The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, OK, Professor Frey invited me to give the two lectures this week on first order equations, like that one, first order dy dt.
And the lectures next week will be on second order equation.
So we're looking for, you could say, formulas for the solution.
We'll get as far as we can with formulas, then numerical methods.
Graphical methods take over in more complicated problems.
This is a model problem.
It's linear.
I chose it to have constant coefficient a, and let me check the units.
Always good to see the units in a problem.
So let me think of this y, as the money in a bank, or bank balance, so y as in dollars, and t, time, in years.
So we're looking at the ups and downs of bank balance y.
The rate of change, so the units then are dollars per year.
So every term in the equation has to have the right units.
So y is in dollars, so the interest rate a is percent per year, say 6% a year.
So a could be 6%-- that's dimensionless-- per year, or half a percent per month if we change.
So if we change units, the constant a would change from 6 to a half.
But let's stay with 6.
And then q of t represents deposits and withdrawals, so that's in dollars per year again.
Has to be.
So that's continuous.
We think of the deposits and the interest as being computed continuously as time goes forward.
So if that's a constant-- and I'll take that case first, q equal 1-- that would mean that we're putting in, depositing $1 per year, continuously through the year.
So that's the model that comes from a differential equation.
A difference equation would give us finite time steps.
So I'm looking for the solution.
And with constant coefficients, linear, we're going to get a formula for the solution.
I could actually deal with variable interest rate for this one first order equation, but the formula becomes messy.
But you can still do it.
After that point, for a second order equations like oscillation, or for a system of several equations coupled together, constant coefficients is where you can get formulas.
So let's go with that case.
So how to solve that equation?
Let me take first of all, a constant, constant source.
So I think of q as the source term.
To get one nice formula, let me take this example, ay plus 1, let's say.
How do you find y of t to solve that?
And you start with some initial condition y of 0.
That's the opening deposit that you make at time 0.
How to solve that equation?
Well, we're looking for a solution.
And solutions to linear equations have two parts.
So the same will happen in linear algebra.
One part is a solution to that equation, so we're just looking for one, any one, and we'll call it a particular solution.
And the associated null equation, dy dt equal ay.
So this is an equation with q equals 0.
That's why it's called null.
And it's also called homogeneous.
So more textbooks use that long word homogeneous, but I use the word null because it's shorter and because it's the same word in linear algebra.
So let me call yn the null solution, the general null solution.
And y, I'm looking here for a particular solution yp, and I'm going to-- here's the key for linear equations.
Let me take that off and focus on those two equations.
How does solving the null equation, which is easy to do, help me?
Why can I, as I plan to do, add in yn to yp?
I just add the two equations.
Can I just add those two equations?
I get the derivative of yp plus yn on the left side.
And I have a times yp plus yn.
And that is a critical moment there when we use linearity.
I had a yp a yn, and I could put them together.
If it was y squared, yp squared and yn squared would not be the same as yp plus yn squared.
It's the linearity that comes, and then I add the 1.
So what do I see from this?
I see that yp plus yn also solves my equation.
So the whole family of solutions is 1 yp plus any yn.
And why do I say any yn?
Because when I find one, I find more.
The solutions to this equation are yn could be e to the at, because the derivative of e to the at does bring down a factor a.
But you see, I've left space for any multiple of e to the at.
This is where that long word homogeneous comes from.
It's homogeneous means I can multiply by any constant, and I still solve the equation.
And of course, the key again is linear.
So now I have-- well, you could say I've done half the job.
I've found yn, the general yn.
And now I just have to find one yp, one solution to the equation.
And with this source term, a constant, there's a nice way to find that solution.
Look for a constant solution.
So certain right hand sides, and those are the like the special functions for the special source terms for differential equation, certain right hand sides-- and I'm just going to go down a list of them today.
The next one on the list-- can I tell you what the next one on the list will be?
y prime equal ay.
I use prime for-- well, I'll write dy dt, but often I'll write y prime.
dy dt equal ay plus an exponential.
That'll be number two.
So I'm just preparing the way for number two.
Well, actually number one, this example is the same as that exponential example with exponent s equal 0, right?
If s is 0, then I have a constant.
So this is a special case of that one.
This is the most important source term in the whole subject.
But here we go with a constant 1.
So we've got yn.
And what's yp?
I just looked to see.
Can I think of one?
And with these special functions, you can often find a solution of the same form as the source term.
And in this case, that means a constant.
So if yp is a constant, this will be 0.
So I just want to pick the constant that makes this thing 0.
And of course, their right hand side is 0 when yp is minus 1 over a.
So I've got it.
We've solved that equation, except we didn't match the initial condition yet.
Let me if you take that final step.
So the general y is any multiple, any null solution, plus any one particular solution, that one.
And we want to match it to y of 0 at t equals 0.
So I want to take that solution.
I want to find that constant, here.
That's the only remaining step is find that constant.
You've done it in the homework.
So at t equals 0, y of 0 is-- at t equals 0, this is C. This is the minus 1 over a.
So I learn what the C has to be.
And that's the final step.
C is bring the 1 over a onto that side, so C will be y of 0 e to the at minus 1 over a e to at.
And here we had a minus 1 over a.
Well, it'll be plus 1 over a e to the at.
So now I've just put in the C, y of 0 plus 1 over a. y of 0 plus 1 over a has gone in for C. And now I have to subtract this 1 over a.
Here, I see a 1 over a, so I can do it neatly.
Got a solution.
We can check it, of course.
At t equals 0, this disappears, and this is y of 0.
And it has the form.
It's a multiple of e to the at and a particular solution.
So that's a good one.
Notice that to get the initial condition right, I couldn't take C to be y of 0 to get the initial condition right.
To get the initial condition right, I had to get that, this minus 1 over a in there.
Good for that one?
Let me move to the next one, exponentials.
So again, we know that the null equation with no source has this solution e to the at.
And I'm going to suppose that the a in e to the at in the null solution is different from the s in the source function, which will come up in the particular solution.
So you're going to see either the st in the particular solution and an e to the at in the null solution.
And in the case when s equals a, that's called resonance, the two exponents are the same, and the formula changes a little.
Let's leave that case for later.
How do I solve this?
I'm looking for a particular solution because I know the null solutions.
How am I going to get a particular solution of this equation?
Fundamental observation, the key point is it's going to be a multiple of e to the st.
If an exponential goes in, then that will be an exponential.
Its derivative will be an exponential.
I'll have e to the st's everywhere.
And I can get the number right.
So I'm looking for y try.
So I'll put try, knowing it's going to work, as some number times e to the st.
So this would be like the exponential response.
Response, do you know that word response?
So response is the solution.
The input is q, and the response is Y.
And here, the input is e to the st, and the response is a multiple of e to the st.
So plug it in.
The timed derivative will be Y.
Taking the derivative will bring down a 1. e to the st equals aY.
A aY e to the st plus 1 e to the st. Just what we hoped.
The beauty of exponentials is that when you take their derivatives, you just have more exponential.
That's the key thing.
That's why exponential is the most important function in this course, absolutely the most important function.
So it happened here.
I can cancel e to the st, because every term has one of them.
So I'm seeing that-- what am I getting for Y?
Getting a very important number for Y.
So I bring aY onto this side with sY.
On this side I just have a 1.
Maybe it's worth putting on its own board.
Y is, so Ys aY comes with a minus, and the 1, 1 over-- so Y was multiplied by s minus a.
That's the right quantity to get a particular solution.
And that 1 over s minus a, you see why I wanted s to be different from a. I If s equaled a in that case, in that possibility of resonance when the two exponents are the same, we would have 1 over 0, and we'd have to look somewhere else.
The name for that-- this has to have a name because it shows up all the time.
The exponential response function, you could call it that.
Most people would call it the transfer function.
So any constant coefficient linear equation's going to have a transfer function, easy to find.
Everything easy, that's what I'm emphasizing, here.
Everything's straightforward.
That transfer function tells you what multiplies the exponential.
So the source was here.
And the response is here, the response factor, you could say, the transfer function.
Multiply by 1 over s minus a.
So if s is close to a, if the input is almost at the same exponent as the natural, as the null solution, then we're going to get a big response.
So that's good.
For a constant coefficient problem second order, other problems we can find that response function.
It's the key function.
It's the function if we have, or if we were to look at Laplace transforms, that would be the key.
When you take Laplace transforms, the transfer function shows up.
Then when you take inverse Laplace transforms, you have to find what function has that Laplace transform.
So did we get the-- we got the final answer then.
Let me put it here.
y is e to the st times this factor.
So I divide by s minus a.
A nice solution.
Let me also anticipate something more.
An important case for e to the st is e to the i omega t. e to the st, we think about as exponential growth, exponential decay.
But that's for positive s and negative s. And all important in applications is oscillation.
So coming, let me say, coming is either late today or early Wednesday will be s equal i omega, so where the source term is e to the i omega t. And alternating, so this is electrical engineers would meet it constantly from alternating voltage source, alternating current source, AC, with frequency omega, 60 cycles per second, for example.
Why don't I just deal with this now?
Because it involves complex numbers.
And we've got to take a little step back and prepare for that.
But when we do it, we'll get not only e to the i omega t, which I brought out, but also, it's real part.
You remember the great formula with complex numbers, Euler's formula, that e to the i omega t is a combination of cosine omega t, the real part, and then the imaginary part is sine omega t. So this is looking like a complex problem.
But it actually solves two real problems, cosine and sine.
Cosine and sine will be on our short list of great functions that we can deal with.
But to deal with them neatly, we need a little thought about complex numbers.
So OK if I leave e to the i omega t for the end of the list, here?
So I'm ready for another one, another source term.
And I'm going to pick the step function.
So the next example is going to be dy dt equals ay plus a step.
Well, suppose I put H of t there.
Suppose I put H of t. And I ask you for the solution to that guy.
So that step function, its graph is here.
It's 0 for negative time, and it's 1 for positive time.
So we've already solved that problem, right?
Where did I solve this equation?
This equation is already on that board.
Because why?
Because H of t is for t positive.
That's the only place we're looking.
This whole problem, we're not looking at negative t. We're only looking at t from 0 forward.
And what is H of t from 0 forward?
It's 1.
It's a constant.
So that problem, as it stands, is identical to that problem.
Same thing, we have a 1.
No need to solve that again.
The real example is when this function jumps up at some later time T. Now I have the function is H of t minus T. Do you see that, why the step function that jumps at time T has that formula?
Because for little t before that time, in here, this is-- what's the deal?
If little t is smaller than big T, then t minus T is negative, right?
If t is in here, then t minus capital T is going to be a negative number.
And H of a negative number is 0.
But for t greater than capital T, this is a positive number.
And H of a positive number is 1.
Do you see how if you want to shift a graph, if you want the graph to shift, if you want to move the starting time, then algebraically, the way you do it is to change t to t minus the starting time.
And that's what I want to do.
So physically, what's happening with this equation?
So it starts with y of 0 as before.
Let's think of a bank balance and then other things, too.
If it's a bank balance, we put in a certain amount, y of 0.
We hope.
And that grew.
And then starting at time, capital T, this switch turns on.
Actually, physically, step function is really often describing a switch that's turned on, now.
This source term act begins to act at that time.
And it acts at 1.
So at time capital T we start putting money into our account.
Or taking it out, of course.
If this with a minus sign, I'd be putting money in.
Sorry, I would start with some money in, y of 0.
I would start with money in.
Yeah, actually, tell me what's the solution to this equation that starts from y of 0?
What's the solution up until the switch is turned on?
What's the solution before this switch happens, this solution while this is still 0?
So let's put that part of the answer down.
This is for t smaller than T. What's the answer?
This is all common sense.
It's coming fast, so I'm asking these questions.
And when I asked that question, it's a sort of indication that you can really see the answer.
You don't need to go back to the textbook for that.
What have we got here?
Yeah?
Is it the null solution [INAUDIBLE]?
It'll be this guy.
Yeah, the particular solution will be 0.
Right, the particular solution is 0 before this is on.
I'm sorry, the null solution is 0, and the particular solution, well, the particular solution is a guy that starts right.
I don't know.
Those names were not important.
And then the question is-- so it's just our initial deposit growing.
Now, all I ask, what about after time T?
What about after time T?
For t after time T, and hopefully, equal time T, what do you think y of t will be?
Again, we want to separate in our minds the stuff that's starting from the initial condition from the stuff that's piling up because of the source.
So one part will be that guy.
I haven't given the complete answer.
But this is continuing to grow.
And because it's linear, we're always using this neat fact that our equation is linear.
We can watch things separately, and then just add them together.
So I plan to add this part, which comes from initial condition to a part that-- maybe we can guess it-- that's coming from the source.
And how do we have any chance to guess it?
Only because that particular source, once it's turned on, jumps to a constant 1, and we've solved the equation for a constant 1.
Let me go back here.
I think our answer to this question-- so this is like just first practice with a step function, to get the hang of a step function.
So I'm seeing this same y of 0 e to the at in every case, because that's what happens to the initial deposit.
I'll say grow, assuming the bank's paying a positive interest rate.
And now, where did this term comes from?
What did that term represent?
The money that [INAUDIBLE]
The money that, yeah?
They had each of [INAUDIBLE]
The money that came in and grew.
It came in, and then it grew by itself, grew separately from that these guys.
So the initial condition is growing along.
And the money we put in starts growing.
Now, the point is what?
That over here, it's going to look just like that.
So I'm going to have a 1 over a.
And I'm going to have something like that.
But can you just guess what's going to go in there?
When I write it down, it'll make sense.
So this term is representing what we have at time little t, later on, from the deposits we made, not the initial one, but the source, the continuing deposits.
And let me write it.
It's going to be a 1 over a e to the a something minus 1.
It's going to look just like that guy.
When I say that guy, let me point to it again-- e to the at minus 1.
But it's not quite e to the at minus 1.
What is it?
t minus [INAUDIBLE]
t minus capital T, because it didn't start until that time.
So I'm going to leave that as, like, reasonable, sensible.
Think about a step function that's turned on a capital time T. Then it grows from that time.
Of course, mentally, I never write down a formula like that without checking at t equal to T, because that's the one important point, at t equal capital T. What is this at t equal capital T?
It's 0.
At t equal capital T, this is e to the 0, which is 1 minus 1 altogether 0.
And is that the right answer?
At t equal capital T is 0, should I have nothing here?
Yes?
No?
Give me a head shake.
Should I have nothing at t equal capital T?
I've got nothing.
e to the 0 minus 1, that's nothing?
Yes, yes that's the right thing.
Because at capital T, the source has just turned on, hasn't had time to build up anything, just that was the instant it turned on.
So that's a step function.
A step function is a little bit of a stretch from an ordinary function, but not as much of a stretch as its derivative.
In a way, this is like the highlight for today, coming up, to deal with not only a step function, but a delta function.
I guess every author and every teacher has to think am I going to let this delta function into my course or into the book?
And my answer is yes.
You have to do it.
You should do it.
Delta functions are-- they're not true functions.
As we'll see, no true function can do what a delta function does.
But it's such an intuitive, fantastic model of things happening over a very, very short time.
We just make that short time into 0.
So we're saying with the delta function, we're going to say that something can happen in 0 time.
Something can happen in 0 time.
It's a model of, you know, when a bat hits a ball.
There's a very short time.
Or a golf club hits a golf ball.
There's a very short time interval when they're in contact.
We're modeling that by 0 time, but still, the ball gets an impulse.
Normally, for 0 time, if you're doing things continuously, what you do over 0 time is no importance.
But we're not doing things continuously, at all.
So here we go.
You've seen this guy, I think.
But if you haven't, here's the time to see it.
So the delta function is the derivative of-- so I've written three important functions up here.
Let me start with a continuous one.
That function, the ramp is 0, and then the ramp suddenly ramps up to t. Take its derivative.
So the derivative, the slope of the ramp function is certainly 0 there.
And here, the slope is 1.
So the slope jumped from 0 to 1.
The slope of the ramp function is the step function.
Derivative of ramp equals step.
Why don't I write those words down?
Derivative of ramp equals step.
So there is already the step function.
In pure calculus, the step function has already got a little question mark.
Because at that point, the derivative in a calculus course doesn't exist, strictly doesn't exist, because we get a different answer 0 on the left side from the answer, 1 on the right side.
We just go with that.
I'm not going to worry about what is its value at that point.
It's 0 up for t negative, and it's 1 for t positive.
And often, I'll take it 1 for t equals 0, also.
Usually, I will.
That's the small problem.
Now, the bigger problem is the derivative of the-- so this is now the derivative of the step function.
So what's the derivative of this step function?
Well, the derivative along there is certainly 0.
The derivative along here is certainly 0.
But the derivative, when that jumped, the derivative, the slope was infinite.
That line is vertical.
Its slope is infinite.
So at that one point, you have an affinity, here, delta of 0.
You could say delta of 0 is infinite.
But you haven't said much, there.
Infinite is too vague.
Actually, I wouldn't know if you gave me infinite or 2 times infinite.
I couldn't tell the difference.
So I'll put it in quotes, because it sort of gives us comfort.
But it doesn't mean much.
What does mean much?
Somehow that's important.
Can I tell you how to work with delta functions, how to think about delta functions?
It's the right way to think about delta function.
So here's some comment on delta function.
Giving the values of the function, 0, and infinity, and 0, is not the best.
What you can do with a delta function is you can integrate it.
You can define the function by integrals.
Integrals of things are nice.
Do you think in your mind when you take derivatives, as we did going left to right, we were taking derivatives.
The function was getting crazy.
When we go right to left, take integrals, those are smoothing.
Integrals make functions smoother.
They cancel noise.
They smooth the function out.
So what we can do is to take the integral of the delta function.
We could take it from any negative number to any positive number.
And what answer would we get?
What would be the right, well, the one thing people know about the delta function is-- and actually, it's the key thing-- the integral of the delta function.
Again, I'm integrating the delta function from some negative number up to some positive number.
And it doesn't matter where n is, because the function is 0 there and there.
But what's the answer here?
Put me out of my misery.
Just tell me the number I'm looking for, here, the integral of the delta function.
Or maybe you haven't met it
[INAUDIBLE]
It's?
It's the one good number you could guess.
It's 1.
Now, why is it 1?
Because if the delta function is the derivative of the step function, this should be the step function evaluated between N and P. This should be the step function, , here, minus the step function, there And what is the step function?
You have to keep it straight.
Am I talking about the delta function?
No, right now, I've integrated it to get H of t. So this is H of P at the positive side, minus H of N. That's what integration's about.
And what do I get?
1, because H of P, the step function here, H is 1.
And here, it's 0, so I get 1.
Good, that's the thing that everybody remembers about the delta function.
And now I can make sense out of 2 delta function, 2 delta of t. That could be my source.
So if 2 delta of t was my source, what's the graph of 2 delta of t?
Again, it's 0 infinite 0.
You really can't tell from the infinity what's up, but what would be the integral of 2 delta of t, the integral of 2 delta of t or some other?
Well, let me put in the 2, here?
What's the integral of 2 delta of t, would be 2H of t. Keep going.
What do I get here?
2
It would be 2 of these guys, 2 of these, 2 of these, 2.
All right?
So we made sense out of the strength of the impulse, how hard the bat hit the ball.
But of course, we need units in there.
We have to have units.
And here, the value for that unit was 2.
Now, I'm going to-- because this is really worth doing with delta functions.
I didn't ask at the start have you seen them before.
But they are worth seeing.
And they just take a little practice.
But then in the end, delta functions are way easier to work with than some complicated function that attempts to model this.
We could model that by some Gaussian curve or something.
All the integrations would become impossible right away.
We could model it by a step function up and a step function down.
Then the integrations would be possible.
But still, we have this finite width.
I could let that width go to 0 and let the height go to infinity.
And what would happen?
I'd get the delta function.
So that's one way to create a delta function, if you like.
If you're OK with step functions, then one way to create delta is to take a big step up, step down, and then let the size of the step grow and the width of the steps shrink.
Keep the area 1, because area is integral.
So I keep this, that little width, times this big height equal to 1.
And in the end, I get delta.
Now again, my point is that delta functions, that you really understand them.
What you can legitimately do with them is integrate them.
But now in later problems, we might have not a 1 or a 2, but a function in here, like cosine t, or e to the t, or q of t. Can I practice with those?
Can I put in a function f of t?
I didn't leave enough space to write f of t, so I'm going to put it in here.
f of t delta of t dt.
And I'm going to go for the answer, there.
My question is what does that equal?
You see what the question is?
I got my delta function, which I only just met.
And I'm multiplying it by some ordinary function.
f of t gives us no problems.
Think of cosine t. Think of e to the t. What do you think is the right answer for that?
What do you think is the right answer?
And this tells you what the delta function is when you see this.
What do I need to know about f of t to get an answer, here?
Do I need to know what f is at t equals minus 1?
You could see from the way my voice asked that question that the answer is no.
Why do I not care what f is at minus 1?
Yeah?
Because you're multiplying by [INAUDIBLE]
Because I'm multiplying by somebody that's 0.
And similarly, at f equal minus 1/2, or at f equal plus 1/3, all those f's make no difference, because they're all multiplying 0.
What does make a difference?
What's the key information about f that does come into the answer?
f at?
At just at that one point, f at?
[INAUDIBLE]
0, f at 0 is the action.
The impulse is happening.
The bat's hitting the ball.
So we're modeling rocket launching, here.
We're launching in 0 seconds instead of a finite time.
So in other words, well, I don't know how to put this answer down other than just to write it.
I guess I'm hoping you're with me in seeing that what it should be.
Can I just write it?
All that matters is what f is at t equals 0, because that's where all the action is.
And that f of 0, if f of 0 was the 2 that I had there a little while ago, then the answer will be 2.
If f of 0 is a 1, if the answer is f of 0 times 1-- and I won't write times 1.
That's ridiculous.
Now we can integrate delta functions, not just a single integral of delta, but integral of a function, a nice function times delta.
And we get f of 0.
So can I just, while we're on the subject of delta functions, ask you a few examples?
What is the integral of e to the t delta of t dt?
It's 1
Yeah, say it again?
It's 1
It's 1.
It's 1, right.
Because e to the t, at the only point we care about, t equal 0 is 1.
And what if I change that to sine t?
Suppose I integrate sine t times delta of t?
What do I get now?
I get?
0
0, right.
And actually, that's totally reasonable.
This is a function, which is yeah, it's an odd function.
Anyway, sine, if I switch t to negative t, it goes negative.
0 is the right answer.
Let me ask you this one.
What about delta of t squared?
Because if we're up for a delta function, we might square it.
Now we've got a high-powered function, because squaring this crazy function delta of t gives us something truly crazy.
And what answer would you expect for that?
1
Would you expect 1?
So this is like?
I'm just getting intuition working, here, for delta functions.
What do you think?
I'm looking at the energy when I square something.
OK, so we had a guess of 1.
Is there another guess?
Yeah?
A third?
Sorry?
1/3
1/3, that's our second guess.
I'm open for other guesses before I-- OK, we have a rule here for f of t. And now what is the f of t that I'm asking about in this case?
It's delta of t, right?
If f of t is delta of t, then that would match this.
And therefore, the answer should match.
Do you see what I'm shooting for, yeah?
It'd be infinity?
It'd be infinity.
It would be infinity.
That's delta of t squared is that's an infinite energy function.
You never meet it, actually.
I apologize, so so write it down there.
I could erase it right away because you basically never see it.
It's infinite energy.
Well, I think you'd see it.
I mean, we're really going back to the days of Norbert Wiener.
When I came to the math department, Norbert Wiener was still here, still alive, still walking the hallway by touching the wall and counting offices.
And hard to talk to, because he always had a lot to say.
And you got kind of allowed to listen.
So anyway, Wiener was among the first to really use delta functions, successfully use delta functions.
Anyway, this is the big one.
This is the big one.
Now, so what's all that about?
I guess I was trying to prepare by talking about this function prepare for the equation when that's the source.
So dy equal ay plus a delta function.
Let me bring that delta function in at time T. So how do you interpret that equation?
So like part of this morning's lecture is to get a first handle on an impulse.
So let me write that word impulse, here.
Where am I going to write it?
So delta is an impulse.
That's our ordinary English word for something that happens fast.
And y of t is the impulse response.
And this is the most important.
Well, I said e to the st was the most important.
How can I have two most important examples?
Well, they're a tie, let's say.
e to the st is the most important ordinary function.
It's the key to the whole course.
Delta of t, the impulse, is the important one because if I can solve it for a delta function, I can solve it for anything.
Let's see if we can solve it for a delta function, a delta function, an impulse that starts at time T. Again, I'm just going to start writing down the solution and ask for your help what to write next.
So what do you expect as a first term in the solution?
So I'm starting again from y of 0.
Let's see if we can solve it by common sense.
So how do I start the solution to this?
Everybody sees what this equation is saying.
I have an initial deposit of y of 0 that starts growing.
And then at time capital T I make a deposit.
At that moment, at that instant, I make a deposit of 1.
That's an instant deposit of 1.
Which is, of course, what I do in reality.
I take $1 to the bank.
They've got it now.
At time T, I give them that $1, and it starts earning interest.
So what about y of t?
What do you think?
What's the first term coming from y of 0?
So the term coming from y of 0 will be y of 0 to start with, e to at.
That takes care of the y of 0.
Now, I need something.
It's like this, plus I need something that accounts for what this deposit brings.
So up until time T, what do I put?
So this is for t smaller than T and t bigger than T. So what goes there?
For t smaller than T, what's the benefit from the delta function?
0, didn't happen yet.
For t bigger than T, what's the benefit from the delta function?
[INAUDIBLE]
For t bigger than T, well, that's right.
OK, but now I've made that deposit at time capital T. Whatever's going there is whatever I'm getting from that deposit.
At time capital T, I gave them $1, and they start paying interest on it.
What's going to go there?
So if I gave them $1 at that initial time, so that $1 would have been part of y of 0.
What did I get at a later time?
e to the at.
Now I'm waiting.
I'm giving them the dollar at time capital T, and it starts growing.
So what do I have at a later time, for t later than capital T?
What has that $1 grown into?
e to the a times the-- right, it's critical.
It's the elapsed time.
It's the time since the deposit.
Is that right?
So what do I put here?
t minus capital T?
t minus capital T, good.
Apologies to bug you about this, but the only way to learn this stuff from a lecture is to be part of it.
So I constantly ask you, instead of just writing down a formula.
I think that looks good.
So suddenly, what does this amount to at t equal capital T?
Maybe I should allow t equal capital T. At t equal capital T, what do I have here?
1
1.
That's my $1.
At t equal capital T, we've got $1.
And later it's grown.
So we have now solved.
We have found the impulse response.
We have found the impulse response when the impulse happened at capital T. That was good going.
Now, I've given you my list of examples with the pause on the sine and cosine.
I pause on the sine and cosine because one way to think about sine and cosine is to get into complex numbers.
And that's really for next time.
But apart from that, we've done all the examples, so are we ready?
Oh yeah, I'm going to try for the big thing, the big formula.
So this is the key result of section 1.4, the solution to this equation.
So I'm going back to the original equation.
And just see if we can write down a formula for the answer.
So let me write the equation again.
dy dt is ay plus some source.
I think we can write down a formula that looks right.
And we could then actually plug it in and see, yeah, it is right.
So what's going to go into this formula?
We got enough examples, so now let's go for the whole thing.
So y of t, first of all, comes whatever depends on the initial condition.
So how much do we have at a later time when our initial deposit was y of 0?
So that's the one we've seen in every example.
Every one of these things has this term growing out of y of 0.
So let me put that in again.
So the part that grows out of y of 0 is y of 0 e to the at.
That's OK.
So that's what the initial.
So our money is coming from two sources, this initial deposit, which was easy, and this continuous, over time deposit, q of t. And I have to ask you about that.
That's going to be like the particular solution, the particular solution that comes from the source term.
This is the solution it comes from the initial condition.
So what do you think this thing looks like?
I just think once we see it, we can say, yeah, that makes sense.
So now I'm saying what?
If we've deposited q of t in varying amounts, maybe a constant for a while, maybe a ramp for awhile, maybe whatever, a step, how am I going to think about this?
So at every time t equal to s, so I'm using little t for the time I've reached.
Right?
Here's t starting at 0.
Now, let me use s for a time part way along.
So part way along, I input.
I deposit q of s. I deposit it at time s. And then what does it do?
That money is in the bank with everybody else.
It grows along with everything else.
So what's the growth factor?
What's the growth factor?
This is the amount I deposited at time s. And how much has it grown at time t?
This is the key question, and you can answer it.
It went in a time s. I'm looking at time t. What's the factor?
Is it e to the a t minus s
e to the a t minus s. So that's the contribution to our balance at time t from our input at time s. But now, I've been inputting all the way along.
s is running all the way from here to here.
So finish my formula.
Put me out of my misery.
Or it's not misery, actually.
Its success at this moment.
What do I do now?
I?
Integrate
I integrate, exactly.
I integrate.
I integrate.
So all these deposits went in.
They grew that amount in the remaining time.
And I integrate from 0 up to the current time t. So you see that formula?
Have a look at it.
This is a general formula, and every one of those examples could be found from that formula.
If q of s was 1, that was our very first example.
We could do that integration.
If q of s was e to the-- anyway, we could do every one.
I just want you to see that that formula makes sense.
Again, this is what grew out of the initial condition.
This is what grew out of the deposit at time s. And the whole point of calculus, the whole point of learning [?
1801 ?
], the integral equation part, the integrals part, is integrals just add up.
This term just adds up all the later deposits, times the growth factor in the remaining time.
And as I say, if I took q of s equal 1-- the examples I gave are really the examples where you can do the integral.
If q of s is e to the i omega s, I can do that integral.
Actually, it's not hard to do because e to the at doesn't depend on s. I can bring an e to the at out in this case.
That formula is just worth thinking about.
It's worth understanding.
I didn't, like, derive it.
And the book does, of course.
There's something called an integrating factor.
You can get at this formula systematically.
I'd rather get at it and understand it.
I'm more interested in understanding what the meaning of that formula is than the algebra.
Algebra is just a goal to understand, and that's what I shot for directly.
And as I say, that the book also, early section of the book, uses practice in calculus.
Substitute that in to the equation.
Figure out what is dy dt.
And check that it works.
It's worth actually looking at that end of what you need to know from calculus It's is.
You should be able to plug that in for y and see that solves the equation.
Right, now I have enough time to do cosine omega t. But I don't have enough time to do it the complex way.
So let me do as a final example, the equation.
Let me just think.
I don't know if I have enough space here.
I'm now going to do dy dt-- can I call that y prime to save a little space-- equal ay plus cosine of t. I'll take omega to be 1.
Now, how could we solve that one?
I'm going to solve it without complex numbers, just to see how easy or hard that is.
And you'll see, actually, it's easy.
But complex numbers will tell us more.
So it's easy, but not totally easy.
So what did I do in the earlier example if the right hand side was a 1, a constant?
I look for the solution to be a constant.
If the right hand side was an exponential, I look for the solution to be an exponential.
Now, my right hand side, my source term, is a cosine.
So what form of the solution am I going to look for?
I naturally think, OK, look for a cosine.
We could try y equals some number M cosine t. Now, you have to see what goes wrong and how to fix it.
So if I plug that in, looking for M the same way I look for capital Y earlier, I plug this in, and I get aM cosine t cosine t. But what do I get for y prime?
Sine t. And I can't match.
I can make it work.
I can't make a sine there magic a cosine here.
So what's the solution?
How do I fix it?
I better allow my solution to include some sine plus N sine t. So that's the problem with doing it, keeping things real.
I'll push this through, no problem.
But cosine by itself won't work.
I need to have sines there, because derivatives bring out sines.
So I have a combination of cosine and sine.
I have a combination of cosine and sine.
So the complex method will work in one shot because e to the i omega t is a combination of cosine and sine.
Or another way to say it is when I see cosine here, that's got two exponentials.
That's got e to the it and e to the-- anyway.
Let's go for the real one.
So I'm going to plug that into there.
So I'll get sines and cosines, right?
When I plug this into there, I'll have some sines and some cosines, and I'll just match the two separately.
So I'm going to get two equations.
First of all, let me say what's the cosine equation?
And then what's the sine equation?
So when I match cosine terms, what do I have?
What cosine terms do I get out of y prime, here?
The derivative.
Well, the derivative of cosine is a sine.
That that's not a cosine term.
The derivative of sine is cosine.
I think I get, if I just match cosines, I think I get an N cosine.
N cosine t equal ay.
How many cosines do I have from that term?
ay has an M cosine t. I think I have an aM, and here I've got 1.
That was a natural step, but new to us.
I'm matching the cosines.
I have on the left side, with this form of the solution, the derivative will have an N cosine t. So I had N cosines, aM cosines, and 1 cosine.
Now, what if I match signs?
What happens there?
We're pushing more than an hour, so hang on for another five minutes, and we're there.
Now, what happens if I match sines, sine t?
How do I get sine t in y prime?
So take the derivative of that, and what do you have?
Minus [INAUDIBLE]
Minus M sine t. That tells me how many sine t's are in there.
And on the right hand side, a times y, how many sine t's do I have from that?
You have N t's
N, good thinking.
And what about from this term?
None, no sine there.
So I have two equations by matching the cosines and sines.
Once you see it, you could do it again.
And we can solve those equations, two ordinary, very simple equations for M and N. Let's see if I make space.
Why don't I do it here, so you can see it.
So how do I solve those two equations?
Well, this equation gives me-- easy-- gives me M as minus aN.
So I'll just put that in for N. So I have N equals aM.
But M is minus aN.
I think I've got minus a squared N plus that 1.
All I did was solve the equation, just by common sense.
You could say by linear algebra, but linear algebra's got a little more to it than this.
So now I know M, and now I know N. So now I know the answer.
y is M, so M is minus aN.
Oh, well, I have to figure out what N is, here.
What is N?
This is giving me N, but I better figure it out.
What is N from that first equation?
And then I'll plug in.
And then I'm quit
[INAUDIBLE]
1 over, yeah
1 plus a squared
1 plus a squared, good.
Because that term goes over there, and we have 1 plus a squared.
So now y is M cosine t. So M is minus aN.
So minus aN is 1 over 1 plus a squared cosine t. Is that right?
That was the cosines.
And we had N sine t. But N is just 1-- I think I just add the sine t. Have I got it?
I think so.
Here is the N sine t, and here is the M cos t. It was just algebra.
Typical of these problems, there's a little thinking and then some algebra.
The thinking led us to this.
The thinking led us to the fact we needed cosines in there, as well as cosines.
But then once we did it, then the thinking said, OK, separately match the cosine terms and the sine term.
And then do the algebra.
Now, I just want to do this with complex.
So y prime equals ay plus e to the it.
To get an idea, you see the two.
And then I have to talk about it.
You see, I'm only going to go part way with this and then save it for Wednesday.
But if I see this, what solution do I assume?
This is like an e to the st.
I assume y is some Y e to the it.
See, I don't have cosines and sines anymore.
I have e to the it.
And if I take the derivative of e to the it, I'm still in the e to the it world.
So I do this.
I plug it in.
Uh-huh, let me leave that for Wednesday.
We have to have some excitement for Wednesday.
So we'll get a complex answer, and then we'll take the real part to solve that problem.
So we've got two steps, one way or the other way.
Here, we had two steps because we had to let sines sneak in.
Here, we have two steps because I could solve it, and you could solve that right away.
But then you have to take the real part.
I'll leave that.
Is there questions?
Do you want me to recap quickly what we've done
Yes
I try to leave on the board enough to make a recap possible.
Everything was about that equation.
We have only solved-- I shouldn't say only-- we have solved the constant coefficient, model constant coefficient, first order equation.
Wednesday comes nonlinear equation.
This one today was strictly linear.
So what did we do?
We solved this equation, first of all, for q equal 1; secondly, for q equal e to the st; thirdly, for q equal a step; fourthly for q equal-- where is it?
Where is that delta of t?
Maybe it's here.
Ah, it got erased.
So the fourth guy was y prime equal ay plus delta of t, or delta of t minus capital T. So those were our four examples.
And then what did we finally do?
So if we're recapping, compressing, we're compressing everything into two minutes.
We solved those four examples, and then we solved the general problem.
And when we solved the general problem, that gave us this integral, which my whole goal was that you should understand that this should seem right to you.
This is adding up the value at time t from all the inputs at different times s. So to add them up, we integrate from 0 to t. And finally, we returned to the question of cos t, all important question.
But awkward question, because we needed to let sine t in there too.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Monday's lecture was all linear equations.
And I thought I would start today with nonlinear equations, still first order.
And we can't deal with every nonlinear equation.
That's too much to ask.
These are going to be made easier by a property called "separable."
So these will be separable nonlinear equations.
And let me start with a couple of examples and then you'll see the whole idea.
So one example would be the simplest nonlinear equation I can think of, with a y squared.
So how to get there?
Here's the trick.
This is the separable idea.
You're going to see it in one shot.
We can separate, put the Ys on one side and the Ds on the other.
So I write this as dy over y squared equal dt.
I put the dt up and brought the y squared down.
So now they're separated, in a kind of hookie way with infinitesimals.
But I'll makes sense out of that by integrating.
I'll integrate both sides.
I'll integrate time from 0 to t. And I have an initial condition, y of 0, always.
And since this one is about y, when t starts at 0, this guy starts at y of 0, up to, this ends at t, so this ends at y of t. OK. Now the point is, also the problem was nonlinear, we've got two separate ordinary integrals to do.
And we can do them.
We can certainly do the right hand side.
I get t. And on the left hand side, what do I get?
Well, that maybe I better leave a little space to figure out this one.
But the point is we can integrate 1 over y squared.
And I guess we get minus 1 over y.
So I get minus 1 over y between y of 0 and y of t. In other words, I'm getting let's see, so what the right, the derivative of the integral of 1 over y squared is minus 1 over y because I always check the derivative gives me that back.
So now I'm ready to plug-in those limits.
So I'll do the bottom limit first because it comes with a minus sign, canceling that minus, 1 over y of 0 minus 1 over y of t. Got it.
And that equals the other integral, which is just t. So that's the answer as it comes directly from integration.
And we can do more.
You can see that finding the solution when these things are separable has boiled down to two integrals.
And we could have a function of t here, too.
And that would be allowed, a function of t multiplying this guy, because then I would leave the function of t on that side.
And I would have to integrate that.
And I would bring the y.
You see, I've just separated the y.
In general, these equations look like dy, dt is some function of t divided by some function of y.
Maybe the book calls the top one g, I think, and the bottom one f. And everybody in this room sees that I can put the f of y up there.
I can put the dt up there.
And I've separated it.
OK.
So that's sort of the general situation.
This is a kind of nice example, nice example, dy, dt equals y squared.
Can we just play with this a little bit?
Let me take y of 0 to be 1, just to make the numbers easy.
So if y of 0 is 1, then I have, I'll just keep going a little bit.
You do have to keep going a little bit because when you finish the integral right there, you haven't got y equal.
You've got some equation that involves y, but you have to solve for y.
So I have to solve that equation for y.
Let me just do it.
So how would I solve it?
And let me take y of 0 to be 1.
So now, if I just write it below, I'm at 1 minus 1 over y equals t. Good?
So I'm going to put the 1 over y of t on that side and the t on that side.
So if I just continue here, I've got 1 over y of t on this side and, do I have 1 minus t on that side?
Yeah.
Looking good.
So solution starting from y of 0 equal 1 is y of t equal 1 over 1 minus t. You could do that.
You could do that.
And I can always, like, mentally I check the algebra at t equals 0.
That gives me the answer, 1.
But let's step back and look at that answer.
I mean, that's part of differential equations is to do some algebra, if possible, and get to a formula.
But if we don't think about the formula, we haven't learned anything.
Right there, yes.
Good.
So what happens?
I want to compare with the linear case that was like e to the t. This was y prime equal y, right?
And that led to e to the t. Y prime equals y squared leads to that one.
So first observation.
I haven't got exponentials anymore in that solution.
Exponentials are just like perfection for linear equations.
For nonlinear equations, we get other functions.
Professor Fry had a hyperbolic tangent function in his first lecture.
Other things happen.
OK. Now, how do those compare if I graph those?
It's just like, why not try?
So they both started at 1.
And e to the t went up exponentially.
E to the t. And, I don't know, we use exponential.
In our minds, we think, that's pretty fast growth.
!
mean, that's the common expression, grew exponentially.
But here, this guy is going to grow faster because y is going to be bigger than 1.
So y squared is going to be bigger than y.
That one's going to grow faster.
Faster than exponential.
This has the exponential growth.
Pretty fast.
Polynomial, of course, some parabola or something would be hanging way down here, left behind in the dust.
But this 1 over 1 minus t, that's going to grow really fast.
And what's more, it's going to go to infinity.
So that y prime equal y squared, the solution to that doesn't just-- e to the t goes to infinity at time infinity.
At any finite time, we get an answer.
Eventually, at t equal infinity, it's gone above every bound.
But this one, 1 over 1 minus t is what I want to graph now.
I believe that that takes off and at a certain point, capital T, it's going to infinity.
It's blown up.
So it's blow up in finite time.
Blow up in finite time.
And what is that time?
What's the time at which the y prime equal y squared has taken off, gone off the charts?
T equal--
1
--1.
Because when t reaches 1, I have 1 over 0, and I'm dividing by 0, and so that's the blow up.
Finite time blow up.
OK.
So this can happen for some nonlinear equations.
It wouldn't happen for a linear equation.
For a linear equation, exponentials are in control.
OK.
So that's one nice example.
Oh, another nice thing about that example.
Well, I say nice if you're OK with infinite series.
I just want to compare.
The book mentions the infinite series for these guys because that's an old way to solve differential equations is term-by-term in an infinite series.
It's sort of fun to see the two series.
Well, because they're the two most important series in math.
Actually, they're the two series that everybody should know.
The power series, Taylor series-- whatever word you want to give it for those two guys.
So let me do them.
E to the t, I'll put that one first, and 1 over 1 minus t. These are the great series of math.
Shall I just write them down and sort of talk through them?
Because this is not a lecture on infinite series by any means.
But having these two in front of us, coming from these two beautiful equations, y prime equal y squared and y prime equal y, I can't resist seeing what they look like this way.
So e to the t, do you remember e to the t?
It starts at 1.
What's the slope of e to the t?
At t equals 0.
So I'm doing everything-- this series is going to be, both of the series are going to be, around t equals 0.
That's my, like, starting point.
So this e to the t thing has a tangent.
It has a slope there.
And what's the slope of e to the t at t equals 0?
1
1.
It's derivative.
The derivative of e to the t is e to the t. The slope is 1.
So that tangent line has coefficient 1.
That's how it starts.
That's the linear approximation.
That's the heart of calculus, is this.
But we're going to go better.
We're going to get the next term.
So what's the next term?
That gave us the tangent line.
Now I'm going to move to the tangent parabola.
So the parabola has got another is still going to be below the real thing.
Can I squeeze in the words "line" and "parab," for "parabola?"
Parabola has bending.
I'm really explaining the Taylor series in what I hope is a sensible way.
Here is the starting point.
This has the slope.
The next term has the bending.
The bending comes from what derivative?
What derivative tells us about bending?
Second derivative.
Second derivative tells us how much it curves.
Well, the second derivative of e to the t is still e to the t. So the bending is 1.
The bending is also 1.
Now that comes in with a factor of a 1/2.
There is the tangent parabola.
And you will see what these numbers become.
Let me just go to, the third derivative would be responsible for the t cubed term.
And its coefficient would be 1 over 3 factorial.
So 2 is the same as 2 factorial.
3 factorial is 3 times 2 times 16.
So the numbers here go 1, 6, 24, 120, whatever the next one is.
720 or something.
They grow fast.
So that series always gives a finite answer.
It does grow with t. But it doesn't spike with t. Now compare that famous series.
And of course, this is 1 over 1 factorial, everything consistent.
Compare that with the series for 1 over 1 minus t. That's the other famous series that they learned in algebra.
I'll just write it.
That's 1 plus t plus t squared plus t cubed plus 1 so on, with coefficient 1.
This had 1 over n factorials.
Those make the series converge.
These don't have the n factorials.
This is 1, 1, 1, 1.
And, well, I could check that formula.
But do you remember the name for that series?
1 plus t plus t squared plus t cubed plus so on?
Algebra is taught differently in many high schools now.
And maybe that never got a name.
I guess I would call it the Geometric series.
Geometric series.
And you see, it's beautiful.
It's the other important series.
But it's quite different from this one because, what's the difference about this series?
Yeah?
It goes to infinity
It's a--
It goes to infinity
It goes to infinity.
But where?
At what time?
At what value of t is this sum going to fall apart?
Blow up?
At t equal 1.
When have 1 plus 1 plus 1 plus 1, I'm getting infinity.
So this blows up.
And of course, we see that it should because this blows up.
Left side blows up at t equal 1, the right side blows up a t equal 1.
Where the exponential series, which is the heart of ordinary differential equations, never blows up because of these big numbers in the denominator.
OK, I'm good for this first simple example, y prime equals y squared.
It has so much in it, it's worth thinking about.
I'm ready, you OK for a second example?
A second important separable equation.
I'm going to pick one.
So I'm going to pick an equation that starts out with our familiar linear growth.
This could be, you know, last time it was growth of money in a bank.
It could be growth of population.
The number of, to a sum first approximation, the rate of growth of the population comes from, like, births minus deaths.
And with modern medicine, births are a larger number than deaths.
So a is positive, and that grows.
But if we're talking about the, I mean, the United Nations tries to predict, everybody tries to predict, population of the world in future years.
And so this could be called the Population Equation.
But just to leave it as pure exponential is obviously wrong.
The world can't grow forever.
The population can't grow forever.
And the, I guess I hope it doesn't grow like 1 over 1 minus t. So this is at least a little slower.
But somehow competition for space, competition for food, for oil, for water-- which is going to be the big one-- is in here.
Competition here, of people versus people, a reasonable term, a first approximation, is a y squared, with a minus, is a y squared and with some coefficient.
That's a very famous equation.
A first model of population is it grows.
But this is a competition term, y against y.
And so, the same would be true if we were talking about epidemics.
That's a big subject with ordinary differential equations, epidemiology.
Or say, flu.
How does flu spread?
And how does it get cured?
So partly, people are getting over the flu.
But then y against y is telling us how many infected, how many new infections.
So we would like to solve that equation.
And it's separable.
I can do what I did before, dy over ay minus by squared equal dt.
And I can integrate, starting from year of 0.
Well, why don't we start from year 2014, with the population y at now-- the present population?
That would be a model that the UN would consider using.
That other people with very important interest in measuring population and measuring every resource would need equations like this.
And then they would put on more terms, like a term for immigration.
All sorts, many improvements have to go into this equation.
Let me just look at this as it is.
Well, I've got two choices here.
Well, it's this integral that I'm looking at.
That is a doable integral.
It's the type of integral that we saw in the Rocket problem.
The Rocket problem was more constant minus.
This was a drag term, when we were looking at rockets.
And this was a constant, say, gravity.
So it was still a second degree.
Still second degree, but a little different.
This has the linear in second degree terms.
If you look up that integral, you'll find it.
Or there's a systematic way to do it.
That's in 1801, I guess, called partial fractions.
It's not a lot of fun.
I don't plan to do it.
It's in the book.
Has to be because that's the way you can integr-- you can integrate polynomials over polynomials by partial fractions.
That's what they're for, but there's a neat way to do this one.
There's a neat trick that Bernoulli discovered to solve that equation, to turn it into a linear equation.
And of course, if we can turn it into a linear equation, we're on our way.
So the neat trick is let z be 1 over y.
You can put this in the category of lucky accidents, if you like.
So now I want an equation for z.
So I know that dz, dt if I take the derivative of that, that's y to the minus 1.
So it's minus 1 y to the minus 2 dy, dt.
That's the chain rule.
Take the derivative of 1 over 1, you get minus 1 over y squared.
Multiply by the derivative of what's inside.
That's the chain rule.
OK.
So I plan to substitute those in here.
So dy, dt, let's see.
Can you see me?
You can probably do it better than me.
So dy, dt is minus.
I'll bring that up.
Dy, dt I'm going to put-- I hope this'll work all right-- for dy, dt, I'm going to put in dz.
Using this, I'm going to put minus y squared dz, dt.
Did that look right?
I don't think I'm necessarily doing this the most brilliant way.
But dy, dt-- I put this up here and I got that-- equals ay.
So that's a over z. Oh, y Is 1 over z.
So get this, I want all Zs now.
So that's this part.
And ay is over z minus by squared is minus b over z squared.
Would you say OK to that?
I've got Zs now, instead of Ys.
I just took every term and replaced y by 1 over z. Y is 1 over z and dy, dt I can get that way.
OK. Yeah.
Now what?
Now look what happens, if I multiply through by z squared or by minus z squared.
Let me multiply through by minus z squared.
I get dz, dt.
Multiplying by minus z squared gives me a minus az.
And what do I get when I multiply this one by minus z squared?
Plus b
I get plus b.
Look what happened.
By this, like, some magic trick.
You could say, all right.
That was just a one time shot.
But it was a good one.
We ended up with a linear equation for z.
A linear equation for z.
And we solved that equation last time.
So let me squeeze in the solution for z, and then elsewhere.
So what was the solution for z of t?
It was some multiple of, no, yeah.
This is perfect review of last time.
We have a constant times z.
And so that's going to go into the exponential.
This will be the, it's a minus a, notice.
That will be the, what part of the solution is that one called?
That's the null solution.
The null solution, when b is o.
And now I add in a particular solution.
A particular solution.
And one good particular solution is choose the z to be a constant.
Then that'll be 0.
So I want that to be 0.
So what constant z makes that 0?
I think it's b over a, don't you?
I think b over a.
Does that work good?
That's every null solution plus one particular solution.
Let me say now, and I'll say again, looking for solutions which are steady states, b over a-- of this particular solution, that particular solution made this 0.
So it made this 0.
So it's a solution that's not going anywhere.
It's a constant solution.
It's a solution that can live for all time.
OK. B over a.
Let me put that word there, steady state.
OK. And now I would want to match the initial conditions using c. Yeah.
I'd better do that.
OK. And I have to get back to y. I have y is 1 over z.
So I'm going to have to flip this upside down.
I'm going to have to flip this upside down is what will actually happen.
Let me make it easy to flip.
Let me, I'll change c, which is just some constant to some constant d over a.
So then it's a is everywhere down below.
And I just write it here in the middle.
That makes it easier to flip.
So finally I get their solution.
Solution to the population equation.
But that's the famous word for it, the Logistic equation.
This is section 1.7 of the text on the differential equations in linear algebra.
It's a very, very much studied example.
It's a great example.
It fits the growth of human population with some, it's our first level approximation to growth of or other populations or other things.
It's a linear term giving us exponential growth, and a quadratic term of competition slowing it down.
And let's see that slow down.
So now that was a bit of algebra.
Much nicer than partial fractions.
The bit of algebra just came from this idea of going to z.
And now I want to go back to y.
So y is 1 over z.
So it's a over d e to the minus at plus b.
That's our solution.
A and b came out of the equation.
And d is going to be the number that makes the initial value correct.
So at t equals 0, I would have y of 0, whatever the initial population is, is a over d. T Is 0, so that's just 1 plus b.
So that tells me what d is.
D equals something.
It comes from y of 0.
So the answer, let me circle that answer.
That answer has three numbers in it, a, b, and d. a and b come from the equation.
D also involves the initial starting thing, which is exactly what it showed.
So you could say we've solved it.
But if you ever solve an equation like this, you want to graph it.
You want to graph it.
So let me draw its graph.
This is important picture.
So here is time.
Here is population.
Here's, maybe it started there.
This is times 0.
And now I want to graph this.
I want to graph that function.
Really, this is where we're going somewhere.
What happens for a long time?
At t equal infinity, what happens to the population?
Does it grow, like e to the t?
Just remember the examples here.
We had a growth like e to the t. We had a growth faster than e to the t that actually blew up.
What about this guy?
What will happen as t goes to infinity with that population?
It goes to?
A over b
A over b.
A over b.
That's the key number in the whole thing.
It keeps growing, but it never passes a over b.
This is y at infinity.
That's the final population.
So how does it do this?
If I draw this graph-- and what about negative time?
Let's go backwards in time.
What is it at t equals minus infinity?
Then you really see the whole curve.
At t equal minus infinity, what is this doing?
0 infinity
It's 0.
Good.
Good.
Good.
T equal minus infinity, this is enormous.
This is blowing up.
It's in the denominator.
We're dividing by it.
So the whole thing is going to 0.
So here's what the logistic curve looks like.
It creeps up.
And it's beautifully, there's a point of symmetry here.
The growth is increasing here.
And then, as a point of inflection you could say, growth is bending upwards for a while.
At this point, it starts bending downwards.
From that point on, ooh, let's see if I can draw it.
It'll get closer, and exponentially close.
That wasn't a bad picture.
The population here is half way.
Here, the population, the final population, is a over b.
And just by beautiful symmetry, the population here is a 1/2 of a over b.
At this point.
If this was the actual population of the world we live in-- I think we're pretty close to this point.
I believe, well, of course, nobody knows the numbers, unfortunately, because the model isn't perfect.
If the model was perfect, then we could just takes the census and we would know a and b.
But the model isn't that great.
But it's sort of, we're at a very interesting time, close to a very interesting time.
I believe that with reasonable numbers, this a over b might be maybe 12 billion.
And we might be, I think we're a little above six billion.
I think so.
So we're a little bit past it.
This is now.
This is halfway.
That's the halfway point.
It's perfectly symmetric.
It's called an S curve.
And many, many equations in math biology involve S curves.
So math biology often gives rise, with simple models, to a kind of problem we've had here with a quadratic term slowing things down.
Enzymes, all kinds of.
Ordinary differential equations are core ideas in a lot of topics, lot of areas of science.
OK. Do I want to say more about the logistic equation?
I guess I do want to distinguish one thing.
Yeah.
One thing about logistic equations and will of course come back to this.
OK. Let me look at that logistic equation.
Here's my equation.
So I've managed to solve it.
Fine.
Great.
Even graph it.
But let me come back to the question, suppose I just look at.
I can see two constant solutions, two steady states, two solutions where the derivative is 0.
So nothing will happen.
So in other words, I want to set this thing set to 0 equal to 0 to find steady solutions.
Steady means the derivative is 0.
So this side has to be 0.
So what are the two possible steady states where, if y of 0 is there, it'll stay there?
0
0.
Y equals 0 is one.
And the other?
A over b
Is a over b.
So steady equal to 0.
And I get two steady states.
Let me call them capital Y equals 0 because that's certainly 0 of, if we have 0 population, we'll never move.
Or setting this to 0, ay is by squared cancel y's divide by b a over b.
So the two steady states are here.
That's a steady state and that's a steady state.
Those are the only two in this problem.
You see how easy that was to find the steady states?
That's an important thing to do.
And then the other important thing to do is to decide, are those steady states stable?
When the population's near a steady state, does it approach that, does it go toward that steady state or away?
So what's the answer?
For this steady state, that steady state, y is a over b.
Is that stable or unstable?
So I'll write the word stable.
And I'm prepared to put in "un," unstable, if you want me to.
This is a key, key idea.
And with nonlinear equations, you can answer this stability stuff without formulas.
Without formulas.
That's the nice thing.
And then that comes in a later class.
But here's a perfect example.
So do we approach this answer or do we leave it?
We approach it, the solutions.
This is stable, yes.
And here's the other stationary point, capital Y.
The other steady state is that nothing happens.
So now if I'm close to that, if y is a little number, like 2, will that 2 drop to 0, will it approach this steady state, or will it leave it?
Leave it
Leave it.
So this steady state is
Unstable
Unstable.
Unstable.
Right.
Right.
With linear equations, we really only had one steady state, like 0.
Once it started, it took off forever.
Here, it doesn't go infinitely high.
It bends down again to that limit, that carrying capacity is what it's called, a over b. I guess I hope you think a nonlinear equation like got a little more to it.
Little more interesting, but a little more complicated, than linear equations.
Yep.
Yep.
Yep.
And similarly, the rocket equation, we could at the right time soon in the course, ask the same thing, a rocket equation was something like that.
What are the steady states?
Are they stable?
Are they unstable?
Can you find a formula?
Here.
This.
We got a formula.
And there are other nonlinear equations, which we'll see.
OK.
I could create more separable equations, but I guess I hope that you see with separable equations, you just separate them and integrate a y integral and a t integral.
Is that OK any question on this nonlinear separable stuff?
Differential equations courses and the subject tends to be types of equations as can solve.
And then there are a hell of a lot of equations that are not on anybody's list, where you could maybe solve them by an infinite series, but not by functions that we know.
OK.
I'm ready to do the other topic for today.
It's the topic that I left incomplete on Monday.
So I'm staying with first order equations, but actually this topic is essential for second order equations.
So I'm going to topic two for today.
So topic two will involve complex numbers.
So we have to deal with complex numbers.
And the purpose of introducing these complex numbers is to deal with what we met last time when the right hand side, the forcing term, was a cosine.
Typical alternating current, oscillating, rotating, rotation.
All these things produce trig functions.
Maybe rotation is more of a mechanical engineering phenomenon.
Alternating current more of an EE phenomenon.
But they're always there.
And what was the point?
The point was we had some linear equation, and we had some forcing by something like cos omega t. Or it could be A cos omega t and B sine omega t. Either just cosine alone, or maybe these come together.
And then the solution was y equals some combination of those same guys.
In other words, what I'm saying is cosines are nice right hand forcing functions.
Fortunately, because we see them all the time.
But they do lead to cosines and sines.
I emphasized that last time.
If we just have cosines in the forcing function, we can't expect that there's any damping, we can't expect only cosines.
We have to expect some sines.
In other words, we have to deal with combinations of them.
And the question is, how do you understand cos omega t plus 3.
Or let me take a first example.
Example-- cos t plus sine t. That's a perfect example.
So what is omega here in this example that I'm starting with?
1
1.
So I just read off the coefficient of t is 1, 1 hertz here.
But we have got this combination.
And the question is, how do we understand that cosine plus sine?
Two very simple functions, but they're added, unfortunately.
And there's a much better way to write this so you really see it.
You really see this.
That's called a sinusoid.
And the rule that want to focus on now is that everything of that kind, of this kind, of this kind, of a cosine plus a sine, can be compressed into one term.
One term.
Of course, it's got to have two constants to choose because that had an a and a b.
This had an m and an n. This had a 1 and a 1.
But the term I'm looking for is some number R times a pure cosine of omega t, but with a phase shift.
So you see there are two numbers here to choose.
It's really like going from rectangular to polar.
Say in complex numbers, let's just remember the first fact about a complex number.
If the real part is 3, and the imaginary part is, let's say 2, then here's a complex number, 3 plus 2 i.
So this was the real axis.
This was the imaginary axis.
I went along 3, I went up 2, I got to that number.
There it is.
I plotted the number 3 plus 2 i in the complex plane.
And for me, that number 3 plus and so on, really saying something important.
And maybe it's not entirely new.
I'm saying something important about complex numbers, this is their rectangular form.
Something plus something.
That form is nice to add to another complex number.
If I added 3 plus 2 i to 1 plus i, what would I get?
4 plus 3 i
4 plus 3 i.
But if I multiply, multiply, 3 plus 2 i times, let's say I square it.
I multiply 3 plus 2 i by 3 plus 2 i.
What do I get?
If I do it with this rectangular form, I get a mess.
I can't see what's happening.
It's the same over here.
This is like having a 1 and a 1, with an addition.
This is like a polar form where it's one term.
OK.
So let me answer the question here and then let me answer the question there.
And then you've got a good shot at what complex numbers can do, and why we like the polar form for squaring, for multiplying, for dividing.
What's the polar form?
Well, I'm using that word "polar" in the same way we use polar coordinates.
What are the polar coordinates of this point?
They're the radial distance, which is what?
So what's that distance?
That's the R you could say.
It corresponds to this R here.
So I'm just using Pythagoras.
That hypotenuse is what?
Square root of 13
Square root of 13.
Thanks.
9 plus 4, square root of 13.
And what's the other number that's locating this in polar coordinates?
The angle.
And the angle.
What can we say about that angle?
Let's call it phi is-- what's the angle?
Well, it's some number.
It's between 0 and pi over 2, I'm sure of that.
What do I know about that angle?
I know that this is 2 and this is 3.
So that's telling me the angle.
Well, what is that really telling me immediately?
It's telling me the
Tangent
Tangent of the angle.
So the tangent of the angle is 2 over 3.
And the magnitude is square root of 13.
OK.
So those beautiful numbers, 2 and 3, have become a little weirder.
Square root of 13, inverse tangent of 2/3.
You could say, well, that's not so nice.
What was I going to do?
I was going to try squaring that number.
So if I square 3 plus 2 i, or if I take the 10th power of 3 plus 2 i, or the exponential, all these things, then I'm happy with polar coordinates.
Like, what would be the magnitude of the square?
And where will the square of that number, so I want to put in 3 plus 2 i squared, which I can figure out in rectangular, of course-- a 9, and 6 i, or 12 i, or 4 i squared, stuff like that.
It's not pleasant.
What's the magnitude, what's the R for this guy?
What's the size of that number squared?
Yes?
Say that again
13
13.
Right.
I just have to square this square root so I get 13.
And the angle will be, what's the angle for the square there?
I don't want a number.
I guess I'm just doing this.
R e to the i phi squared is R squared.
And what's the angle here?
E to the i phi squared is e to the 2 i phi.
It's the angle doubled.
E to the 2 i phi.
The angle just went from phi to 2 phi.
The lengths went from square root of 13 to 13.
Squaring, multiplying is nice with complex numbers.
Maybe can I before I go on and on about complex numbers, I should ask you, how many know all this already?
Complex numbers are familiar?
Mostly.
Correctly, with a wiggle.
OK.
I won't go more about complex numbers.
Let me come back to my question here.
Let me come back to the application.
So here it is with complex numbers.
Here it is with sinusoids.
And the little beautiful bit of math is that the sinusoid question goes completely parallel to the complex number question.
So you have an idea on those complex numbers.
We'll see them again.
Let me go to this.
So I want this to be the same as this, OK. Maybe I'm going to have to use a new board for this.
Can I start a new board?
So I want cos t plus sine t to be some number R times cosine of t 1.
I can see omega's 1, so I just put to 1 minus some angle.
OK. And I want to choose R and phi to make that right.
You see what I like about it?
This tells me the magnitude of the oscillation.
It tells me how loud the station is.
When I see cos t and, separately, sine t, or I might see 3 cos t and 2 sine t. 3 cos t is a cosine curve.
2 sine t is a sine curve shifted by 90.
I put them together, it bumps, it bumps, bumps.
Not completely clear.
It seems to me just beautiful that if I put together a cosine curve that we know, that starts at 0, with a sine curve that starts at 0, the combination is a cosine curve.
Isn't that nice?
I mean, you know, that sometimes math gets worse and worse whatever you do.
But this is really nice that we can put the two into one.
But you see, it's going to-- well, let's do it.
What would R and phi be here?
So I'll use a trig fact here.
A cosine of a difference of angles, so this is R times cosine t, do you member this?
This was the whole point of going to high school.
Plus sine t sine phi.
So now, how do I get R and phi?
I use the same idea that worked last time.
I match the cosine terms and I match the sine terms.
So the cosine t has a 1.
1 cosine t is R cos phi.
That's what's multiplying cosine t. And the sine t has a 1.
And that has to agree with R sine phi.
So I'm in business if I solve those two equations.
And well, they're not linear equations.
But I can solve them.
Of course, the one fact that you never forget is that sine squared plus cosine squared is 1.
Right?
So if I square that one, and square that one, and add, what will I get?
1 squared and 1 squared will be 2, on the left hand side.
On the right hand side, I'll have R squared cos squared, R squared cos squared, and plus R squared sine squared.
And what's that?
What's R squared cosine squared plus r squared sine squared?
R squared
It's just R squared.
So all that added up to R squared.
In other words, it's just like polar coordinates.
R is the square root of 2.
That's telling us the magnitude of the response.
Square root of 2.
You see, it's just like complex numbers.
It's like the cosine gave us a real part and the sine gave us an imaginary part.
And R was the hypotenuse.
And that's really nice.
So R is the square root of 2.
OK. Now, the angle is never quite as nice.
But how can we get something about an angle out of there?
All we could get in this case here was the tangent of the angle.
And I'll be happy with that again here because it's a totally parallel question.
How am I going to get the tangent of the angle?
What do I have?
From these two equations, I want to eliminate R. So how do I eliminate R?
What do I do?
Divide.
Divide.
I guess if I want tangent sine over cosine, I'll divide this one in the top by this one in the bottom.
So I take the ratio.
That'll cancel the Rs perfectly.
It'll leave me with 10 phi.
And here it happens to be 1.
OK.
So what have I learned?
I've learned that when these two add up together, they equal what?
R square root of 2.
You see how easy it is.
Square root of 2 came from the square root of 1 plus.
It's like Pythagoras.
Pythagoras going in circles, really.
Times the cosine of t minus.
And what is phi?
Its tangent is 1, so what's the angle phi?
Pi over 4
Pi over 4.
Right.
So that's the sinusoidal identity when the numbers are 1 and 1.
But you saw the general rule.
Let me just take it.
Suppose this is the output, and cos omega t plus n sine omega t. What is the gain?
What's the magnitude, the amplitude, the loudness of the volume in this when I'm tuning the radio?
What's the R for this guy?
What's this R?
If we just follow the same idea.
So if we have m times a cosine and n times a sine, what's your guess?
What's your guess for R, the magnitude?
I'm guessing a square root of what?
Yeah?
You got.
What is it?
n squared-- [INTERPOSING VOICES]
Plus n squared.
Way to go.
M squared plus N squared.
And the angle is like the phase shift.
I'm not great at graphing, but let me try to go back to my simple example.
If I tried to add up on the same graph cosine t, which would start from 1 and drop to 0, go like that, right?
Something like that would be cosine.
And now I want to add sine t to that.
So that climbs up to 1 back to 0, down.
And now if I add those two, this formula is telling me that it comes out neat.
Neatly.
That one plus that one is another sinusoid with height square root of 2.
If I had different chalk, I've got at least a little bit different.
But does it start here?
Of course not.
It starts here, I guess.
But it goes up, right?
Because this comes down, but this is going up.
All together, it's up to, where is the peak?
Where is the peak on the sum?
So I'm adding, everybody sees what I'm doing?
I'm adding a cosine curve and a sine curve.
And it goes up.
And where does it peak?
What angle is it going to peek at?
What's the biggest value this gets to?
[INAUDIBLE]
At pi over 4, it'll peak.
At pi over 4, it'll be the cosine of 0, which is 1.
It's height'll be the magnitude, the gain, square root of 2.
So it'll peak at pi over 4, which is probably about there, right?
Peak at pi over 4 and, I don't know if I got it right frankly.
I did my best.
That's the sum.
That right there.
The first key point is it's a perfect cosine.
The second key point is it's a shifted cosine.
The third key point is its magnitude is the square root of 1 squared plus 1 squared, or n squared plus n squared, or a squared plus b squared.
So that's the sinusoidal identity.
A key identity and being able to deal with forcing terms, source terms, that are sinusoids.
OK. Now, I'm going to take one more step since we have just like 10 minutes left, and let the number i get in here properly.
Get a complex number to show up here.
OK. Before I start on this, let me recap.
Let me recap today's lecture.
It started with nonlinear separable equations.
And a great example was the logistic equation up there, with the S curve.
That took half the lecture.
The second half of the lecture has started with things real with sinusoids that are combinations of cosine and sine and has written them in a one term way.
And now I want to get the same one term picture from using complex numbers.
OK. OK. And everything I do would be based on this great fact from Euler that e to the i omega t. The real part is cosine omega t. And the imaginary part is sine omega t. That's a central formula.
Let me draw it rather than proving it.
Let me draw what that means.
I'm in the complex plane again.
Real part is the cosine.
The imaginary part is the sine.
That number there is e to the i omega t because it's got that real part and that imaginary part.
And what's its magnitude?
What's the R, the polar distance for cos omega t plus, for this number, which is for this number?
What's the hypotenuse here?
Everybody knows
1
1.
Hypotenuse is 1.
Cos squared plus sine squared is 1.
So e to the i omega t is on a circle of radius 1.
That's the most important circle in the complex world, the circle of radius 1.
And all these points are on it.
And their angles are omega t. And as t increases, the angle increases, and you go around the circle.
You've seen it.
Physics couldn't live without this model.
OK.
So that's basically what we have to know.
And now, how do we use it?
Well, the idea is to deal with the equation.
Like, the equation I had last time was dy, dt equals y plus cos t. That gave us some trouble because the solution didn't just involve cosines, it also involved sines.
Yeah.
So I want to write that equation differently, in complex form.
And this is the key point here.
So I'm going to look at the equation dz, dt equals z plus e to the i t. Well, I'll make that cos omega t just to have a little more, the units are better, everything's better if I have a frequency there.
Units of this are seconds and the units of this are 1 over seconds.
Now, question.
What's the relation between the solution z to that complex equation and the solution y to that equation?
Of course, they have to be related, otherwise it was stupid to move to this complex one.
My claim is that complex equation is easy to solve.
And it gives us the answer to the real equation.
And what's the connection between y and z?
So y's the real part
Y is, exactly, say it again
The real part--
Of z. Y is the real part of z.
So that gives us an idea.
Solve this equation and take it's real part.
If I can solve this equation without getting into cosine and sine separately and matching, I can stay real.
I solve the equation by totally real methods up to now.
Now I'm going to say, here's another approach.
Look at the complex equation, solve it, and take the real part.
You may prefer one method.
You may like to stay real.
In a way, it's a little more straightforward.
But the complex one is the one that will show us, it brings out this R, it brings out the gain, it brings out the important-- engineering quantities are important, if I do it this way.
Now, I believe that the solution to that is easy.
Actually, it is included in what I did last time.
It's a linear equation with a forcing term that's a pure exponential.
And what kind of solution do I look for?
I'm looking for a particular solution.
If I see an exponential forcing term, I say, great.
The solution will be an exponential.
So the solution will be sum.
Z is sum capital Z e to the i omega t. Plug it in.
What happens if I plug that in to find capital Z, which is just a number?
Right.
This is my method.
This is a linear equation, with one of those cool right hand sides, where the solution has the same form with a constant, and I just have to find that constant.
So I plug it in.
Dz, dt.
Take the derivative of this, z i omega will come down.
E to the i omega t. Z is just this, z to the i omega t. And this is just 1 e to the i omega t. So I plugged it in, hoping things would be good.
And they are because I can cancel e to the i omega t, that's the beauty of exponentials, leaving just a 1 there.
So what's capital Z?
What's capital Z then?
I've got a z here.
I better bring it over here.
And I've got the 1 there.
I think the z is 1 over.
When I bring that z over here, do you see what I'm getting?
It's all multiplying this e the i omega t. It's a number there.
Z times i omega and comes over as a minus z.
What do I have multiplying z here?
I see the i omega.
And what else have I got multiplying the z?
Minus
A negative 1 because it came over with a minus sign.
Done.
Equation solved.
Equation solved.
Complex equation solved.
So the point is, the complex equation was a cinch.
We just assumed the right form, plugged it in, found the number, we're done.
But there's one more step, which is what?
Take the real part.
So I have to take the real part of this.
So the correct answer is y is the real part of that number, 1 over i omega minus 1 times e to the i omega t. I'm tempted to stop there, but just with a little comment.
How am I going to find that real part?
And what form will it have?
What form will that real part have?
Yeah, maybe just to say what form will it have?
The real part, it's going to be a sinusoid.
But I have a complex number multiplying this guy.
The real part is going to be exactly of the form we-- well, of course, it had to be the form because that was another way to solve the equation.
It's going to be some number.
And I'll call it g, for gain, times the real part.
And so the real part will be a cosine.
Yeah, it's just perfect.
A cosine of omega t. And there'll be a phase.
Yeah.
i haven't taken that step fully.
I got to that fully.
And then I said that that, if I use some complex arithmetic, will come out to be this.
And you see the beauty of that answer, which was way better than a sum of sines and cosines.
We see the gain.
We see the amplitude.
And we see the phase shift.
Yeah.
So I don't know, that would be a good exercise in complex numbers.
Find g and find phi, in taking the real part of this thing.
Yeah.
It's a pure exercise in using complex numbers.
I don't feel like doing it today.
If we do it, you just see a lot of formulas.
Here, you see the point.
The point was that the complex equation could be solved in one line.
We just did it.
But that left us the problem of taking the real part.
That was the e to the i omega t there.
Left us the problem of taking the real part.
And that's a practice with complex arithmetic.
So you've got the choice.
Either stay real-- sign plus cosine.
And then use the sinusoidal identity, polar form.
Or get the polar form from here.
Same answer both ways.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This week is my second pair of lectures.
Last week the two lectures were about first order differential equations, and this week second order.
Those are the two big topics in differential equations.
Let me start with most basic second order equation.
We see the second derivative and the function itself, and we don't see yet the first derivative term.
This is the nice case, when I just have y double prime and y.
In general, I-- I'm taking constant coefficients today.
Because if the coefficients depend on time, the problem gets much, much harder now.
So let's stay with constant coefficients, meaning we have a mass, for example, we have a spring.
The stiffness of the spring is k, the mass is m, and the y, the unknown displacement, tells us the movement of the mass.
The classical problem.
You will have seen it before.
Because you have an exam this afternoon, I wanted to start with things that-- they are about second order equations, but they're still close to the exam idea, particularly the idea of exponentials.
With constant coefficients, that's the fundamental message.
Exponentials in, exponentials out.
But it's not quite so clear when we had first order, y prime equal ay, we knew that the exponent was a.
The solution was e to the at.
Now we've got second derivatives coming in, and it won't be so much e to the at type thing.
Either the at was growth for a positive, decay for a negative.
Now we're going to see oscillation.
It's still exponentials, but oscillation.
Things going up and down, things going around.
Harmonic motion, you call it.
Sines and cosines.
And sines and cosines connect to complex exponentials.
So that instead of e to the at-- so now oscillations-- they're going to be coming from e to the i omega t. In other words, instead of an a, we're going to have an i omega.
Or, if we like to stay real, we can stay with cos-- cosine-- and sine.
And actually, I've written two real guys there, so I better have two complex ones.
And it will turn out to be plus or minus.
There are two frequencies there.
Plus i omega, minus i omega, and they turn into cosine and sine.
So in this case, with no damping term, we can stay entirely real without creating any problems.
We can work with cosines and sines.
The first question is, what's omega.
What is the frequency of oscillation.
And of course, another similar picture would be a pendulum, a linear pendulum, swinging side to side, keeping time, because that frequency will stay constant.
Always I'll start with zero on the right hand side.
Just look at these equations.
Constants there.
I'm looking for solutions.
And I'm looking for null solutions, looking for the natural motion of this spring, the natural up and down motion of this spring.
Classical problem.
Won't be brand new, but it's the right starting point for the full second order equation.
It'll get a little complicated on Wednesday, when damping gets in there.
The formula got a little messy, because you've got a mass-- you've got an m-- and a k, still, but you also will have a damping constant.
Then complex numbers really come in.
Here they're optional.
So this is my equation to solve.
Because we don't have a first derivative, a cosine will solve that.
So let me look for-- I could look for exponentials.
Maybe I should do that first, look for an exponential solution.
Yeah, that's a good idea.
And let me not jump ahead to know that the exponent has that i omega form.
Let me discover it.
So I look for no solutions-- because I have that zero there-- no solutions of the form e to the st, some exponent.
Plug it in.
That's the message with constant coefficients.
Look for exponentials, substitute them in, discover what s will be.
So let's just do that.
This is the most basic step.
For null solutions will be exponentials, I substitute into the equation.
I get ms squared from two derivatives.
It will bring s down twice.
This is just ke to the st, and I'm looking for null solutions.
Zero.
No forcing.
So this is undamped, unforced.
Undamped, unforced.
Natural motion.
What do I do now?
Plugged in an exponential, got this equation.
And the beauty is that the exponentials cancel.
An exponential is never zero, so I can safely divide by it.
So I cancel those, and I get ms squared plus k equals zero.
The key equation-- and it's so simple-- it's just we're doing algebra now.
The calculus, the derivative we took when we plugged it in, but now it's an algebra question.
And of course, solving that system is easy.
There's no s term, no damping term.
So the frequency, s, is-- put k on the other side, divide by n. s is-- s squared, let's say-- is k over m-- is minus k over m. Critical point.
That tells me, with that minus sign there, that s is an imaginary number.
A complex number has a real part and an imaginary part.
In this case, all imaginary.
No real part at all.
It's natural to think-- s is the square root of that, so I'm going to write-- everybody writes-- s, the frequency s, is i omega.
So if I plug that in, I have omega squared equal k over m. i squared and the minus 1 deal with each other.
So the frequency omega-- here is the great fact-- square root of k over m. That's-- yes?
What difference does having imaginary parts to answer affect the oscillation?
To-- OK. Oscillation, just pure oscillation-- which is what we would see here with no damping-- is the frequency, e to the-- the solution-- the displacement, I could write-- the displacement up and down, y, will involve e to the i omega t, and e to the minus i omega t. We've got second order equation.
Let me just go back to that key point.
When we have second order equations, we look for two-- we expect and we want and we need two-- solutions.
There will be two.
I didn't put it here, and I should.
s, the frequency, is plus or minus i omega, because in both cases when we square, it comes out right.
So we get two frequencies, and here they are.
So let's see how to answer your question.
The presence of this i is only telling me that, essentially, I've got sines and cosines.
That's really what-- when it's a pure imaginary number-- I would call that a pure imaginary number, there's no real part at all-- then equally cos omega t and sine omega t. I can now, if I want, go real.
I can say, OK, these were the general null solutions.
Let me put this down, then.
The null solution-- I'm looking only right now at null solutions-- is some combination of e to the i omega t and e to the minus i omega t. That's what we got from plugging in e to the st, discovering that s was an imaginary number, and we got these guys.
But equally-- equally-- yn is a combination of cos omega t and sine omega t. And maybe you'll like those better.
I think everybody practically likes those better.
Do you see that these guys are the same as these guys?
The c's are a little different because, well, we know that we can switch from one to the other.
We remember that basic fact that e to the i omega t is cos omega t plus i times sine omega t. You're used to maybe seeing that omega t as theta, e to the i theta is cos theta plus i sine theta.
And e to the minus i omega t, of course, is cos omega t minus i sine omega t. I hope you won't think I'm filling the blackboard with formulas, because I'm really just writing down-- well anyway, they're beautiful formulas.
So if I have these guys, then I have these and vice versa.
If I have cos-- how would you write cos omega t using the exponentials?
I want to just see totally clearly that I can go back and forth between complex imaginary exponentials and cosines and sines.
So how would I, I want to go in the opposite direction and write the cosine and the sine as combinations of these, just to show if I've got combinations of one, I've got combinations of the other.
Combinations of these are the same as combinations of those.
So what is cos omega t in terms of these guys?
[INAUDIBLE] some of them divided by 2?
Exactly.
If I add those two, this part cancels.
I've got two of these, so I have to divide by 2, as you say.
It's a half of the first plus a half of the second.
And how about sine omega t?
Sine omega t is always slightly more annoying, because it's the one-- it's the imaginary part that brings in an i.
What would be the same formula?
How could I produce sine omega t out of that?
Yes?
The difference divided by 2i
Yes.
If I take the difference, that'll cancel the cosines.
So I'm going to take e to the i omega t minus e-- minus e to the minus i omega t. Take the difference.
But then I've got 2i multiplying this sine.
Up here I had a 2, but now I've got-- when I take the subtract, these i's are in there, so I divide by 2i.
So this just tells me that I can go either way.
Next time, we'll see what happens when there is damping and there are complex numbers instead of pure i omegas.
We're golden here.
We've found the great quantity with the right units.
The right units of omega are 1 over time.
Actually the units are radians per second, would be the typical appropriate unit.
Radians per second.
I'll use the word frequency for that, but there's another definition of frequency, cycles per second.
I just want to think about steady motion around a circle.
So this tells me how many radians per second.
And if this is 2pi-- if omega happened to be 2pi-- then I would go once around the circle.
If omega was 2pi, then when t reached one, I would be around the circle.
Let me draw a circle in a minute.
So there's a 2pi here hiding behind the word radians.
And in many cases, you'll want also a definition in cycles per second.
So f is omega divided by the 2pi, and that's in cycles per second.
Full revolutions per second.
And that's hertz.
I think I misspoke last time in confusing these two, so let's get them straight here.
There's no complicated math in here, it's just a factor 2pi, but of course that factor is important.
So a typical frequency in everyday life would be like f, 60 cycles per second, 120pi radians per second.
So I'm going around in a circle.
Now I'm ready to have initial conditions.
This connects, again, to the afternoon exam.
We found the general solution with some constants, like here.
Let's keep that real form.
And now those constants get determined by the initial conditions.
Conditions plural, because we have an initial position, like I stretch it-- maybe I stretch it and let go.
Maybe I stretch the spring and then I let go.
What happens?
By stretching it, I'm giving it an initial displacement.
And I'm giving it zero initial velocity, because I stretched it and just let go.
Another possibility would be to strike it.
If I hit that mass, that would be a different initial condition.
What would be the initial condition if it's sitting there in equilibrium quietly minding its own business and I hit it?
Then I've given it an initial velocity, with initial displacement zero.
So those would be the two extreme possibilities.
Pull it down, let go, or strike it when it's sitting in equilibrium.
Anyway, we've got two initial conditions.
You see why-- y double prime is showing up because essentially we've got Newton's Law.
This is Newton's Law.
Mass times acceleration is equal to minus ky-- that's the, with the minus sign, and the all-important minus sign, that's the acceleration.
That's a force, sorry.
Mass, acceleration, this thing with a minus sign is the force, and the force is pulling back.
If y is stretching, the force is restoring.
Let me just go ahead with what you know.
The initial conditions.
And I want to solve my double prime plus ky equals 0.
So I'm still talking about the unforced with given y of 0 and y prime of 0.
Just think for a moment.
Could you do that?
This is the most basic second order equation.
We know what the solutions look like.
Let's do this one in a box, cosines and sines.
We know what omega is.
Omega had to be square root of k over m. Then the equation was solved.
All I've got left is to get c1 and c2.
All I have left is to match-- choose c1 and c2 to match the two initial conditions.
So let me just do that.
What are c2 and c2?
At time zero, I have to match an initial displacement.
So at time zero, this is a 1, cosine of zero is a one, and that's a zero.
So at t equals 0, I have y of 0.
The displacement matches c1 times cosine of omega t, which is a 1, plus c2 times 0.
I'll put it in there plus c2 times 0.
C1 cos 0 plus c2 sine 0.
I've learned c1.
And also-- what do I do next?
I want to get c2.
And where is c2 coming from?
Now I would like to know what's the coefficient of the-- the initial conditions are supposed to determine that coefficient.
It'll be that initial condition that determines it.
y prime of 0.
The initial velocity should match the derivative.
OK, so what's the derivative?
y prime.
So the derivative of the cosine will be a sine.
And that will disappear at t equals 0.
The derivative of this sine will be a cosine with a factor omega.
So I'll have y prime of 0 will be the c2 omega cos of 0.
Which is what?
That tells me c2.
You could do all this without my pointing the way.
I'm solving this equation.
I have the solution in general form with two constants.
Now I'm determining those constants, and cosine and sine just determine them perfectly because cosine is 1 and sine is 0 at the start.
So we've got the answer.
The solution is y of t is c1 is y of 0 cos omega t, and c2 is-- now you'll notice, c2 is y prime at 0 divided by omega.
y prime of 0 divided by omega sine omega t. There we go.
Finished.
Finished.
Unforced problem solved.
Everybody in this room could get to that point.
Let me make some comments about that.
It's a combination of cosine and sine.
They're both running at the same frequency, omega.
I'm going to give a special name to that frequency, omega, this famous formula, all-important.
Lots of physics in that formula.
I call that the natural frequency, because the next step will be to drive the system by a driving frequency, which would be different from omega.
So we need to-- we've got 2 omegas.
Actually when I first wrote the book I thought, we've got to keep these two separate.
Everybody has to keep them separate.
My first attempt was to use little omega and big omega for the two.
I concluded after looking at it for a while that it was better to be more conventional.
People had figured out a good way to do it.
And the good way is to call this the natural frequency and put a subscript, m. So all the omegas that you see on this board should be omega n. I can change them all, but let me just change it here.
I'll change omega n. So that's omega n. We've only got that one omega right now, because we don't have a driving term yet.
So natural frequency has the advantage, which kind of made me smile, that the n stands for natural, and everybody calls it the natural frequency.
And n also stands for null, and we're talking here about the null solution, because there's no forcing.
So I could have subscripts on all the y's.
Eventually I'll need subscripts on the y to separate what we've done.
This is really yn of t, the null solution.
Good?
Now we could take one more step.
This is a combination of cosine and sine.
And we learned last time that that could be put in a polar form, but I don't plan to do this.
Let me just say I could do it.
This would be some amplitude, some gain-- maybe g for gain-- no, a for amplitude is good-- times-- what is this second optional form, which I'm just going to write here, say that we could do it, remember a little about it, but not make a big deal-- what is it I'm after here?
I'm looking to write this combination of cosine and sine, which is two oscillations, a cosine curve and a sine curve, but with the same frequency.
Then I can combine them into a single cosine, a single cosine of omega nt.
But now what else have I got in this form?
There's a phase shift, minus phi.
Thanks.
So there's an a phi, two constants, or there's y of 0, y prime of 0, two constants.
Let me not write again the formula for a or for phi, I don't plan to do anything with it.
It just could be done.
In other words, what we've done so far is just to see that the single spring oscillates with the frequency omega n. That's really what we've done.
A single spring oscillates with a frequency omega n. Saying that makes me think, let me look ahead to the linear algebra part of the course.
So where is linear algebra going to come in?
It's going to come in for a system of springs.
When I have another spring.
Can I draw another spring and another mass and another spring?
Say six springs, six masses.
Then-- and they could be different k's, different m's, or not.
Then we've got six displacements-- six differential equations-- coupled together, because the whole system is coupled together.
So what happens at that point?
That's the point where linear algebra, where matrices are coming in.
You want to see what's the point of matrices.
It's not a separate course by any means.
It's a most necessary part, because a single spring happens in reality but also systems today are coupled.
Big, actually, there are many, many things.
You have an electric circuit with thousands or tens of thousands of elements.
You have a coupled system with many gears, many oscillations going on.
So we need matrices at that point.
Can I even just add one more word about the language?
When we had-- here we have a frequency of motion for one spring.
What are we going to have for two springs or six springs?
The motion will be a combination of six different frequencies.
And so you'll see that it's a much more interesting, much more not so simple motion.
A combination of six pure frequencies.
And those frequencies are determined from the six eigenvalues of the matrix.
I'm just using that word looking ahead.
We will have a 6x6 matrix to describe the coupled system.
That matrix will have six eigenvalues.
It will tell us six natural frequencies, and our solution will be a combination of all six oscillations.
Here, it's 1.
Here it's 1.
That spring is not there.
So the problem we've solved now is the fundamental, basic problem, and I have to-- next step is forcing.
I now want to add a force that drives the motion.
In general, it could be any function of time.
Calling it f of t. So that's what I'm going to put in now.
But in reality, very, very, very often f of t is also a simple harmonic motion.
It's also a cosine.
But at a different frequency, at a driving frequency.
So I'm going to-- the next equation to solve is to put in cosine-- let's stay real for now-- at another, driving frequency.
At a driving frequency.
And of course, it could have an amplitude.
But let me take that amplitude as 1 to keep things simple.
So now I'm talking about forced motion.
Can we solve it?
How can we solve this equation?
Let me take out the 0 or-- take out the 0-- equals cosine omega t. With a different omega.
If the two omegas were the same, if the driving frequency is the same as the natural frequency, the formulas have to be slightly adjusted.
There's still an answer, but it's a case of resonance and you have to look separately.
But let's say, no.
Let's say omega d is different from omega n. How are you going to solve this?
I have to think myself.
How do I solve that.
Let's start a fresh board.
my double prime plus ky equals cosine of omega dt-- or often, I won't put the d. I don't have to put the d anymore.
Omega will now represent the driving frequency, because I've got omega n, the natural frequency, as the square root of k over m. What am I looking for now?
I found the null solution.
I'm looking for a particular solution.
I'm trying to keep the whole thing systematic.
Null solutions are now dealt with.
Took a little more time than just ce to the at for first order equations, because we've now got a two-dimensional collection of null solutions, but we've got them.
Now I'm taking a forcing term.
So I'm looking for a particular solution.
I'm looking for any solution to this equation.
I'm looking for a particular guy.
What do you suggest?
Again, it's a neat problem because of that particular forcing term, a cosine, an oscillation.
So I'm going to look for yp is some gain times [INAUDIBLE].
This is the next and, fortunately, a highly, highly important case, in which the particular solution has the same form as the forcing term.
It's just a multiple of the forcing term.
That's best possible.
That's best possible, is to have the forcing term reveal to me-- the forcing term immediately reveals a particular solution.
Once I know what I'm looking for, what do I do?
Substitute it in.
So I substitute that particular solution in here.
And notice everything is going to be a cosine, mgy double prime.
So what do I get when I plug this in for that guy?
I want to-- you can do it quickly, but let's stay together and do it together, because we can with this case.
What happens when I plug that in and take its second derivative?
I get the g. And then what's the second derivative?
[INAUDIBLE]
We have a negative, because two derivatives of the cosine bring out a minus omega d, will come out twice.
And I'll keep writing omega d for a moment, but then I'll give up on the d. Cosine of omega dt.
And then k times this, g cosine of omega dt, equals the forcing term, cosine of omega dt.
It worked.
This is one of that small family of nice functions where the solution has the same form as the function.
Actually that list of what you could call best possible forcing functions, where the form of the forcing function tells you the form of the solution.
That's a small family.
But it's fortunately a very important one.
Cosines, sines are included, and we'll see all the other guys that are included.
Most forcing functions we couldn't just assume that the solution had the same form.
It's only these nice ones.
But cosines are nice.
So what do I do now?
Everything is multiplying cosines, so I just look at-- I have minus m omega squared g-- g is going to factor out-- minus m omega squared and a k times g. Let me remove that off for the moment.
I'm canceling cosine omega, so my right hand side is 1.
That's it.
We looked for a solution with that simple format, and we found it.
Now we know g, the gain.
So the solution is-- this is g is 1 over k minus m omega squared times cosine of omega t. And omega is omega d. Omega is omega d now.
Does that look good to you?
This is the periodic solution going at the driving.
This is what the-- this g is the gain, the driving force.
The driving force is 1 times cosine omega d, then that 1 gets multiplied by this number.
This is, you could say, the amplifying factor.
I guess frequency response would be the right word.
Can I bring in that word, response, again?
Response is a word for a solution.
It's what comes out.
When the input is this, a pure frequency, the output, the response, is a pure frequency-- same frequency, of course-- multiplied by that.
That is the frequency response factor.
Notice we could write that a cool way, by remembering that omega squared-- that's wrong as it stands.
What have I forgotten in writing k minus m omega squared in that denominator?
I forgot a subscript, which is n. Which is n. This is n. This is-- is that right?
No.
Is it?
Or is it d?
Maybe I didn't make a mistake.
Is it d?
You're seeing a kind of critical moment.
Which is it?
[INAUDIBLE]
It's d, isn't it?
Yeah.
It's d. Sorry.
It's d. But when I see this and remember what omega n squared is-- omega n squared is k over m-- I can see that I can get an omega-- I can use this in here to make it even more interesting.
So it'll be equals-- let me get this box ready-- cosine of omega dt divided by-- now I just want to rewrite that.
I want to take out an m. I'm going to write this as m times k over m. m times k over m. Safe to do that.
Now I have a factor, m, that I can bring out.
And what is m multiplying?
That's the neat thing.
What is m multiplying?
k over m is-- omega n squared.
And this is minus m omega d squared.
Minus omega d squared.
That's pretty terrific.
The gain is this multiplier, 1 over m, times that.
And we see that the gain is bigger and bigger when the frequency is near the natural frequency.
And of course everybody has seen the pictures of that bridge-- wherever the heck was that bridge?
Somewhere in the Northwest, I think.
You know the bridge I'm talking about?
[INAUDIBLE] Tacoma, Washington
Yeah, I think Tacoma, that's right.
The Tacoma Narrows Bridge.
Right.
Tacoma, Washington.
Where the natural-- when you build a bridge, you've built in a natural frequency.
And then when traffic comes, it's doing a driving frequency.
And if you haven't got those two well-separated, you're in trouble, as this shows.
Or similarly, when an architect designs a skyscraper, there's going to be a frequency of oscillation, a natural frequency, at which that skyscraper swings.
And then there's wind.
Actually I talked yesterday to the-- by chance, the math department is not a very party-going department, but once a year we let it out.
And so we had our party at Endicott house out in the suburbs, and all the usual people-- that's all the professors I know-- came, of course.
But also, there was a really cool person.
He's the key architect for Building 2.
You've noticed that Building 2 is under wraps and we're moved out.
And we move back in January 2016.
So we've been out a year and a quarter and we have another year and a quarter to go.
It's going to be cool.
And you may say, well, who cares.
But the key point is Building 1 is next, and Building 1 is going to have the same cool addition of a fourth floor.
We're putting in a fourth floor, which all the-- Buildings 3, 4, 5, 6 go up to four, but Buildings 2 and 1 stopped at the third floor.
But there's a lot of space up there under the roof.
And they've discovered they could put a fourth floor up there.
Here was one interesting thing, though.
These buildings that we're sitting in are sinking.
You know that MIT was built on marshy land, just the way the Back Bay-- which is like the greatest idea in the history of Boston, the Back Bay and the dam that makes the Charles River beautiful-- was built by bringing in trainloads of earth from Needham.
So whole mountains and hills in Needham have come into Boston and come here.
So anyway, we're sinking.
You may say something like 3/16 of an inch a year is not something to worry about, but now it's been more than 100 years that these buildings have been here.
Anyway, not good to sink faster.
So the weight had to be controlled.
So by putting in a fourth floor, that put in a lot a new weight, and faster sinking, probably by some formula here.
Probably there.
So the weight had to get subtracted out.
It turns out that the ceiling, the roof to Building 1-- Building 2 and no doubt to Building 1-- was more than a foot thick of concrete.
Really heavy.
And some more asbestos probably, which we don't want to think about.
That's much reduced.
A whole lot of weight came out of the roof.
I think they probably did the calculation right, so we won't get rain coming through, but it won't weigh as much and the fourth floor is acceptable.
All this was a big decision by MIT to pay for that, or to raise money and pay for the new fourth floor.
But it's going to be fantastic.
And it'll be fantastic in Building 1 also.
So all that is discussion of that formula.
That's the frequency response, this factor to frequency, omega d, or omega, is this factor.
I guess I should say something about resonance.
What happens when that formula breaks down?
When the driving force equals the natural frequency, then we're dividing by 0, and something is different.
The formula isn't right anymore.
What enters in the formula-- let me just tell you what enters, and then we'll see it in a simple example.
When I have this repeated thing, two things are equal, what tends to happen is a factor, an extra factor, t, appears.
So an extra factor, t, will appear in the case omega n equal omega d. The solution, y, will be some factor, I'll still call it g-- no, I don't want to call it g, let me call it a.
There'll be a factor, t, times cosine of omega t. So in this case, there's really only one frequency.
We're driving it.
So the oscillation grows.
As you know, when you push a child on a swing, the whole point of pushing that child is to push at the natural frequency.
You wait for this swing to swing back naturally and you drive it again with that-- at that-- maintain that frequency.
And of course you see the amplitude-- the child swing higher and higher.
Presumably you stop pushing before disaster for that child.
But that's a case of resonance.
And it's what happened in the Tacoma Narrows Bridge, and there was nothing to-- nobody stopped, traffic just kept coming.
The movie is amazing, because there's one car that shows up after it's already swinging wildly, some crazy person still driving across.
And you might think, OK, that's ancient history.
But you know the bridge in London, the pedestrian bridge, the Millennium Bridge-- it's just a walking bridge across the Thames-- a big feature of modern London, and it had the same problem.
It was swaying.
People could not walk across.
They couldn't keep their balance.
So they had change it.
So it's not trivial to anticipate.
So now we've solved it-- we've solved the the null equation with no force, and we've solved the driving force equal to a cosine.
And of course, we could do a sine.
What other driving force should we do?
I think we should do a delta function.
I think we have to understand the fundamental solution is the case when, if we can solve it-- there's always this general rule, if we can solve with a delta function, that will give us a formula for every driving force, because every function is some combination of delta functions.
So if we could do it with a delta-- really the great right hand sides are-- well, cosines and sines I'll include as great right hand sides.
Those are the exponentials in disguise.
So the great right hand sides are really exponentials at different frequencies and delta functions.
Delta of impulses.
So now I want to find the impulse response.
That's the next-- that's really a job.
At this point, in these last 20 minutes when I solve my double prime plus ky equal a delta function-- well, what I was going to say was I'm now taking you to something that you won't see on the exam this afternoon.
But maybe you will.
Delta function, right hand side.
I haven't seen it yet.
Or I haven't looked recently.
You won't see second derivatives, I guess.
So what is it?
So now this is of the form with an f of t, a very special f of t. And that very special f of t makes that extremely easy to solve.
That's really my point here, is it it's going to be a cinch to solve that, and we practically have done it already to solve that with a delta function.
And the reason is sort of physical.
We have here our spring.
And what am I doing with that force?
I'm hitting the mass.
I'm striking the mass.
Let me say, and I'll write it on the board, the point I want to make about this.
That point is that this equation with a delta function force starting from 0-- say, y of 0 equal y prime of 0 equals 0, let's give it starting from rest-- it starts from rest by hitting it.
And that hit, that impulse, is in no time at all.
It's not stretched out.
It's hit over one second.
So this has the same solution.
This is the beauty.
This is why we can solve it so easily.
Same solution as-- let me write it and see what you think-- as my double prime plus ky equal 0.
We know how to solve those.
With-- it's still, when I hit it-- when I hit it, what happens in that split second?
In that split second, it doesn't have time to move.
It doesn't move.
It still has y of 0 equals 0.
But in that split second, we've given it a velocity.
We've given it a velocity.
And that velocity will be y prime.
The initial velocity is 1-- because here I had a 1-- over an m. We have to have the units right.
So here's a point, and we will stay with it.
We'll come back to this point next time.
Maybe the first thing for you to take in is the fact that it's such a nice thing.
We have this equation with this mysterious delta function, and I'm saying that the solution is the same as this equation with no force, but starting from a mass.
I'm tempted to take an example to make this point.
Let me take an example where the whole thing is a lot simpler.
y double prime equal delta of t. I've taken the spring away, so the k is gone, the mass is 1.
What's the solution to y double prime equal delta of t?
If we concentrate on this example, we're good for today.
So my point is the same solution as-- now, what's the other problem?
I'm just repeating here, but making it simple by taking k equals 0 and m equal 1.
So the same solution as y double prime equals 0, with y of 0 equal what, and y prime of 0 equal what.
I just wanted to repeat here what I've said there, and then we'll solve it and we'll see that it's all true.
If I look for a solution to y double prime equal delta starting from 0-- this was starting from 0-- if I say that's the same as this, what should y of 0 be here?
Zero
Zero, right.
It hasn't had time to move.
It hasn't had time to move.
But in that instant, what happened to y prime?
It jumped to 1.
That's right.
That's right.
Exactly.
Now just solve that equation for me.
Solve this example for me.
Suppose y double prime-- yeah.
Here we go.
What's the solution if y double prime is 0?
What are the solutions to y double prime equals 0?
[INAUDIBLE]
Constant and linear.
a plus bt, right, have second derivative 0.
Now what's the solution that starts from 0 that kills the a and has slope 1?
What's the answer to that question?
[INAUDIBLE]
t. t. The solution to this equation is a ramp.
It's zero everything in this course, is zero up until time 0.
At time 0, in this example, all the action happens.
Everything happens.
And what happens is it gets a velocity of 1, and the solution is y equal to t. y is 0 here, of course.
At that point, that's the key point, t equals 0, right there-- it gets a slope.
We don't have a step function.
There's no jump in y.
The jump is in y prime, the y prime the velocity jumped from 0 to 1.
That's exactly-- I think when I introduced delta functions and drew a picture.
What is the derivative, the first derivative, y prime, for that guy?
Let's just review, because this is what we've seen already.
The first derivative is--
[INAUDIBLE] step
A step.
And the second derivative is delta.
The second derivative of this is the first derivative of a step.
The derivative of a step is 0 everywhere except at the step, at the jump when it jumps to 1.
So that's the solution in this example.
And now to end the lecture, let's solve it in this example.
Again, let me just say-- why do I like this forcing term?
Mathematically, I like it because, if I can solve that guy-- as we're doing, we are solving it-- if I can solve that one, I can solve all forces.
Over here, I could solve when I had a very happy f of t, a perfect f of t, where I could guess the answer and push through.
Now with a delta, I can build everything out of delta functions.
That's why I like it mathematically.
Why do I like it physically?
Because it's a very physical thing to have an impulse.
That happens in real time, in real things.
And by the way, let's just, before I write down any more formula, what would-- I would like to be able to solve it for a step function.
[?
Heavy thud.
?]
I would like to be able to do that one.
I'm going to have to erase something, or I'll write it right above just for the moment.
I would also like to solve my double prime plus ky equal a step function.
So I would call the solution, y, the step response.
And what would be a step function start?
A step function start would be like turning a switch.
Suddenly things happen.
That's forcing by a step, so I'm looking for the step response.
And how do you think these two are related?
I look at the relation at the right hand sides.
What's the relation of this step to the delta?
Yeah?
One's a derivative
One's a derivative of the other.
And we've got linear equations.
So the right hand sides.
The step response, y step, and the delta response, y delta-- I'll use a different letter for this because it's so important.
One is the derivative of the other.
The great thing about linear equations is we have linear equations, differentiation, integration.
Those are linear operations.
The step function is just like a steady-- anyway.
I was going to-- I won't-- is the integral of the delta.
Step function is the integral of the delta, so the step response is the integral of the delta response.
I guess to finish the lecture, why don't we solve this problem, which looks tricky because it's got a delta.
Instead, we'll solve this problem, which doesn't look tricky at all.
It's exactly what we started the lecture with.
Zero forcing and some initial conditions.
So let me just finally make space for the big deal from today's lecture, which would be the fundamental solution with a force by a delta.
I'm just going to write down the answer when you tell me what it is.
What's the answer to that?
What's the solution to this second order constant coefficient unforced equation with those initial conditions?
We probably had it here.
I may just have erased it.
But now let's get it.
So y is y delta.
This is the impulse response.
y of t-- and I'll give it later another name.
So here's a perfect review question.
What's the solution to this problem?
Everybody remembers-- what are the solutions, what's the general form for the solution to the equation?
I'm reviewing today's lecture.
The solution to that equation looks like what?
[INAUDIBLE]
It's a cosine and a sine, right.
And then how much of a cosine do we have and how much of a sine do we have?
The initial condition will tell me how much of a cosine we have.
And what's the answer?
None?
No cosine.
This condition, this initial velocity, will tell me how much of a sine we have, because the sines are the things that have initial velocities.
So it would be a sine of-- the sine of what?
Square root of k over m, right?
omega nt, right?
And what's the number?
What's the number so this has the right-- let me write again what I want.
I want y prime at 0 to be 1 over m. What's the number that I put in there?
I've got something, its derivative, at zero.
This is some number-- I'll call it little a for the moment, but I want to find out what it is.
Are we right?
Yeah?
I think we're right.
Yeah.
The derivative is at zero, so I just plug that into here, take the derivative at zero-- of course that makes it a cosine, which will be 1-- but it also brings out that factor.
So a times-- well, that factor will be 1 over m, and that tells me what a has to be.
Well, this is omega.
So a is-- this is omega a equal m. This is 1 over m omega.
And that's omega.
Sorry I'm erasing stuff which I-- this is the formula I'm after.
Sine omega t over omega.
I think we're good.
Are we?
Yeah?
Yeah.
I'll come back to this in-- Wednesday is my day to move to damping terms.
I've intentionally stayed with undamped equations here, because you're thinking about that level of equation.
Damping is going to bring in new stuff, and that should wait till Wednesday.
Shall I recap today?
I'll just recap today, and then we're done.
Today started with the unforced equation.
We solved it by assuming-- by not thinking ahead, just assume I have an exponential, because the beauty of exponentials is, when I plug it in, the exponential cancels.
And that told me that s was pure imaginary.
It told me that it had this form, e to the i omega t. And there were two s's.
Two possible s's, plus and minus.
I get to make a little comment about this example here.
What was omega?
What's the natural frequency in this problem?
What's the natural frequency here?
I guess this is a case where-- what's the natural frequency?
I guess this is a case where m is 1 and k is 0, is that right?
This does fit into that pattern, but it's a little special.
This is a case where m is 1 and k is 0.
So what's the natural frequency in this?
Zero
Zero.
Zero.
This is a crazy case of resonance.
It's a case in which the natural frequency and the driving frequency, say in this-- I'll have to do it here-- this simplest of all equations is, in a way, special.
It's a case when the natural frequency is zero and the driving frequency is zero and they're equal.
And what happens with resonance?
What's the new formula, the new term that comes in with resonance?
It's t. You saw it happen for this example, and we didn't have to use the word resonance.
We knew that we had a ramp.
We just used the word ramp, not resonance.
But this is a case of resonance.
When omega n is zero and omega d is zero.
And the factor t up here.
Anyway.
Just that small comment there.
And now, just going back to the recap.
The recap was, we tried exponentials.
We learned that they were pure oscillations.
We realized that we could do cosines and sines instead, and we did.
And we took off.
We got the formula.
Then of course the-- so this is section 2.1 of the book.
And it goes through all those steps carefully.
Section 2.2 of the book tells us about complex numbers, and section 2.3 brings damping in.
So that's what's coming next time.
So the recap again.
We found the null solution, we found a particular solution-- oh there's just one comment I want to make, and then I'm done.
Where was our particular solution?
Yeah.
This was our particular solution.
Here's my comment.
Here's my comment.
Suppose I want to solve this basic equation starting from a given y of 0 and a y prime of 0.
I'm going to do it in two parts, I think.
I've got the null solution, and I've got this particular solution.
Now here's my point.
If I want to get y of 0-- how shall I say this.
You can't just put together-- it's an easy mistake to make-- solve the null equation with the initial conditions and then add in the particular solution.
You'd think, I just followed all the rules.
But this particular solution that you added in has a-- at t equals 0, it's not zero.
So you have to change.
So the correct thing, the correct yp plus yn-- let me make that point.
Just a warning.
So in words the warning is, remember that the particular solution has some initial condition-- in that case, g-- and then that is going to affect the right null solution.
So again, y is y null plus y particular-- plus y of particular-- so it's some c1 cos omega nt plus some c2 sine omega nt plus this particular guy, g, cosine of omega dt.
All correct.
All correct.
But now, put in the initial conditions.
y of 0 is given.
And what do I get on the right hand side when I put in t equals 0?
I get c1 here.
What do I get when I put t equals 0 in there?
Nothing.
What do I get when I put t equals 0 in here?
g. So it's not c1 equal y of 0 anymore.
That's the easy mistake that I'm correcting.
When you put in this particular solution, it has an initial value.
That initial value is going to come in here.
So c1, then, the correct c1 is y of 0 minus g. End of story.
Just don't be too quick to just add the two pieces and think you can do them completely separately, because you're putting them together.
And then you have to put them together in the initial condition.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So morning.
This is my fourth lecture on differential equations, that part of the course.
And I haven't said anything about the textbook.
That's Differential Equations and Linear Algebra.
I wrote it because so many courses like this one want to combine those two topics.
They're the two major topics of undergraduate math, after calculus, the two major directions, and the book connects those two directions.
And I just wanted to give you the website for lots of things connected with the book.
So it's the math website, dela for differential equations and linear algebra.
And so today is differential equations, second order, with a damping term, with a first derivative term.
So that in many engineering problems, those coefficients A, B, C would have the meaning of mass, damping, and stiffness.
Mass, damping, and stiffness.
And physically, we know what the mass comes from.
The stiffness comes from a spring, as I drew before.
And traditionally we describe the source of damping as a dashpot.
I guess in my whole life I've never seen a dashpot, but maybe it's-- think of a piston going up and down within a cylinder of oil or something, with resistance.
OK, so that's the left-hand side.
Linear constant coefficients still.
We don't have formulas, it's not easy to see what's happening when things are varying, when the equations non-linear, so this is the starting place.
OK. With some forcing.
So this is damped forced motion.
This is the ultimate within linear equations.
OK. And now, what's the forcing?
So now, always we solve first for the null solution.
With no force, what are the natural motions?
And we'll find a formula for those.
Then the big one, the special right-hand side is always an exponential.
So this is going to be y is going to be the null solution yn, and we'll get a formula for that.
This, the exponentials.
Always, the response to an exponential is an exponential.
That's like the most important fact in getting solutions.
So the response will be some ye to the st at the same frequency-- well, I say frequency.
If s is a real number that would be a growth or a decay, but very frequently s is an imaginary number, like last time.
And this is comes from a rotation or an oscillation.
And then, the complete picture comes from being able to solve it with an impulse.
So then y is the impulse response, which I write as g of t. Let me introduce just that letter g. That stands for growth factor in first order equations stands for Green's function, so that word Green is getting in here, his name.
And it represents the impulse response.
OK, one, two, three.
And then the point of doing this one is that then we can do it, then we can get a formula for any f of t. So this is the pattern that we followed for first order equations.
We followed it for second order equations that didn't have damping.
And now we're doing the big one with damping.
So what's the damping going to do?
What's going to be effect of damping?
Say if I had a right side of one, just a unit force?
The damping.
So I'll still have oscillation-- you'll see this-- I'll still have oscillation, but its amplitude damps out.
Like that like everything we know, like something swinging back and forth, but friction is it is eventually damping out that motion.
So it'll be oscillating with exponentially decaying amplitude.
Let's find this solution.
OK. How do we find the null solution?
I've got constant coefficients here.
So the nice, the right thing to look at is exponentials.
In first order equations it was e to the at.
Here, we'll have to see what it is.
So I'm going to try a particular-- I'm going to look for the right exponentials.
Certain exponentials will be null solution.
Can I do it?
So I take that equation and put in y equal e to the st.
Remember now, I'm doing 0 here so it's not the same s as the right-hand side.
OK. What happens if I put in-- so I'm looking for the null solution.
I'll try y equal e to the st. Plug it in.
I get m. Two derivatives gives me s squared e to the st, b.
One derivative gives me an s, e to the st, and k gives me k e to the st. And that's supposed to equal to 0 for the null solution.
OK. Have we made a good guess at what will work?
Yes, because I can cancel e to the st, and I come to the most important equation.
The equation that governs everything in this whole lecture is ms squared plus bs plus k equals 0.
That's called the characteristic equation, or it has many names.
It's obviously the big deal.
It's the thing.
It's an equation for s, for the special frequencies that solve the null equation with no force.
OK. And how many values of s do we expect?
Two.
So I expect solutions s1 and s2, and then my null solution is e to the s1t is a null solution.
E to the s2t is another one.
My equation is linear.
I can multiply that by any constant.
I can multiply this by any constant.
And I can superimpose, i can add, I can combine, because I can do linear algebra here.
This is the most important operation in linear algebra, multiply things by constants and add.
That's called a linear combination.
It's the basic operation in linear algebra and it's a basic operation here, because we're doing linear algebra with functions.
Do you see that that's it?
We've got it, except we should really write a formula and draw some pictures to show s1 and s2.
What would be the formula for s1 and s2, to the two solutions?
We remember that from school, it's the quadratic formula.
The two solutions to this.
Everybody remember this one?
Let's see if I do.
Does it start with-- a minus b.
And then there's going to be a denominator that I'll remember, which is 2a.
No.
I said a but I mean m. 2m.
And now this is plus or minus-- what goes into here?
B squared minus 4ac.
The key quantity in this whole business, b squared minus 4ac.
Ah, what is it?
Mk, thank you.
4mk.
Great.
And can I just remark, a little remark about units.
B squared has the same units as 4mk.
It has to or such a formula would be crazy.
So we will see that actually there's something called the damping ratio that involves the ratio of these guys.
OK, that's the formula.
But it's not like-- OK it's got a square root, and what's the point of-- the thing we have to remember with this square root is that if it's the square root of a positive number then we have a plus or minus, ordinary real numbers.
If this thing is negative, then what?
What's up if b squared is smaller than 4mk?
So if b squared is-- if there's not much damping, if it's underdamped, b squared would be smaller than 4mk.
And what then?
We've got a negative number here.
When we take its square root we have an imaginary number.
That's oscillation.
Under-damping is going to show oscillation.
Let me draw this.
Let me draw that curve for different choices of m, b, and k. This is a good picture to see.
So let me draw first of all one that has b equals 0.
OK.
So there's a curve.
What I'm drawing is, up on this curve is this ms squared plus bs plus k. Ms squared and bs and k. And here, this one is one with no damping at all.
This is just s squared plus 0 s plus 1.
That's what that curve is That's the example we saw before.
Y double prime plus y. Y double prime plus y equals 0.
This is coming from Y double prime plus y equals 0.
This is pure oscillation.
OK. Now let me bring in some damping.
Now, as I bring in damping the curve will move.
And it takes a little patience to see where it moves.
Let me have a little damping a little damping, so s squared plus 1 s plus 1.
I think the curve-- so this is s here.
And it's a parabola.
Everything I draw here is a parabola, is just that different parabolas come from different choices of b.
So I think that this choice, I think it goes a little bit like that.
That's the next one.
It goes down a bit.
Now, let me do s squared plus 2s plus 1.
Now, let's see, I really should stop for a moment and solve the equation, find the roots for each of these guys.
And then I'm going to have an s squared plus 3s plus 1.
So we're doing something very straightforward, parabolas.
But it shows us the different possibilities.
And we could give them names.
OK.
So I would call this one undamped.
And what are the roots of that equation?
S squared plus 0 s plus 1 equals 0.
What are the s1 and s2 for that guy.
So the roots of s squared plus 1 equals 0 are?
Stay with me here.
S squared plus 1 equals 0, that's the equation that has roots
I and minus i
I and minus i. I and minus i.
So this is undamped.
S1 is i and s2 is minus i.
That's the pure oscillation.
Pure oscillation, that's the case where b is 0 here, I have a square root of a negative number, and it gives me plus or minus 2i, and then the 2s cancel and I get plus or minus i.
So I can always go back to this but I'll try to choose numbers that come out nicely.
Now, what happens with some damping?
This guy.
What are the roots for this one?
Well, I better use this formula.
Now, I'm always keeping m equal 1 and k equal 1, in all those samples.
But now b has increased to 1.
So if b is 1, what do I have?
I have s is-- the roots are minus 1, plus or minus the square root-- And it's going to be just 2 down below.
What's in the square root, the all important square root.
When m and k and b are all 1.
Just do the calculation with me, so you see it.
It's negative?
Three
Negative 3.
So what's the point there?
The point is, this is going to be square root of 3i or minus i.
We have oscillation at a frequency square root of 3, and we have decay from s minus one-half.
The real part of this is giving us the drop off.
We didn't have any drop off at all in this case.
They were pure imaginary.
Now the s1 and s2 are whatever I have a minus 1, plus or minus square root of 3i over 2.
Those are the roots.
All right, I'm ready for this guy, and it's particularly nice.
It's particularly nice.
What do you see for s squared plus 2s plus 1?
When you see this parabola, now b has moved up to 2.
What's up with that one?
It's going to be-- let's see-- that looks like a perfect square to me.
Right, inside the square root is 0.
Right, exactly.
Inside the square root, b squared is 4, and 4mk is 4, so I have 0 inside this square root.
So now I have s equals minus 1 plus or minus 0 over 2.
That's when this was the case, when b moved up to 2
[INAUDIBLE] minus 2
I'm messing it up?
[INAUDIBLE] equals minus 2, plus or minus 0
Minus 2 plus or minus 0.
Thank you.
So what do I have?
What are the roots of this guy?
Negative 1.
What's the other one?
Negative 1.
A double root.
It's critical damping.
Critical damping-- it's not underdamped.
It's not overdamped.
It's right on the borderline.
And I see that, when you first saw quadratics, before anybody brought up that awful formula, you would have factored this into s plus 1 squared equals 0, and you would have discovered that s was minus 1 twice.
A double root.
So the picture there would be-- there's minus 1.
Yeah.
Everybody recognize that this, we're hitting 0, height 0.
We're hitting 0 twice at s equal minus 1.
So this is now the case.
This was b equal 0, no damping.
This was b equal 1, under-damping.
This is b equal 2, critical damping, just on the border.
And what do you think the s squared plus 3s plus 1 is going to look like.
Again, we can find it.
No, let me do the s squared plus-- let me take b equal 3 and find the roots and draw the picture.
If you're with, me that picture and these formulas tell you the difference between these four cases.
So what do I get?
Minus 3 plus or minus the square root of, 9 minus 4, is 5, over 2.
So I have two negative roots.
I have decay.
I have decay at a fast rate and a slow rate but both are giving decay.
So the curve now is coming down here and back up there, and it hits there, and it's got the two roots.
These two roots are x and x.
So these are s1 and s2.
Let me copy them over here.
S1 and s2 are minus 3 plus or minus the square root of 5 over 2.
Yeah.
Real.
Real roots.
So this is two real roots.
This is a double real root.
This is two complex roots.
And this is two pure imaginary roots.
The four possibilities.
The famous four.
OK. And for me, that picture is a-- so this was the b equal 3 curve, more overdamped.
It's a little interesting that overdamping has this root that's pretty near 0.
So overdamping doesn't mean that you go to 0 real fast.
Actually, the one that goes fastest to 0 is the critical damping.
Then, as b grows, one root gets closer to 0, so it's like slower decay, and another root is going off to fast decay.
I think you have to know those guys, because the physically that's very important, where's the damping.
But we've now found the null solution completely.
The null solution completely is-- let me write it again here-- is anything times e to the s1t and anything times e to the s2t.
And where do those constants, c1 and c2 get decided?
By the?
[INAUDIBLE]
Initial conditions.
Right.
These two conditions determine c1 and c2.
OK. Are we good?
So the null solution has already separated the different cases that depend on how much damping.
I'm ready for number two.
Number two now.
OK.
So the idea is I now have a forcing term, some frequency e to the st. And I will assume that it's not-- s not equal s1 or s2.
That's to make my life easy.
Just as last time, the formulas will break down.
And I'll have to put in something different if there's resonance, if the driving force is at the same frequency as a natural frequency.
In that case, there's a resonance.
And the way you spot a resonance formula is there's an extra factor of t. There's a growth of t. Up here, we're seeing no factors of t. Over there in the exponent, of course.
I mean, down below, there would be a t times e to the st. And the book does that carefully.
I don't see that it has a place in the very first lecture on this topic.
OK.
So this is saying no resonance.
All right.
What's the solution then?
The solution is a multiple of e to the st.
The input was e to the st.
The response is a multiple of e to st, so that's the frequency response.
Capital Y is telling us the response to s. And it's a very, very important function.
It's called the transfer function.
It's just the key to everything.
I probably say that so often, that this is the key to everything.
Well, it's partly because I just have one lecture to do a big part of differential equations, and it's got some key ideas.
And the first key idea is s1 and s2.
And the second key idea is Y.
And let's go for it.
All right, so this is the s1 and s2 picture.
I'll move that up, and now, in the equation, try Y e to the st. We hope it works.
What is this?
I'm trying, this is my solution, and it's a particular solution now.
I got the null solution.
I've moved to a particular And this is when the force is e to the st.
In other words, I'll just say again, if we have an exponential force, try an exponential response.
1803 would call this the exponential response formula.
So you could use the word exponential response, very appropriate.
You could use the word frequency response.
That frequency response is kind of the right word when s is an imaginary number giving an oscillation frequency.
OK, I'm going to plug that in.
So into the differential equation, m. Second derivative of that is s squared Y e to the st, right?
That's m Y double prime.
The beauty of exponentials.
Take every derivative, just brings down an s. The next term.
What's the next term?
Maybe do it with me, do it for me.
What happens when I plug in this as Y into b, so there'll be a b Y prime.
So what's Y prime?
[INAUDIBLE]
s, Y, this constant, e to the st. And then the final term is k times this Y itself, no derivative, so it's just Y e to the st, and that's matching e to the st. That's the force.
This is f. F, this is an exponential force.
Of course, I could and should have a constant here to give me the units of force.
Let me just keep the formula as clean as possible by taking units, so that's one.
OK what do I do?
[INAUDIBLE]
I cancel.
The nice part of 287 is canceling e to the st. That's the most fun you get.
And now, I have Y's on the left.
So Y is-- can I see what Y is?
Y is this 1, divided by the coefficient of Y.
And what's the coefficient of Y?
We know it.
We've seen that coefficient.
Y on the left is multiplied by ms squared plus bs plus k ms squared, again, ms squared, bs, and k. Multiplying Y, I divide by that, so I put it down here.
Ms squared plus bs plus k. And because s is not one of the roots, that's not 0, so we're golden.
That problem took five minutes.
The null solution took half an hour.
The exponential response is clear.
And you can see what it would be.
And let's give it a name.
This is the transfer function.
Widely used name.
Other names could be given but that's the best.
So it's a function of s. It's a function of s. And so, again, when f is e to the st, it sort of transfers the input into the output.
That's the way I think of the transfer function.
Here is the input.
The output is just multiplied by the transfer function.
And the transfer function is just that nice expression.
Just that nice expression.
So we are golden for a frequency, for a linear equation with an exponential forcing function.
What would be another example?
I'm using mass, dashpot, spring here.
If this was in electrical engineering, what three things would I be using instead of mass, dashpot, spring.
The three guys would be?
What would correspond to the dashpot?
So can I just draw here a little-- let me put on a low voltage and put on something that does this, and then something like this.
And then there's something like this.
And give me a break, tell me what these things are.
This guy is?
Inductor
An inductor.
This guy is a?
Resistor
Resistor.
Now that's the one that's like damping.
This resistor here is like damping.
Like the damping term, or maybe 1/b.
I'm not getting its units right because I haven't got any equation here at all.
Resisting is-- there's friction in that resistor.
It burns up heat.
And similarly, the dashpot slows things down.
And then this guy is a--
[INAUDIBLE]
Capacitor, right.
In other words, you can do the mechanical application and the electrical application with exactly the same ideas, just a change of letter, and of course, different units, but same problem.
OK, so that's a comment that you've seen before.
What else do I want to comment on?
Because this example was really so straightforward.
I think what I want to mention, and this is important, is that this is the central starting point for the Laplace transform.
So I can't do Laplace transforms all today by any means, and so Professor Fry will talk about the Laplace transform next week.
But what is the point of the Laplace transform?
The point of the Laplace transform is to get your money's worth out of the simple formula for exponentials.
Having an exponential there turns the whole differential equation problem into an algebra problem.
We just have quadratic equations.
We just have a division by a quadratic.
That's the great thing about the Laplace transform.
It turns the t domain, the time domain, where we have exponentials, into the s domain, the exponent domain, the frequency domain, where we just have quadratics.
And then first order equation's just linear.
And we can even get from second order of the quadratic to linear, because I can factor that guy.
If I factor into s minus s1 times s minus s2, I've two linear pieces.
And that's the first step in the Laplace transform, in the algebra.
So all of the algebra in the Laplace transform is this algebra for e to the st. And then the job of the Laplace transform-- and this is the tricky part.
So let me even take a little board space on that.
So this is like a heads up for next week.
So the Laplace transform.
OK.
So for any forcing function, f of t. That's the thing that we're going to take the Laplace transform of and the response and its response, y of t. OK.
So here's the idea of transforms in general.
I choose some terrific functions like exponentials.
So I want to convert my problem to exponentials.
e to the st for all s, s between let's say 0 and infinity.
So what do I do?
I take my function, and I figure out how much of every exponential is in it.
That's the Laplace transform.
I take my function f of t, and I go to what you'll see next week, its Laplace transform.
Let me call it capital F of s, or it's sometimes written the Laplace transform of F of s. Something like that.
I'm not going to do that.
I'm not going there.
So F of s is like the amount of a particular exponential in my function.
If my function is just the sum of two exponentials, then the Laplace transform just is a big bump on one and a big bump on the other.
But most functions, like some forcing function, has some e to the st is for all for a whole range of s. So I figure out how much of this is.
OK now second step.
Using linearity, I can solve the problem for when the right hand side is just that.
Then solve for right hand side, F of s, e to the st.
Solve for each frequency, each s separately.
And what does that mean?
That means just dividing by this.
So this was the easy step.
So this is step one, is take the Laplace transform.
The Laplace transform tells you how much of each exponential is in it.
Now step two is a cinch.
Step two is a cinch.
I just multiply by the transfer function.
I divide by this, bs plus k. And now, what's step three.
Step three, I have the solution for each separate exponential, but I've got a whole lots of exponentials, so I have to do an inverse Laplace transform, add up, figure out what function has this Laplace transform.
And that's often the place where the algebra gets harder.
In principle, we can do it for any F of t. We can take its Laplace transform, we can solve for each frequency in the transform, we can assemble them all together.
That's the inverse Laplace transform, so this is the inverse Laplace to get the solution y of t. So this is really the Laplace transform of the solution, and we have to get back to the solution itself.
Can I just let you think about those ideas?
I'm not up to describing the algebra here.
The point is the Laplace transform takes this into separate exponentials.
Each of those right hand sides is simple algebra, divide by that.
And then you have the job of okay, what function has this Laplace transform, to go backwards.
And that's usually the hard part.
So people make tables of Laplace transforms.
Everybody remembers the Laplace transforms of a few functions.
You could say that those few functions are the golden functions of differential equations.
The golden functions of differential equations are the ones where you know their Laplace transform and you can go back and forth easily.
And what are those golden functions?
Well, you might guess exponentials, simple polynomials, 1 t, t squared.
You can do Laplace transform for those.
What else do you think would be nice?
Cosine and sine.
And you could multiply those together.
We could deal with t times cosine t. So a little bit different.
But having said that, the reality is I've given you the whole list of nice functions.
Those functions show up in every simple method for solving equations.
There's a method called undetermined coefficients and what does it amount to?
I'm sorry, I don't have time to say it this morning, but it can come later.
Undetermined.
It just means if the right hand side is one of these nice guys-- shall I write down again the golden functions?
e to the st is like the platinum function.
And then some golden functions are like t and t squared and so on.
Of course, when we get that, we're close to cosine omega t, and sine omega t. All these guys, their Laplace transforms are nice.
We can deal with them completely.
Or multiply any of those together.
And when the right hand side is one of these golden functions, you can write down the answer.
We've focused on this one because it's the platinum one.
And we did these two too, because they come from s equal i omega.
And then these guys are a little bit of juice.
But that's it.
I'm sorry the list isn't longer.
It'd be nice to have--and of course, people for centuries have worked with the next hardest functions.
You know, the silver functions.
Famous functions have names like Bessel's function or Legendre's function.
Others where you can get pretty far.
Those are the best.
Then you have the famous ones where you get pretty far, in the web has the Laplace transforms, and then you get the general function, F of t. Oh, could you get anywhere with delta of t?
Oh, yes.
Does it belong on the list of golden functions?
Yes, it does.
I almost forgot it, and it's like-- I'll call it-- Yeah, delta of t. Yeah, that's a beauty.
That's a beauty.
The Laplace transform of delta of t happens to come out 1 over s. You can't ask for more than that.
Or maybe it's one.
Yeah, it's probably one.
Yeah, the Laplace transform of that is one.
Yeah.
It's got all exponentials in sort of, you could say, equal amounts.
OK.
So that's some thoughts about that about Laplace transforms, just sort of the big picture that takes the differential equation, turns it into an algebra problem, and then at the end, you have to get back, and that's the part that's not always doable.
OK.
So what's left for today is this guy.
Now this one now.
Have I got to that point?
So this will be the final ideas in this course, this four unit core elective.
What is the impulse response when there's damping?
What's the impulse response when there's damping?
OK. OK.
So that means that I would really like to solve m y double prime, b y prime, k Y equals an impulse, delta of t. Because if I can solve this, then I can solve everything.
And I can solve this.
It turns out to be easy.
Now why is it easy?
You might think, my gosh, we've got second order equations here, we've got a delta function there, where do we go?
And so my advice is go this way.
The solution to that is the same.
This with y of 0 equals 0 and y prime of 0 equal 0.
So you have a spring, so again we have a spring with a mass.
And that spring is in a damper.
So can I just, without knowing what I'm doing, draw a damper around it.
So the idea is I'm striking that mass at time 0.
Striking that mass at time 0.
What happens?
What happens immediately?
And then I don't touch it again.
I strike it and that's it.
I've set off, and what have I-- so my point is this has the same solution as m y double prime plus b y prime.
And Ky equals 0.
Nothing happens beyond time 0.
But what are y of 0-- Well, let me give this a letter g, just to emphasize it's special and deserves its own name.
So now, what is the starting position and the starting velocity for this picture?
So I'm saying that the y here is the same as the g there.
And really if you see it physically then you see it best.
So again, at the instant t equals 0, I'm hitting that mass with a unit impulse.
So what is the position of that mass, instantly after I've hit it?
0
Still 0.
Good for you.
Good for you.
And what is the velocity of that mass instantly after I've hit it?
1
1 is essentially the right answer.
1 is the right answer.
But the way I've set it up is, there is an m there.
If I put an m there, which would be nicer, then the answer would be 1.
But the way I've written it here, I have an m there, and I haven't fixed the units.
So that turns out to give me a 1/m there.
I could explain why it's a 1/m, but let me just for the moment say it's my fault.
It's because I didn't get any units right that we have 1/m.
No big deal.
But now, what's good here?
What's good?
This was a problem with a mysterious delta function.
This is a problem with 0.
And the only price we're paying is the impulse gave the mass a little velocity.
And you can imagine that the velocity gives it is 1/m because the strike didn't tell us about that.
So what I'm saying is we can solve that equation for g. We can find g, and in fact, we have.
So this really brings the lecture full circle.
What do I have here?
I have a null solution.
So this g here is a null solution.
So what form does it have?
What can you tell me about g of t?
And it's the same as y of t. So y is the same as g, and what's the form for it?
Yes?
Tell me?
I'm taking it back to the very beginning of the lecture where I solved it with a 0
[INAUDIBLE]
C1, thanks
[INAUDIBLE]
e to the?
C1, e to the s1, t, the two s's, s1 and s2, the special two s's, the special roots of the key equation.
That equation gives us s1 and s2, by this formula, or in a homework or an exam problem, we hope that it would come out easily.
So this is that part of it.
What's the rest of it?
C2 e to the s2 t. The impulse response is the particular null solution that starts with a shot, starts with an impulse, starts with a strike.
OK.
So I just have to find c1 and c2 here.
All right?
We've come back to the basic problem in differential equations.
We've got the solution.
We've got two constants.
We've got two equations.
We just plug that function into that equation.
It gives us one fact about c1, c2.
This gives us the second fact.
We solve them.
Why don't I just write down the answer?
It turns out to be e to the s1 t, and the other guy will come in with an opposite sign e to the s2t over s1 minus s2.
I think that gives us the c's.
Oh, there'd be an m. There'd be an m times this, because of my messing with units.
So can we just check?
At t equals 0, what do I get out of this?
0
0.
What I'm supposed to.
What's the derivative at t equals 0?
The derivative at t equals 0, this derivative is going to bring down an s1.
The derivative here will bring down an s2.
At t equals 0 the exponentials will all be one so I'll just have the s1 minus the s2.
It'll cancel that and it'll be 1 over m. So this is the neat formula for the impulse response.
That's the neat formula for the impulse response.
And then why-- can I use this little corner of the board?
Why do I want that impulse response?
What can I use it for?
It gives me the answer, not just for the impulse but for everything.
The particular solution is for any forces, force, I multiply by whatever the-- let me write the formula and I'll show you what it says.
Yes, there is the formula.
I'm sorry it's squeezed.
But really, the goal here was simply to get a handle on what is the response to any f. And again, I look at that this way.
F of s is the input at time s. G is the growth factor over the remaining time up until time t. So Y at time t, I take all the inputs up to time t, And each input gets multiplied by its growth factor.
It was e to the a, t minus s in the first order equation.
Now we've got two exponentials.
But that's the solution of the general problem.
So we have now in one lecture completed a solution to the second order constant coefficient differential equation.
Right.
Yeah.
By finding the impulse response.
Yes?
[INAUDIBLE] would we still be able to [INAUDIBLE] if s1 is equal to s2?
Ah, if s1 equals s2.
That's the case where formulas need a patch.
They need a patch.
If s1 equals s2, what do you think happens?
If s1 equal s2, everybody sees, I have 0/0.
And so this is like a technical question that I wasn't going to ask myself.
You asked it.
You're responsible.
What do we do for 0/0?
What did you learn in calculus?
Who's the crazy guy who figured out how to deal with 0/0?
In a way, calculus is all about 0/0, right?
Delta y over delta x, they're both headed to 0.
And suppose you have-- let me take the most famous example of 0/0.
It's like sine x over x, as x goes to 0.
So x going to 0.
Sine x goes to 0, x goes to 0.
What's the answer, by the way?
What happens to that ratio as x goes to 0?
This is maybe the most famous example.
Sine x over x, when x gets very small, is close to?
1
1, thanks.
And now just help me out with the name of the guy.
It's a crazy spelling name, and do you remember?
L'Hopital.
L'Hopital.
Everybody despises him.
Probably hi friends despised him.
But anyway, L'Hopital says in a situation like that, when you're going to 0/0, you're allowed to do something a little strange.
You're allowed to take the derivative of the top, so it has the same limit.
Instead of looking at this 0/0, you can take the derivative of the top, cos x, divided by the derivative of the bottom, 1.
And now you can let x go to 0, and you get the right answer.
So this gave a 0/0.
Unclear.
Fuzzy.
This gives-- what's the right answer then?
Just tell me again.
When x goes to 0 this becomes?
1
1.
Right.
So that's L'Hopital.
So that's what I would have to do here.
I would take the derivative of this, and the derivative of this-- the only sort of tricky part is it's the s derivative.
It's s1 going to s2.
Let me just tell you the result.
Since you asked.
A factor t comes out.
It's t e to the s1, or s1 is the same as s2, divided by the m. It actually looks simpler.
There's only one.
This is in the case s1 equal s2.
So s1 is the same as s2.
I just chose s1.
Yeah.
L'Hopital gives a simpler answer.
And it's got this suspicious and recognizable factor t. That came from L'Hopital.
OK, I won't do that stiff.
So let me say again, we've now done the second order constant coefficient equation I do just have 10 minutes of something to make it better.
And that is that the famous quadratic formula for s, for s1 and s2 is not beautiful.
It's correct.
It's correct.
But it's a little bit of a mess.
You've got three things, b and m and k playing around.
And we saw in this picture, we saw all the differences.
I guess in this example I kept m1 and I kept k1, and I increased b. I could do other examples where I increase the k, I make it stiffer and stiffer.
All these examples.
And engineers have worked for 100 years to see, out of this formula what are the important parameters, what are the important numbers, and hopefully, where possible dimensionless.
So I just want to- the final minutes would be-- back to high school-- playing with this formula, to get better numbers in there.
May I do that?
I just think, because then you'll see-- I learned this, actually, so this is like something math professors have no reason to do.
Look at that.
That's the formula.
End of story.
But the Web, 1803 website, has a class in which Professor Miller from the math department was teaching this subject, doing these, exactly these, but also Professor Vandiver from Engineering was putting in his suggestion of what are the good parameters?
What are the parameters that engineers look at?
So that would be my final comment, and I won't do it as well as Professor Vandiver did.
But can I just take that-- let me erase these two special examples, and look at this question.
Again, the book will do it.
So one nice-- b/2m is a pretty natural parameter to use.
Let me introduce that as one of them.
I'm going to, by taking ratios like b/2m, let me call that p. Let me call b/2m.
So that's a ratio of damping to mass.
And then this has got to come out simpler.
What does that come out?
If you'll allow me, I'm going to open the book so I don't write the wrong thing here.
This is in the book, on page 99.
The title is Better Formulas for s1 and s2.
Better Formulas for s1 and s2.
And here's my first better formula.
You can see that I get a minus b plus or minus the square root of something.
And that something will turn out to be p squared.
And then it'll be a minus something.
And that something will turn out to be the natural frequency squared.
Isn't that nice?
So what's the natural frequency?
Somehow, the natural frequency's coming in from this and this?
And just remind me what that second parameter is, omega n squared, the natural frequency of oscillation with no damping.
Tell me again what that is, because that was the fundamental ratio from last time.
It's central to all of engineering.
It's?
k/m
k/m.
Thanks, k/m.
So I believe-- and maybe Professor Fry could make this assignment a homework question, which is just algebra question.
Everybody sees that I have a minus p here.
And with a little care you get p squared, which is quite nice.
Which is quite nice.
And so we see that the decision between overdamping-- remember now?
Overdamping is when you damped so much that this became negative and you got an imaginary number in there.
Underdamping, it's still positive.
Overdamping, it's negative.
And so really that separation between overdamping and underdamping is the ratio of p to omega n. P to omega n is the damping ratio.
I think.
There may be a factor too.
Let me try to-- everybody sees that's the battle between these, if you accept that formula, and if it's in the book it's got to be right.
OK. And so the damping ratio is just that.
That's the damping ratio.
Now that's called zeta.
The Greek letter zeta.
I'm not Greek and not good writing zeta.
So I have unilaterally decided to use a capital zeta, which is a Z. Zeta is the Greek letter for Z. I could try to write it but you wouldn't be impressed.
So it's that damping ratio.
So now what does this mean?
Z smaller than 1, z equal to 1, c greater than 1?
Tell me what those-- obviously, when it's smaller than 1, p is smaller than omega n. Yeah, so what's going on here?
Which one is underdamping, which one is critical damping, which one is overdamping?
Because there's no difficult stuff here.
We're coasting in the last minutes here by just choosing words and notation that have turned out over a century to be more revealing than b squared minus 4mk, and this Z.
So z less than 1 will be what?
Z less than 1 will be p smaller than omega n. p smaller than this.
What's the story on that case then?
[INAUDIBLE]
That is underdamping, I guess.
p small has to go.
p is like b, the damping.
And small damping is underdamping.
So this is underdamp.
Underdamp.
And that's the case, in which we're going to have some imaginary stuff.
We're going to have some oscillation with the decay coming from there.
Now, what about z equal to 1?
z equal to 1 means p equals omega m, so that equals that so it's a big 0 in there.
What case is that?
[INAUDIBLE]
Critical damping.
It's this case in that picture.
It's that case with a double 0, equal s's.
Formulas that have to take account of that, this is critical.
And then finally, z greater than 1 is what?
Overdamped.
Overdamping.
Z bigger than 1 means p is big.
P big means b is big, damping is big, it's overdamped.
Overdamped.
So we've got it down to one parameter, the damping ratio, to tell us these things.
Rather than previously we had to say is b squared smaller than 4mk?
Is it equal to 4mk?
Is it bigger than 4mk?
Now we've got those words down to a single number z.
And let me just write next to us here that the z turns out to be the ratio of the damping to-- I think it's right-- it's the damping divided by the square root of 4mk, I think.
Can I just put a question mark there.
You couldn't mess around with the letters p and z.
But to get some variation from some other, but the point is, you see how much cleaner that is compared to this?
You're directly comparing that number to that number.
And that ratio is that number.
Yeah.
So all the formulas come out nicely.
Yeah, the formulas come out nicely.
And I guess what we see here-- final comment-- what we see here is what is the frequency of underdamped oscillations.
So I want to be in this underdamping case where there is oscillation.
There is an imaginary number coming out of that.
But there's also a real number.
Is the frequency of underdamped oscillation the same as omega m, the natural frequency?
No.
The frequency of that number-- so final comment, let me put it just here.
I would like this whole thing to be i, to give me oscillation, times omega d, the damped frequency.
And let me just say what that is.
So omega d, the damped frequency, squared, is this omega natural frequency squared minus the p squared.
If I had longer and we didn't have blackboards already full of formulas, I could-- it's the thing whose square root we're taking here.
So this is minus p plus or minus i, omega damped.
Omega damped is this square root.
There, we succeeded to fit in the better ratios, the good quantities to look at.
So again, the good quantities to look at are p, z, the damping ratio, omega d the damped frequency.
I think in a first lecture you could say, well, we already had correct formulas, we should just leave it there, and that's absolutely true, but anyway, this is what-- these are the letters people have introduced to make the formulas easier to understand in an engineering problem.
OK.
I'm all done except for questions.
Yes?
Don't ask me about resonance again.
Yes, OK.
Yes?
In the case of where we have the delta function, what is the velocity [INAUDIBLE]?
What is the what?
Why is the velocity equal to [INAUDIBLE]?
A-ha.
Okay.
You're right on the ball.
The question is where did this come from.
Where did that come from?
Can I tell you?
So if I integrate everything here, if I take the integral of everything, between 0, a little bit left of 0-- can I call that 0 minus?
Just a little bit left of 0.
This is crazy.
No math professor or whatever should ever do this.
To a little bit right of 0, just a real short time.
So what am I going to call a little bit right of 0?
0 plus.
OK now what is that integral?
Between a little left of 0 and a little right of 0, you know what the integral of the delta function is.
It is?
1
1.
Good.
Now, what are these ridiculous things?
Well, y is not changing in this tiny, tiny time.
So this is something, it's not getting big.
I'm integrating it over this tiny little time.
It's nothing.
Forget it.
Similarly here, y prime, the velocity is not climbing to infinity.
There's no-- and I'm just integrating over this infinitesimal little time.
Nothing here.
So this term has to be responsible for the 1.
And now you can tell me the integral of m y double prime.
What's the integral of m y prime?
m y prime
m y prime.
So m y prime plus, at 0 plus, minus m y prime at 0 minus.
But at 0 minus, it's 0.
You see what's happening?
And on the right side I'm getting a 1.
This is-- no person who had any skill with a blackboard would allow this to happen.
But that happened.
OK.
So these lower order terms are typical of math.
Lower order terms in the limit, forget them.
This is the top term, and it has to have something there, because it has to balance the 1.
And what it has is the jump in y prime.
So this is the instant jump in y prime in velocity, is times m gives 1.
So the instant jump jumped us from 0 to 1 over m. That's where I came from.
Well, that was a good question, and a kind of crazy answer but there it is.
OK, so we've got a mention of the Laplace transform as the algebra tool that works when you're staying with exponentials and nice functions.
And you'll see more of that.
So it's a frequently used tool to turn problems into algebra.
My name's Lorna Gibson.
I'm the professor for 3.054, it's a course on cellular solids.
And I've been working on cellular solids since I was a graduate student, since I did my Ph.D. And cellular solids are materials that are made up of an interconnected network of struts or plates.
And there's examples like engineering honeycombs and foams, and there's lots of examples in nature.
Things like wood and cork and there's a type of porous bone.
And there's lots of examples in medicine too.
Tissue engineering scaffolds, for example.
So my background is in civil engineering, and in civil engineering we study structures.
And typically people think of large structures like bridges or buildings.
But in fact when we analyze the cellular solids, we use the same kind of mechanics.
It's just the scale is very much smaller.
So we're looking at structures where the scale might be hundreds of microns or millimeters, things like that, but the same sort of mechanical principles apply to that.
OK, so I grew up in Niagara Falls, in Ontario.
And people always think of Niagara Falls as being the waterfall and all the tourist stuff, there's a casino there now.
But in fact, there's loads and loads of big civil engineering works in Niagara Falls, mostly associated with the hydroelectric power station.
So when they make hydroelectric power in Niagara Falls, the power station is actually about a mile downstream from the Falls.
And what they do is they have a big hydraulic gate that goes into the river and it diverts water from the river above the Falls into a whole series of canals and tunnels and there's a big reservoir where they store water.
And then the water from this reservoir goes into the penstocks, the tubes that go down to the turbines and then make the electricity.
Niagara Falls is not a big town, but if you drive around Niagara Falls, you see these canals, you see the reservoir, you see the big power station.
And so there's these really huge, impressive civil engineering works.
And my father worked for an engineering company in Niagara Falls and they specialized in the design of hydroelectric power stations, and I think that's how I got interested in engineering.
So I've been interested in bird watching for some time.
Mostly just because birds are beautiful and there's all sorts of interesting behaviors you can see with birds.
But since I started doing research on cellular solids and, in particular, teaching this course, I realize there's lots of examples of things about birds that have to do with cellular materials.
So for instance, some people had once told me that woodpeckers avoid head injury and brain injury by having a special cellular material in between their brain and their skull.
And that this acted kind of like a foam in a bicycle helmet.
That it would absorb the energy of the impact.
And I thought oh, well, I like bird watching and I study cellular materials, I should find out about this.
So I started looking into it and people had looked at the anatomy of the woodpecker skull and brain.
And, in fact, there is no special cellular material.
But by that point, I was kind of hooked.
And I actually did a project at one point looking at why it was that woodpeckers don't get brain injury.
And it's largely a scaling law.
It has to do with the fact that their brains are very small.
Another aspect of birds that has to do with cellular solids is how birds make themselves very light.
And here we have an owl skull.
This owl, unfortunately, had an accident with a car.
But somebody picked up its body and took it to Mass Audubon, and I got this from somebody at the Massachusetts Audubon Society.
And if you look at the skull-- I don't know if you can do a close-up here-- if you look at the skull, you can see there's a dense layer of bone on the outside and there's another dense layer bone on the inside, and there's a sort of foamy layer bone in between.
And that's called a sandwich structure.
And this foamy type of bone is called trabecular bone.
And that's one of the things that I study.
And it turns out that particular structure gives you a very stiff, strong, lightweight structure.
So you can see an example of how cellular materials are used in engineering but here sort of manifested in the owl's skull in making the skull very light.
Another part of the course is that I also have a project in the course.
And students do the projects in pairs.
Partly, because I just think it's nice for them to have somebody to work with.
And they've done a huge range of projects.
I let them do whatever project they want.
It has to have something to do with cellular solids.
And some of them have done, sort of, analytical things where they've done numerical finite element analysis of some cellular structure.
Some of them done very experimental things.
And some have been a lot of fun.
So, for instance, almost every year there are students who want to work on food foams.
And one year, for example, there was a group made bread.
And they looked at sort of the processing of the bread.
And they looked at how if you used more yeast, or you let the yeast rise, or the bread rise for longer, how that affected the cellular structures.
So they sort of changed the chemistry of how they made the bread.
And then they looked at the micro structure of the bread that they got.
I think they even did some mechanical tests of the bread.
So that was kind of fun.
And I think, probably, the most interesting project any student has done was on elephants skulls.
So you could imagine an elephant skull is huge, and it's bony.
So it would be very, very heavy, if it didn't have some pores in it.
And elephant skulls, it turns out, have some very large pores in them.
I think partly to reduce the weight of the skull, and the head, and the bone.
The neck has to, kind of, carry it all.
So these two students came to me.
And they said they had heard somewhere or they'd read somewhere that these pores in the elephant skull had an effect on how the elephants perceived sound and the acoustic transmission of sound waves through the skull.
And they wanted to do a project on elephant skulls.
So I was kind of intrigued by this.
I love all natural history kinds of things.
And I've worked before with people at Harvard's Museum of Comparative Zoology, where they have bones.
They have stuffed bodies.
They have all kinds of animals over there.
And I called up a colleague over there, and it turned out they had elephant skulls over there.
So I went with one of the students.
And it was an attic of the building, this kind of dingy place.
And there was this huge room.
And it was full of elephant bones.
And they had several skulls, which are like the size of this table.
They're huge.
They're this big.
And some of the skulls were cracked.
And you could see these big pores in the skulls.
And then the students found out that University of Texas at Austin has CT scans, Computed Tomography scans of all sorts of bones.
And sure enough, they had elephant skulls scanned.
And so they got a three-dimensional representation of the elephant skull through this University of Texas at Austin program.
And then they used that as input to a 3D printing set up.
So they 3D printed a sort of mimic of the elephant skull with some ceramic powder.
And they made a skull was about this big.
And then they wanted to look at the acoustic properties of it.
So what they did was they suspended the skull from a string.
And they had a speaker, and the speaker had a sound.
And they put an accelerometer on the skull.
And they measured the vibration of the skull.
And then to compare it with the skull that didn't have these pores.
And they got the CT image from Austin.
And they 3D printed this dolphin skull.
And they did the same thing.
They suspended the skull from as a thread.
And they measured the vibration.
And they could show-- and I've forgotten the details of their results-- but they could show, basically, that the two skulls had a different frequency response to the vibrations.
And they thought maybe part of it was because the pores.
So these two sections here are two sections of their 3D printed elephant skull.
I don't have the entire thing.
But you can see this is the orbit, where the eyes would have gone in here.
And if I turn it over and you look inside, you can see these pores in here.
And also if you look at this section lower down on the skull, you can see this whole porous structure here, as well.
And if we flip it over there's a little bit more over there.
So that was probably the most intriguing project that the students did as part of this course.
That was quite something.
So typically in the course we start off talking about the structure of these cellular materials.
We do some modeling of honeycomb and foam type materials, and that takes not quite half the term, but most of the first part of the term anyway.
And I give them more problem sets at the beginning of the term, and they don't really start on the projects at the beginning.
So I give them some background information to get them going.
And I assign the project at the beginning of the course, and I think there's three kind of deadlines.
One is they have to give me a proposal.
I think that's typically about a month into the course, and that can be fairly brief, but I want to at least know they've got a team.
They've got an idea.
They've got some idea of how they're going to carry out their project.
Then in about another month they have to send me a sort of update on the project, and by that point, I would have expected them, if they're doing a literature review, to start reading some papers and be able to tell me something about the background to their project.
If they're doing experiments, I would have expected them to at least gone into the lab and maybe made some materials, or bought some materials, and done some preliminary tests.
And I give them feedback at that point.
And at the end of the term, they hand in the final project.
And I see them twice a week in class, and I don't really have a formal recitation, but what I do do is every week that a problem set is due I have office hours.
And in fact, I don't actually have it in my office.
I book a room, and we do a little tutorial.
So there's times throughout the term they can see me and come and ask me questions about the project as well.
So the question is what kind of feedback do I give the students and how do they interact with me on the projects, and typically there's about 20 students that take the course.
So if they do it in pairs, there's roughly 10 projects.
Some of the projects students just do literature searches, and I don't really get that involved with those.
They're perfectly capable of going to the library and doing a literature search.
And some of the students who are taking it are graduate students, and they often do finite element numerical calculations.
And again, I give them some advice about how to set it up, but often they have experience doing this, and it's sort of applying what they already know to something new, but they kind of know what to do.
The way I get the most involved is if students do experimental projects.
So for instance, in this elephant skull project, I had this connection at the Museum of Comparatives Zoology, and I took the student up there.
They had this idea about 3D printing it, and we have a 3D printer in the department.
And one of the technical staff, Mike Tarkanian, is very, very good with the students and very helpful in getting them set up on the 3D printer, so he helps with that.
And I give them sort of general advice.
When they said now we want to measure some sort of acoustical response, I suggested maybe you could suspend it from a wire, or thread, or something and then put an accelerometer and measure the vibrations.
So I try to give them some general advice like that, but they then carry the experiment out themselves.
They have to figure out how to actually put it into practice and how to do it.
And I think they get a big kick out of that.
MIT students enjoy that kind of thing so that's kind of fun, and obviously I was very delighted too with this elephant skull project.
So there's different things I try to help with-- mostly giving general advice about how they can do their experimental projects.
So what are some of the challenges that students encounter in their projects.
So I think one thing is students sometimes are little over ambitious in what's possible to accomplish, because typically we have to cover some material before they can even start the project.
So typically they don't start the project until a month or six weeks into the term, and the term is only three months long more or less.
And so they really have a fairly limited amount of time-- maybe six weeks or eight weeks, something like that-- to actually do the project.
So if they want to make materials, if they want to do some sort of processing to make a foam, for example, they don't really have a lot of time, because often it takes some trial and error to be able to do that.
So the projects have to be fairly focused so that they can actually get something interesting out of it in a fairly short amount of time.
And sometimes students might want to do something where they would have to order materials, and it may take a few weeks for the materials to come in.
So again, I try to discourage them from doing projects where there's going to be a long lead time on getting some critical thing that they need for the project.
So there are some limitations, partly because of just the time we have for them to actually do it.
And they're not just taking my course.
They're taking other courses, so there's sort of a limited amount of time they can spend on it, too.
So really the way I set up the lectures is I write out notes for myself of what I want to cover, and the notes are pretty detailed.
And in the past I always just had the notes for me.
And even though they were reasonably neat and I could read them, I didn't hand them out to the students.
But because I've turned my fall course into an MITx course and I want to make the lecture notes available for that, when I was doing that course for MITx I made really nice notes.
And a friend of mine who lectures, I was asking her how she did it, and she said she actually goes and measures the chalkboard, and she measures the aspect ratio, how tall to wide the chalkboard is.
And then she sets up her notes so that they're the same aspect ratio, and she plans out exactly where she's going to put everything on the board on her notes.
And so I started doing that.
And when I'm doing the lecture I put it all on the board, and I find a lot of students-- even though I hand the notes out now-- they like to write their own notes.
And I think it helps them pay attention in class and helps them kind of focus on the material.
So another thing I do in the lectures, when I first started lecturing and for a long time I just focused on the engineering, you know, on the equations, this is the derivation, this is an example, this is how you use this.
And it was all just about kind of the engineering of whatever I was working on.
And I found over the last few years, actually in both courses, in the fall one on mechanical behavior and in the spring course on cellular solids, I look for more interesting examples and stories, kind of stories about the people who discovered some of the principles that we talk about.
I tell them stories about engineering situations that came up and there was some interesting thing happened.
And the students love it.
I mean, they really like having the kind of hard core mechanics broken up with some sort of stories.
So I do that a lot more now than I used to do.
So probably most lecturers have some kind of interesting example or historical thing that I talk about.
So for instance, when I teach the fall course, the mechanical behavior materials, one of the first things we talk about is stress.
So stress is a force per unit area.
If I take this piece a wood and I pull on it like this I'm pulling on it with a force that goes out like this.
Stress is just that force divided by that area.
And the unit of stress in the SI system is called the Pascal.
And it's named after Blaise Pascal who's a French mathematician.
And a couple of years ago I was in France for a conference and I was in a little town called Clermont-Ferrand, which is in the middle of France.
It's a pretty little town.
And I'm just walking around one day kind of seeing the square and the cathedral and all this stuff, and I see there is sign, there's like Pascal something or another.
And this was apparently the site of Blaise Pascal's house.
And right next door to it is Cafe Pascal.
So of course, I have to take a photograph of Cafe Pascal.
And in this other course, when we get to the bit about stress and I tell them about the Pascal I show them the picture of the Cafe Pascal.
And the other amusing thing that they kind of get a kick out of because of Boston, you know how in Boston there's the Freedom Trail and there's a red line goes around all these historical sites all this colonial and revolutionary stuff around Boston.
It's kind of cool, all the tourists to it.
Well in Clermont-Ferrand there's a Pascal Trail.
And there's little metal medallions of the portrait of Blaise Pascal put in the sidewalk and you can walk around Clermont-Ferrand doing the Pascal Trail.
So I kind of keep an eye out for stuff like that and I put that into the course now, and I never used to do that kind of stuff.
I have a picture from the Library of Congress, which I went to just for fun a while ago.
And one of the main buildings is this old, beautiful historical building for the Library of Congress.
And they have a marble staircase that goes up the middle.
And the staircase has all these little cherubs.
And there's an agriculture cherub and he's holding a sheaf of wheat.
And there's a wine cherub and he's holding a little thing of grapes.
Well, it turns out there's a mechanics cherub and he's holding a gear.
So at the end of the first lecture I show them the mechanic's cherub with the gear.
So there's these cute little things.
So I just keep an eye out for these cute little things.
And last fall, this past fall, one of the students came up at the end of the first lecture and he said, I really like art.
And he says, is there going to be more art in the class?
And I'm like, not usually.
This has kind of exhausted my art in mechanics.
And I said, but I'll keep an eye out.
And through the term there actually were different things that I saw that had to do with mechanics and art I showed the class.
So one of them was at the Peabody Essex Museum this fall there's been and display of the mobile sculptures of Alexander Calder.
And they also made these very large, I think they're called stabile sculptures, as well.
The big sail at MIT is one of his sculptures.
And these mobiles are actually a really nice example of free body diagrams in mechanics and balancing the forces.
So anyway, I got a picture of one of his mobiles and I went up to the exhibit.
And when I was at the exhibit I found that he did a degree in mechanical engineering.
He actually was a mechanical engineer.
And so the students were very, very tickled by the sculpture, the fact that he studied engineering.
So anyway, I look out for stuff like that.
So there's a lot of images in the class, partly because we're studying these materials and you can see just with the ones sitting in front of me, they have this porous, cellular structure.
So I show lots of images of materials.
We also look at how the images deform under load.
And I think, perhaps, that's something that might be a little unexpected.
So for instance, we have a stage in the electron microscope where we can actually deform things in the microscope.
And the stage is set up that it has a load cell on it.
And we could also measure how much it deforms, the materials.
So we can actually watch the materials as they're deforming.
So even though the cells may be 100 micron size, you can watch how they deform and how they fail, and you're going to learn a lot about the mechanics from these sorts of observations.
So we have both video of these deformations and also still photography at different time points that we show.
Another interesting little video clip that I show in the class is we look at the interactions between biological cells and tissue engineering scaffolds.
So for example, people who've looked at trying to heal, say, burns in skin where there's a large area of skin missing, one of the ways that you can do that is by using a collagen-based tissue engineering scaffold.
And when you have a burn in your skin and it just heals the normal way and you get all the scar tissue forming, that scar tissue is thought to form in conjunction with a process of what's called wound contraction.
So cells will actually migrate into the wound bed and actually mechanically pull the edges of the wound together, and that partly closes the wound and then scar forms as well.
And those two processes are thought to be related to each other.
So I've done a collaboration with Professor Ioannis Yannas here at MIT who developed one of these scaffolds for burn patients.
And he and I have been interested in this wound contraction problem.
And it turns out if you just put fibroblast skin cells into a dish of culture medium with one of these tissue engineering scaffolds, they will contract the scaffold.
And you can use an optical microscope to actually watch the cells do this.
So you can focus on an individual cell and you can see the cell elongating.
You can see the scaffold itself contracting.
And you can see this whole process.
So this is one of the little video clips that we show in class.
And the students always find that fascinating.
So I think one of the special things about this course and about my kind of research on cellular materials is that I spend a fair amount of time talking about materials in nature.
So I talk about wood, for instance, and what it is about the cellular structure that gives rise to the density dependence of wood properties and the anisotropy in wood properties.
I talk about trabecular bone, and I talk about the structure and properties of the bone, but also we do a little bit of modeling on how you might look at bone loss and osteoporosis.
So if you lose a certain fraction of the bone density, what residual strength would you expect the bone to have.
We have a project on bamboo right now.
We talk about the structure of bamboo.
Bamboo is actually a grass.
And this is a Chinese species of bamboo called moso bamboo, and you can see how big this one is.
They even get bigger, maybe six or eight inches across.
And what we're interested in doing is making something called structural bamboo products.
And this is an example of a bamboo oriented strand board.
So the same way people take wood, and they chop wood up into strands and make oriented strand boards for housing construction, you could, in principal, do the same thing with bamboo.
And we have a project that's in collaboration with some colleagues at the University of British Columbia.
They're the ones who are actually making the bamboo oriented strand board.
And with some architects in England, in Cambridge, England, we're looking at things like how you might modify wood building codes that talk about wood structural products, how you would modify that for bamboo structural products.
And what we're doing here at MIT is we're looking at the structure of the bamboo and doing some modeling of the mechanical properties of the bamboo itself.
So that's one example.
Here's another example.
This is a bamboo laminate.
So this is a little bit like a glue laminated wood member, but in fact, this one is made out of bamboo instead of out of wood.
So it's the same kind of idea.
Let's see.
We've also had a project in the past on cork.
So this is a cork from a wine bottle, and we've looked at that.
Cork has an unusual mechanical property.
You know, if you take a rubber band and you pull on it, if you can make it longer this way, it gets narrower that way.
Well, if you load cork in one direction, if you, say, pull on it or compress it, it doesn't get any wider or narrower in the other direction.
It just stays the same kind of size.
And you can show that that's related to the structure of the cells.
The cells are like little bellows, or like a little concertina.
So you can imagine if you have a little concertina, and you push on it this way, it doesn't really get any bigger that way or smaller that way.
It just stays the same dimension.
And the cork cells look a little bit like that, and that's why they do that.
So we have all these natural materials.
We talk a little about the hierarchical structure in plants.
The cell walls are fiber composites, and then there's a cellular structure.
And plant materials have a sort of hierarchical structure, with several different levels of hierarchy, and we talk about that in the class as well.
And we talk a little bit about why that makes these materials mechanically efficient and how you might look at designing engineering materials based on that.
So there's a little bit of biomimicking in the class as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so what we're going to talk about in this course are materials that have a cellular structure.
So they're all very porous.
And typically they have low volume fractions of solids, like less than 30% solid.
And we're going to talk about different types of cellular solids.
So one type are honeycombs.
And this would be kind of your standard hexagonal honeycomb, something like that.
We're going to talk about foams.
And I'm sure you've all seen polymer foams.
Here's an aluminum foam.
We'll talk about other sorts of foams as well.
We're going to talk about some medical materials.
And I brought in some bone samples, some trabecular bone samples here.
I brought in a little tissue engineering sample here.
And then, we're going to talk about materials in nature that have a cellular structure too.
So we're going to talk a little bit about wood.
And I brought in a piece of balsa wood.
I'll pass these around in a minute so you can play with them.
This is the lightest wood.
And I brought in lignum vitae.
This is the densest wood.
Lignum vitae is so dense that it sinks in water.
And I have a couple of projects right now on natural materials.
So I thought I'd talk a little bit about those too.
We have one on bamboo and making structural products out of bamboo.
So this is like a beam made out of bamboo.
And this is a piece of oriented strand board made out of bamboo.
So the same way you have like wood oriented strand board, you can do the same thing with bamboo.
And we have a project that involves this, so I thought talk a little bit about that later on in the course.
So we'll talk a little bit about the processing, how you make these materials, the structure.
We'll talk a little bit about how we do the modeling.
So we're going to start with the honeycombs, partly because they have a nice, simple unit cell that you can repeat and you can analyze fairly exactly.
And then we're going to talk about modeling the foams.
And then once we've got the modeling background, so we're going to get equations describing the mechanical properties, the stiffness and the strength, once we've got that, then we can apply it to lots of things.
So we can apply it to understanding the trabecular bone, the tissue engineering scaffolds, or how cells interact with scaffolds.
We're going to look at energy absorption in foams because they're good for absorbing energy.
We're going to look at a lightweight sandwich panels as well.
So we're going to look at all these different things.
And I have some slides I'm going to go over today that have more pictures of all of this.
And I'll pass this around to a sec, but let me go through the logistics first.
So this first hand out has a few of the kind of details.
There's two books that I've coauthored with colleagues.
The one on cellular solids, if you wanted to buy a book, is probably the most relevant one to buy.
If you don't want to buy it, you certainly don't have to.
I'm sure the library has multiple copies of it.
And the other more recent one is called Cellular Materials in Nature and Medicine.
And that's a little more specialized.
And again the library has that.
So you don't need to run out and buy that.
You can just get it there, but let me just mention there's that reference and it might be helpful.
OK, so let me talk a little bit about the projects.
What we do in this class is everybody does a project.
And I like for you to do it in pairs, just so that you have somebody to work with.
I think it's nice to have somebody to do it with.
And the project has to be something on cellular materials, but I really leave it pretty open ended what the project is.
It's really up to you to decide what you'd like to do.
And to give you some idea what people have done, people have done all kinds of stuff in the past.
So people have worked on negative Poisson's ratio honeycombs.
I didn't bring any of those in with me, but you can design these honeycombs so that they have this property that when you push it this way-- instead of if you make the strain smaller in this direction, it gets wider in this direction normally, but with negative Poisson's ratio materials, it will contract in that direction.
So I've had people do projects on that.
I've had people work on osteoporosis.
I had a group once who worked on elephant skulls and I brought part of their project with me.
So it turns out elephant skulls have a sort of porous layer to them.
And this is-- they have these large pores in the top of the skull.
I don't have a whole skull, but this is part of it.
And what they did was they had heard that elephant skulls have these pores and that the pores somehow affect sound transmission and how the elephant hears.
And so they wanted to do a project on that.
And that was really all they knew to start with, elephants have pores and we want to do a project on it because it's cool.
So I helped them put together this project.
We went up to Harvard.
We went up to the Museum of Comparative Zoology where they have elephant skulls, which are like about this big around, huge skulls.
And some of them had the outer part of the bone was broken and you could see these big pores.
So they got some nice pictures of elephant skulls.
And then they found that the University of Texas at Austin has computer tomography images of all sorts of bones of different animals.
And sure enough, they had elephant skulls.
So they got the file for the micro CT image, which gives you the sort of 3-D picture of the skull.
And then they use that to 3-D print small versions of the skull.
So that's what this is, they printed smaller versions.
And these were just a couple of slices that I got from them at the end project.
And then what they did with one of the skulls, they suspended it from a wire, and they took speakers, and they had sound that vibrated the skull, and then they put an accelerometer on the skull and they measured the vibration response of the skull.
And they also made, I think, it was a dolphin skull, which did not have these pores, and they kind of compared the vibration response from the sound for these two skulls with two different structures.
So that was a project for this class.
People have worked on tissue engineering scaffolds.
Often there's people who work on food foams.
So one year, people worked on bread.
They did bread processing.
They made bread by having different amounts of yeast, different rise times, different ingredients.
And they made bread.
And I have a little-- I like these historical things.
And I have a little thing here I wanted to show you.
So the people in 3032 last year, last fall, will know that I went to England in November.
And I went to the Royal Society for an editorial board meeting.
But I also went and looked at their archives.
And one of the things they showed me was this article, which is in the sort of an original, sort of archives of the Royal Society from 1600s.
And it's by a guy called John Evelyn.
He's famous for writing a book called Silva about trees and wood.
But he's written this article here.
And I love the title.
It's "The Several Manors of Making Bread in France, Where by General Consent, the Best Bread in the World is Eaten," by Mr. Evelyn.
So here we have in the Official Royal Society bread making science.
And in fact, the article was several pages long.
There was quite a lot on bread and how you make it in France, where the best bread is eaten.
So if you want to do something on food foams, people have done meringue before, various things.
They look at typically changing something about the recipe, or the composition, or the processing, or the baking.
And they look at the structure.
Sometimes they look at mechanical properties too.
So anyway, there's a whole list of things there.
You can think of other things.
If you look through the books, you might get some ideas as well for projects.
So what I'd thought I'd do next is I wanted to just kind of give an overview of what we're going to talk about.
And I've got a bunch of slides.
Let's see I forgot some things, because I do that.
Books, yeah, OK, let me pass some of these things around so you get to play with them too.
So here's some honeycombs.
I don't know if we can pass all these things around because it's a little unwieldy.
So this is an aluminum honeycomb.
I like bringing toys in.
These are little rubber honeycombs.
This is a little ceramic honeycomb.
This is a little paper honeycomb.
And let's see, we have some foams here.
So here's a metal foam and a ceramic foam.
And this is a sort of, not quite a foam, it's made of hollow spheres by centering hollow spheres together.
So you can play with that.
And what else do we have?
We have little lattice things here.
So here's a sort of 3-D lattice material.
So this has sort of a cellular structure, but it's very, very regular.
It's not like a foam.
So that's called a 3-D lattice material or sometimes a 3-D truss.
And what else should we pass around?
We need to do the bones.
Here's the wood.
You can feel how different the densities of the two woods are.
I would like these all at the end because I show these around for different classes.
Here's some bone, you can see of the bone looks like the foam.
And this is a tissue engineering scaffold for generating skin.
All right, so while those are getting passed around, I'll talk a little bit about what we're going to cover in the class with some slides.
OK, let's see, should I dim the lights?
Would that be a good thing, Craig, if I dimmed the lights?
They're kind of preset, you can try maybe two
OK.
Doesn't seem to-- there we go.
How about that?
That OK?
All right, so I like these historical things.
So this is a picture of Robert Hooke's drawing of cork from his book Micrographia.
And he was the first person who used the word cell to describe a biological cell.
And it comes from the Latin cella, which means a small compartment.
So you can think of the cells as small compartments.
That kind of makes sense.
And he kind of very modestly says, "I no sooner to discerned these, which were indeed the first microscopical pours I have ever saw, but me thought I had with the discovery of them perfectly hinted to me the true and intelligible reason for all the phenomena of cork."
So he's saying by looking at the structure of cork, he thinks he understands everything about the properties of cork-- very modest.
But in fact, this is kind of the foundation of material science.
So material science is all about looking at the structure of materials and trying to say something about the properties of the behavior of the materials.
And that sentence kind of sums that up.
So that's why I like that sentence.
So what we're going to do is look at different kinds of cellular materials.
We'll look at engineering ones.
And these we typically refer to as honeycombs or foams.
Honeycombs have two dimensional prismatic cells, while foams have three dimensional polyhedral cells.
And we'll look at applications for the honeycombs and foams in things like lightweight sandwich panels, in energy absorption devices, and things like thermal insulation.
We'll talk a little bit about the thermal properties.
We're also going to talk about cellular materials in medicine, so trabecular bone.
We'll talk about osteoporosis and how loss of bone reduces the strength, and how you might estimate that.
We'll talk about tissue engineering scaffolds, something about their mechanical properties.
You may think the mechanical properties aren't probably the most important thing.
But in fact, the mechanical properties do have some effect on how the cells interact with the scaffold.
So we'll talk a little bit about cell scaffold mechanics.
And then we're going to talk about cellular materials in nature at the end of the course.
So we'll talk a little bit about honeycomb like materials, like wood and cork, and foam like materials, like the trabecular bone.
There's also a type of tissue in plants called parenchyma that looks just like a foam.
And there's some sponges that have some interesting features.
And often, in nature the cellular material appears in combination with some solid material.
And so it's sort of a structural component.
And you can see sandwich structures in nature, leaves and skulls.
You can see materials that have density gradients, palm stems and bamboo are examples of that.
And you can see materials that have cylindrical shells with compliant cores, and things like plant stems and animal quills are like that.
So that's kind of the range of materials that we're going to talk about.
And one of the interesting things about cellular materials is that you can make cellular materials out of almost anything now.
And this is really a huge range of materials and lots of different applications for this.
So one of the fundamental things we're going to do is look at the mechanisms by which the materials deform and how they fail.
And we'll use a structural analysis to obtain the bulk mechanical properties, so things like the stiffness, the moduli, the strength, the fracture, toughness.
We'll look at how you can control the design of the microstructure to get the properties that you might want, and also how you might select for the best material for a given engineering application in engineering design.
So we're going to get those three things.
So let me start by showing you some more examples of engineering cellular solids, and in particular, some micrographs.
So that you can see what it looks like on a small scale too.
So these are the honeycombs.
These are the sorts of things that I'm passing around now.
The aluminum one, paper resin, and the ceramic ones.
The aluminum and the paper resin ones are typically used in the cores of sandwich panels.
And the ceramic ones are used in the catalytic converter in your car.
So what they do is they block off every other cell at one end and then the opposite cells at the other end and exhaust gas from your car is forced to go through those channels in the honeycomb.
And the walls, if you look at that triangular one, those triangular walls themselves are porous, and they're coated with the catalyst, which is platinum.
And that forces the exhaust gas through the wall in contact with the platinum, and then comes out the other end.
And they're ceramic, obviously, because the gas is hot and they need something that has a high thermal resistance.
So these are some examples of honeycombs.
These are some examples of engineering foams.
When I tell people I work on foams, they always think of polymer foams, like polystyrene or something.
And there's lots of polymer foams.
But you can actually foam any materials now.
There's metal foams.
There's ceramic foams, and glass foams, carbon foams, all sorts of foams.
So those are some examples.
You can see when you look at these images here that the foams have a low volume fraction of solids, like if you look at say this polyethylene one here.
Say we look at this guy up here, then you can see there's not much solid, there's a lot of gas.
So the volume fraction of solids is fairly low on that foam there.
So one of the things we're going to talk about is how the volume fraction of solids affects the properties.
You can also see on the top left and the top right, the top left one has what we call open cells.
There's just edges along the polyhedra, there's no faces over the membranes.
And the right hand one is a closed cell foam.
So there's like membranes that cover the faces of the cells.
So we're going to talk a little bit about the differences and the behavior of open cell and closed cell foams too.
These are food foams.
So I've already said you might want to do a project on food foams.
And these are just some examples of different kinds of foods that are in fact foams.
And it turns out the food industry spends quite a lot of time and effort thinking about the mechanical properties of food.
And it turns out if the texture of the food isn't right, then people don't like the way it feels in their mouth.
There's something they actually called mouth feel.
So it turns out if your cereals too soggy, it's icky.
If it's too crunchy, it's icky.
So it's sort of a happy medium.
And food companies spend quite a lot of time and money worrying about the mechanical properties of food.
This is an example of showing that the cells could be antisotropic, the cells could be elongated in one direction.
For instance, in the top one on the polyurethane foam.
And if they're elongated in one direction, it's not too surprising, you might have different properties in that direction from the plane perpendicular to it.
And then the bottom images is of pumice.
Pumice is a volcanic rock.
And you can see how the pores are kind of flattened out there.
And they're flattened out because that was once molten lava.
And the molten lava was flowing down a mountain side of a volcano.
And as it flowed, it got sheared.
And the shape of those pours reflects the shearing as the molten lava flow down the volcano.
And so this kind of sort of stretched out cell shape is going to give you antisotropic properties, different properties in different directions.
This is the 3-D truss that I'm passing around.
I don't know if it's exactly the same one, but it's a similar one.
And these trusses are triangulated structures.
And we'll talk a little bit about their properties too.
And then we also are going to talk about some applications.
So obviously, these materials are mostly air.
And that gives them a low weight.
And that means they're often used in structural sandwich panels as the core of the panel.
And these panels have stiff faces separated by a lightweight core.
And the idea is to make it a little bit like an I-beam.
So the way you have the flanges on the I-beam, the faces are like the flanges, and the porous core is like the web of the I-beam.
They can also undergo large deformations at relatively low stress.
And that means they can absorb a lot of energy.
So if you think of the energy as the area under the stress strain curve, if there's big strains and big deformations, then there's going to be a large area.
And that sort of energy absorption occurs at a fairly low stress.
So typically, when you want to absorb energy, it's not just how much energy you want to absorb.
You have to do it without actually breaking the thing you're trying to protect.
So you don't want to generate high stresses as you go along, and foams are good at this.
Foams are also good at being thermal insulators.
They have a low thermal conductivity.
And that's because they're largely made of gas and the gas has a lower conductivity than the solids.
So that gives them a lower conductivity.
And they have a large surface area.
And the smaller the pore size, the bigger the surface area per unit volume.
And that makes them good for things like carriers for catalysts.
And that's why they're used for these catalytic converters too.
OK, so here's some examples of cellular materials in medicine.
So here's some examples of trabecular bone.
Trabecular bone exists at the ends of your long bones.
So say in your hip or in your knee.
It also exists in your vertebrae in the middle of the spine in the vertebrae there.
And it also exists in your skull.
And you can see it's a porous type of bone.
It looks very similar to the foams and the sorts of mechanical models we make for foams can be applied to the bone as well.
And so that's one of the things we're going to do later in the course.
These are two slides showing what happens when people get osteoporosis.
The left hand slide is from a 55-year-old female to the same bone, the same slice.
And the right hand one is from an 86-year-old female.
This thing here, row star over row S, that's the relative density, the density of the bone divided by the solid that it's made from.
That's the same as the volume fraction of solids.
And so on the normal bone on the left it's about 17% solid.
And on the osteoporotic bone on the right it's, about 7%.
So you can kind of see the bone density has gotten lower, partly by thinning of the struts, but partly by resorption of the struts, as well.
And obviously the one on the right is going to have a lot lower strength than the one on the left.
These are micro CT images of bone.
And again, you can see how the structure looks different at different relative densities.
The one on the left is sort of in the middle at around 11% dense.
The one in the middle is the most dense 25%.
And the one on the right is 6% dense.
So it's not too surprising that the one on the right would have a much lower strength from the other ones.
And we'll look at how we can model that.
This is just showing some deformation in bone.
I have a colleague, Ralph Mueller who's got a micro CT machine, which allows you to do compression tests in the micro CT.
So he can make these sort of images where he scans it at zero strain.
He compresses it a little bit.
He scans it again.
He compresses it a bit more.
He scans it again.
And he these are stills from his images, but he makes animations from this.
And if you look at the top right up here, you see these struts here.
They're pretty straight in this one.
They're a little bent and starting to buckle here.
And then if you look at that one strut there, you can see how it's buckled right over.
So you can look at the deformation mechanisms by looking at the CT scans and things like that.
People are starting to think about using metal foams for coatings of orthopedic implants.
So one of the issues with implants is that say you have a hip implant or a knee implant, you remove the bone that's preexisting, and then you replace it with some sort of implant.
Typically, the implant has a stem that fits into the hollow part of the bone and then has a sort of joint piece to it that fits into the joint.
And you can get some loosening of the stem in the remaining bone.
And one idea is that you use porous coatings to minimize that.
And right now, typically what they do is center beads, metal beads on to the stem.
And another idea is maybe you could use a metal foam.
And these are some different types of metal foams that people are looking at.
Another type of cellular material in medicine is a tissue engineering scaffold.
And this just shows some different examples made by different processes.
And we'll talk more about these later on in the course.
This one here at the top left is a collagen based one made by a freeze drying process.
And I don't know if you saw MIT's website yesterday and today, Ioannis Yannas was the one who developed this.
And he's just being inducted into the National Inventors Hall of Fame.
And this is what he really was inducted for is he's invented a skin-- well, he calls it a dermal regeneration template for regenerating skin, mostly in people with serious burns.
Then these are some other sorts of scaffolds that are made by different processes.
This is by a sort of rapid prototyping process here.
The bottom two, these are kind of interesting.
These are actually the extracellular matrix in the body.
And they've had all the cells removed from it.
So these tissue engineering scaffolds are really designed to mimic the extracellular scaffold in your body or extracellular matrix in your body.
And you can see how when you remove the cells, the structure of those two things looks a lot like a closed cell foam.
So that's the kind of structure you're trying to replicate.
We'll look a bit at cell mechanics.
This is a cell contraction of a scaffold.
So here these sort of very thin transparent bits of the collagen based scaffold, and this is a fiber blast on it right here.
And I had a student, Toby [?
Fryman, ?]
who worked with me and Ioannis Yannas on this.
And you can see from the video the cell is actually contract in the scaffolding making it deform.
And you can calculate what forces the cell must be imposing on the scaffold by knowing something about the geometry of the struts and how the cells attached.
And then it's going a little bit more.
So we'll talk more about that.
And then there's a final picture down here, where you can see these two, the two points up here and down here have now been brought pretty much right together.
So we'll talk about that in more detail.
We've also done studies on cell attachment and how that attachment rate or the amount of cells that attach is related to the surface area, the surface area per unit volume.
So these are just some tests from that done by [?
Fergal ?]
O'Brien, a post-doc that worked with me.
We've also done some studies on cell migration.
And Brendan Harley was the student who did this.
And he stained the cells with one stain and he stained the scaffold with something else.
So red are the scaffold and green are the cells.
And then he used a confocal microscope to track the cells.
And he tracked the cells and where they moved versus time.
And if he has the location at different times he can get the velocity.
And one of the things he did was he changed the stiffness of the scaffold and he found that the migration speed depended on how stiff the scaffold was.
So he was looking at sort of interactions between mechanical properties of the scaffold and behaviors like cell migration.
And then we're going to look at materials in nature.
So here is wood.
So you can see the cellular structure of wood.
It's a lot like the honeycombs.
It has sort of a prismatic structure.
That one happens to be cedar, but other woods look similar.
Now, this is balsa wood.
And this is showing just how the balsa deforms.
I think this was loaded from top to bottom.
And this is at zero load.
And then this is more load, and more load, and more load.
And if you look at that cell there with this little kind of tear in it here, that's the same as the cell down here, and that's the tear there.
So you can see how the cell walls bend and how they deform.
And you can model that using the honeycomb models.
This is just another image showing actual failure of wood, buckling of the cell walls.
This is cork.
So these are modern scanning electron micrograph of cork.
And one of the interesting things is the cork cells have these little corrugations.
You see how they're not flat, they have little kind of wrinkles in them.
And that gives rise to sort of an interesting property of cork.
If you take cork and you load it.
So here we were pulling it in the direction of these arrows, pulling it like along this direction here.
And again, this is the same set of cells.
That tear there is the same as that tear there.
And you see all these little corrugations here, they've all straightened out when we're pulling on it.
And the Poisson's ratio of the cork is zero.
It's kind of like a bellows.
Like if you had an old camera, or you have an accordion bellows.
If you pull the bellows in and out, it doesn't get any wider this way or the other way.
You're just sort of opening the bellows and closing the bellows.
And the cork cells are doing kind of the same thing.
And it gives them this Poisson's ratio of zero.
Which it happens is one of the things that makes it easy to get the cork into your wine bottle, because as you're pushing on it, it's not pushing out in all directions.
This is only for this one direction, but it's not pushing out in all directions.
These are parenchyma cells in plants, in carrots and potatoes.
All those little blobs in the potato, those are starch blobs.
This is called a Venus flower basket sponge, and Joanna Aizenberg, at Harvard, has studied this quite a lot.
This has a hierarchical structure.
If you look at the overall sponge and then you look at each of the sort of struts that make up the lattice, that too has a hierarchical structure.
And she's looked at the optical properties of this glass sponge.
It's kind of a beautiful thing.
And then there's some cellular structures in nature as well.
There's sandwich structures.
There's density gradients.
And there's tubes with a cellular core.
So here's some examples of that.
Here's the iris leaf, you know the iris plant has these long kind of leaves that stand up like this.
And it's just like a sandwich panel.
The parenchyma are kind of like a foam in the middle here.
And there's very dense fibers called sclerenchyma that run up and down the length of the leaf.
And they're like fibers in a fiber composite.
And here's a bull rush or a cat tail leaf.
And they're like little I-beams almost.
It's like a whole series of little I-beams.
And again that's sort mechanically efficient.
These are examples of sandwich structures in bird skulls.
Some of you know I'm a birder.
So I sort of sneak in bird stuff from time to time.
But you can see how these birds skulls are all sandwich panels, and obviously birds want to minimize their weight for flight.
And this is one of the ways that they do that.
This is a horseshoe crab, sort of similar kind of thing.
This is from Mark Myers in San Diego.
He did a study on the crab in its shell as a sandwich.
And this is the ever so handsome cuttlefish.
And the cuttlefish has something called a cuttlefish bone.
This is the bone here and the bone is made up of these kind of sandwich type structures.
The cuttlefish is related to octopus and squid and things like that.
And it's hard to see in this picture here, but these little things here are actually like little tentacles.
There's several tentacles that it eats stuff with.
The cuttlefish is actually a mollusk.
All those things are mollusks.
It's called a fish, but it's not really a fish.
And here's an example of a natural material that has a radial density gradient.
Have you ever noticed if you look at a palm, like you see those pictures of Hollywood in LA, and the palm trees, you know, they line the street.
But they're all about the same diameter from the bottom to the top.
And when you think of like an oak tree, it's not like that.
It's big diameter at the bottom, skinny diameter at the top.
So when wood grows, the wood has more or less the same density in the bottom and at the top.
So as it's growing, the density is more or less the same.
And it resists the bigger loads from getting taller by adding circumference.
So it gets wider and wider as it gets older and older.
But palms don't do that.
They come out of the ground a certain diameter.
And most palms just grow that same diameter.
As they get taller and taller, you can imagine there's wind forces, and different kinds of forces are on it, the stresses get bigger and bigger.
And the way they resist those is that the cell walls get thicker, and they preferentially get thicker on the outside.
And if you think of moment of inertia, remember moment of inertia is increased more with the material on the outside of a beam.
And that's kind of what the palm is doing.
So if you look at young cell walls and old cell walls, here's some SCM pictures of young ones and it's sort of skinny.
And SCM pictures of older ones, and the cell wall has gotten thicker.
So we're going to talk a little bit about that.
And it turns out, this is an incredibly efficient way to deal with getting taller and needing to resist bigger loads.
Another material that has a radial density gradient is bamboo.
So this again shows these sort of dense sclerenchyma fibers.
Do you see these kind of dense parts here?
And you can see there's not very many of them here.
And there's more and more as you get towards the outside there.
So there's a density gradient there.
So we'll talk about that.
And some plant materials have a cylindrical shell with a compliant core.
Plant stems or commonly like.
This is a milkweed stem.
And you can see it's got these sort of fibers that are almost completely dense.
And then a sort of lower density core, cellular core, here, and a void in the very middle.
And you can show that that core helps prevent local buckling.
So if you take a drinking straw and you bend it, you get that kinking kind of failure.
You can show that having this sort of foam like core in the middle helps to resist that.
Imagine you have a drinking straw and now you put foam in the middle.
It's going to be harder to get it to kink like that.
So that's what the plants are doing.
Animal quills do the same thing.
That's a porcupine and a hedgehog quill.
And all of this stuff is in these two books.
So it doesn't really matter to me if you go out and buy the book.
I don't make very many much money on these books.
So this is not an income producing thing.
But those are the books that all these pictures have been taken from.
All right, so I think that's my sort of introduction to the class.
Are there any questions about how we're organized or what we're going to be doing?
Are we good?
It's OK?
OK.
So then I think what I'm going to do for the rest of the time is start the next section of the course which is on processing of cellular materials.
Now, I have another little hand out here.
So I don't know if I'll remember to do this for every lecture, but I like to have a little outline, that partly makes me be organized.
So it's just a little outline for the lecture
[INAUDIBLE]
You asked me to do what?
Put the room light back up
Put the what up?
Lights
Oh, the lights.
I'm going have another set of slides though.
So let me get out of the intro slides.
I know I have another set of slides.
So I'm going to just leave the screen up.
And kind of put stuff on the board and talk about the slides
That's fine
You're good?
OK. OK.
So I wanted to talk a little bit about processing of cellular solids and then, next time we'll start talking about the structures.
It seemed good to talk about the processing before we got to the structure.
So I'm going to talk a little bit about honeycombs and how they make honeycombs, and then foams, and then lattice materials.
Yeah?
The slide you're showing with the the shell with the foam inside it, are there techniques for analysis of it?
Well, I don't think we're going to get into all the gory details, but I can certainly give you references.
That's something that one of my students did at one point.
And in fact, I've been collaborating with Jennifer Lewis up at Harvard and she has a student who's making sort of cylindrical shells with foams out of ceramic foam.
So he's 3-D printing sort of with coaxial nozzles a cylindrical shell that's pretty solid.
And then a ceramic foam on the inside.
And that's one of the things he's playing around with.
So he's looking at ways you might make engineering versions of that.
So I wanted to start with looking at honeycombs and how they make honeycombs.
And I thought what I'd do is I've got some slides.
And I'm going to talk about the slides.
And then I'll put some notes on the board to kind of describe what we're doing, OK?
So this is the first sort of slide on the honeycombs.
And the two main techniques that they're made by, especially those aluminum honeycombs and paper honeycombs that I passed around, one technique is an expansion process.
So what they do is they take flat sheets of some metal, say aluminum, and they put little stripes of glue on it in different places.
So these little kind of specialty things are where the glue goes.
And then they stack those guys up in a sort of particular arrangement.
And then what they do is they pull it all apart, kind like a paper doll thing.
They pull it all apart and when they pull it apart, they get the hexagonal shape.
So let me just show on the board how you do the gluing and how that works.
So they would start with some sheets.
Say we start with two sheets like that.
They'd put some glue down, say there.
And then there's a gap.
And then they put glue on the opposite side over there.
And then there's another gap.
And then they put glue there.
And then they do the same positions but the opposite sides on the next sheet.
So they do that.
And then if you glue those together-- well, let me do another one.
Maybe do a couple more.
So then if I do one like that, it's glued there, and there, and there.
That guy gets there, and there and there.
And when glue that-- when you push that together and then take it apart, you've got something that looks like this.
So say I call that one, two, three, and four.
Then this would be 2 and 3.
OK, so this thing here is that.
Where it's not glued, you get them doing that.
And then it's glued again down here.
And so you get this kind of pattern.
And one of the things about these honeycombs that are made by the expansion process is these inclined walls have a thickness t. And because there's two sheets up here, the vertical walls have a thickness 2t.
So that's typically what you see in the commercial honeycombs that are made by this way.
And this process is used for aluminum honeycombs, for paper resin honeycombs, for Kevlar honeycombs.
And I'll just say note that the inclined walls have a thickness t, and the vertical walls are 2t.
So that's the expansion process.
And the process that's commonly used for honeycombs is called a corrugation process.
And for the corrugation process, it's just like the lower schematic here shows.
You take a flat sheet.
So you've got a roll of a flat sheet.
And you've got some rollers that have the right shape to give you the corrugated profile that you want.
You pass the sheet through the rolls and you get individual sheets out.
And each sheet is kind of a half hexagon.
And then you put the sheets together and that forms the whole hexagon.
So you have a flat sheet that's fed through a shaped wheel to form half hexagonal sheets, which you then bond together.
And it's the same kind of thing that the inclined walls have thickness t and the vertical walls have thickness 2t.
And this corrugation process, you can only really use it in materials that you can deform a fairly large amount to get the corrugations.
So typically, this is for metals.
And aluminum is probably the most common metal that this is used for
How are the corrugated sheets attached to each other?
I think they're just bonded with epoxy.
Yeah, so obviously if you wanted to use it for high temperature performance-- you know, all of these things are bonded with some sort of epoxy or some sort of resin.
So there's an issue if you wanted to use it higher temperatures.
So another process that's used to make ceramic honeycombs is an extrusion process.
And you just take a ceramic slurry and you pass it through a die.
And you can make a ceramic honeycomb by doing that.
And I believe that's how they make the ceramic honeycombs I passed around, the catalytic converter ones.
Other techniques involve rapid prototyping.
You can 3-D print honeycombs.
And Jennifer Lewis has a project on 3-D printing of honeycombs up at Harvard.
And one of the interesting things they're looking at is not just printing with an ink, but printing with a fiber reinforced ink.
So they're making cell walls of the honeycomb that are fiber reinforced.
And one of the tricks is trying to orient the fibers in the way that you want them to be oriented.
So there's rapid prototyping techniques as well.
You can use also selective laser centering.
Let's call it selective laser scanning.
So you can have a photosensitive polymer and use a laser to cure that and build up a honeycomb type structure.
And you can also cast honeycomb structures.
So those rubber honeycombs that I pass around, those are made by casting.
You take a liquid silicone rubber and you add a hardener and you pour it into a mold.
Another kind of interesting way that people have made-- well here's another example of the honeycombs that are 3-D printed.
And this is an example of-- or a couple of examples of looking at a bio carbon template.
So what that means is that these materials are based on the wood, but none of them are actually wood.
So what they do is they take wood, like they take pine, or they take beech or something.
They take some kind of wood and they carbonize it.
So that they do the same processes as used for making carbon fibers.
So you put the wood in an inert atmosphere.
And you pyrolyze it.
You heat it up to I think 800 degrees C. And all you're left with is the carbon.
And it preserves the structure.
And you replicate the structure.
You just get the same structure.
There's some shrinkage.
The shrinkage is about 30%.
But you get the same structure as the wood.
So this material up here is actually all carbon.
It's just replicating the wood that was used, the pine that was used.
An then what people have been doing is using that carbon structure and then infiltrating that with gaseous silicon.
And they form silicon carbide.
So these structures down here are all silicon carbide replicas of wood.
And they're thinking about using that for things like filters for high temperatures or for catalyst carriers.
And one of the attractive features is wood has fairly small cells.
The cells around 50 microns across.
And so you get a large surface area.
And this is a similar thing here.
These two are the carbon template.
And here they've used silicon and they've actually filled in the voids.
And so they've got silicon carbide where the cell walls used to be.
And they've got silicon where the void used to be.
So people are playing around with this is another way of making a honeycomb type of structure.
And they use other kinds of plants besides wood as well.
But that's the kind of general idea.
So the idea is that wood has a honeycomb like structure.
And the cells are fairly small.
the cells are in the order of 50 microns sort of in diameter and maybe a few millimeters long.
And this bio carbon template replicates the wood structure.
So the wood is pyrolized at 800 degrees C in an inert atmosphere.
So say an inert gas.
And that gives you the bio carbon template.
And you maintain the structure, although there's some shrinkage.
structures And then this carbon structure can then be further processed.
So for example, you can infiltrate it with a gaseous silicon.
And you end up with a silicon carbide wood replica.
So possible applications are things like high temperature filters, or catalyst carriers.
I think that's it on the honeycombs.
Are we good with all sorts of methods?
And my little talk here on processing is certainly not comprehensive.
I'm sure there's other ways people have developed.
These are some main ways.
All right then, I want to talk about foams as well.
People have developed different types of processes for different types of solids, so polymers, and metals, and ceramics.
So I just go through each class of solid and talk about that.
So the idea with polymer foams is that you want to introduce gas bubbles into either a liquid monomer or a hot polymer.
And then you want the bubbles to grow.
And then you want to stabilize them and solidify it by other cross linking or by cooling the hot polymer.
So there's a variety of ways of doing that, but let me just put that down.
So there's a few ways to get the bubbles in there in the first place.
One, is just by mechanical stirring.
So if you've ever made meringue, you know what that is, you just take a whisk and you beat egg whites and bubbles.
The air will get enveloped in the egg whites.
They also do that with polymers.
Or you can use a blowing agent.
And there's several varieties of blowing agents.
So the blowing agents are divided into physical and chemical blowing agents.
And the physical ones, they force the gas into solution under high pressure, and then you reduce the pressure, and the gas bundles expand.
So you can use physical blowing agents.
Or you can introduce liquids that, if you're using a hot polymer, that at the temperature of the hot polymer, they form a gas.
So that the liquid just turns into a gas.
And that would form vapor bubbles.
And then the chemical blowing agents.
There's a couple of different ways that those work.
You can either use chemical blowing agents where you have two parts that react together to form a gas.
And so that gas then blows the foam.
Or you can have a chemical blowing agent that reacts with the polymer to form a gas and that blows the foam.
So either way.
And you can have them decompose on heating.
So the same kind of thing.
Evolved the gas when it gets into the hot polymer.
So there's these different ways of blowing the foams.
And there's many, many different types of these blowing agents.
But, these are kind of the general techniques.
And whether or not the foam forms an open cell or a closed cell structure depends on the rheology of the polymers, so the viscosity of it, and also the surface tension.
Another way to make a foam is to make something called a syntactic foam.
A syntactic foam is made by taking thin walled hollow spheres and then using, say a resin, like an epoxy resin, to bond them together.
So you end up with something that's porous.
And you've got the void from the hollow sphere, but you don't foam it in the same way that you blow bubbles through it in some way.
One other thing about polymer foams is they sometimes have a skin on the surface.
So when you blow them, say you've got a mold, there will be a skin that forms against the mold, and sometimes the process is designed in a way that the skin is thick enough that it acts like a skin in a sandwich panel.
So they control the mold in a way and the blowing process so that they get a foam in the middle and thicker skins on the top and the bottom surface.
And that forms a sandwich panel.
Those are called structural foams.
Let's see.
So I think what I'm going to do next is the next section's on metal foams.
And I've got a few slides on that.
So I think I'm going to run through the schematics and just talk about it.
But, I'll put the notes on the board next time.
And there is one thing I forgot to do at the beginning.
I like to tell you a little about me.
And I want to hear about you.
So I wanted to leave a few minutes for that.
So let me just wait until people are finished writing stuff down.
And I'll go through these in a few minutes, and then we'll through it in more detail next time.
And I'll write notes down.
OK?
Are we good?
So there's a whole variety of ways of making foamed metals.
And most of them have been focused on aluminum.
But you could in theory do them with other types of metals.
So this was one of the first processes and it just involved taking a molten aluminum, so here's the aluminum down here in a crucible.
They added silicon carbide powder to it and then they just used a stirring paddle, like they just stirred it up and mixed gas in that way.
And they found that they got bubbles that rose up.
And then they used conveyor belts to just kind of pull the foam off.
And the thing about the silicon carbide was that if you didn't have that, then the bubbles wouldn't be stable enough that you could do this.
The bubbles would collapse before you got to be able to pull them up.
But the silicon carbide I think makes the aluminum melt more viscous and it helps prevent sort of drainage and collapse of the bubbles.
And so that's one way.
And there's a type of foam called Cymat, and this is an example of the foam that's made with that process.
Maybe I'll bring it next time and we can pass it around.
Another method is to use a metal powder and titanium hydride powder.
Then you can consolidate that.
So here's-- it's hard to see the writing, but this is a aluminum powder.
This would be the titanium hydride powder.
You mix them together and then you compact them.
You press them together.
And the titanium hydride decomposes and forms the hydrogen gas at a temperature at which the aluminum is not really quite molten but it's kind of viscous-y, kind of softening.
And so when the aluminum is soft like that and the titanium hydride decomposes and forms the hydrogen gas, you get a foam from that process.
And I think, somewhere, yeah, this foam here I think was made by that process.
Then in a similar thing, you can just put titanium hydride powder into molten aluminum, and again the titanium hydride powder evolves the hydrogen gas and you get foamed aluminum from that.
And I think this foam here was made with that process.
They all look kind of similar.
Another method is by replicating an open cell polymer foam.
So I think I passed an open cell polymer foam around.
And that's made by taking an open celled-- an open cell aluminum foam-- it's made by taking an open cell polymer foam, you fill up all the voids with sand.
You then burn off the polymer, but now you've got sand in all the sort of places where there were voids.
And then you infiltrate molten metal into that.
And then you get rid of the sand.
And then you're left with an aluminum foam that replicates the polymer that you started off with.
So this replication technique.
There's a vapor deposition technique.
And this was developed by Inco to make nickel foams.
So they take again an open cell polymer foam, that's kind of this thing here is, and they infiltrate into it nickel CO4.
The only teeny detail that's a problem with this process is that happens to be highly toxic.
So they put this gas through here.
And then they get nickel depositing on the polymer and they burn the polymer out and they center it.
So it is possible to do this.
They have done this.
But it's not that practical because of the toxicity of the gas.
Now another method is something called entrapped gas expansion.
And here, what you do is you take a can, like a metal can.
This one's titanium, a titanium alloy.
And then you put a titanium powder in here.
You evacuate the can.
So the can has a little valve on it, so you can evacuate it.
And then you backfill it with argon gas and you pressurize the argon gas.
So you've got a powder with sort of pressurized gas inside of a can.
And then you hot isostatically press it.
So you heat it up and you press it uniformly in all three directions.
And you compact it.
And then, if you want you can roll it.
Sometimes people roll it because they want to make sandwich panels, and they want to have a certain thickness, and they want to have faces on the panel.
But then you heat that up.
And as you heat it up, the gas evolves again.
And the thing expands and you get a foam that way.
So that's another method.
Another method is by making hollow spheres and then bonding the spheres together.
And in this process-- this was developed at Georgia Tech.
They used the titanium hydride again.
They made a slurry of the titanium hydride in an organic binder in a solvent.
And then they had a little kind of needle that they injected gas.
And so they had this slurry and they were blowing gas through this needle and they got hollow spheres of the titanium hydride.
And then they heated it up, and again evolved the hydrogen gas off.
But now they're just left with titanium spheres.
And then they bonded the spheres together.
And these aren't titanium.
This is an iron chromium, like it's not quite a stainless steel.
But this is the same thing.
I can pass that around next time too.
Those are the little beads that they make.
And then there's also fugitive phase technique.
So you can take say salt particles, put them in a mold and pour a liquid metal into that, and then, leach the salt away.
And I think that's it for the metal foams.
So I think I'm going to stop there for today in terms of the lecture.
I'll go over that again next time.
And I'll write stuff on the board.
But I wanted to tell you a bit about me.
So the people in 3032, they already know me because they had me in the fall.
But I see some unfamiliar faces.
So I thought I would tell you a little about me.
So I grew up in Niagara Falls in Canada, big power station.
Lots of big civil engineering works in Niagara Falls.
And my father worked at an engineering company that specialized in the design of hydroelectric power stations.
It was founded by the guy who designed the Niagara Falls power station.
And then I went to university in Toronto.
And I did a degree in civil engineering in Toronto.
And then when I finished my degree in civil engineering, I wasn't really sure what I wanted to do next.
And I applied for some jobs, and I applied to graduate school.
I applied to MIT.
And I didn't get in-- ouch.
But I did OK, it turns out.
And I ended up-- I had an advisor when I was in Toronto who had taken a sabbatical in Cambridge, England.
And he said he thought I might enjoy Cambridge, England.
And I ended up going to Cambridge, England to doing my PhD there.
And I worked on the cellular solids for my PhD.
And it was a nice combination because I was interested in material behavior and mechanics, but I had a background in me in civil engineering.
And these are just like civil engineering structures, but they're on a little teeny weeny scale, not like big buildings or bridges or something, like little teeny things.
And really all of this has come out of doing that PhD in Cambridge.
But when I was there, I never even thought about being an academic.
And I never applied for any academic jobs.
I didn't think I wanted to be an academic.
And I went and got a job in Calgary in the oil business.
And I was working at a consulting firm that did work for the oil business.
I hated it.
I just hated it.
It was like I had a boss.
I hated having this boss.
And, you know, the projects were too short term.
The winter in Calgary-- if you think this is bad, you've seen nothing.
Like less snow, but cold.
I mean like 30, 40 below, everyday, cold.
Real cold.
So I stayed there one winter.
And somewhere along the way there, I decided maybe the academic job thing would be good.
And I just sent my CV out to a bunch of Canadian universities.
And I ended up getting a job at the University of British Columbia in Vancouver.
And I lived in Vancouver for two years.
And I probably would have stayed there, except there was a gigantic recession, and it was all very depressing, and there was no money.
And that the universities in Canada are almost all run by the provincial governments.
And the government had no money.
It was all, you know, frustrating.
And I sort of thought, oh, I'll look around and see what else I can get.
And I answered a little ad in Civil Engineering Magazine for a job at MIT.
And I got the job at MIT.
And I was in the civil engineering department for about 12 years.
And then I moved over to the materials department.
Because my work started off on sort of sandwich panels and structural things.
And then it kind of became more biomedical stuff and had less and less to do with civil.
And I've been in the materials department since then.
And this is kind of what I do, this kind of work.
So that's kind of my little five minute story.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I think, last time, we got as far as talking about processing of foams and we talked a bit about processing of polymer foams.
And today, I want to pick up where we left off.
And I was going to talk about processing metal foams.
We'll talk a little bit about carbon foams, ceramic foams, and glass foams.
We'll finish this section on processing today and then we'll start talking about the structure of cellular materials.
And hopefully-- we won't finish that today, but we'll finish it the start of tomorrow's lecture.
And then, we'll start doing mechanics of honeycombs tomorrow.
OK?
So that's the scheme.
I thought what I'd do-- I have a whole series of slides that show, schematically, a variety of different processes for making metal foams.
So I thought I would just go through the slides- and I think I did this really quickly last time-- but I would write down a little bit of notes on each of the slides so that you've got some notes on it, too.
This is the first method here.
And many of these methods were developed for aluminum foams.
But you could, in principle, use them for other types of foams, as well.
This first method here-- let me see if I can get my little pointer.
[INAUDIBLE] The idea here is that you take molten aluminum.
Down in this bath here, you've got molten aluminum.
And they put, into the aluminum, silicon carbide particles.
And the silicon carbide particles adjust the viscosity of the melt.
They make it more viscous.
And then they just have a-- I've got this thing here-- they've got a tube here that they blow gas in with.
And if you go to the bottom of the tube, there's an impeller, or a little paddle, that stirs the gas up.
And so the gas just forms in the molten aluminum here.
And then they have conveyors which pull off the molt-- or the metal foam, and then it cools.
The idea is that if you just had the aluminum, you couldn't really do this because the bubbles would collapse before they cooled down and became a solid.
But adding the silicon carbide particles increases the viscosity of the melt.
It helps prevent drainage of the foam.
Normally, if you have a liquid foam, just from gravity, you're going to have some drainage.
It's going to tend to-- some of the liquid is going to tend to sink, just from gravity.
By putting the silicon carbide particles in, you increase the viscosity.
It helps prevent the drainage.
And you can get the foam bubbles to be stable.
Let me just write one little note here.
The first method here involves just bubbling gas into the molten aluminum.
And that molten aluminum is stabilized by silicon carbide particles.
Sometimes people use aluminum particles and the particles increase the viscosity of the melt.
And that reduces the drainage and it stabilizes the foam.
That process was developed at Alcan in Canada and at Norsk Hydro in Norway.
And this foam here is an example of the foam that they've made.
And you can kind of see-- there's a density gradient in this foam.
I'll pass it around.
You can see it.
But the bubbles are smaller down here and there's fewer of them than there are up here.
And that's partly from the drainage.
The molten aluminum is draining down and you get more liquid at the bottom and then you get more solid at the bottom.
So that's the Alcan foam.
OK. Another-- there's half a dozen of these processes for metal foams.
Another version of the process involves taking metal powder and combining it with titanium hydride powder.
And then you consolidate it and you heat it.
So if I can show through the schematics here.
In the first step, you take the two powders, you mix them up.
So the second thing down here is mixing the powders up.
And then you press them together.
There's a dye here that you press it, and then you get pieces.
And then you can heat that up.
And the way that this works is that the titanium hydride decomposes at a temperature of about 300 degrees C. And if the other powder is something like aluminum-- aluminum melts at something like 660 degrees C-- the aluminum has become soft at 300 degrees C, but it's not molten.
And the titanium hydride-- when it decomposes-- forms hydrogen gas.
So the hydrogen gas forms the bubbles that you need to make the foam.
Are you riding your bicycle?
You are tough.
Very tough.
I ride my bike, but I gave up a couple weeks ago.
Well, three weeks ago, I guess.
When it started snowing, I gave up.
The idea with this process is you can use titanium hydride powder with aluminum and the aluminum becomes soft at the temperature that the titanium hydro decomposes.
And when it forms the hydrogen gas, that gives you the bubbles, and so you get the foam made by that process.
And that was developed by a place called Fraunhofer.
And I have one of their little foams here.
This is an example of their foam made by that powder metallurgy process there.
OK. Let me just write a little note about that.
So you can mix up titanium hydride powder with a metal powder and then heat that up.
When you do this, you need to have a metal powder that's going to be deforming by, say, high temperature creep at the temperature that the decomposition of the titanium hydride happens.
And for aluminum, that works.
So you need to have the metal material be able to deform fairly easily, in order to get these bubbles to form a nice foam.
But that's another way you can do it, is by consolidating these two powders.
Here's another way to do it.
You can also make use of this property of the titanium hydride-- that it'll decompose and form hydrogen gas-- by just putting it into molten aluminum.
And in this example here, you've got an aluminum melt in here.
And this time, they've added 2% calcium to it.
Again, to adjust the viscosity.
And then they add the titanium hydride in this step here and they've got a little mixer thing here that's spinning around and will mix it.
And then they'll put a lid on it to control the pressure and the titanium hydride will decompose, the hydrogen gas will evolve, and you'll get the foam made by that method, too.
So you can stir titanium hide right into a molten metal, as well.
OK.
So that's another method.
And I think that was made by something called the Alporas process.
And this is an example of one of their foams.
I'm pretty sure that's an Alporas foam there.
Yep
[INAUDIBLE]
They can't really control it perfectly.
In that first example that's going around-- maybe you might have missed it-- there's this drainage.
The first one's made by a molten process and you get drainage.
And you get different sized bubbles.
And you get different-- you get a density gradient in the thing, as well.
So you can't control these things perfectly.
OK.
Here's another method for the metal foams.
This one here involves replication.
In this method here, what you do is you start out with an open-cell polymer foam.
In this step up here, there's an open-cell polymer foam.
You fill that with sand.
So you fill up all the open parts of the cells with sand.
Then you burn off the polymer.
And so you've got a little channels where the polymer used to be.
And then you infiltrate those channels with the molten metal that you want to use.
So you replicate the polymer foam structure.
This is the infiltration process here.
This little thing here is your furnace.
And then you get rid of the sand and then you're left with a metal replica of the-- a replica of the original polymer foam.
And this example here is one of these things that's made by this replication process.
So that's an open-celled aluminum.
I think the-- if you look at the density of these things, they're fairly low.
And so there's quite a large volume of pores and they're all interconnected.
So it's not that hard to get the sand out.
This method would involve replication of the open-cell foam-- the polymer foam-- by casting.
So you fill the open-cell polymer foam with sand.
You burn off the polymer.
And then you infiltrate the metal into that.
And then you remove the sand.
OK?
Then, another process involves just using vapor depositions.
So you take an open-cell polymer foam again.
Here-- let me use my little arrow.
Here's the open-cell polymer foam up here.
And you have a furnace here with a vapor deposition system.
And they use a nickel CO4 system to do this.
You then burn out the polymer.
You're left with a metal foam with hollow cell walls, where the polymer used to be.
And then usually, what they do is they center it.
They heat it up again to try to densify the walls.
The only teeny weeny problem with this process is that the gas they use this incredibly toxic.
And so it's not cheap and it's got health hazards, as well, associated with it.
But you can do it.
That gives you a nickel foam when you're finished with that.
And you could also use an electrodeposition technique that's similar.
OK. That's another method.
Another method is shown here.
This is the entrapped gas expansion method.
And what they do in this method is they have a can and the can has a metal powder in it.
It's whatever metal you want to make the foam out of.
In this example here-- it's probably too small for you to read in the seats-- but it's titanium.
They've taken a titanium alloy.
They've got a powder of titanium alloy.
They then evacuate the can.
They take all the air out of it.
And then they back fill it with argon gas.
They put in an inert gas in there.
And then what they do is they pressurize and heat the thing up and so the gas is internally pressurized by doing that.
And then, sometimes when they do this, they want to have a skin on the two faces.
So in this next little image here, it's done where they roll the can and produce a panel that's got solid faces.
And then when you heat it up, the gas expands and you end up with a sandwich panel by doing this.
So this bottom figure down here-- I'm not having much luck with the pointer.
This bottom figure down here, they've heated it up.
The gas expands and you've got this solid skin on the thing, which is from where the can use to be.
So that's trapping of a gas.
OK?
There's a couple more methods for the metal foams.
One involves centering hollow metal spheres together.
And the trick there is to make the hollow spheres.
And the way that can be done is by taking titanium hydride again, if you want to make titanium spheres.
You put it in an organic binder-- in a solvent-- so you've got a slurry here.
And you've got a tube that you blow gas through.
And as you do that, you get hollow titanium hydride bubbles.
And then you can do the same thing where you heat that up.
The hydrogen gas evolves off, and you're left with titanium.
You're left with titanium spheres down at the bottom here.
And then you can pack those together and press those and form a cellular material that way.
You can also center hollow metal spheres.
And the last method I've got for the metal foams is that you can use a fugitive phase.
With the fugitive phase, you would take some material that you could get rid of at the end of the process.
Say, something like salt, that you could leach out.
Here, we have our bed filled with salt.
And then you would infiltrate that with a liquid metal, typically under pressure.
And then, after the metals cooled, you leech out the salt.
You get rid of that.
You can pressure infiltrate a leachable bed of particles, and then leach the particles out.
OK. We have a whole variety of methods that have been developed to make metal foams.
And most of these have been developed, probably, in the last 20 years.
Something like that.
Some of them are a little bit older than that.
But there's been a lot of interest in this recently.
Those are all methods to make metal foams.
I wanted to talk just a little bit about a few other types of foams.
People make carbon foams.
And they use the same kind of method as they do to make those bio carbon templates I told you about.
When you take wood and you heat it up in an inert atmosphere and it turns into a carbon template, you can do the same thing where you take a polymer foam, you heat it up in an inert atmosphere, and everything except the carbon is driven off.
And you're left with a carbon foam.
It's the same process they used to make carbon fibers.
There's carbon foams.
There's also ceramic foams that you can make.
I brought the little sample of ceramic foaming again.
You can pass that around.
And those are typically made by taking an open-cell polymer foam and passing a ceramic slurry through the polymer foam so that you coat the cell walls.
And then you fire it so that you bond the ceramic together and you burn the polymer off.
And you're left with a foam that's got hollow cell walls.
You can also make ceramic foams by doing a CVD process on the carbon foam that you could make by the previous process.
And people also make glass foams.
And to make glass foams, they use some of the same kinds of processes as people use for polymer foams.
I'll just say similar processes to polymer foams.
OK. That covers making the foams, and I think we talked about the honeycombs last time.
I wanted to talk a little bit about making what are called 3D lattice materials, or 3D truss materials, as well.
Let me [?
strip ?]
that up there.
Yeah?
[INAUDIBLE]
Chemical vapor deposition
[INAUDIBLE] use this for metal foams?
Well, people were quite interested in using them for sandwich panels-- the cores of sandwich panels-- lightweight panels.
There was interest in using them for energy absorption, say, car bumpers.
The automotive industry was quite interested in this, in terms of trying to make components with sandwich structures that would be lighter weight, or energy absorption for bumpers.
Or filling up-- if you take-- say you take a metal tube.
If you think of a car chassis and it's made of tubes, if you fill those tubes with a foam-- especially if you fill them with a metal foam-- you can increase the energy absorption quite substantially.
So when you have a tube in a chassis, if it's loaded axially, it will fold up and you get all these wavelengths of buckling.
And if you've put a foam in there, it changes the buckling wavelength and it increases the amount of energy you can absorb.
So not only is the energy absorbed by the foam itself, it actually changes the buckling of the tube so you can absorb more energy.
So there was a lot of interest in that.
There was an interest in using them for cooling devices for, say, electronic components.
The idea was you would take, say, an aluminium open-cell foam and you would flow air through that.
And say you have your device that's generating heat, you'd have a foam underneath it.
And the aluminum conducts the heat fairly well.
And then you would blow air through it to try to cool it off.
So there were a bunch of different applications people have had in mind for them
What about glass?
Glass foams, I think, are largely used for insulation in buildings, believe it or not.
I think, actually, one of the dorms at MIT-- maybe the Simmons dorm-- has a glass foam insulation
[INAUDIBLE]
Well, I think because the foam-- because the cells are closed, the gas is trapped in the cell.
Whereas with a fiberglass, gas could move through the fibers more easily.
So I think that's partly how it works.
OK. Well, let me talk a little bit about the lattice materials, too, and how we make those.
We're going to start talking about the modeling of honeycombs and foams.
And when we do that, we're going to see that if we have a structure that deforms by bending, the properties vary with the amount of material, in a certain way.
But if we have materials where the deformation is controlled by axial deformation, the stiffness and the strength are going to be higher at the same density.
People made these lattice-type materials to try to get something with a more regular structure, and especially a triangulated structure.
You see how these things are like little trusses?
Triangulated?
Triangulated structures, when you load them-- say I load this like this-- there's just axial components-- axial forces in each of the members.
And so, theoretically, this would be higher stiffness and strength for a given weight than, say, a foam would be.
So people were interested in these lattice material.
This one here is made of aluminum.
And I wanted to talk a little bit about how you can make these things.
One way you can do it is by injection molding.
And this here is just the centerpiece of something that would look like this.
So there would be a-- I didn't bring it, but there's a top face and a bottom face that fit onto this.
And they would be injection molded as three different pieces, and then assembled together.
So you can make a mold in this complicated geometry, and you can make a lattice material by injection molding.
We'll start with polymer lattices first.
One way is injection molding.
Another way to do it is by 3-D printing.
You can generate a structure like that by 3-D printing.
You can also make trusses in 2D and you can make them so that you can snap fit those together.
So you can make little 2D trusses.
Here's a little truss here and here's a little piece of a truss here.
And you can make a little snap fit joints.
Do you see how these ones have little divots in them?
And you can make it so that these things will fit together.
I think these guys-- can I do it?
No.
You'd have to take-- oops.
Wait a minute.
No, it's not that way.
There we go.
So you can snap them together like that.
OK.
I can't get it to-- there we go.
So you can do that.
And you do that over and over again.
And if you do it over and over again, you get something that looks like that.
OK?
You can make a snap fit thing.
Let me pass all these guys around and you can play with those.
That's the injection molding one.
This is the snap-fit one.
Got that?
Let's see.
I think I have a little picture here.
This is an example of the snap fit truss here.
It's the thing that's getting passed around.
And another clever way that was developed was by taking a monomer that's sensitive to light.
You take a photo sensitive monomer.
And you put a mask on top of it and the mask has holes in it.
And then you shine collimated light on it.
You shine, say, a laser on it.
And the light goes through the holes in the mask.
And it starts to polymerize the polymer because it's photosensitive.
And then, as the polymer-- as it polymerizes and becomes solid, it then acts as a waveguide and draws the light down deeper into the monomer.
And so the polymer acts as a wave guide.
It brings the light down.
And this is a schematic over here.
This is a schematic showing the set up.
And these are some examples of some 3D trusses that they've made using this technique.
And one of the nice things about this is you can get a very small size cell size.
So this is-- I think that bar-- it says 1,500 microns.
That's what?
One and a half millimeters.
So you can get a nice, small cell size if you want that.
Let me write that down.
You take a photosensitive monomer and then you have it in a mold beneath a mask.
And then you shine collimated light on it.
And as the light shines on it, it polymerizes the monomer.
So then it solidifies and then it guides the light deeper into the monomer.
OK. That's that.
And then finally, there's metal lattices, as well.
And so this is, obviously, a metal lattice here.
It's an aluminum alloy.
And the metal lattice is made by taking that polymer lattice that was made by the injection molding technique.
You coat that with a ceramic slurry.
You burn off the polymer, and then you infiltrate the metal where the polymer used to be.
OK. That's the section on processing of the honeycombs and the foams and the lattices.
So there's a variety of different techniques that people have developed for making these kinds of materials.
And I thought it'd be useful to just describe some of the techniques.
As I think I said last time, this isn't comprehensive.
This doesn't cover every technique.
But it gives you a flavor of what techniques people have developed for making these kinds of materials.
OK?
Are we good?
We're good.
OK.
The next part, I want to do on the structure of cellular materials.
And I have a little overview.
I don't think we'll finish this today, but we'll finish it tomorrow.
Yeah?
[INAUDIBLE]
Down here?
What happens to the ceramic?
They get rid of the ceramic.
Typically, the ceramic is not very strong and it's just fired enough so they can infiltrate it with the metal.
And then they-- yeah, I think with mechanical smushing around, you can get rid of the ceramic.
And the ceramic's brittle, so if you break the ceramic, you're not going to break the metal
I'm wondering if you could make a type of metal lattice [INAUDIBLE] with reducing [?
the ?]
oxides?
I guess you could, if you could-- but you'd have to then make the oxide in that shape, too.
You've always got to make something in that shape
[INAUDIBLE]
Yeah, maybe you could make a foam.
But to make these lattices, you need this really regular kind of structure and be able to control the structure.
OK. Let me scoot out of this set of slides and get the next set up.
OK. We want to talk about the structure of cellular solids.
And we classify cellular materials into two main groups.
One's called honeycombs.
This thing down here is a honeycomb.
And honeycombs have polygonal cells that fill a plane and then they're prismatic in the third direction.
So you can think of them as just being a prismatic-- and they can be hexagons, they can be squares, they can be triangles-- but you can think of them as prismatic cells.
And the cells are just in a 2D plane.
And then we also have foams.
All of these ones over here are foamed materials.
And they're made up of polyhedral cells.
The cells themselves are three-dimensional polyhedra.
And this slide here shows a number of different types of foams.
These ones are polymers up here.
These are two metals.
These are two ceramics.
This is a glass foam down here.
And this is another polymer foam down here.
OK?
[INAUDIBLE]
No.
I just know that

I took those pictures so I know that.
No, you can't tell by looking at them.
In fact, that's one of the things about how we model the cellular materials.
The fact that their structure is so similar is what gives them similar properties.
And they behave in similar ways because they've got similar structures.
OK. We've got 2D honeycombs, where we have polygonal cells that pack to fill a plane.
And then they're prismatic in the third direction.
And then we have what we call 3D cellular materials, which are foams, which have polyhedral cells.
And then they pack to fill space.
The properties of all of these materials depend, essentially, on three things.
They're going to depend on the solid that you make it from.
If you make the material from a rubber or from an aluminum, you're going to get different properties.
So they depend on the properties of the solid.
And some of the properties that we're going to use that are important for this type of modeling are a density of the solid-- which I'm going to call rho s-- a Young's modulus of the solid-- which I'm going to call es-- and some sort of strength of the solid-- which I'm going to call sigma ys for now.
And you could think of other things.
There could be a fractured toughness of the solid.
There could be other kinds of things.
One thing that the properties of the cellular material depend on is the properties of the solid.
Another is the relative density of the cellular material.
And that's the density of the cellular thing divided by the density of the solid.
And that's equivalent to the volume fraction of solids.
So it makes sense that the more solid you've got, the stiffer and stronger the material's going to be.
So it's going to depend on how much material you've got.
And it also depends-- the properties also depend on the cell geometry.
The cell shape can control things like whether or not the honeycomb or the foam is isotropic or anisotropic.
You can imagine, if you have a foam, and you've got equiaxed cells, you might expect to have the same properties in all directions.
But if you had cells that were elongated in some way, you might expect you'd have different properties in the direction that they're elongated relative to the other plane.
So cell shape can lead to anisotropy.
For the foams, you can also have what we call open-cell and closed-cell foams.
If you look at this slide here, and we look at this top right images-- these two up here-- the one on the left in the top is an open-celled foam.
There's just material in the edges.
There's no faces.
And so a gas can flow between one cell and another.
And then if you look at the one on the right, this is a closed-celled foam.
There's faces.
If you think of the polyhedra, you've got solid faces covering the faces of the polyhedra.
For an open-cell foam, you've only got solid in the edges of the polyhedra.
And the voids are continuous, so they're connected together.
And for a closed-cell foam, you've got solid in the edges and the faces.
And then the voids are separated off from each other.
So we'll say, the cells are closed off from one another.
Another feature of the cell geometry is the cell size.
And the cell size can be important for things like the thermal properties of foams.
It's important for things like the surface area per unit volume.
But typically, for the mechanical properties, it's not that important.
And we'll see why that is when we do the modeling.
OK.
Yes?
For the closed-cell foams-- because we can't really see it without cutting it, is it that all of the faces are closed?
Or is it like some fraction of the faces are closed?
If you look at this one on the top right here, they're pretty much all closed.
But the reason we have this little picture down here is some of them are closed and some of them are open.
So you can get ones that are in between.
But typically-- this is kind of unusual.
Usually, they're either all open or all closed.
If we look at the mechanical properties of cellular materials, typically the cell geometry doesn't have that much of an effect.
The relative density is much more important.
The relative density, we define as the density of the cellular solid.
And when I use a parameter like rho or e or something, if it's got a star, it's for the cellular thing and if it's got an s, it's for the solid.
So rho star is going to be the density of the cellular material.
And rho s is going to be the density of the solid it's made from.
And so the relative density is just rho star over rho s. And I just wanted to show you how this is the volume fraction of solids.
So rho star is going to be the mass of solid over the total volume.
Imagine you've got a honeycomb or a foam and you've got, say, a unit cube of it, the sum total volume of the whole thing-- the density of the cellular material is going to the mass of the solid over the whole volume.
The density of the solid is going to be the mass of the solid over the volume of the solid.
This is really just equivalent to the volume fraction of solids, how much solids you've got.
And that's also n equal to 1 minus the porosity.
Typical values for cellular materials-- I think last time I passed around one of those collagen scaffolds-- those tissue enineering scaffolds.
It was in a little plastic bag.
That collagen scaffold has a relative density of 0.005, so its 0.5% solid and 99.5% air.
And if we look at typical polymer foams, the relative density is typically between about 2% and 20%.
And if we look at something like softwoods-- wood is a cellular material.
And we look at softwoods, the relative density is usually between about 15% and about 40%.
Something like that.
OK?
As the relative density increases, you get more material on the cell edges, and if it's closed-cell foam, on the cell faces.
And the pore volume decreases.
And you can think of some limit.
If you keep increasing the relative density more and more and more, eventually you've got-- it's not really a cellular material anymore.
It's more like a solid with little isolated pores in it.
And so there's two bounds.
And the density has to be less than a certain amount for you to consider it a cellular material in the models that we're going to derive to be valid.
And if the relevant density is more than a certain amount, people model it as a solid with isolated holes.
If I have a unit square of material, if it's a cellular material, you might expect that you've got pores that would look like this.
And you've got relatively thin cell walls, relative to the length of the material.
And for a cellular material, typically, the relative density is less than about 0.3.
And when we come to the modeling for the honeycombs and the foams, we're going to see that the cell walls deform, in many cases, by bending.
And that you can model the deformation by modeling the bending.
And that the bending dominates the behavior if the density is less than about that.
And at the other extreme, you can imagine if you had just little teeny pores.
I have a little pore here and one here and one there and one there.
That's not really a cellular material.
It's just got a teeny weeny little bit of pores.
And that could be modeled as isolated pores in a solid.
Each one is acting independently.
And people have found that that is appropriate if the relative density is greater than about 0.8.
And then, in between, there's a transition in behavior between the cellular solid and the isolated pores in the solid.
OK?
Are we OK?
The next thing I wanted to talk about was unit cells.
Especially for honeycombs, people often use unit cells.
A hexagonal cell is an obvious one to use to model this kind of behavior.
For honeycomb materials, you can have unit cells and you can have different ones.
On the left here, we've got triangles, in the middle, I've got squares.
On the right-hand side, I've got hexagons.
And you can see, even if you have a certain unit cell, there's also different ways to stack it.
So the number of edges that meet at a vertex is different for, say, this example on the top left and this example on the bottom left.
Here, we've got six members coming into each vertex, and here, we've got four.
And again, this stacking for the two square cells is also different.
So you can have different numbers of edges per vertex.
Another thing to note that's kind of interesting-- if you look at the honeycomb cells here, this one on the top left-- this equilateral triangle one with the stacking-- and this one on the top right-- the regular hexagonal cells-- those two are isotropic for linear elastic behavior, whereas all the other ones are not.
So we have 2D honeycomb unit cells.
We can have triangles, squares, and hexagons.
They can be stacked in more than one way.
And that gives different numbers of edges per vertex.
And in that figure, a and e are isotropic, for linear elasticity.
OK.
When we come to modeling the honeycombs, we're going to focus on the hexagonal cells.
We'll talk a little bit about the square and triangular cells, as well.
And then, for foams, when you look at the structure of a foam, it's obviously not a unit cell that repeats over and over again.
But people started off trying to model the mechanical behavior of foams by looking at periodic repeating polyhedral cells.
And there's three cells here that are prismatic.
We're not really going to talk about those beyond this.
So they're not really physically realistic or interesting.
But people would use these two cells here in initial attempts to model foams.
And this one here is called the rhombic dodecahedra.
Rhombic because each of the faces has four sides and dodecahedra because each polyhedra has 12 faces.
I forget if I've bored you with my Latin already.
Hedron means face in-- oh, this is Greek, I think.
Hedron means face.
Do is two, deca is 10.
So dodeca is two plus 10.
It's got 12 faces.
OK?
So that's the rhombic dodecahedra over here.
And then this bottom one down here is a tetrakaidecahedra.
It's a similar thing.
Tetra's four, kai mean and.
Four and 10-- tetra kai deca-- it's got 14 faces.
OK?
And those two pack to fill space.
I think those are the only uniform polyhedra that pack to fill space.
Here is the 3D foams.
We have the rhombic dodecahedron and the tetrakaidecahedron.
And the tetrakaidecahedron packs in a bcc packing.
Initial models for foams-- they took these two unit cells.
And what they would do is have an infinite array of them to make up the whole material.
And then they would isolate a unit cell.
And they would apply loads-- some say compressive stress, for example.
And then they would figure out what the load, or force, was in every single member, and how much that member deformed.
And they would figure out the component of the deformation in the same direction that they were putting the load on.
And they would figure out things like a Young's modulus, or they would figure out when there was some failure of one of these struts, and they would figure out a strength for the foam.
But you can kind of imagine, geometrically, not that easy to keep straight.
A little bit complicated.
So one way to model foams is by using these unit cells.
But we're going to talk about a different way to do it, as well.
OK.
So those are unit cells.
When they make foams, as we just talked about, one way to make a foam is by blowing a gas into a liquid.
And if you blow a gas into a liquid, then the surface tension is going to have an effect on the cell geometry and on the shape of the cells.
And if the surface tension is isotropic-- if it's the same in all directions-- then the structure that you get is one that minimizes the surface area per unit volume.
And so people were interested in what sort of cell shape minimizes the surface area per unit volume.
And Lord Kelvin, in the 1800s, was the person who worked this out.
And this is called the Kelvin tetrakaidecahedron.
And it's not just a straight tetrakaidecahedron.
There's a slight curvature to the cells here, to the faces.
And you can kind of see it in some of the edges here.
Like if we-- let me get my little pointer.
If you look at that edge, it's not straight.
This edge here is not straight.
It's got a little bit of a curvature to it.
But this minimizes the surface area per unit volume.
And then more recently, in the 1990's, there were two people-- Dennis Weaire and Robert Phelan-- discovered that this structure here-- which isn't a single polyhedron, but it's made up of a few polyhedra.
That has a slightly smaller surface area per unit volume.
Smaller by 0.3%.
So, a tiny bit smaller.
OK. Let's see.
What I'll say here is that foams are often made by blowing a gas into a liquid.
And if the surface tension controls and it's isotropic, then the structure will minimize the surface area per unit volume.
OK. That's relevant if the foam is made by blowing a gas into a liquid and surface tension is the controlling factor.
Sometimes foams are made by supersaturating a liquid with a gas, and then you nucleate bubbles, and then the bubbles grow.
So there's a nucleation and growth process.
So that's a little bit different.
And if you have a nucleation and growth process, you get a structure that is similar to something called a Voronoi structure.
In an idealized case, imagine that you have random points that are nucleation points and that you start to grow bubbles at those nucleation points.
So you start off with these random points.
And the bubbles all start to grow at the same time and they all grow at the same linear rate.
If you have that situation, then you end up with this Voronoi kind of structure.
And I've shown a 2D version of it here just because it's easier to see in 2D, but you can imagine a 3D system.
And in order to make one of these Voronoi honeycombs, you can imagine-- if you have random points-- here, say that little point there is one of the nucleation points, and here's another point here-- you form the structure by drawing the perpendicular bisectors between each pair of points.
This is the bisector between these two points.
Here's a bisector between those two points.
And then you form the envelope of those lines around each nucleation point.
And that, then, gives you that structure.
And you can see, this structure here is kind of angular.
It doesn't look that representative of something like a foam.
But if you have an exclusion distance, where you say that you're not going to allow the nucleation points to be closer than some given distance-- your exclusion distance-- then you get this structure here.
And this starts to look a lot more like a foamy kind of structure.
So these Voronoi structures are representative of structures that are related to nucleation and growth of the bubbles, or nucleation and growth processes.
Let me write down something about Voronoi things.
And these Voronoi structures were first developed to look at grain growth in metals.
They weren't developed for cellular materials.
But you can use them to model cellular materials, as well, as long as it's a nucleation and growth process.
We'll say that foams are sometimes made by supersaturating a liquid with a gas, and then reducing the pressure so that the bubbles nucleate and grow.
So initially, the bubbles are going to form spheres.
But as the spheres grow, they start to intersect with each other and form polyhedral cells.
And you get the Voronoi structure by thinking about an idealized case in which you randomly nucleate the-- you have nucleation points at a randomly distributed space.
They start to grow at the same time and they grow at the same linear rate.
OK.
The Voronoi honeycomb, or the foam-- you can form that by drawing the perpendicular bisectors between the random points.
So each cell contains the points that are closer to the nucleation point than any other point-- or any other nucleation point.
And if we just do this process as I've described here, you end up with a Voronoi structure, where the cells appear kind of angular.
And if you specify an exclusion distance, where you say the nucleation points can't be closer than a certain distance, then the cells become less angular, and of more similar size.
OK.
So are we good with the Voronoi honeycomb nucleation and growth idea?
Alrighty.
All right.
If we think about cell shape-- if we start with honeycombs and we just think about it hexagonal honeycombs, if I have a regular hexagonal honeycomb so that all the edges are the same length and this angle here is 30 degrees, then that is an isotropic material in the plane in the linear elastic regime.
One of the things we're going to do is calculate-- if I loan it this way on, what's the Young's modulus?
If I load it that way on, what's the Young's modulus?
And we're going to find they're the same, in fact, no matter which way on I loaded it.
It would be the same.
But if I now have my honeycomb, and imagine that I stretched it out-- and I'm kind of exaggerating how much we might stretch it out.
But if we did something like that, it wouldn't be too surprising to think that the properties are going to be different if I loaded it this way on and that way on.
And in terms of the cell geometry, I'm going to call that vertical cell edge length h. And I'm going to call this one-- the inclined one-- of length l. I'm going to say that angle is the angle theta.
And the cell shape can be defined by the ratio of h over l and that angle theta.
OK.
When we derive equations for the mechanical properties of the honeycombs, we're going to find that they depend on some solid properties.
Say, the modulus of the honeycombs can depend on the modulus of the solid.
It's going to depend on the relative density raised to some power.
And we're going to figure out what that is.
And then it's going to depend, also, on some function of h over l and theta.
And that function really represents the contribution of the cell geometry to the mechanical properties.
OK. That's the honeycombs.
It's fairly straightforward to characterize the shell shape for the honeycombs.
It's a little more involved to do it for the foams.
And the technique that's used is called the mean intercept length.
At least, that's one technique that's used.
Let me wait until you've finished writing because I want you to see the picture as I talk about it.
OK?
OK.
Here's-- whoops.
My pointer keeps disappearing.
This top left picture here shows an SEM image of a foam.
And you can see, you've got some big cells and little cells and there's no obvious way to characterize the cell shape.
And what people do to calculate this mean intercept length is they would take an image.
They would then sketch out just the cell edges that touch a plane's surface.
So all these black lines are just the-- if you took your-- if you put ink on your foam and you just put it on a pad and put it on a piece of paper, you would get this outline of the edges of the cells, where they intersect that plane.
And then what people do is they draw test circles.
Here's the test circle here.
And they draw parallel equiaxed, or equidistant lines.
So the lines here are parallel.
They're all at, say, zero degrees.
And they're all the same distance apart.
And then they count the number of intercepts.
They count-- say we went out here.
The cell wall intercepts here.
There's one that intercepts here.
And then, it'd go up here.
Here's another intercept.
Here's another intercept.
So they count the number of intercepts of the cell wall with the lines.
And then they get a mean intercept length, which is characteristic of the cell dimension.
And then what they do-- because this is just in one orientation-- you would then rotate those parallel lines by, say, 5 degrees and do it all over again.
And get another length at 5 degrees and one at 10 degrees one at 15.
And so you get different lengths for the intercepts as you rotate your parallel lines around.
And then you make a polar plot, and that's what the thing is down at the bottom here.
And so you plot your mean intercept length as a function of the angle that you measured it at.
And you can fit it to an ellipse.
And if you do it in three dimensions, you fit it to an ellipsoid.
And the major and minor axes of that ellipse, or ellipsoid, are characteristic of how elongated the cell is in the different directions.
And the orientation of that ellipsoid is characteristic of the orientation of the cells.
Those of you who took 303, too, you remember Mohr's circles?
Is this beginning to look familiar?
It's the same kind of idea as Mohr's circles.
Same way we have principal stresses and orientation of principal stresses, now we have principal dimensions and the orientation of the principal dimensions.
So it's the same kind of idea.
OK?
Let's see.
I feel like I'm getting to the end here.
Maybe I'll stop there for today.
But next time, I'll write down the whole technique about how we get these mean intercepts and get this ellipsoid.
And I'm going to write the mean intercepts down as a matrix.
But you could also write it as a tensor.
And there's something called the fabric tensor, which characterizes the shape of the cells.
And as you might imagine, the same is with the honeycomb.
If you have equiaxed cells in the foams, you might expect you would get isotropic properties.
If you have cells that are stretched out in some way-- so you've got different principal dimensions for them-- then you've got anisotropic material.
And you can relate how much anisotropy to the shape of the cells.
OK.
I'm going to stop there for today.
I'll see you tomorrow.
Seems very sudden.
I'll see you tomorrow.
I'll pick up and I'll finish this section on the structure.
We've got a bit more to do.
And then we'll start looking at honeycombs and modeling honeycombs.
The honeycombs are simpler to model just because they have this nice simple unit cell.
So we'll start with that, and then we'll move from there to the foams.
OK?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so I guess I should start.
So I think last time we were talking about cell structure and cell geometry.
And I got as far as putting this image here up.
And I talked a little bit about how this works.
And I was going to go over it again and then write the notes down today.
So we'll start from there.
So we're going to do a little bit more on cell structure today.
We're going to talk about some topological laws for cellular materials for polyhedral cells.
And then we'll start talking about modeling honeycomb materials, and talk about how we look at the mechanical properties of honeycombs.
So I think where we left off last time was we started talking about mean intercept length.
So the idea is that with-- hello, hello-- if you have a honeycomb, it's fairly easy to define the cell shape in terms of the ratio of the cell edge lengths, h over l, and the angle that the inclined cell wall is to the vertical one.
But for a foam it's a little more difficult.
And what people do is they measure this mean intercept length.
So last time I talked about it briefly.
So the idea is you take an image of the structure you're interested in.
You sort of draw out the outline of a planar surface.
And then you put on that a test circle.
And you superimpose on the test circle equidistant parallel lines.
So say we start off at theta equals 0.
We then count how many times the cell walls intercept those lines.
And we get a number of cells per unit length.
And that gives us an intercept length for that orientation.
Then we rotate it around a little bit, say 5 degrees or something.
And we measure a new intercept length for that orientation.
And we keep doing it all the way around, 180 degrees.
And then you get-- then what you do is you plot those points-- if I can find my little pointer-- we plot those points.
And it will form an ellipse.
And the major and minor axes of the ellipse correspond to the principal dimensions of the cells on that plane.
And then you can do the same for perpendicular planes and form an ellipsoid.
And you can get the three principal dimensions.
You can also get the orientation of the cells by getting the orientation of this ellipse, or the ellipsoid in three dimensions.
So let me just write down some notes that kind of summarize all of that.
So I'm going to say that for foams, we characterize the cell shape and orientation by using these mean intercepts-- mean intercept lengths.
So we consider a circular test area on a plane section.
So you want to use a circle and not say, a square or a rectangle.
Because if you had a square, then you'd have different total lengths of line, depending on what orientation you took it.
So you want to use a circular test section.
And then you draw equidistant parallel lines.
So for example you might start at theta equals to 0.
And then you count the number of intercepts of the cell walls with the lines.
So if we say that Nc is the number of cells per unit length of line, then we can get the intercept length for that orientation.
Say for theta equals to 0, it works out to 1.5/Nc.
So it's not just 1/Nc, because you may not be cutting the cell-- you know, you're cutting the cell-- say you've got a three dimensional cell like this.
You're cutting it at different places along here.
So people have worked out the stereology of that.
And there's a constant that's 1.5 that fits into there.
So then you just repeat that process for different increments of theta.
So you increment theta by some amount, something like say, 5 degrees.
And then you repeat that whole process.
And then you plot a polar diagram of the intercept lengths versus theta.
And you then fit an ellipse to the points.
And if you did it in 3D, you'd have an ellipsoid.
And then the principal axes of the ellipsoid are the principal dimensions of the cells.
And the orientation of the ellipsoid is the orientation of the cells.
Hello.
And you can write the equation of the ellipsoid.
So it would be something like Ax1 squared plus B x2 squared plus Cx3 squared plus 2Dx1x2 plus 2Ex1x3 plus 2Fx2x3.
And that would all equal 1.
And you can write those coefficients as a matrix.
And if you do that, the first three, A, B, C, are the diagonals, and D, E, F are the off diagonals.
And you can also represent this is a tensor.
And if you do, it's called the fabric tensor.
So the fabric-- I think they use that term for other types of materials-- it says something about the orientation of the material.
And if you have this matrix here, if D, E, and F are all of 0, then A, B, and C are the principal dimensions of the cell.
So if all non-diagonal elements are 0, then A, B, and C are the principal dimensions.
Are we good with how this works?
So it's kind of a crank and churn kind of thing.
But it gives you a way of characterizing the cell shape and the orientation of the cells in the foams.
So the next thing I wanted to talk about was the connectivity of the cells.
So imagine we have an array of cells.
And if we have an array we have vertices that are connected by edges.
And edges surround faces.
And the faces enclose the cells.
And there's something called the edge connectivity, which is usually given the symbol Ze.
And that's the number of edges that meet at a vertex.
And there's a face connectivity, and that's the number of faces that meet at an edge.
And it's very common for honeycombs to be three connected.
So Ze is 3.
And it's common for foams to be four connected.
And there's some topological laws that are kind of interesting.
And so we'll get into those after this bit on connectivity.
So for the connectivity we have vertices, which are connected by edges, which surround faces, which enclose cells.
So we're going to talk about the vertices, the edges, the faces, and the cells.
So the edge connectivity, Ze, is the number of edges that meet at a vertex.
And for a honeycomb, say a hexagonal honeycomb, Ze is 3.
So I have a little honeycomb here.
If we look at the number of edges, there's three edges, connect at a vertex.
So Ze is 3.
And for a foam, Ze is typically four.
So I'll just say typically here.
And if I go back, if you look at the sketches here of the rhombic dodecahedra and the tetrakaidecadedra, if you look at the tetrakaidecadedra, you can see the number of edges.
So here's one edge.
Here's another one here.
There's another one here.
And there's another one here.
So that's the four edges that are meeting at that vertex there.
So if you look at arrays of cells, you can convince yourself that Ze is typically 4 for a foam.
And then we also have the face connectivity, Zf.
And that's the number of faces that meet at an edge.
And that's typically three for foams.
And so again if you look at these pictures you can sort of see how the face connectivity is three.
So if you look at this bottom one here, there's faces here.
And then there's another face-- well, let's see if we can-- it's kind of hard to show.
If you look at this one here, there's this cell.
There's that face there.
And then there's another one coming out of the page, I think.
OK.
So that's connectivity.
And the next thing are some of these topological laws.
So the first one I'm going to talk about is called Euler's law.
If you remember Euler from buckling, same guy.
So Euler's law relates the number of vertices and faces and cells and edges for a large array of cells.
So we can relate the total number of edges, so I'm going to call that E, vertices V, faces F, and cells C. That total number is related by Euler's law for a large aggregate of cells.
So in 2D Euler's law is that the total number of faces minus the number of edges plus the number of vertices is equal to 1.
And in 3D, you just add a minus the number of cells in front.
So minus the number of cells, plus the number of faces, minus the number of edges, plus the number of vertices is equal to 1.
So there's some interesting things you can figure out using Euler's law.
And one of the things I wanted to look at was if we had an irregular honeycomb that was three connected.
So say we have a honeycomb that's not just a regular hexagonal honeycomb, or not even one that's got repeating hexagonal cells.
But say we had a honeycomb where some of the cells had 5 sides to them, some of them had 7, some of them had 6.
You can ask, what's the average number of sides per face?
And you can use Euler's law to figure that out.
So we're going to look at an irregular three connected honeycomb.
Let's see, I can start that up here.
So when I say irregular, I mean we've got cells with different numbers of sides.
So we have an irregular-- oops.
You OK, Greg?
--that is three connected.
And what is the average number of sides per face?
And I'm going to call that n bar.
So we're told that it's three connected.
So that's saying that the edge connectivity is 3.
So there's three edges coming to each vertex.
So imagine that you had just a regular hexagonal honeycomb, like this.
Here's our vertex here.
And there's our kind of three edges that come into it.
And each vertex is going to have half of each edge, right?
Because the next vertex-- each edge is shared between two vertices.
So if Ze is equal to 3, then the number of edges per vertex is 3/2.
Because each edge is shared between two vertices.
And I'm also going to define something I'm going to call Fn.
And Fn is going to be the number of faces with n sides.
So I could have some number of the cells have six sides, some number have five sides, some number have seven sides.
So these are-- Fn is the number of faces with some particular number of sides, n. So if we have Fn is the number of faces with n sides, then if I sum up n times Fn and divide it by 2, that's the number of edges.
So imagine I have all these faces.
I take n times Fn.
So say we had four five-sided faces.
That would be four of those.
There's be 20 edges associated with that, 20 sides.
Then I have some number of six-sided, some number of seven-sided.
I add them all up.
And I have to divide by 2 to get the number of edges, because each edge separates two faces.
OK. And then I can use Euler's law.
So I can say F minus E plus V is going to equal 1.
And here I've got an F minus E plus-- I can put V here.
V's going to be 2/3 of E. So I've got that.
So that's the same as F minus 1/3 of E is equal to 1, like that.
And then for E I can substitute this thing here in.
So that gives me F minus 1/3 of the sum of n times Fn over 2 is equal to 1.
And then what I'm going to do is multiply everything by 6 so I can get rid of my denominator here.
So 6F minus sum of nFn is going to be 6.
And then I'm going to divide that through by F. So that's 6 minus sum of nFn over F is equal to 6/F.
And then if I let F go to a large number-- so say I've got a large aggregate of cell.
I've got lots of faces.
I'm going to let F become large.
So if F becomes large, then 6/F is going to tend to 0.
Let me get the other rubber
Professor?
Mhm?
That's-- F is like the total number of faces, not the total number of faces per cell
F is the total number of faces.
And Fn is the number of faces with n sides.
We've got some number with 5 sides, some number with 6, some number with 7.
So we're almost at the end here.
So 6/F goes to 0.
And so that says that the sum of n times Fn over F is equal to 6.
And that just is n bar.
That is the average number of sides per face.
This is your-- kind of your total number of sides in the whole thing.
And you're dividing by the total number of faces.
So that is the average number of sides per face.
So what this is saying is if you have a three connected honeycomb, the average number of sides per face is always 6.
So that if you introduce a cell with 5 sides, somewhere you have to introduce a cell with 7 sides, so that they compensate and you come back out to 6.
And I have a little soap bubble picture here that kind of illustrates that.
So here we have a soap honeycomb.
So you can make these just by putting two glass sheets close together with a soap bubble froth in between them.
And you can kind of see in this picture here, they've numbered how many sides each cell or each face has.
So here's one with 5, here's one with 7.
And I'm not going to add these up all together.
But you can kind of see that the 5's and the 7's kind of compensate for each other, and that the average works out to about 6.
And this is true for a large aggregate of cells.
So if you only have a few, it's not going to work out perfectly.
You need to have a large aggregate to have it work.
Yeah?
And that 5, 7 balance doesn't have to be touching, right?
No, no, no, overall.
Yeah, overall.
And, you know, you could have one with 4 sides, and you'd need two 7's or one 8 or something.
But you'd need to have that balance out and match up.
OK.
So that's the Euler law.
Now there's a couple more of these kinds of things.
There's something called the Aboav-Weaire law.
And so the Aboav-Weaire is sort of related to the Euler because it's looking at this idea that if you have a three connected honeycomb, if you introduce a 5-sided cell, you have to introduce a 7-sided cell to compensate.
And what Aboav noticed was that generally cells with more sides than average have neighbors with fewer sides than average.
Let's see.
So we'll say the introduction of a 5-sided cell requires the introduction of a 7-sided cell.
And I'll just put-- this is all for a 3 connected net, a 3 connected honeycomb.
So that generally, cells with more sides than average have neighbors with fewer sides than average.
And in 3D, you could say the same thing about the faces.
The cells that have more faces than average have neighbors with fewer faces than average.
So Aboav made observations of this.
And it was Dennis Weaire who made a proof of it.
And the equation that they came up with relates the average number of sides in the cell surrounding a candidate cell.
Let's see-- are we going to be able to put it in here?
Maybe.
So say you have a candidate cell and say it has n sides.
So you look at one particular cell and you say, count up, it's got n sides around it.
And then you count up the number of sides of all the cells surrounding it.
It has n neighbors because it has n sides, so then the average number of sides of the n neighbors is called m bar.
It has n sides.
So then the average number of sides of its n neighbors is m bar.
And m bar is equal to 5 plus 6 over n for a 2D honeycomb kind of cell structure.
OK, so that's the Aboav-Weaire law.
And then there's one more of these things.
The last one is called Lewis' rule.
And Lewis looked at biological cells and 2D cell patterns, and he found that the area of the cell varied linearly with the number of the sides.
And he found just an empirical relationship.
It looks like this.
We can say what everything is in a minute.
So he found that the area of a cell with n sides, that's A n, was linearly related to the number of sides, so this n over here, and naught's just a constant and A n bar is the area of the cell with the average number of sides.
So here, A n is the area of cells with n sides and A n bar.
And then n naught is just a constant.
And he found that for 2D cells n naught was equal to 2.
And if you look at voronoi honeycombs-- remember last time we talked about those voronoi honeycombs-- you can show that this holds for voronoi honeycombs.
And then you could write a 3D version of this as well.
And in 3D, it's the volume and faces instead of the areas and the number of sides.
So you can say that the volume of the cells with f faces relative to the volumes of the cells with the average number of faces, they vary linearly with the number of faces.
So there's sort of an exactly analogous expression here.
And here, this f naught is another constant.
And in 3D it's about equal to 3.
So these are all just kind of interesting topological rules that are nice to know about.
OK. Yeah?
Basic question.
In two dimensions, what is a face?
Like what does that refer to?
So-- and let me put my little slides again.
So say we have some honeycombs like this, then say we look at this guy.
So that's a vertex there, that's an edge, and this thing in the middle here is the face.
So in 2D, the face and the cell is kind of the same thing.
All right?
And then--
[INAUDIBLE] sides, what's different with the edge?
It's not quite.
Like if I count up the number of edges, I say one, two, three, four, five, and I count those up like that.
But if I say sides of the face-- see that's a face or it's a cell-- I would say that had six sides.
Right?
So typically when people say it has-- a cell or a face has so many sides, they count up how many all around it.
But because each side is-- or each edge is-- shared between two faces, the number of edges is actually half that.
Right?
Because there's one edge, but if I count up the number of sides for this face and I count up the number of sides for that face, I'm counting that twice.
So it's a little bit-- so I try to see edges when I'm talking about adding them all up and I try to say sides when I'm talking about, here's a face, how many sides does it have.
OK?
It's a little bit confusing.
Anybody else?
[INAUDIBLE] what is n bar naught?
n bar naught, did I put an n bar naught?
Oh, that's where I didn't erase this enough.
There you go.
It's just n naught.
OK.
So we've been talking about the structure of the honeycombs in the foams.
And what we ultimately want to do is be able to model the mechanical behavior or the thermal behavior, some sort of behavior of the cellular material.
And for looking at mechanical behavior, there's three main approaches that people take.
So you have to model the structure somehow.
So there's three main approaches.
And the first one is to use the unit cell.
So say, for example, for the honeycombs, if you have a hexagonal honeycomb you'd just use that unit hexagonal cell.
And that's what we're going to do probably starting later on today.
So for the hexagonal honeycomb, it's kind of obvious.
You would use a unit cell, the thing's periodic, you figure out how that unit cell behaves, you're all set.
For a foam, it was not so obvious a unit cell to use.
But the thing-- people use different things, but one of the common ones they use is a tetrakaidecahedron.
So the nice thing about the tetrakaidecahedron is that it packs to fill space.
So it's a repeating single cell.
Packs to fill space.
But, in fact, real foams aren't tetrakaidecahedrons, so this is a bit of an idealization.
So we'll just say foam cells are not tetrakaidecahedrons.
OK, so that's one approach.
And then a second approach is to use something called dimensional analysis.
So in dimensional analysis, what you want to do here, with this technique, is model the mechanisms of deformation and failure in the structure.
But you don't necessarily represent the cell geometry exactly.
And so what you do is, you say one thing is proportional to another and there's some constant of proportionality, and you just wrap all of those constants of proportionality up at the end.
So for instance, when people look at foams, the geometry of the foams is kind of complicated.
There's cells with different number of faces, there's different sizes of cells, it's kind of a mess.
So we could just say the geometry is complex and it's difficult to model.
And with dimensional analysis, instead what we do is we model the deformation mechanisms and failure mechanisms.
And you can get quite a bit out of just modeling those.
So when we look at modeling foams, we're going to do this, and you'll see how it works.
And then the third method is to use finite element analysis.
So this is a numerical technique and it's a very standard numerical technique.
And one of the nice things about this is you can apply it to random structures.
So for example, those voronoi structures we saw, if you want to try to see-- say you had a certain amount of material and you had a honeycomb that was a regular hexagonal honeycomb.
You had the same amount of material and you put it in a voronoi honeycomb, and you want to know how does having a random material affect the properties relative to the uniform material?
You could figure that out using finite element analysis.
So you can apply it to random structures.
And another thing you can do with finite element analysis-- and people in the orthopedics end of the world often do this-- if you're interested in trabecular bone, that porous type of bone I showed you the first day, people can take micro-computed tomography images of trabecular bone and they get a file which basically says, every voxel, says if it's solid or it's void.
And you can use those files as input to a finite element analysis.
And so you can analyze exactly how a piece of a trabecular bone would deform.
So it's nice for that.
So one of the things we're going to talk about later is we'll look at the structure of trabecular bone and the sort of mechanics of trabecular bone, and I'll show you some results that a former student of mine did, where we look at-- say you have a sort of intact structure of a certain density and then you reduce the density by fitting the cell walls or you reduce the density by removing cell walls-- because in osteoporosis sometimes the walls resorb all together-- and you can see what residual strength you would have for some given amount of density loss.
So it's good for looking at that kind of a thing.
You can also use it for looking at local effects.
So you can look at defects, for instance.
So if you think of the trabecular bone as having missing trabeculae, that would be a defect in the structure.
You can look at that.
You can look at size effects.
So when we were studying some of those metal foams, some of them have very large cells, and if you have a small sample relative to the cell size, you may only have four or five cells across the dimension.
Where you've cut the cells, you've got edges that are less constrained than in the bulk of the material because at the outer edge you've cut them.
They're not connected to anything else.
And so you can look at edge effects that relate to the size effects in foams as well.
So these are basically the three approaches that people use for modeling cellular materials.
And what we're going to do is we're going to start with looking at honeycombs and we're going to look at these hexagonal kind of honeycombs like this.
And one reason to start with them is that they have this unit cell structure.
And if you can analyze how that unit cell deforms and fails, you can say something about the whole structure.
And it turns out that the honeycombs, they deform and fail by the same mechanisms as the foams.
So if you can understand through this simple structure, it gives you a lot of insight into how the foams behave.
So if I deform this a little bit, the cell walls bend.
And you can show that if you deform this guy a little bit, the cell walls bend.
And so you can learn a lot by looking at the honeycombs and then sort of applying that to the foams.
So we're going to start off-- so that's the end of the section on sort of the structure of the cellular materials-- and now we're going to look at modeling the mechanical properties and we're going to start with the honeycombs and then we're going to do foams.
So when we talk about the honeycombs, we've got this hexagonal structure here and we're going to call properties in this plane the in-plane property.
So if I load it this way on or that way on, those are in-plane, and if I load it that way on, those are out of plane.
So think of the cells are in the plane and the prismatic direction is the out of plane.
And clearly, the honeycombs are going to have similar properties this way and that way, but they are going to have very different properties that way.
So we're going to start with the in-plane behavior.
So the honeycombs have these prismatic cells and they're widely available in different materials, polymers, metals, ceramics.
And they're used in a variety of applications.
So one of the most common is to use them in sandwich panels.
So I brought a couple of sandwich panels with me today.
So here's a couple of sandwich panels that have honeycomb cores.
This one's an aircraft flooring panel.
It's got carbon fiber faces and a Nomex core, honeycomb core.
And this is an aluminum honeycomb.
It's got aluminum honeycomb core and then aluminum faces, and it's kind of amazing how stiff that little panel is.
And each of the pieces is really not that stiff at all, but the thing put together is quite stiff, and we'll talk more about that when we get to sandwich panels.
So it's used in sandwich panels.
They're used for energy absorption.
So sometimes you'll see there's some natural disaster area and they fly in helicopters and they drop big crates of supplies, and they will have it like a pallet with a big crate thing.
Often they have a honeycomb, like a metal honeycomb, that is kind of oriented this way.
So the pallet would be like this and the honeycomb's like that.
And the idea is that when they drop it-- they sort of bring it down as close as they can-- and then the honeycomb absorbs some of the energy from the impact.
And they're also used, I think, sometimes in car bumpers.
So they're used for energy absorption.
And they're used as carriers for catalysts.
So the catalytic converter in your car looks like this, this is kind of the material that's used in the catalytic converter in your car.
And the way that works is, the cell walls here are actually porous and they're coated in the platinum, which is the catalyst, and half of the cells are blocked off on this end.
So every other cell on this end is blocked off and then every other cell over here, the opposite ones, are blocked off.
And so the gas is forced down a channel but then through the wall and then out the next channel.
And that's where the reaction actually occurs, is in the cell wall.
So they're used as carriers there.
And some natural materials also have a cellular structure, have a honeycomb structure.
So for example, things like woods and cork have a honeycomb structure, too.
So I said the mechanisms of deformation and failure in the hexagonal honeycombs parallel those in foams.
So we can learn a lot about foams by understanding the honeycombs.
And then, similarly, the mechanisms of deformation and failure in triangulated honeycombs parallel those in the lattice materials.
Remember I brought those lattice materials in?
The sort of trust type materials.
The triangular honeycombs have a behavior similar to those lattice materials.
OK.
So let me scoot over here.
OK.
So let me scoot out of this one.
OK.
So here is our kind of hexagonal geometry.
This is kind of an idealized geometry here.
And, as we talked about in this section on the structure, we're going to call these vertical walls, we're going to say they have a length h, the inclined walls have a length l, the wall thickness is t, and there's going to be an angle between the horizontal and that inclined wall of theta.
And I'm going to define three axes here, an x1, an x2, and an x3 axis.
So the x1, x2 plane is the in-plane and x3 is out of plane.
OK?
So if we load our honeycomb up, we get stress strain curves that look like this.
So the ones on the left here, over here these three are all in compression, and the ones on the right, these three are all in tension.
OK, so let's talk about the compression ones first.
And let's start at the top.
This is in elastomeric material, so like one of these rubber honeycombs.
This would be a material that yields plastically, so like an aluminum honeycomb.
And this would be a honeycomb that fails in a brittle manner, like one of those ceramic honeycombs.
OK?
So if we go up to the elastomeric one, up here, if I compress it I get a linear elastic part first, and then at some point, I get a stress plateau where the stress is almost constant for strains that are quite large.
And then finally, the stress starts to rise quite sharply at the end here.
So the strain here goes from 0 to 1.
So that's a strain of 100%.
You've completely flattened the thing there.
So that's a large strain.
So initially, when we're loading it up to smaller strains like this, we've got linear elastic behavior and these cell walls bend.
And you can relate the modulus here-- let's see where'd my little arrow go-- you can relate this Young's modulus here to the bending of those cell walls.
And we're going to-- I don't know if we'll finished that today-- but we'll start that today.
Then-- oops, lost my arrow, where'd my arrow go-- then at the stress plateau here, that plateau is related to collapse of the cells.
So if I have my little rubber honeycomb and I load it, at some point, the cell walls buckle.
So you see how they've buckled there?
And that stress plateau, I can smush that to quite large strains that are roughly constant stress.
OK?
So what's happening here is that we're buckling the cells.
And as we go along here, the buckling deformation gets bigger and bigger.
And then if I smush it and I've got it kind of like that, at some point it becomes much harder to press it together again because the cell walls are now touching each other and they're pressing against themselves, and to get a certain amount of strain, it gets much more difficult to do that.
And the stress required to do that gets much bigger.
And that's what leads to this last piece of the stress strain curve here, which is called densification because you've almost eliminated the pores.
It's hard to eliminate them entirely but the porosity has gone way down by the time you're up there.
So this type of stress strain curve where you've got linear elasticity and then a stress plateau and then the densification is classic for compressive behavior of cellular materials.
So elastomeric ones will have a plateau that's related to elastic buckling.
I lost my arrow again.
There we go.
If we had a metal honeycomb, we'd again have linear elasticity related to cell wall bending.
This plateau here would be related to yielding of the cell walls.
So say we had aluminum honeycomb, the aluminum could yield, and that would again cause this stress plateau.
And then we get densification.
If I had a brittle honeycomb like the ceramic one, we'd have initial linear elasticity.
Then you've got a stress plateau that's kind of a lot of up and down here.
And the serrated nature is related to the fracture of individual cell walls.
So it goes up and down because when you break a cell wall, the stress drops off, and then the other cell walls will try to pick the stress up again.
And so each one of those little up and downs corresponds to breaking a cell wall.
But if you kind of took an average of that, you can see there's a stress plateau and then there's the densification region.
So in compression, the shape of the curves is very similar and the mechanism of the plateau varies a little bit.
So let's see if I can write some of that down.
So in compression, we can say we have three regimes of behavior.
So we've got the linear elastic regime initially and we're going to see that's related to cell wall bending.
Oh, thanks.
And then we've got a stress plateau.
And for elastomeric materials, that's caused by buckling of the walls.
For metals, it would be caused by yielding.
And for ceramics, it would be caused by a brittle crushing.
And then we've got densification.
And that's related to the cell walls touching.
And if we increase the ratio of the thickness of the cell walls relative to their length, we're going to increase the stiffness, so the Young's modulus of the honeycomb.
The stress plateau I'm going to call sigma star.
And we decrease the strain at which that densification occurs, which I'm going to call epsilon D. So you see over on the right hand side of these plots here, that strain there is the densification strain, epsilon D. So that's in compression.
And these materials are very often loaded in compression.
In tension, we still get linear elasticity initially.
And that's going to, again, be related to the bending of the cell walls.
But if we look at the stress plateau, if you look at these three curves here on the right, the stress plateau only exists if the material has a yield point and you get some plastic yielding there.
If you have an elastomer, if I pull on this-- you don't get buckling in tension, so you're not going to get a stress plateau in tension.
And for a brittle material like one of those brittle honeycombs, if you pull that in tension, you would just get a crack propagate and the thing would break into two pieces and so you don't get a stress plateau there.
So the stress plateau in tension only exists if the material yields.
So you don't get any buckling in tension.
And for a brittle honeycomb, you would just get fracture.
OK?
These inclined walls are going to bend.
We're going to see that in more gory detail in one minute.
If you can wait one minute.
OK.
So are we good with a stress strain curves yet?
More of an abstract question, but is it possible [INAUDIBLE] to get collapse this way before the plateau?
I haven't seen that.
But maybe it it's possible, I don't know.
I don't think so, but maybe.
Anybody else?
OK. OK, so here's some photographs of honeycombs.
So these ones are white but these ones are just the same.
So most of them are just a regular hexagonal honeycomb and one of them is this funny shaped honeycomb here.
These two guys at the top, these two here, are unloaded.
And then the rest of them have some loading.
So if you look at this one down here-- so here I'm taking the honeycomb and I'm loading it.
This is the x1 direction, that way on.
And if you look very carefully, you can see what happens is these vertical walls just move sideways and that is going to zoot, with the sound effects.
And then the-- I can't help making sound effects-- and then these guys here bend.
OK, so this guy here-- it's maybe a little hard to see it on the image but it's actually a little bit bent there.
So that's loading it this way on.
And then similarly, if I take it and I load it that way on, that's the x2 direction.
And now these guys here, the vertical guys, are just going to compress a little.
But these guys here, the incline guys, are still going to bend a little bit.
And I've got some schematics that's going to show that a little bit better.
And then if I shear it, if I took it like this and I-- I can't do it because my hands aren't glued to the rubber-- but if I could sort of shear it this way on, then you would get this kind of deformation here and that also involves bending.
You can imagine these guys here would bend if I do that to them.
And then the buckling deformation looks like this.
If I scooch that like that, it should look kind of like that picture there.
OK?
So that's kind of what the deformation looks like for these sorts of honeycombs.
So these were elastomer rubbers.
These are just some images from an aluminum honeycomb.
So here's, the top one's the undeformed honeycomb, the middle one's loading it in the one direction from left to right, and the bottom one's loading it in the two direction from top and bottom like that.
OK?
So you can imagine that you've got little cell walls.
If you load it up high enough, those walls are going to yield and we're going to see that the yielding in the walls causes the formation of something called plastic hinges and the walls can rotate and they can produce these kind of shapes.
OK.
So then this is sort of a schematic stress strain curve here.
And this is showing what happens if you increase the thickness of the walls relative to the length, or you increase the relative density, you increase the volume per action of solids.
And this kind of shows the different regimes.
So over here-- well let's, first of all, start with the different relative density.
So this one here is the lowest relative density, then this is higher, and higher, and higher.
And not too surprisingly, the more you increase the density, the more material you've got, the stiffer it is, the stronger it's going to be.
And the more material you've got, the sooner it's going to densify.
If you've got more solid in here, you're going to reach that densification strain at a smaller number.
OK, so the shape of the curves looks like this.
And you can define these three kind of regimes.
So everything in here is linear elastic, everything in this big sort of envelope here is the plateau region, and everything up here is this densification region.
So this is just a bigger picture kind of plot.
OK, so let me skip through all of that.
All right.
OK, where are we?
Chalk?
Here we go.
OK, so I think I mentioned this before, but let me just go over it again.
So there's three types of things that affect the properties, the mechanical properties, of the honeycombs.
And probably the most important thing is the relative density of the honeycomb.
So remember, we said this was the same as the volume fraction of solids, and for a hexagonal honeycomb, you can show that's equal to the thickness to length ratio times h over l plus 2 divided by 2 cos theta times h over l plus sin theta.
And if you have regular hexagons, so h over l is equal to 1, all the sides are of equal length, theta is equal to 30 degrees, the relevant density is 2 over 3 times t over l. So it just goes linearly with t over l. The thickness to length ratio of the walls.
So it depends on how much solid you've got.
Depends on the solid properties.
So the Young's modulus of the solid, yield strength if it's a metal, some sort of fracture strength if it's brittle.
It also depends on the cell geometry, which we can describe with h over l and theta.
So if we think of a cell here-- that's our edge length h, that's our edge length l, that's our angle theta, here's the cell wall thickness t, and then we've got some set of solid properties here.
OK?
So that's kind of the set up.
And we're going to define x1, x2 axes like that.
And we're going to make a few assumptions just to make life a little bit simpler.
So we're going to assume that t over l is small, so that also means the relative density is small.
And what that means is that we're going to be able to neglect axial and shear deformations.
So you can imagine, if I have a thin wall and I'm applying loads that produce moments and produce bending, if the wall is very thin then the axial and the shear deformations are going to be small.
I'm also going to assume the deformations are small.
And what that means is that I'm going to neglect any changes in the geometry of the cell during the deformation.
And I'm going to assume that the cell wall is linear, elastic, and isotropic.
And we're going to start off with looking at in-plane behavior, and we're going to start with the elastic moduli.
And if we look at the elastic moduli, we're going to be talking about Hooke's law.
And Hooke's law and the elastic behavior of the material can be described by a set of elastic constants.
And if you recall, the number of independent elastic constants, how many constants you need to describe the material, depends on its symmetry.
And these materials are orthotropic.
So the regular hexagonal honeycomb is actually transversely isotropic, but imagine that h was not equal to l, then it would be orthotropic.
So remember, orthotropic means that you can rotate the structure 180 degrees about three mutually perpendicular axes and the structure looks the same.
So if I take this and I do that, it looks the same, and if I do that, it looks the same, if I-- no matter how I rotate this, about three mutually perpendicular axes, the structure remains unchanged.
So it's orthotropic.
So I'm going to write down Hooke's law for our orthotropic material, and then we'll talk about the constants that we're going to work out.
OK, so this is Hooke's law for our orthotropic material.
And let me just remind you what our notation is here.
So epsilon 1 is epsilon 1 1.
Epsilon 2 is epsilon 2 2.
Epsilon 3 is epsilon 3 3.
Epsilon 4 is gamma 2 3.
Epsilon 5 is gamma 1 3.
And epsilon 6 is gamma 1 2.
So these are the normal strains here, epsilon 1, 2, and 3, and these are the shear strains here, epsilon 4, 5, and 6.
And you remember this convention where the subscripts add up to 9.
So 4 plus 2 plus 3 is 9, 5 plus 1 plus 3 is 9, 6 plus 1 plus 2 is 9.
And then the stresses are a similar thing.
So sigma 1, sigma 2, and sigma 3 are the normal stresses.
And then sigma 4, sigma 5, and sigma 6 are the shear stresses.
And for the in-plane moduli, so we're dealing with the x1, x2 plane, there's four independent elastic constants.
So we could think of it as E1, E2, a Poisson's ratio 1 2 and a shear modulus in the 1 2 plane.
OK?
And the compliance matrix is symmetric, so there's the reciprocal relationship between the moduli and the Poisson's ratios.
And then the notation I'm going to use for Poisson's ratio, I'm going to say that mu i j is minus the ratio of the strain in the j direction divided by the strain in the i direction.
OK.
So what we're going to do next is we're going to calculate some of the elastic moduli.
I'm going to show you the derivation for E1 star and mu 1 2 star, and you can get the other two in a similar way.
So I'm not going to do all of them but next time we'll do the derivations for the Young's modulus in the Poisson's ratio.
OK?
And then we're going to talk about the out of plane direction later and we'll get the moduli for the out of plane direction as well.
OK?
So I think I'm going to stop there for today.
And then we'll start doing the derivations next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so it's five after.
We should probably start.
So last time we were talking about honeycombs, and I just wanted to quickly kind of review what we had talked about, and then today I'm going to start deriving equations for the mechanical properties of the honeycombs, OK?
So this is a slide of our honeycomb setup here.
These are the hexagonal cells we're going to look at.
We talked about the stress-strain behavior.
The curves on the left-hand side are for compression and the ones on the right-hand side are for tension.
And so what we're going to be doing today is we're going to start out by calculating a Young's modulus, this slope here.
We're going to calculate the stress plateaus for failure by elastic buckling in elastomeric honeycombs, by failure from plastic yielding in, say, a metal honeycomb, and by failure by a brittle crushing in, say, a ceramic honeycomb.
And if we have time, we'll get to the tension stuff.
I don't know if we'll get to that today or next time.
So we're going to start calculating those properties today.
And these were the deformation mechanisms.
Remember, we said the linear elastic behavior was related to bending of the cell walls, and then the plateau was related to buckling if it was an elastomer.
And the plateau was related to yielding if it was, say, a metal that had a yield point.
And then this was sort of an overview of the stress-strain curve showing those different regions, OK?
So what I'm going to talk about today to start is the linear elastic behavior.
And we're going to be starting with the in-plane behavior.
So in-plane means in the plane of the hexagonal cells.
And then next time we'll do the out-of-plane behavior, this way on.
So if I had to form my little honeycomb like this, what initially happens is the inclined cell walls bend.
So if you can see over here, we've kind of exaggerated it on this sketch.
So this wall here is bent.
This one here just kind of moves along, goes for the ride.
And this guy here is bent.
So this is for loading in the what we're calling the x1 direction, sigma 1.
And the same kind of thing happens when we load in the other direction, in the sigma 2 direction, these guys still bend.
Now the honeycomb gets wider that way.
It gets shorter this way, wider that way.
And we can calculate the Young's modulus if we can relate the load on the beam in the moments to this deflection here, right?
So the Young's modulus is going to be related to the stiffness and the stiffness is going to be related to how much deformation you get for a certain amount of load that you put on the beam.
So I'm going to calculate the modulus for the x1 direction, the thing on the left there.
And you can do the same thing for the x2 direction on the right, but I won't calculate that because it's exactly the same kind of process.
OK, so let me start here.
Get my chalk.
So I'm going to draw a one-unit cell here.
So here's my unit cell there, like that.
And this member here is of length h. That member there is of length l. That angle there is theta.
I'm going to say all the walls have equal thickness, and I'm going to call it t. And I'm going to define an x1 and x2 axis like this.
So the horizontal is x1 and the vertical axis is x2.
And I'm going to say that I apply a sort of global stress to it, sigma 1.
So there's a stress in the one direction there, sigma 1, OK?
And I'm going to say my honeycomb has a depth b into the page, but the depth s-- is because the honeycomb's prismatic, the b's are always going to cancel out of all the equations that we're going to get, because everything's uniform in that direction.
And we can think about a unit cell here, and in the x1 direction, we could say the length of our unit cell is 2l cos theta.
So in the x1 direction, that's our unit cell there, and that's 2l cos of theta.
And in the x2 direction, you might think that you go from this vertex up here down to that vertex there, but if you did that, then on the next layer of cells, you wouldn't have the same distance.
So the unit length in the x2 direction is actually from here to here, and then you can see the next cell, you would get the same thing.
You get this bit here from the inclined member, and then you would get h down here from the next member.
So this bit here is equal to h plus l times sine of theta.
So I can say in the x2 direction, the length of the unit cell is h plus l sine theta, OK?
So that's kind of the setup.
And then what we want to look at is that inclined member that bends, we want to look at how this guy bends under the load.
And if we can relate the forces on it to the deflection, and we need the component of the deflection in the one direction, then we're going to be able to get the modulus.
So I'm going to draw that inclined member again over here, and it's going to see some loads that I'm going to call p, and that's going to cause this thing to bend.
So I've kind of exaggerated it there, but there's the bending.
And there's some end deflection there, delta.
And there's moments at either end of the beam, or either end of that member, as well.
And this member here has a length.
That length is l. OK, are we good?
So it's just kind of the setup.
And I'm going to draw the deflection delta bigger over here.
So say that's delta.
That's the same parallel as this guy here.
What I'm going to want is the deflection in the x1 direction, and when I come to calculate the Poisson's ratio, I'm going to want the deflection in the x2 direction.
And if this angle here is theta, between the horizontal and the inclined member, then this angle up here is also theta, and so this bit here is delta sine theta.
And this bit here is delta cos theta.
Ba-doop-ba-doop-ba-doop.
So the Young's modulus is going to be the stress in the one direction divided by the strain in the one direction.
So I need to get the stress and the strain in the one direction.
So here the stress in the one direction.
If I'm applying my load, like this, sigma 1, the stress in the one direction is going to be this load p-- so this load, say, p on this member here, divided by this length here, the unit cell length, and then divided by b into the board, the width end of the board.
So sigma 1 is going to be p divided by h plus l sine theta times b.
And epsilon 1 is going to be the strain in the one direction, is going to be the deformation in the one direction divided by the unit cell length in the one direction.
So that's going to be delta sine theta divided by l cos theta.
So here, even though I've said this unit cell is 2l cos theta, there'd be two of the members here that would be twice that deflection in the one direction.
So it's, for one beam, it's delta sine theta over l cos theta.
Are we OK so far?
So far so good?
So for the hexagons, because we're going to figure out the equations more or less exactly, we're going to keep track of all the geometrical factors.
When we come to the foams, we're not going to keep track of all the geometrical factors.
So one of the things that makes us look a little kind of hairy is just the fact that we're keeping track of all these sines and cosines and all the dimensions and things.
All right, whoop.
So I need to be able to relate my load p to my deformation delta to get a stiffness out of this, to get a modulus, OK?
So the way I do that is, remember in 3032, we did those bending moment diagrams and we did the deflection of the beams?
This is where this comes in handy.
So I'm going to draw my beam a little bit differently now.
I'm going to turn it on its side.
So this is still my length l, but I'm going to turn on my side just so that you can see it the same kind of way we did the bending moment diagrams.
So this is still my length l across here.
And there's end moments, M and M here.
And P sin theta is just the perpendicular component of the load p. So p sin theta is just the component perpendicular to my beam.
So I could draw a shear diagram here and I could draw a bending moment diagram here.
And, if you remember, the shear diagram, if I have no concentrated load along here, and I have no distributed load along here, if this is zero, down here, it's just going to go up my P sin theta, and then be horizontal, and then come down by P sin theta, OK?
So that's the shear diagram.
And then the bending moment diagram, I'm going to draw down here.
So I've got some moment at the end here, and this would tend to bend like that.
So this would be a negative moment.
Remember, bending moments were negative if there was tension on the top, and they were positive if there was tension on the bottom.
So over here we'd have tension on the top, so that would give us a negative bending moment.
And then, if you also remember, the moment at a particular point is equal to the integral from, say, A to B-- well maybe I should write this another way.
And B minus Ma is the integral of the shear diagram between the two points.
A little sloppy.
OK.
So if I know how I have some moment here minus M, if I integrate this shear diagram up, then this is just going to be linear here, and then I'm going to be at plus M over there.
So if you look at this shear and bending moment diagram, it's really just the same as the shear and bending moment diagram for two cantilevers that are attached to each other.
So let me just draw over here what the cantilever looks like.
Let's see.
So imagine I just had a cantilever like this, and I have some force F on it like that.
And I call this distance here capital L, and I'm going to call that deflection capital delta, like that.
If I drew the shear diagram for that, there'd be a reaction here, F, there would be a moment here, FL.
Doot, doot, yup.
So this would look-- whoops-- it's a little too long.
Shear diagram here would look like this.
That would be zero.
This would be FL.
And the moment diagram would look like this.
Whoops.
A little too long again.
And that would be minus FL.
And that would be zero.
So do you see how the shear and the bending moment diagram here are really just like two cantilevers, OK?
So I know that the deflection for a cantilever, delta, is equal to F capital L cubed over 3EI.
It's kind of a standard result.
And so I can take this and apply that to this beam here.
So instead of working everything out from first principles, I'm just going to say that my beam here is like two cantilevers, and instead of F, I've got P sin theta.
And instead of capital L here as the length, I've got l/2 because l/2 would be the length of one of the cantilevers.
OK, so for the honeycomb, I've got two cantilevers of length l/2.
So delta for the inclined member on the honeycomb is going to be 2-- because I've got two cantilevers-- the force, instead of having F, I'm going to have P sin theta.
And instead of having capital L, I'm going to have l/2, all cubed.
So this is like F capital L cubed over 3.
And here, the modulus that I want is the modulus of the solid cell wall material, so I'm going to call that ES, and over the moment of inertia.
So you see how I've done it?
Is that OK?
So then I can just kind of simplify this thing here.
I've got P sin theta l cubed.
1/2 cubed is going to be 1/8.
So this is going to be 2, if that's 1/8, times 3 is 24.
So 2/24 is 12.
So I've got delta for my honeycomb member is P sin theta l cubed over 12 EsI.
And here I is the moment of inertia of that inclined member of the honeycomb.
And that's BT cubed over 12, OK?
So B is the depth into the board, and T is the thickness.
We'll cube that and divide by 12.
It's a rectangular section.
Yeah?
What was ES again?
ES is the Young's modulus of the solid that it's made from.
So clearly, if my honeycomb is made up of these members, whatever material the members are made of is going to affect the stiffness of the whole thing.
Are we good?
Because once we have this part, then we just combine these equations for the stress and the strain in the one direction.
And we have this equation relating delta and P, and we're going to be able to get our Young's modulus, OK?
We're happy?
OK. All right.
So I'm going to call the Young's modulus in the one direction E star 1.
So everything with a star refers to a cellular solid property, and 1 because it's in one direction.
So that's going to be sigma 1 over epsilon 1.
So if I go back up there, I can say sigma 1 is equal to P divided by h plus l sin theta b.
And epsilon 1 is equal to delta sine theta in the denominator over l cos theta.
And now instead of having delta here, I can substitute this thing here in for delta.
And then I'm going to able to cancel the P's out.
So delta was equal to P sine theta l cubed over 12 Es, and there was an I, a moment of inertia, and I was equal to bt cubed over 12.
And let's see here.
So that's delta.
And there's another sine theta here so I'm just going to square that sine theta there.
So now the P's cancel out.
The b's are going to cancel out.
The 12s are going to cancel out.
And I'm going to rearrange this a little bit.
So I'm going to write Young's modulus of the solid out in the front.
Then I've got a term here of t cubed and I'm going to multiply that by 1/l squared, and then everything else-- well, let's see.
We can take this l cubed here.
I can take that.
Put it underneath that, so that's going to give me t/l cubed.
And then I've got an h plus l sine theta here, and I've got an l there, so I'm going to take that to be h/l plus sine theta.
Boop-boop-da-doop.
So I've got this term with [?
h/l's ?]
in the thetas.
There's a cos theta from the numerator here.
This term here turns into h/l plus sine theta, and then I've got my sine squared thetas down there.
And that's my result for the Young's modulus in the one direction.
OK?
Let's make sure that seems right.
It seems good.
OK.
So one of the things to notice here is there's three types of parameters that are important.
So one is the solid properties.
So the Young's modulus of the solid comes into this.
So the stiffness of the whole thing depends on the stiffness of whatever it's made from.
There's this factor of t/l cubed-- that's directly related to the relative density or the volume fraction of solids.
So what this is saying is the relative density goes as t/l, so the Young's modulus depends on the cube of the relative density.
So it's very sensitive to the relative density.
And then this factor here really is just a factor that depends on the cell geometry.
Remember when we talked about the structure of the honeycomb, we said we could define the cell geometry by the ratio of h/l and theta, OK?
And since we often deal with regular hexagonal honeycombs, I'm just going to write down what this works out to be for regular hexagonal honeycombs.
So for a regular hexagonal honeycomb, h/l is 1.
All the members have the same length.
And theta's 3, and the modulus works out to 4 over root 3 times Es times t/l cubed, OK?
So do you see how we do these things?
So all the other properties work in a similar kind of way.
You have to say something about what the sort of bulk stress is on the whole thing and relate that to the loads on the members.
You have to say something about how the loads are related to deflections, or when we look at the strengths, we're going to look at moments and how the moments are related to failure moments of one sort or another.
But it's all just like a little structural analysis, OK?
Are we good?
You good, Teddy?
I thought you were going to put your hand up?
No?
You're OK?
OK. OK, so the next property we're going to look at is Poisson's ratio.
And I'm going to look at it for loading in the one direction.
So Poisson's 1 2, say we load uniaxially in the one direction, we want to know what the strain is in the two direction, it's minus epsilon 2 over epsilon 1.
And again, if I look at my inclined member, and I say that member's going to bend something like that, and that's my deflection delta there, and, say, got the same x1 and x2 axes.
And again, if I look at delta here, it's the same little sketch I had before.
That's delta sine theta.
And this is delta cos theta.
I'm going to need those components to get the two strains in the different directions.
So epsilon 1 is going to be delta sine theta over l cos theta.
And if I'm compressing it, that would get shorter.
And we get-- and epsilon 2 is going to be delta cos theta divided by h plus l sine theta.
And that would get longer.
So these two have opposite signs, and so the minus sign is going to disappear here.
Doodle-doodle-doot.
So then I can get my Poisson's ratio by just taking the ratio of those two guys.
So I could put a minus sign there and say that's the opposite sign to epsilon 2.
Then this would be delta cos theta divided by h plus l sine theta.
And epsilon 1 would be delta sine theta over l cos theta.
And the thing that's convenient here is that the two deltas just cancel out.
So the Poisson's ratio is the ratio of two strains.
Each one of the strains is going to be proportional to delta, and so the two deltas are just going to cancel out.
And so I can rewrite this thing here as cos squared theta divided by h/l plus sine theta times sine theta.
And so one of the interesting things to notice that the Poisson's ratio only depends on the cell geometry.
It doesn't depend on what solid the material is made from.
It doesn't depend on the relative density.
It only depends on the cell geometry.
Oops.
OK, and then we can also work out what the value is for a regular hexagonal cell.
And if we plug-in h is equal to l and theta's equal to 30, you get that it's equal to 1.
So one is kind of an unusual number for a Poisson's ratio.
When we think of most materials, it's around 0.3, so it's kind of unusual that it's that large.
The other thing that's interesting is that it can be negative.
So if theta is less than 0, then you can get a negative value.
If the cos squared is going to be a positive value, but you've got a sine theta down here, then that's going to give you a negative value.
So you can get negative values.
So let me just plug in an example.
So say h/l is equal to 2, and theta is equal to minus 30 degrees, then this turns out to be 3/4.
So cos of 30 is root 3/2, so square of that is 3/4.
h/l is 2, sine theta is 1/2, but it's minus 1/2.
So 2 minus 1/2 is 1 and 1/2.
And then the sine theta is minus 1/2.
And so it works out to be minus 1 for that particular combination.
And I brought my little honeycomb that has a negative Poisson's ratio in.
So this guy here-- let's see, I don't think there's an overhead here.
No overhead?
Guess not.
I'll just pass it around.
So if you take it, put your hands on the flat side and load it like this, and don't smoosh it like that.
Just load it a little bit, because you [?
want to be ?]
linear elastic.
If you load it just a little bit, you can see that as you push it this way, it contracts in sideways that way.
So don't smash it.
Just load it a little bit and you can kind of see it with your hands.
And if you put on a piece of lined paper, it's easier to see it.
OK, so that's kind of interesting.
So are we good with getting the Young's modulus in the one direction and the Poisson's ratio for loading in one direction?
OK, so you can do the same sort of thing to get the Young's modulus in the two direction and the Poisson's ratio for loading in the two direction.
And you get slightly different formulas, but it's the same idea.
And you can also get a shear modulus this way, and in-plane shear modulus.
It's a little bit-- the geometry of it's a little bit more complicated.
So all of those things are derived in the book, in the cellular solids book.
So if you wanted to figure those out, look at that, you could look at the book.
So let me just comment on that.
All right, so those are the in-plane linear elastic moduli, and remember we said that four of them describe the in-plane properties for an anisotropic honeycomb.
And you can use that reciprocal relationship to relate the two Young's moduli and the two Poisson's ratios.
All right, so the next thing I wanted to talk about was the compressive strength.
So let me just back up here a second.
So if we go back to here, remember we had for an elastomeric honeycomb, this stress plateau was related to elastic buckling.
So we're going to look at that buckling stress first.
And this plateau here is related to yielding.
And then we'll look at the yielding stress next.
And then this plateau here is related to a brittle sort of crushing, and we'll do that one third.
So we're going to go through each of those next.
And this is kind of a schematic for the elastic buckling.
So when you look at the elastic buckling, one of the things to note is that when you load the honeycomb this way on, if you load it in the one direction, you don't get buckling.
It just sort of continues to-- whoops, if I can keep it in plane.
It all just kind of folds up, so you just get larger and larger bending deflections.
You don't really get buckling.
But when you load it this way on, these vertical members here, the ones of length h, they're going to buckle.
So, see if I do that, my honeycomb looks like those cells up on the schematic there, OK?
So, whoops.
So we're going to look at the compressive stress or strength next.
That's sometimes called the plateau stress.
So we can get cell collapse by elastic buckling, if, for instance, the honeycomb is made of a polymer.
And then the stress-strain curve looks something like that.
And what happens is you get buckling of those vertical struts throughout the honeycomb And then you could also get a stress plateau by plastic yielding.
And what happens when you get plastic yielding is you get localization of the deformation.
So one band of cells will begin to yield initially, and then as the deformation proceeds, that deformation ban will propagate and get bigger and bigger, and you get a wider and wider band of cells yielding and failing.
So you get localization of yield, and then as deformation progresses, the deformation band widens throughout the material.
So if I go back and look-- if I look at this one here, when you look at this middle picture here, you can see how one band of cells has started to collapse and started to fail.
And as you continue to compress that in the one direction, this way on, then more and more neighboring cells are going to collapse and the whole thing will get wider until the whole thing has collapsed.
And that's kind of characteristic of the plastic failure.
And then the third possibility is brittle crushing.
And then you get these kind of serrated plateau.
And the peaks and valleys correspond to fractures of individual cell walls.
OK, so we're going to start off with the elastic buckling failure.
And I'm going to call these plateau stresses sigma star, for the sort of compressive strength.
And el means it's by elastic buckling.
And as I mentioned, you don't get it in the one direction.
The cells just fold up.
You only get it for loading in the two direction, so it's going to be sigma star el 2.
Oops, need a different piece of chalk.
So you get this elastic buckling for loading in the x2 direction, and the cell walls of length h buckle.
And you don't get it for loading in the one direction, the cells just fold up.
So again, let me draw a little kind of unit cell here.
And here is our stress sigma 2, like that.
And here's our little wall of length h that's going to buckle.
So if I load it up, initially it'll be linear elastic.
And then eventually, at some stress, it will get large enough that this wall here will buckle.
And we can relate to that plateau stress, or that compressive stress, to some Euler buckling load.
So you remember, if we have a pin-ended column, so just a single column, pins on either end, the Euler buckling load says you get buckling when the critical load is equal to some end constraint factor, n squared.
So n squared pi squared E, and here it's E of the solid, I over the length of the column, and in this case, the column length is h-- so h squared.
OK, so that's just the Euler formula.
And here, n is an end constraint factor.
And if you remember for a pin column, so if our column is pinned at both ends like that, and just buckles out like that, then n is equal to 1.
And if the column is fixed at both ends, something like that, then the column looks like that and then it's equal to 2, OK?
So if I know what the end condition is, I know what n is and I can use my Euler formula here.
So the trick to this is that it's not so obvious what n is.
Yes?
So, when you're loading in the x2 direction here, the first thing you're going to get is the incline members deforming?
Yeah
And then at some point, you hit a P critical that will cause the vertical members to buckle?
Exactly

Exactly.
That's exactly right.
Hello.
So the trick here is that we don't really know what this n is, initially.
They're not really pinned, pinned; they're not fixed, fixed.
And if you think about the setup with the honeycomb here, the constraint on that vertical member depends on how stiff the adjacent members are.
So you can kind of imagine, if I'm looking at one of these vertical members here, if these two adjacent inclined members were big honking thick things, it would be more constrained.
And if they were little thin, kind of teeny little membranes, it would be less constrained.
And you can think of it in terms of a rotational stiffness, that when the honeycomb buckles, you kind of see the member h goes from being horizontal to sort of it buckles over like this.
But that whole end joint, see the end joint at the top here or the end joint at the bottom, that whole joint rotates a little bit.
And so there's some rotational stiffness of that joint.
And that rotational stiffness depends on how stiff the member h is and how stiff those inclined members are.
So there's a thing called the elastic line analysis that you can use to calculate what n is.
And basically what that does is it matches the rotational stiffness of the column h with the rotational stiffness of those inclined members.
So we're not going to get into that.
I'm just going to tell you what the answer is.
But if you want to go through it, it's in an appendix in the book.
So you can look at it, if you want.
So here I'm just going to say that the constraint n depends on the stiffness of the adjacent inclined members.
And we can find that by something called the elastic line analysis.
And if you have the book, you can look in the appendix and see how that works.
But essentially what it does is it matches the rotational stiffness of the column h with the rotational stiffness of the inclined members.
So what you find is that n depends on the ratio of h/l.
And I'm just going to give you a table with a few values.
So for h/l equal to 1, then n is equal to 0.686.
For h equal to 1.5, it's equal to 0.76.
And for h/l equal to 2, it's equal to 0.806.
OK, so now if we have values for n, we can just substitute in to get the critical buckling load.
And if I take that load and divide it by the area of the unit cell, I'm going to get my buckling stress.
So it's pretty straightforward from this.
So my buckling stress is going to be that critical load divided by my unit cell area.
So it's divided by the unit cell length in the x1 direction to l cos theta times the depth b into the page.
So it's equal to n squared pi squared Es times I.
And I is bt cubed over 12.
Divided by the length of the column, h squared, and then divided by the area of the unit cell, 2l cos theta b, OK?
And I can rearrange that somewhat to put it in terms of dimensionless groups.
So if I pull all the constants out, it's n squared pi squared over 24 times the modulus of the solid, t/l cubed in the numerator divided by h/l squared times cos theta in the denominator.
So again, you can see that the buckling stress, the compressive sort of elastic collapse stress, depends on the solid property.
So here is the modulus of the cell wall in here.
Depends on the relative density through t/l cubed.
And then it depends on the cell geometry through h/l cos theta, and n depends on h/l as well, OK?
And then we can do the same thing where we figure out what it is for regular hexagonal cells.
And it's 0.22 Es times t/l cubed.
And then we can also notice that since E in the 2 direction, for a regular hexagonal cell, E is the same in the 2 direction and the 1 direction.
It's isotropic.
So E2 is also equal to 4 over root 3 Es times t/l cubed.
That's equal to-- whoops-- it's equals to 2.31 Es t/l cubed.
And we can say that the strain at which that buckling happens is just equal to a constant.
And for regular hexagonal honeycombs, it works out to a strain of 10%.
Are we good?
So we have a buckling load.
We divide by the area.
The only complicated thing is finding n. And you can find it by this elastic line analysis thing.
So each of these calculations is like a little structural analysis, only on a little teeny weeny scale of the cells.
So you see where my background in civil engineering comes in handy.
Yup.
OK.
So the honeycombs involve the most sort of complicated equations.
When we come to do the foams, we're going to use a dimensional analysis and all the equations are going to be much simpler.
So this is the most kind of tedious part of the whole thing.
So the next property I want to look at is the plastic collapse stress.
Say we had a metal honeycomb and we wanted to calculate the stress plateau for a metal honeycomb.
So we have this little schematic here, and say we load it in the one direction again.
So we're loading it here.
And we've got some load P, like that.
And if we have our honeycomb, we load it this way on, initially, the cell walls bend.
And you have linear elasticity and you have some Young's modulus.
But if you have a metal, if you continue to deform it and you continue to load it more and more, eventually you're going to hit the yield stress and the cell wall.
So the stresses in the cell wall are going to hit the yield stress.
And initially, the stresses are just going to be-- remember, if you have a beam, the stresses are maximum at the top and the bottom of the beam.
So initially you're going to hit the yield stress at the top and the bottom of the beam first.
But as you continue to load it, you're going to end up yielding the cross-section through the entire section.
So the entire section is going to be yielded.
And once the entire section yields, it forms what's called a plastic hinge.
Once the whole thing's yielded, then you can add more force and the thing just rotates.
And because it rotates, it's called a plastic hinge.
You know, if you take a coat hanger, and you bend it back and forth and bend it back and forth.
If you bend it enough, you form a plastic hinge because it just can bend easily.
So these little schematics here, if you look at the, say, one of these inclined members, the moments are maximum at the end.
So Remember when we had the linear elastic deformation and I looked at the little bending moment diagram?
The moments are maximum at the ends, and you're going to form those plastic hinges initially at the ends.
And so these little ellipsey things here, all the ends, those kind of show where the plastic hinges are.
So those plastic hinges are forming.
So here's for loading in the x1 direction, and here's for loading in the x2 direction, there.
So the thing we want to calculate is what stress does it take to form those plastic hinges and get this kind of plastic plateau stress?
OK, so we can say we get failure by yielding in the cell walls.
And I'm going to say the yield strength of the cell wall is sigma ys.
So sigma y for yield and s for the solid.
And the plastic hinge forms when the cross-section has fully yielded.
So let's look at the stress distribution through the cross-section when its first linear elastic.
So say that's the thickness t of the member.
And if the beam was linear elastic, the stress would just [?
vary ?]
linearly, like that, right?
And this would be the neutral axis, here, where there is no normal stress.
So that's what happens if it's linear elastic, and I'm hoping you remember something vaguely like that.
Sounds good?
But as we increase the load on it, and we increase the sort of external stress, this stress in the member is going to get bigger and bigger, and eventually, that's going to reach the yield stress, OK?
And once that reaches the yield stress, if we continue to load it, what happens is the yielding propagates down through the thickness of the thing here.
So we get yielding through the whole cross-section.
So let me scoot over here
Professor?
Yup?
When it starts to yield, does this curve change?
Yes.
I'm going to draw it for you
Oh,
That's the next step.
That would be the next thing.
OK, so once the stress at the outer fiber is the yield strength of the solid, then the yielding begins and it progresses through the section as the load increases.
So the stress distribution starts to look something like this once it yields.
OK, so that's sigma y of the solid.
Actually, let me rub that out because then I can show you something else.
So in 3D, this would be through the thickness of the beam.
That would be the thickness of the beam there.
And boop, boop.
It would look something like that.
OK?
And then this is still our neutral axis here.
And then eventually, as you load it more and more, the whole cross-section is going to yield.
Whoops.
And I'm assuming that the material is elastic, perfectly plastic.
So the stress-strain curve from the solid I'm idealizing as-- whoops.
That's not quite right.
I'm idealizing as that, OK?
So when you get to this point here, the entire cross-section has yielded, and that means you form the plastic hinge.
The idea here is that the section then just rotates like a pin.
All right.
So we can figure out the plateau stress that corresponds to this by looking at the moment that's associated with the plastic hinge formation.
So there's some internal moment associated with that.
And then equating that to the applied moment from the applied stress.
So doodle-loodle-oot.
Let me see me, maybe back up here.
So there's some-- if I have the stress distribution here, I could say this whole kind of stress block is equivalent to some force acting out like that and some force acting out like that.
It would be sigma ys times b comes t/2 would be f. And I can say there's some plastic moment.
If I think of the force here and the force there, they act as a couple and they have some moment, and that's called the plastic moment.
So that's like an internal moment when the plastic hinge forms.
So I'll say the internal moment at the formation of a plastic hinge.
I'm going to call that Mp, for plastic moment.
And we can work out Mp by looking at that stress distribution when the entire cross section has yielded.
The force F is going to be sigma ys times b comes t/2.
It's the stress times that area.
And then the moment arm between the two forces is also t/2.
And so that plastic moment is just sigma ys bt squared over 4, OK?
Are we good?
Sonya?
What's the second [INAUDIBLE]?
OK, so this is the force.
This thing here is the force F. And I have to-- if I'm getting a moment, I'm saying that that force, if I doot-doot-doot-- the distance between those two forces there is t/2.
So each force acts through the middle of the block, and so the distance between [?
it is ?]
t/2.
And I'm going to equate that moment to the applied moment from the sort of applied stress.
And then if I go back to my inclined member-- whoops, let's see.
Let me get a little more inclined.
That's my inclined member, there, of length l. I've got modes p that are applied at the end from sigma 1.
And I've got moments that are induced at the ends.
And that angle there would be theta.
This length here is l, like that.
And if I just use static equilibrium on that, I can say that I've got 2 times the moment, so I've got one at each end-- they're both the same sign-- minus P. And then the distance between these two P's, say I take moments about here, I've got M applied plus M applied, I've got minus P times l sine theta.
That's equal to 0.
So the applied moment there is just Pl sine theta over 2.
So now what I'm going to do is I'm going to equate this applied moment with this plastic moment, and I'm going to relate P to my applied stress sigma 1.
And then I'm going to get a strength in terms of the yield strength of the solid, there's going to be a t/l factor and there's going to be some geometrical factor.
So that's just the last step.
Boop-ba-doop-ba-doop.
So we get plastic collapse of the honeycomb.
And the stress I'm going to call sigma star plastic with a 1, because I'm going to look at the one direction.
And that happens when that internal plastic moment equals the applied moment.
So let's see.
I've got that.
Let me also write down over here, I've also got this sigma 1 is equal to P over h plus l sine theta times b.
So here I can write P in terms of sigma 1, in this thing.
And then write that, get the applied moment in terms of that, and then equate it to that.
So this term on the left-hand side corresponds to this expression for the applied moment where I've plugged in.
For P, I've plugged in sigma 1 times h plus l sine theta times b.
And that's my plastic moment on the right-hand side.
So if I just rearrange this, I can then solve for this plastic collapse stress.
So it's equal to the yield strength of the solid times t/l squared, and then times another geometrical factor.
2 times h/l plus sine theta times sine theta.
Doop-doop-doop.
So the same kind of thing, there's a solid property, a t/l, a relative density term, in then a cell geometry term.
And we can calculate with this for regular hexagonal cells.
And we can do a similar kind of calculation for loading in the other direction.
And you can get a shear strength if you want to do that, too
If you're going in the other direction, only the E or the M apply changes, right?
[?
Or ?]
like that section
Yeah, this thing here is the same
That stays
Right.
And this is-- there's a different geometry to it.
Because now you're loading it this way on.
OK, so we've calculated an elastic buckling plateau stress and a sort of plastic collapse plateau stress.
And if you have thin enough walled, say, even aluminum honeycombs, then the elastic buckling could precede the plastic collapse.
And so I'm just going to work out what the criterion would be for that to happen.
So the two stresses can be equated.
And then that's going to give us some criterion.
So the two are equal, I'm just going to write down the equations that we had.
So the buckling stress was n squared pi squared over 24 times E of the solid times t/l cubed divided by h/l squared times cos of theta.
And the plastic collapse stress for the 2 direction was sigma ys times t/l squared divided by 2 cos squared theta.
So I can write this-- because this has a t/l cubed term, and that has a t/l squared term, I can write this in terms of a t/l critical.
So if I leave it t/l here and I put everything else on the other side, I've got 12 over n squared pi squared, then h/l squared over cos theta times sigma ys over Es.
So if t/l is less than that, I'm going to get elastic buckling first.
And if it's more than that, I'm going to get plastic yielding first.
And we can work out an exact number for regular hexagonal honeycombs, so I'm going to do that.
So if I have a particular geometry, I can figure out what n is.
So for regular hexagonal honeycombs, t/l critical just works out to 3 times the yield strength of the solid over the Young's modulus of the solid.
So if we know that ratio of the yield strength of the modulus of the solid, we can get some idea of what that critical t/l would be.
So we'll do that next
And you said if t/l is less than that critical, then you're going to get the yielding first
No, if it's less, you get the buckling first.
If it's really skinny, it tends to buckle first.
So, for example, for metals, the yield strength over the modulus is roughly 0.002, like the 0.2% yield strength.
And so that means that t/l, the sort of transition or the critical value is at 0.6%.
So most metal honeycombs are denser than that.
That's a pretty low density.
But if we look at polymers, you can get polymers with a yield strength relative to the modulus of about 3% to 5%, and then that critical t/l is equal to about 10%, 15%.
So low-density polymers with yield points may buckle before they yield.
So we have one more of these compressive plateau stresses, and that's for the brittle honeycomb.
So I don't think I'm going to finish this today, but let me set it up and then we'll finish it next time.
So the idea here is that if you have a ceramic honeycomb-- remember I showed you some of those ceramic honeycombs-- that if you compress them, they can fail by a brittle crushing mode.
So ceramic honeycombs can fail in a brittle manner.
And again, initially there would be some cell wall bending, but at some point, you're going to reach the bending strength of the material.
And bending strengths are called modulus of rupture.
So you reach the modulus of rupture of the cell wall.
So I'm not going to write the equations down today because we're not going to get very far, so I'll do that next time.
But we're going to set this up exactly the same as we did for the last one, for the plastic yielding.
But instead of getting that sort of blocky, fully yielded cross-section stress distribution, we're just going to have the linear elastic stress distribution, and when the maximum stress reaches that modulus [?
rupture, ?]
the thing's going to fail.
So the form of the equations is going to be very similar to what we had for the plastic collapse stress, but there's a slightly different geometrical factor-- that's all.
So we'll do that next time.
And then next time we're also going to talk about the tensile behavior of honeycombs in-plane.
We'll work out a fracture toughness and then we'll start talking about the out-of-plane properties, as well.
So on Wednesday, we'll do the out-of-plane properties, OK?
So hopefully we'll finish the out-of-plane properties Wednesday.
And then next week, I was going to talk about some natural materials that have honeycomb-like structures, so things like wood and cork, OK?
All right, so this is the kind of most equationy lecture in the whole course.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
I should probably start.
Last time, we were talking about the honeycombs and doing some modeling of the mechanical behavior and we started off talking about the in plane behavior.
We're talking about loading it in this direction or that direction there.
And we talked about the elastic modulus.
I think I derived a Young's modulus for the one direction, a Poisson's ratio for loading in the one direction.
And then we started talking about the stress plateau and we went over the elastic buckling stress, for one of these elastomeric honeycombs like this.
And we went through the plastic collapse stress, for, say, a metal honeycomb that would yield.
And I think I started talking about a brittle honeycomb and brittle crushing.
The idea with a brittle honeycomb-- like a ceramic honeycomb-- is it could fail in a brittle manner.
And the failure is going to be controlled by the cell wall in bending.
And when that bending stress reaches the modulus of rupture, or the bending strength of the material, then you get wall fracture.
I think that's where we left it last time, right?
I had written down something about cell wall fracture.
Now, I wanted to do the little derivation.
Here's our little schematic up here.
Here's the honeycomb.
You've loaded it with sigma 1 here to such an extent that one of these cell walls has reached the modulus of rupture and has broken.
And this is the little free body diagram that corresponds.
I'm going to go through sigma 1 for loading in the one direction.
This is the same thing for loading in the two direction.
And the result for that's in the book.
OK.
If I have loading in the one direction, I can relate that horizontal force p to the stress in the one direction.
The little p is equal to sigma 1 times h plus sin theta times b.
And remember, b's the depth into the page.
And I'm going to define sigma fs as the modulus of rupture of the cell wall material.
It's the bending strength of the cell wall material.
And we're going to say that we get fracture of that bent wall when the applied moment is equal to the fracture moment.
From the plastic collapse stress from last time, we had the applied moment was equal to p times l sin theta over 2.
That was just using static equilibrium, looking at that free body diagram of the beam.
And if I write p, in terms of sigma 1 up here, I can just write that like this sigma 1 times h plus l sin theta times b.
And then I've got this other term of l sin theta and we divide that whole thing by 2.
That's the applied moment.
And we're going to get fracture when we reach the fracture moment.
I'm going to call that mf-- the moment at fracture.
Last time, we figured out a plastic moment to form a plastic hinge.
And this is an analogous thing.
But in this case, remember, if we have a beam and we have the stress profile through the cross section of the beam, it's going to look something like that.
So for our beam, that's going to be the thickness of the beam there.
So if it's linear elastic, we get the maximum stress at the top and the bottom.
And the neutral axis is here in the middle.
There's no stress there.
This is the normal stress distribution here.
And as we increase the stress for a brittle material that's going to be linear elastic till fracture, this is going to stay linear like this until we reach this modulus of rupture stress here.
When we reach that stress, then we're going to get fracture of the beam.
And we can say that there's some moment associated with that.
I could say that this stress block here is equivalent to some concentrated force and this stress block down here is also equivalent to the-- it's going to the same magnitude, but the opposite direction force.
And I can get the fracture moment by figuring out how big those forces are and multiplying by this moment arm between the two forces.
OK?
The magnitude of those forces is just going to be the volume, essentially, of this stress block here.
Imagine there's stresses there.
It's a triangle, so the area of it's going to be a half times t over 2 times of sigma fs.
And it's going to go b into the page.
So if you think of the force-- if this was the stress-- if that stress was constant, it would just be sigma fs times b times t. But it's not constant.
It's a linear relationship.
So I'm taking the area of that triangle.
That's the force.
And then I want to multiply that times the moment arm.
And the moment arm between those two forces-- each of these forces acts through the centroid of the area.
The centroid of the area is not in the middle for a triangle, and that total distance is 2/3 of the thickness, t. OK?
That's the moment arm that you get by figuring out where the centroid of these areas are.
I multiply that times 2/3 t and one of the 2's is going to cancel.
I can rewrite that and sigma fs times b times t squared over 6.
OK?
I think, last time when we talked about the plastic moment, we did a similar calculation and it worked out to sigma y bt squared over 4.
So now this is sigma fs, the modulus of rupture, times bt squred over 6.
The 6 is just slightly different because we've got a triangle here instead of a square shape like we had before.
And now I can get the brittle crushing strength and compression by just equating that applied moment to this fracture moment.
And if you do that, the result you get is this plateau stress for brittle crushing and compression.
In the one direction, it's sigma fs-- the modulus of rupture of the cell wall-- times t over l squared.
And then divided by a geometrical factor.
And her regular hexagons, it works out to 4/9 of the modulus of rupture times t over l squared.
OK?
Are we good?
We've got the in plane compressive properties now.
We've got the elastic moduli and we've got the three plateau stresses that correspond to the three mechanisms-- to the elastic buckling failure mechanism, the plastic yielding mechanism, and then the fracture mechanism for brittle crushing.
OK?
If you think of the stress-strain curve of these materials in compression, the stress strain curves all look something like that.
And now we've figured out equations that give us the modulus here and our collapse stress there.
OK?
So we can describe that stress-strain curve.
All right.
That's compression.
And the next thing I wanted to talk about is tension.
And if we think about the tensile behavior, the elastic moduli are going to be just the same.
So the moduli are the same in tension and compression.
And then, if we think about the stress plateau, we don't really have a stress plateau for an elastomeric material because there's no elastic buckling.
If you pull it in tension, you're not going to get buckling in tension.
You only get buckling if it's in compression.
If you have a material that yields like a metal, you can get a plastic collapse stress and a plastic plateau.
And that's very similar in tension and compression.
There's a very small geometrical difference, but you can, essentially, ignore it.
If you're loading the material in compression-- and imagine this was a metal-- if you load it in compression, the cell walls are getting a little further apart when I compress it.
And if you're loading in tension, like this, the cell walls are getting a little closer together.
So there's a small geometrical difference.
But if we ignore that, we can say that the plateau stress for plastic behavior is about the same in tension and compression.
And so really, the only property that's left is to look at a brittle honeycomb.
And for a brittle honeycomb, you can have fast fracture and we can calculate a fracture toughness.
So this next slide describes the fracture toughness calculation that we're going to do.
Here's our honeycomb.
I'm going to load it in the sigma 1 direction here.
I've turned the honeycomb 90 degrees, so this is still sigma 1.
And imagine now that we've got a crack here.
And I'm going to consider a situation where the crack is very large, relative to the cell size.
So it's not a crack in the cell walls.
It's a crack that goes through multiple cells.
I'm going to assume the crack is large, relative to the cell size.
I'm going to assume that the bending is the main deformation mode.
And what I'm going to do is look at-- if I have my crack tip here, I'm going to look at this cell wall a just ahead of the crack tip.
And I'm going to say, that cell wall is bent.
And I'm going to figure out something about the stress in that cell wall and look at when that fails.
And I'm going to assume that the cell wall has a constant modulus of rupture.
So the cell wall has a constant strength.
You can imagine the cell wall could have little tiny cracks in it, too.
And if a cell wall has a bigger crack, it's going to fail at a lower stress.
But let's imagine that the cell walls are all the same strength and they all have a constant modulus of rupture.
Let me write some of this down.
In tension, the elastic moduli are going to the same as in compression.
There's no elastic buckling in tension, so that's not going to happen.
The plastic plateau stress in tension is going to be very similar to that in compression.
As I mentioned, there's a small geometrical difference, but we're going to ignore that.
And then, if we had a brittle honeycomb, like one of those ceramic honeycombs I showed you, then we can have fast fracture.
What we want to calculate is the fracture toughness.
And I'm going to make a few assumptions here.
I'm going to assume that the crack length is large compared to the cell size.
And if I do that, I can say that I'm going to use the continuum assumption.
Hello.
We'll come back to that.
I'm going to say that axial forces can be neglected.
We're just going to look at bending forces.
And I'm also going to assume that the modulus of rupture is constant for the cell wall.
First, let's just think again about the continuum.
Imagine we just had a solid and we have a plate of the solid and it's loaded in tension with some remote stress-- some far away stress-- sigma 1.
And the plate has a crack of length 2c perpendicular to that normal stress.
And we're going to look at the stress-- local stress at the crack tip-- some distance r ahead of the crack tip there.
In fracture mechanics, it's been worked out what that local stress field is.
And it depends on the crack length, and then how far ahead of the crack tip you are.
So you can say that if you've got a crack length of 2c in a linear elastic solid, and the crack is normal to a remote tensile stress-- which I'm going to call sigma 1-- then that crack is going to create a local stress field at the crack tip.
And we're going to use this equation for the local stress field.
The local stress field is equal to the far away field divided by-- or multiplied by the square root of pi c and divided by the square root of 2 pi r. So there's a stress singularity at the crack tip.
And then the local stress decays as you move away from the crack tip
And what is r?
r is the distance from the crack tip.
So if that's the tip of my crack there, then r is my distance out.
OK.
In the honeycomb wall, if we look at the crack here, and then we look at that cell wall a that's just ahead of the crack tip, that cell wall is bent.
So in the honeycomb, we're going to be looking at the bent cell wall.
And that wall is going to fail when the applied moment equals the fracture moment.
If we look at wall a, we could say that the applied moment is going to be proportional to p times l. Getting ahead of myself there.
I'm going to do this-- because it's hard to say exactly where the crack tip is because there's a void there.
I'm going to use that argument here where I make everything proportional.
The moment's going to be proportional to p times l on wall a.
And the fracture stress is going to be proportional to sigma fs times bt squared.
Last time, we said it was sigma fs bt squared over 6.
It's the same thing.
I'm just dropping the 6 out.
And then I can also say that this applied moment, if it goes as pl-- p is just going to be my local stress times lb.
And then I multiply times l. So if you think of just thinking about-- if you got a load p on this member here, l, there's going to be some local stress there.
And p is just going to be that local stress times the cell wall length times the width into the page.
And then, that local stress, sigma l, I can replace with that equation over there.
So that local stress is going to go as sigma 1 times the root of c over the root of r. And I'm going to say the distance ahead of the crack tip goes as l. Instead of having r, I'm going to say it's l. It's not necessarily exactly l, but it's going to be some fraction of l. That's my local stress there.
And then I've got an l squared times b.
And if I set that equal to the fracture moment, that's going to be proportional to sigma fs bt squared.
Are we good here?
You have to think of the crack tip.
And there's some local stress field ahead of the crack tip.
And we're saying that the load p is equal to that local stress times a cell length times the depth into the board.
And then multiply it times l to get the moment.
And then I replace that local stress with this standard equation for the remote stress and the crack length and the distance ahead of the crank tip.
So here, the b's are going to cancel out.
And now I can solve for a fracture stress in the one direction.
And that's going to-- well, let me get proportional-- that's going to be proportional to sigma fs.
Then there's going to be a t over l squared.
And then, this is going to go as the square root of l over c like that.
And now, if I want to get a fracture toughness, the fracture toughness is just the strength times the square root of pi times the half crack length c. So here, my fracture toughness is just that strength times the root of pi c. So I can say that's equal to a constant times sigma fs times t over l squared.
And now, times the square root of l. These root of c's have canceled out.
So that's my equation for the fracture toughness.
And one of the interesting things here is that the fracture toughness depends on the cell size.
This is the first property that we've derived an equation for where it depends on the cell size.
OK. And here, c's just going to be a constant.
All right.
Now we've got a set of equations that describe the in plane properties.
We've got equations that describe the linear elastic moduli in the plane.
We've got three equations that describe the compressive stress for elastic buckling failure, for plastic yielding failure, and for a brittle crushing failure.
And we've got an equation for the fracture toughness, as well.
OK?
We've got a description of the in plane behavior of these hexagonal honeycombs.
The next thing I wanted to do was talk a little bit about in plane behavior, but for a different cell shape-- for triangular honeycombs.
Because they deform by a different mechanism.
And they can be used to represent the lattice materials that we looked at earlier, too.
If we have a triangular honeycomb with triangular cells, triangulated cells behave like a truss.
And you can analyze trusses by just saying that the joints are pin jointed.
There's no moments at the end of the joints or end of the members.
And the forces are all axial and so the behavior's a little bit different.
I wanted to show you how these triangular honeycombs work, too.
I can scoot this up.
Imagine that you've got a honeycomb that's an array of triangular cells like this.
And say we're applying some bulk stress sigma to it, like that.
And say it's got a depth b into the page.
When we have a triangulated structure like this, it behaves like a truss.
And we can analyze it as being a pin-jointed structure.
There's no moments at the joint.
And if it's pin-jointed and there's no moments at the nodes, then we just get axial forces along the members.
And even if the nodes were fixed-- as they are in these ceramic honeycombs-- you can show that if it's triangulated, even if you accounted for any bending, it really is a very tiny contribution to the deformation in the forces.
It's less than a couple of percent.
I'll say even if the ends are fixed-- I'll just say the bending contributes less than 2% to the forces in the deformation.
If I have a triangular cell like that, and say I pick a unit cell like this, and I say that the bulk stress produces a load of p on the top and p over 2 at each of the bottom nodes there, then the force in each member is going to be proportional to p. And for a given geometry of triangle, you can figure out exactly what the force distribution would be in each of the members.
But I'm going to use one of these proportional arguments again, just to get a general result.
Because I don't really care that much about the details of the geometry.
OK.
If I have a little set up like this, I can say that the overall stress is going to be proportional to p over lb.
And the stress in each member is going to be proportional to p over l times the thickness-- or b times the thickness.
This is the overall stress.
The overall strain is going to be proportional to some deflection of the triangle divided by the length.
So if I said, say, this length here was a length l. And then the deflection of each member is going to be proportional to p times l over es times the cross sectional area of the member, and that's just b times t. OK?
So this is the stress on the whole thing, the strain on the whole thing, and relating the delta to the p. And then, the modulus of the whole honeycomb is going to go as the stress over the strain.
So that's p over lb divided by delta over l. These l's here cancel.
And delta here is pl over es bt, and so the b's cancel and the p's cancel.
And the modulus I get for the honeycomb is just some constant related to the cell geometry times the modulus of the solid times t over l. And if you did an exact calculation for equilateral triangles, you'd find that that constant's 1.15.
The interesting thing to note here is that the modulus for these triangular honeycombs goes as t over l, not as t over l cubed.
For the hexagonal honeycomb, it went as t over l cubed.
And here, because the deformations are axial-- not bending-- it's much stiffer.
And it's much stiffer to have one of these triangulated structures.
I'll just say, here, that the modulus goes as t over l cubed for the hexagonal honeycombs due to the bending.
One of the reasons that people are interested in those lattice materials is that they, too, have moduli that go as t over l. That basically go with the relative density, rather than with the relative density cubed.
So they're much stiffer than, say, a hexagonal honeycomb.
OK?
Are we good with the triangulated honeycombs?
Yes?
What is c?
c's just a constant related to the cell geometry.
For equilateral triangles, it's 1.15.
You could work it out, but it just makes the whole thing a little more complicated to do that.
OK. That's the in-plane behavior.
And next, I wanted to talk about the out-of-plane behavior.
Remember, we said the hexagonal honeycombs are orthotropic and the orthotropic materials have nine elastic constants.
And we've figured out four so far.
We've figured out the four in-plane elastic constants.
There's five out-of-plane elastic constants to describe the elastic behavior completely.
And so we want to talk about these other elastic constants.
The honeycombs are also-- I should just back up a little bit.
The honeycombs are used in sandwich panels.
And when they're used in sandwich panels-- I brought a little panel in with carbon fiber faces and a nomex core.
If you bend that panel like that, you're going to get shear stresses in the core.
And the shear stresses are going to be going this way and this way on, and that way, that way on.
And so those shear stresses are out-of-plane.
They're in the x1, x3, or x2, x3 planes.
And so you need the out-of-plane properties for the shear properties in the sandwich panels.
Honeycombs are also sometimes used as energy absorption devices.
Not these rubber ones, but imagine there was a metal one.
And when they're used for energy absorption devices, they're typically loaded this way on.
Again, that's the out-of-plane direction and you need the out-of-plane properties.
And for the out-of-plane properties, the cell walls don't tend to bend.
Instead, they just extend or contract.
And you get stiffer and stronger properties.
Let me just write something down and then we'll start to derive some of those properties.
The cell walls contract or expand instead of bending, and that gives stiffer and stronger properties.
OK. OK.
There's five elastic constants in the out-of-plane directions.
We'll start with the Young's modulus.
And if I take my honeycomb and I load it in the x3 direction-- just taking this thing here and just loading it like that-- the cell walls just axially contract and the stiffness just depends on how much cell wall I've got.
So the modulus in the three direction is just equal to the area fraction times the modulus of the solid.
That's just the same as the volume fraction, or the relative density.
So it's quite straightforward.
The cell walls contract or extend axially.
e3 is just es times the relative density.
And that's just es times t over l. And then there's a geometrical factor here.
h over l plus 2 over 2. h over la plus sin theta times cos theta.
Again, a little bit like those triangular honeycombs.
The thing to notice here is that in the three direction, the modulus goes linearly with t over l, whereas in the in-plane directions, it goes with t over l cubed.
So there's a huge anisotropy in the honeycombs because of this difference.
Imagine a honeycomb might be 10% dense.
t over l might be something like a tenth-- 0.1.
So e star 3 is going to 0.1 of es, roughly, and in the other direction, it's going to be 1/1000th.
So there's a huge anisotropy because of this.
Let me just-- square honeycombs.
This just shows looking at the out-of-plane directions and the different stresses and properties that we're going to look at here.
The next one we're going to look at is Poisson's ratio.
And first, we're going to look at loading in the x3 direction.
And if we load it in the x3 direction, the cell wall's just strain by whatever the Poisson's ratio is for the solid times the strain in the three direction in the other two directions.
We'll say for loading in the x3 direction, the cell wall's strain by nu of the solid times whatever the strain is in the three direction in the other two directions.
If we load it in the x3 directions and everything contracts by that much in the other two directions, that just means that the Poisson's ratios-- nu 3 1 and nu 3 2 are going to be the same.
And they're just going to be equal to the Poisson's ratio of the solid.
So if each wall is going to contract by that amount, the whole thing's going to contract by that amount.
And that's going to give you that Poisson's ratio.
Let me just say, here, also-- and remember that I'm defining nu ij as minus epsilon j over epsilon i.
We're loading in the three direction here.
And then you can get the other two Poisson's ratios using those reciprocal relationships.
So nu 1 3 and nu 2 3 can be found from the reciprocal relations.
And remember, those relationships come from saying that the compliance tensor, or the stiffness tensor, is symmetric.
We can write, for instance, that nu 1 3 over e1 is equal to nu 3 1 over e3.
So I can write that like that.
And then I can say nu 1 3-- that is going to equal to nu 3 1 times e1 over e3.
And we just saw that nu 3 1 was equal to nu s. And we see, from before, the e1 is equal to some constant.
Let me just call it c1 times es times t over l cubed.
And e3 is going to be some other constant times es time t over l. The es's are going to go.
And if t over l is small-- even if it's say, a 10th-- and this is going as t over l cubed and that's going as t over l, then I can say this thing is about equal to 0.
It's going to be small.
It's not to be exactly [INAUDIBLE] 0, but it's going to be small so we're going to say it's 0.
So I'll just say for small t over l. And then, similarly, nu 2 3 is going to be close to 0, as well.
So there's the Poisson's ratios.
We've got the Young's modulus, the Poisson's ratios, and next we want to get the Shear moduli.
And the shear moduli is little more complicated.
The cell walls are loaded in shear but the neighboring cell walls constrain them and they produce some non-uniform strain.
I'm talking about shearing it this way on.
You can see on this figure here, we're talking about shearing it, like tau 2 3 or tau 1 3, this way.
And so each wall is going to shear, but the walls are attached to each other so they can't just do it independently.
They have to be constrained by each other.
And the exact solution is a little bit complicated.
And I'm just going to give you an estimate of what that modulus is.
And we're going to see that it depends linearly on t over l, as well.
I'll just say the cell walls are loaded in shear.
An estimate is g star 1 3 is equal to g of the solid times t over l times a geometric function.
It's cos theta over h over l plus sin theta.
And for regular hexagonal honeycombs, it's 1 over root 3 times gs times t over l. Again, just note the linear dependence of the modulus on t over l. And in the book, there's a method using upper and lower bounds that gives an estimate for g 2 3.
I'm not going to go into it.
I just want you to notice that the shear moduli go as t over l, just like the Young's modulus does.
And Sardar, who's sitting in the back there, has done even more involved calculations and analysis of the shear moduli of the honeycombs in this direction.
So I'm not going to go into all the gory details on that.
OK. That gives us the moduli now.
So now we've got all nine elastic moduli.
OK?
And the next thing to do is, then, to figure out the compressive strength.
So we're going to look at compression again, and then we'll look at tension.
If we look at compressive strengths, again, we've got different modes of failure.
And if I have an elastomeric honeycomb like this one here-- if these cell walls were a little longer, I might be able to actually do it.
If you compress this enough, you produce buckling in the cell walls.
And this is a schematic of this buckling pattern here.
And you can see there's a diamond pattern where it alternates up and down in the different cell walls.
We're going to do some approximate calculations, but you can see the idea of how the material behaves in this direction, just from these approximate calculations.
OK. Say we have our honeycomb like this, and here's the prism axis this way.
And now, we're going to load it up with some stress in the three direction.
I'm going to call this sigma star elastic 3 when it buckles.
And what we're going to do is just look at a single plate.
And look at the buckling of a plate.
We're going to analyze it just looking at a single plate and then adding up how many plates we have per unit cell.
It's actually more complicated than this because, obviously, the plates are attached together and there's some constraint by attaching the plates.
But we're not going to worry about that.
If you have a column-- just a, say, circular cross-section column-- and you apply a compressive load to it, it buckles at the Euler load.
And similarly, there's an Euler load for plates.
And that equation is usually written as a p critical is equal to some end constraint factor.
For plates, it's usually called k instead of n. So this is an end constraint factor.
It depends on the modulus of the plate.
It goes as t cubed.
Then, there's a factor of 1 minus mu of the solid squared and the length of the plate.
Say this plate here-- actually the width of the plate there is h and the length here is b.
And this thickness here is t like that.
Here, k is an end constraint factor.
And it's going to depend on the stiffness of the adjacent cell walls.
If I had a honeycomb, and say it was-- these walls here-- the adjacent walls-- were thicker, then you can imagine those thicker walls-- it'd be harder to get them to deform.
And the end constraint for the plate is going to depend on those thicker walls.
So that the end constraint, k, depends on these-- say I'm looking at this wall here of width h here, then how stiff these other two walls are is going to affect that end constraint factor.
What we're going to do is just do something very approximate.
We're going to say if these vertical edges here-- if this edge here and that one there-- if they were simply supported-- if they're just pinned to the next column, the next member-- then k has some value.
And if they're fixed, it has some other value.
And we're going to pick a value in between.
So we're going to do something very approximate.
I'll say if those vertical edges are simply supported-- that means they're free to rotate-- then k is equal to 2.0.
And this is if b is bigger than three times the length.
So this is h here, or we could say l. Either way.
It's really the-- it's the length when we look at the honeycomb this way on, but it's the width in that picture there.
And if the vertical edges are clamped, or fixed, then k is equal to 6.2.
These are values you can look up in tables of plate buckling.
And we're just going to approximate it by saying k is equal to 4.
We're just picking a value that's in between those two.
And then, the p total is going to be the sum of the p criticals for the columns that make up a unit cell.
For the unit cell, I have one wall of length h and two of length l. And if you just take that total load and divide by the area of the cell, you get that this compressive strength for elastic buckling is approximately equal to es over 1 minus nu s squared times t over l cubed.
And then there's a geometrical factor here.
And if you had regular hexagonal cells, this buckling stress works out to 5.2 times es times t over l cubed.
If you remember, for the loading in the two direction-- in the in-plane direction-- it has the same form.
And goes as es times t over l cubed, but it's much smaller.
This number here was, I think, 0.2.
It was much smaller.
So it has the same form, but it's a lot bigger.
OK?
Are we good with that?
The idea is we just use the standard equations for plate buckling.
We make some estimate of what that end constraint factor is.
And we just have an approximate calculation here.
OK. That's the elastic buckling.
If I had a metal honeycomb, then it might not fail by elastic buckling like that.
Instead, we'd probably get yielding.
If it was dense enough, we could just get axial yielding that-- if you just loaded it, you'd have axial forces.
And at some point, you'd reach the yield stress.
And so you can get failure by just uniaxial yield.
That's one option.
And if you get that, then it just depends on how much solid you've got again.
So it's just the yield strength of the solid times the relative density.
But usually, the honeycomb is thinner walled than that.
And usually, you get plastic buckling proceeding that.
In plastic buckling, you can think of it as-- say if you have a tube-- this is just shown for an individual tube here.
You can see how the tube folds up.
And you can get that same kind of thing with the honeycomb.
Here's a single tube.
It's been loaded along the prism axis of the tube.
And you can see, you get these folds, and the more you load it, the more number of folds you get.
And the more the folds concertina up.
To do an exact analysis for the honeycombs, you would have to take into account not just one tube, but the constraint of the neighboring tubes again.
And again, that gets to be a complicated, messy thing.
So again, we're going to do a more approximate thing.
What we're going to do is just say that we have members that are folding up like that.
So the same geometry.
But we're just going to look at a single cell wall and see what the single cell wall does.
And someone else has done the more exact calculation.
We'll just compare our approximate calculation to the exact one.
OK. We're going to consider an approximate calculation.
What we're going do is look at our isolated cell wall.
And if you look at the figure here, the wall is going to be vertical, initially.
And as we load it, eventually it's going to buckle and we're going to form one of those plastic hinges in the middle here.
And then, the thing is then going to rotate about that plastic hinge and just fold up.
So at the bottom here, it's completely folded up.
OK?
And we're going to do a little work calculations.
We're going to look at the internal work done and we're going to look at the external work done.
The external work is just going to be this load p times that deflection delta that the p moves through.
And if we say this is half of a wavelength-- if you think of this thing going through multiple wavelengths, just consider when it folds up like that, that's a half of a wavelength.
It would go two of those to get a full wavelength.
That's lambda over 2.
And so to go from this stage to that stage over here, the external work done is going to be approximately p times lambda over 2.
Say that it's thin and that 2t is small compared to lambda.
So it's going to be about p times lambda over 2.
And then, we're also going to look at the work done by the plastic moment.
And when we form the plastic hinge here, there's a plastic moment.
And that moment is going to rotate through an angle of pi.
So we start out straight here, we end up folded up like that, and we've gone from straight to that.
We had to go through 180 degrees to get there.
So it goes through an angle of pi.
And if you have a moment going through a rotation, the work done is the moment times the rotation.
We're going to equate those two works done.
We're going to look at the rotation of the cell wall by an angle of pi at the plastic hinge.
Our plastic moment-- it's going to be the yield strength of the solid again times t squared over 4, the same as when we were talking about the plastic moment before for the other loading direction.
But now, instead of multiplying this times b, we're multiplying it times 2l plus h. That's the length of the cell wall that's associated with one cell.
And now, it's not b because now we've turned the thing the other way on.
We're loading it the other way on.
And this plastic hinge-- if I think of-- if this was b before.
And now that b is l plus 2h-- or 2l plus h, rather.
That's the dimension of the-- let me draw a little hexagon so maybe you can see.
OK. Now we're forming a plastic hinge halfway down the board.
Imagine that this has some length b that way and we're halfway down the board.
And now, the plastic hinge has to form all the way around these members here for one cell.
Or you could think about it as this guy plus these guys is one cell.
You can think about the unit cell different ways, but it's one h plus two l's.
OK?
Are we OK with that?
OK. Then the internal plastic work is that plastic moment times the rotation pho-- or pi, rather.
Sorry.
Are we OK with this?
That the work done is m times our angle?
Imagine-- let me get rid of my honeycomb here.
Imagine you have a point here and you have some force over here.
Let's call that f. And say, the force is at distance r from f. And say that it moves through some distance.
The moment here would be r times f. And if that rotates, say, through some angle-- let's call it alpha-- and here is f here, then this distance here that the force moves through is just r times alpha.
So the work done is going to be r times alpha times f, or just the moment times alpha.
OK?
So that's all that we're doing.
OK. That's the internal plastic work.
And now we have to look at the external work done.
And that's equal to p times lambda over 2.
Here, lambda is the half wavelength of the buckling.
I'm going to say for these tubular kinds of things, it's in the order of l. So if you look at that last slide here-- oops.
Rats.
How'd that happen?
Let me scoot back down here.
There.
If we look at that guy again, the magnitude of the buckling wavelength is on the order of l. And here, below p, can be related to the stress in the three direction.
We'll just multiply it times the area of the unit cell.
And so if I equate the internal work and the external work, I can say p times lambda over 2 is equal to pi times my plastic moment.
And then, for p, I can write sigma 3 h plus l sin theta times 2l cos theta.
And then, lambda is l divided by 2 is equal to pi.
And then I've got my plastic moment over there.
And then if I solve for sigma 3, that's my compressor strength.
I've got pi by 4, the strength of the solid, sigma ys, times t over l squared.
Then h over l plus 2 divided by h over l plus sin theta times cos theta.
And for the regular hexagons, this works out to about 2 sigma ys times to over l squared.
And the exact calculation for regular honeycombs is equal to 5.6 times sigma ys times t over l to the 5/3 power.
This power here-- 5/3-- is a little less than 2.
And that's because the additional constraint of the neighboring cell walls.
But the main thing we're interested in, in these sorts of calculations, is the power dependence on the density and this simple calculation.
Obviously, it's not exact, but it gets you close.
OK.
I'm just going to wait for people to catch up a little.
OK.
The next property I'm going to look at is out-of-plane brittle fractures.
Say we loaded in tension, and if we had no cracks in the walls, we'd just see uniaxial tension and the strength would just be the strength of the solid times the relative density times the amount of solid.
We'll just say if defect free, the walls see uniaxial tension.
And then the fracture stress in the three direction is just equal to the relative density times the fracture strength of the solid.
If the cell walls are cracked, and if the crack length is very much bigger than the cell length, then the crack propagates normal to x3.
Then we can say the toughness gc-- or the critical strain energy release rate-- is just equal to the volume fraction of solid times gc for the solid.
And then the fracture toughness, k1c, is equal to the square root of the Young's modulus times gc.
And that's just equal to the relative density times the modulus of the solid.
And then the relative density times the toughness of the solid.
So it's just equal to the relative density times the fracture toughness of the solid.
It's just straightforward there.
Then we've got one last out-of-plane property.
And that's brittle crushing and compression.
And if we have some compressive strength of the cell wall-- say I call it cs-- then it's just the relative density times that strength.
And for brittle materials, that crushing strength is typically around 12 times the modulus of rupture, or fracture strength.
We could say that's about equal to 12 times the relative density times sigma fs, a fracture strength.
OK. That's the modeling of the honeycombs.
I know there's been a lot of equations and derivations, but that's the basis of a lot of the things we're going to do in the rest of the course.
The modeling we're going to do on the foams is based on this and the mathematics is just easier because we're going to use these dimensional arguments.
We're not going to figure out all these geometrical parameters.
Before we get to the foams, I wanted to talk a little bit about honeycombs in nature.
And we've only got a couple minutes left, so I won't really get that far.
But I wanted to talk a little bit about honeycomb materials in nature.
And the two examples we're going to talk about are wood and cork.
I'm going to talk a little bit about the structure of wood next time.
Then, we'll see how we can apply these models to understanding how wood behaves.
And we'll see how you can use these models to predict the density dependence of wood properties and also the anisotropy in wood properties.
And I guess we'll probably, maybe, start it Wednesday next week.
We'll talk about cork, as well.
Those of you who took 3032 know that I like cork because of Robert Hooke and his drawing of cork.
And I made a new video that I'm going to show you.
Remember in 3032, I showed you the video from the Bodleian Library, where they had the first edition of Hooke's Micrographia.
Well, it turns out Harvard has a first edition.
Harvard has three first editions.
Yeah.
Exactly.
MIT has zero first editions.
Gee, why does that surprise me?
And I have a friend who's a librarian at Harvard and she arranged for me to go and make a little video with the first edition of Micrographia.
So I can-- I don't if we'll play the whole thing, but I'll show you the first little bit of it.
And you can watch it at your leisure.
And Sardar came.
You came and saw it with me.
You came and saw the first edition with me, right?
Yes
Yeah.
Yeah.
It's very beautiful and you'll see some of the nice drawings.
And I talk about the cellular structure of some of the drawings.
So we'll talk about wood and cork next time.
But I think I'm going to stop there because that seems like enough equations for now.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I should probably get started.
So I just wanted to mention this Friday the libraries are having Furry Friday.
So they have therapy dogs come, and if you like dogs, it's kind of fun to go get cuddled by the dog.
The other thing I wanted to mention, last term, there was a student taking 3032 who was interested in art.
And I kept trying to find art pictures for him, and he's not here.
But I thought everybody else can like the art too.
So I belong to the Peabody Essex Museum in Salem, Mass.
And they have an exhibit right now on wood and on sort of using wood as a sculptural material.
And this is one of their posters to advertise it.
So this thing was carved out of a single piece of wood, I think.
And they've got lots of other sort of sculptural wood.
So I thought you might like to see that.
If you wanted to go to Salem, there's a couple of options, if you don't have a car.
You can take a commuter rail to Salem.
You can also take the ferry.
And if you take the ferry to Salem, it's like a five minute walk to where the ferry lets you off to get to the Peabody Essex.
And it's a kind of neat museum.
It's not too big.
It's kind of small.
But it's a beautiful building, and they have neat stuff there.
So you could go to the Peabody Essex Museum.
Hmm?
It's very nice
Yeah, you've been there?
Yeah, it's really nice.
So I was going to talk about honeycomb-like materials in nature today.
And I'm going to talk about wood today, and I might finish this today.
I might not.
And then I'm going to talk about cork for a little bit on Wednesday, and then we'll start talking about foams after that.
So I have a couple of sort of cute little language historically things.
And you know how I like that stuff too.
So I have two things about words that are related to wood.
So the word "materials--" you know where the word materials comes from?
It comes from the Latin.
So there's a Latin "materies materia."
And materies materia means "wood" or "the trunk of a tree."
So if you think of studying materials, in olden times that was like studying wood.
And another cute thing that I found was that in old Irish the names of the first few letters of the alphabet are named after trees.
So the letter A, that's called alem in old Irish, and alem is the word for elm.
And B is-- I don't know if I'm saying these right.
It's called beith, and that's the word for birch.
And C is call.
That's the word for hazel.
And D is dair, and that's the word for oak.
And so they sort of named the letters of the alphabet after different kinds of trees, different kinds of woods.
So I just thought those were kind of interesting historical things.
So I wanted to start by talking about wood structure.
And then we're going to look at how would deforms and fails, and talk about the data that people have measured for the wood properties, things like stiffness and strength.
And then we'll talk a little bit about how the honeycomb models can be applied to understanding the mechanical properties of woods.
So this is kind of a generic trunk of the tree here.
And we're defining three axes.
The radial axis comes radially out of the tree.
There's the tangential axis, so that's the x1 and the x2 axes.
And then there's the longitudinal or the axial axis, x3.
So if you think of the wood as being, in a very, very simple way, just like the honeycomb, the radial would be this way on.
The tangent would be that way on.
And that axial would be that way on.
So it's like that.
And if you neglect the growth rings, you can say that other woods orthotropic, and that's typically what people do.
They neglect the growth rings, and they say that it's orthotropic.
And the density of the woods, the relative density ranges from about 5% for balsa wood to about 80% for lignum vitae.
So I brought in some pieces of wood.
So this is balsa wood.
You're probably familiar, making different kinds of models with balsa.
So balsa's very light.
It grows in Ecuador.
And it's the lightest wood.
And this is lignum vitae.
You're probably not so familiar with that.
This actually grows in Florida, and it's the densest wood.
It has a relative density of 0.8.
And it's so dense, that if you put it in water, it sinks.
So it's a very dense wood.
And the way the wood cells grow is that if you look at the sort of structure here of a tree, there's the bark on the outside here, and then there's the kind of wood cells inside the bark.
And there's a layer of cells in between the bark and the wood called cambial layer.
And that's really the layer of cells that are alive and are dividing.
So if you think of the wood cells, they're living when they're in that little cambial layer there.
And they're dividing.
And that cambial layer, the cells have a plasma membrane and a protoplast.
And then they sort of exude the plant cell wall.
So a little like bone cells, like if you think of the bone in your body, there's osteoblasts and osteoclasis, different kinds of bone cells.
But the bone cells secrete the sort of collagen and the calcium phosphate that are the sort of hard mineral part of the bone that you think about in the bone.
And that's not a living thing.
The cells are the living thing.
That's like an extracellular matrix.
In the trees, it's a little bit the same.
So there's the living cells that are just under the bark, and they have this plasma membrane and the protoplasm.
And over a few weeks they excrete the plant cell wall, and then they die.
So the living cells die, and you're left with the plant cell walls.
And then as the tree grows, you're always having a layer of these cambial cells, and it forms bark on the outside and wood on the inside.
So there's sort of a layer of cells that are differentiated, such that on the outer layer they form the bark, and on the inner layer they form the wood.
And as the tree grows, that cambial layer is kind of expanding out radially.
So let me write some of these things down.
Let's see.
So let me just write down those two little word things because I think they're cute.
So the word materials is from the Latin materies materia.
And that means "wood" or the "trunk of a tree."
And here's the little old Irish thing.
It's not like I think-- I'm not going to put this on the test or something.
I just thought it was cute.
So the letter A is alem, which is elm.
And letter B is beith, which is birch.
Letter C is call.
That's hazel.
And D was dair, and that's oak.
So that's just for general interest.
So then the wood structure we can think of it as orthotropic, if we ignored the growth rings.
And if you have a sort of large diameter tree and you take a piece of wood not from the very center, but from somewhere near the outside, then that's not a bad approximation.
Now, the relative density of the woods ranges from about 0.05 for balsa to about 0.8 for lignum vitae.
Any Latin scholars here?
I took one year of Latin in high school.
Anybody take Latin?
No, no Latin.
So I think lignum vitae I think is "tree of life."
"Vitae" is the sort of life.
And when it has this ending A-E it means "of life."
So I think that's the "tree of life" is lignum vitae.
So trees have cambial layer beneath the bark.
And the cell division occurs in that cambial layer.
So the new cells on the outer part turn into bark, and the new cells on the inner part turn into wood.
And then we have the living plant cells that have the plasma membrane and the protoplast.
And those cells then secret the plant cell wall, which sort of surrounds them.
So in trees, the living cells lay down the plant cell wall over a period of a few weeks.
And then the living cells die.
Oops.
Back.
Here.
Now you always retain a layer of those cambial cells.
So you may have heard if you have a tree and you cut a ring around the tree through the bark, if you go into those cambial cells and you destroy them, you kill the tree, because you're killing that layer of living cells.
So then we want to look at the cellular structure of the woods as well.
And I've got a couple of slides here.
This one is of softwoods.
And softwoods have two types of cells.
That have tracheids, which are the bulk of the cells here, and the tracheids provide structural support.
And the tracheids also have little holes along the length of them at their ends called pits, and those pits allow fluid transport up and down the tree.
And then the softwood also has these ray cells here.
So those are examples of ray cells.
So this is a transverse section.
This is a longitudinal section here.
And the rays are parenchyma cells which store sugars.
So softwoods have tracheids and rays.
And then hardwoods, here's an example of a hardwood oak.
They have three types of cells.
There's cells called fibers, so these guys all in here would be fibers.
They provide the structural support.
They have vessels, these really large cells that provide fluid transport up and down the tree.
And they also have rays.
So here are some rays here.
And again, those rays are parenchyma cells that store sugars in the tree.
So let me just write down what all these cells are.
So in softwoods most of the cells are these tracheids, so they make up the bulk of the tree, something like 90% of the tree.
And they provide structural support.
They have holes in the cell wall for fluid transport, and those are called pits.
And to give you some idea of what size they are, they're are a few millimeters long, so something like 2 and 1/2 to seven millimeters long.
And then they're tens of microns in the other two directions, so they're something like 20 to 80 microns across.
And the cell wall thickness, t, is usually a few microns, so something between about two and seven microns.
So typically, the denser the wood, the thicker the cell wall's going to be.
Whoops, let's see if I can get the rays down here.
Put it on the same board.
So the rays are parenchyma cells that store sugar.
And then the hardwoods have three types of cells.
They have the fibers that provide the structural support.
And the amount of cells that are fibers varies, depending on the species, but it's usually somewhere around 35% to 70% of the cells.
And then they the vessels, which are the sap channels.
That provides for the conduction of fluids.
And that's between about 6% and 55% of the cells.
And then, again, there's rays that store sugars, and they usually make a boat 10% to 30% of the cells.
So there's the structure of this sort of cellular structure, at this kind of length scale of tens of microns.
And then there's also a structure within the cell wall itself.
And the cell wall itself is made up of cellulose fibrils in a matrix of lignin and something called hemicellulose.
So if you look at the cellulose structure, the cellulose has a regular structure, a sort of periodic lattice.
And it's crystalline for most of the length of the fibrils.
So this is the structure of the cellulose here, and this is showing it at a slightly larger length scale.
It might have a crystalline region here and then a non-crystalline region here.
And these macro fibrils, which are made up of bundles of micro fibrils, are about 10 to 25 nanometers.
And each one of the micro fibrils might be three to four nanometers across.
So you have these cellulose fibers.
And then the cell wall is made up of different layers.
So there's what's called the primary wall here, which has a random arrangement of the cellulose fibrils.
Then there's an outer layer here.
These are all called secondary layers.
This is S-- I think that's S1.
Yeah, it's S1.
And it has this arrangement of the fibrils.
Then there's a layer called S2, and it's generally the thickest layer in the cell wall.
And the cellulose fibrils are aligned not perfectly vertical, but a little off the vertical.
And the angle between the vertical and the orientation of the cellulose fibers is called the microfibril angle.
And then there's a third layer here, S3, with, again, a different winding of the fibers.
So because S2 layer is the thickest layer and because the fibrils are closest to the vertical axis, the S2 layer actually contributes the most to the longitudinal modulus and stiffness and strength of the cell wall.
So that's kind of the arrangement of the cell wall.
And then so that one cell would have that.
Another cell would have that.
And in between the two, there's a layer called the middle lamella that kind of glues them together.
So that's the arrangement of the cells.
Let me scoot over here.
So the cells are often modeled as a fiber reinforced composite that has four layers to it.
And in each layer there's different volume fraction of the fibers and different orientation of the fibers.
So the cell wall has this fiber-reinforced structure.
Here's the cellulose fibers in a matrix of lignin and hemicellulose.
And there's four layers, each with the fibers in a different orientation.
And then there's the middle lemella between the two cells.
So in doing the modeling of a material like wood, you need to know what the properties of the cell wall material are, because, obviously, the properties of the wood would depend on the cell wall properties.
And it turns out that they're similar.
They're not exactly the same, but they're similar in different species of wood, so we're going to call them more or less the same.
So the density of the solid is 1,500 kilograms per cubic meter.
The modulus of the solid in the axial direction is 35 gigapascals.
The modulus in the tangential direction or transverse direction is 10 gigapascals.
And the strength of the solid in the axial direction is 350 megapascals.
And the strength of the transverse direction is about 135.
So here A means Axial direction, and T is transverse.
And just for comparison, if you just look at cellulose, cellulose has some pretty amazing properties.
The modulus of cellulose is about 140 gigapascals, which is very high for a polymer.
And the strength of cellulose fibers run between about 700 and 900 megapascals.
So the cellulose fibers have very impressive properties.
And that's one of the things that gives wood very good properties.
So the next thing is I want to show you some stress-strain curves for wood.
And you'll see how similar they are to the honeycombs that we looked at before.
And then we'll look at how the cells are deforming as they're getting loaded.
And from that, we're going to do some modeling.
So let me just wait till people get caught up.
Are we caught up?
More or less?
OK.
So these are all compression curves, so I'm just going to talk about compression.
So these are curves for different types of woos.
And on the left, the wood is loaded in the tangential direction.
So in terms of the sort of honeycomb model, it's loading at kind of this way on, like that.
And on the right, are a set of curves for wood load it in axial compression.
So in axial compression loading, we're loading it that way on.
And we've got different species here.
So the lowest density is balsa, around about 100 kilograms per cubic meter.
The densest species on this plot is beech, which is around 700 kilograms per cubic meter.
And then there's pine and willow, some other species in between here.
So you can see the shapes of the curve look just like the curves that we had for the honeycombs.
So here there's a linear elastic bit.
There's a stress plateau.
And there's a densification bit.
And then, as the density goes up, it gets stiffer, and the strain at which the densification occurs gets smaller, and the strength gets higher.
And if we look at the axial properties, the shape of the curve is similar.
We get linear elastic stress plateau densification.
But if you look at the scale here, this scale goes from 0 to 100, whereas that scale went from 0 to 20.
And so the stiffness and the strength along the grain are much higher than they are across the grain.
And you probably already know that.
Wood is stronger and stiffer along the grain than across the grain.
So that's what the stress-strain curves looked like.
And the fact that we're getting the curves that look like that makes us think maybe the mechanisms of deformation and failure are similar to the honeycomb, too.
So here's a set of curves for balsa all plotted on the same scale.
And, again, you can see for loading across the grain, either in the radial or the tangential direction, the stiffness and the strength is a lot less than if you load it in the axial direction.
So a number of years ago, we had a project on balsa.
And the thing we were interested in doing was looking at how the cells deformed and failed.
And because balsa's a low-density wood, it was easier to see the deformation in the cells, because the cells were thin.
So that's why we chose balsa.
I actually have a project on balsa right now.
And [?
Sardar ?
], my postdoc, is doing more detailed kind of finite [INAUDIBLE] modeling, trying to represent the structure of balsa.
And I think I mentioned, the reason we're interested in it is the balsa's used as a core in sandwich panels in wind turbine blades.
It's actually the best material that they can find, it's better than any engineering material.
So that's comparing the three curves for balsa.
And then if you look at a specimen that's loaded in the [?
SCM ?
], with a loading stage, you can measure the stress-strain curve, and you can take photographs of what the cellular structure looks like at different stages of loading.
So here, this picture one, is unloaded.
And these four images here are looking at the same section of cells, the same area of cells.
And you can see there's a big vessel here, and that's the same vessel there.
So here, this image two, is at this point on the stress-strain curve.
Here's three, at that point.
And four is at that point.
So if you look at this carefully-- and I've got another higher mag picture I'll show you in a second-- you can see that what's happening is the cell walls are bending.
So it's kind of like taking my honeycomb like this, and I'm doing that to it, and the cell walls are bending, so just the same as the honeycomb.
And then eventually, if I load it enough, you get to this sort of densified stage.
And you're doing this, and the stress-strain curve increases sharply.
So here you can see how the cells have densified over here.
It kind of looks a lot like my honeycomb when I-- maybe I do it this way-- when I smush it up like that.
It looks kind of similar.
So if we look at the higher mag picture, again, these four images are the same area of the cells.
And if you look at that a little bit of crud, it's the same on all four of them there.
So this is the unloaded one.
And these were loaded from top to bottom.
And this is loaded to some extent/ that's loaded more, and that's loaded more.
So if you look at this cell here, it's got this little tear on it.
So you can sort of find it again.
If you look at that cell there, that's what it looks like unloaded.
And here you can see-- see that wall there?
You can see how it's bent up.
So it's bent like the honeycomb walls.
And here it's bent even more.
And eventually, it has this sort of a shape here, and it's deformed permanently.
It's formed one of these plastic hinges.
So it's like the aluminum honeycombs almost, that it's filled like that.
So in the balsa wood, when we load it in the tangential direction, we're getting bending of the cell walls and then yielding and plastic hinges forming, just the same as we would in an aluminum honeycomb.
Are we good with that?
Well, I'll go through all three directions.
And then I'll write down the notes.
So this is loading the balsa in the radial direction.
And these things here are the rays.
So we're loading it in that direction.
And here you see this also bending occurs, but the rays act a little bit like fiber reinforcement.
So the rays are a little bit stiffer, and they sort of reinforce the thing a bit.
And this is the loading platten here, and you can kind of see that the failure starts at the loading platten, and as you sort of load up more, it progresses in from the loading platten.
So we're going to look at the modeling of the balsa in the radial direction, and we're going to count for the rays, at least in a crude way.
And then when you load them balsa in the axial direction, initially you don't really see much happening.
So if one is unloaded, one's down here.
And two is at this peak stress up here.
And really, if you look between one and two, you just don't see an awful lot of difference.
And that's because what you're doing is you're taking the wood and you're loading it this way on.
And it's so stiff, you just don't see much deformation.
So there's not really much to see.
But then eventually, something starts to fail.
And in this case, what fails are the end caps.
So the balsa wood has these long cells here.
But then at the end of the cells, there's little caps on the ends.
And the cells kind of fit together like that.
So that eventually, if you keep smushing it, those end caps start to fail.
Here you can see how bright it gets, and the cells are starting to crush together and kind of fail those end caps.
And in fact, each one of these serrations here, if you look at, say, from that peak up to that peak, that corresponds to a length of about the length of the cell, or the length of the cell between the end caps.
So in axial deformation you're just actually deforming the cells until you break those end caps.
If you look at denser words, they fail in slightly different ways.
This is a Douglas fir, which is much denser.
This particular specimen, the whole thing is kind of buckled over.
So it's not really so representative of the structure itself.
This is Douglas fir in radial compression.
You can see this picture, it looks just like what I showed you for the balsa wood, that sort of propagation of the failure.
These long things here are the rays.
And this is a Norway spruce in axial compression.
And this is fairly common in denser words.
You get this buckling formation.
And what happens is, I think, you get some yielding of the cell walls initially, but that leads to buckling, like a plastic buckling.
And you can see on this higher mag picture down here, you get these really small wavelength buckles in the cell wall.
And the two-- you get a plane that kind of shears over itself.
And you can see in the top image, this top half has shared over relative to the bottom half.
And all the deformation is in this little band here.
So this stuff here is all going on in that band up there.
So let me write down some notes about how these things to form and fail.
And then we'll get to the modeling in little bit.
So we can say the stress-strain curves resemble those for honeycombs.
And I'll say the mechanisms of deformation and failure are most easily identified in low density balsa wood.
So for balsa, if we look at the tangential loading, we see bending of the cell walls and then eventually plastic yielding.
And for radial loading, the rays act as reinforcing.
And for axial loading, you get axial deformation and then the failure of the end caps.
And I'll just say failure by plastic buckling is also observed, say, in the denser woods.
[INAUDIBLE] So then we can look at some data for the properties of woods.
And these charts plot relative Young's modulus and relative strength against relative density.
So here the modulus of the wood is divided by the modulus of the solid cell wall material.
And here we've normalized everything by the modulus of the solid cell wall material in the axial direction, because the cell wall itself is anisotropic.
And so here's the relative modulus, and here's the relative density.
These are log-log plots.
And we see that when we load the wood in the axial direction, the moduli is just linearly related to the density.
And when we load it across the grain, it varies with the cube of the relative density.
So do you remember our little honeycomb models?
If I took the honeycomb and I loaded it this way, it went as the cube of T over L. And that's because the bending.
And so the wood doesn't lie perfectly on that cube line, but it's fairly close.
And then similarly, if we took the honeycomb and we loaded it this way on, it deformed axially.
The modulus depended linearly on the density.
So you get the same kind of relationships there.
And then if you look at the strength, the strength along the grain goes linearly, and the strength across the grain goes with the square.
And we'll see when we get to the modeling in a minute, that if we loaded, say, an aluminum honeycomb this way on, the strength would go linearly with the density, if we're just yielding the cell walls.
And if we loaded it this way on, it went as the square of T over L. So these things kind of correspond.
And you can see the structure of the wood is a lot more complicated than just a simple honeycomb.
And so these models are sort of first order, and they're fairly crude.
They don't try to capture every detail of the wood structure.
But they can give you a sense of where the wood properties are coming from.
So let me just write down some of these observations.
So the data for the wood-- the modulus along the grain goes linearly with density.
It goes more or less as the cube for loading in the tangential direction.
And the radial direction is somewhat stiffer different than that.
The strength in the axial direction goes linearly with the density.
And the strength across the grain goes with the square of the density.
And then there's data for the Poisson's ratios too.
So let me just write them down.
So the modeling based on the honeycomb is sort of a simplified model that gives you kind of a first-order description of the behavior.
And it doesn't really attempt to capture all the details of the softwood and hardwood structure.
And in the equations, I'm going to take the cell wall properties along the grain, or along the axial direction.
And we're going to have a bunch of constants that describe the cell geometry, and those constants are also going to reflect the cell wall anisotropy.
So we can model the wood structure as something that's a bit more of a simplified thing, just like this.
And we say we've got cells that are roughly hexagonal, and then we've got some cells that are more or less rectangular that are the ray cells.
And if you look at lots of micrographs, you can get some idea what the dimensions of the cells are.
And these dimensions were measured for a particular density of balsa wood.
So if we look at the linear elastic moduli, we can start off with a tangential loading.
And if we have the tangential loading, we can model it as a honeycomb loaded in the plane, and we get cell wall bending.
And from the cell wall bending in the honeycomb model, you would get that the tangential modulus varies with the relative density cubed.
And the structure's not quite that simple.
There's ray cells.
There's end caps.
And they act to stiffen it a little bit.
And the data lie a little bit above this line.
Then if we look at the radial loading, the rays kind of line up with the radial direction, and the rays act as reinforcing plates.
And so you can just use kind of an upper-bound composites idea to get the modulus.
And the rays tend to be a bit denser than the fibers.
So if I say a Vr is the volume fraction of rays, and R is the ratio of the relative density of the rays compared to the fibers, so it's rho over rho S for the rays divided by rho over rho S for the fibers.
And that varies a little bit from what one species to another, one specimen to another.
But it's something a little over 1, something between 1 and 2.
Then I can say the modulus in the radial direction is the volume fraction of the rays times R cubed times the tangential modulus plus 1 minus the volume fraction of rays times the tangential modulus.
And that works out to be about 1.5 times the tangential modulus.
I wanted to work this out in terms of the tangential modululs, so I've put this in terms of the tangential modules in the first term there.
So we get that the radial modulus is slightly larger than the tangential, but also goes roughly as the cube of the density.
And then for the axial loading, we just have axial deformation in the cell wall.
And the Young's modulus just varies linearly with the density.
So these are kind of simple models, but they kind of explain to first order the density dependence of the wood moduli and the anisotropy.
So it's kind of nice because they're fairly simple models, and it gives you kind of a big picture.
So if you wanted to know the modulus of a particular piece of wood, this probably isn't the best way to figure it out.
But if you wanted to kind of compare how do woods behave in general and how does the density affect the properties and why are they anisotropic, this is a pretty good way to do it.
We could also look at the Poisson's ratios.
And just because I didn't want to write them down again, I've just left on what the data were down here.
But let me just write what the model would give us for nu RT and nu TR, the model would give us one if we had regular hexagonal cells.
And these are the values we get here.
This might be 0.6, 0.7 would be a typical value, somewhere around 0.4 in there, so they're not quite one, but they're close to it.
And I think the reason they're a little less is because the rays in the end caps provide some constraint.
If you have the honeycomb, if I just had these cells, and I squeeze it like this, these guys can move out.
If it's a regular hexagonal honeycomb, the strain that I'm applying here is equal to the strain going out that way.
But if I have rays this way that sort of constrain it or end caps, it means that the Poisson's ratio is going to be a little bit less.
So I'll just say constraining effect of the end caps and rays-- constraining.
Then for nu RA and nu TA, the model says the value we would get would be zero.
And these are pretty close to zero.
They're not quite zero, but pretty close.
And then the last pair nu AR and nu AT, the model says that we would get nu of the solid.
And the data's close to 0.4, which we might expect would be about the nu of the solid.
So, again, there's some variation in the Poisson's ratios.
They're not all just one number.
But you can see these ones here are about zero, and that's roughly what the model says.
These ones here are closer to 1.
And then these ones here are closer to what you might expect for a solid material.
So it gives you the kind of general idea.
Are we good?
We're good, yeah?
So we can do a similar thing for the compressive strength.
So for tangential loading, we get plastic hinges forming and the bent cell walls, just like in an aluminum honeycomb.
Then we get that the strength over the cell wall strength goes as the relative density squared, so just like the honeycomb.
the radial loading, we can do the composites thing again.
So we can say the strengths in the radial direction is about equal to the volume fraction of rays times R squared times the tangential strength plus 1 minus the volume fraction of rays times the tangential strength.
And for balsa, I have some values here.
VR is about equal to 0.14.
R is about equal to 2.
And so the radial strength is about equal to 1.4 times the tangential.
And in higher density woods, the value of R is a little bit smaller, and in general, the radial strength is a bit larger than the tangential, and both depend on the density squared And then for axial loading, if the failure's initiated by yielding in the cell walls, then the axial strength's just going to depend linearly on the density.
So the idea with these models isn't that they kind of describe a particular piece of wood exactly.
It's more that it gives you a general picture of how the cells are deforming and failing, and how the properties scale with density and why the wood's anisotropic.
Are we good?
Yeah?
Caught up.
So there are a couple more sort of interesting things we can do with looking at the wood properties.
So we've been talking about how to model the cellular structure.
But people have also looked at how to model the cell wall as a fiber composite.
And this plot and the next one kind of show you how you can combine all of that together.
So remember, I said the modulus of the cellulose was around 140 gigapascals.
So here's the modulus of the cellulose, at least the crystalline part of the cellulose plotted in that little envelope there.
The lignin and the hemicellulose have a modulus around 2 or 3 gigapascals, so it's down there.
And if you made composites with cellulose fibers in lignin and hemicellulose matrix, those composites would have a modulus that fell in this envelope here.
They've got to be in between those two limits, right?
The modulus have to be between those two limits.
The density have to be within the densities of the constituents.
And if you look at the modulus of the wood cell wall, it lies in this envelope here.
Along the grain it'd be here, and then across the grain is further down here.
So the cell wall modulus is in here.
And then if you take that cell wall and you make it into the honeycomb-type material that wood is, if you load it along the grain, you're going to get this linear dependence of modulus on density.
And if you load it across the grain in the radial or the tangential direction, you're going to get this cubed dependence here.
So here's a set of data for different woods of different densities.
And that envelope kind of encompasses all of them.
But if you look at the slope of that data, it's roughly equal to a slope of 1.
And so it corresponds to that equation there.
And similarly, here's a set of data for different species of woods of different densities loaded perpendicular to the grain.
And they lie on a line that has more or less a slope of 3.
And this set of data here along the grain intersects the wood cell wall towards the top of that envelope, and this set of data here intersects closer to the bottom of that envelope for the cell wall material.
So this gives you a way of sort of putting everything together on one plot-- the cell wall as well as the cellular structure.
So that plot does it for the modulus.
And you can do the same kind of thing for the strength.
Here's the cellulose up here.
Here's the lignin down there.
Here's the wood cell wall, the composite made from those two.
And then here's data for different kinds of woods loaded along the grain and for load across the grain.
So it gives you a way of putting all this modeling into one set of plots.
So let me just write a couple of little things about that.
So we could say you could model the cell wall as a fiber composite.
And you can use the composition upper and lower bounds to give an envelope.
And then you can also show the cellular solids models on the same plot.
So overall, it shows you how the hierarchical structure fits together and can be modeled.
Now there's some more cute things we can see.
So another thing I want to talk about is material selection, because it turns out wood is very good compared to other materials in certain applications.
So we're going to look at, say, having a beam of a given stiffness at a given span, and say it's just a square cross-section beam of edge length T. And the question is, what material would minimize the mass of the beam?
So say we have some span we have to have.
It's got to have some rectangular cross-section, some given stiffness.
And the question is, what's the material that minimizes the mass?
So there's a little short calculation we can do to figure that out.
And then I've got another plot, and you can compare different materials on this other plot.
Then you'll see how good wood is compared to other materials.
So from beam of a given stiffness and given span, and say it's a square cross-section, then the question is, what material minimizes the mass of the beam?
So the mass is just going to be the density times t squared times l And if it's a beam, say it's got some central load on it, a concentrated load, the deflection's going to go as pl cubed divided by some constant and divided by the Young's modulus and the moment of inertia I.
So the stiffness, if I just rearrange this, the stiffness, p over delta, that's going to go as p over delta CEI, and I's going to go as t to the fourth over l cubed.
And then I can solve that for t squared.
And I want t squared because I'm going to plug it back into the equation for the mass.
So if I solve this for t squared, I've got my stiffness p over delta.
I've got l cubed divided by CE.
And then I take that whole thing to the 1/2 power like that.
And then I plug the t squared back into the little equation for the mass.
So I've got density minus p over delta times l cubed over CE.
And we'll take that whole thing to the 1/2 power-- [INAUDIBLE] another l. And so to minimize the mass, you want to look at the material properties.
And here, the material properties are the density and the Young's modulus.
And to minimize the mass, you want to minimize rho over E the 1/2 power.
Or conversely, you want to maximize E to the 1/2 over rho.
So if you just had a bar that you were just pulling on, you would just want to maximize E over rho.
But if it's a beam and bending, it works out that you want to maximize E to the 1/2 over rho.
And if we look at the next slide, this next slide then plots on a log-log scale, it plots the modulus on this axis and the density on that axis.
And here this plotted data for lots of different materials.
So there's engineering alloys.
Metals are up here.
Engineering ceramics are here.
Composites are here.
Polymers are down here.
Elastomer is way down here.
Foamy things, down here.
And this envelope here is woods.
And notice log scale here.
The lowest stiffness polymer foams here are 0.1 gigapascal, and diamond is up here at 1,000 gigapascal.
So there's like five orders of magnitude difference in the modulus here.
So then, if you look at the bottom right corner here, there's a bunch of old dashed lines.
And this red one here is E to the 1/2 over rho.
So if it's log-log plot, E to the 1/2 over rho's going to show up as a straight line.
And every point on that line has the same value of E to the 1/2 over rho.
And the material that would be the best for a beam of a given stiffness would be the one that has the biggest value of E to the 1/2 over rho.
And if I move the line up to the top left here, I'm increasing E. I'm decreasing rho.
It's got the biggest value of E to the 1/2 over rho.
So the materials that are on this line here, they all have the same value of E to the 1/2 over rho.
And they've got the biggest value-- well, virtually the biggest value.
So let's look at what those materials are.
There's things like engineering ceramics, like diamond that maybe are not the most convenient thing to make our beam out of, and tend to be brittle and might break.
So we have some issues.
There's engineering composite, so things like carbon fiber reinforced plastics.
And at this sort of tip of the composites, there'd be things like unidirectional fiber composites.
And then here's other woods down here.
So the woods have the same performance index, this is called, the same value of E to the 1/2 over rho as the best engineering composites.
And so they have very good properties for their weight.
And one of the interesting things is if you look at this performance index of E to the 1/2 over rho, this is the performance index for the wood.
This is for the solid cell wall material that the wood's made from, so E to the 1/2 of the solid over rho for the solid.
And from the modeling of the wood, just looking at the axial modulus, this thing here is equal to that times rho S over rho.
So if you look at this, this is the performance index for the wood.
This is the solid it's made from.
This number here is bigger than 1, right?
Because the density of the solid is bigger than the density of the wood, and so this is saying the wood is more efficient than the thing that it's made from, than the solid that it's made from.
And so that's the sort of plot for the stiffness.
And there's a similar plot for the strength.
That if you do the same little kind of calculation, you find that the performance index for the strength is some failure strength raised to the 2/3 power over a rho.
And again, here we're plotting strength versus density on a log-log plot.
And this red line here is the strength of the 2/3 over rho.
And again, if we scoot over here so we have a parallel line, every point on that line has the same value of the strength to the 2/3 power over the density.
And these are the materials that have the highest values.
And again, here's engineering composites.
These are ceramics.
But the ceramics, they have a high compressive strength, but they tend to be brittle.
So it's not really a practical strength.
These are metals in here.
And here's the woods down here.
So it's kind interesting just to see that the wood has such a good property.
Yes?
So I realize why this is valuable setting up the problem this way.
But if you're actually trying to design something, why would you want to fix your cross-section?
You could change your material and change your cross-section
So this is the starter version of this problem.
And there's another part two of the problem is to change the shape.
And you could look at what shape's efficient.
There's something called a shape factor that gives you the efficient shapes.
So you could take the material and turn it into a different shape and have a more efficient thing because it was a different shape.
So you can account for that
So then if you varied, like let's say you made your cross-section smaller, like even if it was still square, you could just still make it smaller
Yeah.
I'm saying we've got a given stiffness.
So if we're given a certain stiffness and a certain span, we would need a certain cross-section to get to that stiffness.
Are we happy?
OK.
So that's one thing.
Let's see here.
So let me just write a few more notes about the material selection, and then there's one more thing I wanted to show you about the woods.
Hmm?
C is just a constant.
So it's just a number.
So if you had a beam in three-point bending, then C would be three.
If you had a beam that was simply supported with a central load, C would be 48.
C is just a number
One more question.
So follow your line there, and the choice is really just about cost
No, it's not about cost.
There's nothing on cost here.
It's all really about the properties.
What's the best combination of properties to minimize the mass, and then which material has that combination of properties.
You can do charts like this that include cost.
You can make these charts with whatever property
[INAUDIBLE].
I guess maybe there's a difference off two or so of strength
Between pine and balsa?
Yeah, maybe more than that.
I think-- I can't quite see where-- pine's close to 100, and balsa's, I don't know, 20 or something
[INAUDIBLE]
Yeah, and it's not-- the point of this isn't so much looking at the absolute value of a strength.
It's looking at the value of this performance index.
And what you want to do is maximize that index to get the material that's going to minimize the mass.
So let me just read a couple notes about this.
So we have these-- these are called material selection charts.
So you plot the log of one property versus the log of some other property.
And then we have a line of constant E to the 1/2 over rho.
I'll just say it's shown in red because you're going have the same plots.
And the materials with the largest values are in the upper left.
So the woods have similar values to engineering composites.
And you can do a similar thing for strength.
So I have a few more minutes.
So I have a few minutes, and I want to talk about a couple of uses of woods.
So one is in old ships.
So I don't know if you know Professor Lechtman has this course Materials and the Human Experience, and they talk about sort of ancient uses of materials.
And I did a section, a module, on woods and the use of woods in old colonial ships, like The Constitution that's in Boston Harbor.
So this is kind of a schematic of an old ship.
And the thing that was interesting and the thing I talked about in this module was that people chose particular species for particular parts of the boat.
And they would choose a particular species depending on its properties.
And a lot of the hull was made of oak.
So oak's a very dense wood.
But they would get something they called straight oak, and they would get something they called compass oak.
And you can see this little thing down here, this little kind of schematic here, this little sketch.
This is straight oak, just a straight trunk.
And this thing here would be the compass oak.
And what they would do is they would use the straight oak for straight parts of the boat, so something like this, these pieces here.
And then they would actually look for trees that had the curve of the branches to match some part of the boat that they were looking for.
So, for instance, if you have the hull out here and the deck here and they had their cannons here, there's something called a knee, which is sort of a bracing piece that goes between the deck and the hull.
And that bracing piece is curved.
And they would actually look for trees in which the branches curved at the same kind of curvature as they were looking for in that piece.
And then they would use it for that piece.
And the advantage of this is they basically had the grain running along the curve, and so they got the best properties out of the wood by doing that.
So they had this straight oak and compass oak, and that was one cute thing.
And often they used white oaks.
And I brought a piece of white oak in.
You can see how dense it is.
And the US Navy often used something called live oak.
Live Oak grows in the South.
Anybody from the South?
You see these big trees with huge sort of spreading branches.
Those are the live oaks.
And apparently, the US Navy, I read somewhere, still has a forest somewhere with live oak for doing things like repairing The Constitution.
So let me just pass those guys around.
So those are a couple of oaks they would use for the hull.
Then they would use white pine for the masts.
And the reason they used white pine for the mast is because the white pine grows very, very tall and very straight.
And white pine was actually like a strategic resource in the 1600s, the 1700s.
And it turns out that when the British Royal Navy was doing all that colonial stuff in the 1600 and 1700s, Britain actually ran out of trees for masts for boats, and they would actually import masts from New England.
And there were these people called surveyors who would go around and they would mark certain trees that were supposed to be saved for these masts for the British Royal Navy.
And the thing was that the size of the boat and how many cannons you could put on the boat depended on how big the mast was.
So the size of the boat depended on the mast, because the mast height controlled how much sail area you could get.
So the taller the mast, the more sail/ the more sail, the bigger the ship.
The bigger the ship, the more cannons.
And so having these tall Eastern white pines was a sort of a strategic resource.
And I have a piece of white pine.
Unfortunately, my dog got to this one.
And be careful.
It's a bit splintery.
But you can see it's a lighter kind of wood.
And if you go around New England, if you go to the arboretum, you can see white oak.
You can see Eastern white pine.
The other wood they used is lignum vitae, that first dense one that I passed around.
And if you notice that lignum vitae has kind of a waxy feel to it.
And they used that in the block and tackle, so like pulleys and stuff like that.
And it was thought to be self-lubricating because of that kind of waxy layer on it.
And because it's very dense, if you think of like a block and tackle and you've got like a rope going over a pulley, you've got a pressure from everything sort of fitting together and the bits bearing against each other.
And the fact that was very dense made it very good for the block and tackle.
And so they used the lignum vitae for that.
And there's one other cute story about lignum vitae.
I don't know if any of you've ever read Dava Sobel's Longitude.
Anybody read that?
I'm a sucker for those history of science books.
So her book Longitude is about the development of an instrument to measure longitude.
Originally, they could get the latitude from the stars, but they were really bad at getting the longitude.
And so boats would go off, and they wouldn't really be able to figure out where they were, until they had a method to measure longitude.
And there was some British board of something or another.
They put forward a prize for somebody who could produce a way of measuring longitude accurately.
And there was a guy called John Harrison, and he built a clock.
He built a very accurate chronometer.
And if you knew when sunrise was and sunset was, and you knew the time and where you left, you could figure out where you-- it's kind of like time zones.
You could figure out where you are today.
And he built a chronometer, and one version of his chronometer used a lignum vitae for the same reason, because it was very dense, and it was very stable.
And the clock that he eventually won the prize with was in the 1700s, 1759.
I think they went on some trip with it.
It was 81 days at sea, and it lost five seconds over 81 days.
So that's pretty impressive for 250 years ago.
So that was the lignum vitae in the clock.
I have one more picture, and then I can finish up the thing on wood.
And we'll start the cork next time.
So this is another example of using wood.
And this is sort of a more modern use.
So this bridge here is made with a glue-laminated wood.
So this big beam here, the big arch, is made up of sections of wood which are glued together.
And you can glue the sections in a curved shape if you want.
They sort of have molds to do that.
And when they make this glue-laminated wood, they cut the defects out.
So they cut knots out, and they control the pieces of each laminate that they use to get the best quality.
And the glue-laminated wood actually has better properties than just two-by-fours or whatever you would cut down, lumber that you would cut from a tree.
So glue-laminated wood is kind of a nice kind of wood structure that's used now.
And you see it all the time in things like ice rink arenas, like large spans.
It's kind of beautiful.
You can see the wood grain in the curve in the wood grain when they make these things.
So that's the wood lecture.
I'm going to stop there.
So next time I'll talk about cork.
I just have a little bit about cork.
And then we'll start talking about foams.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I wanted to talk a little bit about cork partly because cork is kind of interesting.
Cork has a structure that's a little bit like one of those honeycomb things.
What I'm going to do is I'm just going to talk and go through the slides.
I'm not going to write the notes on the board.
There's only a few pages of notes, and it's in the Stellar site.
I'm not going to write notes for this because it's just really for fun.
This first slide starts off with historical uses of cork.
Cork was used by the Romans.
They used it for the soles of sandals, the same as we do.
And they used it for stopping bottles of wine, the same as we do.
But they didn't realize that you could just use the cork.
They would take the cork, put it in the wine bottle, and then they would use pitch which is a tarry stuff from a tree.
They would use the pitch and seal the bottle with the pitch.
In the 1600s, there were some Benedictine monks that realized that you could just use the cork and not use the pitch.
They were the ones who really perfected the use of corks in wine bottles for sealing it.
So what is cork?
Cork is the bark of a tree called Quercus suber, the cork oak.
Here's a piece of the cork bark.
All trees have a layer of cork in them.
But the thing that's different about Quercus suber is that it's very thick and the corks are obtained from this thick layer.
Quercus suber is a Mediterranean type of tree.
It grows, Portugal is the main place that exports cork, but also places like Algeria, Spain.
You can grow it in California.
The cork is kind of unusual because-- let me scoot onto the next slide-- it's kind of unusual.
So here's a little picture.
I went to Portugal when I was a graduate student doing this project.
Here's the cork tree here.
Here's the little mini we rented.
Here's the cork being harvested here.
Cork's unusual because you can remove the bark from a cork oak tree and it regrows.
So most trees if you did this, you would kill the tree.
But cork doesn't get killed by doing this.
What happens is they plant trees.
You have to wait something like 10 or 15 years for the tree to get big enough to harvest the cork.
And then the first harvest is poor quality and they don't use that.
And then you have to wait another 10 years or so before you can actually harvest the cork.
So you can imagine if a cork orchard or forest gets chopped down to build a skyscraper, or apartment buildings, or something, it's not an economically feasible thing to plant cork trees these days.
So when the trees are cut down, they don't tend to get replanted at this point.
And there's a number of artificial substitutes for corks.
You've probably seen wine bundles with foam plastic corks in them too.
OK.
The reason it's called Quercus suber is that the cell walls in this particular type of cork oak are covered with a waxy substance called "suberine."
That's where the Quercus suber-- "Quercus" is "oak."
Every oak is Quercus something.
"Quercus alba" is white oak.
Quercus just means oak.
If we look at the structure of the cork, we can see that it's got these different views of the cork that are seen here.
This is a drawing by Robert Hooke in the 1600s from his book Micrographia.
He was the first person to really draw cork like this.
You can see he drew sort of boxy cells on this side.
And then, the other perpendicular plane, the cells have this structure here, sort of more rounded.
And here's a little sketch he's got of the cork tree.
Over here are SEM pictures.
These two planes here correspond to this plane in Hooke's drawing.
This plane here corresponds to this plane here.
This over here is Hooke's actual microscope.
I think the Royal Society still has that microscope that Hooke used in the 1600s.
Some of you know that Hooke wrote this book called Micrographia.
He got one of the first microscopes.
He looked at a lot of different materials, and he drew these beautiful drawings.
He wrote a page or two about each of the drawings.
Harvard has a first edition of Micrographia, and I made a little video on it.
So I thought I'd show you the video because the drawings are beautiful.
But the url's on the slides, and so you can watch it yourself.
There's more on this and on how he came to be so good at making scientific apparatus, how he came to do the Micrographia book.
At the end of it, there's a comparison of a number of his drawings with modern SEM images of the same thing.
He had this very famous picture where he draws a flea.
Don Galler, the person who runs the SEM for me, I had him put a flea in.
And he has essentially the same kind of image.
The thing that's spectacular is you can see how much of the detail that Hooke was able to capture in his drawings.
There's some really beautiful drawings.
OK.
So let's go back to cork.
Hooke was the first person to use the word "cell" to describe biological cells, and he described the cell in cork.
That's the structure looking at the SEM micrographs and his optical micrographs.
These are just some more higher resolution, higher magnification, images.
One of the things that you can see is that the cork has these little corrugations on the cell walls.
See those little wrinkles?
All the cell walls have those little wrinkles.
This plane here is the perpendicular plane.
If you look down into the cells, you can see those blurry things.
Those are the corrugations in the cell walls.
That's the structure.
Here's a schematic.
Here's the cork tree.
The cork is the layer just beneath the bark.
This is a picture of how the cells are oriented relative to their radial, and tangential, and axial directions.
You can think of them as roughly hexagonal.
They've got these little corrugations.
This is a schematic of an individual cell.
We measured the dimensions of the cells, and these are some average dimensions here.
Typically, the cells are tens of microns long, and the cell wall is about a micron thick.
Something like that.
OK?
One of the things we're going to see is that corrugated structure gives rise to some of the interesting properties of cork.
If we load cork and just do mechanical tests on it-- this is just a uniaxial stress-strain curve.
You can see the stress-strain curve looks like all these other curves we've seen for honeycombs.
There's a linear elastic part here.
There's a stress plateau here.
And then there's a densification part here.
Typically, the relative density of cork is around 0.15.
Something like that.
It densifies at a strain of something less than 0.85.
It's a stress-strain curve.
And when we did our little project on cork, we measured the properties in the three directions because in one direction, it's roughly hexagonal cells.
That plane is isotropic.
The e1,e2 plane is isotropic.
This compares the measured values of the properties versus what we calculated from the honeycomb model.
And really, we just used the sorts of equations that we talked about in class over the last couple of lectures, apart from loading in the x3 direction because in the x3 direction, you've got those corrugated walls.
And you have to take those corrugations into account.
So there's another complication that I'm not going to go into.
But there's a sort of modification you can do to account for that.
But you can see there's actually pretty good agreement here between the elastic moduli that we measured and what we calculated.
The compressive strengths down here are not quite so good.
They're off by a factor of two, more or less.
But they're in the right ballpark for the cork.
So those are some of the structures, some of the properties.
One of the interesting things about cork is what it's used for.
The uses of cork really exploit the mechanical properties.
Obviously, it's used for stoppers for bottles.
I brought a champagne cork along with me, and I brought a couple of other little pieces of cork here.
I'll pass those around in a minute.
One of the things to look at is just the still wine cork, which is the one on the right, and the champagne cork, which is the one on the left.
If you notice the still wine cork is just made of one piece of cork that's cored out from the bark.
And if you notice these little channels.
These little channels here are called "lenticels."
On the still wine cork, they go this way.
And on the champagne cork, they go that way.
They're oriented perpendicular.
It turns out that they are normal to the isotropic plane in the cork.
If you look at the champagne cork, this plane here is the isotropic plane.
If you think of that being put into a bottle, I think part of the reason they orient it this way is because it gives you a uniform compression against the neck of the bottle and gives a nice seal.
So that's one of the things about corks.
Another thing that's interesting is that cork has a Poisson's ratio equal to zero if you load it in that direction.
Let's see.
Did I not bring that picture?
Maybe I didn't bring that picture.
Hang on a sec.
Nope.
I guess I didn't.
Sorry.
I thought I brought a picture where I had the deformation of the cells when you load it in that direction.
When you load it-- so say the cells are corrugated this way on.
When you load it that way on, it's like having a bellows and folding up a bellows, or unfolding a bellows.
So when you load it that way on, there's no expansion or contraction this way on.
And so you get zero Poisson's ratio.
And if you think of trying to get the cork into the bottle, it's rather convenient to have zero Poisson's ratio because you don't get as much expansion.
As you're pushing it into the bottle, you don't get as much expansion that way out, pressing against it.
In fact, if you compare wine corks with rubber stoppers, this is a kind of typical rubber stopper.
Wine corks are always just cylinders.
In fact, even the champagne corks are cylinders when they start off.
When they put it into the bottle, it's just a straight cylinder.
It gets deformed like that from being in the bottle for some time.
They have straight sides and you can just squeeze them.
There's these funnel things that wine makers have for putting the cork in the bottle.
You can just squeeze them into the bottle top.
You can do that because the Poisson's ratio is zero and because the Young's modulus and the bulk modulus are both small.
But if you look at a rubber stopper, rubber stoppers always have these tapered sides to them.
And that's because the Poisson's ratio of the rubber is 0.5.
As you squeeze it in, it's trying to move out this way.
You couldn't get the stopper in unless it had those tapered sides.
So that's sort of an interesting thing about cork.
Let's see.
Another application of cork is it's used for gaskets for the same sorts of reasons.
It's relatively compliant.
It takes up any slack between two pieces that you want to press together.
It's often used for musical instruments that come in pieces.
Things like clarinets, there'll be a piece of cork-- you a clarinet player?
Yeah?
Yeah.
One of the interesting things about the clarinet is that, if you can see here at the ends, there's a piece of cork there.
And I think there's a piece of cork down there.
And the other pieces mate up with that.
The cork provides a seal.
And the way the cork is cut, it's cut in such a way that those lenticels go radially out like this, which means that the plane of isotropy and the direction that's got the zero Poisson's ratio is that radial direction.
When you're squeezing, say, the second part onto it, the cork does not expand circumferentially.
So as you're squeezing it this way, it doesn't expand that way.
It doesn't wrinkle or anything on your other part.
They use the cork in a particular orientation for that reason, I think.
So it's used for gaskets It's also used because it's got a good friction property.
It's got a property that is taken advantage of in things like flooring and shoes.
Cork has a high friction even if it gets wet.
Some sources of friction are from adhesion, from a surface effect.
Then if, say, the floor gets wet, then you break that, and it could be slippery.
But the source of friction in cork is from an energy loss and dissipation as you're deforming it.
Imagine you have a wheel here.
The wheel is rotating on this cork floor.
And here, a piece of cork is getting deformed as the wheel rolls over it.
As it gets deformed, there's some histeresis loop.
Cork has quite a lot of damping in it.
There's quite a lot of energy lost in that histeresis loop.
What that means is that's characteristic of the cork itself.
It's not a surface effect.
That means that if you use it for floors or for shoes, it doesn't lose that friction and damping when it gets wet.
Here's some measurements we did of the coefficient of friction for cork versus doing it dry and doing it with a liquid, soapy surface.
You can see the soap doesn't make any difference.
Cork is seen as a very attractive material for things like flooring.
It's actually not a cheap material to make your floors out of, but it's an attractive material for flooring.
Part of the reason it's used for floors and for the soles of shoes is because of these friction properties.
Another feature of cork is that it's got very small cells compared to a lot of engineering polymers.
The cells are on the order of tens of microns, whereas many polymer foams, the cells are hundreds of microns or millimeters.
We'll get into this later, but this plot here is really saying that the thermal conductivity of a cellular material depends, in part, on the cell size.
The cell size for foam plastics is in here.
And that for cork is down here.
Because it has a smaller cell size, it has a lower thermal conductivity.
Cork was at one point used to some extent for thermal insulation.
If you go to Portugal, where cork comes from, there's hermit caves.
There were these old hermit, religious people who had holed up in a cave.
And they would line the caves with cork to try to make it a little more insulated, a little more warm.
The other place you see this is if you look at cigarettes, you know cigarettes have that little brown tip on the part that touches your lips?
That's meant to look like cork.
And if you look, it has little dots on it.
The little dots are the little lenticels.
Apparently, they used cork originally in cigarettes as a sort of thermal insulation between the cigarette and your mouth.
So it was used for that too.
And then, one final thing.
Cork's also used, obviously, for bulletin boards.
If you push a pin into cork, then you get this local deformation here.
Here's our pin.
And here's cells locally deformed.
When you pull the pin out, you'll recover some of that deformation because the deformation is elastic.
So the hole will partly close.
So that's my little spiel on cork.
And that's just because it's interesting.
There's no test on cork or anything like that.
OK.
So are we good with cork?
Any questions?
We're OK?
OK. Let me scoot out of there.
Then the next part of the course, I wanted to talk about foams.
Let me just park the cork thing.
Let me pass these corks around.
So you can play with those too.
Oops.
There's little bits of cork.
There you go.
There's the champagne cork, the rubber cork, some little cork layers.
OK.
So the next part of the course, I wanted to talk about foams.
And I want to talk about how we model the mechanical behavior of foams.
If we look at the stress-strain curve for foams, these are some examples for foams made out of different materials with different characteristics.
The polyurethane and the polyethylene here are examples of elastomeric foams, really.
This one here is an open-celled foam.
This one's a closed-cell foam.
Polymethacrylamide is a polymer that has a yield point.
Mullite is a ceramic.
You can see the shapes of these curves resemble the shapes that we saw for the honeycombs.
Right?
They look exactly the same, in fact.
And the mechanisms of deformation in the foams are very similar to the honeycombs.
Even though the foams have a much more complicated geometry, we can use some of the ideas from the honeycombs to understand how the foams behave.
So that was part of the reason for doing the honeycomb analysis.
Let me back up.
These curves here were all in compression.
These curves here were all in tension.
So again, these ones in tension also look like the curves for the honeycombs.
Remember, in tension, we don't get any elastic buckling.
So if the foam was made of an elastomer, we don't see any stress plateau.
If the foam is made of material with a yield stress, then we get a very short yield plateau because of a slight geometrical difference between pulling and compressing the foam.
And if it's a brittle material, then we just get fracturing.
There's going to be some fracture toughness that's going to characterize it.
We can look at the deformation and the failure in these foams and look at the mechanisms.
And what we're going to do is model the mechanisms and not worry too much about the cell geometry.
So we're going to use dimensional arguments here.
Here's a foam in compression.
It was compressed from the top to the bottom.
And you can see this strut that's circled in red.
This is unloaded.
And then, this is after some load.
You can see this has bent somewhat.
And then, you can see this vertical strut here.
As the load gets larger, you can see that strut's buckled.
In an elastomeric foam, you get bending and buckling just the same as we did in the honeycomb.
Then here's a metal foam.
You form plastic hinges in the metal foam.
So here's a cell wall here.
And it's a little bent to start with at zero load.
But you can see it becomes more bent in this image over here.
And here's a cell wall that's more or less vertical.
And you can see that wall buckles.
It's a plastic buckling in this case.
There's a permanent deformation there.
Here's a brittle foam.
And you can see cell walls in this foam fracture.
So this region here is equivalent to this region here.
That little glitch there is the same as that little glitch there.
You can see there's a couple of cell walls here that are fractured.
So we get fracture.
The idea is is that the mechanisms of deformation in the foams parallel those in the honeycombs.
OK?
All right.
So let me write some of this stuff on the board.
We're going to start off by talking about open-celled foams, so foams where there's just solid in the edges, but not in the faces of the polyhedral cells.
Then we'll talk about close-cell foams where there's solid in the faces, as well.
But the open-celled ones are easier.
So we'll start with that.
In compression, we see the same three regimes as we did before for the honeycombs.
There's a linear elastic regime that corresponds to bending of the cell walls.
There's a stress plateau.
And for elastomeric foams, that corresponds to buckling.
For metal foams, that corresponds to the formation of plastic hinges.
And then, for ceramic or brittle foams, that corresponds to brittle crushing, so fracturing of the cell walls.
Then, if you load the foam up to higher strains and higher stresses, eventually you get to densification.
And in tension, just like the honeycombs, for the elastomeric materials there is no buckling.
We can get a stress plateau from plastic hinges if there's, say, a metal foam.
And for a brittle foam, we would get a fracture toughness and brittle fracture in tension.
So the idea is the mechanisms of deformation and failure just parallel what we've seen in the honeycombs.
So we'll start off with the linear elastic behavior.
And we'll start with open-cell foams.
The initial linear elasticity is due to bending of the cell walls.
And if the thickness of the cell edges relative to the length is small, the bending dominates the deformation.
But as the thickness to length ratio increases, then axial deformations can become important too.
What we're going to do is we're going to consider dimensional arguments.
We're going to set the dimensional arguments up so that we replicate the mechanisms of deformation and failure.
But we don't worry too much about the cell geometry.
What I'm going to start with is considering a cubic cell.
And I've arranged the cubic cell so that the cell edges are staggered.
That's going to give me the bending.
The edge length is going to be L. I'm going to say we have a square cross-section, t squared.
Here's our idealized model here with a cubic cell.
All the members have a length, l. All of them have a square cross-section, t squared.
That's an open-cell model.
We've got just solid on the edges and nothing on the faces.
The idea is that if I bend that, or if I load that in compression, so I apply, say, a stress out here that puts forces on those members there, because I've staggered these cell walls with these ones here, we're going to get bending in this cell edge here.
That bending is going to be what we model.
I'm going to set this up so that one thing is proportional to something else.
These relationships are going to be true regardless of the cell geometry.
So I could have picked a tetrakaidecahedra if I wanted to, and I would have had these same relationships.
I'm just picking a cubic thing because it's easier to think about.
So first of all, we look at the relative density.
Remember, the relative density is the volume fraction of solid.
So it's the volume of the solid over the total volume.
And that's going to go as t squared l over l cubed, or just t over l all squared.
You remember for the honeycomb, the relative density went linearly with t over l. For the open-celled foam, it goes as t over l squared.
The moment of inertia in this case is going to go as t to the fourth.
Remember, we have a square section, t squared.
So if it's bh cubed over 12, b is t, h is t. It's going as t to the fourth.
Then what I'm going to say is the stress is going to go as F over an area length squared.
OK?
So if I look at my little square thing here, I look at having my force here.
Here we have a force f. And it's acting over an area that's somehow related to l squared.
Right?
I don't know exactly what that constant is, and I'm going to not try to calculate that.
But it goes as F over l squared.
Similarly, I can write that the strain is going to go as delta over l. So the strain is going to go as this bending deflection here, that delta divided by the height of the cell.
And that's also l. Then I also know from structural mechanics that delta is going to go as Fl cubed over E of the solid and I.
Then I'm just going to put all these things together to get the modulus.
The modulus of the foam is going to go as the stress over the strain.
If I plug in what I have for the stress, it's F over l squared.
If I plug in what I have for the strain, it's delta over l. So this is F over l and delta.
I'm going to replace delta by Fl cubed over Es.
I'm going to use, instead of I, I'm going to use t to the fourth.
Then the F's are going to cancel out.
I've got that the modulus goes as the modulus of the solid times t over l to the fourth power.
Then I can put that in terms of the relative density.
It's going to go as the relative density squared.
So I can summarize all of this by saying that the Young's modulus of the foam is some constant C1, I'm going to call it, times the modulus of the solid times the relative density squared.
OK.
So this has the same kind of form as those equations we had for the honeycombs.
Right?
There's a solid-cell wall property.
The solid module's here.
For the honeycombs, I put it in terms of t over l. But the t over l was related to the relative density.
How much solid you've got is reflected in the relative density.
And then, this constant C1 wraps up all the geometrical constants that I've said, one thing's proportional to another, and something else is proportional to another.
C1 just wraps up all of those.
OK?
I'm just going to say here C1 includes all the geometrical constants.
We have to get C1 by looking at data.
If we look at data for the Young's modulus, we find that C1 is just about equal to one.
People have also done more sophisticated analyses than this.
There's a group of people who looked at doing a full-scale, structural analysis of an open-celled tetrakaidecahedral cell.
Remember, I said they pack to fill space.
So you can look at a unit cell.
They also made their cells such that the thickness along the length of the cell was not constant.
The thickness varied as something called a "plateau border."
If you have a foam that's made by surface tension, the edges will tend to have these plateau borders.
And the thickness will vary along the length of the edge.
So when they did all this whole, complicated thing, they could calculate a value for C1.
They calculated 0.98.
So it's very close to 1.
I'll say analysis of open-cell tetrakaidecahedron cells with these plateau borders give C1 equal to 0.98.
OK.
So that's the Young's modulus.
We can also look at the shear modulus.
The shear modulus is also going to be controlled by bending of the cell walls.
And so the shear modulus is just going to be some other constant times Es times the relative density squared, so a similar kind of relationship.
It's just a different constant.
And if the foam's isotropic, and if the Poisson's ratio is a third, then C2 is equal to 3/8.
Remember, if we have isotropy, then the shear modulus is equal to E over 2 1 plus nu.
And so you can get the C2 from that if you say nu is a third.
Then we can also get Poisson's ratio for the foam.
If the foam is isotropic, so we'll say for an isotropic foam here, nu is equal to E over 2G minus 1.
That's just rearranging this expression here.
And because E and G both depend on the relative density squared, they both depend on Es squared, that's all going to cancel out.
So this is going to be equal to C1 over 2 C2 minus 1.
So that's going to equal to a constant.
That constant's going to be independent of whatever material the foam is made from and the relative density.
The constant just depends on the cell geometry.
Remember, in honeycombs we found the same thing.
The Poisson's ratio for the honeycombs only depended on the cell geometry.
It didn't depend on the solid properties.
It didn't depend on the relative density.
So this is an exactly parallel thing here for the foams.
Yeah?
I have a silly kind of question.
What is the difference between foam and the honeycombs?
Oh.
The honeycombs have cells in a plane, and they're prismatic in the third direction.
And the foams have polyhedral cells.
You know what a tetrakaidecahedron, a 3D, polyhedral cell.
OK?
The honeycombs are prismatic.
And the foams have polyhedral cells.
OK?
Are we good?
OK.
So there's a couple more interesting things about Poisson's ratio.
The same way we can make honeycombs with negative Poisson's ratios, we can also make foams with negative Poisson's ratios.
They do it the same way as for the honeycombs, really.
The honeycombs had negative Poisson's ratios if the cell walls looked like this, this sort of arrangement.
So that sort of a thing.
We said that theta was negative for the honeycombs.
And if you invert the cell walls on a foam, you also get negative Poisson's ratios.
And the way they do that is they take a thermoplastic foam, and then, they load it hydrostatically.
So they compress it in all three directions.
And they smush the cells in on each other.
And then, they heat it up to a temperature above the glass transition temperature while it's still smushed.
And then, they cool it down.
So they end up with that structure frozen in.
And if they do that, they get a negative Poisson's ratio.
I'll just say they invert the cell angles analogous to the honeycomb.
They load the foam hydrostatically and heat to a temperature above Tg.
And then, they cool and release it.
I have a photograph here of a foam with a negative Poisson's ratio.
You can see how the cells have been smushed in.
It's the equivalent of the way it's done for the honeycomb.
OK?
Are we good?
OK. That's the linear elastic moduli for open-celled foams.
The next thing I wanted to do was closed-cell foams.
If we look at a closed-cell foam, we can idealize it in this kind of a way here.
I've set it up so that the edge thickness is different from the face thickness.
That's really representing the fact that in foams, many foams are made using a liquid, and the foaming is controlled by surface tension.
Often, the surface tension draws material into the edges and away from the faces.
So the faces tend to be thinner than the edges.
When we have deformation of the closed-cell foam, we've got bending of the edges the same as we did for the open-celled foam.
But the faces can stretch.
So they can have an axial stretching.
You can think of that as a cell membrane stretching here.
So imagine if I either pull on the foam or I compress it, there's going to be some axial load in the faces.
So when we analyze them, we have to account for both bending of the edges and axial deformation in the faces.
I'll just say we have edge bending as in open-cell foams.
And we also get a face stretching.
Another thing that can happen in the closed-cell foams is we can get compression of the gas.
In an open-cell foam, the gas can move from one cell to the next.
But in a closed-cell foam, the gas is trapped.
And as the volume of the cell changes, the gas gets compressed.
So we have another effect here.
So we'll say for polymer foams, surface tension tends to draw material to the edges during processing.
We define two thicknesses, one for the edge and one for the face.
And then we apply a force to this cubic structure.
And we can do an analysis a little bit like what we did for the open-cell foam.
Let me rub all this off.
OK.
I'm going to set this up a little bit differently than for the open-celled foam.
We're going to do a work argument.
We're going to look at the external work done by the force F going through a deformation, delta.
And that's going to have to equal the internal work done by the edges bending and by the faces stretching.
So let me set that up.
We're going to say the external work done, that's going to be proportional to F times delta.
So delta is how much the whole thing is going to deform.
Then, I've got internal work from bending of the edges.
That internal work is going to be proportional to F over delta times delta squared.
I'm going to end up with an expression where everything's in delta squared.
So I want to keep the delta squared there for now.
And F over delta is the stiffness.
That's going to go as E of the solid times I over l cubed.
I is going to go is Te to the fourth here because it's I of the edges.
I've also got internal work from stretching of the faces.
That internal work is going to be-- I'm going to run into my other equations here.
Let me put it down a little.
That's going to go as the stress on the face times the strain in the face times the volume of the face.
Or I could write that, instead of stress of the face, I can put it in terms of Hooke's law and say it's E of the solid times the strain in the face squared times the volume of the face.
And I can replace the strain in the face by delta over l. So it's delta over l squared.
And then, the volume of the face is going to be t of the face times l squared.
Then I want to balance the internal work and the external work.
I can say F times delta is going to equal some constant I'm going to call "alpha" times E of the solid times t of the edge to the fourth, that's this guy up here, over l cubed times delta squared plus some other constant I'm going to call "beta" times E of the solid times delta over l squared tf l squared.
So far, I've got this in terms of the force I'm applying.
But I want to get a modulus of the foam out of this.
I want to relate this force here to the modulus of the foam.
I can say the modulus of the foam is going to be related to F over l squared, that's the stress, divided by the strain, delta over l. I can write the force is proportional to the modulus times the deflection, delta, and times the member length, l. And then, I'm going to plug that guy into this expression up here.
And I'm going to get a delta squared on the left.
And I'm going to get delta squareds in each of the right-hand terms.
And so I'm going to cancel out the deltas.
If I put all that together, I have the modulus in the foam times delta squared times l. There's a delta here, and there's a delta there.
That gives me delta squared.
And that's going to equal alpha Es Te to the fourth over l cubed times delta squared plus beta times Es times delta squared tf, if I cancel out one of those l squareds.
So here I can get rid of the delta squareds in all these terms.
And if I just divide by l, I'm going to have the modulus of the foam.
I get a term here that it goes as alpha E of the solid times t of the edge over l to the fourth power plus beta times E of the solid times tf over l. Are we good?
OK. And what I'd like to do is instead of putting it in terms of te and tf, I'd like to put it in terms of the relative density.
I'm going to look at two limits.
I'm going to say if we just had open cells, if there were no faces on the membranes, if we just had open cells, and we had a uniform t, then the relative density would go as t over l squared.
If I just had closed cells, and I had a uniform t, then the relative density just goes linearly with t over l. The relative density is the volume fraction of solids.
It's the volume of the solid over the total volume.
For a closed-cell foam with a uniform t, the volume of the solid is going to be t times l squared.
And then, the volume total is l cubed.
So it's t over l. Now, I'm going to define one more thing.
If I say that phi is equal to the volume fraction of the solid in the edges, then I can say te over l is some constant times phi to the 1/2 times the relative density to the 1/2.
And I can say tf over l is equal to some other constant, c prime, I'll call it, times 1 minus phi, that's how much is in the faces, times the relative density.
Then I could combine all of this.
Can I put it in here?
Maybe I'll just stick it down here.
Hang on.
OK.
Put it up here.
This is my final expression here.
And this term here arises from the edge bending.
This term here arises from the face stretching.
So I think I should wait a bit for you to catch up.
The idea is the edges are bending.
The faces are stretching.
We're looking at the work done by deforming the edges and the faces and equating that to the work done by the externally applied load, f. OK?
That gives us two terms, one that accounts for the edge bending, and one that accounts for the face stretching.
To be comprehensive, we want to take into account the compression of the gas, as well.
So there's one more term I'm going to add on to that.
Typically the gas effect is only significant for elastomeric foams.
If you had a metal foam or a ceramic foam that was closed-cell, it wouldn't really contribute much.
But just to be complete, we'll go through this.
The idea here is we say we've got a cubic element of the foam.
Initially, it has a volume, v0.
If we apply a stress, a uniaxial stress, there's some change in the volume of the foam.
So there's some volume, v, after we compress it by some strain, epsilon.
If we can figure out what the volume is relative to the initial volume, we can figure out how the change of the amount of gas goes from the initial state to the compressed state.
Then we can use Boyle's law.
The idea is P1V1 equals to P2V2.
Then we can figure out, using Boyle's law, what the pressure must be.
And then, that pressure is what's contributing to the modulus.
When I do this, I'm going to write down some equations that just have the main results.
There's a whole bunch of algebra just to get from one to the other.
And I'm not going to write them all down.
When I write the equations, don't panic if it's not obvious how you get from one to the other.
In the notes that I'm going to put online, there's all the details of how you get from one to the other.
The idea is that we start with a cubic element of foam.
Initially, before it's loaded, it has a volume V0.
Then we apply a uniaxial stress.
And we say the axial strain in the direction of the stress, I'm going to call epsilon.
Just from the geometry, you can calculate the deformed volume.
So after you load it, the volume is V. And that volume on loading, V, divided by the initial volume, V0, you can show.
It's fairly straightforward.
It's 1 minus epsilon times 1 minus 2 times the Poisson's ratio of the foam.
Here, I'm taking compressive strain as positive.
And if you do this whole volume thing, you'll get terms in epsilon squared and epsilon cubed.
But because if it's linear elastic, epsilon is going to be small.
I'm going to ignore the epsilon squared and the epsilon cubed terms.
That's the total volume of the foam after and before the compression.
And then, what I want is the volume of the gas.
And the volume of the gas is just going to be the volume minus the volume of the solid.
And I can get that by just subtracting off the relative density.
So Vg over Vg0.
Again, Vg0 is the volume of the gas initially.
Vg is the volume of the gas when I'm compressing it.
Remember, the relative density is the volume fraction of solids.
So I'm essentially subtracting out the amount of solid to get the amount of gas.
Then we can use Boyle's law.
Here, p is going to be the pressure after the strain and p0 is going to be the initial pressure.
The building seems rather making unhappy noises for some reason.
I'm not sure what that's all about.
There'll be some initial pressure in the gas even before the strain, or before the stress is applied.
That's p0.
So the pressure we need to overcome is p minus p0.
Again, I'm missing out a bunch of steps.
But using that expression and this expression and this expression, you could find that p prime is equal to p0 times epsilon times 1 minus 2 nu divided by 1 minus epsilon 1 minus 2 nu minus the relative density.
Then the contribution of the gas to the modulus you can get by just taking the derivative of that pressure with respect to the strain.
So remember, modulus is stress over strain.
It's the same kind of idea.
So I'm going to call it E star g for the contribution from the gas.
So that's going to dp prime d epsilon.
And that's going to equal p0 times 1 minus 2 nu over 1 minus the relative density.
OK?
As I said, I'll scan the pages that have the details and put it online.
The final expression we get combines all of these things.
The modulus of the foam relative to the solid is phi squared rho over rho s squared plus 1 minus phi, that's the amount of material in the faces, times the relative density and then plus this gas compression term.
Like I said, for most foams, the gas compression is negligible.
So if p0 is equal to the atmospheric pressure, so about 0.1 megapascal, then the gas term's negligible, except for elastomeric foams.
So that's the Young's modulus.
Then we can do a similar thing for the shear modulus.
And the thing to notice in shear is that when you shear something, there's no volume change.
So if you shear it and there's no volume change, there's no gas compression.
There's no pressure built up because there's no change in the volume.
So for the shear modulus, you just have the first two terms.
And if the foam is isotropic and you use a third, then this constant here is going to be 3/8.
And then, Poisson's ratio is just going to be a function of the cell geometry, again.
And roughly, we could say it's going to be around a third.
OK. Are we good?
Let me wait a little bit.
So the idea is we're looking at the deformation of the bending in the cell edges, and the stretching in the cell faces, and the gas compression.
And we're accounting for those different terms.
The next thing is to compare these model equations with some data.
And here's data for Young's modulus.
Here, we're plotting the relative Young's modulus, so the modulus of the foam divided by the solid against the relative density.
Again, these are log-log scales.
On this plot, everything with open symbols is an open-cell foam, and everything with filled symbols is a closed-cell foam.
Here's our equation for the open-celled foams, the simplest thing that goes with density squared.
And that's that thick line there.
And you can see these are all open-cell foams.
There's some more up here.
So that gives a reasonable description of that data.
And then, these are two lines for closed-cell foams for different values of phi, so different values of the amount of solid in the edges.
And you can see all the filled symbols are the closed-cell foams.
And they're in between this line here and this line here, basically.
So that gives a fairly good description for the modulus of the foams considering how crude this model is.
We're not trying to account for the cell geometry.
We're just modeling the mechanisms of deformation and failure.
And then, here's a similar plot for the shear modulus.
Here's for open-cells, 3/8 times the relative density squared.
These are open-celled foams down here.
These are closed-cell foams up here.
If the amount of material in the faces is small, then you would just get the shear modulus varying with the relative density squared.
Then here's data for Poisson's ratio.
We don't really know what the constant is because we don't know what all these geometrical parameters are.
But here's a value of a third.
And you can see there's a lot of scatter here.
This is like less than 0.2.
This is more than 0.5.
And the scatter really represents differences in the cell geometry.
If the foams were, say, anisotropic, and the cells were stretched in one direction, then you would get different values of the Poisson's ratios.
So that's the Poisson's ratios there.
I'm thinking I'm going to stop there because that seems like enough equations for today.
And then, next time we'll start doing the stress plateau and we'll figure out the elastic collapse stress from buckling and a plastic collapse stress from yielding, and so on, and so on.
We'll finish doing the modeling next time.
And we'll probably start doing a little bit of other stuff on foams next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
And I really wanted to show you my little hook video and I downloaded it so I thought we'd start just by watching that and then I'll pick up about modeling phones.
So this takes like nine or 10 minutes, but I just thought it was cute.
And I made it and I want you to see it.
So let's do that to start.
[VIDEO PLAYBACK] We're here at the Harvard University Botany Library, looking at a first edition of Robert Hooke's Micrographia, published-- How do I get rid of the bar, Greg?
Oh, there it is.
show the microscopic structure of materials.
And it has a number of remarkable drawings in it.
Here we see drawings of silk.
These are two different silks.
On the top here, we have a fine-waled silk.
And in this more details drawing down here, you can see the patterned weaving of the silk.
The bottom image here is a drawing of watered silk.
And over here, there's another higher magnification image.
And you can see the pattern here is more sharply angled.
And it appears that this sharper angle here gives the different texture to the surface finish of the silk.
So here we see a drawing of charred wood.
And one of the things I find interesting about this drawing is how similar it is to modern electron micrographs which we've seen before.
And in this drawing, we can see two of the main features.
We see these small cells, which are fibers that provide structural support to the tree.
And we see these larger cells here, which are vessels which allow fluids to go up and down the tree.
And here we see a drawing of the surface of a rosemary leaf, with the unexpected, tiny, little bars.
And this is something that you can only see with the microscope.
You wouldn't expect to see those when you just feel the surface of the rosemary leaf.
So it's kind of interesting that with the microscope, you can see these features that are invisible to the naked eye.
One of the main themes of material science is that the property of materials are related to their structure.
And so being able to see the structure at a microscopic scale is very helpful.
And today, we can even see the structure at the atomic scale.
Robert Hooke understood this idea.
And in the description of the cork, Hooke states, "I no sooner discerned these-- which were the first microscopical pores I ever saw-- but methought that I had with the discovery of them, perfectly hinted to me the true and intelligible reason for all of the phenomena of cork."
So what he's saying here is that by looking at the structure and looking at the cells here in the drawing, he thinks he can understand the properties of cork or the phenomena of cork.
What was it about Robert Hooke that allowed him to make this book?
Why was it him and not somebody else?
Well, Robert Hooke had kind of an interesting history.
He grew up on the Isle of Wight.
And as a boy, he loved making drawings.
And he got quite skilled at making drawings.
The other thing was, he loved making models of things.
He made models of ships.
He made a wooden clock that was a working clock when he was a kid.
And as a teenager, he moved to London.
And he became an apprentice to Sir Peter Lilley, who was a famous painter of the time.
So his drawing was good enough that he would be working with a very well-known painter.
After he did that, he went to the Westminister School.
And he studied classics.
He studied mathematics.
But he also learned to use a lathe.
And this was also very helpful in him making various sorts of apparatus.
And as a student at Oxford, he worked in the lab of Robert Boyle.
And his job in that lab was to develop scientific apparatus.
And he did things like he built pumps that allowed Robert Boyle to do the experiments that led to Boyle's Law.
When he returned to London after Oxford, he became the Curator of Experiments at the Royal Society.
And one of the things he did was he got a microscope.
He improved that microscope, increasing their magnification, which was what allowed him to make the beautiful drawings that we see today.
And here in the preface of the book, we see that he even made a drawing of his microscope.
So this thing down here-- this is Robert Hooke's microscope.
The development of new microscopes with higher and higher magnifications continues to this day.
Scanning electron microscopes were invented in the 1960s.
And today, we have transmission electron microscopes and atomic force microscopes with even higher magnifications.
At these higher magnifications, we can see details that Hooke was unable to see because of the limitations of the microscope that he had-- the optical microscope.
But it's interesting to see today the images we see in a scanning electron microscope at a similar magnification to those that Hooke saw in his optical microscope.
And it's remarkable to see how many of the features that we see in these much more fancy microscopes that he was able to capture in his drawings with his simple optical microscope.
So here we have a picture of cork.
We have Hooke's drawings showing two perpendicular planes.
We also have this nice, little drawing of a cork branch here.
Cork is the bark from the cork oak tree.
And in Hooke's drawings of the microstructure, we can see these cells here are roughly box-like.
They're more or less rectangular.
And these cells here look more or less circular.
So there's these two different perpendicular planes in the cork.
And when we look at these scanning electron micrographs, we can see very similar structure.
There are some cells that are roughly boxlike, and others that are more or less hexagonal or roughly rounded.
One feature that Hooke was not able to see, though, that you do see on the scanning electron micrographs, is the waviness in the cell walls.
And that was because the resolution of his microscope was insufficient to see that level of detail.
And here in this illustration on the bottom here is a drawing of sponge.
And when we look at the scanning electron micrograph, we see that the structure is remarkably similar to what Hooke has drawn.
So here we have Hooke's drawing of feathers.
And we can see he's made several drawings at different length scales.
And if we look at this one here, we see the barbule.
And you can see these little hooked regions there.
And those hooks lock into the little feathers over on this side over here of the adjacent barbule.
And in the higher magnification picture, you can see on one barbule, there's hooks on one side but not on the other.
And it's this hooking of the two sections together that allows the feathers to maintain a smooth surface for the wing when the bird is flying.
And you can see the same sort of thing when you look at the electron micrograph.
So you can see the little hooks on one side of the barbules.
And you can see how they interconnect together with the next barb.
One of the most reproduced images from Hooke's book is that of the flea-- this image we see here.
And you can see why.
It's a gorgeous image.
And it shows details that people had never seen before.
People were amazed to see that the little flea that they might have found on their dog or something was actually made up of this compound body, with all these little plates and little hairs here.
And you can see these little tiny claws on the legs, and the legs have all these hairs.
Nobody had any idea that this is what a flea actually looked like.
And so it was an amazing drawing.
And it was something that people were just stunned by when Hooke's book came out.
And if we look at a modern electron micrograph, we can see it's remarkably similar if we look at the same magnification.
So Hooke showed many of the same details, showed some of the same hairs on the legs, showed the same sorts of plates, showed the claws at the ends of the legs.
And our modern image is probably from a different species of flea.
We don't know what species of flea that Hooke actually looked at.
But you can see there's a tremendous similarity between the two images.
And it's remarkable how many of the features that Hooke was able to capture in his drawing.
And here we have the compound eye of the fly.
And this, again, was astonishing to people in Hooke's day.
And even today, people look at this image, and they're pretty amazed at the detail in this drawing.
And again, we can compare this with a modern electron micrograph.
And again, you can see the similarities between what Hooke saw and what we see in a modern scanning electron microscope at a similar magnification.
In the 1980s, atomic force microscopes were invented, which have a resolution down to tens of nanometers.
And today, there's transmission electron microscopes, which allow you to see the atomic structure.
So for instance in a crystal lattice, you can see the individual atoms and the regular crystal structure.
Today, most experimental studies of materials include photographs of the microscopic structure of the material taken through some sort of microscope.
And the remarkable thing is that all of these studies really trace back to this book here that we're looking at today-- to Robert Hooke's Micrographia.
[END PLAYBACK] There you have it.
So I just thought that was kind of cute.
You might enjoy that.
So that was that.
All right, let's get out of there.
Stop.
So let's go back to the foams.
So I think last time, we got as far as talking about the linear elastic behavior of foams and modeling that.
But we didn't quite get to looking at the compressive strength of the foam.
So I think we got as far as comparing the models with these equations here, and these plots of the data.
And what I wanted to pick up with today was looking at the compressive strength.
And we'll look at the fracture toughness as well in tension.
So we're going to start with nonlinear elasticity and the elastic collapse stress.
So if we have an open-cell foam, the derivation for the elastic collapse stress is really pretty straightforward.
We say the elastic collapse occurs when the cell walls buckles.
So in this schematic here, you can see the vertical cell walls have buckled.
And so there's going to be some Euler load that's related to that buckling.
And that's just the usual Euler load-- n squared, the n constraint factor, pi squared E. This is going to be E of the solid, I over l squared-- the length of the member.
And then the stress that corresponds to that is just going to be proportional to that buckling load over the area of the cell, which is just l squared, so just P critical over l squared.
So that just goes as Es.
I is going to go as t to the fourth, because we have that square sectioned member.
And now this is going to be l squared.
And that's an l squared.
So that's l to the fourth.
And so if I combine all of that together, I can say that the elastic buckling stress is going to be some constant-- and I think we're up to C4 now-- times the Young's modulus of the solid times the relative density of the foam squared.
So that's our equation for the elastic buckling stress.
And if you compare this with data, you can make an estimate of what C4 is.
And we find that C4 is about 0.05.
And you can also say that 0.05 really corresponds to the strain at which the buckling occurs.
Because the Young's modulus goes as the constants 1 times Es times the relative density squared.
So the strain's just going to be the stress over the modulus.
So that does correspond to the strain.
So that's saying that buckling compressive stress occurs at a strain of about 5%.
So that's open cells.
And then if we look at closed cells, if you recall when we looked at the moduli we looked at a couple of extra terms.
One was associated with face stretching for the modulus.
And the other was associated with the compression of the gas.
For the buckling, the faces don't really contribute that much, because typically the faces are very thin relative to the struts.
And because they're so thin, they buckle at a much lower load, and they don't contribute too much.
So we're not going to worry about that contribution.
So I'm just going to say that the thickness of the face is often small compared to the edges.
And that really is from the surface tension in processing that draws material away from the face and into the edges.
There can be some contribution from the internal pressure.
So if the internal pressure is greater than atmospheric pressure, then the cell walls are pre-tensioned, and you'd have to account for that.
So the buckling would have to overcome that pressure as well.
So then you would have the buckling stress would just be what we have up there-- C4 times Es times the relative density squared.
And then we just add on that factor P0 minus P atmospheric.
The thing with the gas which tends to affect more than the buckling stress, though, is the post-collapse behavior.
So let me just show you a couple of things here.
So here's some data for the elastic collapse stress.
And you can see on the y-axis, we've got the stress normalized by the Young's modulus of the solid.
And on the x-axis, we've got the relative density.
And that solid line there-- sort of solid, dark line-- is that equation there, which is the same as this one up here.
And you can see the data lie fairly close to that.
But what's interesting is if you look at the-- why is this not working?
Maybe my batteries finally died.
If we look at the post-collapse behavior, you can see if these are the stress-strain curves, they're not flat here.
They have some rise to them.
And this is a closed-cell foam.
And you can imagine as you're compressing the closed-cell foam, you're reducing the volume of the cell.
And as you doing that, you're increasing the pressure inside the cell from the gas.
And you can calculate what that is.
And I'll do that in a second.
And if you subtract off that gas pressure contribution, that works out to this line here.
Then these lines will be more flat, like this.
And we already really pretty much worked out that gas contribution.
So I'll just say for the post-collapse behavior, the stress rises due to the gas compression.
And that's as long as the faces don't rupture.
So if you have an elastomeric foam, typically they don't rupture.
And what we had worked out before was that that pressure-- we called it P prime-- it was P0 minus P atmospheric-- that was equal to P0 times the amount of strain, epsilon, times 1 minus 2 times the Poisson's ratio divided by 1 minus epsilon times 1 minus 2 nu minus the relative density.
And once you get to the buckling stress, then the Poisson's ratio becomes 0.
So if you take a foam-- so I brought a little foam in so you can play around with this one-- so if you take a foam like this and you compress it, once you've buckled it like this, it's not getting any wider this way.
And part of the reason for that is you've got all these pores in here.
And the cells just collapse into the pores.
They don't really need to move out sideways.
So you can smush that yourself, and try to convince yourself that the Poisson's ratio is just 0.
Yes, Matt
[INAUDIBLE] I guess I want to measure [INAUDIBLE] the gas contribution?
Yes, so there is a strain rate effect with these things.
But I wasn't going to get into that here.
If you look in the book, it's described in the book.
So I think there's two things.
One is that the solid itself can have a rate dependency.
And then there could be something connected
[INAUDIBLE]
Yeah, I mean, I'm not going to go into that here.
But one could look at that.
So let me just write down one more thing here, because if we let nu be 0, then this thing here becomes simpler.
So we could say the stress post collapse as a function of strain would be our buckling stress and then plus this factor here.
So that curve on the bottom over here-- if this is the stress-strain curve-- this little dashed line here-- that's the gas contribution.
And that is this term here.
So you can kind of see how the shape of the curves reflects that gas contribution.
And when you subtract it out, you get pretty much a horizontal plateau over here.
Are we happy?
Yeah?
[INAUDIBLE]?
This is for the closed cell.
Because the closed cell are the ones that are going to have the gas pressure.
If it's open cells, the gas can just move out of the cells
[INAUDIBLE]?
Oh, sorry, that was to show you that the Poisson's ratio was 0.
And that's true for both of them.
So then we can look at the plastic collapse stress.
Say we had a metal foam.
And we do a calculation a little bit like the one we did for the honeycombs, too.
So we say the failure occurs when the applied moment equals the plastic moment.
And the applied moment is proportional to the applied stress times the length cubed.
So I'm going to call that applied stress-- our strength sigma star plastic times the length cubed.
So if you think of, say, the little schematic up here, the force is going to go with stress times the length squared.
And the moment's going to force times the length.
So it's the stress times the length cubed.
And then the plastic moment goes as the yield strength times the thickness cubed.
And then if I just combine those, I get that the plastic collapse stress in compression is another constant-- I'm going to call it C5-- times the yield strength times the relative density to the 3/2 power.
And if we look at data, we find that the constant is about equal to 0.3.
And if I go to the next slide, here's a plot of the yield strength or the plastic collapse strength of the foam divided by the yield strength of the solid, plotted against the relative density.
And that dark, bold line is this equation here.
And you can see the data lie pretty well on that line.
There's one data set that's a little bit above the line.
But you can see the slope of that data set is still about 3/2.
OK, and the same as in the honeycombs, we could say that we can get elastic collapse before the plastic collapse if we were at a low density.
You can get the same thing in the foams.
And you calculate out what the critical relative density is for that the same kind of way.
So we can say we can get elastic collapse precedes the plastic collapse if the elastic buckling stress is less than the plastic collapse stress.
So all we do is make those two things equal to figure out the critical relative density where you get the transition from one to the other.
So the relative density has to be less than 36 times the yield strength of the solid over the Young's modulus of the solid squared in order to get buckling before yielding.
And let's see, where can I put that?
So for rigid polymers, that ratio of the strength of the solid over the modulus of the solid is about one over 30.
And so the critical relative density for the transition is about 0.04.
So you'd have to have a pretty low-density foam, but it's possible.
And for metals, that ratio is about 1/1,000.
And then the critical transition density is less than 10 to the minus 5.
So essentially, it never happens for the metal foams.
And then for the closed-cell foams, we could include the terms for face stretching and for the gas.
But in practice, the faces don't really contribute very much.
And typically for foams like say metal foams or a rigid polymer that had a yield point, the faces rupture.
And then if the faces rupture, then you don't get the gas compression term, either.
So I'll just write the full thing down.
But typically, you don't need to use it.
So the first term would be from the edges bending.
And the second term would be from the faces stretching.
And this would be from the gas.
But in practice, the first term is really the only one that is significant.
So for closed-cell foam, this equation works pretty well, too-- the same one as for the open-cell foams.
OK, so if we had, say, a ceramic foam that was brittle, there'd be a brittle crushing strength.
And then we get failure when the applied moment M is equal to the fracture moment Mf.
And this works very similar to the plastic yield strength.
So we find the applied moment goes as the global stress times the length cubed.
And the fracture moment goes into the cell wall strength times the cell wall thickness cubed.
So the brittle crushing strength goes as another constant-- let's call it C6-- times the wall strength times the relative density to the 3/2 again.
And C6 is about equal to 0.2.
And typically, ceramic foams have open cells.
So I'm just going to leave it at the open-celled formula there.
So there's one last thing for the compressive behavior, and that's the densification strain.
And we just have an empirical relationship for the densification strain.
So if you compress the foam and you get to very large strains, then the cell walls start to touch, and the stress starts to rise steeply.
And there's some strain at which that occurs.
And we call that the densification strain.
And in the limit, the modulus at that point would go to the modulus of the solid.
If you could completely squeeze all the pores out, the stiffness of that would go to the modulus of the solid.
And you might expect that that densification strain is just 1 minus the relative density, but it actually occurs at a slightly smaller strain.
So in a large compressive stress, or strain, I guess we could say, cell walls touch, and we start to get this densification.
So the modulus in the limit would go to the modulus of the solid.
And you might expect that the densification strain was just equal to 1 minus the relative density.
But it occurs at a little bit less than that.
So empirically, we find that it's just 1 minus 1.4 times the relative density.
And then I have this plot here, which is really just fitting a line to that data for densification strain.
So those equations describe the compressive stress or the compressive behavior of the foam.
So we've got the moduli, we've got the three compressive strengths, and we've got the densification strain.
So what we're going to do later on in the course is we'll use those models to look at how we can use foams and things like sandwich panels and looking at energy absorption.
And we'll also look at these equations in terms of some biomedical materials-- looking at trabecular bone, and looking at tissue engineering scaffolds.
So there's one last property I wanted to go over, and that's the fracture toughness.
So if we were pulling the foam in tension, and we had a crack in the foam, we'd want to know what the fracture toughness would be for a brittle foam.
And this follows the same sort of argument as we had for the honeycomb.
So all of these equations really are just following the same kinds of arguments.
But you can kind of see how having the honeycomb calculations makes it easier to do the foam ones.
So we'll do the fracture toughness calculation, and then I want to talk a little bit about material selection and selection charts for foams.
So that's less equation-y.
OK, and we're just going to look at open cells here.
So imagine we have a crack of length 2a.
And we have some remote stress applied, so remote tensile stress, so I'm going to call that sigma infinity-- the far-away stress.
And then we have a local stress on the cell walls.
I'm going to call that signal local.
So I have a little schematic that kind of shows what we're doing.
So we're pulling on it.
There's some crack.
The crack length is large compared to the cell size.
And we want to know what the fracture toughness is.
So we can say from fracture mechanics the local stress is going to be equal to some constant times the faraway stress times the square root of pi a over the square root of 2 pi r. And that's at a distance r from the head of the crack tip.
And if we look at our little schematic here, we could say it's hard to say exactly where the crack tip is, but it would be somewhere in here.
And we'd say this next unbroken cell wall is a distance r ahead of the crack tip.
And that r is going to be related to l. It's going to be some function of l. So I can say the next unbroken wall ahead of the crack tip at some distance r is going to be related to l. And that's subject to a force, which is going to be the local stress times l squared.
So that force is going to go as local stress times l squared.
And the local stress-- I can substitute this thing here in-- that's going to be proportional to the faraway stress.
And I'm going to get rid of the pi's.
And I'm going to substitute for r. I'm going to put in l. So it's going to be proportional to the faraway stress times the root of a over l and times l squared there.
And then we're going to say, again, the edges are going to fail when the applied moment equals the fracture moment.
And the fracture moment is going to go as the modulus of rupture of the cell walls times t cubed.
And the applied moment is going to go as f times l. And I've got f from up there, so that goes as the faraway stress, sigma infinite, times the root of a over l. And now I've got l cubed, because there's an l squared there and there's an l down here.
And then if I just equate those, then this is going to go as sigma fs times t cubed, like that.
So then I can say the fracture strength is equal to my faraway stress.
That's going to go as my modulus of rupture times the root of l for a times t over l cubed.
And then my fracture toughness is going to be this tensile stress times the root of pi a.
So there's going to be some other constant here, which I'm going to call that C8.
We've got the modulus of rupture of the solid.
I've got the square root of l, and I'm going to multiply it by pi so it's like other fraction mechanics kinds of equations.
And then we multiply that times the relative density to the 3/2 power.
And here, if we look at data, we find that that constant is about equal to 0.65.
And here's another one of these plots.
So here I've normalized the fracture toughness of the foams by the modulus of rupture of the cell walls times the root of pi l. So I've taken the cell size into account here, and I've plotted against the relative density.
And that equation there is the same as this equation I've got down on the board.
And this is the only one of the properties that we've looked at that depends on the cell size.
There's a cell size dependence here.
All right, so I think that's all the modeling of the foams.
Are we good?
I gave you a lot of equations.
We're good?
All right.
So I want to talk about how we might design foams to improve their properties.
And then I want to talk about how we might select foams for certain applications and look at selection charts.
So when we've been talking about the foams, especially the open-cell foams, we've been saying their deformation is largely by bending of the cell edges.
And if we could do something to increase the stiffness of the edges or the strength of the edges, then that would increase the overall properties of the foam.
And there's a couple of ways to think about doing that.
So the foam properties-- if the foam is controlled by bending of the edges, and the edges have some flexural rigidity, EI, if we could increase that EI of the edges, we would increase the properties of the foam.
And one way to do that is by making the edges hollow.
So if we had hollow edges, and you had a tube, then that would increase the EI.
And we can work out how much it's going to increase them.
And I have a little example here of-- a natural example of hollow foam struts.
So this is a grass.
I don't know what kind of grass it is.
I just saw this grass.
And we picked some different grasses, and we took some SEM pictures.
And it has a really kind of common structure for grasses.
It's very common for grass stems to have sort of a solid outer part and then a foam-like inner part.
It's so common that botanists have a name for it.
They call it the core-rind structure.
And if you take one of these grass stems, and you look at the sort of foamy bit in the middle, and you do a SEM picture of that, you can see that the little cell walls are actually little hollow tubes.
So one of these things-- it's a little hollow tube.
So what I wanted to do is work out how much the modulus of the foam would increase if you could make all the edges into little hollow tubes.
So we're going to start by saying the foam behavior is dominated by cell bending, so edge bending.
And the foam properties can be increased by increasing the EI of the cell wall.
So there's a couple of ways we could do that.
So the first one is looking at hollow walls.
So imagine I have a thin-walled tube-- just a circular, thin-walled tube.
There's my little wall there.
It has some radius little r, and a wall thickness t. And then imagine I have the same amount of mass, but now I have a solid circular section.
And I'm going to say the radius of that is big R. So for our thin-walled tube, the moment of inertia is pi r cubed times the thickness, t, if it's thin.
And for our solid circular section, I is going to be pi big R to the 4th over 4.
And if I say I want to set this up so that the masses are equal, then the areas of the cross-sections have to be equal-- say it's from the same material.
So the masses are going to be equal if pi R squared is equal to 2 pi r t. So I'm going to solve here for R. So the pi's are going to cancel out.
So the masses are equal if R is equal to the square root of 2 times r times t. And then what we're going to do is see how the big is the moment of inertia of the tube relative to the solid.
And the tube is pi little r cubed t. And the solid was pi R to the 4th, divided by 4.
And I'm going to get rid of the R here, and get rid of the pi's there.
So R to the 4th is going to be 4r squared t squared.
So the 4s are going to go.
And this boils down to r over t. So if I had a thin-walled tube, the moment of inertia is going to be r over t bigger than if I had the same mass in a solid circular section.
So you can see for the little plant here, by making a thin-walled tube, you're increasing the stiffness of the foam with the same amount of material.
That's the idea.
And you can do a similar kind of analysis for other properties.
So that's if we have hollow tubes.
So another option is we could have cell walls that are sandwich structures.
So imagine if the cell walls themselves were little, tiny sandwich structures.
So when you have a sandwich beam, what you have is too stiff, strong faces that are separated by some sort of porous core, like a honeycomb or a foam or balsa wood.
And the idea with the sandwich structure-- if I draw a little sketch of the sandwich, here's my faces.
So imagine those are solid.
So they might be aluminum sheets, or they might be fiber reinforced composites.
And then we have some sort of cellular thing here as the core.
And the idea is, that's analogous to an I-beam.
So in the sandwich beam, we have two, stiff, strong faces separated by a lightweight core.
So the core is typically a honeycomb, or a foam, or balsa wood.
And the idea is, you increase the moment of inertia of the cross-section with little increase in weight.
And if you think of an I-beam, an I-beam has a large moment of inertia, because you're separating the flanges by the web.
And the sandwich beam works in the same way.
You're separating the faces by the core.
But the core doesn't weigh very much, because it's a cellular thing.
So the faces of the sandwich are like the flanges in the I-beam.
And then the core is like the web.
So the idea is to make something called a micro-sandwich foam.
So what you want to do is make the cell walls into sandwiches.
And one way to do that is to disperse a large volume fraction of thin-walled spheres into the foam.
And you have to get the geometry right to make it work.
So let me draw a little kind of sketch here of how it works.
So here's our thin-walled spheres.
And then you're going to distribute those in a foam.
Here's another sphere over here.
The spheres are not perfect.
Let's say there's another one in here.
And then the idea is this stuff in here would be the foam.
So these guys are hollow spheres.
And say the spheres have a diameter D. And say they have a wall thickness here of t. And say that the separation of the spheres I'm going to call c. You can see that there.
And then the cell size of the foam I'm going to call e. So there's a bunch of parameters you have to kind of play with to get this to work.
So you have to have thin-walled spheres so the faces are thin.
The sandwich panels work best when the faces are thin.
So you need the thickness of the sphere to be much less than D. You need the faces to be stiff relative to the foam.
So you need the modulus of the sphere material to be greater than the modulus of the foam.
And you need the volume fraction of the spheres to be relatively high to get the spheres close enough together for this to work.
So you want that volume fraction to be something like 50% to 60%.
And for the foam, you need to have the foam cell size less than the separation between the spheres.
You need to have a number of-- you can't just have one pore in here.
That's not really like a foam.
It won't behave like a foam as a continuum.
So you need to have a number of different cell sizes in between each sphere.
And so you need the cell size of the foam to be a lot less than the separation of the spheres there, c. But if you can control this geometry, you can get the sandwich effect.
And you can get improved properties by doing that.
So there's ways you can play around with the structure of the foams to improve their properties.
So that was one thing I wanted to say.
Another way to improve the properties of a foam-like material is to use one of those lattice materials.
So we've been talking about ways to improve the bending stiffness.
But if you could get rid of the bending altogether and have axial deformation in the cell walls, that would be much stiffer.
And you can get axial deformation by having those 3D truss kind of materials.
So I have a picture of this.
There we go, so there's one of those 3D truss materials.
So another alternative is to sort of get rid of the bending altogether, and to try to make a truss-type material.
So there's various ways to make these.
I think that we talked about a few of them earlier on.
And you can analyze them as truss-type structures.
And I can just run through a sort of little dimensional argument to get the modulus.
So the modulus is going to go as the stress over the strain.
The stress is going to go as a force over a length squared.
The strain's going to go as a deformation over l. So this is just like what we had before for the foams.
But in this case, the deformation is going to go with the force times the length over the area of the cross-section divided by Es, because we're pulling it or pushing it axially.
So that goes as Fl over t squared Es.
And if I just put that back in the equation here for the modulus, I get that we've got F over l. And I've got delta here, so that's F l t squared Es.
And you just get the modulus goes as the modulus of the solid times t over l squared.
And that goes as the modulus of the solid times the relative density.
So for the open-celled foams, the modulus went as the relative density squared.
So if it was 10% solid, the modulus would be 0.01.
And this is saying if it's 10% solid, the modulus is 0.1.
So it's much bigger.
So this is all sort of well and good.
The only difficulty is that when you look at the modulus, you can do reasonably well.
But when you look at the strength, some of the members are going to be inevitably in compression.
When you have these truss materials, some members are going to be in tension.
Some members are going to be in compression.
And the compression members tend to buckle.
And once the compression members buckle, then you're back to the same kind of strength relationship that you have for the foam.
So that's one of the difficulties of this.
So let me say that the strength-- so if the strength was controlled by uni-axial yield, it would go linearly with relative density.
But if it goes with buckling, it goes as the square.
So I'll just say the compression members can buckle.
And say you had a metal lattice.
Then there's some interaction between the plastic behavior and the buckling.
And you use what's called the tangent modulus instead of just the Young's modulus.
And the tangent modulus is lower.
And there's also what's called knock-down factors that can be large, too.
So the knock-down factor can be like 50%.
So the measured strength can be half of what you thought it was going to be.
This should be a squared over here.
Sorry.
So even though the stiffness of these 3D trusses can be quite good, the strength often isn't quite as good as one might hope.
So that's one of the issues with them.
All right.
So do you see the idea, though, with all these different micro structures, is that you can control the structure in a way to try to increase the bending stiffness or get rid of the bending stiffness and increase the axial stiffness?
So there's things you can do to play around with that.
And I wanted to talk a bit today about material selection charts for foams.
So when we talked about woods, we started talking about this.
Remember, I derived a little performance index.
We said if we had a material and we wanted to have a given stiffness, and we wanted to minimize the mass, we had that performance index that was E to the 1/2 over rho.
And we had a chart of modulus versus density.
And we saw that wood was really good.
You can do that for other sorts of properties, not just modulus.
So you can make-- depending on what the mechanical requirement is, you can work out different performance indices.
So I want to go into that in a little bit more detail.
So the question is, how do we select the best material for some mechanical requirement?
So in the wood section, we looked at the minimum mass of a beam of a given stiffness.
And we saw that the performance index was E to the 1/2 over rho.
So let me do another one of these little examples, and then I'll show you some more of them.
So another example would be what material-- minimize the mass of a beam of a given strength or a given failure load.
So we'll call the failure load Pf.
And we can see the maximum stress in the beam is going to be the moment in the beam times the distance from the neutral axis y, and divided by the moment of inertia.
So here, M is the maximum moment in the beam.
And y is the maximum distance from the neutral axis.
And I is the moment of inertia.
And I'm going to say i goes as t to the 4.
And I'm going to define a failure stress of the material sigma f. So sigma max is going to go as my failure load times the length.
That would be the moment.
The distance from the neutral axis is going to go as t. And the moment of inertia is going to go as t to the 4th.
And that's going to be the failure strength there.
So I can solve this for t. And then I'm going to write the mass in terms of t, and put that in there.
So here t goes as Pf l divided by sigma f. And that's going to be to the 1/3 power.
I guess I can scoot over here.
Then we can say that the mass M goes as the density of times t squared times l. So the mass M is going to go as rho times l times t squared.
So that whole thing goes to the 2/3 power.
So if we look at the material properties, the mass goes as the density times the failure stress raised to the 2/3 power.
So if we want to minimize the mass, we want to minimize rho over sigma f to the 2/3, or we want to maximize sigma F to the 2/3 over rho.
So that's the performance index for that case.
So we can obtain these performance indices for different loading configurations and different mechanical requirements.
And I don't want to go through a whole lot of them, but I'm going to put this up with the notes.
So this is from Mike Ashby's book on Material Selection in Mechanical Design.
And this is a whole series of these performance indices for different situations, for things loaded in torsion, for columns and buckling, for panels and bending.
So these ones are all for stiffness.
And they all involve a modulus raised to some power divided by a density.
So a tie in tension, c over rho, the beam in bending is E to the 1/2 over rho.
A plate in bending is E to 1/3 over rho.
So you don't need to memorize those.
But you can see you can derive these for different situations.
And here's another one for strength-limited design.
So the shaft is, depending on what the specifications are, it's the strength raised to the 2/3 power over rho.
The beam loaded in bending-- the top one there-- sigma f to the 2/3 over rho.
That's what we just did.
So there's all these different kind of performance indices.
So depending on what your situation is, you would pick one of these indices.
And then what you can do is use these material selection charts, which plot one property against another on log-log scales.
And because all of these performance indices involve a power, they always end up being a straight line on your log-log plot.
And here this one, I think, is the same as what I showed you for the wood.
This one's the modulus here plotted against density.
So foams are down here.
And other engineering materials are over here.
And these guidelines here are the different performance indices.
So this one's E over rho.
This one's E to the 1/2 over rho.
This one's E to the 1/3 over rho.
And for this case here, as you move the lines up to the top left-hand corner, E is getting bigger.
Rho is getting smaller.
And so the actual value of the performance index is getting bigger.
So you can use this to select a material.
So we've made these charts for foams as well.
So here's a couple of charts for foams.
And I think what I'm going to do is just go through them quickly.
And there aren't really that many notes, so I'll just put the notes on the website.
And you can come and write all the notes down.
And then we can finish this today.
So this one here is the Young's modulus versus density.
And these are all sorts of different foams.
So the low modulus ones tend to be flexible.
The higher modulus ones tend to be more rigid.
And you could use this to select foams, if you wanted.
You can also see what the range of values is.
So the values of the modulus here goes from a little less than a 100-- because this is two orders of magnitude here, I think, each one of these-- down to about 10 to the minus 4 or a little less than that.
So there's a huge range.
There's almost a range of a factor of a million in those moduli.
And the same with the strengths here.
The strengths go from 10 to the minus 3 mega-pascals up to about maybe 30 mega-pascals, something like that.
And you can see for the modulus and the strength, things like the metal foams are good.
The balsa's good.
Here's the balsa up here.
Metal foam's up there.
So you can kind of see the range of properties that you could get.
And then you could also-- need a drink, hang on.
You can also plot the specific property.
So here's the compressive strength divided by the density plotted against the Young's modulus divided by the density.
And here you want to be up at this end.
So you would have a high strength and a high stiffness.
So the balsa and the metal foams are good up here.
This next plot-- this is the compressive stress at 25% strain.
And this is the densification strain.
And if you think of having your stress-strain curve looks like this, something like that, so you could say that's a strain of 0.25 and that's the stress that corresponds to that.
So that stress times the densification strain, which is out here someplace, is an estimate of the energy underneath the stress-strain curve.
So you can think of this right-hand plot here-- those dashed lines-- these lines like this and this and this-- each one of those corresponds to how much energy you would absorb under the stress-strain curve.
So points that lie on here would have an energy of 0.001 megajoules per cubic meter.
And over here, we're at 10 joules per cubic meter.
So again, the balsa and the metal foams are good over here.
So you can use these plots to try to identify foams for particular applications.
And I think there's a couple more.
It doesn't have to be mechanical properties.
Here is thermal conductivity versus compressive strength.
So you can imagine if you wanted some insulation, you wanted to have a certain thermal conductivity value, you probably also need at least some minimal compressive strength.
You could also have something like a maximum service temperature, that maybe the foam is going to melt at some temperature.
You can't go beyond that.
So there's some property there.
And I think there's one more here.
You can look at things like the density in terms of the buoyancy of a foam, if you have some buoyancy application.
And you can look at cell size on this one here.
And cell size can be important for things like filtration and catalysis.
So the amount of surface area goes as 1 over the cell size-- the surface area per unit volume.
And so the cell size can be important for those sorts of applications.
So the idea is, you can make these material selection charts for foam.
And you can put data on there.
And you can compare foams.
And you can use these performance indices.
So I'm going to leave it at that.
There is a little bit more notes.
But I'll just put them on the website, and you can get them from there.
So I think we're good for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu
So what I wanted to do today was talk about thermal properties of foams.
And foams are often used for thermal insulation.
And that's always closed cell foams that are used for thermal insulation.
And we'll see why.
And the foams tend to have a low thermal conductivity.
And that's largely because gases have lower conductivity than solids.
And if you have mostly gas, you're going to have a lower conductivity.
So they have a low conductivity because they have a high volume fraction of gas.
And they've got a low volume fraction of the solid.
They also have cells.
And the heat is transferred partly by radiation and convection.
And if you have small cells, you reduce the amount of convection and radiation.
And we'll see that.
So that, by having a cellular structure, and in particular, by having small cells, you can decrease the heat transfer.
OK, so let me write some of the stuff down.
So closed cell foams are widely used for thermal insulation.
And the only materials with lower thermal conductivity than closed cell foams are aerogels gels.
And I'll I talk a little bit more about aerogels later on today.
But the difficulty with aerogels is that they tend to be very weak and brittle, like ridiculously weak and brittle.
So we had a project on aerogels a couple of years ago.
And the students who I was collaborating with would make aerogels.
And they'd bring it up to my office.
And I would pick it up and like-- I would pick it up like this, and it would break.
So they have very low thermal conductivity, but they're very brittle.
And I brought a few of our samples of aerogels, just so you can see what they look like.
And I'll pass them around in that little tube, so you can kind of play with them.
OK, so we're going to focus on foams.
And whoops-- And we can say the low thermal conductivity of foam arises mostly from the high volume fraction of gas and that the gas has a low lambda, a low thermal conductivity.
So lambda is thermal conductivity, so I'm just going to put lambda there.
Then it has a small volume fraction of solid, which has a higher thermal conductivity.
And then the foams have a relatively small cell size.
So one of the things we're going to look at is how does the cell size effect the thermal transfer, the thermal conductivity.
OK, so there's lots of applications for foams.
And I guess one of the main ones is in buildings, insulating buildings-- also insulating refrigerated vehicles, things like LNG tankers.
So there's lots of the applications for using foams for thermal insulation.
Foams, in addition to having a low thermal conductivity, they also have good thermal shock resistance.
So thermal shock is if you have a material, you heat it up, and then you suddenly cool the surface of it, for example.
So say you takes something and you quench it in water or quench it in some fluid, then the surface, it wants to shrink because its temperature drops, but it's connected to everything underneath it and it can't really shrink.
And so it's constrained and you can get cracking and spalling.
And so, it turns out foams have a good resistance to that thermal shock kind of loading.
And we'll see why that is, too.
Roughly, you can see if the thermal expansion strain is the thermal expansion coefficient times the change in temperature.
And the stress that you might generate is just going to be related to the modulus times alpha times delta-t. And because we're going to see that alpha for the foam is the same as alpha for the solid, but the E foam is going to be a lot less that E of a solid would be.
So because the modulus is smaller, you would get a better thermal shock resistance.
OK, so I wanted to go over a couple of sort of laws of heat conduction, so we can talk about what thermal conductivity is and how we define it.
So the first one here-- --first one here is for steady state conduction.
So when we say steady state conduction, what we mean is that the temperature is constant with time, the temperature doesn't change with time.
So time's not going to come into the equation here.
And heat transfer for steady state conduction, where there is no change in the temperature with time, described by Fourier's Law.
And that says that the heat flux q is equal to minus lambda times the gradient in temperature.
And if you want to think about just a one diversion of that, it's equal to minus lambda times dt by dx.
So here, q is our heat flux.
So that would have units of joules per meter squared per second.
So how much heat transfer per unit area per unit time.
Lambda is the thermal conductivity.
And it has units of watts per meter k, so degrees kelvin.
And then delta or-- and then this is our temperature gradient.
OK, so that's Fourier's Law, and we're going to use that later on when we talk about the foams.
And then, just so that we have things a little more complete, if you have a non-steady heat conduction, if the temperature varies with time, then there's a difference equation that involves the thermal diffusivity.
So if we have non-steady heat conduction-- so t varies with time.
I'm going to call a time tau.
Then a partial differentiation, the partial derivative of temperature with respect to time is equal to the diffusivity, that's given the symbol a, times the second derivative of temperature with respect of distance, so with respect x squared.
So here a is the thermal diffusivity.
And it is equal to the thermal conductivity divided by the density and divided by the specific heat.
So here, rho is the density and cp is the specific heat.
The specific heat is the heat required to raise the temperature of a unit mass by the unit temperature.
And so, the density times cp is the volumetric heat capacity.
It's how much energy you would need to raise a certain volume by, say, 1 degree k instead of a certain mass.
OK, so on the table here, on the screen, we have different materials.
And we have the thermal conductivity lambda.
And we have the thermal diffusivity, a.
And I guess I should also say a has units of meters squared per second.
So this table is arranged in order of decreasing thermal conductivity.
So here's copper at the top, 384, watts per meter k. Here's, you know, different metals.
You've got aluminum.
Here's a couple of ceramics.
They're about a factor of 10 less than the metals.
Here's the polymers, another factor of 10 less than that.
And here's some gases.
Air is about 0.025.
Carbon dioxide is less than that.
Triclorofluoromethane, which used to be used as a gas in foams because it's got such low thermal conductivity, is 0.008.
But it's no longer used because it's a-- what you call it?
A fluorocarbon.
Anyway, it decreases our ozone layer.
So they don't use that anymore.
Now here's some wood.
So that's one sort of cellular solid.
And they're around 0.04-- something like that.
And here's a group of polymer foams.
And they're a little over 0.025.
So if you think of-- if you had the gas, air-- if the air was the gas inside the foam, 0.025 is lambda for the gas.
So you're not going to get lower than that.
And you have to use a low conductivity gas to get these values, like 0.025 here, 0.020, 0.017.
And then, hear some other sorts of sort of mineral fibers, glass foams, glass wools.
OK, so that's just a table so that you have some data there.
All right?
Yes?
[INAUDIBLE] --foams, if they are closed cell, with a different gas rate.
Because if they're open cell--
--Right.
The gas is going to--
--it would just always be air
It's going to go.
And in fact, one of the difficulties with using the lower conductivity of any gases is there's a phenomenon called aging, that if, you know, you've got your gas inside your foam, it's going to diffuse out into the air.
And air's going to diffuse in.
So over time, the thermal conductivity tends to increase because you're getting air coming and the local-- conductivity gas going out.
But I think, typically, that process takes a number of years.
It doesn't happen in a week.
But if you're designing a building and want the building to be there for 50 years, it occurs faster than that.
So it's not ideal from that point of view.
All right.
So let me talk a little bit more about thermal diffusivity.
Let me scoot over here.
So materials with a high value of that thermal diffusivity, a.
They rapidly adjust their temperature to their surroundings.
So if they have a high value of a, what it really means they've got a, say, a high value of lamda-- so high thermal conductivity.
And, say, a low value of this volumetric heat capacity.
So it doesn't take much energy to change their temperature.
And they also conduct heat well.
So they tend to adjust their temperature to their surroundings quickly.
OK, so then, let's talk about the thermal conductivity of a foam.
So I'm going to call that lambda star.
So the star is the foam.
And then we'll talk about-- lambda s will be the lambda for the solid that it's made from.
So if you think of the thermal conductivity of the foam, there's contributions from different types of heat transfer.
So you could have conduction through the solid.
I'm going to call that lambda s. You could have conduction through the gas.
You could have convection within the cell.
So convection has to do with having, say, within the cell, it might be a different temperature on one side of the cell to the other side of the cell.
And the warmer side of the cell, the gas is going to tend to rise to the warmer side and fall to the cooler side.
And you get a convection current set up.
So you can get heat transfer from that.
And you can also get heat transfer by radiation.
So radiation can cause heat transfer, as well.
So we're going to have contributions from conduction through the solid.
So the amount of conduction in the foam from the solid-- I'm going to call lambda star s. So lambda s would be the conductivity of the solid.
And lambda star s is the thermal conductivity contribution from the solid in the foam.
So we get kind of-- through the solid.
We have conductivity through the gas.
So it's lambda star g for gas.
And then we could have convection within the cells.
We'll call that lambda star c. And then we could get radiation through the cell walls and across the voids.
We'll call that lambda star r. And so, the thermal conductivity of the foam is just the sum of those four contributions.
So we're just going to go through each of those contributions, in turn, and work out how much thermal conductivity you get from each of them.
And it turns out most of the thermal conductivity comes through the gas.
So if we first look at just conduction through the solid, we've got that contribution to the conductivity of the foam from the solid, it's just equal to some efficiency factor times the thermal conductivity of the solid times the volume fraction of the solid or the relative density.
And here, eta is an efficiency factor.
And it accounts for that tortuosity in the foam.
So if you think of the solid in the foam, it's not like we have little fibers that just go from one side to the other like this and the heat just moves along those fibers.
You know, the foam cells have some complicated geometry and the heat has to kind of run along that complicated geometry.
And people have made estimates of what this is.
And it's roughly a factor of 2/3.
So I guess it would depend on exactly the foam cell geometry.
But typically it's around 2/3.
So that's conduction through a solid.
That's straight forward.
Conduction through gas is similarly straightforward.
It's just the conductivity of the gas times the amount of the gas.
And the volume fraction of the gas is just 1 minus the volume fraction of the solid.
So it's just 1 minus the relative density.
So the conduction through the gas is just lambda g times 1 minus the relative density.
So we can do a little example here.
And you can see how much of the conduction comes from the solid in the gas.
So for example, if we look at a foam that's 2.5% dense and say it's a closed cell poly-- what are we doing-- polystyrene.
So the total thermal conductivity of the foam is about 0.04 watts per meter k. And the thermal conductivity of polystyrene is 0.15 watts per meter k. And the thermal conductivity of air is 0.025.
So let's assume it's just blown with air.
And then if I just add up, what's the contribution of conduction through the solid and conduction through the gas-- so I just use those two little equations-- conduction through the solid-- it's going to be 2/3 of this value of lambda s times the amount of the solids-- that's 0.025 and then plus lambda g, which is 0.025 times the amount of the gas, which is 0.975.
And if I work those two things out, this is 0.003 and this is 0.024.
So that total is 0.027 watts per meter k. So you can see if the total is 0.04, most of it's come from the gas.
A little bit's come from the solid.
And the rest is going to be from convection and radiation.
And that's typical.
And that's the reason that they sometimes use low thermal conductivity gases to blow foams for thermal insulation because the gas makes up such a big fraction of the total conductivity.
If you can reduce that, you reduce the overall conductivity.
So, we'll say foams for insulation are blown with low conductivity gases.
But as I mentioned, you have this problem with aging that, over time, that gas is going to diffuse out and air is going to diffuse in.
Then the overall thermal conductivity of the foam is going to increase.
So that's the conduction.
And then the next contribution is from convection.
So imagine we have one of our little cells here.
And it's hotter on that side than it is on that side.
And hot air is going to rise.
Cold air is going to fall.
So you get a convection current set up.
And because of the density changes, you get a buoyancy force in the air.
So that's kind of driving the convection.
But you also have a viscous drag.
So the air is moving past the wall of the foam.
And there's going to be some viscous drag associated.
And how much convection you can get depends on the balance between this buoyancy force and the viscous drag.
So we'll say the gas rises and falls due to density changes with temperature.
And the density changes give rise to buoyancy forces.
But we also have these viscous forces from the drag of the air against the walls of the cell.
So air moving past the walls-- this is kind of a fluid mechanics thing-- so that air is a fluid.
And in fluid mechanics, they often use dimensionless numbers.
And there's a dimensionless number called the Rayleigh number.
And the Rayleigh number, you can think of it-- it's not quite the balance of the buoyancy force against the viscous forces.
But it involves those forces.
And convection is important if this Raleigh number's over 1,000.
And here's what the Rayleigh number is.
It's the density of the fluid times the acceleration of gravity times beta.
Beta's the volume expansion coefficient for the gas-- times the temperature change.
And we're going to look at a temperature change across a cell.
And then, times the length.
That's going to be the cell size.
And we divide that by the fluid viscosity and the thermal diffusivity.
So let me write down what all these things are.
So rho is the density of the gas.
So the g's gravitational acceleration.
Beta is the volume expansion of the gas.
And for a constant pressure that's equal to 1 over the temperature.
Then delta tc is the temperature difference across a cell.
And l is the cell size.
Mu is the dynamics viscosity the fluid.
And a is our thermal diffusivity.
So what I'm going to do is just work out, for a typical example, how big of a cell size do you need to get this Rayleigh number to be 1,000.
And we're going to see that, typically, that cell size is big.
It's like 20 millimeters.
So in most foams, the convection really isn't very important at all.
So it's typically-- people don't worry about convection.
And let me just show you how that works.
So for our Rayleigh number, which is ra-- for the Rayleigh number to be 1,000-- say we had air in the cells.
And say the temperature was room temperature.
Then the volume coefficient of expansion is just 1 over t. So it's 1 over 300, say.
degrees k to the minus 1.
Let's say our change in temperature across one cell was 1 degree k. Bless you.
The viscosity of air is 2 times 10 to the minus 5, pascal seconds.
The density of air is 1.2 kilograms per cubic meter.
And the thermal diffusivity for air is 2 times 10 to the minus 5 meters squared per second.
And if you plug all of these into that equation for the Rayleigh number and you solve for the cell size, you find that the cell size, l, is 20 millimeters.
So that says convection is only important if the cell size is bigger than that.
And so most foams have cells much smaller than that.
And convection is negligible.
So I have enclosed cells and the heat's not transferred so easily from one cell to another by the gas moving.
And by having small cells the convection drops out.
So you don't have to worry about that.
So the last contribution to heat transfer is from radiation.
And there's something called Stefan's law that describes the heat flux for radiated heat transfer from a surface at one temperature to another surface at a different temperature across a vacuum.
So we can say we have a heat flux qr not from a surface of one temperature.
So I'm going to call that t1-- to one at a lower temperature.
I'm going to call tnot-- with a vacuum in between them.
So this is [?
Stefan's ?]
law so this is the radiative heat flux is equal to the emissivity of the surfaces, which is beta 1 times a constant called Stefan's constant-- sigma times the fourth power of temperatures.
I'm taking the difference of the temperatures so here are the Stefan's Constant-- is sigma.
And that's equal to 5.67 times the 10 to the minus 8.
And that's in watts per meter squared per k to the fourth.
And beta is a constant describing the emissivity of the surfaces.
So it gives the radiant heat flux per unit area of the sample relative to a black body.
And that's a characteristic of the emissivity.
All right, so then, so if we-- yes?
[INAUDIBLE]
Now-- so right now, forget the foam.
We have no foam.
We just have two surfaces with a vacuum between them.
And now I'm going to stick a foam between the surfaces.
And we're going to see how that changes the heat flux, OK?
So the next step is we put the foam between those two surfaces.
And the heat flux is going to be reduced because the radiation is going to be absorbed by the solid and reflected by the cell walls.
And so we're going to characterize how much it's reduced.
So there's another law called Beer's Law, which characterizes the reduction in the heat flux.
Piece of chalk's getting to small OK, so Beer's Law gives us the attenuation, so the sort of reduction in the heat flow.
So qr is equal to qr not.
That would be the heat flux, if we just had the vacuum.
And then there's an exponential law.
And it's the exponential of minus k star t star.
And here, k stars in an extinction coefficient for the foam.
Talk a little bit more about that in a minute.
and t star is just the thickness of the foam.
And then this thing is called Beer's Law.
So we have very thin walls and struts.
And we're just going to consider optically thin walls and struts to make life easy.
Then we can say that, if they're optically thin, they're transparent to radiation.
They're optically thin if they're less than about 10 microns.
Then this extinction coefficient is just the amount of solid times the extension coefficient for the solids.
So it's just the relative density times the extinction coefficient for the solid.
OK, and then I can say, the heat flux by radiation.
I can use two equations to write that down now.
And then I'm going to let them be equal to get the thermal conductivity.
I can say qr is going equal to lambda r times dt by dx.
So that's the Fourier's Law that we started out with.
And then I've also got the qr that I'm going to get by combining the Stefan's Law with the Beer's Law up there.
So if I do that, I get that qr is beta 1 times sigma times t1 to the fourth minus t not the fourth.
So that's the qr not up there from down there.
And then I've got an exponential for the attenuation.
And instead of k star, I'm going to put the relative density of times ks.
and then I've got the thickness of the foam, t star, as well.
OK, so that's qr, but that has to equal lambda times dt by dx.
So I'm going to use some approximations.
Here and I'm going to end up with an expression for the contribution from radiation to heat transfer in the foam.
Yeah?
So when you say optically thin walls, where t is less than 10 microns, you mean like the walls of the foam?
Yeah, yea
So it's different t than the--
t star is the thickness of the whole thing, yeah.
So imagine we had our two surfaces.
And they might be like 100 millimeters apart or something.
t star is the sort of thickness of the foam in between the two surfaces.
And the optically thin is the cell walls, which are microns kind of thickness.
OK.
So I'm going to make some approximations here.
And that's going to allow me to solve for t star.
So I'm going to say that dt by dx x is approximately equal to just t1 minus t not over the thickness of the foam or I'll call that delta t over t star.
And then, the other approximation I'm going to use is that t1 to the 4th minus t not to the fourth is equal to 4 times delta t times the average temperature cubed.
So here t bar is the average temperature, t1 plus t not over 2.
So then, if I use those two approximations, I can write that qr, our heat flux from radiative transfer.
I got the beta 1.
I've got the sigma.
And instead of the difference of the fourth power, I'm going to write 4 delta t t bar cubed.
And then I've got my exponential.
Blah, blah, blah, blah, blah.
So then, here's the relative density times ks times t star, the overall thickness.
That's going to equal the radiative contribution to the thermal conductivity of the foam.
And instead of dt by dx, I'm going to have delta t over t star here.
So part of the reason for doing these approximations I end up with a delta t term on both sides.
Now I can cancel that out.
And if I just take this mess here and multiply it by t star, then I've got lambda r star.
That's our thermal conductivity contribution from radiation.
So one of the things to notice here is that, as the relative density goes down, then the contribution from radiation to the thermal conductivity of the foam goes up.
OK, so this chart here shows thermal conductivity as a function of relative density.
And it breaks down the contributions from the gas, g, the solid, s, and the radiation, r. And you kind of see the gas contribution doesn't change that much.
These are relative densities between a little over 2 and a little less than 5%.
So the amount of gas-- it's mostly gas in all of these things.
The solid contribution increases as the relative density increases.
So you'd expect that.
And then, as I just said, as the relative density goes down, the amount of radiation contribution goes up.
And so you can kind of see how that all fits together.
Another plot that shows the thermal conductivity versus the relative density.
These are for a few different types of foams.
You can see for this plot here, you reach a minimum in the thermal conductivity.
And that's because you've got this trade off between the contribution from the solid and the contribution from the radiation.
And those two kind of trade off and you get to a minimum.
So let me write some of this down.
So I'll just say that-- hang on.
Write this over again.
This is looking at the overall thermal conductivity.
And we can see the relative contributions of lambda, solid, lambda, gas, lambda, radiation.
I'll just say this shown in the figure.
I'm going to say the next figure shows a minimum in the thermal conductivity.
Then I'll just say there's a trade off between the conduction through the solid and I can direction from the radiation.
And then we also have a plot here that shows the conductivity versus the cell size.
And you can see that the conductivity increases with cell size.
And the reason for that is the bigger the cells get, the radiation is reflected less often.
And one thing I wanted to mention with the cell size is that if you look at aerogels, the way aerogels shells work is that they have a very small cell size, a very small pore size.
So typically, it's less than 100 nanometers.
And the mean free path of air is 68 nanometers.
So the mean free path is the average distance the molecules move before they collide with another molecule.
And if your pore size is less than the mean free path, then that reduces the thermal conductivity.
It reduces the ability of the atoms to pass the heat along between one another.
So the way the aerogels work is they have a very small pore size.
And what's important is how big the pores are relative to the mean free path of air.
OK, so that's the thermal conductivity.
I wanted to talk about a few other thermal properties of foams, as well, today.
So one is the specific heat.
And since the specific heat is the energy required to raise the temperature by a unit mass, then the mass is the same-- you know, if you have a certain mass of foam or a certain mass of solid-- the specific heat from the foam is the same as the solid.
So the specific heat for the foam is the same as the specific heat for the solid.
So that would have units of joules per kilogram per degree k. And the next property is the thermal expansion coefficient.
And it's a similar thing.
The thermal expansion coefficient for the foam is equal to the thermal coefficient of expansion for the solid.
So imagine you have-- say you had something like a honeycomb.
If you heat it up a certain amount, every member is going to expand by alpha.
And if every member expands by alpha, the whole thing expands by alpha.
And this is the same.
And it's the same idea with the foam.
So if every member just gets longer by alpha, then the whole thing gets bigger by alpha.
OK, so the last topic I wanted to talk about was the thermal shock resistance.
And thermal shock is the idea is that if you have something that's hot, and say you quench it in a liquid-- so you put it suddenly in a liquid-- the surface is going to cool down faster than the bulk of it.
And because the surface is trying to contract because it's cooling down, but it's attached to the bulk of it and it's constrained, it can't really cool down, then you generate stresses.
And if the stresses are big enough, you can cause fracture and have the thing crack and spall.
So we'll say if the materials is subjected to a sudden change in the surface temperature, that induces thermal stresses at the surface and can induce spalling and cracking.
So we're going to think about a material at one temperature that's dropped into, say, a liquid at a different temperature.
So the surface temperature is going to drop to the cooler liquid temperature and it's going to contract the surface layers.
And the fact that they're bound to the layers underneath that are not contracting as quickly, it means that you generate a thermal strain.
So the thermal strain is going to be the coefficient of thermal expansion times the change in temperature.
So you're going to constrain the surface to the original dimensions.
And then you're going to induce the stress.
So if it's a plane or thing, it's e alpha delta t. And then, there's a factor of 1 minus nu, just because it's a plane, in a plane.
And then you'll get cracking or spalling when that stress equals some failure, stress.
So I can rearrange this and solve for the critical delta t that you can withstand without getting cracking.
So I just rearranged this and say sigma's equal to sigma f. That would be sigma f times 1 minus nu over e and over alpha.
So that's the critical change in temperature to just cause cracking.
So now what I can do is I can substitute in there for what you would have for the foam.
And I'm going to do it just for the open cells just because it's easier to write the equations.
So for the foam, I would have some sort of fracture strength.
So when we did the modeling of the foams, we said that was equal to about 0.2 times the modulus of rupture times the relative density to the 3/2's power and 1 minus nu.
And if I divide by the modulus of the foam, that's es times the relative density squared.
And then we just had alpha for the foam was the same as alpha s. So then, I can rearrange this slightly and say it's equal to 0.2 over the relative density to the 1/2 power.
So I'm canceling out these relative densities here.
And then I can combine all the solid properties together.
And I'm going to say that nu for the solid is about equal to the same as nu for the foam.
So what I can do here is I can group all the solid properties together.
And this just is delta t critical for the solid, right?
So this is saying that the critical temperature range before you get cracking in the foam is equal to the range for the solid, but multiplied by this factor of 0.2 and divided by the square root of the relative density.
So if the square of-- the relative density is going to less than 1.
So this number here is going to be bigger than 1.
So it's saying that the temperature range that will give you spalling in the foam is going to be bigger than the temperature range in the solid.
So the foam's going to be better than the solid, OK?
And that uses our little models from before.
So I think I'm going to stop there, probably cause my throat is starting to get too sore.
There's a little case study in the notes.
And I'll just put that on the notes on the Stellar site.
It's like one page and it's really straightforward.
You can just read that, OK?
So this is the end of the bit on thermal conductivity.
That's just this one lecture.
And this is really the end of the whole section on modeling of the honey combs and the foams.
So that's kind of the first half of the term is modeling the honey combs and the foams.
And the second half of the term, we're kind of applying those models to different situations.
So next week, we'll have the review on Monday, have a test on Wednesday, week after that is Spring break.
I can't believe we're at Spring break already.
And then after that we'll start we'll do the trabecular bone for a week.
We'll do tissue engineering scaffolds and cell mechanics for two or three lectures.
We'll look at some other applications to engineering design, look at energy absorption and sandwich panels.
And then, I'm going to talk about plants a little bit at the very end, OK?
So we've already covered a lot of the kind of deriving equations part of the course.
The rest of the course is more applying the equations to lots of different situations, OK?
So I'm going to stop there just because my throat is giving out.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I was just going to be here to answer questions
Just clarifying, What was the material that we were covering?
In the test?
Yeah
So the test covers everything up to the end of the part on modeling foams, but not the bit on the performance indices, and the material selection charts for foams.
So I think up to the end of the fractured toughness of foams
I see.
OK.
So not covering past [INAUDIBLE]
Not covering thermal properties, no.
It doesn't cover thermal properties.
Here you go.
So you know I got this MacVicar award, and we had a lunch on Friday at the Catalyst restaurant.
And I had to get up and speak.
And just as I got up and spoke, there was a red-tailed hawk swooped by the window.
It was perfect.
It was perfect.
Yeah.
Hi.
So I'm just here to answer questions.
So come on.
Somebody must have questions.
It's all perfectly clear?
You want me to do--
Talk through test one from last year a little bit
You would have to give me test one from last year.
I didn't bring it with me.
I just brought the problem sets
I have it on my computer, and I could read you the problems or just hand you the laptop.
Whichever you prefer
Why don't you hand me the laptop and I'll try to do it.
Is that OK?
OK.
So the question is about the test for 2014.
OK.
So the first question was, describe four processes for making honeycombs, and comment on the type of material usually used for each process.
So I did post the solutions, right?
Did you look at that?
Yeah, I looked at them.
I guess I just feel like I don't fully understand why things are there.
But I can look at it some more
Well, I can go over it, if you want
I'll just look at it some more
No, I can go over it.
So four processes.
Let's see.
So there's the expansion process, where you take sheets and you glue the sheets together, and then you pull them apart.
So you can only really use that process for materials that are going to have large plastic deformations.
So you could use it for metals, some polymers.
But you couldn't really use it for ceramics.
You couldn't use it for glass because as soon as you yanked on it, you'd break the sheets, right?
So--
Is it rigid polymers that you can use it for?
Well, something like nylon you can use it for.
Something that's got a little yield, that will have some sort of yield point.
Not like if you had epoxy, you couldn't use it for epoxy.
So, OK.
So that's one process.
Another process is a corrugation process where you have a wheel that has little gear knobs on it.
And you run your flat sheet through that and it comes out with the half hexagonal profile, and you glue those together.
So again, you need something that's going to yield.
So that would typically be a metal that you would use that with.
Let's see.
Another processor making honeycombs is 3D printing.
You can 3D print honeycombs.
And there's different ways to do it.
One way is by having an ink.
So if you want to print in ink, typically that's some sort of polymer that you're printing.
I suppose physically it's possible to print glass or to print a metal, but you'd have to have some very high temperature setup to do that.
So typically a resin of some sort.
Let's see.
Other ways to make honeycombs.
You can extrude honeycombs.
So the ceramic honeycombs we saw were made by extruding a ceramic slurry.
And typically, you would do that with a ceramic that's a slurry and a powder.
You wouldn't necessarily do that with a metal.
I don't think I've seen any metal honeycombs that are extruded like that.
OK?
Are we good with number one?
Yeah, that makes sense
We now need your password
Oh, sorry.
So is there an actual difference between 3D printing and the extrusion process?
Yeah.
So the extrusion you have a die, and you squeeze the material through the die, right?
So extrusion's kind of like the toothpastey thing.
And 3D printing, you can have an ink or you can do 3D printing where you have, let's say, a powder.
And then you print the binder.
And then you heat it up some way to get the binder to cure.
And then you get rid of the powder that's not bound.
That's another way to do the 3D printing.
OK?
Can you also pour it into a mold?
Yeah.
Yeah, those silicon rubber honeycombs that I showed you, those are all made by pouring a liquid into a mold and then curing it.
Yeah.
Yeah, there's other ways, it just asked for four, so I randomly thought of four.
OK. OK, are we good?
So the next one is a hexagonal titanium alloy honeycomb has h/l is two, theta is 45, and t/l is 0.05.
It says, the end constraint factor for elastic buckling is n equals 0.806.
The titanium has a modulus of 110 gigapascals, and yield strength of 880.
And then you have to calculate some properties.
So would you like me to do that?
Yeah
Yeah, you would?
Does anybody else want that?
I don't see a lot of other people wanting anything else, so I might as well do that.
Let's see.
I would need a piece of chalk.
Here we go.
OK.
So it's a titanium alloy honeycomb, and we're told h/l is 2, theta's 45, and t/l is 0.05.
And we're told that n-- why don't I put the n over here-- n is 0.806.
And we're told the modulus of the solid is 110 gigapascals, and the yield strength of the solid is 880 mega pascals.
OK.
So it says calculate the value of and describe the mechanism of deformation failure for-- and the first part is the Young's modulus in the two direction, e star 2.
OK.
So I don't remember these formulas either, so I need to look at my notes.
And oh, I don't have the formula for e 2 in my notes.
Let me see.
Is it in any of the problems?
I have the formula sheet
You have the formula sheet?
I didn't bring the formula sheet with me.
You have it?
Yeah, I think it's there.
I'm pretty sure it's there.
OK, so this is just like substituting, and there's nothing complicated about this.
So it's equal to Es times t/l cubed times h/l plus sine theta divided by cos cubed theta.
So then you just plug everything in.
So this is 110 gigapascals.
And I'm going to put 110,000 mega pascals, because it's probably going to be less than a gigapascal.
Then t/l is 0.05.
So that's 0.05 cubed.
And then h/l is 2 plus sign of 45 is 0.707.
And then we divide by cos theta cubed, 0.707 cubed.
And then I'm not going to work it out, but that's OK.
I assume that's what's in the solution.
Are we good?
So it's just substituting.
That's all it is.
In the second part-- OK. You need to change the time on this, because it just keeps timing out
Sorry, I don't know how to change it, but I'll try
Oh, is that what this is?
OK, this is the test.
Oh, this is from 2013.
Oh, the next-- if we keep going.
Here we go.
He's got it, so you take your computer
That's probably better
That's perfect for everybody.
OK.
The second part is the plateau stress for loading in the x2 direction.
So that's that.
For loading in the x2 direction, it could either be an elastic buckling collapse stress, or a plastic buckling collapse stress.
So I'm going to calculate both, and then whichever one is lower, that's the one it would be.
So let's see here.
So that's going to be-- I'm missing my formulas again.
Here we are.
So here's the buckling one.
It's n squared pi squared over 24 times t cubed over lh squared times 1 over cos theta.
So you put 0.806 squared in here.
Pi squared 24.
So here you go.
This is t over l cubed times h over l squared.
So that would be 0.05 cubed.
And that would be 1 over 2 squared, and then 1 over 0.707, and then whatever that equals.
OK?
Are we good with that?
And then you'd want to calculate sigma star to plastic.
And that's equal to sigma ys, t over l squared, and 1 over 2 cos squared theta.
All right?
So then sigma ys was 880 mega pascals.
And t over l was our 0.05.
And that's 1 over 2 times 0.707 squared, and then whatever that equals.
OK?
And then whichever one of those would be less is the plateau stress.
So I don't-- yeah?
So I know-- I think I made this mistake in the problem set, but here, because it's titanium, we don't consider it brittle
Right.
Right.
I mean, you could.
But you don't need to
That was something with ceramics?
Yeah, or if it was a glass, or ceramic, or maybe an epoxy.
Something that was brittle.
And besides which, if you look at the question, I only give you a yield strength and a solid modulus.
So to get the brittle thing, I would have to give you a fracture strength
That's true with all the problems
I usually give you what you need.
Especially on the test, I'm going to give you what you need.
You're not going to be looking things up.
OK?
So we're good so far?
OK, let me go back to the question.
So then the third one is the out of plane Young's modulus in the x3 direction.
And that's just going to be Es times the relative density.
And the relative density-- let's see.
So it's Es times rho star over rho s. So that's 110,000 mega pascals.
And then there's also the very first equation on this is the relative density.
So it's t/l times h/l plus 2 divided by 2 cos theta h/l plus sine theta.
So I'm not going to substitute everything in, OK?
So that was it.
It was very plug and chug.
Are you good?
Can I ask a question about a specific question
I'm going to go through the rest of them.
She wants me to do the whole test.
You want me to do the whole test, right?
That would be great, but if other people have other questions it's fine
Why don't I do the whole test.
And then there's going to be time, I think
[INAUDIBLE]
Let me do the test from last unit.
OK, so that's number one.
Or that's one and two.
And then three is, a closed cell elastomeric polyethylene foam has a relative density of 0.05 and a volume fraction of solid in the edges of 0.6.
They give you the Young's modulus of the solid is 0.2 gigapascals.
The pressure within the cell walls is atmospheric, 0.1 mega pascals.
And the Poisson's ratio of the foam is 0.3.
And you're asked to get the Young's modulus of the foam, the compressive plateau stress.
And then there's a question about why does the Young's modulus depend on the solid modulus and relative density, while the Poisson's ratio does not.
So let me go through, then.
So the first one is, what's e star.
And we're told relative density is 0.05.
The volume fraction in the edges is 0.6.
So remember, that was what we called phi.
So phi's 0.6.
The Young's modulus of the solid is 0.2 gigapascals.
The initial pressure within the cells is 0.1 mega pascal.
and Poisson's ratio for the foam is 0.3.
And it's a closed cell.
So if you remember-- oh, let's see.
I think back here, was I supposed to-- I was supposed to say something about the mechanism of deformation and failure for the first one.
So the mechanism of deformation in the modulus in the 2 direction is bending.
The mechanism of failure here is buckling.
The mechanism of failure here is yielding.
And the mechanism of deformation here was axial deformation, OK?
So I forgot to say that.
OK, let me go back here.
So for this one, if it's a closed cell foam, remember there were three terms to the modulus.
There was one from bending of the edges, one from stretching of the faces, and one from the gas contribution if you've got gas inside the cells.
So again, I'm going to have to peek at the equation.
Foams.
Here we go, foams.
OK. And then this gas one.
Get rid of that.
OK, so this one is just the same kind of thing.
It's just plug and chug.
Do I need to put all the numbers in?
No.
I guess I'm a little bit confused why you need the pressure term.
Because you talked about faces bursting
Ah, so if it's the modulus, remember the modulus-- so the question is, why do you need to worry about the pressure because I talked about the faces bursting.
And remember, the stress strain curve looks something like that.
Maybe the slope of the curve is a little bit higher over here.
But this is the modulus down here, right?
The modulus is related to the initial stress strain relationship.
And initially, they're not going to burst.
You'd have to load it up to some amount of stress before the faces burst, right?
So when you're down here, the faces certainly down at the beginning, they're not burst, right?
You have to get some stress before they're going to burst.
And in some materials, when you get up around here, around the plateau stress-- let's just call that sigma star-- then they might burst.
And then the pressure term would disappear and the face term would disappear.
So if I don't tell you to ignore them, if I don't say they're going to burst, or I don't say they're negligible, I would calculate them.
And then if they're small, then you say, well, they're negligible.
OK?
Are we good with that?
You're good?
You're good?
Sardar, you don't need to be here.
But you can stay if you want, but you don't need to be here.
OK, are you good?
Everybody else good?
OK. That was A.
B, what's the compressive plateau stress of the foam.
So here, they want to know what sigma star is.
You're told it's elastomeric.
So if it's elastomeric, it's like a rubber.
It's rubbery.
So if it's rubbery, it's going to buckle.
It's not going to yield.
It's not going to be brittle.
So you can just calculate the elastic stress here.
And if we flip over to our handy dandy list of equations-- blah, blah, blah, blah.
Oh, pooh.
I'm realizing-- yeah, so I don't have the term here for the faces or for the gas, so here we could assume it's going to rupture.
So let's assume it's going to rupture.
So if we assume that it's going to rupture, it would be just like the open celled foams.
And then you can just use that.
And that's been found to work fairly well for the open celled and the closed cell foams
And we assume that faces rupture because--
Well, to be honest, I can't remember if last year I got to this point in the test and somebody said, we don't have the equation, and I gave them the equation with the other terms, or if we just assumed that the faces ruptured.
I can't remember.
To be honest, I think probably for elastomeric foam, you could assume that they'd probably don't rupture unless they're very, very thin
So what kind of foams do they typically rupture in?
So certainly if you had a metal foam, they'd probably rupture.
If you had, say, a polymer foam that was more rigid, like a rigid polyurethane.
So polyurethanes can be flexible, which means they're made out of an elastomer, or they can be rigid.
And the foams that are typically used for insulation, thermal insulation, are typically closed celled polyurethane foams.
And those typically have very thin faces, and they would rupture.
Yeah?
Are you looking for the Young's modulus in this problem?
The Young's modulus was the first part, right?
So part A was the Young's modulus
In B--
In B is the collapse stress, the compressor strength
So I think in my notes, I think it has this.
If the--
Right, if p0 is bigger than-- so what she's showing me in her notes is I had a little note in the class, in the lecture, that if the initial pressure in the cells is greater than atmospheric, then the cell walls are pre-stressed and you have to overcome that in the buckling
Is that atmospheric?
That is atmospheric pressure
So you don't need the [INAUDIBLE]
No.
Yeah.
OK?
OK. Yeah, you don't know that that's atmospheric.
So do you ever do things in PSI?
No, you don't.
Because when I was a student a long time ago, the thing I remember learning was atmospheric pressure is 14.7 PSI, more or less.
And the conversion between mega pascals and PSI is there's more or less 145 PSI to a mega pascal, so the atmospheric pressure is about 0.1 mega pascals.
Yeah?
I don't know if this is a silly question, but for part B, how do we get from the Young's modulus of the foam--
Oh, sorry, sorry, sorry, sorry.
I put the wrong thing down here.
Sorry, my mistake.
OK, now you happy?
Sorry.
OK, shall I move on to the next part?
Another question?
Well, I had a question about number two, but maybe we can come back to that--
OK, let me finish this, and then we'll go back to number two.
So this one here, the part C is why does the Young's modulus foam depend on the solid modulus and the relative density while the Poisson's ratio does not.
So when I write the equation for the Young's modulus, the solid modulus comes into it, the relative density comes into it.
And remember when we had the Poisson's ratio, it's just a constant that depends on the cell geometry.
So here's C, nu star just as a constant.
And that constant just depends on the cell geometry.
So if you think of the Poisson's ratio, it's the ratio of two strains, right?
So say I have my foam here.
So say that's just a block of foam.
Little cells in it here.
Little cells.
And say I press on it this way here.
And let's call this the one direction and the two direction.
If I press it in the two direction, the Poisson's ratio is then just nu would be-- let's see, this would be 2, 1.
It'd be the strain in the one direction over the strain in the two direction.
So it's the ratio of two strains, right?
And if you think of our model for the elastic behavior of the foam, each of those strains is going to be related to some bending deformation in the cell walls or the cell struts.
And this strain here is going to be-- let me make this proportional.
It's going to be proportional to a delta over l. And this one here is going to be proportional to the delta over l. So this might be delta in the one direction, and this will be delta in the two direction.
But those two things are both going to be related to the bending deflection of the beams, right?
And since both of those deltas are related to the bending deflection of the beams, we could write them-- if you want, I could write that as f l cubed over E of the solid times t to the fourth.
And then that's times 1 over l. And then this thing here is also f l cubed over E of the solid t to the fourth 1 over l. So everything cancels out except the geometrical constant.
And if you remember when we did the honeycombs, it looked exactly the same.
When we looked at the Poisson's ratio of the honeycombs, we had the strain in one direction over the strain in another direction.
And each of those strains was related to some component of delta.
There was a delta 1 and a delta 2.
But the delta 1 might be delta sine theta and delta 2 was delta cos theta.
So if the deltas are the same, then it all just cancels out.
And all you're left with is a geometrical constant.
OK?
Do you get physically why that is?
So I have a question about this.
Because we're given most of the equations that we need, is it only in conceptual questions that we should know how we actually derived that version?
I'm not going to ask you to derive that equation for a closed cell foam
I meant this last part
Well yeah, you should be able to explain that.
But I mean just at this level.
Nothing very mathematically involved
OK, cool
OK, are we good?
So that was the end of the test for the undergraduates, OK?
And then for the graduate students, just like the problem sets, I just have an extra question.
And that's what I did this year, too.
So the graduate students have one extra question.
So you and you.
Is anybody else a graduate student?
I think it's just the two of you.
You're post post-graduate.
OK. OK.
So let's see.
So this one says, the performance-- so this is on the performance indices which I told you you didn't need to know for this test, partly because, remember, we missed two lectures.
We're not exactly on the same spot as we were last year.
But I can do it if you want.
Do you want me to do it, or should we do other questions?
What do the grad students think?
Sure.
OK.
So the question is, the performance index to minimize the mass of a beam of a given bending stiffness, length and square cross-section is e to the one half over rho.
So you remember, we derived that e to the one half over rho in class.
In the section on wood, we saw that this performance index for wood is higher than that for the solid cell wall material in wood.
Do you remember that?
The e to the one half over rho for the wood was, I think, rho s over rho star to the one half times Es to the one half over rho s. So explain why wood has a higher value of e to the one half over rho than the solid cell wall material.
And then part B is, suggest a design for an engineering material based on wood that has high values of e to the one half over rho.
So one way to explain it is to say that if you're looking at e to the one half over rho, you can say for wood, E over Es is equal to rho star over rho s for loading in the axial direction.
So this will be for loading actually along the grain.
And that's what we were looking at.
So that's what I'm talking about here.
So I think-- let me just see if this is right.
Yeah.
So this equation here is exactly the same as that equation there, right?
And this is basically saying that this is the performance index for the wood.
This is the performance index for the solid.
And this factor here is bigger than 1, because the solid density is higher than the wood density.
OK?
So really, all you have to do is say that for the wood, the modulus in the longitudinal or the axial direction along with grain varies linearly with the relative density.
And it probably would be a good idea to say that this is a result of the cell walls deforming axially.
So when you take the cells, if you think of the wood cells as being something like that and you're loading it this way on, the cells just actually shorten, and the modulus depends on the-- it just is the volume fraction of solid times the modulus of the solid.
And that's where this comes from.
And once you have this, that basically gives you that.
OK?
Are we good with that?
So that's why it's higher.
Another way to look at it as sort of more of a hand-wavy argument is that if you have a certain amount of solid-- so say you have a certain mass of solid.
If it's solid, it takes up a certain cross-sectional area.
So say that your beam's a certain length, that's going to have a certain cross-sectional area.
And if you have wood, if you have a cellular material, if you have the same mass, you're essentially making the dimensions of that piece bigger.
So you're moving the material further away.
And as you're making it bigger, you're increasing the moment of inertia.
And so you're increasing the bending resistance of it.
That's another, more hand-waving way to talk about it.
And then the second part is to suggest a design for an engineering material based on wood that would have high values of e to the one half over rho.
So remember when we looked at those material performance charts, we said that wood was similar to engineering fiber composites.
But those data for fiber composites are assuming that it's solid, the fiber composite's a solid.
So if you could take fiber composites and make little tubes of fiber composites and assemble the tubes together so that it was like wood, would get something that would be even higher.
So if you could make, say, a fiber composite honeycomb material, and you'd want to have the fibers aligned along the prism axis of the honeycomb, then you would get higher values.
It would be the same-- it'd be this sort of argument again, but now with a fiber composite.
So you'd want-- if this was your fiber composite like this, you'd want the fibers-- well, in wood, they're at a little bit of an angle.
But say they were lined up like that.
You'd want them something like that, then loading it that way on, right?
And if one way to think about those charts is if you-- say we had a plot.
And say this was log of the modulus and that was log of the density.
And I think I'll just draw the envelope.
So foams were somewhere down here, metals were somewhere over here, and with elastomers we're somewhere in here.
I think ceramics were up here.
And then I can't remember exactly where composites were, but composites were around about here.
I'll just say FRC for fiber reinforced composites.
And I think woods were kind of in here.
Something like that.
And then we had our performance index, right?
So remember, there was a performance index, something like that.
And that slope of that was e to the one half over rho.
So every point on that line had the same value of e to the one half over rho.
And essentially, if you had the fiber composite and you made a honeycomb out of it, you would be taking the data from here and shifting them out that way.
You'd be pushing them out over here, so you'd get a higher value of that performance index.
OK?
So that's the test from last year.
That's the end-- yeah?
So for along right here or different it would be cubed
Yeah, so the thing about the honeycombs is--
The opposite
Right
[INAUDIBLE]
It'd be worse, that's true.
So the thing about the honeycombs is they're very stiff in the axial direction, but you pay for that in the other directions.
And it's the same for wood.
So the wood is very good when you load it along the grain, but you pay for it the other way.
But if you think of from the tree's point of view-- if you're a tree.
So here's my little tree.
So here, say we have a tree trunk, and we have some branch.
Branch over here, branches, tree.
So the grain is lined up this way.
And then when there's a branch, the grain turns around and goes that way, right?
So if you think of the tree as a whole, the whole tree blows in the wind like this.
So it's like a column like this, and everything's lined up that way.
And you're loading it this way.
So that is the stiff direction, right?
And if you're a branch, the branches are more loaded by gravity.
So they're loaded that way.
And then because the fibers, the grain turns around, they're also oriented in the good direction.
So from the tree's point of view, it's optimized things.
Then you remember when I talked about the old wooden sailing ships, when they made the old wooden sailing ships, if this was the deck here and that was the haul there, they would get pieces of wood to fit in here that were called the knees.
That was the knee.
And they would try to get a piece that was from a branch like this.
And they would try to match the curve of that joint with the branch with the curve that they needed in here so that the grain followed the pattern of what they needed for the boat.
OK. Other questions?
I'm not quite sure what the difference between tangential versus ray here
Oh, OK.
So in the wood, you mean?
Yes
OK.
So can I rub this stuff off?
We're happy?
Let's see.
Say again?
So tangential and radial.
OK.
So say the wood cells look something like this.
So these would be the fiber cells or the tracheids And then the rays typically are more rectangular cells.
So they might look something like that.
And then they would be some more fibers or tracheids, depending on if it was a soft wood or a hardwood.
So these would be either fibers or tracheids in a hardwood or a soft wood.
And then these would be the ray cells in here.
So they have a different structure.
They just look different.
They're different shape
This is the top?
From the top?
Yeah, this is looking from the top down.
And then if you think of the tree-- so the tree's going to have growth rings, right?
So the growth rings are going to look-- obviously I'm not making perfect circles, but you get the idea-- something like that.
And then the rays go this way.
They go radially.
OK?
So this would be the radial direction, and then those are the rays.
Are we good?
So which way's the tangential?
So the tangential would be this way on, OK?
So if I loaded this way like that, that would be loading it in the tangential direction.
And if I loaded it this way, that would be loading it in the radial direction.
The length of the rays runs in the radial direction.
The length this way on.
So this thing here corresponds to one of these lines I've drawn here.
And then these guys here are the stuff in between here
How do you know what the tangential, the Young's modulus and stiffness is?
So say we were loading it tangentially, we're loading it like that.
Then--
If you have a tree, how do you apply a tangential load on it?
Oh, well it's not the whole tree, right?
So say we have a piece of wood that we cut out like this.
So say I have that.
And if I loaded it this way on, I'd be loading it tangentially.
The tree's big, right?
So I'm not talking about loading the whole tree, I'm talking about taking a piece of wood out of the tree and loading it
OK, so you can't really load tangentially for the entire trunk
No, I'm talking about taking a piece out and loading that piece
How about a ray here?
Do you take the--
So the same thing.
You'd take-- say this was the piece of wood that you were looking at.
Now you would just load it this way on.
OK?
I think-- I brought my thing because I have the slides.
Let me see if I can find-- I think there was a slide that showed this.
OK. That was Furry Fridays.
That was the wood sculptor.
Here we go.
OK, so imagine that that cube is your piece of wood that you're loading, right?
So imagine this is the-- you cut a little piece out.
Then you're loading it tangent.
Can you see, then?
You can load it-- you're not loading the whole tree
Yeah, I was thinking about loading the entire tree and then applying the tangential load on it
Yeah, because then-- I see the problem
It's going to give us shears
Yeah.
Yeah, because I could say, well, if I was trying to load the whole thing.
Say I was loading it from here to there.
Well if you look at it one way, it looks tangential.
If you look at it the other way, it looks radial.
So think of cutting a piece out, because that is what you're going to do.
You're going to cut a piece out.
OK?
All right.
Are there other questions?
Yes?
With that formula sheet, do you only give that formula for the honeycombs, or also for the foams?
I'm going to give you-- if you look at, I think, problem set 2, I gave you a sheet that had three pages of equations.
And it looked exactly like this.
So there was one saying, properties of two dimensional cellular solids-- honeycombs.
There was a whole thing of in plane properties and out of plane properties.
That was one page.
The next page was properties of regular hexagonal honeycombs.
And then the next page was properties of three dimensional cellular solids foams
OK, excellent.
Thank you
OK?
Like I just said, I think there's maybe one or two equations missing from this.
But if it was something you needed, I would give it to you.
I would give it to you.
OK?
So I should have scrolled down for it?
What?
You should have scrolled?
So you're like me.
This happens to me all the time.
I have some website.
I'm looking at it.
I'm like, OK.
I got it.
I think I've got everything.
And then I realized I'm supposed to-- I missed something because I was supposed to scroll down
Problem set two was only the honeycombs.
That's why
Well, I think that was probably all we covered was the honeycombs on that problem set
So that's why I only got that part
OK. All right, yeah.
So I'm going to give you, this will be attached to the test.
OK?
So I think on the test that I posted it was attached, wasn't it?
Yeah.
[INAUDIBLE], did you have a question?
For 2 part D, I don't think we went over that.
I was confused as to how-- is it just that equilateral triangular cells always have-- is always truss behavior?
Let's see
Oh, sorry.
It's the 2014 test
2014-- oh, sorry.
I missed a part.
Sorry.
Yeah, so it says, the same titanium alloy is used to make a honeycomb with equilateral triangular cells.
And what is the in plane Young's modulus for loading in the x 2 direction of the triangular honeycomb?
So this is-- say you have cells that look like that now.
And that's x 1.
That's x 2.
OK.
So the Young's modulus for this-- so I happen to remember the formula.
So I guess I'm thinking you might have put this on your cheat sheets.
It's 1.15 times Es times that, times the relative density.
So I'm trying to remember.
Do I have that on here?
I don't have it on here
Are we just supposed to know that?
Are we just supposed to--
Well, I guess what I would hope that you would know is maybe not the constant, but that it should go as Es and linearly with the relative density.
Because it's a truss and because it deforms axially.
I don't really expect that you would remember the 1.15.
Yeah?
So does that mean for all the foams, of which there were like 10 constants, we don't need to write all down, like what C1 equals--
I think that's what this thing gives you.
Let's see.
It gives you all the Cs.
All right, then I'll make sure I give you the Cs.
I'll make sure I give you the Cs.
But who's got a pen so I can write that down to make sure that I do that?
Does somebody got a piece of paper.
Or I could write it down here.
Oh, here we go.
I can write it down this little sticky thing here.
OK.
So I'll stick that on there so I remember to do that
Will you also be writing what Cs?
Because you have, in your equation, C1, C2--
Well, I think I would say-- say I asked you for, I don't know, the yield stress for a foam in compression that yields plastically.
I would say, the constant for that is 0.3.
I wouldn't give you C2.
I wouldn't do it by numbers, I would tell you what the number was for the thing you needed, because I mean the way the numbers are, the only reason they're numbered is because that's the number they are in the book.
They're just ordered sequentially in the book.
But I don't expect you to remember which-- it's C6, or C5 or something.
So anyone else?
Can you explain the difference between uniaxial yield and plastic buckling?
Oh, OK.
So if you have something and it fails by uniaxial yield-- so say you have a honeycomb like this, and you're loading it this way on, OK?
So if you're loading it that way on, these walls of the honeycomb are just axially deforming, initially.
Right?
So the elastic behaviors, they just axially deform.
So it works out that if these cell walls are very thick, then you can reach a yield stress before any buckling occurs.
And then the strength would just be that yield stress of the cell wall material times the relative density.
OK?
But the cell walls have to be thick for that to happen.
So then imagine that the cell walls aren't thick.
Imagine that the cell walls are thin.
So say I have the same honeycomb like this.
If the walls are thin, and say the solid material itself-- so this is for the solid-- it has some stress strain curve.
And it may have a linear elastic part, and then a yield thing like that.
So say this is the yield strength here.
Say we compress that this way on the same thing in the three direction.
Then if a material's got a yield point, there can be an interaction between plastic yielding and elastic buckling.
And you can get plastic buckling.
And the plastic buckling, you're going to get the wrinkles that go along the length of it this way.
remember I showed you that tube that kind of collapsed and folded up kind of thing?
That's plastic buckling.
OK?
And typically, people use what's called the tangential modulus to calculate the buckling stress for plastic buckling.
And the tangential modulus would be something related to the tangent over there.
I don't expect you to be able to derive plastic buckling equations.
But the plastic buckling-- you know what elastic buckling is, right?
Yeah.
Yeah.
So one way to think about plastic buckling is, if you have-- and I'm trying to remember.
This is called the slenderness ratio.
And I'm trying to remember, is that l over r?
Imagine you had just a circular cross section and you had a length, l. So you have a column here, and it's got a length, l, and it's got a radius, r. Like that, OK?
So the longer it gets, the more slender it is, the higher the slenderness ratio is.
And this, I think, is some sort of stress.
If the slenderness ratio of just a single column is short-- if it's stubby, if you had a column that looked kind of like that, It's not going to buckle.
It's going to yield.
And so if you compress that, it would just yield.
And it's just going to yield at the yield stress, right?
It's just going to yield at sigma y of the solid, whatever the solid is.
If you have a long column, it would buckle elastically by an Euler buckling.
And Euler buckling-- let's see.
I'm going to run out of room here.
If you think of it in terms of a stress instead of a load, it's going to be in squared pi squared Es i.
Let's say i goes as r to the fourth.
And this is going to be l squared r squared.
Right?
This is going to be a pi in here.
There's going to be a pie in here.
I might have lost a factor of 4, but it's going to be-- let me make this proportional, OK?
So the slenderness ratio, there's going to be an l over r squared term here.
So I could cancel out the four there and put a squared.
So sigma Euler is going to go as Es times r over l squared.
Like that.
And so this is the Euler buckling stress here, OK?
So this would be elastic.
And right here at this little corner, it turns out life isn't quite that mathematically exact.
If you're near that corner, it's not like here it's buckling elastically, and here, it's buckling plastically.
What happens is, if you looked at data, data might do something like that.
So the sum interaction between the elastic and the plastic.
And that's kind of what's going on with this thing here.
Does that makes sense?
Plastic buckling can-- OK, so if you unload plastic buckling, you get some of the elastic part back?
You're not going to get much back
You're not?
No, because once-- to get the plastic buckling, you're very close to this.
By the time you get that deformation, you've got locally, it's yielded.
It's not all elastic everywhere.
It's going to yield in places.
And once it starts yielding, it's-- if you think of these buckles forming, it's not like you're at one spot on this curve throughout the whole thing.
Some of it's more deformed, and some of it's less deformed.
Let me pull up those plastically buckled columns, those tubes.
Get rid of that one.
Let me try and remind myself where they were.
I think-- honeycombs, I want honeycombs.
Out of plane, that's what I want.
It was this thing here.
So you see when you have-- that's just one tube, but the whole honeycomb would-- imagine that you have groups of tubes put together.
They would have to fail in some compatible way.
But the deformation and the stresses are not going to be uniform through this whole thing, right?
One part of it's going to be at one stress, and something else is going to be at another stress.
So parts of it are going to yield plastically, and you're not going to recover that.
OK?
So in fact, they use these sorts of things for energy absorption devices, like in cars and things like that.
To absorb the energy from the impact.
More questions?
Let him have a turn
Sorry, I have a question about I think 2013
2013,
The last question.
It's about the plastic
2013.
Let me rub some of this stuff off.
OK.
Here we go.
OK. Oh, we haven't covered this at all.
So the last question, this one here?
Right
Yeah, so this question's on energy absorption.
We haven't got there yet
How about--
And the third question's on sandwich structure.
So when I taught the course in 2013, I did the topics in a different order.
So I did honeycombs, and I did foams.
And then I think I did sandwich panels, and I did energy absorption.
And I left the stuff on the wood and the cork to the end.
So we haven't done that.
So don't panic if you haven't-- if you can't do that.
OK?
You should have known.
Come on, you should have known that if it talked about things we haven't covered yet, I'm not going to give it on the test.
OK, what else?
Can you explain more plastic hinges?
Plastic hinge, OK.
So let's just say we have a beam in bending, OK?
And say it just has a load p in the middle, all right?
Are we good?
So this load in the middle, then this reaction is p over 2, and that reaction is p over 2.
And if I drew the sheer force diagram, it'd go p over 2 up, we go over, go p over 2 down, like 2 p over 2 down.
Over and back up.
OK?
And then if I drew the bending moment diagram, it would go up and down like that.
And that would be zero.
And that would be zero.
And this would be PL over 4.
OK?
Are we OK with that?
We haven't got to the end of the answer, but--
I have a question.
We're not expected to--
No, no, you don't need to do this.
I'm just trying to explain it now.
You don't need to retain that information, for heaven's sakes.
No.
Come on, I'm so disappointed
For the moment, is it positive for the counterclockwise turns?
Oh, so you remember for the beam bending, there's a different convention.
It's positive if it's tension on the bottom.
Are you in mechanical engineering?
I though you were in mechanical engineering
No, I take physics
Oh, you do physics.
All I really want to say is the moment's maximum in the middle, OK?
So let me just say the moment's maximum in the middle, OK?
So then let's look at the cross section.
So say I look at a cross section here.
Let's just make it rectangular to make it easy for me to draw.
So it has width, b, and a height, h. OK?
So this would be h on this picture over here.
And remember, the neutral axis goes through the middle on the cross-section here.
And so one half of the beam is in tension, and the other half of the beam is in compression.
So for this situation here, this half of the bean is going to see compression, and that half of the beam is going to see tension, OK?
Are we happy with that?
We're happy with that.
OK. Now let me draw the stress distribution.
So if it's linear elastic and it hasn't yielded yet, the stress distribution is going to look like this.
So this is h again.
That's the height.
And now b is into the board.
And I'm plotting the stress this way.
So this thing here is my neutral axis.
It has no stress.
Remember, there was one plane that has no stress, and for a rectangular cross-section, it goes through the middle of the cross-section.
It goes through the centroid.
Is this ringing a bell?
I'm hoping this is ringing a bell.
Come on.
I know we did this in 302, too.
I know we did.
OK. OK, so this is all linear elastic, right?
So at some point-- so I'm going to get to the plastic hinge.
At some point, If you keep loading it and p gets bigger and bigger, the moment gets bigger and bigger, the stress gets bigger and bigger remember, the equation here for the stress is equal to My over i.
This moment, the maximum moment's going to be this moment here.
The maximum y is going to be h over 2, the distance from the neutral axis.
And i is going to be bh cubed over 12 for the rectangular section.
So if I keep loading it up, at some point, the maximum stress is going to equal the yield stress.
right?
And in our cellular things, is going to equal the yield stress of the solid.
So our beam is one of our edges in the foam, or struts in the honeycomb.
So at some point, is going to equal the yield strength of the solid.
So let me draw the stress distribution again, where we start to have plasticity.
So here, the stress is equal the yield stress.
And it's going to equal the yield stress at the bottom, too, because it's all symmetric, right?
And the neutral axis is still going to be in the middle here.
So it's going to 0 down there.
So initially, when it's just barely reached the yield stress at the outer part of the beam, then this stress distribution would still be linear in between.
But once you start to load it more than that, then the plastic region starts to seep in from the outside inwards.
And what we do here is we assume that the solid is elastic perfectly plastic.
And if you remember, when we said things were perfectly plastic, or if they were elastic perfectly plastic, they look like that.
The stress strain curve for the solid, I'm assuming, looks like that.
So I'm assuming the yield stress in the solid is just a constant.
That if I strain it more, there's no work hardening.
I'm neglecting work hardening
You said inward that the--
In board?
Inward, you said something like--
Inward.
So this is-- let's see.
I didn't bring a bean with me today.
No beam.
Do we have anything beam-like?
Ah, here we have a beam-like thing.
OK.
So say this is my beam, and I'm loading it this way on, OK?
And this is b, and that's h. So this picture here is looking at it that way on, OK?
And this picture here, I've drawn h, but now I'm just looking at the stress distribution across h. And b is into the board.
Is that OK?
Does that answer your question?
No, I mean for the plastic
For this part?
OK.
I'm working up.
I haven't finished it yet.
So this is the same kind of view as here.
I drew it a little bigger.
It shouldn't have draw bigger, I should have drawn it the same height.
But it's the same thing.
OK?
So you'll buy that at some point, we reach the yield strength here.
And if I keep loading it up and I assume that the solid is perfectly plastic, that there's no work hardening, then the stress distribution would look like this.
OK?
And then if it yields more, then it's going to look like that.
And if it yields more, eventually I'm going to get to the stage here, where it's-- let me redraw this.
That would be-- you get the idea, OK?
This will go over here, and down here, and like that.
OK?
OK.
So are we happy with this stress distribution across the cross-section?
Yeah, OK.
So that's when it forms the plastic hinge.
So when it forms a plastic hinge, the stress distribution looks like this.
So these are supposed to be the same size.
They're not quite.
Let's see.
So one of the things I talked about was the plastic moment that kind of characterized that plastic hinge.
And the plastic moment is just the internal amount of moment that the beam can withstand when it's yielded completely across the whole cross-section.
So when we're at this point here.
So you calculate that by saying, that stress there is equivalent to a force.
That stress there is equivalent to a force.
And you get the moment by multiplying those forces times that distance there.
OK?
Because you think of those two forces as being a couple, and the moments the force times the distance between them.
So the plastic moment was sigma ys.
And say we're talking about our honeycomb or foam or something.
That was our cell wall thickness t. So let me call it t instead of h for the foams and the honeycombs.
So this force here is going to be the stress time the area over which it acts.
And let's say we look at it for a honeycomb.
Then I've got the stress is acting over this distance here, and then times the depth into the page, right?
And if it's the honeycomb, that depth into the page is just b, OK?
And then this moment arm here is just t over 2 as well, because that's t over 4, and that's t over 4.
So it's t over 2 again.
So it's sigma ys bt squared over 4 for the honeycomb.
And say we have an open cell foam.
The edges aren't of thickness b, they're of thickness t. So then it's m p is just sigma ys t cubed over 4.
Are we happy?
And really, physically what that is is it means that the beam can't hold any more force.
You can't apply any more force to it.
It's just going to rotate like this once you've gotten to that plastic moment.
That's why it's called a hinge, because it just can rotate like hinge rotates.
Like a door hinge, OK?
How does it rotate?
Well, where's my original picture.
So if this was the beam, when you form the plastic hinge, your beam would just look like that.
And this would be your hinge point.
I'm a civil engineer originally.
We try to avoid this.
So that's why, in the foam and in the honeycomb, that's when it fails is when you get that plastic hinge forming.
OK?
All right.
We have a few more minutes
That kind of looks like plastic buckling
Well yeah, it's not buckling, but it's plastic, yeah.
It's permanent.
Anyone else?
OK. No other questions?
Should we call it a day?
Is that helpful?
All right, then.
It's what I do.
Come on.
It's what I do.
All right.
So I'll see you Wednesday.
And my plan is to grade the tests before spring break, so I shall have it back to you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to start talking about trabecular bone, and we're going to do a bone today and on Wednesday.
And I'm hoping we can more or less finish it on Wednesday.
So hello.
Hold on a sec.
So these are some images of trabecular bone, and you can see that it has a foam-like structure.
And trabecular bone exists in certain places in the body.
There's three main places that it exists.
So it exists at the end of the long bones.
And over here this is a femur.
This is the very top of the femur, and you can see this is all trabecular bone in here.
This is a tibia, and this is the top of your knee there.
And you can see how the bone get more bulbous at the ends, and it's filled with a trabecular bone.
This is a vertebrae.
So vertebrae are actually mostly trabecular bone, and they have a really thin shell of what's called cortical bone, the dense bone, on top of it.
So trabecular bone exists at the ends of the long bones, it exists in the core of the vertebrae, and it also exists in sort of shell or plate-like bones.
So in your skull, for example, there's a layer of trabecular bone in between two layers of the compact dense bone.
And in your pelvis, it's the same thing.
So those of you who took 3032, you remember when I passed around the bird skulls, there was that very porous kind of trabecular bone.
So trabecular bone is of interest medically in three main kind of medical situations.
So the first one is osteoporosis.
So I want to talk a little bit about osteoporosis now, and then we'll talk about it in more detail later on.
I guess, we'll probably start today.
Another medical issue is osteoarthritis, and the properties of the trabecular bone are important in arthritis, and the third issue is in joint replacements.
And so we're going to talk a little bit about osteoporosis, osteoarthritis, and then joint replacements.
And then I'll talk a little bit more about modeling bone like a foam and how it deforms and how it fails.
And then we'll talk a little bit how we can model osteoporosis.
So let me write down some of these things.
I guess we'll start here.
So trabecular bone has a foam-like structure, and what we're going to see is that we can use the models for foams to describe the mechanical behavior of the bone.
It exists at the ends of the long bones.
And at the ends of the long bones, the bones become more bulbous.
And really what that's for is to increase the surface area so that there's cartilage between the ends of the two bones.
So there would be a bone here and a bone there, and there's cartilage in between them that sort of lubricates that joint and makes low friction at the joint.
And the bone gets larger to decrease the stresses on the cartilage.
So if you have the same force and you have a larger area, you're going to have a smaller stress.
So that's why the bone gets bulbous like that.
And then by having the trabecular bone, because it's so porous and lightweight, you're not having a big, dense hunk of bone at the end of the long bones.
So it exists at the ends of the long bones.
And I'll just say the ends have a larger area than the shafts.
And that's to distribute the loads on the cartilage or to reduce the stress on the cartilage.
And then the trabecular bone reduces the weight.
So it also exists in the core of the vertebrae, and in fact, it makes up most of the vertebrae and then in things like the skull and the pelvic bones in shell and plate-like bones.
And so it's the core of a sandwich structure there.
So it's of interest in osteoporosis, in osteoarthritis, and in joint replacements.
So if we start by thinking about osteoporosis, you probably know that osteoporosis is a disease where the bone mass becomes reduced and there's a greater risk of fracture, so there's especially a greater risk of hip fractures and vertebral fractures.
And it turns out in both of those sites, if you just look at these bones here, typically if you have a hip fracture, what happens is the neck of the femur breaks.
So this is called the neck here.
This is called the head, this spherical bit there.
So the neck has a fracture, and you can see most of the bone there is trabecular bone, so it's really carrying most of the load and the same with the vertebrae.
This sort of cylindrical part of the vertebrae here carries most of the load.
It has a shell of really thin cortical bone, but it's mostly trabecular bone.
And when the loads are vertical like this, that's really the trabecular bone that's carrying most of the load.
And people get sometimes what are called wedge fractures where instead of having a sort of a cylinder with parallel faces like this, the trabecular bone fails, and the bone ends up like that so that there's-- yeah, I know.
You make that wincing expression.
It's like ouchy.
And in fact, it's very ouchy for people who get that.
And when you see little old ladies who are all hunched over, that's why.
The bone has actually failed.
It's actually been crushed into these wedge fractures, and there's no way they can straighten it out, and it's quite painful.
So people who look at osteoporosis are quite interested in the mechanical properties of trabecular bone for this sort of reason.
The hip fractures are particularly serious because people become immobilized and then sometimes because they're immobilized, they get pneumonia, and in elderly people they sometimes die.
So something like 40% of elderly patients who are over 65 die within a year of having hip fracture.
So it's not that the hip fracture kills them, it's that they become so immobile, and they can't move, and they can't walk around, and they end up getting pneumonia.
So it's quite a serious thing.
And there's something like 300,000 hip fractures a year in the US, and the cost of treating these hip fractures is something like $19 billion.
So yeah, it's a huge problem.
And as the population is aging, as baby boomers like me get older and older, there's going more people having hip fractures.
So it's a huge deal, osteoporosis.
So we'll say bone mass decreases with age, and osteoporosis is extreme bone loss.
And a little later today I'll show you some pictures of what it looks like when people have osteoporosis.
So the most common fractures are of the hip and the vertebrae, and at both sites, most of the load is carried by the trabecular bone.
And the hip fractures you are the most serious.
40% of elderly patients pass away within a year.
So that's sort of a little introduction to osteoporosis.
The next issue that people are interested in is osteoarthritis.
And in osteoarthritis, there's a degradation of cartilage at the joints, and the stress on the cartilage is affected by the modulus of the bone that presses against the cartilage.
You can kind of magic if you have a fiber compass, for instance, most of the stresses, if you're loading it along the fibers, is carried by the fibers because they're stiffer.
So if you have like say trabecular bone that has varying density, the denser bits are going to have higher moduli, and it's there's going more stress associated with that.
And so the modulus of the trabecular bone can affect how the loads are distributed in the cartilage, and that can affect the damage in the cartilage.
And the shell, as I mentioned before, this sort of shell of cortical bone or the dense bone at the joints can be quite thin.
It can be less than a millimeter.
So I brought my little bones along with me again.
So this is the head of a femur here, and this is a piece of a knee joint here from a tibia.
And you can see just looking at these how thin the cortical shell is.
So you can get an idea of how thin that is.
So osteoarthritis involves a degradation of the cartilage at the joints.
And the stress on the cartilage is affected by the moduli of the underlying bone, and the cortical shell, the totally dense bone, can be quite thin.
So the mechanical properties of the trabecular bone can affect the stress distribution on the cartilage.
And if osteoarthritis gets particularly bad, then sometimes people have joint replacements.
So when it gets really bad, the cartilage is degraded completely, and the bone is rubbing on bone, and that's quite painful.
And when it gets to that point, people generally have a joint replacement.
And so the way the joint replacements are done is say somebody who is going to have a hip replacement, what they do is they chop off the top of the femur.
So they would chop the femur off somewhere around here, and then they have a metal implant that has a spherical ball.
That's like the head of the femur.
And then it has a sort of stem and a shaft here that goes into the hollow part of the long part of the shaft of the femur.
And so they use a number of different metals for this, titanium and stainless steel, and there's a cobalt-chromium alloy are also used.
So you need metals that are biocompatible, aren't going to corrode, aren't going to have degradation products.
And then the bone grows around that implant, and the bone grows in response to mechanical loads.
So the density of the bone depends on the magnitude of the load, and the orientation of the trabeculae depends on the orientation of the principle stresses that are applied.
So let me write that down.
So they cut off the end of the bone, and they insert the implant into the hollow shaft of the remaining bone.
And the metals they use are titanium, stainless steel, and a chromium-cobalt alloy.
And then the bone grows into that implant.
And the bone grows in response to mechanical loads.
So the density of the bone depends on the magnitude of the stresses, and the orientation of the bone depends on the principle stresses.
So one of the issues that comes up in joint replacements is that there's a mismatch in the moduli between the metal and the bone.
So if you think the metal, like something like stainless steel, has a modulus of around 200, 210 gigapascals.
And the cortical bone has a modulus of about 18 gigapascals, and the trabecular bone has a modulus between about 0.01 and 2 gigapascals, depending on its density.
So you're taking the bone out, and you're replacing it with something that's much, much stiffer, and that changes the stress distribution around the remaining bone.
And one of the things that can happen is you can get a loosening of the implant.
So the bone can grow in initially, but over time, you get a different stress field in the bone.
And if you have a different stress field, then the bone can resorb away from the implant and cause what's called loosening.
So if the implant becomes loose, that's clearly not a good thing.
It's a bad thing.
And often orthopedic surgeons don't like to do these joint replacements in young people partly because they don't always loosen, but occasionally they do.
And if they loosen they can go back and do a revision.
But you can kind of imagine after they've chopped the head of the femur off and they put one implant in, it's not that easy to go back in and replace that with another one.
You would need one with a longer stem, and the whole thing becomes a little bit more complicated.
So this issue of stress shielding is what it's called when you have something much stiffer that's shielding the stresses in the bone.
The issue of stress shielding means that they don't like to do the replacements on younger patients unless you can get stress shielding.
And if we just compare-- if we look at the cobalt and chromium alloy, the modulus of that in gigapascals is about 210.
If we look at the titanium alloys that are used, the modulus is about 110.
If we look at the stainless steel-- it's 316 stainless steel-- it has a modulus of around 210.
And then if we look at the bone, the cortical bone has a modulus of about 18, and the trabecular bone has a modulus 0.01 to 2 gigapascals depending on the density.
So after the joint replacement happens, the remodeling of the bone is affected.
So the idea is that the stiffer metal carries more of the load, and then the bone carries less load, and then it resorbs.
And that can lead to this thing called loosening, which is not desirable.
Now, this typically doesn't happen till about 15 years after you've had the implant, so it's not something that would happen right away, but it can happen later on.
So these are all sort of medical reasons why people are interested in trabecular bone because of osteoporosis, osteoarthritis, and joint replacements.
So I wanted to start by talking about the structure of trabecular bone.
And then we'll talk about what the stress-strain curves look like in compression and tension, what are the mechanisms of deformation and failure, and how we can apply our models for foams to the trabecular bone.
So the idea is that the structure of the bone resembles a foam, and here's some SCM images of trabecular bone.
And you can see that the bone has a varying structure.
If it's relatively low density, this is a bone that's almost like an open-cell foam if I didn't tell you that was a bone, you might actually think it was an open-cell foam.
And here's a denser piece of bone, and you can see there's still interconnections between all the openings, so it's not exactly like a closed-cell foam, but it's much denser, and it's almost like there's perforated plates in the structure.
And then as I said the bone can grow in response to loads.
So if you have loads that are more or less vertical, the trabeculae tend to line up and be more or less vertical with some sort of horizontal bracing.
So this is a piece of bone from a knee, the condyle is sort of towards the top of the knee.
And you can see these are sort of plate-like pieces of bone.
They're almost parallel, and not too surprisingly in your knee, the loads are typically vertical, and then there's a little bracing bits that go horizontally here.
So you can get different structures depending on the loading on the bone, and the density of the bone corresponds to the magnitude of the load, and the orientation of the trabeculae corresponds to the orientation of the load
[INAUDIBLE]
Resorb.
So when the bone density goes down, when you lose bone mass, that's called resorption.
So the idea is that the trabecular bone resembles a foam.
And in fact, the word trabecular comes from Latin, and in Latin, it means little beam.
So the foams to form by bending.
They act like little beams, and so the trabeculae are like little beams, even in Latin.
There's a range of relative densities, and you can see in that image up there, you can see that there's a range.
And they range typically between about a 5% in dense and 50% in dense.
So something like 0.1 or 0.2 might be typical.
And the low-density bone resembles an open-cell foam.
And the higher density, it becomes more like perforated plates.
And the structure can be highly anisotropic depending on the stress field.
And then I've got another image here of the trabecular bone.
These images are using what's called micro computed tomography.
So you've probably heard of computed tomography.
Say somebody has cancer, they get put in a CT machine, and they do a scan.
The micro CT is more of a research tool.
It's the same kind of technology, but it's got a much finer resolution, and typically, you put a small specimen into a machine to do this.
So the specimen might be half an inch in diameter and an inch tall, something like that.
So these are done by a colleague, Ralph Muller, who's in Zurich, and this is one of his bread and butter things that he has these images, and he looks at osteoporosis.
And you can see here the difference in the structure for the different densities.
So here's a 26% dense piece of bone in the femoral head.
It looks pretty sturdy and substantial.
Here's an 11% dense piece from the lumbar spine, and here's a 6% dense piece.
And you can kind of see when you go from 26 to 11, the struts get a little bit thinner.
And when you go from 11 to 6, the struts get very thin, and in fact, if they get too thin, the struts resorb altogether, and some of their struts can just disappear.
So when people get osteoporosis, what happens is they first lose bone mass by thinning the struts, but then at some point, the struts just resorb altogether.
And if you think of the struts as a biological material, they have bone cells in them.
So there's little osteoclasts and osteoblasts and osteocytes that live in the bone, the mineral thing, the bony thing.
And those cells have dimensions of 10s of microns, so maybe 20, 30 microns, something like that.
So the struts can't get any thinner than that.
If they get thinner than that, then the cells can't live, and the thing just disappears altogether.
And you can think of from a mechanical point of view, if you lose density by thinning the struts, you can use our sort of foam equations.
And say the density went from 0.2 to 0.1, you could make some estimate of how the modulus and how the strength would vary depending on our foam models.
But if you lose density by resorbing the struts, the struts just disappear altogether, then it's as if you had a steel scaffold or a steel structure of a building.
And now you're starting to remove columns and remove beams.
Yes, I know.
That's not good, not good.
And so we'll talk a little bit more about that when we talk more about osteoporosis, and you can see the consequences of that.
But this image here kind of gives you a little bit of a picture of what the bone structure looks like as it gets less dense.
So I want to talk a little bit more about the bone growing in response to load.
Let me rub off the board.
So you're probably already a little bit familiar with this idea.
So when astronauts go up into space, they often do exercises where they have a treadmill, and they've got springs, and they're pulling on the springs to try to exercise themselves.
And the reason they do that is when they're in microgravity, if they were doing some kind of exercise, they would lose bone mass.
And they will get back to Earth where we have Earth gravity, and they would have a problem.
So you see it in astronauts, in microgravity.
The other place you see this just in everyday life is in professional tennis players.
People have done like x-rays of the bones of professional tennis players, and obviously, they have one arm that they hit the ball with their racquet.
The bones in that arm actually get bigger because they're loading that bone over and over again pretty much every day when they're playing tennis, and they're not loading the other arm.
So their two arms are not symmetrical because of this loading from hitting the racquet over and over.
And the people in 3032 have already seen this, but I couldn't resist bringing up the Guinea fowl experiments again.
So obviously, you can only do x-rays on human.
You can't sacrifice the humans and look at their bones, but you can with Guinea fowl.
And so people have done experiments where they run Guinea fowl on treadmills, and they have one set of Guinea fowl that they run on the treadmill that's horizontal.
They have another set of Guinea fowl that they run on a treadmill that's inclined to 20 degrees, so one would think they might have more stress on their bones from that, and then they have a control group that they don't run on the treadmill at all.
And then what they do is they have a forced plate on the treadmill so as the Guinea fowl is running, they measure the maximum force in they're taking high-speed video.
And then they measure the angle of the knee at that point at which the force is maximum.
And they can see there's a change in the angle of the knee when they put them on the inclined treadmill, not too surprisingly.
And then these are juvenile Guinea fowl that haven't completely matured their bones.
And then after about six weeks of this, they sacrifice the Guinea fowl, and they do scans on the bone, and they look at the orientation of the bone, and they measure what's called the orientation of the peak trabecular density, which is a way of characterizing the orientation of the bone.
And they find that the angle of the knee when the Guinea fowl are running changes by about 14 degrees.
And it turns out the angle of the bone, the orientation of the bone also changes by about 14 degrees.
So the bone has remodeled to match that change in the forces that are applied as the Guinea fowl are running on a treadmill.
So this is all a demonstration just to show that bone grows in response to load.
So let me write down some of this stuff.
So I will say astronauts-- did you see Michael Collins is going to come to the talk at MIT?
When I was a kid in '60s, he was one of the Apollo astronauts.
He was like one of the first NASA astronauts.
Anyway astronauts, so in microgravity, they would lose bone if they don't exercise.
And tennis players, the bones get larger in the arm that they hold the racket with.
And then I'll just write a little bit of notes about the Guinea fowl experiments.
So this was done by-- it's in a paper, Ponzer et al 2006.
So they have one set of Guinea fowl that run on a level treadmill, they have another set that run on a inclined treadmill, and it's inclined at 20 degrees.
And then they have a control group that doesn't run on the treadmill.
And then they measure the angle at the knee at the moment of peak force on the treadmill.
And after six weeks, they sacrificed the Guinea fowl, and they measured the orientation of the peak trabecular density.
And they find that the knee flexion angle changed by 13.7 degrees.
And if you compared the inclined versus the level treadmill, and they found the orientation of the peak trabecular density, which they called OPDD, also changed by 13.6 degrees.
So the idea is that the orientation of the trabeculae changed to match the orientation of the loading.
Then I have a little video here.
Do you like video?
So I have a colleague who's at Harvard who studies animal locomotion, and they didn't do this set of experiments, but they do do experiments on Guinea fowl running on treadmills, and thought you might find this amusing.
So let me see if I can make this work.
[VIDEO PLAYBACK] -Sometimes you walk into a lab and you just think this is what science is all about.
-I just put the Guinea fowl on the treadmill, and this is something that we commonly do.
-Welcome to the Concord Field Station, a defunct Nike missile base turned scientific menagerie.
It's owned by Harvard, and biologist Andy Biewener is the director here.
So think of it as-- -A research lab facility for doing comparative biomechanics and physiology of largely animal movement.
-And the birds are just the tip of the iceberg.
-So do you want to see the baby goat and the emu?
-Obviously.
-OK. We keep it because it's sort of like a mascot.
There used to be a lizard colony.
You can hear the African greys.
Then the jerboas are housed in this room here.
This is where we originally did our pigeon flight studies.
So usually the ones with claws and sharp teeth and aggressive behaviors, you want to watch out for them.
-As you might expect.
But did you know that Guinea fowl-- -They're really lovely to work with.
-Sometimes.
Or that-- -Rats are not very good on treadmills.
-Yes.
That's what a rat treadmill looks like.
And this?
-And this was historically a treadmill of note, the treadmill that they first taught kangaroos on and showed that kangaroos stored energy in their tendons enough that they don't actually increase their metabolic rate when they hop at faster speeds.
-These are the kind of discoveries made here with the use of high-speed video and x-ray machines and semi cooperative animals.
But beyond the basic biology, Biewener says engineers are using this research to build better robots, and it can help improve medical treatment for people with movement disorders.
Today the big excitement at the lab is happening here.
Ivo Ros is studying how heart rate changes when cockatiels fly at different speeds.
So this is a way to look at how much energy it takes to fly, and that cord is measuring heart rate.
But instead of the birds flying faster, the wind changes speed.
-I'm going to turn it on then.
-It's hard to fly fast, and it's hard to fly slow, Ros says.
So the expectation is that the heart rate should be shaped like a U.
But so far Ros is finding that it's a flat line.
It's like the bird goes into a stress reaction when it takes off.
Is that just because of the wind tunnel?
What Ros wants to know is-- - --whether or not they need to be stressed to fly in the first place.
-That's something that Ros is looking into, but today is mostly about training.
-Keep going.
Come on.
-Imagine you're a cockatiel.
A wind tunnel is kind of a strange experience.
-A bird in a wind tunnel has to confront the fact that the world is not moving past, which defies its normal sensory cues.
-Which pretty well sums up the Concord Field Station generally.
For Science Friday, I'm Flora Lichtman.
[END PLAYBACK] And if read The New York Times, Flora Lichtman used to work for NPR and would make these Science Friday videos for them.
But now she has a gig doing things for The New York Times, and she does science videos still.
I don't know if you quite call them videos, but what they do is they have these little paper puppets, and the paper puppets are animated and re-enact different episodes in science.
And it's kind of amazing how they do these little science videos.
So if you Google Flora Lichtman, you'll see more Science Videos with all sorts of things.
I guess the other interesting anecdote is I went to the Concord Field Station once.
And I had done a study on quills and animals that have quills because the quills have a foamy structure in the middle.
So they're carrot, and they have sort of like carrot-like structures.
And they have an outer shell that's dense, and then they have a foamy thing in the middle.
Anyway I did this paper on quills and how they work mechanically.
And Technology reviewed a little article about it, and they said they wanted to take a picture of me with a hedgehog.
The hedgehogs are little European-- they're like little small, cute things.
And I said, well, if you can find a hedgehog, I'm happy to have my photograph taken.
And they had a hedgehog at the Concord Field Station.
So we went out there, and we had these big leather gloves and took a picture.
And I don't if it was Andy or I don't know who it was, but I said one of the people there, what did you do with a hedgehog?
And he said, well, we tried to do the treadmill study.
But hedgehogs are like porcupines.
When they get scared, they curl up into a little ball.
And so they said they would put the hedgehog down under the treadmill, and they would start it up, and it would make a noise, and it would get scared it.
And it would just go into a little ball and kind of slide along to the end, and then it would kind of get flopped off.
So that was the end of the hedgehog experiments.
But they did have wallabies there the day I went.
So they have all sorts of animals that they put on to treadmills, birds that they fly, so it's kind of interesting to go there.
But the main idea here is that there was this set of experiments with Guinea fowl that showed just how precisely the orientation of the bone matches the orientation of the loads
Was there a difference between the control groups?
Ah, so I have slides.
I have slides.
Hang on a sec.
I got distracted by my video.
Sorry.
So here's the sort of schematic of Guinea fowl on treadmill.
And where's the little doo-da here?
So on the level, the knee flexion angle was whatever this is, 76.3, and here was the 62.6, so the difference is 13.7.
And then this kind of table here summarizes these results.
So this is the maximum trabecular density, and this is the angle.
And here we have the incline.
Let's see, the control was the yellow, and the level was the blue, and they've got the values for that peak trabecular density orientation.
So they've got that for the level.
And then they looked at the difference between the incline and the level in the knee angle.
That's what this thing here is.
And then between the level and the control, there wasn't really any difference in the knee angle because the control ones, they were just walking around.
So that's the sort of slide that has the actual data on it.
All right.
And then I showed you the video.
All right.
So we need to do a couple more things before we get to the modeling.
Let me get a drink.
So if we want to use the models for foams to try to describe the trabecular bone, we need to know something about the properties of the solid.
Remember we used the properties of the solid in the models.
So we want to get the properties of the solid in the trabeculae, and there's a couple of ways you can do this.
To get the moduli, you can use an ultrasonic wave propagation method, and you can measure a modulus that way.
And if they do that, they measure a modulus between about 15 and 18 gigapascals.
Another way to do it is to take a piece of bone, do a compression test on it, measure the modulus.
Before you do the test, you put it in the micro CT machine, and you get a picture of the structure, and then you use that as the input to a finite element analysis.
So the finite element analysis is a computer numerical analysis to do mechanical calculations.
And if you know what the modulus of the structure is, you can back out what the modulus of the solid must have been from the fine element thing.
And those sorts of experiments also showed that the modulus was around 18.
And it turns out that moduli is about the same as cortical bone, and the properties of the solid trabeculae are very similar to the solid cortical bone.
So let me scoot over here.
So if you use an ultrasonic wave propagation, people have measured a modulus for the solid in trabecular bone of 18 gigapascals, or you can do a finite element calculation based on micro CT data for the structure.
And then you measure the overall modulus for the trabecular bone, and then you back out the modulus of the solid.
And people who've done that have gotten values of around 18 gigapascals too, and that's very similar to what the cortical bone is.
And so we're going to use the following properties for the solid in the trabecular bone.
We're going to say the density is 1,800 kilograms per cubic meter.
The Young's modulus is 18 gigapascals.
The yield strength has different values in tension and compression.
It's about 182 megapascals in compression.
And it's about 115 megapascals in tension.
So those are the solid properties.
So then if we look at the compressive stress-strain curves, they have the shape shown on the screen there.
And you can see how similar the stress-strain curves are for those for a foam.
So there's the same three regimes that we see for the foam.
There is a linear elastic regime over here, there's a stress plateau here, and there's some densification regime here.
These are three curves for three different relative densities.
As the relative density goes up, the stiffness goes up, the plateau stress goes up, and the densification strain goes down.
So is the same as the foams that we've looked at before.
And if we look at the mechanisms of deformation and failure, people have looked at this.
These are on a whale vertebrae.
So these are tests that are done in a micron CT machine, again, by Ralph Muller's group.
And here the specimen is unloaded, and here's the same specimen loaded.
So you can see this platen has come down a little bit.
And if you look at this column here, this trabecular here, you can see it's bent out and bowed out more.
And people have found that usually the linear elastic behavior is controlled by bending of that trabeculae, and the plateau stress is usually controlled by some sort of buckling.
But it's not elastic buckling.
You don't recover it.
If you take a piece of bone and you compress it, it's going to have a permanent deformation.
So it's inelastic buckling.
And I think we have some more pictures.
This is another example from whale bone from Ralph Muller's group.
So here's the bone unloaded.
Here it's loaded to 4% strain, here it's to 8%.
And you can start seeing right in this area here if you compare with up there, it's starting to form.
And if you go up here to 12% strain, you see that strut right there.
That was this guy up here, and you can see that it's buckled right over.
So people have made measurements like this in observations, and you can actually see the buckling.
And people have also done finite element modeling.
They can take a micro CT scan and input that to the fine element model.
And then if they do the compression and they input the properties of the solid, they can see that they get a buckling kind of failure.
If you have trabeculae that are very aligned-- we have more.
Here's one more in the buckling.
So this is one of Ralph's little movies.
So when it unloads, it looks like it recovers, but this is all just an animation.
He takes several stills and puts them together, and it doesn't actually recover.
It's just the way that it shows.
But again, these are two different specimens of different densities.
You can see how the struts deform.
They bend and then they buckle.
Let me stop there, and I'll put some stuff on the board.
So we'll say the compressive stress-strain curve has the characteristic shape of cellular solids.
And the mechanisms of deformation and failure, usually there is bending followed by, usually, inelastic or plastic buckling.
And sometimes if the trabeculae are aligned like that knee that I showed you, or if the trabecular are aligned or if they're very dense, then the actual deformation is important.
And I'll just say people have found this by making observations using micro computer tomography or by finite element calculations.
And this is a stress-strain curve and tension here, a tension you get failure at small strains, then you get micro cracks in the bone.
And these next plots just show some data for the bone.
So we're plotting the Young's modulus here.
So this is a relative Young's modulus, the modulus of the bone divided by the solid cell wall material.
Here's the relative density.
Here's data for lots of different specimens of bone.
So some of this data is for human bones, some is for bovine bone.
Sometimes the data is taken where the orientation of the trabeculae doesn't line up with the direction of the loading.
So you might have trabeculae that are oriented this way, but you're loading it this way.
There's sometimes different strain rates.
Let's see.
There's different groups, and so there's a huge scatter in the range of the data.
But you can see if you look at it broadly and you look at that whole cluster of data, the data lie close to a line of a slope of 2.
And if you think of the open-celled foam model and you had bending of the cell walls, you'd expect that the modulus would vary [?
to the ?]
density squared.
So that's kind of the limit of how we do the modeling.
We're really just interested in seeing how the properties vary with density.
If you had a particular piece of bone, I don't think you could use the models to exactly predict what the modulus of that bone would be.
And here's the compressive strength here.
So this is the relative compressive strength.
Here we've normalized it with the yield stress of the solid bone, and here's the relative density.
And you can see, again, this line is of slope 2, so that kind of speaks to the buckling-type failure mode.
And I think I have another one here.
This is the tensile strength.
So if you pull the bone in tension, you wouldn't expect to get buckling, you'd expect to get plastic yielding.
And if you got yielding and you use the open-cell foam model, you'd expect a slope of 3/2, so this line has a slope of 3/2.
And this line is sort of towards the upper bound of that set of data.
You can imagine a line that went through it a little bit lower but the same slope.
And so these open-celled foam models, they don't predict the properties of a particular piece of bone because the bone can have some anisotropy to it.
The orientation of these things may not be perfectly lined up with the loading.
But overall the models give you a sense of how the bone is deforming and failing.
So let me write some of this down.
So that's data for the modulus, the compressive strength, and the tensile strength.
And those have been on those plots.
Those values are normalized by data for cortical bone.
I thought somebody was talking.
It's just the chair squeaking.
And as I said the spread in the data is large, and that's due to anisotropy in the bone and misalignment between the bone orientation and the loading direction.
So when people first started doing tests on trabecular bone, they typically were orthopedics labs.
And the orthopedics labs tended to initially cut the bone specimens on anatomical axes.
So they would do you know the superior-inferior, or the medial-lateral, or the posterior-anterior.
But the bone orientation didn't line up with those directions.
So the bone might have been this way, but they were loading it this way, and so that gave this misalignment.
And there could be some variation in the solid properties too.
So you could imagine some solid might have more micro cracks than another.
So if you took say human bone of different ages, you might expect the older bone to have more micro cracks in it.
So these plots put a lot of data together, and then the lines are based on models for open-cell foams.
So the relative modulus goes roughly as a relative density squared, and the cell walls are bending.
And the compressive strength goes roughly as the modulus squared, and that's related to this plastic buckling.
And then the tensile stress or tensile strength depends on the formation of plastic hinges, and it goes roughly as the density to the 3/2 power.
And one observation that people have made is that in compression, if the modulus and the strength both go as a density squared, then the ratio of the strength to the modulus is just a constant, and that, in fact, is just the strain at failure, or the strain for that say, the plateau.
And that's a strain of about 0.7%, and that's pretty consistent in trabecular bone.
Let's see.
And we said sometimes the bone was relatively aligned.
So here's that picture of the femoral condyle again in the knee, and you can see the bones lined up.
If you have bone that's lined up like that and you load it along the direction of alignment, then you can get axial deformation in the trabeculae.
And then you would expect the moduli would go linearly with the density.
And here's some data for the Young's modulus and the compressive strength of bone that was fairly aligned.
So this was selected to be aligned.
So here's the modulus here.
And the square data points are the longitudinal direction, and the little diamond, these little stars, are transverse.
So here's a line of slope 1, and again, they don't all lie perfectly on that line, but roughly the slope is about 1 there.
And then similarly here, this is the compressive strength.
Now the little squares are the longitudinal data, and they're not exactly on a slope of 1, but they're more or less on a slope of 1.
So I'll just say in some regions, the bone may be aligned.
And then axial deformation is important.
And then you would expect the modulus to go linearly with the density and the strength to go linearly with the density in the longitudinal direction.
Then finally I wanted to finish up the bit on the modeling by making one of these plots a little bit like we did for wood.
So here's the Young's modulus of bone plotted against the density.
The trabecular bone is down here.
It's sort the lowest density.
And then this is the collagen that's in the solid part of the bone, and this is hydroxyapatite, the mineral.
So the modulus of hydroxyapatite is around 120 gigapascals, and the modulus of collagen is somewhere around 5.
And if you make composites of collagen and hydroxyapatite, their moduli are going to be in this envelope here, and compact bone, the modulus fits in around here.
Remember I said it was around 18 gigapascals.
So then if you take a compact bone and you turn it into trabecular bone, you'd expect the modulus would go down along a slope of 2.
So here's our little slope of 2, and more or less that's what you see with the trabecular bone.
So the idea is that the models give you kind of a general idea of how the bone is behaving, but it's not really meant to predict a particular piece of bone.
Because a particular piece is going to have a particular geometry.
Typically they're not equi ax and isotropic.
All right.
So are we good with the general overview?
Are we good with how fewer equations there are now that we got past the first part of the course?
So I'm going to talk a bit more about osteoporosis, and I'm going to talk about some modeling that my group did to look at the consequences of osteoporosis.
And then later on we're going to talk a little bit about using metal foams as a possible replacement material for a trabecular bone as well.
And I have a little bit of a talk on using trabecular bone in evolutionary studies to see whether or not a species was bipedal or quadrupedal.
So I think I talked about this a little bit in 3032, but I have more slides and more stuff I'm going to talk about.
So let me get myself organized.
So osteoporosis comes from the Latin, and it actually means porous bones.
So osteo means bone, and not too surprisingly porosis means porous.
So this next slide gives you some idea what osteoporotic bone looks like.
So the top slide is normal bone in a 55-year-old woman.
These are sections from the lumbar spine.
And that bone up here is 17% dense, so the relative density is point 0.17.
And this is a section from the same area of bone in an 86-year-old woman, and it's 7% dense.
So you can see there's a huge difference in the density, and you can start to see what happens when you lose bone mass.
So if you look at this bone up here, it's all well connected.
Each little trabeculae is connected to its neighboring friends.
And you can see down here, I mean, you look at this and you kind of go ouch just looking at it.
Because this piece of bone here is just kind of dangling off, not connected to anything.
And you can see the struts have gotten thinner, so they've lost bone mass by thinning.
And then as I said when the thickness gets less than they are roughly equal to the size of the cells, then the cells can't live anymore, and the bone strut just disappears altogether.
So it's not too surprising that if you lose this much bone mass, there's mechanical consequences, and there's a greater risk of fracture.
And as I said the two most common types of fractures are hip fractures and vertebral fractures.
So let's see here.
So as people age, everybody loses bone mass.
And happily for you and not so happily for me, the bone mass peaks at about 25 years old.
So you're probably either not at the peak or just barely at the peak.
And then it decreases after that every year.
I'm considerably older than you.
And in women, when you go through menopause, the cessation of estrogen production increases the bone loss.
And so typically, osteoporosis is most common in post menopausal women.
And osteoporosis is defined as a bone mass 2.5 standard deviations or more below that of a young, normal mean.
So it's not like you fall and break your hip and they say you have osteoporosis.
It's based on the bone mass.
And as I said, the trabeculae thin and then they resorb completely.
So anybody here take Latin?
Yes.
I did Latin for one year in high school.
So trabeculae, with an E on the end here, trabeculae, I suppose, is the plural.
Trabecula with an A is singular.
And that comes from Latin.
So you don't say trabeculas.
That's a no-no.
All right.
Let me get rid of these.
So if we saw that the strength of the bone varies as the density squared, you can begin to see how sensitive the strength is going to be to this bone mass loss.
So say you went from a density of 0.2 to a density of 0.1, then the densities changed by a factor of 2 so that the densities gone down by 1/2, but the strength is going to go down by a factor of 4.
You're going to have the strength to be a 1/4.
And so you're going to have a big change in the strength.
And you can imagine is the trabeculae thins, this buckling gets easier to happen.
And then once the trabeculae begin to resorb as they disappear altogether, it's like I said.
it's like having a building's framework.
Now, you're removing beams and columns, and the strength is going to go down even more dramatically.
And the way we model the osteoporotic bone is we use finite element analysis.
So before we talked about using the unit cell for the honeycomb.
But to use the unit cell, you have to have repeating unit cells, and obviously, you don't have that.
You've got local variations in what the structure looks like.
And we also used a dimensional analysis.
And the dimensional analysis relies on the geometry being similar from one specimen to another, and you can't really rely on that either for the osteoporotic bone.
And so what we've done is-- and this is what other people do as well, is use finite element modeling to represent the bone.
So initially what we did was we used a 2D Voronoi model.
So remember we talked about Voronoi honeycombs and Voronoi foams.
So I like to start out with simple things, so we started out with 2D Voronoi model for a honeycomb.
Then we did a 2D representation of vertebral bone.
And then we had a 3D Voronoi.
And I had a couple of students who did this.
Matt Silver was the one who did the first two, and Sereca Vagilla was the one who did the last one.
So let's see.
I think that's probably a good place to stop there for today.
So next time I'll talk about the modeling of osteoporotic bone, and we might talk a little bit about metal bones as a substitute for trabecular-- metal foams as a substitute.
I don't think we'll get to the evolution stuff.
We probably won't quite finish next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time we were talking about trabecular bone and that it's this porous kind of foam-like type of bone.
And we talked a little bit about the modeling.
And I think I got as far as starting to talk about osteoporosis.
And I wanted to talk today about how we can model osteoporosis using those voronoi honeycombs that we talked about a while ago when we were talking about the structure.
So Bruno has a project on trabecular bone for the class.
And he needed bone samples.
And so we talked about different kind of bone samples we could get.
And the thing is if you get human bone-- well, there's all sorts of issues about just handling human bone and permissions, and it's complicated.
So that was too complicated.
We've used bovine bone before.
You just go to the slaughterhouse and get bovine bone.
But one of the things with trabecular bone is because it grows in response to loads, the geometry of it can be different, sort of architecture can vary from one spot in the bone to another.
And I have a colleague who started doing tests on whale bones, just because it's a way of getting nice, uniform bone.
So I was on a Ph.D. Committee a few years ago for a student who was in the Woods Hole program.
She was at MIT, but was doing Woods Hole thing.
And I don't know if you've heard of the Atlantic right whales, the North Atlantic right whales.
They're endangered.
There's about 500 of them left in the world.
And they migrate between like typically the Bay of Fundy and off the Florida coast.
So they go up and down the coast.
And they sometimes get hit by ships.
And then bones break and that kills them.
And her study was on ship impacts on right whales.
And so I got to know people at Woods Hole who worked on whales.
So Bruno, I called up my friend at Woods Hole.
And the Woods Hole guy didn't have any bones.
But he put me in touch with somebody at the Mass Fish and Wildlife Department.
And he had a couple of whale vertebrae that he was willing to give up.
So I got one of them for you.
So here it is.
So I went out to the Mass Fish and Wildlife yesterday.
And they produced this bone for me.
I could either pass it around.
It's not too heavy.
Or you could come up and look afterwards.
Shall I pass it around?
Or do you want came up afterwards?
Maybe come up-- pass it?
OK.
So one of the things is our like vertebrae, there's these things that stick up like this.
There's one that's missing off of this bone.
There should have been one down here too.
But this is sort of what's called the body of the vertebrae.
And in human vertebrae it's about that big.
But it's the same kind of general structure.
There's these kind of bony plates that come off.
And the body is almost all trabecular bone.
So you can see on this side, this is a growth plate here.
But on this side, there normally would have been a thin shell of the cortical bone.
And when I pass it around, if you look up at this point here, you can see there's just a little bit of that left.
But it's kind of gotten worn out.
And the rest of this, if you look, you can see it's the trabecular bone.
And you can see it's pretty uniform, which is why I thought this might be good for your tests.
So I thought you should get in touch with Mike Tarkanian.
And I talked to him a little bit about cutting it with a water jet cutter.
So I think we could use the water jet cutter.
And I think I emailed you.
If you could cut it in half, I'd like to have some pictures of it cut in half.
And he's got a diamond corer, cylindrical corer.
So you could make little cylindrical specimens.
And I know if you want to do compression tests or beams
Well, I was planning to do compression tests
Yeah, so you could probably use some kind of a bands or something to cut them up.
So, and this is where the spinal column goes through in the whale.
So there you have it.
Anybody want to ask me anything about the whale bone?
OK, so let me pass that around.
And I guess I have to tell you a couple of other stories.
So while I was there, they have a brand new building.
And it's all got solar panels and geothermal heat.
And it's all very groovy.
And the guy who I was talking to about the bone, he wanted to give me a little tour of the building.
And he said, oh, you've got to see your freezer.
OK, so the freezer.
So he opens the freezer door.
The freezer's like a room.
And there's like a bear.
I'm serious, like a bear on the floor, like a dead bear on the floor of the freezer.
And he said it was like a two-year-old bear that had, I guess, just come out of hibernation a couple weeks ago.
I think it had gotten hit by a car or something.
And somebody must have called them up.
And they have it.
So they had this bear.
They had like several deer.
They had a coyote.
They had like boxes full of all kinds of animals.
So anyway it was kind interesting to see all these animals there.
And you know what he said about the deer?
He said, normally deer in Massachusetts don't starve.
Like if you live in the suburbs, you actually have a problem with deer eating your vegetable garden because there's deer all over the place.
But he said, this winter, you know how much snow we've had and how cold it's been?
He said, deer have been starving this winter.
And I think a couple of the deer they'd had had actually starved to death.
And people had called up.
And they had come and kind of collected the carcasses.
So anyway that was my little trip out to Mass Fish and Wildlife yesterday.
OK, so let's go back to talking about the bone.
And I think last time we kind of left off more or less here.
So this was a slide of what osteoporosis looks like.
And you can see the bone loss is a combination of thinning of the struts and resorption of the struts.
And we wanted to try to model this, sort of an engineering sense.
And the way we did that is we used our voronoi honeycombs.
So the bone has an irregular structure.
And we wanted to look and see if we could model something that had an irregular structure.
So we used the voronoi honeycomb.
And if you remember when we talked about the structure of cellular materials earlier on in the course, we said that these are generated by putting down random seed points and then drawing the perpendicular bisectors.
And if you have a constraint that the seed points can't be closer than some exclusion distance, then you get structure where you have cells that are not all exactly the same size, but roughly the same size.
So that's what we did here.
We got this structure here.
And I had a graduate student, Matt Silva, who did a lot of these studies.
And then he used this and analyzed it using finite element analysis.
So the first thing he did was he calculated the elastic moduli of the structure.
So he applied loads.
We calculated deformations.
Figured out elastic moduli.
And so this plot here shows a comparison between the analytical equations that we derived at the first part of the course and what he calculated for these voronoi honeycombs for the final element analysis.
So here's Young's modulus down here, for example.
In the closed form, the line, that's just the analytical equation we had originally.
And the little dashed line is his finite element.
Here's the Shear modulus down here.
And here's the possum's ratio up here.
So you can see, there's a pretty good agreement between these two things here.
So let me write some of this on the board.
And then I can keep going after that.
So for the 2D voronoi honeycomb, we have the random seed points.
And we used the perpendicular bisectors.
And we used a minimum separation distance between the points.
So we generated the structure.
And then we did a finite element analysis.
And from that, we calculated the modulus.
And what we found was the finite element analysis results were pretty close to our closed form analytical model.
And if we think about the modulus, the modulus is the average stiffness over the whole honeycomb.
And when we look at the strengths next, the strength is going to be related to the weakest few struts.
And we're going to find that the strength doesn't work quite the same way.
So first, we got the modulus.
That's sort of the simplest thing to calculate.
And then after that, we wanted to calculate the compressive strength.
So we did a similar thing.
We set up the voronoi honeycomb in the finite element analysis.
We had a few more elements along the length of each strut.
And then we modeled the elastic buckling and the plastic failure behavior.
And we looked at honeycombs that had different densities.
And the lowest densities failed by buckling.
And the higher densities failed by a sort of plastic yielding.
And we assumed that the cell walls were elastic, perfectly plastic.
Remember we said if we have a material where-- this is for the solids-- so say this is the stress in the solid and that's the strain in the solid.
If it behaves like that, we say that's elastic, perfectly plastic.
So we modeled the walls as elastic, perfectly plastic.
And we made the ratio of the solid modulus to the yield strength similar to what it would be for bone, which is about 0.01.
And for that particular value, the transition between the elastic buckling and the plastic yielding failure was at a relative density of about 0.035.
So then what we did was we analyzed structures that were a little lower than that were equal to that and then a little higher than that.
So we would try and see what happened with these different failure modes.
And then we found that if we look at the compressive stress strain curve, this model here had a relative density of 15%.
And the transition occurred at a relative density about 3.5%.
So this one here failed by a plastic failure.
You can see, if we unload it, there's some plastic deformation.
We get a little strange softening here, which is kind of characteristic of the plastic failure.
When we look at the overall deformation of the honeycomb, we saw local kind of failure, like we do in aluminum honeycombs.
So that was the kind of stress strain curve for a relatively dense honeycomb that failed by yielding.
And then this is one for a much lower density.
This is now point 1.5%.
And this one fails by elastic buckling.
And if we load it up and unload it, we recover most of the deformation.
Barry, do you think you could make that stop?
Yeah
There's a chance it could be--
Below.
Just because we're recording it.
It just doesn't seem very good.
OK, so what we did was we made five different voronoi honeycomb.
So we had five sets of seed points.
And we had five slightly different geometries.
And then we averaged the results of those.
So if we make that calculation, this is the strength of the voronoi honeycomb here divided by the strength of the periodic regular hexagonal honeycomb.
And this was plotted against relative density.
And you can see the strengths for the voronoi structure are a little bit lower than for the regular periodic hexagonal honeycomb.
And they reach a minimum here.
And the minimum's around about 0.05 relative density.
So this was the 1.5.
That was a 3.5.
That's 5% and 15%
Why is there like a wide gap between 0.05 and--
Well, I think because we felt-- I think this one failed by some combination of-- there was a limit to how many of these we were going to do.
And this is what we chose to do.
We wanted one that we knew was going to fail by plastic yielding.
And that was this one.
And we wanted one that we knew it was going to fail by elastic buckling.
And we wanted a couple in between.
So we didn't do-- I guess we were lazy is the real reason there's not another point in the middle there
It just seems like there might be a variable--
You think it might go [CAREENING NOISE] like that.
Except there's a physical reason why this happens.
And I'm going to get to that in a minute.
So let me I'll finish explaining this, and then I'll put the notes on the board.
So first of all, the strength is less than the regular hexagonal honeycomb.
And then there's also this minimum here around about 5% density.
And I think the reason that the strength is not the same as in the voronoi in the periodic honeycomb is because if you look at the distribution of strains-- or you can think of the distribution of stresses-- so these were the normal strains at the nodes in the honeycombs.
And the distribution here is for the voronoi honeycomb.
So the voronoi honeycomb, you have lots of members of different lengths.
There are different orientations.
And so there's some distribution of stresses and strains in each member.
And that gives you the distribution.
In the regular hexagonal honeycomb, if you look at just the nodes, there's really just the vertical member, which has a certain strain.
And all the vertical members are going to have the same strain at the nodes.
And then the obliques members, if you just look at the nodes, there's just going to be a maximum tension and a maximum compression at the nodes.
Because there's a unit cell and it repeats, all the oblique ones are going to have the same maximum and the same minimum.
So the dashed lines here are for the regular hexagonal honeycomb.
So the thing to observe here is that the voronoi has some strains and correspondingly some stresses that are outside the range of the regular periodic honeycomb.
And so if there's parts of it that are seeing higher strains and higher stresses, it's going to fail at a lower load.
So I think it's this distribution because you've got this random structure, and you've got different lengths and different orientations of the members.
So that's one reason why these strengths are less than the periodic structure.
I think there's a minimum here, because-- I think before the test I mentioned there's some interaction between elastic buckling and plastic yielding.
And when you get that interaction, that also reduces the strength.
And so there's a minimum near where the crossover is between the elastic buckling and the plastic yielding.
So let me write some notes of the compressive strength.
So we'll say the cell wall-- so the cell wall elastic, perfectly plastic.
And the yield strength relative to the modulus for the solid was 0.1, which is pretty much what it is for bone.
And we assumed that the possum's ratio of the solid was 0.3.
And for this value of sigma Ys over Es, the transition between elastic buckling and plastic yielding is at about 3.5% relative density.
So then we made models with densities that were a little bit less than that and a little bit more than that.
And then the strengths we got from the voronoi were between about 0.6 and 0.8 times what we got from the periodic.
So then what we looked at were these maximum strains of the nodes.
And we found that because the voronoi honeycomb had a much broader distribution of those strains, that led to the lower strengths.
And then we found the minimum strength was at a density of about 5%.
And if you think just about a pin ended column, and you make a plot of the strength-- so I'm just going to say the strength of that columm-- against l/r, the slenderness ratio.
So say it's a cylinder, l would be the length.
r would be the radius.
If you just had Euler buckling, you get a curve that looked like that.
The Euler buckling load is pi squared EI over l. squared.
So I goes as r to the 4th.
And if we get the stress, then it's going to be that divided by the area of the column.
So it's pi squared E. Moment of inertia is pi r to the 4th over 4 l squared.
And the area is pi r squared.
So it goes is as r over l squared, or 1 over l over r squared.
So this would be the Euler buckling.
So that's elastic buckling.
But at some point, the column is going to yield.
If I make it really, really short, then it's going to yield before it buckles.
And at some point, this would be the real stress here.
And this would be failure by the plastic yielding.
And in practice, there's not like a sharp corner here.
You know, if you made columns of progressively longer length, and you tested them, the little short ones would yield.
So they'd be along here.
But they don't kind of yield and the next one buckles.
In fact, you get something like that.
So that when you're near that transition, when you're near this point here, the actual failure stress is a little bit less than that.
And I think that's partly what's going on over there.
It's the minimum.
OK, so those two studies, we just looked at the modeling and the strength of honeycombs.
And we were kind of looking at how does the random structure change the properties.
So the randomness of the structure didn't really change the modulus much at all.
But it did affect the strength.
And then the next thing we did was we looked at putting defects into the bone.
So we knew that the bone, partly the density is reduced by thinning of the struts and partly by resorption when there's a lot of density loss, a lot of bone loss.
So we wanted to look at putting defects where we actually removed some of the struts.
So we did another series of voronoi models.
So we did another series of tests where we looked at it if we have a certain density that we start with and then we look at losing the same bone mass or relative density in our honeycomb, and we look at what happens if we thin the struts versus if we remove struts.
So these plots here show that.
And Matt Silva also did this.
So this is the residual modulus plotted against the reduction in relative density.
So residual modulus is the modulus after we've reduced the density by some amount relative to the initial modulus.
And if we have an intact honeycomb where we just thin the struts, we don't remove any of the struts, if we have an intact honeycomb, as you reduce the density, the modulus just goes down like that.
So that's really just the same as those analytical formulas that we had.
But if you start removing struts, not too surprisingly, the modulus goes down substantially more.
And at this value here, I think it was 30% or 35% density reduction, you reach what's called the percolation threshold.
At the percolation threshold, you have two separate pieces of material.
So obviously the mechanical properties are going to go down to zero when you reach that percolation threshold.
And then this plot over here is the same sort of thing, but now for strength.
So it's the strength of the bone, or the strength of the honeycomb, after you've either thinned or resorbed the wall, divided by the strength of the intact honeycomb.
And again, you're reducing the density here.
And this is for the intact model where you're just thinning the struts.
And this is for the models where you're removing the struts.
So you can see that if you think of-- this is kind of a simple model, but if you think of the bone, if you first thin the struts, you're going to lose a little bit of strength.
But then if you start removing the struts, you're going to lose a lot of strength.
So that's really where people run into-- there's much higher risk of fracture once you get to the point where you might be resorbing struts.
OK, so let me see, what's next thing?
OK, let me just finish the slides, and then I'll put the notes up.
So this is the same sort of thing, just plotting the strength and the modulus on the same plot.
So you can see the shape of the curves is very similar.
The modulus is a little bit more sensitive than the strength.
And here we are thinning, and here we are removing the struts.
And then the next step was that we made a model that was more similar to bones.
So let me write down the notes for this.
And then we'll do that one that's more similar to bone.
Thought it didn't makes sense.
OK, so then we were interested in trying to model something that was a little closer to the structure of bone.
And so we set up this model here.
So we started with just a square voronoi.
So you just force the points into a square, voronoi, or a square pattern, you get a square voronoi.
And then what we did is we just perturbed the points a little bit.
And we got this perturbed voronoi array.
And so we made this model here.
And we took a piece of vertebral bone.
And we measured the angle of orientation of a lot of the struts.
And we matched our voronoi model to that distribution in the bone.
So our model looked something like this.
So you can kind of see how it's more or less vertical and horizontal, but not exactly.
And here's a sort of comparison with a slice of vertebral bone.
And again, because the loads are more or less vertical in the bone, the trabeculae tend to orient that way.
And then have some horizontal struts.
So here you can see on the left, we've got a voronoi model that's more or less representative of the bone.
And we've removed some of the struts.
So we're going to the same thing with this model.
We thin the struts.
And we remove the struts.
And we try to see what the residual strength is.
And you can see there's for of at least a 2d model this isn't a bad representation of the vertebral trabecular bone.
And this was the stress strain curve for both the specimen of the vertebral bone that was tested and then the honeycomb model that we made to kind of match it.
So a similar kind of behavior.
This is how our model failed, this sort of a local band of struts that fail.
Let's call it local deformation band.
And then what we did was we thought about changing the density.
And what we did was we removed either horizontal or vertical struts, or we thinned either the vertical or the horizontal struts.
So these are the same kinds of plots as I showed before.
This is the residual modulus.
This is the residual strength.
This is the density reduction.
And here we're reducing the density by making struts thinner.
So it's still intact.
We haven't removed any struts.
But each of the struts gets thinner.
And this top line is for thinning the longitudinal or the vertical struts.
And this was for thinning the transverse or horizontal struts.
And then this is for removing struts here.
So we're removing a bigger number.
So the more we remove, the more the density changes.
And then this plot here is for the strength.
So again, this is thinning.
So we're moving either the horizontal or the vertical ones-- I'm sorry, thinning either the horizontal ones or the vertical ones.
And then this is for removing struts.
And again removing more to reduce the density more.
So you can kind of see we're kind of working our way to more complex models here.
So this one here was looking at the bone.
Let me write some notes for this.
And then we did a 3D voronoi model.
So I'll do that one next.
Oop, over here.
So I'll just say the model was adapted to reflect the trabecular bones study in the vertebrae more.
So we perturb a square with array of seed points to get a structure that was more like the bone.
And then we looked at the reduction in the thickness and the number of strides in the longitudinal and transverse directions.
OK, and then the next model we did was the 3D version.
So we made a same kind of thing.
But with the 3D model, we had fewer cells in that model, because once it goes to 3D, you've got more struts in each cell.
But this was the same idea.
We uniformly thinned the struts, or we removed the struts.
And again, you can see removing the struts is a much bigger effect.
And also the other thing to look at here is for 3D the percolation threshold is 50%.
So it kind of makes sense that if it's in 3D, and you've got struts in all directions, you're going to have to remove more of them.
You're going to reduce the density more to break it into two separate pieces at the percolation threshold.
So that was the 3D model there.
And then this is just a comparison of the 3D with the 2D for the modulus.
We just did the modulus for the 3D structure because it sort of gets computationally more involved.
So the 3D, these two lines here corresponded to removing the struts and the change in the modulus.
And the little triangles were the 3D voronoi calculation that we did.
The little crosses here were the same sort of calculation done for tetratridecahedron.
One of my former students had loaded that.
And then at the bottom here are the lines for the 2D structures, for either a regular hexagonal cell or for a 2D voronoi cell.
And you can see, there's not a huge difference whether or not you take a regular structure or a voronoi random structure.
But there's a fairly significant difference between the 2D and the 3D.
So the 3D, just you have to remove more material before you get the same reduction in modulus.
So let me write some notes for that.
So it's the same kind of analysis, but just with a 3D model.
So do you see the idea with these models?
It was an attempt to look at a way that you could computationally estimate how much modulus loss or strength loss you get by either thinning the struts or removing the struts.
So I gave you a way to model the osteoperotic bone.
Yes
Can you clarify the percolation threshold?
So the percolation threshold is say you have some network and you start removing things.
If your remove enough, you have two separate pieces of things.
If you remove enough struts, you have two separate pieces.
That's called the percolation threshold.
So I think it's-- you want to know why it's called that?
So I think that originated because it wasn't used in this kind of context.
I think instead it was used in a context where imagine you have 2D plate.
So you put 2D holes in it, little circular holes.
And they we're looking at flow of a fluid through the plate.
So you can imagine, if you put enough holes, eventually they line up or they-- line up is not the right term-- but there's enough of them that they connect.
And you end up with two separate.
You have a path through for the fluid.
That's called the percolation threshold.
But in mechanics it's sort of two separate pieces.
OK, does that makes sense?
So you know, at the percolation threshold, the stiffness is zero because you have two separate things and the strength is 0
So the two doesn't have to be like separated by--
It could be-- yeah, yeah, it doesn't have to be a straight line
Does it make a threshold between whether everything is interconnected?
Yes.
Or there's some path that separates them.
Yeah, that's what it is.
OK, so that's the end of the bit on osteoporosis.
I had a couple more things I wanted to talk about on bone.
So the next part is on the idea of using metal foams as a bone substitute materials.
So when they make hip implants-- so they're typically metals.
They're titanium or stainless steel or something.
Often what they do is they coat the outside with some sort of porous coating.
And the idea is the porous coating allows the bone to grow in.
So you can kind of imagine, like especially on the stem and around the head of a femur, you'd like the bone to grow into that to attach.
And having a porous coating helps.
And there's a couple of ways they do it currently.
They use porous sintered beads.
So they put little beads of metal on the outside.
And the idea is that the bone grows into the little gaps between the beads.
Or sometimes if they have-- not so much for hip implants but sometimes when people have say car accidents and their face get sort of smashed up, and they have to have, say, a plate put in their face, and they need like a flatter plate, they use like a wire mesh.
And they have sort of a wire mesh that goes on the outside.
And it's the same thing.
It's the idea to try to get the bone to grow into that plate.
So some people are thinking instead of using the porous sintered beads, or instead of using one of these wire mesh plates, that you could use metal foams.
And so there's been some interest in using metal foams in coatings of implants.
And longer term, there's been some interest in trying to make sort of a vertebral body that would involve using a metal foam from the vertebral body.
So you know that whale bone that we just passed around, that vetebral body, that cylindrical part, it's almost all trabecular bone.
So there's some interest in trying to use metal foams for spinal surgeries.
Maybe not to replace the whole body.
But to use in part of the surgery.
So I have a little slide here which shows a bunch of metal foams that people have made with the idea in mind that perhaps some of this could be used in orthopedic surgery.
So these are some different kinds of metal foams.
And these ones are made from titanium or tantalum.
So typically, the metals that they use in orthopedic implants are the cobalt chromium alloys.
Titanium, they sometimes use tantalum or stainless steel.
And they use those because they're biocompatible.
And they're very corrosion resistant.
So these ones here are mostly titanium.
And this is one that's a tantalum.
So let me just go over how they make them.
And then I've got another slide that goes over it in more detail.
And I'll write a few notes down.
So this guy on the top left up here, that's made by taking an open cell polyurethane foam, like a seating foam, like a cushion.
And then what they do is they heat that up in an inert atmosphere, so that they are left just with the carbon.
So it's a sort of vitreous carbon.
And then what they do is they coat that carbon by a CVD process with tantalum.
And so they end up with a tantalum foam with a very thin layer of carbon at the core.
The carbon makes up something like 1% of the final composition and the tantalum is the 99%.
So that's sort of a replica process there.
This one here is made by another replica process.
They take an open cell polyurethane foam.
They infiltrate it with a slurry, which has the titanium hydride particles in it.
Remember when we talk about processing of the foams at the beginning, I said, if you heat up the titanium hydride, eventually the hydrogen would be driven off, and you'd be left with the titanium.
So they do that, and then they sinter it, and they get a titanium foam.
This one is made by a fugitive phase process.
So the idea with a fugitive phase is you burn off some phase.
So you could mix powders, consolidate the powders, then you burn off one of the powders.
And you're left with the other one.
And then you need to sinter that together to make it have some reasonable mechanical properties.
This one here is made by using a foaming agent.
This one's made by expansion of argon gas.
I think when we talked about the metal foams, we talked about the idea of packing, say, titanium powder in a can.
And then you evaporate the can.
And you then pressurize the can with argon gas.
And then you heat treat it.
So you heat it up.
And as you heat it up, the argon gas expands.
And you're left with the pores.
This one's made by a freeze casting process.
I have a slide.
And I'll talk about that in a minute.
This one's made by a selective laser sensory.
So it's like a 3D printing.
But instead of printing in ink, this time you have a bed of a powder.
And you've got a laser that selectively sinters.
So you turn the laser on and off.
And where it's on, it's going to bind the material.
When it's off, it's not going to bind the material.
And then you raise the thing.
You make a little bit more powder.
You do it all over again.
And you can get different patterns.
And then this is made by a sort of process in which they take powders and press them and ignite them.
But I think that's not very commonly used.
So this is some more details about how they might do it.
So this is the fugitive phase process here.
You could take a titanium and a filler powder, pack them together.
You know, you'd mix them up, pack them together.
You would raise it to a certain temperature to decompose the filler.
So typically the filler decomposes at a lower temperature than what you sinter the powder, the metal powder that's left.
So you decompose the filler.
You drive that off.
And then you sinter the metal powder.
And you're left with porous titanium.
This one's the expansion of the foaming agent.
So you take your titanium powder.
You might have a binder and then a foaming agent.
Mix those all together.
They heat them up until typically the binder becomes a liquid.
And the foam foams up the liquid binder.
Then they drive off the binder.
And then they sinter at a higher temperature the titanium.
So you you've got a porous titanium left.
This is the freeze casting or the freeze drying process.
So here they would take titanium powder and put it in agar.
And the agar's in water.
So the agar is like a jelly, like a gel.
But it's mostly water.
And then if you freeze it, what happens is the water freezes.
And it drives off the titanium agar into the interstitial zones between the frozen ice crystals.
So these little dots here are the ice crystals.
The ice crystals are growing as it gets colder.
And as the ice crystals get bigger and bigger, you're left with the titanium and the agar in between those ice crystals.
And then if you sublimate the ice off, you're left with a porous thing.
And you can sinter the titanium if you want to make it a little more dense.
And this is a rapid prototyping thing here.
So you spread the powder.
You could either print a binder or you could use a laser to sort of almost weld the particles together.
Then you would drop the piston, put more powder down, have the binder go again until you made your product.
And then you would get rid of all the unbound material.
And you'd have your final product.
OK, so these are some of the methods they can use for making metal foams.
I don't know, should I write notes?
Or are you good if I just put the notes on the website?
I'll put the notes on the website.
And then this is a stress strain curve for a titanium foam.
So it looks like all these other kinds of foams and bone and wood and everything else that we've looked at.
And this is some data for titanium foams that are made by different processes.
And we've just taken different data from the literature and put it together.
So this is the modulus here.
This is a slope of 2.
You can see some of the data is sort of near that slope, but below it.
And there is obviously a large spread in the data too.
But if you go back and look at the different types of structures, then not all these have this kind of typical foam-like structures.
The structures aren't all quite like a foam either.
Yeah?
So is there any objective in this to make the foam similar in structure to the bone that will be growing into it?
Or does it just need to be scaffolding?
I think for this, they just want to make a porous thing.
And they're thinking about coatings.
So the coatings aren't necessarily similar to the bone.
When we finish the section on bone, we're going to start talking, I think, about tissue engineering.
And when we talk about tissue engineering, people have made scaffolds to try to make them the same shape as the anatomical part that they're trying to mimic.
And then they make a cellular kind of core.
So they have made some more of an attempt to do that.
I could give you a sneak preview.
Would you like a preview?
So these are some scaffolds that are generated from a making different kinds of tissue.
These aren't all from bone.
But this one here, for example, is for regenerating bone.
And they've printed it in this exact geometry, because that's going to replace some anatomical piece.
And they want it in that geometry.
So I'll talk more about that.
But so for the scaffolds, they sometimes do that.
But not so much for these bone coatings.
Let's see.
Se we did that.
We did that.
So these are the data.
And then over here, there's the compressive strength.
And again, you know, this is our line with a slope of 3/2.
Again, there's a lot of scatter, because there's a lot of variation in the structure of these things.
But it's in the ballpark.
So are we good with metal foams?
And there's just like a page and a half of little notes.
Should I just put that on the website?
You're good?
OK. OK, so the last topic I wanted to talk about on bone has to do with how people look at the structure of bone in evolution and in evolutionary studies.
So the idea here is, in particular in looking at sort of pre-human species, sort of hominid species, people are interested in when primates went from being quadruplets to bipeds.
So obviously, bipedalism, walking on two legs, is characteristic of us.
And people would like to know if they find some fossil-- you can't just tell from the fossil directly is it a biped or a quadraped.
You can't see the species moving because it doesn't exist anymore.
So you'd like to have some way of making some estimate of whether or not it was a biped or a quadruped.
Let me get my notes together here.
So I wanted to kind of look at the big picture a little bit and look at the evolution of different species.
And this is a phylogenetic sort of chart.
And this is kind of-- have you heard of the tree of life?
This is like the tree of life.
So this piece of it is-- metazoa is for animals.
So not plants, animals.
And this goes back about 1.2 billion years.
So these are millions of years ago.
And then these are different sort of eras and ages that are defined.
And when we have a branch here, this is a common ancestor.
And then this is a branch in one direction, and that's a branch in another direction.
So these points here, like 1 and 2 and so on, the implication there is that there was a common ancestor to everything that traces back to there.
So this point here, 1, is 1.2 billion years ago.
So this was sort of very early kind of species.
And the very first things were, well, multicellular things.
I mean, there were little amoeba type things.
But the more sort of sophisticated animals were sponges.
And there's three kind of classes of the sponges.
There's calcarea.
And they're called calcarea because they're mineralized.
And they're mineralized with a calcium carbonate.
And then there's another branch of them called-- I don't know if I'm going to say this right, but hexactinellida.
And those have glass.
Those are called glass sponges because they have SiO2 is the mineral in those.
And then there's these guys here the demospongiae.
I think some of those have calcium carbonate and some of them don't.
Oh, no, some of them have silica.
And some of them don't.
So these things here are sort of very early multicellular structures.
And the mineral in them is either calcium carbonate or silica.
And then if we move up, I've sort of highlighted a few of them.
The cnideria-- I know there's a C, but that's actually-- when I say s-nideria, my biologist friends laugh at me.
And they say no, no, no, it's nideria.
The cnideria are the corals and the jellyfish.
And corals are also mineralized with calcium carbonate.
And you can see they branched off something like a billion years ago.
Then we get up here.
These are the mollusks.
So the mollusks are things like bivalves, like if you like to eat claims, things like that.
So bivalves, snails, and things like octopi, octopus.
So those are all molluscs.
And molluscs, when they're mineralized, also are calcium carbonate.
So those are the calcium carbonate kind of shell.
So we haven't got up to anything bony yet.
Bone is collagen plus a calcium phosphate.
So we haven't gotten to anything that's a calcium phosphate yet.
Then another large class is arthropoda.
That's insects and spiders and crustaceans.
Those all have a chiton skeleton.
So if you think of like the exoskeleton of an insect or a spider, those are chiton.
And crustaceans, things like lobsters, those also have a chiton shell.
And in crustaceans, it might be mineralized.
But again, the mineral is a calcium carbonate.
So all the way up here, most of these things, if there is any mineral, it's calcium carbonate.
And if we get up finally to the vertebraes, the vertebraes have the calcium phosphate and have a bone, like what we think of as real bone.
So the vertebrates obviously involve things like mammals, birds, snakes, and fish.
So this is kind of the big picture going back.
And sort of one of the interesting things is that bone doesn't come along until you get somewhere over here.
And I have one more little, nice slide here.
So when I talked about the sponges, they were these guys here with the glass sponges.
Joanna Aizenberg at Harvard did a nice study on glass sponges.
And she looked at this one here.
It's called the Venus flower basket is kind of the common name.
And it has this hierarchical structure.
And it's remarkably stiff and tough.
And what she did in her paper was look at optical and mechanical properties.
But they looked at the structure at different length scales.
So there's kind of a cellular structure at this length scale.
It's kind of a tube.
This was just a picture I took in a natural history museum.
But if you look at higher magnification, each little strut is made up of sort of fibers and has a hierarchical structure itself.
So that's just one of the sponges.
And here's a similar chart for the vertebrates.
So this point here is where the other chart kind of branched off.
And if we start with the earliest things again, the earliest ones with the most common ancestor back here is something called cyclostomata.
And those are things like jawless fish, so things like lamprey and hagfish.
Do you know what a hagfish is?
It's this kind of eely thing.
And I have the video for you if you don't know what a hagfish is.
So let me get out of here because it's just an amazing thing, the hagfish.
OK, so let's see, I got my sound on.
[VIDEO PLAYBACK]
Here, at the University of Guelph, about an hour outside Toronto, materials scientist Atsuko Negishi and biologist Julia Herr think that these lovely creatures, called hagfish, may revolutionize how we make strong materials
These are specific hagfish.
They're well known for their unique defense mechanism
So if I wanted to see this, what would we do?
Like could we poke at it with a stick?
I think the best way to do it is to reach in there and grab one
Oh, my gosh, look at that disgusting-- oh, no, I've been slimed.
I feel like an outtake from Ghostbusters.
Look at the quantities of this stuff.
[END PLAYBACK]
He used to do this for The New York Times.
And I think he's got his own going on, but he used to make these little videos.
And he had a show on PBS a year or two ago all about materials.
And there were like four different episodes.
And he talked about different kinds of materials.
And he went to different labs.
But he's quite a character.
But anyway the hagfish have this defense mechanism where they make the slime.
And I don't know if you know Professor McKinley over in mechanical engineering here at MIT, but he collaborates with those people of Guelph.
And he studies what he calls non-Newtonian fluids.
Well, a lot of people call non-Newtonian fluids.
A Newtonian fluid is a thing like water, where the viscosity is a constant no matter what sort of strain rate you shear it at.
And a non-Newtonian fluid does not have that property.
The viscosity changes.
And some of them have a strength, like in the hagfish one has a strength.
So Gareth's studies things like this kind of hagfish slime.
I don't know if he still has them.
He used to have hagfish in his lab over him building, whatever it is, 1 or 3 or something over there.
OK, so that's what the hagfish are, just because that's amusing.
And they don't have bone.
So they and the jawless fish don't have bone, even though they're called vertebrates.
Then the next sort of most recent thing is the chondrocytes.
Those are the cartilaginous fish, so things like shark.
So sharks don't actually have bones.
They have cartilage.
And they have a little bit of mineralization in the cartilage, but they don't actually have bone.
And the first thing that actually has a bone is the ray finned fish, which are these guys here.
And that occurred about 450 million years ago.
And then everything in the vertebrate since then is bony.
So there's coelacanth-- I don't know if you've ever see those.
Every now and then they find one of these things in Florida or something-- lung fish.
There's these guys here, which are the frogs and the toads and the salamanders.
So it's finally getting to be spring after the winter from hell.
And the salamanders are going to come out into the vernal pools and mate.
And it's cute.
So anyway, they come out this time of year.
And then there's the amniota, things that have eggs of one sort or another.
So that includes us, the mammals, the birds, the snakes, and the turtles.
And so those would have branched off about there.
So the last thing I wanted to point out here is that there's this huge kind of diversity of animals that have evolved over millions and millions of years.
And that the first ones that were mineralized used the calcium carbonate and that the bony type materials didn't really evolve or didn't appear until about 450 million years ago.
And then these are the vertebrates that have these kind of bony things.
So as I've said many times now, the bone grows in response to loads.
And the bone structure reflects the mechanical loads and the function.
And evolutionary studies have looked at both cortical bone and trabecular bone architecture to try to say something about the locomotion of the animal or of the species.
So there's ones that look at cortical bone.
But I'm just going to talk about one that deals with trabecular bone.
So this study was done by a group of peoples, the first author was Rook.
And what they studied was the ileum.
So this is a pelvis.
And there's the ileum is one of the bones in the pelvis.
And they found fossils of a hominid species that was about 8 million years old, called Oreopithecus bambolii.
And bambolii refers to the place in Italy where these fossils were found.
And so they found two-- or at least somebody found two pieces of an ilium.
And they took x-rays.
And they made a digital reconstruction so that they would get one ilium-- it turned out that the two pieces were two different parts-- so they could make a whole one out of the two pieces.
And then they looked at the structure of the trabecular bone.
And they compared that structure to the structure of the trabecular bone in humans and other primates.
And they wanted to see is it more like the humans, which are obviously biped, or is it more like some of the primates, which are quadrupeds.
So this just shows for a human and a non-human primate what the structure of the ilium is.
And these little black boxes with the letters are what are called anatomical landmarks.
So they're sort of comparable spots on the bone of different species.
And what they were doing was looking at the trabecular architecture at these different spots.
So you can kind of see how they're more or less analogous in the two different species.
And this is the digitally reconstructed ilium that they put together from their fossil species.
And again, these little letters refer to these anatomical landmarks.
And then what they did was they compared the Oreopithicus, the fossil, with the human and then three non-human primates.
So these four images are all from the fossil.
These are the human.
These are champi, siamang and baboon.
And this square here corresponds to that one there.
This is B, corresponds to that one.
And C and D are those two there.
So they had two fossils, they made the digital reconstruction.
They looked at certain areas.
And then they looked at the same or analogous areas in these other species.
And they tried to look for similarities and differences in the bone structure.
So let's look at this first box A.
You can see there's a very white bit here.
And the white corresponds to more dense.
So there's a white bit that's more dense in their fossil.
And in the human bone, you see is a similar sort of band right there.
And if you look at the other non-human primates, that band is missing.
So that says to them, OK, this feature, this one feature at least, is more similar to the-- sorry, in the fossil here is more similar to the human than it is to the non-human primates.
And then they had three other landmarks that they looked at like that.
So this one here again is a sort of dense regions.
So you can see that white dense region.
There's some there.
There's some here.
So those are the fossil and the human.
But there's a teeny bit here and a teeny bit there.
But it's not as pronounced in the non-human primates.
Yes
From I get at least so far, the portions of the bone that are dense versus not dense seem less relevant to the direction of loading than the orientation of the foamy parts of the trabecular bone
So the density reflects more or less the magnitude of the stresses.
So if the stresses are higher, it's going to be denser.
And the orientation of the bone, like whether or not which way the struts are oriented, that reflects the sort of ratio of the principal stresses.
So if the principal stresses go in a certain direction, the bone's going to tend to line up in that direction.
That's what that Guinea fowl study was kind of about.
OK?
Are we good?
OK. And then these other two, so in B-- and you can't really see it from here, but they looked at the sort of architecture of the trabecular bone.
And they said that it was more similar in the fossil in the human than it was in these other three primates.
And this region here, this looks very similar to that bit there to me.
But I think there was some other feature about this region that they were looking at.
And again, it was more similar to the human than it was to the non-human primates.
So by just looking at the pattern of the bone and the density of the bone and comparing it to these other species, they said that the-- and you know the hip, because we're standing like this, you would kind of expect to see differences in the hip.
That's why they wanted to look at the ilium.
So the conclusion they made was that these observations suggested that the species was bipedal, or at least spent some of its time as a bipedal animal.
And I think that might be it.
Do I have more I have one little summary here.
So just to summarize what we've done on bone this week.
We talked about the structure of the bone.
We talked about mechanical properties in the foam models.
We talked about osteoporosis in the voronoi models, how you can try to represent the loss of bone and the loss of bone strength using those models.
We talked just a little bit about the idea of using metal foams as bone substitutes or coatings as implants, and then this little bit on trabecular architecture and evolutionary studies.
I have some notes, but I think we've got just a couple minutes left.
So maybe I'll just scan those and put them on the website?
Are we good with that?
I have a question about what we just looked at about the different species.
We always consider on the density changes.
Can there always be changes in the solids?
So it changes a little bit from one species to another.
So the question is, does the solid properties of the bone, the solid bone itself change?
So if you look at cortical bone in different species, it changes a little bit, but like 10%, not a huge amount.
So the two most common things people have compared are bovine bone and human bone.
And there is a slight difference between them.
But it's not a huge difference.
One of the other things people have looked at in osteoporosis that I didn't really talk about was there's another whole set of things that can go on that reflect what you're talking about.
So they talk about bone quality as well.
And when they talk about bone quality, they're talking about are there micro cracks in the solid.
So you might imagine as you get older, it's not just that you lose the struts or that the struts get thinner, but the solid bit itself has more cracks in it.
So you can imagine if the solid bone itself had little micro cracks, then it too would be weaker.
And then you think what I put up was bad.
It gets even worse if you put that in as well.
So, yes, people do look at bone quality, which is sort of looking at with age.
And typically it's fairly elderly people that the bone quality is an issue.
I guess there are certain diseases where it's an issue.
But in osteoporosis, it's sort of elderly people.
Any thing else?
Should I stop there for today?
So there were hardly any equations in this.
Did you know that?
So we got to the part where there's lots of equations.
So next week I'm going to talk about tissue engineering.
I think I'm going to talk a little bit about different kinds of scaffolds, how they make scaffolds, how the scaffolds fit sort of into an anatomical things, what is that supposed to represent, how they use them clinically a little bit.
And we had a research project on osteochondrol scaffold, so scaffolds for replacing bone and cartilage.
And I'm going to talk a little bit about that as sort of a case study in tissue engineering scaffolds.
And I have some stuff on cell mechanics and how biological cells interact with scaffold.
I don't if we're going to get to that next week or not, but somewhere close to that.
So the next bit is on tissue engineering scaffolds, osteochondrol scaffolds, cell mechanics.
And there's not that many equations in that part either.
So OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time, we finished talking about trabecular bone.
And what I wanted to talk about this week was tissue engineering scaffolds.
So the idea with tissue engineering is you want to be able to repair damaged or diseased tissue.
And typically, that's done by regenerating the tissue in some way.
So in your body, many types of cells, like maybe not blood cells, but most types of cells for sort of structural tissues are attached to an extracellular matrix.
And this is sort of a schematic of the extracellular matrix here.
And the composition depends on the type of tissue that's involved.
For example, in skin, it would be collagen and something called glycosaminoglycans and elastin.
In bone, it would be collagen and a mineral-- a calcium phosphate mineral, hydroxyapatite.
So the composition varies.
But the idea with the tissue engineering scaffold is that you want to make a material that essentially substitutes for the extracellular matrix.
And it does so on a sort of temporary basis.
So the idea is you put something in the body.
The cells attach to that, whatever scaffold you put in.
And the scaffold has to be made in such a way that the material-- that the cells, they can migrate through it.
They can attach to it.
They can differentiate.
They can proliferate.
So the cells can do all their normal function.
And the idea is that as the cells are doing their normal function, they then secrete the natural extracellular matrix.
And the engineered thing that you put in is resorbed.
So there has to be a balance between the rate at which the scaffold you've made resorbs and the rate at which the cells are depositing the new native extracellular matrix.
So that's one of the key things about this.
So this is just an example of sort of a schematic of an extracellular matrix.
In this case here, there's collagen fibers.
So these guys here collagen fibers.
And these kind of hairy-looking things are proteoglycans.
So they have a core of protein with sugars kind of hanging off of them.
And there's different kinds of GAGs, they're called, that hang off of them.
So one's chondroitin sulfate.
The CS here stands for Chondroitin Sulfate.
There's one called dermatan sulfate.
And there's one called heparin sulfate.
So there's different of these glycosaminoglycans.
So let me write down some of this, and then we'll kind of get into this.
So what I wanted to do today was show you some examples of tissue engineering scaffolds.
Show you some of the sort of design requirements.
So you have to have, obviously, a material that's porous so the cells can get in there.
And that's where the cellular solids comes in.
So we'll talk about some scaffolds, some of the sort of design requirements for them.
And also, we'll talk a little bit about processing of the scaffolds and mechanical properties of the scaffolds.
So I'm hoping I can finish most of that today.
And then next time, I have a little case study on osteochondral scaffolds.
So osteo means bone.
And the chondral means cartilage.
So this was sort of a two-layer scaffold that we developed in collaboration with some other people at MIT and some people at Cambridge University.
And it went from a research thing to a startup thing and being used clinically.
So I was going to talk about that Wednesday.
OK.
So let me just get started here.
So the goal of tissue engineering is to regenerate diseased or damaged tissues.
And in the body, the cells attach to the extracellular matrix.
And that's sometimes called the ECM.
Sometimes, the scaffolds are called scaffolds.
And sometimes, they're called matrix because the extracellular matrix is called matrix.
So then, the composition of the ECM depends on the tissue, but it usually involves some sort of structural protein.
So something like collagen or elastin.
It also typically involves some sort adhesive proteins.
So something like fibronectin or laminin.
And it involves these proteoglycans, which are the core of protein with a sugar hanging off them.
Whoops.
And the sugars are typically glycosaminoglycans.
And for short, people call them GAGs, just because it's easier to say.
So some examples of the GAGs are chondroitin sulfate.
And we're going to talk more about that a little later because that's one that we've used in making collagen-based scaffolds with [INAUDIBLE].
So for example, if you look at the composition of extracellular matrix in something like cartilage, it has a collagen component and a hyaluronic acid component and some GAGs.
And this hyaluronic acid is a proteoglycan.
If you look at bone, extracellular matrix in bone, it's made up mostly of collagen and hydroxyapatite.
And if you look at skin, skin is made up of collagen, elastin, and proteoglycans.
And the idea is that the cells have to be attached to this extracellular matrix in order to function.
Or, they have to be attached to this or to other cells in most cases.
So next week, I'm going to talk a bit about cell mechanics.
And we have some video where we watch a cell deforming a scaffold.
And I don't have videos to show you, but I had a student who took videos of cells migrating along with scaffolds.
We'll look at how the stiffness of the scaffold affects how the cells migrate along the scaffold.
So that's kind of the native ECM in the body.
And the idea with tissue engineering is that you want to make a scaffold that's porous that mimics the extracellular matrix in the body.
So, say, there's some tissue that's damaged.
Say there's damaged cartilage.
You want to provide sort of an extracellular matrix that's a sort of synthetic thing that's going to provide the same function as the ECM in the native tissue.
So people have been working on scaffolds for regenerating all sorts of different tissues.
And probably, the most successful one has been used to regenerate skin.
And there's been scaffolds available for regenerating skin for probably almost 20 years now.
And one of the first ones was developed by Professor Yannas in mechanical engineering here at MIT.
And it's actually still sold by a company called Integra.
But this research to develop scaffolds for lots of different tissues, orthopedic tissues, things like bone and cartilage, cardiovascular tissues, nerve-- like Professor Yannas works on peripheral nerve these days.
People have looked at trying to make scaffolds for gastrointestinal tissues.
So all sorts of different tissues.
And at MIT, there's quite a lot of interest in this.
There's a lot of people working on it.
So Bob Langer works on it.
Linda Griffith.
Sangeeta Bhatia.
There's really quite a number of people at MIT who work on it.
Those are just some of the people at MIT.
So the idea is in the body, the cells are constantly resorbing the extracellular matrix and depositing more.
So if you think about bone, for example.
Remember we said bone grows in response to load?
Even in healthy bone, the bone is constantly being resorbed and deposited.
And in normal bone, the rate of resorption and the rate of deposition is roughly the same.
When people get osteoporosis, the thing that happens is that that balance gets out of whack.
And so it's not being deposited at the same rate it's being resorbed.
And the idea with the tissue engineering scaffolds is that they degrade over time.
And that the cells that were attached to them are forming their own extracellular matrix.
So there's kind of a balancing act between the cells depositing the native ECM and the tissue engineering scaffold that was provided, say, by the clinician being resorbed.
So the scaffolds are actually designed to degrade.
And controlling that degradation rate is one of the design parameters of the scaffolds.
OK.
So that's kind of the overall, kind of big picture.
And what I wanted to talk about next was some design requirements for the scaffolds.
So if you think about it, there's different sort of ways you can think about what the requirements are.
So you have to make the scaffold out of some solid.
And there's some requirements for the solid.
So obviously, you want a solid that's biocompatible.
That's one kind of main requirement.
Another requirement is that not only the solid has to be biocompatible, but when the solid decomposes, if it decomposes into other components, they have to be biocompatible too.
So you don't want the solid to degrade during this resorption process into toxic components.
That would be a bad idea.
And then, the other thing is the solid itself has to promote cell attachment, and cell proliferation, and cell migration, all these kinds of things.
So we're going to talk about some different materials for the scaffolds.
And some of them are sort of native proteins.
So some of them are things like collagen.
Collagen already has binding sites for cells to attach to it.
Obviously, it's one of the proteins in the native ECM.
There's also a number of synthetic polymers you can use.
And with the synthetic polymers, they don't have natural binding sites for the cells.
And so you have to coat them with something else.
So you have to coat them with, say, adhesive proteins so that the cells will attach to them.
So we're going to talk about the requirements for the solid.
Then, you make the solid into some sort of porous, foamy thing.
I think I have some slides here.
Here's an example of a collagen GAG scaffold.
And this is one of the ones that is made in Yannas' lab.
And you can see it looks a lot like a foam.
It's very, very porous.
And there's some requirements for the sort of cellular structure, the foamy structure of the scaffold as well.
So typically, you want interconnected pores so it's easy for the cells to migrate in.
Typically, you want pores to be within a certain range.
It turns out if the pores are too small, it makes it difficult for the cells to get in.
Sometimes, they can by eating away at the material.
But typically, the pores-- you want them to be bigger than a certain size.
You also want them to be smaller than a certain size because how much specific surface area, how much surface area per unit volume you've got, depends on the pore size.
The smaller the pores, the more surface area per unit volume you have.
And then, the number of binding sites you have for cells to attach to it depends on that specific surface area.
I'm going to write all this down, so I'll do that.
So there's requirements for sort of the pore structure.
And then, there's also some requirements for the whole scaffold itself.
So for instance, it's got to have some minimal mechanical integrity.
So there's sort of some requirements for that.
So there's requirements for the solid.
There's requirements for the sort of cellular structure, the porous structure.
And there's requirements for the overall scaffold.
So let me write some of these things down.
So there's some requirements for the solid.
So it must be biocompatible.
It must also promote cell attachment and proliferation in the cell functions.
And then it must degrade into nontoxic components.
So there's some requirements for the cellular structure, too.
And what you want to have is a large volume fraction of interconnected pores.
And so you want that to facilitate the cell migration.
And also, the transport of nutrients into the cells.
So you also want the pore size to be within a critical range.
So you need the pores bigger than a lower limit so the cells can migrate through easily, can kind of get in there.
And you want the pore size to be less than an upper limit to have enough surface area to have enough binding sites to actually attach cells.
And for different tissues, there's different critical ranges of the pore size.
So for example, for skin they found that you want to have a pore size between about 20 microns and about 150 microns.
And for bone, the pore sizes that people tend to use are between about 100 and 500 microns.
So there's the pore size.
And one other feature is that the pore geometry should be conducive for the cell type.
So lots of cells are somewhat equiaxed.
Maybe a little elongated.
But if you look at something like nerve cells, like peripheral nerve, they're incredibly elongated.
So you want to have pores in the scaffold that are also very elongated.
And then for the overall scaffold, it needs to have some mechanical integrity.
You guys OK?
Yeah?
We're good.
Oh, achy.
Yeah.
I know.
So you want to have some overall mechanical integrity.
I mean, the thing has to be put into the body in surgery.
And people are going to be pushing and poking at it.
And so it has to have some just overall mechanical integrity.
Also, it turns out that if you put stem cells into scaffolds, the types of cells they differentiate into depends in part on the stiffness of the scaffold.
So you want to be able to control the stiffness of the scaffold
How do they do this research?
Do they use animals?
Or they do it in vitro?
Yeah.
So the question is, how do they do the research?
So they do a sort of series of different things.
So at one level, you could have the scaffold and you'd put cells on it.
Say you're making a bone scaffold.
You'd put osteocytes onto it.
So one level, you just put cells onto it.
And you want to see, are the cells attaching?
Are they dying?
Are they proliferating?
So sometimes, what people will do is put the-- they'll seed a certain number of cells at a certain time.
Say, time 0.
Then, they'll look at how many cells are attached at 24 hours or 48 hours.
And you kind of see the cell attachment.
You can measure relatively easily.
Another thing people do is animal studies.
So for instance, Yannas does research on peripheral nerves and scaffolds for peripheral nerves.
And they cut a piece out of the sciatic nerve of rats.
So obviously, they have a surgeon and do a surgery thing with it.
You can't just kind of do this in the lab.
You've got to get permissions and stuff to do it.
And then, they put in the scaffold.
And the scaffold's actually in a tube.
And so they put the two stumps of the nerve end at either end of the tube, and then the tube's filled with a sort of porous scaffold that we're talking about here.
And then, they wait some period of time.
And they take video of rat running, things like that.
They then sacrifice the rat and they do histology.
And they look at the sort of cross-sections and see what it looks like.
And so I'm going to talk next time about this osteochondral scaffold we worked on.
So we did cell studies.
We did goat studies.
We put into goat knees.
There was a longer term sheep study.
And then, the student who's in Cambridge, England, started up a company and he ended up getting approval in Europe to start clinical trials.
And then he worked with an orthopedic surgeon who started putting it in people.
But typically, they're looking at cells, looking at animals before you get to the people stage.
And one of the things people do when they're making these scaffolds is you want to use materials that already have some sort of regulatory approval.
So say FDA approval or approval in Europe.
So typically, people don't start with a brand new material from scratch.
Because to get approval for that would just take a very long time.
So typically, people start with-- the solid material is already approved for some other sort of use.
OK.
So one requirement for the overall scaffold is it has to have sufficient mechanical integrity.
Sufficient.
And then also, as I mentioned, the stiffness of the scaffold can affect differentiation of cells.
And the other thing that is really a factor for the overall scaffold is you want to control the rate of degradation of the overall scaffold.
So you want that rate to be matched to the rate at which the new tissue is forming.
So it has to degrade at a controllable rate.
OK.
So I want to talk about the materials that people use.
And you can kind of break them down into a few classes.
So one class is natural polymers.
So things like collage.
So you can get collagen.
And that's an example up there of a scaffold that's made with collagen.
Another class of materials is synthetic biomaterials.
And if you've had stitches or surgery, you may know that some of the sutures they use are resorbable.
So some of those polymers that they use for resorbable sutures are also used for tissue engineering scaffold.
And then, there's also hydrogels that people use as well.
So those are probably the three main groups are sort of natural polymers, synthetic biopolymers, and I guess the hydrogels are sort of a subset of the sympathetic biopolymers.
So collagen is probably the most common kind of natural polymer that's used.
They also use GAGs.
And this scaffold up here is made by making a coprecipitative collagen with a GAG chondroitin sulfate.
People also use alginate.
I think one of the project groups-- you guys are going to make some sort of alginate scaffold, right?
No?
[INAUDIBLE] foamy thing?
Yeah.
And people also use something called chitosan, which is a derivative of chiton, which is what's in the exoskeleton of insects and things like lobsters.
So those would be all examples of natural polymers that can be used and people have tried.
I'm going to talk a bit more about collagen, just because it's the most common one.
So collagen is a major component in the natural extracellular matrix.
And not surprisingly, it has binding sites for cells to attach to it.
So if you use that, that kind of takes care of that issue.
Let me put it down here.
So collagen exists in many types of tissues.
Exists in skin.
Exists in bone, cartilage, ligament, tendon-- cartilage.
So it's very common.
It has surface binding sites for cells.
It has a relatively low Young's modulus.
So the Young's modulus is a little less than a gigapascal.
But you can increase the modulus by either cross-linking or by using it in conjunction with some synthetic polymers.
And I'm going to talk a little bit about how you make these scaffolds up here.
And the first step in making those scaffolds is you put the collagen in acetic acid, and then you add the glycosaminoglycan and it forms a coprecipitate.
And the fact that it forms a coprecipitate with the glycosaminoglycan, the GAG, means that you can use a freeze-drying process.
And that's how that's made.
Collagen is one option.
So then synthetic biopolymers is another option.
And as I said, typically they use the materials that are used for resorbable sutures.
So there's several of those.
There's something called PGA.
That's polyglycolic acid.
And something called PLA.
That's polylactic acid.
And then you can combine those two and make something called PLGA polylactic co-glycolic acid.
And you can control the degradation rate of these things by controlling the molecular weight.
And in this case, you can also control it by controlling how much of each of those things you put in.
And there's another one called polycaprolactone.
So those are several synthetic biopolymers that people use.
There's lots of different materials, but these are just some typical ones.
And then another class are hydrogels, which are produced by cross-linking water soluble polymers to form an insoluble network.
And those are typically used for soft tissues.
Sometimes, they're used for things like cartilage.
And again, there's a few different materials that are commonly used.
One's PEG, Polyethylene Glycol.
One's PVA, Polyvinyl Alcohol.
And another one's PAA, Polyacrylic Acid.
So for these synthetic polymers, there's many different processing techniques available.
And I'll talk a little bit about some of the processing techniques.
But one of the limitations is they don't have natural binding sites.
And you have to coat them with some sort of binding agent, like an adhesive protein, to get the cells to attach to them.
And then, as I mentioned before, you have to make sure that whatever material you choose, if it's a synthetic material that when it degrades, it's not toxic to the cells.
Because you don't want to have some sort of toxic reaction or inflammation.
OK.
So there's a couple more things about materials.
So those are all polymer-based materials.
When people are trying to make scaffolds for bone tissue engineering, they also include a calcium phosphate mineral.
And there's different versions of the calcium phosphate.
So they can include-- you can buy, for instance, hydroxy powders now.
There's another calcium phosphate called octacalcium phosphate, which will, with water, turn into hydroxyapatite.
So typically, there is this mineral, some sort of calcium phosphate, is combined with either collagen or with one of these synthetic biopolymers.
And one other option is something called an acellular scaffold.
And what that is they take some natural tissue and they remove all the cell material from it.
And so when they remove all the cell material, what they're left with is the native ECM.
And that's called an acellular scaffold.
So it's a native ECM with all the cell matter removed.
And they remove the cells by-- they can use sort of a physical agitation or chemical, or using enzymatic methods.
Using something like trypsin to get rid of the cell.
So there's ways that they can get rid of the cells.
OK.
So are we good so far?
So there are some requirements for what materials we kind of use, what the cell structure should be, and these are some examples of typical materials.
So I wanted to talk a little bit about the processing of the materials.
Let me wait until people catch up a little.
Oh, and I have some scaffolds I was going to pass around.
So this big sheet is a piece of the collagen GAG scaffold that I showed a minute ago.
And then this little piece is a mineralized version of that.
So this has the collagen plus calcium phosphate plus hydroxyapatite in it.
OK.
So this slide shows some examples of different scaffolds that people have made.
And I was going to talk a little bit about some of these methods.
And why don't I talk about them, and then I'll write some notes on the board.
So this top one here on the top left, that's the collagen GAG scaffold that's made in Yannas' group.
And that's made by a freeze-drying process.
So you put the collagen in acetic acid, then you put in the GAG.
The GAG and the collagen form a coprecipitate.
And then you can freeze that.
And if you freeze it, what happens is-- it's just like if you freeze saltwater.
The water freezes.
The pure water freezes.
And you've got increasingly higher brine content in the bit in between the water grains, or in between the ice.
So you get the sort of solid ice forming.
And the collagen and the GAG are kind of squeezed into the interstitial bits between the ice crystals.
And then if you sublimate the ice off, you're left with this porous kind of structure that looks like a foam.
So that's made by a freeze-drying process.
And I'll go over it in a bit more detail when I write the notes on the board.
You could also foam some of these polymers.
So just blowing a gas.
The same way you can blow a gas through engineering foam.
You do the same thing with some of these polymers.
This one's made by foaming.
You can have a fugitive phase process.
So this is made by salt leaching, the second row on the left there.
So you could imagine you could take a polymer powder.
You could mix it with salt.
You can sort of mix them up, combine them together.
You heat it up to get the polymer to melt and to sort of form a connected mass.
And then you leech out the salt.
And then you get pores where the salt was.
This one here is made by an electrospinning process.
So you have a nozzle.
You feed the polymer through the nozzle.
Then you have plates that are charged and you get fibers forming and kind of scattered in different directions by the electrospinning process.
Then, this one here represents scaffolds that are made by things like 3D printing, selective laser centering.
You can have laser-sensitive polymer.
And you can produce scaffolds that way.
I think the geometry of this one matches some part in the body.
I think it was a knee or something like that.
And then, these two examples down here are the acellular scaffolds.
That's what I was talking about at the very end there.
So those are from porcine pork heart tissue.
You know, pig heart tissue.
And those are mostly elastin.
And they've had all the cell matter removed from them.
OK.
So you can kind of see that these synthetic scaffolds here have a structure that's not so different from these native ECM scaffolds down here.
OK.
So let me write some of the things about the processing on the board.
Let me rub this off.
Start over here.
So these freeze-dried scaffolds are used for skin regeneration.
And I think I have some more slides here.
So it's kind of a two-step process.
In the first step, you make what they call a slurry.
So you make the slurry by taking the collagen.
And for skin, I think you want type 1 collagen.
Yeah, skin is type 1 collagen.
There's different types.
You put it in acetic acid, and then you add the GAG.
And we use chondroitin 6-sulfate is just the particular GAG that we use.
And one of the things that the acid does is that it swells the collagen.
And collagen has a sort of periodic structure in it, sort of periodic banding.
And the acid destroys that periodic banding structure.
And that helps increase the resistance to having some host immune response.
So that you remove the immunological markers and it makes it less likely that the scaffold's going to get rejected by the body.
So then when you put the GAG in, you form a coprecipitate.
So this next step just shows kind of mixing the whole thing up.
And then you've got kind of a little slurry that you can store.
And then, this is the freeze-drying step here.
So you put the slurry, the suspension, into a pan.
Kind of just like a cookie sheet, really.
And then you freeze it.
So if you think of this phase diagram here, where you have temperature and pressure.
So here we have liquid, solid, and vapor.
So if you start off at this point here, you freeze it.
So you've reduced the temperature.
So that forms the ice.
And the ice is surrounded by the collagen and the GAG fibers.
And then if you do the sublimation step, you reduce the pressure and increase the temperature a bit.
And then you get over to the vapor end of the world.
And then you're left with this porous scaffold.
And then, let me see.
Let me do one more step here.
And then you can control the size of the pores by controlling the freezing temperature.
So the size of the pores is exactly related to the size of the ice grains that are forming.
And the faster it freezes, the smaller the grains are going to be.
And then, the smaller your pore size are going to be.
So you can control that.
The type 1 collagen is mixed with acetic acid.
And it then swells the collagen and disrupts periodic banding.
And it removes immunological markers.
And then you add the GAG, the chondroitin 6-sulfate.
And then that cross links with the collagen and forms a coprecipitate.
And then you can freeze dry that to get the porous scaffold.
And typically, the relative densities of these scaffolds is very low.
So typically, they're 0.5% dense.
The relative density is 0.005.
So they're 99.5% air and the rest of it's the collagen and the GAG.
And the pore sizes are typically between about 100 and 150 microns.
And Yannas uses the same scaffolds for the nerve regeneration.
And he uses a directional cooling.
And that then elongates the pores, so that-- the idea is that they elongate so the nerves kind of grow along that length.
So that's one way.
Another way is leaching a fugitive phase.
So let's see.
I think-- yep.
Here we go.
Back to there.
So if you look at the one on the second row on the left, that's done by using salt as the fugitive phase.
People use other things.
You can use wax.
Paraffin wax works as well.
So it doesn't have to be salt.
There's different things you can use.
So you combine a powder of the polymer with your fugitive phase Say, salt.
Then, you heat it up to get the polymer to bind.
And then you leach out the salt.
So you can control the porosity by the volume fraction of the fugitive phase, and then the pore size by the size of whatever the fugitive phase is.
Another technique is electrospinning.
The idea is you produce fibers from a polymer solution that you extrude through a nozzle.
And then you apply a voltage across some plates to spin the fibers.
And then you get a network of these fibers.
And typically, their micron-scale diameter.
And the last method I'm going to talk about is rapid prototyping.
So you can think of using 3D printing.
Or you could use selective laser centering or stereo lithography using a photo-sensitive polymer.
So the idea is-- you know how this works.
You just build up layers of solid, one layer at a time.
And then you can make complex geometries with that.
So that's one of the advantages.
If you wanted to make a part to fit a particular place in the tissue, then it's convenient that you can control the geometry of the whole part.
OK.
So that just kind of summarizes very briefly, kind of how the tissue engineering scaffolds are meant to work, what kinds of materials people make them from, and a few of the processes.
And there's many, many processes.
These are just sort of a few common processes.
I wanted to talk a little bit about the mechanical behavior because that's kind of what I do.
And so this next plot just shows a stress strain curve in compression for a collagen GAG scaffold.
And I'm hoping that by now you're getting the idea all of these cellular materials have this kind of shape of a curve.
So there's the same kind of linear elastic regime, and then a collapse plateau.
These collagen things, as you can imagine just pressing them in your hand, they fail by buckling, by inelastic buckling.
And then there's a densification regime.
So they look like all the other kinds of curves that we've got.
One of the things we've done in the modeling is typically in the model, we want to be able to calculate or measure the modulus of the solid or the strength of the solid from which the thing's made.
So [?
Brendan ?]
Harley was one of my PhD students.
And he took a little microscope, cut a little strut.
The struts are very small.
The pore size is 100 microns.
So the struts are on that order.
He glued one end of the strut to a glass slide, and then he used an AFM probe to do a little bending test on that little strut.
If he could measure the deflection, he knew the length.
He knew the geometry of the strut.
He could figure out what the modulus was for the solid.
So he backs out what the modulus is.
So he did these measurements on a dry strut.
And then by comparing the overall modulus of a dry scaffold with a wet scaffold, he estimated what the modulus of the wet scaffold or the wet strut would be.
So the modulus of the dry collagen GAG was 672 megapascals.
A little less than a gigapascal.
And wet it was about 5.
So there's a huge difference between the wet and the dry.
OK.
So let me just write a few notes about that.
So in compression, there's the three regimes that we see for all these cellular materials.
And so you can estimate the modulus by using the model we have for the foam for the modulus.
The modulus of the foam divided by the modulus of the solid goes as the relative density squared.
And that's related to bending in the cell wall.
And then, the collapse plateau is related to elastic buckling.
And so that's equal to some constant times E of the solid times the relative density squared.
And that's related to elastic buckling.
And then we measured E of the solid doing this little AFM beam bending test.
OK. And for one of these low-density scaffolds, we measured the modulus.
We measured the buckling strength.
And we got pretty good agreement by using these equations here.
And the good agreement was if this constant was 0.2.
That was the strain, that buckling.
Yeah
So what does it mean to be wet in here?
And why is it so much lower?
Well, we make the scaffolds by this freeze-drying thing.
And like this thing I passed around, that was dry.
We just immerse it in water.
We just put it in water.
And then, he does the test.
So he does the test on the whole scaffold dry, and then he does the test on the whole scaffold wet.
And I assume there's some sort of bonding that gets disrupted by having the water.
I don't know the details of how that works, but there must be some change in the bonding to make that happen.
So let's see.
I'm trying to see.
What else should we do today?
Oh, yeah.
So we get pretty good agreement with this sort of simple foamy model.
One of the things we did find was that when we tested higher-density scaffolds, the agreement wasn't so good.
And I think this was because when you get higher density, it's just hard to get the collagen GAG mixture to mix in with the acetic acid.
And so we ended up getting inhomogeneous scaffolds with sort of large voids in them.
So in order for the modeling to work, you have to have a scaffold that's relatively homogeneous.
You don't have kind of big defects in it.
I think that's all I'm going to say about that.
I have one more slide here.
So those are sort of three-dimensional scaffolds we've been talking about.
Sort of foam-like scaffolds.
People have also made honeycomb-like scaffolds as well.
And this just shows some examples of some honeycomb-type scaffolds.
So this one here, I think was made in Sangeeta Bhatia's lab.
You can sort of think of it as a hexagon, but it's also kind of triangulated as well.
These two here were made by George Engelmayr.
He worked with Bob Langer at one point.
And these two scaffolds here, they're both sort of rectangular cells.
And they were designed to look at how the cell geometry or the pore geometry affected the sort of morphology of the cells that attached.
So if you have different, say, [?
porous, ?]
you get different morphology in the cells that you're trying to attach.
And then, George Engelmayr also made these scaffolds here.
And those were designed to be anisotropic and have different mechanical properties in different directions.
And I think what they had done was try to match the anisotropy in the mechanical properties to anisotropy in heart tissue, in cardiac tissue.
So these are some examples of honeycomb-type scaffolds.
So let me just write down a few things about those.
So I think these were more-- obviously, the scaffolds are used-- the ultimate goal is to use them clinically.
But sometimes, people make scaffolds just to study cell behavior.
And some of these, I think, were made just to study how the cells would behave on them.
So they're sort of idealized to do that.
So that triangulated.
It kind of looks like a hexagon, but there's also sort of triangles in there.
I think the thing they were looking at with that was transport of nutrients to cells.
And from a mechanical point of view, if it's triangulated you'd expect, say, the modulus of that to go linearly with how much solid there is there.
So the rectangular honeycomb and the diamond shaped pores, they were used to study the effect of pore geometry on the cell orientation.
They used fibroblasts.
And I'm going to call it the accordion-like honeycomb.
That's mechanically anisotropic.
And the mechanical anisotropy is matched to the cardiac tissue.
OK.
So I think I'm going to stop there for today.
That's sort of the end of this part.
And next time, I'm going to talk about the osteochondral scaffolds.
I don't think it really makes sense to start it for two or three minutes.
So I'll start that tomorrow.
Or Wednesday.
And we should be able to finish that on Wednesday.
So one of the things that these honeycomb-type scaffolds kind of suggests is that the scaffolds are used both to try to regenerate tissue in the body in clinical applications, but they're also used as sort of an environment for cells, in order to study cell behavior.
So next time, I'm going to talk about an osteochondral scaffold.
And the idea with that was to try to use it clinically.
But next week, I'm going to talk about cell mechanics a bit.
And when people study cell mechanics, or look at the mechanics of biological cells, not the cellular structures.
So when people look at trying to study how cells behave, they need some environment to put them on.
Typically, people started by just using flat 2D substrates.
But the flat 2D substrates are kind of easy to study, but they don't really represent the tissue in the body.
And so people are now using tissue engineering scaffold as an environment that they can control to study how cells behave.
So they study cell attachment, cell proliferation, cell migration, and cell differentiation all by using the scaffolds as kind of a controlled environment.
So we'll talk about that, probably not-- I don't know if we'll get to it on Wednesday.
But we might start it on Wednesday and finish it next week.
OK?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
What I thought I would do today is two things.
I wanted to give this talk about osteochondral scaffolds, and this is just slides.
I'm not going to write anything down.
So there aren't going to be any notes.
And I'll put the slides on the Stellar site, I don't know, this afternoon or tonight or something.
And this isn't going to take the whole time we have today, so the other thing I thought I'd do is I wanted to walk you through this little booklet that I handed out about how to write a paper.
So this is by Mike Ashby, who you know, I've written books with.
He was my PhD advisor and we wrote the books on cellular solids together.
He has written many books.
I looked him up on Amazon this morning, and even though I know he'd written many books, I was shocked how many books.
They had 58 listings for just books with him as a co-author and some are second editions and third editions.
He's written a lot of books.
He's written a lot of materials, papers.
And he's, in the materials community, he's seen as a very clear and lucid writer.
So he's written this thing for his students and I thought we could just walk through it and I can talk to you a little bit about writing.
Because I know you're not quite there with your projects, but later on in the term you're going to want to write up your project, and it's not too early to start thinking about writing.
And there's some really good advice and it's short and it's to the point.
It's really helpful, that little brochure.
So we won't read every single thing, but I wanted to kind of walk through some of the main points in it, and that pretty much should take the hour.
So last time we talked about tissue engineering scaffolds, and today, I wanted to do a case study, and this was a project we had here at MIT and in collaboration with some people in the materials department at Cambridge University.
And we made what we called an osteochondral scaffold.
So osteo means there was a part for regenerating bone and chondral means there was a part for regenerating cartilage.
And really the point of the scaffold was to try to repair small defects in cartilage.
And I'll explain why we did the bone thing as well.
So this is just a little outline.
I'm going to talk a little bit about what the structure of cartilage is.
We have a little schematic of that.
Then talk about how small defects in cartilage are currently treated.
So these are things that, say you're an athlete, you might tear some cartilage, that kind of thing, not like you have osteoarthritis and you need a new knee.
It's not going to do that.
So I'll talk a little bit about cartilage and the current treatments.
I'll talk about some of the things we thought about in making an osteochondral scaffold, like what parameters were important.
And we based it on that collagen-GAG scaffold that I talked about last time.
So the collagen glycoseaminoglycan scaffold that we talked about last time.
And what we did was we had a layer that was a collagen based scaffold for the cartilage and we had another layer that was a mineralized version of that.
And one of the main things we did in this project was figure out how to mineralize that scaffold.
And then we made this two-layer osteochondral scaffold.
So I'll talk about that, OK?
So are we good?
So this is a schematic of articular cartilage.
Articular just means it's in a joint, so between like, say, two of your long bones, and there's several regions.
So this shows both the cartilage and the bone underneath it.
And so this top layer is called the superficial layer, and then there's a transition zone, and then there's this deep cartilage here.
And the little white lines represent how the collagen is oriented in the cartilage.
So the collagen is oriented, more or less, vertically here and then it becomes more kind of woven and horizontal towards the top.
And so those three different zones, the collagen is oriented differently.
Then there's a region down here called the tidemark.
Everything above here is just cartilage and everything below is calcified to some extent.
So this next layer down here it's more cartilage-like, but it's got some calcification in it as well.
And then here's the compact bone.
They call it subchondral bone.
It's the bone below the cartilage.
So chondral is cartilage.
And then here's the trabecular bone.
We talked about trabecular bone a couple of weeks ago, OK?
So one of the things is there's different types of collagen and the different types may have slightly different fiber structures or slightly different compositions.
They're all related, but they're slightly different types.
And bone has what's called type I collagen and cartilage has type II collagen.
So when we made the scaffold, we wanted the bony layer to be made with type I and the cartilage layer to be made with type II.
So I think with the 3032 people have probably seen it.
I think I did a version of this for you guys, didn't I at the end of term, something?
Yeah, but there's other people, so it's really for them.
So if you think of articular cartilage, it has difficulty repairing itself and one of the reasons that it's difficult for the cartilage to repair itself is that there's no blood supply in it and another reason is that there's not very many cells.
So chondrocytes are the cells in cartilage and if there's a low volume fraction of the cells, then it's not so easy for that small number of cells to actually produce the extracellular matrix, which is kind of what you think of as the cartilage.
And it can be damaged, either as I said, from sports injuries, typically that's what young people get, they tear their cartilage in some sporting injury accident, or from osteoarthritis.
So these scaffolds we're talking about, they're not really meant to repair large amounts of cartilage that are damaged.
And as I said, the cartilage has a poor capacity for self-repair.
So there's several treatments, there's three treatments that are given currently and the most common one is called marrow stimulation.
And some of these orthopedic treatments are fairly crude when you look at it.
So this marrow stimulation, what's involved is they take a drill.
So here's the drill.
And they basically drill through the cartilage.
So this would be the cartilage layer here.
And they drill down into the bone and they want to get down into the trabecular bone.
So they want to go below the cortical or the subchondral bone and they want to go down into the trabecular bone.
And the reason they want to do that is the trabecular bone has bone marrow in it and the bone marrow has mesenchymal stem cells, and they want those mesenchymal stem cells to move up into the cartilage layer.
So that's what this red glob is here.
The idea is that you've made a hole and now that hole is going to fill up with a blood clot, which will have these mesenchymal stem cells, and that those will form cartilage.
So it does work to some extent, but it's not really a great result.
There's something like 75,000 of these done a year.
As I said, this is the most common kind of repair.
The next most sophisticated type of repair, what they do is they take plugs of bone and cartilage from another place.
So say this is where the defect is and they want to repair that.
They take bone and cartilage from this spot up here.
So they sort of drill out little cylindrical cores and then they plug them into this bit here.
And that's called an osteochondral autograft.
Sometimes it's called mosaicplasty because they build up a little mosaic from all those little pieces.
And then they just leave these donor sites, where they took the plugs from, they just leave those empty.
So they try to take the bone and cartilage from regions where the loads are lower, but they end up leaving holes, which is not so desirable.
And again, those holes may fill a little bit by the previous mechanism.
Then the most fancy method that they use now is called autologous chondrocyte implantation.
So what they do is they harvest cartilage cells from the patient.
They then take them to a lab and they culture them for two or three weeks and they get them to multiply and proliferate and grow and then they re-implant the cells.
So this works fairly well, but the difficulty is it involves two surgeries, so one to get the cells out and one to put them back in, and there's the cost of the cell culture.
So this is a much more expensive procedure because of the two surgeries and the cost of doing the cell culture, but I think it's the method that works the best.
And when I talked about this in 3032, I mentioned Dara Torres.
Dara Torres is an Olympic swimmer.
She's 47.
She began swimming in the Olympics in 1984, more than 30 years ago.
And she has swum in the Olympics, not every one, but up until 2008.
And even 2008, she won three silvers that year.
She was, I think, the oldest person who's ever won a medal in the Olympics.
And she had this surgery done at the Brigham a few years ago.
And it used to be, they've stopped running these commercials but the Brigham for a time was running these commercials, that featured Dara Torres and saying basically that she had this surgery done there.
I'm assuming she was happy with it.
OK, so that's what they do currently and what we were thinking of and what other groups have thought about is could you repair damage in the cartilage by using a tissue engineering scaffold?
So what we were thinking about when we tried to do this project was we wanted to use a healthy articular cartilage joint as a model for our scaffold.
We wanted to have a layer that would go down into the bone so that you would have access to those mesenchymal stem cells.
And that's why we wanted an osteochondral scaffold, so that we would have a layer for the cartilage but also a layer that would go down into the bone.
We wanted to be able to control scaffold parameters, things like the mineral content and the pore size.
Remember, I said these tissue engineering scaffolds, the pore size is one of the parameters, that you want to have the pore size in a certain range.
And we wanted to use materials that would be appropriate for approval from things like the FDA.
So typically, when people make tissue engineering scaffolds, they don't start with some material that's never been approved before.
They start with something that already has approval for something else and that's what we wanted to do too.
So this was our idea of what we wanted the scaffold to look like.
We wanted an unmineralized type II collagen scaffold up here that would be for the cartilage.
We wanted a mineralized type I collagen scaffold down there for the bone.
And we wanted some region that had some gradient in mineralization because it would be like that layer of cartilage that was slightly mineralized.
So we wanted to duplicate that whole structure.
So that was our picture of what we wanted to do and we had a pretty good idea of how we were going to make this.
We were just going to use that same process that [INAUDIBLE] developed for the skin scaffolds.
He was involved with this project but just used type II collagen instead of type I.
The challenge was really figuring out how to make the mineralized collagen scaffold and then how to get this gradient in the mineralization between the two layers.
So this, I think, I showed you last time.
So this is just the method we used to make the collagen-GAG scaffold.
So we take type II collagen.
We put it in acetic acid.
Remember, that destroys the periodic banding of the collagen and it improves the immunological response.
We add the chondroitin 6-sulfate, the GAG, to crosslink it.
We then just make a slurry out of that.
So we mix that all together.
We keep it as a slurry.
And then the second stage is you put the slurry or the suspension into a pan, and then you do the freeze drying process.
So you go through this process where you start at this room temperature and atmospheric pressure.
You cool it down to freeze it, and then you sublimate it, and you're left with a very porous collagen-GAG scaffold.
So these are pictures.
I think this was with a type I collagen, but it looks the same with the type II collagen.
So that's what the structure looks like.
I think I showed you this last time.
We can control the pore size by controlling the freezing temperature.
So the way the freeze dryer works is there is shelves that you put these pans on, and these cooling elements go through the shelves, and you can set the temperature of those shelves.
So we would set the temperature of those shelves to different values and we got different pore sizes.
So the colder the shelf temperature was, the faster the freezing, and the smaller the size of the ice grains, and then the smaller the size of the pores.
And this is just the mechanical response again.
So we did mechanical tests on it.
These are some of the numbers for the properties.
So we tested it dry and wet, and we measured a modulus and a buckling collapse stress for it.
So those are just some values there.
And then it came to making the mineralized collagen-GAG scaffold, and one of the students in Cambridge, England was really the main person who did this, Andrew Lynn.
And he worked with Brendan Harley, who was our student here, and after they graduated, I had another student, Biraja Kanungo, who worked on this too.
So Andrew Lynn really developed this technique.
And this involved taking the collagen and the GAG, so the same as for the other scaffold, but this time, if we want to make a mineralized scaffold, we somehow have to get calcium phosphate into it.
So it's got to have sort of a hydroxyapatite-ish type of component to it.
So this time we used phosphoric acid instead of the acetic acid and we put some calcium salts into the mixture as well.
And Andrew was really the one who figured out how to do this and what salts to use.
And then the process was very similar.
So we have this slurry.
We mix the slurry up.
We did a freeze drying process, and then we crosslinked it with a chemical crosslinker called EDAC.
So it's just a chemical that you put into this and it crosslinks it all.
So that was how we made the mineralized scaffold.
The mineral we got was something called brushite, which is a calcium phosphate, but it's not exactly the same as hydroxyapatite.
And we could control the amount of brushite by different weight fractions or volume fractions by controlling how much of the calcium salts we put in and what the molarity of the phosphoric acid was.
If you take brushite and you put it in water, it then converts to octacalcium phosphate and then to apatite by a hydrolytic conversion.
So the apatite is related to the hydroxyapatite in bone.
And this was the structure of the mineralized scaffold that we got.
So you can see it looks a little different from the collagen-GAG scaffold.
It's much denser.
Typically, the densities were like 5% or 10% dense.
And remember, the collagen scaffold was 0.5% dense.
So it's a lot denser.
But if you notice, there's a few things to notice here.
So one is the pore size, that's a 500 micron bar.
And we could make pores between about 50 and 1,000 microns, depending on the freezing conditions.
And the range of pore sizes we were shooting for was somewhere between 100 and 500, so we could get in the right ballpark with that.
And you can see, just looking at that picture, if that's 500 microns, those pores are somewhere of that order.
Another thing to look at is this image here, and this just shows, the white little dots are the calcium phosphate mineral, and it just shows that the mineral is uniformly distributed throughout the thickness of the scaffold.
So some people had tried to make scaffolds for regenerating bone where they take, say, one of those polymers for resorbable sutures or they take collagen and they coat it with hydroxyapatite, but the shortcoming of that is that the scaffold is going to resorb over time.
The cells are going to secrete enzymes, which are going to eat away at the scaffold.
And if the scaffold resorbs, you're eating away at the hydroxyapatite first and then you're left with whatever polymer is underneath that.
Whereas this gives you a more uniform composition throughout the thickness of the scaffold, and you've got calcium phosphate everywhere throughout the thickness of the struts in the scaffold.
So that's the structure of that.
We wanted to make sure that the mineral was uniformly distributed throughout the scaffold, and we did some micro-CT imaging, some micro computer tomography.
And this is our sample here, and this red line just says that is the plane at which this image was taken, and the black is the mineral.
And then here's a lower plane and here's another image.
And you can see the calcium phosphate's pretty uniformly distributed throughout that specimen there.
And this was just another way of looking at the same thing using EDX in an SEM to look at where the calcium was and where the phosphate was.
So again, this is uniform distribution of the mineral.
We did mechanical tests on these scaffold as well.
So we measured moduli and collapse stresses.
We get the same kind of stress strain curve as all these other cellular solids.
This was done by- Biraja Kanungo was the student who did this bit.
One of the things we found was that with these mineralized scaffolds you could manually compress them.
If you pushed them down and you hydrated them, they would recover all the deformation, but we were increased in improving the mechanical properties of the mineralized scaffold for improved handling during surgery.
And there's another reason that I'll get to as well.
So the second reason is that, if you look at how bone itself forms, it forms from a collagen precursor, so bone in your body.
And there's something called osteoid, which is this collagen-based precursor to bone, and it has a modulus of about 25 to 40 kilopascals and Angler showed that if you have mesenchymal stem cells and you put them on substrates of different stiffnesses, they differentiate into different kinds of cells, depending on the stiffness of the substrate.
So the substrate stiffness can affect what kind of cells you get.
And the idea here was we thought, well, if we could get a stiffness that was close to this osteoid, what the natural bone formation has, then that might help the mesenchymal stem cells differentiate into the bony cells that we want them to.
So we wanted to try to reach a stiffness of this in the wet state.
And if I back up here, you can see the stiffness we had was around four in the wet state, four kilopascals and we want to get to 30 or 40, something like that.
So these are our equations for the modeling of the mineralized scaffold, the open celled foam models for the modulus and for the collapse strength.
So we could change different things.
We could change the solid properties or we could change the relative density.
The geometry of the thing is probably not going to help us too much.
So basically, that's what we did.
We first started off trying to increase the mineral content.
We thought if there was more mineral content that would make it stiffer.
So Biraja made these more highly mineralized scaffolds and these are just some SEM images of those.
But the thing he found was that the properties actually got worse when he had the more highly mineralized scaffold.
The modulus went down and the strength went down, so that wasn't very helpful.
And when he looked into it more detail, he found that he had more voids in the cell walls and he had more disconnected walls.
This shows some of the micrographs, so this isn't really very quantitative, but you can see there's holes here.
There's a few holes here, but there tended to be a bigger volume fraction of holes in the more highly mineralized scaffolds and more walls that were disconnected.
So we realized increasing the mineral content wasn't going to work very well.
And then the second thing he tried was increasing the relative density.
So we started off at this density of 4 1/2% dense and he developed a method of increasing the relative density up to about almost 20% here.
He did this by, first of all, he tried to just mix more of the constituents into the slurry, that's kind of the most obvious thing.
But as you add more constituents, it gets harder and harder to mix the thing up and have it homogeneously distributed.
So in the end, that's not how he made these things.
He started with the starting mixture and then he had a vacuum system for sucking water out of it.
So he would reduce the amount of water, which essentially increased the amount of solids that was in there.
And you can see in this last one here, the most dense scaffold, he was sucking the water in one direction, and he sucked it so much that he was starting to get the cells collapsing.
So this one here, these cells that have this sort of elongated orientation, that's because the cells are starting to collapse because of the vacuum that he was applying.
But he could get a pretty good difference.
This was almost 5%.
That's almost 20%, so roughly a factor of four.
There are, yeah, four difference between them.
So then he did mechanical tests too, and he measured the relative density.
Here's the moduli wet and then the strength dry and the strength wet.
So you can see here, if you look at the dry moduli, for instance, when you go from this relative density to that relative density, from about 14% to 19%, the modulus actually drops down.
And if you look at the structure, I think, it's because you've got this flattened structure here.
You've collapsed the cells a bit already.
So there's a maximum density that you might want to go to, probably somewhere around 14% or 15%.
But the wet modulus we've got here is around 35 kilopascals, so that's close to the target that we had.
It's close to what we wanted to have.
So one way you could get the right or the appropriate modulus is by increasing the density to that value.
Another way is by playing around with the crosslinking.
So those values were for non-crosslinked scaffolds.
So here, this is the 14% dense scaffold wet, non-crosslinked it was around 35 kilopascals.
This is a dehydrothermal treatment, just basically heating it up.
It gives you a higher modulus.
And then this is a chemical crosslinking technique that increases the modulus again.
So if you put all of this together, you can show these results for the modulus on one table.
So these are all the wet modulus.
So it's pretty clear by playing around with the relative density and the crosslinking, that you can get a scaffold in that range.
And the idea is that that would help get the mesenchymal stem cells to differentiate into these osteoblast-like cells.
So then we wanted to also see if our models for the cellular solids could be applied to this scaffold, and we needed the solid properties.
So Kristyn Van Vliet helped us with this, and we isolated a single strut, bonded it to a glass slide, and, I think, I mentioned this last time, we used an AFM tip to then do a little bending test.
So I think the one I mentioned last time was for the collagen-GAG scaffold.
Then we also did the same thing for the mineralized scaffold.
And here we measured a modulus of about seven gigapascals, so that's a dry modulus.
And just for comparison, the modulus of the solid and trabecular bone is something around 18.
So it's lower, but it's in the same ballpark.
And by nanoindentation, we measured a strength of about 200, and that's similar to what you would get in trabecular bone.
So the solid in the struts themselves is not exactly like trabecular bone in mechanical properties, but not too far off.
And then here's a plot of the scaffold modulus divided by the solid modulus against the relative density.
And this line here, the curve, is a squared relationship.
So that's what we'd expect for the foam models.
And this is the strength.
These things fail by a plastic or a brittle failure, and this curve is a three halves power with relative density.
And again, that's what you'd expect from the cellular solids model.
So that gives you a reasonable description of the behavior of the mineralized scaffold.
So again, these were the considerations in trying to make the osteochondral scaffold.
So now we have a collagen scaffold and we have a mineralized scaffolding, and we want to put the two of them together.
So we wanted to use the joint as a model, and we wanted to have some intermediate layer that had some gradation in the mineralization.
So that was the next step.
And the way we did that was we used what we fancily called liquid-phase co-synthesis.
This just meant that we took the mineralized collagen-GAG slurry, we poured that into a mold, and then we poured the non-mineralized slurry into the same mold.
And then we just allowed the two slurries to interdiffuse for some time period.
And then we did the freeze drying step.
So the idea was, that if you poured the one on top of the other, the mineralized one is denser and then you put the less dense one on top, but over some period of time, there will be some diffusion between the two.
And then we just did the freeze drying step.
So this is the scaffold we ended up with.
This is a micro computer tomography image.
So here's the collagen-GAG scaffold for the cartilage on top, and here's the mineralized collagen-GAG calcium phosphate scaffold on the bottom for the bone.
And that just kind of shows what it looked like.
The porosities and the pore sizes we got, the collagen-GAG was about 98% porous and had a pore size of around 650 microns.
The mineralized scaffold was 95.5% porous and had a pore size of 400, roughly, microns.
And then this is what the structure looked like in the EDX.
You can see, this is the collagen-GAG layer, this is the mineralized layer, and there's some zone in between that's a little bit mineralized, not as much as the bony layer but more than the cartilage layer, and the same with the phosphorus.
So we have this collagen-GAG slightly mineralized layer, and then a more mineralized layer And then finally, there was a student, Scott Vickers, who worked with Myron Spector at the Brigham.
And oops, oops.
No, I don't want the weekly updates, thank you.
Sorry.
Well, let me just get rid of this.
Oop, where's my little mousy mouse?
There we go.
OK, so Scott Vickers worked with Myron Spector, and Myron had a surgeon who could do animal studies.
So we did some animal studies on goats.
I think there were six goats at the Brigham.
And they took a plug out of the knee of the goats, and they put our scaffold in.
So this is one of the surgeries as they're about to poke the scaffold in.
And then, Scott waited, I think it was four months, and then sacrificed the goats, and then did the histology.
And this is one of the images from his PhD.
So this staining shows that you've got tissue growing in.
The scaffold was where these little black dotted lines are.
So you had bony tissue in here, and there was a cartilage-like tissue formed at the top.
It wasn't perfect articular cartilage, but it was something similar to that.
And really was as far as we took the project with the funding that we had.
We had a-- I don't know if you remember that-- well, it's beyond before your time.
But Cambridge and MIT had a big research collaboration and the student exchange was part of that.
And this was done through that research collaboration between Cambridge and MIT.
So this was as far as we took it with that research funding.
Andrew Lynn, who was the student in Cambridge, who developed the mineralized scaffold, he then started up a company called Orthomimetics, and he had longer term animal studies done, and he took it a little further.
In Europe, there's something called CE Mark approval, and CE Mark approval means you can start doing clinical trials.
So he never got FDA approval for it, but he got approval to have clinical trials in Europe.
And the first clinical use was in February of 2009.
And they started off using it for the donor sites for the mosaicplasties.
Remember the second method I talked about?
They take plugs out of one region and put them into the region with the damage.
So what they were doing was using our scaffold to fill up these donor sites here and they found that worked quite well.
And then eventually, they started using it for the primary sites as well.
And as of about April a few years ago, April 2012, they had treated about 200 people with this
Wait, so why not just directly start putting it in the-- why start with the--
I think because these were supposed to be sites that were less loaded, weren't as highly stressed, and they thought that was a more, not as critical a place to put them in.
Yeah?
So what is different about this scaffold?
Is it the lack of the dense bone that allows all of the marrow cells to migrate up to the cartilage?
Well, I think, there's not that many people that have made osteochondral scaffolds, so it's good that we've got these two layers.
And the idea that you try to get the stem cells up into the cartilage.
The stem cells will differentiate into the bone or the cartilage, I think, partly depending on the stiffness of the surrounding tissue
But they don't do that in normal bone because of the dense layer?
No, I think they would.
But the thing is, I mean, in some ways, that very first technique where you just drill holes in, I mean, in some ways, that's what it's counting on, right, is that the marrow mesenchymal stem cells are going to differentiate either into the bone or into the cartilage.
But it's just got a hole to differentiate into.
So this gives the cell something to attach to and I think, gives a better result.
Yeah?
So is part of the scaffold and this bone, are they removing the original bone?
Yeah, they remove-- yeah.
Let me back up a step.
So when they do this thing here, so this is in a goat, but they would do the same thing in a person.
So when they drill the hole to put that in, they go through the cartilage, they go through the compact bone, the dense bone, and they go into the trabecular bone, because you don't really get into the marrow until you're in the pores of the trabecular bone.
OK?
Are we good?
How do they attach it?
How do they attach it?
Yes
I think it's just a press fit.
I think they just drill a hole and stick it in.
And these are all in joints, right?
So there's always another bone pressing against it.
I mean, in the surgery, they kind of peel things apart so they can do the surgery but there's another bone pressing down on it.
So I don't think there was any glue or anything.
So there's that.
So this is just a summary.
So we were able to make this two-layer scaffold with a gradient interface, and we tried to make it so that it mimicked the osteochondral tissues.
And this freeze drawing process allowed us to control things like the mineral content, the porosity, the pore sizes.
And then we used materials that had already been approved for medical devices.
And this was funded by a number of places.
So the Cambridge MIT Institute was that collaboration I mentioned.
I have a chair and I used some money for that.
Brendan Harley was one of the students involved and he got a fellowship from MIT.
And Andrew Lynn got a fellowship through the Cambridge Commonwealth Trust and through St. John's College in Cambridge, that's his college there.
So he had had funding through that.
So I think that's the end of that talk.
So are we good with scaffolds?
Yeah, you're good?
So obviously, this probably whole course is on tissue engineering.
This is just scratching the surface and giving you an introduction to it, but I wanted to show you how a lot of these scaffolds look a lot like the foamy materials that I work on, and that you can use the same models for trying to understand the mechanical properties of the scaffold.
And even though the scaffolds are acting in a biological way, there's actually a connection between the mechanical behavior of the scaffolds and the biological response.
So I'm going to talk on Monday about cell/scaffold interactions, so how the environment biological cells are in can affect how they behave.
So we're going to talk about things like cell adhesion and cell migration and cell contraction, contractile behavior.
So I've got another talk a little bit like this and that means I'll have a few notes I'll put on the board on Monday about how the environment that the cells are in affects how they behave, OK?
So this is leading up to that.
It's sort of a similar thing.
So I thought for the rest of the time today, because I knew this was going to end early, I thought what I would do is just switch gears and talk about writing.
So normally when I give a class, I don't talk about writing all that much.
I talk about the technical stuff.
But it turns out writing is actually a huge part of what scientists do and engineers do and whether or not you end up with an academic job or a job in industry or working for government, no matter where you are, you are going to have to write things.
And the better you write, the better off you're going to be.
So I made copies of this brochure and I have a few little slides.
And you're going to have to write up your project report for me, and I thought it might be helpful to just go through this little brochure.
So I'm not going to write stuff on the board.
Everything is pretty much in this brochure, but I thought I'd just walk you through some of the main parts of it.
Oh, did you get one?
Here, have one.
So you might want to think about this when you're writing up your project report for this class, but this really is, it's general.
It's really for any time you have to write something.
This is helpful.
So Mike Ashby put this together.
And as I mentioned, he's done a lot of scientific writing and especially in material science and engineering.
He also makes paintings.
This is one of Mike's little paintings of him writing a paper.
And what I was going to do is just walk through it.
So there's just a few figures, and mostly it's text, but let me go through some of the figures.
So I'm going to just turn to what's page three, OK?
So one of the things that he says to start, and I think this makes a lot of sense, is that you can think of writing a paper the same way you can think about designing something in engineering.
So when you think about design, there's different stages of design, right?
There's a conceptual design, where you just decide roughly what the thing's going to be.
There's embodiment, where you work out a lot of the more details.
You design one version of it.
And then there's the detailed design, where you do all the fine-tuning.
You do all the final design things.
And before you really start your design, if you were going to design an engineering thing, you would first of all think about what's the market.
And if you're writing, the market is your readers.
And so when you think about what you're going to write, you have to think about who's going to read it.
And it's the same thing when I give a talk.
When I give a talk, before I make a single PowerPoint slide, the first thing I think about is who am I talking to and what do they already know?
What's the audience?
What are they looking to get out of the talk?
What am I trying to convey in the talk?
And the writing is the same thing.
You have to think about your audience.
So for instance, I have technical talks.
Obviously, I go give technical talks at meetings, but I do other sorts of talks too.
I've got the woodpecker talk.
I go give the woodpecker talk at the Mass Audubon.
And people come to listen to that talk who are interested in birds but they're not engineers.
So when I do that talk, I have to make it so somebody could understand it who's intelligent but they're not necessarily an engineer.
So I give a different kind of talk when I do that than when I give an engineering talk.
And I got invited to a student dinner.
I'm going out to somewhere with some students on Friday.
I'm going to give that how I became a professor talk, and when I do the how I became a professor talk, it's more general, and so it's a different audience that I'm thinking about.
And in some ways, when I do these talks, it can be the same group of people, but depending on what I'm talking about, the way they look at it is different, OK?
So the market thing is something to think about.
So if you're writing a thesis, your market is your PhD committee, who's going to read the thesis and examine you on the thesis.
If you're writing a paper, you've got to think about the market as being other people in your research field, some of who are going to be the reviewers, who are going to be reading it and criticizing it.
If you're writing a general popular science book, it's a different kind of audience.
If you're writing a research proposal, the audience is going to be the funding agency.
Are they interested in what you're talking about, but also reviewers, who are going to be deciding whether or not to give you the money and what the criticisms are.
So you have to think about who the market is.
So that's one thing.
And one thing that think about in doing the writing-- you know Mike and I have written several books together, and when I tell my neighbors I've written books, they somehow think-- their first idea is that we start on page one and we start writing the book from page one and then we work our way through to page 500.
And that's not how we do it at all and that's not how I write papers.
That's not how most people write papers.
You've got to think about the big picture and think about, roughly, what goes where.
And then maybe think about a draft that gets the scientific facts right.
You put the information down, and then you try to figure out about how do you make the style really nice?
How to make it read well?
How do you make it easy to understand?
How does one paragraph lead into the next paragraph?
So it's an iterative thing.
It's not like you start at page one or line one and you just start writing.
So it's an iterative process, the same as engineering design is an iterative process.
If you were going to design, I don't know, a skateboard or something, you wouldn't just think you were going to make one and that would be it.
That's how writing is, and often, students don't see it as this iterative thing.
OK, so that's that page there.
Let's see, I already talked about market.
I think I have another little slide about-- here's another slide here about markets.
So this is what I just said.
Who are the readers?
How are they going to use it?
So you've got to think about who you're writing for or if you're giving a talk, who's going to look at that.
OK, now the next phase is to make what Mike calls a concept sheet.
I would just think of this as an outline.
And I always make some sort of outline before I try to write anything.
And there's different ways you can do that.
The thing that Mike's got here and there's a nice little figure on page six, which is what I've got up here, is he takes a big piece of paper.
In Europe, it would be known as the A3.
Here it would be known as the eight and a half by 17.
You take a big piece of paper.
This stationery company actually makes paper with big blank space in the middle and little note space on the outside or you could just use the back of it.
So you take a big piece of paper and you may think this is douffy, but it actually helps.
So you take your big piece of paper, and you just make boxes about each topic.
You're going to have an introduction, you're going to have methods and materials, you're going to have a result section.
And you think about what should go into each of those boxes.
And what you're trying to do here is think about the whole paper and how it all fits together and what goes where and what are you going to include and what are you not going to include.
So people think about writing as sitting at the keyboard and typing, but that's just the-- how when you get a problem on a problem set, you turn and crank, that's the turning and cranking part.
The thoughtful part is figuring out what to put in, what to leave out, what figures you want, how you organize the whole thing, how you put it all together, and this helps you do that.
And you can make this-- it's just for you.
It doesn't have to look great.
You can make this messy if you want.
It doesn't really matter.
But it's good to have some sort of an overview of how you want to put the thing together, and that's what this stage helps you do.
So Mike's put other little things here.
So see papers by so-and-so and so-and-so.
So you've got an introduction.
You want to talk about something.
You know there should be some references go here.
Maybe you think there's some extra references you haven't got.
Maybe something in the method, there's some analogy you can use here.
Maybe you need a figure, needs a good figure.
You don't have to actually have the figure.
You just say I need a figure and you have a vague idea what the figure looks like.
Here there's some discussion point.
Discuss this with collaborators, Ed, all this stuff.
So you just put down roughly what needs to go where and you think about the whole thing and how it's all going to fit together.
And then as you work through it, it looks more like this.
So this would be on page seven.
And this is filled in, and if fact, this particular one, what he's written down here is the overview he made for making this booklet, OK?
So the things refer to this book.
So here's the introduction.
Here's the little chart we went through and through, the concept design, embodiment, and detail, blah, blah, blah.
Here's the need, the market need.
We just talked about that.
Here's the concept thing that we we're just talking about now.
Then we're going to talk about each of these different phases.
And his initials are MFA, Michael F. Ashby.
Think out, that means think about this more.
I haven't figured this out yet.
I need an example here, all these kinds of things.
So this is the way he does it.
There's a couple of other ways you can do it.
One way is to just make a bullet outline, that's typically what I do.
You know you're going to have an introduction, these different headings.
And you might put in the bullet outline these same sorts of things, like I need this figure or we need to get one more set of data or I need to look up this reference.
So it doesn't have to be finished, but it's a thing that tells you what have you got, what do you need to do to make it all come together.
So that's one way to do it.
Another way to do it is by thinking about what figures you want in the paper.
So some people, before they write any words, they say, well, I know I want to have these figures, and then they build the words around the figures.
And they can even sketch out what the figures are.
Some people even start before they start the project, they say what kind of figures do I want at the end of the project?
And they don't know if the data is going to do this or that.
They don't know which way the data is going to go, but they say I want to have a plot of one thing versus another thing.
Oh, you're looking like this is not OK?
No
No?
OK, yeah, some people do this.
So even when you're ready to write, you could say to yourself, well, in the methods, do I need a figure that's a schematic of some apparatus?
In the results, how am I going to present the results in the discussion?
Do we need some other kind of figure?
So if you have a set of figures, you can work the text around those figures.
So that's another way to do this.
So those are just three options, but all of them have in common that you think about the whole paper and how it all fits together and what goes where and what do you have already and what else do you need to get, OK?
So that's that.
And then what I was going to do-- I think that's probably the last figure that's-- yeah, OK.
So there's just this little thing here.
So what I was going to do is just talk about some of the other things in this little booklet that work through each of these stages of the writing.
So that's the concept, and then the next phase would be what's called the embodiment, if you think of the design language, and that would be the first draft.
And I think most people find the most difficult thing is to write the first draft.
I mean, I find that the hardest.
Once you've got something, editing it is relatively straightforward.
You go, oh, this piece should go over there or there's something missing.
But getting the first draft down is the hardest thing.
And one of the things-- it's like when I talked about writing the book, you don't write the draft sequentially either.
I typically tell students to start with the materials and methods because that's the most straightforward and it's the easiest to write.
There's not a whole lot of mystery to how you actually did something you've already done.
So I tell people don't write the introduction first.
That's a bad idea because it's actually often not so easy to write the introduction.
So I tell people to write the materials and methods first.
Write the result section because you've got your results and you know what the results are going to be.
So those are the two easiest things.
And often, I think, what people find is if they find it hard to write it's because they don't know what they want to write or they haven't thought things through.
They haven't got their thoughts together.
And if you've got your thoughts, if you know what you want to say, then the writing becomes easier.
So one of the pieces of advice Mike gave me, and it's in this booklet too, is in the first draft what you should try to do is just get the facts down.
Just get the information down and don't worry about if it doesn't sound quite right or if this sentence doesn't lead into that sentence.
Don't worry about the style of it at all.
Just try to get the facts down.
Just try to get the information down.
And once you've got the information and you've got some kind of framework-- I'm not saying that you don't want to make the style good, you do, but you don't need to do that the first thing.
The first thing is just to get the facts down, and then you can edit it later on.
It's more of this iterative process.
OK, so the first draft, the most important thing is just to get the facts down.
Then, I'm not going to read all of these things because you can just read them yourself, but I'll just comment on a few things.
So one thing he's got-- I'm on page eight now, is the abstract.
So the abstract should be concise.
It should be fairly short and you want to tell people why you're doing what you're doing, what you did, what the key result is, and what the conclusions are.
So it can be pretty short.
There's a section on the introduction.
You can just read that.
Let's see here.
Yeah, I think, you can read yourself these other things.
I don't want to go through the whole thing.
All right, so that's that.
OK, so you get to the end of the first draft, there's a little section on figures here.
Let me just make some comments on that.
People who are busy and may not want to read every word that you've written will look at the figures, and they'll look at the figures to make some judgment about does this look useful?
Does this look like you've got something I want to spend more time on?
So it's important to make the figures easily understandable and to be fairly self-contained.
So you want to make the figures clear.
You don't want to have too much information on it so that people can't follow what's going on, and you want them to illustrate the points that you're trying to make.
So the figures are very important.
And then when you've got the first draft finished, the next step, which I find tremendously useful and students often don't get that this is a step.
The next step is you put it somewhere and you don't do anything with it for a while.
And this is why it's important not to write it up at the last second.
Because there's something about, you work on something, you put it to one side, you don't do it.
You just leave it and you go do something else.
And then when you come back to it, all of a sudden, you see things that you didn't see the first time through.
So the next step is you just put it to one side for a couple days.
But you can only do that step for a class if you started before the night before it's due, OK?
So that's why you have to start earlier.
That's why I'm telling you this now, OK?
So you put it aside and then you go have a cup of coffee.
You see there's several cups of coffee that Mike has happily drawn for us here.
OK, then the next part of this is all about grammar and sentence structure.
I'm not going to go over that.
You can read all that yourself.
But there is one amusing thing I want to tell you because there's a cute story.
I'm on page 16 now.
There's a thing about spelling.
Mike is a terrible speller.
When I was his student, he used to-- there were no word processors.
And we used to write things and he would write things and I would say, I can't read your writing.
And he says, ah, well, I make up for my bad spelling with my bad writing.
So I couldn't tell if he had spelled something right.
The other thing we found was that, we wrote the cellular solid book, the first edition was in the 1980s.
And I grew up in Canada, then I lived in England, then I moved here.
He grew up in Australia, then he moved to England, then he moved to the States then he moved back to England.
And words that end, that we spell I-Z-E, like normalized, like normalized variables.
They're I-Z-E here.
In England, they're all I-S-E. OK, and then some of the words, it turns out, Canadians spell with an I-Z-E and some with I-S-E.
So anyway, we wrote the entire fricking book.
We were almost ready to send it off, and we have all those graphs where the axes are normalized, and we must have used the word normalized like 100 times, and we realized half the time we had spelled it I-S-E and half the time we had spelled it I-Z-E. And then we had to go back and find them all and fix it all.
So anyway, that's just our little story about spelling.
OK, there's a thing about punctuation.
You can read that.
That's kind of boring.
I wanted to move on to the bit about style, because style is important too.
And people remember papers that are well written.
Obviously, papers that have valuable scientific information are memorable too, but if it's well-written, it's more memorable, and the style really is important.
And it's the same with giving a talk.
You can convey the same information, but if you don't have it presented in a clear way, people aren't going to remember it.
So the first rule-- so I'm on page 20.
The first rule, now, is to be clear.
And being clear, it doesn't mean having long-winded sentences with lots of jargon.
It really just means make it short, make it concise, make it clear what you want to say.
So Mike has several examples here of headlines from the newspapers, which were not clear.
So I'll just read these out because I find them amusing.
"Red Tape Holds Up New Bridge."
So, OK, maybe the red tape temporally held it up, but not spatially held it up.
OK, and then here's another one.
"Something Went Wrong in Jet Crash, Expert Says."
In fact, you hear this all the time on the news now.
Obviously, something went wrong.
You don't have to be an engineer to figure out something went wrong in a jet crash.
Then, "Chef Throws Heart In To Help Feed the Hungry."
OK, well, maybe that's a little too much.
This is my favorite one, "Prostitutes Appeal to Pope."
OK, there's different ways you can appeal.
You don't have to appeal to the Pope in that way.
And then, "Panda Mating Fails, Vet Takes Over."
I don't think the vet really took over with the mating, but you know what they mean.
OK, so one thing is clarity.
Another point is don't waffle.
You want things to be concise and get to the point.
So he's got this example here from what he cites as a well-known but anonymous materials text.
"The selection of the proper material is a key step in the design process, because it is the crucial decision that links computer calculations and the lines on an engineering drawing with a real or working design."
But what does it say?
It says material selection is important.
So don't say something in some long-winded way that you could say in a short way.
OK, let's see.
So let me move on to the next page.
The next step that I-- let me emphasize one more time, is 8.5, revise and rewrite.
So writing really is iterative.
Those books that you've seen that Mike and I have written, I cannot tell you how many drafts of each chapter we went through over and over and over again.
So to make it good, you have to do it over and over again.
OK, there's that.
Let me see.
Is there anything else I have here?
Ah, here's a couple things that are not so obvious.
So one is, for each paragraph you should have a good first sentence.
You should tell the reader something they don't already know.
So there's some examples here, and you can read through them.
But it is really helpful for each paragraph to have a good opening sentence.
Another thing is at the end of the paragraph it's good if there's a sentence that links it to the next one.
So it hints at what the next paragraph's going to be about, so it leads the reader through it, and it is a logical progression.
So an opening sentence for the paragraph and the final sentence, you might want to look at in a bit more gory detail.
OK, and then there are some references at the very back, at page 25.
And I brought a couple of books in that I have found helpful.
So you may have seen these.
One is called The Elements of Style and this is by Strunk and White, so William Strunk, Jr. and EB White.
You guys know EB White, Charlotte's Web, same guy.
He's written this small, small, not too long to have a look at book about style.
So there's some rules of usage, which is grammar stuff, but there's principles of composition and there's a chapter called "An Approach to Style."
And so here's some of the headings.
Place yourself in the background.
Write in a way that comes naturally.
Work from a suitable design.
These people are not engineers, but they're saying work from a suitable design.
It's like that concept thing.
Revise and rewrite, did I mention that, revise and rewrite?
Do not overwrite.
Don't make it too complicated.
Avoid the use of qualifiers.
If you say something's very stiff, what does that mean?
You can just say it's stiff.
You don't have to say it's very stiff.
Very compared to what?
Let's see.
Use orthodox spelling, blah, blah, blah, blah, blah.
Avoid fancy words.
OK, be clear, all this kind of stuff.
So Strunk and White is old but good.
And then, there's this book here by Bill Bryson.
Bryson's Dictionary of Troublesome Words.
So this is, it is like a dictionary.
It goes by letter and has different words, but it talks about how people sometimes use a word thinking it means one thing when it doesn't, it actually means something else.
And so it's just words that people use commonly that they sometimes confuse with what they really mean.
So it's a very handy thing too.
Do you know Bill Bryson?
Very funny travel writer.
If you ever go somewhere that Bill Bryson has been you should get the book he's written about it because he's very funny.
OK, so I think that's it.
Oh, and then, I think, the very end of this booklet has some examples of good writing and bad writing.
So you can have a look at that too.
All right, so are we good on how to write a paper?
Do you have questions on how to write?
Writing is important.
So I have one more writing story.
So when I first got this job at MIT, I was living in Arlington, and my mom comes to visit me from Toronto.
And I'm work, work, work, work, working, working, and my mom says to me, she says, my, you spend a lot of your life writing.
I'm like, Mom, that's what I get paid to do.
I get paid to write.
Like she always thought I was should be spending maybe 30 hours a week lecturing or something like that.
Like she, well, you're a teacher.
You only teach three hours a week, what is this?
I'm like, I write, Mom.
That's what I do.
So anyways, so writing is important and giving presentations is important.
And no matter what you end up doing, you're going to end up having to do those two things.
And if you do it well, it's going to make your life better, and if you do it badly, you're not going to be so good.
So that's my message.
So I'm going to stop there because I haven't got the next lecture ready, and I can start that on Monday.
So Monday we'll do the cell mechanic stuff.
And then let me just-- because we're getting ridiculously close to the end of term.
Like four weeks from today is the last test.
I know, it's shocking.
So there's one lecture on cell mechanics, and then there's some more engineering applications of foam.
I'm going to talk about energy absorption in foams and foams in sandwich panels, that kind of stuff.
And then, we're going to talk about some natural materials again.
We're going to talk about natural structures, so like natural sandwich panels and natural cylindrical shells with foamy cores.
So I've got lots of nature things at the last part of the course, and you know how I like those nature things.
So there you have it.
Bob's your uncle, as my mother would say.
You know what Bob's your uncle is?
Bob's your uncle is an English expression.
It just means there you have it.
There you have it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So last time we were talking about tissue engineering scaffolds.
And what we're going to talk about today still has to do with tissue engineering scaffolds, but we're going to look at it from a different perspective.
So last time we were looking more at sort of a clinical perspective, and looking at those osteochondral scaffolds for repairing small defects in cartilage.
And today what we're going to talk about are how cells-- how biological cells, interact with the scaffolds.
And there's various kinds of interactions.
So we're going to go through a bunch of these.
So the first one I'm going to talk about is degradation of the scaffolds.
Then we'll talk about the cell attachment.
Cell morphology-- so the shape of the pores in the scaffold can affect the way the biological cells-- what shape they have.
Biological cells could also contract the scaffold and apply mechanical forces.
So we're going to talk about that.
The stiffness of the scaffold and the pore size can affect the speed of cell migration.
And the stiffness of the scaffold can affect the differentiation of cells, so from one cell type to another.
So I thought today I'd talk about that.
This probably won't take the whole hour.
The next topic is on energy absorption in foams.
And so we'll probably start that towards the end of the lecture.
OK.
So the idea here is that we're looking at how scaffolds are being used, really, to provide a 3D environment to characterize the behavior of cells.
And in particular, how the cells interact with their environment.
So let's write that down.
So how the cell behavior is affected by the substrate it's on.
OK.
So the first thing we're going to talk about is scaffold degradation.
And if you think of the native extracellular matrix, the cells secrete enzymes which resorb that matrix and then they also deposit new matrix.
So it was kind of like what we were talking about with the bone.
The bone is always being resorbed and deposited.
And if there's a balance between that those two, then the density of the bone stays the same.
And if one of the rates gets out of whack, then you get osteoporosis and you lose bone mass.
So the idea is that in just the native extracellular matrix, the cells are producing enzymes that degrade the scaffold.
And those enzymes are also going to degrade the tissue engineering scaffolds as well.
And you want to be able to control the rate of degradation, versus the rate at which the native extracellular matrix gets deposited.
Excuse me, sorry.
So you can kind of imagine if the tissue engineering scaffold did not resorb quickly enough, you'd have scaffold there.
And the cells would be trying to put down their own extracellular matrix, and there wouldn't be a place to put it.
And if it resorbs too quickly, then the cells don't have something to attach to.
So there has to be a balance between the rate at which the enzymes are resorbing the tissue engineering scaffold, versus the rate at which the cells are depositing their own extracellular matrix.
So in the native extracellular matrix, the enzymes produced by the cells are resorbing the extracellular matrix.
And then the cells are also synthesizing so they synthesize ECM to replace it.
So the cells are also going to degrade the tissue engineering scaffold that you put in.
And the length of time that the scaffold is insoluble, or so that it remains in the body as a solid, is called the residence time.
And so then we require the scaffold degradation to occur over a time that balances with the new ECM synthesis.
And so the scaffold residence time must be about equal to the time required to make new native extracellular matrix.
So the degradation rate depends on the composition of the scaffold, on how much cross linking there is, and on the relative density.
Obviously, the more scaffold there is, the longer it's going to take to degrade it.
And with synthetic polymers you can vary the molecular weight of the polymer.
And sometimes if you have copolymers, one may degrade faster than the other.
And you can control the balance of how much of each copolymer you have.
And for natural proteins, like collagen, you can control the amount of cross linking.
So you can do the cross linking by various techniques.
That's what's called physical methods.
There's something called dehydrothermal treatment, where you heat the collagen up to 105 degrees C in a vacuum, in a dry environment.
And that eliminates water and causes more cross linking.
There's a UV treatment, ultraviolet light treatment, you can use.
And there's also chemical cross linkers you can use.
So there's different chemical methods you can also use to cross link the collagen.
OK.
So the next thing I wanted to talk about was cell adhesion.
And let's just wait a minute for people to catch up.
Are we just about there?
So this next slide shows a sort of schematic of how a cell would adhere to a substrate.
So down at the bottom here, all these little squiggly lines are representing the extracellular matrix in the native tissue.
Or you can think of it as a, say, a collagen scaffold.
But here we have the ECM.
And this little blob here is our cell.
This is the nucleus of the cell here, the little green blob in the middle.
And the cell is attached to the ECM through something called focal adhesion points.
And this schematic here is a blow up of that focal adhesion.
And at the focal adhesion there's proteins called integrins.
And integrins pass across the cell membrane.
So the idea is the integrins attach to ligands on the extracellular matrix.
And then they also attached to the sub membrane plaque within the cell.
And then that plaque attaches to the side of skeleton.
Things like actin filaments within the cell.
So this is what attaches the cell as a whole to the extracellular matrix, is these focal adhesion sites here.
And different kinds of cell behaviors-- obviously, things like cell attachment, but also things like cell migration, are affected by those focal adhesions there.
So we have that the cells attach to the ECM at focal adhesions.
And sometimes you see those referred to just as FA.
And at the adhesion point the cell has integrins.
And the integrins are transmembrane proteins, so they go across the membrane.
And they bind to like ends on the ECM.
And then the other end of the integrin is attached to the submembrane plaque within the cell.
And then that connects to the cytoskeleton.
And then different kinds of cell behaviors-- so for example, things like adhesion, and proliferation, and migration.
And that cell contraction, we're going to talk more about that in a minute.
They all depend in part on this adhesion between the cells and the extracellular matrix.
And the biological activity depends on how many binding sites there are.
So if you think of the extracellular matrix, it's got these ligands and it depends on the density of binding sites, how much interaction you can get.
So things like how much cell attachment you can get, depends in part on just how many of these binding sites you've got for the cells to attach to.
And that density of the binding sites or the density of the ligands depends on the composition of the scaffold.
But also on the surface area per unit volume of the scaffold.
So if you think of first, just the composition, if you have native proteins, like collagen, they have binding sites themselves.
They have native binding sites.
But if you think of synthetic polymers, like the resorbable sutured type of polymers that we talked about, they don't have binding sites and you have to coat the scaffold with some sort of adhesive protein.
And then the surface area per unit volume of the scaffold is related to the pore size and the relative density.
Let's call the specific surface area, surface area per unit volume.
And if you think of having some scaffold that's like an open celled foam, you can roughly calculate what the surface area per unit volume is.
So say each strut was a cylinder, then the surface area of each cylinder is going to be 2 pi rl.
If each one has a radius r and a length l. Say we had n of them, that would be your surface area.
And the volume of the whole scaffold, or one cell, would go as l cubed, the length of each strut cubed.
So if we just forget about all the constants here.
Forget about n. This just goes as r over l times 1 over l, and that goes as the relative density to the 1/2 power times 1 over the pore size.
So the specific surface area depends on the relative density and on the pore size.
And if you have a tetrakaidecahedron cell, you can work out exactly what that relationship is.
It's sort of a model.
And that gives you the relationship there.
And in this particular case, I think the relative density was 0.5%.
And so it's a constant over the cell size.
So one of the things we did in my group was look at how cell attachment varied with this specific surface area.
So we seeded cells onto scaffolds of different pore sizes.
We kept the relative density constant and we changed the pore sizes.
Remember I said, when we make these scaffolds by freeze drying we can control the pore size, by controlling the freezing temperature.
And we see that it's just a linear relationship between how many cells attach, or the percentage of the cells that were seeded that attach, and the specific surface area.
In here we used MC 3T3 cells.
It's sort of a standard cell one that you can get.
So Fergal O'Brien was the post-doc in my group who did that.
So I'll just say we find cell attachment is proportional to the specific surface area.
OK.
So that's the cell attachment.
So you can see how the scaffold design is going to affect how the cells attach.
So there's some relationship between them there.
Another thing people have looked at is cell morphology.
And so if you change, the sort of, orientation of the pores, how does that change the orientation of the cells?
So this was a study done in another group.
So here we have randomly oriented fibers that make up the scaffold.
And here they're not perfectly oriented this way, but more or less.
And then these are cells that have been seeded onto them, so that the green staining is the cells.
And you can see if the scaffold is random, the cells themselves line up with that fiber structure and become more or less random.
And if the scaffold has fibers that are aligned, then the cells, they also line up and be aligned.
So the morphology of the cells can be affected by the orientation of the scaffold pores.
Also the cell morphology can be affected by the stiffness of the cells.
Or the stiffness of the substrate.
So this is a substrate.
Here this was a PEG-fibrinogen hydrogel.
And they varied the cross linking of this hydrogel.
So they got different modularly for the hydrogel.
So these numbers here, are all the stiffness of the four different hydrogels.
And you can see the cell morphology changes from being a spread out thing on the least stiff substrate, to being just a little spherical or circular blob on the most stiff substrate.
So the cells respond to the substrate.
And so how the cells behave, depends in part on their environment.
So I wanted to also talk about womb contraction.
And talk about how cells contract scaffolds as well.
So one of the things people have found when they look at say, skin and regeneration of skin-- so say you had somebody with a burn and the surgeons will clean the burnt out.
And then what will happen as it heals, is scar tissue will form.
And the scar tissue forms in conjunction with the wound contracting.
So cells will actually migrate into the wound bed and they'll pull the edges of the wound together to try to close the wound.
And they won't close it completely, but they'll partially close it.
And that's called wound contraction.
And that is thought to be associated with the formation of scar tissue.
So the cells can actually apply mechanical loads.
And they can contract the wound.
And one of the things that Professor Yannas found was that if you use one of his collagen and gag scaffolds, you can inhibit that wound contraction.
And if you can prevent the wound contraction from occurring, you also prevent the formation of the scar tissue.
And that allows normal dermis to form.
So you get normal skin.
So this photograph here is of somebody who had burns over their entire torso.
And they put this tissue injury scaffold on this part at the bottom, but not on that part at the top.
And you can see these lines here are contracture lines from the scar formation.
And you can see this skin down here is relatively normal.
And in fact, when people look at the histology of the skin the forms using these scaffolds, they find that it is pretty much the same as normal dermis.
It doesn't have sweat glands and it doesn't have hair follicles.
So you can't sweat from that skin and you don't grow hair.
But apart from that, it's more or less normal dermis.
So this observation that if you can inhibit the womb contraction, you can prevent scar formation and you can get normal dermis to form.
That's led to some interest in just seeing how is it that the cells do this contract l process.
I think hitting the thing and my battery is dead.
So one of the things people have done, is they've just taken what's called, free floating scaffold.
They've just taken little disks of scaffold and put it in a cell culture medium in a Petri dish.
And they find that if you put, say fiberblast on it, the fiberblast will contract that scaffold.
And people have measured how much the diameter of the scaffold changes.
And so they've kind of measured this contraction just by-- it's almost like measuring a strain.
And what we wanted to do is we wanted to try to measure the forces that were involved.
So we first developed something called a cell force monitor, and I'll show you that.
And then we tried to calculate how much an individual cell could apply in terms of the force.
So we used this scaffold here.
This is the same collagen GAG scaffold I showed you before.
And here's the cell force monitor.
So that's just a schematic of holding a piece of the scaffold between two clamps.
So here it is in elevation view.
And then I'll just build the whole thing up, so you can see how it works.
So it's on a base plate.
It's attached to a horizontal stage that's adjustable.
Then there's a very thin beam here.
So this is another adjustable stage here, and this very thin beam here.
And that's attached to one end of this clamp.
And here's the matrix.
And this is attached to this other adjustable stage here.
And then when we have a proximity sensor-- so what's going to happen is, this is fixed over here.
The scaffold is going to contract with the cells applying these contract l forces.
This beam here is going to bend and the proximity sensor is going to tell us how much it's bent.
So we can measure how much that's bent.
If we know how much that's bent, and we calibrate the beam, we can figure out the force in the beam.
OK.
So we can figure out how much is the total force that the cells are contracting with.
And then this just is a little silicone well with some culture medium.
So that's the whole setup there.
Toby Fryman was a student who did that, who's married to Professor Van Vliet.
And I have a very big soft spot for both of them.
So anyway, that's the set up.
And the thing that Toby measured was the force, by measuring how much that beam deflected.
And he measured the force over time.
And he found that if he put say, a certain number of fiberblasts onto the scaffold, the force would increase and then reach an asymptotic point.
And you could describe these curves by this equation here.
Here's the asymptotic force.
And it's a 1 minus exponential of minus time over a time constant tao.
And then this number here is the number of fiberblast that were attached at 22 hours.
So he ran these tests for 22 hours.
And when he was finished, he could count the number of cells that were attached in the scaffolds.
So you would just wash off any cells that weren't attached and you can do accounting of how many cells are left.
And one of the things that he found was that if you plot that asymptotic force-- if you plot through this force over here, against the number of cells that were attached, you just get a linear relationship.
And the slope of that is roughly the force per cell.
And that's about one nano neutron.
Now this is a little deceptive because not all the cells are contracting.
And not all the cells are lined up in one direction.
So there are cells in different orientations.
But just as an order of magnitude the cells are applying something like one minute per cell.
So that's the effect of the cell number.
Another thing he did was he looked at what happens if you change the stiffness of that beam if.
You make that beam in the device different stiffnesses, how do the cells react.
And so the stiffness here are the stiffness of the system.
So there's 0.7 newtons per meter up to ten, so it's a factor of a little over ten difference.
And you can see the displacement per cell changes.
The stiffer the system is the less the cells can displace it.
But if you then plot the force per cell, you find that the force per cell is about the same.
So you develop about the same force.
So that suggests the cells are capable of applying a certain amount of force, and not any more force.
No larger force.
So he did that.
Then we were interested in what was the mechanism of this.
How were the cells applying this force?
Because I was kind of surprised to find out the cells even could apply forces.
So we were interested in understanding the mechanism of this.
And one of the things we knew that we didn't quite figure out how this all worked together was, we knew that the cells elongated.
If you just take a substrate, like even just a 2d substrate, and you put cells on it they'll be rounded to start out with.
And over time, over a few hours, they'll spread.
And that's pretty standard.
Many types of cells will do that.
So we knew the cells were starting off as rounded and they were spreading.
So the cells are getting longer, but our whole scaffolds getting shorter.
And so it wasn't obvious how was the cells going longer, but the scaffold's getting shorter.
And so the next thing we thought we would do is just watch the cells and see what they did.
And so we measured the aspect ratio of the cells at different time points.
And we did this by just impregnating the scaffold in the cells at different time points with a resin, and then using a stain, and then using digital image analysis.
So what we found was that the fiber of the fiberglass morphology looked like this.
So the long thready things of the scaffold, and these little blobs here are the fiberblast of the cells.
So here at time 0 you can see-- like I said, the cells are pretty rounded they're not very spread out.
Here at eight hours you can see-- here's a cell that's gotten longer.
Here's another one.
This guy here is still rounded, it's not doing much.
22 hours, again, some of the cells are quite elongated.
Some of them are still not that elongated.
So they don't all become active.
But one of the things we noticed, if you look at this image here, you can see these cells are attached at one end, and at the other end.
But they're not attached in the middle.
There's sort of a gap between the cell and the strut.
And this is another example here.
Here's a cell here, and this is the collagen GAG strut that it's attached to.
And you can see it's attached to the two ends, but not in the middle.
And this starts to explain how it is that the cells are elongating but the scaffolds getting shorter.
It's that the cells are just attached at two ends.
And the cells are moving along a strut and they're attached to the two ends.
And if you think of the cells attached through those focal adhesion points, they're applying tension to the cell.
And the actin filaments in the cell are in tension.
Obviously, filaments can't be in compression.
They're only going to be in tension.
And what happens is that puts the stress into compression.
And if the struts in compression, at some point it's going to buckle.
And you can see this strut here has basically buckled under that cell.
And so if the cells are getting longer, and they're buckling the struts, then that's going to shorten the struts and the whole scaffold is going to get shorter.
And so then Toby plotted the aspect ratio of the cell, so that is a measure of their elongation against the time.
And again, he found one of these curves with the same kind of form as the curve for the forced development.
And he found the time constant here for the change in the aspect ratio was about five hours.
And for the development of the force it was about 5.7 hours.
So the time constant for the elongation of the cells, more or less matches up with a time constant for developing the force.
So that's what that says.
And that suggests there's a link between the elongation of the cell population and the macroscopic contraction of the population.
So then we wanted to take it one step further.
And we wanted to look at what the cells were doing live.
Like as they were doing it.
So Toby devised this little schematic thing here.
So he had just an optical microscope.
He had a microscope slide with a fairly thick well in it, so that we could put culture medium in the well.
We put a cell seeded matrix in here.
And he had a heated stage here.
And then he took little videos of what the cells were doing.
And this required some patience because as you could see not all the cells did anything.
Some of them just sat there and did nothing.
So he would set this up for a day, and watch a cell, and it would do nothing.
And then he would have to find another cell.
But he did find some cells that were responsible for the contraction.
And that was it was kind of neat.
So here's the scaffold again.
All these little bits here are the scaffold.
This is a strut of the scaffold.
And this is a fiberblast parked on the scaffold.
And this has a little video here.
And you can see what's happening is the strut here is starting to buckle.
And you can see these two sides here, those two things are coming closer together.
So they originally were this piece here, and that piece there.
And now they're at that point there.
And then if I let it go a little bit longer, it continues to do that process.
And then the final thing-- this kind of smushed up mess here is these two things having me brought completely together.
And this strut here is some strut down over here.
So you can see how the cells are elongating and causing contraction of the scaffold.
Here's a series of stills taken from another video that he did.
So this sort of square thing is the scaffold.
So b is the scaffold.
And a, this little blob here, is the fiberblast.
And you can see, even from this image to this one, you can see that the fiberblast has spread a little.
Do you see how it's kind of oozed out along the scaffold there.
And eventually it attaches over here.
And you can see that it's buckled this strut underneath it.
And here it's a little bit more deformed.
It then grabs on down here somewhere and deforms it even more.
So you can see that's more deformed.
And then Toby put alcohol on the whole thing, which kills the cells and the cell let's go.
And you can see you recover some of the deformation.
You don't recover all of it, but you recover some of it.
This was another example.
And this was kind of interesting.
Here there was a scaffold junction where there were three struts that came together, a little bit like a strut.
And there was a little cell right there.
And you can see the cell elongates.
You see how this elongated and its grabbing on up here somewhere.
But the amount of force the cell was kind of pulling with must have been less than the-- or rather must been more than the force of the focal adhesion.
Because what happens was eventually the focal adhesion let go.
And the cell kind of bounces back and ends up over here.
So the cell was kind of snapped back on to the other focal adhesion over here.
And here it's rounded again.
And here it elongates again.
And then this focal adhesion lets go and now it's moved back over to there.
So these struts here are so stiff.
They're much stiffer, I think, partly because they're triangulated.
And it looks like they're just shorter and a lot thicker.
The cell isn't being able to deform those.
But it's elongating and then focal adhesion was letting go.
So this is a little schematic of what we thinks going on.
So the cell starts out-- it's some elongation here.
It's attached at that point.
It's attached at that point there.
And the cell is getting longer.
And if you think about it as the cell's getting longer-- if you think about the Euler Buckling formula, the buckling load goes as 1 over l squared.
So the longer the length of this piece of the strut of the scaffold underneath the cell is, the smaller the load it takes to actually cause it to buckle.
So at some point it buckles like this.
And this is just a little force diagram.
So the actin fibers are in tension and the matrix strut is in compression.
Sometimes we saw some bending.
So you could see if a cell was spanning between two struts, you could get the cell bending the struts as well.
That was another possibility.
And so we think that the cell elongation was related to the contraction.
The time constants for the two things were almost the same.
And as the cell elongates there's a gap between the cell and the matrix on the central portion.
And then the cell is adhered at the periphery of the adhesion points.
And then the tensile forces in these act.
And filaments inside the cell induce compression in the strut, and that causes buckling.
And then Toby graduated.
And then I got another student, Brendan.
And Brendan saw what Toby did and he wanted to do a little more with that.
Brandon was also involved that osteochondral project that I talked about last time.
And Brendan this other thing as well for his project.
So he wanted to measure the force of an individual cell.
So when we had that cell force monitor, that was the total force of all the cells in that one direction.
But Brendan wanted to know if he could measure the force of a single cell.
And now that we knew that the contractal process was related to buckling, We thought, well, we could just use Euler's formula.
If we knew what the modulus of the solid was, and we knew what the dimensions of the struts were.
So that would allow us to calculate the contractile force of a single fiberblast.
So I think I've shown you this thing here.
So Brendan was the one who did these experiments.
He cut a single strut out of the scaffold.
And the single strut is about 100 microns long.
He used a microscope to do this.
He then glued it onto a glass slide and he used the atomic force microscope probe to bend the strut like a cantilever beam.
And he measured this displacement here.
And from that he could back out what the modulus of the solid was.
He did these tests in the dry state.
But we could extrapolate to the wet state from looking at the behavior of the whole scaffold.
So he had a modulus for the wet scaffold solid.
And then this is our formula for Euler buckling here.
So that's just the standard formula.
I had a student from civil engineering, who looked at hydrostatic loading of a tetrakaidecahedral cell and he looked at buckling.
If you had a tetrakaidecahedral cell and you load it in all three directions.
He looked at the buckling.
And he had calculated that the n constraint factor-- the n squared was point 0.34.
So we have some idea of what that n squared value should be.
Although it's somewhat of an estimate.
I had a UROP student who took Toby's images and measured the dimensions of the struts.
So he measured the diameter and the thickness of the struts.
And from that, we just plugged everything into the Euler formula.
And we found that the average single cell force is somewhere between about 11 and 41 nano neutron.
It was something like 26 nano neutrons.
So it would make sense that it's more than the one nano neutron per cell because not all of those cells were active and they weren't all going in the same direction.
So Brendan Harley and Matt Wong did that part of the project.
OK.
So that's the contraction.
Are we good with contraction?
So it's kind of interesting that cells will contract and we can measure some forces.
So the next type of interaction between the cells and the scaffolds that I wanted to talk about is cell migration.
And these are some studies from the literature.
These are two different studies.
But the top one here, they've measured migration rate as a function of the cross linking treatment of a scaffold.
And the decreasing stiffness goes this way.
And so they're seeing that the speed of migration-- this is in millimeters per day.
Cells don't move too quickly.
They go millimeters per day.
But you can see that the migration speed, the speed at which the cells can move, depends on the stiffness of the scaffold that they're attached to.
And in this study on the bottom here, what they did was they had just a flat 2d substrate.
Just a flat polymer.
And what they did was they cross linked one part of the polymer more than the other part of polymer.
So over here, this was the less cross linked.
That was the soft part.
And this was the more highly cross linked.
This was the stiffer part.
And they found that if they put a cell on the soft part it would migrate onto the stiff part.
But if they put a cell on the stiff part, it would start going this way towards the soft part.
But when it got to the interface it would just spread out along the interface.
And it wouldn't go into the soft part.
So the cells were somehow sensing the stiffness of the substrate.
And for some reason, I don't know what, but for some reason these particular cells seem to prefer being on the stiff substrate.
So this is just really showing that there's some interaction between the substrate stiffness and the way the cells are behaving and migrating.
And then Brendan also wanted to study this.
And he got some of the collagen GAG scaffold.
He made some of the scaffold.
And he stained that with a stain that made it turn red.
So these lines here are all red struts in the scaffold.
And then he put fiberblasts on to the scaffold and stained them green.
So all these little blobs here that are green are the cells.
And then he used confocal microscopy.
And the confocal microscopy allowed him to look at a certain volume of the scaffold.
And he had some software that would track the centroid of each cell as it moved through the scaffold.
And so he had a thing he called spot tracking.
So each of these little spheres here corresponds to a cell.
And the white box is the volume of material that you could see in the scaffold.
And this color scale here really corresponds to time.
So I've forgotten which round.
I think blue is the original time 0, and then red is maybe five seconds, and yellow was 10 seconds.
The different colors correspond to different times.
So he could track the path of each cell and also what the position was at different time points.
So he knew what the position was at different time points.
And obviously from that, he could get the speed of the scaffold.
And he did these experiments on scaffolds of different stiffnesses, as well as, different pore size.
And here you can see the cell speed.
He's measuring it in microns per hour now.
The cell speed increases at first and then decreases with the strut stiffness.
So we don't know exactly why this is.
But there is an effect between the stiffness of the scaffold and the migration speed.
And another thing he did was he looked at how the cell speed varies with the pore size.
And as the pore size gets smaller, the speed goes up.
And we're not entirely sure why that is.
But I think that might be related to this binding site thing too.
As the pore size goes down, the number of binding sites is going to go up.
And if you think of the cells migrating by having these adhesion sites, and the adhesion sites are just at the ends of the cells, and the cells kind of putting out a little extension, and then looking for somewhere else it can bind.
The more binding sites there are, the faster it's going to find a binding site.
And the faster, I think, it's going to move on.
So I think that the cell speed depends on pore size, at least in part because of the increase in the binding sites with smaller pore sizes.
So pore size and the migration.
And then the last thing I wanted to talk about was cell differentiation.
And this is a study study by Engler.
And one of the things he found was he put mesenchymal stem cells on 2d substrates.
Just flat 2d substrates of different stiffnesses.
And again, he could control the stiffness by cross linking.
And what he's showing up here in the first bit is that he's looking at the stiffness of tissues of different kinds.
So here's brain type tissue.
Something like one kilo pascal.
Muscle might be something like 10 kilo pascal.
And collagenous bone-- this is sort of the osteoid that is the precursor of bone, not the bone itself.
Is about 100 kilo pascals.
And what he did was he put these mesenchymal stem cells-- so here's his cell onto his substrate.
And he varied the stiffness of the substrate.
And then he looked at the shape of the cells.
So here's the least stiff substrate, so between point 1 and 1 kilo pascals.
And here's 4 hours, 24 hours, 96 hours.
And these cells formed long processes extending beyond the cell body.
And they looked kind of like neurons.
So they he called those neuron like.
Then there's an intermediate stiffness of substrate here.
And these cells became even more elongated.
And became something like a muscle cell, myoblast like.
And then cells that were put onto a substrate that was between about 25 and 40 kilo pascals, they developed a shape that was something like an osteoblast, like a bone cell.
So one of the things he was looking at here, was how the stiffness of the substrate affected how a stem cell might differentiate into different cell types.
And another thing that he did was he looked at different cell markers.
And he found that the cells were expressing markers that were corresponding to the types of tissue.
So I couldn't tell you the names of all these things and what they are.
But I think the red here is expressing more of a particular marker.
And I think these wounds were related to nerve tissue.
These wounds here, were related more to muscle tissue.
And these wounds here were related more to bone tissue.
So the things the cells were expressing also seemed to correspond to the different types of tissue that they were corresponding to.
So I'm just going to end this part by going through a little summary here.
So what I've tried to show you today is different types of cell behavior that are affected by the scaffold.
And they're affected by things like the number of binding sites, by the pore size, by the stiffness of the scaffold.
So we started with a cell attachment.
We saw that the cell attachment increases linearly with a specific surface area.
We saw that the cell morphology depends on the orientation of the pores.
And that kind of makes sense, they got to line up with the pores.
We talked about the contraction behaviors.
So the cells bind at the periphery, the cells elongate, and that causes this buckling.
And you can calculate the buckling forces.
It's around 10 to 40 nano neutrons.
We looked at the cell migration speed.
That increases with the stiffness of 1D fibers.
And we looked at cell migration in the collagen gag scaffolds.
So that depends on the stiffness of the pore size.
And then there was this final study on the cell differentiation.
So I wasn't going to write any notes on this because the slides I think pretty much explain it.
So I was just going to put the slides on the website at the end after today's lecture.
So are we good with how cells and the scaffolds of the environments interact?
Because I think it's not so obvious that this actual mechanical environment makes a difference.
People think of the chemical, the biochemical environment.
That obviously affects the cells.
But people don't think at first that something like the sort of structure of the pores, the pore size, or the orientation of the pores, or the mechanical properties are going to affect how the cells behave.
But in fact, they do.
So that's it.
And this is all various people who worked with me on these projects.
So it was a lot of fun.
OK.
So hang on a sec here.
What's this all about?
I'm going to get rid of that.
Go away.
Here we go.
OK.
So are we good with cells and substrates?
Yeah?
OK.
So let's just take a little moment and I'll rub the board off.
And then we can start the next bit.
OK. OK.
So that's the end of the medical material stuff.
So we talked about the bone.
We talked about the tissue engineering scaffolds.
And then we talked about the cell scaffold interactions.
So now we're going to go back to more engineering topics.
And the next thing I wanted to talk about was energy absorption in foams.
So foams are very widely used for energy absorption applications, things like bicycle helmets, different kinds of helmets.
You buy a new computer, it comes in foam packaging.
And the reason foams are used so much is they're extremely good at absorbing energy from impact.
And in fact, they're better than the solid that they're made from.
So let's just look at this curve here for a minute.
So here's a stress strain curve in compression for the foam.
And the material that it's made from would have the stiffness something like this.
It would be much, much stiffer than the foam.
And if you think about how much energy you can absorb, the energy you can absorb is just the area under the stress/strain curve.
That's the energy you can absorb in a given volume of foam.
And so when you're thinking about these energy absorption problems, it's not just that you need to absorb a certain energy.
You need to absorb it without exceeding a certain peak stress.
So whatever it is you're trying to protect, at some point it's going to break.
This is what you want to avoid.
You want to avoid it breaking.
So you don't want to have a stress bigger than the stress that's going to break whatever it is, your computer, or your head, or whatever.
So say you have a given peak stress that you can tolerate here.
And we've normalized things by the solid modules.
But just say that's a peak stress here.
The foam is going to absorb this amount of energy up here, this whole little shaded region.
And the solid is going to absorb that little, teeny weeny bit in there.
So what you want to do is absorb the energy without exceeding a certain peak stress.
And the foam is always going to be better than the solid that it's made from.
There's a couple other things that make the foams good because they're more or less isotropic, maybe not perfectly.
But roughly, they have the same properties in all directions.
Sometimes you don't know what direction the impact's going to come from.
And so if you've got the same properties in all directions or roughly the same, that's a good thing.
You also want the protective thing to be light.
If you're paying for shipping for your computer or whatever, the fact that the packaging is light makes the shipping easier.
If you have a helmet for your head, you don't want some big heavy thing.
You want something fairly light.
And foams are cheap.
So the fact that they're roughly isotropic, they're light, they're cheap, this all helps as well.
But from a mechanical point of view, foams are very good at absorbing energy.
And so what we're going to do in the next-- the rest of this lecture and on Wednesday-- we're going to see how we can convert these stress/strain curves into what are called energy absorption diagrams.
We're going to look at some energy absorption diagrams that we just measure from the stress/strain curves.
And we're going to look at how we can predict the energy absorption diagrams as well.
OK.
So the main idea here is that the impact protection has to absorb the energy from the impact but without exceeding a certain peak stress.
So the direction of loading may not be predictable.
And foams are good because they're roughly Isotropic.
And they would have the same energy absorption capacity from any direction.
And foams are also light and cheap.
We can say for a given peak stress the foam is always going to absorb more energy than the solid it's made from.
So other things that make foams good are that they have a capacity to undergo large deformations.
And they do that at roughly constant stress.
So that if you look at the stress strain curve for the foam, you're going to be able to absorb all this energy under here.
And these strains that the foam might go to might be 0.08 to 0.09, so huge strains on an engineering scale.
And then this is your energy-- would absorb is that area under the stress/strain curve.
So I wanted to say something about strain rates too.
So typically we're going to be talking about problems of impact.
And in impact, the strain rates are typically on the order of 10 to 100 per second, something like that.
We're not going to talk about things like blast.
If you have a blast loading, then you have to take inertial effects into account.
And blasts involves strain rates, which are 1,000 to 10,000 per second, much, much higher.
So we're going to talk about strain rates that are about 10 to 100 per second, maybe a bit more than that.
And for instance, you can roughly estimate what one of these impact rates would be.
So you had something that you dropped from a height of 1 meter.
Then the velocity on impact is just if you just equate the potential energy with a kinetic energy.
The velocity and impact is just the square root of 2gh.
So g's the gravity acceleration.
And h is the height.
So that's the square root of 2 plus 9.81 meters per second times 1 meter.
And that comes out to 4.4 meters per second.
And say you had some foam packaging that was 100 millimeters thick.
Then you could say roughly that the strain rate would be approximately equal to that velocity over the thickness, so 4.4 per second over 0.1 meters.
That' would be 44 per second.
So it's somewhere in that range.
Obviously, the thickness could be a little bit smaller, it could be bigger.
But it's in that ballpark.
And if you do tests on servo controlled instrons or you do a drop hammer test, you can get strain rates in that ballpark.
OK.
So we're talking about impact and not blast.
OK.
So most of the energy that's absorbed is really absorbed in that stress plateau.
So if you think of the stress/strain curve, most of the area under the stress/strain curve comes from the area from underneath the stress plateau.
So the mechanisms of absorbing the energy are going to be mechanisms that are associated with a plateau stress.
So for elastomeric foams, we've got elastic buckling of the cells.
And one of the advantages or disadvantages-- depending on what you want-- of this is that the deformation is recoverable and you got to have rebounds.
So if you have an object and you drop it onto elastomeric foam, it's going to bounce around like that.
So the elastic deformation is going to be recovered, and you're going to get rebound.
If you have a foam that has a plastic yield point or is brittle, then the deformation is going to be largely from dissipating plastic work or work of fracture.
And in that case, there's no rebound.
But once you've loaded it, you've crushed the thing, and you've permanently deformed it, and you can't use it again.
So sometimes if you ride your bicycle like I do, if you have a helmet, you should wear your bicycle helmet.
If you have a problem, if you have an accident, and your helmet get smooshed, that's it.
You have to throw your helmet away.
You can't use it again.
And this is why.
[INAUDIBLE], even if it doesn't get smooshed, if you hit your head at all, [INAUDIBLE].
Exactly.
[INAUDIBLE] Yeah.
You need a new helmet.
Yeah.
Go ahead.
Talk about helmets because I'm on a helmet conversion thing.
Yes.
You've got to wear your helmet.
And you should change it every now and then.
Anything else you'd like to add about bicycle helmet safety?
No, absolutely.
You've got to wear your helmet.
So I know several people who would have had their head smooshed had they not been wearing their helmet.
So you have to wear your helmet.
Let's see.
OK.
If you think about natural cellular materials, things like wood, they often have cell walls that are fiber compensates.
And you can dissipate energy by mechanisms related to the fiber nature, so by things like fiber pull out fracture.
And then you can also have open cell foams with fluids.
You can have fluid within the cells.
And if the cells are open cells, the fluid effect is really only going to be important if the cells are extremely small or the fluid is particularly viscous, or the strain rates are very high.
So in most cases, the fluid effects aren't important in open cell foams.
But, for example, you could try to make an open cell foam that had more energy absorption by putting a fluid into it.
So you could put glycerin into the fluid, and that would increase how much energy it would absorb.
Or, you can put this honey into it.
That would make it more energy absorption.
And enclosed cell foams, you may have an effect of the gas within the cells.
But it's really only going to be significant if you have elastimeric foams where the cell faces don't rupture.
The cell faces rupture, then the gas is just going to flow out of them, and that's not going to do much.
So the next step is I want to go from having the stress/strain curve that we've become very familiar with, and make something with that that is a little easier to see graphically that shows how much energy we can absorb.
Remember, what I said what we're really interested in is absorbing a certain amount of energy without exceeding a certain peak stress.
So what I'm going to do is plot another plot that's based on that.
It's going to be the energy absorbed.
So w is going to be energy absorbed per unit volume.
And I'm going to plot that against the peak stress.
OK.
So we're going to look at three different regimes here.
We're going to look at what happens in the linear elastic part, what happens in the stress plateau, and then what happens in the densification part.
So let's think about the elastic regime first.
And if I moved up-- say I moved up to some point right there where the little x is on the stress/strain curve.
Then the amount of energy I absorbed would just be equal to this little bit here.
And if I moved up, and then the peak stress would be this peak stress there.
We'll call that sigma p1 and w1.
And if I moved up over here, I'd be at w2.
And that would be sigma p2, right?
And if I know the modulus, I know what that relationship is.
And I get a relationship.
And these are going to be-- I'm going to do this on log scales here.
There's going to be log, and that's going to be log.
I'm going to get in that linear elastic regime.
The energy is going to go as the peak stress squared over 2 times the modulus of the foam.
Remember, energy is a half stress times strain.
And I can say strain is sigma p over e. So it's 1/2 sigma p squared over e. So on my log1 plot here, this is just going to be a straight line like that.
And then I'm going to get to this value here.
I'm going to get to my collapse stress here.
So let's call that single star.
And at that point, the more I go along here, every point I go along, like that, I'm going to absorb more and more energy.
But the stress isn't going to go up at all.
So then this thing here is going to go like that because I'm absorbing more and more energy.
But the stress just stays the same.
So this is good news if we want to absorb energy.
And then once I get to the densification point, then it's going to do the opposite thing.
As I go along here, at each increment I'm not absorbing that much more energy.
But the stress is going up.
So at some point it turns and starts to look like that.
So this part here corresponds to linear elasticity.
This bit here corresponds to the stress plateau.
And this bit here corresponds to densification.
And the point where I would like to be is right here, because here I'm going to absorb the most energy possible through the peak stress.
So you can think of that as sort of an optimal point.
And I'm going to refer to that as a shoulder because it's the shoulder between where the curve bends over again.
So I've only got a couple minutes left.
But let me just show you one thing and then we'll talk about this more next time.
So I've just done this for one relative density.
But if you look at the screen, you can imagine I would have stress/strain curves for lots of different relative densities.
And let's say these are all at the same temperature and all at the same strain rate.
And I could draw a curve that looks like that for each stress/strain curve.
And if I did that, I'd get a family of them.
So this is our energy absorbed here.
I've normalized it by dividing by the solid modulus.
This is our peak stress here.
And I've normalized that by dividing by a solid modulus.
And I've got a sort of family of these things, right?
They all have the same shape.
But they shift depending on the relative density.
And then the thing that makes life good is that these shoulder points you can connect with a line.
And you can mark off the relative density for those shoulder points on each line.
And then the last step you can do is you can just plot these lines.
And you can repeat this for different strain rates.
So this would be a family of these guys here.
There's a family of those lines at different strain rates.
And then you would join up the points that correspond to each relative density.
So you can make a drawing that looks like this that summarizes the most energy you can absorb for a certain peak stress for foams of different relative densities tested at different strain rates.
You could do it for different temperatures if you wanted to.
So next time, we'll talk about that.
But I'm going to stop there for today.
OK?
Are we good?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so last time we started talking about energy absorption, and I wanted to try to finish that up today.
And then next week we would talk about sandwich panels and using honeycombs and foams in sandwich panels.
So I think we got as far as the idea of introducing what these energy absorption diagrams are.
So let me run through this little sequence again, and then I'll put the notes up on the board.
So the idea is that you have your compressive stress strain curve-- so here's a series of curves for different densities of a foam.
And we would do these all at the same strain rate and temperature, so that those aren't variables.
And then what we would do is we could turn those into energy absorption diagrams.
So notice here, this is a log plot.
So here's that energy absorption here.
Here's the peak stress here-- so that's the peak stress up to some level of energy that you've absorbed.
And we've normalized both of those by the solid modulus.
So say we look at one density here, say we look at the lowest density-- 0.01.
So here's our curve here for the test.
If I go up to some stress level that's still in the linear elastic region, then that's going to translate into somewhere along here on the energy absorption curve.
And one of the things we're going to do today-- I'm going to show you how you draw these, and how you can set them up.
And you can either get them from experiments-- so this is, say the top curve were experiments, doing it from experiments-- or you can use the models for the foams to do it, as well.
So we'll see how you can do from both ways.
So this would be the linear elastic bit here.
This vertical part-- where the energy is increasing, but the peak stress isn't increasing-- that corresponds to the plateau here.
And then this part here, where the energy is not increasing very much, but the stress increases a lot-- that corresponds to the densification part over there.
So what you would do is you would do tests on foams of different densities, and from the test data, you could draw an energy absorption curve for each density.
So you plot-- there's four different densities here, so it forms a family of these curves.
And then what you do is you say-- and I think we talked with this last time-- that the place you want to be is at this shoulder point.
You want to be absorbing as much energy as possible at that plateau stress.
So you want to be at the point just before it turns around to the densification regime.
So you can mark that little shoulder point for each density.
So here's like 0.01, here's 0.03, and so on.
And then those points can be connected by a line-- so that heavy line then connects those shoulder points, or those optimum points.
And then what you can do is then repeat this whole process for different strain rates.
So this family of lines here is at different strain rates-- so this set that goes this way, corresponds to this line here.
And if you do them at different strain rates, where the shoulder appears at a slightly different point-- and so if you mark those shorter points, you can draw these lines here that connect up for one relative density.
So this line on the left hand side here, these are all relative densities of 0.01, these ones are all relative densities of 0.03.
So this diagram down here, doesn't really look like this at all-- it doesn't look like the basic energy absorption diagram.
But it actually has information for what the optimum would be for a range of densities and a range of strain rates.
So typically, these foams are viscoelastic and they have some strain rate sensitivity.
So you'd like to get the strain rate sensitivity into it.
And it's not shown here, but you could do the same thing at a constant strain rate and varying the temperature, too.
So if you had things at different temperatures, you could do the same kind of idea.
OK-- so are we good with how this works?
Because now I'm going to write some notes on the board so you have it in your notes.
So the idea is that you turn your stress strain curve to look something like that, into an energy absorption diagram.
And you can plot on log log scales, the energy versus the peak stress.
And you get something like that.
And this point here, I'm going to call the shoulder point.
And that would use the material in the most efficient way-- you get the most energy absorption without getting higher than that plateau stress.
So we can say at this stress plateau, the energy increases without much increase in the peak stress.
And then as the foam densifies, then you get an increase in that peak stress, with little increase in the energy absorbed.
And so ideally, you want to be at that shoulder point.
So to construct these energy diagrams, you can test the series of foams at different relative densities and constant strain rate and temperature.
And then you make that plot of the energy absorbed normalized by the solid modulus, versus the peak stress normalized by the solid modulus for each curve.
And typically what people do is they would take the solid modulus at a constant strain rate and temperature, so you don't have to introduce that, as well.
Then you would mark the density for each of those shoulder points, and then you would connect them.
And then you could repeat this whole process for different strain rates.
And then you would draw the final diagram at the bottom, where you have this family of lines that describes the shoulder points for different densities and different strain rates.
And you could treat different temperatures in the same way, if you wanted to.
You would hold the strain rate constant, and vary the temperature.
so it's kind of a nice way of just putting a lot of information in one diagram.
So one of the things about this is that, because we're normalizing by Es, and if you think of elastomeric foams, elastomeric foams, both the Young's modulus depends on Es, and the plateau stress depends Es.
So the Young's modulus depends on the stiffness of the solid, and also because the plateau stress is related to elastic buckling, it also depends on the modulus of the solid.
So for elastomeric foams-- it's because you normalized it with respect to Es-- one of these diagrams will represent all elastomeric foams.
So that's rather a nice thing-- so you can have different elastomeric foams, but one of those diagrams is going to represent all of them.
So we can say, elastomeric foams can all be plotted on one plot, or one curve, since both the modulus and the plateau stress are related to Es.
So if we look at this next figure here-- maybe I'll just wait a minute for people to stop writing.
Some people write faster than others.
So if we look at this next plot here, here's a compressive stress and strain.
These tests are done for one density, but at different strain rates-- so it's kind of the other version of this.
But here's the stress strain curves here, and here's the energy absorption diagram that's derived from those.
And then here's a summary diagram that has the different strain rates, and that would have different densities here.
And the idea is this diagram here could represent all elastomeric foams.
So this has been put together for polyurethane, but it should be able to represent other sorts of flexible elastomeric foams.
Sorry?
In those diagrams, then, the intersection of this strain rate line and your relative density line should be the shoulder position?
Yeah, exactly.
And so this line here is for 0.01, and that one's for 0.03.
So 0.02 is going to be-- you'd have to interpolate somewhere in between there.
So we've just put on certain values, because we're not going to put on a million values.
We just put on certain ones, and then you can estimate where other densities would appear on there.
OK?
So here's another example here-- these are curves for two different foams.
So here's a polyurethane a polyethylene, so these are both elastomers.
And one is the dashed line, and one is the solid line.
And you can kind of see how the lines mesh up.
So here's a density of 0.01 for the polyurethane.
Here's 0.05 for the polyurethane.
And here's 0.06 for the polyethylene.
And you can see how the 0.06 and 0.05-- they're not quite on top of each other, but they're pretty close to coming on top of each other.
And then 0.1, 0.12-- and so you get a family of them for the different densities.
And you can also do this for materials that have a yield point, so polymethacrylimid has a yield point, so you do exactly the same kind of thing.
So here's the energy absorption diagram that's been developed from the stress strain curves.
But now this curve here, or this set of curves here, is really just valid for one ratio of sigma [?
ys ?]
to Es-- so the solid yield strength of the solid modulus.
So it's valid for whatever it was for that particular type of foam-- the polymethacrylimid.
So I'll just say, if we have foams that are made from a material with a yield point-- and so they have a plastic collapse stress-- then the curve will be valid for foams with the same ratio of [?
sigma ys ?]
to Es.
So in that case there, for the polymethacrylimid, that ratio is about equal to 1 over 30.
So that plot would probably give not a bad description of other foams, with the same value of sigma ys over Es.
So the idea here is we can generate these diagrams either from data-- the way those ones have been done-- or from the models.
So another way to generate these is to think about the models that we have for the foams, and the foam behavior.
So we have an equation that describes the Young's modulus, we have equations that describe the plateau stresses, we have an empirical equation for the densification strain.
And we can use those to generate these diagrams.
And they're kind of useful, because they show you what's going on a sort of mechanistic basis.
So this is a diagram here that's been generated for open cell elastomeric foams.
Here's our energy absorbed, here's our peak stress down here.
This little inset is sort of a schematic of the idealized foam behavior.
So here's the Young's modulus, here's the elastic [?
collapse ?]
stress, here's the densification over here.
So obviously it's kind of a very idealized set-up.
But we can generate this diagram-- and I'm going to go through the equations that will let us do that.
So one thing to note is here's the curves for each density.
Here is this line that connects the shoulder points.
There's a couple of other lines on here that I just wanted to mention something about.
If you think about the densities-- like that's 0.01, this is 0.03, that's 0.1-- if you had a fully dense solid that was made of the same elastomer, you could plot the curve for that, and that is going to show up over here.
So this is kind of an upper bound.
It can't get any more from that.
And we've also got a dotted line here, which takes into account fluid flow within the cells.
So there can be some fluid flow, and because we haven't talked about that, we're not going to go into that.
So we can just ignore that dotted line for now.
So let me go through how we can do the modeling.
So we're going to divide the stress strain curve up into bits, and I'm going to write equations for the energy absorbed for each bit.
So we're going to start with a linear elastic behavior.
And I'm going to-- let's see-- yeah, so let me just note here that this is the densification strain out here.
And this strain, epsilon naught, corresponds to the strain at which we first reach the stress plateau.
So here, for the linear elastic part, I'm going to say that the strain is less than that.
So the strain is less than absolute naught.
And then I can say, the energy absorbed-- so if you just remember from Hooke's law on linear elasticity, energy under the stress strain curve for the linear elastic part is 1/2 of sigma squared over E. So I'm going to call sigma-- whatever the stress is going to be, the peak stress that we get to.
And now we're going to divide by E of the foam-- because these were our foams here.
And I can use our model here to say-- and now I've got to divide that by Es, because I've normalized here.
So because I know from my modeling that the Young's modulus of the foam is just equal to Es times the relative density squared for the open celled foam, that means I've now got an Es squared in the denominator, so I've got a sigma p over Es squared.
And then I've got a 1 over the relative density squared term.
So that factor there, that equation there, gives you these first set of lines here.
Gives you that bit, and this bit, and that bit.
So it gives you those first parts of the energy absorption diagram.
And then if we look at the stress plateau-- so here we're going to say that epsilon naught is less than epsilon is less than the densification strain-- so we're on the plateau somewhere.
And now the energy absorbed is just going to be our plateau stress times the amount of strain we've got.
So that's epsilon minus epsilon naught.
And if I normalized with respect the solid modulus, I can write down that my plateau stress is 0.05 times the relative density squared, and then multiply that times epsilon minus epsilon naught.
So that equation then corresponds to these vertical parts-- so this part here, that part there, this part here.
It corresponds to those vertical lines on the figure.
Vertical lines on the diagram.
And then the plateau stress is going to end at the densification strain.
And at that point, the energy diagram is just going to become vertical.
OK, and then the last part-- I'll try to rub this off a little better.
And then the last part is when we're at the end of this stress plateau, and the strain is equal to that densification strain.
And so the amount of energy we absorb here is really going to be the maximum.
So this is the energy that's going to correspond to that shoulder point that I've been talking about.
So I'm going to call that W max, and normalize that with respect to Es.
And that's then going to be our plateau stress times the densification strain.
And the densification strain was just 1 minus 1.4 times the relative density.
So here I'm assuming that the densification strain is very much bigger than the strain at which the buckling first occurs, and I'm neglecting that linear elastic part.
So then we could say that the optimum foam is at that shoulder point.
And I can say that the peak stress at that point is just equal to the plateau stress.
And what I want to do is get an equation for that solid line up here that connects all those shoulder points.
So I want an equation, in terms of the energy, and the peak stress, instead of in terms of the density.
So what I'm going to do a solve this for the density, and then plug that back into there.
So I get that the relative density is, then, 20 times the peak stress over the solid modulus, and I take the square root of that.
And now I can substitute this up here for the relative density.
I think I need another board.
So then I've got-- this is my peak stress, or my plateau stress, over Es, and this is all just the densification strain.
And that's just 1 minus 1.4 times the relative density.
But now I'm putting the relative density in terms of the peak stress, or the plateau stress.
If I just simplify that slightly with the constant, it's 1 minus 6.26 times sigma p over Es to the 1/2 power.
So that equation there describes the-- oops, no more updates.
No updates.
Go away.
Ah, so that equation there describes this line here that connects those shoulder points.
And that's the line you're the most interested in, because each of those shoulder points is a point where the foam is being used in the most efficient way, or the optimum way.
And then, let's see-- is that going to fit?
No-- let me try the other board.
And then the last thing we can do is calculate that line that corresponds to the dense solid.
Yeah?
On those ones over there, it says it corresponds to the vertical lines on the diagram.
And then later it says then it becomes vertical.
Is that referring to two different diagrams?
So this stress plateau equation here corresponds to these vertical lines here.
So for relative density of 0.01, it corresponds to that part.
For 0.03 it's this part.
And your 0.01 it's that part
But then some of the [INAUDIBLE] it says, then w [?
versus ?]
sigma becomes vertical.
Should that--
So, well the plateau stress ends at the densification strain.
So the plateau stress ends here.
Oh, let's see-- and then it becomes-- should be horizontal.
Sorry.
OK, sorry.
Happy?
And now I'm going to rub that all off.
OK, did everybody get this?
I can rub it off?
OK, so then the last part is what happens when the foam is densified.
And if it was fully dense-- and you never really can get to this point-- but if it was fully dense, you would get rid of all the pores, and it would just be a solid.
And then the energy absorption curve would be the curve for the solid.
So I'll just say, when fully densified, I'm going to say a curve approaches that for the solid.
And for the solid, you would just have that W over Es is equal to 1/2 the peak stress squared over Es.
So this model curve, the curves have the same shape as when you get the diagrams from the experiments.
And you can see how the different mechanisms of deformation and failure contribute to the diagram, where the diagram comes from.
And I guess one other point is you can see that the foams are always going to be a lot better than the solid.
And remember this is a log log curve, so that a foam that has a density of 3% here, there's a huge difference in the peak stress.
So say you wanted to absorb this amount of energy up here, for a foam that was 0.03 dense, the peak stress would be a little less than 10 to the minus 4, normalized by the modulus.
And for the solid, it would be 10 to the minus 2-- so it's orders of magnitude better to have the foam rather than the solid.
All right, now let's see what else we have.
So we could do a similar thing for closed-cell foams, and you get diagrams that look like this.
One of the differences with the closed-cell foams, if you assume that the faces don't rupture, the plateau stresses and horizontals-- remember we had that gas contribution, and you can take that into account?
So I'm not going to go over the details of that.
The next one I wanted to talk about was looking at foams that have a yield point.
And again, you can generate a similar kind of diagram, but now, instead of having an elastic failure here, you've got a plastic failure-- you form plastic hinges.
And then again, this diagram is less general than the one for elastomeric foams, so this diagram would be valid for whatever ratio of sigma ys over Es you've the calculation for.
So this one here is for 0.01.
So let me just run through the same kind of calculation for the plastic foams.
Can I rub this off, and then I can use this board to start here?
OK, so the linear elastic part is just the same as for the elastomeric foams.
So you get W over Es is 1/2.
Sigma p over Es squared times 1 over the relative density squared, so it's just the same thing.
And the stress plateau-- you get w over Es is just the plastic collapse strength times the strain range that you go up to.
So if you remember the plastic collapse strength was 0.3 sigma ys times the relative density to the 3/2 power, and then times the strain range.
And then at the end of the stress plateau, you've got the maximum energy absorbed.
So normalize that by Es.
And that's going to be your peak stress over Es times the densification strain again.
So this bit here is the densification strain.
And then you can do a similar thing to figure out the equation of that line that joins the shoulder points.
So the first step is to solve for the relative density there.
So if this part here-- 0.3 sigma ys times the relative density to the 3/2 power is the plastic collapse stress, then at the densification point, the relative density, you just rewrite that and it comes out to the 2/3 power, because you turn the power around.
And then you just substitute this up in here.
So you get that the maximum energy absorbed, normalized by the solid modulus, is your peak stress, times 1 minus 1.4 times this thing in brackets to the 2/3 power-- the 3.3 times the peak stress over the yield strength of the solid.
And then if I just rearrange that, and get the constants, it's 1 minus 3.1 times our ratio of the stresses there.
So maybe I'll just put over here-- the curves are less general than for elastomeric foams.
So each family of curves would be for a particular ratio of the solid yield strength to the solid modulus.
So you get the idea?
It's fairly straightforward.
So I wanted to finish up this topic by giving you a few examples of how you can use these curves.
So the next thing is to look at the selection of foams for impact protection.
And typically, you're given some information about the objects-- so typically you want to protect some object.
It could be a computer, it could be your head, some part of your body.
So typically you know something about the object you want to protect.
So you might know its mass, you might know the area of contact, you might say, well, if it's my computer, I want to make sure it doesn't break if I drop it from a height of a meter.
Or you could say whatever, [?
maybe it's ?]
2 meters.
But you pick something.
And so there's a certain amount of energy you know that you need to absorb.
And you may know that whatever the component is, or the body part, or whatever-- there's some maximum acceleration you can tolerate.
So you might say, well, I want to make sure my computer doesn't break under acceleration of so much.
So you're given the acceleration.
And so if you know the mass and the acceleration, and you know the area of contact, you can figure out a force over an area, and that gives you the peak stress.
So typically, you know those things in the problem.
And typically the problem involves choosing a material, or choosing-- like choosing what kind of material do you want to make the foam out of, and what density of foam do you want to use, what thickness of foam do you want to use?
So I've got a couple of examples just to show you how this works.
So let me just write down a couple notes, and then the rest of it I think I'm just going to take from the slides.
So typically, you know what it is you want to protect, and you know something about it.
So if you know what it is, typically you know what the mass is.
You might know what the contact area would be.
Say a maximum drop height, maximum tolerable acceleration.
So say if you're worried about brain injury-- and making a helmet-- you might know what the maximum tolerable acceleration would be.
So typically, you know what the peak allowable stress is, just from whatever the object itself is.
And the variables that you have to play around with-- variables-- are things like the foam material, the foam density, and the foam thickness.
So I have a couple of examples that have different setups here.
And we'll just see how the thing works out here.
So the first example, we're told the mass of the packaged object is 1/2 kilogram.
We're told the area of contact between the foam and the object is going to be point.
0.01 of 1 meter squared.
And we're told that it's supposed to be designed to withstand a drop of 1 meter-- so if the drop height is 1 meter, then the velocity is just the square root of 2gh.
So g is gravity, so you can work out the velocity on impact would be 4.5 meters per second.
And if you have the velocity on impact and the mass, you can figure out the energy to be absorbed.
You can say mv squared screwed over 2, or you could say it's mgh-- either way.
So here it's going to work out to 5 joules.
And in this case, we're told that the maximum deceleration is 10g-- so then if that's 10g, the maximum force is the mass times the acceleration.
That works out to 50 Newtons.
And then that gives us a peak stress, the maximum allowable peak stress of the force over the area it's 5 kiloNewtons per meter squared.
And in this case, we're told that the foam is going to be a flexible polyurethane, and it has a solid modulus of 50 megapascals.
And so we can calculate this normalized peak stress.
So the normalized peak stress here is 10 to the minus 4.
So in this problem here, we need to figure out what's the foam density, and what's the thickness of the foam to protect the object?
And so that last slide is just summarized here.
And I realized that when I was going to put this up, the font is going to be kind of small, so I blew it up a little.
So we have figured out that we're at sigma p over Es-- the peak stress normalized by the solid modulus is 10 to the minus 4.
And we know that we're going to use a flexible elastomeric polyurethane, so we pull out our diagram for elastomeric foams.
And this sort of hashed band here corresponds to all the different strain rates, and all the different densities.
So I haven't plotted each individual line, we've just got this band that represents the whole thing.
So we know that our normalized peak stress is going to be somewhere along this line here.
That's from everything that's given, and what we can calculate.
And we want to know what density of foam to use, and what thickness.
So the way you approach this is the thickness is going to affect the strain rate.
So the strain rate's just going to be the velocity on impact, divided by the thickness-- or it's an approximation for the strain rate.
So we don't know if we're at this point down here, or if we're at this point up there, because we don't know where we are in the strain rate end of things.
So the way you solve this is you just guess a thickness.
And if you guess a thickness, you can calculate a strain rate.
Then if you calculate the strain rate, you know where in this band you are.
You can figure out a value of W over Es.
And you can use an iterative process.
And the way this is set up is we've chosen two very different initial thicknesses, and the point of doing that is to show you that it converges very quickly.
So the first iteration on this side here, we've chosen the thickness of a meter, which is probably unlikely that we need a meter of foam.
And on the side here, we've chosen a millimeter-- 0.001 meters.
So probably, we need more than that.
So we probably need to be somewhere between those two bounds.
So if the thickness was a meter, then the strain rate turns out to be 4.5 per second.
And then we know where we are in this diagram, and we can read off a value of W over Es.
So we know we're on this line here, and for a particular strain rate, we can read off the W over Es.
So here's the value-- 5.25 times 10 to the minus 5.
And if we know Es, which we do, we can then calculate the actual energy absorbed per unit volume.
We get W-- so W is [?
2620 ?]
joules per cubic meter.
And we can use that value-- because that's an energy per unit volume-- we know the area of contact, and we can use that to get another value of the thickness.
So we use that to calculate the next iteration of the thickness.
So U is the total energy in joules, and W is the energy per unit volume.
So U, the energy in joules, is going to equal the energy unit volume times the area times the thickness.
So we know what the area is, too.
We can then calculate a new thickness.
So now our new thickness is 0.19 meters.
We can use that to get a new strain rate-- that's 24 per second, and go through the whole thing again.
And we end up with a revised energy of 3,300 joules per cubic meter.
Now if we started at the other end, if we started with the first guess was a millimeter, then the strain rate is 4.5 times 10 to the minus 3 per second.
That gives us a different value that we read off here for W over Es, and a different value for W. And then we use this value here for W to make another guess for the thickness.
So that value is 0.14.
And then we go through the whole thing again-- we get a revised strain rate a revised W over Es, and a revised W. And you can see after just two iterations, these two things are almost exactly the same.
And then on the third iteration, they both would give you a thickness of point 0.15 meters.
So even though you can pick wildly wrong first iterations, it converges very quickly, and it's a fairly simple calculation to do.
So you know the thickness that you want is in here, and then you can get the optimum density here.
So you know that your sigma p over Es is along this line of 10 to the minus 4--we're somewhere in here.
These two strain rates here-- one was 24, one was 32-- so the final value is going to be somewhere around 30.
And if you look on this thing here, you can see there's a line that corresponds to 10, there's a line that corresponds to 100.
We're going to be right around in there.
So the relative density is going to be right around 0.01.
So you can use the diagram to get the thickness and to get the density.
OK, are we good?
We're good?
So I've written some notes, and I'll just scan those and I'll put them in the Stellar site.
So here's another example here-- and in this example, it's set up a little bit differently.
So in this example here, we're not told the material, but we're told the thickness of the foam.
So this time we want to get the material, and we want to get the density of the foam.
So here we've got the specification.
We're told the mass is 2 and 1/2 kilograms.
The area of contact is 0.025 meters squared.
We're told the thickness-- here thickness is 20 millimeters.
We've got a drop height of 1 meter again.
And the velocity of impact is then going to be 4.5 meters per second again.
And since we know T, we know that the strain rate is going to be around 225 per second.
We can calculate the energy absorbed-- [?
MGH-- ?]
25 joules.
We can calculate the energy absorbed per unit volume, because we've got the area and the thickness-- so I'm just going to divide that by the area and the thickness.
Now we have 5 times 10 to the 4th joules per cubic meter.
And we're told that we're supposed to design it so the package can withstand a deceleration of 100g-- and so that gives you a maximum force.
And here we've got a maximum allowable peak stress of 10 of the 5 Newtons per meter squared.
So here we have our curve for the elastomeric foams again-- let's assume it's going to be an elastomeric foam, but we don't know what kind of foam.
So the way you solve this problem is that you make a guess for what Es is.
So remember, these diagrams are all normalized by Es.
So to plot a point on there, we need to know what Es is.
So here, we're going to make a guess, and we're going to assume that Es is 100 megapascals to start.
And if we had that value of Es, we know W, up here, and we know sigma p there.
So we just divide those values of W and sigma p by Es, and we get these two values here.
And we plot those two values on the thing here.
So here's our point A-- and that corresponds to that first guess of 100 mega Pascals for the modulus of the solid.
So that's not necessarily the final answer, that's not the right answer-- that's just somewhere to start.
So the thing to notice is, if we have that point there, the strain rate there is probably not quite right.
This upper bound here is-- let's see.
Got to get closer.
Oh, let's see-- that's 10 to the minus 2, that's 10 to the 2.
So the strain rate we want to be is closer up to here, it's not quite down there.
So we're not at the right strain rate.
But the thing to notice is that if we draw a line of slope 1-- so there's this dash line of slope 1-- when we move up and down that line, we're just changing Es.
Because everything is normalized with respect to Es-- if that line has a slope of 1, then we're just moving up and down with respect to Es.
So what we do is we scoot up the line to get to the point that's on the right strain rate.
So if this was a strain rate of 100, or around 200, we want to be at point B.
And then from point B, we can read off what's the value of W over Es, and sigma p over Es.
And from that, we can back out what the solid modulus we want is.
So these are the values that we read off the chart.
This gives us an Es of 28 mega Pascals.
And again, you can go to this more detailed diagram here, and if you read off the sigma p over Es, and the W over Es, the density you want is about 0.1.
So it tells you the modulus of the solid, and from that you can pick a solid, and it tells you the density.
Are we good?
OK, so I have one more.
So there's a slightly different way you can do it, too.
So now I want to talk about bicycle helmets-- you're my bicycle helmet person.
So this is another little case study that involves a slightly different way to do this.
And this involves a slightly different diagram.
So the idea here is to choose a material for a bicycle helmet.
And as you probably all know, the bicycle helmets have a hard shell, and they have some sort of foamy liner.
And the foam's usually around 20 millimeters thick.
And you want something that's light, because you don't want your helmet to be too heavy-- but you want something that will absorb the energy from the impact.
So I've set this up here-- so we assume the mass of the head is about 3 kilograms.
And we assume that it can withstand a deceleration of something like 300g-- and so then you can get a force mass times the acceleration, because you have the force.
And I've assumed an area of contact of something like 0.01 meters squared, so that gives you a peak stress.
So this method here is based on the idea that you have these material selection charts for foams.
Remember we talked about that earlier.
And this is the compressive stress at 25% strain.
So the idea is that axis there is meant to represent the plateau stress, and this axis here represents the densification strain.
And these dashed lines here correspond to basically a value of W-- and energy absorbed per unit volume.
So this is an energy absorbed per unit volume of 0.001 megajoules per cubic meter, 0.01, and so on.
So if you know the peak stress that you can tolerate, for the numbers I gave you it works out to be 0.9 mega Pascals-- so here it's just a little less than 1.
So that's the peak stress that you can tolerate there.
And you can see that the material that's going to absorb the most energy-- so you're absorbing more energy as you move over this way on the curve on the plot-- so the material that's going to do the best is something like an expanded polystyrene that's 5% dense.
So I've highlighted that in red-- expanded polystyrene that's 5% dense.
And then this is the densification strain here.
And you know that the lines of the energy absorption are just the stress times the strain.
So you can basically just read off from here that expanded polystyrene that was 5% dense would be a good choice for a bicycle helmet foam.
Would you like to add anything about bicycle helmet foams?
I think that is exactly what they use, yes.
So this is just another way to do this kind of thing.
And I did one more little calculation here-- if you know the thickness of the foam is, say, 20 millimeters, and we've estimated the area of the contact, you can figure out what the energy absorbed is per unit volume, and from that you can back out the energy in terms of joules, and from that you can back out the velocity that's the maximum speed that one would want to get dinged at on your bicycle.
And the maximum speed works out about 22 miles an hour.
So that's just another example.
Are we good?
OK-- yeah, exactly, your head would be hitting the ground at 22 miles an hour which, would be ouchy.
Which is why you want to wear your helmet, because your skull will not be happy if that happens.
And your brain will not be happy.
And you will not be happy.
And your mom and dad will be really, really, really unhappy.
So you don't want that to happen.
so I have a few minutes left-- and I know I've done this for the people in 3032, but I was going to buy woodpecker talk, because it's about energy absorption.
So if you don't want to watch it, if you've already seen it, you can go.
But there are some people-- you guys haven't seen it, and you haven't seen it.
There's a few people that haven't seen.
And it's cute, involves birds.
And it involves energy absorption-- so I thought I would do this woodpecker talk.
So Barry, this is all just slides.
I don't know if you want to do the lights differently.
I'm not going to write anything on the board, I'm just going to talk
Well, if it will help them see the slides better then certainly
Yeah, I think maybe we could turn the lights down a little, please
Let's try this one
Oh, there we go.
That's good.
Yeah, now I don't feel like I'm looking in the spotlight so much.
That's good.
OK, so you guys know that I like to watch birds.
And you know that I work on foams.
And if you look at bird books, sometimes the bird books say that woodpeckers can withstand the impact from pecking because they have a special material between their skulls and their brains.
And I thought, oh, well I study foams, and I'm interested in woodpeckers-- I should find out what this special material is.
So I started looking into this.
And it turns out there is no special material.
People have looked at the anatomy of woodpecker brains and skulls, and there is no special material.
But it turns out there were also a group of neurologists in the late 1970s who got interested in why woodpeckers don't get brain injury, and they took high speed video of a woodpecker pecking.
And it was kind of amazing what they found out.
So it turns out the woodpeckers can withstand incredibly high decelerations-- much higher than our brains could withstand.
And so I kind of got interested in this, and I decided to try to figure out how it works.
So here's an acorn woodpecker.
And let's see-- I got a little video here.
Wait a minute-- there we go.
There we go.
So here's a little acorn woodpecker.
These live in California.
Anybody from California?
Yeah-- have you seen them?
No, OK. Well you have not been looking carefully.
So here's our little acorn-- and you can see they peck-- oh?
Like around San Francisco
[INAUDIBLE].
[LAUGHS]
So they're pecking, and when they're pecking-- they don't just go bonk.
They do this repeatedly-- they can peck at 10 or 20 times per second.
So the question is, why don't they get brain injury?
So, first of all, why do they peck?
So as we in that little video, that woodpecker was foraging-- it was trying to get little things out of the bark.
And woodpeckers eat insects, and so the bird books say that they can actually hear insects scurrying around under the bark, and they'll peck at the bark and forage to try to get insects.
And there's one other anatomical feature of woodpeckers, which is kind of amazing-- and you can see it in this picture here.
So this is the woodpecker tongue, and the tongue is connected to something called the hyoid process.
And the hyoid process wraps around their eyeballs.
And then when they peck-- I mean, the idea is they're making a hole in the tree, and they've got to get their tongue into the hole to get the bug.
And the end of their tongue has little barbs on it.
And when they contract this thing here, their tongue scoots out and gets the little bugs.
So they pick partly to forage, but they also build something called cavity nests.
So they'll find a tree that's started to rot, and they'll drill a sort of horizontal hole, and then they'll drill a cup underneath that, and they lay the eggs and have their nest at the bottom of the cup.
And then they also-- especially at this time of year, in fact, today I heard woodpeckers drumming-- so it's one of these mating things, one of these courtship things.
So woodpeckers will peck on a hollow branch or a hollow tree, just to make a big loud noise to say, here I am, looking for sex, I'm ready, this is my territory.
And so in the spring-- so this time of year you hear woodpeckers drumming.
And I think naturally they do this on hollow trees to try to make a big sound, but they have adapted to metal downspouts, which are also very effective for making this loud noise.
And people who've-- I wrote a paper on this-- people who have read my paper sometimes email me and say, oh, I heard you know about woodpecker pecking.
How can I get them to stop drumming on my downspout?
Because it's kind of annoying, if you're the human inside the house.
So anyway, they peck for these reasons.
And then acorn woodpeckers are special-- acorn woodpeckers are what I think of as the champions of pecking.
And they do one more behavior-- so here's from David Sibley's beautiful Guide to Birds, here's the acorn woodpecker.
They store acorns in what's called a granary, and they look at old tree trunks that are beginning to rot, and they pick holes in the tree trunk and they store the acorns in the holes.
So see all those little dark dots on that trunk?
Those are all holes that the acorn woodpecker has pecked.
And they'll do this-- they live in social groups, so there might be like 10 or 20 of them living together, and they'll peck like 10,000 holes into their granary.
There will be thousands and thousands of these holes.
And here we have the acorn woodpecker with the acorn in its beak, and you can see some of these holes have acorns, and some of them are empty.
So here is the acorn woodpecker in action.
And here's the little video again-- I won't play the video.
So I often give this talk for non-engineers, so I'm going to explain some things-- there's going to be some writing on the slides that you guys already know.
So the impact force depends on the deceleration-- how quickly the brain stops when the beak hits the tree.
And I should mention, these videos are from the Cornell Lab of Ornithology-- they have an amazing collection of bird audio, like bird calls, bird songs, and bird photographs and videos.
It's incredible, the collection that they've got.
And then I explain what acceleration is-- so I'm going to put acceleration in terms of gravity.
And when the beak hits the tree, there's going to be a deceleration on impact.
And just as a comparison, human brain injury occurs roughly at about 100g.
And so the question is, how much deceleration can the woodpecker brain take?
And that's where the neurologists in the Bay Area come into the picture.
They found out that there was a park ranger-- I think maybe at Point Reyes, just north of San Francisco.
And he had an acorn woodpecker that, I don't, had an injured wing or something.
Anyway, he had this acorn woodpecker that he kept.
And they were able to use this acorn woodpecker, and they took high-speed video of the woodpecker pecking.
So from the high-speed video, the video that they took went at something like 2,000 frames a second.
So they have a picture of where the head is every 2,000th of a second.
So if you know the position at these times, you can get the velocity, and you can get the deceleration.
So they measured the deceleration, and they measured some amazing things.
So they measured that on impact, the woodpecker's bill was going something like 15 miles an hour.
And the decelerations were up to 1,500g-- so many times more than what we can withstand.
And they also measured the stopping time-- and they thought it was between about 1/500th to 1/1,000th of a second.
And that's going to be important later on.
So one of the interesting things about this was how they got this whole thing set up.
So I don't know how many of you do UROPs-- but you know, part of the thing in doing UROPs, and doing experiments in the lab is just how do you do the experiments?
So you know, the park rangers got the acorn woodpecker, that's all very good.
But they have to get the woodpecker to peck in front of a camera, and they have to get the camera to turn on as it's pecking-- so there's some experimental challenges.
So the important thing, the critical thing, is the date of the paper.
This was written in 1979, which meant they probably did the experiments in 1978.
And I was a graduate student in 1978.
And I can report there were no Apple computers in 1978-- there were no laptops.
You couldn't just kind of do your little PowerPoint slides.
And most offices had something called an IBM Selectric typewriter-- I don't know if you've ever seen the old IBM typewriters.
But the typewriters, when you typed on the typewriters, they made these noises, and it sounded kind of like a woodpecker pecking.
And the ranger had one of these typewriters in his office, and he had discovered that if he typed on the typewriter, the woodpecker thought, oh, there's another woodpecker.
I'll start pecking.
And so they used the typewriter as a way to get the woodpeckers to peck.
And I think they had some old stump, and I don't know if they put nuts or peanut butter or something into the stump to get the woodpecker to peck at it-- because they needed to peck at a particular spot, so they have the camera all set up.
So anyway, they had this whole arrangement to do this high-speed video of the woodpecker pecking.
OK, so it goes that up to 1,500g, which is kind of amazing.
And then, remember also, they do this repeatedly-- they do it at like 10 or 20 times a second.
Then this is just explaining what stress is-- but you already know what stress is, so I'm going to skip over that.
And so the way you can think about this is to think about it in terms of a scaling argument.
So imagine there's the brain and there's the skull, and when the head hits the tree, the brain's going to accelerate, and the brain and the skull are both going to decelerate.
And you can think of the stress as the force over the area.
So the force is the mass times the deceleration over the area.
So that value for the woodpecker, you can say, is roughly equal to the same values, but for the human.
So what that argument relies on is the idea that the stress to cause damage in the woodpecker brain is similar to the stress to cause damage in the human brain.
And that's not totally unreasonable.
So if you look at bone, for example, like when you measure the strength of the whale bones, it's not going to be that different from what you would measure from human bones.
So when you look at a particular type of tissue, and you look at the strength of it in different species, the properties aren't that different from one species to another.
So let's say the brain tissue gets damaged at the same stress in the two species.
So we can write this sort of equation down.
And the mass is going to depend on the density of the brain tissue times the volume-- and the volume goes as the radius cubed.
And the density is going to be the same for the woodpecker, or for the human-- so assume the brain tissue has the same kind of basic stuff.
So the mass goes as the radius cubed, and the area of contact is going to go as the radius squared.
And so there's a radius term.
So you can say the radius times the deceleration in the woodpecker should be equal to the radius times the deceleration of the human.
And obviously, the woodpecker radius is going to be a lot smaller, so the woodpecker deceleration is going to be a lot bigger.
So part of the thing is the brain is just a lot smaller.
Then there's another factor that comes into it-- these are photographs of an acorn woodpecker skull and a human skull that I got from the Museum of Comparative Zoology at Harvard.
So this is looking down on the top, and this is an elevation view.
And if you think of the brain as roughly a hemisphere-- I mean, obviously it's not a perfect hemisphere, but let's say it's roughly a hemisphere-- the orientation of the brain in the skull is slightly different in the birds and in the human.
So if you think of it as being this way on, and in the woodpecker it's turned roughly this way on, and the contact area-- think of this as the contact area.
The projected area would be a full circle, and in the human, the brain is more this way on.
And then the projected contact area is a semicircle.
So there's a factor of 2 difference, just because of the orientation of the brain.
And so I've put the factor of 2 that accounts for that.
So then I could say the deceleration that the woodpecker can take is twice the ratio of the radii of the human and the woodpecker brain, times the deceleration the human can take.
And this was around 100g, remember.
So then I wanted to know what the ratio of the sizes was, and I thought, oh geez-- I'm going to have to mess around with skulls, and try to make some sort of measurements, and this is going to be a drag.
And then I found this paper-- I couldn't believe it-- I found this paper called "Brain Size in Birds."
And this guy had table after table after table of the mass of the brain in different birds-- and he had the acorn woodpecker, lucky for me.
So if you just assume that the brain is roughly a hemisphere, if you have the mass, you can work out roughly what the radius is.
So it turns out there's a factor of 8 difference between the radii.
And so the human brain is about eight times the size of the woodpecker.
And then if you take into account this factor of 2, that means the woodpecker should be able to tolerate a deceleration of 16 times what the human can.
So remember, I said the human can take about 100-- so this would get the woodpecker up to 1,600.
But they measured 1,500.
And I'm a civil engineer, originally, I like big factors of safety-- that's a little too close for comfort for me.
And so it turns out there's one more factor that matters.
People have studied human brain injury pretty extensively, and one of the things they've looked at is how much acceleration you can tolerate without injury, relative to the duration of the impact.
So this is from a car crash conference-- so here's the tolerable acceleration, and here's the duration of the impact.
And typically, for human head impacts, the deceleration occurs over a few milliseconds-- like over 3 to 10 milliseconds.
So if this is the duration of the typical head impact for a human, here's the range of the tolerable accelerations between about 80g and 160g-- so you know, I said around 100.
So we can take this curve, and now we have this factor of 16, and we can just scale it up by our factor of 16 for the woodpecker.
So if I scale it up by the factor of 16, we're there.
But the duration of the impacts was more around 1/2 a millisecond, to a millisecond, so I've extrapolated this a little bit.
And the duration of them is up in here.
So this is saying the woodpecker can take these sorts of decelerations here, and that adds on another factor of 4.
And these were the measured decelerations-- 1,500 was about the biggest, and I think it went to about a few hundred g. So there's really three factors-- one is the small brain size, so there's this sort of scaling factor.
One is the orientation of the brain, that was another factor of 2.
And then there's this duration of impact, which is a factor of 4.
And so that's how you can get this huge decelerations that they've measured in the woodpecker when they're pecking.
So now I have just a couple more slides.
So woodpeckers have various adaptations to pecking, too.
So one of the things is they have amazingly stiff tail feathers-- you see the tails here?
You can't quite see it because this didn't reproduce quite properly, but the woodpecker is perched on a tree here, and the tail is pressed up against the tree.
And if you think about pecking, like imagine if its tail was not pressed up against the tree-- it would be kind of grabbing on with its feet, and it would be trying to peck, and it would be hard to push.
You know, you need something to push against.
And so it's got these stiff tail feathers, which make it easier to push against the tree.
And so here's the stiff tail feathers here.
It's also got kind of unusual feet for birds-- it's got two toes forward and two toes back.
And again, if it's grabbing with its feet, that helps it get some purchase to push against.
That's called zygodactyl in the bird world.
So it's got these adaptations, the pecking.
And then finally, I just like this slide here from The New Yorker, because it's a woodpecker pecking out a woodpecker, so it seemed kind of cute.
And several people helped me with this project.
Trey Crisco studies head injury at Brown.
He actually does work for the NFL on football brain injuries, and looks at football helmets.
Sharon Swartz is a friend of mine at Brown, who's a biologist.
She studies bat flight.
And she just thought this was kind of a cool project.
So, in fact, I was walking the dog last week, a few days ago, and I saw bats flying around overhead at night.
And it was so great to see the little bats.
So I had to immediately email Sharon, and say, bats-- there's bats in my neighborhood.
Andy Biewener runs the Concord Field Station, and studies animal locomotion, and I talked to him about it.
Jeremy Trimble was the one who gave me the skull pictures, and Matt Dawson was a student helped me do some of the images.
So that's my woodpecker talk.
So that's the end of energy absorption, I just thought that was kind of amusing.
So next time, we'll start talking about sandwich panels.
So, let's see-- Monday's a holiday.
I will be in Toronto on Monday, seeing my family.
In fact, I'm going to see one of my old professors-- I'm going to my old fluid mechanics professor on Monday and have lunch.
I'll see my family on the weekend.
And so we'll meet Wednesday next week, and I'll start the bit on sandwich panels.
So I think there's two lectures on engineering sandwich panels, and then there's a lecture on natural sandwich panels-- so bird skulls, for example, are natural sandwich panels.
So I'll talk about that.
And then I think the last lecture-- there's only a handful of lectures left-- the last one I think I talk about natural materials, because you know, I like that.
So I just do that for fun.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so we should probably start.
So last time we finished up talking about energy absorption in foamy cellular materials.
And today I wanted to start a new topic.
We're going to talk about sandwich panels.
So sandwich panels have two stiff, strong skins that are separated by some sort of lightweight core.
So the skins are typically, say, a metal like aluminum, or some sort of fiber composite.
And the core is usually some sort of cellular material.
Sometimes it's an engineering honeycomb.
Sometimes it's a foam.
Sometimes it's balsa wood.
And the idea is that what you're doing with the core is you're using a light material to separate the faces, and if you think about an I-beam-- so if you remember when we talk about bending and we talk about I-beams, the whole idea is that in bending, you want to increase the moment of inertia.
So you want to make as much material as far away from the middle of the beam as possible to increase the moment of inertia.
So if you think about an I-beam, you put the flanges far apart with the web, and that increases the moment of inertia.
And the sandwich panels and the sandwich beams essentially do the same thing, but they're using a lightweight core instead of a web.
And so the idea is you use a lightweight core.
It separates the faces.
It increases the moment of inertia.
But you don't add a whole lot of weight because you've got this lightweight core in the middle.
So I brought some examples that I'll pass around and we can play with.
So these are some examples up on the screen, and some of those I have down here.
So for instance, the top-- turn my little gizmo on-- the top left here, this is a helicopter rotor blade, and that has a honeycomb core in it.
This is an aircraft flooring panel that has a honeycomb core and has carbon fiber faces.
So that's this thing here.
I'll pass that around in a minute.
This is a downhill ski.
This has aluminum faces and a polyurethane foam core.
And that's the ski here.
And it's quite common in skis now to have these sandwich panels.
This is a little piece of a small sailing boat.
It had, I think, glass fiber faces and a balsa wood core.
And I don't know if any of you sail, but MIT has new sailing boats.
Do you sail?
I do not much these days
OK, but those little tech dinghies that you see out in the river, those have sandwich panel holes to them.
So those are little sandwich panels.
This is an example from a building panel.
This has a dry wall face and a plywood face and a foam core, and the idea with panels for buildings is that usually they use a foam core because the foam has some thermal insulation.
So as well as sort of separating the faces and having a structural role, it has a role in thermally insulating the building.
The foams are a little less efficient than using a honeycomb core.
So for the same weight, you get a stiffer structure with a honeycomb core than a foam core.
But if you want thermal insulation as well as a structural requirement, then the foam cores are good.
And these are a couple examples of sandwiches in nature.
This is the human skull.
And your skull is a sandwich of two dense layers of the compact bone, and you can see there's a little thin layer of the trabecular bone in between.
So your head is like a sandwich, your skull is like a sandwich.
And I don't know if I'll get to it next time, but in the next couple of lectures, I'm going to talk a little bit about sandwich panels in nature, sandwich shells in nature.
You see this all the time.
And this is a bird wing, here.
And so you can see there's got the dense bone on the top and the bottom, and it's got this kind of almost trust-like structure in the middle.
And obviously birds want to reduce their weight because they want to fly, so reducing the weight's very important.
And so this is one of the ways that birds reduce their weight, is by having a sandwich kind of structure.
So I have a couple of things here.
These are the two panels at the top there.
This is the ski, and you can yank those around.
I also have a few panels that people at MIT have made.
And I have the pieces that they're made from.
So you can see how effective the sandwich thing is.
So this was made by a guy called Dirk Moore.
He was a graduate student in ocean engineering.
And it has aluminum faces and a little thin aluminum core.
So you can see, if you try and bend that with your fingers, you really can't bend it any noticeable amount.
And this panel here is roughly the same thickness of the face on that.
And you can see how easy it is for me to bend that-- very easy.
And this is the same kind of thing as the core.
It's thicker than that core, but you can see how easy it is for me to bend this, too.
So each of the pieces is not very stiff at all.
But when you put them all together, it's very stiff.
So that's really the beauty of this.
You can have lightweight components, and by putting them together in the right way, they're quite stiff.
So here's another example here.
This is a panel that one of my students, Kevin Chang made.
And this has actually already been broken a little bit, so it's not quite as stiff as it used to be.
And you can kind of hear, it squeaks.
But you can feel that and see how stiff that is.
And this is the face panel here.
And you can see, I can bend that quite easily with my hands.
Doodle-doot.
And then this is the core piece here.
And again, this is very flexible.
So it's really about putting all those pieces together.
So you get this sandwich construction and you get that effect, OK?
[?
Oop-loo.
?]
All right, so what we're going to do is first of all look at the stiffness of these panels, calculate their deflection.
We're going to look at the minimum weight design of them.
So we're going to look at how for, say, given materials in a given span, how do we minimize the weight of the beam for a given stiffness?
And then we're going to look at the stresses in the sandwich beams.
So there's going to be one set of stresses in the faces, and a different kind of stress distribution in the core.
So we'll look at the stress distribution.
And then we'll talk about failure modes, how these things can fail, and then how to figure out which failure mode is dominant, which one occurs at the lowest load.
And then we'll look at optimizing the design, minimizing the design for a certain strength and stiffness.
So we're not going to get all that way through everything today, but we'll kind of make a start on that.
OK.
So let me start.
So the idea here is we have two stiff, strong skins, or faces, separated by a lightweight core.
And the idea is that by separating the faces, you increase the moment of inertia with little increase in weight.
So these are particularly good if you want to resist bending, or if you want to resist buckling.
Because both of those involve the moment of inertia.
And they work like an I-beam.
So the faces of the sandwich are like the flanges of the I-beam, and the core is like the web.
And the faces are typically made of either fiber reinforced composites or metals.
So typically, something like aluminum, usually you're trying to reduce the weight if you use these things, so a lightweight metal like aluminum is sometimes used.
And the cores are usually honeycombs, or foams, or balsa.
And when they use balsa wood, what they do is-- I brought a piece of balsa here-- what they do is they would take a block like this and chop it into pieces around here.
And then they would lay those pieces on a cloth mat.
So typically the pieces are maybe two inches by two inches.
They lay them on a cloth mat, and because they're not one monolithic piece, they can then shape that mat to curved shapes.
So it doesn't have to be just a flat panel.
They can curve it around a curved surface if they want.
So we'll say the honeycombs are lighter than the foams for a given stiffness or strength.
But the foams provide thermal insulation as well as a mechanical support.
And the overall mechanical properties of the honeycomb depend on the properties of each of the two parts, of the faces and the core, and also the geometry of the whole thing-- how thick's the core, how thick's the face, how dense is the core?
That kind of thing.
And typically, the panel has to have some required stiffness or strength.
And often what you want to do is minimize the weight for that required stiffness or strength.
So often these panels are used in some sort of vehicle, like we talked about the sailboat, or like a helicopter, or like an airplane.
They're also used in like refrigerated trucks-- they would have a foam core because they'd want the thermal insulation.
So if you were going to use it in some sort of a vehicle, you want to reduce the mass of the vehicle and you want to have the lightest panel that you can.
Yup?
So if you saw the base material and you'd have the [INAUDIBLE] sandwich panel, that piece [INAUDIBLE] the sandwich panel with something [?
solid ?]
in the middle?
Well--
So, as we're getting [INAUDIBLE] aluminum piece that's was as thick as a sandwich panel
Yeah
[INAUDIBLE]
Oh, well, if you have the solid aluminum piece that was as thick as the sandwich, it's going to be stiffer, but it's going to be a lot, lot heavier.
So the stiffness per unit weight would not be as good.
OK?
So we're going to calculate the stiffness in just one minute.
And then we're going to look at how we minimize the weight, OK?
OK.
So what I'm going to do is set this up as kind of a general thing.
We're just going to look at sandwich beams rather than plates, just because it's simpler.
But the plates, everything we say for the beams basically applies to the plates.
The equation's just a little bit more complicated.
So we're going to start with analyzing beams.
And I'm just going to start with a beam, say, in three-point bending.
So there's my faces there.
Boop.
And I've made it kind of more stumpy than it would be in real life, just because it makes it easier to draw it.
And then if I look at it the other way on, it would look something like that.
So say there's some load P here.
Say the span of the beam is l. Say the load's in the middle, so each of the supports just sees a load of P/2.
And then let me just define some geometrical parameters here.
I'm going to say the width of the beam is b.
And I'm going to say the face thicknesses are each t. So the thickness of each face is t. And the thickness of the core is c, OK?
So that's just sort of definitions.
And I'm going to say the face has a set of properties, the core has a set of properties, and then the solid from which the core is made has another set of properties.
So the face properties that we're going to use are a density of the face.
We'll call that row f. The modulus of the face, Ef, and some sort of strength of the face, let's imagine it's aluminum and it yields, that would be sigma y of the face.
And then the core similarly is going to have a density, rho star c. It's going to have a modulus, E star c. And it's going to have some strength, I'm going to call sigma star c. And then the solid from which the core is made is going to have a density row s, a modulus Es, and some strength, sigma ys, OK?
So the core is going to be some kind of cellular material, a honeycomb, or a foam, or balsa.
And typically, the modulus of the core is going to be a lot less than the modulus of the face.
So I'm just going to say here that the E star c is typically much greater than Ef.
And we're going to use that later on.
So we're going to derive some equations, for example, for an equivalent flexural rigidity for the section, an Ei equivalent.
And that has several terms.
But if we can say the core stiffness is much less than the face thickness, and also if we can say the core-- the stiffness is less and also the thickness of the core is much greater than the thickness of the face, a lot of the expressions we're going to use simplify.
So we're going to make those assumptions.
So let me just draw the shear diagram here.
So V is shear, so that's the shear diagram.
We have some load P/2 at the support.
There's no other load applied until we get to here.
Than the shear diagram goes down by P, so we're at minus P/2.
Then there's no load here, so this just stays constant, and then we go back up to 0.
And then let me just draw bending moment diagram.
The bending moment diagram for this is just going to look like a triangle.
Remember, if we integrate the shear diagram, we get the bending moment diagram.
And that maximum moment there is going to Pl over 4.
OK.
So initially, I'm going to calculate the deflections.
And I don't really need those diagrams for that, but then I'm going to calculate the stresses, and I'm going to need those diagrams for the stresses.
So just kind of keep those in mind for now.
So to calculate the deflections, sandwich panels are a little bit different from homogeneous beams.
In a sandwich panel, the core is not very stiff compared to the faces.
And we've got some shear stresses acting on the thing.
And the shear stresses are largely carried by the core.
So the core is actually going to shear, and there's going to be a significant deflection of the core and shear as well as the overall bending of the whole panel.
So you have to count for that.
So we're going to have a bending term and a shear term-- that's what those two terms are there.
So we're going to say there's a bending deflection and a shear deflection.
And that shearing deflection arises from the core being sheared and the fact that the core, say, Young's modulus or also the shear modulus, is quite a bit less than the face modulus.
So if you think of the core as being much more compliant than the face, then the core is going to have some deflection from that shear stress.
OK, so we're going to start out with this term here, the bending term.
And if I just had a homogeneous beam in three-point bending, the central deflections-- so these are all the central deflections I'm calculating here-- with Pl cubed over it turns out to be 48 is the number, and divided by EI.
And because we don't have a homogeneous beam here, I'm going to call that equivalent EI.
And to make it a little bit more general, instead of putting 48, that number, I'm just going to put a constant B1.
And that B1 constant is just going to depend on the loading geometry.
So any time I have a concentrated load on a beam, the deflection's always Pl cubed over EI, and then the sum number in the denominator and that number just depends on the loading configuration.
So for three-point bending, it's 48.
For the flexion of a cantilever, B1 would be 3.
So think of that as just a number that you can work out for the particular loading configuration.
So here we'll say B1 is just a constant that depends on the loading configuration.
And I'll say, for example, for three-point bending, B1 is 48.
For a cantilever end deflection, then B1 would be 3.
So it's just a number.
So the next thing we have to figure out is what's the EI equivalent.
So if this was just a homogeneous beam, and it was rectangular, E would just be E of the material and I would be the width B times the height H cubed divided by 12.
So here we don't quite have that because we have two different materials.
So here we have to use something called the parallel axis theorem, which I'm hoping you may have seen somewhere in calculus, maybe?
But, yeah, somebody is nodding yes.
OK, so what we do, what we want to do is get the equivalent EI-- I'm going to put it back up, don't panic-- of this thing here, right?
So I want-- this is the neutral axis here, and I want the EI about that neutral axis there.
So, OK, you happy?
There.
OK, so I've got a term for the core.
OK, the core, that is the middle of the core, right?
So for the core, it's just going to be E of the core times bc cubed over 12.
Remember, for a rectangular section, it's bh cubed over 12 is the moment of inertia.
And here our height for the core is just c, OK?
And then if I took the moment of inertia for, say, one face about its own centroidal axis, I would get E of the face now times bt cubed over 12.
So that's taking the moment of inertia of one face about the middle of the face.
And I have two of those, right?
Because I have two faces.
And the parallel axis theorem tells you what the moment of inertia is going to be if you move it, not to the-- you don't use the centroid of the area, but you use some other parallel axis.
And what that tells you to do is take the area that you're interested in-- so the area of the face is bt, and you multiply by the square of the distance between the two axes that you're interested in.
Oop, yeah.
Let me change my little brackets.
Boop.
So, oop-a-doop-a-doop.
Maybe I'll stick this, make a little sketch over here again.
OK, all right.
So this term here, Ef bt cubed over 12, that would be the moment of inertia of this piece here, about the axis that goes through the middle of that, right?
Its own centroidal axis.
But what I want to do is I want to know what the moment of inertia of this piece is about this axis here.
This is the neutral axis.
So let's call this the centroidal axis.
And the parallel axis theorem tells me what I do is I take the area of this little thing here, so that's the b times t, and I multiply by the square of the distance between those two axes.
So the distance between those axes is just c plus t over 2, and I square it.
And then I multiply that whole thing by 2 because I've got two faces.
Are we good?
[INAUDIBLE]
Yeah?
The center [INAUDIBLE] and the [INAUDIBLE], are those Ed's or Ef's?
These are Ef's because this is the face now, right?
So this term here is for the core.
So here the core is E star c. And these two Ef's are for the face up there, OK?
Because you have to account for the modulus of the material of the bit that you're getting the moment of inertia for.
Are we good?
OK.
So now I'm just going to simplify these guys a little bit.
Doodle-doodle-doodle-do-doot.
OK?
So I've just multiplied the twos, and maybe I'll just write down here this is the parallel axis theorem.
Doot-doot-doot.
Yes, sorry?
So for the term that comes from the parallel axis theorem, why do we only consider Ef and not [INAUDIBLE]
Because I'm taking-- what I'm looking at-- so the very first term, this guy, here--
Yeah, [INAUDIBLE]
Accounts for this, right?
And these two terms both account for the face
Oh, OK, so the face acting--
Yeah, about this axis.
So the parallel axis theorem says you take the moment of inertia of your area about its own centroidal axis, and then you add this term here.
But it's really referring to that face, OK?
Let me scoot that down and then scoot over here.
And this is where we get to say the modulus of the face is much greater than the modulus of the core.
And also, typically c, the core thickness, is much greater than t, the face thickness.
So if that's true, then it turns out this term is small compared to that one.
And also this term is small compared to this one.
And also this term, instead of having c plus t squared, if c is big compared to t, then I can just call it c squared, OK?
So you can see here, if Ec is small, then this is going to be small compared to these.
If t is small, then this guy is going to be small.
So even though it looks ugly, many times we can make this simpler approximation.
OK, so we can just approximate it as Ef times btc squared over 2.
So then this bending term here, we've got everything we need now to get that bit there.
So the next bit we want to get is the shearing deflection.
So what's the shearing deflection equal to?
So say we just thought about the core, and all we're interested in here is what's the deflection of the core and shear?
And so say that's P/2, that's P/2, that's l/2.
We'll say that's-- oops.
That's our shearing deflection there.
We can say the shear stress in the core is going to equal the shear modulus times the shear strain, so we can say P over the area of the core is going to be proportional to the Young's modulus times delta s over l. And let's not worry about the constant just yet.
So delta s is going to be proportional to-- well, let me [?
make it ?]
proportional at this point.
Delta s is going to equal Pl divided by some other constant that I'm going to call B2, and divided by the shear modulus of the core, and essentially the area of the core.
And here B2 is another constant.
So again, B2 just depends on the loading configuration.
Yeah, this is a little bit of an approximation here, but I'm just going to leave it at that.
OK, so then we have these two terms and we just add them up to get the final thing.
Start another board.
OK.
So that would give us an equation for the deflection.
And one thing to note here is that this shear modulus of the core, if the core is a foam, then we have an equation for that.
We also could use an equation if it's a honeycomb.
But I'm just going to write for foam cores.
Whoops.
This is for-- that will be for open-cell foam cores.
Oops, don't want to-- and get rid of that.
We won't update just now, thank you.
OK, so the next thing I want to think about is how we would minimize the weight for a given stiffness.
So say if we're given a stiffness, we're given P over delta, so I could take out the two P's here.
If I divide it through by P, delta over P would be the compliance, P over delta would be the stiffness.
So imagine that you're given the face and core materials, and you're told how long the span has to be, you're told how wide the beam is going to be, and you're told the loading configuration.
So you know if it's three-point bending, or four-point bending, or a cantilever-- whatever it is.
And you might be asked to find the core thickness, the face thickness, and the core density that would minimize the weight.
So I have a little schematic here.
I don't know if you're going be able to read it.
So I'm going to walk through it and then I'll write things on the board.
Whoops, hit the wrong button.
OK, so we start with the weight equation here.
The weight's obviously the sum of the weight of the faces, the weight of the core, so those two terms there.
So I'll write that down in a minute.
And then we have the stiffness constraint here.
So this equation here is just this equation that I have down here on the board, OK?
Then what you do is you solve that stiffness constraint for the density of the core.
So this equation here just solves-- we're solving this equation here in terms of the density, and we get the density by substituting in this equation here for the shear modulus of the core.
So you substitute that there.
It's kind of a messy thing, but you solve that in terms of the density.
Then you put that version of the density here in terms of this weight equation up here.
So then you've eliminated the density out of the weight equation, now you've just got it in terms of the other variables.
And then you take the partial derivative of the weight with respect to the core thickness c, set that equal to 0, and you take the partial derivative of the weight with respect to the face thickness, t, and you set that equal to 0.
And that then gives you two equations and two unknowns.
You've got the core thickness and the face thickness are the two unknowns.
And you've got the two equations, so then you solve those.
So the value you get for the core thickness is then the optimum, so it's going to be some function of the stiffness, the material properties you started with in the beam geometry.
And similarly, you get some equation for the optimum face thickness, t. And again, it's a function of the stiffness and the material properties in the beam geometry.
Then you take those two values for c and t, those two optimum values, and plug it back into this equation here, and get the optimum value of the core density.
And so what you end up are three equations for the optimal values of the core thickness, the face thickness, and the core density in terms of the required stiffness, the material properties, and then the loading geometry.
So I'm going to write down some more notes, because I'll put this on the Stellar site.
But it's hard to read just here.
So let me write it down and I'll also write out the equations so that you have the equations for calculating those optimum values.
So before I do that, though, one of the interesting things though is if you figure out the optimal values of the core thickness and the face thickness and the core density, and you substitute it back into the weight, and you calculate this is the weight of the face relative to the weight of the core, no matter what the geometry is, and what the loading configuration is, the weight of the face is always a quarter of the weight of the core.
So the ratio of how much material is in the core and the face is constant, regardless of the core-- of the loading configuration.
And this is the bending deflection relative to the total deflection.
It's always 1/3.
And the shearing deflection relative to the total deflection is always 2/3.
So regardless of how you set things up, the ratio of what weight the face is relative to the core and the amount of shearing and bending deflections is always a constant at the optimum.
OK, so let's say we're given the face and the core materials.
So that means we're given their material property, too.
And say we're given the beam length and width and the loading configuration.
So that means we're given those constants, B1 and B2.
If I told you it was three-point bending, you would know what B1 and B2 are.
So then what you need to do is find the core thickness, c, the face thickness, t, and the core density, rho c, to minimize the weight of the beam.
So there's two faces, so the weight of the face is 2 rho f g times btl.
And then the weight of the core is rho c g times bcl.
So I'm going to write down the steps and then I'll write down the solution.
So you solve.
So you put this equation for the shear modulus of the core into here, and then you rearrange this equation in terms of the density of the core here.
So you have an equation for the core density in terms of that stiffness, and then you solve the partial derivatives of the weight equation with respect to the core thickness, c, and put that equal to 0.
And then the partial of dw [?
over ?]
dt and set that to 0.
And if you do that, you can then solve for the optimal values of the face and core thicknesses.
Yes?
[INAUDIBLE] for weight, what is g?
Gravity

Just density is mass, mass times gravity-- weight.
That's all it is.
And then you've got a version of this that's in terms of the core density.
You can substitute those values of the optimum face and core thicknesses into that equation and get the optimum core density.
And then in the final equations, you get, when you do all that, and I'm going to make them all dimensionless, so this is the core thickness normalized by the span of the beam is equal to this thing, here.
So you can see each of these parameters here, the design parameters that we're calculating the optimum of.
I've grouped the constants B1 and B2 together that describe the loading configuration so you'd be given those.
C2 is this constant-- oop, which I just rubbed off-- that relates the shear modulus of the foam core.
So you'd be given that.
These are the material properties of the-- you know, say, it's a polyurethane foam core, this would be the density of the polyurethane.
Say it's aluminum faces, that would be the density of the aluminum.
so you'd be given that.
You'd be given the stiffnesses of the two materials, the solid from which the core is made and the face material.
And then this is the stiffness here that you're given, just divided by the width of the beam, B.
So the stiffness, you'd be given the width B.
So you're given all those things, then you could calculate what that optimum design would be.
So the next slide here just shows some experiments.
And these were done on sandwiches with aluminum faces and a rigid polyurethane foam core.
And here we knew what the relationship was for the shear modulus.
We measured that.
And what we did here was we designed the beams to all have the same stiffness, and they all had the same span in the width, B, then we kept one parameter at the optimum value and we varied the other ones.
So here, on this beam, this set of beams here, the density was at the optimum.
And we varied the core thickness, and we varied the face thickness, and the solid line was our model or our sort of optimization.
And the little X's were the experiments.
So you can see there's pretty good agreement there.
Then the second set here, we kept the face thickness at the optimum value and we varied the core thickness, we varied the core density.
So the same thing, the solid line is the sort of theory and the X's are the experiments.
And here we had the core thickness of the optimum value, and we varied the face thickness and the core density.
So you can kind of see how you can see this here.
And over here, just because I forgot to say it, this is the stiffness per unit weight, over here, OK?
So these are the optimum designs here, all right?
So there was pretty good agreement between these calculations and what we measured on some beams.
Do I need to write anything down?
Do you think you've got that?
Yeah?
I was just going to ask, for the optimum design column that you have there, do those numbers like fall out of these equations if you do the math?
They do, yeah.
I mean, it's-- yeah, exactly.
So if you remember the equation we had for the weight, so the weight is equal to 2 rho f gbtl plus the density of the core, bcl, so if you plug these things into there, then-- so this is the way to the face, that's the way to the core, then it drops out to be a quarter.
So it's kind of magical.
I mean, you have this big, long, complicated gory thing, and then, poof, everything disappears except a factor of 1/4.
And the same for the bending deflection.
So we had those two terms, so there was the bending and the shear.
If you just calculate each of those terms and take the ratio of 1 over the total, or the one over the other, everything drops out except that number.
So that's why I pointed it out, because it seemed kind of amazing that everything would drop out except for that one thing.
OK, so then the next thing-- so that's the stiffness in optimizing the stiffness.
Are we happy-ish?
Yeah?
OK.
So the next thing-- oh, well, let's see.
I don't think I need to write any.
I think if you have that graph, I don't really need to write much down.
So the next thing then is the strength of the sandwich beams.
So let me get rid of that.
You guys OK?
Yeah?
Yeah
Yeah, but you're shaking your head like this is very, very helpful for me
[INAUDIBLE]
Oh OK, that's
[INAUDIBLE]
That's OK, you can do that.
I don't mind.
But as long as you don't have questions for me.
OK, and so the first step in trying to figure out about this strength is we need to figure out the stresses in the beams.
So we need to find out about the stresses.
And we're going to have normal stresses and we're going to have shear stresses.
So I'm going to do the normal stresses first and then we'll do the shear stresses.
So you do this in a way that's just analogous to how you figure out the stresses in a homogeneous beam.
So we'll say the stresses in the face-- normally it would be My over I. M is the moment, y is the distance from the neutral axis, I is the moment of inertia.
So this time, instead of having a moment of inertia, we have this equivalent moment of inertia.
And we multiply by E of the face.
So you can think of this as being the strain essentially.
And then you multiply by E of the face to get the stress.
The maximum distance from the neutral axis, we can call c/2.
So that's y.
Then EI equivalent we had Ef btc squared over 2.
And then I have a term of Ef here.
c squared.
So one of the c's goes, the 2's go, the Ef's go.
Then you just get that the normal stress in the face is the moment at that section divided by the width, b, the face thickness, t, the core thickness, c. And I can do the same kind of thing for the stress in the core, except now I multiply by the core modulus.
So if I go through the same kind of thing, it's the same factor of M over btc, but now I multiply times E of core over E of the face.
And since E of the core is a lot smaller than E of the face, typically these normal stresses in the core are much smaller than the normal stresses in the face.
So the faces carry almost all of the normal stresses.
And if you look at an I-beam, the flanges of the I-beam carry almost the normal stresses.
So I want to do one more thing here.
I want to relate the moment to some concentrated load.
So let's say we have a beam with a concentrated load, P. So for example, something in three-point bending, typically we're interested in the maximum stresses, so we want the maximum moment.
So M max is going to be P times l over some number.
And this B3 is another constant that depends on the loading configuration.
So if it was three-point bending, B3 would be 4.
If it was a cantilever, B3 would be 1.
So if I put those things together, the normal stress in the face is Pl B3 divided by btc.
OK, so that's the normal stresses.
And then the next thing is the shear stresses, and the shear stresses are going to be carried largely by the core.
And if you do all the exact calculations, they vary parabolically through the core.
But if we make those same approximations that the face is stiff compared to the core, and that the face is thin compared to the core, then you can say that the shear stress is just constant through the core.
So we'll say the shear stresses vary parabolically through the core.
But if the face is much stiffer than the core and the core is much thicker than the face, then you can say that the shear stress in the core is just equal to the sheer force over the area of the core, bc.
So here, V is the shear force of the cross-section you're interested in.
And bc is just the area of the core.
And we could say the maximum shear force is just going to be V over-- actually, let's make it P, P over yet another constant.
And B4 also depends on the loading configuration.
So if I was giving you a problem, I would give you all these B1, B2, B3, B4's and everything.
So the maximum shear stress in the core is in just the applied load, P, divided by this B4 and divided by the area of the core.
OK, so this next figure up here just shows those stress distributions.
So here's a piece of the cross-section here.
So there's the face thickness and the core thickness.
You can think of that as a piece along the length, if you want.
This is the normal stress distribution, here.
So this is all really from saying plane sections remain plane.
These are the stresses, the normal stresses in the core.
And you can see they're a lot smaller in this schematic than the ones in the face.
And then this is the parabolic stress in the core.
And similarly, there'd be a different parabola in the face.
And these are the approximations.
Typically these approximations are made so the normal stress in the face is just taken as a constant.
The normal stress in the core is often neglected.
And here the shear stress in the core is just a constant here.
So the two things you need to worry about are the normal stress for the face and the shear stress for the core.
Are we good?
We're good?
Yeah, good-ish.
OK, so if we want to talk about the strength of the beam, we now have to talk about different failure modes.
And the next slide just shows some schematics of the failure modes.
So there's different ways the beam can fail.
Say it's in three-point bending just for the sake of convenience.
One way it can fail is, say it had aluminum faces.
This face here would be in tension, and the face could just yield.
So you could just get yielding of the aluminum.
That would be one way.
It could be a composite face and you could have some sort of composite failure mode.
You can get more complicated failure modes for composites, but there could be some sort of failure mode.
This face up here is in compression, and if you compress that face, you can get something called face wrinkling.
You get sort of a local buckling mode.
So imagine you have the face, that you're pressing on it, but the core is kind of acting like an elastic foundation underneath it.
And you can get this kind of local buckling, and that's called wrinkling.
That's another mode of failure.
You can also get the core failing in shear.
So here's these two little cracks, denoting shear failure in the core.
And there's a couple of other modes you can get, but we're going to not pay much attention to those.
The whole thing can delaminate, and, as you might guess, if the whole thing delaminates, you're in deep doo-doo.
Because, remember when I passed those samples around, how flexible the face was by itself and how flexible the core is by itself.
If the whole thing delaminates, you lose that whole sandwich effect and the whole thing kind of falls apart.
We're going to assume we have a perfect bond and that we don't have to worry about that.
The other sort of failure mode you can get is called indentation.
So imagine that you apply this load here over a very small area.
The load can just transfer straight through the face and just kind of indent the core underneath it.
We're going to assume that you distribute this load over a big enough area here, that you don't indent the core.
So we're going to worry about these three failure modes here-- the face yielding, the face wrinkling, and the core failing and shear, OK?
So let me just write that down.
And then you also can have debonding or delamination, and we're going to assume perfect bond.
And then you can have indentation, and we're going to assume the loads are applied over a large enough area that you don't get-- So you can have different modes of failure, and the question becomes which mode is going to be dominant?
So whichever one occurs at the lowest load is going to be the dominant failure mode.
So you'd like to know what that lowest failure mode is.
So we want to write equations for each of these failure modes and then figure out which one occurs first.
So we'll look at the face yielding here.
And face yielding is going to occur just when the normal stress in the face is equal to the yield stress of the face.
So this is fairly straightforward.
So this was our equation for the stress in the face.
And when that's equal to the face yield strength, then you'll get failure.
And the face wrinkling occurs when the normal compressive stress in the face equals a local buckling stress.
And people have worked that out by looking at what's called buckling on an elastic foundation.
So the core acts as elastic support.
You can think that as the face is trying to buckle into the core, the core is pushing back on the face.
And so the core is acting like a spring that pushes back, and that's called an elastic foundation.
So people have calculated this local buckling stress, and they found that's equal to 0.57 times the modulus of the face to the 1/3 power times the modulus of the core to the 2/3 power.
And here, if we use our model for open cell foams, we can say the core modulus goes as the relative density squared times the solid modulus.
And so you can plug that in there.
So then the wrinkling occurs when the stress in the face, the Pl over the B3 btc is equal to this thing here.
OK, so one more failure mode that's the core shear, and that's going to occur when the shear stress in the core is just equal to the sheer strength of the core.
So the shear stress is P over B4 times bc, and the shear strength is some constant, I think it's C11, times the relative density of the core to the three halves power times the yield strength of the solid.
And here, this constant is about equal to 0.15, something like that.
So now we have a set of equations for the different failure modes, and we could solve each of them, not in terms of a stress, but in terms of a load P. The load P is what's applied to the beam, right?
So we could solve each of these in terms of the load, P. And then we can see which one occurs at the lowest load, P. And that's going to be the dominant failure mode.
So one way to do it would be to, for every time you wanted to do this, to work out all these three equations and figure out which one's the lowest load.
But there's actually something called a failure mode map, which we're going to talk about.
So let me just show you it and we'll start now.
I don't know if we'll get finished this.
But there's a way that you can manipulate these equations and plot the results as this failure mode map.
And you'll end up plotting the core density on this plot, on this axis here, and the face thickness to span ratio here, and so this will kind of tell you, for different configurations of the beam, different designs, for these ones here, the face is going to wrinkle, for those ones there, the face is going to yield, and for these ones here, the core is going to shear.
So I'm going to work through these equations, but I don't think we're going to finish it today.
So this is just kind of where we're headed is to getting this map.
So we'll say the dominant failure mode is the one that occurs at the lowest load.
So the question we're going to answer is how does the failure mode depend on the beam design?
And we're going to do this by looking at the transition from one failure mode to another.
So at the transition from one mode to another, the two modes occur at the same load.
So I'm going to take those equations I had for each of the failure modes, and instead of writing this in terms of, say, the stress in the face, I'm going to write it in terms of the load, P. So using that first one over there, the load for face yielding, I'm just rearranging that.
It's B3 times bc times t/l times the yield strength of the face.
And similarly for face wrinkling, I can take this equation down here and solve it for this P here, OK?
And then I can take that equation at the top and solve that for P2 for the core shear, and that's equal to C11 times B4 times bc times sigma ys times-- oops, wrong thing-- times the relative density to the 2/3 power.
OK?
And then the next step is to equate these guys.
So you get a transition from one mode to the other when two of these guys are equal to each other, right?
So there's going to be a transition from face yielding to face wrinkling when these guys are equal.
And I'm not going to start that because we're going to run out of time.
But let me just say that I can pair these two up and say there's a transition between those two.
And that transition is going to correspond to this line here, OK?
So at this line here, that means you get face yielding and face wrinkling at the same load, OK?
And then if I paired up-- let's see here.
If I paired up face wrinkling and core shear, these two guys here, I'm going to get this equation here on that plot.
And then if I paired up these two guys here, the face shielding and the core shear, I would get that line there, OK?
So once I have those lines, that tells me, you know, anything with a lower density core and a smaller face thickness is going to fail by face wrinkling.
Anything with a bigger density is going to fail by face yielding.
And anything with a larger face thickness and a larger density is going to fail by core shearing.
And so you can start to see that it-- I'll work out the equations next time, but you can start to see that it kind physically makes sense.
Intuitively, this face wrinkling, it depends on the normal stress in the face, in compression.
So obviously the thinner the face gets, the more likely that's going to be to happen.
So it's going to happen at this end of the diagram.
And it also depends on that elastic foundation, on how much spring support the foundation has, right?
So the lower the core density, the more likely that is to happen.
Then if you, say you have small t, so the face is going to fail before the core, as you increase the core density, you're making that elastic foundation stiffer and stiffer, and you're making it harder for the buckling to occur.
It can't buckle into the elastic foundation, so then you're going to push it up to the yielding.
And then as you make the face thickness bigger, as t gets bigger, then the face isn't going to fail and the core is going to fail.
So you can kind of see just looking at the relative position of those things, they all kind of make physical sense.
So I'm going to stop there for today and I'll finish the equations for that next time.
And we'll also talk about how to optimize for strength next time.
And we'll talk about a few other things on sandwich panels.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, then, I guess we may as well start.
So what I wanted to talk about today was natural sandwich panels and sandwich beams.
So there's lots of examples of sandwich structures in nature, and we've been looking at the engineering sandwich structures.
And we've seen that you can get a lightweight structure by having this sandwich construction.
And so there are several examples I was going to talk about today.
And I think because this isn't really on the test, I'm not going to write a lot on the board.
So there's some notes.
I'll just put them on the website, and you can look at that if you want.
Because we have kind of a shorter time today.
I'll just try and talk and explain what's what.
Hey, Bruno.
How are you?
So this is the first example.
So many leaves of Monocotyledon plants have a sandwich structure.
And this is an iris plant and iris leaves.
And for those of you in 3032, I think you know that these are glass flowers.
So the Harvard Museum of Natural History has a glass flower collection that was made in the 1800s.
And there was a botany professor there who made these as sort of a lecture demonstration vehicle.
And so he would bring then to class, and he would show different things about the plants with the glass flowers.
But now they're just in the museum, and they're very realistic.
So I just wanted to show you those.
So let's see, it's not working.
Turn it on.
There we go.
So if we look at a cross-section of an iris leaf, it looks like the diagram on the left.
So here's the iris.
And you can see there's these kind of solid fibers, and those solid fibers are called schlerchyma.
And they only exist at the top and the bottom of the leaf.
So I went out this morning.
And if you look outside of the Stata building, there's that little kind of river-y thing, and there's some iris leaves growing there.
So I went and got some iris leaves.
And you can tell we had a horrible winter because usually when I give this lecture in the spring, the leaves are like twice as big.
But this year, they're just little, short, wimpy ones.
But I'm going to pass it around.
And if you just like move your thumb over the top, you can feel little ridges, little bumps.
And those little ridges that you can feel are these little schlerchyma fibers.
So you kind of see they kind of stick up a little.
And so when you move your thumb over it, you can feel that.
And then you can see that the middle of the iris leaf has this kind of foamy-type structure here, and that's called parenchyma cells.
So you can think of the leaf as very much like one of the sandwiches.
This is like a fiber-reinforced composite at the top and at the bottom.
And then this is kind of like a foam core in between, separating the fiber-reinforced faces.
And so the iris leaf behaves mechanically like a sandwich beam.
So I'm going to talk a little bit about how we can actually demonstrate that using the equations that we developed in class.
This is another example.
This is, I guess, what Americans call the cat tail, but Canadians and English people call it a bull rush.
And you can see this is a slightly different construction, but there's the same sort of idea.
So instead of having a foamy core as in the iris leaf, you've got these kind of webs here that go in between the top and the bottom, and that forms like a series of I-beams almost.
And you can think of that also like a sandwich panel or a sandwich beam.
So you've got two stiff top and bottom pieces, and then you've got these kind of webs that separate them, kind of like a honeycomb core would be.
So that's another example of a leaf that has the sandwich-type structure.
And this is very common in these Monocotyledon leaves.
So if you think of a cat tail or you think of an iris, they tend to be kind of narrow at the base, maybe an inch or two wide at the base, and they can be quite tall.
The iris leaves can get two or three feet tall.
The cat tails can get five or six feet tall.
And they stand up more or less straight.
They bend over a little, but they stand up more or less straight.
And this sandwich structure is one of the things that lets them stand up straight at a fairly low weight.
And from the plant's point of view, there's a sort of metabolic cost associated with making more material.
So it we can minimize the amount of material, it's a better thing for the plant.
These are some other examples of grasses that are sandwich-type constructions.
This is from some papers by Julian Vincent.
And the black little circles here are the schlerchyma, are those sort of dense fibers.
Then you can see in both of these cases, the dense fibers are on the outside, and the parenchyma cells, which is the white, are on the inside.
And so this is sort of another set of micrographs of the iris.
So this is just showing the outside, and these are the ribs viewed from the outside.
And this is the core, just sort of viewed along the length of it.
And so you can idealize the structure as being like a sandwich that's got sort of fibers on the top and on the bottom.
So the top and the bottom are like a fiber composite.
And the middle part, with the parenchyma cells, is kind of like a foam.
And so we did a little project on iris leaves, and we wanted to see if you could show that they behave mechanically, like a sandwich beam.
So you remember that we had that equation for the deflection of the sandwich beam.
There were two terms.
There was a bending term, and then there was a shearing term.
And so we took some little sandwich beams.
We cut little kind of rectangular beams.
We hung little weights.
We measured how much they deflected, and we wanted to see if we could use this equation to predict their stiffness and how much they deflected.
So to do that, we needed to know a bunch of things.
We needed to know some of the geometrical parameters.
So we needed to know what volume fraction of the face is those solid ribs, how thick's the core, how thick's the face?
And so we measured a bunch of these geometrical parameters.
We tested it like a cantilever so we knew what B1 and B2 were for the cantilever.
We knew how long the beam was, so we know what l is.
We knew what loads we applied, so we knew what P was.
But we needed to make some estimate of what the face modulus was and what the core shear modulus was, too.
And so we made some estimates of that.
So this table here just shows some of the dimensions of the leaf.
The leaf tapers, and this is at the thin end, so here's the face thickness.
Here's the sort of length of this.
Some square cells in the face.
This is the core thickness here.
This is the dimensions of the core cells.
This is the diameter of the ribs, the spacing of the ribs, the volume fraction of solids in the ribs.
And we did that at different lengths along the different positions along the length of the rib, or length of the leaf.
So we had the geometrical parameters, but we needed to get this E of the face and G of the core.
And to do that, we looked at the literature.
And people had done tests on the fiber parts of leaves.
They'd done little tensile tests, and they'd measured modulii between about two and 20 gigapascals.
And then we did some tension tests on the iris leaf.
And in tension, those ribs are going to take most of the stress.
And if you know the volume fraction of the ribs, you can back out what the stiffness of the ribs must have been.
If you know the stiffness of the ribs, you can figure out the stiffness of the face.
So we calculated that, and then we looked at the literature.
And people have done tests on parenchyma cells and different types of tissue on things like apples and potatoes and carrots.
And these are the values for the Young's modulus they get.
They're between about 1, and the highest one was 14 megapascals.
But most of these values for the Young's modulus are around about four.
And the shear modulus is roughly about half of the Young's modulus.
So we said the shear modulus was around two.
So we have these values we could plug it in and then calculate what the stiffness would be for the iris leaf.
And so this was a little analysis we did.
So this was the measured beam stiffness up here.
We had four beams, and they were different stiffnesses.
They all had the same length.
They all the same face thickness.
The core thickness varied.
They all had the same width.
We cut them to have the same width so we could calculate a flexural rigidity.
That's the EI equivalent.
We could calculate the bending deflection term, the shear deflection term.
And this is the calculated beam stiffness.
And then this is the ratio of the calculated over the measured.
So it's not exactly right.
Obviously, there's some difference here.
But it's in the same order of magnitude.
It's in the same ballpark.
And one of the complications that we didn't really try to take into account was that the leaf isn't a nice rectangular structure.
The leaf has this kind of curved cross-section to it.
And we made a bit of an approximation to that, but it wasn't that close, really.
We could have probably done better on that.
But I think the idea that the iris behaves like a sandwich is a reasonable one.
So that was the iris leaf.
And then I wanted to show you some other structures in nature that are sandwiches.
So this is a seakelp, help like a seaweed thing, in New Zealand.
This is the largest intertidal seaweed.
The fronds, the sort of long pieces of it, are up to 12 meters long.
So that's almost 40 feet.
So 40 feet is probably like from one side of this room to the other side of the room.
It's quite long.
And you can see, if you look at this section here, this is all like a honeycomb-type section here.
And the honeycomb is like a honeycomb in a sandwich, and the top and the bottom faces are like the face of the sandwich.
So this would be like the face here.
That would be the honeycomb core.
And that would be the other face on the other side over there.
And those honeycomb-like cores, apparently, have some gas-filled pockets that then provide buoyancy to keep the whole thing floating.
So it photosynthesizes.
So one of the things about these leaves is that they have multiple functions.
It's not just that they have to have a certain stiffness so they don't fall over.
The plant wants to photosynthesize, so you want to maximize the surface area as well, and you want to have exposure to the sunlight.
So there's a number of things that the plant's trying to do in having this structure.
So that seakelp is one example.
These are skulls from birds.
And so this is a pigeon here.
This is a magpie.
If you come from the West you see magpies out West.
You see them in Europe as well.
And this is a long-eared owl.
This long-eared owl's around here.
And I brought in a couple of bird skulls as well.
And you can see that all of those birds skulls are sandwich structures.
The one for the pigeon has sort of a foam-like core here.
And you can see that the two faces aren't sort of concentric for the pigeon skull.
They sort of not following each other.
But here, this would be, say, on the top shell of the magpie, where the two, the inner and outer face, are sort of concentric.
Then you get these kind of little ribs of trabecular bone in between them, and then the same with a long-eared owl.
You get these little ribs in between them.
And so you can see that there's a sandwich structure there.
And obviously, birds want to be light.
They have to be light to fly, to take off, and so they want to be light.
So I've got two skulls here.
And I'll pass them around.
Please be careful because they're kind of delicate.
This one is from a screech owl, and you see screech owls around here.
This was a screech owl that had an intersection with a car.
Yeah, so the skull fractured, but you can see the sandwich right there.
You see the two little bits?
So you can see the inner plate and the outer plate and the foam, the trabecular bone.
So that's the screech owl.
And this is a red tail hawk.
So you can't really see the shell and the sandwich structure here.
But I want to pass it around just so you can see how light it is.
So it's amazingly light.
So a red tail hawk is probably about this big, something like that.
And this is one of the things that makes them very light.
So those are the bird skulls.
Oh, yes, so now I have to tell you about the owl.
So I think the people in 3032 have heard this before.
But the other people haven't.
So one of the things about the owl is if you look at the whole skull, if you look at this picture here, one of the things is that this bone here is not symmetrical with that bone there.
Normally, when you think of a body, you think of the bones being symmetrical.
But those bones are not symmetrical, and those bones are near where the ear is.
And it turns out on owls, at least on some owls, the ears are at different heights on their heads.
And people think that one of the things that allows the owls to do is it allows their hearing to sort of pinpoint where something is.
And owls can catch little creatures at night, but they can also catch little creatures underneath the snow.
So they can catch things that they can't even see.
And they have a number of adaptations to improve their hearing, but this is one of them.
So here's a little owl Allison Curtis is a Canadian friend who lives in northern Ontario, and this is looking out of her living room window.
And that's a barred owl.
And you can see the barred owl has caught this little vole here.
And you can see in the background it's winter in Canada.
and there's snow all over the place.
So this owl has probably caught that little vole underneath the snow.
And then it's come to eat it.
And this is another picture of-- you can see this is where an owl landed in the snow.
It's wings hit the snow, trying to catch something underneath.
And this is another kind of beautiful print of the owl's wings hitting the snow in the winter time.
So did I show you the fox video?
Should I show you the fox video?
You saw it, right?
I think I showed it last time in 3032.
But you guys haven't seen it.
Let me show you the fox video because foxes do the same kind of thing.
Their ears are the same as ours.
They're in the same position.
But they have this-- let me see.
Where's the sound thing?
We don't really need the sound for this, but there's BBC sound.
So we get this music, even though the fox can't hear the music.
Here we go, fox no drive.
Check this out.
Is it going to come up?
Is that going to play?
OK [VIDEO PLAYBACK] -It listens for the tiny sounds of its prey moving about below
So you see how it cocks its head, and it does this with its head?
It's putting its ears at different heights when it does that.
So check this out.
And look carefully, you can see the little animal it's got in its mouth when it comes out.
There's a little tail.
So part of the reason dogs and foxes and coyotes do that thing, I think, is because they put their ears at different heights, and it helps them pinpoint where something is.
[END PLAYBACK] You know I love these Nature videos, right?
So that's the fox video.
Let me see if I can stop that.
So that's one of the interesting things about owls.
Let me go back to my little PowerPoints.
So here's another example of a creature that has a sandwich-type structures.
So here's the sandwich here.
Here is the ever so charming looking cuttlefish.
And the cuttlefish is not actually a fish.
It's a mollusk.
So it's related to things like octopus, things like that, and squids.
It's a cephalopod.
And you can't see it so well in this picture, but I'm going to show you something else and you see it.
It's got like little tentacles.
These things here are actually separate little tentacles.
And because it's not a fish, it doesn't have like fins that can kind of swim with.
And it's got this thing called the cuttlefish bone.
And this is a cuttlefish bone here.
And that bone has the sandwich structure here.
And it's not actually a bone.
It's really a shell.
It's a calcium carbonate thing, not a calcium phosphate thing.
But the cuttlefish can control how much air goes into those little pockets.
And it can control its buoyancy by controlling how much air goes into those little pockets.
And I brought with me a cuttlefish bone.
Have you ever owned like, I don't know, like a parrot or a pet bird?
Apparently, pet birds love to sharpen their beaks on this cuttlefish bone.
So if you go to a pet store, you can buy this stuff.
So you won't be able to see the little sandwich structure because it's a very small length scale.
But you can kind of see there's a sort of different material on the inside than there is on the outside of that.
So do people know the other thing that cuttlefish are famous for, besides the bone?
Change colors.
Can I show you a video of cuttlefish changing color?
Yeah, of course.
So let me get rid of this again.
Go back to this.
Let's see, somewhere-- where's the cuttlefish?
Here we go.
Did I do it?
Is it thinking?
Here we go.
Where's the cuttlefish?
So this is another one of these Science Friday videos from National Public Radio with Flora Lichtman.
[VIDEO PLAYBACK] -OK, let's play a game.
[GAME SHOW MUSIC PLAYING] [APPLAUSE]
See it?
-Biologist Sarah Zielinski took these shots.
And if you needed a helping hand to find the cuttlefish, don't feel bad.
-I've certainly taken photos in the past then come back to look at them and gone, I'm sure there was a cuttlefish in there somewhere!
-These cephalopods are master camouflagers.
But while they're hiding their body, they're revealing something about their mind, or at least their visual system.
-In very simple terms, they can tell us what they can see by the body patterns they produce on their skin.
-They produce these body patterns by expanding or contracting chromatophores, these little ink sacks on their skin.
And they use different displays for different reasons, like for male-to-male combat.
-Two males will turn into each other and pass these kind of waves of dark chromatophores over a really bright sort of iridescent stripey body pattern and somehow solve these combats.
Eventually, one male gives up and goes away.
-And then there's this unsolved mystery.
It changes color when it grabs a snack.
-That doesn't make perfect sense because it seems to make it very conspicuous.
So one theory is that it's just a happy signal of how excited it is to have caught something, some response that it doesn't have any control over.
-But most of the time they seem to be using their chromatophores more intentionally, primarily to blend in.
-Because otherwise they're more likely to be eaten, so it's very important they don't make mistakes about ambiguous visual information.
-And ambiguous visual information is specifically what Zielinski's interested in.
So here's the experimental setup.
Print out laminated patterns, like this checkerboard, and stick them in a tank.
-And we place the animals in the tank.
And we record the body patterns that they produce.
-You're seeing them on squares, but they do the same thing on top of circles.
They produce-- - --the disruptive pattern, where you get these blocky components of high-contrast components.
-But when you put a cuttlefish over squiggles, it produces-- - --a sort of mottley pattern, where you get these little groups of dark spots showing across the body.
-So what happens when you put a cuttlefish on something in between, when you put them on incomplete circles?
When we see something like this, our visual system likes to fill in the blanks, something we do constantly, Zielinski says.
-The reason why cartoons and sketches work is because we can recognize objects based on their edges alone.
-And we can identify objects even if they're broken up or-- - --have an object that is occluded by another object.
That's no problem for us.
We can still work out what the object is most of the time.
And I was interested to know whether cuttlefish can solve similar problems.
-And Zielinski and colleagues report this week that cuttlefish do seem to-- - --fill in those gaps and interpret those little segments as a whole circle.
-Or anyway, the broken circles prompted the same camo pattern as full circles.
So if you're wondering, uh, I see these as circles, too.
What's the big deal?
The weird thing here is that there's no reason why cuttlefish, which are-- - --invertebrates, and they're in the same group as slugs and snails.
- --should see the world the way we do.
-Yes, it's like they're alien, but we also seem to have so much in common with them.
-So the next step?
-Because we can't share the perceptive experience of a cuttlefish, it's hard to know exactly what it is that they're doing to fill in that missing information.
And I want to try to get a better grasp on that and also see whether they actually respond to true illusory contours.
-So you're going to show optical illusions to cuttlefish?
-(LAUGHING) That's what I'm hoping to do, yes.
[END PLAYBACK]
So let's go back to sandwiches.
I think I have-- do I have one more?
There we go.
So horseshoe crab shells, so different sorts of arthropods, the shells are sandwiched too.
This is from Mark Myers' work.
So we're looking at the cross-section of a horseshoe crab shell.
So again, it's the same idea-- the animal wants to minimize the amount of material or minimize the weight, and this is a way of doing that.
And I went to the Galapagos about a year ago.
And there was a place where they had these giant Galapagos tortoise shells.
And one of them was broken, and you could see there was a sandwich structure in the Galapagos tortoise shells.
These Galapagos tortoises, their shell is like this big.
They're gigantic.
They're huge.
So those are my examples of sandwich panels and beams and shells and whatnot in nature.
So the idea is that nature too wants to minimize weight and minimize the amount of material, and the sandwich structure is a way of doing that.
So I have one more thing I wanted to talk about today.
So this isn't quite sandwich structures, but it's looking at another kind of natural structure that is designed to reduce the weight of plant stems, in this case, palm stems.
And there's a couple of interesting things about this.
So when you look at palms, like let's pretend we're not in Boston.
We're in California, where they have palms.
And we're in LA, and they don't have winter.
And if you look at the palms growing, when the palm's short, it's about this big in diameter.
And as it gets taller and taller, the diameter doesn't really change.
It gets taller and taller and taller, but the diameter doesn't change, at least in some species.
Whereas if you think of a tree, a tree starts out with a little skinny diameter.
And as the tree gets taller, the diameter gets bigger.
And it sort of tapers and does that whole thing.
So palms don't do that.
And palms are not trees.
They're a botanically different thing from trees.
So here's a coconut palm.
And so the question is, as the stem gets taller and taller, how does it resist the bending loads that get bigger and bigger?
So probably, the main load on these sorts of things is from the wind.
And often these plants are in areas where they have hurricanes.
And you see them in hurricanes, you see the pictures of the palm stem blowing way over.
And so how do they resist the larger internal stresses as they get taller and taller, if the diameter doesn't get bigger and bigger?
And the way they do that is that they deposit additional layers of cell wall as the plant ages.
So if you think of a tree, when a tree grows, it just deposits more and more cells.
And the cells have roughly the same thickness.
So there's ones that are deposited in the spring have thinner walls.
Then the summer and the fall have thicker walls.
But more or less, it's similar.
Whereas the palm, it deposit cells, and then as the trunk of the palm gets taller, as the stem gets taller, it deposits more layers on the cell wall.
So this is an example in an SCM.
You can see here this is a young cell, and it's got-- this one that's not marked is a primary cell wall, and then this is the first layer of the secondary cell wall.
And then this is an older palm.
And you can see here it's got more layers, and so the cell wall itself has gotten thicker.
So that means that the density of the tissue changes as the palm ages.
And it does so in a very kind of clever way.
If you think of the palm as being like a cantilever that's vertical and it's bending in the wind, when we have a cantilever beam or any kind of beam, the stresses are going to be biggest on the periphery, right?
They're going to be biggest on the outside.
And if you think of the palm as having a circular cross-section, that outer periphery is going to see the biggest stresses.
So it would make the most sense if that was the densest tissue.
And that's exactly what the palm does.
So there was a nice study done by Paul Rich quite a number of years ago.
And he studied palms in Central America and looked at the density and measured the mechanical properties.
And I'm going to talk about his stuff today.
So the white is the low density.
The gray's the medium, and the black's the high.
So you can see the low density's on the middle of the young stem, and just at the very base and then the periphery is the dense tissue.
But as the stem gets taller and gets older, then stuff that was low density is now high density.
And only the very middle here is the low density.
And that some stuff that was low density has turned to middle density.
And some stuff that was low density has turned to high density.
So it's done this by adding more and more layers to the cell wall, making the cell wall thicker and making the cells themselves denser.
So this is looking just at a single palm.
So each one of these lines is a single palm.
And this is looking at how the density changes from the periphery to the center of the palm.
So if you cut the palm down and, say, we take a little sample radially from the middle to the outside or from the outside to the middle, he then measured the density.
And it's probably easiest to think about the dry ones because that's kind of what you would compare wood to.
So the dry densities varied from about one gram per CC, that's about 1,000 kilograms per cubic meter, down to almost zero in this particular species here, probably like 50 or something like that.
And if you compare this with woods, this little arrow here is the density of most common woods.
So if you looked at pine and spruce an oak and maple and ash and hickory, they would all be in that little range there.
So a single palm stem can have a bigger range of densities than many different species of wood.
So it has this kind of profile of the density.
And the thing I was interested in is seeing how mechanically efficient that was to put the denser material at the outside.
So I looked at the stiffness of the palm, and I also looked at the strength.
So I just replotted that data on this slightly different axes here.
So this is the radial position relative to the outer radius, and this is the density.
And I subtracted off the minimum and then took the range.
And for this species here, the minimum density was almost zero.
So this expression simplifies to something like that.
And just because it's mathematically simpler, that's what we're going to look at.
So the density goes roughly as the radius squared.
And Paul Rich also did a lot of mechanical tests on the palm, and he took out little beams of different densities.
And he measured the stiffness and the strength of the beams.
So he measured the modulus of elasticity here versus density.
And he measured the modulus of rupture here.
And these are all along the grain.
And he found that the Young's modulus varied with the density to the 2.5 power, and the strength varied as the density squared.
And if the-- [BUZZING SOUND] Oh, hello.
[LAUGHTER] So these were just sorts of empirical findings that he made.
If you have prismatic cells and you deform them axially, and the cell wall was the same in the different specimens, then the solid modulus would be a constant.
And you would expect that the modulus of the beam would go just linearly with the density, sort of like a honeycomb loaded [?
at a ?]
plane.
But what he measured was that the modulus and the strength varied with some power of the density.
And the reason for that really was that the cell walls of the denser material had more layers.
And in the additional layers, the cellulose microfibular angle was probably different, so that the different layers had different stiffnesses.
And if you have layers of differences, then you're going to get this power relationship.
So what I then did was I took his data, and I tried to see how efficient that would be in bending.
So he had found that the density varied with the radius raised to some power.
This power n was 2, but I wanted to do it just for a general case, so I said I was just n. And he said that he found that the modulus varied with the density raised to some other power m. And for him, m was 2 and 1/2.
And so I could write just another equation saying that the modulus goes as the radius to the mn power.
And then you could do a little calculation where you work out with the equivalent flexural rigidity is.
So you have to integrate up.
You kind of say you have a little band at a certain radius.
That radius has a certain modulus.
And you can figure out the moment of inertia that goes with that particular radius.
And then if you integrate it up over the whole thing, you can say that the flexural rigidity for the gradient density is some constant times pi times the outer radius to the fourth power divided by those two powers mn plus 4.
So m was the power here for the modulus.
And n was the power there for the density.
And then you could compare that with having the same mass just uniformly distributed over the whole cross-section.
And then if you take the ratio of the flexural rigidity for the density gradient versus the flexural rigidity for the uniform density, you can show that it's this equation here.
And then if you plug in these measured values for those exponents for n and m, you find that the flexural rigidity with the gradient density relative to the uniform density is a factor of 2 and 1/2.
So the stem is 2 and 1/2 times stiffer by having that density profile.
So there's a huge sort of mechanical advantage to doing that.
And just sort of physically, if you know the stresses our biggest on the outside, it would make sense to put the denser material on the outside.
And then the other thing I looked at was the strength of the palm.
So imagine this is our very schematic palm here, and then there's a circular cross section.
So I wanted to compare the bending stress distribution with the bending strength distribution.
So the stress goes as the modulus times the strain, just Hooke's law.
And here we're assuming that plane sections remain plane, like that's the standard assumption of bending.
So if you assume plane sections remain plane, then the strain goes with the curvature times the distance y from the neutral axis, the distance from the middle.
So this distance here would be the dis-- [?
same ?]
at loaded with a loaded p here.
That distance would be y there.
And then I can plug in some things here.
So instead of E, I'm going to plug-in my relationship with the radius to that mn power.
And here's my curvature, and instead of y, if I say that some radius, I'm going to say y is our r cos theta.
And so I'm going to say that the stress goes-- [SNEEZE] Bless you.
Goes as radius raised to some power mn plus 1.
And again, for the species I know what n and m are, so the stress goes as the radius to the sixth power.
And then I can also compare with what Paul Rich had found for the strength.
He found that the strength-- so sigma star is the strength-- was proportional to the density raised to some power q, and that power was 2 in the measurements that he made.
And so I can say that the strength goes as the radius to this power nq, so to the fourth power.
And then if I plot the stress distribution and the strength distribution-- so imagine, this is through the cross-section here.
So this is the diameter of the stem.
And this is the neutral axis here in the middle.
The strength goes as that solid line there.
It goes as the fourth power.
And the stress goes as that dashed line there, as the sixth power.
So they're not exactly on top of each other, but they're very close to being on top of each other.
So basically what the palm has done is it's arranged the material in such a way that the strength matches the stresses that are applied to it.
So if I just had a constant density, my stress profile would look like that.
And if I had a constant density, the strength profile would kind of like that.
So the strength here would be a constant, and this would be the stress here.
So the stuff in the middle, it's much stronger than it needs to be.
Whereas the palm has arranged things so that it's got just the right amount of strength for the stress, as a function of the radial position.
So it's kind of a clever thing.
So that's kind of a beautiful thing.
And I think that is it.
I think that's-- yeah, that's the end of it.
So all these images came from this other book that we wrote.
And if you wanted to get the sources, you could get them from there.
So that all I wanted to talk about today was some examples of sort of efficient mechanical design in nature and the sandwich panel structures as one, and these radial density gradients is another.
We have a project on bamboo right now, and the bamboo also has a radial density gradient, and it's the same thing.
The densest material's on the outside, and the least dense is on the inside.
So I think I'm going to stop there for today.
So what I was going to do on Monday is talk a little bit about bio-mimicking.
And that won't take the whole class at all.
And I thought we could spend the rest of the class on Monday just doing a review.
So the test's on Wednesday.
So if you want to bring questions, that would be a beautiful thing.
I can't really can I review the whole last six weeks or something in an hour and a half or something.
So if you want to bring questions, I'll be here and we can just go over questions.
Does that sound good?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, well, I guess I may as well start.
I don't know if anybody else is going to come.
So I wanted to finish up by talking a little bit about the biomimicking.
And some of these examples, you've seen before.
But I just thought I'd put them all together, and we could look at them as one thing.
So if you remember, when we talked about wood, one of the things that I showed you was that people have taken wood and pyrolized it.
So they get a carbon template of the wood cells.
And then they infiltrate that with silicon carbide-- or, with a silicon vapor infiltration.
And they make a silicon carbide ceramic.
So they can get a replica of the structure.
So sometimes when people say "biomimicking," some people think of it as replicating something.
But it doesn't have to be just replicating something.
It can be also just some design inspired by the biological material.
But this thing really is a replica.
And this was another version where there was the-- they took the silicon carbide material and then infiltrated that with liquid silicon to get a fiber-reinforced material, really.
So there was the wood composites we talked about.
Jennifer Lewis's group up at Harvard is doing 3-D printing of honeycombs.
And one of the things they've been interested in is not just printing of the pure resin, but having a fiber-reinforced resin.
And the first thing they did was they made these honeycombs like this.
And they had small silicon carbon fibers-- or, were they silicon carbide?
Maybe it was carbon fibers-- in the ink.
And so they would just-- if the ink was being laid down, the fibers would just line up in the direction of the ink.
And so the fibers tended to be in the plane of the honeycomb.
And if you think of things like wood, you want the fibers to normal to that plane.
And more recently, they've-- hello.
Oh, hello.
Oh, look.
Oh, look, almost everybody is here.
So more recently, they've got a technology now where they're rotating the nozzle as they print the honeycomb.
And as they rotate the nozzle, they get some change in the orientation of the fiber.
So they're beginning to be able to make honeycombs that are fiber-reinforced.
And they can get the fibers aligned with the prism axis of the honeycomb, which is more or less what the wood does.
So I wasn't going to write anything on the board today.
I was just going to go over some of the slides and do the review.
So they're beginning to make honeycombs that have the same sort of structure on the cell wall level, or at least a similar structure, as to what the wood composites have.
So that's another example there.
This is just another close-up of their fiber-reinforced walls in honeycomb specimens.
We talked about trabecular bone, and we talked about the fact that people are starting to look at using foamed metals as coatings on orthopedic implants.
And there's been some interest in looking at using foamed metals for more permanent parts of the body, more permanent bone parts, things like vertebral cages, stuff like that.
And so this is just an example here, with the trabecular bone on the left and the tantalum foam that's made by replicating an open-celled polyurethane foam on the right.
And you can see the similarity in the structures of those two things there.
Then we talked about tissue engineering scaffolds.
And if you remember, these two scaffolds here on the bottom were made from pig heart tissue.
And they're made by just removing all the cells.
So that actually is the natural extracellular matrix.
And then, these other structures up here, these were all engineered tissue engineering scaffolds that are made in a synthetic way.
And the idea is to try to mimic the extracellular matrix in the body.
So you can see the similarities there.
And then more recently, we were talking about sandwich panels.
So this is the example from the helicopter rotor blade.
That's from an aircraft flooring panel.
And this was the Irish leaf, and these were the bird skulls.
So the same sort of idea, that there's these engineering lightweight structures, and there's also similar things in nature.
And then, I think, last time, we also were talking about palms.
And the palm stems had density gradients in them.
And one of the things we showed was that by having that density gradient, the stress distribution across, say, the radius of the palm was almost matched by the strength distribution.
So it was a very efficient way to use the material.
And at MIT, that was a student in architecture who was looking at doing this with concretes with aerated or foam concretes and making a radial density distribution.
So he made beams.
He made columns.
He made different kinds of things.
With the concrete, there's a little bit of a limitation, because the concrete is much stronger in compression than it is in tension.
So if you had a beam loaded in tension or a concrete column that might buckle, you'd still have to have some reinforcing bars in there to take the tensile loads.
And one thing I think I didn't really talk about was animal quills and other sorts of plants stems.
And many of these have a structure that's made up of a cylindrical shell with a foam or a honeycomb core.
So here are some examples in nature.
If you look at grass stems, there's a dense layer on the outside, and then a foamy layer on the inside, and then just a hollow layer in the middle.
And if you look at porcupine quills-- this is a porcupine quill-- these are made of keratin.
They're like modified hairs.
So it has a dense layer on the outside, and then this foamy stuff on the inside.
This is a hedgehog spine here.
And again, you can see there's a dense layer on the outside and these ribs on the inside.
And this is the toucan beak-- you know, the toucans that live in Central America.
And the beak has a foam core.
Again, they're keratin structures, and yet the outside is solid.
But the inside has a foamy structure.
And Mark Myers did a paper on this a while ago.
So we got interested in these structures that have a solid shell on the outside but a foamy thing on the inside.
And we wondered if there was a mechanical reason for that.
And you can show that, at least in some of them, if, say, you have a grass stem-- it's really common in plant stems.
Do I have some more plant stems?
Yeah, here we go.
Here's a milkweed stem.
So it's got these dense fibers with this foamy core here.
And blue jay feathers-- feathers have this as well.
So they have an outside layer on the quill that's solid, and then an inside layer that's foamy.
If you look at things like plant stems, they blow in the wind.
And you can look at the buckling resistance of the stem.
And because there's this shell with the foam-like core, it's not just the overall buckling of the whole thing.
You can get that local face-wrinkling mode again.
So the outer shell can wrinkle.
And you can show that having a foam-like core helps prevent that wrinkling from happening, the same as with the sandwich panel.
You remember we talked about the face wrinkling on the sandwich panel?
Well, on these stems, you can get wrinkling of the outer shell.
And the foam helps prevent that from happening.
And you can show that you can actually-- for the same buckling load, you can reduce the weight of the plant stem, or the bird feather quill, or whatever by having that foamy core.
So people have looked at this too.
And there's a group in Germany who had looked at the idea of mimicking the horsetail stem.
So this is a plant stem here, the horsetail plant.
And they made something they called a technical plant stem, where they made this structure here.
And they made it out of fiber-reinforced composites.
And you can see, the little holes here represent those holes there, in the plant stem.
So the idea was to try to get something that was good at resisting the buckling, but at a lower weight.
And they were doing that with this thing here.
And there was a group in Japan that did a similar thing.
They used-- I think they took a copper tube, and then they took copper and aluminum wires and filled the copper tube with the wires.
They extruded that.
And then they melted out the aluminum, which, I think, also helped to soften and make the copper bond together.
And they got these structures here.
And you can see, that's similar to some of the plant stems as well.
So these are all examples of cellular structures that have-- mechanically efficient structures.
They're lightweight, and they're strong, and they're stiff.
And these natural structures have been mimicked in engineering applications.
So that's really all I wanted to talk about today.
But I think we wanted to use the rest of the class as a review.
So I haven't made a one-hour summary of the last six weeks, because that's not really possible.
So I thought I would just answer questions.
If you have questions, I'll try and answer them.
So for the test-- so, the test is on Wednesday.
You can bring one 8 and 1/2 by 11 sheet.
I wasn't going to give you all those honeycomb and foam equations, partly because-- the only thing I would really want you know is open-cell foams.
And I was hoping, by now, that you might have registered those equations somewhere in your brain.
So I'm not going to ask you for-- some equations are obscure.
I might ask you-- expect you to know what the Young's modulus of an open-celled foam is by now, or the axial modulus of a honeycomb or something.
But I don't think I'm going to ask you anything like, calculate air pressure contributions to the modulus of the closed-cell foam.
I don't think we're going to do that on this test.
So I think you should know what the modulus of a-- the Young's modulus of an open-celled foam is, the shear modulus of a foam, because you need that for the sandwich panels.
But you don't need reams and reams of those equations, so I wasn't going to give you those this time.
OK?
So Jenny, did you have-- so I finished the biomimicking thing.
We're just going to do a review.
I don't know if you want to stay or if you want to go.
You want to stay?
Well, whatever.
So do you have questions, Jenny?
I do.
I just wanted to have question 4 from the last pset reexplained.
Because I know that you explained it to in office hours, and I know that the solutions are online, and I looked at them.
I'm still confused
OK.
So, you're going to have to remind me what problem 4 is, because I don't remember
Question 4 says, polymethacrylate foam at solid strength of 3.0-- or, solid Young's modulus, rather, of 3.0-- gigapascals is being considered for the energy absorption layer in a bicycle helmet
OK.
I'll tell you what, I think I have it on my little disk here.
Maybe that's easier, because then I can read it
[INAUDIBLE]
Yeah, let me just see if that's going to come up.
There it is.
OK.
So this one here about the foam, about the--
The one with the graphs
--energy absorption?
I'm just a little bit confused by the graphs
OK. Let me see if I can make this bigger.
Hang on a sec, my computer's thinking.
OK. OK. And I think I gave you-- right, I gave you this graph here.
Right?
Yes
OK.
So can you read what I've got here, or is that-- should I make it bigger?
I can't really read it, [INAUDIBLE]
Does that help?
OK.
So I have to admit, when I put this together, somebody-- I can't remember who it was-- told me you thought it was overconstrained.
And it turned out it was overconstrained.
And then I said, forget the thickness.
Forget that I've given you the thickness, right?
So disregard the thickness
No, the velocity
Oh, speed-- the speed?
The speed-- all right, OK, the speed.
No, I don't want to register.
OK.
So from what I gave you, you can figure out the normalized peak stress, right?
So you can get-- so if I do this, is this good?
You can see what I'm pointing at?
So you can get, the peak stress is just the mass times the acceleration over the area-- are we good with that-- divided by Es, which I gave you.
So I think most people probably got the peak stress here.
And then, because I had given you the velocity and the thickness, I was thinking you could calculate the strain rate.
But if I don't give you the velocity, say we're not calculating the strain rate at this point here, OK?
So here, we're-- did I put this-- I don't know if I have the graph on this solution.
There we go.
So this point here is the 2.5 times 10 to the minus 4 for the peak stress.
And if you're not given the velocity, I think what I thought you would do then would be just assume a velocity.
And then you can check it at the end.
So since I had given you this velocity of 12, let's just say that's what we assumed.
OK?
So then you get a strain rate of 480 per second.
So let's just say we assumed that velocity.
Then we could scoot back over here.
So we know we're on this line here for the sigma p over Es.
And we want to be up towards the top of these different strain rates.
So if you look at the strain rates, see, the very last one at the top is 1,000 per second, and the next one's 100 per second.
So we're halfway in between those.
And they're so close together, you can't really read the difference.
But we're up here somewhere.
So then we read off a w over Es for that.
OK?
Are we good?
Then, if we know the w over Es, this is the number, here, that I read off.
You know what the Es is, so you can get w. If you can get w, w is in joules per cubic meter.
It's in energy per unit volume.
But you know the area, and you know the thickness.
So you can get the energy in joules.
And then-- oh, did I-- I must have rubbed that off.
Didn't have it on this version.
The version in my notebook, I think, calculated what the velocity would be that corresponds to this.
And I think it turned out to be 8 meters per second or something.
So I had assumed 12, and I think it worked out to 8.
And on those log log graphs, whether or not it's a strain rate of 480, or a little bit less, or a little bit more, you can't read the difference on these things.
OK?
Can you explain how this graph relates to the other graphs from lecture that were simpler?
Because we had ones that were all density, and ones that were all the same strain rate.
And I guess I'm just confused why
Oh, OK.
Hang on.
So let me see if I can-- hang on.
No, I think I know what you mean.
Let me see.
I want to pull up the lecture notes.
I think I'm finding the right thing.
Here we are.
So in the lecture notes, there was a thing that looked like this.
Is that what you're talking about?
Yeah.
So the top set are the stress strain curves.
So those are OK. You do a compression test, you measure that.
Then, the middle set, you take, say, for one density, for one stress strain curve-- say that's your curve, the middle one there, 0.03.
Say you loaded it up to some point, or say you looked at some point here, on the curve.
You would figure out, for that stress, what's the area under the curve up to that stress.
So you'd have a stress and an area under the curve up to that stress.
And you could then-- say we know what this foam is.
It's-- I don't know-- a polyurethane or something.
So say we know Es.
Then we could divide those two numbers by Es.
And we would plot that one thing.
Let me just walk over here.
So this is the 0.03.
So if it's in the linear elastic reading, it would be somewhere in here.
Then I would scoot along, say, to here someplace.
I would do a whole bunch of points all the way along there.
And at every point, I would say, what's the stress, and what's the energy absorbed up to that stress.
OK?
And then I would plot-- doot, doot, doot, doot, doot-- up here, I would plot all those points.
OK?
And then when I got to this part here, that corresponds to that part over there.
OK?
Are we all good with that?
OK.
So then we repeat-- so we get one curve on the middle chart.
Then, for the different densities and the different stress strain curves, we plot a different curve for each of the different densities doing the same process.
And the thing we notice is that these points here are really the optimum point.
Because at that point there, you absorb as much energy as you possibly can for that stress.
OK?
And we notice, happily, that those points lie on a line, basically, on a straight line.
And we tick of what the different densities are-- so 0.01, 0.03, 0.1, 0.3.
OK?
And that line there, and all of these stress strain curves, and all these lines here, curves here, they all correspond to one strain rate.
All right?
So now I could take that line there for that first strain rate, and it would be one of these thinner lines here for a particular strain rate.
And I would mark off-- just the same as I've got here, 0.01, 0.03-- I'd go 0.01, 0.03, 0.1, 0.3.
All right?
And then I would repeat this whole process again for a different strain rate.
So I'd go back to doing some mechanical tests at, now, a new strain rate.
And typically, the new strain rate is going to be bigger or smaller by a factor of 10 or so, because you're not going to see much difference in the behavior unless you get big changes in the strain rate.
So you're going to change the strain rate significantly.
You get a new series of these stress strain curves.
Then you get a similar-- very similar, but not quite the same-- series of these curves here.
And what you'd find is you'd have another line here that would be offset a little from the first one.
And the positions of where these densities were would also be offset a little from the first one.
And then you'd draw the second one.
So the second strain rate line would go here.
And the little density positions would be offset a little bit, and you would mark them off.
And you basically repeat it for different strain rates, and then you build this thing up.
And then the density lines connect too.
Is that OK?
So that's how you generate that.
So the idea is, this bottom diagram is really summarizing all of those shoulder points.
The bottom diagram is really summarizing all of these points where the stress starts to scoot up at the densification machine.
Yeah?
So in question 3, we were constructing the graph in the middle for this particular one.
What confused me is that on the graph, the variable on the x-axis is stress, whereas in the question, we were given the stress.
So I drew something that looked as it should have looked.
And it turned out to be right, but I'm very confused as to why
OK. Let me try and get rid of that.
And I'm going to have to remind myself what the question is again.
OK.
So we have an open filled aluminum foam.
I asked you to write the equations for each of the different regimes.
And those were straight out of the notes, I think.
Right?
Yeah, I think the three inputs were different relative densities
Yeah, and then you had to construct the energy absorption curve based on those equations.
And I give you three relative densities.
So this was my solution.
So all this stuff here, I think, was straight out of the notes.
So there was the-- oops, let me back up so we start at the beginning.
So there was the linear elastic part.
There was the stress plateau.
That was just as it starts to densify.
And then, there's that line joining up the points.
So you were OK with all of that?
Yeah.
No, the only part that confused me is because once I simplified-- I inputted all of what we were given such that I had equations in terms of the relative density so I could just apply it to the different ones given.
But then I wasn't very sure how-- because I got constants, I wasn't really sure how-- the slopes should look like, because they were constants
So you got-- I'm not sure what you mean about you got constants.
So what I did was I said, well, I know that the diagram has to have this basic shape, right?
In the first part here is where it's linear elastic.
And this part here is the stress plateau.
And that part there is the densification.
OK?
And I said, well, if I can find these two points that correspond to the change between the linear elastic part and the stress plateau and the point that corresponds to the stress plateau and the densification, then I've got the diagram.
Right?
OK?
Yeah, I think that--
Oh, can I stop talking now?
I mean, if you wanted to--
Well, it's for you.
So you're OK?
I think
You
So what confused me about this question was that in order to find the two points, you'd need to know the strain at which that [INAUDIBLE].
So I wasn't sure how you would find that strain
Yeah, I think I didn't tell you.
Let's see.
Well, let's see.
Do you have to have the strain?
You have that.
I think if you have-- I think you don't have to have it in terms of the strain.
I suppose you could put it in terms of the strain.
I mean, that would be a different way to do it
But isn't the equation for the stress plateau already in terms of the strain?
Yeah, but this assumes that the stress plateau is just perfectly flat, right?
So this thing here would be useful, the strain at which the plateau starts.
But I think that's the same as saying-- that's the same point as saying you're at that point there, because that's where the plateau starts.
OK?
So I think there's different ways you could try to approach this.
But I think-- let's see, I'm just trying to remember what I did here.
Yeah, so what I did here-- so to get point a, I said, well, this is the equation for the linear elastic bit, right?
And this is the equation for the stress plateau.
Where I'm at point a here, this stress is the stress plateau.
So that's why I've put sigma star plastic there.
And then I said, this is-- the sigma star plastic is-- and this works out to some number here, some number times the relative density to the 3/2 power.
And then I just made this little table here, where I said, OK, these are the densities I need to get the curve for.
This is going to be my sigma star plastic for those densities.
And from that, then I can get this column here.
It's basically just that equation with this substituted in.
And do you see that that corresponds to the point a?
So I mean, you could do it by figuring out what that strain is and then figuring out all the points along that line up to that strain.
But you don't have to do it that way.
Do you know what I'm saying?
So you're saying-- if I had-- so say I have my idealized stress strain curve like that, right?
You're saying, well, I had to know that strain there so that I know where this stops, right?
And I'm saying that corres-- and you could do it this way if you wanted to.
You could say that this is the energy absorbed up to that point.
And if you know that the modulus goes as the relative density squared times Es, and you know this, if you know those two things, you can figure out what that strain is.
So you could do it that way if you wanted to, but I just did it a slightly different way
Yeah, so I don't understand how you just were able to get rid of the epsilon minus epsilon on that curve
Oh, let's see.
So I think, in the first part, where it's linear elastic, I just go up to epsilon 0, right?
So this part here doesn't really involve the epsilon, because I've gotten rid of it by taking the square of the stress.
And then here, the other point I'm looking at is this point here.
And I've assumed that epsilon 0 is much smaller than epsilon d, and I've ignored it.
And that's, I think, how I did it in the notes in the class.
OK?
So when you get to that point, this strain here is usually a few percent at most.
And this strain here is typically 80% or 90%.
So it's pretty common to do that.
OK?
Other questions?
So don't forget, the test covers everything from the thermal conductivity of the foams, it covers the stuff on trabecular bone, sandwich panels, and then, the energy absorption stuff.
Yeah?
Are you a little exhausted already?
Yeah.
So I'm guessing that this week and next week, you have everything due.
You have papers, and projects, and tests.
Do you have very many exams left, like final exams?
Yeah, you've got finals?
[INAUDIBLE]
Oh.
All right.
Anybody else have any questions?
Because I think we can just go and do other things if nobody has any other questions.
So the test is on-- so let's just review where we're at for the rest of the term.
So the test's on Wednesday.
And you can bring one cheat sheet, but I am not giving you all those equations with honey combs and foams.
So I think, on your cheat sheet, you would want to put Young's modulus of an open-celled foam, shear modulus of an open-celled foam, compressive strength of an open-celled foam, shear strength of an open-celled foam.
But I don't think there's going to be anything more complicated than that.
And Monday, I was going to do the how I became a professor talk.
So if you've seen it and you don't want to see it again, you're welcome to not come.
But for you guys, I just talk about how I got here.
And it's about my life.
It's not about cellular solids or anything.
And I don't know, you guys liked it.
You like it, don't you?
Yeah
Yeah.
Yeah.
So if you want to come, I'll do that.
And then Wednesday, I thought, I just need to collect the projects.
I wasn't going to do anything on Wednesday.
And so I've been thinking about the how I became a professor talk.
And I think-- so I have this idea that students would like to hear more of these talks from other faculty.
Would that be true?
Yes
Ah.
Because I've been in touch with Cindy Barnhart, and we're going to try and organize something for the fall.
So I'm going to approach some other professors-- not just in our department, across all of MIT-- and see if I can get other people to do the how I became a professor talk.
So you would like that?
OK, yeah.
So we'll see if we can make that happen.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
--not of the vindictive sort.
You skip class, you've skipped a lot of important stuff.
But I'll get you on the quiz, that's all.
Kidding aside, there were a number of things that I passed out.
I think nobody needs a set of problem set number 13.
That was the one that had something on symmetry constraints and working with second-rank tensors.
If anybody missed that, you can see me during break.
Some people, I think, missed problem set 14.
And that's the one where you are invited to diagonalize some tensors, either using the method of successive approximations or the direct diagonalization- by-an-eigenvalue procedure.
Anybody need one of that?
And I'm sure that nobody has a copy of problem set number 15, which deals with piezoelectricity.
And I know you don't have it, because I just put it together.
So I'd like to hand it out and invite you to explore things that deal with third-rank tensors.
And I hope that, even though doing the problem sets is optional, particularly at this juncture in the semester when things have come to a set of successive crunches.
But if you don't know how to do it, for goodness' sakes, come see me.
Or raise it in our next class.
You know, some question like, I haven't the foggiest idea how to do problem number two.
Could you say a little bit about that, please?
And I'd be happy to oblige.
All right.
I will have for you next time the quizzes and also all the problem sets which will have been turned in up to that point.
And what I spent my time doing instead is writing out notes for those people who missed the last lecture, and also notes covering what we're going to do today.
Because it is very exquisitely intensive, algebraically.
It's not hard, but there are a lot of variables with a lot of subscripts.
So let me pass this around.
I'll split it up into packs.
These are notes on some basic relations in electromagnetism which you may or may not have forgotten.
Take one off the top.
And it's coming at you from either side, so you're going to pass it back.
And the notes also cover everything that we're going to do on piezoelectricity.
Most of it will take place today.
And I can zip along a little more rapidly if you have notes to follow.
I would like to ask you when you get a set of the notes-- I could really kick myself-- the introductory discussion reminds you of the definition of a dipole.
And down in the middle of the page, on the cover sheet, two different types of polarizability are defined.
And one of them involves the separation of charge on an individual atom.
And that is called, quite appropriately, the electronic polarizability because it involves polarization of the electrons and protons on the individual atoms.
And then there's another type of induced dipole moment that comes when the structure is ionic.
And then an electric field will pull positive ions in one direction and negative ions in the opposite direction.
And that is very often referred to as the ionic polarizability.
And it's easy to keep them straight.
One involves electrons, which all atoms have.
The other involves ions.
And not all structures and materials have ions.
So the second one is unique to ionic structures.
And then, these are sometimes also referred to as the dielectric polarizability.
And I meant to purge that from the notes.
And as I put this together to xerox it, I grabbed the uncorrected sheet.
So please, just below you see the displaced positive and negative ion on the middle of the page, cross out "dielectric" polarizability.
And change that to "ionic" polarizability.
And I didn't catch that.
There's one other little typo as we go partway through.
In any case, we'll talk about third-rank tensor properties.
We'll introduce some other ones other than piezoelectricity later on.
But piezoelectricity is one of the primary examples of a third-rank tensor property.
And it's one that has a lot of applications in devices-- pressure sensors, audio equipment, electronic devices.
It's a very important property in terms of devices and present-day technology.
But there are others.
And we'll cover those in due course.
Some of them are rather exotic.
OK.
But to review these basic concepts in electromagnetism, we remind you again of the definition of a dipole.
We mentioned this last time.
But to quickly review, a dipole is a pair of charges of opposite sign but equal magnitude, separated by a separation, d. And then one defines a dipole moment, which has a vector character, as the product of one of the charges of magnitude, q.
And the vector that separates the two charges in the sense of the vector is defined as going from the negative charge and pointing towards the positive charge.
It's purely a definition.
But the reason it's convenient is that the vector sense and the product of charge and separation comes up again and again in all sorts of problems, among them definition of the piezoelectric effects.
Also a dipole in an electric field is going to experience a torque because the electric field will pull on the positive charge in the same direction.
It will pull on the negative charge in the opposite direction from the field.
And that's going to create a torque on this little gizmo.
OK. Now it's important, too-- and interesting to note-- that there are three different kinds of dipole moments.
There are some molecules-- and water is the primary example.
Water has seen an asymmetrical arrangement of hydrogen, relative to the oxygen ion to which they are connected.
And that gives water a permanent dipole moment, which is what makes water such a darn good solvent.
And its ability to dissolve primordial juices probably accounts for our fact, intelligent design notwithstanding, of why we are here today.
On the other hand, dipoles, as I was just saying, can be induced when you impose an electric field on matter.
And these are of two kinds.
One is the dipole moment that is induced on an individual atom.
And that results in displacement of the positive nucleus relative to the negative electron shell.
It's found that the dipole moment is proportional to the magnitude of the electric field.
And the proportionality constant, alpha, is called the polarizability.
And for an individual atom, as I said a moment ago, it's defined as the electronic polarizability.
We write it as a scalar quantity, but actually, by now you're probably sensitized to being a little bit skeptical when things that relate to vectors are described as a scalar.
And in fact, the electronic polarizability is not a scalar, it's a tensor.
And one should really write that the i-th component of the dipole moment is given by alpha i,j times the j-th component of the electric field.
Second type of induced dipole moment involves, again as we said a moment ago, the effect of imposing an electric field on an ionic structure.
And again, the field will pull the positive ions in one direction and negative ions in the other.
And here quite clearly, if this pair of ions is in a structure, and that structure has some symmetry, we really have to consider this second origin to induce dipole moments as a tensor.
And this is referred to as the ionic polarizability.
So both of these types of dipole moments will be present in matter, in general.
The relative importance of each depends on whether the electric field is a static field or an oscillatory electric field.
And then the frequency dependence of these two polarizabilities has a consequence on the magnitude for fields of different frequencies.
And not surprisingly, the ability of the ions in the structure to polarize is going to poop out with high-frequency electric fields a lot quicker than just the displacement of the light electrons about a positive nucleus.
So there is a frequency dependence of the net polarizability.
We won't go into that.
But just keep in mind that at very high-frequency electric fields, the ionic polarizability will damp out.
OK. Then we went through a rather simplified but amusing model for the electronic polarizability.
And there's some rather severe assumptions that are made.
But making those assumptions let's you get a rigorous result, which tells you something about the electronic polarizability.
So we model the electron distribution on the atom as a uniform charge density in a distribution that goes up to some radius, r, and then quits.
So there's a sharp cutoff to the distribution of electrons, which is obviously ridiculous.
You know there are a collection of wave functions that give you charge probabilities that tail off slowly to large distances, getting progressively smaller and smaller.
So this is not terribly realistic.
And then the other thing we assume to make this model, something that we can solve exactly, is that the nucleus and the electron distribution displace as units.
In other words, we start with a sphere of electrons, the center displaces, but it stays a sphere of uniformly distributed electrons.
And then having made those assumptions and having lost any credibility for the model, if we carry through to see what the model predicts, it's rather interesting.
We use a fact, again, known to freshman and sophomore students of electromagnetism, that a charge inside of a uniform distribution of charge experiences no force.
And that's surprising, but it's something that you are very often invited to do on problem sets.
So therefore, if the center of the electron distribution is displaced from the nucleus, then the restoring force between the electron sphere and the nucleus is simply the coulombic force between a nucleus of charge plus ze, and a fraction of the total number of electrons, namely that fraction of the electrons which are contained within a sphere that has a radius equal to the displacement.
And that geometry and that algebra's carried out for you on the bottom of the first page.
If you set that up and ask what the dipole moment will be, it comes out beautifully simple.
It comes out to be equal to whatever proportionality constant you use in Coulomb's law.
I use rationalized MKS units from force of habit.
So there's a 4 pi epsilon 0 in there, then times the cube of the radius of the electron distribution, times the electric field.
So the two items of note that come out of this simplified treatment is first of all, the induced dipole moment is proportional to the applied electric field, which is what we assumed.
And so therefore, the polarizability, which is the quantity that relates the dipole moment to the magnitude of the field, is a constant.
And it turns out to be equal to 4 pi epsilon 0, times the radius.
So not only does this tell us that the electronic polarizability is something that relates dipole moment in direct proportion to the magnitude of the field.
And secondly, the electronic polarizability involves the radius of the charge distribution, cubed.
And even on a qualitative basis, this is interesting.
It tells you that high-atomic-number, big, fat ions are going to have a very, very large polarizability.
And things way down in the periodic table, like beryllium and lithium, and other low-z atoms, are going to be tough little nuts that don't display much polarization at all.
And in point of fact, several individuals have tabulated empirical sets of electronic polarizabilities.
One of the earliest ones are the so-called TKS values published a long time ago by Tessman, Kahn, and "Wild Bill" Shockley.
And I give you the reference to those.
There is another set of values that were assembled by a fellow at DuPont named Bob Shannon.
And I'll give you a citation to those values.
But in any case, if you look at these values, you find that the cation that has highest electronic polarizability is thallium, way down on the bottom of the periodic table, next to lead.
And that's just a big, fat, flabby atom that can be deformed very, very easily.
How do you get these polarizabilities?
Well, the equations at the bottom of page two-- which we won't make any use of but, nevertheless, will tell you where they come from-- there's a relation between the square of the index of refraction and the sum of the polarizabilities of the individual species, times the number of those species per unit volume.
And that is something that's called the Lorentz-Lorenz equation, equation.
I can't resist saying everything twice, Lorenz and Lorentz.
You can put this in another form that involves the molecular weight of a molecular structure and the polarizability per molecule.
It's the same equation, but for organic compounds.
It's a useful form.
And it turns out that the dielectric constant of the material is directly related to the square of the index of refraction.
So you can write those two equations in terms of either the dielectric constant or the index of refraction.
And if you substitute dielectric constant in place of n squared in those equations, the equations get new names.
And they're not called the Lorentz-Lorenz equation, equations.
They're called the Clausius-Mossotti equations, which shows you that sometimes fame and immortality can be gained simply by a trivial substitution of variables.
Nice to keep in mind if you can find something like that.
Finally, we don't see dipole moments on individual atoms or molecules.
We see evidence of polarization in bulk.
And on page three is a little model that reminds you of what the polarization of a material is.
And that's defined as simply the dipole moment per unit volume.
And that's represented by a capital P rather than a small p. And this is also a vector quantity.
And we can see how this is related to the individual dipole moments in the solid by dividing the solid up into individual cells.
And I use the term of "cell" loosely.
Is this a unit cell?
Is this a box around each of the atoms?
It really doesn't matter.
Because all of these little dipoles on the individual unit cells are packed together back to front in the solid.
So internally, the negative end of one induced dipole moment is always adjacent to the positive end of the neighboring dipole moment.
And everything cancels out internally, except for the two surfaces of the solid.
So macroscopically, if you impose an electric field and it induces dipole moments, what happens is you see that there is a charge on the surface of the piece of material.
And that is a real charge.
It's a bound charge.
You can't draw that charge off as a current.
Because the minute you reduce and eliminate the electric field, those dipoles disappear.
And the charges all go back to where they came from.
So this is the thing that keeps the exact model that we use for one of the cells in the solid [INAUDIBLE] of no importance whatsoever.
You could say that the dipole moment on each atom is the charge times the separation of the positive and negative charge.
And qi is the induced charge.
The number of cells per unit volume, if we say that they're little cubes of the same edge length, delta, is unit volume one divided by the volume per cell, which will be delta cubed.
And the dipole moment per unit volume is the number of cells per unit volume times the polarization on each.
And you find that everything drops out.
And the polarization indeed turns out to be equal to induced charge per unit area, whether that's induced charge per unit area of one of our cells or on a square centimeter of material.
So polarization, dipole moment per unit volume is numerically equal to, and physically equivalent to, an induced charge per unit area.
OK.
So much for freshman electromagnetism in fast forward.
And now I'd like to turn to something that involves tensors and materials.
As I mentioned a moment ago, piezoelectricity, literally "pressure electricity," is a very technologically important property, useful property, but also provides a nice example of a tensor property that has to be defined in terms of a sensor of third rank.
There are a number of different piezoelectric effects.
The first one is the so-called direct piezoelectric effect.
And it refers to the fact that if you subject a material to an applied stress, it will develop a surface charge.
Remove the applied stress, the surface charge goes away.
The surface charge can be described in terms of a polarization, a dipole moment per unit volume.
And it will be in direct proportion to a stress tensor sigma j,k.
And so what we'll assume-- and this is an assumption.
And it's followed, except for very extreme conditions, that each component of the polarization, p sub i, is given by a linear combination of every one of the nine elements of stress, sigma i,j.
Oops, not i,j.
I have to use a different index.
I'm used to writing i,j.
So each component of the polarization, where i ranges from 1 to 3, is given by a linear combination of all nine of the elements of stress, sigma j,k.
And the proportionality constants, di,j,k, are known as the piezoelectric moduli.
And this is called the direct piezoelectric effect.
OK. That's simply an assumption that the polarization is proportional to the applied stress.
We know that stress, however, is a field tensor of second rank.
We know that the polarization has the character of a vector, charge times the length.
We know how second-rank tensors transform.
We know how first-rank tensors transform.
And therefore, knowing that, we can say that the coefficients, the array of coefficients, di,j,k, will transform like a third-rank tensor.
And therefore they are a tensor.
So we will have three components of polarization.
And we will have nine components of stress.
And so there will be 27 piezoelectric moduli, di,j,k.
So things go up rapidly in terms of number of coefficients as the rank of a tensor increases.
Now let's take a look at the nature of these equations.
We'll have, for example, the x1 component of P being given by d1,1,1 times the element of stress sigma 1,1.
The next term will be d1,2,2 times sigma 2,2.
I'm putting down the tensor components first.
But this is just an equation.
I can write the terms in any order.
Next will be a d1,3,3 times sigma 3,3.
And the next one would be a d1,2,3 times a shear component of stress sigma 2,3 and other terms.
I don't want to write out all nine of them.
Because I think I have written enough to make my point.
The point is that the value of i is always tied to the component of the resulting vector that we're defining.
But the second pair of subscripts, j and k, always go together as an unseparable pair with the component of stress that they modify.
So the 1,1 here goes with the 1,1.
The 2,2 goes with the 2,2.
The 3,3 goes with the 3,3.
So why in the world, if they're always going to go together, do we have to use two indices to define the piezoelectric moduli?
Why don't we use a single symbol?
Well, there is a good reason for not doing it.
But we'll issue that caveat later on.
We could use an a, a b, and a c, or an alpha or a beta or a gamma, or some other esoteric symbol.
But what makes sense since we're using Arabic numerals to represent subscripts, let's write the pair of indices in terms of an index that's related to the stress tensor, which is sigma 1,1; sigma 2,2; sigma-- whoops.
Sigma 1,2; sigma 1,3.
And then comes sigma 2,1; sigma 2,2; sigma 2,3; sigma 3,1; sigma 3,2; sigma 3,3.
We know that this is a symmetric tensor.
So these off-diagonal terms are always equal to one another.
So we're numerically going to have to enter each of those twice.
So to define a single index that represents the components of stress.
And I'll describe it in this fashion so you can remember, as a mnemonic device, what makes this work.
We'll go down the main diagonal of the stress tensor, this fashion.
And then having reached the bottom, we'll go up along the right-hand side and then jog over to the left to pick up the last term.
And we'll define a number associated with these indices that goes in the form 1 to 2, to 3, to 4, to 5, to 6.
So we'll just use these three integers, ranging from 1 to 6, to represent pairs of integers in the stress tensor.
So things tidy up quite nicely then.
This would be P1 is d1,1 times sigma 1; plus d1,2 times sigma 2; plus d1,3 times sigma 3; plus d1,4 times sigma 4.
And now, uh-oh, Houston.
We've got a problem.
Because there's another term in here that is d1,3,2 times sigma 3,2.
And sigma 3,2 is required, because the stress tensor is symmetric, to be numerically equal to sigma 2,3.
So this is really d1,2,3 times sigma 2,3.
And then we've got a d1,3,2 times a sigma 3,2.
I know that this is equal to this.
But is there any reason d1,2,3 has to be equal to d1,3,2?
I can't see any reason why.
OK. Let me confide in you that, yes, they are equal.
But we can't show it or claim it on the basis of what we've got before us right now.
It's going to come later.
But we can show that they are equal.
OK.
But equal or not, this means we'll have a term d1,4 sigma 4, and we're going to get it in there twice.
So when the subscripts go up to 4, we're going to get a 2 out in front.
And then we'll get for the term sigma 1,3 and 3,1 we'll have a d1,5 times a sigma 5.
This would be sigma 1,3.
And then we'd have another d1,5 for sigma 5 again.
And this would be sigma 3,1.
So what are we going to do?
We're going to say, well, it's nice we're getting rid of an unneeded subscript.
P1 is the d1,1 times sigma 1; plus d1,2 times sigma 2; plus d1,3 times sigma 3.
Are we then going to say 2 d1,4 times sigma four, and say that the coefficient here is di,j when i is equal to 1, 2, 3.
But the coefficient is 2 di,j when j is 4, 5, or 6?
Hell of a matrix that would be.
That's going to be a bother.
Since it doesn't seem that we can measure these two coefficients, d1,2,3 and d1,3,2, independently anyhow, let's just write our relation in reduced subscripts with the two added together as the element d1,4.
So we are going to define d1,4 equals d1,3,2 plus d1,2,3.
So that's a definition.
And d1,5 will be defined as the sum of d1,1,3 plus d1,3,1.
And we're going to define d1,6, finally, as d1,1,2 plus d1,2,1.
OK.
So if we do that, then and only then are we entitled to write a nice, simple, compact little nugget that says that in our redefined form, Pi is equal to di,j times sigma of j.
And i goes from 1 to 3, and j goes from 1 to 6.
And this is kind of a neat compact, easily managed outcome.
So the moral of this story is, I like to think, is you can have your cake if you eat its 2.
Oh come on, this is a tough, tough crowd.
You're just all worn out from the MRS meeting.
That'll do it to anybody.
So why do we worry about the fact that to the direct piezoelectric effect should be a third-rank tensor?
The answer, my friends, is that this is no longer a tensor relationship.
It's a matrix relationship.
A tensor is a matrix, but it's a matrix with a difference.
It's a matrix for which a law of transformation is defined, if you change axes.
There's no such law defined for the di,j's.
So they qualify as a matrix, but they are not a tensor.
So to attempt to say-- as you might be inclined to do when we raise the issue of what the symmetry restrictions are on these tensors-- if you attempt to say, well, I'm going to find d2,1 prime.
And that's going to be c2,i c1,j times di,j.
Wrong.
You go down in flames because that's just nonsense.
That's just nonsense.
So this is a matrix relation.
And if you want to do exercises such as slice a piezoelectric wafer out of a piece of quartz, and then having done so, refer the properties to axes taken along those edges of the plate that you've cut, you've got to-- in order to find the piezoelectric moduli for that plate in that new coordinate system-- you've got to be prepared always to go back to the full tensor notation.
If you want to derive symmetry restrictions, which we're going to do whether you want to or not-- but we won't do them exhaustively-- you've got to go from this matrix notation back to the full three-subscript tensor notation.
OK?
Yes, sir?
Couldn't you extend c sub i,k's to be [?
3,6's ?]
and extend most others [INAUDIBLE]?
Oh, yeah.
No, if we would write all these down, we'd have-- I don't know if that's what you're asking-- but we'd have P1 is equal to d1,j times sigma sub j; P2 is equal to d2,j sigma sub j; and P3 is equal to d3,j times sigma sub j.
So I just did that for the line with i equal to 1, because I was too lazy to write down all three relations.
Is that what you're asking?
No, I'm saying you could have a transformation model if you were to extend a c sub i,j matrix to be 3,6
No.
It just won't do it.
I mean, think of what these indices are.
They're 1, 2, and 3 standing for the x1 direction, the x2 direction, the x3 direction.
What's the x5 direction?
It's just not defined.
Good try, but you just can't do it
I'll think of something next time
Oh, I'm sure you will.
Whether it will work or not is an absolutely different question.
OK.
So beware.
Do not try to transform tensors of third rank expressed with two subscripts to other coordinate systems.
All right.
What I would like to do now, then, is to examine the symmetry restrictions that are imposed on a third-rank tensor by crystallographic symmetry.
And these are embedded in the notes that you have.
But it's going to be bothersome to leaf through those.
So I separated out the pages separately.
And I'm going to look at a few transformations so that you can see how they work, and then point out some very, very curious consequences of this, which are non-intuitive.
So first of all, how would we do it?
If we want to get the form of di,j, the matrix for-- let's do a very, very simple one.
Well, let's do one that we did last time, for those who were here, for inversion.
For inversion, the direction cosine scheme is c1,1, 0; 0, 0-- I'm sorry.
For 1 bar, the transformation of axes is c1,1, 0, 0; 0, c2,2, 0; 0, 0, c3,3.
So we'll take di,j, change it to the full three-subscript notation, di,j,k.
And the law for transformation of the third-rank tensor is di,j,k prime is equal to ci, capital I; cj, capital J; ck, capital K; times d, capital I, capital J, capital K. We've not done this much.
But we've mentioned quite some time ago that this is the way a third-rank tensor would transform element by element.
The only terms that are non-zero in this array, when the symmetry transformation is inversion-- where x1, x2, x3 goes to minus x1, minus x2, minus x3-- is whatever i is, minus 1; whatever j is-- only the diagonal terms are there, and they're all minus 1-- whatever k is, it's going to be minus 1, times di,j,k.
So we find that for inversion, di,j,k prime is always, regardless of what the three indices are, is going to be minus di,j,k.
And if inversion is a symmetry operation which the crystal possesses, we're demanding that the transform index be identical to the original index.
But there's a minus sign in there.
So we can say that every single element vanishes, has to vanish.
So any crystal that has inversion in it is not going to be able to display the piezoelectric effect or any other third-rank tensor property.
The electro-optic effect, piezoresistance, and there are a whole slew of them, which are examined for you in part on this sheet.
One third-rank tensor property is the direct piezoelectric effect, which we've been discussing as our example.
There is something called the converse piezoelectric effect, which describes the phenomenon where an applied electric field creates a strain.
So the direct effect is you [?
scush ?]
your material, you develop charge.
The converse piezoelectric effect says if you apply an electric field that's going to move the atoms around and induce polarizations, you are going to create a strain.
And the absolutely mind-boggling thing is that the same array of 3 by 9 coefficients describe both the direct piezoelectric effect and the converse piezoelectric effect.
So in one relation, we have that each component of polarization is di,j,k times the nine elements of stress.
And that means we have to write three equations in nine variables, the elements of stress.
And the converse piezoelectric effect, we're developing a strain.
And there are, therefore, nine elements of strain.
And we're applying a vector, e sub i, a field.
So there are three components of that vector.
27 elements, 3 times 9.
27 elements, 9 times 3.
So both of these are third-rank tensors.
Strain transforms like a second-rank tensor.
Field vectors transform like a first-rank tensor.
Therefore, the coefficients are elements of a third-rank tensor.
But what boggles the mind is that the same 27 numbers describe these two seemingly disparate phenomena.
You don't prove this by symmetry.
You don't prove this by the nature of stress and strain.
This hinges on the thermodynamic argument, which I'm not going to go into.
But what you do if you're willing to stretch tensor notation a little bit-- you can't have a 3 by 9 array when you've got a tensor of second rank on the left and a vector on the right.
You have to write this in this fashion, that di,j,k times ei gives you the element of strain, epsilon j,k.
And that's not proper tensor notation.
But we're not going to quibble.
Because if you allow this little departure from convention, then you can write both the converse piezoelectric effect and the direct piezoelectric effect in terms of the same matrix.
But again, that is not intuitively obvious.
It does not have to be the case.
And it hinges on an argument in thermodynamics.
Some other examples of piezoelectric relations that I've mentioned in the notes.
If you apply a stress and you get a polarization, that stress produces a strain.
So you also have to get a polarization if you're applying a stress, and write it in terms of the strain that's produced.
Similarly, if you apply an electric field and it produces a strain, the crystal must be in a state of stress.
So there must be relation between stress and applied electric field, which will also be a third-rank tensor.
Those two relations are represented by coefficients given the symbol e. And again, the same array of 27 elements describes both effects.
And these are not dignified with any special names.
They're just tensor relations that have to be true because of the fact that stress and strain are coupled by elastic relations.
Not surprisingly, then, the coefficients e must somehow involve the piezoelectric moduli di,j and the elastic properties of the material.
Another effect which is a very interesting one is the electro-optic effect.
If you apply an electric field to a material, you change the birefringence of the field.
The birefringence is defined as the difference in index of refraction for light polarized in two orthogonal directions.
Another tensor effect-- and I'll give you a note defining the terms next time we meet-- there is a piezoresistive effect, which you don't see talked about very much.
But that's a property that Texas Instruments was interested in at one time.
And I have a set of sheets defining those relations that the presenter of a paper at a meeting, one time, kindly gave to me.
OK.
So what other symmetry restrictions are there?
Having shown that inversion will not permit any third-rank tensor property, we have gone from 32 point groups, lost interest in the 11 centrosymmetric point groups.
And so there are only 21 piezoelectric point groups.
And the way we would plod through all 21 of them would be to simply define, starting with a twofold axis.
Let's say a twofold axis parallel to x3 would correspond to a change of axes ci,j that describes the new axes in terms of the original ones that consisted of minus 1, 0, 0; 0, minus 1, 0; 0, 0, 1.
So if we look at an element in the piezoelectric matrix-- something like d1,6-- d1,6 actually corresponds in tensor notation to d1,3,2 plus d1,2,3.
And d1,3,2 prime is going to be c1,i c3,j c2,k times all of the original tensor elements di,j,k.
The only term of the form ci, something that is non-zero is c,1,1.
And that has a value, minus 1.
The only term of the form c3, something which is non-zero is c3,3.
And that turns out to be plus 1. c2, something, the only form that's non-zero is c2,2.
And that has value, minus 1.
And this should be times d. And the only value of i that stayed was 1.
The only value of j that stayed was 3.
And then only value of k that stayed was 2.
So this says that d1,3,2 should be equal to d1,3,2, which I don't like, unless it's supposed to be negative.
I did something wrong here.
Well, you see how easy it is, even if it didn't turn out right.
And the form of the tensor for monoclinic crystal of symmetry 2, with a twofold access parallel to x3, has as shown in the lower left-hand corner of the handout on symmetry restrictions, it has eight non-zero terms.
If you do the same thing for a mirror plane perpendicular to x3, you find that there are 10 non-zero terms.
So we don't have the situation where all of the point groups that are able to show the property within a given crystal systems like monoclinic have exactly the same form of the property tensor.
In fact, you'll notice a curious correspondence between the restrictions for symmetry 2 and symmetry m. All the terms that are 0 in symmetry 2 are non-zero in symmetry m, and vice versa.
All the non-zero terms in symmetry 2 are 0 in symmetry m. And the reason for that is simply that this is the form of the direction cosine scheme for symmetry 2.
The form of the direction cosine scheme for symmetry m, where the m is perpendicular to x3, would have the form 1, 0, 0; 0, 1, 0; 0, 0, minus 1.
So ci,j, for a mirror plane perpendicular to x3, is exactly the negative of the direction cosine scheme for a twofold axis parallel to x3.
And since the number of direction cosines is odd, this means that everything that has an equality between the di,j,k's for m would have the transformed element be the negative of the original one for 2, and vice versa.
So that's why there's this complementary form of the tensors for the two monoclinic symmetries.
Point out a couple of curious things in the tables.
You really have to go through 21 of the symmetries independently.
And you find that some of them do come out the same.
Symmetry 4 bar 3m and symmetry 2:3 have restrictions of the same form.
But for the most part-- Yes?
I know why this is wrong
Why is that wrong?
Because your notation of d1,6 is in fact not d1,3,2 but d1,2,1.
Since you have two same indices, [?
d3 ?]
and minus [?
d1.
?]
OK. d1,6; d1,6.
Ah, of course.
Of course.
d1,2,6 is the one up here.
And that's 1, 2 and 2, 1 for the strains.
OK.
Thank you.
So this is d1,1,2 plus d1,2,1.
And so we would have 1 by 1,k and 2,k.
1, 1, and 2.
And 1, 1, 1.
1,i; 1,j; and 2,j for this one.
So we'd have c1,1 c1,1 c2,2.
c c1,1 is minus 1; c1,1 is minus 1; c2,2 is minus 1.
So d1,1,2 is minus d1,2,1, which means they have to be identically 0.
Thank you.
I'm sure nobody cares at this point.
Very good.
OK. Another curious thing that happens is that for cubic symmetry 4:3:2, it's acentric.
But every single modulus is 0.
And the reason, the explanation, is there's so many different transformations which have to leave the tensor invariant that the poor tensor just can't do it.
It gives up, packs up, and goes home, leaving all the elements zero.
Just no way you can get all the qualities to be satisfied.
Another curious result that I point out for some of the hexagonal symmetries for symmetry 3:2, for symmetry 6:2:2, and for 3 over m and 6 bar 2m, all the elements of the form d3, something are identically 0.
Which says you simply cannot create a polarization that has a component P3.
You just cannot create a polarization perpendicular to the axis of high symmetry.
Yeah.
Got a question?
So all of the cubic that has 4:3:2 symmetry or [?
4:1:0 ?
], that means you can't get--
You just don't have any piezoelectric effect, even though the crystal is not centrosymmetric, requiring that all those transformations leave the tensor invariant; simultaneously, require that everything has to be 0.
One final thing, and then I'm running a little bit over.
But I'd like to go on to other aspects of piezoelectricity during the next hour.
Remember something that we've shown and which I've asked you to look into again on one of the problem sets, and that is that the trace of the strain tensor gives you the change in volume.
The first 3 by 3 blocks of this tensor that we've been looking at, were we to write the elements of strain, epsilon i,j, in terms of d-- epsilon j,k-- in terms of di,j,k times e sub k, it is this block in here which enters into the terms epsilon 1,1; epsilon 2,2; and epsilon 3,3.
These terms give you the fractional change in volume.
And the elements that are involved in the piezoelectric matrix are these terms in here.
So if all of those terms are 0, it turns out that when you apply a stress and look at the electric field, or apply an electric field and look at the strain, if you apply a field and all of those nine terms are 0, you cannot create a strain.
So there's no volume change.
There can be a strain, but no volume change.
The only deformation you can create is pure shear.
So if you drive a piezoelectric oscillator with an electric field, for those property tensors for which that first 3 by 3 block are all zero, the thing can shear-- in the water, for example-- but it can't pulse.
It can't have a volume change.
And if you were to create a transducer for sonar applications, what you would want to do is to have piezoelectric device which expanded this way, to create a sound wave going through the water.
If it just goes back and forth, it's going to slosh back and forth in the water and not create any sonic wave that could be used in sonar.
Now that is true for elements being identically 0.
It turns out it's also true for elements such as 4 bar, where two of those terms in the 3 by 3 block are the negative of one of the other.
These will also have no volume change.
So that's another very curious property of piezoelectric response that follows from these symmetry restrictions.
OK. That's enough for our first session.
When we come back, we'll look at the converse piezoelectric effect.
And we'll also look at representation surfaces for specific piezoelectric devices and ask if it's possible for a third-rank tensor to have a representation surface that's analogous to the representation quadric for second-rank tensors.
So I'm sure you'll all want to come back and hear the answer to that question.
The following content is provided by MIT OpenCourseWare, under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
The final quiz is scheduled for a week from today, on December 8.
Following that weekend, we'll have one class on the 13th.
And the last days of classes are Monday, Tuesday, and Wednesday, the 12th, 13th, and 14th.
So I know a number of you spoke up last time and said that you have a conflict on Thursday, the eighth, with a quiz in another class.
So could I see a show of hands again of how many of you have this conflict?
So three, six people, seven people, you're going to enroll in the other course just so you can put off taking the quiz.
That's unfortunate, I don't think anybody should have to take two quizzes in one day.
We can't move it up.
We'll have to move it back.
So I don't know if I am violating any Institute rule, but I know that it is strictly illegal to give assignments that are due after the last day of classes, let alone have a quiz after the last day of classes, which has not been previously scheduled as a final examination.
So we can't give it after Wednesday, the 14th, the last day of classes.
So, do any of the six impacted people have a strong preference for when we should schedule the quiz?
Monday?
Due it Monday?
It doesn't have to be that soon.
Monday?
Why not Tuesday?
Because Tuesday we have a class
Or Wednesday?
Well we would have a Thursday class instead.
Correct?
No.
If we're doing it after the quiz next week, there's a class.
We have a last meeting here on the 13th.
And I'm not going to shift it for other people, just for the impacted.
So of the six who are entitled to vote, how many would prefer to have it Monday, the 12th?
Three and I know how it's going to turn out.
And for Wednesday, the 14th?
Three
I can't take the test because-- not because of another test-- I'm out of town.
And I might come back Saturday.
And Monday is just a little quick to take it after we get back
I think I'm going to have to make an executive-- yes
Professor, when you mean by conflict, do you mean having an exam at the exact time?
No, it's on the exact same day.
Considering this is a two hour examination, to have another examination exactly the same day, if you're not brain dead after an hour or two, you will be pretty close to it.
So I think that is an unfair penalty to pay.
I think, to keep it as close as possible to the quiz that the rest of the people will be taking, why don't we make it-- since it's a tie vote-- on Monday, the 12th.
Do it sooner, rather than later.
And I'll let you know next time of where we will hold it.
I'll have to arrange a room for it.
So that will not be a make-up quiz.
It would be a made-up-- invented-- quiz.
And, as I say on Monday, I should have all of the homework and the previous quiz to return to you.
Reason you don't have it now is all the little crabbed handwriting that you see before you in the form of these notes, which takes forever.
All right, so since we've satisfied the unpleasant aspects of the end of the term, let's get back to discussing some of the other piezoelectric effects that we have defined.
And then also ask the question rhetorically, is there anything like a representation surface for a third-ranked tensor property?
And it doesn't look promising.
But there are some things that we can do to discuss variation of properties with direction, and we'll see directly what those are.
But first let's look at the converse piezoelectric effect.
And this again is a third-ranked tensor property, but what we do is to have the elements of strain, epsilon ij, a second-ranked tensor.
And to take advantage of this curious relation between the directed converse effects, we define the converse piezoelectric effect as giving you nine elements of strain in terms of a third-ranked tensor dijk times e sub i.
So what is not standard is our convention for the order of the subscripts on the moduli.
And we make up the rules, we can do it any way we like.
And the advantage of defining it this way, in nonstandard tensor notation, is that we can use the same coefficients for both the direct and the converse effects.
So let me-- to illustrate what these equations look like-- write out a few examples.
Epsilon 11 would be d1jk times e sub 1.
I'm writing it in simple fashion, and not expanding fully, just to save space and time.
Epsilon 22 would be d2jk times e2.
Epsilon 33 will be d3jk times epsilon 3.
And let me stop after these six terms, and write, at least for these, an expansion.
Because this gives us some interesting information.
I'm not doing the equal signs in here.
So this would say that the element of strain epsilon 11 is d111 times e1 plus d112 times e2 plus d113 times e3.
And I want to say this is 11k.
And I want to say that this is 22k.
The next term would be epsilon 22, and that would be d122 times epsilon 2 plus d212
Wouldn't the 2's be balanced?
Hm?
Wouldn't that be d2's?
Yeah, you're right.
This should be epsilon jk equals dijk times e sub i.
So this is all e1.
This is e1.
And this one goes with this one.
This 2 goes with this one.
You're right.
d2 and this is the d311 times e11.
This would be 122 times e1 plus 222 times e2 plus d322 times e3.
And the fourth one would be e33 equals d133 times e1 plus d233 times e2 plus d333 times e3.
OK, these elements here are the three that appear in the box that I had indicated for the terms djk.
When we write the direct piezoelectric effect, this would be the box of coefficients.
When we write the converse effect, this is the box of coefficients.
And now what I wanted to point out is that delta v over v is equal to episilon 11 plus epsilon 22 plus epsilon 33, which is the trace of the strain tensor.
And we're going to get one set of terms which depends on the x1 component of the field, another set of three terms that depend on the x2 component of the field, and another one on the x3 component.
If these expressions sum to 0, then there would be no volume change.
So this first set of 3-- 3 of the first 6 equations-- give you an indication of when the application of a field will result in no volume change in the sample.
And that again, I remind you, can be for two reasons.
It could be because all of these six of the nine piezoelectric moduli are 0.
Or alternatively, if you examined the form of the piezoelectric moduli matrices that is required by symmetry constraints, you'll find that there are a substantial number of point groups for which some terms are 0.
But then there is, in addition, an equality between some of the other terms, which make the volume change zero.
Even though, not all of the elements within this 1/2 box of moduli are zero.
So that is an interesting effect.
And it turns out that the majority of the non-centrosymmetric point groups do not have a volume change, when you apply in a field in any way you choose.
A few do, but it's a minority.
Alright, but this is not the main point of writing this.
I want to-- at this point-- point out that this tells you about the volume change.
And then we would have additional terms, we would have a term of the form e1 something like e123 or e132.
And this would be e14.
This would also be e14 since we replace both of those subscripts by a single subscript.
And the tensor elements that would go in here would be d123.
We just want one of the them.
d123 times e1, and then we want d223 times e2, and then d323 times e3.
We're writing one of the specific equations for the shear strings.
If we write the expression for d132, this is going to be d132 times e1 plus d232 times e2 plus d332 times e3.
There are two interesting consequences of this.
The strain tensor is symmetric.
And this element of strain-- why have I got three subscripts in here?
Don't want that one in there.
These two strains are equal.
And therefore, if we would apply just an e1 for example, let e be equal to just a component of field along x1.
The strains have to be equal.
But we have two different tensor elements here.
And the only way that strain can be symmetric, and it's defined as such, is that d123 be identical to d132.
And that resolves the issue that came up in connection with the direct case electric effect.
We said the direct piezoelectric effect depends just on the sum of the elements dijk and dikj.
And since they're lumped together, all we can measure is the sum.
And, so, we'll just have to call that a single matrix element.
The converse piezoelectric effect tells us these tensor elements have to be equal, if the strain tensor is to be symmetric.
So that says that, since we defined d16 as the sum of d123 plus d132, this says that d132 is equal to d123 is equal to 1/2 of d16, just making the equality in the reverse direction.
The converse effect let's us say that 123 has to be 132, that any ijk has to be equal to a dijk.
So if I tried now to write this first expression in the reduced subscript notation, e23 is what we let e5 be.
And now we have this equal to and in our reduced subscripts, we have this as 1/2 of d16.
And the field that's multiplying this is piezoelectric modulus is e1.
And that messy factor of 2 has come back to haunt us again.
It's like trying to stuff a jack-in-the box back in the box.
It keeps popping up.
We ate the factor of 2 in defining the matrix representation of the piezoelectric electric modulus.
And now when we try to go to a reduced subscript notation for the converse piezoelectric effect, we've got a 1/2 in there.
And similarly, the second equation would be e5-- same result epsilon 5-- and it's 132, but 132 is 1/2 of d16 times e1.
And we have a similar mess for the other coefficients here.
So what do we do?
Do we say that the relation between strain epsilon j equals dij e sub j has 1/2 in front of several of the coefficients and not in others?
Well, we can't really absorb the factor of 2 in the definition of the piezoelectric moduli, because we've already done that.
So the only thing we can do is to say that we will have to take the off diagonal strains, and define them as having a 1/2 in front, and we add these up.
So we will have to write matrix strain in this reduced subscript notation.
We'll have to take e11, epsilon 12, epsilon 13, epsilon 21, epsilon 22, epsilon 23, epsilon 31, epsilon 32, and epsilon 33.
And in converting this to matrix form, we'll call this epsilon 1, analogous to what we did for the tensile stresses.
We'll call this epsilon 2 and this epsilon 3.
And then for all of the off-diagonal elements of strain, in order to avoid the factor of 2 popping up in front of the matrix representation of the piezoelectric moduli, we're going to have to put in here 1/2 of epsilon 4, 1/2 of epsilon 5, and 1/2 of epsilon 6, and same for the off diagonal terms 1/2 of epsilon 5, 1/2 of epsilon 4, and 1/2 of epsilon 6.
So the moral of this story is that you can't win, but if you play it right, you can come out even.
So only if we define the reduced subscript strains in this fashion, can we write an expression of this form.
So this algebra is carried through for you for the other elements in the notes.
But this is the way we are forced, unless we want to have a factor of 2 in some terms, and not in others, is the way we have to define matrix strain.
All this is formalism and definition, but I'd like to now do two things.
First of all, give you some examples of real numbers for piezoelectric moduli, and then ask the question about representation surfaces.
Once again, these numbers are in the handout for you, so you don't have to make note of them.
But one of the very important piezoelectric materials is the quartz form of SiO2.
SiO2 has many polymorphic forms.
Quartz is the form that's stable at room temperature, and it has point group 32 asymmetric.
And there are higher temperature polymorphs of SiO2.
There's a phase transition in quartz to a more symmetric form, and then there are cubic forms at the highest temperatures.
Now, quartz is not the material that displays the largest piezoelectric moduli.
But it has the following advantages.
One is it is a naturally occurring material that is very inexpensive.
So it's not an exotic expensive material.
Very stable.
It's not water soluble.
Extremely tough.
You can take a thin wafer of quartz, for example, if you want to make a monochromator for a diffraction experiment.
You can take a thin wafer of quartz, and bend it like this, and it does not break, very elastic.
And if you want a material that's going to earn its living by being squished, you want something that doesn't plastically deform and something that is very hard and resistant to stress.
So quartz, even though the moduli are not the largest, is a very attractive material, and is used in a variety of devices.
There was a time when CB radios were very, very popular.
Everybody had to have one in their car.
I guess so they could pretend that they were truck drivers.
But anyway, you don't have them anymore now that cellphones have come in in existence.
But for your CB radio, you needed something called a crystal.
And they were fairly expensive.
And the number of channels on which you could communicate dependent on the number of crystals that you could plug into your CB radio.
The so-called crystal was exactly that.
It was a little black box that looked almost like a transistor.
And there were two leads coming out of it.
If you ever got curious and broke this thing open, what you found was a nice wafer of quartz.
And on the wafer of quartz-- brazed onto it-- were two wires.
And that's all there was in the box.
The crystal really was a crystal.
And the crystals had been very precisely ground to thicknesses such that when a field caused these wafers to hit a resonance, that resonance would be at exactly a particular frequency.
And that was the frequency of that channel.
One of the crises, during the Second World War, is that the highest quality natural crystals of quartz come from Brazil, and during the conflict the sea channels were essentially blocked.
And so, people-- in order to make all these communication devices-- had to learn how to synthesize crystals of quartz synthetically.
And there are a number of companies, such as Sylvania up on the North Shore, that developed entire buildings devoted to growing single crystals of quartz.
And they're big tanks like something out of the aquarium.
And in the center of the tank is a rod, and seeds of quartz are placed on the rod.
And the thing very slowly rotates around in this solution.
And on the rod, eventually, are single crystals of quartz that are this size.
And its a very spectacular thing to see.
In any case, symmetry 3 2, and the moduli that are 0 and non-zero.
If we refer the reference axes to a set of coordinates with x1 in this direction, and x2-- since it has to be orthogonal to x1-- in between the two-fold axes, and x3.
And for all materials of commerce that are anisotropic, there has to be some standard for defining the choice of axes.
For example, crystallographers would say the unique axis should be along the z direction, the x3 direction.
But why isn't x1 and x2 in between the two-fold axis?
There's some professional society that is responsible for giving standards for representing property measurements in some mutually agreed upon form.
And this is the standard set of axes for the quartz and symmetry 3, 2.
The moduli, dij, not dijk, but dij, these two are constrained to be equal.
This one is 0.
This is minus 0.67, 0, 0, 0, 0, 0, 0, 0.67, 4.6, 0, 0, 0, 0, 0, 0, 0.
One of the strange materials for which no field can create a strain-- field along x3 cannot create a strain.
And these are all in units of 10 to the minus 12 coulombs per Newton.
An example they give you here, if you apply a field of 100 volts per centimeter, which is not terribly large, but would be comparable to what you have in some electronic device, perhaps.
So this, since our units are MKS, this would correspond to 10 to the 4 volts per meter.
The strain epsilon 1, which is d11 times e1.
It turns out to be minus 2.3, which means it contracts.
That's the significance of the negative times 10 to the 4.
And that turns out to be 10 to the minus 12 times 10 to the fourth.
That turns out to be a strain of minus 2.3 times 10 to the minus eighth.
10 to the minus eighth is not exactly a large point strain.
You're not going to see the crystal wafer twitch and jump, if you apply a field of 100 volts on it.
But yet, even a strain of this sort is more than enough to be useful.
But this is just to illustrate that quartz is not the most sensitive of piezoelectric materials.
Another one that I give you data for is so-called ADP.
And this is a widely used material.
This is ammonium dihydrogen phosphate.
And this is one of the family of salts that have very large piezoelectric responses.
The nice part about it is that it's water soluble, so you can grow very, very large crystals easily from solution.
The nasty part about it is that it is water soluble, so you have to be careful to protect this material from moisture if you're going to use it in any sort of device.
But if we look at the moduli, relative to the standard axes, and this has point group 4-bar 2m.
So x3 is taken along the 4-bar access.
Then there are two-fold axes and orientations like this.
And since they are orthogonal, you can take both x1 and x2 along the two-fold axes.
And the numbers here for dij, are 0, 0, 0, 1.7, 0, 0, 0, 0, 0, 0, 1.7, 0.
This is one of the interesting tensors where there's a diagonal row of non-zero terms off on the right hand side.
And finally, the big surprise is the third modulus this is 51.7.
So you can see this has a very strong effect.
This is over 10 times the maximum piezoelectric modulus in quartz.
So this is a material that's very commonly used in transducers.
This is again times ten to the minus 12 coulombs per Newton.
One of the very, very exciting developments in recent years is a class of materials that are perovskites.
And they are very, very new.
I gave you the reference to the first one that was reported, and that was just in the spring of 2000.
And these are perovskites.
Perovskites are materials that in the type form are cubic.
But depending on composition and temperature, they can transform to a distorted version of this very simple cubic structure that is tetragonal.
And this material can exhibit piezoelectric effects.
And whether it distorts or not depends on the relative sizes of what goes into the perovskite.
Perovskite has a composition ABO3 like barium titanate is one example.
And the material has two different cations.
And they have different valences so they have different sizes.
And I won't bother to describe the structure, but it is only a very restricted locus in the field RA versus RB, where both the A and the B can remain in contact with the oxygen without distortion.
And it turns out to be a line that does something like that.
Any other perovskite-- and there are lots of them in this field of radii-- has to have one of the ion sort of flopping around.
And if that gets too serious, the structure distorts so that all these ions can remain in contact with the oxygen.
OK in order to do that, many of them distort to tetragonal forms.
Others distort to super structures, which have very, very large unit cells.
But in any case, when you're right at the phase boundary between the distorted structure and the true perovskite structure, these materials sometimes have very, very complicated x solutions of the two phases, on a very sort of fine scale.
And it's not known exactly why they have this property.
But they have absolutely enormous piezoelectric moduli, very close to this phase boundary.
And the references that I give you-- here-- is a compound that is a lead titanium zinc niobate.
And the complicated composition is to get you close to this phase boundary.
This has a d333 that is greater than 2,000 picocuries per Newton.
And pico is 10 to the minus 12.
So this is a piezoelectric electric modulus that is 10 to the 3 times d11 for quartz.
So 3 orders of magnitude stronger than this very commonly used piezoelectric material.
Some of these materials have strains getting close to 1%.
So this is something that will actually twitch on the lab bench, when you apply a field to it.
So these are entirely new.
People still don't know the origin of this behavior.
And it's still under considerable study.
So this is a new family of materials.
It's very exciting, and undergoing a lot of investigation and development work at the moment.
Alright, we are almost out of time.
Time goes fast when you're having fun.
Let me raise the question that we'll consider next time, which will be one of our last lectures.
And that is, is it possible to create representation surfaces that tell you how the piezoelectric properties of a particular material will vary with direction?
Well, vary with what?
Well, we talk about the direct piezoelectric effect.
This gives us components of-- well let's look at the simpler one, in terms of what we apply.
We have the converse piezoelectric effect that says that epsilon ijk is going to be dijk times e sub i.
So we've got a piece of material, and we apply a field, e sub i.
So we can vary this in space relative to a coordinate system x1, x2, x3.
But how in the world are we going to show what happens?
So I always do an extra thing in here.
There are 9 components to the strain tensor, which is symmetric.
So they're really six responses that are unique.
So yes, we can define the direction of the applied field.
But they're going to be 6 different strains.
So we're going to need six representation surfaces.
One for each of the three tensile strains, and one for each of the three shear strains.
So you can't do it with a single surface.
So you can't do much other than say, there are certain responses which are intended to emphasize one particular sort of strain or one particular sort of polarization.
So one of the things we might do is to cut a very thin plate of something like quartz, and subject it to a uniaxial stress.
So let's say sigma along the x3 axis.
So we're looking at a very restricted strain tensor.
That's 0, 0, 0, 0, 0, 0, 0, 0, sigma 3.
And in response to that strain, there are going to be three different components of the polarization.
Polarization is manifested as a charge per unit area.
So if we make a very thin plate-- to be sure there will be charges induced on these thin edges-- but if it's got a surface area that's a large compared to the area of these thin edges, we are going to be measuring primarily p3-- the component of polarization that's normal to this surface-- and that might have a charge per unit area that is comparable to these other two charged surfaces.
But because the area by design of our specimen is so large, the response that would be most easy to detect, and which would be the largest response, by design, would be p3, which is the charge per unit area on this surface.
So this is an effect we can define for a particular sample, and for a particular special form of the generalized force.
And we then can ask, what is the value of the single modulus that relates p3 to sigma 3?
And that's a question we can ask.
And we can plot that response as a function of direction of a plate that we consider as being cut out of a single crystal, and different orientations, and then ask how this modulus-- which connects the two-- changes with the orientation of x3.
So that is a question we can ask.
And these surfaces are absolutely wild, highly anisotropic, can be identically zero in certain special directions, and they are very interesting, and a lot of fun to look at.
So we'll take a quick look at a couple of those, which won't come as a surprise because they're already worked out for you in the notes.
So we'll take a look at one of those.
And in the problem set, I invite you to amuse yourself by looking at such representation surfaces for two other point groups.
And with that, having kept you til five after the hour I will quit.
Today, we are going to change gears after the first hour to a very different set of topics.
What I wanted to do today was to take a look at notation for space groups.
And this is a question that is far from trivial.
There are conventions, and we'll see the reasons for them with a few examples.
And then I would like to pass around representations of some non-trivial space groups.
First, as they appear in the old International Tables for X-Ray Crystallography, Volume 1.
And then second, to give you a look at the larger, heavier, and much more expensive Volume A, which gives much more information which is both its liability and its advantage.
There are things in there that are not in the international tables.
But there's so much information and so cluttered that it's really overwhelming.
But I'll give you a chance of what their representations of the space group look like.
Let me begin by passing around an example of the information in the International Tables, Volume 1 for one of the orthorhombic space groups.
And it all fits on one page.
It's not terribly complicated.
But it is considerably more complex both in the arrangement of atoms and in the cluster of symmetry elements, compared to the very simple monoclinic space groups which we looked at in their entirety and in considerable detail.
So first of all, some indication of what's on the lower part of the page.
This is exactly analogous to what you've seen and presumably are familiar with for the two dimensional plane groups.
There are many possible origins that could be selected for the 0, 0, 0, position relative to which the atomic locations are expressed.
So the first thing, just below the picture of the arrangement of atoms and the depiction of the arrangement of symmetry elements is a statement of where the origin is.
So it says, origin at the inversion center 2/m.
And if you look at the arrangement of symmetry elements, there is nothing but the representation for screw axes normal to the page.
But there is, off on the right-hand side, a solid chevron, and that is the indication for a mirror plane.
Something again that is nowhere stated, but always assumed is that the two variables x and y or the axes a and b are assumed to run from upper left to lower left for a and the coordinate x and from upper left to upper right for the axis b.
And therefore, plus c comes out normal to the plane of the projection.
Here, for the first time, we're seeing a complex space group that really requires the full range of symbols for representing the atomic locations.
You will see, for each circle, a vertical line dividing it in half.
These are two atoms then that are superposed in projection.
One of them, corresponding to the little comma inside the circle, is the enantiomorph of those circles which are unadorned with that little comma.
And they are, therefore, of opposite handedness.
Alongside of each circle, you will see two symbols giving the elevation of that atom within the cell.
The one at the furthest upper left corner says 1/2 plus.
So this says that the coordinate is z, whatever that is for the representative atom, plus 1/2.
The half of the circle that has the comma inside has a minus sign next do it.
That means that it is at an elevation minus z.
Well, where is the representative atom?
This is tucked just inside the corner of the cell where the vertical line divides the circle into 1/2.
That is a plus, so that's an atom at plus z.
The right-hand side of that circle has a comma.
It's an enantiomorphic motif, and that exist at 1/2 minus z.
So you find labels on all of the atoms in the cell.
There are a total of four clusters of eight.
So are a total of 16 atoms within the unit cell.
If you look at this pattern and reflect on it for a moment, so to speak, you can see that the same cluster of eight sits at the corners of the cell as in the middle of the cell.
Exactly the same.
So therefore, this is a lattice that is side-centered.
There's a lattice point at the corner of the cell and one lattice point, in addition, at 1/2, 1/2, 0, in the center of the cell.
The c-axis comes out of the plane of the paper.
This then is a side-centered c-lattice.
And that is the capital letter symbol that appears first in the symbol for the space group.
It is a side-centered lattice, and the extra lattice point is in the middle of the face out of which c comes.
Then come, in the space group symbol at upper left-- this is the abbreviated form-- mcm.
And what I wanted to go over this example specifically for, there are three directions in an orthorhombic symmetry, no one any more or less special than any other.
So if you look at the arrangement of symmetry elements, there's a 2 sub 1 screw axis coming out in the direction of c. The symbols running left to right, arrows and barbs, the arrows stand for two-fold axes.
The barbs stand for 2 sub 1 screw axes.
And running up and down parallel to the a-axis, again, arrows that indicate two-fold axes and barbs that indicate 2 sub 1 screw axes.
So which is the special direction?
Which one should come first?
It's not like a cubic symmetry, where the direction of the four-fold axis is always along the edge of the unit cell, or a tetragonal symmetry, where the four-fold axis is always taken to be the direction of c. Here, that's not specified.
So again, the first convention is that the magnitudes of the translations determine a, b, and c in orthorhombic.
Because there's nothing more or less special about the symmetry of these three orthogonal directions.
And the convention which we've mentioned earlier is that the magnitude of b is greater than the magnitude of a.
And then seemingly logically, c is the smallest.
So if this is the direction of a and this is the direction of b, then c comes up.
And this is the intermediate length translation.
This is the maximum translation.
And normal to the blackboard is the shortest translation.
What symmetry goes first?
And here again, for orthorhombic, we need a convention.
And the space group symbol has-- I [INAUDIBLE] I use lambda for this, but-- a symbol for the lattice type.
And we've seen that the capital C that comes first indicates that there is a c-centered lattice.
And then what one does for orthorhombic is give the symmetry elements in the order abc.
So the axis that is parallel to a over the plane that is perpendicular to a.
Next, the axis that's parallel to b over the plane that is perpendicular to b.
And then the axis that is parallel to c over the plane, if any, that is perpendicular to c. So then this is a convention for orthorhombic.
And again, this is the only crystal system where conventions this elaborate are necessary.
If this were tetragonal, 4 or 4/m would always come first because that's the direction of c. So even the order of the symmetry elements is different.
But we've got another problem.
If we look at the axes, if any, that are along a, we've got this symbol that stands for a two-fold axis and this single barbed arrow that stands for a 2 sub 1 screw.
So what kind of axis is along a?
There are two different kinds of axes.
So in here, we could put either a 2 or a 2 sub 1.
And if we look at the planes that are perpendicular to a, there's two of them.
There's a dashed line, and this is a glide where the direction of tau is in the plane of the paper so that we're looking edge on in the glide plane and perpendicular to tau.
And then there is a solid line.
That's the symbol for a mirror plane.
This would be a glide in which tau is a long b, so this is a b-glide.
So those are the two sorts of symmetry planes that are perpendicular to a.
Again, there are two of them.
So we could put in either m, or we could put in a for the plane that is perpendicular to a.
So already, there are four different possibilities for the first character.
So obviously, what I'm leading up to is the fact that we're going to have to have conventions of necessity to come to a symbol on which everyone agrees.
I think you get the idea now, so let me move along quickly along the direction of the arrow for a two-fold axis, a barb for 2 sub 1 screw axis.
So again, there are two kinds of symmetry axes that could go in here for the axes that are parallel to b, either 2 or 2 sub 1.
If we look at the planes that are perpendicular to b, there's a dotted line.
And this is a glide for which tau is directed upwards.
We're looking directly along the orientation of tau.
The axis that is normal to the board is the c-axis.
So this is a c-glide.
And then appears a dash-dot line alongside of it.
The dash-dot line is a glide plane where you're looking neither perpendicular to tau nor parallel to tau, but halfway in between.
So this is a glide plane where tau is equal to 1/2 of a plus 1/2 of b.
And that's a diagonal glide represented, just to confuse the innocent, by the symbol n. So two choices for axes along b.
Two choices for the glide plane, either c or n. And finally, mercifully, along the direction of the c-axis, which is easiest to visualize, because things are either along it or parallel to it, we've got only a 2 sub 1 screw axis.
So this is the only symbol which is not ambiguous.
And then perpendicular to that, we have the symbol for a mirror plane, the solid chevron off to one side.
And then the chevron adorned with a diagonal arrow and that is, again, a diagonal glide n. We notice that the center of the cell at z equals 0 is the same as the grouping of atoms at the corner.
So that is a side-centered lattice.
The centered lattice point is coming out of the face which c emerges from.
And so this is a c-lattice.
You have a question?
Professor, for the first term [?
x of the ?]
plane perpendicular to a, why is it a and not b?
I'm sorry.
You're absolutely right.
I wrote it down as a b-glide.
Perpendicular to a is a b-glide.
Good.
Congratulations.
So we need a convention.
And the convention for establishing a hierarchy of symbols is easy to remember.
It's if you have both a two-fold screw axis and a two-fold axis, the order of preference is that the 2 is chosen in preference to the 2 sub 1 screw symbol.
So by this greater than, I mean the preference for choosing it is greater than the symbol to the right.
So we get rid of the 2 sub 1 here.
We pick the 2.
Get rid of the 2 sub 1 here, and that stays at 2 sub 1.
For planes, you pick m in preference to an a-glide, in preference to a b-glide, in preference to a c-glide, in preference to a diagonal glide, in preference to a diamond glide, assuming you have a choice of two different planes.
So in this case, we picked the m in preference to b.
Here, we've got a c-glide and an n-glide.
We picked the c, and it's in preference to the n. And here again, for the plane that is perpendicular to c, we picked the m in preference to the n. So the full symbol would be C, 2 over m, 2 over c, and 2 sub 1 over m. Now I'll peek and see if I got this right.
Yes, I did.
How about that?
And the short symbol would be simply Cmcm, which indeed is what we see listed in the upper left-hand corner of the page.
This is a space group based on the point group 2/m, 2/m, 2/m, which is D2h.
And there is the large number 17 in the superscript, which is the 17th orthorhombic space group, which Schoenflies was able to derive in the order in which he did them.
Any questions?
Yes
If you look at the other symbols to describe the space group, would we construct the same space group using that function [INAUDIBLE]?
Yes, you could.
And let me say, yes, it's possible.
But also to answer a question, you might have about the short form of the symbol, we're throwing away information.
We've thrown away the fact that there are two-fold axes, and a pair of directions, and 2 sub 1 only in the third direction.
Is it possible that two space groups could have exactly the same three symbols in the short form?
And the answer is no.
Because if you think about it, you start with a lattice type.
If you put a mirror plane in one orientation, a mirror plane in the other orientation, that really is all you have to specify in order to determine every other symmetry element that's present.
You simply combine those operations, and you find that when you've taken these symmetries and rotations and combine them with the lattice translation, it comes out to one and the same unique result.
So your question was what happens if you take another three symbols.
You end up with the same result.
Three of them, really, are enough to specify what the group will turn out to be.
Now the other thing I wanted to do was to show you how remarkably the symbol for the space group will change if you take a different set of axes simply because the relative lengths of the cell edge for this arrangement of symmetry elements have changed.
This takes dominance.
This determines the labels that go to the axes.
And then the symbol for the plane group, and what is a b-glide, and what is a c-glide follows from this choice of axes.
So let me take another example.
And I think if you follow what I've done so far, I can zip through this a little more quickly.
Somebody pick a new direction for this axis.
Let's not pick a.
We have used that for this.
So somebody pick a b or a c. Do you want to pick a b or a c?
[INAUDIBLE]
b.
So we're going to change this direction to b.
Somebody want to pick a label for this axis?
c
c. And nobody gets to pick this axis.
If you think you can, you're out of here.
Because that now has determined the coordinate system.
So we've got b this way, c this way.
We want a right-handed abc convention.
So a goes up.
So abc right-handed system.
So exactly the same arrangement of symmetry elements except that we've now changed the labels on the axes.
So we have changed the name that we have to apply to the glide planes.
So let me now proceed to write down here the new collection of symbols which would appear for this setting of exactly the same space group.
First of all, the centered lattice point is now in the middle of the face out of which a comes.
So this is an a-lattice, side-centered a.
If we look along the direction of a, all that we have for an axis is 2 sub 1.
Then we have these two planes.
One is an m. One is a diagonal glide.
So that is m or n. If I look along b, I have the choice of either 2 or 2 sub 1 as both are present.
Perpendicular to b, however, is a glide for which tau is along the direction of c. So that is a c-glide.
And then the other symbol for an axis that's perpendicular to b is a mirror plane.
So it's either c or m. Then finally, along the direction of c, I once more have two-fold axes and 2 sub 1 screw axes.
And then perpendicular to c is a diagonal glide.
And this is a glide.
The dotted line indicates that the direction of tau is along a.
So I have a choice of either a or n in here.
So putting down the symbols in the order of preference, this would be A, 2 sub 1 over m, 2 over m, 2 over a.
Or the short form of the symbol would be Amma, which I submit doesn't look anything like Cmcm.
And when you start out in diffraction, many people have a first disconcerting experience.
You spend probably the better part of a month taking single crystal diffraction patterns.
You come up with a trial space group for your particular material.
And then you go to the International Tables, and you say, oh, it's not in there, what did I do wrong?
And you go back, and you check all your calculations.
You look once more at all of the systematic absence of reflection.
See, I have it right.
Why is it not in there?
Well, the answer is more than likely that you have a different setting for the labels that you applied to the axes.
And you have, therefore, metamorphosed the symbol for the space group from one form to a symbol that is very, very different.
How do you tell, when you have a particular space group, what all of the possible symbols for one and the same symmetry might be?
So let me now pass out two things.
If you have a question like that, you can bet your bottom bippy that it's in the International Tables if you look in the right place.
So first of all, let me do something I should have done early on.
I meant to hand out a copy of the space group that I drew up here.
If I can find it again.
It's buried with all of the stuff that I brought in.
So I do have a copy for you, but it's gone at the moment.
Let me pass out a listing of all of the possible variants of the space groups for the 230 three-dimensional space groups.
And it starts out simple for triclinic symmetries.
There's only two possible symbols period, P1 and P1 bar.
And then when you come to monoclinic, the Schoenflies symbol has the marvelous property that it is independent of the orientation and labelling of axes.
It depends only on the point group that the crystal has.
For monoclinic, either C2 or Cs-- that's a mirror plane-- or C2h-- that's 2 over a horizontal mirror plane.
But the labels can still change depending whether in the first setting either a, b, or the diagonal of the oblique net has to be labeled a and c. So at the top of the column, you'll see the permutation of the two symbols a and b or b and a.
And again, the change is significant.
Number eight, as indicated in the left-hand column, can be Am or Bm.
And for the groups that are based on 2 over m, you can see, again, there are different symbols.
A, 2 over b, A, 2 over a.
Again, not at all similar.
Next, come the orthorhombic space groups.
And you can amuse yourself by looking through those.
Again, what is called the standard setting under the heading symbols for various settings, this is the one that's listed in the tables.
But then for various permutations of a, b, and c, you can have five other possibilities.
And you can marvel at how the symbol for the space group changes, as you do nothing but change the labels a, b, and c that goes onto the axes.
So there are tons of orthorhombic space groups because you have three different axes to play with, and three different planes, and four different lattice types.
So there is an absolutely mind boggling collection of orthorhombic space groups.
It is the most densely populated crystal system.
For tetragonal, no real alternative.
Something like P4, 4 is in the direction of the c-axis.
There are two-fold axes parallel to that, but the symbol that is used for the symmetry part of the space group symbol looks very, very much like the symbol for the point group.
So these are pretty much self-explanatory.
Notice that there are two kinds of lattices for tetragonal crystals, either primitive or body-centered.
So you see families with P for primitive or I for body-centered.
But notice how many you can get out of a single type of axis and a single lattice.
Numbers 75, 76, 77, and 78 are a four-fold axis and a primitive lattice, a 4 sub 1 screw, a 4 sub 2 screw, or a 4 sub 3 screw.
Then you do the same thing with I.
And so it goes.
Then it picks up at the top of the page again.
And you see, again, different possibilities for the symbols.
Continuing still on-- and I think you have the idea now-- same for trigonal crystals.
They chose to list those separate from hexagonal.
And then hexagonal symmetries.
The number is fairly large here because when you take an axis and add it to a primitive hexagonal lattice-- for number 168, for example-- you get P6.
Change the six-fold axis to a screw axis.
There are five different types of six-fold screw axes.
So there's a P6 sub 1, a P6 sub 2, a P6 sub 3, a P6 sub 4, and a P6 sub 5.
And then finally, cubic.
A fair number of cubic symmetries.
But the edge of the cell is always along the direction of the four-fold axis or the two-fold axis, depending on whether the symmetry is based on 2, 3 or 4, 3, 2.
So there are really no alternative symbols for different settings of the symmetry elements relative to the edges.
So there you have them.
All 230 of our cast of characters.
But in forms of the very different mantles in which they can be cloaked.
What else did I want to do?
I want to contrast what is done for us in the old Volume 1 of the International Tables.
And I will pass out an example for one of the tetragonal space groups.
This is the one with maximum symmetry 4 over m, 2 over m, 2 over m. Sorry.
Can you pass those back?
And this is something that is analogous to P4mm in two dimensions with the four-fold and two-fold axes extended parallel to the edge of what is now a tetragonal cell.
And then there are two-fold axes perpendicular to the four-fold axis.
So actually 4 over m, 2 over m, 2 over m has been dropped in at a lattice point of a primitive tetragonal lattice.
The chevron in the upper right of the arrangement of symmetry elements is the mirror plane that's perpendicular to the four-fold axis.
The other mirror planes are the same as in the plane group P4mm.
And the glide plane, which now would be called a diagonal glide, is exactly the same as in the two-dimensional symmetry.
Notice, however, the exquisite number of general and special positions that are present in the space group.
The general position has the letter U.
You've gone through almost the entire alphabet before you've labeled all of these positions.
And again, either on the mirror planes, but now there's a vertical mirror plane, a diagonal vertical mirror plane, a horizontal mirror plane, mirror planes halfway along each of the axes, mirror planes halfway along the diagonal translation.
So there are lots of positions of point group m. Lots of positions with point group 2mm.
And then finally, 2 over m, 2 over m, 2 over m. And the highest symmetry, 4 over m, 2 over m, 2 over m, which occurs at the origin, at the center of the cell, and at the positions 0, 0, 1/2 and 1/2, 1/2, 1/2.
Then I'd like to pass around one more example of a space group from the International Tables.
And I know I copied it, and I don't have it with me.
I left some stuff behind, which we'll get after we take our break.
No, here it is.
This is an example from Volume 1, the older edition, for one of the cubic point groups.
And this is a very high symmetry.
This is symmetry 4 over m, 3 bar 2 over m, the highest symmetry cubic point group dropped into a body-centered lattice.
So this turns out to be, in the long form, I, 4 over m, 3 bar, 2 over m. Let me pass these back.
So the first thing you'll notice is no picture of symmetry elements.
How do you draw something where the symmetry elements are not merely parallel to the plane of the depiction or perpendicular to it?
Here, you've got mirror planes that are inclined to the paper, axes that are inclined to the paper.
How do you represent them?
You'll notice also the enormous number of atoms in the general position.
96 atoms.
Drop in one atom at xyz, and all hell breaks loose.
You get 95 other atoms.
And again, when the lattice is centered, they do not list all 96 sets of coordinates.
They denote at the top of the page 0, 0, 0, 1/2, 1/2, plus.
That means because the lattice is body-centered to those 48 positions that are listed, you add 0, 0, 0, which is easy to do, and then add 1/2, 1/2, 1/2 to x, y, and z.
And these are the atoms that hang at the centered lattice point.
Again, because the symmetry is so high, surprisingly, there are not that many special positions.
Because all of the symmetry elements can serve as special positions are related by the symmetry that's there.
So this is all that you see for the body-centered lattice with 4 over m, 3 bar, 2 over m dropped into it.
And again, you can describe some very, very complicated crystal structures with a very brief set of notes, give the value of the single lattice constant a.
And then say, you've got an atom in the general position 96l with x equals something, y equals something, z equals some number.
And another atom in position 16f x, x, x, and just the value of x would be given.
So you've described a structure that has roughly 125 atoms in it with just that modest specification of atomic positions.
Let me now pass out for you some samples of what the space groups look like in Volume A of the new International Tables for Crystallography.
Not X-ray crystallography, but just crystallography in general.
And I think you will be blown away because the amount of information that's there is almost suffocating in its detail and its density.
The first sample space group that is given here is, not by coincidence, one of the tetragonal space groups, the one that we just discussed, P, 4 over m, 2 over m, 2 over m. They've done some very useful things.
They have specified the asymmetric unit within which you need specify coordinates of atoms.
And here, they say that you have to tell what's in the range x equals 0 to x equal 1/2, y equals 0 to y equals 1/2, and z equal to 1/2, but greater or equal to 0.
Then they give a representative set of symmetry operations which generate all of the atoms within the unit cell.
The symmetry operations are not independent, but they list 16 symmetry elements.
And now if you look to the next page on the list of 16 atomic coordinates in the general position, you will see a number in parentheses in front of each one.
The 15, for example, tells you that you get this atom from the atom at xyz by the operation of symmetry element 15, which turns out to be a mirror plane parallel to the x minus xz direction.
And this is very, very useful if you want to describe a structure and you've labeled your atoms silicon 1 and silicon 2 if they're two different kinds of silicon ion in the structure.
And then you're talking about a silicon, oxygen, silicon bond, and it's not at all clear which of the symmetry related atoms are involved in that bond.
Well, the presentation of a standard numbering of atoms related by symmetry lets you say that, for example, this is the bond between silicon superscript 5, oxygen, silicon superscript 14.
And you can identify the coordinates of the atoms that went into that particular bond angle or bond distance.
And that's very nice.
All sorts of information a la group theory.
The maximal non-isomorphic subgroups, the maximal isomorphic subgroups of lowest index, the minimal non-isomorphic supergroups.
At one time, I may have known what all that meant, and I've long since forgotten and have never regretted having forgotten.
So this is very exotic, higher-level information about the subgroups that exist for the space group.
Notice how much more information is present though, compared to the depiction of the same space group in the earlier International Tables for X-Ray Crystallography.
Well, more or less at random, I picked out some other space groups.
I, 4 sub 1, amd.
So this is 4 over m, 2 over m, 2 over m in which the 4 has been replaced by a 4 sub 1 screw axis.
The mirror plane perpendicular to the edge of the cell replaced b an a-glide.
The mirror plane for the second mirrors that's perpendicular to the cell edge, the a-glide is diagonal.
And then perpendicular to the diagonal, two-fold axis is a d-glide.
Again, the different operations are identified by a number, the glide plane, and the inversion centers, and two-fold axes that are present.
And then in the coordinates of the general position, you are given the nature of the symmetry element that produces the atom at a particular set of coordinates from the atom that's in the general position.
Now again, the maximal non-isomorphic this, the minimal non-isomorphic supergroup, and so on.
Lots of information.
Then some examples of still higher symmetries.
P6 sub 3 over mmc based on 6 over m, 2 over m, 2 over m with the 6 replaced by a 6 sub 3 screw axis and one of the mirror planes replaced by a c-glide.
If you go on, here comes the really exciting part.
The next page gives you a depiction of a cubic space group.
Heroic.
What they do is show stereographic projections at locations, such as 0, 0, 0 and 0, 1/2, 1/2, I believe it is.
And then the best part of all, down at the bottom of that page, you see the arrangement of atoms in stereo.
And if you're one of these people who can stare at the thing cross-eyed and let your eyes sort of blonk out, you can watch the two halves merge.
And all of a sudden, zing, the thing leaps out of the page at you in three dimensions.
If you don't have that ability to cross your eyes and see stereographic projections, you have to get a little viewer.
But nevertheless, if you want to see it in three dimensions badly enough, you can do it.
So there you are.
For the first time, pictures of the general position in cubic space groups.
I think that's kind of fun.
Sometimes I gaze at it for so long I'm afraid my eyes are going to get stuck.
And then I'll be in real trouble.
Here's another one.
P 4 over m, 3 bar, 2 over m. You notice the problems that they have in indicating the orientation of three-fold axes which are not parallel to or perpendicular to the paper.
You have to do that with a little stereographic projection.
And then finally, after we've done a couple of more cubic space groups, you come to the one that I gave you the handout from the International Tables for X-Ray Crystallography.
I, 4 over m, 3 bar, 2 over m. And look at all the information that is there that no attempt is made to give you in the earlier tables.
So this is for better or for worse what space group tables look like.
This is the information that hopefully you'll be equipped to use.
If nothing else, if you can't derive these things or reproduce the arrangement of symmetries from the symbol, if you ever have to construct the atomic arrangement from a material from the crystallographic notation in which the atomic arrangement is provided, you can hopefully go to the International Tables, know where to look, and how to go about identifying the coordinates of all of the atoms within the unit cell.
Timed myself to finish just at five of the hour.
So let me, before you leave, give you one final problem set on symmetry, problem set number 10.
And this is related to identifying what the symbols used to give atomic locations represent.
First problem asks you to look at a pair of atoms and determine the symmetry element which has related them.
The second problem on the second sheet asks you to do what we did here for this orthorhombic space group.
Determine the symbol when a, b, and c are particular translations of the three that are unique.
And then change the orientation, and see what the space group symbol morphs into.
Let me finish with one final handout.
And that is could you please summarize what we spent the last month and a half doing?
Gladly.
I have summarized everything that we did and the theorems that we used to derive it on one piece of paper.
So all of symmetry theory is here admittedly written in a rather tight hand.
But here is an indication of everything we did to get from a definition of basic operations, a flowsheet that gets us down to 230 three-dimensional crystallographic space groups.
So I'll pass that around for your awe and amazement.
Sorry I gave a big pack out on the right-hand side of the room.
They'll come around to you.
So let us take our usual 10 minute break.
All right, now for something completely different.
Before beginning though, I would like to raise a procedural question.
It was my intent that quiz number two, which is like a little more than a week away, was going to be part symmetry and part tensors.
I don't want to have your fortunes based by weight of 2:1 on symmetry theory as opposed to properties.
What I would suggest, and you can express your opinion either way, is that we postpone quiz number two by maybe as much as 10 days, so that it can be half symmetry and half the introductory discussion of tensors.
And if that does not create a conflict with some of your other classes, I would propose that as a suggestion.
If not, we can just have on the quiz what we're going to cover of tensors in the next couple days and have the rest symmetry.
So what I am suggesting is the quiz was scheduled for a week from November 8, which is a Tuesday.
What I would suggest is putting that off to the 18th, and that will give us four extra lectures-- six extra lectures-- on tensors and we can make the quiz 50-50
What does that do to quiz number three?
Quiz number three is going to stay in place because it's right at the end of the term
What about the subject in three?
It will be all tensors.
It will be all tensors and properties.
And the third quiz is set up for longer than you think.
It's December 8, so it'll be about three weeks
So everything from here on?
Yeah.
Does that make sense?
Or is somebody going to have a big, traumatic responsibility at that point?
I don't know, but I think it would be probably be easiest for us not to have it three days before close of exams
OK, OK, that's what I wanted to hear.
How about if we went just a little bit further then and did it not on the-- when was the suggestion?
The 18th.
Let's say we did it on Tuesday the 15th?
Does that conflict with anything?
That's a little bit
We were doing it on the 18th
Is that Friday?
[INTERPOSING VOICES]
I'm sorry, 18th.
I said it would be 17th.
Excuse me
That definitely would be a little better
OK, let me allow you to think about that, and that would be pushing it back one week and maybe it'd be-- we'll see how much material on tensors we cover.
So let's table it for now but let's say, tentatively, let's consider seriously moving it to the 15th.
OK?
But you guys have final say if there's a preference that you have.
OK let's talk about properties in first a general and rather philosophic way.
When we discuss properties we very commonly lump together a set of behaviors which have some common phenomenology.
So for example, we will talk about mechanical properties.
And that's a basket in which we place many different sorts of behavior.
We place things like fracture toughness, yield strength, elastic constants, a whole variety of things involving strength deformation and so on.
Or another thing we very often lump together in a basket is something called electrical properties.
And we talk about electrical conductivity, ionic conductivity, electronic conductivity.
We talk about dielectric constants, we talk about permittivity and permeability, magnetic properties-- a whole bunch of other classes of behavior, which have some sort of root or description in terms of electromagnetism.
There are though a lot of strange aspects to properties.
There are some properties that are determined for material which no longer exists when you're through determining the property.
You have a sample you want to know what the yield strength is.
So you put the gradually increasing load on it until finally, POP, it breaks.
And now you know what the yield strength is for a material that no longer exists.
Another example is an old one.
Back before the days of X-ray diffraction when people would try to characterize material, anything that is very intensely colored usually looks black, and that's not a very definitive property.
The color of something is a very prominent characteristic.
So for mineralogists, in particular, anybody going out into the hills and wanting to be prepared to determine what a particular rock that they tripped over was had something on a rope around their neck called a streak plate.
And what the streak plate was was a little rectangle of porcelain.
And they would take this black mineral and rub it on this piece of porcelain and it would leave a mark-- that was called the streak.
And what this was was actually little bunch of fragments that rubbed off on to the porcelain because the porcelain was white and the fragments were very small.
A black piece of rock could give you a streak that was brown or green or deep orange.
And so you really were determining the color of the material when it was in a form finely divided enough that light could pass through it.
So the streak test was a very important diagnostic for determining minerals.
There are some properties that we refer to as structure sensitive.
In the sense that their value-- conductivity is a good example-- the value of ionic conductivity depends on the impurities and point defects that are present in the material.
So to say that the ionic conductivity of a certain material is such and such, you have to specify the purity and perfection of the material or the property's meaningless.
There are some properties that are not even single-valued.
And the best example there are dielectric or magnetic properties.
If you plot the magnetization of a material, the magnetic moment per unit volume as a function of the applied magnetic field B, the material is initially non-magnetized.
When you apply B, you get a magnetic moment per unit volume that eventually saturates.
And then if you remove the magnetic field, the material keeps some of its magnetization.
It doesn't go to zero when you reduce the field to zero.
You have to reverse the field to get the property to go to zero and then magnetize it in another direction.
And if you continue to cycle the magnetic field, you get a behavior like this.
This is hysteresis, and this type of behavior is called generally hysteretic.
Same thing would be a relation between the displacement vector and an applied electric field.
But clearly you can have any value of the magnetization in this range.
And if you relate the magnetization as a function of the applied field, you can get any value of the proportionality constant you like between a negative maximum and a positive maximum including zero.
So there's a property that's not even single-valued.
Depends on the past history of the sample.
And then there are even more peculiar properties.
There are properties that are-- we call them composite properties and very often qualitative.
What do they mean by a composite property.
Let me give you one example-- The property fuzzy.
If I say fuzzy, you know exactly what I mean.
It means something that has a diffuse reflectivity, something that has a surface texture that's yielding.
It's not like rubbing a wire brush.
It's soft and giving, a whole collection of different properties.
But yet when I say fuzzy, you know exactly what I mean.
Let me not be so facetious as talking about a fuzzy property.
People such as ceramists or powder metallurgists very often will take a powder of a material and consolidate it by compacting it and heating it, very often subject to a compacting stress.
And when you do that little necks grow between the particles and they hold together and the material is centric.
So you refer to a property of a powder as being centerable or the centerability of a powder.
You know exactly what somebody means by that.
But what does it depend on?
It depends on the surface energy, it depends on vapor pressure, depends on bulk diffusion coefficients.
It depends on surface diffusion coefficients.
And all of these things have to be just right to make the powder something that easily densifies upon heating in the application or not of pressure.
So you know exactly what I mean by centerability, but it is a very complex property.
And was one that really was not understood until probably the late 1950s.
Until then, it was an art that was entirely empirical.
And it was somebody here at MIT, a fellow named Robert Coble, who developed a theory of centering that was the first really workable theory that described densification by heating and compaction.
OK, composite property that depends on many different individual properties of the material.
OK, what we are going to examine here are equilibrium properties that can be rigorously defined and measurable.
So we're going to leave out of the picture things like fuzzy and centerability.
And let me give a very nice, obscure definition of what I mean by a property.
And what I mean by a property, in terms of a formal statement, is the response of a material to a specific change in a given set of conditions.
So the response of the material to a specific change in a set of conditions that relates independent properties and dependent properties for a particular process.
So I'll state that again because I think it's terribly elegant.
So a property is the response of the material to a specific change in a given set of conditions that relates independent and dependent quantities in a particular process.
Now there are a lot of properties that have their roots solidly embedded in thermodynamics.
So let me give you a few of those.
What we'll talk about when we specify a property is something that we will refer to as a displacement.
We'll talk about a generalized displacement in response to a generalized force.
So some of the thermodynamic quantities that are related in this fashion-- if we list some forces and some displacements, the thing that happens as a result of that stimulus that's applied to the material.
Temperature can be regarded as a force that results in all sorts of processes as a result-- thermal energy flow, thermal expansion, all sorts of things.
But one of the things that will happen in response to a temperature change thermodynamically is an entropy.
So you can view entropy as a generalized displacement resulting from the application of temperature as a generalized force.
Another example is electric field and the thing that happens there, and electromagnetism, is the quantity, D, which is displacement.
Still another example, stress, and the result of applying a stress, among other things, is a strain.
What is special about these forces in this displacement is that their product is, in each of these cases, energy, the change in internal energy of the particular body.
And when that is the case, these are said to be conjugate-- a conjugate force and displacement.
And there are other examples that one can come up .with.
Now to talk about a specific set of properties.
Very often, and this crept into the earlier discussion, very often the thing that we do to a material, as a generalized force, is a vector.
So very often the thing that we do to the material, apply an electric field, apply a magnetic field, apply a temperature gradient, apply a tensile stress, it has the characteristics of a vector, magnitude and direction.
And in many cases, the thing that happens as a generalized displacement, we may call this in general, q, is also a vector.
An example is if we apply an electric field as a generalized force, one thing that might happen is a current flow, which is also a vector.
And we are accustomed to writing, q, the generalized displacement, as a proportionality constant, sigma, which is in the case of electric field and current flow, the electrical conductivity.
Let me call it, in general terms, proportionality constant, a, times the applied vector, p. Is this something we would like to stick with as a general relation?
Well let me submit that writing an expression of this form makes an inherent assumption.
Namely that this will be true for small, and we'll have to define for each property what we mean by a small, applied vector.
Let me give you an example.
If p were-- the applied vector was electric field, and the resulting vector was current flow, and the relation between those is the conductivity-- a relation of this sort says that if you double the field, you double the conductivity.
You triple the field, you triple the conductivity.
Obviously this can't go on indefinitely because all a sudden, POW, dielectric breakdown.
The sample evaporates at the smoke.
And again, you have the property of a material which no longer exists.
So a lot of properties, we inherently assume that the applied vector is small in order to write something in an expression of this form.
So for conductivity, dielectric breakdown is going to destroy the nice linear relation between current flow and electric field.
That's not always the case.
Let me give you an example of another property.
And this is a property, magnetic susceptibility, which relates the magnetization, which is magnetic moment per unit volume, and relates that to an applied electric field-- an applied magnetic field, H. And the proportionality constant, represented by a Greek chi, is the magnetic susceptibility.
So let me give you a specific example.
Suppose we have a chunk of glass and the glass contains a dilute concentration of iron.
And iron carries a permanent magnetic moment.
And the reason I want to make it a dilute concentration of iron is I don't want these magnetic moments close enough that they can interact with one another.
I want this to, therefore, be a dilute system.
So we have different iron atoms in this.
Each iron atom has a magnetic moment.
And then we put this in a magnetic field, H. And the magnetic field acts on each of these little magnetic moments just as though it were a compass needle.
And so it will try to take each of these moments and drag it into coincidence with the magnetic field.
But at a finite temperature, temperatures making these magnetic moments jiggle around, so at a finite temperature, the magnetic moments will just not simply zing into coincidence with the magnetic field.
You'll have to increase the magnetic field to make it larger.
When that happens, more and more of the magnetic moments will come into alignment.
And if you put on a really strong magnetic field, then every single magnetic moment will be dragged into exact alignment with the magnetic field and the system has saturated.
There's no way you could squeeze further magnetic moment per unit volume out of it.
So again, you would expect, for this particular property, magnetic susceptibility of a material, if you plot it as a function of the applied magnetic field-- linear maybe be at low applied fields-- but eventually if you make the field strong enough, the system is going to saturate.
And then again, no longer will the direction of the magnetization, and that magnetic moment, be parallel to H. But the property becomes nonlinear if the applied vector is strong enough.
This is an example of a property where you would not go wrong at all by stating direct proportionality.
For ordering to occur, temperature and applied field go hand in hand.
Increased temperature tends to create more disorder.
Increased magnetic field tends to align the moments.
In the days when MIT had a national magnet laboratory, the magnet laboratory held the record for the strongest magnetic field ever produced artificially by man.
And if you took that magnetic field and applied it to this system of dilute iron in a glass, you would have to lower the temperature of the sample to about 3 Kelvin before you would begin to see saturation.
So even the strongest field that you could produce in a laboratory environment would not succeed in producing non-linearity until you cooled the sample down almost to absolute zero.
So here would be a case where under any practical consideration whatsoever, assumption of strict proportionality would be right on the money.
You'd be absolutely correct.
But there's another assumption built into this statement.
P, the generalized force, is a vector in the cases we're discussing now, and q is also a vector.
So when we write an expression of this form you're making another assumption.
And that is that the vector displacement that results is always exactly parallel to the vector that you apply.
Do I make a big deal out of this?
Isn't that always going to be the case?
I mean whoever heard of taking a piece of metal and putting an electric field on it in this direction and having the current run off in this direction.
It's absurd.
Or maybe it isn't so absurd.
So let's think of some of the atomistics of this process.
Now, since I know I'm among friends, I will not hesitate to display my ignorance, total ignorance, of polymer chemistry.
So suppose we had a piece of polymer.
That's what a polymer molecule looks like.
It's a more or less linear molecule, and so these might, in a very highly ordered polymer, be chains that lined up like this.
And suppose we now put an electric field on this polymer and asked how the current will flow.
Again, it would not be absurd to say that an electron sitting on this polymer chain in response to this field would be constrained only to flow in the direction of the chain and would find it rather difficult to hop from one chain to another.
So maybe, just maybe, we could apply an electric field to a polymer and it would have a flow in this direction, that would be pointing in this direction, and the current flow, J, would be in that direction.
So should we maybe rethink this idea of the direction of the generalized displacement being not parallel to the direction of the generalized force.
OK, these are hypothetical examples.
Let me now give you an example of a real property for a real material, for which the thing that happens, the vector that happens, is decidedly not parallel to the direction of the applied vector.
OK, thermal conductivity is something that relates a heat flux, usually represented by the symbol K, and relates that to a temperature gradient, dt dx, which is a vector.
The thing that gives rise to thermal conductivity can be either propagation of radiation as in propagation of light traveling through a transparent material.
But the other mechanism for thermal conductivity is modes of lattice vibration.
When a material is hot, the atoms are jiggling around and you can make the displacement of an individual atom be represented by a sum of waves moving in all different directions with a variety of wavelengths in a variety of amplitudes.
Take all those waves, add them together at a particular time, and you get the displacement of the atom.
Propagation of heat by this mechanism, by modes of thermal vibration, can be very, very anisotropic and probably the best example of this is-- get a single crystal of graphite.
And have the layers, the graphite sheets, which are hexagonal rings in which each carbon has three neighbors, and have the sheets be parallel to the surface of an extended two-dimensional slab.
Now we can't do that.
Single crystals of graphite don't occur in sizes like that.
But what you can do is make a material called pyrolytic graphite that you make by having a reaction in the vapor phase and having the soot that's formed settle down onto a substrate.
And what happens is you nucleate a graphite crystal in one orientation, and that grows preferentially with its layers parallel to the surface on which it's nucleated.
And these nuclei occur at random.
So you get a bunch of single crystals of graphite, which all have their layers, their three coordinated layers at parallel, but they are oriented at random about their c-axis.
And that's a material that's very easy to prepare.
And it's called pyrolytic graphite, and it has interesting applications and properties.
OK, now I'm going to suggest an experiment to you and I'll caution you, please do not try this at home.
Get yourself a piece of pyrolytic graphite.
Put a Bunsen burner underneath it.
Play the Bunsen burner on the bottom of the sheet of pyrolytic graphite.
And to determine temperature, we'll take a ball of cotton and put it directly above the flame.
And then take another ball of cotton and put it down at the end of the sheet.
If you do that and bring up the Bunsen burner, this piece of cotton will sit there and sit there and sit there and sit there.
And this piece of cotton will instantly burst into flame.
So here's a case where dt dx, the thermal gradient, points in this direction.
And the heat flow goes off in a direction at right angles to the temperature gradient, almost exactly at right angles.
It turns out that the thermal conductivity in this direction, the value of K in the-- I want to put crystallographic directions on this.
But K parallel to the layers is equal to 10 to the third times K, perpendicular to the layers.
So there's an anisotropy of the property that amounts to a factor of 1,000.
Very dramatic.
So what I'm leading up is that in a general relation between a generalized displacement, p, and a generalized force-- I'm sorry, I'm using q-- p and q-- we just can't simply put some constant in front.
Because that assumes inherently that the direction of what happens, the vector that happens, is exactly parallel to the direction of the applied vector and that generally is not true.
The difference may be small but it has to be taken into consideration.
All right, let me now suggest a way of patching this up.
What I'm going to do is assume-- and it is an assumption-- we're going to assume that each component of the vector that happens is given by-- in fancy terms, the vector that results as the generalized displacement is given by a linear combination of every component of the vector that's applied.
And that's what we've defined as the generalized force.
So how would we express this analytically?
I am going to use as my coordinate system, not x, y, and z as we usually do, I'm going to call it x1, x2, and x3.
The reason is we'll see some unique properties of these indices that are very useful algebraically.
So I'm going to now take my applied vector, p, and I'm going to assume that it has three components, p1, p2, and p3 along x1, x2, x3, respectively.
My coordinate system, although I did not state it explicitly, is going to be a Cartesian coordinate system, not a crystallographic coordinate system.
So what I'm proposing here is that we take each component of the resulting vector q-- let's say q1-- and we'll assume that that's given by a linear combination of all three components of the vector p. So it'll be some number times p1 plus some number times p2 plus some number times p3.
And those numbers in general will be different.
So let's say I call the coefficient a, and now I'm going to define a convention that will stay with us.
I'm going to define each of these coefficients, a, in terms of the index of the component of the generalized displacement which is being computed, and the coefficient modifies the component of the generalized force for that particular term.
So I'm going to call this a11, where the 1 goes with this and the 1 goes with this.
I'm going to call this a12, so that the 1 again says it's a contribution to q1.
The 2 says that this term modifies p2.
.
And this similarly would be a13.
For the term component of the generalized displacement, q2, I'll use the coefficients a21 times p1 plus a22 times p2, plus 23 times p3.
And q3 similarly will be a31 times p1, plus a32 times p2 plus a33 times p3.
So I've got three simultaneous equations then, one for each component of the vector that results, the generalized displacement.
So I can sum up this set of three equations by saying that the i-th component of q, where this is some particular component, q1, q2, or q3, is given by the sum over j from 1 to 3 of aij times p sub j.
So I've got ai1 times p1, ai2 times p2 plus ai3 times p3.
So this is in a nice, compact little nugget the expression that we are assuming will apply for all sorts of physical properties in which this is a vector and this is a vector-- electrical conductivity, magnetic susceptibility, thermal conductivity, and so on.
OK, this is a bad point to introduce something as hairy as what comes next.
But I've got one minute left, and I think I can do it.
I'm going to introduce something called the Einstein convention after old Albert Einstein himself.
I'm sorry, but nobody said everything this term had to be easy.
So let me introduce, if you're ready for it, the Einstein convention.
Old Albert, I think, was just as lazy as anybody else.
And he said it is going to be a bloody pain in the butt-- I don't know if he put it exactly this way.
It's going to be a pain in the butt to write this summation every time we want to combine three terms in a linear combination.
So the Einstein convention is let us throw out the sigma and write this expression just as qi times aij p sub j.
And whenever we see a subscript repeated, summation over repeated subscript is implied.
OK so that's the Einstein convention.
It will save us the trouble of writing in a lot of summation signs.
But I would caution you that this convention, compact and convenient as it is, can define some polynomials that are nightmares.
So let me give you one example of this, and this is actually a physical property.
Let me say that Cijkl is given by ai capital I aj capital J ak capital K, al capital L times let's say D, capital I, capital J, capital K, capital L. That actually, believe it or not, means something physically and we're going to get to that in due course.
But what this is, taking the Einstein convention into account, this is a quadruple summation over capital I, capital J, capital K, and capital L. These are very often referred to in this business as dummy indices, meaning that they don't really mean anything physically.
They're just indices of summation.
I would caution you that this is a term that does not permute.
If I say those are dummy indices, it means one thing.
If I say those are indices, dummy, it means something completely different, and you're apt to get a poke in the nose.
OK so these are dummy indices.
Now, what does this represent?
I won't say what it represents physically, but this consists of terms in four variables times a coefficient.
So there are five quantities in each term.
If I sum over capital I, J, and K from 1 to 3, there are going to be 81 such terms, each with five elements in each term.
So it's going to be on the order of 405 terms in this summation, and we've collapsed the whole thing down, 405 terms in this nice summation.
So it is a great facility for writing down expressions explicitly, but these terms can hide a terribly, terribly complex polynomial.
All right, I think that's a good place to quit.
It continues to amaze me, though, how some absolutely trivial convention, when first proposed by a great man, will carry that man's name no matter how stupid it is.
So this is called the Einstein convention by everybody who works in this field.
Another one, just briefly to finish up, when you discuss dislocations, you talk about a circuit of steps around the dislocation.
That's called a Burgers circuit.
And if you do the same circuit in a corresponding piece of material that doesn't have a dislocation, if this circuit closes, this circuit fails to close by something that's called the Burgers vector.
Every book on dislocation theory says, Shockley called this good material.
I'm sorry, Shockley called this good material, and Shockley called this bad material.
Big deal-- good material, bad material because it's got a dislocation in it.
But that is identified with Shockley's name in every discussion of dislocation.
I'm just jealous because nobody has ever said according to me, such and such is-- all right, see you later in the week for more great things.
All right, I would like to then get back to a discussion of some of the basic relations that we have been discussing.
We didn't get terribly far, but I'd like to start with the Cartesian coordinate system that we set up.
Rather than using x, y, and z, I'm labeling the axes x1, x2, and x3.
And we'll see that the subscripts play a very useful role in the formalism we're about to develop.
Now, the first thing we might want to specify in this coordinate is the orientation of a vector and its components.
So let's suppose that this is some vector P. And what I will do to define its orientation is to use the three angles that the vector makes, or the direction makes with respect to x1, x2, x3.
And we could define these angles as theta1, that's the angle between the direction and x1, theta2, the angle between our direction or our vector and x2, and finally, not surprisingly, I'll call this one theta3.
So the three components of the vector could be written as P1, the component along x1 is going to be the magnitude of P times the cosine of theta1.
The x2 component of P would be the magnitude of P times the cosine of theta2.
And P3, the third component, would be the magnitude of P times the cosine of theta3.
Now, we will have so many relations that involve the cosine of the angle between a direction and one of our reference axes that it is convenient to define a special term for the cosines of these angles.
So I'll define this as magnitude of P times the quantity l1, magnitude of P times l2, magnitude of P times l3, which is a lot easier to write.
And we will define these things as the direction cosines.
With these equations it's easy to attach some meaning to the direction cosines.
Suppose we had a vector of magnitude 1, something that we will refer to as a unit vector.
And if we put in magnitude of P equal to 1, it follows that l1m l2m l3 are simply the components of a unit vector in a particular direction along, obviously, x1, x2, and x3, respectively.
Trivial piece of algebra, but it attaches a physical and geometric significance to the direction cosines.
Now, the vector is something that could represent a physical quantity.
In any case, it is something that is absolute.
And it sits embedded majestically, relative to some absolute coordinate system.
The magnitudes of the components P1, P2, and P3 will change their values if we would decide to change the coordinate system that we're using as our reference system.
So the next question we might ask is, suppose we change the coordinate system to some new values, x1 prime, x2 prime, and x3 prime?
And I'll illustrate my point with just a two dimensional analog of this.
This is x1, and this is x2.
And this is my vector P. And I change x1 to some new value, x1 prime, and change x2 to some new orientation, x2 prime.
Then clearly the component of P on x1 has changed its numerical value if I refer it to x1 prime instead.
And similarly, this value would be the component P2.
If I change the direction of x2 and draw a perpendicular to x2, this would be P2 prime.
So if I change coordinate system along the fashion I suggested, the three components of a vector, P1, P2, P3, are going to change to some new values, P1 prime, P2 prime, P3 prime.
OK.
So the question I'd like to address next is given the change of coordinate system, and given the three components of P in the original coordinate system, how do I compute the values of the new components P1 prime, P2 prime, P3 prime?
I'll say it in words, and then we'll define a mechanism for specifying the change in coordinate system.
What I'll say is-- and this was apparent in the sketch that I just erased-- the new component of the vector P1 prime is simply going to be the sum of the components of P1, P2, and P3 along the new x prime direction.
So I'm saying that this is going to be the sum of the component of P1 along x1 plus the component of P2 along x1 prime and the component of P3 along x1 prime.
So in short, I'm doing nothing more complicated than saying, I can get the values of the new components if I take the vector P, split it up into its three parts, and then find the component of each of these three parts along the x1 prime access, then do the same thing for the x2 prime axis, and then the same thing for x3 prime.
So I'm going to need, now, a notation for a change in a three-dimensional Cartesian coordinate system.
So here is x1, here is x2, and here is x3.
And I will change them.
And again, I'm always keeping the coordinate system Cartesian.
So here's an x1 prime, here's an x2 prime, and then x3 prime will point out in some direction like this.
I don't want a prime on that.
So I'm going to say now that the component of P along the new x1 prime is going to be the magnitude of P1 times the cosine of the angle between x1 and x1 prime, plus P2 times the cosine of the angle between x1 and-- am I doing this right?
P1 onto x1 prime.
And I want P2 onto x1 prime.
So this is going to be the angle between x2 and x1 prime plus P3 times the cosine of the angle between x3 and x1 prime.
Well, we used C's or l earlier on to represent a direction cosine.
Let me define Cij as the cosine of the angle between x1 prime, xi prime, and x sub j.
So that means I can write this expression here in this nice compact form.
With our definition of direction cosines, I can say that P1 prime is going to be equal to C1 1 times P1 plus C1 2 times P2 plus C1 3 times P3.
Can write that P2 prime in the same way.
It's going to be the cosine of the angle between P2 prime and P1 and x1, plus the cosine of the angle between x2 prime and x2 times P2 plus C2 3 which is the cosine of the angle between x2 prime and x3, times the component P3.
And in a very similar fashion, P3 prime will be C3 1 P1 plus C3 2 times P2 plus C3 3 times P3.
So here is the way a vector will transform.
And we can write this compactly in matrix form.
We can say that P sub i prime, where this is a column matrix, one by three, is going to be equal to Cij, a three by three matrix times the original components of the vector P sub j.
And just to cement the notation that we're using, if I put my old axes up here, x1, x2, x3, and put the new axes, x1 prime, x2 prime, x3 prime, down this way, then the cosine of the angle between the quantities that are in this column and the quantities that are in this row would be C1 1, C1 2, C1 3, C2 1, C2 2, C2 3, C3 1, C3 2, C3 3.
Nothing fancy except the notation.
It's the description of some very simple geometry.
This array, Cij, is something that I will refer to as a direction cosine scheme.
Let me pause here and see if that's all sunk in, whether you have any questions on this.
One of the nasty properties of what we're going to be doing for the next month or so is that the notions are really very, very simple, but the notation is horribly cumbersome and complex.
So it takes a bit of getting used to in application to actual real cases before you feel fully at home with it.
OK. Just a matter of definition so far.
Let me note that this direction cosine array is going to be useful for defining how a vector changes as we go from the original coordinate system to a new coordinate system.
But this direction cosine array will also tell us how the axes in one coordinate system are related to the axes in the new coordinate system.
It follows from the fact that the axes themselves can be regarded as unit vectors.
And we said that the components of a unit vector are the direction cosines of that vector, relative to a coordinate system.
So let's ask, what are the new components of x1 in terms of the original axes unprimed.
Well, x1 prime is going to be the unit vector x1 times the cosine of the angle between x1 and x1 prime, plus the unit vector along x2 prime times the cosine of the angle between x1 prime, and x2.
And that's C1 2.
Plus x3 regarded as a unit vector times the cosine of the angle between x1 prime and x3.
So we can actually write an equation for unit vectors along each of our new axes.
And they will go as C1 1 times x1, C1 2 times x2, C1 3 times x3, plus C2 2 times x2 plus C2 3 times x3 times x3 prime.
And x3 primal will be C3 1 x1 plus C3 2 x2 plus C3 3 x3.
So this, then, is an equation between the unit vectors along the three reference axes in the new coordinate system relative to those in the original coordinate system.
And the direction cosine scheme does the job.
OK. We could, using the same argument, give the array that specifies the reverse transformation.
If we would change our mind, for example, and say we don't like what we've done, let's write the original coordinate system x1, x2, and x3 in terms of the unit vectors along the new axes.
And we can use exactly the same array.
We can say that the original x1, in terms of the three new axes, x1 prime, x2 prime, and x3 prime, is going to involve the cosine of the angle between x1 prime and x1, and that is C1 1, plus the cosine of the angle between x2 prime and x1, and that's C2 1, plus the cosine of the angle between x3 prime and x1, and that C3 1.
If we continue on this, if you have the idea, the angle x2 is going to be given in terms of x1 prime, x2 prime, and x3 prime, as the cosine of the angle between x2 and x1 prime, and that is C1 2.
Here we want the cosine of the angle between x2 and x2 prime, and here the cosine of the angle between x3 prime and x2.
And you can see the way this is playing out.
C1 3 times x1 plus C2 3, x2 prime plus C3 3 times x3 prime.
So there's the reverse transformation, using the same array of coefficients as we did the first time.
So it turns out if we write this symbolically in a compact form, xi prime is given by Cijx sub j.
And the reverse transformation using the same direction cosine says that xi is going to be Cji times x sub j prime.
In other words, the reverse transformation, let's write it as Cij minus 1, the inverse transformation, turns out to be simply Cji.
And that, in matrix algebra, is written as the transpose of the original array of coefficients.
And transpose is either given by a squiggle, a tilde on top of the matrix.
Some people like to use a superscript T. But we'll use this particular notation.
But you can see either notation used to indicate the transpose.
The array Cij, which has this property, and it also has another property which I won't bother to prove, but the determinant of Cij is equal to 1.
And this is something called a unitary matrix.
Unitary matrix has the property that the inverse matrix is the transpose.
We will very, very shortly start writing down numbers for some specific transformations.
And then I think that will give us a little facility in doing these manipulations.
Comments or questions?
Is this old stuff or old stuff for which the notation is still confusing?
All right.
Let me point out something that is perhaps apparent to you.
And that is that not all nine of these numbers are independent.
There are relations between them.
And let's point out some of these relations.
C1 1, C1 2, C1 2 represent the components of a unit vector along x1 prime, in the original coordinate system of the elements in any row is equal to 1.
Because these are the components of a unit vector.
And the magnitude of a unit vector is 1.
In the same way, if we look at any column of terms in this matrix, for example, C1 1, C2 1, C3 1, these are terms that represent the cosine of angles between x1 in the original coordinate system and x1 prime, x2 prime, x3 prime, our new coordinate system.
So this gives us the magnitude of x1, but x1 is a unit vector.
So it follows that the sum of the column C1 1 squared plus C2 1 squared plus C3 1 squared also has to be unity, because that gives us the magnitude of a unit vector along x1, So the sum of the squares of elements in any column of the direction cosine is unity.
These expressions are useful.
But they have one ambiguity.
That is the cosine of an angle can be either positive or negative, depending on whether the angle is less than 90 degrees or greater than 90 degrees.
These relations involve the squares of direction cosines, and therefore we can't tell whether the direction cosine itself is positive or negative.
So let me put down a limitation here.
And that is we cannot tell the sign.
Every time I point this out to people I wince inside.
Because I once spent two weeks trying to debug a computer program, and it wasn't working.
And it turns out the reason it wasn't working properly was that I didn't realize that you cannot tell the sign when all you know is the squares of the direction cosines.
So I remember this as a rather pointed observation.
Happily, there are other relations among this array of coefficients.
This row of terms represents the components of x1 prime in the original coordinate system x1, x2, x3.
This row immediately below it represents the components of a unit vector x2 prime relative to the original coordinate system x1, x2, x3.
Our coordinate systems are Cartesian.
Therefore, this unit vector has to be perpendicular to the unit vector along x2 prime.
And that means their dot product has to be 0.
So let me indicate that this way.
The unit vector along x1 prime dotted with the unit vector along x2 prime has to be 0.
And that dot product is going to be C1 1 times C2 1, that's the product of these two terms, plus C1 2 times C2 2 plus C1 3 times C2 3.
And that has to be 0.
And this involves only the first product of the direction cosine.
So to make it come out 0 when we add up the magnitudes, we will get the sign of the direction cosine.
So this is a much more powerful relation.
And similarly, the product of the coefficients in the first and the third row have to add up to 0.
And the second and third row have to be 0.
So there are three different relations we can write between products of corresponding coefficients in the rows.
So to sum up in words, the sum of the corresponding elements-- of the product of-- in any pair of rows of Cij must be 0.
But we're not done yet.
If we look at the columns in this array, this represents the components of x1 relative to x1 prime, x2 prime, x3 prime.
And these terms here represent the components of x2 relative to x1 prime, x2 prime, and x3 prime.
And for similar reasons, the dot product of those two vectors has to be 0.
So we can say that in addition, the sum of any sum of pairs of corresponding coefficients in any pair of columns must be 0.
So we're working here on the direct matrix of the transformation.
We've seen that the reverse relation, the inverse matrix of Cij is Cij transpose.
And therefore the inverse matrix has to have this same relationship that the products of terms and rows or columns, any pair of rows or columns has to be 0.
Now, there's one other pair of relations among the coefficients, which is not quite so geometrically obvious.
And I won't attempt to prove it.
I'll just state it.
I said a moment ago that these are unitary matrices.
The determinant of the coefficient Cij then has to be unity.
But interestingly, it will be plus 1 if one goes from a right-handed system to a right-handed system.
That is to say the set of axes x1, x2, x3 might be right-handed.
And if the new coordinate system x1 prime, x2 prime, x3 prime is also right-handed, then the determinant of coefficients is plus 1.
On the other hand, if one goes from a right-handed system to a left-handed reference system or from a left-handed one to a right-handed coefficient, then, interestingly the determinant of coefficients is minus 1.
So the determinant of the matrix of the transformation is plus 1 if you go to coordinate system of the same chirality.
It's equal to minus 1 if you go to a coordinate system of changed chirality.
All right, so to repeat something I said at the outset but which you now probably truly believe, the elements in the direction cosine scheme that get you from one coordinate system to another have lots of inter-relations.
And all of these coefficients are not independent.
There are these relations that couple them.
How are we doing on time?
We mentioned last time that a large collection of physical properties of materials are properties that relate a pair of vectors.
So let me, to make this specific, talk about a particular physical property, electrical conductivity.
And electrical conductivity relates a current density vector, and that it charge per unit area per unit time to an applied vector, and that vector is the electric field vector.
And the electric field has units of volts per unit length, so volts per meter in MKS.
And provided the electric field that's supplied is not too strong, it turns out that every component of the current flow is given by a linear combination of every component of the applied electric field.
So the flow of current along x1 will be given by a proportionality constant, an element sigma 1 1 times the x1 component of the electric field.
Let me write it out the first couple of times we do this.
Sigma 1 1 times E1 plus sigma 1 2 times E2 plus sigma 1 3 times E3.
J2 will be sigma 2 1 times E1 plus sigma 2 2 times E2 plus sigma 2 3 times E3.
And J3 will be equal to sigma 3 1 times E1 plus sigma 3 2 times E2 plus sigma 3 3 times E3.
Looks formally like the relation between unit vectors that define a coordinate system.
Number of subscripts is the same, but actually this is something that's completely different.
It's dealing with vectors that have some physical significance.
So in compact reduced subscript notation, this is the definition of electrical conductivity.
This matrix that relates the electric field vector to the current density vector is said to be a tensor of the second rank.
OK, tensor.
First thing you might say, why do you call it a tensor, dummy?
It's a matrix.
It's a plain old matrix.
There's a subtle but very important difference.
A tensor is a matrix with an attitude.
And I'll make the distinction clear a little bit later on.
But there are tensors also of higher rank.
These expressions where summation over repeated subscripts is implied can hide, as I indicated last time, some absolutely horrendous polynomials.
But tensor at very least is a term that makes the faces of all who hear it pale, and makes the knees of even the very strong to weaken.
And in case you don't believe that, I'll show you what I have to wear whenever I give these lectures.
And consequently it's kind of scuzzy and worn out.
But I have to put on these knee braces from wobbling braces.
And you can see what it says on here.
"Tensor."
So that's a consequence of this frightening definition that we've just made.
Let me next set the stage for what we ought to do next.
E sub j represents the components of an electric field, x1, x2, x3, in a first coordinate system.
ji represent the components of the current flow in a coordinate system, x1, x2, x3.
If we were to change coordinate system for any reason, these three numbers would wink on and off.
Some might go negative.
The magnitudes would change.
And as a result, the components of the current flow would have to do the same thing.
Because the components of these vectors, without changing anything physically, have to change their numerical values if we refer them to a new set of reference axes.
If we change coordinate system and these numbers change, and if we change coordinate system, these numbers change, we're still applying field in the same direction.
The current still flows in the same direction.
But the components we use to define these two vectors change.
And it follows just algebraically, the elements of the tensor have to change and link into different values.
It follows automatically.
So a question, then, is that if we have a coordinate system, x1, x2, x3, and we change it into a new coordinate system, x1 prime, x2 prime, x3 prime, then j sub i changes to some new values, j sub i prime.
E sub j changes to some new values, E sub j prime.
And therefore, of necessity, sigma i j, the conductivity tensor, has to change to new values sigma ij prime.
So I'll let you rest up to brace yourself for this.
The question is, how can we get sigma ij prime, the nine elements of the tensor in the new coordinate system, in terms of the direction cosine scheme that defines this transformation and in terms of the elements of the original conductivity tensor?
And this, my friends, is what makes a tensor a tensor and not a matrix.
I can write a matrix for you, a really lovely matrix.
Let's put in some elements here.
Let's put in 6.2, square root of minus 1e, and 23.
And as other elements, I'll put in pi 23.4, 6, and 0.
It's a perfectly good matrix.
It's just an array of numbers, any numbers, real or imaginary, or whatever I like.
So this is a matrix.
What a tensor is, is a matrix for which a law of transformation is defined.
And that's what makes a tensor a tensor.
What does it mean to take this two-by-four matrix that I just wrote down?
How do I transform that to a different coordinate system?
It's meaningless, just an array of numbers.
It's an array of numbers that has some useful properties, like matrix multiplication and the like.
But to talk about transformation of this set of four ridiculous numbers to a new coordinate system is something that's absolutely meaningless.
Not so for something like conductivity or the piezoelectric moduli or the elastic constants.
These change their values.
There's a law of transformation when we go from one Cartesian reference system to another.
So what we will do when and if you return is to derive a law for transformation for second-rank tensors, and then, by implication, look at higher-rank tensors and decide how they would transform.
But why would you want to do this?
Why would you want to muck things up and have to worry about transforming these numbers?
Well, let me give you just one simple example.
Suppose we had conductivity of a plate, of a crystal.
And what would you do?
You'd measure it relative to a set of axes, which, if you have a little fragment of crystal, you have no reference system.
So say that the axes x of i are taken relative to the lattice constants of the material, so relative to the edges of the unit cell, possibly.
Then you decide that this material really has some useful properties, and you would like to cut a piece out of it so that you get a plate for which the maximum conductivity in that plate is in a direction normal to the plate.
So you know just what sort of plate you want to cut out.
You know what the direction cosines are.
But once you've cut a plate from the crystal, the tensor relative to the old axes, x1, x2, x3, is not going to be terribly useful.
You're going to want to find the tensor relative to this as one set of axes, and these perhaps as a new set of axes within the plane of the plate.
So there's a good example.
Cut a piece from a crystal and cut that piece so that the extreme values are along x, y, and z for the new coordinate system.
Then you will be faced with the necessity of transforming the tensor from one coordinate system to another one.
Or you might measure the thermal conductivity tensor.
You might want to cut a rod out of the material so that the maximum conductivity or the minimum thermal conductivity is along the direction of the rod.
You might want to use that as a push rod to hold a sample in position and not have it be a big heat sink for the temperature that's inside of your sample chamber.
So I've hopefully convinced you that there are lots of cases where it would be necessary and convenient to transform the tensor that describes a property to a new coordinate system.
All right, so let us take our break now.
Some internal clock always tells me when it's five of the hour, unless I get really excited about something.
And it is indeed that time now.
So let's stop.
OK, ready for more?
I defined a problem for you.
Now let's address it.
Said that if we have one coordinate system, and if we have some vector, q, that's defined as a second-rank tensor, aij times some other vector p sub j-- and let me digress in passing.
I am very careful to say "a second-rank tensor" and not a "second-order tensor," because higher-order order terms means negligible and non-important.
And when I say "second-order tensor," I don't mean to say it's not important and negligible.
It's very important, so I say "rank," which has some sort of dignity to it.
So I don't like the term "order," because it has another meaning.
OK, so here is a tensor that relates a vector pj to give us the components of a vector qi.
If we change coordinate system, the components of p, representing exactly one in the same vector, wink on and off and take different values.
The values for q take on different values, and therefore, of necessity, the three by three array of coefficients, which relates these different numbers, must also change its numerical values.
And I hopefully convinced you at the end of last hour that there's times when you might actually want to do this when you're cutting out a particular sample from a single-crystal specimen.
How do we get the new tensor in terms of the direction cosine scheme that specifies the change of axes and the original tensor?
So I am going to refer to my notes quite closely here, because I want it to come out pretty and not have to redefine variables when I'm done.
So let's start with the original tensor relation, q sub i equals aij times p sub j.
Now, what we want is something of the form q sub i prime equals aij prime times p sub j prime.
And we know the relations forward and reverse in terms of the direction cosine scheme cij So let's begin by writing qi prime in terms of qi And we know how that is going to transform.
It's going to be elements of the direction cosine scheme cim times q sub m. So that will give me the i-th component of q in the new coordinate system.
I know how q arises from the applied vector p. So let me write qm in terms of the applied pj.
And this is going to be aij times-- be careful of my variables here-- this is going to be a ml times p sub l. And that is going to, from my definition of a second-rank tensor, give me the m-th component of q.
So far, so good.
I've got two different repeated subscripts here, so this is a double summation.
Now, I'll have what I want to have, namely a q sub i prime on the left-hand side and a p sub j prime on the right-hand side if I can express the original components of p in terms of the new components of p. And I do that by the reverse transformation.
So let me now write a ml.
And then in place of p sub l, I will write cjl times p sub j prime.
Notice the inverted order of the subscripts.
That is the reverse transformation that's going to give me p sub l. So you really have now what I'm after.
This is a triple summation in m, l, and j. I can write the terms that are in what is going to be a triple summation over a product of terms.
I could write these terms in any order.
So to simplify it, let me write q sub i prime is equal to cim cjl times a ml, times p sub j prime.
And now, hotcha, I've got an expression that has q prime on the left and p prime on the right, and paying close attention to my notes so that the subscripts all came out the way I would like them to.
So what this says is that the transform tensor aij prime is going to be equal to, by definition-- or what we've shown here, it's going to be equal-- just picking off terms-- it's going to be cim cjl times a ml.
Just picking off these terms.
m has no physical meaning, because m simply is an index of summation.
l has no specific meaning.
That is just an index of summation.
But the i and the j do have meaning.
They go with the i and j on the particular tensor element that we were attempting to evaluate.
So in other words, to be specific, if we want the new value of the tensor element a1 2 prime, it's going to be c1 something, c2 something, and those somethings m and l would vary from 1 to 3.
So if I write this out not in the reduced subscript notation but put a summation sign in there, so a12 prime is going to be the sum over m and the sum over l of terms c1m-- the first index is always 1-- c2 something-- first index is always 2-- and then m and l take on all possible values.
OK, we've got two results.
We learned how to transform a vector.
And a vector, if you will, is simply a tensor of first rank.
And transforming the vector, we summed over all three components of the original vector.
And the coefficients in that summation were one direction cosine.
Now we're transforming a second-rank tensor.
Again, each new element is a linear combination of all nine of the elements in the original tensor, and the coefficients are a product of two direction cosines.
So we've got two points.
Let's draw a line through them, and we can say that, in general, any new tensor element of any rank-- and you could prove it through exactly this method by going up, now, to the third rank, fourth rank, and so on, and writing substitutions of this form-- it turns out that a new tensor element aijkl however far you want to go, is going to be given by a linear combination of direction cosine element ciI, cj capital J, ck capital K, cl capital L, times however far you have to go times aijkl, and so on.
So these are the true indices.
These have physical meaning.
These have relevance to how a particular property will behave.
The capital I, capital J, capital L, and so on, are what we referred to last time as dummy indices.
These are indices of summation.
But the first index on the direction cosines has specific meaning.
They are tied to the indices on the subscript of the term that you would like to evaluate.
So we've got now a very profound relation for a tensor of any rank.
And really, it just involves substitutions using the reverse or the forward transformation until you get one element on the one side related to another element on the right-hand side.
And this is a specific element in the new tensor, in our case, of the second-rank tensor, aml.
This is all very abstract.
It is something that we'll have to do a couple of times for a real problem before you see how it works out.
The number of elements that figure into these transformations is really astronomical.
Suppose, for example, the tensor involved were something like the elastic stiffness tensor, which is represented by c. And we need four subscripts.
This is a tensor or fourth rank.
If we wanted to transform a particular stiffness to a new coordinate system, we would need a summation ci capital I, cj capital J, ck capital K, cl capital L, times all of the elements in the original tensor, aijkl.
A fourth-rank tensor consists of an array of 9 by 9 terms.
So there are 81 of these.
We'd have four direction cosines out in front, and there would be a total of 81 times 5 characters that we would have to write.
So to do the complete tensor transformation, we would have to write on the order of 400 quantities to get just one of the 81 elements in the new transform tensor.
So the total number of elements we'd have to write to do this would be 81 squared times 5.
That's a lot of elements.
We'll do a few of these transformations directly, but let me assure you that if we do a transformation that is going to involve symmetry, a lot of the direction cosines, if we're lucky, will be 0.
So it's not quite as onerous as it seems.
So we would make use of this sort of formalism if we wanted to go from one set of reference axes to a new set that might represent a special specimen that we cut out of a crystal.
But there's another formal way in which we could make very profound and non-intuitive use of these relations.
Crystals, except for the abominable triclinic crystals, have symmetry.
If a crystal has symmetry, you can transform the solid physically by that symmetry operation.
And you have to measure the same property before and after.
So suppose we have a crystal that has a twofold axis.
And this crystal is something that looks like this.
So this is side A, and this is side B.
We could move the crystal by a 180-degree rotation.
Put it down.
I won't draw it, because it's going to look exactly the same, except now this thing-- I won't draw, and I do draw it-- this is face A, and this is face B.
If we had electrodes on the crystal before and after that transformation, we have to measure, let's say, the same electrical conductivity for both orientations of the crystal.
Now, moving a crystal relative to some coordinate system, relative to a pair of electrodes that we're fastening onto the crystal, is exactly the same thing as doing the reverse transformation of the coordinate system.
That's a vague, strange-sounding term.
So suppose we have a crystal with a fourfold axis with four faces, A, B, C, D. And here are our electrodes.
To move the crystal relative to the electrodes by a 90-degree rotation would involve rotating face D up to this location.
A would move to this location.
C would move to this location.
B would move to this location, and we'd fasten electrodes on the crystal again.
If the crystal originally had a coordinate system such that this were X1 and this is X2, moving the electrodes onto a different direction on the crystal is the same as moving the crystal in the opposite sense.
So we could either envision moving the crystal relative to the electrodes like this, or we could move the electrodes relative to the crystal by the reverse transformation.
And the result is the same.
So what I'm saying is that if a crystal has symmetry-- and let me be specific and suppose that our crystal has a twofold rotation axis along X3.
Let's ask how that twofold access would change the coordinate system relative to the crystal.
That's the same as moving the crystal relative to the coordinate system.
It's going to take X1 and move it to this location X1 prime.
It's going to take X2 and move it through 180-degree rotation.
This is going to be X2 prime.
And if the twofold axis is along X3, X3 prime is the same as X3.
So what is the direction cosine scheme for this change of axes?
You might immediately start working and saying, well, C11 is the cosine of the angle between X1 prime and X1.
That's 180 degrees.
Cosine of 180 degrees is minus 1.
But let me remind you that the direction cosine scheme, c ij, simply gives us the relation between the new axes x sub i prime and the old axes, x sub j.
So let me, just by inspection, write down the relation between these two sets of coordinate systems.
So X1 prime is equal to minus X1.
X2 prime is equal to minus X2.
X3 prime is equal to X3.
So the direction cosine scheme for this particular transformation is simply minus 1 0 0, 0 minus 1 0, 0 0 1.
So I just evaluated a nine-element direction cosine scheme by inspection, if I can write the relation between the coordinate system before and after the transformation.
OK?
And as we examine higher symmetry, the same is going to be true for the threefold axis, let's say, along the 1, 1, 1 direction of a cubic crystal, for a sixfold axis, and so on.
So we'll be able to write the direction cosine schemes for a symmetry transformation simply by inspection.
So for a twofold axis along X3, this is the form of the direction cosine scheme.
So now let me transform the elements of a second-rank tensor term by term and see what we get.
Suppose I want-- let's stick with conductivity as an example.
Suppose I want the value for the conductivity element sigma1 1 prime.
That's going to be c1 something, c1 something, because these are the elements that go in here, times every element of a conductivity tensor sigma lm.
The only element of the form c1 something that is non-zero in this row c11, c12, c13, is the term c11.
In the same way in the same row, the only term of the form c1 something is c11.
So this is going to be simply c11 times c11 times sigma11.
That's the only term that survives.
c11 has a numerical value of minus 1, and that says that sigma11 prime is equal to sigma11.
So is there any constraint, any restriction on sigma 11?
No, sigma11 could be anything it likes.
So there's no constraint.
Let's do another element.
Let's see what sigma12 prime would be.
This will be c1 something times c2 something times sigma something something.
The only element of the form c1 something that is non-zero is c11.
So I'll put in a 1 for the l. The only direction cosine element of the form c2 something which is non-zero is c22.
So I'll put in c22 and let n be equal to 2. c11 is minus 1, c22 is minus 1.
So again, this gives us something not terribly interesting.
Sigma 12 prime is equal to sigma 12.
So there's no constraint, at which point you're probably getting very restive, say, this is not telling us anything.
So let me shake you up by doing one further transformation, and that is to find the value for c13 prime.
And that would be c1 something, c3 something times sigma lm.
The only form of this term of the form c1 something, that's non-zero is c11, as we've seen.
So I'll put in just that single term and replace l by 1.
The term of the form c3 something that is non-zero is c33.
And I'll put in a 3 for m, and this is then c11 times c33 times sigma13.
1, And c11 is minus 1. c33 is plus 1.
And that says that sigma13 prime is minus sigma13.
But if this is a symmetry transformation, the tensor has to remain invariant.
And if we're to have sigma13 equals minus sigma13, there's only one number that can make that claim, and that's 0.
So sigma13 is identically 0.
And that places a rather severe constraint on the way in which the crystal is going to relate an applied electric field to a current flow.
Sigma11 is anything.
Sigma12 is anything.
Sigma13 has to be identically 0.
Now, let's cut to the bottom line.
The direction cosine scheme is diagonal, so we can say that for any element that we pick to transform sigma ij prime is going to be cii, the diagonal term which has the second subscript equal to the first, times cjj times sigma ij.
And this says that if we have i or j is equal to 3, then sigma ij has to be 0, because we're going to have a minus 1 times a plus 1.
If neither i or j is equal to 3, then again, we will have a minus 1 times a minus 1.
There will be no constraint.
And the only final possibility is that both i and j are equal to 3.
That would be the single element c33.
Then we would have plus 1 times plus 1 as the product of direction cosines, and there will be no constraint.
So for a crystal that has a twofold axis, and in which that twofold axis is along the direction of x3, the form of the tensor will be sigma11, sigma12, 0, sigma21, sigma22, 0, sigma31, that's going to be equal to 0, and sigma33 has no constraints.
So rather than having nine elements, there are only five independent elements rather than nine.
And there is now another relation that can occur in a second-rank tensor.
The off-diagonal terms sigma12 and sigma21 do not have to be related.
But for most-- but not all-- most second-rank tensor properties happily have sigma ij identical to sigma ji.
In other words, the tensor is symmetric across its principal diagonal.
That is a condition that does not arise from symmetry That depends on the specific physical property.
So let me emphasize that this depends on the tensor property.
And for a great many physical properties-- conductivity, diffusivity, permeability, susceptibility-- you can show that the tensor has to be symmetric.
But there are a lot of tensors, particularly for the more obscure physical properties, where to my knowledge, this proof has never been given.
And along the same lines, it is well known that there is one physical property for which this is not true.
This is the thermal electricity tensor.
So for at least one property, you can show for sure that the tensor does not have to be symmetric and that for a crystal of symmetry 2, this term and this term are definitely not equal.
All right, let us do another transformation for another symmetry, and we can see that it goes fast when the direction cosine scheme is relatively sparse.
Let's ask the restrictions, if any, that are imposed by inversion.
So what is the direction cosine scheme here?
Here's x1, here's x2, here's x3.
Then operation of inversion at the intersection of these axes will invert the direction of x1 prime to here.
It'll invert the direction of x2 prime here.
It will invert the direction of x3 prime to here.
So the relation between the reference axes is that x1 prime is equal to minus x1.
x2 prime is equal to minus x2.
x3 prime is equal to minus x3, so that the form of the direction cosine scheme crj is minus 1, 00, 0 minus 1 0, 00 minus 1.
Slightly different from that for a two-fold axis for which the first two diagonal elements were minus 1, the third one was 0.
Well, let's jump right to it and see if we can generalize this.
It's a diagonal direction cosine scheme once again.
And this says that if we transform a particular element sigma ij, it's going to be given by cil, cjm, sigma lm, where l and m are variables of summation.
The only ones that survive are the ones for which i equals l and for which j equals m. So it's going to be cii, cjj times sigma ij.
Regardless of the values of i and j, the diagonal terms are always minus 1.
And therefore, sigma ij prime is always going to turn out to be equal to sigma ij, so there's going to be no constraint on any element.
And this shortens the job that's facing us immeasurably.
So let me write that down, because that's important.
Inversion imposes no constraint on any second-rank tensor property.
So if we stay with monoclinic crystals, we looked at symmetry 2.
Symmetry 2 over m is equal to 2 with an inversion center put on it.
But inversion doesn't require anything, so the symmetry constraints for 2 over m have to be the same as for symmetry 2.
We look at the constraints that might be imposed by a mirror plane.
A mirror plane plus inversion is 2 over m. 2 over m has to be the same as 2.
So this will be the same as 2.
And now we've shown that for any monoclinic crystal, regardless of whether the symmetry, the point group, is 2m or 2 over m, the form of the tensor has to be exactly the same.
So for any monoclinic crystal, namely 2m or 2 over m, this is the form of the tensor where the twofold axis, again, is along x3, the mirror plane would have to be perpendicular to x3, and for 2 over m, both of the preceding conditions.
So how about that?
Let me issue a caveat, because we're almost out of time.
There are five independent elements.
And that's true.
But elements sigma12, sigma21, sigma13, and sigma31 are 0 only for this arrangement of axes relative to the symmetry elements.
If you wanted to take a different set of axes, you know how to get the tensor for that set of axes.
Each tensor element is going to be given by a linear combination of these five non-zero elements.
And if the orientation of the axes relative to the symmetry elements is quite general, all nine elements of the tensor will be non-zero.
There will be only five independent numbers, which composes each of those nine elements, and they will be given by a product of two direction cosines, sometimes each of these five non-zero elements.
But there will be no zeros in this array at all for an arbitrary set of coordinate systems.
Something that I think I'll ask you to do as a problem, because it's really easy to do, if the tensor is symmetric, which most of them are, one thing that you can show quite directly is that a symmetric tensor remains symmetric for any arbitrary change of axes.
And that, again, is something that's fairly easy to prove, and I'll let you have the fun and exhilaration of doing that for yourself.
OK, so this means that for everything except thermal electricity, you really have to transform, at most, only six elements if you go from one coordinate system to another.
That's still a lot, but it's considerably better than transforming all nine.
So if the tensor originally is symmetric in one coordinate system, it stays symmetric in any other coordinate system.
Now, one thing that I should mention-- I passed over it rather quickly-- we said that a property of a direction cosine scheme is that it is what's called a unitary transformation.
And it has the property that the determinant of the coefficients is plus 1 if the axis retains the same chirality.
The determinant is minus 1 if you change the handedness.
That is only true for what is called a measure-preserving transformation.
That's what it's called.
And when it's a measure-preserving transformation, then the direction cosine scheme is a unitary matrix.
What is measure-preserving transformation?
If it's a right-handed system beforehand, there's no squishing.
It doesn't go to an oblique coordinate system.
Cartesian stays Cartesian.
If the reference axes are of equal lengths, they don't stretch upon the transformation.
You can define transformations like that if you like, where the angles between them go from orthogonal to oblique after the transformation, and the units of length along the three axes change dimension.
But then the determinant of the coefficients is not unity, and a lot of the nice, convenient properties that we've seen here do not hold.
All right, that is a good place to stop, I think.
Next week, no quiz.
If you came in late, we're going to postpone the quiz for a week.
And let's see, look at all I can ask you just after one lecture on tensors.
What we will do next is explore the form of the tensors for other crystal symmetries.
It goes fairly quickly.
And then having done all that, I'll show you how you can determine the symmetry constraints by inspection for a tensor of any rank.
And you're going to despise me for that, but this was useful, because we can get used to manipulating the notation.
But there is a method called-- appropriately enough-- the method of direct inspection where you can very quickly and very easily do the symmetry transformations.
So all this and more will be revealed next time, which is going to be a lot more beneficial than taking a quiz.
Do them individually so I can continue to put names and faces together.
I'm happy to announce that the registrar has now got everybody's photograph online for registration in the course.
So anonymity is a thing of the past, so you have to watch your step from now on.
I handed back, to those who didn't get it, problem set number four, which asked you to tackle some patterns, nontrivial patterns.
And actually, that was a dirty trick, because we hadn't, at that point, derived the plane groups, and you really didn't know what to do or what to look for.
But nevertheless, it got you thinking about patterns and some of the symmetry elements which we had discussed up to that point.
At this point we have derived exhaustively every last one of the 17 plane groups.
So now you are armed with this new-found power, and when faced with a pattern, you should know exactly what to look for and how to go about deciding what plane group it is.
At the very least, you'll have the drawings of the arrangement of symmetry elements in the plane groups before you, and you can work by the process of elimination.
For example, high symmetry usually hits you right between the eyes, and if something is square-ish, you can pretty quickly guess that it's based on a square lattice.
And if it has a square lattice, there jolly well better be a 4-fold axis in there that makes it square.
If you can find the 4-fold axis, then you have to ask yourself only three questions.
So 4-fold axis, fine.
Is there a mirror line in there?
Yeah.
Does the mirror line go through the 4-fold axis?
Then it is P 4 MM.
And you know just where to look for everything else, including these very subtle glide planes that are hard to spot.
If there is a mirror plane there, but it doesn't go through the 4-fold axis, then it's P 4 MG. And if there is no mirror plane, then it's P 4.
So just by asking one or two simple questions, you can narrow it down to what the plane group has to be.
This is another indication that the informed intellect is always more than a match for sheer, raw native intelligence.
If you know what to look for, it's a lot easier.
Because you really didn't have much practice with patterns, we're having a quiz, as you know, next Thursday, that will cover up through completion of the plane groups and not the material we've been doing now.
So I think it might be of use to you to have some practice analyzing a few more patterns.
So there are four additional patterns in this problem set.
As always, it's optional, but if you would like to try them, and you want to see if you've got them right, come in and see me tomorrow or on Thursday morning.
I'd be happy to go over them with you on the spot.
So this is for practice.
And some of you did extraordinarily well on the first try.
Others, I think, could use the additional practice.
There were a few people who identified the plane group correctly, but got the name that's assigned to it wrong.
And one other very confusing thing is that there is one plane group that has the symbols G and M, and another plane group which has the symbols M and G. So if you found a mirror plane and a glide plane is an independent symmetry plane, when is it MG, and when is it GM?
I have a little mnemonic device.
GM, general manager, is the guy who sits on top of the organization.
So GM should be the plane group that has the highest symmetry.
P 4 GM, P 4 general manager.
Now that is-- hey, it works for me.
But another way of saying it is that there are two plane groups, one has MG and the other has GM, and I like cars, and I think an MG is much classier than anything that General Motors, GM, puts out, so MG should be the one of highest quality, highest symmetry.
And that's just the reverse.
But whatever works for you.
You could keep them straight through that simple algorithm.
And as they say, if it works for me, but if it doesn't work for you, don't use it.
All right.
What I will bring in during intermission, for those of you had trouble identifying translations in the patterns that we handed out earlier, I've taken these and put them on overhead transparencies.
And I'll have two of each.
So if you don't see the symmetry or translations that are present, you can actually take one pattern and physically move it and lay it on top of the other one, and that's a good way to convince yourself what a translation looks like when it occurs in a pattern.
So I'll bring those in at our break between class.
All right, any questions before we move on?
Any questions that have arisen as you have gotten ready for the quiz?
You haven't gotten ready for the quiz yet, so there are no questions.
That's OK.
I know how things work at MIT.
You deal with one crisis at a time.
Any questions?
Anything you want to go over?
There was one interesting wrinkle in a problem that I had not encountered before, and this was the one that asked you to look at, in two dimensions, a plane with indices h and k. And then, when h and k were mutually prime, to move that plane by the translations plus T1 and plus and minus T2, and then show that the number of intervals between the origin and the intercept plane, the one that hit lattice points on both translations, was equal to h times k if they were mutually prime.
That is true only if the lattice is primitive.
And what the problem said was to pick one of the cells that you used in the first problem, number one.
Well, that problem asked you to begin with identifying different primitive cells.
If you take a multiple cell, this operation of going plus and minus T1 does not put a lattice line through each of the lattice points.
If you picked a double cell, that process decorated only half of the lattice points with planes, and the other half sat there with nothing hanging on them at all.
The key to the difference, if you looked at a double cell, was that if h plus k was even, then you automatically got a plane on every lattice point.
If h plus k was odd, as it would have been for the plane 2,1 for example, 2 plus 1 is 3-- hey, this isn't even one of my good days-- then half of the lattice points did not have planes hanging on them.
Now, there's great relevance of this observation to diffraction, and you probably are all familiar, if only vaguely, with the magic rules that say, if h plus k is equal to 3 pi plus 4, then the intensity is identically 0.
Well, for a double cell, the lattice planes that are repeated by translation diffract x-rays, and there is no reason why the intensity should be something other than 0.
So here comes, a la Bragg, an x-ray beam coming in at angle theta, and then you say you get diffraction when scattering from this lattice plane is exactly in phase with this one.
And this gives the familiar relation that an integral number of wavelengths is equal to 2 d sine of theta when the crystal diffracts.
So there is exactly-- this says there's exactly 2 pi phase difference or n lambda path difference between these two planes.
Now, if the lattice would be a double cell, then there is an additional lattice point in here that does not get a plane hung on it.
So if the lattice is a double cell, there's another plane that has to hang on this lattice point, and that one is exactly out of phase with this plane, and the intensity is 0.
So this observation that a non-primitive lattice has a interplanar spacing that is a sub-multiple of that of a primitive lattice gives some insight into why certain reflections-- certain diffraction maxima-- are identically 0 in intensity for a crystal that has a non-primitive lattice.
So that is something I had not noticed before.
I should have, but I will phrase the problem a little bit more precisely in the future.
All right.
So to conclude my preamble, I hope you'll try playing with some of the four additional patterns that I handed out, just to give yourself some practice.
And the implication of this is that you're going to see a pattern on the quiz, and I will tell you that you will.
So if you want to see how you did on the patterns that I distributed, please come in and talk to me about them.
All right then.
Let me remind you where we were last time.
We started to begin to build a framework of symmetry elements in three dimensions.
And we asked the question, what would happen if we take a first rotation axis, A alpha, combine it with a second rotation axis, B beta, in such a way that they intersect at a point.
This means that their operation and reproducing atoms or motifs is going to leave at least one point in space unchanged, and that will be the point of intersection.
We ask ourselves, what will be the combined effect-- we have two operations in space-- what would be the combined effect of rotating alpha degrees about A followed immediately by beta degrees about B.
So what we're going to do then is to take a first motif-- and let's say it's left handed.
Being a left-handed person, I like to give right handed motifs and left handed motifs equal time.
If we rotate that through an angle alpha, and this is number 2, it will stay left handed.
Then if we rotate that by B beta, it'll move it over here to number 3 and it will stay left handed as well.
And the question is then, what net operation is equivalent to the combined operation of these two transformations?
And to specify the type of operation is really a no-brainer.
All of these motifs are of the same corality so the only thing that can relate them is translation or another rotation.
And clearly the first and the third have no reason to be parallel to one another, and the distance between them is going to depend on how far they are away from the axis, so translation won't do the job, a and the only thing that's left as a net operation that's equivalent to those two steps is rotation about a third axis C. And what we're going to do today is answer the question, where is axis C located, and what is the angle of rotation, given the value of alpha and beta and the angle between these two axes?
And let's define that as a lowercase c. So clearly the location of the axis and the amount of the rotation is going to be a function of alpha, beta, and the angle between them.
We want this to be a combination of operations that exists in a symmetry operation.
And if this is to be a crystallographic symmetry, these will be restricted to the angular throws of a 1-fold, 2-fold, 3-fold, 4-fold, a 6-fold axis.
We can take these two at a time, ask what the net effect is if we combine at a given angle c. And what comes out here must be a rotation which is also crystallographic.
So there are going to be severe constraints on this combination.
Two rotations about an intersecting point will always be a third rotation, but if this is to be a set of operations in a symmetry group, the result must be crystallographic.
That's a tough problem, and how will we undertake it is going to be non-intuitive.
OK, the problem is most readily treated with spherical trigonometry.
So on the surface of a sphere, I'm going to map the point at which A alpha protrudes-- and I'll call this point A-- and then I'll mark out the point where axis B beta exits the sphere, and I'll mark that point B.
And this is the angle C. And we said that in spherical trigonometry, the measure of the length of the arc separating A and B is given by the angle subtended at the center, so the length of this distance between A and B is the angle c. Again, it sort of boggles the mind when you measure lengths in terms of degrees rather than some metric unit.
All right.
So I will now not bother to show the sphere on which the geometry is taking place.
I'll just draw A and B, and this is the arc between them, c. And somewhere or other there will be some third axis, C, which is going to be the combined effect of the rotation about axis A and axis B.
So what I would like to do is to locate the position of this axis C. In order to do that, I'll have to know what the angle between A and C is, and I'll call that, by analogy to what I've done here, I'll call that b.
And I'll want to know what the angle between B and C is, and I'll call that angle a.
So again, going back to three dimensions momentarily, if this is the rotation operation C gamma, and this is A alpha, and this is B beta, the axis c is this, the angle b is this, and the angle a is this.
OK.
It's a non-trivial problem and it is not by accident that the solution to this problem was first given by a very, very famous mathematician, Leonhard Euler, and this construction that we're about to go through is called Euler's construction.
All right.
Let me find where these different locations are going to be.
We've specified the location of point A and the location of point B, and we know that the angle between them is the length of the arc ab, which is the angle between A and C. So let me now do some constructions.
Let me find a great circle that by design is alpha over 2 away from the arc ab, and that is by construction.
And I'm going to say, then, that if A alpha works in this direction, the operation of A alpha is going to take this great circle and move it over to a great circle which is alpha over 2 on the other side of the arc ab.
Fine, you say, so what?
Well, just going to leave those there for now.
I'll have B beta work in the same sense.
And I'm now going to create a line here that by construction is beta over 2 on one side of the arc ab, and if I let B beta go to work, that will map this great circle over to a new location beta over 2 on the other side.
What has this done for me, other than perhaps confuse me and clutter the diagram?
Well, now I'm going to determine unequivocally the location of the axis C, and where it emerges from the reference here.
And how will I do that?
I'm going to use a definition that may have seemed trivial the first time we made the observation.
I said that a symmetry element is the locus of points that is left unmoved by an operation.
OK?
I rotated by A alpha from here to here, that took everything along this line and mapped it to a new location here.
I took this line and rotated it by B beta, and that took everything along this line and moved it to a new location.
So my question now is if I rotate by A alpha and then rotate in the same direction by B beta, what point is left unmoved?
It's only one point that can make that claim, and that is where these two great circles intersect.
The rotation A alpha will take this location-- and I'm going to call it C because I've identified now what it is-- it's going to take C and move it over to here, call that C prime, and then B beta takes that point and only that point, and restores it back to its original location.
So this, then, ladies and gentlemen, is where the rotation axis C gamma pokes out of the sphere of reflection.
Still don't know what this angle is in here, and I would dearly love to know what these arcs b and a are, and then I will have specified all three of the interaxial angles between A, B and C. OK, let me do something quite similar to what I did before.
I'm going to again let A alpha work on a particular point, and then let B beta map it.
So here's A alpha, here's B beta.
And now I'm going to look specifically at how these two rotations transform point A, where A is the point at which axis A alpha pokes out of the sphere.
A alpha, when it acts on this point, does nothing to it.
It leaves it alone.
B beta is going to map A to a new location, A prime.
Now, doing A alpha and following up by B beta is supposed to be equal to the rotation C gamma.
So that says that this point and this point must be related by the rotation gamma.
So say that again.
We're doing exactly what we did here except we're starting with an initial point, not this arc, but we're starting with the specific point A, operate on it by A alpha, it twirled around but stays put.
Rotate that by B beta, it goes through a total angle beta to this location here.
The net effect of getting from A to A prime is supposed to be the rotation C gamma, so this angle is then gamma and this is the location of C. OK?
OK, one other step that's a fairly easy one.
This length is equal to this length, because they were produced by rotation.
This side is common to these two triangles, and this angle then is beta over 2, this is beta over 2.
And if these two triangles, A, B, and C, that triangle is similar to A prime BC, and therefore I can say that angle A prime CA is identical to ACB, so therefore this angle has to equal this angle, and if the total angle is gamma, this is gamma over 2, and this is gamma over 2.
So now let me extract from this the information that I would like to use.
Here are three axes, A alpha, B beta, and C gamma.
This angle in here is gamma over 2.
This angle in here is beta over 2, and this angle in here is alpha over 2.
And let me emphasize that in this magic triangle, out of which we're going to extract some dazzlingly profound stuff, it is half the angular throw of the rotation axes that appear in here as these spherical angles, and not the entire angle of rotation.
So here's how properties of the three rotation axes are related one to another.
And now, we introduced without proof last time something called the law of cosines in spherical trigonometry.
And I not only do not want to prove it, but I have no idea how I would go about doing so, but that doesn't prevent me from using it.
So if here are three edges, a, b, c, and three angles in there, A, B, and C, we said that the law of cosines in spherical trigonometry, analogous in a way to the law of cosines and plane geometry, except since the lengths of the triangles are measured in degrees, there are trigonometric functions of these angles that appear in the law of cosines.
This says that cosine of b cosine of c plus sine b sine of c times cosine of a is equal to cosine of a.
So this now is an interesting relation that we can apply to this spherical triangle, which connects together the three rotation axes.
Let me apply it to find the angle c which we have picked as the angle between the initial two axes a and b.
That says that this should be equal to cosine of b cosine of c, the angle between the other two axes, plus sine of b sine of c times the cosine of angle a, and angle a is cosine of alpha over 2.
So all these quantities that we'd like to determine are hooked together by the law of cosines.
And this is a lovely relation, but it doesn't do us a bit of good, because in this relation we know only one quantity, and that is the rotation angle of a.
We can pick the angle between a and b, that's this, but I have no idea what these other angles are.
That's what I'd like to find out.
I'd like to find out the angles at which three rotations have to be combined in order that rotation about one followed by rotation about the second be the third.
So this equation is a beautiful equation, but it involves everything that I don't know and only one quantity that I do know.
So it looks as though we're up the creek.
Yes, sir?
So you're looking for cosine c, so shouldn't it be cosine b cosine a?
Oh, I'm sorry.
Yeah, I did that wrong.
Yeah.
You're absolutely right.
This should be cosine of a, and this a goes with this alpha over 2.
Absolutely.
Sorry about that.
So anyway, the point still stands that what this equation involves is the three interaxial angles, and I would like to know how I could combine a and b to get it to come out to a crystallographic rotation c, and where that location is relative to the first two axes.
So it involves everything I don't know, and only one thing that I do.
But now we introduce another curious aspect of spherical triangles, which I mentioned last time.
You may have thought that that's interesting, but who cares?
Here are the three points, A, B, and C, and these are the three arcs little c, little a and little b.
And then we said we could construct something called the polar triangle of ABC.
And what we would do, we would find the pole of arc b, and that will be some point B prime.
We'll find the pole of arc a, and that will be some point A prime.
We'll find, similarly, the pole of arc c, and that will be some point C prime.
And now we can connect together A prime, B prime, and C prime, and get something that's called the polar triangle.
And now comes the useful part.
We said that a curious property of the polar triangle is that the side of the polar triangle plus the angle opposite it add up to 180 degrees.
In my original triangle, this is beta over 2, this is gamma over 2, and this is alpha over 2.
So the length of this side is going to be 180 degrees minus beta over 2, the length of this side is going to be 180 degrees minus alpha over 2, and the length of this side is going to be 180 degrees minus gamma over 2.
And now let's use these angles and these lengths in the law of cosines, and I'll leave out the little bit of intervening algebra.
And what we will get out of this is that cosine of c-- and I'll solve for that-- is equal to cosine of alpha over 2 cosine of beta over 2 plus cosine of gamma over 2 divided by sine alpha over 2 sine beta over 2.
And that is something we can sink our teeth into and run with, because now I can ask the question, suppose I want a to be a 2-fold rotation axis, b to be a 3-fold rotation axis, and c be a 4-fold rotation axis?
Then the value of alpha over 2 is half of 180 degrees or 90.
Well, you can see I put in half the value of the rotation axes.
And then I solve for c, and that is the angle at which I have to put axis a and b together to get c to turn out to be whatever angle gamma over 2 is.
So I can do this systematically now without thinking.
And I can set up the problem by taking the crystallographic rotation axes and combining them together three at a time in all possible combinations.
Right?
In addition to this relation, I have two other relations.
And let me assemble them off to the left, because we have to solve three equations to find out the nature of the combination that is required.
So just permuting terms, the angle between A and B, c, has to follow from cosine of c equals cosine of alpha over 2 cosine of beta over 2 plus cosine of gamma over 2.
Notice that the single term by itself is the cosine of half the angle of the opposite rotation axis c. Then in the denominator is the sine of these two angles.
And so just permuting terms, one can see that cosine of b is going to turn out to be cosine of alpha over 2 cosine of gamma over 2 plus cosine of theta over 2 divided by sine of alpha over 2 sine of gamma over 2.
And a third analogous expression will give me the angle that will be the one between B and C. And this will be cosine of beta over 2 cosine gamma over 2 plus cosine of alpha over 2 divided by sine beta over 2 sine gamma over 2.
OK?
So now we don't have to think anymore.
It's just plug and chug.
And I'll pause to suck in air and let you catch up, and then we'll set up the problem and look at a few solutions.
And all this, in the event that you're thoroughly bewildered, is in the set of notes that I handed out last time.
So you can read it over at your leisure
Will this stuff be on the quiz?
No.
Quiz will go up to the end of the two-dimensional plane groups and stop.
We won't say anything three dimensional.
OK, let's, then, if there's no objection or complaint, look at possible values for-- let me do it the same way that I did it in the notes so that it's consistent-- let's put down the value for axis b, the rank of axis b and the rank of axis a.
And A could be a 1-fold axis, B could be a 1-fold access, and we could take a 1 with a 1 with a 1, a 1 with a 1 with a 2, a 1 with a 1 with a 3, a 1 with a 1 with a 4, and a 1 with a 1 with a 6.
This is clearly impossible.
If I did nothing, and followed it by doing nothing, and wanted it to come out to be a 6-fold rotation, you'd all be spinning on your axes like tops right now.
So you can't do nothing and follow it by doing nothing and have it come out to be a net rotation.
So these are impossible.
So we don't have to consider those.
A could be a 2, though, and I don't want to do 2, 1, 1, because I've got a 1, 1, 2 here.
The order doesn't make any difference.
So I'll start with a 2, 1, 2, a 2, 1, 3, a 2, 1, 4, and a 2, 1, 6.
So those are four combinations that I should be examining.
I could look at a 3 with a-- 2 with a 1, I have here in the form of 2, 1, 3, so the next one I would want to look at is a 3, 1, 3, a 3, 1, 4, and a 3, 1, 6.
And let me put in a couple more here.
If B were a 2, I should look at a 2 with a 2 with a 2, a 2 with a 2 with a 3, a 2 with a 2 with a 4, 2 with a 2 with a 6.
And then A 3 with a 2-- and I've got 3, 2, 2 up here, so I'll start with 3, 2, 3, 3, 2, 4, 3, 2, 6, run out of room here, but there should be a similar entry with a 4 and a 6.
So this sets up the problem.
If you count up the number of ways one can do this, we only have to consider the off-diagonal boxes here, because interchanging a and b, for example, looking at 3, 1, 3, that's going to be the same as 1, 3, 3 up here.
So it's just the off-diagonal boxes that we have to consider.
So there should be a 3, 3, 3 in here, a 3, 3, 4, and a 3, 3, 6.
So what we would have to do in order to determine the unique combinations is to look at all of these combinations in turn, and I'm going to not try all of them.
I will do one that's going to be clearly impossible.
So let's look at 2, 1, 4.
So here A corresponds to a 2-fold axis, B corresponds to a 1-fold axis, and C would correspond to a 4-fold axis.
So could we combine these three axes at appropriate angles such that a 2-fold followed by a 1-fold is equivalent to a 4-fold?
This clearly isn't going to work.
If I do a 180-degree rotation then don't do anything and ask is that equivalent to a 4-fold rotation, that is saying that the 2-fold axis should be identical to the 4-fold axis, and that is not going to work.
So let me do now a generic family that I know turns out to be possible.
Let me look at an n-fold axis with a 2-fold axis with a 2-fold axis, and I can show that this combination is possible for any integer n whatsoever.
So this will include a lot of non-crystallographic symmetries.
So let's say that this is C, this is A, and this is B.
But make it C, B, A if you'd like.
OK, so my first equation says that cosine of c, the angle between A and B, should be equal to the cosine of alpha over 2 times the cosine of beta over 2 plus the cosine of gamma over 2 divided by sine of alpha over 2 sine of beta over 2.
B is a 2-fold axis, so alpha is equal to 180 degrees.
A is a 2-fold-- I'm sorry.
B beta, A alpha.
Beta is equal to 180 degrees, the angular throw of a 2-fold axis.
Alpha is equal to 180 degrees, the angular throw of a 2-fold axis.
And gamma is equal to whatever 2 pi over n would be.
That's the throw of the n-fold axis than I'm letting be equal to C. So the cosine of A and B, to get the result of rotation A followed by rotation B being equal to the net rotation of an n-fold axis is that the cosine of c should be the cosine of 180 degrees over 2 times the cosine of 180 over 2 plus the cosine of gamma over 2, and gamma is whatever the rank of the axis determines, and that's divided by sine of alpha over 2, and alpha is 180 and sine beta over 2, and that's 180 over 2.
So the cosine of c is going to be the cosine of 90, which is 0, times the cosine of 90, which is 0, plus the cosine of gamma over 2, whatever that might be, over the sine of 90 which is 1, sine of 90 which is 1.
So this says that cosine of c is equal to cosine of gamma over 2.
So the angle between axis A and axis B ought to be equal to one half the angular throw of rotation axis C. So let's start putting down some of this information.
This says that if this is axis C gamma, then A pi and B pi, the 2 180-degree rotations, should be at an angle gamma over 2.
We still need values for b, and we still need a value for a.
So let's find out what those are.
And let me start over here at the left again, because I've got the relation that I need for b sitting here.
The cosine of b is equal to the cosine of alpha over 2, and that is the cosine of pi over 2, plus alpha is a 90-degree rotation.
Then cosine of gamma over 2, whatever gamma happens to be, plus the cosine of beta over 2, and that's cosine of pi over 2, and this is all over sine of pi over 2 times the sine of gamma over 2.
So this is going to be cosine of pi over 2, which is 0, plus cosine of pi over 2 divided by sine of pi over 2, which is 1.
This gamma over 2-- no, cosine of pi over 2 is 0.
So this is 0 plus 0 times sine of gamma over 2, whatever that turns out to be.
So cosine of b is 0, and that says that the angle b between axis A and axis C turns out to be 90 degrees.
So in order to get pi followed by c gamma to be equal to b pi, I've got to make b be equal to pi over 2.
And if I put it in this orientation, then a pi followed by c gamma is going to be equal to b pi.
One final angle, and that's the value for a. OK, cosine of a should be equal to the cosine of beta over 2, that's pi over 2 times the cosine of gamma over 2, whatever it is, plus the cosine of beta over 2, and that's cosine of pi over 2, over sine pi over 2 sine of gamma over 2.
And this, as for b, turns out to be 0 plus 0 over sine pi over 2, which is 1 times sine of gamma over 2.
So cosine of a turns out to be 0, and this says that the angle a is also pi over 2.
So we've got a whole-- Yeah
Did you really need to go through all three equations?
Yeah, because I had to show that all three work.
And in general, if I combine, let's say, a 4-fold with a 3-fold with a 2-fold, which is something I want to do, all three angles a, b, and c, will be different.
OK?
So the answer is yes.
And there will be a few cases where a value for one angle will exist and the value for the one or two others will be impossible.
And that's also something that I have to know.
So repetitious as the exercise might be, the answer is yeah, you do have to do all three
[INAUDIBLE]?
Oh, you don't have to do different permutations.
That's just a question of labeling.
OK?
So n with a 2 with a 2 is the same as a 2 with an n with a 2 is the same as a 2 with a 2 with an n, that's just labeling.
So that's why my boxes, when I filled them out, the list got shorter and shorter, until finally for a 6-fold axis, it would be just 6, 1, 1, 6, 1, 2, 6, 1, 3, and so on.
Just that one entry in the box where I enumerated what should be considered.
Well, here is a whole slew of possible solutions and a lot of them are non-crystallographic, but still possible.
This says that a combination of a 2-fold axis-- but remember now that these are equations and operations.
But the only operation that's present for a 2-fold axis is a rotation a pi, b pi or c pi.
So I could combine three 2-fold axes that are mutually orthogonal, and that is an allowable combination.
And what we are obtaining here is a sort of scaffolding, a framework, based on pure rotation operations, that by themselves will be an allowable 3-dimensional point group, but which also provides a framework, a Christmas tree, that we can decorate with mirror planes and inversion centers to get still additional groups of higher symmetry.
So here's one possible crystallographic o combination of rotation axes.
What we will use to denote this combination is the same rule that we use for our other notation.
We will make a running list of the independent operations that are present, and what we have combined here are three distinct independent 2-fold axes.
A solid that would have this symmetry plus some other symmetry would be an orthogonal brick with one 2-fold axis coming out here.
This is the operation c pi, another 2-fold axis coming out the front, and this would be the operation a pi, and another 2-fold axis coming out of this face, and this would be the operation b pi.
Now let me show you-- it's rather amusing-- that what we have done really works.
We've shown supposedly that a pi followed by b pi should be equal to a net rotation c pi about an axis that's orthogonal to the first two.
So let's pick a motif, and for convenience I'll put it at one corner of this brick.
Here's object 1, I rotate it by 180 degrees about a.
Here sits object number 2, same corality, and then I rotate it 180 degrees about B beta, and that's going to give me number 3.
What is the net way of getting from 1 to 3?
Holy mackerel.
It's a net 180-degree rotation about c pi.
It really works.
Or I could do the operations in a different order.
I could rotate by d, rotate by c, and the way I get from the first to the third is a rotation a pi.
So that is a self consistent set of rotation axes.
That is 2, 2, 2.
Let me do one more.
Well, no, let me take a break here and let you absorb all this, and then we'll look at some remaining ones, and this will include some that are non- crystallographic.
And that's perfectly OK.
But they're lovely groups, they constitute groups, but they won't be groups that can occur in combination with a lattice.
Any questions about where we left off-- up to where we left off?
OK, what I'll do then is give you a few more examples of the combinations in 22 to show which ones we have to retain as frameworks for crystallographic point groups and which ones exist as groups but which involve rotational symmetries that are not permitted to a lattice.
So we've seen a combination of three orthogonal twofold axes and then projection that would look like this.
And the international symbol for that point group is just a running list of the different axes that are present, 222.
The next group in the sequence would be 3 2 2, where we took a 120 degree rotation.
We combine that with a twofold axis perpendicular to it and the new twofold axis comes out and reminds you again of things that are quite clear but which are easy to forget-- that this angle here is 1/2 of 2 pi over 3.
Don't forget that 1/2.
So the neighboring twofold axis is 60 degrees away and then if we allow these axes to operate on each other, the net symmetry consistent set of axes looks like this.
But let us look at a solid that has this symmetry.
And such a solid would be a trigonal-- triangular prism.
And again we can use the corners of this polyhedron as the reference locations of our motifs.
So let us put a first motif here, number 1.
Let's rotate it by 120 degrees to get a second motif here.
And then let us rotate that one down by a twofold axis coming out of one of the edges.
That will give us number 3 that is down here or of the same chirality.
And how do we get from 1 to 3 directly in one shot?
And the answer is about a twofold axis that comes out of the face of the prism.
So again the prism that we've used as our reference is a prism that looks like this.
We've got one twofold axis coming out of the face.
The next twofold axis is the one that is equivalent to a threefold rotation followed by a twofold rotation.
Now here we hit a situation in deciding on the name for the combination that is analogous to what we found in two-dimensional plane groups for 3mm.
We saw that only one mirror plane was distinct.
We look at this arrangement of axes-- this was axis a, alpha, this was axis b, beta.
But if I repeat one of these twofold axes by 120 degree rotations, this one is the opposite end of this one, this one is the opposite end of this one, and this one is the opposite end of this one.
So there are only three kinds of-- there are three, twofold axes and they are related by the 120 degree rotation.
Another way of saying that is that if we look at a trigonal prism, each of the twofold axes comes out of an edge and out of the opposite face.
So there they all do the same thing in this regular prism-- twofold axes extend between corners of the opposite face.
So there are only-- there's only one kind of twofold axis present in terms of being symmetry independent.
So just as we called 3mm, 3m, and we call this one 3 2 because all the twofold axes are symmetry equivalent.
The next one that is crystallographic would be a combination of a 90 degree rotation with a pair of twofold axes that are normal to it and separated by 1/2 of pi over 2, the [?
throw ?]
of a fourfold axis.
And the symbol for this one would be a fourfold and now there are two different kinds of twofold axes.
And if we look at a regular square prism, we can again show that what we've been demonstrating for these other prisms is true.
One twofold axis would come out of the face, the other twofold axis would come out of the midpoint of an edge, and the fourfold axis would come up here.
And a 90 degree rotation, from here to here, followed by a twofold rotation about this axis, gives us as a net effect a 1, 2, 3.
And that rotation from 1 to 3 about this twofold axis gives us the combined mappings.
Notice that the order in which we do them is unimportant.
For example we could do the same thing but do two 180 degree rotations about the twofold axes.
Let's say we start by doing a rotation about this twofold axis.
From here down to here.
And then we do a twofold axis about the-- twofold axis that comes out the face and that would take number 2, and rotate it up to number 3.
And the way we get from 1 to 3 directly is by a rotation of C pi over 2 about the square face of the prism.
So do them in any order you like-- two 180 degree rotations, or a 90 degree rotation, one of the two full rotations with 90 is the other twofold-- the other type of twofold axis.
The 90 degree rotation plus the second type of the twofold axis is the same as the first.
So the result can be permuted and turns out to be the same combination.
The next one that we would hit if we proceed systematically is non-crystallographic.
And this would be a fivefold axis.
And I'm foolhardy to even start trying to sketch this in three dimensions.
Nevertheless, nothing ventured, nothing gained.
Start with a twofold axis out of one of the edges and rotate from 1 down to 2.
Follow that by a twofold rotation about the twofold axis that comes out of the face.
And that gives us one number 3 up here, and lo and behold, the way you get from 1 to 3 directly is by a rotation through 1/5 of 2 pi.
So this would be the non-crystallographic point group, 5- well it's not going to be 5 2 2.
And that has a whole bunch of twofold axes separated by 1/10 of 2 pi.
And just as in 3 2, twofold axes here all come out of the face and out of the opposite edge.
So this would be called 522.
A nice symmetry but not crystallographic, so we can promptly forget about.
So what comes out of this is a family of groups that are all of the form n 2 2.
The crystallographic ones are 222, 32, 422 which we've looked at in detail, and one that I won't draw because there's so much symmetry it get's messy.
This is 622.
There is a Schoenflies notation.
You remember the language that we encountered for our two dimensional point groups.
We used m for mirror in the international notation.
We used C standing for cyclic group, subscript s standing for Spiegel in the Schoenflies notation.
The Schoenflies notation for all of this family of symmetry is Dn, and the D stands for dihedral.
And the reason for that name is that the difference between all of these groups, besides the n-fold axis, is this angle between adjacent twofolds and this is a dihedral angle.
I have a set of planes passing through a common axis; the angle between those planes is termed a dihedral angle.
So this is called D for dihedral and then a subscript that gives the rank of the axis.
So Dn generically.
This is D2, this is D3, this is D4, and this is D6.
Comments or debate?
Yes, sir
[INAUDIBLE]
[INAUDIBLE] --out of this face.
So I got from here down to the diametrically opposed axis.
And I rotate it about the adjacent one which comes out of an edge and-- what did I do here.
Here I did A pi over 2 from 1 to 2, and then I did B pi, where this is B pi, and they turned out to be C pi, which is this one here.
So going from here to here down to number 3 is the same as going from 1 to 3 in one shot about a twofold axis normal to the face.
And actually I'm courageous to try to do this in three dimensions.
We could do it in projection and then things are used-- this for a point that's up, and use this for a point that's down.
And then what we've done is to go from 1 that's up, to 2 that's up, and then we rotate it about this twofold axis.
That was 3, that's down.
So when we get into complicated symmetries where you just can't do a proper job drawing them in an orthographic drawing, we'll do them in projection and use a solid dot for something that's up and an open circle for something that's down.
Is there any other way we can combine things?
Well what you would have to do is use these three relations, and plug and chug your way through all of the combinations which were enumerated in the handout.
And I'll save ourselves a lot of work by saying that there are only two more combinations.
And these are combinations of axes at angles that have relevance to directions in a cube.
One of them is a twofold axis with a threefold axis with a threefold rotation.
Again remember these are not equations in symmetry elements.
This is really A pi, combined with B 2 pi over 3, combined with C 2 pi over 3.
And the angles that fall out of this are all crazy things like 109 point something degrees and they make no sense whatsoever.
They're not nice things like some multiples of 2 pi, 90 degrees or 120 degrees.
And they make no sense at all until you refer them to directions in a cube.
And this one, 2 3 3, consists of a twofold axis coming out of the face of the cube, a 120 degree rotation, so this is B 2 pi over 3, this is A pi.
And the other one, C2 pi over 3, corresponds to a threefold axis coming out of another body diagonal.
I don't know if I want to be gutsy enough to try to illustrate that that really works.
But one thing that we should do is to let these axes go to work on one another, and see what comes out.
First thing we can say is that this threefold axis-- if we extend it, it comes out of the bottom diagonal of the cube.
This one, if we extend it, will come out of this diagonal of the cube.
So there's a threefold here, and a threefold here.
The twofold axis gives us a threefold axis that will come down this way.
So there's a threefold axis here and a threefold axis coming out here.
And then the twofold axis will rotate this threefold axis over to the remaining pair of corners.
So we have created this by looking at a twofold rotation, combined with the threefold axis, combined with the rotation of another threefold axis.
But in point of fact, if you let the twofold axis operate on the threefold axis coming out of one body diagonal, you get threefold axes automatically out of all body diagonals.
So this is given the international symbol 2 3, because if you start with one twofold axis out of a face normal and one threefold axis out of-- along a body diagonal, the twofold axis, acting on that threefold axis gives you one along every body diagonal.
And the threefold axis-- let me point out that a cube standing up on its body diagonal, with a threefold axis coming out here and these faces sloping down into the blackboard.
If I have a twofold axis coming out of one face, the threefold axis puts a twofold axis on this face and rotates again 120 degrees and puts a twofold axis on this face.
So the threefold axis relates all twofold axes coming out normal to the cubed face.
And they were really just two independent axes in this combination.
So 2 3 is the international symbol-- one kind of twofold axis, one kind of threefold axis-- inclined at these crazy angles that are in a cube.
The Schoenflies notation for this is T, and that stands for tetrahedron.
And let me try to convince you that if I look at a tetrahedron, that that is the arrangement of pure rotation axes in a tetrahedron.
And the way to show a tetrahedron is to inscribe it in a cube.
So if I connect these two faces together and these two faces together, that will define for me a solid that has four triangular faces.
Easier to recognize it when we put it up on one face.
So this is a tetrahedron.
Schoenflies symbol is T. And I love to get to this part of the semester and be at this point at the end of the hour, because if I draw a stereographic projection of the twofold axes, in this symmetry, I can take a couple of more twofold axes and add it to this combination which destroys the group.
But lets me wish everybody a happy Halloween and exit to a stunned silence at the end of the hour.
So this is a nice point group for October.
There is one more and that is the highest symmetry of all.
And this is a combination of a rotation-- A pi over 2, a 90 degree rotation, with a rotation B 2 pi over 3, rotation through one third of the circle, and the rotation C pi.
And the directions between the axes that come out there don't come out some multiples of 2 pi.
Again they are crazy angles that make no sense at all unless you refer them to directions that occur in a cube.
The fourfold axis is in the direction that corresponds to the normal to a face.
The twofold axis corresponds to a direction out of one of the edges.
And the threefold axis corresponds to a direction that is a body diagonal.
So again this is a mess.
If we let those axes work on one another however, we will produce, more readily appreciated in a stereographic projection, fourfold axes along the directions that correspond to face normal.
So the cube, threefold axes coming out of the body diagonals and twofold axes in between all of the fourfold axes in directions that correspond to the lines from the center of the cube out through the edges.
So this is a group which would be called 4 3 2.
There's one kind of fourfold axis related to all the others by the other rotation axes that are present.
One kind of threefold axis that is related to all of the other threefold axes along the body diagonal by other rotation axes that are present.
And twofold axes, one kind, all coming out of the edges.
So this is the group that, in international tables, is called 4 3 2.
And the Schoenflies notation, this is called O.
And that is the general reaction when one sees this lovely combination.
You go ooh and O is what it's called.
But the O doesn't stand for a gasp, it stands for an octahedral.
This is the symmetry of an octahedron-- rotational symmetry of an octahedron.
So that's it.
That is the bestiary of ways in which you can combine crystallographic rotation axes in space about a fixed point of intersection.
And there are eleven of them.
There are the axes by themselves-- 1, 2, 3, 4, and 6.
There are the dihedral groups, 222, 32, 422, and 622.. And then the two cubic groups, T and O.
In the international notation, I'm mixing metaphors.
These are 23 and 432.
I call to your attention the insidious similarity of the two combinations of axes, 32 and 23.
When the 3 comes first, this is a group of the form n22.
When the 2 comes first, that is the tetrahedral group.
So if you count them all up, there are 4, 2 is 6, and 5.
There are eleven axial combinations.
Quite a few more than the situation in two dimensions where we just had single rotation axes 1, 2, 3, 4, 6.
Now we have those as in two dimensions but the dihedral groups and the two cubic arrangements of axes as well.
So we don't have to stretch our vocabulary too much more to be all inclusive here.
OK, comments?
Takes your breath away, doesn't it?
All right, let me indicate the next step in outline.
What we will next do is introduce our remaining two symmetry operations into the picture.
We have the eleven axial combinations.
As I said, we can regard these as a framework that we can decorate with mirror planes and or the inversion center, which has not appeared until this point because inversion is inherently a three dimensional transformation.
So what we're going to do is to take these axial combinations and add another symmetry operation to the group.
And this as-- we used the term earlier, this is an extender.
We have something that constitutes a group by itself then we muck things up by adding another operation.
Remember in all of this we are combining operations, not symmetry elements.
So we'll take an axial combination and add the reflection sigma, a reflection operation sigma.
Or take a rotation and combine it with the operation of inversion as an extender.
So let's itemize the sorts of extenders we should consider.
And the ground rules are that the extender should leave the arrangement of rotation axes invariant.
Because if it doesn't, we are going to create a rotation operation that does not conform to the constraints that we used in Euler's construction.
So for example, if we take 222, which contains the operations A pi, B pi, C pi and identity, that's the group 222.
If we would add to the arrangement 222 a mirror plane that snaked through some arbitrary fashion like this, that mirror plane is going to reproduce the twofold axis over to here.
And that is either going to not constitute a group because this twofold, this twofold, and this twofold don't conform to Euler's construction.
Or alternatively, if we put it in carefully at 45 degrees with this twofold axis, we're going to get twofold axes that are 45 degrees apart and that's going to change this into a fourfold axis.
So if the addition of a mirror plane does not leave the arrangement of rotation axes-- rotation operations-- invariant, we're either going to get something that's impossible and does not constitute a group.
Or else we're going to get a combination of rotation operations of higher symmetry which we've already found because we went through that process of combination using Euler's construction in an exhaustive fashion.
So the rule then is that if we add the reflection operation sigma, then the arrangement of rotation operations must be left invariant.
OK let's look at the single axes.
If there's an n-fold axis, the ways we can add a reflection operation to that axis is to pass the reflection operation through the axis.
And this is going to look very much like the two dimensional point groups of the form nmm except that rather than having a mirror line, imagine the whole works as extending upwards along the rotation axis and space.
So this extender is called a vertical mirror plane.
The other way we could add a mirror plane to an n-fold rotation axis is to put the mirror plane in an orientation that's perpendicular to the rotation axis.
That didn't exist in two dimensions because that plane that's perpendicular to the axis is the plane of our paper.
And unless we wanted to have a two-sided group that was on both the top of the paper and the bottom of the paper, and we make up the rules since it's our ball game.
And that could be a group and these would be the two-sided plane groups, a plane point groups, but we didn't do that here.
Adding the mirror plane, the reflection operation sigma, in a fashion that is normal to the rotation operation, A 2 pi over n, is another distinct combination.
And this is called, very descriptively a horizontal mirror plane.
That looks like about all you can do except for the cases where we have more than one kind of rotation axis present.
So let me use 422 as an example.
We could add a horizontal mirror plane perpendicular to the principal axis of symmetry, and that would be the horizontal sigma.
We could add a vertical mirror plane.
And now there are two ways we can do it.
We could put the mirror plane, the operation sigma, through the fourfold axis and in a fashion that was perpendicular to the twofold axis.
And we will retain the term of vertical sigma for that addition.
But the other thing that we could do would be to put the reflection operation interleaved between the twofold axes.
That's going to take this one and flip it into this one, this one flip it into this one, flip these back and forth, and that doesn't create any new reflection.
And this is referred to as a diagonal reflection plane.
Diagonal to what?
Diagonally interleaved between the twofold axes.
And these are distinct additions and they will lead to different groups.
In as far as addition of reflection operations is concerned, that's about all we can do that's distinct.
And notice that this is for the groups Dn, and tetrahedral, and octahedral only.
The distinction here is not defined for just a single axis.
So there are three possible extenders here-- a vertical mirror plane, a horizontal mirror plane, a diagonal mirror plane, added to each of the eleven arrangements of rotation axes.
And then the final extender that we could add is to add inversion.
And the symbol for the inversion operation is 1 bar.
And obviously if one point in space is going to be left invariant, you either add this on a single axis, and that is what you'd have to do for the groups Cn, or at the point of intersection.
And that would be the case if more than one axis, and that's the groups of the form n22, T, and O.
That's it.
That's the job.
So we should consider each of these possible additions of an extender systematically.
I don't propose to do every single one independently.
If we do a couple, you'll get the general idea.
And I think because I have an honest face, and you've come to trust me, I can just describe the remaining results to you and we won't grind through every single one.
Now the enormity of what I've proposed becomes apparent when I say that we now are going to have need of a number of different-- what I call combination theorems, that let us complete the group multiplication table.
And deduce, as a consequence, which symmetry operations must come into being because of these additions.
So we'll want to know what happens when you add a vertical sigma to a rotation operation, A 2 pi over n. We'll want to know what happens when you add a horizontal sigma to a rotation operation, A 2 pi over n. And we're going to want to know what happens when you add a diagonal mirror plane, this really is a special case of the vertical mirror plane.
And we'll want to know what happens when you add an inversion center to a rotation axis.
So let me do a few of these and then next time we can start off and start driving the three dimensional symmetries.
So let's just repeat the ones that we've already done.
We said that if we have a rotation operation A alpha, and we put a reflection plane through it, that A alpha followed by a reflection plane passing through it-- let me call this sigma V-- because this is the so-called vertical, up orientation.
We've already seen that in two dimensions.
This is a vertical mirror plane, sigma prime, that is going to be alpha over 2 away from the first.
So that is something that we've already seen in it's entirety in the two dimensional point groups.
There was m sigma, there was 2mm, and that was C2V, 3m, that's C3V, 4mm, and that was C6V, and 6mm, and that's C4V, and that's C6V.
So that is the result that we obtained for two dimensions.
And you can see now the reason for distinguishing the mirror plane by saying it is a vertical mirror plane because this is in the three dimensional sense.
It's vertical parallel to and passing through the rotation axis-- no longer a rotation point but a rotation axis.
So we've got those theorems.
What happens if we take a rotation operation, A pi, and put a horizontal reflection operation through it normal to that axis?
So I'll call this sigma h, a horizontal operation.
OK what we have to do is draw it out once and for all.
Here's the first one, let's say it's right-handed.
We'll rotate by A pi to get a second one which stays right-handed.
And then we'll reflect it down in the horizontal mirror plane to get a third one which is left-handed.
And now the question is, how is number one related to number three?
Anybody want to hazard a guess?
We've got to go from a right-handed one to a left-handed one.
But these two guys are oriented anti-parallel to one another.
So how do we relate the first one to the third one?
I heard somebody mumble softly enough to remain anonymous.
Inversion.
Right.
So as we go along making these combinations, if we had not been bright enough to think of the operation of inversion as a general transformation where the sense of all three coordinates is changed, we would have stumbled over headlong right here.
Combine a rotation operation, A pi, with a mirror reflection that is perpendicular to the axis.
The way you get from one to three in one shot is by inversion through a point that is at the intersection between the rotation axis and the mirror plane.
So rotation followed by rotation in a vertical mirror plane that's perpendicular to the axis is the operation of inversion at the point of intersection.
So again, if we had not been clever enough to invent it or tell you about in advance, there it is.
When we start forming the group multiplication table, we would have had to have defined this operation to describe this relation
Is that sigma sub h?
Ah yeah.
Let me write that, sigma h. I thought that V didn't look like a V, so I changed it so it looked like a V but it shouldn't be a V. That's a horizontal mirror plane.
OK this is a new combination and if we see what operations are going to be present in the group, we've got the two operations of the twofold axis, 1 and A pi.
And what we have added is sigma h as an extender.
So 1 and A pi, this little box here is the subgroup that we know and love as the twofold axis.
And then we'll write sigma h here and now let's fill in the group multiplication table.
Doing the identity operation twice is identity.
Doing the identity operation followed by A pi is A pi.
Identity followed by sigma h is sigma h. Identity followed by A pi, sigma h, lets me fill in those boxes.
Do a rotation twice, that's the identity operation.
Do a rotation of A pi and follow that by a reflection, A pi followed by reflection.
This is the inversion operation.
And so I should add inversion to my list of operations since it's come up.
So 1 followed by inversion is inversion.
A pi followed by inversion is the horizontal reflection.
Horizontal reflection followed by A pi is the same as inversion.
Horizontal reflection followed by horizontal reflection is the identity operation-- brings me back to where I started from.
And a horizontal reflection followed by inversion is the same as A pi.
So I'll have another object down here to complete the three dimensional arrangement.
And this is the group for operations-- A pi, inversion, horizontal reflection, and the identity operation.
So what do we call this one?
The operation A pi plus a horizontal reflection operation gives rise to a group that is a twofold axis of its operations, perpendicular to a mirror plane.
And this written as a fraction means that the 2 is perpendicular to the mirror plane rather than being parallel and in the plane of the mirror plane.
So two codes for writing symbols.
A 2 followed on the same line by the m means that the m is parallel to 2.
We know that when we write them as a fraction, that will be our way of designating that this mirror plane is perpendicular to a twofold axis.
OK it's the witching hour, 4 o'clock exactly.
That's when we ought to quit, so let's stop there.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available ocw.mit.edu
All right.
The quiz on Thursday will cover up through piezoelectricity.
You've had a set of notes in your hands that cover just about everything that I wanted to say about piezoelectricity.
There are other modulae that one could talk about.
But these follow quite directly from the one or two that we will do.
So we will not have anything to say about elasticity until the last lecture of the term, which is a nice outcome because you certainly don't want to take a quiz on forthright tensors.
You'll spend the entire hour just writing out all these cumbersome equations.
All right.
So the quiz will cover up through piezoelectricity, including a few of the representation surfaces that we'll examine today.
The thing that we'll be looking at is-- I'm sorry.
You have a question?
Yes.
A question.
Well, you said [INAUDIBLE] example
I'm sorry.
Third-rank tensors

Also, we had just a little bit of that going into the second quiz.
And we didn't really ask anything about that on the second quiz.
So it'd be third-rank tensors.
But you have to use second-rank tensors to define third-rank tensors.
So it really will be not the emphasis, but certainly you should be familiar with the early part of what we did with second-rank tensors.
All right.
So today we'll look at some representation surfaces to your wonder and delight at how incredibly anisotropic the variation of these third-rank properties are with direction.
One of the things that I love to do for problems when we get to these different piezoelectric effects is make up hypothetical devices just for fun.
So among those that I've invented, the first one is a earthquake sensing device because it's well known that California is going to split in half, and half will fall into the sea any day now.
So if this is the San Andreas fault, I have developed large, prismatic monoclinic crystals that I embed into the San Andreas fault at regular intervals.
The bottom of these crystals is grounded.
There's an electrode at the top.
And if the San Andreas fault starts to move, and there is shear on these crystals, there will be a charge developed on the top.
And I ask you to relate the charge on the top in terms of the piezoelectric modulae to the shear along the San Andreas fault.
So there's a very clever little device that clearly is going to be lucrative because these crystals will have to be huge in size.
And they'll cost more even than silicon.
Another device that I've invented is used down at the Boston fish pier.
This is a crystal on which I hang a pan, and you put fish in it.
That creates a tensile stress in this direction.
We measure the charge on this face.
And we put a little meter that measures the charge that's accumulated.
So this is how the fishermen can weigh their fish after they haul them in off the boat.
So you see this is really practical material we're dealing with.
This has applications in all realms of life.
So today I'd like to show you, also, another problem that sets up.
And I pass this out just so you can, again, have a look at some of the questions that you should be equipped to answer.
So here-- not fully expecting anybody to do it, but you can see the sorts of problems one might ask.
Here is problem set number 16, which asks you, should you be so inclined, to think about manipulations of third-rank tensors.
But then the second question is an idea for a device which the Center for Material Science and Engineering is considering employing here in building 13.
And you can read all about that..
This is the famous soup cell.
I think probably you will see some sort of for fun modulus for a particular device on the quiz.
I haven't made one up yet.
But I think you'll have a look at something like that.
And when we do the longitudinal piezoelectric modulus for quartz, which we'll do momentarily, this will give you an idea of how to set up these expressions.
The problem, basically, is that the direct piezoelectric effect measures a polarization in terms of all of the elements of applied stress.
So there is a modulus dijk times all of the elements of stress, sigma jk.
So this, until you use the condensation of subscripts, contains nine terms going this way and three equations going this way.
As we discussed earlier, since only six of the nine elements of stress are independent, you can condense this down into a 3 by 6 array of terms.
The problem in doing that, though, even though one has only 18 different modulae to work with, instead of 27-- that's a considerable economy in notation, and an elimination of a great deal of redundancy-- the matrix form of this relation-- and I emphasize that it is a matrix, and it's no longer a tensor-- the matrix form cannot be transformed.
Again, there are only six terms in the matrix elements of stress.
And there are three equations, again, one for each component of polarization.
So instead of having 27, one has only 18.
But if you are considering changing the reference axes-- and that is one of the things that it's interesting to do for these various modulae that one can define-- change the orientation of a particular rod-shaped specimen that you cut out of a crystal to different crystallographic orientations and then ask how the scalar modulus changes as you change the direction in which you've sliced out the wafer or the rod of material.
In order to do that, you want to transform the piezoelectric tensor to a new set of reference axes.
And you cannot transform the dij's because they are a matrix and not a tensor.
And no law of transformation is defined.
So what you have to do in any generic problem of this sort is, for a particular single crystal, look up the form of the dij matrix that will have the equalities between tensor elements written in and the 0's, those modulae which are identically 0, entered into the array.
And then you have to work your way backwards to get to the full tensor notation.
So we'll see this when we look at the modulae for symmetry three two.
You'll have to write in the exact matrix subscripts without absorbing the equalities in the notation.
And then you'll have to expand the matrix terms into full three-subscript tensor terms.
Then if you want to transform the axes, which is to say you want to cut out your specimen, be it a plate or a rod, in a different orientation, you have to transform the full three subscript tensor elements to a new setting.
And then if you want to continue to work in that setting in the compact matrix form, collapse it back down to matrix form, insert the equalities, and then you're back to where you started from.
So this problem of seeing how modulae that describe different phenomena vary with crystal symmetry, you have to go through this problem of expanding the compact form and then collapsing back down when you've got it as a function of some orientational angle or in terms of a coordinate system that you want to work in.
The problem is exactly the same for elasticity.
And we'll look at some of these modulae next term.
You've heard these names before, I'm sure-- Young's modulus, shear modulus, and so on.
We'll take a look on next Tuesday, a week from today, at Young's modulus, which is one of the more important ones.
And probably are used to seeing this in the form of information for a polycrystalline material, which is essentially isotropic.
The modulus is much more interesting for single crystal materials.
And then the surfaces that are defined particularly for the lower symmetries are absolutely wild things with lumps and wiggles and lobes and things of that sort, nothing like the dumb old, uninteresting ellipsoids that we encountered for second-rank properties.
So let's, then, take a look at how we had set up and defined the direct piezoelectric effect.
We set this up as a proper tensor relation, saying that P1 is equal to d1.
V1 And then, you'll recall, we have nine elements of strain-- sigma 11, sigma 12, sigma 13, sigma 21, sigma 22, sigma 23, sigm a 31, sigma 32, and sigma 33.
But the tensor is symmetric, so we really only need to enter into our relation six of these nine terms explicitly.
And what we did was to replace the two subscripts on the elements of strain, which, again, you need if you want to refer to those elements of stress through a different coordinate system.
You have to know the elements of stress in tensor form.
But we convert it to six terms by going and replacing the pairs of subscripts with a single one, two, and three, marching down the diagonal of the tensor this way and then marching up the right-hand side, calling two, three, four, and calling one, three, five, and finally ending up in this slot here.
And we call that six.
So that was the notation we used to get to a matrix representation.
But the place where all this started is-- and I'll write just one line of this to be merciful-- d 111 times sigma 11, so this pair of subscripts goes with this pair, plus d 122.
I'm putting that one in next because we're going to number the terms for stress in this order, one through six.
Times sigma 22 plus d 133 times sigma 33 plus d 123 times sigma 23 plus d 132 times sigma 32 plus d113 times sigma 23 plus d 132 times sigma 32.
And finally we end up in slot number six.
And we have a d 112 times sigma 12 plus a d 121 times sigma 21.
Look at that.
There's an equation that covers two whole blackboards.
So if we now condense this down to matrix form we would say that P1 is d 11 times sigma 1 plus d 12 times sigma 2 times d 13 times sigma 3 plus-- and now we have this messy problem with the 2's-- we have a d 14 times sigma 4 plus, again, a d 14 times a sigma -- 2 3 is equal to 32, so I can call this 4-- and then these terms become 15 sigma 5 and, again, a 15 times sigma 5 plus a 16 times sigma 6 plus d 16 times sigma 6.
Now we have to make a choice.
Either we are going to have, in a general matrix relation, that P sub i is equal to dij times sigma j, if j is equal to 1, 2, or 3.
But it's equal to 2 dij times sigma j if j is equal to 4, 5, or 6.
And that is something we like to avoid, if possible.
That's ugly.
That's ugly.
And it's going to be a hell of a matrix if we have factors of two in front of some of the matrix elements but not in terms of others.
So this is something we could do.
Hey, it's our ballgame.
We make up the rules.
But that's going to be an ugly thing to have to deal with.
So instead, as we mentioned last time, what we will do is to lump together these terms and define d 14 not as these individual tensor elements, but define those matrix elements as the sum of these two elements.
And then we saw before that-- we saw last time in our earlier meeting-- that from the converse piezoelectric effect, which expresses strain in terms of an applied field, where the elements of strain 1, 23, and 32 appear in separate equations.
And knowing that the same array of piezoelectric coefficients amazingly describes the converse piezoelectric effect as well as the direct piezoelectric effect, we know that d 123 equals d 132.
And that is from the converse effect.
So equivalent to saying this is to define d 14 as twice d 123 because these two elements are equal.
So we're eating the factor of two here so that we can write a nice matrix relation that doesn't involve a factor of two.
So making this combination of terms for the shear stresses, we would have simply d 14 sigma 4 plus d 15 times sigma 5 plus d 16 times sigma 6.
And we can say, in general, for the other two equations, by analogy to this one, that P sub i is dij times sigma j-- a nice, neat matrix relation but one for which, unfortunately, there's no law of transformation for the matrix modulae dij.
If you want to change to another coordinate system, we have to be prepared to resurrect this full three subscript notation on the piezoelectric modulae.
OK.
Comments or questions at this point?
All right.
If not, let me remind you that in the notes which I distributed last time, there are summarized all of the constraints imposed on the piezoelectric modulae for single crystals.
And, again, these constraints, these requirements that the tensors remain invariant for the change of axes produced by a symmetry element that the crystal possesses, these transformations show that no third-rank tensor property can exist in a crystal that has inversion.
So the 11 [INAUDIBLE] group, so-called, that possess inversion have absolutely no property and can be not considered further for third-rank properties.
And then one must consider all of the 32 minus 11 21 point groups that lack conversion separately.
There's no reason why they should behave the same way.
Remember that for second-rank tensor properties we pulled the argument that inversion imposes no restrictions or constraints whatsoever on second-rank properties so, therefore, two different symmetries that differ only by the presence or absence of an inversion center.
That is to say, you change 2 to 2 over m if you add inversion.
But the argument was inversion requires nothing, so the constraints or symmetry, too, look exactly like those for symmetry.
And here you've got to plod through every single one of the non-centrosymmetric point groups separately.
And they all have tensors that have different forms.
So what I'd like to do is look at one specific one.
And that is the matrix for symmetry 32.
And that is a point group that you'll recall has a threefold axis and twofold axes at intervals of 60 degrees.
And in your list of the qualities and absences, 32, where the 3 is parallel to the axis x3, and the twofold axis is parallel to x1.
So we're defining this as x1, this as x3, and x2 comes out halfway between the two.
So let me draw this looking down along the threefold axis.
These are all twofold axes.
And we'll take x1 in this direction.
x2 pokes out in between two twofold axes.
And x3 comes straight up along the threefold axis.
With that coordinate system, the form of the piezoelectric modulus matrix has this form-- d 11 minus d 11 0 d 14 0 0 0 0 0 d 15 minus d 14 0 and in the bottom row d 31 d 31 d 33 0 0 0.
So this matrix with the equalities put in is obtained by looking at the full three-subscript tensor and requiring that it look the same before and after any of the rotations involved by these three distinct axes-- the threefold and the pair of twofolds.
OK. Now, there are many different scalar modulae that one could define, some of them serious and worth the consideration because of the application in devices or other practical situations.
The modulus that I'd like to examine is something called the longitudinal piezoelectric effect.
And let's emphasize, again, that it is impossible to come up with one representation surface that fits every need because, again, the direct piezoelectric effect relates the components of a vector to a tensor, sigma ij.
There are six independent tensor elements.
So how can you describe how this vector is going to change as you change orientation of a crystal whose behavior is described by all of these modulae?
But I would point out, however, that there are only two distinct-- oops, I'm sorry.
This is d 14.
I don't know how I made that d 15.
There are only, in this array-- and I slipped a notch.
Excuse me.
Last couple of days are such that I am not able to even read from my notes.
So these bottom lines, my apologies, are all 0.
And there are two modulae, d 11 and d 14.
So there are two independent numbers, but they appear as different matrix elements and, therefore, different tensor elements, as well.
And this should be minus 2 d 14.
I'm sorry.
I slipped down a notch, and I got some for 32 and some for 6.
And I'm glad I found it at this point, or I'd really be in deep trouble.
OK.
So two numbers and that is, in fact, the form of the matrix for symmetry 32.
So the longitudinal piezoelectric effect is one of the representation surfaces that gives you the way in which the polarization will change for one very, very specific type of stress.
In particular, what we'll do is set this up as a coordinate system.
And we will look at a coordinate system where this is the reference axis, x1.
We'll come down with a compressive stress, sigma 11, along that axis.
And, therefore, we're applying a uni-axial stress, which has just one component of stress.
So that's very specialized.
In general, there would be six different components of stress.
But we're looking at one specific stimulus applied to this crystal plate.
In response to that sigma 11 there are charges induced on all of these surfaces.
And these charges, this charge per unit area, is proportional to the component of polarization P1, the component of polarization P2 along the surface out of which x 2 comes, and the charge per unit area or the polarization along x3.
So this would be P3.
And this would be the direction of x3.
Now I deliberately tried to show this sample as a thin wafer, which has a much larger surface area here than it does on the other two surfaces-- a much smaller area there.
Therefore, since polarization is charge per unit area, if this area normal to x1 is a very large area, there's a lot of charge accumulated there.
It's going to be easy to measure.
If we make the wafer vanishingly thin, then the charge per unit area is high, but the total area is small.
So there's going to be a negligible accumulation of charge on these two side surfaces.
So the longitudinal piezoelectric effect and the longitudinal piezoelectric electric modulus is an effect, where we look at the component of polarization, P1, in response to an applied stress, sigma 11.
So it's that simple.
Look at all the terms we've thrown out.
We've thrown out a whole bunch of elements of stress, which we could impose if we wanted to.
And we've thrown away two of the three components of polarization by designing a specialized sample.
So all that's left then is that P1 equals sigma 11.
And the relation between those two parameters is the 111.
So all this is going to hinge on one single piezoelectric electric modulus, d 111, and how that changes with direction.
So this is for one orientation of a plate.
And I had not specified how the orientation of this plate is related to the symmetry axes.
So let's do that now.
What I'm going to assume is that this is a crystal.
It doesn't look like it's hexagonal.
But imagine that this is a crystal of quartz.
And we could look at an x1 that's in this direction.
And imagine that we have cut out of this crystal a wafer that has a normal along x1.
And then relative to this coordinate system, if this is x1, the modulus d 111 would tell us what charges accumulated on these two surfaces.
But now, what we could do if we wanted to know how this modulus changed with direction would be to cut out a plate, a thin plate, in another orientation, where this is x1 prime.
And this has changed relative to the orientation of the cell edges in the crystal.
The crystal is fixed.
We're just cutting a wafer out in a different orientation.
So this is x1 prime.
We're going to, again, squeeze it with a tensile stress sigma 11 prime.
And we'll ask how the polarization P1 prime is related to sigma 11 prime.
And the answer is that P1 prime will be a tensor element d 11 prime times sigma 11 prime.
So what we are asking, essentially, is how does d 11 transform when we take the direction of x1 in a different orientation and, thus, change the value of d 11 prime?
It's going to change all of the piezoelectric matrix elements.
But we're looking at an effect in a sample that is deliberately prepared such that we will measure only the surface charge given by P1 prime.
And, therefore, the way in which the properties of this plate change as we vary the way in which we've sliced it out of the single crystal is going to be simply the variation of d 11 prime with direction.
So this is the general nature of what we will do when we define any of the scalar modulae related to the piezoelectric modulus tensor.
We can change our notation a little bit in that we have a modulus which I'll define as a scalar modules d. And that d is going to be d 111 prime.
And I know how to evaluate that.
d 111 priime will be C 1l, C1m, C 1n, where these are direction cosines, times all of the elements in the original tensors, dlmn, in the tensor referred to the original coordinate system.
So even though this looks simple-- it's just one modulus-- when we transform it, we've got a product of three direction cosines out in front at every single one of the 27 tensor elements in the original tensor.
So it's not as trivial as it seems.
So this is how this modulus that relates compressive stress to induced surface charge will change with orientation.
But what are these direction cosines?
These are the direction cosines-- not the full direction cosine matrix.
These are the direction cosines for x1 prime.
OK?
So we can get rid of this two-subscript notation if it's understood that these are the direction cosines of x1 and simply call these l l, l m, and l n, just as we did for the direction cosines of a vector because we're only concerned about the orientation of one of the axes, namely x1 prime.
We don't care diddly-bop about x2 prime or x3 prime because these don't enter into the modulus that we have defined.
And this will be times dlmn.
OK. Is what we're doing clear?
So we have defined this particular effect in terms of those of the 27 piezoelectric modulae which are necessary to describe it.
And then we've established how they will change with a change of the direction of one particular direction.
And we don't care anything about x2 or x3.
All right.
Now we go through this process of inserting for matrix notation with the equalities built in.
The proper matrix notation in the first term is d 11 in matrix notation.
That's this term up here in the upper left-hand corner.
The second term, the term that we've written as minus d 11, that's not d 11 at all.
This is, by definition, d 12.
And we need the true subscripts, if we're going to transform this.
So this really is not even a matrix because the subscripts have lost meaning, and we're just using them to identify equalities.
Then comes a 0.
And next comes something that we've labeled d 14.
And the subscripts there are correct.
That is, indeed, the fourth term in the first row.
But we're going to want to convert d 14 into a tensor element momentarily.
Now let's get the rest of the terms that are non-zero.
This is really d 25.
So the next term that is non-zero is d 25, which just happens, because of symmetry, to be equal to minus d 14.
But this is the true matrix subscripts, and this is the true matrix subscript here.
The next term over to the right is the fifth and final non-zero term.
This is minus 2 d 11.
And this is really d 26.
Those are the true matrix elements.
So we put in the proper matrix subscripts.
And now the next, final, step in the expansion is to convert these terms into actual tensor elements.
So this is d 111.
And these are tensor subscripts, so this is something we can transform.
This is d 12.
In tensor notation this is d 122.
And that's something that has a law of transformation.
d 14 is really d 123 plus d 132.
We lumped two tensor elements together to define this modulus.
Minus d 14 that appears in the next to the last non-zero spot is d 25. d 25 is really d 231 plus d 213.
Then, finally, d 26 is d 121 plus d 112
Shouldn't that be d221?
Sorry.
d 26, you're right.
That's down in the second row.
d 221 and d 212.
Now we've got something we can transform.
The law for transformation is l sub l, l sub m, l sub n, dlmn.
So And these are the direction cosines of x1.
So this term will transform as l 1, l 1, l1 times d 111 .
1 The next term will transform as l 1, l 2, l 2 times d 122.
And that will be d 122 prime for different orientation of x1.
This will be two terms.
This will be l 123.
And I can write them in any order.
So this is l 123 times d 123 plus d 132.
And these terms prime, when we change axes, are going to be equal to l 2, l 1, l 3 times d 231 plus d 213.
And this last term will be l 1, l 2 squared times d 221 plus d 212.
All right.
So this now is our new tensor element, d 11 prime.
And that's given by this sum of terms.
So we'll have a first term l 1 cubed times d 111.
And if I look through these other terms, that's the only term in l 1 cubed that I'll have.
The next term will involve the product of three cosines-- l 1 and l 2 squared times d 122.
And if I go down here, here's an l 1, l 2 squared again.
So I have plus d 221 plus d 212.
And then, finally, the other coefficient that I have is plus l 1, l 2, l 3-- which is what this should be.
And that will be times the sum of terms d 123 plus d 132.
Up here, you've got the same thing again-- plus d 231 plus d 213.
And I have a total of 1, 2, 3, 4, 5-- 1, 2, 3, 4, 5 terms.
OK, that is how the longitudinal piezoelectric modulus will change as we change the direction of the normal to the plate that we have cut out of the crystal.
So these are direction cosines relative to the crystallographic axes.
l 3 is the angle between the normal to the plate and the threefold axis.
l 1 is the angle cosine to the angle between the normal to the plate and one of the twofold axes.
And l 2 is the direction cosine for the normal to the threefold axis and the twofold axis.
OK.
So we've got it now in terms of tensor elements.
And now-- yeah?
Is it at all reasonable to assume instead of taking those sums in d 123, d 122, just saying 2 d 123?
Is that OK in assuming?
Or not necessarily?
Well, we could do that.
But I did it the long way to not obscure what we're doing.
OK?
This is a well-defined summation over subscripts.
And we're going to collapse immediately down to the sums.
And we're going to replace the equalities.
So let's see what comes out of this, if we now, having reached the zenith, having transformed the tensor elements, go down and replace this with a consolidation of terms and an insertion of the equalities between the matrix elements.
OK The first term is d 11.
So I will have l 1 cubed times d 11.
Notice I'm getting third powers of direction cosines, which is going to be what causes the exotic nature of these anisotropies.
And then I have a product of l 1 and l 2 squared.
And this is d 12.
And this second term is-- where did it go?
This is d 21 plus d 212.
And that is what we call d 26.
And then, finally, this product of three different direction cosines-- l 1, l 2, l 3.
And we have d 231 plus d 213.
And this is d 25.
And, again, an l 1, l 2, l 3 times d 14.
And the second term here is d 25, if I've done it correctly.
d 25 -- this is d 24.
And this one is d 24.
2/4 OK. Let's now insert the equalities-- back to where we came from.
d 11 is d 11. d 12, however, for symmetry 32-- I'm going to my handy-dandy chart of symmetry restrictions.
I don't want to do that.
That's fourth rank
It's still on the board
It's still on the board?
Yes.
Thank you.
When your nose is in it, it's hard to see.
d 12 is minus d 11.
1 And d 26 is minus 2 d 11.
So these two terms can be consolidated.
l 1 cubed plus l 1, l 2 squared times d 11 minus d 11.
So these two terms die.
And I have a minus 2 d 11 that's left.
If I insert the equalities here, I'll have l 1, l 2, l 3. d 25 is minus d 14.
And here's a d 14 itself.
So these two terms die.
And then I had d 25.
And that is--
You don't have d25
I don't have d 25.
Where did I get the extra one?
[INAUDIBLE]
OK.
I'll take your word for it.
And I know how it has to turn out.
OK.
So these two terms kill each other.
And I'm left with, then, an l 1, l 2, l 3.
Or have I left something out?
This is d 1-- this is d 15 and d 231 and d 23 -- uh, this is d 2 -- and the combination of 13 and 31 is d 25.
Right?
And if I look at my equalities, this is l 1, l 2, l 3 minus d 14 plus d 14 plus d 25.
And d 25 is minus d 14
Why would you add this d25?
Let me check my notes and see what I've got here
[INAUDIBLE]
OK. See what I -- d 25 is minus d 14.
And then I have just a d 14.
I don't know -- I see what I did.
I put it in the wrong slot.
This is d 14.
And d 25 is the one that's minus d 14. so this term dies, which is nice because that cross term is messy.
So what I'm left with, then, if I check against my notes, is d l 1 cubed plus l 1, l 2 squared times d 11.
And then I have minus d 1 minus-- this doesn't belong in here.
This wants to end up being l 1 cubed minus 3 l 1, l 2 squared times d 11.
The first term is l 1 cubed.
That's correct.
The second term is l 1, l 2 squared.
We've got a minus d 1 plus minus 2 d 11.
So I have minus 3.
It should be a minus.
Yeah, that carries down to a minus.
So I have minus 3 l 1, l 2 squared all time d 11.
OK.
So what we have ended up with is an expression for the longitudinal piezoelectric modulus as a function of orientation.
The surprising thing is that l 3 does not appear here at all.
It doesn't depend on the angle between the normal to the plate and the threefold axis.
It depends only on one modulus, and that is a remarkable thing.
This says that the shape of this surface is independent, essentially, of the property, any property that relates the one one prime to a uni-axial stimulus, sigma 11.
And you measure a vectory component in the same direction is always going to have this universal surface.
And it involves just a geometric term and then one modulus that changes the magnitude of the longitudinal piezoelectric modulus but does not change the asymmetry.
So let's see what this function looks like as a function of direction.
Maybe we better wait for that until we come back because that's going to take a few minutes.
So this is what we found.
That is correct.
And we have to now decide what this looks like, which will take a few more minutes.
But let's stop here rather than run late.
All right.
Let's take our 10-minute break as usual.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license, and MIT OpenCourseWare in general, is available at ocw.mit.edu
OK. Let's resume.
I cut things off at a time when we had the final answer.
And I left you hanging because we don't know what the final answer is telling us.
This says that as we change the orientation of the normal to the plate, relative to x3 and, secondarily, as we change the angle between the threefold axis and x2, we get this strange third-rank trigonometric function.
Let's convert these cosines into the appropriate angles.
This was the twofold axis.
And this was x1.
This was the direction of the threefold axis.
This was x3.
And this is the direction x2.
And that comes out in between a pair of twofold axes.
And we may want to look at this from up above, relative to these twofold axes that occur.
This thing here looks like a trigonometric identity, doesn't it?
Let's let this angle here be theta.
And l1, then, is cosine of theta.
And so our geometry is that this is l1 cosine of theta.
This is l2, which is the cosine of the angle between our direction and x2.
And that is the cosine of pi over 2, minus theta.
And that, then, is equal to sine of theta.
So this identity, as you all know-- I'm not telling you anything that you don't already know-- this is d is equal to d1,1 1, times the cosine of 3 theta, right?
You knew that.
Believe it or not, it is.
It's one of these obscure trigonometric identities, which wouldn't occur to you in a million years unless you go digging through some handbooks.
So this is rather astonishingly simple.
It's simply a cosine function.
But the interesting thing is it goes as a function of 3 theta.
So this goes through one cycle between a pair of adjacent twofold axes.
So it starts out as d1,1.
And it finishes up at d1,1.
And in between it is minus d1,1.
And so it goes down.
And now we come to something that I do differently than most people do.
Most people will say it goes as minus d1,1.
But minus d1,1 I like to show as a lobe going off in this direction, with a minus sign
How do you get that cosine of 3 theta?
Hmm?
How do you get d equals d1 cosine 3 theta?
How do I get that?
Yeah
That is just a trigonometric identify, believe it or not, a well-known trigonometric identify.
Actually, it's an exceedingly obscure trigonometric identity.
So the way this is going to go from x1 is it's going to be a positive lobe around x1.
There's going to be a negative lobe along x2 and then a positive lobe again about the twofold axis that's 120 degrees away.
And then a negative lobe, and then a positive lobe opposite this negative lobe, and a negative lobe opposite this positive lobe.
So it's a six-membered-- six lobes.
There is always a positive lobe opposite a negative lobe.
And what this means is that the charge is of opposite sign on opposite ends of the twofold axes.
So the response peaks up on twofold axes.
Now the thing that I don't like is that what Nye does is to say, OK, this is a negative value.
So you should plot the radius in a negative direction.
And that puts it over here, right on top of this positive lobe.
So what Nye shows is the polar plot of this result, in the plane of the twofold axes, is simply this.
This is x1.
And he shows a lobe here.
And he shows a lobe here.
And he shows a lobe here.
And if you interpret that as a polar plot you say, well, I know what the value is in here.
It's decreasing.
And there's nothing going on in here.
So the response must be 0.
And then it starts coming up again.
And there's a response in different directions here.
And then it goes back down to 0.
And in this range, there's nothing going on.
And that's not true.
What's going on is a negative value of the modulus.
And to me, that becomes abundantly clear if you just put a sign that labels the sign of the modulus within those lobes.
Now I submit, that's pretty anisotropic, isn't it?
Yes, Steve?
Did you just arbitrarily choose where your positive and negatives go, is it [INAUDIBLE] out here?
No.
This comes out here.
When l1 is 0, that's-- excuse me.
When the angle is 0, l1 is plus 1.
Remember that l1 is the cosine of this angle.
When that angle is 0, then the value is plus 1.
So I was careful to put the label x1 on this.
Now the other thing that we should examine is how these lobes vary in a direction perpendicular to x, in a direction that includes x3.
So let us look at how the function varies in a direction perpendicular to-- that includes x1 and x3.
OK.
Call this angle phi.
And in the x1, x3 plane, d1,1,1 prime turns out to be l1 cubed d1,1.
Before you go to our general expression.
And l2 is cosine of 90.
This thing drops out.
We're left with simply d1,1 prime equals l cubed times d1,1.
So this then goes as cosine cubed of phi, which is a lobe that starts out at plus 1.
And then because it's cosine cubed, this dies out very, very rapidly.
So these lobes are very flat in the x1, x2 plane, die out very rapidly as the cube of phi, where phi is the angle between the twofold axis and x3.
So the interesting thing about this surface is that if you decided to pick up the random fragment of crystal and determine what its Piezoelectric Modulus is.
One way of doing this, a poor man's test for piezoelectricity is to just clamp a fragment of crystal between two electrodes and then hook this up to a variable frequency generator that sweeps through a range of frequencies, changes the frequency of a voltage across these plates, and then comes back and sweeps again.
Or alternatively, have a knob one on that lets you change frequency.
Then put a pair of earphones across the crystal.
And if we would do that, by having this variable frequency generator, as we turned it, when we hit a resonant frequency that set up either a full wavelength of a vibrational wave in the crystal-- or half wavelength, or one wavelength, or 3/2 wavelength-- there'd be a resonance.
And the capacitance does something crazy.
It does like this as you go through the resonant frequency.
And then your simple detector-- a pair of earphones-- as you tuned the frequency, you would hear static as you went through this discontinuity in capacitance.
Or alternatively, you could put this on a cathode ray tube and just have a sweep frequency and put the voltage across the electrodes on the oscilloscope screen.
And you would see something like this, and then this discontinuity, and then maybe second harmonic.
And it would work just fine.
This is a poor man's way of detecting piezoelectricity.
But what if-- what if-- you happen to have your piece of quartz with the electrodes directly on the c-axis?
The modulus would be 0, and you wouldn't detect anything.
Or if you put on the probes such that they were in a direction that was in between these lobes where, again, the Piezoelectric Modulus has gone to 0.
You wouldn't find anything.
So measuring the Piezoelectric Modulus for a random chunk of material is dangerous, if you just look at one direction and say nothing's going on; it's not piezoelectric.
The other thing is the material could have a piezoelectric response that's so weak you just can't measure it.
Yeah, OK?
This stuff's only for single crystals, right?
Because if you [INAUDIBLE] polycrystal material, you're getting [INAUDIBLE] averaging
You're averaging over all directions.
So this is for a single crystal.
And that's the only time you get these exotic, very anisotropic surfaces.
Another method that I've seen in a rather old book is really nice.
What you do is you cut your crystal into a little plate.
And then you drive the plate by means of putting electrodes on it and hook this up to a variable frequency.
And if the material is transparent, you will set up-- depending on the velocity of sound in the material-- you will set up standing waves when you hit the right frequency.
And these lines that I've drawn are places where the displacement is always 0.
And these things that I've indicated at little squares is a region where the displacement goes up and down between its maximum and minimum extreme.
Now if this crystal is transparent and you shine a light through it, these little regions bounded by lines of zero displacement act like little lenses.
So if you pass a beam of light through it, you get a bunch of right maxima, little focused spots on the sheet.
And you can determine for a particular frequency if this is the 1 and 1/2 wavelengths along a dimension that you know, you can again find the Piezoelectric Moduli.
So there are lots of ways of detecting this effect.
But when you have a single crystal, you've got to really look at this as a function of direction, to be absolutely sure.
One other Piezoelectric Modulus is in the problem set.
4 bar 3m, this is the structure of sphalerite and a lot of the compound semiconductors.
This is not in the notes.
But if you try the problem, what you find is, again, a highly, highly anisotropic behavior.
And this is the direction of the unit cell of the crystal.
You find that the piezoelectric response is a series of very sharp lobes, positive and negative, going along the directions of the body diagonal, namely the 1, 1, 1 directions in the crystal.
And then there's another lobe that goes like this, negative, positive.
Another lobe that goes up here, positive and negative.
So very sharp lobes along the body diagonal, alternately positive, negative, positive, negative as you go around 0, 0, 1 plane.
OK. That's what the longitudinal piezoelectric effect is like.
You can invent single-crystal devices for weighing fish.
And then you have an interesting question.
If you hang the weight on the crystal, and it's a flat plate, how would you orient the crystal to get the optimum sensitivity?
So you define a scalar modulus for the crystal in the particular direction you're exerting a uniaxial stress.
And then you express this module in terms of theta, the angle of rotation, within the crystographic plane.
In the case of the sup cell if you looked at that geometry, that is a flat plate subject to compression, where you measure the surface charge perpendicular to the direction of the uniaxial stress, not parallel to it as we've done here.
And then there's a question if you rotate the plane of the crystal about the imposed stress, what orientation gives you optimum, maximum response?
So there are fun things that you could do with that.
We have a quiz next time.
There's a lame-duck period after the quiz will be over.
And I want to say a little bit about the tensor aspects of elastic moduli.
Mechanical behavior is something that you have or will cover in great detail in a graduate-level class on mechanics, but I think very few people who deal with things other than cubic materials or with single-crystal materials where the elastic constants are functions of crystal symmetry and direction.
So let me, at least at the time available, set up a definition of stress in terms of strain.
Stress is a second-rank tensor.
There are six unique components.
So we could write an expression for sigma 1,1; sigma 2,2; sigma 3,3; sigma 4,4; and sigma 5,5.
And sigma--
--1,2
No, I want to go down like this.
So this would be sigma 2,3; sigma 1,3; and sigma 1,2; eventually to be known as sigma 1, sigma 2, sigma 3, sigma 4, sigma 5, and sigma six.
But actually, if we're writing out a full tensor relation, we should write down all six elements here.
So let me do that and put in a 3,2.
Put in all nine of them, because before we condense to matrix form, that is what we're going to have.
So we'll have a sigma 2,3; we'll have a sigma 3,2; we'll have a sigma 1,3; a sigma 3,1; a sigma 1,2; and a sigma 2,1.
And we can express each of the stresses in terms of the elements of strain.
So I'll have an epsilon 1,1; an epsilon 1,2; an epsilon 2,2; an epsilon 3,3; an epsilon 2,3; an epsilon 3,2; an epsilon 1,3; an epsilon 3,1; an epsilon 1,2; and an epsilon 2,1.
Nine terms, nine terms.
And in between is going to be a tensor consisting of 9 by 9 elements.
There will be 81 different coefficients in here.
If we want to derive symmetry constraints, we're going to have to, for every one of those 81 elements, do a transformation.
And each transformed element is going to be a sum of four direction cosines times one tensor element, repeated 81 different times.
Not something to be undertaken on a short afternoon.
OK. What are the tensor elements in here?
They are represented by the symbol c. This would be c1,1,1,1.
This would be c1,1,2,2 times epsilon 2,2; a c1,1,3,3; a c1,1,2,3; and so on.
And these c's, in one of the great perversions of scientific notation, are called stiffnesses.
The other thing we could do is to write strain in terms of stress.
And really, stress is something you can do to the crystal.
So stress is an independent variable.
And this is something that I feel more at home with.
So I'll have an epsilon 1,1; 2,2; 3,3; epsilon 3,2; 2,3; epsilon 1,3; epsilon 3,1; epsilon 2,1; and an epsilon 1,2.
Again, nine elements of strain.
And these will be given by coefficients times each of the nine elements of stress, sigma 1,1; sigma 2,2; sigma 3,3; sigma 2,3; sigma 3,2; and so on for nine different components.
Andy the coefficients here are designated with the symbol s. These are the tensor elements.
So this is s1,1,1.
This is s1,1,2,2.
And the s's stand for compliances.
Now English is a very strange language.
But to call compliance s and to call stiffness c is surely a perversion that can be designed for no other purpose than to confuse the introductory student and give the instructor some feeling of superiority.
I don't feel superior, I feel embarrassed that I have to explain this.
Stiffness, c; compliance, s. It's got be one of the atrocities of scientific notation.
To remember which goes with which, I'll tell you what works for me.
If it doesn't work for you, forget it.
The s goes with sigma.
And the little c goes with the epsilon.
That's a c with a bar in it.
That works for me.
If it doesn't work for you, use your own mnemonic device.
English does a lot of things strangely, but nothing is as perverse as this.
Now what's some examples?
Consider this.
You park your car in your driveway, but you drive your car on a parkway.
Why don't you drive your car on a driveway and park your car in a parkway?
No, it's the other way around.
It's almost as bad as this.
What are some other ones?
In my basement, I have something that's called a hot water heater.
You don't heat hot water, you heat cold water.
Why isn't it called a cold water heater?
Even better, I once worked for a year and a half in Switzerland.
And in the kitchen of my home I had, in German, an elektrowarmwasse rsheissungsapparat.
That's one word that's 11 syllables.
The Germans have a knack with the language that we have never approached in English.
But there's another one you've probably heard.
You know what the prefix "pro" means.
That means for, and "con" means against.
And so progress is moving forward, and I leave it to you to decide what Congress means.
OK.
So I could go on and on.
I probably shouldn't, unless you egg me on.
No, I won't.
But anyway, there are-- OK, I'll give another one.
Why is what a doctor does called "practice"?
This is not reassuring at all.
Let me invite you over to my practice.
And why is it when you take an airplane your flight ends with a terminal?
That's doesn't sound very encouraging either.
So anyway, language is silly.
And I guess if you want to call c stiffnesses and s compliances, it's OK.
The verbal description of these terms means something.
Because if the stiffness is very high, that says that a little bit of strain requires that you haven't posed a very large stress.
So stiff is when the material is resistant to stress.
So you have to have a very, very large strain to get yourself a given level of stress.
Conversely, if the material is very compliant, a very small stress for a compliant material should give you a big strain.
And that's exactly what a large value of s will do.
It will give you a large strain for a relatively small stress.
So the words describe what's going on.
OK.
There is great utility in reducing these equations to a matrix form, which is going to cut us down from a 9 by 9, or a tensor with 81 elements, to a smaller number of subscripts and a smaller number of matrix elements, so if we let sigma 1 be sigma 1,1; if we let sigma 2 be equal to sigma 2,2 and sigma 3 be equal to sigma 3.
And then if we write the inequalities we've had before, and we let epsilon 1,1 to be replaced by epsilon 1, then we would have out in front here a term simply c1,1, a 1 for the sigma and a 1 for the epsilon.
The next term would be c1,2 times epsilon 2, where obviously epsilon 2 has replaced epsilon 2,2, and 1 stands for the two indices, 1,1.
So this will go c1,3; epsilon 3.
And then you hit these messy factors of two.
You'll have a pair of terms.
You'll have c1,4.
And that will stand for a term c1,1,2,3.
And then there'll be another term c1,1,3,2.
And if you want to write this as a matrix, c1,4 has to be equal to the sum of these two things, times epsilon 4.
That's the only way you can write it as a matrix.
And if you think it was bad worrying about a factor of 2 in front of half of the terms on the right-hand side of the equation, we're going to be dealing with factors of 2 for terms in the upper 3 by 3 array.
We're going to be worrying about a factor of 4, somehow or other, down in the lower 3 by 3 array.
So the way you handle the factor of 2 really is a nightmare for relations in elasticity.
But continuing on with the first line, c1,5 times epsilon 5 would be a combination of c1,1,3,1 plus 1,1,1,3.
And then finally, when you get to c1,6 times epsilon 6, that would represent a combination of c1,1,2,1 plus c1,1,1,2.
You have to make a decision where to swallow the 2.
And probably the best thing I can do is to give you conventions for relabeling stress, strain, stiffness, and compliance, which I'll pass around to you.
No big deal, except that this is a case where you can eat the 2's early on, but you can't even break even.
So very, very briefly, what you do is you convert the tensor elements of stress to matrix elements in exactly the same way we did with piezoelectricity.
For strains, we do exactly the same thing that we did with the converse piezoelectric effect.
The factor of 2 that you swallowed in definition of stress pops up to haunt you when you deal with the strains.
And you cannot write a nice matrix relation unless you divide the off-diagonal elements of strain by 1/2.
And that was exactly the same thing we encountered with the converse piezoelectric effect, which related strain to an applied electric field.
In defining the matrix stiffnesses, the c's, you forge straight ahead, and you let ci,j,k,l, which is identical to ci,j,l,k-- that's the term that relates the two equal shear stresses-- and you define that as a matrix term with two subscripts.
No factors of 2 or 4 are involved.
Then when you hit the s's, there's a nightmare.
si,j,k,l is sm,n if m and n are 1, 2, or 3; that is to say, not 4, 5, or 6. si,j,k,l, which is the same as s,i,j,l,k, has to be defined as one half of sm,n, where m or n is 4, 5, or 6.
And then finally, you have to throw in a factor of 4 when both m and n are 4, 5, or six or, in other words, m and n are not 1, 2, or 3.
So it is a mess.
And these are rules that you have to bear in mind if you're ever going to go from matrix form, which works absolutely lovely in a fixed coordinate system.
But if you have a single crystal and you want to refer it to different axes, you have to be prepared to resurrect the full tensor form.
And then and only then can you work symmetry transformations.
I don't want to go through the simple algebra of expanding and contracting these terms.
So the next two sheets show you how you condense from tensor to matrix notation, and then go back from matrix notation to tensor notation.
This is for the compliances si,j,k.
And the next page does the same thing, if you care to write stress in terms of strain.
And there are a lot more factors of 2 that appear there.
But again, it shows you how you go from tensor notation, for subscripts on the c's, down to matrix notation with 2 subscripts, and then go back up again to tensor notation.
So it's a tedious business.
And you've got to keep careful tracks of your factors of 2 and your factors of 4.
How about symmetry restrictions?
Holy mackerel.
Transformation of 81 different elements.
Well, in order to do the tensor transformations, you have to go to the full-force subscript notation.
Only then is a law of transformation defined.
Fourth-rank properties are even tensors.
So mercifully, there are not as many different possibilities.
All point groups that differ by presence or absence of inversion have exactly the same form of the tensor.
So symmetry 2, m, and 2/m look alike.
Symmetry 2, 2mm, and 2/m, 2/m, 2/m look alike.
So there's only one orthorhombic tensor, only one kind of monoclinic tensor, only one kind of triclinic tensor, and so on.
And the only place that you have more than one form of the tensor for a particular set of point groups is looking at the point groups that are based on single-rotation axes, something like 4, 4 bar 4m, and those that are based on the axial arrangements 4, 2, 2.
So they're different, just as they were for second-rank tensor properties.
For cubic, a rather remarkable result.
There are three independent compliances.
But cubic crystals are not elastically isotropic.
They are anisotropic.
And it's just the nature of the tensor that requires that.
So rather than letting you hang by your thumbs wondering what's on these pages, let me pass around the summary of symmetry constraints.
If you understood what we did for third-rank tensors or even second-rank tensors, you how to do this.
And fortunately, the terms almost over.
So I can't make you do it, which is probably an enormous relief to you.
Some additional bits of information.
The transformations for hexagonal crystals are complicated because threefold and sixfold axes do not change the reference axes x1, x2, x3 into one another.
And the equations that relate the individual tensor elements are more complex.
For the threefold axis, for example, 2 s1,1 minus 2 s1,2 is equal to s4,4.
So they're not simple relations between them.
And that's simply because you're not changing one reference access into another.
You're changing x1, for example, into a linear combination of x1 and x2.
Two things down here of interest.
I said that cubic crystals are not isotropic.
What would have to be the case if the material were to be isotropic?
Well, it turns out that if you do this for the compliances, s, if 1/2 of s4,4 is equal to s1,1 minus s1,2, then the material is elastically isotropic.
If you do this in terms stiffnesses, the c's, then 2 c4,4 is equal to c1,1 minus c1,2 if the material is supposed to be isotropic or going to be isotropic.
Then there is one other inequality that depends on structure.
And this is something known as the Cauchy relation.
And it depends on the interatomic forces.
The first condition that has to be met is that the forces between the atoms should be central forces.
What is a central force?
Well, this is a case where the attractive force between the atoms is directly along the line joining their centers.
Isn't that always the case?
Don't crystals hold together because there's an attractive force between atoms?
Well actually, for metallic crystals and ionic crystals, maybe that's not a bad assumption, particularly for ionic crystals.
But if you had a covalent material, where the bonding was due to overlap of orbitals like this, then the thing that holds the crystal together is overlap between these orbitals.
And the force holding the atoms together is a force that goes through this shared electron pair.
And that's not a central force.
And so the Cauchy relation generally fails rather badly for covalently bonded materials.
Second assumption is that each atom is at a center of symmetry.
And the reason for that is so the force in the plus-x direction is the same as the force in the minus-x direction.
And that is not true for any material that has tetrahedral coordination.
So it's not true for any of the forms of SiO2.
It's not true for any of the compound semiconductors that are based on tetrahedral units.
So there, the atom inside of these tetrahedra is decidedly not at an inversion center.
And finally, you assume that the crystal is under no state of initial stress.
Because if it is under initial stress, you've squished it, you've stretched the bonds.
And the forces-- not to a major degree, to be sure, but-- the forces will not truly be perfect central forces.
So if all three of these assumptions are satisfied, then you have an additional equality c1,2 is equal to c4,4.
What I'll do next time is I'll bring in some examples of data for stiffnesses and compliances and, in particular, show you how some of these elastic tensor elements change with temperature.
And we can examine them to see how well the Cauchy equality or the isotropy condition is satisfied.
All right.
That's about all that we'll do about the definition of fourth-rank tensor properties.
On next Tuesday to wrap the term up, we'll look at the variation of some elastic moduli, such as Young's Modulus or the shear modulus, for different symmetries as a function of direction in the crystal, OK?
But you all knew that.
That is quite a mouthful.
So if you like, you can refer to it as SST.
And today, we're taking off.
So my name, for those of you who don't know me, is Bernardt Wuensch.
My room number is 13-4037, The office that's become a legend in its own time.
And my extension number is 3-6889.
And hey, just to get you sensitized and thinking about the right things, let me point out that my extension number has a point of 180 degree rotational symmetry right in the middle.
You can pick it up, turn it head over heels by 180 degrees.
And it's mapped into coincidence with itself.
Just happened to get it.
You might think I would have had to have fought for years to get an extension number like that.
But no, it just happened to come my way.
OK, some words about the formalities of the subject.
First of all, the format of the class is unusual.
We meet four hours a week, but because most if not all of you are graduate students anxious to get some work done in the laboratory, we do this in two two-hour chunks.
So we meet Tuesday and Thursdays for two hours.
Two hours is a lot of time for anything, however good.
So what we do is to take a long intermission halfway through and let you go out and enjoy what's left of the lingering summer for 10 or 12 minutes.
And then, come back refreshed and we will resume.
Most graduate students like this arrangement because it gives them a chance to duck out and make a setting on a furnace or turn something off in the laboratory.
And it works out better for them than having a one hour time chunk every day of the week or four of the five days of the week.
In any case, nobody's complained about it.
So I assume that will work satisfactorily for you as well.
The other question that one immediately asked at the beginning of the term, how many quizzes?
And we're supposed to tell you that straight up.
There will be three quizzes.
No final examination-- do I look like the kind of Scrooge that would prevent you from getting a good flight home at Christmas time or have you working, cramming for a final examination a few days before Christmas?
And for my part, I can remember the good old days-- or not so good old days-- when I did give a final.
And there I would be, lying down on my stomach under the Christmas tree grading final examinations.
And every time one of my little kids would come near, I'd lash out with my foot and say, get out of here, kid!
Can't you see Daddy's got papers to correct?
Well, no Scrooge.
No final examination.
We'll have three quizzes.
The quizzes will also be a little bit unique.
Since we have two hour chunks of time, I found by experience that if I give the quiz during the first hour everybody is sitting around glassy-eyed, absolutely brain dead and pay no attention to the lecture that follows.
If I give the quiz in the second hour, everybody is pretending to pay attention and then sneaking surreptitious looks at their notes just so they can have everything packed away before they have to write on paper.
So my quizzes are two hour quizzes which lets you not work for two hours, but gives you all the time you could possibly want.
And people start leaving after about an hour and a quarter.
But you can stay for the entire two hours if you want.
Even at that, I found by experience that if I give you two hours for the quiz, after about an hour and a half, everybody's looking out the window, looking back at the ceiling, not a single pencil is moving.
Then I will say, OK, you all done?
Everybody starts writing again and going through their papers once more.
And even after two hours I find that in order to get the quiz papers, I have to plant one foot on the edge of your table, grab hold of your quiz with both hands and drag it out of your clutching fingers.
So you can take the full two hours.
But it should not be necessary for you to consume that much time.
The quizzes will come-- and this is something else we're supposed to explain to you-- they will come at one third of the way through the term, two thirds of the way through the term, and 2.983/3 of the way through the term.
And if you're wondering where that number comes from, this lets me put the quiz just before the final week of the term when we're not supposed to give examinations.
So there'll be three quizzes.
You will have opportunity for lots of practice with problems.
We will have on the order of 15 problem sets.
And for the most part, they will be very short, or modestly short, and designed to give you some practice in working with the material.
Because as the nature of this subject begins to unfold, you'll see that it involves a type of mathematics that you've really perhaps not had much practice with.
It involves geometrical relations.
And to really master it, you have to work with the material and get some practice.
Another question that's perennially asked if not outright raised in private, what do the quizzes count?
What do the problem sets count?
The answer to that is that the problem sets will count a lot towards your understanding of the material.
But I'm not going to grade them.
And I'm not going to factor them in along with the quizzes to decide your final eventual fortunes in this class.
The problem sets, moreover, there are a lot of them.
But they will be optional in the sense that if you do them, I will carefully correct them, add words of inspiration and advice, correct things where you've gone wrong and then return them to you as quickly as possible.
But if you how to do the problem, you say, ah!
Why does he wants us to do this and waste our time with a silly problem like this?
Don't do it.
Don't do it, because if you know how to do the problem set, that's fine.
And you've got better things to do with your time.
And I've got better things to do with my time if the feedback is not going to be of benefit to you.
So I hope you will do them.
And as I said, if you do them, I will correct them promptly and thoroughly.
And if you haven't got the foggiest idea how to do the problem, do what you can.
And then, write down a plea of help-- I don't understand what's going on here!
OK, and then I will take the time to write out what's going on, hopefully to your benefit and use.
Will I turn out solutions?
Only on an individual basis in the fashion that I've just described.
I find that if I write out a solution to each of these problems, you don't do them.
And you say, oh, so that's how you do that, throw it in a file, and not look at it until the night before the quiz.
So there will not be solutions handed out other than correction on an individual basis on your papers.
Is that all that I wanted to say?
I think that's about all for the formalities.
Actually, I should add one postscript to say the problem sets don't count anything toward your final grade.
They do in one minor sense.
When you have a large class and you plot up the grades and there are no lumps with gaps in between, there comes a point where you have to separate one grade from another.
And if you've done well on the quizzes and you've done well on the problem sets and there's just one quiz that's a little bit, I say, OK, he or she had a bad day that afternoon.
And I'll give you the benefit of the doubt.
And even though I'm not a vindictive sort, if you're right on the fence and you haven't done any of the problems, then without malice I say, gotcha!
And you go down [INAUDIBLE] on the low side of the barricade.
And I think that's only a natural indication because there are some cases where, with Solomonic judgement, you have to decide who gets what grade.
OK, let me say a little bit about the texts.
There are a number of books that deal with crystallography.
For the most part, though, they consists of an introductory chapter.
Every single book on the solid state feels compelled to write some sort of half baked chapter on crystal structure or crystallography.
And usually, these chapters consist of big tables.
And they say, there are 14 of these.
There are 17 of these.
There are 32 of these.
There are like 230 of these.
There are 1,170 of these.
And then, that has all the excitement and stimulation of reading the telephone directory.
It's a crazy cast of characters, but it's awfully hard to see the plot.
So what we will do all the way through is derive everything.
So you can not only see how it turns out, but why it has to be that way.
And that's the way, in my opinion, one really learns this material.
A couple of other very pedantic comments about the material.
In the early part of the term, the first half in fact, we're going to use plain old geometry.
Now, geometry really doesn't cut much mustard around the Institute.
If you can't integrate it or take its Fourier transform, that's a mathematics you don't have to take seriously.
Well, geometry is a perfectly valid branch of mathematics.
And one can do what we're going to do in more complex terms using the language of group theory.
And we will, in fact, use a little bit of that later on.
But for the most part, just diagrams with simple geometry are going to be one of the principle tools in the initial part of the class.
About halfway through, we'll switch over to something that is much more mathematical in the traditional sense.
Here, a little bit of linear algebra and matrix algebra will help you.
If you haven't had that or haven't looked at it for a while, we'll build it up from ground zero so that you'll be able to fully understand it.
We'll hit a few eigenvalue problems towards the end of the term.
If that doesn't get your adrenaline pumping, that will be developed in a physical context so that you're doing the sort of problem before you even know what it's called.
So it's going to be a user friendly course that doesn't rely on something that you may have had two or three years ago.
OK, but the other thing that I wanted to say was that this class is not like many classes in that you talk about something for one week.
And then, you put it aside and you talk about something completely different the next week.
Our first half of the course will be one long process of synthesis.
We're going to start out very, very simply with little mapping transformations.
This is picked up and rotated and slid over to here.
And you'll say, ho-hum, let's get on with it.
Come on, go faster.
But we'll build on this and then build on what we've just done to what comes next.
And unlike most of the classes in science that you take where you start with general terms and you zero in on some little nugget like f equals ma, e equals mc squared, lambda equals 2d sin (theta), a little nugget like a bullion cube that you can drop in your pocket.
And then when you need it later on, you pull it out and add hot water.
And then, you have a tool that you can use.
We will do something that's completely different in its structure.
It will start out simple.
It will grow.
It will blossom like an elegant [?
Filigree ?]
structure that gets more and more complicated and diverges rather than converging to a nice, tight, little nugget.
It's going to get very, very complicated.
And the reason for doing this gradually and thoroughly is so that you can understand the complexity and where it comes from.
OK, so my moral here is keep up.
It may seem easy when you start.
But we're going to assume that you've got that down cold before we go on to the next step.
OK, texts.
Apart from these half baked treatments which I just keep [INAUDIBLE] on, one of the very best books is by an old MIT guy, Martin Buerger, who was one of MIT's most distinguished faculty.
He was the very first faculty member to be honored with the title Institute Professor, the very first one.
Chairman of the Faculty, all sorts of awards from professional societies-- he has a book called Elementary Crystallography.
This is published by Wiley.
There's some who dispute the term "elementary."
But he really has a book which uses, at the outset, nothing more than geometry.
He doesn't throw in comments like, "It can be shown that," or "By further work, it turns out--."
He does everything for you.
Everything is down there so you can see how it's done and what the results are.
To me, it is the best book on the subject.
That's the good part.
The bad part is that it's been out of print for about 15 years.
So what I am going to do is to make-- now that I know how many of you are going to be present-- I'm going to make a Xerox copy for you of the first half of the book.
What a department!
What a class!
You're going to get a classic text, 50% of it, without spending a nickel.
And that'll be the text for the first part of the class.
We will be doing some derivation that are not in this book.
And for that, I will have notes that I have written out.
And you'll get Xerox copies of that.
So we'll have lots and lots of handouts during the course of this semester.
I'd like to call your attention, though, to two other books.
These are not textbooks.
These are reference books.
And you can see from the shape of this one that this is one of my favorite volumes.
It's thoroughly worn out.
This is something that is called The International Tables for X-ray Crystallography.
And it is published by an organization called the International Union for Crystallography.
The funny sounding term, "International Union for Crystallography," sounds like an organization under which diffractionists go out and strike for higher pay.
But no, this is actually a federation of all of the national societies of crystallography from all over the world.
And among the useful things that they do, besides having a splendid conference every couple of years, is to publish these tables.
And volume one is called Symmetry Tables.
And everything that we will derive and all of its properties-- physical and geometrical-- are tabulated in this book.
It is, however, a reference book and not a textbook.
You don't learn it for the first time from this book.
But in terms of generating atomic arrangements from the data that's present in the literature, looking at the arrangement of symmetry elements in space, and how they move atoms around, it is the code book that tells you how to crack the arcane language in which diffraction and structural results are recorded and find out how to unravel it.
I call also to your attention, although it will not be germane to this class, there are four other volumes.
Volume two is called Mathematical Tables.
And this has all sorts of useful stuff.
If you've ever done diffraction, you know that depending on the symmetry of the crystal, there are some planes for which h squared plus k squared plus l squared divided by 2 pi is not a reflection [INAUDIBLE] if the crystal is green, and other arcane rules like that.
All of these are summarized in these books.
There are quantities that you need to calculate, things like interplanar spacings.
Tables are available there.
So this is a handy thing primarily for diffraction.
Volume three is called Physical Tables.
And this is where you find things like absorption coefficients for x-rays and for neutrons.
It's where you find the latest values of absorption coefficients, neutron scattering length.
And since these things are derived experimentally, the values improve and change from time to time.
So this is where you find the most up to date values of physical constants and items that are necessary for diffraction.
It never ceases to amaze me how somebody who has the good fortune of having to use the diffraction for a thesis will labor carefully over making the measurements and reducing the data.
And then when it comes to using a wavelength, which is how the final numbers will be determined, goes to an appendix of a book on diffraction that was published 20 years ago.
And that's not the most up to date value.
Scattering powers of x-rays by the electrons on the atoms are calculated from wave functions, which constantly get better from year to year.
And the value of the scattering powers of the function of angle gets better from year to year.
So this is where you want to go if you need any of that physical data.
And finally, volume four is-- it's not its title, but it's essentially an update of the Physical Tables, giving later values which came out about 10 years later.
OK, this series was getting out of hand.
So I have to bend my knees and use two hands when I pick up this one.
This is a continuation of the series, essentially.
But this one is called International Tables for Crystallography, period, no x-rays because neutrons and electrons are just as important today for doing scattering experiments.
And this is International Tables for Crystallography.
No x-ray in there.
And there are now something like six volumes out.
They're not called one, two, three, and four, but they're called A, B, and C to avoid confusion.
And volume A is one called Space Group Symmetry.
And then, there are a whole series of other ones.
As I say, I think there's six of them that give physical data and all sorts of useful guides.
I have mixed feelings about the new series.
You will see that it is about three times as large and three times as heavy, which means it's nine times as expensive.
And to me, it's almost the case for most people of a situation where if it wasn't broke, you shouldn't fix it.
And what they've done is that they've put in all sorts of esoteric theory which probably is going to be of interest and use to perhaps 5% of the readers.
But nevertheless, if you wanted, you'll find it there, which is something that could not be said before.
They've added a few things which are useful, but a lot of additional information which you don't really need.
And you pay for that whether you want it or not.
Nevertheless, it's been done.
You can't buy the old volumes any longer.
You have to buy the new volumes.
So anyway, this is what you'll find in the library now.
Maybe they do still have the old volumes, one through four.
This, we will make reference to in the course of the term.
I will give you some copies of certain pages in here as handouts when we need them for purposes of illustration or for use.
But I spent the last five minutes just to make you aware of the existence of these books.
And these are really the penultimate source of information and numerical quantities that will be used in diffraction, one of the principle applications of crystallography.
I think I have just enough enough-- to start things off, I have a syllabus for the course that is, in very dense form, exactly what we will be covering this term.
And I'd like to lead you by the hand through this.
All right, what we will be doing in the first half of the term is something that is known as crystallography.
OK, the meaning of the word is almost self-explanatory.
The first part is crystal.
We're going to be dealing with the crystalline state of matter.
To me, amorphous materials, although they may be important, have all the interest of a piece of steak before it's been cooked.
The atoms in amorphous materials are fine.
But they really get interesting when they organize themselves into an ordered fashion.
So the name is self-explanatory.
The first part, crystal, means we're going to deal with the crystalline state.
What does the graphy mean?
That means mapping or geometry.
And let me give you an example of a few other words that have the same sort of structure.
Geo-- the Earth-- followed by graph, geography, is the mapping of the Earth.
And there are many other terms that involve these two separate parts.
Crystallography, though, is very often subdivided into different flavors.
There is something well defined called x-ray crystallography.
And this is the experimental determination of the crystallography of a material using diffraction, usually x-rays because they're relatively inexpensive and they're widely available.
But increasingly, neutron scattering or electron scattering is used for this purpose.
And there are a number of very powerful, very exciting sources of neutrons, either from reactor sources of unprecedented intensity or from what's called a spallation source, where an entire synchrotron is built just to direct a beam of particles onto a heavy metal target.
And those high energy particles split off neutrons from the nuclei of the target material.
Doesn't really matter what the material is.
It helps if it's a heavy metal.
The nice thing about these sources of neutron radiation is that they're so expensive they are all national facilities.
And the consequence of that is that anybody with a good idea and a project worth doing can apply for beam time.
And if it's a good problem, you get it.
So you're using a facility that cost $1 billion.
You have people whose sole function in life is to help you do the experiment and make sure you're doing it properly.
And this is a very, very exciting time to be somebody working with diffraction using these neutron sources.
There's another branch of crystallography which is called optical crystallography.
And this is the characterization and study of crystalline materials using polarized light.
You can identify unknowns using their optical properties if they're transparent about 10 times faster than you can do with x-ray diffraction.
It's a technique that today is little used.
But it's a very powerful technique.
And all it takes is a microscope, and you're off and running.
Some other flavors of crystallography, well, I'll mention the one that we're going to use.
What we're going to talk about is something called geometrical crystallography, to distinguish it from these other branches.
And this is synonymous with symmetry theory.
So that's what we'll do for the first month and a half or so.
All right, let me introduce now some basic concepts.
Geometrical crystallography is the study of patterns and their symmetry.
So let me give you an example of some very simple patterns that extent in one dimension.
And let me put in a figure.
The thing that is in the pattern is something that's called the motif.
And let me use a plump, little fat comma.
And I'll make a chain of these things extending in one dimension.
The nice thing about this fat little comma is that it is a figure which, in itself, has no inherent symmetry.
So it is asymmetric, without symmetry.
And imagine this is being repeated without limit in both directions, both to the left and to the right.
We then draw another pattern with a different sort of motif.
And let me use a rectangle with one concave side.
OK, and I think you get the picture of this one.
And imagine that as extending without limit indefinitely to the left and to the right.
Then, I'm getting tired of inventing new motifs.
So let me use the same motif the second time, but arrange it in a slightly different way.
And again, imagine that as extended indefinitely.
OK, having now generated these three patterns in two dimensions but extending periodically in two dimensions.
Let me ask the question now.
And even if you have not the foggiest idea, you have a 50% chance of being right.
Are any of these patterns the same?
Or are they all different?
Are any of the patterns the same?
Or are they different?
Well, that's a-- yeah?
[INAUDIBLE]
OK, that is an answer that's right because the bottom two involve the same sort of figure.
They have the same sort of the motif.
They both have the same rectangle with one concave side.
And that's a valid answer.
Do you have a different answer?
The first and third are the same
First and third are the same.
Why do you say that?
They both have [?
rotational symmetry.
?]
OK.
This is the point I was trying to introduce.
And that is your choice of answering the question, one is the nature the motif.
And you're absolutely correct.
This pattern and this pattern are both based on the same motif.
But in patterns, we are less concerned with the motif that is in the pattern than we are with the relations between one motif and all of the others.
And in that context, the first and the third pattern, although they look entirely different, are really exactly the same sort of pattern.
So let's begin to analyze what sort of operations are in these patterns that take one motif-- and obviously, they're all the same-- and relate it to all of the others.
First of all, there is an operation which I'll call translation for obvious reasons.
And I'll represent that by a vector, T, since a translation has magnitude and direction but no unique origin.
I could take this pair of objects sitting nose to nose, pick them up, slide them over by T, put them down again.
And I have the relation that gives me this neighboring pair.
Pick it up again, move it to the right by the same translation in the same direction, put it down again.
And I've got this pair.
So that is one operation that can exist in patterns.
This is the operation of translation.
So let me call that by a vector relation.
And it has magnitude.
It has direction, but no unique origin, just like a plain old vector.
So in other words, I can't say that the translation moves us from here to here or from here to here.
It's all the same thing-- magnitude and direction, no unique origin.
In fact, all of these patterns have translational periodicity.
There's a translation in this bottom pattern and another translation from here to here in the middle pattern.
The thing that makes a crystal a crystal is that it is an arrangement of atoms or molecules which is related one part to another by the operation of translation.
If you don't have translational periodicity, you do not have a crystal.
So that comes to the essence of what crystallography is about.
You can imagine, in one sense, the generation of this pattern by a rubber stamp sort of operation.
Suppose I have a rubber stamp.
And I put on the rubber stamp the pair of motifs like this.
Pick it up, move it over, chunk.
Pick it up, move it over, chunk.
And I can stamp out the pattern in that fashion.
Notice that my statement about no unique origin in these terms can be stated that it doesn't matter where the two motifs are on the stamp.
they could be up in the upper left hand corner, right in the middle, down in the bottom.
As long as I move the stamp through the same distance and the same direction, I get the same pattern.
Now, that's not bad for an introduction.
But I want to be more general than this because when I deal in terms of a rubber stamp operation, that is a transformation that involves taking one little chunk of a two dimensional space, picking it up, and putting it down in another location to another unique location in space.
So I'm going to now make another generalization that operations, which we've begun to define, act on all of space.
So I don't want you to think of this repetition in terms of a rubber stamp, although we could get the pattern that way and it's conceptually appealing.
But I'm going to say now that this string of motifs has translational periodicity if, when I pick it up, move it by T in a particular direction, and drop the whole infinite chain back down again, it is mapped into congruence with itself.
Which leads me to another definition-- an object or a space possesses symmetry when there is an operation or a set of operations that maps it into congruence with itself.
In other words, in plain words, you can't tell that it's been moved.
OK, is there anything else that is a transformation which leaves the set invariant?
OK, if we look at the first pattern, there are [?
rho ?]
sides such as this one here, or this one here, or this one here, about which I can rotate one motif into its neighbor or, for that matter, pick up the entire chain and flip it end over elbow through 180 degrees.
And it will be mapped into coincidence with itself.
And that is an operation, and another sort of distinct operation of transformation.
And this is one that I could call rotation for obvious reasons.
And there are two things I have to tell you about a rotation operation.
The first one is the point about which the rotation takes place, and that's going to be some point.
And let me call this point here A.
So this will be some labelled point that is the location of the rotation axis.
But then, the other thing that I have to tell you is the angle through which I'm going to rotate.
And I'll append to the A as a subscript the angle of rotation.
So this particular operation, called a twofold rotation because it rotates through half of a circle, would be the operation A pi.
This point is A.
We rotate through an angle pi.
This pattern here has also rotational symmetry.
In addition to the translation, there is a rotation operation, A pi, in the lower pattern.
So the follow who is unfortunate enough not to have a seat-- and I should have given you this one a long time ago.
I'll give that to you as your reward for giving the best answer.
And you get a seat wherever you would like to place it.
The first and the final pattern are the same in the sense that they contain two operations, translation and rotation.
This pattern is a much more interesting one.
This also has a rotational symmetry, A pi.
It also is based on a translation.
But now, there's another operation that we can do to leave the pattern invariant.
There exists [?
rho sides ?]
that pass through the center of this rectangular figure across which I could flip an individual motif, or for that matter the entire pattern, from left to right.
It's a reflection sort of operation.
So this is a new type of transformation.
So we'll add that to our list.
And the symbol that's usually used to indicate the locus of this operation is m, standing from mirror.
And that does it for these particular patterns.
Three sorts of operations-- translation, rotation, and reflection.
And in fact, that is all you can have in a two dimensional space-- not necessarily a rotation that's restricted to 180 degrees.
If these patterns are translational periodic in more than one direction, you can have higher symmetries.
One of the things I would like to suggest to you is that you look around you in everyday life at the sort of patterns that enrich your environment.
I see a one dimensionally periodic pattern there, the black and white stripes.
It's translationally periodic, going up and down.
It also has mirror planes running through the black stripes and the white stripes.
I see another two dimensional pattern back there.
That has translation.
But you could rotate-- no, you can't do anything.
That just has translation, nothing else.
Get a new shirt.
That's not terribly interesting.
There's another one there that's so complex I don't think I can look at it without climbing all over him and drawing some translational vectors and things like that.
But that's a nice periodic pattern.
That's a good one.
But there's lots of stuff like that.
Look at the grills in the ventilators.
They have mirror planes.
They are translationally periodic in one direction.
We've got floor tiles.
These are lovely because these have examples of 90 degree rotational symmetry.
Same is true of the tiles up in the ceiling, same sort of pattern.
And there's another pattern with four-fold symmetry in the grills that are underneath the fluorescent fixtures.
So symmetry is everywhere.
It surrounds us.
We wear it.
We walk on it.
We sit on it.
And think how much richer your life will be when you can understand this part of your environment.
Hey, that's a good, chauvinistic note, overstated, on which to end.
So why don't we take our break?
I'll hang around if you have any questions or get you a copy of anything that came around that you missed getting one of.
And it is now, according to my Timex watch, about three minutes before the hour.
So let's take a break and stretch for 10 minutes.
Ya'll come back because I've got your name's on a list.
OK, let us resume.
I had no idea how many people would be here today, and I think I made 25 copies of the handout.
And I see 25 names on the list.
And that means that two people did not get a copy of the syllabus.
Does anybody need a copy?
That's strange.
OK. All right.
We covered some introductory material, and I think we've covered enough that you can do a problem set.
So it gives me great pleasure to hand out problem set number one.
OK, you can think about that.
It is the sort of problem that will either take you two minutes or two hours or infinity.
So don't spend too much time on it, but I would like you to put your name on it and turn it in either at the end of the hour or next time so I can make comments if there's something that's mostly right but not quite right.
Let's return to these three simple patterns that we put on the blackboard.
And let me make another point about symmetry.
The people who sensed that this pattern and the one on the bottom were the same because they had the same motif in them, that they had the same rectangle with one concave side.
And I drew a mirror line in here because that locus, when I review this is a reflection from left to right, left the motif, as well as the entire pattern, unchanged upon making that transformation.
Is there not also a mirror line there?
Worked for this motif.
Why not for this motif?
Well, the answer is no, that this is not a mirror line because the symmetry transformations acts on everything, and not just one little bit of space.
And if I would take this chain of objects that's translationally periodic with a translation running this way, and I reflected that, I should have another chain running like this.
So the direction of the translation vector is not left invariant by this reflection.
So the conclusion here, and it's a subtle one, matter of definition, almost, is that the transformation, the symmetry transformation, if it's to be a symmetry transformation, acts on all of the space, and not just on one local domain.
So let me give you an example of a pattern that doesn't involve translational periodicity.
So let me try to make a star as carefully as I can.
What sort of symmetric does that have, or would it have had I drawn it more perfectly?
Well, that would be a five-fold rotation axis in the middle because I could rotate through one fifth of the circle.
And any of those rotations twice or three times, or just 2 pi over five, would be something that maps the pattern into congruence with itself.
There are also mirror lines that go from one tip of the star to the other end.
So that is an example of a pattern, non-periodic, but one that has five-fold rotational symmetry and mirror symmetry.
Now, if I put that star in a box and ask, what is the symmetry of that space?
There's only one operation which is common to the star and to the enclosing rectangle, and that's this mirror plane.
So the symmetry acts not just on one little part of the space, but it has to leave everything invariant.
So in that sense, going to this pattern here, this is not a mirror plane because it doesn't leave the entire pattern invariant.
That plane would reflect this one up to here, and we don't have anything there.
So the space is not left invariant.
One further definition.
We defined what we mean when we say a space or an object has symmetry.
We said an object or a space possesses symmetry when there is an operation, or set of operations, that maps the space or the object into congruence with itself.
Let me make another definition, and that a symmetry element-- another bit of terminology-- is the locus of points that's left unmoved by the operation.
Left unmoved or left invariant.
So for some specific examples, this vertical line is a locus which is left invariant by either the five-fold rotation or any of the mirror planes passing through the points that would be true of the star.
So for the star, these are all symmetry elements.
For the net combination of the star and the enclosing rectangle, the only thing that leaves a space invariant is this line.
The locus that's left and moved is this line, so we refer to that as a mirror plane.
Now, these may be seeming kind of definitions.
Nice to have, but what use are they?
We will use some of these definitions to answer a question which may seem tricky.
If we do a couple of things in sequence, for example, what is the net consequence of doing, let's say, a rotation combined with a reflection?
You can answer that question by saying, what has been left unmoved?
And that is the locuses of whatever net transformation results from a combination of two or more.
So that, again, is abstract, but we'll use that later on.
Any question on this?
Let me summarize very quickly what we have found for two dimensions.
We found that there are, in a two-dimensional space, three kinds of operations.
There is the operation of translation, which we'll call by the vector, T, corresponding to that transformation.
And there is an operation of reflection.
And the locus of the plane, in which the reflection occurs, we'll call a mirror plane.
And that's a linear locus.
And then, we've seen in these two patterns here an operation of rotation.
In particular, in these two-dimensional patterns, we saw the rotation operation A pi.
Now, I would put forth for your consideration something that is a profound conclusion.
These are the only single-step transformations that can exist in a two-dimensional space.
These are the only ones that can exist as single-step operations.
We can view these as operations that result in a transformation of coordinates.
And in two dimensions, if we have some position, x, y, in the space, what the transformation of a translation does is to take x and add a constant to it.
It takes y and adds, perhaps, a different constant to it.
Do the operation a second time, and we'll go to x plus 2a and y plus 2b.
So analytically, we can look at these symmetry operations in terms of the transformation of a representative coordinate.
If we have a reflection plane, and let's set up a coordinate system where this is y and this is x.
We have an object here at the location x, y, and we reflect it across this locus.
It goes to minus x, y, if the mirror line runs through the origin and is perpendicular to x.
So one example of a transformation by reflection is that x, y goes to minus x, y.
And this is a case where the mirror plane is perpendicular to x and passes through the origin.
If we do the operation a second time, minus x, y would get mapped back into x, y again.
It comes back to where it started from.
So this would be the first reflection.
This would be the second reflection.
And we saw in the patterns an example of one other transformation.
Let's suppose there was an operation, A pi, at the origin, and this was x, and this was y.
We started out with a motif here at x, y.
If we rotated that by 180 degrees, it would go down to a location minus x, minus y.
So the operation of a 180-degree rotation is going to analytically correspond to a transformation of going from x, y to minus x, minus y.
If we perform it again, it would go back to x, y. OK, let's look at this in more general terms.
In a two-dimensional space, we've got two dimensions to diddle with.
We can change the sense of no coordinate.
That's translation.
We can change the sense of one coordinate.
That's going to be reflection.
We can change the sense of both coordinates.
That's going to be a rotation.
That's all we can do.
So these are the three basic operations in a two-dimensional space.
I gave you special cases to make things easy, but regardless of where the mirror plane is, parallel to or perpendicular to an axis or not, and whether it passes through the origin or not, a mirror plane has the operation of reversing the sense of one direction.
Just the sense of one direction that is reversed.
And the rotation, be it a rotation through 60 degrees or 90 degrees or 180 degrees, is always taking both coordinates.
It's making a transformation of both coordinates.
If you only have two coordinates with which to play, that's all you can do.
Let's do some giant extrapolations.
If we have a strictly one-dimensional pattern where there's x and nothing else, than they're only going to be two coordinates.
And there are going to be only two ways we can transform coordinates.
So in a one-dimensional space, we can change the sense of no coordinate, and that's going to be the operation of translation, or we can change the sense of one coordinate, and that's going to be the operation of reflection.
No rotation in a one-dimensional space.
Now, let's extrapolate in the other direction.
In a three-dimensional space, the sort that we're going to be concerned with when we want to describe the symmetry of real crystals, you've got three coordinates to permute.
So it follows then, without saying what they are, in 3D, there are going to be four distinct one-step operations.
And then five dimensions?
Hey, that's a nice thing about mathematics.
You could play any game you like.
Not only that, but you make up the rules.
In a five-dimensional space, there's going to be six transformations.
Would we ever want to worry about five-dimensional crystallography?
Well, let me hang out a teaser and not answer the question.
Yeah, there are crystals for which as many as six-dimensional symmetries are necessary.
Wow.
Doesn't that blow the mind?
We'll return to that, and I'll explain why later on.
Another thing you might ask, why did I sneak this in?
Why did I say one-step operation?
Well, it's something we should worry about, and unfortunately, we will.
What if you take a motif, translate it, rotate it around a couple of times, reflect it, bounce it up and down three times, and then put it down?
How do you get from the first motif to the final one there?
Is there an infinite number of operations?
Mercifully, no.
The number is small and very finite.
And we will systematically, in another week's time, examine specifically two-step operations.
And as with many things that we'll encounter, we might not be clever enough to think them up.
But when we start putting things together into a synthesis, suddenly we're going to stumble over something we don't know how to explain, and we will have arrived, like it or not, at a new feature which we perhaps hadn't anticipated.
OK, any question at this point?
All this has been in a way of general introduction.
We're going to now take things more slowly and proceed one step at a time.
I would like to confine our attention for the moment on two-dimensional symmetries and examine the sorts of patterns that can exist in two dimensions, fabric patterns, floor tile, grillwork, and so on.
And we've seen that, basically, there seem to be three operations, three kinds of operations, translation, reflection, and rotation.
That's an infinite number of operations because we are not specifying whether or not the rotation angle is restricted to any particular value.
No reason why it should be.
There are lots of rotational symmetries that are absolutely lovely.
But let's build things up.
And I would like to first look at the operation of translation, which we've said a great deal about to this point.
Translation has magnitude.
It has direction.
So it acts like a vector.
But just like a vector, it has no unique origin.
Perform the operation twice, and you have a position that is two translations removed from the origin.
Do it three times, you have a translation that's three times out.
If a motif sits here, the motif must sit at the end of this translation in the same orientation parallel to itself.
It must exist at the end of two translations.
And if the operation acts on all of the space, if we say a translation is present, we really imply that there's an infinite row going to plus infinity and back to minus infinity.
And there is a motif hanging at the terminal point of every vector.
Now, we can summarize this periodicity with a convenient device.
Let's take some fiducial point and summarize the translational periodicity by saying that something that is hung at one point, either here, or maybe hung also off in some other direction relative to the translation, that something hung on one of these points is automatically reproduced for us at every point.
So what we have done through this device is defined something that is called a lattice point.
And this is an abstraction of the translational periodicity.
There is an array of points, geometric fictions, which we have constructed.
And we ascribe to this geometric fiction the property that anything hung at one of these points, be it a benzene ring or be it a Santa Claus on Christmas wrapping paper, is understood to be automatically reproduced at every other one of these points.
It is this array of fictitious points that is the proper designation of what we refer to as a lattice.
So a lattice is an array of fictitious points that summarizes the translational periodicity of the crystal.
It has a property to repeat that something hung at a particular disposition relative to that point and with a particular orientation is understood to be hung at every other lattice point in exactly the same way.
So that is a lattice.
And this is one of the most abused terms in crystallography.
We talk about the sodium chloride lattice.
The sodium chloride lattice is a set of points that are placed at the corners of a cube and in the middle of all the faces.
This is the NaCl lattice.
If I choose to decorate that lattice with one sodium and one chlorine, then I have atoms sitting at these lattice points.
And that is the sodium chloride structure.
That is the proper term for the atomic configuration.
So lattice is a geometrical term, and it's an abstraction.
Structure is the actual atomic arrangement.
Now, since I realize already that I am among friends, I can confess that I very often recklessly abuse the term lattice.
If I talk about lattice energy, lattice diffusion, lattice vibration, I'm not talking about abstract points bobbling around or something going through this array of little points.
I mean, I should talk about structure diffusion, structure energy, structure vibration.
But man, that just doesn't have the established terminology, and it doesn't have the zing and music of something like lattice vibrations.
So I do it all the time.
Don't tell anybody else that I said this to you, frankly.
But it's never going to be stamped out.
But now you perhaps are informed enough to at least blush slightly when you talk about lattice energy or lattice diffusion, realizing you're using the term incorrectly and that you know better, but everybody else does it, so you do the same thing.
So that is the definition of lattice.
Now, suppose I take this space, to which I've added a first translation, and I'll call it T1, implying that I'm going to add something else to this space, which I'm free to do.
I can put in a second translational periodicity because this is a two-dimensional space.
How do I do this?
And the answer is very carefully because the second translation could not go in the space parallel to the first one if I put in a second translation, T2, which is totally incommensurate with T1.
The things blow up in my face.
I don't have a lattice.
I will get lattice points all over the place.
So this is impossible.
So if T1 is not equal to T2, this space self destructs.
If T1 is a multiple of T2, then if I say a translation exists of length T1, and I add a second translation twice T1.
I've already got those lattice points.
And that's nothing new.
So if I want to say there's a second translational periodicity in the space, the only thing I can do is pick a T2 which is not parallel to T1.
And then this T2 will pick up everything in the space.
It's going to take these lattice points and generate them at equal intervals, T2.
But for that matter, it acts on everything in the space.
So we could think of this translation, T2, as moving this entire infinite string of lattice points separated by T1 and giving me a whole string of lattice points.
So now, having taken two noncollinear translations, those translations will imply a two-dimensional space lattice in which motifs will be hung at translations nT1 plus mT2 where n, m are integers that go from minus infinity to plus infinity.
OK, so this is a two-dimensional space lattice, or sometimes it's referred to by the term a lattice net.
Good term.
It looks like what fishermen throw in the water to snag fish.
So it is a net, in terms of something that we're familiar with in everyday life.
All right.
So we've specified a space lattice, but it is a highly redundant pattern.
We've got a doubly infinite set of lattice points.
And the unique nature of the pattern, the structure, is going to be whatever is associated with one lattice point.
So if we specify what's going on in the vicinity of one lattice point and establish that at every other lattice point, we have the entire infinite two-dimensional structure.
So let's ask now, how we can define the area that is unique to one lattice point.
And there are several ways of doing this.
We can specify T1, and then specify T2.
We'll repeat T1 up to here.
T1 will repeat to T2 over to here, and we will have defined the area that is uniquely associated with one lattice point.
So if I can tell you what's going on within the confines of this parallelogram, then I have given you the unique part of what is hung at a lattice point, and which is reproduced only by translation.
And this is a very important construct.
It is something that is referred to as the unit cell, or sometimes just cell for short.
And now we encounter a curious ambiguity.
T1 and T2 imply an array of lattice points.
And this particular choice of T1 and T2 define a cell.
But the reverse is not true.
If I give you-- and what do I want to say?
That a particular lattice does not specify a unique unit cell.
Or, stated another way, there are many different choices for T1 and T2 that would specify the same unique area.
I could take this as a T1 prime, and then I would have a cell that looks like this.
And that would also define the area associated with one lattice point.
It's not clear this oblique thing with one very long T1 prime would have very much to commend it, but there are many ways, many choices, for T1 and T2, to find exactly the same lattice.
We could take this as T1, this as T2, same lattice, same array of lattice points.
Take this as T1, this as T2, same array of lattice points.
Take this as T1, this as T2, that's yet another choice.
So there are an infinite number of translations.
Special name for this, to introduce a bit of jargon again, these are very often called conjugate translations.
So all this is still nothing more than simple geometry, but if you invent some fancy words, you really have to do that to impress your friends.
Yeah, you had a question here?
Yeah.
So you can define magnitude for T1 and T2, all those constants.
But you're changing the directions of T1 and T2, and you're saying, even though you're changing those directions, it's still the same unit cell?
Yeah, provided I have some new translation like this one here, which is really this T1 plus this T2, this would define a very, very oblique cell that looks like this.
But yet, the terminal points of T1 prime and-- I need a term for this.
I'll call this T2 prime.
The terminal points here are going to be exactly the same as the nodes that are defined here.
So they are two choices for one in the same lattice.
OK, so the implication of this is we're going to have to have rules.
And some of these make common sense.
You could pick, in a two-dimensional lattice, some absolutely ridiculous unit cells defined in terms of very long vectors that define a cell that is a very, very oblique cell.
So it's the lattice that's defined by this translation here.
And the next translation parallel to this one would go way up to something like this.
So there's a T1.
There's a T2.
This crazy cell here works.
That's the area that's associated with one lattice point.
But clearly, it has absolutely nothing to commend this choice.
There's nothing to be gained by using these long translations that make very extreme intertranslation angles.
Your intuition would say, why would you want to do that, you dummy?
Let's take these as the translations, which is something I sort of naturally did all along.
And what are we doing?
We're picking the shortest translations.
So there's one very common sense rule.
Another rule, getting a little bit ahead of the game, but suppose I examine the lattice that describes the arrangement of four floor tiles.
If I take a lattice point right at the point of intersection of the joins between the tiles, that is a cell that is exactly square.
And it's exactly square because there's a four-fold axis in that pattern that leaves things invariant after a 90-degree rotation.
So if that's the nature of a lattice, if it in fact is constrained because of the symmetry that is there to have two translations identical in length, in fact, identical in every way.
Pick those as the choice of the cell to emphasize that special key feature of the lattice.
So a second row, which is a second and final one, is to pick a T1 and T2 that displays the symmetry, if any, of the lattice.
Which introduces us to a feature which we'll elaborate much more later on, that the translational periodicity and the symmetry of the lattice are two things that go together.
That the fact that there is translational symmetry drastically reduces the number of symmetries that you could have, the fact that there are symmetries possible for presence in a lattice restricts the number of different kinds of cells.
So these are two aspects of the pattern, the symmetry that's in it and its periodicity.
OK, but these are the only two rules that we really need to pick what's called the standard cell.
Take the shortest translations that are available to you, and pick translations that display the symmetry that may be present in the lattice.
Any questions or comments?
Any comments?
OK, I think I have time for one last major point of discussion.
And what we are going to embark on now is a process of synthesis, which will occupy us for a couple of weeks.
What I'm going to do is start with a translation.
And this defines an infinite string of lattice points.
Now, I know that in two dimensions, I have two kinds of symmetry operations that are present, either rotation or translation.
So now, I'm going to ask the question, what happens if I define a lattice, or at least one translation in a lattice, and now I add to that lattice an operation of rotation, OK?
I can do that.
We've seen examples of translationally periodic patterns that have rotational symmetry.
So let me suppose I add to this space a rotation operation, A alpha.
And there's no unique origin to the translation.
There is no unique location for a lattice point.
So I can put the operation A alpha in at my designated lattice point.
Now, if I do that, all hell breaks loose because now I have a rotation operation A alpha.
This has a translation coming out of it.
That translation will be repeated up here, an angle alpha away.
A alpha acts on everything, so it's going to take this translation and move it over here to a location for another translation.
This is a lattice point.
This is a lattice point.
And this business is going to go on until it comes around full circle.
Let me focus my attention on just one of these translations, and this will be this one up here, the one that is alpha away from the first in a counterclockwise direction.
So here sits another translation, and that means this is a lattice point.
At this end of the translation, the same thing is going to happen.
The operation A alpha is moved to this location at the end of the translation.
That means that anything coming out of this lattice point must also be repeated at angular intervals, alpha.
And now I'm going to focus my attention on this translation here.
And there will be a translation that goes up like this, and this is a lattice point.
And now, in the words of that famous musical, there's big trouble in River City.
Because we started out by saying that everything in the space was periodic at an interval, T, a translational interval, T. This is T. This is T. This is T. Here we have a lattice point.
This jolly well has to be T as well, or we've contradicted the basic assumption of our construction.
Well, that's over restrictive.
This doesn't have to be T, but it has to be some multiple, p, of that translation.
p could be 0. p could be 5.
But it has to be an integral number of translations because this translational periodicity has to work everywhere, including up on the top of this trapezohedron.
So that's a constraint.
This angle is alpha.
We cannot let alpha be arbitrary because the only way we can add a rotation operation A alpha to a lattice is for a value of alpha which makes this translation be a multiple of the original one.
Now, let me take this geometry, and I'm going to extract the basic constraint from it.
This is some integer, p times T. This is T. This is T. This is T. This is alpha.
Let me lickety split drop down a perpendicular to the original translation.
This is T times the cosine of alpha.
This is T times the cosine of alpha.
This total length is T. This length in here is p times T. And now I can go away from the geometry to an equation, something you probably prefer to deal with.
And what this constraint is expressed analytically is that my original translation, T, minus twice T times the cosine of alpha has to come out equal to an integer, p times T. And there's my constraint.
Alpha has to satisfy that condition.
Well, I can immediately cancel the T and write this as one minus 2 cosine of alpha is equal to an integer, p. And it figures that that has to be the case because none of this construction depends on the size of the original translation that I took.
And now, let me solve for the values of alpha which are compatible with a lattice.
This says that cosine of alpha is 1 minus p over 2.
And unless that condition holds, my combination is incompatible.
So I'm going to let that stew with you until next time.
But what we've set up is something where we can just plug and chug, put in different values of p. And if I start out with a value of p, and let's let p be equal to 4, and then find one minus p over 2, which is supposedly the cosine of an angle, alpha.
If that's 4, I will have minus 3/2.
And the value of alpha obviously does not exist.
Cosine of alpha cannot get greater than 1.
If p is equal to 3, then 1 minus 3 over 2 is minus 2 over 2, or minus 1.
You like the way I do that arithmetic in my head just like that?
And the angle whose cosine is minus 1 is 180 degrees.
And what that says is that a two-fold axis works.
So I can drop a two-fold axis into a net.
And what that's going to do is take my original translation, rotate it 180 degrees, and the second translation is going to sit here.
Rotate it 180 degrees in the reverse direction, and then the second lattice point sits here.
And lo and behold, just as advertised, the distance between the first lattice point and the final lattice point is three translations.
So I can put a two-fold axis in any lattice whatsoever because this is compatible simply with a lattice row.
So one possible combination of rotation in a lattice is going to be any lattice whatsoever, and what we can add to this is a rotation operation A pi.
And we'll have two full rotation operations which are translationally equivalent.
All right.
Several integers to go.
We would want to try p equals 2.
That's going to work.
p equals plus 1 is going to work.
p equals 0 is going to work.
And we will find a very limited number of rotational operations that are compatible with a lattice.
And this is going to give us a small number of the possible combinations of lattice and rotational symmetry in two dimensions.
So we'll pick up from there next time, and we'll very quickly determine the remaining possibilities and take a look at what the arrangement of symmetry elements look like in these lattices.
OK, once again, I have some extra copies of the syllabus if somebody did not get one.
And I'll have extra copies of the problem set.
OK, why don't we you get started again?
I can understand why there is an air of excitement in the room since tomorrow's a holiday.
But we still got two hours before the day is over.
Any questions on what we have done up to this point?
No, that's good.
Let me erase some of this art work then and ask why one would indulge in this rather bizarre exercise of taking the elements of something that represents a physical property and constructing a surface out of them.
Well, the name suggests that maybe these surfaces, be they hyperboloids of one or two sheets or imaginary ellipsoids, have something to say about the property that the tensor that formed the coefficient of these functions is doing.
So let me take the case of an ellipsoid.
That would be the case where all the coefficients are positive.
And let's let this be x1 and this x2.
And let me ask a first question.
If this is to be a well deserved function, does the surface transform in exactly the same way as the tensor elements?
In other words, if we take the tensor in one coordinate system and then change the coordinate system to a different coordinate system, are the coefficients in front of x1, x2, and x3 still the elements of the tensor in that new coordinate system?
So let me show you that this is, in fact, the case.
Suppose our original relation, sticking to conductivity, is that x sigma ij xi xj equals 1.
And then for some reason or another, we change axes.
So the new equation will be some different coefficients.
Because as we change axes, they're going to wink on and off, still the same surface, but now referred to different axes xi prime and xj prime still equal to 1.
And let me now use the reverse transformation to put xi prime in the terms of xi.
If we do that, we can say that xi prime is cli x sub l prime.
That's the reverse transformation.
So notice the inverted order of the direction cosine subscripts.
And then xj prime is going to be equal to cmj times x sub ep prime.
To find this, let's just rearrange these terms in a trivial fashion.
Then I will have cx sigma ij cil cmj times xl prime xm prime equals 1.
And what is this?
This is a summation of direction cosines where the first subscript goes with the subscript on sigma prime.
What did I do wrong?
This is mj.
Why is that not coming first?
[INAUDIBLE]
Well, what I would like to get this in the form of is a summation of sigma ij prime over two dummy indices l and m. What did I do wrong?
Ah, yes, right, right, right, right, this is li.
Thank you somebody said it.
And my nose was right in it.
And I didn't notice it.
OK, this is a summation of all the original elements sigma ij prime times dummy indices l and m. And this actually is sigma ijn.
So it's the same set of coefficients.
And that is a technicality, which probably you wouldn't want to worry about.
So anyway, the elements of the tensor transform in the same way that the equations for the surfaces transform.
So if you change coordinate system, the coefficients that you should use should be the elements that are in the transform tensor.
OK, now I'm going to ask a question.
Suppose I define some direction by a set of direction cosines li and I ask the value of the radius of the surface in that direction.
So this is some radius vector.
The radius vector will have components, R1, R2, R3 or Ri.
And each of those components will be equal to the magnitude of R times the direction cosines li.
Yes, sir?
Could you briefly rego over what you just stated before you erased it [INAUDIBLE]?
Oh, OK.
I'm sorry.
I said aij xi xj equals 1 is the original equation for the quadric.
If we change the coordinate system, we're going to have some new elements aij prime times xi prime xj prime.
Because we've got a new coordinate system.
And therefore, the coefficients in front of the equation for the quadric have to change.
And now what I did was to express xi prime and xj prime in terms of xi and xj using the reverse transformation.
And xi prime in terms of the original x's will be c sub l sub i x sub l prime.
Usually, we write it xl equals cil x sub l. This is the reverse transformation where the order of the subscripts is reversed.
So this will give me x of i prime.
The expression for x sub j prime will be given by cmj x sub j where m is a dummy index.
And that's equal to 1.
And if I just rearrange this, then I will have cli cmj times aij prime equals 1 times xl xj.
And this is going to be alx lxm.
And this will be alm times xl xm equals 1, which is the equation for the quadric in the original coordinate system.
I think that was better when I was standing in front of it so you couldn't see it.
It's just that the surface transforms formally to the surface that we would create if we used the new tensor elements for the same change of coordinates system, which you would probably will be willing to take my word for
[INAUDIBLE]
This is a direction cosine for the--
[INAUDIBLE] right below
Right below?
[INAUDIBLE]
cli cmj aij prime xl xm, and then I collected the cli cmj times alm.
And that is the transfer.
Yes?
In the previous [INAUDIBLE], you defined the derivitave side as being xi prime equals cil xl.
[INAUDIBLE]?
You can say that x prime is cil times x sub l. And if we want to use the same direction cosine scheme but do it in reverse, we would say that x sub l is equal to c-- yeah, OK, x sub i is cli times x sub i, right, no, times x sub l-- is that right-- x sub l prime
On the second line to the third line, you say that xi prime equals cli [INAUDIBLE].
It's either xi equals xcli xl prime or xi prime equals cil
OK, you want to say this.
xi prime and cil becomes xl, yes, yeah.
And then mj times x-- that's x sub m. OK, and then this says that cil cjm times aij prime should be alm, right, OK, OK.
So it's OK then.
OK, that was supposed to be a small point that everybody would accept.
But now it's done correctly.
OK, let's get back to this, which is more hair raising and exciting.
What is the radius of the quadric in a given direction that we specify by three direction cosines l1, l2, l3?
So there's some radius from the center of the quadric out to the surface in that particular direction specified by three direction cosines.
And we know what those three components are, sub i, are going to be in terms of the direction cosines, magnitude of R times li.
And those points at the terminus of the radius vector go to one point on the surface of the quadric.
So these values of R are coordinates that satisfy the surface of the quadric.
So we can say that just substitute Ri for the different values xi in the equation for the quadric.
And this says that sigma ij magnitude of R times li, that would correspond to x sub i times magnitude of R times l sub j.
That's the magnitude of xj.
That should be equal to 1.
And if I rearranged this slightly, this says that the magnitude of R squared is going to be equal to 1 over sigma ij times li lj.
So the radius of the quadric gives me not the value of the property in that direction.
This is the value of the property that will occur in the direction specified by the direction cosines lj.
The radius of the quadric is going to be equal to 1 over the square root of the value of the property in that direction.
So this is why it's called the representation quadric.
If you construct the surface from the tensor elements, you will have defined a quadratic form, which has the property that, as you go in different directions look at the distance out to the surface of the quadric in that direction, that that distance squared is going to be equal to 1 over the property in that direction or, alternatively, the radius is going to be 1 over the square root of the value of the property.
There is an enormous implication here.
This says that the value of the property, if the quadratic form is an ellipsoid, the value of the property as we go around in different directions in the crystal is going to be a smooth, uniformly varying function.
They're going to be no lobes sticking out.
They're going to be no dimples, no lumps.
It's going to be an almost monotonous property, not very interesting.
It's going to change in a uniform way.
And in fact, we could put the two surfaces side by side.
If this is the value of the quadric as a function of direction, if we make a polar plot of the property as a function of direction, it's going to look sort of like the reciprocal of this.
The minimum value of the property will be in the direction of the maximum value of the radius of the quadric.
And the maximum value of the property is going to go in the direction of the minimum value.
So the value of the property as a function of direction is going to be a quasi-ellipsoidal sort of variation but not really an ellipsoid.
It's going to go as this inverse square of the radii in ellipsoid.
But the thing is it's going to be something that varies uniformly between extreme values of the maximum and minimum value of the property.
We are, I assure you, for higher ranked tensor property going to look at some absolutely wild surfaces with surfaces that do have lobes and extreme values and very, very irregular variation of properties with directions but not for second ranked tensor properties.
There will be a few variations on this theme, which are also interesting to touch upon.
In principle, we can get other quadratic forms from the tensor if some of the elements of the tensor are negative.
And in particular, what would we see-- and I'll draw this relative to the principal axes of the surface-- what would we see for an hyperboloid of one sheet?
This is the sort of surface that might result if one principal value of the tensor had a negative value.
Well, this is the quadric.
And this is a radius in one particular direction.
What would this say about the property?
Well, let me, to make it clear, look at one of these sections through the quadric that our hyperbola.
In directions like this, we have a radius that is a minimum out to the surface.
In other directions, the radius gets progressively larger.
And then if we plot the reciprocal of the square root of that, that says that, in this direction, we get the maximum value of the property.
And as the direction approaches the asymptote, the reciprocal will go down to 0.
But how did I get two y-axes here?
OK, so the maximum will be in the direction of the minimum radius.
And then it will go down to 0 for the asymptote.
And that's going to be symmetrical on either side of this principle axis.
So what then are the radii in directions outside of the asymptotes of this hyperboloid?
Within this range, the radius is imaginary.
But if you square it, you get a negative number.
So within these two lobes, the value of the property is positive.
As you go away from the asymptotes, you'll get another lobe like this and another lobe like this where the value of the property is negative.
How in the world could you get anything that looked like that?
Well, a good example of a property that has this behavior is thermal expansion.
And let me give you two examples.
The structure of selenium and tellurium is a hexagonal structure.
And there are chain-like molecules that are pairs of bonds that rise up around a threefold screw axis.
So this is two coordinated atoms in the structure just spirals up in this triangular spiral around the threefold screw axis.
So this is a material in which the bonding is very, very anisotropic.
The bonding within these covalently bonded spirals, which are like springs that you might put on your screen door in the summertime, the bonding is very strong.
Between these individual molecular chains, the bonding is very weak.
So what happens when you heat this stuff up?
It expands like the dickens in the direction of these weak bonds.
But the spirals are, in part, held in this extended form by repulsive interactions in like chains.
So when these spirals move apart as a result of large thermal expansion in the plane normal to the chain, the chains relax a little bit.
So you have a large positive thermal expansion here.
But in one direction along the normal to the hexagonal in the structure, the thermal expansion is negative.
And that gives you a variation of property with direction that looks exactly like this.
Negative value of the property, that means the structure, contracts in that direction, positive values this way.
The structure expands.
So there's a good example of this.
There's another example of a material, which, again, has a negative thermal expansion in one direction.
And this is calcium carbonate, which is a complicated hexagonal structure.
But it looks very, very much like the structure of rock salt in a distorted form.
These are the calcium atoms.
And calcite is CaCL3 And the calciums are in a face-centered cubic arrangement just like in rock salt.
The carbonate groups are very tightly bonded little triangles with the carbon in the middle.
And these things are arranged on the edges of the cell.
And this is going to be very schematic.
These triangles are normal to the body diagonal of the cells.
So you have one family of triangles that are all parallel to one another.
On the next edge, the triangles are anti-parallel to this first orientation.
So think of rock salt.
Tip it up on its body diagonal.
Every place you have a chlorine anion, place a triangle in an orientation that is perpendicular to the body's diagonal of the rock salt structure.
Again, these tightly bonded little triangles don't do much as you increase temperature.
But the bonding between the calcium and the triangles is rather weak.
So again, you find that the structure expands in one direction so strongly that it contracts to the other direction so calcium carbonate.
The calcite form also has the distinction of having a negative thermal expansion coefficient.
We'll talk quite a bit about expansion coefficients as one example of a tensor property.
Interestingly, and we'll say more about this and I'll give you some references when we come to this point, can you get a material that has a negative thermal expansion coefficient in all directions?
No, that seems as though it would violate in a flagrant way some vast law of thermodynamics that things have to increase their volume when you increase temperature.
There were, however, a few odd ball materials that over a very, very limited temperature range would contract but only by a tiny amount in all directions.
And then one of the very interesting discoveries of recent years done primarily by a crystal chemist named Art Slate, who's out at the University of Oregon, he discovered a family of materials that have large negative expansion coefficients in all directions over a considerable range of temperatures, like 100 degrees.
So these are materials, when you heat them up, they contract amazing as that seems.
And there's a structural reason for it.
These are tetrahedral frameworks in which one corner of a tetrahedron is dangling.
And as you heat it up, this tetrahedral corner can get closer to other tetrahedra, which takes energy to do.
But in so doing, you actually are changing the net volume occupied by the solid so very, very anomalous set of compounds.
More of these have been found.
There are probably a dozen materials now that have negative thermal expansion coefficients in all directions.
So is the imaginary ellipsoid a viable representation quadric for real materials?
Yes, in the case of thermal expansion coefficients.
OK, so the representation quadric then is a dandy device for seeing with one function how a property will vary with direction.
For an ellipsodial quadric, the variation of the property itself with direction is not really ellipsodial but quasi-ellipsodial in that there's a small principal axis, a large principal axis.
And the large value of the property goes with the short axis of the quadric.
So we have a nice device for representing the value of the property as a function of direction.
It looks as though we've lost all information about the direction of the resulting vector, the generalized displacement.
It looks as though that's not in here at all.
Well, there was an ad years ago for a bottle spaghetti sauce, which offends many cooks who are very fiercely proud of their own spaghetti sauce.
So here's a wife who's using the canned stuff.
And her husband comes home and looks at it very, very skeptically and says, where's the basil?
And the housewife says, it's in there.
It's in there.
And he sniffs again and says, where is the basil?
And she says finally, it's in there.
It's in there.
Well, this is a similar situation.
Where is the direction of the generalized force?
And I say, it's in there.
It's in there.
Not obvious, but it's in there.
So let me now show you where it is lurking.
I think we have time to carry this through.
Let me describe how it's embodied in the quadric and then prove to you that this is indeed the case.
And I'll use a general quadric in the form of an ellipsoid.
That looks more like an egg than an ellipsoid.
It has a pointed end.
OK, this is something that's called the radius normal property.
And what it says, in words, is that, if you pick a particular direction relative to the quadric, we know that its length is going to be inversely proportional to the square root of the value of the property.
If you want to know what the direction of the resulting vector is, for example, in our relation for conductivity now getting very warm, it says the current flow is given by a linear combination of every component of the electric field.
The radius normal properties is that, if you want to know where J is for this particular direction, look at the point where the radius vector intersects the surface of the quadric and, at that point, construct a perpendicular to the surface, which, in general, will not be parallel to R. And this will be the direction of J.
It won't give you the magnitude.
But it'll give you the direction of it.
OK, let me now prove to you that the quadric does have this property.
In our conductivity relation, the direction of J is going to be given by the tensor relation.
Namely that J sub i is equal to sigma ij times E sub j, which can be written as sigma ij times the direction cosines of E sub j times the magnitude of E. I'm going to want to compare this expression, with which you're very familiar term by term, with the normal to the surface that we can compute from its derivative at that point.
So this will say that J1 is equal to sigma 1, 1 l1 times the magnitude of E plus sigma 1, 2 times l2 times the magnitude of E plus sigma 1, 3 times l3 times the magnitude of E. And I don't need to write much more.
J2 will be sigma 2, 1 times l1 magnitude of E plus sigma 2, 2 l2 times the magnitude of E and so on.
What is going to be the normal to the surface as a function of direction?
OK, to do this I'm going to say that the normal to the surface is going to be, if I have some function of xyz, the normal to that function is the gradient.
Let me say that that's G of xyz.
And we can say then that the x1 component of the normal is not going to be equal to but proportional to the gradient of the equation for the quadric with respect to x1, dx2, and dx3.
So it's going to be proportional to the differential of the function that gives us the surface with respect to x1 times i plus the differential of the function with respect to x2 times j plus dF dx3 times k. So this is the normal entirely.
So if we split this into components, N1 is going to be equal to dF dx1.
And if I differentiate the equation for the quadric, that's going to be 2 sigma 1, 1 times x1.
Remember the equation for the quadrant is sigma 1, 1 times x1 squared plus sigma 1, 2 times x1 x2.
So if I differentiate-- let me write it down.
If I take those terms and differentiate with respect to x1, I'm going to get 2 sigma 1, 1 times x1.
Here I'll get sigma 1, 2 times x2.
But then down in this line where I have a sigma 2, 1 x2 x1, if I differentiate with respect to x1, I'll have another term sigma 2, 1.
And if I differentiate with respect to x3, I'll have sigma 1, 3 plus sigma 3, 1 times x3.
And a similar thing for the x2 component of the normal, that's going to be proportional to the gradient with respect to x2.
And that's going to be equal to sigma 1, 2 plus sigma 2, 1 times x1 plus 2 sigma 2, 2 times x2 plus sigma 2, 3 plus sigma 3, 2 times x3 and similarly for N3.
OK, I've made my case if I can demonstrate that each component of J, J sub i, is proportional to each component of the gradient to the function that defines the quadric.
If you look at them, they're close but no cigar.
The first term is OK.
I've got a 2 out in front of sigma 1, 1 for the x1 term.
Here, though, I have the sum of two off-diagonal terms.
And for this expression, I have just sigma 1, 2.
And down here I've just the single term sigma 2, 1.
So the conclusion we're forced to draw is that these two expressions would have components of a vector that are parallel to one another if sigma 1, 2 was equal to sigma 2, 1.
Because if these were equal, I could write just twice sigma 1, 2.
And down here I could write twice sigma 2, 1 and do that all the way through these two expressions.
And then this factor of 2 would just be part of the proportionality constant.
So the radius normal property will be true or valid only for symmetric tensors.
That is it's going to be true only if sigma ij is equal to sigma ji.
We mentioned when we first started talking about second ranked tensors that a tensor does not have to be symmetric.
And showing that the tensor is symmetric has nothing to do with symmetry.
Because it's symmetric in an algebraic sense not in the literal sense of geometrical symmetry.
And there are some properties, the thermal electricity tensor is one notable example, where the tensor is a second ranked tensor but it decidedly is not symmetric.
So this is OK for most properties but not all.
OK, so here are properties of the representation surface that we can construct from the representation quadric.
And what I would like to do following this and after the inevitable unpleasantness of a quiz is to look at some specific properties that illustrate second ranked tensor properties.
And in particular, I would like to look at some other sorts of second ranked tensor properties that represent generalized forces, namely stress and strain.
You've seen these before in other contexts perhaps not actually defined rigorously in terms of tensors.
Because, obviously, stress and strain can be regarded as generalized forces.
A stress is something that you can apply to a solid.
And that will cause various things to happen.
In addition to mechanical behavior, different properties and effects can result.
So that is going to involve a generalized force, which is a second ranked tensor.
And a thing that might happen could be a vector.
It could be another second ranked tensor.
And this is going to introduce us to the nasty world of higher ranked tensor properties and their representation surfaces.
So this is a nice place to quit and pause for a quiz.
When we resume, we'll look at stress and strain in terms of our tensor algebra.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.MIT.edu
This, as you know, is the last lecture of thermal tensor properties of crystals for the semester.
Come on, can't somebody go, aww, just to make me feel good?
At least nobody said yay, so that makes me feel good, too.
So I guess on balance I come out feeling pretty happy.
What I'm going to do today is to talk a little bit more about forthright tensor properties, which we've not said much about.
And I'm going to look at some scalar moduli and look at them for a couple of different symmetries and restrictions on the compliances.
The surfaces that you might expect turn out to be really, really weird.
Because these will be fourth order variation with the direction cosines.
So when you take trigonometric functions and raise them to the fourth power, you get severe anisotropies for a great many of the scalar moduli.
I'm sorry to say, years ago when I made the acquaintance of a computer programmer in course six who was looking for something challenging to do in connection with material science and engineering, I said, wow, have I got something for you.
How about doing some computer graphics?
And this was a few years ago when such products were fairly rare.
And how about making a program that would let us see visually how some of these scalar moduli will vary with direction and provide the provision so you can turn them over in space, just as you would pick up a model in your hands and rotate it around till you finally came to appreciate it?
And then for something like a triclinic crystal, where there are almost a couple of dozen different moduli, how about allowing people to scale up or scale down the value of one of the elastic constants and let them see how the shape changes?
And oh, that was-- he went to work on it for a semester and came back.
And it was a gorgeous thing.
You could take a monoclinic representation of three dimensions of Young's modulus.
Say you want to change S31.
And it would change by a factor of 10.
You'd see a big nose blow out on the surface and then contract down again.
And you could turn around.
Anyway, the program having told you how marvelous it is, it's no longer supported because of the operating system that it operated under having done defunct.
But anyway, that was a fun thing to play with
What operating system?
I don't even remember anymore.
It was a fairly obscure one.
Was not only the system, but it worked through a software package which operated on that system.
All right.
So we're going to talk today about fourth rank tensor properties.
The most important-- in fact, probably the only ones you've ever heard of, are the elastic, stiffnesses, and compliances, which are ways of representing strain in terms of stress or stress in terms of strain.
I think I'll sort of see how the time plays out.
If I finish just about everything that I'm hoping to say by on the hour, I think I'll just go an extra five or 10 minutes.
And then comes a great, grand, and traditional event at MIT.
All semester long I've been grilling you with problem sets and quizzes, and you've had to dance through your steps just because I told you to.
Now how's this for reciprocity-- you get to evaluate me.
You get to grade me.
MIT, at the end of every semester, has a course evaluation.
It's submitted anonymously by you, so you could really vent your spleen without ever having called to task for it.
So when we finish, either for our break or at the end of a slightly extended first session, I will bow, to thunderous applause, I hope, and exit.
And then you will be left alone to fill out the course evaluation.
Corinne will collect them from you, and she will deliver them over to department headquarters.
The results will be tabulated, and so will the comments.
But they will not be in your hand, all of which I recognize, having spent hour after hour grading your quizzes.
So they will be submitted to me in a completely sanitized fashion.
I won't know who said what.
A final question that I will answer before you ask it, what about the quizzes?
How are you doing on them?
Well, I have come to hate them.
I think I will give one-question quizzes from now on the future.
It takes me about five to seven minutes to grade each question.
And there are about 25 of you.
So it takes me, to grade one question, somewhere between two and 1/2 and three hours.
And there are 10 questions on the two quizzes remaining to be graded.
And so I let you do the arithmetic, and you can see what sort of torment I've been living in the last few days in return for the torment I put you through for two brief hours.
So wow.
I will leave it to you to decide who's getting the worst of this deal.
OK, fourth rank tensors.
Elasticity's probably the only one you can think of.
Let me give you an example in this handout, an example of another forthright tensor property.
And this is the piezoresistive effect, the way electrical resistance of the material changes in response to an applied stress.
It's a pretty exotic property.
I had never heard of it until I was at a meeting of the American Crystallographic Association and somebody, believe it or not, from Texas Instruments-- so you better bet that this property is useful in technology-- fellow named Ahmed Amin got up and gave a talk on the piezoresistive effect.
And he had marvelous slides at the outset that defined the effect So what you have here on these few sheets is a few pages that define the piezoresistance effect.
We're not going to do anything with it.
It's a fourth rank tensor, so it will proceed to transform like other forthright tensors, such as elastic stiffness and compliance.
I should warn you about these following pages, which were slides that he used at his talk.
He seems to have done the lettering with a magic marker.
Because all of the I's and J's are indistinguishable, which makes interpretation of what he's saying here a little bit challenging.
And then the other thing that throws one off is that he uses x to represent stress.
I don't know what field or in what discipline the elements of stress are called xij, but this is what he does.
And once you figure that out, the interpretation of what he has said here is fairly apparent if you think about it.
Anyway, he takes a state of zero stress and strain and expands it as a series, and then picks off different coefficients.
And one of them is a set of moduli which he calls pi, pi subscript ijkl, which relate stress, xkl in his notation, to a change in the density, delta row ij.
So here's a fourth rank tensor property that will indeed have to conform to all the restrictions for the moduli that we have defined for the elastic stiffnesses and moduli.
All right.
Let me then turn to stress and strain.
We introduced the relation between stress and strain, but didn't really go into detail on the bizarre absorptions of factors of two or four that have to be done in order to make this come out in a nice, clean matrix form.
So let me remind you of where these silly factors came in.
We took our stress tensor, sigma 1 1, sigma 1 2, sigma 1 3, 2 1, 2 2, and 2 3, sigma 3 1, sigma 3 2, and sigma 3 3.
The tensor had to be symmetric.
The off-diagonal term sigma ij had to be equal to the term sigma ij.
And this was for the reason of mechanical equilibrium.
These off-diagonal terms, sigma ij and sigma ij, we saw, exerted a torque on a body.
And unless they were equal, the body would undergo an angular acceleration.
And then we renumbered these according to the convention that as we would march down the diagonal of the tensor and take pairs of subscripts, 1 1, 2 2, 3 3, represent them by a single 1, 2, and 3, then march up the side to define a sigma 4, sigma 5, and a sigma 6.
If you remember that little algorithm you can always keep straight what these reduced subscripts represent.
So with that definition, then, the stress tensor reduced to sigma 1, sigma 6, sigma 5, sigma 6, sigma 2, sigma 4, sigma 5, sigma 4, sigma 3.
And that takes a tensor and degrades it to a matrix.
Because there is no law of transformation for this representation of the elements of stress.
OK, then we do something very similar with the strain tensor, epsilon 1 1, 1 2, 1 3, epsilon 2 1, epsilon 2 2, epsilon 2 3, epsilon 3 1, epsilon 3 2, epsilon 3 3.
And we saw that for physical reasons this tensor also had to be symmetric.
We had to have epsilon ij identical to epsilon ji, the reason being that if these off-diagonal shear strains were not equal, the state that we would be defining was one of actual deformation combined with rigid body rotation.
So unless this was the case, the definition performed by the tensor epsilon iij would define rigid body notation as well.
OK. We use the same process of renumbering to convert this from a tensor to a matrix.
And we got this into a form, epsilon 1, epsilon 6, epsilon 5, epsilon 6, epsilon 2, epsilon 4, epsilon 5, epsilon 4, epsilon 3.
But in order to do that, we saw we had to consume a factor of two.
So there is a factor of two built into the off-diagonal terms.
And what we have put in here is we converted this to actually epsilon 1, 1/2, epsilon 6, 1/2, epsilon 5.
And I should have written that this way to begin with.
So all these factors of two appear so that when we combine these epsilons we get a nice, clean matrix relation between stress and strain that doesn't involve factors of two.
And that's a great convenience if we're going to be doing all of our deformation within the framework of one coordinate system.
OK.
So if we now use these definitions to write the stress in terms of strain, we will set up our fourth rank tensor property.
If we do this first for the stress in terms of strain, you would have a tensor element of stress sigma 1 1, which would be equal to C1 1 1 1 times epsilon 1 1 plus C1 1 2 2, epsilon 2 2 plus C1 1 3 3, epsilon 3 3.
And then we would have, in addition, off-diagonal elements of strain.
We'd have C1 1 2 3 times epsilon 2 3 plus C1 1 3 2 times epsilon 3 2 plus C1 1 2 1, epsilon 2 1 plus C1 1 1 2, epsilon 1 2 plus C1 1-- and I should have made this 1 3 so that the other integers come out right.
And C1 1 1 2 times an epsilon 1 1 1 2 plus a C 1 1 2 1 times an epsilon 2 1.
So that is one line of the relation between stress and strain.
And when we condense this to a matrix form, it is surprising, once we expand it once more, at how cumbersome this expression is.
Now, why do I remind you of all this?
Isn't it nice to work in the form of a fixed coordinate system where we can use a matrix for the Cijkl's?
The answer is, fine, unless you want to change the coordinate system.
And you might want to do that for practical reasons such as cutting out a specimen which makes it convenient to define the stiffnesses and compliances relative to a coordinate system taken along the logical directions in the specimen.
And the other reason for doing it, and that is what I want to do a little bit of this afternoon, is to derive the symmetry restrictions on the stiffnesses and compliances.
You cannot transform a matrix representation of the stiffness or the compliance.
You can only do this for the full-blown tensor arrangement.
So having collapsed down, which we'll do momentarily just to remind you of how it goes, we will have to expand again to derive symmetry restrictions.
Do not exit at this point.
We're not going to do any of these calculations in their full glory detail.
We'll just set up the problem and then jump immediately to the outset.
So how does this pay out if we try to make the condensation?
I remind you again that the symbol C stands for stiffness, and the symbol S, which we'll see later, the Sijkl's are call compliances.
And the perversity of that semantic description of these tensor elements is perverse for reasons that I have never really been able to understand.
So if we go down to matrix form, we'll write instead of sigma 1 1, simply sigma 1 1.
We'll call this C1 1 times epsilon 1 plus C1 2 times epsilon 2.
So far so good.
C1 3 times epsilon 3.
And now we have problems, because epsilon 2 3 was defined as 1/2 of epsilon 5.
Now we'll have a C1 5 here, and then a C1 3 2 is C1 5 again.
And this also is 1/2 of epsilon 5.
And then similarly, this is C1-- I'm sorry.
This is C1 4.
This is C1 5 times 1/2 of epsilon 5 plus C1 5 times 1/2 of epsilon 5 plus C1 6 times 1/2 of epsilon 6 plus C1 6 times 1/2 of epsilon 6.
So this, given the way in which we had defined matrix strain, initially in connection with piezoelectricity when we entered the realm of third rank tensors, this will play out OK.
This says that sigma 1 is simply C1 epsilon 1 plus C1 2 times epsilon 2 plus C1 3 times epsilon 3.
And now the 1/2 gets absorbed, and it's simply C1 4 times epsilon 4, and so on.
If we look at another line, one that involves some of the off-diagonal terms and strain, if we, for example, look at sigma 2 3 and we write this down as sigma 2 3 times 1 1 times epsilon 1 1 plus C2 3 2 2 times epsilon 2 2 plus C2 3 3 3 times epsilon 3 3 plus C-- and I'll just write down a few of these additional terms.
This would be C2 3 3 2 times epsilon 3 2 plus C2 3 2 3, epsilon 2 3, and then other terms for additional terms, and epsilon ij with i not equal to j, which are going to behave the same way.
So if we convert this, we go to sigma, and we would call this sigma 4.
This would be C4 1 in matrix notation times epsilon 1 plus C4 2 time epsilon 2 2 plus C4 3 times epsilon 3.
And now this matrix element would be C4 4.
And in place of the tensor element of strain epsilon 3 2, we would write 1/2 of epsilon 4.
And the next term is again a C4 4 times a 1/2 epsilon 4 and the terms in epsilon 5 and epsilon 6 would behave the same way.
So you can see that the factor 2 is absorbed, and this becomes simply C4 4 times epsilon 4.
So all the factors of 2 have disappeared.
And we can say, provided we remember the way we have condensed the elements of tensor strain, we can say with complete confidence that in our matrix notation, sigma i is Cij times epsilon j where i goes 1, 2, and 3, and j goes 1, 2, 3 all the way up to six
[INAUDIBLE]
No.
No, no, these are the-- yeah, I'm sorry.
Yeah, inj.
So it's a six by six matrix, right.
Right you are.
And this is a six by six.
And in here are 36 businesses stiffnesses.
OK, any comments other than, yuck?
And again, if you stay in one coordinate system it's not so bad.
In fact, instead of having nine by nine 81 terms you have 36 stiffnesses.
And that's a great convenience.
Unfortunately, things are not quite so simple if we work with the compliances represented by the symbol S. And if we attempt to write strain in terms of stress, we'll have an S1 1 1 1 times sigma 1.
We'll have an S1 1 2 2 times sigma 2 2 plus an S1 1 3 3 times a sigma 3 3 plus an S1 1 2 3 times a sigma 2 3 plus an S1 1 3 2 times a sigma 3 2, and an S1 1 2 3 3 1 times sigma 3 1 plus an S1 1 3 times a sigma 1 3, and then other off-diagonal terms which I won't bother to mention.
So if we convert this to matrix form, epsilon 1 1 would be replaced by the matrix term epsilon 1.
S1 1 1 1 would become S1 1, and this would be sigma 1, and an S1 2 times a sigma 2, plus and S1 3 times sigma 3.
And now we have a problem, Houston.
Because we would have an S1 4 times a sigma 4 plus an S1 4 times sigma 4.
And now in the sigmas there is no factor 1/2 in the matrix representation of stress as there was with strain.
Because we ate the factor of 2 in defining strain.
But now there is no factor of 2 and stress.
So we're going to have to do something with that.
So this would give us some additional terms, S1 5 times sigma 5, and then an S1 5 times a sigma 5 again.
So what are we going to do?
Again, we're stuck.
We can either say that epsilon i is equal to Sij times sigma j if j is not equal to 4, 5, or 6.
Or we can absorb, again, a factor of 2 in the definition of the compliances.
And that, again, since we will usually be working in one, and the same coordinate system, is the convenient thing to do.
So what we will do is to define this as S1 4 times sigma 4.
And in so doing, we have to combine the factor of 2 into the definition of S1 4.
So S1 4 would be equal to 1/2 of S1 1 2 3 plus S1 1 3 2.
So it's going to be 1/2 of one of these equal terms that involve one on-diagonal subscript and one off-diagonal subscript.
So this is the only way we're going to be able to write a nice, clean matrix representation.
Things get worse
Sir?
Yes?
You're sure of your [INAUDIBLE]?
Yeah, this would be equal to 2S1 4.
I'm sure it's 1/2.
I'm not sure of what term would go in front.
So I would have 2S1 4 times sigma 4.
And if I want to write this-- I'm sorry.
Let me go back to the full matrix expression.
I would have S1 1 2 3 times sigma 2 3 plus S1 1 3 2 times sigma 3 2.
I know that sigma 2 3 is equal to sigma 3 2, so I could write this as S1 1 2 3 plus S1 1 3 2 just simply times sigma 2 3, one of them.
Now what I would really like to do is to write this as S1.
Let's change this to sigma 4.
I would like to write this as S1 4.
So it follows then that S1 4 is equal to S1 2 3 plus S1 1 3 2.
And that is the other way around, isn't it?
Thank you.
I knew there was a factor of 1/2, but it goes in here
And then there's no 2 at all.
If you wanted to say that S1 1 2 3 is equal to S, 1 1 3 2, then you would have S1 4 equal to 2S1 1 3 2
OK, you're right, you're right.
So it's this.
Let's leave it at that.
But if these are equal, we could say-- and we did show that the compliance tensor is symmetric, so we could say that this is equal to 2S1 1 2 3 or 2S1 1 3 2 as they're equal.
I told you it was going to get bad, but I didn't think I would be contributing to it the extent that I am.
I don't know if you really want to see this, but this gets even worse when we deal with something like epsilon 2 3, where this is a term that would be replaced by epsilon 4.
And then all of our absorbed factors come back to haunt us.
Let me go through this quickly.
This would be S2 3 1 1 times sigma 1 1 plus S2 3 2 2 times sigma 2 2 plus S2 3 3 3 times sigma 3 3.
And then we'll have these terms, off-diagonal terms S2 3 3 1, sigma 3 1 plus S2 3 1 3 times sigma 1 3, and so on, other terms.
OK, and going from epsilon 2 3 to epsilon 4, we have to put on a factor 1/2.
And that says that that 1/2 is going to create problems in the first term in this expansion.
We'd call this sigma 1 1.
We'd like to call this S4 1.
But what is S4 1 from what we have defined here?
It's the sum of two tensor elements.
So we will have to define, now, again, S4 1 equals the same as we did before.
It's going to be equal to the term S2 3 1 1 plus S3 2 1 1 1.
And therefore if I put a 1/2 in this definition, then that will cancel, so far.
But then in the terms that come down in the lower quadrant of the matrix-- and I will just right in two terms.
We've got an S2 3 31 that we would write as S4 4 times sigma 4.
And then I would have another term, S4 4 times sigma 4.
But the S4 4 really is a sum of two tensor elements.
So my definition of an S4 4 is that it should be--
Isn't that sigma 5 [INAUDIBLE]?
You're right, you're right.
Yeah, this is 5.
So this was right, 3 1, and this is sigma 5.
You're right.
And I go to S4 5, yeah.
No, sigma-- OK, yeah.
OK, let me cut to the chase.
What we're going to have to do is to put in not the factor 1/2 that we had here, but there's going to be a factor 4 that-- something like S-- which one are we dealing with?
Things like S2 3 3 1, something like S4-- and we're dealing with 5.
S4 5 is going to be defined as S1, S2 3 1 3 plus S2 3 3 1 plus S3 2 1 3 plus S3 2 3 1.
So we're going to have to put in a factor of 1/4 in front of the S4 5 in order to accommodate for those terms.
So our definition, then, in going from tensor compliances to matrix compliances is much more complicated.
And the rule-- and just to summarize this, I gave you a handout last time.
The necessary conventions to absorb these factors of 2 and 4 is that-- and I didn't bring it with me.
OK, I just simply did not bring it with me.
So I wouldn't attempt to summarize it.
All right.
So hopefully I've convinced you of nothing other than that these conversions are messy.
But the summary is that for the compliances, the Sijkl, you take this equal to Slm if l and m are equal to 1, 2, or 3.
Then you have to replace Sijkl by 1/2 of Slm if i, j, or kl is 4, 5, or 6.
And then you have to write Sijkl as 1/4 of Slm if i, j, and kl are 4, 5, or 6.
But once you're into the coordinate system, which will be fixed in matrix notation, things are simple, since you don't have to worry about these factors of 2 or 3.
The reason I did this is I would like to now talk a little bit about how one can define important scalar moduli that describe a particular phenomenon.
If you ask the question, how do mechanical properties vary with direction, we've got six unique elements of stress.
And we've got six independent elements of strain that can work.
And if you wanted to show how each one of those elements varied with the other one, you'd need 36-- 6 times 6, 36-- representation surfaces and this is not going to be fruitful.
So just as we did for piezoelectricity where we could define a scalar modulus which represented the component of polarization normal to a very thin plate-- making it a thin plate means you're going to measure primarily the charge on the large surface, because polarization is charge per unit area.
So if you looked at the surface of a plate with x1 normal to the plate, you are going to have a specimen that primarily gives you a measure of P1, or the charge on a surface normal to x1.
And then you could apply any of six different states of simple stress.
And one thing that you could do is squeeze it with the tensile stress in the same direction as the normal to the surface.
And that was the longitudinal piezoelectric modulus.
Then on the quiz you looked at another modulus that related a component of polarization on another surface in response to a tensile stress that was not parallel to it.
There are a number of such moduli and mechanical properties.
Probably the most important one is something called Young's modulus, which I'm sure you've all heard of.
And Young's modulus involves taking a very long rod of the material and hanging a weight on it.
And that weight will induce a strain.
And if we take this as the direction of x1, we will, with Young's modulus, relate the change of length to the initial length of the rod.
And my question now is, rhetorically, do we want to do this in terms of stiffnesses or compliances?
Does it make any difference?
Yeah, makes a lot of difference in terms of the simplicity of the result that you get.
Suppose we wanted to do this in terms of the compliances.
What we'd be then applying would be a sigma 1.
And this would be given by the compliance C1 1 times the strain E1.
That looks like exactly what we want.
This delta l over l for this one-dimensional specimen with a one-dimensional directional applied stress, this would be simply epsilon 1.
And that's force per unit area here.
This would be sigma 1 1.
And it looks as though what we want is a C1 1 1 that relates a sigma 1 1 to an epsilon 1 1.
Is this going to be a definition that I would want to make for describing this?
My colleague here shakes his head very seriously.
No.
Do you want to share your reservation?
Because as I recall, Young's modulus, once strain is weaker output [INAUDIBLE] in terms of stress you're going to want a compliance
That's one answer
You could theoretically get either one [INAUDIBLE]
If I could restate your objection, we can't impose a strain to get a stress.
We really stresses the independent variable, practically speaking.
We can stress it, but we can't instantly say, [INAUDIBLE] develop a epsilon 1.
Yeah?
[INAUDIBLE]
You betcha.
That's the reason.
Added onto this is not only this term, but there will be a C1 2 times epsilon 2 plus a C1 3 times an epsilon 3, and so on.
So we're not going to be able to get a nice, tidy relation between the tensile strain that we're measuring and the tensile uniaxial stress that is produced there.
So the relation of stress in terms of strain that involves the stiffnesses is just not going to work.
But if we would instead express epsilon1 in terms of a compliance, a matrix compliance S1 1 times sigma 1, that's all that she wrote.
We're measuring this by design by selecting an elongated specimen for which we'll primarily be seeing epsilon 1.
This is not to say these other strains exist.
There will be lateral strains here that will be epsilon 2 and epsilon 3.
There will be shear strains.
Sounds mind-boggling.
You pull the sample in this direction, and what it does is shear.
Well, if this were a single crystal, that would happen.
And these would be other components of deformation.
But if we measure a pi sigma 1 and measure epsilon 1, then this is the way we have to define it.
And the definition of Young's modulus is that 1 over S1 1 is sigma 1 over epsilon 1.
And this is Young's modulus.
I would hasten to observe that it is unfortunate that this very practical modulus involves the S's, which have all these complicated factors of 2 and 4.
And therefore, if we try to go to a particular symmetry and ask the question, how does Young's modulus change as we cut this rod in different orientations from the single crystal.
So I would like to illustrate for you how we would do this.
I'm going to take a very, very simple example.
If you look in the table of symmetry restrictions that I passed out a couple of times ago, prior to the quiz, for an isotropic the tensor has a different form than it does for a cubic crystal.
Cubic crystals are elastically anisotropic.
But the form of the stiffness tensor for an isotropic material was S1 1, S1 2, S1 2, 0, 0, 0, S1 2, S1 1, S1 2, 0, 0, 0.
Then the diagonal terms, if the material was isotropic, was 2s1 1 minus S1 2 for this term, 0, 0, 0, 0, 0, 0, and again, a 2S1 1 minus S1 2 plus a 0, and then 0, 0, 0, 0, 2S1 1 minus S1 2.
OK. Let me now derive Young's modulus as a function of direction and show that, in fact, this says that regardless how you orient the rod, the value of Young's modulus will stay the same.
Let me also do something else first, though, which is something we should have scratched our head over when we first encountered it.
Here's something curious.
When we looked at symmetry constraints on single crystals, we found that for a cubic crystal, C6 6 had to be equal to 1/2 of C1 1 minus C1 2 if the stiffnesses were to be independent for all symmetry transformations.
For the compliances, however, we found that S6 6 had to be equal to 2 times S1 1 minus S1 2.
Two very different equalities between tensor elements to make the elastic behavior be invariant to the symmetry transformations of a cubic crystal.
How come?
How come these are so different?
Well, they have to be, in terms of the tensor, the same sort of a quality.
And the reason they don't look the same is because of our different absorption of the factors of 2 and 4 in defining the stiffnesses an in defining the compliances.
Remember that for the Cijkl's there was no factor of 2 or 4 introduced in defining the matrix elements.
So this one is OK.
This is a true equality between tensor elements, for any tensor whatsoever of fourth rank for a cubic crystal.
For these terms, these are off-diagonal.
This is an off-diagonal compliance.
And our definition is that Sijkl is equal to Smn, for m and n not equal to 4, 5, or 6.
Here's this crazy thing again.
It's equal to 1/2 of Smn for m or n equal to 4, 5, or 6.
And it's equal to 1/4 of Smn for m and n equal to 4, 5, or 6.
So S6 6 here is actually 4S1 2 1 2.
And that's supposedly equal to 2S1 1, which is really S1 1 1, minus S1 1 2 2.
And this says that 1/2 of S1 2 1 2 is equal to S1 1 1 1 minus S1 1 2 2.
So this is exactly the same constraint on tensor elements, when we take out these factors of 2 and 4 that have been absorbed.
So this, in fact, although it looks very different with those factors absorbed, is exactly this same relation.
So the symmetry constraint is the same
How'd you get 1/2 there?
Hmm?
How did I get 1/2 here?
It's 4S1 2 2 had to be equal to 2S1 1 minus S1 2.
And this was the equality.
I'm working up this way.
So this was the statement in the matrix forms of the compliances that I handed out.
Looks different from the term on the left.
So I wrote this now in terms of the matrix elements.
And the factor of 4 comes in here.
The 2 was here in the equality that had to be held that the tensors stay invariant.
And if I bring this 2 over on the left-hand side, it's exactly the same as when--
Shouldn't it be 2 now?
[INAUDIBLE].
[INAUDIBLE] the right side
No, this is just replacement of the matrix terms with the definition of the tensor
Right, but [INAUDIBLE]
You're just dividing 4 by 2, right?
Yeah, exactly
So wouldn't that make [INAUDIBLE]?
Oh, ha,
You want to be able to [INAUDIBLE] by 4, so it'd be S12 1 2 equals 1/2 the difference to be equal to that
OK, OK.
Thank you.
OK, let me then go back to what I started out to do.
And I will write just one line of how we would go about expanding this just to indicate how intricate it is, and then I'll give a few examples for two point groups of how Young's modulus varies with direction.
OK. What we're going to say, then, is that one of these compliances, S1 1 1 1 prime is going to be equal to l 1/4 times S1 1 1 1 plus l 2/4 times S2 2 2 2 plus l 3/4 times S3 3 3 3 .
And just now doing what we know, that any Sijkl prime is going to be Cii, Cj capital J, Ck capital K, Cl capital L-- these are direction cosines now, not stiffnesses-- times Sijkl.
So it's a quadrupole summation over all of the non-zero tensor elements.
So what I'm doing now is taking these terms and expanding them to the full tensor form.
So S1 1 1 1 prime, that's the first term in the transformed tensor.
That's going to be these terms plus l1l2 squared times S1 1 2 2 plus S2 2 1 1 plus l1 squared l3 squared times S1 1 3 3 plus S3 3 1 1, and then still another term, l2 squared l3 squared times S2 2 3 3 plus S3 3 2 2.
And then there will be terms of the form l2 squared l3 squared times, again, four terms, S3 2 3 2 plus S1 2 3 3 plus S2 3 3 2 plus S2 3 2 3, and then similarly, terms in the squares of l1 and l2, and these would involve four terms of the form S1 2 1 2 plus permutation 0, and then l1 squared l3 squared times, again, four terms, S1 3 1 3 and permutation 0 for four terms.
OK.
So what do we have now?
We have an expression for S1 1 1 1 prime.
And this is, in fact, 1 over Young's modulus E. And we've done this summation over the supposed non-zero tensor elements in the matrix for an isotropic material.
And if I simplify this, and this will be just one more tedious and then we can see, that is for these equalities, we ought to, for l1, l2, l3, get the same value, S1 1.
And that is indeed what happens.
So the summation, if I simplify, is l 1/4 times S1 1 plus l 2.4 times S2 2 plus l 3/4 times S3 3.
And then a collection of terms plus l1 squared l2 squared, S1 2 plus S2 1.
And then similar terms in l1 and l3, l1 squared l3 squared, S1 3 plus S 3 1 plus l2 squared l3 squared times S2 3 plus S3 2.
And then some terms that stand by themselves, l2 squared l3 squared times S4 4 plus l1 squared l2 squared S6 6 plus l1 squared plus l3 squared times S5 5.
And consolidating this, this is going to be equal to l 1/4 plus l 2/4 plus l 3/4 times S1 1.
And then combining terms in the second power of direction cosines, l1 squared l2 squared plus l1 squared l3 squared plus l2 squared l3 squared.
And this is all times 2S1 2 plus 2S1 1 minus 2S1 2.
So S1 2 drops out.
And all this will be simply l 1/4 plus l 2/4 plus l 3/4.
And then I'll add in the other terms here that involve products of squares of direction cosines.
And all this is times S1 1.
But this turns out to be equal to simply l1 squared plus l2 squared plus l3 squared, the sum of the squares of the direction cosines of our rod, quantity squared times S1 1.
And this term inside the parentheses is 1.
So indeed, S1 1 prime is equal to S1 1.
So the value of Young's modulus has not changed with direction if the form of the compliance tensor is like so.
And I think you would probably have taken my word for that, but it was intended primarily as an example of how, when S1 1 prime is the reciprocal of Young's modulus, how you would set up a transformation for S1 1 prime.
And the form of this polynomial would be exactly the same, even for a triclinic crystal, if you included all the non-zero terms in this fashion.
All right, so let me wrap things up in another couple minutes for some cases that are real symmetries, where Young's modulus is anisotropic.
And working in exactly the same way for the tensor that is the appropriate one for a cubic crystal, we would lift out the form of the stiffness matrix.
The reciprocal of Young's modulus is S1 1 prime, so what we're saying is we have a long, skinny rod.
This is x1.
And what we're doing, if this is a single crystal, is examining how Young's modulus changes as we change the direction of the rod to a new orientation, x1 prime, that's described by direction cosines l1, l2, L3, relative to x1.
The expression that results by exactly the process that we muddled through a moment ago is that for a cubic crystal, the form of S1 1 prime as a function of direction does indeed give anisotropy.
Isaiah It's S1 1 minus 2 times S1 1 minus S1 2 minus 1/2 of S4 4.
Remember, for a cubic crystal, 1 1, 2 2, and 1 4 are the only non-zero terms.
And then this is times a polynomial l1 squared l2 squared plus l2 squared l3 squared plus l3 squared l1 squared.
So this is, again the reciprocal of Young's modulus.
And it turns out to have a constant term, S1 1 , which is what we found for the isotropic material.
But from that, as the direction cosines change, we subtract off a term that is a linear combination of 1 1, 1 2, and 4 4.
So the question is, is this positive or negative?
The direction cosines are all squared, so this term here is always going to be positive.
So are we going to take the thing that we found for an isotropic material, which was a constant, Young's modulus, which was 1 over S1 1 prime, and that's equal to the Young's modulus E. Are we going to add or subtract something to it?
Well, it turns out that if you look at real materials, it can be either positive or negative.
It's usually positive.
So that says we are going to subtract off a term which goes as products of squares of direction cosines.
And this is something that's going to be zero along the direction 1 0 0.
So along the reference axes x1, x2, x3, which if you remember, where this form of the matrix came from was taking the axes along the edges of the cubic crystal.
It turns out that this is going to be equal to 1/3 along the direction 1 1 1.
So if this term is plus, nothing gets added onto the surface in the directions that correspond to the four-fold axes or the twofold axes of a cubic crystal.
Nothing gets added on if it's plus.
But along the body diagonals, this takes on a value of 1 3, so a positive thing gets added on.
And the best way I can describe this surface-- it's not a simple surface-- it looks like a cube with fuzzy edges.
So we had a cube and started to dissolve it.
So this is how the reciprocal of Young's modulus varies with direction.
If, on the other hand, this term is negative-- so this is what you get if it's positive.
If it's negative, again, you start with the basic isotropic variation of 1 over S1 1 1.
If it is negative, and there's one metal, that's molybdenum-- molybdenum has a negative value of these compliances.
It looks like a surface that has indentations along the 1 1 1 direction.
So it looks like something with a dimple in it.
There is a final case, which is not realized for any material that I know of, if that term is 0, then it's isotropic.
Let me give you one other variation that comes straight out of [INAUDIBLE].
And this is for a hexagonal crystal that might be a hexagonal close-packed crystal.
For zinc specifically, which is a hexagonal close-packed metal, the form of the matrix is S1 1, S1 2, S1 3, 0, 0, 0, S1 1, S1 2, 0, 0, 0, S3 3, 0, 0, 0, S4 4, 0, 0, S4 4, 0, and S4 4.
In the values of these specific compliances are 8.4 for S1 1, for S1 2 1.1, for S1 3, minus 7.8, for S3 3, 28.7, for S4 4, 26.4.
And these are all in units of 10 to the minus 12 meters squared per Newton.
That's good old MKS units.
It turns out that there are 10 to the 2 meters squared per Newton per 1 centimeter squared per dyne, if you like CGS units.
The form of the reciprocal of Young's modulus, it turns out to be a surface of revolution.
This is x3, which is the direction of C. And that's the surface of revolution and the value of S1 1 prime as a function of the angle theta.
That's the only parameter we need examine the variation with since it's a surface of revolution.
S1 1 prime is equal to S1 1 times the sine of theta to the fourth power plus S3 3 times the cosine of theta to the fourth power plus S4 4 plus 2S1 3 times sine squared theta, cosine squared theta.
So it's a fairly exotic surface, even as a surface of revolution.
And what this looks like, it's something that peaks out at 3 times 10 to the minus 12 centimeters squared per dyne along the direction of C. It's something that comes down very sharply as you approach the normal to the C axis.
And then there's a cute little wiggle just near the axis.
So it looks something like that.
Not a terribly isotropic surface, and a variation of Young's modulus of 3 to 1 in the direction parallel to C and perpendicular to C. So there are lots of exotic surfaces of this sort.
All right.
I did run a little bit over.
And I will absent myself and let you express yourself candidly on the questionnaire.
Except for these numbers that I just put up on the blackboard, I haven't really said anything about, much about, numbers.
So let me pass around some examples, not for metals, which we just looked at, but for oxides.
And this is interesting, because you'll remember there was an additional equality between the stiffnesses called the [?
Kowshi ?]
equality, which was supposed to hold for physical reasons, not for reasons of symmetry, if the forces were central, if the crystal was under a state of no stress, and if all of the atoms were situated at a center of symmetry.
The first set of data that you have on the top sheet is for MGO, which is cubic.
It's predominantly ionic compounds.
And the atoms are all octahedrally coordinated.
So all of the requirements for the [?
Kowshi ?]
equality should hold.
And you can see they're not really exact.
And that's probably due to the covalent character.
The two different sets of data here are for measurements by two separate observers.
The second page is for aluminum oxide.
And again, you see the order of magnitudes of the numbers on the order of 10 to the 12 dynes per centimeter squared in general, and variation from one to another for a lumen of a factor of about four.
I meant to bring the book in, but I left it behind in my rush to come off.
Where do you get these numbers?
And it's hard to really find.
And the book from which I took these data are by two workers, Simmons and Wang.
And it's called Handbook of Elastic Constants, I think that is.
And this is a book that was published by MIT Press.
Valuable repository of data, but it's all numbers.
And there are many, many materials in here.
Simmons and Wang, interestingly, are geophysicists.
Why should geophysicists care about stiffnesses?
Why?
Because these guys are going all over the face of the earth setting off little bits of explosive that send off seismic waves.
And they probed the interior of the earth by the propagation of elastic waves.
And the elastic waves depend on the square root of stiffnesses over the density of the material.
So they're very, very interested in the elastic properties of particularly rock-forming minerals.
Another interesting comment on this book is that there is no set of data for a single triclinic crystal.
Too many stiffnesses that are independent, I guess, and too few materials that, fortunately, are anisotropic and triclinic.
OK, I am going to quit.
And thank you for your attention.
You've been a good class.
And we've covered all sorts of exotic aspects of the behavior of crystalline materials.
And you might think with some justification that you know all there is to know about symmetry and tensor properties of materials.
But nevertheless, I would caution you that you don't know everything.
So I have a final handout fill in the chinks, the cracks that we've not been able to fill.
And this is a set of data that is titled "Think You Know Everything?"
And this will fill in some things that you really don't know.
And it has a lot of very useful items on here.
For example, there are 293 different ways to make change for $1.
You didn't know that.
2/3 of the world's eggplant is grown in New Jersey.
I'm a native of New Jersey, and even I didn't know that.
The longest one-syllable word in the English language is "screeched."
OK. And so it goes.
In most advertisements-- this is one I never checked out-- in most advertisements the time displayed on a watch is 10 minutes after 10:00.
Don't know why.
How many ridges are around the edge of a dime?
118.
And how many little dimples are there on a regulation golf ball?
If you count them up, you'll find that there are 336.
So I'll end with one for the benefit of some of our visiting students.
In England, the Speaker of the House is not allowed to speak.
There is my oxymoron of the day.
And with that I shall leave you, and I hope you enjoy these.
It's one of these things that circulates around on the internet.
So as the sheet finishes, now you do know everything.
So I'll leave you, and again I thank you for your faithful attendance.
And you shall see me when you come to call for your quizzes.
And I will, as most of you requested, send out an email to let you know when you can come and pick those up.
So again, thank you.
Enjoy the mid-term break.
And take advantage of some of the really interesting things that go on extracurricularly around the institute.
So that's it.
Thank you, and au revoir.
[APPLAUSE]
All right, I feel almost as though I should introduce myself all over again.
It's been a week and a half since we had a lecture.
So let me begin by reminding you of what we were doing.
We had derived all of the plane groups, and for better or worse had put them behind us.
And then we moved into three dimensions, where things get a lot more involved and a lot more complicated, and to say the least, a lot more numerous.
And the first question we asked was to say, when we're in a three-dimensional space, we can combine a first rotation operation with a second rotation operation, B beta If we're to begin by deriving point groups-- that is to say, the least [INAUDIBLE] point in this three-dimensional space is not going to move.
[INAUDIBLE] two axes to intersect a point.
For a space group, they could be parallel to one another.
But that's gonna be an infinite set of symmetry elements and operations that extends through all space.
Then we asked the question-- rhetorically, because you knew I was going to answer-- what is the net result of a sequence of rotating from a first object to a second, and then picking up the second and rotating it to an angle beta about second axis.
Begin the third one here.
What is the net effect?
And again, we could do that by the process of elimination.
It has to be either translation or another rotation, because these are the only two generic sorts of operations which leaves the chirality unchanged.
And I think I convinced you that indeed there was some third axis, C, which rotated directly from the first to the third by some different angle, gamma.
So that is the consequence of combining the first two rotation axes.
What they would anticipate is that the location and also the value of the rotation [INAUDIBLE] depends on alpha and beta, and also [INAUDIBLE] at which we combine them.
And that's what we'll see.
You could do this in a number of ways that if you don't [INAUDIBLE] that gamma would turn out to be a crystallographic rotation.
And then your result would be true, but it would not be a rotation operation which could exist in a three-dimensional point group.
So using the genius of Leonhard Euler and a construction known as Euler's construction.
We set up a little spherical triangle, which we could analyze.
Let me tell you a little bit about Euler, because he's a remarkable individual.
The first remarkable feature of Euler is that he's Swiss, and there are not many world-class famous people who are Swiss, simply because the population is so small.
The probability of somebody rising to heights is constant among all populations.
If you have a small population, there are not going to be very many.
And to demonstrate that point, can somebody identify some other citizen of Switzerland who rose to great heights, as world-famous as [INAUDIBLE]?
Think of one other person?
I'm fairly pressed to do so myself.
There's my uncle, but he actually didn't amount to much.
But there's an artist, Paul Klee, who is world-class.
He was one of the early modern artists.
And Switzerland has just finished constructing a marvelous museum on the outskirts of the capital city, Berne.
It's a structure that is supposed to mimic the rolling countryside of the central part of Switzerland.
So it's a series of cylindrical structures, glass in front, glass on top.
And it divides the area into three.
Two of them are exhibit spaces and one is a space for scholars and researchers.
And it's an absolutely marvelous structure.
It appeared in the pages of Time Magazine when it opened [INAUDIBLE] about two months ago.
Anyway, Euler was born in 1707, so he operated a long time ago.
And he died in St. Petersburg on September 18, 1783.
That is exactly 350 years and one day after my birthday.
That's another remarkable thing about him.
He was 76 years old when he died.
And that time of primitive medicine and plague, not many people got to live to their 60s and 70s.
Euler studied at the University of Basel under the Bernoullis.
I think you've all heard of the Bernoullis.
There's a very famous principle of physics known as the Bernoulli effect, which stated in its simple practical form says that if you have the Sunday paper on the front seat alongside of you, and you drive your car with the windows down, the paper will blow out the window.
That's Bernoulli's principle in action.
Euler got his doctorate from Basel at age 16.
It sort of leads one to the rhetorical question, how come you guys have been spinning your wheels for so long?
But then I said, he was an unusual individual.
The Bernoullis went to St. Petersburg in Russia, under Catherine the First.
Russia was trying very hard at that time to enter the ranks of the Western world as a full member.
Euler followed them a little bit later on.
And he succeeded one of the Bernoullis as a professor of mathematics in 1733.
Then unfortunately, two years later in 1735, he lost the sight of one eye.
And why?
Because at that time, astronomy had been using this newfangled telescope which had recently been perfected.
One of the hottest things going was studying the heavens looking through a telescope.
And if you wanted to look at the sun, people knew nothing about the damaging effect on retinas of the sun's rays.
So he lost the sight of one eye.
1741, he went back to Europe again, to Berlin.
Why?
Because the reigning monarch in Russia at that time was called Ivan the Terrible, and that says reams about why Euler would want to get out of Russia.
But then 1776, he went back to St. Petersburg under the next monarch, who was Catherine the Great.
And that says why one would be interested to go back.
Finally, in 1766, he went fully blind.
Did that slow him down?
Not one bit.
He published in his lifetime 800 papers.
You talk to some big cheese around MIT, they've published maybe 200 or 300 papers, and that with the assistance of an army of graduate students, and also, one might add, the assistance of Xerox machines and word processors.
So back in the days when you wrote everything out by hand with a [INAUDIBLE] quill pen, 800 papers is an absolutely unbelievable accomplishment.
It took 35 years after he passed away to publish everything that he'd written.
People had kept busy publishing what he did.
And among his accomplishments, he was one of the first people to apply real hardcore mathematics to astronomy, to make it quantitative.
He was one of the first to suggest that light was a wave form, and that color was a function of wave length.
That was astonishingly precocious.
And then, lest he seem like an egghead who spent all his time staring through telescopes and working out theorems to use in crystallography, he also wrote a popular account of science for the general public, which was published in 1768.
And that book was published for 90 years, three generations of people kept gobbling up [INAUDIBLE] pretty good.
He impinged upon our own language and activities in several important ways.
He was the one who used lowercase i to define the square root of minus 1.
We can thank Euler for that.
He was the person who used e to define the constant, 2.71828182845904523536.
And he was the person who first used f to stand for function.
So he contributed not only a lot of good mathematics, but a lot [INAUDIBLE].
So this does not have to be easy.
Euler was a great guy.
And this geometry of rotations about different axes is something that also survives in a mechanism that involves achieving angular core rotation on a axis by rotation on two orthogonal arcs.
And that's something that's called an Euler Cradle.
And that is geometry that;s used in a great number of mechanical devices.
In any case, back to instruction for our purposes.
The thing that we would like to do is let alpha and beta take on all possible crystallographic values, namely 360 degrees or onefold axis, although we know that that's not gonna work.
Twofold, 180 degrees.
Threefold, 120.
Fourfold, 90.
Sixfold, 60.
And let that give the values to alpha and beta, taking two at a time.
And then let us ask the question, at what angle should we combine these two axes to get gamma to be a crystallographic rotation axis?
And if it is crystallographic and not something like 37.9234 degrees, what are the remaining axes with interaxial angles B and A?
So this is the problem that Euler's construction solved.
And I won't go through all the arguments that we need to set this up.
But what we found after some f sleight of hand when we were working on the polar triangle with spherical trigonometry, what we found was the result that said that if we want to combine two axes, alpha and beta, so that the third one turned out to be a rotation of gamma, then the cosine of the angle between A and B should be the cosine of alpha/2, cosine of beta/2 plus cosine of gamma/2, divided by the sine of alpha/2, sine beta/2.
So if you pick your alpha and beta, and you decide what you would want these first two rotations to turn out to be.
And generally it's not gonna work.
But there are a surprising number of cases where it does work.
So you specify the combination you had.
You also determine the angle between A and C. And you have to also determine the angle between the axes B and C. And there are similar sorts of expressions that one obtains simply by [INAUDIBLE] alpha and beta again.
So then we set it up just by looking at all possible combinations of twofold, of two different rotation axes, and a third, which the net effect might be.
We're not interested in permuting A, B, and C. And A equal to C equal to B is just as interesting or not as A equal to B equal to C. We don't care about permutations.
And we generated-- just as we [INAUDIBLE] a week and a half ago-- a set of combinations that we should consider, what the axis A would be, what the axis B would be, and them different choices for the axis C. So A could be 1.
B could be 1.
And we could look for 1, 1, 1; 1, 1, 2; 1, 1, 3; 1, 1, 4; 1, 1, 6.
Those are legitimate combinations?
Those are absurd combinations, because doing nothing about the first onefold axis, doing nothing about the second onefold axis could hardly result in the net effect of the 90-degree rotation by the third axis.
And [INAUDIBLE] suggested is that sitting around and doing nothing twice was equal to a rotation [INAUDIBLE] its junctures [INAUDIBLE] we'd find ourselves spinning on our axis like tops.
Twofold axis.
1, 1, 2, we have here, so we don't have to consider 2, 1, 1.
But we should consider 2, 1, 2; 2, 1, 3; 2, 1, 4; 2, 1, 6, and so on.
If we filled out this whole table, last time you got a copy of it and some notes, and all that remains then is to quote [INAUDIBLE] and see where we get allowable axial combinations.
And not surprisingly, there are so very, very few.
And we showed-- again, when we finished up last time-- that you can always take any n-fold axis that has a C gamma that's equal to C 2 pi over n and combine it with twofold axes at right angles, provided you make the angle between the twofold axes equal 2 of gamma over 2.
So the crystallographic possibilities for C are, first of all, C could be a twofold axis, in which case you could combine with it a pair of twofold axes, and this 1/2 of 180 degrees would also be a right angle.
We could let C be a threefold axis, in which case the twofold axes have to be at right angles, two- to threefold axes.
And they should be combined at half the angular throw of the threefold axes, which is 60 degrees.
Two more possibilities are four [INAUDIBLE] pair of twofold axes at right angles.
Of the angle between them, half of the throw of a fourfold axis would have to be equal to 5 degrees.
And the last one is sixfold axis with a pair of twofold axes.
Add angles to it.
And a third [INAUDIBLE].
So notice the insidious fact that the angle between the twofold axes is always a half, 1/2 the rotation angle in principal axis C are not equal to this rotation axis.
The other thing we saw that is that these twofold axes are different, distinct, symmetry-independent axes.
They're different in that the principal axis C never rotates this axis into the second one, and therefore demands that whatever's going on around one twofold axis be identical to what's going on at the other twofold axis, different in the sense that they function in different ways in the pattern, or if you're describing the symmetry of an object.
So probably the best demonstration of this is a regular prism with a triangular shape or with a square shape or with a hexagonal shape.
And the adjacent twofold axes for these prisms would come out of the normal to a face.
And then if the second twofold axis is going to be 45 degrees away from the first, the other one has to come [INAUDIBLE].
So yeah, they function in different ways in the space.
One is normal to the face of a regular prism, if that's what's in our space.
The other one comes out of the edge.
Similarly for a sixfold axis, a hexagonal prism has one twofold axis coming out normal to a face, the adjacent twofold axis coming out to an edge.
And as advertised, that angle is 33.
So they function in different ways.
The only exception to that, again, is the [INAUDIBLE] threefold axis.
And the twofold axes there, which were 60 degrees, come out of one side from a corner of the triangular prism.
On the other side, that was [INAUDIBLE].
So all of the twofold axes were the same thing.
There's only one independent kind of twofold axis, just as there was only kind of mirror plane in the combination of a mirror plane passing through a twofold axis.
The names for these are always, as we've done in the past, a running list of the independent symmetry operations that are present.
So this general one, n, 2, 2, with the n-fold axis for some generic sort of a twofold axis.
This one would be 2, 2, 2.
This one would be 4, 2, 2.
And this one would be 6, 2, 2.
This [INAUDIBLE] with a threefold axis is, again, called not 3, 2, but 3, 2, because there's only one kind of twofold axis, just as there's only one kind [INAUDIBLE].
We pause [INAUDIBLE], see if you have any questions.
These are the crystallographic combinations of this [INAUDIBLE].
There is no reason why you should not in something that doesn't have to be compatible with a lattice, combine an n-fold axis of any sort with twofold axes or right angles.
And indeed, if I look at my old friend, the saguaro cactus, we can add anything like 28- up to 32-fold symmetry.
This cactus stem had a 28-fold symmetry.
It would be a twofold axis coming out of the string with one of the ribs, another twofold axis coming out of the crevice between the pair of these ribs.
And if I took that thing up, very carefully because of the spines, and with great effort because it weights several tons, and rotated it about one axis, and then rotate it again about a second axis, turning it upside-down, [INAUDIBLE] axis would be rotation to 128 [INAUDIBLE].
Valid symmetry, but not crystallographic.
OK, any comments or questions?
Get to know these results, because the exercise that's going to occupy us for the next week is going to be asking how we can decorate these frameworks with mirror planes and with inversion.
If you want orientation, we could add another operation to the collection of axes while it pops up.
Where [INAUDIBLE].
Comments or questions?
OK, there are only two other combinations that are crystallographic that involve directions that are not simple.
And one of them involves a combination of a threefold axis with twofold axes that come out of directions at a normal to the face of a cube.
So these turn out to be very, very strange angles which make no sense at all until you refer them to directions in the cube.
The direction of the threefold axis turns out to be correspondent with the angle of a cube.
The direction of the twofold axis corresponds to the normal two faces.
You can show-- I did show, and I don't think anybody really followed me, so I had to hand out that [INAUDIBLE]-- that if you start with one twofold axis and one threefold axis, what you're going to get is a threefold axis coming out of all of the [INAUDIBLE] diagonals, but they're all equivalent to this threefold axis.
And twofold axes come out to normal for all the faces of the cube.
So there's only one kind of twofold axis and one kind of threefold axis, so even though we got this by combining a pair of twofold axes-- I'm sorry, a pair of threefold axes-- that's what happens when you stay up late.
You forget about this one.
A pair of threefold axes at the diagonals and one twofold axis.
And this is the combination that is called 23, because there's only one sort of twofold axis and one sort of threefold axis.
And I'd like to point out and I'd like to warn you of traps when we come across them, make sure we don't [INAUDIBLE] across them.
Notice the insidious relation of this pair of integers to the symmetry that we label [?
232.
?]
32 is a threefold axis with a twofold axis normal to it.
23 is this combination that involves corrections in a cube.
Now, let me pause parenthetically with an aside.
You might say, how can this be?
Here is a cube.
That cube has got a fourfold axis about it.
Don't call that a twofold axis.
A cube has a fourfold axis coming out of it.
OK, let me give you an example in real life.
There is a fairly common mineral, iron disulfide-- pyrite.
This forms nice, shiny cubes, but the cube faces have striations on them.
If you look at them, there's a set of lines running this way.
And what those lines are if you look at this crystal face with a magnifying glass, is that these are little steps of a second face.
And this is a face of the form hk0.
And this oscillates back and forth, and there seem to be lines scribed on the surface.
So this sort of a [INAUDIBLE] crystal growth of one face which never really develops is not that uncommon.
There's a threefold axis coming out of the corner here, but this is really a surface that is left at variant only by 183 [INAUDIBLE].
You cannot rotate that surface 90 degrees.
The orientation of the lines have changed.
But there is a bona fide threefold axis coming up there, so these striations, if I rotate them to this face, will go in a way like this.
This edge turns into this edge, and therefore the lines will run down like this.
And if I rotate [INAUDIBLE] n by 90 degrees, the striations on the adjacent face will run down.
So there's a decorated cube.
And if you rolled it up and say how can I move this cube around and leave its appearance totally unchanged?
the answer is, rotate it by 120 degrees [INAUDIBLE] diagonal.
But we can only rotate it 180 degrees around the face.
So there's an example of a crystal [INAUDIBLE] on the arrangement of rotation axes 23.
The final one, the highest symmetry of all, involves a fourfold axis coming out of a direction normal to a face, a threefold axis coming out of the [INAUDIBLE] diagonal, and a twofold axis coming out normal to an edge.
And if you let these axes work on one another, there's a twofold axis that comes out of every edge, and a fourfold axis that comes out of every face.
And this one is named 432, because it's a combination of a fourfold, a threefold, and a twofold.
That is the symmetry to the cube.
And for crystallographic symmetries, that's about as complicated as it gets.
Now, if we look at the regular solids that we've encountered here, with symmetry 23, there is a regular [INAUDIBLE] consisting of four triangular faces.
That's a tetrahedron.
And for 432, one of the polyhedra that can form from the crystal [INAUDIBLE] is an octahedron.
These were the lovely solids called Platonic solids, after Plato, that we used as our trophies early this afternoon.
So this is an octahedron.
Let me finish up before our break by asking is there any other regular polyhedra that can result from a combination of rotation axes that are not crystallographic?
Now, that's a tough question to ask.
You instantly scan your knowledge of geometry.
Clearly, there are a lot of prisms [INAUDIBLE] infinite number of prisms [INAUDIBLE] n22.
But there's only one other combination of axes non-crystallographic which results in a regular polyhedron.
And this is a combination, believe it or not, of a fivefold axis with a threefold axis.
This is a fivefold axis and a threefold axis and a twofold axis.
And these result in a regular solid called an icosahedron.
And that is so complicated that I won't attempt to draw it.
But having said so, it looks like this.
It has diamond-shaped faces.
And there are five of these that come together in a [INAUDIBLE] form.
So here's one of the [INAUDIBLE] diamond-shaped faces, another diamond-shaped face, and then the two other ones that come in like this.
So there are 1, 2, 3, 4, 5 faces, so this is the orientation of a fivefold axis.
The twofold axis comes out of a place where two of these edges are shared.
And the threefold axis-- [INAUDIBLE] triangular faces.
[INAUDIBLE].
So here's the fivefold axis.
These are the twofold axes [INAUDIBLE].
And these are threefold axes.
But you know all this.
Is there anybody who [INAUDIBLE] show me that they've never seen an icosahedron?
Anybody ever who has not seen-- have you seen it?
[INAUDIBLE].
Here-- I spared no expense-- is a live icosahedron for those who would like to look
How many sides does it have?
He'll count them for you.
I don't remember.
I know the number of faces and the number of edges, but not the [INAUDIBLE]
[INAUDIBLE] faces
I think it's-- I'm not sure [INAUDIBLE].
Not sure
But you said you know the number of faces
Yeah, but I can't tell you everything I know [INAUDIBLE]
Yeah, yeah
There's another figure which also has this symmetry, and it's a regular solid.
And this is called a rhombic dodecahedron.
And, wise guy, this has 12 faces.
And the faces are pentagonal.
And there are three pentagonal faces that come together at the threefold axis, a fivefold axis out of each of the pentagonal faces, and a twofold axis out of the edges.
The face that this has 12 faces at regular intervals leads an entrepreneur who was familiar with injection molding to cast these little things as a plastic, and then puts a month of the year on each of the 12 faces and made a nice little desk calendar.
[INAUDIBLE] something that reminds you of symmetry as well [INAUDIBLE].
So this has fivefold faces, 12 of them, and that's the rhombic dodecahedron as opposed to the normal dodecahedron which is [INAUDIBLE]
So are those both [INAUDIBLE]?
I'm sorry?
Are those both [INAUDIBLE]?
The icosahedron, [INAUDIBLE]?
Yeah, this has a fivefold axis coming out of the pentagonal [INAUDIBLE].
Is that what you're asking?
And the threefold axis, there are three of them that, come together, and they do something like this.
So here's the threefold, here's the fivefold, here's the twofold.
Here we have to see it [INAUDIBLE].
Look for somebody who's got one of these desk calendars.
[INAUDIBLE] used to sell them.
[INAUDIBLE].
OK, this sets up the next stage of our game.
We've got these arrangements of axes.
And if you count them up on the fingers of your hands and one toe, there are 11 of them.
There are the axes by themselves-- onefold, twofold, threefold, fourfold, sixfold.
There are these so-called dihedral combinations, where the only thing that changes from one to the other is the symmetry of the main axis, the [INAUDIBLE] symmetry, and therefore the dihedral angle between the twofold axes.
And these are 222, 32, 422, and 622.
And then the two cubic arrangements, 32 and 432.
So when we return, we'll ask the question, how can we add mirror planes for an inversion center to this combination of axes?
And these are going to give us new symmetries involving not only rotation, but [INAUDIBLE] help with rotation inversion as well.
And the constraint in doing this is that we have to add the reflection plane and the inversion center in such a way that it doesn't create any new rotation axes, because we have systematically derived all of the possible combinations of crystallographic rotation axes.
So if the addition of a mirror plane, creates a new axis, that's going to be something that you already have with a combination of a greater number of rotation axes.
For example, if you take a single twofold axis and a mirror plane of an angle, that generated another twofold axis 90 degrees away.
But we've already got that.
If a mirror plane moves an axis to an angle that doesn't correspond to one of these arrangements, it's going to be impossible, because we have systematically derived, using Euler's construction, all the combinations of rotation operations that are possible.
So that's going to be the constraint.
We want to add mirror planes or an inversion center in all possible combinations.
And this means we're gonna need a theorem.
What happens when you add a mirror plane to a rotation operation?
We're already familiar with one of them.
You take an axis and you pass an mirror plane through it, you get another mirror plane that is rotated about the axis, away from the first, by half the rotation angle of the axis.
So let's take a breather, and let us resume in about 10 minutes.
To mention, most everyone did extremely well on the quiz.
But I sense that there's still some of you who have not yet come to terms with crystallographic directions and planes, and you feel a little bit awkward in distinguishing brackets around the HKL and parentheses around HKL.
And there are some people who generally get that straightened out, but when I said point group, suddenly pictures of lattices with fourfold axes and twofold axes adorning them came in, and that isn't involved in a point group at also.
Again, a point group the symmetry about point.
A space group is symmetry spread out through all of space and infinite numbers.
So let me say a little bit about resources.
I don't know whether you've been following what we've been doing in the notes from Buerger's book that I passed out.
That was hard to do is because we did the plane groups, and he doesn't touch them at all.
So now we're back following once again Buerger's treatment quite closely.
So read the book.
And if you like, I can tell you with the end of each lecture, this stuff is on pages 57 through 62.
The other thing.
As you'll notice, this nonintrusive gentleman in the back is making videotapes of all the lectures.
These are eventually going to go up on the website as OpenCourseWare.
We were just speaking about that, and it takes a while before they get up, but I have a disk of every lecture.
So if there's something you didn't follow or a place where I chewed my lines and you want to go back over it again-- not that that happens very often-- you are more than welcome to ask me to borrow and borrow the disk if you want to review it.
So that's another resource.
And then I will regularly throughout the term give hard-copy handouts of some of the things that we're doing, particularly when it involves geometry.
And when we move on to three-dimensional geometry, unless of the graphics is really tight and precise, it's hard to follow what's going on.
So in that vein, one of the first things I wanted to pass out-- the only other one for today-- is a demonstration that in fact in the Group 23 all you need is a single twofold axis oriented along the normal to a face and a single threefold axis coming out of one body diagonal.
And that gives you all of the axes you are going to get into 23.
So when this comes around, there are number of steps that you can perform letting the axes work on each other.
And if you start with just the single twofold axis and the single threefold axis-- which the symbol suggests is all you need, there's only one kind of each-- if you look at a cube along its body diagonal, the twofold axis that's coming out of one face gets rotated into directions normal to all the other faces if you rotate by 120 degrees.
So the little diagram in the upper right hand corner of the sheet hopefully convinces you of this.
Now we've got three mutually orthogonal twofold axes and one threefold axis coming out of a body diagonal.
So the vertical twofold axis swings that around by 180 degrees to give you another threefold axis along a body diagonal.
And I labeled that one 3 prime in the middle diagram at right.
And then if you take the axis 3 prime and repeat it by the second of the twofold axes that we have along face normals, that in the middle diagram on the right hand edge takes 3 prime and repeats it to 3 double prime.
Then finally, the third twofold axis that we generated repeats the threefold prime axis to the remaining threefold axis along the fourth body diagonal.
So the results start with one twofold axis oriented along the normal to a cube and one threefold axis along the diagonal of the cube, you get axes coming out of all faces and all diagonal.
And there's a staple on that sheet for reasons that I don't understand, but it there was, probably on the surface of the Xerox machine when I went over there.
So let me now return to our next step, and that is to add what in the language of group theory is called an extender, a new symmetry operation that can be added to a preexisting group that will generate new operations.
And let's see what sort of theorems we need to describe what we should look for and in which particular orientation.
We've got these 11 axial combinations, and these are frameworks that we can hang mirror planes on.
So let's look at a first simple combination.
Suppose we have a twofold axis, and the only nontrivial operation there is a rotation through 180 degrees.
So this an axis A pi.
And again, the ground rules are that if we're to add a mirror plane to this axis, which along with identity constitutes a group, we can't create any new axes.
So there are two ways we can do this.
One is the three-dimensional analog of something that we have already done, namely to pass a mirror plane through the twofold axis.
And this is the group that we found as a two-dimensional Point Group, and we called it 2mm.
And to do that, we used the theorem that says that if you take a rotation operation A alpha and combine it with a reflection operation that goes through it, you get a new reflection plane, sigma prime, that's at an angle alpha over 2 to the first.
So what gave us the second mirror line in two dimensions, in three dimensions that would give us another mirror plane that's at right angles to the first.
And this will give us a three-dimensional symmetry, which is also called 2mm.
So it's nothing more than taking the two- dimensional Point Group and letting it come out at the board at you-- Man, that'll give you nightmares when these things are coming out of the paper at you-- and that is a valid group 2mm.
There's another way that we can add a mirror plane to a rotation axis though which will not create any new axes, and that's to add the mirror plane-- the reflection operation sigma I should say since it is a combinations of operations-- exactly perpendicular to the locus of the rotation operation.
And that reflection operation sigma then just flips the rotation axis end to end.
And the rotation operation just swirls the locus of the reflection plane around within its own locus and doesn't create any new reflection plane.
So in the particular combination A pi, if we add a mirror plane perpendicular to that as the operation sigma, this would be then all of the operations of a twofold axis over all of the operations of a mirror plane.
So we've got now a combination of a twofold axis with a mirror plane.
We've got the same thing here.
To distinguish these two combinations, we'll write this as a fraction 2/m.
And that literally in words is the way we've added the twofold axis.
It is sitting over the mirror plane and gets reflected down into its other end when the mirror plane act on.
So it's just language.
It's nice though, as we said some weeks ago, if our language have some descriptive content to so when we look at it we can remind ourselves of what it means.
So two 2mm means the mirror planes are parallel to the axis that contain it.
2/m means the twofold axis is over the mirror plane.
It goes through and pierces it.
What I would like to ask though is have we got a group yet?
And let's take a first object-- let's call it right handed-- rotate it by 180 degrees.
You get a second one, which will stay right handed.
And then repeat it by a reflection operation in the mirror plane, and we'll get a third operation.
Reflection changes chirality, so the third one is left handed.
So we are performing the sequence of operations A pi followed by sigma.
And let me append just so we make no mistake and not confuse it with 2mm that the reflection operation is normal to the mirror plane.
So question, what is the net effect?
And lo and behold, we have stumbled over-- if we had not been clever enough to invented it and suggested it early on-- the only way you can get from 1 to 3 in one shot is to turn it inside out and change its chirality by projecting it through a point which is the location where the twofold rotation pierces the mirror plane.
And what this is going to do is to change the sense of all three coordinates.
This is going to take the coordinates XYZ of object number 1 and change them into minus x, minus y, minus c. And what we're doing is inverting the motif, turning it inside out as it were, into an enantiomorph.
And we have discovered as soon as we combine a pi with a perpendicular reflection plane, a new operation which we'll call in words inversion.
And the symbol that used to describe this is a one with a bar over it, and we'll see why later on.
But this is a onefold axis with an inversion center sitting on it, and that's what an inversion center by itself is.
The implication is that they're going to be other axes that we can abbreviate such as 3-bar, 4-bar, and so on.
So this is analogous to a situation that will come later where we really need a new notation.
So we've fell headlong over a new type of symmetry operation, and we should consider taking inversion and adding that to the rotation axes by themselves as an extender.
Obviously, if we take inversion and add it to a mirror plane, we're going to get A pi.
If we take a twofold rotation, add it perpendicular to a mirror plane, we get inversion.
If we put inversion and put it on a 180-degree rotation, we'll get the mirror plane back.
So these things all permute one to another.
You may even remember some time ago we asked in general terms when do two operations permute without changing anything.
And the answer is if these operations to leave the locus of the other one alone.
And the mirror plane obviously leaves the locus of the rotation operation unchanged.
The rotation operation spins the mirror plane around in its own plane and doesn't create a new axis.
And the inversion center leaves the mirror plane alone and takes the twofold axis top to bottom.
So those three operations, inversion A pi, sigma are the three nontrivial operations that exist in the space.
The fourth one is the identity operation.
So here is the set of elements in the group that we will call 2/m.
2/m implies three operations inversion, a 180-degree rotation, reflection in a plane perpendicular to the axis, and the identity operation.
And I'll leave it to yourself for you to convince yourself that I can rotate and then reflect or I can reflect and then invert.
And all of these operations do not create any new motifs in the set.
The group multiplication table, in other words, contains just the four elements, 1, 1-bar, A pi, and sigma.
So the full pattern that corresponds to 2/m consists of four objects.
It'll be a fourth one down here.
The twofold axis tells you have this fellow is related to this one.
Inversion tells you how this one is related to this one.
And the mirror plane tells you how this one, number 1, is related to number 4.
And the identity operation tells you how 1 is related to itself.
So, again, as we've seen in the set of operations that constitute a group, there's a one-to-one correspondence between the transformations that are elements of the group and the number of objects in the pattern.
So we've made one combination, and what we found from this is a new transformation inversion that involves changing the sign of all the coordinates in a space through a point which is called the inversion center.
Questions?
If not, let me quickly rattle off other Point Groups in this family.
We could take the operation A pi/2 in 90-degree rotation and add this perpendicular to mirror plane.
Let me now say something that I've said again many times before.
The pattern of objects that will result is the pattern of objects that's produced by the initial group-- let's say a fourfold axis-- repeated by the extender.
And the operation sigma perpendicular is the extender.
So the pattern, without making any big deal about it, is going to look like this square of objects reflected down below the mirror plane.
So it'll be one going down like this.
One like this.
One like this.
The operation A pi sits perpendicular to the operation of reflection, so there will be an inversion center that arises at the point of intersection.
And indeed the square above can be inverted through this point-- little hasty repairs there-- and every one up above gets inverted down to an enantiomorph below.
So all of these guys up on top are right handed.
All these guys down below are left handed.
So this is another group.
This is 4/m in international notation, a fourfold axis perpendicular to a mirror plane.
There is also, unfortunately, a Schoenflies notation.
The international notation tells you what you've got, a twofold axis or a fourfold axis perpendicular to a mirror plane.
The Schoenflies notation tells you how you derived it.
And the symbol for a twofold axis is C2, so this is a twofold axis.
And what we did was to add an extender consisting of a horizontal mirror plane.
Schoenflies calls this one C2h; Group C2, which is a twofold axis; a horizontal m is the extender.
So Schoenflies tells you how you make it.
The international notation tells you what you get as a result.
Schoenflies notation here would be C4, that's the symbol for a fourfold axis, and the extender is an h. And then without making any big fuss about it, if I do the same with a sixfold axis, I will have six objects related by a sixfold rotation axis.
If I take that sixfold axis and put a mirror plane perpendicular to it, these will be reflected down to a hexagon of enantiomorph equidistant below the mirror plane.
That's not terribly good, but it's not terribly bad either.
So this would be called 6/m, Schoenflies notation C6h.
And the operation A pi exists in a sixfold axis, so there is an inversion center but also arises as a new symmetry element at the point of intersection.
So with reckless abandon, you can continue on here and derive noncrystallographic groups for all the even-fold axes and derive an 8/m and a 12/m and 16/m.
Lovely symmetries.
I wouldn't want to draw them, but they're still symmetries.
They all have an inversion center in them, but they're noncrystallographic.
So we don't have to worry about them for prism purposes.
I left one out because it introduces a complication that is kind of curious and interesting.
Any questions on what we've done here?
Yes?
The 2mm, it's just the same-- Schoenflies notation is just C2--
Schoenflies notation for three dimensions is exactly the same as in two.
So the three-dimensional version where this extends in a direction that is normal to the two-dimensional space of our two dimensions.
Two dimensions it was this.
Now just imagine them coming out of the blackboard at you.
The symbol for this one was 2mm.
The symbol for this one is also 2mm, the same thing.
The mirror plane is a vertical mirror plane.
So the Schoenflies notation is exactly the same as what we used per two dimensions.
It's called C2v.
We've added a vertical mirror plane.
And again, horizontal and vertical.
Horizontal is horizontal with respect to the axis of higher symmetry.
Vertical is vertical with respect to the two-dimensional space of the two-dimensional Point Group, parallel to the axis in three dimensions.
So that's the vertical indication.
So let's, though, tuck that away for future reference.
We've got two kinds of extenders.
We've got a horizontal mirror plane, and we've got a vertical mirror plane, and these are extenders that we should consider adding.
So we've taken care of 2/m, 4/m and 6/m.
2mm, 3m, 4mm, and 6mm are just the extensions into a third dimension of what we've seen and come to love in the two-dimensional space.
Let me now turn to the threefold axis.
And this is a curious one.
Threefold axes require fewer symbols to indicate the vertical mirror planes because there's only one independent one.
But let's see what would happen.
And now I'm not going to attempt to draw these in three dimensions anymore.
I'm going to use a stereographic projection.
And what I'll do is use a solid dot for a point that's up above the equatorial plane and an open dot for one that's down below the equatorial plane.
So my stereographic projection of 4mm would look like.
This is the fourfold axis.
This is the mirror plane.
And I've got one up that gets reproduced by the axis to give me a set of four.
All of these are, let's say, right handed.
And then directly below them is another set of four repeated by reflection, and these are all left handed.
And then there's an inversion center at the point of intersection.
And I'll indicate that by the little open circle sitting right on the fourfold axis.
So there is a projection of what 4/m looks like.
So let me now do the same thing for a threefold axis.
And I'll add to the triangle of points that a threefold axis would generate.
So these guys are all of one chirality.
Let's say right handed.
Then I'll reflect them down, and I'll get three objects that are down.
And that's what 3/m looks like.
Is there an inversion center here?
No.
No, because the operation of A pi is missing.
And it was the horizontal mirror plane combined with A pi that gave us the inversion center with all of the even rotation axis.
So one of the things we have to say here is that there is no 1-bar that's present, which means in this instance, unlike the other ones, we have another option.
So we can use the operation of inversion as an extender too.
So we're going to get another group out of the threefold axis besides this one.
And this one we will name 3/m or C3h in Schoenflies notation.
They're six objects here, so there should be six operations in the group.
So let me number these guys up on top as number 1, number 2, number 3.
And 1 is related to itself by inversion.
There's an operation A 2 pi/3.
And that tells me how the one that's up is related to the second one that's up and how the left-handed one that's down is related to the one that's directly below number 2.
There is an operation A 4 pi/3.
And that's the same as saying 8 minus 2 pi/3.
And that tells us how the things that are separated by 240 degrees are related, both up and down.
I know how 3 up is related to 3 down and how 1 up is related to 1 down and 2 up is related to 2 down.
This is all with the horizontal mirror plane, which I'll call sigma h. That is a total of one, two, three, four-- whoops-- one, two, three four operations.
I need six.
Let's ask how is this one that's up related to this one that's down?
I just got rotation operations and reflection.
The only way I can get from this one number 1 to this one number 3 left that's down is to take two steps to do it.
I can't get from this one up here to this one down here unless I rotate 60 degrees and then invert.
If I move over to 3-bar.
This is A 2 pi/3 with 1-bar as an extender.
The pattern would, again, look like what are threefold axis does.
But then if I repeat this set of three by inversion, the two triangles above and below are skewed.
The ones down below are enantiomorphs.
The three that are up are of opposite chirality.
And this is a new type of pattern.
And in international notation, what do we call this?
It's a threefold axis.
But how do we indicate a symbol for three with inversion sitting on it?
Let's ask if we know how each of these objects is related to each of the other.
So here's 1, 2, and 3; 1 goes to 2 by A 2 pi/3; 1 goes to 3 by A minus 2 pi/3.
Let's put some numbers on here for the ones down below.
Let's call them 4, 5, and 6; 1 goes to 6 by an version.
How do I get from 1 up to 4 that's down?
I can do that only by taking two steps.
Rotate 1 from here to number 2.
Don't yet put it down.
First invert it.
So 1 to 4 involves the operation A 2 pi/3 followed immediately by inversion.
And I go from 1 up to 5 down by doing the operation A minus 2 pi/3 followed immediately by inversion.
And then 1 goes to 1 and itself by the identity operation.
So I have six objects, one, two, three, four, five, six operations.
Yes?
In the cases where you're rotating and inverting, does it matter which way to the other?
No, it shouldn't because they leave each other alone.
So I can rotate from here to here and invert.
Or I can invert from here to here and then rotate.
It's the same transformation.
Again, they permute if the two loci of the two operations leave the other locus alone.
Maybe the enormity of what we've shown here has not sunk in.
This is a new two-step operation.
We can't describe it any simpler than saying, rotate and not put it down yet, follow up by inversion.
Yes, sir?
Couldn't we express that in another way by sort of extending the three-directional glide plane by saying invert, then transform by some vector that's parallel to the glide plane?
Maybe they do in space group, but as soon as we introduce a glide plane, you've got an operation that's half a lattice translation.
And that means you've got to have a lattice translation and double the lattice translation, so--
Oh, we don't have to worry about that
--when we're in a space group, yeah.
That could be present.
But not for a point group because the ground rules are at least one point has to remain immutably fixed in space.
So this is a two-step operation, and what we're going to call it is rotoinversion.
It consists of as a first step an operation by rotating alpha from 0.1 to a virtual point number 2.
But before you put it down, you will invert it to a new object number 2 which is of opposite chirality.
So here then are the operation of the group that results when you combine a threefold rotation axis and add to it an version center as an extended; A 2 pi/3; A minus 2 pi/3; a rotoinversion operation through 2 pi/3 and then inverting; a rotoinversion operation of A minus 2 pi/3 followed by inversion.
And the symbol that is used to represent that new two-step operation is putting a bar over the top of the symbol for the axis.
And then, finally, we have inversion by itself.
So that's a group rank 6.
The Schoenflies notation is called C3i because we got this group by adding an inversion to C3, the threefold axis.
The international notation picks up on putting a bar over an axis to indicate a rotoinversion operation.
So this is called 3-bar in the international notation.
So there is a new group, and it is an oddball.
It sort of stands alone from the other groups of the form C3h.
This we derived by using the rotation of the threefold axis and adding 1-bar as an extender.
So there's no mirror plane in this
That's not the same as--
That's the same as 3/m, no.
3/m is C3h.
3-bar is C3i, different extender added to the same subgroup 3
What was the definition of 3/m?
Oh, we never really finished that.
That if we need the six operations that control the group, we'll have a sixfold rotoinversion axis.
But this pattern looks just like the triangle produced by 3, and we add an reflection operation as an inversion, and the 3 go down.
So if we ask how every one of the top is related to one underneath, that's by this horizontal mirror plane.
If I want to know how I get from this one that's up to this one that's down, then I've got to rotate through 60 degrees and invert.
So that would be a rotoinversion operation.
Let us to extend this idea of a rotoinversion operation.
And we would find this eventually in adding different extenders and falling headlong over this rotoinversion operation as we did here with 3-bar.
But let me in this case start by defining a 4-bar operation.
And this would contain the operation A pi/2 followed immediately by inversion.
And we'll call this step A pi/2-bar.
So let's try to do that and see what we get.
Start with a first point, number 1.
And that's up, so it's a solid dot.
And let's say it's right handed.
If we combine that with a rotation of 90 degrees.
Not yet put it down.
That's a virtual motif.
Before putting it down, we inverted it.
We would get one that's down, and it would be left handed.
Do the operation again.
I'll put the little tadpole inside.
Do the operation again.
Rotate 90 degrees and invert.
We're back up again.
So this was 1.
This is 2.
This was 3.
And that's up.
Do the operation again.
Rotate and invert.
And here is number 4, and it's down.
Do it a fifth time, and we're back to where we started.
So this is a crazy pattern.
It's a pair of objects that's up and a pair of objects that's down.
So there's a twofold axis in there.
That twofold axis A pi leave the pattern invariant.
But there is no way of specifying the relation between the two that are up and the two that are down other than doing this two-step process of rotating 90 degrees and then inverting.
So there is actually in this pattern a new type of operation analogous to 3-bar, and it's called a 4-bar axis.
And it's indicated geometrically by drawing a square because there's a 90-degree angular symmetry to this.
But a twofold axis inscribed inside of it because this is a pattern that has a twofold symmetry.
So something that has this symmetry is the symmetry of a tetrahedron.
And if we draw a line from the upper edge to the lower edge, this is the locus of a 4-bar axis.
International notation this is called 4-bar.
That's how we generated the pattern.
The Schoenflies notation is an S, little bit of exotica.
This geometric solid is something that's called a sphenoid.
And sphenoid.
Is the Greek word for axe.
And you can imagine a handle put onto this thing, and it does look kind of like an axe.
You could splits firewood with a thing like that.
It looks like a tetrahedron, but in a tetrahedron, it's either elongated along the 4-bar axis are squished.
It doesn't have to be regular.
So this is called S4, and the S stands for sphenoid
Is it part of a regular tetrahedron?
No, no.
A regular tetrahedron would be something where all of the edges had equal length.
And what we're doing is taking one edge and the edge that's opposite it and either stretching it or squishing it.
So there are two edges.
This one, and this one, which are the same length.
And then these four inclined edges have a different length.
It could be either elongated or squished.
But it's not a regular tetrahedron.
If those three distances were equal, then geometrically it would be a tetrahedron.
But strictly speaking, a tetrahedron is not a tetrahedron, just as a square prism with eight sides approximately equal can't claim to be a cube unless there's symmetry present that demands that this be true.
In this case, the 4-bar requires that these four edges inclined to the 4-bar axis be of one length.
And this have to have the same length.
But there's nothing that constrains all six to the edges to be of identical length.
So it's not a tetrahedron, so squished or deformed tetrahedron.
So there is another two-step symmetry element that we would not have been clever enough to think of had we not discovered this sort of rotoinversion operation when we added inversion to a threefold axis.
A 3-bar axis is a step that's present when you add inversion to a threefold axis.
So 3-bar, what we call it for short, is identical to a threefold axis plus inversion sitting on it.
A 4-bar is not equal to a fourfold axis with inversion added to it.
A 4-bar is something that you cannot describe any more simply than saying there is a two-step operation in there, and it's a group of rank 4.
Let me finish by setting up the task of going through this systematically.
We have 11 axial combinations 1, 2, 3, 4, 6, 222, 32, 422, 622, 23, and 432.
So there are 11 of those.
We want to examine as extenders a vertical mirror plane that would be one extender.
We should add that to each of these symmetries.
We already done a lot of these.
We've done pretty much up here.
We could add a horizontal mirror plane.
Or we've encountered inversion when we added a mirror plane perpendicular to an even-fold axis.
We could add inversion as an extender.
And to be complete, we should add to our list of axes in quotation marks, the 4-bar axis, having discovered it.
And does that do it?
Is there anything else we could do to these axes that would leave them invariant?
What about 2-bar?
2-bar; 2-bar would be rotate 180 degrees and invert.
So 2-bar is identical to a horizontal mirror plane.
So that's nothing new.
We're already running a little over time so-- Yeah?
3-bar?
3-bar is 3 plus 1, and we call at 3i, so this one down here.
This is 3-bar.
We describe it for short as that, but it really is a threefold axis with an inversion center sitting on it.
This thing is distinct because it's not a fourfold axis with inversion sitting on it.
A 4-bar is a 4-bar is a 4-bar.
You can't decompose it as a twofold axis is a subgroup.
That's only half the story.
I don't want to keep you anxious, not anxious to find out what the answer is but anxious to get on your way and go home.
So let me submit that when we have more than one rotation axis, such as 222 or as in 32, if we put the mirror plane in normal to the principal axis, we'll call that a horizontal mirror plane.
If we add the mirror plane through the principal axis, we could pass it through the threefold axis and the twofold axis, pass it through the vertical twofold axis and the horizontal twofold axis.
We will call this a vertical mirror plane.
And that's all we could say for a single axis, the mirror plane was perpendicular to the axis or passed through it.
But when there's more than one axis, another thing we could do would be to snake the mirror plane in between the twofold axis.
In that case, this twofold axis gets reflected into this one.
But I haven't created any new axes.
So that is going to leave the results of Euler's construction unchanged.
I can't similarly put a vertical mirror plane through this first twofold axis but in between the other two.
In that case, this is no longer 222 because these two mirror planes are equivalent by reflection, so I want to drop that at very least.
So in any case, without belaboring the point, it's late.
I could do for each of the groups that involved more than one axis I could add a diagonal mirror plane, or I should try to add a diagonal mirror planes.
And this means interleaved between axes that are present in these combinations, added such that no new axis is created.
But the addition clearly is going to be a new disposition of symmetry elements arranged in any different fashion and space.
So the game's afoot.
This is what remains to be done next.
What I'll do for next time is prepare a chart that looks like this that has the results of all of the unique combinations shown and then hand out pictures of stereographic projections of all the results.
I think once you know how to play the game to go through and do every single one in detail is probably not necessary.
If you know how to do some of them and you know all the tricks for adding extenders, you could do it if you had to.
All right.
So, again, sorry we started late and sorry that we last long as well.
Good afternoon and welcome back.
I'm glad you all came back even though there was a problem in finding a seat for every one last time.
I would like you to turn in the problem set if you've been able to do it.
If you haven't been able to do it, that's no great problem, but I hope you find it mildly amusing.
I have given that problem set out a couple of times over the years, and I can recall one very pale student who appeared at my door the next morning saying it's the first problem set, you've only talked an hour and I can't do it.
And then there was another group of three who formed a consortium to attempt to solve the code using a computer.
They didn't get very far either.
This is an interesting sort of problem because it requires you to think in a slightly different direction than you're accustomed to thinking, and therefore it's a little bit amusing.
This is not my creation, it came from a book by a man named Polya, the title of the book is Mathematics and Plausible Reasoning.
And he gives this as an example of a problem that can be solved only if you think in a slightly different direction.
I'll give you another example of a problem from his book.
Suppose, not that the problem arises in this era when you do all of your graphics on a computer console, but suppose you had to, in solving a problem in short notice, draw a circle that had a diameter of 4 inches.
And you went looking for your pair of compasses which you never use very often, and when you found them, they were rusted solid, and they were open to a distance of 5 inches.
OK, so there's the problem.
You have a pair of compasses that can only draw a circle that's 5 inches in diameter, you must draw a circle that's 4 inches in diameter.
What sort of construction, what sort of mapping out of arcs that you connected together could you do to create the circle of smaller diameter?
Anybody have an idea?
It seems impossible doesn't it?
Well suppose you got yourself a little block of wood that had a height that was equal to the square root of 5 squared minus 4 squared.
And so you have one end of the compass is up on top of the block of wood the other end of the compass traces out a circle that has a smaller diameter.
It's fairly obvious.
But what you have to do is to think of a problem that has poked your nose into two-dimensions and think of it in terms of a three-dimensional problem, and then the answer is easy.
So that was the sort of thought provoking thing that Polya presented in one portion of this book.
OK, if you enjoyed that problem, I have another one for you in a similar vein.
And perhaps you'll enjoy this one as well.
So while I'm talking, let me pass this around, I think there's enough for everyone.
Last time I got so caught up with the displaying the heft of the International Tables for X-Ray Crystallography that I forgot to mention entirely that there is another text that we will use in the class.
And this we will not need until halfway through the semester, and therefore I did not feel terribly remiss in not mentioning it.
It's a book by somebody named Nye, and it's called the Physical Properties of Crystals.
And this is published by Oxford University Press and the publication date of the original addition was in 1967.
This is a book that is really, I don't think I'm being extravagant in calling it a classic.
It is a beautifully written book.
The first 2/3 of it deal systematically with tensor properties, the particular sort of mathematics that is used to set them up, transformation of axes, and then looks at specific physical properties and numbers that have to be described in terms of tensors.
This is actually the third book in a sequence.
It was a book by Wooster that covered things very similarly, but the notation that was used for the tensors was not the modern current notation.
And then the subject started in the form of a third book, Woldemar Voigt.
And the title of this book is Lehrbuch der Kristallphysik.
And this was published back in 1910.
This is the first time anybody had anything to say on the matter.
It in fact is a big fat book that contains some topics that are not covered in Wooster and Nye.
Nye's book is beautifully written.
The first 2/3 of it concerns tensor formalism and the last 1/3, which we will not touch at all, deals with thermodynamic relations between different properties that are represented by tensors.
So I don't recommend you go out and buy this book until you determine whether or not you need it because I'll try to make the lectures self-contained and I'll have lots of notes and handouts.
The problem with Nye's book, and any book that is intensely mathematical, is that you can't jump in on page 73 to find the answer to a specific question.
Because when you go there, it will say as we showed back in Chapter 4, now what is he talking about?
So you go back to Chapter 4, and Chapter 4 says, starting with our definition of Chapter 2, and you have to go back and read Chapter 2.
So it's awfully hard to pick something out to answer a specific question.
You have to really go all the way through it.
The good news is that Nye's book has been published in paperback.
And it is available at the COOP and paperback means cheap, cheap, or relatively inexpensive.
No books are really cheap these days.
So we will cover material that's in there, we'll have a slightly different emphasis, but the notation and the general mathematics that's involved is in Nye's book.
The second thing that I mentioned last time is that there is a very, very nice and thorough and geometric treatment of crystal symmetry.
And I said that's the good news.
The bad news is that it's out of print, so I promised, what a guy, that I give you a Xerox copy of the first half of the book.
So here is the text that we'll use in the first part of the term.
I included at the beginning, the table of contents, so you can see each other topics that are covered in the book.
We will not go through all of the material that's covered.
There are a lot of different symmetries to be derived, and it turns out that if you get the general idea and you can summarize the results, there's no need to derive every single one of them.
It's nice to know that there's a place where you can find out how it is done if you really have a particular question.
Did everybody get a copy or are there a few who did not?
I made extras, OK, nobody in need of one, great.
OK, let me now start with a general rhetorical question.
Crystallography, as we mentioned last time, is the geometry of crystals.
It's the geometry of patterns and the sorts of symmetries that are in those patterns.
Now you might ask yourself, why should I as a material scientist or a physical scientist of some sort, worry about this stuff?
I'm not training to be a wallpaper designer, I'm going to do physical things.
I'm going to heat things and measure properties, and that sort of thing.
Well there are at least three answers to that question.
First of all, whether you like it or not, the arcane language of symmetry is the language that's used to describe crystals.
It's the language that's used to describe structures.
The normal thing that you do when you're trying to describe verbally a ball and pin model of the geometrical arrangement of atoms in a crystal is to say the red balls are at the corners of the cube, the green balls are in the middle of the edges, and the chartreuse balls are sort of tucked up inside one of the corners, but slightly closer to 1 face than to the other 2 faces.
The point I'm trying to make is that is a language that has limited utility.
It deals, it's capable of dealing only with the simplest sort of atomic configurations.
So there is a general language based on symmetry theory, based on group theory, that is universally used to describe atomic arrangements.
So instead of saying red balls at the corners of the cube and green balls in the middle of the faces, I can say space group 4 over m3 bar 2 over m, atom a in position for b, m3m, atom b in position for c, m3m.
That's what rock salt it.
And that is the way, not only it, but especially more complicated structural arrangements are described in the literature.
So this is the language of describing such arrangements.
And finally, sooner or later, I bet you that every one of you will be involved with some crystalline material, and the first question you will answer is what is its structure, what is the atomic arrangement.
That's where properties start.
And you'll go a book or a set of volumes that describe structural data.
There's a big long compendium of books that fill about that much of a bookshelf which are called structure reports.
They started a number of years ago to compile all of the structures that had been determined within a given calendar year.
They did a pretty good job of staying caught up back in 1915 and 1920.
And then as it became easier to obtain such results, partly due to the advent of rapid large computers, they fell further and further behind and I think now they are about 5 or 10 miles-- 5 or 10 years, miles as well.
But this is one of the places to go to look up, without going to the original literature, whether the material that you're interested in has had its atomic arrangement determined.
When you go there you're going to find the atomic arrangement, not in terms of red balls at one position on the cell, but you're going to find it in terms of the language of symmetry theory.
So one of the things I hope you'll be able to do by the time we finish this time together, is to be able to go to such literature and know exactly what to do and where to go to reconstruct the geometrical arrangement of the atoms.
OK, so hopefully you're at least mildly convinced that this exercise is going to be worthwhile.
Before we continue where we left off last time, I would like to say a little bit about the language in which these geometries are described.
And we mentioned last time without thoroughly demonstrating why that in a 3-dimensional space there are 4 basically different kinds of operations.
And one of these is something that all crystals must by definition display, and this is the operation of translation.
Analytically it can be described as a mapping in which every coordinate in a space xyz is mapped to a location x plus some constant, y plus some constant, z plus some constant.
And if you do the operation again, this would go to a location x plus 2a y plus 2b, z plus 2c.
A feature of translation that is unique to this particular symmetry transformation is that it has no origin.
If I have a pair of motifs that are related by translation, we could think of them as being related by a vector, magnitude and direction, that takes this motif and moves it to this location.
We said that more generally we should view these operations, not just acting on one little domain and space, but acting on everything.
So this implies that there be a infinite chain of motifs if the operation of translation is to be present.
Because only that infinite, doubly infinite string is consistent with all of space being mapped into itself.
Like any vector, there's no unique origin.
You could say it extends from here to here, or from here to here, or any other choice of translation, provided the direction and the magnitude are the same in every choice.
As a result, it's not really possible to specify the locus of this particular operation.
It has magnitude and direction, but no unique origin.
What we can do through a very neat device is to nevertheless, take some reference point and have each of these reference points separated by T, and have each motif lurking off in space in exactly the same location and distance from this point that we've constructed.
And this array of abstractions, so these geometrical abstractions, these points, are what are called lattice points, and they are a very neat summary of the translational periodicity of the crystal.
It is absolutely essential not to mix up these lattice points which are a construct that we have created, and the atoms themselves that are present in a crystal.
The atoms are atoms and they're not necessarily the lattice points.
Another way of saying that not all atoms of the same chemical species need be translation equivalent.
We'll see some examples of this later on, so do not mix up the atoms and the lattice points.
When I talk about the sodium chloride lattice, I mean an array of points in space that are located at the corners of a cube and in the middle of the faces of the cube.
If I talk about the arrangement of sodium ions and chlorine ions, that is the sodium chloride structure and not the sodium chloride lattice.
And then last time I apologized for usage so as not to appear hypocritical.
Everybody talks about lattice vibration, lattice energy, lattice dynamics, and so on, but that's a misuse of the term.
But nevertheless, it is much more musical than saying structure energy, structure vibration.
So we'll go on misusing the term lattice I'm afraid.
OK, let's look at another operation.
Here we change the sense of no coordinate.
Let's next look at an operation that might take xyz, and map it into minus xyz.
This would be a situation where if I set up a coordinate system, here's x, here's y, here's z.
What I've done is to take an object that sits off here, at a coordinate plus x, and I've changed the sign of x so that this object now sits off here.
This is exactly what happens when I take something and reflect it in a mirror.
And if that's not immediately obvious, it just so happens, not at all by accident, I brought along with me a mirror.
OK, here is one hand, and if you look in the mirror, there is the other hand.
It's the same distance behind the plane of the mirror, two coordinates have been left unchanged.
The two coordinates within the plane of the mirror, if that is my choice of the reference system, and one of them has been reversed.
Now I'll take the second motif out of the geometric construct, and I'd like to point out one very curious feature of the pair of motifs that's generated by this transformation.
They're both the same thing, clearly.
But no matter how I try, I cannot move one so that it coincides with the other.
And we intuitively appreciate this difference by saying we actually use our hands by analogy.
We say one is left-handed and one is right-handed, and they are not congruent.
The fancy name that's used to describe this relation is to say that they are enantiomorph.
Another term that's used, particularly in chemistry, is to say that they are chiral.
Which one is the left-handed one, which is a right-handed one?
This is what I call my left hand, this is what I call my right hand.
But can we distinguish them physically, any other way?
No, these are terms that have come in to regular use in both our everyday language and also in science because we use our hands instinctively as readily available examples of enantiomorphs, readily at hand, I might say to almost make a pun.
One of the really brilliant figures in physics was a man named Richard Feynman.
Recently deceased, Feynman gave a very famous series of lectures on science at Cornell University.
And he has one entire vector that was devoted to the difference between right and left.
And he comes up with a funny story, he pretends that the hero of the story is someone who's trying to communicate with beings in outer space and suddenly he gets lucky and he gets a response to his message.
And they work out a way to communicate and eventually they try to describe each other to the other individual.
Well what they look like?
Well we're bipedal, and we have two organs related by reflection that let us sense light and form images.
And we have an aperture through which we ingest things that can be metabolized.
And our circulation and body works because we have a pump on the left hand side that circulates fluids through our-- wait, I don't understand, what's left?
So then there follows along this course on how to define left in an absolute sense.
How do you describe to someone what makes your left hand left, and your right hand right without being anthropomorphic about it.
So it goes on and on and on and he gets into physical phenomena which are objective and independent of a human being.
And finally he comes to the anisotropic emission beta particles in radioactive decay.
And that depends on direction relative to the magnetic moment and that defines an absolute sense of right and left.
So that's something physical, it doesn't depend on the nature of the human being.
And then Feynman wraps up his story by saying, if finally our extraterrestrial being travels to space, gets out of his spaceship and he walks forward to greet you and you put out your right hand, and he puts out his left hand, get out of their fast because it means he is made out of anti-matter.
And nuclei of matter in this anisotropic emission of data rays shoot out the beta particle in one sense, anti-matter shoots out the beta particle in the chiral sense.
So it's a cute little story that emphasizes the problem in defining absolutely left from right.
But they're of opposite handedness that we can say.
Mirrors are interesting things and I brought along a couple of mirrors and they have very, very peculiar characteristics.
And I would invite you to come up and look at these in private because if I hold them up in front of you, you're not going to be able to see what I'm doing at all, although I kid myself that you can, and I walk around and show you what I'm doing.
Here is something scientific, it's a chemical compound carbon dioxide, and-- OK, and if I hand this down you can see carbon dioxide in the mirror.
Can you see that?
Uh oh, it's not reflecting-- sorry, I didn't turn it on.
Now I think we can get it.
And is it working now?
OK, now it's working.
You can see why this is a very special kind of mirror because it reflects only red letters and it leaves the black letters unchanged.
You're going to have to come up, I see some of you straining your necks, you'll have to come up and look at that in person.
But the black letters are completely unchanged, the red letters are reflected into letters of an opposite chirality.
It's a very special kind of mirror.
I've got another kind of mirror that works in a different way.
Where Is my other piece of paper?
OK.
This is an interesting mirror because it reflects only male names are not female names.
This was a very topical sort of mirror a few years ago when there was a lawsuit.
There was a college down South called the Citadel which would only admit male applicants and not female applicants.
So I claim that this was a mirror that I got from the Citadel because it doesn't change the male names, but does change, does reject or reflect, female names.
So you can play with this during our break.
But if I look at myself in a mirror, I take a look at myself.
If I wink my left eye, the in there winks his right eye back at me.
So I'm not really seeing myself, what I'm seeing is my enantiomorph.
Doesn't that shake you up?
You have never ever seen yourself in exactly the same way as other people see you.
You are only familiar with your enantiomorph.
Does that make a difference?
Well I'll bring in something that I put together and I couldn't put my hands on.
We are very, very sensitive to the symmetry in our faces.
And if they are reflected left to right, you surely are going to look different to the other person.
And the way to see that is to take a photograph of somebody and cut it down the middle, and put the two different sides reflected left to right.
And the expression on the person's face, and the general spirit that that image conveys is entirely different if you use the one half of the face reflected left to right, and the other half reflected left to right.
So think of this, when you look in the mirror, you see your enantiomorph and other people see you differently.
Let me ask you to scratch your head now.
Is there any time when, in point of fact, you may have seen yourself without being reflected into the enantiomorph?
Yeah
Picture?
Absolutely.
A Photograph or a TV monitor.
When you look at a picture on television, you can read all the signs, and they didn't make up special signs in reflection so that they'd look right when they photographed you.
So photography or a video camera does not change the chirality.
But I've got another way in which I can see myself in the exact same chirality.
And this I can't really convince you of, you'll have to try it.
If I put two mirrors together at 90 degrees, and then adjust them so that I am looking right the point of intersection so that my two images coincide, then I see myself in the normal way.
If I now blink my left eye, this guy blinks his left eye at me too.
This is really astounding, two mirrors at 90 degrees, if you look at yourself right at their point of intersection, give you a non-chiral image of yourself.
So I invite you to come up and try that, that's truly astounding.
So why should somebody in material science or chemistry care about chirality?
Does it really make any difference?
Let me give you a little experiment that you can try.
Suppose you have a little cell on a couple pieces of Polaroid, and the cell has a glass front and a glass back and you fill it with sugar solution.
And then you pass a beam of polarized light through the sugar solution, and what happens is that the sugar solution rotates the direction of polarization in proportion to the thickness of solution at the light is passed through, in proportion to the concentration of sugar.
The point of polarization gets rotated.
Now that's pretty curious, so you scratch your head about that.
Why does that happen?
Well, maybe I'd better go back and try it again.
And a day or two later, you go back and you repeat the experiment.
And once again, the point of polarization rotates, but it rotates in the opposite direction.
The reason for this is that if I was not careful in cleanliness and there were some little bugs lurking in the corners of that cell and when they sense the sugar solution, they said wow, free lunch, and they crawled out And gobbled it up.
It turns out, those guys can gobble up just the sugar of one chirality.
Sugar is a chiral molecule.
And in fact there is a product that's called invert sugar and this is sugar that is all of one handedness.
But everything in the world around us, everything from sugar beats to sugar cane to other things that make sucrose, manufacture sugar of one chirality, not mixed.
All chiral molecules that are produced by living organisms are all of the same kind chirality.
If we make them synthetically, there's no reason to favor synthesis of one molecule or the opposite handedness, so synthesized molecules are of equal proportion in the left-handed chirality and the right-hand chirality.
This means that in the case of pharmaceuticals at the very best, you are going to use only half of the product that you've made.
There is a pharmaceutical product that is prescribed for attention deficit disorder, this is called Ritalin, and only one chirality of the Ritalin molecule does anything for you.
The other part is just metabolized and doesn't do anything.
But there are other much more sinister cases.
There was a serious problem about 20 years ago, primarily in Europe, where a particular pharmaceutical thalidomide was prescribed for pregnant women, it was to act as a sedative.
Only one chirality of the molecule did this, the other chirality tragically caused birth defects.
So you have to be very careful about the chirality of the pharmaceutical molecule that you synthesize.
Another example, there is-- I don't remember the name of it.
This is something that is taken to-- this is something called Ethambutal which is used to treat tuberculosis.
Only the molecule of one handedness does this, the other one causes blindness.
That's really a sinister and antiomorph.
Then there's some even crazier examples.
Ibuprofen is a chiral molecule, and this in a most remarkable situation is a molecule which you're body converts to the molecule of the chirality that has the intended purpose.
So here your body is clever enough to change ibuprofen into the molecule which is the one that you need for its pharmaceutical effect.
OK, so mirrors are interesting things.
I would invite you to come up and play with the special mirrors that do strange things and see yourself as others see you.
And now I would like to continue on in this discussion to mention the ways in which we can represent a mirror plane in a graphic language.
This is what a mirror plane does, it changes the sense of one coordinate.
If there is a locus across which that transformation is performed, we would like first of all, an analytic symbol.
Some way of indicating the presence of that particular operation in the pattern, and a mirror is very descriptive so the symbol m is used to represent the presence of a mirror plane in a particular symbol.
We might want to indicate a specific operation.
There are only two operations in the case of a mirror plane reflecting left to right and reflecting right to left.
But there are other operations such as rotation.
If we have a 16-fold rotation axis, there is one operation that consists of rotating 1/16 of 2 pi, another operation that will also leave the space invariant that's rotating 2/16 of 2 pi, and so on.
So an individual operation is something that we will want to designate.
And for a mirror plane, something that is used commonly in physics is to use an operation sigma for a particular reflection.
This is not done in Buerger.
If you get into reading it, he uses m for both.
And then finally, it's going to be convenient when we have a pattern before us to use a geometric symbol to indicate in the pattern the locus of this particular operation.
And what we use in the case of a mirror plane is a bold line.
And that if this were the pattern and we wanted to indicate where the mirror plane was, or the mirror line in 2-dimensions that relates those two motifs, we draw it in thusly.
We began last time to examine the properties of rotation, but that's another sort of symmetry and that is a rotation which takes place about a rotation axis.
The symbol that is used to represent the collection of operations, the analytic symbol, is based on the fact that the angular rotation, alpha, has to be equal to some sub-multiple of 2 pi.
2 pi over n, where n is some integer.
And the reason for that I think is quite clear, if I take a particular motif and rotate through an angle alpha, if I am not rotating by some sub-multiple of 2 pi, I'll just go round and round and round and I will never get a finite set of objects that is separated from its neighbor by the same angular interval alpha.
This will only happen if alpha is an integral sub-multiple of 2 pi.
And the symbol that is used for the collection of operations that is usually embodied in a rotation axis is n, the same n that is in the denominator.
The symbol for individual rotation, so we mentioned last time we have to specify the location of the point about which we rotate, and we have to indicate the angle alpha through which we've rotated.
So A alpha will be an individual operation, and the geometric symbol will be an n-Gon which has the symmetry of the rotation axis.
So for a sixfold axis we would use a hexagon.
For a fivefold axis we will use a pentagon, for a fourfold, a square, for a threefold, a triangle.
Now an n-Gon with 180 degree rotation is a line segment.
And that would be easily overlooked and it's not very aesthetic, so here we indulge in a little bit of artistic license and fatten out the middle of the line segment to get an oval with pointed ends.
And that's the symbol for a twofold axis.
What about a one-fold axis?
One-fold axes exist anywhere, so you can sprinkle them around with reckless abandon.
A one-fold axis has no symmetry at all, but that is a very nice symbol to use for no symmetry at all.
So symmetry 1 is the absence of symmetry.
So it does come up occasionally in notation.
Now if you look at what we've done so far, we have a transformation that changes the sense of no coordinate.
We have a transformation that changes the sense of two coordinates, one coordinate, no coordinate, one coordinate.
Rotation is interchanging the sense of two coordinates in a plane, and in a 2-dimensional pattern, that's all there is.
But for a 3-dimensional space, we have the option of changing the sense of no coordinate, the sense 1, the sense of 2, or change the sense of all 3 coordinates.
So if this is x and this is y and this is z, and up here in space lurks my initial motif, if I change the sense of x, the sense of y, and the sense of z, namely take xyz and map it to minus x, minus y, minus z, what I'm going to do is to essentially turn the object inside out.
And if my initial one was right-handed, I will produce a chiral object, a left-handed object.
This is an operation which is called inversion.
In this operation of turning the object inside out if you will, is inverting it to a new location.
And this analytically is the exchange in coordinates provided the point of inversion is at the center.
The analytic symbol for inversion is 1 with a bar over the top, pronounced 1 bar.
And I'll have to leave to later indication of exactly where that notation comes from.
The individual operation is also called 1 bar, and the geometric symbol that is used to indicate the location of an inversion center is a tiny little open circle large enough so that you don't miss it, but not so large that it might be confused with an atom in a drawing of an atomic arrangement.
So in this case, we would adorn our sketch was a little circle at the origin if that was the point through which the space was being inverted.
So that, ladies and gentlemen, is our basic bag of tricks in 3-dimensions.
Let me point out that inversion can exist in 3-dimensions only because I have to have 3 coordinates to play with or else I cannot define the operation.
Suppose I have a mapping operation xyz that goes to minus x, minus y, minus z, and I get rid of z to make it 2-dimensional.
Then my transformation is xy going to minus x, minus y and that's exactly what a twofold axis does.
So inversion, when you throw out the third coordinate, looks like a 180 degree rotation.
So you need 3 dimensions in order to define that transformation.
If we really wanted to go crazy, we could go on to say what happens in 4-dimensions?
There should in principle be five different operations and yes, mathematically you can define them.
They're very difficult to draw because we have to have some sort of operation that take something and pulls it out of our 3-dimensional world.
We have no idea where it went and then all of sudden, [POP], it pops back into our space.
But mathematically there are cases when you need a fourth variable to describe the symmetry of an arrangement.
And this generally occurs in something called a modulated structure where there's a periodic change in some variable other than the atomic positions.
And let me give you two quick examples without going into it exhaustively.
One characteristic of an atom besides its location and its atomic mass and things like that, is perhaps a magnetic atom that has a magnetic moment attached to it.
There are magnetically ordered structures.
One of them looks exactly like rock salt.
And I'll draw just the magnetic cations which sit in locations like this.
And the magnetic moment here is up, the magnetic moment here is up, the magnetic moment here is down, the magnetic moment here is down.
So what I've drawn here is no longer the lattice and in fact, the lattice constant of this material looks like a rock salt as far as the atomic positions are concerned, but the magnetic moments have to be continued on in another direction and some extra distance.
Actually some examples of this sort of behavior is FeO, cobalt oxide, nickel oxide.
All of these cations are magnetic, they have magnetic moments which are ordered and the unit cell turns out to be when you take magnetic moment into account, a larger cell, a super cell.
There's a more interesting type of magnetic structure though in which the magnetic moment is inclined relative to some translation in the structure.
And the magnetic moments all lie on the generators of cones, but as you walk along the chain of atoms, the orientation of the moment rotates to different orientations.
There is a family of materials that are said to have cubicle spin structures in which the periodicity of the march of the magnetic moment around the surface of the cone occurs with a period that is in commensurate with the spacing of the chain of atoms.
So strictly speaking, this material does not have a lattice in this direction, so it's not a crystal unless you use a fourth variable to describe the periodicity of the orientation of the moment.
And one final one at the risk of carrying this too far, here's a pattern that is based on a square lattice.
It has a fourfold axis in it unless I make the pattern out of squares that are black and white, make a checkerboard.
This is now no longer a fourfold axis because I can't rotate 90 degrees and leave the pattern invariant.
So this is an example of something called a black-white symmetry, or a color symmetry.
And it requires more than just 4 operations to describe the relation between one motif and another.
We need a fourth operation, switching of a color from black to white, or switching it from white to black, that's a forth operation.
Again, within the confines of a pattern that exists in our space.
Yes?
Does that actually have-- so you're saying it doesn't add value to the [INAUDIBLE]?
I did say no rotational symmetry if it has a fourfold axis here but this used to be a fourfold axis, and that now changes into a twofold axis.
And then I have the problem of describing how this square is a square exactly like this square except for its color.
So I need then an operation which rotates 90 degrees and switches from white to black.
And then rotates 90 degrees again and switches from black to white.
So there's a fifth operation, a color change that is necessary in a 3-dimensional space or a forth operation, a color change in a 2-dimensional checkerboard for example.
So there are lots of nuances to symmetry theory, it's mathematics and the nice thing about mathematics is it's your ballgame, you could make up the rules and as long as you play according to those rules consistently, then you've got something that people can't quarrel with.
OK, I think my internal clock has just told me that it's five minutes of the hour and it's time to take our break.
Come up by all means and play with the mirrors if you'd like and we'll resume the lecture part of our discussion in 10 minutes.
Another strange observation about mirrors that I've never really understood-- the mirror plane is reflecting me left to right, so it looks as though a mirror has a grain to it.
It knows what line to reflect me back and forth.
But if I kept the mirror in fixed orientation and I lay down, the thing should reflect me side to side, but it doesn't.
It still reflects me from top to bottom.
So how can that be?
Why does a mirror plane, if I hold it in one orientation, appear to have a direction across which I'm reflected but it doesn't follow me if I move?
Know what I mean?
You have any explanation of that?
Hmm?
Rotate your eyes, too
Rotate my eyes.
I can roll them around.
I can't rotate in any other fashion.
That's strange.
I mean, you look at yourself every morning-- several times, perhaps, and you're reflected always left to right.
And if you turn the mirror, it doesn't reflect you top to bottom.
Or conversely, you can leave the mirror alone and you can-- why does a mirror plane just reflect you left to right?
Ah.
I'll let you stew about that one for a while.
And I can tell you that I know of three papers in scientific journals that tried to explain this.
I can give you literature citations, but you think about it until our next meeting.
Why does it know which way I'm oriented?
Ah, I can't think about it.
Alright, back to more straightforward questions.
We are now about to embark on a grand process of synthesis which will take us the better part of half a semester.
And we've identified our four basic operations-- four basic one step operations-- namely translation, reflection, rotation, and for the time being I'm going to look just at two-dimensional symmetries, so I'll leave inversion out of the picture.
It's only defined in three dimensions, and the logic which I will follow will be to first build up two-dimensional symmetries and then we'll turn them into three-dimensional symmetries by picking another translation that's not coplanar with the first two.
So we're going to look for the time being at just two-dimensional symmetries.
The nice thing about doing this is that the number of two-dimensional symmetries is relatively small, so we can derive them rigorously and exhaustively.
To do so in three dimensions is a much more time consuming and elaborate exercise.
It's no different in principle, there's just more work.
So if we do two-dimensional symmetries first, it's an easy case.
We can do it rigorously and completely, and then what we'll do is just look at a few examples in three dimensions and look at how the results are designated and tabulated.
So the first combination I will choose to make is one that I set up last time and should not have started when there was no time to finish it.
So let me take an initially pristine space and say that to this I'm going to the operation of translation.
That immediately implies a string of translations and a string of lattice points, but I'm just going to focus my attention on the first one.
Then what I said I'll do is add to the space a rotation operation A alpha.
I'm going to for convenience put it right at the lattice point, but as there is no unique origin to the translation, I can start and stop the translation anywhere I like.
So here's my translation.
The rotation operation is going to take that translation and repeat it at angular intervals alpha so that I get a radiating porcupine-like sheaf of translations all coming out of a common point.
Alpha, we observed, has to be a submultiple of 2 pi, so I will have a cluster of translations separated by the equal interval alpha-- angular interval alpha-- and I'm going to choose to focus my attention on just the first one.
So this, since it's repeated by rotation, has the same length T. And then I'm going to look at the other end of the translation and say that similarly I must have a set of equally spaced angular-wise translations all separated by alpha, and I'm going to choose to focus my attention just on this one.
So this angle is alpha, and this angle here is alpha.
And now there's big trouble in River City.
This is a translation, here's a lattice point, here's a lattice point.
This is a translation-- a lattice point sits up here.
This is a translation-- a lattice points sits up here.
Now I've got two lattice points eyeball to eyeball, and they jolly well better be separated by either the interval T or some multiple P times T, where P is some integer.
It would be quite all right if there were two translation separation or five translation separation, but it must be some multiple of translation or I have violated my initial premise that everything in this space is periodic at a translational interval T. So let me take this geometrical constraint and very quickly convert it into analytical form so that we can proceed to systematically find out what the possibilities are.
Let me drop a perpendicular down to the original translation.
So this distance in here is PT, this distance will be T times the cosine of alpha, and this distance in here will be T times the cosine of alpha.
So in analytic form, then, I can say that my original translation T is equal to T cosine of alpha plus T cosine of alpha.
That's 2T cosine of alpha plus PT.
And the first thing we can see is that the magnitude of T drops out because this construction and the constraint it embodies in no way depends on the magnitude of the translation.
So this says, then, that 1 is equal to 2 cosine of alpha plus the integer P. And if I solve for the value of cosine of alpha, cosine of alpha will be 1 minus an integer P divided by 2.
So there is the constraint that must be followed if my construction is to be self-consistent.
So we've got this now in a plug and chug situation.
So what I'm going to put down is the value of P, taking all possible integers for which a value of cosine of alpha exists.
I'm then going to evaluate cosine of alpha, which is 1 minus P over 2, and then I'm going to identify the n-fold axis that corresponds to that particular value of alpha.
Let's put in P equals 3, and this is as far as we got last time.
Well, if we put 4, then cosine of alpha is minus 3/2-- it's not defined.
If we drop P down to 3, then cosine of alpha is minus 1, 1 minus 3 over 2.
And the angle whose cosine is minus 1 is a-- let me put down the value of alpha and then the n-fold axis.
The angle whose cosine is minus 1 is 180 degrees, and that would describe quite nicely the rotational throw of a twofold axis.
If I let P drop down to the value 2, then I have minus 1/2 of the cosine of alpha.
The angle whose cosine is minus 1/2 is 120 degrees.
And guess what?
That's the angular throw of a threefold axis.
Let P drop down to 1, and then cosine of alpha is 0.
The angle whose cosine is 0 is 90 degrees, and that would be a fourfold axis.
Looks like we're done, except P could be equal to 0.
In that case, cosine of alpha is plus 1/2.
The angle whose cosine is plus 1/2 is equal to 60 degrees, and that's a sixfold access.
What about negative integers?
Minus one, will that work?
This says that cosine of alpha is 1, and the angle whose cosine is one is 0 degrees or 360 degrees.
And that would be no rotational symmetry all.
It's rather amusing that the trivial case of no symmetry at all also falls out of this construction.
So this is a momentous result.
We've shown that if you're going to have a pattern that has a repetition by translation in it, the number of rotational symmetries that can be added are either no symmetry at all, a twofold rotational symmetry, threefold, fourfold, or sixfold.
In other words, the axis-- no symmetry at all, twofold, threefold, fourfold, or sixfold, nothing else.
This tells you a lot about the shapes that you can see macroscopically on crystals.
You could have a crystal that had the shape of the trigonal prism, and that would be perfectly fine.
You could have a crystal that had the shape of an orthogonal brick that would have twofold axes coming out of the faces.
You could have a crystal in the shape of a hexagonal present, or you could have a crystal in the shape of the square prism.
But something thing like a crystal with a pentagonal cross section that would be a fivefold access-- that is strictly forbidden, because the external shape of the crystal has to reflect the internal symmetry among the arrangement of atoms.
There are lots of things in nature that have crystallographic symmetry.
There is a little cactus that looks like this with some little spines coming out like this on the top, and its proper name is astrophytum myriostigma, also known as the ornamented bishops cap.
Beautiful example of fivefold symmetry, but the cells inside of that cactus can not have the same size and shape and be repeated by translation.
There are flowers that have very common examples of fivefold and even sevenfold symmetry.
There's one little purple flower that looks like this that comes out in the spring, and that's called periwinkle.
Fivefold symmetry-- fine for a plant, but if you got down inside the stem of this flower, the cells cannot be repeated by translation.
There are astronomically high symmetries.
These big giant cacti in the Southwest, the saguaro, they have rotational symmetries that run from 18-fold, 19-fold, 23-fold, so that if you picked up one of these guys very carefully, because they're covered with spines, and if you could lift it-- which would be very hard because they weigh a couple of tons-- take that guy and rotate them through 1/27 of a circle, and if he had 27-fold symmetry, you could plop them down and you couldn't tell that it had been moved.
There is no crystallographer who can resist cacti.
All sorts of symmetry, even symmetries that violate crystal graphic symmetries-- wonderful textures and colors.
And they have another really remarkable property, which commends them as house plants.
If they die, this sheath of spines stays intact until someday when you're watering it, you brush against it and you poke a hole right through the spines and there's nothing inside.
So a house plant that dies and you can't tell for two years or so is a very good plant to have as a companion.
So cacti have all sorts of strange symmetries.
Fine, but you can say something about the internal structure of that flower or that cactus.
But this little almost trivial proof has told us something else about crystals, because the presence of translation imposes a constraint on the rotational symmetry that can be present, and the rotational symmetry tells us something about the nature of the two-dimensional lattice which can accommodate that symmetry.
So let's go through this list once more, and let's pay attention to the value of P. For a twofold rotational symmetry, what we would do would be to take this original translation, we put the twofold axis here, and that takes the translation and rotates it around.
So here's a lattice point here, one here, one here, and the twofold axis here, takes the translation and we rotate in a counterclockwise sense.
Here's another lattice point here.
P was equal to three translations, and I'll be darned if that isn't exactly what we have.
Three translations from this lattice point A-- let me put some labels up here.
This was lattice point A, this was lattice point B, and this was our translation PT.
Three translations, just as advertised.
What constraint does this put on a lattice?
None whatsoever, because all this says is that if you have a translation that translation must be repeated into an extended one-dimensional row.
So you can put this an any 2D lattice whatsoever.
In other words, the magnitudes of the two translations-- let's call them T1 and T2 are under no constraint to be related one to another-- and this angle between them, alpha, can be anything that it likes.
You could have a twofold axis, but that requires simply a lattice row parallel to T1 and a lattice row parallel to T2.
The next integer we hit was minus 1/2.
That was cosine of minus 1/2, this was P equals 3, and that corresponded to something that could accept a threefold symmetry.
So let me put a guide to the I in here and make an equilateral triangle.
If we translate and rotate up by 120 degrees using a threefold axis and then rotate minus 120 degrees about the other lattice point, that puts another translation up here, and now-- I'm sorry, this was P equals 2 And as required, this is PT, and that's equal to two translations.
This has put a constraint on the sort of lattice which can exist in this space because we have two translations, T1 and T2, which are equal in magnitude, identical in magnitude, because they're related by symmetry.
And consequently, we have defined a space lattice, a two-dimensional space lattice.
We'll call this T1 and call this T2.
This lattice has the constraint that magnitude of T1 must be identical to the magnitude of T2, and the angle between T1 and T2 is again identically 120 degrees.
Not 119.9, but exactly 120 degrees because there is a threefold axis in here that demands that that be so.
So this is a very specialized kind of lattice, restricted to have two translations identical in magnitude.
And if there's a threefold axis there, you should be able to find these two.
And if there's a threefold axis there, then the angle between these two specialized translations is 120 degrees.
Our next magic integer was 1, and that corresponded to a fourfold axis.
And if we do what we claimed we did in that construction, we'd put a fold axis here.
That takes T1 and rotates it exactly 90 degrees to a translation T2.
Once again, two noncollinear translations, so we have defined a two-dimensional net.
If we complete the cell, this is one translation.
PT is equal to one translation in here, and this angle is 90 degrees because it's produced by a fourfold axis.
So we have a very special lattice.
Again, the magnitudes of two translations are identical-- not approximately the same, they're identical-- and the angle between them is identically 90 degrees.
Only a couple to go.
P could be equal to 0, and that was the case for a 60 degree rotation, a sixfold axis.
So again, let's draw what came out of this particular special case.
Here's T1, here is T2.
This angle is 60 degrees exactly because there's a six-fold rotation axis here, a sixfold rotation axis here, and the rotation of 60 in the opposite sense gives us another translation here.
These two lattice points coincide and there is PT equal to 0T, and these two points coincide.
Now if I complete a standard unit cell with T1 T2 as I've done in other cases-- this was T1, this was T2, this was a translation which I'm not going to use.
So this is the shape of the lattice and these now are lattice points with a sixfold axis on them.
The dimensional specialization is exactly the same as I found for a threefold axis, T1 identical to T2.
If I pick this cell, T1 to T2 can be described as an angle of 120 degrees.
Exactly the same lattice that we found for a threefold axis.
So with this simple minded little construction we've found two profound things-- that there are five kinds of rotation axes, including the onefold no rotational symmetry at all.
And it turns out that there are one, a general lattice, a hexagonal lattice, and the square lattice.
There are three kinds of two-dimensional lattices that are required by these symmetry elements.
So these guys require that there be three lattice of different specializations that are able to accommodate them.
So let me call this a parallelogram net, and that has T1 not equal to T2, the angle between T1 and T2 general.
And this is a lattice that can accommodate either no symmetry at all or a twofold rotation access.
Then there was a net that I'll call the hexagonal net, and this had T1 identical to T2 in magnitude, and it had the angle between them, the angle between T1 and T2 as identically 120 degrees.
And this could accommodate either a threefold or a sixfold axis.
And then finally, the general net, which I call a parallelogram net, and that has magnitudes of the two translations not equal to one another.
They can have any values they like, and the angle between T1 and T2 is completely general.
And that's exactly what I had up here for the-- oh, we did that once already.
The one that I'm missing, the third one, is the square net.
Square net has T1 identical to T2.
In magnitude, the angle between them is exactly 90 degrees, and that is required by a fourfold axis.
Let me pause here to see if there are any questions.
Yes, sir
When you write on the last one with the sixfold, it's only 120.
You could have also written 60, correct?
Yes, I could have.
And this lattice is actually the same as what I found for a threefold axis.
I could pick either this or this as the cell, but the two translations in those two cells are equal.
And again at various stages along the way, we'll need a convention.
And if I have a net that looks like this, a parallelogram, whether a specialized parallelogram or not, I have a choice of two angles that I could use.
We call that alpha.
This is going to be 180 degrees minus alpha.
Which do I pick?
You need a rule, and the rule is that for labeling the cell, pick alpha so that it's greater or equal to 90 degrees.
That's pure convention, but you want to have a rule just like a language so people use the same words to describe the same thing.
Here we want to use the same geometry to describe one and the same thing.
The other convention is that there are no unique translations that define a net.
We could take linear combinations of these vectors and define the same lattice, so we need another rule for selecting the standard translations-- again, so that two people can do an x-ray diffraction experiment and report the results in terms of the same lattice, and this is a fairly reasonable thing.
This is pick the two shortest translations, and that clearly makes sense.
There's absolutely nothing at all to commend a cell that has this as T2 and this as T1 so you get a long, skinny oblique things.
Your natural inclination would pick the two shortest translations in the net.
OK, so these are conventions.
This has nothing to do with the nature of the symmetry or what makes it unique, but just so that people have one defined way of labeling things
Right.
[INAUDIBLE] my question was actually that--
It was a good answer, even if was not to the question that you asked
I said the sixfold and the threefold are exactly the same, but then I realized they are because there is no crystal that can have threefold symmetry without sixfold
You were doing fine.
You should have quit just before that last statement.
How can you have a hexagonal lattice that sometimes has sixfold symmetry and sometimes has threefold symmetry?
Well, let me give you an example for that, and it makes a very useful point.
Let me draw two hexagonal nets, and in this one I'll put a threefold axis.
So I'll have one motif here, I'll go 120 degrees away.
Here is a motif here, and I'll go 120 degrees away, and here is a third motif.
So these three guys form a triangle about this lattice point.
And we would have about the other lattice points at the corners of the cell exactly the same triangle of motifs, and the same thing over here.
So there is a pattern.
It has a hexagonal lattice, and it has a triangle of objects related by a threefold axis.
And now let me take exactly the same lattice, and now I'll put in instead a sixfold axis.
And that means I'm going to have a hexagon of objects, and it'll do something like this.
And I don't want to push my luck and try to draw that twice, but there would be a hexagon here and a hexagon of motifs here, another one at this lattice point, and another one at this lattice point here.
Same lattice, same shape, same dimensions-- although that's not critical-- but one of them has only a threefold axis.
One of them has only a sixfold axis in it.
Why?
Because I decided to put a threefold axis in this pattern, and I decided to put a sixfold axis in this pattern, but they both end up being contented and happy with a lattice with the same degree of specialization.
Now we will come as we progressed a little bit further that we have to go in two dimensions to the reverse situation.
We have a particular symmetry and it's happy with two different sorts of lattices with different shapes and different specializations, and that's going to come up directly.
Any other questions?
Yeah
In this example, the sixfold [INAUDIBLE] axis [INAUDIBLE].
It's just that [INAUDIBLE]
Yes.
You're saying that here hanging at this lattice point is something that has sixfold symmetry.
Here is something has only threefold symmetry.
The nature of the lattice and the symmetry that is in that lattice are two inseparable aspects of the pattern.
But often, as we've seen here, there's more than one possibility for a given lattice.
And as we'll see very shortly, for some other symmetries, for one given symmetry, there are two kinds of lattices.
But nevertheless, the thing to keep fixed in your mind is that we call this a hexagonal lattice.
Why?
Because this translation is equal to this one, and they are exactly 120 degrees apart.
That lattice can have that specialness only if there's either a threefold or a sixfold axis in it.
So the specialization of a lattice is inseparable from the symmetry that is in the lattice that demands that specialization of the lattice.
Conversely, a lattice can have the specialization only if you place in a symmetry which demands precisely that specialization.
So if you measure a lattice, and this turns out to be 119.99 degrees and these turn out to be 3.21 angstroms and 3.21 angstroms, that is not a hexagonal lattice, because there's no symmetry in there that demands that this angle be 120 degrees.
And that may seem to be an academic fine point, but we'll see that in due course the properties of a crystal depend on the symmetry of that crystal.
If the crystal has symmetry, the property also has to have that symmetry.
And it is the atoms inside the cell which determine the symmetry of the property, and the properties is one aspect of the symmetry that goes along with lattice dimensions and lattice angles.
Is that clear?
So let me say it again, because this is an important point.
The specialness of a lattice is inseparable from the symmetry that is existing in that lattice that demands that specialness.
So if you have a crystal with three orthogonal translations that is equal in length as you might care to measure, that crystal is not cubic unless there's symmetry in that lattice that demands that the edges of the cell conform to the geometry of a cube.
Any other questions?
Let's then in the time that's remaining look at the other symmetry element that could be present in a two-dimensional crystal, and that's the mirror plane.
So here's a mirror plane.
That's the one remaining symmetry element in two dimensions, and let's ask how we might combine in this space along with the mirror plane a translation.
If I just pop in a translation, and call this T1, and for convenience, I'll take the lattice point on the mirror plane.
Here's another lattice point that sits here.
The mirror plane acts on everything.
It's going to take this lattice point and flip it over to here, and it's going to take the translation that goes from the origin lattice point to this lattice point and give me a T1 prime that sits here.
And now I have two non collinear translations, so these have defined for me a cell that looks like this.
This is T1, this is T2, and this is some angle alpha between them.
That's a special lattice.
This is a lattice which has, just as in the hexagonal or square net, two translations that are identical in magnitude.
Why?
Because they're repeated by a mirror plane, and the angle between them-- the angle between T1 and T2-- is completely general.
It can be anything it likes.
It can close up to a very narrow angle or it can open up to an almost flat angle.
That's a new kind of lattice.
None of the preceding lattices could make that claim.
Now let me point out that this is for the first time a case where it would be much to our advantage to not choose a cell that contains one lattice point.
Let me put some dotted lines in here.
And let me submit-- these will also be lattice points-- that I could pick a larger cell that would have this as T1, it would have this as T2, and the angle between them now would be identically 90 degrees because of the geometry that gives us a rhombus here.
This would have two translations that are not equal in magnitude, and it would have an angle between these two translations that is identically 90 degrees, but it is no longer a cell that contains one lattice point.
It now catches a second lattice point that's in the middle.
So this is a double cell, and it has a rectangular shape.
It's a centered rectangular lattice.
Being a double cell that's redundant has twice the area that is unique in the pattern, and anything that's hanging up here is going to be hanging down here, so it has a twofold redundancy.
But the thing that you get in return for paying the price of that redundancy is a cell that has a right angle in it.
And as we'll see, we're going to use the edges of the unit cell as the basis of a coordinate system for describing what goes on at positions xy within the cell.
And the advantages of an orthogonal coordinate system, whenever you can take advantage of it, far outweighs the price of pain-- twice the area to describe the same pattern.
So this is what is generally taken as the standard cell.
It's a double cell.
But I think you're used to that sort of compromise, because you've all heard of face centered cubic lattices.
The primitive cell in a face centered cubic lattice is a rhombohedron.
But, oh, that Cartesian coordinate system is so great to use rather than something that has an oblique coordinate system.
So is this is a definition of convenience.
Notice the curious duality-- either special relation between the translations angle general or special relation between the translations, an angle special.
General translation, special angle, relation between the translation, general angle-- so, it's a curious sort of duality.
You can have one but not the other or vice versa.
So this is a fourth sort of lattice.
Number one, number two, number three, and now we have number four, which is a centered rectangular lattice.
And this particular lattice has T1 not equal to T2 in magnitude, but it has the angle between T1 and T2 exactly 90 degrees, and it's a double cell.
It's centered.
Are we done?
The reason I asked that silly rhetorical question is that obviously I suspect that we're not done, and what else might we do?
We got our centered rectangular net by starting with a translation-- starting with an mirror plane, really-- and then we added a general translation, and that reflected it across and gave us a diamond shaped net which we could define as a rectangular double shell.
Does that always happen?
Do I always get that oblique diamond shaped cell?
No.
Suppose I put in my first translation deliberately in a fashion such that it was at exactly right angles to the mirror plane.
That mirror plane will then reflect the translation and change its direction, and now I have generated a one-dimensional lattice row with translational periodicity T1.
And I've got a translation that's exactly perpendicular to the mirror plane.
How do I now make a space lattice, a two-dimensional space lattice?
And the answer is very carefully.
Suppose I throw in a second translation T2 and the mirror plane reflects it across to here.
This interval between lattice points up here is totally in commensurate with the first translation, and that won't work.
It's violated my initial choice of the translational periodicity unless I do one of two things, and let's try them both and dispose of this quickly-- this is straight away.
Here's my one-dimensional lattice row.
I do not violate this periodicity T1 if I do two things.
I could pick T2 so that it fell exactly along the mirror line, and that's going to generate for me a new type of lattice in which this is 90 degrees and these two translations are unequal in length.
So this is a lattice that has a rectangular shape.
It's a primitive rectangular cell, and it has T1 not equal to T2 in magnitude, and it has T1 and T2 exactly 90 degrees apart.
The second choice that would not result in any contradiction would be to have this as T1, and then pick T2 very carefully so that it spanned the mirror line with one half of T1 exactly up here and one half of T1 exactly here.
And this would be compatible with the separation T1 down at the start of this translation.
I think you can see that what this is going to give me is the centered rectangular net right back again, so this is the centered rectangular net-- nothing new.
But we did pick up one additional two-dimensional lattice of distinct character.
Number 5 is a primitive rectangular network, and it has the characteristics T1 is not equal to T2 in magnitude.
Just as in the centered rectangular net, T1 is at exactly 90 degrees to T2, but this is a primitive cell.
And that's it.
We really set up the ground rules for the geometry of a periodic two-dimensional space.
There are five kinds of rotation axes-- one, two, three, four, and six.
Each one requires one or more of the specialized two-dimensional lattices.
We have a case where interestingly two different symmetries are compatible with a lattice of the same specialization.
In the case of the [?
hexagon ?]
on that, either three or sixfold symmetry could require that.
In the case of the mirror plane we have one symmetry element, M, that can fit into two different kinds of lattices.
So in one case, the same lattice can take two different symmetries.
In this case, two different lattices can accommodate the same symmetry.
So that's the story for two dimensions, and we have just one final thing to do.
That is to add to the lattices that we have found, and there are five of them, and add to the lattice point the symmetries that we have found require them.
And there are a limited number of these-- one, two, three, four, or sixfold rotational symmetry in a mirror plane.
And when we have finished these additions, we will end up with a combination of lattice and symmetry, which is something that is called a plane group-- group because the operations that are present in the space follow the requirements of the mathematical entity called a group, plane because this is a distribution of symmetry elements throughout a plane and not just fixed at one particular point.
Let me finish with some general observation, and we will obtain some of these rules later on.
Suppose I take a pristine space-- and this blackboard is no longer pristine-- and I put in the space a first operation, and there's a motif in there.
This first operation moves around this first motif and gives me a second one, number two.
Then I say let me put in another operation.
I'd like to combine these things and see how many different combinations I have, so I put in operation number two.
Operation number two will take the second object and repeat it to a third object.
Number two is identical to number one, so this gives me a third object reproduced from the second by operation number two.
Now I have a space, and sitting in it are two different operations and three different motifs.
Motif number one and motif number three are the same darn thing because they've been repeated by symmetry steps, so there must automatically be some third transformation that is equal to the combined effect of going from one to two and then from two to three-- going from one to three directly.
So this is another truth about these symmetries.
Whenever you take two operations and combine them in a space, the net effect of those two operations is equal to a third operation.
So a question we're going to ask all along the way until we are at the end of the month-- if you take a translation and combine it with a mirror plane, what new operation has to arise?
If you take two rotation operations and put them together, what third net operation has to arrive?
If we have some general rules, then we can automatically say, OK, I'm going to take a square lattice and I'm going to put in a fourfold axis.
What else is going to pop up elsewhere within the cell?
We'll be able to do this systematically but fairly automatically.
So that's where we're going from here.
When we're done, we will have derived systematically and rigorously the sorts of symmetries that can combine symmetries such as rotation and reflection with a lattice, and we will know completely the different sorts of patterns that exist around us in two dimensions-- in floor tiles, brick work, wrapping paper, and plaid shirts.
We'll pick up at this point on Thursday.
Let me caution you we are going into territory that is not covered in Berger's book.
I've just passed it out to you.
What we've said in the early parts of the term are in there, but we're going to do things in a slightly different way and then return to his text later on.
All right, my trusty $5 Timex watch says it's five after the hour.
So I guess we might as well begin.
I have corrected all the problems that were turned in-- the puzzles, not problems, puzzles.
And I just yesterday got hold of the photographs of you few people.
And I'd like to hand out the papers individually so I can start to put names together with faces.
So I'll turn them back individually next time.
I have one additional problem set that for you.
And again, this is in the form of a puzzle.
Be assured that this is the last entertaining problem set.
From here on, we move into drudgery-- tedious, drawn out things that hopefully will teach you something.
But this is the last one that's done with a little bit of tongue in cheek humor.
This one is a little more subtle than the other two.
And for this one, I will hand out a solution because I would like to use these problems to make a couple of basic points about the nature of symmetry theory in crystals.
So see how you do on this one.
OK, also before we begin, he gets a little bit uneasy before a crowd.
But I brought a little friend in with me today.
And could someone hold the box, please, so I could take him out?
Sorry.
He gets a little bit nervous before a crowd.
Come on, take it easy.
Take it easy.
This is Rodney.
He's an old friend of mine.
Rodney is a crystallographic abomination.
Rodney has five-fold symmetry-- impossible in crystallography, an abomination.
Why did Rodney decide to do this?
Anybody have any idea?
Why not six fold symmetry?
He's got five-fold symmetry.
Yuck!
But why?
There's always a reason for things if you look carefully enough and know what questions to ask.
Yeah?
He had narrow symmetry
Yeah, he had narrow symmetry.
But he could do that-- what if you were square?
Why not a square starfish?
Why five?
He did this for a reason.
He's been doing this for probably a couple of million years, he and his ancestors.
There has to be a reason.
OK, let me turn him over.
And maybe that will give you a clue.
The backside is not nearly as nice as the front side.
And in fact, if you look at Rodney carefully, Rodney has sutures between these five arms.
OK, this is what he looks like from the top.
But if you look at him from the bottom, there are sutures between these arms that do this.
And these sutures are weak points in his structure.
And by having five-fold symmetry, he puts the suture between this pair of arms opposite the midpoint of the opposite rib.
And that gives him a great deal more structural integrity.
So when a large fish comes along and gloms onto one of his arms and shakes, he doesn't split in half and end his career prematurely.
What happens instead, at very worst, the tip of the arm breaks off.
And then, Rodney gets to scamper away and make more starfish or do whatever his purpose in our environment is.
If he had an even-fold symmetry, that wouldn't happen.
He would really be done in by a fish grabbing hold and shaking
What if [INAUDIBLE]?
That's a very good question.
If he had only three arms and a fish glomed on and one arm broke off, his mobility would be greatly impaired.
In a way, he would go hip, hop, flop.
Hip, hip, flop.
Hip, hip, flop.
Hip, hip, flop.
It'd be a pathetic thing.
So five is much more advantageous than three
Why not seven?
Why not seven?
That's a good question, too.
Would you believe that on the Australian Barrier Reef, there are starfish with nine-fold symmetry?
Quite odd.
That's really a creepy thing with all those arms flopping around.
But as many things in our environment in our natural world, there's a reason for things if you can only think what it might be.
So we gave you another example of a non-crystallographic symmetry, the large cacti of the Southwest, the saguaros.
And there are lots of other ribbed casts.
They have symmetries that range between 19 and 23 fold.
Why do you think they have this rib structure?
Why can't they have a nice, smooth trunk like a birch tree?
Any idea there?
I'll give you a hint.
They live out in the desert.
There's not much water
[INAUDIBLE]
Hm?
More surface area for the [INAUDIBLE]
Yeah, but that's bad.
That means they would lose their water.
So things that don't want to lose water through their surface-- you're right-- try to minimize their surface.
But you're onto something.
And it is exactly the reverse of that.
Water is scarce.
So when it rains, this cactus wants to soak up the moisture and store it.
If he has this pleated structure, he's able to expand.
If he had a smooth surface with minimal area, he would just pop.
And that would be the end of him or her, as the case may be.
"It" is probably the appropriate term.
But anyway, there's the reason for these many, many ribs.
It gives them the ability to expand, store water, and then contract when the water evaporates.
So again, it's always interesting to look at the world around us and see examples of things that are strange because it's the strange things that very often can lead to penetrating conclusions.
All right, last time we had talked about chirality in finite atomic structures, molecules.
And I had pointed out-- but did not make the obvious observation-- I cannot think of a more dramatic connection between structure and properties, which is what material science is all about, than the very, very different properties of molecules that are of opposite handedness.
We pointed out, for example, that many pharmaceuticals are of chiral molecules that exist in right handed and left handed forms.
And very often, one form of one handedness has a very different set of properties, and in particular pharmacological action, then it it's enan enantiomorph.
So in the case of thalidomide, one handedness of the molecule acts as a tranquilizer.
The other handedness acts as a source of birth defects.
One of the other ones less familiar to me, but this is [?
ethambutol.
?]
It's used as a drug to treat tuberculosis.
The molecule of opposite handedness creates blindness.
The drug Ritalin that is used to treat hyperactivity, one handedness of the molecule is absolutely useless.
So you only use half of what you pay for.
So it's of great interest to pharmaceutical companies to be able to synthesize a molecule of one particular handedness with little if none of the other one.
And I looked up a number since we met last time.
It turns out that in 1997, enantiomorph pure drugs were a $400 billion dollar industry.
So again, this is structure property relations in action.
So let me ask you another question.
It turns out that products that are derived from nature-- a good example being sugar, which is produced by sugar beets or sugar cane-- produces a molecule only of one handedness.
They are [?
enantiomer ?]
pure.
But every synthetic compound that can exist in right handed and left handed forms occurs with equal probability.
So how do you make a pharmaceutical of one particular handedness?
And this is an interesting question because I think it was two years ago the Nobel Prize in chemistry was given to three people for their work in being able to synthesize molecules of a given handedness.
Anybody an idea how you do it?
I don't have an idea how you do it, but what do you mean by one handedness versus--?
OK, left handed or right handed.
So my left hand is exactly the same as my right hand, but I cannot match them into congruence with one another.
And we don't know how to describe this other than we used the term derived from our physiology.
We say left handed and right handed.
So when I was referring to a molecule of one handedness-- and I'll bring in a Xerox copy of a couple of examples.
There are many examples of fairly common molecules which can exist in one form in which the hydroxyl group points out this way, then another form where the hydroxyl groups points out this way.
Sorry, you should have asked that earlier
So if it's not the same molecule, it would be that molecule but--?
Chemically, it is exactly the same concept.
But the way in which those component models are configured has the same number of side groups, the same number of appendages.
But they, don't have a better word.
One is left handed.
One is right handed.
I'll give you some examples next time.
Somebody else, you had a question?
I was just going to see if I could venture a guess how they do it.
I was going to guess they use an electric field that has its bearings to set the right handed course [INAUDIBLE] that would make [?
enantiomorphs ?]
or more advantageous than [INAUDIBLE]
That's not a bad idea.
Anything that could bias the energy of one configuration over the other would do it.
But actually, no reason why you should know how to do it.
If you knew how to do it, maybe you could have gotten the Nobel Prize instead of these other guys.
But actually, you do it through catalysis.
And you use a catalyst which itself has chirality.
And that chirality favors the formation of one molecule over the other almost exclusively.
And that's why living plants make-- or animals-- make chiral molecules in just one handedness as well.
So it's using a catalyst that also has some chirality
How do you get the one hundred catalysts [INAUDIBLE]?
How do you have it--?
OK, there are lots of materials.
Any crystalline material that does not have a mirror plane in it can exist in chiral forms.
And one of the classic examples of this is quartz.
Quartz is SiO2.
And quartz has actually only a threefold symmetry.
But it forms crystals that look prismatic, as though they were hexagonal prisms.
And if I draw two of these prisms side by side, you could not tell which prism was left handed and which was right handed.
But quartz ends up growing with a couple of little facets on it that look like this.
They spiral up this way in a crystal of one handedness and they spiral up in the opposite direction for a crystal of opposite handedness.
So the external faces on the crystal show you that this crystal shape cannot be mapped one into another.
They are of opposite handedness.
And they have very different properties.
And so here's an example.
I'm not answer your question.
I'm talking around it very adroitly in the hope that if I talk enough you will forget what you asked.
But how do you make a chiral catalyst?
Well, there are some materials if you prepare them in a single crystal form, once you nucleate one enantiomorph, that will continue to grow.
So I would presume one could do exactly the same with a catalyst.
Actually, the properties of quartz are strikingly different for these two different forms.
If you would subject the crystal to compression, this crystal would develop a positive charge on the surface.
This crystal would develop a negative charge on the surface.
That's a property that we'll discuss in detail later on.
That's something called piezoelectricity, literally pressure electricity.
So the nature of the charge induced on the crystal is different for the crystals of the two chiralities.
OK, any other comment or question?
OK, last time we had taken the first small steps in what will turn out to be a long and protracted process of synthesis.
We've covered by now some of the general notions of operations that can exist in a pattern, two dimensional pattern of wallpaper or fabric or a three dimensional pattern that constitutes a crystal.
And we said that in three dimensions, there are four basic different operations that can exist that tell you how one part of the system is related to another.
There is an operation of translation.
And this has all the characteristics of a vector, magnitude and direction.
We summarize that translational periodicity by a set of fiducial markers that are called lattice points.
And analytically, the transformation takes the coordinate x, y, z.
And if you do the operation once, maps this to x plus a, y plus b, z plus c where the translation has components a, b, and c in x, y, and z directions.
Another operation is the operation of reflection.
And what this operation does is to take the coordinates of an initial part of the space atom or location and maps it into some location where one direction is reversed in sign.
Which direction that is or whether it's a pair of directions will depend on how the reflection locus is located relative to the coordinate system.
Next, there was an operation of rotation.
And this involves repeating things at angular intervals.
Take the space or the location of the atom and rotate it through some angle, alpha, about some rotation, a.
And in a sense, this is a periodicity which is reminiscent of translation except translation strings things out in a line.
Rotation wraps up the directions about some central axis.
But it's also periodic.
Then finally, in three dimensions, there is another operation called inversion.
And what this does is-- to could describe it as such in literal terms-- take something and turns it inside out to a point.
So if the inversion point were at the origin, it would take x, y, z, and map it into minus x, minus y, minus z.
And then, we concluded last time with a definition of a vocabulary for indicating analytically a single operation, the set of operations that constitutes the symmetry element, and then a geometric symbol for the locus of that operation.
And for translation, we could represent a single operation by t. The set of operations could be-- and I'll use this customary set of braces to indicate the set-- this would be a one dimensional lattice, a set of operations, p, where p is an integer times t. The geometric symbol, it's convenient to use an arrow extending from one lattice point to another.
But I emphasize that there's no unique origin to the translation.
We cannot be entitled to say it sits here as opposed to down here or down here.
There are an infinite number of parallel directions that could specify a given interval between lattice points.
For rotation operations, the notation for a single operation in as much as we must specify two things, the point about which we rotate and then the angular interval about which we perform the rotation.
And so we need to specify two things, the location of the axis, a point a, and the angle, alpha, through which we rotate.
The set of operations, the rotation axis, is indicated by an integer, n, where the rotation part of the operation is 2 pi divided by the integer, n. So we would use, for example, 2 to represent a rotation axis where things were moved through 180 degrees, 3 for a symmetry where things were mapped through intervals of 120 degrees, and 22 for my good friend, the Saguaro cactus, which is left invariant perhaps by a rotation through 1/22 of 2 pi.
For the geometric symbol for a rotation axis, we will use a little polyhedron.
Triangle for a threefold axis.
Square for a fourfold axis.
For a twofold axis, the polyhedron that had that symmetry would be a line segment.
So we'll take a little artistic license and let the middle of the line segment bulge out a little bit.
Finally for inversion, for reasons that will not become clear for another meeting or two, the single operation is denoted one bar.
The set of operations of which there are only two, inverting and coming back again, also indicated by the symbol one bar.
And the geometric symbol for the locus is a little, tiny circle that's open large enough to be noticed, but not so large that it looks like an atom in an atomic arrangement.
So there's our basic bag of tricks.
And last time, we had already used some simple geometry to come to a rather astonishing conclusion.
Yes, sir?
Reflection?
Reflection, that's this middle one.
Whoops, not going to help.
Thank you very much.
I get excited and carried away.
OK, reflection, the second operation.
Single operation, although crystallographers don't often use it in condensed matter physics, a single operation is represented by the symbol sigma.
Symbol for a mirror plane is m, easily appreciated.
And the geometric locus across which things are reflected left to right and vice versa is done with a bold, solid line when you're looking down parallel to the surface of the reflection locus.
Then, I will now be caught up.
And I can set off on something new.
We had said that in a pattern, very often more than one of these operations are combined in a single space.
And one of the things that must be present-- if the symmetry of the entity we are discussing is a crystal-- one of the things that must be present is a translation.
And so I asked last time the question, what if we want to say that our space has a translational periodicity described by t. And then, I add to one lattice point a rotation operation, A alpha, that has to get translated to another location where that operation, A alpha, exists.
And if A alpha exists and I repeat that translation by a rotation alpha, I'm going to have a sheath of translations all separated by a distance alpha.
If I single out the one that is removed from the first by an operation of rotation A alpha in a counterclockwise direction and single out from that sheath another one that is alpha away in the clockwise direction.
Then, I have lattice points at the end of all of these translations.
But these two guys up here are also lattice points.
And therefore, I have mucked up to the entire construction unless I can claim that this distance between these two lattice points is either T or some multiple P times T, where P is an integer.
And then lickety split, before you could catch your breath in wonder, I dropped a perpendicular down to the original translation.
This distance in here was PT.
This distance on either side was T times the cosine of alpha.
And that led me to a constraint that alpha, that rotation operation A alpha, if it were to be combined with translation, would be restricted to values such that the cosine of alpha was equal to 1 minus an integer P divided by 2.
Wow, simple, but so incredibly profound.
What came out of this was that alpha could either be 0 or 360 degrees.
That was a one-fold axis.
Alpha could be 180 degrees.
That was a two-fold axis.
120, which would be the angular rotation of a three-fold axis.
90, that would be the operation of a four-fold axis.
Or 60, that would be the operation of a six-fold axis.
It's incredible.
Anything, any pattern, any crystal that is based on a lattice can contain only these five rotational symmetries including no symmetry at all.
So this says a lot about what the morphology of crystalline materials are permitted to be.
Then, we looked at the nature of the lattices that were described by these angular rotations.
And we found that there were only three types of lattices that were required by rotation.
These were two dimensional lattice nets.
One was a general oblique net which had two different translations, some arbitrary angle between them, two translations, T1 and T2, which were not equal to one another.
We would call this the oblique net.
Then, there was another net that was square in the sense that two translations, T1 and T2, were identical to one another-- not close, but identical.
And the angle between them was identically 90 degrees.
This oblique net could accept either a one-fold axis or a two-fold axis.
This was the specialized shape that was required by a four-fold axis.
And then finally, there was another net that looked oblique except it was a specialized oblique net.
It had two translations, T1 and T2, which were identical just as in the square net and an angle between them, which was exactly 120 degrees.
And this same sort of net could be compatible with either a three-fold axis or a six-fold axis.
Then, we took the only other operation that could be present in a two dimensional pattern, and this was the operation of reflection, a mirror point, m. And we saw that this by itself could require two different sorts of specializations in a lattice, a lattice in which two directions were exactly orthogonal but the lengths of the translations were unequal.
And then, a diamond shaped net-- and this was a new wrinkle.
This was a net that had two translations that were Identical in length and an arbitrary angle between them.
Or alternatively, if we chose to pick a redundant lattice, one that had two translations, T1 and T2 prime which were different but an angle between them that was exactly 90 degrees, so this one we would refer to as a rectangular lattice, a rectangular net, and this one as a centered rectangular net-- same shape, but an extra lattice point in the middle.
Why pick a double cell, which is redundant, takes up more, and doesn't convey any more information?
And the advantage comes when you want to analytically describe the relation between different directions or the positions of atoms within the cell.
In that case, an orthogonal coordinate system, even if it's not Cartesian, is of immense advantage as opposed to an oblique system.
OK, slightly out of breath, but that is everything we've covered to this point.
And you can find this material discussed in the first chapter, in the introduction, and then half of the second chapter of Berger's book if you want to read over it.
OK, any questions or comments?
All right, I wanted to say a little bit more about lattices and something with which you are no doubt familiar, but perhaps have some questions about why one does it in such an obscure, difficult way.
And let me introduce what's done by an analogy which we have around us in everyday life.
Let me draw a system of streets and avenues for a city that might be one like Boston where the streets were laid out by cows wandering around in search of forage.
And suppose somebody met you on the street and said, may I stop you here at point A and ask you a question?
How do I get from point A to point B?
That's not a vector.
That's a point.
How do I do that?
You would not say, go 342 meters to the east and 26.4 meters to the south.
That's perfectly logical that is admirably Cartesian but you wouldn't do it that way what we just said you would say go one block straight ahead, and then turn one block to your left, and then go down the street to your right and turn to your right again, and you'll be right there.
You can't miss it.
Of course, the person in Boston would miss it and would have to stop a second time and ask again and by a method of successive approximations eventually get to where he or she would like to be.
I submit if a system gives you a grid, it makes sense to use that grid as the basis vectors of coordinate system even if the intervals are different, in two different directions, and even though the intervals might be not vectors that are orthogonal to one another.
If the system is periodic and has this grid work to it, it makes sense to use that as the basis of a coordinate system.
So if we have a lattice that is oblique and there are lattice points defined by these two translations, T1 and T2, it makes great sense if you want to specify the vector that runs from this lattice point to this lattice point to do that by saying that this is a lattice point that is at the end of the vector 1 T1 plus 2 T2.
And that, just as a system of streets and avenues, is a lot more efficient than saying go 12.2 angstroms in the x direction and go 15.3 angstroms in the y direction.
What we have defined here is something that is called a rational direction.
And a rational direction is something that is going to be a direction that extends between lattice points.
And what's rational about it is that the coefficients in front of T1 and T2 that would appear would be integers.
And an integers is sometimes referred to as a rational number.
That would be quite different from a vector that extended from a lattice point to, let's say, some atom that is within the boundaries of this cell.
And this might be a radial vector from the origin to one lattice.
And that could be used to specify the atomic coordinates.
This is going to complete the dichotomy, a feature that is referred to as irrational.
Why?
Because the components of the steps along T1, x, and along T2, y, are going to be fractional numbers.
So you'll hit irrational notation for directions that go to different locations with a unit cell.
You'll use rational vectors in directions that extend between integral numbers of lattice points.
These directions occur all the time when one discusses the physical properties of crystals.
I'm sure in any class that you've had that's discussed mechanical properties, one of the favorite topics is to discuss the ways in which close packed metals can deform.
And this can be discussed easily in terms of one layer of close packed spheres sliding over another.
And one of the ways that the layer can slide to go from one set of close packed hollows to another is in a direction like this.
And that turns out to be a rational direction.
And these are directions in which a close packed metal will easily deform.
So directions of easy plastic deformations almost always involve rational directions.
So there is a unique application of these sorts of features in the discussion of mechanical properties.
Another thing that can happen is that a particular structure, if it is weakly bonded or relatively weakly bonded in one direction, if you come down with a chisel on that plane and give it a little tunk with a hammer, the crystal will fall exactly in half and give you two very smooth surfaces on either side of some plane.
And the reason is if you look down into the guts of the crystal and look at the atoms and how they're bonded together, there may be some direction of weak bonding between atoms that is very easily separated by some sort of mechanical force.
This is true even for crystals that are very strongly bonded.
The way in which people begin to facet diamonds, for example, is to sit for a month and study the diamond and then decide how you're going to put a straight edge to it and give it a little tunk so that it falls neatly into two pieces at the sides that you want to facet.
Doesn't always work, so you sit for a long time to figure out just what you're going to do.
I think diamond cutters burn out early and have a useful career that last perhaps 3 and 1/2 years.
And then, they're burned out.
I don't know that for sure.
But it must be a nerve wracking way of making an existence.
Not a far-fetched story.
At one time, I was doing a diffraction study on a material which was a low temperature face, very interesting, but formed stable only at such low temperatures that you could not make it by cooking the components.
But it had been found as a mineral in the Swiss Alps, all sorts of exotica attached to this particular material.
There was a particular place up in the Alps which had a very unusual chemistry involving lead and arsenic and thallium and sulfur-- not what you find in your usual backyard rock pile.
And there was something like 37 unique minerals that were found in this little spot that were known nowhere else on the face of the Earth.
And I was very interested in the atomic arrangements in one of them.
So I wrote to the British Museum that had found the sole supplier of this material that-- back in about 1901 or so-- I wrote and I said, do you have any crystals of this stuff?
Yes.
Would you like to study it?
And I said yes, I would.
Well, we'd be happy to send it.
But please be careful not to damage the morphology because it is a very rare material.
So I said fine.
I'm as careful as the next meticulous guy.
They sent two crystals.
The crystals measured-- one of them was 2/10 of a millimeter in diameter.
One of them was about 0.15 millimeters in diameter.
And I was supposed to take off a piece for study without damaging the morphology.
Man, I felt just like one of these Dutch diamond cutters.
I sat and I looked at those two little suckers under the microscope for, I think, a week.
And then finally, the moment of truth, and I took a needle and popped off a corner.
And it went into two pieces.
One went for a microprobe analysis to determine the composition.
One of them went to a single crystal x-ray diffraction study to determine the atomic arrangement.
Both analyses can be done on a very small piece of material.
So my story about the diamond cutter agonizing before looking for a cleavage or a fracture surface is not just made up fiction.
It is something that, when you work with crystals very often, is encountered.
OK, cleavage planes are also rational directions.
The cleavage plane of rock salt is legendary.
And it's nice, really, to spend an afternoon splitting it up just to see how neat it falls apart.
OK, how are we going to denote rational planes?
Talked about rational and irrational directions, and that's easy.
How are we going to describe rational planes in crystals?
And this is rather bizarre.
So let me tell you what you do, which is something you've probably heard before, and then explain why one does it this way.
Let's suppose we have three vectors that are the vectors we will use to define the lattice.
And if there is a plane hanging on one of these lattice points, everything is translationally periodic, so there must be a similar plane in the same orientation that passes through all the lattice points.
So let me look at a very special plane, namely one that cuts a lattice point at an integral number of translations, T3, and an integral number of translations, T2, and an integral number of translations, T1.
And I don't want any common factor here, so let's look at this point here.
And now, let's connect these lattice points together.
And that will define a plane uniquely.
And what is special about this point is that it hits a lattice point on each of these three axes.
And this is a plane that I'll call the intercept plane.
There will be a similar plane hanging on all of these other lattice points.
But those planes will not hit a lattice point on the directions of T1, T2, and T3.
This is the first one out from the origin that does that.
Now, we will use this oblique set of vectors, T1, T2, and T3, as the coordinate system for specifying direction and angles.
So let me ask a question now that's going to blow you away.
What is the equation for the locus of this plane If I use this oblique coordinate system based on T1, T2, and T3 as my coordinate system?
Holy mackerel.
What a way to wrap up a Tuesday.
Actually, it's very, very easy.
First of all, it has to be a linear equation.
So it's going to be linear in x, y, and z.
So something times x, something times y, something times z equals a constant.
That's going to define, if I use the proper coefficients, the equation for this plane.
And let me determine quite easily what those coefficients are.
What is the coordinate of this point, which sits on the plane and therefore must satisfy this equation?
Well, y is 0. z is zero.
So this is the point-- if I am a translations out from the origin, this is the point x equals A.
So if I'm three translations out from the origin, that integer is the coefficient in front of x because when y and z are 0, if I make the constants on the right hand side equal to 1, this is the point x equals a.
And it's nice to have one on the right hand side.
That's about as nice and neat and tidy a constant as you could have.
We do the same thing for this point at the end of two translations, T2.
let's say in general that for this intercept plane, we are two translations out along T2.
So for the point 0-- this point, 0-- if I put the number of translations, b, in front of y, This is the point y equals b when x and z are 0.
And in the same way, if we are C translations out along T3, C times z-- that integer-- is equal to 1
Can you explain it again?
[INAUDIBLE]
I'm sorry.
What?
[INAUDIBLE]
Yeah, I'm saying I'm going to [INAUDIBLE] my definition at a plane which cuts the direction of T1, which is what I called the variable x, at a translations out.
In this case, this is 3.
This is 2.
And this is 1.
So for this particular example, I would have 3x plus 2y plus 1z equals 1 because when y is 0 and z is 0, the point on the surface is-- 1/3.
Sorry.
Good question.
OK, that's a lot better.
I would've gotten in much deeper trouble if you had let me persist in that.
So thank you for the correction.
But now, I would like to have integers out in front here.
So let me multiply both sides through by A times B times C. And now, I do get something with integers in front of x and integers in front of y.
And this would be A times C, and integers in front of z.
And that's A times B equals-- and now on the right side, I would have a times B times C. So everything, every coefficient and the term on the right hand side are products of integers.
So they are all integers themselves.
OK, let me next ask the question these planes passing through all of the other lattice points that are contained within this little tetrahedron of volume are going to be equally spaced.
How will I claim that these planes are all equally spaced?
Well, that's easy just in general terms.
Here is a lattice point.
There's a plane passing through it.
There's some lattice point neighboring it that's out at distance, D. Here is another lattice point.
It has a plane passing through it.
There must be a plane out here on D. And I'm not going to be able to have the same environment for every lattice point unless all of these planes have the same spacing from one another.
So to say that they pass in lattice points in the environment of everybody lattice point is by definition identical is possible only if the planes are equally spaced.
Next question-- and I'm not going to be gutsy enough to try to demonstrate this in three dimensions, so let me do it in two.
Let us ask the question, how many planes are there hanging on all the lattice points between the origin and the intercept plane?
And let me demonstrate that for a two dimensional case.
And I will refer you to a drawing in Berger's book because this is not convincing unless you do it absolutely exactly.
And to do that in front of a live audience with chalk is not always the easiest thing in the world.
So here is T1.
Here is T2.
And let me look at this plane.
This is the intercept plane.
A is equal to 3 and B is equal to 2.
Now, what I'm going to do is to use plus and minus the translation T1 to repeat this intercept plane.
OK, I go minus T1.
And that's going to give me a plane in here.
I'm going to go minus T1 again.
And that's going to give me a plane in here.
So I have split the interval between the origin and the intercept plane into A parts.
OK so far?
Let me now use the translation T2.
And I will use the plus and minus the translation T2 to take this stack of A planes and repeat it B times.
And that's going to take the plane at the origin and move it up to here.
It's going to move the planes into stacks like this.
I will map the stack of A planes B times.
And that is going to give me A times B intervals except if a special condition applies.
And then, I will go back to it in a moment.
This is the equation of the intercept plane.
I will argue that the three dimensional analogy of what I've done is that there will be, if I let the translations T1, T2, and T3 go to work, there will be A times B times C intervals between the origin and the intercept plane.
So if this is the intercept plane and this is A times B times C times the spacing between the planes out from the origin, then the first plane from the origin is going to happen to intercept which is 1ABC to the first.
And ABC divided by ABC turns out to be 1 for any value of A, B, and C. So the first plane from the origin is going to have the simple equation be BC times x plus AC times y plus AB times z equals 1.
And the second plane out from the origin will have the same coefficients except the term on the left will be 2.
Third plane will have 3 on the right and finally the integer ABC when we get to the intercept plane.
OK, now I'm ready to make a momentous definition just as we get five of the hour.
B times C is an integer.
Let me define that integer by a single integer.
And just for the heck of it, I'll call it H. A times C is an integer.
Let me call that integer K. And AB is an integer.
Let me call that integer L. So using T1 and T2 and T3 as the basis vectors of my coordinate system, the first plane from the origin has the equation Hx plus Ky plus Lz equals 1.
And these three integers, H, K, and L, are said to be the Miller indices of this plane
Yes, sir?
Just on that figure, [INAUDIBLE] connect to the lattice points.
I'm trying to make sense of why did you put the planes in between them?
OK, what I did was I started with just these three hanging on the lattice point separated by T1.
And then, I move this by T2.
OK, so T2 has a plane in the middle of it.
And it's going to move that up to here and give me another plane.
If you just draw this out-- and you're going to get a problem that asks you to do this-- since these integers are mutually prime, the second translation is going to interleave planes between the first set.
And it gets even worse in three dimensions because the third translation, which is the axis C translations out, is going to interleave that set of AB planes C times.
And they have to be at equal distances.
Otherwise, the definition of the lattice point having identical environment is violated.
OK, these are the infamous Miller indices that are the downfall of all freshman who take 309.1.
It is not necessarily a very straightforward definition.
But the advantage of it is that it lets you analytically describe the locus of the lattice planes in a very, very convenient way.
Actually, let me tell you something maybe you have not heard before.
These indices were not invented by a fellow named Miller who was an Englishman who wrote a book, an early book on crystallography that incorporated this notation.
The guy who invented them and first proposed them was Auguste Bravais, a very famous French crystallographer-- mathematician, actually-- who gave his name to the three dimensional space lattices.
These are universally known as the Bravais lattices, 14 of them.
It's a great name.
It's music.
it rolls of the tongue-- Bravis, Bravais.
Actually, Bravias wasn't the first guy to try to derive them.
The first guy who tried to drive the lattices was somebody named Frankenheim.
He blew it.
He didn't get 14.
He only found 13.
So there's a moral here.
If immortality is your game, get it right the first time because everybody has heard of Bravais, Bravais.
Nobody has heard of Frankenheim.
And for that, I say thank god.
What a name!
It makes you think of somebody who's got plugs in his neck and walks around like this and has a green face.
Frankenheim-- bah!
Bravais, Bravais, it really sings to you.
OK, I'm getting silly so it's time to quit and take a break.
Come up and say hello to Rodney.
All right.
The end of this lecture generated so much excitement that we've continued on into the start of the next segment.
So I think we'd better begin.
What I told you, as several people were clever enough to observe, that the number of planes between the origin and intercept plane is equal to A times B times C, where the intercept is A T1 out plus B T2 out equals C T3 out.
That is true only if the numbers A, B, and C are mutually prime.
If it's something like 4, 2, 1, you won't get that number of planes.
You'll get a submultiple.
And the reason for that is that some of the planes, when you generate them by these three different translations in different directions, some of them are mapped on top of existing planes.
And I'm not going to attempt to prove this, but let me just tell you the result, that if A and B contain some common factor, p, and B and C contain some common factor, q, and in the worst possible case, A and C contains some common factor, r, then the number of intervals is equal to A times B times C divided by p times q times r. And probably the best way to show that is let you show, on a problem set in the very near future, to actually map out the planes in the two-dimensional situation, for a and b that are mutually prime, and then do the same thing for a and b that contain some common factor.
So this is the so-called Miller indices, the Miller-Bravais indices, and everybody hears about them, but it is not at all clear why one takes this very indirect and non-intuitive way of defining the orientation of a plane.
And the reason for this, ultimately, is that in x-ray diffraction, you want to integrate up the scattering of all atoms in a crystal.
So you want to integrate over a plane that contains the atoms, and this locus has a convenient form only if you choose these indices to find the orientation of the plane.
And what comes out of this, as I said to a couple of people during intermission, is that the amplitude of the scattered wave has some summation over all of the atoms.
And in that phase of that ray, that is e to the plus 2 pi i, hx plus ky plus lz.
Very, very simple form.
And it has that form only because the locus, the way this plane is defined was done very, very carefully.
So it's something beyond just simple geometry that forms a reason for it.
Let me say a couple more things in general about the Miller-Bravais indices.
They are very, very intuitive, counter-intuitive.
Let's suppose that this is a, and this is b, and this is c. And I put down here, again, the equations of the entire set of planes in the stack.
And let's look at the first planes and ask what the intercepts are.
The intercepts on x, y, and z are going to be 1 over h on x, 1 over k on y, and 1 over l times z.
So the first plane from the origin is going to be 1 over h of the a translation, 1 over k of the b translation, and 1 over l of the c translation.
So this gives us one way of determining, given a particular rational plane, show it relative to the three translations.
From the first plane from the origin, h is going to be 1 over the intercept on T1.
I'll call it a, which is what we usually do.
k is going to be 1 over the intercept on b, and l is going to be 1 over the intercept on c for the first plane from the origin.
So if you have a drawing of the set of planes relative to the translation, such as the one that I obliterated from just a moment ago, reciprocals of the intercepts of the first plane from the origin give you h, k, and l very easily.
The thing that is counter-intuitive is that, if you look out of context at the three indices, h, k, and l, you have the feeling that as l gets larger, the intercept of the plane on c should gradually creep up from here, to here, to here, and so on, as l gets larger.
Actually, the reverse is true.
As l increases, the intercept on c gets smaller and smaller and smaller and smaller, until finally, when l is infinity, the intercept is 0.
So it's counter-intuitive.
The intercept on the axis gets larger as the Miller index gets larger.
Let me give you a proprietary Uncle Bernie's secret method for determining Miller indices without stress and strain.
This works every time.
Find the intercepts of any plane.
And we know from these equations what those are going to be.
For the nth plane, the intercepts are going to be n over h, n over k on b, and n over l on c. Take the reciprocals.
And the reciprocals are going to be h over n, k over n, and l over n. And then find whatever integer, n, you must multiply those reciprocals by in order to convert them to mutually prime intercepts, and you've got the Miller indices in a no-brainer, automatic fashion.
Another bit of jargon.
We will also use the basis vectors to define locations and to define directions in a lattice.
To distinguish three integers that indicate a plane, and you all know this from previous experience, the parentheses says plane, an individual plane.
There are other times when you would like to use the indices of one representative plane to indicate the entire set of planes that are related by the symmetry of the crystal.
So let me give one specific example.
Suppose we had a cubic crystal, where these are the three translations that form the edge of the unit cell.
And let's say that this is a1, this is a2, and this is a3.
I'm using a1, a2, and a3, not a, b, and c. But let's not go there.
OK, these are lattice points.
And if I look at this plane that's perpendicular to a1 and cuts it at one lattice point, l, this would be the 1, 0, 0 plane.
And I would indicate that that is an individual plane that we're talking about.
But if this crystal is cubic, all six faces of the set, 1, 0, 0; 0, 1, 0; 0, 0, 1; and then the plane that's in the opposite direction, minus 1, 0, 0; 0, minus 1, 0; and 0, 0, minus 1.
These six planes define the surfaces of the cube.
So there are times, since these are equivalent to planes and will have the same properties, the same hardness, the same [?
edge ?]
bits and everything else, there are times when we may want to refer to the entire symmetrical set.
And we do that.
We call that set a form.
It is a symmetry-related set.
And you can pick any one you like.
Generally, you pick one with simple indices, like the one that is perpendicular to the translation a.
And then you indicate by braces, which is the symbol for set.
This means the set 1, 0, 0, and that would correspond to all six of these faces.
The number, if you do this in reverse, and I'm going to give you the indices, what are the separate faces?
That depends on the symmetry.
So to give you an example, if I have a unit cell that has three orthogonal translations, and this is a, and this is b, and this is c, so that there is no cubic symmetry to it.
This face, which would be 1, 0, 0, and the one that cuts the axes at minus a, minus 1, 0 0, those are going to be the only faces with a single 1 and a pair of 0's which are equivalent by symmetry.
So in this case, for this brick shaped unit cell, 1, 0, 0 would mean this pair of two faces, rather than six.
So to go from a representative face to the set of planes, you have to know the symmetry.
If you can write the planes on the basis of symmetry, take one representative and let that be the form.
So it leads one to observe that crystals undergoing plastic deformation have bad form.
Yeah?
Is that different from when you talk about a direction?
Yes, yes.
You will see in just a moment that we use the same system of three integers, but in order to distinguish which one he's talking about when one's thinking about integers with a different set of numbers.
Again, if we use the lattice as the basis of a coordinate system, another thing we might want to specify is the orientation of a rational direction.
And let me go to a two-dimensional case.
Assume that c goes straight up.
Directions of easy plastic deformation will very often involve integral number of translations.
For example, this direction here is obtained by going one translation in the a direction plus one translation in the b direction.
So we could use the pair of integers 1, 1, and if we are normal to see we'd use a 0 for the third integer.
But how are we going to tell whether we're talking about a direction or a plane?
And the way one does that is to use square braces to indicate one particular direction.
You may want to also indicate the location of a particular atom within the unit cell.
In order to do that, one could really use the location of a vector, the components of a vector, xa plus yb plus zc, the vector from the origin to that atom as coefficients which correspond to the atom location.
So if that is the case, the numbers are usually not integers.
And one simply uses the three integers in an unadorned fashion.
So the lattice translations give you the basis of a coordinate system, and we use those to specify features, rational or not, of planes and directions and coordinates.
OK, any questions here?
OK, we are next going to take the first small step in a process of synthesis.
We have found that there are, in two dimensions, five distinct kinds of lattices.
The oblique, the rectangular, the centric rectangular, the square, and the hexagonal.
Five kinds of lattices.
We've shown, to this point, that each of these lattices can accommodate a certain rotational symmetry.
1 or 2 here.
m and m for the rectangular and centric rectangular.
4, the fourfold axis for a square.
3 or 6 for the hexagonal lattice.
So what I would like to do next, after we make a brief direct diversion, is to combine with each of these nets one of the two symmetries which can be accommodated in that net.
And what we will have determined then are the patterns' lattice and symmetry, which can exist in two dimensions.
The sort of things we look at actually, like the pattern of square tiles on the floor, or the two-dimensional pattern that is on a fabric design.
So we'll derive those exhaustively.
The thing that results is something that is called a plane group.
And I'm going to take this through in detail because there are not that many combinations that can be made.
And the number is enough to count on your fingers and toes.
You've got to use both, so they're a fair number, but not a staggering number.
If we were to do this exercise in three dimensions, there would be 230 distinct combinations.
And that's too much for anybody to go through.
I know of very few people who work professionally in this area who've actually sat down and derived every single one.
If you know the principles of the derivation and know how you could go about doing it if you were forced to, that's sufficient.
The results are tabulated in an admirable compilation.
We mentioned that at the beginning.
This is the International Tables for X-ray Crystallography, Volume 1.
So what we should end up with is a way for you to go to this reference data and know how to interpret it and make use of it.
But this is a nice microcosm.
There are not many plane groups, and to do this exhaustively, will let us see the sort of reasoning that goes into it.
I have to tell you that this is not in Buerger's book.
We're going to do two-dimensional crystallography first, and then extend it in a third direction.
In fact, I know of no book that does this.
And so you're going to get a unique, handwritten document by none other than yours truly that will do this in detail every step of the way.
And I don't know of any other place where this is done.
Now, what is the origin of this name?
Plane, well, these are symmetries confined to a plane.
And then there's this mysterious term, group.
And the corresponding thing in 3D is something called a space group.
This is a language derived from a branch of mathematics that is called group theory, and it is exactly the language that describes things that we've mentioned conversationally, such as if you take two transformations and perform them in succession, they're going to give you a third that has to be present.
If you've got two present, there must be a third.
We talk about symmetries being self consistent.
If you have a rotation operation that is not an integral submultiple of 2 pi, it's not going to be self consistent because you go around, and around, and around, and you will never come exactly full circle.
So group theory is a branch of mathematics that deals with concepts of this sort.
So it's not very hard to develop, and it gives its ideas to the notation that is used in crystallography.
So let me say a few words about group theory.
What is a group?
A group is a set of elements-- And what are elements?
Elements are things-- For which a law of combination is defined.
And the nature of the elements can be very, very different things.
They could be numbers.
They could be complex numbers, and the law of combination here might be multiplication.
They could be matrices, for which the law of combination is defined as matrix multiplication.
Or they could be pots of different color paint, for which the law of combination is defined as mixing the two colors 50-50.
I don't know any practical application of that sort of group, but nevertheless, it follows the definition, a set of things for which a law of combination is defined.
And in addition, which satisfies three so-called group postulates.
And the group postulates are the combination of any two elements is also a member of the group.
The second group postulate is that the identity-- I'm going to call it just identity-- and let me define, identity is doing nothing-- Is also a member of the group.
And if we define the identity operation as I, this means that for every element, I followed by a or a followed by I, is the same as a by itself.
If the law of multiplication were just arithmetic multiplication, then number 1 would be the identity operation because 1 times any number gives you the number.
Any number times 1 gives you the same number back again.
And the third postulate is that, for every element, an inverse exists-- let's call the elements a and a minus 1-- such that a followed by a minus 1 is equal to the identity operation, and the inverse element followed by the original element, a, is also the identity operation.
So if the law of combination were defined as multiplication, you would have to find, for every number, another number, which multiplied by it, gives you the number 1.
OK, let me now illustrate with a couple of simple examples.
Let's consider the set of numbers 1, n minus 1.
The number of elements in the group is said to be the rank of the group, or some people like to use the term order.
And I don't like that term because second order or fourth order is a term applied to quantities which are not terribly important.
So we expand this except for higher order terms, which are negligible.
And I don't like that connotation.
So I will not use order, although it's sometimes used to talk about the rank of a group.
The rank or order of the group is simply the number of elements contained in a set.
So let's show that the numbers 1 and minus 1 constitute a group of rank two.
Is the combination of any pair of elements also a member of the group?
Well, to do this, it's convenient to set up an array called the group multiplication table.
And what you do is you simply put the elements of the group along a row.
And in this case, it's 1 and minus 1.
And then you put the same elements along a column, and then you combine them.
1 times 1 is 1.
Minus 1 times 1 is minus 1.
1 times minus 1 is minus 1.
Minus 1 times minus 1 is plus 1.
Sometimes, as I'm giving this part of the lecture, somebody passes down the hallway, and they hear 1 times 1 is 1.
1 times minus 1-- and they go in reverse to see what in the world could be going on in this high-powered mathematical lecture.
That's why I close the door.
OK, so any combination of elements is also a member of the group.
Is the identity operation present?
Yes because minus 1 times plus 1 or plus 1 times minus 1 gives you the same element back again.
So for every element in the group, the combination of any two elements is present.
For each operation, we've shown that an inverse exists and for every element there's an identity operation, and the inverse exists.
So the pair of numbers, not terribly exciting, 1 and minus 1 where the law of combination is defined is a group of rank 2.
I'll give you another example.
I won't bother to carry out all the terms, but the numbers 1, minus 1, i, and minus i, where i is equal to the square root of minus 1, and where the law of combination is defined as multiplication, constitute a group of rank 4.
With this simple introduction, I think one can see that symmetry transformations can be regarded as operations that are elements in a group.
And let me give you a simple example.
Let's look at a combination.
We still haven't considered the mind-boggling possibility of having more than one symmetry element present in a space at the same time.
But let's suppose we have two mirror planes that are completely orthogonal, and see what sort of pattern they would generate.
Here's a first motif.
If I call this mirror plane m1, and this mirror plane m2.
m1 is going to take this object and reflect it across to here.
m2 will take this object and reflect it down to here.
Take this one, and reflect it down to here.
And now I know how this is related to this, how this is related to this, how this is related to this.
But this one and this one are exactly the same thing.
And those two objects are not related by reflection.
If this is a right-handed one, this is a left-handed one, this is a left-handed one, and this is a right-handed one.
So if something relates these two, it has to be something that does not produce an enantiomorph.
The only thing that's possible then would be translation.
And these two guys are not parallel to one another.
You can't get this one by just sliding this parallel to itself.
The only thing that's left is rotation.
And, lo and behold, we can get from this one to this one by a 180-degree rotation.
And we can get from this one to this one by a 180-degree rotation.
So here's an example of a way in which we've combined without showing why this is possible, or how I got there.
Here's a combination of symmetry elements that gives us a possible arrangement of objects in space.
It's going to be convenient to have a notation to indicate such combinations.
And we call them Harry, George, and Sam, or some popular, affectionate name.
Or call them number 1, number 2, number 3.
But a nice notation is going to be a descriptive one which tells you what you have.
And what is done in crystallography is to indicate these possible combinations by a running list of the different kinds of symmetry elements that are present.
And here, we've got a twofold axis, for which the symbol is 2, and two mirror planes that are different and function in different ways in the pattern.
So this combination is called 2mm.
OK, that's getting ahead of our story.
But what I wanted to do now is to show you that this set of symmetry elements contains four operations that constitute a group.
There's the operation of a one-fold axis.
That's the identity operation.
There is a reflection across the first type of mirror plane.
And notice, now, the utility of having a symbol that indicates a specific operation rather than a symmetry element.
There's a second reflection operation, sigma 2, and there is a rotation operation through a 180 degrees, A pi.
So I'd like to show that these four operations constitute a group of rank 4.
So how do we show this?
We first set up the group multiplication table.
And then we simply combine these four operations pairwise.
Put the same 1, sigma 1, sigma 2, and A pi down this side of the array.
Doing nothing, doing nothing, is the same as a one-fold axis doing nothing.
Doing a reflection sigma 1 followed by doing nothing gives you sigma 1 back again.
Doing a reflection sigma 2 followed by doing nothing gives you sigma 2 back again.
And similarly, rotating A pi followed by doing nothing is the same.
If I go along the columns, doing nothing followed by sigma 1 is sigma 1.
Sigma 1 and 1 combined is sigma 2.
A pi combined with sigma 1 is A pi.
So far so good, but not surprising.
Doing sigma 1 and following it by sigma 1 is reflecting left to right across mirror plane 1, and then reflecting right back again.
So that is going to be a one-fold axis doing nothing.
Doing sigma 1 following up by sigma 2 would reflect across, and then follow up by reflection in the second mirror plane.
I get from the first one to the final one by the rotation A pi.
So this is A pi.
And if I do the first reflection, sigma 1, and follow that by A pi, that gets me from number 1 to number 2 to number 3.
And I get from number 1 to number 3 directly by the reflection operation, sigma 2.
OK, you notice what's happening?
I get the same four elements back again in every column or row.
Sigma 1, 1, A pi, sigma 2.
And if I rattle these off quickly, this as A pi.
This is identity, and this is sigma 1.
And this will be sigma 2.
This will be sigma 1.
And this will be the identity operation.
So the first group postulate is satisfied.
These four elements combined pairwise always give you nothing but one of these elements back.
An identity operation is present because the one-fold rotation axis is the same as leaving the thing alone.
And we've seen that for every operation, an inverse exists because 1 occurs once in each row and column.
So sigma 2 is its own inverse, and for sigma 1, sigma 1 is its own inverse.
For A pi, A pi is its own inverse.
So an inverse exists.
So we've got a group.
And this is another way of saying, in general terms, if we put two mirror planes and a twofold axis together in this specific fashion, these operations, when they go to work on an initial motif, reproduce a finite set of objects and not a clutter that just fills space and never closes upon itself.
Yes, sir?
I see why you have sigma 1, sigma 2, A pi in that group, but why do you have the 1, which is a conversion?
OK.
If I didn't put it in-- no, that's the identity operation.
This is a one-fold axis.
Just this 2 was a twofold axis.
So one-fold axis is a nice symbol for identity because it doesn't do anything.
It picks something up and puts it right back down where it came from.
OK. Yeah.
That's deceptive because 1, when we're talking about multiplication of numbers, also functions as the identity operation.
Yeah?
If all these symmetries are commutative, [?
that is, ?]
if they all commute, do you really [INAUDIBLE] bottom diagonal [INAUDIBLE]?
In general, yes
Generally?
Yeah.
Actually, this is a special group because if I have two elements, a, and take b and follow it by a, that is the same result as a followed by b.
And a group that has this character is said to be abelian.
There's a standing rotten joke in mathematics that asks rhetorically, what is purple and commutes?
The answer is an abelian grape.
I don't think it's terribly funny either, but if you had a gaggle of mathematicians here, they would chortle and double over in laughter.
Ha, ha, ha, abelian grape.
Well, I don't feel badly.
You react about the same way to some of my funny things, too.
But you get used to that sort of thing.
All right.
The feeling I want to leave you with at this point is the idea that group theory and some of its concepts are exactly what we're meaning when we grope for a definition of the fact that a certain collection of symmetry operation should be finite.
That is to say, it's finite.
The operations combined on each other have to give a finite number, must constitute a group.
And the main reason for being familiar with this is that group theory forms the basis of a lot of the words that are used to describe the combinations of symmetry elements.
For example, what we have derived here, the symmetry that's called 2mm, is something that's called a point group.
Point because there is at least one point in space that's left unchanged, and that's this point of intersection of all three symmetry elements.
And it's called a group because the individual operations of these symmetry elements, when combined according to group theory, can be shown to constitute a group.
The patterns that we get when we add symmetry elements to a lattice are groups in the sense, if you regard the pattern as extending infinitely in all directions, that the combination of any two elements gives you something that's also in the group.
But the number of elements is infinite.
For example, in the lattice, here's T1.
Here's T2.
Every other lattice point can be described as some combination of T1 followed by T2.
But the numbers in front of those translations can be infinite.
If we put a symmetry element in this lattice, like a twofold axis, for example, the translations reproduce this twofold axis to an infinite number of locations.
But yet, all of the group postulates are satisfied.
The number of elements in the group does, however, not have to be finite.
So that is another distinction that's worth making, that there are infinite groups and there are finite groups.
And the number of elements can be infinite.
Collection of symmetry elements that leave one point in space unmoved is called a point group.
If we do something like this, put a twofold axis in combination with a pair of translations, that is an infinite set of operations that acts on all of space.
This is something that's called a space group.
Another designation, distinction, that's sometimes useful to make are crystallographic point groups, such as this one, 2mm, as opposed to combinations of symmetry elements, which are perfectly lovely and which are valid groups, such as a combination of rotations of 2 pi over 5, or a combination of a fivefold axis with a mirror plane.
Perfectly lovely symbols, constitute groups, but these are non-crystallographic These would be non-crystallographic point groups.
Satisfy all the requirements of the group, but if it's to be in a crystal, the fivefold axis must be combined with translation, and that's impossible.
So you can have non-crystallographic point groups.
There are no non-crystallographic space groups because they are, by definition, something that involve translation yet constitute groups.
All right.
Set the stage for next time, so you all come back excited.
We have shown that there are a limited number of symmetry elements that are possible in a lattice, rotation 1, 2, 3, 4, and 6 in a mirror plane.
And these are now examples of what we would call crystallographic point groups.
These would be the sort of symmetry elements that are candidates for two-dimensional patterns.
But are these the only symmetries that can be added to lattices?
Could we not combine a mirror plane with a twofold axis?
The answer I say to that is, sure.
You bet you can.
You just did it for us.
But could you combine a mirror plane with a threefold axis, a mirror plane with a fourfold axis, a mirror plane with a sixfold axis?
The answer to all of those questions are yes.
So there are going to be spaces here, where there will be additional groups.
And when we're done, we will have the two-dimensional crystallographic point groups.
But now I have to be able to complete something.
If I take A pi and combine it with a mirror operation sigma 1, what do I get?
If I take A 2 pi over 3, a threefold rotation, combine that with a mirror plane that passes through it, what do I get?
Again, a characteristic of symmetry theory is that whenever I take a motif and repeat it by operation number 1 to get a second motif, and then I repeat that by operation number 2, I have three things that are identical.
And there must be some way, some operation, that automatically arises that tells me how number 1 is related to number 2.
Number 1 is related to number 3.
Tied in with group theory, this says that if you combine operation 1 in a space, if it's to be a group, if operation number 2 is present, you must combine those two steps, and whatever pops up must be a member of the set of operations that are present.
So we're going to have to come up with something that I call combination theorems.
And this is simply expressing the result of a combination of two elements in a group multiplication table.
It goes with a caveat.
If you're so excited about this stuff, you're going to run home and start reading Buerger's book as soon as you get there.
You can regard these symmetry transformations as operators.
And you're familiar with other sorts of operators like d by dx, d by dy.
And when you apply them in succession, you actually write it as a product.
d by dy followed by d by dx you write as d squared dxdy.
Or you write it as d squared dx squared when you differentiate twice.
You don't mean you take the differential of the function with respect to x and then square it.
You mean you do this operation twice.
So these are all operators.
And what we usually understand is that if you do d by dy first and then follow that with d by dx, the sequence of operation goes from right to left.
And I think that will come right quite naturally to you.
You do that with differentials and other sort of operators.
That is what we'll do.
Not Martin Buerger.
A very strong-minded person, and he did what he thought was sensible.
And he said we read things from left to right.
We read everything from left to right.
So I will be damned if I'm going to write d by dx, d by dy to be a sequence of operations that goes from right to left.
So all throughout Buerger's book-- and it's easy to get adjusted to, if he writes A dot B, he means B followed by A.
Whereas normally, in the calculus of operations, we will use, as the rest of the world does, this is A followed by B.
Trivial point, but one that can throw you for a loop the first time you see it in Buerger's book if you follow it.
Distinction does not matter if the group is abelian.
So that the order of operations doesn't make any difference.
OK, that's enough for one day.
Beginning next time, know we will start to begin this process of synthesis.
And the first thing we'll ask is, how do you combine a rotation operation with a reflection operation?
See you next week.
Having looked at the quizzes in a preliminary fashion, I come to the conclusion that some notes on tensors would be useful for the remainder of the term.
So at great pain and personal sacrifice I will endeavor to [INAUDIBLE] All right, let me remind you of where we were a week ago-- before a slight unpleasantness intervened-- and we had just begun to look at the properties of a very useful surface, the representation quadric.
And we said that to define it we would take the elements of a second rank tensor, something of the form A11, A12, A13, A21, A22, A23, A31, A32, A33.
And we'd use those elements as coefficients in a second rank, a quadratic equation, of the form Aij Xi Xj equals 1.
OK, so the Xi and Xj are the coordinates in our three-dimensional space.
And the set of coordinates that satisfy this equation-- setting the right hand side equal to a constant-- will define some surface in space.
And it's going to be a surface that is a quadratic form, so it is going to be one of four different surfaces that can be defined by such an equation.
One would be an ellipsoid, and in general this would be an ellipsoid oriented in an arbitrary fashion with respect to the reference axes.
And it would have a general shape, so the three principle axes of the ellipsoid would be of different lengths.
Then we saw we could also get a hyperboloid of one sheet.
One sheet means one surface-- continuous surface.
This was the surface that had an hourglass like shape that pinched down in the middle, and the cross-section, in general, would be an ellipsoid.
And the asymptotes to this surface would define a boundary between directions in which the radius was real, which meant a positive value of the property, and a direction in which the radius was imaginary, even though that may boggle the mind.
And if you take an imaginary quantity and square it, you're going to get a negative number.
So this surface would describe a property that was positive in some directions and negative in magnitude over another range of directions.
Then the third surface was a hyperboloid of two sheets.
And this was a surface that looked like two [INAUDIBLE] with an elliptical cross-section that were nose to nose, but not touching the origin.
And in this range of directions between the asymptote to those two lobes the radius was imaginary, the property negative and then there were a range of directions that would intersect the surface of either of these two sheets in there in those directions the property would be positive.
Then the fourth one-- which is encountered only rarely, but as we pointed out does exist in the case of thermal expansion as a property-- and this would be an imaginary ellipsoid.
This would be an ellipsoid in which the distance from the origin out to the surface was everywhere imaginary.
And this would be a property then that was negative in all directions in space.
And that doesn't seem to be a physically realizable thing, we pointed out that for the linear thermal expansion coefficient there are a small number of very unusual materials that when you heat them up contract uniformly in all directions.
Very remarkable property, but that does indeed, thankfully, provide me with an example of an imaginary ellipsoid quadric that does represent, indeed, a realizable physical property.
OK, this surface, we said, had two rather remarkable properties.
The first property was that if we looked at the radius from the origin out to the surface of the quadric, a property of the quadric is that the radius out in some direction-- that would be defined in terms of the three direction cosines-- the radius has the property that it is given by 1 over the square root of the value of the property in that direction.
Or, alternatively, the value of the property in the direction is 1 over the magnitude of the radius squared.
OK, so this is what gives us the meaning of the imaginary radii, the imaginary radii 1 squared would give you a negative property.
But again, I point out, it's obvious here-- but we have to keep it in mind-- that the value of the property and the magnitude of the radius are related in a reciprocal fashion, not only reciprocal, but reciprocal of the square.
So if this is the quadric A, a polar quad of the value of the property A as a function of direction would have its maximum value along the minimum principal axis and, correspondingly, the minimum value of the property along the maximum radius of the quadric.
So that is sort of counter-intuitive, you think big radius, big value of the property, no, goes not only as the reciprocal, but as the reciprocal of the square.
So the value of the property with direction is not a quadratic form any longer, it's based on a quadratic equation, but when you square the radius it's no longer quadratic.
Looks as though we have the value of the property as a function of direction, but looks as though we've lost information about the direction of the resulting vector.
The direction that we're talking about is always the direction in which we're applying the generalized force, the electric field, or the temperature gradient, or the magnetic field.
But the direction of the thing that happens is defined by the basic equation that gives us the tensor.
But as I demonstrated with you last time, that information is in the quadric also, and this is the so-called radius-normal property.
Which doesn't involve exactly what it sounds like, but what it tells us to do is a way of determining the direction of what happens is to go out in a particular direction, along some radius that will intersect the surface of the quadric at some point.
And then at the point where the directional vector emerges, construct a vector that is normal to the surface.
And so if this then were the direction of an applied electric field, the direction to the-- normal to the surface of the quadric where that direction intersected-- the surface of the representation quadric-- this would be in the case of conductivity the direction in which the current flows.
So everything that you care to know about the anisotropy of the property that's described by a given tensor, and about the direction of what happens-- the generalized displacement as you apply a vector-- is contained within the quadric.
But one caveat, this holds only if the tensor is symmetric.
And I think this is where we finished up just before the quiz, only if Aij equals Aji does the radius-normal property work.
OK, any comments or questions on this?
OK, let me now turn to a practical question of interpretation.
If you were indeed to measure a physical property as a function of direction, and you get a tensor that is a general tensor, A11, A12, A13.
A21, A22, A23.
A31, A32, A33.
All the numbers are non-zero.
You know that the diagonal values in the tensor are going to give you the value of the property along X1.
Or, in general, the diagonal terms give you the value of the property along Xi.
Aii gives you the property along Xi.
You have not the foggiest idea, if the quadric has a general orientation, what the maximum and minimum values of the property might be, and these might concern you.
Given the tensor that you've determined, what are the largest values, what is the largest value of the property, what is the smallest value of the property?
And if the crystal has any symmetry these directions might be the direction of symmetry axes in the crystal.
So for many reasons you might want to know the maximum and minimum value of the property.
And then for some applications for a real chunk of crystal that you've examined relative to an arbitrary set of axes, you might want to ask how should I orient that crystal so that I can cut a rod from it that has, let's say, the maximum or minimum thermal conductivity.
If you're using a piece of ceramic let's say, as a probe into a furnace, you wouldn't want that probe to conduct much heat, so you'd like the minimum thermal expansion coefficient, for example.
If you were making a window out of a transparent single crystal, it'd be a very small window, but you might want the smallest thermal conductivity normal to the surface, and therefore you might want to make your single crystal window oriented in a particular direction.
So I hope I've convinced you with enough straw questions that we can knock down easily that, yes, this would be an interesting thing to know.
So how can we do this?
Let me show you some simple geometry that let's us set up an equation for finding these directions right away.
And it'll be based on the following observation, that when the direction of, let's say, the applied electric field is a long one of the principal axes, then and only then is the direction of the generalized displacement exactly parallel to the radius vector.
We go off in any other direction other than a principal axis, the generalized displacement is not parallel to the radius vector out to the surface of the quadric.
We look at the equation of the property A11 X1 plus A12 X2 plus A13 X3.
This is the X1 component of the generalized displacement.
Similarly, A21 X1 plus A22 X2 plus A23 X3 is going to be the X2 component.
And A31 X1 plus A32 X2 plus A33 times X3, guess what?
That's going to be the X3 component of the displacement.
Now we said that these X's give us the direction of the displacement as we let the coordinates of that point X1, X2, X3 roam around the surface of the quadric.
When we land at a point on the surface of the quadric which is where a principal axis emerges, then, and only then, is the resulting vector parallel to the applied vector.
And this means each component of the resulting vector has to be proportional to a vector out to the point at that location, X1, X2, X3.
So what I'm saying then is that when we are on a principal axis the X1 component of what happens is going to be proportional to X1.
And the X2 component of the vector that happens has to be proportional to X2, and this is the radial vector out to that point.
And, similarly, the X3 component of what happens has to be parallel to the coordinate X3, the vector out to the surface of the quadric along X3.
So let me make this an equation, now, by putting in a constant for the unknown proportionality constant.
And what I'll do, just for arbitrary reasons, is call that proportionality constant "lambda."
So this, then, is a set of equations that will give me, if I solve for X1, X2, and X3, the coordinates of one of the three points that sit out on the surface of the quadric at the location where a principal axis emerges.
So let me rearrange these equations to a form that I can solve.
And I'll make the equations homogeneous by bringing lambda X1 over to the left hand side of the equation, and let me point out that this is the same lambda in all three equations.
There's a proportionality constant between the radial vector out to the surface of the quadric and each component of the vector that results.
So my set of equations will be A11 minus lambda X1 plus A12 times X2 plus A13 times X3, and that's now equal to zero on the right.
Next equation would be A21 X1-- and notice I'm not combining A12 and A21 into numerically the same constant, representing it by the same quantity, I'm just leaving them be separate and independent at this point.
And then the middle term here is A22 minus lambda times X2 plus A23 times X3 equals 0.
And the third equation, you can anticipate how that turns out, it's A31 X1 plus A32 times X2 plus A33 minus lambda, and that's equal to zero.
So this, then, is a set of linear equations and they're called homogeneous equations.
And that has solved our problem for us, all we have to do is solve for X1, X2, X3.
Except that if the nine coefficients Aij are all arbitrary and lambda is a constant this set of equations has only one solution, and that solution is X1 equals X2 equals X3 equals 0, and that indeed will wipe out every one of these lines.
And that is not a very interesting solution.
The only case in which this is not the only solution is that if the condition that the determinant of the coefficients is equal to zero, then we can find a real set of X1, X2, X3.
So the condition that a solution exists is that A11 minus lambda A12 A13, and A21 A22 minus lambda A23, A31 A32 A33 minus lambda, this determinant has to be equal to zero.
All right, so we know how to expand the determinant, we'll do a little number crunching, and I'm not going to write it down explicitly, but we'll have a term A11 minus lambda and this will be among other terms in the expansion A22 minus lambda A23 A32 A33 minus lambda and then there'll be another determinant that involves this term A12 plus its cofactor plus A13 times its cofactor.
Unfortunately none of these terms are zero in general, so I'm going to have three terms here.
If I expand this term, I'm going to get something in A11 minus lambda A22 minus lambda times A33 minus lambda and a bunch of other terms that I won't bother to write down explicitly.
The only point I want to draw at this particular juncture is to note that this is a third rank equation.
In lambda.
It's a cubic equation.
And what this means is that there are going to be three roots.
And lets call them lambda 1, lambda 2, and lambda 3.
And that's the way it should be.
There are three principle axes, so we should have three values of the constant lambda that we could put into this equation that will let us solve explicitly for the coordinates X1, X2, and X3, which sits on the surface of the quadric where our principal axis pokes out.
Three different principal axes, so this should be three solutions to the third rank equation that we've defined here.
OK I think you've all seen this sort of problem before.
The lambdas are called characteristic values.
Or, on the basis that things sound more impressive in German, these are called the eigenvalues, meaning the same thing.
So using nothing more than the properties of the quadric and some linear algebra we have stumbled headlong into an eigenvalue problem, which is a well known sort of mathematical problem associated with a large number of physical situations and a lot to do with physical properties in particular.
OK, now I have a question for you.
How we going to solve this silly thing?
I know-- well the answer is I poke it into my laptop, and out comes the answer.
But that's not satisfying, we should know how to do this if our batteries were dead and we needed the answer in a hurry.
Well, I know how to solve a second rank question.
If I have equation of the form ax squared plus bx plus c equals 0, I know the solution to that.
It's etched indelibly in my memory and it says that x equals minus b plus or minus the square root of b squared minus 4ac all over 2a.
Impressed you, didn't I?
You know why I remember that?
I learned high school algebra from a throwback-- a teacher who was a throwback to the Elizabethan period.
You know how when people go to a remote island they suddenly find an example of a species that everyone thought was extinct?
Well, to secondary education [?
Ms. Dourier ?]
was a species that had long gone extinct, but happened to be preserved at the high school that I attended.
[?
Ms. Dourier ?]
was a very tall woman, ramrod straight, with a pile of snow white hair on the top of her head.
She invariably dressed in a long, black skirt, and a white blouse that had puffy sleeves, and a collar with frilly things on the top, and, invariably, a black ribbon around the frilly lace collar top.
Her mode of enlightened education was to have all of the students in the class have a notebook that was open.
And one hapless member of the class was called upon to solve a problem in real time, and as the student recited we all wrote down what the student was saying in our notebooks.
All the while [?
Ms. Dourier, ?]
holding a ruler like a swagger stick, strode up and down the aisles, and woe be unto the poor student who faltered slightly, let alone get something wrong.
And that would bring a slap of the ruler down on the desk and a sigh of disgust, and then, all right, you take it, Audrey.
And then Audrey would begin to recite until she screwed up and we would all copy in our notebooks.
That tyrant so terrorized and intimidated a bunch of kids in a redneck, working class high school that we really learned algebra.
So to this very day, as a reflex action, minus b plus or minus the square root of b squared minus 4ac all over 2a.
But I don't have the foggiest idea how to solve a third order equation until I look it up.
You probably never had to do it, so let me, for your general education and amusement, pass around a sheet that tells you how to solve a third rank equation.
OK the first step is to convert an equation in, let's say, y squared plus y cubed plus py squared plus qy plus r equals 0 to a so-called normal form where you get rid of one of the terms.
So if you make a substitution of variables and let y be replaced by x minus the coefficient p/3, then you get an equation that has a cubic term, linear term x, and a constant.
And the constants a and b are combinations of p and q and r in the original equation.
And then the solutions, not well known to be sure, are there's a first solution x is equal to a plus b, and then, something messy, x2 and x3 are minus 1/2 of a plus b plus or minus an imaginary term-- those are the second and third roots-- i times the square root of 3 over 2a minus b where A and B are complicated functions of a and b.
Capital A and capital B are functions of little a and little b.
And then if the coefficients in the original equation are real, then you get one real and two conjugated imaginary roots.
If this combination of terms b squared and a squared are greater than 0, it is b squared over 4 plus a cubed over 27 equals 0.
If it's that, then you get three real roots, two of which are equal.
In the most general case that we would be encountering here is b squared over 4 plus a cubed over 27 is less than 0, and then there are three real unequal roots.
Unfortunately, if b squared over 4 plus a cubed over 27 is negative that term appears inside of a square root sign in the solutions.
So the only case that we'd really be interested in doesn't work for the solution.
But fortunately there's another solution and that's given at the bottom of the page, and if you really wanted to solve a third rank equation by hand, these solutions would do it for you.
But I think you'd much prefer to have your computer solve the equations for you, but let me show you a way of doing this without any computer, without solving any equations.
And it is a very clever method of successive approximations that's based on the properties of the quadric.
So let's suppose we have a tensor, and that tensor has some set of coefficients Aij, all of which are non-zero.
And I'll assume although this will work for other quadratic surfaces that the quadric has the shape of an ellipsoid.
All right let me now pick some direction at random, actually I don't have to pick it at random, but I'll show you what the shrewd first guess would be.
So let's say we picked this direction.
And let us find the direction if this is-- let's do it in terms of current and conductivity.
Let's let this be the first guess for the applied field.
That's clearly not going to be one of the principal axes, and let this be the first resulting direction of current flow J1.
So what I'm finding then is a J sub i in terms of an Aij times an E sub J, and this is my first result, and this was my first assumption.
Let me now let the direction of J be taken as the direction of the applied field for a second guess.
And we could normalize to a unit vector, but we don't even have to do that.
So let's simply say that my second guess for the electric field that I hope will point along one principal axis is E2, and E2 I'll take as identical-- let's say proportional-- to J1 from my original guess.
So this then would be E2, the second guess.
And if I find the direction to the normal-- to the quadric in that direction, this would be my second result for the current flow J, I should have put this in parentheses to indicate that these are not components of E and J.
And you can see what's happening, look at where I am, I started out here after just one iteration I have defined this as the potential direction of one of the principal axes.
So I'm going to find E1-- the direction of the field-- E1 that comes from sigma ij times Ji, I'll take my second guess E2 either as identical to J1 and this is going to give me then a new value for my second iteration, this would be J2.
And this process is going to converge very, very rapidly on the shortest principal axis.
As you can see in two shots I'm pretty close to being parallel to this direction, which would be X2 in my illustration.
Now if I wanted to-- if I wanted to see if I was close to convergence-- this is going to be awkward because if I don't normalize the magnitude of the resulting vector, J is going to get larger and larger and larger.
So periodically I would want to normalize-- take the magnitude of J, divide that into the components, and then I'd have a unit vector.
If I wrote a computer program to do this, I would do the normalization each time to test convergence.
Now, this is my kind of procedure, because if I'm doing this by hand and I screw up and I make a mistake and my answer is thrown off a little bit, if I continue to iterate, the thing will proceed to continue to converge to the shortest principal axis.
So I can make a mistake and it will correct itself and come back again, and that's my kind of solution.
Yeah, Jason?
It's going to give you the minimum value of the property, right?
No.
It's going to give you the minimum principal axis, that's going to be the maximum value of the property.
So that's one out of three, hey, that's not bad.
What do I do for the others?
Let me tell you without proof to find the maximum principal axis.
And that would be the minimum value of the property.
What you would do is exactly the same procedure, and you would operate on not the tensor, but on-- using as a matrix of coefficients-- the matrix of the tensor inverted.
And I assume you know how to find the inverse of a matrix, except that you don't have to even find the inverse, the inverse is going to be a collection of functions of the original elements divided by the determinant.
You don't have to worry about normalizing, all that's important is the relative value of these coefficients.
Yes, sir?
If your tensor is symmetric, can't you say that they'll just [INAUDIBLE] on to another?
That's what you're going to do.
But when you know just one, that won't work.
The other two are floating around somewhere and you don't know in what direction.
So if you do this, then you have two out of the three and now very shrewdly, as you point out, if we have two of the principal axes and the tensor is symmetric, then we can automatically get the direction of the third.
If it's not symmetric, then you have to use the set of equations, and you have three principal axes as eigenvectors, as they're called in eigenvalue problems.
And they do not have to be orthogonal, but you have the components in a Cartesian coordinate system so you can find the angle between those axes.
OK?
Yes, sir
With respect to whether the matrix is symmetrical or is not symmetric, the quadric is always-- has three principal axes at right angles right?
Only if the tensor is symmetric.
Only if the tensor is symmetric.
And let me put your question aside for about five minutes, and I want to take an overview of what we've learned about the geometry of the quadric and the symmetry restrictions that we can impose on the tensor in a formal fashion, and see how they compare and how one allows an interpretation of the other.
So I'm going to answer your question, I'd like to wait for about three minutes.
Other questions?
OK, again this is all very symbolic, I'm not giving you an explicit answer, I can't do it.
But what I can do you-- do for you is, ha ha, do to you.
I almost slipped and said that, what I can't do to you, is give you a problem that asks you to solve for the principal axes, for example of a property tensor.
I will be merciful and perhaps not have all nine coefficients non-zero, I will have one of them zero, or maybe two of them zero.
So again to try this it is very instructive and it will cement what the individual steps that I performed here actually involve.
Now, a question that I thought you were going to ask about principal axes is that in order to do what I did here I am using the radius-normal property, and the radius-normal property only works for a symmetric tensor.
So sounds like I'm swindling you, except that if I use this procedure, the radius-normal won't actually be identically parallel to the generalized displacement.
But it's not going to be terribly different from it, and I'm going to get something that again will converge towards the shortest principal axis.
And once I'm close to the principal axis it is true-- symmetric or not-- that the only three directions before which the generalized displacement is parallel to the radius vector are the principal axes.
But anyway this is a dicey situation if the tensor is not symmetric, and it probably comes as a great relief to learn that there's really only one property tensor of second rank that's known for sure to be non-symmetric.
So in most cases-- 99 out of 100, if not more-- we will be dealing with property tensors that are symmetric, so we could use this procedure.
So again this property converges remarkably well, and as I say it has the admirable quality of being self-correcting if you make a mistake, if you're grinding this out by hand.
But a computer program that you write that just simply takes the direction of J, normalizes it to a unit vector, applies that as the generalized force, finds a quote "J" as a second iteration, normalizes that, and then you can check to whatever degree of convergence you wish to have in your solution.
And it's a simple matter to set up a program to do this.
All right, I think we still have some time, so let me now put everything together.
And look at what we've seen of the geometric properties of the quadric, and the tensor arrays that we found for different symmetries.
For triclinic crystals, for which the recommended procedure is to leave by the nearest exit and work on something different instead, you would have nine elements altogether that are necessary to define the property.
I'll discuss this in terms of tensors that gives you an ellipsoid.
The argument is not different for hyperboloids of one or two sheets, and we can't draw imaginary ellipsoids, but an ellipsoid is a much easier thing to draw.
OK, triclinic symmetries, either one or one bar, nine elements in the tensor.
No constraints whatsoever on the shape of the quadric or on its orientation relative to our coordinate system.
So, let's total up now looking at the quadric how many degrees of freedom there are in this situation.
We have three principal axes, so those are three degrees of freedom.
The orientation of the quadric is arbitrary, so there are three orientational degrees of freedom.
And that comes out to six.
Supposedly nine degrees of freedom, what are the other three?
Again, if this is a general tensor, which is non-symmetric, the eigenvectors-- the principal axes that you have to choose to squeeze this thing into a diagonal form-- become non-orthogonal.
OK?
So you have another three degrees of freedom that specify the mutual directions of the eigenvectors-- the angles between them.
OK?
The directions in which you have to pick your coordinate system, X1, X2, and X3, that force this thing into a diagonal form is going to involve a coordinate system in which these three angles are not 90 degrees and are fixed if you're going to squeeze this thing into a diagonal form.
And that's it.
So we add these three interaxial angles in, they're a total of nine degrees of freedom for a general non symmetric tensor.
To convince you that these interaxial angles for the eigenvectors really are variables, let me tell you something that we may prove later as a recreational exercise, or I may give it to you on a problem set, it's not difficult to prove.
And that is the result that a symmetric tensor remains symmetric for any arbitrary transformation of axes.
That is from one Cartesian coordinate system to another.
Now a very salutary effect of this is that the tensor has nine elements.
And if the tensor is symmetric-- if you want to transform that tensor-- you only have to do the off-diagonal terms, because whatever it turns into when you change the axes, these off-diagonal terms are going to be equal to the off-diagonal terms in the new tensor.
OK?
So this means that only six-- if the tensor was symmetric in one coordinate system, only six elements have to be transformed.
Actually, it's better than that, you don't have to do six, you only have to do five.
And this is the last diagonal term that can be obtained from the trace of the tensor.
And that would be the trace of the tensor T prime after transformation because A11 prime plus A22 prime plus A33 prime has to be equal to the original trace T, which was A11 plus A22 plus A33 in the original system.
So if the tensor is symmetric, you really have to crank through a transformation for only five of the nine terms.
Now what is the relevance of this to what I just said?
If we have the tensor diagonalized, to a new form A11, 0, 0, 0, A22 prime, 0, 0, 0, A33 prime, that is a special case of a symmetric tensor.
So if you decide to go from the coordinate system that produced this diagonalized form to some other coordinate system, the new tensor that you're going to get is going to be symmetric.
But the thing was not originally symmetric, so how can that be?
The answer is you can put it in diagonal form only in a non-Cartesian coordinate system.
And, therefore, that is evidence that the reference axes-- the principal axes-- cannot be orthogonal.
Otherwise you could take the diagonalized tensor and crank it back to some arbitrary Cartesian coordinate, and suddenly it would go to a symmetric tensor when it was not symmetric to begin with.
OK?
QED.
So you can diagonalize a general tensor only if you take the axes along the eigenvectors, which cannot be orthogonal.
OK, I saw you raise your hand, I wasn't ignoring you.
That was it?
Good
I wanted to make sure that Cartesian [INAUDIBLE]
It's always nice to see the class one step ahead of what you're doing.
All right, let's look at a couple of more crystal systems in the form of tensors in those systems and show that that in fact does correspond to the geometric constraints on the quadric as well.
For monoclinic crystals-- and this is symmetry 2, symmetry M, and symmetry 2/M.
The form of the tensor when we took the coordinate system along symmetry elements was A11, A12, 0-- terms with a single three vanished-- A21, A22, 0, 0, 0, A33.
So this was the case where X3 was along the two-fold axis and or perpendicular to the mirror point.
OK, number of independent variables to describe this property is five.
And if we look at this in terms of a constraint and shape in orientation of an ellipsoidal quadric, says that one of the principal axes, if the quadric is to remain invariant, has to be along the two-fold axis and or perpendicular to a plane that contains the mirror plane, if a mirror plane is also present.
So what are the degrees of freedom here?
This has to be along one reference axis, x3.
And, indeed, this form of the tensor occurred only when the two-fold axes were parallel, 2 X3.
And this ellipsoid then was left with one degree of freedom.
If this is the direction of the reference axis X2, we have a degree of freedom in-- shouldn't have called these X1 and X2, these need not be the coordinate systems.
Looking down on the quadric along the two-fold axis, the section of the quadric perpendicular to the two-fold axis can have a general shape, and it can have one degree of freedom in the orientation of the principal axis relative to the coordinate system.
So we have three parameters to specify the principal axes, since this is a general quadric.
We have one degree of freedom in orientation, and that adds up to four, but we said five.
So what's going on here?
Again this is a question of where the eigenvectors point.
If this term is not equal to this term, in order to force the tensor into a diagonal form, you have to pick eigenvectors in the X1, X2 plane, which will force the tensor into a symmetric form.
That is, to make this term equal to this term.
It's only a pair of axes involved, so there's only one angle between these two eigenvectors, which is a variable.
So for a nonsymmetric tensor plus one angle between the eigenvectors.
So that comes out, a-ha, five.
And again the way to see that is I will never get the tensor into a diagonal form making all the off-diagonal terms zero when I'm starting with a tensor which is not symmetric, and I know that a symmetric tensor, even in the special case where the off-diagonal terms are zero, is going to go to a symmetric tensor in another coordinate system.
So I know that there has to be an additional degree of freedom.
OK, the rest come very, very easily, so let me take just a couple of minutes to finish up this stage of the discussion, and we will shortly go on to something different.
The next step up in symmetry is orthorhombic.
And if we pick arc-- and this could be 222, 2MM or 2/M 2/M 2/M.
We found that when we took X1, X2, and X3 along two-fold axes that the tensor took a diagonal form A11, 0, 0, 0, A22, 0, 0, 0, A33.
That says that the quadric if it's an ellipsoid-- or any other quadratic form-- has all three of its principal axes along the reference system X1, X2, X3.
So the only degree of freedoms here-- and things are again starting to set up like supercooled water-- the only degrees of freedom are the three principal axes.
And that's it, the orientation is fixed.
And, moreover, the tensor is diagonal in this reference system-- it's going to be symmetric in this reference system, it's going to remain symmetric in any other coordinate system.
So the tensor is always symmetric, the eigenvectors then are always orthogonal, even if the coordinate system stays Cartesian.
And there are three variables, three degrees of freedom.
And finally two remaining cases can be disposed of quickly.
We saw that for an arbitrary theta that is a symmetry element-- so let me not say a symmetry-- for an arbitrary rotation about X3.
This was the ingenious little proof in which we transform the tensor by an arbitrary rotation theta about X3.
We found that the form of the tensor was A11, A12, 0, and now I get to use tensor notation by calling this term A12 again and not the proper indices A21, so I'll put quotation marks.
"A22" was equal to "A11."
So going to an arbitrary rotation theta this must be the form of the tensor we discovered, and this will cover then fourfold axes, three-fold axes, and sixfold axes.
These two elements are constrained to have the two values, these two are constrained to have the same values, so the tensor is always symmetric.
Symmetric relative to these axes, symmetric in all coordinates, and any choice of coordinate system, then.
And that means that if the quadric has this property, it's going to have one principal axis along X3, it's going to be circular in the plane that contains X1 and X2, so there are only, count them up, there are only three degrees of freedom.
In terms of tensor elements, these degrees of freedom are A1, A12-- in the diagonalized form-- and A33.
And there are three degrees of freedom in terms of the independent elements as well.
So this is for anything that involves rotational symmetry other than 180 degrees.
And, finally, for cubic crystals, if a symmetry operation other than 180 degrees gives us this form of the tensor, two off-diagonal terms are zero, two diagonal terms equal.
If we impose this along all three reference axes, then the form of the tensor is going to go to a diagonal form with all diagonal terms equal.
The quadric that corresponds to that form is a sphere that says that there's one quantity, one degree of freedom in the form of the tensor.
There are zero degrees of orientational degree of freedom.
Thus the spherical quadrics stay spherical for any orientation.
So again, one independent element in the tensor, just one degree of freedom in the quadric-- namely its radius-- and, again, the formal constraints that we obtained by symmetry transformations agree with those that are the same degrees of freedom when you want the quadric to coincide with the axes.
Yes?
Speaking of the circle and the exponents to play only if A12 equals zero?
These two were equal, but non-zero.
These two are equal, this one was independent.
And that would be for the specific case of a fourfold axis.
Now if we put a fourfold axis along X1, that's going to involve these two being equal and the non-zero elements become-- let's see-- along X2 it would be A13 and A23.
No.
OK, we put the four-fold axis along X2, then this one would be A22, A11, and A13 would have to be equal, and 21 and 12 would have to be zero, 23 and 32 would have to be equal.
I think that's the way it'll go, and these three have to be zero.
So when you impose all three constraints, here we've had these two equal, now we have these two equal, put the four-fold access in the next-- in the remaining direction, and again it has to be diagonal, and pairwise all of the off-diagonal terms will be required to be zero
I mean, when you diagonalize your matrix the diagonal term are going to be all different in this case if A12 is different from zero
This is not a final result, this is an intermediate step.
So we impose this constraint, plus this constraint, and that's actually going to give us all the qualities we're going to get, but we'd want to do the same thing for a four-fold axis along X3.
You put them all together, the fact that these are zero will wipe out all of the off-diagonal terms, and these things-- for another choice of axes, these two would have to be zero.
And for another choice of the orientation of the four-fold axis, this and this would have to be equal.
And, finally, this and this would have to be equal.
So this is what we found, in fact, when we cranked through the symmetry transformations, and this is what we would get when we said that the quadric has to have a shape that conforms to cubic symmetric
My question was in the case of the arbitrary relation of those three.
All three degrees of freedom, are they, in fact, the three principal axes?
For this case here?
Yes
For a symmetric tensor, they are two principal axes.
OK?
If the tensor is nonsymmetric, then these two don't have to be equal any longer, and they give you the extra degree of freedom.
OK?
What are the three degrees of freedom in this case?
I just want you to know I appreciate your question
May I make a suggestion?
Yeah
I think you need two degrees of freedom to specify the first one, and since it's symmetric and since it's orthogonal you'd only need one more to find the second one and then another to divide the third.
So that's three degrees of freedom to find three principal axes in an orthogonal system
Not sure I like that, because this has to be the shape of the quadric for an arbitrary rotation angle
Does that angle between X1 and X2-- does it have to be 90 degrees if it's not connected?
That's correct, that's correct.
OK, let's not tie up the whole audience, let me-- let's talk about this and I'll think about it too.
But you make a very, very good point, but let's not settle it in real time.
We'll throw everybody out, we'll close the door, and you can come back and see who won the scuffle, OK?
OK, so let's take our break, I think you're more than ready for it.
OK, its always disappointing when, just as you're ready to tie everything up in a nice, neat little package and leave the room to a round of applause, you screw up the last little detail.
Anyway, what's wrong here is, I said for an arbitrary rotation around X3, and I'm comforted only by the fact that nobody else caught this little glitch either, the answer elements are a11, a11, a33, a12, not 0, and -a12, 0, 0, 0, 0.
So there's a skew-symmetric form when there's just a single rotation axis.
So this would be asymmetry n and this would be one of a family of two different symmetries, the dihedral groups n22, the two-fold axes would require that these be equal, so, 0, 0, 0, a11, 0, 0, 0, a33.
So this is not the form of the tensor for a four-fold axis, it's the form for a crystal with symmetry 422.
And same for a three-fold axis.
This is the form for 32 and 622.
For just the single rotation axis of any sort other than 2, these two terms are equal in magnitude, but non-0.
Now if you look at the equation for the tensor, the term in a12, x1, x2, and a21, x1, x2 are opposite in sign and disappear.
So the quadric is, in fact a surface of revolution So the question is why are there two degrees of freedom rather than three?
And I think the answer to that is going to lie in what is required of the eigenvectors when the off diagonal turns are skew-symmetric, and that is something I've not thought about before.
All right, so that one minor loose end sticking out of the lid as we try to close the box on this segment of our discussion.
I'd like t move on to a different set of considerations.
One of the things that we can subject a crystal to as a generalized force is not merely a vector force, but stimulations that are more complicated.
And I would like to in particular now, as a prelude to get into getting into property tensors of higher rank.
I'd like to .
first discuss stress, and then define strain, and you've all heard of these questions in other contexts.
I'm going to introduce it, though, in terms of the tensor notation that we've used for other sorts of tensors, just so that we have everything in terms of the same language.
So let me introduce stress by supposing I have a volume element in a solid, and it will not surprise you that I'm going to define the coordinate system in the solid by three axes, X1, X2, X3 in a Cartesian.
system, and I will look at a surface on the exterior of the solid that cuts axis X1 at A, axis X2 at B, and axis X3 at C. Then I'll assume that I have a force density, force per unit area applied to the surface.
And if this is force per unit area, why don't I call it a pressure?
Well, a special case of what we normally think of as a pressure is a hydrostatic pressure, and that is something that's always exactly normal to the surface.
So I don't want to call it a pressure to mislead you into the assumption that this force per unit area, which is a vector, has to be normal to the surface.
So I'll call it a force density, which becomes a hydrostatic pressure when that force vector is normal to the surface.
And I'll call that force density vector K, and it will have three components, KI.
Now, the solid is in equilibrium, I will assume.
And if that external force is not balance on the other side by balancing forced, the volume element in the solid would undergo a linear acceleration, or an angular acceleration at the very least.
So I will assume that this force on the outside, KI, which is a force per unit area.
So to make that a force, I will have to multiply that force density by the area of the surface ABC, on which it acts.
And I'll say that this has to be balanced for each of the three directions, X1 and X2 and X3 by a force per unit area acting on these internal surfaces.
So I'll assume that there is a force acting in the x1 direction, and I'll call those a sigma IJ.
And I'll say that there's a first force acting in the X1 direction on-- now I need to somehow specify the orientation to the surface, because on this internal surface there's also a force acting in this direction.
Now on this back surface there's also a force acting in this direction.
So the forces acting on all three internal surfaces in the X1 direction have to balance exactly the X1 direction, the X1 component of the force density.
So let me turn this to K1, and I'll say this has to be a force per unit area acting on area.
First of all, I'll call the origin O on area AOC.
Plus a force acting in the X1 direction on area A-- whoops, what did I do?
I wanted to do this first on area BOC, excuse me.
And a force in the X1 direction acting on area AOC.
And finally a force acting on area AOB.
And these are all acting in the X1 direction, and I now would like to introduce some notation that indicates the direction of the direction orientation of the surface on which that force acts.
And I will use the normal to these internal surfaces to define that.
So the force per unit area on area AOB, that is a surface whose normal is X3.
And I will call this component sigma 13.
And I'll call this one area AOC is normal to X2.
That's why I did that little sleight of hand as I began this, and finally area BOC has as its normal X1.
So my use of subscripts here is that this first subscript is the direction in which the force acts.
And the second subscript will be the normal to the internal surface.
So I've set up algebraically a system of subscripts that works.
We'll see that these have some physical significance in just a moment.
So what will I call these?
Well, let me get this in a slightly different form.
Let me take the left hand side and divide area ABC into all of these other areas.
So this then will have sigma 11 times area BOC over area ABC.
And not surprisingly, sigma 12 will then have multiplying it a term area AOC divided by area ABC.
And this is all looking very cumbersome, but you'll be amazed at how this cleans up in just a moment.
And this will be area AOB over finally area AOC.
ABC again.
OK, so how can we tidy this up?
Well, let me imagine that I have some surface, a planar surface, and its normal points in this direction.
And I'm going to take that area and project it onto another surface whose normal points in this direction.
And let's suppose that the angle between these two directions is phi, and the angle between the two surfaces is phi.
The area of this original surface is A, and its normal points in this direction.
And I project it onto a surface whose normal is an angle phi away.
This new area, A prime is equal to A times the cosine of phi.
So if we accept that, then the first line of this equation, which will be eventually a set of three equations, this is the ratio of area BOC over area ABC, and what is that?
That is the angle between the direction of the force density and area ABC.
And here is area ABC.
That's the normal here.
And the first term involves area BOC, and that is the direction of X, the normal to that is X1, and therefore the ratio of these two areas is the same thing as the cosine of the angle between K and X1.
And this second ratio of areas is going to be sigma 12 times the cosine of the angle between the force density vector K and X2.
And the third term is going to be sigma 13 times the cosine of the angle between K and X3.
What these cosines are are just the cosines of the angle between K and the three reference axes, X1, X2, and X3.
So this is simply the direction cosine L1 of K. And this is the direction cosine L2 of K. And this is the direction cosine L3.
So the X1 component of the force density vector is going to be equal to a term sigma 11 times L1, plus the term sigma 12 times L2, the term sigma 13 L3.
And if I would write down similar balances, forces for the X2 component of the force density vector, not surprisingly these things are going to give me coefficients that I'll label sigma 121, sigma 22 times L2, plus sigma 23 times L3.
And in general then the requirement that this body be in equilibrium states that each component of the force density vector should be given by a coefficient sigma IJ times L sub J.
These are the components of the force density vector.
This is the direction cosines of the direction of K. This is a unit vector in the direction of K. This is the direction of a vector.
These are the components of K. And therefore these coefficients sigma IJ relate to vectors, and they must be a second rank tensor.
And these are called, as you all know, it comes as no surprise as the components of stress.
So the reason that these are really tensors, and not just a vector force per unit area essentially comes down to the fact that the behavior of the body, dynamically and mechanically, is going to depend not only on the direction of the force, but on the direction of the surface on which it acts.
Quite clear here, if I shove, is a force density, and that's a vector.
But the way in which this desk responds to my pushing on it with a force vector of that magnitude is going to depend whether I do this, which is going to compress it, or whether it takes the same force and direct it this way, which if I don't move the thing is going to result in a shear.
So the behavior of the body will depend on the direction of the force per unit area that I apply, and the direction of the surface on which I apply it.
In this case, that would be the surface ABC.
Yes, sir?
You say that the force density is not parallel to the normal of the surface?
That's correct
So when you do the ratio of [INAUDIBLE] isn't it supposed to be [INAUDIBLE] of the normal to the [INAUDIBLE] and X1?
Another way of defining it.
This is a vector, and what I'm doing is splitting that vector up into three internal forces that act on surfaces whose normal are here.
So actually, what I'm going to do is to take the way in which the internal force pushes back.
I see, that's what's confusing.
If the force that I'm putting on the body, I didn't mean to indicate any particular direction, but if this component of K points in this direction, this force that balances it has to go in the opposite direction
So you're saying that there were surface ABC is defined as normal to K?
ABC is not normal.
No.
ABC is not normal to K. All I'm saying is that it if it were normal to K, then the balancing force would be just a function of this orientation of the surface.
But if I allow this to be a general force per unit area, then the balancing force on the inside depends on the angle between K and the angle between this internal surface
I don't see the relation between K and the ratio between other ends.
BOC and ABC [INAUDIBLE]
OK.
The reason that there are three areas, is that what's troubling you?
Because I'm really now splitting the balancing force up into a force per unit area on this surface, and a force per unit area on this surface, and a force per unit area on this surface.
And that's so that I can look at these different components of resistance in terms of a Cartesian system.
Clear that the ratio of these two areas does go as the-- I erased it now, that is going to go as the-- This is A prime, and this is the original A, the relation between those areas is given by the cosine of this angle phi.
If I [INAUDIBLE] this down in this projection, and this is the area of A.
This is the direction of the normal to the surface ABC.
And I'm going to let this be the direction of the normal to one of the internal surfaces, and these internal surfaces are going to have as their normal either X1, X2, or X3
So [INAUDIBLE] between the normal to ABC and X1
And X1, yes.
And that's what I'm saying its, and that is going to be the same thing.
As the cosine of that angle is just going to be the direction cosine of the normal to ABC relative to X1, X2, X3.
OK?
So in the end, you're saying that K is parallel to the normal of ABC?
No, I'm not.
No, I'm not.
This is a relation between the areas.
And these are the forces acting on those internal surfaces.
And I'm saying there are three of them, and those three components that are all acting in the X1 direction will depend on the ratio of these areas.
The ratio of this area is the cosine of the angle between X1 and K. OK.
This you're OK with?
Yeah
OK. And what I'm saying now is that there is acting on this internal surface something that has to balance the component of K, which points in the direction of X1.
And the component of K before we divided out the area, the component of K1 times the area gave me the net force.
So this K1 now is a force per unit area, the component of K per unit area that points along the X1 direction.
Now as a force per unit area I multiply it by an area that gave me force.
There are three forces internally that balance this force, and they all act in the X1 direction
You're just [INAUDIBLE].
Shouldn't it be the cosine of the angle between the normals of [INAUDIBLE]?
Yeah.
That's what I'm saying here.
That for X, for this surface, yeah, that's what I'm saying.
One of these surfaces here, if we look at the first term, this is the direction of X1, and the force here will be the force along K. And if I take this force and put it down in X1, this is going to be the force times the cosine of the angle.
And the two areas are going to go as the cosine of the angle
[INAUDIBLE] cosine of the normal [INAUDIBLE]?
OK.
This is indeed the ratio between the internal area and the external area.
Since the internal area has a normal that's either X1, X2, or X3 the angle between the normal to these internal surfaces and K is going to be the direction cosine of K. I suggest since we're getting out of time quickly we either resolve this after the end of class or leave it until next time.
That's a bad note on which to end, too.
The point I want to leave, still subject to resolution and debate is that the coefficient sigma IJ I would like to claim relate to vectors.
One is the direction of the external surface, and if it relates to vectors, then this set of coefficients qualifies as a second rank tensor.
And if you accept that, however grudgingly, everything that we've said about second rank tensors holds for the quantity sigma IJ, which are called the elements of stress.
In particular, we can talk about a stress quadrant.
We can talk about the value of a stress in a particular direction.
If we have a stress tensor in which show all the terms are non-0, it can be diagonalized.
We can also say that if the crystal has symmetry, for example, if the crystal is cubic, you can say that the form of a second rank tensor for a cubic crystal has to be diagonal.
And these diagonal elements represent compressive stresses, so you can say that you can only subject a cubic crystal to compressive stress.
On the other hand, if the crystal is a triclinic crystal, you can say that there's a sigma 11, there's a sigma 12, a sigma 13, a sigma 21, a sigma 22, sigma 23, sigma 31, sigma 32, sigma 33.
So for a triclinic crystal, there appear to be nine elements, but next time we will show that the nature of stress is the body has to be in equilibrium requires that the stress tensor be by definition symmetric if there's to be no net couple to force on it.
Going to buy this?
You didn't buy the things that I said that perhaps were slightly incorrect.
This is massively incorrect.
Jason?
You can rotate a cubic crystal [INAUDIBLE]
No.
Because say a tensor of this form acts just like the scalar.
Just like a cubic crystal, you don't suddenly get off diagonal properties if you transfer it to a different axis.
So unless you tell me what's wrong here, I'm going to leave this on your plate until next week.
And I don't want you fretting over the weekend.
This is clearly nonsense, and it is a contradiction that arises only if you carry too much of what we've been doing for the last month with you into a discussion of stress and strain.
Stress is something that you impose on a crystal.
It has nothing to do with what kind of crystal it is and what's going on in the interior of the crystal.
There may be constraints on the coefficients that relate stress and strains, and there surely are.
But I can take a cubic crystal, and I can squeeze it, and I can twist it and share it anyway I like.
So this is an important distinction between the tensors that represent physical properties, which we've been discussing up to this point.
And the properties such as conductivity, diffusivity, susceptibility and so on and all their many varieties, these are something which we'll refer to as property tensors.
And they are tensors demonstrably, but they are tensors which describe the physical behavior of a crystal.
Nye uses a term that I don't think is self explanatory, so I don't care for it.
He calls these field tensors.
That sounds like some description of a corn patch that requires a tensor.
But no.
We probably don't really think of what we mean when we say a field.
When we say an electric field, you think of an e-vector.
You think of a vector.
But what a field is is just a set of coordinates, XYZ, and that describes an area or a volume.
And just like the things that corn grow in are referred to as corn fields, it's an area that has corn on a lattice very often also.
So a field is just a set of coordinates in space to which a value of something is a sign.
So if we have an electric field, and we express that electric field as a function of X, Y, and Z, that is a field of vectors.
If we take something that has intrinsically, that intrinsically needs to be described as a tensor, then we have to every point in space XYZ a tensor sigma IJ, whose value and whose components value vary with position within the space.
So talking about the field as assigning values of something to every coordinate in the space that could be a tensor.
What we refer to as an electric field really is a field vector.
So that's Nye's term, but the fact that it took me three minutes to explain why he uses it explains as well why I don't care for it.
So we'll talk about these as property tensors.
And property tensors are something that's innate to the material.
Something like stress, strain, and other second rank tensors are external stimuli, just like an electric field.
And they have value that is imposed on the crystal, and the stress field is defined as a function of position.
So that's the distinction between a property tensor, which is subject to symmetry constraints, and a tensor that represents a stimulus, an external stimulus applied to the crystal, and it's defined as a function of position.
And that is a field tensor.
OK, so that's the difference.
Since this is something that's imposed externally, it can have any form of the tensor that you wish to oppose.
And there are no symmetry restrictions on it.
The only restriction is, as we'll show next time, that it must be symmetric if the body is to be in equilibrium.
All right.
It is time to quit, and that I think is a good place to leave things.
And we'll say more about stress next time.
We are thundering rapidly to the midpoint of the term.
I think a week from this coming Thursday will be exactly halfway, and the halfway point is when I usually try, give or take a day, to change gears very abruptly and start looking at the symmetry of physical properties and tensors and actual discussion of physical properties that are anisotropic.
So we'll be finishing up our discussion of symmetry fairly quickly.
We still have, however, some major derivations to perform, and we'll get into one of these today.
Last time we had begun the process of deriving what are called the point groups as opposed to the plane groups.
The plane groups were combinations of symmetry with a lattice that extended throughout a plane in space.
A point group we've seen in the form of two-dimensional point groups.
We're now going to today derive the three-dimensional point groups.
And, again, the name stems from the fact that these are clusters of symmetry about a common point so that at very least that one point in space stays put.
So, hence, point groups-- these are symmetries about a point.
And to put some embellishing adjectives on here, these will be the three-dimensional crystallographic point groups.
Because there are an infinite number of point groups, we're considering just those symmetries that can be combined with a lattice and therefore are permissible to crystals.
I'd like to point out in case you've not been making much use of Buerger's book that we actually are getting back to Buerger's treatment.
We'll do some of the derivations over the next few days with a little bit more detail and with a slightly different procedure than Buerger.
But the bases covered are the same.
So let me just point out where some of this material is in Buerger's book.
Buerger does Euler's Construction, and he does this in a once over lightly.
I think this is not the best part of the book.
This is on pages 35 to 43.
In particular, he sort of leaps ahead quickly and just looks for one angle between rotation axes and doesn't get the-- do specifically the ones that are 90 degrees and are not as interesting.
Some of the combination theorems are in Chapter 6, and there you'll find the statement that two reflections at an angle mu is equivalent to a net rotation of mu, except Buerger doesn't call them sigmas.
He calls them m's.
He uses the same symbol for individual operations and for the symmetry of them, and I think that's a little untidy.
Also, the theorem that says that if we have a mirror plane that's perpendicular to a twofold rotation axis, that gives rise to inversion center at the point of intersection.
And then he launches into a rather condensed version of the derivation of the point groups on pages 59 to 68.
Then, in Chapter 7, derives the two dimensional lattices or the plane nets.
This is on page 69 to 83.
We've done that differently and much more exhaustively by deriving the two-dimensional plane groups and the lattices are part of the plane groups, and we've gotten them automatically.
There is a place where in the derivation of the space lattices, he's a little bit incomplete and actually does something that's wrong A-ha-- I showed Buerger was wrong at one place in his book.
But anyway, this is where the material that we're covering now is mentioned in Buerger's book.
Last time, we had hit a new surprise.
We had asked if we take the basic rotational symmetries and add an inversion to something like three-- so, three is a subgroup, the operation of inversion is an extender.
We found that there was a new operation that came up, and this was a rotoinversion operation.
And then the resulting point group, which we called 3-bar-- the symbol itself, 3-bar, is a threefold rotoinversion.
And if we were not clever enough to invent a rotoinversion operation, we would have stumbled headlong over it in adding inversion as an extender to a threefold axis.
And a rotoinversion axis is something that involves in general rotating through some angle alpha to get from a first object to a second object of the same handedness and then not yet putting it down, first inverting it through axis-- through a point on the axis-- to get a second object.
I shouldn't call this second because this is not rotation with inversion simultaneously but a two step operation.
So we rotate from 1 to this virtual object also of same handedness and then before putting it down, we invert it to get number two, which is left handed.
And the operation of getting there in two steps but in one repetition is the operation of rotoinversion.
That operation was one of the members of the group that results when you add inversion as an extender to 3.
But, really, this was not something that was a new sort of symmetry element.
It was an operation that came up, but we could regard the point down that we've labeled 3-bar as simply a threefold axis with an inversion center sitting on it.
But then we looked at another one.
And by anticipation, I said we ought to look at these.
And we looked at what happened when we took a fourfold rotoinversion operation where we would take the first object number one, rotate it through 90 degrees, and not yet put it down.
We'd first invert it through a point on an axis, and we got one that was 90 degrees away but pointing down in of opposite handedness.
If we repeated that operation several times, we found that we got two objects up above a plane normal to the rotation part of the operation and passing through the point, which we inverted.
So there's one right handed one up here, a second right handed one up here, and then two down below, which were of opposite chirality, number 3 and 4.
And that is an entirely new beast.
We have no way of describing the relation between these four objects other than saying rotate, don't put it down, first invert.
Rotate, don't put it down, first invert again.
So this is something entirely new.
It's a two step operation.
You cannot describe it any more simply than saying take two steps to do the repetition in the same sense that the glide plane that we discovered in the plane groups had a step of translation followed immediately by reflection, and that was a new sort of operation.
So here's another example of a two-step operation.
The symbol for this point group is 4-bar, and the symbol for it-- if you want to indicate its locus, is, it's got a squareness to it because these objects as mutually are separated by 90 degrees, but it really has only a twofold symmetry, so the symmetry for a 4-bar axis.
It's not really an axis at all.
It's a 4-bar point, because only a point is left invariant.
But the symbol for a 4-bar "axis" is this square with a little twofold axis inscribed in it.
Alright.
We ought to really, then, step back and ask how many different rotoinversion axes there might be, and add these to our bags of tricks.
And what I'm going to invite you to do is to examine this systematically on a problem set which I will pass around, along with several other problems for your consideration, as well.
And having stumbled onto that combination, another thing we might ask to broaden our bag of tricks still further is to say is there such a thing as a rotoreflection axis?
And we called n-bar a rotoinversion.
I wrote a reflection axis is indicated by an n with this little squiggle on top.
Anyone who has studied Spanish realizes that the proper name for the squiggle is a tilde, T-I-L-D-E. And this would be an operation where there's a locus in which we'll perform the reflection part of the operation, so this would be a 2 step operation that involves taking a first one and rotating it to a virtual object number two but not putting it down.
Before putting it down, we will reflect in a planar locus.
So this is number two, and it will be of opposite chirality.
So that's another operation that involves rotation coupled as one part of a step that involves R, the second of our three translation free operations, namely reflection and inversion.
So are these something we should consider?
What I've invited you to do is to draw out for n equals 1 to 8 the patterns produced by rotoinversion and the patterns created by rotoreflection, and then see if you regard these as new operations or whether they can be broken down into the simultaneous presence of more than one of our old friends such as 3-bar being a plain old threefold axis with an inversion sitting on it.
The fact that I'm asking you take your time to do this suggests that yes, you are going to find some rotoinversion and rotoreflection axes, which are inherently 2 step operations and can't be decomposed.
What I'd like to do now, then, is to systematically look at the axial combinations.
These are of the form n one, two, three, four, and six and the dihedral symmetries.
These are of the form n22, or just 32 in the case of the one with the threefold axis.
And then the two cubic arrangements 23, twofold axis out of the face normal to a cube, threefold axes out of all of the body diagonals, and 432.
And what I'm going to do with you is to use these as worksheets to guide the logic of what we're doing, and we won't do every single one of them.
I think once we do a few, the general principle is clear and the results there are before you.
But what we'll take, then, is the axes n, the axes n22, and then the two cubic symmetries 23 and 432.
Then, finally, we found 4-bar as a different rotation axis-like symmetry element, so I'll add that to the list.
Then consider what we can add to these symmetries as extenders, and these are an additional element that we can add to what is already a self-contained nice little group.
We muck things up by introducing another operation, and then we'll have to take combinations of that operation with all the symmetry elements that are there in the parent subgroup and see what new symmetry elements arise.
The ground rules in these additions are fairly simple but nevertheless profound.
The addition of the extender cannot create any new symmetry axes by its operation, and the reason for that should be quite clear.
We obtained these combinations of rotation axes using Euler's principle, and that was thorough and orderly, systematic and complete.
These are the only arrangements of crystallographic rotation axes that are possible.
So if we add an extender that creates, for example, a new twofold axis in n22, it's either going to be something that we already have in this list, and therefore not interesting, or something that just cannot constitute a group.
We'll take combinations of operations and we will never find a closed finite set.
So if we add a mirror operation, a reflection operation as an extender, the addition of the reflection operation sigma must be either perpendicular to the axis.
And what that's going to do is just flip the top part of the rotation axis and reflect it down to the bottom axis.
Nothing new was created.
And it could alternatively be parallel to the axis and passing through it.
In that case, we've already seen the consequences of this addition in deriving the two-dimensional point groups, namely to 2mm, 3m, 4mm, and 6mm.
No new axis is created.
One of the additions that is something we mentioned last time is that in the case of the dihedral groups where we have an n-fold axis and then twofold axes arranged in some fashion-- not in some arbitrary fashion.
It's going to be 2 pi over n times 1/2, depending on the n-fold axis-- we could pass the mirror operation through the twofold axis.
And that's what we'll call a vertical mirror plane.
But when there's more than one axis present, we could also put the reflection operation diagonally in between the twofold axes.
So we're going to refer to this as a diagonal mirror.
It's snaked in between the axes that are present.
So, perpendicular reflection operation as an extender, one parallel to the axis and passing through it, or one that is diagonal passing in between them.
And finally, that exhausts what we can do with reflection, but we could add inversion.
And if we're not going create new axes, and we're going to get a point group-- a finite set of objects-- the inversion center has to be on one of the single axis, namely these groups n, or at the point of intersection, if there's more than one.
So there's the job laid out for us.
And we've got two theorems to aid us in quickly finding the new operations that arise-- in effect, taking a shortcut to establish the group multiplication table, and the two theorems are as follows.
We saw that if you take a rotation operation A pi, that takes a first object that's right handed and rotates it to a second object that is also right handed.
Then follow that by reflection in a mirror plane that is perpendicular to the axis.
The net effect of going from 1 to number 3, which is left handed, is to create an inversion center at the point of intersection.
So there's a theorem that we can write down once and for all and say that A pi followed by reflection in a mirror plane that's perpendicular to axis A has, as a net effect, the operation of inversion.
And if this is not a twofold axis, it's something like a fourfold axis or a sixfold axis, that contains the operation A pi as part of the operations contained in that axis, and that's going to create an inversion center as well.
So for any evenfold axis, add a perpendicular mirror plane, and automatically the inversion center comes up.
And we can permute the order of these operations if we rotate by 180 degrees and then invert the net effect is reflection in the perpendicular mirror plane.
So this is something that you permute around into three different combinations.
Does the order of the operation make a difference?
And the answer is no, the order is not important.
For example, rotating and then reflecting is the same as reflecting and then rotating it.
It went up at the same point, number 3.
Remember a fancy word for this when we define what's meant by a group, that these groups are going to be what's called Abelian, and an Abelian group is where any combination of operations a followed by b is identical to b followed by a. I told you that great joke among mathematicians, what is purple and commutes?
And the answer is an Abelian grape.
Ha, ha, ha-- I think that's stupid, but it drives mathematicians into guffaws of laughter.
A second theorem, and this is one that we've already seen in two dimensions, is that if you take an operation A alpha, pass a reflection operation sigma 1 through it, that to the effect of that is going to be another mirror plane that is alpha over 2 away from the first.
Notice I'm a little bit sloppy in talking about reflection operation and mirror plane because if a reflection operation is there, that's all I need to have to say that this is the locus of a symmetry point, a mirror plane.
So this is another one.
Do you think that this an Abelian combination to say that A alpha followed by sigma 1 is equal to sigma 2.
Is this Abelian?
No
No, it's not.
And how do you answer that?
Not through any means more profound than just drawing it out and seeing what you get.
Let's take 3n as an example.
So let's do the operation.
In this order, do the rotation A 2 pi over 3 to go from this one up to this one, and then let's reflect across this horizontal mirror plane.
So this is 1 right handed, this is 2 right handed reflect, here's number three and it's left handed.
So that's A 2 pi over 3 followed by reflection in sigma 1.
And if we do the operations in reverse order, reflect in sigma 1 to go up to here, so this would be 2 prime and then rotate by 120 degrees.
We would go to here to here, and that is not the same location as this one.
So this is not equal to sigma 1 followed by A 2 pi over 3.
How can you tell when is the combination of operations is going to be Abelian and when is it not?
It sounds rather clumsy to state it in words, but whenever the two symmetry operations leave each other unmoved, then you can permute the order.
For example, in this combination here the rotation leaves the mirror plane unchanged.
It just spins around in its own plane.
The mirror plane leaves the rotation unchanged, it just reflects it end to end.
Here this rotation axis and these mirror planes obviously do not leave each other unchanged because the threefold axis rotates the mirror plane to two other new locations.
So whenever that's the case, if the operations do not leave each other unchanged, the order makes a difference.
So that's a useful thing to keep in mind.
Let's now turn to these sheets that I passed out, and you might want to unstaple them because they're designed to go side by side, and there are three pages.
What I've done across the top of the pages is to give the different combinations of crystallographic rotation axes that are possible.
So going from the first sheet across to the last, there's the rotation axes by themselves-- one, two, three, four, and six.
Then there are the groups of the form n22, 222, 32, 422, 622, and then the two cubic combinations 23 and 432.
So there are the 11 possible combination of crystallographic rotation axes, and stuck off by itself at the end is the oddball 4-bar.
For the axes by themselves, the diagonal mirror addition is not defined.
The diagonal mirror position by definition snakes in between axes that are there in the axial combination.
If there's only one axis, that's not defined.
But you could add a horizontal mirror plane.
A horizontal mirror planes adds a onefold axis.
It's just a mirror plane.
So that's called m in the international notation for mirror.
And it's called C sub S, C because it's a cyclic group, and the S stands for Spiegel.
That's the Schoenflies notation.
If we add a-- take two and add a horizontal mirror plane-- this is the thing that we sketched out here-- this gives a pattern of four objects, two that are up above the mirror plane of one chirality, two down below the mirror plane of opposite chirality.
These projections of the patterns are down along the rotation axis, and when you see a dot in a circle, the circle represents the point that is down and the dot is one that's up on top.
So there are two pairs of objects-- two right handed ones that are up and two left-handed ones that are down.
And that, in international notation, is 2/m the twofold axis is over a mirror plane.
So that's the way to make sense of the symbol and remember what it means.
In the Schoenflies notation at C2.
You've added a horizontal mirror plane as an extender.
Going down the remaining ones in that list because they're all very, very similar-- add a mirror plane to a threefold axis, the triangle is reflected down to a triangle of motifs of opposite chirality, so that's 3 over m. For sure, it's called 6-bar, but let's forget about that.
The Schoenflies notation C3H, and similarly there's a 4 over m, a square above, and a square of opposite handedness below the mirror plane.
The horizontal mirror plane in all of these cases is shown as a bold line.
They've added fainter vertical lines to give you an angular reference.
Don't be confused by that in the Xerox copy-- the weight of the lines is not distinguished.
So the only symmetry element in 4 over m, for example, should be shown by this solid bold circle that is in the plane of the paper.
The two crossed lines are lighter and those are just as mutual orientations.
And finally, there's another one of this form which is 6 over m. On all of the evenfold additions-- that is, to say everything but 3m-- if you add that horizontal mirror plane, it's going to be perpendicular to an operation A pi.
Therefore, inversion pops up as one of the operations in the group, and 2 over m, 4 over m, 6 over m, those regular polyhedra of objects-- can be inverted through the point of intersection of the axis in the mirror plane, and that leaves the set unchanged.
So if we go down to the bottom of the list and say, can we add inversion to these axial arrangements, in the case of 2, 4, and 6-- no, adding inversion doesn't give you anything new, because that's already there in the groups of the form CNH, or 2/m, 4/m, 6/m.
Diagonal is not defined, so the only additional one that we could pick up by adding inversion is adding inversion to a threefold axis, and that's one we did the other day.
Adding inversion to a threefold axis gives us three that are up of one handedness, three that are down of opposite handedness, and the orientation of the triangle is skewed.
That is called 3-bar.
But it's just nothing more than a threefold axis with an inversion center sitting on it.
Schoenflies' notation is C3i.
Then the only other additions to the single axes are adding the mirror plane in a vertical fashion passing through the axis, and these we've already seen in two dimensions so we don't have to spend much time on them.
There's 2 mm, which is C2V.
3m-- all the mirror planes result upon adding a single mirror plane to the threefold axis-- 4 mm, and 6 mm.
These are just three-dimensional extensions parallel to the axis of the symmetry that we already derived for a plane in space in the two-dimensional point groups.
So we're almost half done.
It's easy, isn't it?
Any questions at this point?
So the only ones that are really new that we haven't seen already in two dimensions are the rotation axis perpendicular to the mirrored plane.
That brings us to roughly the middle of the second sheet, and the arrangement of axes 222 represents three mutually orthogonal twofold axes.
If we add a horizontal mirror plane, that's going to put an inversion center at the point of intersection because that horizontal reflection is going to be sitting normal to a twofold axis.
And then since the other twofold axes see an inversion center sitting on it, there's a mirror plane perpendicular to those other twofold axes.
So adding again to go through that again, adding inversion to the point of intersection of the twofold axes creates a mirror plane perpendicular to each of those twofold axes.
So it becomes 2/m 2/m 2/m-- each of the twofold axes acquires a different sort of mirror plane perpendicular to its orientation.
And 2/m 2/m 2/m, even though I love saying it, is kind of a mouthful.
So that's very often abbreviated to just mmm, which really is a nice exclamation when you see what a lovely symmetry it is.
The pattern shown in the diagram to the left is just the pattern of 222 with the objects reflected up or reflected down, and you get a total of 8.
That means that there are eight operations in the group, and you can add up quickly what they are.
They are the three operations of rotating by pi, the operation of identity, then three mirror planes because the mirror planes all indistinct, and then the operation of inversion.
So there are the eight operations that are present that generate the objects from a single one.
Add a horizontal mirror plane to 32 and you get a threefold axis perpendicular to the threefold axis, so you write that as 3m.
That mirror plane now passes through the horizontal twofold axes, and a twofold axis with a mirror plane passing through it wants another mirror plane 90 degrees away, so a new mirror plane pops up passing through each of the twofold axes and perpendicular to the horizontal mirror plane.
Those mirror planes are in the same direction as the twofold axis.
They're not perpendicular to it, and when symmetry elements are parallel to a common direction you write them out on the same line, so here's a case of a mixed metaphor.
One mirror plane that's perpendicular to an axis, the threefold axis, so you write that as 3 over m. The other twofold axes have mirror points passing through them, so you write that as 2m and the threefold axes makes all of those 2m symmetries equivalent to one another.
Pattern, and this is true for all of these symmetries, is nothing more than the pattern of the subgroup repeated by the one operation that you've added as an extender .
So if you look at the pattern of 32, object all of the same chirality, all apparently up and down, and add the horizontal mirror plane.
The one that is up goes down, the one that's down goes up, and you get a set of four around each of the twofold axes.
422 behaves very similarly.
The pattern is just the pattern of 422 reflected in a plane, so you have a set of 16 objects-- eight of them up of one chirality, those are the solid dots, if you will, and then another two, four, six, eight that are down of opposite chirality.
You have a mirror plane that you added as the horizontal mirror plane perpendicular to the fourfold axis.
All of the horizontal twofold axes see a mirror plane passing through them.
So they've got to have a mirror plane that's 90 degrees away, and those are the vertical mirror planes.
Either the point is obvious now or you're not following me at all, so I'll simply say on the left-hand column of the final sheet, 622 with a horizontal mirror plane goes to 6/m, 2/m, 2/m, called D6h in Schoenflies.
I'll leave the cubic ones until last.
They're not nearly as bad as they seem.
But obviously they have a lot of symmetry all over the place, and we'll want to take a closer look at the patterns.
The next extender is a vertical mirror plane, and we've already got the groups of the form cnv, because they are just extensions in a third dimension of one set we've seen in two dimensions.
If we try adding the vertical mirror plane, which is defined as passing through the principal axis of high symmetry and perpendicular to the twofold axes, we've already encountered those in every group except 32.
2/m, 2/m, 2/m already has the vertical mirror plane.
The same is true of 3/mm2.
The same is true of 4/m, 2/m, 2/m and 6/m, 2/m, 2/m.
The vertical mirror plane comes in automatically when we add the horizontal mirror plane as the extender, So nothing new there at all.
And now we come to one that is interesting.
This is the diagonal mirror plane.
And I'll do this one slowly and then in some detail because it's another example of how we would trip over something if we were not clever enough to think of it.
Let me begin by drawing the three orthogonal axes of 222.
So we have one object that's up and one object that's down.
The twofold axis perpendicular to the board will take this one that's down and rotate it to here and take this one that's up and rotate it to here.
So that is the pattern of 222.
We can snake a mirror plane in between the twofold axes, and a mirror plane passing through the vertical twofold axis has to be accompanied by one that's 90 degrees away, half the throw of the axis, which comes from our theorem that says A pi dot sigma vertical has to be equal to a sigma prime that's vertical and pi/2 away from the first.
In terms of the pattern, this second mirror plane that comes up is going to reflect this one across to here, it's going to reflect this one across to here, it's going to take this one and reflect it over to here, and take this one and reflect it to the right, as well.
So now we've got a total of eight objects.
This one is right handed, and this one is also right handed because we repeated it by rotation.
Then we reflected that pair, so these two are left handed, and reflect it across the other diagonal mirror plane.
They're back to right handed again, and these have to be left handed as well, so there's the pattern.
Is there an inversion center in this pattern?
The answer is no, because there is no mirror plane that is perpendicular to a twofold axis, so this point group has no inversion in it.
It's said to be acentric, without a inversion center.
Do we know how everything is related to everything else?
We know how this one is related to this one by twofold rotation.
We know how this one is related to this one, and that's by reflection across the mirror plane.
How is the first one related to this one?
How do you get it through 90 degrees and then pop it down and change the chirality?
[INAUDIBLE]?
You can't do that in one step.
You've got to do just what I said in words.
Rotate it 90 degrees, don't put it down yet, and reflect it down in a horizontal mirror plane.
This horizontal mirror plane is not a symmetry element, it's part of the operation that's necessary to get us from this guy over to this guy of opposite chirality.
So this is an example a new type of operation, a 2 step operation.
This is a 90-degree rotoreflection axis.
And again, I feel compelled to put the axis in quotation marks, because really it's a pair of operations that leaves only a point unmoved, so it's really a point symmetry element.
There's another way of describing the same thing, and that would be rotoinversion.
We could rotate by 90 degrees in the reverse sense, not yet put it down, and invert, and we get, again, the relation between-- what did I do?
Rotate it here, yes-- rotate to here and then invert, and we got that one.
So that's another way of defining the relation.
A 90-degree rotoreflection is a minus 90-degree rotoinversion.
So you pays your money and you takes your choice, and what the groundbreakers who went before us did was to take rotoinversion.
as the operation to survive.
So here is, if we were not smart enough to invent it, our 4-bar rotoinversion axis.
So if we hadn't been clever enough-- and it would take a pretty clever, devious mind to invent a 2 step rotoinversion operation-- as soon as we added this sort of extender, a diagonal mirror plane to a group of the form n22, we would have found that there was a new sort of transformation that could not be described any more simply than to say take two steps to do it.
Rotate 90 degrees and then invert.
So we have to add the operation of A pi over 2-bar to our basic bag of tricks for generating patterns.
The group 32 is also a group to which we can add a diagonal mirror plane.
And if you do that, again, the bold lines that are mirror planes in that group and the axes that are faint lines are easy to mix up.
But if we examine that group without the confusing lines, here are the twofold axes.
And remember that they are all equivalent to one another by the threefold axis.
So this axial group is the group 32.
If we put down a mirror plane interleaved between the twofold axes, notice that each of those mirror is perpendicular to a twofold axis.
So we can write this as 2/m.
2/m means there's an inversion center at the intersection of the twofold axes with the mirror planes.
And a inversion center sitting on a threefold axis makes it a 3-bar axis, so this group is called 3-bar 2/m.
And the pattern is what a twofold axis would do.
These would be up-down-right ones.
Reflect those and you get an up-down pair of left ones.
Reflect again and you get an up-down pair of right-handed ones.
Reflect yet again, and you get an up-down pair of left-handed ones.
Reflect still once more, and you get a down-up pair of right-handed ones.
One more time, up-down left-handed ones.
So you get a total of 2, 4, 6, 8, 10, 12 points.
So this is 3-bar 2/m in Schoenflies notation D3-- that's the symbol 32-- with a diagonal mirror plane added.
You indicate that by a d in the subscript.
OK.
I think that's probably a point where you're more than ready for a break.
We've got very few yet to do.
One surprise is that if you add diagonal mirror planes to 422 and 622, you get perfectly lovely exquisite groups but they're not crystallographic, so we don't include them in our list.
But we'll discuss them when we come back, and then take a look also at the cubic symmetries.
Impossible to draw, but really not all that difficult to understand.
So let's stop at this point and take a stretch.
People are stretching already.
They need it.
We'll resume in 10 minutes' time.
--questions about what we've done.
I think it's been fast, but hopefully if you understand the principles, we didn't go too fast.
But any questions on what we've done?
I guess you haven't had a chance to think of questions yet.
Let me take care of our oddball symmetry at the end of the chart that I handed out-- and this 4 bar-- and ask what we can do there.
Having discovered the four bar operation in 2, 2, 2 with diagonal mirror planes, we can consider that as a new type of symmetry element.
And this is the symbol for it.
And this would take a pair of objects and repeat them by a 180 degree rotation.
And then another pair of opposite handedness, opposite chirality would be rotated 90 degrees and inverted, rotated 90 degrees and inverted.
And I mentioned last time that a solid that has this shape is something called a sphenoid.
And the 4 bar axis takes a pair of faces that are up and a pair of faces that are down and skewed by 90 degrees.
Now what can we do?
If we add a mirror plane that's perpendicular to the 4 bar axis, the right-handed one goes down, the left-handed one comes up.
And it becomes simply 4 over m, which we've already got.
If we add a vertical mirror plane through the 4 bar, if you look a bit earlier at 4 bar 2m, now we've got the 4 bar, now we've got the mirror plane.
Not surprisingly, we get the two mirror planes.
And so S4v is going to be the same as 4 bar 2m.
And that's something we already have D2d.
So that exhausts the possibilities.
4 bar just stands by itself.
At a vertical mirror plane, it's D2d 4 bar 2m.
At a horizontal mirror plane, it becomes 4 over m. The diagonal mirror plane is not possible.
There's nothing to place the mirror plane diagonal to.
And if you add inversion, it changes into 4 over m. So this one is an odd group.
It sits by itself.
Except that we could start there, and add a vertical mirror plane, and get 4 bar 2m once more.
That leaves the cubic ones, which are really easy to deal with because a cube has such high symmetry.
We all know what cubes look like.
But when you show arrangements of motifs, it gets a little bit confusing.
So let's take a look at the tetrahedral groups.
T is a combination of twofold axes coming out in directions corresponding to face normals to a cube.
So these are the twofold axes.
And then there are threefold axes coming out of directions that correspond to the face diagonal.
So this is the jack-o'-lantern stereographic projection, which is something I love to draw at this time of year.
What will the pattern look like?
Well, it's going to look like 2, 2, 2.
So that's a subgroup.
So let's draw in a pair of objects on either side of the twofold axis, and that's the pattern of 2, 2, 2.
Now this guy here is lurking off of a threefold axis.
So that threefold axis is going to repeat it three locations that are 120 degrees apart.
And so there's going to be a triangle of objects up above.
This threefold axis is going to reproduce this into a triangle of objects that are down below.
The twofold axis will take this triangle of objects and move it over to here.
Also, up.
And the twofold axis that's vertical will take this triangle and move it down to three that are below.
So it looks very complicated in projection.
But it's simply a planar triangle of atoms here rotated 180 degrees.
So you've got one like this, one like this.
And then down below, two other planar triangles of objects.
Remembering what the symmetry of a tetrahedron looks like, you can draw this arrangement of symmetry elements relative to a tetrahedron.
And the threefold axes now are coming out normal to the faces.
The twofold axes coming out normal to the edges.
And imagine that we put a triangle on this face, a similar triangle on the face behind, and a triangle pointing in the other direction down this way.
These are the three below.
So that's the pattern of 2, 3.
Take a tetrahedron, smack a triangle on the two upper faces, and an equilateral triangle on the two lower faces.
And the threefold axis that comes out here comes out of the center of this triangle and out of the center of this triangle.
The next one that you can get from this is Th, the tetrahedral arrangement of axes with a horizontal mirror plane.
If you put in a horizontal mirror plane, it's going to take this triangle and reflect it down.
It's going to take this triangle and reflect it up.
And they will overlap in projection.
And so that's the pattern for Th.
There's a mirror plane perpendicular to a twofold axis.
That creates an inversion center.
So we can call this a 3 bar axis.
So that's Th 2 over m 3 bar.
So just have triangles on the upper and lower faces as well.
The one remaining one is Td.
Here are the twofold axes.
Here are the threefold axes.
And now if we put a diagonal mirror plane in, diagonal with respect to what?
Well, diagonal with respect to these twofold axes.
So this mirror plane goes down like this, passes through a twofold axis.
There must be a mirror plane 90 degrees away.
And the threefold axis is going to repeat these mirror planes so that we get mirror planes that are at an angle with respect to the vertical twofold axis.
And this is Td.
And this, in the international notation, is called-- if we can figure it out-- the diagonal mirror planes with respect to these twofold axes have changed them into 4 bar axes.
So this is called 4 bar 3m, which doesn't look cubic at all.
So that's a little bit deceptive.
I am foolhardy for even trying to do this.
And so I don't think I'm going to do it.
4, 3, 2, we know what that looks like.
That is fourfold axes coming out in directions that correspond to the face normals to a cube.
A threefold axis coming out of body diagonals.
Twofold axes between all of the fourfold axes that are normal to the edges of the cube.
So all sorts of rotational symmetry.
That, if you want a pattern, has a triangle that's on about each of the threefold axes.
But the one that is up, points in the opposite orientation of the one that's on the threefold axis coming out of the other end of the cube.
So you have one triangle that's like this, up.
And another triangle like this that's on the other end of the threefold axis that points down into the blackboard.
What can you add as other extenders?
You can put in a mirror plane that is perpendicular to the fourfold axis.
And that leaves everything unchanged.
Now we've got a mirror plane passing through a fourfold axis, so we have to have mirror planes at 45 degree intervals.
And that will create mirror planes there.
This horizontal mirror plane goes through this fourfold axis, so there must be mirror planes at 45 degree intervals.
So there's another one like this and 90 degrees away as well.
The fourfold axis is perpendicular to a mirror plane, as are these other twofold axes, so there's an inversion center.
Here's a fourfold axis with a vertical mirror plane going through it.
And it has to have a vertical mirror plane going like this at 45 degrees away.
So that's the symmetry.
This is the regular symmetry of a cube or an octahedron.
And I will not, even if pressured, try to draw a pattern that conforms to that.
We've got the fourfold axis perpendicular to a mirror plane.
And this is called O sub h, O with a horizontal mirror plane.
The fourfold axis in 4, 3, 2 has got a mirror plane perpendicular to it.
The twofold axes all pick up mirror planes perpendicular to them.
There's an inversion center at the point of intersection.
So we label this axis a 3 bar axis.
So that's O sub h, 4 over m, 3 bar, 2 over m. And this, as I say, is the symmetry of a regular cube or an octahedron.
Nothing else we can do here.
There's so many symmetry axes all over the place that there's no way we can snake in any diagonal mirror plane.
Because there is no second axis of the same kind adjacent to either the twofold, the threefold, or the fourfold axis.
So we can't put a mirror plane in here between the threefold and the twofold.
We can't put a mirror plane in here between the fourfold and the twofold.
So we're done.
And this is the final and 30 second combination.
So there are 32 crystallographic point groups
Professor?
Yes, sir
So if m3m is a regular operation, are there symbols [INAUDIBLE]?
O is the symbol for 4, 3, 2.
And what we added as an extender is a mirror plane perpendicular to the fourfold axis.
That is just one way of adding an extender.
This is also Oi.
If we add an inversion center, we get all this.
And if you look at the ones have been honored by being designated by a symbol, the horizontal mirror plane takes the precedence.
So for example, for 2 over m, 2 over m, 2 over m, you've got two different vertical mirror planes, and you've got one horizontal mirror plane.
But it's called 2 over m, 2 over m, 2 over m. You can call it 2, 2, 2, m, m, m. That's also a possible symbol, but much more of a mouthful.
There's some arbitrariness to the symbol because the arrangement of symmetry elements is what's real.
And we decide how we want to devise a notation to label them.
And as we've seen, there are two different people who adopted a different code for giving them names.
So if it's rational and informative, it's a good notation.
Interestingly, the international notation, on the one hand, and the Schoenflies notation, on the other, complimentary.
The international notation tells you unambiguously what you have.
So 2, 2, 2 over m, m, m tells you three orthogonal twofold axes if you remember what came out of [INAUDIBLE] construction.
And perpendicular to each of them is a mirror plane.
And that's what you've got.
Schoenflies tells you how you derived it.
You took D2, and that's the dihedral group 2, 2, 2, and you added an h, a horizontal mirror plane.
And all hell broke loose.
And you got mirror planes perpendicular to all the twofold axes.
So there's a certain complementarity to the two different notations.
And the people who do diffraction and crystallography for the most part follow the international notation, the Hermann-Mauguin notation.
The people who do condensed matter physics use the Schoenflies notation.
Because it's more inscrutable.
And condensed matter physicists like to be inscrutable because it's how you gain respect.
So both notation survive, but are more prevalent in some disciplines than in others and vice versa.
Let's take a brief look ahead.
What remains to be done?
We now have the 32 crystallographic point groups.
These are the way things can be arranged by symmetry about a fixed point in space.
If we were to proceed in the same way that we did for the two-dimensional space groups, what we should do next is decide what sort of three-dimensional space lattices these symmetries will require.
And then proceed to drop each of the point groups into each of the lattices that can accommodate them.
And then use the tricks that we've used in two dimensions, take mirror planes and replace them by glide planes.
And then having done that, we would take the mirror planes and the rotation axes and interweave them.
And lest that job seem too daunting, let me point out that we already have 17 of the space groups.
All we have to do is take the plane groups, take a translation that's perpendicular to the plane of the plane group, and let all the rotation axes and mirror planes extend up indefinitely in three dimensions parallel to that translation.
So we already have 17 of the three-dimensional space groups.
They look just like the plane groups except they extend in a direction that's perpendicular to the plane of our original two-dimensional group.
So we're already a long way towards deriving a three-dimensional space group without really knowing it.
But what I would like to consider next is the lattices that are required for three dimensions.
And I've already given you how one can approach this problem.
Take the lattices that have been required in the two-dimensional plane groups.
And if the presence of a threefold axis and the base of the cell require, say, net that is hexagonal with a1 equal to a2 identically in magnitude and exactly at 120 degrees with respect to one another.
Let's let the third translation be perpendicular to the base.
If it's hexagonal, we have to have at least a threefold axis here.
And we've found that adding a threefold axis to that net gave rise to two other threefold axes in the center of the triangle.
So one lattice would be one in which the third translation, which we'll label c, is straight up.
In other words, perpendicular to the net that we derived in two dimensions.
What we will have as a constraint is that if we pick a certain translation that is not normal to the plane group, and let's say it terminates here in projection.
So the third translation goes up and over.
If there's a threefold axis at this end of the translation, there must be a threefold axis at the end of that translation.
And that mucks everything up, and we no longer have a group.
We can't have another threefold axis poking down through the plane of this two-dimensional plane group.
But it is perfectly OK if the third translation moves this threefold axis and puts down a lattice point and another threefold axis directly over this one.
Or alternatively, another choice for T3 would be this one.
That would put the lattice point in a threefold axis directly over this one.
So again, we have threefold axes extending normal to the plane of my drawing.
And I've created no threefold axes.
So with a threefold axis, there are three potentially different space lattices, one in which the third translation is normal to the plane of the plane group, another one where the translation terminates over the other two twofold axes.
And so this is the way in which one would proceed to derive the space lattices.
Buerger does not do it this way.
He doesn't look at the plane groups.
He looks just at where the axes sit in the nets.
No mirror planes whatsoever.
And I'll return to the point.
And the place where I think he's wrong is that if you add a twofold axis to a centered net, you get twofold axes in all of these locations.
These are the twofold axes in C2mm.
And if that's all you look at, there's no reason why this should be the plane group.
And it looks as though you can make T3 terminate over this twofold axis.
And that would give you a peculiar triple cell with three lattice points along the long diagonal of a rectangular net.
And you say, wow, 15 space lattices.
We're going to be famous, if not rich.
You can't do that because this plane group can exist only if there are mirror planes in here like this.
And then through these twofold axes, these have to be glides.
So you can't take 2mm and put it on top of 2gg.
It's impossible.
So there are 14 space lattices.
The 15th doesn't exist.
But it looks as though it's possible in Buerger's treatment when he looks just at the placement of the rotation axes alone and not the location of any mirror planes that are in the plane groups.
So that is where we're going to go next.
And that is a process that actually is surprisingly simple.
And we will get the space lattices very quickly.
We'll get a number of space groups along the way, as we've just seen.
And then we will cut to the bottom line and just look at how this information is tabulated in tables for you in a fashion that's analogous to the representation of the plane groups.
So we're pretty close.
We'll be wrapping things up in another two or three meetings.
What I would like to do in the time that remains though, the point groups are very difficult to visualize unless you look at real crystals.
So what I'm going to do is pass around a collection of models of actual crystals.
I'm not sure I can tell you what every one of these models represents.
These are curious things.
I would beg you not to drop them on one of the sharp corners.
These are made out of wood.
They're made out of pear wood in the Black Forest by elves.
No, that last part is not true.
But they are incredibly expensive.
Because the angles in these things have to be exactly right.
So they are made on the same sort of machine that you use for faceting diamonds.
It's something that has two degrees of freedom, so you can get a surface that you want on the material exactly parallel to a grinding surface.
And then you have a provision for advancing it normal to that direction by a controlled amount.
If it were not precise, you would not see sharp edges and sharp corners.
So these are incredibly expensive.
I, therefore, ask it that you hold them tightly with both hands.
Don't drop them on the floor.
And I'll take around a couple of handfuls of these.
I don't know if I have enough for everyone.
With the tables of the point groups in front of you, why don't you try to identify the point group that's possible in each of these models.
This is a big one.
So I'm sure you won't drop that one.
I don't think I have enough for everyone, so I'll start passing them out to every other person.
Maybe you can look on with the person adjacent to you.
Take one of these.
Pass those over.
Here you go.
I gave you that just to be mean.
That is an example of something that's called a twinned crystal.
You can't have reentrant faces.
Yeah, that's actually two crystals that are intergrown
So we can't identify the [INAUDIBLE]?
You can look.
Block one of them out in your mind, and try to imagine what it would look like
[INAUDIBLE]
Yeah.
Here's an example of another one.
That's actually gypsum.
And that's a very common twin in gypsum.
And if you could take this and rotate it-- this one doesn't rotate-- rotate it 90 degrees, then you would have a crystal that had a parallelogram shape.
And actually that's been rotated by 90 degrees about an axis.
That's within the base
No, no, I think that's just 4m or 4 over m
Yeah, there's no mirror there.
That's a 4 bar, I think
Oh, is that the inversion?
There's no-- neither.
I am a touchy-feely guy.
If I see something up here, I look down here and try to feel something that's parallel to it.
But you're right, you're right.
If I rotate 90 degrees-- remember rotoinversion is the same as rotoreflection.
So I could rotate to here and then reflect down, and I get this one
So it's 4 bar
It's 4 bar, yeah.
And you see in the cross section, it is square.
There's a little square on top here, so there is a fourfold aspect to it when you look at planes in a special orientation
I have a question about 4 bar.
4 bar in planes are twofold rotation
A twofold pure rotation.
Probably the best example of 4 bar is a tetrahedron.
Two faces on top, two faces twisted 90 degrees, and inverted down
So that means an 8 bar would be a fourfold proper rotation.
So why cannot I put an 8 bar in the point group?
Because it's only a fourfold rotation, and that's allowed
The reason is that there's no origin to translations.
So if you had translations in an 8 bar crystal, you would have four translations that were repeated by rotation and then another translation 45 degrees and inverted down.
But you can assemble those at this same point.
So you would have eight translations 45 degrees apart.
And that's impossible in a lattice.
Need another one?
I'm sorry
Do you need another one to look at?
No, I'm good
OK.
I can't have you just sitting there doing nothing
So is this half of this [INAUDIBLE] material?
Yeah, that's twinned by a 180 degree rotation on the 1, 1, 1 plane
Oh, yeah, the 1, 1, 1 [INAUDIBLE]
Yeah.
That is actually half of an octahedron.
This is a-- whoops, no, it's not.
Yes, it is.
No, I think that's just the threefold-- 3, 3, 3.
3 bar, 2 over m. And it's been rotated by 180 degrees, which is not a symmetry transformation.
So the 3 bar is rotated 180 degrees
That's a 3 bar, 2 over m?
I think so.
I think it's a 3 bar here, a face here, and a face down below, inverted, 60 degrees away.
And they're mirror planes.
And there two full axes perpendicular to those mirror planes, I do believe
I don't see the twofold axes
You're only seeing half the crystal, and I think that's the reason why.
This is the mirror plane here.
And this edge is parallel to that mirror plane.
And there's a twofold axis that comes out of the middle of that edge.
You can't see it.
That edge.
And the mirror plane is exactly parallel to that.
It's tough to see because you're only seeing half of it
[INAUDIBLE]
Yeah
This [INAUDIBLE]?
Yeah.
This is a fourfold axis perpendicular to a mirror plane.
There's a threefold axis coming out here and a twofold axis coming out here.
And there is a mirror plane perpendicular to the fourfold axis and perpendicular to the twofold axis also under the mirror plane.
So this is 4 over m, 3 bar, 2 over m
[INAUDIBLE]
That's the rotational symmetry.
But you get all sorts of mirror planes coming through here.
So really, it's this one.
Mirror plane this way, mirror plane this way, mirror plane this way.
So if we set it up relative to this, there's the fourfold coming out here, here's the fourfold coming out here.
And there are mirror planes this way and also this way, 45 degrees away.
Nothing to do?
Can I steal one that you're not working with?
There's a--
This one is [INAUDIBLE]
That's 2, 3 with no mirror planes.
Oh, no, no, that's not true.
No, that is 4 bar, 2m, believe it or not.
Look at the three different directions.
These two corners are the same.
There's a little, tiny line segment there
Yeah, so there is a mirror plane right here?
Yeah, but there's no mirror plane going through the other edges.
So there's a mirror plane here, mirror plane here.
And then there is twofold axes coming out of this little straight line segment here
Really?
Yeah.
And that's different from this.
So these two edges are the same
Where is the principle axis?
The principle axis would be this one
This one?
Yeah
So basically, I have this mirror plane right here.
Then twofold axis in the middle of the circle.
So now here, where is it?
Well, it's hexagonal.
So you want to back up a little bit.
Right there
This one?
No, sorry.
This one.
Threefold axis, twofold.
That's perpendicular to a mirror plane
So here's my mirror plane
Coming right out of this little edge here
Let's say like this.
No mirror plane?
No
Which one is this one?
It's like this one
Let's put it right here.
Here is--
Like this?
No, you've got to get it up like this.
OK.
There is a mirror plane
No, this is not.
No, the mirror plane is right here
Yeah, OK.
It goes up like-- oh.
Let me get it set up here.
OK.
There is the mirror plane.
This face is different from the others
And the twofold axes are on the edges
Yeah
So basically, there are three for one mirror plane
Here are the mirror planes coming through this way, this way, this way.
They're 60 degrees apart
You say there are three mirror planes?
Yeah, this one, this one, and this one
I don't agree.
This one is no mirror plane
You're right.
You're right
There is only one when you look from above.
But there are two for just [INAUDIBLE] 45 degrees-- or, not 45 degrees.
That's a triangle
It's a terrible one
Because if we decide that the top is one, the principle--
There's a twofold axis.
That's a mirror plane, and that's a mirror plane.
And it looks like it is 2nn.
I think it's 2nn
OK.
There's nothing further?
Nothing further.
Got that one?
4 bar 2m?
I had to look to find a twofold axis.
That was the tricky part
No, no threefold axis
Well, twofold
Yeah
Finding those was [INAUDIBLE]
That's it
I can see the mirror planes
That's the twofold axis
Yeah
And this two on top and two underneath skewed by 90 degrees, that's a 4 bar.
So that's 4 bar 2m.
You have nothing to do?
Oh, you're talking with them,
What is this one actually?
There is 4, 4, 2, 4, 3, 4, 3
Right, so it's cubic.
And mirror planes are going down through the twofold axis.
So that's enough to tell you that it is this.
So let's set it up here.
Fourfold axes are coming out of the points.
So you've got mirror planes this way, mirror point 45 degrees.
Here is this twofold axis.
Here is this twofold axis.
And here are the twofold axes that are in the same plane.
And here's the other fourfold axis
I see.
That one's not fair.
This is actually two crystals that are grown together by an operation that is not a symmetry element of the crystal.
And actually these two crystals have symmetry 2 over m. And they are rotated relative to one another by a 180 degree rotation.
It's not a symmetry element.
So this is something that's called a twinned crystal
Twinned?
Twinned crystal.
So it's really two of them.
And if I could twist this one around so that the crystal continued on in that direction, it would be something that had a lozenge-like shape.
So I should put that one away.
That's only confusing people
Professor Wuensch?
Yes, sir
Is that a 6 over mmm?
Yes, that's 6 over m, 2 over m, 2 over m. Six folds.
Two kinds of twofold, one out of the edge, one out of the corner.
And a mirror plane perpendicular to each of those.
6 over m, 2 over m, 2 over m
So that's the same thing [INAUDIBLE].
So the 6 over m comes from this way.
And then the 2 over m is there
Yeah, so it's [INAUDIBLE]
Yeah, exactly
Let me put that one away.
It's just confusing people
Oh, that one was easy
That's a twinned crystal.
It was easy?
You found it easy?
Yeah, so 3 over m, right?
Yeah, I guess.
This looks as though it might be the corner of an octahedron because you don't see much of it.
But these two faces and these two faces are not related by a fourfold rotation
Well, either way, there wouldn't be anywhere to put a fourfold rotation
Yeah, right.
So if you look at the whole thing, this is actually two intergrown crystals when you see this sort of reentrant angle.
This is two crystals grown together by an operation which is not a symmetry operation.
So this is something that's called a twin, a twinned crystal
But the symmetry is nevertheless 3 over m, correct?
Of the whole thing?
Yeah, yeah, of the whole thing
And this is just 3
No, this is 3mm.
There's a mirror plane down there and a mirror plane down here
You're right
No twofold axis because this end is different from this end
So this is a regular solid, right?
What is it called?
This is a rhombic dodecahedron
Oh, the one you were talking about the other day
Yeah.
This is 4, 3, 2.
And then mirror planes too all over the place
So all of these solids do not have to be members of this list, right?
Because these are finite objects.
They can be [?
twelvefold ?]
and--
They could be.
These actually, in fact, are models of real crystalline materials.
And they're made with the faces and the relative sizes that these minerals actually have

So somewhere floating around is something that's very characteristically quartz.
And there's one that's a flat one with a reentrant angle in it, that's gypsum
[INAUDIBLE]
Yeah, that's fourfold.
Twofold here.
This goes into this.
No mirror planes because these things are inclined.
So it's 4, 2, 2.
Good.
It's easy when you know what to look for.
When you say, they are only a few possibilities.
And if I see a fourfold axis in it, that narrows it down to two or three
Like that.
That's an inversion center running through here.
And there's no rotation there [INAUDIBLE]
[INAUDIBLE] here?
Here through this way?
No, because this doesn't map [INAUDIBLE].
Does that map that by a mirror?
The mirror [INAUDIBLE]?
It cuts across this way
Yeah, I could buy that
You've reached a consensus?
We've got a mirror, and we've got an inversion.
But we're not sure what else we've got
While I like the inversion, I'd also like the mirror.
I don't see the mirror.
All I see is a really an inversion axis there.
I think I'm missing that mirror that you're saying
There's a mirror that goes down this way.
And there's a twofold axis here
OK, that was the one we missed
So that's 2 over m
Over m. And the inversion just falls out
Yeah, that's in there too.
Inversions are nice.
You can feel inversions.
You put your finger on one face and your finger on the other face.
[INTERPOSING VOICES]
Get it?
So this is this one, right?
Yes, very good.
And that is a silicate mineral called garnet
What's that?
Garnet.
It's actually a silicate mineral, which is used as a gemstone sometimes
So this should be a 4, 3, 2 face
Yep
But I don't know if it's the end of the [INAUDIBLE].
Could be m3m
Where are the-- it doesn't an inversion center
Hmm?
It doesn't have an inversion center here
Oh yeah, it does
Oh yes, it does.
Yes, it does.
Actually, the thing to look for are the high symmetry things.
Your eye spots them.
That's got a fourfold axis in it.
OK So are there other fourfold axes?
Yes, yes.
So that has to be based on 4, 3, 2.
Because you have three orthogonal fourfold axes.
Threefold is here.
Twofold axis is here

OK, thanks
There are-- oh, I already saw those ones
You want to look at some more?
All right
Enough for one day?
Professor Wuensch, I've been staring at this one for such a long time.
And I couldn't match it with anything on here
This looks like a tetrahedron except the face is puckered up into this little thing.
So the way I would start with saying, OK, this is tetrahedral in which case if I make these things flat, that's a perfect tetrahedron.
Four sides.
But what's happened is that this face is not quite 1, 1, 1.
The 4 bar axes come out these three directions.
And they have got twofold symmetry, mirror planes running through the twofold axes.
This is the 4 bar axis with mirror planes running this way, this way.
So this is actually 4 bar.
This is still further.
It's a cubic crystal.
This is based on the tetrahedral symmetry.
Here is the 4 bar axis.
Mirror plane this way, mirror plane this way.
Another 4 bar coming out of these two edges.
And then these are the diagonal mirror planes.
So this is it
I think I was looking for a threefold axis in the center
No, the 4 bar comes out of the edge of the tetrahedron.
And this is the other 4 bar coming out here.
Threefold like this and mirror planes going through the threefold axis like that
I see it
It's hard when you look at this thing.
I never saw this thing before.
What's here?
But if you know the results have to be 1 of these 32 possibilities, and this is obviously cubic, and it's based on the tetrahedron, that means there are only three possibilities.
No mirror planes.
If you find one mirror plane, you ask yourself, does the mirror plane go through the threefold axis, or does it miss the threefold axis?
It goes through the threefold axis, so it's got to be this.
And then you know just what to look for
Is it just [INAUDIBLE] m?
This is a dirty one.
This is a dirty one.
If you look at this very carefully, this face is the same as this face, but it's not the same as that one.
It's a little bigger.
So if you say, what's in here, these two things do not come together at a common vertex like these two.
There's another little line segment between these faces.
So there are no threefold axes coming out of this.
Looks like it might be tetrahedral.
Clearly, a mirror plane going this way.
And then there is also a twofold axis coming out here.
So there's a 2 over m and another 2 over m. I did this once before with somebody, and this is the hardest one in the whole set.
So a mirror plane this way.
There's got to be another mirror plane because there's a twofold axis.
4 bar.
Up, down, up, down.
So this is 4 bar, 2 over m. 4 bar, 2 over m.
I think it's time that we got started.
I haven't given a problem set out for a couple of days.
So I have one that will cover some material that we'll go over with and develop today and some new material that we're not going to introduce for a while
[INAUDIBLE]?
Yes, this is Thursday.
And this is for the near future.
I've given you two-- two a week before.
So it's the adjective you're objecting to not the problem set, of course.
Today we're going undertake another major step in our development of three dimensional crystallographic symmetries.
We have just completed derivation of the crystallographic point groups, so these are the macroscopic symmetries that crystals can have.
And then, by analogy, to what we did in two dimensions, the next step would be to take each of these point groups and hang it at a lattice point of some three dimensional lattice that can accommodate them.
We have not as yet developed the three dimensional lattices, and that will be our agenda for today.
Seems like we've already done that, haven't we?
We showed that in two dimensions, a cell can be oblique, it can be rectangular, it can be centered rectangular, it can be square, or it can be hexagonal.
So these are the different kinds of bases that one can have for the cell, right?
So there should be the same number of space lattices.
Actually it's more complicated than that.
And let me remind you by another handout what the two dimensional space groups, the plane groups, look like assembled in all of their glory on one sheet.
And there you see the different ways symmetry can be placed in these five different kinds of lattices-- oblique, rectangular, centered rectangular, square, and hexagonal.
Now a lattice in two dimensions, a potential base for a unit cell, can be rectangular if and only if there is symmetry within that net.
That demands that it be exactly rectangular.
In other words, if a lattice is to be rectangular in the base of the cell, it has to have in it one of the symmetries that correspond to those in the plane groups that give and require, indeed demand, a rectangular lattice.
Similarly a lattice can be truly square, identically square, only if there's a fourfold axis in it.
That demands that it be square.
The same could be said for either threefold or sixfold symmetry.
One of the groups, P3, [?
P3, ?]
mP3, 1m, P6, and P6mm, has to be in the base of the cell if that cell is to have the specialization.
So let's give an example with a twofold axis.
Let's suppose we have a base of the cell that has an unspecialized shape-- two translations that are unequal in length and an angle between them which is a general obtuse angle.
That's about as general as you can get.
But suppose that this net has as well a two dimensional-- a twofold axis in it.
And suppose we pick our third translation such that it picks up this net which is going to be the base of our three dimensional lattice and translates it over to this point in the base.
Well that translation picks up everything.
It picks up not only the two translations that we've been calling T1 and T2 to this point, but it also picks up all the twofold axes that are hanging on these lattice points.
So if this is the terminus of T3 and projection, it will have picked up a twofold axis and plopped down a twofold axis here where no twofold axis exists.
And we can take one of the theorems that we've seen, namely that the operation A pi, followed by translation, is equal to a new rotation operation through pi located halfway along the translation.
And turn this around and ask what happens when we combine two rotations through 180 degrees about different points.
And let's just draw it out to see what happens.
Let's say there's a twofold axis here.
It takes motif number 1, moves it to number 2.
And then here's a second twofold axis and this takes number 2, rotates it over here to number 3.
I is number 1 related to 3?
But in terms of this first theorem, not surprisingly, we have introduced a translation into the space which is equal to twice the spacing between the twofold axes, which I"ll label as delta.
So in other words, if we put a twofold axis down in this place in addition to the ones that we already have, we have created, in the base of the cell, a new twofold axis in this location.
That just mucks everything up because it's going to rotate the translations that we have, it's going to rotate the twofold axes that we have.
And we will not any longer have a group.
We will not have a set of operations that closes upon itself.
So the conclusion then is that if we're going to safely pick a third translation, T3, and combine it with a net that has twofold axes in it, in order to get a group, we are going to have to ensure that the twofold axes line up in every net in the stack.
And there could be several ways of doing that.
We could pick T3 so that it goes straight up.
And then the twofold axes are just all slit up parallel to themselves, and they remain in coincidence.
Or we could pick T3 so that it terminates over the point that's halfway along T2.
And then this twofold axis gets picked up and put down over one that's already there.
This one gets picked up and put down over one that's already there and so on.
So that's an OK choice.
That's a safe choice as well.
Similarly we could pick T3 such that, within the plane of the net, it had a component that was one half of T1, so that it picked up this net and slid it over to this location.
So once again all the twofold axes lined up.
Or, in the same fashion, let it terminate over the center of the cell.
So those are the four ways we can combine an oblique net that contains a twofold axis in such a way that the lattice points in the existing twofold axes remain invariant and are not moved into locations that create new twofold axes.
OK so let's back off a little bit and start with plane group P1-- no symmetry at all.
So this is T1 and T2.
If there's no symmetry whatsoever in the base of the cell, we are free to pick the third translation in any orientation that we choose.
So trying to sketch it in three dimensions.
If this is T1, and this is T2, T3 can be in any orientation such that not only is this angle general, but this angle is general, and this angle general as well.
And if I would try to carefully complete the parallelepiped so that its geometry looks reasonable.
This is going to be the general oblique parallelepiped with three translations that are unequal in magnitude, and three angles between these translations.
Let me write them as the angle between T1 and T2-- not equal to the angle between T2 and T3 necessarily, not equal to the angle between T3 and T1.
And all of these angles are completely general and can assume any values that they like.
So this then is the lowest common denominator.
This is a totally unspecialized space lattice.
It has the shape of the general parallelepiped-- no special relations between any of the translations whatsoever
[INAUDIBLE]
I say they're-- Oh I'm sorry, magnitude are not equal.
[INAUDIBLE] I meant to put not equal signs there.
Yeah, they can have any length they wish.
They're not equivalent.
Any other questions?
OK let's go back then to the case of an oblique net to which we've decided to add a twofold axis.
And that gives twofold axes in the locations that we've become familiar with in the plane group.
And if I pick the translation so that it goes directly normal to the base of the cell, I'm going to have a new kind of lattice.
The lattice is going to have two angles between axes that are exactly 90 degrees, not approximately so, but exactly so, because only if that translation, T3, is exactly normal to the base of the cell will the twofold axes in all of these nets lineup exactly.
So here we have a degree of specialization.
The translations remain unequal in magnitude.
They can be any length they wish and not change things, but two of the angles-- axial angles, T1 to T2, is identically 90 degrees.
The angle between T2 and T3 is identically 90 degrees.
And the third angle, the angle-- I'm sorry, T1 and T2 is my general angle.
So this can be anything.
So that can be any general angle.
T2 and T3 and T1 and T3, want to be exactly 90 degrees.
So we're getting space lattices that have a degree of precise specialization.
And the reason is that the geometry of the base of the cell is inseparable from symmetry in the base of the cell that demands the degree of specialization that makes the space lattice unique.
So let us look at a couple of the other choices for the third translation that will leave the twofold axes in coincidence.
Another choice would be to pick the third translation such that it terminated exactly over the midpoint of T2.
So here's T1.
Here's T2.
Now what I'm going to do is something I have not done to this point.
Having this translation T3 terminate exactly over the midpoint of a translation [INAUDIBLE] was clearly a specialization.
If I were to go up two translations, T3, I would be directly over the end of T2, and that is a much more convenient way of demonstrating that special feature.
So let me pick a new T3 prime.
It goes up exactly normal to the base of the cell, and in so doing, I will have defined a cell which is a double cell.
And the catch is an extra lattice point in the middle of one of the pair of faces.
So I'll refer to this general sort of situation-- this is a double cell.
And I'll call this, in words, a side-centered cell.
And it has, with this redefinition of translations, exactly the same degree of specialization as before-- magnitude of T1 not equal to the magnitude of T2, not equal to the magnitude of T3.
And it has the same specialization of the angles.
The angle between T1 and T3 is exactly 90 degrees.
The angle between T2 and T3 is exactly 90 degrees.
And the angle between T1 and T2 is general.
General, but by convention we will assume the obtuse angle rather than the acute angle.
OK so by picking this double cell, we've defined a cell that has a twofold redundancy.
But in doing so, we have gained the advantage of showing that this specialized geometry is exactly the same as this one-- in the same sense that in two dimensions, the primitive rectangular cell and the centered rectangular cell were both cousins.
They were both had orthogonal translations.
But the other thing that we gain is that even though we can't have an orthogonal coordinate system, if we use the three translations as the basis of a coordinate system, at least we have two 90 degree angles on our coordinate system.
And operationally, that is going to be an enormous advantage.
Yes, sir
So is T3 [INAUDIBLE]
T3 was selected so that we picked it as 2 T2.
I'm sorry, 2 T3 minus T2.
That was my new T3 prime.
I went up two translations in T2 and then back T2, and that put me directly over the lattice point from which my translations emanate
So your T3 is actually [INAUDIBLE]
It goes to the next level up in the stack of nets.
If I would draw this in projection, which is not nearly as clear in some respects, this would be the base of the cell.
Let me use x's for the next level up.
So here's the net offset by half of T2.
And then if I go up two of these T3's and then back by T2, my next layer up is directly over the first one-- the third layer up is directly over the first
Shouldn't it be T3 prime then?
Yes if I were to be consistent, I should call this T3 prime, which I did here.
But now I should call that T3 as well.
I got a little bit careless because we're going to forget that we ever had this T3-- to find this selected as a stacking vector.
OK well in P2 there are four different kinds of twofold axes.
So we could have the third translation terminate over either of the remaining pair of twofold axes as well.
I think that it's apparent that if I pick T3, the original T3, as one half of T1, so that the next layer up terminates in lattice points in this orientation.
And then I pick a T3 prime, which is equal to 2 T1 minus-- 2 T3 minus T1, I will have gotten again a side-centered lattice.
The only difference is that it's going to be a different pair of faces.
It's going to be the O1Oa faces that have the extra lattice point.
So that is not fundamentally a different sort of lattice.
It is also side-centered.
And differs from our second result only in whether it is the side face with the shortest translation or the longest translations of the base that bears the extra lattice point.
One final choice though does result in a new lattice with special features.
And this would be one where I pick the third translation such that it terminates over the center of the net below.
So let's let this be T1, this be T2, and pick T3 so that it moves the origin lattice point directly over the center of the parallelogram face below.
And then if I go up two translations, T3, and then subtract off T1, and subtract off T2, I will again have a T3 prime, which is exactly normal to the base of the cell.
OK this is a type of lattice that you've come to know and love.
In the cubic system, this is referred to as a body-centered lattice
So then, T3 is just [INAUDIBLE]
T3, the original choice, would pick this original net up and move it so that that net was moved-- such that it's corner was over the center of the original net.
So if we go up two translations, we have lattice point over lattice point, and we redefine T3 by going back 1 T1 and minus 1 T2.
And that makes it exactly normal to the base
Not on the sides?
Nothing on the sides .
Now in fact, if we try to do that, we would have something that's not a lattice.
If we tried to have some additional lattice points in the center and the edges of this upper level, we would have destroyed the original T1 and T2 in the base of the cell.
Let me do one or two more and then I think the way in which one proceeds should be quite clear.
If you take this list of all of the plane groups, if we would like to have a three dimensional cell that has a rectangular base, the base can be rectangular only if there is a mirror plane in that net or a twofold axis with the mirror plane.
The two situations are different.
So let's examine first the case of a rectangular net that has either a mirror plane or a glide plane in it.
And they're both going to have the same sort of constraint.
If I have a mirror plane, that-- let me use a slightly wiggly line so the mirror line is not confused with the edges of the cell.
So here are two translations, T1 and T2.
And if I pick a T3 that moves this rectangular net up and over itself, we have to do so in such a way that the mirror planes coincide.
Because if I have a mirror plane in space that passes in proximity to a lattice point, I can turn my theorem around and say that a translation, T1, combined with a mirror plane that is a reflection operation that's removed from the lattice point, I would have to introduce a new translation to delta which is going to be incommensurate with T1.
So I have to pick my third translation in one of two ways.
Let me indicate that by drawing a circle.
T3 can terminate anywhere over this mirror plane.
I could pick this net up, slide it parallel to T2, and as long as I put it down so that this is the lattice point in the next layer up, that will be a legitimate placement of the net.
Because all the mirror planes are simply slid into coincidence with one another.
So if I sketch that thing in three dimensions-- this is the base of the cell and it has a rectangular shape.
And the third translation goes up at an angle.
That's going to be exactly something that I've made before.
And the only difference is that the oblique angle is now on the vertical plane but that's a right angle.
And this stays a right angle.
So that's exactly a lattice of the shape that we had last time with twofold axes, except this cell is sitting on one of its rectangular sides rather than sitting on its oblique base.
So that is something that's not new.
And if I would pick the translations so that T3 had a component that was one half of T1, plus some amount z that is perpendicular to the base, I can redefine that in terms of a cell that has one pair of faces inclined to one another.
And the difference is going to be that the lattice point would be caught in the center of the cell.
This would be my original translation T3 and I'll redefine that as a T3 prime.
And that gives me two right angles and one general angle.
So those are exactly the same lattices that I obtained from stacking a net with a twofold axis in it.
Let me introduce now another piece of jargon as we go along.
The relative angles between the translations are going to determine the shape of the coordinate system that we pick along those translations to specify the geometry of features within the lattice.
So we've had a first case, and this was just a general oblique net.
So we had a T1 not equal to a T2, not equal to T3, and all three angles were general.
All three pair of axes are inclined.
And this coordinate system, if we want to refer to it in terms of words, is called a triclinic.
This is the triclinic system.
And the triclinic system is short for coordinate system.
So this is as general as things get.
And if this is the shape of the lattice there is no symmetry.
Which is the same as saying that the point group, the only point group that can fit into such a lattice, is point group 1.
And then we hit another sort of geometry.
And that is where the three translations had a general angle-- a third translation that was exactly normal to the plane of the first two.
And one pair of axes is inclined.
And you can see that this notation is a little strange, perhaps, but consistent.
And it says something about the nature of the arrangement of cell edges.
So one pair of axes inclined is called the monoclinic system.
And this can contain lattices that are either primitive, side-centered-- with either the longest or shortest translation in the parallelogram that's coming out of the face that's centered.
Or we saw there was one final possibility.
And that was body-centered.
So we'll refer to these lattices subsequently as monoclinic primitive, monoclinic side-centered, or monoclinic body-centered.
These lattices were compatible with point group 2 or point group m. We obtained a lattice of the same shape with point group m or with point group 2.
If it works for 2 and it works for m, why not both at the same time.
It would also work for 2 over m-- twofold axis perpendicular to a mirror plane.
Any question at this point?
Yeah?
Also 2m, right?
[INAUDIBLE] Yeah
No,because as soon as you got a mirror plane, then you have to have a rectangular net.
So it's going to be-- it's going to fall into this family.
You're right, a mirror plane by itself and we've got that here.
That's where the mirror plane falls in the side of the thing.
You're right.
But if it has a rectangular shape, then we have to have either orthogonal mirror planes and a twofold axis.
The base, if the base-- yeah.
OK. Part of the amusing part of this game is that you can, upon appropriate redefinition of cells, come up with a given coordinate system that defines more than one lattice.
And I'll give you-- because you get a little space doing this-- I'll give you a two dimensional example.
Here is one cell.
This is T1, this is T2.
This is a primitive cell.
Nobody should bother to write this down.
Here's a T1, here's a T2.
So anybody know what that is?
This is a primitive cell.
This is a jail cell.
Couple sniggers, thank you.
Still another one-- this is T1, this is T2.
Anybody guess what that is?
That's a soft sell.
I got a million of them.
Know what that is?
It's an underground cell.
Here's one that's kind of dated-- I don't think anybody will get this one.
Does anybody know what that is?
That was a misadventure of the Ford Motor Company that was named the Ed cell.
Well, I'll give you equal time.
Anybody who, after intermission, wants to respond in kind, I'll let you have an opportunity to do so.
But obviously these are not standard crystallographic cells, but they make for a smile in an otherwise dreary business.
I don't know how to proceed from here.
I have a set of notes for you which carries all this out in incredible thoroughness.
So let me pass this around.
Let you take a copy of this and we won't go over every single thing, but I think just to outline quickly the sort of choices that we have.
We haven't yet finished with the rectangular cell, but if it's got something like 2mm in it.
with twofold axes here and mirror planes running this way-- P2mm in the base, P2gm, P2gg, can be stacked only in such a fashion that T3 brings twofold axes and mirror planes into coincidence with one another.
So T3 could be 0T1 plus 0T2 plus some amount z which is straight up.
Or it could be equal to one half of T1, 0T2, plus z.
That would be making the origin twofold axis fall over this location.
Or T3 could be equal to 0 plus one half of T2 plus z.
And they're both going to, upon redefinition, give the same result.
Or T2 could be equal to one half of T1 plus one half of T2 plus z straight up.
And upon redefinition, the first one that would be a brick-shaped unit cell.
That's going to be a primitive cell.
Picking T3 with a component of either one half of T1 or one half of T2 is going to be something that gives us a cell, which can again be redefined.
This was the original T3 in terms of a T3 prime, which is equal to 2T1 or 2T2 minus [?
2T3 ?]
minus T1 or T2.
This is going to be side-centered.
And the final choice where it is a T3 that's one half of T1 plus one half of T2 can once again be defined in terms of-- redefined in terms of a body-centered lattice.
OK this is a coordinate system where all interaxial angles are 90 degrees.
The translations, however, have arbitrary magnitude with respect to one another.
All of these cells could be described as rhombuses.
In this case all angles in the rhombus are 90 degrees, so this is called the orthorhombic system-- orthorhombic coordinate system.
All the angles between the axes are 90 degrees.
So we can get that out of any one of the rectangular groups that has twofold axes and symmetry planes in it.
And conversely, take one of these lattices and let the symmetry elements that we found in the plane group extend through them and you've got a three dimensional space group.
So without emphasizing the fact is we go along, we're picking up a healthy number of three dimensional space groups as we go along
[INAUDIBLE]?
This is primitive.
It's also orthorhombic -- same dimensionality, three translations, and they're all mutually orthogonal.
That's true of these other cells except they're double cells in the case of the side-centered and double as well for the case of the body-centered.
It's time we introduced some other conventions for notation in describing lattices.
We've been using T1, T2, and T3 to designate the three translations that are used to define the cell.
One has to have some sort of rules for picking a standard unit cell so that two people can do an x-ray diffraction experiment, let's say, on a particular material and end up defining the lattice in exactly the same way.
So let me now list, so that we can use these labels as we go along further, conventions for selecting cell edges.
OK one convention is that one should select the shortest translations in the lattice.
And these shortest translations, if there's no symmetry, will be [?
dignified ?]
by calling them a, b and, c-- standing for the x, y, and z directions.
One picks always, if there's nothing special about the translations in the triclinic system, the magnitude of b greater than a.
And in a rational world-- every so often you come to a point like this and it feels almost the silliest talking about underground cells and jail cells-- in a logical, rational thinking world, you would do this.
This is not what one does-- for reasons that have to do with the perverse nature of crystals.
Take b greater than a, one does that always, but you pick c as the shortest translation.
Why?
It has something to do with the peculiar behavior of crystals.
If you were working with a crystal that had the shape of a needle, in other words, greatly elongated in one direction so that the crystal looks something like this.
How would you draw a picture of that?
Would you put it straight up and down like this or would you put it on its side.
Takes up a lot of room this way.
You'd almost certainly instinctively put it this way.
Now if you didn't know anything about the translations down inside the innards of that crystal-- and this was the position that the people who did crystallography solely on the basis of crystal shape and morphology-- and had no idea if there was even a lattice inside of the crystal that caused this regular shape.
When you draw a coordinate system x, y, z, z always goes up.
Who draws a Cartesian coordinate system with z going this way, x going this way, and y going this way?
It's obscene.
We always put z going straight up and down.
You do the same if you don't know what the lattice translations are with the labels on the axes of the crystal.
So this is a, this is b, and this is c. And there were tons of pages published with drawings of crystal morphology on them.
And then along came x-rays.
And when they begin to determine lattice constants of a the material that looked like this, almost invariably they found the translations inside of a needle-like crystal corresponded to a unit cell in the shape of the pancake.
Crystals, when they have a very, very short lattice vector, almost always grow most rapidly in that direction.
So this is a terrible pickle.
Here are reams of books that published descriptions of crystals that had a needle-like-- an acicular shape, if you want to use a fancy term, which had c as a direction which corresponded to the shortest translation in the crystal.
So what do you do?
You don't want to have to redraw all these figures and redefine all these directions in the crystal.
So what you do is cop out.
And you make c the shortest direction in the crystal.
You think this field is rigorous but people wimp out and they do something that's contrary to the logical thing to do.
Interesting question is why crystals should grow most rapidly in the direction of the shortest lattice vector.
And I think I know the answer to that-- I've never seen anybody express it in the writing.
The clue comes from some early work that looked at the growth of the crystal from solution.
And this was done first with-- I think it was some sort of iodide.
I can look it up-- something like cadmium iodide.
And this material in solution grew very, very slowly in the form of hexagonal plates.
And continued to grow most rapidly in this direction.
This was the direction of the shortest lattice translation.
Then all a sudden something happened.
And suddenly the rate of growth and the nature of the morphology changed.
The crystal took off like a bat in these directions and finally ended up in a needle-like fashion like this.
This was an observation that led to the dislocation theory of crystal growth.
The interpretation was that in the early stages of growth, the crystal is growing by accretion of atoms on to these planar surfaces.
And the probability of an atom sticking is less high when the atom is on the flat surface than when an atom is on the surface that might have a little crevice in it so it can bind to two planes at once.
So the original postulate-- and now we all take this for granted-- is that if there was a flaw such that the crystal developed a dislocation on this surface, and an exposed ledge, atoms would accrete and stick more tenaciously to this exposed ledge of the dislocation.
And this is the so-called dislocation-- screw dislocation mechanism-- for crystal growth.
And after the crystal has grown for a while, you can see spiral steps on the surface of the crystal that correspond to this dislocation winding around.
OK so why should crystals grow most rapidly along the direction of the shortest translation?
It's going to be a lot easier to create a screw dislocation with a Burgers vector for a translation that is short, rather than the translation that's 10 or 20 angstroms that the Burgers vector would have to involve-- many, many layers of atoms.
If the translation is very small, just two or three atoms, it doesn't cost you much work to make a screw dislocation.
So I think that is the reason why crystals with very short lattice translations grow most rapidly in that direction, because it's easier to make the screw dislocation with a Burgers vector that's parallel to that translation.
So here's the first flagrant bit of logic that flies in our face.
Let me give a few more definitions and then we'll return to this.
If we have a crystal in which two translations are equivalent by symmetry-- and we would have such a situation if we picked a translation that was perpendicular to a square net.
And would we call the base of the cell something defined by translations a and b?
If this is a fourfold axis, what goes on in this direction is identical to what goes on in this direction by symmetry.
And if these two directions are identical, why call one of them a and one of them b?
So if two directions are equivalent, and this is true of the square net, what you do is call one a1 and you call one a2-- because they're the same thing.
So let's call them both a.
And the third direction then is c. And in such a situation here, c is always defined as the unique direction.
What do I mean by a unique direction?
This is the direction of high symmetry in crystals that are based on square bases and hexagonal bases.
Yes sir?
You pretty much contradict yourself if c is the longest [INAUDIBLE]
Not always, not always
Well, I mean in that picture
Well I drew it that way.
You know why?
Because if I draw it like this, the top of my cell doesn't get in the way of the bottom of my cell.
And if I want to draw a small c translation, then I get into geometrical complications that doesn't look nearly as nice
Well, then I'm just saying two [INAUDIBLE]
No, no, no.
It's different for different systems.
This is what is true for a triclinic system.
And in the case of a brick-shaped unit cell-- orthorhombic.
So this is done for triclinic.
It's also done for orthorhombic.
Not for monoclinic.
For monoclinic, there is a special direction because there is a direction of a twofold axis or maybe a twofold axis perpendicular to a mirror plane.
In that case, if we were rational, we'd call that 1c.
OK I'll close with an opinion poll.
I will invariably, and I'm sure most people do, draw a crystal with a square base, with the c axis the longest.
And if we have someone drawing a sketch of a cell in which the base of the cell is a hexagonal net, we'd again call one a1, one a2.
And perpendicular to that would be the same direction.
Say look, there I did it again.
I made c longer than a1 and a2.
All right.
Let's look at hexagonal crystals.
Let me ask for a poll.
What percent of all crystals with threefold and sixfold axes do in fact have the length of c greater than the length of a?
50/50?
How many would say equally probable to have it the longest or the shortest?
No opinion.
Let me tell you the astounding fact.
Seventy percent of all hexagonal crystals have c greater than a. Astounding.
Let's look at the next-- haven't really defined these terms-- tetragonal crystals.
These are crystals that have a fourfold axis in them.
What percentage have c greater than a?
This is almost a first cousin of the cubic crystal or all three translations are equal.
How many would say more have c greater than a, fewer have c greater than a?
One vote.
More?
That's a good answer, but it's only about 60 some percent.
So let's go down to orthorhombic crystals.
And orthorhombic crystals are the ones that have a brick-shaped unit cell.
How many have c greater than a?
It's declining.
So if you extrapolate 50/50?
Zero?
Who said that?
Are you a wise guy?
You weren't supposed to give the right answer.
It's 0 percent.
It's 0 percent because there's nothing that makes one direction more special than any other.
So you decide on the labelling of axes on the basis of their relative lengths.
And you, by definition, make c the smallest axis.
Congratulations.
To the rest of you, gotcha.
So that's a good note on which to quit.
And a few remaining mysteries of coordinate systems and lattices will be expounded in the second half of our lecture.
Since z, that's straight up.
And that is going to provide for you a cell that has the shape of a square prism.
This would be a1.
This would be a2.
And this would be a3-- I'm sorry, this would be c. And if you took the choice for the third translation as 1/2 half of a1 plus 1/2 of a2 plus some amount z up above the base of the cell, and then redefined a T3 prime that would be equal to-- sorry.
What am I doing here?
Yeah.
This is right.
1/2 of a1 1/2 of a2, and z straight up.
Define a T3 prime as 2T3 minus a1 minus a2.
And that will define for you, again, a cell in the shape of a square prism with a1 and a2.
And now the translation that went up directly over the center of the base of the net below.
Twice that minus a2 minus a1 brings you back to a third translation T3 prime that's normal to the base.
So both cells have the same shape.
This is a primitive.
This is a body-centered tetragonal lattice.
And the symbol that's used to represent the body-centered lattice is I.
One of the few cases, again, where one has to be bilingual.
The German word for body-centered is innenzentriert.
So there's Schoenflies at work again.
So primitive body-centered represented by I.
And that brings us to one final case, and that is the plane groups that have a hexagonal shape.
And here we have a funny situation in that the same choices of the third translation do not work for all of the plane groups.
If you have a base to the cell-- and again, the two edges of that net are identical by symmetry-- if it has a three-fold axis in it and nothing else, than there are two choices.
You can either have T3 be equal to 0a1 plus 0a2 and an amount z straight up.
And that will give you a cell that is a primitive cell.
This would be a1.
This would be a2.
And just as we did for tetragonal, we'd call the third axis c, This will work for a three-fold axis.
So there is a space group P3.
This will also work for all of the other groups.
So there's P3M1 and a P31M.
And a P6 will work.
And a P6mm will work.
So look it here.
We've got five additional space groups in addition to the lattice type.
This is the primitive hexagonal lattice.
For a three-fold axis, we have another choice.
We could pick T3 so that it took the origin three-fold axis and put it directly over this one.
Looks as though you could have a different distinct lattice if you move the origin lattice point over the three-fold axis that sits at a location 1/3, 2/3.
This one is at 2/3, 1/3.
Let me demonstrate, I think convincingly and clearly, that those two choices are exactly the same thing.
And the way that I can show that is, once again, to draw this net a1, a2.
And suppose I pick the part of the offset of T3 that is within the plane of the net as going from this lattice point to this one here.
Then if I go twice that translation T3, that's gonna put me directly over this three-fold axis.
And if I go three translations T3, I'll be directly over the lattice point on the ground floor.
So putting the original lattice point over this three-fold axis or this three-fold axis is exactly the same thing.
And I can change one into the other just by flipping this thing 180 degrees.
So it's obviously the same choice.
This description of T3, if I define the direction of c now as 3T3 defined in this fashion, minus a1 minus a2, is gonna be a funny situation.
It's gonna be a peculiar sort of double body-centered cell.
Of course, along the long diagonal of the hexagonal cell, I'll have one lattice point that's 1/3 of the way along that long diagonal, and another interior lattice point that's 2/3 of the way along that long diagonal.
OK.
This double body-centered cell, so to speak, actually can be redefined in terms of a primitive cell, not a primitive cell that is this particular description of a primitive hexagonal lattice.
This is not a primitive hexagonal lattice.
But suppose I go from this interior lattice point along the long diagonal of the cell to this location and call that a1.
Remember that there is a three-fold axis sneaking down through the center of this triangle, goes through the lattice point, and comes out the center of the triangle below.
If I go up to this lattice point and up to this lattice point, I have three translations that are straddling the three-fold axis.
And that three-fold axis rotates one into the other.
So this new definition of a cell that is in fact primitive a1, a2-- and they're all equivalent by symmetry.
I always get in trouble when I try to draw it, but that actually defines a cell with a shape that we've not seen before.
That is a cell that has the shape of a rhombohedron.
And as I say, I always get in trouble when I try to draw it.
It's got three translations like this and three translations skewed by 60 degree sitting on top.
Everybody can see that's a rhombohedron, can't you?
Say yes.
Be nice.
So this is another primitive cell that's a choice for this triple cell.
And this consequently gives its name to this lattice.
This is called a rhombohedral lattice, regardless of whether you pick the primitive cell in the shape of the rhombohedron or this peculiar double body-centered cell.
So this is a rhombohedral lattice.
And this is represented in a space group symbol by the letter R, standing for rhombohedral.
So just the three-fold axis with this choice of translations, that would be space group R3
[INAUDIBLE]?
Yeah.
a1 like this.
a2 like this.
a3, I'm taking a right-handed system, comes from the centered lattice point.
So let me draw it in projection.
It's easier to see.
Here's the outline of the hexagonal net.
Here's the three-fold axis.
And I've taken a1 coming up like this.
I've taken a2 coming up out of the board like this.
And I've taken a3 coming out like this.
OK.
So these are the three edges of the rhombohedron.
Notice that I cannot have that lattice with a lot of the hexagonal plane groups.
I cannot do it for six-fold axis.
Because a six-fold axis-- P6 and P6MM have a six-fold axis here and three-fold axes in the middle of the cell.
So I cannot have a rhombohedral lattice for P6 or P6<M.
The primitive hexagonal lattice is the only thing that works.
And then we have this curious situation with the two alternative settings of point group 3M in a hexagonal net, P3M1 and P31M.
For P3M1, I have symmetry 3M at both the origin lattice point and in the center of these two triangles.
So for P3M1, in addition to a three-fold axis, I could put in 3M1, stack up plane group P3M1.
But I cannot do this for 31M.
That's impossible.
Because if you look at P31M, there is symmetry 3M at the origin and only symmetry 3 in the center of the triangles.
So if I try to pick this as a third translation for plane group P31M, I'm taking symmetry 3M and plopping it down on top of symmetry 3, and that wrecks the plane group.
So for the rhombohedral lattice, I can only have 3M1 or P3.
P6 has a six-fold axis at the origin lattice point.
The two centers of the triangles are three-fold axes.
So primitive for the six-fold axis, either P6 or P6MM, is the only choice.
So if anybody's counting-- and I wasn't-- we've got 11 lattices so far.
And the only ones that we haven't been able to get by simple stacking of the plane groups are the systems that you all know as your favorites.
The cubic or isometric stems for the fact that the translations are identical in all three directions.
Let me quickly dispose of those by starting with a four-fold axis coming out of one face of a tetragonal cell and saying now if this is a cubic symmetry, I have a three-fold access that comes out, sort of, in the direction of a body diagonal.
But this three-fold axis is gonna rotate that four-fold access to this direction and to this direction.
We've seen that if I look down along the diagonal of a equally axed tetragonal prism, the three-fold axis takes this face, rotates it into this face, and rotates it into this face.
So I have to have a four-fold axis coming out of every one of the faces.
And having that arrangement of four-fold axes means that not only will the two translations be identical in the base of the cell, as is the case for tetragonal, but if I put in that three-fold axis as an extender, all three edges of the cell have to be identical by symmetry.
So I can do this for a primitive tetragonal lattice.
And that's gonna give me a primitive isometric lattice.
I can do this for a body-centered tetragonal lattice.
So that would be indicated by the symbol I.
So it's a body-centered cubic lattice.
And if you add up everything we've done to this point, there are 13 space lattices, a very unlucky number.
And as you all know, there are 14.
We've missed one.
So what have we not done?
We stacked up all the plane groups in all possible ways.
We've taken the four-fold axes out of the face normals, all three face normals of a tetragonal cell.
And that forced it to become a cubic lattice.
What have we done wrong?
Actually, we can get the fourteenth space lattice by a little bit of sleight of hand that is not obvious.
Let's say that this is the base of a tetragonal lattice that we used as the parent for our cubic lattice.
Let me change that primitive square in the base of the tetragonal cell and change it into a centered square.
There'd be no reason for doing that for a plane group.
But let me notice now that if this is the direction of a four-fold axis, rather than having the three-fold axis come out of this direction relative to this super double square, I can make the three-fold axis come out in this direction.
Ah-ha.
So what I have now is a cell that is centered, has a center in the base and has translations that are perpendicular to the base like this.
And if this is the direction of the four-fold axis and this is the direction of the three-fold axis, that three-fold axis is going to put a centered lattice point on the remaining faces of the cube.
So it's only if I notice I can make a double square base to the cell again, put the three-fold axis of a cubic symmetry along the direction of the body diagonal, I generate lattice points in the middle of the other faces as well, and this gives me the lattice known as F, the face-centered cubic lattice.
And the edges of the cell here are all identical, equivalent by symmetry.
So they're labeled a1, a2, a3.
Have you ever seen a lattice constant for a cubic crystal a0?
Has anybody ever seen that?
A lot of people do it.
And if you see that notation in a publication, you will be entitled to sneer at it knowingly.
It's logical, three, two, one, zero.
By extrapolation, if you've got an a3, an a2, a1, you gotta have an a0, right?
Well, not necessarily.
That sort of algebra doesn't apply to labeling of cell edges.
It comes from a curious historic precedent.
Come back, again, to the fact that for literally more than 100 years, people developed symmetry theory and had a pretty good idea that crystals were based on packing of units, molecules, whatever they were, that were on the nodes of a lattice.
But they couldn't measure the size and shape of the lattice.
And that was the brilliance of the famous experiment by Max von Laue because that, in one fell swoop, show that the mysterious radiation x-rays were just electromagnetic radiation of a very short wavelength.
And that's what he really gained notoriety for.
But at the same time, he proved for the first time, unequivocally, that crystals were based on a lattice.
And that was the other side of his double-edged accomplishment.
Up until that time, people learned about crystals by studying the morphology and measuring the angles between crystal faces and recording different faces on a crystal.
I mean, I can just see one of these old guys working a little bit before his wife made him come down and sit for supper because it was getting cold.
And he'd let out a, whoop, huh?
A 13 27 face.
What the hey.
It's a new face on potassium tartrate, something like that.
This was the way these guys got their jollies.
And on the basis of the angles between faces, as I said on several occasions, they could determine a ratio of axes, a ratio of a to b to c, if their assignment of indices to faces were correct.
And how did you know?
Well, along came x-rays, a and you could measure for sure.
So what people started doing to distinguish a to b to c ratios determined from crystal morphology, they put the zeros on it to indicate real values from diffraction.
And when little people came afterwards and they didn't realize the reason for doing that-- nobody's done this for over 100 years because everybody determines lattice constants using diffraction-- but some people have seen these zeros.
So they say, I'm not doing proper notation.
And you know how furious crystallographers get if you don't use proper notation.
So let me write a0 as the lattice constant of my cubic crystal.
You see this commonly.
And there's been no need for it for 100 years.
So anyway, that's the origin of the subscript zero.
There are a lot of dumb things that take place among normally rational people.
I said a moment ago-- and I crossed my fingers when I said it-- that c is the label that is assigned to the unique axis, the axis that is unique because of symmetry.
Sometimes you see monoclinic crystals, which have a two-fold axis and/or a mirror plane perpendicular to the two-fold axis, labelled with this as b, this is a, and this as c, and the angle between them-- that's the general obtuse angle-- as beta.
There is a story behind this lapse of standardness in notation.
How do people decide on what proper notation is?
Well, there is an organization called the International Union of Crystallography, which is the equivalent of the International Union of Pure and Applied Physics and the International Union of Pure and Applied Chemistry.
And they meet every three years.
And there are special commissions for people who like to fight and argue out conventions and nomenclature.
And this is such a silly abomination that there was a move made when they came out with their new edition of international tables to change this and do it the way you do for all of the other crystal systems, namely a, b with gamma between them, and c coming out this way as the direction of the two-fold axis.
And this would be the mirror plane.
And that would put this in accord with tetragonal and hexagonal and everything else.
And it was logical and seemed the right thing to do.
So how do people decide on nomenclature?
This is something that people fight with with more passion than anything else in science.
So you make a proposal.
The proposal goes to the national committees of crystallography in countries over the world.
They discuss it.
They fight about it.
They vote on it.
Then it comes back to the International Union.
And finally, if everybody seems agreed, they present it at the next meeting of the International Union Of crystallography for a vote.
And that's exactly what happened with the proposal to standardized notation.
This took place maybe, I don't know, in the 1950s.
Everybody was in favor.
The discussion proceeded rationally.
Then this little guy jumps up in the front row, short of stature, balding of head, with a very piercing nasal voice.
And he gets up and he enters into this diatribe.
b-axis unique was good enough for Curie.
b-axis unique was good enough for Mauguin.
B-axis unique was-- and all the names were French.
Actually, he wasn't French.
He was Belgian.
So he went on and on and on.
And he so cowed this group of 300 rational delegates, that they didn't cave in entirely.
What they did was they entered every entry for monoclinic crystals in the international tables twice, once with this set of labels, once with this set of labels.
And to indicate that this was the preferred notation, this is labeled the first setting.
And to indicate second-class citizenship no matter what Curie, Mauguin, Bravais, and all of these other guys felt, this was called the second setting.
And I'm not kidding.
I'll bring in a copy of the international tables for x-ray crystallography.
Every entry for a monoclinic crystals is in there twice
And no one in the past 50 years has tried to get rid of the second set?
No.
Actually, the reason is it is expensive to revise these tables.
And I brought in on the first day, you don't remember, my copy of international tables for x-ray crystallography, which is about this big and cost $100.
And then the new version which is twice as large dimensionally, four times as heavy in terms of weight, and I think about eight times as expensive in terms of actual cost.
So you don't do this casually.
If there's something actually new comes out, you make a little supplement that people can stick in the pages somewhere
Don't they still have PDFs nowadays or something?
The last volume came out before PDF, before very intensive use of computers for such data.
The journals now are all done online.
You submit papers online and they're archived online as well in addition to hard copy.
But that's something that was before the time of the publication of the last set of volumes.
Well, I am almost done with talking about lattices.
But I have a thought question.
You've all heard of this castle in Ireland that has this famous rock in the side of the castle called the blarney stone?
And you, to get the gift of gab, have to lean over this rather dangerous wall and somebody holds your ankles and you go down and you kiss the blarney stone.
And that gives you supposedly the gift of gab.
But kissing is a dangerous occupation.
You've all heard about kissing disease mononucleosis?
And with all those people smooching the blarney stone, I've always been afraid the blarney stone might come down with mono.
And if the blarney stone became ill in this session, would it have to be transferred to the Mono Clinic?
I don't know.
All right.
Here in all their glory are the 14 space lattices, or Bravais lattices as they're finally called.
The labels a, b, c, et cetera, are not on them, but the c axis is vertical.
You'll see that the hexagonal rhombohedral lattice is shown referred to hexagonal axes.
In the international tables, you will find all the information both referred to rhombohedral lattices and to the triple hexagonal cell as well.
Now, into these lattices one should drop those of the 32 point groups that the lattices can accommodate.
That's gonna be a lot of additions.
Besides that, there is the option of having the symmetry elements of the point group not intersect at a common point, but interleave them as we did, for example, with the glide planes in P2GG, and in some of the other two-dimensional plane groups as well.
Then we would do what we also did in two dimensions, take the mirror planes and replace them by glide planes, one or the both.
And you might think that we really have an enormous amount of work to do.
We haven't considered any symmetry combinations that involve a horizontal mirror plane.
For example, we look at what symmetry 2 requires and we found the primitive monoclinic lattice, and the side-centered monoclinic lattice, and the body-centered monoclinic lattice.
So there's more than one lattice type into which we can drop a given point group.
But what would you have if you had a mirror plane perpendicular to the two-fold axis?
What kind of lattice will that require?
OK. Let me, in one masterful swoop, convince you that we have done almost the better part of the work.
Let us ask what does inversion require of a lattice?
What does the symmetry of inversion, which didn't appear in any of the plane groups at all because that's inherently a three-dimensional point group operation?
Well, think about it.
All inversion says is that, if, hey, there's a translation that goes up this way, there should be another translation minus T in the opposite direction.
And any lattice whatsoever can make that claim and satisfy that requirement.
So let me now just quickly say that if we have derived the lattice types, and by implication the space groups as well, for a two-fold axis, for a four-fold axis, for a six-fold axis, and so on, we have automatically satisfied the requirements for any point group that results when you add inversion to a two-fold axis, when you add inversion to a four-fold axis, or add inversion to a six-fold axis.
So it turns out that the requirements of a six-fold axis, which gave us the primitive hexagonal lattice, are also met by 6 over M. The requirements of a four-fold axis, which gave us the lattices primitive tetragonal and body-centered tetragonal, are also satisfied for 4 over M. So, any one of the point groups that differs from the 11-- from the 10 two-dimensional symmetries-- namely 1, 2, 3, 4, 6, M, 2MM, 3M, 4MM, and 6MM-- any symmetry that you get by adding inversion to those two-dimensional point groups is automatically going to be happy in the space lattices that we've derived to this point.
So we have done an enormous fraction of the work that's required to derive space lattices, and the space groups as well.
So there is not too much more to be done.
And what I will do for probably the next meeting is to derive a few of the space groups for the lower symmetries which are not too complicated, just to show you how the game plays out.
And then I will mention a little bit about the special idiosyncrasies of the orthorhombic brick-shaped cells where there is nothing more distinct about one direction and the other.
All the inter-axial angles are 90 degrees.
The translations are general.
So there are peculiarities in notation there that are unique to the orthorhombic system.
And then I want to spend, perhaps, one class talking a little bit about crystal chemistry.
Have any of you had a class in solid state chemistry or crystal chemistry?
OK. One.
One of the main uses of space group theory is the description of crystal structures.
So I think I might, as a minor digression, take an hour to talk about packing and some of the concepts used to describe the close-packed structures and ionic structures and give you a problem that ask you, as I already did, to generate a structure, simple-minded two-dimensional structure, and do some interpretation of it.
So, I think this will be worth saying a little bit about just so that one of the main applications of symmetry theory does not escape without any discussion at all.
We will, after about two more lectures, have a complete change of direction and start talking about properties of crystals and how these properties are impacted by the fact that crystals have symmetry.
So we're gonna use some of our results in symmetry, but not all that much, but develop remarkable mileage in predicting anisotropy in crystals just on the basis of the symmetry which that crystal possesses, that certain physical properties as a function of direction are weird surfaces that have lumps and noses and bulges in them.
And certain properties have this anisotropy universally, regardless of the chemistry or composition of the crystal.
Given the symmetry of the crystal, the anisotropy has to be in this fashion.
We'll be able to show that some properties simply cannot exist, period, regardless of composition and structure in materials that have certain symmetries.
So that'll be the direction we take off in.
I mentioned at the beginning of the term-- you've probably forgotten-- there is a lovely book on tensors and physical properties by a gentleman named J. F. Nye, N-Y-E. And the title of the book is Physical Properties of Crystals.
This is a nice place to go for further reading.
We will go through roughly half of Nye's book with a little more emphasis on the consequences of symmetry than what he does.
But if you want to do some extra reading, I asked the Coop to stock this book.
But I suggest that you not rush out and buy it.
It's available in paperback, which means it's cheap, but only relatively so.
There's no such thing as a cheap technical book, paperbacked or not, these days.
It's also on reserve in the library.
So, see if you need the extra help.
See if you would like to do some reading in Nye.
And make sure you want to before you go out and put your money down at the Coop.
The last part of Nye is an elegant, beautiful part of the book.
It deals with the thermodynamic relations between different properties.
And it's a unique treatment that is developed by Nye.
We won't do any of that, but it's a nice thing to have.
So don't rush out and spend your money.
But if you want additional reading and just going to the reserved book collection is not enough, then Nye is not at all a bad copy of a book to have.
I think most of you have probably seen Nye, not the book but the man.
How many of you have seen the famous Bragg bubble raft movie on disk locations, where he floats sheets of bundles and sheers them and you see his dislocation go zipping across?
At one stage in that movie, there's an assistant who tiptoes, trying to be unobtrusive, across the background.
But the camera catches him scurrying behind Bragg.
That's Nye.
So if you've seen the movie, you've seen what Nye looks like.
OK.
I'll call it quits there for today.
And we'll see you next week on Tuesday.
Everyone seems to be here.
At least all the seats are filled up, so why don't we begin.
One thing that I did finally, to be specific, I told you that we would have three quizzes and that they would be uniformly spaced one third, and 2/3 and 2.89 thirds of the way through the term.
And I figured out where those dates would be.
So since I have them, let me give you the date.
So Quiz 1 is going to be on Thursday, October 6.
Quiz 2 will be on Tuesday, November 8, and Quiz 3 will be on Thursday, December 8.
And this, as advertised, is exactly one third of the way through the lectures.
This is exactly 2/3 of the way through the lectures, and this is the next to the last meeting that we'll have.
This is the week before the final week of the terms.
so I deliberately kept it a little bit away from the usual end-of-term crunch.
Being person who is sensitive to symmetry and order, I find it highly regrettable that we cannot have the quiz on October 8.
Then you would on the 8th of October, November, December, you have a quiz.
Unfortunately, October 8, much as that would beautify the schedule, is on a Saturday, so we can't do it so.
I could do it, but I don't think anybody would come.
I've been, believe it or not, going through these problem sets in great detail.
I didn't want to hand them out until I had the list-- and you may or may not know-- that is made up for the registrants in each class, And we get a nice array of photographs with names and department numbers and years numbers under them.
That something, by the way, that only the instructor in a class requests, not even the secretary can do it.
So this is not widely broadcast.
I found to my dismay though when I finally got it that about one third of the pictures are missing.
And I don't know whether you folks have been photographed fairly late on the term, but a third of the pictures are missing.
I already been putting some faces in my memory banks, but to match up names and faces so that I can hand out the problem sets individually, I need more photographs.
So I'll do it one way or another next time, and we'll see some photographs, additional photographs, have appeared in the interim.
I've given you three problem sets so far, and they were really just for fun as well as to get you thinking about the right sorts of things, but there were little puzzle.
I'm, unfortunately, going to have to tell you from now on no more cutesy little puzzles.
You're going to get real problem sets, long, tedious, drudgery.
They're optional though, so that's the saving grace.
So I'm going to pass it around the problem set 4.
And this asks you to demonstrate some of the things that we talked about last time and convince yourself that they really work and ask you to make some simple applications of Miller-Bravais indices to two-dimensional lattices.
And then finally, the last problem is to get you thinking about patterns.
And I've asked you to identify the lattice points and the symmetry elements in two patterns.
I'm almost willing to wager that not one of you will get it completely right, that there's going to be some little thing that you missed or did wrong.
At one time, I bet a bag of potato chips for everybody in the class against the class wagering against me one bag of potato chips.
Well, I'm going to watch my weight, and I've won that invariably, and I really can't eat that greasy sort of stuff.
In any case, you'll see that in the case of symmetry and patterns it is immensely simplified when you know exactly what to look for, when you know the number of possibilities was finite,.
And when we're through with this, you can ask just one question about a pattern and then know exactly what the symmetry of that pattern has to be and exactly where to look for everything else.
I can guarantee you that by the time you finish this class you will never sit in a bathroom and stare at the tile floor and see it in exactly the same way again.
I think that's a good observation because I like to think that that's what education is all about, not sitting in the bathroom, but seeing things differently after you've had the experience than you did before.
That's what education is all about.
So have fun with the patterns.
And again, it will become-- one seat up here.
Today then we're about ready to embark on an adventure.
As I said last time, we'll develop two-dimensional symmetries first because there are relatively few of them, and one could do this rigorously and in great detail.
We're going to have to touch on things more lightly when we get to three-dimensional symmetries so we can finish at a decent point in the term.
Let me remind you that we have so far identified two types of symmetry that can exist in a lattice, onefold, twofold, threefold, or sixfold rotation axes.
Any number of different rotational symmetries are possible, but if you're going to want them to be compatible with a lattice, you must restrict the rotation axis to one of these five, including a onefold axis, which is no symmetry it all.
And in two-dimensions, besides translation, we saw the operation of reflection.
And now we're going to begin to put things together and make elaborate combinations.
The first thing I will ask is can we have more than one of these symmetry elements present and operating about the same locus at the same time.
The answer to that would be, why not.
Because if we look at a sixfold rotation axis, that's really what a twofold rotation axis does combined with what a threefold rotation axis does.
And we plot these on top of one another, and what we end up with is the arrangement of motifs that is generated by sixfold axis.
So in a sense, a sixfold axis is a twofold axis superimposed on a threefold axis.
That kind of a naive way about thinking of this though.
What we're really saying is that what a sixfold axis has is the set of operations 1 A 2 pi/ 6, A twice 2 pi /6 -- that would be 120 degree rotation-- and A pi, A 5 pi/6.
And there's 1, 2, 3, 4, 5.
And we're missing one.
60, 120, 240.
So that would be A twice 2 pi/3, 4 pi/6 So what we're really saying is a sixfold axis consists of these six elements, and these elements constitute a group because I can combine any two of these rotation operations and find something that's already a member of the set.
By saying that a sixfold axis consists of a threefold axis sitting on top of a twofold axis, what we're saying is that there's one collection of elements here, namely identity, or same thing as A 2 pi and A pi, that is a subset of these, or we could say a subgroup.
So what we're doing in saying that these two sets of rotation operations exist simultaneously, we're saying that the twofold axis is a subgroup of the sixfold axis.
And similarly, we can separate out three other elements, a onefold axis, same as A 2 pi; a 120-degree rotation, A 2 pi/3; and a 240-degree rotation, A 4 pi/3.
And these three operations, a group of rank 2, is what constitutes another subgroup, a subset of elements which by themselves satisfy all of the requirements of a group.
So this is one example of how we can have more than one symmetry element, a set of operations existing about the same locus and space.
This is not how one would go about the deriving these higher symmetry however.
Because what we have to do is to add something to the set, something that's called an extender, and then show that all the requirements of a set being a group is satisfied; namely, that a combination of any two elements in the group is also a member the group, that for every operation an inverse exists, and the identity operation is a member of the group.
So you can build up more higher symmetries, more complex symmetries by adding some operation called an extender and then taking all of the products of these elements and see what new operations arise.
So we can have more than one symmetry element operating about the same locus and the set of individual operations that it embodies.
And here sitting all by itself is a mirror plane.
Why don't we combine a mirror plane with the rotation axis?
And we've already seen examples of that.
For example, in the square pattern that these tiles make up, there's a fourfold axis in the middle of each of the squares that are also mirror planes that pass through that location.
So let me look at a combination that is a little simpler to handle.
And what I'm going to say is here sits a mirror plane with a reflection operation that I'll call sigma 1 for the individual operation.
And let me say that I combine now in that space a second reflection operation about a line that intersects the first one.
And the question I'm going to ask now is happens when I take a first motif, number 1, reflected in the locus of the first mirror plane to get number 2, and then reflect that a second time in the locus of the second mirror plane to get one that sits up here.
So the question is now what is the operation sigma 1 followed by the operation sigma 2 equals to?
We can almost answer by the process of elimination.
If this one is left handed, reflection changes the polarity, so number 2 is right hand.
And if we reflect a second time, the right-handed one goes to a left-handed one.
So what we're asking is how do I get from one left-handed motif to another left-handed motif?
Of the three operations that exist in a two-dimensional space could be translation.
But, clearly, I can't take the first one and slide it parallel to itself and make it coincide with the third.
So translation would not change chirality, but that won't work.
What's left?
Rotation.
Yes.
You want to clarify something first?
That third one, it looks kind of left-handed.
[INAUDIBLE]
Oops.
It does.
It does indeed.
Sorry about that.
When you've got your nose poked right in these things, it's easy to overlook it.
Yes.
Absolutely right.
That should be over canted over a bit more like I had it the first way.
Very good thank you.
I don't want to proceed much.
Yes?
What did you [INAUDIBLE] axis [INAUDIBLE]?
We could do that.
And let's do it, and we're going to encounter this sooner or later.
So we have a first mirror plane reflect from a first one, which is right handed, to second one that's left handed.
And then define a second locus, and that one would take the second one and move it over to here, the same distance on the other side of this mirror plane.
And I'll let you answer the question since you asked it.
How is the first one related to the third one?
Translation
Translation.
Yes
It's translated
Exactly So if this distance is delta, the first and the third are related be a translation which is twice delta.
So that would be in addition where we'll eventually want to consider when we have patterns that are based on a lattice.
But right now, I want to consider symmetry operations that leave at least a point in space and move perhaps a line in the case of the mirror plane.
But back to the original question, what is this first reflection followed by a second reflection?
We've just thrown out translation.
That won't work unless the mirror planes are parallel to one of those.
The only thing left is rotation.
And in point of fact, even looking at this diagram, you can get from the first one to the third one by a rotation.
So the answer to the question is that a combination of two reflection operations is a rotation operation about the point of intersection of the mirror planes.
But we can go further and say exactly how large this rotation is.
Let's draw an extra line in here.
And let's say that this angle is alpha.
And if I repeat this by reflection, that angle is also alpha.
Let's label this angle beta.
Repeat that by reflection, and that angle must also be beta.
The angle between the two mirror planes is the angle mu.
And clearly mu is equal to alpha plus beta.
So the answer to the question quite generally is have two mirror planes intersect at an angle mu, and their successive operation is going to be equivalent to the net result of rotating through twice of the angle between them mirror line.
So we don't have to think about that anymore now.
We know if we combine to mirror planes we get a net rotation.
So that is a basic result, and that's a first example of something that I'm going to call by my own pet term.
I'm going to call them combination theorem, the theorem that tells you what you get when you combine two operations.
And what this is going to give us is a way of filling in one box in the group multiplication table.
So knowing what two mirror planes combined at an angle should be, I can now combine the mirror planes at angles which gives me a rotation operation which is one of the ones that's allowed for a lattice.
In general, if I'm not yet putting these symmetry elements in a lattice, I could combine them in such a way such that I got a 17-fold rotation as the angle.
And then the angle between the mirror planes would be half of 2 pi/17.
Could be a lovely symmetry, nice for a pendant or a ring, but not for a lattice and not for a crystal.
So I'm going to want to combine mirror planes at angles that correspond to rotations that are compatible with the lattice.
So what I could do is take a first reflection locus, sigma 1.
And if I want the rotation angle to be 180 degrees for a twofold axis, I would want the second mirror plane with the operation sigma 2 to be at one half of pi with respect to the first.
And for this specific combination then, reflection in the first mirror planes followed by reflection in the second mirror planes to give me a third one, is going to be equivalent to-- Oops.
Back here-- it's going to be equivalent to a twofold axis.
So sigma 1 followed by sigma 2 at an angle one half of pi is going to be equivalent to the rotation operation A pi.
This is not yet a complete pattern because here's a mirror planes that wants to reflect this motifs as well.
So let's just reflect number 3 across the mirror planes 1.
And then we're going to get a fourth object.
Now let's show that this constitutes a group.
We've got the identity operation doing nothing.
We've got the operation sigma 1.
We've got the operation sigma 2, and we've got the operation a pi.
I'll put the same operations again in the vertical column.
Here's operation 1 doing nothing, sigma 1, sigma 2, and A pi.
So doing nothing and following it by one of these four operations just gives you the same thing back again.
And doing these operations followed by 1 will give me the same operation back again.
If I reflect twice across sigma 1 from left to right and right to left, I'm back to where I started from.
So sigma 1 followed by sigma 1 is the identity operation.
Sigma 1 followed by sigma 2 is what we just did.
That turns out to be the operation A pi.
And sigma 1 reflection and following that by A pi is the way in which I get to the fourth one.
So that is going to be equal to sigma 2.
Well, let me rips through the last ones fairly quickly.
Sigma 2 followed by signal 1.
Sigma 2 followed by sigma 1 is the same as A pi.
Sigma 2 followed by sigma 2 gets us back to where we started from.
That's the identity operation.
Sigma 2 followed by A pi reflect from here to here and then rotate.
That's the same as sigma 1.
And the final sequence, rotation followed by reflection would give me sigma 2.
Rotation followed by sigma 2 will give me sigma 1.
And doing the rotation operation twice is the same as the identity operation.
So are the group postulates satisfied?
Yes.
The combination of any two elements is a member of the group.
For every operation, an inverse exists.
And we can answer that question very easily by merely looking at the column under a particular element, and somewhere we find the identity operation.
So sigma 1 is its own inverse.
So, yes, an inverse exists for every operation.
And then finally, identity is a member of the group.
So everything's lights up.
Bells ring.
This set of four elements is entitled to call itself a group.
And in particular, it is a group of rank 4 because there are four elements in the group.
I show you a cool thing.
I noticed that the number of objects in the pattern, the number of motifs in the pattern, is exactly the same as the order of the group.
Why?
Because these four operations tell you how to get from any operation to the remaining three and tell you how to get it into itself.
So identity relates it to itself.
Sigma 1 reflects it to here.
A pi rotates it down to here.
Sigma 2 reflects it down to here.
So there's always a one-to-one correspondence between the elements that are in the group and the rank of a group and the number of objects that are in the pattern.
Yes, sir?
How do you show if the inverse exists again?
The inverse exists if I can find something that combined with an element in my basic set gives me the identity operation.
So it's the operation followed by its inverse has to be the same as doing nothing.
So if I look under each element and find the identity operation, then sigma 2 is its own inverse.
If I look under A pi, A pi is its own inverse.
This is generally not the case.
This is a very simple symmetry.
So really we can work with commas and little figures, and that's one way of getting these symmetries.
But group theory I think you can see is a very nice, very elegant language for describing some characteristics of these patterns.
Now little bit of jargon.
As if this material were not confusing enough, there are two different languages that are used to denote these unique combinations.
This notation where we simply have a number for a rotation axis or an m for mirror plane is something that was first proposed by two mathematicians, Hermann and Mauguin.
And this was adopted as the preferred notation in the international tables, that hefty tome that I brought it on the first day of classes.
So this is also referred to synonymously as the international notation.
Let me put these things out horizontally.
For rotation axes by themselves, the symbol in Hermann notation is simply n. And for lattice, n-- as we've seen many times-- is restricted to what in Hermann-Mauguin notation would be either 1, 2, 3, 4, and 6.
There's another type of notation that is use quite frequently by condensed matter physicist, and this is called the Schoenflies notation after one of the mathematicians who was the first to derive the three-dimensional symmetries.
This was a curious thing.
There were three people in different parts of the world about the turn of the century-- and this was before email or even air mail-- who were trying to derive the three-dimensional symmetries.
Schoenflies was one of them.
And all three of them got to the same place at about the same time, and the results were obtained.
But each one used his own method and had his different notation.
Schoenflies was one of the people who did this.
Schoenflies was a mathematician and based his notation on group theory.
Something called a cyclic group is what the C stands for.
And a cyclic group is one in which all elements are power, so to speak, of some basic operation.
For example, a fourfold axis consists of the set of operations A pi/2; a 90 degree rotation; A pi, which can be written as A pi over 2 squared; A pi over 2 squared; [?
then we need ?]
A pi/2; and A pi/ 2 again; A 3 pi/2, which can be viewed as doing A pi/2 three times; and finally, the identity operation, which is the same as A 2 pi, which is equivalent to doing the basic operation A pi/2 four times.
So the group of rank 4, there are four elements in the group, each one is a power of the basic operation A pi/2.
So a rotation axis by itself as a cyclic group.
Schoenflies indicates these generically by C subscript n, where n is the rank of the axis.
Hence, this is C1.
This is C2.
This is C3.
This is C4.
And this is C6.
A mirror plane by itself is indicated m in the international notation.
A mirror plane is also a cyclic group.
There are two operations, reflection and reflection back to where you came from.
Doing the reflection operation twice is the same as doing nothing.
So Schoenflies also call this one C. And the one thing that is without meaning in English is his subscript S. And Schoenflies was German, and the S stands for spiegel, which is the German word for mirror.
We have in many cities a paper that's called the Daily Mirror.
In Germany, there's a paper called Der Spiegel.
So they use it the same way in everyday life as well as in notation.
So C sub S in the Schoenflies notation is a mirror plane.
Cyclic group the S stand for spiegel.
For the combinations of which we have seen only one, in the Schoenflies notation these are all of the form Cnv, a rotation axis Cn with a vertical mirror plane passing through it.
So this one, for example-- we've yet to look at the others-- would be C2v.
So we'll have to draw the first, and then I'll give you the complete set.
So this is a rotation axis with a mirror plane passing vertically through it.
Let's do a few more.
And I think having done a couple in great detail, we'll see what the others will look like quite readily.
Let's take a fourfold axis and add to the rotation operations A pi/2, a first mirror plane that I'll label sigma 1.
I can permute the order of operations and say that sigma 1 followed by the rotation operation A pi/2 is going to be equal to a second reflection operation that is equal to one half of pi/2 away from the first.
So I'm claiming that if I reflect in the first mirror plane and then rotate by 90 degrees that should be another mirror plane at 45 degrees to the first.
So let me do exactly what I advertised.
Here's the first, one right handed.
Reflect across to get a second one, which is left handed.
Reflect, then rotate, and that would bring my left-handed one up to this location here.
So here's 3, and it's left handed.
And lo and behold, just as advertised, I get from the first one to the third one by reflecting across a mirror plane, which is one half of pi/2.
If let these symmetry elements operate on each other, what I'll end up with is a set of mirror planes at 45 degree intervals.
And what I'll have is a pair of objects hung in the same fashion at every mirror plane.
Is that too fast?
You want to go through that a little slower?
Oh, go ahead say, go through it more slowly.
Everybody's afraid to say, yeah, yeah, and been seem like a class dummy.
Do you want me to do it more slowly?
I see people still writing, so I think what you'd rather have is me be quiet for a bit while you catch up to where I am.
In the Schoenflies notation, this lovely thing here is C4.
It's a rotation axis of rank 4 with a vertical mirror plane added to it.
And the international notation, now the Hermann-Mauguin notation, is just a running list of the individual symmetry elements that are present.
And now I really are going to have a mouthful.
This is a fourfold axis, so the symbol for that is 4.
This is a mirror plane.
This is a mirror plane.
This is a mirror plane.
This is a mirror plane.
This is a mirror plane.
This is a mirror plane.
That's a mirror plane, and that's a mirror plane.
So it looks as though I should call this 4mmmmmmmm, which comes 4mmmmmmmm.
Well, that is a typical reaction.
It's a lovely symmetry.
But that is a mouthful.
So it isn't really necessary to give all these m's So the international notation is a running list of the independent symmetry elements, and that's the new wrinkle that I'm introducing.
It's a running list of the independent symmetry elements.
And all these mirror planes are just different sigma that exist as operations in the group.
But how many different kinds of mirror planes are they?
Well, there are two different kinds of mirror planes, both in terms of the way in which they function in the pattern.
The two motifs related by reflection hang close to this mirror plane, but they're widely separated for this mirror plane.
So the motifs do different things relative to those two kinds of mirror planes.
Another way of asking what's independent is if I start with this 1 mirror plane and repeat it by 90-degree rotations, I'll get these 4 mirror planes 90 degrees away.
So they are not independent in that these mirror planes are all related by the rotational symmetry that's present.
You don't get this mirror plane in any fashion other than saying, if I combine the rotation operation with this reflection operation sigma, the net result is this reflection plane.
So there are two mirror planes that are distinct in this arrangement of symmetry elements, distinct in the sense that they function in different ways in the pattern; distinct in the sense that no other operation that is present will throw these two operations into one another.
Another example, and this is in fact symmetry 4mm, is the square tile.
If you look at the mirror planes there, they are 45 degrees apart.
But one of those mirror planes comes out normal to the edge of the tile.
The other mirror plane comes out of the vertex of the tile.
So they are different in the way they function in the space which has this symmetry.
So, mercifully, we don't have to call this 4mmmmmmmm.
We drop the last 6 m's, and this one is called simply for 4mm, 2 kinds of mirror planes with a fourfold axis.
If you're familiar with the way in which these were derived, the symbol tells you exactly what you've got.
So it's a very useful notation for these symmetries.
Let me pause here, give you a chance to catch up.
Yes, sir?
So wait, all you did here is take two separate mirror planes at an angle of 45 degrees between, and you ended up with a fourfold symmetry
Exactly
But you didn't put on the fourfold symmetry, it was just what fell out of it?
No.
I Got this mirror plane to begin with by combining this reflection operation sigma with the operation A pi/2, and that's where this operation came from.
But a general theorem that says if I have a rotation operation A alpha and combine it with a reflection operation sigma, the combined effect is a reflection operation sigma 2, which is alpha/2 away from the first.
So, again, showing you for this now rather messy diagram, if I start with the reflection operation that takes 1 of them throws of the 2 and then rotates 2 up to location 3, the way I get from 1 to 3 in one shot is to reflect in this locus sigma 2.
Yes, sir?
You could have started with any one of those two and got the third
Absolutely.
Absolutely.
The operations that are present operate everything in the space.
So when I say that there's a mirror plane here that relates this one to this one, it also relates this one to this one, this one to this one, this one to this one, that mirror planes operates on everything.
You can't say that a symmetry operation grab this little packet of space and moves it to another packet removed from it.
To say it's a symmetry of a pattern or of a crystal, it has to leave everything invariant.
OK.
But, again, in terms of the language of groups theory, if you combine this pair of operations, then when you combine everything that you get pairwise, you will get in this case a total of, how many operations?
How many operations are in this pattern?
I said a moment ago the rank of the group is the number of operations that are present.
It's the number of objects that are in the group because each operation in a group tells you how to get from anyone to all of the other.
So, 2, 4, 6, 8, this is a group of rank 8, and there are 8 operation that are present.
And I can rattle them off quickly.
Four operations, identity, 90, 180, 270 for the fourfold axis.
That's 4.
This mirror plane.
This mirror plane.
The mirror plane.
And this mirror plane.
That's 8.
Four for rotation.
Four for reflection.
Other questions?
All right.
Let me then wrap up this quickly and get onto something new.
Did I hear the hiccup or a question?
No.
It was a hiccup.
Not a yawn I hope.
Let me do the highest symmetry of all in two dimensions, and this is if I take a sixfold axis and combine it with a mirror plane.
So here's the first operation sigma.
This one is something of a bear to draw.
This is sigma 1.
Sigma 1 followed by the operation A 2 pi/6 should be equal to a reflection that is an angle one half of 2 pi/6 30 degrees away from the first.
So this will be sigma 2.
So if I reflect from 1 to 2 and then rotate by 60 degrees, I'm going to get one that sets up here.
And the way I get from number 1 to number 3 in one shot is by a reflection sigma 2.
Now if I draw in all of those are mirror planes when they are repeated by the sixfold axis, I shouldn't have given this my 6mmmmmm treatment because there are 12 mirror planes.
Two objects hanging on this one in one fashion spaced in a different way that are belly to belly.
They're back to back here, so it does something different in the pattern.
A pair hanging here.
A pair hanging in here.
A pair hanging and here.
A pair hanging in here.
And finally, a pair hanging in here.
So there are a total 2, 4, 6, 8, 10, 12 motifs in this pattern.
Pairs hanging on mirror planes that are 30 degrees apart and hanging disposed and pointing in different ways on the adjacent mirror planes.
So this is one that we would call in international notation 6mmm and in Schoenflies notation C6v.
So we're making extraordinary progress here.
The one that I did for you initially was C2v, and that's 2mn in international notation.
The only one that I left out to this point is a threefold axis compared with a vertical mirror planes.
And let's start with the operation A 2 pi/3, combined with that a first reflection operation sigma 1 that passes through it.
And for reference, I'll drawn in some lines that are separated by intervals of 60 degrees.
So this reflection from 1 to 2 followed by a rotation 120 degrees should give me this one as number 3.
The first is right handed.
The second is left handed, and the third stays left handed.
And lo and behold, the way I get from 1 to 3 directly is by a reflection sigma 2 across a mirror line that is-- I'm sorry.
This is number 1 down here-- across a mirror line that is 30 degrees away from the first.
If I would complete the pattern, reflect this one across to here, take this pair and reflect it or rotate it up here, and I have six objects.
So this is a group of rank 6.
And it has characteristics that are quite analogous to those we did for the other rotation axes in all respect except one.
The international symbol for this combination is C3v.
We've got a threefold axis.
We've got the mirror plane that we added.
And then we've got a mirror plane that is 30 degrees away from the first-- I'm sorry-- 60 degrees away from the first, one half of 120 degrees.
I claim that this is not the proper symbol because the mirror planes that are listed should be the symmetry-independent, distinct sort of mirror planes.
And is that the case with this one?
The answer is no.
It isn't.
Here is one mirror plane.
It's got a pair of motifs hanging on it.
This mirror plane here is a mirror plane that has a pair of motifs hanging it.
Same is true of this mirror plane.
So all six of these mirror planes are doing the same thing in the pattern.
Each one has a pair of a motifs on either side of it in the same fashion.
And I can get one mirror plane and the motifs hanging on it by a rotation of 120 degrees.
So there's only one kind of mirror planes, so we don't need that m. So all of the other rotational symmetries, m, a onefold axis with a mirror plane; 2mm, 4mm, 6mm have two kinds of mirror planes.
C3v has only one kind of mirror planes.
And another way of showing that is if I look at a trigonal prism.
It's got a mirror plane coming out of a corner and out of the opposite face.
Mirror plane coming out of the corner and out the opposite face.
Mirror plane coming out of a corner and out the opposite face.
And that's exactly the rearrangement of symmetry elements here.
So each mirror planes when drawn in relative to a trigonal prism, which is a body that has this symmetry, each mirror planes does exactly the same thing, as opposed to the square tile or square prism that has one kind of mirror plane that comes out of faces and one kind of mirror plane that comes out of corners.
So there are two different types of mirror planes.
Let's add them all up, summarize, and we've got the cast of characters that we can be use in deriving two-dimensional symmetries.
We've got C1, which is a onefold axis; C2, twofold axis; C3, that's a threefold axis; C4, and that's a fourfold axis; C6, that's a sixfold axis.
Then we've got the additions that involve adding a mirror plane to a rotational axis.
So this is simply m and CS in Schoenflies notation.
Then we added a mirror plane to a twofold axis.
We've got C2 with a vertical mirror plane, 2mm; C3v; and not 3mm, but only 3m; C4v, and that's 4mm; and C6v, and that's 6mm.
Add them all up, and there are 10 unique possibilities, no more, no less.
Yes, sir?
I'm missing a point.
Why is there the mirror plane sigma 2 rather than rotating sigma 1 is not different?
How do you?
The same thing is going on, on this mirror plane as on this mirror planes, on this mirror plane.
Or if you like to see it in terms of a geometric solid, the mirror planes here in a trigonal prism all of them do the same thing.
They come out of one corner and out of the opposite edge.
Each of these has one pair of objects hanging on it.
And that is true of all three of them.
And if you like, this end of the mirror plane here is nothing more than one that's related to the first one by one of the rotations of the threefold axis extended back in the opposite direction.
So the mirror plane doesn't just work up here.
It works down here as well.
It works all through the space.
So any way you want to look at it and whatever terms work for you, each of these mirror planes is the same, but they have a different type of end to them.
There's something different hanging at one end than at the other end.
Think of it in terms of actual physical object
[?
What about the ?]
case of the fourfold [?
there?
?]
In the fourfold, one comes out of the face.
One comes out at the edge.
The motifs are oriented and spaced at a different distance from each of those neighboring mirror planes.
So these two are different.
The fourfold axis never rotates this one into this one.
These two are not different because the threefold axis rotates, if you do it twice, this end into this end of what is one in the same mirror plane as what's down on here.
So what's confusing you I think is that there's a polarity to the mirror planes.
Both ends of the mirror planes do not have the same disposition of motifs on them, but there's nothing that says that this has to be the case.
The motifs could just be at one end.
And that, if you think about it a little bit if it help, figure out-- not on company time though, but on your own time-- what a fivefold axis does.
And a fivefold axis is a non crystallographic symmetry.
But it turns out that C5v, which is a 5 with an m, has only one kind of mirror plane in it as well and for exactly the same reason.
A regular figure that has the symmetry is a pentagon.
A mirror plane comes out a corner and out of the opposite face.
And that's true for all of the mirror planes that are in there and separated by one half of 2 pi/5
That's just whether or not the [INAUDIBLE] axis is [INAUDIBLE]?
Yeah.
That's what I said.
Yeah.
So before I go through all of the rotation axes with vertical mirror planes, the international notation would be 2mm, 3m, 4mm, 5m, and 6mm, 7m.
For the odd symmetries, there's only one kind of mirror plane in a pattern, in the tile, or whatever you want to have.
So there are 10 distinct possibilities, and these are called the Point Groups.
Why?
Because they are clusters of symmetry elements about at least one point that's fixed and embedded immovably in space.
They're called groups because, as we've seen for one simple example, the collection of operations follows the postulates for the set of elements which we have defined as a group.
More specifically, we could call these the 10 two-dimensional crystallographic graphic Point Groups, which is more of a mouthful.
But it emphasizes the fact that there are lots of two-dimensional Point Groups, but the two-dimensional crystallographic Point Groups are those that involve rotational symmetry that are compatible with a lattice.
Hence, they are crystallographic.
But the number of Point Groups is actually infinite if you include the ones that are not compatible with translation.
All right.
I think that's probably a good place to quit.
And guess what?
My internal clock has told me that this is the time for our break.
This is one crossroads in our development.
And what we'll do next for the faint hearted who may not want to have more of this stuff for one day than what we've just done, we're now going to do the final penultimate combination, and say we have also shown that there are 5 two-dimensional lattices.
And the final step will be to say if we have a pattern that has symmetry and is based on translation, we can obtain these by taking each of the 10 crystallographic Point Groups in turn and dropping them into each of the five lattices that can accommodate them.
We would not try, for example, to take a Point Group like 4mm and try to drop it into the hexagonal lattice.
It's not going to fit.
But there will be two, maybe three ways in which we can add a given Point Group to one of these lattice types.
And what we've done when we finished is we have exhaustively derived the symmetries' translational lattices and symmetry operations of reflection and rotation that are possible for two-dimensional crystal.
You might say, well two-dimensional crystal, I've heard that people who do thin film work to make monolayers and really make a two-dimensional crystal.
That's not something you could say only a few years ago.
But why do we worry about two-dimensional symmetries if we're not to be wallpaper designers?
Well, actually, I'll give you one example.
One of the difficulties you had in conveying the nature of a crystal structure is taking something that's fairly complicated in three dimensions and getting it onto a two-dimensional sheet of paper.
And what you invariably do is you project the contents of the unit cell down along one of the cell edges.
And what you have then is a two-dimensional crystal structure with a lattice and symmetry.
So you'll see two-dimensional representations of translationally periodic arrangements of atoms quite frequently when you look at projections of actual crystal structures, which is the only reasonable way to convey the information of any structure once you get on beyond the baby stuff of sodium chlorides, zinc, sulfide and body-centered iron.
OK.
I'm going to pause, suck in air.
I am happy to turn you loose for 10 minutes and will meet here again, let's say, at 10 after 3:00.
That I know of that derives two-dimensional symmetries.
And so, what I will give you instead is a set of notes-- nothing fancy, just handwritten-- but which will carry out each of the derivations in detail, so you'll have something to fall back on if it seems overly complicated.
Any questions on what we've done to this point?
We've really set forth the ground rules and components of our final step.
And that, as I mentioned earlier, is to combine one or more of these collections of symmetry elements around a fixed point in space, with one of the five two-dimensional lattices that can accommodate them.
And when we find a successful, feasible combination, we will have derived something that we could call the two-dimensional crystallographic space groups.
These are collections of operations which really pick up everything-- the whole space-- move it, tumble it end over end, and throw it back down into coincidence with itself.
So hence the term space group, rather than point group, where it's only a point that's left invariant at worst.
OK, so what we're going to do then is to take each of the point groups in turn, and combine each of them, when possible, with one of the five lattice types.
And I remind you that those are the oblique lattice or parallelogram lattice.
And this was what was required by two and one, no symmetry at all.
Then there was the primitive rectangular net, and the symmetry element that required that was a mirror plane, or we could have, depending on how the mirror plane was aligned relative to the translation, we could also have a lattice that was non-primitive, a double cell.
And that was a centered rectangular net.
And either of those was compatible with a mirror plane.
Then we had a square cell-- square net or square cell-- and that was what was required by a 4-fold axis.
And then the hexagonal net, and that could be required by either a 3-fold or a 6-fold.
Well, if you add up the number of point groups that we've listed here, there are only six, but there are 10 crystallographic point groups.
So what about the ones that we have not yet considered adding to a lattice, and these are the nets of the form Cnv.
So let's consider, in turn, each of those point groups and see what they would require of the lattice that could accommodate them.
But let's do 2MM first.
2MM, the 2 is compatible with an oblique net.
The mirror plane wants either a rectangular net or a centered rectangular net.
So all we can do is to add this to a rectangular or centered rectangular net.
Let's put a 2-fold axis at the lattice point.
The mirror plane has to be aligned along the edge of a rectangular net.
If there are two of them, you just put one along each of the edges of the rectangular net.
A mirror plane in a net could either go in like this, or could go, for the centered net, in like this.
So what we're saying, 2-fold doesn't care, add a mirror plane, it means that the parallelogram has to become a rectangle.
You can put the two mirror planes of 2MM in along the two edges of the rectangle, or do this equally well with a centered rectangular net, put the mirror plane in parallel to the edges, just like we did in CM, put the 2-fold axis in at the lattice points.
So there's a lattice point at the corner, and also a lattice point in the middle of the cell.
So for the rectangular net, we could put in either M or 2MM in both of these rectangular nets, the primitive and the centered.
4-fold access requires that the net become still more specialized and have the shape of a square
Professor?
Hmm?
We can't see on this side
Oh, I'm sorry.
I'll pace back and forth, which I usually do.
Then everybody will get equal time.
OK, so I'm glad you spoke up.
4-fold axis requires a square net.
There are two mirror planes, 45 degrees apart, and those mirror planes have to be along the edges of a rectangular net.
Put one of them in this orientation, and that surely is along the edge of a rectangular net, along the edge of a net that's not only rectangular but square.
But it has a higher symmetry, and that's fine.
Looks as though this diagonal mirror plane might cause troubles, but let's note that if we look at the diagonal translations in this lattice, that they will define a centered rectangular net, so putting the other mirror plane in this way puts it along the edge of a centered rectangular net.
So this mirror plane is perfectly happy as well.
So in the square lattice we could pop either symmetry 4 or 4MM.
3-fold is an interesting one.
That's the odd symmetry that does strange things.
Here is a hexagonal net, and that's what a 3-fold axis requires.
Two translations, identical in magnitude, and in terms of what goes on along them, in exactly 120 degrees apart.
Now, I don't readily see any rectangle along the edge of which I could align the mirror plane in 3M.
But it's the oblique shape of this net that throws you off.
Let's notice that if I draw several unit cells side by side, here is a double cell that has a rectangular shape.
OK?
Take 2T1 plus T2 as one edge of the cell, and leave the other one as T2, and we have defined a cell where that is identically a 90-degree angle.
So one of the mirror planes of 3M could go in this orientation, and the other one would be 30 degrees away.
So this would be the orientation of the two mirror planes.
So we're putting in 3M with M perpendicular to the translations that define the primitive hexagonal cell, T1 and T2.
But there's another way you can do it.
Let me draw the same net, the same hexagonal net, and look at a different kind of double cell.
OK, here's my hexagonal net, two translations, T1 and T2, with a 3-fold axis at the lattice point.
And now I will again draw a couple of extra cells-- double T1, and let's double T2.
And then let me point out that I can put in the mirror plane this way and along this way, and here is a double cell, and I can put the mirror plane along this direction and this direction.
So this is crazy.
This is the same lattice and the same point group, but there are two different ways I can align the point group in the lattice.
I can put the mirror plane in such that it's perpendicular to a cell edge, or put the mirror plane in 30 degrees away from that orientation such that it is along the edge of a centric rectangular net.
So I could add 3M as well with the mirror plane aligned parallel to the translations in the net.
So lots of funny permutations here.
Here I have one symmetry element, two different lattices that I can combine with it.
Here I have one point group, 3M, but two different ways in which I can set it relative to the edges of the cell.
Yes?
Why are you putting two mirror planes in each of those figures when you really only could do one mirror plane?
Actually, I need-- well, there are going to be a radiating sheaf of mirror lines coming out of that 3-fold axis
An angle of 60 degrees
They are at an angle of 60 degrees.
It has to be 60 degrees, and that is indeed what I have here.
I've just drawn part of 3M, and if I draw the remaining mirror planes, they would do this.
OK, I think that's fairly intelligible.
Another reason for notes is to draw these diagrams when they're reasonably complex.
On a blackboard, when you're rushing to get through it, is something that's easier said than done, even though you may have done it a number of times.
So there is the job that we've laid out for ourselves.
Five lattices, 10 point groups, and we can drop each of the point groups into one or more of the lattices of the five that we have determined.
As soon as we combine things, we're going to have the situation that we've encountered before.
In general, if I have an operation number one that produces one transformation of coordinates, and combine it with an operation number two which has another change of coordinates, there must automatically arise some third operation.
And we can write it in terms of the language of operators by saying operation number one followed by operation number two is equivalent to an operation number three.
And in terms of the pattern, this says that whenever you throw two symmetry transformations together, a third one pops up.
You can't push it down.
It's going to be there once you add the first two combinations.
You may have to be really clever to see where it is and what it is, but it's going to be there.
It's going to be there.
It comes in automatically.
So this general class of statement, this equality in operations, is something that I like to call, for short, a combination theorem.
And we've seen several examples of this already, notably the one in the last hour that said whenever you combine two mirror lines at an angle mu, you automatically create, like it or not, a rotation operation of a 2 mu.
So as soon as we start making these combinations, we're going to find a new operation arising.
So let's start with the simplest one.
Add a 1-fold axis to a general oblique net.
So there is T1, there is T2, we don't put anything in it, it's just a simple array of translations with lattice points that we can put at the terminal point of these translations.
There's no symmetry.
If we put in one atom, that atom hangs on every lattice point, and that's all there is to it.
It's so simple it's not only uninteresting, it's ugly.
It's just a simple atom arranged in a periodic array.
As with everything we've done so far, it's convenient to have a notation to indicate which particular plane group one has, and the symbol here is to give side by side the type of lattice as the first part of the symbol, and then the point group that you have added.
So what do we have here?
We've taken a primitive oblique lattice, and we've added no symmetry at all, a 1-fold axis.
And here people who have derived symmetry theory assume that the reader knows something.
And so you're going to get a lot of respect when you show your colleagues that you know what this notation means.
All you have to do, if you know the symmetry, is to state whether the lattice is primitive, as it has to be for an oblique net, or whether it's centered, in the case of the rectangular net.
Three dimensions is a lot more complicated.
If the lattice is cubic, it could be primitive, it could be face-centered, it could be body-centered.
So you need a symbol that tells the type of lattice.
In this case it's primitive, and the point group that we've added is 1, so this rather boring ugly thing here is called P1.
Another bit of code.
We're going to do eventually the same thing in three dimensions.
We're going to take just a general oblique three-dimensional lattice, and not adorn it with any sort of symmetry at all, and that will also be called P1.
But to distinguish the two, capital P1 means a three-dimensional lattice, and we won't be working with those for a while.
We haven't even enumerated them.
A lowercase p, that is the symbol for a two-dimensional net.
In other words, a plane group rather than a space group.
So here's the first, P1, and it really doesn't give shivers running up and down my spine to look at it.
So let's do one that's a little more interesting.
Let's take a 2-fold axis and combine it first with a primitive rectangular net.
I'm sorry, what am I saying?
I want an oblique net.
So the symbol-- and I'm assuming this is going to be unique, and we haven't seen anything with a 2-fold axis in it, so it is.
So this would be P for a primitive lattice, 2 for the symmetry that you've added to the lattice.
And since you are all cognoscenti, I don't have to tell you it's an oblique lattice, because you know, don't you, deep down in your heart, that a 2-fold axis requires only that the lattice be an oblique lattice.
Nothing special is required.
So what we will do is to take this general oblique lattice with two translations, T1 and T2, and to this-- remember, we don't deal with symmetry elements.
We have to do this operation by operation.
So what we're doing is taking the operation A pi, the only non-trivial operation contained in a 2-fold axis, and putting that in at a lattice point.
OK. Bells ring and whistles go off.
We don't know what happens when we make this combination.
And let me do this in more general terms, so we can derive the theorem once and for all.
Here's a translation, and let me add to the end of that translation a rotation operation A alpha.
In this case, it would be the operation A pi.
What's the net result of rotating and then translating?
That's a non-trivial question.
So if I'm doing this for a general rotation of alpha, let me take a translation, and I'm going to make my construction in the following way, because I'm really clever.
And I'm going to put this translation T at one half of alpha on one side of the perpendicular to T. And then I'll let that rotation go to work.
And it'll take the translation and move it to alpha over 2 on the other side of the perpendicular.
And now I'll let the translation move to this rotated translation, and if it does so, it's going to move the translation over to here.
Here's a motif hanging on this translation.
It's rotated over to here, and then gets slid over to here.
This one is right-handed, this one is right-handed, this one is right-handed.
How do I get from this one to this one?
Only two ways, translation or rotation.
If we rotate, about what point are we rotating?
Now I'm going to use a definition which seemed trivial.
I said, a symmetry element is the locus of points that is unmoved by the operation.
And if we decided, because these two motifs are not parallel to one another and have the same corrality, that the rotation-- has to be a rotation that relates the two.
When asked finally the question, what point is left unmoved?
Ha.
It's this point where these two translations come together at a single point.
That's the point that's left unmoved by rotating from here to here and then sliding over by the translation T. And I can say exactly where that rotation is.
It's going to be along the perpendicular bisector of my original translation.
And where is it going to be located?
This line makes an angle of alpha over 2 with respect to the perpendicular to T. So that angle is also alpha over 2.
This line and this line both make an angle of alpha over 2 relative to the normal to the translation, so this angle is also alpha over 2.
And look how this has turned out.
The combination of a rotation with a translation is another rotation, B, about a different point but through the same net amount alpha as the original translation.
On top of that, really to nail this down, where should we look?
This distance in here is T over 2, and the tangent of alpha is T over 2 over this distance x.
And if I just solve for that distance, x is equal to T over 2-- I'm sorry.
I got this garbled.
T over 2 over x is equal to the tangent of alpha.
That's what I wanted to do.
So if I solve for x, that distance x is T over 2 over the tangent of alpha, and I can write that as T over 2 times the cotangent of alpha over 2.
So here it is, done properly at last.
Go up a distance x along the perpendicular bisector of T, and you go up a distance T over 2 times the cotangent of alpha over 2.
So this is a general result for any rotation operation.
Combine a rotation operation A alpha with a perpendicular translation-- perpendicular to the rotation axis A-- and the result is a new rotation operation through the same angle, but at a different location.
It's at a distance x, which is equal to T over 2 times the cotangent of alpha over 2 along the perpendicular bisector of T. So this is a general theorem.
This is a fact of two-dimensional life that is always going to be true regardless of alpha.
And the only constraint is that the translation has to be added perpendicular to the locus of the rotation axis, and that has to be the case in two dimensions, because the rotation axis has to always be normal to the space, the two-dimensional space.
In three dimensions, we'll have to generalize this to make the translation have arbitrary orientation relative to the rotation axis, but we've got enough to chew on at the moment.
Yes
[INAUDIBLE]
The distance x is the distance up along the perpendicular bisector.
I'm sorry.
I said it but I didn't write it in.
OK?
Let me go through it again since we've got everything on the board.
We start with the original object, rotate it by alpha, and then we then further map it to a new location by the translation T. Both motifs are right-handed, but they're not parallel, so therefore, they have to be related by a rotation.
If we ask what is the locus about which this location has occurred, we use the fact that the locus of a rotation axis is the only point that's left unmoved by the net transformation, and the place where this translation intersects the translation that has been shifted over by T is a point that sits up along the perpendicular bisector of the translation by a distance T over 2 times the cotangent of alpha over 2.
So that's in the abstract.
Let's now look at a particular application of this to our addition of a 2-fold rotation to a translation.
So here's the translation T. We're going to put a 2-fold axis and the only operation which it possesses, namely A pi.
And according to our theorem, up along the perpendicular bisector of T by distance x equals T over 2 times the cotangent of alpha over 2 should be a new operation B pi.
So in this case, this would be T over 2 times the cotangent of pi over 2.
Cotangent of 90 degrees is 0.
So this says that you go 0 of the way along the perpendicular bisector, which means you stay right at the midpoint of T. So this says that if you rotate through 180 degrees, follow that by this translation, you should find that the net effect is a rotation of pi over 2 about this point.
Boggles the mind, but let's show that it works.
Here is an initial point, initial motif number 1, and it is right-handed.
We rotate by 180 degrees.
Here's number 2, it stays right-handed, and I pick it up and I move it by this translation, and it moves from here to here.
Here's number 3, it stayed right-handed, and the way I get from 1 to 3 in one shot is to rotate 180 degrees about the midpoint of the translation.
How about that?
People are so astounded that they're stunned, except for the people who couldn't see what I was doing, and they're moving back and forth frantically trying to see what I wrote.
I'll let you catch your breath, and then let's do this, use this to derive the two-dimensional space group, the plane group that results when we add a 2-fold access to an oblique net.
And it's turned out very simply, because if I've got the operation B pi here, that implies that I've got a 2-fold axis, because that's the only operation that exists within a 2-fold axis.
So I'll start by putting in a 2-fold axis here.
I'll combine A pi with this translation T1, I get a new rotation operation, B pi, in the middle of this translation, so there's a new 2-fold axis there.
The 2-fold axis here hanging at this lattice point as well, and at these other lattice points.
Combine the rotation A pi with this translation T2, I get a new operation, B pi, sitting in the middle of this translation, so I've got all I need to say that there's a 2-fold axis there.
That 2-fold axis will get translated down here by T1.
This 2-fold axis will be translated over to here by T2.
Then the only other thing that I have to combine this translation with is the translation T1 plus T2, and that combined with A pi says I should have B pi in the middle of that translation as well, so I'll get another 2-fold axis in the middle of the cell.
And that's it.
We've got our first nontrivial two-dimensional space group.
And what do we call this?
Confusing as hell, some might say, but crystallographers would say that it's a primitive lattice combined with a 2-fold axis, this is P2.
I don't have to tell you that it's an oblique net, because you know that that is all that a 2-fold axis requires, that the net be oblique.
What does the pattern look like?
I'll say another thing that's so simple, it takes a while for it to sink in.
The pattern of a plane group is the pattern that is generated by the point group merely hung at every lattice point in the net.
So if this is what a 2-fold axis does, relates a pair of motifs like this, a pattern for P2 is this pair hung at every lattice point.
So all of the objects are of the same pirality, all either left-handed or right-handed.
I've got the same pair at every corner of the cell.
And notice that these new symmetry elements that popped up just like what we found in the point groups, the new mirror planes that came arose as a consequence of the combination that we had made.
And they were independent in most cases, 3M being the one case where they weren't.
These additional three 2-fold axes are independent 2-fold axes.
They're different kinds of 2-fold axes than the ones at the corner.
Not only are they not equivalent by translation, but they do different things relative to the motifs that are in the pattern.
This 2-fold axis has a different relation relative to the two closest motifs than does this one, than does this one.
They relate different objects pairwise in the pattern.
And no two of these 2-fold axes describe the same pairwise relationship.
So these are independent 2-fold axes.
Independent in the sense that they do different things relative to the motifs in the pattern, different in the sense that there is no other operation that throws this one into this one, and therefore, of necessity, would require that the pair about this 2-fold axis be the same as the pair about this 2-fold axis.
OK.
So there is a first nontrivial two-dimensional space group.
I should mention-- probably an obvious fact-- that this also has a cousin in three dimensions.
If I just imagine all of these 2-fold axes extending out through space perpendicular to the plane of the blackboard, and take another translation that is also perpendicular to the blackboard, I would have the three-dimensional space group which has the same symbol except capital P, indicating that it's a space lattice.
So almost at no extra charge, we've done all the work to determine one of the three-dimensional space groups.
Yes?
If you change the order of the combination of the translation rotation, do we get the same operation?
OK. That is a good question.
I really meant to mention it earlier in connection with the point groups.
Does interchange of the order of the operations in a combination change the result, or change, in other words, the symmetry element that is equivalent to the net combination of the two?
How many think yes, it does change?
Could you say that again?
OK, if we have two operations A and B, is operation A followed by B the same result as the operation B followed by A?
Does the order make a difference?
And how many say yes--
The order makes a difference?
The order does make a difference.
How many say no?
Well, this is one of these happy occasions where I can say you're all right.
Sometimes yes, sometimes no.
So let's ask when does it make a difference and when does it not.
Let's look at one of our point groups.
Let me take a non-trivial point group.
This would be point group 4MM.
OK?
So those are all mirror planes of two different kinds.
And let me quickly sketch in the pattern.
The pattern is just a pair of things hanging on the mirror plane, and that pair is rotated by 90 degrees.
OK.
Suppose we look at the operation A pi over 2, a 90-degree rotation in a counterclockwise direction, and let the other operation be sigma 1, this reflection plane here.
Suppose we reflect and follow that by A pi over 2.
We reflect to here and then we rotate by 90 degrees, and that's going to give us the number 3 sitting up here.
So we reflect it and then rotate it.
How about if we first rotate that same object by A pi over 2 and follow it by sigma 1.
So we'll rotate number 1 to here, so this is 2 prime, and you can see we're already in trouble.
If we now reflect along sigma 1, this is 3 prime.
So in one case we ended up here, in the other case we ended up here.
So the order did make a difference.
Let us look at another example.
Let's look at 2MM.
So here's an operation sigma 1, here's an operation sigma 2, so let's first do sigma 1 followed by A pi.
So here's object 1, reflect to here, and then we rotate by 180 degrees, and this is number 3.
Now let's do the operation A pi and follow it by sigma 1.
So we'll take number 1 and rotate it up to here.
So here's number 1 prime.
And then we reflect by sigma 1, and 3 and 3 prime are exactly the same.
So it didn't make a difference.
So the answer is sometimes yes, sometimes no, which is interesting, but even more interesting is how can you tell?
Yeah, you have a question?
In that first identity you drew--
Yep
The right side, it seems like your commas are drawn backwards and that just affects--
Could be.
Mm, no, they're-- ah, these are drawn backwards.
They should all have their tails pointing in.
No, it doesn't work, actually, because even if I screwed up on the orientation, the positions are different.
Again, first time I reflected, and then rotated, so this is 3.
If I rotate and then reflect, I'm way over here.
So this is one location, this is another location, and even if I screwed up by having the tails pointed the wrong orientation, still not the same.
But here, clearly, reflecting and rotating is the same as rotating and reflecting.
I end up in the same place.
So how can you tell?
It's a very difficult thing to prove that this is the case, but you can interchange the order of operations when the two symmetry operations that are involved leave each other untransformed.
And if the two symmetry operations move one another, then changing the order changes the net result.
It doesn't change the nature of the operation, but it does change the locus about which it operates.
Let me show you what I mean.
Here's 2MM.
The 2-fold axis takes this mirror plane, turns it upside down.
It takes this mirror plane and flips it over.
The mirror plane is left unchanged in both cases.
The mirror plane has the 2-fold axis sitting right on it, so it doesn't change the location of the 2-fold axis when it acts on it.
And similarly, this mirror plane takes this mirror plane and reflects it, and it doesn't change anything.
So all of these operations-- the three independent operations, sigma 1, sigma 2, and A pi-- all leave their locus of operation unchanged.
Whereas that, clearly, is not the same for a 4-fold axis and a mirror plane.
The mirror plane, when it acts on the 4-fold axis, leaves it unchanged, but the 4-fold axis, when it acts on the mirror plane, moves it into a new, entirely different location.
And when that is the case, the order does make a difference.
So I can take that corny joke of the mathematicians, what is purple and commutes?
An Abelian grape.
I can say, what is purple and commutes?
It's a purple 4-fold axis in a mirror plane, a purple 4MM.
OK, what other aspect of the plane groups and three dimensional space groups is an analytic representation of the way the plane group maps atoms around?
And I see that it is five of the hour, so let me just make one final statement to indicate what I'm going to talk about next time.
Here's the arrangement of 2-fold axes, and if you have an atom sitting in here, and this is T1 and T2, and we use this as the basis of a coordinate system, so this atom sits at a location x along T1 and y along T2.
And then they ask, what other atoms are related to it?
Well, that 2-fold axis will move x to minus x, and move y to minus y, and those are the only two atoms I get.
So a characteristic of this plane group, if I were describing to you a crystal structure which had this symmetry, I might say something like a lithium in a position with coordinates xy, and therefore minus x minus y would also have to occur.
I have an oxygen at a different number, x prime y prime, and that must then also occur at minus x prime minus y prime.
And it looks as though any time you throw in an atom at a location xy, you get a symmetry related companion to it with negative coordinates.
And this is what is called the general position of the space group.
There's nothing special about it, and you put in an atom here, you get another atom here.
This is going to be a nice economical way of describing an atomic arrangement, particularly if you realize that there are almost 150 atoms that appear in the unit cell for some of the cubic symmetries.
Drop in one, wow, all hell breaks loose, 150 other ones.
But what you can do is codify the coordinates of all these symmetry-related atoms.
And that is a great utility in describing crystal structures, because I only have to give you the coordinates of one representative atom.
But this is not the only thing that happens.
This is the general position because it's general relative to its location with respect to the symmetry elements.
If x and y were both 0, I would have one atom moving to the origin to coincide with the second one.
And if I put one in at 0, 0, that's all I'm going to get.
The 2-fold axis is just going to twirl it around, and the translation will repeat it to the other lattice points.
So if I put in an atom at 0, 0, that's all I'm going to get.
And the same thing would occur for any of these four independent 2-fold axes.
If I let x and y migrate to 0, 1/2, this atom will join together with this atom, and I would get just one atom sitting there.
So there's a position also of the form 0, 1/2, and a position 1/2, 0, and a position 1/2, 1/2 that are the location of these four distinct 2-fold axes.
OK, to have a shorthand way of referring to the general position in these positions, which are called special positions-- what's special about them is that they're right on top of a symmetry element, and whenever that happens, the number per cell is a sub-multiple of the number in the general position, because coalescence has occurred.
And so they're three independent 2-fold axes, so there are four locations where you get only one atom per cell rather than two.
And then to give you a way of referring to these positions readily, they are labeled starting with the most specialized, just going through the alphabet, position a, position b, position c, position d, position e, which is the general position.
And then another piece of information that's almost trivial here, but is not for more complicated symmetries, what's called the rank of the position, and that is the number that you get per cell, one per cell, one per cell, and for the general position you get two.
So the rank of the position is just the number per cell.
And then, if it's a special position, the atom has to sit at a site of some symmetry, and in this case, the atoms all sit at 2-fold axes for the special position.
The general position always has site symmetry 1 always.
So that's the characteristics of the way in which this symmetry can move atoms around.
And this is of great utility in interpreting structures.
For example, suppose our motif was not an individual atom, but was a molecule.
And suppose you had one molecule per cell.
That would tell you the molecule has to fit in a special position.
And the special positions are all 2-fold axes.
So if the molecule has to sit on a 2-fold axis, the configuration of that molecule has to have at least a 2-fold symmetry, or else you can't fit in one per cell in this particular plane group.
So you can use this information to determine structures.
I'm going to give you a problem a little bit later on when we get to three-dimensional crystallography, and I'm going to give you the magnitude of the cell edge and the density of rock salt.
And from that, in 10 minutes, you can find out that there's only one possible structure for rock salt.
The Braggs fiddled around for the better part of a year doing x-ray experiments and trying to calculate what the intensities would be for different atomic positions.
You, in another two weeks, can get the same answer in 10 minutes knowing only the lattice constant and the density and the space group.
So a lot of physical consequences of the nature of the structure, the confirmation of the molecule, the extent or absence of solid substitution of one species for another, can be dredged out of the properties of the space group.
OK, I've run 2 minutes over, but we took our break two minutes later, so let's call it a day and I shall see you on Thursday, hopefully armed with pictures so that I can personally present each one of you with your corrected problem sets.
We had begun to discuss, really what is, although you would not gather it from last time, a fairly straightforward matter.
Set up our usual Cartesian coordinate system.
And then look at the surface of the solid, and applied to this solid a force per unit area, which we'll represent by a vector k. And then put on some labels here.
Let this be O.
Let this be A, let this be B, and this be point C. Then we proceeded to-- I will say, attempt to-- set up a balance of forces between the external force per unit area and the internal forces per unit area, which have to push back on the external force if the solid is to remain motionless.
The thing that I did not carry through is that there are two directions involved.
One is the direction to the normal, to the particular surface that we're looking at.
We'll call that N. And there is the direction that k, the vector k, makes with respect to the three reference x axes, x1, x2, x3.
Let's take k and immediately split it up into the three components, k1, k2, k3, because throughout, we're going to be dealing with the balance of those three forces per unit area in the directions of x1, x2, and x3.
We want to take this surface area and resolve it into three separate areas that are normal to x1, x2, x3.
And what I did at the end of last hour was to look at k and try to express these internal areas in terms of the direction cosines of k. Forget that.
We've split up k into its three components, and I didn't draw in the direction of the normal to the surface N last time.
So I kept saying, direction cosines of k, and people in the audience said, no, no, no, no.
And I said, yeah, because you see, that's the ratio of these two areas.
And everybody said, no, that's wrong, and I said, ha.
And that's where we ended my most inglorious hour.
And they've got this all down on tape.
My reputation's going to be ruined.
So, anyway, let's now do it properly.
And what I will say is that in the x1 direction, we have to have internal forces that push back, and we have to balance those forces in all three directions.
So I'll say is that in the x1 direction, and this as you saw last time, gets very cumbersome because we have to deal with all these clumsy areas, that the force per unit area on ABC has to be balanced for each of the three components of k by internal forces acting along x1, x2, and x3.
So we can say is that the first in the x1 direction, the component of k that acts on the surface ABC in the x1 direction has to be balanced by internal forces.
And let's look first at the x1 direction.
So there are three internal surfaces.
They are the area BOC.
That is this back surface here, and the normal to that surface is x1.
So let me define some forces per unit area that are internal, and I'll use the term sigma for that.
And I will use a first subscript that gives the direction in which that force per unit area acts, and I'll use a second subscript that indicates the normal to the internal surface.
So area BOC obviously, as we observed a moment ago, has x1 as its normal.
But there are three internal surfaces.
There will also be an internal area, AOC, on which a force per unit area acts in the x1 direction.
So I'll call this one by analogy to what I did a moment ago.
I'll call this sigma 1, because it acts in the x1 direction, and it acts on the surface AOC, which has x2 as its normal.
Then finally, there will be a third force per unit area, sigma 1 3, acting in the x1 direction, on a surface whose normal is perpendicular to-- along x3.
The surface is perpendicular to x3.
And that's area AOB.
OK, so this is just a simple balance of forces, external and internal balancing forces, which act in the x1 direction.
Then we'll write a similar thing for the x2 direction.
And we'll say that the x2 component of force per unit area external to the volume element acting on area ABC is going to be balanced by, again, forces now acting in the x2 direction.
So first, there will be a component of 4 sigma 2 acting on the area that's normal to x1, and that's the same area BOC, plus another force per unit area acting in the x2 direction on the surface, whose normal is a long x2.
And that is, once more, area AOC.
And that would be a force per unit area that I'll define, because it's my ball game and I own the ball.
So again, that would be sigma 2 2, and then finally a sigma 2 3 times the area whose normal is x3.
And I think I need not write the third term that follows automatically from what I've written so far.
So now comes the geometry part, where I'm going to get rid of these messy areas, and I'm going to divide through by the external area on the surface of the solid.
I will then write accordingly that k1 is equal to sigma 11 times the ratio of area BOC divided by area ABC plus sigma 12 times area AOC divided by area AOB-- ABC, excuse me.
Then finally sigma 1 3 times area AOB divided by area ABC.
So now I'll introduce a little bit of geometry that caused such difficulties last time.
If I have an area and project it onto another area that makes an angle C with respect to the first, this area A prime is equal to the area A times the cosine of Phi and you could define that Phi in a different way, namely take the normal to one of the two areas and let M prime be the other normal, so C then is exactly the same thing as the angle between the normals to these two surfaces.
So let's now look at what the ratio of these two areas that we have is.
What we're dealing with here is an angle between the normal to the surface, not k, but the normal to the surface, and either x1, x2, or x3.
So the ratio of-- let's look at one of these specific terms.
The ratio of area BOC through the ratio of the area ABC should be the cosine of the angle between the normal to the surface and the normal to area BOC.
Well, the normal to area BOC is x1, so that first ratio of areas is just going to be the cosine of this angle.
And that cosine is just the direction cosine of N1 along the x1 of M along the x1 direction.
So this is just the direction cosine between N, the normal to the surface, and x1.
And let me say then that, by definition, I'll let the direction to the normal be its three direction cosines, L 1 plus L 2, plus J plus L3 times k. These are the direction cosines of the normal to the surface, and not k. So this then simply comes down to sigma 11 times L1, the ratio of the area AOC to ABC is going to be the angle between N and x2.
And that's the angle whose direction cosine is L2.
And similarly, the ratio of these two areas will be cosine minus 1 of L3.
So this cumbersome construction with ratios of areas just reduces to sigma 11 L1 plus sigma 12 times L2 plus sigma 13 times L3.
And the other two equations will follow a similar form.
K2 will be sigma 21 times L1 plus sigma 22 times L2 plus sigma 23 times L3.
Or in general, case of I will be given by sigma I J times L to J.
Easy.
Nice and compact.
Nice and simple.
What can we say about this?
K is a vector which has components k1, k2, and k3.
The direction cosines can be, as we said a moment ago, can be regarded as the components of a unit vector pointing along the normal to the surface.
So k is a vector.
These three direction cosines are the direction cosine to a unit vector normal to the surface.
If we were to be so foolish to try to change the coordinate system., having just gotten it right for the first time in one coordinate system, we know exactly how the two vectors will change because we have a law for transformation of a vector.
If we were to change the coordinate system, the components of k would wink on end of and take new values and we know exactly how the components of a vector will change as we change coordinate system.
OK, if this transforms like a vector and this transforms like the vector, then the 3 by 3 array of coefficients which relate those to quantities must be a tensor.
So we've proven rigorously that sigma IJ is a tensor, in particular, it's a second rank tensor.
And were we to change coordinate system, we know from what we've done in the past, exactly how the numbers that make up this 3 by 3 array will transform.
These quantities, sigma IJ, are something that you've probably encountered, introduced to be sure, probably in a little different way.
These are the elements of stress, and sigma IJ is the stress tensor.
We can immediately, on physical grounds, established that sigma IJ must be a symmetric tensor.
So let's set up two axes, x1, and x2, and x3 obviously is normal to the board.
And let us look at some off-diagonal terms like sigma 12 and sigma 21.
Sigma 1 2 would be a force acting on the x1 direction on a surface whose normal is x2.
So this is going to be a sheer stress and that, to prevent any translation of the volume element, must be balanced by a sigma 12 on the other face that acts in the opposite direction.
We're going to need a convention for defining sign, S-I-G-N, of what constitutes a positive element of stress and a negative element of stress.
And I'll define a plus sigma IJ as a force per unit area acting in the plus xi direction on a surface whose normal is plus xj.
So if this is a force per unit area acting on this surface, and it acts in the direction of x1 on a surface whose normal is x2, what I've shown here is a positive sigma 12.
A sheer force going in the opposite direction would be a negative sigma 12.
A positive sigma 11 would be a stress that's tensile, a negative sigma 11 would be a stress that's compressive.
So here's a sigma 12, a sigma 12 on the opposite face.
If we went away and turned our back, this volume element would start accelerating, getting an angular acceleration, would start furiously spinning about the x3 axis like a top.
So something, if this volume element is, by definition, an equilibrium, something has to balance this couple.
And if it's to stay put, we must have a sheer force like this and a sheer force like this.
And that's the only thing available from the nine elements of stress that will give a net torque on the volume element.
So what are these?
This is a stress that acts on the surface whose normal is x2-- whose normal as x1 in the x2 direction.
So this is sigma 21.
It acts in x2 direction, and its normal is x1.
An x, and then [?
plus ?]
x2 direction on [?
I ?]
surface, which has a normal that is along x1.
So the only way we can avoid an angular acceleration is to say that sigma 12 has to be identical to sigma 21 if there's no rotation about x3.
And looking at the couple's that act about x2, and x3, we could say in general that sigma IJ is going to have to be equal to sigma JI for a volume element in equilibrium.
So stress then is equilibrium is in place, has to be a symmetric tensor.
It is, however, a plain old second-rank tensor like all of the other second-rank tensors that we've spent the last month talking about.
And I think I tried to swindle you on this one before.
If this is a second-rank tensor, the stress tensor must be symmetric and diagonal for a cubic crystal.
In other words, the stress tensor sigma IJ must have the form sigma 0 0, 0 sigma 0, 0 0 sigma because we've shown that that is what is required of a tensor if the crystal has cubic symmetry.
So surprisingly, you cannot subject a single crystal with cubic material to sheer stress.
Any quarrel with that?
Should we move on?
We better not, but somebody better raise an objection.
That's not true, obviously.
I can take a cubic crystal and shear the stuffings out of it.
So what's going on here?
The tensor property that you are applying to [INAUDIBLE] property of the [?
crystal.
?]
But if it is an outside force, it doesn't [INAUDIBLE]
This is one of your good days
[INAUDIBLE] society [INAUDIBLE]
That makes it an especially good day because you're paying attention to what I said for a change.
The distinction here, and I did make it last time-- it was one of the things that I did correctly-- that there are really two kinds of things that have to be described by second-rank tensors.
There are the things that we have been spending most of our time on, and these are property tensors and-- I mentioned last time, too, so I know I said this before-- Nye calls them field tensors.
I'm sorry.
Nye calls them property tensors, also.
And these are something that depends on the structure and bonding and other characteristics of the material.
The stress is something that we we're taking and imposing on an innocent, unsuspecting crystal, and that is something we do.
And we can, by design, twist it, shear it, stretch it, as hard as we like, provided we don't break it.
And so this is something that is externally imposed on the crystal, and this is something that's called a field tensor.
We mentioned last time the meaning of field.
The field is a volume of space and a field tensor is a space in which a value of the tensor is defined as every point, x1, x2, x3, in the space.
Just as an electric field is a vector field and you define the electric field we as a function of every point x1, x2, x3, in the space.
So there's no reason whatsoever that there need be restrictions on the tensor that we apply.
And when we can cross out that interesting, but incorrect, supposition.
But they can be a lot of different forms, special forms, of stress tensors which have special forms simply because of the nature of what we do.
And these have restrictions and equalities that have no counterpart in property tensors.
For example, we can have the cancer sigma 00000000, which is something we could never have identically so for a property tensor.
And what does this represent?
This is a tensor for which sigma IJ is 0, unless I equals J equals 1.
So this is the only thing that's going on, is a force per unit area along the x1 direction.
So this is a uniaxial stress.
And that is a stress field which we could apply on a piece of material very easily.
Just screw a hook into the material and hang a weight on the hook and you've got uniaxial stress.
We can also have something that's a little harder to visualize, a biaxial stress.
And this would be a stress that had elements sigma 11, register 0-- at least when we refer to principal axes-- 0 sigma 2 2 0 0 0 0.
When it's in diagonal form, two of the diagonal terms are 0.
That doesn't seem to make sense because, if this is a force per unit area along x1, and this is a force per unit area along x2, can't we add up those vectors and say that that's a uniaxial stress?
Depends on your coordinate system.
If you want an example of a biaxial stress, imagine some firemen underneath a burning building, and they're holding out a blanket for somebody to jump into.
You have one pair of firemen pulling in this direction to keep the blanket taut, and another pair of firemen are not quite so vigorous, and they're pulling in an orthogonal direction.
And those two forces, sigma 11 and sigma 22, create a biaxial stress event material.
You could change the form of the tensor by rotating the coordinate system, but this is one example of a biaxial stress.
The final general form, a triaxial stress, obviously by extension, is going to be where of all three diagonals, sigma 11, sigma 22, and sigma 33, are non-zero when you put the tensor in diagonal form.
This would be a general triaxial stress.
And an example of that would be when the jumping person lands on the blanket, there's going to be a force that his or her mass exerts on the blanket, and that would be a third sigma 3 3.
So that would be a triaxial stress.
Let me give you a couple of special cases of specialized stress tensors.
And suppose I had a triaxial stress, but all the diagonal terms were equal.
That's something that I tried a moment ago to hoodwink you into believing was required by a cubic crystal, but suppose that was the form of the stress tensor.
What would that represent?
[INAUDIBLE]
That is something that's going to be-- yeah, very good.
That's not a pressure.
It's a dilation.
But if I make it into a more familiar form, put it here, minus P, minus P, minus P. It's a force prevented area.
That's where the stress is.
If it's a hydrostatic pressure, it's squeezing the volume element so it has to be, by our definition, acting in a negative x1 direction on a surface whose normal is plus x1.
So minus P, minus P, minus P, represents a hydrostatic pressure.
And now, unless you've seen it before, or you're reading the notes instead of listening to me, let me give you a really strange-looking one.
What is this stress tensor?
Where sigma 22 is opposite in sign but equal in magnitude to sigma 11
[INAUDIBLE]
Yeah.
Good.
The way to see that geometrically is the following.
What we're doing is, we're pushing with a force per unit area in this direction.
That would be sigma 11, and then we squeeze in the orthogonal direction.
And that is a sigma 22, which is negative.
If we rotate the solid by 90 degrees, then we've got this, plus this, acting on this face, same forces per unit area as we have here.
And that is a net force per unit area like this.
On the opposite surface, we've got this force per unit area and this force per unit area, component going in this direction.
And that is going to be a force that goes like this.
That's going to be a force per unit area like this, and similarly, this combined with this is going to be a force acting in the reverse direction.
That would be, again, this with this, and this would be this with that.
So clearly, that is a body that's being subjected to pure sheer, and if we rotated the axes x1 and x2 by 45 degrees to an x1 prime and x2 prime like this, and then transformed the tensor, it would take the form 0 sigma sigma 0 0.
And then, what do we want now?
This would be the only thing we get.
Except I should be careful to put a sigma prime on this because, when we change axes, these sigmas will not be of the same magnitude as the original ones.
There's going to be a square root of 2 coming from cosine of 45 degrees in there.
But, anyway, just rotating the axes 45 degrees of as I've indicated schematically here with vector sums is going to give me two off diagonal terms equal in magnitude that would correspond to sigma 12 and sigma 21.
So those are some special forms of a stress tensor when you place it in a diagonalized form.
But let's conclude this part of our discussion by observing that everything that we've said about second-rank tensors holds for the stress tensor.
For example, you can always select new axes that put the tensor in diagonal form.
So that's one general property.
You can define a stress quadric, and we'll do that with the equation sigma ij xi xj equals 1.
Why not?
It's a second-ranked tensor.
We can take the elements and construct a surface from it.
This quadric will have the properties of any other quadric for a second-rank tensor.
In particular, if we construct the quadric, it's now for sure going to be able to be either an ellipsoid, an hyperboloid of one or two sheets, or in this case, even an imaginary ellipsoid for a stress that's everywhere compressive.
The quadric could have its diagonal elements all negative.
The radius of the stress quadric in a particular direction is going to be the 1 over the square root of the property in that direction.
What does that mean, stress in a given direction?
Well, let's again remember how we have defined the stress tensor.
A force per unit area has three components, k1, k2, k3, and they're equal to sigma ij times lj.
So the value of the property in this direction is going to be the value of the property in the direction defined by [?
elsa ?]
j and the values of [?
elsa ?]
j are components of the unit vector that point in that direction.
So if this is the direction of k, [?
off ?]
like this, the value of this quadric in this direction is going to be the value of the part of k that's parallel to the radius vector over the magnitude of the radius vector.
But the [?
radius ?]
vector is a unit vector, so it's simply going to be equal to k parallel.
And if k parallel is the part of the vector that is along the surface normal, that is going to be the component of k which is purely tensile.
This is a force per unit area, has parts that are, components that are normal to the surface normal.
Those are shear components.
The component that's directly along the normal to the surface is going to be a purely tensile component.
And, therefore, the radius of the quadric in a given direction gives us the tensile component of the force density vector k that is transmitted across the face-- so let's not say face, but interface-- whose normal is in the direction of R. So this is the radius vector.
That's the direction of the normal to a surface and 1 over the radius of-- The radius of the quadric in that direction is going to be 1 over the square root of the tensile component of stress that's transmitted across an interface in this direction.
How are we doing on time?
Any questions at this point?
Can you explain how you know that's [INAUDIBLE]?
Remember what our radius normal property said.
If we have a tensor like good conductivity, where a current density is related to an applied electric field, [?
essa ?]
j and we've-- unfortunately, that's segment 2.
This is the conductivity tensor.
What we said is that, if we take the components of the tensor, sigma ij, and construct the surface sigma ij xij equals unity, that will be some sort of quadratic form that has the property that the radius in any given direction is 1 over the square root of the value of the property, sigma conductivity in this case.
Bad case.
I shouldn't have picked things where the sigma is also involved.
And the value of the property in the given direction is going to be the parallel component of what happens over the magnitude of what you do.
So in the case of electrical conductivity, if we apply a field in this direction, the value of the property is going to be that part of the current flow, which is parallel to the electric field, divided by the magnitude of the electric field.
So that is a relation that you know by heart and have come to not only know, but to love.
So what are we saying here?
For the stress tensor, we can take the elements of the stress tensor and construct a quadric.
And the thing that's related here is a force per unit area that we're applying to the surface of the crystal, and that's transmitted through the volume of the crystal by the relation that we have defined as the stress tensor.
So, unfortunately, again, sigma Ij.
And now what is involved are the direction cosines of the normal to the surface.
So what is the applied vector?
The applied vector is a unit vector, and this is the normal to the surface.
And the surface in question would be a surface like this.
The thing that results is a force per unit area k. So what we're doing is taking k, the value of sigma, in that direction.
A scalar quantity will be the part of k that's parallel to the normal vector, divided by the magnitude of the unit vector.
And that's, a magnitude of n is 1, so that's just k parallel.
So this is the surface.
This is its normal.
And what the radius of the quadric is going to give us is that part of the force density, which is the force density vector resolved onto the normal vector.
And this is a tensile.
This is the part of the tensile stress that's transmitted.
This is the total stress that has a sheer part and a tensile part.
The radius of the quadric is going to give you the part that's transmitted across the surface in this orientation.
That is purely a tensile force.
That make sense?
I should have done it without sigmas being present in both tensors.
So everything you want to know about stress that's applied to a solid is contained in a second-rank tensor that is symmetric and obeys all of the characteristics that we derived earlier for property tensors, even though this is a field tensor, and not a property tensor.
Let us then turn to the next thing that I want to discuss, again something you've heard before.
But we'll do it within the context of the tensor algebra that we've developed before.
And this is the concept of strength.
And let me introduce the subject by a one-dimensional analog.
Suppose we have a very thin elastic band fastened to some immovable surface, and this will be the direction x.
And then we pull on it.
And here again, we have to be complete and note that there are several different kinds of behavior.
One type of behavior is a case where the elastic band stretches, and if there's some point P on the initial state that point P will move to a point P prime that is displaced from the original location by a vector U.
There's another point Q part way along the elastic band.
If the thing that we have fastened to the elastic band slipped, and there's no deformation at all, Q would be displaced by the same amount U if there was just translation of the elastic because it wasn't fastened down solidly.
But if it's stretching here between this point and this point, it's going to stretch up here, too.
And, in general, it will move over to some point Q prime, and this displacement vector is going to be the original U plus delta U.
And what we care about in defining strength is not the absolute coordinates, but the interval between these points.
This separation is U, and this separation is U plus delta U.
Now this is a common, but not a necessarily unique, sort of behavior.
This is something where the displacement does not change along the length of the elastic bands.
This is something that's called homogeneous deformation.
And this is characterized by a displacement view that varies linearly with position along the elastic band.
Does something like this.
It's possible, in a [?
grade ?]
of material, if you'd like a real example, that U, as a function of distance along this one-dimensional object, does something like this.
Why not?
That's what might happen for example in-- that's almost an oxymoron, a one-dimensional elastic band whose cross-sectional area changes with distance in that one dimension.
So this is homogeneous deformation, as opposed to inhomogeneous deformation.
There's another type of behavior, and this is apparent if you place a thin piece of metal or plastic, let's say, between the pair of grips in the Instron machine, and after deformation, it does something like this.
There's an inhomogeneous deformation, which is to referred to colloquially as necking.
A very good example of that is what happens when you grab hold of some cheap plastic shrink wrap that has been shrunk around a package that has been mailed to you.
And you want to get it off, so you grab it.
And it doesn't deform elastically.
What happens is that it necks in places where there's a stress concentration.
And you get something exactly like this.
And if we were to indicate that sort of behavior, U as a function of position would be one U, and then it would increase very rapidly.
The [?
UDX ?]
would be very much larger than the places which had uniform elastic deformation, so it would be a nonlinear behavior of this sort.
And that would be necking.
If you've never noticed that sort of behavior before, I give you an assignment tonight of going home and getting a piece of Saran wrap from your kitchen drawer and grabbing hold of it and pulling it.
And it deforms non-uniformly in a fashion that's indicative of necking.
OK, it's just about five of.
Why don't I stop there.
And what I would like to do next is to cast this relation, which is a one-dimensional stream, into a three-dimensional situation.
And we'll look at some of the properties of the description of a body that's been deformed in three dimensions.
Resume by going back to our one-dimensional body that has undergone some elastic deformation.
And what I would like to do now is to distinguish between displacement of an object and fractional change of length, which turned out to be measured by the same thing, that thing that we're going to name strain when properly defined.
OK, here is our one-dimensional case.
And we said that originally some point, P, at a location x gets mapped to a point P prime that is at x plus some displacement U.
So this is the displacement vector U.
Our point Q, which is originally at some location x plus delta x, where delta x is the original separation between P and Q, gets mapped to a point Q prime, which is going to be equal to a whole collection of terms.
It's going to be equal to x plus delta x, the original location, plus the linear variation of U with x that has to go U times x plus delta x.
And if we simplify this a little bit, Q prime is going to be at a location, factoring out x plus delta x, x plus delta x times 1 plus e
Is e represented here?
e is the linear relation between displacement U and position along the body.
OK, so what has happened here?
The relative change of length is going to be P prime Q prime minus PQ.
And if we just substitute in our two expressions, the relative change of length is going to be equal to delta x times 1 plus e minus delta x.
This is the relative change of length.
And I should label this such.
That's going to be equal to delta x times 1 plus e minus delta x divided by delta x.
And that simply going to be e, which is equal to delta U over delta x.
And that is what we define as strain.
And that is a dimensionless quantity.
Because it has units of length over length.
All right, how can we patch this one-dimensional sort of behavior up to a three-dimensional situation?
What we are saying in this one-dimensional case is that the displacement of a particular point P varies linearly with position in the body.
So let's generalize that into three dimensions by saying that the component of displacement U1-- and this is going to look like an algebraic identity.
It's going to be the way in which U changes with x1 times the position x1.
So this is saying that the displacement will vary linearly with x1.
And the rate will be du1 dx1.
But also U1 is going to change with the coordinate x2.
And the coefficient there will be du1 dx2.
And there's going to be a third term that will be the way in which U1, the x1 component of displacement, changes with x3 times the position in the body x3.
So this is saying exactly the same thing that we did in one dimension, that displacement depends linearly with position, except that there's now three coordinates for position.
And the displacement will change with an increment of a change in position along each of those three axes.
Similarly, we can say that U2 is going to be equal to the way in which U2 changes with x1 times the position x1, the way in which U2 changes with x2 times the coordinate within the body x2, and the same for U2 dx3 and the actual positional coordinate x3.
And so, in general, we're going to propose that the i-th component of displacement is given by the way in which displacement changes with x sub j times x sub j.
Or you could do the same thing, not in terms of actual displacement, but differences in displacement, the same way we could define strain as the fractional change of length of our line segment on the elastic band or, alternatively, as the shift in position.
So we could define it also as the difference in distances or displacements let me call them.
And that really is a trivial change.
We'll say the change of a length along x1 delta U1 is going to be the way in which U1 changes with x1 times delta x1 plus the way in which U2 changes with x1 times the way in which U1 changes with x2 times delta x2 plus the way in which U1 changes with x3 times delta x3 or, in general, that the fractional change in length is going to be equal to dui dxj times delta xj.
All right, so what we're going to define now is dui.
Dxj will be defined as something like strain.
It's not exactly strain yet.
And this is going to be a measure of deformation.
I'm hedging my words because of something that we'll want to impose upon the proper strain tensor.
But first, let's see what these three-dimensional terms mean.
And to see that, let me look at a place within the body.
And I'm going to look at a line segment that goes between a P and Q that is oriented along x1.
And I'm picking that deliberately.
Because I want to keep things simple and isolate one of these terms so we can identify its meaning.
So let's say here we have two points P and Q separated by a distance delta x1 prior to deformation.
And now we squish the body.
And what happens is that P will move to a position P prime.
Q will move to a position Q prime.
And this will be the new line segment, P prime Q prime.
This will be the original delta x1.
To that we're going to tack on an instrument which is delta U1.
But then there will also be a delta U2.
And clearly what has happened is that the length of the line segment P prime Q prime has changed.
But also the orientation of the line segment has changed by an angle, which I'll define as phi.
So again, we have a line segment.
We deform the body.
P is displaced.
Q is displaced.
The component of that line segment along x1 has changed in length by an amount delta U1.
The coordinate along x2 will change by an amount delta U2.
So we not only change the length.
But we rotate the line segment.
We can express these in terms of our general relation that I proposed a moment ago.
Delta U1 is going to be equal to du1 dx1 times delta x1.
And that is going to be given by the element epsilon 1, 1 times delta x1.
The value of x2 was going to be equal to du2 dx1 times delta x1.
And that's going to be by definition e2, 1 times delta x1.
And the reason this is so simple is that I initially picked the line segment which was parallel to x1.
So not all of the four terms that would be present in the x1, x2 system have come in.
So there's no contribution of delta x2 because of the fact that I've picked this special orientation.
OK, the fractional change of length resolved on x1 is going to be, well, it's going to be exactly delta x plus delta U1 quantity squared, it's going to be this horizontal line segment, plus delta U2 quantity squared all to the power 1/2.
And now I'm going to say that delta U2 is negligible compared to this big delta x plus delta U1.
And I'll say that this is approximately equal to delta x plus delta U1, OK, just taking the whole works outside of the square root sign.
So delta U1 over delta x1 is going to be equal to the term e1, 1 from this expression here.
So the term e1, 1 represents the tensile strain along x1.
So we can see how these derivatives are going to enter into changes of length.
The P prime Q prime has also been rotated by phi.
And we can say exactly what that is that the tangent of phi is going to be given exactly by delta U2 divided by the original length of the line segment delta x1 plus the little increment of displacement along x1.
And this is-- I'll walk down here so I can see it.
This is delta U1.
This is delta U2 over times delta x1-- sorry, I can't see what I've got down here-- delta U1 plus delta x1.
OK, so it's the amount of displacement along x2 over the original line segment delta x1 plus the change in displacement delta U1.
And clearly delta U1 can be claimed to be small with respect to delta x1.
So this is approximately equal to delta U2 over delta x1.
And tangent of phi is going to be tangent of a very small angle.
So this will be delta U2 over delta x1.
And so this angle phi is, for small strains, going to be equal to delta U2 over delta x1.
And that is the definition of our element of strain e1, 2.
So e1, 2 corresponds to a rotation of a line segment that was originally parallel to x1 in the direction of x2--
[INAUDIBLE]
e2, 1, I'm sorry, e2, 1, yeah, along x1 in the direction of x2.
And that is counterintuitive as I have just demonstrated.
e2, 1 is a rotation of a line segment along x1 in the direction of x2, which is just the reverse of the subscript.
So eij, a general off-diagonal element of the array eij, is going to be a rotation of a line segment initially along x sub j in the direction of x sub i.
And for very small strains, numerically that term eij will give an angle in radiants
For the equation you have over there, should it be delta x plus delta x2?
No, I would say this--
Because aren't you saying delta U1?
Oh, OK, you're right.
Yeah, I didn't take the difference here
Delta U1 is basically negligible?
Yep, yep, so I'm saying that that [INAUDIBLE] should be delta x1--
Plus delta U2?
--plus delta U2.
You're right.
No, I'm throwing this out.
And that's right.
So I'm saying that this is essentially delta x1 plus delta U1.
I'm saying that this is negligible.
So strictly speaking the distance between P prime and Q prime is the square root of this squared plus this squared.
But if this thing is tiny, P prime Q prime is essentially going to be this distance, delta x1 plus delta U1.
So that's right.
OK, so we have something that looks like it measures deformation.
But I would like to ask if this is a suitable measure of deformation.
And I would like to show that, unless the tensor eij is symmetric, that we have included in our definition of strain rigid body rotation as well as true deformation.
So in order to demonstrate that, what I'm going to look at is a case where we have actually, by the way in which we apply a stress to the material, actually done nothing more than rotate x1 to x1 prime and rotate x2 to x2 prime.
And in general, for a real amount of deformation, that is something that is going to happen.
Let's say I decide to deform this eraser in shear before your very eyes.
And I try shearing it.
Nothing much has happened.
And I squeeze a little harder.
And finally, I've got it wrestled down into an orientation like this.
And when I finished, this was the original position of the eraser.
Here's x1.
Here's x2.
And by the time I've wrestled it around to a deformed state, it sits down like this, maybe deformed a little bit.
But would you say that this angle here is a measure of deformation?
No, that's clearly a rigid body rotation.
And what I would intuitively do, if I wanted to measure true deformation, if this were the original body and after deformation it went to a location with some deformation to be sure but this has been rotated to here.
This has been rotated down.
And I wouldn't want to say that this is a measure of deformation.
This would be e1, 2.
What I would want to do intuitively would be to position the body symmetrically between the axes and then say that this is a measure of shear strain.
So let me show you now in a more rigorous treatment that, if there is a component of rigid body rotation, let me show you what a rotation of the coordinate system to a new orientation x1, x prime would do.
So let's say that this is an original point Q1.
And after deformation, it moves by a rotation phi that's equal to e2, 1 to a new location Q1 prime.
Let's suppose that this is a position Q2.
And after what we think is the deformation, this moves to a location Q2 prime.
And this would be e1, 2.
That's equal to minus phi.
This is e2, 1.
And that's equal to plus phi.
So what we would do is like to get rid of that rigid body rotation.
And what I'll do is to show that, if we have a point that's out at the end of a radial vector, R, which has coordinates xi, x1, x2, x3, and if we have a displacement, which is absolutely perpendicular to that radial vector, that this would be characteristic of rigid body rotation.
And I will say that, if Ui is perpendicular to the radial vector xi, then U.R should be equal to 0 for every position xi.
So if we just multiply this out, this is saying that Ui Ri should be equal to 0 for rigid body rotation.
And let's simply carry out this expansion.
And I will have then eij xi xj forming this dot product.
And that is going to be 0 for a rigid body rotation.
And if I expand this, this will contain terms like e1, 1 times x1 x1 plus e2, 2 times x2 x2 plus e3, 3 times x3 squared.
And then they'll be cross-terms e1, 3 plus e3, 1 times x1 x3 plus e1, 2 plus e2, 1 times x1 x2 plus e2, 3 plus e3, 2 times x2, x3.
And my claim now is that, if this represents rigid body rotation, then that should be 0.
That is to say the displacement is absolutely perpendicular to the radius vector for small displacements.
This must be 0 for all xi.
And the only way that's going to be possible for any value of x1, x2, x3 is that each of these six terms vanish individual.
It's the only way I'm going to be able to get the whole thing to disappear for any value of coordinate x1, x2, x3.
So we're going to have to then have e1, 1 equals e2, 2 equals e3, 3 equals 0.
All the diagonal terms are going to have to be 0.
In order to get the fourth term to vanish, I'm going to have to have e1, 3 equal to the negative of e3, 1 and e1, 2 the negative of e2, 1 and e2, 3 the negative of e3, 2.
So what is this going to look like for the part of the e tensor that corresponds to rigid body rotation?
The diagonal terms are all going to be 0.
And the off-diagonal terms are going to be the negative of one another.
So this is e1, 2. e2, 1 would have to be equal to minus e1, 2. e3, 1 would have to be equal to the negative of e1, 3.
And so we will have something like this.
So this suggests that our definition of the true deformation, which will be a tensor epsilon ij-- I bet you kind of guessed I was going to call it epsilon and not E. I can write this as a sum of two parts.
I'll turn that around and say I'll write epsilon ij plus another three by three array omega ij is equal to the tensor eij.
That's a novel idea.
This is addition of two tensors element by element.
And if I do that and define epsilon ij equal to 1/2 of eij plus eji, so for the diagonal elements that is going to take 1/2 of e1, 1 plus e1, 1.
And that's just going to give me e 1, 1 back again.
And I'm going to define the terms omega ij as 1/2 of eij minus eji.
And if I do that, the resulting tensor will be symmetric.
OK, so from tensor eij we can split it up into two parts and a part omega ij.
And I won't bother to write it out.
But you can see that omega ij plus eij is going to be equal to eij minus 1/2 of 2eij minus 1/2 of 0.
And that's just going to be eij.
So this plus this is indeed going to give me the tensor eij.
So given a tensor eij, which is not symmetric, I will define eij as the sum of two parts, a tensor epsilon ij, which is true strain, and a part omega ij, which is rigid body rotation.
And now at last we have a satisfactory measure of true deformation in terms of the displacement of points within a deformed body where that displacement can arise from either rigid body rotation and or deformation.
So finally, we have a tensor epsilon ij, which is the strain tensor.
And now I can finally make my claim that, if this really is true deformation, that for cubic crystals-- you can see the same [INAUDIBLE] window coming again-- since second ranked tensors have to be symmetric for cubic crystals, the form of strain for a cubic crystal can only be this.
It has to be diagonal if the tensor is to conform to cubic symmetry.
And you say, ah, ah, you tried to pull that on us with stress, too.
And I can squish a crystal anyway I want.
But you can only develop a strain if you deform a crystal.
And the symmetry of the crystal is going to determine how the crystal deforms.
So therefore, a cubic crystal should be able to deform only in a way that stays invariant to the transformations of a cubic symmetry, right?
So cubic crystals can only deform isotropically.
So if I wanted to design springs for an automobile so that elastic energy could be stored most efficiently in a solid, I would want to make these automobile springs out of a triclinic metal.
Because then the tensor could be one of general deformation.
OK, well, this, obviously, is a swindle like my assertion that stress could only be an isotropic compressional stress for a cubic crystal.
But the argument is a little more convoluted.
And strain, indeed, is also a field tensor.
Because, even though I can only create the deformation by application of a stress or perhaps an electric field or something of that sort in that the link between the deformation and what I do to the crystal is coupled by a property of the crystal, which has to conform to the symmetry of the crystal, nevertheless, given a tensor that describes deformation, I can always, independent of the symmetry of the crystal, devise a particular set of stresses that would produce any state of strain I want.
I would have to design, though, a particular state of stress to produce a desired state of strain.
And that coupling has to conform to the symmetry of the crystal.
But there's no reason why strain itself has to conform to the symmetry of the crystal.
I can create any state of strain I like by choosing the appropriate stress.
OK, so that is a swindle.
But everything now that I can say about the behavior of a second ranked tensor applies to the strain tensor.
I can take a strain tensor epsilon ij.
And I can perform the surface epsilon ij xi xj equals 1.
And that will be the strain quadric.
The strain quadric will have the property like any quadric constructed from a tensor that the value of the radius in a particular direction is going to be such that the strain in that direction is equal to 1 over the radius squared.
So what does that mean?
Well, we have to look at what the strain tensor is relating.
The direction, remember now that the definition of the strain tensor is that U sub i equals epsilon ij times x sub j.
So the quote "applied vector" is the direction in a solid.
And we will get, in general, a displacement, U, which is not parallel to the direction that we're considering.
And the epsilon in a particular direction is going to be equal to the part of U that's parallel to the direction of interest divided by distance in that direction.
And what this is going to give us is the tensile component of deformation in the direction that we're examining.
Now, the radius normal property works in this case.
And what the radius normal property says, if you look at a certain direction in the solid and then look at the normal to the surface at that particular point, this will be the direction of what happened.
So I've not drawn this correctly.
This should be the direction of U.
And this is the radio vector.
So the normal to the surface gives us the direction of the displacement that's going to occur.
And the reason I can say that is by definition that that property holds only for a symmetric tensor.
And by definition, we have defined the strained tensor such that it is symmetric.
So the radius normal property holds.
If we want to change from one coordinate system to another, we do that by the all for transformation of a second ranked tensor that, if we change coordinate system, the elements of strain change to new values epsilon ij prime that are given by cil cjm times epsilon lm, the same law for transformation of a second ranked tensor that we have seen earlier.
Finally, we can, because there is a quadric, we can change that by finding eigenvectors and eigenvalues, take a general form of the tensor epsilon 1, 1, epsilon 1, 2, epsilon 1, 3, and so on, and by solving the eigenvalue problem, convert this to a value epsilon 1, 1 prime 0, 0, 0, epsilon 2, 2 prime 0, 0, 0, epsilon 3, 3 prime to find out what the extreme values of tensile deformation are and the orientations within the original description of the body in which these extreme values occur.
OK, I'm just about out of time.
I'll save until next time another neat feature of the strain tensor.
And that is contained within these elements is information on the volume change that the body undergoes.
And this is something called the dilation.
And we'll see that this is related in a very simple way to the elements of the strain tensor.
OK, we've spent a lot of time developing the formalism of stress and strain.
And next thing we'll do is to move on to some interesting properties of crystals, which are defined in terms of tensors of rank higher than two.
So we'll look at some third ranked tensor properties.
That will include things like piezoelectricity, the stress optical effect, and things of that sort.
These are going to be really anisotropic.
Second ranked tensors were anisotropic enough.
But the variation of property with direction was a quasi-ellipsoidal variation When as 1 over the square of the radius of the quadric.
For higher ranked tensor properties, we will encounter some absolutely weird surfaces with dimples and lumps and lobes, not this smooth quasi-ellipsoidal variation, a property with direction, which was the case for second ranked tensor properties.
So we will get into some exotic anisotropies very shortly to finish up the semester.
So I will probably see some of you at the MRS meeting next week.
I won't see you on Thursday for reasons that I need not elaborate upon.
So I hope you have a happy holiday.
Relax, suck in air, and come back refreshed next Tuesday for some really wild anisotropy of physical properties.
I wanted to make a few comments on the third problem set, which was a funny puzzle.
And it was intended to be entertaining and get you thinking about things related to symmetry in an amusing context.
And there were two problems.
The first problem had a platter on which there were arranged cherries with either two or three berries to a spray, and apples and pears.
And they were either black or white, which didn't make then particularly appetizing.
But in any case, there were three platters and the fruits marched around on the platter.
And the puzzle was, what would be the fourth platter in the sequence?
This didn't involve symmetry in any sense of the word.
But yet, what was going on from picture to picture was a mapping of the motifs from one location to another.
Not just rotation or translation or anything nice in the crystallographic, but still they were being moved from one location to another.
And just about everybody got the right answer just by deduction.
But the reason I liked that problem, and the reason I gave it to you, is that it introduced another sort of transformation.
One of the pairs of fruits not only change locations, but they switched from black to white as you went from one position to the other.
And that is another sort of transformation we can introduce.
And this is something we could call color symmetry.
And I would observe that this is something that you are already very familiar with in patterns, in particular, in the pattern that is represented by a checkerboard where you have black squares and white squares.
This location here is a bona fide four-fold axis, as is this location here.
But there other locations, such as the corners of the black and white squares, where, indeed, there is a 90 degree rotation that takes this square and transforms it into this square.
But in doing that transformation, you change it from a black one to a white one, and then from a white one to a black one again, back to a white one.
And that is a color symmetry.
You could make it red and green if you like to be a little more attractive.
Similarly, there are mirror lines like this that are true mirror lines.
The pattern is left invariant.
There's another locus such as this one here where when you reflect you go from a black one to white one and then back to a black one when you do the operation twice.
So there are black-white mirror planes as well as regular mirror planes.
There are two-fold axes but all of the two-fold axes are black-white axes, white to black, black back to white.
So that's a new sort of transformation that we can develop if we wanted to, switching the color.
Does this is have any utility in the physical sciences?
Be nice for wallpaper designers, but does it have utility in physical sciences?
And the answer is yes.
Because what is going on here is that there is a binary character assigned to each of the motifs.
So any time the atom or whatever you have, the molecule, has some sort of binary characteristic that it can exist in two distinct states, the arrangement of that sort of motif requires a black-white symmetry.
And one example in crystals that requires this sort of symmetry is the case of a magnetic structure-- and I think I mentioned this last time-- where the magnetic moment can either point up or down.
Well, saying that you have an atom with spin up and an atom with spin down is the same thing as saying, in terms of transformations, having a black atom or a white atom.
It's a binary character that switches from atom to atom.
Could you have three color symmetries?
Yeah.
You can.
There is a Dutch artist named Maurits Escher who spent much of his, career deducing patterns, and did this in a purely intuitive way.
I mean you look at these patterns and say, wow, that guy is some crystallographer.
But I heard him, while he was still alive, speak twice.
And he describes it in metaphysical terms.
This is the twoness merging with the fourness, and just completely intuitive.
But his patterns are gorgeous.
And he has a number of very nice ones that are three-color symmetries.
And maybe later on in the term, I'll bring in a projector and show you examples of some of these.
They're very entertaining.
All right.
So the purpose of that first puzzle was to introduce you to black-white symmetries.
Then the second puzzle on the sheet was a pretty dumb thing.
There was a stack of blocks and you said how many-- I asked how many blocks are there in the stack?
And you could count them up, 1, 2, 3, 4.
And If you did it right, you came out with 32.
So that was pretty dumb.
Well, this was intended as an example of how one can use symmetry in the solution of a physical problem.
Let's suppose that you were a cowboy and you had to go out and buy horseshoes for a herd of horses.
Would you count the number of feet?
No.
You'd count the number of horses and multiply by four.
OK.
So there you'd be using symmetry.
Now The point of this problem really was-- You could have had this one.
We have just the exact number seats.
The point of this problem was that you have to, in assigning a symmetry to a system, usually make physical assumptions.
When you do a derivation, as we've been doing for the last several days, we say this is a four-fold access, and there's no ambiguity whatsoever.
Because it's my ball game, and I want it to be a four-fold axis, But when you're given a pattern or given a crystal and you have to assign a symmetry to that crystal, you invariably have to make assumptions.
The person buying the horseshoes, for example, has to assume that five- and six-legged horses are pretty rare articles and that a horse with only three or two legs is going to be of little utility to the rancher.
It Would probably have a limited existence at the ranchers expense.
So you're assuming that all horse have four legs.
In looking at a crystal, though, things are not that easy to decide.
And let me give you a few really practical examples.
A lot of mineral crystals grow in fissures in the rock.
And let's suppose we've got a crack, and bubbling through this crack is a solution that contains silica.
And as the solution cools off, it will deposit a crystal of quartz.
And this crystal of quartz would probably grow looking something like this.
And you would knock the crystal out of the rock.
And you would say, what is the symmetry of this crystal?
It doesn't appear to be anything hexagonal about it.
But what's happened is this crystal if it had developed in a uniform environment would have had a hexagonal shape.
But because the nutrients are flowing in from one direction, it tends to elongate in that direction.
So how could you assign the symmetry to that crystal that you dug out of the rock?
Unfortunately, crystals do not come with a legend on the bottom that says quartz SIO2 space group P3 subscript 121, made in USA.
You have to yourself assign the symmetry.
And what you would do is you would look at this face and this face and you say, gee, these angles, when I measure them, all turn out to be 120 degrees.
This face has a luster on it that looks like the luster on this face.
This face has little edge pits on it, and they point in the same way as the edge pits on this face.
I can measure the electrical conductivity and find that it varies in a fashion consistent with a hexagonal crystal.
And you would do any physical test that you decided you would need to make.
And then when you were all done, you would say everything that I can measure about this crystal other than it's shape is invariant to a 120 degree rotation.
Quartz has a hexagonal shape very often, but actually only this face, this face, and this face are symmetrically equivalent.
And there's only a three-fold axis in there.
And if you look very carefully at this crystal, you could see some difference there, too.
The neighboring faces very often have striations, very faint striations on one face, no striations on the neighboring face.
So you find out all that can.
You have to make some assumptions, and then on that basis decide the symmetry.
Let me give you another example.
This is a real one.
And one of the things that it's fun to do is to grow crystals from solution.
The Science Museum has little kits that you can buy that have some powder that you could heat up in water and then put it in a beaker and put a string in there, maybe with a little crumb of the stuff that you dissolved.
And one of the crystals that is nice for this purpose is alum.
It's a cubic crystal.
And if you let the stuff set there, it will form a nice, lovely equiaxed cubed.
Science Museum usually puts a pinch of another salt in there so the crystal is not only pretty and shiny, it's purple or it's the orange.
Now, unless you attach this little seed crystal carefully, there would be a good chance that it would fall off and land on the bottom of the beaker.
When that happened, your crystal would not have the shape of a cube.
It would have a shape in which these two top edges had length L and these faces on the side were exactly half that length.
And what's happening is that this crystal is growing in an unconstrained environment.
The nutrient is coming on and crystallizing from all directions.
In this case, nutrient can't get in from the directions that are below the crystal.
So the growth in that direction is eliminated, and this height is exactly half that at the edge of the crystal.
So that's sort of a very specialized case of a crystal growing in a constrained environment.
I'm digressing now, but let me make a few more remarks.
The shape that a crystal has-- and the shape is in crystallography designated by a special term.
It's called habit.
Crystals that undergo plastic deformation never die, they just develop bad habits.
Oh, come on.
This is a tough crowd.
I thought that was pretty funny.
Anyway.
The habit that a crystal has can depend very much on impurities in the solution or melt from which it grows.
And let me suppose I start with a little crystal that looks like this.
And there are two different kinds of faces on there.
And let's suppose one of them grows rapidly and one of them grows slowly.
Which face do you think would be the one that eventually forms the boundaries to the crystal volume?
The fast-growing face or the slow-growing face?
Fast-growing face
Fast, slow are both possibilities.
Let's do a time lapse experiment.
Let's suppose that this is the fast-growing face.
And, after a certain amount of time these faces would have all advanced to this location.
And this is the slow-growing face.
And after the same amount of time, it will have advanced to this location.
After the next time increment, this one would have grown up to here.
This would only have grown that far.
And you could see the only thing that's gonna be left is the slow-growing face.
So it's sort of counter intuitive.
The faces that survive to bound the crystal as rational planes are the slow-growing faces, not the rapidly growing faces.
This is of great use to the people who want to control crystal shape.
And this is very often in very every day, ordinary contexts.
The people who grow table salt, for example, want to grow a crystal that has nice pointy edges so when it comes down on your tomato, it doesn't bounce off.
It sticks to it.
So the shape of table salt makes a difference in how it behaves.
And the way you change the shape is to find something that will be absorbed preferentially on one set of faces that you would like to have be the faces that appear on the crystal.
And that will do the job for you.
I'll close with just one final example of this in real, every day industry or technology.
I once had a phone call from a company that made salt for sprinkling on roads in the winter time, particularly around New England when the roads tend to ice up.
And in an earlier era it, was politically correct to use sodium chloride.
But you wanted the sodium chloride, if you could, to have an acicular, a needle-like, shape to improve its flow properties so you didn't waste a lot when you shook it out of the back of the truck.
And the material that they use for that-- you think rock salt in the environment is bad-- they used a cyanide compound to modify the growth habit and make it needle-like.
And they wanted to get away from that before they were really dragged into some sort of trial by the EPA.
And did I know of something that could change the shape of the salt crystals for them?
And the answer to that question was quite straightforward.
No.
That was the end of the conversation.
But it was an interesting example of how surface chemistry can change crystal shape.
OK.
So all that long-winded explanation was a justification of the meaning of those two final two puzzles.
And the final thing I have to say about them is that these puzzles were lifted from a puzzle book published by MENSA.
Have any of you heard of MENSA?
MENSA is an association of self-declared geniuses.
So this was a puzzle book for geniuses.
And you all did horribly well on that third problem set.
Everybody at MIT is bright.
But you folks are not only bright, you are geniuses because you cracked the MENSA puzzles.
OK.
Enough enjoyment and fun and games.
Let's get serious, back down to what we were doing earlier.
We had established in earlier discussion that there are a very limited number of ways in which we can arrange a mirror plane and a rotation axis about one fixed point in space.
And these, accordingly, are called the point groups.
And in particular, they are the two-dimensional crystallographic point groups.
We had rotation axes by themselves, 1, 2, 3, 4, and 6.
And these are the only ones we need worry about because these are the only rotational symmetries that are compatible with translation.
Then we had a mirror plane.
And then there was no reason why we could not combine a mirror plane with these rotational symmetries.
And in doing so, we used a theorem that said if you take two reflection operations, sigma 1 and sigma 2-- and again, I use sigma to represent an individual operation of reflection-- and a mirror plane is the locus of operations of reflection, and then reflecting back again, there are two operations involved.
And if these are combined at some angle mu, sigma 1 followed by sigma 2, when they are separated by an angle mu is the same as the net transformation of a rotation about their point of intersection through twice the angle which the mirror planes are combined.
And that led us-- in addition to a one-fold axis, a two-fold access, a three-fold axis, a four-fold axis, and a six-fold axis-- let us combine reflection with these axes to get symmetry combinations that we named according to the symmetry elements which were present in the final combination.
We used two mirror planes if the mirror planes that arose where independent.
Independent in the sense that the two-fold axis never turned one into another.
And independent in the sense that if we draw a representative motif, the way those motifs were disposed about one mirror plane was different from the way they were arranged about the other one.
For a four-fold axis, we could add mirror planes.
And they would have to be at half the angle of the four-fold axis.
And as we saw, it was kind of a mouthful to call this 4MMMMMMMM.
There were just two kinds of mirror planes, so this one was called 4MM.
6MM, analogously, was a six-fold axis with mirror planes separated by 30 degrees, half the throw of the six-fold axis.
And the one that I left out is 3, not m m but 3M because a three-fold axis has mirror planes separated by 60 degrees.
And the same thing goes on at each of these mirror planes, pair of objects hanging at one end of the mirror plane, the other end is unadorned.
So they all behave the same.
There's only one kind of mirror plane.
The three-fold axis maps one mirror plane into another, and so they have to be the same.
So therefore, only one M appears in the symbol.
Independent of this, we had, when we combined reflection or rotation with a lattice, found that there were a very limited number of lattices in two dimensions, the oblique lattice which had T1 not equal to T2 and T1 inclined to T2 by some general angle.
The next degree of specialization was for a two-fold axis-- so either a one-fold or a two-fold axis could fit in here.
For a three-fold axis or a six-fold axis, we had to have a lattice in which the two translations were identical to one another, identical in magnitude and identical in the way the atoms were ranged relative to these translations.
And the angle between T1 over T2 had to be identically 120 degrees.
And by convention, we take the larger of two angles, 120 rather than 60, to define the inter axial angle.
A four-fold axis required a net that was exactly square, not approximately square but exactly square.
So T1 and T2 had to be identical in magnitude.
The angle between them had to be exactly 90 degrees.
And close is no cigar.
It has to be exactly 90 degrees because that angle is generated by symmetry.
And then at the place in which I'm gonna pick up today in just a moment is the case of a mirror plane and reflection, we saw depending on how we arranged the translations relative to the reflection plane, could give us either a lattice in the shape of a rectangle-- and here T1 and T2 made an angle of exactly 90 degrees.
But the magnitude of T1 could be anything it liked relative to the magnitude of T2, and then a double cell, which had the shape of a diamond.
But it was operationally much to our advantage to pick a double cell which had a right angle in it.
And the reason for that was that it is a nightmare to do calculations of inter-atomic distances and angles or angles between faces in an oblique system.
The right angle makes these calculations very simple.
And the fact that this cell tells you twice as much of the area as you really need to know about is a small price to pay.
So this is the rectangular net, as we'll call it in words.
And this is one that we'll call the centered rectangular net.
OK.
So let me pause here and suck in air and see if there are any questions.
This is where we left off.
And now having everything spread out on the board, we're going to start to make combinations and continue that process.
Yes, sir
So the centered rectangular has the same exact [?
intercept ?]
as the rectangular but the only thing you're doing is you're deriving it based on symmetry that you're using like a diamond kind of thing to get the center one?
Yeah.
So all these arose when we did it slowly and systematically.
We said let's let this be the location of the mirror plane.
What happens if we combine it with a translation?
We can take one lattice point on the mirror plane since there's no unique lattice point.
And then the mirror plane is gonna reflect this over to here.
So here's a first translation.
I've got a second non-colinear translation.
Wham-o.
Instant lattice.
So what I'm doing is taking this diamond shape and I am taking one translation here-- let's put some labels on this.
Let's call this T1 and this T2.
So this translation here, call it T1 prime, is my original translation T1 plus T2.
In this translation, T2 is the negative of the-- Yeah.
It's gonna be minus T1 plus T2.
So I've taken linear combinations of the two vectors of the diamond-shaped cell.
And again, this is contrary to the rules we have that say pick the shortest two translations.
Why?
The final larger area of volume then you need.
And the answer to that is occasionally the provision of a coordinate system, which is far easier to work in, is a small price to pay for that redundancy.
OK. And then the primitive rectangular net, just to remind you again, we said is there any case in which this is not true?
And that case was if you take T1 exactly perpendicular to the mirror plane and then you define only a lattice row.
So that gives you a second choice for T2.
You can't pick it anywhere you like, otherwise it gets reflected across just as in the first case, and you have two translations that are not compatible.
And the only way that they are compatible is that if you make the translation T1 straddle the mirror plane.
And that's what we've already got up here.
So that's how we got the rectangular net.
We could get a second distinct sort of lattice only if this translation was exactly in the plane of the mirror line.
Yes, sir
But the number and kind of symmetry elements are the same for both, right?
Exactly
Uh, so--
So there's a curious sort of duality here.
Here in the case of a hexagonal net, this is just a lattice of a very specialized shape.
And does that have a six-fold axis in it?
No, not unless I decide to put one in.
I could put in a three-fold axis, alternatively.
So here, I have one lattice that can accept two different kinds of symmetry.
Here I've got the reverse situation.
I have two distinct kind of lattices, both of which are happy and content with the same symmetry on them.
So there's going to be a far greater number of combinations of lattice with symmetry than simply a one-to-one correspondence between lattice types and point group types.
OK. Any other questions?
All right so let me now shift down to low gear and remind you of one thing that we did last time.
We asked what happens if I can combine a rotation operation, a alpha, with a translation?
Call this T1.
Sounds like something I already did in showing that the values of alpha are restricted to values.
But what I'm going to do now is use something that looks as though it's a similar starting point to arrive at a different result that we saw last time.
I'm going to deliberately look at a line that is alpha over 2 on one side of the perpendicular to T1.
And then the operation of the a alpha is going to take this translation with a lattice point of necessity at the end of it, and it's going to move it over to here, alpha over 2, on the other side of the translation of the perpendicular.
Then I'm going to pick this up with the translation T1 and move it and everything along it to a location like this.
And my question now is, what is a alpha followed by the translation T1?
The handedness of the objects have not changed.
So this has to be either translation or another rotation.
And what we can very quickly show is if upon dropping a perpendicular down to the original translation, these lines are parallel.
This line cuts across it.
So this angle is alpha over 2.
And this line is inclined to the perpendicular by alpha over 2.
So this angle in here is also alpha over 2.
So the answer is that if I rotate from one location to another by rotation alpha and then pick up that motif or the entire space and translate it by the original translation T1 to get a third one-- this is number 1, right-handed, say, number 2, right-handed.
This is number 3.
Stays right-handed.
Has to be related by a rotation.
And we next ask, what point is the point about which the rotation occurs?
Then we used-- what at the time seemed like a trivial observation-- that a rotation axis in space or a rotation point in two dimensions is the locus that is left unmoved by the rotation.
So if rotating from here and translating over to here is equivalent to a rotation and if the locus of the rotation has to be the point that's left on moved, bingo.
That's where the rotation occurs.
And it's through the same angle, so we can label this an operation b alpha.
And we know exactly where that point is gonna be.
It's gonna be along the perpendicular bisector of the original translation.
And it's gonna be up a distance x which is T over 2 times the cotangent of alpha over 2.
So now we've got another, what I call, combination theorems.
And again, this is nothing more than knowing how to complete the product of two operations in establishing the group multiplication table.
so let me write it down in the form of a combination theorem.
This says that a alpha followed by a translation is another rotation-- in the same sense, both counter clockwise in this example-- another rotation b alpha in the same sense about a point that is always T over 2 times the cotangent of alpha over 2 along the perpendicular bisector.
So that's how you would go about making an entry in the group multiplication table.
Sorry.
I put you off
Yeah.
Can you just, like, I just want to see what you did actually to the motif in that drawing
OK. Rather than taking an abstraction, an arbitrary line that is at alpha over 2, I put a motif in the space that doesn't necessarily hang on this particular locus that I drew for reference.
And the rotation through alpha would move it to here.
It's lagging behind this first line.
It lagged behind the second one by the same amount.
Then I pick it up and I move it by the same translation and that slides it over here, still canted over to the left.
And now what I claim is that I get from the first one to the third one-- and actually should have raised your hand on a different matter.
This is out in front of the translation.
So number 3 should sit here, also to the right to the translation.
And the way I get from 1 to 3 in one shot is by rotating alpha b. OK. Now, a cautionary note just so you do not use your new power recklessly.
Let me emphasize that this is an escalation in operations, not in symmetry elements.
And the reason for that is that a rotation axis, in general, contains a number of different rotations.
An n-fold axis has n different rotation operations implied in it, including the identity operation of rotating 360 degrees.
So if I say I'm gonna drop a four-fold access in a lattice, what I'm doing is dropping into the lattice a 90 degree rotation, a 180 degree rotation, and a 270 degree rotation.
And the location of the new operations depends on alpha.
So these new operations are not gonna pop up at the same location.
They're gonna be sprinkled over different locations.
So as an abstraction, that may be hard to appreciate.
But we're going to straight away derive a couple of more additions.
And then when we see what happens upon adding a rotation axis to a lattice in a couple of nontrivial cases, I'll just summarize the results and assume that you'll be able to derive them on your own.
The last time we had got around to doing just one of these combinations.
And we ask what happens when you combine with a translation a rotation operation a pi.
So we'll take some motif that sets up here, right-handed, rotate it by 180 degrees to get a second one, and then pick that up and translate it by T. So I'll get a third one sitting down like this of the same handedness.
And the question now is how do I get from 1 to number 3 directly?
Well, let's use our theorem.
Our theorem says we should go a distance x along the perpendicular bisector, which is half the magnitude of T times the cotangent of pi over 2.
The cotangent of pi over 2 is zero.
So we go up a distance x.
That's equal to 0, all in the perpendicular bisector, which means we stay put.
And-- in this class, there is always truth in advertising.
Around this locus here is a rotation operation b pi, which takes the first one into the third one.
OK. And now, lickety split, I will derive again-- and I apologize for going fast because I did do it last time.
Let's ask what happens when we combine a two-fold axis with a parallelogram net?
So we're taking a two-fold axis plus a parallelogram net.
In this two dimensional lattice there are two translations, T1 and T2, of arbitrary magnitude with some angle between them that's arbitrary.
And to this lattice point, I'll add a two-fold axis.
A two-fold axis involves the presence of two operations, the identity operation, and that we can forget about, and then there's the operation a pi.
By combine a pi with T1, I'll get an operation b pi that sits at the midpoint of T1.
If I combine a pi with T2, we get an operation c pi at the midpoint of T2.
If I combine a pi with T1 plus T2, I'll get another operation d pi at the center of the cell.
And we don't have to ask what happens when we go three translations over.
There's gonna be another operation of 180 degree rotation on that translation.
But we really only need bother about what goes on on the edges and in the interior of one unit cell, because whatever's there has to be repeated by translation.
So, in making these additions, we need consider only independent translations-- that is, not related to one another by the rotational symmetry-- independent translations that terminate within the unit cell.
And that means for the addition of a two-fold axis, I need do only what I have, in fact, just done.
I want to consider a pi with T1, T2, and T1 plus T2.
If I have the operation a pi or b pi or c pi or d pi at these four locations, that is all I need have present to say that I have, in addition to the two-fold axis that I added to the lattice point, a two-fold axis halfway along T1, a two-fold access halfway along T2, and another one smack in the middle of the cell.
These are lattice points.
So those axes have to be repeated by translations, this one from here to here, this one from here, this one from here to here.
So I'll have two-fold axes in the middle of all of the edges of the cell plus this fourth one right in the middle.
So this is an example of a two dimensional space group.
It's a symmetry that acts on all of space.
And the names that we will give to these combinations is a symbol for the symmetry element that we added, in this case a two-fold axis.
And then we'll specify the nature of the lattice to which we've added it.
And all that I have to say is that the lattice is primitive because you're all aware at this point that a two-fold axis requires only a primitive, a parallelogram net.
The only place we'll need a special symbol is when we add a mirror plane to either the rectangular net or the centered rectangular net.
And then we'll have to specify which type of lattice of that shape, primitive or centered, we're dealing with.
Yes, sir
So all you did with that derivation there is prove that you don't have to worry about any rotations outside of that?
That's correct
That's correct?
Yep.
Which is very fortunate 'cause there's an infinite number lurking outside the boundaries of the cell.
I'll show you another general truth.
What does a pattern look like?
The pattern looks exactly like the pattern of a two-fold axis with that pattern of motifs hung at every lattice point.
These new two-fold axes don't do any further repetition of the motifs.
They don't do any repetition of the motif.
They just express relation between things that you get when you take this pair and then repeat it by the translations.
So this two-fold access, for example, relates this to this.
This two-fold axis relates this one to this one, and this one to this one, and so on.
They're just relations that exist between the things that you get when you add the pattern of a two-fold axis to a lattice point.
So another generalization of what we're doing here that turns out to be valid, the pattern of motifs for a particular plane group is nothing more than the pattern of the symmetry element-- symmetry element or symmetry elements, sometimes we'll be adding more than one-- that we've added to a lattice point.
So in other words, if we take 2MM and drop that into a rectangular lattice, the pattern is simply gonna be these four atoms related by 2MM hung at every corner of the lattice.
If you add it to a centered lattice, these would be hung at every corner of the lattice, and also at the centered lattice point.
So it's that simple.
If you look at some of these high symmetry space groups or plane groups, wow, there's symmetry all over the place.
You say, how can I draw a pattern for that?
Easy.
Just do what the point group does and drop it in at the lattice points.
So that turns out to be a universal feature of all space groups and plane groups.
OK. Somebody had a question over here and I cut you off.
Yeah
Yeah.
On that two [INAUDIBLE] diagram, wouldn't you want to use four of those because you said it was-- you don't concern ones that are-- you're only concerned independent ones, the ones you can't get by--
You're absolutely right.
And let's look at this pattern again.
Here are a pair that are hanging close to this two-fold axis.
Here's a pair hanging close to the two-fold axis, very different arrangement, different arrangement relative to this two-fold axis, different arrangements relative to this two-fold axis.
So what we're saying is-- and this is the way, in fact, we derive them-- that these four two-fold axes are all distinct.
But the ones that are along the edges are just related by translation.
So why do I have to put them in?
You just put them in to just show the symmetry?
Yeah.
Exactly.
Exactly.
'Cause if I say this is a lattice and I have two-fold axes here, that's an incomplete representation because if these are translations, I jolly well better have the same thing at all the lattice points.
I'm just emphasizing the property of a lattice which is understood to be present in all of these combinations.
OK. Other questions?
And I would remind you that a reasonable question or request would be, could you do that all again, please, at half speed?
And I'd be happy to comply.
But really this is ground that we covered last time.
And I'm just reinforcing it.
So now that we've run out of time, we can go on to something new.
Let me do, in the couple minutes remaining, let me make one more addition just to show you how things change when you add a rotation axis that has more than one rotation operation.
Let me ask what happens when you add a four-fold axis to a lattice.
We've seen that a four-fold axis can coexist with no lattice other than 1, which is dimensionally square and which has the same thing hanging about, two translations that are exactly 90 degrees apart.
So what we're doing is taking a four-fold axis and drop it in at the corner lattice point and then step back before all hell breaks looks.
Now, first thing we're gonna do is to get the four-fold axis at every corner lattice point.
And then we're gonna use our theorem, taking into account that when I say I'm adding a four-fold axis, I am adding the operations a pi over 2, a pi, a3 pi over 2.
But let me call that, instead, a minus pi over 2, the same thing and a little easier to deal with.
And then the operation a2 pi, which is the same as the identity operation.
And that's dull and uninteresting, so we won't consider that addition.
We've already done this, a pi, in deriving P2.
And that's the nice thing about this derivation.
It's gonna snowball because as the symmetry gets higher, we've already done the work for the subgroup that contains the elements that are present in the higher symmetry.
So we'll get an operation a pi over a pi here.
We'll get an operation a pi here.
And if we take the diagonal translation, T1 plus T2, I'll get the operation a pi here.
Or I should probably call them a pi, b pi, and c pi to indicate that these are different two-fold axes.
OK. Let me emphasize again that this theorem that we have is a theorem in individual operations and not symmetry elements.
If I say a four-fold axis is present, it means that all of these operations exist about that locus.
I cannot say that a four-fold axis sits here.
I cannot say a four-fold axis sits here.
I'm gonna have to take each of these combinations in turn and then step back and see what operations exist about the various loci within the cell.
So the one that's really different is the combination of a translation T with the operation a pi over 2.
And my theorem says if I put in a motif and rotate it by 90 degrees to here-- so this is number 1 and this is number 2-- and then translate it to a location that's related to the first by symmetry-- by translation to get number 3-- they will be related by an operation b pi over 2 that sits up along the perpendicular bisector by an amount x, which is T over 2 times the cotangent of alpha over 2, T over 2 times the cotangent of 1/2 of pi over 2, 45 degrees.
The cotangent of 45 degrees is unity because the two edges of the triangle are equal in length.
So this says I should go up along the perpendicular bisector by an amount that is half the length of the cell.
And that puts me right in the middle of the cell.
So the operation b pi over 2 sits not at the edge of the cell, but at the center of the cell.
And that, I think you will agree, even in this hastily sketched diagram, is the way number 1 is related directly to number 3 in one shot.
Now I could do it on the cheap from here.
But let's show that we have first an operation a pi over 2.
We've already got the operation c pi.
Let's now combine the operation a minus pi over 2.
And that says that for c minus pi over 2, I would go up the perpendicular bisector a distance T over 2 times the cotangent of minus 1/2 of pi over 2.
And the cotangent of a negative angle is minus that.
So I go a distance minus T over 2 along the perpendicular bisector.
And what that is gonna do is to bring me to the center of the cell that's directly below.
And this will be the rotation c minus pi over 2.
But everything has to be translation equivalent at the interval T. So I can just move this operation c minus pi over 2 up to the center of my original cell.
So now I have at the center of the cell all the operations of a four-fold axis.
So a four-fold axis exists in the middle of the cell.
And let me write the results in here.
I've now got all of the operations of a four-fold axis sitting here.
I've got all the operations of a two-fold axis sitting here and here.
So this is the final result, if I translate the two-fold axes over to the other edges of the cell
What is that angle [?
by the d project?
?]
That should be 90 degrees?
That should be 90 degrees, if I drew it carefully.
Yeah.
So let me give the final result with a pattern in it.
These are the two translations, equal in length, four-fold here, four-fold here, four-fold here.
They are all the same four-fold axis.
Four-fold axis in the middle of the cell.
Two-fold axis in the middle of the cell edges.
And the pattern of objects is the set of things that are rotated by 90 degrees that form a square about the corner of the cell.
Another one here.
Another one that sits up here.
And another one that sits like this.
Four of them on the corners of the square.
And that's all we're going to get in this pattern.
Again, it's just a pattern of a four-fold axis that is hung at every lattice point of the square net.
And these other symmetry elements that arise simply are things that relate to the squares that are hanging at every corner of the square cell.
OK.
So the square in the center of the cell, for example, relates these four.
And you can pick any other four and they'd be related as well.
The two-fold axis relates this to this.
This two-fold axis relates this to this and this to this.
So the pattern is just the square produced by four stamped out at every corner of the square net.
And it's that simple.
The name of this thing-- what have we done?
We've taken a four-fold axis, we've put it in a primitive net.
I don't have to tell you it's squared because in your heart of hearts know that that is what a four-fold axis requires.
So this describes completely the combinations that's been made.
And this is a representative pattern.
A pattern of this sort is one that we've seen several times already in describing sample symmetries.
That is the plane group of square floor tiles.
It's the plane group of the square mesh that's in the overhead lighting fixtures.
It's the pattern that is present in the tiles up above the lighting fixtures.
It's a pattern that is very commonly present in square shirts, but not too often.
It's kind of dull and uninteresting.
Yes.
Fellow back there against the wall wearing P4, probably didn't know until now.
Thank you.
Actually, he's a shield.
I made him come in.
I called him up last night and said be sure you wear the P4 shirt.
And he said, OK, I will.
All right.
I'm getting silly.
So that's tells me it's time to quit.
So let's take our usual 10 minute break, And then we'll resume.
And I think I'll go lickety split through the remaining plane groups that consists of combinations of a rotation with a lattice.
OK.
So do come back.
Any questions before I obliterate all this lovely geometry?
No.
OK. We handled them during intermission I guess.
Let me do a couple more plane groups just very quickly to show you how they come out without going through all of the steps, because I think we've seen now what one has to do to derive these.
There are two that are left.
One is a threefold axis, and if that's all the symmetry we've put into the lattice, we're combining a threefold rotation axis with a primitive lattice, and we know that this has to be hexagonal net.
Because from our depth of experience, we know that is the shape of a lattice that is demanded by a threefold axis.
It has two translations, T1, that are identical in length.
And this angle between them is exactly 120 degrees.
That's what we saw a threefold access require.
So now what we are doing is putting in a threefold axis at one lattice point, and this means we are adding the operations A 2 pi over 3.
A minus 2 pi over 3.
I'll choose to define a 120 degree rotation that way, and the operation A 2 pi, which is the identity operations.
So I'm adding three operations.
Putting the threefold axis in at all these locations.
They will all be translationally equivalent.
The pattern of this particular plane group is going to be the pattern of a threefold axis.
And so there will be one motif here, 120 degrees away.
There will be a-- sorry to wipe out those operations-- 120 degrees.
There'll be another object located here.
And 120 degrees again, there will be another object located here.
So these guys will fall at the corner of an equilateral triangle.
And the same will be at the other corners of the net, and that, as I've said before, is the pattern of that plane group.
And that's a plane group that we would call P3.
Primitive lattice that has to be hexagonal, and what we've added to it is a threefold axis.
OK, let me down here just indicate the combinations that we would do.
And I could do it up on top, but I can work with this translation as well, which is easier to reach at this late hour in the afternoon.
Let me combine A 2 pi over 3 with this translation here.
And that says we have to get an operation B 2 pi over 3.
That is located at the original translation T times the cotangent of 1/2 of 2 pi over 3.
And that's at 1/2 of the cotangent of 60 degrees.
And this turns out not to be any nice neat number like 0 or 1/2, but if you evaluate what the distance up along the perpendicular bisector is by this amount, where you come out is right in the center of this triangle.
Trust me.
A little bit of trigonometry will let you convince yourself of that.
So if I rotate 120 degrees.
Bring this one to this one, and then translate over to here, the way the first one and the second one are related is by a 120 degree rotation about the center of this triangle.
If I combine the operation A minus 2 pi over 3, with this translation, I go down a distance x of minus this amount and that puts me in the middle of the triangle that is directly below.
And I can move that back up.
And now I have all the operations of a threefold axis about this location.
I can do the same thing with the threefold axis at another lattice point.
Another thing I could do is to just rotate this by 120 degrees.
And by either route, you find there must be a threefold axis here.
Notice that these threefold axes will once again, as advertised, simply take things that are at the different corners of the cell.
For example, this threefold axis will tell you how this motif is related to this one is related to this one.
And so it goes through the other threefold axes as well.
This threefold axis will tell you this one is related.
No, I don't want to draw it in.
So this is P3, a pair of threefold axes in the centers of the triangle, and another one at the corner of the cell.
And now I am going to do the remaining combination of a rotation axis with a lattice, so quickly, and it's going to take your breath away.
And that is seemingly the most difficult and complex one of all.
This would be P6.
Sixfold axis plus a primitive lattice.
And we know it also has to be hexagonal.
And that will be called P6.
What we're adding to the lattice is a sixfold axis.
And now what I'm going to do as a shortcut is to say a sixfold axis also contains all the operations of a threefold axis.
So I can take P3 and drop it right on top of P6, and that's going to give me threefold axes here.
Sixfold axis not only contains 2 pi over 6 and 2 pi over 3, it also contains the operation 2 pi over 2.
This is the operation of a twofold axis, and that says I have to, in addition to the two full rotation that sits here get twofold rotations in the middle of every one of the translations T2.
So actually, this is going to be P2 superimposed on P3.
And that's going to give me these twofold axis and this threefold axis.
And the only thing I have to do is to show you what A 2 pi over 6 combined with one of these translations is going to do.
What is A 2 pi over 6 combined with this translation?
And it's going to be new operation B 2 pi over 6, and it's going to be located at 1/2 of T times the cotangent of 1/2 of 60 degrees, cotangent of 30 degrees.
And the cotangent of 30 degrees is 2.
So this is going to be at 1/2-- no, what do I want to say?
This is 30 degrees.
This is one.
This is two.
Cotangent of 30 degrees is this over this, and that is-- no that's one
It's 2 pi [?
squared 3.
?]
Yeah.
OK, that's right.
So actually, what that does is to say the sixfold axis sits right up here.
So I don't get any new sixfold axis rotation of 60 degrees here, followed by translation is the same as 60 degrees about here.
So this is P6, and there is a lot of pure rotation axes combined with lattices.
We've got P1, P2, P3, P4, and P6.
Now, what I will do eventually, when we're all done here, the plane groups are not derived in any book or set of tables that I am aware of.
And next time, you will get some notes that do this in very slow motion fashion and give you all the individual steps.
But the international tables does give you diagrams of the resulting plane groups, very nice carefully done figures along with the representative arrangement of motifs that they generate.
But we're not done yet.
We have not let mirror planes enter the picture.
And so unless there is dissension or debate, I'd like to consider what happens when we take a mirror plane, and we can combine that with two different kinds of lattices, a primitive rectangular net or a centered rectangular net, which is called C. And in order to do that, we need yet another combination theorem.
Here are the lattice points, and let me first derive the plane group that is called PM.
And what I'll do is to put the-- let me use a squiggly line here, not because I'm excited or nervous about this, but just to distinguish it from the edges of the cell.
So here is the operation sigma.
The pattern that is going to be displayed by the plane group is once again just the pattern produced by a mirror plane hung at every lattice point.
But now we need a theorem.
What we have here is the operation of reflection, followed by a translation that is perpendicular to the locus of the reflection line.
And we will ask what is that?
Again, you get the answer by just looking at once and for all, and say, if here is a first one and it is right-handed, and I reflect it to one number to a second one, number two, which is left-handed, and then move that by translation here to get number three, which stays left-handed if I move by translation.
And ask now how was one related to three.
The chirality is changed.
Reflection is the only thing available to us, and lo and behold, if I say there is a new mirror plane here, that tells me how this is related to this, and this one is related to this one.
And this to this and this to this, so the answer to this question is that a reflection combined with a perpendicular translation is a new reflection operation sigma prime that is located at a distance removed from the first by 1/2 of that perpendicular translation.
And again, it's a plane old mirror plane just like the first one, but notice that the disposition of objects relative to the mirror plane in the center of the cell is quite distinct from the disposition of objects relative to the first mirror plane, so this is a second mirror plane.
It is an independent mirror plane from the first.
So that is plane group PM.
If there's symmetry in this business, you might ask is there a plane group AM?
The answer is yes in three dimensions.
There is a space group, PM, and there's a space group AM.
So there's AM and PM, and there is symmetry, and all is well in the universe.
OK, we're making great progress here.
And we'll be fairly well along before we have to bring things to a close.
Let's do the second addition that's possible with a mirror plane.
And that is to take a reflection operation and add it to the translations that are present in a centered rectangular net.
We've already done all the work for PM, so we can use that as a starting point.
The pattern of this plane group is going to look like a pair of objects related by reflection.
But now, we'll have an extra pair hung at the centered lattice point as well.
And the first thing we can note is that this mirror plane is no longer independent of the first one.
What goes on at this mirror plane is something that's related to what happens at the origin lattice point and mirror plane, and therefore, these two mirror planes are going to do the same thing.
The pattern of this plane group, we've taken a mirror plane and added it to a centered rectangular net.
This is called a C lattice, standing for centered.
And correspondingly, the symbol for this plane group is CM.
We know how this one is related to this one.
This one related to this one.
And now, we've got something of a problem.
All of these motifs are equivalent.
How is this one related to this one?
Or in more general terms, what we're asking is suppose I have a reflection operation sigma, and I add the translation not at right angles to it, as I did here, but place the translation at an angle with respect to the reflection plane.
So what we're going to do is to take a first motif that's right-handed, reflect it to a second one, which is left-handed.
And then slide it along so that it sits up in the same position relative to this centered lattice point.
So here's number three, translation leaves it left-handed.
So I've taken the operation of reflection combined it with a translation that has a perpendicular component plus a parallel component, perpendicular and parallel meaning the orientation of these two components of the translation relative to the reflection operation.
Anybody want to hazard a guess on how that first one is related to the third one?
Number one is right-handed.
Number two is left-handed, so it's got to be reflection, right?
If I put a reflection plane in here, this one ought to be tilted like this.
That's not going to do the job.
Anybody got any idea?
Yeah
What is that, T, T1 [INAUDIBLE]?
OK. T parallel plus T perpendicular means it's a component of this translation.
This is T. This has a part T perpendicular, and it has a part T parallel relative to the initial mirror plane.
My friends, we have just stumbled headlong over a new type of symmetry operation, which we have discovered upon making this combination of mirror plane with a centered lattice.
And it's come up to smack us rudely in the face even though we may not have been clever enough to think of it in advance.
This is a new type of operation, and it is an operation that cannot be reduced to one of the simple operations that we've defined so far.
You've got to take two steps to get from number one to number three.
The way you can do it is to reflect along a locus that is one half of the way along the perpendicular part of the translation, exactly the same location as we found the symmetry plane positioned when the translation was normal to the first mirror plane.
But yet we can't put the object down yet in the position that would be produced by translation, because our translation is inclined to the mirror plane.
So I've got to take a second step.
Reflect.
Don't yet put it down yet.
Before you put it down, slide it up parallel to the mirror plane by an amount that's equal to the part of the translation that is parallel to the mirror plane.
So to summarize this before all these words get too confusing, I'm saying that a reflection operation combined with a general translation that has a perpendicular component in the parallel component relative to the locus of reflection is going to be a new operation, which I'll write as sigma tau, a reflection part and a translation tau that is parallel to the mirror plane, and tau is equal to the part of the translation that is parallel to the mirror plane.
Astounding.
This is a two-step operation that cannot be described in terms simpler than saying do two steps to do it.
And we'll see as we go along, particularly into a three-dimensional space, that there are other two-step operations as well.
Now you're all familiar with a pattern like this, because in very short order when New England's winter descends upon us, as you go slogging along from your room into the Institute, your footprints will make a pattern like that in the snow.
Exactly what we've got here.
Reflect across and slide.
Reflect across and slide.
Reflect across and slide, and this is the glide component tau.
And this is an operation that is called a glide plane.
And it's a new type of symmetry operation.
It can only exist in a pattern, which has translational periodicity.
And if we were not clever enough to invent it, we would see it as soon as we combined a mirror plane with a lattice that was non-primitive and had a translation parallel to it.
So it's a very, very descriptive name, the glide plane.
Reflect and glide, reflect and glide, reflect and glide.
It sounds like something you'd be doing in the Arthur Murray dance studio, very melodious.
Reflect and glide, reflect and glide.
It's a nice operation.
All right, so what has happened then when we add a mirror plane to a centered rectangular net is that we get a new operation coming in, something completely new.
We've got a symbol to represent an individual operation, sigma the symbol for reflection with a subscript tau.
And the pattern we've already drawn, but let's do it again in a tidy fashion.
Exactly as advertised, the pair of objects related by reflection hung at every lattice point of-- holy mackerel, look at this.
That would give you the willies.
That's the nature of the motif.
Get that out of there.
Mirror planes going through the lattice points.
Glide planes halfway in between.
The mirror lines and the glide planes tell us how things on the right side, for example, of the lattice point are related to the motif of opposite chirality on the left-hand side of the centered lattice point.
And this is a plane group that is called
Question
Yes, sir
How can we just define a new operation that's a two-step.
Seems like we could do this forever, define two steps of any new operation, like you said, with two steps
That's a good question.
We found this, because we tripped headlong over it.
And we say OK, there it is.
We've got to deal with it.
But you're right.
How do you know that rotating once reflecting, and then turning end over end three times is not a new operation that cannot be decomposed, is the word that's used for it, into something simpler?
The answer is you've got to try it.
If they're there, when we make these combinations of a symmetry operation, and now the symmetry operation can be a two-step symmetry operation, now that we've discovered that, combine that with lattice and with rotation and with reflection, and ask what is the relation between the motif at the beginning and the motif at the end.
If you can't describe it any more simply other than a hop, skip and jump, you've got to introduce the hop, skip, and jump as an element that goes into the derivation of these groups.
Now before you get concerned and fill out an add/drop card, I have to reassure you that there are a couple of two-step operations that we have yet to discover, but there are no three-step operations that are necessary.
Whew.
Feel better now, don't you?
I was just going to say it's kind of arbitrary, because you have a and b, you're just applying the third [INAUDIBLE] T sub a and b, [INAUDIBLE] trivial multiplication tables
Well, actually, this is something that is distinct.
I mean here is the group, and you cannot describe the relation between everything that is in this pattern, which was obtained simply by taking the operation of reflection-- we know how to peacefully coexist with that-- and placing that pair of objects at lattice point of a centered rectangular net.
So that's nothing really freaky.
I mean it's a straightforward addition, but if it's a group, you have to know when you combine all the operations pairwise that it'd be able to show that these operations are members of a group.
And the answer in terms of the language of group theory, if you combine a reflection with a translation that has a component that is parallel to the reflection plane, then there is a new operation that comes up that has to be in the group multiplication table, and the operation has a reflection part and a translation part
I have a question
Yes
How do we go from the lower left to the upper right in one operation?
The lower left to the upper right
No, the upper right diagonally.
Across the diagonal, the upper right corner of the square
Upper right here?
Yeah.
And the left-handed guide below that
The left-handed guide below
No.
Down there
Where is down there?
The lowest edge of the--
Down here?
Yeah
OK. From this one to this one?
Yeah
OK. What I can say is-- may sound like I'm slipping off the hook too easily-- we said that we really only want to consider operations that terminate within the cell, and the way I get from here to here is to reflect and then translate up by the diagonal translation.
So that's something that lies outside the cell.
So I can always knock off an integral number of translations or add on an integral number of translations to any mapping transformation, modulo T. OK?
But surely, if you've done, one operation then it's [INAUDIBLE]?
From here to here?
No, not necessarily.
If I give you a very simple pattern and plane group P1, and you ask how do I get from this one here to this one that sits up here?
If it's outside the cell, I've got to go translation that's outside the cell.
But it's not any new translation or any new sort of operation.
This is sort of in the same category.
Yeah
You can make a glide plane in the center of the cell
Yeah, OK.
Thank you.
There is a glide operation that goes from here to here and then translates up by one full translation.
But a glide operation that is an integral number of translations says that you're dealing with-- there's another object that is removed by a translation that is the same thing.
So it is first and that simpler one that would be inside of the cell and would be only the unique sort of translation you need to consider.
Thank you.
That's a good question.
That was a good answer to his question.
In fact, anybody want to take over?
I didn't do well on that one.
Yeah
I was curious.
Does the glide plane exist maybe in this case because we're not using a primitive cell?
If you were to consider this [INAUDIBLE] as hexagonal?
OK, let me answer that question by saying that that's our first encounter with it.
But now, that it exists, that is a symmetry element, which we should consider adding to lattices in addition to pure reflection.
So let me proceed now to do another plane group since we had discovered the operation of glide.
And I will take a primitive rectangular lattice, and I will now add to the lattice point, not a mirror plane, but a glide plane.
OK.
The pattern is going to look like the pattern of glide, have things left and right on either side of the glide plane.
The same thing here.
And there is the pattern.
This is a pattern that's called PG.
And what I have to ask is what is the guide operation sigma tau combined with, let's say this translation T1.
And the answer is that if I reproduce number one to number two of opposite chirality by the glide operation sigma tau and then follow that by T1 the first and the third will be related by a new glide operation sigma tau prime that's located at half perpendicular part of the translation away from the first.
And that really is a generalization of this relation here.
If we make the pure reflection operation a glide operation, combine it with a translation that is perpendicular to it, it turns out that the net result is a new glide operation, sigma tau prime.
The two taus are equal, and this occurs at one half of the translation T perpendicular removed from the first.
And the complete generalization would be to say if I have a translation that has a parallel part and a perpendicular part relative to the glide plane, what I will get is a new glide operation sigma prime that's located at 1/2 of the perpendicular part of the translation from the first, and it will pick up a glide component that's equal to the original tau plus the part of the translation that's parallel to the glide plane.
So this now is this theorem involving translation and reflection type operations in its most general form.
OK.
So this is a new group called CM, and it consists of pairs of objects.
Sorry, we did that earlier.
This is PG, pairs of objects related by a glide plane, hung at every lattice point of the primitive rectangular lattice.
Is there a CG?
Actually, there is not, and let me show you why, and then I think we're just about quitting time.
If there is a centered lattice, and I hang a glide plane at the lattice points, we'll have glide planes in the locations of PG, the pattern will look like objects repeated by glide.
At this lattice point, and now we're going to have another object hung at the centered lattice point, and it's going to be in positions like this.
Now, you can see just by looking at the pattern that there is going to be-- whoops, what did I do here?
I want to move this one up to here, and I want to move this one up to here.
This one should be in this location.
If I look at that pattern, what I've done is to create halfway along the-- quarter of the way along the translation a pure reflection operation.
And I can find that using my general theorem.
I have a glide operation, sigma tau.
I combine with that the centered translation, which is 1/2 of T1 plus 1/2 of T2.
I've taken 1/2 of T1 plus 1/2 of T2, that's the centering translation, combined that with sigma tau.
I'll deftly jump to the side, so the people against the wall can see it.
I've taken a glide operation, combined it with this translation.
This should be a new reflection type of operation that will be located at 1/2 of the perpendicular part of the translation, and that's at 1/2 of 1/2 of T1.
And it will have a glide component tau prime, which is the original tau, which was 1/2 of T2.
And to that we add the parallel part of the translation, and that is 1/2 of T2.
So this is rewritten in slightly different form, is simply a new glide plane sigma prime, which has a glide component equal to the entire translation T2 at 1/4 quarter of T1.
And this is the same as a mirror plane at 1/4 T1.
And this, if we compare it with CM, is exactly the same thing.
Identical to CM with an origin shift of 1/4 T1.
So there is no CG in what we picked up in terms of rectangular nets and symmetry planes is PM, PG, and CN.
So there are three groups involving orthogonal nets in a single symmetry plane.
That is a good place to wrap things up.
And we'll next turn very, very quickly equipped with a set of notes, which summarizes the result, to what happens when we take two MM and put it into a rectangular net, and take two MM and put it into a centered rectangular net.
And then things get interesting, because we've got two planes.
We can make a both mirror planes.
We can make them both glides, or make one a mirror plane and one a glide plane.
So there are three possibilities with the addition of two MM to the net.
More than enough for one day.
It's Thursday.
Take the rest of the week and weekend off by doing no symmetry.
There's no assignment, and we'll have at it again next Tuesday.
Why don't we get started.
Good afternoon-- to stretch the literal meaning of the word considerably.
What a miserable day out there.
I get to and from the boonies via the commuter rail.
And this morning, we sat there for an hour part way in because a tree had fallen across the railroad track.
I don't know who went out to do it, but somebody had to get a saw and saw the tree up and take it off the tracks before we could proceed into Boston.
Even with that, I was sort of reluctant to get off the train.
I saw how it was pouring outside.
So I am highly flattered that you decided to trudge through the puddles and show up this afternoon.
All right, this is a momentous occasion because we are exactly at this point, halfway through the term.
And this is when we are about to change gears from our discussion of symmetry theory and switch over to physical properties of crystals and the way in which symmetry impacts on those properties.
What I wanted to do primarily today, though, is to say a little bit in a very fast and very simple way about the nature of crystal structures because that is one of the primary uses of symmetry theory.
It's to describe the periodic arrangements and symmetrical arrangements of atoms in crystals.
So it's the language that's necessary to describe such arrangements, particularly when they're beyond the red balls at the corner of the cube and black balls in the middle of the faces level of structure.
Let me first, for your amusement and edification, pass out another problem set.
This has only two problems on it.
You are not really equipped to do it yet because we haven't talked about space groups.
So don't worry about doing that now.
But I wanted to get it in your hand so that you could look it over.
It's a good problem set on which to end this part of the discussion-- but there's another one that I'll be passing out next week-- because this is an example of how you can use the notions of symmetry that we have established to say some fairly profound and non-obvious things about the nature of atomic arrangements.
So that's the purpose of this problem set.
It is to give you an opportunity to see the surprising things that can fall out of our material that we now have at our disposal.
Let me begin by telling you where we left off last time.
We had completed deriving the so-called Bravais lattices, the three-dimensional space lattices.
The process by which we did this was to take our two-dimensional space groups, the plane groups, and say that, if we add a third translation, we will have generated a space lattice.
But these different nets in the base of the cell are special and determined by the fact that there are symmetry in these nets.
So if we take, for example, a net that is exactly square-- it's square because there's a fourfold axis there.
And that fourfold axis, in a space lattice, pokes not only into the base of the cell but extends up through space.
So you must pick a third translation such that the symmetry elements line up.
We would take each of the crystallographic plane groups and pick a third translation.
You end up with almost all of the lattices that you're going to get except for the ones that involve cubic symmetries that are inherently three-dimensional.
And the reason for that is that the more complex three-dimensional point groups are, for the most part, obtained by adding inversion to the simple symmetries that exist in two dimensions.
But inversion doesn't require anything of a lattice.
So you can immediately say that 2M and 2/M are compatible with the same sort of space lattice because the symmetry 2/M is nothing more than what you get when you add inversion to either a twofold axis or a mirror plane.
And, finally, another way of classifying crystals, and a broader way of classifying them, is according to the coordinate systems that are necessary to describe the lattices.
And these are called the crystal systems.
And, as a mnemonic device for keeping straight what the crystal systems are as opposed to the crystal classes, which is another word that we haven't used but it's another word for the point groups, the crystal systems goes with coordinate systems.
And it's simply a word to describe the shape of the lattices, family of lattices, that we've described.
And we distributed a handout last time that gave the names of these systems and the relations that exist between the edges of the unit cell, which we use as the basis for our coordinate system and also the labels that we put on them.
I have another sheet, which I'll pass around, which is nothing new.
It just assembles-- I'll give you two sheets, which I'll pass around.
The first is an assembly of the crystallographic point groups in a somewhat neater and tidier way than we had on the sheet that we passed around last time.
So this is a picture of all 32 of the point groups.
Again, beware of the fact that mirror planes and guides to the eye that split things up into 90 degree segments or 60 degree segments come out deceptively similar on these sheets.
But there with the representative pattern are the arrangements of symmetry elements, a representative pattern of motifs, and the international and [INAUDIBLE] symbol for all 32 of the three-dimensional point groups.
And then the next sheet I'll pass around takes the point groups and groups them together with the lattices that can accommodate them and the crystal system that describes the arrangement of axes in the lattice.
So this is, in a nutshell, everything that we've covered up to this point.
And, at this point, I have to say that there is some disagreement on how many crystal systems exist.
And even though people who work with symmetry and mathematics are usually rational, almost to a fault logical individuals, there are several junctures as you build up a body of knowledge where you could go either way or adopt either convention.
You sort of got to pay your money and take your choice.
And this occurs in our arrangement of point groups and lattices among the crystal systems.
This occurs in the hexagonal crystal system.
Symmetry 3 and the other point groups that can be derived from it can be placed either in a hexagonal lattice, that is to say, a third translation C that is exactly at right angles to a pair of translations, A1 and A2, which are 120 degrees apart.
Or a three-fold symmetry can go into this funny double-body centered hexagonal lattice, which could be defined, if we were masochistic and enjoyed working in an oblique coordinate system, could be defined instead in terms of a rhombohedral lattice, which had the special characteristics that three translations are identical by symmetry, called therefore A1, A2, and A3; and the angles between the more equal alpha 1, alpha 2, alpha 3; and they could be anything we wished.
So that is a unique sort of crystal system and unique sort of coordinate system.
There are those who would say that if this is the case, we really have seven crystal systems-- triclinic; monoclinic; orthorhombic; tetragonal; hexagonal; and then this one that is compatible only with a three-fold axis and that is called the trigonal system or, by some folks, the rhombohedral system; and then, finally, the cubic.
Well isn't that a rational and logical thing to do?
Yeah, question at this point?
Do you have any extra copies of the sheet [INAUDIBLE]
Oh, yes.
There should be more than enough floating around.
This one here?
[?
The ?]
[?
chart ?]
Yeah, [?
not the ?]
[?
chart ?]
OK.
I'll give you-- I think there are more back there.
But why don't you take one for now?
Yeah, so it's a perfectly reasonable thing to do is to set up the trigonal system as a separate coordinate system to describe the rhombohedral lattice.
The problem that creates, however, is that if you do this, there is no longer any one to one correspondence between the point groups and the crystal systems.
A crystal with symmetry 3, for example, can fit into it either in a hexagonal lattice or a trigonal lattice.
US So sometimes a crystal with point group 3 would have to be called a hexagonal crystal, sometimes a trigonal system.
But which is it?
Well it can be either.
So admitting the trigonal system as a separate crystal system breaks down the one to one correspondence between point groups and crystal systems.
The very pragmatic reason for not admitting the trigonal system as a separate crystal system is that, although you can in this funny triple hexagonal cell define a primitive trigonal cell that has a special geometry, nobody in their right mind, as I indicated earlier, would want to work in such a coordinate system.
Better to take the triple cell, pay the price of three-fold redundancy, and yet have two interaxial angles that are 90 degrees and one that's specialized at 120.
That's a much more convenient reference system for our description of crystal properties.
One final thing I'll comment on but won't say anything about very much-- I always do that.
I say I won't say anything about it but, and then launch into a 10 minute digression.
Off in the right hand corner is something called the Laue group of which there are 11.
The Laue groups are those of the point groups that contain the operation of inversion.
And the reason they're called the Laue groups is that one of the primary ways, other than looking at crystal morphology to determine the symmetries of a crystal, is to look at the way it diffracts x-rays.
So if we had, for example a tetragonal crystal and there were lattice planes in here that were related by 90 degree rotations, then if we brought in an x-ray beam at such an angle to satisfy Bragg's law and saw some sort of intensity that came off, if we rotated the x-ray beam 90 degrees, we ought to see diffraction at the same angle and with exactly the same intensity.
So either by moving the x-ray beam, or simpler thing would be to leave the x-ray beam fixed, since it's attached to an x-ray generator that weighs a ton or more, instead move the crystal relative to the x-ray beam.
Now what would you do in this way of thinking to determine whether a crystal had the operation of inversion in it?
If this is the set of planes with indices HKL, you, again, would bring in an x-ray beam at an angle theta, in out.
Look at the intensity from the set of points HKL.
And then look at diffraction, both in terms of the Bragg angle theta and in terms of the intensity.
Look at that diffraction from the planes that were related to HKL by symmetry.
Well what plane is related to HKL by symmetry?
It's these planes down here which have indices minus H, minus K, minus L. So if we did the corresponding experiment to what I did to the left, we'd bring in an x-ray beam like this.
It would be diffracted at some angle theta.
And we'd look at the intensity for the set of planes minus H, minus K, minus L. Is that going to be any different?
No.
Right-minded person would say, why should this intensity be different?
All we're doing is bouncing the x-ray beam off the opposite side of one in the same set of lattice planes.
So why should the intensity be any different?
In this case, clearly if the set of lattice points is rotated by some finite angle, we are not going to see equivalent diffraction from planes that are 90 degrees apart.
So the upshot is that the diffraction symmetry of a crystal, the way in which planes with indices that are related by symmetry diffract in terms of position of the beam and intensity, the diffraction symmetry of a crystal always looks as though that crystal possessed an inversion center.
So, therefore, to-- and I qualify this by saying, to a good approximation-- the only point groups that you could determine and distinguish from one another by x-ray diffraction are the point groups that contain an inversion center.
Almost, but not quite.
If we look a little more closely at what is doing the scattering, it is not the Bragg planes.
These are imaginary, shimmering, absolutely planar things that are constructs that we use to interpret the process called diffraction.
Inside the crystal what is actually doing the diffraction is a collection of atoms, each of which are scattering the x-radiation.
And that physically is what causes the diffraction.
And the resultant intensity is the addition of all the little wavelets scattered by these separate atoms.
So in terms of the atomistics of what is involved, to look at diffraction from the set of planes HKL and contrast that with the diffraction for the set of planes minus H, minus K, minus L, we're coming in from opposite directions.
And here the beam would hit-- let's call the atoms, if I did not do so already, let's call them A and B.
A for a and b for big.
Diffraction from the set of points HKL would pass through the sheets of A atoms first and then impinge on the B atoms, whereas in the opposite direction, the x-ray beams would impinge upon the B atoms first and then hit the A atoms.
Does that make a difference?
Aha.
Believe it or not, it does.
And I can't explain why without going into a very detailed discussion of diffraction.
But if you compare the intensities that are scattered by these two sets of planes, which are related by inversion and therefore just opposite sides of one another, it does make a difference which atom the x-ray beam encounters first.
And the difference in intensity is on the order of 3% to 5%, although by picking a radiation appropriately for the particular chemical species in the crystal, you could increase this perhaps to 10% or 12%.
But that is something that you could not do until synchrotron radiation came along, which could actually tune the wavelength to a particular value that you wish to use and not use something that was spewed out by an x-ray tube that had a target such as molybdenum or copper or chromium, perhaps.
OK, so you can with this careful measurement of intensity distinguished between the opposite sides of the given set of Bragg planes, unless, of course, the crystal did have inversion in it.
And then there would be no difference at all.
It requires a very careful experiment, though, because there are many things that could make the intensities different.
For example, absorption by the crystal.
If the crystal has an irregular shape, the absorption, which can easily be factors 10, 20, or even more, the absorption would be different for x-ray beams coming in those two orientations.
So in any case, as I say, I won't say anything about it.
But 10 minutes later, that's the meaning of the Laue groups.
These are the point groups that have inversion in them.
That effect that I just described, by the way, is called anomalous scattering or anomalous dispersion.
There's really nothing anomalous about at all.
It has to do with the way electromagnetic radiation interacts with the clouds of electrons that are distributed about the nucleus of each atom.
Where we will go from here to finish up symmetry theory will be to now take the 32 crystallographic point groups and to drop those point groups into each of the distinct 14 Bravais lattices.
And then we'll need combination theorems to deduce where the additional symmetry elements arrive.
And when we go through that process, if we were to do this exhaustively, we would find that there are 230 distinct three-dimensional symmetries that involve point group symmetry operations and lattice type.
I saw several faces blanch.
Don't worry.
We're not going to do that.
What we will do next time is do a small handful and then conclude by showing you how this information is tabulated in the international tables.
And the quick, immediate answer to that question is that the problem set that I passed around has two pages for two different point groups that show the way that this information is tabulated in the international tables.
So that's where we'll end up.
And we'll come pretty close to completing that when we meet on Thursday.
I heard no sighs of relief.
I heard no cheers.
So you are presumably still engaged in this material somewhat.
I have two things I would like to do.
We talked about crystal systems and point groups.
And let me now ask, do you think there is a crystal known for every one of the point groups?
Think so?
I saw some skeptical shakes of the head.
The answer is yes, but just barely for some of the point groups.
One of the very rare point groups, in terms of structures that populate it, is our high symmetries but which lack inversion or mirror planes.
One very, very rare point group is 432-- just axes arranged in a cubic arrangement with no mirror planes or inversion centers.
And my rationalization of that is that if an arrangement of atoms has to conform to all of these rotational symmetries, it's pretty hard for it to do it and have that assemblage of atoms assume the lowest possible energy state if it doesn't pick up in the process inversion or mirror planes as additional symmetry element.
Sort of satisfying for me.
But that's not a rigorous argument.
OK, let me now take a poll.
Do you think, using the broader cut of the crystal systems, do you think there's a crystal known for every crystal system?
Are there triclinic, monoclinic, orthorhombic, tetragonal, trigonal, hexagonal, cubic crystals known?
Yeah.
That's a pretty realistic expectation.
And that is indeed the case.
But now let me cut this a little finer.
Which do you think is the most popular crystal systems for real materials and which do you think is the rarest?
OK. Who thinks triclinic crystals are the most common?
OK, there are pessimists.
What is the definition of a pessimist?
The pessimist is somebody you should borrow money from because they'll never expect to be paid back.
Who would say cubic crystals?
There are naive [?
optimists.
?]
Now I will reveal all.
The answer is, it depends.
Depends upon whether you're talking about, interestingly, inorganic materials or whether you're talking about organic materials.
If you look at inorganic materials, the most popular crystal systems are-- and the results are, first of all, cubic.
Is this not a benign world in which we live.
Most inorganic materials-- if you're a material scientist who works with metals and with ceramics, you're going to work with cubic materials, thank God, most of the time.
The second most popular-- by a considerable margin but there are still a lot of them-- is orthorhombic.
And the third most popular, shortly behind orthorhombic and almost in a tie, monoclinic and tetragonal.
We'll launch into a quick overview of crystal symmetry.
And we'll see why inorganic materials have high symmetry.
It's true, in particular, when the material is held together by ionic bonding.
If the bonding is ionic, it's non-directional and atoms of opposite charge will therefore try to get as close as possible to the atoms of opposite charge.
And because atoms of like charge repel one another, the surrounding atoms will try to be as widely separated as possible.
And this grouping of atoms, positive around negative, tends to have a lot of symmetry.
And, therefore, the way in which these groups are linked together has a lot of symmetry.
OK, I said organic materials.
And I'll pass around some actual numbers that were assembled a long time ago but are still, in terms of proportion, still valid.
The organic compounds are broken down into whether these are compounds based on rings or extended, lopsided molecules.
And the numbers are very different depending on that.
But, for organic molecules [?
material, ?]
monoclinic is the most common, then orthorhombic.
And shortly after orthorhombic comes way, way behind is tetragonal and triclinic, which are almost tied.
And I think I can rationalize this observation, as well.
Organic materials contain discrete molecules.
And if we've got something-- and I'm going to, since I know I'm among friends, display my abysmal ignorance of organic chemistry.
I am unapologetically and unabashedly inorganic and even non-metallic in my research interests.
So let's suppose we have something that's composed of various strange rings with little side groups sticking off like this, another one off like this, and then maybe another ring up like this.
That, to me, looks like a reasonable molecule in terms of my ignorance of organic chemistry.
So if there are these finite groups and you're going to put them together in a crystal, what are you going to try to do?
They're going to be held together by very weak van der Waals forces, which are non-directional.
So think of the outline of this molecule as being some ugly thing like a ham hock.
And then your problem, if the forces between these molecules are non-directional, how do you close pack ham hocks?
The way you would do it would be to put down a row of them in a close-packed sequence.
So we'll put down things that look like this.
And then, maybe to make them fit together, dovetail another one in here.
And maybe something like that.
And then for the next row down, and you would maybe put the nose of one in here.
And then put the other one in here, the nose of one here, and something like that-- being very schematic.
The lattice that describes this packing is not going to be orthogonal.
It's not going to be square with high symmetry or even have two orthogonal translations.
It's going to be something that has this as translations.
There's going to be an A and a B that is not at right angles to it.
And the B translation will have different links, simply because the thing that you're putting together in close packing has a very irregular shape.
So I think that is the reason why molecules, discrete molecules that try to achieve something that is as close packed as possible, will have translations in them that are of unequal length and making arbitrary angles with respect to one another.
So having kept you guessing for so long, let me pass out another sheet.
This was a compilation that was done a long time ago.
And there were only 9,000 crystals for which complete crystallographic data had been obtained.
This was way back in 1967.
As I say, there's much more data now, easily in order of magnitude.
But I don't think the proportions will change very much.
They'll stay about the same.
Let me pass out these sheets.
So make sure you memorize them because the first question on the next quiz is, how many out of 8,759 crystals are actually triclinic?
I would never do that.
All right, any comments or questions?
Yes, sir
Why would organic molecules of such odd shapes even be crystalline to begin with?
Why wouldn't they just be amorphous?
OK. Clearly the interatomic forces are very directional and very strong.
And that's going to hold the molecule into a fixed configuration
Didn't you say there were van der Waals forces?
But they're these-- even for something that is not ionic or metallic, there still are weak attractive forces between these molecules.
Why?
Well, the charges are not uniformly distributed over these molecules.
So there are going to be some van der Waals forces that tend to hold them together.
And a van der Waals force is essentially non-directional.
So if you have this weak force, you're going to be able to squish the crystal into something amorphous very easily.
And organic materials are usually very subject to plastic deformation.
But if you assemble the molecules carefully from solution, or by vaporization, condensation, they're going to want the order if they are solidified with enough mobility to rearrange themselves in the lowest energy configuration.
I think what you're saying is these are not going to be very strongly bonded crystals.
They're going to melt at low temperatures.
But, nevertheless, prepare them properly, and the lowest energy state will be ordered
I'm sure [?
they're ordered.
?]
But would it be such that you have [?
translational ?]
symmetry?
Who's to say it's going to have a definite pattern that will repeat itself?
Yeah, if there are attractive forces-- let me again use a hand-waving argument.
If you've got a bunch of particles and they aggregate here in the lowest energy configuration, that is going to be the lowest energy configuration.
And a similar group of atoms will want to do the same thing here.
And if there are interactions between these groups, they're going to arrange themselves in an order configuration.
That is simply saying-- this is a very weak qualitative argument-- says if you have a configuration which is the lowest energy arrangement here, any other place in the system where those atoms get together and congeal are going to assume that same configuration.
OK?
Other comments or questions?
This is all very qualitative but actually concerns matters that we are describing a language to describe
[INAUDIBLE] Yes, there are.
If you look at this table that came around, if you look at cubic organic materials out of-- there-- what is the number?
117 out of what looks to be several thousand.
Yes, there are cubic organic crystals, but very few in number.
Hexagonal-- 56 out of a rather large number.
All right, when we took a poll a week and a half or so ago and said, how many of you had seen any material and crystal chemistry in solid state chemistry in a previous class on structure and bonding or crystal chemistry, and only a few people tentatively twitched a hand slightly.
So what I'd like to do for the rest of this session, since this is what symmetry theory is developed to describe, I'd like to say something about structures and why they form the way they do to give some qualitative insight.
This is a fat pack of notes.
And I'll go over it quickly with you and lead you by the hand through it.
But this is not material for which I'll hold you responsible.
In fact, I'll turn my back and those who want to get home early because of the rain can tiptoe out.
And I won't even notice who you are unless everybody goes.
And then I'll feel very hurt because I wrote these out.
And it took a lot of trouble.
All right.
I have to apologize and say that most of what is described in these notes applies to ionic bonding and, to a lesser extent but still in large measure, to metallic structures.
And what is the reason for that?
It's because the metallic bond, to a fair approximation, an ionic bond rigorously, if you believe Coulomb's law, is a non-directional sort of bonding.
So the structures that are assumed, the structures that are the lowest energy configuration turn out to be determined by interatomic distances and ionic or metallic sizes.
The Coulombic interaction that holds ionic compounds together is a central force.
And one of the features of electromagnetism is that, if you have some distribution of charge about center and another distribution of charge of a different sort about a another center, the cohesive force between, let's say, a positive ion and a negative ion is, along a line, joining their centers.
And, secondly, you can take, once you're outside the distribution of charge, you can clump all the charge together at the center of the distribution.
So that's what makes the ionic bond so simple.
And that's why everybody who talks about bonding and structure discusses ionic bonding because organic chemistry is just too complex in comparison.
So let's look at a given cation.
And cations generally tend to be smaller than anions because you've removed electrons from the outer configuration.
We'll look at some effective sizes of ions in just a bit.
But you've stripped electrons off to make a positively charged ion.
You've added electrons on to make a negatively charged ion.
So when you add electrons, the ion functions has [?
always ?]
had a larger radius.
So let's bring up a B minus ion as a neighbor.
And for those two species, there will be, if we put energy as a function of separation D, there's going to be a Coulombic attractive force that goes as 1/D.
But when the electron configurations get in to close proximity, at small separations, the electrons will start to push one another, beginning with the outer electrons first, into higher energy states.
And the energy of interaction, due to the interaction of orbitals, will increase the energy of the system.
And the result then is that the net potential, as a function of distance for an isolated ion pair-- I was looking for some colored chalk and it's gone-- is something that rises steeply at low separations and then levels out more gradually at larger separations.
So this then would be an equilibrium separation, D0, for that ionic pair.
If we got energy out, and for that isolated ionic pair it'll be just exactly this much energy, E0.
If it weren't for a pair of ions-- let's take another A and bring it into the system.
And there's a lot of room to put still another A around.
And we can add that to the system.
And every time we bring up an additional neighbor, we get approximately this much energy out of the system.
But not quite because there will also be repulsive interactions between the ions of like charge.
OK?
But to a good approximation to within about 10% to be quantitatively, every time we bring up another neighbor, we get out the same amount of energy again by forming this group.
And all that works splendidly until-- let me now look at the arrangement of anions about the little cation because they will have to have a shell as well.
So if we have a small cation and we bring up a big fat anion, there will be, first of all, two consequences.
These ions of like charge will interact repulsively.
So the lowest energy situation is going to be one in which these neighboring ions arrange themselves so that the repulsive energy is minimized.
And we're going to get the maximum separation between all of these anions and get the minimum energy when they arrange themselves on the corners of a regular polyhedron.
And that's where symmetry starts to come in to the importance of ionic structures.
That, for this particular configuration of anions, the B ions might be on the corners.
And I am not drawing them in contact now just so it'll be easier to draw.
The B ions will be equidistant and tend to arrange themselves on the corner of the regular polyhedron.
If we got energy up by bringing up extra neighbors, we will reach a point where the B ions will essentially be in contact.
And the A ion is rattling around in a hole.
It can touch perhaps two neighbors.
And we will get the full amount of energy on.
But for the other two neighbors, we're going to be out here on this curve.
And we won't get nearly as much energy out.
But at the same time, we're paying the price energetically of all of these repulsive interactions between the surrounding neighbors of like charge.
So the second consideration is that, as we said, first of all, this group of atoms, which is referred to as a coordination polyhedron, it first of all will be symmetric.
And then the number of neighbors, the number of ions of one charge that are coordinating the others-- and that's something called the coordination number, introduce some jargon-- has an upper limit that is going to be determined by the size of the ions.
So these are the two basic tenets of ionic crystal chemistry.
So having established those points, we can do some geometry and assume that the ions are hard spheres or at least elastic spheres with some stiffness because it's going to take energy to perturb the orbitals of the electrons and, just in terms of simple geometry, calculate the range of radius ratios RA/RB where A is the central ion in the polyhedron and B is the surrounding ion, and do this for different coordination numbers, picking polyhedra in which the surrounding ions of like charge are as widely separated as possible.
So starting at the beginning, coordination number one would be a pair of ions.
Obviously no constraints whatsoever on the relative sizes.
RA/RB can be anywhere it likes between zero and infinity.
For coordination number two, again the surrounding ions want to be as widely separated as possible.
So it'll be a linear chain.
But, again, the central ion can become as small or as large as it like.
So the range of radii is between zero and infinity.
For coordination number three, the polyhedron will be an equilateral triangle.
And if you calculate the geometry, when the B ions are exactly in contact, you have a little 30, 60, 90 triangle in which one edge of the triangle is RB and the other is RA plus RB.
And you can manipulate that into the form where you can show that there will be a lower limit RA/RB, which is 2 over the square root of 3 minus 1.
And that turns out to be a magic number, 0.155.
Going to coordination number four, you'd say, aha.
That's going to be square group.
No, a square group would not keep the surrounding ions of like charge as widely separated as possible.
And the geometry which would keep them as widely separated as possible but yet in contact with the central sphere would be a tetrahedron.
The Bs would be arranged on alternating vertices of a cube.
The magic number there turns out to be 0.225.
Coordination five-- very, very rare.
Five is a crystallographic abomination because five-fold symmetry does not conform to a lattice.
But there would be, in principle, one five-fold group.
And that would be a trigonal prism where you have three Bs arranged on an equilateral triangle about the central A.
And then you have one A up here, another B up here, and another B down here.
So this, then, is a bipyramid, a triangular bipyramid.
And you can derive a range of radius ratios for this.
And this is a dandy problem to give on a problem set.
But you find that the range of radius ratios RA/RB is the same as for six-fold coordination, which is the next one considered on the handout.
So we six coordination with the corners on a regular arrangement would be an octahedron.
And the permitted range there is 0.732 to infinity.
Five coordination, the trigonal prism, leads to exactly the same range of radii.
And the reason is, if you look at the geometries, this triangle here is the same for both an octahedral coordination and a trigonal pyramidal coordination.
Notice as we chug on that the lower limit to the radius ratio is gradually increasing.
And if we go to coordination number eight, cubic coordination, it goes up to 1 to infinity.
And for coordination number 12, RA/RB has to be exactly 1 period.
And that would be what you achieved in the close-packed number.
OK, turning the page.
There is also going to be a range of radius ratios that's determined by the number of As that can be packed around a central B.
And doing exactly the same calculation, if, for example, four-fold coordination permitted in terms of RA/RB, the range is 0.225 to infinity.
If we want to know the limits for packing of A around B with tetrahedral coordination, this would give us numbers RA/RB that were exactly the same range of numbers because we've just changed the labels.
If we want to put everything on the same basis, in terms of one particular radius ratio, if we do anion to cation radius ratio, the range would be 0.225 to infinity.
If we take the reciprocals of these numbers for four-fold coordination of B, this would be 1 over 0.225 to 1 over infinity.
So this will give us-- this is zero.
This is infinity.
So there'll be a range of radius ratios, 0.225 less than RA/RB, less than 1 over 0.225, which turns out to be 4.44.
So this would be the range of permitted radius ratios for CNA equals 4 and CNB equals the same thing, 4.
So there's going to be an upper limit that's imposed by the packing of As around B and a lower limit that's imposed by the packing of Bs around A.
But what are going to be the coordination numbers of A and B?
Let me finish this up.
And then we can take our break.
I'm going to now restrict things rather drastically.
I'm going to consider binary compounds in which all cations have the same coordination and all anions with the same coordination.
And the implication there is there's only one type of coordination number for both.
And that's not always true in compounds.
Sometimes you find a given species that has two different coordination numbers in a more complex compound.
If that's the case, if this is the A ion and this is the B ion, let me consider the chemical composition of a bond.
And that's going to translate into the same thing as the compound.
So let's suppose there's a number of Bs around the As, which is, by definition, the coordination number of A, and a number of As around a B and that is, by definition, the coordination number of B.
So let me look at a wedge that's associated with this bond.
And this is an amount of an A atom, which is 1 over the coordination number of A.
And if I do the same thing for the B ion the part of the B ion associated with one bond is going to be 1 over the coordination number of B.
So we can say then that the composition of a bond is going to be A subscript 1 over the coordination number of A, B, 1 over the coordination number of B.
The chemists abhor fractional subscripts on a chemical composition.
So let's multiply this by the product of the two coordination numbers.
And what we'll find is that the formula of the compound with integer subscripts is going to be A, C, and B; B coordination number of A.
And that is a subject to a very drastic set of assumptions that each species A and B has just one coordination number.
OK, so doing this in terms of composition, just turning this result around.
And then we'll quit and take a stretch.
This says that if we look at a particular composition, AnBm, something like TiO2 or Al2O3, then the coordination number of A in proportion to the coordination number of B is going to be equal to the ratio of M to N. And the more ions of one charge that we can pack around the other, the lower the energy will be.
And, therefore, the coordination numbers will be the maximum permitted values of CNA and CNB, subject to the constraints of radius ratio.
In other words, to conclude very quickly with one specific example, if we look at a compound like Al2O3, an important ceramic and a gemstone, this says that the coordination number of Al to the coordination number of the oxygen has to be equal to three to two.
So possible structures might be aluminum with coordination number three, oxygen with coordination number equal to two.
Or aluminum could have six coordination.
The oxygen could have four coordination.
Or the aluminum could have 12 coordination.
And the oxygen could have eight coordination.
Those three possibilities.
Next we have to use the ionic sizes to see what constraints there are that are going to limit the maximum coordination numbers because the greater the number of neighbors, the lower the energy.
And, in the case of aluminum oxide, it turns out that it's six to four.
And this is determined by the radius ratio.
And you know what?
If we used the ionic radii for Al3 plus and O2 minus, we were right one the nose.
That is the nature of the structure of aluminum oxide-- aluminum with octahedral coordination by oxygen and oxygen with a tetrahedral coordination of aluminum.
And that is the actual structure.
The three-dimensional linkage would have to be such that ions of like charge have the maximum possible separation.
And to keep you beyond your patience for just a little bit, what is that going to be?
We can use very simple considerations there.
If we have coordination polyhedra and we want to join them together, the worst possible situation is to have them share edges because the central ions then had minimum separation.
The better situation would be to have to the polyhedra share corners and have the ions in the center of these triangles form a straight line with the shared ions.
For a three-dimensional polyhedron, the sequence of increasing repulsive energy would be shared faces for something like cubes, shared edges-- that would be lower energy but not the best you could do.
Shared corners would be the best possible arrangement.
As you make each of those transitions, you take the central ion and keep the separation between the neighboring ion of like charge as low as possible.
The interaction energy would be as low as possible.
All right, so let me stop there.
I think we're probably about halfway through.
We can then determine ranges of radius ratio for various stoichiometries.
And all this is just a fantasy unless we can say, do ions have sizes that are the same from compound to compound?
And, if so, how do we determine them?
I hope that intriguing pair of questions will keep you here, dry and warm, rather than heading off early.
OK, so let's take our usual 10-minute break and resume to wrap this up.
All right, let's resume our discussion.
And we'll finish it for sure by the end of today.
And as you say, you can tuck this away for study on your own or for future reference or whatever you want to do.
So if we could assign sizes to atoms and ions, we could do a lot towards predicting what sort of structures they might form if we knew the composition.
The composition-- again, assuming there is only one type of anion and one type of cation, the composition is determined by the valence.
The composition fixes the ratio of coordination numbers.
And the structure will have lowest energy when we have the maximum number of anions around cation and vice-versa.
So we're in great shape.
And this is simple, idealized, but very powerful stuff if we knew the sizes that were assumed by ions.
And that, really, is an absurd notion, to talk about an atom as though it behaved like a sphere with finite dimensions.
If we plotted the density of the electrons as a function of distance R from an atom, it does something like this.
At very small distances, that defines a shell of a rather small volume, so there are not many electrons.
Rises up to a maximum, and then tails off gradually.
So what is the radius?
The maximum in the distribution of density.
So the notion of radius is absolutely nonsense in terms of the actual distribution of electrons on the atom.
However, one can get from your fraction measurements a determination of the lattice constant.
And even with routine methods, you can get the dimension of the lattice to extraordinary precision.
The typical lattice constant is about 10 Angstroms.
You can get this through routine methods to plus or minus 1 in the fourth decimal point.
So this is a precision of 1 part in 10 to the 5th.
You can get atomic positions when they are not fixed by symmetry to get a coordinate to plus or minus 0.yyyy plus or minus 1 in good cases.
So that is 1 part in 10 to the 5th.
Missing a factor or 10 somewhere.
This is plus or minus 1.
That's 1 in 10 to the 4th.
So you can get this information very accurately.
One thing I always like to point out here is that a thermal expansion coefficient for most inorganic materials is something that never gets much lower than about 1 or 2 times 10 to the minus 6.
So if you're going to push the measurement of a lattice constant to the ultimate accuracy, you'd better have a thermometer on the top of the X-ray generator, or your measurement is meaningless.
The thermal expansion can be more than the uncertainty in your measurement.
So you should report, for very precise measurements, the temperature at which you made your measurement.
OK, you can get interionic distances with very high precisions.
And if you look at a series of compounds-- and these are the ones that are discussed in the notes, which have the rock salt structure type.
So here is the cation, here is the anion.
The anion-cation distance is simply equal to 1/2 of A for the rock salt structure.
So that's easy.
You don't even have to know the atomic positions.
Just measure the lattice constants.
And then you would find, for example, that R, NA plus R, CL is equal to 1/2 of A for NaCl.
Very precisely know the sum.
How do we split this up?
Well, let's look at another compound.
Let's look at R, NA plus R for bromine.
And that would be 1/2 of the lattice constant of NaBr.
Hm.
More data, but we haven't solved anything.
We have three radii, only two measurements.
And you can see, if you try to push this further, you can never get enough information to split up the interionic distances into sums of individual radii in any sort of unique fashion.
So you have to start somewhere.
And there are different sets of ionic radii that were proposed early in the game.
And they are discussed in the notes.
I give you an example of some of the sums of radii for the oxides and sulfides and selenides of the transition metals.
You can see that as the anion atomic number increases, the size of the lattice constant goes up.
So you could get, for example, the difference in size between a sulfur and an oxygen and the difference in radius between a selenium and a sulfur.
But you could never get the absolute values.
There are four early sets of radii.
And they are tabulated for you part way through the notes.
And there's a cautionary lesson here.
One the early series of radii-- see, even the old workers in this field appreciated the role of ionic sizes in determining structure.
So they wanted to set up self-consistent schemes of radii.
And the earliest one is by, really, the founder of crystal chemistry, a man named Goldschmidt.
Linus Pauling got into the act very early on.
If you look at this table that I copied from these separate publications, four different people-- Goldschmidt, Pauling, Zacharias and Ahrens.
Some people left out the radii of certain species.
Other people left out different radii depending on the compounds that they looked at.
If people looked at the same ion, they very often got very, very different values, depending on their starting point.
Look at the size of barium, for example.
Zacharias said 1.29.
Ahrens said 1.34.
Pauling said 1.35.
Goldschmidt said 1.43.
Depends on the starting point that they used for the initial radius.
And once you've determined that, everything falls into place.
Another basis for determining radii, Bragg determined a number of the structure of important silicates.
Why silicates?
Because as you walk along the ground, you're walking on silicates.
These are part of the minerals that form the Earth's crust, so there was a lot of interest in those.
Bragg, from his early work way back in the late 1920s, found that the size and shape of the Si04 tetrahedron forms tetrahedral coordination because silicon is so small compared to oxygen.
The interionic distances were always about the same.
The tetrahedron was always regular.
So what Bragg said was, a-ha, this is a tough little nut in silicate structures.
Doesn't deform.
The size is always the same.
It's always regular.
The reason must be that the oxygens are just in contact.
So he used the dimensions of an Si04 tetrahedron, took the radius of oxygen as half the edge.
Pauling used a different starting point that has been followed.
There's still another way of determining radii.
Is there any physical property of a material which might also be shown to be proportional to the size of the ion?
And the answer to that, which I'll pass over quickly, is yes.
Quantity called the ionic electrical polarizability, which is the relation between an applied electric field and the dipole moment that is induced on the atom when you separate the electron cloud and the nucleus.
And a simple model shows-- and this is carried out for you-- that you can relate the ionic electronic polarizability to the volume of the atom, and it turns out to be beautifully simple, that the polarizability is simply 4 pi epsilon 0 times the radius of the ion cubed.
So therefore, you can split up ionic radii in proportion to the cube root of the polarizability of the ions.
Well, that trades one problem for another.
How do you get polarizabilities?
It turns out that the index of refraction of materials is given by the sum of the number of each species per unit volume times the ionic polarizability of that species.
And there is a famous equation called the Lorentz-Lorenz equation.
I can never resist calling it not the Lorenz-Lorentz equation but the Lorenz-Lorentz equation equation because it sounds like you're saying everything twice.
If you note that the dielectric constant of a material is related to the square of the index of refraction, instead of n squared, you can replace the n squared by epsilon, the dielectric constant, and then the equations become known as the Clausius-Mossotti equations.
And I always take that as a nice illustration of the fact that a very simple change of variables can be all it takes to become famous and immortal.
[INAUDIBLE] what happened to Clausius and Mossotti.
Where do you get polarizabilities from?
This was looked at by a number of people.
One of the most complete were three people, Tessman, Kahn, and Wild Bill Shockley of transistor fame.
So you can find these numbers, and you can get a self-consistent set of radii.
But note in the table that I give you here that the individual numbers are very different.
And two caveats that I have to issue is you can't mix ionic radii in different people's tables because they have different starting assumptions for the standard radius.
If you do that, you're mixing apples and oranges.
And I wish I had a cookie for every half-baked book on chemistry that said, hey, Ahrens has got something for americium and Pauling doesn't.
Let me put the value for americium in there.
Totally wrong that they are established on different bases.
So you can't mix apples and oranges.
You have to use a self-consistent set of radii.
The ones that were followed until very recently are Pauling's radii.
And Pauling's radii are based on a standard radius for oxygen, which is 1.40 Angstroms for the radius of oxygen.
The other thing that I have to issue as a caveat, which I hope you won't forget, is that radius ratios do a pretty good job of predicting the expected coordination number.
But if you are very, very close to a limiting critical radius ratio, all bets are off.
Because then you're really asking the question, if I have this polyhedron of ions, if I squish them a little bit so that I increase the coordination number, does that give me a lower energy in return for the price that I have to pay to deform the ions that I'm packing together?
And that is not possible to answer on the basis of a rigid sphere model.
So when you're very close to a critical radius ratio, you really can't say for sure whether the structure will go to the lower coordination number or the higher coordination number.
So you have to be careful of that.
Finally, people have continued on.
And stuck in the margin of these older radii is a reference to a paper by two people, Bob Shannon, who was a crystallographer at DuPont, and Charlie Prewitt.
I'm very proud that he became so notable, because he was my office mate in graduate school.
So I like to promote his radii whenever I can.
And in Acta Crystallographica in 1969, they published a set of radii, which is in the nice typeset version that's Xeroxed directly from their paper in Acta Crystallographica.
And the headings need explanation.
There's the ion and its valence, then the electron configuration.
And what's interesting is that some of the transition metals, for example iron, can be in a high-spin or a low-spin configuration depending on whether the moments on the electrons are all parallel or anti-parallel.
High-spin and low-spin configurations give ions with very, very distinct radii.
So the second column, EC, stands for Electron Configuration.
Then comes coordination number.
And here is another feature that we really should not overlook, that as even rigid spheres-- here's an A, and here's a B-- acquire progressively higher coordination numbers-- let's look at just coordination number three and coordination number four, because they're easy to draw.
In addition to the attractive force between anion and cation, there is also a repulsive force between the anions.
So here's the repulsive force between this B and its neighboring B and this B and its neighboring B.
And the resultant is a force that tends to push apart the A and the B ion.
For four-coordination, remember first of all, the Bs are closer, so the repulsive force is stronger.
But there are four of them, as well.
So this turns out to be a larger repulsive force.
So the conclusion is that the radius of A would be expected to go up slightly as the coordination number increased, simply because there is a repulsive interaction between the coordinating Bs.
And that will tend to push them apart more strongly as the coordination goes up.
So the third column in the Shannon and Prewitt tables is coordination number.
And if you look at several ions, they're-- for example, barium has 6-coordination, 7-coordination.
8, 9, 10, and even 12.
And as you go down that list of radii, it gets larger as the coordination goes up.
And it's a relatively small effect that's on the order of 5 or so percent.
But still, when trying to calculate an interionic distance, you should use the radius that is appropriate for the coordination numbers.
The fourth column is spin.
And the spin stands for high spin or low spin.
And you will find that for the transition metal atoms-- for example, CO2 plus low spin or high spin, and note the rather different radii for these two configurations of the electron moments.
Yeah
Under coordination number, what's the difference between 4SQ and just Q?
You would guess that the lowest energy configuration for four-coordination would be tetrahedral, because that keeps the anions most widely separated.
There are cases where the coordination is plainer.
So the "SQ" stands for square, and that's unusual.
And you'll notice that that appears for things like silver, which has a considerable covalent character to it.
For gold, 3-plus.
That's one of the noble metals, again, that has a considerable covalent character to it.
So you'll notice that as the coordination number goes up, you get a different radius.
Now, there are then a set of numbers, and these are numbers that are based on different starting points.
The furthest column to the right, quote "IR" quote, this is based on Pauling's starting point.
And Pauling's starting point, if you go way down the list onto the second page, is that for 02 minus for oxygen in six-coordination, the standard radius from which everything else hinges is 1.40.
The Pauling radii had been so commonly used that Shannon and Prewitt decided to use 1.40, Pauling's radii.
So all of their numbers are compatible with the early Pauling radii, except they're based on thousands more crystal structure determinations.
There is another column, second from the right, called CR.
And that stands for Crystal Radii.
And there were two people that proposed that Pauling's radii really didn't give a feel for the proper sizes of the ions.
And they went on at great length to explain what "proper sizes" is.
And the only difference for the sizes that are based on, quote, "crystal radii" is that if you go all the way down to oxygen, the size of the oxygen ion in six-coordination is given by 1.26.
So all of the anions in the crystal radius column are smaller by 1.40 minus 1.26.
I can do that in my head.
That's 0.14 Angstroms.
Run down the list.
They're 1.4 Angstroms smaller for the anions.
And the cations are 0.14 Angstroms larger.
So there are two alternative sets of radii, one based on the traditional Pauling radii, 1.40 for oxygen in six-coordination, and the Fumi-Tosi radii, 1.26, which maybe give a better picture of what the sphere sizes actually, quote, "look like," unquote.
But note the tendencies.
There is a progressive change-- whoever's starting point you use, there's a progressive change in size with coordination number.
And let me give you, if I may, at this point, give you-- again, a little knowledge is a dangerous thing.
One of the structures which I worked on many years ago was a puzzle, and this was the prototype fast ion conductor silver iodide.
And nobody had been able to determine the structure for several reasons.
Nobody had ever made a single crystal of it, and all the diffraction data you had available for the structure were five or six intensities.
And there were about seven parameters necessary to fully describe the structure, so you were out of luck.
We learned how to make single crystals.
But before that-- no, he will go nameless.
He said, OK, let's see what's going on here.
Oh, we'll get the radius of silver.
We know that in silver iodide, the iodines are in body-centered cubic packing.
And we know what the lattice constant is, because we got that from powder diffraction data.
Huh, the silver ion is too big to fit in either the tetrahedral interstitial site or the octahedral interstitial site in silver iodide.
So he said, therefore, the structure has to distort, and it's really tetragonal.
But there are going to be domains.
a tetragonal domain in one orientation here, a tetragonal domain in another orientation here.
So he gets himself a random number generator and takes numbers one, two, and three and puts it on a cubic lattice to determine what the distortion will be.
Said, hey, I got a bunch of ones here and a bunch of threes here.
This crystal is built of domains.
And the reason the silver ions migrate rapidly is that a domain can pop from one mode of distortion to another, and that's essentially going to transport the silver ion from one end of the domain to another.
This guy got two papers out of it.
What he had done was he looked up the radius of silver in Pauling's tables.
And he didn't realize that all of Pauling's original radii were normalized to octahedral six-hole coordination, which makes the radius larger.
And that's why the silvers didn't fit in any of the interstices.
If you take the tetrahedral radius for silver, which is where it actually sits in the structure, it fits as snugly and nicely as you please in the tetrahedral interstitial site in the body-centered cubic arrangement.
So this guy made an idiot of himself because he didn't realize the basis on which these tables had been established.
But he's nameless, so I can call him names.
So anyway, again, a little knowledge can be a dangerous thing.
Moving on, in the few minutes that we have left to devote to crystal chemistry.
Pauling's radii, and even better still now, the Shannon-Prewitt radii, let you predict nearest-neighbor configurations-- coordination numbers with fair degree of confidence.
Pauling-- and this was one of the things that he received the Nobel Prize for-- proposed a set of five rules that are called Pauling's Rules for ionic structure.
The first thing he said was something that we've already stated, a coordination polyhedra of anions is formed about each cation, and the cation-anion distance is determined by the sum of the radii, and the coordination number is determined by radius ratio.
So there are three principles in here.
The two most important ones is that expressing faith in a set of radii to determine interionic distances and the fact that if there are radii that can be assigned to the atoms, you can predict a coordination number.
So that's Pauling's first rule.
Actually, Pauling's first rule was first stated by Goldschmidt.
But to have a complete listing of guiding principles, he included that there and didn't claim it was his own.
But then he does something that is rather interesting.
This is the only one of the rules that is quantitative.
If you had an ionic structure, and you could determine its unit cell, the structure should be electrically neutral if it's composed of ions that are charged.
So therefore, you would not make a structure in which all the cations were up in one corner of the cell, and all the anions were down in the opposite end of the cell.
That would be something that might be electrically neutral, but it would be a high-energy configuration.
So Pauling's second rule is a very cute way of expressing the fact that a structure should be atomistically electrically neutral, as well as microscopically.
And he did that in the following way.
He said, let us look at the coordination number the cation, A, and look at the coordination polyhedron.
And let's define as the bond strength, S, as the ratio of the cation charge plus Q divided by the coordination number of A.
And then he said that in a stable structure, if we look at the anion, the sum of all the bonds donated to-- he visualize the charge being donated to the anion.
The sum of all of the bonds from the same cations or from different cations, if that's the nature of the structure, should be equal to the charge of the anion.
So let me say "cation" here.
So it's a cute way of expressing local electroneutrality.
Actually, that's been superseded by an extension of this general principle.
And I may not have time to discuss it today, but I'll give you a handout next time.
It's not in the notes, either.
This is a relatively recent tool that most people are not familiar with, so I'll return to it briefly next time.
The next rule says, OK, we know how to determine the coordination numbers.
To describe the structure, we should be able to say how these coordination polyhedra fit together.
So looking at the coordination numbers-- well, let me do a three-dimensional case, because that's the realistic situation.
Let's suppose we have coordination number eight, which we often do.
And besides, that's easy to draw.
So he says that in a stable structure, the coordination polyhedra tend to share corners.
And I might say also that they tend to have the line between cation, anion, cation to be a straight line, as opposed to sharing edges.
So for eight-coordination, this might be a pair of cubic coordination polyhedra sharing edges.
And especially corner sharing is more favorable than sharing faces.
It's a very specific statement on how the coordination groups whose geometry is determined by size should fit together in a structure.
And it has a very simple basis, an almost trivial basis.
As the polyhedra of the given size share corners and then edges and then faces, the cations inside of these polyhedra are progressively getting closer and closer together.
And clearly, that increases the repulsive energy.
So the cations are most widely separated if not faces, not edges, but corners tend to be shared.
Puckered a little bit.
Maybe this is not exactly a 180-degree bond angle.
But corner sharing is still favorable.
And rule D is essentially an appendix to that, and it says, and how, if the charge of the cation is high and the coordination number is low, the higher the charge of the cations, the stronger that repulsive interaction.
The smaller the coordination number of the cation, the closer together they become in, let's say, face sharing for tubes as opposed to face sharing for tetrahedra.
The lower the coordination number, the closer together the cations come, regardless of the precise shape of the coordination polyhedra.
So rule D says, and how for high charge and low coordination number.
The last rule, sometimes called the "rule of parsimony," the "rule of stinginess," says that the number of different kinds of atoms in a structure tends to be small.
You can read in two different interpretations of the term "different kinds of cations," and both of them are right.
You could mean different in terms of different chemical species.
In other words, if you take a pot and dump half of the elements in the periodic table into the pot, melt it up, and then cool it down to crystallize the phase, you're not going to incorporate all of those ions into one crystal structure.
They have different sizes.
They have different coordination polyhedra.
The different coordination polyhedra have different sizes.
So to try to fit all of those things together in one structure is just going to cost you a lot of energy.
It's going to be much more efficient to form two or more phases, which have much lower energy, and then pay the price of an interface between them.
So have a chemically complex melt or solution, you're going to get different phases.
You want to incorporate everything into one structure.
The other interpretation of different is in terms of their coordination number and their coordination polyhedra.
If the size is such that a particular cation wants to have tetrahedral coordination, you would expect them all to have tetrahedral coordination and not have a mixture of cubes and tetrahedra and octahedra.
If it works for one coordination through the chemical species, it'll work for them all.
That rule tends to have some, but not terribly many, exceptions.
But there are cases where you find two different coordination numbers.
But very often, that's because of a covalent character.
So there's some discussion and elaboration of each of those rules.
I'm just about out of time, and therefore out of crystal chemistry.
Another entirely different approach to structure is to say that the anions are the big things.
And if the energy of the structure is lowered by taking ions of like charge and getting them as close as possible to the larger ions, say the anions of opposite charge, there is a strategy for making the arrangement.
And I think it's the same strategy that I use when I'm packing a suitcase for a trip.
Take the big things-- the books, the shoes-- and arrange them in the suitcase as densely as possible.
Then I take the little things-- the toothbrush, the socks, and so on-- and I stick them into the interstices that are left.
And this other view of examining structures and interpreting says the same thing.
Take the big things-- not the shoes but the anions-- and arrange them as densely as possible.
And this gets into the question of close packing of spheres.
Then take the little guys, the tiny little cations, and stuff them into the holes that are left in the close-packed arrays.
So another way of interpreting structures and predicting structures is to look at the sort of interstices that exist in a close-packed array.
Regardless of the stacking sequence of spheres, there are always two kinds of interstitial sites, one tetrahedral, one octahedral.
And there are two common, but not exclusive, types of packing arrangements, which you probably know.
There is the cubic close packed array.
There's a little diagram of close-packed sheets here.
And there are two sorts of interstices shown in the figure on the first page of the notes on packing considerations.
There's one set of triangles that points in one orientation that forms a hollow in which you can drop down a second layer of spheres, another triangle that is the closest triangle to the first one that points in the opposite direction.
Each represents a potential location for the next close-packed sheet, but those two different locations are too close together to both be used.
So you have to pick one or another.
And if you label those three different locations A, B, and C, any arrangement of spheres that does not place two letters in succession will give you a close-packed arrangement with exactly the same density.
So ABC AC, AC, ABC, AC, AC is a packing of close-packed spheres that is exactly the same density as AB, AB, which is hexagonal close packed, or ABC, ABC, ABC, which could be shown to be the face-centered cubic structure.
So that is another way of looking at structures.
And specifying the space filling in simple ionic structures is another way of looking at structures.
And at that point, I'll quit.
There are some examples of close-packed structures and derivatives thereof that form the last pictures.
And I'm going to stop and suck in air and mention as one last thought that only five of Pauling's rules are stated here.
There was a sixth Pauling's rule, and that's not commonly listed.
And I'll tell you what it is.
The sixth rule says, massive doses of vitamin C can cure the common cold.
Actually, that was something that Pauling pushed.
And interestingly, about three years before he passed away, Pauling came to MIT and gave a lecture in 6120.
And as you might expect of MIT undergraduates, after he finished his lecture, there were a few questions.
But there were about a dozen students who came up with bottles of vitamin C and asked Pauling to autograph them for them.
And he did so with good humor and thought he had perhaps won a few converts in the process.
So that's it.
Getting silly, so it's time for all of us to go home.
My watch says five after 2:00, so why don't we get started?
This is going to be a strange lecture.
I'm expecting some wise guy in the back to say, all your lectures are rather peculiar!
But I have a phone call from Europe coming in sometime around 2:15, 2:30.
Somebody's coming to visit us next week, and we're going to set up a seminar.
So I don't know exactly when it's going to come in, but I have to be in my office.
So what I will do is adjourn at quarter after, 20 past the hour.
And then so I don't keep you here sitting restively wondering when I'm going to come back, we'll adjourn until 3 o'clock.
So you seem to have been taken longer and longer breaks during intermission.
So I'll give you a full 3/4 of an hour so you can get it out of your systems.
And then, we'll take shorter breaks from here on in.
OK this is going to be pretty much our last lecture on symmetry.
And we will begin, at least in half of the next class on Tuesday, to begin talking about physical properties, and in particular tensor properties.
But I'd like to say a little bit about the derivation of space groups and take a look at how this information is tabulated for you by the kind folks who prepare the international tables.
And there are really very strict parallels between what we did in two dimensions.
And that's the reason why we did it so thoroughly and systematically, except that there are three dimensions.
And because we are taking 32 point groups and putting them into 14 lattices, there are a lot more combinations to be considered.
And there are a lot more of them that are unique.
Also, they're a little bit more difficult to visualize and depict.
So there are some new conventions in preparing a representation of three dimensional symmetries on a sheet of paper or page of a book which, of necessity, has to be two dimensional.
So we'll go over some of those conventions.
But the main reason for going a little bit further is that there is another surprise lurking in there for us as soon as we begin to combine translation in three dimensions with a symmetry element.
But let me go through in some detail the monoclinic space groups.
So we have at our disposal to decorate with symmetry elements two lattices.
And they are a primitive monoclinic lattice with two translations, a and b, that define an oblique net, and a third translation, c, at right angles to that net.
And then, either two choices for a double cell-- and I'll draw these in projection because I can do both of them with one diagram.
Either this is a and b and the extra lattice point is in the center of the cell halfway up from the base.
This would be body centered.
And that's represented by I for innenzentriert.
In German, that's German for body centered.
Or the alternative would be to pick a cell like this, in which case the extra lattice point gets caught in the center of one of the faces.
So this depiction here would be a side centered lattice.
And it could have the extra lattice point in the middle of the face out of which b comes.
And in that case, the symbol for the lattice is B.
Or alternatively, without changing any of the nature of the specialness of the lattice, the extra lattice point could be in the middle of the face out of which a comes.
And that would be an A lattice.
With the first, setting with the c axis unique, there is no c lattice because the oblique base of the cell would not give us anything new if it were centered.
All right, so there are those two lattices.
And then, there are three point groups.
And they were 2, m, and then a combination of a twofold axis perpendicular to a mirror plane.
So it looks as though in principle there are going to be 6 combinations.
In point of fact, there are a lot more because of the surprise that we have in store for us.
Before proceeding further, though, let me ask rhetorically the question.
Which lattice type would you pick as the standard one?
And here, there's a major decision that has to be made.
Should the labels on an axes give a unique symbol for the lattice?
In other words, if you have the double cell for a monoclinic crystal, should you define the cell to make it body centered always, or to make it A centered or B centered always?
Or should the labels be defined by the relative lengths of the axes?
So those are the two possibilities.
And there's no right or wrong way.
You pays your money and you makes your choice.
And on pragmatic grounds, this is the convention that's followed.
And it's purely a pragmatic decision because lattice constants have been determined for literally tens of thousands of materials.
Sometimes, the lattice constants are known and the space group is not.
But it would be terrible if the label that you applied to the axis could be C attached to the longest one, the shortest one, or the intermediate one.
It would be impossible to calculate any database for lattice constants displayed by particular materials.
So there's all this-- this is what the decision was made.
This is the decision that was made many years ago.
And for monoclinic crystals, the labeling of the axes is determined by the c axis being the unique axis.
That is, the axis that is along the twofold axis or perpendicular to the mirror plane.
And then, the labels b and c are defined by magnitude of b greater than the magnitude of a.
So I inherently drew this is in the proper fashion.
So the labels are applied to monoclinic crystals in that fashion.
The price you pay for that is that the symbol for the lattice for a monoclinic crystal that has the double cell could either be A, B, or I depending on the relative lengths of the axes.
If these were the two shortest, it would be a body centered lattice.
On the other hand, if this and this were the shortest pair, then it would be a side centered lattice.
So the symbol for the lattice type will bounce around depending on the dimensions of the translations.
So that's a complication in the space groups.
In the two dimensional plane group, that issue never came up.
We just took B greater than A for the rectangular and the oblique nets.
OK, so let me do quickly a couple of the monoclinic space groups.
Let's take a twofold axis and put it into a primitive monoclinic net.
And this is exactly the same as our plane group, P2, with the twofold axes extended parallel to the c translation.
So we would have twofold axes in all of these orientations.
And that's simply P2, little p in two dimensions, with the twofold axes extended normal to the plane of the plane group.
So this is p2.
This is capital P2.
And that is way we distinguish plane groups from space groups.
So the symbol for the lattice is a capital symbol for the space groups.
So I didn't have to use any new combinations theorems.
I simply say there's a third translation perpendicular to the net.
So everything protrudes out of the plane group into a third dimension.
Let me now combine a twofold axis this a and this b and this c with a lattice that is not the primitive lattice but one of the double cells.
And I'll take it, for convenience, to be the body centered flavor because it's easier to draw.
And the operation that we've added to the lattice, if this is a combination of a twofold axis, is the symbol A pi.
And I know what happens if I combine A pi with a translation that is perpendicular to the rotation axis.
I get a new rotation axis, B pi, that's halfway along the perpendicular part of the translation.
And that's where all of these additional twofold axes came in.
But if a lattice is a body centered lattice, there is now a translation that goes up to the centered lattice point.
So what's that going to be?
What is T followed by A pi followed by T, where T is 1/2 A plus 1/2 of B?
And that is a part that is perpendicular to the axis, and then a third component, c, which is parallel to the axis.
As with all these theorems, what you do is you draw it out once and for all and see what it turns out to be.
So let's do that.
Let's say that this is the translation that goes up to the point 1/2, 1/2, 1/2.
Here is the parallel part-- the perpendicular part, rather.
This is 1/2 of A plus 1/2 of B.
And then, we have a part that's parallel to the rotation operation, A pi.
And that is 1/2 of c. So let's just do it.
Here's my first object number one.
It's right handed.
I'll rotate 180 degrees to get a second one that's right handed.
And then, I will translate it up to here.
And here sits the third one.
It's also right handed.
How do I get from one to three?
It's not really clear how I do that.
And again, I'll draw it in projection because this is looking pretty messy in three dimensions.
So here's my first one.
I rotate 180 degrees to get a second one.
The chirality is all the same.
And then, I translate up to this point.
So if this one is at z and this one is at z, this one here will sit at z plus 1/2.
And this is the third one.
Anybody got any idea?
Can't be reflection.
They're of the same chirality.
It can't be translation because the orientation is different.
Yes, sir?
Then it has to be some form of rotation
Right, has to be.
But how do we get the object up to a different elevation?
Well, we've stumbled headlong over a new type of operation, just as we discovered the glide plane when we started putting mirror planes into a lattice in combination with translation.
The only way I can get from the first to the third is to rotate 180 degrees about exactly the same point that I would have before, namely at 1/2 of the part of the translation that is perpendicular to the operation.
But then before putting it down, I've got to take a second step.
I've got to slide it up by a translation component that is parallel to the axis about which I've rotated.
So to complete my theorem, A alpha followed by a translation that has a perpendicular part plus a parallel part is a new type of operation that involves rotating through 180 degrees.
And this is located at 1/2 of T perpendicular.
But it has a translation component which I'll call generically tau.
And tau is equal to the parallel part of the translation.
So if I separate out this new sort of operation in all of its grandeur, what it does is it takes a first object, rotates us 180 degrees, does not yet put it down, first slides up by the amount tau.
Doing the operation, again rotates it 180 degrees, doesn't yet put it down, slides it up by tau.
Doing it again moves us to another one up here.
So what we've generated is a helical spiral of objects about the central axis about which we're rotating.
And this is a new two step operation that's called a screw axis.
There is a persistent rumor that it derives its name from the effect that it has on 360 quizzes.
But that is just an ugly rumor.
There's nothing to that at all.
So this is a new two step operation that we'll call, in general, A alpha tau.
And what it will do is to generate a screw like distribution of objects.
For those of you who have come in late, since you've been demonstrating a predilection towards longer and longer breaks, we're going to adjourn now and have a 40 minute break so you can get it out of your system.
And we will resume at exactly 3 o'clock sharp.
And we will launch into an elucidation of the nature of screw axes.
And again, for those of you who were not here at the very start of class, the reason is that I have a phone call that had to come in about this time of day and no other time.
So rather than keep you shifting, wondering whether I'm coming back, we'll start at three when I should be done with my business.
OK, so that's it.
You missed all the important discussion of what's going to be on the next quiz, and nobody will tell you because they want to keep class average down.
All right, so sorry for the longer than usual interlude, but we'll start again promptly at 3:00.
Let the minutes of the proceeding show that I re-entered the room at 3:00 and 12 seconds, true to my word.
OK, I wanted to give you one example of a screw axis that you're probably familiar with in everyday life.
That was a telephone pole that I was very familiar with when I was a little kid because we used to love to wait until it got dark and our parents couldn't see us.
And then, we'd climb up the thing.
And wow, you could see all over the whole neighborhood.
I'm surprised nobody said, what?
They had electricity when you were a little kid?
Yes, they did.
But now it's all underground.
So kids, I don't know what they do for recreation these days.
OK, let me pursue this question of screw axes.
We've seen what a 2 sub 1 screw access looks like.
I just erased it.
But it consists of objects of the same chirality extending left and right on either side of the principal axis.
The symbol for a twofold axis is this.
The symbol for a twofold screw axis is the symbol for a twofold access with alternate sides extended like a propeller, a very descriptive symbol.
The thing is rotating around.
And you can think of these as the little ribbons that you used to have on the handle bars of your bike when you were a little kid.
And as you pedaled along, these things fluttered out behind, and it was really cool.
OK, let's move on to the next family of screw axes.
And let me look at an operation A2 pi over 3 with a translation component, tau.
And these things are going to get awfully cumbersome to draw.
So let me use a device-- I wish I could claim credit for it.
But actually, a student in my class one year said, hey, I've got a really cool way to draw these things for you.
Let's imagine that this is a screw axis.
We put a cylinder of paper around the screw.
And then, let's divide up the surface of the paper in terms of end segments.
So if this were a threefold screw axis, these would be 120 degrees segments.
And then, let's draw horizontal lines on the sides of the cylinder.
Then, if we want to see how the screw axis reproduces a pattern from a given motif, you just fill in these boxes.
And then when you're all done, cut the cylinder and you can draw the pattern quite nicely on a two dimensional surface.
So this is what I call the unrolled cylinder device.
Somebody very imaginative invented it.
His name, unfortunately, is lost to history.
But I didn't want to claim credit for it myself.
So what is the pattern of a threefold screw axis going to be?
Let's let the difference between boxes be the translation component, tau.
And so what we will do is take an initial motif, rotate it 120 degrees, slide it up by tau relative to the axis about which we're rotating.
Performing the operation again would involve rotating 120 degrees, sliding up by tau.
Doing it yet get again brings us back down full circle.
So this would be the translational periodicity along the direction of the screw axis.
And this would be the value of tau equal to 1/3 of the translation.
So that is what a threefold screw access would look like.
And the pattern obviously, if we would keep repeating it, would do something like this-- so perfectly interpretable, easy to draw.
But things are actually more interesting and more complicated than this.
We are stating two things about this operation.
We're specifying a translation.
But then the other thing that we're specifying is the translation component tau.
And why should we be constrained to say that tau can only be one nth of the translation?
If we do the operation n times, then doing the operation n times would give us a total translation of n tau.
But there's no reason why that has to be one translation.
Why not two?
Why not three?
Why not four?
So the only real constraint is that n tau has to be some integer, m, times the translation that is parallel to the screw axis.
And this means that the value of tau is not just equal to 1 nth of-- restricted to 1 nth-- of t. Tau can be m/n times T. And that is perfectly compatible.
You do the operation n times.
You're going to be directly above where you started.
That's going to be a translation vector.
But why not two translations, or three translations, or four translations?
So there are infinitely more screw axes than just the n subscript something translations.
So let's look at some of the possibilities.
For a threefold screw axis, tau could be equal to 0T Tau could be equal to 1/3 of T. Tau could be equal to 2/3 of a translation.
Tau could be equal to 3/3 of the translation, 4/3, 5/3, and so on, on and on and on.
We could fill the whole board with possible screw axes having different values of tau.
Now, let me convince you, hopefully easily, that if tau is equal to 3/3 t, tau would be an integral number of translations-- in this case, one.
Down here, 6/3 t would be two translations.
We're already going to have those operations in the pattern when tau was equal to 0T.
So let me show you what I mean if it's not clear.
Here's a trio of objects on either side of the rotation part of the operation.
So what I'm saying is tau would be equal to two translations.
That would, for some perverse reason, defining this as the pitch of the screw.
Is it clear that that is going to be an operation that I already have when I say there's a threefold axis and a translation, T, parallel to that threefold axis?
That's going to give all sorts of screw operations.
But they are going to be integral multiples of the translation that's parallel to the axis.
So the rule is that we can always subtract off one translation from any of these definitions of tau and reduce it to something smaller.
So we can always define tau less than or equal to T without any change or artificial restrictedness on the nature of the pattern.
So that says that for a threefold screw, for a threefold screw axis, there are only three that we should consider.
Tau equal to 0, and that's a pure threefold rotation axis.
tau could be equal to 1/3 of a translation.
And now, we introduce the notation used to designate screws.
If n tau was equal to mT, the symbol for the screw axis is n, the rank of the axis, with m as a subscript.
So the pattern that we just drew here would be designated 3 subscript 1.
And that's says automatically that the value of tau was equal to 1/3 of the translation that's parallel to the axis.
So the only other ones we have to consider besides 3 and 3 sub 1 is 3 sub 2, where tau would be equal to 2/3 of T. Let's see what that looks like by quickly drawing it out again using this dandy little unrolled cylinder device.
Let's let this be my first motif.
And I'll take this as the value of tau, two boxes up.
So I rotate, slide up by tau, rotate, slide up by tau, rotate, slide up by tau, and I'm back up here.
And here is my translation.
tau is equal to 2/3 of a translation.
And my translation, then, should be equal to 3/2 times tau.
But this is supposed to be the translation.
And there's nothing in this box.
OK, this introduces another very important aspect of the patterns produced by screw axes.
And it's sufficiently important we must use first the basic screw operation, A alpha tau and the translation in terms of which we have defined tau.
So use the spiral that you have defined by stating A alpha tau is the basic operation.
But then, don't quit.
You're saying another thing, that tau is a certain fraction of a translation.
And you have to use that translation to generate additional objects in the pattern.
So in this pattern that we've drawn here, this is supposed to be a translation.
So I have to take this one and slide it up by T. I have to take this one and slide it up by T. I have to take this one and slide it down by T. So I'm using a different kind of shading to indicate the ones I produced by A alpha tau and the ones that I produced from that helix by using the translation, T. So this is the final pattern.
It has a basic screw operation that's equal to 2/3 of the translation.
And it has a translation that's 3/2 of tau.
So everything's fine.
Yeah?
For the very top one where you have it colored in, [?
should that stay ?]
outside the box?
Yeah, well, it's supposed to be another row
Shouldn't that be a box lower [INAUDIBLE]?
Yes, it should indeed.
Thank you.
When your nose is right in the middle of the thing, sometimes you don't notice that as readily as somebody in the audience.
OK, so that's the pattern of 3 sub 2.
And now, if we realize that we must use both the basic operation, A alpha tau, and translation to fill things in, it's clear that if we try to have tau greater than the translation, the translations, when we apply them, would give us a screw that had the possibility of being redefined in terms of a shorter tau.
OK, one thing that is apparent if you look at this pattern is that 3 sub 2 produces the same sort of helix but in an enantiomorphic sense.
This one is a spiral that goes this way.
And this one increases when we rotate in a clockwise fashion.
So are they distinct?
Yeah, they really are because you can have both the left handed spiral and the right handed spiral together in the same space group.
I don't remember whether that's true for the threefold screw axes or not.
I'm not sure.
I have to check that.
Yes?
So really [INAUDIBLE] enantiomorphic because they're all left handed.
So it's just minus 1, isn't it?
Good for you.
I was about to make that point.
The sense of the spiral is enantiomorphic in the sense that this one is a right handed spiral.
This one is a left handed spiral.
But they're not truly enantiomorphic patterns because if we start with a right handed motif, this one also has a right handed motif.
So I should qualify that statement, which I was about to do.
It's the sense of the spiral which is enantiomorphic.
This is not to say that one has motifs of one chirality and the other one has to have the opposite chirality
Couldn't you just [INAUDIBLE] being minus 1?
Just [INAUDIBLE] minus 1?
Yeah, I could do that, I could do that.
And remember that we can add or subtract a translation at will from tau.
And if I took a full translation and subtracted it from 3 sub 1, I would get-- I get tau equal to minus 2.
Nope, doesn't work that way
[INAUDIBLE]?
Yeah, it's the same tau but in the opposite sense, yeah.
OK, so what comes out of this is that there are three kinds of axes that involve a 120 degree rotation.
Three, which we can view as 3 sub 0, 3 sub 1, 3 sub 2.
The symbols that are used for them now, we know and love the triangle that represents the locus of a threefold axis.
Four 3 sub 1 and 3 sub 2, what one does is to extend the edges of the triangle.
There's the streamers on the bike handle again.
And if you look down on the pattern and use a right handed spiral, then you extend the streamers, the edges of the triangle, that goes down into the board this way.
And 3 sub 2 would-- if it was in the same pattern-- you would indicate by extending the opposite pair of the opposite set of edges
So would they be different space groups?
[INAUDIBLE]?
Yes, there is a P3 sub 1.
And there is a P3 sub 2.
And they are distinct space groups
So how can you distinguish [INAUDIBLE]?
You would find out where the atoms sit.
And in one case, the spiral would occur in a right handed fashion.
And in the other one, from a left handed fashion.
You can determine the position of the atoms unambiguously.
OK, let's do one that's more interesting where the rank of the rotation part of the operation is higher.
And let's look at fourfold screw axes.
So here, we would consider tau equal 0, tau equals 1/4 of a translation, tau equal to 2/4 of a translation, and tau equal to 3/4 of a translation.
So let's take our cylinder that was formerly wrapped around the rotation access, straighten it out.
And I won't bother to draw the pattern for a fourfold access.
But for a 4 sub1 1 screw axis, this would be tau.
And that would be one, two, three, one quarter of a translation that's parallel to tau.
Or conversely, so T is equal to 4 tau.
Or conversely, how is equal to one quarter of the translation.
So start with a first motif, rotate 90 degrees, slide up by one quarter of the translation.
Rotate and slide, rotate and slide.
Now I've come full circle.
Rotate and slide.
So this, then, is the pattern for the screw axis that I'll label as 4 sub 1 by analogy to what I've done with threefold screw axes.
Over here, let me immediately jump to 4 sub 3.
I'm going to have to be a little more stingy with the spacing of my boxes or I'm going to run out of room.
So if this is 4 sub 3, turns out that tau should be equal to 3/4 of a translation.
And conversely, the translation should equal to 4/3 of tau.
So that's the situation that I would obtain if one, two, three boxes gives me the length of tau and four boxes give me the length of the translation.
OK, so again I'll use different sorts of shading to indicate which way I have gotten this.
I would rotate 90 degrees.
I would jump up one, two, three boxes.
So here's the next one.
Rotate, one, two, three boxes up brings me to here.
Rotate, one, two, three boxes brings me up to here.
Rotating again, sliding up three boxes brings me to here.
OK, so there's the basic helix.
And how is indeed equal to 4/3 of the translation.
But I can't quit yet.
I've used the basic operation, A alpha tau.
Now, I've got to fill in with translation.
And according to my rule, the translation should be equal to 4/3 of-- I'm sorry, this should be the translation is 4/3 of tau.
So this should be the translation running up one, two, three, four boxes.
So that's-- I'll use a different shading here.
That would fill in one here.
One, two, three, four boxes up brings me to here.
One, two, three, four boxes up would bring me to here.
Go down by four-- one, two, three, four-- brings me to here.
And let me use still a different shading here.
One, two, three, four brings me down to here.
Filling in quickly, one, two, three, four.
Another one would sit here.
One, two, three, four, another one would sit here.
One, two, three, four, another one would sit here.
And you can see what is happening here, that 4 sub 3 looks exactly like 4 sub 1 but in the opposite sense.
So it's the same relation as between 3 sub 1 and 3 sub 2.
What do we use as symbols?
This is a 4 sub 1 screw.
And again, if we look down on the pattern from the top, use a right handed rule and extend every edge of the square, that would be the symbol for 4 sub 1.
4 sub 3 is the same sort of spiral, but in the opposite direction.
So we will, with a right handed rule and going between neighboring closest motifs, we would extend this, extend this, and extend this.
So that's the symbol for 4 sub 3.
With that, we come back over to here.
And that is everything except 4 sub 2.
And again, I'll mark up a cylinder that's been split into quadrants, draw reference horizontal lines.
This is 4 sub 2.
And we define this as tau.
Then, the translation, this should be equal to 1/4 of the translation parallel to the axis.
And conversely, the translation should be four halves.
So let's draw the pattern.
This was the first one.
Rotate, slide up by tau.
This is the next one.
Rotate and slide, rotate and slide.
Rotate and slide begin, rotate and slide.
So this turns out to be the nature of the helix that is produced by A pi over 2 tau.
But this is not yet a pattern that has the translational periodicity that I have claimed.
That's supposed to be the translation, twice tau, up to here.
So I would have two slide this one up to here.
I'd have to slide this one up to here.
I'd have to slide this one up to here.
And something's wrong here-- one, two, three, four.
I think I went too far up for one of them.
This one was one, two, three.
This one, I went up too far.
I skipped one.
What did I do wrong?
You did the rotation wrong
Hm?
The two [INAUDIBLE]
OK, let me do this all over again.
Again, I can't see standing right on top of it.
It's not a very good attempt to make it look easy.
Let's split this up into four quadrants.
If this one comes out quite differently, I'll make sure I have it right.
So this is my translation, one, two, three.
This is the translation.
And that should be equal to four halves of tau.
So this, then, is my value of tau.
It should go up two boxes to here.
And that should be equal to 1/2 of T. So I want to rotate, slide up two boxes, rotate, slide up two boxes, rotate, slide up two boxes, rotate, slide up two boxes, rotate, and slide up two boxes.
And now, I have to fill in with translation.
So I will take this one and go up one, two, three, four translations.
That's what I did wrong.
I'll take this one and go down one, two, three, four translations.
Take this one and go up one, two, three, four translations.
Take this one and go one, two, three, four translations.
Take this one and go one, two, three, four translations.
And now, lo and behold, what I have is one, two, three, four.
Curious sort of thing-- I have two motifs at every level.
And they are 180 degrees apart.
So a 4 sub 2 screw axis produces a pattern that looks like this.
Pair of motifs like this 180 degrees apart, pair of motifs skewed by 90 degrees to that pair, another pair translationally equivalent to the first.
But what is the chirality of this spiral?
3 sub 1 was right handed.
4 sub 1 was right handed.
4 sub 3 was left handed.
4 sub 2 was right in the middle.
Is that right handed or left handed?
The answer is it's both.
It's both.
There's one spiral that goes up this way.
And there's another spiral that goes up in the opposite sense.
So it's both.
It's a left handed spiral and a right handed spiral together in the same pattern.
And the pattern for this screw axis, the symbol for this screw axis if it occurs in a pattern, is every other edge of the square extended.
Which do you extend?
Well, it really doesn't matter because it's both left handed and right handed simultaneously.
So you can use either one.
Yes, a couple questions?
Can you explain why you explain you did that shading?
I just wanted to differentiate the ones that I used.
They're all the same.
And where they come from doesn't affect what the basic pattern looks like.
But I tried to make clear which ones I got by using the operation A alpha tau.
And for 4 sub 2, which I just did twice, once correctly and once incorrectly, these with shading were produced by the operation of rotating and sliding up by 1/2 of the translation.
The other ones, the ones for which I used this shading, I filled in by using what the translation was, namely twice tau.
So it's just a way of keeping track of what operations I used to produce everything that's in the pattern.
And for 4 sub 3, I had to use three kinds of shading.
Yes, sir?
Should I [INAUDIBLE] the fact that this was actually two superimposed twofold axis separated by 1/2 T?
Yes, you could read that into it.
And you would be very observant because a 4 sub 2 screw axis contains a twofold rotation as a subgroup
[INAUDIBLE]
And since you were shrewd enough to deduce this-- we ask the question with glide planes when we're dealing with two dimensional plane groups.
Is a glide plane a candidate location for a special position?
In other words, if I move the atom directly onto a glide plane, is there coalescence?
The answer was no.
Suppose I asked the same question now of a screw axis.
Is a screw axis a candidate location for a special position?
The answer is sometimes yes, sometimes no
Well, only if you have [INAUDIBLE]
Yeah, so actually if I move the atom, the first atom, onto the locus of the rotation part of the operation, I get pairwise coalescence.
And instead of getting all of these, I'll have just a string of single atoms separated by half a translation, half as many.
So that's a special position.
I don't get the full number out when I throw one into the space group.
Now, there are an infinite number of screw axes.
My old friend, the saguaro cactus, which has a 19 to maybe 23-fold rotational component to its symmetry, does not, if I look at it carefully, have the same thing on every one of these ribs because the tufts of spines spiral up like this.
So a saguaro cactus, if I take the spines into account, doesn't have 22-fold rotational symmetry.
It has some sort of 22-fold screw axis.
And I've never been determined enough to risk puncturing my finger by going down around a saguaro and seeing if there's some other tuft at the same level and whether it's a 22 subscript 3 or 22 subscript 2 screw axis.
But a saguaro cactus does have a screw axis as its symmetry element.
OK, let us generalize on the basis of what we've done so far.
For any screw axis whatsoever, crystallographic or not, we will have an n sub 1 screw, an n sub 2 screw, where tau is equal to 1 nth of a translation, tau equal to 2 nths of a translation.
And we will go all the way up n sub n minus 1, n sub n minus 2.
And always, regardless of n, the ones at either end tend to be spirals in an enantiomorphic sense.
And n sub 2 and an n sub n minus 2 are spirals.
If you end up with an odd one in the middle, this has no chirality to the spiral.
There's a left handed one and a right handed one.
On the other hand, for something like two and four and six, the ones at opposite ends of the series are enantiomorphs in the sense of the rotation.
For any six-fold screw axis-- and I will pass out the nature of the pattern without taking time to draw it-- 6 sub 1 and 6 sub 5 are enantiomorphous.
6 sub 2 and 6 sub 4 are enantiomorphous.
And 6 sub 3 stands alone.
6 sub 3 consists of a pattern of three objects on a triangle pointing in one sense, three on a triangle pointing in the opposite sense.
This is the value of tau.
And that is equal to 3/6 of the translation.
So no left handed or right handed sense.
6 sub 2 consist of pairs of objects separated by 60 degrees.
6 sub 4, exactly the same pairs, but they go in the opposite sense.
6 sub 1 is just a single spiral going up with atoms at intervals of 1/6 of the translation.
6 sub 5, a spiral in the opposite sense.
OK, with that, let me pass out a sheet that has patterns done in a decent fashion for all of the crystallographic screw axes.
Bear in mind that there are fascinating, interesting, and intriguing patterns for non-crystallographic screw axes.
And let me say a few words about the symbol used in the space group for glide planes.
Any questions on this, by the way?
I don't want to have beat it into the ground.
Yes?
I have a question on symbols.
How do you know which way the arrows point on the triangle or the squares?
Oh, here?
Yeah
OK.
Probably the best thing to do is to look at the patterns that I just passed around if you got one yet.
If you look at 3 sub 1 and look down on it from the top and indicate the sense of rotation that gets you from the one above to the one that's directly below, your hand will curl around in a clockwise fashion.
And the little tails on the symbol for the threefold axis trail out behind the way in which you're rotating.
If you look at 3 sub 2 and look down on it, you would have to, using your right hand, rotate in a counterclockwise sense.
And therefore, the streamers go out again in the opposite direction.
Same for 4 sub 1.
There, every edge of the square is enclosed.
If you look at how you have to rotate to get from the one on top to the one immediately below using your right hand, again, you have to rotate in a clockwise sense.
And therefore, the edges that are elongated trail out in the opposite direction.
And finally, drawn in a decent fashion are 6 sub 1, 6 sub 5, which are spirals in the opposite sense.
6 sub 2, 6 sub 4, pairs of things at each level.
And the level is 1/3 of the translation.
6 sub 3, triangles of objects at intervals that are equal to 1/2 of T, 3/6 of T. OK, does that answer?
That is really a convention of some higher order.
But nevertheless, it is a convention and it is consistent.
Other questions about screw axes?
Yeah?
So in nature, is there a difference between the right handed screw [INAUDIBLE]?
That's like asking is there something like a Coriolis effect in crystals.
You know the Coriolis effect that if you're in the southern hemisphere, the water gurgles down the drain one direction.
If you're in the northern hemisphere, the opposite direction?
There was actually a very famous scientist, what was his name?
I think it was Serret.
Serret, so famous I can't think of his name.
I remembers his name, Serret He developed a very famous theorem in kinetics, and that was thermal migration, migration driven not by a chemical potential gradient but by a temperature gradient.
And it's very difficult to measure.
And only recently have people been testing that theory and making measurements.
He asked this very same question.
And he said, I think it was potassium chlorate that he looked at which has no mirror plane or inversion in it.
So it occurs in left handed and right handed forms.
He asked, is there something like a Coriolis effect that would give you more right handed crystals in the northern hemisphere and more left handed crystals in the southern hemisphere?
So what he'd do, he drew a lot of crystals.
And he started counting.
And he counted, and he counted, and counted.
And he couldn't tell if there was a difference.
And counted, counted, counted.
And still, after lots and lots of counting, still could not get a result that was statistically significant.
And he went nuts.
Sorry to tell you this.
So I think it remains to be seen whether there is indeed a Coriolis effect in crystal shapes.
And that's true.
It's tragic.
We're all laughing at the poor guy.
But he actually just couldn't stand it.
He counted so many crystals, poof!
That's a true, unfortunate story.
No, I don't think there is any difference between left handed crystals or right handed crystals.
Remember my story about sugar and its chirality and the little bugs that can only eat one chirality.
Other questions of the less frivolous nature?
That was not frivolous.
That was a good question.
OK, well let's say a little bit about glide planes and how they appear in space groups.
Things were very easy in two dimensions because the glide plane was a glide line.
And everything had to be confined to a two dimensional space.
So we used a solid line to indicate a mirror plane, the locus of the operation, sigma.
And we use a dashed line to indicate the locus of an operation, sigma tau.
And tau was in the direction of the dashes.
Well, I've given it all away with the little diagram that I passed around.
Things are much more complicated in three dimensions.
If this is the locus of a glide plane and this is the direction of tau in a space group, we could be looking at that glide plane in a total of four different ways.
We could be looking at the glide plane from the side, edge on, and perpendicular to tau.
We could be looking at the glide plane edge on and along the direction of tau.
Or we could be between these two orientations and looking at the glide plane edge on but in between normal to tau and parallel to tau.
And finally, we could look at the glide plane down from the top.
If we view the glide plane from above, for example suppose we have a monoclinic crystal and we put a glide plane in the base of the cell which has tau in this direction, the way you indicate that is with a little chevron off to the side.
And if this is the direction of tau, you put a little barb on the chevron that indicates the direction of tau.
So the three possibilities for a monoclinic crystal is that how could be this way, in which case you use a chevron like this.
Or how could be in this direction.
And then, you use a chevron with a barb attached like this.
Or how could be a diagonal glide, in which case the atoms would be up and opposite chirality and down and back up again.
In that case, you add a little barb in between the two directions to indicate the direction of tau.
We put some labels on this, this a and this b.
This situation would be a tau that is equal to 1/2 of b.
In this case, how would be equal to 1/2 of a.
In this case, how would be equal to 1/2 of a plus b.
This is designated as a b glide.
And rather than having the symbol n appear in the symbol for the space group, a b would appear.
This is an a glide.
And the a would appear in the symbol.
And this is a diagonal glide.
And that, for reasons that I have never found anyone able to explain to me, is called an n glide.
I know of no language in which the word for diagonal begins with an n. But that's what it's called.
Now, those same glides can be viewed from directions that are parallel to the plane of the glide plane.
Looking normal to tau is exactly the same situation we had in plane groups.
So one uses the dash.
If you're looking along the glide plane end on but in the direction of tau, you use a dotted line.
It's easy to keep straight which is which.
If there's a little arrow in there and you look at it from the side, you see a line segment.
If you look at it from the end, you see a dot.
So it's easy to remember this convention.
If you're neither perpendicular to nor parallel to the glide plane, you just mix the two symbols.
So you use a dashed dot line.
So those are the symbols for glide.
Glide plane is something like a piece of wood.
It's got grain to it.
The direction of the grain is the direction of tau, if you'd like.
All right, in the hand out, I give you some examples of the way in which a pattern of objects that has glide in it would be interpreted in terms of a geometric symbol.
At the bottom right of the page, the left hand diagram has atoms related by a glide plane in the base of the cell where tau is running left to right.
And so you have a chevron with an arrow on the right hand branch.
If you have atoms alternately left handed, right handed at a level and 1/2 plus that elevation, then you're looking at a glide plane edge on and down along the direction of tau.
So you would indicate the locus of that glide plane by a series of dots.
What other type of glide plane is possible, not for monoclinic, but for lattices that have a centered lattice point in the middle of the cell?
So this takes a special type of Bravais lattice.
And this would be a lattice, for example, that had lattice points at the corners of the cell and another lattice point in the middle of the cell.
One face of the cell is centered.
Then, you have not only translations a, b, and a plus b, which can serve as directions for glide.
But you have another translation that is not present in the primitive lattice.
And this is 1/2 of a plus 1/2 of b.
And if that's a translation, you could have a glide plane parallel to the base of the cell and have tau equal to 1/2 of 1/2 of a plus b, namely have this little vector in here be tau.
And the symbol for that kind of glide is d. And I do know what that stands for.
That stands for a diamond glide because diamond, one of the very simple structures for an element or any compound and one of the early structures to be determined experimentally, does have a d glide in it.
So the name of the compound, diamond, gave its name to the type of glide plane.
OK, I've got still 10 more minutes.
What I would like to do next is to look at some of the other monoclinic space groups and also examine the way in which this information is presented for you in the international tables.
So I've got a big, fat pack of material that shows you all of the monoclinic space group, that is the space groups derived from point group two, the space groups with point group m, and the space groups with point group 2/m.
And we've done a few of these.
But what we haven't looked at is the Shcoenflies notation for the result or the way in which this information is presented in the international tables.
To introduce you to these gradually, this is the monoclinic space group-- these are the monoclinic space groups as presented in the old international tables for x-ray crystallography.
A little bit later, I will pass out to you a few higher symmetry space groups that are represented in both the old international tables and the new international tables.
And as I indicated somewhat disparagingly earlier on, there is so much information in the new version that they're very, very hard to use.
But OK, if you open the hand out, you find space group P2.
So that's a twofold axis added to a primitive monoclinic axis.
You see on the left hand page this situation, this ridiculous situation, that exists only for monoclinic crystals the first setting, where the axes are a on the left hand side running down, and b on the top running from left to right, and c coming straight up out of the page.
And the second setting, where the unique axis is b, and that means if you're going to draw the space group with c coming up, a down, and b to the right, b now is the direction of the twofold axis.
So the diagram looks completely different.
But it's the same space group except that the first setting is tilted on its side so that a comes down, b is from left to right, and c, which is now not a direction of the twofold axis, comes out.
So there's some jargon here, that geometric jargon.
The atoms occur at different elevations.
So in these two cases, the atoms occur only at elevations plus z or minus z.
And you see next to the little circles that represent the symmetry equivalent positions for the general position a plus.
And that means elevation plus z.
And for the first setting, all the atoms occur at an elevation plus z for the general position.
For the second setting in the right hand diagram, you're looking at the twofold axis from the side.
And therefore, one atom is at plus z.
The one that's related by symmetry gets rotated down to minus z.
So you see a little plus next to one atom, a little minus next to the next to it.
We need a symbol for a view down along the locus of a twofold axis.
And that's the little pointed oval that we became familiar with with the plane groups.
When you're looking at the twofold axis from the side, how are you going to indicate that?
Well, the convention is to use an arrow.
And that's what you see on the right hand side for the second setting of P2.
If this were not a twofold axis but a 2 sub 1 screw axis, it would be a one-sided barb.
So looking down at these two axes with rotations of 180 degrees, this is the symbol for two viewed from the side.
And a one-sided barb would be the symbol for a 2 sub 1 screw axis.
Mercifully, these are the only screw axes that need to be depicted in a view from the side.
So there's no standard symbol for a sixfold access or a fourfold axis looked at from the side.
In boldface, in the outer corner of these pages, you see the international symbol, which is what we used for the plane groups, the symbol for the lattice type, primitive, except now we use uppercase symbols for the lattice type to distinguish the space groups from the plane groups.
Underneath it, you see a C2 superscript 1.
This is the Schoenflies symbol.
You'll recognize C2 as the Schoenflies symbol for a twofold axis.
Schoenflies' symbol for the space group was to add a superscript, namely the first one that old [?
Artur ?]
got, the second one that [?
Artur ?]
got from this point group, the third one he got from the point group, and so on.
So it's pretty much an arbitrary order, except he starts with, as you'll see, a twofold access, then replaces the twofold axis with a screw access.
So if you turn to the next page, you see symbols for not twofold axes, but 2 sub 1 screw axes along the edges of the cell.
You see now a different sort of symbol alongside the atoms.
There's one at plus z.
That's the representative one at x, y, z.
If you repeat it by a screw rotation, you rotate it 180 degrees and slide it up by 1/2 of [?
c.
?]
So you see the symbol 1/2 plus.
So plus is plus z.
1/2 plus is 1/2 plus z.
Over to the second settings, trying not to sneer too vigorously, you'll see pointed barbs.
That's the symbol for a twofold axis viewed from the side.
And then, one of them is plus.
The one that's a little long parallel to the 2 sub 1 screw axis goes to the other side and goes from plus z down to the minus z.
Let me ask you a question which people usually don't think about.
If you look at the first setting and ask, what are the lengths of the translations?
That's a.
And that's so many angstroms.
And that's b.
That's so many angstroms.
If I ask you that for the second setting, the answer is a little trickier.
What is the length of this translation?
That's b.
What is the length of this translation?
I'm sorry, that's c. And this translation is a, right?
Wrong.
That is no longer a.
And the reason for that is that when you have a cell with non-orthogonal angles in it, let me indicate for the most general case for a triclinic crystal.
So this is a.
This is b.
And then the third translation normal to that plane is c. And if we have a structure with atoms in these locations and we want to project the structure along c, you do exactly that.
You don't plop the atoms down onto the base of the cell when you project it because if you did so, the atom that is up in the neighboring unit cell would come down to a different location.
And you would not have a pattern that was periodic based on a lattice.
You get that only if, when you're projecting in an oblique lattice, you project along the translation.
And then, and only then, can you end up with a pattern that looks like it has translational periodicity and really does.
So if a cell is oblique, you do not project just by plopping the atoms down on the base of the cell.
You project them parallel to the translation that extends up above the direction onto which you're projecting.
So if our cell in the second setting is monoclinic and this is a and this b and c comes up, if we look at that cell from the side as is done in the depiction on the right hand, then we let this be the direction of b.
So that is b.
But the thing that is at right angles to b is going to be a times the cosine of beta.
And it's not a itself.
So this is a right angle.
But this is not the lattice translation.
It is the lattice translation times a trigonometric function.
The triclinic case is a hairy beast.
To calculate the length of these axes and projection, it's clear what they are.
But to get these values involves alpha and beta and gamma.
And it's a very, very complicated function
So the angle between a and c in general is not 90 degrees in monoclinics?
The angle between depends on how you label your axes.
For monoclinic, if you do the first setting, then by definition this is a, and this is b, and this is c. So these are always by definition 90 degrees.
If you use the second setting, then this is the direction of b.
And this is the direction of a.
And this is the direction of c. That keeps the system right handed.
Then, this is a right angle and this is a general angle
But all you're doing is taking that top one and just [?
tilting it over ?]
No, I'm doing more than that because if there were a twofold axis, that would be the direction of the twofold axis.
Now, this is the direction of the twofold axis.
The other thing that happens is the symbols for the glides change.
So if you look at space group number seven, which is a case where the mirror plane has been replaced by a b glide in the first setting, and that means the glide is perpendicular to the twofold axis.
If the twofold axis is now the direction of b, the glide plane turns into a b glide.
But let's just thumb through them quickly.
And then, we're actually two minutes past my promised quitting hour.
First one, P2.
Then, replace the twofold access by a 2 sub 1 screw.
So that's Schoenflies symbol C2 superscript 2.
Then, do this with centered lattices.
The international table chooses to use a side centered cell.
So that's the third one you can get from a twofold axis.
So Schoenflies calls it C subscript 2 superscript 3.
And if you add that to a side centered lattice, you get screw axes interleaved between the twofold axes.
Put it on its side, it stays C2 superscript 3 in the Schoenflies notation.
But the side centered d become side centered c because it's b that comes out of the oblique net.
So the international symbol tells you exactly what you have.
The Schoenflies symbol is arbitrary.
It tells you the point group and it tells you the order in which Schoenflies derived them.
But that non-informative nature to the symbol has a blessing.
And that is it doesn't change with different labellings of a, b, and c. It doesn't change with a being shortest, b being shortest.
It just sits there and its dumb, uninformative fashion.
And that is nice.
So both of them have survived.
OK, so could we get another space group out of this by replacing the twofold axis by a 2 sub 1?
No, because we've already got 2 sub 1 screw axes in C2 superscript 3.
So then, Schoenflies moves on and begins to work with symmetry planes.
So number six puts a mirror plane in the primitive monoclinic lattice.
That becomes CS.
That's the Schoenflies symbol for point group m. And it's the first thing you can get.
Another piece of convention for displaying the representative pattern of atoms, in the first setting, the two atoms sit directly above one another.
So there are two new things here.
A vertical line through the atom indicates that there are two of them that are superimposed in projection.
And moreover, the little tadpole has appeared inside the frog eggs in the left hand part.
So this says on the right hand side of that split circle, the atom is at plus z.
On the left hand side with the comment to indicate an enantiomorph is 1 at minus z that sits directly below the first one.
And you'll notice in the diagram of the symmetry elements is a chevron working off the lower right hand corner of the cell.
And that's the symbol for a mirror plane that's being viewed directly from above.
And then I'll just go a couple more.
You could replace the mirror plane with a glide plane.
And that's a b glide.
Now, you can see the atoms going up at plus z, slide along by 1/2 z, and popped down to minus z, going down to an enantiomorph, so there's a comma inside the circle.
The chevron in the lower right hand corner of the diagram of symmetry elements now has acquired an arrow, a barb, that indicates the direction of tau.
The second setting, again, flops the thing on its side so that you're looking now down along the b glide.
And so you see a dotted line that indicates a glide plane viewed edge on in the direction of tau.
Then, you can put a mirror plane for number eight into a side centered b lattice.
And that gives you another space group.
And then, you can put a glide plane into the b lattice.
And that, interestingly, gives you a different space group.
Number eight has an alternating mirror plane with an axial glide.
Number nine has an alternating axial glide alternating with a diagonal glide-- two glide planes.
Not a mirror plane and a glide, but two different kinds of glide planes.
And then, you've used up all the tricks you can pull there.
So now, we take the last monoclinic point group, 2/m, drop it into a primitive lattice.
Replace the twofold axis with a 2 sub 1 screw axis.
That gives you P2 sub 1 over m. Then, leave the twofold axis alone and leave the plane alone and put 2/m in a side centered cell.
And then, replace the mirror plane by a glide.
That's number 13.
And then finally, the summa cum ultra, replace the twofold axis y a 2 sub 1 screw and replace the mirror plane with a glide, number 14.
And then, do the same thing in a centered lattice.
So we get a total of 13 different space groups from two lousy kinds of lattices and three possible point groups.
You get 13 different distinct space groups.
OK, that's a quick overview.
Notice that just as was the case for the plane groups, the tables go on to list the way in which atoms are related by symmetry and the number per cell and the site symmetry just as in the plane groups.
And the way a three dimensional structure is going to be described to you is exactly analogous to what we did with two dimensions.
Magnesium in position 3b, symmetry 6/m, oxygen in position 12d, symmetry 1.
And then, the tables give you the coordinates of all the symmetry S atoms.
OK, thank you for your patience.
That was a long stretch in one shot.
I'll say a little bit, very little, next Tuesday about the convention for orthorhombic space groups because there, no one direction is any more or less special than any other.
So the symbol for the space group changes all over the place when the axes take on different lengths, relative lengths.
But the symmetry is exactly the same.
So we'll see you--
All right, we've been slogging our way through derivation of the plane groups.
And I think I'll do a few more, because we'll stumble across some major tricks in deriving a subfamily of them.
But to not get lost in the forest because of all the trees, I have a set of notes.
They are handwritten because my secretary would resign if she had to fit in all these figures and subscripts and strange symbols.
So, they are as neat as I could make them.
Sorry to say that, in running them through the Xerox machine in an attempt to get everything on one sheet, some of the last lines got clipped.
So I'll run these through again and give you a copy that's minus those truncations.
All right.
What we've been doing so far, to have a brief reprise, was to take the symmetries, the 10 two-dimensional plane group symmetries.
And they were one-, two-, three-, four-, or sixfold axes, a mirror plane, 2mm, 3m, 4mm, 6mm.
So there are 10 of them.
And these are the so-called crystallographic point groups.
Crystallographic because we considered only those rotation axes that are compatible with a lattice.
And they are point groups because they leave at least one point in space, invariant-- stays there rigidly fixed.
And they are groups because the collection of operations that are present satisfies the postulates of the mathematical entity called a group.
We've then are in the process of taking these 10 symmetries and adding them to the 5 two-dimensional lattices.
The parallelogram net, the rectangular, the centered rectangular, and the square, and the hexagonal.
And clearly, we can't put each of those point groups in every one of the lattices.
For example, the lattice has to be square for either 4 or 4mm, so we would attempt to place only those two of the 10 point groups into a square net.
We've gone through quite a few of them.
I won't bother to draw the pattern of the symmetry elements.
But we put no symmetry at all in the general parallelogram net.
And that is plane group P1.
Maybe I will draw the figures, after all.
P2 was a twofold axis dropped into a net, which had to have no specialized shape, simply because a twofold [INAUDIBLE] axis requires nothing but a lattice row if one translation is combined with it.
The threefold axis could fit into an equilateral net.
And there's the threefold axis we added to a lattice point.
And we have two additional threefold axes in the centers of those 2 equilateral triangles, which each make up half the cell.
P4 was the square net.
We put a fourfold axis at the corner of the cell.
We've got another one in the middle.
Two folds in the middle of the edges because of the 180-degree rotation that is built into the fourfold axis.
And P6.
We put that into a hexagonal net.
We've got sixfold axes that we dropped in at the lattice point only, nothing else.
There's a 120-degree rotation at the lattice point, and that gives us the threefold axes that are present in P3, as well.
There's a 180-degree rotation contained in a sixfold axis, and that gives us the twofold axes in all of the locations of P2.
OK, I haven't drawn in any representative patterns, but let me remind you again.
The pattern that is characteristic of every one of the plane groups is just the pattern that the point group that you've placed at the lattice point would produce.
And that pattern is, in turn, hung at every lattice point of the two-dimensional cell.
So even though the huge number of symmetry elements that's present in the higher symmetry point groups is rather intimidating-- you say, wow, how would one draw a pattern for that-- the pattern is nothing more complicated than the pattern of the symmetry that you've placed at the lattice point.
And all of these other symmetry elements arise to express relations between the motifs that you've placed at the initial representative lattice point.
Deriving these, we used one theorem, which it's well to remind you of.
And that is that if I have a rotation operation A alpha, follow that by a translation that's perpendicular to the rotation axis, what I get is a new rotation operation is B alpha.
And it's located in a very specific location.
And that is at a location that's a distance x equal half the magnitude of the translation times the cotangent of alpha over 2.
So this combination term, if nothing else, reminds us that these combination theorems, as I call them, are not equations in symmetry elements.
They are equations in individual operations.
So, for example, if I combine the fourfold axis with the translation, the 180-degree operation that's present in a fourfold axis puts B pi here.
The 90-degree rotation puts B pi over 2 here.
And the 270-degree rotation-- which I can define just as well as A minus pi over 2-- puts the operation A minus 2 pi over 2 down here.
So it's not an equation in symmetry elements.
It's an equation in individual operations.
Then some peculiarities started to arise, which we perhaps might not think of.
If we put a mirror plane in a primitive rectangular net, that gave a group that we called Pm.
If we combine that with a translation, we need a theorem that says what happens if you combine a mirror reflection operation with a translation that's perpendicular to it.
We sketch that out, and once and for all could decide that it's going to be a new reflection operation that's located halfway along this perpendicular translation.
So this reflection operation sigma, combined with this translation, gave us a new reflection operation, sigma prime, halfway along the cell.
And, of course, we have this one hanging at a lattice point as well.
And then came the interesting one.
When we combined a mirror plane now with the centered rectangular net, we have all of the mirror planes that we have here, because this primitive rectangular lattice is a subgroup of the centered lattice.
Then the interesting thing happened when we combined the reflection operation sigma with this translation that went to the center of the cell.
And that gave us a transformation that was something we had not encountered before.
We took an object, reflected it in this plane, and then translated it down to a centered lattice point over here-- to give us one that sat here-- and then asked, how did I get from the first one, a right-handed one perhaps, to a second one that's a left-handed one, and then a third left-handed one that sits down here?
The answer is that we found there was no way we could specify getting from number 1 to number 3 in a single shot.
We had to take two steps to do it.
And there was nothing more simple than that.
We had to first translate down by the part of the translation-- let me call it tau-- which is equal to that part of the translation t, which is parallel to the reflection operation.
And then we had to reflect across, and that would get us from the first to the third.
That was a new sort of operation.
We'd indicate its locus by a dashed line to distinguish it from a mirror plane.
And it has a translation part and a reflection part.
Doesn't matter what order in which we do them.
We get to the same location if we reflect first and then slide or slide first and then reflect.
So this was a new operation that I'll represent by sigma tau-- looks sort of like a symbol for reflection, but the subscript reminds us you've got to translate by an amount tau that is parallel to the initial mirror plane.
And this gave us a new theorem that a general translation that had a part t perpendicular and a component t parallel, when it followed a reflection operation, was equal to a net effect of reproducing the object by a glide plane, sigma tau prime.
It had tau equal to the part of the translation that was parallel to the reflection part of the locus.
And it was located always at one-half the perpendicular part of the translation.
So using that theorem and completing the mirror planes that hang at the lattice points, we have a very interesting group that consists of a centered lattice, mirror plane hung at the corner lattice point-- this is also a lattice point, so we automatically get the mirror plane in the middle of the cell-- and then in between, we get this new 2-step operation, the glide plane.
And the pattern that's representative of this plane group is, as advertised, just what a mirror plane does.
And that is hung at every lattice point of the centered rectangular net.
So the glide plane, this new operation that popped up, does what the new operations did in all of the other preceding groups.
It tells you how do you get from the pair that you've hung at the lattice point to these that sit in the center of the cell.
They're related by translation, but the relation of one to another of these motifs is by the glide plane.
So that is an operation which has arisen in the group.
And this, then, is a group that we would call Cm.
C for a centered rectangular net, as opposed to the primitive net, which is the one we got immediately before.
Again, I sound like a cracked record sometimes, but let me emphasize the simplicity of these patterns.
Again, the pattern consists of what you would get when you hung a motif on one lattice point, and then that is repeated by m, which is the symmetry you've placed at a lattice point.
And that's hung at every lattice point of the centered rectangular net.
The glide planes just express relations between things that you already have when you've hung the motifs on the lattice point.
Any questions after this brief reprieve?
Comments?
Yes, sir
You called those [INAUDIBLE] glides, that's a sigma tau?
Yeah.
The relation between this one and this one, I've called sigma.
That's the reflection operation.
The relation between this one and this one down here would be the operation that has a reflection part and a translation part, tau.
Let me point out something that's worth observing when we start making some more complex additions.
We said early on that we only have to consider translations that terminate within the unit cell.
Because everything is translationally periodic-- not only the atoms in the motifs but the symmetry elements as well.
But the observation that I want to make is that if we put a glide plane in.
And let's do that for the direct addition of a glide plane to a lattice point.
And having done that, we have the potential of possibly having derived a group that we would call Cg.
OK.
This diagonal translation-- this is T1, and this is T2.
We might ask, what is the reflect [INAUDIBLE] glide operation that sits at the origin lattice point, followed by T1 plus T2.
Well, what does our theorem tell us?
It says that we should get a new glide plane.
It should be at one-half of the perpendicular part of the translation.
And the perpendicular part of the translation is T2.
So this is at one-half of T2, and it should have a glide component equal to the parallel part of the translation, and that is T1.
T1 plus the original glide component, tau.
What is this?
A glide component of half of T1 plus the entire T1.
If we ask, does that make sense?
Yeah, we would glide down to here and then we would translate down to here.
And that would give us sitting at half of a path of T1, a new object that sat here.
Is that a glide operation?
Yeah, it is.
But it is one that is not really distinct from the glide plane that sits here.
Because if we have a glide operation with tau equal to one-half of T1, and if this sits in a lattice, then there's going to be a glide plane has tau equal to 1 plus one-half of T1.
And there will have to be another glide operation that consists of 2 plus one-half of T1.
And the reason is that everything is periodic at an interval T1.
So the moral that I'm trying to draw here is that one can add or subtract to identify the actual nature of a symmetry.
One can add or subtract an integral number of translations.
And that permits one to reduce any tau to a sigma tau prime, such that tau prime is always less than the translation that's parallel to the tau.
In other words, lop off this translation of an entire T1, this translation of the entire T2, and you have identified the basic nature of the glide operation that sits here as something with a translation that's half of T1.
The translations that move motifs out of the cell may be related by a glide operation that involves an integral number of T1s plus half of T1.
But it doesn't change the basic nature of the simplest glide step that's in there.
That's a very obscure explanation of probably something that didn't puzzle you in the first place, but it's worth saying when we make some of these additions.
Let's finish this off.
We have another translation in here, and that's the translation T1 plus T2 over 2.
So, what would we get if we took the glide operation sigma one-half of T1 and followed that by a translation T1 plus T2 over 2.
And that operation, again, would be equal to a glide plane sigma prime with the tau equal to the original glide component plus the part of the translation that is parallel to the glide plane.
And it'd be located at one-half of the perpendicular part of the translation, which is T2.
So it'd be at one-half of one-half of T2.
So this says that the combination of the glide operation with the centered translation is a glide plane with a glide component T1.
And it's located at one-quarter of T2.
And that, indeed, is what you would do if you reflected and reproduced the object by a glide down to here and then translated by T2 over to here.
OK. No, I'm sorry.
We glided and then we added on the parallel part of the translation.
So we would end up down here.
And if one sits here, there has to be one repeated by T1 up here.
And this is exactly the same thing as a reflection plane that's been introduced.
So here is a case where we could subtract off the entire translation T1 and say this is identical to a mirror plane passing through the origin, a pure mirror plane sigma that is at one-quarter of T2.
So, let me clean this up and show you what we have.
Completing the operations, we would have a pair of objects related by glide, like this, that is hung at every lattice point.
It's also hung at the centered lattice point.
And what that is going to give us is a pair of objects that sit like this.
And what has come in as a result of those combinations is a mirror plane interleaved between the glide planes.
And this is exactly what we have in Cm, which was also interleaved mirror planes and glide planes with the origin shifted by one-half one-quarter of T2.
So this is not in a group.
Proceeding logically, we'd take Cm and replace the mirror plane by a glide.
When we do that, we have a consistent group.
But it turns out to be exactly the same arrangement of symmetry elements and exactly the same pattern as Cm, but with a little nudge over along T2 by one-quarter of that translation.
Yes
But in Cm, didn't we have mirror planes going through the lattice points?
Yeah.
And what I'm saying is here there are glide planes going through, but the lattice point is arbitrary.
I can put it anywhere I like.
And if I decide to put the lattice point here, that turns it into Cm.
OK?
So, let's put the two of them side by side.
Cm was this mirror plane, mirror plane, mirror plane glides, and the atoms motifs did this.
A lattice point here in the center.
And I'll deliberately, to make my case, offset the cell by one-quarter of T1.
Now I've got a glide plane here, same as this one.
Mirror plane at a quarter of T1.
Glide plane here at the centered lattice point.
Mirror plane here, glide point T2 away.
And here are the lattice points.
But the pattern of objects looks like this.
Armed in advance on what the thing has to look like.
Looks like this.
And this is exactly the same pattern of motifs.
OK.
So, coming out of this consideration, we have with the rectangular nets Pm, Cm, Pg.
But Cg, if we try to construct it, was the same as Cm.
And that exhausts the possibilities for a single symmetry plane and the rectangular nets.
OK. Let me move on then to the next step.
And I'm going to skip over a threefold axis.
And I'm going to look at the square net combined with the other symmetry that would require a net with this dimensionality.
And this would be a primitive square lattice plus 4 mm.
We've already done almost all the work.
So as we derive the symmetries that are subgroups of these higher symmetries, we've done P4.
And that says that if we put a fourfold access at the origin, that fourfold axis, we'll get another one in the center of the cell.
And we'll get twofold axes in the midpoints of all the edges of the cell.
And then 4 mm has one kind of mirror plane.
Two Ms because there are two kinds of mirror planes.
The mirror plane says, hey, if I'm in a lattice, I want to be at right angles to a rectangular or a centered rectangular net.
Well, OK.
This one is happy, because a square net is a special case of a rectangular net.
The two translations are merely equal.
So he is happy.
Combine that with T2, and we'll get another mirror plane like this, and another mirror plane like this.
Fourfold axis is going to rotate that mirror plane, so we'll have mirror planes running this way, and this way as well.
And now we have a different kind of mirror plane that we tried to put in in this location.
This mirror plane says, hey, I have to be parallel to the edge of a rectangular net or a centered rectangular net.
So if we look at the translations that are parallel to and perpendicular to this translation, lo and behold, this mirror plane is aligned along the edge of a centered rectangular net that has the additional specialization of being a centered square.
But that mirror plane now is perfectly happy.
And he says, OK, I'll hold my piece.
I have the arrangements of translations relative to my orientation that makes me happy.
So I can say now that there is a mirror plane running this way.
There has to be another mirror plane 90 degrees away from those orientations.
And this, then, is going to be the location of all the mirror planes in that net.
And at no time did the chalk ever leave my fingers.
Just doing what we said we're doing.
Putting in 4 mm, making sure the requirements imposed on the lattice are those that that symmetry element demands.
And here we go.
Except that there is now one other combination that we have not considered.
Here is a mirror plane.
And now this mirror plane is diagonal to our translation T2.
And here is a mirror plane.
And that mirror plane is diagonal to our translation T1.
So what is that going to require?
Here is the mirror plane, that's the operation sigma.
And here is our translation, T1, down to here.
We have a theorem that says that a reflection followed by a translation that has a general orientation is going to give me a new reflection operation, sigma prime, that's located at one-half the perpendicular part of the translation.
And it's going to have a glide component that is equal to one-half of the parallel part of the translation.
So what has that told us?
To get from this mirror plane down to here, we've gone this far in a sense that is perpendicular to the mirror plane, so this is T perpendicular.
And we have a part of the translation that is parallel to the mirror plane.
This is T parallel.
So the plane that results is going to be a glide plane in here.
It's located at one-half of the perpendicular part of the translation.
And that is one-half of one-half of T1 plus T2.
And it has a glide component equal to T parallel, which is equal to one-half of T1 plus T2.
So we get a new glide plane in here.
And that will require glide planes through similar arguments that go down the diagonals of the cell, like this.
And let me convince you that that, in fact, is an operation that must arise if I place, at the origin of the cell, a set of objects that have 4 mm symmetry.
We have one like this, one like this, one like this.
Another pair hanging here.
Another pair hanging here.
And if we do a reflection operation, let's say this pair up to this pair, and then slide it down by the diagonal translation-- and we have the same set, again, hanging down here at the diagonally opposed lattice point.
Reflect across, slide down to here.
The way in which you get that is, believe it or not, to reflect across this glide plane.
How do we get there?
We reflect across.
We translate down to here.
And the way I do that is by reflecting across this diagonal glide.
That probably has convinced no one.
But map it out with your own pattern, and I think you'll agree.
Let me observe, while you're considering that.
When we derive groups based on 3m or 6mm, we're going to have other cases where a translation along the edge of the cell is inclined to a mirror plane.
And the effect of combining a translation with a mirror plane that's inclined to that translation always has the effect of interposing a glide plane halfway in between the mirror planes that are related by translation.
So, let me say that in general, because that will be an observation that'll let us identify glide planes quickly.
So the general resolve is if we have a translation inclined to a mirror plane, which of necessity is repeated by translation, a quite general result is that a glide plane is interleaved always between the two.
And they'll be parallel because they're related by translation.
So we'll see some more cases where we can immediately state that without further thought.
OK.
There is something that we might consider doing.
I'd like to put it off, though.
Here we start with a mirror plane.
Can we replace the mirror plane with a glide plane?
The answer is yes, we can.
But it's not at all clear if I take a fourfold axis and put a glide plane through it, what this plane has to be.
So I'd like to leave this one for now and come back to that.
OK.
In the notes, I've tried to do all of these derivations thoroughly and logically.
So if this is a bit fast or me waving my hands, saying a glide plane is here, and this, that, and the other thing, when you can't really see what I'm pointing at, I think if you refer to the notes that'll be clear.
But yes, you had a question?
So you said that this glide plane [INAUDIBLE] one-half T perpendicular.
What did you write underneath that?
I guess that maybe is where your T and one-half T perpendicular is where the glide plane is
OK.
I was just demonstrating that, in fact, for this mirror plane with the diagonal translation, the glide plane would come in at one-half of the perpendicular part of the translation.
And the part of the translation that-- we were combining this fourfold access with this translation.
So the perpendicular part of the translation is half of the body diagonal.
And if a glide plane comes in one-half of the perpendicular part of the translation, that put it parallel to the mirror plane and at one-quarter of the way of the translation.
So this little scrawl here says it one-half of T perpendicular and one-half of one-half of the body diagonal.
So it's one-quarter.
All this equals one-quarter of T1 plus T2.
OK. Now something unexpected happens.
If we would move along, I skipped over 3m.
We know what P3 looks like.
And that is a hexagonal net, an equilateral net.
And we have a threefold axis that we've added to the lattice point.
And then we found additional threefold axes in the center of these triangles.
And, once again, the pattern that has this symmetry is just a triangle of objects hung at every lattice point.
And now we want to add a mirror plane to that threefold axis.
So we have a primitive equilateral net plus the two-dimensional plane point group 3m.
The question is, which way should we orient the mirror plane?
Well, a mirror plane says, I want to be along the edge of either a centered rectangular net or a primitive rectangular net.
There's nothing about this lattice that looks rectangular.
Yeah, there is.
I see a couple of people doing this, and they have the right idea.
Here's a lattice point.
If we go more than one unit cell in this diagram, here's a lattice point.
Here's a lattice point.
Here's a lattice point.
Here's a lattice point.
Go down to here and go over to here.
And lo and behold, here is a centered rectangular net, hiding incognito in a hexagonal net.
Well, that helps.
I can put in the mirror plane.
But let me point out-- this is getting very messy, so I'll redraw it on a smaller scale.
This will be worth doing so that it's perfectly obvious.
I'm going to draw a number of equilateral nets.
The reason I'm drawing more than one cell, I would like to point out that here is one centered rectangular net.
And it has its edge perpendicular to the translations in the hexagonal net.
But there is also a rectangular net that goes down like this.
That's a centered rectangular net that has its edge along one of edges of the unit cell.
So I have two possibilities.
One is to draw the rectangular net like this.
The other one is to draw the rectangular net like this.
And having confused you thoroughly, let me simply draw this larger cell again.
And say that I could, if I wanted to, put the mirror planes in in these directions, along the edges of the cell.
OK. And in this case, if you're convinced that there's a centered rectangular net sitting there.
Let me clean this up.
And here's one mirror plane.
60 degrees away is another mirror plane.
And there'll be another mirror plane at this lattice point doing this.
So now I've got 3m at every one of these threefold axes.
And I got all those simply by repeating these mirror planes by translation.
But in one case, the mirror planes are perpendicular to the cell edge.
This being T1, let's say, and this being T2.
In this case, the mirror planes are along the edges of the cell.
This being T1, and this being T2.
So, holy mackerel!
One point group.
Two different space groups, which differ in the way in which the symmetry is oriented relative to the lattice.
And if you think back a little bit, this particular net, with this dimensional specialization, was happy with a sixfold access as well, and will be compatible with 6mm.
So, really, these two different orientations for the mirror planes, even though we haven't got there yet, are the two different orientations which are both present simultaneously if we were to put 6mm into this hexagonal net.
So, same point group.
Same lattice.
Two different plane groups that depend on the orientation of the symmetry relative to the lattice.
So there isn't even a one-to-one correspondence between the point groups and the plane groups.
The pattern for either of these is, once again, going to be just what the pattern of the point group does.
If we start with a motif here and repeat that by the threefold axis and the mirror planes, we would have motifs like this.
And here the translation points out in between the mirror planes.
In this case, the mirror planes would repeat the objects in two locations, like this.
So, indeed, both patterns have 3mm symmetry.
And what's different is the way the translations come out in between those mirror planes.
OK. Now our notation has come up against an impasse.
We said that the notation for the plane group should be the symbol for the lattice type, which the initiated know has to be a hexagonal net.
The point group that we've added, which is 3m, but now how do we distinguish the two different orientations of the mirror planes?
And the way around that is to actually use three symbols.
And the second symbol will be what is perpendicular to the cell edge.
And the third symbol will be what's parallel to the cell edge.
So this one would be called P3M1.
And this one, just to distinguish it, would be called P31M.
So if this weren't bad enough, now you've got almost the same symbol with a permutation of two of the terms in it.
In this case, the mirror plane is along the cell edge along the translations.
In this case, it's perpendicular.
But we're not quite yet done, because we've got translations that are inclined to mirror planes.
And now you can see how prudent I was in saying quite generally, without trying to figure out what is the perpendicular part and what is the parallel part, here's a translation and it's at an angle to a mirror plane.
And therefore we get a glide that has to be halfway between these mirror planes.
And there'll have to be a glide halfway between these mirror planes, and a glide that's halfway between these mirror planes.
In this case, the mirror planes are this way.
Again, a translation that's inclined to the mirror plane.
We have to have a glide that is midway between these mirror planes.
In this case, they're a little easier to identify.
They sort of make a triangular box that surrounds the threefold axis.
So now we see another difference in the plane groups and the reason why they're so very distinct.
And that is the rearrangement of the glides relative to the threefold axes are quite different in these two cases.
So these are two quite different symmetries.
Notice that when we start identifying special locations in these two plane groups, here there is a location only of symmetry 3mm for a point group, 3m for a point group.
Here there are two different locations, one that's symmetry 3m, the other one is just symmetry 3.
So this is location of point group 3 and a location of point group 3m.
So the sort of special positions that exist in these two plane groups will be very, very different.
One more addition, and then I think we can leave with a light heart because we should be done.
What about a primitive hexagonal net combined with 6mm?
And if you think I'm going to try to draw that, you're crazy.
But I can say quite simply and hopefully convince you that this looks like P31M plus P3M1 right on top of one another, because we've got mirror planes that are 30 degrees apart.
So, as a schematic direction and an invitation to do this at home in your spare time, is to take what P6 looks like.
And that's sixfold axes at the lattice points.
Threefolds here.
Twofolds here.
And then, directly on top of this, place the mirrors and glides in this orientation, and the mirrors and glides in this orientation.
So we will have mirror planes coming out like that, and glide planes in between all the parallel mirror planes.
As I say, I'm heroic but not foolish.
Yes
Can you just explain that derivation again?
Your P3 [INAUDIBLE]
OK.
The object of this additional bit of confusion is to put in the symbol the point group, which is 3m, and then tell you how the mirror plane in a point group is oriented relative to the edges of the cell.
And we could call them anything we wanted to.
We could call them P3M perpendicular, subscript perpendicular.
Or P3M subscript parallel to indicate mirror plane parallel to the cell edge.
And mirror plane perpendicular to the cell edge.
Or we could call them some obscenity, which would be very descriptive, as well.
But this is just a way of keeping track of what's parallel to the cell edge and what's perpendicular to the cell edge.
So, this middle symbol is what's oriented perpendicular to the cell edge.
And in this case, this is the mirror plane and that is perpendicular.
All the mirror planes are perpendicular to the edges of the cell.
And then parallel to the cell edge, there's no symmetry at all.
So that's 1.
1 stands for no symmetry.
In this case, the mirror plane is along the cell edge, so there's an M in the third position.
And perpendicular to the cell edge, there's no symmetry at all-- no symmetry plane at all.
So it's purely arbitrary, but we need some mechanism for distinguishing which one we're talking about.
That pretty much works as well as anything.
And it's a minor perturbation of what we've done for the additional plane group symbols.
All right.
It's time for our midafternoon break.
It's a nice place to quit, because you're probably feeling good.
We've done this, and we can move on to something new.
Actually, there's one more trick.
And unless you were really clever, you wouldn't have thought of it.
But there's another small family of plane groups that we can get through one particular device.
I shouldn't have told you that, because you'll be less inclined to come back now.
But we're almost through.
And I think it'll be very clear what these additional possibilities are.
Examine every possible means for combining the symmetry at once, but there is a seemingly paradoxical trick that we can yet pull.
And let me indicate what is true here for 2mm.
OK, so this is a twofold axis.
Has mirror planes perpendicular to it.
If one of these is a mirror plane, the other one has to be a mirror plane as well.
So there's no way we could make one a mirror plane and one a glide plane.
OK, that requires a net that is exactly rectangular.
So let's put in the twofold axis.
And I add one to the corner of the cell.
As we well know we have to have twofold axes at all of these other locations.
We want to put a mirror plane in the cell.
We could pass it through the twofold axis, and that would be the same as P getting to P2mn back again.
But why do we have to put the mirror plane through the twofold axis?
We have to have the twofold axis left unchanged when we add the mirror plane, because if we created a new twofold axis we create a new lattice and we'd wreck the lattice that we've constructed.
But why, why, oh why couldn't we put the mirror plane in like this?
That's going to leave the twofold axis alone.
It's going to leave the translations invariant.
Why don't we do that?
Why not?
So here, trick number five, or wherever we are now.
You can add the symmetry elements of a point group to a lattice, but not necessarily at the same point.
You can interweave them.
But the constraint is that this addition must leave the axis invariant.
Let's leave the twofold axis invariant.
And vice versa.
That is to say the mirror planes can't create new twofold axes, the twofold axis can't create new mirror planes.
And let's make sure we understand the reason why.
If I've got a twofold axis here and a twofold axis here, I have to have a translation that's twice their separation.
If this distance is delta, then I have to have a translation that's automatically created at twice delta.
This is the same as saying twofold axis with a translation gives you a twofold axis halfway along.
If I take the twofold axis, I get the translation back as a consequence.
So what I'm saying is if we take the first twofold axis and repeat number one to number two, and then add another twofold axis which repeats number two to number three, you have a third one that is related by translation of twice delta.
So if you're going to interweave the symmetry elements, you have to leave the arrangement of symmetry elements at the same intervals, otherwise you would have created a translation or more that is incommensurate and incompatible with the other ones.
So this is a third trick.
And we are going to need a new theorem.
Let's first of all look at the possibilities.
We can have the twofold axis interweave with a mirror plane.
How about putting a glide plane in between the twofold axis.
In other words, replace the mirror plane by a glide plane.
This twofold axis hangs off here, reflect it across and slide it down to here, and you get this twofold axis here.
So that's also a self consistent compatible arrangement.
How about a centered net?
And that's more difficult.
A centered net has twofold axis and all the places of C2, and then twofold axes at locations like one quarter, one quarter as well.
We can't put a mirror plane in here.
That's not going to work, because it will generate a new twofold axis, and that changes the translation T1 to half its value.
We can't even put a glide plane here because this would slide down to here, reflect across, and that's not going to work.
So the number of possibilities is limited, but we'll see that there's one other case as well for a fourfold axis.
OK, we need a new combination theorem that says if you have an operation a, pi, and combine it with a reflection operation that's removed from it by some distance delta, what is the result?
That general theorem has worked out for you on the notes, but I think what I'll do it in the interest of time is simply look at how this arrangement of symmetry elements would move things around.
And here I've taken the motif and hung it at the lattice point, and repeated it by the twofold axis.
The mirror plane would take this and reflected it across to here, then take this one and reflect it across to here.
And do the same thing down at the bottom of the cell.
Let me now ask you what sort of plane has arisen that would be perpendicular to the mirror plane?
Could be interweaved or passed through the twofold axis.
Anybody want to hazard an answer?
OK. We have to have some sort of correspondence theorem that says if you've got a twofold axis, have one plane, you've got to have another plane 90 degrees away.
Do I know how everything is related?
I know how this is related to this.
I know this is related to this.
That's by a reflection plane.
I know how the twofold axes relate things.
How is this pair related to this pair?
And the answer is that the way they are related is by rights and lefts on here.
This is right, this is right.
Reflection changes handedness to left, so I've got to have some way of getting from this pair to this pair that involves a change of handedness.
And I see nobody is really jumping out of their seat, but there is a glide plane in here.
Take this pair, slide it along by half of T2, and flip it across by reflection.
And there is a glide plane right here.
And therefore of necessity there's a glide plane here.
And this glide combined with the perpendicular translation will put a glide plane in here as well.
And you look in your tables.
This is plane group number seven.
And this one has a twofold axis.
It has a primitive rectangular neck, so this is called P2m, and now the second plane at right angles is not an mirror plane, it's a glide plane.
So this is called P2mg, or there's a shorthand form that leaves out the two.
PMG is the shorthand symbol for it.
So there's a new plane group that is based on orthogonal symmetry planes.
One a mirror, one a glide, and twofold axes.
Let me ask you now what is the point group of a crystal that would have this relation between the atoms that are down in the guts of the crystal?
What would be the point group of the crystal?
We've got two different point groups.
We've got point M, we've got point group two.
What would be the point group of the crystal?
And this is a new problem, a new concept.
All of the plane groups that we looked at until this point, the symmetry elements on an atomic scale down inside the plane group were exactly the same as the point group that we had added to the lattice points.
But here we've got two different point groups that had been put into the lattice.
So do we say that this crystal has symmetry two or symmetry M?
Or just point group 2m.
That won't work because if you have one mirror plane in a point group, you have to have another.
What this introduces is a very subtle point.
The point group is inherently a macroscopic symmetry.
When I say a crystal has a particular point group, what I mean is that if I look at the exterior faces of the crystal, I would say there's a twofold axis here.
And that's about all, if these spaces are pair wise distinct.
I have a crystal that looks like this externally.
I would say that that crystal has symmetry 2mm based on the faces.
If I found that the etch pits on the surfaces were not quite the same on this face and this face, I would have to throw out the twofold axis, perhaps, which means this mirror plane would go out as well.
But also I would do things like look at the optical properties.
I would measure the conductivity as a function of direction.
I would look at the slip systems and yield stresses in different directions, and the assignment of a point group is an experimental observation that requires that you look at all possible properties.
Not only shape, but properties like conductivity and the magnetic susceptibility, and also perhaps the way the crystal diffracts X-Rays.
That's another physical property.
And then when you've done as many measurements as you choose to make, you say as far as I can tell, all behavior of this crystal is determined inconsistent with this particular point group.
But you're doing macroscopic things.
You're doing macroscopic things.
When you talk about the plane group or space group, you're talking what goes on on a local atomistic scale in the unit cell.
So what would be the point group of this crystal?
It would have a twofold axis that would be manifested externally.
It would have a mirror plane, which makes the left hand side want to be the same as the right hand side.
And if this were not a mirror plane here, but a glide plane, what you would say is that this face, if I reflect it and slide it along by one angstrom is going to become this face.
And this face here, if I reflect it and slide it back by one angstrom is going to become this face.
But what will that do to the external shape?
I just extend all these surfaces, and it's going to be exactly what I've drawn here.
How would you distinguish a mirror plane from a glide plane?
If a crystal has a mirror plane, a face that sits here passes through some atoms.
Those atoms are repeated by reflection.
And being slit up by an amount tau which is on the scale of atomic dimensions, and that would give rise to a face here.
But can you macroscopically assign any difference to the fact that two faces atomistically don't meet, but are separated by 3.2 angstroms?
No.
What you see is one face like this, and one face inclined to it with a slope that is the same for either a glide plane or a mirror plane.
So another truth about crystals is that macroscopically a glide line plane manifests itself as a mirror plane.
So paradoxically this crystal, which only has a twofold axis and one mirror plane down in its guts is going to look as though it has point group two in it, even though there is no sight, atomistically within this arrangement of atoms that has symmetry 2mm
I have a question
Yeah?
Sorry
[INAUDIBLE]
That's a very good question.
And actually, to answer it properly requires knowing something about diffraction.
The symmetry of the diffraction pattern that you observed would look as though it had a mirror plane in it.
Which is to say if we put a beam of white radiation down along this direction of a crystal, imagine these things all extending out in a third dimension, that what we would see among the arrangement of spots is a twofold axis in the center of [INAUDIBLE] photograph we'd see a mirror plane running this way, and a mirror plane running this way.
So we would see spots on the [INAUDIBLE] pattern, which would look like this.
So the glide plane would also behave it were a mirrored plane.
And, maybe, if you think of what goes on with the fraction.
If you ask yourself, if I brought in x-rays this way and diffracted them off a layer of atoms related by this glide plane, and then I can't do this actually to the crystal.
But, suppose I then inverted the direction of the incoming beam, and slitted along by five angstroms.
Doesn't make any sense.
The diffraction from these planes is going to look the same whether I bring in from one side or the other.
So a glide plane, in diffraction symmetry, manifests itself as a mirror plane.
So how can you determine the presence of glide planes using diffraction?
And you can.
And the answer is, that the glide plane causes the intensity diffracted it from planes with certain indices to be identically zero.
You've probably heard of these magical extinction rules they're called.
And it says that if h plus k, plus three times L is two pi plus four, and the crystal is green, then you've got such and such, the reflection being absent.
Actually it is-- I'm getting distracted-- but, it turns out that glide planes affect just a sheet of reflections in reciprocal space.
And the reason some reflections are absent is that that sheet of reflections corresponds to what the projection of the structure onto that plane would do.
And, if you project a structure that has a glide operation onto the plane of the glide plane, the structure looks like the lattice translation is half as big.
And, if that's the case, the diffraction spots are twice as widely separated.
And it turns out that all of the reflections that have an index corresponding to the direction of tau are absent if that index is odd.
So, if you knew a little bit about it to begin with, that perhaps answers the question.
So yes, you can determine the presence of glide planes.
And later on, we'll talk about screw axes.
You can determine their presence unambiguously, but you do that not the symmetry of the diffraction effects, but from systematic absences.
OK, another thing I suggested we could do is to put a glide plane in between the two-fold axes instead of a mirror plane.
And this says that I have a pair of atoms repeated by the two-fold axis, the glide plane would take that pair, shift them down by half of the cell, and reflect them over.
So this would be the pattern of objects.
This is not a lattice point, because if these are right handed, that's a right-handed pair, this is a left-handed pair.
Now let me try you again.
I know how these guys left and right to this glide plane are related.
How about these ones that are up, to the ones that are down here in the middle of the cell?
There's got to be, from our correspondence principle, some plane going in this direction.
Anybody want to hazard a guess at what it is?
The answer to that question resoundingly is, no, not at this hour the afternoon, thank you.
Yeah?
Left one
Yeah, very good.
Right ones have to go left ones, and they're going to go left and right about these planes here.
And, if I combine this glide plane with t2, there has to be another glide plane in here.
If I combine this glide plane with t1, there'll have to be another glide plane in here.
OK, and that turns out to be plane group number eight, P2gg, not P2mm, both mirror planes have become glide.
All right, we're almost there.
If you look at the hexagonal symmetries, there is just no way that you can interweave things, and an attempt is made in the notes that I handed out.
You just cannot do it.
But with P4M, P4, there is one final possibility.
And I'll just set that up and not carry them out.
OK, I'm going to put on just P4 with twofold axes in here.
Now we have planes that go in orientations like this.
And then there is another plane 45 degrees away that is a symmetry independent plane.
And the question now, I'll let you have a crack at it, is there any way in which I could take planes in the diagonal orientation or parallel to the cell edge and interweave them in between the rotation axes in such a fashion that the rotation axes were left invariant?
Again, conversely if they're not left invariant, we will have created new translations and wreck the lattice.
So how could we slip in planes, either reflection or glide, and leave the axes invariant?
One quarter and three quarters?
Yeah.
Do you want parallel to the cell edge?
Horizontal
Horizontal?
Yeah.
This [INAUDIBLE] here has a fourfold axis up, a fourfold axis down, a fourfold axis up.
Twofold axis down, twofold axis up, twofold axis down.
So this would be a lovely location for a glide plane.
So that's one.
From the fact that I am placing the same diagram on the board yet again suggests that there's another way.
That's a perfectly good suggestion, and that's one of the things we would have to examine.
Is there any other way I could put in a mirror plane or a glide plane and leave the axes invariant?
Yeah?
[INAUDIBLE] 45 degrees?
45 degrees.
Through here.
I don't want to do it through here because the fourfold axes sit on the same side, but you want to put it in like this?
I don't think you can do that.
Glide plane won't work here.
How about a mirror plane?
There's no mirror plane that goes through these twofold axes for P4mm.
What we had there were mirror planes here, and then going this way were glide planes.
We've already got that.
How about putting in a mirror plane like that?
This goes into this.
Halfway along there would have to be another mirror plane in that orientation.
Here's a twofold axis that demands that there has to be another one if it has a mirror plane passing through it.
So there's another way.
One final one, and I will not keep you guessing anymore, sneaking a peak at the answer.
There is another way you could do it, and that would be to put the glide plane down diagonally through the cell.
That is put the glide plane here.
That and take this twofold axis, reflect it across to here, and slide it down to here.
So this twofold axis would be related to this one by glide.
So those three possibilities are available for point group 4mm and a square net.
If you try them, and the simple way of doing it without any general theorem would be to draw in the way the symmetry elements repeat the motifs, and then ask what they require for the remaining symmetry elements.
All of these give the final result.
So they yield the same result.
You can develop a general theorem for what you obtain if you have an operation a, alpha combined with a plane, and make it a general plan, a glide plane that is removed from the axis by some distance delta.
And that theorem is written out for you in the notes on the very first page.
You take a glide plane, and this is a specific example, a, pi, if you have a glide plane that misses a twofold axis by some distance delta, then the effect of those two successive operations is a glide plane at 90 degrees to the first, and it's removed from the twofold axis by two delta.
It has a glide component, excuse me, has a glide component two delta.
So there's a general theorem for the twofold rotation.
The general theorem for a fourfold or a threefold rotation is considerably more complex to derive, but that's contained for you in the notes as well.
All right, we've come to the end of one other major part of our story of symmetry.
These are the periodic patterns in a two dimensional space.
And these are then the 17 crystallographic plane groups.
Crystallographic is rather redundant because they are groups that have [?
in ?]
translation and therefore these are the symmetries, the two dimensional symmetries that are suitable for the atomic arrangements in crystals.
As a little bonus, we also have, as a result, 17 of the three dimensional space groups.
Just imagine each of these plane groups with a translation perpendicular to the plane of the plane group, and that would leave all the symmetry elements coincidence.
So you could pick a third translation at right angles to the blackboard of any arbitrary length, and you would have a three dimensional lattice that contain the symmetries of a plane group.
I think I mentioned earlier that the way you distinguish a plane group with a twofold axis in it from a space group with a twofold axis is that a lowercase letter for the lattice type implies plane group, and an uppercase symbol, capital P stands for a primitive lattice in a space group.
So something like lowercase p4mm is a plane group, capital P4mm is a space group with a third translation of arbitrary length at right angles to the plane group.
We talked about plane groups and space groups, let's take a giant leap backwards.
How about the one dimensional space groups?
Bet you were wondering about them all along and were afraid to ask.
So what would be the story for one dimension?
It's nontrivial.
A one dimensional space group would be a lattice row.
You have just one dimension to play with, and you can make that space periodic by translation.
So there's one lattice type.
Just the lattice row.
And what sort of symmetries could you place in that lattice?
Now you have to define your ground rules.
In our two dimensional symmetries, we did not permit any operation which would pick the two dimensional space up and rotate it and flip it over and put it down again, because that is a transformation that takes you out of that space and pops you back into it.
And we will not in our discussion of three dimensional crystallography consider symmetries operations that take something, suck it up into a fourth dimension that we can't comprehend, and then all of a sudden pop it down in again, and suddenly it appears.
I mean, that sounds bizarre.
We wouldn't want to do that.
So for plane groups, we did not allow for any operation, say a twofold axis that would flip the object over and turn it upside down.
But why not?
I mean this, is mathematics.
If it's your ballgame, you can make up the rules.
So why couldn't we have a twofold axis in the plane or the plane group?
It would have to leave the net invariant.
Well actually such entities do exist, and they have been derived.
They're called the two sided plane groups.
And if you want to allow that when you make up the game, you can actually do that if you wish to allow it.
And also there could be a mirror plane in the plane of the space that would take the top side and relate it to the bottom side.
So you can do that, you can permit that.
That's a different beast entirely, but you can allow that to be one of the transformations.
So for our one dimensional space, I would submit to be consistent with what we've done and just finished in two dimensions.
And what we will do in three, we will not allow for any operation that will take the space and transform it into a second or third dimension, and then put it down again.
So that being the case, the only operation that is possible is a mirror point that would reflect things left to right.
And let me illustrate now with some patterns.
There's the lattice point.
So let me put in some motifs.
and I have to make a one dimensional motif, but to distinguish the ends I'll take a little artistic license and make one end of the motif a little fatter than the other.
Or alternatively, I could put a little one dimensional headlight on this thing that would shine just in one direction.
So here's a motif hung on every lattice point, and so this is no symmetry at all.
So the symmetry part of the symbol would be one, and the lattice is primitive, and what do you think people use?
We've used lowercase symbols, we used uppercase symbols, what's left?
Yes?
Greek?
Good try.
Gothic.
Actually, what you do is you use a Gothic symbol with all these nice little shaded angular shapes.
That's a Gothich P, isn't it?
Yeah, that's a Gothic P. It is.
It actually has thick and thin parts, curved parts [INAUDIBLE].
So that's P1.
What about another symmetry?
Well if we have a mirror point in there, another possibility would be to have these motifs point alternately left and right.
So I'll take some non one dimensional license and say that there would be mirror points at these locations.
Really nice symmetry, but actually it would be totally wasted on these poor denizens in a one dimensional world.
Life would be dull and uninteresting because they couldn't see anything except the motif on either side of them, because everything is constrained to one dimension.
There's an old saying about sled dogs that says if you are not the lead dog, the scenery is not very interesting.
And the same is true for these poor people in a one dimensional world.
They could never see the elegance and beauty of that.
But in any case, that would be Gothic P, no snickers, please, and that would be m. And those are the only two possibilities and a one dimensional space.
We have a couple minutes to go, but I think this is a good place to pause.
And I'll give just a brief indication of coming attractions.
We're now going to look at point groups in three dimensions, and a good way of entering this much more complex situation is to go back to something analogous to what we did for two dimensional symmetries.
And I'm going to first consider the arrangement of rotation axes in three dimensions.
We do not have the constraint that all the rotation axes be perpendicular to the plane of a two dimensional space, being therefore more properly rotation points.
We've got three dimensions we have to view a rotation axis as extending infinitely in a direction, and there's no reason why we can't have another rotation axis at an angle to it.
But you know that already.
Everybody's heard about cubic crystals.
Even if you think all crystals are either body centered cubic, primitive cubic, face centered cubic, or complex, four kinds of lattices.
But you've heard of cubic crystals, in the cubic crystals clearly the rotation axes are inclined to one another and arranged spacially.
So we're going to ask the nontrivial question, how can we combine more than one rotation axis at a time about a common point in space?
And the constraint is going to be that a rotation about the first axis followed by a rotation about the second axis, wherever it is, is going to have to turn out to be something that is crystallographically compatible with a lattice.
So that's the constraint.
It's not an easy question.
And just to make you feel proud of yourself when we get through this, this uses a geometry that is due to one of the great mathematicians of all time, Leonhard Euler.
Probably the most remarkable thing about Euler was that he's Swiss.
How many world class scientists have you heard of that come from Switzerland?
Not very many, and the reason is it's such a small country, and if genius occurs as a certain fraction of the population, that's not going to happen very often in a country like Lichtenstein or Switzerland.
Have you ever heard of anybody prominent in science who came from Lichtenstein?
No, probably not.
OK, this then is going to be a nontrivial piece of mathematics for us, largely because it involves spherical trigonometry.
Which I'm sure if you've ever heard of, you've forgotten, and probably don't see any utility in it.
OK, with that exciting prospect in hand, I look forward to seeing you on Thursday.
I think you will get going.
A lot of people are not here, but I think if the MRS meeting were going on across the river I would be there too if I didn't have a class to go to.
We had last time finished a discussion strain.
And we introduced strain by defining it as a displacement, each component of a vector displacement U as being given by a set of coefficients, eij times x of j, which says that the displacement in a deformed body varies linearly with the position of the particular point in the body.
We could also express the same thing in terms of change of length.
The change in the vector U would be given by-- we show the same set of coefficients-- eij times delta x of j.
So whether you want the displacement vector or you want the fractional change of length, you get this by an expression of the same sort.
We saw though that unless we define the coefficients correctly we could have a situation where a body is not only deformed but is rotated as well.
And we saw that in general unless we define these coefficients carefully we would include in this tensor a component of pure rotation, rigid body rotation.
And this would not measure deformation.
However, regardless of its suitability or unsuitability, the positional vector is a vector.
It transforms like a vector.
The displacement U is a vector, and it will transform like a vector; ergo, the 3 by 3 array eij is a second-rank tensor and we'll transform like a tensor.
We showed though that we could take eij and get rid of the pure rotation that might be contained in those coefficients by writing it as a tensor epsilon ij plus another tensor omega ij.
And that was a new wrinkle, the concept of adding two tensors together to get a third tensor.
We did run into that before.
And we define the elements of the two tensors epsilon and omega; epsilon ij was given by one half of eij plus eji, and omega ij is given by one half of omega ij.
And notice the order of the subscripts here in this difference.
That is a matter of definition, but it defines which term is omega ij and which is omega ji minus omega ji.
I'm sorry.
This should be eij minus eji.
And we showed that at a tensor of this form which would have all of the diagonal terms zero and the off-diagonal terms equal to the negative of one another that a tensor of this form does indeed correspond to pure body rotation.
We showed that by looking at a point at the end of a positional vector U and looked at a displacement which would be exactly at right angles to it.
And the U would be given by the tensor relation.
We showed that U dot r is identically zero when the displacement vector is given by this tensor with diagonal term zero and the off-diagonal terms equal to the negative of one another.
So we have then from a general tensor of the form eij constructed in the terms epsilon ij something that is a measure of true deformation.
It is a symmetric tensor by definition.
And therefore in addition to all of the other properties of second-rank tensor, namely having a law of transformation and staying symmetric for any arbitrary change of coordinate systems, the radius normal property also works.
Hence, the representation surface that we construct from the tensor epsilon ij xi xj equals 1, gives us the strain quadric.
And it has the property that the radius gives us the value of strain in a given direction.
And the value of the radius literally is going to be the vector that's on the left hand side, the magnitude of U times the component of U that is parallel to the radial vector per unit radial vector.
And this then is the tensile strain, the fractional change of length as we look in this direction.
The radius normal is going to give us the direction of the total displacement U.
And the value of the property in that direction is going to be the product of that displacement with a unit vector in the direction of interest.
And therefore that is the tensile strain as we said before from the general properties of the quadric.
So everything we said about second-rank tensor and in particular symmetric tensor holds for the strain tensor.
We can talk about diagonalizing the strain tensor into a form epsilon 1, 1, epsilon 22, zero zero, epsilon 33.
And I'll remind you that we know how to do this to set up the normal equations and solve for the eigenvalues and then look for the principal axes of a tensor.
There are special forms of the strain tensor.
Something that 1 diagonal goes into this form is called plane strain.
And I love the melodic nature of that, the plane strain.
If this were strain induced by a Modernistic transformation, we could call it the Bain plane strain.
If we diagonalized it, it would be the main Bain plane strains.
And if that deformation took place in an aircraft window, we could call it the plane-pane main Bain strains.
And we could continue on indefinitely, but I think you're finding this tiresome.
Another form of the strain tensor that doesn't look what it actually represents is epsilon 1 zero minus epsilon 1 zero zero zero zero.
That looks peculiar, very special.
But, again, this is analogous to what we found for stress.
If we rotate this tensor 45 degrees, we would find that treat tensor was diagonally, had all the diagonal elements zero.
And we would have the epsilon 11 prime-- let's just call it epsilon prime-- and this epsilon prime zero zero zero.
And this is pure shear Something that we'll show shortly, and in fact I left it to you as a problem on a problems set.
The sum of the diagonal elements, epsilon 11 plus epsilon 22 plus epsilon 33 is something that's defined as the trace of the tensor.
And for the strain tensor, the trace turns out to be equal to delta V over the fractional change in volume.
And we'll show that in just a little bit as well.
So a characteristic of pure shear is that it does not result in any change of volume of the solid.
There's deformation, but the volume stays exactly the same.
Let me then proceed to show that the trace of the strain tensor is indeed the change in volume.
And want I'll do is to look at an element of volume that is oriented along the principal elements of strain.
So if this as x1, this is x2, and this is x3, the strain tensor I'll assume is oriented.
So we have a term epsilon 11 zero zero zero zero; epsilon 22 zero zero zero; epsilon 33.
So let's suppose that the solid originally has edges L1, L2, and L3 along x1, x2, and x3, respectively.
So the initial volume before any deformation will be simply L1 times L2 times L3.
Let's suppose now we impose the strain, and the volume then we'll increase to some value of V plus delta V. And the length L1 will change to a value L1 times 1 plus epsilon 11.
That is to say it'll be L1 plus epsilon 11 times L1, which can be factored out in this fashion.
L2 changes to L2 times 1 plus epsilon 22.
And L3 changes to L3 plus epsilon 33 times L3, which I'll write in this fashion.
So expanding this product, it'll be L1, L2, L3.
And then if I simply multiply out these terms, I'll have 1, and then I'll have epsilon 11 plus epsilon 22 plus epsilon 11 times epsilon 22.
And this will be times 1 plus epsilon 33.
And if I carry out that multiplication, that will be 1 plus-- and I'll get exactly these terms again-- epsilon 11 plus epsilon 22 plus epsilon 11 times epsilon 22.
And then a term epsilon 33 plus epsilon 11 epsilon 33 plus epsilon 22 times epsilon 33.
And then a third-order term, epsilon 11 times epsilon 22 times epsilon 33.
Now having gone through the painful process of expanding that, I will proceed to say that the epsilons are always going to be very small.
Elastic strains are typically on the order of 10 to the minus 6.
So a product of two of those strains is 10 to the minus 12.
And a product of three of those terms is going to be on the order of 10 to the minus 18.
So this last term is going to be minuscule.
And now I use a higher order term intentionally rather than a higher rank term.
This is really a term of higher order.
The same is true of all the peer-wise projects of this [INAUDIBLE].
So I now have left if I turn out these higher order terms to pasture, I'll have 1 plus epsilon 11 plus epsilon 22 plus epsilon 33.
And the rest is negligible.
And this will be the original volume V times 1 plus epsilon 11 plus epsilon 22 plus epsilon 33.
So delta V over V, if I subtract off V on the left, delta V over V then is simply going to be equal to epsilon 11 plus epsilon 22 plus epsilon 33.
So this is indeed the fractional change of volume, and it's the trace of epsilon ij.
Now, physically, you wouldn't expect the volume to change if I change my reference axes because that's a scalar quantity.
So I've shown by a hand-waving, backdoor argument that in fact the trace of a tensor does stay invariant for at least this particular tensor.
But it's fairly straightforward to show that for any tensor aij the trace of the tensor is invariant when you change the coordinate system.
And that's not hard to do.
If you're clever, it takes about three lines.
So I'll invite you to the exhilaration of discovering that for yourself on a problem set.
Any questions or comments?
All right.
Having defined strain as a symmetric second-rank tensor when we factor out rigid body rotation, we can view strain as a physical property.
And I think I tried to hoodwink you last time by saying this is a property of material.
So therefore strain tensor has to also conform to all of the symmetry restrains that we have derived very exhaustively.
And you said, no, no, no, no, that can't be; I don't believe that I can only give a uniform uniaxial deformation to an orthorhombic crystal.
And I say, yeah, but to get the deformation, you have to squeeze the crystal.
And that is going to be linked to the deformation by the elastic constants, and they are physical properties.
And that maybe makes you think a little bit.
But actually even though operationally you impose a strain by imposing a stress or by doing something else to the crystal that results in a strain, you can nevertheless pick any stimulus you wish to achieve a desired state of strain.
So there is no symmetry constraint on the form of the strain tensor that can result.
So the stain tensor then is not a property tensor.
It is also a field tensor just like stress.
So it can have any form we choose to create in it.
So we now have the potential of having stress as a field tensor.
And we have strain as a field tensor.
And within limits, we can create strains of these two field tensors of any form that we wish.
And we can now regard these as things that we do to crystals and look at a whole range of properties that result when the generalized force is not a vector, a tensor of first rank, but the generalized force is a-- let's say-- a stress tensor.
It's a generalized force which itself is not a first rank tensor.
It's a second rank tensor.
So we'll see a whole collection of interesting properties that fall into this category.
But first I'd like to look at one remaining second-rank tensor property, and that is thermal expansion.
And this is a curious second-rank tensor property, and we couldn't discuss it until we had talked about strain.
And let me introduce it by one-dimensional example.
Let's suppose we have a rodlike specimen of length L. And let's suppose it is in equilibrium at some temperature T. And then we heat it up to some temperature T plus delta T. In response to that particular change in temperature, the length of the rod will increase to a length L plus delta L. It expands.
And we would find experimentally that delta L, first of all, is proportional to L. If you make the rod twice as long, the amount of expansion in an absolute sense is twice as large.
And secondly, provided you're well above very, very low temperatures, close to absolute zero, delta L will also be proportional to the change in temperature.
Thermal expansion is one of those properties which thermodynamically must go to zero as temperature goes to zero.
So the thermal expansion coefficient that we've yet to define will be a function of temperature as we get down to very low temperatures on the order of a few degrees Kelvin.
So we can define then the linear thermal expansion coefficient for this rod as 1 over L, delta L over delta T. So this is something that will give us the fractional change of lengths.
And what we can do now in general is to say that if we create a general strain, not a tensile strain, we have a general strain epsilon 11 plus epsilon 12, epsilon 13, and so on, has to be symmetric.
So epsilon 21 is equal to epsilon 12.
We'll say that this tensor epsilon ij will be written in general for the three-dimensional case as a second-rank tensor aij times a scalar quantity delta T. The strain is the field tensor.
That's the response.
The stimulus that creates this response is an incremental change in temperature.
But as this is a second-rank tensor and this is a scalar, an array of coefficients in which we multiply every element of a tensor by a scalar is also a tensor.
So this is called the linear thermal expansion coefficient tensor.
And it is a symmetric tensor by definition since a measure of true strain is by definition a symmetric tensor.
So this is the linear thermal expansion coefficient tensor
Quick question
Yes
[INAUDIBLE] the temperature gradient [INAUDIBLE] and talk about the directionality say between here and here because of the temperature [INAUDIBLE]?
Oh, yeah.
Sure.
Sure if you wanted to do that.
That assumes that the thermal conductivity would be very small so that the temperature inside the body would not attempt to [INAUDIBLE].
Yeah.
Yeah.
In the same way that you could create strains that are in homogeneous as well.
We'll now for the first time start talking not in terms of abstractions but in terms of some real physical properties.
What is the range of linear thermal expansion coefficient tensors?
And unlike many physical properties, that's an easy one to answer, 10 to the minus 6 degree C per degree C. Materials have a very, very small range linear thermal expansion coefficients.
And let me give you some examples for real materials.
They go up to maybe 10 to the minus 5 for a very soft weakly bonded materials.
But the for metals-- and I'm giving you a single number now because these are averages for polycrystalline materials-- the value of A times 10 to the 6.
For lead, a low- melting metal, is 28, 2.8 times 10 to the minus 5.
For copper, 18.
For iron, 12.
Four tungsten, a very refractory metal, 5.
And for diamond, a very strongly bonded materials, it's 0.89 times 10 minus 6.
If we look at compounds, again, the average linear thermal expansion coefficient times 10 to the 6 for polycrystalline material.
For aluminum bromide, hardly a technology material of commerce, but in light of this because this has one of the largest linear thermal expansion coefficients of any material, very weakly bonded to compound.
For a more typical ionic compound, NaCL, 40 times 10 to the minus 6.
For calcium fluoride, this has a bivalent cation in it, so the material is stronger, more strongly bonded, 20.
For MgO, both a divalent cation and the divalent anion, as you would expect, the thermal expansion coefficient drop still more.
For Al2O3, a trivalent cation that goes down to 8.8, and that's really an average because this is a hexagonal material.
And a second-rank property tensor for a hexagonal crystal has to have two equal diagonal elements and one independent diagonal element when you diagonalize the tensor.
Spinal, MgAl2O4 this is representative of all of the ferrites for example, 7.6.
Silicon carbide, a very refractory covalent compound, 4.7.
For glasses, here's a surprise.
You might think that a glass might have about the same linear expansion coefficient as the crystalline form of the same composition.
But a sodium calcium silicon oxide glass, so called soda-lime silicate glass, is 9, fairly low.
A borosilicate glass, like the Vicor that we love to make glass apparatus out of, 4.5.
And one of the near record holder is fused silica, silicio glass, and here a measly value of 0.5.
So glasses have very low linear thermal expansion coefficients compared to crystallize materials that are predominantly ionic.
And the reason lies in the nature of the structure.
Something like MgO has ordered ions, and when you heat it up, it expands isotropically.
A glass is this random network of edge-shared polyhedra.
So when you heat it up, yes, the dimensions of the polyhedra increase in proportion to the change of temperature.
But the framework because it's so meandering and open can buckle to accommodate the increased size of the tetrahedra that are in the linkage.
So the result is the network buckles, but the overall microscopic thermal expansion is very, very slight even though the tetrahedra are changing dimensions in the same degree that they are in these crystalline inorganic nonmetallic materials.
Let me give you a handout not that it's up to date, but there was a very nice, convenient one-page article in the Journal of the American Ceramic Society a number of years ago that lists thermal expansion coefficients for low expansion oxides.
So let me pass that around and let you take off for copy with that.
And that gives some examples of the value of this property for, again, for polycrystalline materials, which if you looked at a single crystal would show anisotropy.
And here you'll find a few materials in the list near the top they're arranged in order of increasing linear thermal expansion, you'll find some that are just about zero when you have a polycrystalline form of the material.
And you'll have some that have this very unusual behavior where the linear thermal expansion coefficient is actually negative.
So let me pass this around for reference.
Now this may seem to be a curiosity that you can fine for a polycrystalline material for which the individual grains expand anisotropically a bulk linear thermal expansion coefficient of zero.
Why should one care other than that being a curious result?
Can anybody guess?
Can you say that again?
That these are materials which in a polycrystalline form with random grain orientation have a linear thermal expansion coefficient this is essentially zero?
[INAUDIBLE]?
Yes.
Exactly.
The materials out of which you build furnace linings and tanks for melting glass or smelting steel have to be made out of polycrystalline materials.
You can't have a monolithic single crystal that's large enough to melt a significant amount of glass in for example.
And being polycrystalline, when the material expands, there is a great deal of stress between neighboring grains because they're randomly oriented.
If the bulk microscopic thermal expansion coefficient is zero, that is going to be refractory that is very, very resistant to thermal shock.
When you heat it up, you don't find intergranular cracking.
So the materials that have essentially zero spatially average thermal expansion coefficients are very attractive for refractories.
You can pick up any issue of the Journal of the American Ceramics-- not the journal, but the Bulletin of the American Ceramic Society and you will find refractory companies hustling materials like Cordierite, which is a silicate material that on average has one positive thermal expansion coefficient, one negative thermal expansion coefficient.
And the volume change of the individual grains is essentially zero.
So that makes a dandy material for refractories.
Cordierite has another interesting property that is a really nice example of anisotropy of a physical property.
If I ask you to rattle off some properties which you know to be very anisotropic, one of them that you wouldn't think of is color.
Your t-shirt is pink, so how can that be a function of direction?
Well your t-shirt is not a single crystal.
And their single crystals which if you hold them up to light and pass through a plain polarized beam of light the crystal has a very different color for one direction of polarization then another.
And these colors, interestingly, very often are strikingly different.
They're crystals that are green in light polarized in one direction and red for light polarized in the opposite direction.
And there is in the old Icelandic sagas-- few people have heard of them.
The Icelandic sagas which were written in about the year 500 are one of the early forms of Western literature that is really great, enduring world-class literature.
In the sagas, there's one episode where Thor is sleeping and Olaf comes along and steals his sunstone.
And Thor is upset he takes his battle as and cut Olaf's head in twain.
And people puzzled for hundreds of years, what is sunstone.
This doesn't compute in today's age.
And finally a Norwegian archaeologist came up with a hypothesis that in Iceland there are remarkably large and remarkably transparent and perfect single crystals of Cordierite.
And Cordierite is strikingly pleochroic.
That's color that is a function of direction.
And what this archaeologist theorized is that the Vikings would take a sunstone when they went sailing on the North Sea, which is notoriously damp and overcast and gray, and your principal means of navigation was the sun.
But when the sky was overcast, you couldn't see the sun.
But the sun as it comes through the cloud gives you light that's very strongly polarized.
So the theory is that the Vikings would have very perfect crystals of transparent Cordierite, would hold them up to the sky, and turn them around until the Cordierite crystal lit up a bright, golden sunny yellow.
What better name for the sunstone.
You found the sun from the direction of polarization of the light scattered from the cloud when you've got the crystal oriented just so.
In a different orientation, it would look murkier, look sort of a bluish purple for the other principal direction of the birefringence.
So there is putting crystal anisotropy hundreds of years ago to a very useful purpose to navigate on cloudy days.
Unfortunately, the Vikings used it to sail down the British coast and sacked the next village.
So it shows you that even the most simple of science can be put to application in war research.
Anyway, won't go there anymore.
So index of refraction and color can also be a strong function of direction The numbers that I put on the blackboard suggest that the magnitude of thermal expansion coefficients are influenced very strongly by the strength of the interatomic bonding.
And this shows up in a very, very striking way if you group together classes of comparable materials.
And if you plot the value of the linear thermal expansion coefficient in units of 10 to the minus 6 per degree C as a function of the melting point in degrees Kelvin, the numbers range from about 400 K for materials like sulfur up to about 3,700 for tungsten.
And the variation is so beautiful it could make you cry.
It goes as the inverse of the melting point.
And way up here are materials like for sulfur and lithium.
And the way out here are materials like tungsten.
And their relation is given by a equals 0.020 over the melting point in degrees Kelvin.
So very strong correlation between melting point and linear expansion for simple elements.
If you do the same thing for compounds, you find a similar parabolic relation except it is offset.
For oxides and halides, many of them with the rock salt structure, so they are isotropic.
And again, if you plot here the melting point in degrees Kelvin, the numbers here range up to 60 times 10 to the minus 6.
And again, a variation with 1 over T, but offset from the origin.
And the curve here is that a equals 0.038 over T, the melting point T sub m in Kelvin, minus 7.0 times 10 to the minus 6.
Finally, let me finish with some data for single crystals that provide examples of and anisotropy.
These are all hexagonal materials.
And I will give you the-- not all hexagonal.
I take that back.
But they're all uniaxial crystals.
The value of the thermal expansion coefficient along C and the thermal expansion coefficient that is parallel to C, so these are the elements that would be in the diagonal isothermal expansion tensor.
So for Al2O3, which is a hexagonal rhombohedral oxide, the two expansion coefficients are 8.39 and 9.0 times 10 to the minus 6, not terribly anisotropic.
But the structure of alumina is a close-packed arrangement of oxygen with aluminum filling 2/3 of the available octahedral holes.
So it's hexagonal only because of the sites that are filled in the array.
For TiO2, which is [INAUDIBLE] and therefore also uniaxial, 6.8 perpendicular to C; 8.3 parallel to C. For zirconium silicate, 3.7 and 6.2, almost a 2:1 difference.
For the quartz form of silica, which is hexagonal, 14 perpendicular to C; 9 parallel to C. For carbon, the graphite form, a layer structure-- you can almost guess how this is going to turn out.
It's going to be humongous perpendicular to the lawyers and very small within the plane of these tightly bonded hexagonal nets.
And that is indeed the case, 1 perpendicular to C; 27 parallel to C. So there is an anisotropy of a factor of 27.
A few more.
Aluminum titanate, perpendicular to C minus 2.6; parallel to C 11.5.
A very well known example of anisotropy, extreme anisotropy, where one principal coefficient is positive and two are negative is the calcite forms calcium carbonate, minus 6 and 25.
For zinc metal, a hexagonal close-packed metal, surprising degree of anisotropy, 14 and 64.
And for tellurium, which is a chain structure like selenium, 27 perpendicular to C and minus 1.6 in a direction parallel to C. Even simple hexagonal close-packed metals can do strange things at low temperatures.
For zinc, the expansion coefficients perpendicular to C and parallel to C as a function of temperature go at 300 degrees C from 13 and 64.
At 150, the values have dropped to 8 and 65.
At 60 degrees K perpendicular to C, the thermal expansion is negative and has not dropped terribly much of at all parallel to C. So this is a hexagonal close-packed structure lot.
A lot of action and strange things going on in the plane at the close-packed layers, but not very much change in a direction perpendicular to the layers.
Let me raise one more issue.
If we were to look at some of these strangely anisotropic materials which had negative thermal expansion coefficients in one direction and positive in another, what would the representation quadric look like?
Suppose we looked at calcite, for example, which has a large negative thermal expansion coefficient.
And if we look at the thermal expansion quadric when the tensor was referred to the principal axes, parallel to C-- well, the thermal expansion tensor aij along its principal axes would be along A1 in a direction perpendicular to C, it would be for the data given here, minus 6, zero, zero, zero, minus 6, zero, zero, zero.
And A33 is 25.
So that's very, very anisotropic.
But what is this going to be?
This is going to be a quadric that has the shape of an hyperboloid of two sheets.
So it's going to look like this.
And there'll be an asymptote like this and then asymptote like this.
Along the asymptote, the radius of the quadric is infinite.
So therefore along these directions the strain is equal to zero.
Strain goes as 1 over the radius squared.
In this range of directions here, the radius is imaginary.
But remember that the strain is given by 1 over the radius squared.
So this says that the strain is negative, so the material has contracted.
And finally, in this range of directions where the radius is finite and positive, if the value of strain is 1 over radius squared, this says that the maximum deformation is along the direction of the C axis, and it's positive.
The material expands along C; the minimum radius of the quadric corresponds to the maximum thermal expansion.
So let me close with a question.
Along the asymptote of the quadric, the strain is zero.
Does this mean that this direction in a crystal of calcite does not move, no deformation at all when you heat it up?
I see faint shakes of the head.
I don't know if that's just in awe of what's going on here or an opinion on the question.
Do you think it'll be no strain?
It would be [INAUDIBLE]
Good answer.
Let's remember a property of the representation quadric.
The direction of what happens is normal to the surface of the quadric.
The thing that is happening here is the displacement; ui is given by epsilon ij times U sub j.
So when we look in this direction, the displacement, which would be normal to the surface of the quadric, gives us this as a displacement.
So along the asymptotes of the quadric, you can't say that there's no deformation, but that the deformation corresponds to pure rotation and not any fractional change of length.
In these directions, the length changes, but it is a decrease in length within the asymptotes.
For those directions, the strain is an extension.
And again, if you want to know the direction of the displacement, you find the normal to the surface of the quadric in the direction of interest, and that's the direction in which the radius vector points.
All right.
That's I think a good place to quit.
I think I'll say a little bit more after the break about the atomistic reason for increases in interionic separations, which we can get some insight into from a simple model.
But before we disperse, I don't know if everybody got a copy of problem set 13.
If anybody didn't, there's some extra copies up here.
But I will with great pleasure pass out to you problems set number 14, which has served two purposes.
One is to have you show for yourself that the trace of a second-rank tensor-- I called it second-order tensor-- second-rank tensor the sum of the diagonal elements is invariant for a change of reference axes for a general direction cosine scheme for the change of axes.
And then the other two questions are to give you some practice in diagonalizing a second-rank property tensor which is not in diagonal form.
And I asked you to do this in two ways.
One is by direct solution of the secular equation and finding the eigenvectors.
And then the third example, which is for a completely general tensor, to do this by the method of successive approximations.
And you'll find after a couple of iterations you get fairly close to convergence.
So I shall pass this around.
And again, I think there's something to be learned from that.
Let's take our stretch then.
I'm sure that lecture involved a lot of stress and strain.
Ha, ha.
Let's resume in 10 minutes.
And we'll then move on to some additional higher rank tensor properties, which will be very, very interesting.
Go for it.
Any questions before we push bravely onward?
OK if there's no question on any of this I shall clean off the board.
And the last thing I wanted to do in connection with thermal expansion is to examine how the macroscopic linear thermal expansion coefficient relates to the nature of the potential minimum between-- in the attraction between-- neighboring atoms.
So let me suppose we have a pair of atoms separated by some distance, d, and there is a cohesive force between them or else the material would not be in a solid state.
And the nature of the potential-- and I'm going to call the distance between the atoms, x-- and this is going to be the energy.
And I'm measuring now relative to-- let me call this d as I did over here-- so the energy as a function of d is going to consist of a very short range, repulsive interaction, that rises very, very rapidly when you attempt to squeeze the atoms any closer than a fairly small value.
And then there'll be some sort of cohesive energy.
And everybody loves to model ionic compounds because in an ionic compound this cohesive force would be purely a Coulombic force that would be, in rationalized MKS units, 1 over 4 pi epsilon zero times the product of the charges and divided by the distance between them.
Taking those two potentials together, and you've all seen this many times before, there is a minimum in the energy.
And then absolute zero-- that is the separation at which this atom pair would sit-- and this would be the binding energy of that ionic pair.
If you carry this through to get the entire energy of the crystal, you find that the energy of the crystal goes as 1 minus 1 over-- haven't introduced this term yet, so let's hold off on that.
What I'm going to do now is to take a look at this potential and expand it around the equilibrium separation just with an empirical collection of terms.
But from the shape of this well, we can see why the bond length should increase as the energy of the atomic pair increases.
At absolute zero, the separation will be this equilibrium separation, d zero.
But as you increase the temperature and pump vibrational energy into this pair of atoms, they will get a certain increase in energy.
And as a result, the pair will be able to vibrate between these two limits.
Put in some more thermal energy by raising the temperature, and the pair will be able to vibrate between these limits-- larger limits.
And so the average position of the atom, the separation of the atoms, is something that is going to increase with increasing temperature.
And that is purely a consequence of the asymmetry in this potential well.
If this were a Coulombic bond, we know exactly how to model the attractive part.
The repulsive part is a little harder to model because that is something that's inherently quantum mechanical.
As the ion pair gets close together, the electrons start to perturb each other's orbitals.
And a very common thing to do is to model this as 1 over d to the sum power n. And n is typically on the order of 8 to 12.
So this is something that rises very, very steeply.
I was once at the Gordon conference where a fist fight almost broke out because somebody was giving a presentation and started from the beginning by introducing this model to the potential well.
And he says, "As usual, we'll write this as a power law, 1 over d to the n." And somebody in a front row jumped up and said, "what do you mean usually?
Sensible people use exponentials."
And the speaker said, "No, I'm going to use a power law."
That's dumb.
Exponentials work better.
Power laws, exponentials, power laws, exponentials.
But in point of fact they're both grossly empirical and one is not better than the other.
The algebra changes a little bit, but people get very, very emotionally attached to their physical models
[INAUDIBLE] law in the exponential
Good, you should've been there to break up the fight.
If you'd gotten between them, you'd have been likely punched in the nose.
[INAUDIBLE] OK so I'm going to use the power law because as we've seen they're the same thing as exponentials.
So now what I'm going to do is expand this in a purely empirical way-- measuring distance from the minimum of the potential well.
So what I'm going to say is that the potential as a function of x-- and now I won't call it d because I'm going to measure distance, x, from the minimum in this well.
I'm going to say V is a function of x.
There is a well in which the-- which keeps the atoms separate.
I'll assume that is a parabolic potential well, so it'll go as first some constant c times x squared.
Then the thing that causes thermal expansion is the asymmetry.
And I want the asymmetry to increase the energy more rapidly on the side of negative x than on the side of positive x.
So I'll put in a term, g x cubed.
And I put a negative sign in here because I want it to go up as x becomes negative.
That means the ion pairs are decreasing at separation.
And finally, a very fine touch, if x becomes very large, the potential well should soften.
The edges of this are starting to level out and get lower than the purely parabolic part for very large separations.
So that's what I'll use as a very simple model for my potential.
This is what gives the asymmetry, and this is what gives softening.
And now I'm going to ask, what is the average value of x?
The probability of finding a particular x is going to be proportional to e to the minus V of x over kt.
So I can say that the average value of x is going to be the integral from minus infinity to plus infinity of the value that I'm trying to find the average of, namely x times the probability of finding that x.
And that's going to e to the minus V of x over kt.
And I will integrate that over x.
And then I will normalize by a term which is the integral of the probability from minus infinity to plus infinity of e to the minus V of x over kt.
So that's a general expression for an average.
It's the thing you want to average times the probability of finding the particular value divided by the integral of the probabilities.
OK now let me do some very straightforward things and I'm just going to rattle these off very quickly.
We'll expand this in terms of a series of terms.
And I'm going to expand both the numerator and denominator using the approximation that e to the u is approximately equal to 1 plus u.
So the numerator-- x times e to the minus cx squared over kt.
I'm going to take the higher order terms and say that e to the gx cubed over kt times e to the minus-- this is a minus sign out in front-- times e to the fx fourth over kt, can be approximated by, since these are very, very small terms, 1 plus g x cubed over kt times 1 plus fx fourth over kt.
So that's going to be, if I put an x e to the minus cx squared, I'm going to have a term 1 plus g x cubed over kt, plus f x fourth over kt.
And then a very, very minuscule term, f times g times x to the 7th over kt.
And I'm going to throw that out.
If I look at these integrals one at a time, the integral from minus infinity to plus infinity of x e to the minus cx squared over k t, that's the first term in expansion, times dx.
That's going to be zero.
Because for positive x, the thing that we're integrating is plus.
For negative x, the thing that we're integrating has the same exponential but has a minus sign.
So that integral is zero.
So the next term I want to look at is x times gx cubed over kt.
And for this I turned to my handy book of integrals.
The integral from zero to infinity of x to the 2n, where that quantity is even, times e to the minus a x squared dx, is equal to, as you all know, 1 times 3 times 5 all the way up to 2n minus 1 divided by 2 to the n plus 1 times a to the n times the square root of pi over a.
And proof of that the integral is left as an exercise to the reckless student.
Yes?
The x cubed [INAUDIBLE]
That's why this is a 2n here.
So I can say that the x cubed term is--but we don't have an x cubed term.
The next term is going to be equal to x fourth.
And this I'm going to drop because that's going to be x fifth.
So the integral from minus infinity to plus infinity of x to the fourth power times g over kt times e to the minus cx squared over kt dx.
If I use this formula, turns out to be a very, very simple thing.
It turns out to be 3/4 times the square root of pi times g times kt over c to the 5/2.
And kt is to the power 3/2.
So that is-- if I throw away the term in x times x fourth which is odd, that will vanish and I've then thrown out only a really large term in x to the 7th.
For the denominator, the integral that is there before us is the integral from minus infinity to plus infinity of e to the minus cx squared over kt times 1 plus g x cubed over kt.
This thing is going to vanish because x is odd.
And then I'll have plus fx fourth over kt.
That will not vanish but it's going to be a relatively small term, so I'll go like that.
Which means I just have to integrate this Gaussian factor.
And that turns out to be equal to 2 times 1/2 times square root of pi kT pi times T over c. So this is the denominator.
And if I take the ratio of those two integrals, I have that the average value of x will be equal to 3/4 square root of pi times g times kt to the 3/2 over c to the 5/2, divided by the square root of pi times kt over c to the power 1/2.
And if I carry through the algebra correctly, this turns out to be 3/4 of g over c squared times kt.
So there is the average distance between this pair of atoms as a function of temperature in terms of characteristics of the potential.
The first thing that is very gratifying is that the bond length, according to the simple model, should increase linearly with temperature.
So it tells us we would expect things to expand linearly with temperature which corresponds to reality.
The next thing we'll see is that the amount of expansion goes inversely as the square.
That's a strong dependence-- inversely as the square of the parabolic part of our potential.
That was the c-- c was the coefficient in front of cx squared in our potential.
So the stronger the bond, the smaller the thermal expansion coefficient is in proportion to the reciprocal of the square of that term.
And then finally, the last important term to survive is g, the asymmetry.
And the thermal expansion coefficient should increase with the first power of the term that describes the asymmetry of the potential well.
So your intuition would tell you that that's the way the thermal-- linear thermal expansion coefficient should vary.
The interesting thing is that it depends more strongly on the parameter that gives the parabolic part of the potential well.
And secondly, that it does predict a thermal expansion coefficient that increases-- the separation that increases with-- in proportion to temperature.
And therefore there is a constant linear thermal expansion coefficient.
Now all this is true at elevated temperatures.
At low temperatures, thermodynamics kicks in.
And it turns out that the linear thermal expansion coefficient, as does energies of vibrating atoms, must go to zero as temperature approaches absolute zero.
So a typical variation of the thermal expansion coefficient with temperature is something that does this.
The same way as-- the same sort of variation as things like heat capacity.
And this temperature in here is on the order of the Debye temperature of the particular material.
All right, so much for linear thermal expansion.
And I would now like to turn to something much more interesting.
And that is physical properties that involve property tensors of rank higher than two.
One of the important and more interesting of these is a property called piezoelectricity.
Piezo means pressure, so the property literally means pressure electricity or pressure induced charge.
Now what is the charge?
First of all, this does not describe a property where-- if I were to pick up a chunk of crystal, and I drop it because I just got an electric charge.
These are not mobile charges, these are bound charges.
And the reason for the induced charge is the fact that we are squeezing or stretching an ionic material, and that's moving charged ions around.
So what we're going to do if that process involves motion of charged ions is to induce a dipole moment per unit volume.
I have a little primer on some quantities in electromagnetic theory and I-- I brought them with me upstairs to look them over and I forgot to bring them down.
But I don't think we'll need them in what's left of this hour.
So let me say that for matter and bulk, the quantity that is involved in piezoelectricity is the polarization.
And polarization is dipole moment per unit volume.
And a dipole moment, to refresh your memory, is that if I have a positive charge plus q and negative charge minus q separated by a distance, d. The dipole moment, which I'll indicate by a lowercase p, has a vector quality and it's equal to the magnitude of one of these charges times the distance between them.
And we'll define the distance as going from the positive charge to the negative charge.
That's a crazy way of defining a pair of charges of equal but opposite sign.
The reason for defining a quantity like the dipole moment is that this combination of charge and separation of the charges comes up in all sorts of problems in electrostatics and electrodynamics.
And this is the delight of people who teach elementary physics.
You are given, on the first quiz, a question that says-- calculate the electric field that is created by a set of dipoles on the Great Court which spell out MIT.
And we can come up with all sorts of crazy configurations of dipoles and then calculate the field that they produce.
And the field turns out to be something that always involves the product of charge and separation in a vector sense.
Probably the best illustration of this would be if I have a dipole charge plus q, charge minus q, separated by d. And I put it in the electric field.
The electric field, which is defined as the force per unit charge on a positive charge is going to pull the positive charge this way.
And it's going to pull the negative charge this way.
And what that's going to do is to place a torque on the dipole.
And that torque is going to be equal to a vector product of the electric field and a vector.
You can express it in terms of the separation of charges defined in a vector sense going from the positive charge to the negative charge.
So there's an example of how the torque on a dipole directly involves the product of charge and the separation between them and the angle between the field and the dipole.
So it's a useful quality and, without further apology, we will use it in our discussion.
Now polarization-- dipole moment per unit volume-- is just going to be the dipole moment of one dipole pair times the number of them per unit volume.
And that will be a macroscopic measure of polarization.
OK, if we subject a piece of material that contains ions to a stress- so here is a stress tensor, sigma ij.
Let me call it jk.
We will create reorientation of the dipoles.
We will have a polarization, a dipole moment per unit volume.
This will be a vector because there will be some net dipole moment per unit volume.
And that will just be the sum of all the little individual dipole moments on the ions-- ion pairs.
And if the stress is not too large, every component of polarization is going to be given by a linear combination of every one of the components of stress.
So we will have an array of coefficients, d ijk The i goes with the P and the j and k goes with the elements of stress.
This is something that's called the direct piezoelectric effect.
It's the basis of a lot of every day devices-- pick ups on the old records which nobody uses anymore, cigarette lighter which nobody uses anymore because everybody's quit smoking-- but worked the cigarette lighter, in some of the more modern incarnations of the lighter, was the piezoelectric effect.
It's important to note that the charges that are involved in the piezoelectric effect are bound charges.
Again you can't get a shock by picking up a piece of a strongly piezoelectric material like quartz.
Might say then, well if there's no spark, how could a piezoelectric material in a cigarette lighter cause the lighter fuel to ignite.
The charge doesn't move.
How could there be a spark?
How could the lighter fluid be ignited?
Anybody got an idea?
Charge can induce current flow in circuit elements that are removed from the charge.
So if you have a little capacitor in the lighter that is near the piezoelectric element that will develop the charge, you can induce current flow in a circuit as a result of that capacitor experiencing the electric charge of the piezoelectric induced charge.
And that's how the little piezoelectric cigarette lighters worked.
OK, this stress-- array of stress elements, nine of them, constitutes a tensor of second rank.
Polarization is a vector per unit volume.
This is a tensor of first rank.
This is a tensor of second rank.
Therefore it follows that this array of coefficients, d ijk will be three equations involving all nine elements of stress.
So it'll be a total of 27 coefficients in the array.
And this array constitutes a tensor of the third rank.
So this is clearly a property tensor which relates a generalized force-- the stress tensor to a generalized displacement, which is the dipole moment per unit volume.
Since this is a property of a crystal, that tensor is subject to symmetry restrictions.
In particular, if there is some direction, cosine scheme Cij, that corresponds to the operation of a symmetry element, then it follows that if we evaluate the new tensor elements, d ijk prime in terms of the old elements, that is going to be a transformation of the form Cil, Cjm, Ckn times all 27 of the original tensor elements, d lmn.
And the l, m, and n are dummy indices, just as with our second rank tensors.
And the ijk are real indices that go with the subscripts on the particular tensor element.
So there are 27 coefficients.
And each transformed element is going to be a sum therefore of 27 coefficients times-- each times three direction cosines.
So each transform tensor element is going to be given by a sum of 3 times 27 quantities.
And to do the complete transformation would involve doing it 27 times.
So 3 times 27 squared is the number of terms that we're going to have to write down to transform the entire tensor.
Not a terribly encouraging prospect unless you use the method of direct inspection, namely that the tensor transforms like the product of the corresponding coordinates.
And then it works fairly quickly, particularly for symmetry transformations, where there are a lot of zeros in the direction cosine scheme.
So we're going to have to do this for each of the crystal-- crystal point groups.
So we're going to have to do this exercise of 3 times 27 times 27 times the 32 point groups that exist, which becomes an even more daunting task.
But let's do one such transformation and let me show you that it, in fact, it's going to go fairly easily.
So let me follow the tracks of what we did with second rank tensors and examine the restrictions imposed by inversion.
OK so the transformation of the axes, as we've seen before, that's produced by 1 bar.
That's going to take x1 and change its direction to minus x1.
It's going to take x2 and change its sense to an x2 prime that points in this direction.
And take x3 and change its sense to something that points down here.
So that's x3 prime.
So the relation between the axes, before and after inversion, is that x1 prime equals x1-- minus x1-- x2 prime is equal to minus x2.
And x3 prime is equal to minus x3.
And we've seen this before.
It's deja vu all over again.
So Cij, the direction cosine scheme, is equal to minus 1, 0, 0.
0, minus 1, 0.
0, 0, minus 1.
So let's transform a representative tensor element for a change of axes that corresponds to that direction cosine scheme.
And that's the same as physically inverting the crystal and demanding that the property be unchanged if that is a symmetry transformation which leaves the crystal invariant.
So if we look at some d ijk prime, that's going to be Cil, Cjm, Ckn, d lmn, where this is a sum over the index l, the dummy index m, and the double index n, from 1 to 3.
If we sum over Ci something, the only term of the form Ci something, regardless of what i is, is going to be the diagonal term Cii, the diagonal direction cosine Cii.
In other words, if i were 1, the only term of the form C1 something that is non-zero, is C11.
If I sum over m, the only term that is non-zero is C, with m equal to j.
So this is going to be Cjj.
And if I sum over n, the only term that is going to be left is Ckk.
And all this will be times d ijk, the single tensor element.
Regardless of the values of i, j, and k, they are all equal to minus 1.
So the operation then of inversion, then says that for the transformed crystal, d ijk prime for any i, j, and k, is going to be equal to minus d ijk.
But if this is a symmetry transformation, the tensor element has to be the same before and after.
And the only number which can be equal to the negative of itself is zero.
So we have, with one swell foop, transformed 27th tensor elements-- each one of them a sum of 27 elements in the original tensor.
So the conclusion is, if a crystal has the operation of inversion, all d ijk vanish.
What does that mean?
Means the property just doesn't exist.
So no crystal which has an inversion center can display piezoelectricity.
Which means that none of the 11 Laue groups can be piezoelectric.
And the only point groups for crystals that can be piezoelectric are the 32 minus 11 equals 21-- and this isn't even one of my good days-- acentric point groups-- the noncentrosymmetric point groups.
And, these, in principle, can be all different.
So we can't get away with just looking at the restrictions of Laue groups as we did with second rank tensors.
We're going to have to look at every one of the point groups that have no inversion center.
And these can, in principle, be different, and they usually are.
Let me give you a physically-- physical feeling for why the piezoelectric effect cannot exist for a crystal that has an inversion center in it.
Let's suppose we cut a wafer from our single crystal, regardless of orientation.
And we subject it to a compressive stress.
We will see-- and I've got a primer on some simple electrostatics that may be useful-- that a polarization in the x1 direction manifests itself as a charge per unit area on the surface and an opposite charge on the other surface.
So P1 corresponds, numerically and physically, to a charge per unit area on a surface that's normal to x1.
OK now let's suppose that that crystal has an inversion center in it.
And that means we should be able to invert the crystal and measure exactly the same charge per unit area.
Well, if this is surface a, and this is surface b, we can imagine ourselves doing the thought experiment in which we invert the crystal.
So this surface becomes b and this surface becomes a.
And we apply the same sigma along x1-- a compressive stress.
It's really doing exactly the same thing that we were doing before-- inverting the crystal.
And now we say that the property has to stay the same.
So now we are demanding that surface b have a positive charge per unit area, and surface a develop a negative charge per unit area-- completely the reverse of what happened here.
But yet what we're doing is just squeezing the plate.
The crystal can't tell which side is up, which side is down.
We're applying the same compressive effect.
So these two distributions of charge are not the same and can be the same only if that charge is zero.
So physically we can see why the crystal can't be piezoelectric.
If you don't like to invert the crystal-- if you can't do that physically-- one thing we can do is leave the crystal alone and invert the compressive stress.
So this is sigma top and this is sigma bottom which is exactly equal to it but opposite in direction.
If we invert the stress relative-- relative to the crystal-- then what we've done is to take sigma top-- leave the crystal alone-- so this is now the sigma that was squishing the top.
Here's the sigma that was squishing the bottom and well, that's clearly doing the same thing.
So inverting the stress, relative to the crystal, is the same as inverting the crystal relative to the stress.
And nothing is supposed to happen but clearly the sign of the charge is going to be reversed.
So that just won't work.
The charge has to disappear.
OK this is the direct piezoelectric effect.
It can exist only for crystals that lack an inversion center.
Let me mention some other piezoelectric effects.
So we mentioned that a polarization, given by a third rank tensor times an applied stress, sigma ij, is the direct piezoelectric photoelectric effect.
However, if we have a stress, we also have a strain.
So another piezoelectric effect, which is not dignified with a name, is that we can apply a stress that creates a given state of strain.
And we again, we'll get a polarization per unit-- a charge per unit area of polarization.
And this is a set of coefficients which are represented by e. And we can write a tensor relation of this sort-- that components of polarization, charge per unit area, normal to axis xi, is given by a linear combination of every one of the components of strain.
This is something that is not dignified by any name.
And clearly, the piezoelectric coefficients, e, have to be related to the piezoelectric coefficients, d, by the elastic constant somehow-- unless we are doing the same thing.
We've got to create a stress, operationally, in order to create the strain.
So a stress creates a strain.
And that has to describe the same polarization if we've done the same thing.
So the e's and the d's have to be related.
Another piezoelectric effect involves a applying an electric field.
And if we apply an electric field, we will in general create a strain.
This direct piezoelectric effect is three equations that involve a linear combination of nine elements of stress.
This piezoelectric relation here is going to involve writing an equation for each of nine elements of strain.
And there would be three components of the electric field vector.
So it'll be three this way.
So both of these relations involve 27 elements.
So again we're going to have a third rank tensor because the electric field is a vector-- strain is a tensor.
And now the mind boggling thing that I will leave you with is exactly the same set of coefficients, d ij describe both effects.
And this effect is given a special name.
This is called the converse piezoelectric effect.
And it's not at all clear how strain, in terms of electric field, should be described in terms of the same array of 27 numbers as polarization per unit volume in terms of stress.
And that is an argument that is based on thermodynamics.
So that's a good place to quit.
We'll have some fun in looking at a few symmetry restrictions imposed on the acentric point groups, 21 of them.
We're not going to do all 21.
If you know how to do one or two, and marvel over the results, you can do them all.
But one thing that we'll be asking is how can we represent the variation of the piezoelectric effect with orientation of the crystal.
And the answer is we can't.
We can't.
Because what is the direction of what we're doing?
There are nine elements of stress, not just three components of a vector which defines an orientation.
There are nine parameters here.
And there are three components to the charge per unit area-- that's a vector.
So all that one can do for these higher ranked tensors is to look for a special, generalized force.
And then perhaps for a very special form of that generalized force, that tensor, look at how the polarization changes.
But you're going to have to look at six different representation surfaces-- each one which will describe a different piezoelectric effect.
So this will become less simple in many respects.
But correspondingly, more interesting because we'll have some really wild, mind boggling representation surfaces for piezoelectric effects.
OK so we'll stop there and resume on Thursday.
And I'll let you get back to the MRS if that's where you earlier today.
I think it's about five after the hour so we ought to get started.
Before we forge bravely ahead, I'd like to make sure that everybody has a copy of the things that we've been handing out.
There is a copy of the handwritten notes on the derivation of the 17 plane groups.
Need a copy of that?
There you go.
And then there is, in addition, a set of diagrams from the international tables that give very nice pictures of all of the plane groups in the arrangement of symmetry elements.
That's the only other thing that was handed out up to this point.
For your continued edification and amusement, I have another problem set.
And let me say something about this problem set.
The first problem gives you some angles between crystal faces and asks you to deduce a possible set of lattice translations which are consistent with those angles.
And in the days before diffraction, that was what crystallography was all about.
It really was mapping the geometry of crystals.
And the interesting thing is, you could measure these angles very precisely with a device called a reflecting goniometer, Provided you had a crystal with nice, shiny faces on it.
You had a two circle instrument, and you could adjust the crystal so that a light beam, a very finely collimated beam of light, was focused into an eyepiece.
And you could measure angles to within not one minute, but one second of arc.
And from this you could deduce if you could assign Miller Indices to the faces.
You could deduce not the absolute magnitudes of translations, but you could deduce the ratio of them.
And along came x-ray diffraction and shook everything up.
Turns out, sometimes you were right and other times you were wrong, because you had determined a self consistent set of indices, but not the ones that were correct.
So this is to give you a little look at the early days of crystallography, and to see if using the way Miller Indices are defined you can calculate the ratio of axes.
Not terribly demanding, but worth doing.
And we had talked earlier about these combinations of symmetry elements as constituting the elements of a group.
And in the second problem I ask you to take a not terrifically simple, but a fairly complex point group, 4mm, and show that, indeed, the operations that are present satisfy the group postulates.
And that is worth doing once so that you convince yourself these really are groups.
And then the third problem is easy to state and it is diabolically tricky.
So I'll let you have a go at it, but don't beat your brains against your desktop for an entire evening over it.
Try it.
If you don't see the solution to it, all will be revealed later on in class.
But it's based on the fact that the plane groups, which we've now derived, 17 of them, these can be viewed as the base level of a three dimensional space group.
And for each of them, you could take another translation, t3, that was perpendicular to the plane of the group, and just imagine all of those rotation axes and all of those mirror planes not being two dimensional operations, but three dimensional operations.
The only one where you could pick the translation, generally, would be the oblique lattice, p1, no symmetry at all.
And there you could pick the translation in any orientation you wished.
But for each one of the 17 two dimensional plane groups, there is a corresponding three dimensional space group.
So we got 17 three dimensional symmetries for free without really doing any additional work.
However, there's one thing that one should not gloss over.
We found that a rotation point in a two dimensional plane allowed only a very limited number of lattices.
But we were not thinking of things in terms of three dimensions.
So what I'm inviting you to do in the third problem is to consider, now, that construction that we did where we showed that rotation angles were restricted to values of cosine of alpha equals 1 minus p over 2.
Now generalize that to a three dimensional situation, where the translation does not have to be perpendicular to the rotation axis, but can be inclined to it.
Are there additional rotation axes allowed?
Are their fewer?
Something we ought to examine.
But there's a little bit of ingenuity that you have to use in that proof.
But I invite you to have a go at it on your own.
So with that tantalizing introduction that will want to send you running home and start it as soon as we finish our class this afternoon, let me hand out, without any further blather, problem set number five.
Pass that in the back corner.
Last time, without saying much about them, I handed out this extract from the international tables that summarized the 17 two dimensional plane groups and their properties.
And I'd like to spend a few minutes going over the considerable information that's contained on these pages.
Each one is treated in the same way, and let me start with one that's almost trivially simple, and that's the two-fold axis in the oblique net.
Across the top of the page you find, in boldface, the symbol for the plane group.
And then, just counting in order of increasing symmetry, the number of that particular plane group in the set.
And then a symbol that really has its full meaning only in three dimensions, but it gives the symbol for the lattice, again, 2 and then a 1 and a 1, which says that if this were a three dimensional space group, there would be a two-fold axis in one direction and no symmetry at all in the other two directions.
And then, next comes the symbol for the point group of the crystal.
And we derived this particular group by dropping point group 2 into a lattice.
And then the coordinate system that is necessary to describe the features of this plane group, if you take the edges of the unit cell as the basis of a coordinate system.
So this is something called a crystal system.
So for each of the plane groups, you have this information spread across the top of the page.
Then underneath that is a diagram that, by means of open circles, shows the way in which that particular symmetry moves a motif where atoms, in the case of a crystal, moves atoms around.
And for p2 we have the pair of motifs of the same chirality related by a two-fold axis.
And that, then, is hung at every lattice point of an oblique net.
And then, immediately to the right, is the arrangement of symmetry elements in the group.
Then, the way in which that particular plane group can move atoms around to generate a structure is summarized for you.
And the first thing they have to state is where you're going to choose the origin.
There is no unique lattice point.
It's convenient to take the origin at a location of high symmetry, and in this case the smart thing to do is to take the origin at one of the two-fold axes.
That's a non trivial question, because there are some groups that have different kinds of rotation axes.
P4 has a four-fold and the two-fold.
Taking the origin of the coordinate system at a location of high symmetry, then, gives you the scope of picking either a two-fold or a four-fold axes.
Yes,sir?
On the left hand figure, what's with the-- those aren't mirror planes [INAUDIBLE] are they?
On the--
[INAUDIBLE]
Here they have this.
That's just to split things up into quadrants, so you have a feel for how the atoms hung at the lattice points split up into different quadrants.
That is a very good point, though.
Mirror planes are shown as bold lines.
Sometimes the outlines of the cell look pretty bold themselves.
So if you turn the page to pm, if you have the notes with you, can see the lines that are reference lines for the cells are lighter in their weight than the symbols for the mirror planes.
But it's a very subtle difference.
And when you Xerox it a couple of times it becomes almost indistinguishable
Can you just say, one more time, [INAUDIBLE]?
That means that this is the location of 0,0.
And it is always assumed, but nowhere stated specifically, that the origin is in the upper left hand corner.
And what is also assumed, but never stated, is that the x-coordinate in the a-axis goes down and the y-coordinate in the b-axis goes from upper left to the right.
That is nowhere stated anywhere in the international tables, but that is the direction that is assumed for the reference axes.
So we'll take our origin of the two-fold axis.
That's a sensible thing to do.
And then there are two ways you can look at how the particular plane group generates a pattern.
You can do it with little circles or little commas, or something like that.
In other words, do it graphically.
But a way to communicate the atomic arrangement in a structure, which is going to be the most free of ambiguity and rigorous, is to do it analytically.
And that's what the tables do for you next.
They give an analytic description of the way in which the symmetry elements will move an atom around.
For this particular plane group, things are very simple.
This is the direction of x, this is the direction of y.
And if we plop one atom in here, it will be at a location x and y.
So that's what happens when you drop one atom in it.
That atom gets hung at every lattice point, and it gets rotated by the two-fold axis.
So you're going to get two of them per lattice point.
Somewhat arbitrary which pair you take.
The coordinates of this atom are x and y.
The coordinates of this atom inside of the box is 1, minus x and 1, minus y.
That would do it, but it makes sense, esthetically, and to see that the atoms are related by symmetry, if, instead, you specify as the two atoms per cell the two that are hanging at the lattice point.
And in that case, the coordinates of the first, if they are x and y, would be minus x and minus y.
If you have a pair of numbers, 0.283 and 0.456, then minus 0.233 and minus 0.456 leaves no doubt that these are positions that are related by symmetry.
If you have coordinates, like 0.2, 0.3 and then 0.8, 0.7, without doing some arithmetic in your head, it's not clear that those are going to be atoms related by symmetry.
So the coordinates that will be stated for you, then, will be the coordinate of the pair of atoms at the lattice point, and some of the coordinates would be negative.
This is something called the general position.
And that transformation of coordinates is different for every one of the plane groups.
That's what makes them different.
They're different in the way they move around an atom in a general location and fill space with it.
So that's the general position, and it is unique for each plane group.
And there are several ways in which this information is conveyed to you.
Move this over a little bit; x, y and minus x, y.
The first thing that's a characteristic of the position is the number per cell.
So you get the number 2, or this is sometimes referred to as the rank of the position.
Drop in 1 and you get a second one out related by symmetry.
The next is, and I'll go to the last next, this is the site symmetry.
And by definition, this is always 1, no symmetry at all, for a general position.
So why make a big deal about all of these characteristics of the general position?
The reason is that there are locations that are termed special positions.
And let me clean up this right hand part of my diagram.
And say that, for plane group p2 you will always get two atoms per cell if you place an atom in an unspecialized location.
What would be a specialized location?
Suppose we would drop the atom down right smack on top of that two-fold axis?
Then that two-fold axis is just going to twirl the atom around on its axis and it's not going to map it into a second location.
So following the general position is always provided to you a set of special positions.
And what's special about them?
They are on a symmetry element.
You can look at the characteristics of a special position in two ways, either geometrically or you can do it analytically, as I'll show you in just a moment.
Imagine that the atom is to sit here, x and y migrate progressively towards the location 0,0.
And then the other atom related to it by two-fold symmetry will move towards it until finally the two of them will merge into just one single motif.
So when that happens you get not two per cell, you get only one per cell.
And the reason for that is the site symmetry is 2.
It's on a two-fold axis.
But there are other two-fold axes as well.
What if the element migrated to the position 0, 1/2?
If that were the case, if the representative atom moves to here, this one up here will migrate down and the two will merge into one atom that sits at just 0, 1/2.
So there'll be another type of special position, one per cell, also, on a two-fold axis.
This one was at 0,0, and this one is at 0, 1/2.
You remember, we made a point of saying that there are four different kinds of two-fold axes within this plane group.
Different in how they're positioned relative to a pattern, different in that they are not mapped into one another by some other symmetry element which is present.
So each of these four locations is a location of another general position of rank 2, sitting on a two-fold axis.
x and y migrate down to 1/2, 0, and this one and this one will come together.
So there's another one, giving you one per cell, sits on a two-fold axis, and this would be at the location 1/2, 0.
And finally there's another one in the center of the cell, and if you let the atom migrate to that location, again, the atoms will coalesce pair wise into a single one.
So there'll be another one, one per cell, and also on a two-fold axis, and this would sit at the location 1/2, 1/2.
And those, then, are the characteristics of this particular simple plane group.
There's another symbol that's added, and this is something called the Wyckoff symbol.
Wyckoff was a crystal chemist who attempted valiantly, back in the early days of the century, when there were a couple of dozen structures determined per year because it was such a grand, new adventure, and few people knew how to do it.
But Wyckoff tried to summarize, between one pair of hard covers, all of the crystal structure determinations that had been performed in a single year.
And he went at this courageously for perhaps a dozen years, and then results began to accumulate so rapidly he just said, the heck with it, clapped covers on it, and that was the end of the series of books.
So they don't get very far, but it was a real useful contribution at the time.
This is just a shorthand way of referring to the position.
And he starts with a for the most specialized, runs the way up through the alphabet until you've assigned a letter to each of the positions.
This is a nicety.
All these special positions, for example, sit on a two-fold axis, and you don't really have to specify the coordinates, because they have to be either 0 or 1/2.
So this gives us a way of saying you have an osmium atom in position 1a and you have an oxygen atom in position 1c.
And it's a nice shorthand way of avoiding mentioning things which could be calculated once and for all and not stated explicitly.
So this, my friends, is the language in which you will see structural datas cited, when people have determined, using diffraction methods, the locations of all of the atoms within the units, some of a particular material.
If the coordinate is variable, with today's techniques and high speed computation, you can get x as a fraction of a cell to usually at least plus or minus 1 in the fourth place.
So it's data that can be determined extremely precisely.
Yes sir?
I didn't quite get the exact definition of the general position.
It's a set of points related by--
The general position, it's a set of equivalent positions that are related by symmetry.
Yeah.
And the general position, yeah-- Each of these is called a set of equivalent positions, equivalent in the sense of being related to one another by symmetry.
And the entire set is sometimes referred to as an, singular, equipoint.
Sort of a condensed jargon for a set of equivalent positions.
So when one speaks about an equipoint of rank 2, in this case, or rank 1.
Let me just, even for this trivial two dimensional symmetry, give you an example of some information that falls out of this immediately.
First of all, you can use these special positions only once.
If you put an atom in position 1c at 1/2, 0, it's used up.
You can't put another atom in there in a given structure.
Secondly, if you think more globally, not in terms of atoms, which, to a good approximation, are spherically symmetric.
But think in terms of a molecule, if you're a polymer scientist or an organic chemist.
and not place an individual atom, but place an entire molecule.
If there's one molecule per cell in this particular symmetry, that molecule has to sit on a two-fold axis.
And that means the configuration of that molecule is going to be limited to having to conform to a two-fold rotational symmetric.
So just from the density in the lattice constitute, if you find there's one molecule per cell, you know that molecule has to have two-fold rotational symmetry.
So you can say something about the structure of the molecule.
So there are lots of ways in which the symmetry information has something to say about the arrangement of atoms in the structure.
Any further questions?
Yes?
How did they decide, between b, c, and d, which was the most special form?
That is a very good question.
And the way it's decided is by you getting there first before anybody else did it.
So everybody follows, now, the conventions that are given in the international tables.
That's sort of the dictionary of all these terms.
But you're absolutely right.
What is more special about this one than this one?
Well, they're all locations of the same symmetry.
You can turn one into another by changing the origin of the cell, where you're going to define a lattice point.
So they are thoroughly arbitrary.
What is generally done is to take 0, 0 for your particular choice of origin as position a.
And then what should come next, 0, 1/2 or 1/2, 0?
The truly observant among you will notice I snuck a quick peek at the international tables before I wrote one of these down.
I can never remember which is which.
And it is arbitrary.
Usually you end up with the one that has both coordinates non-zero, but that is code that's embodied in the international tables.
But that's a good question.
Yes, sir?
How did you decide to make the motifs [INAUDIBLE] special decisions?
Oh, I just changed the value of x and y.
Those are numbers that can be, for the representative atom, each of them can be between 0 and 1.
So what I just said was, what will happen if we let x and y take on not general values, but very special values?
And the special values are going to be things like 0 and 1/2, if I have picked my origin at a two-fold axis.
It's another reason for picking your origin at a location of high symmetry
But how about the other non-origins of these special decisions [INAUDIBLE]?
There are two ways you can do it.
One of them we just did.
We just looked at the geometry and we said, if this representative atom, this is the one we define as being at location x, y.
If x and y both approach 0, this atom and the one over here are going to migrate towards the origin lattice point and eventually just coalesce.
If I let x and y and migrate, x go to 0 and y go to 1/2, then this one and this one will come together and coalesce.
So that's why I did that, because I knew that the number I would generate would be smaller than the number that I would get for a set of coordinates for an atom that was off the symmetry element.
I said there's another way of doing it.
For something this simple it is not really that profound, but it is a way of checking whether you've counted the same position twice.
I said you can imagine these special positions arising when x and y assume special values.
So let's let, for example, x be 1/2 and y be 0 and plug those coordinates into my equation for the general positions.
So I'll put in 1/2 for x, 0 for y, and then I'll put in minus 1/2 for x and minus 0 for y.
But what goes on at minus 1/2 has to be the same as what goes on at plus 1/2, if the structure is based on a lattice and periodic.
So what I'm really getting is 1/2, 0, 1/2, 0 twice, which is the same as saying, if x is 1/2 and y equals 0, I'm putting two atoms together right on top of one another.
So you can do it analytically.
And for the more complicated symmetries, where there might be for 20 or 48 atoms, that's a good way to see if you've looked at the same position twice.
So we're going to do a couple more before we move on, and this one is almost trivially simple.
Any other questions before we go forward?
Let's take one that is slightly more complex.
And I'll go to one of the rectangular groups.
And let me look at, let's take plane group p2mm, which has a fair amount of symmetry to it.
So this is the group that we got by putting 2mm into a lattice that had to be rectangular, then.
And across the top of the page, this is number six, you'll see that it is rectangular.
It has to have a lattice in a rectangular shape because of the symmetry.
Next comes the short hand symbol.
And I always feel a little bit apologetic when I have to point out the existence of this, and the reason is that people who work with symmetry are no more or less lazy than any other human being.
And some of the symbols become really ungainly when we go to three dimensional symmetry.
And something like a 2 sub 1 over a 2 over m 2 sub 1 over d is a mouthful.
And actually what's done is to just specify the minimum amount of information, which defines the symmetry.
So here, two m's that intersect have to give you point group 2mm.
Then comes the full symbol of the plane group, p2mm.
And for those who are counting, this is number six and the shorthand symbol is pmm, where the short symbol for the point group, along with the lattice, is sufficient to tell you that this is 2mm placed at the nodes of a rectangular lattice.
Then, underneath, comes the representative pattern.
Again, there's a cross in the middle of this just to split it up into quadrants.
The symmetry is one that we've already encountered and was fairly easy to come to terms with.
Two-fold axes at the corners of the cell, and in the center and in the midpoint of the edges.
That's just like p2, except the mirror planes that are present require that that parallelogram in p2 straighten out into a rectangle.
And then we put 2mm at the lattice points so the mirror planes run down through the cell like this, up and down and left to right.
So here is all the symmetry here.
Pattern, I'll say it again.
It's so easy to forget.
The pattern is nothing more than the pattern that 2mm produces hung at every lattice point of a rectangular net.
And all of the symmetry planes and symmetry axes that arise are just ways of defining all of the relations that exist between these things when you perform that process of addition.
So this is, then, the arrangement of all of the atoms in the cell and then some.
And there's a new symbol that's introduced, and somebody asked about that last time.
To me it looks like a little tadpole inside of a frog egg.
Has anybody seen a frog egg?
That's just what it looks like.
It's a little tiny tadpole in there, waiting to hatch out.
So here is our little tadpole.
This is used to indicate an enantiomorph.
If this were a motif with chirality, this one would be right handed, this one would be left handed.
So that indicates all of the atoms or molecules that are the same chirality.
The purists among you will notice that the dummy that produced this diagram did not show the tadpoles conforming to the two-fold symmetry.
Both of the commas point in the same direction.
Tsk, tsk.
Little oversight.
So this, then, is the arrangement of motifs in the pattern.
And now let's proceed to analyze that, according to the nature of the positions that are available of different sorts.
Again, coordinates have meaning only if you define the origin.
And nobody in their right mind would want to pick an origin that is off, at least, the symmetry plane.
But the thing to do is to take the origin at 2mm.
Again, I'll remind you that x goes down this way, y goes this way.
So if I have a representative atom at x, y, I'll have another one at minus x, plus x, another one at minus x, minus y, and one at plus x, minus y.
So there are the coordinates of all the symmetry of related atoms.
So putting down the general position, I get four of them if I place one in the cell.
By definition, the general position is always at a site of symmetry 1.
So this is the site symmetry.
Then we just rattled of the coordinates.
They're x,y, minus x, y, x, minus y, and minus x, minus y.
There are lots of special positions here, now, because we have not only two-fold axes with mirror planes intersecting at them, but they're also locations of just a mirror plane alone.
And there are a total of four distinct, independent mirror planes, this one, this one, this one, and this one.
So we're going to have four positions where the atom sits at a location of symmetry 2mm.
And those are going to be analogous to the positions that we found for the two-fold axes in p2.
And, not surprisingly, the coordinates look very much the same.
0,0 puts us on a location of symmetry 2mm.
0, 1/2 does the same, 1/2, 0 and 1/2, 1/2.
So there are four positions of symmetry to 2mm.
And then there are four positions just on a mirror plane, but not on a two-fold axis.
And we could do that for the mirror plane that runs along the x-axis, or the mirror plane that runs along the direction of x at y equals 2.
And the order in which you pick them is rather arbitrary, but there will be a pair of positions at x, 0 and minus x, 0 that would happen if the y-coordinate was exactly-- let me draw what the position is, just off here to the right in a small diagram.
This would be where you put the atom on the mirror plane running through the origin, then these things would give you only a pair.
The next one is at a position x, 1/2, and that's where the mirror plane would be the mirror plane that is along the position y equals 1/2.
And here we would get atoms coalescing like this.
x is general, but y is exactly 1/2.
So we'd have x, 1/2, minus x, 1/2, and I better put these off to the right, here.
The remaining two are 0, y and 0, minus y.
That is also a position of site symmetry m. And that would be a pair of objects that has x equal to 0. y is anything, so that's this mirror plane here.
And in that case we'd have a pair of atoms, left to right, reflected by the mirror plane that goes through the origin.
And then last one, thank goodness, is one at a location 1/2, y and 1/2, minus y.
And this would be the mirror plane exactly at 1/2.
y can be anything.
x is 1/2.
So that would be a pair that sits somewhere on either side of the mirror plane running through the center of the cell.
That's also a site of symmetry m. Now we assign the Wyckoff symbol by working our way up the alphabet.
So this is a, this is b, this is c, this is d, this is e, f, this is g, this is h, and finally we end up at position i.
For all of these, we get two atoms per cell.
These have rank 2.
For each of these we get just a single atom
Can you explain the two atom per cell thing?
How do you [INAUDIBLE]?
Which one would you like me to do, that last one?
1/2, y?
[INAUDIBLE]
So if x is 1/2 and y is anything, that's this locus here.
And that is a mirror plane.
And if we let this atom move from its location, x,y, down to this location, the one in the lower part of the cell is going to move up and these two will coalesce to a blob that sits there
That would be 2m?
So this would be site symmetry m. And instead of getting four per cell I would get two per cell; this one and the one that's reflected across.
Now the other way of doing it is to not think about it all.
Say here is x,y, minus x, y, x, minus y, minus x, minus y.
And then let's make one of the coordinates specialized.
Let's let x be exactly 1/2.
So I'll get 1/2, y, minus 1/2, y, and then I'll get 1/2, minus y and minus 1/2, minus y.
Looks like four atoms, except if I have one at plus 1/2, the one in the neighbor unit cell is at minus 1/2.
So actually I've had the atoms coalesce pairwise, because these coordinates are actually identical to saying 1/2, y plus 1/2, y, 1/2, minus y, and minus 1/2 is the same as plus 1/2, minus y.
So you can do it that way.
Just put in a special value for x or y, turn the crank on the general position, and you'll find you're going to get the same thing twice.
And if we look at position a, which is 0,0. put in 0 for x, 0 for y, then put 0,0 for all these other four positions, you're going to get 0,0 four different times.
I'm going to do just one more, and the results are here for all of the remaining 14.
So I don't think there's any need to do them all.
One of the things I would like to examine with you is a group that has a glide point in it, and see how that changes things.
Any other questions on this?
Can you just go over the rank of fours?
Four--
Four means four per cell
Four per cell
And if you count them around the origin as 1, 2, 3, 4.
If you count the number that's caught within the box it's 1, 2, 3, 4
And site symmetry of 1?
That's site symmetry 1 because 1 is no symmetry at all, and that is what's general about it.
It's not sitting on a symmetry element which would then fail to reproduce it.
And again, if the motif was not an atom but was a molecule, if you find only one molecule per cell from the density in the cell dimensions, that molecule has to have symmetry 2mm, because it has to sit.
If there's only one per cell, that set of atoms has to sit at a location that conforms to symmetry 2mm.
Let us go to one more.
And I'm going to take p2mg, because it's one that has a glide plane in it.
This is pmg for short.
This is number seven.
P2mg is the proper symbol, and that's point group 2mm.
And, again, the coordinate system is rectangular.
This is one that we derived last time.
The arrangements of elements in this is, once again, a rectangular lattice, two-fold axes.
If we take the origin at 2, would be at the corner of the cell and in the midpoint of the edges.
Then there's a glide plane passing through the two-fold axes.
And there is a mirror plane passing in between the two-fold axes.
And here's a good example of an arrangement of symmetry elements that has a mirror plane in it, and then a line that just indicates the edges of the unit cell.
And can you distinguish the bold line from the light line?
Barely.
But the bold line is the mirror plane.
And what does the arrangement of atoms look like?
This is a tricky one, because there are two different symmetry elements; a two-fold axis, and that's going to take an atom at x, y and repeat it to minus x, minus y.
But here, now, is a case where there's another symmetry element that does not intersect the first one.
So that mirror plane is going to reflect this atom down here to an enantiomorph and reflect this atom down here to another enantiomorph.
So if this is x then this is y, this one sits at minus x and minus y.
For this atom here, y is the same as before.
But if this distance is x, the distance up from the center line will be x, so this second coordinate is 1/2, minus x. I'll do that again since this is pretty small and tight.
This distance is x.
We reflect the atom across a mirror line at 1/4, and therefore the one towards the center of the cell sits up above the location 1/2 by the same number, x.
So this is 1/2 minus x, y.
And by the same argument, this one here is at minus y, and it sticks out beyond the position x equals 1/2 by the same amount, x.
So this one is 1/2 plus x minus y. Nontrivial.
So if I began to tabulate these-- We've picked the origin at 2.
Didn't have to do that.
If you wanted to, you could take the origin at m if you wanted to, and just switch everything by 1/4 of the coordinate x.
We found four per cell.
The site symmetry is 1, and the atom locations were at x, y, minus x, minus y.
And then we had 1/2, minus x, y and 1/2, plus x, minus y.
So the coordinates are getting permuted around in a very more complex fashion than just changing sign.
Special positions.
Just as before, any of these two-fold axes is going to be a site for a special position.
Let x and y go to 0.
This pair at the origin will coalesce and this pair will coalesce at a point that's halfway along the cell.
So there will be two of those, one at the origin, one in the middle of this edge, and their locations would be 0,0 and 1/2, 0.
And this is a site on a two-fold axis and we have only two per cell.
Would this be a location for a special position, this two-fold axis?
It was before.
Is it now?
Key point is independent sites of symmetry.
There's a mirror plane here.
What goes on here has to be exactly the same as what goes on here.
So this is not an independent special position.
In the same way, there's a mirror plane.
And if I let the atom migrate down to the mirror plane I will have the mirror plane at a location 1/4, y.
And these will be on a mirror plane.
1/4, y would be a first, 3/4, minus y would be the second one that I get, and they would coalesce pairwise.
Is this is another mirror plane that could be regarded as an independent position?
This mirror plane, again, is not independent, because it's related to the one we just considered by the two-fold rotation about the center of the cell.
So there's only one kind of mirror plane in the structure.
But, going back to the two-fold axes, there is a two-fold axis here.
There's another one that is halfway along y.
So there's another two-fold axis that is independent at 0, 1/2, 1/2, 1/2.
And this is the entire set.
So there are three special positions of rank 2, two atoms per cell.
We make the most specialized one be named a, work our way up to c, and the general position, then, is referred to as d. Again, so what I did was to say that this mirror plane is related to this one by symmetry, and if there's something on this mirror plane as a special position, I'm automatically going to get it on the lower mirror plane.
This two-fold axis is the same as this one.
So whatever is going on as a result of specializing the location on this two-fold axis is going to be provided for me automatically at x equals 1/2 by the action of this mirror plane.
There's nothing that throws this two-fold axis into this one, though, so they're both independent.
Now, one of the symmetry elements that I steadfastly ignored was the glide plane.
What happens if I place an atom right on the glide plane?
A glide plane, if you use a motif that has some handedness to it, we repeat the atom by translating half of a translation, not yet putting it down, first reflecting it across.
This is an enantiomorph.
Doing it again gives me an atom that's related to the first by a translation.
What happens if I let the atom migrate onto the glide plane?
Let this atom move down to here.
Now it sits here.
This one, repeated from it by glide, will migrate up to here.
Nothing happens.
It's just that the amount that the atom is displaced from the locus of the glide plane is different in the two cases.
But there is no coalescence of the atoms, because of this translation component to the glide plane.
So a conclusion, then, is that glide is never a candidate location for a special position.
This is a good example of a case where I can check what I've done, and to see if I counted the same position twice.
I said that both of these locations are not special position.
The mirror planes are related by symmetry.
Suppose I were not clever enough to notice that, and I said, OK, there's going to be a special position, 1/4, x, 3/4, minus y.
That's what happened if I use this mirror plane as a special position.
What would I get if I took 3/4, y and 1/4, minus y?
Looks like a different specialized position.
But let me put this number into my expression that is the formula for the general position.
If x is 1/4 and y is y, then this position here would be minus 1/4 and plus y, 1/2, minus x would be 1/4, and y is y.
Over here, I got the same thing back again.
And 1/2, plus x would be 3/4, and minus y is minus y, minus 1/2 and 3/4, plus 3/4 are the same thing.
So what I've gotten is the position 1/4, y, and 3/4, minus y twice.
So what this is telling me, analytically, if I just plug in the coordinates of what seem to be a distinct, special position is, I get the same atom on top of itself twice.
And it looks exactly like choosing the first mirror plane.
So that's a good way of checking, particularly in a very high symmetry where there are all sorts of sites of possible point groups that could cause coalescence, to just crank out the coordinates and see if, in fact, you have exactly the same pair with the same coordinates that correspond exactly to the x and y of some previous location that you identified.
I think, hopefully, you have a feeling for how it works.
If you look at some of the higher symmetries, the number of atoms that constitute the full equipoint set of equivalent positions.
For p6mm there are 12 locations in the general position, and because the coordinate system is oblique, x and y get permuted into linear combinations of one another.
So that's really, extremely complicated.
If you look at some of the square symmetries, the number of the special positions is very, very high.
There are two positions in p4mm, there are two positions of rank 1.
There is a position of rank 2 that sits on 2mm.
Three different mirror planes, and the general position with rank 8 requires working your way all the way through g in the alphabet.
Yes?
I had a question about the p2gg [INAUDIBLE] whether they're at the entrance of the [INAUDIBLE]?
For p2gg, the only site of point symmetry is a two-fold axis.
So the only thing that you could use for a special position in p2gg is a two-fold axis.
And there are two different two-fold axes.
They relate in a curious way because of a glide plane.
And this is a case where the site of a special position is not related by rotation or reflection, but by a glide.
P2gg looks like this.
Two-fold axes in the old familiar places, 0, 1/2.
The glide planes are here, here, here, and here.
So which two-fold axes are equivalent?
The glide would take this two-fold axis, reflect it down, and slide it over to here.
So whatever goes on here goes on at this two-fold axis
Sorry.
I guess my question was, can you put the glide planes at any edges of this sites?
[INAUDIBLE], or--
You could.
You could, but you would lose, then, the apparent similarity in coordinates in that, if you put the glide plane at the two-fold axis, if this is x, y, this one over here is at minus x, minus y.
And then if you repeat it by glide, that would slide over to here and then be reflected down here.
So there'd be one pair up here and one pair down here of opposite handedness, so you would not see the similarity.
No, that's right.
If you took the origin at the glide plane, you would not see the simple relation between the numbers that you do when there is that a rotation axis of a mirror plane.
But you can take the origin anywhere you want.
And the advantage of doing it at a symmetry element is then the numbers that describe where the atoms sit are simple permutations of sign, or adding 1/2 to the number of the preceding atom that was just mapped to a new location.
I think you're more than ready for a break.
Let's resume in 10 minutes, and we'll move on to something completely different.
OK. Before we get started, I'd like to deal with a small matter of some unpleasantness.
The class is sort of like a football game.
When there's two minutes to go, you shoot off a pistol.
But when there are two meetings to go until we have the quiz, we shoot off a pistol to wake you up.
We're scheduled to have a quiz on October 6.
And even though October seems far away when you're still in September, that is going to be a week from this coming Thursday.
So if you have problems that you're working on, try to get them to me on Thursday, or just come slide them under my office door, if I'm not in.
And I'll have them back for you on this coming Tuesday.
The way the material has played out is that we're really at a nice, convenient juncture between one chunk of interconnected material moving on to another.
So I think the first quiz we'll confine to two-dimensional symmetry.
And beginning in about two minutes flat, we'll begin to move into-- take the first small steps, at any rate, into three-dimensional symmetries, which will be much more complicated in which we will not deal with the exhaustiveness that we have been able to afford the luxury of in two dimensions.
Before I begin, let me-- does everybody remember their spherical trigonometry?
Has anybody had spherical trigonometry?
OK. What I will then do is give a short primer on some of the definitions and concepts in spherical trigonometry.
And then we shall immediately use this to combine rotation axes in space.
Pass those back
I had a quick question on this
Oh, sure, please
With the limiting possible--
Yep
--reflections and conditions, that does say they have to be figured out?
Actually, that was a good question and something that we are not going to use at all in the symmetry tables.
Somebody asked what about this notation in the far right-hand edge of all of the plane groups-- conditions limiting possible reflections.
One usually doesn't put a two-dimensional crystal in an x-ray beam fairly often, although I suppose a thin film actually is almost a two-dimensional crystal.
But you've probably all heard one way or another about the magic conditions relating the Miller indices of a plane that will require that the intensity diffracted from that set of planes is identically zero.
And they're linear combinations of H, K, and L. And one of the rules is that if H plus K plus L is even or not, the intensity may be zero.
These are rules for systematic absences.
And the corresponding information is given for you here for these not terribly realistic real two-dimensional crystals.
So, for example, if you turn to number seven, P2MG, it says conditions limiting possible reflections for the general position.
For H and K, there's no condition.
For H zero, H has to be even, if the reflection is to have non-zero intensity.
And for the last two, for the general reflections, H, K, if there's an atom occupying the position either zero, zero or the position zero, one half, then for the general planes with indices H, K, you will see intensity only if H is even, as well-- same as the condition above.
This is something that is not generally known that everybody knows-- that if the crystal is face-centered cubic, there is a pattern of absences.
But there are additional absences if the atom is in a special position.
And this can very often be used to advantage because you can single out certain classes of Miller indices for which one atom in the structure will not diffract or which another atom in the structure will not diffract.
And that can be of great utility in unraveling a structure.
We'll see some examples of this in useful form when we deal with three-dimensional space groups.
And the corresponding sheets for the three-dimensional symmetries will be handed out to you-- some of them, not all of them.
Let me take a little bit of time to remind you, if you've forgotten them, but to inform you of certain definitions and trigonometry in spherical geometry.
Spherical trigonometry differs from plane geometry in that all of the action takes place on the surface of a sphere.
And it'd be nice if I had a spherical blackboard.
Actually there is one in the x-ray laboratory that I can draw things right on that spherical surface.
But this is a sphere.
We'll see directly that the radius of the sphere is not important.
So we'll take that as a unity, which is a nice, even number.
And as my dichotomy of the afternoon, if we pass a plane through that sphere, if the plane hits the sphere, it will intersect it in a circle.
If the plane passes through the center of the sphere-- and we've assigned the radius of the sphere as unity, then this is a circle that's referred to a great circle.
Sounds like a value judgment, but it's simply saying that's as large as the circle is going to get is when it passes through the center of the sphere., it would have unit radius, just as the sphere does.
So if you take any other plane which intersects the sphere but doesn't pass through the center, it's going to have a smaller radius.
And this is something that's called a small circle.
OK, so if all of the action is going to take place on the surface of the sphere, and we have two points on the sphere, A and B, sitting on the surface of the sphere, how do we measure the separation of A and B?
Well, if you think in terms of a normal three-dimensional person, you say, zonk, connect them by a line.
And that's the distance between A and B.
Now, you can't do that because all the action has to take place on the surface of the sphere.
People who deal with spherical trigonometry all the time are airplane pilots.
And if your pilot is going to take you from New York-- where would you like to go?
Paris?
That sounds like a nice place.
But if you're going to go from New York to Paris, you don't plow your way through the intervening earth.
You follow something that is at a constant radius out from the center the Earth, at a height of 5,000 feet above the surface of the Earth, an additional radius.
So the way we'll define distance is to pass a great circle through A, B, pass a plane through A, B in the center of the sphere.
And then we will define distance between A and B as the smaller of the two angles subtended at the center of the sphere.
So this is a more reasonable looking great circle.
If this is point A and this is point B, pass a plane through the center of the sphere, O, and through A and through B.
And then we'll measure the length of the arc AB in terms of the angle alpha subtended at the center of the sphere.
So it's a crazy notion.
We're measuring distance in terms of an angle.
And if that's an angle, we can take a trigonometric function of that angle, like sine or cosine.
And that blows the mind that you can take trigonometric functions of a distance.
But we can.
We'll see it's going to be useful to us, too.
And then I emphasize again, we'll take this as the smaller distance.
It'll be 360 degrees minus alpha.
That would be the long way around from A to B.
All right.
We've defined now how we will draw distances between two points.
Suppose I have three points on the surface of the sphere-- A, B, and C. I can pass a great circle through A and B. I know how to do that.
I can pass a great circle through A and C. I know how to do that.
And I can pass a great circle through B and C. So now I have defined something that is referred to as a spherical triangle.
We know how to measure the length of the spherical triangle.
Let's call the arc opposite the point of intersection A as little a and the length of the arc opposite B as a distance and angles little b, and the distance from A to C as little c. But there's something in between these arcs that looks like an angle analogous to the angle in a planar triangle.
And how can we define that?
Well, the arc AB is defined by a plane, a great circle.
The arc AC is similarly defined by a plane that passes through a, c in the center of the sphere.
And what we will define as the spherical angle BAC is the angle between the great circles that define the two different arcs.
So if there's one great circle that defines the arc from A to B and another plane that defines the arc from A to C, we'll define as the spherical angle between those two arcs the angle between the planes that define the great circles.
So we're going to call this angle in here between these two planes as angle BAC.
Another construct that is a useful one-- suppose I look at the plane that I've used to define a great circle and at the center of the sphere construct a line that is perpendicular to the great circle.
And if I extend that line, sooner or later it's going to poke out through the surface of the sphere.
And I will refer to this point as the pole of arc AB, or the pole of the great circle that we've used to define arc AB.
So the North Pole is actually the pole of the great circle that defines the equator.
And clearly there are two poles.
There's one in either direction.
So there's a North Pole and a South Pole to this arc AB and to the great circle that defines it
I have a question.
Why couldn't you define the angle BAC as the angle [INAUDIBLE]?
You want to make a tangent here?
Yeah
I don't know if that's really defined.
In other words, if I'm saying I want a line that is tangent to the sphere, it doesn't fix its orientation
In the plane of the great circle
OK.
In the plane of the great circle.
Suppose you could if you wanted to.
There are trigonometric qualities to defining the angle in the way that we have.
And the construct really is not something confined to the surface of the sphere, and everything else that we are doing is.
OK?
So it's sort of the non sequitur because we started out by saying that everything has to take place on the surface of the sphere.
There's things that we would do in our three-dimensional world, like defining distances between points as the shortest straight line, that are ruled out in spherical trigonometry.
And I think something similar could be levied at your proposal.
And the answer is we just don't do it that way.
That's the real answer.
OK.
If I have not boggled your mind so far, let me go a bit further with another useful construct.
We can see how we can define a pole of a great circle or a pole on an arc that is a portion of a great circle.
Let me take a spherical triangle, A, B, and C. And I've got three great circles now, which have formed those arcs that make up the sides of my triangle.
Let me now find the pole of arc CB.
And that means we're going to go out 90 degrees to that plane through the center of the sphere.
And that's going to define some point that I'll call A prime, that is the pole of arc BC.
And I'm going to do the same thing for the other arcs that are sides of my great circle of my spherical triangle.
I'll find the pole of arc AC.
And I'm going to label that point as B prime.
And, finally, there'll be another pole that is the pole of arc AB.
And that's going to define a point C prime.
Now I've got three points, I can connect these together and make another spherical triangle
How do you know to determine where the pole is?
If you think of it in three dimensions, I got three different arcs.
And for each one of them I am drawing a perpendicular to the plane of that great circle and looking at the point where it emerges.
OK.
So now if there's another arc, there'll be another great circle coming around like this.
And I look for the pole of it.
And that would be another one of the corners.
This thing that I've constructed is bizarre.
But it's given a special name.
This is called the polar triangle.
And it has some useful properties.
A property of the polar triangle is that the two triangles, A, B, and C, and A prime, B prime, and C prime are mutually polar.
That is, if I use the spherical triangle ABC to define and locate the three points A prime, B prime, C prime-- now if I reverse the process and find the whole of arc A prime C prime, that turns out to be point B.
And if I take the arc of A prime B prime, that turns-- I'm sorry-- take the arc of B prime C prime, that turns out to be point A.
So the two triangles are mutually polar.
The polar triangle of the polar triangle is the triangle that we started with.
Yeah?
I guess I missed that.
So if you take B prime through C prime, then all of that is going--
Yeah, I'm saying that the pole of this arc, A prime C prime, is this point here.
And the way I can show that is to say that we got B by looking at the pole-- I'm sorry-- I got C by looking at the pole of the arc AB.
So B is 90 degrees away from C prime.
I found point A prime by finding the pole of arc CB.
So B is 90 degrees from any point on that arc.
So it's 90 degrees away from A.
So B is 90 degrees from A prime.
B is 90 degrees C prime.
And, therefore, it has to be the pole of that arc.
Now that was a little too quick.
That's written down in the notes.
And that's why I wrote them out.
One final thing and then we can put circle trigonometry to one side, and this is something that is not all obvious.
If we look at a spherical triangle and simultaneously the polar triangle-- so let's say this is ABC.
And here is the polar triangle A prime, C prime, B prime.
It turns out that the spherical angle in one circle triangle and the length of the arc opposite it, namely this arc B prime C prime, are complementary-- supplementary, not complementary.
And the way one would do that is to say that the measure of alpha is the length of this arc here.
And this total side, B prime C prime, is equal to this arc plus this arc minus this length.
And these two arcs are 90 degrees.
So let me do as I've done in the notes.
Let me call this point P prime.
And I'll call this point Q prime.
So my argument, it says that B prime is the pole of arc AC.
And, therefore, B prime Q prime equals 90 degrees in length.
And then I would say that C prime is the pole of arc AB.
And, therefore, the distance C prime P prime is also exactly 90 degrees.
And that says that B prime Q prime plus C prime P prime-- if I add those two together, it has to be 180 degrees.
But I can write B prime C prime as B prime P prime plus P prime Q prime plus Q prime C prime plus the side P prime Q prime.
And that's 180 degrees.
But these three things that I've lumped together here are exactly the same as the length of the spherical polar triangle A prime.
So what we've shown then is that A prime plus alpha is equal to 180 degrees-- QED.
So this angle plus the side of the polar triangle add up to 180 degrees.
And that is not obvious at all.
One final relation-- and this I will simply hand to you on a platter.
I'm not about to derive it.
Sides and angles in planar geometry are related.
And there's a particularly useful relation in plane geometry that's called the Law of Cosines.
So this is in plane geometry.
And if you have a triangle that has sides a, b, c-- a general oblique triangle-- and it has angles A, B, and C, the Law of Cosines says that the side A is determined by c and b and the angle between them.
And that's clear.
If I specify this length, specify this length, specify that angle, things set up like a bowl of supercooled jello.
And the triangle's completely specified.
So a squared in the Law of Cosines is b squared plus c squared minus 2bc times the cosine of angle A.
In a spherical triangle there is a similar sort of constraint.
If we have a spherical triangle with sides a, b, and c, and spherical angles capital A, capital B, capital C, in the same way as specifying the spherical angle A and the lengths of the two sides c and b, specifies and fixes the spherical triangle entirely.
This side must be determined by the length of side c, the length of side b, and the angle between them.
And that, since everything is in terms of angles, is something that doesn't involve squares.
It involves totally trigonometric expressions.
And it turns out the cosine of this missing side a is given by the product of the cosines of the two other sides.
So as I said, you can take a trigonometric function of a length, which sounds like an oxymoron.
And it's the product of the cosines of the two known sides and times the sine of b sine of c times the cosine of the spherical angle A.
And that is also called the Law of Cosines.
And this is the corresponding case in spherical geometry.
OK.
So there's some machinery-- yes, sir?
What's the difference between the sines of lowercase a and--
OK, the angles are the capital letters.
This would be the angle between the great circles that defines the--
Since the radius is one, there's no difference between the angles and the-- [INAUDIBLE]?
No.
This angle is something, for example, we can choose.
And depending on how long we want this arc to be, we can put the arc BC anywhere we like
Does that mean your radius [INAUDIBLE]?
No.
This would be, say, two points of the spherical triangle.
Now we can pick any third point on the surface of the sphere, connect that with great circles, and here is a spherical triangle.
OK?
So I see.
I think I see what your problem is.
Here are the two planes.
We define the angle of the spherical triangle as the angle subtended buy the great circle.
So this is the definition of A.
But now the other two points on the spherical triangle can be any point on these great circles.
So this can be point B, and this can be point C. And my spherical triangle can be something like this.
So the arc that defines the spherical angle A is a value that is independent from the length of the arc AC.
That would be what is subtended at the center of the sphere.
I'm going to have time just to set the stage for how we're going to use these relations.
And the problem that I would like to raise and then apply spherical trigonometry to is the question if I go into three-dimensional space, there is no longer any requirement that rotation axes be all parallel to the same direction.
In two dimensions the rotation points were really-- could be viewed as axes that were always perpendicular to the plane of the blackboard, the plane of the plane group.
But now when I'm dealing with three-dimensional spaces, this could be the operation A alpha.
And there is no reason whatsoever why we should not try to combine with this first operation a second rotation B beta-- a rotation through an angle beta about this axis B and a rotation of angle alpha through this axis A.
If we're going to come up with a crystallographic combination, the angles alpha and beta have to be restricted to the angular rotations of either a onefold, a twofold, a threefold, a fourfold, or a sixfold axis, if the combination is going to be crystallographic.
Yes, sir?
You're just rotating around those lines?
I'm rotating around those lines, yeah.
So I'm saying that we're going to rotate an angle alpha about axis A.
And now what I'm going to raise as a rhetorical question-- what is the rotation A alpha followed by B beta?
So I rotate through the angle alpha about A.
So here's my first motif, right-handed.
Then I'll rotate alpha degrees.
And this here's number two.
And that will stay right-handed.
Now, I will place on axis B the two constraints.
These have to be crystallographic rotation angles, namely 360, 180, 120, 90, or 60.
And I'll also, since I would like to obtain point group symmetries initially, I will require that axis A and axis B intersect at some point.
And one of the variables in the combination will be this angle between the two rotation axes.
So let's complete our combination of operations.
I'll rotate from one to two by A alpha.
If the first one is right-handed, the second motif is right-handed, as well.
And then I will rotate beta degrees about B.
And here will sit number three.
And it will stay right-handed.
So, again, the $64 question that we raise periodically-- what operation is the net effect of two successive rotations about a point of intersection?
[INTERPOSING VOICES]
Lots of opinions.
Let's sort them out.
I heard translation.
Well, let's put down what it could be.
We know that only translation and rotation leaves the chirality of the motif unchanged.
So it's got to be one or the other
Since there's no reason for the general orientation of the two to say it has to be rotation around the third axis, so the question is what angle and what--
OK. That is exactly the problem.
Now that we know the problem, we can go home early because we know what we're going to do next time.
Well, let me expand a little bit.
It can't be translation because clearly the separation of number one and number three depend on exactly where I place the first one.
If I place it a little further out from A, then it's going to rotate to here.
And then B is going to swing it off to some different location.
So it can't be translation.
And I don't think these guys, if I rotate this way and then I rotate this way, are going to be parallel to one another.
I doubt that very much.
So it's got to be a rotation.
So without knowing how to find it, let's say that we can get from number one to number three in one shot through rotation of an angle gamma about some third axis, C. So the answer, in general, without being specific, is A alpha followed B beta has got to be equal to a third rotation, C, about a direction that has the same point of intersection with the first two axes.
Now we've got some really, really tough constraints.
Alpha is restricted to one of five values.
Beta is restricted to one of five values.
The third rotation, gamma, jolly well better be a crystallographic rotation and not something that is not a sub-multiple of 2 pi.
And even if it is a sub-multiple of 2 pi, it has to be either 0 degrees, 120, so on.
It has to be one of the crystallographic rotation angles.
So what sort of relation can we get that would give us, first, the value of gamma in terms of alpha, beta, and the angle at which we combine them?
So taking A and B as the five crystallographic rotation axes two at a time, we want to put them together, if we can, such that the angle makes the third rotation axis also be crystallographic.
And then we would have to find its location.
It looks like an impossible constraint .
It looks absolutely impossible to do.
We've got to put this first in quantitative form and then simply put in the values for alpha, beta, and gamma and find the angle that they have to be combined on to make this, if possible, be one of the crystallographics sub-multiples.
That is not an easy problem to formulate.
And, as I said a couple of times ago, the geometry that is the basis of this derivation was originally proposed by Euler.
And it's known as Euler's Construction.
I will have for you next time my own set of notes on this.
We have finished with two-dimensional crystallography.
So we are back to Buerger's book again.
We had that little interlude.
Buerger deals with Euler's Construction.
But I don't think he's at his best in this particular section.
So we'll take it a little more slowly.
And next time we'll get around to deriving Euler's Construction.
The following content is provided under a Creative Commons license.
Your support will help MIT Open Courseware continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Open Courseware at ocw.mit.edu.
Hi I'm Jocelyn and today we're going to go over fall 2009 exam 1, problem 1.
So to begin with, we always want to read the whole problem.
So in this one it says uranium metal can be produced by the reaction of uranium tetrafluoride with magnesium in a sealed reactor heated to 700 degrees C. So we know that we're dealing with a chemical reaction here.
The byproduct is magnesium fluoride.
To ensure that all the magnesium is consumed in the reaction, the reactor is charged with excess uranium fluoride, specifically 10% more than the stoichiometric requirement of the reactor.
To produce 222 kilograms of uranium, how much uranium tetrafluoride must be introduced into the reactor?
So that's what the question is asking us.
Then it says we assume complete conversion of magnesium, express your answer in kilograms.
So we know that the question is asking us for the uranium tetrafluoride needed to make 222 kilograms of uranium.
Right?
That's the last sentence.
The sentence that's before that, however, also gives us very important information.
It tells us that we need the stoichiometric equivalent plus 10%.
And that's very important to keep in mind because, as you'll see later, that's where a lot of people got tripped up on this problem.
So then let's go to the first two sentences in this problem and extract the information given there.
We know that we're dealing with a chemical reaction.
So when we are dealing with one of those we always want to put down the chemical equation.
So we have uranium tetrafluoride reacting with magnesium to give uranium with a byproduct of magnesium fluoride.
And it tells us that it that it's at 700 degrees C. That's superfluous information but you can always put reaction conditions above the arrow just to keep track.
So now that we have this equation are we ready to move on?
No.
Right?
We need to have a mass conservation.
We need to balance this equation.
So to determine if this is balanced or not, we want to keep track of the amount of uranium, fluoride and magnesium on each side of the equation.
So we have uranium, fluoride and magnesium.
And then we have on the product side-- and that's not right.
This is the reactant side, right?
And over here is the product side.
So on the reactant side we have 1 uranium, 4 fluoride and 1 magnesium.
On the product side we have 1 uranium, 2 fluoride and 1 magnesium.
So we see that we have an imbalance in fluoride.
To fix that we can put a stoichiometric coefficient in front of the magnesium fluoride on the product side in order to have 4 fluorides.
We put a 2 there.
And we change that to 2.
But we also have to change the magnesium.
Right?
Now, although we fixed the problem in the fluoride, we have an imbalance in the magnesium.
So to fix that, we put a stoichiometric coefficient in front of the magnesium, switch our accounting down here, and we see that we now have conservation of mass.
The two sides are balanced.
So with our balanced equation we can now move on to what the question is actually asking us.
And we know that we have, we want to make 222 kilograms of uranium.
We then need to figure out how much uranium tetrafluoride is needed.
This chemical reaction gives us mole ratios.
But we are, right now, have kilograms.
So the first thing we need to do is convert kilograms to moles.
And we do that using stoichiometry or unit conversion.
So the other thing we need to know is the molar mas of uranium.
So this is something you can look up on your periodic table, online, whatever resources you have available.
So the molar mass of uranium is 238 grams per mole.
Now we just do a simple unit conversion to get the moles uranium that we want.
So first we need to convert to grams.
That's 1,000 grams.
And then using the molar mass, we convert to moles.
And because I have grams up top here I need to put the molar grams down here in order to cancel everything out.
And that gives us 933 moles uranium.
OK so we have the number of moles uranium we want to make.
Now we can use the chemical equation over here to determine how much uranium tetrafluoride we need.
So the chemical equation tells us that we need a 1:1 ratio of uranium tetrafluoride.
However this is where we need to go back to what the problem was actually asking us.
And remember that we need the stoichiometric equivalent, which is given by this mole ratio, plus 10%.
So the amount of uranium tetrafluoride we need is the stoichiometric equivalent plus 10% times that stoichiometric equivalent.
And that equals 1,026 moles of uranium tetrafluoride.
So we found the amount of uranium tetrafluoride we need.
Are we done?
No.
Right?
If we go back to the question we see that it asks for our answer expressed in kilograms.
So that's why it's always important to keep track of what the question is actually asking you and to make sure you've answered that.
So we have uranium tetrafluoride in moles.
Now we need to convert it to kilograms.
Again we need the molar mass.
This time of uranium tetrafluoride.
And it equals-- so we have 1 uranium per molecule and 4 fluoride.
And again we just look up that molar mass of fluoride on the periodic table.
And that equals 314 grams per mole.
So we do our unit conversion again.
And this is my favorite way of doing stoichiometry unit conversions.
You may have another way that you learned in high school or later on.
But this way I know that I'm keeping track of my units and everything's canceling out correctly.
So we have 1,026 moles uranium tetrafluoride.
We use the molar mass we just determined.
And remember that this is an industrial scale process, so we actually care about the amounts in kilograms.
And we get 322 kilograms uranium tetrafluoride.
Going back to what the question was asking us, we know that we now have the correct answer.
And we can put a box around it.
So that on the exam whoever's grading you knows that that is your answer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Hi, I'm Jocelyn, and we're going to go over Fall 2009, Exam 1, problem number 2.
So as with every problem, we want to first read the full question.
In box notation, give the complete electron configuration of each of the following gas-phase species: calcium 2 minus and magnesium 4 plus.
So the first thing to note here, is you need to know what electronic configuration is.
If you don't, you might want to go back and review that material, but it is basically assigning electrons to orbitals.
The next thing you need to know is what box notation is.
And Professor Sadoway introduced that in class, and you'll see, as I go through this, a refresher of that, if you forgot.
So to assign electrons to orbitals, we first need to know how many electrons we have.
So let's start with the first part of this problem, which is calcium 2 minus.
We will look at our periodic table, and see that neutral calcium has 20 electrons.
So calcium with no charge has 20 electrons.
The 2 minus tells us that it has 2 extra electrons.
Right?
Because electrons have a negative charge.
So we have a total of 22 electrons that we need to assign into orbitals.
In order to actually do the assigning, we need to follow the three rules that Professor Sadoway introduced in class.
So I will put that over here.
The first rule is that you need to fill them, at least for a ground state species, in order of increasing energy.
That is, you want to fill up the lowest energy orbitals first, and then move to the higher energy orbitals.
This makes sense because ground state species want to have the lowest total energy possible.
So the first is-- Now, how do we figure that out?
Well, there's a few ways.
One way is a nice trick that you can do on the side of your page or something.
And it goes like this.
So we have our s, p, d, and f orbitals.
Right?
We know that s can be in the first energy level up.
We know that p starts at the second energy level, d starts at the third, and f starts at the fourth.
The tricky part is that you don't just fill up 1s, 2s, 2p, 3s, 3p, 3d.
There's a little bit of mixing up of the energy levels versus the actual energy of the orbital.
So a nice side of the page trick is to draw diagonal lines down through these numbers.
And if you follow these arrows, you'll fill up in order of increasing energy.
For example, after you fill up the 2p, you're going to fill up the 3s then the 3p, the 4s, and then you'll go to the 3d.
If that doesn't make sense, you might want to go and review a little bit about the orbitals in your textbook, or other resources.
The other way you can think about the ordering of the energy of the orbitals, is to be familiar with the periodic table.
So we're just going to have an aside over here.
And a general, very rough sketch of the periodic table would look something like this.
OK?
You have your alkali, alkali earth metals over here.
You have aluminum, oxygen.
Your halides, your noble gases over there.
But the way I drew it here is in what we call blocks.
So this is your s block, and then we have our p block over here, d block, and the the f block is down here.
So once you get more familiar with electron configurations, you can actually looks towards the periodic table, and know, depending upon where your element is in the periodic table, what its electron configuration is.
Or at least the electron configuration of the valence electrons.
OK.
So moving back to our trick, which we'll use now, we know that we have 22 electrons, and we know the order that we need to fill the orbitals.
So let's start writing out our boxes, which is how Professor Sadoway wanted us to answer this problem.
So we have 1s, our 2s, 2p.
If you're unsure why I'm only drawing 1 orbital for s and 3 orbitals for p, again, you might want to go review that beginning orbital material.
So we filled, we have 1s, 2s, 2p, 3s, then we're going to do the 3s here, we're going to do the 3p.
And after the 3p, we do the 4s, and after the 4s, we do the 3d.
Which has 5 boxes, right?
So now that we know which orbitals we're going to fill, we need to know some more rules about filling them.
So the second rule, over here, is the Pauli exclusion principle, which says that no two electrons can have the same quantum numbers.
For us, that means that we basically can only have two electrons per orbital, and of opposite spin.
So that's something we need to remember for this.
The third is Hund's Rule.
And that is, we want to have as many unpaired electrons as possible.
So that will come into play when we're filling up our boxes in just a second.
If these are confusing, again, we want to look at the lecture material.
So I forgot my boxes here.
Now let's start the fun part of filling up the elections.
So we fill up 1s, spin up and down, the 2s, spin up and down.
So that's 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, and we have 2 more.
So in the 3d, we're going to do 21, 22.
And I separate them out because of Hund's rule.
We want to maximize the number of unpaired spins.
They both spin down, or both spin up, but they will be the same spin, and they will be in different orbitals.
So this was the answer for this part of the problem.
Now let's move to the second part.
And maybe you can try by yourself, and then we can do it here.
All right.
Now we're going to do the second part of this first part of the problem number 2.
So it asks us for the electron configuration in box notation of magnesium 4 plus.
So again, we're going to determine how many electrons we have.
Which, neutral magnesium has 12 electrons.
But the 4 plus signifies that we have 4 less electrons than neutrality, and so we subtract 4, and we have 8.
With 8 electrons to work with, we again want to write down our boxes using the energy level tricks that we have.
And now we can fill them up.
The Pauli Exclusion Principle tells us 1 spin up and 1 spin down per orbital, and Hund's rule tells us that when we fill up this 3p, we're going to try to maximize the number of unpaired electrons, and then, when we have to, we pair up the spins.
So that's another main mistake that people make, is they pair up too many spins, violate the Pauli exclusion principle, putting 2 electrons of the same spin in one orbital, or something of that nature.
So this again is the correct answer, and would-- oh!
I'm sorry.
This is 2p.
I'm sure you guys caught that.
And would have awarded you full points in this part.
All right.
Now we're going to start the second part of this problem, part B.
So first we want to read what the question is asking.
Give the chemical identities of the species with these ground state electron configurations.
So now we're working in the opposite direction.
In part A, we were given a chemical species and asked for the electron configuration.
Now we are given the electron configuration, and asked for the chemical species.
So the first one has the electron configuration of xenon.
So this is a noble gas configuration, plus what turns out to be 27 electrons.
One way you can do this, is to go to xenon, which is in the fifth row of the Periodic Table, and count through 27 electrons, remembering that when you get to the d on the sixth and seventh row, you need to go to the f block, which is signified in this problem by having f block electrons.
Count through the f block, the d block, and end up at thallium.
Or if you are familiar with the Periodic Table enough that you can recognize the valence electrons are in 6p.
And the 6p only has one electron in it.
So that tells me, xenon is in row 5.
This element is going to be in row 6.
I go to the first column of the p block, which signifies that there's one electron in that 6p orbital, and I see that it's thallium.
That's the quicker way to do it.
On an exam, you might want to be that familiar with the periodic table to be able to do that, because you're certainly time-crunched, a lot of times.
So because that's a neutral atom, that method will work very easily.
The second part is a little trickier, because it has a charged species.
So we have a net charge of 4 plus, an electron configuration of argon and 3d5.
This may seem a little confusing at first glance, because we always fill the 4s before the 3d.
But when you're ionizing an atom, it turns out you take electrons from the 4s first, even though it's of lower energy.
So that's a little subtlety that's not too important to this problem, but it may have tripped you up when you were looking at this, and certainly caused some problems for the people in this class.
Again, we go to the periodic table and we see where argon is, and then we go to the next row.
The way I like to think about this, is this is the electron configuration, missing 4 electrons.
To find the element that has this charge, I'm just going to add back in the electrons, go to that position in the periodic table, and that is my elemental species.
So we want argon plus, instead of 5 electrons-- I'm sorry, the problem says this is a 3-- instead plus 3 electrons, we are going to go to argon plus 7 electrons.
And going through the periodic table, you'll see that that leaves you with manganese.
Adding the 4 plus, because that is part of this chemical identity, we get the answer of the problem.
So again, the charged species make this a little more complicated.
The easiest way is to go back and add in the missing electrons, because that is your chemical species, and then remember to signify that there actually is that charge for your answer.
So moving on to part C, it asks us, write the quantum numbers of one of the 3d and one of the 4s electrons in iron.
So this requires a knowledge of what quantum numbers are, and what values they can have.
So let's start with the 4s, because that's a little more straightforward.
He tells us that we have an n quantum number, l quantum number, m and s. And, hopefully, we all know that n is the principal quantum number.
It's basically your energy level.
l gives you what type of orbital you're in.
Right?
If you're in the s, the p, the d. So it just tells you your orbital shape, basically.
m gives you the angular identity of the orbital you're in.
We didn't really talk about that in depth.
We just need to know, it tells you which of the p orbitals you're in, or which of the d orbitals you're in.
So I'll say, which orbital.
And the s is your spin.
Right?
So for a 4s electron, it doesn't actually matter what the identity of our species is.
So that he tells us is iron is kind of extra information.
A 4s electron already has 3 of its quantum numbers set.
So we know that n equals 4, for s, l equals 0, and m, which orbital, s only has 1 orbital.
So m always equals 0 for s, and your spin.
And I'm going to make that a script, just so it's less confusing, is plus or minus 1/2.
So the answer to this problem could have been 4, 0, 0, 1/2, or 4, 0, 0, minus 1/2.
Three of them are set by the fact that this is the 4s orbital.
The spin can either be plus or minus 1/2.
If you're unsure of where I got these numbers, and why they have to be such, you may want to review the lecture on that.
So moving onto the 3d quantum numbers, we know that n is 3, l for d is 2, m is negative 2, negative 1, 0, 1, or 2, right?
Because there's 5 different orbitals in a d set.
And then again, s can be plus or minus 1/2.
So picking one set of those, you could have said 3, 2, minus 2, plus 1/2, or any combination of 3, 2, and the m and the s. You don't have to get the exact same answer as your neighbor or someone else you're-- well, you shouldn't be looking your neighbor's paper, or a classmate, but these do signify that you're in a 3d electron.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Hi.
I'm Jocelyn, and we're going to go over fall 2009, exam 1, problem number 3.
With every question, we want to make sure we read the full problem first.
So answer the following questions about the difluoro-iodate ion.
Draw a three-dimensional representation of the molecular geometry around the central atom, not simply the Lewis structure.
Show all atoms and bonds between them.
Because this part seems pretty separate from the rest of them, we're going to start with that.
So we have a difluoro-iodate ion.
It has a positive charge.
And it asks us to draw the molecular geometry in a 3D representation, and not just Lewis structure.
However, to do the molecular geometry in 3D, we need to know the Lewis structure.
So we're going to start with that.
To draw the Lewis structure, we first need to figure out which atom is the central atom.
And it's always a good idea to do that by looking at the relative electronegativities.
As we know, fluorine is the most electronegative element in the periodic table, and so that's probably not going to be in the middle.
Therefore, iodine, being the less electronegative, is going to be in the middle, and it will have fluorines on either side.
This is our rough sketch of the molecular geometry.
Not too important right now.
Next we need to know how many electrons we have.
Iodine has seven valence electrons, fluoride has seven valence electrons, and we have a positive charge, so we know that we're missing one.
And that gives us 20 electrons.
Now that we know how many electrons we have to work with, we need start filling them in.
I always start with the outer electrons.
Those are usually, hopefully, the most electronegative, and therefore, the most likely to have the octet rule satisfied.
Also, if you remember, fluorine is in the second row, and therefore needs to have the octet rule satisfied.
Iodine is a little bit lower in the periodic table, so the octet rule isn't as important.
It can have an extended octet if we need it to.
So we will work with iodine second.
Let's start out with satisfying the octet rule for the fluorine, putting first the seven valence electrons that the fluorines brought themselves, and then noticing that each needs to share one from iodine, so that we have a full octet.
So each has one bond with iodine.
We've used 8 electrons, or 8 per fluorine.
So we have a total of 16.
We have 4 left, and we're going to put those on the iodine.
Counting up how many electronics are around iodine, 1, 2, 3, 4, 5, 6, 7, 8, we see that the octet rule is also satisfied for iodine.
This is looking like a pretty good Lewis structure.
Now, just to make sure it's really good, we're going to look at the formal charge.
So if you recall, the formal charge is the number of valence electrons that the element usually has, and minus the number of unshared electrons, and the number of dots.
So this sum subtracted from the valence electrons will give us the formal charge.
Looking at the fluorine, we have seven valence electrons.
We have 1, 2, 3, 4, 5, 6 unpaired, and one bond to the fluorine.
So that gives us a formal charge of 0.
The same goes for the other fluorine.
So I guess its formal charge is 0.
Now for the iodine, we have-- I'm going to do the formal charge down here-- it again has 7 valence electrons, usually.
We have 4 unshared electrons, and 2 bonds.
That gives us a formal charge of plus 1.
Because we have a charged species here, the full molecule has a plus 1 charge.
There's no way we cannot have a net formal charge.
Right?
We need to have a net formal charge of plus 1.
And it makes sense that that formal charge is on the least electronegative atom.
So this looks like the right Lewis structure for our purposes.
One more thing that you might want to write down.
I'm not sure if points were taken off from this.
But technically, if you have a charged species, put brackets around your Lewis structure, and put the net charge on the outside.
This is not, however, our final answer, right?
This is the Lewis structure.
But the question asks for the molecular geometry to be represented.
So if we remember how we determine molecular geometry, we need to look at the number of electron domains around the essential atom.
So we're going to move over here, and I'll redraw our Lewis structure.
And remember, an electron domain can be either an electron pair, or a bond.
So we see that iodine has 4 electron domains.
And if we remember our skeletal geometry, we know that that's tetrahedral.
Thus, to draw this in a more 3D fashion, we might want to say that one of the electron pairs is here, one of the electron pairs is over here, and then we have a fluorine coming out of the board, and a fluorine going into the board.
We didn't really go over how to do three-dimensional drawings.
So as long as you show something like, you know the fluorines will be an angle, not straight from the iodine, and that the electrons be on the other side, something to that effect.
A little bit more detailed than just a Lewis structure, was what Professor Sadoway was looking for in this problem.
And that would be your answer.
And for this one especially, I'd say it's important to box your final answer.
Because you hopefully would have figured out the Lewis structure, but this is not the correct answer.
So you always want to signify what answer you want to be graded.
Part B says, name the type of hybrid orbitals that the central atom forms.
So we are almost all the way to answering this question, so I'll put it down here.
Remember, hybrid orbitals are-- we have hybrid orbitals, because we can't really say what orbitals exactly are bonding to which atoms.
We want to have 4 equivalent orbitals for this system, right?
We have 4 electron domains, so we want 4 equivalent bonding orbitals.
And from our talks earlier in the lectures about hybridization, we know that if we want 4, we take 1S and 3P, and that gives us the hybridization of the SP3.
if we only had 3 electron domains, it would be SP2, because we only use 3 orbitals.
And the number of orbitals that go into the hybrid is the number of orbitals you will get out.
So we know that our hybridization is SP3.
And that goes along with the fact that our shape is tetrahedral.
Now we need to name the molecular geometry of this molecule.
so we already have the molecular shape and the kind of spacial arrangement is tetrahedral.
However, when we're naming the molecular geometry, electron pairs no longer matter.
So for part C, tetrahedral is our skeletal geometry, right?
It shows us our special arrangement.
The molecular geometry is ignoring those electron pairs.
So it is called bent when we only have 2 bonds.
And I think about that because in spectroscopy and ways that are used to study molecules, you can't see the electron clouds very well.
So they could only see the iodine and the two fluorines, thus looking like a bent structure.
So hopefully you're sticking with me here, because this problem has a lot of parts.
And we're going to move on to part D now.
Part D asks, is difluoro-iodate polar or nonpolar, and explain.
If you put down polar or nonpolar and did not explain, we assume you guessed, and you didn't get any points.
Explaining shows that you know why, and you understand the concepts behind this, and thus deserved points on this problem.
So there's two different parts of this problem.
First, to have a polar molecule, you need to have polar bonds.
So I would first ask myself, are the bonds polar?
We have a difference of electronegativity that you can look up on your periodic table and know is not negligible.
And so yes.
A difference in electronegativity signifies that our bonds are polar.
However, not all molecules that have polar bonds are themselves polar, right?
if we have spacial symmetry, those polar bonds could cancel out.
So now we need to make sure, is the molecule polar?
And for that, we need to have answered the first questions correctly.
Right?
We need to know that this is not a linear structure.
If this was a linear structure, those polar bonds would be geometrically symmetric, and cancel each other out.
But we all are very smart, and we got this part C right.
We know that we have a bent structure.
Thus, I'll rewrite this over here.
We know that if we draw in our polarity vectors, we can realize that we'll have a net dipole pointing down.
So the answer to this is yes, because of the difference in the electronegativity, and because of the spacial asymmetry.
If you said both of those things, you would get full credit.
If you said one or the other, you'd probably get part credit.
But knowing to look at these two things shows that you fully understand what it means to have molecular polarity.
All right.
So now we're going to move to part E. This part is changing gears a little bit, so if you didn't quite understand the first part, you can still try this one.
Part E, again, what is the question asking us?
Determine the maximum wavelength of electromagnetic radiation capable of breaking the IS bond.
So we need to find the maximum wavelengths which corresponds to the minimum energy.
Right?
Because we have e equals hc over over lambda for electromagnetic radiation.
We're trying to figure out the lowest energy of photon that will break the IF bond.
Now, what are we given to answer this question?
We're given the energy of the homogeneous fluorine-fluorine bond, is 160 kilojoules per mole.
And the energy of the homogeneous iodine-iodine bond is 150 kilojoules per mole.
And in order to find the wavelength that will break the IS bond, we need to know the energy of the IS bond.
Right?
That's what we're working towards in this problem.
So given the homogeneous energies, do we have a relationship that will help us determine the energy of the IS bond?
If you recall, in lecture 9, Professor Sadoway went over the Pauling formula for determining a heterogeneous bond energy from the homogeneous bond energies, and the electronegativity.
So I'll just write that up here.
The Pauling formula is, in a generic form, the energy of an AB bond equals the square root of the product of the homogeneous bond energies plus a constant times the difference between the electronegativities squared.
And I just said what all of these terms mean.
Because even if you had this on your equation sheet in the test or something, a lot of people wrote this down, and then used it incorrectly.
So it's really important to know not only what equations you have, but especially what each of the terms in the equations mean.
So we're given the homogeneous bond energies, and we have a periodic table, or some other resource, that has the elecronegativities.
Next step is to plug in the numbers and find the energy of the IF bond.
So we have the square root of 150 kilojoules plus.
And the exact values you get for the electronegativity may vary, depending on the source.
But they'll be close enough, and relatively, probably very-- the difference between them will be very similar.
So don't worry if your numbers were a little different than mine.
So we wrote all that out, and plugging it into the calculator, you get that the IF bond has an energy of 323 kilojoules per mole.
Now that we've found the energy of the IS bond, are we done with the problem?
You have an answer.
You might be ready to move on to the next question.
However, this isn't what the question is asking us, right?
We are asked for the maximum wavelength that can break a bond of that energy.
So we need to go back to our energy relation to our wavelengths.
And we know that the wavelength equals hc over the energy.
Now here's where some people tripped up a little bit.
You have a value for the energy.
h and c are constants.
And you find the lambda.
However, if you plugged that all in, your answer would be 6.15 times 10 to the negative 31 meters.
Some people, that was the answer that they found.
And it makes sense.
You have an energy, you have some constants, and you can get your lambda.
However, if you think about it, does this wavelength actually make sense?
The gamma ray radiation has a wavelength of about 10 to the negative 17.
That's very, very, very energetic electromagnetic radiation.
So this is 14 magnitudes smaller than gamma ray radiation, and it just doesn't make sense for a bond energy of this much.
Plus it doesn't-- it just-- you should look at the answer you get, and try to be comfortable with if it makes sense or not.
So where people got tripped up, is in the units.
We found Pauling's formula gives you the bond energy in kilojoules per mole.
However, if you look at Planck's constant, it has units of joules time seconds.
The speed of light is in meters per second.
And you want lambda in meters.
To make everything or cancel out, that means we want the energy to be in joules.
Not joules per mole, because we're looking at the wavelength of one photon, right?
We're looking at the energy of one photon to break this bond.
So we want the energy of the bond, not per mole of bonds, but just of one bond.
So instead of just having this equation, we want to have put in h times c over 323, change it to joules, and then multiply by-- sorry, moles are on the bottom, 1 mole and Avogadro's number.
And Planck's constant of the speed of light, you should have on your equation sheet, or somewhere else, some other resource that you have.
You don't need to memorize those in most cases.
If you remembered to divide by Avogadro's number, you got 3.7 times 10 to the negative 7 meters, which equals 370 nanometers.
And if we recall, the light we can see is in the 400 to 700 nanometer range.
So this puts us in ultraviolet radiation, which makes sense that it would take that much energy to break a bond.
The other problem some people had was forgetting to convert from kilojoules to joules.
That would still give you a physical answer.
But again, I can't stress unit cancel out enough.
So make sure you know that since your Planck's constant is in joules, you want to make sure you have joules on the bottom.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, I'm Sal.
Today we will be doing problem number 4 of fall 2009, exam 1.
Now before you attempt the problems there's certain background information that you should know before attempting it.
One is knowledge in ionic bonding.
Because the problem deals with ions.
Two, how to draw an energy-level diagram.
That's very important.
And three, what Madelung's Constant is.
Now before attempting the problem, it's a good thing to read the whole problem in detail and make sure you don't skip anything out of it.
So that you get all the information that's given to you.
Because that's very important.
So the problem reads as follows.
For a given cation, c and anion a, show that the following four energy states on the same energy-level diagram.
1, Ions in infinite separation.
2, 9-pair c and a, c stands for cation, a stands anion.
3, An ion line, c a c a c a, just a repetitive line of anions and cations.
And 4, crystalline solid of c a.
So a three-dimensional crystal structure with t and a.
And it says also that, assume that the comparison is based upon identical numbers of ions in all four states.
The diagram need not to be drawn to scale, however you must convey relative values of different energy states.
So that's the problem reads.
Now if you recall from lecture there's an important equation that actually describes the energy between two ions, between a cation and an anion.
And this energy is directly proportional to the product of the two charges.
And it's inversely proportional to the separation distance between them.
And it's multiplied by some constants.
So what does this mean?
Well the fact that it's directly proportional to the product of your two charges means that your energy for any ion or anything that ionically bonds, it's going to be negative.
Because your z minus will be negative and that's the whole value of your energy will be negative.
And that's what dictates stability.
How negative the energy is.
So with that in mind, we can go ahead and start the problem.
So we have a cation a and anion c. Now I like to draw diagrams, or draw little pictures because that helps me when I'm solving the problem.
So I'm just going to draw pictures of a cation and an anion.
So I have a cation, c and then I have an anion, a.
So c and a, and the value of r naught from our equation is pretty much the separation distance between the two.
So project this up, this is r naught.
And if you look from the picture, r naught is just simply the radius of your cation plus the radius of your anion.
Now this is already describing the energy between just a cation and an anion with these magnitude of charges.
So all that tells me is that, this is already answering number 2, pretty much from the problem that we're asked for, which is the ion pair.
So the problem asked you to draw an energy-level diagram.
That's what I'm going to do over here.
So I'm going to go ahead and start drawing the energy-level diagram.
Now I'm going to go ahead and label my energy axis to be positive to be up.
And I'm going to draw a baseline of 0 here.
So I know that to be able to incorporate my ion pair-- like I said the energy is negative, so if the energy goes up to here and let's say the value of the energy here is 0, then that ion pair should lie somewhere below it.
So I'm going to go ahead and draw this line.
And I'll label this, s 2 for our energy level diagram.
So this is the ion pair.
Now number 1 says ions at infinite separation.
What is the energy of two charged species that are separated by infinity?
Well if you look at your equation, your equation tells you that it's inversely proportional to the separation distance.
So all that tells you is that if you divide by infinity, your energy should be essentially 0.
Which makes physical sense because those two charged species can't feel each other when they're separated.
So this is actually already my number 1, which is 0.
Separated away.
So now we need to find what the energy is of an ion pair.
Which is the line of c a.
So we want to draw a little picture.
It's pretty much going to be a repetition of that.
And if I draw a line-- that's not very straight-- I'm going to draw a bunch of cations and anions.
Essentially I can just imagine that it extends to infinity in both ways.
So with my energy between these two ions right here is given by my equation, then I should be able to figure out what the energy is of the whole system.
Because we're talking about electrostatic interaction, which you can just add linearly.
So I can make one assumption.
But look at my cation and anion pair, I can go ahead and simplify my life by letting z plus be 1.
So I'll write that down.
I'll let z plus equal to plus 1, which is the charge on our cation.
And I'll let z minus equal minus 1, which is the charge on the anion.
So that's to help with the math.
So if I come back over here and if I look at my equation line then I know that if I draw-- let's see-- a reference access here.
Then the separation between the cation and anion is r naught.
And between the cation and the other cation is going to be 2 r naught.
And it's going to extend for that.
So if I go ahead and relabel my energy equation for a line.
I'll go ahead and call it e line of function of r naught.
It's simply going to be the first one, which is going to be negative e squared over 4 pi epsilon naught, r naught, multiply the 1 over minus 1 over n. So that's the energy between these two ions.
So now because the energy is electrostatic, this cation also feels the repulsion of this other cation.
So therefore with that you have got to add what that interaction is.
So the product of the two charges is plus 1, which gives it a plus in front of your equation.
And I end up getting e squared over 4 pi epsilon naught.
Now here's something that you should pay particular attention.
The separation between these two cation is not r naught.
It's actually 2 r naught.
Because of where exactly where it sits on the line.
So my separation distance now becomes 2 r naught and it's multiplied by the same 1 minus 1 over n factor.
Then if you repeat it again, you're analyzing the interaction energy between the cation and the other anion that's next on the line.
So as you can see you end up getting this nice-- 4 pi epsilon naught, 3 r naught, 1 minus 1 over n. And essentially it repeats until it ends.
Now I'm just analyzing on the right side.
Because it extends, it's an infinite line, because it extends from negative infinity to infinity, then you can also assume that there's an anion to the left.
So all you really have to do is, you take your overall energy for the right side and you multiply it by 2.
Because it's additive.
So this whole thing gets multiplied by 2.
And that's essentially what the energy is of a line.
Now in order to be able to know where it lies on the energy-level diagram, it's important to try to get it into the same basic form as the energy of an ion pair, just something simple like over there.
So just by looking at the equation I know there's a lot of factors here that are common.
Like that e squared, 4 pi epsilon naught, even r naught and this factor can all come out of the equation.
You can take it out.
So if I do that, my equation of my line then simply becomes negative e squared over 4 pi epsilon naught.
Now I'm going to go ahead and take out the r naught as well.
And I'm going to multiply it by the factor.
And essentially what stays from here is just your 1 over 1.
From this one it's your 1/2 and then your 1/3 and then we're happy.
And because I also took out the negative, I want to make sure that I take out the negative from all the other ones too.
So this product then gets multiplied by 1 minus 1 over 2, plus 1/3 minus 1/4 plus... and it goes on forever.
Now it so happens that this is actually a series that we have an answer or an answer to.
And this product that's being multiplied by your fundamental base of your equation is just simply-- oh sorry I forgot the times 2, don't forget that, that's very important because you're summing from both sides-- this essentially right here it's simply arised because of your geometry.
You're analyzing your systems.
And this value happens to be greater than 1, which is a good thing.
And the value of this is actually 1.386.
So the fact that the whole equation is still negative, because we're talking about coulombic interaction between a negative charged species and a positive charged species.
And we're multiplying that by a value that is greater than 1 this means that if this is 1 right here, then our energy level diagram should lie somewhere below it, like right here.
So this is 3.
And that's your ion line.
Now in real life we know that crystals don't exist in lines really.
Or they're not just an ion pair.
it's literally something that is three-dimensional and has an ordered structure.
Sometimes could be different geometries, but normally it's a cubic structure.
So because that forms in nature, then that tells me that the energy of a three-dimensional crystal, which is the fourth one that we need to put on the energy-level diagram, the value should be greater than 1.386.
And this value of 1.386 is actually Madelung's Constant, which you should know.
It was a required preliminary to the problem.
So I know that below this value of 1.386 I have my three-dimensional crystal.
Now an energy-level diagram is pretty much quantized by different steps.
Now what is quantizing our problem here?
Well this is 0 because our energy is 0.
So we can almost assume that we're multiplying-- or that the Madelung's Constant for ion pair with infinite separation is 0.
And this is 1, so we can assume that the Madelung Constant is 1 or we know that it's 1 for the ion pair.
So if I start writing this out on the side here-- give me some space-- I know that's one.
I'm going to go ahead and label this by Madelung's Constant, which is 1.386.
So I come over here.
Relabel this.
This value here's is 1.386.
And this value here has to be greater than 1.386 for a three-dimensional crystal.
And this little diagram here is the answer to your question from the exam.
So when you take the exam you don't really have to go through all this math to answer the problem.
If you know the material from lecture, then the solution should be very straight-forward very easily.
And also if you keep in mind that there's a lot of material that you need to know before you take the exam, and knowing how to absorb that material and apply it will help you in solving these type of problems, which can be pretty difficult.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu.
Hi I'm Sal.
Today we're going to be doing problem number 5 of fall 2009, test 1.
Make sure that you read the problem in full detail before attempting the problem.
And there's also some things that you should know, pretty much background material.
Before attempting the problem.
And that's what I need, which is W I N. You will win if you know how to manipulate these equations.
The first equation is the energy of two charged species as a function of their internuclear separation.
And the other equation is a function of just the separation between two ion pairs at equilibrium, which is given by r naught.
And r naught is just simply the sum of the radii of the cation and the radii of the anion.
So if you look at this equation there's two components to it.
The first component is actually the component that is responsible for the attraction of your charged species.
And the second component is the repulsion of it.
Because essentially ionic bonding happens because of your electrostatic interaction between an cation and an anion.
So with this in mind we can go ahead and read the problem and solve it.
So the problem reads as follows.
On the same graph below-- and this is the actual graph that we're going to be doing the problem on-- it says on the same graph below, for one, BeF2, so beryllium fluoride, and two, BeO, which is beryllium oxide, sketch the variation of potential energy with internuclear separation arc between a cation and an anion pair in each compound.
The diagram need not to be drawn to scale however you must convey the relative magnitudes of key features.
So the first thing that I would do is write down my compounds that I'm using.
So I'm focusing on beryllium fluoride, is the first one.
So if I look at beryllium fluoride, I'm going to look at the relative magnitudes and charges when they're in their cation and anion state.
So I know that for beryllium, beryllium forms a cation that is 2 plus.
So it has a positive charge of 2, which means that it lost two electrons.
So I can say that z plus for beryllium is plus 2 and for fluorine, when it's in the anion state, my charge is equal to minus 1.
So 2 plus minus 1.
So with this in mind, you can take this and it will help you solve the problem.
Because like I said, the first part of the equation is pretty much the attraction part, which involves the product of your charges.
And I want to go ahead into the same thing for-- so this is for BeF2.
And for BeO we have the oxygen 2 minus.
So my anion has a z minus of minus 2 and my cation just has z plus of plus 2.
Now if you look at your equation over here.
There's two things that will dominate at certain parameters.
Now this value of n is known as the Born Exponent, and that has a value between 6 and 12.
So it's always greater than 1.
And this is actually what's nominating your repulsion, so I'll do some colored chalk.
So for the attraction part I'll circle it.
And the fact that it's being attractive means that one of these charges has to be negative that will make the overall component negative.
Right here because you'll have q1 times q2.
And in our case if we look up beryllium fluoride, you have beryllium 2 plus you have fluorine minus, so if you make the product of those two you get minus 2.
And you have this other component, that is the repulsive part.
So if I was to draw these independently, not as a sum, I know that for my attractive part, which is the negative, it should look something like so, as a function of r. And my repulsive should look something like so.
And this makes sense because as your radius becomes smaller and smaller then essentially if you get below 1, your potential energy curve for your-- the component for your repulsion-- spikes up.
So if I was to superimpose these two, then I should expect that the curve to look something like so.
Now this is good because this is exactly how the curve should look.
And there's some characteristic features here that we want to go ahead and use BeF2 BeO2, draw on the same diagram to compare together.
And this point right here on your r-axis is simply just your r naught.
So it's just the sum of your cation radius and your anion radius.
And this valley here is your energy at r naught.
Which is the second equation that you should know how to apply.
So if I want to analyze my system, how do I know which one has a bigger radii than the other?
Well I know that my cation is common.
So in both cases I'm dealing with beryllium cations.
But I'm dealing with a different anion.
Now I know that both fluorine and oxygen anions have the same number of electrons, from looking at the Periodic Table of Elements, or their charges that they have.
But fluorine has one more proton.
So what that does is that it allows the fluorine to pull the electron cloud tighter together and that decreases your radius.
So just by looking at the Periodic Table of Elements, I can already assume that my radius for my fluorine should be less than the radius of my O2 minus anion.
So if I look at my system than I know that if I want to look at the internuclear separation, the r naughts of the two compounds, then I should assume-- or have an educated guess based on my data that I have here-- that r naught for beryllium fluoride should be less than the r naught for beryllium oxide.
So this allows you to put the position of your radius on your curve.
So I'm going to go ahead and just redraw the a new axis over here just to compare both of them so you won't confuse yourself.
So we know this is the energy of our system.
And we concluded that our, that the radius of your beryllium fluoride is less than the radius of beryllium oxide.
So I can go ahead and mark this as r naught of BeF2 and I'll mark that as r naught of BeO.
So this tells me exactly where the low point of my potential curve should lie.
But how do I know how deep it sits in the well?
Well that's going to be dictated by the equation that governs the energy at r naught.
And that's our simple equation of when it's the function of r naught.
And if you look at the equation, the equation is just the product of your two charges.
So if we come back and we look at our analysis here, we know that if I multiply these two charges together, you know the combination of these two, should be, q1 or q-- beryllium 2 plus, I'll call that q1 and I'll call this one q2.
The product of these two gives you minus 2.
But for this one, q1 q2 gives us minus 4.
So this tells me that the electrostatic interaction for beryllium oxide is a lot stronger than beryllium fluoride.
And it makes sense because you have plus 2 and minus 2 being attracted to each other instead of plus 2, minus 1.
So on my graph I can assume, I can guess, that my value of my beryllium oxide should sit lower, more negative, on the potential curve than beryllium fluoride.
So if I designate this as the low point for my beryllium fluoride then my curve for this system, for that particular compound should take a certain type of shape.
And so again, this is your energy, the bottom point of r naught.
And if this is the energy given to the fact that we have a higher magnitude when you multiply the two charges together, then it should sit lower on the potential well.
And this will be my low point.
So my curve should also look something like so.
So if you were able to do this on the exam you would have gotten full points, which is good.
And, again the key points about this problem is just knowing what equations to use by reading the problem.
The problem talked about internuclear separation, well we've learned in class how to treat those problems and how to use equations to be able to describe the way that the energy should look, which is a good visual.
And by looking at the Periodic Table of Elements, just thinking about how many protons, how many electrons certain cations have, you should have an estimated guess of what your ionic radius should be.
And with that you can go ahead and look at the math.
Look where that parameter plays a role in your equation.
And then decide whether or not it'll be greater or less or what have you.
So with that in mind, I dare you to try the problem now and give it a good shot.
The following content is provided under a Creative Commons license.
Your support will help MIT Open Courseware continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Open Courseware at ocw.mit.edu.
Hi I'm Sal.
Today we're going to solve problem number 6 of exam 1 of fall 2009.
Now before you attempt a problem, there are a lot of things that you should know, that is background material that's going to help you solve the problem.
Especially during an exam, given that this was the hardest on this exam.
So you should be able to finish this within 15 minutes or so.
So before you start the problem you should know what the energy is of a charged particle in an electric field, which is given by this equation.
The conversion from an electron volt to a joule, which is just a conversion of energy and this should be given to you on your table of constants.
Also the energy associated with an emission spectrum, which is given by this equation where k is actually the ionization energy of hydrogen, which is 13.6 electron volts and the z squared is the atomic number of your atom of interest.
And then the n f and the n sub i are your transition states.
The DeBroglie wavelength, which is given by lambda, is equal to a Planck's Constant divided by the momentum.
And always remember that you need to conserve energy.
This is what's going to get you the full points on the problem.
So the problem, it's best to solve if you draw a little image as you read the problem.
So the problem reads, atoms of ionized helium gas, He plus, are struck by electrons in a gas discharge tube operating with the potential difference between the electrodes set at 8.8 volts.
So I'm going to go ahead and start drawing an image.
So let's say I have two plates where the voltage between these two is set at 8.88 volts.
And what's happening here is that you're accelerating electrons through here.
So you can imagine that-- say this is positively charges and this is negatively charged-- that an electron starts from your negative plate and then it accelerates toward your positive plate and you can imagine your plate having hole or something small where your electron can then exit and then be in free space under the influence of no potential.
So then these electrons are actually being aimed towards your helium plus ions.
So that's your target.
Now if I continue reading the problem, it says that the emmision spectrum includes the line associated with the transition from n equals 3 to n equals 2.
Calculate the minimum value of the DeBroglie wavelength of scattered electrons that have collided with helium plus and generated this line in the emission spectrum.
So it's an energy conservation problem.
And all this tells you is that, you have an electron that is being accelerated through your potential.
The electron is going to strike a helium plus atom and upon that you detect an emission spectrum.
Well what does that tell you?
That tells you that that electron actually excited an electron in the atom.
The electron had to transition to a higher state to absorb energy and then decay and in the process of decaying it emits a photon, which is what you detect.
So this information is given to you.
But the problem asks to find the value of the DeBroglie wavelength.
So since this an energy conservation problem, and this is the very first step that happens and I know that my energy equation for a charged particle is simply q times the voltage.
So I know that q, which is a certain value of charge, the product of these two when I'm multiplying by 8.8 volts, this yields 8.88 electron volts.
So it's a new unit of energy where we just saw the conversion.
And the reason why this takes this shape is because one electron has the charge of 1.6 times 10 to the negative 19 coulombs and that's where the conversion from electrical volts to joules come from.
So I'm going to go ahead and call this e sub i initial.
Because this is our initial energy that we're given.
So we just start with this energy, nothing else.
Nothing else was given from the problem.
So we want to go ahead and conserve this.
So our final state, the combination of whatever we are solving, should not have a higher energy than what initially we started with.
So because the problem says that we have an emission spectrum, I'm going to go ahead and draw another image.
I'll label this one, initial and I'll label this one, final.
So my final image, you can imagine a helium plus atom.
And the electron strikes the helium plus atom and we detect a photon and you're getting a scattered electron.
That's what the problem says.
Scattered electron.
So what this tells me is that this energy now is going into creating a photon and scattering an electron from your atom.
So if I was to equate those two or what not I can already say that, well in my final state I detect an emission or an emission spectrum.
And I know what the transition says.
It goes from n equals 3 to n equals 2.
That's what the problem tells us.
So if I take my equation, my delta e of my emission spectrum, is simply going to be negative k, which is 13.6 electron volts.
And I'm going to multiply by the atomic number, squared now.
This is where people also make mistakes during the exam.
The atomic number is not 1.
It's actually 2 because we're talking about helium.
And the atomic number is not the number of electrons or values in terms of an ion, but it's actually the number of protons that you have.
So helium has 2 protons.
So I can multiply this by 2.
And I'm going to square it.
And I know that I'm going from n and equals 3 to n equals 2.
So if I plug this into my equation I have 1 over 3 squared minus 1 over 2 squared, which is 9 and 4.
And if you plug this in, and assuming that your calculator works and is correct, you get a value of 7.56 electron volts because that was the unit that I was using.
Everything else is unitless, so my energy is now 7.56 electron volts for my transition.
Which covers this part for my photon.
So now we're left with what the energy of this is.
So what is the energy of the scattered electron?
Well if I equate the initial to the final I know that the final is a composition of the transition in my scatter state.
So I know that e initial, which is 8.88 electron volts, this equals to the energy for the emission spectrum, which is 7.56 electron volts.
So I'll put the units in square brackets so you don't get confused.
And then plus the energy of the scattered electron.
So if I subtract the two, I get the energy of my scattered electron.
And that pretty much gives me a value of-- after I subtract that-- e of scattered electron is equal to-- well the distance between these two is actually 1.32 electron volts.
But don't stop here because the problem didn't ask you to figure out the energy of the scattered electron.
It actually told us to figure out what the DeBroglie wavelength is of this scattered electron.
Now the fact that the electron is scattered, to me that means that the only energy that this particle has is kinetic energy.
So it's a classical form of energy.
So I can go ahead and I can equate this to just 1/2 mass of the electrons times my velocity squared-- whatever velocity it has.
Now I'd don't need to figure out what the velocity is.
This is just important to figure out what the DeBroglie wavelength is.
So this is good.
Now if we look at what our DeBroglie wavelength is, we know that lambda is equal to Planck's Constant divided by p. And Planck's Constant has a value of 6.6 times 10 to the minus 34 with units of joules seconds.
So this is important too.
Always keep track of your units because dimensional analysis will help you a lot when solving these problems.
A lot of the times you're given energy values in electron volts but your solution will have a constant that doesn't have an electron volt, that will have a joule, so you're forced to make that conversion or else your problem is going to be wrong.
OK so with that in mind, we know what h is.
But what about p?
Well p is momentum.
And classical momentum is just mass times your velocity.
So this is just mass times velocity, but this is the mass times our velocity.
We know what the mass of the election is.
That's given on our table of constants.
But the velocity, we don't know what it is but we can grab it from our classical energy.
So if I look at my energy-- from the side so I'm going to look at my energy-- I know that my energy-- I'm going to call this scat, for scattered-- the energy of my scattered electron is simply 1/2 the mass of the electron times our velocity squared.
Now if I multiply both sides by 2m, I'm going to go ahead and get us to the point where I can get an equation that is just my mass times v. So I'm going to multiply both sides by 2m, So I get 2 mass of the electron, e scat of the electron, equals-- if I multiply this by 2m, the two cancel, so I get an m squared.
m squared times v squared is essentially just m times v squared.
And this becomes just mass of your electron v squared.
Now if I take the square root of both sides, I end up getting a nice relation for just m times v. So I know that m times v now is just simply equal to this-- canceled, the square root cancels the square.
The square root of 2 mass of the electron times the energy of the scattered electron.
So if I go back to my DeBroglie wavelength, I have m v. So I can take that value as a function of energy and just substitute it into my equation.
And that'll help us get the answer because we know what the energy of the scattered electron is.
We calculated that on the first part.
So with that in mind we'll come over here.
Again this is 6.6 times 10 to the minus 34.
And the units are joules seconds.
And down here I have the square root of 2 times my mass of the electron, which is on your table of constants.
And it's just 9.11 times 10 to the minus 31.
And this has units of kilograms.
And the energy of my scattered electron, which is 1.3 times 1.32 and this units of electron volts.
Now this has units of electron volts.
This has units of joules.
This problem's a little bit-- now you're not going to get a good answer with that unless you make the conversion.
I pointed it out, the bullet point in the beginning was that 1 electron volt is equal to 1.6 times 10 to minus 19 joules, so if I simply multiply this by that conversion, just 1.6 times 10 to the minus 19, this has units now of joules per electron volt.
So this has units of joules per electron volt.
The equation turned out to be pretty long.
But this cancels the electron volt and now has units of joules.
Which if you go through the math, you're going to go ahead and at the end get a unit of just meters.
Because joule is just kilogram, meters squared per second squared.
So all that factors out and you end up getting that.
If you go through the math and you get your math right, then this should yield a value of 1.06 times 10 to the minus 9 meters.
So this is the value that will get you the right answer on the exam.
Again this is lambda for your DeBroglie wavelength because that's what the problem is.
So the problem asked for the DeBroglie wavelength but it give you all this information to get to the point where you needed to solve.
And by conserving energy and knowing the right conversion factors between energies you're able to get an answer, which is good.
It's really easy to complicate things and get a wrong problem now.
I remember from grading the exams, the most common error that people faced was actually letting the energy of the electron be the energy of a photon.
Now that can't be because the energy of a photon is just your Planck's Constant times a frequency.
And that's the energy for massless particles, your electron has mass so if it's moving it's not going to be h nu.
The energy is going to be kinetic energy.
1/2 mb squared.
If it's not in an electric field, that is.
So with that in mind just be confident when you solve these problems and make sure that you know exactly what energy equations to use to get the right answer.
The following content is provided under a Creative Commons license.
Your support will help MIT Open Courseware continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Open Courseware at ocw.mit.edu.
Hi there.
Welcome.
I'm Brian Spatocco.
We're going to be going over exam 2 from the fall 2009 semester.
The logical place to start on this exam is problem 1.
And what I usually do for all the problems is discuss the things we need to know before we start it.
So I would recommend reviewing these concepts before attempting the problem.
And then attempt the problem and then identify your weak spots.
So here's what I personally found to be important information to know before attempting the problem.
So I call it what I need, (W I N), if you want to win.
There's four things.
The first are Miller Indices.
This is a crystallographic notation for figuring out and communicating planes and directions in crystals.
So review Miller Indices We have plane/direction notation.
I'll explain a little bit more what that is in a second.
Definition of planar density.
And for that matter, linear and three-dimensional density as well.
And 4, the concept of a crystal basis.
And this is actually the concept that most students had trouble with on this particular question.
So those are the things I would recommend you look at before starting.
And if you haven't, check it out now.
So let's start off with part a of problem 1.
And so the question asks us to draw the crystallographic feature indicated and label it clearly.
It doesn't actually say if we're drawing planes or directions and that's left to the exam taker to identify.
So the first thing I note on that problem, part a, is that we're asked to draw 0, 1 bar, 2 bar.
And 1 1 1 bar.
So the first thing I notice about these two particular crystallographic features is how they're actually encapsulated.
So the first one is in parentheses.
The next one is in square brackets.
That actually tells us some information about the problem.
The first part, a part one, is 0 1 bar 2 bar.
So you actually need to know what this implies.
Parentheses mean we're looking at a particular plane orientation.
Now I'm not saying a plane.
I'm saying a plane orientation.
Which means there are infinitely many planes in a crystal which have this orientation.
OK so that's distinguishing between just one plane and a crystal.
The second part, b, we have square brackets.
That implies we're either looking at a direction or a place in a crystal.
We also have things like this, which we're not seeing on the exam this time but may show up later.
We have these curly brackets and we have these angled brackets.
The curly brackets denote a family of planes.
And the angled brackets denote a family of directions.
And it's markedly different than a particular or plane orientation and a particular direction.
So let's do some visuals here to explain what we mean.
So let's start off with a 1.
Now we're asked for 0 1 bar 2 bar.
Now the bars indicate negative signs.
So you would know that from your Miller Indices review.
Now the way to do this problem-- what I find is the easiest way we reviewed in recitation-- was to not be afraid to move your origin.
A lot of people are afraid to move their origin around but we have to remember in a crystal, basically what a crystal is, it's a unit lattice repeated infinitely in three dimensions.
So it's OK to move your origin as long as you choose some reasonable point.
So in this problem we're looking in the back bottom left corner.
That's our defined origin on the paper.
But to do this problem, I would recommend choosing a new origin.
I chose this one.
And that's really going to facilitate doing this problem a lot more easily.
So let's look at what we have to dran now.
0 1 bar 2 bar.
The 0 indicates that we're never actually intersecting the x-axis.
OK, so at no point will this plane-- we know it's a plane because the parentheses-- intersect the x-axis.
The 1 bar indicates that we are intersecting the y-axis at 1 lattice parameter spacing.
Miller Indices are a little tricky because they're somewhat inverted.
1 implies you are going the full distance of the unit cell.
2 implies you're going half the distance.
3 implies you are going 1/3 the distance and so on and so forth.
So for 1, we're going to go the distance of 1.
We're going to move negative 1 in y.
So we're going to move negative 1.
We're going to go to here.
That's in our y.
And when we actually make this a little clearer.
I've chosen a new origin.
So here's my y prime.
Here's my z prime.
Here's my x prime.
So we're moving negative 1 and y.
We're going to move half, negative 1/2 and z.
And we're also told that this plane doesn't intersect the x-axis.
So it must run parallel to that asix.
Which tells me that it's going to run along the x-axis.
We're going to get a plane like this.
And here's our plane.
That's the answer to part a 1.
Pretty easy and most students got this correct.
For part a 2 we're asked to draw a direction or point to a particular place in space.
I also moved my origin on this problem.
And I found it easiest to redefine my origin up here.
Which means that my new set of axes are going to be x prime, y prime.
I'm going to keep z because z didn't move at all.
OK note how, when I'm drawing these things-- we're going to draw this one now-- that the way I choose to move my origin generally tracks with the negative signs that I have.
This problem I had a negative y and negative z, I moved in the negative y, negative z directions.
Here I had a negative z.
So I just moved up in the negative z.
That kind of told me how to move my origin.
It made life a little easier.
And now it's pretty easy from here.
Now what we're going to do is draw 1 in the x, we'll go 1 in the y, we'll go negative 1 in z.
We're pointing to this point.
There we go.
That's our direction.
So pretty easy.
Part a 1.
Part a 2.
So we're batting 100 right now.
And just don't be afraid to move your origin.
That's the take-away from this part.
The problem, of course, gets a little harder.
That's something that we pride ourselves on in 3.091.
If it was always this easy, everyone would pass the class and get 100s.
So the second part, part b, is asking us to calculate the density of atoms in a plane.
So this gets back to what I was saying before.
We need to understand the definition of a planar density.
OK, so I've sort of defined it here.
This is rho.
Not p. rho planar.
And rho planar, I define-- to help me, maybe it'll help you-- is the number of full atoms in a particular plane-- in this case we're looking at 0 0 1-- divided by some area.
Because the plane is denoted by an area not a volume.
Don't be afraid, I mean densities don't always have to be mass over volume.
It can be some unit over some specific distance or area or volume.
So that's general density.
So we're looking at the number of full items in a plane divided by the area.
We're told that we're looking at Dalium, of course named after Salvador Dali.
And so we're also told that we're looking at the 0 0 1 plane.
I've drawn a bcc crystal here.
As a scientist or an engineer, when you're given a bcc fcc, simple cubic, the first thing you put on that paper is the actual crystal structure.
So I drew the Dalium up here.
Not a real element, but a make-believe one.
And we're going to identify our 0 0 1 plane.
This is good practice.
So there's our x, y and our z is right here.
Our 0 0 1 plane, so we're not intersecting x.
We're not intersecting y.
We're only intersecting z.
So here's our 0 0 1 plane.
Let me sort of bring this down for you.
So we can look down on it from above.
It's a square, with sides a.
And here's our atoms.
So now the question is, how many full atoms do we have and what's the area?
The area's obviously a squared.
And the number of full atoms in this area is we have four quarters of an atom.
When you're in two-dimensional space an atom looks like a circle to you.
When you're in three-dimensional space it looks like a sphere.
So if you lived in the two-dimensional plane, you would see a quarter of an atom, a quarter of an atom, and so on.
You'd have one full atom in this plane.
So this is sort of what we're looking for.
And you say, oh, well I'm done.
A lot people left it at that.
But that's not the answer we're looking for.
We want an actual numerical answer.
So the question now becomes, what is a?
So we're going to digress and find a and then we'll know exactly the answer to the planar density.
So a-- this is where you have to be a little clever.
How do you find a?
The lattice parameter, the lattice spacing.
So the best way to do this is to think both small scale and large scale.
So first let's think about a unit cell.
Let me ask you, what's the three-dimensional density of a unit cell.
So how many atoms do we have per unit cell?
We have our bcc.
We know in bcc there are two atoms.
We have eight corner atoms and they're each 1/8.
We have one body atom.
So there's two atoms.
So we have 2 atoms per-- what's the volume-- well the volume is a cubed.
Now we got a cubed.
We didn't really make any progress here, we still have an unknown variable.
But the question is, how do we get solved for a cubed.
So let's zoom out from that unit cell.
Let's zoom out to a mole of material.
OK, so a mole of material, how many atoms are in a mole of material?
Well by definition, it's Avogadro's Number.
6.02 times 10 to the 23.
And then what's the volume of a mole of material?
Well looking back at the problem, we're given the molar volume.
Which is something that I would recommend you write down.
You know all the information you're given, see what you've got.
The volume of a mole is the molar volume, which we're given.
So now it's pretty easy.
We know Avogadro's, we know molar volume.
So we can solve for a.
So not too bad.
You'll find that a is about 2.8 times 10 to the negative 8 centimeters.
You could give me 2.85, 2.79.
We're worried about the concepts here, not the numbers.
So don't go back and waste a lot of time getting the exact number.
That's our a.
And now that we've got a, we've got our planar density.
Very easy.
I'm going to write rho plan, just to save some time.
And that equals 1 over this number squared.
That's one atom, by the way.
And that's all squared.
And we're going to find in the end that our planar density is going to be equal to 1.27 times 10 to the 15.
And the units are important.
That's atoms per centimeter, squared.
If you don't give us the units then we have no idea if you're right or wrong.
So always include your units at the end.
So hopefully that helped.
We're going to take a second and clean off the boards.
And we're going to start with part c. OK we're back.
We're going to finish problem 1 now from the 2009 exam 2.
We're on part c. So thus far in part a, we've talked about Miller Indices, we've talked about the plane and direction notation with the parentheses and the brackets.
We've also talked about planar density.
And I would invite you to review those things.
Now we're going to go to part c, which is really the concept of the crystal basis.
And this was the part of the problem that most students lost points on.
So it's the thing that I would emphasize the most on this problem.
The problem asks us to-- and I put a sad face because that's how people felt after this part-- the problem asks us to draw a particular plane in a fcc crystal.
So we're given Magnesia, MgO.
They want us to draw that plane and show the atoms.
And we also want you to show the relative sizes of the atoms.
You know we have anions and cations.
And we actually give you the sizes of those ions, so it shouldn't be too tricky on that part.
But I think the best way to start is to go over the answer that was given that was wrong.
And then we're going to talk about why it was wrong and how to get the right answer.
So to start off, we want you to draw this crystal.
The best place to start is by actually drawing the crystal in 3-D and then we're going to take the projection after that.
So people think, MgO, they see fcc, and they say, oh that's not so bad.
I know what fcc looks like.
Let me draw fcc.
I'm going to use for this problem, I'm going to use this red chalk to represent magnesium and the blue chalk to represent oxygen.
OK so magnesium, we're going to have it as a circle like this.
This is magnesium, 2 plus.
And we're going to have the oxygen-- it's going to be slightly bigger because it's an anion.
This is O 2 minus.
Maybe I'll color this in too.
People said, OK easy.
fcc.
This is a simple problem, easy points.
They drew what they thought to be an fcc crystal.
So they did-- let's say they put the Mgs on the corners.
And they said, OK now I need my faces.
So maybe the oxygen are the faces.
So I'm going to put in my 6 face atoms.
So here are my 1 side here, and I've got the bottom and the top.
There's my 6 faces.
And they said, OK, easy.
I'm done.
And then they took the projection of 0 1 1.
We also know, when you put the axes on here just to make it clear.
And here's our z, You know 0 1 1, we know it doesn't intersect the x but it intersects the y and the z at one lattice spacing.
So x doesn't intersect.
It intersects z here, intersects y here's.
We've got this kind of plane going on here.
We've got something like this.
And they just drew this.
It's a little bit of 3-D visualization But they drew that and they got 0 points.
The reason that that is not a correct answer is because that is not what MgO-- it's rock salt structure-- that's not what it looks like.
And the reason they got it wrong is because they didn't understand the concept of the basis.
So I'm going to make a statement right now.
This is a really important statements so I urge you to think about it.
A crystal equals a lattice plus a basis.
Here's the difference.
A lattice is a collection of points.
So you have an fcc lattice, you have a bcc lattice, you have a simple cubic lattice.
Those are points where atoms-- I use atoms plurally, perhaps-- can exist.
OK, a basis I like to think of as a stamp.
A basis represents the most fundamental pattern that you stamp at every lattice point.
So let's draw an fcc lattice.
So I'm going to drawn an fcc lattice on this cube right here.
I'm going to put points.
So I'm going to make them white.
They're just points.
I got all the corners.
They're hard to see.
But they're all the corners.
I got the faces.
Side, back, front and bottom.
You're saying to yourself, that's like the same thing you just drew.
It's really not.
What I drew the first time, I put atoms in.
That's a crystal I drew.
I just a lattice right now.
This isn't a crystal, this is a lattice.
So to actually answer the question, we want to figure out, we want to draw the 2-D projection of a crystal.
So we've got to draw the crystal eventually.
Now we're dealing with MgO.
We're told MgO is face-centered cubic.
Which means that our basis must have 1 Mg and 1 O atom.
So I'm going to do that right now.
I'm going to draw what I'm going to call to be my stamp.
You can use whatever method you think is best.
But I call this my stamp.
I'm going to draw it like this.
So here's an oxygen atom.
Here's a Magnesium atom.
And this little white dot is the point that's going to get stamped.
And here's my stamp.
And the idea is I'm going to take the stamp, this basis-- that's what it is, it's a basis-- and I'm going to stamp it at every lattice point.
OK so note I have 8 corner lattice points and I have 6 face lattice points so I'm going to be stamping 14 times.
So I'll do the first couple very slowly.
I'll take my stamp and I'll stamp here.
Here is my O.
Here's my Mg.
I'll stamp over at this corner.
Here's my O, here's my Mg.
I'll stamp in the face.
Here's the face, the front face one.
Here's my O, here's my Mg.
So you can see what's happening.
Let me just circle it for you.
I'll circle one of them.
Here's my stamp.
I'm going to go ahead, I'm going to finish drawing this in.
It's going to get a little messy for a board.
But I hope that you can go home and do it on some graph paper as well.
All four corners.
The face is here.
I have a tendency to leave off faces sometimes.
Let me make sure I've got them all.
I already did that one.
Yep that's right.
Is that right?
No.
OK, now let me finish with the magnesiums.
The magnesiums-- it's going to be hard to see-- exist at the center of every edge.
So it's a little tricky but let me explain what we're looking at.
What you should be looking at.
We have what looks to be oxygen in the face center cubic arrangement.
And we have magnesium at all the edges.
So you're thinking, this is weird this isn't face center cubic.
Look at the magnesium, this doesn't look face center cubic.
Well the thing I would challenge you to do at home is actually draw this unit cell out 4 more times.
And then you will actually be able to find face center cubic magnesium lattic cell in that larger unit cell.
So this is actually the answer.
This is what a rock salt structure looks like.
This is MgO.
You have oxygen, looks like your traditional face center cubic.
And you've got these magnesium stuck on all the edges.
So now we're just going to take our projection.
Let me use white chalk for that.
Or rather we're going to take our plane.
We're looking at the 0 1 1 plane.
Here's our axes again.
Remember it's always right-handed.
We're going to do 0 1 1.
Not intersecting the x and were going to intersect z and y.
So we're going to come down across the face.
We've got this.
This is pretty terrible looking.
But this is one of the tricks of the problem.
You have to be able to visualize these in 3-D space.
So let's slowly transcribe what this plane here looks like projected out on the board.
We know that at our corners and also at the faces of this cube, we've got oxygen.
So here's a corner.
These are our corners.
At the faces, two faces, we've got oxygen.
What else do we have?
We know we must be hitting some magnesium.
Well we know that at all the edges-- here's an edge, here's an edge-- we've got some magnesium.
What about the center?
Well the center we do too because we know that when we made this stamp, the bottom face with this oxygen, we also stamped halfway up a magnesium.
So there's actually a magnesium right here.
It's there.
It's right there.
Believe me.
There's our magnesium.
And this is what we've got.
This is the answer to the problem.
Notice how I've giving my oxygens larger than my magnesiums and I've got it looking like this.
In fact let me let me emphasize this even more.
The fact that you could actually have drawn this differently.
You could have drawn it like this.
And it would have been exactly the same thing.
You would have gotten full credit.
You could have drawn this.
This is the thing that I would recommend you try at home to prove that this is true.
These two are equivalent It's the same thing.
It all depends on what you choose as your stamp point.
So I could have chosen to put my red at every point, my magnesium.
So this is the answer to part c. This part, most people did not get correct.
Less than half the class got this correct.
So think about a little bit.
Identify crystal as lattice plus basis.
And then think about the full scope of the problem.
What do we go over?
when went over Miller Indices, went over designation and crystallographic notation, so plane and direction notation.
We went over the definition of planar density.
Think about what linear density means, think about what three-dimensional density means.
Don't be afraid to not use mass.
And the last thing we just covered was the crystal basis.
So review those and hopefully you did well.
Thanks.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. We're still on Exam 2 of the fall 2009 class.
We're going to be doing problem number 2 now.
This is the x-ray problem, as I like to call it.
It's kind of exciting.
Let's talk about what we need to know before we actually reasonably try to attempt this problem.
So I would review these things again.
We want to know emission line nomenclature, how to name emission lines.
We want to know Moseley's Law.
We want to know Bragg's Law, and I would also emphasize, I'll get onto it later, you want to know how to derive Bragg's Law as well.
We want to know, this is pronounced Bremsstrahlung, it's a German word which means breaking radiation.
So Bremsstrahlung radiation.
And we also want to know some reflection rules, and I'll sort of elucidate that a bit more.
But let me start you off in that direction.
So getting on to part A, we are told that somebody has been horsing around with our x-ray machine.
We've all had this problem before.
And they've changed the target.
So let me first tell you what a target is, and how the x-ray machine looks.
Then we'll be able to understand what exactly it is we're trying to figure out.
An x-ray machine, basically the x-ray source involves electrons being accelerated into a target material, a metal.
And then that gives off a whole bunch of electromagnetic radiation, so photons.
Some of those photons we'll talk a little about later, in the Bremsstrahlung spectrum, are very characteristic of the material.
So I'm going to call, you know, we have long wavelength photons, short wavelength photons.
Some of the photons have a very characteristic wavelength or frequency that corresponds to a particular electron transition.
We're going to talk more about this in a little bit, but I'm going to call this the K-alpha photon, OK?
So that just happens to be in metals, x-ray.
We know that for particular materials, we have particular K-alpha wavelengths.
And what's basically happened in this problem is that, you know, maybe we knew what this material was before, and then all of a sudden we weren't around, somebody switched it on us, and now we have to figure out what it is.
Don't you hate when that happens?
So the best way to approach this problem is to understand Moseley's Law, and then have it on your equation sheet.
We allow all of our students to have an equation sheet, and also a periodic table.
So don't panic if you can't memorize this equation.
Our students are expected to, not to memorize it, but to understand it.
So this is Moseley's Law.
And Moseley's Law basically tells us, let me go through these variables.
We have the wave number.
We have a Rydberg constant.
This is a bunch of physical constants, sort of agglomerated into one big one, to save us some time.
We have the final energy level, where the electron ends, and we have the initial energy level where it begins.
And then we have z, which designates our element.
So it's the number of protons we have in the nucleus, for example.
And then we have this sigma.
OK?
So, you know, in order to solve this problem, we want to know what element.
So obviously, we're looking for z.
Now the question is, do we know everything else?
So yeah, we definitely do.
Let's just go through each individual thing that we know.
So we know Rydberg constant.
That's just, you know, have that on your equation sheet.
It's a bunch of constants agglomerated together.
We know that, for K-alpha we're told, this is K-alpha radiation.
Let me draw a cartoon for you here.
OK?
This is our nucleus, OK?
This is n equals 1, this is n equals 2.
I'm going to go kind off the board.
This is 3.
OK?
And K-alpha corresponds to an electron dropping down from n2 to n1 to here, and then giving off a photon.
So I'm going to draw a photon as a squiggly line like this.
OK?
So this is going to be h nu.
This will give you our K-alpha radiation.
So we know all of those things.
We know our wave number here.
So we know pretty much everything.
But what is sigma?
Well, sigma is the correction factor that Moseley found.
Now, if you're looking at this and you're thinking, wait, this looks a lot like the Rydberg equation, it's because it's sort of is.
The only difference is that the Rydberg equation is only for hydrogenic-type atoms.
OK?
So we're talking about hydrogen, we're talking about helium plus 1, lithium plus 2.
We can't assume that our target is hydrogenic, so we can't use the Rydberg equation.
So we're using Moseley's equation.
And what Moseley found, just to show you where the sigma sort of comes from, and this is actually reviewed in, I think, lecture 17 of the online postings.
What Moseley found was that if you plot your wave number versus your z, you've got points like this.
You've got some sort of line.
OK?
So this is what Moseley found.
And what Moseley basically did, was he fit an equation to it.
And the equation looks a lot like the Rydberg equation.
And because these don't intersect the origin, we've got this fitting factor right here.
We know that for a K-alpha transition, we're looking at a sigma equals 1.
OK?
So we're good.
We know our sigma.
We know our n's.
We know n final is going to be 1.
We know n initial is going to be 2.
We know the Rydberg constant, and we know our wave number, because we're given our wavelength here.
So we can go ahead and actually just calculate z.
And let's do that right now.
So let me rewrite this equation so that it makes more sense.
Rydberg constant.
1 over.
our n final.
That's 1 squared minus 1 over, our initial point is 2 squared.
We're looking for our z, and we're going to subtract off 1, because we're dealing with a K-alpha transition.
So you're expected to either remember that, or have it on your question sheet.
There's no way to derive that unless you've got all this data, which you won't have.
So we've got this squared.
We know everything, so we can basically reduce this.
This is why on the answer sheet, you see something like this.
You kind of skip all these steps, and we went right to this.
3/4 z minus 1 squared.
And now you can just solve for z.
No problem.
OK?
So that's 4 over 3 lambda Rydberg constant to the half plus 1.
And that's going to give you about 23.
You might get 22.9.
You might get 23.1.
We're talking about 23.
That's going to be the vanadium.
That's the answer to the first part of this problem.
OK. We're going to take a second and clean up, and come right back and finish the problem.
We're going to start part B now of problem 2 on the second exam.
So we basically have used the Moseley equation to find out an element.
And now we're going to switch the problem up a little bit more, and we're going to ask you to find the smallest defraction angle for a particular situation.
So what's the situation?
Let's go back to our x-ray machine.
The way you actually do x-ray defraction, you know, it's acronymized XRD.
The way you do it, is you have to generate x-rays.
So we accelerate an electron into some material.
That material gives off a whole bunch of photons.
You then filter all the photons, so you only pick up a certain wavelength.
And we're going to use the K-alpha wavelength.
And then you get these K-alpha radiation, which then probes your sample.
So here we go.
This is what we're looking at for this part.
We've got K-alpha radiation coming in, and it's hitting our structure, which has a crystal, hopefully.
You're going to do XRD.
And then you're getting off some defraction, anything that's reflecting off of the crystal in some way, like this.
So the thing you need to know to do part B, is you need to know Bragg's Law, Bragg's equation.
And that's number 3 on the win list.
So I drew this picture here just to define the variables, but one thing I would stress for you at home, is to please, go home and, you know, search online, Google image search even.
Look for a picture like this.
Search Bragg's Law and look for a picture like this.
And it's actually from this.
It's very easy, just using geometry, to derive Bragg's Law.
Bragg's Law is just the geometric interpretation of this picture.
OK?
So Bragg's Law, in its traditional format, is n lambda equals 2d sine theta.
And I've defined, I drew the picture to show the variables here.
We have n, which is-- we'll talk about n in a second.
We have lambda, which is our incoming radiation.
We have d, which is the spacing between planes.
Whatever plane orientation you're looking at.
2d is the spacing, is there because you need to come in, and then come out again.
So you're doing twice the distance between the planes.
Theta is the angle at which you're incident on the material.
And n is going to be your defraction index.
We're not going to worry about that in this class.
We're going to assume it's one, for this class and for this exam.
So the question is asking us to find the smallest possible angle of defraction.
What that means, is that we're getting all this radiation, it's hitting our material, we're moving our material around.
The question is, what's the smallest angle at which we're going to see a peak on our XRD graph?
I rearranged the equation here, so theta equals the inverse sign of lambda over 2d.
And we're looking for as small a theta as possible.
This question is really just mathematical manipulation.
I need to know a couple important details that hopefully you've read about in advance.
So we want to find the smallest theta possible.
That's the objective for this problem.
I've drawn the arc sine function here.
To minimize theta, we want to have the argument, which I call x here, we want the argument to be as small as possible.
The smaller it is, the closer to 0 you are for y, which is theta, in our case.
So we want to minimize the term in parentheses.
And the way you do that, small theta means small lambda over 2d, which means you want to have a large d. And the reason I'm not using lambda, the wavelength, as the knob, is because lambda is set.
Lambda is predetermined by the target material you've chosen.
So for our problem, we have titanium as our target material.
Which is, we can't change the lambda K-alpha.
It's set by that material.
So we have lambda K-alpha-- we have k-Alpha coming off, which is characteristic of titanium.
So we can't use lambda as a knob.
The only thing we can mess around with here, to figure out this problem, is we have to look at d. So we want to have a large d to minimize this, and to minimize your theta.
To get a large d, we need to know the equation for planar spacing.
OK?
This is your equation for planar spacing.
And as we sort of talked about in problem 1, in the prior video, d planar spacing is the distance between one plane of a particular orientation, and another plane of the same exact orientation.
OK?
So we're looking at maybe a 011 plane in this unit cell, and a 011 plane in this unit cell.
What's the distance between the two?
That's what this d is.
OK?
So we want to have a large d. And let's look at what we have in the equation for d. We have these hkl.
These are the coefficients the Miller indices of the plane.
And we have a, which is the lattice constant of the material.
We're looking at-- what are we looking at here?
We're looking at tantalum.
We can't change the lattice constant for constant temperature and pressure.
So this is nonnegotiable.
We can't change what a is.
The only things that we can toggle, the only switch we can toggle now, is h, k, and l. So the only thing we can do, is look at different plane orientations in the crystal.
So this is the logic I used to go through the problem.
So you want to have as small an h, k, and l as possible, because the smaller these are, the larger d is, et cetera.
So the final piece of information where we have to be clever is knowing how to get the smallest h, k, and l. We're told that we're looking at tantalum.
And the students in this class have access to a periodic table, which has a vast quantity of information about all the elements in it.
One of the things it has is the crystal structure of pure elements.
So for tantalum, we're looking at a bcc structure.
So we're looking at bcc.
bcc, from your reflection rules, which is the number 5 on the things we need to know to do this problem-- for bcc, you have to have your h plus your k plus your l-- let me write it over here-- h plus k plus l-- must be even.
That's a rule, OK?
Proving that is a little bit beyond the scope of this class, but that's a rule that we went over in class, and it's in the lectures as well.
So we have a couple things now.
We want to minimize h, k, and l, and they have to be even.
So basically, what that means is, we have to go through a couple permutations.
So let's think about it.
0 0 0.
That's even, but that doesn't really correspond to a plane.
That doesn't make, you know, that's not going to help us.
1 0 0, or 0 1 0.
So let me write some of those down.
Let's look at, you know, 1 0 0.
Well, that's not even, so we can't look at that.
So what's the next smallest thing we could go?
How about 1 1 0?
Those are the smallest coefficients we can have for a plane which has the rule that the h plus k plus l must be even.
So this could be 1 1 0, this could be 1 0 1, this could be 0 1 1.
But what we're basically talking about, if you remember from the first problem, is the family of 1 1 0 planes.
So I plugged in just 1 1 0 for h, k, and l. You get a d of 3.31 over the square root of 2.
And then with that d, you know that you've just maximized the size of your d, which means we've minimize the size of lambda over 2d, and that means we've minimized our theta.
And now for theta, we can easily calculate that we're looking at an angle, if we plug in for d, which is 3.31 over the square root of 2, here.
And if we plug in for lambda, which is given to us as 2.75 angstroms, we get an answer of 36 degrees.
Put that right in the middle.
And I put in the middle to emphasize that this is sort of the logic you have to go through, sort of a, you know, circular logic, to get to that answer.
So that's the answer to part B.
And remember, this is pretty much math.
Just manipulation until the very end, where we discussed the reflection rules for bcc.
I want to go on to the last part now, and I'm just going to erase this.
We only need a little bit of room.
This last question was actually my favorite question in the entire class.
And actually, we only had 3 students get it right, out of 480.
So this was a great question, and if you get it right, then you're doing really well in the class.
So let's look at C. It's really a chatty question.
We're not looking for a number.
The question basically asks us, we want you to talk about, to sketch the emission spectrum of an x-ray target that's bombarded by photons instead of electrons.
So let's go back over to what we said in the beginning.
When we generate x-rays, generally the way we do it, is we accelerate an electron into a target material.
In part B, the target material was titanium, but it could be anything.
A very common target material is copper.
copper K-alpha is a very standard target material, or wavelength to use.
You accelerate your electron, and you generate the spectrum of electromagnetic radiation.
OK?
So this is our spectrum.
We have large wavelength, we have small wavelength, we have some wavelengths in the middle.
And generally, what we would see, coming back over here, I'm going to draw it like this.
This is what you'd normally see, if you're using electrons.
Professor Sadoway refers to this as the whale-shaped curve, probably because we're in New England.
But this is what you would normally see.
Now, this y-axis is the intensity.
That's basically the number of photons you count at some specific wavelength.
And you see these spikes.
And these spikes correspond to K-alpha.
This is K-beta.
We'll talk about what they are in a second.
L-alpha and L-beta.
And et cetera.
You'd have M-alpha, N-beta.
You go all the way down.
But they have very low intensity.
So this is what we would normally see if we use electrons to hit our sample.
But we're not using electrons.
And this is actually the answer we got for probably 90% of the solutions from students.
They just drew the Bremsstrahlung radiation, Bremsstrahlung, breaking radiation.
And they walked away, and thought they had full credit.
But in reality, this is not the correct answer, because you think about what's actually causing this whale shape.
Once you understand where the whale shape comes from, then you understand what happens if we're using photons instead of electrons.
So let's actually understand where these characteristics come from.
There's two things to pick up from this plot.
We have a whale shape.
We've also got these spikes.
So first, let's talk about where the whale shape comes from.
Let me just do it with blue.
We'll talk about the spikes in a second, because they're actually a different phenomenon.
The whale shape comes from-- let me draw it for you.
Let's zoom in.
Let's zoom in on our target here.
So here's our target material.
We have an electron hitting it.
It's just a metal.
And we've got photons coming up.
Let's zoom in really close to the surface of that target material.
Let's go back over here.
So we've got-- looks something like this-- we've got an electron, which I'll draw in yellow, coming in.
So here's our electron.
It's about to hit our material.
Notice I've drawn it with some crystal structure, because that's what we're going to talk about, really zoomed in, the atomic level.
Now this electron, we've talked about this in class.
What can happen is that this electron could come in-- here's its path, normally, if it wasn't going to get deflected-- it can come in, and it can be deflected.
OK?
So it can get deflected off-- let me draw the following path in blue.
So it can go off like this, like this.
It could actually go straight through, without being deflected at all.
It could actually be reflected back, like this.
And these different reflections correspond to the electron being accelerated.
It's getting accelerated.
If we define our system like this, so here's our x and our y, the electron's getting accelerated in the y-direction.
This is just a simple location, but what happens, is when you accelerate a charge, you generate radiation.
You generate electromagnetic radiation.
Accelerating charge.
So, you know, here in the first path, where it comes through, and it gets deflected only by a little bit, you generate low energy radiation, low energy photons.
So what we're talking about here is a very large wavelength photons coming out.
So we have an electron.
It gets deflected.
It's accelerated.
And from that point, we're also generating a photon.
Same thing for all these other paths, but for the larger angle deflection.
So this one here.
And as you actually start deflecting back, you're generating very high energy electromagnetic radiation.
And this is all because the electron is basically, you know, is getting accelerated.
You can actually have, you know, the maximum energy photon you can give off here is a photon that corresponds to this electron coming in and getting stopped completely by the atom.
Think electrostatic repulsion.
So it comes in, and it gets stopped dead in it's tracks.
So you have some energy, it was moving some kinetic energy, and now has 0 kinetic energy.
The energy of your photon that gets given off is basically the difference in the two.
And that's what we call here, on this plot, the short wavelength limit.
We're going to write SWL.
Because this is lambda, which means that energy moves in the other direction.
Lambda goes up, energy goes down.
Lambda goes down, energy goes up.
So this is our short wavelength limit, which means that is the maximum, the highest energy we can generate from this setup here.
So all I'm saying is that you can get deflections in any angle across this way, you can get deflections in angle and generate any number of wavelengths of photons with the minimum wavelength being this one, that's generated from this situation.
So I'll just make it very, very small for you.
So you generate all these wavelengths, you create all this energy of photons, and that's what basically corresponds to the whale-shaped curve.
You're more likely to get deflections that correspond to something around here, because you have higher intensity, and over here, you're less likely to see those happening.
So that's our whale shape.
And now, let's return to the problem.
Let's ask, when we use photons, what's the situation?
Well, photon means we're no longer looking at an electron.
Our photon is coming in.
And the thing about photons, is that they're not going to get deflected like that.
They don't have a charge.
There's no electrostatic propulsion, for example.
So a photon will either only be absorbed and re-emitted, or it will go through the material.
So what that means is that there's no process now that will accelerate a charge to create Bremsstrahlung radiation.
And if there's no process to accelerate a charge, then there's no way to get Bremsstrahlung radiation.
So the first thing you need to do to get most the points on this problem is to say, look!
There's no Bremsstrahlung radiation anymore.
There's no charge being accelerated.
So that's sort of the first answer.
The second thing to realize is that these spikes still exist.
Because the reason for these spikes is a different mechanism than it is for the Bremsstrahlung.
These characteristic peaks correspond to the movement of an electron to a different energy level, and then a dropping back down.
So it's very characteristic of the material itself.
So a photon can still come in.
It could still liberate or move an electron.
It could just knock an electron out of its shell.
Perhaps n equals 1.
And then the n equals 2 electron will drop down and fill in the n equals 1 shell corresponding to K-alpha radiation.
If you had n equals 3 electron dropping down to n equals 1, you'd have your K-beta radiation.
And likewise from, you know, 3 to 2, 4 to 2, L-alpha, L-beta.
So that's the key take-home message here.
The message is that your Bremsstrahlung, the thing that you see in the notes in the class all the time, the whale shape with the spikes, that's two specific mechanisms.
The whale, the Bremsstrahlung, corresponds to the breaking of an electron.
These peaks correspond to the ejection of an electron, and an electron's moving around energy levels within the atom.
OK?
So our learning objectives for this problem, the things that we've taken home from the problem overall, is that we know, for example, Rydberg equation is only for hydrogenic-type atoms, and we know how to use the Moseley's equation now.
That was part 1.
Part 2, we know Bragg's Law, and how to use it, and what all the variables mean.
That's something you should learn and take home.
And I really highly recommend that you derive Bragg's Law from the geometric interpretation.
And part C, we really wanted to probe and understand who conceptually understood what was happening.
And part C tells us, the thing that we learned from it is that this Bremsstrahlung radiation with the peaks that we saw in class, that's two separate mechanisms occurring.
So that's problem number 2, and I hope you did well.
The following content is provided under a Creative Commons License.
Your support will help MIT Open Courseware continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Open Courseware at ocw.mit.edu.
OK we're going to do problem 3 now on exam 2 from the fall 2009 class.
Let's talk a little bit about what the problem concept is.
This is doping.
We're going to talk all about doping semiconductors.
As we say chemistry, where doping is legal.
So we're going to look at the things we should probably know before attempting the problem.
You want to review your doping principles.
How does doping works.
What are the mechanisms, your conduction mechanisms?
So why does the material conduct?
What's actually happening?
Your donor and acceptor levels, understanding which is which.
And your p versus your n type doping.
What those mean.
So look at those real quick and do the problem.
Let's go on.
I wrote down the information that we're given in the problem.
The problem part a asks us to find specifically how much gallium is needed to reach a certain carrier concentration in our germanium.
So we're doping gallium into germanium.
We're given the fact that the germanium band gap is 0.7 electron volts.
We know that we want to achieve a carrier concentration of 3.091 times 10 to the 17 carriers per centimeter cubed.
And we're working with 1 kilogram of germanium.
So we need to achieve this density or this carrier concentration in one kilogram of germanium.
I've drawn schematically-- now remember the germanium crystals are in 3-D so the bonds don't actually look 90 degrees like this-- but I've drawn just for visual a germanium crystal.
So this is what a pure germanium crystal would look like.
I'm going to draw the band structure for the pure germanium crystal now as well.
When you see a problem that gives you your band gap and it tells you this is about doping, the first thing you want to do is just get some points.
Put some stuff on the paper that you know to be true.
So let's draw the band structure.
I'm going to abbreviate conduction band with cb and valence band with vb.
So here's my conduction band, here's my valence band.
We're told that this is here.
We know our band gap energy is 0.7.
OK and I'm going to use this blue to show where electrons are.
This just shows that there's electrons in the valence band going all the way up to the top of the valence band.
That's how the valence band level is defined.
So this easy points.
You've got this on the paper, we're good to go.
Now what we're talking about is putting gallium into germanium.
So let's do that.
Let's take out this germanium.
And let's put a gallium in its place.
We'll put it in red.
OK so this isn't exactly correct yet.
Germanium makes 4 bonds.
If you look at the periodic table you'll see that it has 4 valence electrons to make bonds.
But gallium does not.
Gallium only has 3.
So what that means is that one of these bonds can't exist like this anymore.
So the way I like to draw it, is we still have an electron from this germanium but we're missing one for this gallium.
And this is what a hole basically is, schematically, in doping.
So we're going to write this as hole plus.
And the reason it has a positive charge associated with it is because in our neutral crystal with no charges you having an electron there.
And you've just removed an electron and now it's positively charged.
You have a positive charge in this region of the crystal, which corresponds to a missing electron.
So we just created a hole.
We have a hole here and then it's going to result in what we call-- I'll use red again-- an acceptor level.
And remember this band diagram is an analogy for, these are the energy levels where your electrons can exist.
So we have these energy levels and the valence band and we've got energy levels up in the conduction band.
What's just happened is that this dopant has created an energy level slightly above the valence band.
If you have a donor level from a different type of doping-- which we'll talk about in a second-- you'll create a level right below the conduction bands.
So we've got some points.
We understand the system now.
It's just going well.
So let's try to do a.
So a, as I said before is asking us to figure out the actual amount.
We're looking for grams.
This is what we're looking for.
Grams of gallium.
That we need to put into a kilogram of germanium to create a carrier concentration of 3.091 times 10 to the 17 carriers per centimeter cubed.
Not too bad of a problem.
This is basically just stoichiometry.
Dimensional analysis.
Here's how I did it and many people did it this way.
I start off.
times 10 to the 17.
And I always keep my units.
That's really important.
So we're going to write this as carriers per centimeter cubed.
Now we're also told in the problem, we have the additional information that the temperature is high enough that all of the sites are ionized.
What that means is that-- let's take a look at this band diagram-- we have electrons existing in the valence bands up to that level.
We've doped and now the temperature's high enough that these electrons can be excited, one of them in this case can be excited to this level here, the acceptor level, creating, of course, a hole in the valence band.
That's what it means that it can be fully ionized.
So basically every gallium atom that we put into our material creates a carrier.
And why do I say that?
Because conduction is either, the movement of electrons in the conduction bands, which we don't have.
Or it's the movement of holes in the valence band, which we now have.
So we're talking about conduction.
And so for every gallium atom we put in, we're creating an acceptor level, which takes an electron up, which creates some conduction in the valence band.
Now if we add lots and lots of gallium atoms-- there's a lot more gallium atoms in here-- we're going to actually see many more of these levels showing up.
And you're going to get what almost looks like very thin bands, an acceptor band or acceptor level there.
So we start off like this.
And I went into that's digression because now I'm just going to say that for every atom we put in we get 1 carrier.
And then we're going to say-- I want to be sure I get this right-- we have 6.02 times 10 to the 23 atoms per mole.
That's right.
And then we have 1 mole.
And notice how I'm being very careful about the dimensional analysis here.
Because if you put something in the wrong top or bottom you're off by 46 powers.
So don't make that mistake.
We saw that a lot on the exam.
So we're good here.
Let's do our dimensional analysis.
We start off, we're looking for this particular concentration carriers per centimeter cubed.
We have 1 atom creates 1 carrier.
1 mole has this many atoms.
And one mole of-- we're talking about in this case-- gallium.
I'll put Gallium here to be clear.
1 mole of gallium has this much mass.
And you're left with something like this.
You're left with 3.58 times 10 to the negative 5 grams per centimeter.
Grams gallium per centimeter cubed.
So we're looking good.
We're pretty close to the answer but we don't actually have the answer yet.
Remember we're looking for total grams of gallium.
Not how many grams per centimeter cubed.
That's pretty easy because we're told the we have 1 kilogram of germanium and we know the density of germanium from our periodic table.
So looking at our periodic table we can look up germanium's density and we can then back out how many centimeters cubed of germanium we have.
We'll do that right here.
So 1 kilogram of germanium is 1,000 grams, times-- we have the density, which is going to be times 1 over the density, rather-- grams per centimeter cubed.
And that's going to give us 187 centimeters cubed.
And now we're good.
Because we know here we have 3.58 times 10 to the negative 5 grams of gallium per centimeter cubed of germanium.
We have this many centimeters cubed of germanium.
Multiply them together and you get the answer, which is 6.69 times 10 to the negative 3 grams of gallium.
So that's part a.
Just dimensional analysis and stoichiometry.
But I stress, be careful about the way these things are.
Even I pause to make sure they're correct because if you get this wrong, the whole problem's wrong.
You're off orders of 46.
You're off by orders of 1,000 here, so just be careful.
Part b.
We've actually already answered when we drew it up, we threw down the things we knew.
We knew that we were creating a hole.
And a hole corresponds to p type doping.
How do I remember p and n?
It's easy.
p means positive.
We've created a hole, which has a positive charge.
n, I think of as negative.
So if you had put something else in there like arsenic, arsenic would've had an extra electron compared to germanium, which means you have a negative charge.
So n negative.
So we have p type doping.
We're done with part b.
Easy.
Part c. I'm just going to erase some of this stuff here and we're going to draw a couple more of these band diagrams.
So basically what we're asked to do now is to think about, what does this band diagram look like at different temperatures?
Because it actually looks slightly different.
I've drawn it here schematically.
But let's actually think about what it looks like in real life.
So let's do 2 more of these.
And we'll use that one as well for part c. OK so I'm drawing my conduction bands.
These are my conduction bands.
These are my valence bands.
I have the same band gap for all of them.
So you know band gap here, here's the band gap.
It's always worth putting down.
If you know it's true, put it down.
And we know that in both cases-- let me just get rid of this thing here; we're going to start from the beginning; put our electron back-- we've created all these acceptor levels.
Because we've doped with more than one gallium atom.
We have many, many, 10 to the 17 gallium atoms.
Which sounds like a lot, but in a mole we have 10 to the 23 spots in a mole of material.
So it's 6 orders of magnitude less.
So we have these systems and we're asked to do them at-- I'm going to keep going back and forth-- 300 k, 4.2 k and 1200 k. That's Kelvin.
Let's go with the extremes first.
And we'll do the middle one, 300 k, once we have an idea of what's happening.
So basically, in the very beginning when we to dope in a material, time t equals 0 and no electrons have had a chance to move around.
You have all your electrons in the valence band, you have these open holes in your acceptor levels.
And the reason that electrons would move between levels is if they have some energy associated with them.
Now why would something have energy in a crystal?
Well that's because of thermal vibrations.
It's a thermal energy they have.
So it's completely based on the temperature at which the crystal exists.
So at a very low temperature-- let me write that up there for you, this is our basic equation-- at a very low temperature we have a very low thermal energy.
k
this is the Boltzman's Constant.
At a very high temperature, we have a very high thermal energy, which means that these electrons have the ability to move between levels more easily.
So here's our very low temperature.
At a very low temperature we can actually calculate the energy.
We'll find that the energy here is about-- just so you have a scale-- it's about 0.00036 electron volts.
Now you say to yourself, wow that is a lot less than 0.7 electron volts, which is the energy of this gap.
So there's no way these electrons here can even make it up to this acceptor level.
That's not exactly true.
This is best thought of as an average thermal energy that an electron will have.
It's based off of a distribution, if you will.
So most electrons will have around this energy.
But some will have a little bit more and some will have a little bit less.
And very, very, very few at the tail of that distribution will have enough energy to actually make it up to one of these levels.
So the way to do this problem, to be actually fully correct, is to say that and explain it.
I actually, when I did this problem on the answer key, I have an electron, one electron going up into one of these spots and the rest are all empty.
I denote that with a little bit of blue here.
But everything else is empty.
We've got a hole here.
So there is a finite, very small, amount of conduction.
But it's extremely small.
And it's extremely unlikely to have electrons moving around but probabilistically it will happen.
That's 4.2 Kelvin.
Very, very low.
Let's go to very, very high.
Starting at the beginning again, we know all of our electrons are in the valence bands.
At 1,200 we have an energy of about 0.1 eV.
Still less than 0.7.
We have a significantly larger proportion electrons in that distribution, which are at 0.7.
So we actually do have electrons that will move up into the spots.
And even many that'll move up here as well.
Because they can overcome this 0.7 eV.
So we've got some electrons, they're going to move here, they're going to fill up most of these.
Because this delta here, the difference between our acceptor level and our valence band, is actually quite small.
It's extremely small.
That's another reason why we have an electron moving up at 4.2.
I should have mentioned that.
This is very small.
This gets pretty much completely filled.
And I'll denote that with a blue line here.
And we have some electrons, even going up into here.
What that means, that that has to correspond to electrons being depleted out of the valence band.
This is the way you would draw this picture.
So we lose these electrons from the valence band.
Some jump up to the acceptor levels and the rest will go up to the conduction band.
So this is 1,200.
We have more than enough energy to reach this acceptor level and we have probabilistically some electrons will make it up to the conduction bands.
That's the best way to think about it.
Now let's go back to our middle temperature.
I think the best way to do this one is to just go somewhere in the middle.
That's the safest play, right?
So we're at a temperature which corresponds to about 0.025 eV.
Now that is obviously smaller than the gap.
Probabilistically we'll still get some electrons in the gap.
Diminishingly small.
But we'll put one up there.
OK maybe one goes up there.
And it's still much bigger than the delta here between the acceptor level and the valence band top.
So these electrons will still pretty easily make their way into the acceptor level.
Now I'll put a little bit here.
We're going to lose electrons very slightly.
That's how I would have done that problem.
It sort of looks like that on the answer key.
Now I want to emphasize that, what is what's the importance of this?
Well when you dope, you're creating conduction.
Why is that?
Because the more we dope, the more holes we have that can be played around with.
So we have more and more of these acceptor level energy levels.
And what happens is these electrons at finite temperatures-- temperatures above 0 Kelvin-- will move into those levels.
And they will cause conduction in the valence band.
And you know at high temperatures, back over here, we have electrons moving to the conduction band, which is traditionally what we think of when we think of conduction, electrons moving around.
So we have in this case conduction in the conductor band as well as in the valence band.
So that's this problem.
This problem was generally pretty easy; many students got it right.
The learning goals and objectives we had from this problem were, number 1, to understand doping principles.
What it means to n and p type dope.
Holes versus electrons and how you get them.
So if you have germanium or silicon, those are very common examples and you dope things into them.
So gallium or arsenic or aluminum, for example.
We talked about conduction mechanisms-- just like here-- and we also talked about the donor and acceptor levels.
So the donor levels are just slightly below the conduction band and are full of electrons.
And the acceptor levels are just slightly above the valence band and initially holes.
So that's probablem 3 and I hope you had good luck at It.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, I'm Jocelyn, and today we're going to go over to the Exam 2 from Fall 2009, and Problem 4, Parts (a) and (b).
As always, we're going to start by reading the question.
Boron exists in the gas state as the dimer of boron 2.
Explain how the fact that B(2) is paramagnetic, two unpaired electrons, implies that, in this molecule, the pi 2p orbitals must lie at a lower energy than do the sigma 2p.
OK, so what is this question actually asking you to do?
Well, it's asking you why the energy of the pi 2p is less than the energy of the sigma 2p, given that boron 2 is paramagnetic.
So the first thing we need to know is what paramagnetic means, and he actually gives that to you in the problem, so it means that you have two unpaired electrons.
So this is unpaired electrons.
We are asked here then to give a reason why the fact that there are unpaired electrons implies a certain ordering of the sigma and pi 2p orbitals.
So the first thing we want to do, probably, is recognize that these are molecular orbitals, not atomic orbitals.
Therefore, we probably want to start by drawing the molecular orbital diagram for boron.
Now, how do we draw a molecular orbital diagram.
Well, the first thing to do is to start with the atomic orbitals.
So boron has five electrons, as we can find out from our periodic table, and they occupy the 1s, the 2s, and one electron in the 2p.
Again, if this seems unfamiliar to you, you might want to go back to earlier in the course, where we went over electron configuration.
However, when we're looking at molecular orbitals, we actually only care about the valence electrons, because those are what are involved in bonding.
The core electrons are much, much lower in energy, so they aren't really involved in the bonding.
So we're actually only going to look at the electrons in the second energy level.
So we have three valence electrons per boron, and they are in the 2s and 2p, which we remember, there are three p orbitals in each energy level.
So now let's draw those in.
We have 2s and 2p.
Remember to always have a scale here, and we're in an energy space, and this is not to scale, because we're just looking at relative energies, we don't care about the actual difference in energy.
And because we have two borons-- so this is boron one-- I'm going to draw the same thing over here.
Make sure they're at the same energy level because they are the exact same atom, and therefore, the same atomic orbitals.
So this is, again, boron, and we have three electrons for each.
We don't have to fill them in, but I'm going to do so, just to keep track.
And again, we're filling according to the orbital filling rules gone over earlier in the class.
So now that we have our atomic orbitals, we are going to combine them to, like, molecular orbitals.
And we want to remember that s orbitals combine to make a sigma and a sigma bonding and a sigma anti-bonding.
Remember that whenever you combine atomic orbitals, you create-- if you're combining two atomic orbitals, you create one of a higher energy and one of a lower energy, because the net energy stays the same.
Oh sorry, that's crooked.
All right, now we'll move on to the 2p, and because the problem told us that we need to explain why the pi bonding orbitals are lower in energy than the sigma bonding orbitals, we're going to start with that configuration.
The actual order depends on the molecule, and so without any extra information, we wouldn't really know if the sigma was above or below the pi.
So as the question, kind of, hints at: p orbitals make two pi bonds and one sigma bond.
This is due to the orientation of the pi bonds, or the p orbitals in space.
Then, as always, we need to put in the anti-bonding just to make sure we keep track of all of our orbitals.
OK, so we combined one, two, three, four, five, six, seven, eight atomic orbitals, and we made one, two, three, four, five, six, seven, eight molecular orbitals.
We want to make sure we always have the same number of orbitals before and after.
So now we can fill in the electrons into the molecular orbitals: that's these in the middle here.
And we use the same rules as we do when we're filling atomic orbitals.
So we start at the lowest energy, pairing spins, if we only have one orbital in the energy level; but as here, we have degenerate orbitals, so we can actually have unpaired spins.
So we have one, two, three, four, five, six electrons, and we have three from each boron atom, and so, we have the same number of electrons in our molecule.
Now that we have our molecular orbital diagram, we can go back to the actual question.
Some students stopped here and were like, I made the molecular orbital diagram.
I'm done.
But we need to remember that the question is asking us why the configuration we drew makes sense because of the unpaired electrons.
So if we go back to our molecular orbital diagram, we see that because the pi 2p have two degenerate orbitals, we can have unpaired electrons.
If we think about the sigma being below, we would have paired up those electrons, and therefore, gotten 0 unpaired electrons, and it would not be paramagnetic.
So let's just try that out.
Move over here, and we're going to draw just the molecular orbitals; since we already did the full process of making them from the atomic.
So this is the sigma 2s, sigma star 2s.
And now we're just looking at what if the sigma 2p was lower in energy than the pi 2p.
And again, we would fill up our orbitals, and we see that because this lowest occupied molecular orbital is singularly degenerate, we pair up our electrons, and so if this were the case, boron would not be paramagnetic.
However, because it is paramagnetic, we know that the pi 2p is lower in energy than the sigma 2p.
So that's what this question was asking, and if you said all those things, great job, and you answered the question correctly.
Now we're going to move on to Part (b), and it's very related to Part (a), so we're going to use the same diagrams here.
So it asks, is the gas molecule B(2) 2 minus more or less stable than the gas molecule to B(2).
Explain.
So here he's asking about the boron dimer that has a negative 2 charge-- that is, it has two extra electrons.
So we're still talking about boron; and thus the atomic orbitals that we're combining are exactly the same.
I'll write that down.
So we have we still have the 2s and the 2p from each of the boron.
That's not the same energy level, and they still make the same molecular orbitals, but we have two extra electrons.
So when we go to fill up our molecular orbitals, we now have eight instead of six.
So not only is the boron 2 minus no longer paramagnetic, because we don't have unpaired electrons, but we also have a difference in the bond order.
So if you don't remember what bonding order is, it is when-- I'll just put it here-- it's how we determine the strength of the interaction between the two atoms.
And because he's asking is the boron 2 minus more or less stable, we want to figure out the strength of the interaction, and that will tell us which is more stable.
So the bond order is the bonding electrons minus the anti-bonding divided by 2.
Sorry that's a little messy, but I hope you get the idea.
So for boron 2 minus, which we're talking about over here, the bonding order: we have two, four, six bonding electrons, electrons in bonding orbitals.
And we have two in anti-bonding orbitals, divided by 2 and that gives us a bonding order of 2.
So that's equivalent to saying there's a double bond between the boron 2 minus, the two borons in that molecule.
So let's move back to our diagram for the neutral boron dimer, and we have two, three, four bonding, and two anti-bonding, and so that equals 1.
So in the neutral dimer, we have a bonding order of 1, and in the charged 2 minus dimer, we have a bonding order of 2.
From this, we can see that the 2 minus has more bonding interaction, and so, it will be more stable.
So again, you can't just put that it's more stable, we need to have an explanation for why, and this is a way to explain that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, I'm Jocelyn, and today we're going to go over Fall 2009 Exam 2, Problem number 5.
As always, we're going to read the problem first.
Which compound do you expect to have the higher boiling point: HF or ammonia?
Justify your choice with an explanation, using narrative or cartoons or both that makes reference to the operative chemical bonding.
So first, we are trying to figure out which has a greater temperature of vaporization, or boiling point; and we're looking at HF and ammonia.
Now when we're talking about boiling points, we're talking about breaking a certain type of bond, and that type of bond is intermolecular bonds.
We're not going to be breaking the actual molecular bonds bonding hydrogen to fluorine or nitrogen, we're breaking the bonds in between molecules of HF or ammonia, and so what types of bond do we have to look at?
And Professor Sadoway, in class, went over a couple of different types of intermolecular bonds.
And I'm going to write them down in order of increasing strengths.
So the weakest intermolecular bonds we have are Van der Waals.
The next is induced dipole-dipole.
Next we have dipole-dipole, and finally a special type of dipole-dipole is the hydrogen-bond, which is the strongest.
So here we're increasing strength.
Van der Waals is an induced dipole-dipole.
That means that although your atom or molecule does not have a net dipole, the electrons can shift, and at some point will cause a difference in polarity between different sides of the molecule.
Induced dipole-dipole is two different molecules interacting: one that has no net dipole and one that does.
Dipole-dipole: pretty self-explanatory.
You have two molecules that have dipoles, and they are interacting.
And then hydrogen-bond is when you have hydrogen bonded with x, where x is nitrogen, oxygen, and fluorine.
We call that a hydrogen-bond, because these are really, really strong dipole-dipole interactions and cause much different behaviors in molecules, and so we, kind of, separate it out at the subset.
So now that we've reviewed intermolecular bonds, let's go back to the problem.
So we were asked to figure out the difference, or which one has a higher vapor pressure, or temperature of boiling, so that's a relative thing.
And as we now know that we need to look at intermolecular bonds, a good place to start is with the Lewis structure, because that will tell us if we have dipoles or not, and then we can see what the strongest intermolecular bond is that we need to break to boil these molecules.
So starting with HF, we have hydrogen bonded to fluorine.
We know that fluorine has a much higher electronegativity than hydrogen, so we're going to have a net dipole.
Well, we have a polarized bond and a net dipole towards fluorine.
This causes there to be a partial positive charge on the hydrogen and a partial negative charge on the fluorine.
So what is our strongest intermolecular bond that we have for HF?
Going back to our list of bonds, we see that we have a dipole, so we might first think about dipole-dipole.
However, we need to remember about hydrogen bonds, and here in HF, we have hydrogen bonded to fluorine, and so that would be our strongest interaction.
So let's write that down to keep track.
Now for ammonia, we have nitrogen bonded to three hydrogens, and again we have a difference in electronegativity, and so each of these bonds is polar.
And because of the asymmetric geometry, we will have a net dipole, and each of these have a partially -- sorry, the hydrogens have a partially positive charge, right, because they are less electronegative, and the nitrogen has a partial negative charge.
So, which is our highest or strongest intermolecular bond that we have here.
Again, we have a net dipole, and so we have dipole-dipole interactions, but because hydrogen is bonded to one of the special three atoms, we have hydrogen bonding, and that is the strongest.
So, that is what we care about.
So now, we're, kind of, at a roadblock here; both have hydrogen bonding, which is the strongest intermolecular bond.
So how do we decide which one is actually going to be stronger than the other, and therefore, have the higher boiling point.
Well even though they're both hydrogen bonds, they will have difference in strength, and that's going to depend on the partial positive and partial negative charges, because all of these interactions are due to coulombic attraction.
In fluorine, we know that fluorine is the most electronegative element and has a higher electronegativity than nitrogen, so just to keep track of our thought process here: this will cause the partial positive on hydrogen to be larger in the HF than in the ammonia, and so from this we can say that the hydrogen bonding in HF will be stronger than the hydrogen bonding in ammonia, and therefore, the t evaporation of HF will be greater than the t-- the boiling point of ammonia.
So now we're going to move on to Part B.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So now we're going to start Part (b).
First we're going to read the question.
To which does an atom of argon form a stronger bond: another argon atom or an atom of krypton.
Justify your choice with an explanation, using narrative or cartoons or both that makes reference to the operative chemical bonding.
So again, he's not just asking you for one or the other, but that you give an explanation to show that you understand the concepts that are being tested.
Here we're asked: does argon bond stronger with another argon or krypton.
And some students were confused by the usage of the term bond here.
But remember that bond doesn't necessarily mean a covalent or an ionic bond, it can also mean intermolecular bonds, which we have been using in Part (a), so it might seem logical that we would think about intermolecular bonds in Part (b).
Now we need to look at the specific species we are asked to consider here.
Argon and krypton are both Nobel gases, and what do we know about Nobel gases?
They have complete octets; they don't bond very often, and so, especially to each other.
Therefore.
we will probably be looking at intermolecular bonds, as I said before, but also the weakest form, which is Van der Waals.
We're looking at Van der Waals, because each of these are a mono-atomic species.
That is, they just are gases as one atom of argon and one atom of krypton.
And so, Van der Waals is the only type of interaction they can have.
As I said in Part (a), Van der Waals has to do with the movement of electrons in an induced dipole.
And what does that mean?
So, we're going to talk a little bit about polarizability.
And polarizability is basically how easy is it to induce a dipole into the atom or molecule.
And because these are non-polar atoms, obviously, we want to think about what can cause a difference in polarizability between argon and krypton.
So if you think about it, we're going to do a very simplistic model of the atom.
We have a cloud of electrons, and to get intermolecular interactions, we want to have a partial negative and a partial positive induced in the atom.
And this can be due to just spontaneous fluctuations in the electron density or to an external charge or another molecule or something.
But here we have two non-polar atoms, so we want to see which one will inherently be easier to polarize.
And so if we go back to our two species here, we have argon, which has 18 electrons and krypton, which has 36 electrons.
So based on our discussion of polarizability, we can see that krypton is going to have a greater amount of polarizability, because it has a larger cloud of electrons.
Those electrons are bound less strongly.
As we know from our periodic trends, krypton is a larger atom, and so we are going to say that the polarizability of krypton is greater than argon.
As we talked about in Part (a), even though these had the same type of intermolecular bonds, if one of them exhibits a higher difference in partial charges as krypton will, then it results in a stronger bond.
And so the answer here would be that krypton has a stronger interaction because it is more polarizable.
And that because is important.
Remember, when you're answering a question, you don't just want to put down the answer, you want to show that you know why that's the answer, and that you understand the concepts.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
I'm Sal.
Today we're going to be solving problem one of exam three of fall 2009.
Now before you attempt the problem, there's a couple of things that you should know or have background knowledge of these materials before starting.
First thing is understanding the properties of crystal defects and for this problem particularly, Frenkel defects, which we'll talk about.
The formula, which is an Arrhenius relationship, as a function of temperature and energy enthalpy of the fraction of vacancies that are formed in a crystal and the conversion again between one electron volt and to the joule.
This will help you solve the problem.
So the problem reads as follows.
Silver bromide, which is AgBr, has rock salt crystal structure.
So it's an FCC Bravais lattice with the ion pair Ag plus and the Br as the basis-- Br minus the basis.
The dominant defect in AgBr or silver bromide is the Frenkel disorder.
So it's telling you what the dominant defective is.
So the question asks, for part A, so I'll put A here.
Part A for the question asks, does the Frenkel disorder in silver bromide create vacancies of silver plus, vacancies or Br minus or both?
Explain.
Dionic radii are 0.67 Angstroms for silver plus and 1.96 Angstroms for bromine minus.
So that's data that's given to us.
I'm going to go ahead and write that down.
So I know from reading about point defects and crystals that a Frenkel defect is pretty much formed when you have an ion pair of dissimilar sizes.
So if one of the radii-- like, say, your anion is a lot bigger than the radius of your cation, then one should expect that you would have a Frenkel defect to form.
And what a Frankel defect is-- which you should know from reading the material-- is that it's when one of the ions in your crystal leaves and leaves back a vacancy-- so just migrates-- just hops out of its lattice site and leaves back empty spaces, which is called a vacancy.
So we're given two things.
If I draw a little picture-- I can call this my silver and I'll call this my bromine.
And I'm given the fact that the radius of Ag plus is equal to 0.67 Angstroms-- and an Angstrom is just 10 to the minus 10 meters-- very small number.
And I'm also given the radius of bromine, the anion, is 1.96 Angstroms.
Now I would argue that these two have very dissimilar radii.
Obviously, the bromine looks to be about three times bigger than your silver.
So by understanding the definition of a Frenkel defect, I can claim that I would expect the smaller ion of the two to be the one that leaves the vacancy behind, hence forms the Frenkel defect.
So for part A, do I expect it to create silver plus vacancies or Br minus vacancies or both?
I would expect just to be silver plus, given the size of your cation.
So if you were to write on this problem, just expect only Ag plus, which is your silver cation, or the smaller one.
Expect Ag plus smaller ion to create a vacancy and hence, this leads to the Frenkel defect.
So yes, I should expect it to form a defect now.
If I was given a cation that had a similar radius that I-- I wouldn't know if I could argue the fact that this will form a Frenkel defect or not because the conditions are that you have to have the similar radius between your cation and your atom-- and it makes sense because the fact that your silver is small-- it has more freedom to hop around.
So it requires less energy for this smaller atom to then start hopping around your lattice site without penalty or without major penalty compared to your bromine ion.
So therefore, this is the one that should be expected to form that.
OK.
So that's part A.
And part B reads-- calculate the temperature at which the fraction of Frenkel defects in a crystal with silver bromide exceeds one part per billion.
The enthalpy of Frankel defects and formation, delta h sub f, has a value of 1.16 electron volts per defect.
And the entropic pre-factor A has a value of 3.091.
So it's giving you data.
Now the way I would solve this problem is that the first thing I would do is that I would write down what my data is.
So I'll call this data and the first thing is that our fraction of vacancies that are formed in silver bromide is one part per billion-- so 10 to the -9.
That's a fraction so no units.
The second thing that we're given in the problem is that our energy of formation-- our enthalpy energy-- is 1.16 electron volts per defect.
This is the energy penalty for every time one of your silver cations jumps out of sight to create that.
Nothing is free.
Everything requires energy.
We're also given the fact that A, the entropic pre-factor, is 3.091-- with no units-- and given our equation that I showed you on the bullet point.
That you should know-- you can go ahead and see what's missing.
So if I write down my equation-- this is all-- I'll box this off as my data because I'm going to refer to this to solve the problem.
And the problem talks about temperature.
So I know that f sub v is going to equal to the pre-factor times the exponent of negative delta h, of formation over kb t. Now you notice that kb wasn't given to you and kb is both in its constant.
Now a lot of student forget that they have a table of contents in front of them and that value is in there.
So if you don't know it by heart, then you want to make sure that you reference to that table because a lot of information will be given to you, because you're expected to look at the table of contents or your periodic table.
But if I include this as my data-- I'm going to go ahead and write it over here-- that kb-- half the value of 1.38 times 10 to the -23 and this has units of joules for degree Kelvin.
So this is something that you should pay particular attention to because now kb is in joules for Kelvin, but our energy was given in electron volts.
Now this is the number one thing that will take off points.
You'll get points taken off if you don't notice this-- that you need to go ahead and do the conversion from electron volts to joules or joules to electron volts.
Either which way, you're going to get the answer.
But the problem asks, calculate the temperature.
The very first sentence.
So what does that mean?
Well, I'm going to go ahead and look at my equation and I'm going to look at the data that I have and the constants and see if I'm missing anything else because obviously you can't solve an equation that has two unknowns and just one equation.
Solve the equation, you've got to only have one unknown for the equation.
So f sub v-- no.
Check-- no.
That's given.
Is A given?
It's right here.
That's given as well.
What about delta A sub F?
Well, that's the energy-- the energy penalty to create a defect and kb, which we got from our table of contents, and t. So this concludes that the only thing we're not given is temperature because that's what we're asked to solve.
So I need to do some math on here to go ahead and solve for temperature.
And the first thing I want to do is take the natural log of both sides.
So by taking the natural log of both sides-- so natural log of f sub v-- this equals to natural log of A plus the natural log of the exponent part and we know from math that the natural log of an exponent cancels each other out because they're inverses.
So the natural log of exponent, of negative delta h sub f, over kb t-- this cancels that and we just get the natural log of A plus negative because you have a negative up here-- delta h of formation divided by kb t. So the only thing we don't know here is temperature.
So if I can rearrange this equation and solve for temperature, I can get an answer.
So if I do that-- what's the best way of doing that?
Well.
I can add-- I can move this to the other side and move this to the other side and that gives me delta h sub f divided by kb t equals natural log of A minus natural log of your vacancy fraction and I can then multiply both sides by t so it cancels this one and it arrives over here and then divide both sides by ln of this.
So I'll go ahead and do that.
So I'll multiply this by t. Multiply that by t. So that cancel that and I end up having delta hf over kb t-- the t got canceled-- this equals to the natural log of A minus the natural log of your vacancy fraction times t. So now if I divide both sides by this factor, I solve for temperature.
So this gives me an isolation that t ends up being delta hf over kb times the natural log of A minus the natural log of the vacancy fraction.
Now all we have to do is plug in the numbers, but again, if you look at the units of Boltzmann's constant, which are joules per Kelvin and the value that we're given up here is electron volts for defect, we want to go ahead and convert the electron volt to the joule because it'll just be easier to do the math that way.
So I can go ahead and write it out and I do that by multiplying the top by that conversion factor and if I do the math, t ends up being-- we have 1.16 of this-- has units of electron volts for defect and then I want to get joules because that's what Boltzmann's constant has.
So in order to get joules, if I multiply this by the conversion factor which is 1.6 times 10 to the -19-- this has units of joules per electron volt.
Now the electron volts cancel and I get-- the numerator has units of joules per defect.
That's good because that's going to cancel with the units in Boltzmann's constant.
And then I put in 1.38 times 10 to the -23-- this is joules per Kelvin-- and this factor is multiplied by the natural log of 3.091 minus the natural log of 10 to the -9.
So unitless-- so if you do the math out, you end up getting a value that at the end of the day, your temperature-- this t right here-- will be equal to-- let's go ahead and write it over here.
So now that we go ahead and do the math-- that way it's nice and clean.
We know that the t, the temperature that we got from plugging all that math-- ends up being just 615 Kelvin.
And where did the Kelvin come from?
Well, it came from your joules per Kelvin, from your Boltzmann's constant, because that's what the dimensional analysis does.
So this is the answer.
This is good.
The problem doesn't say show the answer in degrees Celsius, but since everybody knows degrees Celsius-- everybody in science relies on degrees Celsius-- you can just convert this over to degrees Celsius, which happens to be 342 degrees Celsius.
So this right here is your answer to part B.
And again, I want to express again that the crucial part in getting this is knowing the fact that you're given an energy in electron volts, but the constant that you use has units of joules.
So you need to convert to get the right unit out or else you will get not Kelvin-- you would get something different here, which wouldn't make any sense given the question that was asked.
So with that, again I advise that-- always read the problem in detail.
Look at the units that you're working with and make the appropriate conversions because everything should be on your table of contents.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
I'm Sal.
Today we're going to solve problem number two of test three of fall 2009.
Now before you attempt the problem, there are certain things that you need to know to solve the problem.
One is mechanical properties of metal and also the properties of amorphous metals, which are known as metallic glasses, and how to draw crystal structures given to you.
The problem reads as follows-- for a given alloy composition, explain why the yield strength of the amorphous form, the metallic glass, is greater than that of the crystal form.
Now in order to answer this question, you want to start thinking about the mechanical properties that these two raw materials have.
So if I look at a metal, I know that the way a metal deforms before it reaches classic deformation-- it deforms the planes sliding past one another-- so it's known as slip.
So I know that metals-- they form via slip and slip is facilitated by dislocations and you should know what a dislocation is by now, which is just a line imperfection in your crystal.
So slip is facilitated by dislocations.
So I know that my metal has planes that are arranged because it's a nice crystallographic structure-- and that the slip-- when it undergoes deformation, that this is actually assisted by dislocation.
So having dislocations helps to form our metal.
So now I want to start thinking about what a metallic glass has-- so the amorphous form.
So if I look at the metallic glass-- and all I'm doing here is comparing the basic properties between the two-- so this is going to get me metallic glass.
I know that for metallic glass, there's no long range order.
So the fact that there's no long range order means that these metallic glasses don't form a perfect crystal array.
And that lack leads to metallic glasses having no dislocations.
So if a metal slip is facilitated by dislocations and if a metallic glass doesn't have dislocations, then I would assume, just by comparison between the two, that the threshold for one of the materials to yield would be due to the fact that-- whether or not I have dislocations.
So with this argument, I would say that the metallic glass has a higher yield strength than my metal because of this property.
So that-- if you did that-- if you went ahead and wrote all that out for your answer in part A, you should be able to get most of the points for this problem.
Now part B-- it asks, on each of the following separate drawings of one face of an FCC unit cell-- FCC stands for face centered cubic-- indicate one of each of the following.
So now we're going to move on to part B-- and part B wants-- there's three scenarios.
One-- it wants us to indicate on a face of a crystal, a substitutional impurity-- so substitutional impurity.
So if I look at-- if I draw a face-- here.
I can-- so to find out what the definition of a substitutional impurity is, I know that it's not going to be anything that can be just embedded in between atoms-- like an interstitial.
I know that it's going to be a substitutional impurity.
So if this is one of the faces of your crystal-- and it's FCC, which is face centered, from the face-- the face is centered on the center of an atom on your face.
Then I would say that if I substitute this out for a different atom of type B compared to these atoms, then that's a substitutional impurity because that atom is not the same as the other.
It's like having silver and gold, for example, in composition.
And then it also asks for the second part.
So that's number one.
For number two, it wants us to demonstrate a vacancy.
Now a vacancy is simply an atom missing from a normal lattice site on your crystal array.
So if I draw again another face, then I can leave that blank and I can go ahead and I can point and I can say that the vacancy is actually-- sits right here.
So this is my vacancy.
There's no atom that's there.
It moved out of its place.
So that's another defect that we put in there.
And for three, it asks, an interstitial impurity.
So what's interstitial?
Well, an atom that can fit in between two atoms that are different than the one that fits.
So if I draw again an FCC face, I notice that there's certain spots where an atom of a different composition can fit and I can go ahead and I can just put a-- that's an atom of different composition and that can very simply be your interstitial.
So that's your interstitial impurity.
So knowing the definitions, you're able to answer this problem very simply.
There was no math involved in this and all you needed to know was knowledge of our mechanical properties of metals versus amorphous metals, which are known as metallic glasses.
And knowing that information, you're able to answer a question like this, which is pretty simple after you finish and you think about it.
And if you didn't know what FCC was-- which stands for face centered cubic, you could have probably lost a lot of points on this problem.
But if you know that, if you know how to draw it, then you're good to go.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
Jocelyn here and we're going to go over fall 2009 exam three problem number three.
As always, let's read the question first.
Calcium ammonium phosphate dissolves in water according to the dissolution equation, for which the value of the solubility product, ksp, has been determined to be 4.4 times 10 to the -14.
Calculate the solubility of the compound in water.
Express your answers in units of molartiy, ie moles of the compound per liter of solution.
So the first thing we want to do is write down what the question's asking.
Part A-- how much calcium ammonium phosphate can dissolve in water?
And we're going to call this cs-- or the saturation concentration.
Next we need to figure out how to find this out, right?
So it gives us that the ksp-- 4.4 times 10 to the -14.
So we need to know something about the ksp and although it's called the solubility product, it's the same as other equilibrium constants.
So equilibrium constants tells us about the concentrations at equilibrium and therefore the maximum solubility.
Looking at the equation, we have calcium ammonium phosphate dissolving-- which is a solid-- dissolving into calcium ions, ammonium ions and phosphate ions.
As with every equilibrium constant, we can write down an equation relating to the concentration of the species.
So our ksp is the product of the concentration.
And a common mistake made on this problem was that people included-- students included the calcium ammonium phosphate solid on the bottom as you normally would-- where you would normally put the reactant.
However, remember that for equilibrium constants, we don't put solids in there because the activities don't change-- or the concentration doesn't change.
It doesn't really make sense to add it in.
So here we just look at the solvated ions and we see that we have a 1:1:1 molar ratio here.
That's going to make our life a little bit easier.
So now we need to figure out what the solubility of the compound is and to do that, we want to look at this chemical equation.
We see that for one mole-- so one mole of calcium ammonium phosphate dissolved gives you one mole of the calcium ion, the sodium ion and the phosphate ion.
Thus, we can say the amount of calcium ammonium phosphate dissolved, which we called Cs, is going to be equivalent to each of these concentrations.
Because we're asked for how much of the compound is dissolved, but we're given the ksp, which is in terms of the concentrations of these solvated ions.
I'm sorry.
This doesn't equal the product.
These are all equal.
Now we can plug the Cs into our ksp equation that we had before.
So instead of the concentration of calcium ions, we have-- it's equal to the concentration of the compound dissolved.
The same goes for the ammonium and the phosphate, because again, we have 1:1:1:1 stoichiometric coefficients.
And we can be a little more concise.
Now it's just a matter of solving for the saturated concentration.
So doing some algebra here.
Then we plug in the numbers.
And we get that the solubility of calcium ammonium phosphate is 3.53 times 10 to the -5 moles-- molar, sorry.
So that means 3.53 times -5 moles per liter of water.
That's not very much.
So we can say generally or relatively this calcium ammonium phosphate is not that soluble in water.
So let's move on to part B.
Part B says, calculate the solubility of calcium ammonium phosphate in 2.2 molar calcium bromide.
Express your answer in units molarity.
Assume that in water, calcium bromide completely disassociates into calcium 2 plus cations and bromide anions.
This is basically asking us for the same value.
It's asking us for the solubility of calcium ammonium phosphate, but under slightly different conditions.
So we want to write those conditions down.
Now we have to ask ourselves why would the fact that we have a concentration of calcium bromide affect the solubility of calcium ammonium phosphate?
And the thing to remember here is the common ion effect, right?
2 molar calcium bromide will-- when dissolved in water, the concentration of calcium from just the calcium bromide will be 2.2 molar, right?
Because we're told that it completely disassociates.
So if we look at the disassociation reaction, we see that for every mole of calcium bromide, we get one mole of calcium and two moles of bromide.
So having calcium bromide will alter our answer from the previous problem because the ksp will always be the same.
It's an equilibrium constant, right?
So even though it has to do with calcium ammonium phosphate, we have to take into account that we already have calcium in the system.
So before we start plugging in any numbers, we should think about this problem.
Do we think that already having calcium in the system will increase or decrease the solubility of the calcium ammonium phosphate?
Hopefully we can agree that it would probably decrease the solubility because you already have those calcium ions.
So putting more calcium ions into the water is going to be harder and therefore less calcium ammonium phosphate will dissolve.
Now that we know what kind of number we're looking for, we can start doing the actual calculation.
So again, we'll start with our equation for the ksp, which for calcium ammonium phosphate has not changed.
I'm just going to rewrite it on this board here.
But now instead of having each of these be equal concentrations, we already have some calcium in the system.
So we need to take that into account.
And I'm going to call the saturation concentration Cs star because we're under different conditions, right?
The calcium bromide does not contribute any ammonium or phosphate ions so those concentrations will just be determined by how much of the calcium ammonium phosphate dissolves.
From before, we can use what we found in part A to simplify this a little bit.
So in part A, we know that without anything else, without a common ion effect, that we have decided will decrease the solubility.
We have 3.35 times 10 to the -5th molar solubility.
Therefore, we can say with good certainty that our saturation concentration with the common ion effect will be much, much less than 2.2 molar because 2.2 molar is much greater than our pure solubility.
Going back to our equation over here, that means we can have-- and now we have a fairly simple algebraic problem.
Dividing by 2.2 and then taking the square root, we get that the saturation concentration under these conditions-- 2.2 molar calcium bromide-- will be 1.41 times 10 to the -7 molar.
And going back to our answer from part A, we see that this is indeed a lower solubility.
There's less calcium ammonium phosphate that can be dissolved and that makes sense from our previous thought about this problem.
And so now that you've found the answer, box it and we're done.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
Jocelyn here and today we're going to go over fall 2009 exam three problem number four.
Starting with part A-- let's read the problem.
In the 1920s, Jack Breitbart of Revlon laboratories found that acne could be treated by the use of benzoyl peroxide.
The oxygen-oxygen bond in peroxide is weak and under the influence of modest heating, benzoyl peroxide readily decompose to form free radicals according to the reaction given.
The rate of decomposition is measured at 92 degrees C at various concentrations and found to be-- and you're given two different concentrations and two different rates.
So part A asks, determine the order of reaction for the decomposition of benzoyl peroxide.
The first thing to do is, when asked for the order of the decomposition reaction, we want to think about the general rate loss, right?
We're given concentrations.
We're given rates.
We're asked for the order.
Something you might want to have written down on your equation sheet or look up is the general rate law equation.
Remember that students taking these exams did have an equation sheet so they could have this ready.
The important part is that you know when to use this equation, what each of the terms mean.
So r is the rate.
k is a rate constant.
It is a function of temperature, but if you're at the same temperature, that rate constant will not change.
It's constant.
c is your concentration.
And n is your reaction order.
So this would be the place to start for this question.
And now we know that we're solving for n. So what is n in this case?
Looking at what we're given, we can write two different equations.
The first line gives us a specific rate for a specific concentration.
The second line gives us another rate for a second concentration.
So now we have two equations and two unknowns.
That means we can solve for one of the unknowns and then could solve for the other one if we wanted to, but here we only care about n. So how I'm going to choose to solve this and you may choose to do it differently-- I'm going to divide the first line by the second line and I get rate one over rate two equals c1 divided by c2 all to the n power.
So that made the k drop out, which is one of the unknowns that I don't care about because the problem isn't asking for that.
Now all we need to do is solve for n. So if we remember algebra, we're going to take the log of both sides.
Because of the properties of the log, we can bring the n out front.
So n equals-- and plugging in the numbers given in the problem-- they're given in the same units so everything will cancel out and that's good-- we get that the n equals one.
So this is a first order reaction.
Now you could've probably done that by inspection if you're familiar with the rate law and how to determine that by just seeing that as the concentration went down by a fourth, the rate also went down by a fourth.
However, the question asks for you to determine the order of reaction and so we were looking for more of a derivation of the problem instead of just an inspection because that shows that you have a familiarity with the general rate law equation.
So now moving to part B-- we are asked, on the plot below, sketch the variation in energy with extent of reaction for the decomposition of benzoyl peroxide.
Assume that the ratio of activation energy to-- delta e, which is the energy of the reaction-- equals -2.5.
Label the energy states and label the delta e of reaction, the activation energy for the forward reaction and the activation energy for the backward reaction.
So that's asking us to do a lot.
So first we're going to write down, what are we actually asked to do?
So the main thing is that we're asked for the variation in energy with the extent of reaction.
So as the reaction progresses, what's the energy of the molecule?
The energy state that it's going through?
In that plot, we're asked to label the energy states of the reactant and product and we're asked to label the delta e of reaction.
So what's the net change in energy as well as the activation, both forward and backward?
I would always write this to the side or scrap piece of paper or something because this is a lot of things to put on one plot and this will be a nice reference to help us to make sure we answered all the things the question is asking.
So on the plot given, which I'll reproduce here, it has axes of extent of reaction and energy.
The first thing to do would be to label the beginning and start energy states.
So because we know at 92 degrees C, this-- the benzoyl peroxide-- decomposes readily, we can assume that there's a decrease in energy.
So we'll have our products start at a higher energy than our-- sorry-- our reactant start at a higher energy than our products.
And because the question asks us to label that, we're going to put in the actual labels.
And so that's your reactant and then we have the radical product.
Now that we have our beginning and end energy states, we need to think about what's happening in the middle.
A clue is that he talks about the activation energy, right?
And we know that most processes that we talk about have a certain activation energy.
You can't just go straight from the molecule to the radical.
You have to have a little bit of energy cuffed.
And so we know we're going to have some type of hill that we have to go over.
Furthermore, in the problem he states that the activation energy divided by the delta e is -2.5.
So we know that because this is a negative delta e-- so this is our delta e here and it's negative-- our activation energy is two and a half times as large as that net change in energy.
So we want to kind of just eyeball that in here.
You didn't have to be exact, but relatively-- it's going to be larger than twice the distance between here.
And remember that activation energy is the hill, the energy costs that you have to go through to get to your lower energy state in the product size.
So now that we have all of our energy states, we want to draw a smooth curve to show the variation of energy with the reaction.
So connecting all these points, we get two valleys and a hill, right?
We have two stable energy states and we have an activated complex corresponding to the activation energy there.
So we have our energy diagram, but we need to go back to what the question is asking and see that we've labeled the energy states, we've labeled the delta e of reaction, but we need to label both the forward and backwards activation energy.
And this is one of the things people had issues with-- the backward activation.
So forward activation is pretty self explanatory, right?
It's the energy we need to overcome the activation of this process and so this is the Ea of the forward.
Now the activation energy of the backward reaction is just the same concept, but going backwards.
So if we started down here, what is the energy we need to get over this activation barrier?
And so that's this full energy, this full amount of energy here.
And so we can see that it's bigger than the activation energy for the forward reaction by the delta e of the reaction.
So going back to our checklist here, we've answered all over the questions and so we can move to part C. So part C asks us, on the same plot above-- which we have over here to the right-- sketch the variation in energy with extent of reaction for the decomposition of benzoyl peroxide under the influence of a catalyst.
So with a catalyst-- first to answer this question, we need to know what a catalyst does.
And in class, we learned that a catalyst will lower the activation barrier, thus allowing for the reaction to proceed faster.
You don't necessarily get more product because that's governed by a different set of laws, but you do get the product faster.
And so the way we can change this is that instead of having our activation barrier up there, we can say it's going to be somewhere here.
It's going to be lowered by the presence of the catalyst.
So we want to make a smooth curve again and connect this lower activation barrier.
And you'd want to label this-- I'll label it in white-- this is the catalyzed curve there.
As long as you've showed that the activation barrier was lowered by the presence of a catalyst, that's fine.
That was what the question was asking for.
So we've labeled everything, answered all the questions and we're done with this problem.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, I'm Sal.
Today, we're going to solve Problem #5 of Exam 3 of Fall 2009.
Now before you start this problem, there are a couple of things that you need to know in order for you to fully understand the science behind it.
And so what you need to know is Fick's first law.
Not just knowledge of it, but how it works.
Also, Fick's second law.
And not necessarily the equation form, but the solution to Fick's second law, which is a function of concentrations, and which we will refer to to solve the problem.
And also know popular dislocations, how they help your material and whatnot.
So the problem reads as follows.
Specimens of steel are being surface hardened by the introduction of carbon.
The concentration of carbon at the free surface of the steel is kept constant at c sub s. Chemical analysis of a number of specimens indicate that after a time-- t sub 1-- the carbon concentration has the value of c star at a depth from the surface-- x1-- shown below, which is shown on your graph.
So this is given to you.
So this pretty much is your concentration profile as a function of position.
And it's telling you what's happening.
So a good thing to do is draw a picture of the science that's happening here.
And they tell you, first of all, I'm going to draw it underneath here that you have a rod of steel.
And so this is steel.
And I have a certain fixed concentration here at the surface of carbon.
So I have some type of a carbon flux in my surface.
And this is a visual image of how it looks.
And what the problem says is that you have a certain concentration on your surface, which is given by c sub s. And then it starts telling you that chemical analysis was able to go ahead and measure this concentration at two different values inside your position at different times.
So if I continue reading the problem, it says that chemical analysis of a number of specimens indicate that after a time-- t1-- the carbon concentration has a value of c star.
So this one right here-- this is x1-- so this has a value of x1 at t1.
And it also tells you that after a certain time-- t2, which is equal to 2 times t1-- the carbon concentration has a value of c star, so the same value at a depth of x2.
So now, this thing says that at a depth of x2, at a time of t2, we measure the same concentration, c sub star in the specimen.
And it tells you that x2 equals 2x1, and it also tells us that t2 equals 2t1.
Now they tell us that for a reason, because it's probably expected for us to use this to solve the problem, so don't forget that.
So the question is, under these conditions, would you describe the rate of carburization as one, steady state diffusion.
Two, transient state diffusion.
Or three, not governed by any diffusion, i.e., rate limited by some other process.
Now you have to justify your choice.
So this is pretty much a problem of process of elimination.
And for part a, which is-- I'll label this part a-- it wants us to do three things.
So the first thing, it asks us if it's steady state.
So the first thing to ask yourself is, well, what's steady state?
And the definition of steady state is that whatever phenomenon is happening doesn't vary with time.
But the problem already tells you that it varies with time.
It tells you that at a certain time you measure this and at another time you measure this, and you have this flux coming in, and this is what's happening.
So from just reading the problem, I can already say that this process can't be steady state, because your concentration profile varies as a function of time.
So for one, I would write, can't be steady state due to the concentration being a function of time.
So one is out, so we can scratch that.
It's not a steady state.
Now two.
This is where it gets a little bit more complicated.
Two is a transient state diffusion.
Well, how do we know whether or not this is governed with time?
Well, you would argue, because this is not steady state, then it has to be time dependent.
But, there are certain rules that you need to know to be able to make that justification.
First of all, if it's time dependent, then whatever your concentration and whatever your position in your time is, has to fit into your solution to Fick's law.
So if I look at my solution to Fick's law right here, I know that on the left side of my equation, I have the concentration at any point in x-- that's what c stands for-- minus the concentration at my surface, which is given as cs-- sorry, this is supposed to be sub s-- divided by the initial concentration.
So c sub naught is if there was already some carbon in my steel, minus, again, the surface concentration.
This equals to an error function-- which is the solution to Fick's second law-- and this error function has the values of position, diffusion coefficient, and time.
So in order to verify if it's a transient state, we're going to go ahead and take that equation and break it down.
So if I assume that there's no carbon, I can go ahead and write this down.
Assume c0 is 0, just to make the equation easy.
I can, because there was no information given about that.
I can go ahead and-- because we're trying to figure out if it's a transient state-- write this down.
So c minus c sub s over negative c sub s. This equals to my error function, which is x divided by 2 squared of Dt.
And I know that at x1 and time-- t1-- I have this value of my concentration, c sub star.
So I'm going to go ahead and plug that in here into this equation.
So I know that my concentration is c sub star.
So that's what we measured, c sub star minus c sub s over negative c sub s. This equals to my error function, x1 over 2 square root of Dt1.
So this is for the first time point, x1, t1.
Now I want to go ahead and do the same thing for the second point.
And if I do it for the second point, I know that just by looking at the equation, I have the same concentration at x2, and I have the same surface concentration.
So what that tells me is that this value is going to be the same for x1, t1, and x2, t2.
So if I go ahead and plug this into my equation.
I can go ahead and look at this and say, this is for t equals 1 for that, and this is going to be-- or, t equals t sub 1, sorry-- and this is t sub 2.
I have c star minus cs all over negative cs.
This equals to my error function of x2 over 2 square root of Dt2.
So I know that this value is the same as this value.
So therefore, this should be the same as this.
The argument in both your error functions.
So we have to go ahead and prove that.
Well, how are we going to do that?
Do I know x2 as a function of x1?
I do, because that's given in the data.
And I know again, that x2 equals 2x1 and t2 equals 2t1.
So what I want to know, does x1 over 2 root Dt1, does this equal to x2 over 2 root Dt2?
Well, I'm going to go ahead and plug in my values that I know, and this becomes 2x1 divided by 2 times the square root of D-- and t2 happens to be 2t1.
So those two cancel each other out, and I end up getting that this is actually equal to x1 over square root of 2Dt1.
Now this and this are not the same.
Because this has a square root of 2, where the other one is just a 2.
So because it did not satisfy the solution to Fick's second law, we can say that it's not case two either.
This is not transient state diffusion.
So therefore, by the process of elimination, it has to be governed by some other phenomena.
And that is the answer to this problem.
The fact that it's governed by something that we don't have enough information to answer.
We don't have enough data points to answer.
So that's part a, which is good.
Now part b asks, if the steel specimens in part a were replaced with steel specimens of identical composition but with a dislocation density 10 times greater than that of the steel in part a, how would the rate of uptake of carbon change?
Explain.
Again, this requires you to know properties of dislocations, and what dislocations do to your materials.
Your metals, for example.
What is a dislocation?
Well, a dislocation is-- if you have a bunch of a plane of atoms-- what a line dislocation is, or an edge dislocation is, this line defect such that it just terminates.
So you have a plane of atoms that just comes to a point and then it stops.
And what do you create here?
Well it creates void, and void facilitates diffusion.
It increases-- in our case, we're talking about carbon diffusion into steel-- so I would argue that the more dislocations that you have, the greater the diffusion should be.
Because you have more void-- more spaces-- for your carbon atom to jump into and from.
So for part b, just by arguing the fact that dislocations represent defects in your crystal-- specifically a defect that can create void-- then with that argument, the fact that more void means higher diffusion, I would conclude that-- for this part-- that my rate of diffusion of carbon should increase when using steel of a greater dislocation density.
So the rate of carbon uptake or diffusion of carbon into your steel increases as you increase your dislocation density.
And with that, you're to be able to solve a lot of these problems.
And these are common in 3.091.
And again, knowledge of Fick's first and second law, how to use the solutions to Fick's first, second law, and knowing what a dislocation is, that will help you a lot in solving these problems.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, I'm Sal.
Today, we're going to solve Problem 8 of the Final Exam of 2009 for 3.091.
The problem reads as follows.
The phospholipid-- the chemical formula is right here-- is added to hexane-- which is C6H14, which is a hydrocarbon-- in sufficient quantities to enable interaction between phospholipid molecules.
Draw a cartoon depicting one possible molecular arrangement that results.
You may stylize your parts of the phospholipid molecule with the squiggly line-- which is shown in the exam-- for a hydrocarbon chain.
So if we look at our molecule here, the first thing that we want to understand is what the structure is.
And as you can see, the molecule has a nonpolar component and it has a polar component.
So for the problem, it will be good to isolate the two with different colors, and then we'll stylize them such that the nonpolar part is the zigzag line and the polar component is the red circle.
So we want to go ahead and draw the arrangement together when it's added to hexane.
So the easiest thing to do is to draw a picture.
So we come over here to the right.
And this is part a. I'm going to draw a beaker with hexane.
And I know that as soon as I look at my chemical formula for hexane-- which is C6H14 and is a hydrocarbon-- I know that it's nonpolar.
So I know from my knowledge of 3.091, when chemical bonding, that like dissolves like.
So the molecules that are polar are going to want to stick together, and molecules that are nonpolar are going to want to stick together.
So by looking at my phospholipid, I already know that-- the problem tells me that-- I'm adding a sufficient amount such that the molecules can interact with each other.
So the only thing that comes to mind for me that will be a possible arrangement, would be that-- in the hexane, which is nonpolar, and if we have a distribution of these phospholipid molecules-- then one possible arrangement will be that you have a bunch of your polar heads interacting together with the hydrocarbon components out here interacting with the hexane.
Because the hydrocarbon component again is nonpolar.
The hexane is nonpolar.
So those want to interact with each other.
But the polar component, which are the heads for the molecule-- the phospholipid as a molecule-- are going to go ahead and interact with each other.
And pretty much if you were to write this down on the exam, you would get full points for part a.
Now we'll move on to part b.
Part b says, to the beaker described in part a, an equal volume of liquid ammonia-- NH3-- is added.
The mixture of hexane-NH3-- or ammonia containing the phospholipid-- is now subjected to vigorous agitation for several minutes and then allowed to equilibrate.
Draw a schematic representation depicting one possible molecular arrangement that results.
The values of density are 0.65 grams per centimeter cubed for hexane and 0.6 grams per centimeter cubed for liquid ammonia.
Assume that hexane and liquid ammonia are mutually immiscible.
So there's a lot of information within the paragraph.
And the first thing that it gives us is the densities between the two components.
It tells us they're immiscible, so immiscibility means that they're not going to mix.
The question that you want to answer now is, which layer is going to be at the top?
Which layer is going to be at the bottom?
Well, the more dense layer should be at the bottom, and the more dense happens to be hexane.
So if I go ahead and draw my beaker again for part b, and then I mix my components.
And now I have two components that are separated by an immiscible boundary.
And I know that the density of hexane is given by 0.65 grams per centimeter cubed.
And the density of liquid ammonia-- which is NH3-- is given as 0.60 grams per centimeter cubed.
So I know that the hexane is going to be down here, so I'll go ahead and label this C6H14.
And I'll label this as NH3.
So one important thing to know is, what is the structure of ammonia?
Well, we know that ammonia happens to be a polar molecule.
And its structure is simply given by-- you have a central nitrogen atom, and you have three hydrogens and a lone pair of electrons-- and the lone pair of electrons is what gives the ammonia the structure.
Hence, you have a polar dipole moment on your molecule.
The fact that this is polar means that it's going to interact with the polar head of your phospholipid molecule.
And we already know we have hexane plus your phospholipid.
Well, if in our arrangement with the absence of ammonia the heads group together because that was the only polar component in the mixture, then over here I should expect the heads to rearrange themselves such that they are in contact with the ammonia.
So one possible arrangement for the system would be that I would have my polar part of my molecule interacting with the ammonia, and the hydrocarbon chain interacting-- which is the nonpolar component-- interacting with the nonpolar mixture.
So if you were to draw this schematic, then you would get full points on the exam as well.
So that's again, like dissolves like.
And if you fully understand the nature of secondary bonding, then this problem shouldn't be that difficult to go through.
Now there's a final part to this problem, which is part c. And part c says, to the beaker described in part b, a small amount of iodine-- I2-- is added to the mixture, and then subjected to vigorous agitation for several minutes, and then allowed to equilibrate.
Explain what you expect to observe.
Again, we're going on this whole issue of like dissolves like, or that polar molecules want to be with polar molecules because they should be miscible together.
And we are given that now we're going to add I2-- iodine.
Well, iodine is a linear molecule that's nonpolar.
There are no other atoms that are bonded to it that can give any polarity.
So if I was to go ahead and draw this diagram, the first thing that comes to my mind is that the iodine should mix with the hexane, because essentially it's a nonpolar component mixing with another nonpolar component.
So if I draw my diagram, I'm going to go ahead and draw the two immiscible regions.
And one question that a lot of students have is, how do you know that the iodine is not going to be on top of the ammonia?
Well, iodine is pretty dense.
It's actually more dense than these two molecules over here.
So the fact that it's more dense, I know it's going to want to sit at the bottom where the other nonpolar molecule is at.
So I would argue that down here we would have hexane-- C6H14-- mixed with iodine.
Over here, we will still have our ammonia.
And then our polar heads would still be-- the polar part of the phospholipid molecule will be in contact or bonding with the ammonia.
And the nonpolar tail would be with the nonpolar components which are hexane and iodine.
So if you did this on the exam, you get full points.
And being able to fully understand secondary bonding will help you solve this problem in no less than 10 minutes.
So that's it.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
I'm Sal.
Today, we're going to solve problem nine of fall 2009, the final for 3091.
The problem reads, the phase diagram of the binary system neodymium, praseodymium and EPR is given below.
There are two allotropes-- alpha, which is hexagonal close pack-- HCP-- and beta, which is body center cubic-- BCC.
Now this problem obviously is about phase diagrams.
So before you attempt to solve the problem, you need to make sure that you understand how to read phase diagrams and also how to apply the lever rule and know where in the phase diagram to apply the lever rule.
So if we look at our phase diagram, which I have drawn over here for part A-- the part A asks, explain why a lenticular phase diagram is to be expected for the binary system.
So this is a lenticular phase diagram and a lot of students when answering this problem get confused and say, oh, because the components are soluble together, that's why you get a lenticular.
That's correct, but that information you can get from the phase diagram.
It's obvious by looking that you have an alpha phase that you have from zero to a 100% that can be missable together, but it doesn't really explain the answer.
Now the best thing to do is to look at the periodic table of elements and look at the physical and chemical properties of the elements.
So here we're comparing to-- we're comparing neodymium and praseodymium, which are two components that are making the alloy.
So here this is atomic percent praseodymium on your x axis and on your y axis, you have temperature in degrees Celsius.
So at zero percent praseodymium, we have a 100% neodymium and at 100% praseodymium, we have zero percent neodymium.
That pretty much comes from the phase diagram.
So why would these two have formed a lenticular phase diagram?
Well, if you look at the periodic table, you'll notice that these two elements lie right next to each other.
So what does that mean?
That means that chemically, they're pretty much almost the same.
To answer this question for part A, I would look at it and I would notice that actually both these elements have the same crystal structure.
So I would write down, they have the same crystal structure.
They also have pretty much the same electronegativity or very similar electronegativity.
So very similar electronegativity.
Also, they have pretty much the same atomic radii--atomic radius.
And also very similar, very close to each other-- the same density.
So the density is almost the same.
So because these two components have the same properties together-- the same size, same crystal structure, then you should expect that they should form a missable lenticular phase diagram, which is what we have here.
So if you were to answer that for part A, that would give you pretty much all the points and that hints that you understand how to look at the chemical properties of the elements and the physical properties of the elements and compare them together and rationalize why the phase diagram looks the way it does.
Part B asks-- I'll come over here to do part B.
At each point, I identify all the phases present in equilibrium.
II-- state the composition of each phase and III-- calculate the relative amount of all phase presence.
And it asks us to analyze the three points that we have on our phase diagram.
So we have one for I, II, III-- So number one-- if we look at our phase diagram, we lie exactly where your alpha phase is.
So to answer the first part of the problem is identify all the phases present-- we only have the alpha phase here.
So if you were to write down alpha phase, that's correct.
The part two says, state the composition of each phase.
So what's the composition of the alpha phase in here?
Well, if I trace down my line down to my x axis, I notice that for the atomic percent of praseodymium, pretty close to about 18%.
So it's 18 atomic percent here.
So to answer the question for part I, I would notice that-- I would write down that it's alpha phase.
For the second part, we have 18%-- 18 atomic percent praseodymium, which leaves behind about 82% atomic percent of neodymium.
And for part three-- part three asks about the relative amounts of the phases present-- we only have the alpha phase in that part so I would say that we have 100% pure alpha, which is-- if you write this, that's correct.
Now for the second point that we're going to analyze in our phase diagram number two-- if we come over here and look at it, you'll notice that now we lie in between or between two phases so where the lenticular region is at.
So here, I don't have pure beta or pure liquid.
I have a combination of both beta and liquid and this is exactly where you need to apply the lever rule.
You can't apply the lever rule here because it's just 100% pure phase.
You can only apply when you're in a region where you have two phases that coexist in equilibrium.
So if I go ahead and analyze part two-- part I-- the phases that are present are beta and the liquid phase.
For part II, it asks, what are the compositions?
Well, if I blow up my lenticular phase diagram-- so I'm sitting right here and this is sitting at around 62 two atomic percent of Pr-- praseodymium and it asks about the phases or the relative composition of each phase.
Well, over here, I have beta, over here I have the liquid phase and in between I have the beta plus the liquid that are coexisting in equilibrium.
So if I want to know how much-- what's the composition of my beta phase that I have here-- well, I'm going to have to go to the line where both intersect with the beta and the beta plus liquid intersects.
And if I analyze this point-- if I look down what the composition of that is for that line, I see that I have about 56 atomic percent of praseodymium.
So to start answering part two, I would say that if I look at beta-- in the beta phase, I have 56% or atomic percent of Pr and if I have 56% of Pr, then this leaves just 44 atomic percent of neodymium and if I look at my liquid-- so I'm going to just move this over here.
I look at the liquid, which is going to be where it intersects in the liquid region and if I trace it down, I see that this is about 70% or so.
So here I have around roughly 70%.
If you trace it down to your x axis, this is 70% or 70 atomic percent Pr, which leaves 30 atomic percent of neodymium.
So if you were to write this down for the liquid, just write an arrow here so I won't have to rewrite that-- that would be the answer for part two because again, we're analyzing this point, which is point number two and we're in between two phases.
So that covers the second part of part B.
And for the third part-- I'll go ahead and I'll put up here.
That one now is where we have to apply the lever rule because now we have to figure out exactly how much of each phase is present at equilibrium.
So if I look at my diagram, it's going to be very simple.
If I want to know how much beta I have in this mixture, you're simply going to apply the lever rule and for beta, it's going to be whatever this magnitude is or whatever the difference between these two points are over the length of the whole existence at that temperature of the length of the whole composition that you have.
So if I do the simple calculation for beta-- beta is going to be simply 70 minus 62 over 70 minus 56 and this essentially just equals eight over 14, which is about 57 atomic percent praseodymium.
So-- sorry-- not percent Pr-- percent beta phase.
So in beta, I have 57%.
So that means that for my liquid-- I can do two things for this problem, for the liquid.
I know that everything is normalized to 100%.
So the difference between 157 will be the liquid composition where I can go ahead and apply the lever rule again and do the math.
So if I apply the lever rule again for my liquid phase, it's going to simply be 62 minus 56.
We're coming back on our-- looking at the liquid phase that is coexisting in this equilibrium at this temperature and I want to know how much liquid-- it's going to be the magnitude of this over the total length of my composition there-- that's what I'm doing here.
Again, this is 70 minus 56.
So this would end up being just six over 14, which is essentially just-- if you have 57% here, if you do the math, you're going to get 43% of liquid.
So part three-- when you apply the lever rule, you see that at this point you have more beta than you have liquid-- and it kind of makes sense because if you look at your diagram, your point is sitting closer to the beta phase.
So I would expect intuitively that I would have more beta present then the liquid present and that's exactly what the lever rule tells us.
So that's point number two for part B.
And if I look at point number three now, which I'll write right here-- this is essentially the same thing as point number one.
But now point number three-- we're no longer lying in alpha.
We're lying in the beta phase.
So if I trace down what my composition is here, I'm about 78% of Pr-- so to answer the first part I-- beta-- the second part II, which is dealing with the compositions of that beta phase is going to be 78% or 78 atomic percent Pr and that leaves just 22 atomic percent neodymium-- and for the third part III-- what is the relative amounts of the phases present?
Not the composition, the phases.
I'm exactly where only pure beta exists so I can say that I have 100% beta.
So that pretty much does it for this problem.
Again, knowing how to read phase diagrams, knowing how to apply the lever rule and where to apply the lever rule will allow you to very easily solve the problem.
A lot of students make the mistake of trying to apply the lever rule when you have a pure phase.
I mean, that makes no sense because said lever rule pretty much calculates what fraction of the phases are present in that equilibrium mixture.
So keep in mind that you can only do that when you have a region in your phase diagram, when you have two phases that are coexistant in equilibrium.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
Jocelyn here.
And today we're going to go over fall 2009's final exam, problem 10.
As always we're going to start by reading the question.
You have 333 milliliters of alkaline solution at a pH equal to 9.9.
You wish to neutralize this by reacting it with 222 milliliters of acid.
What must be the value of the pH of the acid?
So first off let's write down the information that was given and the information that he's asking for.
So we have 333 milliliters of a solution with pH equal to 9.9.
And he's asking you what must the pH be if you want to put in 222 milliliters in order to completely neutralize that basic solution.
And the key here is to recognize what neutralize means.
And a neutralization is when you take acid, or protons, and base and it reacts to form water, thus creating a neutral solution.
So the next step would probably be to figure out how much excess hydroxide we have in this solution.
And then we can figure out how much hydrogen we need to react with that hydroxide and to form water in order to completely neutralize this solution.
So first step is to find out what our concentration of OH minus is.
And we do that by remembering a few things about acid/base solutions.
Right?
We know that if you have the pH plus the pOH it equals 14.
And this is from the equilibrium constant of this neutralization reaction.
And then we can figure out what the pOH is because we're given the pH of our initial solution-- 9.9-- and that equals 4.01.
Now using the definition of pOH, which is that the pOH equals the negative log of the concentration of the OH minus, we can rearrange that and get that our OH minus concentration equals 10 to the negative pOH.
Right?
Just using our log rules there.
So this gives us that our current concentration of the hydroxide ion is 10 to the negative 4.1.
That gives us the concentration of the hydroxide, but what we really want to know, if we're going to use our neutralization reaction here, is we want to know how many moles of hydroxide we have in that solution.
And we can just do that with stoichiometry.
Right?
The concentration is in moles per liter.
And we know-- so our moles hydroxide equals our concentration, which is 10 to the negative 4.1 moles per liter.
Times-- if we want to get rid of that liter unit we multiply it by our volume, which is 333 milliliters, converting it from milliliters to liters.
And this gives us 2.65 times 10 to the negative 5 moles of OH minus in solution.
So that's just the first step of this problem.
Now that we have the number of moles of OH minus we need to find the number of moles of H plus necessary to react with this excess hydroxide, and then we will be able to find the concentration of hydrogen, or protons, in that second solution.
So first off we're going to look at our neutralization reaction over here and realize that it's a one-to-one stoichiometric ratio.
That means in order to react with 2.65 times 10 to the negative 5 moles of hydroxide, we need the same number of moles of hydrogen.
So over here I'm going to say that this also equals the moles of H plus needed.
All right.
Now that we have our moles H plus needed, we need to do the same process here but in reverse.
So first we need to find the concentration of protons that we have.
So given that we want to only have 222 milliliters of this solution, right?
So we have-- and we just divide that by our volume, 0.222 liters.
Right?
Because we divide by 1,000 to convert from milliliters.
And that equals 1.19 times 10 to the negative 4 moles per liter.
So this is another problem where you could get the answer but not actually be answering the question.
Right?
Because this is the concentration of protons that we need, but now we need to convert that to pH.
So moving over here we'll just write the final steps.
We know the definition of pH is the negative log of the concentration.
And so because we know the concentration of protons we know that the-- we just need to plug this into our calculator and we get our answer, which is a pH of 3.92.
Put a box around that one because it's our correct final answer, and you'll be good to go for this part of the problem.
Now moving on to part b we are asked, name the conjugate base of each of the following.
So we'll just start part b right here.
And the first thing to recognize is what does a conjugate base mean.
And in an acid/base pair, or when you have a kind of dissociation of an acid, in the most simple terms we can think about it just losing a proton.
In the leftover molecule, that's the conjugate base right there.
So in this problem we're asked if the molecule loses a proton, what is the resulting molecule?
Ion, basically.
So part i we have HPO4, and that will dissociate to hydrogen phosphate ion.
Simple enough.
This is your conjugate base.
So that's your answer for part i.
Moving to part ii.
We have CH3NH3 plus.
So it's kind of like an ammonia ion but not quite-- sorry, ammonium.
And we know that this dissociates by losing a hydrogen off of the nitrogen, because that's where the positive charge is.
It makes more sense there.
And we get-- sorry, keeping the same order as above-- a proton plus.
And this would be your conjugate base.
If you were a little bit confused on this one it might help to draw the Lewis structure, and you would see that the formal charge does actually reside on nitrogen and, therefore, it makes sense that the nitrogen would lose a proton.
In addition, if you think about the ammonia/ammonium ion relationship you know that you have NH3 going to NH4.
And in this case we just have one of the hydrogens replaced with a methyl group, and so you can make the analogy there that, oh, the hydrogen would come from the nitrogen in that case.
All right.
So hopefully this was pretty straightforward.
As long as you're comfortable with what a conjugate base is and what the dissociation reactions look like for acids.
So now on part c we are asked to classify each of the following as a Lewis acid or a Lewis base.
Now first we need to remember what a Lewis acid and a Lewis base are.
So if you remember from lecture or in your reading, we know that a Lewis acid is an electron-pair acceptor and a Lewis base is an electron-pair donor.
These definitions are just a little more general than the classic proton acceptor or proton donor.
The first part is asking us about the cyanide ion.
And to kind of clarify it for yourself and because we're asked about Lewis acids and bases, which are to deal with electron pairs, it might be helpful to draw the Lewis structure.
So for this molecule a good Lewis structure looks like this, and it has a net negative charge.
And you can see that we have a couple of lone electron pairs there that aren't involved in bonding.
And because it has a net negative charge you can imagine that these electron pairs would want to kind of grab onto something else that's a little bit more electron positive.
And because of that we know that this is actually a Lewis base.
It acts as a base and would grab something like a proton and thus become more, I don't know, stable in some cases.
All right.
Moving to part 2.
We are asked about water, which is a molecule that we deal a lot with in acid/base chemistry.
Again, you might be able to answer this right off the bat, but we're going to look at the Lewis structure here.
Water has two lone pairs.
So that means it could very well act as a Lewis base.
Hopefully that makes a little bit of sense.
Right?
Because these electrons can go and grab something like a proton and form the hydronium ion with a positive charge, thus acting as a Lewis base.
It's important to remember that even though we're creating an acid in the hydronium ion, this water molecule in converting to the acid acts as a Lewis base.
However, we also know that water can lose a hydrogen and accepts those electrons to form the hydroxide ion, and thus acts as a Lewis acid.
So in the case of water either Lewis acid or Lewis base was a correct answer in this case.
Now moving on to part d. We will go over here, I think.
I'm sorry.
d, not b.
Again, it's going to be about acids and bases.
Consider the effect each of the following substances has on the ionization of the weak base ammonia.
For each state whether the substance, one, suppresses ionization, two, enhances ionization, or, three, has no effect on the ionization of ammonia.
In each instance give a reason for your choice.
Going back to the first part of this question we see that it's asking us about the ionization of ammonia.
A good place to start from here would be to write down the reaction, just so we have that on our paper, in our mind, ready to think about it.
So we have ammonia-- and aqueous means that it's in water, right?
And that is reacting with water to form the ammonium ion-- sorry-- by adding a proton and a hydroxide ion.
OK.
So this is the reaction we're asked about.
Now we need to look at each of the three substances listed and decide how it affects this ionization.
So number i, we have KOH.
Again, a good place to start is to write what happens to potassium hydroxide when it goes into water.
And we know that KOH is a strong base and will dissociate into potassium plus hydroxide.
So we see that it produces hydroxide.
And this should indicate to you that, well, it has the same product as one of the products of the ionization of ammonia.
And if we remember back a little bit before this section, we know that if you have the same ion in solution it will definitely affect the solubility, or ionization.
So that's one way to think about it, is that I now have a common ion effect that will, because there's more hydroxide in solution now, it will push this reaction, this equilibrium, back to the left.
Another way to think about that is Le Chatelier's Principle.
Kind of the same thing, although it's a little bit more versatile, I think, so pretty easy to think about.
Right?
If you add a product your reaction will shift to the left.
If you add a reactant your reaction will shift to the right.
So here we're adding a product, our reaction will shift to the left.
Same as a common ion effect, these two combine to decrease, or both suggest that we will decrease, ionization.
Moving to the second part of this problem.
We're asked about hydrogen chloride.
Again, let's look at what that does in water.
We know that it's a strong acid, so it will dissociate into hydrogen, or protons, and chloride.
Now this one's a little bit trickier because we don't have either of these ions in our initial ionization reaction, but if we think about the fact that ammonia is a weak base we can write an additional ionization reaction.
So that additional would be ammonia reacting with a proton to form ammonium.
Right?
So this is another way that the ammonia molecule would be ionized.
And we can see that if we add protons from this acidic molecule there we will increase the ionization, because it will allow the ammonia to work more as a base because of this reaction here.
So we are increasing ionization.
Another way to think about this is completely valid and kind of the same thing, but if we look at this ionization reaction here where the hydrogen chloride is dissociating into protons and chlorides we can recognize that these protons will then react with the hydroxide that is produced in the initial ionization reaction.
This will result in a decrease of hydroxide in solution, right?
And so by Le Chatelier's Principle we know that a decrease in a product causes the reaction to shift towards the product, to make more hydroxide.
So that's just another way of thinking about the same process, and it results in the same conclusion that addition of hydrogen chloride increases the ionization of ammonia.
OK. Moving to part iii.
We have ammonium chloride, and this will dissociate into ammonium and chloride.
And even if you're not sure how much it will dissociate, it will still dissociate to some amount.
So it will have at least a little bit of an effect on our initial ionization reaction.
So here we see the dissociation produces ammonium ions, and just like the first case that is one of the products in our ionization reaction.
And so Le Chatelier's Principle, common ion effect, whatever you want to call it, tells us that it will shift the equilibrium to the left and decrease ionization.
So with that we're done with problem 10 from the final.
It had a lot to do with acid/bases, so hopefully you feel more comfortable with them after this.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi.
Jocelyn here.
Today, we're going to go over fall 2009 final exam problem number five.
As always, we're going to start by reading the question.
The skeletal structure of the amino acid alanine is given below as it exists in its neutral zwitterion.
To the right is shown its titration curve in aqueous solution.
The abscissa expresses concentration in terms of degree of protonation so that a value of 0.5-- that the neutral ion is the only species present.
At a value of zero, alanine is totally deprotonated and at a value of one, alanine is totally protonated.
So let's move to part A.
What is the hybridization of each of the three carbons in alanine?
So to begin with this one, I would start by rewriting the structure of alanine because as it's written now, it's not as clear what the hybridization of the alpha, beta and gamma carbons are.
So starting with the alpha carbon, connecting the beta carbon-- and here is the gamma carbon-- and of course, we don't want to forget the amine group.
So this looks fairly similar to what is on the page, but I've kind of expanded the structure so that it's more straightforward for us to determine the hybridization.
The alpha carbon is the central carbon here and we see that it has four bonds and four electron domains.
If we go back to the beginning of the class, we learned that when you have four electron domains around a central carbon, we have Sp3 hybridization.
Going to the beta carbon here, we see that it also has four bonds, but one of them is a double bond to oxygen.
So we need to remember from our earlier lessons that the double bond only counts as one electron domain.
Therefore, this carbon has three electron domains.
And going back to our hybridization rules, we know that this gives us an Sp2 hybridization.
Lastly, we have our gamma carbon and it gives us one, two, three, four bonds and four electron domains.
Therefore, our gamma is also Sp3 hybridized.
This problem on the final was dealing with first beginning material, but then in part B and C, we're going to move on to the material we've learned more recently.
So moving to part B-- we are now are dealing with the titration curve that is given to us.
So this looks hopefully fairly similar to what is on your page and the question-- it's asking us to indicate on the titration curve-- one, the PKA for protonation of the zwitterion.
Two, the PKA for deprotonation of the zwitterion.
And three, the isoelectric point.
So before we start that, let's talk a little bit about this titration curve here.
What is it showing us?
What is it depicting?
What I like to do is-- we see that we have two buffered regions.
That right off the bat should tell you that there are two different equilibriums going on.
There are two different protonation/ deprotonation reactions.
So I'm going to split this right down the middle here.
And that way we can think about the two protonations/ deprotonations separately.
So on the left side here, we have the more acidic portion of our titration curve and even if you didn't quite understand his description of the abscissa, we know that because this side-- the solution has a lower Ph, it's going to be dealing with the more protonated species of the alanine.
This-- on the right side, we have the higher Ph, so we're going to have a more deprotonated alanine on this side.
And he already told us that we had the zwitterion in the middle.
So next to 0.5, we're going to write a generic term for a neutral acid.
If we look back to our structure, we see that we have-- the neutral zwitterion has one acidic proton here on the amine group and it has a neutral charge.
So that's why I'm going to depict that zwitterion as HA Moving to the left, more acidic portion of the titration curve, we have-- we know that there's a protonation going on.
So I'm going to write that as H2A plus because whenever we add a proton, we're adding a positive charge.
And then, at the more basic side, we have doubly deprotonated alanine and so we have just our backbone, which I'm going to call a and a negative charge.
So from the question, we should know that this is true, correct?
Because he told us that we have the totally protonated alanine on the left side.
We have the zwitterion in the middle at 0.5 and we have the totally deprotonated on the right side.
OK. What else can we know from this titration curve or what can we add to it so that we can understand it a little bit better?
Well, let's write down the equilibrium reactions that are happening that make this buffer region here.
On left side, we have the doubly protonated alanine going to proton plus the zwitterion.
And that's what's creating this buffered Ph region.
On the right side here, we have the zwitterion in equilibrium with-- again, a proton and the doubly deprotonated alanine.
These are things that are true for any titration curve you see.
Depending upon the number of protonations-- the number of times a buffer region might change, but for each buffered region, you can always determine an equilibrium constant that is causing that stability in Ph.
OK. Now let's go to the question.
We need to indicate on the curve the PKA of both this equilibrium equation and this equilibrium reaction.
So to do that, we hopefully will remember the Henderson-Hasselbalch equation and that gives us a relationship between the Ph of the solution and the PKA of the acid that's in solution.
So writing down the Henderson-Hasselbalch equation-- we have-- not the Ph-- and this is the same equation that Professor Sadoway derived in lecture.
And here, I've written it in the most generic form, but no matter what our equilibrium species are, we just know that we put the more basic species on top and the acid that got deprotonated on the bottom.
So for the first point of this, we are looking for the PKA of the protonation of the zwitterion.
So we know that it's an equilibrium between the doubly deprotonated and the zwitterion.
That is the equation we're looking at.
So let's write down our Henderson-Hasselbalch for that and I'm going to call this PK1 just to keep them separate.
Remember here, my Ha is on top because we have-- our doubly protonated species is the acid in this case.
Now in order to put on the titration curve where the PKA is, we look at the Henderson-Hasselbalch equation and see that where the Ph equals the PKA is when the concentration of the acid and its conjugate base are equivalent.
because if Ha equals H2a plus, the log term goes to zero and we have our Ph equals PK1.
Going back to the titration curve, we see that the concentration between H2a plus and Ha is equal right in the middle.
We can assume that this is a to-scale concentration abscissa here.
And so there we go up and we can say, OK, this point on the titration curve here corresponds to our PK1.
So from the Henderson-Hasselbalch equation, we can see that given the titration curve, this is where our PK1 is and it's about two.
For our purposes, that's-- we weren't asked to estimate it, but if I had to, I'd say it's around two.
Now we have to do the same thing for the other side of the titration curve.
So if you haven't done that already, maybe you can stop now and try that on your own because it's the same process.
OK.
So it's do of the second one-- which again, is just the Ph is equal to the PKA when the two species are equal, except for now we have different species.
So basically we're looking for-- our new Henderson-Hasselbalch equation is-- with using the new equilibrium reaction, our doubly deprotonated alanine is now on top and our zwitterion is on the bottom.
So this might be a little confusing because we have the same species going from the top to bottom in our Henderson-Hasselbalch equation, but remember that we're always talking about the acid is in the denominator and the conjugate base is in the numerator here.
But either way, we again want to look for where the Ph equals the PK2 and so our zwitterion concentration will equal our doubly deprotonated alanine.
And that occurs right here.
So the last part of this question is to indicate where the isoelectric point is and the important thing for this part of the problem, of course, is to know what the isoelectric point meets-- and it's where the dominant species in the solution is the neutral zwitterion.
And Professor Sadoway already told us where that is and it is where we're at 0.5 here, which we've already indicated as where the zwitterion is dominant.
And so we just can go up here and circle that and we can call that number three.
So you didn't have to go through all of the work I did and if you were familiar with the titration curve, you could have simply put these down, but understanding where these points come from is quite important for understanding acid base chemistry and how amino acids work.
All right.
So let's move to part C. In part C, we're asked to draw the skeletal structure of alanine when it is solvated in an aqueous solution at extreme acidity-- ie, Ph less than one.
So I'm going to write that down and then we can go back to our titration curve and see where that is because we've already spend a lot of time understanding our titration curve and that tells us a lot about the acid base chemistry of alanine.
So at the titration curve, we see that at a Ph less than one, we're going to have mostly our doubly protonated zwitterion.
That said, now we need to go back and look at the structure of alanine and we can put in the protonation.
So we're given the zwitterion form of alanine, but at very low acidity-- sorry-- high acidity, low Ph.
This carboxy group is going to be protonated.
And so we have both of our acidic protons are on the alanine and this would be a structure of alanine at a Ph less than one.
Now moving to part D-- we are asked for an aqueous solution of alanine-- calculate the ratio of the concentration of neutral alanine zwitterion to the concentration of deprotonated anion when the Ph is 8.091.
So a couple things I would write down.
Ph equals 8.091 and we are asked for the ratio of the concentration of the zwitterion to the doubly deprotonated anion.
When we're asked for this, hopefully you think first, Henderson-Hasselbalch-- I can use that.
We're given a Ph.
We can find the PKA and we're asked for the ratio and concentrations.
So let's write down the Henderson-Hasselbalch equation again.
And because we're asked for the ratio, we don't need any more information except for the PKA.
Luckily, from part B, we know what the PKA is of this equilibrium between the zwitterion and the doubly deprotonated anion.
So moving back to your titration curve, we see that the PKA is roughly 10-- doesn't need to be our-- our estimation doesn't need to be exact, but if it's in the ballpark range, then you have the right idea.
So I'm going to write that.
We can plug that in just down here.
Log of our concentration.
That's a log term.
And from here, it's just algebra.
So simplify a little bit.
If we bring the 10 over to this side, we get -1.909 equals the log of a minus over-- I'm sorry.
I'm getting a little bit crooked here.
And then if you take both sides of the power of 10, we'll get-- the ratio equal to this.
And if we take the reciprocal of this, we'll get a nicer answer.
We'll get HA over A minus and it won't be a fraction over here.
So I suggest that you do that and you'll see that our final answer is-- let's put it right here.
81.1:1-- the ratio of our zwitterion to the doubly deprotonated anion.
And then you've answered all parts of the problem.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Hi I'm Jocelyn and we're going to go over fall 2009, the final exam, problem number 6.
It says, the dipeptide shown below is derived from alanine and serine.
And I'll write up that structure here.
And we know it's a dipeptide because we see that it has two-- what look like-- amine groups and two kind of carboxyl groups.
So that's an indicator that we're dealing with a dipeptide not just one amino acid.
Something to notice right off the bat is that we're given the zwitterion form.
We have the amine group at the end here is protonated and the carboxylic group on the other end is deprotonated.
So moving to part 1.
It asked us to indicate the position of the peptide bond.
Hopefully this is a pretty straightforward task.
We realize that the dipeptide bond is what makes the dipeptide.
It's the bond between the amine group of one amino acid and the carboxyl group of another.
Right there.
So circling that carbon nitrogen bond would get you full credit on that part.
Moving to part 2.
We are asked to draw the skeletal structure of each constituent amino acid as it would be present in an aqueous solution of an extreme basicity, i.e.
pH greater than 12.
So before even starting to write we want to think about what extreme basicity means.
Well it means that both our amine group and our carboxylic acid group are going to be deprotonated.
So let's start with alanine first.
We have the central carbon.
The amine group.
And the carboxylic acid group.
Now I just drew that totally protonated.
However we want to draw it totally deprotonated because it's in basic solution.
So I'm just going to erase the extra hydrogens.
And we have a deprotonated amine group and a deprotonated carboxyl group.
And that is what it would look like in basic solution.
Now serine is a little different because you have a side group with functionality.
It has not just carbon and hydrogen.
First drawing up the backbone here, we have a central carbon, which has a methyl alcohol group off of it.
And attach the carboxylic acid.
And amine group on the left here.
So this one I drew as the zwitterion, which would be in moderately or kind of neutral.
It depends upon the amino acid.
But again we want to deprotonate our amine group here because pH greater than 12 will not have a protonated amine group.
Now the question is, what do we do with the side chain here?
It has an H attached to an oxygen.
Does that come off in super basic solution or not?
A lot of students got tripped up on this.
An alcohol actually has a pKa of about 16.
And so it won't be deprotonated in an aqueous solution.
Therefore we want to keep this hydrogen on.
It will not be-- this will not be a charged oxygen in the basic solution that we were asked about.
So now we are going to move on to the second part of this problem.
The second part asks us about the tertiary structure of the length of protein that is shown.
I am not going to draw the whole length of protein but we're going to-- at the four specified locations-- we're going to indicate what we would call the tertiary structure; or what functionality is causing the tertiary structure.
So looking at that protein chain.
Professor Sadoway has indicated four different distinct parts of the chain.
And the question is asking us, at each of the four numbered positions name one chemical change to the environment of the protein that would destabilize the associated feature in the tertiary structure.
Explain the relevant chemistry so it's asking us to say what can we do to disrupt that tertiary structure?
And the first thing we need to do is figure out what the tertiary structure is.
So if we look at number 1, we see that it has two sulphur atoms connected together.
So we call that a disulfide bond.
The second part, we see that we have a phenyl group, so it has a very electron-deficient hydrogen interacting with a deprotonated carboxyl oxygen anion.
And so we have a hydrogen bond there and that's what's causing those amino acids to be pulled together.
So let's just say hydrogen bond.
At the third number we see that we had a deprotonate carboxyl group and a protonated amine group interacting.
And that's not quite a hydrogen bond but it's a negatively charged part of the molecule interacting with the positively charged part of a different molecule.
And so we call that electrostatic interaction.
It's similar to a hydrogen bond in that it's a intermolecular interaction that's causing these molecules to move closer, preference to be closer together.
And now we come to the fourth part.
Which is hydrophobic interactions.
So we see that there are a lot of just carbon hydrogen chains.
No oxygen is around, no amine groups around.
Purely carbon and hydrogen.
And because this is an aqueous solution, those hydrophobic parts of the molecules will preferentially go together and kind of make a little-- almost like an-- oil droplet in water.
They're trying to decrease the amount that they experience the polarity of the water.
And so the interactions are hydrophobic.
So now that we know what the functionality at each of these points in the DNA structure are, we can go back to what the question is actually asking us, which is, how can we actually disrupt these features.
So the first one is the disulfide bond.
And in class Professor Sadoway talked about how to disrupt the disulfide bond.
And one of the ways is to put extreme heat on to the protein.
And that breaks the disulfide bond and you can straighten out that portion of the DNA.
This is how you straighten your hair if it's curly or other types of things with the disulfide bond.
So heating the disulfide bond to extreme heat will cause it to break and thus the protein will lose that, that folding or whatever the disulfide bond imparted to that portion of the protein.
Now going on to number 2.
We have hydrogen bonding.
And we see that it's between a alcohol group on a ring there and a carboxyl group.
So one way to disrupt that hydrogen bond is to make the carboxyl group no longer deprotonated.
So if we put back on the hydrogen to that oxygen it will become less partially negative.
It will have a lower interaction with that hydrogen.
And so we can say, to protonate that carboxyl we want to make the solution more acidic.
So we can add acid.
So the disulfide we're going to heat, the hydrogen bond we're going to add acid.
Looking at number 3 we see that we have a electrostatic reaction.
And so we want to not have that electrostatic reaction.
And one way to do that is to make one of those entities not have a charge.
So one way to do that would be to reprotonate the carboxylic acid, just as we did in the previous step.
Or we could deprotonate the amine group.
And that would cause it to lose its charge and thus the electrostatic interaction would be weaker or gone.
So to deprotonate the amine group we should know that we want to add base.
And the final part is, the portion with hydrophilic interactions.
And here we want to figure out a way to make those hydrophobic portions of the amino acids not hold as tightly together.
And one thing we learned about in class is something like soap.
Something that has a hydrophobic side and a hydrophilic side.
And so if we put soap in with this protein it can kind of stabilize the hydrophobic part and make it stretch out a little bit more, not ball up so tightly.
So soap or detergent would be a good answer for this one.
And so for this question, as long as you indicated the relevant chemistry, which is specified what tertiary structure you were looking at and how your suggested procedure would disrupt that tertiary structure, that's what Professor Sadoway was looking for and that would mean that you answered the question completely.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Hi there.
I'm Brian.
We're going to be doing problem number 7 of the 2009 final examination.
So the way I like to approach these problems is to identify what I think is important to know before attempting them.
Things that you should review that will help you successfully navigate the problem.
I call this the what I need, W I N. So for this particular problem, which I think of as the phase diagram problem, you need three things critically in order to successfully complete it.
Number one you need to have a general understanding of what a phase diagram is, how to read it, how to write it, how to interpret the information it gives you.
You can find a lot of this in chapter 9 of Shackleford.
Along the same lines you should probably know how to go in between binary and unary phase diagrams.
How they are related at least.
In this problem it's not absolutely necessary to know this.
But in similar problems we might ask, this is critical to understand how these two are interrelated.
The second thing you should know is some language like critical point and triple point.
Where these are and what they mean on the phase diagram.
And the third thing is the phase coexistence.
So what does that actually mean?
How do you find it on the phase diagram?
So with these three particular points, I'd review them and then I would recommend attempting the problem afterwards.
So we're going to start the problem.
We're going to start over here.
I'm actually going to start this from a reverse direction.
And review really quick what a binary phase diagram is.
because this is actually the phase diagram which you will see most often as you progress in your career as a scientist or an engineer.
This is something you'll see a lot of the time.
It'll have either one element and another element.
Or one compound and another compound along the axis.
And it shows the different phases which exist at certain temperatures and compositions.
So in general it's important to sort of understand what a phase is.
So a phase, is a chemically and a structurally homogeneous material given certain state conditions.
So a phase diagram depicts picks all these phases.
It's the master plot of phases that are in equilibrium given a particular set of state conditions.
When I say state conditions I'm actually referring to things like temperature, pressure, I even sometimes consider composition to be one of those state conditions.
So in a binary phase diagram, what we're looking at is the temperature versus the composition.
So if you have some composition of silicon and germanium at some temperature, you know exactly what phase it should be at thermodynamically.
It doesn't actually mean it will be, but it should be.
So that's what a binary phase diagram looks like.
So this is something you'll see a lot and this is something you probably will become used to seeing a lot as you go on in your years.
We're going to come back to this in a second.
But first, for now, we're going to it actually attempt the problem that's given.
And then I'll tie both of those together for you.
So let's go over and let's actually look at the problem now.
We'll go back to that in a second.
We're asked to draw a unary phase diagram for silicon.
And we're given some critical information.
Specifically on the problem we're given the triple point, we're given the critical point, and we're asked to draw this.
Not to scale, of course, but what it should generally look like.
So the first thing I do is label my axes.
Pressure, temperature.
And then we're given certain data.
So I'm just going to throw it down right in the beginning.
We're given pressure and temperature for triple point.
We're given pressure and temperature for the critical point.
So I'm going to put those points down right now.
Because I know that's part of the answer.
So let me first put down the triple point.
We're told the triple point occurs at 0.15 atm.
And we're told that that is at a temperature of 1415 celsius.
So temperatures are going to be in celsius and our pressures are going to be in atmospheres.
So we're going to go up to find this point.
In fact I'm going to give it a different color.
That is our triple point.
I'm next going to plot the critical point here.
We're told that that occurs at a pressure of 6,600.
So not to scale of course.
But that's going to be somewhere all the way up here.
Now we're told that that is at a temperature of 4880.
So here's a point here.
So we've just put two points down.
But that's not enough to complete the problem.
It doesn't actually answer all the questions.
The problem actually also asks us to provide the normal boiling point and the normal melting point of silicon.
So you're not actually given those temperatures because you've got to be clever.
And all the students in our class are given a periodic table during the exam.
So I would encourage you to have a periodic table with this type of data.
So if you look at the periodic table for silicon you can then find your melting point and you can find your boiling point.
So this data, a normal melting point and a normal boiling point will occur at 1 atmosphere.
Let me just put 1 atmosphere on the pressure here.
Again not to scale.
So anything that occurs at this particular pressure, this isobaric line, is going to be normal.
So our normal melting point occurs at 1414, which is right next to 1415.
There we go.
And our normal boiling point occurs at 3265.
These are approximate numbers.
So we'll put that here.
And now if you've seen a unary phase diagram before, which I hope you have during your studying, you'll know that it looks somewhat like a y shape.
So one thing we're going to do now is we're going to connect these dots.
We know that at very high temperatures we're going to have what's a gas, basically.
And at very low temperatures we're going to have something like a solid.
And we have some liquid in the middle.
And the way we're going to delineate these phases is to connect these dots.
And this is what our unary phase diagram is going to look like.
So in order to complete this problem-- it was actually not too bad of a problem-- all you have to do is label some particular aspects of this graph.
So number 1, we have a solid region, we have a gas region.
That implies this must be the liquid region.
I'll put an L there.
You're asked to identify 1, 2 and 3 component regions.
Or three 1-phase regions, 2-phase regions.
So number 1, you could have chosen for a 1-phase region, you can chosen a point here in the gas, a point here in the liquid, or a point here in the solid.
I'll just choose one in the gas.
So something here, let's call that i, for answering part i.
You could have chosen a point along any of these lines for a 2-phase equilibrium.
Which implies that at that point, so at that specific pressure and at that specific temperature, you have both phases of equilibrium.
Let me just choose one and explain it.
So let's choose something here.
Between 0.15 and 1.
So let's choose this point.
There we are.
So at this point, at whatever pressure we're at and whatever temperature we're at here, we have both liquid and gas in equilibrium.
They both coexist.
You're not tending towards one or the other.
They're both there.
So that's part B, the 2-phase coexistence area.
So we put that here.
i i.
And the third part is the 3-phase coexistence, which is-- almost you can pull it out of the name-- the triple point.
At the triple point-- and there's only one of these-- you have all three, gas, liquid and solid coexisting together.
So we've already sort of made that.
That was our blue dot.
1, 2, 3.
That's that point right there.
So not to scale.
But this is the answer.
This is full credit.
One thing that's really interesting to note is that on this particular answer we have a negative slope on our solid-liquid coexistence line.
Now you've probably seen this before in your studying and it occurred with water.
Most materials actually will have a positive slope.
If you think about it it's sort of counterintuitive.
What this implies is that if I'm at a constant temperature and as I raise my pressure I'm going to go from the solid to the liquid.
Intuitively, we would think that the solid is more densely packed and therefore as we increase the pressure we're going to tend towards a solid.
That's not the case in water and in this problem is not the case with the data we're given for silicon.
In this case as we raise the pressure we're actually turning into a liquid.
It's kind of counterintuitive.
Another way of actually saying that is the liquid phase at this particular temperature is more dense than the solid phase.
And you see that with water, which is why ice floats on top of water.
So that is our unary diagram.
That is the full credit on this problem.
I do want to tie it in really quick though, with what a binary diagram is.
Because this is a very important concept to get and a potential problem.
So we're going to go back over to binary phase diagram over here.
And so what we're looking at on this binary phase diagram is actually, it's the temperature versus the composition at a specific pressure.
Now for a binary phase diagram you're going to assume-- unless you're told otherwise-- that it occurs at 1 atm, 1 atmosphere.
If I want to go between the binary and the unary that we just looked at, you need to add your third axis.
So if we had a third axis-- let me draw that here.
This is going to be our z-axis.
There's the z.
We know that if we put a pressure along this axis, we're going to be able to extract what we had for our unary diagram.
So let me do that for you.
I'm going to call this the pressure axis.
Going back in and out of the board.
And actually this is at 1 atm.
Which implies at some point farther back we have the origin.
You can just do this lightly so it doesn't get too confusing.
We have an origin for our pressure.
So at this point we have 0 pressure.
We don't work with negative pressures.
0 pressure.
Which implies-- let me just draw the Cartesian form of this-- so we have composition, we have temperature, we have pressure.
That's our x, y, z.
Now what happens is that as we increase pressure what happens to, for example, this point?
Now what is this point?
This point is the point where we go from the solid to the liquid.
That's the melting point.
What happens to the melting point as we increase or decrease our pressure?
Well that information is given on our unary phase diagram.
Let's go back really quickly.
As we increase our pressure we can see that what's going to happen with our point is it's going to go up.
So what basically we have to do is we need to take-- you have to think three-dimensionally-- and take this graph and you're going to take it an spin it and put it on to one of those planes there.
So we're going to get an idea.
This is something that I would challenge you, the listener, to do and think about and do on your paper after you've completed this problem successfully.
I'm going to draw it right now.
And I'm going to let you look at it and think about it, what it means.
So let me do that.
Let's take a look now.
This is our temperature axis.
Here's t. Here's p. Remember unary was p versus t, so you've got to do a little flipping.
But let me just draw it now.
It kind of goes up to the words.
But you get the idea.
This is-- if you do a little flipping and I challenge you to do that, some three-dimensional thinking-- this is the unary phase plot that we gave for our answer.
And the point I'm trying to make is that you can go in between your unary and your binary diagrams by understanding that.
For example, binary diagram is at a constant pressure and your unary diagram, unary means that it has one material, one composition.
So in this particular problem I've created germanium to be our b element.
And we're operating here with pure silicon.
So you can go in between if you have the grasp of that concept.
So I would challenge you to think about that.
If you've gotten the problem right and you understand this concept, I think you're good to go on phase diagrams.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, welcome back.
We're going to be looking at problem number 11 in the fall 2009 final exam.
This is one of those problems on the exam that kind of tests a whole bunch of other stuff we couldn't get into one big question.
So it's looking at a bunch of different areas.
But the main themes are going to be bonding and reaction rates and half lives.
So the things I think you need to know, the W I N list as I usually call it, you need to understand reaction rate orders.
What they mean, how they apply to reactions.
What a half life is and how you use that.
And the third thing is melting points and how they are related to intermolecular forces and how intermolecular forces affect melting points.
So review these three things and once you've got them under your belt then let's attempt the problem.
So let's start the problem.
The first part is asking us to determine the half life for a particular reaction.
So there's two places you could start.
We saw both answers.
You could start with the definition of half life.
If you've got it down on your equation sheet, you could just say, well I know this is the equation which is the natural log of the starting concentration-- c naught-- divided by the current concentration that you're looking at-- c-- equals kt, where t is time and k is your constant for k. Or your reaction rate constant.
Or-- and this is the place that I would recommend starting because it's universally useful to use for doing kinetics problems and also for doing half life decays-- you start with your rate equation.
This is a very general rate equation, which I'm going to show you how it can be transformed to solve this very simple problem.
We're told here that we're looking at the half life of a particular reagent, it's not listed.
And we're also told that it falls by factor of 11 in 11 minutes.
Because we're looking at one material, we don't need two different items.
So we're just going to look at a.
Let's forget about b.
There's no b. OK, we're just looking at how a changes over time.
We're also told that it's first order.
So we're given this information in the problem.
First order means that you have an n equal to 1.
So n equals 1, which turns this equation into k times the concentration of species a.
We also know that rate is nothing more than the changing concentration with respect to the change in time.
So in calculus that would be da / dt.
So looking at this now, we've already got a differential equation, which we can solve and given certain boundary conditions we can apply as a naught as our starting concentration, we can create a new equation.
So the way we're going to do this is we're going to do the calculus-- which I'm not going to do for you here-- it's very simply you're going to find this equation.
This is all coming from the very general rate equation that we started with.
So there we are.
That's where we're at and you'll notice if I bring this a to the other side, I take the natural log, I put a negative sign in there, I return back to my original equation that I sort of luckily-- or didn't luckily-- have on my equation sheet.
So now we've got this equation.
And now we can actually go ahead and solve the first part of the problem very easily.
So I'm going to move over here.
We're going to start off.
We're going to say we're told that in 11 minutes we have now 1/11 of the concentration we originally had.
So let me write that mathematically for you.
I'm going to write a naught as my starting concentration of a.
So in 11 minutes we're going to have 1/11 a naught a naught e to the negative k. And that occurs in 11 minutes.
Just express the statement mathematically.
We've can cancel out these a naughts so they don't matter.
Which is good because we don't know what it is in the first place.
And now we can rearrange this equation pretty simply.
We can just do it as, solve for k, our reaction rate constant.
That's going to be negative 1/11, natural log of 1/11.
That's k. And now we have a value for k. Now that we know the value for k we can go back and find the half life.
And half life is simply defined very, very similar to how we approached the problem to begin with.
Half life is how long it takes for the concentration to drop from its original value to half of that original value.
So let me write that now in mathematical terms.
In this problem we know k now but we don't know the half life.
We don't know t 1/2.
I'm going to call it t 1/2 to designate it and to mark it differently than before.
Again we can cancel these off.
And now we can solve.
Because we know k. So let's plug in everything.
We can reverse everything.
So let me do that for you.
We're going to have this.
Natural log of 1/2 being equal to negative k, t to the 1/2.
And now we can solve for our t 1/2 because we know that k-- let's bring k to the other side, and the negative sign as well-- k is going to be negative 1/11 natural log 1/11.
We can then change natural log 1/2 into natural log 2.
That actually looks like our original half life equation if you had that on your equation sheet.
So when you solve this out, you're solving for t 1/2, t 1/2 the long and the short of it is 3.18 minutes.
The main takeaway from this problem is that you don't need to have memorized any number of equations to do it.
You can start from a very general equation, which is the rate equation, which we showed in the beginning, and go from there using logic and then just mathematically represent what the problem is telling you to do in the first place.
So that's part A and that's the majority of the points on this problem.
Moving over, part B and C are very quick, very short.
Just to check if you know the facts.
Let's do them right now.
We're asking you-- we're going to do B 1 and B 2 together-- to tell us which one of the compounds has a higher boiling point.
And we give you two different cases.
In each case there's two compounds.
So number 1, you're given silene and phosphene.
So the first thing you should do is actually draw those molecules out.
So that's what I did.
Silene looks like this.
And then phosphene-- put the bonds in there-- phosphene looks something like this.
So now we have to sort of identify which one of these has a higher melting point.
Now in order to do that you need to have reviewed and know that melting point is related to the intermolecular forces between the between the molecules.
The forces between the molecules are dependent on both their geometry and their chemistry.
So knowing that, if something has stronger intermolecular bonds, it'll have a higher melting point.
So let's take a look at these two molecules.
Which one has stronger intermolecular bonds?
We have in silene case, we have van der Waals forces, they're generally very weak.
We've got those tpo for the phosphene.
But in phosphene we additionally have a dipole.
We have partial-- we put maybe delta minus here and delta plus here and here on this side.
So our molecule is actually polarized.
There's a plus region and a minus region.
And because of that you have much stronger intermolecular forces between the phosphenes then you do the silenes.
And because these have stronger intermolecular forces, this one has a higher melting point.
The last part of the problem, part 2.
We try to throw a little trick in there with you.
We look at ammonia and phosphene.
So now we're doing NH3 and PH3.
And they actually both look identical in terms of their geometry.
So let me do that now.
It actually looks tetragonal if you were to draw the structure out.
There's ammonia and here's our phosphene again.
You can tell they look the same.
So I think you have to sort of reason out which one has the higher melting point, which is the same thing as saying which one has stronger intermolecular forces.
A very common answer we got was that the nitrogen is a smaller atom, therefore everything is held more closely, therefore the forces are stronger.
That's not really the answer that is most dominant and it's also not the answer we were looking for.
The correct answer here is that even though both of these have the same-- probably very similar-- van der Waals forces, they're both polarized molecules, nitrogen has hydrogen bonding.
So remember the hydrogen bonding elements you should probably know are fluorine, oxygen, chlorine, nitrogen, four of them.
So nitrogen will hydrogen bond, which means that this nitrogen will have a hydrogen bond with some other ammonia molecule over here.
This H might be bonded to an N somewhere else.
And because there are additional hydrogen bonding force in the system, the intermolecular forces are stronger and more prevalent in ammonia, which means that ammonia will have a higher boiling point, which is true.
The answer here is ammonia.
So all you have to take way from this is that the mechanical properties as well as the melting and boiling properties of a material are very much governed by the intermolecular forces that exist in that particular material, that the liquid or the solid state.
And so in this case we're looking at these compounds and we've identified this one as being a higher melting point because it has a polarity whereas this one doesn't.
And in this case where we try to trick you up, we identified that nitrogen has this additional hydrogen bonding force.
And that's all there is to it.
The following content is provided under a Creative Commons License.
Your support will help MIT Open Courseware continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Open Courseware at ocw.mit.edu.
Hi.
I'm Brian.
We're going to be going over problem number 13 of the fall 2009 final exam.
I like to think of the theme of this problem as polymers.
So that's the overarching concept.
But I'm going to give you some things I think are important to review and to know before actually meaningfully attempting this problem.
So there's three things that I that I sort of thought were very important.
The first one is polymerization processes.
So we learned in class and in the book as well that there's two ways that we've covered that you can create a polymer.
And that is through either addition polymerization or condensation polymerization.
And I'll talk about that more in a second.
The second thing you want to know is basically molecular weight and how it applies to polymers.
And you want to know about elastomers for this problem.
If you don't know about elastomers then you'll have trouble with the last part of the problem.
So that's kind of what I would review.
And then once you've got that under your belt give the problem a try.
So we're going to go and we're going to start the problem now.
We're given in the problem the 6-aminohexanoic acid, which I've drawn here.
And we're also given at the beginning of the problem the structure of nylon 6.
This is the actual reaction that's used to create nylon.
So what I'm going to do is I'm going to show you-- or ask the question of how do we create nylon from 6-aminohexanoic acid.
And that's actually going to answer part A and part B to this problem.
So to create a polymer-- in this case we're talking about the polymer nylon 6-- you actually have to add together many mers.
So merge is the term we used to denote the single unit which is repeated n times to create the poly mer-- polymer.
So in this case the mer, the individual unit of nylon 6, is 6-aminohexanoic acid.
And that's this.
That's the same thing right there.
So we're going to add these together and we're going to start creating our long chain of nylon 6.
Remember, nylon 6 a polymer is a really, really long chain.
Think of it as spaghetti, very long spaghetti.
It isn't just two molecules that react.
So let's just start somewhere.
It's got to begin somewhere.
You have to have two molecules at some point in the beginning reacting with each other.
So let's look at our two molecules of 6-aminohexanoic acid-- I have to keep looking back at how to say that-- so let's take a look and we have to think about now our two possible processes or routes towards polymerization.
There's addition and there's condensation.
Now addition, as you probably read and heard, requires there to be a double bond present between two carbon atoms.
The reason for this is because usually have a free radical coming in, which will then initiate.
That's why oftentimes the free radical molecule or free radical provider is called the initiator.
You need an initiator to come in to break that double bond and then create another free radical.
Which then lets the process continue along in a chain like that.
If we look at our molecule here, we don't actually have any double bonds between two carbon atoms.
We have a double bond with an oxygen but that's not sufficient for an addition-type reaction.
For a condensation reaction, what that basically implies is two mer units, or two things coming together to react and they give off a byproduct.
So that's how I remember condensation, very often the byproduct is water.
You can have other byproducts, like HCl or any other sort of small molecules can be given off.
But most times what we'll see in 3091 will be water given off in condensation, so it's sort of intuitive to call it condensation.
In this problem, if we look at this, we have a molecule that has the Hs on one side and has Os on the other side, we can actually almost see this immediately as being a condensation reaction.
What's going to happen is we're going to have this O reacting with the positive end of this molecule, which has Hs.
And we're going to have an H2O liberated and given off.
We're going to have the new molecule formed.
Let me draw the molecule for you.
Let me write it out.
So we've just begun forming our polymer and this is a polymer which has an n of 2.
It has 2 mer units building it.
Notice I've kept the ends here.
The positive and the negative here.
In reality what's going to happen is we have a big vat of this 6-aminohexanoic acid and they're going to keep reacting.
So we might have another one of these molecules float over and react with this O.
Or conversely, we could have another one of these molecules come over and react with the H. So this thing is going to keep building.
And the way these polymerization processes begin is the control of some parameter of the system.
It could be temperature, pressure you can add something in to sort of initiate it.
But that's how this is going to start.
So we've actually sort of answered part a and b.
We've come to the conclusion that this is obviously a condensation reaction.
Because we're going to give off-- I left that out there, see if you guys caught it-- H2O.
So we're losing this O and two of these Hs and we're forming water.
And that's the byproduct.
And a byproduct immediately should tell you we've got condensation going on.
So condensation polymerization, part A.
Part B, we've actually already done to sort of logic out the answer to part A.
This is the answer to part B.
Part B is just the reaction.
I'll read it very specifically.
Write the reaction that converts two molecules of 6-aminohexanoic acid into a dimer of nylon 6.
So we have two molecules forming the dimer.
You are going to have trimer, and it goes on and on.
The most common mistake on this problem was to give us the full repeating units.
Let me just show you what that would look like.
And then we'll move on with the problem.
So the most common mistake on this problem was to not have these end units.
And instead say, oh it's created this huge, long chain.
So I'm going to write it as follows.
And this is the symbol to imply that it repeats.
Likewise over here.
So people would give us this.
And that's not correct.
And the reason that's a problem is because not only do you lose points on this part, you also lose points on the next part.
So part C is now asking us to calculate the molecular weight of a molecule given a certain n. n is the degree of polymerization.
So in this case, n equals 2, we've got a dimer.
In part C, we have n equals-- surprisingly-- 3,091 You'll see that's a recurring theme in this class.
So we have n equals 3,091 And the question is, how do we approach the molecular weight problem?
Well we're told that we have 3,091 one of these mer units making up this polymer chain.
So let me just write this down.
We're going to form an equation.
If we take the mass of the full chain and divide it by the mass of a single mer unit we should be able to extract the number of units in the chain.
So let me write that out for you.
So the total molecular weight of the molecule, divided by the weight of a particular mer unit.
OK so we're going to put that as mer.
And so this is easy.
Let's first identify what our mer unit is.
And this is why it was easy get tripped up on this problem.
Because if you made a mistake on the part B, you have trouble on part C. What is our mer unit?
So in our mer unit, what we're talking about is a chain which has 3,091 of these linked pieces here.
So let's look at this.
What do we have?
Here's our mer unit.
Let me circle it.
We have 6 carbons.
We have 11 hydrogens, 1 nitrogen and 1 oxygen in this mer unit.
We're not going to take the full, complete end unit.
We're not counting this O.
We're not counting those 2 Hs.
We're dropping off 18 grams per reaction into water.
So this is our mere unit.
Let's calculate the mass of our mer unit.
So it's going to be-- let me write this here-- we're looking for molecular weight.
We're going to divide that by the mass of the mer unit.
Let me just write out the full line of what it would be.
Approximately 6 times 12 plus 11 times 1.
So this is 6 carbons, times the mass of carbon.
11 hydrogens times the mass of hydrogen.
We're going to have 1 oxygen times the mass of oxygen.
And we're going to have the nitrogen as well.
So let me add that down here.
1 nitrogen times the mass of nitrogen.
I rounded.
That's OK. As long as you get the idea right.
So now we can easily just solve for the molecular weight.
And we'll find that molecular weight in this problem will be equal to 3.5 times 10 to the fifth grams per mole.
There's another way to do this problem, actually.
And what you can do is you can calculate n times the number of this-- this particular molecule-- and you can subtract off 3,090 orders.
18 grams.
3,090 times eighteen.
You subtract that off.
Think about a little bit why it's 3,090 and not 3,091.
So that's just maybe a brain teaser for you.
That is the answer to part C. And now part D asks us if we're able to convert this into an elastomer.
So this is where it's really required that you understand what an elastomer is and what that implies and what the structure looks like.
So an elastomer-- and I sort of paraphrased this from the book and added my own thing to it-- is basically a randomly oriented amorphous polymer with some cross-linking such that you have the ability to move the chains but not slide them completely over each other.
So in order to have an elastomer you must have cross-linking.
In order to have cross-linking, what must you have?
Well from lecture and from the book we know that to cross-link you must have double bonds in your carbons.
So you must have 2 carbons-- going back over here-- which are double-bonded together.
And why is that?
Because what happens is those double bonds will be broken by some initiator element-- many times we'll have sulfur-- will come in, break the double bond and then you'll have links forming between things.
So let me actually give you real-life example of this so you can see what I'm talking about.
We're going to show disulfide bridges bounding and creating cross-links in a molecule.
Something we're very familiar with is rubber.
And we're also very familiar with car tires.
The big difference between those two things is that rubber has vulcanized Which means we're actually putting sulphur into it to create a different type of polymer.
We're looking at a cross-linked, much stronger rubber which we use on our car tires.
So polyisoprene looks something like this.
You should be familiar with this notation by now.
We have this denotes it repeats.
We have our double bonds, we have our carbons and hydrogens.
This is polyisoprene.
And this is what we're going to add sulphur to vulcanize rubber.
We're going to create vulcanized rubber.
sulphur generally actually looks, it's a ring of 8 sulphur atoms.
We'll then add it to rubber, maybe heat it up mixed.
And you'll make what you're used to on car tires.
So what's actually happening here is that you can see we have this double bond between carbons.
The sulphur's going to come in, it's going to react with this double bond, and is actually going to pull it off.
And it might form a chain like that.
And then this sulfur will then connect to another one of these polyisoprene molecules that was originally double bonded.
And I'm not going to draw the rest of it out.
And it's going to react, kill this double bond, and it's going to connect it.
So basically what's happened in this process of forming sulfide bridges is you've got this polymer, polyisoprene, connecting to this polymer, polyisoprene, and you have cross-linking going on.
And that's what we would need to have in our case for the nylon 6 in order to get an elastomer, And because nylon 6, because our mer unit, 6-aminohexanoic acid doesn't have double bonds, we can't break them.
We can't network.
We can't cross-link.
And therefore we can't make an elastomer.
So the answer to this problem-- in a very long-winded manner-- is no.
You cannot create an elastomer given that mer unit.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
We, we are solid state devices.
This is soft matter.
My eyes are photo detectors.
Band gap of about two electron volts.
They're just not made of a compound semiconductor inorganic.
And this motion is just conformational changes in a polymer.
See it's all chemistry.
The rest is stamp collecting.
So the principles that apply to inanimate objects apply to animate objects.
And the solid-state chemistry umbrella is very broad.
If I say solid-state chemistry, people are apt to think of electronic devices, transistors, what have you.
But that extends to the life sciences.
I mean we, as human beings, are chemical machines.
So this is the classical view.
Electronic structuring informs bonding but then in 3.091 bonding informs atomic arrangement, which tells you everything.
The shape of all things, it's all electronic structure.
There's nothing more.
The difference is that the focus in the chemistry department, their focus is the molecule.
Whereas in 3.091 I like to think of the focus being aggregates of molecules, namely solids.
Because engineering systems are made of solids.
If you're steeped in chemistry from high school, you've had the AP curriculum, again 3.091 might be something that is a fresh look at chemistry from a slightly different perspective.
What I wanted to play for you as you're leaving today is an old Beatles song, Lucy in the Sky with Diamonds.
Being able to bring in subject matter, music, art, literature, into a chemistry class and make the connections to the world around us, I think is most important in the 21st century, when problem solving even in the technical domain is going to require skills that build bridges to the humanities, arts and social sciences.
And so I like to think of 3.091 not as a chemistry class, but rather a chemistry-centered class.
The majority of students taking 3.091 at MIT may never take another chemistry class.
So I ask myself what topics are critical for someone who might major in electrical engineering?
Or for someone who might major in physics?
Or what's the chemistry that's critical to an MIT student who might major in political science or economics?
And that informs my choice of topics.
And it turns out that that doesn't diminish from the education of the student who may choose to major in a chemically intensive discipline such as material science, chemical engineering, or even chemistry itself.
I don't view the presentation of the subject matter as a linear track, where this is the beginning and this is the end.
I view it more along the lines of a spiral.
And so the first time you encounter a certain concept it's at a rather superficial introductory level.
And then at some point later in the semester you spiral back and see it again.
And then maybe there's a little more context and a little more intensity.
Maybe there's a little more mathematics associated with it.
And then you'll see it applied later again.
And through that revisiting-- repetition by design-- I think that key concepts get driven home.
Some people are shocked that I stand in front of a chalk board and I write on it and I believe that that's still a valid means of teaching.
For the first-time learner I think it's important to activate all of the learning channels.
The act of drawing and inspiring them to draw is important.
And maybe the person at home ought to put themselves in a chair in the auditorium and be taking notes as though they are attending the lecture.
Watching those videos, especially for someone that has had a fair bit of high school chemistry, you're going to hear a lot of the same things but maybe coming at you at a slightly elevated level.
But not so elevated that you won't follow it.
And I think that's good for a high school student to be talked to at a higher level.
I think the videos also have utility for people that are interested in specific topics.
I mean for example, if somebody needs to know something about x-rays, or the use of x-rays, dropping in to that three-lecture sequence on x-rays could be quite useful.
I think it's possible to sample with a goal of mastering a particular concept.
So clearly for the learner out in the World Wide Web, going to recitation and interacting with a recitation instructor is not practical.
So in lieu of the recitation section OCW has created these help session videos.
We actually hired some recitation instructors from the fall of 2009 and asked them to work through some sample problems and what you see in the help session video is essentially what would have happened in a recitation section.
I issue the homework questions with model solutions at the beginning of the unit.
So the homework becomes a study aid.
And I recommend to students that they try the homework problems without referring to the answer key initially.
And to turn to the answer key when they reach an impediment.
What's the message of 3.091 for the man on the street?
An understanding of those relationships.
Electronic structure to bonding.
Bonding to atomic arrangement.
Atomic arrangement to properties.
Is the key to understanding the world around us.
And in particular it's the key to fueling invention.
And I believe in practice very much and in the application of science in service for humanity.
Why do we worry about designing a car that's a zero-emission vehicle, as if to do no harm to the environment?
Why don't we go even further and say, what about a car that purifies the air as it drives down the road?
So that the air in the wake of the car is cleaner than the air in front of the car?
And I'm finding that increasingly that's a message that resonates with the students.
They want to take the tools of technical analytically centered education and put them to work to solve societal problems.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started.
Settle down.
Settle down.
Welcome to 3.091.
I'm Donald Sadoway and I'm going to be your instructor this semester.
So what I want to do today is to introduce my plans.
I've got plans for you.
Plans for learning.
I want to talk a little bit today about those plans, introduce myself, introduce my class curriculum-- the path forward.
So let me begin by saying that 3.091 is the most important class you will take at MIT.
It's true.
But, you know, anybody who stands in front of you to lecture should say the same thing about his or her class.
If they don't believe that they shouldn't be standing in front of you lecturing.
The difference is when I say it, I'm right.
[LAUGHTER]
And it's not just an opinion, it's because of the content of 3.091.
3.091 is about chemistry.
It's about the central science, but it's not just about any old chemistry.
This isn't a class that you could take anywhere else in this country.
This is the only place that you can take 3.091.
3.091 is solid state chemistry.
Now why do we talk about solid state chemistry?
Because engineering systems are made of solids.
Now I know what you're thinking-- oh, he said solid state, he's going to talk about the chemistry that constitutes his laptop computer or the chemistry that constitutes this laser pointer.
And that's true.
We will talk about that chemistry.
But we will also talk about soft matter.
We as human beings are chemical machines.
When this hand changes shape, it is a polymer that is changing conformality.
These eyes are photodetectors, band gap of about two electron volts.
They're not made of gallium nitride.
They're made of organic compounds.
Inside, what supports us, it's a ceramic skeleton.
So solid state chemistry describes life science as well.
Well, what we're going to learn in 3.091 is the rudiments.
We're going to learn the rudiments.
So what I'm going to do today is now go through some of the basic organization.
So tomorrow you're going to meet your recitation instructors and get to know each other and get to know the recitation instructor.
Today I'll say a few words about myself so you know who's standing in front of you.
I graduated from the University of Toronto in chemical metallurgy, came down here in 1977 to spend a year as a postdoctoral fellow and, I guess, I lost track of time.
Now what's my research?
My research is in electrochemistry.
Electrochemistry is the most important branch of chemistry.
Do you notice some theme in my professional life?
See, I have tenure.
So what does that mean?
It means you find your passion and pursue it.
You don't waste time on trivia, all right?
And that's what I urge you to do: Find your passion and pursue it.
So what's my passion?
My passion is electrochemistry in nonaqueous media.
Anything but water.
Let the rest of the world work on water.
I work on molten salts, ionic liquids, and polymers.
And what's the reason for this?
There's an application.
I'm in the School of Engineering.
So I'm interested in environmentally sound technologies for metal production, all right?
Right now looking at titanium, iron recycling as well.
I'm also interested in electrochemistry as it applies to energy storage, energy storage for mobile power.
See this gal here?
She's got the cell phone.
She's not even looking where she's driving.
[LAUGHTER]
So making safe batteries out of earth-abundant, accessible materials for portable power, ultimately to drive the car with electrons, electric fuel, to eliminate this country's dependence on imported petroleum.
We can do it.
How?
By inventing.
By inventing.
And the way we're going to invent is to learn the lessons in 3.091 that will give us the chemistry we need to invent a battery that can send that car 250 miles on a single charge, and put it in a show room for the same price as a car with an internal combustion engine.
The only thing that stands between that image and where we are today is invention, and the requisite material is right here in this class.
We're also looking at colossal batteries.
Store the grid.
Enable renewables, wind, solar.
And then, lastly, let's never forget about dreaming.
So if we want to dream and imagine that man will return to the moon or maybe even go to Mars, we're going to have to be able to produce oxygen.
We'll live like the pioneers, produce oxygen from local resources, produce structural metals, and even produce photovoltaic silicon so they can generate their own energy from local resources.
And so electrochemistry is the key enabler for that as well.
So now let's turn to the whole underpinning of 3.091.
If we take a look at the performance of any engineering system it's a combination of the design and the construction.
Now the construction is both the workmanship and the choice of materials.
Now how do we choose materials?
We choose them on the basis of their properties.
So, for example, here's an application: a beverage container.
This one I'm holding is made out of an aluminum alloy.
It's a metal.
When I was your age, they had steel beverage cans.
You can take this same beverage and contain it in a glass bottle.
You can contain it in a polymer bottle.
Why do we make those various choices?
Because they have the right mix of properties.
Now how do you determine the properties?
Properties obviously are determined by the composition.
We wouldn't make this thing out of sodium chloride.
It would dissolve.
So composition is important, but atomic arrangement is important as well.
Example is 1/2-inch-thick pine board compared to a 1/2-inch-thick piece of plywood.
Both pine, but the 1/2-inch-thick pine board is one solid pine board.
I can take that pine board and I can fracture it if I strike it along the grain.
But the 1/2-inch-thick plywood, I can't do that because it's 1/8-inch sheet pine cross-laminated.
1/8 inch north-south, 1/8 inch east-west.
If I try to cut through that board, I can't advance the fracture.
Same composition, different atomic arrangement.
So the thesis of 3.091 is that the electronic structure of the elements holds the key to understanding that relationship between composition, atomic arrangement, and properties.
And once you have properties that's how you make your selection.
And away we go.
And that's how we got the syllabus of 3.091.
So the syllabus has two major blocks.
The first block is general principles of chemistry-- it's going to be about the first five weeks-- and that's the same stuff that you get here, 5.111, 5.112, the basics of electronic structure and bonding.
And then we part company and in 3.091 we dig down into solid state chemistry.
You don't have to take notes.
You can relax.
All this is going to be burned to PDF and posted at the website.
So everything that you see in this class that goes up on the screen is archived for you.
This lecture is being video recorded right now and within an hour will be posted at the website.
So in the unlikely event that you can't quite come to class, it is available.
[LAUGHTER]
So now what I want to do is introduce some of the operational aspects of the class.
And we had some handouts going around.
If you didn't get them we can get you extras.
But again, this is all posted at the website.
So you know who I am.
The key person in this operation is my administrative assistant, Hilary Sheldon, and she's just down the hall.
You can roll a penny down the hall from here and get to my office.
So if you tell me you tried looking for me, you couldn't find me, I know you couldn't have been trying very hard.
The text is a two-volume set consisting of the text Chemistry: Principles, Patterns, and Applications by Averill and Eldredge, and then a second volume that consists of miscellaneous readings taken from other Prentice Hall texts.
And it's identical in content to the blue book that was used last year and the year before.
So if you get a copy of that blue book, you don't have to buy these.
I don't work for Prentice Hall, I'm not trying to sell books, but we will take the homeworks out of the books to a great extent.
And the page numbering is the same, so anything this year or the last two years will be fine.
If you have to buy something, you'll have this.
This one will come with a CD that will get you into a mastering chemistry, which is a sort of a tutorial, computerized system to help you with certain homework problems.
The lectures, Monday, Wednesday, Friday, here in this room at 11 o'clock.
The lecture starts at five minutes after the hour-- gives you five minutes to get in-- and then the lecture stops at five minutes to the next hour-- time for you to leave and then for the next class to arrive.
And what I'm going to try to do is establish some plug flow here.
So what I'm going to try is to have everybody leave by the exit to the north here to my right, and there's also an exit over the top back.
So leave to your left and that way it'll be easier for the next class to come in.
And maybe we can persuade the people here before us to do the same and then it'll be easy to change.
I don't know why but there's this sort of-- everybody has to charge to this door.
And I just stand here and watch people collide.
And I don't know.
Just interesting social experiment.
And there are two doors.
I don't understand why people do this, but I'm not in the social sciences.
Recitations.
Recitations will meet Tuesdays and Thursdays.
So you'll go to the same hour on Tuesday and Thursday.
The section should be roughly 20 students.
And that's where the question and answer occurs.
Here it's largely I talk, you listen.
I've got time for a question like, hey, shouldn't that be a minus sign?
I can take something really quick.
But where you get to really interact with the instructor is in recitation.
And I've given a direction to my recitation instructors not to give you a fourth and fifth lecture.
You control the content of the recitation.
So you have to come to the recitation prepared with questions.
So they're supposed to walk in and say, good morning, good afternoon, what are your questions?
And you can say, I didn't understand the last five minutes of the lecture yesterday.
Would you do number five on the homework?
Would you go over secondary bonding?
That's the sort of thing that's supposed to happen in section.
You've been assigned by the Registrar.
You are forbidden to change your section on your own.
You can't just squat in another section.
We do this with the intention of trying to keep the enrollment roughly uniform.
We don't want sections growing to 35 because then that limits your ability to interact.
If you've changed classes or you've picked up a UROP, or what have you, and your situation has changed, please go down the hall and meet with my assistant Hilary, who will then arrange to move you.
And that way we can keep some control over the section size.
Again, you may not change simply on your own.
And it has to be with cause, academic cause.
You can't go in and say, I was assigned to the 9:00 am section and I don't do mornings.
That's not going to work.
Homework.
Homework is very important here.
Homework is a little bit different, though, in 3.091.
You're not going to be asked to turn in homework for grading.
Instead, at the beginning of each unit, you will be given homework with the model solutions right at the beginning.
This is a study aid, and you can use the model solutions to help you understand the homework material.
Now since we're not asking you to turn anything in and we do want to stimulate interest in the homework, we found a way to stimulate interest.
And that is that once a week in section you will have a 10-minute quiz based on the homework.
That will ensure that you've at least looked at the homework.
And those weekly quizzes will be graded, and the aggregate of all of those quiz scores will constitute your homework grade in the subject.
And you must take those quizzes.
If you don't take the quiz-- you got sick or some personal emergency came up-- contact your recitation instructor and within a week of the date of the original quiz arrange to take a makeup.
If you don't take the makeup, we're going to give you a zero.
And there's no dropping of lowest score.
Somewhere there's this in the lore here that, oh, you can drop the lowest score, the lowest two scores.
People come to me with this proposal and I say no.
[LAUGHTER]
So let's get started.
This is homework number one.
It's assigned today.
We're going to save paper-- just go to the website-- and you'll be tested on Tuesday, next Tuesday, September 15, at the beginning of section.
At the beginning of section there'll be a 10-minute quiz based on that homework, and you can get the homework and so on.
By the way, here's what the homework looks like.
Chapter 1, Chapter 3 of Averill.
Averill is the major text.
We just refer to it by the author's last name.
So taken from Averill.
By the way, I don't like to use this unpleasant term "test."
I like to refer to it as a celebration of learning.
[LAUGHTER]
So it's a celebration.
We're going to start celebrating on Tuesday.
There's the website by the way.
You probably want to bookmark that.
You're probably going to go to that a lot.
We're going to keep celebrating, and so we will have some monthly celebrations, monthly tests, and they've already been scheduled.
And you will write those tests during the normal class time.
But on those days we will spread you out a bit.
I like to see some vacancies.
So you will not be sitting one next to another.
There'll be at least one human vacancy next to every human, and that way you've got room to spread out, keep your eyes on your own work.
So those are the dates and there'll be more of that in time.
Now when you take the monthly test, I allow you to use aids.
So everyone will be given in recitation tomorrow a Periodic Table of the Elements, very nice one, laminated one.
So you take that with you to the test.
In fact, you should take this with you everywhere.
[LAUGHTER]
If I run into you at Harvard Square, I want to see the Periodic Table, because every educated person has the Periodic Table.
You get a table of constants.
So you'll have one of these.
You'll get that tomorrow as well.
Paper copy, and on there are all the constants so you don't have to remember that the permittivity of vacuum is 8.85 times 10 to the minus 12 farads per meter.
It's on there.
I urge you to use that when you do your homework.
So it's always amusing to me during the time of the first monthly celebration, somebody's looking at this as though it's today's newspaper and they're looking, where is that?
Sort of revealing about the intensity with which the homework has been embraced.
Calculator, something to calculate with, and I don't care if you bring in a graphing calculator or a mainframe.
I don't care.
[LAUGHTER]
And you're allowed an aid sheet, an 8 1/2-by-11 sheet of paper.
And you can write on the front, you can write on the back.
You can photoreduce previous exams down to micro-dot size.
I don't care what you put on it, but with this, you have then no excuse to say I really understood this stuff, but I couldn't remember a formula.
Actually, many students tell me that the act of preparing the aid sheet organizes the subject matter in their minds.
They bring this with them to the exam, and they never consult it, but it just soothes the nerves and makes sure that everybody does well on the monthly test.
The weekly test, no.
The weekly test you bring your Periodic Table and table of constants, but no aid sheet.
For the weekly test, it's a very concentrated amount of material.
And I'm not going to test your memories, because that doesn't prove anything to me.
And then, of course, the celebration of celebrations is the final exam.
[LAUGHTER]
That's huge.
That's a huge celebration.
It's three hours.
See, these are really 50 minutes.
This is three full hours.
The time and location will be set by the Registrar.
We should know by October 1.
The final exam period is December 14 to 18.
And so what I urge you to do is as soon as that schedule comes out and we know when all the final exams are going to be held, then you book your passage for the holidays.
Do not get that order reversed.
You cannot come to me and say, I got a really good price on a ticket.
I'm going Acapulco on the 15th of December, and your exam is on the 16th of December.
I'll say, you just got a zero on the final.
You have to be here for that.
You know why?
I have to be here for that, you can be here for that, too.
There are about a quarter of a million students in Boston.
It's a great college town, but at Christmas time, it's pandemonium at Logan Airport.
So you want to book your passage early, but you can't do it until you know what your final exam obligations are.
Grading.
Freshman, you know, it's pass/no record.
Pass/no record.
And so that means that if you struggle and things don't go well, you don't have any blemishes on your record.
Unfortunately, some of the upperclassmen will tell you as pass/no record, you now, barest pass imaginable/ no record.
Well, I hate to let you know this, but increasingly I am being asked by medical schools, law schools, scholarship providers to reveal the scores on the freshman year.
So think about that before you call Hilary in late November saying, what do I need to get a 50?
I need a 32 on the final?
OK.
But Lord help you, you get a 31 you go down.
[LAUGHTER]
Upperclassmen get the luxury of the entire alphabet: A, B, C, D, F. The final grade composition.
1/6 for homework-- that's the aggregates of the weekly test scores-- and 1/6 for each of the three tests.
And then the final is 2/6, or 1/3.
I didn't want to get into transcendental numbers, so I made it 33 exact, 16.75 exact.
I could have made it 16.77, I could have made it 16.81, I could have made it anything I want.
I'm the professor.
But I chose 16.75.
Bottom line here is it's really dumb to fail the final.
If you passed the final, you're pretty much assured that we're going to be favorably disposed to passing you.
But you fail the final that says two things.
It says, number one, you don't have a grasp of the overall year, but it also sort of indicates that you ran out of steam partway through the semester and stopped working.
It's very bad.
You get a lot of good advice from upperclassmen, but sometimes-- I'd ignore that one that says, hey, you don't have to do that well on the final.
Anyway, so you have to get a C level as a freshmen or greater.
And, by the way, we do not grade on a curve.
I've seen it in magazines this last year that MIT grades on a curve.
I don't where they get that from.
I don't grade on a curve.
Your success does not come at the expense of your neighbor.
As far as I'm concerned everybody in this class can get an A.
Again, I'm the professor, all right?
So you say, well, how do you know that 50 is the right number?
Why isn't it 55?
Why isn't it 75?
Well, I know.
I know.
How do I know?
Because when we grade, we set up the point scheme so that if the student has the mastery of the barest level of competency of the key concept the point scheme must reflect a passing grade-- 5 out of 10, 5 out of 9-- and you propagate that through.
I don't care how much is written, if it doesn't demonstrate basic mastery of the key concept the point scheme must give 4, 3, 2, 0.
Maybe a 1.
And so that, if you propagate it through the whole semester, means 50 is a pass.
There's some wiggle room there.
How do I know 49.7 is a fail and 50 is a pass?
That's when I call your recitation instructor.
[CELL PHONE RINGING]
Let's kill that.
We're going to get to that in a second.
Let's call the recitation instructor.
And what happens?
I ask the recitation instructor, well, what can you tell me about this young lady?
And, oh, she came to all my classes, she tried really hard, she came to office hours.
I don't think the exam is a proper reflection of her understanding.
I'll listen to that and maybe we'll promote.
If, on the other hand, I get the response, I never saw her, didn't come to class.
I repeatedly reached out to her, ignored my entreaties, she's going down.
[LAUGHTER]
OK. Website.
This is what the website looks like.
There's a number of tabs here.
The readings are there, the videos are there archived.
The schedule-- what's going to be coming up.
So, for example, this is what's going on today.
It says what the topic is, roughly what the readings are.
I know today you didn't come to class having read.
And it's OK, we'll get through it.
But from now on I urge you to do the readings.
A couple of topics I'm required to talk about.
My management requires me to do so.
Academic honesty.
There's a lot of texts here, but in plain English, this says don't cheat!
You know what this means!
Now I don't want to hear, well, in my country, the custom is-- I don't care what they do in your country!
You're in my class, and if you cheat in my class you will pay for it.
Very simple.
Accept information of any kind from others-- wrong.
Represent somebody else's work as your own-- wrong.
You know this.
It was Juvenal, the Roman politician, said men need not so much be instructed as reminded.
I'm not telling you anything you don't already know, but I'm going to say it so no one can say, well, nobody told us it wasn't OK to erase our answers and hand them back in for more points.
I just did.
All right?
So if in the unlikely-- but it happens every so often.
Maybe every two, three years.
And people get caught.
You know why they get caught?
Because for the first time in your life you're being taught by people who are as smart as you are.
[LAUGHTER]
My TAs are really smart.
And it never ceases to amaze me-- somebody succumbs and does something dishonest, and they get caught!
They get busted!
And what happens then?
Then I get angry.
You know why?
Because we can't settle that in my office.
You can't come and cry in my office.
I have to take this episode to a committee here at MIT called the Committee on Discipline.
It is staffed by faculty, by administrators, and by students.
And the case is brought before the COD at a hearing and a punishment is decided.
And it can be anything from suspension to expulsion.
And of the three categories, faculty, administrators, students, which category do you think is most severe on infringers?
Students
Your peers.
You got it.
You know, because people my age, old faculty, they'll be like, oh, they're just kids, whatever.
The students say, no!
Throw them out!
[LAUGHTER]
So it's not going to be me that's going to expel you, it's going to be your peers.
So don't do it.
And if you're ever in an exam and somebody's pressuring you, just raise your hand and ask to be reseated.
For all I know, the guy next to you has got a bad case of B.O., you just want to move.
We will not ask any questions, so take yourself out of the situation.
Conduct yourself appropriately.
Classroom behavior.
Now this is the first lecture, there's a whole bunch of violations in here right now.
So I'm going to say it now and we'll fix it for next time.
We've got 425 seats here, we've probably got 475 people, and there's only one way we can make this system work and we have to observe certain rules of decorum.
And I make the rules.
So if I want to maintain a fertile learning environment I'm going to ask for these rules to be observed.
No talking at all.
No talking.
Little conversation here, little conversation here-- it disturbs people.
I don't want any food or drink.
No food, no drink.
One exception is the professor.
[LAUGHTER]
Because I do not want to have my throat get so dry that I can't finish talking my way through the end of the class.
Otherwise, no food or drink and no disruptive behavior.
No horseplay or anything like that.
And wireless communication devices must be silenced.
Cell phone goes off, you get up, you leave the room.
That's it
Is water OK?
If you need water because it's some kind of a health thing, fine.
But I do not want to see a whole bunch of people drawing on water bottles.
Don't need it.
Don't need it.
You can go 50 minutes without your little-- whatever.
[LAUGHTER]
You know why?
I'll tell you why.
It's not because I'm trying to be a control freak.
This is a chemistry class, but it's chemistry-centered.
You didn't just come to MIT to learn some geeky techno stuff.
You're preparing for a professional career, and part of that is how you behave, how to act as a professional.
And you cannot learn behavior by doing problem sets.
How do you learn behavior?
You learn behavior by observation.
And how does one teach behavior?
By modeling.
If you're in some high-level committee meeting and all of a sudden your cell phone goes off and you're scrambling and you can't find the damn thing, it's in the bottom of your briefcase, right?
I can't tell you how vulgar that is.
You're in a professional setting, you commit professional suicide.
And I need to tell you that.
If you think it's OK you're wrong.
So let's get in the habit.
Disable the damn thing.
What do you need?
You're going to call your stockbroker?
What's so important right now?
And on an exam, that thing goes off I take the exam in front of everybody and-- [MAKING RIPPING NOISE]
--we introduce a defect, it's called a tear.
[LAUGHTER]
And then I put a zero on it like this, with a circle around it.
It's called a donut.
[LAUGHTER]
All right.
Now let's talk about some lighter things, more upbeat things.
Look, I want you to succeed in this class.
Now how are we going to succeed?
You go to the listing in the bulletin you'll see this, right?
And you zoom in here it says 5-0-7.
What's this mean?
Well the 5 is 5 contact hours.
3 here and 2 with your recitation.
The 0 is lab.
There's no lab with this class.
So what's the 7?
The 7 is the reading, the homework, preparation, et cetera.
I pledge to you, you give me 7 hours-- you go to the 5 hours contact and 7 hours-- you will not just pass this class, you will flourish in this class.
How do I know this?
Because I used to chair the committee on admissions and I've read applications.
I know the quality of individual in this room.
You should have seen, there were 11,000 applications piled up like this, each dossier like this, and they just get copied down on a four-by-six card.
We get here on a Presidents' Day weekend and we got stacks and stacks of the cards.
We're looking at your whole academic life is on a four-by-six card.
And I pick that up and I go, yes.
Literally at this speed.
About 45 seconds.
No.
[LAUGHTER]
Hell no!
[LAUGHTER]
That's how I spent Presidents' Day weekend.
And so you got through a very grueling selection process.
What's the corollary of that?
Listen carefully to this: Everybody in this room has the intellectual apparatus to pass 3.091.
The only people who fail 3.091 are people who choose to fail 3.091.
They choose not to come to class, they chose not to go to recitation, they chose not to work the homework.
I don't know why they make those choices.
But I guarantee you if you give me this amount of time, you'll do well.
It's straightforward.
Number one problem you are going to face this fall is not secondary bonding, it's time management.
So I used to call this strategies for-- what did I used to call it?
I forgot.
I used to have it something like, survival strategies.
But I don't want you to survive, I want you to flourish.
So I call it recipe for success.
There are different venues for learning.
OK?
So lecture, that's here.
That's my responsibility.
Recitation, that's Tuesdays and Thursdays.
That's my staff.
Now reading, that's you.
Homework, that'd be you.
Weekly quizzes is you, monthly tests is you, final exam is you.
[LAUGHTER]
What we have here is a partnership.
See?
[LAUGHTER]
So I'll do my part, you do your part.
You know, one of the things beyond the basic learning of the chemistry that we're going to attempt here is a transition.
You are going to change in ways you can't imagine.
I want you to think about the way you are right now, and I'm going to ask you to think about how you feel about yourself on the last day of class.
You will be amazed, and it's not because you know a few more chemical equations.
And what one of the things that I want to see happen and to help facilitate is the transition from student, which you are as a high school graduate, to scholar.
And the difference between a student and a scholar is a scholar takes ownership of his or her learning.
So you're going to take ownership of it.
You know, is this going to be on?
Are we responsible for this?
Go back to high school.
Here you're a scholar.
You say, how can I learn more about this?
That's the difference.
So I think we've covered enough.
I think we don't want to just have the whole day, welcome to MIT, welcome to MIT.
So what we're going to do is we'll get into the lecture, and in the very brief amount of time we have left I'm going to talk about the beginnings of chemistry.
We're going to talk about taxonomy, classification, nomenclature.
And to help introduce this I'm going to refer to the writings of William Shakespeare.
We're going to integrate some humanities here.
We're going to read from Romeo and Juliet.
Maybe there are people here who were admitted on the strength of the performance in Romeo and Juliet.
Maybe one of you was Romeo, one of you was Juliet.
Maybe one of you did the lighting.
Maybe one of you did the set design.
Maybe one of you took the tickets.
Somehow you were involved, right?
Because we're all involved.
So let's take a look.
Act 2, Scene 2.
Romeo: "But soft, what light through yonder window breaks?
It is the east"-- that's the east.
East Campus is that way, right?
[LAUGHTER]
"It is the east, and Juliet is the sun."
That has nothing to do with nomenclature.
Maybe photon emission, but-- that's a joke.
We'll get to it.
Now Juliet: "O Romeo, Romeo!
Wherefore art thou Romeo?
Deny thy father and refuse thy name.
Or, if thou wilt not, be but sworn my love, and I'll no longer be a Capulet."
See she's a Capulet, he's a Montague.
You know, the Hatfields and the McCoys.
And this is a metaphor.
It's as old as literature.
It's about warring factions, two communities that can't stand each other based on prejudice.
And then these two youngsters fall in love, and how love triumphs over hatred, and so on.
It's powerful stuff and it's written here.
Romeo: "Shall I hear more, or shall I speak at this?"
Remember, she's up high, he's doing "Shall I hear more, shall I speak at this?"
Fellows, the answer to that question is don't speak.
Don't interrupt her.
All right.
So now she goes, "'Tis but thy name that is my enemy.
Thou art thyself, though not a Montague.
What's Montague?
It is nor hand, nor foot, nor arm, nor face, nor any other part belonging to a man.
O, be some other name!
What's in a name?
That which we call a rose by any other word would smell as sweet."
That's properties, right?
"So Romeo would, were he not Romeo called, retain that dear perfection which he owes without that title.
Romeo, doff thy name, and for that name which is no part of thee take all myself."
Beautiful writing.
400 years ago and it's just fantastic.
Well, that's no good.
That's not the way it works in science.
In science we have to agree-- [LAUGHTER]
We have to agree on the name.
And so this is taken from Chapter 1 of the text, and this is the classification of matter.
This is about stuff and the different forms of stuff.
Over here we have the simplest form of stuff, which is the element.
And we're going to start here in 3.091 and we're going to work our way all the way through this table, starting with electronic structure and how electronic structure governs stuff.
So let's start with a history lesson.
We'll start with a history lesson.
And the history lesson goes like this.
What are the origins of chemistry?
The ancient Egyptian hieroglyphs refer to khemeia, which was a chemical process for embalming the dead.
You know the Egyptians were very fixated on the afterlife.
And the chemists, the chemists were revered in that society-- not like here.
They were revered in that society because they knew how to prepare the body for the afterlife.
Embalming is a chemical process.
A few years ago I was in London.
I toured the British Museum and came upon this.
This is 18 inches tall.
It's the mummified cat.
It's a fantastic example of mummification.
It's a beautiful-- most people go zooming right past it.
They're looking at the big dinosaurs and all that other nonsense.
But this, this is beautiful.
Mummified cat.
And then khemeia expanded to other chemical processes: dying of cloth, glassmaking, and metals extraction.
The chemist could take dirt and turn it into metal.
Sorcery.
Some things haven't changed.
I'm in that tradition: producing metal from dirt.
And they were always looking for an overarching theme, to unite the heavens with the simple elements.
So the seven known, naked-to-the-eye, astronomical bodies were associated with the seven known chemical elements.
And you could even talk in this priestly language.
So if you wanted to make a bronze, which is an alloy of tin and copper, you could say, well let's have the confluence of Jupiter and Venus.
It's all there.
Mercury.
Why do we call it Mercury?
Mercury's a metal that's liquid at room temperature.
It's fast.
Because Mercury is the planet that moves quickest around the sun.
Very nice.
So this is what we knew 2,400 years ago.
We had the seven metals, carbon, and silicon.
And the beginning of the shift from practice to theory, or from craft to science, is with the work of Democritus.
Democritus, who lived around 400 BCE.
Democritus, who was Greek.
He described the physical world as consisting of a combination of void plus being.
Void plus being.
These are lofty words.
Listen to this.
Void, being not some little equation.
It's big, big ideas.
And how did he describe void?
Void is something that we would recognize, in his language, as akin to what we recognize to be vacuum.
And being, he said, was comprised of an infinity of atoms.
He coined the term "atom."
"Atom" comes from the Greek tomoi, which is to slice.
And then if you put "a" in front of it as in apolitical or amoral, cannot be sliced: indivisible.
The atom cannot be sliced, and so to these atoms he attributed these properties.
They're indivisible and eternal.
I mean, this takes you all the way to E equals m c squared.
From 400 BCE, and that's all he had to work with.
This is brilliant.
Absolutely brilliant.
But to show you that things don't always go in a linear fashion for the better, along comes Aristotle.
Aristotle, another Greek, and he decided, nah, we don't need all this carbon, sulfur, and so on.
He said, we will have four essences that will describe the earthly world.
Four essences.
And here are the four essences.
Earth, water, air, fire.
So those are the Aristotelian essences.
And there's actually a fifth essence that describes the heavens.
That's why we say something is quintessential.
It's heavenly, it's et cetera.
So these are the four essences.
All right?
And then it gets even worse because he has compounds, a combination, so you can take fire plus earth and make dry, earth plus water make cold, air plus water make wet, air plus fire make hot.
This is nuts.
[LAUGHTER]
And it dominated science for a long, long time.
Because we know really earth is an aggregate.
It's an aggregate of different minerals which are compounds.
Water, as you know, is a compound H2O.
Fire is the product of combustion.
It doesn't even belong in this set.
And air is a solution.
It's a homogeneous mixture of nitrogen, oxygen, argon, rising levels of carbon dioxide, sulfur dioxide.
And if you're next to an aluminum smelter, tetrafluoromethane.
And so on and so forth.
And finally this thing was knocked down.
So we can look at this chart and go back to this one and say, now we can make the connections.
All right.
Here's the way the elements looked at the time of the American Revolution.
The alchemists gave us arsenic, antimony, and bismuth in the 12th, 13th, 14th centuries.
In 13th century India, there was zinc isolated.
There's a tall zinc pillar still standing there.
Platinum is an American metal.
It was unknown to the Europeans.
It was discovered when the Spanish came to the Americas.
Actually, it's kind of ironic because plata is silver, so platina is sort of like a diminutive of silver, which in point of fact is backwards because silver melts at 962 degrees Celsius, platinum melts at 1768.
It is nobler than silver.
It has catalytic properties.
It is even fashionable in jewelry now if you know how to work with it, because gold melts at 1063 and platinum melts at 1762.
A lot of jewelers can't work with platinum.
It's too high-melting.
It's a fantastic metal.
It's an American metal.
[LAUGHTER]
It really is.
I mean it's an American metal.
I didn't make this up.
It's true.
And then we see these other elements.
Discovered, discovered, discovered, discovered.
Now we see modern science.
So what does it mean, discovered 1766?
There was no hydrogen before 1766?
They found it?
No.
It means that Cavendish isolated it and documented its properties.
Hence, it was discovered.
Now I've been cheating here.
This is actually lined up in a way that we already know, where the story is going to end with this, with the Periodic Table.
I'm just lying them on the table in a way that makes it possible to anticipate.
But this was the first table of elements that is of record, and this was by John Dalton, the English chemist, who put these in order of atomic mass.
They're organized in ascending order of atomic mass.
And he also started a system of chemical symbols.
And you can see that iron has an "I" with a circle around it, and zinc has a "Z" with a circle around it.
And the Swedish chemist Berzelius said, you know I don't think the French are going to like to use "I" with a circle around it for iron, or Germans aren't going to like that.
We better choose something a little bit neutral.
So they chose Latin.
And that's why iron is "Fe" for ferrum, and gold is "Au" for arum, and so on.
But this was the first attempt.
John Dalton actually was a polymath.
He was also working on vision, human vision, and he suffered from an affliction that is present in about 10% of men, which is red-green color blindness.
And he did the original work on red-green color blindness.
In fact, in some circles it is known as Daltonism.
It's the same man, John Dalton.
Other classifications.
Dobereiner in Jena talked about triads.
And if you took the atomic mass of chlorine, added it to the atomic massive of iodine divided by two, you get something that's not too far off the atomic mass of bromine.
Newlands was a musician and he talked about octaves.
So if you start here, if this is a diatonic scale, so this is C, D, E, F, G, A, B, C. So potassium lies an octave away from sodium.
He was ridiculed.
They said, have you considered perhaps putting the elements in alphabetical order?
They were cruel.
Scientists can be very cruel to new ideas.
And, in fact, in your book this is some of Newlands work, and you can see to what extent it helps understand things.
But the first proper organization came in 1869 with Mendeleyev, and also 1870 with Lothar Meyer in Tuebingen, the Periodic table as we know it.
And this is the set of elements that were known at the time.
And this is a page from the paper in which Mendeleyev published the Periodic Table.
And here's the smoking gun right here.
It's right here where he says, in this new system, in this proposed system, there are very many missing elements.
Very many missing elements-- that was the key.
Because he knew that even though arsenic has the atomic mass next highest to that of zinc not to put arsenic under aluminum, not to put it under silicon.
It had more in common with phosphorus.
So he put it under phosphorus, and so did Meyer.
But what Mendeleyev did, which was a first, he said, there are missing elements here.
There is an element that lies between silicon and tin, and I predict what it's mass will be.
I predict what it's chemical formulation will be, how it will react with oxygen.
The predictions.
And how did he get this?
Because he used to travel by train and he would sit in the train station on his trunk playing solitaire.
And he'd be going down one suit, and then there'd be a gap.
And he'd go down the other suit and he could go farther.
And so the concept of, I know there should be an eight of spades here, but I have to stop at the nine.
I've got a seven but there's something in there-- triggered his imagination and led him to have the courage to say, there are elements there and I will predict their properties.
And when they were discovered how close he was is shocking.
Absolutely shocking.
So we'll get to that next day.
But before you go I want to leave you with this.
This is the portrait of Mendeleyev that you typically see.
This man on in years, disheveled, sort of mad scientist look.
Don't remember that.
Look at this.
This is Mendeleyev, age 35, when he proposed the Periodic Table of the Elements.
OK. We'll adjourn.
We'll see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Tuesday will be the first weekly quiz, celebration that is.
That will be at the beginning of recitation.
You'll have 10 minutes.
It'll be a short one-pager.
You just write on the page.
All you bring is your periodic table, which you should have gotten in recitation yesterday.
Periodic Table, table of constants, calculator, something to write with, but no aid sheet on the weeklies.
Readings: Readings, I urge you to read before you come to class.
And so if you go to the website, you can go to Schedule, and in the schedule, you'll see stuff like this that tells you what the readings are for the day.
As I mentioned last day, the lectures are being videographed and posted probably within an hour on the website and any of the images that I show are also recorded, burned as PDFs and uploaded.
So you don't have to be put in high-speed stenographic mode in order to attend class.
What's the other thing I wanted to tell you?
If you're new to the class or if you need to change your recitation section because your conditions have changed, do not simply go to the other class.
We're trying to regulate enrollment, particularly on Tuesdays.
If the TA shows up expecting 20 students and has 20 copies of the quiz and 25 people show up, that's not a recipe for success.
So you must go to my administrative assistant, Hilary Sheldon in order to change recitation.
And if you need any of the handouts and so on, it's just down the hall here in Building 8, Room 201.
I think that's all that I had to say.
If you go here the videos are all listed.
So last day we started talking about taxonomy and that led us to the beginnings of atomic theory.
We visited with Democritus, 400 BC.
We had that detour with the idiocy of Aristotle, and then eventually got back to our senses, and we saw John Dalton with his table of the elements, and then ultimately onto Mendeleyev.
And I wanted to pick up the thread there.
But before doing so, draw attention the fact that John Dalton did more than simply develop a set of fonts for us.
So he proposed the model of the atom and this goes back little over 200 years ago, and these are the features of the model.
First of all, that matter is composed of atoms that are indivisible and indestructible.
So that goes all the way back to Democritus, nothing new there.
All atoms of an element are identical.
Atoms of different elements have different weights and different chemical properties.
This is the emergence of modern material science, the connection between properties and elements.
So the weight arguably is one of the properties, but the only way they could distinguish elements at that time was by their atomic mass.
Atoms of different elements combine in simple whole-number ratios to form compounds.
Well, that makes sense because they're the elements.
They are the elemental building blocks.
If I told you you could build a structure made of blocks and part-way through your construction I say, why don't you cut that block in half, you'd say, well, then the block isn't the building block.
It's the half-block that's the building block.
So axiomatically if these are the elements they must combine in simple whole-number ratios to form compounds.
And lastly, atoms cannot be created or destroyed.
Well, he wasn't foretelling E equals mc squared.
What he was saying was that if you take elements and you combine them to form a compound, if you subsequently decompose the compound you get the elements back as they were.
So those are the features of John Dalton's model.
And we fast forward to 1869.
And this is the knowledge that was available at the time in terms of the elements that had been isolated and characterized.
And it was with this set of elements that Mendeleyev operated.
On file cards, in his breast pocket, he carried with him everywhere.
And he wrote down the names of the elements and their atomic masses and their properties and whatever else he could use the way of characterizing them.
And during the course of writing a textbook-- he had just finished a chapter on the alkaline metals and he was sitting in the railway station playing solitaire, and boom!
The flash came to him that you don't put arsenic underneath aluminum even though it's next in mass to zinc.
You move it over and you don't even put it under silicon.
You put it under phosphorus.
And furthermore, what he said was there's going to be an element here discovered under silicon and it will have these properties.
And let's look a little bit more deeply at the properties.
But before doing so I want to say that by announcing this prediction of what the element should be that's missing is that we start to see the evolution of principles of modern chemistry.
First of all, he recognized the pattern.
So did Lothar Meyer in Tuebingen.
So they both proposed a periodic table of the elements, but where Mendeleyev pulled away from the pack and distinguished himself was that he developed a quantitative model.
And I haven't shown you the quantitative aspect yet.
That's coming next.
That explains the observations, and that's good.
You might say, well, that's just curve fitting if you're a cynic.
But it makes predictions that can be tested by experiment, tested by experiment.
So let's take a look.
He said that under silicon, but above tin, there would be an element.
He called it eka-silicon.
Eka is a Sanskrit word, which means one after.
So this is the element one after silicon.
It was eventually isolated and given the name germanium.
Mendeleyev said it would have an atomic mass of 72 grams per mole.
In fact, it's 72.59.
He said it would have a density of 5.5 grams per cubic centimeter.
It's 5.36.
This is 1869.
He said that it would have a high melting point, whatever that means, and it melts at 958 Celsius.
It's compounds.
he said it would form a dioxide with a high melting point and a density of 4.7.
It forms a dioxide and its density is 4.70.
In fact, there's a story about a French mineralogist who came upon some of the stuff that ultimately became germanium dioxide, measured its density and reported it to Mendeleyev in a letter, saying, you know I measured the stuff and it's 5.3 grams per cubic centimeter.
Mendeleyev wrote him back and he said, make the measurement again.
You're wrong.
He wrote back three months later and said, I measured it again.
It's 4.7.
That was the genius of Mendeleyev.
To go way out on a limb and make those predictions.
And so I've made the case for the table of the elements.
Why do we call it the Periodic Table?
What's the periodic about?
Well, the periodic-- take a look here, if you go to the website there's a tab called Courseware and there's a tab called Periodic Table.
And you can go to the Periodic Table and ask the software to plot property.
So for example this is boiling point versus proton number or atomic number.
And so you see the boiling point varies as you move from low atomic number to high atomic number.
But it's not totally random.
It's not a Gaussian distribution.
There are features.
It goes up and down, up, down, up and down.
If you train your eye a little bit you'll actually see some regularity, a pattern there.
Maybe that's not so good.
Let's look at this one.
This is electrical conductivity.
And again, up, down, up, down and look at those red lines.
There, there, there, there.
Don't you see something?
That's a pattern.
And they're almost equally spaced.
So that was where Mendeleyev announced his Periodic Law, where he said the properties are related to the identity of the atoms.
And furthermore, he announced, that the properties are a periodic variation in atomic mass.
So let's get that now Mendeleyev's Periodic Law, and the properties of the elements vary periodically with atomic mass.
That was Mendeleyev.
So now that we know that we can go forward, and here's now the full-blown Periodic Table according to the framework that Mendeleyev established.
Now if you look at this carefully, you'll see down here things get whited out and there's these strange notations, uu, m, and all that stuff.
What's that all about?
This is where the super heavies lie.
These are all synthetic elements.
Transuranic, they're made by high-energy reactions, so-called high-energy physics in what you might call accelerators, atom smashers, what have you.
And there's only three places on the planet where you can conduct such reactions.
One of them is in Darmstadt in Germany.
One is in Dubna, just outside of Moscow.
And if you want to stay home-- and eschew the frequent flyer miles-- you can go to Berkeley, California.
These are the three places where we have the accelerators capable of making such compounds.
And so, take a look carefully at what the nomenclature is.
The way you name them is by using these Latin ordinals.
So un, bi, tri, quad and so on.
So if you wanted to name element 115, it's ununpentium.
You want the ium ending.
And you can make these up.
You could make up element 205 if you want to or whatever.
My favorite is 111 because that's unununium.
But there they are, so you can have fun with those.
But with time, the elements are being named and these have been synthesized since your version of the table was printed.
And so number 110 is named Darmstadtium in honor of the team at Darmstadt that first isolated it.
And number 111 was just named two years ago and the name is roentgenium after Wilhelm Roentgen, who discovered x-rays.
Now what is it about discovery?
Well, here's an example of one such reaction that would give you an element.
So if we had access to one of these devices we could take, for example, lead and nickel and accelerate them to very, very high energies.
And then we could make 110, ununilium.
Or now we'll call it Darmstadtium plus neutron.
And in doing so we've generated the new element.
But we can't just say we've made the element and publish.
We have to be able to characterize it.
Remember the reason that we gave Cavendish the credit for discovering hydrogen wasn't that he's the first to know that hydrogen exists, but he isolated it and gave it value.
So if you look at the rest of the periodic table, you get things like boiling point, melting point, density, electronegativity, first ionization energy.
There's a lot of information there.
If you go down here there's nothing.
It's all blanks.
These things have very, very short lifetimes.
Fractions of a second.
But you have to isolate them.
There's certain criteria before you can publish.
And all this is regulated by this governing body called the International Union of Pure and Applied Chemistry, So UPAC, the organization that finally rules on the legitimacy of any of these.
And actually there have been some retractions in recent years, where people published claiming-- I think there was a report out of Berkeley claiming that they'd synthesized 115 and then subsequently that was retracted because they couldn't support the property measurements Last thing is, if you're interested, want to do some extra reading, there's a fantastic book about Mendeleyev.
He was the youngest of 14 children, came out of a very poor family in Siberia and rose to be a giant of his day.
He was a polymath.
He, among some of the other things he did, he worked for the Ministry of Weights and Measures under the czar.
The czar was interested in taxation of alcohol.
And if you mix equal volumes of water and vodka you don't get additivity.
So 100 mL of water plus 100 mL of vodka doesn't give 200 mL.
It gives less.
And so Mendeleyev did a study to determine what the optimum ratio is so that people couldn't misrepresent the amount of alcohol in the beverage.
And set the standard at 40% alcohol by volume, which is used the world over to this day.
He also came to the United States in 1876 to go to Titusville, Pennsylvania, where the first oil well was drilled.
And did an exhaustive study of what was the American petroleum industry at the time.
And then went back to Imperial Russia and did the same survey for the Czar in Imperial Russia, including a report that recommended how to develop the natural resources of the time.
He was really an amazing man.
He wrote text books and so on.
And nobody in science-- I would venture to say-- has not heard of the periodic table.
Mendeleyev died in 1906.
The Nobel Prizes were first offered in 1901.
So there were five years where he was close to the top for winning the Noble Prize but was eked out by somebody else.
When you look back at those other Nobel Prizes, they were deserved.
But none more so than that for Mendeleyev.
So ironically the man who gave us seminal knowledge of all chemistry was never awarded the Nobel Prize.
And there's probably a lot of politics in there.
And as I said last day, here's the typical picture of him.
In this he sort of looks like a street person, disheveled and so on.
But this was the man that gave us the periodic table.
That's him at age 35 when he annunciated the Periodic Law.
So good for him.
Alright.
So now, let's take a look a little deeper about the properties of the elements.
How do we understand the properties of the elements?
For the properties elements we're going to have to look inside the atom.
If you did your reading you undoubtedly came across this table.
Which at first pass, deconstructs the atom into three simple particles: the electron, the proton and the neutron.
Here are their symbols, e, p and n. And they're distinguished by charge and mass.
So the electron has charge, minus 1.6 times 10 to the minus 19 Coulombs.
And a very low mass: 9.11 times 10 to the minus 31 kilograms.
The electronic charge is balanced by the protonic charge.
The atom is net neutral.
So the proton has a charge of plus 1.6 times 10 to the minus 19 Coulombs.
The neutron, as the name implies, has 0 charge.
The proton and the neutron have very nearly equal masses, however.
Right.
And just a word about the units.
The units here are given in terms of the Systeme Internationale.
So when we use the term, C, capital C is for the Coulomb.
And that's the unit of charge.
And it has an uppercase letter because it's named after a scientist.
In this case, the French scientist, Coulomb, whereas the gram is not named after a scientist and so it's lowercase.
And then we can amplify by powers of three.
So if I want 1,000 of these, I put a lowercase k here.
If I put an uppercase K, I end up with the unit Kelvin, which is the unit of temperature named after Lord Kelvin.
And all of this is known as SI units, which is the International System.
And it's not because the scientists don't know how to develop an abbreviation.
This was originally developed when French was the international language of science.
So this is known as the Systeme Internationale and all of these units were defined at that time.
And the term SI sticks that's the legacy.
All right, so now if we go to the Periodic Table.
When we start looking at the elements, we can look at any entry on the Periodic Table, and we have the chemical symbol that I'm designating here as uppercase X, and this was originally John Dalton with the I and the circle around it.
And about 30 years later the Swedish scientist Berzelius suggested that we use neutral units and so therefore we have the Latin coming in for many of the elements, such as iron, Fe, ferrum, and gold Au, aurum.
In the upper-left corner, we have the quality I'm representing here, A And A is the mass number.
Some people call it the atomic weight.
And it is the sum of the masses of all the constituents.
So it's the sum of the mass of the protons, so it's the proton number plus the neutron number plus the electron number.
But since the electron weighs 1/1800 of what these others weigh, you normally don't consider this.
It doesn't matter.
So just adding protons plus neutrons gets you to what we call the atomic weight.
And then down in the lower left corner we have Z and Z is the proton number.
And as the name implies, it's equal to the number of protons in the nucleus, which then equals the number of electrons outside the nucleus in the neutral atom.
Now I'm specifying neutral atom, because it's not necessary for atoms to be neutral and we'll take a look at those in a moment.
A point about redundancy here.
We don't really need the proton number and the chemical symbol because the proton number really defines.
The proton number is like the Social Security number.
This is the identity number of the atom.
If we change the the proton number, we change its identity.
So for example, I could write sodium.
Sodium 23 and 11.
I don't need the 11.
11 means it's sodium or sodium means it's 11.
So I could just write this as 23 sodium.
So I know it's sodium, that means it's got 11 protons and 23 minus 11 must be neutrons.
Or if I wanted to be a smart aleck, I could write this : 23 11.
That's sodium.
I don't need to put anything here.
But there's no smart alecks here, of course.
So for example, we could then show this reaction as-- this is what?
208.
This is lead, 208.
And nickel, 62 gives us Darmstadtium with a value of 269 and the neutron is 1.
You can see how these reactions can be made to go.
Now atoms don't necessarily have to be net neutral.
We can have something that is net non-zero charge.
Net non-zero charge on the atom gives it the term, ion.
Ion is an atom with net non-zero charge.
And we have two cases where the atom is net positive.
If the atom is net positive that's the result of electron deficiency.
The atom is electron deficient.
And we term such an atom the cation.
There's two types of ions.
The cation.
And then we have something that is net negative.
If it's net negative, it means it's electron-rich.
That is to say, there are more electrons than protons and the net negative ion is called the anion.
And you can try to figure out ways.
I sometimes think that cation has a t, which looks a little bit like a plus sign.
Anion has five letters, minus has five letters.
And they both end in n, but this has an n, which is negative or something.
You'll figure something out.
Now we've talked about varying charge at constant proton number.
But the other thing we can do is we can look at-- you can vary the neutron number.
Since the neutron has no charge you can vary the neutron number and not harm the identity and still have a neutral atom.
So vary neutron number at constant proton number.
And let's see what that is.
That gives you something that looks like this.
So for example, if you if you look at carbon, the atomic mass that's shown here is 12.011.
And you'd say, well, gee, if it's got 6 neutrons and 6 protons, why isn't that 12 exactly?
Well, this is the answer here.
You can vary the neutron number at constant proton number.
So let's take a look at how that plays out.
The way that plays out is as following.
Let's see I'm going to make a little table here.
So we'll start with carbon 12.
Carbon 12, so that means-- now I know what I'm going to do.
I'm going to bring this down and make some headings for me.
This will be my proton number and this will be my neutron number.
And finally I'm going to talk about abundance, natural abundance.
So carbon 12, since it's carbon, axiomatically it must have 6 protons.
And 12 minus 6 is 6, so it's got 6 neutrons.
And this is the dominant form of carbon.
If you took a chemical analysis of the carbon you'd find that over 98%, 98.892% of the carbon atoms that you examined would be of this form, carbon 12.
Now there's also carbon 13.
Has to be 6, otherwise it's not carbon.
That means it's got 7 neutrons.
And it's a minority species.
1.108%.
And then there's a third form of carbon and that's carbon 14.
Again, has to be 6 and it's got 8 neutrons.
And it's found in vanishingly small quantities, one part in 10 to the 12.
Or we could call it ppt, parts per trillion.
So this is same atomic number, same proton number, same Z but different mass numbers.
Different A's.
So all of these variants of carbon are found on the same place, the same spot on the Periodic Table.
The Greek word for same is iso, and the word for place is topo, so these are called isotopes.
The isotopes of carbon are species that have identical proton number but different neutron number.
How about the units?
What are the units here?
Well, we have to give some kind of unit.
I've been sort of freely going around and counting protons as one and so on.
And here's the standard.
The standard for mass is defined, and the definition goes like this.
If you take carbon 12, which we just introduced to you, and we say that we're going to specify a mass of 12.000 grams exactly for a specified quantity, in other words, a specified number of these atoms.
We have to say we'll take a certain number of these carbon atoms and specify the mass of that number is 12 exactly.
And specified number of atoms being the mole.
The mole.
And it turns out that the mole has a value of 6.02 times 10 to the 23rd.
How do they get that number?
A little bit more in the way of definitions.
It was a concept put forth by a professor.
So we're going to take some time on it because we respect professors, in this class at least.
And so this was a concept put forth by Professor Amadeo Avogadro.
Professor Avogadro, who was a professor of physics at the University of Turin, Torino.
And he was a contemporary of John Dalton's and they were both studying gases.
And it was Avogadro who taught us that, when you keep the pressure constant equal volumes of different gases contain equal numbers of molecules.
It doesn't matter if you have argon, which is by itself atomic, or we have oxygen, which is diatomic, or you have methane, which is CH4, five atoms making a compound.
Equal pressure, equal volume, equal numbers of those species.
So that was Avogadro's Law.
So let's put that down.
At constant pressure equal volumes of different gases, contain identical numbers of atoms.
Equal volumes of different gases contain equal numbers of molecules.
And here I'm using the term molecule as a counting unit.
So it could be, strictly speaking, an atom or it could be diatomic and so on.
That's what it was.
And out of honor for Avogadro, we name the number of atoms in the mole the Avogadro number.
Which I've written 6.02 times 10 to the 23rd.
Now how do we determine Avogadro's number?
That's an interesting story.
So first of all, we need two pieces of information.
Because we're going to do this by the noblest form of chemistry, electrochemistry.
So the first thing we're going to do is we're going to look at the work Michael Faraday in England.
And what Michael Faraday did is he studied the electrodeposition of metal.
And specifically he passed current through a cell and he electrodeposited silver.
So he starts with silver plus, that's silver a cation, and by the action of electric current attaches an electron to silver and renders it neutral.
Silver, which now plates out onto an electrode and they measured the mass.
They measured the mass of silver-plated and they compare it to the amount of charge that was passed.
They measured the charge.
And you can get charge, because you know current.
So charge is simply equal to the integral of the current times the time.
You know the current, that's easy.
And what Faraday found was that to make what we now know to be 108 grams of silver, 108 grams of silver, which we're going to subsequently recognize as the mole, which is identical to the amount, the number of particles in 108 grams of silver, is equal to the number of particles in 12 grams of carbon.
Sort of an Avogadro-type harkening.
He found that that is-- the equivalent requires 96,485 Coulombs.
So you can say 1 mole of electrons gives me 1 mole of silver, so that's the charge on 1 mole of electrons, where Coulomb is the elementary charge, because we know 1 electron per 1 silver atom deposited.
So now if I know that's a mole of electrons, I need to find a charge on one electron, divide through and I get the Avogadro number.
And to finish the story we have to wait about 50 years and come to the United States, where it's Robert Millikan, Robert Millikan at the University of Chicago doing the oil drop experiment through which we learn the elementary charge.
And here's the cartoon of the oil drop experiment.
I took this from a different text.
It's not shown in your text.
So I actually did this experiment as a sophomore at the University of Toronto.
They had us repeat some of the great experiments of physics, the ones that were accessible, obviously.
I couldn't do high-energy physics in an afternoon.
That would have taken me a little bit longer.
But we did this one.
And so it consists of an atomizer, sort of a perfume atomizer, in which there's oil.
And by the action of atomization we form a shower here, a very, very fine dispersion of tiny droplets of oil.
And then-- this cartoon is hard to make sense of so I fixed this-- we shine high-energy radiation on this.
And by the action of high energy radiation we take these neutral droplets and we turn them into ions.
We eject electrons.
And so now these are charged.
And then we charge the plates.
So if we have neutral species and they simply come out of the atomizer, they'll settle under gravity.
But now if they're charged and I put a charge on the plates-- let's say as here the upper plate is positive-- if any of these particles is charged positive, the action of the electric field will accelerate the descent, because the bottom plate is negative attracting and a positive plate at the top is repelling.
And vice versa.
If I have a particle that's negative, the upper positive plate will actually cause it to slow down, and in the extreme, it may actually start to rise.
And so what Millikan did is a set of experiments in which he studied all the different particle sizes.
See this telescope?
Right over here is Millikan.
And Millikan's sitting there and he's squirting and he's watching.
He's measuring the settling velocity.
And he changes the magnitude of the electric field.
He changes the intensity of radiation.
He changes the nozzle.
He changes everything he can.
And what does he find?
He finds that the distribution of velocities is not continuous.
It's not continuous.
You think, well, gee if you just keep dialing you should get every variation of velocity.
Well, he doesn't.
He finds that he gets variation down to a single value, below which he can't go.
He determines that electric charge is quantized.
That is to say there's a base unit.
It's an element.
I just talked to you about the elemental building block.
That's an element in mass space.
Now I'm going to go conceptually into charge space.
There is an elemental building block of electric charge.
Electric charge is quantized.
And he found that the elementary charge, which we gave the symbol, e. e is not the symbol for electron.
e is the symbol for elementary charge.
It has a value, if you convert it to modern SI units, of 1.6 times 10 to the minus 19 Coulombs.
So now I can take these two pieces of information, Faraday which is up here.
This is known as the Faraday Constant.
Script f, Faraday constant.
So if I divide the Faraday constant, which is the charge on a mole of electrons, by the elementary charge, which is the charge on one electron, presumably I should end up with the Avogadro number.
It should be the ratio of the Faraday to the elementary charge.
And it gives us-- for the third time this morning-- 6.02 times 10 to the 23rd.
If you like per mole, yes or no, doesn't matter.
So now, what's the atomic mass unit?
Now we can say the atomic mass unit is, 1 atomic mass unit then must equal what?
It's going to equal 1/12 of the mass of carbon 12.
1/12 of the mass of carbon 12 divided by the Avogadro number, which gives us 1.661 times 10 to the minus 27 kilograms.
Now be careful because the system is just a little bit rickety.
You know we went SI, but look, this is still defined as 12 grams.
And so sometimes if you look depending on where this is, 10 to minus 27 kilograms or 10 to the minus 24 grams.
Just be careful.
If you ignore this you'll be off only by factor of 1,000.
That's a joke.
But it's lost here.
People are too serious.
We'll lighten you up.
All right, so enough of the history.
Let's now do something dynamic.
So far we've been studying static elements.
But chemistry is really the action of elements in motion.
So how do we describe a chemical reaction?
Let's look at that.
What are the rules to describe a chemical reaction?
Write an equation.
Write the equation of the chemical reaction subject to these rules.
There are two simple rules.
One is conservation of mass.
We've been told the repeatedly since Democritus, conservation of mass.
And the second thing, we use Dalton's Law of Molar Proportions.
That is to say, the building blocks in integer ratios.
And so I thought I'd do this in context.
So I've got a specific example here.
So this is something that I'm interested in.
Some of my research is in metallurgical extraction by benign processes.
What you're looking at is a billet of titanium.
To give you a sense, you can see the stairwell back here.
So this is about 4 feet, a little over a meter here.
So you can see this is one honking big piece of titanium.
This came out of the primary reactor, the Kroll reactor and this is subsequently swaged and hot worked and so on to form these billets.
So this is the first step of turning dirt into metal.
That's called titanium sponge.
And titanium sponge occurs inside a Kroll reactor.
It occurs inside a Kroll reactor, which was invented by a man of the surname Kroll in Luxembourg in the 1930s.
And then with the advent of World War II, he decided to be smart, to get out.
And he ended up in Oregon where he became a professor.
So he's known as Professor Kroll, although the truth be told he really made his discovery before he became a professor.
But he's still a professor and so we'll honor him.
And so the Kroll process for making titanium centers around this reaction.
Here's the reaction written according to the rules above.
We take titanium dioxide, which is found in the Earth and by some prior chemistry convert it to titanium tetrachloride, and in a reactor that I'm going to show you in a moment, we react titanium tetrachloride with magnesium.
And magnesium has a higher affinity for chlorine than does titanium and steals the chlorine from titanium to form magnesium chloride, leaving behind titanium metal.
Now we have to have conservation of mass.
So you can see, I've got 4 chlorines on the left but only 2 chlorines on the right.
So I'm going to put a 2 here and double the magnesium chloride.
But now I've got 2 magnesiums on the right and only 1 on the left.
So I'll put a 2 in front of the magnesium and now we have a balanced equation.
And here's what the reactor looks like.
You can imagine a giant vessel with a pressure seal on the top and a couple of valves, big enough to make this.
So this is about 15 feet by 30 feet.
And so we introduce titanium tetrachloride, which is a gas, and magnesium as a solid and heat to 900 degrees C. And at 900 degrees C, if you look on your Periodic Table you'll know that magnesium melts at 650 degrees C. So we have a liquid sitting here, titanium tetrachloride here, and this thing is sealed.
It's called a bomb reactor.
Nothing can get in, nothing can get out.
The pressure builds up here.
And right at this interface the titanium tetrachloride reacts with the magnesium according to this reaction.
Now this is very interesting.
It's beautiful reaction because the titanium tetrachloride is a gas; magnesium is a liquid.
Magnesium chloride is a liquid, but it is of different density, and it is insoluble in magnesium, and titanium melts at 1670 and it's a solid.
So what happens over time is this.
The magnesium chloride that forms pools underneath the magnesium liquid, gets out of the way so that we can continue to keep this interface clean and have the reaction proceed.
You don't want to reaction where reactant A reacts with reactant B, makes a product that covers the interface and now the product is in the way of future reaction.
So this is very elegant because I don't need any fans, I don't need any nose propellers, nothing.
By density the magnesium chloride settles and the titanium settles.
And it's sitting here at the bottom.
And you can imagine if we do this long enough, this titanium at the bottom will continue to build until it looks like this.
As long as you keep feeding TiCl and Mg. See I'm talking metallurgy now.
TiCl and Mg, that's what you make.
So that's how we make titanium, first step.
And so suppose you get hired and it's your first day on the job and you're working at Cambridge Titanium and the boss says let's put in 200 kilograms of TiCl and we'll put in 25 kilograms of Mg. And the question is, what is the yield?
What is the yield?
How much titanium are we going to make?
Well, you say, just multiply it out.
But first you have to see if things are in balance.
We have to study the stoichiometry of the reaction.
Stoichiometry, what does this mean?
It's from the Greek, stoicheia, which has to do with measurement proportions.
So if these are not put in to the reactor in proportion to what they are in the equation we're not going to get the yield here.
So first thing I gotta do, this is in moles, this is in kilograms.
So I have to convert the kilograms to moles and then maybe I can make some sense of this.
So if I divide by the atomic mass of titanium, four times the atomic mass of chlorine and convert; I will discover that I have 1,054 moles of TiCl.
And I've got about 1,029 moles of magnesium.
Well, this equation says I need 2 times the amount of titanium tetrachloride.
Well, it's obvious to the naked eye, 1,029 isn't two times 1,054.
So I've got a problem here.
I'm not going to get as much titanium as I put in.
Titanium chloride.
This yield is going to be restricted.
It's going to be restricted by-- this is sort of a chain is as strong as its weakest link-- the yield is restricted by the amount of limiting reagent.
And in this case, magnesium is-- this is less than 2 times the mole number of titanium chloride.
So this means this is the limiting reagent.
Alright so now if we use that principle then I'm only going to get as much titanium as I had magnesium and you can see from the stoichiometry here, if I've got 1,029 moles of magnesium I'm going to have half of that number of titanium.
So therefore the amount of titanium is equal to 515 moles of titanium.
And you notice I'm not obsessed about a significant figures and so on.
It's a metallurgical plant.
Half of 1,029 is 515.
Is it 514.5?
If you wish.
I don't care.
So 515 moles and then I convert that, which gives me 24.7 kilograms of titanium when I use that amount of magnesium.
And if you go to the text, Section 2.7, you'll see the nuts and bolts of how to run these reactions.
For those of you who had a lot of chemistry in high school, I know this is review, but I want to bring everybody up to the same page.
So we're starting with this.
All right.
I think that's a pretty good place to stop with the delivery.
But I don't want you moving.
You don't move yet.
Because the last 5 minutes I'm going to still continue to talk.
But on a slightly different topic.
And so I don't want to hear the binders snapping and so on.
We're here; you paid your money.
Five more minutes.
Five more minutes and then you're out there.
Out there, then begins le weekend.
But not until then.
So a couple of things.
First is, the music today.
I try to link the music thematically.
So the music playing today was Polovstian Dance number 17 from Prince Igor, by Borodin, Aleksandr Borodin.
Why were we listening to this music?
Well, because I insisted that we listen to it.
Well, what about Borodin?
Borodin lived in Saint Petersburg.
He was a friend of Mendeleyev.
OK, that's cute but more importantly Borodin wrote his music in his leisure time.
He had a day job.
His day job was professor of chemistry.
And he worked at the Medical Surgical Academy in Saint Petersburg.
He was an exceptional human being.
In those days, women were forbidden to attend institutions of higher education.
He set up an entire curriculum for women in a night school at the Medical Surgical Academy.
He cavorted with artists and therefore obviously his politics were radical.
And they were trying to reform the political scene in Czarist Russia at the time.
And he was also quite a bon vivant.
And he died on his feet dancing at a ball.
So that's the way to go.
Having a great time.
That was Borodin.
One other thing before you go.
You were very very dour, so I thought I'd try to put you in a good mood to the extent this is possible with this group.
And I wanted to share with you some news.
There's been a new element discovered.
You know these atoms smashers, they're always working.
And so the discovery of the heaviest element known to science has been reported.
The element, tentatively named administratium.
I don't know if UPEC is going to go for this, but you can suggest names.
So they're going to name is administratium, the discoverers.
It has no protons or electrons.
So that means its atomic number is 0.
It does have one neutron, 125 assistants to the neutron.
75 vice-neutrons and a 111 assistants to the vice-neutrons.
This gives it a mass number of 312.
The 312 particles are held together in the nucleus by a force that involves the continuous exchange of meson-like particles called memo-ons.
There's no electronic mail, because there's no electrons.
There may be neutronic mail but we don't know yet.
Now you've already learned something today.
You know something.
Since it has no electrons, what do we know about its chemical reactivity?
It's inert.
It has no electrons.
It can't exchange.
So this is chemically inert.
So you say, how did they detect it?
Because it seems to impede every reaction in which it is a present.
According to the discoverers a few nanograms rendered a reaction that normally takes a fraction of a second, it took now four business days to conduct that same.
There are a few other properties.
We know so far that it's radioactive.
And we're going to study radioactivity later, so there's a little bit of foreshadowing.
It has a half-life of about three years, at which time it stops decaying and instead it undergoes a reorganization, in which the vice-neutrons, assistants to the neutrons and assistants to the vice-neutrons, exchange places.
Some studies indicate that the mass actually increases after each reorganization.
So you can imagine now we'll have something like this.
See how this increased?
So if they occupy the same place, they have the same proton number, but a different neutron number, in the case of administratium, they're called isodopes.
So with that I will say, have a good weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Time to learn.
Time to get back to business here in 3.091.
So let's do the announcements first.
Tomorrow quiz one, based on homework one.
Ten minutes.
Bring your Periodic Table, your table of constants, something to write with and a calculator.
There's about a dozen or so copies of the text on reserve at the Hayden Library.
I want you to be aware of that.
Now some of you approached me last day and said is it true that we have to memorize the Periodic Table?
Yes.
But not the whole Periodic Table.
I know some of you are going to say, well that's so 19th century, rote learning and so on.
To which I say, you're wrong.
Every educated person should know that potassium lies under sodium, which lies under lithium.
And you are going to be educated whether you like it or not.
But this is a lot to learn and so we're just going to look at the main block.
We're not going to ask you to memorize the lanthanides, actinides and the super heavies.
We'll leave that out.
So it's quite straightforward.
We've been doing this for many years.
This is from 2004.
And it's not that hard.
I give you the blank here.
And I even put the numbers on for you.
And all you have to do is add the two-letter or one-letter symbol.
You don't have to give me the full name of the element.
You don't have to get me its density or its electron affinity.
You've just got to put argon, in there.
Ar.
That's all you've got to put in there.
OK?
And every year people grouse when I announce this.
But you'll find out that you'll breeze through it.
You'll have 10 minutes.
This will be on September 24.
So next week there will be two minor celebrations.
One on Tuesday and one on Thursday.
So it's going to be a very festive week.
And people blast through it.
It's one mark off for every error down to zero.
We don't give negative scores.
And people just go right through it.
And then they urge me to make sure that in future years I victimi-- I mean I ask students to continue this.
People are very proud of the fact they know the S, P, and D blocks of the Periodic Table by memory.
But I don't want to neglect, disrespect the lanthanides and actinides.
So we're going to have two contests.
Mnemonics to help remember the names of the lanthanides and actinides.
That's tricky to remember.
So these are mnemonics contests.
So here are some examples.
Here's one, see lanthanum, cerium, praseodymium, neodymium and so on, all right?
Obviously this one came from industry because of the slur against the Academy.
Obviously, somebody in industry has no idea how hard we work here.
And plus, look at this: "to dramatically help".
This gigantic split infinitive.
Clearly, clearly someone from industry.
Here's another one that would help you memorize, I think.
Now, this is obviously not referring to 3.091.
It must be referring to some other chemistry professor here.
I'll let that go unchallenged.
Now, occasionally people ratchet up.
And this was about six, seven years ago, a young man by the name of Blake Stacey, who was aware of the fact that there are 14 lanthanide elements because there are fourteen F electrons.
And an Elizabethan sonnet has 14 lines.
And so he took Sonnet 57 by Shakespeare in reference to the fact that lanthanum is element 57.
And he wrote this in reference to Sonnet 57.
And, of course, he knew he was going to win when he made samarium the word smelting, and being the chemical metallurgist I am, he had me at the fifth line.
Actually, I showed this to Professor Sonenberg in Humanities, and she said, actually, this is pretty good.
That's one of those times you look at and say, you wouldn't see this at Harvard.
So now I'm going to show you one.
Sometimes people ratchet up.
A couple of years ago, there was a video.
So let's take a look at that one.
[VIDEO PLAYBACK]
And she goes through various poses as a superhero of different elemental value.
[END OF VIDEO PLAYBACK]
But we don't have that much time.
So anyways, September 25, 5:00 p.m. Eastern Daylight Time.
Submit your entry, whether it's the big canisters of the Hollywood footage or whether it's just a 14-line mnemonic device.
And the prizes-- it's a contest, there are prizes-- I have ties and scarves from the American Chemical Society.
[LAUGHTER]
What are you laughing at?
These things!
Now look at me!
This is going to be stylish.
They're black.
They've got elements from the Periodic Table on them.
Very hot.
Very hot.
They're black.
They look great.
And we're going to present them on the Friday of the parents' weekend because the parents eat this stuff up.
So now I think it's time to get back to learning.
So last day we ended with the stoichiometry, and I showed you the Kroll process, how to make titanium.
So the question that we never got to was how do we tell that magnesium has a higher affinity for chlorine than titanium does?
How do we know that magnesium is going to reduce titanium tetrachloride to a sponge?
So for that we have to look inside the atom.
We have to look inside the atom.
So that's what we're going to start doing today.
And we're going to begin, as often we will, with a history lesson.
So let's go back for a status report around the end of the 19th century.
So what did we know towards the end of the 19th century?
The atom is electrically neutral.
The negative charge is carried by electrons.
We knew that there was some carrier of negative charge.
We knew further that this electron is very, very tiny.
Small mass.
And so if it has a very small mass, then by difference, then the bulk of the mass of atom must be contained in the positive charge.
So now the question is, what is the spatial distribution of charge inside an atom?
You might say, well why do we want to know that?
Well, we're trying to get a physical model of the atom so that then we can start to address the question of chemical reactivity.
So we have got to know where all the players are before we can attribute action to them.
So let's now take a look at models of the atom.
For the first modern model of the atom we go to J. J. Thompson.
J. J. Thompson, he was a professor of physics and director of the Cavendish lab at Cambridge University.
The other Cambridge.
Cambridge University.
And this Cavendish Lab was named after Henry Cavendish.
The same one who is credited with the discovery of hydrogen in 1766.
He never married, and when he died he bequeathed this fortune to Cambridge University, and they took that money and established a physics laboratory, which is still in operation to this day.
So what did J. J. Thompson tell us?
He said the electrons were distributed throughout a uniformly charged positive sphere of atomic dimensions.
So that's the text.
Now let's make this cartoon, because that makes more sense.
Visualize it a little bit better.
So it looks a little bit like this.
So here's a sphere of atomic dimension.
Electrons are these tiny little things distributed throughout the positive sphere.
And this is massive because the electrons have very, very low mass.
And just to be clear, there are no protons.
No protons.
So this isn't positive.
It's a positive chargeness.
It's just a big positive blur.
And then the little electrons inside.
So the electrons are negative, and they're tiny and low-mass, and they're mobile.
He didn't say much about how they move, but you can imagine that they move, and maybe they even spin.
And so on.
And this was known as the Plum Pudding Model.
There's a big cultural bias here.
So even though I grew up in British Canada, I never had this stuff.
But my understanding is plum pudding is this dish that's served at holidays and Christmas-time.
So the custard is the charge, the positive charge.
And these little bits of fruit are the electrons.
And they're throughout.
At Christmas-time, they might even put little charms in here.
Like a little wishbone or something for good luck.
And you pull it out and all that kind of stuff.
Yeah that's merry, old England.
So this is the Plum Pudding Model.
And that's what was in place.
By the way, we've been using the term electron, but J. J. did not like the term electron.
Actually, he won his Nobel Prize for the-- quote, unquote-- the discovery of the electron.
Just as Cavendish didn't discover hydrogen, he characterized it.
J. J. Thompson characterized the electron.
What did he in particular characterize?
He characterized the charge-to-mass ratio.
He made the first quantitative measure of the charge-to-mass ratio of the electron.
But he never used the term, electron.
He called it the corpuscle.
He called it the corpuscle of electrical charge, whereas this is really sort of an elementary charge.
The corpuscle.
And fortunately there was another British chemist.
In fact, he was an electrochemist.
His name was John Stoney, who, while I think that Thompson was a brilliant man, I wasn't crazy about his literary choices.
And it was Stoney who said, let's call the term electron, coming from the Greek term, elektra, which is the Greek word for amber.
And you know if you rub amber, you'll build up static charge, and so on.
That's where we got the electron from.
So that looks pretty good.
So that's our theory.
And what happens next?
Well, what happens next is that we've got to put this theory to the test.
And the person who puts the theory to the test is a man by the name of Ernest Rutherford.
Rutherford was an interesting character.
He came from New Zealand and he was born on a farm.
It was a family with 12 children.
You notice Mendeleyev came from a family of 14, this guy comes from a family of 12.
So you learn how to survive.
He worked on a farm so he was very handy, and that came in to play as his career progressed.
He was a brilliant experimentalist.
He was able to do experiments where others failed.
And he did this experiment where he was a professor of physics at Victoria University in Manchester in the UK.
Now, he came out of New Zealand, got a scholarship to study at Cambridge University.
And he was actually a research student for J. J. Thompson.
And he conducted his thesis research under J. J. and during that time his thesis research was about the properties of charged particles.
And they were very interested in gases and radioactive elements, ionizing radiation, the whole thing.
The Cavendish lab was a beehive of activity.
And what he did under his PhD was he understood both the alpha particle and the beta particle.
First, he identified what they are.
The alpha particle was determined to be the helium nucleus.
So this is helium that's lost both of its electrons.
So it's just protons and neutrons in the nucleus.
Completely naked.
No electrons.
And then the beta particle, which other people in physics had been referring to this mysterious beta particle, he made the connection between the beta particle and the electron.
So the same electron that's over here and attributed to the negative charge within the nucleus is also a free particle that can go through space.
And these are taken from your reading.
And you can see that if you send the particle beam through a charged plate, the negative plate here being the lower plate, the positive particle, which in this case is the alpha particle, will bend down towards the negative plate.
And the negative particle, being the free electron, is bent in the opposite direction.
So these are the kinds of experiments they would do.
And they also looked at their ability to penetrate different media.
So when it comes to their ability to penetrate solids, they differentiate themselves.
So they alpha particle is quite poor at penetrating solids.
So you see in the cartoon there, the alpha particle is stopped even by that little sheet of paper.
So it doesn't do so well here, whereas the beta particle, the electron, can go zooming right through the paper and is stopped by the lead.
And then over here they were also studying their ability to ionize gases.
So if you take a gas and you bombard it with alpha particles, will you get a lot of ionization?
Yes or no, et cetera.
And what they found was that it was a complementary, that the alpha particle is pretty good at ionizing gases, whereas the beta particle is poor at ionizing gases, and in fact, is stopped.
This is why, if you have an electron beam you need a vacuum.
Otherwise, the electron will be dissipated in very, very short order.
So after doing this work with J. J., he got a teaching job in Canada at McGill University up north.
And while he was there he continued to do work on the origin of alpha particles and won the Nobel Prize for his work while he was at McGill.
And so when he determined that the alpha particle origin in the decomposition, the disintegration of elements.
And believe it or not, in this work back in 1906, 1907, he was already anticipating nuclear fission because he was saying that if you put thorium or polonium into a tube, you can expect that it will emit alpha particles.
Well, conservation of mass says they have got to be coming from somewhere.
And if they're not coming from out here, they have got to be coming from inside.
This was brilliant work.
This is Rutherford, young man in Montreal, gets the Nobel Prize in 1908.
And then he goes to Victoria University, and he sets up this experiment.
Critical experiment.
I mean, imagine this guy.
I mean, he's already got the Nobel Prize.
He could just put his feet up.
No, he keeps going.
So this is the experiment.
It's called the Rutherford-Geiger-Marsden experiment.
Because Rutherford was the brains behind it and Geiger and Marsden were the guys in the lab.
And so here's the experiment.
So what he's going to do is he's going to put this model to the test.
We're going to see if this model makes sense or not.
So he's got a gold foil-- now they used different elements, but ultimately they loved gold because gold, first of all, is noble and it doesn't oxide.
So it's pure, even in atmospheric oxygen.
And secondly, we've known since antiquity we can physically deform gold and make it into something sub-paper thin.
So that's what they did.
They beat this thing down to 600 nanometers thick, 0.6 micron.
And they used alpha particles here.
Radium, polonium, thorium as the source in the lead container and so on.
And this stream of alpha particles comes zooming out at energies of 7.68 million electron volts per particle.
So this is like a cannon with a pumpkin in front of it.
And they're going to shoot these at the foil.
So what's the purpose here?
They want to see what's going on.
And if you imagine that we are shooting alpha particles, which are positive, and I've got a wall here of all of these nuclei that are essentially positive chargeness, uniformly distributed with these tiny, tiny little electrons of almost no mass, you'd expect that most of these particles are just going to go zooming on through.
Beyond a certain thickness, I might start seeing some attenuation.
But if I get the gold film really thin, I should just go blasting these all through.
Well, what they find is, two things.
First of all, most of the particles do go through.
When they go through, they're deflected a little bit.
They don't just go through, they are deflected.
And most of them are deflected through low angles.
But a small number of them are deflected through high angles.
They almost go straight back at the source.
So if you've got that set of data, that doesn't support this model.
This model cannot be sustained in the light of those data.
It's a very important experiment.
By the way, how do they count these things?
Well, Geiger is the same guy who give us the Geiger counter.
Geiger was a technician that worked for Rutherford.
And he developed this screen, they called it a scintillation screen.
It was coated with zinc sulfide.
And so when alpha particles bounced and they hit the screen, they would excite electrons in the screen and then cause a scintilla, a spark.
And then what you do is you just get a graduate student or some other source of abundant, cheap labor-- and you have a whole bunch of graduate students sitting here-- and they count.
That's how it was done in good old days, 1909.
And so they counted what the distribution was.
So that's the experiment.
So this is taken from your book.
So this what you'd expect.
And this is what they saw.
So this is small angles and this is large angles.
Theaters of the angle here.
Now Marsden, who's the third?
There's a third person.
Marsden.
Marsden's not too different from you.
Marsden was about 20 years old at the time.
He dropped out of school and then he came to Rutherford, and he said, I'm looking for a job.
In those days, you couldn't return to college.
If you dropped out, finished.
You had to stay all the way through.
Once you dropped out, they slam the door behind you.
But Rutherford, he was different.
He looked at people on the basis of their intrinsic value.
He didn't care about title.
And so he said to Marsden, OK, you want to do something for me?
Here's your question, I like you to imagine an alternative model of the atom.
And here's the alternative model of the atom.
What Rutherford said was, suppose instead, if this is the dimension of the atom, instead of having positive charge uniformly distributed, he said, what if we have positive charge concentrated at one point and then out here we have the negative charge?
So here's the positive charge.
And here's a whole bunch of negative charge.
And now here's the next one.
And here's the next one.
And so on.
And now we're going to shoot these helium nuclei.
See what's in between here?
This is a void, taking from Democritus.
And so now if I shoot the positive helium nuclei here, if they get close to this positive charge they'll be deflected.
If they don't get close to the positive charge, they just go right through.
And once in awhile they get almost on axis, in which case they fling back.
This model would be consistent with the data set I just showed you.
And then to go further, he says to Marsden, I'm going to tell you what the distribution of high-angle scattering was, you tell me what is the ratio between the radius of the nucleus, where the positive charge is to the-- excuse me, I'm showing you the diameter, so this is the diameter of the nucleus versus the diameter of the atom.
What would be the ratio of the two?
I mean, it's gold, so I know I got 79 plus here.
And I've got 2 plus.
And I've got a 79 plus.
What would be the size of the sphere of 79 plus that would deflect something of 2 plus through this angle?
So that's Marsden.
It's not bad for a kid that's been out of school for a couple of years.
That's his problem.
So he did.
He did the calculation.
He did the calculation and Marsden figured out that the radius of the nucleus to the radius of the atom is on the order of about 1 to 10,000.
That means most of the gold foil is void.
Most of the gold foil is nothing.
And you've got all this mass, highly dense, concentrated here.
So this is the Rutherford model, which was called the nuclear model.
This was proposed by Rutherford because it has such a thing is a nucleus.
He coined the term, nucleus.
So that's a big step forward from this thing here.
So you'd think that the physics community would be ecstatic.
That now we've got a model that squares with the data.
Well, what do you think the reaction to Rutherford's announcement was?
Condemnation.
Derision.
People didn't like it.
They laughed at him.
They said, well, this is crazy.
There's a couple of problems with this.
So strong negative reaction to Rutherford's model.
And here the two major objections that people posed.
First of all, they talked about nuclear collapse.
They said if you've got a system of negative particles revolving around a positive nucleus, electrostatic forces are going to cause them to be drawn to one another.
What's going to keep the electrons from collapsing into the nucleus?
That was the first objection.
Coulombic attraction.
Coulombic, electrostatic, same thing.
If you want to use a man's name, Coulombic.
If you don't want to use a man's name, electrostatic.
Coulombic attraction between plus and minus.
And the second one was, OK, suppose for some reason we don't understand that the electron doesn't collapse, it stays in orbit, it's still not going to work.
Because you run out of energy.
If you take a charged particle, a charged particle that accelerates consumes energy.
Acceleration, as you know, can mean either change in speed or change in direction.
So if the electron is revolving at a constant speed, it has to change direction; otherwise, it will fly out of the room.
So changing direction means acceleration, which means what keeps the energy in the electron?
Why doesn't the electron just run out of energy and just stop?
Even if you can prevent it from collapsing, it will just stop.
So this is the problem of energy deficit.
Energy deficit.
Well, what powers the electron?
What powers the accelerating electron?
So that's a problem.
So now the story continues.
The next chapter is 1912.
Because this is 1911.
In 1912, a young Danish businessman by the name of Niels Bohr.
Niels Bohr just finished his PhD in Copenhagen and he got a fellowship from the Carlsburg Brewery Foundation.
Carlsburg Brewery put up money for science.
So he won a Carlsburg fellowship.
And he was a young physicist.
And he knew that in the UK there was a controversy brewing between the model of Rutherford and the model of J. J. Thompson.
So he says, you know what I'm going to do?
I've got a year, I got money.
I'm going to spend six months with J. J. Thompson, I'm going to spend six months with Ernest Rutherford.
And that's what he did.
So he goes and he spends six months at the Cavendish lab with J. J. and then he goes up to Manchester and he spends three months at Manchester with Rutherford.
Why didn't he spend six months?
Because he got a job offer and he went back to Copenhagen to start his teaching duties.
And what he did is he developed a quantitative model for us.
He developed a quantitative model.
And that's all part of what I showed you the other day.
We recognize patterns.
That's what the Rutherford, Geiger, Marsden experiment gave us.
The pattern of the distribution of alpha particle deflections.
So now we need to develop a quantitative model that will explain the observations, and if it's a really good model, it will make predictions that, again, can be tested by experiment.
So we ratchet back and forth between experiment, model.
Experiment, model.
So he proposes the model, not with J. J., because J. J., I don't know what he did there.
This is where he wants to propose the model.
Quantitative model.
To Rutherford's atom.
Let's take a look at what we've got there.
Now, big lesson here.
How do you announce a model?
You don't put an ad in the newspaper.
This is the scientific literature.
This is not Google.
This is not Wikipedia.
This is a primary source.
This is how science is communicated.
So this is a journal.
It's called Philosophical Magazine and Journal of Science.
It goes all the way back to the 1600s.
In fact, when I was a freshman at the University of Toronto we had copies of Phil Mag going back all the way to the 1600s.
And these were not facsimiles, these were real copies.
And I was in the stacks and I was reading something by-- I don't know-- Lord Kelvin, you know late 1800s, and I said gee, well, if I've got Lord Kelvin, then maybe I can go over here to the early 1800s and read Humphry Davy and, gee, if can keep going, and going and I pull out a leather-bound edition of Phil Mag from the 1660s, and I open the book and there's a paper about gravity by Isaac Newton.
And that's how science is communicated.
So I urge you to go to the library and read primary sources.
Most of this stuff is available online.
You can just sit at your desk and you can zoom in to history.
So this is Phil Mag and this is London, Edinburgh and dublin.
See this is 1913, so Ireland was not a free state yet.
So Dublin was a British city.
London, Edinburgh, Dublin.
There's a close-up of it.
July 1913, On the Constitution of Atoms and Molecules by Niels Bohr, Doctor of Philosophy, Copenhagen.
And it was communicated by Professor Ernest Rutherford, FRS, Fellow of the Royal Society.
So, let's read.
It's in English.
In order to explain results of experiments on scattering of alpha rays by matter, Professor Rutherford has given a theory of the structure of atoms.
You see the dagger here?
Down here, Ernest Rutherford, Phil Mag, duh-duh-duh-duh.
There's a citation.
See, attribution.
According to this theory, the atoms consist of a positively charged nucleus surrounded by a system of electrons kept together by attractive forces from the nucleus.
That's what keeps them from just fleeing anywhere, and they don't go in, but they vwomp.
The total negative charge of the electrons is equal to the positive charge of the nucleus.
Further, the nucleus is assumed to be the seat of the essential part of the mass of the atom and have linear dimensions exceedingly small compared with the linear dimensions of the whole atom.
This is beautifully written.
Clear.
It's textbook quality and it's written by a man whose native language isn't even English.
Read the literature.
The number of electrons in an atom is deduced to be approximately equal to half the atomic weight.
Great interest is to be attributed to this atom model.
You have to know a little bit about Bohr.
Bohr used this phrase, great interest.
That was his way of saying, embroiled in controversy.
If someone said something that he didn't agree with, instead of saying, I think that's nonsense, he say, very interesting.
Very interesting.
So great interest is to be attributed to this atom model.
That means people are in pitched battle on both sides.
For as Rutherford has shown, the assumption of the existence of nuclei as those in question seems to be necessary in order to account for the results of experiments on large-angle scattering of the alpha rays.
Another footnote: Geiger and Marsden.
Geiger and Marsden published their results in this journal.
In an attempt to explain some of the properties of matter on the basis of this atom model, we meet, however, with difficulties of a serious nature arising from the apparent instability of the system of electrons.
Difficulties purposely avoided in atom models previously considered.
For instance, in the one proposed by Sir J. J. Thompson, which is to say, mmm, you know, it's curious.
You condemn the Rutherford model because he's got electrons in motion, which means they're accelerating in energy deficit.
But in the Plum Pudding Model, the electrons are in motion, and J. J. Thompson is silent about it.
So this is a very aimed barb right at J. J. Thompson done elegantly.
With this elegant slap in the face.
You know this is how scientists do it.
They don't go, oh you're ugly , and your mother dresses you funny.
Instead they write things like this, difficulties purposely avoided in atom models previously considered for instance.
So this is sort of yeah, how about you, huh?
But anyway, we can go on and on.
But down here, the result of the discussion of these questions seems to be a general acknowledgment of the inadequacy of classical electrodynamics in describing the behavior of systems of atomic size.
So he's saying that you can't use classical electrodynamics down to subatomic dimensions.
He's saying that those models, any more than you can use planetary models down to human dimensions, this, in many respects, is foretelling nanotechnology.
He's saying the properties we know, of how things behave in the Newtonian world are not necessarily valid at atomic dimensions.
This is very important stuff.
Anyway, so what I did is I went through the paper and I've reduced the content of the paper to these things called postulates.
So Rutherford's atom is correct.
That's the first thing.
Classical electromagnetic theory not applicable to the orbiting electron.
And you're going to get-- the PDF and all this is going to be posted, so don't feel like you have to be a super stenographer here.
Newtonian mechanics is applicable to the orbiting electron.
So you can see, wait a minute it's sort of like cafeteria physics here.
Electrodynamics doesn't apply, Newtonian mechanics does apply.
And the answer is, yeah.
That's what we're going to do here.
We'll build a model and we're going to try these ideas out.
And if they're crazy, how are we going to know they're crazy?
Because there's going to be data that refute.
But if we get data that supports all this, then there's only one conclusion.
This thing makes sense up to a point.
The energy of the electron is conserved in the system.
It's bond is kinetic plus potential.
The quantization through angular momentum.
And lastly, the Planck-Einstein relation applying to electron transition.
So now I'm going to take all this and I'm going to write out the model for you.
Now, I don't want you to say, oh, he's going to derive something.
Do we have to know derivations?
This is the only time I'm going to derive something for you.
The reason is, it teaches the model.
It teaches how the model is based on assumptions and so on.
But I don't expect you to memorize things because that's not the way things work.
So let's go starting with this point here.
This point here is Bohr model atom.
So it's Bohr's representation of the Rutherford model.
We can call it a nuclear model or a planetary model.
Because you've got orbiting electrons.
Now, first important point.
The Bohr model is the simplest model.
It works for a one electron system.
So he has a single electron.
Planetary model, single electron.
And this orbits the positive nucleus.
So you might say, one electron, well, what's that mean?
Well, obviously it means atomic hydrogen.
But it could mean helium plus.
That's a one electron system.
It could mean lithium 2 plus.
It could mean roentgenium 110 plus.
That's a one electron system.
And this is all gas.
Gas phase.
No solids here.
Gas phase.
So it's an isolated nucleus with one electron around it.
That's how we're going to begin.
And here's what it looks like.
I'm going to put the nucleus, first of all not to scale, because we're not going to draw 10,000 to 1.
Over here is the nucleus with z positive charge.
And then out here is the electron in orbit.
This is the lone electron in orbit.
And this distance is r. The distance from the nucleus.
You're going to say, well, is that from the outside or the inside?
It's 10,000 to 1; it doesn't matter.
Forget about it.
So now let's use this concept.
We're going to say it's a conservative system and so the energy of the system is simply going to be the energy of the electron.
You might say, well, why don't you consider the energy of the nucleus?
Because the nucleus is, relatively speaking, stationary.
To use Rutherford's colorful language, he says when an elephant has fleas, it's the fleas that do the jumping.
So you don't care about that energy of the nucleus in this case.
So the energy of the electron is the kinetic plus the potential.
So the kinetic energy, we're saying it's 1/2 mv squared because it's Newtonian mechanics apply.
See number 3.
So that's 1/2 mv squared, where v is the velocity.
So that's the kinetic, and then what's the potential energy?
The potential energy here is electrostatic.
Because that's the energy that's stored here is due to the Coulombic forces between them.
So that's going to be q1, q2 over 4 pi epsilon 0 divided by r. So what I'm going to do here, we have to just for convention I'm going to always choose that q1 in my derivation, q1 will be the charge on the nucleus.
so q1 will equal z, which is the proton number here, times the elementary charge.
Remember e is the elementary charge.
It's not the charge on the electron.
So sometimes I use the word e with a minus sign, meaning the electron.
But this is e meaning the elementary charge.
z times e and then q2, which is the charge on the electron, is minus the elementary charge e. So we're always going to have minus e out here.
What varies is z.
And so we're going to make some substitutions.
So this is going to be what?
It's going to be 1/2 mass of the electron times velocity of the electron squared.
And now q1 is ze and q2 is minus z, so that's minus ze squared.
And then that'll be over 4 pi epsilon 0 r. And I'm going to call this thing equation 1.
And by the way you can find all of this information by looking at your table of constants.
So see, epsilon 0 sitting here.
Permittivity of vacuum.
There it is, 8.85 times 10 to the minus 12 farads per meter.
Elementary charge 1.6 times 10 to the minus 19 Coulombs.
Electron mass, 9.1 times 10 to the minus 31 kilograms.
And what's this 4 pi epsilon 0 doing here?
Well we know this is going to give us joules.
And this is going to give us joules.
And the epsilon 0 is the factor that renders electrostatic units on to the same plane as mechanical units.
If I plug in-- this is going to be Coulombs, it's Coulombs squared, and this is going to be meters, and now this things is farads per meter.
And I got farads per meter times meters divided by Coulombs squared.
I don't care.
This, with impunity is joules.
Because this is an energy unit.
And this is Systeme Internationale.
And this is kilograms times meters per second, meters per second and how many apples in a bushel and-- forget it!
Put this in kilograms.
Put this in in meters per second.
With impunity, it's in joules.
So that's what that thing does there.
Alright.
Good.
So let's keep going.
Postulate 3.
Newtonian mechanics applies to orbiting electrons.
So orbiting electron, it's in a stationary orbit.
So that's the thing.
So postulate 3 implies that the sum of the forces acting on the electron must be 0.
This is a force balance, right?
If the forces are out of balance, the electrons are either going to move closer to the nucleus or move farther from the nucleus.
So that's equal to, again-- it's a sum of a dynamic force and a Coulombic force, or electrostatic.
Here I just noticed a nice parity here.
So these are technical terms, but we can give them human terms.
So the dynamic force, there's a Newtonian force, this is a Newtonian force and this is a Coulombic force, after the two people that are associated with the ideas.
Charges interaction are Coulombic, blah, blah, blah, blah.
So now lets substitute in.
So we know that if you've got something orbiting, something tethered, then the dynamic force on that, that pulls the object away on a tether is mv squared over r. And the electrostatic is q1 q2 over 4 pi epsilon 0 r squared.
Force goes as 1 over r squared.
Energy goes as 1 over r. How do you know?
Well, one way to think about it is, you know energy is the result of a force moving through a distance.
You say, I'm working really hard, and I say I don't see any force moving through a distance.
That's a joke.
So here's energy.
So force.
So if this is 1 over r squared, the integral of 1 over r squared is minus 1 over r. If you get these backwards, you integrate 1 over r, you're going to get natural log of r, which makes no sense.
That's one way.
Or the other way is, you can just remember.
So anyways we're going to plug into those values.
And what do we get?
What we get there is this.
We get mv squared over r. q1 is z times e. q2 is minus z.
So it's minus ze squared over 4 pi epsilon 0 r squared.
And I'm going to call this equation 2.
So now we keep going.
Next one, energy quantized through its angular momentum.
This is really critical.
This is the major breakthrough by Bohr.
Now what do we mean by quantization?
I told you that in 1909, Millikan had figured out that electrical charge is quantized.
Now quantization had already been annunciated by Max Planck in 1909.
You see, the classical theory of radiation gave this prediction for intensity as a function of wavelength.
They called this the explosion, the collapse of the theory.
As you go to low wavelength, the intensity goes without abatement.
But these are the spectra that were measured.
Depending on the temperature of the gas, you get this distribution.
So how to get the distribution to turn around at lower values of wavelength?
And in order to do that, Planck suggested that light is composed of energy packets, or quanta.
That light is a stream of individual energy bits.
And the elementary unit of electromagnetic radiation is the photon.
And you can say, well I don't understand, where goes he get that from?
That's not the way to think about it.
Say, if you make this assumption you get these curves.
And then figure out what it means.
You can't make everything anthropomorphic.
If this were the case, and furthermore if the energy of one of these quanta was related to its frequency-- and wavelength and frequency are inversely related-- then the proportionality constant is h, which we now have designated the Planck constant in honor of Planck, then you get this.
People grew to accept it.
1900, he gets a Nobel Prize for it.
La de da!
But who knows what light is anyways.
It has no mass, it's kind of mysterious.
1900, the whole concept of action at a distance was strange to people.
The notion that something could affect you, from me, without physical contact, this was a new idea.
We take it for granted, but this was very new.
So it's kind of mysterious, they said, OK, he's a physicist, who knows?
And it worked!
But it's light.
Now what Bohr is going to say is the electron moving in orbit, he says, I'm going to apply the same concept of quantization to the electron moving in orbit.
Well, that's different.
And using this value he gives us mvr, which is the angular momentum is quantized n h over 2 pi.
h is the Planck Constant and n is an integer.
1, 2 3 4, et cetera.
So this is the quantum condition and this will be equation 3.
And I have three equations and three unknowns, and what I'm going to do next day is come back and show you the solution of the equations and how they give us a set of equations that allow us to compare with data.
So let's jump to the thing.
And by the way the Planck constant is there.
Now Niels Bohr eventually won the Nobel Prize.
He was widely revered.
He achieved the stature on a par with Einstein.
If you go to Denmark, it's still not part of the European Union, you can still get the 500 Kroner note if you change a 100-dollar bill there.
And there's Niels Bohr with this pipe.
And you can see the concept of things rotating around it an so on.
There's a young Niels Bohr.
This is Bohr with Werner Heisenberg.
Heisenberg was a post-doc who worked with Bohr in 1925 and annunciated the Uncertainty Principle, which we will study a few lectures from now.
This is a chemical conference sponsored by chemical company Solvay.
Brussels 1930, there's Bohr and Einstein.
They loved hats.
This is the Hamburg, this is Borsalino.
Here's Bohr mixing it up with royalty.
This is a young Queen Elizabeth of England.
There's Prince Phillip.
This is the royalty Danish family.
Bohr loved music and here is with Louis Armstrong undoubtedly talking about the resonant structures within the tube here and how to get the various harmonics and overtones.
Now last thing, a little bit about hydrogen.
We've been studying hydrogen here, the one electron atom.
But there are isotopes of hydrogen.
The original was discovered by Henry Cavendish.
There is one form of hydrogen that has a neutron in addition to the proton.
That's the deuterium, from which we can get heavy water, D2O.
This was discovered here in New York at Columbia University by Harold Urey.
And then lastly, there's a second isotope of hydrogen that consists of two neutrons and a proton.
That's tritium and it was discovered at the Cavendish lab by Ernest Rutherford.
He was active right up to his death.
In fact, it is argued that Rutherford did his best work after he got the Nobel Prize.
Your book is showing this.
It's nice to follow.
We started with Dalton and now we're here and then later, in another week or so, we'll be down here.
And here's the sort of the intellectual road map that takes you to modern atomic theory.
So with that I'll dismiss the class and we'll see you on Wednesday.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
OK, settle down.
Let's get started.
One announcement, yesterday we had our first weekly I would say minor celebration.
And by and large, it went well.
Please make sure that you go to your assigned recitation.
If you miss your recitation, you need to get down to see Hillary so that we make sure that we have enough copies of the weekly quizzes on hand.
If you've joined the class, you have to check in with her.
She's down the hall here in room 8-201.
And you'll be assigned to a section.
What else do I have by way of announcements?
Oh yes.
Just reminding you, this was from 2003.
See it doesn't change.
It's the same s-block, p-block, and d-block elements.
So that's coming up a week from tomorrow, two celebrations next.
And of course the contests, the contests with hot prizes.
All right.
Let's get down to business.
Last day we looked at the Rutherford-Geiger-Marsden experiment.
Oh there's one other one.
If you look at the readings, there's this one section called the archives.
My predecessor, Professor Wit, wrote a set of lecture notes.
And they look something like this.
And some students have said that they find these a little more expository in certain sections on certain topics than the book to be.
And I have no preference.
But if you take a look at this it'll say LN1, lecture notes 1.
If you go to this, read this.
If you find that's helpful, good.
If you don't find it helpful, than stick with the book.
Just letting you know what that is.
All right, so last day we looked at the Rutherford-Geiger-Marsden experiment in which a high energy beam of alpha particles bombarded a thin, gold foil.
And on the basis of the scattering results, namely most of the particles went through with minor scattering.
And a tiny fraction of them were scattered through large angles.
The Thomson plum pudding model was rejected in favor of Rutherford's nuclear model of the atom.
And then subsequently, Bohr came up with the quantitative representation off the Rutherford nuclear model.
And we were partway through the treatment of Bohr last day when we adjourned.
So let's right pick up the thread from where we left off.
And so just to remind you, the Bohr model is for a 1-electron atom gas phase.
So this is either atomic hydrogen, it could be helium plus lithium 2 plus roentgenium 110 plus Its doesn't matter how many protons.
there's always only want electron.
And it's a planetary model.
The positive charge concentrated in the nucleus, Z is the proton number.
And at a distance r from the nucleus is a circular orbit in which resides 1 electron.
It has a charge of minus E. And I just designated them q1 and q2.
You could have done it the other way.
But I had to choose something.
So we went through and looked at the constitutive equations here.
So first of all, the energy of the system-- and this is only going to be the energy the electron.
Because we assume that the nucleus is far more massive.
And so we don't have to get into things like reduced mass or anything like that.
So just measure the energy of the electron.
1/2 mv squared is the newtonian component.
And then coulombic energy that's stored is z times e squared over 4 pi epsilon zero r, where epsilon zero is the permittivity of vacuum.
And it's the factor, the 4 pi epsilon zero, is the factor that allows us to take electrostatic energies and put them on the same plane as mechanical energies.
When we run through this, we always end up in joules.
Then there's a force balanced to make sure that the electron neither falls into the nucleus nor flees and breaks free of the atom.
And the force balance is if you put a ball at the end of a string, and you whip it around on a tether, you have a centrifugal force that's trying to get the ball to break away.
And then the string is pulling in.
So the pull in, in this case, is the coulombic force.
And the force that makes the ball want to flee is this mv squared over r. And that must be net zero.
Otherwise we're going to have a shift in orbit.
And then lastly we have the quantum condition.
And this was the breakthrough of Bohr where he enunciated that the quantum condition is going to give us this energy level quantization.
And this was a big departure from what had been in the past.
The only antecedent idea of this nature was the work by Planck who said that light is quantized.
But as I told you last day, who knows what light really is.
The Newtonian notion of a ball orbiting was very compelling here.
And the notion that the movement of the electron could in some way be discontinuous was quite a major departure.
So I left you last day with three equations and three unknowns.
And what I'm not going to do right now is solve the equations.
Because I've been lecturing long enough to know that's the way to kill interest, quench a lecture.
And so if you really want to go through the algebra, be my guest.
You're smart enough to do that.
Instead I'm going to show you the results.
But those are the three equations that you need.
So we have an equation in r. We have an equation at v, and an equation in e. So let's go after them.
If you first look at the solution for r. This is the radius of the orbit of the electron, the orbit of the electron.
If you go through and solve, you'll end up with this, Epsilon zero times the square of the Planck constant divided by pi times m. It's always the electron.
So this is the radius of the electron orbit.
This is the mass of the electron times the square of the elementary charge-- that whole thing I'm going to group-- times the square of the quantum number divided by z, the proton number.
So what we see here?
Well, everything inside the parentheses is constant.
These are all constant.
Pi obviously is geometric constant.
And the rest of these are constants you could look up in your table of constants.
And we noticed that there is a set of solutions to this.
The radius of the electron can occupy various discrete values defined by n. So we say that the radius takes on a plurality of values a function of n. And furthermore, the functionality goes as the square of n. It's n squared times a constant, where that constant is inside those parentheses.
And we notice that because the r goes as n squared it's nonlinear.
It's nonlinear.
This is so important I'm going to write it down one more time.
So the radius of the electron orbit takes multiple values.
It takes multiple values.
And they're discreet.
The physicists like to use a different term.
When something is discretized, the physicists say it is quantized.
So these values are quantized.
You cannot continuously vary the radius and nonlinear values, multiple values.
So let's plug in.
Because I want to get a sense of scale.
So let's look at the simplest one.
The most primitive 1-electron atom would be hydrogen.
In which case, Z equals 1.
So I've just got a proton orbited by an electron.
So look at atomic hydrogen.
So in that case, Z equals 1.
And I'm going to look at n equals 1, which is the lowest number here, right?
R scales as n squared.
So the lowest value or r is obtained when n equals 1.
And this is termed the ground state.
The ground state.
So I want to ask what is the radius of the electron orbit ground state in atomic hydrogen?
And if I plug in these values, I'll call this r sub 1.
It turns out to be 5.29 times 10 to the minus 11 meters, or 0.529 angstroms.
I love the angstrom.
It's a great unit.
It's a great unit.
It's not an SI unit.
But I like the angstrom.
I'll show you why.
If you try to express this in SI units, well there's 10 to the minus 11 meters.
The Si units go in units of clusters of 1,000.
So for example, you've got the meter.
You've got the kilometer.
You've got the micrometer.
You've got 10 to the minus 9 meters, which is the nanometer.
This is a Goldilocks problem.
This one's too big.
And then the next one down here is 10 to the minus 12 meters, which is the picometer.
So this is either 52.9 picometers, or a 0.0529 nanometers.
And that's no good.
I want numbers like 3, 7, simple to remember.
So 0.529, this is about 1/2 angstrom.
It's good to know.
But you try to publish, you know what happens in a scientific literature today?
The Literary Lions that control the journals, they'll circle that and say you have to convert to SI units.
And so they have to right some goofy nanometer thing or something.
I know you think I'm crazy, but I love the angstrom.
So there.
Anyway, so here it is.
Once you know that this is 0.529, this is on your table of constants.
It's right on your table of constants.
So you don't have to go and calculate all this stuff.
Which means if you do your homework with your table of constants, you will know where those numbers lie, as opposed to opening this thing up for the first time on the first celebration of learning on October the 7th, and with 47 entries and they're tiny, tiny font.
And you're wondering where is that thing.
Just a word to wise.
So now we know what this is.
We know this is 0.529 angstroms.
So now I can write an equation for the radius of a 1-electron atom anywhere, anytime.
r of n is going to be equal to a naught which is this value here.
And it is termed the Bohr radius.
So you can write it as a function of the Bohr radius, times the square of n divided by Z.
So that's for all 1-electron atoms, gas phase.
And you can see that as Z goes up, the r goes down, which makes sense.
So suppose instead of hydrogen, we talk helium plus What's the only difference?
Helium plus has 2 protons in the nucleus.
Which means that the coulombic force of attraction between the same 1-electron and now a doubly charged nucleus is going to be stronger.
So the first orbit is going to get pulled in.
And all the other orbits are going to get pulled in.
By how much are they going to get pulled in?
By that much.
So this is the functional representation of all of that physics.
All right, there's three equations, three unknowns.
Let's look at energy.
So if you go through and solve for energy, you get this one.
Minus this big monstrosity, mass of the electron to the fourth power of the elementary charge times 8 times the square of the permittivity of vacuum times the square of the Planck constant, all times the square of the Proton number divided by the square of the quantum number.
And I just to make sure everybody is with me here.
I always want to write n here.
n equals 1, 2, 3, takes on integer values.
And we'll say it again here.
N equals 1, 2, 3, et cetera, et cetera.
So again we say we see that e is a function of n. It's discretized.
It's quantized.
Why?
Because once you impose the quantum condition here on angular momentum, it propagates through the entire model.
So radius this quantized, energy is quantized, you're going to see velocity is quantized.
Because the quantum condition is pervasive.
So e to the n, and I'm going to take this whole quantity in parentheses and just call it giant K. These are all positive quantities.
Mass is positive.
And squares and fourth powers of numbers must be positive.
So this is K times Z squared over n squared That's good.
And we can go and evaluate K. And when we evaluate L in SI units, we get 2.18 times 10 to the minus 18 joules.
This is joules per atom.
Or you can multiply this by Avogadro's number.
If you multiply it by Avogadro's number, then that will give you 1.312 megajoules per mol.
So that's the energy of the electron in the ground state of atomic hydrogen.
And then we can mediate that with Z and n, and go to electrons that are outside the ground state, above the ground state, or ground state electrons in atoms that have more than 1 proton, or both.
And so let's take a look at the graphical representation of that.
So instead of Cartesian coordinates, because this is spatial distribution, I'm going to go to energy coordinates and give you an energy level diagram.
So again, not to scale.
Because this thing goes is 1 over the square.
So that's going to be messy.
So let's start here.
And on the left side I'm going to designate the energy.
And on the right side I'm going to designate the quantum number.
I'm going to start down here.
That's the lowest energy.
You see these are all negative values, first of all.
They're all negative values.
Because Z is a square, n is a square, and K is a positive quantity.
So n equals 1 is the lowest state.
It's the ground state.
It has a value of minus K. So I'm going to do this one just for atomic hydrogen.
So I'm going to write atomic H. So now Z equals 1.
You can do it later for Z equals 2, 3, whatever.
So this is atomic hydrogen.
So ground state energy is minus K. What happens if we go to n equals 2?
n equals 2 it becomes K divided by 2 squared 4.
So it should be 3/4 of the way up.
I'm not going to go quite 3/4 of the way.
Because I want to leave room for some fine structure.
That's why it's not to scale.
All right so this is minus L over 4.
What if we go to n equals 3?
Well it's not symmetric here.
It's nonlinear.
But this should be really what?
Minus K over 3 squared is 9.
You get the picture.
3, you can go 4, 5, and so on until n equals infinity.
What happens when n equals infinity?
I've got minus K over infinity, which is vanishingly small, zero.
Where is the electron when n equals infinity?
r is n squared times the Bohr radius.
That's a great, great distance away.
What does it mean?
Physically it means that the electron is so far away that it is no longer bound.
It's no longer part of the atom.
And when it's no longer part of the atom, and the potential energy that's stored is a result of the charges coming together from infinity.
I'm starting to talk like someone out of 802.
What's the energy if I take 2 charged particles at infinite separation?
I bring them into some finite separation.
Voila.
There it is.
So when they're in infinite separation there's no energy stored.
Hence, you are at that point.
So n equals infinity means r equals infinity, which means E equal zero.
There's no stored energy.
So this means the electron is no longer bound.
And therefore, if it's no longer bound, we have a term for it.
It's called free.
It's a free electron.
And if the electron is free, then the atom is electron deficient.
So if the electron is free, that means the atom is now an ion.
Because it's lost an electron.
It's no longer net neutral.
Or we say an atom hasn't turned into an ion.
Or the electron has been ionized.
What's the energy for that?
We can calculate what that energy is.
It would be called the ionization energy.
So if I started with an electron down in here, and I sent it all the way to infinity.
See that's an energy space.
Which is the equivalent in Cartesian space to go from here to infinity, same idea.
Do you see the models?
This is Cartesian.
It's like a. Map This is energy coordinates.
It's different.
And you're going to be able to think from one model to another.
What's the energy consequences?
What are the Cartesian consequences?
Until we get to the point where there is no Cartesian representation.
Because the abstraction level is too high, we'll have to content ourselves with this.
So get comfortable moving from there to there.
And then some day we're going to say it's too complicated.
There's no Cartesian thing.
We'll be comfortable by then with this.
OK so now we're taking an electron from here up to here.
While we ask, what is the ionization energy?
The ionization energy must equal the delta E of the transition.
So what's that?
The delta E of the transition is going to equal always E final minus E initial.
E final minus E initial.
Which is equal to E at infinity minus E1, the ground state.
Well E infinity, we just said, is zero.
And the ground state energy is equal to minus K. So minus minus K is K. So you also get the energy.
From infinity down to ground state is the ionization energy.
So we can define the ionization energy in terms of this transition.
Define ionization energy as the minimum energy to remove an electron from the ground state of an atom in a gas phase.
So that means there's no solids, no liquids.
There's no work function here.
There's no lattice energy, and so on.
So there's a definition of the ionization energy.
And we can be a little bit more elaborate.
Even though right now I'm just going to do a little break here.
I don't want to mislead people.
But just an aside.
I'm nonlinear.
I can have multiple conversations at once.
And you are capable of stacking.
So we're going to break now.
We're not going to talk about 1-electron atom.
We're going to follow the thread of ionization energy.
I'm going to take lithium.
Lithium in its normal state has 3 protons, 3 electrons.
So I'm going to take lithium gas.
And I'm going to ionize it and make lithium plus.
So this is a lithium plus ion in the gas phase.
It's still got 2 electrons.
So this isn't Bohr model stuff.
But anyway, here we are.
So the energy for this action would be called the ionization energy.
Because I took a neutral atom, and I pulled an electron out of the ground state, and so on.
So this is an ionization energy.
But now I can continue this process.
And I can take Lithium plus.
And I can lose an electron from that, which will than give me lithium 2 plus.
And this is called also an ionization energy.
This is called the second ionization energy.
So this is the first ionization energy.
But just as when you write an equation, when the coefficient is 1, you don't write the 1.
I don't write 1 lithium here.
I know it's 1.
This is the second ionization energy.
And then I can keep stripping away electrons.
And I can take lithium 2 plus in the gas phase, and take away that electron leaving me with just the lithium nucleus, lithium 3 plus, plus electron.
And that's called the third ionization energy.
OK, now what can we say?
What's the relationship here between any of this except the definition and the Bohr model?
well?
The Bohr model applies only to 1-electron atoms.
Are there any 1-electron atoms on this board?
So we can calculate the energy, the third ionization energy.
We can get that from the Bohr model.
You can do it in your head.
You can do it in your head right?
It's just K times Z squared right here.
It's going from 1.
So if this is 2.18, it's going to be 9 times that trivially.
OK, so this is just 3 squared times K. And this when you have to get from the literature.
So you have to go to primary sources.
Which is why you're going to learn how to use the proper database.
And this one here also you get from the literature.
But this is on your periodic table.
Your periodic table, one of the data points it gives is the first ionization energy of all of the elements.
And so even though lithium normally is a solid at room temperature, the ionization energy for lithium as given on your periodic table is for this reaction.
It's for the gas.
Anyway.
So that's little aside.
All right, the last quantity that we could get from the Bohr model is v, the velocity.
And I solve for the velocity.
We don't talk about this very much.
But we're going to do so once today.
And here it is.
So I went through the algebra.
And you get nh over 2 pi mr, where r is the radius.
There's the quantum number.
This is quantized as well.
So I regrouped this.
And I already have a nice, cool expression for r in terms of the Bohr radius.
So I use that because that's on the table.
So this is 2 pi times the mass of the electron times the Bohr radius, 1/2 angstrom, times Z proton number divided by n, where n equals 1, 2, 3, and so on.
So let's again get a sense of scale.
So let's try for sense of scale.
Let's do velocity of the ground state electron in atomic hydrogen.
So that means Z equals 1, n equals 1.
So I plug in the numbers.
And I get v1 for hydrogen, atomic hydrogen, gives me 2.18 times 10 to the 6 meters for second.
I don't know.
Is that fast?
Is that slow?
I don't know.
But I do know this much.
I know that the speed of light is equal to 3 times 10 to the 8 meters per second.
So 10 to the 8 divided by 10 to the 6.
2 and 3, that's roughly 1, speaking as an engineer.
Who cares?
So this is about 1% of the speed of light.
That's pretty good.
That gives me something I can hang on to.
I would say that if this thing is zipping around at 1% of the speed of light, I would say that's relatively fast.
One more time, 1% of the speed of light.
That's relatively fast.
Remember last day I told you that Bohr simply dismissed the concept of the use of classical electrodynamics down to atomic dimensions.
Well here's another example of why a lot of these assumptions aren't going to work so well.
We're talking about the ground state electron velocity in this putative planetary model of a 1-electron atom.
You're already getting into relativistic effects.
So, just another example.
By the way, why do we use the letter c for speed of light?
It comes from the latin word celeritas, which means swiftness.
And we get the modern word acceleration, deceleration from that.
OK. All right.
So the Bohr model, we've now rolled it all out.
We have the energy portrait.
We have the radii, discrete.
We have quantization.
And we have velocities if we ever want to look at those again.
Now what's the next thing we do in science?
We compare the predictions of the Bohr model with data.
Are there any data to support this?
Because remember, all Rutherford said was plum pudding doesn't make sense.
Instead I'm going to concentrate the positive mass in the center.
And then Bohr came along and said, not only is it going to be a planetary model, I'm going to have circular orbits.
So now we've gone a long way from Geiger-Marsden.
So is there any data?
Well there were data in 1853.
Remember, Bohr published this in 1913.
In 1853 there was a spectroscopist by the name-- I'm going to tell you his name-- in Uppsala, Sweden.
And his name was Angstrom.
Angstrom was doing experiments on hydrogen in gas discharge tubes.
So he measured emissions from gas discharge tube.
And it was filled with various gases including atomic hydrogen.
And in order to take his data, what he used was this device here-- there's the Bohr radius, just an example.
He used the prism spectrograph.
So here's a gas discharge tube.
And I'm going to show you the physics of that in a second.
Basically you've got a pair of electrodes.
You've got gas in the tube.
And as this cartoon shows, you apply a potential across the electrodes.
And beyond a certain threshold potential, the tube begins to glow.
And the glow goes in all directions.
And a blinds you when you're in the lab.
So what you do, is you cover this up a bit.
You have a narrow slit.
And then you force the light to come through in a thin ribbon, and then expose it to a prism.
What the prism does, is it takes the light and breaks it into its components, sort of rainbow-like, and magnifies the difference.
As refraction goes, different wavelengths will refract different amounts.
And then you shoot this across the room.
There's two counters.
One is a scintillation screen and an army of graduate students who sit there in the dark.
But they're no good because you can't stick them into the publication.
You need to have data that people will rely upon.
So instead you use a photographic plate.
And if you put even a tiny, tiny angle of separation across a great enough distance, you start to get enough line splitting that you can see.
And then you go backwards.
And you know the geometry here.
And you can figure out what the wavelength must have been to go this distance, et cetera, et cetera.
And these are all color coded, not because they had color film in those days, but just to let you know that 656 nanometers, if you were the graduate student sitting there, you'd see a red line, a green line, a blue line, and a violent line.
So that's how he made the measurements.
And he published those measurements.
And so they lay.
And then the story gets a little thicker.
In 1885, there's a Swiss high school math teacher.
I mean, I can't make this stuff.
This is true story.
There's a Swiss high school math teacher by the name of J. J. Balmer.
We had J. J. Thomson.
Now we've got J. J. Balmer.
And J. J. Balmer, he loved to play with numbers.
And he was studying this set of lines.
And he was trying to come up with a pattern.
Can you see a pattern there?
410 434 486, 656, do they go squares?
Are they primes?
What's the pattern there?
So Balmer puzzled over this for awhile.
And he finally came up with the equation to represent those lines.
So he studied Angstrom's data found the pattern.
And here's the pattern that he found.
He said that those are wavelengths.
If I take instead wave number, nu bar is called wave number, which is the reciprocal of the wavelength.
So if I take the reciprocal of the wavelength, I end up with those 4 lines conforming to a series that goes like this.
1 over 2 squared minus 1 over n squared, where n equals 3, 4, 5, 6.
And there's a constant here which we're going to designate r. And the value of R-- I'm going to put that on the next board-- the value of R as expressed in SI units today, would be 1.1 times 10 to the 7 reciprocal meters.
Wavelength is a meter.
Reciprocal wavelength or wave number must be reciprocal meters.
So how this all of this support the Bohr model?
Well in order to explain it, I've got a first tell you what the physics of the gas discharge tube are.
So let's go inside the gas discharge tube and understand those physics.
So here's the gas discharge tube.
It's made are borosilicate glass.
And we fill it with gas.
And in this case, the gas is going to contain among other things hydrogen.
So this is a hydrogen gas phase atom.
And this is probably at low pressure.
And then I said I need electrode.
I'll get the electrodes inside.
So I've got to have a really good glass blower who can make a glass to metal seal, and have a feedthrough to an electrode.
So this is still a vacuum seal.
How do they get the gas in the first place?
We don't show you this in the books.
I'll tell you because I did this in my Ph.D. What you do is you have a little side tube here.
You evacuate.
This goes to a vacuum pump.
And then over here you have a gas source.
You evacuate.
Put in the gas to whatever pressure you want.
And then the glass blower disconnects.
This pulls this down to a reduced pressure, and seals it.
But the books don't show you that.
That's a secret.
So we've got a gas at low pressure.
And I've got an electrode over here and an electrode over here.
And they're connected to a variable voltage power supply.
So I'm going to put an arrow with the V meaning it's variable voltage.
I can change the voltage.
And this convention, this is the negative side.
So this means the electrons leave the power supply and go like this.
Which means this electrode will be negative.
And this electrode will be positive.
And thanks to Michael Faraday, we will call this one the cathode.
And this one we will call the anode.
Now what happens?
We start turning up the pressure, turning up the voltage rather.
Low pressure here, but electrical pressure is voltage.
The voltage gets high enough, eventually the electrons will boil off the cathode.
And this is a gas at low pressure.
And they will accelerate from rest and go all the way across the tube and crash into the anode.
And we complete the circuit.
But if there's a gas in here, some of these electrons are going to hit the gas molecules.
And when they hit the gas molecules, if they have enough energy to do so, they will cause electrons inside the gas molecules to be excited.
And if they're excited enough, the electrons will jump up to a higher energy level.
But they can't be sustained.
Because this is a ballistic collision.
It's a one of.
It's like a bowling alley.
One ball, one pin, one impact.
Now the pin is in the air.
What happens to the pin?
It falls back down.
Why?
Because gravity pulls it down.
In this case, you've got the energetics pulling the electron back down.
Now the electron goes from high energy to low energy.
And when that happens, the energy difference is given off in the form of a photon.
So I get photon emission when this falls back down.
And this photon has a wavelength.
What I'm going to show you is that the set of lines that you get from exactly this configuration using this equation give you the Balmer series.
So now you've got a model that Bohr postulated for atomic hydrogen on 1-electron atom that exactly predicts that set of 4 lines.
Which were measured 50 years before.
So let's go.
So first of all let's get the energy here.
And I'm going to get the energy of this electron.
This electron I'm going to call a ballistic electron.
Why do I call it a ballistic electron?
Because it's not bound.
It's free.
It boils off the cathode, flies through free space, and crashes into an anode.
Clearly it's not part of an atom.
But there's a second electron in this story.
And it's the ground state electron in hydrogen.
And it lives here.
So what's the energy of the ballistic electron?
Well that's just 1/2 mv squared And where did it get its energy from?
It got its energy from the power supply.
And what's the electrostatic energy?
It's the product of the charge on the species times the voltage through which it was accelerated.
So away we go.
I know the charge on the electron is minus E. Whatever the voltage is there, 1 volt, 10 volts, 100 volts, whatever.
Away we go.
By the way, I'm going to show you just one other thing in terms of order of magnitude.
The kinds of voltages you see along here are 1 volt, 10 volts, that sort of thing.
So suppose, to get an order of magnitude, suppose we had a species of charge E. So in other words, it's only 1 times the elementary charge.
A species of charge E influenced by voltage of 1 volt.
So this is 1 in the voltage units, and 1 in the elementary charge units.
How much energy would that be?
That's equivalent to making 1 volt and accelerate an electron from rest across this gap.
And the result would be, the energy then would simply equal 1.6 times 10 to the minus 19 coulombs times 1 volt.
And what's the energy going to be?
Well I've got coulombs times volts.
And I don't know how I convert one to the other.
I don't have to.
Why not?
Because that's an SI unit.
And that's an SI unit.
This is an energy.
So with impunity, I write 1.6 times 10 to the minus 19 joules.
That's the beauty of SI units.
So that's a good news.
I know it's joules.
The bad news is I hate this number.
It's a stupid number, 1.6 times 10 to the minus 19. it's crazy.
Why don't I come up with a number like 3, 7?
So what I could do, is I could define.
I could define a unit such that when the elementary charge is accelerated across the unit voltage, I would call that unit 1 electron volt.
And so somebody thought of this before me.
And hence, this is the unit of the electron volt.
It takes these crazy things that we've been spewing here up until now, and rationalizes them into numbers that people can carry around in their heads.
So what's K now?
K is 2.18 times 10 to the minus 18 joules.
Yuck!
Let's convert that to electron volts.
So I divide by 1.6 times 10 to the minus 19.
And I got 13.6 electron volts.
You'll remember that on your death bed, ionization energy of atomic hydrogen.
Maybe we don't have to give out tables of constants.
You just know this stuff.
It's OK. All right.
Now one last thing about this.
So this has got gas in it.
This is the cathode.
And this beam of electrons, back in the 1800s, there was a popular term, it was called the ray.
So instead of a beam of light, people refer to the ray of light.
So then when they got to particle beams, they talk to them as rays.
So this is now not an electron beam, it's an electron ray.
And it comes off the cathode.
And it's in a vacuum tube.
So this could be called a cathode ray tube, a CRT.
Now see, I could flatten this.
And I could spray it with phosphors.
And then I could put some charge plates here.
The electrons have a negative charge.
So if I charge these plates, and I was clever about how I charge them and varied the charge, I could raster the electron beam like this about 30 times a second all up and down the screen.
And then I could put some program signal in there.
And I could sit here.
And I could watch TV.
It all started with the gas discharge tube.
It says nothing about the content unfortunately.
Very nice physics, but no content.
All right.
So now we've got the electron.
The electron is moving, the ballistic electron.
Now I want to look at what happens when the ballistic electron smashes into one of those hydrogen atoms.
So let's go back over here.
So here's the incident particle.
And it's going to be an electron in this case.
So I'm going to designate this electron.
This is my ballistic electron.
So here's the incident electron.
And this is ballistic, just to be clear.
It's the ballistic incident electron.
Now this is a mixed metaphor here.
Because I'm representing this in Cartesian space.
But I've moved into energy space.
So some people are going to get really upset.
Because they're going to say, well this is Cartesian, but this isn't.
It doesn't matter.
It's my lecture.
It's my model.
It works.
I'm the professor.
So we've got this mixed metaphor here.
But anyways, it helps a lot.
So what happens when this thing comes in?
It depends on how much energy it has.
Now if the incident energy, if E incident is tiny, nothing happens.
This thing just zooms right on through.
But if the incident energy, if E of the incident ballistic electron is greater than delta E for any transition that's feasible-- and in this case I'm going to assume I don't have thermal distribution of electrons.
If I gave you Avogadro's number of hydrogen atoms because of the thermal distribution of energies-- and we'll come back to this later-- there might actually be, at any moment, some electrons that are thermally excited above the ground state.
We're going to forget about that for now.
We're going to spiral up the learning curve here.
So first time we're going to assume all the electrons are in the ground state.
If I don't enough energy to go from n equals 1 to n equals 2, nothing happens.
If I have more than enough energy to go for n equals 1 to n equals 2, I will take that energy.
And the electron will jump, steal that amount of energy, and then this thing moves on-- and I'm purposely making this vector shorter than the incident vector-- with that amount of energy raw.
And this is called the scattered electron.
Go back to the bowling ball analogy.
The bowling ball comes in with a certain energy, hits the pin, continues to roll.
But you know that there's a loss of kinetic energy in the bowling ball.
That's what we're seeing here.
It's purely ballistic.
Now let's say we do have more than enough energy.
Suppose I have enough energy to go from n equals 1 halfway between n equals 2 and n equals 3.
There's only n equals 1, n equals 2.
I can't take the electron up to n equals 2.3.
It's unallowed.
These are the only allowed states.
So that differential amount of energy then resides with the electron that's ballistic.
And it moves on here.
So we've got conservation of energy here.
We can say that E incident will then equal the energy that's lost in the transition plus the energy that's still left with the scattered electron.
And we can calculate what that transitional energy is.
That transitional energy is going to equal 1/2 mv squared incident.
This is the velocity, the incident electron.
What's the energy to go from n equals 1 to n equals n?
Whatever it is, its minus K times Z squared If it's hydrogen, Z is 1.
It's going to be 1 over nf.
The final quantum number squared minus 1 over the square of the initial quantum number.
And then, what's left over after this has been robbed from the incident ballistic energy is 1/2 mv squared of the scattered ballistic electron.
And the quantization dictates that only if E incident is greater than delta E going 1 up to n-- here I'm assuming everything is in its ground state.
Later on we're going to be more sophisticated.
But for first time through, all are ground state electrons.
I have to have enough energy to go at least to n equals 2.
I can go to n equals 3, m equals 4.
In principle, if this thing had more than 13.6 electron volts, what would happen?
It would kick this electron out.
Gone!
And you'd have 2 free electrons.
So if it's greater than this, then the consequence is electron promotion.
So we're moving along.
But this excited state is unstable.
This excited state is unstable because it's like the bowling pin that got thrown up.
So what happens?
The electron standing up there on n equals 2, for example, looking down to n equals 1.
And it falls.
When it falls it gives off radiation.
And that radiation is conservation of energy there.
And what we know is when it gives off an energy of the emitted photon, the energy of the emitted photon must equal delta E of the transition falling from 2 to 1.
And we know how to calculate that.
That's just that thing flipped around.
And this thing is equal to what?
This is equal to h nu.
According to Planck it's hc over lambda is equal to hc nu bar.
You know what this one is.
This is minus KZ squared 1 over-- in this case it's going to be-- 1 over nf squared 1 over 1 squared minus 1 over, in this case, 1 over 2 squared And you can generalize this 1 over nf.
So where am I going with this?
Well I'm going to flip all of this around.
And when I flip it all around, what I'm going to end up with is this equation here.
And better than that, I'm going to end up with this equation.
And I'm going to end up with this as the constant.
And when I get that, we're going to say Bohr has done it.
The data support the theory.
So that's what we're going to do.
But I think we're going to stop at this point today.
So let me just jump here.
I mentioned to you that if you go here on your periodic table, there's the 13.6 electron volts.
In the case of lithium, this is 5.4 electron volts.
So you can see all the various values.
This by the way, people no noise.
No noise.
It's 11:52.
I'm holding court until 11:55.
I'm simply changing topics.
It's not, oh this is the part where I can talk to my neighbor.
What are the rules?
No talking.
No food.
No horseplay until 11:55.
Then, still no horseplay, gentle talking, no food, no drink.
This is a sacred space.
I'm not kidding you.
Do you know why?
In this secular America, this is sacred space because this is where people learn.
The lecture hall is sacred space.
Now, here's the 13.6 electron volts.
And there's the 1.6 times 10 to the minus 19 joules.
And if you multiply those 2, you'll get the 2.18 over here.
All right.
Here's a cartoon showing the photon, higher energy orbit, lower energy orbit, electron emission transition, right out of your book.
And there's a postulate 6.
And we're going to finish this up at the beginning of the Friday lecture.
So here's the whole series from n equals 3 to 2, n equals 4 to 2, and so on.
See how that works.
OK. One of the things that we've learned here, is that it doesn't matter what the incident energy is here.
The emission is characteristic of the energy levels inside the gas.
Instead of using an incident electron, I could use an incident proton.
I could use an incident alpha particle.
I could use an incident neutron.
Anything that has enough energy to kick this up from n equals 1 to n equals 2 will result in photon emission of this frequency.
That means the set of those lines is unique to the target.
And this is the beginning of chemical analysis.
I can use this to characterize species, to it, stars.
How do we analyze the composition a stars?
Well, do we send a NASA spaceship out 25 light-years and grab some gas and bring it back to the lab?
No.
All we've got is the spectrograph.
The star is hot.
That means thermal excitation and cascading down with photon emission.
And the lines we get are related to the energy levels within the stars.
So from a distance of a 100 light-years, I can tell you what the composition is.
And if there's two gases there, what if there's hydrogen and helium?
the helium lines will be there.
And they'll be superimposed on the hydrogen lines unless they lie directly on top of one another.
I'm going to be able to figure out what's there.
That's how it works.
Now here's a story about an astronomer, Cecilia Payne.
She's the first woman graduate student in astronomy at Harvard.
She went on to chair the Faculty of Arts and Sciences, awarded tenure, but denied a professorship for 18 years because she was a woman.
And in her thesis, she was the first person to figure out to the sun is dominantly hydrogen, not iron, which is what most astronomers thought.
Why?
Well, because the earth is made of iron.
Meteorites are made of iron.
The whole universe must be made of iron.
Never mind the fact that the sun is glowing.
So here's how spectroscopy works.
Look at this.
See, what does this mean?
This is an analogy.
What could that mean?
Well, they said iron again.
I know this is misspelled.
But maybe it's just a glitch in the instrumentation.
So most people would look at that and say, the message is iron.
But it's not about the word.
It's about the pattern.
It's about the pattern.
And that's what's spectroscopy is.
By the way, if you somehow didn't catch this lecture.
And you walked in, and all you saw was those four lines.
You know those four lines.
That set of lines is characteristic of atomic hydrogen, and nothing else.
By the way, those lines are very faint.
This is not to scale.
And they're so faint, they're ghostlike.
And what is the latin word for ghost?
Specter.
So what is a spectrum?
It is a set of ghostlike lines.
To this day, the term spectroscopy refers to the ability to study data that are so faint they're ghostlike.
All right, we'll see you on Friday.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, a couple of announcements.
Next week two minor celebrations, maybe receptions we would call them.
Quiz 2 Tuesday based on homework 2, and periodic table quiz Thursday.
I provide the numbers, you provide the letters.
I guess that's how it works.
And the contest ends Friday five PM.
Last day we looked at the Bohr Model and we developed equations for the radius of the electronic and the orbit of the one electron atom.
The energy of the electron and the velocity of the electron.
And we found that for all of these they were a function of n. Quantum number.
n takes on discrete values.
One, two, three, and so on.
We say that these energies radii velocities are quantized.
They take discrete values.
And then later in the lecture we started looking for evidence.
And we found ourselves in an exercise of reconciliation with data taken by Angstrom about 50 years earlier and fit to an equation by J.J. Balmer.
And we were part way through that and adjourned.
So I'd like to pick up the discussion at that point.
I've done a different drawing of what's going on inside the gas discharge tube.
Last day I had the ballistic electronic here, and this is boiling off the cathodes.
The cathode is inside the gas discharge tube and this electron, if the voltage is high enough, will leave the cathode and shoot across this low pressure gas, which contains, among other things, atomic hydrogen.
And I'm trying to depict the atomic hydrogen atom here.
Here's the proton, which is the nucleus-- that's the sum total of the contents of the nucleus-- and here's the lone electron that is orbiting the nucleus at some initial value.
n sub i.
Could be ground state.
Doesn't necessarily have to be.
With some thermal energy this could be n greater than one.
Then we reason that if the electron, ballistic electron, and it's trajectory across the gas discharge tube over to the anode, which is charged positively.
If it collided with this electron it could impart some of its energy, thereby promoting the electron from ni up to nf, the final level.
And the electron would be up here.
And for this transition there would be an energy cost.
That energy cost is delta e. Delta e is the energy to go from ni to nf.
And so the kinetic energy-- half mv squared of the incident electron-- is diminished by this amount.
And the electronic continues on it's merry way at a slower speed.
We assume it's mass doesn't change.
The only way we can change its energy is to slow it down.
And there's a conservation of energy, so the sum of the energy of the scattered electron and the transition energy of the electron within hydrogen must equal the incident kinetic energy of the ballistic electron.
But there's more.
This is a-- no pun intended-- a one shot deal.
This is ballistics, and so the electron is not sustainably promoted.
It falls back down.
And when it falls back down we have the transition energy now given off.
Here, to promote we had to call for energy.
When the electron falls down it gives off that energy.
And that energy is given off in the form of an emitted photon.
And it's that emitted photon and it's that emitted photon that ultimately gives rise to the lines.
The lines in the spectrum are generated by the emitted photon here.
Everything else is preamble to this event, and this event gives rise to the emitted photons.
And I think that's about where we got last day with the reconciliation.
So let's look carefully here.
We recognize that there needs to be conservation of energy again.
In other words, the energy of the emitted photon, the energy of the photon, which we know from Planck is h times nu.
I'm trying to distinguish.
This, I'm making nu, the Greek symbol nu.
And I put a little descender on it.
It looks like a v, but I put a little ascender here to distinguish it.
This is lowercase v, as in mv squared.
This is nu.
So h nu, or it could be hc over lambda.
Or it could be hc nu bar.
Three ways of writing the energy of the photon.
And that must equal delta e of the transition.
So let's keep going.
We know that the delta e, the transition is given by the Bohr Model.
Delta e transition will equal e final minus e initial, which will be minus kz squared-- I'm writing this generally, in this case with atomic hydrogen z as 1-- with minus k times 1 over n final squared minus one over n initial squared.
What we can do then is equate these and roll them around to isolate nu bar.
nu bar, then, equals minus kz squared over the product of the Planck constant, the speed of light, 1 over nf squared minus 1 over ni squared.
Now for the Balmer series, that is to say the Balmer series of lines, that turns out to be a series where all of the transitions end up on n equals 2.
We said nf equals 2. z equals 1 because we're talking about atomic hydrogen.
Then what we have is a set of translations that go 1 over 2 squared minus 1 over ni squared.
ni must be greater than nf, so ni must come from the set 3, 4, 5, et cetera.
Furthermore, I'm going to put z equals 1.
Let's evaluate k. We know that's 2.18 times 10 to the minus 18 joules, or 13.6 electron volts.
And we know the Planck constant, 6.6 times 10 to the minus 36.
And this is 3 times 18 of the 8th all in SI units.
So this gives me 1.1 times 10 to the 7th in reciprocal meters.
And if I put all this together I end up with exactly they equation that was published by Balmer.
Exactly Balmer's equation in 1885, rewritten to express it in SI units.
1 over 2 squared minus 1 over-- I'm just going to put ni squared-- or ni equals 3, 4, 5, 6.
This is Balmer exactly.
Balmer exactly.
So the assumption of this planetary model, with all of the restrictions that Bohr placed on it in order to get this set of equations, reconciled with laboratory data.
Very, very significant.
So here we are.
Those four lines all can be derived from the Bohr model.
And here's another cartoon from your book and showing what I'm trying to depict here, namely it's the falling down, the return to the state from which the electron was promoted that generates the photon.
And the set of those lines is what gives you this.
There so there's the validation of the sixth piece.
So Bohr Model agrees with Angstrom's data, but it also suggests other experiments.
Let's think about this for a second.
OK, here's another cartoons from your thing.
You know, I told you that this thing is unstable.
And in the Balmer series it goes from n equals 2 up.
But there's a ground state, n equals 1.
What was wrong with those electronics in Sweden in 1853 than Angstrom could never find any electron that would fall all the way down to the ground state?
What's wrong with it?
Well, here's the answer.
It has to do with instrumentation.
So this is an example where science goes further thanks for the advent of new instrumentation that allows this to make measurements that previous people couldn't make, even though they were very competent experimentalists.
Angstrom could have found n equals 1 series, but couldn't see them because he was using a photographic plate.
This shows you the range of sensitivity for photographic plates.
Here's the electromagnetic spectrum.
Out here you have low energy radio waves and up here you have x-rays and gamma rays and so on.
And the visible spectrum is parked right here in the middle, and here it is unpacked for you.
And it roughly runs from 400 to 700 nanometers.
That's the invisible spectrum.
So wavelength increasing from left to right, which means energy frequency in wave number increase from right to left.
They're complimentary, right?
e nu, nu bar on the top, lambda is on the bottom.
Some spectra are plotted in lambda, some are plotted in wave number, whatever.
And by the way, I want to show you the power of knowing a few things.
I don't expect you to know a lot of facts, but I expect you to know a few things.
Every educated person ought to know that the visible spectrum runs round numbers 400 to 700 nanometers.
But look, I can take 700 nanometers and use this formula and convert it to energy.
And I'm going to get something like 3 times 10 to the minus 19 joules.
Yuck.
Instead, I go in electron volts.
1.8 ev.
Over here, 400 nanometers is 3.1 ev.
So round numbers, the visible spectrum spans 2 to 3 electron volts.
Our eyes are photo detectors that operate on the band width 2 to 3 electron volts.
that's easy to remember.
Those are good numbers.
So where does that leave us?
It leaves us here.
We go back and we see these numbers, 656, 486, 434, they're all in the visible spectrum.
So I went and I did a little calculation.
I said, well what would I have if we'd gotten the wave length for the transition from 2 down to 1?
This is n equals 2 down to ground state.
If you plug in the numbers to the Bohr model, you'd find that that would give you 122 nanometers.
122 nanometers?
Well, 122 nanometers is going to put you way over.
It's too high energy, right?
122 nanometers is going to put you off to the left there into the ultraviolet, where the photographic film was not sensitive.
So he couldn't measure those lines.
So now I'm going to end by putting the master equation that captures all of this.
And the master equation that captures of all of this is here.
It's that nu bar goes as r times z squared, 1 over nf squared minus 1 over ni squared.
So this is the most general form for all 1 electron atoms.
That's why I've got z squared in there for all 1 electron atoms.
And this is called the Rydberg equation.
Named after another Swedish spectroscopist at the University of Lund.
I think a Swede would probably pronounce this something Rydberg, but you don't have to say that.
You can just say Rydberg and it'll be fine.
And in honor of Rydberg, the constant here is given the symbol, capital R. The capital R as the Rydberg constant and it has a value of 1.1 times 10 to the 7th reciprocal meters in good ST units.
Well, there was more evidence for the support of Bohr's Model.
More evidence for the support of Bohr's Model.
By the way, as the detectors got better and better we could get more and more lines.
You see these, as you get higher and higher series ending on higher and higher end numbers, you move off into the infrared.
Because this is not to scale.
These n equal 4, n equal 5 are closer and closer to closer together in terms of energy.
They're farther and farther apart in terms of spacing, but they're closer and closer together in terms of energy.
Because they're farther from the nucleus.
You say, gee, shouldn't it cost more energy to go farther?
Uh-uh.
Because you're farther from the positive nucleus.
Be careful.
Don't let your intuition send you in the wrong direction.
It's all about Coulombics.
Anyway, so the Lyman series ends at n equals 1.
And these are different scientists.
Paschen, Bracket, Pfund, Humphreys, and so on.
So maybe, I don't know, if somebody hasn't claimed n equals 214, all the lines that end there, you know, maybe that could be your name on the series.
As if anybody cares.
Looks like this quantum condition is validated.
See this is really important because this was the big break away from classical theory.
That the motion of a body, something with mass, could be quantized in its behavior shook this physics community.
But this reconciliation of the data says that assumption is valid.
There's more that happens.
So in 1913 in Berlin-- remember 1913 is when Bohr published the paper-- 1913 in Berlin there was James Franck and Gustav Hertz.
James Franck and Gustav Hertz.
And they were conducting experiments on gas discharge tubes.
Only they filled a gas discharge tube, instead of with hydrogen, they filled it with mercury vapor.
So gas discharge tube-- GDT-- gas discharge tube containing mercury vapor.
The same thing.
Put the electrodes, connect on a power supply, and started varying the potential.
So I'm going to show you what they found.
This is the Planck voltage, and this is the current between electrodes, or if you like, across the tube.
Between the electrodes, or through the tube.
Or if you like, tube current.
Meaning from one electrode to the other.
The tube current.
Well, low voltage, low current.
High voltage, high currently.
They get up to a certain value of voltage, all of sudden the tube starts glowing blindly and the current falls to 0.
Then they continued to raise the voltage.
More voltage, more current, up, up, up, up, up.
And then they get to another critical value of voltage, even more intensity.
And then the current falls to 0.
So you look at those data and say, well, what's that got to do with the Bohr Model?
Because mercury is not a 1 electron atom.
It's got a boat load of electrons.
This is not a 1 electron atom.
So you say, I know what it is.
It's ionization energy.
Must be ionizing the mercury.
So you go to the periodic table and you look up the ionization energy of mercury and you discover that that's 10.4 volts.
10.4 electron volts is the ionization energy.
And this first null is at 4.9 volts.
Well, 4.9 is a long way from 10.4.
And this second null occurs at 6.7 volts.
6.7 volts.
So what's this telling us?
What this is telling us is that when you get to a value of 4.9 volts, you've hit a certain value that allows you to promote electrons within mercury between one level and the next level.
And those electrons are being promoted and then cascading down.
And they're cascading down and they're emitting in the visible and it's blinding you.
Say, OK, so what does that mean?
Well, it means that the Bohr Model, which is for a 1 electron atom, assumes that energy levels within it are quantized.
These data indicate that on the basis the behavior of this gas discharge tube, there must be quantized energy levels inside of mercury, which means all atoms have quantized energy levels.
You understand?
Everything is quantized.
That's really powerful.
It starts off with this nerdy little 1 electron atom, and now he's applying it across matter.
And this is gas, this is more elaborate gas.
Heaven forbid, it might exist in liquids and solids.
So that's the Franck, Hertz experiment.
So his stock goes way, way up as a result of that.
And they win a Nobel Prize.
Here's James Franck.
Here's Gustav Hertz.
You know what the Hertz is.
200 kilohertz, so on.
That's Hertz.
James Franck was at Gottingen when he won this, but he ultimately came to the United States when the political changes started occurring in the thirties in Germany.
Franck decided to seek safer surroundings and ended up at the University of Chicago, where there is to this day the James Franck Institute of Physics.
Very, very high-end physics institution.
So this is good.
But all good things come to an end.
So 1913 was a bittersweet year for Bohr.
Because he got some good news, but he also got some bad news.
So now I want to move over to limitations of the Bohr Model.
Limitations of the Bohr Model.
So I know what you're going to say-- well, it only talks about 1 electron atoms, so that's a limitation.
No, there's more to it than that.
Even the 1 electron atom model doesn't capture everything.
I'm going to summarize the limitations.
I'm going to show you three, and they all fall under the general umbrella of fine structure.
Fine structure.
In other words, the Bohr Model is good, give us the big lines, but when you start looking more carefully it fails to capture some of the physics.
So first of all, let's go back to some earlier data.
1887.
1887, there were already data out there that were going to give heartburn to the Bohr Model.
And those data were taken by Michelson and Morley.
Michelson and Morley.
Everything I've taught you so far, with one exception, has been European science.
Americans were not active in science because this was a young country.
We were really good engineers because we were blockaded by the rest of the world.
We had to live by our wits-- that's where you get the term "yankee ingenuity."
Science was hifalutin stuff.
We didn't have time for it.
But towards the latter half of the 19th century, we started moving into fundamental science.
The first American to win the Nobel Prize was Michelson.
Michelson was doing work at Case in Cleveland, which eventually became Case Western Reserve University.
So he was at Case in Cleveland and he was studying optics.
And he was a brilliant experimentalist.
In fact, he made the first reliable measure of the speed of light.
Back before 1900.
What they were doing is they were looking at Angstrom's lines and they noticed something peculiar.
If you take a look at even this drawing, you notice the red line is a little bit fatter than the others.
Now you might just say, well, that's just the artist taking liberties and somebody didn't catch it in proof reading.
But in point of fact, what he found was that if you look at that line, which is really the line for the 3-2 transition-- the 3-2 transition in the Balmer series-- what you find is that if you look at the photographic plate more carefully, you find that this thing in fact is a pair of lines, but very, very closely spaced.
This is known as a doublet.
Two lines very closely spaced, centered at 656 nanometers.
And with his interferometer he gets super, super good data.
And he could split the doublet.
Well, what's that mean for Bohr?
Bohr has no way of explaining this.
If you look at the Bohr Model, you've got n equals 2, you've got n equals 3, alright?
So this is energy 2, energy 3, right?
And so when the electron falls from 3 to 2, we get a photon of a certain value.
It's going to be nu 3 to 2.
That's the frequency or wave number, what have you.
Now, the fact that you've got a doublet here means that there must be two transitions, but darn close.
There's either a 3 and a 3 primed, or there's a 2 and a 2 primed, but it's not simply 3 and 2.
So that piece of information runs counter to the Bohr Model.
Bohr Model is silent about it.
It gets the big picture, but if you look more carefully it can't capture the doublet.
And Michelson ultimately gets the Nobel Price.
And I think I've got him here.
There he is.
The Nobel Prize.
By the time he got the Nobel Prize he was at the University of Chicago, but he did the work that won the Nobel Prize for him at Case.
So sometimes when you see even Millikan, Millikan did his work at University of Chicago, but eventually took a position at Caltech.
So the Nobel Prize says, Robert Millikan, Caltech.
But he didn't do that work at Caltech.
He did it at Chicago.
Anyways, you can go to the Nobel website.
You can read about these people.
And what's really cool is when you win the Nobel Prize-- you notice I didn't say if-- I say, when you win the Nobel Prize, what you do is you get on an airplane, you go to Stockholm, and then you go and you have dinner in this beautiful hall.
I've been there and it's gorgeous, gilded and so on.
Very nice kitchen, excellent wine list.
And-- yes-- and you can go there and they serve meals.
the menu is taken from previous Nobel Prize dinners.
So you can sit and-- whatever it is, it could be the Nobel Prizes of 1927 and that's what's going to be on the menu today-- and after the dinner they have a presentation ceremony with the King of Sweden.
You get your Nobel Prize, and then people listen to your lecture.
And those Nobel lectures are really, really expository.
So if you want to go and read the Nobel lecture that Michelson gave on the occasion of winning the Nobel Prize, you'll probably learn all of about this and more.
It's really, really good, so go there and read.
Now back to the story.
Second problem with the Bohr Model.
1896-- see, all this data had been accumulating-- 1896, there was a postdoc by the name of Zeeman.
Piet Zeeman.
He was a postdoc at Leiden.
Leiden in Holland under Lorentz.
You'll learn about the Lorentz force when you study 802.
What he was doing-- again, gas discharge tube.
So this was gas discharge tube, and what Zeeman was doing on his postdoc was in a magnetic field.
These people were doing all sorts of experiments.
They were trying to block out the whole experimental space.
So one guy, his specialty is high energy.
One guy's specialty is low pressure.
These people are taking a gas discharge tube and putting in the jaws of a powerful, permanent magnet and then measuring the spectrum.
And what he found was that for certain lines, this was the rest-- b, I'm going to use as magnetic field-- in the absence of applied magnetic field you have a line.
And this is not a doublet, triplet-- it's just a plain old line.
Well behaved line.
But when they take that gas discharge tube and put it into a magnetic field, they see a plurality of lines.
And furthermore, the spacing-- I'm going to use c here-- the spacing in the lines is proportional to the intensity of the magnetic field.
No magnetic field, single line.
Modest magnetic field, modest amount of what is called line splitting.
So a modest amount of applied magnetic field, modest splitting.
Intense magnetic field, intense splitting.
Bohr Model is silent about that.
Because you know, if you've got different lines, it means you must have different energy levels.
It's as though the energy level diagrams opens up in a magnetic field.
The Bohr Model can't account for that.
And parenthetically, they got the Nobel Prize, too.
So there's Piet Zeeman.
Got his PhD in 1896.
He's got his Noble Prize, 1902.
He's off to a good start, I'd say.
And there's Lorentz.
Two of them.
We'll get to him in a second.
So third piece of bad news for the Bohr Model.
And that comes, again, in 1913 in November.
In November of 1913 there was a man by the name of Stark in Germany.
And Stark was doing analogous experiments.
He was studying gas discharge tube in electric fields.
Obviously, you've got an electric field across the electrodes to excite the electrons.
But he's taking a whole gas discharge tube and putting it between flights and then applying an electric field.
And what did he find?
He found the same sort of thing.
He got line splitting in an E field, and furthermore that extent of splitting, extent dependent upon the intensity.
E intensity.
So no field, no splitting.
Modest field, modest splitting.
Intense field, intense splitting.
Well, again, that's a headache for the Bohr Model.
So this is all three problems, and it's all under aegis of fine structure.
So we know the Bohr Model has its limitations.
OK, Stark.
I know he's got his Nobel Prize.
There he is.
So 1913 ends on a sour note.
But people don't give up.
1916, Arnold Sommerfeld in Munich.
He was a professor of physics and he proposed modifications.
Modifications to Bohr Model.
It's a patch, we would call it a patch.
going?
To put a patch on the Bohr Model.
And what's he going to do?
What's the gist of his idea?
Well, he retains the planetary structure.
He liked that idea-- nice orbits, so on.
But he took a page out of Kepler's book.
The planets in the Kepler model, when they revolve around the sun their orbit is not circular.
It's elliptical.
So Sommerfeld said, why don't we give that a try?
What if we said the electronic orbit can be elliptical or circular?
And he was quite specific.
He said, suppose-- and this, again, is not to scale, but to emphasize this is going to be elliptical or circular, but very, very mild eccentricity.
What I'm going to draw for you is extreme eccentricity to make a point.
But suppose we had the circular orbit as I'm drawing it now, and then we had an elliptical orbit that is centered on that circle.
So it's mild eccentricity.
We might have another one-- let's do one more.
This is good enough.
The gist here is that we have a circular orbit and an elliptical orbit, but the bandwidth here is very, very narrow.
So this is very, very thin.
And it's sort of like an egg shell.
So if I asked you, what's the dimension of an egg?
You'd say, well, it's dimension of the surface of the egg.
Then I'd say, but the egg shell has some thickness, right?
But that thickness is relatively small in comparison to the total dimension of the egg.
So an analogy.
He said that the range of distance from the nucleus, whether it's circular or elliptical, is very, very narrow.
So we can say the set of circular and elliptical orbits lie within a shell, as in egg shell.
So this is a shell model.
It's a shell model.
So now how do you designate the different orbits?
You've got some that are circular, some that are elliptical.
He needs to distinguish them and he needs to be able to label them.
So he introduces new quantum numbers to allow us to name them.
So let's go and take a look at the quantum numbers that Sommerfeld gave us.
So he starts off with n. He retains that from the Bohr Model and he calls that the principal quantum number.
And it's primary attribute is size.
It captures the distance, the principal r from the nucleus.
And it takes values 1, 2, 3, all the way up to infinity.
So n equals 1, small radius.
n equals 10, large radius.
Oh, by the way, there's another numbering system.
This is what we use, but the spectroscopists use letters.
The spectroscopists use letters-- why?
Because remember the Balmer series?
Everybody was hooked on the Balmer series and it ended up being n equals 2.
And then later with better detectors we find there's an n equals 1.
So the spectroscopists said, we're going to get fooled again.
So we're going use letters.
And we're going to start with the letter k. It's in the middle of the alphabet.
That way if we discover even lower energies, we've got some head room here, we can label those.
But we never found any.
So if you go over to Building Thirteen and you do some x-ray refraction and you use the line that emanates from a copper target, n equals 1-- it's called the k alpha line of copper.
To this day.
So k, l, m, and so on.
You can't get to infinity, obviously.
You know, I didn't think this thing through.
Now the l. l is, what's his name?
Sommerfeld.
And it's called the orbital quantum number.
Why?
Because he said that the electron is in an orbital instead of an orbit.
Orbit is Bohr, orbital is Bohr-Sommerfeld.
And it speaks to the shape.
Somehow, I've got to distinguish between elliptical and circular.
And it takes values 0, 1, up to n minus 1.
So the n number controls the range of l. And again, the spectroscopists, they're real number weenies, they're afraid, so they use s, lowercase.
See this is uppercase, this is lowercase.
s, p, d, f. For sharp, this is the sharpest line from the l equals 0-- then the principal because as you go to z they all seem to converge and look like hydrogen-- d is diffuse, f is fine, and then after that they ran out of ideas so g and h. So you'll talk about the one s-orbital, meaning n equals 1, l equals 0.
And there are some values here for shapes.
I'm going to put that right above it.
When l equals 0, l equals 0 means you have a circular orbit.
And when l equals 1 it's elliptical.
And when l equals 2 it's much more complex, and we'll just leave it at that.
1, 2, and 3.
So there's l values.
And then m is the magnetic quantum number.
And it talks about orientation.
I'll show you what I mean by that in a second.
The values are governed by l, which is governed by n. Starts at l, l minus 1, goes down through 0, goes to minus values and ends at minus l. So for example, we could do something like this-- when n equals 1, then l most equal 0, so therefore m must equals 0.
So this means for n equals 1, it's only a circular orbit and this thing is going to be immune to line splitting in a magnetic field.
When n equals 2, l can equals 0 or l can equal 1.
When l equals 0, m equals 0.
That's boring, that's circular.
But here's another possibility.
And that is, when l equals 1 then m can equal 1, 0, and minus 1.
Now I said it has something to do with orientation.
Most of quantum mechanics doesn't translate into the Cartesian world, but this one does, mercifully, and I think it's a cute analogy.
If I were to tell you that I've got three different quantum numbers and I've got an elliptical thing-- and one way to think, see, the number 0 looks like a circle and the number 1 has some asperity associated with it, so you can think of that as the ellipse-- so I know that I can, with no prior knowledge of where the true origin of the universe it is, I can arbitrarily define a set of rectangular coordinates, orthogonal coordinates, x, y, and z.
And that means I could put one orbital here, one orbital here, and one orbital here.
So those are three orthogonal orientations, which I think is consistent with the fact that m takes on three values.
OK, that's cute.
So that's as far as Sommerfeld went.
I'm going to go and do something as a retronym.
I want to get the fourth quantum number up here now, but we're going to pause the story.
We're going to fast forward to 1925 so I can get the last quantum number up here.
And that's called the spin quantum number.
And it takes values plus or minus a half.
Where did that come from?
Well, in 1922-- oh, you know everybody's getting Nobel Prizes and I didn't give Niels Bohr his proper recognition.
He gets the Noble Prize, as well.
Oh, when Sommerfeld turned 80, they had a symposium in his honor and they published a book.
And the book had papers and well-wishes, papers that were given at the symposium.
And in the front they had Sommerfeld's picture and they also had this twin picture, this diptych.
So on the right is Sommerfeld, and on the left is the same picture but they've morphed it.
Now remember, there's no Photoshop.
Horrors, there's no Photoshop.
Can you imagine?
So how could they do this?
They had to take the negative, which was a photographic plate, and when they were printing the negative using a light box they had to hold the negative on an angle to get the distortion.
And in holding it on an angle to get the distortion, they turned this image into something that was a little more spread out.
And the caption that went with this, "To Arnold Sommerfeld, who taught us that the circle is the degenerate form of the ellipse."
Now that's geek humor.
I mean, they laughed.
They thought that was so funny, hahaha.
You know.
They were having a great time.
It was Germany, and nineteen twenties.
And there he is.
OK, so now let's go to 1922.
This is the Stern-Gerlach experiment.
Very interesting experiment.
This is really physical vapor deposition.
Over here I've got a crucible and it's full of molten silver.
So Stern and Gerlach were studying the magnetic behavior of liquid metals.
So what they were doing is they had this-- over here you see it's red even though it's silver, because this is that about 1,000 centigrade.
Silver melts at about 960.
Everything, I don't care what it's color is at room temperature, at 1,000 degrees it's red.
It's called red hot.
All right.
So this is red hot silver and there's a vapor here and there's a slit and the silver atoms come out of the slit.
And they go across over here to a substrate and then they pause it on the substrate.
So making little band of silver on the substrate.
And furthermore, he sometimes put them through a magnetic field.
So he's got a slit here that narrows the beam, and then he sends it through a magnetic field that is asymmetric.
It's divergent.
Can you see here?
Look at the end.
The south pole as a tip and the north pole is this arc, this cup.
So the field lines don't go just directly from tip to tip, they go from tip off to the side.
So you can see the divergence of the magnetic field.
And so he looked at what kind of deposits he got as a function of the magnetic field.
Here's what the observed.
Very puzzling.
The whole thing was about Maxwell's equation.
So he's got a silver beam.
And when it went directly from the furnace to the substrate he just got the shadow of the slit.
And they have the split crosswise with respect to the divergent magnetic field.
So if you look at the substrate you just see a band.
So this is a band of silver and you can imagine there was a slit out here and it just cast a shadow and that's the band of silver.
This is PVD, physical vapor deposition of silver.
Now when b is not equal 0, what would you expect?
You'd think the beam would bend, right?
So what do you think happens?
The beam bends up?
The beam bends down?
Or beam bends to the right or to the left?
Think about it.
I don't want to hear your answer.
Think about it.
What do they observe?
What they observe is, if this is where the original one is.
Two.
The beam splits in two.
And it gets two deposits, one above, one below, of equal intensity.
That's a problem.
Beam splitting.
But now it's a beam of matter.
Beam splitting.
Boy, they had them scratching their heads on that one.
No way to explain that.
So along come a couple of graduate students.
1925, couple of graduates.
So this is 1992 in Frankfurt.
1925, two graduate students in Leiden, again.
Gaudsmit and Uhlenbeck.
They're just like my TA's.
Grad students.
And they looked at this thing, I don't know, maybe sitting around over a beer one night, and they said, you know, so far what we've been saying is the electron revolves around the nucleus.
And sometimes it revolves in a circular orbit, and sometimes it revolves in an elliptical orbit, but here's the electron revolving.
And they said, what if in addition to revolve, the electron rotated so that it's going like this?
[GESTURES] But there's two choices.
It can be going like this, or can be going like this.
[GESTURES] Now, it's a charged species and it's rotating, which means that it's going to have a magnetic moment depending on rotation.
And now I'm going to send it through a divergent magnetic field.
Doesn't it follow to reason that if I put it through a magnetic field and I've got some of them doing this and some of them doing that, they're going to go in different directions?
Opposite directions?
And what do you think the numbers are?
If I give you Avogadro's number of silvers, you think I'm going to get a dominant clockwise and a minority anti-clockwise?
No.
We're going to get equal numbers.
Some are going to spin like this, some are going to spin like that.
And you're going say, but electrons don't spin, they're not doing this.
But if you model them as though they are doing this, you get those results.
Those results make sense.
And so they introduced the spin quantum numbers.
And I think these are the ones that have been erased.
Such is education.
OK.
So you know it.
S plus or minus a half.
By the way, Gaudsmit and Uhlenbeck were here during World War Two.
They worked in Building Four.
You go down the corridor, Building Four just off the Infinite Corridor, there's a plaque there for the Radiation Laboratory.
That's where they worked, in the Rad Lab.
That's where radar was first engineered.
There was work in the UK, there was work in other places.
But this is the Radiation Laboratory, started here and they were both here at the time.
OK, well, I think that's a-- So this is a plate from the paper.
No magnetic field with the magnetic field.
And by the way, why did they choose silver?
They chose silver because it's atomic number is 47-- it has an odd number of electrons.
You're going to learn later that you get two electrons in an orbital, and if you have two electrons, one will be spin up, one will be spin down.
There's no magnetic moment.
So they were clever about choosing an element that had an odd number of electrons so that there would, at the end, be an unpaired electron.
And there's Otto Stern with his Nobel Prize.
And he came to the United States, as well.
And you're going to see the ascendancy of American Science as people flee Europe up in the nineteen thirties.
And America is the beneficiary, and then you see American science rise.
But for now it's European science.
OK, so I'm going to talk a little bit about hydrogen and transportation.
And we're going to talk about the Hindenburg because it was full of hydrogen.
And to give you a sense of scale, this is what a 747 would look like and is what the Titanic would look like.
It was almost as long as the Titanic.
It was built in Germany by the Zeppelin company.
And the Titanic, the Hindenburg rather, was LZ129 serial number.
That's Luftschiff Zeppelin.
Airship Zeppelin.
135 feet in diameter.
804 feet long.
How long is a football field?
So that's a big boat.
Seven million cubic feet of gas, giving you 112 tons of useful lift.
You ever have to lift something very heavy, there's your sky crane.
So why are they using hydrogen?
Well, when the Nazis came to power in Germany, Congress passed the Helium Control Act.
The dominant supplier of helium to the world was the United States.
Helium comes from helium wells in the earth.
And so as of 1933 the United States refused to sell Helium to Germany, so the engineers were forced to use hydrogen.
Next best thing.
Here are some posters.
"Only 2 1/2 half days to Europe."
And here one, a German one.
"And now over the North Atlantic."
That's Manhattan.
That's the lower tip of Manhattan.
There's the Chrysler Building, look at that.
Now look at that picture, isn't that magnificent?
10 transatlantic flights, 1936.
1002 passengers.
Cruising speed, 78 miles an hour.
Took two and a half days.
By the way, 100 feet in diameter-- when people traveled, they traveled in style.
They had a ballroom there and a grand piano.
People didn't sit like this.
[CROUCHES] With a plastic knife and fork.
That's progress, right?
Two and a half dancing, tux, tails, champagne.
Now, like this.
[CROUCHES] May sixth, 1937.
Arrival of first flight to the U.S. while docking in Lakehurst, New Jersey.
Why were they docking in Lakehurst, New Jersey?
If you go to the top of the Empire State Building, look and you will see at the corners moorings.
Moorings sticking out.
The plan was to dock airships at the Empire State Building.
So you'd come in from Europe, you'd dock at Fifth Avenue, get on the elevator, and there you were.
When they tried to dock the air currents were so violent that they couldn't safely dock the ship.
So then they moved across to the fair grounds at Lakehurst, New Jersey, where obviously this mooring is much closer to the ground than the top of the Empire State Building.
The wild currents, they're bad, but they're manageably bad.
At the top of the Empire State Building, impossible.
There's another image.
So what happened?
It did not explode.
It did not explode.
It couldn't explode.
Seven million cubic feet of hydrogen to explode requires seven million cubic feet of oxygen instantaneously.
And air is 20% oxygen.
So it was a very violent fire, roman candle from the point of egress of the hydrogen.
Most of the people on board walked off the Hindenburg.
Most of the people walked off the Hindenburg uninjured.
They think it was electrical discharge in the vicinity of a hydrogen leak.
Recent research has indicated the skin was made of resin finished with a lacquer dope.
And then to make it shiny they put aluminum powder.
And why they put iron oxide on the inside I don't know, but this is what NASA uses for solid rocket motor grains.
So when this thing catches fire, this is a thermite reaction and could be very violent.
And this spelled the end of rigid error ships in commercial air transportation.
Now this a U.S. Navy airship filled with helium.
And there was a small gasoline fire, look what happened.
Again, it was the skin.
That's a blow up of that one.
So I looked at that and I thought, geez, that looks Lichtenstein, doesn't it?
You know this one?
This one.
Look at that.
Look at that.
So you know, I can be an artist, too, right?
OK, I'm going to tell you one more story.
Another Niels Bohr story.
So in 1896 there was a guy, and astronomer at Harvard called Pickering.
Pickering at Harvard, 1896, and he was studying the lines in star light.
And he attributed to some of the spectra that he was getting, he said he was seeing atomic hydrogen in star light.
And then there was a fellow in London called Fowler.
And Fowler, in 1912, reproduced the experiments in the laboratory.
He put gas in a tube and got the same thing in the lab.
So this guy's at Harvard and the other guy is at London.
Well, Bohr looks at this stuff and he says, you guys are wrong.
You guys are wrong-- your lines are off by a factor of 4x.
You've got the right series, but you got the wrong element.
What you guys are looking at is helium plus.
You're not looking at hydrogen, and you know, from-- it goes to z squared.
So the lines are going to be shifted by factor of four because this z is equal to 2.
So Fowler was a pompous ass and he didn't like being called on his bad science.
So he does a calculation and he looks more carefully and he says, Bohr, you're wrong.
In fact, our lines are off by 4.0016.
Now don't laugh.
The reason is the spectroscopy was so precise that they could go to five significant figures.
So Bohr says, hmm.
And he goes back and he says, you know, we've been doing all these calculations with a one electron atom just neglecting the center.
So he redoes the calculations for the entire Bohr model, including considerations of the mass of the nucleus and the mass of the electron in the form of the reduced mass.
The reduced mass is-- you're learning this in-- the reciprocal of the sum of the reciprocals.
And when he does that, he gets that the value of the line shift should be 4.00163.
So he says, you guys are wrong.
It should be 4.0016.
You got 4.0016, you idiots.
You're looking at helium plus.
That was Bohr.
Did not want to get into an argument with Bohr.
All right.
Have a nice weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, a couple of announcements.
Tomorrow we've got Quiz Two.
Based on Homework Two And then, Thursday, we'll have the periodic table quiz.
I give you the numbers, you give me the letters.
And Friday is the last day for the contest.
I'll have office hours.
I know when others have office hours.
I'll be available 4:30 to 5:30 down the hall in my office.
Where is Grant's, who's buried in Grant's Tomb?
Grant.
Where are my office hours held?
In my office.
Good.
Last day.
Last day.
It was a rough weekend.
I can't get a smile, can't get a laugh.
So, last day we looked at the validation of the Bohr model.
And we had two pieces of experimental data.
First were the hydrogen spectrum lines, measured in 1853 by Angstrom and fit to an equation by J.J. Balmer in 1885.
And, secondly, we saw the Franck-Hertz experiment, in which we were able to get the sense that energy levels within a multi-electron atom, like, mercury, those energy levels are also quantized, which gave credence to the quantum condition.
Which said that the movement of an electron, the movement of an electron, is quantized.
And then we started looking at the limitations of the Bohr model, problems with the Bohr model.
And I said, in two words, fine structure.
And we started looking at fine structure.
We see that the 656 nanometer line in hydrogen, in point of fact, is a doublet.
It's actually two lines very closely spaced.
Bohr model is incapable of explaining this.
Zeeman looked at gas discharge tube, measured hydrogen spectrum in a magnetic field, and found line splitting proportional to the intensity of the field.
Stark did similar experiments only in an electric field, and also found line splitting, the degree of which was proportional to the intensity of the field.
And the Bohr model is incapable of explaining this.
Sommerfeld, in 1916, put a patch on the Bohr model and proposed that the electron moves not only in a circular orbit but also in elliptical orbits.
And there's a plurality of these orbits.
But overall, the distance from the nucleus to the electron orbits is more or less given by the principal quantum number n, and the degree of eccentricity is tiny.
But, more or less, described by n. And then to further depict what's going on, he introduced the orbital quantum number or, as your book calls it, the azimuthal quantum number.
And the magnetic quantum number.
And further said that the energy of the electron is given by the specifications according to all three quantum numbers.
And then, lastly, in order to get completeness, we jumped ahead in time to 1922 with the Stern-Gerlach experiment, which was the beam of silver.
Atoms through a divergent magnetic field, the beam split in two symmetrically about the point at which the beam would land on the slide in the absence of a magnetic field.
And that can led to some deeper thinking by two graduate students in Leiden, Goudsmit and Uhlenbeck, who proposed the notion of electron spin.
And from that came the fourth quantum number, s, and we're going to throw that into the mix even though Sommerfeld didn't give it to us back in 1916.
But I just want to move forward with all of them.
And we recognize that s could take on two values, plus or minus 1/2, or we could call it spin up or spin down.
And, as you're going to learn in 802, the convention in electromagnetism is the right-hand rule.
That is to say, the thumb indicates the vector and the curl of the fingers indicate the rotation.
So we would argue that this is an electron spin from looking top-down anticlockwise and then vice versa.
So that's as far as we got.
And so now what I'd like to do is to take a look at the periodic table and get a sense of electron filling.
And whether that explains the trends in the periodic table.
And for that I'm going to go to table 6-3 in your reading.
And all we're going to do, basically, is say, well can we can we use this idea of n, l and m, and explain the order of filling in a periodic table.
So when n equals 1, l must be equal to 0.
And the m must be equal to 0.
So that means there's only one orbital.
And, in that orbital, we can have two electrons.
So we've got the possibility of two different electrons going into the 1s orbital.
Then we go, n equals 2. l can take a value of 0.
Which is the same as what we had before.
Just one orbital in that subshell.
Or we can have l equals 1, in which case m can take three different values; minus 1, 0, and plus 1.
So there's three orbitals there.
3 plus 1 is 4.
And that gives us the possibility of putting 8 electrons in.
Because this is n, l, and m. And then we've got the choice of s plus or minus 1/2.
And we'll do one more.
We'll go to n equals 3.
So at l equals 0, we just have 1.
When l equals 1, we have the same thing as we had with the 2p.
And then when l equals 2, we have what's known as the 3d.
Remember the spectroscopists there.
They don't like numbers.
So they use s, p, d, f. But we're smart.
We can go 0, 1, 2.
It didn't hurt us.
And we go minus 2, minus 1, 0, 1, 2.
So there's 5.
So 5 plus 3 is 8, and 9.
9 times 2 is 18.
Now let's go to the periodic table and see if this reconciles.
So, when we have 1s, there's hydrogen, helium.
And then the next is n equals 2.
So that should give us 4 times 2 is 8.
So, 3, 4, 5, 6, 7, 8, 9, 10.
So there's the 8 different electrons that get us all the way over to neon.
And now let's go to n equals 3.
So, I have 1, 2, 3, 4, 5, 6, 7, 8.
And then I'm over to 4.
4s.
But look, this is saying I should have 18.
18 electrons in n equals 3.
So what I'm pointing out here is that there's a disconnect, there's a disconnect between the populating of electrons just in ascending quantum number, and the way the elements are arranged in the periodic table.
There's some other factor at work here.
So we want to take a look at what that could possibly be.
And, so what we need is to go to a modified energy level diagram.
A modified energy level diagram, that can explain what the filling sequence is in the periodic table.
And the modified energy level diagram is drawn on the basis of the Aufbau principle.
The Aufbau principle.
Aufbau, German, meaning construction.
I think it actually means out-build.
But, anyways, it generates the filling sequence.
So, there are three parts to the Aufbau principle.
And the Aufbau principle is going to govern, it's going to govern or direct the electron filling sequence.
Directs the electron filling sequence.
So, the first component of the Aufbau principle is the Pauli exclusion principle.
Pauli exclusion principle.
Wolfgang Pauli was an Austrian.
He did his Ph.D under Sommerfeld in Munich, and then he post-docced with Born in Gottingen and on to Niels Bohr Copenhagen.
These people worked together.
They traveled from lab to lab, and it was a very vibrant conversation going on.
Eventually, he became a professor physics in Hamburg.
And the Pauli exclusion principle, simply stated, is that in any electron system, each electron has a unique set of four quantum numbers.
A unique set of four quantum numbers.
Any atom set, four quantum numbers are unique for each electron.
For each electron.
So you can think of this as the set of n, l, m and s as sort of the social security number, if you like, for each of the electrons in the set.
And he eventually gets the Nobel Prize for this, in 1945.
Virtually everybody I'm going to talk about today, with the exception of Sommerfeld, gets the Nobel Prize.
Sommerfeld is a very interesting character, though.
While he himself never won the Nobel Prize, many, many of his students and proteges won Nobel Prizes, to which you must include that there was something very, very special about the quality of the mentoring that he gave, that so many of his proteges went on to win the Nobel Prize.
So this is the first part of the Aufbau principle.
The second part is that the electrons fill from lowest to highest energy.
So electrons fill orbitals.
You can think of the orbitals as placeholders.
They're really energy concepts.
But we populate orbitals from lowest to highest energy.
From lowest to highest energy.
Or if I wanted to be a wise guy, I would say, I'm going to define energy in such a way as I get that as the filling sequence.
One's got to fit the other.
And the energy is itself a function of the four quantum numbers.
So energy, once I specify n, l, m, and s, you can give me the energy and away we go.
And the thing is that you need to realize that the energy levels are actually a function of electron occupancy.
So you can look carefully, if you get down into the d orbitals and the f orbitals, as you add one more electron moving one element to the right on the periodic table, sometimes you see the population not adding by one, because the addition of an electron changes the relative energies of the various orbitals.
So let's remember that, that energy is a function of electron occupancy.
Energy is a function of electron occupancy.
And then the third part of the Aufbau principle is called Hund's rule.
Hund's rule.
Now, Hund is German for dog, but this isn't named after a dog.
This is named after Friedrich Hund, a professor of physics at Frankfurt.
He taught for many years.
He died at the tender age of 101.
And he spoke about degeneracy.
Not societal degeneracy, but degeneracy in an atom.
And this degeneracy is the condition where you have a plurality of orbitals at the same energy level.
So, in orbitals of equivalent energy, in orbitals of equivalent energy, we strive for unpaired electrons.
This is filling, now.
This is how to direct the filling sequence.
Strive for unpaired electrons.
Let's say unpaired electron spins.
Let's make it a little clearer.
Unpaired electron spins.
So I'm going to give you an example, work through an example.
So, what have we got here?
Here's carbon.
And if you look at any element on the periodic table, you'll see this green.
And this is the electronic configuration.
This tells you what the sequence looks like for that particular element in the neutral form, et cetera, et cetera, subject to many, many considerations.
So let's look at carbon.
So if we look at carbon, it tells us 1 s2, 2 s2, 2 p2.
So what's all this mean?
Well, it's all written in code for us.
So the first number here is the n number.
So this means n equals 1. s is the spectrocopist's notation, so this means that l equals 0.
And then the 2 here means 2 electron occupancy.
So the 1s orbital is filled to the tune of two electrons.
And the 2s orbital has two electrons, and the 2p orbital likewise has two electrons.
So now we want to do is put these electronics into their orbitals.
And we can use, there's a variety of notations.
This is one I'll use today.
This is a box notation.
So this is 1s.
This is 2s.
And then this is 2p.
If you recall, 2p there's going to be three of them.
If you go back here, 2p has three orbitals in that sub-shell.
And we said that this is the one time that m makes some sense with respect to Cartesian coordinates.
m is -1, 0, +1 so we can call this 2p x, 2p y, and 2p z.
And I don't know which is which.
I don't know what the arbitrary standards of x, y, and z, I don't know where the origin of the universe is, so I can't tell you all this.
But, arbitrarily, if I choose one as x, I know where the other two are according to right-hand rule.
So let's start filling.
This says 2, so I'll put in one spin up, and one spin down.
Now it says 2s 2.
One spin up, one spin down.
And now here's where the Hund rule comes in.
I've got to put two electrons into these three boxes.
Now, I could be a librarian and start left to right, and make them nice and neat.
Or, I could just put them in wherever I want.
What the Hund rule says is, strive for unpaired electron spin.
So for the Hund rule, you'd put them in both same spin and in different orbitals.
And then when you get to nitrogen, nitrogen will have a third one here.
And when you get to oxygen you've got three plus the fourth one's going to be one of these three and I don't care.
I mean, some people are really anal about it, and they want you to go from left to right.
As long as you've got two of them unpaired and one of them paired for oxygen I'll be happy.
So now we can use this concept and look at the energy level diagram for multi-electron atoms.
And this is taken from the book.
So this is different from the energy level diagram for hydrogen, because you can see some compression here.
And I want you to know first of all, all of these values are negative.
It's true that energy increases vertically, but the zero is way up here.
So these are all negative values.
And I want to zoom in here.
Because the energy difference between 1s and 2s is large.
And we know in hydrogen it's 3/4 of the total energy difference.
Because there's some compression in a multi-electron atom.
But nevertheless, n equals 1 to n equals 2 is a huge energy difference.
So let's zoom in on here.
And what do we see?
Well, 2s, 2p, 3s, 3p and look.
4s lies below 3d.
4s lies below 3d.
So that tells us, there's my 3s.
3s 1, 3s 2.
There's 3p x1, 3p y1, 3p z1, 2, 2, 2.
And now what's next?
It says go to 4s.
So potassium is 4s 1.
Calcium's 4s 2.
And scandium is 3d 1.
So this energy level diagram is the map.
It tells you how to put electronics in sequence.
So, this now gives us the rational basis for e equals n, l, m and s. All right.
Well this is nice.
It's been graphical, and so on.
Now I want to go to the same position.
I want to get back to this, I want to get back to this point.
But I want to go by different route.
I want to go by different route, and for that we're going to go by wave mechanics.
So, same destination.
Same destination, via wave mechanics.
Now, you don't have partial differential equations as a prerequisite for 3.091, so I'm not going to go through the math.
I'm going to give you the features of the wave mechanics.
So that later on you're going to spiral around and study this again.
You'll have seen it before.
And again, there's going to be people involved, and they're all giants in modern physics.
The first one is de Broglie.
The first one is Louis-Victor de Broglie.
Let's get his name on the board.
Louis-Victor de Broglie, he was an aristocrat from Normandy, who had gone to the Sorbonne.
He was studying humanities.
Political science, literature, and around about the time of his senior year he decided to switch horses for graduate school and forget about a career in the diplomatic corps, do a Ph.D in physics.
So, he did a Ph.D in physics.
And in 1924 he published this Ph.D thesis, beautiful piece of elegant writings.
Less than 30 pages long.
And I'll give you sort of the summary, the one-liner that summarizes his Ph.D thesis.
Most of you are going to have to write a thesis.
The word thesis comes from the Greek, and it means, sort of, my position.
My statement.
Before I have a thesis, I have something that is not a thesis.
It's my trial balloon.
That's a hypo-thesis, a hypothesis.
Now, the key to writing a good thesis is to ask a really good question.
If you ask a pedestrian question, you're probably going to get some pedestrian answers and ho hum.
If you ask a really interesting question, you give rise to the possibility of interesting answers.
And what de Broglie did is, he asked a really interesting question.
So here's his question: He says, if a photon, which has no mass, photon's just an energy packet.
If a photon which has no mass can behave as a particle, and we've seen.
We model ray optics as particle beams.
And a photon, and Max Planck said equals h nu got no mass, but we can think of it in our little anthropomorphic brains as, photon photon photon.
So if a photon, which has no mass, can behave as a particle, does it follow that an electron, which has mass, can behave as a wave?
It's beautiful.
Let's do it one more time.
If a photon, which has no mass, can behave as a particle, does it follow that an electron which has mass, can behave as a wave?
So he asked the question.
And he answers it.
In less than 30 pages.
So, let's get this on the board.
Because this is beautiful.
It's like, ask not what you can do for your country.
If a photon, which has no mass, can behave as a particle, or can be modeled as a particle, behave really means, so that the theoreticians can model it as a particle, does it follow that an electron, which has mass, can behave as a wave?
See, if you understand the question then the impact of the answer, and the answer is, if it does, this is what its wavelength is going to be.
So de Broglie said, the wavelength of an electron, if it were to behave as a wave, would be given by the ratio of the Planck constant to the Newtonian momentum, which you know from 8.01 is simply h over the product of electron mass and its velocity.
So that's de Broglie's thesis.
So let's take a look what we can do with this.
Now, you remember in the Bohr model, recall Bohr.
Well, Bohr taught us that mvr, that's the quantum, condition, mvr equals the ratio of h over 2 pi times n, where n takes on the discrete values.
1, 2, 3, et cetera.
That's the quantum condition.
Now let's take this idea of de Broglie.
And first of all we have to put the electron in its orbit.
So we'll put the electron in its orbit.
And now I'm going to have it behave as a wave.
So if it behaves as a wave, I'm going to draw it as a wave.
Now, why did I draw it this way?
There's two kinds of waves in this world.
There's standing waves and there's traveling waves.
Now, this orbit's station, it's in this orbit.
So it better be a standing wave.
I think there's a cartoon in the book.
There you go, standing wave.
In order for this to be a standing wave, there's a geometric constraint on this.
Listen carefully geometric constraint.
I'm not saying anything about quantum mechanics.
Geometric constraint, the geometric constraint for a standing wave is what?
You know what this distance is, right?
I mean, this is not to scale.
This should be 10,000:1, in which case these ripples are barely visible.
But here it looks kind of exaggerated.
This is a hyper wave.
This is emphasis added in proof.
So that means that the circumference here, 2 pi r, the circumference must be an integral number of wavelengths for a standing wave.
But we know that from de Broglie, I can write n lambda as n h over m v. I've just put in de Broglie's definition of the wavelength of an electron.
And now I can cross-multiply and I get mvr equals h over 2 pi times n. Which is Bohr's quantum condition.
So we've got validation of the Bohr model, so that's a pretty compelling case that maybe the electron really does behave as a wave, and that explains why we have the quantum condition that we do.
So de Broglie, that's his Ph.D in 1924.
Einstein read the thesis, loved the thesis.
But we don't care what Einstein says, because he's a theoretician.
So one theoretician praising another theoretician.
That's not how science works.
How does science work?
Data.
We need data.
And the data come in 1927.
1927, at Bell Labs in New Jersey.
At Bell Labs in New Jersey come the critical data.
And they were taken by Davisson and Germer.
Davisson and Germer.
Davisson and Germer were studying crystals.
They were studying crystals of various elements, and in particular metal crystals.
Metal crystals, by X-ray analysis.
And in order for you to appreciate what I'm going to show you of Davisson and Germer's work, I'm going to take you back to high school to those thrilling days with the wave tanks.
Remember the wave tank?
This is the top view, this is the side view of the wave tank.
And you might have some kind of a mechanical device here that has a paddle.
And it starts vibrating up and down, and it starts sending waves into the tank.
So the waves come like this, from the edge it looks like this.
Remember that?
Sure you do.
You're toying with me.
Oh, I don't remember anything, we never did that.
Sure you did.
OK, so you can send waves down.
Now, what we can do is, we can put a dam here.
I'm going to put a dam.
And depending on, if this is the wavelength, this is the wavelength, it's the distance between two successive crests.
And, if this spacing here, the gap between the wall and the edge of the dam, d, if d is greater than lambda, the waves just propagate but for the place where they're blocked by the dam.
So you get, you essentially cast a shadow.
You've seen that.
And so I could model this system as a beam.
This is a beam.
This is a water beam.
This is a water beam shadow.
And this is equivalent to ray optics.
Straight lines, if something gets in the way it's opaque.
Blocks transmission.
End of story.
So you've seen all that.
But you also did this other experiment, I'm willing to bet.
So let's do the other experiment.
We're going to do the same thing.
I'm going to send waves down here.
Only, this time we're going to make the dam a little bit different.
This time we're going to bring the dam in from the wall.
And we're going to put a tiny opening.
And I'm going to go some more into the tank.
And then another tiny opening.
It could be the same.
It's probably best to keep it the same dimension.
Now, in this case, the spacing, d, is much less than lambda.
d is much less than lambda.
And what happens in this case?
When d is much less than lambda, you don't get the shadow.
You don't get something like this instead, remember.
You've got the rings.
This is called diffraction.
Diffraction.
And there is no way to explain diffraction modeling water as a beam.
You must implore the wave-like behavior of water in order to explain diffraction.
Explain only by invoking wave-like properties.
So with wave-like properties, we get something that makes sense in terms of the data.
Now, let's do this same experiment, let's do this same experiment on a metal crystal.
So if you go back to the gas discharge tube.
Remember the gas discharge tube that we were looking at for lecture after lecture?
If you take a look and go through the energetics of it.
If you put one volt across the plates, you know the energy is going to be a product of charge times voltage.
And that's equal to 1/2 mv squared.
Which you know from 801 is p squared over 2m.
And p is equal to h over lambda.
Pretty soon you can come up with the wavelength.
And that will give you the wavelength of the electron.
This is a ballistic electron now.
See what I'm doing?
See, once I said that this indicates that the electronic in stationary orbit can be modeled as a wave of a certain wavelength, so now the free electron.
It's got m, it's got v, there's Planck constant.
I can go ahead and compute its wavelength.
I can compute the wavelength of a baseball.
So, you go through it.
And you get a value of about 12 Angstroms.
12 Angstroms.
Now, if I want to see whether there's wave-like properties, I need to have a condition that gives me diffraction.
So I'm going to have to find something that gives me a dam with an opening that's less than 12 Angstroms.
If I'm going to use 1 volt.
So what can I do?
Well, turns out you're going to learn this in greater detail later, but if this is a crystal of nickel.
Crystal of nickel, the atoms are arranged in regular arrays.
And this is what the face of nickel looks like, 4 atoms each at the corner.
And one in the center of that face.
This distance is 3.53 Angstroms.
Perfect.
Perfect.
So what Davisson and Germer did is, they irradiated this.
They irradiated this first with X-rays on the order of lambda, on the order of, say, 10 Angstroms.
And what did they get?
This is the output.
This is the output.
You get a diffraction pattern.
It's a set of rings, concentric rings.
So this is the X-ray diffraction.
This is the X-ray diffractogram, if you like.
And then, what did they do next?
They irradiate the same crystal with an electron beam.
lambda of the electron beam, 10 Angstroms.
And what did they get?
Are you ready?
Drum roll.
Now, there is no way that you can get a ring pattern from a beam of electrons acting as a particle beam.
The only explanation for this is that the electrons were behaving as waves of this value to give us the same spacing as we got with X-rays.
And you're comfortable if I say that X-rays are a type of light.
So, therefore, it's got wave-like properties.
And it's got particle-like properties.
Well, now I've just made the point that this is the electron diffractogram.
So this is evidence of electron diffraction.
And this shook the world.
Because now it's real.
There's no way you can get this otherwise.
This is electron diffraction.
This was 1927.
1929, de Broglie gets the Nobel Prize.
1937, Davisson gets the Nobel Prize.
So this means the wave-particle duality is complete.
It applies not only to light, but it applies to matter.
So, wave-particle duality, they call it.
Wave-particle duality is complete.
Matter can act as waves.
Electromagnetic radiation can act as particles.
So, sometimes people refer to de Broglie's accomplishment as matter waves.
Matter waves.
And what do you call the behavior of billiard balls banging around and so on?
We call that mechanics.
So now we're going to use what might seem as an oxymoron, contradiction, wave mechanics.
Wave mechanics.
That means matter behaving as a wave.
But still behaving as a matter.
So this is the dynamic.
So that's pretty good for de Broglie.
So let's go to number two.
I said there were going to be three people here.
Number two is Werner Heisenberg.
Werner Heisenberg.
Heisenberg studied with Pauli.
Sommerfeld.
Did his Ph.D in Munich in 1923.
Got his Ph.D with Sommerfeld at the age of 22.
And then he decided to take a postdoc with Bohr.
And he was working with Bohr for a couple of years, he was feeling a little bit burned-out.
And decided to take three weeks off.
Went up to a deserted island off the coast of Norway.
Came back three weeks later with the mathematical formulation of quantum mechanics.
I'm not kidding you, that's what he did in his time off, to kind of unwind.
And so one of the things that he used as a critical piece of this derivation is that the position and velocity of an electron cannot be fully specified.
They cannot be fully specified below certain limits.
There's a threshold below which we can't go.
This is sort of like, if I asked you to time a 100-meter sprint, which typically takes less than 10 seconds.
But I give you a clock, and the nearest unit on the clock is the minute.
So you wouldn't be able to distinguish.
So, he says-- and the reason for this is, it's a consequence of quantization.
Light itself is quantized.
So, at some point you're asking for a continuous splitting and splitting and splitting into finer and finer time segments.
And you can't get there.
So we already knew this from Planck.
And so one of the ways that he expressed the inability to go below a certain threshold is the uncertainty principle.
The uncertainty principle.
And that's unfortunate that, see, it was originally published in German.
And the idea really is the indeterminacy principle.
But, English says uncertainty.
So it's a limit to determination, but there it is.
And so one expression of this is, the product of the velocity.
Only, he wrote it in terms of momentum.
So the mass isn't going to change.
So, think of this as the uncertainty and the velocity.
And the uncertainty in the position.
So this is the x-coordinate of particle.
You can break its trajectory into three orthogonal components.
So the uncertainty in the x direction of the momentum times the uncertainty in the position is greater than or equal to the Planck constant divided by 2 pi.
The Planck constant divided by 2 pi.
And so what this means is that we're going to see a transition in models from individual atoms.
If we want to describe what happens with individual atoms, we need what is known as a deterministic model.
Sort of, Newtonian mechanics.
You tell me the initial position and velocity.
You tell me the forces, and I can predict where it's going and where it's going to be.
So that's deterministic models.
So, deterministic models describing individual atoms are going to give rise to probabilistic models.
Probabilistic models.
And probabilistic models obviously can't be talking about individual atoms.
Must be talking about ensembles of atoms.
So I can't say where any individual atom will be, because I don't have the ability to do so.
But I can tell you, if you give me a large number of them, I'll tell you roughly what the expected outcome could be in terms of energy.
And ultimately predict the spectrum, and so on.
So instead of chicken and egg we have now chickenality and egg-ness.
Everything is just sort of getting a little bit murky.
Little bit murky.
You can do a calculation on this.
You can do a calculation on this, a very simple take.
Take the Bohr model and take the ground state electron in hydrogen.
n equals 1 in atomic hydrogen.
And you know this is about half an Angstrom.
So the distance across here is 1 Angstrom.
So make 1 Angstrom your uncertainty, and you'll find that the uncertainty in the momentum is on the order of 15%.
But what this is saying is, when you get down to atomic dimensions, you can't just shine light on it and reveal what's going on.
Because you're going to disturb the very thing you're trying to measure.
Some people say that, every time you try to work at the atomic level, it's as though you're trying to take a picture with the sun at your back and your shadow is in the picture.
So you can't get there without disturbing the very thing.
Another way to think about it is, the photons that are capable of this resolution are going to have such high energies they'll knock the very thing you're trying to measure.
All right.
He gets Nobel Prize in 1932.
And then the third one is Erwin Schroedinger.
Let's get him up here.
Erwin Schroedinger.
Also an Austrian, University of Zurich.
He too was burned out.
They get burned out, these guys.
So at Christmastime, 1925, took a vacation.
At Villa Herwig, in Aurora.
And comes back two weeks later with the wave mechanics formulation, of quantum mechanics.
See, sometimes going away on a vacation.
So he took de Broglie's notion of the electron as a wave and wrote equations to model wave-like behavior.
So, let's look at how to get there.
And here's what he did.
So, you know, for example, that we could start with a violin string.
And it has a geometric constraint.
It must be fixed at both ends.
And if I pluck that string, it can vibrate as long as it conforms to the geometric constraint of a standing wave.
So here's one possibility.
Wherein we would call this n equals 1.
I have simply the entire string vibrating in the matter that's shown.
But here's a second possibility.
I could have it vibrating as is shown here, n equals 2 with a node in the middle where that node doesn't move at all.
The string is stationary at its mid-point.
And what's the characteristic here?
This is operating at a certain frequency.
Let's say it's middle C. And this is the overtone.
This is the first harmonic.
And it's going to be an octave higher, because it's as though we have two strings, each fixed.
See, from a physics standpoint I could literally cut this string in half and fix it there, and this is now n equals 1 for the half-length.
So it's going to have the same pitch as the half-length.
Which means that this is an octave higher, and this is going to be two octaves higher, and so on.
And all of these conform.
So you get a plurality of solutions.
You get a plurality of solutions.
And the solutions look something like this.
They'll eventually teach you the string as a simple harmonic oscillator.
Simple harmonic oscillator.
And it has equations that look like this.
If you want to plot its position, this is x going from 0 to l. And this is the y-coordinate.
So you can, for example, write something like this.
So, the function will look like this.
Some pre-multiplier times cosine of kx plus another pre-multiplier b times the sine of kx.
And the geometry will dictate that the value of k is n pi over l. So, pi over l is the geometry.
And n takes multiple values.
Just as you see here, there's not a unique solution.
So listen carefully.
Wave equation, plurality of solutions.
But subject to some constraints.
Subject to some constraints.
So what Schroedinger did is, he wrote a wave equation to describe the motion of electron in its orbit.
And guess what he gets?
He gets a plurality of solutions.
And when you look at the plurality of solutions, the plurality of solutions ultimately map into what we know as the distinct values of n, l, m and s. See, this is the one-dimensional.
So this is giving us n numbers.
n equals 1, this is now n equals 2.
So I'm getting quantum numbers here.
Now, if I did this in three dimensions, I'd have a plurality of quantum numbers.
And Schroedinger gets us all the way to n, l, m and s. And so here's what it looks like.
This is the equation, it's a wave equation, so there's a double derivative in space.
There's a forcing function.
And this is i, square root of minus 1 in a time base here.
So it's a harmonic kind of equation.
Psi is the wave function, it's an abstract concept but we'll show you how to make sense of it.
And these are the various solutions, the plurality of solutions.
And we can now map those into what we know as 1s, 2s, 2px, 2py, 2pz, et cetera, et cetera.
And you see this number a sub 0.
That's our Bohr radius.
Comes right out of the equations.
0.529 Angstroms.
So, this is quite good.
But, as I said, the psi is the wave function.
Or psi, however you want to call this.
This is called the wave function.
Wave function.
And we have plurality of solutions.
We call these Eigenfunctions.
Eigenfunctions.
And the closest we can get to something physical is the product of psi and its complex conjugate.
And that is related to the probability of finding the electron.
Probability of finding the electron.
Which, in essence, gives the boundaries of the orbitals.
So now I'm going to put, we're going to get a Cartesian shape.
I told you, 1s, 2s.
What do they look like?
Here's what they look like.
So these are the square of the wave function plotted.
So this is in a radial distribution function.
It's only in one direction, out from the radius.
Now, if you whip this around in 3-D, you'll generate the surface.
But already you can see, here's 1s, and it peaks at about 1/2 an Angstrom.
And here's 2s, and it peaks at 4 times the Bohr radius.
And psi squared 3s about 9 times.
This is from your book.
So it's a maximum.
But there's some uncertainty.
See, it's not a simple line fixed at 0.529 Angstroms.
This is another way of plotting.
So these are spherical.
What we were calling circular now becomes namely spherical.
And there's this node here.
This is the p orbitals.
They're dumbbell-shaped.
With two lobes.
And if you have a single electron, it doesn't reside in one lobe.
It can jump from one side to the other.
You might say, well, how does it get from one lobe to the other when halfway between, there's a nodal plane.
It has zero probability.
Well, it's behaving as a wave.
Behaves as a particle, you can't get through a wall that says zero permission.
That's how, you can transfer energy from here to here and have that node's perfectly stationary.
Anybody skip rope?
You know how this works.
Now, this is where I quarrel with the book.
This is another drawing.
But I'm uncomfortable with the fact that they chose different colors.
Because I think to the first time learner, you might be tempted to think, well, one electron lives here and the other electron lives here.
No, the electron, if there's only one, it can go from one to the other.
If there are two, they can go from one to the other.
And, see, they do this all the way through.
So please don't start rationalizing in your mind that one electron goes in the yellow and the other electron goes in the grey.
There's the d. Aren't they pretty?
If you find the f orbitals, that's wild.
So I think this is probably a good place to stop.
We've got a few minutes here.
If you want to read more about uncertainty, this is a very nice book by David Peat that goes into the meanings, including this indeterminacy, and so on.
Good book here on hydrogen.
Please, I don't want noise.
We've got, still, a few more minutes.
Still got a few more minutes.
This book here talks about hydrogen.
Goes right back to Democritus.
One chapter is a beautiful thing on the use of hydrogen as a potential fuel.
All the Bohr and whatnot.
This is a play that won the Tony Award in the year 2000.
Written by Michael Frayn.
And it's about the fact that Niels Bohr was the mentor to Werner Heisenberg.
And now it's 1941, the Nazis have invaded Denmark.
And Bohr is essentially waiting to get out of Denmark before the war overtakes the rest of Europe.
Meanwhile, Heisenberg is now the head of the Nazi equivalent of the Manhattan Project.
And he goes to Copenhagen to visit his old mentor.
That's a fact.
They have dinner.
That's a fact.
They go for a walk.
That's a fact.
They never speak to each other after that night.
That's a fact.
So the question is, what went on that night.
And that's what Michael Frayn uses as the dramatic point of departure.
So, did Heisenberg go to Bohr to get Bohr's opinion about nuclear weaponry?
Did he try to find out whether the Allies were working on a bomb?
Did he go to say, look, we should on both sides not develop nuclear weaponry?
What went on in that conversation?
And so you see Bohr at the center.
There's Heisenberg who's the one electron.
And there's Margaret, who is Bohr's wife, who was the observer.
And so there's the play between the uncertainty in quantum mechanics and the uncertainty in human relations.
It's a really nice play.
Here's a rendition of it.
This is Stephen Rea playing Bohr, Francesca Annis playing Margaret Bohr, and playing Werner Heisenberg is, do you recognize him, it's Daniel Craig.
This is a book that came out not too long ago about Heisenberg.
A lot of controversy about him.
Some people accused him of being a collaborator.
Other people say that he was absolutely brilliant in the right amount of foot-dragging.
He did not want to give Hitler nuclear weapons.
If he'd been a total disaster he would have been replaced by somebody who might have been more zealous.
And if he went too fast, he might have figured out how to make nuclear weapons.
So, very interesting book about him.
And here's a nice photo.
This is Bohr.
This is Werner Heisenberg.
And this is Wolfgang Pauli.
Undoubtedly talking about what goes on when you pluck that string.
So we'll see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, a couple of announcements.
Tomorrow will be the Periodic Table test.
I provide the numbers, you provide the letters.
Friday, the contest ends, 5:00 pm.
Get the submissions to me, either over the internet or if you have something majestic, you have to cart it in with the riggers.
Get it to my office.
Last day, we had a lesson on the Aufbau principle.
And the Aufbau principle gave us the-- There's still too much talking.
Still too much talking.
I will tolerate zero, zero talking.
When you walk through that door, I'm assuming it's an act of free will.
And when you walk through that door, we enter into a contract.
The contract goes something like this: You have certain expectations of me.
You expect me to come to class prepared.
You expect me to treat you with respect, not to use vulgar language, not to say things that are insulting, insensitive.
And I have certain expectations of you.
I expect you to come to class prepared.
I expect that you've done the reading.
And I expect that you're going to sit quietly.
Absolutely quietly.
Because it's my duty to preserve a fertile learning environment.
If you don't want to learn, I don't care.
I don't care.
But if you impair the ability of your neighbor to learn, I will take action.
It's very simple.
Very simple.
Where was I?
I think I was talking about the Aufbau principle.
And the Aufbau principle tells us what the filling sequence of electrons is in a multielectron atom, which ultimately we rationalized in terms of the four quantum numbers and the filling in ascending order of energy.
And then we tried to understand what's behind the Aufbau principle.
And lastly, we took a journey through central Europe in the 1920s and 1930s.
And we met de Broglie, who gave us the concept of matter waves.
If matter has wave-like properties, the wavelength would be given by the ratio of the Planck constant to the Newtonian momentum.
Heisenberg taught us that we can't deal with precision down to atomic dimensions, and there's a certain degree of uncertainty or indeterminacy.
And it's given by this relationship here.
And then, lastly, we saw Schroedinger, who wrote the wave equation saying that if we have wave-like properties then let's model the atom as a wave, and he gave us this equation here, which is essentially the equation of the simple harmonic oscillator.
The simple harmonic oscillator, which allows you to tell us what's going on with the plucked string.
And it's essentially just this, the double derivative x double dot plus kx goes to some function.
And the solution of that is a bunch of sines and cosines.
And that's this equation, but dressed up for night-time.
It's a little bit more sophisticated and gives us the three dimensions, eigenfunctions that ultimately, we can take and generate the pretty pictures of the orbitals.
So, today what I want to do is go a little more deeply into properties.
And if you learn nothing else from me in 3.091, this is what I want you to retain.
It's that electronic structure dictates properties.
Electronic structure dictates properties.
You know this on your death bed.
It's that simple.
Electronic structure dictates properties.
Then we're going to take 14 weeks and describe electronic structure, right?
But that's it.
That's it.
So I want to go back to this.
We got into this energy, Aufbau principle because we saw on the Periodic Table there's some problems here.
If we just go in terms of quantum numbers, we see there's two elements in n equals 1 shell.
There's 8 elements in n equals 2 shell.
And, if we just kept going, we would expect to find 18 elements in n equals 3 shell, but instead we found only 8.
And that's where the journey began.
Well, there's a way to sort of help remember what the filling sequence is.
And that rule is encapsulated by the n plus l relationship.
So for equivalent, n plus l values, for equivalent n plus l values, you fill in ascending n. Fill in ascending n. So let's take a look.
So, here's Aufbau.
n plus l rule.
Here's an example where we have 3d.
d means l equals 2. p means l equals 1. s means l equals 0.
So 5 plus 0 is 5.
4 plus 1 is 5.
3 plus 2 is 5.
So what do I do?
Well, it's saying go in order of ascending n plus l. And so, you can snake your way through this n plus l, and get the filling sequence for the whole Periodic Table.
So let's take a look.
We'll start with 1s.
Well, that's easy.
That's hydrogen and helium.
Then we go next to 2s.
There's lithium and beryllium.
And then next goes 2p.
I mean, nobody's going to try to stick 3s ahead of 2p.
That's pretty straightforward.
2p, and that gets us all the way over to neon.
And then we wrap around over here to 3s.
That gets us from sodium to magnesium.
And then we jump to 3p, gets us from aluminum over to argon.
And then, we don't go to 3d, because 3p is 3 plus 1 is 4.
4s is 4 plus 0 is 4.
So it says we fill in ascending n. So we go to 4s next.
And that gets us potassium and calcium.
Then comes 3d.
So that gets us scandium over to zinc.
Then comes 4p, gallium to krypton.
5s.
I'm having fun here, I'm going to keep going.
5s gets us over to strontium.
Next comes 4d.
Gets us all the way over to cadmium.
5p gets us to xenon.
Now it's turn 6s cesium, barium.
4f.
We've got to jump down here.
Finally we're going to get to the f shell.
See?
The 4f is in here.
Now, this is an item of convenience.
Strictly speaking, lanthanum is 57, cerium is 58.
So all of these elements belong in here.
But if we were to put this to scale, the Periodic Table would be way over here.
It wouldn't fit on a page.
So people, over time, have gotten used to just putting it down here.
But these elements are in the middle here.
So you go 4d, 5p, 6s, then 4f.
Finally we get over to the edge here, to erbium, and then we jump over here to hafnium, 5d, over to mercury, 6p, et cetera.
So the n plus l rule gives you the filling sequence in ascending order.
That's good.
So we've got a nice compact way of grabbing this e goes to n plus l. Now let's look at some properties.
We said it's a table of the elements, but it's a periodic table.
So let's see what this periodicity looks like.
Now, here's a variation of first ionization energy.
So here it is in kilojoules per mole, and here's my pet unit, the electron volt per atom.
Because here's hydrogen, of 1.6 megajoules per mole, but I like the 13.6 electron volts because I can remember 13.6.
I can't remember 1312 megajoules per mole, kilojoules per mole, whatever it is.
13.6 electronic volts.
So here's hydrogen, there's helium.
And then we drop down to lithium, and then we move across up to neon, and so on.
So you can see 1s, we filled the 1s shell.
Now, we're going to go to shell n equals 2.
Here's 2s.
And then we go to 2p.
And you can even see, look, boron, carbon, oxygen, the ascending value of the first ionization energy.
And then there's a little jog here at oxygen.
Because here we've got the three unpaired electrons for nitrogen.
And then there's the oxygen starts to pair, and we continue to neon, and so on.
3s, 3p, 4s.
So you can see the relationship between that property, and the place, the element, in the Periodic Table.
So how do we measure these values?
Well, I thought it might be a good opportunity to look at, revisit the whole question of gas dynamics, and also understand the measurement.
Measurement of ionization energies.
And the technique that's used is called photoelectron spectroscopy.
Photoelectron spectroscopy.
All right.
And it's got a three-letter initialization, PES.
I want to show you a cartoon of how this works.
Actually, I took one -- this is taken from the text we used to use.
I like the text that we're using now better, but there are a few things in the other text that were good.
So, we've scanned these few pages and posted them at the website if you want to go through and read this stuff.
So let's jump over to that.
So here's an example of a terrible drawing.
I look at this and I haven't got the faintest idea how this thing works.
It's not the artist's fault.
Somebody should have proofread this thing.
What they should be doing is showing something like this.
So, let's see what's going on.
So what we're going to do is, we're going to bombard a specimen right here.
Which they never show.
I suppose this atom beam is supposed to be colliding with the photons.
That would be quite an apparatus to build, let me tell you.
Instead, what you have is, you have the apparatus sitting so that you've got the material in the center of the chamber.
And you irradiate with photons.
And the photons have very, very high energy, and they dislodge electrons.
When they dislodge electrons that's the ionization event.
And then what we do is, we measure the velocity of those ejected electrons.
And if we know the velocity, we know their energy.
We know the energy of the incident energy and by difference, we calculate the binding energy of the electron and the atom.
So let's take a look.
I'm going to draw a cartoon here that gives you a sense of what's going on.
So, here's the n equals 1 shell, or the k shell.
And then I'm going to show, say, a second shell here.
One, two, three.
So this is the l shell, or n equals 2.
And you might say, gee, he's got that wrong.
It's kind of simple, and so on.
It's as complex as it needs to be for the explanation.
Don't let accuracy trump clarity.
The real accurate picture here is too detailed to convey what I'm trying to convey.
So, this is the specimen, right?
Let's call this the specimen.
And I'm going to irradiate the specimen with some radiation of high energy.
And so, I generally indicate the photon by a squiggly line with an arrow.
That's to indicate it's got wave-like properties.
And I'll usually write h nu next to it, to indicate that it's a photon of energy h nu.
So the photon comes in, and if the energy is high enough, it will dislodge an electron.
So this electron is dislodged.
It's now ballistic.
It's rendered ballistic.
It's free.
It's ionized.
It's gone.
It's gone.
And what we're going to do is, we're going to take an energy balance here.
And at some point you may come back and learn some quantum mechanics.
This photon is annihilated at this collision.
So the photon doesn't act the way an incident electron does where it loses some of its energy.
All of the energy is lost.
So this photon's gone.
All of the energy is given to this electron.
So let's do an energy balance here.
So I can say that the energy of the incident photon, the energy of the incident photon, is now going to be given to the electron.
It's now got some kind of kinetic energy.
E kinetic plus the energy it took to pull it out of the atom.
Plus E, let's call it binding.
E binding And we know the incident energy of the photon because we're running the experiment.
So we set the value of the incident energy of the photon beam.
And then we send this to a detector.
This goes to a detector.
And the detector measures the velocity and ultimately gives us the energy of the scattered electron.
This is called the dislodged electron, or photoelectron.
It's the electron that was kicked out by the photon.
So it's known as the photoelectron.
Photoelectron.
Pardon me, let's try that again.
Photoelectron energy is measured at the detector.
And then by difference, we get the binding energy.
We get the binding energy.
So this is h nu.
This one here is going to be a 1/2 m b squared, and then by difference we get the binding energy.
And what kind of photon energies do we need?
We need fairly high energies.
And so typically, we might use wavelengths down around 1 angstrom, which then makes it an X-ray.
And so, over here in Building 13, we have such instrumentation, and the material scientists would call this XPS.
X-ray photoelectron spectroscopy.
And if you don't want to blast all of the electrons out of the specimen and just get the most weakly bound, then you want a lower incident photon energy, which means a longer wavelength.
You might come in at around 100 angstroms, which is ultraviolet.
You know, people, the general public, is so afraid, they don't know science.
If they hear X-ray they get all panicked.
You know, it's radiation.
Their children are going to die and all this kind of stuff.
So, what people will do is, they'll say instead of soft X-ray they'll say hard ultraviolet.
And then the public thinks, oh, as long I wear sunscreen I'm going to be OK.
So if you use hard ultraviolet, it's called UPS.
That's nothing to do with brown trucks.
It's ultraviolet photoelectron spectroscopy.
And collectively, this is known as PES-- photoelectron spectroscopy.
All right.
So now, let's go back to this schematic.
Now you can see how bad this schematic was that these photons are coming in as so.
And striking the specimen here, ejecting the photoelectrons which then are focused and ultimately measured here at the detector.
So there's the energy balance.
The energy of the incident photon is distributed across the kinetic energy of the photoelectron plus the binding energy, and this is how we measure all of these quantities.
Since you're using very, very high intensity radiation, there's nothing saying that you're going to eject only one electron.
You can eject a plurality of electrons.
And the different electrons have different binding energies.
So you're going to get a set of binding energies, which means you'll get a set of lines.
You'll get a spectrum of energies.
So this is what the spectrum looks like, and this is, by the way, just a general way of looking at any spectrum so that you train you eye to know what to look for.
If I look at any spectrum, I'm going to have a plot of some kind of intensity, some kind of intensity unit versus some kind of energy unit, versus some kind of energy unit.
And sometimes, energy increases from left to right.
But sometimes they might put energy increasing from right to left as you see in these spectrum.
But energy is along here.
And intensity is related to population.
All other things being equal, if I have a specimen that has two times the concentration of active species, I'm going to get twice the strength of the line.
So the intensity measures population.
The energy is related to the characteristic binding energy of a particular electrons, so the energy is related to the identity of the species.
This is how we can identify certain species.
I've already shown you if you walk in and you see those four Balmer series lines on a spectrum, you know that's characteristic of atomic hydrogen.
Nothing else.
So those are energies.
And then, where do we go?
So we have a line here, but we know that real data have a little bit of a spread to them.
So you don't see discrete lines; you'll see a spread centered at about the value.
So let's look at these.
These are highly stylized, but here you can see-- here's boron for example.
So these are the 1s electrons of boron, and they're held at a binding energy of about 19.3 megajoules per mole.
And then this is 2s, and here's 2p.
Now there's a difference between 2s and 2p, but they're of the same order.
They're roughly about a megajoule per mole.
There's subtle differences.
And this goes back to Sommerfeld, who said that even though there are s and p orbitals, there's a circular orbit and there's an elliptical orbit.
But they're roughly in the same shell and that's what you see.
But there's a huge difference in going from shell n equals 1 to n equals 2.
This is not to scale.
You see they got a little break there.
And now look, what's the electronic structure of boron?
1s2 2s2 2s1.
There are two 2s electrons, but only one 2p electron.
And this is trying to show that the height here is twice the height of the 2s peak.
So you see that we've got two electrons here, one electron here, and it's easiest to pull out the 2p electrons first.
And you go on, there's beryllium and so on.
So we see that we can build data in this fashion here.
We keep going.
Here's carbon, which is 2, 2, 2.
And then oxygen has 2s2-- pardon me.
1s2, 2s2, 2p4.
So this is two electrons, this is four electrons.
Showing that the 2p height is twice the 2s height.
And then finally, neon is 2s2, 2p6.
So this is roughly three times the height of the 2p.
But all of the 2p numbers are roughly of the same order of magnitude and decidedly smaller than what's going on at 1s.
And what else do you see?
Well, the carbon has 6 protons, so the inner electrons of carbon are held not as tightly as the inner electrons of neon, which has 10 protons.
All of this comes out.
The data are all good.
So now what do we want to talk about?
What's all this about?
I said electronic structure leads to properties.
The first thing we want to talk about is reactivity, chemical reactivity.
Well it's pretty clear from looking at these XPS data that those inner shell electrons aren't going anywhere.
They're too tightly held.
So first thing is the only electrons that we can even think about involving in chemical reactivity must be the electrons in the outermost shell.
These are called valence electrons.
The outermost shell is called the valence shell.
So let's get that down because that's probably important.
So chemical reactivity determined by only electrons in outermost shell.
And we're going to term that the valence shell.
And those electrons are called the valence electrons.
And, these are the only ones that we expect to see involved in chemical reactivity.
And we want to have a measure.
You can see that there are subtle differences between the energies that hold the valence electrons of carbon as opposed the valence electrons of neon.
And so what we can do is have a measure of the ability for valence electrons to react.
That is to say, to participate in chemical activity.
And that measure is called the average valence electron energy.
And there's no special symbol for that, so we just call it AVEE.
And it's just the values of the valence electron energies averaged.
So if I want to take the average valence electron energy for say, oxygen.
So I can take it right off of there.
I can see I'm going to need two times the ionization energy of 2s plus four times the ionization energy of 2p.
And those data are up there.
All divided by 6.
2 plus 4, the total number of valence electrons.
And if I go through the math up there I get 1.91 megajoules per mole, which I prefer to expresses as 19.8 electron volts per atom.
And when we go through and calculate the values of average valence electron energies, we find trends.
And here's what the trends look like.
So you have, first of all, following along with the ionization energies, very similar values.
Pardon me.
So here's the plot of average valence electron energy in bar heights arranged as the elements are found on the Periodic Table.
So you have hydrogen here, helium on this scale would be way up.
And then here's lithium, beryllium, boron.
And so you see a monotonic increase as you move from left to right.
And you see a monotonic decrease if you stay in the same column from low mass to high mass.
So what's going on there?
Why do we have that change?
We have similar electronics structures.
We have similar electronics structures in a given column and lithium is s1, sodium is s1, potassium is s1, rubidium is s1.
And let's put some values on here.
Let's put some values.
For the majority of elements in the Periodic Table, we get values of average valence electron energy less than 11 electron volts.
And when we have such values, we have the following properties: the valence electrons are weakly held.
The valence electrons are weakly held.
How do we know this?
Because the binding energy isn't so strong.
That means the electronic are weakly held.
So we say that the element is a good electron donor.
It's a good electron donor and we term this a metal.
The property that makes an element a metal is that it has a low value of average valence electron energy and therefore, it is a good electron donor.
And it turns out about 75% of the Periodic Table is made of metals.
At the other extreme we have high values of average valence electron energy.
High values.
So when the average valence electron energy is high, we have the complementary set of properties.
So that means the valence electrons are tightly held.
Valence electrons tightly held.
Tightly held.
And so the element is a poor electron donor.
But complementary fashion, it is a good electron acceptor because when it gets near electrons it tends to grab them and hold on to them.
So this is a good electron acceptor.
And we term such a element a nonmetal.
And so roughly, about 25% of the Periodic Table is non-metallic.
And I think I've got that shown here.
And you see just a few elements off to the upper right.
And then in the middle, we've got this-- about a half a dozen elements and they have values of average valence electron energy intermediate between 11 and 13 electron volts.
And so, they can behave either as electron donors or electron acceptors depending on who else is in the room.
So if you put an element like silicon with a very strong metal, silicon will act as an electron acceptor.
If you put silicon in the presence of something that's a very strong nonmetal, silicon can act as an electron donor.
It has that dual property.
And these elements are called semimetals or metalloids.
And I think I've got some images to portray that.
There's the semimetals.
And if you look carefully at the Periodic Table that you have, you'll see that there's this red staircase here.
So there's silicon, arsenic, tellurium, antimony and so on that act as semimetals.
They can behave in both fashions.
All right, so now with this classification scheme, let's start thinking about chemical reactivity.
And seeing if we can, on the basis of where the elements are found on the Periodic Table, start to make some judgments.
Well I know one thing right off the bat that along the right-hand column, the noble gases are chemically inert.
That much we know.
Noble gases are chemically inert.
I know there's a Nobel Prize out there for getting xenon to combine with fluorine, and the chemistry textbooks love to point out these exceptions.
But by and large, the group 8 elements are chemically inert.
So what do we know about their electronic structure?
They all have the same electronic structure ns 2, np 6.
With the exception of helium.
Helium, because it's 1s 2.
There's no p. So beside helium, starting with neon, we've got 2s2, 2p6.
Argon is 3s2, 3p6, and so on.
n2 p6, it has 8 p electrons.
It has a full valence shell.
And because the last electrons are p electrons, this is termed octet stability.
There's something about having a full shell that renders the system satisfied energetically.
And it doesn't want to react anymore.
So we say octet stability, and if you want to be a wise guy, parenthetically you whisper helium is due at stability.
But you know, whatever.
All right, so let's make this octet stability a hypothesis and see how far we can go with it.
So the next one I want to look at is sodium.
Now sodium, it's got the electronic structure 1s 2, 2s2, 2p6, 3s1.
Now, I really don't need to talk about anything up to n equals 3 because this is not valence electrons.
I just put it up there for completeness, but this is all we have to worry about.
Now we know that sodium is a metal.
It's a metal.
It's got an average valence electron energy of about 5.2 electron volts, which happens to be the ionization energy.
Because that's trivial; it's only got one electron, so the average valence electron energy is equal to the ionization energy.
So it's a metal.
So it's a good electron donor.
But let's think about that.
If we could get rid of that electron in sodium, we could then turn it into something that has the same electronic structure as neon.
And why do we want to do that?
Well because there seems to be an energy well there.
If we can make something neon-like, then it's going to be happy.
So let's do it.
Let's write a reaction sodium to lose an electron and become sodium plus.
And sodium plus now is this minus 3s1.
So it's isoelectronic with neon, and neon has the octet stability.
But charge neutrality forbids me to say, OK, lose an electron.
It won't.
The only way for sodium to lose an electron is to find an acceptor.
So if you put a good electron donor in the presence of a good electron acceptor, then the donor can give an electron to the acceptor and both profit from the transaction.
So, where are we going to find a good electron acceptor?
Well I just told you, the nonmetals are the really good electron acceptors.
So let's go zooming over to the right-hand edge of that row and look at chlorine.
We need an electron acceptor.
So choose just for argument's sake, chlorine.
It's got an average valence electron energy way, way up there.
So let's go.
Chlorine, it has the electronic structure of neon plus 3s2, 3p5.
So chlorine, if it could acquire an electron, would then become isoelectronic with argon.
So imagine chlorine plus electron becomes Cl minus and Cl minus is isoelectronic with argon.
So now I've got two elements.
One that's one electron richer than inert gas structure, and one that's one electron poorer than inert gas structure.
If I put them together, put them both together in the same reactor, they will react and electron transfer occurs via electron transfer.
Chemical reaction occurs.
Chemical reaction occurs through electron transfer.
And what's the purpose of electron transfer?
It's to achieve full valence shell.
Achieve full valence shell occupancy, if you like.
Achieve full valence shell occupancy.
So, now we've got it.
But there's more.
There's more.
What happens after the electron transfer step?
We don't have neutral species anymore.
The sodiums have given up their electrons to become sodium ions.
And the chlorines have acquired electrons to become chloride ions.
And these are free.
Let's make this simple.
Let's make it a gas phase reaction.
So sodium vapor and chlorine gas have become sodium ion vapor and chloride ion vapor.
What do you know happens when you have charges of opposite value, opposite polarity?
There's a coulombic force of attraction.
So the sodium will attract the chlorine.
So let's put the two together.
So sodium attracts chlorine, but it's not over.
There's more sodiums, there's more chlorines.
So chlorine attracts sodium, which attracts sodium.
Which could attract more chlorines.
Which could attract more sodiums.
And what do you see happening here?
Well we're now starting with a gas phase and we're forming some giant atomic aggregate.
This is a giant ion aggregate.
What's the state of matter going to be if this thing's honking big?
It's going to be a solid.
We're going to form a solid starting with those two gases: sodium vapor and chlorine vapor.
And what about the solid?
What do we see about the atomic arrangement here?
All the sodiums are the same size and all the chlorides are the same size.
They have the same coulombic forces of attraction in between them.
Can you see that they're going to form not just a solid, but they're going to form a solid consisting of atoms in a regular array?
This is going to be an ordered solid.
And there's a plain Anglo-Saxon word for an ordered solid, it's called a crystal.
Now look at how far we've come.
Look at how far we've come.
10 minutes ago we hadn't done anything about chemical reactivity.
We made one observation.
We had the tools of course.
We were ready.
We're ready for discovery.
We had the tools.
We had the average valence electron energy.
We knew that helium, neon, argon, krypton, xenon, radon, et cetera, all have this inertness.
We postulated that the octet stability was some-- as a sweet spot.
And then we went with it.
And operating with that one postulate, we get to electron transfer and we're concluding that if I take a really, really good metal-- see, I can generalize this now.
There's nothing peculiar about sodium that makes it react this way and no other element will react this way.
I could replace sodium with any other metal.
I could say any metal, any metal will react with any nonmetal in order to achieve octet stability.
So now you can take big pieces of the Periodic Table and conclude that entire sets of elements will act in this way to form ionic compounds, crystals, atoms of regular periodicity.
That's a nice pun there, right?
We started with the Periodic Table and now we have periodic spacing of atoms.
I think that's beautiful.
Obviously this view is not shared by the rest of the audience.
This crystal consists of ions.
It's held together by what chemists like to refer to as bonds.
This is called an ionic bond.
And what you're witnessing here is ironic bonding via electron transfer.
So we've now categorized the first form of primary bonding.
Sodium chloride, magnesium oxide-- ionic bonding.
And ionic bonding must give us solids at room temperature.
If you've got ionic bonds and you don't have a solid, clearly you are at elevated temperature where at elevated temperature, the thermal energy is disruptive enough to break those bonds.
Those are very, very strong bonds.
And what we're going to do next day is look at the nature of those bonds.
So I think I'm going to stop the formal lesson at this point.
And by the way, perhaps I failed to mention on the first day of the class.
What I'm doing is I'm lecturing for, out of the 50 minutes, about 45 on hard core topics.
And then, about the last 5, 7 minutes, I want to go to something related to these-- chemistry in the world around us.
That's why you see this little break.
But it doesn't mean that class is dismissed.
There's a difference between changing topics and dismissing class.
I don't know how people confuse the two.
But somehow I've inadvertently communicated to you evidently.
So we're not dismissing yet.
We're going to try to apply this knowledge somehow.
So what I wanted to do was to show you what happens with these ionic solids in commerce.
And in particular, I wanted to talk about metallurgy and the best form of metallurgy, which is of course, electrometallurgy.
Which is the kind of research that I'm involved in.
And electrometallurgy is quite pervasive.
The aluminum beverage can is made by electrometallurgy, as is magnesium.
So I'm going to talk to you a little bit about magnesium.
This is a bar of magnesium.
Obviously, it's magnesium because if it were steel, it would be so massive that I wouldn't be able to fling it around like this.
The density of iron is 7.87, the density of magnesium is about 1.76.
It's less than 2.
It's 2/3 of the density of aluminum.
It's lighter than aluminum.
Now perhaps some of you've been told that magnesium burns.
Well, let's see what happens.
There's a billet of magnesium and this is the key made of steel.
We're OK. We're OK.
It's not going to burn.
That's a myth.
Magnesium in powder form, magnesium in ribbon form will burn.
The surface to volume ratio is so high that the oxidation generates a lot of heat.
But here the surface to volume ratio is so low that it's fine.
In fact, I went to school at the University of Toronto.
And at the time the department head was a man by the name of Pigeon who invented a process during World War II to make magnesium cheaply.
And when he came into the classroom to lecture on the day that he would talk about magnesium, in those days they had lab benches at the front of the class with sinks and gas outlets.
And he would put the magnesium billet on a stand and light a Bunsen burner under it.
And of course, people in the class sort of leaned back, and he would lecture for 45 five minutes with the Bunsen flame burning the whole time.
And then towards the end he would extinguish the flame, let the billet cool.
And then pick it up and go back to his office with it to make the point.
All right, so how do we make these metals?
Well, what we do is we reverse the electron transfer step.
And we start with the cations and anions, and we force the electrons back onto the cations anions.
Thereby, creating the neutrals again.
This is called electrolysis.
So cation plus electron gives us neutral.
Anion minus electron gives us neutral.
So in the case of magnesium, we start with magnesium chloride, which dissolves and forms magnesium cations and chloride anions.
So, at elevated temperature-- in this case, at about 700 degrees Celsius, the ionic solid becomes an ionic liquid.
And it's clear, colorless, and has the fluidity of water.
And this is a very simple schematic, so we've made the cathode of steel negative and on the negative electrode, magnesium ions are reduced to magnesium, which is a liquid at this temperature.
And these are little bubbles of magnesium and they pool here.
And we collect them and syphon them off.
And on the other electrode, we're making bubbles of chlorine.
And they rise and we collect them.
So we reverse nature.
If you put chlorine in the presence of magnesium it wants to form magnesium chloride.
Here we do the reverse.
So electrolysis undoes the spontaneous electron transfer of ion formations.
So I decided I better get something more authoritative than my little cartoon.
So first I looked on the library website and I found that there was an article in this volume of Advances in Molten Salt Chemistry.
And there's this article here, an authoritative article about chemistry and electrochemistry of magnesium production.
And so I decided to turn to the article and let me read the first paragraph of this authoritative article.
A cubic kilometer-- what is a cubic kilometer?
It's a cube, 1 kilometer on edge.
A cubic kilometer of sea water contains approximately 1 million tons of magnesium.
1 million tons of magnesium as the salt magnesium chloride decahydrate dissolved in sea water.
More than has ever been produced in one year by all the magnesium plants in the world.
The world production of magnesium right now is about 600 million tons.
No, furthermore, sea water contains only 3.7% of the total magnesium present in the earth's crust.
Clearly magnesium resources are ubiquitous and virtually inexhaustible.
So when people tell you we've got to recycle because we're running out of resources, I read this-- I don't know.
You might want to recycle for other reasons, but you can't make the case it's because of scarcity.
This is magnesium that has been made by this electrolytic root.
And what's the value of this?
It can substitute for steel in automobiles.
It's density is less than 2.
Steel is about 8.
Factor of 4 you lightweight the vehicle, which means per unit distance traveled, less fuel consumed.
Fewer emissions.
And less fuel, which means less dependence on imported petroleum.
So why aren't we doing this?
Why aren't we lightweighting our vehicles with magnesium.
Because the bonds are so strong the energy required to make this metal is so high that it's very costly.
If I give you a light bulb that costs three times what the other light bulb does and burns only twice as long, you won't buy it.
If it costs three times as much and burns 10 times as long, then you're willing to pay the premium.
So what we do in my research group, among other things, is study the electrochemical processes by which we make these metals in order to make the process more efficient.
Thereby reducing the cost, making these lighter materials more competitive.
Thereby, making the world a better place by electron transfer in service of humanity.
And I invite you to do the same.
But in order to do so, you have to learn the lessons of 3091.
Because after all, this is the most important subject you will take at MIT.
All right, we'll see you on Friday.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
OK. OK. Settle down.
Settle down.
Settle down.
So the weekend doesn't begin until after the 3.091 lecture.
OK. A couple of announcements.
Tuesday, Quiz 3 based on Homework 3.
The final exam has been scheduled.
In fact, all the finals are scheduled now, so please consult the Registrar's listing.
The celebration of celebrations, the 3.091 final exam, will be Tuesday, 15th of December in the morning, 9:00 am to 12:00 noon over in the Johnson Athletic Center.
So I urge you to go through the final exam schedule and then make your travel plans when you know what your last obligation is.
And book them soon, because we've got about a quarter of a million students in the Boston area and the semesters all come to a close within a very narrow time window, and everybody's trying to get through a wormhole called Logan Airport.
So you want to be ready.
Do not try to leave town before you've met all of your obligations.
You can't leave before you've finished all of your finals.
But you know what they are now, so call your travel agent or make your internet booking.
And an announcement for Professor Paul's section.
Owing to the holiday on Monday, he's going to have office hours from noon to 1:30 today.
So if you're in Professor Paul's section, you'd like to catch up with him, noon till 1:30 today.
All right.
Let's get to the lesson.
The last day we started looking at octet stability, and we looked at octet stability and what it means in terms of shell filling and some sweet spot in energy with respect to reactivity.
And a filled shell leaves us with this electron configuration, ns2np6 for n greater than 1.
In the case of n equals 1, there's just ns2 for helium.
Otherwise, ns2np6.
2 plus 6 is 8.
There's the octet stability.
It's observed trivially in noble gases.
And then we saw that it can also be observed in certain ions.
And how do we get to octet stability?
Electron transfer.
And here's the prototypical reaction.
I like this.
This is very iconic.
It puts it in broad terms.
The marriage of an electron donor with an electron acceptor leads to the formation of a cation and an anion, thanks to electron transfer from the donor to the acceptor.
It doesn't end there, though.
You have cations and anions in the gas phase and they're attracted to one another by coulombic forces, or electrostatic forces, and that leads to ionic bonding.
And today I want to go deeper into it and study the energetics of ion-pair formation.
So let's do it.
And what I'm going to do is study this with reference to an example of sodium chloride.
So we will go through the energetics of sodium chloride, which we know is going to exist as Na plus and Cl minus-- sodium being the electron donor and chlorine being the electron acceptor.
So what I'm going to do is show you a plot of energy as a function of separation.
So you're going to have to be pluralistic here in your ideas.
So, for example, if I wrote this word by itself you don't know if I'm saying lead or I'm talking about the metal lead.
The only way you know is in context.
And so you have to know in context here.
So r is not the radius of the atom.
In this case it's the symbol.
And I'm using this symbol not to confuse you.
This is what professionals use.
So if you go to the text and the literature you'll see r. This is the interionic separation, and it's determined nucleus to nucleus.
So I'm going to put a sodium ion at the origin.
This is the origin.
Energy is on the ordinate.
So I'm going to model this as a hard sphere.
So this is sodium ion, Na plus, and it's got some finite radius.
And its radius, I'm going to call it r-plus.
That's the radius of the cation.
And next to sodium I'm going to put the chloride anion.
And it's bigger.
It's bigger.
It's not to scale, but it's bigger.
So this is chloride, the Cl minus, and it has its radius, rCL minus.
And then the interionic separation is measured from the sodium nucleus to the chloride nucleus, and it's given the symbol r0.
That's the interionic separation.
And we could we write that r0, in fact, is strictly r-plus plus r-minus because we're using a hard sphere model.
So there's no shrinkage when the two touch.
So that makes sense.
Now what I want to do is calculate the energy here.
The only energy that we have here is electrostatic.
So we're going to start and say, imagine if we had these two ions separated by infinite distance and we brought them to a separation of r0.
How much electrostatic energy would be stored there?
So I can write that as E, and I'm going to call it attractive.
There's a force that attracts these two ions together, and that's given by Coulomb's Law.
So that's the product q1 q2 over 4 pi epsilon0r, in general.
I'm making this as a function of r and then we're going to figure out how to get to r0.
Let's put a little ledge in here.
So q1, I'm going to make that the sodium just for grins and chuckles.
So q1 is equal to the z, the valence on the sodium, times the charge, the elementary charge.
So the charge on the sodium is plus 1-- so it's plus 1e-- and q2 is going to equal the charge on the chloride times e. And in this case the chloride ion is negative 1, so q2 is minus e, q1 is plus e. Now if instead I were doing magnesium oxide, magnesium would be plus 2, oxide would be minus 2.
So that's where the charge on the ion comes into play.
So we can put that in here and will make this z-plus times e, z-minus times e. So there's q1 q2 over 4 pi epsilon0 r. And in this case, for sodium chloride, this is plus 1 times minus 1 is minus 1 e squared over 4 pi epsilon0 r. So this is a function of r. This e is 1/r.
That's the hyperbole, and I can draw that here.
So it's going to look something like this.
All right?
It's going to be a right like this.
Good.
So that's the attractive force.
Now how do we avoid the ions just blending together?
Why do they stop at r0?
What puts the brakes on the coulombic force?
Ah!
For that we have to look at the fine structure of sodium.
So let's look at the fine structure of sodium.
So sodium is net charge plus 1.
But sodium more properly, if I go to the next level of structure, is really 11 protons in the nucleus and it's got electrons around it-- in total, 10 electrons.
So it's net charge is plus 1, all right?
Now if I'm way over here and I look at sodium I just see something of charge plus 1.
And, in fact, I could model sodium as just a point charge, a plus 1, and do all the electrostatics and I would be perfectly accurate in my estimation.
But as I get closer I see there's fine structure.
It's like so many other things in life.
You know, from a distance it looks really good, and you get up close you go, eww.
[LAUGHTER]
I'm not going to mention any names, all right?
But it's Friday.
Be careful.
So as I get up closer I realize that it's plus 1, but the plus 1 is plus 11 mediated by minus 10.
So when things start getting close together this exterior of negative charge becomes manifest, palpable.
So let's look at the chlorine.
The chlorine's coming, sees plus 1, plus 1.
It starts getting closer and closer.
And now what happens?
The negative electronic configuration on the outside of sodium is interacting with the negative electronic configuration on chlorine, and the result is you have electron-electron repulsion.
And we've got an equation for that, and that's given by E repulsion is equal to some constant b over r to the n. And this n is called the Born exponent.
It's not the quantum number.
So r today means interionic separation, n means Born exponent.
And you have to determine it by experiment and the value lies between 6 and 12.
And we have to determine b by experiment.
So let's say this thing's got a-- pick a number in the middle.
Say it's 8.
So let's plot r to the eighth.
So that's going to hug the abscissa, and it's going to come in really, really close.
And then somewhere around this distance here, where the electron-electron repulsion starts to be felt, this thing takes off.
But 1 over r to the 8 goes way, way up fast.
So instead of being a gentle curve it's more like a hockey stick shape here.
And so now the net energy is the sum of the negative attractive energy and the positive repulsive energy.
And if we sum the two what are we going to get?
Well, way out here 1 over r to the 8 is negligible.
And the net value-- so E net, in red-- is essentially equal to E attractive.
And at very, very low values of r, r to the 8 dominates r. So we have this as the net value.
And the two sum-- and I can't quite do it because of the way this is drawn, but I think you can see.
These two eventually equilibrate and go through a minimum.
So I'm going to cheat a little bit here and I'm going to add these two in such a way as to go through a minimum value of energy at r equals r0, which is the sum of E attractive and E repulsive.
So we can sum those two and let's see what we get.
At r equals r0, E net is equal to its minimum value.
So how do you find a minimum in a function?
You take the derivative.
I'm going to put some math to work here.
So we'll take dE by dr and set it equal to 0.
And the value of r at which it equals 0 is termed r0.
OK?
So we'll just go through and take the derivative of that thing.
And I'm not going to go through all the math but just show you the set-up here.
What's the other thing that we know?
dE by dr represents what?
That's force.
So I could say that at r equals r0 the net force is 0, which is what you'd expect.
Because if the net force isn't 0 it's going to either push the ions farther apart or draw them closer together.
So this is mathematics imitating reality.
What a concept.
Math working for you instead of you working for math.
So what do I have?
I have everything in these equations except the value of b. I'm assuming we know the value of n from experiment.
We don't know the value of b.
And so you can solve to get the value of b, and once you've got that you can put everything together and give an expression for the energy of the system at r equals r0.
When you plug everything in you get this: z-plus, which is the net charge on the cation, times z-minus, which is the net charge on anion, divided by 4 pi epsilon0 r0.
1 minus 1/n, where n is the Born exponent.
And this is valid at r equals r0 only.
If you're not at r equals r0 then you can get the value of b and then put it into this expression, where E net will equal E attractive plus E repulsive.
So there it is.
And so this represents-- this is the energy of a single ionic bond, because that's all the energy that's there.
It's the single ionic bond.
And the second thing that we realize is plus times minus is net minus, so this means that it's negative quantity, as it should be.
It has to be a negative quantity.
All right.
So what do we have here?
What we've seen by going through this derivation is the recognition that the ionic bond is electrostatic attraction mediated by electronic repulsion.
It's the balance of the two.
And those words sound so good to me that I'm going to write them down.
Electrostatic attraction mediated-- another lovely word-- mediated by electronic repulsion.
So that's how you get to the final setting here of the interionic separation.
So what are the characteristics?
What does this lead to in terms of characteristics of this bond?
Characteristics of the ionic bond.
First of all, it's omnidirectional.
This is a concept based on the fact that the electric field radiates in all directions uniformly.
So the negative field coming from the chloride ion is uniform in all directions.
There's no preferred direction.
Omnidirectional.
OK?
E field-- oh, I better not say that.
Electric field, not the energy field, radiates in all directions uniformly.
And that's going to have consequences.
I'm not just telling you this because we like cataloguing things.
This isn't a bookkeeping class.
So we're going to come back, use this fact.
And the second thing is that the bond is unsaturated, which is a chemical way of saying that a given ion can bond to more than one other ion.
In other types of bonds that's not the case.
A given atom can only bond once and then it's done.
Whereas in this case the ion can bond to a plurality of other ions.
So ions bond to more than one.
OK?
Plurality of bonds is formed.
They're polygamous, if you like.
So what does that mean?
That means that here is what happens.
We've got the blues as the sodiums, and for any given sodium it forms bonds without limit until the number of bonds is stopped by physical limitations-- not because the E field was saturated.
It's unsaturated.
You just can't jam any more chlorides physically around the sodium.
That's why the sodium is only bonding to the number of chlorides that it bonds to.
There's no intrinsic limitation.
So what happens when you get to this situation where you have omnidirectional forces, unsaturated bonds, and ions that you can model as hard spheres of constant radius?
All the sodiums have the same radius, all the chlorides have the same radius.
You make a 3-dimensional ordered array.
So you can make an infinite atomic ordered array, which we use the simple Anglo-Saxon word last day to describe: crystal.
You form a crystal.
And as a result ionics have to be solid at room temperature, because if you've got thousands and thousands of atoms together in one aggregate they're not going to float.
They're going to settle.
Put another way, the strength of the bond, the amount of energy in here, is so great that the thermal energy of the room isn't great enough to disrupt this bond.
It's a combination of unsaturated, omnidirectional, and high energy.
So we form solids at room temperature.
OK. Now I want to show the energetics of that one because this is good.
You know, I promised you I wouldn't do derivations, so I'm not going in detail on this.
I'm giving you just enough so that I can introduce the characters here.
You know, how else am I going to introduce the Born exponent?
Am I'm just going to say, there's this exponent n, the Born exponent.
We're going to introduce it in context.
So now you know what the energetics are.
So now I want to prove to you energetically along this line that crystals will form.
So let's imagine-- we're going to do this thought experiment.
We're going to take three ion-pairs of sodium chloride.
So here's three ion-pairs of sodium chloride.
And I want to compare these three ion-pairs.
So this is an ion gas.
And the distance between ion-pairs is great enough that one pair doesn't affect the other.
The electrostatics are only strong within the pair.
So we'll just label this infinity with quotation marks around it.
They're very far apart.
When a physicist says they're very far apart, very is code for infinity.
So this one doesn't interact with this one, which doesn't interact with this one.
And I want to compare the energetic state of the ion dispersion to what would happen if I were to put all of those in a single line.
Plus, minus, plus, minus, plus, minus.
So then this has a certain energy state, it's the energy of the ion line.
And I want to show you that there's an energy decrease in collecting all of these and ordering them into a line.
So there's more energy in a line dance than in ballroom dancing.
That's what we're going to say ultimately.
OK?
So let's compare the energies.
That's all we're going to do.
So what's the energy of this?
Well, we're not going to do 3 versus 3 Let's think big.
It's Friday.
So let's take Avogadro's number of pairs, shall we?
So the energy of the ion dispersion would then equal-- that's the energy of one pair, and they're infinite distance apart, so there's nothing to be gained by putting them in the same chamber.
So it's just going to be N Avogadro, if that's the number, times the energy evaluated at r equals r0.
So we're done.
And we know what that is.
I don't have to rewrite it for you.
OK.
So now what I have to do is get an estimate of the energy of the line and show you that the line is at a lower energy state.
Well, let's see.
Jocelyn, take the top one and the middle one, please, but not the bottom one.
Thanks.
OK.
So now let's look at the energy of a line.
So we're going to do-- here's my line.
I'm going to get the colored chalk.
Green and blue.
You know, some people think being a professor is so cool because you get to travel, you get to research, and so on.
It's the colored chalk.
[LAUGHTER]
It's the colored chalk.
All right.
So there's a sodium.
And on each side of the sodium we'll put a chloride.
All right?
And I'm going to just keep going this way.
Here's a sodium and here's a chloride.
All right.
And now what I'm going to do, I'm going to start here at this sodium and I'm going to count the energy, electrostatic energy, that's in this system.
So we're going to count to the left, we're going to count to the right, and then we're going to multiply it by the total number of ion-pairs.
And I know that the ones on the end aren't the same, but the number of ends over N Avogadro, that's peanuts.
The edge effects are negligible because there's such a giant middle.
That's how you model this stuff.
You don't obsess over the fact that the last ion doesn't see anything on that side.
You just do it and forget about it, because you know you wouldn't do it for 5.
This would be a big problem.
But if you have Avogadro's number, who cares?
It's called risk assessment.
All right.
So let's look at the energetics here.
So what we've got is the energy of the ion line.
OK?
So let's start with this central one.
And separated by distance r0 is the chloride, and there's an attractive energy here.
So that's going to equal minus e squared over 4 pi epsilon0 r0.
Now I'm going to keep going, because the field is unsaturated and goes in all directions.
So this sodium, that's a distance 2r0 away.
It's got a repulsive force exerted on this sodium.
So let's add that.
So that's going to be plus.
A repulsive force raises the energy of a system.
That's e squared over 4pi epsilon0 times 2r0.
And let's keep going.
So now let's go 3r0 away.
So 3 times r0, that gets me out to the chloride over here.
Let's put 3r0.
Now that will take me from the center of the sodium to the center of the next chloride.
And that's going to be attractive.
So that'll be minus e squared over 4pi epsilon0 times 3r0 plus, et cetera.
So you see how this goes.
So you go all the way out, you add them all up, and you go the other way.
And so on and so forth.
So that's how the derivation goes.
You might say, hey wait a minute.
What happened to the Born exponent and the repulsive energy term?
Well, where's the repulsive energy term going to be felt?
It's called electron-electron repulsion.
Axiomatically, the electrons here are nowhere near these electrons.
See, you only have to count it for the nearest neighbor.
So we can patch that in at the end.
And we do.
I haven't forgotten.
But I'm not going to spend a whole day trying to derive this thing.
I'll show you how it starts to evolve.
At some point you end up with something that looks like this.
e squared over-- in fact, I'll put the minus sign up here.
Minus e squared over 4 pi epsilon0 r0.
And you're going to double it, because you're going to go one side and the other side, and you're going to get a series that looks like this: 1 minus 1/2 plus 1/3 minus 1/4 plus blah, blah, blah.
Yeah.
OK.
So what does this look like?
I've really broken this into two pieces.
So this coefficient out in front here, you should now be able to repeat this in your sleep: e squared over 4 pi epsilon0 r. This is electrostatics, isn't it?
Electrostatics.
This is the consequence of Coulomb's law.
What's this second term here?
What's this all about?
Geometry.
This is dictated by atomic arrangement.
So I could calculate this if I took, instead of a line, what if I put them in a sheet subject to the constraints of those sizes and plus 1 and minus 1?
So what would be?
I'd start at the sodium and count how many chlorides?
If I'm on a plane there'd be one, two, three, four.
And then how many sodiums?
Well, they'd be on the backside of each of the chlorides.
And I'd add them all up in 2-space and I'd end up with another coefficient here.
All right?
And we compress all of this into a coefficient which we call the Madelung constant.
And it's a function of the atomic arrangements.
So different crystal structures have different Madelung constants.
It's named after a German professor, Madelung.
In 1910, he published calculations for the energy of a system of point charges-- just abstract theoretical paper.
And then about 10 years later another German professor by the name of Paul Ewald-- he did his PhD for Sommerfeld-- he published a paper in which he actually made the calculation for ion crystals, and he came up with this constant.
And to show you the class of the guy, instead of naming the constant after himself he named it after Madelung.
Now that's class.
So Madelung did the first calculation so he gets named.
So now what we're going to do is we're going to multiply by the N Avogadro, because I've got N Avogadro of these things, and we're going to put in the Born exponent patch and so on.
And here's what the final expression looks like for the line.
There are a few algebraic tricks that I'm not willing to do in class because I don't think that's a profitable use of our time in a chemistry class.
But if you want to try the derivation I have it in full and we can compare notes.
So once we get the patch in it's going to look like this.
It'll be minus.
There'll be the Madelung constant times N Avogadro e squared-- and this is already assuming it's plus 1, minus 1.
If this were magnesium oxide there'd be a 4 in here.
4 pi epsilon0 r0 times 1 minus 1/n, where n is the Born exponent.
So compare this to this one here.
What's the only difference?
The only difference is the Madelung constant, right?
It's the only difference.
So E of the pair dispersion is really equal to E of the line divided by the Madelung constant.
So what I'm trying to prove to you is that E line is more negative than E of the dispersion.
So it all hinges on the magnitude of M. If M is greater than 1 we win.
If M is less than 1 I've just proved to you that water runs uphill, so that's a bad day.
All right?
So let's calculate the value of M. And you can go to your algebra books.
And you've got this series natural log-- and engineers write natural log "ln."
I know the mathematicians write "log."
Uh-uh.
Engineers-- uh.
That's 1 plus x. OK?
Natural log, 1 plus x.
You can expand this as x minus x squared over 2 plus x cubed over 3, dah, dah, dah, dah.
You know, look that one up.
Set x equal to 1, which is what we've got, right?
Because we've got 1 minus 1/2, 1/3, dah, dah, dah, dah.
Go through it and you'll get the value that M, according to this, will give you 2 times the natural logarithm of 2, which is 1.386-- which is greater than 1.
And so we're golden.
That means that the energy of the line is lower, more negative, than the energy of the dispersion.
So I'm going to do this pictorially.
Let's make an energy level diagram.
And the energy level diagram will look like this.
So up here the energy is 0 and everything is negative.
So if I put this as minus 1 unit.
All right?
These are all negative values increasing in this direction.
So this is the dispersion of ion-pairs.
So all we've done is take a cation and put it to the anion.
We've just seen that if we do the calculation for the line it's 1.386 times whatever this is.
So that gives us-- this is for the ion line, which I'm going to take the liberty of calling a 1-dimensional crystal.
It's a 1-dimensional ordered array.
And what I can do is go to the 3-dimensional array, start at the lower right-hand corner with that chloride.
Calculate the distance to each of the nearest neighbor sodiums.
Go through the geometry.
The next nearest neighbor chlorides, the next nearest neighbor sodiums.
And you'll build an infinite series and you'll evaluate it.
And in three dimensions it's even lower.
It's 1.7476.
So this is for the 3-dimensional crystal.
This is for the ionic crystal.
All right?
3-D crystal.
So what this is showing is that the system keeps making more and more bonds.
Why does it make bonds?
Because the more nearest neighbors it has the lower the energy goes.
So making a 3-dimensional crystal is energetically favored.
Now there are different Madelung constants for different crystal structures.
You say, well wait a minute.
How do you get different crystal structures?
Suppose instead of sodium it's potassium.
What's the only difference?
Potassium is plus 1.
What's the difference?
Size.
Potassium is larger.
They're not going to pack quite the same.
And so depending on the relative ion sizes-- I mean what if I have something like silver iodide?
Iodide is huge.
Silver is so small it'll fit into the interstices between touching iodines.
So it's going to have a different crystal structure, and the different crystal structure will give us a different Madelung constant.
And there it is.
So we've come a long way with that little assumption of octet stability.
So now let's take a look at what the properties are of these things.
They're solid at room temperature because we've got strong bonds.
High melting points.
Bonding is related to melting point.
Now think point is dictated by bonding, because now you're comparing thermal energy versus the cohesive energy of the crystal.
So tightly bonded substances melt at high temperatures, weakly bonded substances melt at low temperatures.
Transparent to visible light.
How do I know that?
Because I'm the professor.
No.
How do I know that?
How do we think about it when someone says to you is something transparent, in this case to visible light.
Well, what I do is I say, here's the solid.
This is the ionic solid and here is visible light, h nu.
And I'm going to write 2 to 3 electron volts per photon.
And what happens when I want to decide whether this is transparent to visible light?
Now look at the modeling here.
Photon is a squiggle.
I don't know if that's Cartesian space.
This is like a crystal, right?
I'm speaking California.
It's like a crystal.
So now if I go inside I want to make the energy diagram.
So what's the energy diagram look like?
All right?
So now the question is how does-- if this is the energy diagram of the crystal and I make this the energy of the photon of visible light, I'm going to compare how much energy the photon has versus how much energy it takes to excite electrons all the way to a new state.
Because if you don't excite them all the way to a new state-- they can't go part way, so nothing happens.
And if nothing happens the photon goes through and that's transparent.
Now what do I know about the binding energy and the energy level diagram of Na plus?
Well, it's isoelectronic with neon.
And I know that neon has an average valence electron energy of about 20 electron volts.
So my guess is the visible light is not going to do anything.
And so it just passes right on through.
See, we got all that.
Electrical insulator.
How do I know that?
Well, all that glitters is not gold, but it must have free electrons.
And these electrons are all bound, and they're tightly bound.
Hard and brittle.
If it's going to be ductile the atoms need to be able to slide over one another.
Well, there's no way these can slide over one another because to slide over one another requires that at some point the two sodiums are going to be nearest neighbors, and the repulsive forces are so high the crystal fractures.
So if you try to deform an ionic solid you will get it moving in accordance to the elasticity.
So force will be proportional to the extension-- Hooke's Law-- but if you try to plastically deform it, you shear it.
Soluble in water.
We'll come back to that later.
Melt to form ionic liquids.
And good for electrolytic extraction of metals.
I showed you magnesium last day.
Today I'll talk a little bit about aluminum.
But that comes later.
OK.
So where do we find elements that are going to form ionic solids?
Well, you go back to the beginning of the lecture.
What do you look for?
You've got to find the box of really good electron donors and the box of really good electron acceptors.
That's where you go.
So the good electron donors are at the left side and the good electron acceptors are at the right side.
So if you take sodium plus chlorine you get sodium chloride.
If you get calcium plus fluorine, calcium fluoride.
Magnesium plus oxygen, and so on.
OK?
Yeah, this shows.
Yeah.
Aluminum plus oxygen, yeah.
There's Max Born.
Max Born, he got a Nobel Prize.
And so did Fritz Haber, but we're going to come to him in a minute.
All right.
So now I want to do this energetic calculation one more way, because right now we've been operating with ion gas but sodium isn't found normally in the form of an ion gas.
So let's do something that starts with elements found in nature.
So we want to form an ionic crystal from elements in their natural state.
And what's going to happen is en route we're going to be able to define a few more terms.
So that's going to make everybody happy because we get more definitions.
And this is called the Born-Haber Cycle, named after Born and Haber.
And what we're going to do is we're going to-- pardon me, there's a C in there-- we're going to invoke Hess's Law.
And Hess's Law is sort of like Kirchhoff's law for electric circuits.
Hess's law says that the energy of a chemical change is path independent.
So energy change in a chemical reaction is path independent.
It's sort of like potential energy in Newtonian mechanics.
It doesn't matter if you take the elevator to the top of the Hancock Tower or if you walk up the stairs, the change in potential energy is the same when you express it from the top of the Hancock Tower, although you might be somewhat more exhausted having walked the steps instead of taking the elevator.
But you don't get any credit in terms of the potential energy.
So let's use Hess's Law in order to describe the formation of sodium chloride.
So I'm going to start with sodium as it's found in nature, and I'm going to talk about room temperature.
So sodium is a solid at room temperature, and I'm going to react it with chlorine gas.
Chlorine's a gas at room temperature and it's a diatomic molecule.
And we're going to react it to form sodium chloride, which is a solid and a crystal.
Now you might say, well isn't that kind of redundant?
No, because later on I'm going to teach you about a form of solid matter that does not consist of atoms in a regular array-- disordered solids.
So we're specifying, I want to form crystal and solid sodium chloride, because that's the reaction that would occur if we were to do it in the lab.
And what have we calculated so far?
What we've calculated so far is this.
We've calculated chloride ion in the gas phase plus sodium ion in the gas phase reacting to form the crystal.
And we've called this the energy of crystallization.
That's this Madelung stuff.
This is the Madelung energy here.
All right?
I'll even put M here.
Madelung energy.
And why am I using H?
Because that's what the books use.
H is enthalpy, and for condensed matter the difference between enthalpy and energy doesn't amount to much.
So H, just for the record, is enthalpy.
And we've been operating with E as energy.
It's almost equal to E, which is energy for condensed matter.
For the gas phase it gets hairy.
OK.
So I want to get us from sodium solid, chloride solid over to here.
So how am I going to do that?
Well, first of all, I know how to make sodium ion gas.
I start with sodium gas and then by ionization I make the electron plus sodium ion.
So this is called the ionization energy, isn't it?
This is the ionization energy.
Sodium gas goes to that.
And now how do I get sodium gas from sodium?
Well that's just called sublimation.
So this I'm going to need delta H of sublimation.
Sublimation is the conversion of solid to vapor.
And you can look that up on the Periodic Table.
I'll show you how to get that in a second.
And now how do I get chloride gas?
Well chloride gas is going to start with atomic chlorine gas, but instead of losing an electron I've got to acquire an electron.
And this action of adding an electron is sort of an inverse ionization, and this is called electron affinity.
And there are tables of electron affinity.
So each element has the ability to lose an electron, it has the ability to gain an electron.
Losing an electron is ionization energy, acquiring an electron is electron affinity.
And just as with ionization energies, if you have multiple electrons you have a first electron affinity, second electron affinity, and so on.
And now how do I get to atomic chlorine?
I've got to dissociate diatomic chlorine.
So this is called dissociation.
So that's the whole thing.
And I'm going to now put some numbers on here.
Let's see.
I'm going to call sublimation step one, dissociation step two, ionization I've got here is step three, electron affinity is step four, and crystallization or Madelung is step five.
And so working off of Hess's law we can say that the total energy required for the formation of the crystal-- delta H for the reaction.
What's the reaction?
The reaction of sodium plus chlorine to make sodium chloride-- is going to be the sum of all of the constituent components.
The sum of all the delta's H. Not delta H's.
delta's H, like attorneys general.
All right?
So now let's add these up.
So we go number one.
Number one I can look up on the Periodic Table.
Where is it?
There's Fritz Haber.
So that's given here.
If you look on the Periodic Table that'll give you the number.
And for sodium it's 108 kilojoules per mole.
Number two, get that from tables.
It's 122.
Number three is just the first ionization energy.
You look it up on the Periodic Table.
First ionization energy of sodium is about 5.3 electron volts, which turns out to be 496 kilojoules per mole.
But look, these are all positive energies and we're trying to make a net negative energy.
So these three steps are all raising the energy of the system.
Finally, acquiring an electron by chlorine is going to decrease the energy of the system, because chlorine is a good electron acceptor.
So that's minus 349.
But watch this people.
The energy in forming the crystal from the discrete ion-pairs is 787 kilojoules per mole, which gives us a net value of minus 410 kilojoules per mole.
So what this is showing you is what the relative contributions are of the different components of that thing.
So here it is in graphical form.
There's the vaporization of sodium.
This is the dissociation of chlorine.
This is the ionization of sodium.
All positive energies.
And now electron affinity.
And look at this contribution from the Madelung energy.
So when things crystallize a lot of heat's given off.
In fact, we can use that in cooling and moderating climate if we're clever about it.
All right.
Now just to show you what the different values are here that you're going to go in and get the lattice energies, you need to know the various r values, you see.
The r0 is simply going to be the r-plus and the r-minus.
So lithium fluoride, there it is.
There's the lattice energy and it's based on the combination of lithium cation and fluoride anion.
So they've gone through and calculated these values.
So there's sodium chloride is 787.
And, you know, you even get things like the boiling points, melting points.
So sodium chloride, for example, melts at about 800 degrees Celsius.
Now if you take magnesium oxide, magnesium oxide is, look, 3,700 versus 700.
And the melting point of magnesium oxide is 2,800 degrees C. Look at aluminum.
Aluminum plus oxygen, look at the end binding energy there.
It's phenomenal, which means it might be useful for-- I'm standing underneath the shuttle here.
This is the tiles underneath the shuttle, and they're made of aluminum oxide because the Madelung energy is so high so it's got the thermal shock resistance.
So now you know how to go and design things for thermal ablation resistance.
All you need is this table.
That's all.
No, you need a little more than that, but this is a good place to start.
If you don't understand this table I don't want you working on the project.
OK?
What else?
All this shows you-- yeah, we're going skip that.
All right.
So now we've got a few minutes here.
What I want to do is last day I talked about magnesium, today I want to talk about aluminum.
And it is also made in an electrochemical process.
In this case the electrodes are horizontal.
We feed aluminum oxide in and we pass current-- and huge currents.
This thing typically runs at 300,000, 400,000 amperes and about 4 volts.
So with the cathode we are running-- remember, we're running nature in reverse.
Instead of the electron donor that aluminum is, we're shoving electrons onto aluminum ion and converting it back to aluminum.
Unfortunately, on the anode side we have to use carbon, and the carbon itself is consumed.
So we consume about a half a ton of carbon to make a ton of aluminum.
So aluminum smelters generate a lot of greenhouse gases.
So you can see this is like a drafting pencil: it's constantly being fed.
And to give you a sense of scale this is probably about 10 feet across, and this gap is about 2 inches, and this is about, I don't know, a foot and a half deep.
It's going to get about 1,000 degrees Centigrade-- liquid aluminum and liquid salt.
So this is what a smelter looks like.
What's the sound of electric current?
Yeah.
The only sound you hear is the fans on the top that keep the place clean.
There's the busbars that are bringing in the current.
And all of these various posts are these things.
So all of the magic is occurring below the floor here.
It was invented simultaneously in the United States by Charles Martin Hall and in France by Paul Heroult.
In the same year they filed patents independently and eventually crossed license when they collided at the World Court.
So this is what happens.
We dissolve aluminum oxide in a molten fluoride called cryolite, which originally came from Greenland, make liquid aluminum carbon dioxide.
Now if we wanted to make this truly green, we want to eliminate greenhouse gas emissions, you need to find an inert anode.
So on an inert anode aluminum oxide would be converted into aluminum and oxygen.
So not only would you not produce greenhouse gases but you'd produce tonnage oxygen, which is marketable.
So some of the work that goes on in my lab is in advanced materials with a view to trying to find an inert anode that would then make this process very, very clean and justify substituting aluminum for steel in cars.
By the way, when you have the field, even though it's a DC current it's a divergent field.
And this is me in a magnesium smelter just in Utah.
There is the busbar for the magnesium cell.
So I'm about, oh, two meters away from the edge of the busbars.
The magnetic field is so high I've got one, two, three, four, five paper clips standing against gravity.
And I asked them for paper clips and there was no office there.
They managed to find a few.
I wanted to see how many would go.
I'm willing to bet I could probably have about seven or eight paper clips up before the gravity would cause them to collapse.
That's the intensity of magnetic fields in these smelters.
So when you drive up to the smelter for the tour you park a fair distance away, you leave your wallet with the credit cards in it, because this is the biggest bulk demagnetizer you can imagine.
If you've got a watch that's got hands that move they're going to be going like this.
[GESTURING]
Yeah.
This is the shuttle.
This is forged aluminum wheels.
The shuttle lands on Centerline racing wheels.
They're forged at a place in California.
They're made of aluminum alloy.
And I don't know if you can read on the side here, it's a little bit light, but these are special tires.
They're made by Michelin So, anyways, the whole message here is learn the lessons here in 3.091 and then we can work together to make metal in an environmentally acceptable way.
And just before I send you on your way for the weekend I thought I'd tell a little joke related to the uncertainty principle.
So the joke goes like this.
Heisenberg is racing down the Autobahn and he gets pulled over by the state trooper who comes to the car and says, where's the fire buddy?
Do you know how fast you were going?
And Heisenberg looks at him he says, no, but I know where I am.
[LAUGHTER]
All right.
Get out of here.
Have a good weekend.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
This Wednesday will be the first celebration of learning.
Test 1 on Wednesday, October 7, you will write during the normal class time.
So you'll have 50 minutes.
And we want to have a little bit of comfort here, so you won't be sitting cheek to jowl.
So before long I'll have the room assignment.
So some of you will write in here.
We'll have fewer people than seats so that there'll be vacancies next to each person.
And then some will write in a few of the other locations, probably 26-100 and the gym above the Walker Memorial.
And we'll get that out to you.
And next week on the 6th we will have no weekly quiz, because enough celebrating.
No point in testing you on the 6th and then on the 7th.
There will be, of course, the mini-celebration tomorrow, quiz 3.
And I'll be available for office hours later today.
Oh, and the coverage, just to remove the mystery, will be right up to the 7th of October.
I've been doing this for over 30 years and I've learned that in order to inspire interest on the part of the student it really pays to have the coverage of the celebration extend up to the lecture before the celebration.
Now obviously I'm not going to drill deep on something I taught you on October 5, but it would be a good idea to stay awake during all of the lectures between now and then.
So last day we talked about ionic bonding.
And ionic bonding occurs with electrostatic attraction between ions that have formed through electron transfer.
And we saw the energy of the ion pair given by this formula, where we have Coulomb's law with the Born exponent.
And then this is plotted.
This is E at r equals r0, and we learned that there were two terms.
The attractive term, which is the Coulombic force here shown.
And then there's a repulsive term, which results from electron-electron interaction when the two lines get very, very close together.
And this is taken from your text.
And I think they did a very nice job here of illustrating as you go to high values of r they're depicting that you have the ions separated by considerable distance.
And there's a certain amount of stored energy, but not a lot.
And then if you go much, much closer than the hard sphere sum of the ionic radii, I think they're depicting here that there's some squashing of the electron clouds.
And you can see that the energy has gone way, way up.
So this is an unfavorable situation, meaning that the energy here is greater than 0.
And there's a sweet spot here at 236 picometers, which represents the ideal location.
And that is the sum of the radius of the sodium ion and the radius of the chloride ion.
And so you can see how energy tracks.
And if you go far, far, far away to the point where they're at infinite separation, there's no energy stored.
So everything makes sense.
And then we said, well what happens if we keep packing these things?
We rationalized that they would continue to do so and ultimately form a 3-dimensional crystal.
And so you can see there's a lot of similarity between what's above and what's down here.
This has been written for a 1-1 system.
In other words, a cation plus 1 and an anion minus 1.
But it could be mediated by the valences.
And what we have here is the structure factor.
This Madelung constant tells us how much we get decrease in the energy of the system by going to a 3-dimensional array.
So depending on the atomic arrangement we'll have a different value of Madelung constant.
We saw that for sodium chloride the value is 1.7476.
And different crystals have different things.
And what determines the crystal structure?
It's a combination of the size of the two ions and their valence.
So what we saw for sodium chloride, this is a structure type.
So obviously sodium chloride is sodium chloride crystal structure, but there is an entire suite of compounds that have radius ratios and charges that end up with a sodium chloride type crystal structure.
And then towards the end we started looking at the Born-Haber Cycle.
And the purpose of the Born-Haber Cycle was to give us a sense of scale of the various constituents in the formation of a crystal.
And what we noted, the takeaway message from the Born-Haber Cycle, is that this enthalpy of crystallization, which is basically this term here, is huge.
It's huge.
It was the big component of the energy required to form the crystal.
It's large and it is negative.
So what I want to do today is start by talking about shortcomings of the business of ionic bonding.
See, how did we get to ionic bonding?
We started with this idea of octet stability.
Octet stability was the driving idea behind all of this.
Octet stability, and in the case of ionic bonding this was via electron transfer.
And so that got us a long way, but it has its limitations.
So let's put up some new data.
So suppose I look at compounds like H2, N2, O2.
Do these things form ionic bonds?
How does octet stability play here?
And so let's start by looking at hydrogen.
So if we took hydrogen and started with atomic hydrogen and added an electron to it, then we would form an anion known as H minus.
And H minus looks pretty good because it's isoelectronic with helium.
So maybe this isn't going to be so bad a day.
But if we're going to have a bond then we need to form an H plus.
So let's do that.
So that would be, then, H goes to H plus, plus an electron.
And that's really nothing more than a proton.
So that doesn't look too appealing.
That's probably a high energy state.
And besides, in the same location at the same time-- in other words, same temperature, same conditions-- half of the hydrogens have to acquire electrons and half of the hydrogens have to lose electrons.
And that's not going to happen.
They're either going to have a propensity for electron gain or a propensity for electron loss.
So it looks like ionic bonding is not going to help us explain the formation of molecules such as H2, N2, and so on.
So who came to the rescue in this case to get us out of the conundrum?
G.N.
Lewis.
G.N.
Lewis was actually born in Weymouth, Massachusetts and he finished his PhD at Harvard in 1899.
And then, like so many Americans of the day, went off to Europe and he postdoc'd in Europe for a while.
And then he came back and got a job at MIT.
And he taught at MIT from 1905 to 1912.
And then in 1912 he was lured to the West Coast where they were starting to establish the chemistry department, the University of California at Berkeley, and he went out to Berkeley and that's where he spent the rest of his career.
And we can speculate why he went.
Maybe he was fed up with the weather here.
Actually, today is one of those few days-- write it down, because one of the few lovely days in Massachusetts.
So G.N.
Lewis, what did he say?
He said, well I've got an idea here.
He said, what if hydrogen achieved shell filling not by electron transfer but by electron sharing.
So he posited the idea of shell filling by electron sharing.
This is in contrast to electron transfer.
So let's see.
Oh.
There's an image of G.N.
Lewis.
He died, actually, on the job.
He came back to his lab one day after lunch and hit the floor.
So he worked right to the very end.
Here's some data taken from a lab notebook and memo, actually.
1902.
And what do you see here?
Well, he developed a notation for us, and we still use this notation to this day: Lewis notation.
So here's lithium and he's got one electron.
But we know lithium has three electrons but only one valence electron.
And then there's beryllium and magnesium-- two electrons.
Aluminum with three.
Here's fluorine chlorine, and when they ionize he puts the eighth electron right here.
And look at this one for silicon.
He's got probably some kernel inside the atom, thus.
So he's even starting to think about concentric shells.
This is 1902.
Remember the Bohr model isn't until 1913.
So you can see people struggling.
And notice that we have eight electrons in a shell-- that's where we're getting the octet stability-- and he's using cubes.
Now we know that the cube isn't the shape of the shell, but it's a pretty good device to help you keep track of electron number-- because there's eight corners on a cube.
So it's another example of how that's not what it is but it's a really good model and it keeps you out of trouble and allows you to go forward.
So this is going back-- way, way back-- for G.N.
Lewis.
Now let's use this idea and account for the formation of H2.
So here's hydrogen, and using the Lewis notation we'll put a dot here for its one electron.
And we'll bring in a second hydrogen and we'll use a cross, or an x, to indicate the electron from the second hydrogen.
And now we're going to double count-- in other words, double attribute.
These are shared electrons so they count for both atoms.
Double count the shared electrons.
And when you do so what do you come up with?
Well, the element on the left has two electrons and, therefore, is isoelectronic with helium.
OK?
Maybe it was a California thing.
They were sharing.
And then there was sort of another California concept, like.
So it was like helium.
And then on this side this is also sharing.
And it's kind of like helium.
So now we've achieved the stability of the filled shell by sharing the electrons.
OK. And I think I even have another slide of how-- this is the more modern version of it.
Electron dot.
So the nucleus and the inner-electrons are contained inside the chemical symbol.
And, actually, this goes all the way to modern quantum mechanics.
Density functional theory does the same thing: lumps all of the inner-shell electrons plus the nucleus into one piece, and then the valence electrons are outside.
And so starting in 1902 with some little dots and crosses we go all the way to DFT today.
All right.
So let's do another one.
How about nitrogen?
Let's try nitrogen.
So when we going to nitrogen we know the valence electron's 2s2 2p3.
So put nitrogen here.
One, two, three, four, five.
Now these three electrons here are according to the Hund rule.
So it's px, py, pz, and this is the 2s2 sitting over here.
And I'll bring in a second nitrogen, and there's its 2s2.
2px, 2py, 2pz.
And now what do I have?
Look at the nitrogen on the left.
Two, four, six, eight.
So the nitrogen on the left feels as though it has access to eight electrons.
The nitrogen on the right-- two, four, six, eight-- it feels as though it has access to eight electrons.
So both nitrogens are isoelectronic with neon if we push on this concept of electron sharing.
Now there's a second thing I want to do.
It's to draw attention to two types of orbitals.
So these three orbitals in the center consist of electrons that are shared.
So these are going to be called bonding orbitals.
And "bonding" and "blue" both begin with a "b", so I'm going to denote the bonding orbitals, or bonding domains, as distinct from the nonbonding domains in red.
Red are nonbonding domains.
Always two electrons per orbital.
They like to live in pairs.
That's the way it works.
OK?
And so each one of these pairs is a bond.
So I can then write nitrogen with three lines through it indicating I have a triple bond.
Three pairs of electrons, three bonding domains, triple bond.
This is all in formation according to the concept of electron sharing.
And Lewis coined a name for the type of bond that is formed in this way.
He said we get bond formation involves cooperative use-- sharing, cooperative-- of valence electrons.
So now we can take the "co" symbol here and the "valence" here and come up with the term "covalent bond."
Covalent bond, thanks to G.N.
Lewis.
So, again, to make sure we're very clear, ionic bond results from electron transfer, covalent bond results from electron sharing.
Now we can do this-- so let's go to heteronuclear molecules.
These are homonuclear.
So let's go to heteronuclear molecules.
And so let's see.
I've got some rules up here, I think.
Yeah.
Drawing Lewis structures.
So let's go to a heteronuclear molecule.
And I'm going to choose as an example sulfuryl chloride.
And I don't expect you to be able to name these things on site.
I will always give you the name.
I'll say sulfuryl chloride, parenthesis, SO2Cl2, blah, blah, blah.
OK?
So sulfuryl chloride.
I want to put up the Lewis structure of sulfuryl chloride.
So center the element with the lowest average valence electron energy.
So it turns out that the average valence electron energy stack like this.
Sulfur is the lowest, then chlorine, and then oxygen.
This is this ranking of average valence electron energies.
And you'd be given those data.
So it says put sulfur in the center.
So I'll put sulfur in the center.
And then what does it say?
We're going to count all the valence electrons.
So sulfur over here is 3s2 3p4.
So that gives me six valence electrons.
And there's two oxygens.
And oxygen lies above sulfur, so that's 2s2 2p4.
So that's 2 times 6.
So that's 12.
All right?
Let's put the 6 over here.
And then there's chlorines in this compound, so that's 3s2 3p5.
So that's 5 plus 2 is 7, 2 times 7 is 14.
And we add this whole thing up, we get there's 32 valence electrons.
And draw a single bond from each surrounding atom to the central atom.
All right.
Again, this is a model.
I'm not saying that this is the shape of the molecule, but it's a way to count.
All I'm doing is trying to keep track of bonds and paired electrons.
So I can put chlorine on either side.
And I'll put an oxygen below and an oxygen above.
All right.
So that's already two, four, six, eight.
So I'm losing eight.
So 32 minus 8 is 24.
And so with the 24 that means I've got 12 pairs of electrons to place.
So let's start putting the Lewis structures up.
So chlorine consists of one, two, three, four, five, six, seven.
And I'll do the same thing on the other side.
Two, four, six, seven.
And oxygen has six, so that's two, four, six.
And then the lower one, same thing.
Two, four, six.
And sulfur has six.
I'm going to use x's for sulfur.
So I'll put one x with the chlorine, another x with the chlorine.
Two with the oxygen, two with the oxygen.
And so now we're in pretty good shape, right?
We can identify bonding and nonbonding domains.
Here's the bonding.
One, two, three, four.
And then the nonbonding.
Looks like there's 12.
And sure enough, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.
The 12 nonbonding domains.
4 bonds.
And we have the Lewis structure for this particular compound.
And one last little piece worth pointing out.
Notice that in the bonds to the chlorine you have two electrons, as you need.
One electron comes from the chlorine and one electron comes from sulfur.
But in the bonds between the sulfur and the oxygen the sulfur's so desperate to form a bond that it actually donates both electrons to the bond.
And oxygen's happy because it's isoelectronic with neon, and sulfur's happy because it's going to be isoelectronic with argon.
But, you know, it has to go to some lengths.
So this is called a dative bond, when both electrons come from the one element.
OK. Well, this is great.
I'm going to do one more.
How about methane?
CH4.
So I'm going to start with carbon.
Carbon's going to go with this on there.
And a carbon is 2s2 2p2.
And so I'm going to use the box notation now.
See, this is the Lewis structure, this is chemical equation, now we're going to a box structure.
We can move fluidly from one model to another.
We had cubes up there.
It's all good.
So this is 2s.
And now this is 2px, 2py, and 2pz.
Now according to this, 2s2, that gives me an electron pair.
And now I've got 2p2, which according to the Hund rule goes in like this.
Well I've got a problem here.
How many unpaired electrons?
Two.
Now what's my maximum number of bonds I can form by electron sharing?
It's two according to this.
So the best I can do, best possible here, is CH2.
And that's no good.
We know from mass measurements it's CH4.
The stoichiometry's CH4.
And besides, what are these orbitals going to look like?
These are the p orbitals, so they're dumbbell-shaped and they're orthogonal, right?
They're 90 degrees, which means if I formed this thing-- which is called methylene-- if I form methylene I'd end up with CH2 looking like this, which has a dipole moment.
And we know from spectral measurements and electrical properties measurements that this thing is symmetric.
So this thing-- electron sharing isn't working.
It's not working.
So we need another patch here, and that patch comes from none other than Linus Pauling.
Another American.
You see, it's American science today and it's in the 20s.
That's why we have Gershwin playing at the beginning.
Celebration of American science.
So Pauling was born in Portland, Oregon.
He was the son of a pharmacist.
And he went to Caltech, got his PhD in 1925.
So the Rhapsody in Blue came out in 1924 when he was just hunkering down to his thesis.
Probably listened to it, got some pleasure out of it, as most people did.
So then he finishes, 1925, at Caltech in Pasadena and he goes to Europe.
Now he chose wisely.
He chose four postdoctoral positions.
These are people he postdoc'd with.
First with Sommerfeld, then with Bohr, then with Schrodinger, and then finally with Bragg.
You'll learn about Bragg.
Bragg got the Nobel Prize for X-ray diffraction.
So that's not a bad preparatory start.
So he comes back and teaches at Caltech.
In fact, I have a picture of Linus Pauling.
There he is.
That's a middle-aged Linus Pauling, probably around the time he got the first of two Nobel Prizes.
So what did Pauling do?
Pauling said, why don't we mix the orbitals?
They're all in the shell n equals 2, and what we're trying to do is to fill the n equals 2 shell.
So how about mix?
Let's mix 2s and 2p states in order to maximize the number of bonds.
Remember, when you form a bond you decrease the energy of the system.
Four bonds is a greater decrease in energy than two bonds.
So the system, if it could-- and they're all within the same shell.
You notice he didn't say, gee, if mixing 2s with 2p is good let's go get some 1s.
Well 1s is down in n equals 1, and there's no way you can mix.
They had to be in the same shell.
Mix states in order to maximize number of bonds that can form.
It's all about maximize the number of bonds that can be formed.
And this, of course, is by electron sharing.
We're talking about covalent bonds here.
OK?
And he termed these mixed orbitals "hybrids."
Termed the mixed orbitals as "hybrid orbitals."
They're cross-breed-- part s, part p. So now let's look at the energy level diagram-- or the box notation.
Forgive me.
So I'm going to mix s and p. So I've got a single s and I've got three p's, so this is called sp3.
Each one of these is a mixed sp3 hybrid orbital.
And I've got four of them.
And how many electrons do I have?
Four.
So now I use the Hund rule and in go the electrons.
One, two, three, four.
And now I have the ability to form four bonds.
But it gets better.
Here's the next thing.
These are degenerate.
They're all in the same state.
That's why we're using the Hund rule.
And so degeneracy in energy implies degeneracy in spatial orientation.
So what does that mean?
It means that if these are four bonds equivalent, then the way those bonds will arrange themselves in space is to be equivalent.
So if I've got a central carbon here and I'm going to put four sticks from the central carbon so as to make the four sticks symmetrically disposed in space, that dictates the architecture of the molecule.
And how do I put four sticks off of a central point symmetrically disposed in space?
One, two, three, and four.
This is meant to be the corners of a tetrahedron.
Each one of these is 109 degrees apart.
And this describes a tetrahedron.
So now I've got carbon in the center, and now I've got the hydrogens at the four corners of a tetrahedron.
There is the structure of methane.
And each of the hydrogens has a shared electron with the carbon, making it isoelectronic with helium.
And the carbon has four of its own electrons, four shared with the four hydrogens to make it isoelectronic with neon.
So everybody's happy.
Shell filling, and it's all good.
So now it's symmetric and it has no net dipole moment.
Everything squares with the data.
Well, good for Pauling.
But he went further.
He went much further.
What Pauling wanted to do was to make it quantitative.
And so he wanted to have something analogous in covalent bonding to what we have in ionic bonding.
So what Leslie is now rubbing off the board there is-- no, keep going.
It's good.
It's OK.
This is a rule of academics: You always erase that which you will refer back to.
We need more boards in here.
How many boards do I fill in a period?
9, 18?
Maybe what?
We need about 24 boards.
That's a good lecture.
All right.
So here's what Pauling was thinking about.
He was thinking about the analogy, for example, if I want to get the energy of magnesium oxide I can use the formula that Leslie has just erased, and it looks like this.
So if all I need to know is the radius of the anion, the radius of the cation, it's charge, and the Madelung constant and then I just plug in, I get the crystallization energy.
But then suppose instead of magnesium oxide I want to go to magnesium chloride.
I can use the same formula only I need the Madelung constant.
This is the Madelung constant for magnesium oxide.
If I have the Madelung constant-- forgive me, script M-- for magnesium chloride, and I know the ionic radius of magnesium cation and chloride anion, away I go again.
I need this.
I need, of course, the Born exponent.
This Born exponent and away we go.
The same formula applies.
So I can build with a library of basic physical data.
So what did Pauling do?
Pauling said, what if we can do the same thing for covalent bonds?
Is there some kind of an analogy?
So he said, let's take a look at an arbitrary heteronuclear compound.
So I'm going to do this with HF, hydrogen fluoride.
So, first of all, let's build a hydrogen fluoride molecule.
H with its one electron, and fluorine with its seven.
So now hydrogen sharing an electron with fluorine is isoelectronic with helium.
It's happy.
And fluorine sharing the electron hydrogen is isoelectronic with neon.
It's happy.
So again we see shell filling by electron sharing.
So what Pauling wanted to ask is, can I get a measure of the HF bond energy knowing only the bond energies of H-H and F-F?
So then if I knew all the homonuclear bond energies and then I mixed these to make heteronuclear bonds, is there a path from homonuclear bond energy to heteronuclear bond energy?
So let's look and see what the numbers are.
So hydrogen.
The hydrogen bond's fairly strong.
It's 435 kilojoules per mole.
That's mole of bonds.
435.
Fluorine-flourine is 160.
And so what do you think the value of the H-F bond should be?
Well when I first look at this I say, well it's part H and it's part F, so it's somewhere between 435 and 160.
I don't know if it's the arithmetic mean-- you know, add these two and divide by two-- or maybe it's the geometric mean-- multiply them together and take the square root-- but it's got to be somewhere in between.
What do the data show?
The number's 569, which is greater than 435.
So I take a bond of 435 and a bond of 160, I put them together I get 569.
That's very, very strange.
But Pauling was smart.
Pauling said, I have an explanation.
He says, suppose when these electrons are shared in between the two atoms, suppose they're not shared equally.
Suppose there is a displacement of the electrons.
So instead of putting them dead center, as I've been doing up until now, suppose the electrons are actually drawn closer to the fluorine.
So we still have octet stability, or in this case duet stability, but the sharing of the electrons is not equal.
So this is charge displacement.
And what does charge displacement constitute?
Well, charge displacement means stored energy.
And Pauling quantified that stored energy.
And so what he did is he said that you increase the bond strength by thinking of it as a two-step reaction.
So in the heteronuclear bond that is a bond between two different atoms.
So in a heteronuclear bond we form by what-- and this is my coinage, you don't see this anywhere in the book-- two-step, share and then pull.
So share is, as the name implies, we share electrons to achieve octet stability.
But then because we have unequal atoms we pull towards one of the atoms.
And which one do we pull towards?
Well, we pull towards the one that's got a greater appetite for electrons.
And we've already gone through this concept.
Which atoms on the periodic table have the highest appetite for electrons?
The nonmetals.
The weakest appetite is the metals.
The metals are good donors, the nonmetals are good acceptors.
And fluorine's up in the top right corner, so fluorine has a very, very high appetite for electrons.
And, indeed, in this bond the electrons are pulled to the right.
And why Pauling got the Nobel Prize and Lewis didn't-- it's my theory-- is that Pauling was quantitative.
So he came up with a quantitative measure.
He devised a quantitative measure for the degree of unequal sharing, thereby allowing us to make these calculations with some accuracy.
And he called that quantity electronegativity, and it's denoted by the Greek symbol chi.
OK?
And he devised a whole scale.
How did he get the scale?
He looked at bond energies for all sorts of pairs of elements across the periodic table and went through an exercise with pencil and paper that today we would call multivariable regression analysis.
And came with a set of-- [SLIDE APPEARING]
Oh, I'll come back to this.
This is the structure of methane.
This is the s and p. Oh, let's take a break.
You can stack.
All right.
So this is what methane looks like.
There's the s, there's the p. And the sp hybrid looks like this.
It's sort of an asymmetric dumbbell.
And these four things stick out.
And then you bond the hydrogens and there's the methane.
OK.
So here's what the electronegativity scale looks like.
It looks a lot like the scale for average valence electron energies.
The nonmetals have the highest appetite for electrons period, which means in a bond they're going to hog the electrons.
And the nonmetals have the weakest appetite, and so they're going to end up having the electrons in a covalent bond pulled away from them.
So nonmetals have high electronegativity, metals have low electronegativity.
And now here's taken from the text.
And you see that the electronegativity is periodic.
If you go across a period the metal has the lowest value and the nonmetal has the highest.
And there's fluorine, number nine, at a value of about 4.
It's got the most intense appetite for electrons.
And then you jump down here to sodium, et cetera, et cetera.
Here we are going across the lanthanides and whatnot.
And this is taken from your text.
There's fluorine, 3.984.
That's the thing.
And down here we have very low values of electronegativity.
So with electronegativity we are now able to make calculations.
And this is the Pauling formula for calculating the bond energy in a heteronuclear bond starting from homonuclear bond energy.
So let's continue with the HF.
So if I want to get the bond energy of HF I'm going to take-- and this is the Pauling formula-- the geometric mean.
So I take the bond energy of hydrogen-hydrogen times the bond energy of fluorine-fluorine, and square root.
So that's the geometric mean of the two.
And then comes the Pauling piece that gets him the Nobel Prize.
You take the difference in the electronegativity between the two elements squared, and then the factor 96.3 gives us the unit consistency with kilojoules per mole.
So the greater the difference in electronegativity the greater the contribution here in terms of the deviation from just the geometric mean of the two homonuclear bond energies.
Or put another way, if you have a homonuclear atom such as H2, if it's chi H minus chi H is 0, so this second term goes to 0.
And obviously when fluorine is one of the members you're going to get a very, very high number, because this has the most-- and it doesn't matter which order you put them in because you're taking the square, so it's always going to come out positive.
And I want you to appreciate the sense of scale here, so if we go in here we'll multiply.
This is going to be 435 times 160.
And I'm going to take the square root of this.
And then this is 96.3.
And you look on your periodic table this is 2.2 for hydrogen.
Fluorine is 3.98.
And I know there are different tables of electronegativity.
I don't care.
Just whatever you've got on your periodic table.
The one in the book is a little bit different but it all comes out in the wash.
So you multiply all this out, and we find that the first term is 264 kilojoules per mole and the second term is 344.
So this second term is even greater than the first term.
So the amount of energy in that electron displacement is substantial.
And if you sum the two of these you get 608.
Now you might say, well wait a minute.
The real number is 569.
But 608 takes you in the right direction and accounts for the contribution of electron displacement.
264 is just plain wrong.
So this was an important start for Pauling.
And he has labels on these two contributions.
This first term, which is just the combination of the homonuclear bond energies, is called purely covalent.
It's the purely covalent contribution.
And it's what I've been referring to as the sharing.
This is what you get from sharing.
OK?
And then this second term here with the difference in electronegativity is what you get from what I've been calling the pull on the electron pair.
And Pauling called this the partial ionic character.
He's not saying that there's electron transfer, but it's a move in that direction.
Partial electronic character.
So what I've done here is I've decided I'll make a sort of a panorama of what we've seen up until now.
And so I'm going to make something called the electron sharing meter.
All right.
So if I look at a homonuclear system like hydrogen.
So my meter reads neutral.
So the arrow's at 12 o'clock.
The electrons are shared equally.
And then if I go to HF what do I have?
Well, I know that the fluorine is pulling the electrons.
And so we can designate that by writing delta minus, delta plus.
delta the physicists use.
The lowercase Greek delta means little bit of.
All right?
So delta minus means it's a little bit negative.
And we've got charge neutrality, so if the fluorine end is a little bit negative then the hydrogen end has to be a little bit positive, which means this thing has a net dipole moment.
It's a dipole.
And the arrow points to the negative end.
One way to think about it is I put a little slash there and that starts to look a little bit like a plus sign.
You can come up with your own way to remember it.
So it's got a little bit of a dipole moment.
And people depict dipoles usually as ovals, and they'll put a minus end and a plus end.
So it's net neutral but the charge is not uniformly distributed.
OK?
So our sharing meter in this case is going to show something to the right.
We've got electrons that are unequally shared, and that moves over to the right.
And, you know, the dipoles have interesting properties.
Oh, there's a plot of electronegativity 3-bar in the bar plot.
And actually this is an interesting one.
Just parenthetically, you see hydrogen here?
Hydrogen's weird.
They put it in the periodic table above lithium but it's not an alkaline metal.
And you can see it just doesn't belong there.
And there's a lot of conversation about putting it maybe somewhere centered above the p block elements, because it certainly doesn't belong next to helium.
But it probably doesn't belong above lithium either.
Anyway, I thought that was very interesting.
I can tell from the response of the class, why does he care?
[LAUGHTER]
All right.
Now this is really-- I'm going to use an adverb here-- this is really important.
All right?
So here's HCl, is a cousin of HF, and you see in the upper frame it's just a bunch of HCl molecules just bopping around any which way.
So there's the delta plus and the delta minus.
Now if you take these dipoles and you put them in an electric field they will align themselves, and the positive ends will face the negative plate and the negative ends will face the positive plate.
And there's energy stored when the random orientation goes into an ordered orientation.
This is the principle behind a capacitor.
A capacitor is nothing more than a whole bunch of aligned dipoles.
So if you want to invent a supercapacitor that we can use on a car to extend the range of the automobile so we can reduce our dependence on imported petroleum, you're going to look for molecules that have a honking big dipole moment.
That way you get more energy per unit electric field.
So, again, a simple idea that tells me how to go and invent.
I can go back to my office and go and invent something right now just based on this lecture 9.
[LAUGHTER]
See, you go and invent.
You start the company, you make the billion.
Remember good old Professor Sadoway at MIT, and established the fellowship for students, and so.
All right.
But you have to know what a dipole moment is.
Got to know what a dipole moment is.
OK.
So there's the dipole moment.
And then lastly I'm going to put sodium chloride.
So what's sodium chloride look like?
Well it's Na plus and Cl minus.
So the electron has transferred completely.
So this isn't even sharing at all.
So this is really bury the needle.
This is not sharing.
In this instance the sodium doesn't even get visitation rights to the electron.
The electron's gone.
Whereas here hydrogen gets to see the electron on Saturdays kind of thing.
Depends what kind of lawyer fluorine had.
That's what it all boils down to.
All right.
This is the same thing that I just showed you.
But you see, the textbook gives you, as the name implies, dense text.
I gave you the sharing meter.
The sharing meter is far more expositive.
All right.
And then, finally, the percent ionic character is given by this formula here.
So this is 1 minus the exponential.
So the exp term, this exponential of-- what is it-- minus 1/4 times the difference in electronegativities squared.
This notation means e base natural logarithms, minus 1/4, blah, blah, blah.
That's what this thing is.
So if you plug in, multiply by 100% you get something that goes from 0 to 100.
So obviously when delta chi is 0 you get 0%.
e to the 0 is 1, 1 minus 1 is 0, and so you have no ionic character.
And so if you plug in the numbers for HF-- so you're going to take this difference here, square it-- it ends up giving you 1.8, which gives you a value of about 56% ionic character.
So it's as though the electron is sort of half transferred.
But you might also look at it from this perspective.
If you take 344-- because this is the partial ionic character, which is the energy of electron displacement over the total energy in the calculation-- that turns out to be 57%.
So this stuff makes sense.
There's a sensible metric here at work.
And so this is what Linus Pauling got his Nobel Prize for, and it's the description of polar covalency.
And polar covalency is operative when you have heteronuclear bonds, because the two different elements don't share the electron equally.
And the Pauling formula allows you to calculate that.
And his formative book was written in 1937, called The Nature of the Chemical Bond.
OK.
So turning to the last five minutes, I want to bring to your attention some covalent molecules.
Today we're going to talk about Freon.
Freon was an invention, it was a designer chemical, invented by Thomas Midgley.
This is me.
I named him "sp3."
That's his nickname.
Thomas sp3, for the hybridized orbital.
So he was working at the Dayton engineering laboratories in Dayton, which was owned by General Motors, and he was working in the 20s at a time when there were no refrigerators in American kitchens.
The only refrigerants that were used were either toxic or flammable, things like ammonia, methyl chloride, sulfur dioxide.
And you read about horrible accidents.
People making ice cream at some plant and the compressor blows up and two or three people are killed.
So it was deemed unsafe in the American kitchen.
In the 20s Midgley discovered this molecule, which looks just like methane only we've replace the hydrogens with two chlorines and two fluorines.
So this is called dichlorodifluoromethane and it's a chlorofluorocarbon, a CFC.
And this was fantastic stuff.
It it was colorless, odorless, tasteless, non-toxic.
It was not just used as a refrigerant, it was used in propellant.
When I was your age all of the sprays-- whether it was hair spray, shaving cream, any aerosol-- was propelled by Freon-12.
It was fantastic stuff.
Well, it turns out that in the upper atmosphere-- you know, you go pss pss pss.
You got people all over the world doing this, eventually this stuff starts floating away.
And what turns out in the upper atmosphere where we don't have shielding from ultraviolet-- you know how to do this calculation, because you can look up the energy.
And, in fact, it's part of your homework, where you look at the energy differences and the electronegativity differences, you can compute the wavelength of light that's capable of breaking the carbon-chlorine bond.
And it turns out to be in the ultraviolet.
Once the chlorine is broken you have a chlorine radical, and that chlorine radical goes over here and attacks ozone.
[CELL PHONE RINGING]
Cell phone-- out.
Just get up and leave out of courtesy.
[CELL PHONE STILL RINGING] [LAUGHTER]
Hello?
The first year I was teaching 3.091 there was a Nobel Prize awarded to Mario Molina, who was a faculty member here in Earth and Planetary Sciences who had worked years earlier at University of California, Irvine, and had speculated on the mechanism by which ozone depletion occurs and linked it to rising levels of CFCs.
Initially-- that's why it says a vindication-- people pooh poohed it, said it was crazy.
There wasn't enough of this pss pss to cause any trouble.
But then later with the NASA program they started taking a lot of images and they could track ozone levels in the atmosphere and start seeing that not only was ozone changing but there were actually pockets where ozone was being depleted at an accelerating rate-- because obviously the atmosphere isn't constant composition and constant temperature.
Duh.
So anyways, yeah.
There he is.
And this was the paper that was published in 1974 in Nature.
And this was done before computers.
The PC wasn't invented and commercialized until the early 80s.
So this was typeset, and the person who typeset it obviously didn't take 3.091 because instead of "atom" hyphen "catalysed" we have "atomc-atalysed."
But even ignoring the spelling error in a Nobel Prize winning paper-- [LAUGHTER]
--the Nobel committee overlooked this.
Yeah.
So there it is.
And then they went to HFCs and so on.
There's a lot of activity in this.
And what happened is when we changed from CFCs to HFCs we had to change the design of the compressors.
And what happened was everything got much, much more efficient.
So this was an example of necessity for a change that was driven by concern for the environment.
Instead of putting people out of work and killing an industry, gave us much more efficient refrigeration.
And the last thing I'll show you is this to draw your attention.
This was in your textbook.
This is the cap at the top of the Washington Monument.
The Washington Monument was built to celebrate the American centennial, 1876.
They finished it in 1884.
And this is 100 ounces of aluminum, because aluminum was a precious metal.
It was priced higher than silver.
1884.
Two years later Charles Martin Hall and Paul Heroult invent an electrochemical process that drives the price of aluminum down to the point that we make beer cans-- I mean soda cans-- out of it today.
[LAUGHTER]
And a good example of how chemical innovation can lead to superior products.
I'll see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Settle down.
Settle down.
Settle down.
All right, first announcement is the celebration of learning.
Reminding you that the celebration of learning is a week from today.
Go to your assigned rooms.
A through Ha will write in here.
This group will go into 26-100.
And the last group over to 4-270.
And you will take with you your periodic table, table of constants, aid sheet, something to write with.
And we'll give you paper.
You'll write on the exam paper itself.
And I'll say more about test taking strategies on Monday.
Coverage will be right up through Monday.
With emphasis obviously on the material that you've had some time to digest.
And there will be no weekly quiz next week.
So let's get right into the lesson.
Last day we looked at Louis, who gave us the notion of achieving octet stability by electron sharing.
And that led to the concept of covalent bonds.
And then Pauling helped us understand the energetics of covalent bonding by putting forth the idea in a heteronuclear molecule there's unequal sharing of the electrons.
And out of that emerged the concept of polar covalency and the definition of electronegativity.
And on the slide you see how electronegativity varies across the periodic table.
Electronegativity being a measure of the pull an atom has for electrons within a covalent bond.
And as you would expect, the non metals which are good electronic receptors are also the ones that have the highest electronegativity.
And the metals, over here, which are good electron donors are those that have the the lowest value of electronegativity.
And Pauling was quantitative in his formulation and gave us this equation that tells us how to measure the energy of the x-y covalent bond.
If x is not equal to y, then axiomatically this is going to be a polar covalent bond.
And you take the geometric mean of the homonuclear bond energies.
This is x-x bond energy in x two.
This is the y-y bond energy in y two.
You take the product square root of which.
And then the partial ionic character, this is the contribution to unequal sharing of the electrons.
You take the square of the difference in the electonegativities of the two elements.
And the 96.3 is a factor that allows you to get the overall quantity in kilojoules per mole.
So you need these numbers in kilojoules per mole.
And we further had a formula for the percent ionic character, which you can get by looking at the difference in electronegativity.
And then through this formula, you end up with a scale that runs from 0 to 100%.
And just by way of example, we had a look at H-F. We spent a fair bit of time on that.
Obviously, it's a heteronuclear molecule.
We calculated its bond energy and so on.
And then to indicate polar covalency, we use the dipole notation.
The dipole shown here.
It's just an oval.
It's net neutral.
But the charge is not uniformly distributed.
You see one end is a little more negative.
The other end is a little more positive.
Sometimes people write lowercase Greek delta, indicating little bit negative here, a little bit positive here, net neutral.
There's the arrow indicating the dipole.
And furthermore, we argue that this is a polar bond.
And since this is just a molecule with the two atoms, then this is also a polar molecule.
Which means it has a net dipole moment.
And I made some observations about dipole moments and the ability to store energy capacitively.
And how you'd go about finding a really good capacitor.
And so that all came out of the desire to find something that has a net dipole moment.
And I want to continue that conversation.
And so I want to look at some other elements.
And what I want to look at in particular is the compound methane.
Let's look at methane from this new found appreciation of polar covalency.
So first thing I want to do is to put it's structure up.
We've gone through the Louis notation and the structure.
It forms sp3 hybrids.
And you end up with a structure that looks like this.
These are all at a 109 degrees, symmetrically disposed in space.
And we know since we have a heteronuclear system here, we're going to have some polar covalency.
You can look up that the electronegativity of carbon is 2.55 electronegativity of hydrogen is less than that from its position in the periodic table.
But to be quantitative it's 2.2.
So that means if I look at the carbon hydrogen bond, carbon has the higher electronegativity.
So it's going to pull the electrons.
So that means that the carbon end is going to be a little bit negative.
And the hydrogen end is going to be a little bit positive.
So that means I've got a dipole moment here.
Polar bond.
Everything is the same as above here.
Polar bond.
But, I want to address the question, is the molecule polar?
So the way I interpret the question, is it a polar molecule, I ask is there a net charge displacement?
Well, we're off to a good start here.
We see that we have charge displacement on the bonds.
But, is there a net dipole for the molecule.
So the way I think about that is to say, where is the center of positive charge for the molecule?
Where's the center of negative charge?
So I know all the hydrogens are a little bit positive.
And they're all the same distance from the nucleus.
And I can draw a circle that will capture all four hydrogens.
Or more appropriately, it's a sphere, right?
These three on the bottom don't lie in the plane.
So the corners of the tetrahedron lie on a sphere.
So where is the center of positive charge?
The center positive charge is right here at the center of the molecule.
And where's the center of local negative charge?
It's on the carbon because the carbon is the negative end of all the bonds.
So the centers of positive and negative charge for the entire molecule are colocated at the center of the molecule.
So this means no net dipole moments.
No net dipole moment for the molecule.
So this is a non polar molecule.
It's a non polar molecule consisting of polar bonds.
Non polar molecule.
So there's two ways that I can get a non polar molecule.
One way is to have a homonuclear molecule, right?
So, homonuclear molecule.
Homonuclear molecules axiomatically must be non polar.
Because they have equal sharing.
So if I give you anything that's homonuclear trivially, it's non polar.
So you can look at things like H2, P4, S8.
These things are all non polar.
I don't care what their structures are, it doesn't matter.
This one here is definitely polar.
And then the last one here that we're looking at, the the methane, is non polar because we have spatially symmetric disposition of identical polar bonds.
So spatially symmetric, that means there's three dimensional symmetry.
Spatially symmetric disposition of identical polar bonds leads to non polar molecule.
Because the centers of positive and negative charge are colocated.
So, now it's time to move on.
Now I want to look at covalent bonding from an energetic standpoint.
I want to go back to energy level diagrams.
We've looked at energy level diagrams in the past for atoms, single atoms.
But now, what I want to do is build energy level diagrams for molecules.
So, for that we're going to go back to the Schrodinger equation.
I want energy level diagrams for molecules.
And for that I'm going to call upon the Schrodinger equation.
And we're not going to go through the quantum mechanics in mathematical detail.
We're going to do quantum mechanics pictorally.
We've got a long way.
And in particular, what I want to do with regard to the Schrodinger equation is to recognize that we can move from atomic orbitals to molecular orbitals by exploiting the fact that the Schrodinger equation is a linear equation.
Using the linearity property of Schrodinger equation.
All right, what do I mean by linearity?
Well, I'm going to do just a little bit of math.
Just enough to make you sit up in your math classes and find out that there's some utility there.
Here's what I mean.
Let me talk about what the linearity principal is.
So if I have some equation, f of x, y, and z.
It's a three variable equation.
And the equation is f of x, y, z is k1.
k1 could be a constant, it could be a function, it could be anything.
And let's say that it has as its solution, the solution is s1.
And then I've got a second variant of this x, y, z.
And it equals k2.
So it's the same function but it equals k2 here.
It could be a constant, it could be something else.
And it has as its solution, s2.
If this f of x, y, z is a linear equation, then if I'd give you f of x, y, z equals k1 plus k2, you don't have to go and solve the equation with impunity.
You can write that the solution is equal to s1 plus s2.
And that doesn't hold if it's a non linear equation.
Just put y equals x squared.
And if I tell you y equals one, y equals two, and then I give you y equals 1 plus 2, it doesn't equal the sum of the solution to that.
You can prove it to yourself.
So this is the fact that superposition holds.
You can superpose solutions and build a library.
So superposition holds when you have a linear equation.
And that's all we're going to use in order to make equations for the molecular orbital.
So that way I can write that the wave function of a molecular orbital then is a linear combination of the atomic orbitals.
That's all this is doing.
This is setting the stage for, if you take quantum mechanics later, you'll go through this in gory detail.
But I'm saying that you know enough now to appreciate that we can do what we're about to do so.
That means I'm just going to sum the wave functions of the atomic orbitals.
In this case, i goes from 1 to 2 because there's only two orbitals in a bond.
And there will be some pre factor here, a sub i times ci.
And this whole business is called linear combination of atomic orbitals into molecular orbitals.
So it's an SLI, it's a six letter initialization.
You know like FBI is a TLI, it's a three letter initialization.
This is an SLI, in 3L91 we go big.
This is an SLI, six letter initialization.
So we're going to do some examples here.
And all you need to do in order to run the examples is use two ideas in LCAO-MO.
First of all, conservation of states.
Conservation of orbital states.
And the second one is, we're going to fill the newly created molecular orbitals according to the Aufbau principle.
Fill MOs by Aufbau.
If we do that, we're in good shape.
So let's take a look.
So first think I want to do is just rationalize this one here.
H plus H goes to H2.
I want to demonstrate that there's a rational basis for this.
So what I'm going to do is make energy level diagrams.
So there's an energy coordinate here.
Energy goes up in the vertical direction.
And out here is zero.
And what I'm going to draw for you is energy level diagrams for two atomic hydrogen gas atoms.
So if this is the zero, we know that somewhere down here is the 1s.
And I'm going to be complete in my label.
This is the 1s atomic orbital of hydrogen.
And over here there's a 1s atomic orbital of atomic hydrogen.
And furthermore, I've got an electron sitting in 1s.
Now these are at infinite separation.
These are far apart.
Far, you know what that means.
Far with quotation marks means that they're separate quantum systems.
So I'm not violating the poly exclusion principal by having both of these electrons sitting in the ground state.
So they both have the same set of quantum numbers.
1, 0, 0, 1/2.
Both of them.
Same thing.
Let's put it over here, 1, 0, 0, 1/2.
So they're very, very far apart.
Now if I bring them close enough together to make the molecule H2, what happens is I'm going to violate the poly exclusion principal if I have both of these atomic orbitals at the same level.
Because I start filling them according to Aufbau principal I'm going to end up with more electrons than two at the same state.
So what happens is this splits.
One orbitals ends up at a lower energy and one orbital ends up at a higher energy than the ground state energy in the atom itself.
And so now this is called the sigma 1s molecular orbital.
And the one above it is called sigma star 1s molecular orbital.
And sigma is at a lower energy.
So if electrons populate this orbital, the system's energy will decrease and a bond will form.
So this is called a bonding orbital.
If electrons populate the upper orbital, they will raise the energy of the system, destabilizing it.
And so this orbital denoted with the star is called antibonding.
It's an antibonding orbital.
So now I've got the energy level diagram for molecular hydrogen.
So now let's go and populate according to the Aufbau principal.
I've got two electrons and they go in spin up spin down.
And now you can see that this occupancy put the electrons at lower energy than they were by being in the energy state of the atoms.
And so I can argue that by this diagram, I haven't predicted anything, but I can use this diagram to rationalize that for this reaction delta E is negative.
And we know how negative it is.
It's minus 435 kilojoules per mole.
It's hugely negative.
Minus 435 kilojoules per mole.
All right.
So, that's good.
In fact I think I've got some pictures.
You can go through the whole quantum mechanics.
And just as is the case for a single atoms, you can use the product of the wave function and it's complex conjugate and so on and make plots.
Oh by the way, this was just making the point that you can pull out the electronegativity off of the periodic table.
It's given here.
And the periodic table is pretty good.
But obviously somebody got a little bit ahead of himself.
They called this the first ionization potential, which is the potential you'd put across the plates in a gas discharge tube.
But the unit is not the electronvolt.
If it's a potential, it's a volt.
Somebody here that put this together seemed to think the electronvolt is a unit of potential.
And I want to make sure that nobody in this class believes that.
You got to get a little bit of nail polish or something and cover up the little e there.
Or take the nail polish, paint over potential and write first ionization energy.
One or the other, but not this.
This was a plot if percent ionic characters.
You can see that the strongly covalent compounds down here have very very low electronegativity differences and therefore very low ionic character.
And way up here are the ionics.
And in between is HF.
It's almost at the cusp.
But, can you see that there must be a mistake on this diagram?
Because this thing has got a greater electronegativity difference than lithium iodide and yet lithium iodide has a higher percent ionic character.
How can this function zigzag like that?
Something's wrong.
So when you look at something, you go wait a minute, that doesn't make sense.
So I go back and I say, do I trust anything here?
Trust, but verify.
Read critically.
It's a good book.
But, it's a big book and there's going to be some mistakes.
All right, so here's some pictoral stuff of what we were just doing over here.
So here are the two spherical 1s orbitals and now they come closer and closer together and they overlap.
And this is what the shape of the sigma 1s orbital looks like.
It's like an oval with the two nuclei inside.
Here's taken from a different text book.
The overlap of atomic 1s atomic and now here's the molecular orbital.
Now you can also look at what the shape of the antibonding orbital would be.
This is what the shape of the antibonding orbital would be.
It would have two lobes with a nodal plane in between.
I think this is taken from yet another book.
Oh this is our book.
There we go.
1s, 1s, as there we go.
Hydrogen molecular orbitals.
Now look, suppose we do the same thing for helium.
If we do the same thing for helium, helium starts with two electrons in 1s.
And now it's going to have four electrons.
Two will go in the bonding and two will go in the antibonding.
And the two that go in the antibonding raise the energy of the system more than the two that go into bonding decrease the energy of the system.
There's a net increase in energy.
And this is the way you could rationalize that helium exists as the atom in the gas phase.
You don't see He2 gas molecules.
So using this energy level diagram you can go through an rationalize.
I would never ask you to predict.
I would say, fact, helium is found as the atomic species in the gas phase.
With the use of energy level diagrams, rationalize.
And that's what I would expect you to do.
Take this and show that the two are of different stability.
One other point to to make, the level of 1s.
If I wanted to put helium on this board here.
Not the scale, but just roughly.
Where would helium 1s be relative the hydrogen 1s.
We got three choices.
Same level, closer to zero energy, or more negative.
How do you think about the problem?
What determines what this energy is?
It's the electrostatic force of attraction.
Is the electrostatic force of attraction on the 1s electron in helium greater than or less than it is in hydrogen?
It's greater.
So that means that the energy is going to be more negative.
And so if I were to put over here helium, I'd put down here this would be 1s atomic orbital for helium and then we go through the analysis.
And why does that come into play?
Well you might want to do a heteronuclear molecule.
Suppose you wanted to do the bonding diagram for hydrogen fluoride.
So we'd have hydrogen here and the fluorine and would be here.
And all the fluorine orbitals would be much lower and then they'd combine to make the molecular orbitals.
And I think there's something opportunities to practice that in the homework.
OK let's do one more.
Let's do one more.
How about lithium.
Let's do lithium.
I want to ask, is dilithium stable?
Li2.
And this is all gas phase.
Is dilithium stable?
So I'll start off with here's the zeroes.
The zero of energy for infinite separation.
So this'll be a lithium gas atom.
And this is the putative dilithium gas atom.
We're going to figure out if it's stable or not.
So it's going to have bonding and antibonding orbitals.
And then this is the 1s, 2s.
And this will be sigma 1s, sigma star 1s, and so on.
And then we'll have 2s atomic orbital splitting into sigma and sigma star of 2s.
And now lithium is 1s2, 2s1.
So there's one, two, three.
And one, two, three.
And so now let's use the Hund rule.
So I've got four electrons in the n equals one shell, and they populate in this manner.
And then I've got two electrons that go only into the bonding orbital.
And according to this it appears that lithium 2 is favored over atomic lithium.
And that in fact is the case.
That in fact is the case and so that applies to all of the all of the alkalide metals.
Because they all have the ns1 configuration.
So when you're driving down the highway and you see those orangey yellow low pressure sodium vapor lamps.
What you're looking at is emission not from atomic sodium but from Na2 vapor.
And you've got the energy level diagram to convince yourself of that.
So next time you see those, just smile, knowing that you're looking at disodium, not sodium.
OK, well so far we've only looked at single bonds.
Now I want to look at multiple bonds.
Double and triple bonds.
So let's look at nitrogen now.
N2.
Remember that gave us the the triple bond?
We had nitrogen, one, two, three, four, five.
Second nitrogen, one, two, three, four, five.
So in order to get octet stability, we had three pairs of electronic sharing.
Which then give us a triple bond here.
And so what's that going to look like?
what's that going to look like pictorally?
And the way to think about that is first of all these are all p orbitals.
If we look at, this is 2s2, 2p3.
So 2s and these are the 2d orbitals.
So I've got s is filled first and then I've got the lone electrons in each of the p orbitals.
And these things are shaped like figure eights.
And the p orbital, this is the p atomic orbital.
It has two lobes.
Each of these zones are just called lobes.
Same word is earlobe.
And this zone in the middle, this one point in the middle, is called a node.
And that's a sight of zero electron density.
Zero electronic density.
And remember the electron, even if it only has one electron here, the electron can be in either a lobe.
It can move from lobe to lobe even though it can never be at the nucleus.
And how does it get from one lobe to the other while never crossing the nucleus because it's never supposed to be in the nucleus?
By behaving as a wave.
The same way that you can have a jump rope and you can have a fixed point of zero motion, yet you can transmit energy down the rope passed the node.
So, let's draw these things.
A word about how you draw, you have to use the right hand rule.
Have to use the right hand rule.
So when I put the coordinate system up, it's going to have to conform so that the thumb is x and they go y and z.
And if you don't use the right hand rule later on if you get into electromagnetics, you start looking at forces and vectors, you're going to end up with things moving in the wrong direction.
So we conform to the right hand rule.
And the other thing, and it's kind of nice, it's not mandatory.
But chemists generally like to have atoms bond along the z-axis.
So that means we start with the z-axis here, if I'm going to put my second nitrogen, have them bond on the z-axis.
And so that means if z is in the plane of the board pointing to the right, then pointing up must be y, and then pointing into the board must be x.
So I'll have, here's my px orbital, then the py orbital, and the pz orbital.
And the three of these are all symmetrically disposed around the nucleus.
And next to it at the same kind of floral arrangement.
I'll start with px, py, and pz.
And now these are going to come close together and overlap.
So I want to figure out what those orbitals are going to look like.
So let's start along the z-axis.
The z-axis is the easy one.
So I've got two of these things lying on their sides like infinity signs.
We're going to do quantum mechanics pictorally.
Why?
Because it's a linear equation.
So I can add pictural.
So this is 2pz of one of them.
And a 2pz of the other.
And this is the nitrogen atomic orbital.
And I'm going to smear these things and what are they going to look like?
The nucleus is here, the nucleus is where I'm indicating the dot.
These two combine, very simply, it's going to look like this.
So there's one nitrogen.
Here's the other nitrogen.
And what's this thing?
This is electronic density.
And so this is our sigma bond.
Looks a little bit different from the case of of hydrogen, because hydrogen was the blending of two s orbitals and s orbitals are spherically symmetric.
In this case, we've got two lobes.
But what's characteristic about this one and hydrogen, hydrogen looked like this, remember?
This was hydrogen.
This was H2.
And this was a sigma bond.
The characteristic of a sigma bond is that when you start from one nucleus and you go to the other nucleus, you move through unbroken electron density.
So there are no nodes, no holidays, between the nitrogen nucleus on the left and the nitrogen nucleus on the right.
There is a node here, but that's different.
I can go from one nitrogen to the other with unbroken electron density.
That's what makes this a sigma bond.
So that's good.
So this thing here is going to be called stigma 2p molecular orbital.
Sigma 2p molecular orbital And I think I've got the slide that shows this.
There's some art work.
People really get excited about this.
OK there's dilithium.
Or dipotasium, disodium.
OK so here we are.
This is the smearing of two pz atomic orbitals to make the sigma 2p bonding orbital.
And there just for grins and chuckles is what the antibonding would look like.
But we don't care, because it doesn't form.
Well this is the book.
And you know what I'm going to say.
I've got my little hobby horse here.
I don't know why they change color on the lobes.
Because when I look at that, it starts conjuring up to me the image that one electron stays in the blue lobe and one electron stays in the yellow lobe.
Besides, I've seen them and they're not different colors.
They're the same color.
So now let's look at what happens when we blend off of the z-axis.
So let's blend the two py orbitals.
See what that goes like.
OK so let's do that one.
And that one is going to look like this.
We're to start with, again figure eights.
But now they're their side by side, they're lateral.
So this is 2py atomic orbital.
2py atomic orbital.
And then we're going to blend them along the z-axis to give us, and I'm going to do this stylized, OK?
So there's the the two nitrogen nuclei.
So I put the nuclei up.
And I'm going to smear the upper lobes.
So these two lobes are going to smear.
And I'm going to get really stylized.
I feel like it's France and it's the late 1800s.
So there it is.
And I'm going to smear the two bottom ones.
And it's going to look like this.
So what do I have here?
Now I have two lobes as before.
But if I look at the second nucleus from the first nucleus, not only do I fail to have unbroken electron density, I have zero electron density.
See, this was a nodal point in the atom.
With the two atoms together, the plane orthogonal to the board is a nodal plane.
There's no electron density in the plane orthogonal to the board.
Nodal plane.
So this is definitely not a sigma bond, this is a pi bond.
This is a pi bond.
And it's characterized by smearing of atomic orbitals, just as the sigma bond is.
But it has a nodal plane that separates the two lobes.
I think I've got some cartoon illustrations from other books Here OK this is from one book.
This is good.
They call the px, I call it py.
There it is.
And I'd go further and I'd say that if I were to slice this and look at it from angle, if we were to cut this and look from here, I'd venture that you'd see something that's sort of figure eightish.
Oh here's our book, bless them with two colors.
But anyway, there's what it looks like.
That's the pie.
And then the same thing happens with the x.
So this is this is going to be pi 2py.
This is pi 2py.
And it's a molecular orbital.
And there's going to be a pi 2px.
And it's going to blend front and back.
So we can make a catalog.
So we're going to do quantum mechanics in pictures here.
So I know that s plus s must always make a sigma bond.
There's no other way.
Because I've got electron density all around.
Let's do it pictorally.
That's easy.
And this is sigma.
We know this has to be sigma.
What about something like HF where the H is an s and the F is going to have the one last electron missing in the p orbital.
S plus p must give sigma always.
Because that's this cartoon.
See there's no way that when this smears with this, there's going to be zero electron holidays from the hydrogen nucleus to the fluorine nucleus.
So you're going to end up with something that looks like this.
It starts around and gives you something that's going to be a little bit asymmetric.
So this will also be a sigma.
So I'll just put HF here as sort of prototypical of that.
And then if I take p plus p axially, on axis, that also gives us a sigma.
Because that's this one, the infinity signs.
Two infinity signs give us this.
And then finally, if we get p plus p longitudinally.
So that will give us a pi bond and that's the 88.
8 plus 8 gives me, and this is pi.
So that's quantum mechanics.
The math will follow.
So now what I want to do is go back to this energy level diagram and show how these energy level diagrams can work.
So here's the energy level diagram for nitrogen, N2.
There's two f's, two p, and the scaffolding is in place.
There's the energy levels.
Now here's the molecular orbitals and here are the atomic orbitals.
And now here they are occupied.
So nitrogen has one, two, three, four, five according to the Hund rule.
And here is the set up for the N2 molecules.
So the 2s's and the 2s's go bonding antibonding.
Now I've got three plus three is six.
Two, two, two.
Everything's paired.
So we get the triple bond, 946 kilojoules per mole.
Enormous energy in nitrogen.
Enormous energy in nitrogen.
Now, let's keep going.
Now let's look at oxygen.
This is the scaffolding for oxygen and fluorine.
And there's a little change here.
A little change, I'm going to draw your attention to it.
And you can't predict this.
We would give you this.
I would tell you what the energy sequence is of the energy levels.
But look here carefully, you see in the case all nitrogen, the pi's lie below the sigma.
In the case of oxygen and fluorine the sigma lies below the pi's.
These are tiny, tiny differences.
But, they are measurable.
That's the little difference.
Now let's see what happens.
Actually here's from out text book and it shows the stigma 2pix slowly meandering down, down, down, down, down.
And somewhere between nitrogen and oxygen it crisscrosses.
All right, so now let's fill oxygen.
So oxygen is two, four, five, six.
So two plus two is four.
There's the two, two, two.
And then these last ones go up into the antibonding.
But look at the antibonding.
We have, according to the Hund rule, not two electrons in the first orbital, but one and one.
So we end up with unpaired electrons in the antibonding orbital.
So these offset the three pairs here, and so we end up with a double bond.
And its energy us 498 kilojoules per mole.
Substantially less then nitrogen.
And this energy level diagram rationalizes that.
And then here's for fluorine.
If you go to fluorine it's the same scaffolding only there's two more electrons.
These are paired and you have a single bond.
160 kilojoules per mole.
Now there's a property that we get from the fact that we have these unpaired electrons.
We're going back oxygen now.
We have unpaired electrons in the antibonding orbitals.
We saw unpaired electrons.
What did they do in the Stern-Gerlach experiment?
Changed the magnetics substantially, right?
We ended up with the splitting of the silver bean.
So this will also have an impact.
An impact known as a paramagnetism.
What's paramagnetism?
It's shown in this little cartoon.
If you have a substance here that's balanced and it is paramagnetic, if you engage the magnetic field, the magnetic field will pull on a substance that's paramagnetic.
And oxygen is paramagnetic.
Not only in a gas state, but paramagnetic as liquid.
And here's an illustration from your text book.
You may have seen this and said, yeah a guy is pouring liquid oxygen, wow.
Look carefully now.
The boiling point of oxygen at one atmosphere pressure is 90 Kelvin.
So room temperature is substantially higher.
This is the equivalent of taking water and putting it in an oven at about 300 degrees celsius and watching it pour.
It would still be liquid, but it would be liquid trying to boil.
Why doesn't it all turned to gas?
Because there's a time to heat everything.
So this is sitting at 90 Kelvin, he pouring it down, or she's pouring it down, and these are the jaw of a permanent magnet.
And it's in a gravity field, it doesn't keep falling, it stops.
And it continues to boil away and as fast as you can pour it, it sits between the jaws of the magnet.
Because it's paramagnetic.
You can think of this as the liquid equivalent of iron filings.
If I told you that were these were iron filings you'd say, yeah the iron filings go and they stick to the magnet, that's what magnets do to iron filings.
They do the same thing to liquid oxygen.
And why?
Because of this.
This explains this.
So we can do a lot with these primitive little diagrams.
Let's do one more.
I want to do one more of these things.
I want to go to hybridized systems.
I want to go to a hybridized system.
And I want to look at ethylene.
C2H4 is ethylene.
OK, so if I told you give me the Louis structure of ethylene, you just start going according to the rule.
So I'm going to have carbon, carbon, hydrogen, hydrogen.
And carbon has one, two, three, four electrons.
The hydrogen has one electron.
I'll put the one electron from hydrogen.
And then the other carbon over here, I'm going to give it dots.
One, two, three, four.
And then these hydrogen each have one.
And now let's see what I have here.
These hydrogens are all isoelectronic with helium, so they're happy.
And the carbon now has two, let's see, what does he got?
Yeah, carbon is happy.
Two, four, six, OK good.
So now carbon is.
Happy as well.
So what do we have here?
This is the equivalent to a carbon carbon double bond.
So what have I learned with the nitrogen example?
What I learned with the nitrogen example was, that if I want to make more than one bond, I have to have a combination of a stigma and a pi.
You can only make one sigma bond because there's no room.
Once you've got zero electron density between the two nuclei you're going to violate the Pauli exclusion principle if you have another orbital cutting through that same domain.
So this means that multiple bonds require a mix of sigma and pi.
Multiple bonds require a mix of sigma and pi.
If you have a single bond, all you need is the sigma.
So let's go back to how we got the original hybridization.
Remember, we started with carbon, with the box notation looking like this.
Here's the 2s and the 2p's and we started with just native carbon off the periodic table.
And we have this.
Just if we go according to the 2s2, 2p2.
And then in order to get hybridization to rationalize methane, where we've got four equivalent bonds.
In that case, we hybridize the s and all of the p's.
We took the s and all three of the p's to make the sp3 hybrid orbitals.
And then we were able to take these electrons and put them in one at a time.
And then bring in the four hydrogens and we end up with something that is symmetrically disposed in space.
The carbon, hydrogen and so.
So now I give to you ethylene I say, well how do I make ethylene.
Well, if I started with this sp3 thing maybe I could bring another carbon over here.
And I'd have three hydrogen sticking out.
So there's my sigma bond.
So I'm off to the races.
This is good.
But now I need to build a second bond.
I'm going to throw away a couple of hydrogen.
So i'll throw this hydrogen away and this hydrogen away.
So I'm going to have now C2H4.
I'm almost there.
The only problem is, this orbital is sticking out this way, this orbital is sticking out this way.
And I don't have the license to bend these orbitals.
And build a double bond.
Because they're 109 degrees apart and they're inflexible and they won't bend.
So this hybridization technique will not work as such to give me what I need to build ethylene.
What do I need?
If I'm going to build a sigma and a pi bond, I'm going to need to preserve one of the p orbitals so that it can still be available for lateral smearing Because how do I make a pi orbital?
I make a pi orbital middle with an 88.
So I need to preserve a p orbital so that in both of the carbons I still have this pi bonding capability.
So what I'm going do is instead of taking the s and all three of the p's I'm going to take the s and only two of the p' s and reserve one of the p' s to be sitting there for lateral smearing.
So instead what I'm going to do is this.
So this is going to be an unmixed p and this is going to take the s and not three p' s but only two p 's.
And this would be called sp2.
And now what do I have?
I've one, two, three.
And how do I put three of these in space?
They lie symmetrically in a plane at 120 degrees.
And then this thing is normal to the plane of the board.
It's sticking out.
Actually, I should've maybe drawn it in perspective like this.
And so now I've got the ability to put two of these together.
This'll giving me my sigma.
And then these two things lying on their side will smear with the lobes to give me the pi.
Now that's what gives me the double bond.
And here are the cartoons that show this.
Oh here, I took this from an old text book, it's text book I had when I was your age.
The wrote great books.
And then I flipped this around see.
I did all this just for you.
So I took this one, I flipped the image around.
So there are the sp2's these are the unmixed p's.
And you bring them close together and bingo.
There's the sigma.
That's this one, p plus p axially, that gives me the sigma bond.
And then I smear those two and that gives me the pi.
And there's ethylene.
You put the hydrogens here, one, two, three, four.
Two carbons.
Tada.
Isn't that cool?
Here's from another book.
It's looks like a Boston traffic map, doesn't it?
Just crazy.
This is from our book.
So they're showing you there's the sigmas and the pi's.
There we go.
More pictures.
OK, I'm going to take three minutes at the end here and show you an example of electronegativity at the extreme.
So if you take a look at the periodic table and look at a compound like sodium iodide.
If I just told you sodium iodide, covalent or ionic?
You'd say, ionic.
Because you got something from the the most metallic of the metals and something from the most non metallic of the non metals.
And you're right.
And the delta chi is 1.73.
And the covalent character is high.
And you end up with something that's very polar and so on.
Now here's a very interesting one, if you compare cesium and gold, you get 1.75.
Which is greater than what it was for sodium and iodine.
Now no one would argue that sodium and iodine is covalent, you'd argue that's ionic.
Well by the same metrics, cesium and gold is as ionic.
The plot thickens.
Cesium, if you melt it it's a metal, liquid metal.
If you melt gold, it's a liquid metal.
But if you mix them in equal number.
So you make a alloy of 50 mole percent cesium and 50 mole percent gold and you've got a delta chi 1.75, you've essentially made a cocktail with equal numbers of really good electron donors and really good electron acceptors.
And guess what happens, electron transfer.
And the melt is not metallic, it turns clear and colorless just as molten sodium iodide would be.
It turns into a molten salt.
So cesium gives its electron to gold.
And gold becomes the negative gold ion.
And what color is every ion?
It's got stable octet configurations.
It's got be the same color as neon, argon, and helium.
They are clear and colorless.
Big drop in electrical conductivity and a shift from electronic to ionic conduction.
Metals have electronic conductivity, ions, what do ionic liquids have?
They have like ionic conductivity.
Here's some data from the literature.
This is the log on the connductivity as a function of concentration.
So here's pure cesium over here, here's pure gold over here.
This is 600 degrees C. So the line stops here because gold melts at about 1060.
So gold is a solid beyond this alloy concentration.
But you can see this is roughly what you'd get.
So electronic conductivity up here at about 10 to the 4 siemens per, this is reciprocal ohms, but siemens per centimeter.
And down here, this is very low value ionic conductivity.
So this is a liquid metal.
And this is a molten solid.
And it all happens just when you get very very close to 50/50.
So you end up with something called cesium oride.
And it's sorcery.
You have one vial of liquid metal.
You have a second vial of liquid metal.
You pour then and it turns clear and colorless.
And the conductivity drops three orders of magnitude all because of electron transfer due to this electronegativity difference.
That's so cool.
That is so cool.
OK, with that I'll let you go.
We'll see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Settle down.
Settle down.
The weekend doesn't start until after the lecture.
First we learn.
All right.
So couple of announcements, the big one being the reminder about that little celebration next Wednesday, coverage through the end of the lecture on Monday, there'll be no weekly quiz next Tuesday, and Hilary wanted me to announce that there's a job opening for someone who's in class anyways, and wants to-- if you take decent notes and your handwriting is legible, you want to make $9 an hour, go see Hilary.
All right.
Let's get into the lesson.
Got a lot to cover today.
Last day, we were looking at the formation of molecular orbitals by linear combination of atomic orbitals into molecular orbitals.
So we had our SLI six-letter initialization, and we saw the formation of sigma and pi molecular orbitals.
And the way you distinguish between sigma and pi is based on electron distribution.
Specifically, in a sigma molecular orbital, you have continuous electron density between the nuclei.
That is to say, no holiday.
So I've drawn a sigma orbital here where I've taken two P orbitals lying on their sides, so they're bonding longitudinally and when the bond is formed, the one nitrogen nucleus is here and the other nitrogen nucleus is here and you can see that there's continuous electron density, no holidays between the two nuclei.
So this would be called a sigma bond.
The pi molecular orbital, on the other hand, has a nodal plane containing both nuclei.
So since the nuclei are in a nodal plane, that means that you've got to have two lobes, one above the plane and one below the plane, and that's shown here.
This is the example of nitrogen, where it forms both sigma and pi bonds and to form the pi bonds, you take the lateral smearing.
And so this is what I call the 88.
If you take the 88, you're going to make a pi bond.
So that's lateral smearing of the p atomic orbitals.
And the last thing I want to make sure that you appreciate is that when you form multiple bonds, you have to have sigma and pi.
You can only form one sigma bond because you have no holidays between the two nuclei so now you can't go and invade that same space with a second bond.
So that means you're going to have to get beyond that zone and that means you have to go into pi.
So if we've got only bond, it's a sigma.
If you've got two, it's a sigma and a pi.
If it's three, it's a sigma and two pis.
That's what we saw in the case of nitrogen.
I didn't complete it here, but there's two pi bonds because this is a triple bond.
So there's a sigma and there's two pi's.
One pi is shown here, and that's vertical to the board, and the other pi is the sigma x, which would be orthogonal coming in and out of the plane of the board.
So those smear give us two of these and then we've got the sigma bond.
And we even extended this to hybrid systems, and towards the end of the lecture, we looked at ethylene, and I just wanted to come back to that for a second because I raced through it there.
And so we wanted to look at C2H4, which is ethylene.
And in the case of C2H4, we recognized that when we went through the Lewis structure, we had a molecule that looked like this, and the thing to note here is that we have a double bond.
So once you see this double bond, then that means I'm going to need a mix of sigma and pi.
Now earlier we had studied methane-- methane, which was CH4, and to get methane, we hybridize sp3.
sp3, which gave us four identical struts coming off the carbon.
Well, we recognized that wouldn't work here because we needed to make a sigma and a pi so the gambit here was to reserve a p orbital.
Reserve a p orbital.
Don't combine all of the three p orbitals.
Combine only two of the p's and that gave us sp2 plus a p. And the box notation for that looked like this.
Here's the p orbital that's not been hybridized.
These are the sp2 orbitals that are a mix of S, and I've taken two of the p's and I'm leaving one of the p's unmixed.
And now I've got four electrons to put in here and that's where I ran out of gas last day.
I have to put the fourth electron in here.
These are still degenerate.
So now I've got still the capability of forming four bonds.
You can see-- whenever you see carbon, there's an interesting-- just something to train your eye, if you see carbon in a structure, you need four sticks coming out of the carbon, or else there's a spelling error.
So look here.
This is carbon with four hydrogens.
One, two, three, four.
This is carbon one, two, three, four.
Here's a carbon one, two, three, four.
It's always got four struts.
That's not what's different here.
It's the arrangement of the struts.
These are all four going at 109 degrees in three space.
This one is going to have three of them identical, because we studied last day the principle that equal energies, equal bond energies, imply-- I want to get the spelling here.
Equal bond energies imply equal spatial disposition.
Equal spatial disposition of those bonds.
So if we have four bonds of equal energy, we have the tetrahedron.
If we have three bonds of equal energy, how do I take three sticks coming out of a central carbon and make them uniformly distributed in space?
They're going to lie in a plane and they're going to be 120 degrees apart.
That's the consequence of having this structure.
All of this follow and why did we get to this?
Octet.
So here we are.
Here's the three of them and here the fourth bond is p, and it's normal to the plane of the board, and now we can take one of these and bond it to another one, bring them in along here.
This is an sp2 and an sp2 and they bond to make a sigma bond.
How do I know?
Look, no holidays.
Continuous electron density.
And now in the plane normal to the board, I have lateral smearing, and I make a pi.
So I've got a sigma and a pi.
That gives me the double bond and let me just show you the cartoons here because we've got some nice artistic renderings.
So here we are.
This is the top view of the three sp2's and then the p's are vertical.
so we're going to move along the z axis, bring two of these together-- there's the smearing of the sp2's on axis.
So this is a sigma bond, No electron density holidays from one nucleus to the other, and now we're going to smear the lobes of the p orbitals of the unmixed carbon.
That gives us the double bond and there we are.
This is the same thing taken from your book.
This is the sigma orbital and then chemists like to do this.
You see this sort of slashed line?
It sort of indicates it's receding.
It's going into the plane and then these are coming out.
This hydrogen is coming out at us and that other one's going way, way in the back and this is in the plane and there's the lobes colored is they irritate me always.
By the way, they've chosen to make this the z-axis, which I don't like.
The z-axis should be the axis along which they bond.
And then here's acetylene.
Acetylene is a compound that C2H2.
C2H2, and I'm not going to go through this in class.
You can try it when you have nothing else to do.
and so it's got a triple bond.
it's got a triple bond between the carbons and then hydrogens on either side.
Again, look at the carbon: one, two, three, four.
But now we're going to have to have two lateral smears, so that means I'm only going to take one of the p orbitals and leave two in reserve.
so when I do that, I'm going to end up with this, starting with the box notation.
Here's carbon as it would exist without any hybridization.
An S and three p orbitals.
And what I want to do-- I want to have two pi's and a sigma, so I'm going to just capture one of these and reserve two p's as unmixed, so that will give me this.
This is one S and one p, so this is called sp hybridization.
And then I've got the two p's unmixed, so now I've got one, two, three, four.
are both orthogonal to one another and these two are what?
I've got two struts coming off the carbon.
How do I get two struts coming off a carbon symmetrically disposed in space?
That's it.
So look at the shape of the molecule.
You see it here.
There's the-- the sigma orbital is here and then we have vertical and then out on the plane.
And so you first would say, wow, I understand that.
That looks a lot like nitrogen.
And then, of course, I looked at that, and I said, what about the right-hand rule?
And according to the right-hand rule, the z is going in this direction.
It's a little more subtle, but I pick up on stuff like that.
I can't help it.
OK.
So this is good.
So what I've done here is gone through a number of instances where we start with electronic structure.
sometimes we hybridize, sometimes we don't.
But what follows here?
When we start with the octet rule and we start playing with the energetics and forming the orbitals, can you see that there's a connection between electronic structure, which then dictates the bond disposition in space, which ultimately dictates the architecture of the molecule?
Molecular architecture.
So there's a way of doing this There's a way of doing this systematically.
I just got ahead of myself.
Before we get to that, I wanted to just pause here for a second and reflect upon the properties of the covalent bond as contrasted to the properties of the ionic bond.
So let me get to that first.
Properties of covalent bond.
OK.
The first property I want to highlight is the fact that the bond is saturated.
Remember when we said that the ionic bond was saturated?
So what does it mean?
Saturated?
It means that there's two atoms only.
Once you've got the electron pair between the two atoms, the bond is formed and that's the end of the story, whereas with ionic bonding, you can keep by electrostatic attraction piling on and piling on.
So this is the concept of saturation.
So we have the shared electron pair.
Shared electron pair and that's all there is to the story.
The other thing about the covalent bond is that it is directional.
And so the bond spatial arrangement-- these are the positioners.
You can't put atoms where there are no bonds and since the bonds have a direct spatial arrangement, then that's going to inform molecular architecture.
Dictates molecular architecture.
So what I'd like to do next-- so for example, here's one that we know.
You've seen this one.
This is methane.
This is sp3 hybridization.
So we learn that methane is tetrahedronal.
We say that the four corners of the hydrogen positions form a tetrahedron.
So we call this molecule tetrahedral.
So I could look at other molecules that have sp3 hybridization.
So if I look at something like titanium tetrachloride, which we met earlier to make titanium, this is titanium in the center and the four chlorines.
It's the same molecular shape.
If you see leaded gasoline, the compound in unleaded gasoline is tetraethyl lead.
Lead at the center, ethyl's at the four corners of the tetrahedron.
So all of these-- and same over here.
This one here, sp2, this means the molecule is planar.
The carbons and the hydrogens lie in a plane.
The whole molecule lies in the plane of the board.
If I give you any molecule that's got sp3 hybridization and has one, two, three, four, five, six atoms, in other words, all the bonds have atoms, the molecular is going to be a planar molecule with this, the 120 degrees.
So we've got a grand scheme here that allows us to codify this behavior and the connection between molecular structure and the electronics arrangement and it's written up here under the term Valence-Shell Electron-Pair-Repulsion Model.
Let's write that down.
I'm going to go through some examples.
Valence-Shell-- hyphenated-- Electron-Pair-Repulsion.
That's the model, and it goes by the acronym VSEPR, which is a five-letter acronym.
This is not an initialization because chemists reverse these, and they call it VSEPR, because the V and the S is kind of hard to pronounce unless you have Slavic blood.
So it's called VSEPR, and that is the scheme that's up here.
So I'm going to go through some examples and show you how you can start with electronic structure and figure out what the molecular architecture is.
So I'm going to follow this.
So first thing we're going to do is write the Lewis structure.
And the example I want to start with is sulfur hexafluoride.
I want to-- so the question is, what is the molecular architecture of sulfur hexafluoride?
Can we predict the shape?
And this is an interesting molecule.
It's very dense.
It's about five times-- it's a gas at room temperature.
It's about five times the density of air.
It's got an atomic mass of about 150, and it's used in the light metal industry.
It's used as a blanket gas for casting of aluminum and magnesium to keep the air away from the liquid metal.
Otherwise, the metal would oxidize.
But unfortunately, the sulfur fluorine bond is a really good infrared absorber, and when this stuff leaks out, it's a fantastic greenhouse gas trapper.
It's got a greenhouse gas coefficient that's, I don't know, something like 10,000 times that of CO2.
So there's a lot of conversation about banning this.
If we ban this, we're going to have to find a substitute.
There is no readily available substitute.
And what that'll mean is the cost of aluminum castings and magnesium castings will go up, which means that they won't be used in automobiles.
Instead, cheaper steel will be used in automobiles, which means the mass of the automobile will go up, fuel economy will go down, tail pipe emissions will go up and greenhouse gas emissions will go up.
So how do we get ourselves out of this situation?
Do we ban SF6 or not?
How do you go about thinking about that problem?
Well, if you're in Washington today, it's the lobbyists with the biggest bag of money that comes in.
That's what's going to dictate policy.
But if we lived in an ideal world, we'd have people like you who understand the chemistry and what would be the analysis?
You'd look at all the SF6 emitted from the cast shops in a given year and compare that to the increase in tail pipe CO2 emissions if you replaced the aluminum with steel, and then you figure out what the trade off is.
And maybe what you'd do is set up a policy where you would gradually phase this out with-- heaven forbid-- solid research support to give us what we need.
That way we get clean air and everything else.
OK.
So that's what we would like.
Now let's look at the Lewis structure of this thing because that's the first thing we're supposed to do according to VSEPR.
So I'm going to put sulfur here and I know sulfur's got-- it's 3s2 3p4.
So it's got six valence electrons.
I'm going to put them symmetrically around the sulfur, and then I'm going to bring in the fluorine.
I'll bring in one fluorine, and I know it's got seven valence electrons: one, two, three, four, five, six, seven.
And the same thing with the other fluorines.
So I'm going to put one, and then for the electron pair, I'm just going to draw a line.
OK. You're smart.
You can figure that out.
That's just shorthand.
OK.
Here's another fluorine.
One, two, three.
So there's two, four, six, seven.
There's one, two, three, four, five, six, seven.
One, two, three, four, five, six, seven.
One, two, three, four, five, six, seven.
Beautiful.
OK.
So now what I want to do is identify all of the bonding orbitals.
So I see around the sulfur, I've got one, two, three, four, five, six, six bonding orbitals, six bonding domains, if you like.
Well, that's 12 electrons.
I said octet stability.
Looks like sulfur took that idea and ran with it.
And it turns out that this does happen on occasion in elements that are very electronegative.
Very, very strong non-metals will form what is known as an expanded octet.
And here we've got-- here's a list of elements.
[MICROPHONE ADJUSTMENT]
We need a high-tech tie clip.
Pull out a little more, maybe it's constrained.
It's tethered.
the bond is too tight.
That's what it is.
It's not flexible.
It's covalent, you see.
All right.
So here we are with the elements, and the ones shown in the in the grayish-blue here are the ones that on occasion might form an expanded octet.
Not always, but if you do see 10 or 12 electrons around one of those elements, don't be alarmed.
If you see 10 or 12 electrons around the potassium, that's probably a mistake.
OK.
So we have six bonding domains and what else do we have?
We have six bonding domains, and we have zero non-bonding domains, and in total, we have six electron domains, in other words, six electron pair systems around the central sulfur.
And on the basis of the total of bonding and non-bonding, this is what dictates the skeletal structure.
In this case, the skeletal structure is trivial.
It's the same as the atomic structure because there are no non-bonding domains.
And so I've got six of them.
They're equivalent.
So what's that going to mean in terms of shape?
Well, I'll put the sulfur in the center and I've got one, two, three, four, five, six.
So two in one plane crossways and then two vertical.
And then I put the fluorines-- one, two, three, four, five, six.
So there's the molecular architecture and it's termed octahedral.
I know at first this throws you because you see six struts.
Where do you get the octahedral?
Well, if we connect the four fluorines along with the sulfur, they lie in a plane, and if we were to pitch a tent here, we could pitch a tent to the upper fluorine, and we'd end up with four faces.
So this is Greek.
Eight faces.
So there's four above and four below.
I know it's sort of like a Jasper Johns painting.
You say, octahedral.
You hear octa, you're thinking eight and you're seeing six fluorines or seven.
Anything but eight.
So this is anything but eight, but it's eight faces.
Last thing-- let's look at the energetics.
To hybridize like this, we needed to do something different because we know that the base structure of sulfur would be 3s2 3p4.
So this is 3s and this is 3p.
So that's 3s2, and then 3p4 would be one, two, three-- whoops!
I violated the polyexclusion there.
OK.
There we go.
So I can't form six bonds with this.
I only have two bonds.
So you say, well, okay, let's make sp3.
Well, if I make sp3, sp3 will give me this.
So now I got one, two, three, four, and then I'm going to put five, six.
I'm still down with two bonds.
I need to have six unpaired electrons.
So I've already consumed my s's and my p's.
Where else can I go shopping for orbitals?
I go to the orbital store and the orbital store is over here.
Here's 3d.
One, two, three, four.
So I've got one, two, three, four, five d orbitals.
So what I'm going to do is, I'm going to take the s, I'm going to take all three of the p and I'm going to lop off two of the d's.
So now I'm going to have one, two, three, four, five, six and I'll leave three d's by themselves and here's now sp3d2.
And now I've got six electrons-- one, two, three, four, five, six and now I have six equal energy states, which means I have to have six struts equally disposed in space and everything makes sense.
All right.
Is this polar or nonpolar?
Is this molecule polar or nonpolar?
Nonpolar, good.
Let's look at it carefully.
What do we have?
I've got sulfur fluorine.
Whenever you see fluorine, that's the biggest electron hog on the Periodic Table so you know you're going to have a dipole moment on the bond.
But I have six such bonds and they're symmetrically disposed in space.
All of the fluorines lie on the surface of a sphere so the center of local negative charge is right on top of the sulfur nucleus, which is where the center of locally positive charge is.
No electron displacement, no net dipole moment.
OK.
This is working.
Let's try another one.
Let's see.
What do I have here?
I got bromide pentafluoride.
So start-- let's go back to the VSPER rules.
There we go.
OK. VSPER rules.
So I put bromine in the center and it's got seven valence electrons-- one, two, three, four, five, six-- and I'll put the seventh up here.
And I'm going to bring in the fluorines-- one, two, three, four, five and just as before, there's one, two, three, four, five, six, seven.
So you get a bond with each fluorine.
A bond with each fluorine.
All right.
So what do we have now?
How many electron domains?
One, two, three, four, five, six.
Six electron domains as before, but one of them's nonbonding.
I get to use the colored chalk again.
All right.
So this is one nonbonding domain and five bonding domains.
One, two, three, four, five.
One bond to each of the fluorines, five bonding, and the total is six electron domains, so that means octahedral structure.
The total number of electron domains dictates skeletal structure.
So man said put six.
So I put the bromine here, and I go, one, two, three, four, five, six.
That's the skeletal structure.
Those are the bonds and space.
Now I know one of those is a nonbonding domain.
So where do I put it?
Well, they're all equivalent.
It doesn't make any difference.
Nobody knows where the real x is in this world.
So just for grins and chuckles, I'll make-- I'll put the nonbonding pair down here.
And then I put the fluorines at the other struts.
And so now what's the shape of the molecule?
You don't see these electrons.
And in fact, this is only done by chemistry professors, the other ones, not me.
I only do this to let you know what the books are doing because we all know that those nonbonding electrons are in here.
They're right up tucked against the bromine.
So if you walked in the room, what do you see?
The bromine and the four fluorines lie in a plane, and then you've got this thing sticking up.
So the molecule is a pyramid with a square base.
So it's called square pyramidal.
And how about polar or nonpolar?
Is there a net dipole moment?
Yes or no?
Yes, because we got the-- in the plane with the bromine, everything is centered, right?
The fluorines pull out, but they all pull out symmetrically, but then there's the fluorine pulling up.
So this is polar.
This is polar and it's got a dipole setup like this.
Good.
All right.
Let's do one more.
I'm having so much fun.
I'm going to do as many as I can until I run out of time.
This is the best part because colored chalk.
That's the end.
All right.
Let's do this one.
Tetrafluoroiodate.
IF4 minus.
This is cool because it's got a net charge, but it's a covalent anion.
So same thing.
Lewis structure.
So we're going to put iodine first.
Iodine and it's just like bromine.
It's got one, two, three, four, five, six-- I'm going to put a seventh electron here because I'm anticipating.
Just to mix things up.
See, last time I put the seventh electron up.
I could put the seventh electron down, but it's got a net charge.
How'd it get the net charge?
By accepting an electron.
So that electron's got to be here, too.
So I'm going to indicates that that extra electron's here as well.
So iodine's got eight already.
So we're off to a roaring start.
It's got eight and it hasn't even started bonding yet.
So we know where we're going here.
We're going to expanded octet land.
All right.
So we need four more fluorine.
I'm going to put four fluorines-- one, two, three, four-- and as before, so we've got bonds.
One, two, three, four.
And we've still got some other orbitals lying next to the iodine and they bond to nothing.
So we have two nonbonding, and we have four bonding for a total of six electron domains.
Again, octahedral structure.
Octahedral skeleton.
So I'm going to put iodine here.
One, two, three, four, five, six.
All right.
Now comes the part-- see, we said-- this VSPER consists of valence-shell.
I've been doing a lot of valence-shell thought here, but I haven't said anything about electron-pair repulsion.
Now I'm going to show you how electron-pair repulsion comes into play.
So I can put four fluorines around here in two different ways.
One way is to put the four fluorines into play and then I have the-- these are called lone pairs.
Why are they lone?
Because they don't bond to a second.
So sometimes chemists call these lone pairs.
They're all alone.
It's Friday and all alone.
So here they are.
So that's one way to put them, but there's another way to put them, and that's this one here.
I'm going to go way over to the other side, and here's the other way.
The other way to put them is to put the pairs in the same plane next to each other, because if I put the pairs opposite each other, that's equivalent.
It doesn't matter.
It just-- whether on the x-axis or y-axis.
So if you think about this as a globe, you think about the molecule as a globe, so in this case over there, I put all the fluorines on the equatorial position.
You can think of this plane at the waistline as the equator.
So these are all at the equatorial position, and the electron pairs are at the poles.
So the lone pairs are at polar positions.
In this case, the lone pairs are lying in equatorial positions.
Now what do you know about electrons when they get close together?
They repel.
And it's a really strong repulsion.
That's the Bourne exponent.
Electron-pair repulsion is really strong and it'll jack the energy way up.
So if you've got a choice of putting electrons here close together or here farther apart, which one is going to be the lower energy condition?
This, because this minimizes electron-pair repulsion, and in fact, this is the one that's favored.
So by using valence-shell thought, bonding, nonbonding skeletal structure, and then finally electron-pair repulsion, you conclude that this is the correct form, and so you don't see these electron pairs because we know they're really tucked in tight here.
What do you see if you do a spectroscopic analysis of this thing?
All five atoms lie in a plane.
So this molecule is planar.
It's planar, just as the sp2 ethylene was planar, but this is planar for a different reason.
Is this polar or nonpolar?
See, it's negative.
It's not negative.
If I were a cation, I'd be drawn in like this.
So is it polar or nonpolar?
Where's the net displacement?
The fluorines are pulling away from the iodine, but they're pulling away symmetrically.
So there's no net dipole moment.
It's cool, huh?
No net dipole moment, but net negative charge.
I love it.
It just-- it's the best.
OK.
So now let's look at some others.
We've looked at CF4.
We know CF4.
CF4, you could do this one in your head.
This would be just like methane, right?
This would be tetrahedral.
I'm not even going to write that out.
This tetrahedral and it would be sp3 hybridization and we just did IF4 minus.
See the stoichiometry.
It's four fluorines.
Four fluorines with something and it's not the same architecture.
So stoichiometry doesn't dictate molecular architecture.
Very important.
You can say, well, it's a net charge.
A net charge doesn't do-- it's this.
This is square planar.
Yeah, so I'm going to add that.
It's not only planar, it's planar and it's a square, whereas the other one is trigonal planar.
So this is square planar.
So I'm going to do another F4, just to show you, make the point that this thing doesn't-- so I'm going to do sulfur hexafluoride.
You say, gee, sulfur hexafluoride.
I bet it's just like carbon because carbon, sulfur-- no.
Let's look carefully.
So I'm going to put the sulfur here and we're going to put-- how many electrons?
One, two, three, four, five, six.
OK. And then we're going to bring in the fluorines, one, two, three, four.
So each of the fluorines is going to form a bond.
Do I have to do this?
Can I just do this?
Yeah, that's enough.
I'm not-- I'm raising-- we're accelerating now.
Now I'm going to stop talking and I'll just think about it and you'll get it.
All right.
So here we go.
Bonding domains.
One with the fluorine, two with the fluorines, three with the fluorine, four with the fluorine and there's two electrons, and they'll just sit together in a nonbonding domain.
So how many domains?
One, two, three, four, five.
Five electron domains.
Four bonding, one nonbonding It all adds up.
Good.
But now I got to put five struts.
This gives me the skeletal structure.
As before, skeletal structure.
So I've got to-- how do I put five struts around something uniformly in space?
One, two and I'll put three in the plane.
Three in the plane.
So three at the equator and two at the poles.
That's the skeletal structure.
This is called a trigonal bipyramid.
Because I can make a pyramid above, right?
I got a tent with three corners.
So that's a pyramid with three faces.
So that's called trigonal.
It's trigonal-- three-- and then I've got it above and below.
So that's trigonal bipyramid.
See, you're going to learn all these new words.
When you go out tonight, you're going to be able to impress people to the point they'll walk away from you.
All right.
So now let's start putting the fluorines-- again, we've got choices here.
They're not equivalent.
So one possibility is to put the lone pair at the equator and then the fluorines go here, and the other possibility is to put the-- let's see.
I've got room up there.
The other possibility is-- let's make the trigonal bipyramid-- is to put the lone pair at a polar position and put all the fluorines in the equatorial plane.
And so now we have to think about this for a second and try to convince ourselves which situation has greater electron-pair repulsion.
So the way to think about it is, in which position does that lone pair come into more contact than otherwise?
The other thing to know is-- I didn't say this earlier, but it just occurred to me-- is that if you've got a choice of bonding electrons versus nonbonding electrons, nonbonding electrons want more room.
Why?
Because they're not tightly confined.
When you get something in a bond, it's on axis between the two atoms, and so things are much more tightly confined.
So these things want a lot of room.
So if I'm sitting here, I interact with this fluorine and this fluorine-- these two are far away.
If I'm here, I interact with three fluorines.
So it turns out the equatorial position is favored.
Give more space nonbonding-- place nonbonding domains at equatorial positions in trigonal bipyramid.
That's what number five means.
If somebody asked you to test the microphone, instead of saying number nine, number nine or one, two, three, say, place nonbonding domains at equatorial positions and a trigonal bipyramid.
And they'll ask you to step away from the microphone.
OK.
So this structure here is, in fact, the shape of the molecule.
You don't see this.
And this molecule is called seesaw, like a teeter totter.
You remember, when you were kids.
You don't remember that.
OK.
So how about polar versus nonpolar?
It's obvious.
If it's asymmetric and you've got the fluorines and the non-pair, this is definitely polar.
And where's the net dipole?
Where's the negative side?
Well, along this axis, it's pulled right to the center.
So along this axis, clearly the fluorines are pulling so you have something that looks like this.
And how about the energetics?
How do we get five of these?
If we've got five struts, it means we must have five hybridized orbitals of equal energy.
Well, we got six before and before that, we had four.
Before that, we had three.
Well, now we're going to get five.
So how do we make five orbitals?
So in this case I start-- I want s, p, d, and I want five.
So one plus three is four plus one is five and that gives me sp3d.
Gives me five and then, what do we got?
One, two, three.
It's good.
OK. Now-- so we've covered a lot.
All right.
So some parting comments.
We've been sticking other things-- what we could do is we could do this one.
We could take carbon and I'm going to go now sp3.
only instead of putting hydrogens, I'm going to put carbons here.
So what happens now?
One, two, three, four.
I'll put another carbon here.
One, two, three, four, et cetera.
So this is sp3, all carbon, and this is called diamond.
This is called diamond sp3.
So these are all sigma bonds, all sigma bonds everywhere, sp3.
And these are very, very strong bonds and these are very tight bonds-- and one of the properties of diamond is that it has a very high refractive index because high energy-- and these are tight, tight bonds.
So the electrons aren't easily excited.
It's transparent, divisible light, which means that when light shines on this, there's no expectation, but there is bending.
So it's very, very high refractive index and a high refractive index is what makes this such a troublesome compound because it concentrates light.
So you can imagine there's somebody in a dimly lit cafe, and she's sitting over in the corner with a candle in front, and she's wearing a diamond stud, and it takes those few candellas of energy.
It concentrates, it shoots it in the room, and some unsuspecting fellow gets hit in the eye, is attracted to the person wearing the earrings, and that's when the trouble begins.
So this is very, very dangerous, very dangerous, and it's a consequence of these strong sigma bonds.
But there's another way.
There's another way that we can bond carbon.
We can make sp2 hybrids.
So if we make sp2 hybrids, that now looks like this.
The carbon lies in the plane, 120 degrees, which means we're going to end up with something that looks like this, et cetera, et cetera.
And this is graphite.
This is graphite.
And now we have a mix of sigma bonds in the plane that are very strong.
In fact, they're every bit as strong as these bonds, but there's a fourth.
I said, you always have to find four struts off the carbons.
So where's the fourth strut?
It's the pi bond that comes out of the board.
So there's a pi bond here and a pi bond here and a pi bond here and what we're going to learn later is that these things are delocalized.
Because of the spacing here, the electrons can form a bond here that then forms a bond here, but it can actually resonate in such a way that the electrons can move everywhere throughout the crystal of graphite and this is what gives graphite its electrical conductivity, whereas diamond is a strict insulator.
So these are very strong bonds in the plane, but not between planes, which gives graphite its lubricity.
It's a dry-lock lubricant and so on.
OK.
So this is very good.
So we have a mix of sigma and pi bonds.
And it's an absorber and it's dark.
It's an absorber so it appears black, whereas this diamond is transparent to visible light.
And if you look in your book, this is all categorized for you, everything that we've done here with VSPER.
All right.
Look at this.
All nice big system.
OK.
So here's diamond, which I was trying to draw over here, four struts sp3 hybridized.
this is graphite sp2, and these are pi bonds in between.
OK. Now suppose this fellow asks this girl out, and things get serious and they get-- I'm going to use an adverb here-- really serious and he decides maybe he wants to make a commitment and he wants to buy her a ring, an engagement ring, and the custom is that the engagement ring has a stone in it.
Now being an MIT student, he knows that he wants to have symbolism in that stone.
Now the custom is to get diamond, but it turns out that if you look carefully, the stable form of carbon at room temperature and atmospheric pressure is not diamond, it's graphite.
So if he wants to give his love this stone that symbolizes eternal commitment, wouldn't he choose the stable form and not the one that's metastable and doesn't represent the natural state at room temperature?
That's the truth.
That's the truth.
Graphite is the stable form.
I leave it up to the young man to persuade the young lady that the gemstone is going to have graphite in it and not diamond.
OK. Now something very interesting happened.
What I wanted to play for you as you're leaving today is an old Beatles song, Lucy in the Sky With Diamonds.
And what happened-- before I get to it, what happened was-- and this is even in today's Le Tech if you look on the second page, "New Fossil Skeleton from africa Predates Lucy."
It was just discovered.
I'm not kidding you.
This isn't a joke.
I heard this on NPR last night.
So it's a female skeleton of a pre-hominid, 4.4 million years old, which is a million years older than Lucy.
Lucy was discovered in Ethiopia in 1974 and considered to be 3.18 million years old.
This new one is 4.4 million years old, about four feet tall, and it just turns the clock even farther back.
The interesting thing is the name.
How did the skeleton discovered in 1974 get the name Lucy?
Interesting story.
There were two American archaeologists that were head of the team that did the dig, and they were so excited the night they found this pre-hominid skeleton.
They knew it was a female, and they hadn't dated it yet, but they knew that it was roughly way, way back, but it was definitely female.
So they were back at the campfire, kicking back, drinking, probably heavily, and singing songs, and one of the songs they were singing was Lucy in the Sky With Diamonds and that night, they named the skeleton Lucy.
But the story doesn't end there.
Where did this Lucy in the Sky With Diamonds come from?
John Lennon, one of the-- you may have heard of this group or not, but there was this group called The Beatles.
So one of the lead composers was John Lennon.
His six-year-old son came home from school one day with a piece of artwork, and the father asks, what is this?
He says, it's a painting of my classmate Lucy, and it was Lucy and he had diamonds in the sky.
So it was Lucy in the Sky with Diamonds, and then they wrote the song about it.
The real Lucy just died this week.
She had Lupus.
She died at the age of 46.
And all of these things have come together just because we had this lesson today about covalent bonding, which took us to this, which then took us to Ethiopia, which then takes this up here.
and so with that, I think we'll go for an early dismissal, and I'll wish you a pleasant weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Let's go to the lesson.
So the last day we studied VSEPR, which allowed us to infer the shapes of molecules, covalent molecules.
Today I want to talk about the state of aggregation.
What do I mean by state of aggregation?
Is something a solid, a liquid or a gas at a particular temperature?
So 3.091 is introduction to solid state chemistry.
One of the things we need to know is, under what conditions is the solid state stable?
And the state of aggregation, when it applies to covalent molecules, is going to lead us to a discussion of secondary bonding.
So today's lecture is state of aggregation or secondary bonding.
Now let's just reflect upon what we've learned up until now.
We studied ironic bonding and we knew that ionic bonding necessarily leads to crystal formation because we have unsaturated bonds.
And that leads to an ion array of unlimited size until you run out of reagent.
And something honking big made of ions is going to be a solid at room temperature.
Last day we appreciated with covalent bonding we have two options.
We can either make discrete molecules such as HCl.
And HCl as a discrete molecule, depending on a number of factors that I'm going to show you today, could be a solid, liquid or gas.
And we're going to understand more deeply how to sort out between the three of them.
Or, I showed you at the end of the last lecture, you can make a three-dimensional network and diamond was one example.
If it's a three-dimensional network that is a large array of solid then that means that you're going to end up with a formation of a crystal, which is going to give you a solid.
Diamond is solid at room temperature, graphite is solid at room temperature.
So let's now go to the one case that we haven't dealt with, and that's the formation of discrete molecules.
So let's look at discrete molecules.
And what we want to understand is whether they're going to be solid, liquid or gas at room temperature and what's the relevant physics here in order to make that determination?
The relevant physics is the following.
We're going to compare two forces.
We're going to compare the cohesive force between molecules versus the disruptive force.
And the disruptive force in 3.091 is always going to be thermal.
It's thermal energy.
We know this.
As we heat things they go from solid to liquid to vapor as temperature increases.
So thermal energy plays the role of the destructive force whereas the bonding is something that is the cohesive force.
So let's go back to HCL.
Last day we looked at HCl.
So here's one HCl molecule.
We have a covalent bond.
It's a covalent bond within the molecule.
We know this is a polar covalent molecule, with the chlorine having greater electronegativity and pulling the electrons towards its end.
And furthermore, this bond I want to categorize as within the molecule so I'm going to call it intramolecular.
Intramolecular.
Now if I want to answer the question, is hydrogen chloride going to be a solid, liquid or gas at room temperature, I don't have the information on the board.
The only way I can answer that question is to put another hydrogen chloride.
This simply forms the bond between hydrogen and chlorine.
To determine whether hydrogen chloride is a solid, liquid or gas, I need to look at how one hydrogen chloride molecule bonds to another hydrogen chloride molecule.
So now this end is the delta plus this end is delta minus.
And we have again an intramolecular bond.
So the question is, how does one hydrogen chloride bond to another hydrogen chloride?
This is the intermolecular bond.
Intermolecular bond.
So intermolecular bonds govern the state of aggregation.
And I'm going to show you the different types of intermolecular bonds.
This intermolecular bond isn't the primary bond.
The primary bond is here inside the molecule.
So the intermolecular bond is known as a secondary bond.
And there three types of secondary bonds.
We're going to look at them in turn.
So the first one is depicted here on the board.
And it's called dipole-dipole interactions.
And these obviously occur only between polar molecules.
Operative and polar molecules.
Not in.
Between.
Take out in.
Operative between polar molecules.
So we've made something covalent and now how does one stick to the other?
Alright and I think I've got a cartoon here that shows this.
There we go.
That's taken from an old book that I had.
So you see the H and the Cl and we have a dipole moment and the negative end of one dipole attracted to the positive end of another dipole.
So we have a net dipole moment here and we can measure the distance between the center of the dipole-- center to center spacing and call it r. Center to center dipole spacing.
And say that the energy in dipole-dipole interaction-- I don't expect you to know this by heart and do anything with it, but I just want you to realize that it's a weak force.
This is very weak.
It's proportional to the magnitude of the dipole moment.
You'd expect that.
Weak dipole moment, weak energy.
Strong dipole moment, strong energy.
Turns out it goes as the fourth order of the dipole moment.
And it's inversely proportional to the separation.
Only it's not coulombic, it goes as 1 over r cubed.
And there's a temperature factor.
1 over t. As temperature goes up, this energy goes down.
And these are very weak forces on the order of about 5 kilojoules per mole.
You remember what the crystallization energy of sodium chloride was?
787 kilojoules per mole.
So this is very, very weak.
It's operative at, between, at low temperatures.
At very low temperatures.
So for example, hydrogen chloride, in the case of hydrogen chloride, the melting point of hydrogen chloride is minus 115 degrees C and the boiling point is minus 85 degrees C. You can see at very low temperatures we already have enough thermal energy to disrupt.
So if we have solid hydrogen chloride, the forces between the hydrogen chloride molecules in the solid are these very weak dipole-dipole interactions.
And I think there's a couple more cartoons here.
This is taken from your text.
So there we go.
There it is.
That's the soup that might be HCl liquid.
OK so that's the first type of secondary bond.
Dipole-dipole interaction.
Let's look at the second one.
The second one is called induced dipole-induced dipole.
And it's operative in non-polar species.
Dominant in non-polar species.
Why am I using the word, species?
I'm trying to be a pedant and use fancy words?
No.
Because I'm going to make it generic.
I'm going to show it works in atoms and in compounds.
So as examples, what are some non-polar species?
Well how about something like argon?
Argon if you look on the periodic table it'll show you that it has a melting point of 84 kelvin and a boiling point of 87 kelvin.
So if I cool argon to below 84 kelvin, I get argon ice.
So what are the bonds between one argon atom and another argon atom?
It can't be this.
There's no net charge.
It can't be ionic.
It doesn't form a covalent network the way diamond does.
How do you justify the existence of solid argon?
It's just so cold that it just sits there and freezes?
I mean, how does it bond?
There needs to be some kind of bonding.
And other non-polar species.
So for example, the molecule iodine.
Iodine melting point is above room temperature.
It's a solid crystal at room temperature.
Well there's a strong covalent bond inside iodine.
But how does one iodine bond to another and to another and to another?
And we can even go to polyatomic species, such as methane.
You know there was data from the Cassini Probe.
Look at this.
This is an image from the Cassini Probe.
This is an island group.
The yellow is an island group.
The blue is a methane sea.
A sea of liquid methane on the moon of Saturn, Titan.
So there's liquid methane.
How does liquid methane form?
What causes one methane to bond with another methane?
That's the question that we're wrestling with.
So let's look at it first in a simpler context.
Let's look at it with argon.
So I put argon here.
It's a spherical atom.
And the question we had before says, how does one argon bond to another argon, as has to be the case in the solid or liquid argon.
So we know that this has a lot of electrons and the atom is net neutral.
But the electrons are in motion.
If I had an attosecond camera and I went in here and I went, snap, I could catch a freeze frame where the electrons aren't symmetrically distributed around the nucleus.
This nucleus, the atom is in constant fluctuation.
But net neutral.
Time average it's symmetrical.
So at some moment I might have a preponderance of electrons over here at around three o'clock.
So this end of the argon is a little bit delta minus.
Which means the other end is delta plus.
Now what happens if this is delta minus, this is delta plus?
Can you see that the positive end of this argon atom will then pull on the electron distribution of the adjacent argon atom rendering it delta minus at three o'clock and delta plus?
So this is the dipole in one, induces a dipole in another.
And why does it occur in the first place?
It occurs because the electrons are in motion.
Electrons in motion lead to time fluctuating dipoles.
They're time fluctuating.
Time average, there's no net dipole moment.
I'm not going back on what I said before.
There's no net dipole moment here, but this is time fluctuating dipole.
Time fluctuating dipole.
So who was the first to explain this?
The first to explain this was Fritz London in 1930.
In 1930 Fritz London gave us the explanation because this troubled people for a long time.
And he did so in a quantitative manner.
And the mathematics of the treatment are identical to the mathematics for the dispersion of light under certain conditions.
This has nothing to do with the dispersion of light.
The mathematical formulation imitates the formulation of the dispersion of light and hence the force here is known as the London Dispersion Force.
So people will say that solid argon is held together by London Dispersion Forces.
Or the bond is known by the name van der Waals.
We either call them van der Waals bonds or London Dispersion Forces.
Van der Waals was Dutch.
I know how to spell the word wall.
This is the dutch spelling.
W A A L S. Van der Waals.
So in this system here, the London Dispersion Force or the van der Waals bond is in fact not the secondary bond.
It's the primary bond, isn't it?
It's the only bond.
So by definition it must be the primary bond.
But can you see that in every compound, including diamond, we have time fluctuating dipoles in all of the atoms?
It's not just argon that has time fluctuating dipoles.
Every atom in you and me has time fluctuating dipoles.
But we're held together by much greater forces.
So that's why I say that in this case this is the dominant force.
But it's operative in everything.
Because wherever we have electrons, they're in motion.
So this is the dominant one.
Alright so we can look at it in a few other instances.
So for example we can look at it in the case of iodine.
We can look at it in the case of methane.
All the same.
Time fluctuating dipoles.
And this is a very, very weak force.
So the energy in London Dispersion Forces, or van der Waals bonds, is proportional to a quantity called the polarizability.
The polarizability is-- and this was defined by London-- polarizability is the measure of how easy it is to induce this dipole.
A measure of the ease of electron displacement within an atom.
And it depends on, it's influenced by the size and it's influenced by the number of electrons in the atom.
So what do I mean by that?
Well let's take two systems.
Suppose I've got argon and I'll go up two members in the same series and I've got helium.
So what are the forces that hold helium together?
Same thing.
But the boiling point here is 87 kelvin.
The boiling point here is 4.2 kelvin.
Why does this have such a low boiling point?
Well, polarizability.
Size-- helium is smaller.
So the degree, the corral in which the electrons can roam is smaller.
So the extent of electron displacement is smaller.
And secondly there are only two electrons.
And they're in 1s, so they're tightly held.
And so the delta minus delta plus capable of being created in helium is tiny compared to the delta minus delta plus that can be created in argon.
So the energy goes as polarizability squared and divided by r to the sixth, where this is the separation.
Not the radius but this is separation.
Dipole separation.
And this is a font issue.
This is proportional to alpha.
So you can tell the difference.
This is a proportional that's to alpha.
Of course you can't tell the difference.
This is contextual.
If I just wrote this by itself, you don't know what it is.
My goodness you're nervous.
So nervous.
All right let's take a look at-- here's the cartoons.
Induced dipole.
And here's London's paper.
Here's London's paper as it first appeared in 1930.
Theory and System of Molecular Forces.
And here's some cartoons from your book.
This is helium, showing helium.
It's kind of funny how the artist shows.
Like there's these two helium atoms and they're absolutely immobile and all of a sudden one of the helium atom starts jiggling and then it induces a dipole moment in the other one.
That's not how it happens, but anyway.
Alright this is hydrogen.
Same thing as iodine.
All right so there's delta plus, delta minus.
Because the electrons are moving even though there's a strong covalent bond inside.
So let me just make the point, I want to make sure people are very clear about primary versus secondary bonding.
So if I look at iodine.
I2.
This is a covalent bond.
It's homonuclear.
This covalent bond, there's no net dipole moment.
Here's another iodine.
But then we have time fluctuating dipoles.
So this delta minus isn't because this is the bond here.
This is the primary bond.
This is the primary bond and it's covalent.
And this is the secondary bond.
And the secondary bond is London Dispersion Force or van der Waals bond because this is induced dipole.
So now let's look at-- oh here's another one.
This one gets on my nerves.
Oh actually this is good.
This is this the bad one.
All right so I'm going to show you polarizability.
So here's a series of sp3 hybridized chains.
Propane, octane, icosane.
They're all the same.
They look like this.
Ch is sp3 hybridized.
And so you just have-- so what is that, C 3 H H. So it's just-- so you have 1, 2, 3 4. so there's hydrogen, hydrogen, hydrogen, and on the fourth one I'm going to make it flat instead of trying to make a tetrahedron.
So 1, 2, 3 hydrogens.
The fourth one is carbon.
1, 2 hydrogens, carbon carbon.
1, 2 3 hydrogens.
So this is C3H8.
And compare that to the longer one.
Which is octane, which is C8H18.
So it's 1, 2, 3, 4, 5, 6, 7, 8.
And all you have got to do is, 1, 2, 3, 4, 5, 6, 7, 8.
You put four sticks off of every carbon.
You can't go wrong.
You count them up.
You got C8H18.
Look at the structures.
They are the same.
So how come propane is a gas at room temperature?
Whereas octane, which is the principal constituent of gasoline, is a liquid at room temperature?
It all comes down to polarizability.
This one's got a longer corral.
So if you like the delta minus versus the delta plus, it's basically the same here except that separation is bigger.
All right, so that's pretty good.
This is the one that gets on my nerves.
You see this?
It's exactly what I just showed you.
So here's methane.
And there's the propane.
Here's butane and somewhere between butane and pentane we cross the the line at room temperature.
So pentane is liquid, butane is gas.
And you keep going up, up, up, and finally if you get to C20 you go from liquid to solid because now the van der Waals forces are strong enough that even at room temperature you make a solid.
It looks like paraffin.
That's why you can melt paraffin, because it's got weak van der Waals forces.
So temperature disrupts and it reforms.
If you try to break a covalent bond, you pyrolyze the thing and you don't get it back.
Secondary bonds allow for ready processing.
Here's what bothers me about this.
Instead of talking about polarizability, which is a physical quantity that means something.
They say molecular weight.
Well it's true that these scales, these are heavier and it's monotonic.
But to me what's the relevant physics between the ability to form van der Waals bonds and the mass.
It's just a dumb thing.
It's not a gravitational effect.
So that's why this thing is stupid.
And you see it all over in chemistry books.
And it's just dumb.
And if you put that on my exam you're not going to get points for it.
Because that's dumb.
OK let's go to the next one.
There's a third type of bonding.
There are certain things I feel strongly about and that's one of them.
OK so let's look at the third type of secondary bonding.
The third type of secondary bonding is called hydrogen bonding.
Hydrogen bonding.
It's a type of secondary bonding.
And it occurs between hydrogen and the most electronegative elements, fluorine, oxygen, nitrogen.
Why these?
Because they have very, very high average valance electron energy.
So the average valance electron energy I'm quoting in that second column, not megajoules per mole, but in sensible units like electron volts.
And so you can see, when you get up around 18, 19 electron volts you cross a threshold.
And that's the electronegativity as represented by average valance electron energy.
So you can see that as the electronegativity gets beyond some threshold value, roughly 3, then you can form hydrogen bonds.
So this is owing to high average valance electron energy, or if you like electronegativity.
Which means very strong polarity in the covalent bond.
So you say, well there's polar and there's even more polar.
So let's see what happens.
So I'm going to use a prototypical value here.
I'm going to look at HF.
So if I look at HF let's go through the Lewis structure.
There's H with it's 1 electron.
And F with the 7.
1, 2, 3, 4, 5, 6, 7.
And so we have a covalent bond here.
We know fluorine is the most electronegative so we have a dipole moment here.
Now the dipole moment is very strong here.
This is an accounting procedure to put the two electrons, but it no way represents the physical position of these electrons.
They're brought in very close to the fluorine.
So it's not some symmetrically disposed between H and the F. So I can't tell anything about whether HF is a solid, liquid or gas.
Why?
Because there's only one sitting here.
I have got to put at least one more.
Because this is a primary bond.
And it tells me how H bonds to F. It doesn't tell me how one HF bonds to another HF.
So I'm going to put another HF over here.
So here's another HF.
And it's also dipole.
But there's something special about that.
Already there's a dipole-dipole interaction.
But the hydrogen bond is much stronger.
It's much stronger.
And why is it stronger?
The electron in this hydrogen on the right is pulled towards the fluorine to such an extent that this hydrogen is so denuded of its electrons that it's acting as proton-like.
It's proton-like.
Now don't tell people that Professor Sadoway said that hydrogen inside an HF is a proton.
It's not a proton but it's starting to get more nearly like a proton.
Now what do we know about a proton?
Positive charge.
Tiny high-charge density.
So this hydrogen's looking forlornly over at its electron that's being hogged by the fluorine to a right.
And what do we know about these?
Oh it's time for colored chalk.
It's time for colored chalk!
What color are those?
They're red because they're non-bonding.
And the bonding are the blue in-between.
And what do we know about the volume occupied by a non-bonding pair versus a bonding pair?
It's larger.
Because they're not constrained.
So not only do we have a non-bonding pair, they're not just here.
They're sort of flopping around.
Hanging way out and there's this thing here that's almost denuded of its electrons, so it starts looking over here saying, if I can't get any action over here, what about here?
So that proton starts establishing contact with the non-bonding pair of electrons on the adjacent fluorine And this is the hydrogen bond.
The hydrogen bond is here.
The hydrogen bond is not here.
If you write this I will give you a 0.
I'll give you a 0 with a circle around it.
It's called the doughnut.
That's what you get when you write something so stupid.
This is not the hydrogen bond.
This is the hydrogen bond.
OK?
So it works.
It works.
Now let's see the effect of it.
Let's see the effect.
Alright so here's a cartoon showing that in water the hydrogen-oxygen spacing is about 1 angstrom.
And the hydrogen-oxygen spacing in adjacent water molecules can be less than 2 angstroms.
So there's certainly a difference.
I mean it can't be the same.
If it were the same it would be a covalent primary bond.
You'd have a network.
Now this is interesting here because this shows the values that are given on your periodic table, which were obtained without the use the average valance electron energies.
These trends are correct.
But look at this one.
They've got chlorine up at 3.16.
And all of these have been revised.
Now on a test, just use what you've been given on the periodic table.
But I want you to understand how this is rationalized with better data coming from photoelectron spectroscopy.
So now I want to show you the implications of hydrogen bonding.
The implications.
So I've got 4 homologous series.
So they're all element plus hydrogen.
So let's start with the group 14.
That's shown here in purple.
And what do we have?
All of these, we'll start with methane.
CH4, so that's a central atom.
1, 2, 3 4.
They're all tetrahedral.
Hydrogens at the corners.
Non-polar.
And no hydrogen bonding capability.
So one methane bonds to another methane by weak van der Waals forces.
That's how it does it.
It doesn't matter if I substitute the carbon with silicon or germanium or tin.
I can put SnH4.
And how does SnH4 bond to another SnH4?
It's just by London Dispersion Forces.
That's all that's operative here.
And so we have, what makes sense here, is that the heavier, more massive-- no, the ones that have greater polarizability have a higher boiling point.
Because their van der Waals forces are stronger.
That means you have to go to a higher temperature to achieve disruption.
Same temperature, weak van der Waals force: gas.
Same temperature, strong van der Waals force: liquid or solid.
And you see the boiling point here.
Monotonic from the lightest to the heaviest.
Now let's go to the next one.
Let's go to the green line, group 15.
Well group 15, what's that look like?
Let's look at the structure of group 15.
Group 15 is-- ammonia is one of them.
So we can look at the structure of ammonia.
And if we use VSEPR we'll end up with something that looks like this.
I'll go through the whole thing.
You're going to end up with three bonds like this.
It's a tetrahedral skeleton with a lone pair.
Ah, colored chalk.
So now, what happens?
I can't say anything about this.
Why can't I say anything about whether this is a solid, liquid or a gas?
It's the only one there.
I have got to put another one.
So put another one up here.
N. H. H. H. Same gambit with the hydrogen fluoride.
This hydrogen sees this lone pair and establishes a hydrogen bond.
And that adds to what otherwise would have been a dipole-dipole.
This has a net dipole along moment, agreed?
There's a dipole-dipole moment here.
But this bond is even stronger.
The hydrogen bond is even stronger than dipole-dipole interactions.
So now let's look at the graph up here.
So what about in phosphene, PH3?
No phosphorous isn't electronegative enough.
So in phosphene, in arsine and in stibine we don't have hydrogen bonding.
So you see this series in the group 4 here?
It goes monotonic from the heaviest element down to the lightest element.
But here, heaviest, less heavy, less heavy, and the lightest element that should be down here is up here.
Why is the lightest compound not down here?
Because of the addition of hydrogen bond.
So ammonia is off the line.
This line shows the trend based on dipole-dipole interactions only.
And you can see the difference.
See if you have van der Waals forces alone, versus dipole-dipole, dipole-dipole are stronger.
So SnH4 has a lower boiling point than SbH3.
I can't predict this but I could ask you, if I gave you this data, I'd say, can you explain this to me?
Let's keep going.
Let's go to group 16.
So, telluride, selenide, suphide, the oxide, H2O, water.
It should have a boiling point of minus 100 centigrade were it not for hydrogen bonding.
How does water work?
Again, SP3 hybridization.
Oxygen, 1, 2, 3, 4.
Two lone pairs.
1, 2, H, H. And now what happens?
I bring another water molecule over here and hydrogen bonding.
And that hydrogen bond raises the temperature, raises the requirement for thermal disruption and moves the boiling point of water up to 100 degrees celsius.
It it weren't for this we wouldn't have this conversation.
Because we wouldn't have evolved as a species capable of conducting business at room temperature if water boiled at minus 100 celsius.
Hydrogen bonding is critical.
It's absolutely critical.
So now you know.
And this isn't just some little bit of pedantry for a professor.
This is very important.
Because we're going to learn later that, when it comes to biochemistry, we will appreciate that most biochemicals are made of carbon, oxygen, nitrogen and hydrogen.
And that means you can have hydrogen bonds here.
And you can have hydrogen bonds here.
Hydrogen bonding is critical to life.
So this is a very important thing to know about.
OK we have a minute or two.
So you can see polarizability increases here but hydrogen bonding operative here.
That's explains the mystery.
All right we're going to jump over this because we are out of time.
And so I will simply show you a few pictures.
Pictures!
So this is a conference in Copenhagen in June of 1936.
And there's Fritz London.
But look at who else is at the conference.
Niels Bohr, Wolfgang Pauli, Werner Heisenberg, Max Born.
Remember the Born Exponent?
You better remember it for Wednesday.
This is Lise Meitner, you haven't met her yet.
We'll get to her.
This is Walter Stern.
Stern-Gerlach.
And this is James Franck.
That's just the first two rows!
That's quite a conference.
Here's a picture of Fritz London sitting on a bench in Berlin with Erwin Schrodinger.
Schrodinger is brooding; he's thinking.
Fritz London is smiling.
You know why?
Because he's figured it out.
He's figured out.
It's time-fluctuating dipole, but he's not going to tell Schrodinger.
He's going to say, you have to read about it.
But you have to go to the library.
Because if you don't read the primary sources, if you go to Wikipedia, you won't find this.
Because this is not going to be in Wikipedia in 1913.
OK, last thing-- hold on, hold on, hey wait a minute!
Where are you going?
We're not done yet.
Did the professor say class is dismissed?
No.
So here's a biography of Fritz London, which I would recommend if you have a few minutes and you'd like to unwind.
Go out, sit in the sun and read something like this.
It tells the story of how he came up with these ideas.
The rise of fascism in the '30s.
He comes to the United States, takes a teaching position at Duke University.
All of the people that he met along the way.
All these people that we study, all this stuff, it's all there from his perspective.
And the perspective of his biographer.
Plus the pictures.
So anyway, primary sources.
Go read the primary sources.
All right, class dismissed.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare to continue to offer high-quality educational resources for free.
To make a donation or view additional material from hundreds of MIT courses, visit ocw.mit.edu
OK. Folks, I may not look like Don Sadoway, but for today, I am Don Sadoway.
So let's-- I guess you can hear me pretty well, right?
OK.
Professor Sadoway is in a faraway, terrible place.
It's called Vienna.
Yeah, I know.
We're here.
What can I say?
And he'll be back next Tuesday.
And so my name is Ron Ballinger, and I'm taking his place.
Put a muffler on that.
OK, great.
I'm sure you're all anxious, before we get started, to see that.
Right.
The average-- it was about 66, which is about 10 points lower than the last five or 10 quiz ones.
So it's a little bit lower.
It was a little bit harder test.
I'm sure Professor Sadoway will mention this when he comes back, but for those of you who are below the 50 mark, it's time to do something about it, and when I say do something about it, I'm sure your recitation instructors will be very happy to help out in any way possible, but there are other options, not the least of which is a tutor.
And so if you think you need one and you need to be kind of ruthless about examining yourself-- if you think you need one, go down and see Hilary and there are tutors which are available for individual instruction.
So I think it's a great time to take it take advantage of that because I can guarantee you that test number two will not be any easier.
In fact, it'll be considerably harder because it'll be pretty much new material.
OK.
So that's enough for that.
Last day, we were-- I wasn't here.
I was in Washington.
They're actually trying to build what's called an exoflop computer.
Does anybody know what exo means?
10 to the 18th.
10 to the 18th floating point operations per second.
Why do they need that?
Because when you go beyond 3091 then starting modeling these atoms-- especially F-block atoms-- you need a supercomputer that large and to do one run on a petaflop machine takes 100,000 CPUs 10 hours for one atom.
So solving the Schrodinger equation's a bit tricky.
OK.
Remember last time, we talked about metallic bonding and we're sort of sneaking up on it.
Remember, Paul Drude-- well, let's look up here.
These are the characteristics of metallic solids.
They have high electrical conductivity.
They have high thermal conductivity.
They shine and they have ductility.
So we're going to deal with the first three today, and later on in the course, we'll talk about the ductility part.
But recall that Paul Drude modeled the solid as a series of cations, which amounted to the nucleus plus the inner core electrons and then allowed the electrons-- the remaining, the outer shell electrons, the valence electrons, to float around, called it a free electron gas.
And that model explained the temperature dependence of heat capacity, but it was not so good when it came to electrical conductivity.
It explained the fact that you get electrical conductivity, but not the fact that some materials are conductors and some materials are insulators.
So we need to deal with that and for that, we need to talk to two sets of folks.
The first one is Felix Bloch and he was a 1928-- he was a-- he got his PhD under Heisenberg.
Boy, it would've been nice back in those days, working for those great folks.
And he applied quantum mechanics to solids.
He said, OK, let's consider that in a solid, the atoms are arranged in an array.
What's an array?
Well, it's actually called a crystal.
It's an ordered array of atoms.
We're going to say a lot more about that as we go along, but-- and then he said, well, then let's apply the Schrodinger equation to this system.
Now it's a big difference between the Schrodinger equation for a single atom in a gas and multiple atoms in a solid.
It's a different set of boundary conditions, different set of conditions all together.
And when you do that, you get a set of solutions for the valence electrons which is, as you might expect, it's periodic.
It's periodic and it invokes wave-like properties of the electron and you end up with a set of values of the wavelengths for the electron that are such that it allows mobility, which is, after all, what we're after.
These electrons got to move through the solid if we're going to have conductivity.
And this is an example where classical physics wouldn't allow that.
So you can't get this kind of behavior with classical physics.
So that's one piece.
And the second group, two guys.
Walter Heitler and Fritz London.
We know Fritz London from London dispersion force-- the same guy.
These guys were very, very productive.
And he did a post-doc-- that's another word for slave labor, by the way.
Folks will know that here.
For guess who?
Well, he did it for Schrodinger.
Interesting story, I guess.
These guys showed up for their slave labor at Schrodinger's lab, only to discover that Schrodinger had taken a position at a different university.
So these guys showed up and he said, goodbye.
Anyway, that's a bummer.
OK. And they sat down and they said, well, OK. Let's see if we can go at this from an energy point of view as opposed from the quantum mechanical point of view and let's see if we can apply LCAOMO-- now don't break out in hives because of the test-- to a solid, to large aggregates.
Large what?
Of atoms.
How big is large?
Well, I don't know.
Dream up a number-- say, 10 to the 23rd.
Lots of atoms.
And let's see what happens.
Well, we've already done a little of this in the past.
We looked at the energetics of the stability of hydrogen or the stability of helium.
So we can take that and we can kind of add up and we'll see what happens.
Well, remember, we had atom A and now let's only deal with the valence electrons.
And so this would be 0 and there's another atom A-- 0-- and this would be A2 and we've been through this before.
Let's just take-- we know that we get splitting and we get a sigma bonding orbital and a sigma star anti-bonding orbital.
Well, let's start adding some atoms here, additional atoms.
What do we get?
Well, let's just try for A sub N-- lots of atoms.
What do we get?
We end up with lots of states.
Remember, we have to have conservation of states.
So for every atom we add-- let's say this is copper, for example, which has an s1, one s. It's an odd number of electrons.
So every time I add a copper to this, I add extra states.
And then what do I do?
I start filling them using Aufbau, just like we've done in the past.
Well, if we keep going, and by the way, not to scale, it starts looking like a whole bunch of states.
And this energy level here, these energies, what?
We know for hydrogen, this is what?
Minus 13.6 electron volts.
So what are we dealing with here in terms of state differences?
Well, let's take 10 to the 23rd atoms and let's see if we can calculate energy.
Well, let's take for copper-- the molar volume of copper is what?
7.11 centimeter cube for mole.
All right.
And that's N to the 23rd-- actually, 6.02 times 10 to the 23rd atoms.
And let's just ask ourselves, what's the sort of range here?
Well, we know it's 13.6 here.
We know that's 0.
We're all friends.
So let's call it 10 ev, just for grins.
Not a bad number.
And let's ask ourselves, well, if we've got 10 to the 23rd atoms and 10 ev-- let's take 10 ev over 10 to the 23rd and you get what?
Well, 10 to the minus 22 ev per state.
That's small.
That's really, really, really, really small and if you convert that to joules, you end up with 10 to the minus 41 joules.
That's really small.
So what are we saying?
We're saying that this organization here-- when we and put a lot of atoms in there, we end up with what amounts to a band.
A band of states.
And what do we do?
We populate this band just like we did using the Aufbau principle, but what does it really look like?
Well, in the case of copper, you remember, copper has a 1s electron.
Copper is 3d10s1.
If we start filling start this band, we're going to get-- and so we start filling and we get up here and we find that we end up with a half-filled band, because there are states that are not occupied.
But remember, the distance between this guy and this guy, an occupied and unoccupied state, is only 10 to the minus 41 joules.
So that's pretty small.
10 to the minus 22 electron volts.
To give you an example, 0.025 electron volts, which is, if you want to convert that to temperature, that's about 300 degrees Kelvin.
So very, very small energy.
So that means if I were to take and if I were to apply a potential here, then what happens?
Well, I add a little energy-- and I don't have to add much energy-- and it's very easy for me to promote one or more of these electrons up into the above here and then I can get migration.
So the endpoint is that if we apply a potential, we end up with conductivity, which is what we were after.
Moreover, we can say a few more things.
If I shine-- now this is a sort of mixed metaphor here.
This is an energy diagram and this some plates on an electrode so be a little bit careful.
We shine photons on this thing.
What happens?
What's the energy of the photons, of light?
Between 200 or 400 and 700 nanometers.
It's about one electron volt.
So there's plenty of electron volts here.
I'm going to-- with a metal, not only will I have conductivity, but there'll be enough energy here to promote electrons and those electrons will move around and we'll end up with re-emission and we end up with opaqueness.
In other words, we absorb light and readmit it and so we end up with, in the case of a metal, luster.
So there's a couple of things.
So if we recall back here-- we're talking about high electrical conductivity.
We had-- we need to say a lot more about that, but we're OK here.
But Drude, it was OK as well.
High thermal conductivity, Drude did that.
That was fine.
Luster.
OK, So it works for copper, seems to be OK.
But what about another one?
Like, say, beryllium.
What's with beryllium?
Well, beryllium is 2s2s2.
So now let's draw the thing for beryllium.
We have beryllium.
We have-- must be a 2s and now it's 2p and we populate this.
And now we want to add a beryllium-- n beryllium atoms, all right?
What's happening?
well, it'll be a little bit.
We got this band that we've talked about earlier.
Now we go to fill it.
So we're filling these guys and we fill it, and lo and behold, we find out that we fill it all the way to the top and so we're screwed.
We have no conductivity.
What I can imagine now is there's 100 computers in here.
50 of them have Skype.
Guess what's going to happen?
By the end of the day, there'll be 50 things up there.
So we know beryllium's metal and it has conductivity.
So what's the deal?
Well, it turns out that while beryllium has the 2s band full, the 2p orbitals still are there.
They're still there and so there's going to be a band unfilled.
There's going to be a band for the 2p orbitals.
Well, guess what?
It turns out that the way I've drawn it, the 2p band overlaps the 2s band.
And so what that means is that I can promote into the 2p band and I can achieve my conductivity.
So that's another-- so we solved the problem of the filled s-orbitals, and we've got conductivity in both cases.
So we're OK so far, but Drude wasn't far behind.
What can we say about insulators?
It looks like we have a problem.
Well, let's take a look at another example.
And let's try carbon.
Well, we know carbon is what?
2s2p2.
And so we can go-- and we know, by the way, that most of the time carbon will hybridize and so we'll end up with 2sp3.
So n equals 2sp3 hybridized.
So we see that happen.
And we also know that carbon's not a metal and that we have strong bonds.
In the structure that we're going to talk about, they're covalent bonds and so they're very strong bonds.
So let's do this.
Carbon in the gas phase-- we have 2s and 2p.
We know that what happens is we end up with hybridization and so we ends up and we fill these guys.
So that's what we get.
Now this would be for diamond in the gas phase.
So with this hybridization, we get bands.
We start adding carbon atoms to this and what do we get?
Well, we get a band that's the 2p band.
The sp3 band is different looking than the bands for magnesium or copper or beryllium in that there's a separation between the sigma and the sigma-star anti-bonding orbitals.
So we start doing these guys up using Aufbau, and we discover that there's an energy gap e sub g between the sigma bonding orbital part of the band and the sigma-star band.
So what's happening?
Well, that's actually pretty good sized.
It's about 5.4 electron volts.
Now compare that with what?
Visible light is around one electron volt.
So now we have a very, very different situation and there's some terminology we need to have here.
This is the so-called conduction band and this is the so-called valence.
band.
Valence band and conduction band.
So in order for us to get electrical conductivity, we have to somehow promote an electron across the 5.4 electron-volt band gap.
This is the so-called band gap.
Now we know that visible light is about 1 electron volt so we know that's not going to do it.
And in fact, we might expect that even a diamond is what?
Transparent to visible light.
So if the photons come in, if there's no promotion, there's no mission, transparency to visible light.
So that's a big number.
Let's try another one.
Let's try silicon.
Now I'm going down group four, where silicon is going to be also sp3 hybridization.
So we can do that, only this time over here, carbon is what?
n equals 2.
For silicon, n equals 3 and so we can do the same thing.
Here's the 3s.
Here's the 3p.
And we hybridize and we end up with before and then we end up with a band, same kind of band structure where this is a sigma-star, star, this is e sub g, this is the valence band, conduction band.
Now we have states up here, but in this case, what do you figure?
n equals 3.
So those valence electrons are hanging out further away from the nucleus.
And we know generally that the energy drops off, the valence electron energy drops off as we get further away from the nucleus.
So you might expect that the energy of this gap would be a little bit smaller and indeed.
it is.
Very fortunate for us, it's 1.1 electron volts.
Now we're getting close.
And so this is 1/4 or even 1/5 of that for carbon and remember, visible light is on the order of one electron volt.
So one, one and half electron volts.
So what happens?
This is close enough so that we get semi-conduction.
It's called a semiconductor and we'll make a definition.
If the band gap is greater than 3 electron volts, we call it an insulator.
If the band gap is less than-- well, let's give it a range.
1 to 3 electron volts, we call it a semiconductor and of course, if the band gap is equal to 0, we call it a metal.
So that's a distinction, which is a little bit arbitrary, but pretty good.
So now, if I take a look at silicon, what do I see?
Well, if I had a piece of silicon here as opposed to diamond and I looked at it, it would be gray.
We would have color and the reason it would have color is because I get some promotion here and I get re-emission and so now I get color.
So it's not transparent to visible light.
OK. Let's take a look in general at what we-- what we're talking about is photoexcitation.
Now this is a diagram which is-- you should have in your aid sheet.
Sooner or later, you should have it in your aid sheet.
OK.
So let's draw a general band.
Here's our two bands.
This is the valence band.
This is the conduction band and we have a band gap.
And now what happens if I were a photon here?
Well, I guess it depends.
If the photon-- if e photon greater than e band gap, then what happens?
I get promotion of an electron up from the valence band to the conduction band.
OK.
So then what happens?
Well, the photon is quantized.
It's one shot.
The photon's gone.
Now I guess we need to settle one little thing.
What happens if the photon is really a lot greater than the band gap?
In another words, let's say it's a million electron volts or something like that.
Well, for purposes of 3.091, what we're going to assume is that any excess energy goes to heat.
Let's not worry about what happens for other things.
In fact, that's not far off.
Some of these LEDs that you see, they get warm.
So there is some heat.
So the photon-- once it goes away, I get-- the electron falls back down.
Well, if the electron falls back down, then what do I get?
I get another photon out, but now I've basically built myself a diode or some kind of device.
That photon-- the energy of the photon is equal to what?
It's equal to the band gap, which is equal to HC over lambda.
So I can adjust the wavelength here if I can adjust the band gap.
See where we're going with this?
Well, what happens if I-- let's say I hook this up to a resistor and I draw a current.
Well, as long as I keep the light shining on here, photons, then I can keep promoting these guys, and I can keep drawing current.
So what do I have?
I have something that I can generate electricity with light.
And so that's sort of solar-powered something, isn't it?
We can generate current.
What happens if I take-- and now instead of doing that, I hook up a battery to this thing and now I pump current in here?
I pump current in here.
In that case, I can force the electrons up here.
So I can force electrons to go in the reverse and I could make a photosensor or I could force the electrons up here and let them come back down and I know this wavelength here, I can make a light-emitting diode.
So just this one little concept here, which is very simplified, gives us the basis for photovoltaics.
And that's exactly the way it works.
OK. Let's put this in another way.
Let's plot the percent absorption.
In other words, if the energy's high enough, I absorb-- versus wavelength this way and since energy is inversely proportional to wavelength, we have energy going this way.
Let's put some numbers in here.
Let's say this is 400, this is 700 and this is Professor Sadoway's dreaded nanometers.
So this is visible light and let's put carbon on there.
Well, if you do the calculation, convert iy, it turns out that for carbon, with this band gap, you end up with behavior where if the energy is above the band gap, I get-- well, let's call this 100.
That's 0, right?
I get 100% absorption.
When I get to the band gap, below the band gap, what happens?
Well, I drop off and it goes like that.
In the case of carbon, this wavelength, right, which is, by the way, called the absorption edge-- that number comes out to be about 229.
If I try it with silicon, that number-- so this would be carbon, this would be silicon and carbon-- 1125.
All right.
So the absorption edge for silicon is 1125, which means it absorbs in the visible range and so that's the way we get the luster.
And this would be in the what?
Infrared.
And this would be in the UV region, if you wanted to-- which would be far hard UV and this would be far infrared.
OK.
This is the paper-- Heitler and London's paper.
It's in German and you can see in Zurich.
And this is their original paper.
You can go down to the library and you can get this original paper and if you know how to read German, it's-- we have it.
This is some of the original paper.
You notice the Schrodinger equation up there.
Nasty-- really nasty, but this is the calculations.
This is radius.
This is energy and you can see a point here where you get an energy minimum and that's where the bands operate.
This is another way to look at it.
This would be the bottom of the bonding orbital or the top of the anti-bonding orbitals.
This is another way to look at what we've drawn so far.
Somebody is talking.
Something you guys ought to know, this room is acoustically perfect.
If you pass gas in the back row, I'll hear it, OK?
So if you're talking up there, I'll hear it.
OK.
So you can see what's going on here.
We've drawn that and this another way to look at it.
By the way, there's a mistake in your book that doesn't make any sense.
2s3p-- it's 2p in the description for beryllium.
OK, this is another way to look at it.
This is in the archival notes.
It's one of the few pieces of the archival notes which I don't understand.
So don't worry about that.
OK.
Here's what happens when you-- where we illustrated this where you apply a potential.
What happens is you depopulate the bonding orbitals, and you populate the antibonding orbitals rules, and you get conduction.
This is another-- this is the right way to look at it.
This sort of illustrates the things we've gone through the whole time.
A metal has no band gap.
That no band gap is achieved either by half-filled set of orbitals or overlap between one or between different levels.
An insulator has a large band gap and a semiconductor has a sort of intermediate band gap.
Let's get a little-- sort of wet your whistle for your reading for next Tuesday.
I think we have Monday lecture on Tuesday.
If you go over to the reactor-- they have a reactor here at MIT-- once a week, a truck backs up and it's full of silicon logs.
These things are about 12 inches in diameter and they're about this tall and they bring them into the reactor and they're irradiated with neutrons.
Well, what do you think happens?
Since everybody in here knows the Periodic Table by heart, what's the next atom-- what's the next element over from silicon?
Phosphorus.
OK.
So guess what?
You take and you irradiate the silicon.
You absorb a neutron.
It adds one to z, doesn't it?
And it becomes phosphorus.
Well, phosphorus has one more electron than silicon.
So I have implanted phosphorus into silicon by transmutation.
And there's one more electron that's in there.
Now where's it come from?
I don't know.
The electron bank, right?
But what do you figure the energy of that guy is?
Well, it's a lot higher than the electrons in silicon.
And so guess what?
That electron, we'll find out, doesn't reside down here.
It resides up here or very close to that.
So that's what's called-- and we'll talk about it next time-- it's called doping, only now we're doping it in a special kind of way.
OK. We've got-- let's keep going.
I want to get this one up there.
There we go.
OK. Now-- so far we've talked about photons.
They're a one shot deal.
What about thermal excitation?
Well, we can make our-- the same drawing we have in the past.
We got our material here where we have a valence band, conduction band, and now we put-- we add thermal energy.
By the way, what's the one electron volt in temperature?
Just to give you a feel-- one electron volt is 11,600 degrees Kelvin if you convert that to temperature.
So one electron volt seems like a small number, but in temperature, is pretty warm.
OK.
So now the thermal energy is what?
It's constant.
Unlike the photons, which are one off, right?
So it's constant.
So what happens?
Now I take an electron from the valence band and I promote it up here.
So it looks sort of like-- so far up there with photons.
But there's one critical difference.
It stays up there.
It stays up there because the energy is constant.
So what does that mean?
Well, it leaves behind a broken bond.
And that broken bond has a lot of energy, and in fact, it doesn't like to stick around.
So given a chance, it'll move.
It's like a hot potato and so that broken bond is called a hole.
We're not too imaginative.
It's a hole in the lattice.
But it has high energy.
So what's the deal with this hole?
Well, this is-- a hole is a 0 in a land of minus 1.
So what does it mean?
It's actually net positive.
So now in the case of thermal excitation, I end up with an electron in the conduction band and a hole in the valence band.
So I end up with an electron in the conduction band and a hole and the electrical engineers call this p. They're not very imaginative either-- positive in the valence band.
So we get 241.
We get two charge carriers for every event.
That, we get one.
This, we get two charge carriers for every event.
So now we have what?
Let's ask ourselves, what's the-- can we do a little math on this?
And the broken bond, by the way, is a hole and so the number of electrons is also equal to the number of holes because it's a one for one, and in the electrical engineering world, they say n equals p. And so we can now go back and say something about this conductivity.
We can do some calculations here and we can calculate the conductivity due to the electrical connectivity, and it's really the sum over i of n of i, which would be the number of the population of carriers times-- that's the value of the charge on the carrier-- times something called the mobility.
OK, so what's the mobility?
Well, you could imagine that these electrons have to move back and forth in the lattice and in a metal versus a covalence solid or something like that, it might be a little bit different.
The resistance might be different and so-- in fact, it is.
mu i is equal to the velocity of the charge carrier divided by the electric field.
OK.
So we're almost there.
Now we need to-- and we can simplify this because of the n equals p business, and we can say that the electrical conductivity is equal to what?
It's equal to n sub e times e, which is the electronic charge times mu sub e plus n sub h, which would be the holes, the electronic charge, times mu sub h. But since n is equal to p, we end up with n sub e times the electronic charge times mu sub e plus u sub h. OK. And so that-- what we really need to get is this.
Well, we don't have time to go through that, but from a quantum mechanical calculation, we can get that.
n sub e is equal to what?
It's equal to some constant times T to 3/2 times the exponent of minus E sub g over 2k sub B times T. So what is all this stuff?
Well, this is a constant.
This is the temperature.
This is the band gap.
So it's a representation of the binding.
What's down here?
Well, k sub B is Boltzmann's constant.
This is absolute temperature and so this represents the disruptive force.
So it's a balance between how tightly they're bound and how much energy I've got to do this.
Well, let's-- we've got one minute.
Let's do a quick calculations for silicon, just to close the loop here.
For silicon at room temperature, that number comes out 1.3 times 10 to the 10th per centimeter cubed.
Say well, that's billions.
That's a big deal.
What about copper at room temperature?
Well, that number comes out about 8.5 times 10 to the 22.
Now we're talking big numbers.
So there's a 10 to the 12th difference between these two Well, let's take a quick look before we finish up.
OK.
This is the band gap as a function of position in the Periodic Table.
Now you guys ought to be able to rationalize this.
Lead is a metal, bigger atoms you can see.
10 actually comes in two flavors.
Gray and white and one of them-- I think it's white-- actually forms covalent bonds and they use tin for night vision.
Think about where that band gap is.
But here's the punch line.
We're trying to rationalize this large difference in conductivity.
Well, there's copper up there at 10 to the 7th, there silicon at 10 to the minus 4 and ten to the minus 4, 10 to the minus-- about 10 to the 12th or thereabouts-- and so, lo and behold, this-- we were able to rationalize with these fairly simple models, the range of conductivities that go all the way from metals to semiconductors and even diamond.
Look at diamond-- 10 to the minus 11.
You can rationalize that the band gap-- where's the band gap?
5.4 electron volts.
OK, we're out of time.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so this takes us into the paper by Heitler and London, where, by using the concept of linear combination of atomic orbitals into molecular orbitals for a large number of atoms, instead of splitting one plus one atoms, making a bond into two energy levels of bonding and AN antibonding, we take a large number of atoms, bring them together, and we get a large number of bonding orbitals and a large number of antibonding orbitals.
And they have to lie within a zone of about 5 to 10 electron volts, which means if you put Avogadro's number of these things together, you're going to end up with an energy difference between successive energy levels on the order of about 10 to the minus 20 electron volts, or 10 to the minus 40 joules.
And for all intents and purposes, that is a continuum, and hence, the term band.
And now we start populating this with electrons, and each one of these is a separate state.
And electrons go in two by two, just like Noah's Ark, all the way up-- either halfway up or all the way up.
But now the energy difference is very, very tiny.
And a second instance, we looked at insulators where-- for example, in diamond-- where the bonds are really strong.
So that means if the atomic orbital is roughly at the middle here, the depression to make the bonding orbital is very, very great, and very negative.
And therefore, the elevation to the antibonding orbital is great and relatively positive.
And when we start putting Avogadro's number of these things together, we still end up with the 10 to the minus 40 joules.
But when we get to the top of the bonding orbitals, we have still got 3 or 5 or 5 electron volts to get to the bottom of the antibonding orbitals, hence the formation of the band gap with an energy of-- as I mentioned.
So up here, we have the conduction band where electrons are free to move about.
And this is derived from the antibonding orbitals.
And down here, we have the valence band, which consists of the bonding orbitals derived from sigma and pi.
And then the semiconductor is sort of like an insulator-- as they say in California, it's like an insulator-- but the band gap isn't quite so severe.
It's down on the order of 1 to 3 electron volts, and that gives it unique properties.
Why?
What's magical about 1 to 3 electron volts?
Number one, visible light.
Visible light is 2 to 3 electron volts.
So visible light shines right through an insulator, because visible light has an energy of about 2 to 3 electron volts.
If that's the energy of 2 to 3 electron volts, and this is 6, it zooms right through.
But if it's 2 electron volts and this is 1 electron volt, we're going to get thermal excitation.
I know Professor Ballinger talked about thermal excitation the last day and what happens, and it's very reminiscent of excitation in the case of the Bohr model.
Photoexcitation across the band gap.
Incident photon with energy greater than the energy of the band gap.
And it's the same drill as you've seen before.
That's why those early lessons on-- photon comes in or electron comes in, strikes the Bohr atom, electron rises up, momentarily sits there, falls back down, photon emitted.
It's the same set of ideas.
Only instead of moving from n equals 1 to n equals 2 in the Bohr atom, you're moving from the valence band to the conduction band, sitting in the conduction band momentarily, you fall back down, and you get a photoemission.
So that was covered last day.
And I know towards the end of the lesson, he started talking about thermal excitation.
So let's look at that in a little more detail.
Thermal excitation.
So thermal excitation of what?
Thermal excitation of electrons across the band gap.
So this is matter energy interaction, only in this case, it's thermally inspired.
So let's draw our cartoon again.
Up here, we have the conduction band, and below we have the valence band.
So in the conduction band we have free electrons.
In the valence band is where all the bonds live.
And we've got an energy gap here of some value.
Could be semiconductor, could be insulator.
In this case, we're going to use visible light.
And we've gone through the analysis with incident photon and the emission.
In this case, I want to look at thermal excitation.
So with thermal excitation, I start with an electron down here in a bond.
This electron lives in a bond.
And with thermal excitation, the electron jumps out of the bond and up into the conduction band.
So I'm going to designate the electron in the conduction band with an e minus and a circle around it.
And it leaves behind a broken bond.
As you know, every bond has two electrons in it.
Well, one of the electrons left the bond, leaving a hole.
Little h with a plus sign.
This is a broken bond, and a broken bond is highly unstable and very mobile.
If I put a potential across here.
If I put this material-- and I'm doing a mixed metaphor-- this is an energy level diagram.
But now I'm going to put a couple of plates across an energy level diagram.
But you're smart.
You can understand this.
So this is not energy.
Now I'm going to make the right electrode negative and I'm going to make the left electrode positive.
What would happen?
Well, the electrons in the conduction band are going to be attracted to the positive electrode or repelled by the negative electrode, but the complementary situation occurs in the valence band.
This broken bond is like a hot potato, and it has an effective charge.
This is a zero in a land of minus one.
A broken bond is a zero in a land of minus one.
So therefore, if I make the right side negative, this hole is going to be drawn.
So I have two forms of electronic conduction.
I have electrons moving to the left in the conduction band and I have holes moving to the right in the valence band.
So for every thermal excitation I get two carriers.
Every excitation of electron from valence band to conduction band generates two carriers.
Carriers of what?
Water?
Carriers of electrical charge.
Two carriers of electrical charge.
And what are they?
The electron in the conduction band and the hole-- little h with a plus sign-- in the valence band.
So that's how thermal excitation works.
And you can see that the numbers have to be equal.
So we write, therefore, that the populations are equal.
The number of electrons in the conduction band equals the number of holes in the valence band.
Now electrical engineers have a similar expression, but they don't care about numbers and electrons and holes, they care about the functionality.
And the electron is a carrier of negative charge and the hole is a carrier of positive charge.
So the electrical engineering books will write, thermal excitation gives rise to pair generation.
Pair of what?
Pair of charge carriers.
And the population of negative charge carriers equals the population of positive charge carriers, which in chemistry is the number of electrons equals the number of holes, or the number of broken bonds.
So that's the way it works, but there's a conundrum here.
Thermal excitation is based upon the thermal energy, and we know that the average thermal energy in an environment is roughly given-- I mean there are fancier calculations-- but to a first order, I can approximate it as the product of the Boltzmann constant and absolute temperature.
So at room temperature-- room temperature, I like to say, about 300 Kelvin.
300 Kelvin is 27 Celsius.
That's kind of a warm room, but 300 is a nice number.
All right?
So if you take 300 Kelvin, you get on the order of about 1/40 of an electron volt.
1/40 of an electron volt, which is a lot less than one electron volt, which is roughly the band gap of a semiconductor.
So how is it that we get any thermal excitation of data indicate that there is some thermal excitation even at room temperature?
So the question is, if the average thermal energy is a fortieth of electron volt according to this-- and this is correct-- and the band gap is one eV, then it should be no thermal excitation.
But we know what happens.
So, one eV, which I'm going to say is Eg.
All right?
So how do we get any thermal excitation under these conditions?
So there's an answer to this.
And for this we go to merry old England and James Clerk Maxwell, that you know or will come to know in 8.02 as the annunciator of Maxwell's equations of electricity and magnetism.
He also thought about physical chemistry, and in particular he thought about the physical chemistry of gases.
And what did he teach us about gases?
He said, if you look here in this room, you see nitrogen and oxygen and all the other constituents of air.
But to a first approximation, we have four parts nitrogen, one part oxygen.
The room is roughly 300 Kelvin.
What Maxwell taught us is that velocities of the gas species zooming around in this room are not all equal.
Some of the gas species are moving around as though they're at 500 Kelvin.
And some of them are moving around very slowly as though they're at 100 Kelvin.
On average, it's 300 Kelvin, because we can feel it as 300 Kelvin.
But some of them are zooming around.
Some are really lazy and they're just moving along really slowly.
And furthermore, he was quantitative about it.
He said, I'm going to tell you what the distribution of energies is.
So let's get that on the board for starters.
That in any ensemble of gas species, could be gas atoms or it could be gas molecules or gas compounds, in any ensemble of gas species, that all molecules-- because you can always say the atom is a degenerate form-- all molecules do not have the same velocity.
And furthermore, he gave us the distribution.
He calculated the distribution of velocities.
So that was about 1859.
Forgive me here.
Velocities.
OK. And then Boltzmann came along in 1872.
And Boltzmann generalized this idea.
He made the link between gas velocity and temperature.
So, he took the idea and he said, if the gas molecules move at different velocities, that means they have different energies.
And why do we call this the Boltzmann constant?
Because Boltzmann made the link between particle energy and temperature.
So he said there's a variation of instant temperature.
All right?
So Boltzmann generalized Maxwell's idea to energy and then ultimately temperature.
And so I'm going to show you, instead of the Maxwell distribution, I'm going to give you the Maxwell-Boltzmann distribution.
But before I do, I want to make sure you understand the math and appreciate what I'm going to plot for you.
So we just had a test, right?
In fact, we can go back to the slide.
All right.
So what do we have here?
I could plot on this axis, the abscissa, the grade-- g-- and on the ordinate, I'll plot the number of people who have any given grade-- n of g. Now the class av was 66, all right?
This is going to go from 0 to 100.
Now what we see here is-- and by the way, I'm going to normalize it.
Instead of dealing with 463 or whatever, I'm going to divide by the total class.
So the number is going to go from 0 to 1.
So this a normalized distribution.
What I'm teaching you applies all through science.
As I told you, this is the most important class.
Not because I'm teaching, but because the subject matter is general, just as Boltzmann did.
So let's look at the normalized distribution.
Now the class av was 66.
Now we didn't have everybody getting 66.
That was not the distribution.
That's one solution to the problem, isn't it?
Instead, what we got was we got that.
Now one possibility would be this one, the bell-shaped curve.
Goes like this.
That's bell-shaped.
This is Gaussian.
This is a Gaussian distribution, and its functionality is y equals e to the minus x minus x average, quantity squared.
Now, to have a normal distribution, you have to have a class of normal people, so that didn't happen on this test.
That's a joke.
God, you're so serious.
Have you no fun in your lives?
Have you ever laughed?
[LAUGHTER]
Overachievers, you.
OK.
So we're not going to have a Gaussian distribution.
We're going to have that one there.
It's kind of a skewed Gaussian with a long tail.
All right, so that's the one thing.
But what Boltzmann gave us, instead of the number of atoms of a given grade, he plotted it as a function of energy.
So instead, the Maxwell-Boltzmann distribution has energy as the ordinate instead of the grade, and up here, the number of atoms-- or more importantly-- the atom fraction.
The fraction of atoms in the distribution that have that energy.
And it doesn't look like a normal distribution.
It looks sort of off-normal.
Skewed, like this.
It rises to a peak and it has a long tail that moves off to the right.
So this is the Maxwell-Boltzmann distribution.
It's not y equals x minus x bar squared.
So this is Maxwell-Boltzmann distribution of energies.
And furthermore, what we see here is that the area under this line has to equal 1.
If you integrate the fraction from 0 to infinity, because this asymptotically goes out to infinity, the integral under this line is 1.
OK. Area equals 1, which is the integral of ndE.
Now, because of the asymmetry here, clearly this is the average energy.
The average energy isn't the maximum energy.
Whereas in the Gaussian-- let's keep the Gaussian over here, just for grins and chuckles-- so the Gaussian looks like this.
It's symmetric.
And this is the n of g max and also a g average, whereas here, the average is a little bit to the right.
A little bit higher.
OK.
So now, if this is the distribution, and this at room temperature is 1/40 of an electron volt, and way, way up here is the band gap energy, there's a tiny but non-zero fraction of this distribution that has energy in excess of the band gap energy.
And so this tiny fraction of the total distribution has the thermal energy to allow this excitation and pair generation to occur.
And furthermore, this is a function of temperature.
This is a function of temperature.
So E average is related to temperature.
So, I'll say E average 1 at T1.
And then what I'm going to do is I'm going to heat this to a higher temperature, so I'll use red chalk to indicate it's hotter.
And what happens to the distribution under an increase in temperature?
Well, the average had better move to the right, and that's what happens.
But it doesn't just move to the right, it skews a little bit and broadens.
So in point of fact, at a higher temperature, it looks like this.
OK.
So this is now E average 2 at T2 where T2 is greater than T1.
T2 is greater than T1.
OK, so that's the Maxwell-Boltzmann distribution.
But here's where the interesting stuff occurs.
So I'm going to blow up this high energy end of the distribution.
So again, this is going to be n of E normalized, and this is E. And what we're going to find here is that at T1-- this is the curve at T1, and the area under the line is shown here-- and then-- not to scale-- we're going to put a break in this.
And I'll show you how much of a break is needed.
And then, at T2 I'm going to have this come down.
This is T2.
And the area under the line T2 captures the T1 plus all of this.
And so what we see, is that for a modest increase in temperature, this maximum shifts modestly, but the area under the line in excess of this critical energy increases radically.
And that's the powerful leveraging factor of increase in temperature.
And I've done some calculations just to show you what this does.
The functionality of the Maxwell-Boltzmann distribution is e to the minus some critical energy divided by the product of Boltzmann constant temperature.
So we can put in a value here-- eg-- what have you.
All right.
So I chose for silicon-- for silicon, I know the band gap energy is 1.1 eV-- and I have it at room temperature-- when I go through the calculation-- the fraction underneath that line is 10 to the minus 19.
One part in 10 to the minus 19.
This is the area.
Area under the curve is greater than the band gap energy of 1.1 eV.
So then I said, let's go up to the melting point.
I'm not going to melt the silicon, I'm just going to bring it up to its melting point but not melt it.
So it's like ice cubes at 0 Celsius.
It's still solid.
OK?
The melting point of silicon, you can look in your Periodic Table, it's over 1400 degrees C. And this is 10 to the minus 4.
10 to the minus 4.
Area under curve: da da da da.
So that means that the red area-- A2, the red.
Actually, let's use red chalk so that everybody is in tune.
A2 to A1 is equal to 10 to the 15, whereas T2 over T1-- well, red chalk.
It's my chance to use red chalk and I'm not going to squander it.
T2 divided by T1-- it's about 1700 Kelvin divided by 300 Kelvin-- is about 6.
So for a change in temperature by a factor of 6, I get a change in population of 10 to the 15.
So you can see how modest increases in temperature have a huge impact on thermal excitation.
So now you say, if we wanted to make a device, we can't heat our computers up to 1600 degrees, 1500 degrees C in order to get an adequate number of charge carriers.
There has to be another way to get charge carriers.
I'm going to show you a third way that's called chemical promotion.
So now I want to talk about charge carrier generation.
Charge carrier generation by change of chemistry, and it's going to be specifically by the introduction of impurities.
Charge carrier generation by introduction of impurities.
You might say, gee, aren't impurities bad for a system?
Well, I want to tell you, there are two kinds of impurities in this world.
There are bad impurities which we call contaminants, and that's what you're commonly taught to think about.
That's bad.
But there are good impurities, and these are called dopants.
Dopants are impurities that we are happy to have in our system.
And so by the introduction of impurities of value, we can lead to engineered materials.
So I'm giving you the introduction to high technology, where we're going to control the engineering properties of a material through control of chemistry.
So that's chemical composition tailored-- for specific-- for targeted values of properties.
And that's the essence of material science.
We control the chemistry.
We control the properties by controlling the chemistry.
So what I want to do is give a vivid example of this, and it's not going to be an isolated example.
It's going to be the example that pertains to all of the microelectronic devices in our world.
So that's silicon.
So let's look at how doping works in silicon.
It also applies to germanium, but we don't use much germanium in devices.
It's dominantly silicon.
So what we're going to do is dope.
The gambit here is to dope-- that's introduce an impurity-- with aliovalent impurity.
What do I mean by aliovalent?
You know the word alias?
It's another name.
Alio in Latin means other.
So we're going to dope silicon with something that has a different valence from that of silicon.
Can't before.
So let's look at the simplest example.
And I'm going to put phosphorus into silicon.
I'm going to dope phosphorus into silicon, and phosphorus is Group 5, or more properly, IUPAC calls it Group 15.
So we're going to put Group 15 element into silicon, which is Group 14 element.
So we call phosphorus a supervalent dopant, meaning it has a valence higher than that of the silicon.
So it's a supervalent dopant.
Supervalent dopant.
So now let's look at how that supervalent dopant works.
I'm going to show you some silicon here, and I'm going to give you the silicon.
This is sp3 hybridized.
it's sitting here as a single crystal.
Here are three silicons.
So it's a single crystal.
All these silicons on the bottom row line up.
All the silicons in the center line up.
And there are silicons at the top and they all line up.
So that if I look at it on the edge, it's a perfectly, atomically smooth system.
And I have one of these.
I know Professor Ballinger made reference to it last day.
This is last-generation technology.
It's only eight-inch diameter.
But this is an eight-inch diameter silicon wafer.
So this started as a giant salami, eight inches in diameter-- probably two meters long.
Single crystal.
Every atom here is adjacent to the next atom according to the rules of atomic arrangement.
And when this thing is cut and polished, it's flat to one atom.
It is atomically flat.
And it makes a fantastic shaving mirror.
Actually Dave, can you cut to this?
Yeah, here we go.
Here's the penny.
And this is silicon here.
It's going to drive this thing nuts because it can't focus on this thing, because it's so-- there we are.
So this was a part of a giant salami, and now it's been cut.
And obviously you can't cut it one-atom thick.
It's some submillimeter thickness, because you need to have some mechanical strength to it.
But the surface is absolutely flat to one atom.
All right.
And that's what we're going to dope into.
We're going to dope into this thing, and I'll show you how we're going to dope it later.
Actually, why don't we cut to that.
Here's something that's already been doped and made into devices.
So there's the plain old silicon, which you're having a hard time seeing anything except a little bit of reflection.
And now here's the thing that's been processed, so let's zoom.
Zoom.
OK.
So now you can see all the features.
So these features involved gas phase reaction to introduce dopant atoms into the silicon lattice.
And then we're going to chop these little things up and put them in your cell phones and in your laptops and whatever other devices are out there.
OK.
So this is what we're talking about, and if we live long enough, we'll see one of these.
This is a Pentium chip.
OK.
So where's the chip?
The chip is underneath this piece of gold.
Where did I put my pointer?
Ah, it's over here.
All right.
So the actual silicon device is underneath this.
And all of these are this spider's web of leads.
It's trying to access all the tiny devices.
It doesn't do you any good to have a device density that's greater than your access density.
So the technology that goes into this is phenomenal.
Plus, these things all generate heat.
When you put all these tens of thousands of devices into something the size of your fingernail, you've got a toaster oven working here, and you have to get that heat out.
And this is several generations old.
What you have in your machines today is even denser than this.
And supercomputers?
Forget it.
You have to have those things so aggressively cooled or else they'll heat up and it'll be just like a light bulb, just bursting.
So that's what's going on inside.
So let's see how that happens.
Let's cut back to-- well, you can leave that up.
It's a pretty image and will probably soothe the nerves of the students.
So now I'm going to put phosphorus in here.
I'll put phosphorus in here, but where does phosphorus go?
Phosphorus goes onto a silicon site.
So it actually comes in and sits here.
Phosphorus comes and sits at a silicon site.
Now what do we know about the number of bonds that phosphorus likes to form?
It has five valence electrons.
So it forms one, two, three, four bonds, and it has a fifth electron.
And that fifth electron is sitting there somewhere.
It's sitting there somewhere in the lattice.
All right.
It's sitting somewhere in the lattice because it has no place to go.
Now in energy space, I know where it goes.
So here's the conduction band of silicon, and this is the valence band.
And this is all of the silicon host crystal, because the dopant is in a tiny, tiny amount.
We're talking parts per million.
So for all intents and purposes, it's still a silicon crystal.
It has tiny amounts of phosphorus in it.
So this is the band gap of silicon, where this is about 1.1 electron volts.
And where does this fifth electron go?
This fifth electron sits up in here in the conduction band, doesn't it?
And every time I introduce a phosphorus, I have a fifth electron that goes in the conduction band.
But I don't have to generate a hole.
I don't need to break bonds in order to put electrons in the conduction band.
Because the fifth electron is like the fifth wheel.
You know?
It goes in the conduction band.
So there's no generation of holes here.
All right.
And furthermore, I can control the conductivity.
At the end of last lecture, Professor Ballinger showed you that the conductivity of a substance is related to the number of charge carriers.
So if I want to double the number of phosphoruses, I will double the number of electrons in the conduction band, independent of temperature.
All right.
So for every phosphorus dopant atom, we get one electron in the conduction band.
We get one electron in the conduction band and no holes.
No holes in the valence band.
It's direct.
It's direct.
All right.
Now, electrical engineers would look at this and they'd say, I don't care if it's phosphorus.
They don't care about the chemistry, they care about the functionality.
What kind of carriers am I throwing into this crystal?
For an electrical engineer, there are only two kinds of carriers.
Positive carriers and negative carriers.
For every phosphorus, what's the outcome?
A negative carrier.
So this is now doped silicon.
It generated negative carriers, so this is called n-type.
This is now n-type silicon.
And its properties are governed by the population of these-- because I'm going to show you in a second that the number of these far exceeds the number that are thermally generated-- so the properties here are called extrinsic.
So now we have a crystal that is exhibiting it's extrinsic behavior, thanks to the doping with the supervalent impurity.
One more thing.
One more thing.
Something very, very cool here.
You see this electron?
It's sitting here somewhere in the-- now this is Cartesian map.
I'm looking through a scanning electron microscope.
I see all the silicons.
I get the phosphorus.
And there's an electron in here somewhere.
And this has 15, right?
This has 15 protons.
This has 14 protons.
So in a land of 14 plus, 15 plus is relatively positive, isn't it?
The phosphorus is locally positive in comparison with the rest of the silicon lattice.
And it's pinned, because it's covalently bonded.
And there's an electron sitting here with no place to go.
Do you know of a model that describes the behavior of a one-electron system around a pinned positive nucleus?
The Bohr model
Bohr model?
Just for grins and chuckles, what if we were to use the Bohr model to describe the behavior of that electron?
You can say it's crazy.
That doesn't make any sense, because that's not a one-electron system.
But what if we modeled it as though it were?
I'm going to-- just for grins and chuckles-- get a sense of what that ground state value is.
And recall that in hydrogen, the ground state-- E1, ground state of the electron-- is equal to minus k. Which is minus me to the fourth over 8 epsilon naught squared times Planck constant squared.
Which is minus 13.6 electron volts So what is the energy of that electron there if we use the Bohr model?
Well, we have to modify it a little bit, because the electron isn't moving in a vacuum.
Depending on how far this electron is from the center, it could be moving around, and there are all kinds of silicon atoms and bonds and whatnot.
So there are all kinds of stuff between the electron and the center here.
So I have to use a modified form.
I don't use the permittivity of vacuum.
I map the permittivity of vacuum into the dielectric constant.
This is the dielectric constant of silicon.
Remember there's so little phosphorus there that it doesn't alter the properties.
And then for reasons I can't go in to here, I have to change the mass of the electron into the effective mass-- we saw this when Bohr did the calculation about the lines of helium plus-- effective mass of electron in the conduction band.
So if you put both of those in-- and I've got numbers for this-- this value is 11.7 and the second one is about 1/5 of the rest mass of the electron.
So if you put all of that in, you end up with the energy of this ground state electron equal to minus k times 0.2 divided by 11.7 squared, which equals minus 0.02 electron volts.
0.02 electron volts.
So this is 2/100 of an electron volt below something.
What's it below?
It's below the conduction band.
It's right here.
This is the ground state of the electron from phosphorus.
It's the ground state of the electron from phosphorus.
E1 of electron from phosphorus.
And phosphorus, the electrical engineers call a donor.
A donor of what?
It's a donor of electrons.
So this thing here is called the donor level.
And that's the ground state.
So the electron should be sitting.
Isn't the ground state here lower than the bottom of the conduction band?
Sure it is.
So the electron really should be down in here.
There's the electron.
Now you're saying, how are you going to get conductivity?
Well, we just figured out that the energy level of the ground state of the donor-- so I'm going to call this the energy of the donor level, ground state-- is 0.02.
How does that compare with thermal energy at room temperature?
It's comparable.
So at room temperature with 1/40 of an electron volt, there's enough energy to promote this electron up into the conduction band by thermal excitation from the donor level.
Thermal excitation from the donor level.
And in fact, if we want to do something really, really-- I want to use an adverb here-- really, really cool.
If that's the ground state-- I'm going to blow this up.
So this is the bottom of the conduction band.
And now this is the ground state.
This is the donor level.
This is the donor level, which I was calling E1.
Is it just the donor level here and the conduction band, or can you see that there's an E2 and an E3 and an E4?
And I have a whole set of quantum states from the donor level right up to the conduction band.
And if I have enough energy-- if E thermal is on the order of E donor-- I get promotion of these up in here.
Now what would we say if this were the Bohr model?
You'd have an electron loose in a ground state up in here where it's free to move.
What's that process called?
Ionization.
And you know what the electrical engineers call these electrons?
They call them ionized electrons.
But they're referring to the ionization of the electron out of the donor level into the conduction band, not out of the silicon crystal.
So these are ionized electrons out of donor level.
So we've come full circle.
By the way, this is a calculation.
You know what the measured value is?
It was measured as minus 0.045 electron volts.
You might look at that and say, whoa, that's about two times this.
Hey, wait a minute.
We started at 13.6.
We went from 13.6 to this, and the real value is this, and so we are in good shape.
Now, what happens if I put another phosphorus in here?
Another phosphorus, it's also going to sit at this-- I'm going to do it over here.
If I put another phosphorus, it also has the same donor level.
And another phosphorus, it has the same donor level.
How can I justify this?
Am I not violating the Pauli exclusion principle?
How come all the phosphorus donor states are at the same level?
Or put another way, under what circumstances is this diagram accurate?
If the number of phosphorus atoms that I introduce is tiny-- parts per million-- the separation, the physical separation of this phosphorus from its nearest phosphorus neighbor is so great, that for all intents and purposes, they are at infinite separation.
And therefore, all of the phosphorus donor atoms establish a donor level at the same value.
And if you dope to very, very high levels, you see this break down.
If you go to very, very high concentrations of phosphorus dopant, you will discover what happens if these two energy levels get close enough together in real space.
They have to split.
And so you'll see all of that happening down here at the miniature level.
The last thing I wanted to do was to push the Bohr model just a little bit harder.
And so I've given you an energy, and the energy seems to make sense.
Even though you might say, this is the crazy.
It's not a one-electron system, but it gave us a reasonable number.
Let's just for interest's sake determine: what's this radius?
I wonder how far this electron is away from the donor phosphorus?
So I plug into the r1.
That's the radius of the ground state electron.
And that's the Bohr radius, right?
0.529 angstroms in elemental hydrogen, but now this has to be modified by adding the dielectric constant and the effective mass.
And the number is 30 angstroms, or 3 nanometers, if you must.
All right?
And compare that to silicon-silicon bond distance.
The silicon-silicon bond distance is 2.35 angstroms.
So clearly this electron is very far removed from the central phosphorus.
It's not as I depicted.
It has to be way over here, going way, way around.
This is really, really something.
So now you understand how you can get extrinsic properties and the behavior of the system governed by the introduction of impurities.
What I'm going to do next day, is to show you that at typical doping levels, when you dope, dope at about 10 to the minus 6-- 1 part per million-- 10 to the minus 6 phosphorus per silicon.
It turns out that the number of electrons from this operation is much greater than the number of electrons from thermal excitation, which is tiny.
And therefore, we argue that the properties are governed by the dopant atom, and so we have extrinsic behavior that we see.
OK, well that's a day's work.
David, may we go to the slides, please?
OK, so you saw all that last day, and this, this, this.
All right, there's the insulator, and that.
So I'm just doing the lecture again and fast.
You can read.
You can parse visual images.
See?
There's the n-type semiconductor.
We're going to look at p-type next day, quickly.
There's a silicon crystal.
There's the salamis-- there's the little wafers that are cut.
And there's the spectrum, the x-ray spectrum indicating you have a single crystal.
There's the first transistor-- this was a single crystal of germanium lying here on its side-- the point transistor in the fall of 1947.
Three gentleman at the Bell Labs that demonstrated the rectification.
And they won the Nobel Prize for this in 1956.
Now, you can also make semiconductors that have compounds.
So these are compound semiconductors-- Professor Ballinger showed you last day-- tin, germanium, silicon, carbon.
And look at all of these.
This is a Group 3 element with a Group 5 element.
5 plus 3 is 8.
Group 3 and 5.
This is Group 2 and 6.
2 plus 6 is 8.
It's always about octet stability.
But they make strong covalent bonds that give band gaps all over the map.
So suppose I wanted to have something like a stoplight.
Suppose I wanted a stoplight, and I wanted to make a red light.
Well, I need-- I can go backwards from 660 nanometers, and I can go e equals hc over lambda and calculate-- I need a material with 1.97 electron volts band gap, and I look on here and there's nothing that's exactly 1.97.
Well, I can mix these things.
I can mix them.
I can mix something that has 1.52 with 2.3 and get 1.97.
That's called band gap engineering.
And there's more than one mix here, so why would I choose one mix over another?
It's cheaper, or it's easier to process, or it's stable in the atmosphere.
You don't want a semiconductor that can't stand rain and snow.
And so I'm showing you here a variety of compound semiconductors, and this is the basis of things like-- the LED, the CD reader, the DVD reader is based on light that's generated from these.
What was the big deal about Blu-ray?
Why is Blu-ray so cool?
Because the original reader was red.
And what's the wavelength of red light?
Red light is around 600 nanometers.
And what's the wavelength of blue light?
It's down around 300 nanometers.
So all other things being equal, my stylus is half the size, which means for a given area, I can get double the density of information without changing the size of the disc.
But it wasn't trivial to find a blue laser.
And the person who found the right mix of compounds that could give blue in an efficient manner became a very wealthy individual.
So this is our stoplight now.
These are the materials that go into the stoplights.
No more do you have that giant lens with a 150-watt incandescent bulb.
Now you have the array of these LEDs that are designed to be in this band gap.
So I think at this point, we've probably run out of time.
So we'll resume the discussion tomorrow.
Same time, same place.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK, settle down.
It's time to start learning.
One announcement.
Tomorrow we'll have the weekly quiz.
In lieu of the Tuesday celebration, we're going to move to Thursday, since the holiday moved everything, compressed the week.
So last day, we were talking about doping of semiconductors, and I want to finish up that unit.
Just to refresh your memory, we looked at how we could change the behavior of the semiconductor by introducing impurity atoms.
And when the behavior of the semiconductor is dominated by the impurity atoms, we term that behavior extrinsic.
In other words, it's not the behavior of silicon itself, but it's the behavior of silicon as determined by the presence of some dopant atom.
And we looked at the special case of doping with a supervalent impurity.
And here's the energy level diagram for that situation, where we've got the valence band down here, we've got conduction band up here, separated by an energy gap.
That's all characteristic of plain vanilla silicon.
But then when we dope with the supervalent impurity, there is an extra electron.
And that is put into a donor level that lies just a tiny bit below the bottom of the conduction band.
And then, thanks to thermal excitation, which gives us an average energy of about 1/40 of an electron bolt, we get, for all intents and purposes, 100% excitation of the electronics that sit in the donor level, excitation up into the conduction band.
And this is what electrical engineers term ionization.
And I also reminded you last day that this is the donor level, and for each donor atom, there sits a donor level at the same value of energy, but these are so far apart point, owing to the dilution, the dilute concentration of the impurity, that we don't violate the Pauli Exclusion Principle.
So these are all sitting at the same level and that's why they've given temperature promotes all of them up into the conduction band, where they are mobile.
And then, just to complete the review, we call this n-type because, thanks to doping, we generate electrons in the conduction band, and electrons are negative.
Supervalent doping gives us n-type.
Then the total number of electrons in the conduction band is the sum of those that would have been generated by plain old thermal excitation.
Thermal excitation operates all the time.
So with thermal excitation, recall, we promote from the valence band up into the conduction band.
So we break one of these bonds, shoot an electron up here, and leave a hole behind.
So this generates the pair.
Hole in the valence band, and electron in the conduction band.
That's always operative, but owing to the energies involved, the band gap is on the order of one electron volt, whereas the donor level sits only about 1/50 of an electron volt below the conduction band.
So we get very, very little promotion at room temperature.
And in fact, it's something like 10 to the minus 19 is the fraction of promotion, and if you say roughly 10 to the twenty-third per cubic centimeter, that means you've got about 10 to the fourth electrons per cubic centimeter due to this thermal excitation from the valence band to the conduction band.
Normally, when you dope a semiconductor, you dope it around parts per million level.
And parts per million, that's 10 to the minus 6.
If you take 10 to the minus 6 times 10 to the 23, you get about 10 of the 17.
Now you can see, 10 of the 17 is vastly larger than 10 of the fourth.
So for all intents and purposes, once you've doped something, this contribution is essentially 0, and so the electron population, the conduction band, is that dictated by the impurity atoms, which is why we say that this is exhibiting extrinsic behavior.
The intrinsic properties are not visible, they're not palpable.
They're immeasurable.
So intrinsic is substantially less than extrinsic.
Now, the last thing I wanted to do for you before we say goodbye to this fascinating topic, which lays the groundwork for everything that we know in the modern electronic era, parenthetically, is to look at the other type of doping.
I've showed you how to make an n-type semiconductor.
How do we make a p-type semiconductor?
So in that case, what we can do is dope with a subvalent impurity.
What do I mean by subvalent?
Well, the valency of silicon is 4, so I want something less than 4.
So a good example is boron into silicon.
So boron is group 13, or 3, all right, if you look on your periodic table.
And silicon is group 14, according to the IUPAC notation.
So let's go back and take a look at how that might appear.
So I'm going to draw the silicon crystal.
Remember, this is going into a single crystal of silicon.
So silicon is sp 3 hybridized, and we have silicons everywhere in the structure.
So we're going to put silicons.
And I'm trying to show sp 3 hybridization.
So this is 3 legs each in the plane, then you've got 109 degrees here.
So that's silicon, and so on.
I get the three of them just to complete enough of the picture.
And now what we're going to do, is introduced boron.
And boron goes into the silicon lattice and covalently bonds.
It doesn't go sit in some void space in between the silicons.
It actually sits on a silicon site and substitutes for the silicon.
Now, boron is group 13.
It's got 3 valence electrons.
And so it forms bonds with 3 silicons.
And now there's a fourth silicon here, and that silicon has an electron, but the boron doesn't have a fourth electron.
Here's where it gets interesting.
The driving force to complete the picture here is so great that the system will actually pull an electron out of a silicon-silicon bond-- and so I'm going to use a different color chalk, and I'm going to indicate, just for argument's sake, that we're going to pull an electron out of this bond here, break this bond, and shoot that electron over to here.
That way, we get r months around the boron.
It says p3 hybridized.
It just doesn't have that extra electron.
But now it rips the electron out of this silicon-silicon bond.
And what's the consequence of breaking this bod?
What's the electrical feature that we've created here?
A hole.
So now we've created a hole somewhere else in the crystal in order to satisfy the desire of boron to get that fourth bond.
And now can you see that for every boron that I introduced into the crystal, I'm going to make a hole somewhere in the crystal.
And on the energy level diagram, where do those holes live?
Those holes live in the valence band.
So let's go make the energy level diagram.
So up here we have the conduction band, and downstairs, we have the valence band, as before.
And I'm going to assume that we got the thermal promotion.
But it's so tiny in comparison to the amount of boron we're going to put in.
I'm not going to muddy the water here and show that you've got pair formation.
Because the extent of it is so small, it doesn't make any difference.
So now what happens?
Now, I know I'm going to generate holes here, and I've got to show this energy level.
And this bonding level is different from this bonding level.
Agree?
Silicon-silicon bond has different bond energy from boron-silicon bond.
So that means the top of the valence band is different from-- because this valence band is silicon-silicon-silicon.
So it turns out that, and this is not to scale, that the energy level of this bond is just a little bit higher.
And this is an energy level that involves the accepting of electrons, whereas in the other case, we were donating electrons.
Hence, that's called the donor level.
This is called the acceptor level.
And it's about 1/50 of an electron volt above the top of the valence band.
And so what happens is that in order to make this bond, we move something out of the valence band up into the acceptor level and generate a hole.
And so for every boron that we put in, we have another broken bond.
And all of these lie at the same level, but the dilution is so high, that they're so far apart, that they don't violate the polyexplosion principle.
They're thermally promoted up, and away we go.
So now, under these circumstances, the number of negative charge carriers no longer equals the number of positive charge carriers.
Because I've got all these positive charge carriers.
Thanks to the introduction of boron, p is greater than n. So we've made a p-type semiconductor, by the introduction of a subvalent impurity.
And we can redo the entire central part of last day's lesson, which was the Bohr model.
Now, this is the sexiest part of the Bohr model I've ever seen.
This is really, really cool.
The hole is mobile!
The hole is mobile!
And the boron is a little bit shy of protons, right?
It's 3 in a land of 4.
So it's negative.
So I've got a stationary negative center, and I've got something positive revolving around a negative center.
This is an inverse Bohr model!
Because the Bohr model has a honking big positive center with a dinky little electron revolving around.
We've got the immobile negative center, and the mobile hole revolving around.
Go through, you get the same set of quantum-- I mean, there's a whole bunch of quantum levels in here.
Fantastic!
How far that Bohr model can take us.
So now you know how to make a p-type semiconductor, you know how to make an n-type semiconductor.
And then what you can do is put them together.
And you can put, say, boron-doped silicon opposite phosphorus-doped silicon.
Be careful here.
The fonts are really critical.
This is uppercase P, for phosphorus.
This is lowercase p, for positive doping.
So don't go, oh, P, phosphorus, that means it's p-type.
No, P gives you extra electrons.
It's n-type.
So that's what uppercase P. This is a lowercase p. This is p-type.
So now I've got p-type, n-type, and so what do I have here?
I've got a p-n junction.
And this is the beginning of solid state devices, rectification of A/C current, diodes, you name it.
And it all starts with this, the chemistry.
This is the birth of the transistor, and all the modern electronics that we have based on that.
Later on, we'll show you how you get the boron in, and how you get the phosphorus in.
You don't just come in and sprinkle it on top, and hope that it goes to the lattice site.
There's a lot of processing involved.
OK.
So this is where I want to hold for the unit on semiconductors, and how it fits into the grand scheme of things.
So in the books, you'll see-- this is a very crude drawing, because this is showing n-type semiconductor with grossly exaggerated numbers of donor electronics up here.
There's no mention of the donor level.
So this is sort of semiconductivity, sort of pre-3091.
This is extrinsic semiconduction for idiots, I guess you'd call it.
And then, same thing here.
You see all of the holes in the valence band.
We know better, because we know there's an acceptor level and there's a donor level.
OK.
So now we're going to switch gears.
Now here's the big transition, right now.
If the last two lectures didn't leave high school behind, today we leave high school behind.
So those of you who have been coasting up until now, thanks to your very good high school background, start paying attention.
Because it's high school no more.
So just to remind you where we've come.
We started over here, at the beginning of the semester.
And the big theme is going to be, electronic structure informs bonding, which informs state of aggregation.
State of aggregation is solid, liquid, or gas.
So I want to show you how we get to where we are.
So we did all of these topics, right?
Bohr, or Sommerfeld quantum numbers, multielectron.
And then we came to octet stability.
And with octet stability, we went a long way.
We got into ionic bonding.
We got into completing the valence shell.
And in fact, all of this is a consequence of the octet stability.
And that led us into the various types of primary bonding.
Ionic, covalent, metallic.
And in some instances, van der Waals.
Solid argon is van der Waals bond.
It's the only kind of bonding, so it's the primary bonding.
But in some cases, like HCl, how do you get liquid HCl?
How do you get solid methane?
So we had invoke secondary bonding for some of these covalent molecules, and that invoked dipole-dipole interaction, the London dispersion force, which is the same as the van der Walls bond, but I decided to mix things up, so that both men get a little bit of attention, none of them feel slighted.
And finally, hydrogen bonding, in those special instances.
And that allowed us to then determine whether something was a gas, a liquid, or a solid.
And when it's a solid, we're happy.
Because this is 3091, and we're interested in the solid state.
By the way, what is a solid?
Well, here's an operative definition.
That which is dimensionally stable, has a volume of its own.
It doesn't distort in a gravity field.
All right?
Liquids will distort.
Fluids, gases will distort.
Now, there's two ways of classifying solids.
One is bonding type.
That's what we did here.
But the second way of classifying solids is based on atomic arrangements.
All right?
So let's look at that.
There's only two types of atomic arrangement.
Ordered and disordered.
I love these taxonomies that split everything into a choice of 2.
That way, you just sort of go through life and decide, is it left or is it right?
Is it up or is it down?
Is it ordered or is it disordered?
So what are the characteristics or an ordered solid?
It has a regular atomic arrangement.
That means it's got a long-range order.
And I'll show you that in an ordered solid, I know not only where my next nearest neighbor is, where my tenth nearest neighbor is.
I know where my hundred and fifty second nearest neighbor is, because it's an ordered solid.
And we have a simple Anglo-Saxon word for an ordered solid.
It's called crystal.
So when I say something is a metallic crystal, I don't mean it's something that's got magical properties, and if I put it over my head, I get powerful or something.
It just means I've got atoms in an atomic regular arrangement.
Now contrasting to that is a disordered solid.
And in that case, we have a random arrangement.
But I put an asterisk here because, as they say in California, it's not totally random.
It's only random up to a point.
No long-range order.
There may be short-range order.
In other words, I know what my next nearest neighbor is, but I probably don't know where my tenth nearest neighbor is Because the coordination shell is established, but the next coordination shell, and the next coordination shell, they're not established.
And I'm going to show you, with examples, what that means So we call those kinds of disordered solids amorphous solids, as opposed to the regular solids, which are crystalline solids.
And there's a plain Anglo-Saxon word for a disordered solid.
It's glass.
Now you might think, well, gee, doesn't glass mean that it's transparent in visible light?
No, no.
Maybe up until now, but I want everybody who's ever taken 3091 from me to be disabused of the notion that glass means transparent.
How do we think about the question of transparency?
And here I mean transparency vis-a-vis visible light.
I'm not going to say, what's transparent to x-rays?
That's pedantry.
So how do we think about whether something is transparent to visible light?
We just ask ourselves, is the band gap greater than 3 electron volts?
The band gap is greater than 3 electron volts, visible light zooms right through.
It doesn't have enough energy to excite the electrons, be absorbed, and re-emit.
So that's what transparency means.
So I can have the kind of material that's used in things like eyeglasses.
It's transparent to visible light because of its high band gap.
Diamond, diamond is transparent to visible light, and you wouldn't dare insult diamond by saying that diamond is glass!
In fact, watch the old gangster movies from the 1930s, that was the derogatory term for fake diamonds.
You call them glass.
So how is it that diamond, which is crystalline, it's transparent to visible light, and yet the window glass that we have is transparent to visible light?
It has nothing to do with state of order.
Has everything to do with this.
In fact, David, if we could shoot to the document camera, I'll show you a piece of glass that's not transparent to visible light.
How do we zoom in on this thing?
OK. Auto focus.
So what I'm showing you is a piece of obsidian.
It's a naturally occurring mineral.
It's rich in silica, and it's turned dark because it's got some iron impurity, and the iron absorbs in the visible.
And this is clearly not transparent to visible light, and yet, it is a disordered solid.
This is a glassy solid.
In fact, the white speckles there are devitrified obsidian.
It's actually started to crystallize.
So the crystalline form is the part that's transparent.
If you could peel off this crystalline form, you'd actually end up with something that's transparent to visible light.
So here's an example of where the crystalline form is transparent to visible light, and the glassy form is opaque.
So glass has nothing to do with transparency.
This is how you think about the question of transparency.
OK.
So let's cut back to the slides, David, please.
So we're now going to start by looking at ordered solids.
So we're going to take next several lessons and talk about ordered solids, and then after that, we're going to take a few more lessons and talk about distorted solids.
Now first of all, what's the term crystal?
Where does it come from?
It comes from the Greek word, crystallus, which is the Greek word for ice.
That's where we get the term crystal.
So we're going to start with a history lesson.
That's how we start everything in 3091.
A history lesson.
So we go way, way back to Isaac Newton's time.
The Reformation, Charles, Oliver Cromwell, all those exciting times.
In merry old England, there was Robert Hook.
You know him from Hook's Law, in mechanics.
So he was doing military research in 1660.
He was trying to understand what was the optimal way to stack cannon balls.
So when you're on a battle front, you want to figure out how to keep your material compactly arranged.
You don't lie them on the ground.
So from that, he concluded that crystal that is a regular array must owe its shape to the packing of spherical particles.
So that's what he was thinking, or trying to imagine.
Remember, Democritus said that we have these indivisible particles, but no one had ever seen them before.
Then in 1669, a Dane by the name of Steenson who was working in Italy was looking at quartz crystals.
And he was studying various quartz crystals.
By that I mean, quartz crystals of different sizes.
And what he observed was, didn't matter what the size of the crystal was.
They always had the same angles between corresponding faces.
So if I give you a big crystal and a smaller crystal of the same material, and a smaller crystal, and they all have the same shape, Can you imagine that you might conclude that that is reflective of something down at the atomic level, and if you could get down to the atomic level, that spacial arrangement would hold right now?
Because that's how it grew, right?
It started from a small number of atoms and grew, grew, grew.
Why would it start with one shape, and all of a sudden change to another shape?
It doesn't make any sense.
At least, not to me.
And certainly not to Steenson.
And then we go to Holland, 1690.
Christiaan Huygens.
And he was studying calcite crystals.
And he drew drawings of atomic packing and bulk shape.
This is 1960.
I'm going ot show you an image from his book.
This was drawn in 1690.
See, this is stacking of cannonballs, but he was looking at a calcite crystal.
Actually, Dave, could we cut back to the document camera?
We're going to do a fair bit of this.
I'll show you calcite crystal.
This is what he had to work with.
It looks like this.
To give you a sense of scale-- what do I have?
Do I have anything you'd recognize?
OK, here's a soda can to give you a sense of scale, OK?
This is one honking big calcite crystal.
I got it as a gift from someone who watched 3091 lectures on OpenCourseWare.
And he said, if I ever come to town, can I attend your lecture?
I said, sure.
And he showed up, he attended the lecture.
He was a man probably in his early 40s, and as a pastime he collects gemstones.
And he gave me this giant calcite crystal, because the one I was using before was so pathetic.
He said, I've got to give you one.
And then, I thought, jeez, if I could ever come upon somebody who's in the gold business-- and you know, he might give me a big honking crystal of face-centered cubic metal called gold-- which broke $1,000 an ounce yesterday, but-- anyways, if you're out there, and you're watching these lectures, I'm here.
It's 8201, and you can see any anytime.
My assistant's name is Hillary, she'll take the call.
All right.
So let's go back to the slides.
Dave, could we cut to the slides, please?
Thanks.
All right.
So now-- oh, I'm going to show you a few others.
Remember, here's the idea, that we're looking down to the elemental level.
So if you look on your periodic table, it says 10, you see, 10, it's supposed to be tetragonal.
So here's a crystal of 10.
Dave, cut back, please.
We're going to go back and forth today a lot.
There's a piece of ten 10.
So you might look at it and say, gee, that looks kind of cubic.
But if you look carefully, the vertical dimension is greater than the horizontal.
In fact, it's got a square base.
And then, this is the tall one.
Which, if we cut back to the slides, let's see.
And that's what tetragonal looks like.
And we'll study all of these in a few minutes.
These are some giant crystals I didn't dare drag in.
This is sort of human-sized scale.
This is balsalt, which is a magnesium iron silicate.
It's on a coast of Iceland.
Dave, let's show them a few more.
Here's one more, if we go to the document camera.
This was beryllium aluminum silicate.
This is a barrel.
And you can see, this has got a hexagonal habit.
So something's going on at the atomic level that's different as we go from one crystal structure to the other.
What else have I got here?
Here's some calcite, here's some fluorite.
Look at this one.
This is square pyramidal but it's ionic.
So don't give me any of that sp3 d2 stuff.
Because it's not going to work here.
All right.
Let's go back to the slides, please.
So then we jump to 1781, to France.
Rene Juste-Hauy was at the Sorbonne, and he was studying the cleavage of calcite crystals.
In other words, he was taking these things and breaking them.
So he'd go to work every day and break things.
And he studied the shards, and he found that the shards were all rhombahedral.
the same shape as the parent crystal.
No matter how small the shards got, they always looked like this.
He didn't end up with cubic shards.
He didn't end up with long slivers.
He always ended up with rhombahedral slices.
So the other thing is that he didn't end up with any voids.
No matter how he cut this thing, he never got voids.
So he reasoned that this must be the way things are packing in three space perfectly.
So then he said, gee.
If they pack perfectly rhombahedrally-- we all know that we can stack boxes together perfectly, cubes together.
So he, being a Frenchman, said, what's the mathematical formulation?
They were steeped in mathematics there.
That wasn't a slur against the French.
It's a compliment.
He said, what are the mathematically distinct shapes, that if we stack them together in three space they will fill three space with no void?
And the answer is seven.
Seven space-filling volume elements.
By the way, the gabled milk carton isn't one of them.
And so he called the crystal system, so I call seven distinct shades of milk cartons.
And these are the seven crystal systems, and they're described geometrically, and you'll get a chance to go through them.
The cube is one of them.
Tetragonal.
Here's the calcite, rhombahedral.
There's barrel, hexagonal, and so on.
The seven different ways you can feel three space.
And this is from the archival readings of my predecessor, Professor Witt.
Actually, just to give your a sense, here's the two-dimensional analogy.
If I said to you, we're going to open a company, and we're going to make bathroom tile, and we want to make it sort of an upscale company, you know, not everybody wants boring old square tiles.
But definitely, I can cover two space with square tiles by putting them side by side, one on top of the other.
What's another way I can fill?
I can go with rectangular tiles.
So this dimension is different from that dimension.
What's another way?
I can use a rhombus, right, at 60 degrees.
Or I can use just a plain old parallelepipede at some arbitrary angle.
And if you go over to the Stata Center, to the Gehry building, take a look at the shape that they chose as the unit cell.
This is the repeat group, if you like, or the unit cell, because that's the one that's going to be cookie cutter replicated, side by side.
When you're putting the skin on a building, you have to make everything fit with no holidays, right?
Because you don't want gaps.
what they chose was this area element.
And all the pieces of stainless steel are cut to this shape.
And that's how you make things fit.
How you take something and make it fit around a three dimensional object is tricky.
It's the same thing as the clothing problem, right?
How do you take a flat piece of cloth and make it fit the contour of the human body?
At least, you know, in men's clothes, you know, we've got dimensions.
We've got sleeve length, got waist, chest.
We've got all these numbers.
For a woman, 8.
One number?
One number?
Fashion designers don't know crystallograpy!
If they knew crystallography, they'd know how to specify the shapes!
If you understand this, you can start your own business and make clothes that fit.
And I guarantee you, there's a market out there for clothes that fit.
What about this one?
Pentagon.
We're going to make pentagonal bathroom tile.
Well, when we install these, we're going to have to use two different colors.
You can see this up here, right?
This is pentagonal bathroom tile.
It doesn't fit.
You've got all these white spots.
So this is not a unit cell.
This is not one of the crystal systems for two dimensional.
So this one's no good.
We better put an x through it, in case someone fell asleep, and wake up, and then tell me that that's OK. All right.
So then we move about half a century forward, to the turbulent year 1848, also in France.
Auguste Bravais.
And what Bravais did, is he said, all right.
This is just filling space.
But where are the atoms?
I haven't told you where the atoms are, right?
You know, as far as you're concerned, maybe there's one atom in each of these boxes.
Or maybe there's atoms at the corners of the box.
Or maybe there's atoms at the corner and the center of the box.
Where are the atoms?
I've simply told you that I can fill three space with boxes, or with these rectangular boxes, and so on.
So he set out mathematically to prove how many different arrangements of points there are in space.
It's sort of like saying, if I want to tell you where all the apples are in the orchard, I'm going to tell you where the trees are, and I'm going to tell you where the apples are on a tree.
So where do the atoms go inside these boxes?
Turns out there's 14 different arrangements.
And what do I mean by that?
This is a good example.
So we've already established that the cube is a space-filling volume element.
It's one of the seven.
But look.
If I'm one of the atoms at the corners here, I've got six nearest neighbors.
This is also a cube, but it's got atoms at the corners and an atom in the center.
You might say, well, what's the big deal?
If you're in the atom in the center, and you look to the nearest neighbors, you've got eight nearest neighbors.
Here you only have six nearest neighbors.
So this is spatially differentiated from this.
So what why don't we keep going?
And by the way, the same environment of that central atom that's coded green is the same environment of any of the atoms on the corners.
They're all symmetric.
So both are cubes, but they have different space point environments.
And here's a third one, where we put atoms at the corners and atoms on each of the faces.
So this atom on the face, you can see, it's got one, two, three, four atoms in the plane, three behind, three in front.
It's got 12 nearest neighbors.
This has got eight nearest neighbors, this has got six nearest neighbors.
So Bravais went through and he said, let's take those seven crystal systems.
Let's put atoms in distinguishable ways.
How many distinguishable ways can we come up with atom arrangements?
And the answer is 14.
Now, what we're going to do in 3091, is we're just going to confine our conversation to the cubics.
Because you use orthogonal vectors, they're all the same length, the mathematics are simple, and the principles are the same.
And by the way, boatloads of materials are in the cubic system.
So it's not as though we just chose something that's convenient but irrelevant.
So here are the different Bravais lattices.
So you can see the cubic, there's simple cubic, body-centered cubic, face-centered cubic.
With tetragonal, there is simple tetragonal, body-centered tetragonal.
There's no face-center tetragonal.
If you try to make face-center tetragonal it's degenerate, reverts to one of the other systems.
So these are the distinguishable systems.
And everything, including crystalline proteins-- if the protein crystallizes, it forms atomic arrangements in one of these Bravais lattices.
Everything conforms to one of these Bravais lattices.
And if it doesn't, it means it lacks long-range order, in which case it is a glass.
That's it.
So here's face-centered cubic.
You can see the placement.
But there's something that's been simplified here.
Right now what you see is a single atom, designated by a hard sphere.
But at each of these lattice points, I can put more than one atom.
So let me show you what I mean by that.
What I want to do, is put different numbers of atom arrangements at the lattice point.
So those positions, the distinguishable positions, are called lattice points.
And now I want to define the crystal structure-- the crystal structure is the complete accounting of atomic arrangement.
It's the complete description of atomic arrangement.
And we don't have to go too far, because we just have to get the base unit, the unit cell, and then just repeat the unit cell.
Complete description of atomic arrangement.
And so how we going to go about this?
We're going to start with a Bravais lattice.
And what is the Bravais lattice?
The Bravais lattice is simply a point environment.
And this will mean something to you in about three minutes, when I give you the rest of the description.
So it's a set of points in space and the basis.
And what is the basis?
The basis is the atom group at each lattice point.
So what do we mean by that?
Well, let's look at what's up on the board.
What's up on the board is face-centered cubic.
So let's do all of these.
So we're going to say that the Bravais lattice, in this case, in face-centered cubic, FCC.
That's what this one is, up on the board.
And we've got dots indicate the points.
Because really, the points are nothing.
It's a concept.
See, it's French.
It's a concept, right?
What's this?
See what I'm holding?
I'm holding a Bravais lattice.
It's just a set of points.
It's like a John Cage piece, you know, a bunch of rests.
So if I go to the piano and I play rests in five four time, listen.
Isn't that great?
Man, it's fantastic.
All the different rests, and different-- So this is a set of lattice points.
Now I'm going to put something on them.
You think I'm kidding.
I'm not.
All right.
Now what's the basis?
If I put a single atom, we have what's up on the slide.
And so that's the representation of metal.
So for example, gold is FCC, with a single gold atom at each lattice point.
Aluminum, copper, platinum.
These are all FCC metals.
So there's a single atom at Bravais lattice point, and so the crystal structure is known as FCC.
It's the same as the Bravais lattice.
It's called FCC.
OK. Now, we can also put a molecule at each lattice site.
So I'm going to put up the FCC here.
Here's the FCC.
There's the cube, the set of lattice points, and the lattice points are-- I'm going to just illustrate on the front face.
One, two, three, four, and a fifth atom at the front face.
Now these are the points.
I could either put simple atoms there, and the central image there shows what happens when you have the close-packed arrangement, the atoms are so big that they touch.
Or what I could do, at each lattice point, I could put a molecule.
So for example, I could put methane.
I could put CH4.
A methane molecule.
Because when one of you gets to Jupiter, and you see the methane ice, and you go, oh.
That's FCC.
Because everybody else on the mission hasn't taken 3091.
They don't have a clue what they're looking at.
But you go, that's face-centered cubic.
And they go, huh?
And this is at each lattice site.
You see, my point is, now look, here's the difference.
All five of these, the carbon and the four hydrogens, sit at this lattice site.
All five atoms sit at this lattice site.
It's not that the carbon goes here, one hydrogen goes over at one lattice site, one hydrogen over here.
You don't have the methane straddling the unit cell.
You have all five atoms here, five atoms here.
At every lattice point, I have all five atoms at each lattice point.
So in that case, we also have, it's FCC.
So methane ice is face-centered cubic.
Because it's spherical.
They pack just like gold atoms.
You know, from a distance, we can model this as a sphere, as a point.
Or we could put an ion pair.
An example of that is sodium and chloride, and this is called rock salt crystal structure.
Rock salt.
Let's look at that one.
So at each last lattice point, I'm going to put two ions.
Not one, two.
This is taken from your text.
So you might look at this and say, well, here's the front.
You might look at this and say, well, isn't this kind of like simple cubic?
Because I've got one, two, three, four, but I've got different atoms if I try it that way.
See, this is taken from a different book.
I did some highlighting on it.
So this is sodium chloride again.
One, two, three, four, five.
With this lattice site, I put the chloride atom and the sodium atom.
The pair of atoms are associated with this lattice site.
You might say, yeah, but this is way over here.
Doesn't matter.
I'm going to anchor all of those to this lattice site.
To this lattice site, I'm going to anchor this and the sodium ion.
So this is not simple cubic.
Because if I try to call this simple cubic, I have different atoms at different lattice sites, and that's a no-no.
Has to be the same.
So if I associate the pair, chloride ion and sodium ion, chloride ion and sodium ion, chloride ion and a sodium ion not depicted.
It's off the diagram here.
And I put these dots at the center of each chloride ion.
Can you see that I'm at the face of FCC?
So that's an FCC Bravais lattice, but it's got two atoms, in this case, both ions, associated with each lattice point.
I will come to the dogs in a second.
And now there's a last one I can do.
And that's got two atoms, but not an ion pair.
We're going to put an atom pair.
Got an atom pair, and that looks like this.
I'm going to put a carbon atom, one carbon atom, and a second carbon atom.
And this angle here is 109 degrees.
And I'm going to put this pair of carbon atoms together, and when I do that, and I replicate that through FCC, I end up with diamond cubic.
And I'll show you a three-dimensional example of that next day.
So just bear in mind that if you have an ion pair, you end up with rock salt.
If I show you this pair, this is due to the sp3 hybridization.
And so examples of this one are diamond, not graphite, just diamond, silicon-- so everything that I've shown you about single crystals, and intrinsic and extrinsic semiconduction, is diamond cubic crystal structure.
But it's not one of the Bravais lattices.
It's an FCC Bravais lattice with two atoms per lattice site.
That's diamond cubic crystal structure.
But it extends to two dimensions.
So here's an Escher print.
It's got all these dogs.
In two dimensions, what's the crystal structure here?
Well, there's no such thing as body-centered cubic.
There's no third dimension in two dimensions.
So you've only got simple cubic or face-centered cubic.
So what do we have here?
Well, you see, you've got a problem.
Because the black dogs are facing the opposite direction of the white dogs.
So you can't go one, two, three, four, and make anything that way.
So instead, I put dots on the, I don't know what you call this part of the dog.
I don't know.
This part of the dog.
The leg?
Whatever you call this thing.
Yeah.
You put it on the same spot of the dog-- maybe I should have used his eye, I know how to call it.
Anyway.
So I put these things up, right?
And that means that if I associate the four dogs with this point, the two dogs facing the right, and these two dogs facing the left.
So these four dogs are associated with this lattice site.
Then I move up here, and I've got the same thing.
One, two, three four.
Two dogs facing right, two dogs facing left.
So that means I've got a simple cubic bravais lattice, and the basis is four dogs.
Four dogs as my basis.
And with that combination of a simple cubic lattice and four dogs is the basis, I can make the entire structure.
That's what this is all about.
That's my unit cell, that's my repeat unit.
So there's simple cubic, right there.
So instead of a single atom at each corner, or these are spheres that are touching, we got the four dogs.
Here's another one.
Once I got the Escher, I kept flipping through the book.
I don't know what this thing is, but it's creepy.
Anyway.
It's wallpaper.
If it's wallpaper, it's ordered.
If it's ordered, it's a two-dimensional crystal.
If it's a two-dimensional crystal, it has to conform to one of the 14 Bravais lattices.
So which one is it?
So I looked at this for a while, and I took this happy entity here, and I put a dot in its belly.
So wherever I found one of these, I put a dot.
and then I connected the dots, and what do I get?
I got the rhombus.
Now, what's the basis?
That's the Bravais lattice.
What is the basis?
So I went over here and I said, well, if I start from this thing and I go all the way up, I got some tail, blue tail here.
And this thing way down here looks like it's missing a tail.
So if I start from this, go all the way down to here, and capture all of this material in between and associate it with that lattice point, that can be my basis.
And then I take that set of imagery, and I put it on top of this, where it's centered at about this thing's navel, and put it around here.
You can see, the translation is obvious there, and the translation is obvious here.
There's the wallpaper.
So it's part of a Bravais lattice, only in two dimensions, you see?
This is crystallography.
That's all it is.
Crystallography.
All right.
So the next thing that we have to do, it's kind of a little bit boring, but you need to know this stuff, because it's formative for the next unit.
It's the characteristics of cubic lattices.
Now what do I mean, the characteristics?
I mean the geometric characteristics.
So there's a table here that I've taken out of your archival reading.
If you go to the website, there's a set of notes that were written by my predecessor, Professor Witt.
And if you go and look this up, this unit on crystallography, there's this table.
And it shows such things as the unit cell volume.
Well, what's the unit cell?
The unit cell is this cookie cutter cube that's the repeat unit, and it has a dimension here called a, which is the lattice constant.
It's the size of the box.
So what's the volume of the unit cell Well, duh, it's a cubed.
That's obvious, all right?
Doesn't matter whether it's simple cubic, face-centered cubic, body centered cubic.
How many lattice points per unit cell?
Well, that's-- how many atoms do I get per unit cell?
If you look at this, you say, well gee, it's-- you know, if you've got the simple cubic, you've got them at the eight corners, so it's eight, right?
Well, no, because each atom on the corner is inside this cell, but it's also inside the neighboring cell, and it's inside each of the eight neighboring cells.
So I have eight times one eighth.
So the total amount of atom within the cube is only one.
That's why you've got to go through this derivation.
So here's an example for face-centered cubic.
You see the corners, one, two, three, four?
There's eight of those.
Eight times one eighth is one.
And then you've got these atoms on the face, and there's one, two, three, four, five, six, right?
Six faces.
But each of these atoms on the face is only half inside the cube.
The other half is in the other cube.
So I got six times one half is three, and that gives me four.
So that's how you go through, and figure out what's going on.
And then you can look at things like nearest neighbor distance, and what's the relationship between the radius, if you're looking at hard sphere packing.
So here's a good example.
Here's the hard sphere packing model.
So I've got atoms all the same size, and they're packed to touch.
And they're packed to touch so that in the end, I end up with a face centered cubic arrangement.
And so what's the relationship between the radius and the lattice constant?
And it's just simple trigonometry, right?
4r equals 2a squared, and you go through it, and that's how you end up with all these formulas.
So that's what going on.
All right.
So I think we're going to move to the last five minutes here.
So just a couple of comments.
Today we played music when you came in.
It was Burning Down the House.
An old piece by Talking Heads.
Why was I playing Burning Down the House?
Well, I want you to study this painting.
This is a painting by Georges Braques, from 1908, And it's called The Houses at L'estac, or if I speak Canadian, The Houses at L'estac.
And you see the thing about these houses?
You see the ones in the background have better definitions than the ones in the foreground?
And the ones in the foreground are all distorted, and kind of falling over?
Well, when this was exhibited at the Salon, on the critics lambasted it.
They said, this thing is absolutely childish.
And one of the critics said, this isn't a painting of houses.
This is nothing but a stack of cubes.
And from this painting, and that negative comment, came the term cubism.
And so the entire movement of cubism comes from this.
And since we're studying the properties of cubic crystals, I thought the connection was pretty tight.
And that's why I chose that piece.
Don't go anywhere yet.
We still have a couple of minutes.
It's not better out there than it is in here, I guarantee you.
All right.
So now I want to show you one of the properties.
Why do we care about atomic arrangement Suppose some of you are interested in fiberoptics.
One of the things you have to do in fiberoptics is make junctures, because you can't run a cable 100 miles without making some junctures.
Well, atomic arrangement reflects microscopic properties.
And one of the properties that you can have in a crystal is something called birefringence, depending on the atomic arrangement.
So what happens is if you have a ray that comes in from the left here, it splits into two.
And that's no good, in certain instances.
In other instances, if you want a beam splitter, you want something birefringent.
So look at all these crystals.
Quartz is birefringent.
Ice is.
You've seen it.
Calcite is birefringent.
What do I mean by that?
Well, Dave, can we cut to the document camera?
Right.
So here's a here's a crystal of-- well, this is my handwriting.
It's beautiful.
Let's write-- here's more.
Isn't that beautiful?
OK, enough about me.
Now let's-- All right.
So this is birefringent.
But what happens if we rotate it?
We can change the degree.
And if we're really clever, and we get it onto the optical axis, we can actually get these things to superimpose.
Here's a piece of cryolite.
This is the stuff they make aluminum out of.
They dissolve aluminum in it.
And you can see that, see, I loaned it to somebody, and they dropped it.
And when they dropped it, they damaged it.
That's what happens.
Look, it's all crunched up there.
But anyway, I don't want to get into that.
But so you can see, if we turn this and we get it right on the optical axis, we can actually make these things line up sometimes.
Anyway.
So you can see the property of birefringence.
OK. Let's go back to documents, Dave.
One more thing.
Last thing I want to show you.
You know gold.
Gold looks like this.
It's yellow, right?
Well, depending on how you, you know, those of you have seen certain wedding bands know that if you want to make a white gold, you add nickel to it.
And once the nickel content gets above a certain level, the gold looks white.
In other words, just metallic reflective.
In Eastern Europe, it was common to add high levels of copper, in which case, you get a beautiful rose-colored gold, or pink gold.
I'm going to show you some blue gold.
This isn't a joke.
This is 14 karat.
It's 40 plus percent by weight gold, and the balance is indium.
And when you add indium to gold, you get blue.
Document camera, please.
Just so that you remember.
OK. That's gold.
This is gold foil.
Rutherford, et cetera, et cetera.
You know, this is yellow gold.
Ho-hum.
This is white gold.
Same deal, but with some nickel.
Now it's reflecting off of the yellow gold.
Boy, there's a lot of yellow in here.
Where's that coming from?
It's hard to tell.
All right.
Now Let's look at some blue gold.
This is blue.
So, but it's brittle.
So you know, what are you going to do with something that's brittle?
You can't shape it.
So I have an idea.
So here's my idea.
See, what we do is-- see this?
We do, instead of this, we put this.
We drill a hole in it, we put hands on it, we sell it for a lot of money.
It's blue gold.
It's blue karat gold.
What you can do if you understand crystallography.
All right, get out of here.
I'll see you on Friday.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, Let's get down to business.
Last day we started study crystallography.
And we were looking at ordered solids.
And we recognized the crystal structure is the sum of the combination of a Bravais lattice, which is the way that points are identified in space, plus the basis.
And we saw that we could put a variety of elements at the corners of the Bravais lattice.
And this is taken from the archival notes where we see the simple, single atoms at the various lattice points.
And we recognized last day that we could put clusters of atoms and so on, and even proteins will crystallize in this form.
And actually, to make a point, about five years ago, a group of students came to my office after the final exam, and recognizing that I drink Fresca during the lectures, they came to me with this body-centered Fresca unit cell.
So this just makes the point that you can put anything at the lattice point.
So there it is.
Oh, here's the point.
This is me at a conference in Copenhagen.
This is outside the Admiral Hotel.
And just to show you I'm always thinking about 3.091, I came outside-- and this is a colleague of mine from France, Marcelle Gaune-Escard.
How do we know she's French?
She's got a cigarette.
She's a scientist.
She's a woman.
And she smoking.
And it's fantastic.
And she's a great gal.
Anyways, so there we are out in front of the Admiral Hotel.
And I notice these cannon balls.
And what do you notice about the cannon balls?
They are sitting in one of the Bravais lattices.
So I couldn't resist having my wife take this picture.
Because I knew that this would come back to 3.091.
So here we are with the individual cubic lattices where we're going to focus.
And we started looking at the properties of cubic lattices.
And I started working through this table.
And we got down to nearest neighbors, and so on, and this relationship between the lattice constant and the radius.
And I just wanted to show this last point here about packing density.
Because it comes up considerations of properties of elements.
And so I wanted to go through the packing density calculation.
And so the way we handle the packing density, remember we're looking at a unit cell here.
This is the cube that's the repeat unit.
And it has a lattice constant of dimension a.
And then depending on where we put atoms, I want to show that we've got three different options here, simple cubic, body-centered cubic, and face-centered cubic.
And the last line of that slide shows that the densest packing of atoms-- this is hard spheres, the cannon balls-- the densest packing is face-centered cubic.
And I wanted to show the derivation of that where if we face-centered cubic where the atoms are big enough to touch.
If we have 4 atoms at the corner touching on the face diagonal and then continuing through the cell, we want to get packing density as the ratio of the volume occupied by the atoms-- and here we're modeling the atoms as hard spheres-- divided by the volume of the cell itself.
And so that's pretty straightforward calculation.
We're modeling the atoms as hard spheres.
And we saw last day that if we take the fraction of the atom, 1/8 times 1/8, and 6 times 1/2, we have 4 atom equivalents per unit cell.
And each of these has a volume of 4/3 pi r cubed.
And then we divide that by a cubed.
And then we know the relationship between a and r. This is 4r and that's root 2 a.
So we can say root 2 a equals 4r.
Either eliminate a or eliminate r. We eventually end up with this dimensionless group, which is pi divided by 3 roots of 2.
And that is 74%.
And that's the densest package.
If you go through the same calculation for body-centered packing, you'll find it's 68% packing.
And for simple cubic it's 52% packing.
That's the highest density that you can have.
So I urge you to spend some time.
If you work the homework, you'll work your way through that chart.
So now I wanted to take a few minutes and talk about crystallographic notation.
Because we need to know this in order to describe various features of a cell.
So I'm going to start with a Cartesian coordinate system that obeys the right-hand rule.
So we'll put down x, y and z.
And then we'll put up a unit cell here with edge a.
So we have a on the x, y and z coordinate.
And we know that a equals b equals c. That's the unit vectors here.
There's a little a here, a little b here, and a little c here.
These are all equal to the lattice constant value a.
And all of the angles are 90 degrees.
This is what defines the cubic system.
So I'm going to put several positions here.
So here is the origin.
And the origin we designate as 0, 0, 0.
I'm going to put a position up here, a.
And you just use the same notation as you'd use mathematics.
Only crystallographers don't put parentheses around it.
So what I'm really teaching is notation.
You already know the math.
So a up here is what?
It's 0 units on the x-axis.
It's 1 unit on the y-axis, and 1 unit on a z-axis.
So that would make it 0, 1, 1.
And I'm going to put a b out here.
B is out 1 unit on the x-axis.
It's 0 units on the y-axis.
And it looks like it's up at about 1/2 unit on the z-axis.
So that makes 1, 0, and 1/2.
And so now what I want to do is show you how we designate directions and planes.
You use a slightly different notation from what you used to in math.
So this chart will be posted at the website.
Move the coordinate axes so the line passes through the origin, define a vector from the origin to the point on the line, and choose the smaller set of integers.
So let's do a simple one here.
Let's do from origin to b.
So o to b.
That's this vector here.
So how would we do that one?
Well we're already at the origin.
So that's trivial.
o to b, so we're going from here out to here.
And then what does it say?
It says choose the smallest set of integers, no commas, clear fractions.
So that's going to take us out to 1 0 1/2.
But the crystallographers don't like fractions.
So you're supposed to clear the fractions.
You clear the fractions by multiplying through.
So make that 2 0 1.
It's the same direction.
See if I came out two units here, I'd end up being one unit here.
So it's the same direction.
It really is the same direction.
Don't put any commas.
And put brackets around it.
So that's the 2, 0, 1 direction, which is o b.
And I want to do a different one.
I'm going to go this way, a to o.
So let's look at that one, a o.
And what happens on a o?
Well now I'm going to put the origin at a. I'm going to go to 0 in the x-axis.
I'm going to go minus 1 in the z-axis, and minus 1 in the y-axis.
So that gives me 0 minus 1 minus 1.
But see they don't use commas.
And so this looks kind of goofy.
Because you get these minus signs in here.
So what the crystallographers do is they put the minus sign on top of the number.
So this becomes 0 1 with a macron it.
If you'd studied Latin, you'd know this thing is a macron, a line over a vowel.
So 0 1 bar 1 bar.
And that's the direction a o.
So that's the macron acting as the minus sign.
So that's the a o direction And then furthermore, we can designate families.
This is just the 0 1 bar 1 bar.
If you want to be a hipster with crystallography, you call this 0.
You say it's 0 1 bar 1 bar.
But if you're a nice chemistry student, you'll say zero, minus 1 minus 1.
No we don't do that in 3.091, 0 1 bar 1 bar.
Now suppose I wanted to describe all of the faces, every face.
So what I could do, is I could say I want the set of all face diagonals.
Well, that's a face diagonal.
But what I can do is in a compact notation, I could write 0 1 1 and use the carat.
This is called the carat.
So if I enclose in carats, this means the set up all 0 1 1 type planes.
And so what that means is I can unpack that, and write 0 1 1, 0 1 1 bar, 0 1 bar 1, et cetera.
And what I'll end up doing is describing the set up all face diagonals.
So if I want to talk to you later when we're studying x-ray diffraction, I want to talk about face diagonals, I don't use that word.
We're quantitative here at MIT.
I say look at the 0 1 1 planes.
So you go face diagonals.
That's the compact notation.
If we want to look at all cell edges, what's the cell edge?
Here's the cell edge.
This is an edge here, isn't it?
It's right on the edge of the cell.
So it's 0 on the y, 0 in the z.
It's 1 in the x.
So the set of all cell edges would be 0 0 1.
And you could write 1 0 0, that's OK too.
Or you could write 0 1 0.
That's OK.
But if you want to be a hipster in crystallography, put the zeroes out in front.
So the set of 0 0 1 directions is all the cube edges.
And then lastly, if we wanted to get all the diagonals that starts from here and goes up, we'll have all cell diagonals, all, if you want to call it, body diagonals, not face diagonals, body diagonals.
Body diagonals is 1 1 1.
And the mathematics imitates reality.
How many combinations are there here?
I've got 0 1, and I've got minus 1.
So I have 0 0 1, 0 0 1 bar, 0 1 0, 0 1 bar 0, 1 0 0, 1 bar 0.
There's 6 And how many edges are there?
6.
Mathematics imitating reality, what a concept.
That's great.
OK.
So that's good.
Now we want to look at planes.
Let's look at planes.
Now planes are a little bit different.
This is basically what you know from math with a little bit of notation It's basically the same thing.
Planes are quite different.
So let's go to the planes.
So you know the equation of a plane.
It's x over a plus y over b plus z over c equals 1.
And a b and c are the intercepts at the x, y z axes.
Well, in order to make this notation compatible to what we have here for lines, a British meteorologist by the name of William Hallowes Miller in 1839 said let's define h, k, and l as the reciprocals of these intercepts.
And now we'll write the equation of a plane as hx plus ky plus lz equals 1.
And we'll use those values of h, k, and l as the indicators of a plane.
So here's an example.
So again, I've got the right-handed rule coordinates.
And here's the plane.
And you can see that it cuts the x-axis and a.
It cuts the y-axis at b.
And it's parallel to the z-axis.
So if we were just to use the old mathematics notation, and put the a, b and c, you'd have 1/2 here.
You'd have 1 here.
And you'd have infinity here.
And that looks goofy.
So instead what we want to do is use the Miller indices, which are the reciprocals.
So now the reciprocal of 1/2 is 2.
The reciprocal of 1 trivially is 1.
And the reciprocal of infinity is 0.
And so now you have something that makes sense.
And you put no commas in between.
And you put the whole thing in parentheses.
So that's the 2 1 0 plane.
And there's a really cool property here.
And that is, using this same formulation I just showed you for directions, the 2 1 0 directions-- you see 2 1 0 with square brackets, that's a direction.
2 1 0 with parentheses is a plane.
The 2 1 0 direction is perpendicular to the 2 1 0 plane.
And that's one of the advantages of using Miller indices notation with the classical notation for direction.
Here's another one.
This is 0 1 0.
This is the x, y, and the z.
So this cuts at 1, 0 1 0.
This is 0 2 0.
An 0 1 0 directions are perpendicular to 0 1 0 plane.
0 2 0 directions are perpendicular to 0 2 0 plane.
Oh here's 1 1 1.
That's cool.
See it cuts it at 1 1 and 1.
And 1 1 1 direction cuts through the 1 1 1 plane perpendicular.
This is an interesting one.
See the origin.
The plane cuts through the origin.
Now what do you do?
What do you do?
You move the origin.
Because we don't know where the origin of the universe is.
It's just a unit cell.
It doesn't matter.
So we move the origin up here.
And now what do we have?
We have 1 1 and minus 1.
And just as before, we put 1 with a macron over it.
That's 1 1 1 bar plane.
And there is the 1 1 1 bar direction.
It doesn't matter.
It'll always give you perpendicular.
It's really cool.
All right.
And we can also specify a set of planes.
So instead of using parentheses, we use brace brackets.
And if we use brae brackets, we get families of planes.
So for example, we could say if we wanted a particular face so that the face in the xy plane, facing the xy plane would be 0 0 1.
Right, because it cuts the z.
So this is 0 0 1.
But what if I wanted to say in a compact way, all faces of the unit cell, all of the faces.
So this is the particular plane.
If I wanted to say all faces I would write 0 0 1.
But I can't use any of these containers.
So I'll use brace brackets.
So 0 0 1 in brace brackets means the whole setup, 0 0 1, 0 0 1 bar, 0 1 0, 0 1 bar 0, 1 0 0, and 1 bar 0 0.
So there's only 6.
And there's 6 faces.
Again mathematics imitating reality.
What a concept.
Make the math work for you.
You don't work for the math.
All right and just to make the point about the h k l plane, is perpendicular to the h k l direction.
So that's good.
And the last thing I want to show you that comes out of the use of the Miller indices for planes, is that another property that we'd like to know is the interplanar spacing.
What's the spacing between like planes.
So interplanar spacing, that is to say the distance between like planes, adjacent planes identical index.
I'm going to write it to be specific to say Miller index, h k l in parentheses.
And the property that comes out of this-- and you don't have to derive it.
Somebody worked this out for you about a 150 years ago-- is that the distance between adjacent planes with the index h k l is given by the ratio of a, which is the lattice constant, this is the cubed edge, divided by the sum of the squares of h, k, and l taken to the power of 1/2, So the square root of h squared plus k squared plus l squared divided into the lattice constant, gives you the value of the spacing in between.
And this is the a just to be clear about it.
So for example, I think I've got some cartoons up here that show this.
So here's 0 1 0.
It's trivial.
What's the distance here?
The d 0 1 0 is just day.
How about here?
This is 0 squared, 2 squared.
0 squared, this is d over 2.
So this is a 0 2 0 plane.
But if the distance between 0 2 0 planes is 1/2 a, well, then that's an 0 2 0 plane.
Then this is an 0 2 0 plane.
So point of fact, every 0 1 0 plane is an 0 2 0 plane.
But every 0 2 0 plane isn't an 0 1 0 plane.
You can see that some planes have multiple names.
Here's another one.
Look, this is the distance between adjacent 1 1 1 planes.
It's a over root 3.
So as the index goes up, as h k l goes up, the distance between successive planes goes down.
And the angle between the planes goes up.
The easiest one is orthogonal.
OK, that's good.
That's good.
All right.
I think that's a good place to stop.
I dread these kinds of lessons.
And I'm sorry that the parents ended up here on a day like today.
But there's a little bit of this notation and taxonomy that they have to know.
And so we suffer through it.
And then we get on to the meat.
Now we're ready for some meat.
Now that we can describe crystal structure, we're ready for physical measurement of crystal structure.
So what I want to start today is the characterization of crystal structure by x-rays.
How do we measure?
How do we know that gold is fcc.
We use x-rays.
So today what I want to do is start as I always do on new unit with history lesson.
We'll start with the discovery of x-rays.
And we'll get into the underlying physics.
And the next day we'll look at applications including x-ray diffraction.
So first of all, what are x-rays?
They're a form of electromagnetic radiation.
And they have the property of that.
They have very short wavelength.
How short?
It's on the order of about angstroms.
So strictly speaking, it goes from about 1/100 to 100 angstroms.
So centered at about 1 angstrom is the center of the x region of the spectrum.
And because they're a form of electromagnetic radiation, their energy is given by these equations, hu equals hc over lambda or hc u bar.
Those equations that you've seen over and over again apply for the energy of an x-ray, because an x-ray is identical to a photon of wavelength somewhere down at this level.
Why do we care about this?
Because this is atomic dimensions.
If I want to measure the dimension of the human hair, I don't use a yard stick.
I use something that is of the human hair dimension.
If we want to get into the characterization of atomic structure, we need a measurement tool that is on the order of that blank scale.
And that's why we're going to use x-rays.
Now other property, I said very, very short wavelength.
But because of this inverse relationship between wavelength and energy, I want to draw attention to these superbly high value of energy.
So let's plug in the energy for lambda equals 1 angstrom.
When lambda equals 1 angstrom, we end up with 12,400 electron volts per photon.
Now what's that mean, just to put it in perspective?
Look up here at the table I've shown.
We know the ionization energy, the lone electron in hydrogen is 13.6 electron volts, which is 14 to 2 significant figures.
And then we go to helium.
You pull these off the periodic table.
This is the first ionization energy.
And they're down around somewhere about 10 electron volts.
Look at.
This is 12,400.
So let's go down to the 1 electron atom.
So this is helium plus.
This is lithium 2 plus.
And we see a set of numbers here that follow this sequence 1 4 9 16, which is nothing more than the sequence you would expect for a series of 1 electron atoms.
It's K, which is 13.6 times the square of z/ So z here is 2, z here is 3, z here is 4, and so on.
But even so, this is the last electron of nitrogen.
The most tightly bound electron of nitrogen, 668 electron volts.
That's a huge amount of energy.
But that pales in comparison to 12,400 electron volts.
So this is clearly ionizing radiation writ large.
What do I mean by ionizing radiation?
It has the capacity to ionize everything.
Every electron will be blown away.
This is called knocking the stuffing out of the electron structure.
So that's what we're dealing with.
So now let's go to the origin of x-rays.
So how did we discover x-rays?
Well, it was a dark and stormy night, not unlike the weather today, apropos of the lesson.
It was November 8, 1895.
And it was at the home and laboratory of a professor.
Of course it's a professor, Professor Wilhelm Roentgen.
And he was a professor of physics at the University of Wurzburg in Bavaria, now part of Germany.
And he was doing research, as many people were in the 1890s, on the properties of gas discharge tubes.
He was studying gas discharge tubes.
And he had a special interest in gas discharge tubes.
Remember Lorentz?
He like to put the gas discharge tubes in a magnetic field.
Stark liked the put them in an electric field.
And what was Roentgen's pet project?
Roentgen loved high voltage.
He loved high voltage an low pressure.
He was focused high voltage, low pressure, near vacuum.
And so let's have a drawing of what he used.
And here's an image of the museum that is literally in the house that was occupied by him.
So the apparatus is sitting right there on the table.
And here's the heart of the apparatus.
I'm going to go to a fresh board because I need a little extra room here.
So we know what the gas discharge tube looks like.
It's a glass tube, sealed.
And we've got feedthroughs for electrodes at both ends.
And this goes down to a power supply.
And the power supply that he used was batteries of chromic acid.
And they were open beakers.
And they were down in the basement two floors below.
And he had wires going from the basement up to the second floor.
And he was able to generate about 20 volts.
So he had a number of these in series.
Because this is roughly a 1-volt system.
So round numbers, you got about two dozen of these things in series.
And I'm going to write here, in the cellar.
Now how did he get the high voltage?
He put in here something that we wouldn't recognize today as an inductor.
He called it a choke coil.
And what the choke coil did, is it would charge up until it got to 35,000 volts.
When it got to 35,000 volts, it would discharge and send a current at 35,000 volts to the cathode here.
And with 35,000 volts, the cathode would send electrons boiling off the cathode, heading towards the anode with the kinetic energy associated with 35,000 volts.
And this discharged at about eight times a second.
So I'm going to write 8 hertz here.
So eight times a second this thing, bam, bam, bam, bam.
And the electrons go firing off here.
And this to make sure he had really low pressure, he had the cell open actually.
And this went to a vacuum pump.
He had almost no gas molecules in here.
Now, the way he did the experiment is, he'd set this thing up, started it going, and this was after dinner.
So on a Friday night after dinner, he was upstairs.
And he wanted to measure emissions here.
And so what he had, you either get graduate students to sit there and you exploit their labor, or what you do is you put up a detector here which was a screen.
So this is either a sheet of paper or a sheet of cloth.
So we call this a screen.
And so you're going to look here.
All right you're looking at this, seeing what's coming out of this gas discharge tube.
And it was a screen up paper or fabric.
And it had been painted with a chemical that would glow if it were struck by a particle, or by a photon of a given energy.
And they used in this case, coated with the chemical barium platinocyanide, so cyanide four times.
So you make an aqueous paste of this stuff.
And you gob it on to the screen.
And when the screen is hit by something, it sparks.
And the latin word for spark is scintilla.
Have you ever heard of someone being described as a scintillating conversationalist?
It means they are a sparkling conversationalist.
So this is called a scintillation screen, a scintillation screen.
And so presumably, every time there's an event, the screen glows.
And so Roentgen starts the experiment.
And it's raining outside.
There's thunder.
There's lightning.
There's a street light as you can see right there.
There's windows.
And the thing is glowing.
And he really can't detect what's going on here.
So he decides to cover the tube.
So he puts a wooden box around the gas discharge tube.
So now the gas discharge tube is enclosed in a box.
And he continues the experiment.
And what he sees is glowing on this scintillation screen.
The scintillation screen is glowing even though the gas discharge tube is in a box.
There was another observation he made.
The student earlier in the day, who had painted the screen with the barium platinocyanide.
I guess he was practicing his calligraphy.
And he left a paper towel on the edge of the bench.
And he painted the letter A on the paper towel.
And the paper towel was glowing with the letter A.
And this thing is enclosed in a box.
So he's saying what's going on here.
So he decides that there's some kind of an interaction between the screen and something going on in here.
So the first thing he does is he says, well maybe it's electromagnetic.
Maybe it's charged particles.
So he takes a magnet.
And he moves a magnet around.
And nothing changes.
He kills the power to the gas discharge tube.
Everything stops.
So it's definitely inspired by 35,000 volts of energy going into the tube.
So then he says, OK I'll take a piece of black paper.
And he puts the black paper in between.
And the screen continues to glow.
So then he takes a book.
And he puts the book in front.
And he sees the shadow of the book.
And he's standing here like this.
He sees the shadow of the book.
And he can see the shadow of the bones of his hand.
So he says, you know I don't think it's particles.
I think it's radiation.
I think it's radiation.
And maybe it's something like that radiation that Hertz has been talking about.
It's some kind of radiation.
But I don't know exactly what it is.
It's mysterious.
And x is the letter of mystery.
I'm going to call it x-radiation.
So he's discovered this mysterious form of radiation.
Does he rush to publish?
No.
He's not going to risk his scientific career by announcing that he's discovered a form of radiation that can see inside the human body.
That would be professional suicide.
So he continues to repeat the experiments through November and December of 1895.
And in January of 1896, he finally publishers.
And it takes the world by storm.
By the way, how did he record this stuff?
He used photographic plates.
And he stored the photographic plates in his lab in a wooden file cabinet.
And so many of the plates were fogged even before he got them to the experiment.
Because of the x-ray, they were going everywhere in the lab.
And he was working there the whole time.
So what happens?
What's the world's reaction?
January 16, 1896, New York Times reports radiation that can see through matter and peer inside the human body.
Peer inside the human body, you say.
Well, why is that?
What's the big deal?
Well, in London, immediately there was a market response.
A manufacturer claimed to be a purveyor of x-ray proof lady's undergarments.
I mean if you can see inside the human body, it's Victorian England.
How modest can you be?
This is a calamity.
In France, of course, they don't care about such things.
So they had a different response.
Remember this is where Daguerre invented halide photography.
So there it's very deep.
It's very existential.
Somebody claimed that with x-rays he could irradiate and get a picture of the human soul.
How French.
But I'm sorry to say it, the wackiest comes from guess where?
The United States.
No lesser an institution than the New York City College of Physicians and Surgeons.
They proposed to use x-rays to project drawings from text books of anatomy onto the brains of students thereby creating, and I quote, an enduring impression.
In Iowa, we plumb new depths.
There was a group out there claiming that they could turn copper pennies into gold.
So this was the response.
But the thing that makes it so fantastic is we take it for granted that if I want to exercise power and exert a force on that can, I don't necessarily have to make physical contact with it.
We accept this as given that you an either have direct mechanical contact, or indirect contact, magnetics, what have you.
But in 1896, this wasn't given.
So the notion that I could exert a force on something from a distance was very fresh.
It wasn't so easy.
But there were benefits of the x-radiation already in middle of 1896.
There was a seamstress who had a broken needle in an industrial accident in Scotland buried into her hand.
This is trivial.
Look, all you need is low pressure in a gas tube, and you can set this up in high school.
It's nothing.
People had this set up in dentist offices.
It's nothing.
So this surgeon was able to find where the needle was, and then use precision to go in and take the needle out of the injured person's hand.
In eighteen 1899 they were already using radiation in treatment of cancer.
So what is the relevant physics?
what's the relevant physics here?
Let's look at the relevant physics.
Well, when we're talking about 35,000 volts, and we're talking about energies in the vicinity of tens of thousands of electron volts, you know that this can't come from emission from an excited electron in a gas molecule.
The gas binding energies are way lower than that.
This must come from the solid.
The electron leaves the cathode, and it crashes into the anode.
Here's the smoking gun.
There's something going on here when the electron crashes into the anode and then gives rise to the radiation.
So let's look inside the anode.
So what's the energy level diagram?
Let's say the anode is made of copper.
So what's the energy level diagram look like?
So we know up here we're going to have energy is 0.
And the energy quantum state is infinity.
Down here is the ground state.
And the energy is the ground state energy.
And we know this is up in the tens of thousands of electron volts.
And then up here there's an E2 for state 2.
And up here is an E3, energy 3.
Then maybe an E4 for energy 4, et cetera.
And I'm going to write not to scale.
This is just to give you a sense.
And so the ballistic electron with, in this case, 35,000 volts of energy, comes crashing in to the anode.
And this is one of these mixed metaphors.
So this is a Cartesian drawing of an electron.
All right?
This is the incident electron.
And its energy is given by the charge on the electron, which is E, and the plate voltage, which is 35,000.
So it's got 35,000 electron volts, which turns into 1/2 mv squared.
It goes crashing into this thing.
And it's got enough energy it can kick out top level electrons, mid-level electrons, and in the extreme, it an eject ground state electrons in 1s.
By the way, the spectroscopists, they don't like numbers.
Remember, this is K. This is L. This is M. This is N. So we can eject K shell electrons.
What happens if we create a vacancy down here?
all?
Of these electrons above are unstable.
If its copper, there's 29 of them.
Now there's 28 with a vacancy.
And they fall down.
And when they fall down from N equals 2 to N equals 1, there's a photon emitted.
And that photon emission is in the x region of the spectrum.
But we don't just get one wavelength.
We get a plurality of wavelengths.
n equals 2 to n equals 1, we get a photon.
There's a possibility to go for n equals 3 to n equals 1.
We'll get a photon of even higher energy or a lower wavelength.
Well, heck, if you've got enough energy to kick out an electron from n equals 1, you've certainly got enough energy to kick out an electron from n equals 2.
So you can get a cascade from 3 to 2, which gives us a photon, or from 4 to 2, which gives us a photon.
Can you see that you're going to get an entire set of lines?
We're going to label them because we don't have a single line.
So what do we label them?
We label them on the basis of the destination shell and how far they traveled.
So this line here is known as a K alpha line.
Why is it a K?
Because that's the nf.
That's the n number of the final state.
And why is it alpha?
Because the delta n equals 1.
It came from 1 shell away.
What's this one?
It's gotta be a K line, because we ended at n equals 1.
But the delta is 2.
It went from n equals 3 to n equals 1.
So this is called K beta.
So there's a K beta line and a K alpha line.
This ends at n equals 2, which the spectroscopists call L. So we call this L. Delta is 1 L alpha.
This is L, but delta is 2.
So this is L beta.
And what I can do, is I can plot all of this on a curve showing the spectrum of this particular element.
And what do I expect to see?
I expect to see a spectrum that plots intensity versus some energy coordinate.
It could be wavelength in this direction, which means that energy increases in that direction.
And which has the highest energy?
The highest energy up here is K beta, n equals 3 to n equals 1.
So there's K beta.
And then close to it is K alpha.
And why do I put the K alpha line higher than the K beta line?
I don't know what the relative amounts should be.
But I suspect that if there's a vacancy, it's like musical chairs.
If there's a vacancy in n equals 1, and an electron only has to go from n equals 2 versus n equals 3, the chances are the journey from n equals 2 to N equals 1 is easier, and therefore more frequent than the journey from n equals 3, even though n equals 3 to 1 is greater energy.
But 2 to 1 is more likely.
And intensity reflects frequency.
The wavelength reflects the energy.
And then over here is L alpha L beta.
Now there's one more piece here.
The precise value here is determined by the identity of the element in the anode.
If it's copper-- and I had to do this as a junior at the University of Toronto.
I spent a whole year doing x-ray crystallography.
And I will know on my death bed the wavelength of copper K alpha radiation is to five significant figures, 1.5418 angstroms.
Now if someone had fallen asleep in this lecture and open their eyes right now, and looked at the wavelength of this line as 1.5418 angstroms.
And I erase the letters Cu.
You take one look at that, and you say copper.
Because only copper has a K alpha wavelength of 1.5418, which means I can use x-ray analysis to determine the identity of a unknown specimen.
And furthermore it's additive.
So if I had a brass, which is copper and zinc, the zinc K alpha line is over here.
And I will have both lines.
And the relative intensities of the lines are related to the relative concentrations of the copper in zinc.
So now I have a tool for chemical analysis.
All of this from going upstairs after dinner and horsing around with the gas discharge tube.
OK, so we're going to end the formal lesson here.
But we have a few more minutes.
So everybody is instructed to not make a lot of noise and just sit quietly.
So let's take a look.
This is an image.
This is an image of Bertha.
This is Roentgen's wife.
This is her hand.
He invited her into the lab.
And she stood for a full 15 minutes in front of this tube.
And in that 15 minutes, got a lifetime's worth of radiation.
But what we're looking at here, and what I'm going to point out to you in the last several minutes here, Roentgen was a prissy physicist, a really prissy guy.
And he was German, which meant that the German academics had no contact with industry whatsoever.
On the night of November 8, 1895, he invented three modern technologies.
This is Bertha's hand.
It represents the birth of medical radiography.
Is there anybody in this auditorium who has never had an x-ray, a dental or body x-ray?
It's very rare.
We see them all the time.
We use them all the time.
This morning I had one taken about 9:10, right here.
And I'm OK.
So that's radiography, medical radiography.
Roentgen had a balance in his laboratory.
The justice, and she's got the torsion balance.
So you put the unknown on one side, and then you put weights that are calibrated.
These are made of brass.
And they're in a wooden box.
And so Roentgen has already covered the tube with a wooden box.
And so he took the emission from this, and he radiating his box of weights.
And what he see?
He saw dark regions where the weights were.
Because metals have a different electron density, because they've got a greater atom density than wood.
Wood is a more open structure.
And so by contrast, he could see inside the box.
This is the basis for airport security.
Roentgen had a gun.
He was not holding up convenience stores.
He was a hunter.
He went into the woods.
And he x-rayed the gun.
Now why did he x-ray the gun?
Now the gun is made of steel.
But deep inside the steel, there could be a flaw.
There could be a crack.
There could be a blowhole.
The surface of the steel looks beautiful.
But inside is a crack.
And that crack could lead to failure under pressure.
So Roentgen irradiates.
And we already know from this that different electron densities give different shading, which means if there were a crack right here, you might see it in the radiograph.
And this we use for certification of pressure vessels today.
Certain pressure vessels that have to be welded, have a specification that requires that every inch of the weld be inspected by radiation, to be certain that there are no cracks deep inside the weld that under high pressure could open up, have an explosion, and cause injury, or in the worst case, death.
So what's the outcome of it all?
In 1901 when the Nobel Prizes were offered for the first time, Roentgen by unanimous vote of the selection committee was the first recipient of the Nobel Prize in Physics.
So on that occasion, he was invited to come from Wurzburg and to deliver a lecture in Stockholm.
I've been to the place, it's a spectacular hall, gilded, beautiful champagne reception, elegant dinner.
And he gives a lecture.
And what's his lecture on?
The discovery of x-rays.
He received a huge amount of money at the time in prize money.
And he returned to Wurzburg What did he do with the prize money?
he?
Donated it to the University to be used for scholarships for students studying physics.
And to this day, students who study physics at the University of Wurzburg are beneficiaries of the generosity of the first winner of the Nobel Prize in Physics.
Such was his humanity.
I hope you have a great weekend.
We'll see the students back here on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get right into it.
So last day, we started looking at Roentgen again and the generation of x-rays, which we saw occurred when we operate the gas discharge tube at high voltage and low pressure and this is the image that we saw of his wife's hand.
Birth of medical radiography.
And we started looking at the origin of x-rays and we started looking at the energy level diagram of the target anode in the gas discharge tube and this is the energy level diagram of the target.
So the ballistic electron-- I'm calling it incident-- this is the electron that's making its journey across the x-ray generating tube.
It left the cathode and it's moving across and crashing into the anode.
So this is the target or the anode.
So let's label it as such.
So we're looking at one of these mixed metaphors, where we've got both a Cartesian image here of the electron and we've got an energy image.
So this is the target or it's the anode and it's charged positively and so the electrons are crashing into it.
And we reason that what would happen is, if the electrons, the ballistic electrons, had high enough energy, they could actually dislodge 1s electrons and when they do so, they make the conditions for a cascade.
So you can see electrons falling from n equals 2 to n equals 1 and when they do so, they emit radiation.
They emit photons and these photons are called K alpha.
K because the final shell number was n equals 1.
And remember, the spectroscopists prefer to use letters.
So when the electron ends at n equals 1, that's n final, we call it a K photon.
And furthermore, the subscript alpha means that the delta, the n initial to n final is only one.
So when you go from 2 to 1, you get the K alpha.
And less likely but still possible is the transition from 3 down to 1.
So it's still called a K photon because the electron that generated the photon ended in the K shell, but it traveled from a delta n of 2, went from 3 to 1 so that's a K beta.
Over here, obviously if we have enough energy to kick out K shell electrons, we have enough energy to kick out L shell electrons.
And so if we lose an L shell electron and we have a cascade, then anything that ends in n equals 2 will be called an L photon.
If it comes from 3 to 2, that's a delta of 1.
That's the L alpha.
It it falls 4 to 2, that's a delta of 2.
That's the L beta.
And so you can see you get a set of lines.
And that set of lines looks like this.
We can plot a spectrum of this entire set of lines, which will be characteristic of the identity of the target.
And I think I showed you last day that it's going to look like this, where we will plot intensity and the intensity on a spectrum is related to the frequency of occurrence and on the abscissa, we're going to have some kind of energy coordinate.
We can put lambda increasing from left to right, which means energy increases from right to left.
And we saw that we have this family of lines where we have a discrete line at the energy of K alpha and a second line at the energy associated with K beta and K beta has a higher energy, therefore, a lower wavelength, and the relative heights related to the relative frequency, again, not to scale, but still the general relationship is correct.
The likelihood of falling from 2 to 1 is higher than falling from 3 to 1 and so you'd expect to have the intensity of the K alpha line greater than the intensity of the K beta line and the L lines have to be of lower energy because transitions 3 to 2 are less energy than 2 to 1 so we expect those lines to be out here and again, a relative frequency where the L alpha has a slightly lower energy and L beta has a slightly lower frequency of a current.
So that's the spectrum and this we call characteristic.
Characteristic of what?
Of the identity of the anode or the target.
This is generating the x-rays.
So if I want x-rays of this value of wavelength, I choose the target appropriately.
That's why we have the relationship here.
And then the other thing is that-- it's obvious, but I just want to make sure that we put it up here-- it's quantized because we're looking at the photons coming at discrete values of energy.
So that's what we speculated on.
Are there any data?
The answer is yes and the data come from a young man by the name of Henry Moseley.
He was doing his PhD up in Manchester.
He was is working for Rutherford.
Rutherford was his PhD thesis supervisor and 1913, 1914, he was making systematic measurements.
He was conducting a systematic study of the characteristic spectra of no fewer than 38 elements of the Periodic Table.
And what did we learn from his measurements?
Learned from his measurements that there's was a pattern here and here's what the pattern is that Moseley found.
He found that if he took the value of-- I'm going to just take one line.
If he took all of the K alpha lines from 1K alpha per element and he plotted the value of the wave number of a particular line, he found that the value of the wave number of a particular line scaled with the identity of the element by the square of the proton number.
So, for example, we would have, say here we started with aluminum.
So aluminum proton numbers 13 so 13 squared.
And he went all the way up to gold.
Didn't do all of them but he did, as I say, 38 from one to the other.
And these are discrete values.
You have different elements here and gold is up here and he found that he could take the set of data for, say, all the K alphas and they lie on a line-- new bar-- as if proportional to the square of the proton number and he did the same with other lines-- say, L alpha and so on-- found that they lie on a line.
So what does all of this mean?
Well, let's take a look.
Here's the image of the paper.
High-frequency spectra elements.
Henry Moseley, Master's degree, so he's working on his PhD, and the data were taken as follows in photographic plates.
And so by trigonometry, you can figure out what the wavelength would be of the line that would go to this degree of distortion, et cetera, et cetera.
It's a beautiful set of lines.
Look at how these date are posed.
So there's calcium.
Scandium is frightfully expensive so you don't see it here.
Then there's titanium, vanadium, chromium, manganese, and as you change the element, the wave number, the wavelength, the energy of all of these lines changes systematically in accordance with the square of the proton number, and we got over here to copper.
Next one is zinc.
Zinc melts at 420 degrees Celsius, which is relatively low melting.
And under the bombardment of electrons-- the bombardment of zinc would cause it to melt and so rather than work with zinc, he instead worked with brass because brass is an alloy of copper and zinc-- and look carefully.
You can see that brass has four lines, two lines identical to the lines of copper and these two new lines are the lines associated with zinc.
And I mentioned last day that you could actually deconvolve the complex spectra and identify the constituent elements.
So this was really fantastic, really fantastic.
Si what did we learn from all of this?
What did we learn by these data?
well, first of all, corrected Mendeleyev What do I mean by that?
Not here to say bad things about Mendeleyev, but Mendeleyev had told us that periodicity is a function of the atomic mass.
That's what Mendeleyev said.
Periodicity is a function of atomic mass.
And now Moseley says no.
Moseley says that periodicity is a function of proton number.
And you can see here-- I wanted to show you the-- this is an image taken from his paper.
He worked for Rutherford.
These people were brilliant experimentalists and so here he's got-- the cathode is up here at the top.
You see cathode of x-ray tube and the feed through and so on, so the anode is down here and it's connected and so on, and rather than take the x-ray tube apart in order to change the target, he went to the toy store and he bought-- this is the train from the toy store and he's got different elements seated next to each other on the flat car of the train and he's got feedthroughs with silk fishing line so that after he's done the experiment with one element, he can pull the train car over and change the anode without having to take the apparatus apart.
These guys were very, very good experimentalists.
So here's from his paper.
The author intends first to make a general survey of the principal types of high-frequency radiation and then to examine the spectra of a few elements in greater detail with greater accuracy.
The results already obtained show that such data have an important bearing on the question of the internal structure of the atom and strongly support the views of Rutherford and Bohr.
It's 1913.
Remember, that's when Bohr paper came out.
All these people were working in the lab at the same time, supporting each other.
You see, this doesn't make any sense with a Plum Pudding Model, does it?
See, he continues: "We have here a proof that there isn't the atom of fundamental quantity, which increases by regular steps as we pass from one element to the next.
This quantity can only be the charge on the central positive nucleus on the existence of which we already have definite proof."
See, not only-- remember, the Plum Pudding Model has this big blob of positive charge nests.
There are no protons.
In the nuclear model, we have a nucleus that has positive charge and he's saying, I know there's positive charge, and it increases as you go from one element to the next.
We are therefore led by experiment to view that N-- we use the letter Z today.
He's using capital N. N is the same as the number of the place occupied by the element in the periodic system.
This atomic number-- for the first time, the term atomic number is used.
That's why I was being coy here and I kept saying proton number, because that's the way they knew it.
Now he's say, this is the social security number of the element.
Proton number is then for hydrogen 1, helium 2, lithium 3, calcium 20, zinc 30, et cetera.
We can confidently predict-- look at-- this is the masters student and he's going on a limb.
He says: "We can confidently predict that in the few cases in which the order of the atomic weights A clashes with the chemical order of the periodic system, the chemical properties are governed by N, while A itself is probably a complicated function of N." He's right.
You look at the Periodic Table.
You say, yeah, yeah, they just go in descending mass or ascending proton number.
Look carefully.
Potassium has a lower mass than argon.
And nobody's to-- well, if you were Mendeleyev, you'd say, go back and measure it again.
Well, they have measured it.
These are the accurate values and no one's going to put potassium underneath neon, but it comes next in the order of ascending atomic mass.
Cobalt and nickel-- nickel actually weighs less than cobalt.
But they're transition elements so who cares if you get those two mixed up?
You find them in stainless steel.
It doesn't matter.
This is a good one.
Iodine is lower mass than tellurium.
Iodine is a halogen.
It belongs under fluorine, chlorine and bromine.
You're not going to put it under oxygen, sulfur, selenium.
But look, there's the data.
And then last one that Moseley couldn't have known about is transuranic synthetic element, but that's just the fourth case in the periodic table.
So that gets you the last wedge in Trivial Pursuit.
What are the four pairs of elements that are out of sequence?
OK.
So now we can say that proton number is the atomic number.
It's the identity.
So that comes out of Moseley, comes out of his work.
The second thing that he did as a result of his study of 38 elements is he figured out what to do with the lanthanides.
Remember our friends, the lanthanides.
Place the lanthanides correctly in the Periodic Table.
Many of the lanthanides have a valence 3 so people were struggling.
They were putting them underneath aluminum.
They didn't know what to do with them.
He placed the lanthanides correctly in the Periodic Table.
And it's not as though these were brand new.
Lanthanum itself had been first isolated in 1839, and all through the 1800s, they were picking them up and finally, lutetium was the last one discovered, categorized in 1907.
But people didn't know what to do with them and remember, they were obsessed with atomic mass measurements.
So I'm going to give you this.
I'll give you this piece of information.
There's the atomic mass of lanthanum, and the atomic mass of lutetium is 174.97.
So what?
I don't know what to do with that information.
That doesn't help me at all.
But now comes Moseley and he says, we're not talking about atomic mass.
We're talking about atomic number.
So now I tell you this atomic number is 57.
So where do you put it?
Duh, you put it right next to barium.
There's no debate.
And furthermore, this one is 71-- atomic number.
Well, if this is 71 and this is 57, I can tell you with impunity-- how many lanthanides are there?
There's 14 of them from here to there.
OK. 14 elements, which makes sense because in s we've got one orbital, in p we've got three orbitals, and d we've got 5 orbitals and these are f and there's 7 orbitals.
7 times 2 is 14.
Everything makes sense.
And the last thing that's a corollary to the item two-- we were able to give uranium its proper atomic number is 92.
So now I ask you, how many elements are there up to uranium starting with hydrogen?
92.
All of this comes as the result of Moseley's experiments.
But it's not over yet because he worked for Rutherford, and Rutherford pushed his people really hard.
He wasn't abusive, but he brought out the best in them.
So who else was in the building?
Bohr, and what were they doing?
They were doing theory and so he said, well, that's nice, but he says, I want those things fit to an equation.
So Moseley said, all right.
I'm going to use-- there's already an equation in the building for a new bar.
It's the Rydberg equation.
So I'm going to use a Rydberg-like equation.
So this is what he does.
This is Moseley's fit.
He goes the wave number of whatever the line is-- whether it's K alpha, L alpha, what have you-- is going to go as the product of the Rydberg constant-- 1/NF squared minus 1/NI squared.
So far that's just the Rydberg equation.
But now comes Moseley's contribution.
We're going to put z squared, but one more piece.
Notice the way I've drawn those lines.
They don't go through the origin, do they?
There's an offset.
So he said, let's allow for the offset-- z minus sigma quantity square.
And this is known as Moseley's Law.
And you've got values for sigma.
sigma for K alpha equals 1 and for L alpha equals 7.4.
So now I can-- with impunity, you give me the wavelength.
I can get the wave number-- it's 1 over the wavelength and I can plug into this equation and identify the element or turn it around if I want to get wavelength radiation of, say, 1.25 angstroms, I can plug into this equation and come up with the z, which will tell me what the target element choice should be.
So let's go and plug this in because we're only going to look at these two.
So we can say that the wave number for K alpha-- it's always going to be 1/2 squared minus 1/1 squared, which is always going to come up 3/4.
So it's 3/4 times Rydberg z minus 1 squared.
So this is for the 2:1 or, if you like, L to K transition and then the other one of interest here is nu bar of L alpha.
nu bar of L alpha's going to be 1/3 squared minus 1/2 squared, which becomes 536 times the Rydberg constant z minus 7.4 squared and this is for the transition 3:2 or KLM, M to L. OK.
So there it is.
And now, what's the significance of all of this?
I try to give a physical significance of this sigma, which is known as the screening factor.
We're going to call sigma here the screening factor.
You'll see why in a second.
Why are we calling it the screening factor?
So let's consider what's happening in the case of the K alpha lines.
Let's look at K alpha generation.
So not to scale.
Let's draw-- here's the nucleus with all of its z positive charges and then I'm going to draw the K shell, and its got two electrons in it, and I'm going to show there's a hole here.
Because without this hole, there's no reason for the cascade.
And then I'm going to draw the L shell and it's got-- what?
There's 2 from the s, 2s, and then 6 from the 2p.
At most.
I know this is a terrible model.
It violates all kinds of things, but it's as complex as it needs to be for the explanation I'm about to give and I don't want to load you down with a whole bunch of extraneous information.
So consider the electrons in the L shell.
They see the electron vacancy in the K shell and they're going to fall down.
What's the Coulombic pull of the nucleus on the L shell electrons?
Can you see that it's not z plus, but at z plus screened by 1 minus.
So the nuclear charge is mediated-- in other words, it's reduced by 1 thanks to the presence of the one negative charge here.
So these electrons in L shell feel z less 1 and hence, we get z minus 1 as screening factor.
It's plausible.
It's at least physically consistent.
Now let's look at the next one.
That's the transition from M to L. OK.
So we get to M. We've got 18 electrons, up to maximum of 18, and if they're going to fall down to n equals 2, there needs to be at least one vacancy here.
So now let's use the same logic.
So an electron in the m shell sees the positive charge of the nucleus mediated by the electrons between the shell n equals 3 and the nucleus.
Now, I don't know how many electrons there are.
What's the extreme case here?
Two, four, six, seven, maybe eight, nine.
Because this would still give me the conditions to generate the transition 3 to 2.
I need a vacancy in two.
I don't need a vacancy in one.
So this is the maximum.
so that would be two, four, six, eight, nine.
So it could be z minus nine, but that would mean that all of these electrons are on the same side and I don't have any vacancies lower.
I only have one vacancy here.
That's an extreme.
It's not observed.
And the other extreme is, we blow away all the electrons.
So somewhere in between 9 and 0, it turns out it's 7.4.
I can't predict that it's 7.4, but 7.4 makes sense.
If the number were greater than 9, I would be distressed.
So it makes sense, rationalizes.
OK. And by the way, if you use this formula, Moseley's Law-- let's bring that back down.
Remember, I told you I can still wake up in the middle of the night and quote you the wavelength of copper K alpha radiation of five significant figures.
I had it drilled into me in my junior year.
It's 1.5418 angstroms.
That's the lambda.
So you can use this formula for new bar and you know that new bar is equal to 1 over lambda.
So I can use Moseley's Law.
But true value, lambda-- I love this one.
Watch.
I'm going to do a triple subscript.
Lambda of copper K alpha.
Isn't that cool?
Lambda of copper K alpha is equal to 1.5418 angstroms.
And if you use Moseley's law, you get 1.546 and the delta here is 1/3 of 1%.
This man was a positive genius.
He was heading pell-mell for the Nobel Prize, but he never got it.
Why not?
World War I broke out in 1914 and Moseley was passionately concerned about World War I.
He wanted to fight.
He wanted to fight for the Allied cause and he enlisted in the Army.
Rutherford was furious.
Rutherford called the Minister of War, which is analogous to the Secretary of Defense and said, give him a desk job.
Put him up in Oxford at a military laboratory.
And Moseley said, no, I refuse.
I'm going to fight.
And so he was sent to Gallipoli.
Gallipoli, as you may know, was one of the bloodiest sites of World War I.
A quarter of a million Allied troops and 1/3 sort of a million Turkish troops were killed at Gallipoli.
Almost 2/3 of a million people died for that little piece of land.
And on August 10th, 1915, at the age of 27, Henry Moseley was killed in the battle of Suvla Bay, and they don't give Nobel Prizes posthumously, and so that was the way it ended.
There's a shot of Henry Moseley with one of his books.
The tributes poured in from all over the world.
The physics community was devastated because they knew all of this stuff.
This is the one that really, I think, says it best.
This was written by Robert Milliken, an American.
He's the one that gave us the elementary charge of the electron, from the University of Chicago at the time.
This is what Milliken wrote-- wrote this to Rutherford to read.
"In a research which is destined to rank as one of the dozen most brilliant in conception, skillful in execution, and illuminating in results, a young man 26 years old threw open the windows through which we can glimpse the subatomic world with a definiteness and a certainty never dreamed before.
Had the European War no other result than the snuffing out of this young life, that alone would make it one of the most hideous and most irreparable crimes in history."
That's when American scientists knew to write.
It's beautiful writing.
What a tribute.
OK.
So let's move on.
Now that we've got Moseley's Law, we've straightened out the Periodic Table.
So we keep moving.
And I say, well, I gave you the drawing of the spectrum.
Remember the spectrum?
The spectrum was here.
You see it?
I'll put it up real quick again.
This was the spectrum.
OK.
This is intensity and this is wavelength and this is K alpha and this is K beta and this is L alpha and this is L beta.
These are the data coming from the x-ray tube.
This happens to be a molybdenum target.
Well, it doesn't quite look like what I've drawn, does it?
It looks a little bit like it, but not quite.
This is definitely there.
You can see some of these lines so I'm going to call this spectrum A and it looks like spectrum A has been added on top of something else.
I'm going to call the something else spectrum B and the spectrum B looks like this.
And in New England-- this curve has a shape because it's New England.
This is called whale-shaped.
I don't know what they call it in the rest of the country, but it even looks like a whale.
So you can go on a whale watch.
So it starts-- it's a sharp front, goes straight up, hits the maximum and then-- are you ready for this?
It tails off, all right?
Whales have tails, yes.
So what's the difference here?
Well, the spectrum A, we already observed is quantized, whereas this one isn't.
This one's continuous.
It has continuous values up to this minimum value or, if you like, maximum value of energy, minimum value of wavelengths.
So we got that.
It's not quantized.
And the other thing that's interesting is-- spectrum A, we said, is a function of the identity of the target.
z of the target, whereas this one, it's a function of the plate voltage.
So you can see in the diagram I've shown you, as the plate voltage changes, we get a series of enveloping curves.
So this is V1 and this is V2-- greater than V1.
So this whale-shaped thing is somehow related to plate voltage.
And then beyond a certain critical value, can you see when you're down here at 15 or 20,000 volts, all you got is the whale-shaped curve?
But somewhere between 20 and 25,000 volts, you hit a threshold and now you switch on the characteristic lines.
So with low voltage, you don't get the characteristic lines.
You only get the whale-shaped curve.
And then beyond a certain critical value of V, it's as though all of a sudden these lines appear.
Low voltage, no lines.
High voltage, you get the characteristic lines.
And what can we do here?
Well, we can compute K alpha, L alpha by Moseley's Law.
The continuous spectrum can't do anything with that, with one exception-- here.
So let's look inside and figure out what's going on.
We have a proposal here of what's going on inside that target atom.
So I'm going to make-- this could be the molybdenum target up there.
So these are molybdenum atoms.
And just as in Moseley's figure-- so this is body centered cubic-- I can look that up on the Periodic Table.
I know this is BCC crystal structure, molybdenum atom sitting here.
This is the anode.
It's charged positive and way up top, I get the cathode.
So the cathode is shooting off ballistic electrons and they go zooming across the gap and crash into anode, but up until now, we've just said the anode is some material.
Now we're going to take one more peel off the onion and say that these are discrete atoms.
It's not continuous molybdenum.
It's molybdenum atoms.
And what do we know the molybdenum atom looks like?
It's got a dense nucleus where all the positive charge resides and then there's this almost vacuum-like zone with the negative charge.
So when the electron comes in, this electron sees the negative charge around the molybdenum atom.
What happens?
It's deflected.
Maybe it comes in on a closer angle and it's scattered through a higher angle.
Maybe it comes in almost in between and it hardly moves at all.
Now can you see that when you have a charged species that changes direction, that's called an acceleration, and an acceleration gives rise to an emission of radiation?
So because the angle of deflection-- so this is low-angle deflection, this is high-angle deflection.
So low-angle deflection means low energy emission.
High angle means high-energy emission, and the result is this continue spectrum that I've shown you here.
So this is the result of low-angle scattering of the ballistic electrons.
This is the result of high-angle scattering of the ballistic electrons, and somewhere in the middle here is the dominant angle.
I can't calculate this curve with one exception.
Imagine the electron comes from the cathode, and with all of its energy is dead on and stops, gives up all of its kinetic energy to a photon.
That's the maximum amount of energy possible.
Let's look at that.
That's the case where an electron comes in, stops dead here and then emits the photon.
Sp the kinetic energy is translated into the photon energy.
So we can do that one.
That's a straightforward calculation.
So the energy of the incident electron-- E of the incident electron is equal to product of the charge on the electron and the plate voltage.
Well, the charge on the electron is the elementary charge and the plate voltage is whatever it is, and I'm going to equate that with the energy of the emitted photon, and that's equal to hc over lambda.
So now I can cross multiply and I can call this the lambda of the shortest wavelength.
That's the shortest wavelength on the whale-shaped curve.
So by algebra, you're going to get the product of the plane constant times the speed of light divided by the elementary charge times the plate voltage, which turns out to be 12,400 divided by plate voltage where the wavelength's given in angstroms.
Just try it.
If you put 10,000 volts, you're going to get 1.24 angstroms, which is smack dab in the middle of the x-region of the spectrum, and this is called the Duane-Hunt Law.
That's the only thing we can compute in the continuous spectrum.
So I can get this one.
Lambda shortest wavelength, because shortest wavelength is maximum energy.
Now there's a fancier name for this whale-shaped curve, and it's the scientific community's term, and it's a German word.
It's called bremsstrahlung.
I love it.
You need to know this.
You can impress your friends at parties.
What does bremsstrahlung mean?
Brems is the German word for brake, as when you put on the brakes of a car.
And strahl, strahl is the word for ray.
And ung is like ing.
So this is raying radiation and this is the radiation of braking.
So the electrons are coming in and when they come up against the negative charge of the outer shell the target atom, they are slamming on the brakes and skidding everywhere.
So this is what bremsstrahlung means: braking radiation.
So we're going to call-- I'm going to put a B here-- see that?
I was thinking ahead.
B is bremsstrahlung.
OK.
So we've got a lot going here.
for us.
I think we've explained a fair bit.
Now I want to talk about modern x-ray tubes.
What do modern x-ray tubes have that these primitive ones that Moseley worked with and Roentgen worked with didn't have?
So I want to show you that the modern x-ray tube is the result of improvements made by an MIT alum.
His name was William Coolidge, and he's the class of '96-- 1896.
And he made a number of improvements.
He actually taught for awhile then eventually spent a good part of his professional career working as a research scientist at the General Electric Labs out in Schenectady.
If you go down to the lobby of Building 6, on the south side of the lobby, there's a showcase.
Look inside the showcase.
You'll see there's a little display in honor of Coolidge.
So what's the first thing Coolidge did?
He was an engineer so he was thinking about making things more efficient.
First thing he did is he turned the discharge tube into a vacuum tube.
Remember, Roentgen worked at low pressure, but there was still gas, and he was blinded by the light and so on.
This means you don't get any visible light, and secondly, it's more efficient because if you've got the tube like this with the two electrodes, the feedthroughs and you've got gas inside, some of the electrons are crashing into the gas molecules, and we don't care about the gas molecules.
We want to get the electrons crashing into the target.
So by going to vacuum, this improves the efficiency.
No glow in the visible and higher energy efficiency.
More x-rays out per unit power put in.
What's the second thing he did?
Second think he did was hot cathode.
Remember, you're trying to rip the electrons out of the cathode and send them on their journey.
So Coolidge reasoned that if you heated the cathode, you'd weaken the bonds, and the electrons would come off.
They'd boil off much more readily, OK?
Raise temperature to reduce bond energy, to reduce binding energy of the electrons.
The electrons in the cathode makes them easier to boil off.
Think I've got a cartoon of that.
Yeah, here it is.
So here's the tube lying on its side.
So the cathode is over here to the left.
It's negative.
Here's the anode to the right.
That's this purple thing here and the electrons are moving from left to right and so he's got-- see, you can have multiple electrical signals going through the same conductor.
It's not the same.
You can't have an AC waveform and a DC waveform in the same conductor.
So you've got a big DC voltage between the cathode and the anode and you'd got a separate little circuit going through the cathode, running almost like a toaster to make it super hot.
So the potential between here and here is 35,000 volts.
The potential along this stretch of real estate might be several hundred volts, and furthermore, this could be an AC signal.
It's Coolidge.
He was smart.
So this makes this hot and now for per given voltage, you get much more yield of the electrons.
So that was pretty good.
I like that.
Smart.
Third thing he did was he heated the cathode and he cooled the anode.
Water-cooled anode.
Why?
You got to dissipate the heat.
All these electrons crashing into the anode.
You raise the temperature of that anode so high you'll melt it and so they had to run the tubes intermittently.
They just get a decent signal.
You have to take these x-ray measurements over a long period of time because the amount of x-rays you get is small, but you have to keep shutting the thing down.
Otherwise, you melt through the anode.
So he put that on a water-cooled copper hearth and was able to run continuous.
Continuous operation.
No more pulse current.
But sometimes when nature hands you a lemon, you make lemonade.
So I'm going to turn this thing around.
I'll say, hey, wait a minute.
I don't want any water-cooled copper anode.
Suppose I want to weld some titanium.
Titanium melts at 1675 degrees Centigrade and it's got a voracious appetite for oxygen.
Well, this thing's a vacuum tube.
So what if I were to take my part and I put the two pieces of titanium in the path of an electron beam and I don't cool the titanium?
Eventually, I get the temperature so high that I can weld titanium.
This is the birth of electron beam welding.
And that's how you weld refracting metals.
You get one of those titanium bicycle frames that cost about $4,000.
It might be tungsten inert gas.
It might be electron beam, depending.
So that's the flip side of the technology.
By the way, if you were in attendance observing electronic beam welding, what do you think is being generated in the electron beam welding apparatus?
X-rays, by the boatload So there's a fourth thing-- maybe the most important thing that Coolidge came up with-- shielding.
You see the yellow here?
That's lead shielding.
Why did he choose lead?
Well, it's got a very high z.
That means it's got many energy levels.
So that if you start looking at the energy level diagram of lead, you've got lots of action up in here.
So what can happen?
When the x-rays come with their high energy, the x-rays from the tube on their way out to get you, and everybody standing observing this marvel.
They excite electrons inside.
So these x-rays are absorbed.
The electrons inside the lead rise, cascade down and now they come out.
And what's the difference in energy between the x-ray and these?
The ones up here are much lower energy.
So this is, if you like, a frequency shifter or an energy shifter.
So now we're using photons to excite electrons to generate photons.
And that's how the shielding works.
So you want to absorb and re-emit.
And while we're on the topic, he gave us lead shielding and he gave us beryllium windows because they were using silicate glass, just as you use in your home, but the silicate glass was absorbing some of the x-rays.
So to get higher efficiency, he chose beryllium.
Why did he choose beryllium?
Well, it's got a low z.
So that means it has few energy levels.
So therefore, there's less absorption and re-emission-- again, higher efficiency.
I think this was taken from one of the other readings.
So they flipped it around.
See, here you can see the cathode here, the anode.
That's one of these things.
All right.
So the window is up here.
You don't use lithium because lithium is unstable in the air unless you had a giant room of humidity set at a dew point of minus 38 degrees, see, and then you can use a lithium window.
Otherwise, you use beryllium and lead down here.
Why'd he choose lead?
Well, you're pretty much at the bottom of the naturally occurring elements in the Periodic Table.
So it's the cheapest of all of these heavies.
All right.
So now let's take a few minutes and talk about the use of x-rays in characterizing art.
So this painting here at once time was arguably one of the most recognizable paintings in the western world.
It's The Angelus by Jean-Francois Millet.
It was painted at 1857 to 1859 on commission for an insurance company here in Boston.
The Boston Brahmins loved BA's paintings of rural life in 19th century France.
And this is couple of peasants who are giving thanks to God.
The Angelus is a prayer, and they're thanking God for this pitiful bounty of potatoes evident.
Salvador Dali, as an art student, was required to paint this as part of training.
You know how when you learn to play the piano, you reproduce the works of the Grand Masters.
So when you go to art school, you have to paint.
He hated this painting.
For many years, it was here in Boston, and then it was repatriated around World War I, it hung in the Louvre, and then ultimately, now it's in the Musee d'Orsay.
In 1963 when Dali was at the peak of his career, he asked the curator of the Louvre if she would have this painting x-rayed.
He said this painting spooks him.
He doesn't like this painting.
And they x-rayed the painting.
What did they find?
They found that it had been painted over down here.
Wasn't an art forgery, Millet himself had painted it over.
You may have heard two years ago there was this art find.
It was a van Gogh painting.
They found that van Gogh had painted over one of his own paintings that had never been seen before even though the pencil sketches survived, so they assumed that maybe the painting was destroyed in World War II or some such thing.
It turns out that van Gogh was so poor at one point that he valued the canvas over the art, and he took his previous painting, and he painted over it.
OK.
So this was going on here.
This is the truth.
Now my speculation begins.
So what was underneath here initially?
It was the casket of a baby.
Look at the pose.
These people don't look like they're happy with their bounty of harvest.
They're grieving.
This was this futility of peasant life.
It was a hard life and they lost their baby.
So imagine Millet paints this, puts it on a boat, it comes over to Boston, they unveil it, and the directors of the insurance company go, we can't hang this in the lobby of an insurance company.
That's my speculation.
It didn't take him three years to paint this thing.
I think it spent most of its time on the boat.
They sent it back and said fix it.
So he put the basket of potatoes.
That's my theory.
All right, look at the pose.
This is from the Galleria Borghese in Rome.
I was there about two years ago and saw this and went, wow, referential.
You see, everything's been said.
All of art has been said.
Now we're just recasting it.
So now here's Dali's revenge.
Now this is what he paints.
You see the man is shorter than the woman.
His hat is down a little bit below the waistline.
There's all sorts of psychosexual things going on here, but we're running out of time.
So here's Dali and his father and his dad is saying, see, this is life, et cetera, et cetera.
So you can see the reference.
See, man taller here, woman taller, et cetera, et cetera.
Have you seen this one?
The Hallucinogenic Toreador.
OK, he went outside one day in Manhattan to buy some pencils.
There was a company called the Venus Pencil Company, you see.
And so he came back and made this trompe l'oeil.
So this is the toreador.
Do you see the breast here is the nose?
There is the face.
There are the flies of the Thames and here is the cape and so on.
Symmetry plane-- all of those Venuses facing back, these Venuses facing forward.
There's the bull.
The life force of the bull is a crystal.
What is the crystal structure?
It's one of the 14 Bravais lattices.
If you ignore color, it's simple cubic, isn't it?
And if you don't believe me, check.
And if you don't appreciate it, study.
Here's the symmetry plane.
Atoms, atoms, fly, fly et cetera, et cetera.
What else do we have here?
Oh, this is the one he painted on the occasion of the revelation of the structure of DNA.
It's hard to see, but over here is the double helix of DNA.
This is his wife Gaia and here are people standing in cubic arrays with guns pointing at each other.
And the point that he's making is that now that we've discovered the instruction set for reproduction of life, not just human life, but life, we are still at a point where we can't resist killing each other.
So this was the point and he chose to use cubic arrays, and I would venture to say this is simple cubic.
If you put a fifth person here, it would be-- are you ready for this?
Body-centered cubic.
Oh, yes!
And there's more, but I think we're running out of time so I think at this point we will dismiss the class.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK, Settle down.
Settle down.
It's 11:05.
It's time to learn.
All right.
Last day we were looking at the emissions spectrum of the target in the x-ray tube.
The target is the anode, and it's being a bombarded by electrons that have been accelerated across a potential difference of some tens of thousands of volts, and this was what we saw as the output, intensity versus wavelength.
We see this whale shaped curve which we have attributed to bremsstrahlung, which is the breaking radiation.
And then if the voltage is high enough, high enough to do what?
High enough to eject inner shell electrons, we will get the cascade, and the cascade will give rise to specific values of wavelength associated with transitions within the target atom.
There's too much talking.
I want it absolutely silent or else somebody is going to leave.
That's the only way it works here.
I talk.
You listen.
And if you have a problem with that, there's a door here.
There's a door there.
And there's a door at the back.
So let's get it straight right now.
It's the only way it works here.
Now what happens is that these characteristic lines are calculable in part.
The K alpha line, the L alpha line we saw from Moseley's law, which is up here on the slide.
and the leading edge of the whale shape curve, the bremsstrahlung is calculable by the Duane-Hunt Law.
The rest of this is not calculable.
So there it is.
What I want to do today is harness this radiation.
The reason we were studying x-rays in first place was that we said they had a length scale that was comparable to that of atomic dimensions.
And so now we want to close the circle here and probe atomic arrangement by x-ray diffraction.
And it goes by this three-letter initialization XRD.
And in order to use x-ray diffraction to probe atomic structure, we're going to make another model.
And this model is going to be simple.
And it's going to be inaccurate, but it's going to be good enough to explain the phenomena that we're going to study.
So the first thing I want to do in constructing a model that will allow to use x-ray diffraction, is to model the atoms as mirrors.
That's the first assumption that we make.
We model the atoms as mirrors.
And when we model the atoms as mirrors, this means that we will invoke the laws of specular reflection.
So if this table top is a plane of atoms, and I have a beam coming down at an angle, the laws of specular reflection say that the angle of incidence will equal the angle of reflection.
So that's all we're doing by modeling the atoms as mirrors.
Laws of speculative reflection apply.
So that simply means that the angle of incidence, theta incidence equals theta reflection.
And you'll see how that comes into play in a minute.
And then the second thing we're going to do in order to get intensities is to use the concept of constructive and destructive interference.
OK.
So we apply interference criteria.
And I will explain what that means in a second with a diagram.
But in dense text, it simply means that in phase rays-- it sounds like this rain in Spain falls mainly in the plain-- in-phase rays amplify.
And out-of-phase rays dampen.
So at some point, you'll study this in physics, but we need it right now, so I'll give you a little bit of foreshadowing, and then you'll be ahead of the game when you meet this in physics.
So what I'm going to do is I'm going to show you two rays, and they both have the same wavelength.
And I want to show you in phase.
And so I want to illustrate this concept here, in-phase rays amplify.
So let's give one ray here.
I'm going to depict it as a wave.
All right, so that's ray number 1.
And beneath it, is ray number 2.
It's got the same wavelength.
So that means crest-to-crest distance is the same, and if it's in phase, in phase means that crest lines up with crest, trough lines up with trough.
So I'm going to make another one seemingly identical.
It's supposed to be identical to the capacity of my ability to draw.
And so what this means is that these will amplify.
So 1 plus 2, 1 plus 2 will give me something that looks like this.
I'm going to line it up again.
Only now the amplitude is double the amplitude of a single ray.
That's what I'm trying to depict here.
OK.
So this is amplification.
Now over here, I want to do the same thing.
Only I'm going to have two rays, same wavelength, but out of phase.
OK, so in phase is here, out of phase is to the right.
So we'll have ray number 3, which does this.
And ray number 4, I'm going to line up with the crest of ray number 3, the trough of ray number 4, but it's going to be the same wavelength.
So the distance between successive crests is the same.
Only they're going to be offset by exactly half wavelength.
So the crest here of wave 3 lines up with the trough of wave 4.
The trough of wave 3 lines up with the crest of wave 4, and so on, and so on, and so on.
And if I drew this thing accurately, it would be obvious to you.
But since I can't draw very well, then we're going to take 3 plus 4.
And the sum of the crest and the trough is 0.
The sum of the trough and the crest is 0.
And so the sum here is 0.
All right this is total destruction.
This is cancellation.
So in the extreme, you can have destructive interference that dampens to the point of obliteration.
OK, so that's the two extremes, full amplification and complete cancellation.
So now, I want to take this concept of planes as mirrors and rules of interference criteria, and now what I want to do is illuminate.
And I want to use the full board here.
So here's what we're going to do.
We're going to take in phase, coherent-- that's what coherent means.
Coherent means that the radiation is in phase and monochromatic.
What's that mean?
Monochromatic means one color, which means single wavelength.
So I have something of a single wavelength in phase.
Coherent, monochromatic, incident radiation, so I've got a plurality of rays.
I have this thing flooded by a beam.
So what I'm going to do, is I'm going to draw some atoms here.
And I'll put a second batch on top.
And now I'm going to model these as mirrors.
So I'm going to have the incident beam come in like so.
The incident beam comes in like so at an angle theta.
So this is theta incident.
This is the angle of incidence.
And according to our model, the beam is reflected at the same angle.
So this is theta, reflection of theta.
Reflection equals theta incidence.
So that's the first thing that we do.
And then the second thing we do, is we use the laws of interference.
So I'm going to take a second beam, and it's going to go down to this atom here.
So I'm going to bring it down like so.
And it's going to come in at the same angle.
And it's going to leave at the same angle.
And if you'll excuse the drawing, these lines should be parallel.
They should be coming in parallel, and they should be leaving parallel where these angles theta i and theta r are the same.
Now how do we invoke the interference criteria?
Well what I'm going to do is put a marker here.
I'll put a marker.
And I'll call the first one ray number 1, and the second one ray number 2.
And what I'm going to show is if I start from ray number 1-- maybe it's time for colored chalk-- so if I start from ray number 1, to show that it's in phase with ray number 2, I'm going to draw this little waveform, and they're both lined up.
So you can see crest lines up with crest.
Trough lines up with trough, and they both have the same wavelength.
So that makes the point coherent, monochromatic.
But now things get interesting, because you can see that ray number 1 travels a shorter distance than ray number 2, and we can put on some coordinates here and mark the geometry, so we can say that the point at which the ray number 2 has to start traveling a longer distance, I'll drop a normal down, and I'm going to call this point A.
This is point A.
The bottom here is point B.
And then up here where it catches up with ray number 1, the reflected ray number 1, I'm going to call this point C. So you can you can see that ray number 2 has to travel this extra distance, AB plus BC.
And now, if I want to get out to here where I'm now in the reflected zone, and I want ray number 1 reflected and ray number 2 reflected to continue to be in phase, there's a geometric constraint on the dimension of AB plus BC, isn't there?
That length, AB plus BC must be a whole number of wavelengths.
It must be an integer number of wavelengths of whatever this thing is, otherwise, these two will not being in phase.
So that's the interference criterion.
And n can equal 1, 2, 3, and so on, so this is called the order of reflection.
And then we can go through the trigonometry.
I'm not going to do it in class.
It's just a waste of time.
It'll bore everybody to tears.
But if you want to do that at home at some point when you've got nothing to do, you can go through.
You've got all the numbers here.
You know the value of lambda.
You know the distance AB, and you know what this distance is.
This distance between successive planes is your dhkl, isn't it?
So if you go through all of this analysis, you will show that n lambda is equal to 2 times the d of the hkl spacing times the sign of theta.
And this is called Bragg's Law.
And Bragg's Law governs the reflection of incident radiation by a crystal.
Now you'll notice I didn't put an apostrophe here.
Some people put the apostrophe here, because the man's name was Bragg.
Some people put the apostrophe here because, in fact, it was father and son.
And they both together won the Nobel Prize in 1915.
Now there have been people who lived to see, as Nobel Prize winners, one of their children win the Nobel Prize.
but this is the only time in history a father and son team together won the Nobel Prize for the same work.
And there are undoubtedly people sitting in this room who think the fact that the father and son could work together in physics for an extended period of time alone is deserving of the Nobel Prize.
Now in 3.091, I'm going to keep it simple, always choose first order reflection, always n equals 1 in Bragg's Law.
So therefore, we will write Bragg's Law as lambda equals 2d sin theta.
And to make the point, the d is specific to a particular set of planes.
So it's a d spacing of the hkl planes.
And it's the theta associated with the correct reflection of the hkl planes.
Now how does destructive interference come into play?
Destructive interference comes into play should there be a situation where I have a third ray.
I'm going to bring a third ray in also at the same angle of incidence.
And that ray is halfway between these two rays.
And what situation could that be?
That could be a situation in which the crystal structure has in a plane out from the board, an atom that sits halfway between these other two atoms.
And if that happens, you can see.
We can go through the derivation.
But there's going to be a path length difference that's exactly 1/2 wavelength different.
That's bad.
It's not just there's going to be some measure of reduction.
It's going to be total cancellation.
And so by going through the set of crystal structures and recognizing that when you have 1/2 wavelength difference, you get destructive interference.
And therefore, you will see no line.
The incident radiation will come in at this angle, and at the reflection angle for that particular plane, there will be nothing detected.
So you can say that you have a combination of selection rules that involve the integral of both the rules of interference and interaction with the crystal structure.
So let's generalize this and say if we take interference criteria plus the crystal structure, that is to say, the instant relationship of atom positions for a particular specimen, the combination of that will give rise to the set of expected reflections.
So only when you satisfy the Bragg criterion do you get reflection.
So you move the specimen.
And only at that special angle will you get the reflections, will you get constructive interference.
So this, in fact, is a fingerprint.
This is a fingerprint of the crystal.
In this case, we're not getting the chemical identity.
We're getting the structural identity.
So we can determine if something is BCC, FCC, and so on.
So for example in BCC, that's exactly the case that I just illustrated.
So in BCC we have atoms at the eight corners.
And we have an atom dead center.
There's a central atom.
And the central atom lies in 0 0 2 plane, doesn't it?
The base of this is 0 0 1.
And the top is an 0 0 1.
But the central atom is an 0 0 2.
And 0 0 2 is halfway between successive 0 0 1.
So can you see that light that comes in?
And it's going to be constructively reinforced.
Off of 0 0 1 is going to be destructively reinforced off of 0 0 2, and the result is that in BCC, no 0 0 1 reflection observed.
The same thing happens in face centered cubic, right?
What's the face of face centered cubic look like?
It looks like this, doesn't it?
So there's an 0 0 1.
Here's an 0 0 1.
What about this space centered atom?
Where is it?
That's an 0 0 2.
It's halfway between 0 0 1.
So at the angle where you would've expected, if you use the Bragg Law and calculate the angle at which for your particular crystal, because you know the d spacing.
You fix the wavelength of your x-radiation coming out of the generator.
At the angle where you would expect to see reflection off of 0 0 1, you will see nothing, because 0 0 2 is canceling 0 0 1.
So you don't have to worry about all this.
This has all been tabulated for you.
Somebody has gone through and done all of this.
Well, this is just making the point.
See, there's a simple cubic.
All the planes are reflecting.
There's body-centered cubic.
That's a/2.
There's face-centered cubic a/2.
0 0 2 is going to cancel 0 0 1.
You're not going to see anything there.
So people have gone through.
And they've made this set of rules for reflection.
So simple cubic, you get reflection from all the planes.
Body-centered cubic, you just get these planes here.
And it turns out that there's a simple rule that compactly represents which planes are going to reflect in body-centered cubic.
And in BCC, the hkl, it's the planes for which h plus k plus l is an even number.
h plus k plus l is even.
And 0 was counted as even.
So for example, 0 0 1 h plus k plus l is 1.
It doesn't reflect.
0 0 2, 0 plus 0 plus 2 is 2.
It does reflect.
And so on.
You just go down the whole line.
And so these are in ascending order of h squared plus k squared plus l squared, because that's a nice way of deciding how to add them up.
And now in FCC the selection rule is a little bit different.
In FCC you get reflection from planes when you write the HKL such that h, k, and l must be either all even numbers, or all odd numbers, or some people like to say unmixed.
Some crystallographers say unmixed, meaning you can't have a combination of even numbers and odd number.
So for example, 0 0 1 won't work because 0 is a zero.
That's even.
1 is odd.
But 0 0 2, h plus k plus l all even or all odd, there you go all even or all odd unmixed.
So this will work.
And then so on.
So you can go through and see which ones work.
All right.
So the next one here is in the sequence.
This sums to 1.
And what would sum to 2?
It would be 0 1 1.
And 0 1 1 doesn't work either because zero is even.
And 1 is odd.
The next one of the sequence is 1 1 1.
These are all odd.
So that one reflects.
So in FCC, you don't see 0 0 1, 0 1 1.
But 1 1 1 does reflect.
And then 0 0 2, which is the next one, because 0 squared plus 0 squared plus 2 squared is 4.
This is even even.
So that one works.
So there's the sequence.
And so now what we can do is use this technique in order to make measurements.
But before we do so, I want to show you the experimental measurement, one way.
There's several ways of conducting the measurement.
And so the first way I'm going to show you is called diffractometry.
And you can do this over in Building 13.
If you get yourself a UROP, you might be assigned to make some measurements.
So diffractometry is a form of x-ray diffraction.
It's one of the techniques.
And the way it works is you fix the wavelength and vary the angle of incidence, you don't rotate the x-ray generator.
You rotate specimen and present continually varying angles to the thing.
So this shows the technique in operation.
So you have a specimen sitting here.
The specimen can either be a thin film, or in other instances, we have a finely ground powder.
So each of the powder grains presents a different angle.
And coming out of the collimator is the monochromatic.
We want a single wavelength.
So this collimator means it's monochromatic and coherent.
So that beam comes and strikes the specimen.
And for historical reasons, instead of calling this the angle of incidence, and this the angle of reflection, they call the angle that goes to the detector 2 theta.
In other words, the projection of the beam, this is theta incident equals theta r. So this is really 2 theta incident, or it's 2 theta reflected.
It's the same thing.
So you'll see x-ray data usually reported in units of 2 theta.
And here's the detector.
And then all you do is you rotate the specimen.
And by rotating the specimen with a detector, you're able to get the entire set of reflections.
And whenever you move through an angle that satisfies the Bragg Law, you get a peak in intensity.
And here's what the output would look like.
So you're plotting intensity as a function of 2 theta.
So somewhere along here at around, it looks like about 38 degrees, you have satisfied the Bragg angle, and you get constructive interference.
When you move off of 38 degrees, destructive interference reigns supreme, and you get almost no reflection.
Then you get to the right angle for 2 0 0, you get reflection.
Well, all you get is these lines.
You don't get these numbers.
These numbers you don't get for free.
So here's the experiment that we're going to do.
We're going to run our x-ray generator with a copper target.
Why do I tell you the copper target?
Because you're going to fix the wavelength.
And you fix the wavelength by choosing the target.
So we've got copper target in our x-ray generator, OK?
So remember, the sample is not the target.
The sample is what is being irradiated.
This is being bombed by the electrons in the tube, copper target in x-ray tube, and that means that the lambda of the radiation is lambda copper.
And I'm going to use lambda copper K alpha because I know it's wavelength to five significant figures.
It's 1.5418 angstroms.
And here's the data set.
I looked this up in the literature.
Here's the data set from the experiment.
And that's all you get, a set of 2 thetas.
So here's my challenge to you.
I'm telling you you've got an unknown sample that's cubic, and there's your data set.
The task for you is determine two parts to the question.
And we're going to do it together.
First part, determine the crystal structure.
So it's either FCC, BCC, or simple cubic.
And the second part is determine the lattice constant, the value of the lattice constant.
So you can get quantitative measurements.
So I'm going to show you how to do it.
So how are we going to do it?
We're going to use this technique.
This is my self-help book for you.
And here's the key.
Here's how I came up with this.
There's a way to unravel this.
And the way you unravel this is to take these two relationships.
You have lambda equals 2d sin theta is 1.
And you also know that d-- in fact, I'm going to keep writing hkl subscripts here-- you know the dhkl is equal to a.
That's this lattice constant.
Be sure this is lattice constant.
This is the cube edge measurement lattice constant divided by the square root of h squared plus k squared plus l squared.
So what I do is I combine the two.
If you combine these two, you can end up with this relationship.
If you combine the two, you get lambda squared over 4a squared equals sin squared theta over h squared plus k squared plus l squared.
And this is the key.
Why is it the key?
The value of lambda is set by you.
You chose the target.
The value of a is set by the sample.
That's the lattice constant.
So the ratio of two constants is a constant.
Agreed?
So this is a constant.
So if the left side of the equation is a constant, the right side of the equation must be a constant.
But h, k, and l vary.
And theta varies.
But the ratio of the variation, when mapped into sin square theta over h squared plus k squared plus l squared, this must be a constant.
And that's going to be the way I work through this delightful problem.
So let's see.
Start with 2 theta values, and generate a set of sin squared theta values.
That's what Sadoway says first.
So I took 2 theta.
And all I did was make 1 theta, and then took the sin of it, and then took the square the sin.
So this is the data set transmogrified into sin squared theta.
All right, what's the next thing he says?
Normalize by dividing through with the first value.
So I'm just going to take this whole series and divide it by 0.143.
And now I end up, instead of 0.143, 0.191.
I have 1, 1.34, and so on.
That's what I mean by normalize.
Normalize to the first entry.
OK, What's next thing he says?
Clear fractions.
Clear fractions from the normalized column.
So I'm going to multiply this by a common number.
Because 1.34, remember this is experimental data.
It's noisy.
1.34 looks like 4/3.
Doesn't it?
It's fuzzy logic.
And 2.67 looks like 2 and 2/3, 3 and 2/3.
That's roughly 4.
5 and 1/3.
The 3 looks like a magic number.
So if a multiply this column by 3, I get 3, 4, 8, 11, 12.
What does it say next?
Speculate on the h, k, l values, that if expressed as h squared plus k squared plus l squared would generate the sequence of the clear fractions column.
And then that's going to take me to the selection rule.
So I say, well how do I get 3?
It's 1 plus 1 plus 1.
4 is 2 squared plus 0 squared plus zero squared.
8 is 2 squared plus 2 squared plus 0 squared.
So I'm generating this thing.
And now it's pretty obvious, right?
Because now I've got to use these.
And what do I see here?
Well, 1 1 1 are all odd.
2 0 0, all even.
2 2 0, all even.
Gee, this looks like it conforms to the selection rules for Bragg reflection from an FCC crystal And then to make sure that I'm on to something, what I do is this.
Compute for each theta the value of sin square theta over h squared plus k squared plus l squared on the basis of those assumed values.
What I'm doing is I'm saying if my hunch is right, whatever I choose for the theta and the assumed value h squared plus k squared plus l squared, that should be a constant that doesn't change.
So let's test it.
And sure enough, look.
0.0477, 0.0478, so it looks like I'm on to something.
And furthermore, I know what this is.
That 0.0477 is this.
This is 0.0477.
And I know my lambda.
That's 1.5418.
So now I can calculate my a.
So now I've determined that it's FCC.
Plus if I go ahead and I calculate the a value, I get 3.53 angstroms.
And if it's FCC and it's 3.53 angstroms, I bet it's nickel.
So that's how you index this stuff.
So there we are.
There are the selection rules.
Now here's a little trick.
Let me show you.
Because when I go to index this, the first thing I do is I go for low hanging fruit.
If you start looking for whether it's simple cubic, face-centered cubic, or body-centered cubic, let's look at the sum.
So I've got simple cubic, body-centered cubic, and face-centered cubic.
So the first plane on simple cubic, it's going to give me everything.
It's going to give me 1, 2, 3, 4, 5, 6.
And FCC, what does it give us?
If you start looking down there, it gives you 3, 4, 8, 11, 12.
Well, this is so different.
3, 4, 8, 11 is so different from 1, 2, 3, 4.
What does BCC give you?
It gives you 2, 4, 6, 8, 10, 12.
Can you see you have a problem here?
It's pretty easy to pull out FCC.
But look at these two.
Since you don't know, you're just normalizing.
You can't tell.
If I give you a number sequence that is in the order 2, 4, 6, 8, 10, how do you know that that couldn't be 1, 2, 3, 4, 5?
There's a hook here, though.
Look at the sequence of h plus h squared plus k squared plus l squared.
Do you notice that there's no combination of h squared plus k squared plus l squared that gives you 7?
You get 6.
And then the next one is 8.
But there is a sequence that gives you 14.
So if you have a seventh line, if the seventh line to the first line, the ratio of h squared plus k squared plus l squared for angle number 7 to h squared plus k squared plus l squared.
For angle number 1, if it's equal to 7-- and point of fact it's not 7:1, it's 14:2-- and if it's equal to 8 for the seventh angle-- then it must be simple cubic.
And now you've sorted it all out.
So you're an expert now.
Now you can do it.
OK, so you're going to get some practice on homework.
So you can index crystals.
You can determine a crystal structure.
All right, a couple of other things we can do.
There is a second technique.
And it's called Laue diffraction after von Laue, who got the Nobel Prize in 1914.
He beat the Braggs by one year for this technique.
So he shines.
In this case for Laue, it's a slightly different technique.
What Laue does, is he uses light x-rays.
That means variable lambda.
So how would you get a fixed value of lambda?
Well, you would pick off one of those lines, like maybe the K alpha line because it's a nice, singular line.
And repress the bremsstrahlung and so on.
But if I want a variable lambda, where do I go for variable lambda?
I repress the lines.
And I go for the bremsstrahlung.
So now I've got lines varying all across the x region of the spectrum.
So I use variable lambda.
And I fixed the angle of incidence, whereas with diffractometry, I use a single value of lambda and vary.
So here's the cartoon which shows what's going on.
I'll do it one more time.
So this is called camera obscura, which is darkened room.
That's all this means is dark room.
So let's say I've got the specimen on the back wall of this thing, and what I've got is white x-rays coming in.
So the white x-ray enters through the face.
I've got the plate here, and I've got the specimen sitting somewhere in between.
And what happens is I get a spot pattern.
I get a spot pattern.
And the spot pattern is imitative of the symmetry of atomic arrangement.
So I have to say a little bit about symmetry so we know what that means.
So let's take a look at symmetry.
So I'm going to talk a little bit about rotational symmetries.
So let's look at that.
So first one I'm going to look at is an 0 0 1 plane.
So if I just take a projection of an 0 0 1 plane, it looks like this.
Doesn't it?
It's just the cube edge, or the cube bottom.
So it's got a and a as the edge.
And now, the rotational axis, it's about a normal rotational axis.
A normal rotational axis.
So what I'm going to ask is how many degrees do I have to go through before I get this same image back.
You can see that you go 90 degrees, it'll come back.
If I stop at 45 degrees, it's going to look like this.
You're going to say I know he moved the specimen.
If I go 90 degrees, you can't tell it apart.
So we define fold.
The fold is equal to 360 divided by the basic angle of rotation.
So we would say that this plane here exhibits 4-fold symmetry, OK?
Let's do a second one.
The second one is 0 1 1.
So if I look at 0 1 1, 0 1 1 at this plane.
0 1 1 goes across the diagonal.
So I'm going to take the diagonal here and plot it like so.
So it's going to have an edge of a.
And it's going to have the diagonal, which is root 2 times a.
And if I put a rotational axis in the center of that, clearly if I go only 90 degrees, I'm going to tell that the thing is rotated.
I have to go 180 degrees before I can't tell that there's been any disturbance.
So this one here has 2-fold symmetry.
We'll do one more.
Because there's only three major ones that we have to deal with.
And that is going to be the 1 1 1 plane.
And the 1 1 1 plane.
And the 1 1 1 plane, I think I've got an image of it here.
You can see the 1 1 1 plane drawn in this cube.
So what's the face of 1 1 1 look like?
The face of 1 1 1 looks like this.
That's face of 1 1 1.
And if you go through this analysis, this is root 3a because it's a diagonal of a difference persuasion.
So this is 3-fold symmetry.
This has 3-fold rotational symmetry.
And that means if I take a crystal such as this, for example.
I've told you before, this is my silicon wafer, and this silicon wafer has been cut from a single crystal.
So I'm looking at the edge of an atomic plane.
The question is, which one is it?
The crystal didn't come with a label on the side.
I don't know what the crystal growth axis was.
I started with a single crystal, and I dipped it into molten silicon.
And just like rock mountain candy, I drop the heating coils and cause the liquid to solidify around the seed crystal.
And I grow this salami that's about 8 inches in diameter, about 2 meters long.
So now I go and I cut these things with the diamond wheel.
So I'm cutting them normal to the growth axis, but I still don't know what plane I'm looking at.
So if I use the Laue technique and put the crystal like so, bombard it with white x-rays, and I look at the spot pattern.
The spot pattern is going to give me one of those symmetries.
And if I get a 3-fold symmetry, I know I'm looking at 1 1 1.
I mean, it's not going to be some wacko plane.
It's going to be either face, edge, or diagonal.
And on the basis of the symmetry in the Laue pattern, I can tell which plane I'm looking at.
They use this stuff.
They use it to make the devices that are in your cell phone and your computer.
I'm not just telling you a story.
All right.
So let's look at some others.
We'll go back to the Escher prints.
Everything has symmetry.
OK, so what's this one?
See this?
What's the symmetry?
4-fold.
How about this one?
What do you think?
3-fold.
I even went into Photoshop just to show you how dedicated I am.
So I took this image.
And I told Photoshop to give me 120 degrees rotation.
And I got this.
You can even see the fold in the book there.
So I'm really doing it.
It's coming back.
It's 120 degrees rotation.
How about this one?
That's 3.
Now these are real Laue patterns.
This is 4-fold.
This is obviously 3-fold.
This one is hard to see.
But it turns out it's 2-fold.
The one axis is just a little bit longer than the other axis.
What about this one?
That's our simple cubic puppy.
Yes, 1-fold.
Translational symmetry without rotational symmetry.
You can make this by just moving them sideways.
But you have to go all the way around.
So that's 1-fold symmetry translation.
Look at this one?
What's going on here?
Is that one of the Bravais lattices?
This is called Penrose tile, again rotational symmetry without translational symmetry.
You can take a patch of this and move it rotationally.
But you can't take and cover a wall with it.
So that's rotational without translational.
See?
You can go around this way.
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and reproduce this.
But you can't use this as a unit cell and keep it hopping laterally and tile a wall with it.
Here's another one, not a Bravais lattice.
All right.
So I told you at the beginning of this unit that there are ordered solids and disordered solids.
We've been focusing on ordered solids.
They have a unit cell.
They're periodic.
And we call them crystals.
And next week we're going to start looking at disordered solid.
So they have no building block, no long range order.
And we call them glasses.
For a long time, people thought that solis have to fall into one box or the other box.
And then the best thing that can happen in science is not, oh, yeah.
That's exactly what I expected.
It's, I wonder what that means.
And there was one of these moments in 1982.
In 1982, a man by the name of Danny Shechtman from Technion in Israel, was over here in the United States working at the National Institute of Standards and Technology, which used to be National Bureau of Standards down in Gaithersburg, Maryland.
And he was looking at a set of aluminum-manganese alloys.
So aluminum is a metal.
Manganese is a metal.
You can make a solid solution which we call an alloy.
And these are highly ordered.
And what he found in his Laue measurements, were rotational symmetries that are impossible in a crystal.
He was getting 5-fold symmetry.
You can't get 5-fold symmetry.
There is no set of planes here that will give you a 5-fold symmetry.
They lack translational symmetry.
They were called aperiodic.
And the popular name for them was quasicrystals.
So here's the Laue pattern of one of Danny Shechtman's aluminum-manganese specimens.
I think this is 25% manganese and aluminum if I'm not mistaken.
So let's count, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
You can't have 10 or 5.
Atoms don't work that way.
But he got it.
He got it, 5- fold, 72 degrees.
Now this is really, really exciting.
This is a shot of his lab book.
You're looking over his shoulder.
These are the specimen numbers.
And this is aluminum 25% manganese, April 8, 1982.
So it's just another day at work.
You could be doing this.
And he can't believe what he's getting.
He's wondering what's going on.
You can't contribute this to a calibration error on the instrument.
This thing is on unassailably 5-fold symmetry.
The joke is that this is not far from the Pentagon.
Only within shouting distance of the Pentagon would you discover 5-fold symmetry.
So there it is.
So where have we come with all of this?
We've come with the ability to start with some very simple ideas about x-ray generation.
And we've come to the point where we can characterize crystals, get quantitative measurements of their lattice constants, and by using Laue techniques, investigate symmetry.
OK. Let's adjourn.
We'll see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I know I'm not Dr. Sadoway.
I'm one of Dr. Sadoway's younger colleagues so-- but class is beginning, so let's get started.
Dr. Sadoway, of course, would like you to know that he would very much love to be with you here today and he would probably like that very much more than what he's doing now, which is sitting in a secured area secured by the Secret Service about to meet the President.
So you can ask him all about it when he comes back.
Associated with that, I have one request for you.
Today, when you're leaving, some of you may be used to leaving that way.
Don't leave that way.
Leave any other way.
This way's fine.
In the back is fine.
Just don't leave that way.
OK.
So last time, we talked about x-rays defraction and Bragg's Law.
And x-ray defraction and Bragg's Law has a lot to do with perfect crystals.
So perfection, right?
We've been dealing with perfect crystal so far and today, we're going to be dealing with imperfections.
So defects.
And in the field of material science, the field that I work in along with Dr. Sadoway.
there's a saying due to one of the most famous material scientists that says that crystals are like people.
It's the defects that make them interesting.
So now you've heard that saying and you can roll your eyes from now on every time you hear it because it's repeated it very often.
So let's see.
What do we mean by defects?
Well, there's two types of defects, broadly speaking, that we're going to be talking about, and we can classify those into two categories.
One of them is chemical.
So far, we've been dealing with chemically perfect materials.
So ones that are made up of one element or ones that are made up in stoichiometric quantities of two elements, but you can also have chemical imperfections.
You can have impurities.
You can have alloying elements and we're also going to be talking about atomic arrangement.
And in the case of atomic arrangement, we're dealing with structure, right?
Here we're dealing with chemistry.
Here we're dealing with structure.
You know about crystal structure so you know what perfect crystals look like, but real materials are not perfect, neither in the chemical sense nor in the structural sense.
So perfect crystals don't really exist.
You have always some forms of imperfections and we'll go over those today.
So in the case of chemical imperfections, we have, broadly speaking, two types.
Good ones-- Dr. Sadoway labels them with a smiley face so I will do that, too.
And bad ones are a sad face.
So both are defects and whether they're good or bad depends on their utility.
Are they useful or are they detrimental?
If they are detrimental, we call them impurities.
And if they are good, we call them other things.
For example, dopants.
We studied dopants, right?
Dopants are a type of impurity, they're a kind of imperfection, but they're good.
We use dopants.
We use them in semiconductors.
Also, alloying elements, you can put in the elements yourself.
Those are examples of good chemical imperfections.
When it comes to atomic arrangements, we're going to spend a little bit of time talking about that in more detail in just a minute.
But broadly speaking, they are situations where you don't have perfect crystal in order.
You have disruptions and that perfect crystal in order-- either in the form of missing atoms or extra atoms or differently oriented unit cells of the crystal and so forth.
So we'll talk about that a little bit more in just a minute.
So one thing that I forgot to mention maybe is that we have a test coming up-- That is, I'm sorry.
Celebration of learning, second celebration of learning.
And those are your room assignments.
You don't have to necessarily write them down now, but you'll see them again.
So I just wanted you to see them.
A through Ha in 10-250, He through Sm, 26-100, So through Z, 4-270.
OK, so let's go into the taxonomy of defects.
So I mentioned that there are a number of different kinds of defects, and we can do better than to say simply, defects exist.
We can actually start to classify them.
So there are, broadly speaking, four types of defects, which we classify based on dimensionality.
So there's 0-dimensional defects and 0-dimensional defects are point defects or point defect clusters.
It's just like in math.
A point has 0 dimension, right?
1-dimensional defects or line defects, things that thread through a material like a shoestring or something, and we'll have an example of that in the form of dislocations.
Then we have 2-dimensional defects.
Those are interfacial defects.
So if you have interfaces between two different kinds of materials or if you have two crystals that are misoriented with respect to each other, those are examples of interfacial defects.
And we also have bulk defects-- 3-dimensional defects like inclusions.
We'll go over some of those.
So let's start out with point defects.
Point defects are those 0-dimensional defects.
They're points, just like in math.
And there are a number of different point defects that we can look at.
We have our substitutional impurities, interstitial impurities.
The general overall recurring theme in these point defects, regardless what type they are, is that they are localized disruptions.
So a lattice, a crystalline lattice, is a regular arrangement of atoms, and a point defect is a very local disruption in that regular crystalline arrangement.
And those disruptions can occur on lattice sites.
So basically, on the positions where you would see atoms normally, they can also occur between lattice sites.
So they're localized rifts, you can say, in the periodicity of a crystalline material.
And here's just an illustration of the way that these different point defects play out.
So let's start out by looking at this one.
This is a substitutional impurity atom.
So it's an impurity.
So it's a type of chemical imperfection.
You can imagine that this is, for example, some kind of FCC material like copper, for instance, and suppose that you have an impurity of some sort like iron, and it sits there and it replace one of the copper atoms.
So this is called a substitutional impurity.
It substitutes for the regular atom that would have sat at that location.
Substitutional impurities.
There's another picture.
There's your substitutional impurity up there.
We'll get to the other ones in just a moment.
We already talked about some substitutional impurities.
So dopants.
If you have boron or phosphorus or silicon, that boron or that phosphorus replaces the silicon that would have sat at a certain lattice site.
So the dopants we've studied so far are types of substitutional impurities and we call them good impurities because they give us desirable properties.
They are dopants.
They're good impurities.
Then we have alloying elements.
I am not a very big fan of sodas, but if you were somebody else who is a big fan of sodas, then you might, for example, drink a lot of sodas out of aluminum cans.
Those aluminum cans are really not pure aluminum.
They're alloys.
They're alloys with other metals to give them the good properties that they need in order for the forming process to occur.
So if you ask yourself, how do you actually make aluminum cans?
They're stamped out from sheets of aluminum, but it's not just aluminum by itself.
Aluminum by itself would just tear if you do that.
So if you alloy it, you give it better properties so it's ductile.
You can deform it to very large strains and that's a good thing in the case of alloying elements.
Another one that he showed you was nickel and gold.
That's if you want to change the color of gold, if you want to be very creative when you're proposing.
So that's a good kind of substitutional impurity.
There are also contaminants.
Contaminants are bad kinds of impurities, but before we get to the contaminants, let me just show you what good impurities can do.
So this is the Hope Diamond.
It's in the American gem collection.
You can actually find out more about it.
I'm not a big gem expert, but anybody who looks at this Hope Diamond can immediately see that it's a really pretty diamond.
But you guys are ahead of everybody, because in addition to knowing that it's a pretty diamond, the fact that it has boron impurities in it already tells you what the majority charge carrier is.
So when you go to this gem collection, you can educate everybody else.
So those are all good impurities.
There are also bad impurities-- contaminants.
Lithium in sodium chloride is an example.
So if you were using saline solution, for instance, in an IV and you had lithium in there, that is very detrimental to the person who's receiving that.
That's definitely a contaminant.
You don't want those.
That could make you die.
So let's move on to some other types of point defects.
Here's another type of point defect.
So just now we talked about the substitutional impurity atom.
Now we're going to talk about the interstitial.
The interstitial's very different from the substitutional.
In the case of the substitutional, we have a lattice site, which instead of being occupied by the regular atom that would have occupied it in a perfect crystal, is occupied by a chemically distinct atom.
In the case of an interstitial impurity, that impurity can sit in the space in between lattice sites, so-called interstitial sites.
That's why we call them interstitial impurities.
And interstitial impurities or interstitial atoms can be both chemical impurities, and obviously, they are rifts in the atomic arrangement so they are structural impurities as well.
Why do I say chemical?
The reason is because if this is, for example, iron, if this is an iron matrix and you have a carbon atom sitting in the interstitial site which makes steel, then that's an example of a structural defect, but it's also a chemical defect.
If, for example, you have, on the other hand, iron sitting in a nuclear reactor and it's getting bombarded all the time by energetic neutrons, then interstitials are being created by atoms getting knocked out of their lattice sites and they're getting put into interstitial sites.
So that's creating defects that are chemically the same as the surrounding matrix material, but which are structurally distinct.
So those are interstitial atoms.
So here's another picture of interstitial atoms.
There it is.
It's not sitting on a regular lattice site.
It's sitting in between lattice sites.
It's sitting in the space, in the interstitial space between atoms.
And I already mentioned to you the fact that if you put carbon into iron, those atoms go into interstitial sites and there would give iron some of its beneficial properties, which we look for in steel.
So some of the good mechanical properties.
Here's another example of a situation where an atom goes into a lattice and create an interstitial impurity.
So lanthanum-nickel 5 is a prototype hydrogen storage material.
It takes up a huge amount of hydrogen.
It takes up a greater density of hydrogen than liquid hydrogen, actually, so if you expose this to hydrogen, the hydrogen just goes right in, and it sits in the lanthanum-nickel 5 lattice as an interstitial atom.
So this is considered to be as a prototype hydrogen storage material.
Unfortunately, it's extremely expensive so it's not being used very much these days, but on the other hand, it goes well with our gem theme in this particular lecture in terms of expensive things.
Here's another example of an interstitial impurity.
This one is not an alloying element.
It's a contaminant.
So hydrogen, but this time in iron.
So hydrogen in lanthanum-nickel 5 is good.
We want to store it.
Hydrogen in iron is bad because it actually degrades the mechanical properties of the iron, unlike carbon, which gives us steel, which is good.
You put hydrogen in, it embrittles it.
So hydrogen embrittlement in steels is a big problem.
And it's actually one of the challenges to a hydrogen economy.
If you have steel pipelines or valves or various pieces of machinery, structural components that are made out of iron that are constantly exposed to hydrogen, over time, they're going to brittle.
They're going to become very difficult to use.
It's one of the challenges in that whole undertaking.
OK.
So we're going fairly at a clip here through these taxonomy of point defects.
So in the taxonomy of point defects, perhaps the easiest defect to understand is the vacancy.
So when you think of the vacancy, think of the hole that we talked about in the case of semiconductors.
Vacancy is nothing.
It's a missing atom.
So if you have a crystalline lattice and it's FCC or it's BCC or it's simple cubic, whatever you like, and you know that there's supposed to be an atom at a certain lattice site and it's not there, that's a vacancy.
That's a situation where you have an unoccupied lattice site and there are different ways to form these vacancies.
They can be formed during crystallization.
If you heat up a material, the number of vacancies decreases.
So if you quench it really quickly, you can actually trap the vacancies before they can leave in service under extreme conditions.
I mentioned just a moment ago that interstitials can be created if you irradiate a material, if you bombard it with energetic particles, like neutrons for instance, you'll create interstitials, sure, by knocking atoms out of their atomic sites, but what's left behind after you knock that atom out?
Vacancies.
So actually, you create two defects at the same time: vacancies and interstitials.
So I think we probably have a picture here of a vacancy.
A vacancy is nothing.
It's just empty space.
That's a vacancy.
There you go again.
Nothing.
So we've gone through a bunch of taxonomy, right?
So we know now that there are a number of different kinds of point defects in crystals.
We've talked about interstitials.
We talked about vacancies.
We've talked about substitutionals.
What can we say about these defects quantitatively?
So let's take the vacancy and derive or write down what is the number of vacancies that you can expect to find in a given material at a certain temperatures?
So to do that, let's be a little bit more quantitative.
Suppose that you have a crystal.
I'm showing you a plane, for example, a 1 1 1 plane in an FCC material and how do you create a vacancy?
You simply remove an atom.
So you had an atom.
Now you have no atom.
You've created a vacancy.
When you created that vacancy, you broke all the bonds between the atom that used to be there and the neighboring atoms.
Breaking bonds costs energy.
So it costs energy to create a vacancy.
It costs energy to remove an atom from the place where it would have been because you're breaking bonds.
So in this case, I have six bonds that I broke.
If I were looking at some material, for example, placing our cubic material in 3D, I would find that I would be breaking 12 bonds to the 12 nearest neighbors, and so on for all the different crystal structures.
So we actually then take that information, the fact that we're breaking bonds, and encapsulate it in a single descriptions of how much energy it costs to create a vacancy.
And we call that the vacancy formation energy.
So this is an energy and so it's expressed in eV.
Its units are electron volts.
You can convert them to joules, anything you like.
And if you wanted to then compute how many vacancies there are in a given crystal, well, first of all, it costs energy to make them, so why would you ever even have a vacancy in the material?
Well, no material is perfect.
We know that from studying materials, but what causes it is the fact that at finite temperature because of the Boltzmann distribution that Dr. Sadoway told you about a few lectures ago, just like in the case of intrinsic charge carrier promotion in semiconductors, you can get thermal formation of vacancies.
So that Maxwell Boltzmann distribution can actually cause there to be vacancies despite the fact that there are-- that it costs energy to do that.
So how can we actually use that to express how many vacancies we have in a given material?
So I'm going to write down an expression for how many vacancies you can expect to find at a given temperature on the basis of their formation energy, OK?
So let's do that.
This is going to be the fraction of vacant sites.
So if you have a given material-- FCC, BCC, whatever you like-- you know that there's a certain number of lattice sites per unit volume, and you learn how to calculate those things.
And to quantify the number of vacancies, you have to basically say, what fraction of those sites does not contain an atom?
Is that 1/100 of a percent or is it 1% or how many?
So this is actually going to be expressed as a ratio, which I'm going to call this.
This is the number of vacancies per unit volume and this is the number of atomic sites, also per unit volume, OK.
So this is the definition of the fraction of vacant sites.
And how do we express it?
Well, we express it using a very simple formula.
This formula contains a factor here which is experimentally determined.
This is an empirical factor and then an exponential.
So the exponent we take here, the vacancy formation energy, and we divide it by the thermal energy at the given temperature of interest.
So this is actually telling you that to form vacancies, you actually have two competing factors.
On the one hand, you have the bonding energy that makes the difficult-to-form vacancies because you're breaking bonds, you're taking it an atom out.
On the other hand, you have this thermal energy, Boltzmann constant times the temperature, and the thermal energy is competing with that bonding energy.
And when that thermal energy is high enough, you can actually start knocking atoms out despite the fact that it costs you some energy, and obviously when this ratio is very large, that means that the bonding predominates, and when it gets smaller, that means the thermal energy is more and more sufficient to actually knock atoms out of their positions and cause there to be vacancies.
So when you do these calculations, make sure to use the absolute temperature in Kelvin, and dimensional analysis will tell you the units of the Boltzmann constant have to be energy units per degrees.
So eV's per Kelvin, for instance.
So this is the absolute temperature.
This is the vacancy formation energy and this is the Boltzmann constant.
So what that means is that at any given temperature, you'll actually expect to see some fraction of vacancies.
So let's actually try to do a calculation with an actual vacancy formation energy and see how many vacancies we get at a given temperature.
So to do that, we've been provided with what is the common currency in science, which is published experimental data, which we find from published journals which we find online through, for example, Web of Science.
And from this journal, we find that for copper, the vacancy formation energy is 1.03 electron volts.
Furthermore, in the same journal in the abstract there, you'll find the magnitude of this constant A, which as I told you is experimentally determined.
This quantity A we call the entropy factor.
And even though there's no way to very easily derive it-- I can't tell you what's the entropy factor for iron and you can't really tell me just by thinking about it-- nevertheless, it turns out that this factor usually fall within a certain range.
It's usually between 0.1 and 10.
That's usually the range and what you find As.
And in the case of copper, it's very much in that range.
It's just 1.1.
So let's use this information to actually figure out how many vacancies we expect to see in copper at the given temperature.
OK.
So here's my fraction of vacancies and we know that it is going to be expressed as 1.1 times exponent minus the formation energy, which is 1.03 eV, and then we'll put in Boltzmann's constant, and then let's pick a temperature.
For the moment, let's just take room temperature.
So T, room temperature, and room temperature's about 300 K. So when we put in all these numbers, it's just a matter of calculating.
This is on your sheet of constants.
This is the temperature which we choose at room temperature.
Let's actually write down T, room temperature, is about 300 K, and we find a certain number of vacancies.
And the number of vacancies that we find is 2.19 times 10 to the minus 18 vacancies.
So this is the fraction of vacant sites.
Well, is that a lot?
Is that a little?
How can we determine whether this is a lot or a little?
We compare it to the number of atoms that there actually are, and in the case of solid materials, we expect there to be something on the order-- this has to be compared to something on the order of Avogadro's number, but we can actually calculate more explicitly that this turns out to be something like 10 to the 5th vacancies per centimeter cubed.
And if in a real material, a solid material like silicon, for instance, or boron or whatever you're interested in, your number of atoms is something like 10 to the 23rd, Avogadro's number, right, then this is actually a very tiny number.
Compare 10 to the 5th to 10 to the 23rd.
The number of atoms that are missing at room temperature is very low in copper.
We can compute that using this expression.
However, even though it's low, it's not zero.
And if we continue going down in temperature, we'll find that the number is lower and lower and lower, but it never really goes to zero because we have this expression which gives us the total number.
So let's do another one and this time, let's take a different temperature.
Let's take the melting temperature of copper.
The melting temperature of copper is considerably higher.
It's about 1085 degrees Celsius and we can go through exactly the same calculation.
1.1 times all these quantities here times melting temperature, OK?
And when we do this, we get a vacant site fraction which is 1.67 times 10 to the minus 4 and that corresponds to 1.41 times 10 to the 19th vacancies per centimeter cubed.
So what do we know by-- what can we see now by comparing these two quantities?
Number of vacancies, add room temperature, number of vacancies at melting temperature.
How many orders of magnitude do they differ by?
Huge difference in the number of vacancies that we find at two different temperatures.
And why do we see such a huge difference?
Let's actually write down what that difference is.
Let's write down the ratio.
The fraction of vacant sites at melting temperature divided by fraction of vacant sites at room temperature is something like 7.6 times 10 to the 13th.
That's the difference in number of vacancies you get just by increasing the temperature from room temperature to melting temperature.
Why is that?
Well, one way to look at that is just from the expression that we have to calculate the number.
Here's where the temperatures go.
So any difference in temperature if it's a factor of two or if it's a factor of three or if it's a factor of four, it's going to go into the exponential.
So that exponential is actually making a huge difference as a function of temperature in terms of number of defects that you get.
The higher you go up in temperature, you don't get just-- you go up a factor of four in temperature, you don't just get a factor of four increase in the number of defects.
You get that in the exponential.
So the factor of four is hugely magnified by the exponential.
So that means that at any given temperature, even the lowest temperatures, you expect to see some defects, but if you increase the temperature, you see a hugely larger number of defects.
And you can use this sort of expression for any kind of defect.
So I talked about vacancies right now and vacancies have a specific formation energy, but interstitials also have formation energies, substitutionals also have formation energies.
So you can use this expression to determine the fraction of defects per lattice site for any kind of defect so long as you have the formation energy of that defect.
So just to show you how difficult it is actually to remove defects, if you have a crystalline material, defects want to stay even at the lowest temperatures.
I have this interesting little piece of art to show you.
This particular piece of art was actually discovered, I guess, by some scientist at the University of Toronto where Dr. Sadoway went to school, and this particular-- what do they call it?
They call it The Atomix.
Let me write down the name of it in case you want to look it up because I think they sold a lot more of these to material scientists than they sold to anybody else.
And all this is two plastic plates, two PMMA plates, polymethyl methacrylates, or plexiglass.
And in between them, there's a little hole that's cut, a gap, and in that gap are a number of ball bearings.
And the ball bearings are kind of like atom, right, so they move around.
And here we're going to the document camera right now.
So if you shake these things around and the ball bearings are moving around, that's like introducing temperature.
That's like taking a crystal, melting it, all the atoms are bouncing around.
You will even see some vapor atoms when these sort of leave the surface and fly through the air, but then if you stop, that's like quenching.
That's like suddenly I've taken this crystal, which was originally molten, and I've dropped the temperature.
Here's what I see.
So can you see that?
How to use this thing-- there we go.
So you can actually see a lot of the defects that we were talking about just a moment ago.
Here you can see areas of perfect crystalline order.
Here's another area of perfect crystalline order.
So you have many, many crystals that are adjacent to each other.
They're oriented differently.
So they're forming misorientation defects between themselves.
So for example, here's a grain boundary.
This is a crystalline grain.
And inside this crystal, you see a vacancy.
It's right there, So here's another instance of the vacancy.
There's another vacancy.
And what happens if I try to remove some of the disorder?
By the way, interesting thing is that some of the vapor is also left here.
So there's the vapor.
If I then try to remove some of these defects, I can just tap on this.
If I just continue to tap on it, I'm removing defects.
The whole thing is crystallizing more and more and more so now I put it down again.
OK. You still see the vapor phase.
Now you see a big huge crystal right here with some grain boundaries around it.
Here's another big huge crystal.
Here's another crystal.
There's a smaller crystal.
Each one of them is bounded by grain boundaries, but what do you see in each of these crystals?
There's a vacancy.
There's a vacancy.
There's a vacancy.
And you can actually sit here-- well, not here.
Maybe later, but if you want to take a look at one of these things, you can probably go to Dr. Sadoway's office and pick it up and he'll let you play with it for a little while.
You can try to get all these vacancies out.
You can try to get all the defects out.
No matter how hard you try, if you spent hours, you'll get things to be more and more and more perfect by tapping on it, but there's always going to be a vacancy.
Always.
And even if you are extremely patient, you get things down to just one vacancy and you think that all you have to do now is just give it a little bit more of a tap to remove that one vacancy, well, more often than not, you'll find that with that tap you'll remove the vacancy, but create another vacancy somewhere else.
So you'll always find these vacancies in these crystalline materials.
It's just a consequence of the fact that when you're agitating a material like you're doing here-- you're shaking it-- that's the same thing that happens if you have some finite temperatures, some non-zero temperature in the material, you're always going to be creating some defect.
So that's exactly what this expression is giving you.
It's telling you that there's going to be defects at any temperature, but their number goes up dramatically as you increase the temperature.
So let's go back to the slides real quick now.
OK.
So everything I've told you so far is concerned with defects in crystals that are of one type, so that are made up of a single element.
But in the case of, for example, things like ionic crystals, and ionic crystals are made up of multiple elements with multiple charge states, you can get new kinds of point defects that you can't see in a single element material.
So we'll talk a little bit about those and when it comes to those defects in ionic materials, we have to expand the taxonomy a little bit.
The taxonomy is now going to include defects called Schottky imperfections, Frenkel imperfections, and F-centers.
So let's go to a visualization of what these are.
Here you see an ionic crystal.
So you have alternating types of atoms.
You have these big ones, which are presumably the negatively charge ones, and then you have the small positively charged ones, and here you have an example of a Schottky imperfection.
So a Schottky imperfection, it's kind of like a vacancy.
It's missing atoms.
The main difference between a Schottky imperfection and a vacancy by itself is the fact that in a material that involves charged atoms-- ions, right-- in an ionic crystal, when you take atoms out, you have to make sure to maintain charge neutrality.
So these materials are charge neutral.
They have equal numbers of positive and negative ions, but you want to make sure that you take these out in equal numbers.
So in the Schottky defect, you take out one negative and one positive ion, or basically one unit, one stoichiometric unit.
If you add zirconium oxide, which is ZrO2, you would have to take out three atoms to maintain charge neutrality.
That would be the Schottky defect.
So here is another example, another visualization of a Schottky defect.
So when you take these two atoms out, nothing says that they have to be taken out from right next to each other.
You can take out one here.
You can take out one there.
It's good both ways because in the end, it's just about charge neutrality.
It's about maintaining a charge neutral material.
So we can actually write down reactions to describe the formation of these Schottky defects.
So let's do that.
When we write down reactions to describe the formation of Schottky defects, we first recognize the fact that we're dealing with empty sites.
We're dealing with void.
We want to see how this void, or in scientific parlance, null, is decomposed into two vacant sites in the particular ionic crystal that we're dealing with.
So let's-- to make things specific, deal with sodium chloride.
So sodium chloride, we have two type of atoms.
Sodium and chloride.
And to maintain charge neutrality, we have to remove one of each.
So this is actually-- this null space is actually going to be composed of two vacancies: a vacant site on a sodium lattice where a sodium atom would have been sitting and a vacant site where a chlorine would have been sitting.
And because this entire process takes place in the presence of materials that are composed of ions-- so ionic solids-- these two vacancies, in fact, have some charge them.
So when we think about the charge of these vacancies, what we actually have to think of, it's a little bit counterintuitive.
The sodium in a sodium chloride crystal is charge positive, but what does that make the vacancy?
So if you have a lattice of atoms that are charge positive like the sodium atoms and you remove one of them, what is the effective charge of that vacancy?
You have nothing, as Dr. Sadoway would say, in the land of plus.
So the charge of this vacancy, the effective charge of this vacancy, is going to be negative.
And we mark that with this little thing right there.
So void or nothing in land of plus is negative.
We mark that with one of these apostrophes.
So what about this vacancy?
That vacancy is created on a lattice of sites which are usually charge negative.
So if you have these chlorine atoms that are charged negative and you remove one of them, now you have void in a land of plus.
So this is going to be effectively charge positive and so we have this void in land of minus.
It is positive.
And we do it-- we annotate it with one of these dots.
We can do the same thing for other kinds of crystals.
So I mentioned that in this case, a Schottky defect would just be composed of these two atoms, because these two atoms constitute the structural unit for this crystal.
But if we had some other, more complicated crystal, like, for examples, zirconium oxide, then we would still write down a similar reaction.
So we could put a Schottky defect into zirconium oxide.
OK. And so now we have to figure out how this reaction plays out.
We have to create vacancies.
Here's our vacancy on the zirconium.
And because the structural unit contains two oxygens, we have to create two oxygen vacancies.
And now we have to think about how do we maintain charge neutralities?
What is the effective charge on all of these atoms?
So the zirconium sites are usually the positively charged atoms, so when we remove one of them, that's removing something, that's putting void basically into the land of plus.
So we have now a negatively charged vacancy, and this negatively charged vacancy has four negative charges, an effective minus 4 negative charge.
And the oxygens are the negatively charged ions, and so when we create voids there, these two vacancies have an effective positive charge, and because the charges have to balance, we know that this has to be effectively two negative charges and there are two vacancies.
So we have charge neutrality.
So that's how the Schottky defects work.
OK.
So let's move on to Frenkel defects.
In the case of Frenkel defects, it's really not so different from Schottky defects in some sense, except now instead of dealing with a situation where we maintain charge neutrality by removing two atoms, we actually displace one atom from its initial location to a different part of the crystal.
So we're creating actually a vacancy right there, and this atom, which was originally sitting on a lattice site, is now displaced to an interstitial location.
So that's an interstitial.
So a Frenkel defect is a vacancy and an interstitial.
And here's another view of a Frenkel defect.
Here, I've removed an atom from one of the lattice sites and moved it over to here.
So that's my Frenkel defect.
And in the case of Frenkel defects I can also write down reactions for Frenkel defects.
Let's do that.
OK.
So Frenkel defects usually occur in crystals with widely differing atomic radii, and in ionic crystals, the radius of whatever is positively charged would have to be much, much less than whatever the radius of whatever's negatively charged occasionally, or very rarely, but I suppose it's still possible, you would have to have basically a very big difference in radius between these two cases.
An example of one of these situations is, for example, silver bromide, where here the silver's positively charged and the bromide ion is negatively charged.
And we can write down a reaction to describe the Frenkel defect formation just like we did in the case of the Schottky defect formation.
So in the case of a Frenkel defect formation, how do we actually go about this?
We have, for example, can start out with a silver atom site and we're going to displace that silver atom away from its original site so it now sits in an interstitial position and we're going to leave behind a vacancy.
So let's do that.
OK.
So we're going to have silver interstitial site and let's just be clear-- say that this is on a regular silver site.
And then we have a vacancy left over on the silver site and this is the reaction for the formation of our Frenkel defect.
Frenkel defects, just like Schottky defects, preserve charge neutrality.
So how do we actually mark down charge neutrality in this case?
Well, there's no change in charges in the original state so we'll just mark that with an X.
And the vacancy is sitting in a place that used to be positive so it's a hole in the land of positive so we know that this one's going to be negative.
And this silver ion, which was displaced from its original lattice site, is now going to be sitting in an interstitial site which was originally a land of zero.
So this one's going to be charged positive.
And now we've balanced, once again, this reaction.
We have charge neutrality.
We've created a defect and this is defect, just like in the case of vacancies, costs us energy to make.
So we can actually write down a formation energy for Frenkel defects.
And this formation energy is usually a little bit higher than the formation energy of vacancies so it's going to be something more perhaps on the order of 2 eV.
OK.
So we have gone through-- before I go to this one, let me just mention one last defect that you might run into if you're working in ionically bonded materials.
An F-center is a situation where you have a vacancy with a bound electron inside of it.
And F-centers have the property that they give ceramics a tint, a hue.
They actually scatter light in the visible range so you can see them.
So with a very short amount of time left, I wanted to tell you about a couple of the other defects that you will encounter in crystals and one of the most interesting ones to me is the dislocation and dislocations are line defects.
They're 1D defects.
So if you look in, for example, a corn ear right here, you can see dislocations as the termination of a single atomic plane.
So here's a picture of what a dislocation looks like in a crystal.
You have these atomic planes and here you have one that just ends.
That's a dislocation.
Some of you probably have dislocations in your fingerprints.
So after the class is over, you can look for dislocations in your fingerprints.
They're line defects.
So here's another example of that.
You see that there's an extraatomic plane which ends.
That's a dislocation.
We mark it with one of these Ts.
Here's a picture of a dislocation.
Dislocations actually help materials to deform easily without having to shear off.
Here's a little picture of mine which I really like.
This is the only way that people really had to study dislocations before computers existed.
So now we can study, for example, dislocation by simulating them in real crystals.
And this is a mid-20th century computer.
It's a raft of bubbles.
It's just a bunch of bubbles that somebody blew on the surface, soap bubbles, and you can deform them.
You can deform this raft of bubbles by shearing it and here you see-- what's wrong?
Why isn't it going?
There it is.
There's your dislocation running through and you see lots of dislocations running through this crystal.
This was, by the way, done by Lawrence Bragg, who you heard about in connection with x-ray scattering.
You'll hear about Bragg again when we get to DNA.
So keep him in mind.
The following content is provided under a Creative Commons license.
Your support will MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, big announcement.
Wednesday, no lecture.
Instead, Celebration Part 2.
And I'm going to say a few words about it.
There is the room assignments.
I think they're the same as last time.
If you miss the exam because of illness, there will be a make-up test on November 4th during class.
There's no weekly quiz tomorrow.
I'll be available for office hours 4:00 to 5:30 this afternoon over here at 8-205.
So a couple of things.
I'm going to say few words about the quiz, the test, rather.
The celebration.
So the coverage is going to go starting from the time of the last celebration up through now, but I think in view of the fact that we started defects on Friday, and my thanks go to Professor Demkowicz, who jumped in the breach and lectured in my place so that I could get drawn into some of the activities associated with the visit of the President.
You will not have had the time to go through your recitation cycle on this new material.
So I'm not going to put anything on defects on the exam, all right?
So we'll go up through the end of the unit on x-rays right up through Bragg's Law, indexing crystals.
So that starting Friday, defects.
I'm not going to teach you Friday, teach you Monday, examine you Wednesday and Tuesday.
Tomorrow I expect you're going to have all sorts of review-oriented activities in your recitation.
So I don't want you torn between trying to master a lot of this new material, which is quite different from what you had in high school, and at the same time trying to have a catch up on capturing all that material since the second test.
So that having been said, I'm going to remind you.
I'm going to basically say the same thing I said before the first celebration, but we need to be reminded.
So what's the purpose of the test?
The purpose of the test is to give you a status report and send you a message.
And there's really only two answers here that we're looking for.
You're either going to get a smiley face or you're going to get a frowny face.
That's all it's about.
I don't care whether it's a 78 or a 75, but hard 50 is a pass.
And if you're not mastering the material, I intend to tell you so and then invite you in so that we can figure out how to get you through.
Because everybody in this room has the intellectual apparatus to pass this class.
Some of you just are not managing your time well and you need to be told that.
So we will tell you that.
With one of these.
That's going to be the message.
Go to the right room.
Bring five things with you: Periodic Table, table of constants, your age sheet, calculator, and a pen, something to write with.
You'll write on the question paper.
No wireless devices.
Shut them off.
Don't even bring them.
What do you need your cell phone for?
Don't need it during a test.
We're going to start at 11:05 promptly and stop at 11:55.
What's the strategy?
Read the entire paper.
Try every question.
You know this.
You get part marks if you write something.
If you don't write anything for the question, you get a zero.
So try every question.
And start with the easy one.
The easy one for you may not be the easy one for your neighbor, but how do you know what the easy one is?
I just put them down as they came out of my mind.
I didn't put them down in ascending order of difficulty, chronological order, topical order.
They're just-- they're there.
Leave tracks, tell us how you got to the answer, Work parametrically.
Don't start plugging in numbers on the first line, because if you make a computational error on the first line and drag it through the entire derivation, you'll make a mess of things.
You'll have wavelengths, size of distance here to Chicago, light moving at ten times its normal speed and stuff like that.
It's no good.
Focus on the question.
Don't just say, oh, it's something to do with x-rays, just give me some stuff that you memorized about x-rays, some stuff you wrote down in your age sheet about x-rays.
If it's not about the question that we asked, I'll give you a zero.
I don't care how much you write.
Write on the topic.
And refer to electronic structure.
When you don't know what to say, start talking to me about electronic structure.
That's the theme of chemistry.
electronic structure dictates behavior.
So talk to me about electronic structure.
Do your own work.
And lastly, it says something here, oh, good luck.
That's what it says.
Good luck.
So last day, Professor Demkowicz got to the line defects.
He was talking to us about line defects and how they have an impact on behavior.
So I wanted to pick up the treatment there.
So what's the whole issue behind line defects?
Well, what line defects do is they explain this anomalous behavior where the measured value of yield strength-- so I'm going to write sigma y; sigma is the symbol for strength.
the yield strength as measured is on the order of only about 1/10 of the theoretical value.
It's only about 1/10 of the theoretical value for the yield strength.
You might, say, wait a minute.
How do you get that theoretical value?
You can calculate this.
Not too badly.
If I take a look at a system of atoms, and this could be a metallic crystal, you could argue that there are metallic bonds between all of these atoms.
And if what I wanted to do to shear something-- so this is the yield strength for shear-- if I wanted to shear the top plane of atoms versus the bottom plane of atoms, can you see that to a first approximation, the energy required would be the energy to break each one of these bonds?
Because I have to break the bond to release the plane and then move it.
So if you make an estimation on the basis of bond strength-- so you take individual bond strength times bond density or bond concentration.
When I say density, I don't necessarily mean mass per unit volume.
I could mean bonds per unit area.
You saw your homework questions.
How many atoms per unit length?
And we said what's the linear density?
And some people were saying, but density is mass per unit volume.
No, I'm counting atoms per unit length.
Open your mind, OK?
So bond strength, what's bond density?
That's atoms per unit area.
So we can calculate this and we're way off.
We're way off by about a factor of 10.
So it was the dislocation that allowed us to explain why this happens.
And it was the work of two scientists, one by the name of Orowan in Hungary and Taylor in the UK.
Orowan, after World War II, went to the UK and then ultimately came to MIT.
And he spent most of this later part of his career here in the Mechanical Engineering Department, from which he retired.
And actually, he's too modest to say it, but Professor Demkowicz, his PhD was for Professor Argon, and Professor Argon was one of Orowan's students.
So this is direct lineage.
This is all very clear, so he was the appropriate person to introduce dislocations.
So what did Orowan do?
Orowan came up with this idea of an interruption.
And how did he get it?
It shows how to look around and, in the world, see analogies.
Now, we didn't have the powerful electron microscopes then that we have today.
This was in the 1930s, so people didn't even agree on an atomic view of things.
So people were thinking, thinking, thinking.
So this is the dislocation.
You can see.
Here, we have three rows of atoms, now there's two rows of atoms.
The middle row stops there.
And I'll show you what the consequence of that is in terms of being able to shear the crystal of lower values of applied stress.
So that's the dislocation.
You can see here.
If you break this bond and then propagate that through, you don't have to shear them all.
You break them one at a time.
So Orowan came home one day in Budapest, and the cleaning woman was moving the runner on a hallway, the story goes, and she was about to move the runner.
And Orowan came up the stairs and offered to help her, and she told him, get out of the way.
Academic.
And what she did, instead of pulling the rug, she did what we see in the cartoon here.
She put a little kink in the rug, snapped her foot and propagated the kink and moved the rug a half a foot down the hallway with minimal energy.
Instead of having to pull the rug with it's coefficient of fraction over three, four meters, all she had to do was come up with enough energy to make the kink and then stamp the foot.
This person's really doing it the hard way.
I don't know.
This guy must have taken classes in chemistry or something.
But see, all you've got to do-- once you get that kind you just push and it'll go by itself.
So Orowan said could this be a metaphor for what's going on at the atomic level?
And that's where it came up with this idea of the dislocation.
So there we have it.
So where do we find the dislocations?
Where do we find them?
So what is it?
The dislocation-- I keep talking about it, what is it?
The dislocation, this is, if you like, it's a line defect.
It's a line defect formed by misregistry of atoms.
And with the misregistry, that leads to a reduction in bond density.
And that broken bond is like the hole, if you like, in the valence band.
It can propagate very easily.
We're going to learn a little bit later in the lecture where we find dislocations and how they contribute.
But I want you to hold that in near-term memory.
OK.
So we're going down the list of the different types of defects.
Oh here, I found this off a coffee table book that's indicating-- I think this is a crummy analogy.
It shows this caterpillar moving by.
Instead of moving all of its legs, it causes this thing-- I think this is a bad analogy.
Just look down here.
You see, you breaking this.
And once you break this, you just keep moving it along.
Propagate the single broken bond instead of breaking them all at once.
I don't think the caterpillar moves that way, anyway.
It's cute, but it's wrong.
This is a dislocation.
I don't think it has anything to do with mechanical behavior, but it's a misregistry.
So you can-- these guys, they write the books and they come up with this stuff and you study it, and you're going, I don't get it.
You know what?
You shouldn't get it.
Because this is crummy.
It doesn't make any sense.
How does that impact the yield stress of the corn?
Someday you could write a book.
Better one than this.
All right, so now we're going to go to two-dimensional defects.
Let's go to two-dimensional defects.
So two-dimensional defects, there's a couple types.
The dislocation is one dimensional.
Two-dimensional defects.
There's two types.
One is the free surface.
We're going to call the free surface a defect, and I'll explain why in a second.
And then internal surfaces.
Internal-- I'm going to use surface in quotation marks-- they're interfaces.
And there's different types of them, OK?
An interface.
There's an interface inside the thing.
So what's the physics here?
Surfaces are high-energy regions.
Why?
Because when you're inside the crystal, you have the prescribed number of nearest neighbors.
Let's say you're in an FCC crystal.
You're an aluminum atom.
So you're an aluminum atom and you've got 12 nearest neighbors because you're FCC, except if you're at the free surface.
If you're an aluminum atom at the free surface you don't have 12 nearest neighbors.
Because if I look down I've got aluminum atoms all over.
If I look up, I've got nothing.
So axiomatically, I've got a different number of bonds.
Why do we form bonds?
We form bonds in order to lower the energy of the system.
But axiomatically, the ones at the top can't form the same number of bonds, so they are not in the same low energy state.
So once you get the connection between lower number of bonds, higher energy state, then any place, free surface, internal surface, is going to be relatively high energy.
Relatively higher energy than atoms in the matrix.
Matrix.
What do I mean by matrix?
In the middle of a crystal.
In the matrix, OK?
So that means that it's not quite as tightly bound, so the consequence of this is if they're high-energy regions, so high-energy implies higher reactivity.
And that can be good or it can be bad.
It can be good if you're trying to design a catalyst.
And it can be bad in the case of corrosion.
Where do you think corrosion occurs?
Not deep inside the crystal.
It occurs at the surface along the grain boundary.
So here's a cartoon that shows different regions within the crystal.
And Professor Demkowicz showed you on Friday-- and I'll cut to the thing in a second, the Atomix.
but here we see atoms.
So this is a simple cubic, this is simple cubic, this is simple cubic.
But when they meet they don't line up.
They don't line up because this formed from solidification from the melt.
So there might have been a seed here and it grows outward.
A seed here and it grows outward.
A seed here and it grows outward.
And when they grow outward they impinge, but they're not all growing with the same orientation.
You already know what the planes are.
And they could even be the same plane, but they're starting from a different point.
So there's this misregistry here.
But it's not a misregistry like the dislocation.
It's a two dimensional misregistry.
Actually, let's cut to the document camera.
Dave, could we do that please?
OK, so here's Atomix.
This was, as explained, an object of art made by Francois Dallegret in Quebec in the '60s.
So it's kinetic.
It's supposed to be after the artist, see?
You know, you, too.
You pick it up on a coffee table, you put it down, now you have become the artist.
You become the artist.
Postmodernism.
After the death of the artist.
See?
I'm an artist.
Look.
I'm an artist.
One of my best works.
So a couple of my professors at the University of Toronto were at a cocktail party one night and they saw this thing on a table.
Must've been a wild party because they kept looking at this thing.
Finally decided to write a paper about it.
And as Professor Demkowicz said, I bet Francois Dallegret sold more of these to professors of material science than he sold to art collectors.
So it's 200 ball bearings strategically placed between two plates of plexiglass.
And as he pointed out on Friday, you can't get everything to go away in terms of dislocations, but now you can see misregistry-- pardon me, vacancies here.
Here's a stacking fault and so on.
And now here you see grain boundaries.
There's the free surface.
Here's the vapor phase.
Everything.
We're having a lot of fun, here.
So we can look at it again.
We can change.
We'll try to anneal it.
So how do we anneal it?
You put it in an oven and you eat it.
This is the mechanical equivalent of annealing.
I'm tapping it, for those of you in the back.
Now, I'm going to put it down and you see I have a much more refined structure.
So now you can see a large section here of a single crystal.
Each one of these is called a grain.
A grain is a zone that is a single crystal with some defects.
You can see the vacancies.
And then look at the grain boundary here.
See this is a nice crystal, this is a nice crystal.
So one grain, two grains, and there's the grain boundary where these don't line up with those.
Because they started from a different angle so they can't impinge and blend.
There's grain boundaries here, here, here, and there's beautiful stuff.
You can see dislocations.
Everything.
And I know he showed you at the end the Bragg bubble wrap.
And you could see-- [SOUND EFFECT]
--dislocations zipping.
When people were trying to compress the bubble wrap, the way the bubble wrap was compressing the bubbles didn't all get flat.
The bubbled glided over one another, and the dislocations relieved the stress.
See what I'm telling you here?
What I'm telling you here is something that you don't get in a Gen Chem class anywhere else.
If I told you about the chemical origins of reactivity, you'd go, yeah, that's chemistry.
But what are we talking about here?
We're talking about the chemical origins of mechanical behavior.
Now that's different.
The chemical origins of mechanical behavior: only in 3.091.
And why shouldn't we?
Because at the end, everything is chemistry.
Everything is chemistry.
The rest is stamp collecting.
So the grain-- make sure we know where the grain is.
The grain is like, I want to call single crystal.
The grain is like a single crystal.
And the grains are separated by grain boundaries.
So by managing grain boundaries, control of grain boundaries, or if you like, management a grain boundaries, this has an impact on mechanical properties.
This is called grain boundary engineering.
How we control mechanical properties.
Grain boundary engineering.
So for example, if we want high strength-- and I just told you that the dislocations will allow the atoms to glide.
Can you see that if the dislocation is present in this large grain on the bottom here and the dislocation, just as in the bubble wrap movie, is zipping across, what happens when it gets to the grain boundary?
The misregistry of atoms stops that dislocation dead in its tracks.
So if I want high strength, do I want a large number of grain boundaries per unit area?
Or a small number of grain boundaries per unit area if my goal is to stop the dislocations?
I want a high number.
High strength means fine grain structure.
Fine grained means small grain size.
Fine grain structure.
You can think of it as like a river.
You know, you're running, running, running-- all of a sudden, you stop because you've got to ford the river.
On the other hand, if you want high ductility-- why would you want ductility?
For example, you want to make a beverage container.
How do you make a beverage container?
Not by casting.
You take a sheet of aluminum and you punch it and you cause it to deep draw.
That's atoms sliding over atoms.
So do I want to facilitate that process or do I want to impede the process?
Well, to facilitate the process, I want to make it as easy as possible for those atoms to glide over one another.
So high ductility means coarse grain structure.
Coarse grain means large, if you like, large grain diameter.
Fine grain structure means small diameter.
So there we go.
That's the beginning.
And there's a lot more to it, but I just wanted you to see that there is something.
And David, may we cut back to the slides, please?
OK, I think the next one shows.
All right, so this is a-- oh, we don't want to hear that.
Whoops!
Well, I've got music embedded in this one.
This is Philip Glass.
[MUSIC PLAYING]
Because we're going to talk about glasses later.
Koyaanisqatsi.
It's beautiful.
Can you feel the bass?
There's the subwoofer right here.
You stand right here.
It's like a message.
Low frequency, of course.
All right, so here's this.
But this is a piece of copper.
Now, this was the advent of modern materials characterization when Sorby, in the late 1800s, figured out that if you polish the piece of metal supersmooth-- that's good, huh?
Maybe we better turn the volume down.
I can't compete with these.
There's more of them than me.
I can't compete.
That's called the mute button.
[MUSIC ENDS]
So what did he do?
He polished the copper and then he etched it.
And what happens with an acid etch?
It will etch at different rates because the different grains have different atom densities.
You see, if I take a piece of metal and I polish it super, super bright and I shine a light on it and look at it in a microscope, what am I going to see?
I'm going to see my eyeball, right?
I'll blind myself.
So the key was to etch.
And then when you etch, you differentiate the different grains.
So that's what you're looking at-- the grain boundaries.
And this is now the same thing, only under a field ion microscope.
This is a bicrystal of tungsten.
It's hard to see, but this is all one.
This looks like a lolly pattern, doesn't it?
You can see the symmetry here.
I don't know if you can see right here.
There's a line, sort of v-shaped here.
And that's the second crystal.
So there's one crystal here, a second crystal here.
So this two-dimensional defect is now characterized.
And we'll get to that later.
OK.
So now we want to do three-dimensional defects.
What are the three-dimensional defects?
Three-dimensional defects are important.
And they have beneficial and they have harmful consequences.
Three-dimensional defects.
The dominant mechanism here is coalescence.
So three-dimensional defects formed by coalescence.
And you can coalesce two things.
You can coalesce voids, or you can coalesce-- actually, you can coalesce void, and you can coalesce-- I'm going to use Democritus talk here.
You can coalesce void, or you can coalesce being.
What do I mean by that?
Well, void, this could be a vacancy cluster.
So if you have a large number of vacancies coalescing, you'll actually end up with a hole, a blow hole inside the material.
And as you can imagine, that has the mechanical strength of nothing.
That's bad.
And being?
This could be impurity clusters.
And there's two types of impurities, as we've learned.
There are good impurities, such as dopants, and there are bad impurities such as contaminants.
And what can they do to the mechanical properties if they are bad?
So what I like to say is impurity clusters-- so we've got, for example, there are good ones that are managed.
The ones that are managed can be used to our advantage.
So I'm going to say the managed ones get a smiley face, and the ones that are uncontrolled-- I'll give you a vivid example of that at the end of the lecture.
Uncontrolled, they get a frowny face, and they can lead to major disasters.
So for example, you're really talking about clusters that exsolve as a separate phase.
So these things exist as a separate phase.
So they're no longer dissolved.
And these precipitates, some of these impurity clusters, if they're managed, we can call them precipitates.
i'll give you an example of precipitates.
Some of you might be going home at some point in the not too distant future, maybe for Thanksgiving.
And if you're flying on an airplane, the airplane is made of an aluminum alloy.
Aluminum copper.
And one of the precipitates that you can form is various copper-aluminum compounds.
And these form precipitates that the volume of copper-aluminum 2 is greater than the volume of the constituents.
So if you take a copper atom and two aluminum atoms in the lattice and they react to form copper-aluminum 2, there's a volume increase.
And so that works to give you constituents.
Isn't that great?
I left you in suspense there as to rest of the word.
So what happens?
This works a little bit the way carbon does in iron, where carbon atom is bigger than the interstitial void space in iron, and that force fit leads to a strengthening.
This leads to a strengthening, oK?
So that's good.
So this is called precipitation hardening.
I'm just going write precipitation pptn, precipitation hardening.
Very important in metallurgy.
Whereas carbon in iron is called solution hardening.
Because it doesn't exist as a separate phase.
The carbon atom is sitting on an interstitial site so it's a force fit, but it's still within the lattice, whereas the copper-aluminum actually exists as a precipitate outside the lattice.
So there we go.
In order to give you a little bit more, I said we're going to talk about the chemical origins of mechanical behavior.
And I want to show you how this all leads to strengthening, strengthening in terms of the yield.
So let's talk about deformation and the relationship to chemistry.
There's two types of deformation.
And this really accounts for the importance of metals in the history of man.
Why?
Because they have this unique property.
Until the modern era, when we discovered plastics.
But until modern era, metals are deformable.
They can deform without breaking.
That's what made them so desirable as materials of construction.
So I'm going to talk about deformation, how we deform them.
So really, metallurgy consists of-- really, there's two major divisions of metallurgy.
The first part is chemical, chemical metallurgy, and that's called dirt to metal.
That's the conversion.
This is the intensive chemistry.
And then the second part is called physical metallurgy, which is what I'm going to talk about next.
Physical, which is change of shape.
That relies on the ductility.
So deformation.
There's two types of deformation.
The first one is called elastic deformation.
And it is reversible.
So it is strain only under stress.
Or if you like, displacement only under applied force.
So this is Hooke's Law.
Hooke's Law applies.
So let's take a look.
I'm going to draw a little curve here.
You've seen this before.
I'm going to draw one of these energy curves.
So this is interatomic separation, which we might use the lower case r. And this is the potential energy that's stored between two atoms.
Remember, when they're far apart, they want to attract.
When they get too close together, their electronic shells repel.
So we've got-- time for colored chalk-- we have the attractive component, which looks like this.
And then we have the repulsive component, which looks something like this.
And then we put the two of them together, and I think we did this for ions, but there's an analogous one for other materials, and so you add the two of these, and you eventually end up like this, where this is r naught.
r naught, the optimum separation.
Here's attraction, this is repulsion, et cetera, et cetera.
So this is e minimum.
We've seen all of that before.
Now what's this have to do with mechanical behavior?
Well, I'll take it just a little bit farther.
And so I'm going to take the derivative of that and look at the force.
All right, so now I'm going to take the first derivative of that.
So this is going to be force, which is equal to de dr. Force is the derivative of energy.
And what do I find?
Well, look.
If I take this derivative, below r equals r naught.
I've got a negative slope.
Above r equals r naught-- I have a positive slope.
And at r equals r naught, zero slope.
So I'm want to put that down here.
So it's zero right at r equals r naught, and above r equals r naught it's positive, and below r equals r naught it's negative.
Not to scale.
But one thing is certain.
It is positive above r equals r naught, negative below r equals r naught, and to a first approximation, I can linearize about r equals r naught, Or Emin.
So what's the real shape of this thing?
You could use a Taylor series if you want.
And just take the first-- what's the first term of a Taylor series?
Constant.
Second term, linear.
Third term is squared, da, da, da, da.
So neglect higher order terms.
You get a straight line.
And what does this mean?
It means that if I try to disturb the atoms from one another and move them away from their rest position, there's a force that pulls them back, and the force is proportional to the displacement.
If I move the atoms farther apart by a distance r1, I get a certain force.
If I move them apart by r2-- 2 times r1-- I get twice the force.
It's linear.
Now let's go over here.
Now let's draw another analogy.
So here I've got something that's pegged.
And now I'm going to put a spring onto this secure wall.
So here's my spring, and I'm going to hang a mass on the spring.
Now I'm going to bring this into Hooke's Law.
I'm going to show you that what you know to be Hooke's Law is a direct consequence of this.
The atomic origin.
So what's the force on this?
There's a force on this, and that's equal to product of the mass times the gravitational constant.
And if there were no mass, this spring would be compressed and it might be sitting here.
I will call this x naught.
And now that I put the spring here I'll measures, say, this point.
And this point here is now at some value x.
So this is the displacement, isn't it?
This is the delta x.
So Hooke's Law is simply f equals k delta x, isn't it?
If I double the delta x, requires double the force.
Well, that's this.
This analogy goes right down.
And if I remove the mass it springs back, which is what I meant when I said it's reversible.
This is not permanent.
No stress, no strain.
Strain is the deformation.
No deformation, no force.
If I put a force, non-zero force, non-zero deformation.
By the way, Hooke-- little bit of culture-- Hooke announced Hooke's Law in 1676.
Here's how Hooke's Law was first published.
It's an anagram and it's in Latin.
And it was in a book entitled A Decimate of the Centesms of the Inventions I Intend to Reveal, which is English for, This is a Tenth of the Hundreds of the Inventions I Intend to Reveal.
He did not suffer from an overabundance of modesty.
Actually, Hooke and Newton sparred.
They hated each other's guts.
They fought bitterly.
Two giant egos.
Anyway, so here's the thing.
So in 1679 he published a book called De Restitutiva.
On The Spring.
Very presumptuous.
On The Spring.
And here it is in its unraveled, ut tensio, sic vis.
As the extension, so the force.
Nice, huh?
This is u.
That's why the u, but it's really v. Vis.
We get the English word vim from this.
This is the accusative form of vis.
The Romans didn't care between the u and v. In fact, if you go on the steps of 77 and you come up, it says Massachusetts Institute of Technology, you ever notice this?
Yeah.
I always thought that that's because the Romans, they couldn't-- no.
And I thought, well, maybe it's because it's hard to make a u, but then it says Institute of-- I figure if you can make one of these, you should be able to make one of these.
So if you can tell me why they do it that way, I might be able to find another Periodic Table beach towel.
So that's another thing.
All right.
So that's the elastic deformation.
And now we want to look at the other type of deformation, and that's called plastic deformation.
And that's permanent.
It's a permanent shape change.
So let's get that up here.
Permanent shape change.
And this requires ductility.
Because otherwise you'll have fracture.
If you take something that has no ductility and you apply a super critical force, you just crack everything.
So ductility allows for the atoms to glide over one another.
So what is the mechanism?
The mechanism for plastic deformation?
Plastic deformation in crystals, we're talking about here, not liquids.
It's called slip.
Atom's slip over one another.
So time for one more cartoon.
So I'm going to say, well, what happens if I start again with this?
I'm going to put a thin wire, a thin wire of metal.
Make it an FCC metal, let's say it's copper, and I'm going to put a mass here-- a honking big mass-- it's going to give me a force, and the force is great enough that it is actually going to cause the wire to elongate.
That is permanent shape change.
So what's going on?
Well, let's take this and blow it up.
When we blow it up, we have what's on the screen.
We have grains of different size.
And I'm just going to show atomic planes.
And that's why we have grains, right?
Because the atomic planes don't line up.
There's a misregistry along here.
So now I want to go down to the atomic level.
So now I'm going to take this and blow it up.
And what do I see here?
I see atoms like so.
I'm going to use that hard-sphere, close-packed model.
So here's the force.
The force is vertical, agreed?
We've got the mass and it's pulling straight down according to gravity.
So what's going on at the atomic dimension?
Because ultimately, this is what we're working with.
So what happens?
One possibility, and don't laugh-- this was actually suggested-- when you cause extension in this manner, you actually take some of these and you can turn them into elongated atoms.
No, no.
That's no good.
What in fact happens is slip.
So the force-- in fact, let me be more specific-- the applied force is resolved at atomic level.
So what happens is that the atoms slip over one another because they can slip along the planes that have the least bonding between them, right?
The chain is as strong as its weakest link.
So what will happen here is that in order to affect the elongation, these will move relative to one another along this slip plane.
This is resolved at the atomic level along slip planes.
I'll write that larger for the people way, way up in the back.
I haven't forgotten about you.
And what's the slip plane?
How do I determine which is the slip plane?
The slip plane it's going to be the plane that's strongest because it retains its integrity.
So which plane will be the strongest?
The one with the most bonds.
And which plane has the most bonds?
The plane with the highest number of atoms per unit area.
And that's why we study this stuff.
Because now we know, in an FCC crystal, which plane will slide.
So in FCC, the high-intensity plane is 1 1 1.
So the 1 1 1 planes do the sliding one over another.
And they'll zigzag the way you saw in the bubble wrap movie.
They'll zigzag down, across, across, across, But from a distance, it appears as the thing is elongating.
But at the atomic level, it's zigzagging along the close-packed planes.
Now I'm in a close-packed plane.
By the way, if I have the highest density in the plane of the floor, can you understand that orthogonal to the plane on the floor, I must have the weakest density?
All right, now I'm in the plane.
Which direction in the plane?
Any direction?
I'm going to push, I'm going to push some pasta.
I'm going to push uncooked pasta or cooked pasta?
Do you want to push on a rope?
Do you want to push on a broom handle?
The stiff.
So how do I determine, within the plane, which direction is the stiffest direction?
The one with the most atoms, which is the close-packed direction when I'm in the plane.
So if I'm in the 1 1 1 plane, which direction?
The 0 1 1 direction.
So this is direct line of sight from the original crystallography.
So that's how we get the slip.
And in BCC the highest density plane is 0 1 1 and so on.
And the close-packed plane simply means 1 1 1, because FCC is the closest-packed structure, so that's a close-packed plane.
Otherwise, they're planes of highest density.
It's a trivial-- forget the last column-- it's pedantry.
Forget it.
I'm sorry I have it up there.
So this is the slip system.
So the slip system is the combination of close-packed plane-- or closest-packed plane-- close-packed plane and the close-packed direction.
They all have close-packed directions because we've got atoms touching.
Close-packed direction.
That's how it deforms.
And then if I get a dislocation in a close-packed plane, voom!
Away they go.
1/10 of the applied sheer stress theoretical.
So if you want to get it down to a simple phrase, you can say, what causes slip?
Atoms slip, dislocations glide.
But please don't think for one minute-- listen to me carefully-- a lot of students come away from this and they say, oh, I know why we have deformation.
Because of dislocations.
No, no, no, no.
Atoms will slip no matter what.
It's just that you're going to pay a lot more to cause the deformation.
The dislocations allow you to have slip at lower cost.
Yeah, this is good.
And I think we're probably at the point where I can show you the enormous consequences of a three-dimensional defect.
Why did the Titanic sink?
It's a story of three-dimensional defects and human greed.
So this Tim Fecke at the National Institutes of Standards and Technology with an optical microscope and a rivet from the Titanic.
And what I'm going to show you in the rivet is a three-dimensional defect that took the boat down.
So this is the microstructure and what they're doing is they're showing you a blow up of this section of the rivet.
Now, the rivet is made of steel.
Before I can explain to you what's going on there, I have to show you-- [MUSIC STARTS]
Oh, shut up.
[MUSIC ENDS]
All right, so this is how they made steel.
Imagine this area the front of 10-250 as an open-hearth furnace, and I'm up to about my knees in liquid iron.
And on top of the liquid iron is a slag made of silica, calcia, and alumina.
It's the stuff when it solidifies it makes something not too different from window glass.
But we use it for refining to get some of the impurities out of the metal.
And those impurities go up into the slag phase.
And to keep this thing it about 1,300 degrees Centigrade, you've got burners.
Two.
One at each end like dragons-- [SOUND EFFECT]
--blowing flame into here for about 12 hours.
12 hours.
And that's what it took to make a heat of steel back in 1910.
This was made in Ireland, which was, at that time, part of the United Kingdom.
So on that day that they made the heat of steel that ultimately was shaped into rivets, evidently somebody became greedy, and instead of waiting long enough for the slag to phase separate and form a film on the surface of the metals, much like oil and water don't mix, but you can shake them up in your salad dressing application tool device and then pour it on quickly.
But if you let it sit they phase separate.
Well, somebody didn't allow this to phase separate.
They were told, pour the heat anyways.
So now you have globs of what is the precursor to silicate glass in the metal.
And now we take the metal and we cut it into little billets and the billets are extruded to form rivets.
And then the rivets look like this, where you have glass stringers.
Three-dimensional defects, impurity clusters, uncontrolled.
What are the mechanical properties of steel?
Strong.
Ductile.
Tough.
What does toughness mean?
Impactability.
When you say someone is tough, it means they know how to take a hit, whereas hardness means it takes enormous strength to cause deformation.
They're very different.
You know?
You say someone is tough, you mean one thing.
You say they're strong, it's different.
You can be strong and yet not tough.
You can be strong and brittle, so you can't take a hit.
I know a lot of people like that.
That can't take a hit.
Anyway, so enough about the people.
So here are the stringers.
So what happened?
What do you think the mechanical strength of this is?
If you have an impact like so, instead of having the toughness to absorb the impact, these things broke.
This is like perforations on a sheet of postage stamps.
And so these rivets were on the hull.
And when the ship hit the iceberg, what should have happened is there should have an a small hole, the water would've taken, who knows, a day or two to fill up, and there was plenty of time to get rescue vessels.
Instead, these rivets were brittle, thanks to this.
And so they just unzipped.
Just pop, pop, pop, pop, pop, pop, pop.
Opened up a giant hole and then took on the water.
Now, did nobody check?
Along the way, no inspection of the steel?
Nobody looked?
The guys that we're doing the riveting, they must have known the rivets were acting strange.
They must have broken a lot of them just putting them in.
But hey, it's not my job.
Not my job.
Not my job.
No testing anywhere along the way.
What's shocking is not human greed.
That's as old as humanity.
What's shocking is there was no checks.
So we have a different way to make vessels like this today.
But that's the story, right there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So these are the scores from test number two.
The celebration.
And as you can see, the average has gone down.
So as I've said in the past, I think everybody in this room has the intellectual capacity to be here.
Some people choose to be there.
I don't know why they choose to be there, why they choose not to go to recitation, why they choose not to try the homework, why they choose not to go to office hours, why they choose not to read.
But I guarantee you, when you make that choice you will be here.
And I have no requirements whatsoever to pass a certain number of people.
I can give everybody A's if I want.
And I can take a large number of people-- you know, look at this.
The way this is right now, the failure rate will be abnormally high.
So I would warmly recommend that if you're down here, that you look in the mirror and ask yourself a few questions.
OK. Let's get to the lesson.
We're going to start a new unit today.
The new unit is going to be the second half of the classification of solids.
We looked at solids and we reasoned that there were ordered solids and we've looked at crystals and their arrangements and so on.
And today, we're going to start talking about disordered solids.
And they're characterized by no long-range order.
They have short-range order.
You might know who your next nearest neighbor is, or your second next nearest neighbor, but you certainly don't know who your 10th nearest neighbor is or who your 100th nearest neighbor is.
And these solids are called amorphous solids.
And there's a simple one salable Angle-Saxon word for this.
It's called glass.
So we're going to talk about glasses.
What kind of materials can form glasses?
Well, obviously, materials that have trouble crystallizing.
And they come from a variety of walks of life.
So we have in organic compounds.
Some inorganic compounds can form disordered solids.
And a good example is silicates.
And this is what you know as window glass, glass that's used in bottles, cookware, and so on.
There are organic compounds that can form disordered solids and a variety of polymers.
Things like food wrap and so on, are also prone to forming disordered solids.
Some elements.
Some simple elements can form disordered solids.
A good example is sulfur.
Sulfur can form solid in crystalline form, but more often than not, it can form disordered solids.
And what I'm going to show you towards the end of the lecture is metal alloys.
Metal alloys can form disordered solids, and you're going to see metallic glasses.
And they're typically 80% metal and 20% metalloid.
That is somewhere along that red staircase on your Periodic Table that divides the metals from the nonmetals.
So a good example here is iron 80-- this is on mole basis-- and boron 20, boron being the metalloid.
And some, some do both.
Some compounds can form both crystal and amorphous.
So one example of that is SiO2, silica.
When it forms crystalline solid, we call it quartz.
Crystalline SiO2 is quartz, and amorphous SiO2 is the silicate glass that we know for windows and bottles.
And so as I've said many times, the term "glass" is related to atomic arrangement.
It has nothing to do with the ability to see through something.
Transparency has no place here.
So what are the conditions?
I mean, if you have silica and it can form crystalline or amorphous forms, how does it decide which to do?
And so I'm going to look at conditions promoting glass formation.
And I'm going to form glasses on the basis of solidification.
So I'm going to start from the liquid and go to the solid.
So I'm asking the question, why, on some occasions, does the liquid go to a solid that's amorphous and in other occasions it will go to a solid that is crystalline?
As in the case of quartz.
So there are primarily three factors that it boils down to.
And I like to make the analogy to the game of musical chairs.
So imagine I've got chairs placed around this central table.
You know how the game goes.
There are more people than chairs.
And the music starts and you walk around, and then at some point the music stops and people race for the chairs.
And some people are going to get a seat and they get booted out of the game.
And then one of the chairs is removed and then so it goes.
You know this game.
I know you don't want to admit to playing it, but your little siblings play it.
But of course, you don't play it.
But imagine if you were to play it.
The same situation here.
I've got silicate floating around in the liquid state, and the chairs are the crystal lattice sites.
So the question is, what promotes the ability to get to the lattice sites or not?
So obviously, one of the first things we think about is atom mobility.
Atom or compound mobility.
This is in the liquid phase.
And the way I want to write this, I want to talk about promoting glass formation.
So obviously, high mobility is going to enhanced crystal formation, isn't it?
If you've got high mobility, you're going to be able to find the right lattice site.
So I'm going to talk about if atom mobility promotes crystallization, the reciprocal of atom mobility will promote glass formation.
The second thing is the arrangement of the chairs.
If the chairs are in a simple line, it's easy to get to the chairs.
All other things being equal.
But if the chairs are arranged in some complex formation, it's harder to get to the chairs.
So that has an analogy and that's the complexity of the crystal structure.
The more complex, the more likely you are to form glasses.
And then the last thing, an analogy to the musical chairs, is the rate at which the music stops.
If the music stops suddenly you could get trapped in the liquid state.
If I gradually turn down the volume, you know, OK, wait a minute.
The music's going to stop.
You start moving towards the seat.
So that's the analogy.
Here is the cooling rate.
So a high cooling rate is going to make it more difficult for the system to find the crystal structure because as the cooling starts, the atoms start thinking, gee, it's time to form the solid.
But the thermal energy is removed from the system before the atoms have had a chance to find a crystal structure.
So the reciprocal of atom mobility, we write in a positive term and call that the viscosity.
Something, a fluid, a liquid, that has low atom mobility has high viscosity.
So I'll take this out and instead represent it this way.
This is the way to think about the formation of the amorphous state.
And why are we studying glasses?
Well, in the old days it was bottles and food and cook ware and so on.
Today, fiber optics.
Fiber optics is based on silicate chemistry.
Very important.
So understanding silicate chemistry has high-tech implications.
So let's look at, for the first study, the silicates.
The silicates for their value in fiberoptic technology.
So these are based on SiO2.
And some nomenclature.
This is called silica.
So the oxide of an atom is named by adding the term "a."
So silicon gives us silica as the oxide.
So here's the atom silicon, the basic element.
And if we fully oxidize-- see, here I've taken silicon, I put two oxygens, so I made a neutral compound-- that's silica.
And then if I fully oxidize-- the maximum number of oxygens I can put around is four-- and this is going to have a net charge of 4 minus.
And this fully oxidized anion is called the silicate.
And that's where we get the name of the family of glasses.
So this is fully oxidized.
So now let's look at the structure.
Silicate glasses.
sp3 hybridized.
Just like carbon above it.
sp3 hybridized.
So that gives us the four struts off of the central silicon at 109 degree angles to one another.
And let's put some flesh on the bones here.
So I'll put a silicon here in the center.
One, two, three, four.
But instead of putting silicons everywhere, I'll put oxygens at the end of the struts.
And then I'll put the second silicon, and you can see the oxygen is acting as a bridge between the two silicons.
One, two, three, four.
Let's do one more.
Another silicon.
One, two, three, four.
So again, you can see the oxygen acts as a bridge between the two silicons.
So you see four oxygens per silicon, but I've got two silicons per oxygen.
Hence, the structure looks like SiO4, but the stoichiometry of the compound is SiO2 because the oxygens bridge.
So this doesn't give you any clue as to what the structure should be, does it?
You really have to write this thing out.
Now, here's where the thing gets interesting.
This is a three-dimensional network.
All of these oxygens bridge to silicons.
And I was talking here about viscosity.
You can see that this thing in the liquid state is huge.
It's like a giant battleship, and this is just one.
So now we can have another silicate.
It can form chains.
It can form meshes.
And these meshes entangle.
So viscosity very high.
So when it goes to solidify-- David, if we can go to the document camera, please-- so here you see the silicate.
So the yellows represent silicon.
And you can see the oxygens, here, are at 109 degree angles.
So this is the SiO4.
We'll do some nanotechnology here.
So here's the SiO4.
One, two, three, four.
And now I'm going to make the bridge.
But look.
The bond between the oxygens and the silicon is defined in three dimensions.
It's a 109 degree angle.
But the bond between the two oxygens is not defined.
It's only defined in two dimensions.
So you can see that if I put all of these oxygens on the same plane-- so somebody borrowed this and look, they lost an oxygen for me.
Got a missing oxygen, here.
If you find this thing, please bring it to my office.
So now all five oxygens are on the same plane.
One, two, three, four, and five.
And then up here are the two silicons.
They're in the same plane.
The oxygens are in the same plane.
What I'm showing you here is the beginning of crystalline quartz.
Crystalline quartz, this stuff.
But now what happens if we cool quickly?
This bond is only specified in two dimensions, which means I can hold this bond at the proper value and this is free to rotate.
This is free to rotate.
Look at the autofocus on this thing.
Who designed this?
Boy, what a stupid machine.
So anyway, so here we are.
Here is the point.
This bond is free to rotate, and when it rotates, look-- now this oxygen is no longer in the plane with this oxygen.
And as a result of that freedom to rotate, we can end up running out of thermal energy.
And this is nowhere near its regular crystalline array.
So this gives you the indications of the high viscosity, moderate cooling rate will end up giving you the amorphous silica.
But in all cases, this bond is the linkage through.
So we end up with this three-dimensional network.
So what do we do with to show that we have some evidence for this?
David, may we cut back to the slides, please?
So how do you characterize?
I'm going to use x-rays.
So here's an x-ray diffraction pattern of the-- the upper one is cristobalite, which is one of the polymorphs of crystalline SiO2.
And you can see you have distinct peaks indicative of satisfying Bragg's Law.
There's some width to them, but that's because it's a real crystal.
Down here, this is the amorphous silica.
You don't see all of these features.
You see only one very broad peak.
Now if I said it's crystal-- pardon me-- if it's glass, you'd say, well, it has no long-range order.
So why do we have even this one feature?
What's this one feature indicative of?
Doesn't matter how much disorder there is, I still know that no matter what, I'm always going to have these four oxygens as my nearest neighbors.
So these four oxygens minus any long-range order gives us this one line here.
So we have evidence for it.
Now let's look at the energetics, because clearly, these are two different states.
One of these is lower energy than the other.
Which one is it?
How to think about the problem?
There's a simple way to do it with the tools we have in 3.091.
Energetics can be given by bond formation.
Energetics via bond density.
When things form bonds the energy of the system decreases.
So which one is going to have higher bond density?
Well, the higher bond density clearly is going to be exhibited by the one that has the tighter packing.
And which one has tighter packing?
The disordered solid or the ordered solid?
The ordered solid has much more dense packing.
So the ordered crystal exhibits tighter packing, therefore, this means more bonds per unit volume.
So that means, to me, that the energy of the crystalline state must be more negative than the energy of the glassy state.
That makes sense so far.
But now I want to show you one other thing that we've just inferred from this little exercise.
If we get more bonds per unit volume, then can you see that we've made an association between the binding energy of any arbitrary ensemble and it's molar volume?
So volume now, is a very easy thing to measure, isn't it?
You can see it with the naked eye.
So the volume of something with a given composition-- and we're talking about a mole, we're talking about equal numbers of atoms-- so the molar volume is indicative of binding energy.
It's a one-to-one correspondence.
If you like, the Vmolar is a measure of disorder, meaning the higher the molar volume, the greater the disorder.
Or put the other way, the smallest molar volume is that exhibited by the crystal.
So we have some traces here that come from the reading.
So this comes from the archival lecture notes that were written by my predecessor, Professor Witt.
There's a little typo here that obviously heating means increasing temperature, not decreasing temperature.
Make a little correction here.
So what we're plotting is the volume, the molar volume of silicate glass as a function of temperature.
And imagine we start with some blob of glass-- of some known mass-- so we can divide and figure out what the molar volume is, and we started cooling it.
We cool down until we get to the crystallization temperature of quartz.
And when we get to that temperature crystalline quartz forms and along with it, a tremendous decrease in the volume.
Unlike water ice, which is a rare exception, where the ice occupies a larger volume than the liquid, for most systems, the solid is more compact than the liquid.
And that's what you see here.
There's an abrupt drop here at the melting point.
And then when we continue to cool, there's some thermal contraction.
And as you can imagine a hot solid occupies a greater volume than a cold solid.
And so this is the cooling curve and you can retrace it, heat up, and when we get to the melting point, there's a tremendous expansion and then off we go.
So that's the classical form of the crystallization of a material that is undergoing the normal process of liquid to solid transformation.
I'm going to use some tea colors today.
So we went down the red line.
Now let's go down the green line.
So we're going to go down the green line.
We go down the green line, we're going to use the same stuff, this silicate network, only we're going to cool a little bit faster.
And we cool a little bit faster, we can zoom right on past the normal melting point and create a supercooled liquid.
And that supercooled liquid gets lower and lower in temperature and all the while the volume is shrinking.
Again, a hot liquid what occupies a smaller volume than cold liquid.
You know this from a mercury-- the old days, you remember these old thermometers?
You probably have never seen one of these things.
It's all done digitally.
But we used to have these liquid and bulb thermometers.
You'd have numbers on here, and what happens is the temperature goes up, the liquid in here rises to a higher temperature.
And at a lower temperature this contracts.
Isn't the solid expanding, too?
Uh-huh.
So what's the other thing you learned from this?
It must mean that coefficient of thermal expansion, which we affectionately will call CTE, the coefficient of thermal expansion of the liquid, must be much greater than the coefficient of the thermal expansion of a solid.
Otherwise, the two would expand and you wouldn't get any sensible measurement out of this, right?
Well, we see that on a curve.
We see that on a curve, because when we plot volume versus temperature, when we're up here in the liquid regime we have a steep slope.
This slope here is dv by dt.
And when we get down on the solid regime, it's a gradual slope.
It's still a slope, but it's a gradual because the coefficient of thermal expansion down here is very low.
And that's what you're seeing here.
So at some temperature the system changes from a supercooled liquid to a solid.
But it's a disordered solid because we've quenched in all of that remaining liquid disorder.
And take a look at this-- you've changed from the slope that's characteristic of the coefficient of thermal expansion of a liquid down here to the gentle slope coefficient of thermal expansion of solid.
You know what this proves?
This proves that glass is a solid.
There are many people out there, even in the popular press, who will say, glass is just a very, very viscous liquid.
Nonsense.
Look at this.
It has this coefficient of thermal expansion.
And don't fall for any of that nonsense they tell you when you go to the cathedrals in Europe and the glass is thicker at the bottom because it's been dripping for 400 years, and that proves-- you know why the glass is thicker at the bottom?
Because they made it by spinning.
And when they spun it, it was graded in thickness from the center out.
And now, if you're the glazier and you're putting the glass in a window, which way would you put a pane of glass that had variable thickness?
Would you put the thickest part up or the thickest part down?
It's thicker on the bottom because that's the way it was made.
Lord help the tour guide when there's an MIT student taking 3.091 on that tour.
All right, so here we are.
Look at what you have here?
You see this?
At this temperature-- let's say down here where the lines end, it's room temperature.
At room temperature the volume of the crystalline solid is low.
The volume of the amorphous solid is higher.
That's proving this.
Vmolar is a measure of the disorder.
And sure enough, the glass that was cooled quickly ends up quenching in more of the liquid state disorder, and you see that in terms of what I call the excess volume.
The excess volume is a measure disorder because the crystalline solid non-zero volume.
So we can define, we can go from this directly over and say that v, that's the excess-- that's a pun.
You know, instead of writing this?
This is the way they do in high school.
This is the excess volume.
Bologna.
We don't write like that.
Excess volume is equal to v of the glass minus v of the crystal.
And the greater the excess volume, the greater the degree of disorder.
So let's do one more.
Let's go down the orange line.
The projector isn't giving us a good yellow component, here.
OK, so this is the orange line.
And what's the difference between the orange line and the green line?
The difference between the orange line and the green line is that in the orange line we're going to cool more slowly than we did on the green line.
Because green means go.
So that's the fast one, right?
So here we are.
We still get supercooled liquid, but we get down to a lower temperature before we have ceased to have higher and higher viscous liquid and have formed the solid.
The knee in that curve occurs at a lower temperature.
And that value is called the glass transition temperature.
Why do we call it the glass transition temperature instead of just saying it's a solidification temperature?
Because when I say solidification, before today you would say, OK, liquid became a solid.
But after today, if someone says to you, solidification, you say, do mean ordered solid or disordered solid?
So we distinguish.
So every crystallization is a solidification, but every solidification is not a crystallization.
So here we are.
Look at this-- this has a smaller excess volume.
Because if it was a slower cooling, that meant that's the equivalent of giving more time, more thermal energy, for things to find their crystalline position, which means there's less excess volume quenched in.
OK.
So let's just get that down, define these things.
So this is a measure of disorder.
Or glassiness, if you like.
OK.
So now let's define these two different temperatures so that we have the distinction.
So we have, first of all, the classical one, which is crystallization.
And crystallization represents the reactions of liquid.
In both cases, we're going to convert a liquid to a solid, but a liquid goes to a crystalline solid.
And that occurs at a unique temperature called the melting point.
Imagine if I talked to you about the freezing point of water.
Freezing point of water at atmospheric pressure is 0 degrees centigrade.
If I asked you, well, if I cool the water really quickly or if I cool it slowly, does it make any difference to the freezing point of water?
No.
Why not?
Why isn't all this happening?
Because water is a tiny molecule and so cooling rate has no impact on the temperature of conversion.
Those water molecules will always find their lattice sites.
So this is a function only of composition.
If I put something in the water and I make it impure, I know I can change its freezing point.
If I put salt in water, I will depress its freezing point.
But if I put pure water, it will free at 0 degrees centigrade.
Later on, we'll learn that there are some pressure effects, but today that's not going to elucidate anything.
It'll just be a distraction.
So pure water, always the same thing.
But now if you go to something like silica, which has the capability of forming a glass, the glass formation is a different reaction.
Glass formation is written in this manner.
We will start with supercooled liquid.
It's liquid, but I'm already going to stipulate that it's liquid that's been cooled below the melting point.
Supercooled means it's cooled below the normal melting point.
So a supercooled liquid is going to form a glassy solid, and this occurs at tg.
tg, which is the glass transition temperature.
And that is very much a function of cooling rate.
And, of course, the function of composition.
Obviously, if we change the composition from SiO2, we're no longer comparing apples to apples.
So composition is important in both instances, but only in the case of supercooled liquid do we form the glassy solid.
And the degree of disorder is a function of the cooling rate.
And how is it a function of the cooling rate?
As the dt by dt, right?
Change in temperature with time as dt by dt goes up, the degree of disorder goes up.
So if I want to quench in the liquid state, can you imagine if I had liquid and I could instantaneously cool it?
I would quench in all of the liquid disorder.
If I quench it less rapidly, there will be some time for the atoms to strive for a degree of crystallinity.
So all of this we've said with reference to silicate glasses.
Now, there are other glasses.
What are other glass forming oxides?
Well, what do I have to look for?
I should look for other compounds that form bridging oxygens, right?
That's the key here.
What's the unifying feature?
It's bridging oxygens.
Bridging oxygen leads to glass formation.
So what are other compounds that could give us such bridging oxygens?
Well, you could be lazy and say, well, if silica does it, then why don't I look up and down the column on a Periodic Table.
If you go up to carbon that's no good because it forms gas, but if you go underneath you will form germania glasses.
So this is germania, and the glasses are germinate glasses.
You can also go to group three.
B2O3.
B2O3 forms sp2 hybrids.
And the sp2 hybrids off of each boron, we have three oxygens.
And that oxygen can bond to another boron.
One, two, three.
And this is all going to lie flat in a plane, isn't it?
But this oxygen bridge is only specified in two dimensions, whereas the boron struts are specified in three dimensions, which means this oxygen bond between the two borons doesn't have to lie in the plane.
It could tilt this up and if it does, this is going to have a greater volume.
You can see this with the naked eye.
I mean, I could have just taught the lessen by putting this slide up.
If those are equivalent numbers of borons and oxygens, it's plainly obvious that on the right side you've got excess volume.
This occupies a much larger volume than the image to the left.
The image to the left is crystalline B2O3.
The image to the right is the same stuff, except in many instances this boron-oxygen-boron bond doesn't lie in the plane.
It tilts out of the plane and it causes all sorts of distortions and leads to excess volume.
So this is called a borate glass.
All right, so if we can do it with the borates, we can do it with anything that will form covalent bonds.
So we can do it with P2O5, phosphate glasses.
V2O5, vanadate glasses.
As2O5, arsenate glasses.
And Sb2O5, stibnite glasses.
And these are used, actually, as additives to some of the glass that's put in computer screens so that they will gobble up excess oxygen on cooling and avoid bubble formation, which then makes the glass foggy.
And a foggy computer screen is no fun to try to look through.
So what are the properties of these oxide glasses?
Well, first of all, they're chemically inert.
Why are they chemically inert?
Because they've got strong covalent bonds.
So if you try to react something with them, it's going to take a very special compound that can trigger reactions.
Which is why they're used for bottling beverages and packaging foods.
You put vegetables and fruits in glass jars going back to ancient times.
Until recent times, with the advent of the soda can, there were glass bottles and so.
They're electrically insulating.
Why are they electrically insulating?
Electronic structure.
These are all covalent bonds, high band gap, which is why the amorphous version is used in window glass.
High band gap, which means light goes through.
How do I think about whether something is transparent?
Do a little finger demonstration.
This is the band gap of the glass, and this is the band energy, the photon energy.
If the photon energy is small it goes right through.
That's called transparency, see.
You can study quantum mechanics, but you know what?
This is it.
That's all you have to know.
Now what happens if the band gap is small-- like around 2eV-- and here's the photon energy?
That's called absorption and re-emission.
Transparency, absorption.
That's all it takes.
You think I'm kidding.
That's all you need to know.
So now, the next thing.
Mechanically brittle.
Why are they mechanically brittle?
Strong bonds.
No opportunity for slip.
Please don't tell me-- this is what students tell me every year and they get zero for the stupid answer-- that the reason glass is brittle is it's a distorted solid and therefore has no dislocations.
Uh-uh.
Dislocations take the stress required to cause slip and reduce it to a lower value.
But you cannot cause this thing to slip once it is solidified.
The only way to get this silicon to move relative to that silicon in a vertical shear-- let's say I want to make this silicon move up and this silicon move down-- there's only one way.
It's called break that bond.
When you break that covalent bond, that's called fracture.
So there is no slip allowed because we have strong covalent bonds.
If you want to reshape glass, what do you have to do?
You have to heat it back up above its glass transition temperature.
But you do not shape glass once it is solidified.
You may have tried in the home, and then you end up with something called shards.
And that's the reason.
Optically transparent, we know that one already.
OK. Last thing is the very high melting.
Melting point of silica is over 1,500 degrees Celsius.
You're saying, well, wait a minute.
He's telling me we're going to make beverage containers, we're going to make food containers, we're going to make all sorts of useful objects, but that's going to cost a lot of energy to go away up there to melt this glass.
But it does have desirable properties-- chemically inert, and so on.
So what can we do?
Back here.
Glass transition temperate is a function of cooling rate and a function of composition.
So next step is let's modify the composition of the silicate network in order to drop it's processing temperature so that we can make beverage containers at acceptable energy.
So what's it going to take?
I'm going to have to do something about those bonds.
Because those bonds are what caused me to have to go to such high temperatures.
So in order to reduce the processing temperature of silicate glasses via change in composition-- so this is material science-- change the composition in order to get desirable properties.
And specifically, I want to lower the processing temperature.
Even when I get these things liquid, they don't deform very well, because all these networks are entangled.
So I'll give you another analogy.
Suppose you're in the kitchen and you're going to cook some pasta.
So you take some linguine and it's about a foot long.
So you got this pound of linguine and you cook it in the boiling water and you pour in into a colander.
And you may or may not rinse it with cold water.
Whatever.
Just leave it in the colander for a little while and what you'll find is the whole thing turns into one big mass.
I'm not talking about the stuff got all gooey.
I'm just saying it just hangs together.
And if you're clever you can just gently slide it out.
And it'll slide out of this one big mass.
What's holding those strands of pasta together, by the way?
Well, are they covalent bonds?
Are they ionic bonds?
Are they metallic bonds?
I don't know, if you've got metallic pasta, you've got digestive problems.
So what's the bonds?
It's van der Waals bonds, right?
Now, I can cause them to slip again, can't I?
I can put a little water in there.
And what does the water do?
It goes in between and then it's got hydrogen bonds and they slide.
I can put a little oil in there and then that slides.
or.
The other thing I could do is-- you know, have you ever seen some people, they break the pasta before they boil it?
I did this experiment.
So you take the pound, divide it two halves.
By my math, that's two half pounds.
So you cook the one half pound one foot in length and you cook the other half pound-- break them in three.
So they're about four-inchers.
And then you put them each in a separate colander.
Which one do you think is going to be much more difficult to move around?
It's the long stuff, right?
The long stuff entangles.
So we're going to do the same thing here with the processing.
And basically, you can study and learn pretty much everything you need to know about polymeric networks in the kitchen with a pound of pasta.
All you need to know.
All right, so let's go and look at it.
I want to reduce the amount, the length, of those chains.
If I reduce the length of those chains, I can process at much, much lower temperatures.
So what's my weapon, here?
My weapon here is oxygen.
Oxygen is going to go in there in the form of the oxide ion.
And the oxide ion, O double minus, has a very, very high affinity for the bonding.
And so what'll happen is oxide ion attacks the oxygen-silicon bond.
It attacks the oxygen-silicon bond and breaks it.
It breaks it in two and then incorporates itself into the network in the following way-- you see I have to silicons joined across an oxygen?
This free oxide ion will come in here, interrupt that network in the following manner.
So it's now broken the silicon chain.
Now, I've got conservation of charge.
This was 2 minus.
I don't see any exposed charge here.
I need 2 minus.
This'll be minus 1, this'll be minus 1.
I have conservation of charge, I have conservation of mass.
And what I've done is I've broken the chain.
This is called chain scission.
Shorter chains, higher fluidity.
Higher fluidity, which means lower processing temperature.
Now, where am I going to get my-- I can't go to the lab and get a bottle of oxide anions.
I have to have charged neutral species.
And so what I'm going to look for is an oxide ion donor.
I need an oxide ion donor.
And where do I find an oxide ion donor?
Well, better be something that's going to be a cation, isn't it?
Because if I've got an oxide anion, I need a cation.
And what kind of a cation?
How do I make an oxide anion?
I have to have electrons to give to the oxygen.
So I need a good electron donor.
Or to use a simple Anglo-Saxon word, a good metal.
So a good metal oxide like, say, calcium oxide.
So if I take calcium oxide and I dissolve calcium oxide in silica, it dissociates to give calcium cation plus oxide anion.
And then the oxide anion migrates over here to our pal the oxygen bridge and results in two of these broken pieces.
So we call this one a bridging oxygen, as I've been saying up until now, BO.
And this one is called a terminal oxygen.
This is bridging oxygen, this is terminal oxygen.
Or some people, I don't know why-- they have no literary skills-- they call it non-bridging oxygen.
I hate that because it tells you what it's not.
So you might see non-bridging oxygen.
So all of these up here, the silicates and so on, these compounds that have the ability to form oxygen bridges, these are called network formers.
These are all network formers.
Because they have the capacity to make oxygen bridges.
And then compounds like calcium oxide that have the ability to donate oxide ions that break bridges, these are called network modifiers.
So good examples-- any good metal.
So I can look at lithium oxide, sodium oxide, these will all disassociate to give oxide anion.
If calcium will work and you believe Mendeleyev, then you should probably think about magnesium oxide, calcium oxide, and anything in that series.
A good ionic oxide like lanthanum oxide, yttrium oxide, these will be oxide ion donors.
If you want to get yourself fired, use scandium oxide because it's frightfully expensive.
And you can go to Group four.
You can even use something like lead oxide or tin oxide.
Even they will donate oxide ions.
In fact, you can put a lot of lead oxide in, you'll modify this thing so much that it'll start to allow you to cut crystalline facets and this'll be called lead crystal.
Why do we use lead?
Well, number one, it modifies so I can cut it and I get a straight line instead of that conchoidal glass edge that if you've ever cut yourself you'll never do it again.
The other thing is lead, because it's got so many electrons, has a very, very high index of refraction.
So when you make your billion and you've got your crystal chandelier hanging, of course you want the cut crystal with a high index of refraction because it's going to make that candlelight look really elegant and romantic and so on.
And if you make a super boatload of money, then you're not going to go with lead crystal.
You're going to make diamond pendants, right?
Because they've got a higher index.
And we all want the best index, don't we?
OK, so I think this is probably a good place to stay.
So let's jump to the end, because we've got a few things here.
So I said I was going to show you about metallic glass.
Here's metallic glass.
1959, Pol Duwez, who was a professor at Caltech, reasoned that if he were able to cool liquid metal very quickly, at, say, a million degrees per second, he could freeze the random orientations of atoms in the liquid state and prevent them from finding even something, either simple cubic, body-centered cubic, face-centered cubic lattice.
So he worked with gold silicon.
Gold silicon has a very, very deep eutectic.
Even though gold melts to over 1,000, mixed with silicon, it gets down to about 400 degrees Celsius.
So he dropped this through a little orifice here.
This is molten and it drops onto a water-cooled copper wheel spinning at a very high speed.
And it makes a ribbon some tens of microns wide.
And what came out here was disordered solid.
It was metallic glass.
This was the birth of rapid solidification, and this is metallic glass.
This is metallic glass.
This has no long-range order, no grain boundaries, no dislocations, but it's not transparent.
Why?
Because it's a metal.
And what do you know about band gaps and metals?
And it's different.
It was a different Q-factor.
It has different mechanical properties.
It has no ductility in a classical sense.
This is just for your reference.
This is aluminum foil.
And you know what aluminum is.
It's got ductility.
It's got no grain boundaries so the corrosion properties are different.
It's got no magnetic wall boundaries.
So that a transformer made of this stuff is half the mass of a transformer made of crystalline glass.
It has some other uses.
So it's used in magnetoelastic resonators for theft prevention-- oh, I'm sorry.
I'm in Cambridge.
I can't say theft prevention-- for inventory control.
And so you see that it's 20% boron and all of this metal-- iron, chrome, and moly-- and they put this inside the object in the store.
And then there's a permanent magnet there, iron-cobalt-chromium, that sets the metglass to a field.
And then when you walk through the uprights, there's a 58-kilohertz signal that causes this thing to ring.
So it excites and listens for the ring.
And if you still have that little piece of metglass in there that hasn't been demagnetized-- Bing!
And then, oh, jeez, I'm sorry.
I really am.
I meant to pay for this.
Explain it to the police officer.
All right.
So that's inventory control.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK, OK. Let's settle down.
Weekend is over.
Tomorrow, weekly quiz.
Today I'll have office hours 3:00 to 4:00.
I have to go down to Washington, so I've got to leave a little bit earlier than normal.
So I will be available from 3:00 to 4:00 The lecture has started and there's still way too much talking in here.
Way too much.
You know how much is too much?
Any.
Any.
So last day, we started talking about oxide glasses, and we reasoned that we could have control of the properties by control of the composition.
We started with a network former, which is some oxide that has the capacity for forming covalent bonds through a bridging oxygen.
And then we wanted to drop the processing temperature, and we did so by adding modifiers.
Intermediates, we haven't talked about and we're going to do that in just a moment.
So if you look in the readings, this is from archival notes that were written by my predecessor, Professor Witt, these are compositions of some typical glasses.
I don't expect you know these from memory, but I would expect you, if I gave you the composition, explain to me why the various constituents are there.
So let's try few examples.
The first one is soda-lime glass.
And you see it contains silica, which is the network former.
It contains sodium oxide, calcium oxide, and magnesium oxide.
And these are alkaline earth oxides that are ionic, and so these are acting as network modifiers because they're donating oxide anions that go in and break the silicate chains.
And then there's this Al203 and that's sort of halfway in between, isn't it?
Silica is Group 4, or 14, if you want to use the modern notation.
Sodium is Group 1.
Calcium, magnesium, Group 2.
Alumina is Group 3, and it's sort of halfway in between.
It's amphoteric.
It can either be a former or a modifier.
And in these instances, depending on how much modifier is present, alumina can act as an intermediate.
And what's an intermediate do?
An intermediate is a covalent oxide.
It's a covalent oxide, or an oxide that can act as both covalent and ionic.
But in this instance, it's acting covalent oxide with a different coordination number.
And what does that mean?
Different coordination number?
It coordinates-- remember last day I showed you B2O3?
Borate glasses?
So they have a coordination of three, whereas silicates have four.
And what that means is that there's going to be a mismatch.
There's still strong covalent bonds, but they don't fit quite right.
And that's going to give even more free volume.
And that excess free volume, through covalent bonds, gives you the ability to endure thermal shock.
So if you want to impart thermal shock resistance in a glass, you give it lots of free volume so it can take the rapid change in temperature.
So it's a covalent oxide with different coordination number.
That is, nearest neighbors, in a covalent sense, from that of the network former.
So you can add a borate or aluminate, what have you.
And that'll give you thermal shock resistance.
And so alumina here is acting as an intermediate.
Let's go down here.
There's borosilicate.
borosilicate is the generic term for Pyrex.
So Pyrex is a trade name.
Pyrex was invented by Corning, and it contains silica as the network former.
There's some sodium oxide, potassium oxide-- in rather small amounts.
You can see this isn't a heavily modified network, but look at this.
13% B2O3 and 2% alumina.
That's the modifier.
And what was the hallmark of Pyrex?
It had thermal shock resistance.
So you could take it out of the oven and put it under cold water and it didn't shatter.
And we have the same analogous behavior for glass ware in the laboratory.
For all room temperature and low temperature work in the laboratory, we use Pyrex that has this resistance to chemicals and resistance to heat.
This is the big one, here.
And that gave birth to functional crockery in the kitchen, where you could work in glass instead of in metal.
And down here we see glass-- let's see, well, here's one.
Light flint optical.
See, that's 54%.
It's down to 54% silica.
And look at this-- boatloads of lead oxide.
And the lead oxide is acting as a modifier and also changes the index of refraction.
It modifies so much that we have almost down to the orthosilicate.
So that the chains are modified to the point where they're almost all terminals.
There's very little of this, and most of it is just terminal oxygens.
And that means that it's more nearly crystalline, and therefore, you can cut it.
And this is the lead crystal.
Lead crystal has that high value.
So you can see how that comes out.
And this is also taken from the reading.
It's a plot of viscosity versus temperature.
Only it's a logarithmic plot.
And what do you see?
Here's pure silica.
That's SiO2.
And if I want to take it down to the point where, if the viscosity goes down, you have the ability to work with the glass.
So if I want to melt silica, I've got to way, way up here.
Over 2,000 Centigrade.
So if I want to make bottles, if I want to make cookware, I don't want to run a lehr at this temperature.
Lehr.
L E H R. It's where you melt glass.
Lehr.
I don't want to run a lehr at this temperature.
But you can see if I add modifier, the more modifier that I add, the more I break the network.
So increasing modifier decreases network connectivity, and that means I can go to a lower temperature and get the same level of fluidity.
So there's a whole bunch of definitions here that I'm not going to go over.
You'll have this slide.
But basically, it's just different points in processing.
So if you go up here.
Strain and annealing.
I mean, these are very, very-- you're especially working with solid, whereas softening point, you start getting the glass to flow.
And the working point, that's the viscosity that you have to get below.
Otherwise, the glass is going to be to resistant to flow.
You want to get the glass tacky so that you can put it into an injection mold, shape it, as they do with bottles.
They take a blob of glass and boom, they just throw it into a mold and it sprays out.
And if you've got the right mass and the right spinning, it makes the wall thickness proper.
But you can imagine, if the glass is really, really viscous, it's not going to flow well enough.
If it's too fluid, it'll drip all over.
So there's an optimum in there.
And this is telling you how to figure out what that optimum is.
And you can see that as you change the composition, here's the working value.
This is the viscosity that you have to get below which in order to work.
And you can see that as you add more and more modifier, you can take the temperature and get it way, way down.
So to work with silica you have to be about around 2,000.
With soda-lime you can be down around 800.
So that's going to cut your energy costs, isn't it?
And it's going to make it easier to recycle.
What's the point of recycling if you consume as much energy to recycle as if you started with virgin material?
At least with virgin material you can guarantee the quality of the feeds stock.
So there's got to be some big saving.
All right.
So that gives you a some sense as to what we can do technologically with glasses.
I want to show you one last thing with glasses, and then we're going to move on to another topic.
You know, new week, new day, new topic.
So I want to go back to this curve.
So what am I showing you here?
I'm showing you that as we change the cooling rate, we change the amount of quenched-in excess volume.
So fast cooling quenches in more of the liquid free volume than slow cooling does.
That's why this V excess is small here, whereas V excess is large here for the other one.
I'm going to use that in glass strengthening.
So I want to strengthen.
Strengthening glasses.
We're talking about silicates or borates.
Strengthening oxide glasses.
Glass is a fantastic material.
It's really good in compression, but it's no good in tensions.
You know this.
If you try to bend glass, it'll break.
Why?
Not because it doesn't have dislocations.
It's got strong covalent bonds.
But you know, suppose you want to strengthen the windshield of your car so that when a stone hits it, it doesn't shatter.
What can we do to give it added strength?
So I'm going to show you two ways, and both of them operate under this principle that the yield stress-- this is the stress that will break the glass, and I'm going to say the effective yield stress, what you experience in life-- is equal to the sum of what I'm going to call the natural yield stress, which is the basic property of the glass.
Plus, I'm going to increase the surface stress.
What I'm going to do is I'm going to modify the surface, and I'm going to put an additive stress at the surface, and that's going to be a compressive stress.
So that means now the effective stress that I need to break the glass is going to be greater than it otherwise would be.
So the whole gambit here is surface strengthening.
Service strengthening, which means surface modification.
And I'm going to show you two ways to modify the surface to bring the strength of the glass up.
The first way is thermal.
Thermal treatment.
And the thermal treatment is to strengthen the glass.
Well, let's see.
Let's keep it in the context of the windshield.
The technical term for this, the technological term is tempering.
So what I'm going to show you was how we can look at that volume versus temperature curve and understand how we make tempered glass for windshields and so.
So I'm going to show you a slab of glass.
Here's a slab of glass.
And we just got below the softening point-- so T just less than T softening.
So now this thing's going to start, it's continuing to become more and more viscous and getting closer and closer to glass transition temperature.
And what we're going to do is we're going to introduce air jets at the surface of the glass.
What that's going to do is it's going to cause accelerated cooling at the surface.
And the same thing happens on both surfaces, so I'm going to cut this piece of glass in half, and we're just going to look at the upper surface.
The same thing happens in a lower surface.
So let's now blow this up.
I'm just going to look at the upper-half surface and I'm going to divide it into two zones.
This is the first level, most primitive finite element analysis.
Finite element analysis.
I'm going to divide it in two, and I'm going to say, this has got two zones.
Two cooling zones.
Here is the zone of the center, OK?
This is the center.
And I'm going to call this the inner portion.
And then there's the outer portion.
Well, take a look at this curve here.
Which is going to have slower cooling?
In the center or near the free surface?
The slower cooling is in the center and according to this graph, the slow cooling has a smaller residual volume.
I've written, V-interior.
That's the green line.
And then the upper one is a yellow line.
So the upper one where it's high cooling, it's going to have a higher volume.
So this is the second piece of finite element.
So I'm going to model this one like so.
So this is outer.
And the same thing happens on the other side, OK?
So it's happening on the bottom as well.
But we're just looking at the top because there's symmetry here.
So you see what I've done?
This is longer because that graph says it wants to be occupying a larger volume.
Problem is the glass can't do this.
I can do this with a piece of chalk, but the glass isn't going to look like that.
The glass is going to have a flat edge.
And how can it have a flat edge?
Because the bottom, here-- this is not to scale, let's make it more to scale-- the top is a narrow zone and the bottom is a big, thick zone, isn't it?
So this big, thick zone, which has a small volume, is going to pull on the thin upper zone, which has a large volume, and pull it in.
Can you see that?
And it's going to cause the introduction of compressive stresses.
So we got to compressive stresses simply using that graph and a little bit of air.
So you take that graph, put differential cooling, and now you've introduced compressive stress.
And that's all tempered glass is.
So the V-excess of the outer layer is greater than V-excess of the inner layer, or the interior, if you like.
And why?
Because the cooling rate, the change of temperature with time of the outer zone, is greater than the cooling rate of the inner or interior.
And there it is.
That's the beginning.
And so now to fracture you have to apply a greater stress then you would have otherwise.
So that's good, and that saves a lot of lives.
There's another way.
There's another way to surface strengthen, and that's a chemical treatment.
And again, what am I trying to do?
I'm trying to introduce a compressive stress, but I'm going to use a chemical means.
And this one is called ion exchange.
And this is used in technology, too.
So now I'm going to take a piece of glass here.
This is solid glass and let's put some components in here.
So I'm going to put some silica as my network former.
I'm going to put some sodium oxide.
And I'm going to put some modifier, B2O3.
So I got all three here.
Former, modifier, intermediate.
And just to put a little skin on the bones here, I want to show what the sodium oxide actually looks like.
Sodium oxide goes in as sodium cations, and oxide anions.
The oxide anions go in and they break some of the silica chains.
But the sodiums don't get involved in that.
They just sit around as spectators.
Now what I'm going to do is I'm going to put this, I'm going to soak this in molten salt.
Soak in molten salt.
This is huge area of my own research.
And the molten salt, one example might be I'm going to take potassium chloride.
Remember, we talked about making aluminum or magnesium?
This is one of the constituents of the melt in which we make electrolytic magnesium.
Potassium chloride.
And it exists as potassium cations and chloride anions.
Chlorine is green, except we know-- you know, this is the way chemistry books write it, but that's stupid because this is isoelectronic with argon.
And it's not green.
I know.
I've looked at this stuff.
It's clear, colorless, and transparent.
The chloride ion has to be clear and colorless.
It's got a complete shell, but the chemistry books will make it green.
Anyway, here's the potassium ion here.
And there's a lot of potassium ion here.
There's no potassium ion inside the glass.
There's sodium ion in the glass, there's no sodium ion in the molten salt.
But these are both media in which ions live.
So can you see this is sort of like the perfume bottle?
Things move from high concentration to low concentration.
Some sodium wants to leave and enter the melt.
And some potassium wants to leave the melt and enter the glass.
And where's the potassium going to go?
It has to go where there used to be sodium.
What's the relative size of potassium ion versus sodium ion?
Potassium ion is bigger.
So potassium ion goes and occupies a site formerly occupied by sodium and causes compressive stress, because you get all these big ions jamming in there and that's going to lead to compressive stress through ion exchange.
So through ion exchange we can raised the temp.
Since r, the radius of potassium is greater than the radius of the sodium, we get a force fit, and this is how we got solution hardening.
Hardening is the metallurgical term for raising the strength.
Hardening is equal to strength going up.
So we get hardening by solution.
So this is solution hardening.
Solution hardening because the potassium ion is in there.
So we have surface.
And this gives a lot of strength.
We do the same thing metallurgically if you carburize the surface of a piece of steel.
Tool steels, for example, surface carburized.
So we get hard surfaces that can cut.
You say, well, why don't you just put the carbon in all the way through?
Well, then the tool bit will be brittle.
So you want something that's got toughness in the center so it can take the impact, but it's got surface hardness so it'll cut.
And that's an engineered material.
We've got one set of properties at the surface, and we've got another set of properties in the bulk.
And it's all brought to you by control of chemistry.
And then that's also control of chemistry.
OK.
So I mean, I could talk more and more and more.
Glasses are just fascinating things.
But we've got to get moving.
So we're going to move to a new topic.
We're going to move to a new topic today.
We're going to start talking about kinetics.
Kinetics.
And what is kinetics?
Kinetics is the topic that we put into 3.091, because it's important, of course, but it's all about the study of reaction rates.
It's the study of reaction rates and mechanism.
So why do we study?
Well, we study it because I think it belongs here.
But first of all, kinetics is related to productivity and resource utilization.
You've got two factories.
One produces 200 units per time, the other produces 100 units per time.
The one that's producing 100 units per time probably isn't going to be in business that much longer.
And it's all about understanding how to get more throughput per unit time, and that leads to competitiveness.
So if you're interested in international competitiveness, you've got to know something about kinetics.
The two go hand in glove.
Second thing is energy and the environment.
A number of you have come to talk to me after class, sent emails because you're interested in energy and the environment.
Efficient use of energy involves understanding the kinetics so you can force chemical change with the least amount of energy utilization and do so in a way because its mechanism-- if there's two mechanisms that allow you to get to the same end product, choose the one that has the least toxic impact on the environment.
Otherwise, you're going to have to spend money to avoid the effluents, which then gets you back up to number one.
Your cost of doing business is higher than the other guy.
Guess what?
You're creamed in the marketplace.
And lastly, societal.
And this is one that excites me, is kinetics leads to productivity, which keeps factories open, which keeps people working.
So through understanding kinetics.
You go and you look at places in the United States where factories have closed.
In many, many instances it was because the technology lagged.
If you could just keep things going faster.
You've got to make things-- you know how fast you've got to make them go?
You've got to make them go so fast that even if the foreign workers are paid zero, you can still beat them in the marketplace.
That's your standard.
So that's how you design your processes.
You design your processes so that you can produce at very, very low cost.
And there's a huge range in reaction rates.
You know, you can start the range of rates, rates of reaction.
At the one that you have very, very slow.
And way over here you have very, very fast, all right?
So on the slow end, you have stuff that's sort of cosmic scale.
Geological.
And then over here, what do we have?
We have explosions.
You might say, well, explosions are usually bad, aren't they?
Well, no.
I can give you an example of where explosion is used to your advantage.
The airbag in the automobile.
There is no pump, no mechanical pump fast enough to inflate an airbag on demand.
So how do we make those airbags work?
And I hope none of us ever has to be present at the deployment of those airbags.
But you know, just academically let's talk about how they work.
There's like an explosion that occurs when you're driving.
There's an accelerometer and this is a key piece.
The accelerometer has to make a decision whether you're applying the brakes just because you've lost focus and you're, you know, whoops!
You've had that jerky brake.
Or maybe this is one of those holy-mackerel moments, and your jamming on the brakes, and we'd better deploy the airbags.
Once the accelerometer decides that this is one of those holy you-know-what moments, it sends an electric current through a wire, which then raises the temperature to 300 degrees C and causes this reaction to take place.
When this reaction takes place, this is a solid, nitrogen is a gas.
So the gas has a much, much higher volume than the solid and within milliseconds you get inflation.
Well, that's good and that inflates the bag and prevents you from hitting the hard parts of the car.
But you see the other byproduct?
That's sodium.
And if you look on your Periodic Table, elemental sodium is liquid above 98 degrees Celsius.
So now you've got liquid sodium.
There's no point preserving the safety of the occupants of the car just to cover them with liquid sodium.
So we better do something about that.
So being chemists, what we do is we add potassium nitrate inside the bag, as well.
so.
That that sodium is mopped up with potassium nitrate, which converts it to sodium oxide, potassium oxide, and a little more nitrogen.
[BLOWING SOUND EFFECT] Keep the bag nice and firm.
But now what happens when the Fires show up and they [SOUND EFFECT], you know, they start pouring water on it.
This turns into caustic.
So now you're going to get covered and wet lye.
So that's probably not too good, either.
Just remember, you've been physically saved from smashing your skull on the tempered windshield.
So it's so far, so good.
So now what?
You want a mechanical, you want a Newtonian death?
Or do you want a Coulombic death?
I guess that's the question, here.
And so we're going to keep doing some chemistry here and we're going to add silica.
And what happens if we add silica to sodium oxide potassium oxide?
Well that's a network former, these are network modifiers.
We'll make an alkaline silicate glass and that's OK because we put food and beverages in alkaline silicate glasses.
So everybody is happy.
So that's all the chemistry that goes on inside an airbag.
And we've got to understand the kinetics.
So I hope I've whet your appetite.
OK, that's enough of that.
Now let's get to the real stuff.
So let's write a general chemical reaction.
The general reaction.
What's the formalism if we want to set up the metrics?
So I'm going to write just a plain, old equation.
This is the classical P-Chem stuff.
Little a moles of A, plus little b moles of B go to little c moles of C plus little d moles of D. So this could have been lecture two.
Just a straight chemical reaction.
Remember we were studying stoichiometry?
So the constituents of the left side of the equation are called the reactants, whereas on the right side of the equation we have the products.
And what kinetics gives us, kinetics tells us the rate of conversion.
Or the rate of reaction.
I'm using all these terms so you understand their equivalent.
So we can write something like this where the conservation of mass kicks in.
So I can't make products any faster than I consume reactants.
There's no sources or sinks here.
When I lose reactants, I make products.
So I'm going to say, the rate of change of the total mass of the reactants-- this is summation sign-- I'm just saying the rate of change of the sum of all of the reactions must equal the rate of change.
So the loss rate of reactants must equal the gain rate of all of the products.
That's just simple stoichiometry.
Or some people call this conservation of mass or Law of Mass Action, what have you.
And then we count by concentration, usually.
Just reminder.
This is lowercase c. So this is the concentration of species i goes as the mole number.
n is mole number divided by the volume of the reactor.
So this would be in moles per meter cubed if we were in SI units.
So therefore the rate of change, the ci by dt, is really the rate of change of mole number, isn't it?
It's moles of i disappearing, but the volume doesn't change, typically.
And then the last thing is we can use this idea up here that concentration of products can only equal the loss rate of the reactant.
So I can write, term by term, the normalized rate loss of the concentration of a divided by its stoichiometric coefficient, for example, would equal the normalized weight gain, or pardon me, the concentration gain of the concentration of d. So this is just saying a disappears no faster than d appears, mediated by the stoichiometric coefficients.
So this is just Law of Mass Action, which is stoichiometry in motion, isn't it?
That's all it is.
Now here comes the cool thing kinetic theory.
We look at any reaction, I can tell you right now what it looks like.
If I applied concentration of i as a function of time, I start off with some value c naught, some initial value, and it just attenuates.
We know that's going to happen as we consume the-- it falls and there's some curvature here.
What we're trying to do next is to give some mathematical representation to this.
Can we come up with a mathematical formulation of the shape of that curve?
And we can, and here's the central tenet kinetic of theory.
Kinetic theory says that the reaction rate can be expressed in terms of a driving force.
And that driving force is the instant concentration.
OK.
These are lofty words.
I'll show you what it means with illustration.
So that's the overall idea.
So now let's say that the rate of change, the instant rate of change of concentration ci, is proportional to the instant value of the concentration.
And you know, the concentration is changing, right?
As the concentration decreases, the rate of change decreases.
So just this alone, this idea alone would rationalize a curve like that.
As the concentration falls, the rate of change falls, which means the concentration falls, which means-- and raised to some power-- it's not linear, necessarily-- raised to some power.
There's a power law at work here.
So now let's do this one more time.
I'm going to write it mathematically.
But I want you to first see the word, concepts.
So let's write it mathematically.
That means minus dci by dt-- so this is time rate a change of concentration i, the minus means it's falling-- is equal to the concentration of i raised to some power n. This is order reaction.
It's not mole number over there.
You've got to be pluralistic today.
n is going to be used in different ways.
So this is called order of reaction.
And there's a constant.
See up there?
It's a proportionality.
Here it's an equal sign, thanks to the constant.
And this is called the rate constant.
And this what all I need for plant design.
Suppose I'm running a chemical plant and I'm taking a plus b and making c plus d and the management says, we want to double the rate of productivity.
So you go, I know what to do.
We'll increase the concentration.
They say, by how much?
Well, this allows you, if you know the value of k and n, you want to double this?
Then you know what to do here.
It's not necessarily double the concentration, Because this is a 1.5 power.
By the way, this doesn't have to be an integer.
Could be anything.
Could be not necessarily integer and must be determined by experiment.
You can't look at an equation and say, oh, that's going to be second order because there's a two in front of something.
So here we are, plant designers.
And the management wants to know, how do we make things happen?
So I'm going to go back to that equation over there and I'll say that most generally, the rate of change of concentration of a will then go as rate constant times the concentration of a.
See, the rate of change goes as something to the power alpha.
But it could also be influenced by b.
So I'm going to put-- this is the most general form.
Then once you make your measurements, a lot of these fall out.
I'm even going to put c I'm going to put d. Everything was in that equation.
You might say, well, wait a minute.
How can the rate of consumption of a be influenced by the concentration of d?
d is a product.
I'll give you two examples.
One is that d, when I'm making d, d happens to have some catalytic value.
So when I start making a convert to d, d catalyzes the reaction, which makes it go faster, which makes more d, which makes the reaction go faster, in which case this is going to have a profound effect.
And unfortunately, there are some other situations where a converts to d and d retards the reaction.
And so I start off with pretty decent conversion efficiency, but as I make more and more d, the d chokes the reaction, in which case the value here is going to have negative implications.
So that's why you write it most generally, and then you go and you make some measurements in an experiment, and you figure out what these are.
Some of them might be zeroes, and away you go.
So I'll give you an example of one.
Here's one.
It's Monday after Halloween, so we'll get something kind of toxic.
This was the manufacture of phosgene that was banned by international convention.
It was used as one of the toxic gases for trench warfare in World War I.
So it's made by the reaction of carbon monoxide and chlorine.
COCL2.
This is called phosgene.
It's very bad stuff.
I used it, actually, in my research.
It has really good dehydration properties for salts.
If you've got any moisture in potassium chloride, this'll go after it and turn the water into carbon dioxide and HCl.
But you've got to be really, really careful with this.
Gas leaks lead to a bad day at the lab.
All right, this was banned.
It's still use by tyrants throughout the world, and that's why we have international war tribunals.
So here's the kinetics of it.
Here's the kinetics of it.
dc by dt.
So this is the rate of consumption of carbon monoxide determined by experiment goes as the concentration of carbon monoxide and the concentration of chlorine raised to the power 3/2.
So what I've done is mimic here.
So in this case, gamma and delta are 0.
Gamma equals delta equals 0, and it looks like beta equals 1.5 and alpha equals 1.
So I would say that this reaction is first order in carbon monoxide of order 1.5 in chlorine, or overall, of order 2.5.
Because, right?
This is the order of reaction.
This is c to the power 1, isn't it?
We don't write the 1.
This is great.
So we know, to increase the rate of reaction we increase the concentration.
There's another way to increase the rate of reaction.
What else can we do to increase the rate of reaction?
Increase the temperature.
Things go faster at higher temperatures.
Again, how?
How much?
Suppose I said, I want to double the rate of reaction.
Do I double the temperature?
Do I raise the temperature by 10 degrees?
How do I do that?
So to increase rate of reaction, we can increase temperature.
But how?
And by the way, where is temperature here?
I don't see temperature.
I don't see temperature in this rate equation.
You could say, well, if you increase the temperature you're going to increase the gas volume.
But that's piddling effect, right?
If I double the temperature, I double the gas volume.
It doesn't change the mole number.
The temperatures in here.
That's where the temperature.
Increase temperature, which then operates on the rate constant.
That's the only place left in that formalism.
But what's the quantitative value?
Quantitative value was annunciated to us in 1889 by the Swedish chemist Arrhenius.
And what Arrhenius found, he found that the rate constant varied with temperature in the following manner.
Rate constant goes as the exponential of the ratio of minus some quantity call the activation energy e sub a-- I'm going to tell you what that means in a minute-- divided by the ratio of the product of the Boltzmann constant and temperature.
And there's a constant of proportionality which we denote A in honor of Arrhenius.
Or another way to write this is A times exponential-- are you familiar with this?
Instead of writing e to the something, if there's a messy argument up here you can write it exp.
This is the same as this minus ea over the ratio of Boltzmann constant and temperature.
So what's in here?
What's in here?
Boltzmann constant temperature, we've seen that already.
What's the Boltzmann constant temperature product?
That's the environment.
That's the energy of the environment.
So this is the final environment.
And this thing up on top, ea, is a barrier.
It's a barrier.
Remember we saw the band gap?
Same idea.
Those ideas go all through physical chemistry.
You got thermal energy that allows you to do this 10 trillion times a second.
Everybody in this room is doing this 10 trillion times a second.
That's what kT is.
And then there's a barrier energy.
Every once in a while something happens.
Sometimes one in a million, sometimes one in a billion.
Whatever.
What Arrhenius found was this relationship, which means we could plot the variation of the rate constant.
This is a mess, right?
This is an exponential-- look, the eye can only see straight lines.
What if I show you this?
What's that mean?
I don't know.
Is that first order?
Second order?
Is that Lagrangian?
Is that exponential?
I don't know.
I know it's a curve.
But what's this?
I know what that is.
And you know what that is.
And you can tell goodness of fit.
This is what science is all about.
Forget all the other stuff.
You know, all that UROP stuff.
You're going to work in a lab and discover stuff.
Forget that.
This is what it's about.
You got data.
Data are all over the map.
Here's the thing-- I need to come up with some f function here versus a g function here that linearize my data.
That's all science is about.
And how do I get to f and g?
Well, there's two ways.
One is trial and error.
The other way is physical understanding.
And this is what Arrhenius taught us.
Arrhenius taught us that the f function for how k varies with temperature, right?
The problem, I'm trying to say, k versus t. That's what I'm trying to do.
And if I plot k versus t I get a curve.
No good.
Instead, if I plot some f of k versus g of t, I get a straight line.
And this is all science.
So what's f of k?
The natural logarithm of k, right?
If I've got k equals a exponential, I take the natural log of this, and the natural log of an exponential is just the argument.
And what's in the argument?
What's the T function?
1 over T. So if I plot the natural log of k versus the reciprocal of the absolute temperature, I get a straight line and its slope is minus ea, forgive me, minus ea over k-Boltzmann.
Minus ea over k-Boltzmann.
So now it's linearized and I can look at that and within an instant, anybody in this room can look at and say, the data conform or they don't.
They don't.
So now let's think about this.
We say well, what's the value of this barrier?
It's called activation energy.
This barrier energy is called activation energy.
ea is called the activation energy.
And it typically, it varies, but it has values-- just to give you a sense-- about 1 electron volt.
1 electron volt, which you know now is about 100 kilojoules per mole, isn't it?
I hear the chorus of yesses.
1 electron volt.
What is thermal energy?
What's kT?
kT at room temperature, k-Boltzmann at room temperature is about 1/40 of an electron volt.
Now, we are chemical machines.
This conversation is occurring because all sorts of chemical processes are at work in me and you.
And how's that happening?
When all we've got to suck out of the environment is 1/40 of an electron volt and we need on the order one electron volt to drive certain processes.
I guess we're all dead.
Any ideas?
Ay yay, yay.
You guys need-- what's that thing?
You've got to call your lifeline or something?
Come on.
How does anything happen?
How do we get-- Why does anything happen in this world?
Why should anything happen in a world with an environment of 1/40 of an electron volt?
Catalysts
Oh, catalysts.
Catalysts.
Yeah, yeah.
I just went to the catalyst store and I got catalysts.
Come on
Potential energy
Oh, potential energy.
Jeez, this is good.
Who can give me the wackiest answer?
Can anybody give me the right answer?
How come anything is happening?
Over here.
Again?
Distribution of temperature
Thank you.
Thank you.
For a moment I thought we were all dead.
At some level we were.
So, yeah, it's Maxwell-Boltzmann.
It's this, isn't it?
And here's room temperature.
And here is 1 electron volt.
And this is it.
And if you use this and you put it back to there, pretty soon you derive the Arrhenius equation.
And if you increase the temperature you know what happens.
After you increase the temperature, this.
Time to use a hot color or chalk.
So now this is T2 greater than T1.
That's what's going on.
That's what's going on.
So now what is this activation energy?
What is the meaning of it?
You know, I've told you there's a barrier energy.
What does it mean?
Well, you go to the textbook, you see goofy stuff like this.
And this is not a slam against the textbook.
This is a slam against all chemistry textbooks, because they all write this stupid stuff.
You see?
a plus b goes c plus d, the energy falls and you have to go over this activated complex.
You memorize it, and I ask you to repeat, it and we leave the room and we think, wow, we know physical chemistry.
There's nothing here.
This is nothing.
This is stupid.
Now this is a little bit better.
Can you see this?
This is the Maxwell-Boltzmann.
This is actually a beautiful graph up to a point.
See the Maxwell-Boltzmann?
Low temperature, high temperature.
Only they've turned this thing on its side.
See?
And there's the energy you need to get over the activated complex.
You see what's wrong with this graph?
What's wrong with it?
The product
Yeah, the product is at a higher level than the reactants.
I looked at that and I thought, whoa!
I guess you'll activate them, but they won't go anywhere.
Anyway, so this is all stupid.
What can we do?
What can we do?
So I decided to give you a mechanical analogy.
So imagine this is the loudspeaker.
Can you say the x?
People in the back, can you see the x?
Give me a thumbs up.
Great eyes.
All right.
So what do we see here?
The center of mass, I've indicated here.
All right.
So this is in a certain energy state.
Let's get the right graph up there.
This is awful.
All right.
So this is a plus b.
Now this is c plus d. You see, the center of mass fell.
It's lower.
It's closer to the table than it is here.
Can you see when I go like this, the photon goes?
Watch this.
Yeah, only I can see it.
Your eyes don't go to that end of the spectrum.
All right.
But what's wrong with it?
So why does the box not fall over?
Why does it not fall over?
We agreed that this is a lower energy state, and it is.
You're correct.
So why does the box manage to stay here and not fall over?
[INAUDIBLE PHRASE]
Exactly.
This is the box at 0 Kelvin.
Now we start raising the energy of the box, it starts doing this.
Because above 0 Kelvin, it vibrates.
The higher the temperature, the greater the vibration.
But we're still in trouble here, you see, because you've only got 1/40 of an electron volt.
So now we invoke distribution.
I don't know what's happening with any box.
Heisenberg tells me that.
I don't know what's happening to any box.
But if I take Avogadro's number of boxes I'll get this distribution.
And some boxes will vibrate very little.
And some boxes will vibrate a lot.
And what's the critical level of vibration?
It's to get to here.
Because once it gets to here it's downhill all the way.
And that would be the activation energy.
And the fraction of boxes that get to the value of activation energy tip over and then I restore the distribution, because now this one's out of the game and now I distribute that energy over the remaining ones.
And as I increase the temperature, the amplitude of the vibration increases, the average increases, the fraction in that red zone increases, and at some temperature it's so, so hot that they actually do this, in which case I have equilibrium.
The two states are in equilibrium.
So that's it.
This is what's going on.
So in three space, you can't go from here to here without going to here.
You can try going this way so you-- no, you have to do this.
It still has to be mechanically activated so we can show that this is what activation means.
So we can plot-- this deserves it's own board-- so we will plot something like this.
And now you'll see this.
Sometimes they write this in the books.
It's kind of goofy.
But they write stuff like this.
This is some kind of an energy coordinate.
This is an energy coordinate.
And this is called a reaction coordinate.
OR sometimes they write extent of reaction.
Again, some P-Chem term.
They usually use Greek C just to make it lofty, but it's meaningless.
And then so you write like this-- reactants here, products here, and this is exactly what we're seeing.
What do we see?
We see this.
Here we have the box sitting like so.
And then over here we have the box sitting like so.
And right here we have the box up like so.
And so now you have, here's the initial energy.
This is the energy of the reactants.
E of the reactants.
This is E of the products.
So then this distance here must be delta E of the reaction.
And what's this?
That's the activation energy.
I have to come up with this; otherwise, I can't make the box fall over.
So this is ea to go from here up onto the corner.
And now you understand what all this stuff means.
So that's here.
And this is the activated complex.
What's an activated complex?
It's a box on its edge ready to fall over.
That's what it is.
All right.
I think we're out of time, so let's cut to the end here.
What've we got.
Oh, just some plots.
We'll get to that next day.
All right.
So I want to show you a little bit about first-order reactions.
This is radiocarbon dating.
n equals 1.
And strictly speaking, this is not a chemical reaction, but it is first order.
So I'll lump it in here.
So this is a nuclear reaction.
And what happens in the upper atmosphere?
Radioactive carbon produced by cosmic rays, which generate neutrons in the upper atmosphere and then those neutrons attack nitrogen to make carbon 14.
That's the radioactive form of carbon.
Remember?
It's present in one part per trillion, if you look on your Periodic Table for the isotopes.
Now carbon 14 enters the carbon cycle.
And so in all of us and in all living organisms the ratio of carbon 14 to carbon 12 is one part per trillion.
And carbon 13 is also present.
I don't know.
What is it?
1 point something percent?
The thing is, the carbon 14 is radioactive.
And it decays.
Like this.
But what happens after somebody dies?
Well, they stop exchanging nutrients with the surroundings.
They stop breathing, so their carbon level is pegged as it was at the time of death.
And now it's a one0way street.
It's just carbon decay.
And it turns out you can measure the ratio of carbon 14 to carbon 12 to determine the age, which is given by 5,730 years, which is the inverse of the rate constant.
Now, you can't use this for crime scene investigations, but certainly you can start dating things hundreds of years old, not hundreds of minutes old.
So there's a whole bunch of other things that can be used, not just carbon.
So this is radiochemical dating.
And these are used in trying to nail down art forgeries and all sorts of things.
All right.
So here, you can see these.
These have all been radiocarbon dated.
You know, the Dead Sea Scrolls were radiocarbon dated and they look like they're from about 2,000 years ago.
Makes sense.
You go out over here.
They they found these Indian sandals from Oregon, and then they looked on an electron microscope, and they found something on the sandals that looks a little bit like that.
That's a Nike swoosh.
I don't know.
You people are so-- All right.
So now I want to show you the Shroud of Turin.
The Shroud of Turin, as you may know, for a long time was reputed to be the burial shroud of Jesus Christ.
And this is taken from the National Geographic back in the late '80s.
Back in the late '80s, they did a major study on it.
So this is the artist rendition of how the shroud might have been wrapped around a body.
The question is, what's the date of this?
So we can use radiocarbon dating.
And so here's the formation of carbon 14.
Carbon 14 exchanges with carbon 12 in flax.
So it's got one part per trillion.
When the flax is harvested the carbon 14 starts to decay.
And then we look at the weaving of the shroud and so on.
And they had three different labs, they took samples from the shroud.
And they concluded that the shroud dates from this period-- 1260 to 1390.
And the first time it was mentioned in the literature was around 1354.
So everything seems to make sense that this is not the burial shroud of Jesus Christ.
It's something that has importance to certain members of the Roman Church, but it's scientifically not.
You know, this stirs up a lot of passion.
So people started saying, wait a minute.
Wait a minute.
There was a fire in 1532 and people repaired the shroud and they were using candlelight and maybe the paraffin and mold and so on covered the fibers.
So now, are we looking at the surface effect?
How do we know what really happened?
The only way to know is to take the whole shroud, put in a big Cuisinart and break open the interior.
Well, in point of fact, what they did was take fibers and look inside.
And I think as of the late '90s, the Roman Church says, it's not the burial shroud of Jesus, but it is an object of reverence.
And I think it's a good example of how science sometimes comes up against spirituality, and we have to be careful how we handle it.
It's a delicate matter.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK. Let's get started.
Going to go right into the lesson.
Last day we started looking at kinetics, which was the description of rate and mechanism of chemical reactions.
We looked at the general rate law, which shows us that the instant change in concentration of species i goes as the instant concentration.
So rate is proportional to concentration.
And the order of reaction is shown here as n and proportionality constant is the rate constant k. So we saw that we can make the rate of reaction increase by increasing the concentration in the reactor.
We saw there was a second way to increase the rate of reaction, and that was to raise the temperature.
And this is the Arrhenius relationship that shows that the rate constant changes with temperature according to this exponential relationship.
In the denominator, we have the product of the Boltzmann constant and the absolute temperature, which is a measure of the ambient thermal energy that's available to promote reaction.
And in the numerator we have a barrier energy, which we call the activation energy.
We'll say a little bit more about that today.
And the constant of proportionality out here is A, which is in honor of Arrhenius, who first annunciated the law.
So what I want to do today is to look at some common orders of reactions.
So we're going to look at the values of n that are encountered in not insignificant number of reactions.
So let's start with the simplest one.
The simplest one is first order.
First order reactions.
That means n equals 1 in this rate law expression.
And here's an example of something that we could use.
The decomposition of N2O5 into NO2.
And gives us half a mole of oxygen.
And we can write this rate law in terms of the rate of disappearance of N2O5.
So I can write minus d by dt of-- I'm going to use square brackets to indicate the concentration-- of N2O5.
And this has been measured.
You can't predict this from the stoichiometry of the equation.
You have to measure this, and on the basis of measurements it's found that the rate of change of N2O5 goes as the concentration of N2O5.
And there's a constant out here.
The rate constant.
So just to make the point, I'm going to put a 1 here.
Meaning that n equals 1 and that's why this is a first-order reaction.
But this is messy.
There's too many characters here.
So I'm going to compress this and give myself a nicer font, so I'm going to let the concentration of N2O5, just in this expression, just be c. Lowercase c. That way I can speed this thing up.
So I can rewrite this expression here as minus dc by dt equals k times c. All right.
And so we know what this is going to look like.
They all look like this.
If we plot concentration versus temperature we expect to have the maximum value of this at time zero.
So at time zero I start off with-- if you want c-star or c max-- and you know, this thing attenuates.
But I can't look at that and decide whether it's first order or not.
The only thing the eye can really tell is a straight line.
So what I want to do is I want to modify this thing.
I want to map c into some f of c. And I want to map t into some g of t. So that if I plot f versus g, I get a straight line.
And then I can take one look at that and go, bingo, this is first order.
Or not.
Or maybe it's first order and the data are very noisy.
So what we'll do is we're going to integrate that equation.
So I'm going to play with this a little bit.
So I'm going to transpose.
I'll get minus dc over c equals kdt.
And then I can integrate this because I know at time 0, t equals 0, c equals some value-- I can call it c-star or c naught-- and at any arbitrary time later, I can go down that curve and put some value of time and then I'll end up with some arbitrary value of concentration c. And I know there's some math people in the audience who are probably horrified that the limits and the integrand are both the same.
So I'll put a little tilde here that way I don't go to math jail.
So now I'm not integrating c. If I wanted to be a pedant and really make the math instead of me enslaving the math, the math enslaving me, watch this.
What I could do is I could make this a carrier variable, s. You know what I'm talking about now, right?
But let's get really crazy.
This is how bad the math can get.
You can do this in a chemistry book.
Let's use some Greek.
So I'll make this d xi over xi.
Now I've totally left the class behind, but it's mathematically correct.
What we will do is we'll put the t back and put a tilde over it.
And now everybody is happy, and heaven forbid, the student might even understand the lesson.
All right.
So we do that and what do we get?
The integral of dc over c is a natural log of c. This'll be natural log of c equals natural log of c naught minus kt.
OK, so integrating this thing gives us this expression.
And that means if I plot the natural log of c versus time I should get a straight line.
And furthermore, the slope is equal to minus k. So here's an example that's taken from your book.
It happens to be another first-order reaction and it shows the decomposition of cisplatin.
This is a compound that's used in certain cancer treatments.
And you can cisplatin hydrolyzes to form this monster cation plus a chloride.
And it turns out that the rate of decomposition of cisplatin is first order.
Will you look at that curve?
That purple line?
I don't know what that means.
But watch.
If I instead plot the natural logarithm of the concentration of cisplatin as a function of time, you get a nice straight line, and furthermore, the value of k is obvious from the slope.
So this is an indication of the power of taking the proper order of reaction and then using it in-- if you'll pardon the pun-- straightening things out.
And a couple other things worth noting.
I mentioned at the end of last day, radioactive decay.
Even though radioactive decay is not a chemical reaction-- it has to do with nuclear phenomena-- and we know chemistry is restricted to the life and times of the electron, we don't go into the nucleus.
We simply know there's something in there, there's neutrons, protons, and a whole bunch of other particles.
But chemistry is about the life and times of the electron.
Radioactive decay turns out to be first order.
It's good to know that because then you can appreciate what goes on.
So for example, you could take something like the decomposition of uranium-238.
One of its decompositions gives you thorium-234 plus helium.
And it turns out that this is first order, which means that if you were to take a plot of the natural log of the concentration of uranium versus time, you get a straight line.
And this slope would be the k, the rate constant.
But it turns out that the nuclear chemists, nuclear engineers, prefer not to talk about the rate of reaction in terms of k. But instead, they like to use the inverse of k, which is the half-life.
So instead, express k through half-life.
And the half-life, as the name implies, is the time it takes to decrease the concentration of the species by half.
So if I start off here at c1 and down here I have c2 equals 1/2 c1, then this is the time it takes to cut the concentration in half.
So through half-life-- and we can go through this equation here, just put c naught over 2 and do a little bit of manipulation, you'll end up with the natural log of 2 divided by the rate constant, which is 0.693 divided by k. So you can see that there's a direct relationship.
It turns out that the half-life for this reaction is 4 1/2 billion years.
4 1/2 billion years.
So that means after 4 1/2 billion years the concentration has gone down to a half, but even then it's probably not safe for the kids to go out and play.
So you really need to think about it in this way.
So here it is shown in your textbook.
Concentration as a function of time.
They could have plotted it log linear, then you would have ended up with this.
But anyways, they're chemists, so what can you expect.
So here it is.
Here it goes down to a half, and then this is a half of a half as a quarter.
It's the same time interval.
And then half of a quarter is an eighth, and it's the same time interval.
So you get the sense.
And the most important thing is the half-life is independent of concentration.
It doesn't matter what the starting concentration is-- if you know what the half-life is, you know how much time it takes to get to a half.
And you know enough math.
What if I said, well, what if I didn't want to go to a half?
What if I wanted to go to 10%?
Well, you can figure out how to modulate half-life into tenth-life.
That's trivial.
But the book, being a chemistry book, will show you a table in which they give half-life values for orders of reactions other than 1.
And it's stupid because except in the case of first order reactions, the value the half-life is a function of the concentration.
So what good is a constant that's variable?
I don't know, but I didn't write the book.
The only time I care about half-life is in the case of a first order reaction, which is when you have nuclear decay for sure.
All right.
So that's enough about first order.
Let's go look at second-order reactions.
So here's an example also taken from the book.
So second-order reaction, which means that n equals 2 in the rate law expression.
n equals 2 in the rate law expression.
And the example that they gave was NO2.
See, look.
We went from N2O5 to NO2.
And then we can keep the decomposition going.
NO2 goes to NO plus O2.
And again, I'm going to let, in this case, c will equal the concentration of NO2.
And what you end up with in this case is, if you go through the similar analysis, you get minus dc by dt.
It's been measured that the rate of loss of NO2 goes as the square of its concentration.
So this is based on experimental evidence.
It's not because this is a 2.
This is not.
Please pay attention here.
This 2 is not the consequences of the stoichiometry.
This 2 was determined experimentally.
I could have just written this reaction NO2 goes to NO plus 1/2 O2.
That doesn't make this first order, does it?
Experimentally in the laboratory.
All right.
So now what we can do is we can integrate this thing.
And so we'll get minus the-- let's do it this way.
Boards are cheap, so let's jump down to the next one.
So now I can then put minus the integral of dc tilde-- cross multiply-- so dc tilde over c tilde squared equals k, the integral dt tilde.
And we can integrate this from zero to some time, and this will go from c naught to some concentration.
And that will give us 1 over c equals 1 over c naught plus kt.
So this is now the new, improved way of mapping c versus t into f of c versus g of t. So f of c as 1 over c, which means now if I plot 1 over c versus time, I'll get a straight line.
In this case, it's got positive slope.
It's got positive slope, so that's what it's going to look like.
And this is k. So if I take our data set and I plot it like this, then I can make sense of it.
And I think we've got some data here.
So here's the data from this decomposition.
And again, what do you see?
A line that attenuates.
Can you look at that line?
Can you tell that line is second order, whereas the other one that looks like this was first order?
Of course not.
The only way you can tell his to fit it into some linearizing function.
So look at that.
What they've done, that's the least squares fit.
They plugged it into a least squares fitting equation.
You can fit a straight line through anything.
It doesn't matter.
But what's that mean?
I don't know.
So here's the log.
This is really important-- look carefully.
So somebody took these data and then plotted them as the natural log, which is what you do if it's first order.
And look at what happens-- it gets a lot better but it's still not a straight line.
But if you were in a hurry, you'd just say, force the fit.
You'll get a straight line and you'll assigned the value of k to it and party on.
And it's wrong.
So now let's try this.
Beta.
1 over c versus time.
So what do we get from that?
I get two things.
First of all, the fact that it's a straight line means that it confirms n equals 2 and the slope of that line is the value of k. So in point of fact, what I've been doing here is teaching you how to determine the rate of reaction.
So let's do it.
Let's codify this.
So determination of order of reaction.
So in other words, the way the story goes is I give you a data set that looks like this-- pardon me.
I give you a data set that looks like this and I say, determine the order of reaction.
So there's two general ways to do this.
The first one is called the integral method.
And that's what we've been doing up until now.
The integral method, it's called the integral method because it's using the integrated form of the radio question.
See?
This is the integrative form.
This is the integrative form.
Hence, the integral method, and it's trial and error.
That's what we did.
Trial and error.
But we don't say trial and error because we're at MIT.
So what do we say?
We have a lofty way.
What's the lofty way of saying trial and error?
By inspection.
By inspection.
Never tell anybody you do things by trial and error.
We don't do trial and error.
By inspection.
It's $50,000 a year.
It's what you get.
OK, so here's what I do.
I'm impatient, so I try n equals 1.
If I get a straight line on the log plot, good.
If not, I try n equals 2.
If I got a straight line on 1 over c, good.
If not, go to the second way of determining.
I give up.
Won't waste my time.
It's a two strikes and you're out thing.
So what's the second method?
The second method is called the differential method.
The differential method.
And this is fun because it's data fitting and it gets into some nice detail.
So what I want to do is to define the rate.
What's the rate?
The rate is over here.
Minus dc by dt, right?
You see up in the top left corner the rate law.
Minus dc by dt.
I'm going to call that positive r. I'm just compress-- this is like French cooking.
I started with this.
Look at all those fonts.
Then I compressed it down to c, and now I compressed it even more.
It's just so much information compressed in that little r there.
Poor little r. All right.
So now, let's rewrite the rate law expression.
So that's r equals kc to the n. There's the rate law expression written very compact.
And now I'm going to take the logarithm of both sides.
So taking the log of both sides, I can write natural log of r, the log of a product is the sum of the log.
So that's going to be natural log of k plus the log of, da, da, da.
n log c. n log c. And I don't care what kind of logarithms you use.
You can make this log base c. You can make it log base 10.
If you're feeling like you want solidarity with the Mayans, you can make it log base 5.
I don't care.
But make them the same.
Make them the same.
And then what happens?
Well, let me show you.
So let's say we've got our data, start off with this data set like so.
So this is concentration versus time and we end up with this.
It comes down, down, down, attenuates.
Remember, what are we trying to do here?
We're trying to take this data set and figure out, what is the value of n?
We're trying to determine n. So what this thing says is.
if I make a plot of r versus c so that it's on a log scale, so if I plot log r versus log c, the slope is going to be n. So how do I take c versus t and construct log r verses log c?
That's what we're going to do.
So here we are and I can start here that at time zero.
This is time zero.
That's the slope, right?
The slope here is the rate.
So slope, I'm going to call it r naught.
Because that's the time-- it's dc by dt r naught at t equals t naught.
But there's no t on here.
So what am I going to put?
It's when t equals t naught and c equals c naught.
So now I'm going to take some colored chalk and we're going to show you how to construct this.
So I've got the slope by taking minus dc by dt here.
So I'm going to take this value, r naught, and this value, c naught-- I want to gang them together, I'll bring over here and put them right out here-- so now I've got r naught.
Well, actually I should put the abscissa first.
So this is going to be c naught r naught.
And now let's do it again.
So we'll come down here to some later time.
Let's call this t1.
And at t1 the concentration has fallen to c1, and I take the slope of this.
Why am I taking the slope?
Because it's dc by dt.
That's what the rate is.
So here now I've got slope r1 at c equals c1.
r versus c. Forget time.
Time doesn't belong here.
So take those.
It's a lower concentration, so it's kind of like a lower energy.
So we'll take a lower wavelength here.
So r1 and c1 come over here.
c1, r1.
We'll do one more.
Three's a charm.
Come up to here.
This is t2.
Concentration has fallen to c2, and I've got a slope.
You see how the slope is getting less and less steep?
Because the rate loss says the concentration falls, the rate falls.
So this is slope r2 at c equals c2.
That's lower energy.
So I'll get the orange.
Or maybe this is fuchsia.
So we'll take this one.
And this will put these two together and bring them in like so.
And this is now c2, r2.
too And what's this thing say?
Take the slope.
Look, it's a log, log plot.
If you've got physical data and you don't get a straight line on a log, log plot, you're incompetent.
You're a complete idiot.
All right?
There it is.
And the slope here is n. And where it intersection, obviously I can't get a negative number, so just allow me to fix this just a wee bit.
So there's the axis, and where it hits the axis, this is now log k isn't it?
So I determine the order of reaction.
Good.
So now you know how to do it and you'll get some practice if you work the homework.
All right.
A couple more things to say and then we're done.
So last thing, we talked about increasing the rate of reaction by increasing the temperature and increasing the concentration.
There's one more thing we can do.
And what we can do is use a catalyst.
Increase rate by use of catalyst.
OK. How does the catalyst work?
How does the catalyst work?
Well, let's go back to this diagram here.
Where we've got, this is called the extent of reaction.
Some kind of reaction coordinate.
And over here we had some kind of energy coordinate.
We had this energy coordinate.
Remember, we had this thing?
This silly thing here?
So this is the reactions.
And this is the product.
And we reasoned that the activation energy ea is sitting here.
And we saw with the box, you had to get a box up onto its ear, otherwise it wouldn't fall over and so on.
And there's a finite amount, finite fraction of the Maxwell-Boltzmann distribution.
So if we plot energy and the number of particles in the distribution have of given energy, we get something that looks like this.
Tails off.
And we said that out here we have the fracture under this line.
This is the fraction that has the energy to jump over the activation barrier.
So fa is the-- f is the activated, is the fraction activated.
And those are the only ones that can participate in the reaction.
These other ones can't surmount the activation barrier.
Now, what the catalyst does is it lowers the activation barrier.
Catalyst lowers the activation barrier.
It doesn't change the n points.
The n points represent the energy state of the reactants and the energy state of the products.
What the catalyst does is it finds a low-energy barrier pathway.
A low-energy barrier pathway, which on this curve would mean that if this energy is lower, then the fraction of any distribution that has the requisite amount of energy is much higher.
Can you see the area?
The blue area is much, much greater than the white area.
So that means that this fa catalyzed-- I'm just going to write cat here-- fa catalyzed is much greater than fa in the absence of a catalyst.
And that promotes.
What's an example of it?
Well, in an automobile catalyst one of their reactions we're trying to promote is unburned hydrocarbons.
There's some unburned gasoline that comes out in the form of carbon monoxide.
What we want to do is convert the carbon monoxide to carbon dioxide.
And this occurs on the catalyst and I'll show you more of this at the end of the lecture.
But these are both gas molecules and they're zooming through space.
And if they're going to react with one another, they really need to somehow move almost on parallel lines so they get close enough to each other for a long enough period of time that they can start thinking about lower energy state.
And say, well, gee, if we break this bond and form a new bond, it's going to go to-- you've got to get to know each other, go have a few drinks, and da, da, da.
So it's not happening.
In the gas phase it doesn't happen.
So what we do instead is we provide a service.
This is the catalyst.
The catalyst is a solid surface and the CO sits down next to another CO.
They sit down and now they're in place long enough in the right orientation to then form CO2 and then desorb.
So this is how the catalyst assists.
We call this-- especially, imagine if it's the other way-- what if this carbon monoxide molecule is turn around so that the two carbons are side by side.
That's not going to promote the reaction at all because this thing's got to whip around and so on.
This is a.
Bad orientation And what's the Greek idea for orientation?
It's stereos.
That's what stereophonic sound is.
If I close my eyes, the two speakers give me spatial rendition.
So this concept here, where things are backwards and they're not the right place.
It's called steric hindrance.
This is steric hindrance.
One of the functions of a catalyst is to combat steric hindrance and put things in their most favorable orientation to facilitate reaction.
So what are characteristics of the catalyst?
The catalyst needs to have the following characteristics-- first of all, it has to have an affinity for the reactants.
In this case, the CO.
It has to have an affinity for carbon monoxide.
Affinity for reactants, so they'll be attracted to the catalyst.
But there's a second piece here.
If the catalyst doesn't get rid of the CO2, can you see that after a very short period of time, the surface of the catalyst will be covered with carbon dioxide and then the reaction starves.
It's choked.
So it has to have a disaffinity for the products.
Disaffinity for the products.
And in the real world, such as in the gas exhaust stream of a car, it's not just carbon monoxide.
There's all kinds of other stuff going on in there, and so the third thing that the catalyst needs to have is selectivity.
So it doesn't get distracted by other reactions.
And selectivity, most important when you have a mixture of all sorts of other species.
So that's what the catalyst does.
Now there's one more thing.
Just one second.
Perfect.
Gotcha just in time.
So one more thing here.
One more thing.
We've shown what happens when you catalyze.
So I'm going to put this.
This is ea under the influence of catalysis.
OK.
This is the e of the catalyst.
There are other substances that can retard the reaction.
These are called inhibitors.
And what's an inhibitor do?
It does the complimentary action.
So it raises the activation barrier.
So this is ea under the influence of inhibitor.
And what's the value of that?
Well, if you buy some antifreeze for use in the automobile cooling system.
If you read carefully the label, it's not just ethylene glycol, and water, 50/50 volume percent.
There are other additives.
What are those additives?
Some of them are called inhibitors.
Oh, I didn't spell this right.
No wonder you looked so puzzled.
I thought you didn't understand the concept.
Inhibitor.
So it inhibits the reaction.
What's going on?
You've got the iron or aluminum block of the engine, you got the water pump.
It's taking heat away from the engine, you're heating everything up, stirring it around.
It's a perfect environment for enhancing corrosion.
And we don't want the cooling system to act as the corrosion system for the engine.
And so we add components to the antifreeze too reduce the rate of corrosion, to lengthen the service lifetime of the vehicle.
And so what does the inhibitor do?
It raises the activation barrier for the corrosion reaction and thereby reduces its rate of reaction substantially.
So this is inhibited.
OK, so this is catalyzed, this is inhibited, and you can see that the fraction of active species over the activation barrier, under the influence of inhibition, is much less than the fraction otherwise.
There's a time and a place for all of these wonderful things.
So that's the story of kinetics.
That's the story of kinetics, but it's 11:35 and it's time to start a new topic.
And now, we can erase.
Thanks.
OK.
So let's talk about something else.
Let's talk about something else.
What I want to talk about today is another topic that comes from the general area of rate phenomena.
I want to talk about something else in rate phenomena.
We're going to talk about diffusion.
And that's from the general class of rate phenomena.
That's why it appears with chemical kinetics.
And what do I mean by rate phenomena?
Rate phenomena means d by dt is in the room.
All right?
We're looking at time, domain processes.
You know, chemistry, before you came into this class, you probably thought, I know what chemistry is.
Chemistry is chemicals reacting.
That's what people think.
Isn't that what the ancient Egyptians said?
khemia That's where the guys knew how to embalm the dead and so on.
It was about chemical reactions.
We've been talking about structure, properties, all sorts of things.
Well, now we're talking about rates of reaction.
If we want to run a reaction, there's two things that we need to move around to sustain a reaction.
So transport of two important quantities.
One is matter and the other is energy.
We star something for energy.
As we just saw, the thing doesn't go.
And if we starve it for matter it doesn't go.
So energy in motion is under the general rubric of heat transfer.
But we won't talk about heat transfer because we've only got 14 weeks.
I'd love to talk about it, but we're out of time.
So instead, we will talk about mass transport.
And mass transport in solids occurs by diffusion.
And we know what's critical.
For example, we know in the case of the Hindenburg, the Hindenburg didn't explode.
Why didn't the Hindenburg explode?
Because there were 7 million cubic feet of hydrogen, but we couldn't get 7 million cubic feet of oxygen to the site.
Because of the limitations of mass transport, 2/3 of the people walked off of that vessel.
It was on fire, and many of them died because they panicked and they jumped to their deaths.
Very few people were actually affected by the fire.
So it's a good example of mass transport, negating the rate of what otherwise could have been an explosive reaction.
So I'm going to look at a simple example today.
And what I want to do is look at the doping of semiconductors.
You know, we've studied semiconductors.
Dave, could we cut to the document camera, please?
So what I've got here is I'm going to show you a silicon wafer.
And the silicon wafer has got some features on it.
So it's already been doped.
Oh, that's pretty, isn't it?
So cute.
All right.
So what we've got here, remember this is the 8-inch diameter single crystal.
So this was cut from a single piece of silicon 8 inches in diameter, about 2 meters long.
How did they make a single crystal?
They immersed the tiny single crystal into the melt and gradually pulled the single crystal out.
Sort of the way you can grow rock mountain candy.
And if you're really careful, the entire giant, 2 meter length, 8 inches in diameter, is one single crystal.
And then we take this and we cut it with a diamond saw into wafers, Each of them about a millimeter in thickness.
And, of course, the width of the saw turns into dust So your yield [SNAPS] right off the bat is about 50%.
Because [SAW SOUND] it's all gone.
[SAW SOUND] it's all gone.
And now what we're going to do, is we want to dope it.
So how do I get the boron in here?
We say, why don't we add the boron to the melt, but what if I want to make pn-junctions?
So I want to get some boron and some phosphorous.
If I add the boron and the phosphorous, it then goes nuts.
I need to have p-type against n-type.
So how do I get it in here?
That's we're going to talk about.
We're going to talk about how we make those features.
See those features?
Because if you're really good, you start with those features and you cut them up and then you end up with this, which is this.
This is a Pentium chip.
The amount of material science, physical chemistry in that device is absolutely phenomenal.
What lies underneath this piece of gold is absolutely phenomenal.
It's what powers the information age.
And it all starts with this stuff here.
It's that simple.
So how do we get the boron in?
Dave, let's leave this up.
It's a pretty image.
It might soothe the natives, so let's just leave it up.
By the way, that spider network is all the gold wires that are addressing.
It doesn't do you any good to have a gazillion features there but you can't get information in and out.
So if you've got a gazillion features, you need a gazillion leads to take information in and out.
That's very, very thin gold.
How do you do that?
Not with a paint brush.
It's all with this stuff.
That's why I teach you this instead some of the other stuff.
OK.
So now we're going to dope.
We're going to dope the silicon with boron.
Number one point to remember-- we're going to dope the silicon in the solid state.
We do not add boron to molten silicon.
We're going to add boron to the silicon in the solid state.
So I'm going to draw this wafer.
Only it's not to scale.
It's going to look more like a hockey puck than a thin wafer.
Anyways, imagine it's this.
OK?
So this is the silicon wafer, the silicon wafer on edge.
And this is where all the features were that I showed you.
So we're going to do is we're going to try to introduce boron.
We want to get boron into the silicon, we're going to bring it in from the surface.
So this is a silicon wafer, which means it's a single crystal.
It's a single crystal.
And we could use Laue methods to figure out which is the face.
Is that cut on the 0 1 1?
Is it cut on a 1 1 1?
And so on.
So now we're going to dope with boron.
How do we do it?
We decompose diborane.
We introduce diborane, it's a gas.
B2H6.
And we flow diborane over the surface at some constant concentration.
And what happens is that the diborane decomposes.
B2H6 on silicon.
And I'm going to write silicon with an x.
What's the x mean?
Crystalline.
Single crystal.
Make sure no one is thinking this is being performed in the liquid phase.
And this diborane decomposes to elemental boron plus hydrogen gas.
So hydrogen is a gas, and the boron goes into the silicon.
Boron goes into the silicon where it goes on to the silicon sites.
I could just write a rate constant for this.
We just studied kinetics.
I don't know.
What is it?
Second order?
You know, it doesn't have to be integral order, by the way.
It could be of order 1.5.
Nothing is saying it has to be integral order.
OK.
So now what I want to do is I want to ask, what happens as the boron goes in?
So that's the next question.
As it diffuses in.
So what I'm going to do is I'm going to draw this little cartoon here.
So I want to look at here's the surface of the silicon.
So I've got a gas phase here and I've got boron going into the silicon crystal.
So I'm going to now get a little bit quantitative, if you'll permit.
And so it's going to look like this.
And put it right underneath here.
So this is x. this is depth from the surface of the silicon.
So this is the free surface of the silicon.
And I start off with some concentration surface value, c sub s. Meaning concentration at the surface.
And that's fixed.
So cs fixed by B2H6 concentration in the gas phase.
So that's fixed.
Now, question is, what does it look like?
The advance of the boron?
So this is now the concentration of boron in silicon.
This is the silicon crystal here.
So what does it look like?
Well, one possibility is it goes in like this as a front.
See if I wait, goes in farther and farther and farther.
That doesn't happen.
Sorry.
Take that off.
You know what happens?
Same thing we've been looking at all day.
So now what's the question?
Question is, what's the functionality of this?
Is this 1 over c?
Is this log c?
Is this some other function of c?
I want to map this into some f of c versus g of x so I can get a straight line.
Why do I want a straight line?
Because I want to be quantitative.
Because you are in charge of designing a fab line and they want to know what the resonance time of a silicon wafer is supposed to be at billion dollars multiples.
You know, the units are in billions of dollars.
So you don't want to make the fab line any bigger than it needs to be, the resident's time any longer.
This is only way you're going to be able to design it.
You have to understand this because the design specification is going to be dope to a certain concentration of a certain depth.
So how long does it take to get boron in?
You can tell me.
Set the concentration here and wait 5 minutes, 10 minutes, 30 minutes.
Two companies.
One takes 30 minutes to process the wafer, the other takes 10 minutes to process the wafer.
Guess which company is going to be in business two years from now?
It's not going to be the one that takes a long time.
Hint.
So we need to know what the rules are here.
And the mathematical formulation was set by Adolf Fick.
Adolf Fick, 1855, answered the question, what is the form of that line?
He's an interesting guy.
He was at the University of Zurich.
And it's sort of like the Balmer story.
Remember J.J. Balmer was the school teacher in Switzerland.
This guy was in Switzerland, too.
Something going on there in Switzerland in the late eighteen hundreds.
So remember J.J. Balmer?
He fit not his own data, he fit the data of Angstrom.
Angstrom made the measurements, Balmer fit the data.
Well, same thing here.
Fick did not make the measurements.
He looked at some data that had been taken back in 1833 by Thomas Graham.
Thomas Graham had been making measurements back in the 1830s of gases diffusing through porous membranes.
Or just say, through membranes.
Obviously, if they weren't porous they wouldn't be diffusing.
Redundancy.
Department of Redundancy.
This is diffusing through membranes.
So what happened was that he modeled it.
And here's what he came up with.
He came up with-- oh, by the way.
I didn't mention what he was.
He was a physiologist.
He was a medical doctor.
He just poured through the scientific literature for amusement, evidently.
OK.
So this is what Fick gave us as the model for the data.
He gave us what we know as Fick's First Law, Fick's First Law of Diffusion.
And we'll just call it FFL, Fick's First Law of Diffusion.
And we'll write Ficks' First Law J. J, which is the flux.
This is the unit of measure of mass transport, the flux, which obviously is the mass flow rate.
We said this is mass transport, so this is the mass flow right of species i, OK?
And we're going to say in the x-direction because things diffuse in all directions, right?
If I open a perfume bottle here, the diffusion will take place in all directions.
So I'm going to make this one dimension.
So the flux is a mass flow rate of i in the x-direction.
J sub i in the x-direction.
Remember we said the Chemical Rate Law?
We said the Chemical Rate Law was that the rate of progress of a chemical reaction goes as the concentration.
Well, Fick studied Graham's data and he said, the flux doesn't go as the concentration.
It goes as the gradient in the concentration.
That's how you linearize those data.
So you take the concentration gradient, which is dc by dx.
OK.
So that's the concentration gradient.
Gradient in concentration of i in the x-direction.
You could write this as-- make this vector-- but we're not going to do that.
It would make it one-dimensional here.
And the constant of proportionality is d sub i, which is called the diffusivity or the diffusion coefficient.
And puts the minus sign.
Why?
Well, you know things move from high concentration to low concentration, right?
So I open the perfume bottle, the odor gets-- the odor, pardon me-- the fragrance gets stronger as you get closer to the bottle.
Things move from high concentration the low concentration.
So if I gave you this concentration profile-- what's profile mean?
Profile means-- this is a concentration profile-- if I draw someone like this, this is head on, right?
And if I do the right view like this, this is the profile.
Profile.
Side view.
This is profile.
Concentration profile, obviously, the concentration is higher to the left.
So things are moving from left to right.
But look at the slope.
The slope is a negative number.
So if the slope is a negative number, but I want to get a positive flux, I need to have the minus sign in here.
Diffusivity is a physical constant.
It has to be a positive number.
So this is Fick's First Law.
I'll just give you the units and then we'll stop.
So what are the units?
Well, the units in flux are in SI units kilograms per second.
And we're going in one direction, and so as we're trying to diffuse into this piece of silicon, we have to normalize it per unit area.
So it's kilograms per meter squared per second.
You know that concentration is kilograms per cubic meter.
d by dx is 1 over meter, which means then that the diffuse coefficient or the diffusivity has units of length squared per unit time.
This is the units of the diffusion coefficient.
And we'll get back to business next day.
David, may we cut to the slides again, please?
OK.
So I talked to you about-- oh, yeah.
There's all this.
You see they give you a half-life?
Crazy.
Forget it.
OK.
There's the book talking about activated catalog, da, da, da.
This is Fick's First Law.
This is the paper from 1855.
Go ahead.
"On the Influence of Diffusion" by Dr. Adolph Fick.
So the number one catalyst in the world, number one application is automotive catalysts for exhaust.
So gasoline is octane.
C8H18, which we burn in air.
We burn it in air, we make hydrocarbons.
Pardon me, we burn hydrocarbons, which obviously means the carbon goes to CO2, ideally, and the hydrogen goes to H2O.
But sometimes you get incomplete combustion.
I already showed you that.
Maybe we stop it CO.
But remember, air is 4/5 nitrogen.
So when you burn hydrocarbons you also, at these high temperatures, mix nitrogen and oxygen and make NOx.
And is was actually the precursor to smog.
So what we want to do is to eliminate these imperfect combustions.
So we want to convert the unburned hydrocarbons.
There's actually some of the CH compounds, volatile, then don't get burned.
We want to burn those to H2O in water-- pardon me, H2O in CO2.
The unburned CO has to go to CO2.
You might say, but isn't CO2 a greenhouse gas?
Yeah, it is.
But you know, given the choice of carbon monoxide and carbon dioxide, my advice is go with carbon dioxide.
Carbon monoxide will kill you at about 200 parts per million.
And then NOx, we want to convert that to carbon dioxide.
So GM alone tried over 1,500 catalysts.
Why GM?
Why not the auto industry?
Because the U.S. Government, in its wisdom back in the 1970s, invoked the Sherman Antitrust Act and forbid the automobile industry to collaborate in the search for a catalyst.
And since they didn't understand the science of catalysis it was trial and error.
By inspection.
And it's a huge space, trying things.
But they eventually found, after 1,500 tests, platinum-palladium works for hydrocarbons and CO, and rhodium works for NOx.
It's almost as though mother nature said, I'll give you a catalyst and it's going to be the most expensive part of the Periodic Table.
You buy this stuff at the jewelry store, you see.
Furthermore, this is an oxidation reaction.
This is an oxidation reaction.
This is a reduction reaction.
So if I told you I want you in the same place at the same time on a vehicle moving 80 miles an hour to conduct simultaneous oxidation and reduction reactions with no intervention, you'd say impossible.
They did it.
They did it.
This is the power of selectivity of the catalyst.
So this is an old diagram from GM literature.
Now we have fuel injectors, here, but fuel goes into the engine, out to the exhaust.
There's an oxygen sensor, which I'll show you next day.
Oxygen sensor measures what composition of the output gas is, sends it to an electronic control module, which then goes back into the carburetor.
And the catalytic convert is right here.
And Dave, if you show the document camera.
This is the monolith.
This is a piece of ceramic because platinum and rhodium and all of these other elements are so expensive.
See, this thing's having a hard time.
So we'll cheat a little bit.
We'll put this here and then it'll focus.
All right.
So what you're looking at, these are long channels.
And just to give you a scale, this is about a half of a millimeter.
This is about a half a millimeter square and they go all the way down for a length of about eight inches.
And then the walls, the interior walls, are coated with a very, very thin layer of platinum, palladium, and rhodium.
Because these are expensive metals.
And it's a surface effect.
So you get no value if you have an inch deep of platinum.
The only stuff that works is the free surface.
What you'd really like is a monolayer.
But a monolayer of platinum has no mechanical strength.
So you put the platinum, palladium, and rhodium down on a catalyst support.
and this stuff is made of a ceramic, it's magnesia, alumina silica that's been extruded like pasta, and then fired.
The technology is absolutely phenomenal.
Goes into this.
And then to get the platinum-palladium-rhodium they have a very thin acid wash. chloroplatinic, chlororhodic acid, wash, little bit of surface tension, holds just a little bit of water, fire, out comes the metal.
Just thick enough.
If it's too thick, dilute the concentration of the acid.
But use that precious metal sparingly.
Platinum hit $2,000 an ounce last year.
$2,000 an ounce.
You steal a car, throw the car away, and get the catalytic converter.
So anyway, that's underneath.
That's what's going on underneath the car.
And actually, I'm holding for you the elements of clean air, the catalytic converter.
But the catalytic converter can't work unless we control the air to fuel ratio, which we have to do with an oxygen sensor.
I'll show you next day.
And you've got to feed it into a CPU.
If you don't have a CPU, it can't make rapid feedback to keep adjusting the air-to-fuel ratio.
All right, Dave.
Let's cut to slides and we'll get out of here.
So I'll show you what else is in here.
OK, there's that.
So there's the old thing.
There's the engine, da, da, da.
Exhaust, the control module.
And look, in the old days they actually had-- they look like little ball bearings-- cost of these things was phenomenal.
So last thing, we had to go to unleaded gas.
Unleaded gas was brought in because the lead would be broken down into elemental lead because lead is volatile as an oxide.
The lead alloys with the platinum.
Turns it into a lead platinum alloy.
Lead has no catalytic value.
Because if it did, we would use lead and not platinum.
OK. We'll see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. OK, settle down.
Settle down.
Let's get started.
So last day we started talking about diffusion.
Diffusion is a solid-state mass transport by random atomic motion.
I made the first point last day and today I'll make the second point.
We'll see why it's random atomic motion.
We reasoned that we could describe the ingress of material.
Here I'm showing a concentration profile.
A profile is nothing more than a plot of something as a function of position.
So, this is looking at, for example, the doping of a wafer.
This is the free surface at x equals 0.
As we go to the right, it represents depth.
And we fix the concentration at the surface and the material tails off to zero deep inside the specimen.
Fick in 1855 annunciated the law of diffusion, shown here, that says that the rate of ingress expressed by the flux is proportional to the instant gradient in concentration.
And the constant proportionality is the diffusion coefficient.
This is where all the material science lies.
It's inside the D. That's what we're going to talk about more today.
And so we see things tail off and what we want to do is figure out what the mathematics are that'll allow us to render this thing into a functional representation.
Because all we know right now is that we've got something tailing off.
Ok?
And so this is the gradient multiplied by the diffusion coefficient and you can see that the red line represents the flux.
The gradient is maximum at the surface, so the flux is maximum at the surface.
And as we move farther and farther into the piece, the gradient in concentration becomes less steep, which means the absolute magnitude of the flux becomes less steep, and we go deep inside where there is no doping yet.
The variation of concentration is constantly 0, so the slope is zero, which means the flux is zero.
So the two sort of track, one and the other.
And we recognize these are negative slopes since the minus sign is here.
Even though we have a negative slope, we have a positive ingress.
And then the other thing that we realized was that the diffusion coefficient has a temperature dependence.
And the temperature dependence looks like this.
It's an exponential of some kind of a characteristic energy divided by the product of -- this is the gas constant, this is not the Rydberg constant.
The gas constant is simply the Boltzmann constant per mole.
So if you take the Boltzmann constant, multiply it by the Avogadro number, you get this.
But if you see something times T, you know it's got to be either gas constant or Boltzmann constant.
This two-letter character string has to have something akin to a Boltzmann constant, otherwise you don't end up with an energy term.
And this in the top is a barrier energy.
It's a barrier energy to diffusion and out here we have the pre-exponential.
Unfortunately we didn't give it the letter F in honor of Fick, it's just D naught.
And so there it is.
So if we take this equation and we plot it, natural log of d versus 1 over T, that'll linearize the equation, and we end up with something that gives us a straight line.
And the slope is minus Q over R, where this Q is some kind of a barrier energy.
And the R is the gas constant or the Boltzmann constant.
So this is the gas constant.
It's on your table of constants.
It's got a value in SI units of 8.314 joules per mole Kelvin, OK?
Just the Boltzmann times Avogadro number.
So now I want to invite you to join me in some pattern recognition.
If I give you the temperature dependence of a quantity that looks like this , in other words, the logarithm of that physical parameter versus 1 over T, gives you a straight line.
You know from the work that we did on chemical kinetics, this is the representation of something that is an activated process.
Remember the box that would fall on its side, and it had to go up on its corner?
So we had some physical sense of what activation is.
Now this has a log, something versus 1 over T dependence.
This represents some kind of an activation energy.
An activation energy.
But what's the activated process that's going on here?
So that's where I want to take you into the atomistics.
So here's a cartoon taken from one of the readings.
And it shows a set of atoms here and the atom next to the vacancy wants to move into the vacancy.
And you can see the artist's rendition is showing that in order to get from the current position into the vacant position, it's got to squeeze through this narrow channel.
And it takes energy to move through that channel.
And there's an activation barrier associated with that motion.
Can you see when the atom is sitting right at the saddle point?
The system is highly activated because it's pushing out against those atoms.
And then finally it falls into the new slot and the energy drops.
So the activation energy is associated with moving through this saddle point.
So what really happens here, I'll tell you right now.
First of all, we have to recognize that the system is depicted there, because it's a freeze frame, but we know that above 0 Kelvin, everything is in motion.
So the only way we can rationalize what's going on is to, first of all, recognize that we have a pulsating lattice.
We have a pulsating lattice.
Everything's vibrating.
And what's the heartbeat?
What's the idle speed?
I've told you this before.
Everything that's going on in this room has an idle speed, a heartbeat, of 10 trillion times a second.
That's the Debye frequency, OK?
The idle speed or the heartbeat.
The heartbeat is called the Debye frequency.
Here's frequency, lowercase nu, in honor of Peter Debye.
And it's on the order of about 10 to the thirteenth per second.
10 to the thirteenth Hertz.
That's 10 trillion.
So 10 trillion times a second atoms are going like this.
Pulsating.
All right?
So what has to happen for that operation to occur?
Well, I've got an atom here, vacancy next door, but because it's close packed the atoms on either side close in.
There's no way this can squeeze through.
But, imagine, 10 trillion times a second.
If I'm lucky enough to see the moment when the atom above pulsates up, the atom below pulsates down, a channel can open up and then this thing can squirt through.
Now, that doesn't happen every time.
If it did, history would have ended.
At 10 trillion times a second, it's all over for everybody.
Right?
So what happens?
What's the frequency at which we will get this unique combination?
It turns out that frequency, noted gamma, is called the jump frequency.
That's the jump frequency.
And the jump frequency is on the order of about 10 to the eighth Hertz.
10 to the eighth Hertz, which is about 100 million.
So if you take the ratio of the Debye frequency to the jump frequency, you see you get a value of about one try in about 10 to the fifth is successful.
One try in about 10 to the fifth is successful.
And that explains what's going on here.
So, what now does that mean in terms of Q?
What's the breakdown of Q?
Well, clearly, Q, the activation energy for diffusion, must involve both the energy to form the vacancy -- I'm going to call it delta Hv, and that's the energy for vacancy formation -- if you don't have vacancies, you cannot have motion -- plus the delta H sub m which is the energy for atom migration.
So you have to form the vacancies and then you've got to squeeze through those saddle points.
So that's how we get a sense of it.
And I think I've got another slide.
OK, so they're I've simply put the markings on for you so you can see the atoms above and below jumping in the right direction.
That allows the atom on the left to squirt to the right.
Now, this is also related to energetics.
It's related to energetics because the energy to form a vacancy and the energy to squeeze through those saddle points must be related to the binding energy.
And what else is related to binding energy?
Melting point.
So, this is an elegant plot that simply shows that the energy -- these are all FCC metals: lead, aluminum, silver, gold, copper, and iron.
They're all FCC on this chart here.
Iron has both BCC and FCC, but this is comparing apples to apples.
And you can see that there's lead melts at 327, aluminum at 660, silver 980, all the way up.
Iron 1535.
And you can see that the activation energy for -- that's this Q-value -- the activation energy for diffusion scales with melting point.
Again, an indication that this analysis makes sense.
And then, here's the other one, the interstitial.
If you have interstitial diffusion you've got plenty of interstitials, so you don't have to pay the penalty to create the interstitial, but it's much, much more difficult to squeeze through the saddle point.
So, it turns out that activation energy for diffusion by interstitial means is still fairly highly activated.
So this is in the case of substitutional systems, all right?
This is for substitutional.
Substitutional atoms.
What do I mean by that?
An atom that sits on a lattice site.
If an atom sits on a lattice site, it has to jump to another lattice site, which means there needs to be a vacant lattice site.
On the other hand, if we're talking about interstitial special atoms -- not substitutional atoms, interstitial atoms -- the assumption is that there's plenty of interstitial sites.
There are very few systems that I can think of where the interstitial sites are so heavily occupied that this assumption is invalid.
So in that case, you don't have to pay for vacancy formation.
It's simply equal to the enthalpy, which we know is analogous to the energy for a condensed system of atom migration.
Atom migration gives you the whole story and it goes back to the same picture, and so on.
Now, I said it's random walks, so let's look at another one.
So, this is an interesting example.
This is self-diffusion in cobalt.
Cobalt has a number of isotopes.
One of them is 59 and one of them is 60.
60 is a radioisotope.
Cobalt 60 is a radioisotope and it's used in a variety of technical endeavors.
And so what we've got here is a sandwich of cobalt 59 -- cobalt 60 and cobalt 59.
So even though cobalt 60 has radioactivity and it's got a different number of neutrons in the nucleus, chemically it is identical to cobalt 59, just as carbon 14 is chemically identical to carbon 12.
So there's no concentration gradient.
Fick's Law says it's supposed to go by concentration gradient.
The concentration gradient is zero across that piece.
And yet if you wait for long enough, you will start to see -- the dark atoms here are meant to represent cobalt 60 in the sandwich -- after some period of time, the cobalt 60's will move outward.
And if you wait long enough, the cobalt 60 concentration will be uniform throughout the specimen.
So what's going on?
There's no chemical concentration gradient, and yet over time, the cobalt 60 spreads out.
And the answer is this is happening by random walk.
We've got the pulsating lattice.
Pulsating lattice.
10 trillion times a second those atoms are vibrating and there's going to be some vacancy.
All of a sudden an atom falls into a vacancy.
And the mathematics of this -- look at this, this is no accident -- this curve here represents something that looks like a Gaussian distribution.
At any given time, the amount of cobalt 60 that's veered from the center band can be portrayed by the Gaussian distribution, the bell curve.
And what's the physical model for the bell curve?
It's the drunken sailor.
Or I guess I have to be PC.
I'm not going to just pick on sailors.
Maybe it's the drunken cougar, the soccer mom that went out drinking.
OK, so what happens with the drunken cougar, she comes to the door, and she starts to walk, all right?
And then she walks one way or the other way, until she finally falls.
And if you go through the number of steps that it'll take before the drunken sailor falls, you get this distribution.
It's an old physics problem.
The drunken sailor problem.
And it's a normal distribution.
And that normal distribution tells you that everything's going about by random jumping.
Random jumping, according to this, all right?
So that's why we say that diffusion is a random walk problem.
Before I turn this off, I want you to take a look carefully.
What's wrong with this textbook cartoon?
Any ideas?
It's a substitutional system, right?
It's cobalt and cobalt.
This is absolutely impossible.
What's wrong with it?
There are no vacancies.
There are no vacancies.
It's impossible.
There's no vacancies to jump into.
This thing is jam-packed, so it's physically unrealistic from a thermodynamic standpoint.
There has to be some finite population of vacancies.
Given this system, there's no way that the black atom could jump into the neighboring site, because you'd have to have the entire system open up, and the chances of that happening are vanishingly small.
So it just shows you, just because something costs 150 bucks and is pressed between two hard covers doesn't mean it's full of truth, OK?
Be careful here when you look at this.
So, again, the artist needed to be guided.
We need to put a few of these things out, just to give symbolic recognition to the fact that we need to have some vacancies.
OK. All right, let's keep going.
Let's keep going.
Here is some data.
Here is some data.
Various data.
This is diffusion.
This is the logarithm of the diffusion coefficient versus 1 over T. So let's look at the top.
This is hydrogen in iron.
So temperature increases from right to left.
So, go up, up, up, up, up.
And roughly 900 degrees Centigrade, iron changes from BCC to FCC.
And you know from our previous unit, the number of nearest neighbors in FCC is greater, so there's less void fraction, right?
So that means that it's much more difficult to move.
And you can see, first of all, the order of magnitude drop in the diffusion coefficient, and also, it's subtle, but the slope is steeper, because the activation energy for diffusion is steeper in a closer-packed system.
This is carbon in iron.
Same thing.
Relatively gentle slope at the same temperature, there's a phase change to FCC and it takes much, much more energy per unit increase in temperature to get the comparable increase in the value of diffusion coefficient.
This is iron, self-diffusion in iron.
So this would be radio tracer iron in iron.
Very much lower.
You see these values are up around 10 to the minus 5, 10 to the minus 6.
This is down to 10 to the minus 11, 10 to the minus 12 centimeters squared per second.
Iron diffusion changes at the transition temperature.
Now iron going through FCC iron.
Here's the last one.
Look at this one.
This is carbon in graphite.
So how does carbon -- suppose I had some carbon 14 in graphite.
What's the bonding in graphite?
Silence.
It's covalent.
It's sp2 hybridized.
It's really strong bonds.
The activation energy -- you know, forget just jumping through that saddle point -- you've got to break those covalent bonds.
It's very, very high energy to get carbon to move through graphite.
And so you see very, very low values and very, very steep temperature dependence.
Now, there's another way to think about this.
Again, this is now looking at the atomistics.
So this is the same material trying to indicate that if you use Fick's Law, moving in from the surface into the bulk, we have a certain front.
So this is some isoconcentration line.
So we could take this value off of the graph arbitrarily.
So they're saying c greater than c i.
Because it trails off to zero, right?
It's asymptotic.
There's no sharp cut-off here.
It trails off.
So what they're doing in that graph is arbitrarily saying, let's call this ci.
And where's the front at concentration ci, given a constant value of cs at the surface?
And you can see that there's a near-constant rate of advance through the bulk.
But along the grain boundary you go much, much farther in the same amount of time.
Why?
Because the atoms in the grain boundary are not bounded the same way as the atoms in the bulk.
There's this gap, this misalignment.
And so, it's easier to make that jump, and you can see you get much, much more advancement along the grain boundary.
And this is a crack.
And this material covers the surface.
You get surface diffusion.
Surface diffusion is very important in many processes, and it's very fast, because on a surface, you've got no atoms on the top.
It's almost liquid-like, isn't it?
You've got atoms to the bottom, but you've got no atoms to the top, so you can move unconstrained.
Go back to this one.
Imagine here.
Imagine if you didn't have any atoms on the top, how easy it would be to make that jump.
All right, let's go and see what the data are.
Nothing like data.
And I got data for you.
Let's look.
Click, click, click.
All right.
Here's the data.
This is all for the diffusion of silver.
So I'm going to use this value, this schematic, and I'm going to give you the value for silver diffusion in the bulk, silver diffusion in the grain boundary, and silver diffusion along the surface.
So these are the data.
So this is log diffusion coefficient versus 1 over T. Only, I hate this graph, because somebody got really wimpy.
You know, it bothers them.
You see, when you plot something like this, you end up with, if 1 over T increases from left to right, this really means that high temperature is over here, right?
1 over T increases this way, which means temperature is increasing this way.
So somebody, some weenie, got all nervous here and put the numbers backwards, you see?
That way, the high temperature is over here, but it looks stupid, because when I see an activation plot, I want a negative slope.
It's non-physical to have a positive slope.
So I condemned this and I turned it around.
So now let's look at it the right way.
So this is log d versus 1 over T, and this is the bulk, lattice or bulk.
This is grain boundary and this is surface diffusion, all in solid silver.
OK, look at this.
First of all -- I'm going to jump from here -- one, two, three, four.
Four orders of magnitude faster diffusion along a grain boundary, which makes sense, because it's less constrained.
And now let's go from grain boundary at constant temperature up to surface.
One, two, three.
Three orders of magnitude along the surface.
So you can see the effects.
And there's the melting point of silver, right here.
So at the melting point of silver, the diffusion coefficient is about 10 to the minus 8 centimeters squared per second, which is the diffusion coefficient of virtually every metal at its melting point.
So I happen to know that number.
10 to the minus 8 centimeters squared per second.
Not that I know it as a function of temperature, but I know it can't be any faster than that.
So I said, this is near liquid-like behavior, what's the next thing to look for?
Let's get some data from real liquids and see if this hunch is correct.
What's the number here?
10 to the minus 5 to 10 to the minus 4 centimeters squared per second.
This is diffusion in molten ferrous alloys.
This is manganese in molten iron.
10 to the minus 5, 10 to the minus 4.
Bingo.
It really does behave like a liquid, in terms of diffusion.
Surface diffusion is very fast, very fast.
And it's all explained by this simple model of atom jumping.
And I continued it.
Here's data from glasses.
We studied glasses.
So here's a suite of different glasses.
Log k. This is permeability, which is related to diffusion coefficient.
There's a gas constant in there, but forget about it.
So this is essentially log of the diffusion coefficient versus 1 over T. We've got a family of lines.
And what's the difference here?
This is pure fused silica up here.
SiO2.
There's borosilicate, there's soda lime, and there's lead borate.
So what's happening?
We're adding more and more modifier as we go from top to bottom.
And as we add modifier, we break the length of the silicate network, which means it packs tighter and tighter, and as you get tighter and tighter packing, the diffusion coefficient gets lower and lower.
So train your eye.
Let's pick 200 degrees Centigrade, right here.
So at constant temperature, the more modifier you put in, the lower the diffusion coefficient because things are more tightly packed.
It's like walking down the infinite corridor in the middle of the hour, versus walking down on the hour.
On the hour, there's too many people.
It's tighter packing.
And if you're the least bit civilized, you're going to have to move a little bit slower, whereas, if you walk down the infinite at around now, no problem.
It's liquid-like behavior.
And now this is the isotherm at 300 hundred degrees C. This is from some old Corning literature.
Corning used to make glass, at one time.
And this is the logarithm of diffusivity.
And this is now the set of silica B203 and P205, and these are all network formers, right?
They all form covalent bonds.
So the more silicate, borate, phosphate you put in, the more stretched out becomes the network.
And sure enough, look up here.
See they didn't have names for their glasses like the Malibu or the Corvette or something.
They called their glasses by four-digit numbers.
So someone would say, oh, this customer needs, oh, I'd sell them a 7040.
All right?
Because it has a certain mix of properties.
So forget about what those numbers are, just know that this axis indicates that as you move up, up, up, up, up, the higher diffusivity is associated with a much more open structure.
So, again, a link between atomistic behavior and atomistic structure.
It's all there.
OK.
So let's leave that up there while we go a little bit forward.
OK, so now, so we've been down buried at the atomistic level.
Now let's jump up to the continuum level.
We'll go back to Fick's first law.
And I want to do something a little more quantitative.
So I want to look at diffusion across a permeable membrane.
So this is gas through a membrane.
Gas through a membrane.
And the membrane is shown here.
This is the walls of the membrane.
Membrane has thickness L. Membrane of thickness L. In fact, we'll give it a name.
Let's call it membrane.
All right, so here's the membrane of thickness L, and I'm going to put gas on the left at P1, and it's a constant P1.
So we keep -- we've got a ballast here, so even though something starts diffusing, the amount that we lose doesn't affect the constant pressure over here.
And on the right side, I'm going to put a second pressure, P2.
And just for argument's sake, I'm going to say that P2 is less than P1.
So that means we're going to end up with mass transport from left to right.
And so, I want to ask, what's this going to look like if we use Fick's Law?
And so beneath I'll make a plot, a concentration profile, from x equals 0 to x equals L. And I can morph this into concentration.
We know the gas law is PV equals nRT.
This is the gas law.
P is pressure.
P is pressure.
V you know is volume.
But I'm going to write all these out, because some of these letters are used so many times that we don't know, is this the Rydberg constant?
No, it's the gas constant that you've seen on the first board.
It's the Boltzmann constant times the Avogadro number.
T is absolute temperature, temperature in Kelvins.
And n is not the quantum number, it's mole number.
Mole number.
PV equals nRT.
And we know that concentration of i is mole number of i over V. Both of those appear in here.
So if you take what's underneath there, and I'll expose it in a second, you'll convince yourself that Ci equals Pi divided by RT.
And that's how you get that permeability thing to turn around as well.
OK, so, I started with gas pressure, but I'm now going to write this in terms of concentration.
So I'm going to morph P1 into C1 and over here, P2 into C2.
And I want to plot the profile.
So what's the profile look like?
I'm going to plot the profile at steady state.
At steady state.
At steady state, after I've waited a while, I have a straight-line profile across here, at steady state.
It's pinned at the two ends.
It's pinned at the two ends and varies continuously across here.
And you can see from Fick's Law, that if you have a situation in which the gradient is invariant across the piece, the gradient is -- dc by dx is the same here, as it is here, as it is here, as it is here.
dc by dx is invariant, right?
It doesn't change.
This is a straight line.
Well, if dc by dx is invariant, and d is a constant, then J is invariant, which just means no sources are sinks.
All of the material that goes in, must come out.
This means that J is not a function of time.
J is invariant.
J is not a function of time.
I come back here ten minutes later, I have the same profile.
That's what steady state means.
That's what study state means.
OK, and then, of course, if I wanted to be a little bit pedantic, I could say, in certain systems, the diffusion coefficient is a function of concentration.
So, when the diffusion coefficient is a function of concentration, then even at steady state, you might have some bowing of the profile, but that's a little bit more advanced.
OK.
So, I can describe everything that's going on here.
If I wanted to know how much material leaves from inside over a certain period of time, then I can simply say that total material lost is given by Fick's first law when you have steady state.
It's simply going to equal the product of the flux, which is mass per unit area per unit time times the area times the time.
And that's sort of analogous to Faraday's Law, right?
If I wanted to look at how much electrical charge I get for a certain period of time with current passing, I could say that the total charge is the product of the current times the time.
You know this relationship.
Well, can you see that J times A is analogous to the current?
It's the mass current.
It's mass flow rate.
Mass flow rate per unit area times area gives me something that's analogous to the current, and the time is the time, this is the total material lost, this is the total electrical charge.
These two are the comparable equations.
OK. Now, I want to go to the more sophisticated question, and that is, what happens before steady state?
Before steady state is achieved?
Suppose at time zero, the membrane has no gas in it.
And at time zero, I inflate the left side.
And just to keep things simple, I'm going to set c2 equals 0.
It's a linear equation.
You can always add it, but I just want to keep it simpler for the analysis.
So, suppose at t equals 0, we inflate to c1 on the left side of the empty membrane.
There's nothing in the membrane.
So, let's see what happens.
First, I'm going to solve the problem graphically.
Because we know what's going to happen.
I'm going to have to do the mathematics.
So, here's what we expect.
This is going from 0 to L. And we've got nothing in the membrane on both sides, so that's good.
And then, at time zero, I inflate to c1.
There's nothing in here.
So, what do I have?
Well, I know if I wait long enough, it's eventually going to look like this.
Because that's the steady state solution.
I've put here c2 equals 0, for simplicity, but we can change it.
It's a linear equation.
We can just add.
So, what happens at time 0 plus?
I start getting diffusion in from left to right, following Fick's Law.
So it's going to look like this, isn't it?
This is the approach to steady state.
Some books will call it nonsteady state or unsteady.
Unsteady to me means unsteady, so I refuse to use those terms.
I call this the transient.
That's the literary term.
The transient.
We're talking about the approach to steady state.
And in some systems, you never get to steady state.
But that's OK.
So here's what it's going to look like at some very, very short time afterwards.
It's going to look like that, only somewhat modified for this geometry.
So it's going to look like this, isn't it?
It's pinned at c1, and it's pinned at 0, and it varies continuously.
So this is at time t 1.
What happens at time t 2?
At time t 2, it advances a little bit farther.
But in all instances, the slope and the shape is given by this, Fick's first law.
So this is at time t 2.
And let's do one more.
Three's a charm.
So, let's take a green one.
All right, so here's at t 3, still pinned at c 1 and still pinned at 0.
So this is at t3.
And then finally, this is steady state.
So if we park at any place -- let's park at x1.
Watch here.
Can you see that the slope at t1 is gentle?
The slope at t2 is steeper than t1.
The slope at t3 is steeper than t2, and this is the highest.
So, we go from nothing, to gradual, gradual, gradual, until we finally reach steady state, and we stop.
And then at that point, we just continue to have to mass flowing through.
And look, the concentration is zero, but the flux is not zero, because the flux isn't related to the concentration.
It's related to the concentration gradient.
Zero concentration, but non-zero concentration gradient.
In fact, just for grins and chuckles, I could plot the ascent of this.
I could plot time at x equals x1.
I could plot J.
And what's J?
At time zero, it's zero.
Then it's small, then it's larger, larger, larger asymptotically.
And this is J of steady state.
And this is t1, t2, t3.
That's the approach to steady state.
And you might say, well, this is kind of a professor's pedantry.
It's not!
This is doping of semiconductors.
You don't run processes at equilibrium.
You run them far from equilibrium.
So in the transient mode, doping, outgassing, drying, what have you.
So, now I want to give you the shape of the concentration profile.
So how do I do that?
OK, so the shape of the transient profile is not given by Fick's Law.
The Fick's Law just gives me the gradient.
In other words, here's what I want.
I want C as a function of x at any given time.
That's what I want.
I want to be able to plot the profile.
So, profile is C of x, but C of x varies with time.
So, I need C of x at different times.
What gives me that?
What gives me that is Fick's second law.
Go to FSL, Fick's second law.
Now, Fick's second law, I'm just going to put it up here.
I'm going to show you the solution.
I don't expect you to solve it.
I would, you were all required to have differential equations as a pre-req, but you don't.
So, I'm just going to put it up here so that you see it.
It's a partial differential equation, and it looks like this.
And it's really beautiful.
It's got a beautiful symmetry and the fonts look great, so that's why we put it up here.
So, this is the partial in time goes as the double derivative in space, because we need a two-variable function, right?
We want x and t. And this assumes that the diffusion coefficient is not a function of concentration.
If it is, the equation is messier.
This assumes a constant diffusion coefficient.
So, this, as you can see, is a partial differential equation, which is a bummer, because it's hard to solve, all right?
But, it's linear.
It's a linear partial differential equation, which means the solutions are additive.
We know this already from LCAOMO.
So, I can handle, because I can solve it once and then I can give you ways to patch it, to make it useful.
So, the solution.
First of all, how do I specify this fully?
Every time you differentiate -- I'm going to give you some math here, real math that you can use.
When you differentiate, you throw away information, right?
The derivative of a constant is zero.
So, what was the value of the constant?
It's gone.
You lost it.
So you have to bring it back.
So when I double derivative this, I need two pieces of spacial information.
And when I take a derivative with respect to time, I need one piece of temporal information.
That's math.
All the other lemmas and postulates, forget them.
This is how you use it, all right?
So, I need two pieces of spatial information and one piece of temporal information.
So these specify our boundary conditions, right?
So, what are our boundary conditions.
So, I will need the concentration of all x at time 0.
That's this.
The answer to this question.
And what we're going to do is say that it's a constant.
I don't know what the constant is, it could be zero, it could be any arbitrary number, but I'm going to give you the solution for the situation in which the initial concentration is a constant.
So, you could look at this and say, well, what happens if it's a non-zero constant?
You ready for linearity?
Watch this.
This is how cool linearity is.
You see this set of solutions?
This is the set of solutions for C equals 0.
What if C equals some non-zero value?
I just shift the origin down to here to the non-zero value and everything sits there.
That's what superposition gives you.
That whole set of solutions just gets jacked up by the value of C naught.
If C naught is 0, this slams down onto the x-axis.
That's real math.
You're the master.
The math is the slave.
Never let math enslave you.
I won't.
I refuse.
So that's how you make C equals C naught work.
And then you have to peg the concentration at the surface.
I can give you examples where that's not the situation, where you have a variable concentration at the surface.
The solution I'm going to give you won't work for those.
So, x equals 0 is the surface.
So, at x equals 0 at all time, I have some fixed value, which I'm going to just call C surface.
And I need a third boundary condition.
And the third boundary condition is a mathematical trick.
We're going to say that this only works at short times.
It's called the short time.
And at short times, I'm going to say that -- from the perspective of the diffusion experiment -- at very, very short times, this sample appears to be infinitely long.
Because the amount of material that goes in doesn't hit the other side.
So, at very, very short times, we can pretend that it's semi-infinite.
And so then we can write that the concentration -- this is the short time, which means infinite size, and you'll see that the two are related -- that the concentration at infinity, then, doesn't change from the initial value.
The concentration at infinity for all time must equal the initial value C naught.
And so if you put these three boundary conditions into that equation, you end up with this.
This is the thing.
You get C minus Cs over C naught minus Cs.
This is surface concentration.
This is initial concentration.
This is C. It varies by this amount.
Error function of x over 2 times the square root of Dt.
So, you tell me the time and I can tell you the relationship between concentration and position.
Now I've got to tell you what the error function is.
Here's the error function.
This is this problem before.
This is the solution I'm going to show you.
Here's Cs.
Here's C naught.
The most general case.
C naught could be zero.
And this curve here is given by this equation, OK?
And this is an example of doping, isn't it?
This is how doping would work.
And this is the case where C s is greater than C naught.
If we turn it around and we make the surface concentration less than the initial concentration, we'll get the complementary situation.
The complementary situation looks like this.
So in this case, C naught is up here, Cs is down here, and we end up with this.
So now matter is going out, because Cs is less than C naught.
So this is -- what do you want to call it -- this is effusion, this is some kind of a drying process, outgassing.
And this is doping.
And what's the shape of this curve?
The shape of this curve.
ERF.
That's what math is.
You find a mathematical function that templates for the curve.
And ERF, I'm going to show you, is related to random walk statistics.
So, it's physically validated.
It's the best fit.
I mean, heck, if you find anything that is pinned at two ends and has curvature, you'll get a reasonable fit, but what's the right fit?
The right fit is this.
The error function looks like this.
Here's the error function.
Error function.
I'm going to plot ERF of z as a function of z.
And it goes from 0 to 1.
It looks like this.
All right?
And here's an error function table.
Those are exact values.
And it turns out that ERF is pretty much linear up to about point 6.
Up to about point 6.
ERF of point 6 not to less than about 1% is point 6.
And then as you go farther and farther out, and eventually, ERF of infinity equals 1.
But you don't even need tables for 0 up to point 6.
And you can push it, go to point 6, 5, if you want.
Yeah, let's do it.
Let's push harder.
6, 5.
It's linear.
You don't need tables.
Right?
And so ERF of 0 equals 0.
Oh, let me give you the function.
You know how you can define sine?
Sine is that integral of something over 1 minus blah, blah, blah?
We've got a definition for this one, too.
It's cool.
It's cool.
Here's what the error function looks like.
ERF of z equals the integral from 0 to z of e to the minus u squared du.
You might say, why is he showing us all of this?
Because he wants to torment you, that's why.
No, it's because I'm trying to teach you something.
What's e to the minus u squared?
Actually, I think I can squeeze it in right there.
What's e to the minus u squared du?
This function here, that's the bell curve, isn't it?
e to the minus u squared.
That's this thing.
It's symmetric, right?
It's a minus u squared, so it doesn't matter if u is positive or negative.
And e to the 0 was 1, and then back and forth.
And so what this is doing, is it's integrating from zero out to some value.
That's all.
That's the random walk.
This is the drunken sailor.
How far does the drunken sailor go?
That's the shape of this curve, the integral.
And we'd like it to be so that if you integrate from zero to infinity, the area will be one.
It turns out that this area here is root pi over 2.
So, therefore, to normalize, we put 2 over root pi.
That's how you get the error function.
All right?
So, the integral from zero to zero is zero, and the integral from 0 to 1 is going to be, whatever it is, point 8, 4, et cetera, et cetera.
So, that's it.
That's it.
Now we have the functional relationship to describe any of these curves.
And you'll get some practice in doing that.
And it's a lot of fun.
Because now you can talk about how long it's going to take to dope something, and so on.
Outgassing.
And now I'm going to show you something really cool, and I'm not kidding you.
This has served me well.
I've been here 30 some odd years, and I've been in consulting situations where I can, in my head, figure out order of magnitude of what it's going to take to run a diffusion problem.
Because you've got, up here, C minus Cs over C naught minus Cs equals ERF error function of x over 2 roots of Dt.
So, this runs from 0 to 1.
The right side.
When I went to school, the left side of the equation had to do the same thing as the right side of the equation.
So if the right side of the equation runs from 0 to 1, the left side of the equation runs from 0 to 1.
So, let's pick a -- are you ready -- average value from 0 to 1.
What would you pick?
I'd pick point 5.
Now, I don't have to go to the error function tables because point 5 is less than point 6.
So ERF of x over 2 root Dt is essentially equal to x over 2 roots of Dt.
So, now I've got one half equals one half x over root Dt, from which I can say that x is approximately equal to the square root of Dt.
And I can be sitting in a meeting and someone says, gee, I don't know how long it's going to take.
This thing's going to go down about 100 microns.
And how long is it going to take?
So, I pick a number, like, OK, if this is 100 microns, and this thing here is 10 to the minus 8, now you just take the square root of that, and tell them, oh, it's going to take so many minutes, and they go, huh?
How did you get that?
It's right there.
And I'm not trying to tell them a number for three significant figures.
They just want to know is it going to take a minute, an hour, a day, or are we out of business?
And you can make that calculation with this.
This is so powerful.
And everything I told you about diffusion applies to conductive heat transport.
You can do the same thing for heat transfer, figure out how long it's going to take for a wave to go in.
This is really powerful stuff.
Anyway.
Let's move on.
Well, there's our pal.
OK, so I was going to give you the last element today.
The last element.
Remember, we talked about clean air.
So, I'm showing you the catalytic converter.
That's this thing here, coated with platinum, rhodium, and so on.
And I'm showing you the electronic control module.
That's the CPU.
It's a little bit toned down from a Pentium, but it certainly started from one of these units.
So, we've studied all of this.
Right?
OK. And, so now, I want to talk about this piece here, the oxygen sensor.
why do we need an oxygen sensor?
Remember, the last day I told you that to get rid of NO, the NOx, you have to reduce, And to get rid of CO and unburned hydrocarbons, you have to oxidize.
Well, you have to pick.
You can't have two different atmospheres in the same place at the same time.
Turns out there's a lucky sweet spot.
Can you see that the conversion efficiency is high for both reduction and oxidation reactions if you peg the air-to-fuel ratio at 14.6.
So you can't just set the carburetor or the fuel injectors at some arbitrary ratio, because as you change humidity, as you change temperature -- what happens when the temperature goes down?
How much oxygen is there in the air?
It's still 20%, but PV equals nRT.
You ever try barbecuing when it's -- I mean, I don't know, maybe you're not from the Northeast.
But if you want to barbecue when it's 0 Fahrenheit?
Do you think you get a hotter fire or a colder fire?
How much oxygen is there per unit volume in cold air versus warm air?
Think about it.
It'll be on the next exam.
I'm just kidding.
OK.
So, anyway, here's the sweet spot that gives you both oxidation and reduction.
And so we need a feedback loop.
That feedback loop is provided by the oxygen sensor.
Dave, would you mind cutting to the document camera?
So this is an oxygen sensor that's used on automobiles.
Here's the plug.
This goes into the pin set.
And the oxygen sensor is inside this housing.
This housing, you can see -- OK, what I'm going to show you is what's going on inside here.
This has got all kinds of protective stuff on it so that it doesn't get dented during installation and so on.
But it sits inside the exhaust train.
It sits inside the exhaust train.
Now, may we go back to the slides, please?
Every car that's running a catalytic converter has to have one of those in order to control, right here, to make sure the air-to-fuel ratio is optimum.
Otherwise, the catalytic converter is either not doing a good job on NOx, or it's not doing a good job on hydrocarbons and whatever the other thing is.
Anyway, so here's what it looks like.
So, this is what's inside.
It's a closed, one-end tube made of zirconia.
Zirconium oxide.
And zirconium oxide conducts oxide ions.
And there are platinum electrodes on the front and the back.
And what I showed you was from this side.
And this is the exhaust gas going along here, and this is the edge of your exhaust train.
And two wires coming off of that sensor go to the CPU.
The CPU measures voltage and then from there, it regulates the air-to-fuel mix to the engine, to the exhaust gas, and around and around and around we go, so that when exhaust gas gets to the catalytic convert, it is optimally being converted.
So, we want to get fast response.
We don't want slow response.
And it's a solid-state sensor and solid-state diffusion is slow, but that's we have to rely upon.
So we add a dopant.
We add a dopant to zirconia to stabilize its crystal structure and create more oxygen vacancies to get faster diffusion and a shorter response time.
So what we add is calcia, one of the candidates, and it creates oxygen vacancies, and these oxygen vacancies allow us to get more rapid response time.
So, these oxygen vacancies compensate for the charge imbalance, thanks to the addition of calcia.
So, it's an engineered material.
Without this it doesn't work, all right?
And so, now what I've shown you is that on the basis of this oxygen sensor, we can monitor the exhaust gas flow so as to optimize the conversion here in the catalytic converter and do so by sending information to the CPU on the car.
And the number one consumer of CPU's is the automobile industry, because every car has at least one, if not multiple, CPU's.
So this is a good example of how understanding the lessons of solid state chemistry can allow us to build fuel efficient automobiles that have minimum toxic emissions to the environment.
All right, we'll see you on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we will have weekly quiz tomorrow.
There's been a lot of coverage, and so to focus you a bit, I'm going to confine the weekly quiz to glasses and chemical kinetics.
So you don't have to worry about diffusion.
We'll catch up on that, but I know there's so much material there.
Let's keep it confined to glasses and chemical kinetics.
And I'll be available today 4:30 to 5:30.
If you can't be at that time, send me a note, and I'm sure we can figure out a time to get together.
So what I want to do today is to start a new unit.
We're going to start talking today about solutions, and do some solution chemistry, OK?
Today we talk about solutions.
And you might initially say, why are we talking about solutions?
This is solid-state chemistry.
I think up until now, you've seen that it's pretty rare that we use solids in their pure form.
We usually have mixtures.
So for example, when we studied glasses, we modified the glasses with an alkaline earth oxide.
Well we, in fact, were making a solution of more than one component.
So it's to get certain properties that we study solutions.
Secondly, coming out of liquid phase is a way to make solids.
So in terms of processing, we need to understand something about solutions.
And then lastly, we're going to be studying, towards the end of the semester, a big unit on biochemistry.
And biochemistry, why, you say, biochemistry?
Why is he doing biochemistry?
I thought this was solid-state chemistry.
We are solid-state devices, but we're made of soft matter.
At least the exoskeleton is soft matter.
The endoskeleton is ceramic, right?
Our bone structure is ceramic.
Hydroxyapatite, calcium hydroxyapatite, and the outside is a polymer.
So look at this.
Confirmational changes in polymer.
But the chemistry, the biochemistry, so much of it takes place in aqueous solution.
So hence, we better know something about aqueous solution.
So to this, we go almost right back to the first day.
You remember, was the second lecture, and we showed this figure, and all the different categories of matter.
And we started over here with elements, and we moved into some pure substance compounds, et cetera, et cetera.
Now we're going to move over to here.
So homogeneous mixture containing uniform composition and properties, as opposed to a heterogeneous mix.
So we're now over here.
We're working our way through the diagram.
All right.
So let's get a couple of basic definitions up.
So the solution is really a mix of at least two constituents.
One, the majority constituent is called the solvent, and then we can have one or more solutes.
So the solvent, this is the majority constituent, and then the solutes are the minority constituents.
And in some instances, it's pretty hard to tell which is which.
And as far as types of solutions, I want to broaden this.
You know, when I say solution, you're probably thinking about aqueous solution.
But I want to take a minute and work through this chart.
And this is all posted, so you don't have to write it all down.
Just follow with me.
So most of the general chemistry subjects will just stop at the end of the first line.
They'll treat solutions as aqueous solutions.
But in material science, we think of things broadly.
So a good example of a simple aqueous solution is sodium chloride and water.
Sodium chloride is the solute and water is the solvent.
But you can have a liquid solute.
So wine-- in case you've ever encountered this beverage, it's primarily water, but it can contain up to about 14% ethyl alcohol, and there are other constituents that give the color and the flavor and so on.
And you can have a gas in a liquid, and that would be seltzer, where CO2 is dissolved.
It's actually dissolved.
If you take a look at a bottle of bubbly water on the shelf, you don't see the bubbles.
The carbon dioxide is actually dissolved.
You can have a gas as a solvent, and air would be an example of that, where the solvent is nitrogen.
And then we have as solutes oxygen, argon, carbon dioxide, if you live next to a power plant, it's sulfur dioxide, if you live next to an aluminum smelter, it's tetrafluoromethane, and so on.
So we have all kinds of solutes in the air.
And then we can have solid solutions.
And OK, as soon as you see the word solid, you know, that means you're 3.091.
So what do we see for solids?
Metal alloys.
We saw carbon dissolved in iron.
Well, that's a solution.
It's homogeneous, single phase, just as the definition on that chart 1.11 said.
Semiconductor, boron doping into silicon, the boron sits on a silicon lattice site.
So this is a true solution.
The boron is the solute, and silicon is the solvent.
We can have the ceramic.
We talked about the oxygen sensor.
The oxygen sensor is zirconia, which has been stabilized with the addition of calcium oxide.
As one example.
In contemporary work, they might use another oxide.
But what's the purpose of the calcium oxide?
Well, I told you that it's to increase the vacancy population, give you a rapid response on your oxygen sensor.
And as we're going to learn later, that's true, and there's a second value in putting in the calcium oxide.
And it stabilizes, that's why they use the term stabilize, it stabilizes the cubic form of zirconia.
Zirconia has a different crystallographic modifications.
The cubic one is the one that gives us the best ability to transfer oxygen, and the addition of calcium oxide as a solute, the calcium ions actually sit on the zirconium lattice, that's a true solid solution.
It also makes the cubic zirconia stable, and with Christmas coming, you know, it's the poor man's diamond, and that's the same material there.
And to modify a glass, as I mentioned earlier, adding a alkaline earth oxide breaks the silicate network.
That's a solution.
Now here's an inverse one, where the solid is the solvent, and the liquid is the solute.
So this was dentist's office practice going back to the time when I was your age.
If you had a feeling, the dentist had silver and mercury in the dentist's office, and would add liquid mercury to silver and make up this amalgam, and then jam that into the tooth.
Actually, that was the B part of the operation.
There was this dentist here in the United States, and if I could find him, I would like to have a word with him.
But he had this theory.
You see, the amalgam is a metallic alloy.
And so obviously, it has metallic bonding.
And the tooth is-- it's a ceramic.
So what kind of bonds are going to form between a metal and a ceramic?
They're not very good.
So this dentist had the theory that the way to increase the bonding capability of the amalgam to the tooth was to maximize contact areas.
So when you went in with a tiny, tiny little cavity, the dentist would drill the tooth out, removing about 75% of the volume of the tooth, and then they would make this amalgam and shove that in.
And you'd wear that for about 20 years, until one day you bite into a muffin that's got a little piece of walnut shell in it, and the amalgam flexes, and your thin-walled tooth goes bang!
Like that.
And now you get to go back to the dentist's office for yet some more medieval treatment.
Actually, things have improved somewhat.
Thanks to material science.
But they're not using this amalgam anymore.
But there are a number of us who are still walking around with silver and mercury in our mouths, and-- yeah.
Enough about my dental problems.
Now let's go on to intercalation.
You can put a gas into a solid.
And I told you about this one, where if you want to store hydrogen, and this is a big problem, if we're going to talk about hydrogen-powered vehicles.
A problem as big as, how to get the fuel cell cheap enough, is where are you going to store the hydrogen on the car, and one of the materials that's been proposed is this alloy of lanthanum and nickel that can intercalate huge amounts of hydrogen.
So that forms a solid solution.
So those are examples of solutions, and we're going to go back and make sure that we cover the basic Gen Chem at the top here.
But I want you know that solution chemistry is very broad.
Now when you dissolve something, you actually have things down at the atomic level.
So for example, in brine, you actually have below two nanometers as particle size.
You actually have sodium ions and chloride ions dissolved completely, which means that the solution is clear, colorless, and transparent to visible light.
Water is clear, colorless, transparent to visible light.
If you had sodium chloride, it remains clear, colorless, transparent to visible light.
You can't filter, and you can't wait for the sodium chloride to settle out, because it's bonded within the structure of the liquid water.
And this has implications.
There's so much water on the planet, but there's not so much fresh water on the planet.
And desalination can't be accomplished by filtration.
A filter that had a pore size small enough to trap sodium ions, the pore size would be so small, water molecules couldn't go through.
So this is the concept.
At the other end, you can get something called a suspension, where the particle size is greater than about 1,000 nanometers, and blood is a good example of that, where it's opaque.
Visible light doesn't go through, but you can filter out some of the matter in blood.
And if you let it sit for a while, it'll settle in it gravitational field.
And in between, you have this whole zone of colloids.
And so between colloids and suspensions, the only difference is particle size.
And this whole thing we would just call a dispersion of another phase, either another solid phase, or another liquid phase.
And it's kind of an interesting physical chemistry about the dispersion, what makes it work.
For example, in milk, milk is a good example of a dispersion.
You have a fatty phase, and you have an aqueous phase.
The aqueous phase, where all the minerals are.
Why?
Because the minerals are-- what kind of compound?
They're not metallic, they're not covalent.
They're ionic.
The ionic compounds dissolve in the aqueous phase, and then in the fatty phase, that's where you have the protein and so on.
But the fatty phase is clear and colorless, and the aqueous phase is clear and colorless.
And yet milk is white.
It's, as the term implies, it's milky.
Why?
What's going on?
You have a second phase here.
This could be the fatty phase, and this is the aqueous phase.
They're both clear and colorless, but they have different indices of refraction.
And since this has n fatty phase, and this has n aqueous phase, this interface scatters the light.
So if you want to start a business, if you want to try to tailor the index so that the index of the aqueous phase matches the index of refraction of the fatty phase, you'd have the two dissolve, one and the other, and it would be transparent, divisible light.
So you would have milk that isn't milky.
I mean, the public would be very confused.
But anyways.
So that's what you do.
So why does this thing not settle?
Because they do have a density difference, and again, going back to the days when I was not a college student but a youngster, there was still this form of milk called pasteurized milk.
Well, all milk is pasteurized, but this milk was not homogenized.
And what would happen is, there was this person that would deliver milk.
This was a borosilicate glass bottle.
And here would be the cream.
The cream would rise to the top.
We have all these expressions in our language.
And then this would be the milk here, and you could skim this off for coffee or sugar, and then this would be a low-fat milk.
But people wanted this all mixed, so then they went to homogenized milk.
So what is homogenized milk?
This is your red cap, now.
All the milk is homogenized.
This was a big thing.
This was simply called pasteurized.
I'm going to get to the physical chemistry here.
It's very interesting.
Because this is lower density, and yet in homogenized milk it doesn't rise.
So let's take a look at what goes on in the physical chemistry of homogenized milk, because it's all about these various systems.
So I'm going to take this particle here, and its sum insoluble cluster.
And I'm not specifying the cluster size.
It's probably greater than about two nanometers.
So there are two forces acting on this.
There's a settling force and there's a buoyancy force.
Obviously, otherwise it wouldn't float.
So there's some kind of a buoyancy force.
Settling force and a buoyancy force.
Well, the settling force, this is just the gravity.
Right?
This is the force of gravity.
And we know the force of gravity.
That goes with the mass.
Come on, get that cell phone out of here.
This force of gravity goes as the mass, and the mass, we know, goes as the volume.
And the volume goes as the cube of the radius.
I'm assuming this is a spherical particle, all right?
Now, the buoyancy force.
The buoyancy force is the interfacial force between the two.
There's some binding across here.
Maybe weak van der Waals, or if this fatty phase has molecules in it that are polar, then there could be dipole-dipole interaction.
But in any case, there's some kind of an interfacial force here.
And this is all chemical bonding between solute and solvent.
But you see, the force is a weak force.
If it were a really strong force, it would dissolve this thing.
It won't quite dissolve it, but there is some kind of dipole-dipole weak force.
And this one here is operating across the surface area.
That's the contact.
So this force goes as the area, and area goes as the square of the radius, whereas mass goes as the cube of the radius.
So you know, from your math, that r cubed dominates r squared, but only at large r. It's not always the case, is it?
Maybe before you got here, you thought that.
But now that you've been at MIT a few months, you know that r squared can dominate r cubed at small r. Interfacial forces dominate.
And that's exactly what happens in these dispersions, and that's why they don't settle out.
Now, homogenized milk is simply milk that has been agitated in such a way as to reduce the fat globule size below a critical value so that these interfacial forces hold the fat globules in suspension.
If you waited long enough, they would agglomerate and settle, but that time is probably longer than the shelf life of the milk.
So the milk probably spoils before stuff settles out.
So this is all very important to understand.
The range that exists between insolubility and this sort of clustering, and suspension, and so on.
By the way, a lot of pharmaceuticals are like this.
A lot of pharmaceuticals.
So when it says, shake well before using, they're not kidding.
Because the active ingredient will settle.
And you're drinking just the solvent, just the vehicle.
And all of the potency is on the bottom of the bottle.
Shake that thing up!
I can't say what it does to the taste, but that's another problem.
Actually, I threw in this slide here.
We're not going to spend any time on it, but you can look at it at some point.
This is a whole taxonomy of colloids.
Solid-liquid emulsions, aerosols, they're all part of this magic zone between solubility and just brick, all right?
There's this whole fine, pardon the pun, the whole fine structure.
OK, So let's get to the chemistry.
Obviously there's something to do with bonding here, right?
So here's a simple experiment, and this is taken right from the reading.
So I've just taken this episode that is written up in the reading.
So we've got two beakers here, and in each beaker, we have a bilayer.
We've poured in some carbon tetrachloride, liquid, and we've poured in some water.
And these two are immiscible, because carbon tetrachloride is obviously a non-polar liquid, and water is a polar liquid with hydrogen bonding capability.
And in one beaker, we introduce crystals of iodine.
In the other beaker, we introduce crystals of potassium permanganate.
And then we shake them up, and we wait.
And eventually we see that on the left, the iodine dissolves in the carbon tetrachloride.
And we're using the purple color as an indicator.
And this is kind of cute, because both potassium permanganate and iodine will render things purple.
So you're comparing purple to purple.
They could have chosen something else, but this is kind of cute.
All right.
So here you end up with a solution of iodine and carbon tetrachloride, whereas on the right side, you end up with a solution of potassium permanganate and water, and nothing in the carbon tetrachloride.
So what can we infer from this?
Well, let's take a look at the possible interactions.
So first of all let's categorize H2O.
This is polar, it's a polar liquid, with hydrogen bonding capability.
Carbon tetrachloride is non-polar.
It's very toxic.
When I was a child, we had this in the medicine cabinet.
It's a non-polar solvent.
It's fantastic for getting grease stains off.
Every man had this.
With a little handkerchief, you'd take a little grease stain off your tie.
You can't buy this stuff.
You can't even guy it for research anymore.
It's too bad.
It's great stuff.
Highly toxic, though.
But you know, there's a time and a place.
All right.
So then here's iodine.
Iodine, we know, is non-polar.
It's a homonuclear molecule, it has to be non-polar.
And so what holds iodine together in the crystal?
There's only one bond, and that's van der Waals, right?
It's a van der Waals solid, whereas potassium permanganate is an ionic solid.
Potassium permanganate is ionic, and it consists of potassium cations and permanganate anions.
And what we find is that the non-polar solute dissolves in the non-polar solvent, and the ionic solute dissolves in the polar solvent with hydrogen bonding capability.
So from this, we can infer the general rule, which is encapsulated in the language used in chemistry textbooks.
Like dissolves like.
And what they're really saying here, is that like solutes dissolve in like solvents.
Solute-solvent is like dissolves like.
OK.
It's a good place to start, but it's an incomplete picture.
So I want to show you that there's some sophistication here.
This is taken from one of the other books.
And it shows just the rules for ionic compounds in water.
And I just showed you, from this example, that the ionic compound dissolved in water.
And what you see here is that some ionic compounds dissolve in water, but there's a whole set or insoluble ionic compounds.
So it's not straightforward.
But we know from 3.091, we know that we can make sense of this on the basis of competition.
Competition between what holds the compound in the solid state versus what will pull it into the liquid state.
So for example, we can compare sodium chloride, which we know dissolves in water.
So sodium chloride, as a crystalline solid, will dissolve to form sodium chloride aqueous solution.
Whereas if you look at magnesium oxide, which is also an ionic crystal, it does not, to any standard imaginable, dissolve to form an aqueous solution of magnesium oxide.
And what's the difference here?
The difference here is, compare solvation energy, in other words, the energy that you got by pulling this into solution, and forming bonds between sodium and chlorine in water, with the crystallization energy.
And what's that all about?
Well, that's the Madelung constant, remember?
Madelung constant.
q1 q2 over 4 pi epsilon 0 r, where this is the cation anion.
And you can see here that sodium chloride has sodium cations and chloride anions, and there's a certain binding energy in the crystal.
Magnesium has 2 plus.
Oxide is 2 minus.
The binding energy between magnesium and oxygen is so great that there's no driving force to dissolve.
Now, I don't expect you to be able to look at this and tell me whether something's going to dissolve or not.
But if I were to say to you, explain why sodium chloride forms solutions with water, and magnesium oxide doesn't, that you could go through this rationalization.
And here's a cartoon from the textbook that tries to illustrate this solvation.
Here you can see a crystal of sodium chloride, in the inimitable fashion, as drawn by chemistry books, where the chloride ion is green, and the sodium ion is blue.
And that's OK.
I can live with the color-coding, as long as we agree amongst ourselves, these ions are clear and colorless, because they have octet stability in their electronic shell.
And here's water, with the hydrogen shown in white, and the oxygen shown in red.
And you can see that the oxide end, the oxygen end, the delta negative end of the water, is trying to wrest sodium cation out of the lattice, and ultimately surround it by a cage.
And the same thing here.
You see the hydrogen ends of the water trying to pull chloride out, and ultimately surround it with a water-like cage.
So this is the competition I was talking about.
In the case of sodium chloride, water wins.
In the case of magnesium oxide, water loses.
The binding energy between magnesium and oxygen and the crystal dominate.
So you don't see that solvation.
OK. Let's talk about metrics now.
Let's look at some metrics of solubility.
It's quantifiable.
So we can express a measure of solubility in terms of a quantity known as molarity.
So we can express moles of solute per liter of solvent.
And this is called molarity, and the symbol is capital M. So we can say, for example, a 1 molar solution of sodium chloride in water, we'll write 1 molar NaCl, and then we'll write subscript aq, meaning aqueous.
So it's an aqueous solution at a concentration of 1 mole of NaCl per liter.
And the liter is named after a person, so that's capital L, per liter of solution.
And remember, the solution is the sum of the water plus solute.
For dilute solutions, there's very little difference between the total amount of water and the total amount of solution.
But in certain instances, the presence of the solute actually has a volumetric change on here.
So strictly speaking, it's per liter of solution And as I've shown you, there are degrees of solubility.
So people represent the threshold of solubility as c of the solute.
When c of the solute is less than about a million molar, 0.001 molar, we call this insoluble.
So something is vanishingly soluble, we say that's the value.
So we'll call this the threshold.
And then, something that's quite soluble, we'd say that the concentration of the solute starts to exceed at about 0.1 molar.
And then we would say, that qualifies as soluble.
So now let's look at two extremes in solubility, operating off of this.
So in the one case, we can have complete solubility.
So examples of that, where things are completely miscible in one another.
Complete solubility.
By the way, some people will use the term miscibility.
When something is miscible it means it's soluble.
Same idea.
If something is insoluble, some people might say it's immiscible.
Same thing.
OK?
So complete solubility is ethyl alcohol and water.
You can mix them in all proportions.
Continuously variable.
On the solid alloy is silver and gold.
Silver and gold, you can make alloys of any composition between 100% gold and 100% silver.
Now, that's the exception.
Most cases are situations of limited solubility.
So they go up to a maximum, which we can denote C-star or C saturation.
This is the maximum solubility.
So an example of something that's sparingly soluble in water is silver chloride.
Let's look at silver chloride.
So silver chloride, I'm going to start here, silver chloride as a crystal, and I'm going to dissolve that in water.
So that gives me AgCl aq.
So that's just simply the formation of an aqueous solution of silver chloride.
And when the reaction moves from left to right, we call that dissolution.
The silver chloride is dissolving, and when the system moves from right to left, we call that-- now here I'm going to nitpick with the book.
The book calls the left reaction crystallization.
And that's correct.
It is crystallization in this case, because silver chloride is a crystal.
But it is possible, in other systems, to have the solute come out of solution, and make a solid that is noncrystalline.
And you know that all crystals are solids, but not all solids are crystals.
You can have an amorphous solid.
So what would happen if you were to bring out of solution an amorphous solid?
It would be silly to call it crystallization.
So I prefer to use the term precipitation.
And there's another term that you can use, and I learned this one from reading the literature of geochemistry.
What the geochemists call the reaction going from solution to make a solid, they say the system exsolves.
This is dissolve, this is exsolve.
So this is called exsolution.
It's amazing what you can do when you know a little bit of Latin.
So this exsolves.
So that's the exsolution, or the crystallization reaction.
Now I'm going to show you what won Arrhenius his Nobel prize.
Arrhenius did not get the Nobel prize for his brilliant work on activation energy.
He got his Nobel prize on the theory of electrolytic dissociation, which was, people knew that you could dissolve salts and water, but they didn't know how.
And it was Arrhenius who said that the salts go into solution by dissociating and forming ions.
So goes in as Ag plus Ag plus aq plus chloride ion-- thank you.
And that, ladies and gentleman, was a Nobel prize for Arrhenius.
And you can see that there's a relationship between the amount of silver chloride and the amount of ions.
So there's a mass balance there, that the concentration of silver chloride dissolved, in fact, equals, in this case, by stoichiometry, the concentration of Ag plus, because of the nature of the dissociation reaction on a mass balance basis.
And that also equals the concentration of the chloride ion, OK?
So that's the way we can look at the system.
And how do we know that this thing has limited solubility?
Well, there's various ways of measuring it, and one of them involves conductivity.
Here's the conductivity of pure water.
And you know that water has very, very poor conductivity, and in fact, what's happening here, when we add silver chloride is, we're adding charge carriers, because the audience are charged species.
So they can carry charge.
And you can see that this is a measure of conductivity as a function of silver chloride concentration.
And as you add silver chloride to higher and higher values, the conductivity goes up.
And look at even the tiniest amount of silver chloride has a conductivity that's about, what, half an order of magnitude higher than the conductivity of pure water itself.
So I would say that this is akin to doping, isn't it?
So up here, when you've got 10 to the minus 6 Siemens per centimeter conductivity, that aqueous solution is demonstrating the extrinsic behavior.
This is very similar to doping.
And at some point, we get to this value here, around 10 to the minus 5 moles of silver chloride per liter, and then adding more silver chloride has no impact on conductivity.
Which tells you that you've hit saturation.
This is akin to adding more and more sugar to the cup of tea, until finally the sugar just falls to the bottom.
You can stir all you want, but you won't get it to dissolve, because you've hit saturation.
So that indicates the presence of a saturated solution.
And so we can talk about that value, and we can say that for silver chloride, the concentration at saturation is equal to 1.3 times 10 to the minus 5 moles per liter.
Moles of silver chloride per liter.
And obviously, that's equal to, according to that, it's equal to the silver ion concentration.
I'm going to use square brackets to indicate moles of silver, ion per liter of solution, which is also equal to-- pardon me-- it's also equal to the chloride ion concentration at saturation.
Now I'm going to ask you a simple question.
Suppose I've got a beaker here, and I know the maximum I can get here, the maximum is 1.3 times 10 to the minus 5.
Now this is a really simple question.
Suppose I am about to add silver chloride-- let's say this is 1 liter already.
All right?
I've got one liter.
And you'd tell me, well, you can put in 1.3 times 10 to the minus 5 moles to get the saturation.
Suppose instead of 1 liter of pure water, I had 1 liter of water already containing, say, 1 times 10 to the minus 5 molar silver chloride.
Well, that's kind of obvious, isn't it?
I'm only going to be able to put 0.3 moles in, because 0.3 times 10 to the minus 5, because I've already got silver chloride in there.
That's easy.
Now let's make the question a little more interesting.
Suppose instead of a certain amount of silver chloride in there, I have no silver chloride in there.
But I've got, say, 0.1 molar sodium chloride.
It's a salt.
It's a difference salt.
So the question is, does the presence of a different salt have an impact on how much silver chloride I can put into this solution?
And the answer is, yes.
The answer is yes.
So what we find is that the presence of the other salt, in this case, has an impact, because sodium chloride goes in as sodium plus, and chloride minus.
So there's already a boatload of chloride ion in there, and that has an impact on this relationship.
So how do we answer the question, what is the solubility of silver chloride in the presence of other chloride ions?
And for that, we define something called the solubility product.
And you need it in order to answer the question, how do you determine the solubility of a solute when there are other solutes present already?
And it's denoted capital K, lowercase sp, subscript.
Solubility product.
And it's equal to simply the ion products of the constituents.
So the solubility product of silver chloride is the product of the silver ion concentration, and the chloride ion concentration.
And you know that in a solution of silver chloride alone, if nothing else, that the concentration of silver ion equals the concentration of chloride ion.
That's the whole business of dissolving by itself.
And so I can then just say that Ksp.
will then equal the concentration of silver ion squared, which we also know is equal to the concentration of silver chloride aqueous.
See, all of these are the same.
So this solubility is the same.
I can just put that in, which is just concentration of silver chloride.
Square that.
So if I square 1.3 times 10 to the minus 5, I end up with a solubility product of 1.8 times 10 to the minus 10.
So now I can use this in order to determine how much solubility I get in the presence of another salt.
So in this case, I'm going to put 0.1 molar sodium chloride.
And these are strong salts, so we're going to get complete dissociation.
Gives me 0.1 molar sodium ion, and 0.1 molar chloride ion, when it dissociates.
Now you see the difference.
Because the silver chloride, by itself, gives me 10 to the minus 5 molar chloride ions.
When I add sodium chloride, I get 4 as a magnitude more chloride.
So let's go back to the Ksp.
So Ksp, this is for silver chloride.
Ksp for silver chloride is going to be equal to the silver concentration and the chloride concentration.
And in this case, the silver concentration is just equal to whatever that solubility is.
Because there's only source of silver ion, and it's silver chloride.
So I can write that as concentration of silver chloride.
That's good.
And now this one here is what?
I've got two sources.
I've got silver chloride, I I've got sodium chloride.
So it's going to equal this thing here.
0.1 plus whatever I get from silver chloride.
And it's vanishingly small, isn't it?
The concentration of silver chloride, whatever it is.
This is nothing, so I'm just going to neglect it.
It's dominated now.
The chlorine is flooded by the silver chloride.
And this product is a constant.
That's still equal to 10 to the minus 10.
So I can turn this around and solve for the concentration of silver chloride, which is equal to the concentration of silver io.
And that's equal to, what is it, 10 to of the minus 10 divided by 0.1, which then gives me-- what's the number here?
Plug that in.
And I end up with 10 to the 1.8 times 10 to the minus 9 molar.
Right?
So look.
Look at what's happened by having the chloride present from sodium chloride in such a large amount.
It's repressed.
It has repressed the dissolution.
The presence of chloride then has a negative impact on solubility, and instead of having 10 to the minus 5 molar, it's down to 10 to the the minus 9 molar.
And this effect of repressing solubility by adding a second solute is called the common ion effect, OK?
Solubility repression by second solute is known as the common ion effect.
And this is used in processing.
If I want to trigger the precipitation-- see, if I started with a solution containing 10 to the minus 5 molar of silver chloride, and I throw in some sodium chloride, it'll start precipitating out silver chloride.
So if I wanted to make a fine precipitate of silver chloride, I make a pregnant solution.
And I could drop the temperature, because you can imagine that solubility is a function of temperature, or I could keep it isothermal, throw in some sodium chloride, and out comes silver chloride.
So since the common ion effect on it and its value in processing.
OK. Good.
Well, I think I'm going to hold it there.
I"ll show you just one more thing.
If you've got stoichiometry like this-- this is a difluoride of magnesium.
If it goes into solution, you get magnesium cation plus 2 fluoride anions.
And so if I wrote the Ksp for this reaction, it would be the product of the magnesium concentration and the fluoride ion concentration.
Because there's the two here, this is squared.
So this, too, will transfer up there, and then throw in some sodium fluoride, and cause the other thing to exsolve, and away we go.
All right.
I've got a couple of things to show you.
We're talking a lot about Arrhenius.
This is a book I have.
It's an English translation of a book written by Arrhenius in the late 1800s, and it was printed in English around 1908.
And Arrhenius was a genius.
He wrote on all sorts of topics here.
Biology, physics, you name it.
And one of the things and he was interested in was the origins of the earth.
So this chapter is called, Celestial Bodies as Abodes of Organisms.
Already speculating on whether you could have life as we know it exist elsewhere in the universe.
And one of the things he talks about is global warming.
So this is Arrhenius on global warming.
I'll read you the last paragraph of this chapter.
There had been some major volcanic eruptions that had caused cooling when Krakatoa in 1883 and Martinique in 1902.
Major plumes of soot that caused dramatic decreases in temperature.
So now here comes the last paragraph.
We often hear lamentations that the coal stored up in the earth is wasted by the present generation-- remember, this is written 100 years ago-- without any thought of the future.
And we are terrified by the awful destruction of life and property which is followed the volcanic eruptions of our day.
We may find a kind of consolation in the consideration that adheres in every other case.
There is good mixed with the evil.
By the influence of the increasing percentage of carbonic acid in the atmosphere-- that's CO2-- we may hope to enjoy ages with more equitable and better climates.-- Remember, he's a Swede; it's cold-- especially as regards the colder regions of the earth, ages when the earth will bring forth much more abundant crops than at present for the benefit of rapidly propagating mankind.
So it's interesting to see the world-- it's a great book to read.
And you can see people in the 1830s were already calculating heat transfer coefficients to how much the earth was changing.
We're going to post these to the website.
This was, last year, in the New York Times.
Every Tuesday they have a science section.
And this was about glass.
And very, actually, with your 3.091 knowledge, you'll read this like a newspaper, and it'll be very easy.
And they go through the structure.
Here's the structure of a window glass.
You can see the network, former network modifier, and so on.
And they talk about how difficult it is from a first principle's standpoint to model the structure of glass.
These oxide glasses are complex and not easy to model.
So when you're trying to engineer the glasses, instead of trial and error, it's hard to do so by theory.
And it talks about some efforts at theory, and so on.
And the last thing I want to talk about is bulk metallic glasses.
You recall that I showed you metallic glass that was made by melting gold silicon, and dripping it onto a water-cooled copper wheel that was zooming around to give us a cooling rate of about a million degrees a second.
And those strips had to be very, very thin, the metal strips.
Because you've got liquid dripping down, and we're pulling the solid away, and there's a finite thickness here.
Let's use a Greek letter, xi.
There's a finite thickness xi.
Because what's happening is that this is the water-cooled copper wheel, and you're extracting heat here.
But eventually, the thermal conductivity of the metal is the limiter.
In other words, I don't care how cold.
You can put this in liquid helium if you want.
You can't get the heat through the metal fast enough.
And what happens is, when you look down here, the lower part is amorphous and the upper part is crystalline.
So you don't end up with metallic glass.
You end up with some metallic glass, and the upper part is crystalline.
So what happens when you end up with the limitation being the thermal conductivity of the metal?
At that point, you're finished.
And this was typically on the order of microns.
And then they got up to, sort of, submillimeter.
And that was it.
So the glass that I showed you was foil.
Now what happened was, with more research-- so here we are.
This is Pol Duwez at Caltech, gold silicon.
And this is the thickness in centimeters.
So you were down here at some tens of microns, all right?
Now, in the '60s, research at Harvard uncovered a set of palladium alloys that had better thermal conductivity and, remember the first day, about the analogy to the musical chairs?
These things have slightly more complicated crystal structures.
So for a given cooling rate, the metal has more difficulty finding the proper lattice site.
And they were able to make metallic glasses that were on the order of 0.1 centimeters.
That's still fairly thin.
And then back to Caltech in the late '80s, early '90s, and a man by name of Bill Johnson was able to develop a set of alloys that can be made in bulk form.
Totally amorphous metal.
And these are known as bulk metallic glasses.
And look at the complexity of the alloy designation.
So now you see, well, why are they doing this?
Because look, this is strength versus elastic limit.
So you can either have things like polymers, that you can stretch very, very far, but they don't have much strength.
Or you can have things like certain steels, that are very, very strong, but you can't bend them very far.
And bulk metallic glass has put you right up here.
You have strong alloys that have very, very high elastic limits.
So they make great golf clubs and tennis rackets and so on, because they can flex way back, store enormous energy, and then spring.
So here are some bulk metallic glasses, and here's a classic one.
This is zirconium, titanium, copper, nickel, beryllium.
All right, now how do we get that?
Well, zirconium and titanium are body-centered cubic, we've got copper and nickel are face-centered cubic, and beryllium is hexagonal close-packed.
So the idea here is the principle of confusion.
So the alloy is fighting with itself.
You know, am I face-centered cubic, am I body-centered cubic, am I hexagonal?
And this confusion about what the crystal structure is to be leads to quenching and the disorder of a liquid state, and preventing the formation of grain boundaries, dislocations and so on.
So I want to show you one example besides golf clubs.
Dave, could we cut to the document camera here?
Get this thing down.
So this is a-- I'm not endorsing the product at all, but you know, this is one of the companies that makes, this is SanDisk, I think.
And they make these flash memories.
This just looks like a another piece of metal, and in fact, it's very disarming, because they call this model the Titanium.
Well, it has about 13% titanium.
The interesting thing here, what's so cool about this, this is bulk metallic glass.
And what's attractive about the fact that it's bulk metallic glass is, that it can be shaped by casting, from the liquid state to very, very fine precision.
So you see all of these slots and everything, and on the side, all of this kind of stuff, and on the end.
All of this is done by casting from the liquid state, in one operation.
And this is a clam shell.
There's two pieces here.
I don't know if you can see, but there's two pieces that have been sandwiched together.
You can see on the edge there.
And the impact that has on manufacturing costs is phenomenal, because normally you make the basic shells, then you have to drill, and you've got to auger out, and so on.
This thing, one step operation, including the labeling.
There's no subsequent processing.
And this is done by a company out in Michigan called Liquid Metal.
And they've licensed the technology and so on.
And again, I'm not trying to make a commercial thing, but I'm just trying to show you that these ideas of changing properties for engineered materials, you know, are around us everywhere.
And this is relatively recent.
Bulk metallic glass.
Fantastic example of structure property relations.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So a couple of announcements.
Weekly quiz Tuesday.
And also on Tuesday, I want to draw your attention to the event in the slide.
We have a poster here for a lecture that will occur right in this room, at 4:00 on Tuesday.
And it's entitled the Wolf Lecture.
The Wolf Lecture was established here about 30 years ago in honor of Professor John Wolf.
John Wolf was an antecedent of mine.
He was the person that invented 3091.
And in 1961, he dared, as a metallurgist, to launch a variant of freshman chemistry.
And you're the beneficiaries of John Wolf's initiative.
And he also was a spectacular teacher, quite a showman.
And in his honor, they instituted in the Department of Material Science and Engineering, the Wolf Lecture, which, as the poster says, is the entire community is invited to attend, but it's geared towards freshman.
So this is a lecture that the rules are.
Talk about whatever you want in terms of material science and engineering, but make it entertaining, and make it engaging and accessible to freshmen.
You should go to lectures.
If you can't go to this one, then by all means, go to other seminars.
If all you do when you come to MIT is go to class, you're missing out on some of the richness here.
We have all sorts of people coming through here every day.
And you can go to these seminars, you can learn a lot.
You might say, well, I'm just a freshman, I'm not going to understand everything.
I go to these things.
I don't understand everything.
But I learn something, because the first few minutes of the lecture, if the speaker is any good, he or she is going to set up the topic for you.
If you follow the first five minutes, you'll have something that, you know, gets you oriented to the topic.
And then the other thing, the other reason to go-- you see this on the poster-- room 10-250, reception immediately following.
That means there will be refreshments!
Food, for that, you know, pick-me-up in the late afternoon.
So you go to the seminar, you know, sometimes they have refreshments before the speaker.
I'm not going to tell you have to go into the seminar, having eaten the food and partaken of the refreshments.
You didn't hear me say that.
But anyways, there's always food around here.
Go to this one.
I think you'll enjoy it.
And the speaker is Professor Michael Rubner.
He's a faculty member in course 3.
An excellent teacher.
As you see, MacVicar Faculty Fellow, which means he's been acknowledged as a good teacher.
I think you'll learn a lot.
And he's talking about nature-inspired, so biomimetics, how we see things in nature and then mimic that design in advanced material science.
OK.
Enough said.
Let's get on with the lesson.
Last day we started looking at solutions, and we recognized that bonding is the key to understanding, and it's encapsulated in that catch-all phrase, like dissolves like.
And towards the end, we talked about Ksp as a metric that helps us understand common ion effect.
Common ion effect, again, if I tell you you have, say, 100 units of sodium chloride that can go into solution, you would say, that's the solubility limit.
But then if I tell you, I've got a solution that already contains 25 units of sodium chloride, that's a no-brainer.
You've only got another 75 units to go.
Now I say I've got potassium chloride in there.
So now you stop and think, wait a minute.
It's not sodium chloride, but it is a chloride.
How do I think about this?
Common ion effect through Ksp helps you reason through it.
It's when you have more than one solute, and one of the constituents of the solute is already present.
So today I want to talk about a subset of solutions, and in particular, I want to talk about acids and bases.
And this is important, not only in materials processing, but we're going to have to understand this if we're going to go forward later and talk about biochemistry.
So, you know, they're around us everywhere.
You know, you probably got up this morning, you washed your hair with a pH-balanced shampoo, maybe had orange juice or grapefruit juice with citric and ascorbic acid.
Your radio is powered by zinc alkaline batteries.
I started my car, it's got the lead acid battery in it, and electricity for all my appliances came from coal-fired power plants, spewing out SO2, turning out acid rain.
So we're off to a good start!
It's Friday.
So now we want to go back and understand this from the beginning.
So we're going to start with a history lesson.
And the history lesson starts in ancient times.
Acids were known all the way back in early times for processing of food and materials.
So who among us hasn't eaten something that has been pickled?
Pickle means to have been processed in acid solution.
In fact, the word acid comes from the word acidus in Latin, and acidus means either sour or tart.
But the modern chemistry of acids and bases starts with Lavoisier in France in 1779.
And I'm going to be naming many scientists from different countries today, so I'm going to use the international symbol.
You know, these ovals that you put on the back of the car, so now if you see a car with this on it, you know it's a French registry.
So he's probably got a Peugeot, and he's driving around with his Peugeot.
And what Lavoisier said, in trying to understand acids and bases, is-- it's a really interesting story.
He said that oxygen is present in all acids.
And why?
Because he spent most of his career studying combustion, so he was very concerned about oxygen.
And the interesting thing is that oxygen, the name of the gas, oxygen, actually comes from the greek oxy, which means sharp, and, you know, the particle gen, as in to generate or to be born.
OK?
So it's interesting that this gas is named incorrectly.
It's named for an attribute that is ascribed to acid, and it's wrong.
And it's in other languages, too.
The word oxygen translates into other languages as meaning something sharp, and it has nothing to do with it.
But while we're on the topic of Lavoisier, Lavoisier did study oxygen.
And there was an intense rivalry between Lavoisier in France, Joseph Priestley in Britain, and Scheele in Sweden.
And all three of them were working simultaneously to study combustion and understand oxygen.
And here I've got a very nice image.
This is a portrait of Lavoisier with his wife.
And it's hard to see.
If the lights were a little bit dimmer, you'd see-- notice here, you see all of this chemical apparatus.
There is bell jars, and various glass apparatus.
His wife was his partner in the laboratory.
She was unique among French women in that she read English.
And in his rivalry with Priestley, he relied on his wife in order to read Priestley's writings.
So they worked together.
She even helped him in the laboratory.
They married at the time he was 27 and she was 13, and they were-- what are you so shocked about?
It's France, and it's 1779!
Get over it!
Anyways, she helped him a lot, and we'll say a little bit more about that later.
OK.
So now let's go on to the next part of the history lesson.
So so far, we've got one explanation, and it's wrong.
So let's go to the next one, and we'll go to Britain.
Sir Humphry Davy in London in 1810.
So we'll put GB here, and he's driving around in his little Jaguar, I suspect.
And he said something that was correct.
He said that it's hydrogen present in all acids.
And that was pretty much it.
He was a great scientist, did some very good work, but really didn't do anything quantitative here.
So for the quantitative stuff, we have to wait for almost the end of the century.
And we go up to Sweden again, to Arrhenius.
And Arrhenius, in 1887-- so I'll give him a Volvo.
And he said that the acid is a substance that disassociates to produce protons.
So the acid is defined as a solution that dissociates to give protons.
I'm going to say H plus, meaning the hydrogen ion, or P plus, and we're going to keep using the term proton.
I'm not going to say hydrogen ion, I'm going to say proton.
So proton in solution.
So this is what he defines.
And he further defines the base.
He defines the base as the complement to the acid.
And he says that the base is something that dissociates to give us hydroxyl.
The base dissociates to give OH minus the hydroxyl.
And so now we've got this whole concept of electrolytic disassociation, which is what wins him the Nobel prize in 1903 for this thing.
And then we saw last day how the addition of ions to water gives charge carriers.
And so the presence of protons and hydroxyl is, in fact, the way we have any charge carried through water.
So up until now, when I said electrical conductivity, we were pretty much referring to electronic conductivity.
But there's a second kind of conductivity, and we can have ionic conductivity.
Ionic conductivity, as the name implies, is not by electrons, but by ions.
And this is typically ions in solution.
Ionic conductivity in solution, and a solution that is an ionic conductor is called an electrolyte.
So we have electrolyte in our bodies, saline, about 5% chloride.
We have electrolyte in batteries that are ionically conductive.
And the term simply means, we have conduction by ions.
And so it was the theory of electrolytic dissociation that won the Nobel prize.
So how does this work?
So we can start with something like HCl gas.
I'm going to dissolve it in water, and this will give me proton, H plus.
And I'm going to write aq, meaning that it's dissolved in water, and the chloride ion, also dissolved in water.
And so this gives me an acid.
This is a proton donor.
And now let's look at something like sodium hydroxide.
Sodium hydroxide at room temperature is an ionic compound, so it is a solid, but it is soluble in a polar, hydrogen-bonded liquid, so H20, to give hydroxyl aqueous plus sodium ion aqueous.
So again, we see the dissociation.
And then we can have, from here, a neutralization reaction.
And the neutralization reaction is simply reconstitution of the solvent.
So neutralization, another way to think about it, because we're not going to be confined to water by the end of the lecture, neutralization is simply a reaction that results in reconstitution of the solvents.
So the solvent, in this case, is water.
So how would we reconstitute water?
We combine proton with hydroxyl.
So let's do that and see the result of that reaction.
So if I take, and run it in the vertical direction, proton plus hydroxyl, we'll give water again-- let's give H20 liquid-- and now you can see Na plus Cl will give me NaCl, aqueous dissolved.
So this is what you probably learned in your high school chemistry, that acid plus base ca give you salt plus water.
So you see all of this resulting from just the simple Arrhenius definition.
So this is good.
We've got off to a decent start here with Arrhenius.
But then the theory has its limitations, as they all do.
And how do we discover the limitations?
With some new data.
So let's look at some data.
So it had been known for a long time that ammonia, when it's dissolved in water, can act as a solution capable of neutralizing an acid.
So if ammonia dissolved in water neutralizes an acid, then this must be a base.
But look, there's no hydroxyl here.
So the Arrhenius definition of base is inadequate to account for this.
So let's just get that down.
Ammonia, which you know is a gas at room temperature, will neutralize acids.
But no hydroxyl present.
So something's going on here that we can't account for by the simple Arrhenius definition.
So to get us out of this conundrum, we had to wait until 1923.
And two scientists simultaneously enunciated the same ideas.
So I'm going to put them both down.
1923.
Bronsted, who was in Denmark, and Lowry, who was working in the UK.
Also 1923.
And I'll give him a Jaguar as well.
And so the two of them proposed a broader definition to account for what's going on here.
And what did they say?
They said that acid-- they'll keep the same definition as Arrhenius.
So an acid is going to be a proton donor.
So that's good.
Let's even put here, same as Arrhenius.
But now here's the difference.
The base is no longer confined to hydroxyl chemistry.
They call it a proton acceptor.
And that's different.
So hydroxyl-free.
Now, let's be careful here.
If I propose this new definition, I can't throw out hydroxyl.
So you've got to watch to make sure that by broadening the definition, we don't exclude hydroxyl.
So the theory has to encompass what we already know, and then continue to encompass things that tend to contradict what we already know.
So let's take a look at what this gives us.
So I'm going to write a broader equation here.
I'm going to write this as the acid.
So what do I have?
I have every acid has to have a proton in it, plus some residual.
So I can rewrite the acid in this form.
I'm going to react it with a base.
So according to this definition, this has to be a proton donor.
And this, you're going to watch me on this one.
This is going to be a proton acceptor.
And we'll put some identities here.
Eventually we're going to write this with ammonia in it, but let's just do the broad definition.
So what happens is, if this is a proton acceptor, on this side of the equation, it takes the proton from here.
This is the proton donor.
It gives it away, and we end up with a BH plus.
So this proton acceptor has accepted the proton, leaving behind the deprotonated A minus.
That's a nice little reaction.
But now, if you'd nodded off for the last 90 seconds, and then opened your eyes and said, well, he said acid is a proton donor, this thing can give up a proton.
So OK, I'm going to call this thing a proton donor.
And this is most certainly a proton acceptor.
Look, it has accepted the proton.
So this is going to be a proton acceptor.
So now I've got in this equation two proton donors and two proton acceptors.
And they're linked.
This proton donor, BH plus, is linked to B.
So I'm going to put a little yolk over this one.
And this HA, the original acid, is linked to this base, A minus.
So I'm going to put a yoke over that.
And the Latin word for yoke, to yoke, is jugere, from which we get the word conjugate.
So I'm going to call these things conjugate acid-base pairs.
And every equation has two.
Because you've got a proton donor proton acceptor.
So now this is, follow the bouncing ball.
If you want to understand acid-base chemistry, just follow the proton.
Here's proton here, goes over here.
Here's something deprotonated, now it's protonated.
That's the rhythm here.
So the whole thing is, acid-base reactions can be defined as a proton transfer reactions.
Remember we talked about ionicity and electron transfer in order to achieve octet stability?
So we saw the whole life and times of electron transfer reactions.
Now life and times of proton transfer reactions.
OK. And last thing I want to do, is to just make sure that we see the definitions.
You know, all queens are female, but not all females are queens.
So this one here, what's this?
This is definitely, it's a proton donor.
So it's an acid.
So it's definitely an Arrhenius acid, because it's got a proton, and it's also a Bronsted-Lowry acid.
Now, what about this one?
This is a base.
It may not have hydroxyl in it.
May or may not.
So this is definitely a Bronsted-Lowry base, but I can't say for sure that it's an Arrhenius base, because Arrhenius has to be O minus.
This one here, what about this?
Unless it's OH minus, this one here is, what?
It's Bronsted-Lowry base.
And this one here, this can be both Bronsted-Lowry acid and it can be an Arrhenius acid.
So this is how we can differentiate them.
OK, good.
Now let's go to the ammonia, see if we've got ourselves out of the conundrum so we can write now NH3.
And I'm going to say that this is ammonia that's already dissolved.
So ammonia is already in aqueous solution, and it reacts with water, H20 liquid, to do what?
This is supposed to be a-- if this is a base, remember, this is the thing that we're trying to demonstrate.
If it's a base, according to this, it's a proton acceptor.
So I'm going to stick a proton on this, and I'm going to get NH4 plus the ammonium ion.
And how did I get the ammonium ion?
I took a proton from water, leaving behind OH.
So can you see that how this thing works?
By gobbling up protons, by becoming a proton vacuum cleaner, it takes protons out of water, leaving an excess of hydroxyl.
And in effect, now we've got something that's tantamount to an Arrhenius base.
But it all started off with this.
So now we can link these two.
I'm going to yoke ammonium and ammonia as conjugate acid-base, and there is water and hydroxyl as acid-base pairs.
So again, let's get the colors going.
Here we've got Bronsted-Lowry-- the bases are always going to be in blue.
Blue and base both begin with the letter B.
So this is definitely Brosted-Lowry base, but it's not an Arrhenius base.
This is a Bronsted-Lowry base, and it's also an Arrhenius base.
This one here, we can call this one-- in this case, it's a proton donor, so it's both an Arrhenius acid and a Bronsted-Lowry acid.
But look at this.
You'd say, OK, I know what he's doing.
Arrheius, Bronsted-Lowry, Arrhenius, Bronsted-Lowry.
This is Bronsted-Lowry only, this must be Bronsted-Lowry only.
But look!
This is a proton donor.
It dissociates to give protons.
So this is both Bronsted-Lowry and Arrhenius.
Most people will miss that.
We'll get to see, maybe, on a subsequent celebration, how many of you miss that.
OK.
So.
So now, let's go into the chemistry here.
Because I said up here at the beginning of the lecture-- what is it-- bonding is the key to understanding.
So what is it about bonding here that defines the Bronsted-Lowry base?
So I'm going to write this reaction in this way, now.
I'm going to put-- here's the proton, which is coming from water.
And I'm going to write that with NH3.
I'm going to use the Lewis structure.
So, you know, Lewis structure looks like this, Nitrate has got five valence electrons, three of them are bound, and now I've got this long pair.
So what do we know about the electronic structure of proton?
What does it look like?
What's the Lewis structure of proton?
How many electrons on proton?
None.
This is nothing.
This is very needy.
It's like the neediest friend you have.
You know, the one that's always taking stuff from you, taking your emotional energy, giving back nothing?
That's proton.
That's the human equivalent of proton.
So if proton wants to make a bond to form NH4, proton is going to come over here and exploit both electrons.
So if this is going to be a proton acceptor, the only way it can be a proton acceptor is to have two unused electrons available.
This is going to be a dative bond, agreed?
Dative, because both electrons from the bond come from the nitrogen.
The proton doesn't contribute anything.
So that's the hallmark.
The proton acceptor axiomatically must have an available non-bonding pair.
So wherever you see non-bonding pairs, wherever you see a compound that has this capability, it could serve as a Bronsted-Lowry base.
We know that water, because it has hydroxyl in it, has to satisfy this.
So let's take a look.
H20 we can do similarly.
Oxygen.
It's got 6 electrons, 2 are in the, bonds and we have 2 non-bonding pairs.
So then when we attach proton here, we make something called hydronium.
We make hydronium ion.
So it's a little fancier than just saying, I've got this naked proton with no electrons swimming around in water.
In fact, it's more coordinated like this.
H30 plus, and while we're in the neighborhood, we might as well show.
This is sp3 hybridized.
We've got one, two, three, four orbitals.
Hydrogen's on three.
It's got a net charge of plus 1, and lone pair on the fourth side.
So there's the structure of hydronium.
And we can now write the equation.
We can say H20 liquid plus H20 liquid gives me H3O plus, plus OH minus.
So this is now the self-ionization or self-dissociation reaction.
So now I can pair these as well.
I can yoke these.
I have acid-base pair.
So this is acid here, and the conjugate base is water, H20.
Hydroxyl, we know, has to be a base, and its conjugate acid is water.
So you see, in the same equation, same place, same time, water is acting as both acid and base relative to these two species.
And so we call such a such a compound that acts as both acid and base, we call it amphipathic.
Has two moods.
You know, amphi, like in amphitheater.
Have you ever seen a Roman theater?
It looks like this from the top down.
It's got all this, you know, like this, and all the people are here, and there's the stage up here.
But if I make a theater in the round, that is amphitheater.
Or as it's commonly mispronounced, ampitheater.
3091ers do not say ampitheater, they say amphitheater.
So this reaction here is self-dissociation.
Right?
This reaction is self-dissociation of water.
Or because we're forming ions, it's also called self-ionization.
Self-dissociation or self-ionization of water.
Now, turns out that the chemical's power of H3O plus and OH minus is very high.
So this reaction doesn't go very far to the right.
The amount of the dissociation is tiny, and to be specific, at 25 degrees C, the amount of, if you take deionized, deaerated water, absolutely pure and pristine.
The amount of native H30 plus and OH minus is on the order of one part in 10 million.
So H3O plus concentration would be 10 to the minus 7 molar.
And obviously, from the stoichiometry of the equation, you get the identical amount of OH minus.
Well, that's in the self-dissociation.
And so you have very, very few charge carriers, which is why high-purity water is a very poor conductor of electricity, even as an electrolyte.
And that's, I've told you in the past, that could be one indicator of certain toxins.
If you measure the conductivity of water and you discover it's abnormally high, and it's putatively supposed to be pure water, that's an indication that it's not pure water.
There's some other charge carriers present.
So we can write a Ksp we analogy.
And it's called the water ionization constant, which is the product H3O plus and OH minus.
And you can do the math.
That's 10 to the minus 14 is the product.
And you can use the common ion effect here.
If I introduce some other acid, look at how this equation helps you the same way that Ksp did.
If I have acid, in other words, protons donated from some other source, that means this number is going to be higher than 10 to the minus 7.
If this number is higher than 10 to the minus 7, and the product must be 10 to the minus 14, this must be lower.
So when I have, H3O plus goes up, then OH minus must go down, and under these circumstances, we have something that is called proton-rich.
And we call this acid, simply because the proton concentration exceeds the hydroxyl.
And likewise for the other.
And so we can make a plot of this, and that's shown here.
And all we've got is OH versus H3O plus.
And you can see that it's a right hyperbola.
But it goes back to that comment that I made in the past about how you graph data.
I don't know what to do with that, because it's curved.
I can't tell whether the data are good or bad.
So instead of looking at something like this, y versus x, I want to transform so that can get a straight line.
Because then I can make a judgment about goodness of fit and so on.
So I need some f of x versus g of y to, quote unquote, straighten this out.
And we have that, thanks to another Dane, by the name of Sorensen.
Sorensen was the one that gave us another way to think about it.
Sorensen, who in 1901, and he's got the DK on his-- he's probably got a Volvo, too.
Or maybe he's driving a Saab.
So he was a biochemist at the Carlsberg brewery.
Yes, they had biochemists, because they wanted understand the chemistry of beer production.
What a concept.
Having people that understand the technology of the business.
So he was working at the Carlsberg brewery, and he decided to take this acid-base business and turn it into something that's much more readily recognizable.
And so he defined a concept called the chemical potential, which, if you like, is the reactivity, the chemical potential of hydronium.
And he gave it the symbol.
Lowercase p for potential, and uppercase H as, obviously, for the hydrogen ion.
And he said, I'm going to make this a logarithm.
So it's logarithm, and in those days, everybody was using slide rules, so it's log base 10 of the concentration of hydronium.
And being a good engineer, he recognized since this starts off at 10 to the minus 7, the log of a number less than one is going to be negative.
And who wants to deal with negative numbers?
So you put a minus sign here.
That way, pH normally is a positive number.
And now you can see the results of Sorensen's straightening things out for us.
So now you go over a wider range, and you can see, you have nice straight line relationship, and then you can get data on there, and so on.
And this is taken from your book, and it shows the pH values of some common substances.
If you start here at neutral pH 7, you have milk, you have human blood, normal things that you would expect, fluids and so on, are hovering around 7.
If you take some coffee, you're going to go into the acidic region.
You see tomatoes down here at about 4.
Wine, you'll see in the reviews of wines, they'll say it has balanced acidity and so on.
Yeah, it's down around pH 3.
And carbonated soft drinks, some of them get down to 2.
Vinegar is wine that has spoiled.
Vin aigre, which means eager.
The Middle French eager meant impetuous, or tart, or something.
So this is spoiled wine.
And the pH changes.
So by measuring the pH of wine, you can tell if it's changing or not.
There's lemon juice.
Gastric juices in the stomach can get down to 1, pH of 1.
But you want that happening only at one point in the digestive cycle.
If you run around for long periods of time at pH 1, you will ulcerate, in other words, you'll puncture the walls.
And if that's about to happen, you need to neutralize.
So how do you neutralize?
You go up with something that has a high pH.
So you can start with something like Alka-Seltzer, sodium bicarbonate at about 8.4, or milk of magnesia, 10.5.
Why is it milk of magnesia?
From last day, because it's not a solution, it is a suspension.
The magnesia is in suspension.
The magnesium hydroxide.
And that's one of those mandatory shake well before using, because all the magnesia is on the bottom, and you've got this almost clear, colorless liquid on the top.
So you're just drinking water.
Not going to help you if you've got stomach pain, but gastric juice is down here.
If you're really desperate, don't reach for ammonia.
You have to be patient.
They have to work with milk of magnesia.
OK.
So that's the range.
So what else do we have here?
Let's take a look.
So far we've been assuming that when we add acid to the system, we get 1 to 1 dissociation.
So the next message that I want to give you here, is that not all acids are of equal strength.
See, I could look up here and say, well, maybe the reason that the lemon juice is down it at 2 and the gastric juices are at 1 is simply a concentration effect.
How much acid was introduced.
No, there's another explanation for it.
And that is that certain acids don't fully disassociate.
So for example, if we look at 1 molar HCl, hydrochloric acid, you look at 1 molar hydrochloric acid, that goes 100% into solution and gives us 1 molar H30 plus.
So 1 to 1 correspondence between how much hydrochloric acid we introduce, and how much hydroxyl that we generate.
So this is total dissociation of the HCl.
And as a result, we call this a strong acid.
In contrast, let me show you a weak acid.
So a weak acid is going to dissolve, but it doesn't dissociate.
So there's really two steps here.
First you've got to get the stuff in solution, and then once you get it in solution, it has to break apart.
It's possible to go into solution and not break apart.
In which case, you don't have the protons, but you've got the solution.
That's not an acid.
So a weak acid would be, let's, in contrast, look at a weak acid.
A weak acid would be something like acetic acid, which is CH3COOH.
And this is for historical reasons.
Normally, the proton that's going to dissociate is at the front of the formula, but this was written long ago, before people understood this theory.
And that's the way you'll see acetic acid written.
So really, this is the proton, and the CH3COO is really the acetate anion, Ac, acetate anion.
So we're going to put that into solution, like this.
I'm going to put it into water, H20 liquid.
So this is the salt out of solution, and all we're going to do is make it into the aqueous solution of same COOH, and I'm just going to write aq.
Now, if this were strong acid with impunity, I would write whatever the concentration of acetic acid is, I break it apart, and I get the full amount of the proton.
What happens is that this, now plus H20, gives me H3O plus plus the remaining acetate, which is CH3COO minus.
And now, here's where the weakness is manifested.
The degree to which the acetic acid grabs protons from here, and then ends up being protonated, is very, very small.
It turns out that in the case of 1 molar-- I'm going to write acetic acid this way, now.
HAc.
So this thing here is the CH3COO minus, all right?
It turns out that in 1 molar acetic acid, we get only 0.4% reaction.
0.4% reacts and dissociates.
So to give protons, it hangs onto most of its protons.
So we can then represent this in the form of an acid dissociation constant.
And that it looks like this.
K sub a for acid.
So on the right side, what we're going to do is make a mass balance here.
We're going to take the product of the proton, the acetate divided by the undisassociated acetic acid.
So we'll put H3O plus concentration times the acetate concentration divided by the undissociated HAc aqueous.
So I'm trying to represent what the ratio is here.
And instead of being one to one, it's 10 to the minus 5.
Very, very weak.
You get a little bit, but not much.
Now, you could say, well, 10 to the minus 5, this is, just for reference, it's 100 times 10 to the minus 7, which is the Kw.
So the point here is, if you put in acetic acid, you'll end up with 100 times the proton population that you would have had by just self-dissociation.
So it isn't acid.
It is donating protons.
But it's not doing it a lot.
So this is called a weak acid, because it's a poor proton donor.
And so we can then look at a comparison to, say, over here, HCl.
Let's go back here, and we can write the acid dissociation constant for HCl, for the strong acid.
And in that case, Ka over here, which is going to be the H3O plus Cl minus, H3O plus.
In the case of HCl, we're going to get proton and chloride over undissociated HCl dissolved in water.
And that's 10 to the plus 6, which for all intents and purposes is infinity.
You put in 1 molar HCl, you get 1 molar H plus.
And look at the ratio.
10 to the sixth, 10 to the minus 5.
There's a 10 to the eleventh relationship between the weak acid and the strong acid.
And here's the cartoon that shows this strong acid, put in this amount, 100% dissociation, weak acid gives you small amounts of proton, and a very weak acid is imperceptible amount of disassociation.
And here's a table that quantifies it.
So the strong acids up here with the Ka.
This value of Ka has to be greater than 1, because it's saying that we're getting a lot of dissociation, the reactions moving to the right.
And you have these numbers here.
10 to the ninth, 10 to the eighth, 10 to the sixth.
Those are differences without distinction.
if I say the ratio is a million to one, or a billion to one, for all intents and purposes, you have dissociation.
And then here are the week acids.
Look.
10 to the minus 3.
There's phosphoric, hydrofluoric, and so on, all the way down.
There's carbonic acid, 10 to the minus 7.
So there's the list.
And just another cute demonstration of the relationship between bonding and acidity, the acid strength increases from left to right, and the bond strength increases from right to left.
You know, HF is very strong.
It's a hydrogen-bonded.
Very tight HF bond.
But what determines if it's going to be a strong acid or not?
It's how willing HF is to give up its H. But its H is tightly bound.
So oddly enough, in HF, you don't give up much of the H. HF is over here.
It's less than one.
It's a weak acid.
HCl is a pretty strong acid.
If you want to go heavy-duty, HBr and HI are even stronger acids, because they have a weaker hold on the hydrogen within them, so they are much, much more generous proton donors.
So you can see that from there.
OK.
So what's the takeaway message here?
The takeaway message is that equal acid concentration does not mean equal acid strength.
You have to be mediated by the Ka.
Equal acid concentration, which means, how much did you dissolve?
Does not equal acid strength.
And what's the reason?
Because equal acid concentration, this is a function of solubility, whereas this is a function of dissociation.
So many dissolve, few dissociate.
How about that?
There's a nice tagline you can use.
Use that at a party this weekend.
Many dissolve, but few dissociate.
All right.
Now I want to go very, very high.
Big concept, all right?
Last definition of acid-base comes from the United States.
G. N. Lewis.
Same G. N. Lewis who gave us the Lewis cross and dot structures, covalent bonding, didn't stop inventing.
So we're going to put U.S.A., with a Chevrolet out in California.
And what did G. N. Lewis tell us?
He said, I want to extend the Bronsted-Lowry concept.
But you know, let me give you an analogy.
Have you ever stood on a bridge and watched the traffic go?
You see it starts from a stop sign or a traffic light.
Light turns green, the first car pulls away, the second car pulls away, the third car pulls away.
The other way you can look at it is, when the first car pulls away, there's a car vacancy.
And instead of watching the cars go from right to left, watch the car vacancy move from left to right.
And the two are linked.
And point of fact, if you're the fourth car from the traffic light, light turns green and you hit the horn, why?
You can't drive.
You have to drive only into a car vacancy, and it takes time for the car vacancy to get to you.
When the rate of vacancy flux is not matched by the rate of car flux, then we have a collision, and then we exchange papers.
So it's a mass transport problem.
So now what G. N. Lewis said was, instead of looking at this reaction as a proton transfer reaction, and looking at proton attachment, or proton acquisition, he said, why don't we look at this from the other side of the relationship?
And so he said, let's look at something like, for example, we've got NH3 here.
So let's put the N with the 2 electrons.
And we've been talking about this from the perspective of the bond being formed here through the acquisition of the proton.
Lewis said, I can view this from the other perspective, and say, I'm going to view it from the perspective of the nitrogen.
From the perspective of the nitrogen, this reaction represents the donation of the electron pair.
A very high concept.
So this base is not about proton attachment.
It's about donating an electronic pair.
So let's get that down, because that's really good.
So I'm going to say something that's a base looks like this.
This is what a base is.
It's a lone pair.
This is electron pair, very high level base.
And what's the proton give us?
What we're doing, is we're talking about electrons and donation.
So what's this other thing in that same category?
An electron pair can match up with an empty orbital.
That's the only place electron pairs can go.
They go to the empty orbitals.
So electron pair is a base, and this is just a circle around nothing.
OK?
This is nothing.
But it's a special kind of nothing.
This is called a vacant orbital.
And I'm going to call a vacant orbital an acid, in the most general sense.
So now I'm going to take an electron pair plus vacant orbital, and what did they react to give me?
Covalent bond.
You'd expect that from G. N. Lewis, because G. N. Lewis enunciated covalency.
That's why you'd expect it from G. N. Lewis.
So this is, vacant orbital plus electron pair gives covalent bond.
Very high level.
Very high level.
So what is a base?
A base is now not just a proton acceptor.
It's an electron pair donor, and the acid is an electron pair acceptor.
So now there's no chemistry here.
No chemical identities.
Which means, I can even go from-- I don't even have to be in solution.
I don't have to be in a liquid.
This could be a gas and a solid.
Could be a gas plus a gas.
So this is, like, Darwinian.
You know, we've come out of the primordial ooze, and now we can fly, because we can talk about any chemical reaction we want.
Any state of matter.
It's fantastic.
Absolutely fantastic.
So here, let me do one last color drawing.
We want to do this one here.
This is Lewis and the biggest concept.
So we're going to go like this.
Base like this.
Base with its lone pair hanging out plus acid gives us-- three colors.
This is great.
We've got all this colored chalk, it's Friday, what could be better.
All right.
So now we're going to end up with A, B, there it is.
This is the Lewis concept.
Lewis acid-base concept.
In the broadest thing.
Now I'm going to show you an example of how you can use this.
I'm not going to stop here.
OK, let's go.
All right.
So we're going to talk about acid rain from burning of coal.
You know, about half of the electric power in this country comes from burning coal, and coal contains about 1% sulfur.
You can purify it of sulfur, but it takes money to do so.
A ton of coal will give you about 25 million British thermal units, and that's the number in SI units and joules.
So 3 tons of coal will give you 1 megawatt per day.
By the way, it takes about one gram of uranium to do the same thing.
So someday, when you're sitting there as a policymaker, you've got a choice of 3 tons of coal or one gram of uranium, keep this fact in mind.
A 10 megawatt plant burns 30 tons of coal per day, containing a third of a ton of sulphur, which makes 2/3 of a ton of SO2.
And SO2 is a precursor to acid rain.
Now we can reduce that SO2 emissions by reacting SO2 with a lime, CaO, according to this reaction, to make calcium sulfite.
And now I'm going to bring in this.
Lewis.
Oh, here's from your textbook, there's the scrubbers with the calcium oxide, and water missed, and so on, and eventually they trap the SO2.
They do nothing to the CO2.
So all the CO2 is going up.
So here's calcium oxide, which is a solid.
It's lime.
You know, stuff you sprinkle on football fields.
And this is SO2, and it's got this structure with the resident bond double single.
The oxygen here acts the way it did in the glasses, as a modifier.
It goes in and breaks this double bond, and now we have three single bonds.
So we can look at this from Lewis acid-base.
You know, calcium oxides, electron pair donor.
If it's an electron pair donor, it's a base.
So it's a base.
And you know SO2 better be an acid, or your theory is nuts, because SO2 is a precursor to acid rain.
So it has to be an acid.
It's an electronic pair acceptor, which is what we said.
It's an electron pair acceptor.
So this is a Lewis acid-base reaction that is a gas-solid reaction.
So we started with aqueous solutions, and we end up generalizing all of these types of reactions.
So that's pretty good.
Here's a plot of acid concentration up in the sky, here.
You're Here.
Yeah.
Last thing I'll do, I talked a lot about Lavoisier and Scheele and Priestley.
This is a play, if you've got the little bit of time this weekend.
You might try reading it.
This is Carl Djerassi and Roald Hoffman, both Nobel Prize winners in Chemistry.
Not the same year.
They won independent Nobel Prizes, and actually speak to each other, and they collaborated on this play.
Here's the setup, the premise.
The premise of the play is that the Nobel committee decides to give Nobel Prizes retrograde.
Before 1901.
You see, you can't get a Nobel Prize posthumously.
You've got to do something great, and you've got to keep on living until you get the prize.
So they decided they're going to go backwards, you know, maybe give prizes to people like, I don't know, Maxwell or Faraday or whatever.
And they decide, it should be simple in the old days.
Because we didn't have all this dog-eat-dog competition in science, and big grants, and corporate interests.
Should be simple.
Until they hit the Nobel Prize for the discovery of oxygen.
And then they hit the wall with these three different competitors.
And so the story goes that the three competitors and their wives end up in Stockholm, and the interactions the take place.
And it's really fascinating, because you have Lavoisier, who was the political conservative but the chemical radical, Priestley, who was the chemical radical but the political conservative-- by the way, Lavoisier.
You know how he died?
He was guillotined during the French revolution.
And people think it's because he was a tax collector.
Not true.
He was a tax collector.
The real reason is Marat, who was one of the chief justices of the Jacobin terror, was also a scientist.
And around 1740, Marat had submitted an article for publication, and it was rejected.
And Lavoisier was on the editorial board.
And so sometimes when I get an article to review, and I have to give it a negative review, I think about this story.
And sometimes, I will just say, you know, I'm really busy teaching 3091 and I won't get to this on a timely basis.
OK, people.
Have a nice weekend.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Couple of announcements.
First of all, the Wolf Lecture is tomorrow, right here in 10-250 at 4:00.
As also, there will be the weekly quiz, the mini celebration.
But there's going to be, a week from today, celebration part trois.
Yes, third celebration of learning.
And remember, what we're shooting for is the smiley face.
None of this.
And those of you who are over here on the grade point distribution curve, we want to move you to the right.
So I'll say a little bit more later in the week about what the coverage will be.
I'll be available for office hours later, and I thought to try to cheer you up, and remind you that studying is not something you do only the night before the exam.
You have to be studying on a regular basis.
All right?
So from the iconic film Ghostbusters, there's this one scene.
[FILM PLAYBACK] For a moment, pretend that I don't know anything about metallurgy, engineering, or physics, and just tell me what the hell is going on.
You never studied.
[END FILM PLAYBACK] You don't want to be a character in that scene.
So what are you going to do about it?
You're going to study.
All right.
Let's get down to work.
Today I'm going to take one day and I'm going to talk about organic chemistry.
Now, I'm not going to pretend that after one lecture, you're going to walk out of here and know much organic chemistry.
I'm going to give you, if you want to know organic chemistry, you're going to have to take 512, and many if you will.
But I'm going to give you enough that it will set up for the subsequent units on polymers and on biochemistry.
So we'll know enough that we can move forward.
So let's get into the definition of organic chemistry.
It's the chemistry of compounds, containing both carbon and hydrogen.
There can be, of course, other elements present, but you've got to have carbon and hydrogen to get yourself into Orgo.
And what makes carbon and hydrogen so special?
We've been studying these elements through the semester.
But just to be reminded.
First of all, hydrogen is peculiar because it's got the lowest atomic number.
It's got the lone proton, and a single electron.
And when hydrogen ionizes, it forms the hydrogen ion, or the proton.
That's all that's left.
And this thing's very tiny, and it can form covalent bonds.
It forms covalent bonds as primary bonds, and it can form hydrogen bonds as secondary bonds.
So it's peculiar for that reason.
It does not form ions.
You cannot imagine something is tiny as a proton occupying a lattice site in an ionic crystal.
OK. Now when it comes to carbon, carbon is also peculiar.
It's smack dab in the middle of the periodic table, in the second row.
Four valence electrons, which means that it's very difficult to ionize.
If it wants to achieve octet stability, it's going to have to form covalent compounds.
It's not going to acquire four electrons or lose four electronics.
In rare exceptional circumstances it, might, but commonly, it does not.
And with that intermediate average valence electron energy, it's going to be capable of all sorts of bonding.
And in particular, because of its small size, it's capable, with its small size and the intermediate value of average valence electron energy-- and this is critical-- is that it's capable of forming multiple bonds.
And this is key, as we'll see later when things try to polymerize and form multiple bonds.
And this is not common, for example, not the case if you look underneath carbon.
Silicon, germanium, for example.
Same thing.
Group 4, same kind of mid-stream average valence electron energy.
But they're too big.
Silicon and Germanium do not form double and triple bonds.
So the other thing is that it's capable of self-linking.
It can link to itself, form carbon chains, and also with nitrogen, oxygen, sulfur.
So this gives it-- and let's put up phosphorus as well.
Phosphorus and sulfur.
So with these kinds of links-- we've already seen oxygen links.
We'll see, pretty soon, nitrogen links and so on.
There's millions of organic compounds, and today I'm just going to look at a few.
I'm just going to give you some taxonomy and nomenclature, and we're going to walk through this.
And here's the way to keep it straight.
You know all this stuff.
I'm just going to organize it in your minds for you.
We've already studied the three different types of hybridization.
sp, sp2, and sp3.
And we've already seen that if you're going to have a double bond, you need sp2 hybridization to reserve 1p orbital, to make that pi second bond.
And if you want to make triple bonds, you make sp hybridization, thereby reserving two p orbitals to make two pi bonds.
So this is sigma, sigma plus pi, sigma plus pi plus pi.
That's it.
Now what we're going to do is to is go back and look at this in the context of hydrocarbons.
So hydrocarbons is going to be now the focus here, and the hydrocarbons are the simplest of the organic compounds.
So let's look at hydrocarbons.
I'm going to look at their nomenclature and their properties.
But the structure behind everything, the treatment, is the type of hybridization.
So these consist only of hydrogen and carbon.
Compounds containing only hydrogen and carbon.
So I'm just going to go down that chart.
And we'll go through.
So the first one on the left is the alkanes We'll look at the alkanes.
And the alkanes are characterized by sp3 hybridization.
And so this gives the maximum number of carbon linkages.
Why?
Because you can only form single bonds.
So if I can only form single bonds, every bond goes to a different atom.
Whereas if I form double bonds, then I'm burning up some of the capability.
Because if I used two bonds to get to one neighboring element, instead of two single bonds each to two different neighboring elements, I don't have as many linkages.
So the maximum number of carbon linkages-- and because they have the maximum number, we're going to borrow terminology from the last unit on solutions.
What's the maximum amount of solute you can get into a solution called?
It's called the saturation value.
And they use that term here.
These are called saturated hydrocarbons, because they've got the maximum number of linkages, bonds.
And they're all sigma bonds.
All of this is a consequence of the choice of sp3.
I could have led you through all of this, but we're putting it up quickly here.
All right.
And so they have a common formula.
CNH2N plus 2.
I think I've got some-- yeah, this is taking from one of the other books.
I know you've got a list like this in your own book.
So here they are.
So this is just CNH2N plus 2, and you go down, and there's the nomenclature.
The meth, for historical reasons represents n equals 1.
So here's what the hidden message is.
You name them by the carbon number plus the suffix A N E. So the -anes, the alk-anes, are all these sp3.
So 1 is meth, 2 is eth, 3 is pro, and 4 is but.
And they all have historical derivations.
But- comes from butyric acid, which is the acid that's formed when butter turns rancid, and so on.
After you get up to N equals 4, it's just your Latin.
So you just take your high school Latin, pentane, hexane, heptane, and so on.
And if you didn't take high school Latin, this is a great chance to learn the ordinals in Latin.
So there they are.
Now, I'm not going to expect you to-- if I say decane, you're supposed to slam down C10H22.
But I would expect that I could say decane, and I'd give you that formula and you'd be comfortable with it.
What's the other thing to note here, while we've got this up on the chart.
You can see that all of these are symmetric molecules, and they're non-polar.
They're perfectly symmetric non-polar.
And what happens as the molecular mass increases, the melting point increases, the boiling point increases, and the state of matter transitions from gas at room temperature to liquid at room temperature to solid at room temperature.
So here's an excellent example of polarizability in action.
Because the only way one methane can bond to another methane is by induced dipole-induced dipole interaction.
So the only difference between methane and icosane is that icosane is a honking big molecule, non-polar, but very polarizable.
And so as the polarizability increases, you can see the effect here.
OK, so that's good.
So you get to learn your Latin ordinals.
Now these are also termed straight chains.
They can be termed straight chains, but I want you to look carefully at what's going on here.
So here's a diagram used, the first one on the left, you've seen methane many times.
There's ethane and then propane.
And what do you notice here?
Because of the sp3 hybridization, all of these angles are 109 degrees, including the carbon-carbon angles.
And when the thing is really short, you get this zig-zaggy nature.
So because of the sp3 hybridization, you see the carbon doing this.
This is carbon-carbon, and then the rest, forget it.
I mean, this is the backbone.
But if I get up to icosane and I have about 20 of these things, forget the zig-zag.
From here, yeah, I know there's a little bit of fine structure, but for all intents and purposes, this as a chain.
Straight chain.
And that's where you get the terminology from.
But keep in mind that there is this zig-zagging back and forth, due to the sp3 hybridization.
OK?
So that's the first point.
The second point is that the bonds between the carbons are not fully specified.
So for example, take the carbon here on the left.
The sticks to go to the 3 hydrogens here depicted in white, and then the fourth stick goes to the carbon.
And then off of that carbon comes 3 more sticks, and all of these are 109 degree angles.
All of that specified, and nothing more.
There's nothing saying that these 2 hydrogens in the back have to line up.
These two hydrogens out front have to the line up.
That's all free to rotate.
And so we have that degree of freedom there.
The result is that you can have something that zig-zags back and forth.
So for example, this is called eclipse, because if I were to stand here, these two hydrogens are on a direct line.
So the front hydrogen covers the back hydrogen.
It eclipses.
This hydrogen out front to your and my right is on the same line is this hydrogen to your and my left.
And so if I were to look here from the end, this hydrogen eclipses, and so on.
So if I get something eclipsed, I'm going to have a perfect straight line.
If I have something that's staggered, it's free to zig-zag, but it's free to zig-zag and meander, and this is what happens.
So here's 2 C17H36 molecules, and the top confirmation is more or less straight.
These are more or less eclipsed configurations.
Whereas the lower one, you can see, there's a fair bit of stagger, and as a result, the backbone can actually loop back on itself.
But in both cases, we still call these straight chains.
They're still straight chain.
Because as you start at one end and go from carbon to carbon, you linearly go to the other end.
It doesn't mean it's straight like straight as an arrow.
It just means there's no branches.
So now I'm going to distinguish this from the other one by branching.
So let's just get this up here.
So we have straight chains, is one case.
Straight chains.
And the straight chains can either be staggered or eclipsed.
And the closer you get to eclipsed, the closer you get to-- the more eclipsed, the straighter.
I think you'll understand what this means.
Staggered allows it to make large loops-- I'll just put here, straight but looped.
You can use that to describe some of your friends, too.
All right.
Now-- it's a Monday, it's a joke!
Wake up!
Wake up!
All right.
Now we can look at another one.
Branched chains.
And now, this will be a major departure, whether the chain is super-straight or does a lot of looping back and forth, the branched chain is in marked contrast.
Now let me give you an example of the branch chain.
Best to just do it.
So here's the branch chain.
I'm going to use a butane.
So you know B U T is 4.
So that means 4 carbons, and A means 4 carbons, sp3 hybridized.
So that's it.
So now I'm going to write this.
Here's butane.
C, C, C, C. And I'm going to write this as just a straight chain, instead of doing this.
I'm going to contrast.
I'm going to do 1, 2, 3, 4.
1, 2, 3, 4.
1, 2, 3, 4.
1, 2, 3, 4.
OK?
So there we are.
So this is, instead of writing all this stuff, this is butane, and this is butane.
But now here's the thing.
Four stick rule.
Whenever you're doing Orgo, four stick rule.
That means, four sticks out of every carbon.
So here's the carbon-carbon bond, so I need three more sticks.
1, 2, 3.
And I know these are sp3 hybridized, but just, in compressed notation, you can do this.
And I'm not even going to write all the hydrogens.
Here I'm going to write the hydrogens because it's our first time through, and it's Monday, and everybody needs a little bit of TLC.
But now the TLC is gone.
This is it, all right?
I'm not even going to put the H's on the end.
The stick coming off of carbon means there's a hydrogen here.
All right?
So how about this one?
1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4.
And now let's count the hydrogens.
1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
C4H10.
Butane.
All right.
Now there's another way to write this.
I could go like so.
1, 2, 3.
And I could put a carbon up here.
This is a branch, you see?
It's not in a straight line.
And I'm not talking about eclipsed.
Because if I start at 1n and go down the backbone, I get to the other end, I only hit three.
I never got to this one.
But let's see.
You know, maybe this is different.
So four stick rule.
1, 2, 3, 4.
1, 2, 3, 4.
1, 2, 3, 4.
1, 2, 3, 4.
Now let's count the hydrogens.
1, 2, 3, 4, 4, 6, 7, 8, 9, 10.
This is C4H10.
It's a different molecule, you can see.
With what you've already learned in 3091, this has a different dipole moment, doesn't it?
It's got a different dipole moment.
It's going to have a different polarizability.
But it's a butane, it's a C4H10.
So this is called butane, right?
And this one, over here, historically, was called isobutane.
Because it's an isomer.
It's got the same chemical composition, but it's got a different makeup.
It's got a different structural makeup.
It's got a different-- I'm going to use the material science-y word.
Different structure.
It's got a different molecular structure.
But the chemists going back 50 years ago didn't use these terms.
They called it different constituents.
So I'm going to use that as well.
Different molecular structure or different constituents.
And what do we mean by constituents?
Now comes the color chalk.
So we're going to do, is I want to circle the makeup of this.
So at this end, I've got-- this is a CH3 here, correct?
There's a CH3 over here.
And then in between, I've got CH2 and a CH2.
Now let's go over here and diagram this one.
The isobutane, I've got a CH3 at the end, and a CH3 at the opposite end.
So far, same as the butane.
But now look.
There's a third CH3 here, and then there's this thing, which is a CH.
You could argue it's a CH2 that got methylated, but, you know.
So I'm going to put a CH2 here that's been methylated.
And I'll tell you what that means.
OK?
So it's really a CH2 group that got methylated.
So it's called isobutane, but you can see, it's got different-- all I'm doing here is making the justification for the different constitution, different dielectric constants, and so on and so forth.
So this is where the chemists must come in.
They would say, different constituents.
And so these are isomers that have the same chemical composition, and are different on the basis of their constitution.
So they call these constitutional isomers.
This is a constitutional isomer.
Isobutane is a constitutional isomer.
Now the last piece is nomenclature.
They don't use this today.
Because isobutane was a term that came out of chemical industry.
Because it looks like butane, has got the same molecular weight, but it's very different.
Today we use the IUPAC notation.
Remember this?
International Union of Pure and Applied Chemistry.
They're the ones, when one of you discovers and stabilizes element 113, And you want to name it after dadadadada, you have to get past the IUPAC committee.
OK?
So their nomenclature committee would say, this is an alkane, but it's only three carbon units long.
So it is not a butane.
It is a propane.
It is a type of propane.
But it's not simple propane, because there's a methyl hanging off the side.
So in point of fact, this, in IUPAC nomenclature, is a methyl propane.
And furthermore, if you want to get really pedantic, and IUPAC loves to get pedantic, is what they'll do, is they'll number these, starting from left to right.
So this is the number 1 carbon, this is the number 2 carbon, and this is number 3 carbon.
And the methyl group is pendent on the number 2 carbon.
So if we want to get really, really, really pedantic, and who wouldn't, on a Monday morning, what we would do, is we would call this 2-methylpropane.
So it's a propane, 3 carbon units, there's a methyl group hanging off of number 2.
Now, I don't expect you to be able to do these on sight, but I would like you to be appreciative of what this is.
So I'll give it to you on an exam.
All right.
So we've got the isomers.
And I think I've got some-- I think I've got this.
No.
OK, this is good.
Oh.
There's one other thing I want to talk about with alkanes.
There's another thing that we need to know, and that's about the radicals.
The radicals that form.
The radical is a species with one or more unpaired electrons.
Right?
This is broken bond.
One pair electrons.
That's equivalent to broken bond.
OK.
So this is highly reactive.
Because this unpaired electron is going to go look for a mate.
So if you take a look at CH4-- now I'm going to put the hydrogens in.
So this is methane, CH4.
I'm going to break this bond in the lower right corner, and just have an unpaired electron.
And this is called the methyl radical.
Very reactive.
So you can see that this methyl radical could go up and stick on to the fourth strut of that number 2 carbon, and thereby form the carbon-carbon bond.
So that's why they call this methylpropane.
So the methyl radical.
And you can write it this way, H3C with dot indicating unpaired electrons.
But I want you to be literate with the way chemists will write this in various compact notations.
So you might even see it written this way.
Now, that doesn't mean that the unpaired electron is on a hydrogen.
This is just a way of saying, you know, CH4 is methane, CH3 is methyl, and who cares.
There's only one possible meaning to this, so it's all good.
All right?
Let's do two unpaired electrons.
So I'm going to take two unpaired electrons.
Again, this is out of the alkane, so I'll put a hydrogen above and below.
So that would be like this one here.
You see the CH2?
So it's got carbon-carbon bonds on either side, but in point of fact, this is part of a zig-zag.
So I'm going to break bonds here and here.
All right?
So now I've got the possibility of putting a bond on either side.
And this one here is called methylene.
This is methylene.
This is the methylene radical, and it's got two unpaired electrons.
So we could write that as H2C with 2 unpaired electrons.
And, you know, these are electrons.
They're not just anywhere.
They're sitting in an orbital, and that orbital is as at 109 degrees from the other bond.
So it has to be sitting there in reserve, in waiting.
So if I bring another carbon here, I get the carbon-carbon bond, and I start forming the zig-zag.
And there's one more, a common one that we'll need to know.
And that's from ethane, which is C2H6, right?
Let's start with ethane, C2H6, and I'm going to make a radical out of that.
It will be C2H5, like this.
To which I can add a hydroxyl, and now I'll make ethyl alcohol, and then it's party time.
But so this is how you start.
All right?
This is the ethyl radical.
Starting from methane.
And that's pretty much it.
That's about all I want you to really take away with the alkanes.
C4H10.
Yeah, there you go.
So there you can see, to the left, the zig-zagging of the linear and the butane, and then the methyl propane.
That's good.
OK. Now let's go to sp2.
Now we're going to look at the alkenes.
So alkenes represent-- the alkenes-- this is sp2 hybridization.
And they contain-- this means there's the possibility of a carbon-carbon double bond.
And they're also known as unsaturated.
Because they don't optimize.
The fact that you spent two of your bonds going between two atoms instead of taking one bond to one carbon and another bond somewhere else, means that you're undermaximized.
So unsaturated hydrocarbons.
OK?
And the formula for these is CNH2N, where obviously N is greater than 1.
We can't form a carbon-carton double bond with only one carbon.
So you know, the prototypical one we looked at was this one here, just to C2.
That's the first one.
So it's going to be C2H4.
So carbon-carbon double bond.
There's a sigma bond here, and a pi bond.
And they lie in a plane.
The molecule is planar.
120 degrees here.
So we'll put the hydrogens on it-- you know already, I don't have to put the hydrogens there.
That's good enough.
All right.
So what's this one called?
This is called ethylene, historically.
Ethylene.
But the IUPAC notation is similar to this one.
It's carbon number plus -ene, E N E. So strictly speaking, this is what everybody knows.
But IUPAC insists-- and I don't know.
As you publish, some editors are really fastidious, and they'll force you to change if you use this kind of terminology.
So you're going to have to call it ethene.
Or in the UK they call it ethene.
But everybody knows it is ethylene.
OK.
So there it is.
And you only need 1 carbon-carbon bond.
So you can have long chains where the position isn't fixed.
So let's look at a couple of examples.
So let's look at butene.
OK, let me just write that down.
That's good.
The position of carbon-carbon is not fixed.
It just has to be somewhere in the molecule, which consists only of hydrogen and carbon So here's a couple of examples.
I'll start with this one.
This is going to be-- so this is 1, 2, 3, 4, so that's a but- something.
And it's got a double bond, so that's a butene.
It's all hydrocarbon.
So that's butene here.
But I could also-- well, let's finish this.
How many?
This is 1, 2, 3, 4.
See, I even tried to indicate the 120 degrees here.
This is going to be 1, 2, 3, 4.
1, 2, 3, 4, 1, 2, 3, 4.
All right.
So there's the butene.
But there's another place I could put the carbon-carbon double bond.
I could do it this way.
Make a carbon-carbon single bond, move the carbon-carbon double bond over to the number 2 position.
this is 1, 2, 3, 4.
So this is going to be 1-butene, and this one here is going to be 1, 2.
So we'll make this 2-butene.
And let's keep counting here.
1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4.
And I've proven to you, I hope, by now that I can count to four.
So this is 2-butene, right?
2-butene.
And if you go through this using the same kind of diagrammatic analysis as over here, you're going to come to the conclusion that even though these both have the same composition, which is going to be C4H8, they're both C4H8, but as you diagram them, you'll find that they have different constituents.
So these, in fact, are constitutional isomers of one another.
Constitutional isomers.
Yeah, that's pretty good.
There's even more.
It gets even better.
I want to focus on the bottom one, the 2-butene, and I want to write it out with a little bit more detail.
So let's start with that carbon-carbon double bond.
And we know the carbon-carbon double bond requires planar structures at 120 degrees from one another.
So let's look at that one.
And what we've got here, is on the carbon, I've got on the one side, I've got a hydrogen strut, right?
And on the other side, I've got a methyl.
I've put a methyl here, like that.
OK?
Or actually, let's get-- OK, I'll do that.
I was-- what I wanted to do, to get really-- OK, watch this.
All right.
So what I'm going to do on this side, this side, same thing.
The carbon has a hydrogen strut down here, and on the other side, you've got the methyl.
OK?
So that looks good.
But there's another way to do this, and still keep the carbon-carbon bond in the center.
So I'm going to do that over here.
So 1, 2, 1, 2.
And in this instance, I'll put the same thing I have on the left side, H3C.
So so far, so good.
I've got the same molecule.
But now what I'm going to do, just for grins and chuckles, I'm going to put the hydrogen in the upper position, and the methyl in the lower position.
Now, these are not constitutional isomers.
They have the same constitution, right?
Carbon-carbon double bond, two methyl groups, two hydrogens.
Carbon-carbon double bond, two methyl groups, two hydrogens.
So they're constitutionally identical, yet these have very different properties.
Can you see?
I mean, the dipole moment is going to be different.
Look.
Here's the charge, all up on one side, and here the charge is more symmetrically straddled.
Which means, they're going to have different melting points.
They're going to have different boiling points.
And they have the same chemical formula, and they have the same chemical constitution.
So they're different kinds.
There's some kind of isomers-- what's the difference here?
It's not the constitution, it's the way things are arranged in space!
This is great material science.
Spatial orientation is everything.
And what do you call it when you're listening to some loudspeakers and a really good audio system, and you can close your eyes, and the violins are over here, and the cello is here, and you can hear the percussion way in the back.
What is that called?
Stereophonic sound.
Stereo phonic, meaning you can render spacial resolution.
So these are called stereoisomers.
One's a stereoisomer of the other.
So they're spacially distinguishable.
So they have-- for example, I went and looked this up.
Oh.
Let's give some names to this.
In this case, all the methyls are on the same side, and the hydrogens are on the same side.
So I can a dividing plane here, if you like, parallel to the floor.
And I'll just cut right through the center of the carbons, parallel to the floor.
And so in one instance, all the methyls are on the same side, and here the methyls are on opposite sides of the double bond.
So when they're on the same side, this is called cis-butene.
And let's be super pedantic.
It was a 2-butene, so now it's a cis-2-butene.
This is terrific.
And this is a 2-butene, but it's on opposite side.
So this is called a trans-2-butene.
Cis-2-butene and trans-2-butene.
And I looked up-- so let's see, which one do you think has tighter, which one's going to be more tightly packed?
Which one's going to have the greater density?
Which one's going to pack better?
Cis or trans?
The density here is 0.627, and the density here is 0.61.
OK?
Now, which one's going to have the higher boiling point?
Which one's going to have the higher boiling point?
Gee, that should be a no-brainer.
Once you know which one's got the higher density.
Why does it have the higher density?
Because it's more nearest neighbors is more tightly packed.
So I would bet on the one with the higher density.
And this one boils at, boiling point is 3.7 degrees Celsius, and this one, the boiling point is-- hang on, I've got this backwards.
This is 3.7.
Boiling point is 1 degree Celsius.
So those are the data as they come up.
So stereoisomer, spacially distinguishable.
By the way, we can form multiple bonds.
We can form multiple double bonds.
So they're -enes as well.
So if we have two double bonds in the molecule, so we'll call that a diene.
And we can have-- you can keep going with the Latin, but common ones you might find is the triene.
And this will come back later when we look at polymers.
So if we have two bonds, we will have-- You know, some of that hard luggage that you're seeing that's coming back, they got the roller boards, but instead of the fabric, now they're coming back with the really hard polymer?
That's ABS, and the b is butadiene.
So let's look at a-- here's a pentadiene.
I'll give you a pentadiene.
So that's going to be 1, 2, 3, 4, 5.
It's five carbons, for starters.
And I'm going to put the double bonds on the 1 and the 3 carbon.
It'll be a penta-, it's five carbons.
And it's an -ene, because there's going to be some double bonds here.
And it's a di-, because I'm going to put double bonds at the 1 and the 3.
So 1 gets a double bond, and 3 gets a double bond.
So this is a pentadiene.
OK?
And then the last thing is-- pardon me.
Last thing is radicals.
So here's the common radical.
We'll start with ethylene, and I'm going to make an ethylene radical here.
I'll break off the hydrogen at this point, and I can write that as CH2CH.
And we can't call this ethyl radical, because ethyl was already used for C2H5.
So we need to indicate that there's a double bond, and so this radical is known as vinyl.
This is the vinyl radical.
And so you could react this with chlorine, for example, in which case you'd make vinyl chloride, and then you'd polymerize that, and you make PVC.
And we're going to see that in the next coming days.
Or you could you can put alcohol on there.
I've already told you that.
I don't need to remind you twice about that, I bet.
OK.
So now-- oh, there's one other one.
There's one other one that will come up if you take a lot of biochemistry, and that's the one that comes off of propene.
So if you start with propene, propene's going to look like this.
1, 2, 3.
So there's propene.
3.
There's a saying double bond here, and I'm going to put 1, 2, 3 in here.
1, 2, 3, 1, 2, 3.
1, 2, 3, 4.
OK.
So this is the radical that comes from propene.
So this one is derived from ethylene, and this one here is derived from propylene.
So again, as in this case, we couldn't call this ethyl.
We can't call this propyl, because that's the one that you get from the alkane.
So instead, this one is called allyl.
A L L Y L. And if you take 7012 at some point, Professor Weinberg will refer to these pendant groups.
He likes to pronounce this all-YL.
You'll hear him talk about, and the all-YLS are over here and here.
That's what we're talking about.
OK, good.
So this is C3H7 dot.
Good.
All right.
Moving right along.
Now I want to turn to the alkynes.
That's the sp hybridization.
sp.
And that means there's going to be at least one carbon-carbon triple bond.
One carbon-carbon triple bond.
So the formula there is CNH2N minus 2, and obviously N has to be greater than 1, otherwise you can't form a bond.
And there's very little that you need to remember about this one.
The dominant one that you'll come upon is C2H2.
So according IUPAC nomenclature, the two carbons means it has to be F, and these are all alkynes, so this should be ethyne.
And if you ask for ethyne, nobody knows what you're talking about, although it's very correct.
This is known as acetylene, which is a fuel gas for welding, cutting, torches and so on.
You've got two lines, you've got a big tank of acetylene, you've got a big tank of oxygen.
And this is used because there is enormous energy stored in this carbon-carbon double bond.
And so when you combust this with the stoichiometric amount of oxygen, you can cut through steel, you can use this to make very elaborate glassware with pure silica, making fused quartz glassware.
Very, very high energy contained in here.
And just to give one more example, let's look at something that's got four carbons in it.
So it's going to have four carbons and a triple bond.
So if it's 4, it's going to have the B U T but-, and it's going to need an -yne.
So this will be a butyne.
And depending on where we want to put the triple bond, I don't know, if you want to put a triple bond here, there's 1, 2, 3, 4.
So this we could call 2-butyne, and it's C4H6, If you go through the stoichiometry.
Let's try it, just for practice.
So this carbon on the end is 1, 2, 3, 4.
Look at the second carbon.
It's got one strut bonding to the neighboring carbon, and three struts bonding to the neighboring carbon on the right.
It's saturated now.
Nothing comes of without carbon.
Look at this one.
1, 2, 3, 4.
Nothing comes off of this one.
And then finally 1, 2, 3.
So there's three hydrogens here, there's three hydrogens here, there's your sixth hydrogen.
C4H6.
OK. That's good.
All right.
There's one other set that I want to look at, and these are called the arynes.
These are aromatic hydrocarbons.
Some people refer to them as arynes.
And the most notable one is C6H6, which is benzine.
And benzine's molecular weight was determined long ago, in the early 1800s.
And it was Kekule who, in 1865, dared to suggest that the chemical structure of benzine was a ring structure.
So Kekule suggested a carbon ring.
Up until this time, people didn't have the common understanding that carbons could link to themselves, let alone form a ring.
And so now, if we go through this, we will put, from each of these junctions there's the carbon, so we have to have four struts.
We need hydrogen, so let's put hydrogens 1, 2, 3, 4, 5, 6, So there's the 6 hydrogens.
And now, each carbon needs 4 struts.
So 1, 2, 3, and I'll put a double bond.
That will make 4.
1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, and now this one here.
So you have an alternating structure of double bond, single bond, double bond, single bond on the carbons.
OK?
So that's the way things stood until the twentieth century, when on the basis of new data, it was discovered that all carbon-carbon bonds-- and I don't want to make this a single bond, I'll just say all carbon-carbon bonds in benzine, C6H6, were measured to be the same length.
Well, that's a problem, isn't it?
That's a problem, because we would expect that, you know, we know that carbon-carbon single bond is about 1.47 angstroms, and the carbon-carbon double bond has to be shorter, because it's double-bond, it's tighter, so it pulls things in.
So that's 1.33 angstroms.
And the measurement here was found to be 1.39 angstroms.
And 1.39 angstroms is midway between 1.47 and 1.33.
So people were frustrated with this new information.
And it was Linus Pauling who made sense of this for us.
Pauling, who in 1931 came along and said, what we have a mix?
What if we have a mix of two structures?
And he said, suppose you had something that looked like so.
1, 1, 1.
So here's the set of double bonds off of the ring.
And what if there is a resonance with an opposite hybrid, where this double bond, instead of being here, goes over to here?
So now we have this?
And he said, if the system resonated between the structure on the left and the structure on the right, the time average value of the carbon-carbon bond length would be around 1.39 angstroms.
So this is a resonant hybrid structure, to account for the mix of single and double bonds, which gives rise to the current representation of benzine as simply this.
So we're not saying this is single or double.
It's a resonant between one and two.
OK.
The other thing is, if all of these bonds are the same length, and that means that between the 12:00 position here and the 2:00 position, there is pi bonding, and then along this double bond, there's pi bonding, and then along this double bond, there's pi bonding.
But if all of the bonds are the same length, I can't say that the pi bonds are confined to where the double bonds are indicated, because resonance indicates that it could be between adjacents.
So that means that the electrons, in fact, can go all around.
So resonance, then, means that the pi electrons are delocalized.
And now here's some better drawings.
It's hard to draw that.
So here they are.
OK.
So this is a nicer drawing of benzine.
So you can see, when all of these bonds are resonant, it doesn't matter whether I choose this front one as the double bond, or the one next to it.
They're all the same length, and so the electrons can go all around.
And you can imagine, if I did this with graphite, and I kept going, well, if the electron can go here, then it means it can go here, which means it can go here, which means it can go everywhere.
Which is why graphite is an electronic conductor, because of delocalized pi bonds.
OK.
This is 1-3-butadiene.
Double bond, single bond, double bond.
Again, all the same length, and you can hybridize, and-- This is how you start making electronically conducted polymers.
You just go double bond, single bond.
Double bond, single bond.
Smear.
That's it.
OK. Let's jump over that.
Let's talk about Kekule So he studied architecture, switched to chemistry.
And then he got a job in London, and he used to fall asleep on the bus to his apartment.
And one day he was sleeping on the bus.
He woke up, and he'd been dreaming about carbon forming chains in 1855.
And he proposed carbon chains.
And then he got a job as a professor in Belgium, at the University of Ghent.
And one night he fell asleep at the fireplace, and he dreamt of a benzine molecule as a snake biting its tail while spinning.
And that's where he got the idea of the ring molecule.
All he knew was that this thing had a formula, C6H6.
And for this, he's dubbed the founder of structural chemistry.
So what's this formula for success?
Well, he moved into chemistry from another field.
But to dream, you've got to get some sleep.
And I'm probably talking to one of the most sleep-deprived populations on this campus.
So I urge you to get some sleep, and then maybe you'll be able to dream.
Hey, don't make noise.
We've got two minutes left.
nobody's going anywhere.
All right.
So the next thing is, I want to talk a little bit about octane ratings and automobiles, and how we go from straight chains to chains of different lengths.
If you're looking at combustion, the idea is, you admit gasoline, which vaporizes, and then the piston comes up, and under high compression, the spark plug fires.
When the spark plug fires, this causes an explosion which then pushes the piston down, and then you've got the camshaft that takes the vertical motion and converts it to rotary motion.
Now, if you're running an automobile after a certain period of time, the combustion chamber becomes so hot that just admitting the fuel, having it vaporized, before the piston has risen fully to get maximum compression, this thing could just explode.
And when it explodes, it'll send the piston down, and it's out of sequence with the other pistons.
And that's when you get the knocking.
Sometimes you're driving up hill, you hear this [GRINDING SOUND].
That's the knocking going on.
You need to get a tune-up, or switch to higher octane gasoline.
So what's going on in here?
What you're doing, is you're changing the chemical composition, the constitutional isomerization.
And you do that in the process by which you synthesize the gasoline in the first place.
So the figure of merit is called octane number, which was first instituted here in the United States.
And they started with 2-2-4-trimethyl-pentane.
It's an octane, but it's an octane that's 2-2-4.
Three methyl branches.
So that means there's only five left.
So it's a pentane with three methyl branches.
And that's called 100.
And then is 0 is heptane.
If you put heptane into the combustion chamber, you'll knock yourself silly.
And the engine will just shake and shake and shake.
So gasolines have to be somewhere on that interval.
So there's the-- you know, if you tested and you had something that was trimetylpentane, 90%, versus 10% heptane, you'd call it a 90% octane.
And you can have octane numbers higher than 100, by the way.
It just depends on what the repression is.
And oddly enough, when you when you increase the octane, you make the fuel more difficult to burn.
Because you want it to burn only on demand.
Fuel that just burns whenever it wants leads to random events in the engine, and that's not good, if you want to get good thrust.
So oddly enough, high octane gasoline requires more ignition than low octane.
And the additives are tetraethyl lead or ethyl alcohol.
And from that, you know now, we get E10, which is gasohol, 10% alcohol.
E85 is 85% ethyl alcohol.
And all of this you get by synthesis.
You know, playing with the catalysts, temperatures, and so on, to get the right mix to get good fuel, and get good emissions.
And you can decide amongst yourselves whether going to E85 high levels of ethanol, ethanol derived from corn, is that smart, is that food for fuel?
You know, what's the sensible use of agriculture, algae, switchgrass?
All of that stuff.
I mean, it's all loaded into here.
All loaded into here.
So yeah.
With that, I think we'll adjourn.
I'll see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We had a staff meeting yesterday, and we decided that what we're going to do is, we'll start with defects and solids.
We didn't examine you on it with the last celebration.
So we'll pick it up with defects, and we'll go through to the end of solutions, acid-bases.
So the stuff that we covered on Monday, the nomenclature of organics, we're going to leave out.
Because you don't have enough time with the recitations to really dig into that, and I think it's-- with the test being on a Monday this time, and not on a Wednesday, the schedule is little bit of out of balance from the way I'd like it to be.
But I thought having the third celebration the Wednesday before Thanksgiving was not wise.
I know that a number of you are going to be in transit that on Wednesday.
But let me remind you that Wednesday, we will have a full-blown lecture.
I'm going without any slow down.
So next Wednesday, we will start biochemistry.
So those of you who are not here, please make sure that you have viewed the lecture.
Because when you return on Monday, you will have no idea what's going on.
And it'd be a good idea not to blow biochemistry, because on the final exam-- if you fail the final, it's not good.
It's really not good.
So just to be clear, we're going to start with defects and solids, and go through, and end with acids, bases, and solution chemistry.
OK.
So today I want to start to get some mileage out of what we learned on Monday.
I want to talk today about one of the applications of organic chemistry, and that's polymers.
So we're going to take two lectures on polymers, and then we'll feed into biochemistry, and you'll see a very natural progression from what we're doing now into the biochemistry.
So let's begin with a little bit of reflection on what we've done so far.
Up until now, we've looked at various solid structures.
We've started with single atoms.
We looked at crystals, disordered crystals, and so on.
We've looked at compounds, and we've even looked at small change.
We looked at alkanes, straight chain alkanes, branched alkanes.
And we've looked at network solids.
Here is a regular network.
This is the structure of graphite, for example.
Nice, ordered crystalline structure.
Diamond grows in three dimensions without abatement.
We've looked at disordered networks, such as silicate glasses.
So that's what we've looked at up until now.
What I want to talk about today is polymers.
Polymers are macromolecules.
These are long chain molecules.
By long, we're talking about thousands and thousands of repeat units.
And this is the distinguishing feature between macromolecules that are found in the body, versus macromolecules that are man-made.
Mother Nature is a polymer engineer.
We are a macromolecular organism.
This is all polymer.
It's all polymer.
The changes in shape is all elasticity in a polymer.
And inside, of course, I've got a ceramic skeleton, which keeps the frame in place.
But this is all polymers.
But the difference here is, this is not a repeat unit.
Whereas when we have a man-made structure, it's the same repeating chemical structure.
And the other thing I want to say at the beginning is their importance in commerce.
Polymers are found everywhere today.
Trash bags, auto parts.
And in culture, they're absolutely essential to culture.
Let's think about it.
When you play that DVD, that DVD is information embedded in a polymer.
The magnetic drags.
What are magnetic drags?
These are polymers that have been coated with gammaferic oxide or some other magnetic material.
Without polymers we couldn't have the modern era.
And if you go back before the DVD, with the CD, before that was magnetic tape and cassette, and before that was this.
You might see some of these around.
This is an old way of presenting musical information.
This is the phonograph record, photograph recording.
And this is information that is scribed into a platter, and it's made of polyvinyl chloride.
That's why you hear the term vinyl.
Why is Virgin, the big Virgin enterprise, called Virgin?
Because Virgin Records, when they made their platters, they started with virgin vinyl, not recycled polymer.
And so their platters would lie flat on the turntable.
Whereas some of them you buy, you put them down, and they warp.
It's OK, because the needle would still track, but it looked kind of-- playing like this.
And it's too bad that we've lost this.
Because this gives rise to the real estate possibility for artwork, and liner notes, and so on, that you don't get when you download something for $0.99.
OK?
So an electromechanical device moves along here, and jiggles in the grooves, and those mechanical wigglings get converted into energy that ultimately gets fed through the loudspeakers.
And now let's talk about visual information.
Daguerre invented halide photography in the early 1800s, but we didn't get motion pictures.
Cinematography had to wait for the advent of polymers, because only when we could make material that would come out miles long on which we could coat with halide photosensitive emulsion, could we imagine images flashing past us faster than the persistence of vision.
You need-- what's the persistence of vision?
Around a twentieth of a second.
Motion pictures are 24 frames a second.
So you couldn't have glass plates on which you've got halide film zipping past at 24 frames a second for 90 minutes.
So only with the advent of polymers did we have the birth of cinematography.
So what I'm talking about here changed the world.
It changed the world, and gives birth to the fantastic access that we have to information.
And now, of course, as things move digitally, and they dematerialize, you're going to see a shift away.
But all this stuff was enabled by the advent of polymer chemistry.
So with that as a motivator, let's dive in.
This is good stuff.
It's really, really good.
So now the payback comes.
You know, all those little lessons about, where's the electron, where's the orbital, you know, who wants to learn that stuff.
You have to learn it!
If you're going to write the great novel, you've got to learn how to spell.
All right.
So let's go.
So the polymer, it comes from the Greek poly, meaning may, right?
Poly is many And the mer, the mer is this repeat unit, if you like.
Many units, many units.
So let's take a simple example.
Last day we studied polyethylene.
So polyethylene looks like this.
Double bond, one, two, three, four, four hydrogens.
So this is just ethylene.
And what I can do with ethylene, is I can react it.
Ethylene's a gas, room temperature.
Put it into a reactor and expose it to an initiator.
And the initiator is a radical.
You know what the radical.
It's something that's got this unpaired electron.
It's very active.
And the radical sees a unit of ethylene, which I'm going to write like this.
We don't need to put the 120 degrees now.
And what the ethylene does, is this radical attacks this double bond and figures, well, it's better to have two single bonds than one double bond.
Reacts the double bond, breaks it, and then forms the following.
The radical now bonds to the ethylene, breaks that double bond, and transfers the unpaired electron to the end.
But now this thing is itself a radical, and it's swimming in a gas of ethylene.
So now a second C2H4 comes up against this, and it's turned into the following, are now C C C C C, one, two, three, four, one, two, three, four.
And so on.
And this can go without limit, until we shut down the reactor, or we introduce something that terminates, that will cap this.
All right?
So this is the beginning of it.
And what you see is attachment, attachment, attachment, breaking of double bonds.
And so we have a repeat unit here.
This repeat unit, the mer unit, is this ethylene unit, and it can go to a value of n that is very, very high.
What kind of numbers can we expect to get?
We can have numbers here where n takes values-- and these aren't hard and fast, but just to give you some idea-- you could have a, you know the oxymoron jumbo shrimp.
You could have a short chain polymer.
A short chain, long chain.
You know?
Boy, you're really dead today.
What's the matter with you?
Sleep deprived?
You've-- All right.
So n can go up to about 10,000.
Now what's the atomic mass of this, just as representative?
This is 2 times 12 is 24, 25, or 26, 27, 28.
28's roughly 30, so 30 times 10,000, you've got 300,000.
That's 300,000 grams per mole.
This means you can have molecular weights on the order of a million grams per mole for one molecule.
And by the way, the polymer people, they come from a different branch of science.
So they don't use this grams per mole.
Instead of grams per mole, they like to use the word, the dalton, as a unit of measure.
So you might see in the pollymer literature, you'll see the polymer with so many kDa, kilodaltons, thousands of grams per mole.
That's the nomenclature you will see.
OK. Now, what are the properties of these things going to be?
Oh wait, I'm going to show you something.
To give you a sense.
You need to be awakened.
So let me give you a sense of just how long these molecules are.
So I'm going to give you a little mechanical model here.
I'm going to put up here so we get some-- So what I've got here is just a pull chain.
If you go in the basement, you might see these pull chains.
They're just made of brass, little brass balls, and then they've got links between them.
And when you cut them-- you buy them by the foot at the hardware store.
You put it on a light, you can turn the light on and off by toggling up and down.
So I'm going to say, let's say this distance here represents the distance between mer units.
This distance could be between 2 mer units.
And I figured, let's see, I've got 40 feet here, and I did the math.
So this thing here, with the 40 foot pull chain, has a polymerization index-- let's call this the polymerization index.
So my polymerization index, I calculated it for this thing.
It's on the order of about 3,000.
So that's pretty good.
It's right in the middle of this.
It's actually-- this qualifies as a bona fide polymer.
So let's take a look at what happens here.
This is one molecule.
So what do you know about interactions between molecules?
Now, in the liquid state, what's this going to be?
Is it going to be fluid, or is it going to be viscous?
It's going to be viscous.
Look, this is one molecule, and it entangles with itself.
Now, to make a liquid, I have thousands of these things swimming around, above their crystallization-- oh, crystallization temperature.
Freudian slip.
Above their solidification temperature.
When they solidify, are they going to form an ordered solid?
One of these at each lattice site?
Probably not.
They're probably going to form disordered solids.
So most of the polymers that we encounter are probably going to be disordered.
How is this held together when it forms a solid?
It's polyethylene.
Well, what's your menu?
Is it ionic bonding?
Is it metallic bonding?
Is it hydrogen bonding?
How is it held together?
Weak van der Waals.
Weak van der Waals bonds.
But look at the surface area.
So this is what it looks like.
See, and it entangles, is even entangles with my clothing.
It just grabs onto everything.
All right.
Actually, why don't we-- David, let's cut to the screen.
So here we are.
This is it.
This is the polymer.
And every once in a while, a polymer will do something like this.
It'll say, you know, I still, even though I'm in this big long chain molecule I still want to try to order.
Because I read in 3091 that ordering lowers the free energy of the system...
So what it does, it starts doing this.
And can you see when it starts doing this, that it starts to take on-- if you walked into the room just now, and this were blowing up really high, you might see just this little snippet and say, wow.
This thing is starting to look like a cubic array isn't it?
This is a really good model.
It's a really good model, because this is what really happens.
And so that decreases the energy of the system.
The bonds are greater there.
So what's that going to do to the mechanical properties?
Going to strengthen it, yeah.
It's going to make things stiffer.
Instead of being this soft, squishy polymer, it's going to have some stiffness to it.
And you've come up across that stuff.
The most notable one being the CD case.
The people that make those, I'd like to bring them here and sit them down and teach them some polymer science.
Those things are so brittle.
They crack, right?
There's only two classes of jewel cases.
Those that are cracked, and those that will crack.
Because these people don't know what they're doing.
OK.
So now let's have some time here to codify what we've just see.
So first of all, we suspect that they're going to be solid at room temperature.
Dominantly.
We can engineer them to be liquid, but dominantly solid at room temperature.
And van der Waals bonds abundant.
OK. And the liquid, as a liquid, we expect viscous liquid.
Good.
Got that down.
What else to have to know?
Let's do calculation here.
So I did one here where I said, what's this distance?
We saw last day that this distance, carbon-carbon, remember, we're looking at single bond, double bond.
A single bond carbon-carbon is about 1.5 angstroms.
So this is 1.5 angstroms.
The distance between successive mer units is about 3 angstroms.
And so if you take something 3 angstroms, and you make it this number of mer units, I came up with a polymerization index of 3571 when this thing weighs 10,010 kilodaltons.
And so then that means that you'd end up with a length, molecular length is on the order of 10,714 angstroms, which is on the order of 1 micrometer, which then makes it greater than the wavelength of visible light.
So this is really a different type of matter.
And we can go there.
We've talked about viscous, so I'm going to remind you of two observations here.
Viscous liquids, amorphous solids.
If you put those two ideas together, what else can we call upon?
We can go back to this.
Yeah.
Remember this?
What's this?
This is super-cool liquid, super-cool viscous liquid.
This is the solidification temperature, which in the case of liquid to amorphous solid is the glass transition temperature.
And here we have the excess volume, right?
This is the volume, and then there's some crystalline volume, and so on.
Right?
I can cool at a second rate, and I get this.
So this is slow cool, this is fast cool.
We have a Tg up here.
Tgf I'll call Tg fast, and Tgs, Tg slow.
So that'll give me a different volume.
So this is volume fast and volume slow.
All right?
And since I know that density is equal to mass over volume, so therefore v fast is greater than-- I observe v fast is greater than v slow, so therefore density of the fast cool must be less than the density of the slow cool.
Now I'm going to put some names on here.
I'm going to call this the cooling curves for polyethylene.
These are the cooling curves for polyethylene.
So this, down here, gives me-- this is now high-density polyethylene, and up here is low-density polyethylene.
Same thing, just different processing.
So faster cooling quenches in more free volume.
And since volume is a measure of disorder, right?
What's the difference between great disorder and not-so-great disorder?
David, again cut to the projector, please?
Forgive me, the document camera?
So the difference between high disorder and low disorder is this.
This is the order, here.
So what that tells me is that in high-density polyethylene, there is a greater percentage of zones like this.
And we just reasoned that this is going to give stiffness and so on, and sure enough, for the low-density polyethylene, low-density polyethylene is used in things like food wrap, things like stretch and seal, where you can pull because you can move those macromolecules relative to one another without fracturing the material to stretch it over.
Whereas the high-density polyethylene is used in such things as milk jugs, where you want a little bit of stiffness.
Polyethylene milk jugs are still kind of floppy, but there's a little bit of stiffness to it.
The other thing is, because the low-density polyethylene is dominantly amorphous, with almost none of this second zone here, it's transparent to visible light.
But now, let's think about what's going on here.
Where I've got this chain, and then all of a sudden I get to this.
All right?
So this is clear and colorless.
Right?
It's a high band gap material.
There's no free electrons.
It's going to be transparent to invisible light.
So this is transparent to visible light all along.
But can you see that because I have this zone of ordering, the density of matter here is different, and so therefore, when a photon comes in, the index of refraction-- now, you know, this is one of those days where n is going to come up.
So here n is the polymerization index.
Here it's index of refraction.
So maybe we'll use a different color.
How about that.
All right?
So the green n is index of refraction.
So the index of refraction in the amorphous zone, is different from the index of refraction in the partially crystalline zone.
So index of refraction varies from zone to zone.
Even though each zone is clear and colorless, can you see that this boundary between the ordered and in the disordered zone acts like an interface?
And what happens when you have an interface with a different index of refraction?
It scatters light.
And as a result, the milk jugs, they appear white.
You can't see through them, even though they're constituted of the same continuous material.
But there's these density fluctuations, because this is a higher density than this.
So you can rationalize all of this stuff.
OK.
So this is what we would call partial crystallization in this zone.
And that gives rise to the changes.
Now, how would we distinguish these?
What technique would I use to see if I've got order in this polymer, that I don't know anything about?
How do I interrogate atomic order?
What technique would I use?
X-ray defraction.
Thank you.
Back to the slides, please, David.
I'll find you the piece.
This is called polyethylene, but remember last day, the IUPAC notation for this is ethyne, so this becomes polythene.
And in the UK, it's known as polythene, and there's a Beatles song, Polythene Pam.
The only reason-- I put up here for two reasons.
One is, it has something to do with polymers.
At least they knew something about polymers.
It's apparently, they must have been in a drug haze at this period in their careers.
This is absolutely terrible.
Some of your parents probably adore the Beatles.
If you want to provoke a conversation at Thanksgiving, pull out this one and ask them to talk about the lyrics here.
But this is just garbage.
This is garbage!
This should be trash bag, is what it should be.
But evidently, this woman used to show up at parties dressed only in a polythene bag, a see-through polythene bag, I might add.
So this is a paean to her.
Anyways, it's bad.
Yeah, yeah, yeah.
Anyway.
So here we are.
This indicates chrystalline polyethylene.
Here you can see the C2H4 unit.
And it's attempting to try to occupy, as these beads line up here, they're trying to set up a faux lattice.
And this is what you might see.
And towards the end of the lecture, I'll show you a transmission electron micrograph where you can see this by dying it with different colors.
OK.
So now here's some x-ray defraction.
So this is crystalline, where they zoomed in on one of these zones, and they've just gotten the defraction pattern from that zone.
And you see Bragg peaks.
And then this is in the amorphous region.
it's not featureless.
There's one broad peak.
Why one broad peak?
Because even though there's no long-range order, you have short-range order.
You know that you've got a carbon on either side, you've got hydrogens and so on.
So that short range order gives you the broad peak.
But look at C here.
Isn't this interesting?
That if you have a polymer that has both some order and a lot of disorder, if you take the x-ray defraction pattern more broadly, more globally, you get the additive spectrum.
So you can see that this is amorphous, but there are some zones of crystallinity.
And that's the additivity power of x-ray defraction that allows us to interrogate.
So now let's talk in a little more fine structure about molecular architecture.
So tailoring molecular architecture.
And why are we doing this?
Because we want to engineer these materials for desirable properties.
Like CD cases that break after about one or two uses, OK?
Of polymers.
Actually, there's probably a business opportunity there.
You start a company where you make jewel cases that actually work.
People might be willing to pay, you know, a penny more for something that works.
All right.
So how do we change this?
What I'm going to show is a whole bunch of cartoons of architecture.
But how do we control this?
It's processing.
It's processing.
Processing and in the synthesis algorithm.
I'm assuming I already have polyethylene, so how am I going to tailor it?
It's in the processing.
And so what I can do, is I've got a number of levers I can move.
One is the composition of the polymer, and the second one is, I can use catalysis.
I already hinted at that last day in talking about gasoline.
How do you start with petroleum and get octane and not heptane and so on?
By playing with cataylsis, we can direct certain forms.
You can actually preferably synthesize a certain architecture by using catalysts.
So let's look at these variables.
First one is composition, obviously.
If you change the composition of something, you can very much expect to change its properties.
So what do we have here?
We can-- here's a simple example I can start here with.
This is ethylene.
Or I can put a chlorine here, so this now becomes chloride, right?
This will become, what's the radical, is vinyl, so this is vinyl chloride.
So if I polymerize this, this becomes polyethylene.
If I polymerize this, this becomes polyvinyl chloride, PVC.
Which is why you heard the fellows from Blue Man Group at the beginning banging on PVC tubing to make their music.
All right.
So I use a pure, in other words, only one mer.
It's pure polyethylene.
This it's called a homopolymer.
Only one mer type in use.
If I want to make the polymer analogy of an alloy-- in ally metals, alloy has more than one metal mixed in, or a metal can even have nonmetals mixed in.
Then the polymer term is called copolymer.
I'll call this a copolymer.
And a copolymer has greater than one mer type.
So for example here, this could just be polyethylene.
Whereas here, an example would be polyethylene, and then the notation is hyphen lowercase c hyphen, which is an indication of the fact that you're making a copolymer.
And this copolymer has mer units of ethylene, and mer units of vinyl chloride.
So this is a copolymer of polyethylene and polyvinyl chloride.
And then we can start looking at the various ways of arranging these.
So we can start with, for example, we can have a sequence of random mer types.
So as you're going down the chain, you get either an ethylene, or a vinyl chloride.
So this is called a random copolymer, and it's designated polyethylene-r-polyvinyl chloride.
And actually, I think I've got a cartoon showing this.
Yeah.
So A and B are different mer types.
And so a random copolymer just has A, B, B, A, A, A, B, A, whatever.
And this is controlled in the synthesis process.
So I can take the same mix of A and B, in other words, the same two mer types, but mix them in an alternating sequence.
So they're very regular.
It's 1 mer of ethylene, and then one mer of vinyl chloride, alternating all the way down the backbone.
So this gives you a regular copolymer, and it's designated A for alternating, because we've already used R for random.
So instead of a sequence of random, we have a sequence of alternating mer types.
So that would be, in this case, polyethylene alternating PVC.
So that's two of them.
And then you see the block copolymer shown.
And in that case, the mers grouped into-- they call them blocks, even though it's a line.
You know, maybe it's like walking down the street.
I've walked down one block.
So this is one block, and then there's the next block.
They're grouped into blocks.
So in that case, we have the block copolymer, lowercase b, and then PVC.
And what you see there is a run.
A run of-- So all of the A's are grouped, and then we have all of the B's.
And this is actually an artist's misconception.
In point of fact, most block copolymers that you find, this is the stuff they use for the soles of your sneakers and so on.
They're block copolymers.
They're usually just a diblock.
There's usually just two different mer types.
One half of the macromolecule is one mer type, the other half of the macromolecule is the other mer type.
Sometimes you might find a triblock, where you might find block A, block B, and then another block A.
But this is a pentablock and he's even got ellipses here as though this thing keeps going.
There's no commercial product that looks like that.
It's usually a diblock, occasionally a triblock.
OK?
And then the last thing you can do, as is shown up here, is what is known as the graft.
And in the graft, the thing that distinguishes the graft is mers grouped into blocks.
In this case, a long side chain of other mer.
So that's different from side group that we saw last day.
These are long macromolecular chains.
So you see here the primary backbone is A, and then you have this very, very long macromolecular chain of B.
And this is called the graft copolymer.
So that would be polyethylene, could be the backbone, and then long side chains of polyvinyl chloride.
And all of these have their different architectures.
And I think I have an example here of one.
This one, I think I mentioned last day, when we were looking at the butadiene.
So this is hard-sided luggage.
And if you go to grandma's house, and she hasn't gotten into the digital age, and isn't an old hipster with a cell phone, she's still got the old hard plastic telephone, it's made of this stuff.
ABS, which is acrylonitrile butadiene styrene.
And the backbone is butadiene, and then you have-- OK, so here's the backbone.
It's long butadiene, which we saw last day.
And then you've got side chains of two types.
And this is just indicating A, A, but this is a macromolecule.
This might be 3,000 units long, whereas this is 10,000 units long.
And this might be 2,000 units long.
So you have side chains of acrylonitrile and side chains of polystyrene.
And that's the hard plastic.
It's coming back, because people want luggage that doesn't destroy their contents anymore.
So that's coming back.
OK.
So this is what we can do in terms of composition.
Let's look at, also, the side group arrangement.
And this is called tacticity.
Tacticity, which is the equivalent of what we saw last day as stereo isomerism.
Remember, I showed you the cis and trans on the butadiene.
On the cis, you've got, in one case, you've got whatever the functional group is.
I'm going to put A, A, B, B.
And then so this is the cis version, or I can do a trans version, which is instead, I'll put in A up here and an A down here.
I'll put a B up here and a B down here.
So this is a trans version.
So imagine the analogy for polymers.
So let's look at that.
It's easier to see it, and then we can just document it.
So here's three different polymers, all right?
So I'm going to start with the lowest one.
This is called isotactic, because this is vinylchloride.
There's vinyl chloride by itself.
There's the vinyl radical from ethylene.
And then we tack on chlorine.
And now when we polymerize, we're going to break this double bond in order to link this carbon to the neighbor.
So now the backbone only has single bonds, right?
You start with the double bonded precursor, and now you've got this chain of single bonds.
Right?
But look at the chlorine.
The chlorine could either be below, or it could be above.
Well, in this case, the chlorine is always below the chain.
So this is called isotactic polyvinyl chloride.
Now this one, you know, they're trying to mix two things at once.
I would have shown this with vinyl chloride in all three, but OK.
So imagine now, in this case it's syndiotactic.
This happens to be polystyrene, because this is vinyl benzine, or what's the other way to call it, phenyl ethylene.
You can have either way.
All right?
So you can say you're putting a phenyl group onto vinyl, or you're putting onto the ethylene, or vice versa.
But the radical, this thing here is called styrene, and we're going to break that double bond and away we go.
So this is the benzine ring, and it alternates from above the chain to below the chain.
Above the chain to below the chain.
So in this, this is called syndiotactic.
And the top one is a polypropylene, so you start with propylene, that has the double bond here.
It's got three carbons and the methyl group coming out, and so you break that double bond and make the chain.
And it seems to be on a random basis where that methyl group appears.
Sometimes above the chain, sometimes below the chain.
So those are three different ways of arranging, so three different tacticities.
So we've got isotactic, we've got syndiotactic, and we've got atactic.
So the way to remember them, isotactic obviously means, everything's on the same side.
And when I'm talking about the same side, what is it?
Same side of the backbone.
And we're talking about the position of functional groups, right?
That's what this is all about.
The positioning of functional groups.
In other words, a methyl, or a benzine, or what have you.
So same side of backbone.
Atactic is random, and then by elimination, this one must mean alternating, above and below.
On opposite sides of the chain.
And these have different propensities for crystallization.
Which of these three-- imagine that all three of them were polyvinyl chloride.
Which of the three, atactic, syndiotactic, isotactic, would be most favorable from the standpoint of partial crystallization, to loop back and forth?
The one that's most regular, so the isotactic one has.
OK. Third one.
Third one is the backbone configuration.
This is the configuration of the main chain, all right?
And this is called conformality.
You see, they have different words for everything that came from a different heritage.
So to me, it answers the question, how distended is the chain?
And we saw this last day, when we looked at what happens when we start with something like this, where we have staggered or eclipsed, remember, orientation of the hydrogens here.
In the case of staggered, we get the lower.
Both of these are straight chains, because if you start at one end, you move monotonically all the way down the chain to the other end.
There's no branching here.
But one of them, the lower one, has staggering of that carbon-carbon bond, with the result, things form this giant loop.
So in polymers, imagine this, instead of being 36 units long, imagine it being 3,600 units long.
So now you can have things to do this.
You can even take a little break here and crystallize and so on.
So how do I distinguish that from something that does this?
So how distended is the chain?
This one here is as distended as it can be, and others can be much more coiled.
So this one is called linear chain.
This is branch chain.
Now, this is not graft.
Don't confuse this with the graft copolymer.
Graft copolymer means, I have one mer down the backbone, and a second mer off to the side.
That's a graft copolymer.
This is a homopolymer.
A homopolymer that has different branches.
So all of these, let's put this here to remind us.
These are homopolymer cartoons.
Homopolymers that haven't changed anything.
So this is something that actually has different branches.
So this is branched chain, branch chain architecture.
Good.
So which one of these is going to be harder to crystallize.
Which one is-- well, obviously, I've given it away, I put that little piece of crystallinity in there.
Can you see that when these two solidify, that the one on the right, because it's got these branched chains with the covalent bonds sticking out.
It's not going to pack as well.
And if it doesn't pack as well, it's going to have a higher free volume.
If it's got a higher free volume, it's got a higher degree of disorder.
So the branched chains have a greater propensity for disorder.
So let's put that down.
Branched chain, harder to crystallize.
And crystallize, it's not a giant crystal or polycrystal.
We're talking about the degree to which we can get that amount of ordering.
And then the last one I want to show you is, here's three different chains.
1, 2, and 3.
And what we're going to do is we're going to cross-link.
We're going to form bridges.
These are covalent bridges.
These are not hydrogen bonds.
These are not weak van der Walls bonds.
These are strong covalent bonds, all the way along here, linking backbone to backbone.
So this is cross-linked.
And what do you think the mechanical properties of this are going to be?
If I want extend this, like stretch and seal, I can pull chain number three relative to chain number one quite easily, until this covalent bond has bent over as far as it can, and then what happens?
I can't pull it anymore.
And what happens when I let it go?
It springs back.
So this imparts elasticity.
This makes the polymer rubbery.
Cross-linked polymers are rubbery.
But you know, the polymer people, they want elevated words.
If you say to a polymer person, oh, so you've made a rubber, they cringe.
They want to have a fancy word.
They call this an elastimer.
It's got the word mer in it, so they're happy, and it's elastic, so it's an elastimer.
Rubbery.
So how are we going to make these cross-links?
What are we going to have to look for as an architectural feature to make cross-links?
We look at the backbone.
If we've got a backbone going like this, all these carbon-carbon bonds, and down here I've got carbon-carbon bonds.
If I break one of these bonds in order to go up in this direction, I've broken the chain.
So how am I going to have the bonding capability to form a covalent bridge between chains without breaking the very chain I'm trying to link?
What feature am I going to have to have in the chain?
I need, after polymerization, to still have some double bonds.
And if I've got still double bonds in the backbone after polymerization-- and now would I do that?
I have to have a double bond to break in the first place.
So what would happen if I started with a unit that had two double bonds in it?
That way, I give up one double bond in order to make the chain, and I still have a second double bond.
That's why the rubber has butadiene.
The diene has two double bonds, after polymerization, still has one double bond, and now what I can do is break this double bond and this double bond, and link them.
But if I link them, they're going to be too close together.
They're going to be scrunched in.
I want to have some play, here.
I want to make this rubbery.
So what do I do?
I put a spacer in here.
What do I use as a spacer?
An atom.
And what kind of an atom do I need?
I need an atom that's capable of making one, two covalent bonds.
What am I going to choose?
What atom do you know likes to make linkages?
How did we make silicate?
What's the linker in silicate?
Oxygen.
You could use oxygen, but I want to make this thing even farther apart, and a little bit, you know, oxygen's small and it's got too much strength.
If I want to make it weaker, but something that behaved like oxygen, sulfur, I'd go down one row in the periodic table.
So I'd put a sulfur in here.
And if I wanted to do a really good job, I'll put a sulfur here, since I got a double bond from both sides, I'll put two sulfurs, and make a disulfide linkage, and now I've got the making of rubber.
Disulfide linkage.
And again, the feature, I have to start with the backbone that has a covalent bond at the end.
OK. Well, let me tell you a little bit about the birth of rubber.
It started here in Massachusetts.
It started just across the river, in Roxbury.
Nathaniel Hayward discovered that rubber treated with sulfur was not sticky.
If you take natural rubber-- you ever work with natural rubber?
It's very sticky.
You can't do anything with it.
And Hayward reasoned that by playing with the sulfur, he could-- he didn't understand the molecular architecture.
But he learned that by playing with sulfur, and introducing sulfur to the rubber, he lost the stickiness and he got an enhanced elasticity.
So Charles Goodyear came to Boston from Ohio to meet with Hayward, and he learned about disulfide linkage and so on.
Took the idea back to Ohio.
And one day in the laboratory, he was trying to prepare a batch of sulfonated rubber.
And it was a laboratory accident.
He knocked over the vessel containing this sulfonated rubber, and it landed on a hot stove.
Things were powered by fire.
Landed on a hot stove, and then by heating it, he gave birth to the process of vulcanization.
And hence was born the American rubber tire industry, by that accident, starting with the trip to Roxbury to learn about this.
And you could have done it all with what you learned today.
You're just about 150 years too late.
OK.
I will see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Perfect timing.
Let's turn to the announcements.
You know the big announcement is Monday, Celebration Part three.
I think Hilary sent all of you the coverage.
Just to be clear, start with defects, amorphous solids, kinetics, diffusions, solutions, acids, and bases.
No orgo, no polymers.
We'll catch the orgo and the polymers on the celebration of celebrations, the final exam.
OK, so let's get moving.
Last day we started talking about polymers.
These are macromolecules.
Here we've got an example of a simple compound with a double bond, vinyl chloride.
And by the action of addition polymerization, we can turn this into a macromolecule where you see a repeat unit.
The double bond has been broken to allow but carbon to bond to another vinyl chloride, and so on.
These are very, very large molecules, can be on the order of microns in length, mass measured in kilodaltons, or thousands of grams per mole.
So today I want to go back and look a little bit at polymer synthesis.
Then we're going to talk about properties of polymers, and end with some cultural implications that I alluded to the very beginning.
So we've already seen how this reaction begins, but let's see how it proceeds.
So here's an example of starting with an initiation reaction that generates this hydroxyl radical that starts from a peroxide.
And then you can see, in this first instance, the unpaired electron attacks the double bond, breaks the double bond links, and then moves that unpaired electron to the end, and so on, and so on.
So here I'm indicating that we're going to keep attaching, in this case, ethylenes to make polyethylene.
And then at some point, we decide that we've got the molecule long enough, and then we have to quench the reaction.
So then we add something that will cap this unpaired electron, and this step is called termination.
So this is how we make the long-chain molecules by this particular reaction.
And the thing to know, trivially, is that the composition of the polymer equals the composition of the mer.
You can see it right here, that I start with vinyl chloride.
Yeah, we've changed the double bond to a single bond, but it's still C2H3Cl, C2H3Cl taken many times.
That's it.
OK.
There's another way to make polymers.
And this is really important, not only in commerce, but it's most important in biology.
We do not have the biological apparatus to create radicals.
So during the course of this lecture, we have cell death proceeding in all of us, and cell generation and polymerization reactions occurring, amino acids being linked to form proteins and whatnot.
And that process goes by the second mechanism, condensation polymerization.
And I'll give you a simple example of that where we start with more than one mer, OK?
Involves more than one mer type.
Typically two, all right?
So let's look at a reaction.
So this is going to be a first kind of a mer, and R is just a general placeholder the people use in organic chemistry.
It comes from the word residual, but it's some giant piece of molecular mass, and there's a hydrogen here.
And this one then reacts with a second mer, which I'm going to designate R2.
And the second mer has a hydroxyl.
And what happens in condensation polymerization, is that we link R1 to R2.
R1 is now linked to R2 in a covalent manner.
And then, we take the hydrogen and the hydroxyl, and we send them over here.
So the hydrogen goes here, the hydroxyl goes here, to form water.
And this is typically conducted, in industry, not in our bodies, but in industry, it's typically conducted at elevated temperatures where this eventually goes into some kind of a condenser.
And then the water condenses and makes liquid.
And for this step, the condensation step of the byproduct of the reaction that we name the process.
The process isn't named for what happens here.
It's named for what happens over here.
So this is condensation polymerization, and you can see, we have covalent linkages at both ends, and we can add another R1H here, and then another R2OH here, and so on, and so on, and so on.
So clearly, we're losing some of the mass of the reactants here.
So the mass of the final polymer is less than the sum of the masses of all of the reagents.
So let's make a note of that.
That the mass of the polymer formed by condensation polymerization is less than the sum of the masses of the constituents.
Or let's say reactants, if we like.
Reactants.
Because we lose the condensate.
And so here's an example from commerce.
Oh, before I get to that, I wanted to show you some of the common addition polymers, which ones that you're familiar with that are made by this first process.
So at the top, we have polyethylene.
That's we started with.
And you know that's used in food wrap, it's used in milk jugs, and so on.
There's the polytetrafluoroethylene.
You remove the hydrogens, you put fluorines, and then you make Teflon.
So that's also made by addition.
Polypropylene, also used as separator.
In fact, polypropylene is found in every lithium ion battery.
It's the separator between the two electrodes, and it's engineered so that its glass transition temperature is about 60, 70 degrees Celsius.
So that if there's some event, and you start getting thermal runaway in your battery, and the temperature rises to whatever, let's say the number is 70 degrees C, the pores in the polypropylene collapse.
And when they collapse, they form a barrier, and it kills the battery, and prevents the battery from exploding.
And that's polypropylene.
There's butyl rubber.
What else do we have?
Polystyrene, you know, which is used in these cups and food containers.
Orlon, here.
Polyvinyl chloride, which is used in everything from old vinyl phonograph records to plumbing supplies.
That white PVC tubing that's used for piping.
Polymethyl methacrylate, this is plexiglass, which many of you are wearing in your eyeglasses.
That's this material here.
Butadiene, OK. You see there's the after-polymerization.
There's still one unused double bond, so then we can mix this later on for cross-linking.
And the various rubbers, and so on.
You can see, they all have residual double bonds, which allows them to then participate in disulfide linkages.
OK.
So now here's the example I wanted to show you that you will be familiar with in commerce.
This is the PET, the polyethylene terephthalate, the plastic bottles.
The two-liter soda bottles are made of this stuff.
So you start with this thing, which is known industrially as terephthalic acid, which is really 1-4-benzene.
You see the benzene ring here, so one, two, three, four.
So off the one position and the four position, you have a carboxylic acid, and this is critical in proteins.
You're going to see the COOH structure over and over again.
And that H is detachable.
So this, then, it becomes a proton donor.
If it's a proton donor, it's an acid.
So this is the 1-4-benzenedicarboxylic acid.
And this is ethylene glycol, which you know as the base constituent of antifreeze for automobiles.
It's really ethane with two alcohols.
So it's a diol.
Ethanediol, 1-2.
And you mix the ethanediol with the terephthalic acid, spit off the proton and the hydroxyl, leaving behind this, which then forms the bridge here, and there's the basis for the plastic soda bottle.
This is the same material from which you can make Dacron, which is a fiber used sometimes in textiles, and Mylar, which is used in sheet films for polymer films.
So this is the PET.
And note here, you see this oxygen is acting as a bridge.
It's acting as a bridge between the terephthalic acid residual and the ethylene glycol residual.
So again, we see nature relying on the fact that oxygen forms those two bonds and acts as a bridge for us.
OK. A couple of other things to note.
Here are the typical polymers that are made by condensation polymerization.
All of the nylons.
The nylons are made by condensation polymerization.
And the bond here that exists between the nitrogen and the carbon is called the amide bond.
Even though it's spelled A M I D E, it's pronounced amid.
So these are all nylons.
The generic term is polyamides.
And you see the different types of nylon here.
Kevlar, used as a substitute for steel belting in tires, derives its strength from this, the benzene ring here.
And polyesters, OK?
The Dacron, the Mylar, I just showed you that.
Lexan.
This is used in everything from sports equipment, if you wear eyeglasses in sports, they're probably made of Lexan.
Again, look at these twin benzene rings, and then you've got the linkage on the end here.
And also used when you're sitting seven miles above the surface of the earth.
The only thing that prevents you from being asphyxiated or frozen to death in that airplane is a window made of Lexan.
And lastly, here are the silicones, which are also made by condensation polymerization.
And just a little note here for you.
The person who invented the silicones didn't know his chemistry.
And he thought that the silicone, in fact, had a double bond here.
See, this is acetone, and it from comes from the family of ketones.
So you have methyl above, methyl below, carbon, and a double bond off to the side.
Well, it turns out that silicone doesn't form double bonds.
Everybody in 3.091 knows this.
It's too big.
So it only has a single bond.
If it's a single bond, it's not an -one.
It's just like a pentane, methane, so on.
So this should be a siloxane.
So what people refer to as silicones today properly are called siloxanes.
OK. Another thing to note.
Let's go back to the-- you see here?
We go down the backbone of the nylon, there's the carboxylic acid.
And there's a C with a double bond to the O, and it spit off its H, right?
That was the proton donor.
Look at over here.
There's an amino group here, a nitrogen with a hydrogen.
Well, if I line those two up just right, look at what can happen.
Here's a carbon-oxygen, and here's a nitrogen-hydrogen.
What do you know if you've got hydrogen sitting next to an oxygen?
It can form hydrogen bonds.
And so nylons derive their higher strength from the hydrogen bonds that form between the chains.
Doesn't this remind you of cross-linking?
It's sort of same church, different pew.
It's not as strong as a disulfite covalent linkage, but it does form linkages.
That's good.
It's good because it gives you strength.
But it doesn't come with the penalty that cross-linking comes with.
And what's the penalty?
You can't recycle cross- linked polymers very easily, because you've got these strong covalent bonds in the chains.
If you heat the polymer up, those bonds don't break.
If you heat this up, the hydrogen bonds will let go.
And you can remelt nylons, you can recycle those polymers much more easily.
So I want to make another taxonomy here that distinguishes polymers on the basis of their ability to be recycled.
Very important these days.
One of the first questions people ask if you're using a polymer in a process, is it recyclable?
And what do you do?
You start thinking about the molecular architecture in order to answer that question.
OK.
So let's talk about the ones that can be recycled.
So recyclable ones are called thermoplastics.
Weak van der Waals and hydrogen bonds only.
And this is between the chains.
You don't break the bonds up and down the backbone.
If you break the blinds up and down the backbone, you'll pyrolyze the material.
If you go to a high enough temperature to break covalent bonds in the backbone of a polymer, you're on fire.
So don't go there, all right?
So hydrogen bonds only, so this can reheat and reprocess.
So now we're talking about something that's almost like recycling metal.
You remelt it and reuse it.
The ones that have the strong covalent linkages are called thermal sets.
And the thermal set is difficult to recycle.
This is cross-linked-- difficult.
Notice I'm not writing impossible.
It's difficult to recycle, which is a polite way of saying, you can do it, and it's really expensive to do so.
You might be better off to just start with virgin material.
By the way, where do we get the material?
How do we make polymers?
Where do we get this carbon from?
From petroleum.
You say, well, why don't we use coal?
Well, you need carbon and hydrogen in these systems.
And coal is carbon.
Petroleum is hydrocarbon.
So these are petroleum-derived.
So all these questions about resource utilization and so on come right up in center stage.
OK. Now I want to take a few minutes to talk about use of crystallization to strengthen polymers, OK?
So crystallization for mechanical performance.
We want to improve the mechanical performance.
And I alluded to this last day.
We;ll do this really quickly.
So we've seen that if we want to, we know that if we-- let me give you the little icon here.
So you're going along like this, and some nice, long chain, and all of a sudden, you get a zone of crystallinity.
So I'm talking about adding crystallinity in order to strengthen the material.
And so we know that this has more bond density than the straight chain here, so this is going to be stronger.
So if I want to strengthen the polymer, introduce more crystallization, so what favors the onset of crystallization?
Or we'll say, partial crystallization.
The whole thing doesn't turn into a crystal.
To favor crystallization, what do we do with the various set points that we have in the material?
Well, first of all, we know with composition, composition is one of our levers.
And we want to make it uniform.
So that means homopolymer is favored over the copolymer.
Or maybe there are other instances where you don't want crystallinity.
In the soles of your running shoe, you don't want that turning into something that's semibrittle.
You want it to be soft.
So that's why we use copolymers in the soles of running shoes, in order to disfavor.
And if we want to do something like those jewel cases that I keep railing against, the jewel cases are a homopolymer that have a fair bit of crystallinity.
The second one we talked about last day, tacticity, this is a second lever that you have.
So we know that isotactic over atactic.
The more uniform, the more likely the material will be to crystallize.
So the isotactic polystyrene makes those jewel boxes, whereas the atactic polystyrene makes the food containers that you get when you buy food from the trucks out here.
And number three, we talked about conformation.
And we know that conformation-- linear over branched, right?
Linear over branched.
If we have a branched polymer, it's not going to pack very well, so it's less likely to crystallize.
The linear one, as I've demonstrated in this cartoon, has a much, much higher propensity for crystallization.
All right.
How about properties?
Let's talk about properties.
Properties of polymers.
And I'm talking here about the bulk commodity polymers.
Sure, you can find specialty polymers that have remarkable properties that aren't characteristic of polymers, but I'm not talking about those.
So first of all.
Electrical properties.
They're insulators, electrical insulators.
Why are they electrical insulators?
We're talking about mainly covalent bonds here.
Covalent bonds, which means tightly bound electrons, tightly bound and localized, because you saw that in graphite, they're tightly bound, but they're delocalized.
So tightly bound electrons.
And no ions, because we've now recognized that we can conduct electricity not only by electrons, but by ions.
Tightly bound electrons, no ions.
That means no carriers, that means no connectivity.
Number two.
What's the second property?
Transparent to visible light.
Again, same thing.
Tightly bound electrons, not possible to excite, et cetera.
So these are clear, and furthermore, are generally colorless.
And this is especially in the case of amorphous materials.
Amorphous materials are clear, colorless.
What happens if we get semicrystallization?
Those crystalline zones, as I showed you last day, those crystalline zones are clear and colorless as well, but the index of refraction of the crystalline zone is not equal to the index of refraction of the amorphous zone, and so this boundary, if you will, scatters light.
So we have two clear and colorless zones of different index, and so if we put them together, we end up with something that is opaque, and that's in the case of the crystallinity.
When you have some crystallinity, you'll give up on that.
And actually, I wanted to show you a few things from the popular culture.
So I think what I've got here-- oh, these are the-- yeah, OK.
This is a little scene from the musical Chicago.
It's Mr. Cellophane.
Cellophane is polymer, viscose.
All right?
Listen to the properties.
They get it right.
[MUSIC PLAYBACK] -I tell you Cellophane, Mr. Cellophane, should I bend my name, Mr. Cellophane.
Because you can look right through me, walk right by me, and never know I'm there.
-Oh, I didn't see you.
[END MUSIC PLAYBACK]
OK.
So sometimes the popular culture gets it right.
All right.
So now, what's the third property of polymers?
Chemically inert.
And this is why we use them in food packaging, and also in other forms of packaging.
Again, strong bonds, strong covalent bonds, means that it's not very reactive with the contents of many containers, including foods and beverages.
A fourth property that's exploited in polymers is very low density.
Why?
Well, what are the constituents of the polymers?
Contains things like carbon, nitrogen-- well, let's put carbon and hydrogen first.
Those are the two biggies.
So we've got carbon and hydrogen.
You saw some nitrogen, oxygen, and then trace levels of other substances.
So these are all low Z elements.
So for example, in a beverage container.
I can make a beverage container.
You see a beverage container.
I see a bundle of properties.
What is the bundle of properties?
It has to be some physical rigidity, but not much, because I can squeeze this.
It's OK. And it has to be chemically inert with respect to its contents.
So I could build this out of a borosilicate glass, a glass bottle.
And the density of borosilicate glass is about 3.54.
Or I can make it out of aluminum, and the density of the aluminum is about 2.7.
Or I can make it out of polymer, and the density is down close to 1.
So now if I load up the beverage truck, instead of transporting the contents plus the mass of the container, the higher fraction now is just the contents, and the container weighs less.
When I was your age, we had still soda cans made out of steel.
Steel soda cans.
Yeah.
They didn't have pull tops.
You had to have an opener to open them.
And the density of steel is 7.87.
Iron is 7.87.
So from steel, down to this, down to this, and so on.
So you see the low density has an advantage.
But, you know, we're going to have to go back over here and start thinking about recyclability.
So you see the balancing act here in choosing materials.
You can't just seize upon one attribute and say, oh, this is less dense than that, let's go with it.
But it's made from petroleum, and maybe it's not so recyclable.
So now what do you do?
Maybe you do make it out of steel, because you know, what you can recycle steel.
You can take the door off a 1928 Model-T Ford, melt that door down, and you can put it on a 2010 Taurus.
That's how recyclable iron is.
I'm just telling you what the properties are.
You're going to have to make the tough decisions.
And five, solid at room temperature.
And why are they solid at room temperature?
Because the interchain bonds, even though they're weak, they have a long, long, long, long surface contact area.
So entanglement of van der Waals bonds.
So you have the best of both worlds.
It's solid at room temperature, but in the case of a thermal plastic with modest heating, you can restore back to the original case.
Oh.
Here I have, I found the Polythene Pam.
I'll play you just a little segment of it, the first verse.
The interesting thing to listen to, in many of the Beatles songs, you can't tell this.
In the song, you can really tell the accent from Liverpool.
Listen carefully.
You'll hear the pronunciation.
That Merseyside accent.
And the way they're playing is the old skiffle.
This is how they started, in the clubs in Hamburg and so on.
This is this is not advanced Beatles.
They were playing here, it's sort of little joke in the middle of the Abbey Road album.
They go retro and show the way they were when they first formed the band.
[AUDIO CLIP REMOVED ]
Did you catch that?
The Liverpudlian accent?
It comes complete with the glottal stop?
[IMITATING ACCENT] You know how you eat the tea?
Give me a bottle?
Yeah.
That's all from Liverpool.
Merseyside.
OK.
So now what do I have here?
Oh, here's recyclability codes.
Throw this in, you know, it's a part of the part of the pack.
So there's the polyethylene terephthalate, that I've shown you.
This is high-density polyethylene, PVC, low-density polyethylene, and so on.
And look here.
Invented by, invented by, invented by, invented by.
These are all man-made materials.
They didn't exist before.
Keep going.
Invented by, invented by.
And they're all invented in the 20th century, with one exception.
A pharmacist in Germany by the name of Eduard Simon was playing with the vinyl benzene and managed to make this goo that became what we know as polystyrene.
And nobody could explain.
Remember, in 18-- he was doing this in 1839, before Kekule had even speculated that you can get carbon-carbon linkages!
No benzene rings!
So that he was linking benzene, that they didn't even understand existed as a ring, and he was making polymers that they didn't appreciate that you could even make C17H36, let alone C10,017.
So in 1922, Hermann Staudinger gave the explanation that what we're seeing here is the polymerization into macromolecules.
And for that, he got the Nobel Prize in 1953.
Note, it took 30 years.
You don't invent something that's really, really prize-winning, and on the way home, turn on the radio and discover, oh, I won the Nobel Prize.
It takes time.
And especially, the more remarkable the discovery, it takes time before people accept it and say, this is not a hoax.
This is real.
They didn't have the characterization.
How were they going to verify in 1922?
So you keep waiting, waiting, waiting.
Look at what happened to de Broglie.
De Broglie said, lambda equals h over p, and they went, uh-huh.
And then, as soon as the fellows did the experiment down at Bell Labs and they got electron defraction-- boom!
He gets the Nobel Prize.
They said, you know what?
You're right!
That's how it works.
OK.
This was one of the giants.
This was Wallace Carothers.
Carothers, who for a short time was a lecturer at Harvard.
He left Harvard and joined DuPont, and he gave birth to an incredible array of synthetic materials, as they say here, similar to cellulose and silicon.
Invented neoprene, the first synthetic rubber.
This was very important for the United States and the Allied Powers during World War II, because we did not have access to natural rubber.
And the fact that they were making synthetic rubber here from materials that we had here allowed us to continue building for the war effort.
Invented nylon, in 1937.
An amazing man.
Unfortunately deeply troubled, very depressive.
And two days after his 41st birthday, he checked in a hotel in Philadelphia, and took his own life.
This is Wallace Carothers.
So I'm going to show you now some synthesis.
About ten years ago, I had a live demonstration here.
There was a gal who was doing her PhD over here in Building 13, and she'd come in and do these demos.
But, you know, it's been ten years now.
We let her graduate, so she's not around anymore.
So we took some videos of it.
So the first thing I want to show you is the synthesis of nylon.
So you're going to see this reaction.
So in this case, it's hexamethylene diamine and adipic acid.
And that's going to give you nylon-6,6 plus water, just as this reaction indicates here, and these are immiscible.
See, the adipic acid is in aqueous phase, and the hexamethylene diamine is a non-aqueous phase, so they form two phases right here, two liquid layers, and this reaction occurs at the interface between the two layers.
So you're going to see a close up of the beaker, and she's going to now dip an instrument in, and start pulling the solid.
She's making the solid from two liquids, and she's going to pull it.
And as she pulls it, she clears the interface.
Because once the interface is covered, new reactant can't get there.
So ironically, the product separates the two halves of the reaction from each other.
So you have to get the product out.
This is really cool.
So there's the chemistry.
So let's look.
This is Heidi Burch.
David, could we dim the lights, maybe?
[FILM PLAYBACK]
The derivative of a carboxylic acid then is mercuric chloride.
I have it dissolved in hexane.
My base is hexamethylene diamine, and I've got it dissolved in water.
So to do this right, I have to make them float.
So we're going to pour, if I can get it open
This is the old 10-250, too.
It's got a carpet here
--because it's dissolved in water, which has a higher density than hexane.
And then I have my acid dissolved in the hexane.
So what's going to happen when I pour that in?
It's not going to mix, right?
It'll separate
It's going to phase separate, exactly.
And we're going to have strata.
Is that going to help or hurt our reaction?
It's going to hurt our reaction, right?
To have a good reaction, you need intimate mixing.
Well--
This is like the Uzo water problem
So this is pretty, actually pretty unexciting.
Nothing really interesting has happened.
It's just kind of sitting there.
Nothing's exploding out of the beaker, nothing's coming out.
It's kind of tame.
But if you look carefully, you can kind of see-- where did my pointer go?
You can kind of see this film right there.
See that big bubble?
Well, there's a scum that's floating at the interface between the two layers.
That's actually the nylon that we formed.
What happens is, I mentioned that nylon has that excellent chemical resistance.
Well, when we formed it, it made a barrier film between the two layers so that no more reactants could get through to each other to form more nylons.
So what I have to do is remove that barrier film to allow the reactants to come into contact.
And I'm doing that by drawing off the nylon as it's formed.
Because I'm drawing off the film, are very haphazard.
And what's happening is, the polymer I'm forming has a very broad molecular weight distribution.
It's not all 240 grams per mole or whatever.
And that greatly affects the mechanical properties, and, in fact, it affects them detrimentally.
So the mechanical properties of this nylon that I'm forming aren't very good.
[END FILM PLAYBACK]
OK.
So that's the first one.
Now, the second one I'm going to show you talks about glass transition temperature, and how you engineer a rubber.
So it might shock you to know that a rubber is, in fact, a liquid.
It's a polymer that's above its glass transition temperature.
It might be supercooled liquid, but it's liquid.
And those disulfide linkages give you the flexibility so that the chains can move against one another.
If you drop the temperature below the glass transition temperature, the whole thing turns into an amorphous solid.
So what are the mechanical properties?
What I'm indicating here is the ball, all right?
So the bouncing ball is up here, above the glass transition temperature it's cross-linked and behaves as an elastomer.
If you get the temperature too low, you'll end up solidifying everything, and then it turns into something with the mechanical properties of a billiard ball, all right?
So we call that tough.
Tough means it has impact resistance.
It will absorb a lot of energy.
So if I drop the ball, it won't bounce, it takes the energy of falling and absorbs it and holds it.
So if you want a ball to bounce, you want to be above here.
So what we're going to do, is we're going to take two different polymers.
One that at room temperature is here, and one that at room temperature is here.
The one that's down here, we're going to put in the boiling water and get it over here.
The one that is naturally a bouncer here, we're going to put into liquid nitrogen and send it over here.
So we're going to make a bouncing ball not bounce, and we're going to make a flat unbouncy ball bounce.
We're going to teach these balls.
And what are we going to do?
We're going to teach these balls how to bounce and not bounce on temperature cue, by moving on opposite sides of the glass transition temperature.
So this is a glass, and this over here is a rubber, OK?
All right.
So here they are.
Those are the two rubbers.
One is norbornene, and the other is isoprene.
And both of these, if you go back to the earlier slides, have a double bond in the polymer, And that double bond has now been broken for disulfide linkages.
This is Heidi again.
[FILM PLAYBACK]
OK.
The thing I want to talk about now, is the effect of temperature on polymer molecules.
Basically, when do I get rubbery behavior and when do I get glassy behavior?
Because I can get the same polymer to exhibit rubbery behavior or glassy behavior, depending on the temperature at which I use that material.
As Dr. Sadoway told you, polymers have what's called a glass transition temperature, and the type of behavior that I get is determined by my use temperature relative to my glass transition temperature.
So I have here six little balls with faces painted on them.
The ones with the happy faces are polyisoprene.
The ones with the sad faces are polynorbornene.
Now, polynorbornene is a synthetic rubber that was developed during World War II when we didn't have access to all the rubber tree groves in Southeast Asia.
So the happy balls are happy because their glass transition temperature is minus 67.
So our use temperature at room temperature is much greater that their glass transition temperatures, so they behave as rubber would.
The polynorbornene has a glass transition temperature of 40.
So we're actually below its glass transition temperature at room temperature, and it just dissipates all the energy.
We get a much more glassy-like behavior from this material, and it doesn't bounce.
So what we can do here to demonstrate the effect is we're going raise the use temperature of both materials, and we're going to lower the use temperature of both materials.
So I'm going to drop them in boiling water, and hope that the little faces stay on, because I just painted them on with white-out.
I don't know how that's going to work.
And then I'm going to drop two of them in liquid nitrogen.
Now, these will be our controls.
Remember, polynorbornene, polyisoprene.
Now something to think about, which I know Dr. Sadoway always likes to ask, is why does one have a high glass transition temperature, and why does one have a low glass transition temperature?
And I'll leave you to ponder that little conundrum, while I try to fish these bad boys out.
OK. Oh!
White-out worked.
So this is the polyisoprene.
Can you hear the sound that it makes?
When I drop the one that's at room temperature, I get a nice rubbery sound.
When I drop that one, I get a plastic, hard glassy sound.
I've cooled it below its glass transition temperature, and now I get a totally glassy impact, like if I were knocking pool balls together.
Now let's go in for the polynorbornene.
Oh!
His eye came off.
Same thing.
I pulled the polynorbornene below its glass transition temperature as well, so it, too, exhibits a glassy collision, like a pool ball.
Now, let's go in here.
This is the polynorbornene, so it should still continue to bounce, because we've raised its use temperature further above its glass transition temperature.
That's not the exciting one.
The exciting one is-- if I can get it out.
What happens to the polynorbornene?
This is the sad ball, the one that wouldn't bounce at room temperature.
By raising its use temperature above its glass transition temperature by dunking it in boiling water, I get a rubbery response.
[END FILM PLAYBACK]
Super.
OK. House lights, please.
I don't read well in the dark.
OK.
So I hope you've seen some examples of how these work.
So now I want to talk a little bit about the broader societal issues.
So this, I think, is sort of a pattern of adoption of new materials.
You start off with this wonder, you know, this cool property that you get at the lab bench.
You know, when nylon was invented, it led to democratization.
Do you realize stockings, womens' stockings, were only owned by the wealthy?
Because they were made of silk.
When they were made out of nylon, they became cheap and abundant.
So that changed the way people dressed.
It changed the way people made clothing in a way that was unimaginable before.
So you could, for example, make a billiard ball, or piano keys, without killing an elephant.
You didn't have to use ivory.
You could use polymers.
Huge, huge implications.
So this is one of the first commercial polymers.
It's phenolic resin, and it's made by a condensation polymerization between carbolic acid and formaldehyde.
It was called bakelite.
and its corporate symbol was B with the infinity sign.
Material of a thousand uses.
And this, normally, is-- look at the elaborate art deco coffeepot.
And this is all-- it's thermal insulated, and so on.
These are all plumbing parts made out of bakelite, that previously would be made out of many pieces of metal that had to be machined and fashioned together.
You could take bakelite powder, put it into a die, heat it, and out comes the part, finished, in one step.
So this has a huge implication on productivity.
But then, so now we start talking about substitution, making the cheap things, the piano keys, the fork here, and on and on and on.
But then things that couldn't exist before.
You know, the movie film.
Look at the steering wheel here.
There's the, that's indicating the lady's stocking, and so on.
This is from Look magazine, 1940.
People thought these things were just godsends.
This is Synthetica.
This is a country, and it's got the province of Cellulose, Petrolia, Castphenolic and so on.
This is the 1960s.
This is plastics, the future has arrived.
It's at a museum in New York.
You're looking at the interior of a home with furniture built in, and it's all plastic walls, and formed, and so on.
I think maybe these guys were in a drug haze or something.
Those are the windows.
Maybe they thought those windows were square, rectilinear, I don't know.
So this is going way overboard.
And then let me show you a piece of iconic film from The Graduate.
You'll see a young Dustin Hoffman.
He's just come back from getting a bachelor's degree.
So this could be you someday in the not too distant future.
And he's at a party that his parents are throwing in his honor.
[FILM PLAYBACK] -We're all so proud of you!
Proud, proud, proud, proud, proud, proud.
What are you going to do now?
-I was going to go upstairs for a minute.
-I mean with your future?
Your life?
'Well, that's a little hard to say.
-Ben.
-Excuse me.
-Mr. McGuire?
-Ben.
-Mr. McGuire.
-Come with me for a minute
See the lipstick?
-I want to talk to you.
Excuse us, Joanne
See, it's the '60s.
They blow off the ladies.
Now the men are going to talk.
-I just want to say one word to you.
Just one word.
-Yes, sir?
-Are you listening?
-Yes, I am.
-Plastics.
-Exactly how do you mean?
-There's a great future in plastics.
Think about it.
Will you think about it?
-Yes, I will.
-Enough said.
That's a deal.
[END FILM PLAYBACK]
All right.
So this was 1967, '68.
So what's happened since then in plastics?
Well, all of these other considerations about recyclability, environmental health and safety issues.
In order to facilitate molding, when you want to mold the plastic, to get those bonds to loosen up a little bit, we use a chemical known as a plasticizer.
It's a low molecular weight polymer that helps the-- it's sort of the equivalent of putting a little bit of olive oil in with the pasta, to give it that freedom to move.
And so this, then, when you sit in a new car, the new car smell is the smell of the plasticizer, which is still evaporating.
And some of those plasticizers have molecular architectures that are similar to hormones in our bodies.
And so it's not healthy for us to be breathing this stuff.
So maybe the bright future that was talked about is not quite as bright as some people might have thought it to be.
The other thing is-- you know how I showed you the plumbing parts?
Well, we started making all kinds of things out of polymers.
As a child, I remember my father bringing me, one day, a shoe.
The shoe was 100% man-made.
It was all polymer.
And we thought this was the coolest thing, because of all man-made.
Of course, there was a complete reversal.
People said, you know, your foot can't breathe, and this is a silly thing to have.
But at the time, we started making-- and they were ugly.
And you know, for me, if it's ugly, I don't want to wear something that's ugly.
So what happened was, we started making all kinds of things out of polymers, or as the public knows them, as plastics.
And by the mid-'70s, to call something plastic was to make a derogatory comment.
It's plastic, it's cheap, it's shoddy.
So you know, making silk stockings cheaper as nylon stockings was good.
But if you get them too cheap, they're cheap.
See?
So there's an optimum there.
And all of that went into play.
By the way, how do we get the word plastic?
It comes from the Greek plastikos.
And same word as potter.
So when you're working with clay, you're working with a plastic medium.
It's deformable, it's malleable.
There's Samuel Johnson "let Thy plastick hand."
So all plastics are polymers, but not all polymers are plastics, because some polymers can't be deformed.
If they're below their glass transition temperature, they're brittle solids.
All right.
So I've told you enough about concern.
Let's look at recycling.
In the United States right now, 100 billion pounds annually of polymer, 50 million tons of which 3 to 4 million are recycled.
It's a very low recycle rate.
So this ends up in the waste.
Contrast that to steel.
Steel, we still make 80 million tons a year of virgin metal in the United States.
It's still a world-class steel industry.
You can bet it's world-class.
With our wages and our high standards, if you're still in business making steel in the United States, you better be damn good.
Very good.
So 80 million tons virgin metal, 60 million tons recycled scrap.
We recycle more steel per year from automobiles than we consume building new automobiles.
It's a complete ecology.
And compare that to aluminum.
Aluminum, 4 million tons a year, 1 million tons recycled.
Very high rate on used beverage containers.
But that's the exception.
Because this was DFE, Designed For Environment.
The alloy choices were made in order to facilitate recycling.
So there were certain metals that were forbidden to go into the alloy because they would pose problems later in recycling.
So there are many ways to get the same deep-drawing characteristics.
You know, you make a beverage can out of a sheet, and you snap.
Punch down, punch up, there's no seam on the bottom.
This is made from a single sheet of metal.
And then a second piece comes down on the top.
And so this has only aluminum, silicon, and magnesium.
You could choose other metals, but with aluminum, silicon, magnesium, it is recyclable.
But aluminum ladders-- if you're building an airplane, let's say you're the chief purchasing agent for Boeing.
And you've got a choice of aluminum scrap, which, you know, people might have thrown some old fishing weights with lead in there, and that's now contaminant in the aluminum.
Or you're going to buy high-purity virgin metal, and you're going to bet the company's future.
What are you going to buy?
You're going to put recycled metal into an airplane.
Have the thing fall out of the sky.
No way.
So you have to be careful.
Those alloys, if you take the, I told you, the door off the 1928 Model-T, and you put it on a 2010 Taurus, you can't take the wing off a DC-3 and melt it down and put it on a 777.
Because the top has an aluminum alloy with zinc, and in the bottom one has an aluminum alloy with copper.
And if you co-melt those, you'll end up with intermetallics, and the alloy is unusable.
So you have to separate them.
Now it's not so simple.
So before you go make pronouncements saying, yeah, recycling is great, think about it.
Follow through.
OK.
If you find this interesting, and the impact that polymer had on American culture, this is a lovely book by Jeffrey Meikle on polymers and their use.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So here's the box score from the celebration on Monday.
Class average improved considerably.
A number of people moved from the sad face to the smiley face.
I would still tell anybody who is down over here that the final exam is coming, and if you perform well on the final exam, we'll take a look at the overall performance of the semester.
So get in and talk to us, if you want to, to see what we can do to help you succeed here.
But it takes a little bit of effort on your part as well.
I think that's all I want to say.
There will be a weekly test, a weekly minor celebration on Tuesday, as normal.
So today we're going to start the last of two units.
We have two units left before the end of the semester.
And this one is biochemistry.
And it's one of these integrative topics that's going to bring together a lot of the material that we've looked at in the past.
And biochemistry overview, it's the chemistry of living organisms.
And the first thing to point out is that biochemistry is governed by the same laws that apply to inanimate matter.
And apropos of that, I'd like to read something that was written by the Nobel Prize winner, Richard Feynman.
"If in some cataclysm, all of scientific knowledge were to be destroyed, and only one sentence passed on to the next generations of creatures, what statement would contain the most information in the fewest words?
I believe that it is the atomic hypothesis, or the atomic factor, whatever you wish to call it, that all things are made of atoms.
Little particles that move around in perpetual motion, attracting each other when they are a little distance apart, but repelling upon being squeezed into one another."
So that's the first thing to point out.
That it's the same chemistry that we've been learning up until now.
And in fact, I think I've got a a slide that captures the-- there's the Feynman quote, the last part of it.
Second thing is, by way of introduction, you see a boom around us today in biotech.
And what is the biotech?
It's the molecular biology, it's the commercialization of molecular biology.
Biology, when I was your age, was classification, taxonomy, naming things and figuring out what bin they fit into.
Today it's not that.
It's molecular biology.
Well, what's the molecular science?
Molecular science is chemistry, and it's solid state chemistry.
Biology is solid state chemistry.
We are solid state devices made of soft matter with an endoskeleton that's ceramic, and so on.
But we are solid state devices.
So this is solid state chemistry and material science, and that's why we talk about it in 3.091.
And why now?
Why is the biochem revolution only now?
Well, it has to do with complexity.
You can think about complexity from the standpoint of materials, and in fact, lay out the ages of man.
The Stone Age.
The Stone Age precedes the Bronze Age.
Why?
Because stone is found in nature naturally.
To use stone requires only to cut it and change its shape.
But you make bronze, you have to engage in chemical processing.
There's a chemical conversion that requires a higher level of societal sophistication.
And then iron follows bronze.
Why?
Because iron melts at a higher temperature, so you have to have a higher level of technology in order to make the furnaces and to conduct the reduction reaction to extract iron.
And then comes polymers.
Polymers, the birth of synthetics, come when?
First polymers were around, what, 1829, when the pharmacist was playing with the styrene.
And then it explodes in the 20th century.
And then mid-century we get silicon.
And silicon is even more sophisticated, because we had to understand quantum mechanics to even understand that we needed silicon.
We could convert beach sand into silicon back in 1500, but who knew what to do with silicon in 1500?
And then finally comes the end of the 20th century, and now we're dealing with this stuff.
So that sort of puts things in perspective.
So what we're going to do today is start the treatment of biomolecules.
And the first thing I want to say, is we're going to be talking about macromolecules but not polymers.
So all polymers are macromolecules, but all macromolecules are not polymers.
What's the difference?
In polymers, we have a common repeat unit, whereas in the life sciences, Mother Nature is a polymer engineer gone wild.
She takes different functional groups and puts them at every stage along the way, as opposed to polyethylene, where it is the same functional group at every stop.
So it is exploded polymer chemistry, macromolecular chemistry.
So there's four classifications of biomolecules.
The proteins, carbohydrates, lipids, nucleic acids.
And we're going to look at three of the four.
We'll start today with proteins.
We will study lipids and nucleic acids, but in the time allowed, we're going to lose the sugars.
So you'll take 7.012 or something to satisfy the Institute biology requirement, you'll go much more deeply into it.
But I think you're going to have very good preparation after three lectures here, and you'll be able to put together a lot of the material that we've looked at prior.
So the first thing we're going to look at is proteins.
Proteins are macromolecules, and they're formed by polymerization of monomers known as amino acids.
So before we can talk about proteins, we're going to first look at amino acids.
So let's do that.
And amino acids are substances that have the following skeletal structure.
I'm going to start a new board for this.
So this is the basic skeletal structure of the amino acid.
It's got a central carbon, sp3 hybridized with four struts on it.
Down in the lower left corner, we have the amino group, which is nitrogen and two hydrogens.
So there's one, two, three bonds off the nitrogen, and I want to emphasize but it's still got this unpaired set of electrons, which we know makes this thing a Bronsted base.
And it's going to have a site here for proton attachment.
And that's a big feature here in amino acids.
So this is the amino group.
And it acts as a Bronsted base.
So it's a proton attachment site.
Second strut is hydrogen in every amino acid.
The third site, over here, is the carboxylic acid end, and that's carbon with a double bond, and one, two, three, four, all right?
Every carbon needs four struts, and the fourth one has the OH.
And this is the H that can fall off.
So this is the proton donor here.
This is carboxylic acid end.
So this is three of the four ends of the carbon bonds are fixed in all amino acids.
And then the fourth one is free to be specified.
This is all R. R is a placeholder.
It stands for the substituent.
This is the substituent.
And this is nature's choice.
So you can put anything here.
There are hundreds of amino acids.
There are only 20 seen in protein.
But anything that goes up here, with this structure, is an amino acid.
I don't care.
You could put a cyanide up here, for all I care.
It'll still qualify as a amino acid if these other three sites are filled in the following matter.
So 20 seen in amino acids.
20 of those in amino acids.
And now I want to look at the substituents and break those into categories.
So there's four categories of substituents.
In other words, the 20 here can be broken into four categories.
Four categories of R. And we can group those by their chemical properties.
The first group, I took this from one of the other texts.
Yeah, this is from one of the other texts.
So there's the 20 amino acids.
So I'm now highlighting some of the ones that are nonpolar.
So what do you see there?
The nonpolar ones, you see for R. The R group is above here.
so?
This is just plain hydrogen, which is nonpolar There's a methyl group, CH3, so that's alanine.
Here's a methylene with two methyls, so this is alanine, and so on and so forth.
So the different groups hang off that R position at the top.
So these are nonpolar, and this also is characterized by being hydrophobic.
So that end becomes hydrophobic when we use nonpolar group.
The second option is to have polar substituents.
And some of them even have hydrogen bonding capability.
That's the second category.
And of course those are going to be hydrophilic.
The third group is charged.
The third group actually has a charge end, which becomes a negative ion in aqueous solution.
And these are going to be strongly hydrophilic.
And also-- [BREAK IN VIDEO]
Number nine, number nine, number nine.
Can you hear?
Yes?
Yeah, that's good.
Hydrophilic and acidic.
The acoustics here today are very different, huh?
We're missing a few bodies.
We need to put the crash test dummies in here, and then we'll have the same acoustic sound.
All right.
And then the fourth category gives you a cation, R plus, in aqueous solution, which then, strongly hydrophilic and basic.
So we'll go through.
Now, our bodies can synthesize only 10 of the 20 amino acids, and then the other 10 we have to get from diets.
So when you look at some of these fad diets, you might want to check and make sure you're getting all of the proper nutrients.
Now another of the structural features of amino acids that's important is chirality.
Chirality is a feature of amino acids.
And what is chirality?
I'll show you by way of example.
Chirality talks about handedness.
Comes from the Greek word for glove.
So let me show you molecules that have the chirality that distinguishes them.
So I'm going to give an example with alanine.
With alanine, the R-substituent group is equal to the methyl, CH3.
So I'm going to put CH3 up here for the substituent, and then the other three stations are as we know them to be.
So there's the amino group, there's the hydrogen, and here's the carboxylic acid.
And note, look at the compact notation here.
We write COOH.
This O is not bonded to the O next to it.
Both O's are bonded to the carbon, but this is compressed notation.
COOH, and then this.
So this is alanine, and I could just as easily write the molecule in the following manner.
I would still satisfy the requirement that I have an amino group, a hydrogen group, a carboxylic acid group.
Actually, if I want to get really fancy in chemistry, I could write it this way.
COOH, and everybody knows H can't bond to two species.
In fact, it's the carbon here.
But it's OK.
It's compressed and in the methyl group up here.
And what you notice is that the molecule on the left is distinguishable from the molecule on the right.
These are not the same molecule, even though the same substituents are here, and the other features are preserved.
This cannot be superimposed.
So you could distinguish these two.
If I just threw them on the floor, you'd be able to pick them up and say, one has the amino group in this orientation, and if I put the other one, with the methyl above and the hydrogen below, I will have the amino group on the other side.
And these things also are optically active.
And through their optical activity, we give them their names.
And so let's take a look here.
All right.
This is just a little bit more of the amino acids.
And actually, one of our former 3.091 TAs, Andrew Magyar, who just defended his PhD thesis.
I was on his committee, and he did a splendid job, and used his 3.091 principles to get through his advanced biology.
This is a list which is going to be part of the PDF that I'll post.
But there are 20 amino acids, and as you know, there are 26 letters in the alphabet we use in English.
And so if you want to write them in compressed form, you just use one of these letters.
There's the three-letter notation as well, so ALA is alanine.
And in the past, I used to have contests in 3.091 to ask students to give me the airports, if these were three-letter airport codes.
You see, it always comes around Thanksgiving, and people are busy heading to-- and it turns out that 19 of the 20 amino acids have airport codes that correspond to the amino acid three-letter abbreviations, with the exception of glutamine, GLN.
There's no airport that has GLN as its thing.
That's a piece of trivia that will get you the last wedge in Trivial Pursuit.
And then this is the one-letter abbreviation.
So Andrew has put them in this form.
So there again, I have the basic, the acidic, the aliphatic are the ones that are nonpolar.
And this is another way to look at it.
And here's the base structure, and here's the whole thing in monochrome.
All right.
So that's part of the thing.
OK.
So here's what I'm talking about.
The mirror plane.
So this box can be mirrored as shown.
But as soon as you put a distinguishing mark in the lower right-hand corner here, the box on the left is not superimposable to the box on the right.
They are distinguishable.
And so these things will have different chemical reactivities, not melting point boiling point, but how they react with other molecules.
So here's the chirologics.
Your gloves are chiral.
If you try to put your right-handed glove on your left hand, it's a poor fit, whereas the flask here is achiral.
And now here are some molecules, just to show you how this translates into chemistry.
So this molecule is chiral, this molecule is not chiral.
And it has to do with the number of positions that are distinguishable.
And how does this fit into the grand scheme of things?
It's sort of a form of stereoisomerism, isn't it?
Remember, we looked at the geometric isomers.
Here's an example.
This is dichloroethylene.
We looked at butene.
But here we've got cis-isomer, here we've got the trans-isomer, where the chlorines are on opposite sides of the double bond here.
They're on the same side of the double bond here.
These are types of optical isomers.
These are the mirror images, the same functional groups, but in this case, in one orientation, the plus enantiomer, here's the minus enantiomer.
They're called enantiomers because it comes from the Greek word for mirror.
So we're using the Greek word for glove and the Greek word for mirror.
Now we'll talk about the optical activity, and then we'll take a few notes.
So what's the optical activity?
It turns out that if you put these in aqueous solution, they will cause the polarization vector of light to rotate.
So here's a cartoon that shows a light source with unpolarized light, and then the light goes through a polarizer, which then forces only vertically polarized light.
In other words, the electric factor is in the vertical orientation.
And so now this electric vector enters this tube containing an aqueous solution of something that has optical activity.
And the degree of rotation is proportional, obviously, to the concentration of the chiral species, and also the path length.
So if you have a dilute solution on a long path, or a concentrated solution on a short path, it's the number of collisions.
And ultimately, you have a degree of rotation, et cetera.
And so based on how the electric vector changes is how we got the names for the various enantiomers, because we've got to name them somehow.
Otherwise we can't distinguish one from the other.
So it turns out that the way I've drawn this one, with the amino group in the lower right, and the carboxylic acid group in the lower left, causes the electric vector to rotate clockwise.
So I'm going to write h nu, just so that you know, or if you want to put the electric vector or something.
It rotates page to the right, and so the Latin word for right is dexter.
So this is called dextrorotatory.
This is the dextrorotatory enantiomer.
So now you've got really nice terminology to use at the table over the Thanksgiving turkey, all right?
So this is the D-form, D-circle, or, as every right-handed person knows when speaking to a left-handed person, this is the positive direction, OK?
And now this one causes an anti-clockwise, or counterclockwise rotation of light in an aqueous solution of this.
And rather than use sinister, which is the Latin word for left, and it has all these negative connotations, they went to the Slavic, and shows levorotatory.
So this is levorotatory, L-form, or minus N, with apologies to the left-handed people in the audience.
That's what it's called.
OK.
So these are called enantiomers, and hence, you understand the choice of Man in the Mirror for the opening music.
So 19 of the 20 amino acids are chiral.
The only one that's non-chiral, the only non-chiral amino acid, is glycine.
Glycine has hydrogen as a substituent.
So now you have hydrogens above and below, an amino acid here, and carboxylic acid here.
And this one is achiral.
This is achiral.
So the other 19 so that's 95% of the amino acids, are chiral, and they exhibit this interesting behavior.
Now the other interesting piece is that only the L-enantiomer-- by and large, there a few exceptions that you can find.
There's some research on this, trying to find, always the exception.
But by and large, only the L-enantiomer of amino acids are found in proteins.
So there's no preference in nature for synthesizing one over the other, but somehow, somewhere, a long time ago, there was a preference given to the L-enantiomer.
And so all of us have the L-enantiomer present in the proteins in our bodies.
So I'm going to say only, but you know that there's always going to be some exception here.
But by and large, only the L-enantiomer of amino acids found in proteins.
And you're going to see the protein synthesis in a moment, and try to figure out why that happened.
By the way, carbohydrates, which we're not going to talk about-- carbohydrates are chiral as well.
And in sugars, only the D-enantiomer of amino acids found in sugars.
And you'll see later, this is all important in terms of a lock and key mechanism.
And there are some L-sugars.
L-sugars you know as invert sugars.
And our bodies don't recognize invert sugars.
Our body apparatus requires that there be D-sugars.
There's some science fiction novella I read many years ago about this group of people that's marooned on the proverbial desert island, and it's lush with vegetation, and they're eating all of these fruits and vegetables and wasting away.
And at the end of the novel, it comes to be known that this island, because it was isolated, somehow, back in time, the plant life there had the opposite enantiomer, and so their biological apparatus didn't recognize anything.
They were eating all this stuff, and it was just going right through them with no nutrient value.
So if you're ever marooned, check!
Find out which enantiomer you're eating!
Otherwise you're wasting your time.
Honey has the L-sugar, because bees have a different biological apparatus.
Now, when we synthesize these things chemically, we can get both enantiomers coming out of the synthetic apparatus unless we take pains not to.
And so when both enantiomers are present, we call that specimen, that's term racemic.
So you have both the dextrorotatory and levorotatory present.
Now, normally we take the five minutes at the end to talk about chemistry in the world around us, but this is, you know, day before Thanksgiving holiday, and we've got a natural entry point here.
I want to talk about what the consequence here is for human health that drives directly from chirality.
About 40 years ago, there was a drug developed in Europe called thalidomide.
And it was developed by a German firm called Chemie Gruenenthal.
If I can just spell it.
And it was developed as an anticonvulsant.
And it was a sedative, and it had fantastic properties.
Namely, that it was something that you could use in treatment of people who are very depressive, but they couldn't kill themselves with this.
If you tried to overdose with thalidomide, you would simply go into a sleep, and then you could be rescued Well, that was its original intention.
Just by accident, it was discovered by various people who were using it that it was a very fine palliative for morning sickness.
And so women started using it in the first trimester of their pregnancies in order to calm their morning sickness.
And so there were calls for the importation of this drug to the United States.
And the FDA said no.
And of course there was an outcry from women's groups that said, oh, the FDA is just a bunch of men that are insensitive to women's health issues.
Let's speed this stuff up, let's get it over here.
They're using it in Europe.
What are we doing?
It turns out that this was during the time of President John Kennedy, and he had appointed the first woman director of the FDA.
And she said, it hasn't been tested thoroughly.
We're not importing it to the United States.
People got on airplanes, they used post office, what have you.
They brought it into the United States, and it was used widely in the United States, And then subsequently was discovered that this is a teratogen, which causes severe mutation of the DNA, and children being born minus limbs.
There were grotesque deformities.
And so the position of the FDA was right.
So you say, well, but didn't they test this stuff?
And they did.
Well, didn't they test it on animal models?
And they did.
Turns out the animal models that they used lacked a certain enzyme that's present in our bodies, that converts thalidomide downstream into something that's a teratogen.
It's a really, really sad story.
It turns out now, knowing what we know, only one of the enantiomers of thalidomide causes birth defects.
Not both.
So racemic thalidomide was loaded with both palliative and toxic components.
And so this is a powerful lesson of getting the-- avoiding the toxic enantiomer.
And also, there may be people here who are opposed to animal testing.
And I understand your concerns.
But we're far from being able to sit with the Shroedinger equation and a supercomputer and predict the metabolic activity of these drugs.
And so this is the dilemma for people.
Do you want to test it or not?
Do you want to get these things--?
It's actually coming back now, in the treatment of HIV-AIDS.
It's being used as part of a cocktail with other medications.
But now we know not to give the racemic form, but only to give a particular enantiomer.
Another one that is a chiral molecule is Ritalin, which is used, among other things, for the treatment of ADHD.
And there are some negative side effects of the use of Ritalin.
And again, it's being discovered that it's only one of the enantiomers that has the negative side effects.
And so the whole question of chiral molecules is a very, very hot topic in chemical research.
And so, how do you how direct the synthesis to get to one enantiomer and not the other?
Catalysis.
Again, which is something that is very important and not very well understood.
Highly empirical, still.
So all of those lessons that we've learned are very important.
I think there's a cartoon here that actually shows-- all right, so here's the example.
This is a molecule that's chiral, and it's very primitive.
You've got a square and a triangle and a circle and so on.
And you see, both of these have the same chemical properties.
But when you're looking at this in terms of biological activity in the human body, you can think about a lot of these interactions as lock and key.
Now, there aren't these little holes here.
It's not putting a square peg in a round hole versus what are they really trying to model here?
They're trying to model the types of secondary reactions.
So for example, there could be a polar center here, and this could be charged negatively, and this could be charged positively, and so they will be attracted coulombically.
Maybe this has a polar end, and this has a dipole, and they'll be attracted.
Maybe this is nonpolar, and this is nonpolar, so they all fit, whereas if you turn it around, you've got something that's positively charged here, and this thing's a dipole, there's no adhesion.
So that's where this lock-and-key idea comes into play.
And so what we're going to do later on is to figure out how to match things up in terms of dipole-dipole interactions, coulombic interactions, and so forth.
OK.
So what are the properties now?
Let's look at the properties of these things as just straight chemicals.
So we had amino acid, I had the solid neutral form of the amino acid.
What are its properties?
Not the protein.
Properties of the amino acid.
OK.
So first of all, they are solids at room temperature.
And why would we do that?
Well, because look.
You've got hydrogen bonding capability here, and that hydrogen bonding capability allows it to form bonds great enough to overcome thermal energy.
Molecules like this, do you think they form crystalline solids or amorphous solids?
They're small enough that-- this one looks a lot like methane, doesn't it?
Looks like methane with a couple of extra things.
But from a distance, it's a sphere.
maybe A little bit oblate, but it's a sphere.
So these things form crystalline solids as opposed to amorphous solids.
Their optical properties, I've already shown you.
They're colorless.
They're not good absorbers.
Where's the covalent bonds with tight electrons?
So there's no reason to have excitation and color.
And then, they're moderately soluble in water.
And that water chemistry is what we're going to turn to next.
So now I want to look at how they behave in water.
And where I'm going with this is to show you the first stages of how we animate matter.
Because I talk about inanimate matter.
We are animate matter.
Where does the animation come from?
What's the chemical origin of our animation?
How is it that I can do this?
See, you're looking at changes in the conformation of a polymer.
That's all you're doing.
You're saying, well, how-- he's moving fast.
What's the Debye frequency?
It's 10 trillion times a second.
10 trillion times a second the atoms vibrate in my body, and the speed at which I'm rotating my hand, compared to 10 trillion times a second, very slow, right?
Which actually explains why we talk at the speed we do.
And why is it that a human being can only run-- or not only run, but runs at the speed that he or she does?
It has to do with the chemistry.
With the Debye frequency, and what the skeleton can do-- I mean, why aren't human beings 40 feet tall?
I mean, stand back here and-- all right-- we're going to create this universe.
Why are we this tall, and not 40 feet tall?
Because make a skeleton 40 feet tall would be chemically very difficult.
It's all chemistry.
The rest, as we know, is stamp collecting.
OK, so now let's look at the properties of the amino acids and water.
So I'm going to write them again in a different form.
So here I'm going to put the H2N.
Here's the amino acid N. Here's the central carbon.
And I'm going to put the hydrogen, instead of writing it down here, this is the carbon with its little hydrogen.
This is the carboxylic acid N, and then the substituent group is above.
So this is again the thing.
By the way, there are two carbons here, agreed?
And the central carbon is also known as the alpha-carbon.
This is the alpha-carbon.
That's the beta-carbon, I bet.
All right.
So now what I'm going to do is, I'm going to dissolve this in water at neutral pH.
And what happens?
What happens is that once this thing solvates, the proton falls off the carboxylic end and comes over and attaches to that electron pair at the amino acid end.
And we end up with this, at neutral pH.
H3NCHCOO, and the substituent is a spectator here.
Well, I lost a proton, so this is locally negative.
And I gained a proton with its charge over here.
So this is locally positive.
I conserve charge.
This is not neutral and this is not neutral, but it has a negative end and a positive end, and it's not a dipole.
I mean, this is negative ion-like and this is positive ion-like.
So let's say, net neutral locally positive and negative.
And just as we've fallen in love with some German terminology for words like bremsstrahlung, we have a German word for this one.
This is like a twin ion, or it's like a hermaphrodite dual gender, so this is called zwitterion.
It's got both genders, if you like.
It's positive and negative.
OK.
So there's zwitterion.
And this is the way that we start to introduce the dynamic behaviors.
So now I'm going to show you how to make something come to life.
So what we're going to do, is we're going to show how this zwitterion can respond to changes in its environment.
And we're going to invoke in order to do is to say that to impart animation, we invoke the le Chatelier principle.
It's sort of like Newton's third law for chemistry.
You know Newton's third law.
Every action, there's a reaction.
So le Chatelier was a French engineer, but his name is associated with an important chemical principle.
And it basically says that if you disturb a chemical system, the chemical system will respond in such a way as to minimize the disturbance.
So the example I'm going to give is, I'm going to start with a neutral pH, and I'm going to change the pH.
And then the zwitterion is going to respond to minimize the disturbance.
In other words, if I drop the pH, zwitterion is going to try to raise the pH back to what it was before.
And in doing so, I think you're going to start to see how we can get animation.
So it's sort of like that a chemical disturbance-- this is the le Chatelier principle.
Chemical disturbance is mitigated by a chemical response.
So here's the example that we're going to use.
We're going to say, let's take something and radically change the pH.
So let's drop the pH.
So drop a pH means, the proton concentration is going to go up.
So how would zwitterion respond if it wants to reduce the proton concentration?
Well, it's going to try to gobble up the protons.
Well, how's it going to do that?
Well, we know that there's a proton attachment site.
Look, there's a proton attachment site at the carboxylic acid end.
How do we know?
Because it used to have a proton there.
So it's going to go and vacuum up protons to try to bring the pH back to what it was before.
So let's look at that reaction.
So here's H3NCHCOO minus plus.
So this is neutral zwitterion.
And what it's going to do in response to the increase in proton concentration is try to gobble these up.
And the following is the result.
That proton is now going to attach here, and cap that negative end.
So now we've got a proton on this end, and we've added a proton at the other end.
So what's happened to the neutrality of zwitterion?
It's lost!
Now zwitterion is net positive, but we've started gobbling up protons.
So this reaction is called protonation.
And the reverse reaction is deprotonation.
And this is messy, all these darn characters.
So I'm going to write the neutral zwitterion as just HA.
There's the proton here, and the rest of this stuff is just details.
So that's A.
So HA plus H plus gives me this thing here, which is HAH plus there.
Isn't that cleaner?
That's a lot easier to read.
And we can write an equilibrium constant for that reaction.
I'm going to call this reaction 1.
In reaction 1, I can write K1.
And it's written, the convention is that you always write the reaction constant for the deprotonation reaction.
So I'm going to write the reaction constant for this reaction from right to left.
So that's going to be this.
H plus over HA divided by concentration of HA H plus.
And then we're going to bring Sorensen to the party, and take logarithms, and make it nice and clean.
As we know, when Sorensen enters the room, he straightens things out.
And so this will then become PK, and if I take the log base 10 of this, I'm going to end up with minus log base 10 of H, which then will be PH.
And then this flips over with the minus sign, and becomes log base 10 of the ratio of protonated zwitterion over deprotonated zwitterion.
And this is known as the Henderson-Hasselbalch equation.
And what does it answer?
It answers the question, you tell me what the pH of the system is, and I'll tell you what the ratio of protonated to deprotonated is, because the PK is the function of the R group.
So different R groups have different PKAs, and they'll respond differently.
Some are very aggressive, and some are not so aggressive.
And look at what happens.
At one end, when we have neutral pH, we have all of the zwitterions sitting here.
And as we go to higher and higher pH, more and more of the neutral zwitterion converts to the protonated zwitterion.
And in the extreme, when I've got very, very high proton concentration, I will protonate all the zwitterion.
I'll protonate it out of existence.
And the only amino acid that exists is fully protonated.
So I've got a sliding scale from 100% deprotonated to a 100% protonated.
Well, somewhere along this scale is 50-50.
And what happens when I've got equal amounts of protonated to deprotonated?
That means the concentration of protonated equals the concentration of deprotonated.
I have the log of 1, which is 0, and that's the pH at PK1.
So now you see physically what PK1 means.
It's the pH at which you're halfway in between, which makes sense.
If I wanted to characterize something, it has values between 0 and 100, I'd give you the value at 50, and you know, it varies around 50.
Either 50 plus 50, 50 minus 50.
So that's what the PK is.
Good.
Now let's do the same thing for the opposite side.
Let's, instead of a sudden drop in pH, let's look at a sudden rise in pH.
Now you see, just to finish the point.
Look at what happens here.
What I'm trying to show you is that this change in pH caused this molecule to change.
you I mean, this could be in your stomach.
And now you drink some coffee, or you drink some cola, and the pH goes way, way down, because you've drunk something that's acidic.
Well, there are functional groups in the lining of your stomach that are going to now start protonating.
And when they protonate, they're going to change.
This could have been bonding to something, right?
COO minus could've been bonding to some other molecule that's locally plus.
When the proton comes in here and caps it, that bond is broken!
And that's the beginning of animation.
So that when I drink something, and all of a sudden, my stomach goes into action.
Well, we're going to have to process this stuff.
Have to turn on some switches here.
It's not my brain that's doing it!
It's not that the brain gets all these sensor signals.
We couldn't function that way.
We're functioning just by these things acting right there at the site.
This is the origin of animation.
I'm telling you that this is the secret of life.
I can't tell you the meaning of life.
You have to go somewhere else to answer that question.
But I can tell you where life comes from.
There it is.
So now, sudden rise in pH, this means that the proton concentration goes way down.
See, zwitterion is like one of these friends you might have.
Do you have any friends that are, what do you call them, rescuers?
You know, if they see anything wrong, they're the ones that get involved?
They've got to help everything?
In the extreme, they're quite meddlesome.
Well, this is what zwitterion is like.
See, zwitterion sees a sudden drop in pH and goes, I'm zwitterion!
I can help!
I know what to do, because I'm zwitterion!
I fix things!
And I'm conflict-averse, I don't like this!
OK.
Imagine you are zwitterion.
So here's zwitterion, sitting there, minding its own business [BREAK IN AUDIO]
--for a minute.
Why?
Because it's in a neutral pH solution.
Minding its own business.
That's a chemical joke.
All right.
So there's zwitterion, just sitting there, minding his own business.
And all of a sudden, the pH rises, and the proton concentration drops.
So now we've got an abundance of hydroxyls.
We're in an aqueous solution.
So the zwitterion looks around and says, I've got to fix this, but there's no hydroxyl attachment site!
So the zwitterion can't gobble up the hydroxyls.
So what's the zwitterion do instead?
Instead, it starts shedding protons.
Because if it sheds protons, then what can happen is, if it forms this plus proton, then the proton can react with hydroxyl to form water, and then shoot the pH back to near neutrality.
So let's dump the proton here, and we'll just end up with H2 and CHCOO minus with the R group.
So this is deprotonation in order to address the sudden rise in pH.
And now we can write a K2.
Let's call this reaction 2, and with K2, since it's a deprotonation reaction right away, we can just go ahead and write, this is going to be H plus.
And what's left?
This is just a denuded, deprotonated zwitterion, just the A minus, all over neutral zwitterion.
And it looks like this.
And we can invoke Sorensen and get PK2 equals pH plus log base 10 of the neutral zwitterion over the deprotonated zwitterion.
And there you go.
And then, of course, that 50-50 concentration of deprotonation, the pH gives you the value of PK2.
And now, at one end, I get deprotonated.
The other end, I get fully protonated, right in the middle.
I get the 100% zwitterion.
I said at neutral pH, but that doesn't mean 7.
It means the pH at which this particular species is stable.
And that's different from 7, and it's somewhere in between, isn't it?
Because if I go to very acidic solutions, I will start to protonate.
If I go to alkaline solutions, I'll deprotonate.
So somewhere in between.
And that somewhere in between is called the isoelectric point.
The isoelectric point is the place at which we get the neutral.
So that's HAmax.
The concentration of this is maximal at the isoelectric point, and that's called P sub I, and that's given by PK1 plus PK2, divided by 2.
And that's it.
You don't have to run it.
We're done.
Thank you.
OK.
So this is the simple story.
It gets more complicated if, for example, the R group can protonate or deprotonate.
So now we got two or three-ring circus.
It gets more interesting.
But this is the start, and there's some homework problems that will help you muddle through.
OK.
So I think this is a good place to switch into the final five minutes.
So here's the titration curve.
I'll show that at the beginning of the next lecture.
OK.
So now let's talk about kinetics.
And the extreme in kinetics, I said, is an explosion.
And we're coming upon December 6, which is the anniversary of the largest explosion in the history of the world before the nuclear age.
And it occurred in 1917, and Halifax Harbor, Canada, at that time was a British colony, and therefore Canada's foreign policy was dictated by Britain, which was involved in a war.
And there was a French supply ship and a relief ship, and they collided in Halifax Harbor.
This is what the French supply ship had on it.
Notice 20,000 tons of TNT.
At 8:45 AM, the Mont-Blanc is hit by the Imo.
But the TNT was not ignited, and instead we ended with sparks and a fire.
By the way, this is TNT.
If you look, this is toluene.
Toluene is methylbenzene.
It's just the methyl group.
And this is trinitrotoluene.
And if you go to Wikipedia, Wikipedia gives this formula, which is wrong.
The methyl group is missing here.
Just a word to the wise.
Be careful when you use that tool, I put in quotation marks.
Because it's worth about as much as you pay for it, which is why we teach you to use the bibliographic things.
OK, so there it is, right here.
I didn't make this up.
Look at this.
Completely wrong.
Anyway, so what happens?
"The crew of the Mont-Blanc, aware of their cargo, immediately took the lifeboats, screaming warnings that no one heeded.
They rode for Dartmouth"-- this is across the channel from Halifax-- "leaving the now furiously burning ship to drift towards Halifax, propelled in that direction by the Imo's impact.
The Mont-Blanc drifted by a Halifax Pier, brushing it and setting it ablaze.
Members of the Halifax Department responded quickly, and were positioning their engine up to the nearest hydrant when the Mont-Blanc disintegrated in a blinding white flash, creating the biggest man-made explosion before the nuclear age."
It was 9:05 a.m. on a Thursday morning.
"Over 1,900 people were killed immediately.
Within a year, the figure climbed to well over 2,000.
Around 9,000 were injured, many permanently.
325 acres, almost all of North End Halifax was destroyed.
Much of what was not immediately leveled burned to the ground, aided by what are stockpiles of coal in cellars.
As for the Mont-Blanc, all 3,000 tons of her were shattered into little pieces that were blasted far and wide.
The barrel of one of her cannons landed three and a half miles away.
Part of her anchor shank, weighing half a ton, flew two miles in the opposite direction.
Windows shattered 50 miles away, and the shock wave was felt even in Sydney, on Cape Breton Island, 270 miles to the northeast.
"There was about 20 minutes between the collision and the explosion.
It was enough time for spectators, including many children, to run to the waterfront to watch the ship burning, thus coming into close range."
I mean, they were little kids, and it's a fire, and the fire engines are coming!
It's fun!
It's more interesting than going to school, right?
"It was enough time for others to gather at windows.
Office workers came to the windows, and with the explosion, the glass shattered and blinded them.
"Not surprisingly, the hospitals weren't able to cope with so many wounded.
There was also a desperate need for housing, and the misery was compounded"-- are you ready for this?-- "by the blizzard that struck the city the following day, dumping 16 inches of snow over the ruins.
"With astounding speed, relief efforts were set in motion.
Money poured in from as far away as China and New Zealand, Canadian Government gave--" blah, blah, blah.
"But most Haligonians"-- which is what you call a denizen of Halifax-- "most Haligonians remember the generosity of the state of Massachusetts, which donated $750,000 of money and goods, and gave unstintingly in volunteer assistance through the Massachusetts Halifax Relief Committee."
People from the hospital district here got on trains, and went up there to assist, especially from Mass Eye and Ear, to assist the people that had been blinded.
"Starting in 1971, to this day, Halifax sends an annual Christmas tree to the city of Boston in gratitude."
And there are specifications on what the tree should look like, and how big it should be, and so on, and they bring it over to Boston Commons.
So if, in the days that come, when you get back from Thanksgiving, if you happen to be across the river and you see something that's about 45 feet high and it looks like it came from Canada, this is what it is.
And it's all about the Halifax explosion.
OK.
There you go.
All right.
Well, have a nice weekend.
We'll see you on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK.
The weekend is over.
One announcement.
Last weekly test will be tomorrow, and it'll be based on homework 11, focused on polymers.
And then the following week is the very last week of semester, and there's no testing, no work required of you, when the subject has a final exam.
And this one has a big final exam, big celebration, so there will be no weekly test next Tuesday.
So this is the last one, so get out the Kleenex.
All right.
So today we want to continue the treatment of biochemistry.
Last day we started looking at proteins, and the building blocks of proteins are amino acids.
And so we looked at the basic structure of amino acids, shown here, which is an sp3 hybridized central carbon, which I've designated alpha carbon.
And three of the four bonds are the same in every amino acid.
Every amino acid has, off the central carbon, an amino group.
This is the nitrogen with two hydrogens.
It has a hydrogen, a lone hydrogen, and it has a carboxylic acid, which is this COOH.
Double bond here, I'm looking at four sticks off the carbon.
One, two, three, four.
And this proton here is the one that can fall off and go over to the side.
And when it does so, then we end up with the acid.
It's a proton donor.
And then the fourth bond is this designated R. R is the substituent, and it could be anything.
We could put a cyanide up there, if we want.
Anything that conforms to this architecture with whatever choice of R is still qualified as amino acid.
There are only twenty choices of R that are found in protein.
We also studied chirality, which is handedness.
Some of these amino acid molecules are chiral.
That is to say, if you put this carboxylic acid in the lower left, and the amino group in the lower right, you have molecules that are seemingly identical, but not superimposable.
We call them chiral.
And the two forms are called enantiomers.
And only the L-enantiomer is found in proteins.
And we saw that the L is determined on the basis of polarization rotation of light in an aqueous solution of this.
Except for lycene.
In the case of lycene, we have H up here.
So we have an H above and an H below.
That molecule is not chiral.
So nineteen of the twenty amino acids that we find in proteins exhibit chirality.
And later in the lecture we started looking at aqueous solution behavior of the amino acids.
And what happens is, when they dissolve in aqueous solution, they form zwitterions.
Zwitterions by proton transfer.
So the proton goes off the carboxylic acid and joins the amino end, so you have a plus end and a minus end.
And then we looked at what happens with the behavior in aqueous solution, and we came up with the Henderson-Hasselbalch equation.
And here it is, plotted.
This is the case for alanine.
In the case of alanine, we have the methyl group here.
So the R is CH3.
Everything else is the same.
So you can see that here's the neutral zwitterion, where the H off of the carboxylic acid, has transferred over to the amino end.
We have charge neutrality.
The molecule started off net neutral, and now the plus is over here, conferring a positive charge to this end, and leaving this end less a positive charge, and therefore minus.
So it's globally neutral, but locally positive and negative.
Hence the term zwitterion, because it's essentially dual gender, both positive and negative.
And we looked at the aqueous solution behavior of the zwitterion, and we saw that zwitterion is responsive to its environment.
And we reasoned that this is how you start to impart animation to something.
How do you get something animated?
You get something animated by having it respond to its environment.
A rock sits there.
You can talk to the rock, nothing happens to the rock.
But animated objects, if they're endowed with hearing, if you talk to them, sometimes they respond.
So what's going on there?
It to do with the ability to respond chemically.
And so what we saw here is, when we went into low pH regime-- low pH means very, very high proton concentration.
The zwitterion responds by the le Chatelier principle to try to minimize that acidity.
So it gobbles up protons.
And over here we see that it gobbles up protons by attaching to this carboxylic acid end.
That's a proton attachment site.
And in the extreme, all of the zwitterions have been capped by protons.
So over here we have complete protonation, and that's what we're showing in this graph here, which is just basically the one that's up on the screen, but I'm using different-- see, this is taken from one of the biology readings.
They say equivalence of hydroxyl.
There's no hydroxyl here.
I don't know how these-- you know, the language of some of these branches of science puzzle me.
That's equivalence of hydroxyl, but we don't have hydroxyl!
It's all about protons.
So here it is, in terms of the equivalence of H plus.
So here's the neutral zwitterion.
That's this thing here.
And this H, if it falls off, it can then become the neutral zwitterion.
And then what happens is, as we go to very, very low pH, what happens is that the zwitterion starts attaching protons.
So this is a neutral zwitterion to which we've attached the H. So it's much-- to me this is transparent.
What this reads-- I don't know.
Equivalence of OH?
There's no OH.
That's a way of saying it's fully protonated.
BUt to me that's sort of a backwards way.
Don't tell me what it isn't.
Tell me what it is.
So this is fully protonated over here.
And as the pH rises, the zwitterion feels less compelled to gobble up the protons, so it sheds protons.
And finally the pH gets to a point that's high enough that the zwitterion plays no role, and it exists as the neutral zwitterion.
That's the isoelectric point.
We have a 100% concentration of zwitterion.
Then we go to the other extreme.
The other extreme is the high pH regime, where it's proton deficient.
And so zwitterion responds by-- what?
Throwing out hydroxyls?
No!
In spite of what's written here, there are no hydroxyls.
What zwitterion does, is it tries to shed protons to compensate for the proton deficiency.
And so some of the HAs become just A minus, because the H's are being liberated.
And over here at the extreme, all of the zwitterion has lost its H here.
And so now you have something that's net negative.
Over here you have something that's net positive.
In the center it's net neutral.
So this is the high pH regime, low pH regime, and this is the Henderson-Hasselbalch equation, which answers the question, what is the ratio of neutral zwitterion to either the deprotonated form or the protonated form as a function of pH?
You tell me the pH, and I'll tell you what the ratio is.
Last thing that's really cool about this, and shows you the power of this mechanism of attachment or detachment, is as follows.
Look at here.
This is not exactly to scale, but it's reasonable.
You see the pK?
This is pK1.
This is for the protonating reaction.
So over here I have neutral zwitterion, over here I have no neutral zwitterion, all the zwitterions are fully protonated.
And in between, I've got-- this is supposed to be sort of halfway across here.
So this is 50-50.
But look at what happens.
For very, very large changes in ratio of zwitterion to protonated zwitterion, the pH doesn't change that much.
The pH changes a lot here, and it changes a lot here.
But over a vast expanse of protonation or deprotonation, the pH kind of hovers around pK1.
So the zwitterion is really doing its job.
It's trying to hold that solution from shifting pH dramatically.
We say that zwitterion, by doing so, acts as a buffer.
A buffer holds pH near constant.
And the near constant value it's going to hold it to is its pK.
So in the acidic regime, in the very low pH regime, over a broad range of protonation deprotonation, it's going to be roughly a pK1.
And the same thing happens over here.
You see?
Over a broad range of A minus to HA, the pH kind of holds around pK2.
And then finally, over here, the system gives up and then it shoots up to the 100% deprotonated.
So that's the story there, and you see the isoelectric point in the middle.
Now, so far all we've done is we've said what happens when we have a simple amino acid that has just the one proton attachment point on the carboxylic acid and the amino acid.
But it can get more complicated.
And there is an example in the homework that you might want to look at before the final exam.
And that's the case where the R group itself has the ability to protonate or deprotonate.
Suppose this R group had a COOH ending here.
Can you see that it could also participate?
And so then the isoelectric point isn't simply going to be the average of the pK1 and the pK2.
Because now there's a pK for this thing, as well.
So this is the example of the tritratable-- that's the term they use.
If I could just spell it, I could convey the information.
Titratable R group.
That's what they're talking about.
Otherwise, if the R group is a neutral, it's just a spectator.
Doesn't doesn't participate any more than this.
See, here's a hydrogen.
Look at this one, people.
You see this hydrogen?
It doesn't participate at all.
Please don't tell me at some point that the zwitterion says, I'm out of hydrogens from here, I'm out of hydrogens for here, I'm going to start shedding this.
This is a really strong covalent bond.
It's not going anywhere.
If you want titration, you've got to put the titratable R group up here.
OK?
This is so cool.
So now, I'm going to show you an application of this stuff that's used in biology.
And I know, because one year they gave a crash course for engineering faculty in biology, and we did we actually did this experiment in the lab.
So I'm going to show you how you can use the fact that zwitterion changes in response to its environment in the laboratory.
So what we're going to do is I'm going to make a column here.
This is a column, and it's an aqueous solution, a column of varying pH.
So it starts at the top, and just for argument's sake, I'm going to put high pH here at the top, and low pH here.
So it's an aqueous solution.
You can say, wait a minute.
He just taught us fixed laws.
This is proton deficient this is proton rich.
Aren't the protons going to go north and the hydroxyls are going to go south?
Yes.
So what are we going to do?
We are going to slow down diffusion.
We're going to change the diffusion coefficient, and what we're going to do is, we're going to fill this with a solid suspension and turn it into a gel.
So what's the function of the gel?
Don't say that Professor Sadoway said that he stopped diffusion.
Diffusion goes on, but it goes on at a slow rate.
So over the time duration of the experiment, for all intents and purposes, the pH changes imperceptibly in comparison to everything else that's going on.
It's all about time scales.
You realize, we are all aging.
We've been aging ever since I started this lecture.
But for all intents and purposes, we can view ourselves as being the age we were when we walked in the room.
And even over the semester, chances are.
All right?
Now if we meet back here in ten years' time, expect to see some changes.
Right?
But over the time scale of the experiment, we can say we take diffusion out.
So that means the high pH is here, the low pH is here.
Now what I want to do, what is it I'm trying to accomplish?
I want to separate an amino acid mix.
And just for grins and chuckles, we're going to have A1, A2, and A3.
Three different amino acids.
Which means just three of these things, but three different R values.
I'm going to pour them the top.
I've got this aqueous solution, got all three amino acids.
So what happens when they fall in here?
When they fall in here, they start diffusing, too.
Right?
Well, first of all, they hit a very, very high pH regime.
So what what's the immediate response to those amino acids in a high pH regime?
They're all amino acids.
They're all those rescuers.
And they say, Hi, pH.
It's proton deficient?
I can help!
And they all start shedding protons.
And so within moments, all of those amino acids have been totally-- as they say in California-- totally deprotonated.
And here it is.
So they're all sitting there, in net negative.
They're all sitting there in net negative, right up here.
But I just told you, I shut off diffusion, slowed it down.
But I want to get out of here before dinner.
So I don't want the protons moving, but I do want the amino acids to get moving.
So what do I do?
I put on my elecrochemist's hat, because as you know, electrochemistry is the highest form of chemistry, the most noble form of chemistry.
It happens to be the area that work in.
And so what we do is we electrify the ends.
And we'll polarize the top negative and the bottom positive.
Well, I just poured in a bunch of amino acids that became net negative, and now they're next to the negative electrode.
So what happens?
Now they're in an electric field, and they start moving.
They start moving in the electric field, and they're all net negative.
But what happens, is they move down here.
So this is medium pH.
Can you see?
I've established the pH gradient.
And as they move from high pH to medium pH, the proton deficiency isn't as intense.
So they're responding to their environments.
They're saying, you know, back there I had to get rid of all my protons.
But now, it doesn't feel so proton deficient, so I can start taking protons on.
They respond to their environments.
So here, they're all A minuses.
And if they ever get down here, they'll all be HAA pluses.
And so what are they in here?
They're sort of on their way to becoming HAs.
Because the pH is continuously variable.
So as they become less negative, what happens to the driving force by the negative electrode?
It's weaker.
So the protons were, first of all, they're anions.
They're net anions.
They go, whoa!
Negative!
And they start moving, and then they get less and less negative, and the field gets weaker and weaker.
And eventually, they take on enough protons that they become neutral zwitterions.
And what is the response of a neutral ion in an electric field?
Nothing.
It stops.
And so let's say, this happens to be the isoelectric point of amino acid number 1.
So it gets to the isoelectric point, and it doesn't move anymore.
And number 2 and number 3 keep going, and this might be the PI of have number 2, and this might be the PI of number 3.
And now I want only amino acid number 1 from this mix, I can now go in-- this is a gel.
It's put down on the glass plate, or you can put it in a column.
Now you where they are.
You come in-- because you've got a pH meter, you know where this is.
You know the PI.
Bingo.
That's where they are.
And if I want number 2, they're right here.
And number 3, they're right here.
So this process is called gel electrophoresis.
This is the term.
It tells you nothing about the process.
It tells you how to conduct it.
You make a gel, and you polarize the ends.
Electrophoresis is simply particles moving in an electric field.
You can paint a car this way.
Obviously the car, when the car is made of metal.
If you have a fiberglass body, you can't use electrophoresis.
The car is made of metal, you polarize the car, and the paint, the paint has got dipoles.
And so the paint sticks to the car.
And then you lift it out, let it drip, and then just get the right amount of paint proportional to the charge you put on the car by electrophoresis.
What's that got to do with this?
It doesn't talk about the focusing here.
So the mechanism here is called isoelectric focusing.
So isoelectric focusing is the principle by which we can separate the amino acids from one another.
The gel electrophoresis column is a device in which we do so.
And let's be modern, so we'll give it a three-letter initialization.
IEF, isoelectric focusing.
Isn't that cool?
Yeah.
I thought so.
Anyway.
OK. Everybody's still zoned out from turkey or something.
No response.
This is brilliant.
It's beautiful, and it integrates all of this material, and deadpan.
All of this withholding love.
You've been with your families, I can tell.
All of the old family dynamic, see?
It's here.
You brought it with you.
Leave it!
Leave it at the airport.
OK. Now.
So we've been talking about the amino acids.
We said we're going to use these as building blocks for proteins.
Let's build some proteins now.
So protein synthesis.
So we're going to make macromolecule.
What are we making?
We're making a macromolecule.
Protein is a macromolecule.
You notice I didn't say polymer, because a polymer going to have-- every element along the backbone is going to be the same.
But I'm telling you that we can change the choice of R at every station.
So it's a macromolecule, but it's not necessarily a polymer.
But we do know, if we're going back a macromolecule, we've got two choices.
Addition or condensation polymerization.
So I look at the basic structure-- let's put it back the way we found it, so that people don't leave thinking that we have two carboxylic acids.
So the most the most general form doesn't have a titratable R group, but it does have an R up here.
So I want to take one of these and link it to another one.
So how do I do it?
Well, right off the bat, addition polymerization is no good.
Because even if I had enough energy in my body to form a radical, what could I do with this?
There are no double bonds here!
There are no double bonds here.
I can't break a double bond and then allow this to bond to its neighbor.
So axiomatically, I have to go with condensation polymerization.
OK?
Macromolecule formation by condensation reaction.
I don't even want to say the word polymerization.
By condensation reaction.
OK.
So let's look at an example here.
An example will be this.
I'm going to write, here is the COO R. So I'm going to show this, and then over here-- well, maybe I'd better show some fine structure.
I'll give you a little more fine structure.
So we'll put the double bond up here, O minus.
And then over here, I'm going to put H3N plus.
So here's the neutral zwitterion, agreed?
And I've got some kind of an R group up here.
And I'm going to put next to it a second one of these.
So here's the alpha carbon, and we'll go COO.
And then over here, I'm going to unpack this nitrogen.
It's got one, two, three hydrogens, and it's net positive.
So there's the neutral zwitterion.
So now by condensation reaction, what we'll do is we'll take this oxygen off of the left amino acid, and we'll take two of the hydrogens off of this amino acid, and we will eject a water molecule.
We'll eject the water molecule.
And that's safe.
It's compatible with the body.
And now what's left?
This carbon is naked.
One, two, three.
There's a bond sitting here that used to go to the oxygen.
And this nitrogen, it's sitting naked too.
And so now we can do, is we can make a link between this carbon and this nitrogen.
And this bond is called the peptide bond.
This is the peptide bond.
OK?
And we see that the atomic mass of the dipeptide is less than the sum of the masses of the individual components, because we've lost the water.
So I think I've got some slides here.
What's here?
Oh.
This is just the various, pK1 and pK2.
And then this side chain here, this is the value of the pK should you have a titratable R group.
That's why there's the third column here.
So when you see this-- you see, they're all roughly around two for the highly acidic regime, and all around high nines, low tens for the alkaline regime.
Roughly the same.
All right.
Now connotation polymerization.
This is the formation of nylons.
You see how you bond this mer to the adjacent mer?
Look at this.
The nitrogen, that hydrogen is sticking up.
There's the nitrogen bonding to the carbon.
There's a long pair, and nitrogen-carbon.
Amide bond, peptide bond, same thing.
The same bond in the backbone of nylon is the same bond in the backbone of protein.
Only, the biologists call this the peptide bond, the polymer scientists call this the amide bond.
Think about that when, you know, the holidays are coming, you think you're really special, and you know, the proteins in your body are really bound the same way that nylon is, you know?
All right.
So now let's consider-- and here's some examples.
Here's a peptide.
So it's a long chain here.
So it starts, here's the amino group, H3N.
It's got glycene, and then leucine, and valine, glutamine, et cetera.
So there it goes, all the way around, around, around, around, around.
And here's another one.
There's the amino group, and then, boom, boom, boom.
What's happening here?
All of these click stops along here are this.
H2NCHCOOH.
And the only thing that's changing is the R group.
If this were nylon, it would be the same thing, all the way down.
That's how this differentiates itself.
All polymers are macromolecules.
All macromolecules are not polymers.
These are not polymers, but they are macromolecules.
And what else do you see here?
Look at this.
Cysteine happens to have a double bond.
A residual double bond in its R group.
Well, where you see a double bond, you go, I can break the double bond, and then I can make a another bond.
And that's what happens.
So we break the double bond with the sulfur, and now we make a disulfide linkage.
What do you think the mechanical properties of this protein are?
Mechanically, what do you think this one's like?
It's rubbery.
This is an elastomer, because you can move the top chain vis-a-vis the bottom chain until these disulfide linkages are flattened.
And then if you let it go, it springs back.
It's the mechanical properties of, in this case, this is insulin.
Now I want to give you a sense of the variety here.
How much information-- because ultimately, it's all about reproduction, right?
That's the number one function of any living organism, is to reproduce.
That's what makes it life.
Right?
So it needs an instruction set.
And the instruction set doesn't come in the box.
It comes inside the organism.
And this is the language.
This is all we have to work with.
So here's a simple example.
This is, how many combinations do you have of dipeptides?
So when you start with two amino acids.
So this is phenylalanine and aspartic acid.
So if I told you, you have A and B, and how many AB mixtures can you come up with?
You'd say, well, you've got AA, you've got BB, and you've got AB.
You know, it's sort of like calculus, right?
If you square A plus B, you'd get A squared, plus 2AB, plus B squared.
But because of chirality, AB is distinguishable from BA.
This is cool.
This is, again, math serving us, instead of we serving math.
All right?
So A plus B squared is A squared plus 2AB plus B squared.
But no, no, not in biology.
Because AB is chiral, and therefore distinguishable from BA.
Now you understand chirality.
So here's-- you get four.
Because see, in this case, phenylalanine peptide to aspartic acid is distinguishable from the one on the right.
They're distinguishable.
They're different.
OK, so that let's roll with that.
Let's say, so if I start with two amino acids, and I want to make dipeptides, well, that's trivial.
That's 2 squared, is 4.
So I got four different dipeptides, starting with two amino acids.
Now, suppose I said, how many different dipeptides can we make, period?
Well, I know I've got a library of 20 amino acids that are found in proteins.
So that means the number of dipeptides would be then 20 squared, which is 400.
But that's hardly a macromolecule.
So let's get into the macromolecule business.
So suppose I took a library of twenty amino acids, and I said, how many kilopeptides could I make?
What's a kilopeptide?
A kilopeptide is something that's got 1000 peptide bonds in it.
So that's going to be 20 to the power of 1000, which is 10 to the power of 1300.
And this is small.
n equals 1000.
I'm not stretching the point by pulling something exceptional out.
A polymer, if it were polymer, 1000 units long, you'd say, that's sort of fair to middling length.
I'm not pulling out the longest polymer ever known.
I didn't go to the Guinness Book of Records for this.
This is just plain vanilla mer length.
So look.
How much information can be contained?
That's in a single element.
OK.
So later on we're going to return to this, and appreciate how we can get to such information density.
And this I think I alluded to on Wednesday.
These are the three-letter initializations, and I'll post this at the website.
So these are the airports that correspond to the three-letter initialization.
Then I showed you the one-letter abbreviation for each of the peptides.
You know, it's kind of strange, because the amino acids are what they are.
And most airports, they're not like the main ones.
It's not like Boston or New York.
It's weird places like Australia and Texas and stuff.
You know?
So-- hey, you've got to have a sense of humor!
Come on.
All right.
And there's glutamic acid.
There's no airport that has GLU as its three-letter-- OK.
So here we are.
So now we want to do is want to talk about protein structure.
And so we're going to look at this as material scientists.
We're not going to look at it the way chemists, capital C-chemists, do.
And I think by looking at it from a structure perspective, you get some insights into what we can do with these things.
So protein structure.
And I'm going to use the terminology that the biologists use.
So first of all, they have something they call the primary structure of the protein.
And it speaks to composition, the composition.
Well, what's the composition?
We know that every protein is made of amino acids, and every amino acid is virtually identical, except for the R group.
So when you say the composition, you're really saying, what is the R sequence?
It's the amino acid sequence down the backbone.
Because you've polymerized this thing, or macromolecularized it.
Amino acid sequence, which along the backbone.
Which is really the R groups.
The sequence of R groups, which is going to be the sequence, OK?
All right.
So here you can see.
There's an amino acid sequence.
Number two is the-- these are really-- you know, like the polymer people had conformality.
You know, they had nice, powerful words.
The first order is the primary, and then the second one is the secondary.
I'm trying to make fun of the biologists, and you're sitting there, you know, with that same Thanksgiving dinner table stoicism.
OK.
So what's the secondary?
The secondary structure of proteins speaks to packing.
And what's the gambit here?
Why do they pack in different ways?
Well, just as we saw, you can have a long chain, straight chain.
Sometimes things full back on themselves.
It's still a straight chain, but there's a curve, a bend in it, and so on.
What proteins are going to do in order to pack, is to try to maximize hydrogen bonding.
And this is not even between proteins.
This is a protein forming on itself.
And you could argue, why do you maximize hydrogen bonding?
Because when you make more bonds per unit volume, you decrease the energy of a system.
You've been taught that over and over again.
So these are long chain molecules, but they still can loop back on one another.
And if you've got a choice of forming a dipole-dipole interaction, a induced dipole-induced dipole interaction, or a hydrogen bond, which is going to give you the most decrease per unit bond?
It's hydrogen.
So if you can form hydrogen bonds, that gives you the most impact per bond.
If you don't have hydrogen bonding capability, then you go for dipole-dipole.
And if you don't have dipole-dipole, then it's just the weak van der Waals, or the London dispersion.
So that's why this thing goes for hydrogen bonding.
Because it wants to.
It wants to decrease the energy.
So I have to show you a few cartoons here.
All right.
So here's a sketch taken from one of the readings.
So there's alpha carbon.
This is the carboxylic acid, right here.
And this is the peptide bond between one carbon and a nitrogen in an adjacent unit.
All right?
So there's the peptide bond.
The interesting thing is that even though this is sp3 hybridized, data show that all six atoms lie in a plane.
So what happened last time when we came up with this conundrum?
sp3 hybridized, but the data showed it all lies in a plane.
That was benzene, remember?
And how do we skate with the puck on benzene?
Resonance.
We said it resonates between two structures, and sometimes the nitrogen lies in the plane.
Sometimes the alpha carbon lies in the plane.
But thanks to resonance, all six of these can lie in the plane.
OK.
So now how do we get to hydrogen bonding?
So there's the resonance.
All right.
Good.
So you see, this is free to rotate, right?
Once we accept that these six atoms line in a plane, and these six atoms line in a plane, and 109 degree angle is fixed, but there's that degree of rotation, which is what gave us the original C17H36 folding back on itself, only we're going to fold back on ourselves in groups of six.
So we've got a deck of cards here.
I've got six here, six here, six here.
So what am I going to do?
I'm going to try to make this blue plane lie relative to the yellow plane in such a way as to maximize hydrogen bonding.
Speaking of energy deficit, we need battery.
All right, here we go.
So now we're going to maximize.
Look.
You see, if I rotate this blue plane in just the right way, I can set up to have a hydrogen bond form between this hydrogen and this oxygen, and on the orange plane, an oxygen and a hydrogen.
So now it becomes a matter of figuring out, what are these angles?
You see, there's a phi and a psi here.
What are the angles phi and psi to give the maximum hydrogen bonding between these two six-packs of atoms?
And from that, you get secondary structure.
It's all about maximizing hydrogen bonding.
After that, if you understand what I just said, the rest is trivia.
So here's what happens.
Here's one structure.
And this was first discovered, if you like, revealed, by Linus Pauling, 1951.
Linus Pauling and Corey, 1951, came with the realization that if the protein spirals in a helix, it will line up the maximum number of hydrogen-oxygen hydrogen bonding opportunities, as opposed to going straight.
So it doesn't go perfectly straight, it coils.
And this is two different cartoons trying to show.
So you can see peptide bonds, and then every so often you get a hydrogen-oxyge bond.
Because it spins around, and the idea is, as the helix forms, you can think of it as-- have you ever gone up a spiral staircase?
And you think of each subsequent turn of the spiral as a new gallery.
And what you've got is hydrogen bonds between the galleries.
So you're going up, and there's a hydrogen bond here, and you keep turning, and there's a hydrogen bond here, and there's a hydrogen bond here.
That's the way to get maximum hydrogen bonding.
There it is.
There's another example.
Maybe this is a nicer one.
And there's 3.6 residues per turn.
That means, per turn is 3.6 R groups, which, it's called residues because historically, when people did the chemical analysis, the residue contained the substituent.
That's how they determined what the identity of the protein is.
They can't just look at it.
And you can't use spectroscopy.
Maybe now you can, but in the old days-- I mean, these are all, look.
Low mass.
Hydrogen, oxygen, nitrogen.
They're not distinguishable.
They're all low-Z elements.
So they had to do wet chemistry, and they would actually peel off the R groups, which were called the residuals.
And so 3.6 residuals per turn give you this alpha helix structure.
there's a second structure that works.
The second structure is between two chains.
So in this case, instead of going coiling by yourself, and forming the galleries, what you do is you take two chains side by side, and you pace them so that you get maximum amount of hydrogen bonding.
I showed you this with nylon, how two strands of nylon bond together.
They can form hydrogen bonds.
Same thing here.
And now, since these are jerking in a motion of six in a plane, six in a plane, six in a plane, six in a plane, what you end up with is this pleating.
It's hard to see here.
Maybe if your eye can follow along the top here.
Here's sort of a cream-colored one, orangey-cream, and it's moving sort of southwest to northeast.
And here's one that's sort of a grayish, and it's moving from northwest to southeast.
And then up, and then down.
And zig and then zag.
And this is called the pleated sheet.
This is the beta form.
The alpha form is the helix, and the beta form is the pleated sheet.
Same thing up here in a different kind of model.
Which I look at that and I go, I don't get any information from this thing at all.
But they put it in the books.
Who cares.
This is where you learn something, right here.
Once you appreciate that those six atoms have to be in a plane, and how to maximize hydrogen bonding.
So let's put that down.
This is all Linus Pauling.
So secondary.
So you know that six atoms in plane, which leads to resonance.
And now, from there, you just needs the genius of Linus Pauling, and then you conclude that to maximize hydrogen bonding, you form the alpha helix with hydrogen bonding between galleries.
All right?
And then the second one is the pleated sheet, or people just call it the sheet.
But to me, the sheet means nothing!
It's the fact that it's pleated.
So I'm going to ignore what the books say.
Pleated, pleated sheet.
OK.
So this is hydrogen bonds between galleries of the same protein.
Whereas this is hydrogen bonding between macromolecules.
So if they're both pleated, it's going to work.
Or of course, that could be folding back on itself.
Remember how I showed you the first day the chain, how the chain wanted to zig-zag, to crystallize?
Do you think protein is liable to fold?
You've heard of protein folding?
Why does it fold?
Because it wants to maximize hydrogen bonding!
And when it folds, it's not going to maximize hydrogen bonding unless it is in the beta pleated sheet arrangement.
Otherwise, what's the point of folding?
Because you can't make the link.
Because I'm the oxygen, the hydrogen's way over there.
If that hydrogen were right over here, we could form the bond.
And then the third one is random.
So let's take a look.
I think I've got some images here.
Oh, here's another one, showing the-- yeah.
I'm trying to, I'm really working hard.
Here's six, here's six, here's six, and there's the common carbon.
OK?
And now you can see the hydrogen bonds forming between.
So this could be one protein, this could be another protein, or this could be the same protein that's folding back on itself in a pseudo-crystallization ploy.
Yeah.
All right.
So if you look in the biology literature, you'll see drawings like this.
You've seen these, huh?
You look at them and go, what are they talking about?
Now you know!
This is the secondary structure.
Let's start over here at the amino end.
So you're moving along, and it's just sort of random, and now, all of a sudden, look.
Round and round and round.
What are you looking at?
For this run of length, it's in the alpha helix form!
And then it kind of goes random for a little bit, and then it goes green.
And this green area is pleated sheet.
So if the pleated sheet goes this way, and then it goes random, and then the pleated sheet goes this way, what's going to happen?
there's going to be hydrogen bonding here, hydrogen bonding here, hydrogen bonding here.
And you know, how is it that this thing decides, oh, somewhere around here, I think.
I think I'll just go into an alpha helix arrangement.
Why is it alpha helix here, and not alpha helix here?
Because of the instant choice of R groups!
The instant choice of R groups will dictate whether it can turn around.
Because if you've got glycene with its dinky little hydrogen on one side, you're compact and you can move around.
But some of these other pendant groups are big, and you can't just shove them one into the other.
They'll repel.
They won't fit.
So this dictates-- so in essence, the secondary structure is set, the table is set by the primary structure, isn't it?
This is so cool.
Then there's the tertiary structure.
Because random isn't, as they say in California, it's not totally random.
There's some order to it.
So let's look at the order that comes.
See this?
Look over here.
You see up here?
This is supposed to be random coil.
But look at it.
It's kind of dentate there, isn't it.
See, it's sort of tooth-like.
Do you think that the artist just did that?
Why is it like that?
Because of the molecular forces!
So let's think on it.
Oh!
This is a little piece from Guys and Dolls.
1951.
The same your Pauling enunciated this.
I bet you think that when people talk about the chemistry between folks, that it's a '60s phenomenon.
You know, the hippies, drug culture.
No.
It goes way back.
Goes to the '50s.
Listen to this.
This Sky Masterson, and he's going to tell Sarah-- she's the one that's the, she plays the-- what do you call it-- she works at the Salvation Army mission.
And he's a gambler, which in those days was really, you know, like low-life.
And he's flirting with her, and she, of course, has nothing to do with him, and eventually, they fall in love, of course.
But anyway, so she's told him how she's going to meet the man of her life.
And she's very linear.
He's got all of these characteristics, and so on and so forth.
Real linear.
And this guy's a gambler, so he's going to tell her how he's going to find the love of his life.
[BEGIN FILM PLAYBACK] Do you want to hear how a gambler feels about the big heart drop?
No!
Well, I'll tell you.
Mine will come as a surprise to me.
Mine, I leave to chance and chemistry?
Chemistry?
Yeah, chemistry.
Suddenly I'll know when my love comes along.
I'll know-- [END FILM PLAYBACK] So there you have it.
From 1951.
Chemistry, again, chemistry!
Chemistry is the central thing.
OK. Now let's go forward.
All right.
So now let's figure out why-- and now I want to answer the question, why is this forming a tooth-like structure?
Why?
There's a reason for everything.
All right.
So here we are, moving along a length of random coils.
So we're in zone three here, random.
But it's not completely random.
So this is the tertiary structure, and it has to do with interactions between the R groups.
So let's look at number one, zone one here.
We've got two cysteines, and cysteine both have sulfur in them.
It turns out that the sulfurs can form a covalent bond here.
This is between two R groups along a certain length of chain.
And once that covalent bond forms, this chain now can't just go flopping any which way, because it's being held in place.
It's like you put a prop there.
Over here, what do we have in zone two?
We have one R group that's got a hydroxyl.
Another one's got an oxygen.
There's a hydrogen bond forming between side groups.
And in order to form that hydrogen bond, can you see that this coil is actually bent?
You see, there's a kink here.
This coil should be much longer, if it were straight, distended.
But it's actually compressed a little bit in order to facilitate the formation of this hydrogen bond.
Which means now, this coil will stay in this configuration.
And then over here, what do we have?
Here we have a substituent group that's net positive, here we have a substituent group that's net negative, and we have an electrostatic force.
This is coulombic.
This is plus attracts minus.
And they form.
And over here, this is very interesting.
What do we have here?
We have a whole bunch of R groups that are all non-polar.
And this thing's sitting in an aqueous solution.
And what do you know about non-polar groups in water?
They're hydrophobic!
So I've got this one hydrophobic group sticking out over here.
It's just going to have to take it.
Because it's bound to the backbone.
But I've got another one over here, and the backbone is flexible.
And I've got another one over here.
Can you see that if the backbone folds around, I can collect all the hydrophobic entities, and present to the water the hydrophilic opposite side of the backbone.
Because if the R group here is hydrophobic, look what's on the other side.
These guys.
So if I can take all of these and cluster them, then I sort of minimize the energy.
Because that there is a repulsive energy term, right?
The water doesn't like the non-polar R group.
So let's minimize their contact, you know?
If you've got two siblings that are fighting, you put them at the other end of the table.
You don't sit them next to one another.
So here's what you do.
You cluster all of this stuff, and that causes big hairpin turn.
So this is called hydrophobic interactions.
And that's the tertiary structure.
So you see all of those.
That's great.
OK.
I see we're getting close to the witching hour.
So maybe I'll teach you how to do your laundry and send you on your way.
So this interaction I'm just showing you right here is the same thing that's happening when you do your laundry.
Now, what's a detergent?
A detergent is this long molecule that's got-- you see this zig-zag here?
That's the aliphatic.
That's the hydrocarbons chain.
And it is hydrophobic.
The soil on your clothing is usually some dirt, but it's covered by some grease and oil.
So if you just soak it in water, the water can't get at the grease and oil.
So what you do, is you put these molecules in that are very long, and then they've got this COO ending, right?
And now what happens?
This can form hydrogen bonds to the water, whereas these long hydrophobic tails can bond to the grease and oil.
And then you put this in the washing machine, and what's the washing machine do?
Mechanically agitate.
So you've got these tails, hydrophobic tails, stabbing the grease and oil, bonded to the water, shake the living daylights out of this, and release the grease and oil, which then will release the soil, rinse the whole thing, and you've got clean clothing.
So it's the bonding here.
What you've got is an amphipathic molecule with a hydrophobic tail and a hydrophilic head that allows you to do your laundry.
All right.
Get out of here.
We'll see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
It's 11:05.
OK, let's get started.
Last day we talked about protein structure.
We talked about composition, which is the primary structure of proteins, and that's the instant R group sequence, as you go down the backbone.
The secondary structure proteins is all about packing, and we saw that there were three different structures, the alpha helix, the beta pleated sheet, and the random coil.
And the gambit here is to maximize hydrogen bonding.
And why you choose alpha, beta, or random coil depends upon the instant R group sequence, which is why in some instances, you can't form the coil.
Because the R groups won't allow it.
And then lastly, we talked about tertiary structure, and that's conformation, and that talks about our group interactions, and ultimately explains protein folding.
Today I want to talk about denaturing of proteins, which is disruption of secondary and tertiary structures.
But before we do so, I wanted to show you.
I found another movie reference to chemistry.
This comes from 1957, and the movie is called Silk Stockings.
It's based on an old Cole Porter play.
And this stars Cyd Charisse and Fred Astaire, who were both fantastic dancers.
And I want you to see in this film clip how she carries herself.
And she actually sings this.
She didn't have somebody dub it over.
So the singing is OK.
It's not great, but you've got to see her posture.
She's so graceful.
And Fred Astaire is a terrific dancer, but he doesn't dance in this sequence.
And last thing you need to know here is, this is set during the Cold War.
So she's playing Ninotchka Yoschenka, a good Ukrainian name, and she's a Soviet representative.
And she's fighting with Fred Astaire, who is an American agent.
And they're talking about the differences between the Soviet and the American systems.
So that's the background.
It's 1957.
I guess we better get some volume up here so we can hear it.
And she will explain to you-- and remember, just parenthetically, that my particular interest in chemistry is, of course, electrochemistry, which is the highest form of chemistry.
Now she'll explain it to you.
[BEGIN FILM PLAYBACK] -If you studied, Kamachev, you would know what I am talking about.
-Who's Kamachev?
-He was one of our greatest scientists.
He has proved beyond any question that physical attraction is purely electrochemical.
-You don't say.
-Kamachev has proven it!
-For 30 years, he worked.
-I happen to have worked on the same subject for about the same amount of time, and I have very good reasons for believing otherwise.
-Facts are facts [SINGING] -When the electromagnetic of the he-male meets the electromagnetic of the female, if right away, she should say, this is the male, it's a chemical reaction, that's all.
And though you Fascists may answer with kisses, the same applies when your Mr. And Mrs. Hey-diddle-diddle with middle-class kisses, it's a chemical reaction, that's all.
Say in love, with you, I fall.
And in love, with me, you also fall.
Though the uninstructed faction calls it mutual attraction, it's a chemical reaction, that's all.
[END SINGING] -You don't believe in Kamachev?
-No, ma'am.
[END FILM PLAYBACK]
See the posture?
It's just unbelievable.
Fantastic.
OK.
So let's talk about denaturing of proteins.
So we want to talk about how we can alter the structure.
And we do so by invasive means.
So let's get to the next slide.
So here's the tertiary structure, and I can point to that.
The first thing I wanted to draw attention to, is we can disrupt that structure by change in temperature.
So temperature is the first agent that we can use in disrupting, or denaturing, protein.
And the whole idea here is to break bonds.
But we're only going to break secondary or tertiary bonds.
We're not going to do anything to the backbone.
And a good example is what happens when we fry an egg.
The egg has protein in it.
One of them is ovalbumin.
It's about 90% water, about 10% protein.
And its native state, its natural state, conformation is a tight ball.
And the diameter of that ball is small enough that uncooked egg is transparent to visible light.
Egg white is transparent to visible light in its natural form, but when we heat beyond the denaturing temperature, that changes the bonds in here, and it unpacks.
And when it unpacks, two things happen.
First of all, the length scale changes to be great enough that it scatters light.
So now the egg white appears white.
And secondly, those various chains entangle, and that gives it its rubbery character.
So all of that is happening as a result of the increase in temperature.
So second way that we can operate, is to change pH.
And what that does, is it changes hydrogen bonding and electrostatic interactions.
Now I draw your attention to zone two here.
In zone two, imagine if we take that proverbial drink of cola, and now the pH in here goes way, way down.
You can see this N has got a hydrogen bond over to here.
Now, if there's a proton excess, the proton can cap, because this is a proton attachment site.
If the proton caps this carboxyl, then this hydrogen bond is broken, and now this thing can unfurl.
Over here, you see, this is an electrostatic attraction.
Again, the carboxyl could be kept by a proton, and now the plus minus electrostatic attraction is lost.
So by change of pH, we could denature what we see here.
So there's a good example.
And in fact, I think we do this.
We pickle our foods.
And then this one I have, because of my interest in high temperature electrochemical processing of metals, I've been to Norway many times.
And this is a food that they will never let me eat, because they're afraid if I ever taste this, I'll never return to Norway.
But this is lutefisk, which literally means lye fish.
And lye has high pH.
And this is an example of denaturing protein by going to very, very high pH.
And then you soak it and draw back out and so on.
So there's a good example of food.
Now the third one I want to talk about is oxidizing reducing agents to denature proteins.
And what they do, they can either create or destroy the sulfide linkages, the sulfur-sulfur linkages.
So for example, you see at position number one.
Position number one, we can use oxidizing and reducing agents.
And a good example, that comes from hair.
So first of all, I want to talk to you.
Hair is protein.
And this first example is just, how protein can, one strand can bond to the other by hydrogen bonding.
So up here you have one strand and a second strand, and there are hydrogen bonds in your hair.
Now, what you do when you want to change your hair by blow drying is first you wet the hair.
And the water goes in and interrupts these hydrogen bonds between strands.
So now each of these cross-links that want to form instead terminate with a water molecule.
And now with the water molecules, now you can move the hair one strand versus the other, hold it in place, blow dry, get rid of the water, and form new hydrogen bonds.
So what, on the basis of what I've just told you, how can you account for a bad hair day?
What is a bad hair day?
A bad hair day is a day in which it's rather humid.
And so water can get in from the atmosphere and interrupt these bonds, and then the hair will relax, and go back to a native state that is unfavorable vis-a-vis what you tried to achieve with the blow dryer in the mirror.
So you want to combat this.
So what do you do?
Well, you bring out covalent bonds.
So here's two strains of hair, and there are disulfide linkages between the strands of hair.
So now with a reducing agent, reducing agent is going to be a proton donor.
And can you see, these disulfide linkages are capped with hydrogen, and now they're terminated.
And so now the two strands of hair are free to move relative to one another.
And now when you're at the beauty parlor or the hairdresser, now you can take this hair, for example, and wrap it around-- what do you call this thing?
It's a mandrel in the metallurgical word.
What do you call this thing?
Curling iron or something?
Yeah.
OK.
So here you are.
You wrap this around-- I know what it does.
I don't know the terms, all right.
So now you see, we've got the hair now wrapped around in this fashion, and now we use an oxidizing agent.
The oxidizing agent is peroxide here.
You know the derogatory term, peroxide blonde.
There it is, right here.
So the peroxide goes in, and now it removes these hydrogens.
The peroxide takes the hydrogens away and reforms the disulfide linkages, but now the hair is in this curled state.
You know, you might have the opposite.
Maybe in your native state, your hair, because of the way the R groups are formed along the length of your hair, you know, it's always, the ones with the curly hair want the straight hair, the ones with the straight hair want the curly hair.
And so, you know, maybe you could be running this sequence in reverse.
But in this case, somebody obviously wants curly hair.
Now they form the disulfide linkages.
So this is the object, here, of denaturing.
So now when we go over to here, we have a disulfide linkage, which is forcing this run of the random coil to stay straight.
But if we get, in this case, the oxidizing, the reducing, and what will happen is that we will break this.
So the reducing agents will destroy disulfide linkages, and the oxidizing agents will create disulfide linkages, and thereby we can distort the natural form of the protein.
And here's a fourth example.
There are many others, but I'll just give you four here.
In the fourth one, we can use detergents.
And last day, I showed you the example of how to do your laundry.
And so you've got this long aliphatic tail and the hydrophobic head, and you can imagine if you take such species and you've got this hydrophobic pocket, the species will operate in such a way as to pull these non-polar entities out, and then destabilize this hydrophobic pocket.
So depending on what the nature is of the protein, these various chemical and thermal actions can lead to denaturing.
So this destabilizes hydrophobic pockets.
All right.
So that's pretty good.
All right.
Well, I think this is where I want to leave it with the treatment of proteins.
And I want to move on to a second biomolecule, and that's the lipid.
We'll say a few words about lipids.
Lipids are not classified on the basis of their composition, but rather by their properties.
So they are defined their properties, which is kind of unusual.
Normally we define things in terms of their chemical compositions.
And in particular, they're soluble in solvents of low polarity.
Some of the books say nonpolar solvents, but even something that's polar but only mildly so will work, in solvents of low polarity.
So that's the example.
They're insoluble in water, they're oily to the touch.
And this includes things like fats, oils, these are all lipids.
Cholesterol, hormones.
These are all members of the lipid class.
And so let's look at some examples.
We've got some slides.
So if you start over here with the glycerol, this is a trialcohol.
And you can see the OH.
We're going to drop the H's, and we're going to put these long aliphatic chains on.
There's 14 carbons here, plus the two, so it's 16 carbons long.
And this also is seven plus seven, and then the two carbons.
And the difference here is that in this case, it's all straight chain.
In this case, we go for seven, then we put in a double bond.
And by doing so, when we get to the double bond, that forces everything to go into planar 120 degree arrangement.
So you go for seven however you want, and then there's this rigid 120 degree placement, and then you go for seven however you want.
Well, clearly, the one in the middle is going to pack better than the one on the left.
And so even though they have seemingly the same chemical composition, the one in the middle is a solid.
It's a fat.
The one on the right is a liquid, because it doesn't pack well enough.
The bonds aren't strong enough.
So these are both examples.
And this is called a palmitic acid.
And you can see here oxygen acting as a bridge, as an ester.
So oxygen, over and over again, acting in this way to give us the ability to make these other longer structures.
Now we can replace that oxygen bridge with a dibridge.
So instead of having just an oxygen here, we're going to put a phosphate.
So with the phosphate, we've got a phosphodiester linkage.
There's one ester, there's a second ester.
And why we doing this?
Because nature wants to get the spacing right.
And remember what I just said, because by the end, you're going to see that spacing is everything.
So we can use this, and we can even lose that oxygen, and put something else here.
So here's a-- I'm not expecting you to know these by heart.
I would give you the structure and tell you what it is, tell you its name, and then we can go on from here.
So what do you see here?
Well, this is called a phosphatide.
And this one is a phosphatidylethanolamine, because there's the amine here.
And look at this thing.
Well, you've got a phosphate in the center that's the bridge.
This is the glycerol, the three carbons.
In its most primitive form, we could just have hydroxyls here.
This is the fatty acid off the one end.
Long chains here, not to scale.
So these things should be way, way over to here.
And over here, we have an ethanol amine.
But look more closely.
Well, this is a hydrophilic head, this is twin hydrophobic tails.
Look, this is just hydrocarbon.
It's not soluble in water.
So now you've got this amphipathic molecule.
It looks like a detergent, doesn't it?
This long, hydrophobic tail could stab the grease, and this hydrophilic head can bond to the water.
And now if we shake things up, we'll release the grease that's binding the soil to your T-shirt.
Got the phosphate bridge here, and look even more closely.
There's a minus, and there's a plus!
This is zitterionic, on top of everything else.
So this can function as a zwitterion, too.
And everything we learned about zwitterionic chemistry and buffering, this thing can do.
That's cool.
All right.
So now I want to show what happens when I take a whole bunch of those.
All right?
So let's go back.
I want to keep this one up, and I want to want to do is take a whole bunch of those, and show how lipids can actually operate in order to build complex structures for us.
So what I'm going to do, is I want to represent that molecule.
So here's the hydrophilic head.
That's the amine, right?
And then we've got twin hydrophobic tails.
And that's your carbon sp3 chains.
And I don't want to write that so much.
So I'm just going to simplify that as simply hydrophilic head and hydrophobic twin tails.
So this now represents the twin tails.
Now if I put a whole bunch of these in water, what's going to happen?
First of all, this is hydrophilic.
And what if I were to just pour this into a beaker?
What do you think would happen if I introduced this into a beaker of water?
Well, this is hydrophobic, this is hydrophilic.
Can you imagine that they would all sort of line up like this?
We're trying to stick the hydrophobic tails up into the air.
Because they don't want to be down in here.
This is hydrophobic.
Doesn't want to be here.
So now suppose I put a whole bunch of these in water.
More than four, right?
With only four, that's the best can do.
What if I put some molar?
So then what it does, is it forms a pocket.
It's going to do this.
All the heads are going to find each other, and all of the tails are going to find each other.
Because now they're going to form a hydrophobic pocket.
So they'll do it this way, because they can-- can you see how the heads can sort of make a wall against the water?
And you go on and on and on, not to scale, and then finally you have some end effects here.
So what have I formed here?
I've got something, first of all, I've got a lipid layer here, and a lipid layer here.
So together, I have a lipid bilayer.
In here, there's a pocket.
And now, what I can do with this whole thing, is I can build a cell wall with this.
I've got a cell wall.
I've got an outside and an inside.
So what happens is, when we put all these things together, because of the clustering of the hydrophobic and hydrophilic, we say this thing is endowed with the property of self-assembly.
Very hot topic in material science.
And that leads to cellular structure, and I think we've got a nice cartoon here.
So this is taken from one of the readings.
So you can see this with the plurality of these.
They're actually showing the twin tails.
Now, this was taken from another text.
I like this one, because it shows the hydrophilic top, hydrophilic bottom, and the hydrophobic interior.
And then this thing is called an integral protein.
Why?
Because it's integrated into the cell wall.
And let's think about this protein for a second.
What must be the nature of the R groups in this vicinity of the protein, that it sits where it does?
The R groups around here must be dominantly hydrophilic, otherwise it would get dumped out of that zone.
And in here, it must be dominantly hydrophobic, so that it feels at home with all of these hydrophobic tails.
And over here, it must be hydrophilic.
Now imagine what happens.
I'm going to go back to that drink of Coca Cola.
Forgive me, no brand names.
Cola.
And now the this zone here gets flooded with protons, because the pH is dropping.
So that could cause conformational changes, because pH changes the conformation of the protein, and it could cause this thing to change shape.
It could unfold, or it might unfold in such a way as to open a channel down the center here.
And so in response to a change in pH here, we open up a channel, which means this is acting as a chemical gate.
This is how things are animated!
It responds.
And then once the proton concentration has been depleted and you're back to a more neutral pH, then this thing changes back to its old confirmation, and the gate closes.
It's that simple!
The secondary bonding explains animation.
That's what's happening.
And there's one other cool thing.
Look at this.
If they all have the same head, They're going to close pack!
It's fantastic.
Everything.
Everything you need to know, 3.091 here.
All right.
So what is the key to animation?
The key to animation is self-assembly.
And why do we have self-assembly?
Because we have these molecules with the hydrophilic head and the hydrophobic tail.
These molecules are called amphipathic.
OK.
So this has hydrophobic and hydrophilic components.
So if you take amphipathy, you end up with self-assembly.
That's the key.
All right, good.
All right.
I think that's all I want to say about lipids.
That's all you need to know about lipids, is lipids will give you cellular structure.
Now we've got a little bit of time left, and we're going to talk about nucleic acids.
So what do I want to say about nucleic acids?
Nucleic acids carry information.
They carry information that directs metabolic activity, including replication.
And something can be animated, but it doesn't count as a life form unless it replicates.
That's the characteristic of life forms.
So nucleic acids carry information that directs metabolic activity, including replication.
So these are macromolecules.
Not polymers, but macromolecules.
We'll take a look in a moment at their structure.
They're macromolecules, and the basic structural unit is called a nucleotide.
And it's got three building blocks.
Every nucleotide has three building blocks.
See, what we're doing right now, is we're really tying together all of that chemistry you've been learning the entire semester.
And it's fun to see it actually go to use.
So what are the three building blocks?
Every nucleotide has a sugar.
And we didn't study carbohydrates, but I'll just show you the structure the sugar.
It's got an amine.
You know what that is.
That's the NH something group.
And it's got a phosphate.
So the nucleotide has those three components, So let's take a look at the sugar.
There's really two types of sugars found in amino acids.
There's the ribose-- and the only reason I'm showing you this, is so that you'll understand the terminology.
There's the deoxyribose.
So if you look here, the sugar on the left has this five-fold symmetry, the five-fold ring.
And there's hydroxyls at the number one position, number two position, number three position.
The deoxyribose is missing the hydroxyl at the number two position.
That's the difference.
The reason I'm showing you this, is this is called deoxyribose, and ultimately this is the D in DNA.
So you'll be able to at least hold your own with your course seven major friends, if you have any friends in course seven.
And you can say, I know what the D is.
It's deoxyribose.
OK.
So then the amines.
The amines we've got, there's five of these.
And they split for RNA and DNA.
And they're shown here.
So it's interesting.
There's two of them that are called purines, because they've got this ring structure.
And then the six fold, and then the five fold.
So these are the purines, and then the pyrimidines have just the six fold structure.
There are three of those.
So A, G, C, U, and T. Those are the five different amino acids.
And the difference is that in DNA, you only have A, G, C and T, whereas in RNA, you have A, G, C, and U.
This is the chemistry.
If you take 7012, you'll figure out how all of this other stuff goes.
And this thing here is phosphate.
And why is phosphate present?
Because it's acting as the bridge and a spacer.
So let's take a look at the structure.
OK.
There's more of the amines in nucleic acid.
So this is what it looks like.
So you have a backbone that consists of sugar phosphate, sugar phosphate, sugar phosphate, shown here.
So in this case, you've got the deoxyribose.
So there's the sugar, there's the five-fold symmetry.
Then the spacer, the phosphate, then the sugar, and then the spacer, the phosphate.
And you can see that the sugars are acting as hangers for these amine groups.
And they're called bases, because in fact they act as Bronsted bases.
They're proton acceptors, and in the early days of molecular biology, people determine the chemical composition by wet assay, and these were determined to be Bronsted bases.
And so to this day, people refer to them as base pairs, et cetera, et cetera, even though now we know they're amines, et cetera.
So this is the structure going up.
And you see, at one end, you've got the three, and going up to the five.
The three is, you have the carboxylic acid end, and at the other end you have the amino end.
OK.
So there the structure.
And you have a choice here.
So again, the sugar hanger, the phosphate spacer, the sugar hanger, the phosphate spacer.
And you have a choice of one of these four.
Any one of these four is what's found in a DNA strand, as you go up the strand.
Now, how is information encoded?
Well, the information is encoded-- first of all, we have to recognize what the structure is.
Know that the structure, turns out that it forms a double helix, which is the first secondary structure, right?
It doesn't have.
The primary structure is this instant A, C, G, T sequence all along.
And then the secondary structure is, in order to maximize hydrogen bonds, this forms a double helix with a second chain.
And furthermore, the pairing is such that A always pairs with T, because they have two hydrogen bonding sites between them, whereas C pairs with G, because they have three hydrogen bonding sites between them.
And furthermore, look at the spacing here.
This always just stuns me.
If you look down the center of the strand-- so I've got two strands of nucleic acid.
The center to center spacing is 1.085 nanometers between A and T, and it's 1.085 nanometers to four significant figures between C and G. And you might say, well, is it possible that I could take two of these hydrogen bond sites and line them up to two of the three hydrogen bond sites?
And the answer is no, because the spacing is wrong.
You can't, these two are far enough apart that they won't line up here.
So it's guaranteed that it's always A to T, C to G. All right.
So now you see the double strand.
You have one strand moving up.
You see the pentagon is pointing up, and here you see the pentagon is pointing down, and we have the base pairs in between.
C pairs to G, and T pairs to A.
And down here is the basic end, the acidic end, the basic end, the acidic end.
And so the secondary structure is the double helix.
Why?
To maximize hydrogen bonds in between, and then to keep maximizing hydrogen bonds.
Hydrogen bonds between the galleries.
Maximize, maximize, maximize.
So there's what they call the base pairs, and et cetera, et cetera, et cetera.
This distance is all prescribed by the fact that we have the phosphate present.
OK. Now where's the information here?
I said this encodes metabolic information.
Well, suppose I want to direct protein synthesis.
I want to put amino acids in a sequence.
So I have to be able to call.
You know, I'm sitting here as nature, and I'm saying, I want to make this protein.
So I need this amino acid, and then I need this amino acid, and then maybe that amino acid, and put them in sequence.
So I need 20 different words to call 20 different amino acids.
Well clearly, clearly if I have just A, C, G, and T-- so if I have one-letter words, it won't work.
Because I need to call out 20.
I need 20 amino acids.
So how am I going to get to 20?
I don't have 20 of these.
I'm only using, there's only four of these.
So I say, oh.
I know what to do.
I can use this idea.
The number of words will equal the number of letters that I have in my alphabet, raised to the power, number of letters per word.
OK?
So if I use this idea, then I can say-- what if I have two-letter words?
So in other words, to call out an amino acid, I have to take two base pairs in sequence.
That means I'll have-- there's only four letters.
But if I have two letters per word, that's 16, which is still less than 20, and that's no good, because I can only call 16.
So what does nature do?
What if I had three-letter words?
Three-letter words, then, is a 64, which is greater than 20, and that works.
So now I've got, in point of fact, 61 of these three-letter words.
So I go down the sequence.
First base pair, second base pair, third base pair.
That triad represents one amino acid.
I've got 61 to call out 20, And I've got three left over.
For what?
Well, let's think about this.
See this?
How do you know to read this, defined by their properties?
You know to read from left to right.
You know that there's a space here between words.
So if I just give you this strand of DNA, where do I begin?
Do I take these three, or do I start counting from here or here?
So built into this, it tells you to read either from left to right, or right to left.
It tells you where to start, and it tells you where to stop.
These three-letter words, by the way, are called codons.
So we've got sixty-one to call out 20 amino acids, which means, some amino acids have more than one name.
There's one of them that has six different names.
There's six different codons that can call it a given amino acid.
For some of them it's only one, for some it's two, three, et cetera.
OK.
So that's the information.
Look!
Here it is!
There's the information!
It's all in there.
Just hydrogen bonds like this, hydrogen bonds like this.
It's all bonding.
So here's the codon.
So somewhere here, see, we take this, this, and this.
This triad represents one piece of information.
All right.
So let's go back and look at some of the history.
Who got us to this point?
This is fresh!
A lot of this happened in my lifetime.
All right.
So Oswald Avery-- this is before my lifetime.
He was working at the Rochester Institute.
And he was the first person-- he was born in Halifax, actually.
And he worked professionally here in the United States.
And he was the first person to recognize that nucleic acids store and transmit genetic information.
Up until the 1940s, people thought nucleic acids-- are you ready for this?-- are too complicated.
Their structure was too complex to contain information.
It's precisely the complexity that gives us the abundance of information!
The prevailing belief was that proteins contained the genetic information.
But people couldn't figure out how to make it work.
So he was the first to make the argument that the nucleic acids have the information.
And then the second giant is Erwin Chargaff.
Erwin Chargaff worked at Columbia in New York, and hospitals in the vicinity.
And he was a painstaking analytical chemist.
He did all of this work by analytical chemistry, and he gave us Chargaff's Rules, 1949.
And he said that in any biological system, the concentration of a-- remember, they didn't know the structure of DNA yet.
I'm going to lead up to how we get to the structure of DNA.
So we're going back, this is like a flashback in a movie.
Now we're going back to see how we got to the double helix, all right?
People knew that the concentration of A in any nucleic acid was equal to the concentration of T. They knew that there were sugar present, they knew there were amines present, and they knew there were phosphates present.
That's all they knew, based on chemical analysis.
And now he tells us that, of the amines, concentration of A always equals concentration of T, concentration of G always equals concentration of C. And then the sum of concentration A plus G is the sum of C plus T. That's Chargaff's rule.
And so here we are, Homo sapiens.
31, 31, 19, 19, a little bit of statistical error, et cetera.
But look!
Corn!
Yes!
Corn reproduces using this same code.
All living organisms on this planet use the same code.
Which makes sense!
If I eat corn, and I'm going to get nutrition from it, and it's going to help me build cells, and regenerate my body, and get energy from it, it has to be made of this same stuff, otherwise my body can't recognize it!
Because I'm a biological machine.
This is not a fireplace in here, right?
If it's a fireplace, you can drink something combustible and boom!
You've got energy.
But we can't derive energy from something combustible.
We can only derive energy from something like this.
So, you know, when you think you're really hot stuff, your DNA is not that different from that of an ear of corn.
So a little bit of humility might be in order.
All right.
And then the last person we want to talk about is Rosalind Franklin.
Rosalind Franklin worked at the King's College in London, and her specialty was x-ray diffraction.
And she made painstaking experiments where she could take a strand of DNA with a stainless steel needle and pull it out of an aqueous solution, and hold it in a chamber where the chamber was humidified so that the DNA didn't dry out.
And while it was humidified in this chamber, take an x-ray diffraction exposure for hours, because all of the recording was done by film.
By halide photography, and you had to have long exposure time.
This is one of the most famous images of the 20 century.
This is the image of the DNA from calf thymus, the beta structure.
And this is a Laue pattern.
So the symmetry of this pattern is reflective of the symmetry in the DNA structure, which people don't know yet.
And what you see is five positions.
One, two, three, four, five.
Four is missing.
There's no reflection there.
But it's where you would expect a position to be.
So it's sort of Bragg-like, isn't it?
Not all of the lines are reflecting.
On the basis of this image, this is what we learned.
We learned, first of all, that the Laue pattern is indicative of a double helix.
Based on X pattern.
Remember, they had no computers in those days!
All they had was a pencil and paper.
So to do the Fourier transform, and go back out of K-space, and discern what the pattern is-- you know, today it's trivial!
They did this all longhand.
Number two.
On the basis of that pattern, you learn that it's 3.4 angstroms between nucleotides along the backbone.
We don't know what the backbone is!
We just know that there's sugar, there's amine, there's phosphate.
There's different ways of putting this together to make a chain.
And the third thing that you learn, because the strong, clear lines indicate that the heaviest elements must lie outboard.
What does that mean?
If you've got a double helix, based on the first finding, what's the heaviest element?
Phosphorus.
Because otherwise you've got carbon, oxygen, nitrogen.
Nitrogen is atomic mass 14.
Phosphorus is atomic mass 30.
So it must be outboard.
Otherwise, if it were inboard, there would be other elements farther out, and you wouldn't get the clear lines.
Because the heavy elements are the ones that give the reflection, and if the heavy elements are obscured by lighter elements that are shielding them, you get a lousy reflections.
Blunted reflections, blurry reflections.
So all of that comes out of her findings.
So what happens next?
So there's the 3.4, and 10, and et cetera, et cetera.
All of that happens based on the pattern that I just showed you.
So I'm going to read to you what happens in those days, back in the 1950s.
There was intense competition.
You know, Linus Pauling had proposed a triple helix, and people in the UK were working hard on it, and so on.
A lot of competition.
So what happens is that Watson goes down to King's College in London, and he meets Rosalind Franklin in the hallway, and they get into a big argument.
And she really chews him him out, and he goes running away.
She was very tough.
And so he goes to Maurice Wilkins, who was her boss.
And I'm reading, now, from Freeman Judson's book.
"Wilkins told Watson as they went down the hall that Franklin had found that DNA fibers, when kept wet, yielded a different x-ray pattern, suggesting a second structure.
Fourteen months after the King's colloquium, despite repeated correspondence and conversations, visits, meals together between Wilkins, Watson, and Crick, the possibility of a second structure was news to Watson, he wrote."
Now, this is quoting from Watson's own book, The Double Helix.
'When I asked what the pattern was like, Maurice went into the adjacent room to pick up a print of the new form they called the beta structure.
The instant I saw the picture, my mouth fell open, and my pulse began to raise.
The pattern was unbelievably simpler than those obtained previously, the a-form.
Moreover, the black cross of reflections which dominated the picture could arise only from a helical structure.
With the a-form, the argument for a helix was never straightforward.
With the b-form, however, mere inspection of its x-ray picture gave"-- listen to this-- "several of the vital helical parameters."
I'm going to come back to that.
"The picture that Wilkins showed Watson was Rosalind Franklin's, without her approval."
So then Watson goes on the train back to Cambridge, and sits down with Crick, and in no time, they've put the structure together.
And this is a famous picture.
This is Crick.
This is Watson.
And I don't know if you can see really well.
These are lab clamps that they're using.
This is a lab stand.
They're using lab clamps to put the structure together.
So here's the paper.
The paper as it comes out, Molecular Structure of Nucleic Acids.
And this was the first image of DNA.
And I submit to you that if the earth were to end in a giant explosion, and we were to only send two images in a spaceship out to let some other race discover what we're all about, it would be this image, and the image of the atom.
That's all you need.
Because you have the simple atom, and you've got this, and everything else is just detail in between.
So there's the first image.
All right.
"We wish to suggest a structure for the salt of deoxyribonucleic acid.
The structure has novel features which have considerable biological interest.
The structure for nucleic acid has already been proposed by Pauling and Corey.
They kindly made their manuscript available to us in advance."
These guys never made their manuscripts available to Corey and Pauling.
"Their model consists of three intertwined chains with the phosphates near the fiber axis and the bases on the outside."
See, they got the positions wrong.
"Without acidic--" da, da, da, da, da, and so on.
Whatever.
Now here's-- remember, I've quoted from Watson's book.
"The previously published x-ray data on deoxyribonucleic acid are insufficient for a rigorous test of our structure."
Really?
"So far as we can tell, it is roughly compatible with the experimental data, but it must be regarded as unproved until it's been checked against some more exact results."
Remember, he's writing this after having seen Rosalind Franklin's x-ray pattern.
"Some of these are given in the following communications."
Yeah, the next paper is Rosalind Franklin's paper.
"We were not aware of the details of the results presented there when we devised our structure,--" That's a lie. "
--which rests mainly, though not entirely, on published experimental data and stereochemical arguments."
You know what?
once I've given you the data, and you know what the structure is, you can give me a very, very strong stereochemical argument.
But that's hindsight.
If you don't know what the structure is, you have no idea what the stereochemical arguments are, because you can justify anything.
So this is just absolute lies.
Who are the authors?
Watson and Crick, even though it's based on data that was taken from other people.
So here they are.
This is the acknowledgments.
We're indebted to-- blah, blah, blah.
The experimental results in ideas of Wilkins, Rosalind Franklin, and their coworkers at King's College.
And one of us, Jim Watson, he was the American, and I think there's an irony here, was aided by a fellowship from the National Foundation for Infantile Paralysis.
And I think there's a joke there, if you know anything about Jim Watson.
And now contrast that with this.
This is the publication of the human genome.
And look at all the names here.
Everybody who was involved in the enterprise was named.
This is not the paper.
This is just the names of the authors.
Now, to be literate, there's this one phrase.
Typical British understatement.
Remember, this was about the structure of DNA.
But there's this passage.
It has not escaped our notice.
This is the litotes.
Understatement, right?
"It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material."
That is the understatement all time.
We've got the structure of DNA, and parenthetically, I think this is how you direct replication in all living creatures, by the way.
So when this paper was published.
and they rolled out the human genome as a sort of a literary-- what's the word-- allusion, thank you.
The literary allusion was-- there's a passage in this paper that at one point begins "It has not escaped our noticed," da, da, da, da.
So if you're ever with some Course 7 people, and you want to really get under their skin, you know, you could be at a party or something, and finally, you say you know, it's not escaped my notice.
And they know what you're saying.
It'll make you less popular then you are now.
So let's fast forward.
This is Stockholm, December 10, 1962, the Nobel Prize ceremony.
Here is the King of Sweden.
This is Francis Crick.
This is James Watson.
This is Maurice Wilkins.
No Rosalind Franklin.
This is Kendrew, and I'm drawing a blank.
These two guys were getting the Nobel Prize in Chemistry for hemoglobin structure.
This, if you look carefully, is John Steinbeck, getting the Nobel Prize in Literature.
Over in Oslo is Linus Pauling, getting his second Nobel Prize, but for peace.
So ironically, he's getting his Nobel Prize.
They're getting the Nobel Prize not in Chemistry, not in Physics, but in Medicine.
You know, what's medicine?
I mean it's not really a science.
But anyway.
Here it is.
It's worthy of a Nobel Prize.
So what really happened here?
What really happened?
Why was there no Rosalind Franklin?
She was marginalized as a woman.
She was mistreated by these men.
It was 1955.
You realize in the 1950s-- and I'm not talking about the England of Charles Dickens.
I'm talking about the England after World War II.
Women were not allowed in the common room.
And the common room is where everybody congregates at 4:00 p.m. to drink coffee and exchange ideas.
So she couldn't go into the common room and so on.
And it was just a terrible, terrible story of misuse, of information, lack of attribution, and so on.
If this story were unfolded today, I guarantee you these guys would not be getting a prize, and in fact, the publication would probably be rescinded on the grounds of its fraudulent claims, that they were unaware of the previous information.
So after the paper was published, she was so brokenhearted that she resigned her position at King's College, and took a job at another college in London.
Her terms of severance included her signing a document in which she agreed to abandon all future work in biochemistry.
I mean, imagine that you leave your job, and they tell you what you cannot do!
I mean, obviously, if you were privy to intellectual property and so on, you're bound by confidentiality.
But no one can tell you, you can't go to work for somebody!
And so she changed fields, and made seminal discoveries in two other fields outside of the DNA work.
So why isn't she here?
She died of cancer in 1958, very young.
And you can't get the Nobel Prize posthumously.
So some people are quite outraged by this story.
But there's a symbolism here to have Wilkins.
It's basically saying that you two guys are not the authors of this.
And so some people say that Wilkins is standing in for Rosalind Franklin, and the Nobel committee at least acknowledged that.
So what's the message here?
The message here is, I hope nobody in 3.091 is ever embroiled in a controversy like this.
We have to acknowledge our collaborators.
All of us are here because somebody helped us.
Don't go away.
You haven't been dismissed yet.
This is important.
What I'm telling you is far more important than that structure.
I'm telling you how to stay out of jail.
See, here, you know, 50 years ago, you cheat and steal, you get a Nobel Prize.
Today you go to jail.
It is a Nobel Prize in Medicine, of course, but anyway.
So this is a book, if you're interested in Rosalind Franklin's story, it's fantastic reading.
Real page-turner.
This is James Watson's book.
If you read this, you know, have your you-know-what sensors on all the time.
And here's a picture of Rosalind Franklin on vacation in France.
And we'll leave with The Twist by Hank Ballard, who was another person who was marginalized, because he was black!
And so you couldn't listen to this music, and so there was a cover made.
The version of The Twist that you have heard is probably by Chubby Checker.
This is the original one by Hank Ballard.
It's gritty.
It's very gritty.
All right.
Treat each other well.
Get out of here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So today we're going to start the last section.
We're going to do three lectures on phase diagrams.
And I've given this the label here of Stability, Sustaining the Solid State.
We've talked a lot about the solid state in 3.091 as the vehicle for teaching the rudiments of chemistry.
One of the things we have not talked about, is what are the conditions that sustain the solid state?
How do I know whether something's going to be a solid, or a liquid, or a gas?
This is very important.
For example, if you're in a foundry, you're running places, making auto parts, you have to know what the solidification temperature is of the alloys, so you can get the parts out quickly and keep productivity high.
It's even important in failure analysis.
You know, when they were looking at things like the rubble from the World Trade Center by understanding phase stability, the thermal history was imprinted in the metal.
It's possible to determine what temperatures were achieved, and therefore, what the chain of events was that led to the collapse of those buildings.
Now you might say, well, isn't this straightforward?
I mean, for example, you just look up the melting point or boiling point.
Water, 0 degrees C. Melting point, boiling point, 100 degrees C. Well, suppose you decide to realize your life's ambition.
You're going to go and scale Mount Everest.
So you pay the $10,000, you get a license from the Nepalese Government, and you're at base camp at 20,000 feet.
You're sitting there, you've got your campfire going, you've got a hankering for a hard-boiled egg.
And you start, you put the eggs into boiling water and you cook them 10 minutes, you take them out, and they're still runny.
You say, I must've lost track of time.
So you cook them 20 minutes, and they're still runny.
And you cook them 30 minutes, and they're still runny.
And you eventually come to the realization that at 20,000 feet, the boiling point of water is below the denaturing temperature of egg yolk.
So now the boiling point is a function of pressure.
So you know, we could have one of these Iron Chef cookoffs, only the dish is souffle, only we'll have the contest at Flagstaff, Arizona, where the altitude is so high that the classical recipes won't work, because the boiling point, the atmospheric pressure, and everything conspire so that you're not going to be able to support the souffle.
So you're going to have to understand phase diagrams.
In fact, if you understand phase diagrams, that's your ticket to being a four-star chef in the kitchen.
Well, here's another place we can look at, where pressure has an important role.
Let's look under the hood of a car.
You're running an internal combustion engine.
So this is the engine block, and there's combustion going on inside.
We've got to keep this thing from overeating.
It could damage the metal.
In extreme, you could melt the metal.
So we've got cooling channels in here with water flow.
But then the water's going to heat up, and the water's going to boil.
So we've got, over here, the radiator.
So it's really a double cooling system.
The water cools the block, and then the radiator cools the water.
So how's that going to work?
Well, here, the temperature is on the order of about 90 degrees C, and here the temperature inside the engine block is much greater than 90 degrees C. It's several hundred degrees C inside that engine block, and we've got water going like this.
So we've got cooling water flow in this manner.
And now cold water goes in and hot water comes out, and then we get into the radiator, and we've got channels here, and we've either got a fan or some kind of air flow.
And so now, what's the gambit here?
I want to see what's going on inside this radiator.
So if this is the wall of the radiator, and in here I've got water, and over here I've got air, the idea is to have a high heat flux to get the energy out of the water and cool it down.
Now, if things get really, really hot in here-- let's say you're zooming along down the highway at about 90, I mean, about 65 miles an hour, and all of a sudden you come upon a collision, and you have to go to a dead stop, all that energy in the engine is now dumped, and the water could overheat.
And in the extreme, if it overheats, it can actually go into a boil.
And this is bad.
Because heat transfer is really good between a liquid and a solid, and it's really poor between a gas and a solid.
This is a low heat flux, and this is bad.
High heat flux, that's good.
This is the bubbles.
This is boiling.
And why?
Because what transfers energy?
It's the atoms.
And what's the atom density in a liquid versus the atom density in a gas?
You just have really crummy heat transfer.
That's why insulation involves dead airspace.
Because you have very poor atom density.
So what can we do in order to try to repress the bubble formation?
Well, we said if we went up Mount Everest, we went to high altitude, the boiling point went down.
And why did it go down?
Because the atmospheric pressure is lower.
So why don't we go the inverse here, and we'll put a pressure cap on the radiator, and make the pressure go up, and when the pressure goes up, it exerts a back pressure and represses bubble formation?
So by increasing pressure, we can increase the temperature, and get the temperature up.
So let's say p equals-- if you look on the top of the pressure cap, it will say 15 psi.
And 15 psi is almost 14.7 psi, which is exactly 1 atmosphere, which in SI units is, and I know this to six significant figures, 101325 Pascals.
Or, you know, from Torricelli, 760 millimeters of mercury, that's the column that's supported.
So we're at 2 atmospheres pressure in there, and with two atmospheres of pressure, we can raise the boiling point to 106 degrees C. And then what we can do, is now change the composition.
So we change the pressure, and I change the boiling point.
I change the composition.
add 50% ethylene glycol.
So now I've got a one-to-one mix, ethylene glycol and water.
And that takes the boiling point up to 130 degrees C, or 265 degrees Fahrenheit.
So another example of how we can manage systems.
So when I'm showing you by these several examples, is that the boiling point is a function of pressure and composition.
So we can tune the boiling point.
See we were tuning materials properties before.
Now we're tuning physical chemical properties, because that's what chemistry is all about.
It's about control.
It's about management.
Management to advantage.
So how do we know what to do here?
Can we predict this from first principles?
No.
Systems are still too complicated.
Our models aren't robust enough.
So instead, we turn to an archive.
So we have phase diagrams.
And the phase diagrams guide us in our search for better management of material systems.
And these are stability maps.
They tell us which phases are stable under which conditions of pressure, temperature, composition.
You can consider them as archives.
So that's what I want to talk about, because it's really important to know this stuff.
OK.
So what are we going to do?
So first of all, let's-- I don't want to write on that one.
This is interesting.
This could be a work of art.
I'd hate to desecrate it.
You like?
This is really good!
So I'm just going to leave that.
That's very good.
That's excellent.
OK.
So I've been talking about phase.
What is a phase?
Let's get the right definition here.
What is a phase?
It's a region uniform chemical composition.
Just to get it down.
it's a region in a material, region of uniform chemical composition, and it has these other properties.
Uniform chemical composition.
It's physically distinct.
And I'll show you by examples, but let's just get the definition down.
It's physically distinct, and in the extreme case-- so that means it's bounded.
It's physically distinct.
That's a fancy way of saying that you can put a boundary around it.
It's bounded.
And then the other property that it has, it's mechanically separable in the extreme.
And so that's in the case where you have a multi-phase system.
You can point to the different p. So what I'm going to do, is I'm going to give you some examples.
And so I'm going to let circle p equal number of phases.
I'll give you some examples.
Why am I using circle p?
Because just plain old p is already pressure.
So pressure, so this is kind of a Western motif, you know?
Like a Circle Bar Ranch or something.
So it's a p with a circle.
That's my font.
That's how I say number of phases.
So let's take a look at some examples.
So here's some examples of p equals 1.
So simple one is pure water.
Pure H2O liquid.
It has all of this.
Wherever I have the water, all right, here's some water, it's all of the same chemical composition.
It is physically distinct.
I can put a boundary around it.
And in this case, there's nothing to separate, because it's just a one-phase system.
White gold.
White gold is one phase.
Because we have gold.
Obviously we have gold.
If you market something as white gold, it has no gold, you'll go to prison.
And how do we make white gold go to its white color from the normal yellow?
We add silver and we add nickel.
And these are all FCC metals, and they substitute for one another on the lattice, and if you sample anywhere in the white gold, you get the same chemical composition, and so on.
Air is an example of a one-phase system.
It's a gas, and in decreasing order, we have nitrogen, oxygen.
So it's a solution.
We're breathing a solution.
Nitrogen, oxygen, argon are the main constituents.
There's some CO2, some sulfur dioxide if you live near a power plant, NOx if you're near a tailpipe, and so on and so forth.
And then the last one I'll give as an example is calcia-zirconia.
Or cubic zirconia.
Let's just call it cubic zirconia.
The holidays are coming, so we better get cubic zirconia up here.
That's the poor man's diamond.
Cubic zirconia, which consists of a solid solution of calcium oxide in zirconium oxide.
This is a solid solution, so it's continuous and chemical composition identical.
Now, contrast that with two-phase system.
So p equals 2 looks like this.
So let's look at, instead of simple water liquid, if I have ice cubes in water.
So now I have two different phases, and they're separated by state of matter.
State of matter is the issue here.
Because I have a solid-- this is solid-- and I'm going to put the boundary around them.
The composition of the solid is different from the composition of liquid.
They're both the same water, but you can tell it's a different crystal structure and so on.
And I can put boundaries around, and it's mechanically separable.
You can pull the ice cubes out of the mix.
Milk is two-phase.
Milk has a fatty phase and it has an aqueous phase.
The aqueous phase contains the minerals.
That's why you can drink skim milk and still get your calcium, because calcium, even the general public knows calcium is a mineral.
It's not elemental calcium, it's a calcium compound.
But it's ionic, it's soluble in water, and the fat contains the other parts.
You know from last day, this is one of the lipids.
And so you have fat globules that are mechanically separable and distinct from aqueous.
So they actually have a different chemical composition, different type of bonding.
Right?
This is largely nonpolar, and nonpolar doesn't like to dissolve in something that is polar and hydrogen-bonded.
So they phase separate.
I've been saying phase separate all along.
Now you know what we mean by phase separate.
Here's a third one.
This is really cool, because you won't get this in any other chemistry class, because only 3.091 talks about such things.
David, can we cut to the document camera?
I want to show you a piece of snowflake obsidian.
Obsidian is a volcanic glass.
So here's the piece of obsidian.
And what you're looking at is, the black area is the glassy part, and the white is actually some of the obsidian that has decided to devitrify.
And it is turning crystalline.
And so those are islands of obsidian crystal sitting in a matrix of obsidian glass.
If we go back to the-- I picked that up in Yellowstone Park.
It's fantastic.
There's a wall, just a wall of obsidian that comes down.
And I think I've got an image here.
There it is.
This one I pulled off the internet.
So this is crystalline phase, and this is the amorphous phase.
And the overall composition, it's a glass, it's a silicate.
And why isn't it transparent to visible light?
Because it's got iron in it.
The iron gives it the absorption and capabilities.
The band gap is invisible, so it's black.
But then when this crystallizes, it's got a different index, and it's mainly SiO2.
It's really the cristobalite phase of the obsidian.
So that's cool.
So this is actually differentiated on the basis of amorphous and crystalline of something that's virtually the same chemical composition.
So this could give you an idea of what happens in the area of phase separation.
OK. Now the second thing that I have to bring to your attention, is that these phase diagrams treat systems at equilibrium.
They're stability maps, and they treat systems at equilibrium.
So what is equilibrium?
Let's just get one time up here a definition of equilibrium.
Equilibrium is a condition which there is no net reaction taking place.
The system could be reacting.
If you're looking at the ice cubes in water, the ice could be dissolving, and some of the water could be freezing, but overall, with time, there's no net reaction.
It's a dynamic system with no net reaction.
And it's characterized as the lowest energy state.
Systems strive towards equilibrium.
If it's the lowest energy state, no net reaction then.
If a system is truly at equilibrium, the properties should be invariant over time, because there's no net reaction.
You can't prove equilibrium.
You can only demonstrate the absence of equilibrium.
Because at equilibrium, there's nothing, there's no net reaction.
And it could be attainable by multiple paths.
So in other words, if I have ice as the equilibrium phase, I can make it by freezing water, and get to the same ice at the given temperature and pressure.
Or I could sublime.
But if I specify the pressure and the composition, I should always end up with the same system.
So attainable by multiple paths.
Because it's a stable state, so it doesn't matter how you got there.
It's a state function.
And you know from your physics, a state function has the same value, regardless of how you arrived there.
And then the last thing is, I want to differentiate.
See, these are all single phase, but some of them are just one element, some of them are just one compound, and some of them have a whole mix.
So we have a way of quantifying the degree of chemical complexity by using the term called component.
It's a measure of chemical complexity.
It's the number of chemically distinguishable constituents.
And again, these are definitions.
I'll give you some examples so then the definition comes to life.
I like to think of it as the number of bottles you've got to pull off the shelf to make the final product.
The building blocks.
So for example, water is just water.
You could say, but it's hydrogen and oxygen.
But if you take 560 or some other thermal class, you'll learn that they're bound by mole ratio.
You don't have freedom of hydrogen and oxygen.
So it's just one bottle, whereas to make white gold, you need three bottles.
You need a bottle of gold, a bottle of silver, and a bottle of nickel.
So that's a three component system.
And I'm going to designate the components by c with a circle around it.
Because c without a circle is already composition, concentration.
OK.
So now we can put all of this together, and we'll show the difference between multicomponent and multiphase.
So we'll make a little table here, and here I'm going to talk about one-phase systems, and distinguish on the basis of components.
So if it's single phase single component, that's just water.
One phase, one component.
Simple liquid.
If it's two components one phase, that could be calcia-zirconia.
I need a bottle of calcia and a bottle of zirconia in order to make this two component single-phase system.
And then to have three components, that's the white gold.
It's still single phase, but I need the gold, silver, and the nickel, whereas if I come over here, I've got two phases.
That could be slush.
Slush, that's ice water.
It's just one component.
I just start with water, and drop it down to 0 Centigrade, and I'm going to end up with ice crystals in the water.
So that's one component.
I just needed one bottle.
Literally, one bottle.
All right.
Now what about two components, two phases?
Well, we saw earlier when we started thinking about solubility, we did the bilayer with carbon tetrachloride and water.
Remember, we dropped potassium permanganate and iodine, and the potassium permanganate dissolved in the water, and the iodine dissolved in the carbon tetrachloride?
Two different components.
I need a bottle of carbon tet and a bottle of water.
And they'll phase separate.
They don't dissolve in one another, because this is a nonpolar liquid, and this is polar hydrogen bonded.
That's the origin of phase separation.
It's all about bonding, and bonding is all about electronic structure, and that sounds like a good topic for a class in chemistry.
And then the last one here is obsidian.
The snowflake obsidian, in which the components are SiO2-- so I'm going to have three components and end up with two phases.
SiO2, there's some magnesia, and then the black comes from the magnetite.
Fe304.
So I've got three bottles off the shelf, and I end up with something that is crystalline and amorphous.
So that's where I get the two phases.
Three components, two phases.
It's great.
So now I want to look at some stability maps.
And I'm going to start with the simplest of all, which is the one we know best from human experience, and that's water.
So we're going to look at the one-- so it's a one component.
So for water, we start with c equals 1, and the example is going to be water.
One component phase diagram.
So we don't have a composition axis, because the only thing we can vary is pressure and temperature.
If we change composition, we've lost water.
So it has to be axiomatically, no variation composition.
All we have is that the coordinates of temperature and pressure.
So there it is out of the book, and I'm going to draw it a little bit differently, just to emphasize three of my favorite words.
Not to scale.
Otherwise you can't see some of the subtle features.
So overall, it's a y-shaped diagram.
And this is pressure.
Let's make it pressure in atmospheres.
Then our human experiences is at 1 atmosphere.
So at the highest temperatures, we expect to have vapor, and at the lowest temperatures, we expect solid.
And in between, we expect liquid.
And then if we look at the 1 atmosphere isobar, what's happening on this line?
On the one side of line it's liquid, and the other side of line, it's vapor.
At this temperature, liquid and vapor coexist.
I call that the boiling point.
So I'm going to put 100 here.
And we already learned from our little example of Mount Everest that this locust here, this line, is the liquid equals vapor two phase equilibrium.
And you see how it changes?
If I go to the top of Mount Everest, the pressure is less than 1 atmosphere.
which means the boiling point of water is below the denaturing point.
So this actually gives you the map.
Or if I put the pressure cap on, I go up here.
And you can use this in the kitchen again.
If you love wild rice, and you can't wait 45 minutes when you get home, use a pressure cooker.
What the pressure cooker does, is it allows you to have liquid up to 130 degrees Celsius.
See, you can't steam it.
You've got to soak it in water.
And you know the Arrhenius Law.
If you can get the temperature up by 30 degrees, you'll cut the cooking time in half.
So how do you get water, liquid, at 130 degrees?
Then you can cook quicker!
Pressure.
Seal it.
And then you've got liquid water way, way up here.
Now over here, this is solid goes to liquid.
So this is the freezing point line.
At 1 atmosphere, that's 0.
0 Celsius.
And you know, if you go skating-- so let's say the ice is minus 5 Celsius.
And you're here, but you put your weight on the blades, and the blades have small area.
So pressure is force per area.
So that puts you up here, and now you cross, and now you glide on water.
Film of water.
You ever watch a hockey game?
I grew up in Canada.
We used to say, well, the ice is fast.
Why is the ice fast?
Because you get down to temperatures where you just get the right slickness of water.
If the ice is slow, the temperature is so close that, you know, you get these big, bruising hockey players, and they're moving up into here, and it's like they're dragging.
And if you ever come from Minnesota or North Dakota, if you've ever skated when the temperature gets down about minus 20, minus 30?
It's a different experience, because you never get across this line.
Your blades are just, they can't get a good grip.
See?
Everything you need to know is here.
This is four-star chef, this is prize-winning hockey player.
We haven't got off the 1 atmosphere line yet.
OK.
So now I'm going to say, this is single phase, right?
It's just vapor.
This is single phase, and this is single phase.
It's all solid.
And along this line, it's two phase, isn't it?
Solid-liquid.
This is ice cubes, isn't it.
Anywhere along this line is ice cubes in water.
Look at this one.
This is liquid vapor.
This is solid, liquid-- all three of them coexist here.
This is p equals 3, which means, ice cubes floating in boiling water.
And this happens at only one point.
It's called the triple point, and its coordinates are-- this is not to scale.
So this is 0.01 degrees C. It's a 1/100 of a degree higher.
And the pressure here is 4.58 millimeters of mercury, which then you can convert using the 760 as 1 atmosphere.
Oh, and this is solid-vapor.
You can go directly from solid to vapor down here.
If you hang up your clothes on a line and it's minus 20 degrees outside, first thing that happens, the clothes freeze.
You come back five hours later, and the clothes are dry.
Well, what happened?
Did somebody take them in?
Run them around the clothes dryer, then hang them up for you?
No.
You're going directly from solid to vapor, down in here.
So here's the whole story of the phase diagram.
One last thing I want to point out.
Here, you're at a given temperature and it's vapor.
And then you squeeze, squeeze, squeeze, and now things get closer together, and you go from paper to liquid at constant temperature.
That makes sense, right?
I start from vapor, constant temperature.
I squeeze it, I get liquid, I keep squeezing, I should make solid.
Here's vapor.
I squeeze, I get solid, and I squeeze harder, and I get liquid.
That makes no sense at all, but that's what happens.
What does this tell me?
It tells me that the number of nearest neighbors in the solid is greater than the vapor.
Yeah, I get that.
But the number of nearest neighbors in the liquid must be greater than the number of nearest neighbors in the solid.
This is an exception.
When I see this negative slope, it means that ice cubes are going to float on water.
That's what I learn from this negative slope.
So let's put that down.
dp by dt when dp by dt is less than zero.
This is for solid equals liquid.
That means that the density of the solid is less than the density of the liquid.
All from here.
OK. Good.
So let's look at a few other phase diagrams.
I think I've got a few things up here.
Here's silicon.
Pressure versus temperature.
Its normal melting point is about-- see, now I said normal melting point.
After today's lecture, if somebody says, what's the boiling point of water?
You don't go, 100 degrees.
You say, at what pressure?
And they go, oh, I hate you.
I hated you before.
I'm going to use an adverb here.
Now I really hate you.
OK.
So this is the normal melting point of silicon.
It's about 1430 degrees Centigrade, or a 1700 something Kelvin.
And it also has this negative slope.
And what does that tell you?
That liquid silicon is denser than solid silicon, and it's a good thing it is, because we wouldn't have the microelectronics age if it weren't for that.
We couldn't have Teukolsky crystal growth and everything else that makes silicon dirt cheap.
You know why it's dirt cheap?
Because it's made from dirt.
That's why it's dirt cheap.
It's true.
Here's aluminum.
This is normal.
This is the normal solid-liquid equilibrium.
This is FCC metal, which is the closest packed.
Axiomatically, the liquid must be less well-packed.
All right?
So this is dog bites man, this is man bites dog.
This is the unusual one.
OK. What else do we have?
Nitrogen!
Here's nitrogen.
I drew this.
So there's the 1 atmosphere line.
Nitrogen boils at 77 Kelvin, and it freezes at about 63 Kelvin.
And it's got the positive slope that you'd expect, and so solid nitrogen is denser then liquid nitrogen.
And you can remember 77 Kelvin.
You want to remember a few of these things.
I wanted to point out to you that at atmospheric temperature, ambient temperature, you can remember 20-20.
20-20 vision.
So at 20 degrees C, it's roughly 20 millimeters of mercury, is the vapor pressure of water.
It's not exactly true, but it's close enough.
I think it's really 24.
But 20-20 is nice to know.
And what's the street address for MIT?
77 Mass Ave. That's the boiling point of liquid nitrogen, 77 Kelvin.
So I thought we'd do a few things here.
And I'm going to show you a little bit of fun and games with liquid nitrogen.
But first, I've been instructed I have to practice safe laboratory-- So I'm going to put on my lab coat.
Yes.
It's a nice lab coat.
Mm-hm!
Yeah, look.
You want to know-- let's go back to this.
They have to know where the-- this is not just any old lab coat.
This is a nice lab coat.
This is from France.
It's from MAISON DUTECH.
See?
Notice the collar.
It's not that typical, you know.
MAISON DUTECH.
OK. Let's go back to the phase diagram.
So Dave Broderick is going to help us here.
So we're going to have some fun.
I'm going to put on a face shield so I don't blind myself.
All right.
Here we go.
All right.
So let's get some liquid nitrogen.
Sounds different, huh?
Sounds terrible for me.
All right.
So we've got some liquid nitrogen here, and I'll pour it into a Dewar.
So what happens when you wet a sheet of paper with ink with liquid nitrogen?
What kind of bonding is there in liquid nitrogen?
Look.
No running, no running.
If it's liquid nitrogen.
I tell you, everything you learn here is so valuable compared all the other stuff you learn here-- All right.
So there's a couple of things.
So let's look at glass transition temperature.
So this is a latex glove.
You can see it's above its glass transition temperature, and the cross-links are snapping it back.
What I'm going to do, is take it down below its glass transition temperature, and turn it glassy and brittle.
So that it's below the TG.
Here's some paraffin.
Paraffin, which is also a polymer here.
Take the paper off.
OK.
So here's paraffin.
Look.
Look at the van der Waals bond.
All right?
This is van der Waals bond.
Now we're going to go below the TG.
It's liquid nitrogen!
It doesn't matter.
OK, you can hear it already.
So it's now below the glass transition temperature.
So what else have we got?
Oh, you've heard of the glass flowers at Harvard?
Well, we've got glass flowers here, so.
Here's a rose.
See, in the proteins, you know, it's above the-- you know, it's soft.
But now, we're going to put the-- mm-hm.
Here we go.
You ready?
Oh, come on.
It's not a kitten.
What's the matter?
It's still burbling here.
We have to wait until it's dead.
This is for science.
OK. Now it's below its glass transition temperature.
Oh, I'm sorry, David.
I should put this over on the other side.
But maybe it's the red, or maybe it's because it's a rose.
So here's the chrysanthemum.
Today is your day.
Let's go.
Get this thing moving!
Yeah.
So there you have it.
There you have it.
I think that's-- oh, there's one, may we go back to the slide?
Better put the gloves on for this.
I'll have to write apology letters.
OK.
So we're going to do next, is we're going to look at phase equilibriums.
There's the melting points and boiling point of oxygen, argon, and nitrogen.
And what you notice is that nitrogen boils at a lower temperature then oxygen and argon.
So what I'm going to do, is I'm going to pour some liquid nitrogen into this beaker.
And then inside the graduated cylinder, I'm going to let it sit here, and we're going to condense air.
And the result will be that we're going to start condensing liquid oxygen, and they'll be little bits of argon ice floating in liquid oxygen.
And liquid oxygen is faintly blue, and it's paramagnetic, as you know, because it's got the unpaired electrons, which means that it will levitate in a magnetic field.
I couldn't bring a big magnet in here, but we'll see if we can make some blue stuff.
And it's foaming, because this would be like pouring water into something that's-- I don't know.
300 degrees Centigrade.
But eventually, the heat transfer gets low enough that you have stability here, and-- yeah.
OK. Oh, that's terrific.
OK. What else have we got?
Let's go back to the slide.
God, I hate this thing.
OK.
So you've been looking at that nitrogen.
Let's look at what's next.
Ah.
Now if you solidify nitrogen here, you get a whole bunch of different polymorphs.
Different crystal structures.
So there are about five or six different crystallographic forms of solid nitrogen.
For that, we'd have to bring liquid helium.
4.2 Kelvin.
Very cool stuff.
Here's carbon dioxide.
Now, it also has the positive slope, so solid will sink in liquid.
But look, the interesting point here.
The triple point is 5 atmospheres.
At atmospheric pressure, you go directly from solid to gas.
It sublimes.
And that's advantageous.
When we use carbon dioxide as a coolant, when it gives up its enthalpy to cool, it doesn't then turn into liquid.
If you chill something with ice, with water ice, when it gives up its enthalpy, the object is now swimming in water.
And that's no good.
So hence we have the term dry ice, because we go directly from solid to vapor.
And we want to prove this.
So what we've got here is a big block of dry ice.
So this is at minus 78 degrees Celsius, and it's still stable here.
Well, it's subliming before your eyes.
It's smaller than it was when we started the lecture.
So what I'm going to do now-- oh, I'm not wearing that stuff.
I guess I'd better.
They're going to write me up on some log, teaching you bad practice.
But anyways.
All right.
So what we're going to do, is we're going to test the proposal that this actually goes-- so I've got some water.
Here, David, let's go out to the video.
So let's take some water.
I think we got water from France.
Goes with the lab coat.
This is Evian, OK?
So I pour the Evian into the Florence flask.
This is Florence.
Florence is round, and the Erlenmeyer is the flat one.
So let's move this guy.
You're going in the backstage for a while.
You just keep condensing All right.
So now, this is here.
Now what we're going to do, is we're going to put some dry ice in there, and see if we go directly from solid to a vapor.
Take a chip off the old block, here.
OK.
So now you can see that it's going directly from solid to vapor.
So that's carbon dioxide bubbles.
And we assume that the phase diagrams is correct.
I mean, another thing we could do, is-- I mean, I know what club soda tastes like.
I could just test it, right?
Yeah.
It's carbon dioxide.
Now!
Here's the other thing.
You know, Schweppes likes to pride itself on tiny bubbles.
They have a tagline, Tiny Bubbles.
Right?
So here's Schweppes.
And you can see Schweppes, with their wimpy little tiny bubbles.
Can you get that, David?
But we have, we have big bubbles.
I mean, this is pure.
You can't get any better than that.
I got Evian water, and I got a block of carbon dioxide.
I like making my own carbonated water.
I've taken it to a new level.
OK. What else do we have?
Well, let's see if we made any liquid oxygen here.
Try this.
I don't know if we got anything here.
Oh yeah!
Oh, wow.
Look.
OK.
So that liquid in there-- oh, I see.
You've got a separate thing.
I don't know if you can see.
It's condensing.
That liquid in there, that's liquid oxygen.
I won't drink this.
But I can inhale it.
Pure oxygen.
I could have a cigarette, and just put the cigarette in there.
What happens if I put the lit cigarette into the oxygen?
All that happens is that it burns quickly, that's all.
There's no explosion.
It burns quickly.
If you walk into a room full of oxygen, all that happens is you'll singe your nose, that's all.
There's no explosion.
It's just oxygen.
There's no fuel.
The fuel is just in tobacco, and there's not much.
It's not a problem.
A room full of oxygen?
Pfft!
OK. Let's go back to the slides.
What have we got here?
We've got some examples of other phase diagrams.
All right.
Yeah.
[
FROM 'PHANTOM OF THE OPERA']
Actually, it's quite tasty.
All right.
So.
Now, here's zirconia.
Now, here's the thing to know about zirconia.
Look at all the different phases.
And cubic zirconia is the one that has a high index of refraction, but it's stable only at temperatures exceeding 2000 degrees C. So that's no good.
So what I'm going to show you next day is that by changing the composition, and adding calcium oxide, we open up this zone here that's labeled cubic, and we get it stable all the way down to room temperature.
It's an example of changing composition to get the stability of a phase that we want.
This is carbon.
So down here, the normal form, as we've learned earlier, the normal form is graphite.
Diamond is formed at elevated temperatures and pressures.
And this is liquid carbon.
All right?
Now, to make artificial diamond, you can try putting graphite in an anvil, and squeezing, and squeezing, and squeezing, and coming up into here.
And then it becomes metastable, and then maybe, because the activation energy to jump from that sp3 hybridized state to sp2 is so high, the diamond is stable.
So you don't have to go running home to see if your diamond studs are now turning into graphite.
They won't.
Once you get into this zone, they'll stay.
The activation energy is too high.
You need more than 1/40 eV to do it, is necessary.
But in the 1950s, General Electric, its Schenectady research labs, reasoned that because carbon is so soluble in iron, they could dissolve carbon in liquid iron, and then raise the pressure and temperature, and then change to exsolve and cause carbon to precipitate out of molten iron.
That's the birth of artificial diamond.
That's how they did it.
Using this concept, but then dissolving it in molten iron.
[
'DIAMONDS ARE A GIRL'S BEST FRIEND']
I don't know how that got in there.
All right.
Here's one for mercury.
I looked it up, using the tools that you've been taught here in 3.091.
So this is a paper phase diagram for mercury.
And physicists drive me nuts, because they plot temperature versus pressure.
I want pressure versus temperature.
So I just turned it around.
And so here's the liquid line.
So you know that liquid mercury is going to float on solid mercury, and there's a plurality of solid phases.
And these are all different crystal structures.
Now I've got a treat for you.
I was in Barcelona several years ago, and I saw the mercury fountain that was made by Joan Miro in collaboration with Calder.
Calder has the big sail outside of East Campus.
So Calder collaborated with Miro, back in the '30s and '40s.
And this is in honor of the miners of Almaden, where there are cinnabar mines.
And the miners fought bravely against the Fascist forces of Franco, and Miro wanted to honor them.
So here's their museum.
It's a beautiful place up on top of an escarpment over Barcelona.
And what I'm going to show you is a mercury fountain.
[MUSIC PLAYBACK] This is liquid mercury.
The speed has not been altered.
This is a liquid at room temperature, density 13.5, metallic bonds.
Look at it.
Very high surface tension.
Look at the way it shimmers.
Look at this shot.
This has not been altered.
That's the way it pours into gravity field.
It's really fantastic.
And of course, mercury vapor is toxic, so the whole thing is in a-- [END MUSIC PLAYBACK]
Oh, that's the-- I cut that snippet out of the documentary.
OK, we don't want to see that.
So they have the whole thing inside of a polymethylmethacrylate cage, to keep the mercury vapor from affecting the people that work in the museum.
Here are some phase diagrams from hell.
This is bismuth.
And look at bismuth.
Bismuth is like water, and like silicon.
So the solid is less dense than the liquid.
But look at all of these different phases.
Here's sulfur.
Sulfur is crazy.
Look, phases in the solid phase, there's even phases in the liquid!
Different phases.
They're allotropes.
Here's sulfur.
Elemental sulfur, the disulfide, hexasulfide.
It makes rings, it makes long chains.
And these are called allotropes.
They're different forms of the same element.
So oxygen is O.
There's the diatomic molecule.
And the triatomic molecule, which is ozone, these are all allotropes of oxygen.
What else do we have?
OK.
This is polymorphs.
So this is alpha goes to beta.
See, different crystal structures.
You could see with your naked eye what's going on here.
Different crystal structures.
And then this goes to lambda, and then finally, they're pouring here.
And if you get a very high cooling rate, and you've got those long chains, what happens?
You don't form the crystal.
You form the amorphous form.
So they'll call this forming plastic sulfur.
We know better.
It's amorphous sulfur.
It's not plastic sulfur.
Here's water, a complete diagram.
Now, this is in kilobars.
So there's all the different phases of water.
There's the negative line.
That's the thing here.
But at very, very high.
Look at all the different phases of water.
And they have numbers on them.
If you read Kurt Vonnegut's Cat's Cradle, there's ice nine.
You get that going, it freezes the whole world, right?
But here's, I draw your attention on this one.
You see this point here?
This is ice seven.
At 100 degrees C, it's solid.
You need 25 kilobars.
So we could go over to the lab, take some water, put 25 kilobars on it, make solid ice at 100 degrees C. And it'll be stable.
Look!
This is minus 78.
It's been here for a long time.
So now I come into a cocktail party with this, I drop this ice cube into a glass of aqueous beverage, it sinks to the bottom, and the beverage heats up to the boiling point.
That's what this is telling you!
That'll get you popularity instantly.
How did you do that?
How did you do that?
And up here is the supercritical fluid, the last thing we haven't said anything about.
Look.
If you take a gas, and you keep squeezing it and squeezing it, eventually the gas molecules are close enough together that they're indistinguishable from the liquid phase.
Or if you start here, with the liquid, and you keep raising the temperature, the density goes lower and lower until the molecules are far enough apart that you can't tell this zone, whether it's a rarefied liquid or a highly compressed gas.
The properties criss-cross.
This is supercritical.
It has liquid-like transport properties, and vapor-like chemical properties.
So you can end up doing things that are very, very different.
Now, I'm going to give you an example of decaffeinating coffee by taking the coffee, going up into the supercritical regime, which allows you to extract the caffeine, and not brew the coffee.
So you want to just get the-- well, if you just heat it up, you'll make the coffee.
So here's an example.
It's called solvent extraction.
Historically, this solvent was methylene chloride, which is also used as paint stripper.
We don't do that anymore to coffee.
They did in the '60s!
They did.
They also used trichloroethylene.
It works really well to leech out the caffeine.
How much is left in there for you to taste in your coffee?
I don't know.
Maybe that's why people were so excited in the '60s to drink their decaffeinated Sanka.
I don't know.
But anyway, so you take the green beans in carbon dioxide as supercritical, and drop the caffeine level, and then you play with temperature and pressure to go over the KSP.
The caffeine salts out, and then you recycle the CO2.
OK.
I hope I've given you some introduction to phase diagrams.
And we'll see you on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So.
Last day we started looking at phase diagrams, and we looked at unary phase diagrams, one component systems, and one component system P versus T, pressure versus temperature.
And this is the situation for water.
Water is an exception, because it's got this negative solid equals liquid coexistence curve, but otherwise, you have large, single-phase regions here, which I've designated circle p equals 1, and then along these lines, we have equilibria.
So this is liquid goes to vapor, this is solid goes to liquid, this is solid goes to vapor.
And we know that for water, if we take the 1 atmosphere, if this is pressure in atmospheres, the 1 atmosphere isobar, then that would put this point at 100 degrees Celsius, and this point at 0 Celsius.
And then we have the triple point, which is solid equals liquid equals vapor at 0.01 degrees C, and p equals, I think it was 4.58 millimeters of mercury.
So that's the triple point.
And we looked at a variety of other one-component phase diagrams, and I think we came to a pretty good understanding.
And then up here, we have the supercritical fluids.
So I've got a break in the line here.
And for water, you reach supercriticality at 374 degrees Celsius, and up here at 218 atmospheres.
So these are not phenomenal, these are not geological pressures at all.
And so that's if we want to the decaffeinating of coffee and so on.
And so what happens here, is you have a highly compressed vapor or a highly rarefied liquid, and you end up with solubilizing power of a liquid, but transport properties of a gas.
So this stuff has really good diffusivity.
So it can penetrate into interstices and do its work very quickly, to say nothing of the fact that you're up at 374 degrees Celsius or higher.
So today what I want to do, is I want to look at two-component systems.
So component number is circle c, c equals 2.
So that means now I have to deal with pressure, temperature, and lowercase c is composition.
I want to know how things vary with composition.
So let me give you an example.
Suppose I have two substances, A and B.
So here's substance A, and I've got its one-component phase diagram.
And in this case, I'm going to put solid-liquid vapor in the conventional setting, where the solid-liquid line is-- the coexistence curve has a positive slope.
So this is A, and let's say I've got B over here, and it's got its solid-liquid coexistence curve, and so we've got its melting point, and whatever.
So now the question I want to ask is, what happens if I mix A and B?
And let's just say, here's the melting point of A, so this is at p equals 1 atmosphere.
And here's the melting point of B.
And I want to ask the question, how does the melting point of the mixture vary?
So let's say I'm going to connect these, and I want to ask, what happens if I've got an AB mixture?
So I'm going to put lowercase c here, which is concentration.
So at the one end, I've got 100% A, and over here, I've got 100% B, and I know those melting points.
Question is, what happens when I mix them?
Does the melting point-- is it just a straight line?
Is it a linear variation?
Do you go through a local maximum?
Do you go through a local minimum?
Or do you go wild?
I mean, what happens?
So a question you might ask is, suppose I made a 50-50 alloy of 50% A and 50% B.
If I knew the end-member melting points, could I predict this?
Or if not, do I have an archive?
Remember the phase diagram compendium is an archive.
What is the value of the melting point at 50-50 or 75-25?
So that's the question I want to ask.
Now this is getting really messy, because now I have to plot pressure, and I have to plot temperature, and I have to plot composition.
So this is really crazy, right?
I've got pressure, and I'm going to have composition here, and I'm going to have to have now a third axis, aren't I?
I'm going to have to have a third axis.
I'm going to need a temperature axis.
And I've got to make a three-dimensional drawing.
And this is messy.
But we're in luck, because 3091 is solid state chemistry.
So as solid state chemists, we're more interested in what happens in the solid here.
Now, I've drawn these lines with a slope, but they're really not to scale.
It turns out that, you know, you can move 10,000 feet up or down in the atmosphere and have a huge variation in the boiling point because this line is shallow.
But this line is virtually straight up and down.
You have to go to geological pressures to change the melting point very much.
And so, as a result, this is almost insensitive to pressure, to first order, right?
So we can say solid equals liquid is almost insensitive to pressure, whereas liquid equals vapor is very sensitive very sensitive to pressure.
But we don't care about this so much.
So what I'm going to do is throw it away.
I'm going to throw it away, and just say, let's look at just T versus C. Temperature versus composition on the strength of the fact that the melting point and all of these things are mildly dependent on pressure.
All right.
So now I'm going to give you three different types.
There's many, many different types of binary phase diagram.
So now we're going to be looking at C equals 2.
So instead of a unary, this is going to be called binary.
Binary phase diagrams.
And they come in all shapes and sizes, but they can pretty much be classified in several bins.
And I've made the classification.
So classify binary phase diagrams according to their bonding.
It all comes back electronic structure and bonding.
Vary by bonding.
Because bonding, then, ultimately indicates solubility, and that's what we're looking at here.
We're going to look at how well A dissolves into B, and how well B dissolves into A.
That dictates solubility.
So it all comes full circle.
So I've made this up.
You're now going to find this in the book.
So I just decided, for ease of teaching 3091, is I'm going to just give the different types.
And so I call them type one, type two, and type three.
You're not going to find that in the books.
I made that up, because it's simple.
So type one.
What's type one binary phase diagram, according to me?
What is the type one?
It means complete solubility.
So A and B are completely soluble in one another as solid and liquids.
That's the first thing that you'll see on a type one phase diagram.
And the second one is, a change of state is present.
And you know the only change of state we care about here is solid goes to liquid.
So we're going to show solid goes to liquid, and the thing is totally, as they say in California, totally soluble as liquids and solids.
Now, here's some bonding rules that you can think about.
So what would be the characteristics of two substances A and B that would give complete solid solubility?
Well, one thing is they'd have to have identical crystal structures.
That would heighten the chances of this.
Identical crystal structures.
So if they're both FCC metals, you've got a better chance of making an infinite variation in solution composition than if one is an FCC metal and the other is a BCC metal because FCC will just sit on the lattice site.
But there's more fine structures.
Second one is similar atomic volumes.
So if we're going to have a substitutional solid solution, let's make sure we're replacing oranges with oranges, and not trying to put a grapefruit on a site that normally is occupied by a lemon, because there's going to be a size restriction there.
And the third thing is, even if they have identical crystal structures and similar atomic volumes, you can still run into trouble with respect to complete solid solubility if you have a large variation in electronegativity.
And I'll show you an example of that.
So if you have a small difference in electronegativity, it means there's a low propensity for polarity, and ultimately no chance of electron transfer.
So I think it's pretty cool.
So I wrote those down, but was disappointed to learn that for metals, they were enunciated about 75 years ago by the British metallurgist Hume-Rothery.
So I can't take credit for this.
This is one name.
It's one of these hyphenated British names.
He's Sir da-da-da-da Hume-Rothery.
So he has the Hume-Rothery Rule.
And he called such systems that mix as binaries forming isomorphous, meaning they have the same structure.
So let's take a look at the prototypical isomorphous phase diagram, and it looks like this.
We're plotting temperature versus composition.
We threw away the pressure coordinate.
So I got pure A on the left, pure B on the right.
Little c is concentration, so it varies from 100% A to 100% B.
And the vertical axis is temperature.
So one extreme, I've got the melting point-- this is a melting point of pure A.
And at the other extreme, I've got the melting point of pure B.
So that, we know.
And now the question is, how does melting point vary as a function of composition?
And for an isomorphous phase diagram, it looks like this: lens-shaped.
Up here is all liquid.
You see A and B mix in all proportions.
And down here is all solid.
And furthermore, it's an all-liquid solution.
It's a mixture.
And this is an all-solid solution.
And then comes the piece that if you take enough thermodynamics, you'll be able to rationalize.
I'm simply going to tell you without proof that when you have a multicomponent system, when c is greater than 1-- so we're in that situation.
Now, c is 2.
When c is greater than 1, it is impossible to move from one field, up here, this is-- I'd better put the label on it-- this is a single-phase field.
So up here, p equals 1, because it's homogeneous liquid solution, p equals 1.
When you're in pure materials, you can go from solid to liquid.
But when you're in a two-component system, you cannot move from one single-phase field to a second single-phase field without moving through a two-phase field.
And we'll get to that, what it means to move from one field of p equals 1 to another field of p equals 1 requires traverse or transit across a field of p equals 2.
And so what's in here has to be the end member.
So this must be liquid plus solid.
So I'm going to call it slush.
They have a slush in here.
The other terminology that people give this is, it's lens-shaped, right?
Looks like a lens.
So let's call this lens shape.
So we're going to use the Latin word for lens, which is lens.
But we want to make it adjectival.
So we're going to use the adjective-- what's the genetive form of lens?
Lentis?
So we will call this lenticular.
This is lenticular.
The phase diagram has a lenticular shape.
Looks like a lens.
And you know, just as over here, I showed you, every line represents a coexistence curve, right?
The lines are all coexistence curves.
These lines are coexistence curves.
The line up here is liquid equals vapor, so this one must be the coexistence of the two things on either side.
So what do I have here?
I have liquid.
Here I have slush.
So I'm going to write that.
So I'm going to write the equilibrium, liquid goes to liquid plus solid.
And this line, this coexistence curve, it's called the liquidus line.
This is the liquidus equilibrium, the liquidus coexistence curve, and so on.
And so what's the liquidus?
The liquidus is the lowest temperature.
So you pick a composition.
The liquidus is the lowest temperature at which you can have a single-phase liquid solution, OK?
So liquidus equilibrium.
And the liquidus is the lowest temperature at which all liquid is stable.
You go below that temperature at that composition, you start making solid, because slush requires that there be some solid present.
And then the lower line, it also is a coexistence curve.
So on the one side, I've got solid.
On the other side, I've got slush.
So I'm going to write that one down.
That solid goes to liquid plus solid.
And that's called the solidus equilibrium, or that's the solidus line, the solidus coexistence curve, and the solidus is the complement.
The solidus is the highest temperature at which you can have all solid present.
So you pick a composition, and I'll tell you what the highest temperature is at which you'll have a single-phase solid solution.
So solidus is the highest temperature at which all solid is stable.
All right.
Good.
And I'm going to look at a few of these things.
It's always fun to see if I'm on track with the Hume-Rothery rule.
So here's copper nickel.
They're both FCC metals.
So we've got copper melting at about 1085, nickel melting at about 1455.
And there's the phase diagram, lenticular phase diagram.
So up here is all liquid, and then-- this is all metallurgy terminology.
A solid solution of A and B, they call alpha.
Metallurgist looks at that, goes oh, it must be a solid solution, OK?
So there's p equals 2 in between, p equals 1, p equals-- Here's a ceramic system.
This is nickel oxide, magnesium oxide.
It's not metals, but they have identical crystal structures, very similar atomic volumes.
They're ions, so we have nickel and magnesium substituting for one another on the cationic sublattice.
So they must have very nearly equal sizes.
If I look at this, and I see a lenticular diagram, it means they must have similar atomic volumes, which means the dominant defect in here must be Schottky, not Frankel because Frankel needs a big difference in atomic volume.
If you have a big difference in atomic volume, we wouldn't have a lenticular phase diagram.
So here's-- mag oxide melts at 2800 degrees centigrade.
It's a great refractory.
You can hold molten iron in it.
And here's nickel oxide down here.
All right.
Here's an interesting one.
This is gold nickel.
Both FCC metals.
Shade your eyes from the lower part.
Just look at the upper part.
It looks lenticular.
It's what you'd expect, gold and nickel.
They're both, you know, card-bearing metals.
But you've seen already, with the cesium-gold, gold has a fairly high electronegativity.
And what happens is that you get over here, at about 33 atomic percent of nickel in gold, you have an atomic ratio that allows to have something that starting to approximate electron transfer.
So it's almost as though you have a lenticular phase diagram between pure nickel and this nickel-gold compound.
But up in here, it's nice lenticular stuff.
All right.
What's the next one?
Oh, yeah.
So now I want to go in and I want to start talking about what goes on inside here.
So what I'm going to do, is I'm going to blow this up.
And I'm going to start at 40%, 40 weight percent nickel, and I'm going to say, what happens if we take a crucible that's 40 weight percent nickel and copper, and we cool it from all liquid at 1300, down to all solid at 1200, pausing in that slush zone at 1250?
So we're going to take snapshots and say, what's the contents of the crucible look like at 1300, at 1250, and at 1200, and come out of it, have an appreciation for what all of this stuff means.
So let's start our experiment.
So we're going to have three crucibles here.
And we're going to start with 40% nickel in copper, and that's going to be the experiment we'll perform.
So I've got three crucibles.
And so if I look at the phase diagram, here I am.
This was 1300 degrees C, this was at 1250 degrees C, and then this was at 1200 degrees C. So at 1300, that's trivial.
1300 for this thing, it should be just all liquid.
And maybe it's copper.
I know at this temperature, it's going to be blinding white heat.
So it doesn't matter.
I could use this.
But I felt compelled, it's one of those rare opportunities to use colored chalk.
So I want this to sort of be copper-colored.
The meniscus is going to look like this, because the liquid metals have a very high surface tension.
They want to ball up.
So you're going to have a meniscus looking like this, and this is all single phase, all liquid.
And over here, I'm going to do the easy one.
If we get down to 1200, it's all solid state, right?
At 1200, I'm going to end up with something that's solid, and it's going to be polycrystalline, and these are all going to be grains of alpha.
All solid.
And I'm going to label all of these as alpha solid solution.
They all have the same composition, which is 40% nickel in copper.
And these are all 40% nickel.
Now, according to this phase diagram, when we get down into the center there, something else happens.
We end up in a two-phase regime.
And that two-phase regime is a regime in which certain compositions are forbidden because that's an equilibrium.
It doesn't allow us to have what's in between.
I think the next slide actually shows this.
So what I've designated as c2 is this 40%.
So c2 is now going to park at that point in the middle.
Whoops!
Want to get into that.
So here's where we are.
We're in the center of that two-phase regime.
So this is the solidus.
This is the liquidus.
And we're at 1250. t equals-- And we're at this value.
We started at c2, which is 40%.
But in this regime, 40% is forbidden.
If you stop at 40% at 1250, this says that the stable phases are a liquid and a solid.
But the liquid has less nickel in it, and the solid has more nickel in it.
It's like if this is a solubility limit.
In other words, if you started at pure copper, and you started adding nickel, you can keep adding nickel until you get to this concentration.
You try to add any more nickel, it's like adding too much sugar to water.
What do you have?
The stuff just falls to the bottom of the cup, doesn't dissolve.
That's this.
It's the solid.
Only instead of being pure nickel, it's still a copper nickel solution, but kind of rich.
So I'm going to call this c star.
It's the solubility limit on the liquid side.
And this one here I'm going to call c star on the solid side.
In other words, I could start from this side, and keep adding copper to nickel, and I make a homogeneous alloy until I get to this composition.
If I try to add any more copper, I jump across.
So we're here in the middle.
But now, isn't there something strange here?
It's saying I'm going to have a liquid that's nickel-poor and a solid that's nickel-rich.
And I'm just putting up here what the diagram says.
And we know that FCC metals, which is denser, the liquid or the solid?
The solid.
So down here I'm going to have alpha, OK, a bunch of grains of alpha solid solution, and up here, I'm going to have a liquid solution.
And the composition here is c star liquid, and the composition here is c star solid.
It's different composition from this.
Here the composition is equal to c2, or the 40% nickel.
So I've got disproportionation.
But the total mass, I can't have sources or sinks.
So I have to conserve mass.
So I'm going to ask you to just do the algebra.
You've got two equations and two unknowns.
I know what the n-member concentrations are.
They're given by the n's of this thing called the tie line.
The tie line ties the two ends of the two-phase region together, and then we just do a mass balance.
Well, fortunately, the metallurgists have thought about this for a while, so we don't have to go through and figure out, well, if two apples cost 15 cents-- we don't have to do that kind of thing.
We can just invoke the lever rule, and it will tell us how much of the liquid that's nickel-depleted and solid that's nickel-rich add up to this.
And so let's look at the lever rule.
Basically what's going to happen here is that the stuff that started off as 40% nickel, 60% copper, that's my initial mix, it's going to break into two layers.
It's going to break into a liquid layer, which, if you go to the phase diagram, looks like it's about 38% nickel and 68% copper, and that's what we're going to call c star liquid.
And then over here it's 45% percent nickel and 55% copper.
And that's what we call c star of the solid.
And so it's just a lever.
So there's c star.
This is the c2 that we want.
And we want to ask, what's the relative amount of the liquid phase and the solid phase?
And why they call it a lever rule is, look, if you cooled something that was at this composition, it would be 100% liquid.
And if you cooled something at this composition, it will be 100% solid.
And so it's going to be a constant variation across here.
So you use, you take this, and you know, and I want this amount, I'm taking this versus this, kind of thing.
It's a lever construction.
So we'll just put it down.
So the percent of the liquid in the crucible at 1250-- and then this is a general thing.
Whenever p equals 2, you'll use a lever rule.
Percent of the liquid is given by this one.
The composition of the solid at the end of the tie line minus the composition that you started at.
This is your bulk initial concentration, divided by the length of that tie line.
c star solid minus c star liquid.
And then multiplied by 100%.
So you get a percent, and the number here is going to be 45 minus 40 over 45 minus 32, and that works out-- going to multiply by 100, or else there will be complaints.
And then this turns out to be 38%.
So what that means is that if you take this amount of liquid here-- pardon me-- based on conservation of mass, if you drop and hold at 1250, 38% of that volume is now sitting here in the liquid, which means 62% of it by volume has turned into solid.
And the composition here is different from the composition here, but if you add up all the nickel in the crucible and sum it versus all the copper in the crucible, you still end up with 40% net nickel.
So this very powerful.
Because what you can do here is you can separate metal.
Because I started here with 40% nickel, 60% copper, and now I've got something-- the liquid face, I wrote here, the liquid phase is now 68% copper.
So can you see that if I'm clever about this, I could use this as a technique for enriching.
And if I'm thinking about recycling metals, maybe I need to know something about phase diagrams because it's a very simple way of concentrating impurities or concentrating the desirable stuff.
Let's look at one other thing here that's really very interesting.
Suppose I had something, instead of c2 at 40%, suppose I choose a c1.
So this is 40% nickel.
Now what if I take something that's 35% nickel, and I cool it down to 1250?
What happens?
If I park here at 1250 degrees, according to the phase diagram, that substance has to phase separate, and I'm going to end up with the same thing.
I'm going to end up with liquid and solid.
Now, isn't there a contradiction there?
How is it that whether I started with 40% or 35%, I end up with the same end members?
What's the missing piece?
The relative amounts will be different!
The relative amounts.
Here, look.
As I'm getting closer to the liquid, can you see that axiomatically, I'm going to make more liquid and less solid if I ended up with something over here?
But at 1250, those are the end members.
That's the only stuff that's going to be present.
Ah!
That's so good.
I think I've got some other stuff here.
Oh yeah.
Oh, this was-- yeah.
So p equals 2.
Lever rule.
Whenever you see p equals 2, you have two things you think of.
p equals 2 means phase separation.
And that phase separation means you're going to get different amounts, and the lever rule.
And why I have this Manchurian Candidate-- it's this movie.
I know there's a remake.
The remake is horrible.
The original, if you see the original, is about some fellows that were brainwashed.
They were American prisoners of war brainwashed in North Korea and they're brainwashed to become assassins on cue.
And the cue is the Queen of Diamonds.
When someone shows the Queen of Diamonds, they just go into automatic pilot, and they're supposed to assassinate the political figure.
So for you, your Queen of Diamonds is p equals 2.
When you see p equals 2, you go, phase separation.
Lever rule.
All right?
No guns, just this.
I just want the formula.
You go, phase separation, lever rule.
Whenever p equals two.
So what happens in here?
Phase separation.
Boom, boom.
Right there.
Oh, I'm not supposed to say boom, boom.
Phase separation.
Ta-da!
You know, something.
PC, whatever.
OK.
So let's keep going.
Now I want to look at a second type of phase diagram.
So the first type of phase diagram was complete solubility.
Now the second type of phase diagram has partial solubility.
So let's look at that one.
So i call this type two.
And you don't predict this stuff.
We would give you the phase diagram and simply ask you, you know, you're the specialist.
Tell me happens if I cool this to such and such a temperature?
You can tell me you get phase separation, and the composition of the two end members, and so on.
So the characteristics, the bonding characteristics here, are partial or limited solubility of A and B.
We're doing all of this for a binary system A and B, and no change of state.
So that means either always solid, or always liquid.
That's not to say that the A and B never become solid.
I'm just saying that a type two diagram is limited to a single state.
So let's take a look at the diagram.
The diagram looks like this.
We're going to plot temperature versus composition.
So pure B on the right, pure A on the left.
Concentration is the abscissa, and then the ordinate, of course, is temperature.
And so the shape of the diagram is this.
There's the coexistence curve.
It's called a synclinal coexistence curve.
So above, we have all solid.
And in here, we have two solids.
So using the metallurgical terms, this is alpha plus beta.
Two phase regime, OK?
So let's get those labels up right off the bat, so we know who's where.
So outside the coexistence curve, p equals 1.
Inside the coexistence curve, p equals 2.
And wherever I'm saying all solid, alpha plus beta, I could write all liquid, and then it goes to l1 plus l2.
So maybe we should do that, to show that we're multilingual.
So it's either all solid, or it could be all liquid.
And then this would be liquid 1 plus liquid 2.
And so why do we call it synclinal?
What's this?
This is an incline.
If I put two inclines and I synchronize them, I get a syncline.
And if I don't synchronize them-- these are two ladders.
I could prop them up.
That's called a syncline.
And if I put two ladders like this, if I'm really stupid and I fail physics, if I do this, they fall down.
So this is called an anticline, OK?
That's an anticline.
This is a syncline.
It's not a U-shape, or a hump, or something like that.
This is 3091.
This is a syncline.
Synclinal coexistence curve.
All right.
So what's on here?
What's this equilibrium, then?
Well, the equilibrium must be this equals this.
So that would be, in the case of the solid, it's the solid solution goes to alpha plus beta, or it could be the liquid goes to liquid 1 plus liquid 2.
That's the equilibrium.
You can think about it almost as a solubility.
So let's start over here on the left.
Let's pick a temperature.
Let's call this T1.
So I can put B into A, and I continue to get an all-solid solution, up to this concentration here.
What happens at this concentration?
I've hit a solubility limit.
And if I try to put any more B into A, I get a tie line-- lever rule.
In here is lever rule time, isn't it?
Lever rule.
p equals 2, OK?
So it's solubility limits.
Solubility limits.
All right.
So let's look at some examples.
Oh, here's this one, actually.
That's why I had a-- you know, if you go to lower temperature, gold, nickel-- look!
They actually phase separate.
That's shocking.
I always find this one shocking, because you think gold, nickel, nice FCC metals, they should substitute for one another.
Look what happens.
If you start putting nickel into pure gold at 700 degrees, you get the 10 weight percent, you put any more in, boom.
Right across here.
So this is going to be two phase.
So if you look at the solid, what's that going to look like?
If I'm at, say, 30%, now I look underneath and I've got a polygrain system, and I'm going to have an alpha and a beta, and an alpha and a beta.
I'm going to have two different grains.
And now, what's the relative amount of alpha and the relative amount of beta?
It's given by the lever rule.
OK. Let's look at a few other examples.
Here's hexane nitrobenzene.
And so they've even put l-alpha, l-beta.
They put the-- actually, that's probably a-- it's hard to read.
But I think it's really an F, but it didn't come through very well, so it's the beta fraction, the alpha fraction.
So you can see how you use the lever rule.
So up here it's all p. That's the not pressure, that's my circle p. Single phase, two phase.
Drop down to 290 degrees, it separates into two liquids.
With the hexane-- with two liquids, you'll actually have them floating on top of one another, right?
Well, maybe.
Depending if the density difference is tiny, you might get a dispersion.
I'll show you that in a second.
All right.
So p equals 2, lever rule.
OK?
Hexane nitrobenzene.
Look at this one.
This is surprising to me.
Potassium chloride, sodium chloride.
It's almost lenticular with a little bit of depression, but down here, it actually separates.
This is polymer.
It's a polystyrene, polybutadiene, depending on what the-- at low index, polymerization index, they mix.
At high polymerization index, the two are mixing.
You get two phase, single phase.
Changes the mechanical properties, too, doesn't it?
Now this one-- some systems actually have a lower critical point.
Oh, I meant to tell you.
You see, there's this temperature here.
Above this temperature, they mix in all proportions.
You can ever have phase separation.
This is the maximum temperature at which you have no phase separation.
So this is called the consolute temperature.
And it's usually an upper consolute.
The higher you go in temperature-- it's like saying, do you dissolve more sugar in cold water or warm water?
You dissolve more sugar in warm water.
And eventually, you get to a temperature high enough that you can mix in all proportions, right?
That's the consolute.
Some systems, for entropic reasons-- and again, if you take 560 or some of the other thermal classes, you'll understand this, but for entropic reasons, you actually have homogeneous solutions at low temperature, and at elevated temperature, you get phase separation.
That's funny, isn't it.
So this is water triethylamine, and they mix in all proportions at this zone here.
And then above a certain-- it's called lower consolute temperature.
It actually phase separates.
Doesn't matter.
All you care about is p equals 2, phase separation, lever rule.
The rest is details.
This is a real hoot.
This is water nicotine.
It has a lower consolute temperature and an upper consolute temperature.
So it's got a solubility bubble.
You see?
Out here, things are soluble.
In here, they're insoluble.
So they are emissible.
So we call this zone, this region here under the dome, so to speak, this it's called the miscibility gap, because things in there are not soluble in one another.
But with nicotine, you have a lower consolute and an upper consolute, so you have a miscibility gap that's a complete ring.
That's cool!
I like this.
All right.
So now I'm going to do a little experiment.
We have a few minutes, so I'm going to show you ouzo water.
I'm going to mix it, and I'm going to actually mix ouzo and water, and go into the two phase regime, and then out.
So this is what it is, all right?
It's oily stuff, you know.
If you're of Greek ancestry, you know what this stuff is.
All right?
But it's got licorice, it's got a fair bit of oil in it.
So what I'm going to do, is I'm going to come in here, and I'm going to start adding ouzo to water.
And they're both clear, colorless liquids.
OK, Dave.
Let's go to the-- All right.
So what we're going to do-- this is distilled water.
So I'm going to put some distilled water in here.
A little bit of distilled water.
And this is ouzo.
It's clear and colorless.
OK. Can we show you this?
Let's do this.
OK. Ouzo, it comes from Greece.
All right.
So it's also clear and colorless.
That was the point.
Not to show you the label.
See?
It's clear and colorless.
So this is clear and colorless, and the water, as you know, is clear and colorless.
So now what I'm going to do-- see, I don't have to wear a smock, or goggles, or anything.
This is great.
So what I'm going to do, is I'm going to add some of this stuff.
Clear and colorless.
OK.
It's turned milky.
Why has it turned milky?
Because we've now crossed into here, and the second phase is coming out.
But the density difference and surface tension differences are so slight, that instead of having the second phase float on the first phase, we have a dispersion, G. If you go to the dairy case, and you look at something called milk, you'll have the same thing.
Why is milk milky?
Because the fatty phase is a fine dispersion, and the particle sizes is dadadada, versus the wavelength of visible light, et cetera.
Now if my phase diagram is correct, and I'm in here, if I keep adding ouzo, I should eventually emerge on this side, and instead of having a milky dispersion, I eventually should come here, and now I'll have a homogeneous, single-phase solution.
But now it's going to be ouzo-rich, instead of water-rich.
You never thought phase diagrams were interesting.
You don't know.
There we go.
Je vous presente.
There it is.
So what we've done is, we've come across the thing.
Yeah, that's good.
Just put that there.
Now, David, please, back to the slides.
May I have the next slide, please?
That's a joke.
When you have these terrible speakers at conferences, they get up there, really nervous, and they stand up there, and the first thing they say is, may I have the first slide, please?
That's sort of a gag among scientists.
What's your opening statement?
May I have the first slide, please?
May I have the next slide, please?
OK.
So we've done this.
All right.
Now-- yeah, OK, this is just more.
All right.
So now I'm going to show you absinthe.
Absinthe is the same thing.
Absinthe also comes from the same family.
And there's a little culture here.
It contains wormwood.
The wormwood was up at around 200, 250 parts per million.
And what wormwood does, it's got a hormone they call thujone.
And thujone antagonizes the gamma aminobutyric acid, which moderates firing of the neural synapses.
Basically, we've got this GABA that regulates how-- our brains could work much faster than they do, but the GABA regulates.
If GABA doesn't work, you can start moving so fast that you get muscle thing, and you get epileptic, is one example.
But anyways, what it does, is if you drink this stuff, instead of being a stupid drunk, you become a very high-functioning, very alert drunk.
And I'm going to show you what the consequence is in art.
Anyways, the cultural piece here was that there was a destruction of the French wine industry in the 1800s due to a blight called Phylloxera.
It's like a beetle, and it ate the vines.
And in fact, the French wine industry was saved by the American wine industry.
Vines from New York state were brought over, grafts were made, and virtually all French wine today is on American stock with the exception, occasionally you'll see something says vieille vin.
Old vines.
There was some areas that were spared.
So what's that have to do with anything?
It has to do with the fact that when the Phylloxera hit, the price of wine went very high, and the people at the bottom of the socioeconomic ladder couldn't afford wine.
So they turn to absinthe.
Absinthe was easily produced.
Thirty years later, when the vines come back, the French wine industry wants to recapture the market.
So there's a Faustian bargain between the French wine industry and the Women's Temperance Union to try to disparage absinthe.
And there were a few major show trials that involved vicious murders in which it was alleged that the killer was deranged on absinthe.
And with this, they slandered the name of absinthe so badly that it was eventually banned and literally taken off the market.
I mean-- you know, we had the Rosalind Franklin commentary a few lectures ago.
I hope I never read about you doing something like this in order to help your start-up company take over market share.
That's not the way you're supposed to do it.
Anyway, so what-- now, how did they drink it?
They drank it by mixing-- so it's now back, but it's got very low levels of thujone, so it's-- David, may we go to this?
So we're going to make a mix called the louche.
It's a-- OK, here it is.
And this stuff here is beautiful.
Because it's a green color, a soft green color.
And the French even have a name for it.
They call it the green fairy, la fee verte.
So this is what absinthe looks like.
Beautiful.
Very nice.
And it's got the same kind of licorice smell to it.
And so you make the louche by one part absinthe and five parts water.
So this, at no upcharge, you get this glass.
It's a beautiful piece of, you know, late Belle Epoque glassware.
And the interesting thing is, these people were absolutely crazy, scientifically.
And so what they did is, the markings on this glass are such that if you put absinthe up to this ring here, and then you put water the rest of the way, you get exactly the five to one ratio for louche.
So you don't even have to measure.
You just put like so.
OK. Let's see.
I'm going to get this right!
It's science.
Here's our distilled water.
You see how it's milky?
So there's the louche.
And if you wanted to be a real hipster, you had this special Belle Epoque spoon.
And what you'd put on top of this would be a sugar cube.
You know, this is how it was done.
You have to know some culture.
You don't know.
You've led sheltered lives.
You pour it through here, like this.
So now I'm going to show you the art.
So David, may we cut to the slides again?
So there's the louche.
All right.
So here's the poster.
Here's the man.
He's pouring the water through the slotted spoon with the thing, and there's a very well-to-do lady, you can tell, she's got a beautiful hat, and she's well-dressed.
And he's inviting her to join him for a glass of absinthe.
L'absinthe oxygenee.
That's the name of the company.
The oxygenated absinthe.
C'est ma sante.
This is my health.
Van Gogh painted this.
This is Picasso.
The absinthe drinker.
There's the absinthe again.
This is Picasso after many absinthes.
You can see the slotted spoon and the sugar cube.
What else is there?
That's up to you.
All right.
When you saw Moulin Rouge, you may not have known all of this cultural history.
So now let's take a look at what's going on here.
There's the absinthe, and I think we've got a little-- [VIDEO PLAYBACK] -I don't even know if I am a true Bohemian revolutionary.
-Do you believe in beauty?
-Yes.
-Freedom?
-Yes, of course.
-Truth?
-Yes.
-Love?
-Love?
Love.
Above all things, I believe in love.
Love is like oxygen.
Love is a many-splendored thing-- [END VIDEO PLAYBACK] Love is like oxygen.
Chemistry is everywhere!
All right, so this is what happens eventually.
This is the poster.
This is from Switzerland.
October 7, 1910.
Gentleman, This Is the Hour!
And there's the clergyman with the Bible, and there's the green fairy.
And she has a wand.
She's been stabbed, but she's lying here with a wand.
The wand is an opalescent wand, because this stuff is opalescent.
Last thing I'll show you is some really, really cool chemistry that you must remember.
When Toulouse-Lautrec drank-- let's go back, David, to the thing.
So when Toulouse-Lautrec drank absinthe, and he drank lots of it, he didn't like the milky color.
So he wanted to make the milky color disappear.
So I'm in the two-phase regime, and I've got an oily phase.
And so how do I get rid of the oily phase?
What he did is he added cognac And why did he add cognac?
Not because he was a hard-drinking alcoholic.
It's because if you've got a fat phase here, and you've got an aqueous phase here, and if you add alcohol, you've got CH3CH2OH.
This can bond to the water by a hydrogen bond, and this aliphatic tail can stab the fat and bring them into solution.
That's why you have these recipes.
Do you ever wonder why the recipe says, add brandy, or add this, and then two steps later, it says flame?
You say, geez, I just put the brandy in, now it's vaporizing away.
Isn't that kind of stupid?
No, it's not!
This is what you're doing.
You're cosolvating.
If you're ever making a cream sauce or something, and all of a sudden, everything just curdles?
First you scream.
You go, ahhh!
Phase separation.
The second thing you do, is you get some of this, and you cosolvate it.
All right.
So let's see what Lautrec did.
Lautrec-- I might have to dilute the volume here.
We've got quite a lot in here.
So just to make the point.
So here's the louche, ad now we're going to add cognac.
He called this drink, Le Tremblement de Terre.
The earthquake.
He had a walking stick a meter long that was hollow, and it always had this in it.
Look!
Look at that.
Isn't that something?
So now it's this beautiful blonde, golden color.
It's clear.
So this is all phase separation and so on.
At the end, it's all about chemistry.
All right.
We'll see you on Wednesday for our wrap-up.
Good.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, OK. Settle down, settle down.
All right.
Today is the last lecture.
You think you're happy?
I'm happy.
So what I'm going to do, I'm going to teach for a little bit, wrap it up, and then I've got some comments about the final exam, and then some personal observations, reflections on the class.
So of course, the main announcement is the upcoming celebration of celebrations, a week from yesterday.
The final exam will be Tuesday, 15 December, 9:00 to noon.
I know 9:00 a.m. is kind of early for some of you.
Use the buddy system.
Get to that room on time.
9:00 to noon in the Johnson Athletic Center.
I'll have office hours on Monday, 3:00 to 5:00, and Hilary will find a room.
We'll post it, and you can swing by, if you want.
I guess the other thing I should say is just a reminder about the course evals.
Please take a few minutes and fill out the course evals, if not for me and for your successors, but certainly for our TAs, so that I can write nice things about them when the time comes.
So last day we looked at binary phase diagrams.
And we looked at two different types of binary phase diagrams.
Type one, which is the lenticular, and type two, which is synclinal, and in both instances, we came upon two-phase regimes, and when p equals 2, that causes you to start thinking along the lines of phase separation.
Phase separation then invokes a tie line.
The tie line tells you the composition of the two phases present, and the lever rule tells you the relative quantities, the relative amounts, of the two phases present.
What I want to do today is talk about the last type of phase diagram that we'll speak about in 3091, and that's called type three.
I came up with these very original, easy to remember labels.
So type three.
What characterizes type three phase diagrams?
Well, we have partial solubility of A and B.
We're doing all of this.
A is one of the components, and B is the other component.
So partial solubility of A and B, and change of state.
And the result of this set of characteristics is freezing point depression of both components.
And that's different from the situation in type one.
In type one, you see, if we have B on the right, then adding at A to B, if this is pure B over here, and this is pure A over here, adding A to B causes the freezing point of B to fall.
But adding B to A causes the freezing point of A to rise.
In this type three, we're going to have freezing point depression of both B and A.
That's the difference between the type one and type three.
And actually, I'm going to approach this by a hybridization of sorts.
So I'm going to consider type three a hybrid.
It's a hybrid diagram of the lens and the syncline.
I'm going to blend the lens and the syncline, and show you how to do that.
So let me get one of those up here, so that we can look at it.
So what I'm going to do here, this is a great diagram to look at.
It's gold nickel.
And at the top, we have a type one, and at the bottom, we have a type two.
And what I'm going to do, is I'm going to mix them.
I'm going to hybridize them.
I'm going to bring this bottom one up and have it touch the lens.
And so we get is something that looks like this.
We'll plot temperature versus composition.
Left-- so here's temperature.
Composition.
On the left I have pure A and on the right I have pure B.
And we'll start here with the melting point of pure B.
And when I add A to B, I'm going to get the depression that we see in the type one diagram.
We want to start off like this, and use everything we've learned up until now.
So up here, we have all liquid.
This is all liquid.
And over here, this is a solid solution.
This is pure B, and this is B that has a little bit of A in it.
So we're going to use the old metallurgical definition and call this beta.
Beta meaning, beta is a solution of A and B, but it's very rich in B.
It's almost pure B.
So I'm going to call this a B-rich solid solution of A and B.
And we know from last day, you can't go from p equals 1, let's get those labels up here, you can't go from p equals 1 to another p equals 1 without going through a p equals 2.
So this is single phase and this is single phase, and that means this one here must be two phrase.
And what are the two phases?
The two phases are the phases that are on either side.
So this must be liquid plus beta in here.
And that means-- phase rule-- p equals 2, tie line, lever rule.
So that's what's going on in there.
And then the difference is, on this side, if this is t melting point of A we're going to start off with the same concept here.
That we're going to get freezing point depression of A, as well.
So we'll have the liquid.
Over here we'll have an A-rich solution.
So then I'm going to call that alpha.
So alpha is an A-rich solid solution of A and B.
And then that must mean that in between here, we have liquid plus alpha.
And these keep going.
And in addition, down at the low end, I'm going to start with a syncline.
I'm going to start with a syncline, coming up like so, and coming up like so.
See what I'm doing?
I'm taking the lens.
I'm going to meet the syncline.
So what's here?
This is alpha, this is beta.
What's inside the syncline?
This is miscibility gap, so this is alpha plus beta.
So let's keep the phase labeling here.
This is p equals 2.
And look at how things have worked out so nicely.
p equals 1, p equals 2, p equals 1, p equals 2, p equals 1, p equals 2.
So nice.
And what happens when all this intersects?
It intersects like so.
Try to do this, stretch it a little bit.
So these two lines connect.
They cross at this point.
They touch at this point, the don't cross.
They just touch.
And then this arc comes up like so, this comes down like so, and now we have a line across here, and this comes down like so.
OK?
So the syncline stops at this value, which is the minimum temperature dictated by the two liquidus.
We have a liquidus here and a liquidus here, and this gets you down to the lowest temperature.
And let's look at the phase mix going in this direction.
p equals 1, p equals 2, p equals 1, p equals 2.
What's happening at this point here?
This point, we have three phases in equilibrium.
We have liquid, alpha, and beta.
And this is the special point here.
This is called the eutectic.
This is the eutectic, which is the temperature at which we get liquid, all liquid, at the lowest value.
So I'm going to call this here, I'm going to call this point e, and e is the eutectic.
Lowest melting point on the diagram.
Comes from the Greek, meaning easy melting.
And at the eutectic, eutectic is special.
Here we have p equals e. It's a triple point.
The eutectic is a triple point.
OK.
So now let's take a look at some of the examples of such diagrams.
All right?
This is a good one now that winter is upon us.
This is the antifreeze diagram.
So on the left, I have pure water, and on the right, I have pure ethylene glycol.
And as you can see, if you add glycol to water, you get freezing point depression.
If you add water to glycol, you get freezing point depression with a eutectic here at roughly two-thirds.
This is in volume percent, because that's practical unit.
When you mix glycol with water, you do so by volume, not by mass, or certainly not by moles.
So if you mix two glycol with one part water, you'll end up with a eutectic of minus 69 degrees C, or minus 92 Fahrenheit.
So that ought to take care of your driving needs.
All right?
Now practically speaking, we put an equal volume mix.
So it's about a 50-50 glycol-water water mix.
And that gives you liquidus depression down to about not quite minus 40.
It's about minus 35 Fahrenheit, which ought to take care of most peoples' driving needs.
Remember, what we're trying to do is to prevent the liquid in the engine block from freezing.
because of if it does freeze, you know that ice has a larger volume than water, and that energy is so great that you'll crack the engine block.
So you have to prevent it from freezing, the water from freezing, and so you add the glycol.
If, by chance, the temperature dropped to minus 50, you'd still be OK, because you'd be in the slush zone, two-phase regime.
You'd have some water ice forming, but you'd still have slush.
And by the way, why does this not look like this?
It turns out that the amount of solid solution behavior of a glycol in water is vanishingly small.
So this point is slammed up over here.
So that's why you don't see the right side of the lens.
And likewise, on the other side, the amount of water that's soluble in glycol is vanishingly small.
So according to this scale, you don't see the beta or the alpha phases here.
It looks like it's pure, but it can't be, because of the phase rule here.
Now, you might say, well, why don't we run pure glycol?
Because somebody might say, I'll just run pure glycol.
Well, here's the problem with running pure glycol.
First of all, if you get beyond two-thirds, the melting point rises.
But then you could say, well, but pure glycol, the solid glycol, has a smaller volume than liquid glycol, so it wouldn't freeze and crack the engine block.
But there's another problem here, and that is that the viscosity of water is 1 in these units, centipoise, and glycol is 21.
So you'd probably burn out your water pump by running pure glycol, because it's so viscous, the water pump is going to be fatigued by running.
And here's the point.
This is the 50-50, minus 37 degrees Celsius.
OK. And so what I've done, is I've superimposed the solid-liquid diagrams with the liquid vapor diagram.
And this is one of the times when nature is kind.
So when you add glycol to water, water is on the left here, as you add glycol to water, you get freezing point depression, and you get boiling point elevation.
So adding glycol to water opens up the liquid range.
You want to stave off boiling, because boiling will because lousy cooling, and the engine will overheat, and you want to stave off freezing, which will cause the ice to form and crack the engine block, and you get both.
Then you put the pressure cap, and then jump this up to about 265 degrees Fahrenheit.
So this is a very good example of management of the properties of the coolant through chemical change.
And the binary phase diagram eutectic type is a critical piece here.
Again, apropos of the weather, if you get ice on the sidewalks, if this rain freezes, it's terrible here.
Those of you who are in the Northeast for the first time, this is bad.
Not with what you see now, but when this freezes, it's a skating rink, and you can't see it.
It's black ice.
You're walking along, next thing you're on your-- you're in the horizontal position, all right?
So we can add sodium chloride to water, and when we do, we cause this, a eutectic type diagram, which can give you freezing point depression and prevent ice formation, and if you get enough sodium chloride in, you can get the temperature down to minus 21 degrees Celsius.
And I've color coded all of these.
This is the liquidus, this is the solidus, and then this line here, I can't remember if we gave it a name last day, this is called solvus.
This is the solvus.
This is, we know, liquidus.
This is liquidus, and this one here is solidus.
Yeah.
OK.
So this is all color coded.
And notice that minus 21 Celsius is not too different from 0 degrees Fahrenheit.
Turns out, that's how the Fahrenheit scale was born.
Daniel Fahrenheit was a glassblower in Leyden who also made exquisitely accurate mercury thermometers.
And he used ammonium chloride, not sodium chloride.
And the lowest temperature he could get for all liquid with ammonium chloride, he pegged as 0 on his scale.
And it's roughly this.
It's a little bit different, but you can see the similarity.
Now, you need two points to define a scale.
So he took this mercury in bulb thermometer, put it in the eutectic mixture of ammonium chloride and water, said, that's zero, calls in his wife, tucks the mercury in bulb thermometer under her armpit, and says, that's 96.
And that's the birth of the Fahrenheit scale.
And that's why we have water freezes at 32, and it boils at 212, and all the other nonsense.
You know, it's 98.6, but here it's a little bit colder.
Now, why he chose 96 and 100 instead of 100 or something, you know, you could read in the history.
Maybe he wanted something that was a power of two.
I don't know what the reason for it was, but that's why we have the Fahrenheit scale.
OK. Now, maybe you're a little bit hypertensive, so you don't want to throw sodium chloride on your walk, so you can put potassium chloride.
But you won't get much of a freezing point depression, And point of fact, we usually salt our roads with calcium chloride, which gives you two advantages.
Number one, very, very low eutectic.
Number two, it's a lot cheaper than anything else.
So it's the cheapest one to use, and that's why we use it.
Christmas is coming, and I told you that phase diagrams would save the day.
We saw that the cubic zirconia was stable in pure zirconia only at temperatures above 2000 centigrade, which is useless for a ring, or other such pieces of jewelry.
But by the addition of calcium oxide, we stabilize the cubic form of zirconia all the way down to room temperature.
And now we've got the high index of refraction, clear, colorless gemstone that passes for diamond.
Poor man's diamond.
Cubic zirconia.
And you can see in this regime here.
And notice that cubic zirconia-- here's a eutectic.
Why I'm showing you this is that the zirconia- calcia system ultimately becomes eutectic.
And the eutectic is at about 2300 degrees C. But over here is the solid solution.
Notice, if it's a solid solution, that means it's p equals 1.
Single phase.
As it must be.
If it were two phase, the gemstone would be frosty, right?
You'd have all those internal surfaces, and they'd be reflecting.
So this is clear and colorless.
And what's it bounded by, this region?
Cubic zirconia plus calcium zirconate.
So p equals 1, single-phase field, p equals 2, dual-phase field.
And what's on the other side?
Over here it's tetragonal solid solution, single phase.
Here's tetragonal solid solution with cubic zirconia, two phase.
Here's cubic zirconia, one phase.
Here's cubic zirconia with calcium zirconate.
Two phase.
Same concept here.
p equals 1, p equals 2. p equals 1, p equals 2.
OK.
So now you know.
When you go in jewelry store, ask the guy, is it single phase?
He'll say, get out of here.
Actually, you know how they test?
The index of refraction is so high with zirconia.
It's a little bit less than diamond.
Diamond's up around three.
Cubic zirconia is over two.
One of the ways they test is, diamond has a very high band gap, about 5.5 electron volts.
But it's got the remarkable property, even though it's an electronic insulator, it's a very good thermal conductor.
Most materials that are poor conductors of electricity are poor conductors of heat.
But diamond, for reasons I can't explain with just the utility of a 3091 education, has a good phonon transmission properties.
And the way they tell cubic zirconia from diamond, is they actually put a couple of tips on either side of the stone, and they heat one tip, and measure the time lag for the thermal wave to get to the other tip.
And if it's cubic zirconia, it takes a long time, because cubic zirconia is an insulator.
If it's diamond, it's almost like metallic conduction.
And that's how they can tell a fake diamond from a real diamond.
Here's, you know, I've been drinking from these cans all semester.
And this is the alloy system, aluminum magnesium.
So we put about 1% magnesium into the aluminum.
So here's pure aluminum over here.
It goes down to eutectic.
This is pure magnesium over here.
They both melt at roughly the same temperature.
And by putting about 1% magnesium in here, it changes the metallurgical properties favorably.
Notice we don't want to put 4% or 5% magnesium in here.
Otherwise, as we get down to room temperature, we'll enter a two-phase regime, and that means we won't be able to process.
Because this can is made from a single sheet of aluminum.
The body of the can.
There's two pieces, the top and the body.
There's no third bottom here.
Start with a flat sheet of aluminum and it's punched deep, drawn down, up, down, and it gives you the whole shape.
And you have to have, you know, this is FCC.
You have to have the glide.
Dislocations are gliding, the slip systems are operative, and the magnesium gives you just a little bit of strength.
But if you get into the two-phase regime, the punch, instead of making deep drawing metal, will end up punching holes.
So what's the difference between drawing and punching holes?
It's all in the metallurgy.
So you can see, this is where we are.
OK. Ah!
Microelectronic devices.
How do we hold all those wires together?
With solder.
And this is the phase diagram for solder.
It's made of lead and tin.
And we use eutectic.
The eutectic here is about 62 weight percent tin at 183 degrees Celsius.
I took a class in high school called Industrial Arts.
It was shop.
We had it drilled into us, the eutectic point of soft solder is 374 degrees Fahrenheit.
I will know that on my deathbed.
Plus the wavelength of copper K-alpha radiation.
1.5418 angstroms.
Yeah.
So what's the magic here?
The magic here is, the solder must be low-melting enough that when you're-- remember, you've got your microelectronic device.
You've already got your chip, and you're putting wires.
You want to put enough heat in to solder the wires, but you don't want to have to raise the temperature so much that you can damage the rest of the device.
So you say, well, why don't you get something that's lower melting?
Well, if it's lower melting, you know from running your laptops and charging your cell phones, they get warm.
So it's a Goldilocks problem.
If the melting point of the solder is too low, you might encounter that temperature in service, and pop all of the joints.
If the melting point of the solder is too high, then you have to use so much energy to make the joints that you damage some of the microelectronic componentry.
So this has to be in the right temperature range.
The problem with this excellent, excellent solder is that it contains lead.
No one's going to get lead poisoning from using a device.
But the disposal of the device ends up introducing the lead into the landfill, and then if it soaks in an aqueous solution, you could get leeching, et cetera.
So there's a lot of research right now on trying to find other solders that operate.
And there are other solders, but they're very expensive.
So that, how do you find something that's functional the way this is, nontoxic, and cheap?
They can give you any two of those, but to get all three of them to line up?
Requires research.
You see the alpha phase, beta phase, everything there.
All right.
And here's the lead tin system.
And let's just take this one that happens to be 40%, and let's see what happens.
We'll go through in time.
So we start here, all liquid regime.
And so this bubble shows what you might see on a microscope slide.
So it's all white here, meaning that it's homogeneous single phase.
And then you drop into the two-phase regime, just below the liquidus, and you start seeing these blobs.
Those little blobs are the primary alpha.
That's the solid solution, right?
Because we're in the two-phase regime, so we've got a tie line here.
We get down to a lower temperature, there's more of this stuff.
There's more of this stuff for me, because it's forming in time.
It takes you time to traverse this.
This isn't parking and waiting.
This is just cooling something.
Imagine you're soldering, so you took the temperature up to here, now you're going to let it cool.
Once you cross the eutectic temperature, now it's alpha plus beta.
So now you get this lamellar structure.
Stripes of alpha, stripes of beta.
But the previous blobs that formed the primary alpha, they don't disappear.
The only way they can disappear is by solid state diffusion.
That's going to take a long time.
So I can look at the microstructure here, and I can predict-- not predict-- I can go back and calculate what the cooling rate must have been.
If I cool very slowly, I will have nearly the equilibrium structure.
And so that's why, when I told you about, say, analyzing the records from the World Trade Center, I can look at the metal, and look at the phase distribution, and tell you what the thermal history was.
And so for example, there was some cock-eyed theory saying that the steel was burning.
Somebody said that the steel melted, and so on.
You can look and you say, that did not happen.
Temperatures went fairly high, crossed into some different solid phases, went from BCC to FCC and then back.
But didn't melt.
Because if it melted, we would have seen this kind of structure.
So this is-- every time a plane crashes, this is the metallurgical investigation that is taking place.
The history is written in the microstructure.
So this is a micrograph that's actually taken from some solder.
This is 50 weight percent.
The slide was 40, but this happened to be 50.
It's the one I could find easily.
So 50 tin, 50 lead, and these dark zones are the primary lead-rich alpha.
And it's all metal.
So how do I get the dark and the light?
First we polish, because it's an optical microscope, which has a really crummy depth of field.
So you've got to make it flat, and you polish it, and now you've got a mirror, and you look, and all you see is your eyeball.
So what you do then, is you etch it with some acid.
And the lead will etch at a different rate from the tin.
And the contrast, then, is what you see here.
So this is all the blobs of primary alpha.
Then you go below the eutectic, and, you know, this cute little schematic of stripes-- this is the textbook version of it, but this is the real version of it.
And you see, it's a distorted lamellar structure.
Alternating stripes.
Because you're trying to make alpha and beta.
And so you're making alpha and beta, and the system's trying to make both of it.
So it makes it in striped form called lamellar microstructure.
So I can look at that and I can say, wow.
That must have been cooled at a fairly rapid rate, because you've got a large amounts of primary alpha.
If it were cooled at a very slow rate, the thing should be all lamellar structure, right?
If you're down here, there should be no alpha.
This is not the equilibrium phase.
So this alpha is trapped there.
It's a victim of its history.
So there it is.
That's cute.
That's a nice, it's a beautiful graphic.
See?
There it is.
All right.
So I'm going to show you some phase diagrams from hell.
This is, you're going to be on airplanes.
The airplane alloy is aluminum copper.
It's about 4% copper and aluminum.
Now remember, I just told you about 1% magnesium and aluminum to give you single phase.
In the case of the airplane, we want to make the wings strong.
So what we do, is we come down into here, which is a two-phase regime.
And if we're clever about it, we can control the particle size of that second phase.
And by controlling the particle size, we can cause grain boundaries, which can arrest the dislocations and make sure the wings don't fall off.
This is aluminum manganese.
This is the one that Danny Schechtman worked with to get the quasicrystals.
Look at all the phases.
He's running out of Greek letters by the time you get across this diagram.
Look at it.
But one thing holds.
The two-phase regimes are always bounded by single-phase regimes.
Look.
Here's liquid, here's a two phases, there's a single phase, two phase, single phase, two phase.
Beautiful.
All makes sense.
Now, what happens if you try to take a metal and mix it with a nonmetal?
They don't want to mix.
Look at the big miscibility gap here.
Molten iron doesn't want to mix with molten sulfur.
This is a liquid-liquid miscibility gap.
Over here, you can add some sulfur to iron, you get a eutectic, get a line compound.
And then over here, no dice.
Does not want to mix.
And look at all the solid phases.
Why do we have all these solid phases?
Different crystal structures.
OK.
So now what we're going to do, is I'm going to talk about application of phase diagrams to the making of champagne.
Because champagne relies on phase diagrams.
And then I'm going to make some general comments.
But I'm going to actually show you some champagne.
So before I get into my commentary, I'm going to get some champagne chilling.
So first I have to show you how to chill champagne.
First I will show you how not to chill champagne.
This is how not to chill champagne.
And it's not because, OK, we can put it in there.
Why?
Because, as I showed you last day, with respect to the cooling of the engine coolant by the radiator, if you have-- if this a bottle here, here's the contents, the precious contents.
And I have this with blocks of ice.
The contact between the ice and the bottle is sporadic, and I've got air in between, so the quality of the cooling is very poor.
So instead, what I do is I rely on the phase diagram.
So what do I do?
What do I know about the phase diagram?
if I pour water in here, I will thermostat the entire system, liquid plus solid, at 0 degrees.
Because the ice is probably, I don't know, minus 3, minus 4 Celsius?
But the air is 20, and it gets in there, and it's very few atoms.
It's crummy heat transfer.
Instead, if I flood this with water, the whole thing is thermostatted at 0 degrees C. So now I've got a good delta t, because I started here at about 22 degrees C. I've got a delta t, and it's a fixed law, right?
It's going to be what?
It's going to be the flux.
The heat flux is going to be minus thermal conductivity, dt by dx.
It's Fickian type motion of the thing.
So let's do it.
This is how you properly chill-- I'm told it also works for sodas and fruit juices.
This technique, I mean.
But I do know that it works with champagne.
Now that's working beautifully.
I can just feel the heat being extracted.
OK.
So now let's talk about champagne.
All right.
So we're going to talk about champagne and a great inventor, and the inventor is the Widow Clicquot.
This is the Veuve Clicquot champagne.
And veuve is the French for widow.
So Veuve Clicquot, her real name was Nicole-Barbe Ponsardin.
And in 1798, she married Francois Clicquot.
This is a big French winemaking family.
And he died and left her a widow at the age of 27.
She was very unusual for a French woman in the 18th century, because they were not involved in society.
They were certainly not involved in business.
This woman was different.
She decided to get under the hood, and she took control of the winery.
And I've written here, bold, imaginative management.
There's a fantastic book about her.
It's actually a very good book portraying late 18th century, early 19th century France.
And these are among her accomplishments.
Marketed champagne to all the great courts of Europe.
Champagne was a regional beverage, drunk only in the Champagne region.
And she created the myth, propagated the myth of champagne.
You use it for festive occasions, bought land in the best vineyards, quality control.
It's about the chemistry.
Fought fiercely against counterfeiting.
As she started getting champagne more recognition, people started making sparkling wine and labeling it as champagne.
She used the court system to take them down.
Established strict quality control procedures.
That's all about defending the brand, and what's the quality control?
It's the chemistry.
Produced the first rose champagne.
The Pinot Noir grape has a black skin.
Depress, and then take the juice, and away you go.
What she did, is she let the skin sit with the juice for long enough to give a just a rose tinge.
And then oversaw the invention of new technology.
Remouillage.
And that's where the phase diagrams come into play.
Phase diagrams give us champagne we enjoy today.
So what's the problem?
Champagne is cloudy.
Why is it cloudy?
Well, here's the chemistry.
Here's how you make, this is how you make all wine.
It's one-stop shopping.
You take the grape.
The grape has sugar in the grape juice.
And yeast that attacks the sugar is in the skin.
So the yeast attacks the sugar to make alcohol and CO2.
So ethyl alcohol and CO2, plus some insolubles.
There are some insoluble products of that reaction.
And there's two types of insolubles.
Those that will settle, they're called sedimentary, and those that won't settle.
They're called suspended.
And how do you get rid of the sedimentary stuff?
By mechanical separation.
You let the juice sit, this reaction is taking place, and the sedimentary stuff is turning into gunk, and it sits at the bottom.
And then periodically you siphon.
You siphon the juice off, and you leave all the crud at the bottom of the barrel, and then you go for a few more months, and then you siphon again.
Well, that works for the sedimentary stuff, but it won't work for champagne, because as you're siphoning, you're letting out all the gas.
You see, you're trapping this gas.
The champagne has CO2 that's naturally occurring.
It's not carbonated.
It's carbonated naturally.
Not carbonated, man-made carbonation.
It's natural carbonation.
So how do you separate the juice from the crud without losing the gas?
That's the problem.
And by the way, the other stuff that's suspended, they use tricks.
You know, here's the free body diagram.
Remember, I showed you this for milk.
So what they do, is they add things like eggshell, and the eggshell acts as a coagulant, so the tiny particles that will float agglomerate to become big enough that then they'll settle.
Tiny particles of a given density will float, large particles of the same density will sink.
So you need to coagulate, and do that without spoiling the taste.
Fantastic chemistry.
So she comes along with the answer.
By the way, the early champagne glasses were all cut crystal.
Why were they cut crystal?
Because the champagne was cloudy.
It tasted fantastic, but who wanted-- it looked like swill!
So how to make it clear?
So what they did, is they made champagne bottles with the deep ruts, and they still do to this day.
They have a deep rut here, because the idea was, if you opened it carefully, you wouldn't disturb so much of the crud.
But it gets churned up by the gas.
So what she reasoned was, let's turn him upside down, and let's have the crud settle on the underside of the cork.
And furthermore, 45 degrees is the optimal angle.
You know this from your Newtonian mechanics.
And then you turn them, so you spiral the bottle.
Turn them, and have all the crud collect on the underside of the cork.
I know what you're thinking.
How are you going to get the cork out?
Now you take the cork out, you're going to lose not just the gas.
You're going to lose the liquid, too.
Ah, that's where the phase diagrams come in.
So here's a phase diagram of ethanol water.
And it's eutectic.
Look at, way down here.
In fact, ethanol makes a really good antifreeze.
Look!
A little bit of-- go way down here.
Now, most wines are somewhere between 10% and 15% alcohol, that gives you very modest freezing point depression.
Maybe about minus 5 Celsius.
Which is why if you forget, you put a wine bottle in the freezer and you forget about it, and you've got your freezer really cranked to, say, minus 5 Fahrenheit, you'll freeze the wine, and it'll expand, because it's 90% water, and push the cork out, make a mess.
But remember, you only get down to about minus 5.
Remember, I showed you this one?
It gets down to minus 21.
So what she reasoned was, what if you took the bottle upside down?
You've got all the crud on the underside of the cork.
What if you took the bottle and put it into not water, but what if I put sodium chloride in here?
I still have my ice cubes, only the temperature is now minus 21.
And that's below the freezing point of the alcohol, and now I've frozen an ice plug in the neck.
So now there's an ice plug between the crud and the cork.
So now I can turn the bottle like this, just melt ever so slightly, and the pressure in the bottle will blow the ice plug, take with it the crud out, and then I quickly put the cork on top.
And that's how every bottle of champagne is made.
And it all comes from these phase diagrams.
See?
So you put it in brine.
Minus 21.
But this thing is frozen already.
That red is frozen.
And now I've turned it right side up, out it goes, takes the crud with it, put the cork back on.
So it's a combination of this phase diagram, this phase diagram.
Both eutectic phase diagrams would give us clear champagne.
Positively brilliant.
So there it is.
The invention and-- oh, by the way.
They call this-- the English word remouillage is riddling.
So they have people that actually go, and they, these wooden racks with the slots in them at 45 degree angles, and all these bottles are pointed downward, 45 degree angle.
Guys come in every day, quarter turn, quarter turn, quarter turn.
French are very traditional.
Few champagne wineries have motorized, so they have this thing called giropalette.
Gyrating pallet.
The California champagne wineries call it VLM.
Very Large Machine.
I kid you not.
I didn't make that up.
In California, the giropalette is called VLM, which is Very Large Machine, dude.
I guess.
That's what they say.
California.
All right.
So let's have a few words now about the final exam.
So you know, it's going to be Tuesday, 9:00 to noon, Johnson Athletic.
Three hours.
You have three full hours, not three times 50 minutes.
Three full hours.
But I'm not going to give you three times the work.
It's sort of a hybrid between intensive coverage since T3, so sort of like a T4 and a redemptory part.
It'll go back, extensive coverage of everything.
One aid sheet.
One aid sheet.
I'm saying it three times.
One aid sheet.
Do not come in there with a bunch of aid sheets.
Well, I didn't know!
I thought we could have-- no!
One aid sheet.
That's all you need.
And the TAs are smart.
You know, every year somebody tries to bring the origami and all that stuff.
And they get caught, and then-- you know.
Bring your periodic table, table of contents, calculator, a pen.
No headphones, no audio.
Please.
Because people are writing, they're trying to concentrate.
I don't want to get into a dispute over whether your music is too loud for your neighbor.
So let's just, out of courtesy, no audio.
You can go three hours without your music.
All right?
What else.
This is going to be comparable difficulty to the monthly tests.
Of course, read the entire exam.
You've got plenty of time.
Most people are leaving 11:15, 11:30.
It's not a race against the clock.
And make sure you show your work, justify your conclusions, solve algebraically.
Stay confident.
Do your own work, academic honesty, all that applies.
Now, your overall grade is going to be based on many factors, including the trend.
So we're going to look at how you performed throughout the semester.
You started off really strong, and you crash and burn on the final, or you've done the math and you say, I only need a 37 on the final, and you go in there, shooting for a 37, and you miss and you get a 35, and your total average is 49.5?
You go down.
Because you've got a 49.5 and you wrote a 35 on the final.
I'm just telling you.
Claim, you can have your exam papers back.
But don't come and see us before Christmas.
You can have them back starting IAP.
So starting January 4, you can have your exams back, if you want them.
And there's no time limit for appeal, so you don't have to be bursting into your TA's office on January 4 if you see that we've misgraded something.
You can come and see us.
There will be some security measures in place.
It has not escaped my notice that some people have attempted to erase answers that were written in pencil, and then bring in this newly brilliantly solved problem for regrading.
And that's, of course, qualifies as academic dishonesty, and I'll take you to the committee on discipline if you try it.
But just to inspire honesty on a random basis, we'll photocopy about 5% of the exams.
So if you want to try this, the question you've got to ask yourself is do you feel lucky?
Do you?
All right.
So now it's time for some personal observations.
So I wrote them down.
OK.
So first thing is, you've come a long way.
Syllabus is full.
You think about where we were on day 1, you know, this is equations, stoichiometry, all that baby stuff.
And you know, look.
So be proud of what you've learned.
Number two.
I wish you much success on the final.
Remember, no grade that I assign to you, or any of my colleagues assigns to you has any relevance to your value as a human being.
It's just a grade in a subject.
That's all it is.
What's the worst thing that can happen?
Suppose you get all Fs, and you're required to withdraw from the Institute.
So what?
So what?
You know?
You're all smart people.
I know you're smart.
I used to chair the Admissions Committee.
I know how smart you are.
If you're getting all F's, this is not the right place, it's not the right time for you to be here.
Look, I have no degrees from this place.
I feel great.
I've been teaching here at MIT for 32 years, and I've never enjoyed teaching as much as I have this semester.
This has been really a blast to teach, really.
I can't believe I said that with a straight face.
So of course it's not true.
I want to think my staff.
My TAs, sitting here.
The tutors who helped out.
Professor Demkowicz, who subbed for me that time, when I had the visit of President Obama.
Administrators, the audiovisual techs.
Dave Broderick, who's done a great job of making sure that we have sound and light.
And the academic media production services people, who have recorded the lectures so that on the rare occasion that you didn't make it the class, you're able to catch up.
And the last thing is that, I've got here, there's one more chemistry lesson.
You know, we've learned about the four different types of primary bonding.
Covalent, ionic, metallic, and in some instances, van der Waals.
But there's a fifth type of bond.
The fifth type of bond that I haven't talked about yet.
And it's the type of bond that's been forming in this room over the last 14 weeks.
Yeah, I know.
It's kind of sappy.
I didn't come up with this.
It came from a student.
So the fifth type of bond is-- it's not this one.
[MUSIC PLAYBACK] No, it's not this one.
It's a bond that's formed between people, and this bond has a very special property.
It can never be broken.
I know!
It came from a student.
It's very sappy.
I'll tell you the story.
I'll tell you the story how it happened.
I used to teach 321, which is a core graduate class in kinetics.
And I'd been teaching it for, I don't know, five, seven years.
And I was at a conference, and there's these two people standing there talking.
And I know both them.
One was from industry, and one of them was an alum.
So I go up, and the fellow in industry, says, hi, how are you doing, do you know so and so?
And I said, yeah.
He was at MIT.
I was his kinetics professor And the student, the alum, he says, excuse me, Professor Sadoway.
It's true that I was at MIT, but please do not say that you were my kinetics professor.
You will always be my kinetics professor.
I went, oh, man.
This is really something.
So now the tables are turned.
And, and, you know, I know you're going to go go on and do something remarkable in your life, and I hope it's remarkably wonderful and not remarkably stupid.
But you know, at some point, somebody may say to you, where did you learn your chemistry?
And you'll have to say, Sadoway at MIT.
So whether you like it or not, I will always be your chemistry professor.
So now, let's see.
I think it's time to open up the champagne.
So first thing, we have to teach you how to open champagne.
So there's the foil here.
So we'll take the foil off.
Take the foil off, very carefully.
And then there's the wire basket that keeps the cork on.
How many turns on the wire basket?
Everyone in 3091 knows.
Six turns.
One, two, three, four, five, six.
Six turns.
And so now, I don't want to point that at anybody, because the only thing between me and six atmospheres of pressure is that cork.
And Dave, if we can cut to the document camera.
The cover, the metal cover on this, has the image of the Widow Clicquot.
This is Veuve Clicquot, and this is the image of the Veuve, and she's on every bottle.
She was the one that gave us clarified champagne, so you thank her very much.
All right.
So now what we're going to do, is we're going to open this thing.
To open it, we point it 45 degrees and turn it ever so slightly.
Never point it at people.
No, really, point it away.
You'll put an eye out with this thing.
You turn it very slightly.
Actually, my hands are wet now, so I'm going to use a towel here.
Oh yeah.
Go like this.
You like the suspense, huh?
See that?
No pfff!
That's for football locker rooms.
It's very declasse.
OK. Now we have to have a glass.
So since we have clarified champagne, we're not going to use cut crystal.
We're going to use Baccarat, French crystal that's not cut.
Because we've got nothing to hide, right?
So now we're going to pour this.
And the rut now has some value.
The rut has value, because now what you can do is to pour it like so.
I've been waiting fourteen weeks for this.
All right.
So the surface tension, you see the carbon dioxide outgassing, because it's supersaturated.
Six atmospheres, not one atmosphere.
The bubbles are nucleating on the defects in the glass.
It's all about chemistry.
So there we go.
OK.
So I'm going to propose a toast to the class, to the 3091 class of Fall 2009, to wish you much success in your academic pursues, and much happiness in your personal lives.
And with that, I'm going to find out what's going on inside.
Look at those bubbles!
You know, the legend is, the blind monk, Dom Perignon, when he first drank champagne, said, I feel as though I'm drinking stars.
He's right.
Drinking stars.
OK.
This is to you.
The morphological unit of life is the cell, whether you're a flea or you're an elephant.
Right?
And we all have water as the major solvent.
We all have the same biological molecules that play a major role inside our bodies, like sugar or fat, or amino acids, or nucleotides.
We all have the same ways of making reactions work fast enough so they can succeed inside the cell, and controlling the specificity, and they're conserved from bacteria to humans.
We all use the same vitamins on your vitamin bottle.
And those are also conserved in general from bacteria to humans.
And we also all have the same major regulatory mechanisms, although as you go from prokaryotes to eukaryotes, things get more complex.
And so what we cover in the course is, in fact, most of the things we talk about are bacterial systems because they're better studied but can be extrapolated between the two systems.
So are these things ordered inside the cell?
And how do you study them inside the cell?
That's the major focus of a biochemist.
You want to understand things at the molecular level, but then you need to understand what you see at the molecular level, how does that relate to what's going on inside the cell?
And so you need to go back and forth between inside the cell and understanding the chemical and physical principles by which all the reactions work.
If you look at the vitamin bottle, you have all these vitamins you eat, like, what do they call them?
Biotin, riboflavin, pyridoxal.
So all of those small, organic molecules, and they all have chemical reactivity and they interact with the protein and allow the protein to do chemistry that can't normally be carried out by just the amino acid side chains of the protein in the active site of the enzyme.
So the vitamins you eat are not actually what's interacting with the protein, they need to be modified.
So they are the precursors to what are called co-factors that, again, expand the repertoire of what's found in enzymatic systems.
So you have a whole bunch of organic co-factors that actually can be made spontaneously.
If you throw in some simple molecules like cyanide and stuff, these things all self-assemble, so they're all from the primordial soup.
And in fact, many of these vitamins can do chemistry without any enzyme at all.
But they can't do it specifically.
And they can't do it rapidly.
So we have all these organic molecules that can self-assemble that expand the repertoire, but if you look at a vitamin bottle, you will see minerals.
And that's one thing that I think most biochemistry courses don't talk about, is metals.
And it's now proposed that between 35% and 50% of all the proteins in our body have metals that are essential for function.
And so these minerals that you eat, iron or zinc or calcium, all play a central role, again, in expanding.
They all help in many cases facilitate chemical transformations, and in the way we can understand by understanding basic chemical principles.
So the vitamin bottle, we come back to over and over again through the course of the semester, because all the enzymes in metabolic pathways have different kinds of co-factors that are required to make the enzymes function.
The way we teach the course is we show them that all of these chemical transformations can be described by 10 pretty simple steps.
And if you understand the basic chemistry of these simple steps, you can really understand almost all of the kinds of interconversions you see and basic metabolism.
And what you'll see is while this looks overwhelming, really, with a few central pathways, which is what we focus on in this course-- glycolysis, fatty acid oxidation, biosynthesis, sugar biosynthesis as well as degradation, the Krebs cycle, which feeds into the respiratory chain-- knowing those central reactions-- almost everything in metabolism feeds in and out of these pathways.
And so then, it really is a question of what is the environment like and how do you enhance breakdown of sugar under the appropriate environment versus synthesize sugar under a different environment.
So then it's a question of regulation.
So what you're going to learn in this course is really focused on central metabolism.
And it doesn't matter whether you study a bacteria or a human, the central metabolism is pretty much the same.
The thing that's different is the detailed regulation and the complexity of the regulation.
And we don't really talk that much about regulation in 5.07.
What we do is introduce you to five or six basic regulatory mechanisms that are used over and over again.
But then, regulation is really distinct, even between organisms.
And so then that becomes much more complicated.
And when you go off and become a biochemist, you've got to study your own system and figure out what the environment is.
We spend a lot of time looking at-- all proteins are made from amino acid building blocks.
And so one question I get, and students hate this, is, why do I need to know?
I make them memorize the side chains of the amino acids.
Why do I do that?
Because the side chains of the amino acids are the key to the way all the catalysts in your body function.
A catalyst simply enhances the rate of conversion of some small molecule into some other small molecule, and it can enhance the rate of the interconversion by 10 to the 15th fold.
So if you didn't have that catalyst, you couldn't do anything.
So amino acid side chains play a key role in catalysis.
So thinking about the chemical properties of the amino acid side chains is key to understanding all the transformations in your body.
So, what is the pKa of imidazole?
You remember that?
Huh?
I don't.
I don't
How can you not remember that?
[LAUGHTER] Anyhow, the pKa of imidazole is close to seven, OK.
So those physiological-- everything in the body is controlled.
The pH has to be controlled.
And so that means that you can protonate or deprotonate it.
So knowing that is key to thinking about how the chemical reaction is going to work.
The amino acid side chains now-- everybody thought there were 22 amino acids.
But now we know almost all these amino acids can be modified once they get into the protein, so that's called post-transational modification.
So now we have, probably, another 250 modifications.
And one of the modifications that is essential, not for the catalysis part of proteins, but for the structural part of proteins, is hydroxylation of the amino acid proline.
So if you don't hydroxylate proline, then you can't make this molecule called collagen.
And collagen is a structural protein.
It's 25% of all humans' protein, OK. And it's found extracellularly.
Gram per gram it has the strength of steel.
It has very complicated biosynthetic pathway.
It has amazing tensile strength.
It's found in cartilage and teeth and bone.
And a key component of collagen is this hydroxylated proline.
Because without it, you can't form the actual collagen structure.
So collagen is this long-- most proteins, if you look at the structures, they look like little balls.
They're globular.
But collagen is a fibrillar protein, so it's very long.
it's probably the longest protein, too.
It's 3,000 angstroms long.
And it has three chains initially, and they they're left-handed sort of helixes but not real helixes, and they have to wind around each other to form a right-handed helix.
This all happens inside the cell, then somehow has to get to the outside of the cell.
People are studying that now.
And then it forms additional fibrils, and they become insoluble.
And that provides the strength-- the extracellular structures provide the strength that maintain the cell's shape and viability of the cell.
And a key component of all that is hydroxylation of proline.
And how did they find that?
This goes back to, again, misregulation.
So, in the 1600s, or whenever they used to sail the ocean blue with no food, they didn't have enough vitamin C. So they didn't have any citrus fruit.
So vitamin C has the vitamin ascorbate, and ascorbate plays a key role in the chemistry of allowing the proline to become hydroxylated.
So it turns out, to get the proline to be hydroyxlated, you use, again, metal catalyzed reactions.
So here's iron-2.
And if that iron-2 reacts with oxygen, if it gets oxidized with iron-3, it can't react.
So the function of this vitamin is to keep the iron-2 in the reduced state.
So there's an example.
People study that.
Took them many, many, many years, like 200 years later, when they really understand the details of how this post-translational modification actually occurs.
And we understand a lot about that chemistry now.
And this was the first one of these modifications discovered, and now they're finding this modification everywhere.
So lots of amino acids turn out to be hydroxylated, not just proline.
So these modified amino acids now, because the technology we have is so mind boggling, we can find a needle in a haystack.
Whereas in the beginning we were just trying to figure out what the amino acids were, now we have very, very sensitive analytical methods which allow us to see all of these modifications.
Then the key question is, what is the function of the modification?
And that's not so easy in terms of thinking about regulation, anyhow.
And that's the focus of a lot of people's efforts right now.
When I went to graduate school, which was in the late 1960s, I didn't even know what an enzyme was, because back in those days I'd never had a biology course.
They didn't do biochemistry back in those days.
And I think I heard a lecture by Van Tamelen at Stanford.
He was interested in steroid structures, steroids that have four rings and it's the basis for sex hormones and for cholesterol biosynthesis.
They're all sort of made through a common biosynthetic pathway that we don't really talk about.
These are these five carbon units to form carbon-carbon bonds that I mentioned earlier.
Anyhow, what he showed was you could take 30 carbon strung together in a linear form.
And then somehow these 30 carbons folded up with one enzyme to form these four rings, putting in eight asymmetric centers in a single step at pH7 in 100% yield.
Once I saw that, I said man I don't want to be a chemist.
If you could see how nature designed this, and what is the basis for being able to make these molecules, it was sort of mind boggling to me.
And I think enzymes are like that.
What they do is they've evolved for billions of years to accelerate rates of reactions by factors as much as 10 to the 20th.
So that's really fast, like a bat out of hell.
And so nature has figured out how to control all of this.
But again, she has a limited repertoire of reactions that she catalyzes, but she's extremely good at it.
And so understanding people have studied enzymes forever because I'd like to understand the basic principles of catalysis.
And then if you understood those, can the chemist then take this understanding and translate it into the bigger repertoire of the periodic table you have to be able to do these transformations?
So the mechanisms of rate acceleration, which we talk about in some detail in a lecture, what causes these catalysts to work, and the way you describe how these catalysts work.
It doesn't matter whether you're using a small inorganic molecule or small organic molecules or a protein, the basic principles and thinking about catalysis is exactly the same, except nature has figured out how to do this better than anything man can do.
But again, she's limited in terms.
She's had a billions of years to evolve these catalysts, but then she's limited in the repertoire of reactions that she needs to catalyze.
So how can you not think enzymes are cool?
My lab works on the only cool enzyme in the world-- ribonucleotide reductase.
It's the only way in all organisms that you make the building blocks de novo that are required for DNA biosynthesis and repair.
So if you inhibit this enzyme, you have no building blocks.
You can't survive.
So from a practical point of view, it's the target of drugs they use therapeutically in the treatment of cancer.
And I think in probably not so distant future in the antibacterials because I think there are sufficient differences between humans and bacteria reductases that you could make specific inhibitors.
Why am I interested in it?
Because the chemistry is sort of unbelievable.
So I mean it was the first example where you learn, or you hear about-- you heard from John-- radicals.
They're reactive oxygen species and nitrogen species that you can't control.
They want to pick up an extra electron and form a stable octet.
And if you leave them to their own demise, they react with anything and destroy it.
Well nature has figured out how to harness the reactivity of radicals to do really tough chemistry with exquisite specificity.
And ribonucleotide reductases have been the paradigm for thinking about that.
And from bioinformatics now there are 50,000 reactions in metabolic systems that are going to be radical mediated transformations, yet we never talk about radicals in introductory courses.
So I think that's all going to change.
So why is it unusual?
Well, for the human ribonucleotide reductase, the key to making this work catalytically is the amino acid side chain tyrosine needs to be oxidized to a tyrosyl radical.
So automatically nobody believes that.
A tyrosyl radical in solution has a half-life of a microsecond.
In the active site of these enzymes, the half-life of the enzyme can be on the order four days.
And this radical, which is again one electron oxidized amino acid-- if you reduce it with an electron and a proton, the enzyme is completely dead.
So this was the first example of-- it would be another example of a post-translational modification that we talked about earlier-- modifying your amino acids.
And so nature has figured out a way.
How do you do this oxidation?
She has a little metal cluster right adjacent to where this tyrosine is.
And the function of this little metal cluster is to put this into the oxidized state, which is essential for the way the enzyme works.
So the other thing that's amazing about the enzyme is the chemistry.
There are two subunits.
The chemistry all happens in this subunit, but the tyrosyl radical is there.
And this oxidation-- normally when you do an oxidation the two atoms are sitting within a few angstroms of each other-- the oxidation happens over 35 angstroms.
So that's unprecedented.
It involves hopping radicals which no one has ever seen before.
And so that was another thing that was completely fascinating from a chemical perspective about how the system works.
The other reason that people in biology are interested in this, besides the fact that makes a building block for DNA, is that if you believe in an RNA world where we have a ribosome where a catalysis of peptide bond formation is all with the RNA, not with the protein.
How do you get from an RNA world to a DNA world?
The only enzyme that does that transformation making these building blocks are ribonucleotide reductases.
And there are many classes of ribonucleotide reductases-- one uses this tyrosyl radical-- but they all have the same active site and do the same chemistry, but they have different metal cofactors depending on where they evolved.
And the function of the metal cofactors in all cases, even though one's cobalt, one's iron sulfur cluster, one's manganese, one's iron-- the function in all cases is to generate a radical in the active site and then the chemistry is the same in all these things.
I'm always interested in listening to JoAnne's lectures, and she talks about how she got interested in biochemistry from a background in what I would call more physical organic chemistry by attending a lecture and being completely inspired.
And I am a toxicologist by training.
And how did that actually happen.
I have a similarly inspiring story, so I thought I would tell that.
I went to a lecture about 45 years ago, by a fellow by the name of [Richard] Evans Schultes.
He was the director of the Herbarium at Harvard-- and I'd been to it; it was very interesting-- and a professor there.
And he is arguably the father of the field called ethnobotany.
I went to this talk-- there were only about 10 people at the talk.
He-- because it was a small talk, he gave it sitting down, he asked permission to do that, he's a very polite man.
And it was very informal, only a few slides.
And the slides were mainly of him living with indigenous peoples, Native Americans in the Southwest, Amazonian native peoples, very interesting.
And he talked about, for example, he had just graduated, his undergraduate degree from Harvard.
His parents were eager to have him go to medical school.
He was very interested in botany and in native languages.
So he went to live with the-- I think it was the Kiowa people in the Midwest-- I think it was probably Oklahoma.
And he learned about peyote and a lot of other chemicals that were hallucinogenic.
And he realized, of course, that many of these kinds of compounds that were used in a lot of rituals, were also medicines at other concentrations, anesthetics and so on.
He then went down to South America, living with various tribes.
He'd be gone for up to a year at a time.
There are wonderful stories, well actually terrible stories, I guess, about him paddling his canoe for 40 days with malaria-- it was terrible-- to get to a hospital, things like that.
But anyway these are stories-- I was a young scientist at the time-- these were very influential to me.
And he talked about, I was down in South America and they-- and I found the plants from which the native people got their dart and arrow poisons.
And he said, out of that we isolated curare, and initiated the path toward the clinic of curare as a muscle relaxant.
He said that it didn't work so well.
I remember this lecture like it was yesterday.
He said that it was the beginning of World War II, we were cut off from the rubber plantations in the Philippines and other places.
And since South America had jungles, his form of, quote unquote "military service," involved trying to find sources in the jungle of latex that could be used to make rubber so that we could have an effective war machine.
So anyway, I went up to this fellow after he gave his talk.
And I said, look, I just got to do this kind of stuff.
This sounds really interesting.
And he said, what's your background?
I said, well, you know, I worked as a chemist.
I was a biology major but I worked as a chemist at an industrial consulting company during my undergraduate years.
And he said, oh, you got to go talk with Gerry Wogan at MIT.
He said that he isolates toxins from fungi that you find out in the jungles of Southeast Asia.
And, so anyway, that's how my career began.
As you know, I got to eventually meet Gerry Wogan and work with him.
He had already identified this toxin, aflatoxin.
But he needed somebody to figure out how it worked, and that's how I got my start.
So my interests have been in chemicals from the environment.
It could be a pollutant.
Or it could be a chemical that could be a precursor to a therapeutically useful molecule, and how do they interact with biological systems.
And that's what toxicologists and pharmacologists do.
I work in the field of genetic change.
In a perfect world, you would say, guanine would always pair with cytosine and adenine would always pair with thymine.
It turns out, however, that sometimes chemicals from the environment can react with our normal nucleotides and change their coding characteristics so that mistakes are made when polymerases try to read them.
These are mutations, and mutations cause genetic diseases.
My role in the field of toxicology is as a person, who is both a biologist and a chemist, who studies how chemical damage to our informational molecules is converted into changes in coding that results in genetic change.
I'll emphasize, of course, this is the basis for all genetic disease, but it's also the basis for evolution.
In other words, mutations happen naturally.
That means that we're not all-- we don't all look alike.
And that means there's diversity in a population, and that's because of mutations.
And evolution is a really good thing, because if we were all alike when the environment changed, then the chance of extinction might be very high.
If there's diversity in a population, that's actually a hedge that life uses in order to be able to make sure something's going to survive, because some members in the population, while they may be considered quote unquote "weaker" in the initial environment, when the environment changes-- they're the ones, for example, with hair on them, and survive the global winter that happens after the meteor strikes.
So, we are interested in chemical modification of DNA and RNA as it relates to the causation of disease, but we're also interested in the rates at which genetic change happens in a population, and how that's a good thing, and that it provides for the continuance of life.
One example of our work in evolution comes from recent work that we've been doing on HIV.
When a virus infects one of our cells, our cells respond by trying to limit the growth of the virus.
They try to kill it.
And one of the strategies that used by what's called our innate immune system is to induce a number of enzymes that start to rip apart the DNA bases.
They really take the amino groups off of cytosines and adenines in order to try to convert them into non-coding nucleotides or miscoding nucleotides.
What happens is, if you take a cytosine and you take away it's amino group, it makes it into a uracil, and then, rather than pair with a guanine, it'll pair with an adenine.
Because these are enzymes that kind of move along the viral genome, they create a huge number of mutations.
The process is called lethal mutagenesis, because what happens is, eventually the number of mutations is so large that you can no longer produce a functional protein or nucleic acid.
That's a natural strategy that we use.
And thinking about this, some years ago, given my lab's expertise in knowing about the structural modification of normal bases that makes them mutagenic, we wondered if we could contaminate the nucleotide pool of a cell with mutagenic nucleotides that would force a virus to mutate even quicker.
HIV, it turns out, almost goes extinct, but not quite.
In other words, our innate immune system, in one day, creates every single point mutation in the virus, but it also creates every single drug-resistant variant.
And it just doesn't push hard enough to be able to push the virus to extinction.
So, we wondered whether-- if we could push a little harder by using creatively designed molecules-- derivatives of cytosine, that would pretend sometimes they're a cytosine and sometimes they pretended that they were a thymine-- that we could push the virus over the top.
And we found that it did work, OK.
In other words, we were able to push the virus to a technical state of extinction using our understanding of the chemistry of the molecules that the cell has the capability to use in replication.
It may seem unwise to intentionally put mutagenic chemicals into people, and, obviously, we worked out a strategy to prevent mutations in people in the drug design process.
Specifically, it turns out there are pathways called DNA repair pathways that repair damage in our nuclear genomes, and they happen in the nucleus.
That's where the enzymes are located.
It turns out that the early stage in HIV replication involved replication in the cytoplasm.
There are no repair enzymes in the cytoplasm.
So, the virus has no defense against the mutagenic affects of these compounds.
We picked compounds that, if they were to get into our nuclei-- it turns out that our polymerases don't like them very much, anyway, but if they did get in, they're very rapidly repaired.
So, it creates what we call a therapeutic index which is very favorable in favor of killing the virus but not putting mutagenic chemicals into us.
At the end of the course, when there's so many pathways out there that things start to get really, really complicated, we try to increase the number of physiological scenarios, again, to try to get the students sufficiently interested in a pathway and thinking about how the whole pathway works, not just each individual step-- to get the student to like the example so that he or she will go right in there and study it.
And one of the areas that I think really important these days in basic metabolic biochemistry is in oncology-- cancer biology.
JoAnne mentioned that not that many years ago, academic programs wondered whether teaching metabolic pathways was really the best use of time, because so many other things were happening in the molecular biology revolution.
And certainly in 5.07 we decided to maintain emphasis in this, because we-- just because these are the first pathways that were studied doesn't mean they aren't important.
They're medically very important.
They were originally studied because of medical or economic reasons that-- fermentation, for example-- that these pathways were right in front and center.
They're always very topical.
But what happened in I'd say the last 10 years is that there's been a reawakening of the understanding of how metabolic pathways redirect themselves in order to accomplish disease goals-- for example, with a tumor.
And one of the things we teach is that a tumor is a lot like a running muscle.
And Matthew Vander Heiden, who is here in our biology department and teaches the comparable course is a real expert, a pioneer in this area.
But what we've learned is that cancer cells tend to be addicted to glucose.
They would much prefer glucose over any other fuel.
In fact, they-- when you think about what a cancer cell really is, it's a cell that's not terribly different from all our other cells.
It's acquired some small number-- we'll call driver mutations, maybe seven or so driver mutations.
There are many more passenger mutations, but maybe seven or so genetic differences that separate it from a normal cell.
But among the things that has happened to it is reprogramming of its metabolic needs.
And it has acquired the commitment to unremitting cell division.
If you think about what's needed for a cell to divide, probably the principal structural resource-- which, again, this would come from JoAnne's part of the course-- is membranes.
You've got to recreate the outside of the cell and all those organelles.
And what that means is fatty acid biosynthesis is absolutely going to be paramount.
One way that we now know that tumors work, if you understand all the pathways, is a tumor will consume glucose, it will convert that glucose to acetyl-CoA.
The acetyl-CoA takes a ride on the molecule citrate until it gets put out into the cytoplasm, and then in the cytoplasm it becomes a factory for producing-- out of acetyl-CoA, it produces the lipids that are necessary for membranogenesis.
That's absolutely critical for cancer cell development.
Now, the reason that's important is that understanding it, it tells us that, well, there are certain enzymes in the pathway.
One of them is called ATP citrate lyase.
Another one is called malic enzyme.
So, when you look at the charts, you'll see these things.
These are enzymes that are necessary for cell division, because they're necessary for fatty acid biosynthesis, and they represent good targets-- next-generation targets-- for cancer chemotherapy.
A normal cell, you can tell it not to divide, and it won't divide.
But if you can tell a cell that's committed to divide and you tell it not to divide, oftentimes that cell will kill itself.
So by blocking these critical points-- test points, we'll call them, in this metabolic chart-- you're able to selectively kill cancer cells-- we hope, anyway.
But this study of metabolic biochemistry has identified these new targets.
Another thing that's, I think, an important anecdote about cancer cells is that they need other cells in order to survive.
If you think about the problems that a tumor faces-- remember, it started from a normal cell that acquired new mutations, converted into a cancer cell, and then started to grow out into a tumor.
As it grows, it becomes more remotely placed relative to the blood supply.
OK, now, certainly there will be a response, and the blood supply will try to grow toward the tumor cells.
But the core of a tumor is hypoxic.
It doesn't have enough oxygen to do real respiration.
So, what happens is, the cancer cell tries to hard-wire the glycolysis pathway to be on.
And rough estimates are that through this-- initially, a hypoxia-inducing factor, which is a transcription factor-- you get an upregulation by 10 to 20-fold of almost all of the enzymes of the glycolytic pathway.
So, what happens is, the cancer cell becomes a specialist at glycolysis.
It does high-throughput glycolysis-- glucose to pyruvate.
But, again, we don't have the metabolic equipment to do a lot of-- we don't have the metabolic equipment to be able to do respiration if you're removed from oxygen.
So, the pyruvate gets converted into lactate.
Lactate gets pumped out into the blood, and, as I mentioned before, it acidifies the blood.
That's what happens with a working muscle.
But the tumor sends the lactate into the blood.
It's picked up by the liver.
The liver is a specialist in gluconeogenesis.
The liver then rebuilds the lactate into glucose, sends this back out into the blood.
So, the tumor is then able to re-eat the lactate that it sent out.
So a partnership emerges between the liver-- the gluconeogenic organ-- and the tumor.
So, the tumor finds that it's fed by-- unwittingly-- by the liver.
And the liver is contributing to providing the nutrition that's eventually going to kill the organism if something doesn't go in to stop it.
But, again, understanding the pathways can give you ideas with regard to how to intervene.
So, metabolic biochemistry is pretty critical to understanding this generation's cancer scientific agenda.
If you're a teacher and you're inventing a course for the first time, or revising it a lot, you sit down with your teaching partners and you put on the table all the ideas.
Teaching, you've got an ever-expanding universe of knowledge out there, and you have to cherry-pick the things that are going to be important.
It has to hang together.
One of the strategies JoAnne and I thought, when we went into the current iteration of teaching 5.07 biological chemistry, was to abandon completely, higher eukaryotes, namely us, because genome sequencing projects had sequenced so many bacteria that one could create an entire course in biochemistry that would be very meaningful, just focusing on microorganisms.
We spent a couple of days reading and thinking about it.
That would be the course JoAnne and I would teach, if it weren't for the fact that we actually have-- feel as though we have a commitment to students that are going to go on to medical school and therefore, if we avoided mammalian biochemistry, students wouldn't know anything about the mitochondria and organelles, and things like that that are associated with eukaryotes.
We wouldn't be able to make these connections to disease, the physiological scenarios.
Nevertheless, why were we so interested in bacteria?
What would be an interesting story, that I might be able to tell you, if we had taken that path.
So we have a fellow in biological engineering, named Eric Alm.
And he is an informaticist and an engineer, and an extremely good chemist.
He really knows his pathways.
When he looks at a cell, he thinks about what it is, but also where it came from, in terms of how it evolved from precursors, its family tree, so to speak.
One of the most interesting organisms that he's published on, not too long ago, is an organism called, Desulforudis.
And this would be a wonderful biochemistry course in itself.
He wondered, if he went out and dug up a cubic meter of dirt-- and outside MIT-- and if he did 16S RNA sequencing, how many living things would be there, many thousands, maybe 10,000.
Then he asked the question, what if you went down 100 meters, you know, maybe you see 1,000.
But what if you go down until there's really only one thing there.
And that's what he did, going down two miles into the ground.
And there was a single, species ecosystem called, Desulforudis.
And I remember seeing this paper, and I brought it over to JoAnne.
I was so excited because the last picture showed its metabolic network, its metabolic pathways.
It had everything.
And it makes sense.
It can't rely on other things.
For example, we can't make all of our amino acids.
We got to get them from food that we eat, or in our co-factors, some of our vitamins are made by the bacteria in our gut.
So, if all the bacteria disappear, we would too.
So we rely on other things, but Desulforudis doesn't rely on anything.
So when you look at it's biochemical networks, what you see is that it can fix nitrogen.
It can take N2 and convert it NH3, and then put that into amino acids, and it can make all of its amino acids.
It has a really good pentose phosphate pathway.
It actually uses radiation in a strange way to generate some of the energy that it needs.
It uses it to generate carbon monoxide.
Ultimately, that CO is going to form an acetyl group that will be able to generate all of the organic material inside the Desulforudis.
It's got all kinds of electron transport pathways.
So it's developed enormous versatility by being a single-species ecosystem.
So, again, this was a course where we had to make a compromise because of our clientele.
Teachers have to think about that.
We have to teach to what the people need in order to go on to the next step.
But as sort of a closing thought, I think that it would be wonderful for next-generation biochemists to really turn their attention to the microbial world, to teach this vast biochemistry and understand how bacteria effortlessly, swap biochemical pathways, pick-up entire biochemical pathways without even breaking a sweat.
Whenever they find themselves stressed, they just pick up a new pathway and they survive.
One of the physiological scenarios that we teach, and is probably very common in biochemistry courses, has to do with diabetes.
I'm diabetic, and my great grandfather was diabetic.
His daughter, my grandmother was diabetic.
Five of her six sons were diabetic, one of whom was my father.
So you see a very genetic disease here.
OK?
That's, in my case, type 2 diabetic, so adult onset.
So my cells are insulin-resistant.
So if I eat a meal that has carbohydrate or fat in it, my pancreas is probably responding.
My beta cells in my pancreas are probably OK, but the insulin just isn't able-- but producing insulin-- but the insulin just isn't able to signal my cells to be able to take up the glucose.
As a consequence, I'm somewhat in a technical state of starvation.
My glycogen reserves are therefore smaller than yours, and I can tell this in times.
For example, when I first became diabetic, I loved mountain climbing and hiking and so on, and I noticed that I would go out.
And I'd be totally exhausted for the first 10 minutes or so, and that's because I have very small glycogen reserves.
What I had to do is this metabolic switch over to booting up oxidative metabolism of fats.
That all had to be turned on, and it's hard.
It's a hard transition, but once it got going, as long as I stay aerobic, in other words, I don't go so fast that I go anaerobic, I can climb forever.
Because if I stay aerobic, I've got plenty of fat reserves.
In fact, it's probably a good thing that I'm burning them.
So I'm just a little bit different.
I like to teach this stuff in 5.07, in part because well there's probably not that many kids in the class, of 100, 130, or so, students would be diabetic, but it's about 6% to 8% of the population in the United States right now.
So everybody knows someone who is diabetic, or maybe your parents are, and what are the risk factors?
So we teach that.
There are many different treatments for diabetes.
I'll just use myself as a case study.
One of the things I did to myself is I measure my blood sugar at different times during the day, and my blood sugar this morning was 116.
And my blood sugar at about five o'clock this afternoon will probably be in the high 80s or 90.
My blood sugar after I have a meal might go up to 160, 170, and it shouldn't go that high.
And some mornings, like a couple of days ago, my blood sugar was 146, which is too high.
And that reflects two things, what I ate the night before, but the second thing is the one that's my problem, which is gluconeogenesis.
Gluconeogenesis technically means new synthesis of glucose from non-carbohydrate precursors.
For example, I could take the amino acid aspartate, and I could find a path by which I could convert that into glucose.
I could take lactate, and I could convert it into glucose.
OK, so during the day, I use my glucose, and I stay active, and I walk around.
I ride my bike and so on.
So my glucose level, it's pretty reasonable by the time I go to bed.
But then at night, when I stop moving around a lot, this gluconeogenesis process continues in me, and that's what causes my blood sugar to go up.
The medicine I take is called Metformin.
It has a number of targets, but one of them is one of the enzymes, called PEPCK, Pyruvate Carboxykinase, that's in the gluconeogenic pathway.
Let me say a word about gluconeogenesis, another word actually.
So we all have dinner, like say six to nine o'clock at night.
We go to bed, and there are certain organs in the body, the brain, our red blood cells, that require glucose.
They can't work with anything else.
So gluconeogenesis, by principally the liver, provides a constant stream of glucose to these organs that absolutely require it, like our brain.
Now, when we go to sleep at night, as time gets longer and longer and longer, after the meal, our glucose level, our natural glucose level, is going to start to fall off.
And the liver then compensates by increasing the amount of gluconeogenesis in order to keep our blood glucose at about 100 while we're not eating.
What I do, during the night, is to take this drug that will prevent the switch to produce more and more sugar by gluconeogenesis.
As I mentioned, I'm diabetic.
And when things that I noticed in my dad, before he knew he was diabetic-- he'd come home from the train at night, and my sister and I would rub his legs, because they hurt.
This is one of the problems with diabetics.
His feet hurt.
And I remember my sister Nancy saying once, gee.
Daddy smells like Mommy's nail polish remover.
Which was acetone.
And acetone is one of the three ketone bodies, the other two being Beta-Hydroxybutyric acid and acetoacetate.
So what happens in diabetes is, the diabetic doesn't have ample carbohydrate reserves within their cells as glycogen.
So they're doing a lot of fatty acid oxidation, rather than carbohydrate metabolism.
If you take a fatty acid, and you break it down to Acetyl-CoA, the last step is called Beta-ketothiolase-- backs up, then your two carbons of Acetyl-CoA will be four carbons of acetoacetate.
And then acetoacetate-- a third Acetyl-CoA can add to it to form what's called hydroxy-methylglutaryl-coenzyme A-- HMG CoA.
And then rearrangements can happen that give rise, ultimately, to the three ketone bodies-- acetoacetate, Beta-Hydroxybutyrate, and acetone.
So in situations where-- of starvation-- and diabetes is basically, my cells were in a technical state of starvation.
In a case of starvation, what happens is you get this backup of the lipid breakdown, or catabolism pathway.
And then these ketone bodies flush out into your blood.
In starvation situations, the ketone bodies will act as fuels for your brain and other organs.
It's a difficult transition.
But ketone bodies are-- in the case of diabetics, they're produced to excess, to such extent that they're actually quite harmful to you.
The reason a person smells of acetone is the intermediate ketone body-- the acetoacetate basically decarboxylates.
It's a Beta-keto acid, and so it's a spontaneous decomposition to form acetone.
Acetone is not, to my knowledge, biochemically useful as a kind of an energy source.
But it's a sentinel, or it's something that is smelled, so that it can be diagnostic of the disease.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Thermodynamics, all right, let's start.
Thermodynamics is the science of the flow of heat.
So, thermo is heat, and dynamics is the motion of heat.
Thermodynamics was developed largely beginning in the 1800's, at the time of the Industrial Revolution.
So, taming of steel.
The beginning of generating power by burning fossil fuels.
The beginning of the problems with CO2 and [NOISE OBSCURES] global warming.
In fact, it's interesting to note that the first calculation on the impact of CO2 on climate was done in the late 1800's by Arrhenius.
Beginning of a generation of power moving heat from fossil fuels to generating energy, locomotives, etcetera.
So, he calculated what would happen to this burning of fossil fuels, and he decided in his calculation, he basically got the calculation right, by the way, but he came out that in 2,000 years from the time that he did the calculations, humans would be in trouble.
Well, since his calculation, we've had an exponential growth in the amount of CO2, and if you go through the calculations of -- people have done these calculations throughout times since Arrhenius, the time that we're in trouble, 2,000 years and the calculation, has gone like this, and so now we're really in trouble.
That's for a different lecture.
So, anyway, thermodynamics dates from the same period as getting fossil fuels out of the ground.
It's universal.
It turns out everything around us moves energy around in one way or the other.
If you're a biological system, you're burning calories, burning ATP.
You're creating heat.
If you're a warm-blooded animal.
You need energy to move your arms around and move around -- mechanical systems, obviously, cars, boats, etcetera.
And even in astrophysics, when you talk about stars, black holes, etcetera, you're moving energy around.
You're moving heat around when you're changing matter through thermodynamics.
And the cause of some thermodynamics have even been applied to economics, systems out of equilibrium, like big companies like Enron, you know, completely out of equilibrium, crash and burn.
You can apply non-equilibrium thermodynamics to economics.
It was developed before people knew about atoms and molecules.
So it's a science that's based on macroscopic properties of matter.
Since then, since we know about atoms and molecules now, we can rationalize the concepts of thermodynmamics using microscopic properties, and if you are going to take 5.62, that's what you'd learn about.
You'd learn about statistical mechanics, and how the atomistic concepts rationalize thermodynamics.
It doesn't prove it, but it helps to getting more intuition about the consequences of thermodynamics.
So it applies to macroscopic systems that are in equilibrium, and how to go from one equilibrium state to another equilibrium state, and it's entirely empirical in its foundation.
People have done experiments through the ages, and they've accumulated the knowledge from these experiments, and they've synthesized these experiments into a few basic empirical rules, empirical laws, which are the laws of thermodynamics.
And then they've taken these laws and added a structure of math upon it, to build this edifice, which is a very solid edifice of thermodynamics as a science of equilibrium systems.
So these empirical observations then are summarized into four laws.
So, these laws are, they're really depillars.
They're not proven, but they're not wrong.
They're very unlikely to be wrong.
Let's just go through these laws, OK, very quickly.
There's a zeroth law The zeroth law every one of these laws basically defines the quantity in thermodynamics and then defines the concept.
The zeroth law defines temperature.
That's a fairly common-sense idea, but it's important to define it, and I call that the common-sense law.
So this is the common-sense law.
The first law ends up defining energy, which we're going to call u, and the concept of energy conservation, energy can't be lost or gained.
And I'm going to call this the you can break even law; you can break even law.
You don't lose energy, you can't gain energy.
You break even.
The second law is going to define entropy, and is going to tell us about the direction of time, something that conceptually we, clearly, understand, but is going to put a mathematical foundation on which way does time go.
Clearly, if I take a chalk like this one here, and I throw it on the ground, and it breaks in little pieces, if I run the movie backwards, that doesn't make sense, right?
We have a concept of time going forward in a particular way.
How does entropy play into that concept of time?
And I'm going to call this the you can break even at zero degrees Kelvin law.
You can only do it at zero degrees Kelvin.
The third law is going to give a numerical value to the entropy, and the third law is going to be the depressing one, and it's going to say, you can't get to zero degrees.
These laws are universally valid.
They cannot be circumvented.
Certainly people have tried to do that, and every year there's a newspaper story, Wall Street Journal, or New York Times about somebody that has invented the device that somehow goes around the second law and makes more energy than it creates, and this is going to be -- well, first of all, for the investors this is going to make them very, very rich, and for the rest of us, it's going to be wonderful.
And they go through these arguments, and they find venture money to fund the company, and they get very famous people to endorse them, etcetera.
But you guys know, because you have MIT degrees, and you've, later, and you've taken 5.60, that can't be the case, and you're not going to get fooled into investing money into these companies.
But it's amazing, that every year you find somebody coming up with a way of going around the second law and somehow convincing people who are very smart that this will work.
So, thermo is also a big tease, as you can see from my descriptions of these laws here.
It makes you believe, initially, in the feasibility of perfect efficiency.
The first law is very upbeat.
It talks about the conservation of energy.
Energy is conserved in all of its forms.
You can take heat energy and convert it to work energy and vice versa, and it doesn't say anything about that you have to waste heat if you're going to transform heat into work.
It just says it's energy.
It's all the same thing, right?
So, you could break even if you were very clever about it, and that's pretty neat.
So, in a sense, it says, you know, if you wanted to build a boat that took energy out of the warmth of the air, to sail around the world, you can do that.
And then the second law comes in and says well, that's not quite right.
The second law says, yes, energy is pretty much the same in all this form, but if you want to convert one form of energy into another, if you want to convert work, heat into work, with 100% efficiency, you've got to go down to zero degrees Kelvin, to absolute zero if you want to do that.
Otherwise you're going to waste some of that heat somewhere along the way, some of that energy.
All right, so you can't get perfect efficiency, but at least if you were able to go to zero degrees Kelvin, then you'd be all set.
You just got to find a good refrigerator on your boat, and then you can still go around the world.
And then the third law comes in, and that's the depressing part here.
It says, well, it's true.
If you could get to zero degrees Kelvin, you'd get perfect efficiency, but you can't get to zero degrees Kelvin, you can't.
Even if you have an infinite amount of resources, you can't get there.
Any questions so far?
So thermodynamics, based on these four laws now, requires an edifice, and it's a very mature science, and it requires that we define things carefully.
So we're going to spend a little bit of time making sure we define our concepts and our words, and what you'll find that when you do problem sets, especially at the beginning, understanding the words and the conditions of the problem sets is most of the way into solving the problem.
So we're going to talk about things like systems.
The system, it's that part of the universe that we're studying.
These are going to be fairly common-sense definitions, but they're important, and when you get to a problem set, really nailing down what the system is, not more, nor less, in terms of the amount of stuff, that's part of the system, it's going to be often very crucial.
So you've got the system.
For instance, it could be a person.
I am the system.
I could be a system.
It could be a hot coffee in a thermos.
So the coffee and the milk and whatever else you like in your coffee would be the system.
It could be a glass of water with ice in it.
That's a fine system.
Volume of air in a part of a room.
Take four liters on this corner of the room.
That's my system.
Then, after you define what your system is, whatever is left over of the universe is the surroundings.
So, if I'm the system, then everything else is the surroundings.
You are my surroundings.
Saturn is my surroundings.
As far as you can go in the universe, that's part of the surroundings.
And then between the system and the surroundings is the boundary.
And the boundary is a surface that's real, like the outsides of my skin, or the inner wall of the thermos that has the coffee in it, or it could be an imaginary boundary.
For instance, I can imagine that there is a boundary that surrounds the four liters of air that's sitting in the corner there.
It doesn't have to be a real container to contain it.
It's just an imaginary boundary there.
And where you place that boundary becomes important.
So, for instance, for the thermos with the coffee in it, if you place the boundary in the inside wall of the glass or the outside wall of the glass and the inside of the thermos, that makes a difference; different heat capacity, etcetera.
So this becomes where defining the system and the boundaries, and everything becomes important.
You've got to place the boundary at exactly the right place, otherwise you've got a bit too much in your system or a bit too little.
More definitions.
The system can be an open system, or it can be a closed system, or it can be isolated.
The definitions are also important here.
An open system, as the name describes, allows mass and energy to freely flow through the boundary.
Mass and energy flow through boundary.
Mass and energy -- I'm an open system, right?
Water vapor goes through my skin.
I'm hot, compared to the air of the room, or cold if I'm somewhere that's warm.
So energy can go back and forth.
The thermos, with the lid on top, is not an open system.
Hopefully, your coffee is going to stay warm or hot in the thermos.
It's not going to get out.
So the thermos is not an open system.
In fact, the thermos is an isolated system.
The isolated system is the opposite of the open system, no mass and no energy can flow through the boundary.
The closed system allows energy to transfer through the boundary but not mass.
So a closed system would be, for instance, a glass of ice water with an ice cube in it, with the lid on top.
The glass is not very insulating.
Energy can flow across the glass, but I put a lid on top, and so the water can't get out.
And that's the closed system.
Energy goes through the boundaries but nothing else.
Important definitions, even though they may sound really kind of dumb, but they are really important, because when you get the problem, figuring out whether you have an open, closed, or isolated system, what are the surroundings?
What's the boundary?
What is the system?
That's the first thing to make sure that is clear.
If it's not clear, the problem is going to be impossible to solve.
And that's also how people find ways to break the second law, because somehow they've messed up on what their system is.
And they've included too much or too little in the system, and it looks to them that the second law is broken and they've created more energy than is being brought in.
That's usually the case.
Questions?
Let's keep going.
So, now that we've got a system, we've got to describe it.
So, let's describe the system now.
It turns out that when you're talking about macroscopic properties of matter, you don't need very many variables to describe the system completely thermodynamically.
You just need a few macroscopic variables that are very familiar to you, like the pressure, the temperature, the volume, the number of moles of each component, the mass of the system.
You've got a magnetic field, maybe even magnetic susceptibility, the electric field.
We're not going to worry about these magnetic fields or electric fields in this class.
So, pretty much we're going to focus on this set of variables here.
You're going to have to know when you describe the system, if your system is homogeneous, like your coffee with milk in it, or heterogeneous, like water with an ice cube in it.
So heterogeneous means that you've got different phases in your system.
I'm the heterogeneous system, soft stuff, hard stuff, liquid stuff.
Coffee is homogeneous, even though it's made up of many components.
Many different kinds of molecules make up your coffee.
There are the water molecules, the flavor molecules, the milk proteins, etcetera.
But it's all mixed up together in a homogeneous, macroscopic fashion.
If you drill down at the level of molecules you see that it's not homogeneous.
But thermodynamics takes a bird's eye view.
It looks pretty, beautiful.
So, that's a homogeneous system, one phase.
You have to know if your system is an equilibrium system or not.
If it's an equilibrium system, then thermodynamics can describe it.
If it's not, then you're going to have trouble describing it using thermodynamic properties.
Thermodynamics talks about equilibrium systems and how to go from one state of equilibrium to another state of equilibrium.
What does equilibrium mean?
It means that the properties of the system, the properties that describe the system, don't change in time or in space.
If I've got a gas in a container, the pressure of the gas has to be the same everywhere in the container, otherwise it's not equilibrium.
If I place my container of gas on the table here, and I come back an hour later, the pressure needs to be the same when I come back.
Otherwise it's not equilibrium.
So it only talks about equilibrium systems.
What else do you need to know?
So, you need to know the variables.
You need to know it's heterogeneous or homogeneous.
You need to know if it's an equilibrium, and you also need to know how many components you have in your system.
So, a glass of ice water with an ice cube in it, which is a heterogeneous system, has only one component, which is water, H2O.
Two phases, but one component.
Latte, which is a homogeneous system, has a very, very large number of components to it.
All the components that make up the milk.
All the components that make up the coffee, and all the impurities, etcetera.
cadmium, heavy metals, arsenic, whatever is in your coffee.
OK, any questions?
All right, so we've described the system with these properties.
Now these properties come in two flavors.
You have extensive properties and intensive properties.
The extensive properties are the ones that scale with the size of the system.
If you double the system, they double in there numerical number.
For instance, the volume.
If you double the volume, the v doubles.
I mean that's obvious.
The mass, if you double the amount of stuff the mass will double.
Intensive properties don't care about the scale of your system.
If you double everything in the system, the temperature is not going to change, it's not going to double.
The temperature stays the same.
So the temperature is intensive, and you can make intensive properties out of the extensive properties by dividing by the number of moles in the system.
So I can make a quantity that I'll call V bar, which is the molar volume, the volume of one mole of a component in my system, and that becomes an intensive quantity.
A volume which is an intensive volume.
The volumes per mole of that stuff.
So, as I mentioned, thermodynamics is the science of equilibrium systems, and it also describes the evolution of one equilibrium to another equilibrium.
How do you go from one to the other?
And so the set of properties that describes the system -- the equilibrium doesn't change.
So, these on-changing properties that describe the state of the equilibrium state of the system are called state variables.
So the state variables describe the equilibrium's state, and they don't care about how this state got to where it is.
They don't care about the history of the state.
They just know that's if you have water at zero degrees Celsius with it ice in, that you can define it as a heterogeneous system with a certain density for the water or certain density for the ice, etcetera, etcetera.
It doesn't care how you got there.
We're going to find other properties that do care about the history of the system, like work, that you put in the system, or heat that you put in the system, or some other variables.
But you can't use those to define the equilibrium state.
You can only use the state variables, independent of history.
And it turns out that for a one component system, one component meaning one kind of molecule in the system, all that you need to know to describe the system is the number of moles for a one component system, and to describe one phase in that system, one component, homogeneous system, you need n and two variables.
For instance, the pressure and the temperature, or the volume and the pressure.
If you have the number of moles and two intensive variables, then you know everything there is to know about the system.
About the equilibrium state of that system.
There are hundreds of quantities that you can calculate and measure that are interesting and important properties, and all you need is just a few variables to get everything out, and that's really the power of thermodynamics, is that it takes so little information to get so much information out.
So little data to get a lot of predictive information out.
As we're going on with our definitions, we can summarize a lot of these definitions into a notation, a chemical notation that that will be very important.
So, for instance, if I'm talking about three moles of hydrogen, at one bar 100 degrees Celsius.
I'm not going to write, given three moles of hydrogen at one bar and three degrees, blah, blah, blah.
I'm going to write it in a compact notation.
I'm going to write it like this: three moles of hydrogen which is a gas, one bar 100 degrees Celsius.
This notation gives you everything you need to know about the system.
It tells you the number of moles.
It tells you the phase.
It tells you what kind of molecule it is, and gives you two variables that are state variables.
You could have the volume and the temperature.
You could have the volume and the pressure.
But this tells you everything.
I don't need to write it down in words.
And then if I want to tell you about a change of state, or let's first start with a mixture.
Suppose that I give to a mixture like, this is a homogeneous system with two components, like five moles of H2O, which is a liquid, at one bar 25 degrees Celsius, plus five moles of CH3, CH2, OH, which is a liquid, and one bar at 25 degrees Celsius.
This describes roughly something that is fairly commonplace, it's 100-proof vodka 1/2 water, 1/2 ethanol -- that describes that macroscopic system.
You're missing all the impurities, all the little the flavor molecules that go into it, but basically, that's the homogeneous system we were describing, two component homogeneous systems.
Then you can do all sorts of predictive stuff with that system.
All right, that's the equilibrium system.
Now we want to show a notation, how do we go from one equilibrium state like this describes to another equilibrium state?
So, we take our two equilibrium states, and you just put an equal sign between them, and the equal sign means go from one to the other.
So, if we took our three moles of hydrogen, which is a gas at five bar and 100 degrees Celsius, and, which is a nice equilibrium state here, and we say now we're going to change the equilibrium state to something new, we're going to do an expansion, let's say.
We're going to drop the pressure, the volume is going to go up.
I don't need to tell you the volume here, because you've got enough information to calculate the volume.
The number of moles stays the same, a closed systems, gas doesn't come out.
Stays a gas, but now the pressure is less, the temperature is less.
I've done some sort of expansion on this.
I've gone from 1 equilibrium state to another equilibrium state, and the equal sign means you go from this state to that state.
It's not a chemical reaction.
That's why we don't have an arrow here, because we could go back, this way too.
We can go back and forth between these two equilibrium states.
They're connected.
This means they're connected.
And when I put this, I have to tell you how they are connected.
I have to tell you the path, if you're going to solve a problem.
For instance, you want to know how much energy you're going to get out from doing this expansion.
How much energy are you going to get out, and how far are you going to be able to drive a car with this expansion, let's say, so that's the problem.
So, I need to tell you how you're doing the expansion, because that's going to tell you how much energy you're wasting during that expansion.
It goes back to the second law.
Nothing is efficient.
You're always wasting energy into heat somewhere when you do a change that involves a mechanical change.
All right, so I need to tell you the path, when I go from one state to the other.
And the path is going to be the sequence, intermediate states going from the initial state the final state.
So, for instance, if I draw a graph of pressure on one axis and temperature on the other axis, my initial state is at a temperature of 100 degrees Celsius and five bar.
My final stage is 50 degrees Celsius and one bar.
So, I could have two steps in my path.
I could decide first of all to keep the pressure constant and lower the pressure.
When I get to 50 degrees Celsius, I could choose to keep the temperature constant and lower the pressure.
I'm sorry, my first step would be to keep the pressure constant and lower the temperature, then I lower the pressure, keeping the temperature constant.
So there's my intermediate state there.
This is one of many paths.
There's an infinite number of paths you could take.
You could take a continuous path, where you have an infinite number of equilibrium points in between the two, a smooth path, where you drop the pressure and the temperature simultaneously in little increments.
All right, so when you do a problem, the path is going to turn out to be extremely important.
How do you get from the initial state to the final state?
Define the initial state.
Define the final state.
Define the path.
Get all of these really clear, and you've basically solved the problem.
You've got to spend the time to make sure that everything is well defined before you start trying to work out these problem.
More about the path.
There are a couple ways you could go through that path.
If I look at this smooth path here.
I could have that path be very slow and steady, so that at every point along the way, my gas is an equilibrium.
So I've got, this piston here is compressed, and I slowly, slowly increase the volume, drop the temperature.
Then I can go back, the gas is included at every point of the way.
That's a reversible path.
That can reverse the process.
I expand it, and reverse it, no problem.
So, I could have a reversible path, or I take my gas, and instead of slowly, slowly raising it, dropping the pressure, I go from five bar to one bar extremely fast.
What happens to my gas inside?
Well, my gas inside is going to be very unhappy.
It's not going stay in equilibrium.
Parts of the system are going to be at five bar.
Parts of it at one bar.
Parts of it may be even at zero bar, if I go really fast.
I'm going to create a vacuum.
So the system will not be described by a single state variable during the path.
If I look at different points in my container during that path, I'm going to have to use a different value of pressure or different value of temperature at different points of the container.
That's not an equilibrium state, and that process turns out then to be in irreversible process.
Do it very quickly.
Now to reverse it and get back to the initial point is going to require some input from outside, like heat or extra work or extra heat or something, because you've done an irreversible process.
You've wasted a lot of energy in doing that process.
I have to tell you whether the path is reversible or irreversible, and the irreversible path also defines the direction of time.
You can only have an irreversible path go one way in time, not the other way.
Chalk breaks irreversibly and you can't put it back together so easily.
You've got to pretty much take that chalk, and make a slurry out of it, put water, and dry it back up, put in a mold, and then you can have the chalk again, but you can't just glue it back together.
That would not be the same state as what you started out with.
And then there are a bunch of words that describe these paths.
Words like adiabatic, which we'll be very familiar with.
Adiabatic means that there's no heat transferred between the system and the surrounding.
The boundary is impervious to transfer of heat, like a thermos.
Anything that happens inside of the thermos is an adiabatic change because the thermos has no connection in terms of energy to the outside world.
There's no heat that can go through the walls of the thermos.
Whereas, like isobaric means constant pressure.
So, this path right here from this top red path is an isobaric process.
Constant temperature means isothermal, so this part means an isothermal process.
So then, going from the initial to final states with a red path, you start with an isobaric process and then you end with an isothermal process.
And these are words that are very meaningful when you read the text of a problem or of a process.
Any questions before we got to the zeroth law?
We're pretty much done with our definitions here.
Yes
Was adiabatic reversible?
Adiabatic can be either reversible or not, and we're going to do that probably next time or two times.
Any other questions?
Yes
Is there a boundary between reversible and irreversible?
A boundary between reversible and irreversible?
Like something is almost reversible and almost irreversible.
No, pretty much things are either reversible or irreversible.
Now, in practice, it depends on how good your measurement is.
And probably also in practice, nothing is truly reversible.
So, it depends on your error bar in a sense.
It depends on what what you define, exactly what you define in your system.
It becomes a gray area, but it should be pretty clear if you can treat something is reversible are irreversible.
Other questions, It's a good question.
So the zeroth law we're going to go through the laws now.
The zeroth law talks about defining temperature and it's the common-sense law.
You all know how.
When something hot, it's got a higher temperature than when something is cold.
But it's important to define that, and define something that's a thermometer.
So what do you know?
What's the empirical information that everybody knows?
Everybody knows that if you take something which is hot and something which is cold, and you bring them together, make them touch, that heat is going to flow from the hot to the cold, and make them touch, and heat flows from hot to cold.
That's common sense.
This is part of your DNA, And then their final product is an object, a b which ends up at a temperature or a warmness which is in between the hot and the cold.
So, this turns out to be warm.
You get your new equilibrium state, which is in between what this was, and what a and b were.
Then how do you know that it's changed temperature, or that heat has flowed from a to b?
Practically speaking, you need some sort of property that's changing as heat is flowing.
For instance, if a were metallic, you could measure the connectivity of a or resistivity, and as heat flows out of a into b, the resistivity of a would change.
Or you could have something that's color metric that changes color when it's colder, so you could see the heat flowing as a changes color or b changes color as heat flows into b.
So, you need some sort of property, something you can see, something you can measure, that tells you that heat has flowed.
Now, if you have three objects, if you have a, b, and c, and you bring them together, and a is the hottest, b is the medium one, and c is the coldest, so from hottest to coldest a, b, c, -- if you bring them together and make them touch, you know, intuitively, that heat will not flow like this.
You know that's not going to happen.
You know that what will happen is that heat will flow from a to b from b to c and from a to c. That's common-sense.
You know that.
And the other way in the circle will never happen.
That would that would give rise to a perpetual motion machine, breaking of the second law.
It can't happen.
But that's an empirical observation, that heat flows in this direction.
And that's the zeroth law thermodynamic.
It's pretty simple.
The zeroth law says that if a and b -- it doesn't exactly say that, but it implies this.
It says that if a and b are in thermal equilibrium, if these two are in thermal equilibrium, meaning that there's no heat flows between them, so that's the definition of thermal equilibrium, that no heat flows between them, and these two are in thermal equilibrium, and these two are in thermal equilibrium, then a and c will be also be in thermal equilibrium.
But if there's no heat flowing between these two, and no heat flowing between these two, then you can't have heat flowing between these two.
So if I get rid of these arrows, there's no heat flowing because they're in thermal equilibrium, then I can't have an arrow here.
That's what the zeroth law says.
They're all the same temperature.
That's what it says.
If two object are in the same temperature, and two other object are in the same temperature, then all three must have the same temperature.
It sounds pretty silly, but it's really important because it allows you to define a thermometer and temperature.
Because now you can say, all right, well, now b can be my thermometer.
I have two objects, I have an object which is in Madagascar and an object which is in Boston, and I want to know, are they the same temperature?
So I come out with a third object, b, I go to Madagascar, and put b in contact with a.
Then I insulate everything, you know, take it away and see if there's any heat flow.
Let's say there's no heat flow.
Then I insulate it, get back on the plane to Boston, and go back and touch b with c. If there's no heat flow between the b and c, then I can say all right, a and c were the same temperature.
B is my thermometer that tells me that a and c are in the same temperature.
And there's a certain property associated with heat flow with b, and it didn't change.
And that property could be color.
It could be resistivity.
It could be a lot of different things.
It could be volume.
And the temperature then is associated with that property.
And if it had changed, then the temperature between those two would have changed in a very particular way.
So, zeroth law, then, allows you to define the concept of temperature and the measurement of temperature through a thermometer.
Let's very briefly go through stuff that you've learned before.
So, now you have this object which is going to tell you whether other things are in thermal equilibrium now.
What do you need for that object?
You need that object to be a substance, to be something.
So, the active part of the thermometer could be water.
It could be alcohol, mercury, it could be a piece of metal.
You need a substance, and then that substance has to have a property that changes depending on the heat flow, i.e., depending on whether it's sensing that it's the same temperature or different temperature than something else.
And that property could be the volume, like if you have a mercury thermometer, the volume of the mercury.
It could be temperature.
It could be resistivity, if you have a thermocouple.
It could be the pressure.
All right, so now you have an object.
You've got a property that changes, depending on the heat flow.
It's going to tell you about the temperature.
Now you need to define the temperature scales.
So, you need some reference points to be able to tell you, OK, this temperature is 550 degrees Smith, whatever.
So, you assign values to very specific states of matter and call those the reference points for your temperature.
For instance, freezing of water or boiling of water, the standard ones.
And then an interpolation scheme.
You need a functional form that connects the value at one state of matter, the freezing point of water, to another phase change, the boiling point of water.
You can choose a linear interpolation or quadratic, but you've got to choose it.
And it turns out not to be so easy.
And if you go back into the 1800's when thermodynamics was starting, there were a zillion different temperatures scales.
Everybody had their own favorite temperature scales.
The one that we're most familiar with is the centigrade or Celsius scale where mercury was the substance, and the volume of mercury is the property.
The reference points are water, freezing or boiling, and the interpolation is linear, and then that morphed into the Kelvin scale, as we're going to see later.
The Fahrenheit scale is an interesting scale.
It turns out the U.S. and Jamaica are the only two places on Earth now that use the Fahrenheit scale.
Mr. Fahrenheit, Daniel Gabriel Fahrenheit was a German instrument maker.
The way he came up with his scale was actually he borrowed the Romer scale, which came beforehand.
The Romer scale was, Romer was a Dane, and he defined freezing of water at 7.5 degrees Roemer, and 22.5 degrees Romer as blood-warm.
That was his definition.
Two substances, blood and water.
Two reference points, freezing and blood-warm, you know, the human body.
A linear interpolation between the two, and then some numbers associated with them, 7-1/2 and 22-1/2.
Why does he choose 7-1/2 as the freezing point of water?
Because he thought that would be big enough that in Denmark, the temperature wouldn't go below zero.
That's how he picked 7-1/2.
Why not?
He didn't want to use negative numbers to measure temperature in Denmark outside.
Well, Fahrenheit came along and thought, well, you know, 7-1/2, that's kind of silly; 22-1/2 that's, kind of silly.
So let's multiply everything by four.
I think it becomes 30 degrees for the freezing of water and 22.5 x 4, which I don't know what it is, 100 or something -- no, it's 90 I think.
And then for some reason, that nobody understands, he decided to multiply again by 16/15, and that's how we get 32 for freezing of water and 96 in his words for the temperature in the mouth or underneath the armpit of a living man in good health.
What a great temperature scale.
It turns out that 96 wasn't quite right.
Then he interpolated and found out water boils at 212.
But, you know, his experiment wasn't so great, and, you know, maybe had a fever when he did the reference point with 96, whatever.
It turns out that it's not 96 to be in good health, it's 98.6 -- whatever.
That's how we got to the Fahrenheit scale.
All right, next time we're going to talk about a much better scale, which is the ideal gas thermometer and how we get to the Kelvin scale.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time we talked about the zeroth law, which is the common-sense law, which says that if you take a hot object next to a cold object, heat will flow from the hot to the cold in a way that is well defined, and it allows you to define temperature.
It allows you to define the concept of a thermometer.
You have three objects, one of them could be a thermometer.
You have two of them separated at a distance.
You take the third one, and you go from one to the other, and you see whether heat flows, when you touch one object, the middle object, between those two objects.
Let me talk to you about temperature scales.
We talked about the Celsius scale then the Fahrenheit scale.
The late 1800's were a booming time for temperature scales.
People didn't really realize how important it was to properly define the reference points: Fahrenheit's warm-blooded or 96 degrees, and Romer's 7.5 degrees.
Romer because he didn't want to go below zero degrees measuring temperature outside in Denmark Those are kind of silly.
But they're the legacy that we have today, and that's what we use.
In science, we use somewhat better temperature scales.
And the temperature scale that turns out to be well-defined and ends up giving us the concept of an absolute zero is the ideal gas thermometer.
So, let's talk about that briefly today first.
The ideal gas thermometer.
It's based on Boyle's law.
Boyle's law was an empirical law that Mr. Boyle discovered by doing lots of experiments, and Boyle's law says that the limit of the quantity pressure times the molar volume, so this quantity here, pressure times the molar volume, as you let pressure go to zero.
So, you do this measurement, you measure with the gas, you measure the pressure and the molar volume.
Then you change the pressure again, and you measure the pressure in the volume, and you multiply these two together, and you keep doing this experiment, getting the pressure smaller and smaller, you find that this limit turns out to be a constant, independent of the gas.
It doesn't care where the gas is.
You always get to the same constant.
And that constant turns out to be a function of the temperature.
The only function it is -- it doesn't care where the gas is.
It only cares where the temperature is.
All right, so now we have the makings of a good thermometer and a good temperature scale.
We have a substance.
The substance could be any gas.
That's pretty straightforward.
So now we have a substance, which is a gas, with a property.
So now the volume of mercury, or the color of something which changes with temperature, or the resistivity.
In this case here, our property is the value of the pressure times the volume, times the molar volume.
That's the property.
The property is the limit as p goes to zero of pressure times molar volume.
It's a number.
Measure it.
It's a number.
It's going to come out.
That's the property that's going to give us the change in temperature.
Then we need some reference points.
And Celsius first used the boiling point of water, and called that 100 degrees Celsius, and the freezing point of water and called that zero degrees Celsius.
And then we need an interpolation scale.
How to go from one reference point to the other with this property.
This property, which we're going to call f(t).
There are many ways you can connect those two dots.
If I draw a graph, and on one axis I have this temperature.
The idea of temperature with two reference points, zero for the freezing point of water, 100 degrees for the boiling point of water.
And on the y-axis I've got the property f(t).
It has some value corresponding to t equals zero.
So let's get some value right here.
There's another value connected to this property here, when t is equal to 100, a reference point here.
Now there many ways I can connect these two points together.
The simplest way is to draw a straight line.
It's called the linear interpolation.
My line is not so straight, right here.
You could do a different kind of line.
You could do a quadratic, let's say.
Something like this.
That would be perfectly fine interpolation.
All right, we choose to have a linear interpolation.
That's a choice, and that choice turns out to be very interesting and really important, because if you connect these two points together, you get a straight line that has to intercept the x-axis at some point.
Now what does it mean to intercept the x-axis here?
It means that the value of f(t) for this temperature is zero.
That means that at this point right here, f(t)=0.
That means the pressure times the volume equals zero, for that gas.
And if you're below this temperature here, this quantity, p times v it would be negative.
Is that possible?
Can we have p v negative?
Yes?
No, it can't be.
Negative pressure doesn't make any sense, right?
Negative volume doesn't make any sense.
That means that this part here, can't happen.
That means that this temperature right here is the absolute lowest temperature you can go to that physically makes any sense.
That's the absolute zero.
So the concept of an absolute zero, a temperature below which you just can't go, that's directly out of the scheme here, this linear interpolation scheme with these two reference points.
If I had taken as my interpolation scheme, my white curve here, I could go to infinity and have the equivalent of absolute zero being at infinity, minus infinity.
So, this temperature, this absolute zero here, which is absolute zero on the Kelvin scale.
The lowest possible temperature in the Celsius scale is minus 273.15 degrees Celsius.
So that begs the notion of re-referencing our reference point, of changing our reference points.
To change a reference point from this point here being zero, instead of this point here being zero.
And so redefining then the temperature scale to the Kelvin scale, where t in degrees Kelvin is equal to t in degree Celsius, plus 273.15.
And then you would get the Kelvin scale.
All right, it turned out that this thermometer here wasn't quite perfect either.
Just like Fahrenheit measuring 96 degrees being a warm-blooded, healthy man, right, that's not very accurate.
Our temperature probably fluctuates during the day a little bit anyways, it's not very accurate.
And similarly, the boiling point, defining that at a 100 degrees Celsius, well that depends on the pressure.
It depends whether you're in Denver or you're in Boston.
Water boils at different temperatures, depending on what the atmospheric pressure is; same thing for the freezing point.
So that means, then, you've got to define the pressure pretty well.
You've got to know where the pressure is.
It would be much better if you had a reference point that didn't care where the pressure was.
Just like our substance doesn't care where the gas is.
It's kind of universal.
And so now, instead of using these reference points for the Kelvin scale, we use the absolute zero, which isn't going to care what the pressure is.
It's the lowest number you can go to.
And our other reference point is the triple point of water -- reference points become zero Kelvin, absolute zero, and the triple point.
The triple point of water is going to be defined as 273.16 degrees Kelvin.
And the triple point of water is that temperature and pressure -- there's a unique temperature and pressure where water exists in equilibrium between the liquid phase, the vapor phase, and the solid phase.
So the triple point is liquid, solid, gas, all in equilibrium.
Now you may think, well I've seen that before.
You take a glass of ice water and set it down.
There's the water phase, there's the ice cube is the solid phase, and there's some water, gas, vapor, and that's one bar.
Where am I going wrong here?
The partial pressure of the water, of gaseous water, above that equilibrium of ice and water is not one bar, it's much less.
So the partial pressure or the pressure by which you have this triple point, happens to be 6.1 times 10 to the minus 3 bar.
There's hardly any vapor pressure above your ice water glass.
So this unique temperature and unique pressure defines a triple point everywhere, and that's a great reference point.
Any questions?
Great.
So now we have this ideal gas thermometer, and out of this ideal gas thermometer, also comes out the ideal gas law.
Because we can take our interpolation here, our linear interpolation, the slope of this line.
Let's draw it in degrees Kelvin, instead of in degrees Celsius.
So we have now temperature in degrees Kelvin.
We have the quantity f(t) here.
We have an interpolation scheme between zero and 273.16 with two values for this quantity, and we have a linear interpolation that defines our temperature scale, our Kelvin temperature scale.
And so the slope of this thing is f(t) at the triple point, which is this point here, this is the temperature of the triple point of water, divided by 273.16.
That's the slope of that line.
The quantity here, which is f (t of the triple point), divided by the value of the x-axis here.
So that's the slope, and the intercept is zero, so the function f(t), you just multiply by t here.
This is the slope.
f(t) is just the limit.
As p goes to zero of p times v bar.
And so now we have this quantity, p times v bar, and the limit of p goes to zero is equal to a constant times the temperature.
That's a universal statement.
It's true of every gas.
I didn't say this is only true of hydrogen or nitrogen, This is any gas because I'm taking this limit p equals to zero.
Now this constant is just a constant.
I'm going to call it r. I'm going to call it r. It's going to be the gas constant, and now I have r times t is equal to the limit, p goes to zero of p r. It's true for any gas, and if I remove this limit here, r t is equal to p v bar, I'm going to call that an ideal gas.
See, this is the property of an ideal gas.
What does it mean, ideal gas?
It means that the molecules or the atoms and the gas don't know about each other.
They effectively have no volume.
They have no interactions with each other.
They occupy the same volume in space.
They don't care that there are other atoms and molecules around.
So that's basically what you do when you take p goes to zero.
You make the volume infinitely large, the density of the gas infinitely small.
The atoms or molecules in the gas don't know that there are other atoms and molecules in the gas, and then you end up with this universal property.
All right, so gases that have this universal property, even when the pressure is not zero, those are the ideal gases.
And for the sake of this class, we're going to consider most gases to be ideal gases.
Questions?
So now, this equation here relates three state functions together: the pressure, the volume, and the temperature.
Now, if you remember, we said that if you had a substance, if you knew the number of moles and two properties, you knew everything about the gas.
Which means that you can re-write this in the form, volume, for instance, is equal to the function of n, p, t..
In this case, V = (nRT)/P.
Have two quantities and the number of moles gives you another property.
You don't need to know the volume.
All you need to know is the pressure and temperature and the number of moles to get the volume.
This is called an equation of state.
It relate state properties to each other.
In this case it relates the volume to the pressure and the temperature.
Now, if you're an engineer, and you use the ideal gas law to design a chemical plant or a boiler or an electrical plant, you know, a steam plant, you're going to be in big trouble.
Your plant is going to blow up, because the ideal gas law works only in very small range of pressures and temperatures for most gases.
So, we have other equations of states for real gases.
This is an equation of state for an ideal gases.
For real gases, there's a whole bunch of equation the states that you can find in textbooks, and I'm just going to go through a few of them.
The first one uses something called a compressibility factor, z. Compressibility factor, z.
And instead of writing PV = RT, which would be the ideal gas law, we put a fudge factor in there.
And the fudge factor is called z.
Now we can put real instead of ideal for our volume.
z is the compressibility factor, and z is the ratio of the volume of the real gas divided by what it would be were it an ideal gas.
So, if z is less than 1, then the real gas is more compact then the ideal gas.
It's a smaller volume.
If z is greater than 1, then the real gas means that the atoms and molecules in the real gas are repelling each other and wants to have a bigger volume.
And you can find these compressibility factors in tables.
If you want to know the compressibility factors for water, for steam, at a certain pressure and temperature, you go to a table and you find it.
So that's one example of a real equation of state.
Not a very useful one for our purposes in this class here.
Another one is the virial expansion.
It's a little bit more useful.
What you do is you take that fudge factor, and you expand it out into a Taylor series.
So, we have the p v real over r t is equal to z.
Now, we're going to take z and say all right, under most conditions, it's pretty close to 1, when it's an ideal gas.
And then we have to add corrections to that, and the corrections are going to be more important, the larger the volume is.
Remember, it's the limit of p times v goes to zero, so if you have a large volume with a large pressure, then you're out of the ideal gas regime.
So let's take Taylor series in one over the volume, it's going to be one over the volume squared, etcetera.
And these factors on top, which are going to depend on the temperature, are the virial coefficients, and those depend on the substance.
So you have this p B(t) here.
This is called a second virial coefficient.
And then, so you can get, you can actually find a graph of this B(t).
It's going to look something like this.
It's the function of temperature, as B(t).
There's going to be some temperature where B(t) is equal to zero.
In that case, your gas is going to look awfully like an ideal gas.
Above some temperature is going to be positive, below some temperature is going to be negative.
Generally, we ignore the high order terms here.
So again, if you do a calculation where you're close enough to the ideal gas, and you need to design your, if you have an engineer designing something that's got a bunch of gases around, this is a useful thing to use.
Now, the most interesting one for our class, the equation of state that's the most interesting, is the Van der Waals equation of state, developed by Mr. Van der Waals in 1873.
And the beauty of that equation of state is that it only relies on two parameters.
So let's build it up.
Let's see where it comes from.
Let me just first write it down, the Van der Waals equation of state.
p plus a over v bar squared times v bar minus b equals r t. All right, if you take a equal to zero, these are the two parameters, a and b.
If you take those two equal to zero, you have p v is equal to r t. That's the ideal gas law.
Let's build this up.
Let's see where this comes from, where these parameters a and b comes from.
So, the first thing we're going to do is we're going to take our gas in our box, let's build a box full of gases here.
We've got a bunch of gas molecules or atoms.
OK, there's the volume of a box here.
While these gas molecules or atoms through first approximation, are like hard spheres.
They occupy a certain volume.
Each atom or molecule occupies a particular volume.
And so, we can call b is the volume per mole of the hard spheres, volume per mole that is the little sphere that the molecules are.
So that the volume that is available to any one of those spheres is actually smaller than v. Because you've got all these other little spheres around, so the actual volume seen by any one of those spheres is smaller than v. So when we take our ideal gas law, p v bar is equal to r t we have to replace v bar by the actual volume available to this hard sphere.
So instead of v bar, we write p v bar minus b, equal r t. OK, that's the hard sphere volume of the spheres.
Now, those molecules or atoms that are in here, also feel each other.
There are a whole bunch of forces that you learn in 5.112, 5.111 like with Van der Waals' attractions and things like this.
So there are attractive forces, or repulsive forces that these molecules feel, and that's going to change the pressure that the molecules feel.
For instance, if I have, what is pressure?
Pressure is when you have one of these hard spheres colliding against the wall.
There's the hard sphere.
It wants to collide against the wall to create a force on the wall, and I have a couple of the hard spheres that are nearby, right, and in the absence of any interactions, I get a certain pressure.
This thing would but careen into the wall, kaboom!
You'd have this little force, but in the presence of these interactions, you've got these other molecules here that are watching this, you know, their partner sort of wants to do damage to themselves, like hitting that wall, and they say, no!
Come back, come back, right?
There is an attractive force.
There are no other molecules on that side of the wall.
So there's an attractive force that makes the velocity within not quite as fast.
The force is not quite as strong as it was without this attractive force.
So the real pressure is not quite the same because of this attractive force as it was, as it would be without the attractive forces.
The pressure is a little bit less in this case here.
So instead of this p here.
Now if I re-write this equation here as p is equal to r t divided by v bar minus b, just re-writing this equation as it is.
So the pressure is going to depend on how strong this attractive force is.
So the pressure is going to be less if there's a strong attractive force.
And the 1 over v squared is a statistical, is basically a probability of having another molecule, a second molecule in the volume of space.
So, if the molar volume is small, then one over v bar is large, there's a large probability of having two spheres together in the same volume.
If the molar volume is large, that means that there's a lot of room for the molecules, and they're now going to be close to each other, and so this isn't going to be as important.
So, a is the strength of the interaction, v bar is how likely they are to be close to each other.
And that's going to affect the actual pressure seen by the gas.
And a is greater than zero when you have the attraction.
And that gives use the Van der Waals' equation of state, with two parameters, the hard sphere volume and the attraction.
You don't have to go look up in tables or books.
You don't have to have all the values of the second virial coefficient, or the fudge factor, just two variables that make physical sense, and you get an equation of state which is a reasonable equation of state, and that's the power of the Van der Waals' equation of state, and that's the one we're going to be using later on this class to describe real gases.
Question?
OK, so we've done the zeroth law.
We've done temperature, equations of state.
We're ready for the first law.
We're just going to go to through these laws pretty quickly here.
Remember, the first law is the upbeat law.
It's the one that says, hey, you know, life is all rosy here.
We can take energy from fossil fuels and burn it up and make it heat, and change that energy into work.
And it's the same energy, and we probably can do that with 100% efficiency.
We can take heat from the air surrounding us and run our car on it with 100% efficiency.
Is this possible?
That's what the first law says, it's possible; work is heat, and heat is work, and they're the same thing.
You can break even, maybe.
So let's go back and see what work is.
Let's go back to our freshman physics.
Work, work is if you take a force, and you push something a certain distance, you do work on it.
So if I take my chalk here and I push on it, I'm doing work to push that chalk.
Force times distance is work.
The applied force times the distance.
There are many kinds of work.
There's electrical work, take the motor, you plug it into the wall, electricity makes the fan go around, that's electrical work.
There's magnetic work.
There is work due to gravity.
In this class here, we're going to stick to one kind of work which is expansion work.
So expansion work, for instance, or compression work, is if you have a piston with a gas in it.
All right, you put a pressure on this piston here, and you compress the gas down.
This is compression work.
Now the volume gets smaller.
p external here.
Pressure, the piston goes down by some volume l. The piston has a cross-sectional area, a, and the force -- pressure is force per volume area.
So the force that you're pushing down on here is the external pressure times the area.
Pressure is force per volume area.
That's the force you're using to push down.
Now the work that's it is calculated when you push down with the pressure on this piston here, that work is force times distance, f times I. f is p external times a, times the distance l. So that's p external times the change in the volume.
The area times this distance is a volume, and that is the change in volume from going to the initial state to the final state.
Now we need to have a convention.
We've got force.
Work is force times distance, it's p external times delta v, and I'm going to be stressing a lot that this is the external pressure.
This is the pressure that you're applying against the piston, not the pressure of the gas.
It's the pressure the external world is applying on this poor system here.
OK, but we need a convention here.
The convention, and then we need to stick to it.
And this convention, unfortunately, has changed over the ages.
But we're going to pick one, and we're going to stick to it, which is that if the environment does work on the system, if we push down on this thing and do work on it, to compress it, then we call that work negative work.
No, we call that work positive work.
All right, so that means we need to put a negative sign right here, by convention.
So if delta v is negative, in this case delta v is negative, OK, delta v is negative, pressure is a positive number, negative times negative is positive, work is greater than zero.
We're doing work on the system, to the system.
In this case here, work is positive.
If you have expansion on the other side, if the system is expanding in the other direction, if you're going this way, right, you're going to do work to the environment.
There might be a mass here.
This could be a car.
Pistons in the car, right, so the piston goes up.
That's going to drive the wheels.
The car is going to go forward.
You're doing work on the environment.
Delta v is going to be negative.
w is going to be negative.
Sorry, I got it backwards again.
Delta v is positive in this direction here, the work is negative.
So work on the system is positive.
Work done by the system is negative.
Convention, OK, this negative sign is just a pure convention.
You just got to use it all the time.
If you use an old textbook, written when I was taking thermodynamics, they have the opposite convention, and it's very confusing.
But now we've all agreed on this convention, and work is going to be with the negative sign here.
OK, any questions?
This is an example where the external pressure here is kept fixed as the volume changes, but it doesn't have to be kept fixed.
I could change my external pressure through the whole process, and that's the path.
We talked about the path last time being very important.
Defining the path.
So if I have a path where my pressure is changing, then I can't go directly from this large volume to this small volume.
I have to go in little steps, infinitely small steps.
So, instead of writing work is the negative of p external times delta v, I'm going to write a differential.
dw is minus p external dv, where this depends on the path, it depends on path and is changing as v and p change.
Now I'm going to add a little thing here.
I'm going to put a little bar right here.
And the little bar here means that this dw that I'm putting here is not an exact differential.
What do I mean by that?
I mean that if I take the integral of this to find out how much work I've done on the system, I need to know the path.
That's what this means here.
It's not enough to know the initial state and the final state to find what w is.
You also need to know how you got there.
This is very different from the functions of state, like pressure and temperature.
There's a volume, there's a temperature, than the pressure here.
There's other volume, temperature and pressure here, corresponding to this system here.
And this volume, temperature and pressure doesn't care how you got there.
It is what it is.
It defines the state of the system.
The amount of work you've put in to get here depends on the path.
It's not a function of state.
It's not an exact differential.
So the delta v here is an exact differential, but this dw is not.
That's going to be really important.
So if you want to find out how much work you've done, you take the integral from the initial state to the final state of dw minus from one to two p external dv, and you've got to know what the path is.
So let's look at this path dependence briefly here.
We're going to do two different paths, and see how they're different in terms of the work that comes out.
So we're going to take an ideal gas, we can assume that it's ideal.
Let's take argon, for instance, a nice, non-interacting gas.
We're going to do a compression.
We're going to take argon, with a certain gas, certain pressure p1, volume V1, and we're going to a final state argon, gas, p2, V2.
Where V1 is greater than V2, and p1 is less than p2.
So if I draw this on a p v diagram, so there is volume on this axis.
There's pressure on this axis.
There is V1 here.
There's V2 here.
There's p1 here, and p2 here.
So I'm starting at p1, V1.
I'm starting right here.
And I'm going to end right here.
Initial find -- there are many ways I can get from one state to the other.
Draw any sort of line to go here, right?
There are a couple obvious ones, which we're going to -- we can calculate, which we're going to do.
So, the first obvious one is to take V1 to V2 first with p constant.
So take this path here.
I take V1 to V2 first, keeping the pressure constant at p1, then I take p1 to p2 keeping the volume constant at V2.
Let's call this path 1.
Then you take p1 to p2 with V constant.
An isobaric process followed by a constant volume process.
You could also do a different path.
You could do, let me draw p v, there's my initial state.
My final state here, I could take, first, I could change the pressure, and then change the volume.
So the second process, if you take p1 to p2, V constant, and then you take V1 to V2 with p constant.
This is path number two.
Both are perfectly fine paths, and I'm going to assume that these paths are also reversible.
Let's assume that both are reversible, meaning that I'm doing this pretty slowly, so as I change, let's say I'm changing my volumes here, V1 to V2, it's happening, I'm compressing it slowly, slowly, slowly so that at any point I could reverse the process without losing energy, right?
It's always an equilibrium.
All right, let's calculate the work that's involved with these two processes.
Remember it's the external pressure that's important.
In this case, because it's a reversible process, the external pressure turns out to be always the same as the internal pressure.
It's reversible, that means that p external, equals p. I'm doing it very slowly so that I'm always in equilibrium between the external pressure and the internal pressure so I can go back and forth.
So, let's calculate w1.
The work for path one.
First thing is I change the volume from V1 to V2 The external pressure is kept constant, p1, so it's minus the integral from 1, V1 to V2, p1, dv.
And then the next step here is I'm going from -- the pressure is changing.
I'm going from V2 to V2 dv -- what do you think this integral is?
Right, so this is easy part, zero here.
This one is also pretty easy.
That's minus p1 times V2 minus V1.
p1 times V2 minus V1.
What that turns out to be, this area right here.
It's V1 minus V2 times p1.
This is w1 here.
OK, I can re-write this as p1 time V1 minus V2 and get rid of this negative sign here.
Now V1 is bigger than V2, so this is positive.
So I am compressing, I'm doing work to the system, positive work everything follows our convention.
Number two here, OK, the first thing I do is I change the pressure under constant volume, V1, V1 minus p dv, and then I change the volume from V1 to V2 and then this is p2, dv.
This first integral is zero V1 to V1, then I get minus p2 times V2 minus V1 or p2 times V1 minus V2.
Again, a positive number.
I'm doing work to the system to go from the initial state to the final state.
But it's not the same as w1.
In this case, I have p1 times delta V. In this case here, I have p2 times delta V. And p2 is bigger than p1.
w2 is bigger than w1.
The amount of work that you're doing on the system depends on the path that you take.
All right, how do I, practically speaking, how do I do this?
Anybody have an idea?
How do I keep p1 constant while I'm lowering the volume?
Change the temperature?
Change the temperature, right.
So what I'm doing here is I'm cooling, and then when I'm sitting at a fixed volume and I'm increasing the pressure, what am I doing?
I'm heating, right?
So I'm doing cooling and heating cycles.
So in this case here, I cool and then I heat.
In this case here, I heat and then I cool.
All right, so I'm burning some energy, I'm burning some fuel to do this somehow, to get that work to happen.
All right, now suppose that I took these two paths, and coupled them together.
So in this case, it's the amount of work is the area under that curve.
And in this case here, the amount of work is bigger, w2 is bigger, and it's the area under this curve.
Now, suppose I took this two paths, and I took -- couple them together with one the reverse of the other.
So I have my initial state, my final state, my initial state, my final state here.
And I start by taking my first path here.
I cool, I heat.
So there's w1.
So the w total that I'm going to get, is w1, and then instead of the path from V1 to, from 1 to 2 going like this as we had before, I'm going to take it backwards.
If I go backwards, to work -- everything is symmetric, the work becomes the negative from what I had calculated before, so this becomes minus what I calculated before for w2.
The total work, in this case here, is p1 times V1 minus p2 times V1 minus V2, it's p1 minus p2 times V1 minus V2.
This is a positive number, p1 is smaller than p2.
This is a negative number.
The total work is less than zero.
That's the work that the system is doing to the environment.
I'm doing work to the environment.
The work is negative, which means that work is being done to the environment.
And that work is the area inside the rectangle.
What you've built is an engine.
You cool, you heat, you heat, you cool, you get back to the same place, but you've just done work to the environment.
You've just built a heat engine.
You take fuel, rather you take something that's warm, and you put it in contact with the atmosphere, it cools down.
You take your fuel, you heat it up again.
It expands.
You change your constraints on your system, you heat it up some more, then you take the heat source away, and you put it back in contact with the atmosphere.
And you cool it a little bit, change the constraints, cool it a little bit more, and heat, and you've got a closed cycle engine.
We're going to work with some more complicated engines before.
But the important part here is that the work is not zero.
You're starting at one point.
You're going around a cycle and you're going back to the same point.
The pressure, temperature, and volume are exactly the same here as when you started out.
But the w is not zero.
The w, for the closed path, and when I put a circle there on my integral that means a closed path, when you start and end at the same point, right, this is not zero.
If you had an exact differential, the exact differential around a closed path, you would get zero.
It wouldn't care where the path is.
Here this cares where the path is.
So, work is not a function of state.
Any questions on work before we move on to heat, briefly?
So heat is a quantity that flows into a substance, something that flows into a substance that changes it's temperature, very broadly defined.
And, again, we have a sign convention for heat.
So heat, we're going to call that q.
And our sign convention is that if we change our temperature from T1 to T2, where T2 it's greater than T1 then heat is going to be positive.
Heat needs to go into the system to change the temperature and make it go up.
If the temperature of the system goes down, heat flows down heat flows out of the system, and we call that negative q.
Same convention is for w, basically.
Now, you can have a change of temperature without any heat being involved.
I can take an insulated box, and I can have a chemical reaction in that insulated box.
I can take a heat pack, like the kind you buy at a pharmacy.
Break it up.
It gets hot.
There's no heat flowing from the environment to the system.
I have to define my terms.
My system is whatever's inside the box.
It's insulated.
It's a closed system.
In fact, it's an isolated system.
There's no energy or matter that can go through that boundary.
Yet, the temperature goes up.
So, I can have a temperature change which is an adiabatic temperature change.
Adiabatic means without heat.
Or I could have a non-adiabatic, I could take the same temperature change, by taking a flame, or a heat source and heating up my substance.
So, clearly q is going to depend on the path.
I'm going from T1 to T2, and I have two ways to go here.
One is non-adiabatic.
One is adiabatic.
All right, now what we're going to learn next time, and Bob Field is going to teach the lecture next time, is how heat and work are related, and how they're really the same thing, and how they're related through the first law, through energy conservation.
OK, I'll see you on Wednesday then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of courses, visit MIT OpenCourseWare at ocw.mit.edu
As you can probably tell from the volume of my voice, I'm not Moungi.
I'm Bob Field, and I will be lecturing today as a special booby prize, because Bawendi is so wonderful.
Last time you heard about work and heat.
So there's p v work and it's given by the integral minus p external dv or just the integral from one to two of dw.
And this little slash here means an in inexact differential.
The reason for inexact doesn't mean it's a crummy measurement, it means that it's path dependent, and so the value of this integral depends on how you get from one to two.
w greater than zero means that work is done on the system, and it's really important to keep track of the signs of these things.
For heat, heat is defined, or the energy unit for heat, calorie, is defined as the amount of heat needed to raise one gram of water from 14.5 degrees c to 15.5 degrees c. That great deal of specificity implies that heat is also path-dependent and again we have the convention that if heat is added to the system, the quantity is greater than zero.
OK, so that's a quick statement of what happened last time.
Today we're going to talk about heat capacity.
The first law, a process that takes a gas from p1, V1, T to p2, V2, T examined for various paths.
And what you'll find is that the maximum work out is obtain for a reversible path.
We'll then look at the quantity, internal energy, which we define through the first law, and we think of it as a function of two variables T and V. And whenever we say something like that, we'll be able to write a differential, the partial of u, with respect to T at constant V, dT, plus the partial of u with respect to V at constant T dV.
So, if we write something like this, it implies this kind of equation.
And the main points of the latter part of the lecture is what are these quantities?
How do we measure them?
How do we understand them?
This one turns out to be the heat capacity, and this one turns out to be something that we measure in the Joule-free expansion.
So, that's the menu for today, at least that's what Moungi assigned me to cover, and I'll do the best I can.
So, let's talk about heat capacity.
Heat capacity relates the amount of heat that you add to the system to the change in temperature, and this is the relationship.
The heat-added, temperature, and this is a proportionality constant.
Heat capacity depends on path.
So, here are two kinds of experiments.
Here we have a fixed volume, and we have a little candle, and we're adding heat, and when we add heat, the pressure does what?
It increases and the temperature.
OK, we're you know, this is MIT.
You guys know something.
You should yell it out, not just whisper it.
You have the confidence -- I mean maybe up the street we whisper, but here we know it.
And, so here is a different kind of system where we have a constant external pressure.
We add heat to the system.
And so here the volume can change.
The temperature can change.
And so as we add heat here, the temperature goes up and the volume -- that was even quieter than last time.
OK, but you know it.
You understand it.
Now, the coefficient that relates the amount of heat in to the temperature change is obviously going to be different for these two cases.
in this case, where we have a fixed volume, we say dq is equal that Cv, heat capacity dT, where the v specifies what's constant for the path.
And so this is one path, constant volume.
This has a particular value.
Over here, we have dq=Cp dT, the heat, the proportionality between heat and temperature rise is given by this, the constant pressure heat capacity.
They're different quantities.
If we want to know the total heat added to the system, we can measure it, which is the straightforward thing, but sometimes you want to calculate in advance, or sometimes you want to calculate it on an exam.
So, we do an integral over a path, for the heat capacity along that path, dT.
So, for Cp and Cv, these are often quantities that are measured as a function of temperature, and one could, in fact, calculate this integral.
For most problems on exams, those quantities are constant, independent of temperature.
But there's no guarantee that they will be.
Now I get to tell a fun story.
The relationship between heat and work was initially proposed in the 1940's by Joule.
Now, there are two stories about Joule and how he came to this insight.
One is that he observed when people were machining cannon barrels, a lot of heat was generated, and there was a lot of work done.
And so maybe the work generated the heat.
Or there was a relationship between work and heat.
Well, this is ridiculous because people were making cannons long before 1840.
The other story, which is probably just as untrue, is that Joule, on his honeymoon, took his wife to a mountain resort, and they were sitting at the bottom of a waterfall, and he had this wonderful idea.
And that was the water at the bottom of the waterfall is probably going to be hotter than the water at the top of the waterfall.
So he grabbed his thermometer, and went and made a couple of measurements and discovered the first law of thermodynamics.
In fact, the water at the bottom of the waterfall is hotter than the water at the top of the waterfall.
And the idea was that gravity did work on the water and falling, and that work led to the generation of heat.
If you think about this, you probably can come up with several other ideas for how the water at the bottom might be warmer than at the top.
I mean it's flowing fast at the top, and it's sort of pooled at the bottom, and there is sunlight and all sorts of things that could make this coincidental as opposed to an insightful observation.
But it's a good story, Joule decided that there must be a direct relationship between work and heat.
They are the same quantity.
They are both forms of energy.
And so, we have this cartoon.
Again, we have an open beaker and a candle, and we're putting only heat into this beaker, and the temperature goes from T1 to T2.
And delta T is given by the heat, which has to do with how much of the candle burnt, divided by the constant pressure heat capacity.
Now you could do a similar sort of thing, but instead of having a candle, you have a paddle wheel, and the paddle wheel is spun by a weight that's dropping from here to here.
So, this weight is being spun-- now I'm really fantastic at drawing this mechanical device, but you can imagine that dropping a weight can cause a paddle wheel to turn.
And we know about gravity, and we know about work.
So, if you're doing physics, work is the integral of force, dx, x1 to x2.
Right, that's work.
Now, gravity, well force, is equal to mass times acceleration, and the acceleration due to gravity is g. The force, as this weight drops is constant, and so the work is just going to be m g h, where this is h. So, it's a trivial matter, by looking at what is the weight, and how far does it drop, to say OK, how much work is done by the paddle wheel.
And this is in an isolated, in an adiabatic container.
All the energy that is inserted into this, which might be turbulence initially, becomes heat, or becomes -- it raises the temperature.
And so, again, we see a temperature increase, and we know the work, and the temperature increase, it's a constant pressure thing.
And so we now have two observations.
The same temperature increase, work and heat, and we have a relationship between heat and work.
The first law of thermodynamics is -- I'm fine.
I don't -- I'm all set, I only use this chalk.
I brought it with me.
The first law of thermodynamics, written concisely is dw plus dq, two inexact differentials, integrated over any closed path, is zero.
This is an abstract and powerful mathematical statement of the first law of thermodynamics.
There are much better or more appealing expressions.
One is, du, u is called the internal energy or just the energy, is equal to dq plus dw.
OK, notice we have two inexact differentials and exact differentials.
This is a condition.
If you have a quantity which is constant over any closed path, that quantity is a thermodynamics state function.
So, this observation is equivalent to saying that there must be something that is path independent.
Therefore, we don't have a cross through the d. And the first law says, well heat and work are different forms of energy, and we can add them, and the path dependence of these two things is somehow cancelled in the fact that we have this internal energy.
So, we can also write delta u as integral from 1 to 2 of du.
That's u2 minus u1, and it's q plus w. So, these two quantities, again, are path dependent.
This is not.
That's the first law of thermodynamics.
It seems trivial, but it's really important.
It's almost as important as the second law of thermodynamics, which you'll see in a week or so.
There is a corollary to this.
If we say we have a system, and it's in its surroundings.
We can talk about du for the system well, that's q plus w. And we can say du for the surroundings.
Well the system got its work from the surroundings.
It either had work, got its heat from the surroundings, or it got worked on by the surroundings.
And so, we can immediately write this and then we can write du for the universe, which is system plus surroundings is equal to zero.
This is the corollary.
It's due to Clausius and it says the energy in the universe is conserved.
Fairly powerful statement, and that's another form of the first law of thermodynamics.
OK, enough for really great discoveries.
Now let's talk about some simple observations on isothermal gas expansions.
The purpose here is to look at a series of processes in which temperature is held constant, and we're going to calculate how much work we get from allowing a gas to expand under various conditions.
And what we'll discover is that when we allow the gas to expand reversibly, we get the most work.
This is neat.
This is important.
OK, so we have constant temperature, because it's isothermal.
And let's do a series of experiments.
First let's set the external pressure equal to zero.
So we have an experiment that looks like this.
We have a piston.
We have a pair of stops.
We have another pair of stops.
We start at p1, V1.
and p external is equal to zero.
In other words, this is vacuum.
And so what happens when we remove these stops is that the piston slams up against the next pair of stops.
It goes very fast.
You could calculate the time.
You could do all sorts of stuff, but what is important is that this piston in moving from here to here did no work on the surroundings because work is the integral of pressure dv, and the pressure external is zero.
It doesn't matter what the pressure internal is.
This is a point that is often confusing, because you can think, well maybe I could calculate what the internal pressure is even for this very rapid process.
It doesn't matter.
Thermodynamics is asking you, what work does this thing do on the surroundings or the surroundings do on the system?
And there is no work done on the surroundings because pressure is zero.
So, the work for this process is the integral, or minus the integral, V1, V2, p dV.
And it's p external and p external is zero, so there's no work.
OK, the next example is, well, when this piston slams up against these stops, the gas has expanded -- we're doing this isothermally, so this is in contact with the heat bath.
The gas has expanded.
It has a particular pressure and a particular volume.
Every time you do the experiment in equilibrium with the heat bath at T, you'll get the same p2 and V2.
One can know them.
And so we can then say, OK, let's do this second expansion.
and let's set p external equal to p2.
And we do the same sort of thing.
We have stops here.
We have p1, V1.
And we don't really need stops up here.
We have a weight on the piston, and that weight is chosen so that it acts as though p external is equal to p2.
We know the pressure is equal to force per area.
And so we know the force for a particular mass, and we know the area of the piston.
so we know how to do this, and we know this thing when it hit these stops, the pressure with p2, and the volume was V2.
Now we could put stops here at V2, but this piston would just kiss the stops.
It would stop there because we've chosen p2.
The work for that process is going to be minus V1, V2, p2, because that's what we chose p external to be, dV, and that's going to be minus p2, V2 minus V1.
That's the work.
V2 is larger than V1, and so this quantity here is positive, and we have a negative sign because the system did work.
OK, so now we can take the result from this and put it onto a p v diagram.
p1, p2, V1, V2 -- so here is the initial situation, and here is the final situation.
And the equation of state, pressure versus volume at constant temperature, is going to have some form, let's just draw it in there like that.
So that's an equation of state.
It's like the ideal gas law, and one could know that in principle.
OK, so we did work -- oh, I should mention p times V has units of energy or units of work.
Remember that, because you're going to find lots of thermodynamics quantities, and you're often going to be writing on exams, deriving things on exams, and you're going to almost always want the combinations of quantities to have units of energy.
So, dimensional analysis is extremely valuable in thermodynamics, and here is an example of it.
Anyway, the work associated with process number two is described by this box.
Because we did work at constant pressure, and so it's just volume difference times pressure.
OK, now we can do the process in two steps.
So we start out with our piston.
We have two weights on it.
So we have p1, V1.
We have stops.
We've chosen two weights, so that this is p3.
External pressure is p3.
It's higher than p2.
So we remove the stops, and the piston moves up, and now we remove one of these weights.
and when we remove one of the weights, this gives an external pressure p2.
So, the piston moves up again, one weight.
So here is p2, p external.
Here is p3.
And so we have the work coming from two steps.
And it's trivial to calculate what that will be, and when you do that, if we put p3 in on this diagram, what you end up finding is that you get an initial little bit of work corresponding to V3.
So we break up our work into three pieces, and we get more work out.
So the final process involves do it reversibly, or almost reversibly.
We want p equal to p external for the entire expansion, and p external is decreased steadily from p1 to p2.
How do we do this?
Well, we start out with our usual system, and we have a bunch of little pebbles on it.
Each weighing, say, one 100th of the weight needed to establish p external is equal to p1.
We remove a pebble, system expands.
We remove another one, system expands.
That's equivalent to doing the integral, and so, what we end up getting is that the reversible work is equal to minus integral V1, V2, p dV.
Because what we've done is we forced p, pressure here, to be equal to the external pressure.
We've changed the external pressure slowly, and again this is isothermal.
There is a heat bath here that keeps the temperature constant.
So, the system does work on the surroundings, hence the minus sign.
Now, if this is an ideal gas, we know that pressure is equal to nRT over volume.
So we can put that in here and do this integral.
We have minus V1, V2, nRT over V dV.
These are all constant.
It's isothermal.
We can take them out.
It's a closed system, so the number moles doesn't change.
The ideal gas constant doesn't change, temperature doesn't change, and so we just have minus nRT integral V1, V2, dV over V. Now, we are very gentle in this course with respect to knowing integrals, but this is one you have to know.
The integral of one over a quantity is the natural log.
And so we can write this, minus nRT log V2 over V1.
That should not take a breath.
You know that much about integrals.
OK, now we actually would like to simplify this or to write this in terms of not the volume change, but the pressure change.
So, we have V2 over V1.
Well, what we can write that using the ideal gas law twice, V2 is equal -- pV = nRT.
So V2 = (nRT)/p2, and V1 = (nRT)/p1.
So, the nRT's cancel, and we have p1 over p2.
And so, we can rewrite this as the work is equal to minus nRT log p1 over p2, or nRT log p2 over p1.
Now, p2 is less than p1, so this is a negative quantity.
The system has done work.
So the reversible process, we had this curve, and for the irreversible processes, we got this, and then this, whoops, and now we get the whole thing.
So for the reversible process, the work done is the integral under the pressure volume state function, the function of state.
OK, what time is it?
I'm going to actually get caught up, make up for but Bawendi's slow lecturing.
That doesn't mean it's better, it just means that I'm making up for a problem that he created.
OK, so what do we know so far?
Maximum work out is by reversible path.
Delta u is q plus w. So the maximum work out required the maximum heat in.
So a reversible process leads to requiring certain quantities to be maximized.
Now, we have u, q and w. We have a relationship between them.
Often, for a particular state change, it is easy to calculate two of these, but not the third.
And because there is an explicit relationship between u, delta u, q and w, you can always find the easy way to derive the change in internal energy or the heat or the work.
Even if the thing that you really want is an integral that would be difficult to evaluate.
OK, now, we're going to look at the internal energy, and we're going to pretend that it is explicitly a function of temperature and volume.
We could choose any two quantities, and, in fact, it turns out that these are going to prove, after we have the second law, not to be the best choice.
But it's allowed to say the internal energy is a function of temperature and volume.
When you say that, it implies that the differential is given by this pair of partial derivatives.
V dT, partial of u with respect to the volume holding temperature constant, dV.
So if you say this, it requires you to say this.
Now, the purpose of this exercise is to give you a little bit of practice in figuring out what these quantities are.
And do a little practice in manipulating these differentials.
So, we have, we're interested in the change in internal energy for various experimental constraints.
And so, one constraint is the process be done reversibly.
Well then, du is equal to dq reversible plus dw reversible, which is minus p dV, because p is equal to p external for a reversible process, and we can write that.
We could have isolated.
Suppose we're looking at the system isolated from the outside world.
Well, dq is equal to zero and dw is equal to zero because it's isolated.
So that implies du is equal to zero.
We could do an adiabatic process.
Adiabatic means there's no heat transferred in or out of the system.
We don't say anything about whether the system does work or has worked on -- implicitly here, we're talking about a closed system, so there's no mass leaving the system.
But if it's adiabatic, then dq is equal zero, and for an adiabatic process, then du is equal to dw.
OK, now this is a point we want to be careful.
Be careful.
Suppose we are doing an adiabatic process.
We can do it reversibly, or we can do it irreversibly.
So suppose we do it irreversibly.
Suppose we just remove stops, and the system slams up against the other stops.
Did no work.
Does that mean du is zero?
You bet it does not.
So, it's important to write this little thing on here, du is equal to the reversible work, not just the work.
You can have a process where you can measure an irreversible process, where you could calculate the work done.
It could be zero.
It could be something else.
But if it's not reversible, it's not du.
And you will be invited to make that mistake on an exam, I'm sure.
OK, so, the thing about a state function is that the function has a value for initial conditions and at final conditions.
And the difference between those is what you could measure.
If you measure something else, you won't get du.
Be careful, and this is going to be especially complicated and confusing when we get to quantities that have a more obscure meaning like entropy.
I mean we can, we can sort of understand why OK, the total energy, if we measure it, we measure a process which is not reversible.
Well it might not give the energy change for that process, but when we have a quantity which is more obscure, which the definition of that quantity requires a very specific prescription for calculating, you're going to get into trouble.
So exam this and be sure you understand that.
OK, constant volume.
Well for constant volume, dw is equal to zero.
Why?
How does it do, what is work?
Well it's p v work if the volume doesn't change.
There is no work.
So in this case du is equal to dq, and we put a little v on it to imply the work, the change in internal energy is equal to the heat added at constant volume.
Nothing reversible about it.
It's now, all we have to do is say we're going to have heat at constant volume.
OK, and now we return to this differential.
We want to ask the question, what are these two quantities?
How do we know what they are?
This should be particularly bothersome to you because, as you've already experienced in 5.60, there are a lot of partial derivatives.
there are a lot of variables.
There are a lot of things held constant.
It's easy to get lost in this sea of quantities, none of which have obvious meaning.
So now what we're going to do is start to extract what these things mean.
OK, so for a constant volume process, we can write du, partial derivative of u with respect to T at constant V, dT, plus partial derivative of u at constant V, dV.
OK, for constant volume, this is zero.
So this term is gone, and we rewrite this du, V is equal to du/dT, at constant V, dT v. So we, have a change in temperature done at constant volume, and we have a change in internal energy done at constant volume, and we rearrange this.
And we discover that du/dT at constant V is equal to du/dT at constant V. What, have I done something silly?
Oh well, yes I have.
dq v, so du v is equal to dq v and so what I should have written here, this is true, we can get a partial derivative by taking two total derivatives at the same pressure, at the same quantity, but what I really wanted to do is to write dq v is equal to the partial derivative of T, constant v dT v. OK, and now, we know the relationship between heat and a temperature change is given by a quantity, a heat capacity for a particular path, and here it is.
So what we've discovered from this relationship that du at constant volume is equal to dq v. We have discovered that this partial derivative that appears in the definition, the abstract definition of the differential for internal energy, is just equal to the constant volume heat capacity.
So, in this definition now, we have one term which we know.
It's something that we can measure.
We can measure the heat capacity at constant volume, and now we have another term, and if we can figure out how to measure it, we'll have a complete form for this differential which will enable us to calculate du for any process.
So let me write where I am now, or we are now.
du is equal to Cv dT, plus partial of u with respect to volume at constant temperature dV.
So, we've simplified the expression by replacing one of the partial derivatives by a quantity that we can measure.
And we like to know what about this.
So we need an experiment that will enable us to measure this quantity.
And that's where we get Joule, and I like to say it Joule's free expansion -- it's usually referred to as the Joule free expansion, which sort of implies that no energy flows, which actually is true, but it's an experiment proposed by Joule, and the experiment involves an adiabatic box.
So we have system insulated from the outside world, and we have two bulbs.
There's a valve between them.
And so we have gas and we have vacuum.
So the Joule free expansion involves opening this valve and asking what happens when this gas moves into the other bulb or distributes between the two.
well since the gas is expanding into vacuum no work is done.
Since it's isolated, no heat is added.
So du is equal to zero because dq and dw are both zero.
So we can now take this expression and rewrite it under the condition of du is equal to zero.
So we have du equal to zero, just a zero here.
Cv dT constant u plus the derivative du/dV at constant T, dV at constant u.
This is just a number.
We don't have to specify that it's measured at constant u.
It's just a number.
So now we rearrange this expression, and so we get du/dV at constant T is equal to minus Cv times dT u over dV u or minus Cv partial derivative of temperature with respect for volume a constant, free energy, a constant internal energy.
So, this is the quantity we want.
It's related to the heat capacity, the constant volume of heat capacity and something you could measure.
What happens as you expand into volume?
Does the temperature go up?
Does it not change?
Joule actually did this experiment, and he observed that for the gas expansions that he could do, that the temperature did not increase measurably.
So he made an incorrect conclusion.
Because something was small and unmeasurable, he said, well the best of my knowledge dT/dV at constant u is equal to zero.
And that implies that since the quantity we want is given by this quantity, which is zero times a constant, the quantity we want is also zero.
So it would imply that du was equal to only the first term Cv dT.
Now, this is an important lesson in what you do in science.
You make an observation.
You make an observation doing an experiment that is as accurate as you can do.
And so an experiment said the gas didn't increase its temperature when it expanded the vacuum.
And so the next thing you do is you call up a journal and say I've discovered a fundamental law of nature.
And, so you propose that there is no, that this derivative is zero, and that the internal energy is given simply by this quantity.
It turns out that this quantity here, which is called eta of J the Joule free expansion parameter, is not quite zero.
When you expand a real gas into vacuum, the temperature goes down.
So this is a very small number and for ideal gases, eta J is equal to zero.
This quantity is exactly zero for an ideal gas and we'll discover why eventually it has to do with what we mean by an ideal gas it turns out.
And so for many, many problems, especially on exams, especially on this first exam, you will be able to say that this is the relationship between internal energy and temperature.
That u is a function of temperature only.
It doesn't matter what other things change.
The value of the internal energy is only determined by temperature.
But this is only true for an ideal gas and it's approximately true for other things.
So, delta u is equal to zero for all isothermal ideal gas processes.
And it's approximately equal to zero for all real gas processes.
And so that means that delta u is always calculable from Cv(T) dT for any ideal gas change.
So even if work is done, it doesn't matter.
All you care about is what was the temperature change?
And this is always the easy way to calculate du, and it's often the easy way to calculate either q or w, using the first law.
So, since du is equal to q plus w, and for an isothermal process this is equal to zero, q = -w. For an ideal gas, for an isothermal process.
It simplifies your life enormously.
You don't have to calculate much.
OK, so it's time to stop, I did, in fact, get caught up.
I hope you didn't mind the enhanced velocity.
Now Bawendi can do the beautiful stuff on lecture number four.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional material from hundreds the MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time you talked about the first law of thermodynamics.
And you talked about isothermal expansion, the Joule expansion.
You saw a very important result.
which is that for an ideal gas, the energy content is only dependent on the temperature, nothing else.
Not the volume, not the pressure, it just cares about the temperature.
So, if you have an isothermal process for an ideal gas, the energy doesn't change.
q plus w is equal to zero for any isothermal process.
And you also saw that du then could be written as Cv dT for an ideal gas always.
This is not generally true.
If you have a real gas and you write du is Cv dT, and your path is not a constant volume path, then you are making a mistake.
But for an ideal gas, you can always write this.
And this turns out to be very useful to remember.
OK, now most processes that we deal with are not constant volume processes.
So energy, which has this wonderful property here, du is Cv dt for constant volume process, which happens to be equal to d q, constant volume, because there's no change.
there's no work if you've got a constant volume process.
So du here is a very interesting quantity, because it's related to the heat that's going in or out of the system under constant volume process.
But as I said, we're not operating usually in a constant volume environment.
When I flail my arms around I generate work and heat.
This is not a constant volume process.
If I'm the system, what's constant when I do this?
Anybody have an idea?
What's the one function of state?
I'm the system, the rest are the surrounding.
What's the one function of state that's constant when I'm doing all my chemical reactions to move my arms around?
Temperature?
Pressure?
Pressure, right.
Pressure is constant.
What is the pressure at?
One atmosphere, one bar.
So the most interesting processes are the processes where pressure is constant.
When I had have a vial on bench top, and I do a chemical reaction in the vial, and it's open to the atmosphere, the pressure is constant at one atmosphere.
When you've got your cells growing in your petri dish, the pressure is constant at one atmosphere, even if they're evolving gas, pressure is constant.
So we'd really like to be able to find some sort of equation of state, or some sort of rather function of state that's going to relate the heat going in or out of the system with that function of state, because this isn't going to do it.
du only relates to the heat under constant volume.
And the heat is a really important thing to know.
How much heat do you need to put into a system, or how much heat is going to come out of a system when something is happening in the system?
All right, this is a really important quantity to know.
Your boiling water or whatever, you want to know how much heat do you need to boil that amount of water under constant pressure?
And this is where enthalpy comes in.
You've all heard of enthalpy.
H we're going to write it as the function of temperature and pressure.
And the reason enthalpy was invented was exactly for that reason, because we need some way to figure out how to relate the heat coming in or out of a system under a constant pressure process.
Because it's so important.
And I should add and also under reversible work, where the external pressure is equal to the internal pressure.
OK, so we're going to define enthalpy as u + pV, these are all functions of state here, So H is a function of state, and we're going to see now how this is, indeed, related to the heat flow in and out of the system.
If you have a constant pressure, reversible work process.
Let's take a system.
Under constant pressure T1, V1, going to a second -- this is the system, so let me write the system here.
And it's more dramatic if the system is a gas, p, T2, V2, And let's look at what happens to these functions of state, to H to u under this transformation.
OK, so let's look at delta u. Delta u is q plus w. That's the first law.
And this is a constant pressure path, so now I can write, this q is actually q under constant pressure.
Little p means the path is a constant pressure path.
And I'm doing reversible work.
So that w is minus p, dV where p is the pressure inside the system, minus p delta V. Rearrange that, delta u is plus p delta V is equal to q p. All right, so this is the heat flowing in or out of the system, and these are all functions of state.
This depends on the path.
It tells you right here, the path is constant pressure.
These don't depend on the path, right.
V doesn't care how you get there.
u doesn't care how you get there.
In this case, p is a constant because the path is constant.
So we can bring the p inside, delta u plus delta p V it's q p. Take this delta outside again, delta of u plus p V is equal to q p. And there you have it.
There is the H right there.
The u plus p V. Delta H is equal to q V. And this is the reason why enthalpy was invented, and why it's so important.
Because we want to know this.
So this for a finite change.
If you want to have an infinitestimally small change, you end up writing dh is dq sub p. It's not always equal to the heat.
It's only equal to the heat if your process is constant pressure reversible work.
OK, so this is the kind of, this is the kind of concept that needs to be branded into your brain, so that if I come into your bedroom in the middle of the night and I whisper to you delta H, you know, you should wake up and say q p, right?
Heat under constant pressure reversible work.
This should become second nature.
This is where the intuition comes from.
This is why people right tables and tables of delta H's.
Why you have delta H's from all these reactions, because this is basically the heat, and the heat is something we can measure, we can control.
We can figure out how much heat is going in and out of something.
This is what we're interested in.
OK, so last time you looked at -- any questions on this first?
Yes
[INAUDIBLE] from the T delta V to the delta p here?
What was the reasoning behind that?
p is constant here.
It's constant pressure.
OK, so now, last time you looked at the Joule expansion to teach you how to relate derivatives like du/dV.
du/dV under constant temperature.
du/dT under constant volume.
You use the Joule expansion to find these quantities.
Now these quantities were useful because you could relate them.
The slope of changes, with respect to volume or temperature of the energy with respect to quantities that you understood, that you could measure.
We're going to do the same thing here.
So if we take as our natural variables for enthalpy to be temperature and pressure, and we have some sort of change in enthalpy, dH, and it's going to be related to changes in temperature and pressure through the derivatives dH through the slope of the enthalpy in the T direction, keeping pressure constant, dT plus the slope of enthalpy in the pressure direction, keeping the temperature constant, dp, and these are knobs that we can turn.
We can change of temperature.
We can change the pressure.
These are physical knobs that are available to us as experimentalists.
And so when we turn these knobs on our system, we want to know how the enthalpy is changing for that system.
Because eventually they will tell us maybe things about how heat is changing further on.
OK, but in order to relate turning these physical knob to this quantity here, which we don't have a very good feel for, we've got to have a feel for the slopes.
If I keep the pressure constant.
I change the temperature, what does that mean?
What is dh/dT?
If I keep the temperature constant, and just change the pressure, dH is going to change, but how is it going to change?
What does this mean in terms of something I can physically understand?
That's the program now for the next few minutes.
What are these quantities?
What is dH/dT as a function, keeping pressure constant, what is dH/dp, keeping temperature constant?
All right, let's start with the first one, dH/dT, keeping the pressure constant.
And we're going to look at a reversible process to help us out, but the result is going to be independent of whether or not we have a reversible process or irreversible process.
Constant pressure, that means dp is equal to zero.
So for reversible process, constant pressure, what do we know?
This is already branded in your brain, right?
Reversible process, constant pressure dH=dq.
So we can write that down, dH=dq, constant pressure.
That's by definition of enthalpy.
That's why we created enthalpy.
What else do we know?
Well we can go look up here, looking at the differential, there are no approximations here.
This is just an equality.
I have a constant pressure process.
This term here is equal to zero.
That means that dH is also equal to dH/dT, constant pressure dT.
All right, so now I've got more dH/dT under constant pressure.
dH is equal to this, and it's also equal to this.
All right, these two are equal to each other.
Now, I know how to relate the heat flow to temperature change, through the heat capacity.
dq constant pressure is that heat capacity, and I have to tell you the path for the heat capacity.
So it's C sub p, the heat capacity under a constant pressure path, dT, all right?
So these two are equal to each other.
So, these two are equal to each other as well, which tells me that this derivative, dH/dT constant pressure is Cp.
So now I have my first of my two slopes, in terms of something that's related to my system, the heat capacity of the system.
Something I can measure and I can tabulate, and when I turn my dT knob here I know what's going to happen to the enthalpy.
So this is the first, this is the one.
So it's very similar to what we saw with the volume and the energy, where du/dV under constant temperature was equal to Cv in this case here, right.
and you're going to find that there's a lot of these analogies between energy and enthalpy.
You just change volume to pressure and basically you're looking at enthalpy under a constant -- anything that's done at a constant volume path with energy, there's the same thing happening under constant pressure path for enthalpy.
So you can guess the answer usually that way.
OK, so now we have the other one, dH/dp constant temperature.
How do we relate this to something physical?
Well, it's going to be an experiment, very much like the Joule experiment.
The Joule experiment was a constant energy experiment, right.
Here we're going to have to find a constant enthalpy experiment, and that is going to be the Joule-Thomson experiment.
That's going to extract out a physical meaning to this derivative here.
OK, the Joule-Thomson experiment.
This is going to get us dH/dp constant temperature.
What is this experiment?
You take a throttle valve, which consists of some sort of porous plug between two cylinders that is insulated.
There is insulation here.
Insulation on the bottom.
It's like a, think about a tube, a large tube, insulated tube, with a bunch of -- a frit inside here, which prevents flow of gas, which slows down the flow of gas from one side to the other.
It's a blockage in this tube here.
Then you put two pistons, one on that side here, and one on this side here, and the external pressure here, we're going to call that p1.
The external pressure here, we're going to call that p2, and we're going to do it slowly enough that the pressure on this side of the cylinder is in equilibrium with the external pressure, and the pressure on this side of the cylinder is in equilibrium with this pressure.
But not so slowly that these two are in equilibrium with each other.
So this is restricting the flow, so there's some sweet spot here.
When I'm pushing slowly enough here that the pressure here is equal to that one, but not so slowly that the air flow from here to here is fast, compared to how fast I'm pushing.
You got the picture here?
Any questions on that?
All right, then as I push through, I'm going to start with all of my gas on this side, and at the end, I'm going to have all the gas on the other side.
Let me first ask you this, is this a reversible or in irreversible process?
Right, let me add one more piece of data here which I said in words, but which is actually important to write down before doing the problem.
Is this a reverse -- any guesses?
How many people vote for that this is a reversible process?
I've got one vote back there, two votes, three votes, four votes.
Anybody else?
How many people think this is irreversible?
It's about a tie, and everybody else doesn't now.
All right, I'm going to give you ten seconds, fifteen seconds to make up your mind.
You're not allowed to be on the fence here.
You've got to decide, all right?
This is, you can talk to your neighbors, you know, do a little bit of thinking.
And I'm going to give you ten seconds to figure this out, what your vote is for that.
All right, let's try again.
How many people vote that this is reversible?
That looks like a majority to me.
Irreversible?
Let's look at the show of hands?
All right, so this is the majority here.
Good thing physics doesn't work on the rule of the majority, otherwise we'd be in big trouble.
Wow, let's walk through that.
I'm sorry to say that this is the wrong answer.
OK, why is that the wrong answer?
Well, just think, you know, think about it.
You're pushing through here.
p1 is greater than p2.
What does it mean for a process to be in equilibrium or reversible?
It means at any point you can reverse the direction of time, and it will look fine, right.
So now I'm pushing on this plug here, with p1 greater than p2.
I'm pushing, I'm pushing, I'm pushing.
p1 is greater than p2.
Then I want to reverse direction of time.
I want the arrow of time to go so that the gas goes from p2 to p1.
p2 is less than p1.
Is that going to work out?
If p2, the pressure in p2, is less than the pressure in p1, is the gas going to want to go from p2 to p1 and the whole thing reverse back?
You've got to put more pressure on one side than the other if you want to push that gas through the throttle, right?
So this is where the time scale issue comes into play.
I said let's do this slowly enough that this p1 is in equilibrium with this p1, but not so slowly that this pressure is equivalent to that pressure.
If I do it really, really slowly, so that everything is reversible, well I won't be able to do it, because p1 and p2 are different.
But suppose that I fix my pistons here, with p1 greater than p2, and I don't touch it, eventually p1 will be equal to p2.
I'll come to some sort of equilibrium.
So this is a system which is out of equilibrium.
If I stop, if I move slowly, if I move more slowly, then these two will want equilibrium.
So this is an irreversible process.
The Joule-Thomson experiment is irreversible.
OK, important -- if you are part of this group here, think about it and make sure you understand that.
All right, so this is the experiment.
Now what are we doing with that?
The initial state, let's look at what we are doing.
So initially, we're going to have our piston, there's the plug sitting here.
Our piston on the right side here fully out, and the piston on my right side, your left side, fully inside.
There's no gas on that side here.
So there's p2 sitting here.
There is p1 sitting here, and all of the gas is sitting on that side of the plug.
Then after we're done with the experiment, we'll have transferred all of the gas from one side to the other.
p1 here.
p2 sitting here.
And there's going to be some volume V2 and some volume V1, but are not necessarily the same.
Especially since the pressures are different.
we don't know yet about temperature so I don't know what to say about these volumes because I don't know what the temperatures' are going to do.
OK, so let's go through this and see what we would do which is to calculate the heat and the work.
This is well insulated.
So, what is the -- let's do the first one here.
What's q for this process here?
Anybody?
STUDENT Zero
Zero, right.
This is an adiabatic process.
It's well insulated.
Heat is not going in or out, adiabatic.
q is equal to zero.
So all we need to find out is the work now.
Let's divide it up into the two sides, the work going on on the left hand side, my left hand side or your left hand side, and the work going on on the right hand side.
So let's first look at the left hand side.
OK, so w, first of all, work is being done to the system on the left hand side here.
I'm pressing on the gas.
So I expect that to be a positive number.
The pressure is constant, p. The V goes from V1 to zero.
So we write down p1, V1.
On the right hand side, the work, let's call this left hand side, let's call this right hand side.
Here there's an expansion going on, so the system is doing work to the external world.
This piston is being brought out, so we expect the work to be negative, negative.
And we start out with zero volume.
We end up with a volume of V2, and the external pressure is constant to p2.
Minus p2, V2.
Minus p delta V. So the total work is the work from the left hand side plus the work on the right hand side, which is p1 V1 minus p2 V2.
Which I can rewrite as minus delta pV.
Delta pV is p2 V2 minus p1 V1.
It's the pressure volume multiplied together at the final state, minus pressure volume from the initial state, with the minus sign here because it's the negative of that.
All right, what is delta u?
delta u is q plus w. q is zero.
delta u is just w. So this is also just delta u. delta u is minus delta pV, for the process.
OK, delta H is delta of u plus pV.
By definition, that's how we define enthalpy up here.
H is u plus pV.
Delta H is delta of u plus pV, which is equal to delta u, plus delta pV.
Now delta u is minus delta pV.
So I have minus delta pV plus delta pV.
This is equal to zero.
So this irreversible process, this Joule-Thomson process, is a constant enthalpy process.
Delta h for this process is equal to zero.
Adiabatic q equal to zero.
It's also delta H which is zero.
The two didn't necessarily follow, because remember, delta H is dq so p is only true for a reversible constant pressure process.
This is an irreversible process.
So a priori, this was not necessarily true.
It turns out after you do all the math, it turns out to be delta H equals zero.
All right, so this is the experiment.
How do we go from that experiment to the terms that we're trying to get, these slopes.
Remember, we're trying to get delta H, we're trying to get dH/dT constant pressure and dH/dp constant temperature.
OK, these are the two things were trying to get here.
OK, so let's write down, what we know here.
I'm missing something.
Oh, we already know one thing.
We already know this guy here.
We already did that.
OK, dH/dT constant pressure is Cp.
That was easy one.
So we already know that.
So now we can write or differential dH as Cp dT plus dH/dp, constant temperature, dp.
Now we want to find out what this guy is here.
Now for this experiment, this is a constant enthalpy experiment for the Joule-Thomson experiment, this is equal to zero.
So I can rearrange this to get this dH/dp in terms of things that I can either measure, like the heat capacity, or that I have control of, like dT and dp.
So in this case, dH/dp constant temperature is minus Cp dT/dp, and this is under constant temperature, no, not constant temperature.
Whatever the experi -- for that the experiment is.
For that experiment, the constraint, so we need a constraint here, right, we need a constraint here.
Right?
We need a constraint here.
The constraint isn't constant temperature because the temperature is going to be changing.
It's not constant pressure, because we have a delta p going on.
It's not constant volume either.
The constraint is the constraint of the experiment, and the constraint of the experiment is that the enthalpy is constant.
So the constraints we have here, is the constant enthalpy.
It's the constant enthalpy process that we're looking at.
This we can do experiments on.
It's tabulated in books, and this we can measure in the experiment.
Delta p here is the change in pressure from the left side to the right side, and we can put a thermometer, measure the temperature before the experiment, and measure the temperature after the experiment.
So this is something we can measure.
So now we have this derivative, in terms of physical quantities, things that we can measure.
Things that we can relate to the properties of the substance that we're doing the experiment on.
So this is basically delta T and delta p and the Joule-Thomson experiment.
And so Joule and Thomson did these experiments, and they measured lots of gases, and they found that, in fact, this was something that they could measure.
Sometimes it was positive, sometimes it was negative, and it was an interesting number.
And so they defined them, after many experiments, the limit of this, delta T delta p and the limit of delta p goes to zero as the Joule-Thomson coefficient.
So, basically dT/dp, constant enthalpy is equal to mu, by definition, Joule-Thomson, where mu Joule-Thomson is the Joule-Thomson coefficient, just like you saw last time eta sub j was the Joule coefficient for dT/dV under constant energy.
So there's, again, total analogy here between what we're doing with enthalpy to what you did last time with energy, replace p with T and H with u. Flip those two and you get the same thing.
OK, so now we have dH/dT is equal to Cp, and we can also write, then, dH/dp under constant temperature is equal to minus Cp mu Joule-Thomson.
We have our two derivatives in terms of physical quantities, which is going to allow us, then, whenever we have a change to go back and, when we have a change where we adjust the temperature and the pressure, we'll be able to know what the enthalpy change is.
OK, now let's take two cases.
Let's first start talking about ideal gases.
The last time you saw that for an ideal gas, the energy only cared about the temperature.
It didn't care what the volume was doing.
du/dV under constant temperature was equal to zero for an ideal gas.
And by analogy, we expect the same thing to be true here, because enthalpy and energy have all this analogy going on here.
So let's look at an ideal gas.
So for an ideal gas, we saw that u was only a function of temperature.
We also have the equation of state for an ideal gas, pV = nRT.
We can write our definition of enthalpy, h is u plus pV.
This only depends on the temperature.
pV= nRT.
So we have u only depends on temperature plus nRT.
The only valuable now on this side is temperature.
Pressure and volume have dropped out.
So enthalpy, for an ideal gas, only cares about temperature.
Pressure has dropped out of the picture completely here.
So there is no p dependence here.
H for an ideal gas is only a function of temperature.
This is not true for a real gas, fortunately, but it's true for an ideal gas.
So for an ideal gas then, dH/dp under constant temperature, that has to be equal to zero.
Because temperature is constant H only cares about temperature.
and that's equal to zero.
And if that's equal to zero, that means that the Joule-Thomson coefficient for an ideal gas is also equal to zero.
We're going to actually prove this later in the course.
Right now, you're taking it for granted.
Right now we told you Joule did all these experiments and he found out that for an ideal gas, that the limit in and ideal gas case was that the eta J was equal to zero.
Therefore, from experiments, u is only a function of temperature for an ideal gas, and therefore from these experiments, we come out with delta H dH/dp is equal to zero.
The Joule-Thomson coefficient is equal to zero.
Later we are going to prove it exactly.
but right now you're going to have to take it for granted.
So, if the Joule-Thomson coefficient is equal to zero, just like we wrote, du = Cv dT for an ideal gas, we're going to have dH = Cp dT for an ideal gas as well.
dH is Cp dT.
This term goes away.
dH = Cp dT.
That's the only thing that's left behind.
So if you know the heat capacity, you know the change in temperature, you know what enthalpy is doing for an ideal gas.
This needs to be stressed that this is the ideal gas case.
Now regular gases, real gases, fortunately as I said, don't obey this.
This is important because we use this all the time for when, when we do technology.
One example of Joule-Thomson coefficient being not equal to zero for a real gas that you've like experienced is if you take a bicycle pump, take a bicycle pump, and you're pumping up your tire, you're working pretty hard, so you're getting hot.
But if you touch the valve going into your tire, which basically measures the temperature of the air going into your tire, that is getting hot, right.
So if you've got to pump that tire really a lot, then you're going to you're going to really feel a lot of heat there.
The compression of the, basically it's an adiabatic compression.
You're taking the air inside of the pump and you're compressing it.
You're doing it so fast that there's not enough time for heat to come out of the gas that's inside the pump towards the walls of the pump.
So your time scale it just fast enough that this is basically an adiabatic compassion.
Your compressing it really fast, all right, so you're changing the pressure.
You're changing the pressure, and the temperature is going up.
dT/dp is positive.
dT/dp is positive.
dT/dp is positive, well that's mu JT.
dT/dp is mu JT.
So for a real gas like air, this is a positive number.
It's not zero.
Air is not an ideal gas.
It's one simple example of -- The fact that mu JT is not zero for real gases is how we are able to liquify things like hydrogen and helium, by compressing them and pushing them through a nozzle, and the expansion through the nozzle cools the gas.
Right, in this case here it wouldn't happen if it was an ideal gas.
Or in many kinds of gas refrigerators where you push a gas through a nozzle close to room temperature, what you find is that the gas coming out on the other side under lower pressure is cooler than the gas that went through on the other side.
Real refrigerators actually work with liquids that go into gases so use the latent heat of the liquid, so it doesn't really work like the Joule-Thomson expansion.
So this is real.
This is real, unlike the Joule coefficient which is very small so that most gases have tiny Joule coefficients.
So if you do a Joule experiment, you hardly measure a temperature change.
With real gases, here you do actually measure it.
You can feel it with your finger on your bicycle tire.
OK, so we're going to see this using a Van der Waal's gas.
Let's look at a Van der Waal's gas and see what happens in the Van der Waal's gas.
Any questions, first?
What we've been talking about, the Joule-Thomson experiment, constant enthalpy process?
OK, so let's take our Van der Waal's gas.
Remember the equation of state for Van der Waal's gas is not pV is equal to nRT, but p plus the attraction term.
And then V minus the excluded volume term is equal to RT.
Two parameters, this is the attraction between two atoms or molecules in the gas phase.
This is the repulsion, not the repulsion.
This is the fact that we occupy a finite volume in space, because they're little hard spheres in this molecule.
OK, in a few weeks, you're going to find out that we can calculate dH/dp from this equation of state, and you're going to find out that dH/dp from that equation of state is proportional to b minus a over RT.
This is going to be probably a homework at some point to do this.
For now, let's take it for granted.
Let's take it for granted that we know how to calculate this derivative from an equation of state like this.
But now we're going to use that.
OK.
So since dH/dp under constant temperature is proportional to minus mu JT, then we have that dT/dp under constant enthalpy than it is related to the negative of this, a over RT minus b All right, this is how the temperature changes when you change, when you have pressure changing.
So when you do an expansion or compression, my adiabatic compression of my bicycle pump is dT/dp.
All right.
Or when I do an expansion of hydrogen or helium at low temperature, through a Joule-Thomson experiment, when I want to liquify hydrogen or helium.
I want to cool a gas with a Joule-Thomson experiment, what temperature do I have to be at?
So this tells you that you have to be careful what temperature you're at, because depending on how high, how big this temperature here is, you could either be, have a negative dT/dp, if this first term is small enough, meaning if temperature is very high, then you end up with a negative term.
If the temperature is very small, then one over the temperature is large, and the first term wins and you have a positive number.
So there's some special temperature which is going to depend on the gas where the first term is going to be equal to the second term, where Joule-Thomson is zero, or it's going to behave like an ideal gas.
So when that is the case, we're going to call that temperature the inversion temperature or T inv.
We call that inversion because on one side you end up cooling if you compress.
And on the other side of that temperature you end up heating if you compress.
OK, so there's some temperature, t inversion minus b where the gas behaves like an ideal gas.
The Joule-Thomson coefficient is zero and that inversion temperature, you can solve for it.
Conversion temperature is equal to a over R times b. OK, so when you are at that temperature, everything looks like an ideal gas, as far as the enthalpy changes are concerned.
Now if you're at the temperature which is higher than the inversion temperature, in that case here, a over RT is small compared to b, and this is going to turn out to be negative.
So if you had a high temperature, this a small compared to b.
If you're negative, which means that dT/dp at constant H is less than zero.
So that means that if you compress something it's going to cool.
The temperature rises when the pressure drops, right?
Or in this case here, if I do my Joule experiment delta p is negative, p2 is less than p1, that means that delta T is positive, right?
So in this experiment here, this side is going to heat up.
So for materials where T inversion is low, lower than room temperature, then you would end up heating up in this expansion.
You're basically expanding the gas from one side to the other and the expansion causes the temperature to rise.
If T is less than T inversion, you have the opposite case, and dT/dp is greater than zero.
So in this experiment here, delta p is less than zero.
You need to have this whole thing greater than zero.
So delta T is less than zero as well.
So if you're below the inversion temperature and you do the Joule-Thomson experiment, you're going to end up with something that's colder on this side than that side.
Ideal gas would be the same temperature.
But now, so this is where the refrigeration comes in.
So if you take a gas, and you're below the inversion temperature and you make it go through this irreversible process, the gas comes out colder from that side than that side.
So the work that you're doing to expand, to go through this experiment, ends up cooling the gas.
OK, for most gases, T inversion is much greater than 300 degrees Kelvin.
Much greater than room temperature.
For more most gases, if you do this experiment at room temperature, you end up cooling the gas, and you can cool it measurably, which is why also, your bicycle pump, you know, you push down, you compress, this is an expansion here.
You're expanding from this side to that side.
Bicycle pump you are compressing the gas, which is the opposite, you end up heating up the air in the bicycle pump in your compression.
There are two exceptions to this rule that most gases have a T inversion which is greater than 300 degrees Kelvin, and that's hydrogen.
The T inversion for hydrogen turns out to be 193 degrees Kelvin, and the T inversion for helium turns out to be 53 degrees Kelvin.
And so when, there's a lot of liquid helium that's being used on campus.
We use a liquid helium.
And so in order to make a liquid helium, you can't take helium at room temperature and do this, because if you did, you would just heat it up, because the room temperature is above the inversion temperature, so Joule-Thomson would heat up the helium.
So you need first to take the liquid helium and cool it below 53 degrees Kelvin before you can do the Joule-Thomson to cool it even further to make liquid helium.
So you have to do it in stages.
You take your room temperature liquid helium, and you cool it with liquid nitrogen to 77 degrees Kelvin, the new, you're not quite there yet, unfortunately right?
Then you take hydrogen you cool it would liquid nitrogen to 77, then you can use your hydrogen gas.
Do a Joule-Thomson hydrogen gas which you first cool with liquid nitrogen.
Liquid nitrogen, 77, that's below 193, so you can do Joule-Thomson on hydrogen, cool the hydrogen to below 53, then use your cold hydrogen to cool the helium, and then you can do the Joule-Thomson on the helium to cool it further until it liquifies.
So that's what you do when you make liquid helium.
All right, any questions on this lecture or any of the concepts that we talked about?
The last thing that we're going to do today then is to look at a relationship which is going to turn out to be very useful.
It's a relationship for ideal gases which relates the heat capacities at constant pressure and constant volume.
Cp = Cv + R. This is very useful because often you just have lots of tables of Cp's but sometimes you want to know what the energy change is going to be for the ideal gas, and you know that du is Cv dT not Cp dT.
So, you need to know what Cv is, and if this is true always, then there's a very easy way to go from one to the other.
We're going to do it two ways.
Today we'll do the first way, and then next time we'll do the second way.
The first way is just to turn the crank on the math, and the second way is to do a little bit more imaginative about the process.
OK, let's just turn the crank on the maths.
What do we know about the Cp and Cv?
Well Cp, we already know how to relate it to dH/dT through the slope of the enthalpy, and we also related Cv to the slope of the energy with respect to temperature under constant volume.
And we also know that H is u plus pV.
Right, H is u plus pV.
So we're going to take the derivatives of both sides of this equation here, by, with respect to temperature, keeping pressure constant.
So we have dH/dT keeping pressure constant, is du/dT keeping pressure constant.
Got to keep track of what's being constant, kept constant, plus dPV/dT, keeping pressure constant.
dH/dT keeping pressure constant, that's just Cp, and then we have du/dT keeping pressure constant.
The p is constant here, comes out of the equation, so we have p and then we have dV/dT well it's an ideal gas, an ideal gas for dV/dT for an ideal gas is equal to R over p because pV is equal to RT, right.
So dV/dT is Rp.
So dV/dT is times R over p, the p's are going to cancel out here.
OK, so now it's very tempting at this state to say, oh there's the answer right here.
du/dT is Cv.
There I have it Cp is equal to Cv plus R, right?
But even though it looks like that's the right way to do it, it's actually not right because it turns out to be right by sort of by accident.
But here you've got pressure constant.
du, this is du, not H here.
du/dT is only equal to Cv when the volume is constant, not when the pressure is constant.
So if you're going to turn the crank on the math correctly, you're going to have to change this p into a V somehow.
Because this isn't Cv mathematically speaking.
We don't know what it is yet.
In order to change this from a p to a V, you have to use the chain rule.
So let's use the chain rule.
And then we'll be done.
OK, so u is actually a function of temperature and volume, which in this case here could be a function of pressure and temperature.
So if we want du/dT under constant pressure, you have to use the chain rule.
There's the pressure sitting right here.
It's going to be du/dT under constant volume, plus du/dV dV/dT under constant pressure.
All right, chain rule.
du/dT constant pressure is the direct derivative with respect to temperature here, which is sitting by itself under constant volume, keeping this constant but there is temperature sitting right here too.
That's where that term comes from, du/dV dV/dT.
Now, for an ideal gas, du/dV under constant temperature is equal to zero.
It doesn't care what the volume is doing.
It only cares what temperature is.
If temperature is constant, there's no change in energy.
For an ideal gas, this is zero.
It's not zero for a real gas.
Right, so this whole term disappears and for an ideal gas, it turns out that du/dT constant pressure is equal to du/dT constant volume, but this is equal to Cv for an ideal gas.
It wouldn't be true for a real gas, and this is a common mistake that people make for real gases to equate this.
This is only true for an ideal gas.
Since it's true for an ideal gas, then we can go ahead and replace this with Cv, and then we have Cp with Cv plus R, which is what we were after.
OK, next time we'll do the other way of getting to the same answer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so we ended up last time, we talked about Joule-Thomson expansion, which is an irreversible expansion through a nozzle, through a porous plug, constant enthalpy expansion.
And I want to make sure everybody figured out that it really was an irreversible expansion.
Anybody have any questions on that, because when we voted, the majority of the class thought it was reversible Yes
Just a quick question on some of the constraints, like isothermal, isobaric, isovolumetric expansion.
For isothermal expansion, that means that delta u does not change, but delta q is equal to delta w?
So the question was, for an isothermal expansion, delta u does not change, therefore, dq is equal to dw.
The answer is that's true only for an ideal gas.
For a real gas, that won't be true.
Ideal gas only depends on the temperature, the energy only depends on the temperature.
Therefore, if it's isothermal, the energy of the ideal gas doesn't change.
For a real gas it depends on more than the temperature
Are there any other constraints similar to that [INAUDIBLE]
So, for an ideal gas, the isothermal is the easy one because the energy doesn't change.
If you have an isobaric you're going to have to calculate where the energy changes, and that's a calculation that's likely on the homework and very like on an exam as well, too.
In fact, we're going to do some of that today, OK, calculate delta u.
Any other questions?
Good question.
OK, so we ended up last time showing that for an ideal gas, that there was was a relationship between the heat capacity under constant pressure, and this is the molar heat capacity, that's why we have the bar on top, and the molar heat capacity for constant volume path.
They're related through the gas constant.
OK, this is only true for an ideal gas, and we went through that mathematically where the, with a chain rule.
And I wanted to do it a different way which is a little bit convoluted, but it introduces the idea of a thermodynamic cycle, and it's something that we're going to use a lot in the class.
Because anytime you're working with the function of state, and you're trying to connect two points together in a p-V or a T-V diagram, then it doesn't matter which way you're connecting those two points, as long as you're dealing with state functions it doesn't matter.
It only matters if you're looking at the work or the heat right.
And so sometimes, when you want to calculate what the energy is at the endpoint, the path of the experiment that you're doing may not be an easy path to calculate.
You may have to find a different path which is an easier one to calculate, and it doesn't matter which path you take, because in the end all you care is what the energy is at the end.
so we're going to use this concept of the path to go from the initial point to the end point.
As long as you're dealing with state functions it doesn't matter to get at the same results here.
So we're going to start with a mole of gas, at some pressure, some volume, temperature and some mole so V, doing it per mole, and we're going to do two paths here.
We're going to take a constant volume path.
So V is constant and we're going to change the pressure to some, a little change in pressure, V is constant here, and then temperatures is also going to change.
So this is a path, which is either a temperature change, with the pressure changes at the same time, going up or down, constant volume.
And since the temperature is changing, clearly Cv is going to have something to do with this.
So constant volume path, where the temperature is changing.
So somehow this is going to allow us to get Cv.
Cv is going to be related to this path.
And if I draw a diagram on a T-V diagram of what I'm doing here, so there's temperature axis.
There's the volume axis here.
I'm starting with some V1 here.
And this is going to be here V plus dV.
So I'm starting right here.
This is my initial point.
And the path that I'm describing then, let's assume that we're raising the temperature up is this path right here.
OK, this is my constant volume path going from T to T plus dT.
OK, we'll call this 1 here.
All right, now I want to make a path where out of that path, C sub p is going to fall out.
So that's going to have to be a constant pressure path.
Temperature is going to change but pressure is going to be constant.
So what I can do is I can take my initial point here and create a new, different path, where, which is going to be a constant pressure path.
So at the end of this path the pressure is constant.
The volume is going to change though, to some new volume dV and the temperature is going to change also and I'm going to make it the same endpoint temperature as the endpoint temperature here.
And out of this path, since the pressure is constant but the temperature is changing, I'm hoping that C sub p will fall out of that path of that calculation.
All right?
This should give us something about C sub p. So this will give us something about C sub v. This will give us something about C sub p. Now these two endpoints here are different.
They have a different pressure, a different volume.
They do have the same temperature though.
So the connection between this endpoint here, and that one here.
Same temperature.
It's going to be an isothermal path that's going to connect them.
So I'm going to connect them in an isothermal way, and as, since I'm dealing with an ideal gas, and as we saw from the first question here, isothermal always means delta u equals zero.
So I'm going to write that immediately here, delta u equal zero, because I know the answer to that path here.
All right, so let's go again what our paths are.
Path number 1 I'm going straight up in the V-T diagram.
Path number 2 on my diagram it's a reversible, this path number 2, it's a reversible constant pressure path.
So I'm going to be keeping the pressure constant, but my volume is going to go up.
My temperature is going to go up.
Basically I'm putting heat in this system.
That's going to be number 2 right here.
It's going to be the same temperature as before but the volume is V plus dV now.
That's 2, and then, so I'm heating up the system in this path here, and then to connect the 2 endpoints here, I'm going to connect them through a constant temperature path.
Basically I'm going to cool down the system so that the -- no, I'm not going to cool down, I'm going to compresses the system back to the smaller volume V1 and the constant temperature conditions.
So that's path number 3 here.
And I'm only going to worry about energy in this case here.
Because I know energy doesn't care about the path.
Energy only cares about where I am on the diagram.
And I know energy is related to Cv through Cv dT etcetera.
So if I worry about energy I have a pretty good chance of extracting out these heat capacities, right, and I don't have to worry about exactly which path and I can really mix things up.
So let's start the process.
So the first path then, the first path, constant volume constant V, so I'm going to, again, let's just worry about energy.
So du for this first path here, du for path 1, and write dq plus dw what we usually write.
Now it's constant volume when the volume is constant.
There's no work being done so I can immediately ignore this dw here.
Constant volume dq = Cv dT.
Now it's an ideal gas.
So in some sense, I did too much work by writing this, because I knew already du = Cv dT.
I could have written that down immediately, but I just went through the process of writing the first law, what's dw, what dq, Cv dT, etcetera.
I'm just writing what we already know basically.
Let's look at path number 2.
Path number 2.
OK, let's look at the dw, du again.
du for path number 2. dq like the first law down, dq plus dw, then -- what's dq?
Well it's a constant pressure path.
dq is related to the temperature, the change in temperature.
dT through the heat capacity, happens to be constant pressure, so this is exactly what we want.
Cp, I forgot to put my little bar on top here because it's per mole Cp dT that's my dq here.
The path is constant pressure.
And then the dw.
Well there's a change now in volume and pressure.
So I need, well the pressure is constant, but there's a change in volume.
Minus p dV is the change, is the work, right?
So I can write that down minus p dV for that path there.
OK, now but it's an ideal gas, so I know something about the relationship between dV and dT, because I've got dT here the dV.
I don't want to have so many variables around.
I've got three variables, T, p and V. I know I only need 2, so I can relate dV to dp through the ideal gas law.
P dV is equal to R dT.
pV = RT for 1 mole, so I just take dV here.
dT here because the pressure is constant, so dV is equal to R over p dT.
Then pressure is constant.
So I can instead of the dV here, I can insert this R over p dT.
The p's cancel out, and now I have Cp dT minus R dT.
This is equal to Cp minus R dT.
All right, let's complete the cycle now, the path number 3 here.
Path number 3 is a constant temperature path, and I already wrote the answer.
Constant temperature isothermal delta u is zero.
There's no thinking involved.
It's one of these things like I mentioned last time, if I tell you delta H in the middle of the night what you say, q Cp for reversible process.
Same thing here.
isothermal process, what do you say?
Delta u is zero, right?
It should be completely second nature.
So delta u is zero here.
Delta u sub 3 is, for an ideal gas.
I gotta put that constraint in there for an ideal gas.
Delta u is equal to zero.
So now I'm back at the same place.
I'm back here right.
So path number 1 went from i, let's call this path up here.
went to f, and this is how much energy change there was.
The sum of path number 2 and path number 3 get me to the same place, so the energy change by going through this time path, this intermediate point here back all the way to final state should be the same the red path.
So what I'm saying is that du through path 1 should be equal to du through path 2 plus du going through path 3.
All right, because energy doesn't care about the path.
It only cares about the end points.
OK, so now let's plug what is du through path 1?
It's Cp dT and the du through path 2 is Cp minus R dT and then du through path 3 is zero.
It's the isothermal process.
The dT is cancelled out and voila I have my answer.
Cv is equal, oh Cv plus R is equal to Cp whichever way you want to do it, it's a relationship that we had up here that we wanted to prove.
Now, I wanted to go to through this just to go through one cycle quickly because we're going to be doing these all the time, and the importance of the fact that the path doesn't matter, and you can always connect things together in a way, whatever you want.
Whatever is the easiest.
If you want to calculate a change in delta u.
Now if you look at my derivation here, there's one spot where I could have just stopped and proven my point without going through the whole thing.
So if you look at path number 2 here, the constant pressure path du is Cp minus R dT, du is Cp minus r dT for that path.
What is du for an ideal gas?
It's -- what else do we know that du is always equal to for an ideal gas?
Cv dT right.
So I could have written right here immediately equals Cv dT, and that was the end of my derivation.
I could have done that.
It would've been easier, but I wanted to go through the whole path right?
OK, any questions?
Cool.
All right, so we're going to be doing more thermodynamics processes.
We're building up to entropy and to engines, Carnot cycles, etcetera.
And so we need to build our repertoire of knowledge in manipulating these cycles like the one I drew on the board that's hidden now You've already seen the simple process, which is the isothermal process, and you've seen how to functionalize, or what the function is that describes the isothermal expansion.
Let's say we start from some V1 and p1 here, so high pressure, small volume and we end up with a high volume low pressure, under constant temperature condition.
So there's some path, isothermal, so T is constant.
And we know the functional form for this path, it's the ideal gas law, pV = RT, pV = nRT.
T is constant, so the functional form here is pV is equal to a constant.
All right, or p is equal to a constant divided by volume.
If you want to write a function that describes this line here, it's pressure as a function of volume related to each other with this constant.
So we know how to do this.
So this allows us to calculate all sorts of things to get these functional forms.
That's the isotherm here.
We also know how to calculate the work.
If it's a reversible process, we know the work is the area under the curve.
w reversible.
in this case here let's see this is it's a pressure is going down.
It's an expansion, so we're doing work against the environment, or to the environment.
So work is leaving the system.
So minus w reversible is going to be a positive number.
V1 to V2 and RT -- you've done this already last time or a couple of times ago and you end up with minus nRT log p2 over p1, OK?
So minus w reversible, we're doing work to the environment.
This should be a positive number.
p2 is less than p1, so p2 over p1 is less than one, log of something less than 1 is negative times negative.
So in fact this is indeed a positive number so we didn't make a mistake in our signs.
You always want to check your signs at the and because it's so easy to get the sign wrong, and I was hoping that it was right here because I wasn't sure, but it turned out to be right.
Always check your sign at the end.
All right so we know how to do this.
Now we are going to do the same thing with a different constraint.
Our constraint is going to be an adiabatic expansion or compression.
Adiabatic meaning there's no heat involved, and we're going to see how that differs from the isothermal expansion and compression.
OK, so this is the experiment then.
We're going to take a piston here which is going to be insulated.
It's going to have some volume, temperature to begin with, and then we're going to do something to it.
We're going to change the pressure above, right now there's a p external, which is equal to p on the inside.
OK, we're going to do this reversibly, which means we're going to slowly change the external pressure very, very slightly at a time, so that at every point we're basically in equilibrium, until the pressure reaches a new smaller pressure p2.
So that's going to results in an expansion where the new volume new temperature new pressure and an external pressure, which is p2 which is a smaller pressure.
It's really important that this, that you remember that this is the reversible case.
We're going to also look at the irreversible case briefly.
So on the p-V diagram, then, there's a V1 here a V2 here, a p1 here a p2 here.
This V2 is going to be different than this one here.
A priori we don't know what it is.
We don't know if it's bigger or smaller.
We kind of feel like it's going to be different because it's a different constraint.
This is isothermal.
This is adiabatic, there's no heat.
So there's going to be a line that's going to connect the initial point to the final point, and that line mathematically is not going to be the same as this one here.
And our job is to find out what is the mathematical description of this path, this line in p-V's case that connects these two point.
And again, I want to stress this is a reversible path.
If it was non-reversible, I would be allowed to put an initial point and a final point, but I wouldn't be allowed to put a path between them like this, connecting them together.
Because if it's irreversible, it's very likely that I don't know what the pressure inside the system is doing while this is happening.
It could be very chaotic.
The pressure could not be, might not even be constant throughout the system.
It could have eddies and all sorts of things.
So for an irreversible process, I wouldn't really be allowed to put a path there.
I can do that because it's reversible, and I can get a functional form out.
What we're going to find in this process is that the functional form from this is that p is related to the volume through a constant, but instead of being V to the first power as it was for the isothermal, it's now a V to some constant gamma.
What we're going to find is that gamma is greater than 1.
And if gamma is greater than 1, and you're changing the pressure to the same final point, what we're going to find then is the volume that you go to is going to be a smaller volume than for the isothermal.
We're going to go through these points, so don't worry about getting at all straight right now.
All right, so let's do this.
Let's try to show this.
Let's try to show that in fact this is the right functional form, that pV to the gamma is equal to constant is the right form that describes the path in an adiabatic expansion or compression.
So let's do the experiment.
Let's write everything that we know down.
This is where, the way that we should start every problem set or ever problem.
It's writing everything we know and what it means.
OK, so adiabatic.
All the words mean things mathematically.
Adiabatic means dq equals zero.
Let's write that down.
dq equals zero.
Reversible.
Reversible means that if I looked at dw it's minus p reversible.
It's minus p external dV, that's what it is, but in the reversible process p external is equal to p. That's the only time you can set p external equal to p is if it's reversible.
So this is the external pressure.
This is the internal pressure of the system.
You can only do this because it's reversible.
What else do I know?
It's an ideal gas.
So I can write du is Cv dT and pV is equal to RT.
OK.
So what else can I write?
Well from the first law, du is equal to dq plus dw, and I wrote down everything I knew at the beginning here.
I wrote dq equals zero I wrote what dw was.
So du is just minus p dV.
Now from the ideal gas, I have another expression for du, Cv dT.
So now I have two expressions for du.
That's a good thing, because I can them equal to each other.
So if you get these two guys together you get Cv dT is minus p dV.
So we're connecting temperature and changes in temperature and changes in volume together.
OK, now we don't really want this p here.
This is going to be a function, I only need two variables, and here I've got three variables down.
So I know p is going to be a function of T and V, and I know how to relate p to T and V because I know the ideal gas law.
So instead of p, here I'm going to put nRT over V. or RT over V bar.
So now, this expression becomes Cv dT over T is equal to minus R dV over V. And I also did a little bit of rearrangement.
Yes?
[INAUDIBLE]
Well, let's see.
du is, for an ideal gas, it's always equal to Cv dT.
Constant volume process tells you that if you have a constant volume, then dq is Cv dT.
This is what the constant volume path tells you, that you can equate Cv dT with dq.
The ideal gas tells you that this is always true here.
It's an ideal gas in adiabatic expansion dq is equal to zero.
Adiabatic expansion means you can't use this.
dq is zero.
The that is adiabatic.
It's not constant volume.
You're allowed Cv comes out here for this adiabatic expansion, which is not a constant volume only because this is always true for an ideal gas.
We didn't use a constraint that V is constant.
Just use this as an ideal gas here.
OK, so we've got dT here dV here.
We want to get rid of derivatives.
We want to integrate.
So let's take the integral of both sides, going from the initial point to the final point.
We've got Cv integral from T1 to T2, dT over T is equal to minus R from V1 to V2 dV over V. OK, we do take that integral, and that leads us to Cv log of T2 over T1 is equal to minus R log of V2 over V1, and now we know how to rearrange this, because the Cv times a log of T2 over T1 is the same thing as the log of T2 over T1 to the Cv power, right.
So by rearranging these expression of the logs here, this is the same thing is writing log T2 over T1 to the Cv power is equal to log of V1 over V2 to the R power.
And I've reversed the fraction here because of the negative sign here.
OK, get rid of the logs, and you get that T2 over T1 is equal to V1 over V2 to the R over Cv by taking this Cv and bringing it on the other side here.
OK, so now we've got the initial and final points, a relationship between the temperature and volume for the initial and final points.
Eventually that's not what we want.
Eventually we want the same relationship in the pressure and volume.
We want a relationship in p-V space, not in T-V space.
So we're going to have to do something about that.
But first, it turns out that now we have this R over Cv.
and remember, we have this relationship between R Cv and Cp here, which is going to be interesting.
So let's get rid of R in this expression here.
So R over Cv, R is Cp minus Cv.
We proved that first thing this morning, divided by Cv so this is Cp over Cv minus 1 and because I know the answer of this whole thing here, I'm going to call this thing here, Cp over Cv.
I'm going to call it gamma.
Because I know the answer, right.
I know the answer.
I'm going to call this gamma.
By definition I'm going to define gamma by Cp over Cv by definition.
OK, so that means that this is really instead of R over Cv.
it's really gamma minus one.
So now I've got this gamma placed in there, gamma minus 1, so I've ben feeling a little bit better.
I've gotten V to the gamma power almost to the gamma power, it's minus 1 here.
But it's almost right and I'm looking for something like this here, V to the gamma power pV to the gamma power is a constant to describe that line.
So I just got to get rid of the temperature now.
Somehow I've gotta get rid of the temperature and make it pressure instead.
OK, so let's -- let me get rid of this here, let's use the ideal gas law to get rid of the temperature.
So we have T is pV over R. So now if I look at my V1 over V2 to the gamma minus 1, that's T2 over T1.
I'll replace the ideal gas law for those two T's.
That p2 V2 over R, and then I have p1 V1 over R, the R's cancel out.
Well let's see, there's a V1 over V2 to the gamma minus 1.
There's a V1 over V2 here or V2 over V1.
That's going to get rid of this minus 1 here.
Then I'm going to get, be left with a p2 over p1.
So this guy here gets rid of this, and I have my answer as V1 over V2 to the gamma power is equal to p2 over p1.
Or I can re-write this as p1, V1 to the gamma is equal to p2 V2 to the gamma.
That means that p1 and -- where I am, the point number 1 and point number 2 were completely arbitrary.
They happen to be on the path.
So any point I pick on that path will be equal to p1, V1 to the gamma.
So if I take p times V to the gamma, anywhere on the path, it's going to be equal to the same relation from my first point.
This is going to be true for any point on the path.
As long as I'm on the path, pV to the gamma will be whatever it was for the first point, which is going to be some sort of constant.
All right, so I've proven my point.
I've proven what I was trying to do, which was to show that on this adiabat, everywhere on the adiabat, if you take a functional form that relates p and V together, it's going to have this relationship with gamma as related to the heat capacities.
Gamma is Cp over Cv.
OK, now there's more we can say because we know that's Cp is related to Cv through this R here, and R is a positive number.
So Cp is always bigger than Cv.
In fact, if it's an ideal gas Cv and Cp are well defined numbers, it turns out.
If you have a monotomic ideal gas, OK, so an atomic version, not a molecular ideal gas, then it turns out that Cv is equal to 3/2 R, and therefore, Cp which is just R plus this is five halves R, which means that gamma for a mono atomic ideal gas is 5/3.
So if you have a gas of argon atoms, you know what gamma is.
This is something that you're going to prove in statistical mechanics, and so we're not going to worry about where this comes from.
We're just going to take it for granted.
All right, so gamma is for ideal gas, is bigger than one.
In fact we have a number for it if it's an atomic ideal gas.
So what it means then is if I look at if this relationship here, gamma is bigger than 1, so gamma something bigger than 1 minus 1.
This is, so this is positive here.
It's positive.
So if I have an expansion where V2 is greater than V1, so V2 is greater than V1, so this is a number which is less than one, then I expect then T2 is going to be less and T1, T2 is less than T1.
OK, I have this number here to a power, which is positive, a positive power.
And its number is less than one.
This is going to give me something which is less than one.
So T2 is less than T1, and I have a cooling of a gas.
So whenever you have an adiabatic expansion, the temperature cools, it gets colder.
It gets colder, and if I look at my graph here, then if the volume of the expansion, if the temperature of the adiabatic expansion was colder, that means that -- this is the wrong graph here.
If I want to compare it with the isothermal expansion, which is sitting here.
All right, so gamma, the gas is cooling so V2 is going to be less than it what would be if the temperature kept constant.
So finally, on the same graph, I put down what an isothermal expansion would be, if my final pressure is the same pressure, my volume for an isothermal expansion will be bigger than for an adiabatic expansion.
So when I expand this gas adiabatically and it cools down, why do you think it might cool down?
It cools down because I'm doing work through the environment, but the energy has to go somewhere, right?
There's an interplay between the energy inside the gas which is the heat energy which is allowing me to do all that work to be outside, and so I'm using up some of the energy that's inside the gas to do the work on the outside.
And here, in an isothermal expansion, The bath is putting back the energy that the gas is expanding or using to expand, and so the energy is flowing back into the gas through the environment in the isothermal expansion.
In the opposite case, if you have a compression, then it's the opposite of expansion.
Compression you're expected to heat up, right?
So in the compression, you're going to have V2 is less than V1.
So that T2 is going to be greater than T1.
That's going to be your heating up of the gas.
And that really is the bicycle pump.
That example I gave last time with the bicycle pump was not quite the right example.
But this time this really is the bicycle pump.
That you're pushing really hard on it.
So you start with the bicycle pump at one bar, let's say, right?
And you compress the air in the bicycle pump.
You do it so quickly that the heat flow between the inside of the bicycle pump and the outside is too slow compared to the speed at which you compress.
But you don't compress it so quickly that you're not in there reversible process so you compress that if you were to feel the temperature of the air and you can feel it through the nozzle gives you an idea of the temperature of the air inside your bicycle pump.
You'll feel that it's warmer.
You've just done an adiabatic compression of the ideal gas, you can pretend there is an ideal gas.
And that gives rise to the heating that you can actually measure.
OK, any questions on this part here.
All right, so we've just gone through the reversible adiabatic, and I'm not going to do this in detail, but I want to go through the irreversible adiabatic fairly quickly, and see where the difference is here.
OK, so now we're going to do the same kind of experiment, but irreversibly.
An irreversible adiabatic.
OK, so the experiment is going to be to take my cylinder now, and the external pressure and the internal pressure are not going to be in equilibrium with each other during the process.
I'm going to start with my cylinder here, I'm going to put little pegs here so it doesn't fly up.
There's going to be some external pressure.
I'm going to set that equal to p2.
There is going to be an internal pressure where p1 is less than p2 and there's V1 and T1 here.
The whole thing is going to be insulated.
Then I'm going to release these little pegs here and my piston is going to shoot up now because p2 is less than p1.
So p2 is less than p1, the external pressures is less than the internal pressure.
So it's going to shoot up until the internal pressure and the external pressure are in equilibrium.
So they're both p2.
external is p2 and I have p2, V2, T2, on the other side.
OK, so in my diagram now, I have p1, V1 for my gas.
p2, V2 for my gas, so I know that I'm starting here and I know that I'm ending here, but I can't connect this path here.
I don't know how to do that.
It's an irreversible expansion.
I don't know what the pressure is doing in there, doing that expansion.
It could be quite chaotic.
It could be non-spacially constant.
OK, so but you still know a number of things.
We're now going to be able to write you know dw is minus p dV, but we're still going to be able to write a bunch of things.
We're still going to be able to write that it's an adiabat so that dq equals zero.
We're still going to be able to write dw instead of pV where p is the internal pressure.
dw now is actually much easier.
It's minus p external.
Well p external is actually p2, so that makes it actually easier.
dV we're still going to write that it's an ideal gas.
So du is still going to be equal to Cv dT, and we're still going to be able to use the first law, all these things don't matter where the path is.
Still know that du is dq plus dw.
dq is zero.
dw is minus p2 dV.
So this is what we know, just like what we wrote there.
We write what we know.
I'm not going to turn the crank here.
I'm going to give you the answer.
OK, you use the ideal gas law, etc., then you get a relationship that connects the pressure and the temperature, like here we got a relationship that connected the temperatures and the volumes together.
You can get a relationship that connects the temperature so this is T2 times Cv plus R is equal to T1 on Cv plus p2 over p1 times R. This is going to be the connect, what connects the pressures and the temperature.
And p2 is less than p1, so this number right here is less than 1.
So and Cv plus R, Cv plus something less than R, so T1 better be bigger than T2.
OK, so T2 is less than T1.
Same qualitative result as before.
You have an expansion.
It's and adiabatic expansion.
You get a cooling of the gas.
OK?
Now here's a question for you guys, which we're going to vote on, so you better start thinking about it.
Is the temperature T2 in this process smaller or larger than if I were to do the process reversibly with the same endpoint pressure.
So now, instead of having p external equals to p2 here, I put p1 and I let this whole thing go reversibly.
And I compare T2 irreversible to T2 reversible.
They're both less than T1.
They're supposed, there's a cooling happening in both cases.
One is going to be colder than the other, maybe.
Maybe they're going to be the same.
I don't know.
I'm going to let you try to figure that out qualitatively without doing any math.
See if you can figure it out.
OK, so which is colder?
I'm going to let you think about it for about thirty seconds or so.
You can talk to each other if you don't, can't figure it out.
All right, let's take a vote.
How many people say that T2 irreversible is colder than T2 reversible?
One, two, don't be shy, three four, anybody else, five.
OK, I've got five votes here.
Professor Nelson?
Oh, I'm abstaining
You're abstaining, OK, good choice.
OK, what about T1 cooler than T2?
OK, I've got a lot of votes here.
So the majority of the people, so this is much bigger than five.
OK.
So remember last time the majority was wrong, right?
The majority was wrong last time.
That doesn't mean that the majority is wrong here.
It could be right.
Let me give you another piece of information here that you already know.
Minus w irreversible, this is the work which is done to the environment by the system, minus w irreversible is always smaller than minus w reversible.
You learned that a little while ago.
OK, so this is a little hint.
Let's vote again.
Let's think about it for ten seconds why this could be interesting and relevant, and I want to ask if anybody changed their mind.
Anybody change their mind, change their vote?
Yes You want to change your vote?
To which one?
From T2 reversible is greater than T2 irreversible, saying that T2 reversible is [UNINTELLIGIBLE]
So the question then, which one is colder?
T2 reversible should be colder
T2 irreversible should be colder?
Yes
All right.
OK.
I'm going to keep you there then, the majority.
Yes?
I'm switching to [INAUDIBLE]
You're switching this way here?
So now we have six here.
Anybody else want to switch?
All right, let's take the example of, the extreme example, let's go to the extreme example where p external is really small.
And I have this adiabatic expansion where p external is really small.
it's kind of like the Joule expansion, an ideal gas.
What happened to the temperature in a Joule expansion in ideal gas?
Anybody remember?
Anybody remember?
Joule expansion, this is where we proved that delta u was only dependent on temperature.
Bob Field taught that lecture.
Nobody remembers?
Well, all right, delta u is equal to zero for that.
What happened to the temperature in that expansion?
It's an adiabatic expansion.
Well, we'll revisit that.
Let me ask you another question here.
So w, the work is less for the irreversible process than the reversible process.
There's less work done to the outside.
So there's less energy expanded by the system.
The energy expanded by the system is smaller for the irreversible process.
It's not doing as much work.
It's not pressing against as much pressure.
Right?
p external equals p2 is less than p external equals p internal.
So it doesn't have to do as much work.
It doesn't have to expend as much energy, therefore, the majority in this case is right.
The temperature is not going to cool as much for the irreversible process.
The reversible process is going to be colder because it has to expend more energy.
It's going to have a higher pressure.
The pressure is going to decrease along the way, but it's going to have to do more work because of this right here.
So the majority in this case is right.
Now the Joule expansion, the temperature doesn't change, T equals zero.
eta sub J, the Joule coefficient for an ideal gas, is equal to zero and eta sub J was dT/dV under constant energy.
All right, any questions?
Yes?
[INAUDIBLE]
The only -- a priori, that's true.
But it's hard to imagine an irreversible compression the way we can just imagine an irreversible expansion.
You have to do it awfully fast to get your irreversible compression.
I don't know how to do it really.
I think it's, but it's more complicated, but you're basically right, basically right.
That's why we always talk about irreversible expansion, because it's easy to write down an irreversible expansion.
A compression is a little bit more complicated.
Any other questions?
OK, great.
Monday Professor Nelson who is sitting here will be taking over for a few weeks and then you'll see me again after that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, today we're going to continue with the lecture that was started last time that had the scintillating and descriptive title "some thermodynamic processes".
And we're going to continue along those lines, and really there a couple of things that I want to do to follow up what you saw last time.
One is that you've got introduced and led through a couple of elementary thermodynamic cycles.
And I want to do a little bit more of that today.
And one of the -- there are really two reasons.
One is that as you saw last time doing a thermodynamic cycle can be extremely useful, because if you have one path along which some change occurs, but along which it's not necessarily easy to calculate all of the thermodynamic changes along such a path, it may be irreversible, lots of variables may change.
It can be easier instead of looking at the complicated path to devise a cycle that involves several easier steps, each one of which is facile for the calculation of what happens to the thermodynamic properties.
So in that case, it's much easier to go through several elementary step, and then the one complicated path you know because going around the entire cycle state functions, you know that they won't have any change.
So in that sense, going through a thermodynamic cycle can be useful because it helps you calculate thermodynamic qualities.
But the other reason to go through the thermodynamic cycles and really to develop great facility with them is because there are just an awful lot of things in nature and things that we build that run in cycles, where we want to calculate the thermodynamics, right.
If we build an engine for a car or anything else, it almost always is going to have some key element that's operating in a cycle, otherwise it won't keep going, right.
But it's not just engines that we might build.
It's biological engines, right.
Things that either produce or consume biological fuel to do different sorts of processes.
Also, of course of key importance to understand what the, in that case, biological thermodynamics are.
And again, there'll be key elements that run in cycles and understanding those can be extremely important in understanding how cellular function works.
OK, so what I'd like to do is go through one cycle, and then I'll suggest another for you to work through on your own.
So consistent with the title of the lecture, we've got a section from your last lecture notes entitled "some thermodynamic cycles".
And what we're going to do is go through a cycle, please note these office hours and locations, which I think wasn't specified before.
So we're going to go through a thermodynamic cycle, and here's what I want to calculate when we do this.
Delta u, delta H, familiar state functions, changes in their values, q, w, heat and work.
And I also want to introduce one new function.
I won't give it a name yet, but it's going to have the unusual form dq over T. OK, we'll call it special function.
It's so special that we'll call this quantity dS.
OK, we'll just go through this on a whim, and this is going to be foreshadowing something that'll take on tremendous importance in subsequent lectures, but I'm going to use the opportunity now to just introduce the behavior of it in going around a cycle.
If that's for me, tell them I'm busy please.
OK, so let's look at the cycle we'll go through.
We're going to look at pressure volume changes, similar to what you saw before.
This time, we're going to have initial and final volumes, V1 and V2 and different pressures where we might end up.
And we're going to look at two ways to go to a final volume V2.
One of them is going to end up at pressure p2 and the other is going to end up at pressure p3.
One of them is going to be a reversible isothermal path.
This is going to be our path we'll label A, reversible isothermal which is to say constant T, right?
And we're going to start at temperature T1, so since it's constant temperature, this is going to end up at temperature T1.
And then we're also going to consider a reversible adiabatic path.
This'll be our path B, reversible adiabatic.
And then we'll close this circle, this cycle.
This is going to end up at a different temperature by the way.
You saw this last time in a slightly different way.
Last time what you saw is we compared isothermal and adiabatic paths that ended up at the same final pressure, and what you saw is that therefore, they ended up in different final volumes.
So this is just a little bit different from that.
So this is going to end up at a different temperature, we'll call it T2.
And then we're going to close the cycle.
And so this is a constant volume path then.
This is path C, constant volume.
OK?
So now let's go around the cycle and just compare notes on what happens to the thermodynamic quantities as we do that.
So here's path A, it's isothermal.
And I didn't specify, but let's make sure to do so, we've got a, it's going to be an ideal gas.
OK, so A is constant temperature.
What does that mean for delta u in path A?
Zero right?
Ideal gas, no temperature change, right has to be zero because da is Cv dT for an ideal gas.
dT is zero, there's no temperature change.
OK, how about delta H, zero Cp dT for an ideal gas.
dT is zero.
All right, now, it's reversible.
So that means we can immediately write down an expression for the work.
Since it's reversible, it's negative p dV right?
And since it's an ideal gas then we can replace p, right, with RT over V, right?
We want to do that because we have too many variables here.
We've already got dV we'll get rid of p as an additional variable and replace it with V which is already in here.
So it's minus R T1 dV over V, right?
I can put in T1 because we're a constant temperature.
so now we can just integrate straight away and find that the work in path A it's just the integral of that quantity minus integral of R T1 going from V1 to V2 dV over V. so it's minus R T1 log V2 over V1.
All right, so let's just do a reality check here.
The way I've got this drawn, the volume is going up in the process.
It's an expansion.
So is this system doing work on the surroundings, or are the surroundings doing work on the system?
Somebody say it real loud.
Yes, the system is working on the surroundings, right?
It's pushing out against them.
Work is defined as the work that the surroundings do on the system.
So this is the negative number, right?
V2 is bigger than V1.
This is positive.
This whole thing is negative.
All right, q has to be just the opposite of this because we've already figured out that there's no change in u.
This is just q plus w. There's w, q has to be R T1 log of V2 over V1.
OK, so that means heat is being imparted to the system, right, from the surroundings in a manner that compensates exactly the amount of work done.
OK, All right, so these are the thermodynamic quantities that you're familiar with already.
Let's just quickly look at our special function.
It's not going to be hard to calculate because it's a constant temperature path, so that T in the bottom is just T1, right.
You've already looked at q, right.
So dq A over T1 it's just R log V2 over V1 right.
So there's our special function for this particular path.
OK, so that's path A.
Now let's compare to path B.
So if path B is an adibat, what's zero?
q, it's adiabatic, that means there's no heat exchanged between the system and the surroundings.
So what happens, q B is zero.
There is a change in temperature, right?
We've got, if we say it's a mole of gas, it's going from p1, V1, and T1 to one mole of our gas at p3, V2, and T2.
So unlike before now temperature is going to change.
So that means that delta u isn't zero this time.
du, it's an ideal gas.
So this is Cv dT and of course we can just integrate this straight away.
So delta u B is Cv times T2 minus T1, right.
and similarly dH is Cp dT right.
So delta H in pathway B is just Cp T2 minus T1 okay?
All right, what we've already got that q is zero.
Delta u is q plus w. Delta u isn't zero.
So this must be equal to work, right?
And now we can look at our special function, but since this is zero right dq B is zero, along path B, so our special function, dS, so we're going to go, dq B over T is equal to zero.
OK?
OK, finally let's look at our third path, this constant volume path, that's going to connect T1 and T2, right?
Constant volume, what's zero?
[INAUDIBLE]
A little more noise here.
What's zero?
Work
Work, great.
So, path C constant volume means our work in past C is zero, right?
Now, our temperature is going to change from T2 to T1, and we just saw what happens to u and H when that happens going from T1 to T2, and it's no different.
It's an ideal gas going along these path.
So we can immediately write delta u C is Cv times T1 minus T2.
Delta Hc C is Cp times T1 minus T2, right?
Work is zero.
Delta u is w plus q, work plus heat.
This is zero, this isn't.
This must be heat q B, right.
So again, there are our familiar thermodynamic quantities, but now let's also look at our special function.
It's not going to be zero this time because we have non zero heat exchange between the system and the environment, right.
So are integral of dq C over T, right, which is our heat, but in this case we can do this calculation easily because since work is zero, we can equate dq with du, right.
So we can write this as our integral from T1 to T2, du over T. du is just Cv dT, right.
So this is just Cv log T2 over T1.
OK?
So now we've completed the cycle.
So let's just compare.
Let's compare what happened in path A to what happened in paths B and C. Yes?
[INAUDIBLE]
Ah sorry, yes of course.
Thank you.
Yes, oh yes, so
[INAUDIBLE]
And it's right because, thank you again, it's because we're going from T1 to T2.
So I'm getting confused by the oh sorry, we're going from T2 to T1.
So it's in reverse of the order of the subscripts and I let this confuse me.
There it is.
There it is.
There.
Thank you, any other details which should be pointed out?
All right, then let's proceed.
So what we're going to do is our comparison of what happened on pathway A to what happened in combined pathways B and C, right?
So here are results for pathway A, right, for delta u zero delta H zero.
I didn't actually explicitly write it or did I?
Let's just write it again.
Here's our work in path A, and here's our heat exchange in path A.
Now let's look at the sum of B and C. So B plus C, here is delta uB Cv times T2 minus T1.
Delta uC Cv times T1 minus T2, the sum is zero right.
That's the some of delta u in these two paths, and if we look at delta u in just path A, it's zero.
And it's going to be the same without delta H. The only difference is it'll be Cp instead of Cv, but there it is for pathway B.
There it is for a pathway C. So the state functions that we're familiar with are doing what we expect they ought to be doing, right?
If you go around in a cycle, starting and ending at the same place, the state functions have to stay the same.
They only depend on the state the system is in.
They don't depend on the path, and that's what we've shown, right.
Similarly if we go from this point to this point through path A or if we do it through the combination of these two paths, and the change in u and H in any state function, those have to be the same.
It turns out they're zero in both cases, and that's what we've seen, right?
Now let's compare what happens to work and heat.
So here are expressions for work and heat.
They're opposites for the pathway A.
Here's pathway B and C. In C the work is zero.
In B it isn't, it's Cv times T2 minus T1, right.
The work isn't equal to the work in pathway A, right, because you know work is not a state function it depends on the path right, and there are different amounts of work done on the system, or done by the system on the surroundings in these two different processes.
They're both expansions.
They'll both have net work done on the surroundings, but not the same amount, right.
And of course it's going to be the same with heat.
We've already seen that delta u is zero.
So we know that in each case the heat is going to be the opposite of the work, but the work isn't the same in these two different ways of getting from here to here, right.
So let's just see it explicitly.
Here's our qA.
Here's heat exchanged in pathway A and in pathway B heat is zero, and in pathway C, here is qC it's Cv T1 minus T2.
So again, for both heat and work we don't get the same result.
Now let's look at our special function, right.
So here's path A.
We found that it's R log V2 over V1.
Pathway B is zero.
Pathway C it's Cv log T1 over T2.
All right, let's just look at that a little more carefully.
R log V2 over V1.
B plus C, Cv log of T1 over T2, right.
Just some weird combination of functions, right?
But, of course, it's not quite like that.
Why?
Because this path is an adiabatic path.
And you already saw last time there was this relationship between the temperature and volume changes along an adiabatic path.
So let me just write that down.
Adiabatic reversible path.
What you saw is that T1 over T2 was V2 over V1 to the power R over Cv.
Isn't that something?
So what does that mean?
We take this Cv and put in the exponent here, right.
And we put this R up in the exponent here.
What are we going to discover?
Those two things are equal right, T1 over T2 to the power Cv is V2 over V1 to the power R, so dS over path A equals dS over paths B plus C, okay?
So what this suggests is that whatever this funny special function is, at least for this one cycle that we've, tried it's behaving like a state function, right.
It seems like its change is independent of path.
Start here, end up here, do it through two different pathways, end up in the same place, right.
OK, now that's all the foreshadowing I'm going to give it for right now.
We will certainly come back to this very special function shortly.
Before I move on, I'm just going to put on the board another cycle, and I'm going to urge you to work through that on your own.
It's worked through already, let's see, I believe it's worked through in the notes, yes.
So if you need the help of the notes it's in there, but I would urge you to work through it on your own, and it's the following: let's start with the same path A, all right?
So we've got our reversible adiabatic path right.
And now, try working through what happens if we close the cycle in a different way, right, like this.
So here's a path that's constant pressure, and here is a path that's constant volume, similar to the constant volume path that we did before, but not between the same pressures, right.
So we can label this one D and this one E, and I urge you to just try going through that and verifying for yourself that again the state functions will behave the way state functions should, and you can see whether this special function once again behaves like a state function, right.
And of course you should expect to see that the work and the heat again won't behave like state functions which they are, then you'll see you have different results for those, depending on the path, OK?
Any questions about going through these cycles and using expressions like this and so forth?
Expressions like this you know turn up in various places, like in the equations sheet that appears at the end of exams, right?
Which means that you may not need to be committing it to memory.
On the other hand it means that you should, it's the sort of thing that you should be familiar with so that the need to use it is something that you'll conjure when you're working on a problem and there's a reversible adiabat, and you have temperatures and volumes that change that you might like to relate to each other.
Cycles that are suggested that you go through and aren't gone through in class also sometimes turn up on exams too, by the way.
So that said, let's move on to the next topic, which is thermochemistry.
So, so far what you've seen in most of the examples in the class are essentially mechanical kinds of changes, pressure volume sorts of changes, where the system is doing essentially a kind of mechanical work, pressure volume work on the surroundings or vice versa and heat is being exchanged, and you're calculating the basic thermodynamics just like we went through, of processes of that sort.
Now I'd like to start introducing something that's really central to chemical thermodynamics, mainly chemistry, right.
Let's start looking at chemical reactions, right, and understanding what the changes in thermodynamic properties are that occur when chemical reactions occur, and the immediate application is to calculate the change in enthalpy, which, as you've seen it, at constant pressure which is the condition under which an awful lot of chemistry is done, that's just the heat.
Which means we're going to calculate heats of reaction, right.
You're running some reaction.
It's in the atmosphere.
It's at constant pressure.
You know, you mix acid and base together and the thing heats up like crazy, right.
Or other things might react spontaneously.
You can feel something cooling off right.
I mean simple examples of these happen when you, you know, if you buy cold or hot packs.
You break the seal between them and feel the thing get cold, for example, right.
What's happening there?
Well, the selection of reactants has been done judiciously to provide either heat or to provide something that's cool.
And all of that is coming out of the heat of reaction, whether it's positive or negative.
Of course the biggest practical application is burning of fuel.
You might use the heat to, and convert it into pressure volume work, right?
So if it's an internal combustion engine, you'll do a reaction.
There will be the heat of reaction.
There'll be a volume change, depending on the conditions under which is occurs, and you can extract work through the reaction and so forth.
So let's just to see how you run through some of those calculations.
Now let me just ask who here took 5.111?
And who here took 5.112?
OK, now you've seen some thermal chemistry in 5.111 and 5.112.
So I I'm going to do some review of that, but I'm also going to call on the thermochemistry that I'm pretty sure you're familiar with.
So I won't go painstakingly over every element of the notes here.
Subsequently we'll go on into additional topics that you haven't seen, and I'll treat them in full detail.
OK, so let's do some thermochemistry.
What we want to do is we'd like to be able to predict for any reaction what's the heat of reaction.
And so what we're going to calculator is delta H. So for any kind of reaction we should be able to do that, right?
And we're going to treat constant pressure situations certainly at first and most of the time, right?
Constant pressure, right, reversible delta H is going to give us our heat of reaction, OK. And then if we can also determine delta u, then we know this, we know delta u is q plus w, then we can determine work as well, right?
And typically, we'll be treating at least some cases where we're dealing with ideal gases in which case we can easily get delta u.
And then we'll be able to in a very straightforward way get w. So it's a really powerful and simple formalism, once we set up what is needed the go through it, right?
So let's just consider any reaction.
The one that I've got written down for you is it's the reverse of the rusting of iron, right?
So iron left to own devices in the atmosphere in the presence of a little bit of water, and the atmosphere will start combining with the water form iron oxide.
There's an equilibrium between the two.
So we'll have an iron oxide species.
It's a solid.
It's combining with hydrogen which is a gas to give iron, also a solid and water, right?
So the way I've written this we're imagining the iron, the solid iron immersed in the water as a liquid, could be calculated otherwise as well, but in this case were imaging the water as liquid and going in this direction then it's forming iron oxide and evolving hydrogen gas, okay?
And our heat of reaction or enthalpy of reaction is defined as the enthalpy at constant pressure.
Isothermal conditions, temperature won't change and reversible work.
For Isothermal reaction constant pressure reversible, right?
Of course, they're all sorts of conditions under which a reaction could be wrong in the lab or outdoors or however, right.
But this is the way we're going to define delta H of reaction.
We want to have that definition clear because in fact we're going to, we might want tabulate heats of reaction, right, and of course want to know what the conditions are for the tabulated values apply.
And we're going to want to calculate them from other quantities, and again, we're going to need to know each case what are the relevant conditions?
OK, so what happens?
Well, we should be able to get this.
We should be able to calculate delta H. It's a state function.
If we know the enthalpy of the products minus the enthalpy of the reactants, right.
It's a state function.
And we can we can do this in principle except for one important detail, which is that enthalpy, just like energy u isn't measured on an absolute scale but on a relative scale, right?
You know, if you want to measure the potential energy of something in a gravitational field, you have to define the zero somewhere, right, because it's arbitrary.
You can set it anywhere you want.
It's the same with enthalpy.
Enthalpy is just u plus p V. So there is an arbitrary set point that needs to be defined, right?
Because what you actually measure in the lab are changes in enthalpy, just like what you measure when you look at energy change of some sort, you measure the change in energy, right.
The absolute number that you assign to it is something that's arbitrary.
You have to set what the zero is.
And so there's a well-understood convention for what the zero is.
What we define as zero is the enthalpy of every element in its natural state at room temperature and ambient pressure.
In other words, we choose a convention for the zero of entropy, so that then we can write entropies of products and reactants always referring to the same standard state.
And then we calculate changes, the convention is understood with respect to what is the zero, right.
And so our tabulated values, they'll all work.
They'll all refer to the same standard state.
And we'll be able to use the formalism that we set up.
So our reference point for H, it's 298.15 Kelvin, one bar and in that standard state our molar enthalpy is defined as zero for every element in its stable form.
OK, very important.
OK, now, given that reference point, all we need to do to get the value of enthalpy that we're going to use for each reactant and each product is calculate how much does the enthalpy change to form it from the elements in their stable form at room temperature and pressure, right.
So we're going to replace the H or we're going to put in for the H what we'll call our heat of formation or delta H of formation starting from the elements in their stable states at room temperature and pressure, for each of these things.
And we can do that because now we're going to have instead of just sort of H, it's going to be delta H of formation for each of these things, is going to appear in our calculation of H, but that's OK. Delta H of formation means the enthalpy of this compound minus the enthalpy of its constituent elements in their most stable state at room temperature and pressure.
But we've defined the enthalpy of those elements in their stable state at room temperature and pressure as zero, right?
So we're just subtracting, in effect, zero, right, from the enthalpy of the product, but of course it's important have that established because the heat of formation is something you could measure, right?
You could run the reaction, take solid iron, gaseous oxygen, form iron oxide, measure the heat of formation of it, tabulate it.
We know it.
We know it forever, right.
And for any of the other compounds.
So in other words, by defining that reference state, we can then figure out or measure heats of formation of a vast number of compounds.
We can tabulate them.
We can know them, and then when we have reactions that inter-convert different compounds, we can calculate the heat of reaction is just the difference between the heat of formation of the reactants, and the heat of formation of the products, right.
So let's just write that out.
So first of all, for example, right, molar enthalpy of hydrogen gas at 298.15 K is zero.
Hydrogen gas it's in its most stable state, right at room temperature and pressure.
That little zero, that little superscript means one bar always.
OK. A molar enthalpy of or whatever, iron, as a solid at 298.15 Kelvin is zero.
Iron as an element is a solid.
That's it's most stable state at room temperature and pressure, right, and so on.
And then we can figure out heats of formation.
Now in this particular reaction, I've got hydrogen gas, iron solid.
Those already are elements in their most stable forms at room temperature and pressure.
But this is a compound, right, it has some non-zero heat of formation from the elements.
So is water, right?
So I need to find out the heats of formation of the iron oxide and the water.
And if I do that then I can find out the change in enthalpy of this reaction.
It's just going to be the heat of formation of these three moles of water, minus the heat of formation of the iron oxide.
OK.
So how am I going to do that?
Well, I need to write the reaction that forms that compound from it's elements, right?
And I want to tabulate that for an enormous number of materials, right?
So for example, if I want to look at HBr, there's a simple case, right, hydrogen bromine.
What's my heat of formation?
Delta Hf.
It's going to have our little zero, right?
How do I calculate it?
Well, I need to write the reaction that forms this from its constituent elements.
What is it?
Well, 1/2 H2 as a gas, temperature, at one bar, plus 1/2 bromine, no actually bromine is a liquid at room temperature and one atmosphere.
They form HBr which is a gas at room temperature and one bar, right.
I can measure that.
And the heat of reaction for this, delta H of reaction is equal to delta H of formation, of HBr as a gas at our temperature and one bar, okay?
So that's how we determine our heats of formation.
We measure them for all these compounds.
And then we go back to reactions like this, and we can just very simply determine the heat of reaction because all we're doing is the following cycle.
We go from reactants to products and there's some heat of reaction, and here's the cycle that we have.
We go to the elements in their standard states, in both cases.
So we're really just doing our delta H is the negative sum of delta H of formation for the reactants, all right?
Because here what we're doing is we're going to take apart our reactants and form the elements from them, right?
So written this way, for example, I'm going to pull these things apart, and I'm going to have solid iron, and I'm going to have gaseous oxygen, and I know the enthalpy of that reaction.
That's just given by the heat of formation, the enthalpy of formation of iron oxide.
Here, now I'm going to take those same elements, and I'm going to make the products, right.
I know it's going to work because I already had this reaction, so all the, you know, I'm going to conserve atoms in the right way.
So here, I'm going to have delta H, is just the sum for all the products of delta heat of formation, right?
Here I'm going to put together all the products.
So this is a positive heat of formation.
This is the negative heat of formation, right?
In other words, I've got reactants, and I've got products.
What's delta H of reaction?
It's delta H of formation of the products minus delta H of formation of the reactants.
That's the important message, delta H of formation of the products, minus delta H of formation of the reactants.
Any questions?
All right, tomorrow I'll go through the remaining details.
The truth is after this, it's all arithmetic, right?
We're just going to go through it and execute it a little bit.
And then we'll move on and talk about how we'd actually make some of these measurements of delta H and compare them to what we calculate.
See you tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, so last time we did a little bit more work on thermodynamic cycles, and basically went around a cycle and looked at some state functions, delta u and delta H. Saw that around a closed cycle those zero because they're state functions.
So if you start and end at the same place, they've got end at the same place that they started.
There's no change in them, and then we also looked at some at non-state functions, work and heat, and saw that those aren't zero going around a cycle.
Of course you can do work in a cyclic process, and heat can be exchanged with the environment at the same time.
So we calculated those values too.
And we also looked at this other funny function that's our special function.
We looked at this integral dq over T. It's so special that we called this derivative dS, and saw that at least for the cycle we looked at, it also behaved as a state function behaves.
That is, going around the cycle, it had no net change, and we'll see this come back later on.
Then we went on to look at thermochemistry, and that's what I want to continue today.
So really, the main result that we saw last time is that we can describe the enthalpy of reaction, so if we have reactants going to products, and we'd like to know our heat of reaction or enthalpy of reaction, remember it's a constant pressure as we're considering it, so it's equivalent to heat of reaction, than the way we can do this is construct a cycle where we're going to express these in terms of their constituent elements.
We're going to all the way back to the atoms.
Right, so if we have the elements in their standard states, that is their most stable forms at room temperature and pressure, then what we're really doing is we're saying let's take the reactants and let's pull them all apart into their separate elements, they're separate atoms.
And then put them back together this way, and we'll calculate the enthalpy of this process, the enthalpy of this process.
It's a cycle, so that'll give us the enthalpy of reaction, all right?
So this delta H is our sum for delta H of formation for the reactants, right.
And this is our sum of the quantity for the products, right.
And in this way, then we can express our delta H of reaction is just going to be given a sum of the heats of formation or the difference between the heats of formation of the products and the reactants, right?
Now, of course, we have to weight this by the number of moles of each reactant to each product involved in the reaction.
So to write this a little more carefully, it's, you want to write the sum over i of nu i delta H naught of formation of i summed over the products.
This is per mole minus sum over i, nu i here over the reactants in the same quantity.
Delta H per mole f i, right?
So in other words, were taking each of the products and we're going to weight them by their stoichiomeric coefficients, and we're going to have in this delta H of reaction their heats of formation, and then we're going to have the negative heats of formation of the reactants, because that's being subtracted, okay?
So, just to go through a basic example, let's just look at the burning of methane.
So we've got methane gas, combining with oxygen gas, so you have carbon dioxide gas and liquid water, and we'll consider the whole thing at room temperature and pressure, right.
So let's just undertake this set of processes.
Let's decompose these to the elements.
So in other words, our methane gas, we're going to write the chemical equilibrium between this and the solid carbon.
That is to say let's specify graphite, it's not diamond, right, graphite is the stable form of carbon at room temperature and pressure.
And then, hydrogen gas two moles of hydrogen gas and we're going to have a delta H of formation.
This is for CH4 in the gas phase, right?
And then over here, we have our oxygen but 2 O2 gas going to, well 2 O2 gas -- the point is oxygen already is in it's stable form at room temperature and pressure, right?
That is the elemental form of oxygen, of course, is as oxygen molecules in the gas so there's no delta H formation for oxygen.
The gas is zero, right.
And then second phase of this, now we're going to take the elements that we've produced, and we're going to put them back to make products, right?
So there we've got carbon as graphite, and a solid plus an oxygen gas goes to make CO2 gas, right.
And we've got our positive delta H of formation associated with that.
And finally we've got hydrogen gas plus oxygen gas giving us two water molecules and this is in the liquid phase, and there's some delta heat of formation associated with that, okay.
So there is our set of individual processes that's going to constitute our cycle, right.
If we take the combination of all those, which is to say we just take both steps in this cycle, then we'll have our net reaction, which is to say we'll have this, right.
So what that says is our heat of reaction is our heat of formation per mole of CO2 gas, right.
Plus 2 times the heat of reaction or heat of formation per mole of liquid water minus heat of formation per mole of methane, right.
And things we can easily look up.
So in a simple way, we can determine, we can calculate what the heat of reaction is for something like this, right.
And you'll see in the back of your book, you'll see moderately extensive tables of heats of formation, and if you go online you'll see, you can find extremely extensive tabulated values of heats of formation, that have been measured for an enormous number of compounds.
And from that, then you can look at enthalpies of reaction for countless numbers of reactions, right.
So by using the tabulated data, we can really determined heats of formation for most reactions that you might contemplate, OK?
Of course, the cyclic steps that we've taken to do this apply not just for breaking down reactants and product into the elements in their standards states, but of course we could also look at whole sets of reactions and write cycles as well, right.
So for example, if we wanted to look at the reaction of something like carbon graphite plus oxygen gas to make carbon monoxide gas, well normally this isn't the way the reaction would work.
Normally if you just tried to make this happen what you would wind up with the CO2, not CO, right.
But one way to approach this is you could look at the individual reactions of going, you could go to CO2 and then you can go with CO2 and oxygen and combine, or combine CO and oxygen to get CO2, right?
So normally, CO2 is formed, but you could calculate the heat of reaction.
Of course you could go back to the elements, but you can also say well, let's just take a known heat of reaction for, or heat of formation for CO2, and then also combine that with the heat of reaction for CO plus oxygen forming CO2 and determine this that way, all right, OK, so once you have done this, once we've got the heat of reaction then there are a few simple results that are useful.
One is we know immediately whether when we run a reaction heat is going to be released in the process out to the environment, or whether heat is going to be taken in, right.
So, and of course if you ever used a hot pack or a chemical hot pack or cold pack, you've seen examples of the two of those, where the constituents are selected to give you either heat released or heat taken up, right.
So in particular delta H of reaction less than zero, that means a negative amount of heat, which means that heat is released, right.
Yes, that is it's exothermic.
Positive enthalpy of reaction, which is to say a heat of reaction is positive.
Remember the way we define heat is positive it means that there is heat that is coming from the environment into the system, right.
In other words heat is absorbed or taken up by the reacting system.
That's endothermic, okay.
All right, one more important aspect of this that we need to be able to deal with, and that is the heats of formation that you'll find tabulated typically are going to be given for room temperature and pressure.
And it's very common that you might want to know the thermodynamic condition for reactions, especially at other temperatures, very common, of course.
And it's not hard to see how the heat of reaction at room temperature can be related to they heat of reaction at other temperatures.
So let's just look at that.
So remember your fundamental relation, dH, if you write that as a function of temperature and pressure, partial of H with respect to T, constant pressure, dT, plus partial of H with respect to p at constant temperature, dp.
But if we're working at constant pressure then dp is zero.
So we just got the derivative with respect to temperature.
That's our heat capacity, right, Cp.
And we may have tabulated values of Cp for an enormous number of materials.
So now, if we look at the temperature dependence of delta H for reaction, really what we're going to need to do is look at how that heat capacity changes going from reactants to products, right?
So in other words, d delta H of reaction, with respect to temperature.
That's what we want to find out, right?
So how does delta H change if we change the temperature?
Well, it's just given by Cp dT for the reactants and the products, right?
For the products minus the reaction, that is it's delta Cp, we need the sum over the products.
Nu i, Cp i minus same sum over reactants, i, nu i Cp i, right.
Similar to what we saw before.
Now we need to integrate, right.
So if we integrate between some pair of temperatures, T1 and T2, we do this and we have this quantity, d delta H of reaction with respect to temperature, dT, all this is at constant pressure, right.
Let's just be careful to specify that.
Well, so of course, this is then just delta H of reaction at T2 minus delta H of reaction at T1.
This is looking good.
This is what we want to get, is our heat of reaction at some new temperature T2.
If we already know it at some initial temperature, usually room temperature, T1.
And it's just given by the integral from T1 to T2 of delta Cp dT.
Now in general, this is as far as we can go, but at least if the temperature change isn't too big, for most materials that heat capacities doesn't depend very strongly on temperature, right.
If you take some material, and you measure its heat capacity, right, how much heat do I have to put into it in order to change its temperature by a degree, right.
And I make that measurement at room temperature.
And then maybe I raise the temperature to whatever, room temperature, maybe 20 degrees hotter than room temperature.
And I again say OK, now how much heat do I need to raise this thing's temperature by 1 degree?
The amount of heat I need to put in to do that is not very different from what it was at room temperature right.
In other words the heat capacity didn't change very much.
So, the result of that is that at least for modest temperature excursions, it may be the case that we can simplify this and simply say it's just the difference in heat capacities times the temperature change.
Again, we can't always be assured of that, but it's often the case.
Any questions on any of this so far?
OK, now what I want to do is just describe a little bit of how do you measure all this stuff, right?
So what we've done so far is just lay out in principle how we should describe heats of reaction, and of course assuming that we've got everything in tables we can always look them up, but sometimes you want to make new compounds, right.
Sometimes you'd like to know what the energetics are involved there, right, and so it's useful to be able to actually measure something.
So how do we do It?
This is what's called calorimetry.
So we're going to make measurements to determine heat of reaction values.
Actually, calorimetry is used for all sorts of things.
It can be used to determine, for example, the energetics of phase transitions.
Maybe you're not looking at a reaction, but you've got some new compound, and you're looking at it go from liquid to solid or to gas.
And you can see what the heat involved in a process like that is as well.
So what happens?
Well, the objective in the case of a reaction, we have reactants at some temperature going to products at that temperature, and that is our heat of reaction at that temperature.
That's how it's defined, right, at constant temperature.
OK?
What do we actually measure?
Well, what we can really do is we can put the reactants in some container, put the whole thing in an insulated place, you know, hold things in a big insulated box.
Run the reaction.
Maybe it produces heat.
If it does, then the whole thing will heat up, right?
The stuff that's inside the reacting volume, and whatever's right around it there inside the big box, it's all going to heat up.
So I'll start at some initial temperature, T1.
I won't end up at the same temperature.
It'll be hotter.
How much hotter?
Well, that depends on how much heat was produced.
So in principle, if I measure how much hotter, I can determine how much heat was produced, and from that, I should be able to calculate delta H at T1.
So let's think about how that's really working.
I've really got reactants at T1, plus a calorimeter at T1, and I'm going to end up with products -- well let's let me just write that's over.
In fact, let's start by just putting a little sketch of the whole thing up.
So here's my reacting stuff, maybe it's a liquid.
It's going to take place in there.
It's going to be a constant pressure, it might be open to the air, or even if it isn't, there might be plenty of room, and it's a liquid anyway, so the pressure isn't going to change significantly.
I want to measure the temperature.
And the whole thing is insulated right.
I don't want to have to deal with heat escaping to the outside environment in a way that might be difficult or complicated to measure or calculate.
OK, so this, what I've sketched here would be a constant pressure calorimeter.
There's a reaction.
Normally this is used for a reaction in the condensed phases and liquid usually.
So this is a constant pressure calorimeter.
It's set up to be well-insulated so it's adiabatic.
That's the set up.
We're going to run the reactants, the reaction.
The reactants are going to turn into products.
Let's look at what happens.
So what happens is my reactants at T1 plus calorimeter at T1 turn into products at T2 plus my calorimeter at T2, right.
That's my process.
That's what really happens.
Now that, the enthalpy of that process isn't what I want, right, because the temperature has changed, also the calorimeter is heated up.
So I need to relate this to what I do want.
Let's label this one.
It is adiabatic.
It is taking place inside this thing, and it's a constant pressure, and we'll do it reversibly, right.
So that's what we've got.
Now, we can always take these things the products and the calorimeter at temperature T2 and cool them or warm them to get to products plus calorimeter at T1, right.
Let's label that two.
It's still at constant pressure.
It's reversible.
It's a temperature change.
Now to make that happen, it's not adiabatic, right.
If I wanted to do that, I'd need a heating element or something to cool, so I could make that temperature change happen, right.
Well, so now I can complete the cycle.
I've got reactants and calorimeter at T1.
Fewer products and calorimeter at T1, right.
Calorimeter doesn't change in this process.
So here I've got some delta H associated with changing the temperature.
This delta H though, this is what I want.
This is delta H of reaction at T1, right?
It's isothermal, constant pressure, reversible.
Just what I want.
That's how I've defined delta H of reaction.
This is what I'm after.
All right, so now let's see how we execute it, and do the calculations that allow us to calculate this.
So, one, what's delta H in step one?
Yes, exactly, it's adiabatic, right constant pressure.
it's zero.
Delta H1 is zero, right.
What we're really going to do in practice is we're going to measure, we're going to use our thermometer and say great, how much did the temperature change, right.
So now, then we're going to use what we're going to learn from step two in order to calculate this part, what we could call step three.
OK, so two, that's the crucial part.
It's constant pressure.
It's just given by the corresponding heat, and it's just a temperature change, right.
So we know how to do this, integral from well it's T1 to T2, depending on which way we go of Cp for the whole system dT.
OK?
It's just how much heat is involved when we change the temperature.
Now, the products have some heat capacity associated with them right, it takes a certain amount of heat if we make their temperature change, to either put it in or take it away, depending on which direction the temperature is changing.
Same with the calorimeter, OK.
In actual practice though, the calorimeter, is going to be this big hunk of metal, and really what's going to, there's going to be some stuff in here, right, they'll be a fluid, it will usually be some sort of oil.
There's a pretty big thermal mass.
Now we run the reaction and it produces heat.
Most of the heat is going to warm up or cool off all that oil and stuff.
There's a moderate amount of material in the actual reaction.
You want to design the calorimeter to fulfill that condition, Of course, you can't make it so enormous that even for, that for any ordinary reaction there's not even a measurable temperature change, right, because if you have your enough oil to fill up this room, it would take a huge amount of heat to change it by even a tiny, it's temperature but even a tiny amount.
So the calorimeter is designed sort of to scale to match the maybe the ordinary volume of reactants that you'll put in there, such that, pretty much all the heat's just going to heat up the calorimeter, and there's only a small amount that's going toward heating up the products, right.
So in general, or at least usually it'll be the case, that this is approximately equal to integral from T1 to T2 Cp of just the calorimeter dT.
And that generally is just given by the heat capacity the calorimeter times delta T, right.. Because the heat capacity of the calorimeter just like this thing is not strongly temperature dependent, OK.
So the point is we can calculate delta H associated with this process pretty readily, OK. That leaves three, right.
But delta H for step three must just be the opposite of this delta H because this was zero and we know that were going around a cycle, right.
And that's our heat of reaction.
So we're going to be able to do this, right?
OK, now this is one way to do calorimetry, and it's practical as long as all the materials are in condensed phases, solids and liquids.
If gases are involved it can work, but it can be tricky to keep the pressure constant, right, because that means that now the moles of gas might be changing, and that means in some way the volume has to be adjustable.
You'd need some sort of balloon or membrane that will allow that to happen.
Could be done, but easier is to just do the whole thing at constant volume, right, and just run the reaction that way and redo the calculation to be a constant volume rather than constant pressure calorimeter, right.
And it's not hard to do that.
So let's just look at what happens in that case.
It's almost the same.
So let's see, just to finish the job here though, all this says is delta H of reaction is just negative Cp for the calorimeter times delta T, all right.
so that's what we think we know in constant pressure calorimetry.
Yes, and if we have gases involved, it's pretty similar, but now what will have is something like this.
We'll have a reaction vessel that's sealed, it's constant volume.
That'll be inside our calorimeter.
It's insulated, and there's still a thermometer, so we can measure the temperature.
So this is still adiabatic.
It's insulated, but now it's constant volume, OK. Now, you know with constant volume, now it's not going to be delta H that's straightforward to measure, it's going to be dealt u, all right.
But it's going to be almost the same, right?
So let's just think about what happens here.
Now it's the cycle that we've got will still basically look the same.
That is its reactants at T1 plus calorimeter at T1, going to products at T2 plus calorimeter at T2, right?
And it's still adiabatic, but now it's constant volume.
And it's also reversal right.
So this is our process one.
Now, process two, we're going to end up here with our products, again at T1 plus our calorimeter at T1, right.
So now we have a constant volume reversible temperature change.
And so this part now isn't exactly what we want to determine delta H, because it's a isothermal constant volume, reversible products or reversible process that takes a reactant to T1 to a product of T1.
What these are going to give us are delta u values.
This'll give us delta u of reaction, right, T1, right.
It's a constant volume.
All right, so in the end, we're going to determine delta u here, and then in the end, we're going to have to relate that to delta H, but that's straight forward enough to do.
So let's look at how we'll analyze what happens.
So one adiabatic, constant volume process, right.
What's zero in that case?
In this case delta H was zero in the constant pressure example.
Now we've got a constant volume process.
What's zero?
It's u, because u is to q plus w right, heat and work, but it's adiabatic.
So there's no heat, exchange with the environment, and it's constant volume, so there's no p dV work, right.
So q is zero adiabatic.
Work is zero.
Delta u1 is zero.
And again just in the constant pressure case, what happens that we're going to measure in step one, which is to say in actually running the reaction is the temperature's going to change.
Right?
The whole thing's going to come to some new equilibrium temperature between the products and the oil or whatever's around it, and we're going to measure that.
OK, two, now it's a temperature change, right?
We know how to calculate delta u for a temperature change.
It's very similar here, but what's going to be different from the case with constant pressure?
There's a real important detail that's different if you want to calculate what happens at constant pressure when you change the temperature?
What happens at constant volume?
Looking at this expression, what's got to be different?
Cv, right.
We're not going to have the constant pressure heat capacity, we're going to have the constant volume heat capacity, right.
So delta u for step two, that's our q under constant volume conditions, integral from T1 to T2 Cv for our whole system dT.
Integral from T2 to T2, and now I can separate again the calorimeter from the product, and at least approximately, again, it'll be Cv for the calorimeter dT, which is to say then it's Cv of the calorimeter times delta T. Great.
Three, well delta u1 was zero.
Delta u2 was there Cv delta T, so now all we need since the whole thing is going around in a cycle, Since we know delta u3 must be negative delta u2, and that is our delta u of reaction, right?
So delta u of reaction is approximately equal to negative Cv for the calorimeter times delta T. So, this is what we're going to measure, and this is what we're going to determine.
And now we're almost done, except what we really want is delta H and not delta V, right.
So now we're going to use the fact that H is u plus pV.
Delta H is delta u plus delta pV, and now this is all at constant temperature in the end, right?
We've determined delta u for some temperature T1, right.
So what happens then we're going to use the ideal gas law.
So it's approximately delta u plus delta nRT.
That's a constant.
That's a constant.
So it's delta u plus RT, we can say T1 is the temperature we've used here, delta n of the gas.
In other words, what matters here in changing the pressure volume product?
What matters is we turned some reactants into some products.
How many moles of gas are there in each case, in reactants and products?
If that changes, of course you know that the pressure in there is going to change at constant volume if the amount of gas in there is changing.
And nothing else is going to make a significant contribution to it, OK?
So finally, then delta H of reaction for our temperature T1 is approximately minus Cv of the calorimeter times delta T plus R T1 delta n of gas.
Change of the number of moles of gas, right.
We're going to measure that, and now we're going to determine this.
In practice, we'll already know the heat capacity of our calorimeter, when we buy it, right?
So we don't really need to put in a certain amount of heat and change the temperature of the products and the calorimeter and so on.
What we need to do is just measure how much the temperature changed, OK.
It's worth getting just sort of roughly calibrated how big is this compared to the rest of this stuff.
That is, how different is delta u from delta H, all right.
And of course it's straightforward to do this, and I've written this out in the notes, so I won't re-write the numbers here.
But I gave an example looking at combining HCl and oxygen right.
So 4 HCl plus oxygen gas.
This two in the gas going to water and the liquid, plus chlorine gas at room temperature.
Well, what you find out is delta u of reaction is minus 195 kilojoules for the reaction as written.
And it turns out, as written, if you say OK, how much, what changed for the pV product?
Well here we've got four moles of gas, five moles of gas.
Here just two, so we changed the number of moles of gas by three.
All right, how much did it matter, right?
Well it matters.
It's measurable.
So now delta H of reaction turns out to be minus 202 kilojoules.
So, you know, seven out of a couple of a hundred, right, a few percent, kind of typical.
In other words, if you look at energetics of ordinary reactions where you're, you know, you're making and breaking covalent bonds, there's a fair amount of energy stored in those, right?
The additional change due to changing pressure volume is certainly measurable.
You don't want to just ignore it, but in cases like that, it's usually a small fraction of the total.
A few percent is typical, okay.
All right, let me just go through one numerical example of a calorimetry calculation, OK.
I won't put all the numbers up on the board because our time is running short, but I just want to outline it.
All I really want to do is calibrate you a little bit for what happens with ordinary calorimeter heat capacities that makes the calculation turn out to be relatively easy, right.
So let's just write it out.
Let's take iron sulfide as a solid, plus 11 halves oxygen gas to make iron oxide, also a solid, plus sulphur dioxide gas.
All right.
We'll start at T1 is 298 Kelvin.
T2, maybe we don't know it yet, right.
OK, so we know how to calculate what's going to happen, delta H and delta u, because we can look up the heats of formation and so forth of all the compounds, right.
So if we do that, what we discover is that delta H of formation whoops, something's wrong here.
This is sulphur -- what am I doing, boy, S and it's a two, sorry, and the heat of formation is minus 180 kilojoules per mole, that's oxygen, it's zero.
Boy, I'm, I don't know what is about this reaction that's vexing me but it's not that complicated.
Here it's minus 824 kilojoules per mole minus 297 kilojoules per mole, right.
So that's our input thermodynamic data.
So first of all, let's just do a heat of reaction calculation, right.
Taking the product minus the reaction, right.
So it's minus 824 plus 4 times minus 297 minus 2 times minus 180.
Right?
In other words I've got the stoichiometric coefficients in there and the values, and I'm subtracting the reactants from products wind up with minus 1652 kilojoules per mole.
Well, it depends on what we write, what we consider mole, right, maybe I should just write kilojoules as written.
OK, I'm going to skip the delta and get the change in moles of gas calculation.
It's straightforward to do.
What I really want to do is just give an example of what happens when you throw the thing, the material into a calorimeter and see how much the temperature changes.
So let's imagine we start with 0.1 moles of our iron sulfide, and then we have a stoichiometric amount of oxygen, and the whole thing is done in a constant volume calorimeter, and we see what happens.
Now the crucial element is what is the heat capacity of the calorimeter, right?
And it's, again it's a macroscopic pretty big thing, so typical might be 10 kilojoules per Kelvin, and that's pretty big, right?
Noticed that's not per mole, right.
I mean the calorimeter is a big thing filled the little oil or whatever is inside it, right?
And it's for that whole unit that you've got some heat capacity.
How much heat does it take the warm the entire thing up or the insides of the thing up by a degree?
It's that number right.
Ordinary heat capacities are in Joule's per Kelvin mole, not kilojoules, right.
And what that's telling you is probably your reactants and products, so the amount of heat that's involved in changing the air temperature is going to be negligible compared to what happens to the whole calorimeter.
OK?
So now if we say, OK, we we've done this calculation starting with 2 moles of this, but now we're going to be at 0.1 mole, so we're going to need to divide by 20, right.
So instead of minus 1652, it's going to turn out delta u is minus 1648 kilojoules, so, and then divide by twenty, we end up with minus 82.4 kilojoules, that is, that's the delta u of reaction for what happens if you'd put in not two moles of this, but 0.1 mole of this, right.
Practical amounts is the reason I'm using numbers like this, right.
So now, what's delta T?
Well here's Cv, right, so delta T is just our minus 10 kilojoules per degree oh sorry, it's our minus 82.4 kilojoules.
That's the heat released, divided by minus 10 or 10 kilojoules per Kelvin, right.
It's 8.2 Kelvin.
In other words, how much does the temperature of the whole thing change when you put an ordinary amount of material in there and run a reaction, right.
Well, what do you do?
You calculate how much heat is released in the reaction.
And then what's going to matter is what's the heat capacity of the whole, of the calorimeter?
I didn't even need to know that heat capacity of the product, right.
Because it's effect the thermal mass of the product is negligible compared to the thermal mass of the calorimeter.
Now in real practice, I'll do the calculation in reverse.
I'll measure how much the temperature changed in the calorimeter, right.
I'll know the heat capacity, and what I'll really be calculating is OK, how much heat must have been released in the reaction to make that temperature change happen?
And that in the case of constant volume, in this case that's my delta u, and then I'll add my little delta n term to get delta H. Any questions on calorimetry?
OK. See you Friday.
We'll finish on calorimetry and thermochemistry and then we'll start in on one of the really most difficult topics that we'll deal with all semester, which is a second law and our special function that we've seen just a little bit so far.
The following content is provided under a Creative Commons license.
Your support will help mit OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
So last time we just about finished up the discussion of thermochemisty, and we went through a description of calorimetry, how we actually make measurements of thermodynamic a quantities.
I should mention, which I didn't last time, that you know a calorimeter, apart from measuring heats of formation also is used to measure standard thermodynamic quantities like heat capacities.
You know how much heat does it take to raise the temperature of something by a degree?
Well, you put the material inside the calorimeter, and you have a heater and it's connected up to current source, and you can accurately measure how much heat you're putting in, because you can measure how much current is going into it, and you separately are measuring the temperature of the material, so it's a straightforward way of measuring the heat capacities, thermodynamic data of this sort.
So work on phase transitions, just ordinary properties of material.
Chemical reactions, all the thermochemistry involved is done using constant pressure or constant volume calorimetry in a pretty routine way.
So we talked about how to use the results of measurements like that to calculate reaction energetics, enthalpies of reaction.
And I just want to end the unit on thermochemistry with a real brief discussion of an approximate way of formulating heats of reaction, and that is in terms of bond energies.
Now you know you're probably never going to sit down and do a series of thermodynamic calculations using bond energies.
You could always look up heats of formation of compounds and from those determine very accurately heats of reaction, by looking at the heats of formation of products and the heats of formation of the reactants and subtracting them.
And since that data is available for a really wide range of materials, that's sort of the simplest approach and the most accurate one to determine reaction energetics.
On the other hand, if you just want to think about molecules, you know, you think about bonds.
You know if you say I've got methane or ethane, I'm going to use a fuel and burn it and capture the energy.
How much energy do we have in there?
Well the way you want to be thinking about that is all right, I'm going to take the starting material, the fuel, I'm going to break bonds, and I'm going to form some final product with new bonds.
Roughly what kinds of energetics are we talking about?
In biological systems, it's also super important.
You know you think about ordinary biochemical energetics, what's used is fuel in biology and what are the products that are formed?
And again, it's just super important to be able to think sort of semi-quantitatively in terms of bond energies, because you don't want to always be going and looking up everything in tables.
If you need to do an extensive series of calculations, it's very useful to have the tables.
But if you just want to think about how something is working, you know biological or system or an engine or another application, it's very useful to just be able to think in terms of bond energies.
So let's just go over bond energies.
And you've seen this in 5.111 and 2, so I'm not going to belabor it but they're just a little bit of examples that I want to go through, just so it's fresh on your mind.
So the idea, of course, is to be able to have an idea of how much energy is involved in forming or breaking a single bond.
So, if we just do this by example, we can take methane so it's a real simple case because we know it's just got four identical carbon hydrogen bonds, and we could say OK, let's just break them.
So we'll start with methane gas and go to one atom of carbon in the gas phase, plus four atoms of hydrogen in the gas phase.
The energetics in this case aren't given by the usual heats of formation.
You'd just say, if I want to know about methane, I can look up its heat of formation from the elements in their standards states.
but the difference here is these elements aren't in their standard states.
Carbon in its standard state at room temperature and pressure is graphite, a solid, it's not carbon atoms in the gas phase, and hydrogen in the standard state is hydrogen molecules H2, not individual atoms.
All right, so we'd like to have this bond energy for these things.
So let's just to make a cycle that will allow us to determine this.
So this'll be our first step in that cycle, or one way of getting there.
And this delta H of reaction is going to be by definition four times our bond energy for a carbon hydrogen bond.
Now let's go to be elements that are in their standards states at room temperature and pressure.
So let's go to carbon as graphite.
And let's go to two molecules of hydrogen gas, so this will be step two and this will be step three.
And this will allow us to work because now we're starting in standard states, so we can just use regular heats of formation.
And that's the key.
So if we look at step one delta H1 there as we've defined it, four times our bond energy.
Two, this is just negative heat of formation of methane, which we can look up.
And three, is similar.
Essentially it's the heat of formation of carbon in the gas phase and hydrogen atoms in the gas phase, commonly called heats of atomization, but these are also tabulated just like heats of formation of ordinary compounds.
In fact, in the appendix to your book that has thermodynamic data, you can find carbon gas phase atoms including heat of formation, and same with individual hydrogen atom as opposed to hydrogen molecules in the gas phase.
So those data are available.
So we can write delta Hc from atomization, and two times delta H of H2.
This is the way this is tabulated and this is molar.
And of course now we've written a cycle.
So we know that to get from here to here is the same as to go through these two steps.
In other words delta H1 is the sum of delta H2 and delta H3.
And so this says that our four times the carbon hydrogen bond energy is just given by minus the heat of formation of methane plus the heat of atomization of carbon, plus the two times the heat of atomization for hydrogen molecules.
And if we look up the numbers that are tabulated, what we discover is that our carbon hydrogen bond energy is 416.2 kilojoules per mole.
Very useful to have numbers like that stored in your head.
To have some basic idea of just ordinary chemical energetics in terms of bond energies.
Hydrogen bonds, you know roughly 20 kilojoules per mole, and so forth.
Just having those kinds of energetic enables you to think in a qualitative way about what's likely to happen in an ordinary chemical or biochemical situation.
OK, so this is in a sense the simplest case, because we've talked about the dissociation of a molecule that consists of all identical bonds.
But it's straightforward to go from there to additional bond energies.
So from this we've got a value for a sort of typical carbon hydrogen bond, right.
We can do the same thing now for ethane, and that's got a single carbon carbon bond, as well as a bunch of carbon hydrogen bonds, but now the we know the carbon hydrogen bond energy, we can use that number plus the relevant heats of formation to determine the carbon carbon bond energy.
And we can keep going and extend this to essentially tabulate bond energies for a very large number of different sorts of bonds.
So just as an example, ethane, so now we're going to have -- this I'll write it all out just some we can keep proper track of all the bonds that we're going to be breaking.
So C2H6 goes to two carbon gas phase atoms, and six hydrogen atoms.
So in this case, our, I can write this again as one delta H1 is six times the carbon hydrogen bond energy plus the carbon carbon bond energy that we'd like to determine.
And again we can go through now a similar cycle.
So again this now can be equilibrated with carbon graphite plus three molecules of hydrogen in the gas phase, just like before.
And so from that we can see that we'll end up with an equality between these, and negative delta H naught formation for C2H6 or ethane, plus two delta H of carbon atomization, plus three delta H for atomization hydrogen molecule.
So we've already seen we could look up these numbers and this, and we've already determined a carbon hydrogen bond energy.
So again, if we find all the relevant values, we can determine that our carbon carbon bond energy is 342 kilojoules.
And so on.
So we can keep going and again build up a bigger and bigger inventory of bond energies just to provide us with the kind of intuition that we'd like to have to be able to think about ordinary molecular energetics.
Any questions about bond energies?
All right.
Of course, if we know bond energies, we also could make estimates of heats of formation.
So if we have some new compound, we don't know its heat of formation, we know its structure.
We know what bonds are involved.
We could get an estimate for what we might expect for its heat of formation if we know those, and I think I'll just write this out on the board by example, but not work through the numbers explicitly.
You know, let's say we were going to make, you know, n pentane C5H12.
And of course again keeping proper track of all the carbon carbon and carbon hydrogen bonds.
But now in principle we know the carbon carbon bond energies and the carbon hydrogen bond energies.
So we should be able to figure out the heat of formation of a compound like this, even if we don't know it from a table.
And it's again through a similar sort of cycle so if we start with the atoms in the gas phase, and go to n pentane.
So this is now the bond energies which we basically know.
And again make our cycle of involving the elements in their stable form at standard temperature and pressure.
Then again, here's our one, two, three.
Here's our minus delta H of atomization, and here's our heat of formation.
And that's the point, is that now we can use what we've already elaborated for the bond energies and the known enthalpies of atomization, and we could determine the heat of formation.
If we do that, we wind up with a value of minus 152.6 kilojoules.
If we measure it.
If we look up in a table what the actual value is, we discover it's minus 146.4 kilojoules.
So it's a few percent off.
Why is it off?
Why isn't it right?
[UNINTELLIGIBLE]
Yes, exactly.
You know you can't use the C-H bonds from methane for every C-H bond, or the C-C bond in ethane for every carbon carbon single bond.
There'll be variations, usually of modest magnitude, in the values.
So this isn't going to be an exact calculation, but certainly this is close enough for allowing us to think qualitatively and semi-quantitatively about ordinary energetics.
Another illustration of it is if we look at neopentane -- so same chemical formula.
Same number of carbon carbon and carbon hydrogen bonds.
So according to the way we've done this calculation, we should get exactly the same number, because all we're doing is adding up the total number of bonds, and it's the same.
But what we discover is that for neopentane, delta H of formation, neopentane is minus 166.1 kilojoules.
So again it's different in this case in the other direction.
But certainly close enough that allows us to think reasonably about what might happen in ordinary situations.
Any questions about bond energies?
OK, now we're going to go to a new topic, and let me say, the topic we're going to start now, the second law, is really in a sense, this is the start of the whole rest of the term.
So far, what we've covered essentially is conservation of energy, how you think about energetics and what contributes to them.
That is heat and work.
How to account for them all properly.
What happens under constant volume or constant pressure conditions that led us to the definition of enthalpy right and so forth.
And once you get your arms around these ideas, and it takes some work to do that, but once you do, the actual execution of things can be pretty straightforward, and some of this can seem like just you know a lot of accounting.
It's not.
It takes some effort to become accustomed to it and familiar with how to properly think about heat and work and how they contribute to energy and so forth.
But given that amount of effort, a lot of this can be reduced to fairly straightforward calculational stuff.
What we're about to start is a little different from that in some sense, and really in some ways more difficult because what we're going to try to understand starting with this next set of topics is what determines whether something happens spontaneously?
We haven't really talked about that in any of this.
We've talked about whether reactions are endothermic or exothermic.
But as you're going to see that doesn't determine what happens spontaneously.
It certainly is one of the factors, but by no means the only one.
What determines what happens just by itself?
Or another way of saying that is, if I have some sort of system, it could be an engine.
It could be a chemical reaction.
It could be a biochemical system or some part of it.
What determines where the equilibrium is and the direction that things would have to go in order to reach or approach the equilibrium?
What is the equilibrium state of something?
Actually, what we've discussed so far really doesn't allow us to determine that, and there are a few easy ways to sort of bring that out.
One is you can imagine constructing some kind of cyclic process that you know it could remove heat like a refrigerator does from a cool region and export the heat to a hotter region, like the room, or outside a window if it's an air conditioner or something like that.
Just move the heat.
That's it.
Why not?
I mean you could do that and not necessarily violate the first law.
Or what about, here's a simpler case.
What if I just have a chamber with a divider, and I pump out half of it, and I put gas in this half, so there's gas and there's vacuum, and then I remove the divider.
So I certainly expect that what happens is the gas fills the volume.
Why?
The first law didn't tell me that that needs to happen.
Why doesn't it go back the other way, anyway, even without the barrier?
Why doesn't the gas just by itself all wind up over here?
Let's say it's a pretty dilute gas, so even if it does all end up in this half, you know the energy of any sort of unfavorable repulsions between molecules is negligible.
Maybe they even have weak attractions?
Why doesn't that happened?
It's a good thing for us that it doesn't happen.
Why in this room doesn't the air all just sort of collect over there somewhere?
Or just the oxygen molecules in the air and suffocate all of us.
The first law doesn't have anything to say about that actually.
Whatever the reason it doesn't happen, we're not going to find the answer to that in the first law.
So that's what the second law of thermodynamics and the development of entropy is all about.
It's to help us understand what is the direction of spontaneous change?
What governs that, and what's the equilibrium state that is reached when the spontaneous change occurs?
OK?
So the second law is going to give us basically a principle that'll tell us the direction of spontaneous change.
One way of thinking about that is it's going to allow it to sort of determine the direction of time.
You know if we see initial or if we see one state and we see another state, we'll be able or know, time must have been going this way, right because the spontaneous change would happen in this direction.
So, I'm going to put up a couple of just statements of the second law of thermodynamics.
I'll start with kind of an easy one.
So maybe I'll start with that here.
Here's a statement by Clausius.
The first law says that the energy of the universe is constant.
We're always conserving in one way or another.
And the second law is that the entropy of the universe is increasing.
Now we're going to define entropy presently.
But I'm writing this up just to give you some foreshadowing of how we're going to try to guide our thinking about spontaneous processes.
This tells us about conservation of energy.
This tells us about which direction things move in spontaneously, namely the direction that leads to an increase in this soon to be defined quantity entropy.
It's going to turn out to be a state function, and it'll help us understand the direction of spontaneous change.
You know, like the first law the second law too was developed before there was a very well defined understanding of the molecular nature of matter.
We're going to talk about the second law and entropy in macroscopic terms.
Later on in the semester we'll also discuss them in microscopic terms.
We'll do a little bit of what's called statistical mechanics, of microscopic, basically discussing the microscopic origins of entropy in terms of disorder of molecules.
But really, the thermodynamics of entropy and the second law evolved because people were trying to build things.
It was the Industrial Revolution.
people were trying to build engines, steam engines, and other kinds of devices, refrigerators other things.
And they couldn't understand -- first of all, why couldn't you make it perfectly efficient?
You know, you'd have a steam engine.
There'd be a source of heat.
You could extract work.
Never seemed like you could get it all and turn it all into work.
Why not?
And if not, what were the limitations on it?
What was the best efficiency you could achieve?
So in a very practical way people were motivated to think very hard about these kinds of questions.
So let's just look at some of the things that brought out the need for this kind of consideration.
So let me start by drawing one of the impossibilities.
We could try to build a heat engine.
Where we've got some temperature T1 which is hot, and some amount of heat, q1 goes into our system, and our system runs as a cycle.
It goes around and around, because in any practical situation we're going to need to keep going and repeating the process.
Some amount of work comes out.
We'll call it minus w, because work is always defined as the work that's done by the environment on the system.
So if I'm drawing the arrow this way, then I want to write it as minus w. Now this would be just swell.
This is it.
That's the whole heat engine.
This is running in a cycle.
So we know that, delta u is zero.
So that would mean that, you know, the amount of work out would just equal the heat in every time around.
Never happens.
Can't make it happen.
The only way it works is if instead there's an additional part.
Somewhere heat is also lost to a cold reservoir.
In other words, you know I burned some fuel.
I consumed something to produce my hot source, and I'm going to then try to turn it into work, which I can, but only partly.
There's going to be heat loss somewhere to a colder region, and I can't avoid it.
And it was discovered that this was always the case.
So this can be built where, just to be clear on these.
q1 is a positive number, because again that's heat going from the environment to the system.
This is our system running in a cycle.
So q1 is positive, minus w is positive, that is work is being done on the environment by the engine.
And minus q2 is positive.
And T1 is greater than T2.
That I can build.
Here's another thing we could try to do.
It's sort of the opposite.
We could try to produce a refrigerator but not have to do any work.
So let's go the other way, in other words, let's go from T2 which is cold we're going to go in this direction now, which means q2 will be a positive number and here's our system running in a cycle.
And now here's minus q1 now is going to be our positive number, and we're going to remove heat from q2 and move it, or from T2 a cold region and move it to some place that's hotter.
Also a pretty nice idea.
And also obviously impossible.
It only works if I put work in there to make it happen, then yes I can take heat out of a cold body and move it up into a hotter reservoir of some sort.
So here's a refrigerator.
Again T1 is greater than T2, and in this case q2 is greater than zero.
I'm removing heat, minus q1 is greater than zero, that is heat is going up this way.
Work is greater than zero.
Has to be.
Of course the thing is running in a cycle.
delta u is zero.
So that means that work must be minus q1 plus q2, right?
And work is greater than zero, so that's saying minus q1 right that's a positive number, must be bigger than q2.
In other words, I am taking some heat away from the cold region.
The heat I'm dumping into the hotter region is bigger than the amount that I taking away here, because it's also got this amount.
That's why, you know, if it's a humid or warm summer afternoon and you think oh what I a great idea, I've got this little fridge in my room.
I'm going to open it up and cool the room off.
Turns out not to work too well.
Because of course what the fridge is doing, it's cold inside.
So it's a machine that's removing heat from the inside and dumping it outside.
There are coils in the back of the fridge, and if you touch them they're a little bit warm and so on.
The heat is being lost to the room.
How much heat?
Well, the work that goes in is also added.
So the amount of heat that goes out into the room is bigger than the amount of heat that you're taking away from the space.
That's bad.
So if you try opening the refrigerator door to cool off your room at the same time as you're dumping heat into your room, all you discover is the net effect is to heat up your room.
Because you know the amount of heat that the thing is removing from somewhere on the inside there and struggling to do that because you've left the door to the fridge open, right, is not as big as the amount out heat that's coming out in those coils on the back.
It's not as big because that heat has not only the heat that you're removing from the inside, but also this amount of work.
It's all conserved because delta u is zero.
The thing is running in a cycle.
Try it sometime.
You can verify this experimentally in a very simple way.
It's not a very effective way to work, and it wastes a little bit of energy, but could be worth it in the interest of knowledge and discovery.
Let me write an alternate statement of the second law of Clausius.
All spontaneous processes are irreversible.
When I open that panel in the container, I've got gas on one side and vacuum in the other and it expands in there, it happens spontaneously, and it's irreversible.
It's not just going to go back.
And now let me write a mathematical statement of the second law.
And we'll celebrate it's importance by using this color.
Going around in a cycle, if I integrate the quantity dq reversible over T, I get zero.
Remember this, this is our special function.
Let me just keep going a little more though.
Let me also write another version.
If I write dq for an irreversible path over T, right, less than zero.
It's not the same.
This is a state function.
This isn't.
Remember of course that the heat that's exchange between the system and the environment depends on the path.
If there is a reversible path, and you see how much heat was exchange with the environment, and you look at that divided by temperature integrate over the whole thing, you discover that as long as the path is reversible, it only depends on the states at the beginning and the end of the path.
In other words if you have various ways you can go from state one to two that are reversible, it's always the same.
We saw that for just a couple of examples, a couple lectures back.
Remember, we looked at these thermodynamic cycles and just said, well let's just calculate those.
And those were, those all involved reversible processes.
So, in effect, we looked at different paths to go from one state to another.
We had a cycle, and one path went to here, and the other path went back.
We discovered that, in fact, this one zero around the cycle or the way we expressed it at the time is the value of this was the same regardless of the way we got from one state to the other, whether we took a straightforward route or a more circuitous route, as long as they were reversible.
We're going to define dS as dq reversible over T, and since we're dealing with a state function delta S going from state one to two of dq reversible over T is S2 minus S1 state function.
And S as state function is the entropy, and of course the us, it will always be special.
All right, now, the second law, you know it's difficult to understand and to get your arms around, because you know, do I really calculate this?
I actually have to find a reversible path in order to do the calculation.
Normally, you know, when you calculate delta u and delta h and so forth, you have various ways to calculate them that can be pretty straightforward.
But here, you actually have to determine the heat exchanged between the system and the environment, and you know that does, in general, depend on the path.
But it turns out that this quantity, you know, the differential of that heat over the temperature, also depends on path unless it's a reversible pass.
Any reversible path will lead to the same results.
It doesn't depend on path among reversible paths.
What that means in practice is if we say OK, start here and end here and calculate the entropy, you actually have to sort of figure out a reversible way of getting from here to here so that you can do that calculation.
So it's really very different from ordinary functions that you've been dealing with.
A lot of people, you know, because the second law is difficult to understand, you know you can easily go online and see thousands of violations of the second law.
It's still easy to look, to go find claims of perpetual motion machines.
Some of them cost a lot of money.
You can buy them, see how they work.
But and also, all sorts of schemes for producing energy.
Go online and see, right.
You're just, they're just crazy ideas about getting energy from nothing.
Getting energy from the heat that's just already present in the air.
I mean there's heat in the air, all this motion, it must have energy.
We should be able to extract it and use it.
Turns out we can't do that.
It has to look something like this.
OK.
I want to look at some more statements of the second law.
To do that I just want to indicate one definition which is just to be clear what these things mean.
These are what I've called heat reservoirs or hot or cold bodies.
Here's what I mean to be more specific.
A heat reservoir is basically a very large thermal mass at the same temperature.
And the real idea is that you can bring heat out of it as much as you want and the temperature won't change.
It's an idealization.
right If you have something that's big enough, and it's at a constant temperature, of course you can take heat away, but it's just so enormous that you won't measurably change it's temperature.
In practice you might be able or approach that limit.
OK, now let's look at some more definitions or statements of the second law.
This is from Kelvin of Kelvin temperature scale fame.
It's impossible to convert heat into work with a system that operates in a cycle.
I'm going to write this all out.
I know you've got it in your notes too, but it's super important.
To convert heat into work with a system that operates in a cycle, without at the same time transferring some heat to a colder reservoir.
OK, a couple of things to notice.
One is, you know, in some sense the second law is kind of negative, right?
Must not have been done in the new age.
It tells us what we can't do.
And that really is in a sense what it's all about.
It's telling us what are our limitations of what can be accomplished.
Very important part of this statement -- in a cycle.
Of course any practical engine has to keep going, so there's going to be a cycle in some way or another, but it's important because, you know, there are some -- if I'm not operating in a cycle, I can convert heat into work just fine.
You know, I could have a heat source, got a candle, and I've got a piston, maybe I've got a weight on it.
I've got something stopping it.
And then I remove the stoppers and let it go, and because of the heat, there's expansion.
It pushes up, right, and it'll be up higher somewhere.
Worked!
Heat turned into work.
I don't have a cold body somewhere.
But I only did this once.
There was just one stroke of the piston.
If I want to continue it and run in a cycle, somehow I've got to have a place where the heat goes.
So this sort of diagram describes effectively the way the Kelvin statement works.
This works fine and like we illustrated before, it doesn't work fine if I don't have this part of it.
I won't be able to just take all the heat and convert it to work, like I, in principle, could do, if I don't need to run it in a cycle.
Here is yet another statement by Clausius And we'll end with that one.
Maybe I'll just read this one, so we can not go over the time.
So, it's impossible for any system to operate in a cycle that takes heat from a colder reservoir, transfers it to a hotter reservoir, without at the same time converting some work into heat.
In other words, it's a statement kind of similar to the Kelvin statement, but applying to the case of moving heat from a colder to a hotter region.
Can't do it, unless some work is also converted to the heat that gets released into the hot region.
I think we'll stop there.
Next time what we'll do is lay out a very well-defined kind of engine called Carnot cycle that allows us to very explicitly go through and say OK, let's just work out the steps of isothermal and adiabatic expansions and contractions.
In other words, how does the engine really work?
And just calculate it going around in a cycle and see exactly how all this is born out.
And we'll see that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCouseWare at ocw.mit.edu
Well, last time we started in on a discussion of entropy, a new topic, and I started out by writing out some somewhat verbose written descriptions that had been formulated that indicate certain limitations about things like the efficiency of a heat engine, and what can be done reversibly and so forth.
And these verbal descriptions lead to some pictures that I put up and I'll put up again about how you might try to accomplish something like run an engine or move heat from a colder to a warmer body.
And what some of the limitations are on things like that.
I'll show those again, but what I want to do mostly today is try to put a mathematical statement of the second law in place that corresponds to the verbal statements that we saw last time.
So, just to review what we saw before, we looked at a heat engine where there is some hot reservoir at some temperature, T1, and it's connected to an engine running in a cycle.
We could, write the work that we generate that comes out as negative w. And then there's a cold reservoir at some lower temperature T2.
So the way I've got things written here, q1 is positive.
Heat's flowing from the hot reservoir to the engine that runs in a cycle.
Work is coming out.
A positive amount of work is being generated and is coming out, so minus w is positive, and this minus q2 is positive, that is heat is also flowing into a cold reservoir.
And what we saw last time is that this was necessary to make things work, this part of it in particular.
You couldn't just run something successfully in a cycle and get work out of it, using the heat from the hot reservoir, without also converting some of the heat that came in to heat that would flow into a cold reservoir.
All the heat couldn't get successfully converted into work.
That would be desirable, but it's not possible.
And, similarly, if we try to run things backward and build a refrigerator, so now we have our cold reservoir, and we're going to remove heat from it, and pump it up into a hot reservoir.
And to do this, we saw that you have to put work in, right, you can't just remove heat from a cold reservoir, move it up to a hot reservoir, without doing some work to accomplish that.
So here, q2 is greater than zero.
The work is greater than zero, and minus q1 is greater than zero, that is heat is flowing this way.
So these are just the pictures that we saw last time.
And then we had some statements of the second law of thermodynamics that I won't re-write up here, but, for example, Clausius gave us the statement that it's impossible for a system to operate in a cycle the takes heat from a cold reservoir, transfers it to a hot reservoir, that is acts like a refrigerator, without at some, at the same time, converting some work into heat.
Work has to come in to make that happen.
And we had the similar statement by Kelvin about the heat engine that required that some heat gets dumped into a cold reservoir in the process of converting the heat from the hot reservoir into work.
Fine.
Now what I want to do is put up a specific example of the cycle that can be undertaken inside here in an engine, and we can just calculate from what you've already seen of thermodynamics.
What happens to the thermodynamics parameters, and see the results in terms of the parameters including entropy.
So let's try that.
So, the engine that I'm going to illustrate is called a Carnot engine.
And the cycle it's going to undertake is called a Carnot cycle, and it works the following way: we're going to do pressure volume work.
So this is something that, by now, you're pretty familiar with.
So we're going to start at one, go to two and this is going to be in isotherm at temperature T1, and all the paths here are going to be reversible.
So that's our first step.
Then we're going to have an adiabatic expansion.
So this is an isothermal expansion.
Here comes an adiabatic expansion, to point three.
So at this point the temperature will change.
Then we're going to have another isothermal step, a compression to some point four.
So this is an isotherm at some different temperature T2, a cooler temperature, because this was an expansion.
We know in an adiabatic expansion the system's going to cool.
And now we're going to have another adiabatic step, an adiabatic compression.
And that's it.
That's going to take us back to our starting point.
So that's our cycle.
Of course there are lots of ways we can execute the cycle, but this is a simple one, and these are steps that we're all familiar with at this point.
So this is the picture, and this is a cycle that undertakes it.
So let's just look step by step at what happens.
So going from one to two, it's an isothermal expansion a T1, so delta u is q1, we'll call it, plus w1, for the first step.
Going from two to three, that's an adiabatic expansion, so q is equal to zero in that step.
And delta u is equal to what we'll call w1 prime.
So we'll use w1 and w1 prime to describe the work involved.
The work input during the expansion steps.
Step three to four isothermal compression, delta u is going to be q2 plus w2.
And finally, we're going to go back to one.
We're going to close the cycle with another adiabatic step.
q is zero.
Delta u is w2 prime.
Now, the total amount of the work that we can get out is just given by the area inside this curve.
We'll call it capital W. It's just minus the sum of all these things, because of course these are just defined as the work done by the environment to the system.
So it's minus w1 plus w1 prime plus w2 plus w2 prime.
The heat input is just q1, and we'll define that as capital Q.
And I just want to write those because what I really want to get at is what's the efficiency of the whole thing?
And in practical terms, we can define the efficiency as the ratio of the heat in to the work out.
We would like that to be as high as possible.
So it's just capital W over capital Q, which is to say it's minus all that stuff over q1.
Now the first law is going to hold in all of these steps, and we're going around in a cycle.
So what does that tell us about a state function like u?
What's delta u going around the whole thing?
I want a chorus in the answer here.
What's delta u?
Zero
Excellent.
MIT students, yes!
OK, so the first law, we know that the, going around the cycle the integral of delta u is zero.
And that has to equal q plus w, summed up for all the steps.
Which is to say q1 plus q2 is equal to minus w1 plus w1 prime plus w2 plus w2 prime.
So that means we can rewrite this efficiency.
We can replace this sum by just the some of q1 plus q2.
So sufficiency is q1 plus q2 over q1.
Or we can write it as one plus q2 over q1.
Now that already tells us what we know, which is that the efficiency is going to be something less than zero, right?
Because we've got one and then we're adding q2 over q1 to it, but we've got negative q2 is a positive number.
Heat is flowing this way.
So q2, the heat in this way is negative.
q2 over q1 is negative.
q2 of course is positive.
That is heat flowing from the hot reservoir to the engine.
So that efficiency is something less than one, and we'd like to figure out what that is.
So let's just state that.
Well, now let's go back to each of our individual steps and look, based on what we know about how to evaluate the thermodynamic changes that take place here, let's look at each one of the steps and see what happens.
So from one to two it's isothermal.
And now we're going to specify, we're going to do a Carnot cycle for an ideal gas.
It's an ideal gas.
It's an isothermal change.
What's delta u?
You know, it's like lots of courses and all sorts of things that you're seeing, they're always the same.
What delta u?
Zero.
PROFESSOR NELSON?
Right.
Delta u is zero, and it's also equal to this.
So that says the q1 is the opposite of w1.
It's an isothermal expansion, so dw is just negative p dV.
So it's, this is just the integral from one to two of p dV.
And it's an ideal gas, isothermal, right.
RT over V. Were going to make it for a mole of gas, so it's R times T1, and then we'll have dV over V. So that's just the log of V2 over V1.
Let's have one mole, n equals one.
Okay two going to three that's this, it's adiabatic, right.
So delta u is just equal to the work but we also know what happens because the temperature is changing from T1 to T2.
So delta you is just Cv times T2 minus T1.
du is Cv dT.
It's an ideal gas, and that's equal to w1 prime.
Now, this is a reversible adiabatic path, so there's a relationship that I'm sure you'll remember.
Namely, T2 over T1 is equal to V2 over V3.
Don't be confused by the subscripts.
We're talking about quantities both starting at two and going to three.
So it's V2 and V3.
They're different volumes.
The temperature though at two is the same as it was at one, so it's still a temp at T1, and this temperatures is at T2.
Anyway T2 over T1 is equal to V2 over V3 to the power gamma minus one.
So we're going to kind of store that for the moment.
Now going from three to four right, so we have another isothermal process for an ideal gas, so I won't try to make you sing again so soon.
Delta u is zero.
And so just like here, now q2 is minus w2, that's integral going from three to four p dV.
So it's R T2, right, now we're at a lower temperature times the log of V4 over V3.
So that's our work in this path and heat.
And finally going from four back to one.
So we've already seen that q is zero.
So we know that delta u is just Cv times T1 minus T2.
Now we're going from T2 up to T1.
And this is equal to w2 prime.
And now, just like we had before, again we've got a reversible adiabatic path, so T1 over T2 has to equal V4 over V1, to the gamma minus one, okay.
All right, now, if this is the case, of course, this is just the inverse of this.
So this just must be the inverse of this.
We can combine these two to see that V4 over V1 must equal V3 over V2.
So now we have a relationship between the ratios of these volumes that are reached during these adiabatic paths.
Now, let's just look at our efficiency.
So we saw that efficiency is one plus q2 over q1.
Let's just use our expressions that we've found for q2 and q1.
We're going to have q2 over q1.
R is going to cancel.
We're going to have the ratio of temperatures and the ratio of these logs.
So it's T2 over T1 times the log of V4 over V3, over the log of V2 over V1, but we just arrived at this relationship between these volumes.
And this of course tells us that V4 over V3 is V1 over V2.
Right.
I'm just inverting these.
So here it is.
Here's the V4 over V3, oops, sorry.
V2 over V1, this is equal to the inverse of this.
So the ratio of the logs is just minus one.
So this is just, sorry, I forgot about the one here.
One plus this, but this is the same as one minus T2 over T1.
That's our expression for the efficiency.
All right, isn't that terrific?
Now, we have an expression.
We can figure out the efficiency.
All we need to know is the temperatures.
What's the temperature of the hot reservoir?
What's the temperature of the cold reservoir?
We're done.
That's the efficiency.
As we expected, it's less than one, and of course if we build an engine, we want it to be as high as possible, as close to one as possible.
What that means is we want to run the hot reservoir as hot as possible and the cold reservoir as cold as possible.
In principle, this value, this efficiency, can approach 1 as the low temperature approaches absolute zero.
If this were to be an absolute zero Kelvin, then we could we can have something, wait a minute.
Sorry, it's T2.
As T2 goes to zero, the cold reservoir, then this goes to zero and our efficiency approaches one.
So that would be the best we can do.
And that basically make sense, right?
The whole thing is being powered by sending heat from the hot to the cold reservoir.
The colder that cold reservoir is, the hotter that hot reservoir is, the better off we are.
The more work we can get out of it.
The closer the efficiency will get to one.
So in some sense, the first law would suggest you can sort of break even.
That is, it gives us the relationship between energy and work and heat.
It might suggest that you could convert all the work to heat.
The second law says that really, you can't do that.
The only way you could possibly contemplate it is to be working at absolute zero Kelvin.
Guess what the third law is going to tell us?
You can't get to zero Kelvin.
We'll see that shortly.
But this is those closest you could come at least by trying to do that to having your efficiency approach one.
Now, we've seen that q2 over q1 is equal to negative T2 over T1.
So let me just rewrite that as q1 over T1 is minus q2 over T2.
Or in other words, q1 over T1 plus q2 over T2 is zero.
But q1 and q2, each one of those is just the integrated amount of heat that was transferred going along a reversible constant temperature path.
Which means that q1 over T1, that's this delta S thing that we saw before.
And what we can say about this is it's saying that if we go around in a cycle and look at dq reversible over T it's zero, because that's what these quantities are.
Now, of course, we can run the engine backward and build a refrigerator, and if you've got your lecture notes from last period, your 8-9, well, they're labeled 8-9 lecture notes, I made an attempt to define something which was a little bit misguided.
And so instead of defining efficiency the way you've got it written there, I'm going to define what's called something different for a refrigerator which is called the coefficient of performance.
So it's the following: so we can define the coefficient of performance written as eta as q2 over minus w. In other words, you know, it's how much heat can we pull out of the cold reservoir, for whatever amount of work that we're going to put in in order to do that?
Well of course, we'd like that to be as big as possible.
But of course, this is just q2 over minus q1 plus q2.
Just to be clear, so it's heat extracted over the work in.
So let me just rewrite that as, I just want to divide by q1 everywhere.
So I'm going to write this as q2 over q1 over minus one plus q2 over q1.
I want to do that because we know that q2 over q1 is negative T2 over T1.
So this is negative T2 over T1 over negative one minus T2 over T1.
I can cancel those, and then I'm going to multiply through by T1.
So this is just going to be T2 over T1 minus T2, that's our coefficient of performance.
So it's a measure of how well we can do to run the refrigerator.
So you know, what the second law is doing, in words, it's putting these restrictions on how well or how effectively we can convert heat into work in the case of the engine, or work into heat extracted in the case of a refrigerator.
And it follows qualitatively from just your ordinary observations about the direction in which things go spontaneously, right.
You know the heat isn't going to flow from a cold body to a hot body without putting some work in to make that happen.
And so, we'll be able of follow that further and see really how to determine the direction of spontaneity for a whole set of processes, really in principle for any processes, went analyzed properly.
So the first step to doing that is I want to just generalize our results so far for a Carnot cycle.
You might think well okay, but this is a pretty specialized case.
We've formulated one particular kind of engine, and seen how we can analyze what it does, come up with relations that seem of value for efficiency and other quantities.
How general is it?
Well, let's take a look.
So, what I want to do is make a new engine which really just consists of two engines, side by side.
One is our Carnot engine as we've seen it, and the other is just any other reversible engine.
So generalize our Carnot engine results.
So let's take our hot reservoir and draw it bigger.
We know it has to big anyway, since we can extract heat from it without changing the temperature.
And on this side, we're going to write out an engine, and we're going to say this is a Carnot engine.
so on the side of it we have q1 prime.
We're going to have w prime.
And we're going to have q2 prime, and we're going to draw the arrows in the positive direction in these cases.
Or in the defined directions I should say.
Here is our colder reservoir.
OK, so here is just an engine like what we've already seen, and I'm going to specify that this is a Carnot engine which is to say all the results that we just derived hold for this case.
And side by side, we're going to run another engine.
So it's got q2, and it's got a cycle w. And it's got q1.
So this is some other engine that runs using reversible processes.
So I can define the efficiency of each one of them.
The efficiency for the one on the left is minus w over q1, The efficiency prime for the one on the right is minus w prime over q1 prime.
Now I want to just assume that in some way they're different.
So let's assume that epsilon prime is greater than epsilon.
So in other words, this engine is running less efficiently than my Carnot engine.
It's also reversible.
And now since it's reversible, we can run it forward or backward.
So we can run this one backward and we can use the work that comes out as the input, well sorry, use this work that comes out of the one of the right to run this one backward, which is to say we'll move heat from cold to hot.
So use work out of right-hand side to run left-hand backward.
Why not?
We need work to come in, we might as well get it from here.
So, the total work that we can get out must be zero, out of the whole sum of them.
After all, we're taking the output work that we get from the right, and using it all to drive the left.
So that means that minus w prime must equal w. And w is greater than zero.
that is in this one we have the environment doing work on the system.
OK, now we've assumed that epsilon prime is greater than epsilon.
So we can just write out what those are, minus w prime over q1 prime is greater than minus w over q1.
But we know that minus w prime is the same thing as w. So w over q1 prime is greater than minus w over q1.
Which is the same thing just to be a little bit maybe pedantic because w over minus q1, and the only reason I'm writing that is to illustrate that this says that q1 must be less than q1 prime.
So q1 is less than zero.
Remember this one's running backward, we're pumping heat up.
q1 prime is greater than zero, it's running as an engine.
It's taking heat from the hot reservoir.
Well, that's interesting.
That says that minus q1 prime plus q1 is greater than zero.
Well, it's a pretty interesting result.
There's no work being done on, out of the whole thing.
This is providing work that's being used in here, but if you take the whole outside of the surroundings and this whole thing is the system, no net work, these things cancel each other, and yet heat's going up.
How did that happen?
Well it happened because we clearly must have a faulty assumption underlying what we've just done.
This can't possibly be true.
This is impossible.
So what this says is the efficiency of any reversible engine has to be one minus T2 over T1.
It's not a result that's specific to the one cycle that we put up.
And you know, you could have a reversible engine with lots and lots of steps, but you could always break them down into some sequence of adiabatic and isothermal steps.
So you know your cycle, you know, you could have a whole complicated sequence on a p v diagram of steps going back.
As long as it's reversible, you know what the efficiency has to be, and in principle, you could break it down into a bunch of steps that you could formulate as isothermal and adiabatic.
They might have to be formulated as very small steps, in order to do that, but you could.
What this says too, is this result that we found, right, now of course that's our integral dS is zero.
It's general.
Entropy S is a state function, generally.
Now remember, we went through before how it's a state function but to calculate it, you'd need to find a reversible path, along which you can figure this out.
But the fact is it's a state function, in a general way.
Now, if we go back to our Carnot cycle which is a set of reversible paths, it's useful to compare this to what happens in an irreversible case.
In a real engine, of course, you can approach the reversible limit.
Every step won't be perfectly reversible.
And of course it's not hard to see what happens.
Let's just take our reversible engine and modify it a little bit.
Let's imagine that instead of all the steps being reversible, let's just put in one irreversible step.
Let's do this.
Instead of this reversible isothermal step, Let's make it an irreversible isothermal step.
We can have a different isothermal step.
Of course in the reversible case, you're always pushing against an external pressure, which is essentially equal to the internal pressure.
Let's not do that.
Let's imagine maybe instead we just immediately dropped the pressure and let the system expand against the lower pressure.
Now we know we're going to get less work out of it in that case.
You've seen that before.
And the work in this case is the area inside here.
The work in this step is just the area under this curve.
So in some way we're going to have a difference here between the irreversible case and the reversible one.
So if we do that, then what I want to do is just see what happens to dq over T, so in our irreversible engine, one to two, what we know for sure is that minus w in the irreversible step, that's the work out, extracted in that step, is going to be smaller than minus w in the reversible case.
So w irreversible is bigger than w reversible.
And of course, in either case, delta u is q plus w, so it's q irreversible plus w irreversible, but of course, u being a state function it's the same in either case.
So it's the same as q reversible plus w reversible.
And we've just seen that this is bigger than this, but the sums are equal.
So this has to be less than this.
q irreversible is less than q reversible.
In other words, and irreversible isothermal expansion takes less heat from the hot reservoir than a reversible one does.
It makes sense, right, because you know we got less work out and delta u is the same right, so it must be that less heat got transferred.
So the expansion against lower pressure draws less heat from the hot reservoir right.
So now let's look at the efficiency of our irreversible engine.
So it's one plus q2.
Now q2 was in this step, and we're going to leave that reversible, right, but q1 is irreversible.
And this has got to be less than one plus q2 reversible over q1 reversible, which is to say it's less than our efficiency in the reversible case.
Why?
This is smaller than this, but it's a negative number.
So this negative number has a bigger magnitude than this negative number.
We're subtracting them from one and they're less than one, so this is bigger than this.
And all we know about this is that it's really for some irreversible reversible step.
All we really needed to know about it is that this is smaller right.
So it's going to be the case for any irreversible engine.
And that's the point.
So it's that the irreversible efficiency is lower than the reversible one.
But, of course, since we saw that this is smaller than it was in the reversible case, we can also write that dq irreversible over T is less than dq reversible over T. Which is to say if we go around a cycle for dq irreversible over T, that's less than zero.
It's only in the reversible case that dq over T around a cycle is equal to zero.
So to write that in a general way, it's actually formulated by Clausius.
Going around in a cycle the integral of dq over T is less than or equal to zero.
Never greater.
OK, what we'll see shortly is that this will allow us to see that for an isolated system the entropy never decreases.
It only can go up.
And in fact any spontaneous process will make it go up.
Only in the case of reversible processes.
Of course, then you can see that this will be zero.
Anything else, which is to say any spontaneous process, it'll be less than zero.
We'll see that next time, and then we'll generalize in a broader sense to look at the direction in which spontaneous processes go.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Over the last couple lectures through our consideration of a number of somewhat difficult topics, including these Carnot cycles and specific Carnot engine to represent them, we've tried to define and understand a little bit about this very special function, this entropy.
Where we've got our dS.
It's dq reversible over T. And our treatment of reversible and irreversible cyclic processes, what we saw is that if we go around a cycle and look at dq reversible over T, then this is zero.
Now go all the way around a cycle, of course, this is just delta S for the cycle.
And that's consistent with the idea that entropy is a state functions.
So if we go around cycle the cycle, we have the same ending point, the starting point, same state, then the change in the state function has to be zero.
But we saw that for an irreversible path around a cycle, then we have that, this gives something less than zero.
And so that was expressed in Clausius inequality that includes both these cases.
And now I just want to go through a number of calculations of entropy.
How entropy changes for a number processes, both to just learn more about it in general terms and also just to see how it's calculated and what sort of changes in undergoes for different sorts of changes in state.
So the first very general example that I want to show is the following.
Let's say we've got an isolated system, and it undergoes some irreversible change.
So we'll start at one, and we'll go irreversibly to some new state, two.
OK, now we can always return the system back to the initial state reversibly.
If we do that, then we can't keep the system isolated.
So it won't be isolated anymore.
In other words, in order to arrange things so that you could have a return along a reversible path, it can't be isolated.
In some sense you know that, because when it was isolated it spontaneously irreversibly went from one to two.
And example would be ice melting at some temperature that might be higher than the freezing point.
So you've got solid ice, but the temperature is above zero degrees Celsius, so irreversibly it'll melt.
Systems isolated, it'll melt.
Then you could put the system in contact with a cold reservoir reversibly re-freeze the water to form ice again.
But of course to do that you couldn't keep it isolated.
You need to have it in contact with a low temperature bath.
But again, this is just completely general at this point.
It's some irreversible process returned then along a reversible path.
So let's just look at what happens.
So here's our path B.
Along path A, well it's an isolated system.
So the heat exchanged is zero.
And then, for the cycle based on the Clausius inequality expressed here, the integral around the cycle of dq over T in general, it's less than or equal to zero.
If there was an irreversible process involved, then in fact, it'll be less than zero.
I'll write it here in the more general case, but we know what it's really going to be with the irreversible step.
Well, so we can write this as the integral of dq irreversible over T, going from state one to state two.
Oh, let me rewrite this down here.
So around the cycle, dq over T, it's integral from state one to two of dq irreversible over T, plus the integral from two back to one of dq reversible over T. And of course, we know that that's less than or equal to zero.
But this, we know is zero, right, because this irreversible step is taken in isolation.
The system is isolated.
We've already seen that there's no heat crossing the boundary of the system.
So we just have this.
So of course this dq reversible over T from two to one, that's, what now we've a reversible path, so that's just the entropy at the delta S going from two to one.
It's S1 minus S2 it's delta S, we can write it as delta S backwards, right, we're going back along that reversible path.
Which is of course negative delta S forward.
Because around the whole cycle delta s has to be zero.
So what that says is delta S forward is greater than or equal to zero.
That's interesting.
What that's saying is -- remember, we didn't specify the process.
It's just for any completely general irreversible process.
For any such process, for any spontaneous process, this is going to be the case.
So in other words, for an isolated system, delta S tells us the direction of spontaneous change.
In particular, for the isolated system, delta S is greater than zero for something that's irreversible.
Delta S equals zero for something that's reversible, and delta S is never less than zero.
In a sense, this is a direct consequence of the Clausius inequality.
That's a pretty useful result.
So remember when we started this whole discussion, I tried to emphasize that the first law of thermodynamics told us about conservation of energy.
But it didn't tell us which way something would go spontaneously if you just left the system to its own devices under certain conditions.
This is telling us that.
We can tell which way the system will evolve in which direction it'll go.
This is for an isolated system and we'll generalize this in subsequent lectures.
Now, let's look at what happens if the system isn't isolated.
So now let's go from one to two not isolated.
So of course, delta S I'm going to specify for the system for reasons that'll soon be obvious is S2 for the system minus S1 for the system.
Of course it doesn't depend on the path, even though to calculate it, we need to find a reversible path.
Just to add emphasis to what we just did here, of course, it was the same thing.
In order to calculate delta S along that irreversible path for that irreversible process we needed to construct some reversible path.
And what we did is imagined the reversible path going backwards and then said, OK, fine then delta S for the irreversible process must be the opposite of delta S that we calculated along the reversible bath going backwards.
The point is that to calculate delta S in either direction, we needed to construct some reversible path.
All right, and of course it'll be the same here.
Now, in this case since the system isn't isolated, that means heat's going to flow across the boundary.
So that means heat is going between, it's being exchanged between the system and the surroundings.
So what's happening to the surroundings?
Generally, we've worry a lot about the system, but not much in recent discussions about the surroundings.
But delta S surroundings must be something.
We could calculate it.
And that's depends on the path.
Not because it isn't the state function in the surroundings, of course it is, but because depending on the path, the final state of the surroundings will be different.
After all, depending on how much heat got exchanged, the surroundings got more or less of that heat.
Work might have been done on the surroundings.
So the surroundings are going to respond, are going to change also, when the system changes in this way and it's not isolated.
And how the surroundings change will depend on what path was taken.
And you've seen that before in calculating things like pressure, volume, work along different paths.
Where you calculated delta u for the system and saw that it was the same for different paths but q and w were not the same for the system, and therefore, of course, not for the surroundings either.
So let's just consider an irreversible path.
And in order to think about what happens to the surroundings, let's take the entire universe which consists of the system and the surroundings -- that is a big isolated system, because it's everything.
So it's not somehow in contact with another additional body or universe.
We can treat the entire universe as an isolated system.
So, we're going to think about the entire universe.
We can think about it as an isolated system.
Great.
And we know that delta S for an isolated system, for a spontaneous process, is greater than zero.
That's right there.
So delta S for the universe which is delta S for the system plus delta S for the surroundings, and I'll specify since we're considering an irreversible process, specify that, it's greater than zero.
The whole thing happened spontaneously, and the whole universe is an isolated system.
So if we wanted to, we can write that delta S irreversible for the surroundings must be bigger than minus delta S for the system in order that there sum be positive.
All right?
Now, if we consider a reversible process, then of course we already know what happens for an isolated system in a reversible process, delta S is zero.
So delta S in this case of the whole universe, which is delta S for the system plus delta S for the surroundings in the reversible case must be equal to zero.
In other words, delta S of the surroundings in the reversible case is exactly the opposite of delta S of the system.
Delta S of the system is the same in either case because reversible or irreversible, we're specifying the system, goes between the same states, but what happens to the surroundings is different in the two cases.
So one way of recasting this statement or the Clausius inequality is the entropy of the whole universe never decreases.
Of course, in some sense, that's obvious from this statement, since we're considering the universe an isolated system.
So the entropy in any process can, it can, the change in entropy can either be zero or positive and never negative.
So entropy off the whole universe never decreases.
That's actually a really quite profound conclusion.
All sorts of philosophical and other consequences of it that are sometimes considered in things like evolutionary biology and other things.
And how things can change altogether an entire system of things.
Now, I'd like to just go through a few calculations of delta S for common processes.
We've seen a few general features of entropy.
Let's calculate it for a few things.
So calculations of delta S. And we know the general procedure.
Whatever the process is, we're going to need to find reversible paths, and along a reversible path then, we'll be able to calculate it.
One, let's just take two pieces of material at different temperatures.
The entire system is going to be isolated.
And then let's connect them.
And then let's connect them.
Put some material between them.
So now heat can flow from one to the other.
And you know what's going to happen which is if they start out at unequal temperatures, and you wait a little while, eventually they're going to wind up at the same temperature.
In other words, you know the direction of spontaneous change.
So let's just see what happens to delta S. So isolated system, so there's no work being done.
There's no heat that's being exchanged.
So delta u is zero.
What's dS?
Well it's just the sum of the changes in entropy for the two sub-systems the two separate pieces.
So it's dq1 over T1 plus dq2 over T2, I think there's a type in your notes.
it's plus not minus of course.
But in this case, dq1 is just the opposite of dq2.
Because whatever heat is flowing into T1, into this block, is coming out of this block, or vice versa, depending on which one is hotter.
So this is just equal to dq1 times one over T1 minus one over T2, which is T2 minus T1 over T1 times T2.
So that's dS, and then we could integrate it to get the delta S, but this is sufficient for what I'd like to illustrate, which is just the sign of things.
We know dS must be greater than zero for the spontaneous change that's going to occur.
So let's just make sure that common sense prevails here.
That says if T2 is greater than T1, right T2 is hotter.
Then dq must be positive, so this is positive, this is positive, dS we know has to be positive.
So, it says T2 is hotter.
This being positive means heat is flowing in from this one to this one, from the hotter to the colder body.
So it looks right.
If T2 is lower than T1, if this one is colder, then in order for dS to be positive, which it has to be, this better be negative.
In other words, again heat is going to flow from the hotter toward the colder, into the colder body.
So, as promised, the condition that dS must be greater than zero, guides us and tells us the direction that spontaneous change is going to occur.
We can tell just from this condition which way things have to evolve.
All right, let's try another pretty simple process.
Let's just take a gas in some volume V and over here is going to be vacuum of equal volume, and we're going to remove the barrier.
You know what's going to happen spontaneously, right?
The gas is going to fill the available volume once it becomes available.
So this, we'll make this a Joule expansion, and we'll make it an ideal gas.
So the process is one mole of gas at our initial volume and some temperature, and we'll make this adiabatic, and this goes to one mole of gas at a new volume, 2V and at the same temperature.
All right, so there's no work done.
It's adiabatic.
There's no heat exchanged.
Delta u is zero.
Everything's zero.
Almost everything is zero.
What isn't zero?
Entropy
Entropy.
It's going to tell us which way this whole thing goes.
So this is an irreversible process.
In order to calculate delta S we need to construct a reversible path along which this can go.
So here's a way to do it.
We could compress it back, isothermally and reversibly.
Just like the example I gave, the general example, where I mentioned the ice melting.
In that case, it won't be adiabatic.
We'd have to put it in contact with a heat source of some sort.
So reversible process.
We could compress it back to volume V, isothermally, and reversibly.
Let me say, there's no -- we wouldn't have to consider this process in reverse.
Of course, we could also consider the forward process in the presence of some heat source with isn't isolated and do this, but let's just consider the compression.
So, OK, now we'll have the -- well, once again, delta u is zero.
It's an ideal gas.
There's no temperature change.
But this time q and w aren't going to be zero.
They're going to be some finite numbers that are opposites of each other.
Because we're no longer isolated.
So let's just write out delta S to go backward is dq reversible over T, and that's minus dw over T, not zero anymore.
And where we're going is from 2V to V, we're compressing, and it's an ideal gas.
So this is the same as p dV, but it's p dV over T which is the same thing as going from 2V to V. Now we're going to replace this with RT over V, and the T's will cancel.
So it's R dV over V going from 2V to one V. So it's just R times the log of 1/2.
So now we've just calculated the value of delta S by constructing this reversible path.
Now this is delta S backward.
To do the compression, so of course delta S forward is just the opposite of that.
So, delta S forward is R log 2.
Reassuringly, that's a positive number.
So this process that we considered originally, this irreversibly process in this isolated system, remember, for the irreversible expansion we considered the system isolated, Happened spontaneously, and of course you know that's the case.
Now, I'll just mention, and later on we'll go into this in considerably more detail, but I'll just mention at this stage through a little bit of sort of foreshadowing that everything we've discussed so far about entropy is with a macroscopic picture.
We started with heat engines and Carnot cycles and macroscopic processes, reversible and irreversible processes and so on.
But there's a, and that's certainly the way it was all formulated originally.
Part of the power of thermodynamics is that it doesn't depend on a specific microscopic model of matter.
That said though, we have a pretty good microscopic model of matter.
We know about atoms and molecules, and in fact there is an entirely microscopic formulation of entropy that has to do with disorder and the number of available states, microscopic states available to a system.
So I'll just state what we'll see in much more detail later on.
Which is in microscopic terms, it's going to turn out that the entropy of a system can be given by R over Avogadro's number times the log of the number of distinct microscopic states that are available to a system.
Right so in this isolated system, when we double that volume, each molecule suddenly has twice as many states available to it.
You could formulate that in a lot of ways.
You can sort of divide up the volume into tiny little molecule-sized cubes and realize well, now, there are twice as many of them.
Now if I have one molecule, the number of states doubles.
If I have n molecules, the number of states goes up by 2 to the n. Because for each one of the molecules, I can select any one of those states, and then the next one I can select any one.
So you have this enormous increase in the number of states.
So delta S in this case looks like R over NA log of 2 to the power NA, it's Avogadro's number, for a mole of molecules.
This can come out and cancel this.
There's our R log 2 result.
But the point I'm making here is that could actually be derived from entirely microscopic consideration.
We haven't gone through that yet.
We've done entirely macroscopic thermodynamics so far, but we'll do a little bit of treatment of statistical mechanics, is what is called the whole microscopic formulation of entropy in thermodynamics.
And that's a really important fundamental result.
And we'll get to it in much more detail in a while.
But for now, I just want to continue calculating entropy changes for a few more kinds of processes.
So, how about if we have two different substances and we mix them, entropy of mixing.
So, we start with some number of moles of substance A in some volume VA, and some other number of moles of substance B in volume VB, separated.
And then we remove the barrier, of course we know they're going to mix.
So we'll have some total number of moles, nA plus nB, and they'll be filling some total volume, which is the sum of the two original volumes.
Certainly this is what we expect to happen spontaneously.
So are process is nA moles of A, gas, initial volume VA and T, plus nB of substance B, I think this is miswritten as A there in your notes.
Gas VB and T goes to nA moles of A plus nB moles of B.
Gas, total volume and T. And we'll have constant temperature and pressure.
Well, there was no p v worked done.
The temperature didn't change and it's ideal gases, so delta u is zero, but what about delta S?
Well again, to calculate it, we need to find a reversible path.
This is irreversible if we just remove the barrier and let it all go.
But we could at least imagine a reversible path, and we could actually construct this for the right substances.
We could imagine that we can find some sort of piston here.
And we're going to prepare to push it inward, with a membrane that's permeable only to b.
And over here we could set up the exact same thing, but this one's permeable only to a.
And then we could just push them to recover this state.
And we can do it reversibly.
So we could construct a reversible process that does the reverse of this.
And in this case, it'll be isothermal still, constant pressure, reversible compression of the two gases.
So, reversible compression and demixing.
All right, so what happens?
Well, of course, our delta S of demixing is going to be minus delta S of mixing.
Delta u of demixing is still zero.
Temperature didn't change, ideal gases, and there's some reversible work and heat.
So dw in this reversible case, is just minus pA dVA minus pB dVB, the infinitesimal amount of work done as these things are gradually moved toward each other.
So delta S for demixing, of course it's dq reversible over T, but just like we did right there, of course, in this case that's just negative dw.
So it's integral from V to VA, right, were compressing.
So for substance a, we're going to go back to there.
It was occupying the whole volume, and then it's going to end up in only part of the volume, VA. pA dVA over T. Same thing for B, going to volume B, pB dVB over T. And now we can substitute for pressure, right?
pV is nRT, so it's nA times R log of VA over V plus nB of R log of VB over V. Now, so this is a suitable answer, but I'm going to make if a little easier by putting in terms of mole fractions.
So, mole fractions, of course, XA is nA over n, and XB is nB over n and XA is also equal to VA over V and XB is equal to VB over V for ideal gases, right?
The molar volumes are the same.
We're at the same temperature and pressure.
So that immediately gives us the delta X of demixing is nR times XA log XA plus XB log XB.
And of course delta S of mixing is the opposite of this, minus nR XA log XA plus XB log XB.
Now XA and XB are mole fractions.
They are between zero and one, So their log rhythms are both negative.
There's a negative sign.
So delta S of mixing is positive.
That's reassuring.
That tells us as we expect that the mixing should happen spontaneously and irreversibly when we remove the barrier.
Any questions so far about how we're going about these calculations of delta S?
OK, let's just do a couple more examples.
Here's a really straightforward one.
What if we just heat stuff up or cool it down.
It happens all the time, but it's pretty important, right?
So let's just heat or cool at constant volume.
So, we're going to go from substance A at T1 and some volume going to substance A at T2 at some volume.
We can do this.
Delta S is integral of dq reversible over T. Going from T1 to T2, but we know how to do this, right?
It's the heat that we need.
It's just given by the constant volume heat capacity times the change in temperature.
So this is just T1 to T2, Cv dT over T. And that is just Cv times the log of T2 over T1 if Cv is temperature independent.
That's not always the case.
Usually if it's not too big an excursion of temperature then it's a reasonable approximation.
So that's straightforward.
By the way, notice that delta S is greater than zero if T2 is greater than T1.
1 In other words, if we heated it up, delta S is positive.
Notice also delta S is less than zero if T2 is less than T1.
Delta S is negative if we cooled it.
Now, under certain conditions we've seen that delta S can't be negative.
Why can it be negative here?
[UNINTELLIGIBLE]
Yes, I heard it.
It's because the system isn't isolated, right.
It was for the isolated system that delta S is always greater than or equal to zero.
And you know, for the whole universe entropy never decreases and so on.
In this case, though, you know we're taking a system and we're putting it in contact with a cold bath and cooling it down.
It's certainly not isolated.
Heat is being exchanged.
It's going from the system to the surroundings, to the bath.
So it's fine.
Delta S can be negative.
What would happen if we calculated delta S of a new entire system consisting of the original system plus the heat bath?
What would delta S be then?
We already kind of did that with those two blocks, T1 and T2 in an isolated systems.
What do you think is going to happen?
In other words, if we call our system the stuff that is put in contact with the heat bath, well OK, then we've seen delta S can be plus, can be positive or negative.
Now let's call our system the original system plus the heat bath, and we'll put them all in some isolating box, that's thermally isolating and so on.
And now we'll put our original system, which is now sort of a sub, part of the system, and it's now going to be in contact with the cold bath.
The bath that it's in contact with is colder than it is.
That's why its delta S is less than zero.
But what's delta S to be for the entire new system including the cold bath?
Wouldn't it greater than zero?
Yes, it's going to be positive, because it's an isolated system and something happened spontaneously in it, namely the original system got cooler and presumably the heat bath got at least a little bit warmer.
And it happened spontaneously, which immediately means delta S had to be positive for that entire assembly, of you know the original system plus the bath.
Here's another important sort of process.
Reversible phase change, we'll melt something or freeze it.
How about water -- liquid 100 degrees Celsius one bar goes to water.
Gas 100 degrees Celsius one bar.
Well in that case, you know what the heat is.
It's just the delta H of vaporization, right, at constant pressure.
So there's nothing really much to calculate here.
Delta S of vaporization for H2O at 100 degrees Celsius is just q of vaporization over the boiling temperature, which is delta H of vaporization over the boiling temperature.
So that was easy.
One last thing.
What if we want to calculate the entropy of melting or some phase transition not at equilibrium?
In this case it's at equilibrium because it's water and water vapor at 100 degrees Celsius and one atmosphere or one bar.
So it's at its boiling point.
But what if it isn't at its boiling point?
So what if instead, let's take H2O, liquid, at minus ten degrees Celsius and one bar so it's cold.
It's going to freeze.
So it's going to turn into solid water at minus 10 degrees Celsius and one bar, we're going to have it be isothermal.
Now you know that's going to happen spontaneously, and it's irreversible.
Irreversible, which means we can't just straight away calculate delta S along this path because it's irreversible.
But what we can do, just like we've done before for making cycles, we can make some other set of events that are all reversible.
We can construct a reversible pass that will get there.
How do we do it?
Well you know for the phase change, for the freezing to be reversible, it has to happen at zero degrees Celsius right.
You know that's where you have reversible, freezing and melting.
Reversible equilibrium between liquid and solid water.
So surely that's got to get involved here.
H2O liquid zero degrees Celsius, one bar, in equilibrium with H2O solid at zero degrees Celsius, one bar.
Whatever reversible path we construct that's going to go from liquid to solid water, somewhere along the set of steps, that better be one of them.
So now what we need to do is go from liquid at minus 10 degrees Celsius and one bar to liquid at zero degrees Celsius and one bar.
That's called heating.
We have to heat it.
And here we have to cool it.
And we've already seen all these three processes, right.
So for the phase change, it's reversible now.
This is just delta heat of fusion.
Great.
Because this is now reversible.
This is going to be reversible.
This is going to be reversible.
Great.
So dq reversible, it's going to be the heat capacity at constant pressure, the example we did before it was constant volume for the liquid dT.
Here, dq reversible is going to be Cp solid, dT.
Those won't be the same right?
It's one thing to say the heat capacity of something doesn't change over some small excursion in temperature, but it's another thing when the thing actually changes phase.
The heat capacity of the solid and the liquid won't be the same.
So this we can just finish up in a jiffy, because we're just going to add the three things.
So delta S is delta S of heating, minus, I think there's a plus that should be a minus there, minus delta S of fusion because it's going in the direction of freezing the way we've written it.
Plus delta S of cooling.
So integral from T1 to the melting point of Cp of the liquid dT over T, minus delta H of fusion over the melting temperature.
Plus the integral going from the heat of the temperature of melting to T1 of Cp of the solid dT over T. That's it, all right?
So delta S is minus delta H of fusion over T. That's what it would be for just the reversible phase change happening at zero degrees Celsius.
And then there's this additional part which is, we can write it from T1 to melting point of Cp of the liquid, minus Cp of the solid, dT over T. Now usually we can assume that these will be temperature independent in their own phases.
So we can usually write this as minus delta H of fusion over the melting temperature plus Cp of the liquid minus Cp of the solid log of T of fusion over T1.
Any questions?
What did we do?
We constructed a reversible path going from here to here and calculated delta S that way, and of course that has to be the same as delta S in the one irreversible step.
All right, more on entropy and its consequences for how to figure out what happened spontaneously next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, well last time we finished our sort of introduction to entropy which is a difficult topic, but I hope we got to the point where we have some working understanding of physically what it's representing for us, and also an ability to just calculate it.
So we went through at the end of the last lecture, a few examples where we just calculated changes in entropy for simple processes like heating and cooling something or going through a phase transition where that process, that sort of process is relatively simple because there's no temperature change.
While the ice is melting, for example, you're putting heat into it, but the temperature is staying at zero degree Celsius.
And we saw, how to do these calculations, we need define reversible paths.
So it was extremely straightforward to calculate the entropy of say ice melting at zero degrees Celsius, because there the process is reversible, because that's the melting temperature.
But if we wanted to calculate the change in entropy of ice melting, you know at, once it had already been cooled to ten degrees above the melting point, to ten degrees Celsius, then in order to find a reversible path, we had to say, OK, let's first cool it down to zero degrees Celsius.
Then let's have it melt, and then let's warm the liquid back up to ten degrees Celsius so we could construct the sequence of reversible steps that would get from the same starting point to the same end point, and we could calculate the change in entropy through that sort of sequence.
So now that we've got at least some experience doing calculations of delta S and we're just thinking a little bit about entropy, what I'd like to do is to try to relate the state variables together in a useful way.
And and the immediate problem that I'd like to address is the fact that right now we have kind of a cumbersome expression for energy.
So you know we have u, we look at du, right it's dq plus dw, and you know, I don't like those.
They're path specific, and it would be nice to be able to do a calculation of changes in energy that didn't depend on path.
You know it's a state function.
So in principle it seems like it sure ought to be possible, and yet so far when we've actually gone through calculations of du, we've had to go and consider the path and get the heat and get the work and so forth.
And then we found special cases, you know an ideal gas where the temperature, where the change is only a function of temperature and so forth, where we could write this as a function of state variables, but nothing general that really allows us to do the calculation under all circumstances.
So let's think about how to make this better.
So, and what I mean by that is you've seen examples like, you know, some special examples you saw awhile back.
The case where du was Cv dT minus Cv and this Joule coefficient d v. But you know you still need to find those coefficients for each system.
This isn't a general equation that tells us how energy changes in terms of only functions of state.
Because of things like this and this -- what I'd really like is to be able to you know write du equals something.
And that something, you know, it can have T and p and whatever else I need.
It can have S, H, and of course differentials of any of those quantities.
dT or dp or dS, dH, you name it, but all state variables.
That's what I'd much rather have.
And then, you know, if I want to, if I've got something like that and I want to find out how the energy changes as a function of volume, so I'll calculate du/dV With respect to some selected variable hold constant, I can do it.
Right now, in a general sense, that's cumbersome.
I've got to figure out how to do that for each particular case that I want to treat.
So, let's see how we could construct such a thing.
Let's consider just a reversible process, at constant pressure.
So, OK, I've got one and I'm going to in some path wind up at state two and I'll write du is dq, it's reversible in this case, minus p dV.
And from the second law, we know that we can write dq reversible as T dS.
dq over T is dS or entropy.
So, we can write du is T dS minus p dV.
That's so important, we'll circle it with colored chalk.
That's how important it is.
You know it's a dramatic moment.
So let's look at what we have here.
Here's du and over on this side we have T, we have S, we have p and we have V. Suddenly and simply, it's only functions of state.
Well that was pretty easy.
So, what that's telling us is that we can write u this way, and you know, this is generally true.
We got to this by considering a reversible constant pressure process.
But we know u is a state variable right.
So this result is going to be generally applicable.
And it tells us a couple of things too.
It tells us that in some sense, the natural variables for u are these, right, it's a function of S and V. Those are natural variables in the sense that then it written as functions of those variables, we only have state quantities on the right-hand side.
Very, very valuable expression.
And of course coming out of that then, we can take derivatives and at least for those particular variables, we can see that du/dS at constant V is minus p. And du/dV at constant S is T. All right, those fall right out.
Now we can have a similar set of steps for H, for the enthalpy, so let's just look at that.
So H, of course, it's u plus pV, so dH is just du plus d(pV), and now there's our expression for du.
We're going to use it this way.
So it's T dS minus p dV plus p dV plus V dp.
And of course these are going to cancel.
So we can write dH is T dS plus V dp.
Also important enough that we'll highlight it a little bit.
So again let's look at what we've got on the right-hand side here, T, S, V, p, only quantities that are functions of state.
And of course, we can take the corresponding derivative, so let's also be explicit here, that means that were writing H as a function of the variables S and p. And dH/dS at constant p is T, and dH/dp at constant S is V. So now we've got a couple of really surprisingly simple expressions that we can use to describe u and H in terms of only state variables.
All right?
We also can go from these expressions, using the chain rule, to expressions for particularly useful expressions for the entropy as a function of temperature.
So, you know, from du is T dS minus p dV.
We can rewrite this as dS is one over T du plus p over T dV.
And now we can go back, you know, if we can go back to our writing of u in terms of, as a function of T and V, right.
So we can write here du as a function of T and V, Cv dT plus du/dV at constant T dV.
The reason we're doing that is now we can rewrite dS is one over T times Cv dT, and that's the only temperature dependence we're going to have.
The other part is going to be a function of volume.
So it's, we've got p over T plus du/dV at constant T dV, and what this says then is that dS/dT at constant V is just Cv over T. Very useful, not surprising because of the relation between heat at constant volume and Cv, right.
And of course dS is just dq reversible over T, but this is telling us, in general, how the entropy changes with temperature at constant volume.
We can go through the exact same procedure with the H to look at how entropy varies with temperature at constant pressure, and we'll get exactly an analogous set of steps that will be Cp over T, right.
So also dS/dT at constant pressure is Cp over T. OK?
Now I want to carry our discussion a little bit further and look at entropy a little more carefully and in particular, how it varies with temperature.
And here's what I really want to look at.
You know, we've talked about when we look at changes in u and changes in H, and we've done this under lots of circumstances at this point.
And at various times I've emphasized, and I'm sure Professor Bawendi did too, that when we look at these quantities we can only define changes in them.
There's not an absolute scales for energy or for enthalpy.
We can set the zero in a particular problem, arbitrarily.
And so, for example, when we talked about thermochemistry, we defined heats of formation, and then we said, well, the heat of formation of an element in its natural state at room temperature and pressure we'll call zero.
We called it zero.
If we wanted to put some number on it, and put energy in it, we could have done that.
We defined the zero.
So far, that's, well not just so far, that's always the way it will be for quantities like energy and enthalpy.
Entropy is different.
So let's just see how that works.
So, let's consider the entropy, we'll consider as a function of temperature and pressure.
First let's just see how it varies with pressure.
We're going to see -- what we'll do is consider its variation to both pressure and temperature, and the objective is to say all right, if I've got some substance at any arbitrary temperature and pressure, can I define and calculate an absolute number for the entropy?
Not just a change in entropy, unlike the cases with delta u and delta H, but an absolute number that says in absolute terms the entropy of this substance at room temperature and pressure or whatever temperature and pressure is this amount.
Something that I can do by choice of a zero for energy or for u or H, but here, I want to look for an absolute answer.
All right, so let's start by looking at the pressure dependence.
So we're going to start with du is T dS minus p dV, so dS is du plus p dV over T. Now let's look, T being constant.
OK, and now let's specify a little bit.
I want to make it something as tractable as possible.
Let's go to an ideal gas.
So then at constant temperature, that says du is equal to zero.
So dS at constant temperature is just p over T dV.
And in the case of an ideal gas, that's nR dV over V. And at constant temperature, that means that d(nRT), which is the same as d(pV) is equal to zero, but this is p dV plus V dp.
So this says that dV over V, that I've got there, is the same thing as negative dp over p. Right, so I can write that dS as constant temperature is minus nR dp over p. So that's great.
That says now if I know the entropy at some particular pressure, I can calculate how it changes as a function of pressure, right.
If I know S at some standard pressure that we can define, then S at some arbitrary pressure, is just S of p naught and T minus the integral from p naught to p of nR dp over p. All right, which is to say it's S of p naught T minus nR log of p over p naught.
Right, now normally we'll define p naught as equal to one bar.
And often you'll see this simply written as nR log p. I don't particularly like to do that because of course, then, formally speaking were looking at something that's written that has units inside as the argument of a log.
Of course it's understood when you see that, and you're likely to see it in various places, it's understood when you see that the quantity p is always supposed to be divided by one bar, and the units then are taken care of.
For one mole, we can write the molar quantities S of p and T, is S, S naught of T minus R log p over p naught.
All right, so that's our pressure dependence.
What about that?
We still don't really have a formulation for calculating this, or you know, defining it or whatever we're going to do to allow us to know it.
Well, let's just consider the entropy as a function of temperature, starting all the way down at zero degrees Kelvin, and going up to whatever temperature we want to consider.
Now, we certainly do know how to calculate delta S for all that because we've seen how to calculate delta S if you just heat something up, and we've seen how to calculate delta S when something under goes a phase transition, right.
Presumably, if we're starting at zero Kelvin, we're starting in a solid state.
As we heat it up, depending on the material, it may melt at some temperature.
If we keep keep heating it up, it'll boil at some temperature, but we know how to treat all of that, right.
So let's just consider something that undergoes that set of changes.
So, we've got some substance A, solid, zero degrees Kelvin, one bar.
Here's process one.
It goes to A, it's a solid at the melting temperature and one bar.
Process two is it turns into a liquid at the melting temperature and one bar.
Process three is we heat it up some more, up to the boiling temperature at one bar.
Process four is it evaporates, so now it's a gas at the boiling temperature and one bar.
Finally, we heat it up some more, so now it's a gas at temperature T and one bar.
And if we wanted to, we can go further.
We can make it a gas at temperature and whatever pressure we want.
That part we already know how to take care of, right.
Well, let's look at what happens to S, all right.
S, a molar enthalpy at T and p, where we're going to finally end up, is, well it's s zero at zero Kelvin and one bar or one bar is implied by the superscripts here.
And then we have delta S for step one, and delta S for step two and so on.
So all right, let's, we can label this six so we to all the way to delta S for step six.
Well, so we can do that.
It's S of the material at T and p is S naught at zero Kelvin, plus, here's for process one.
We heat it up from zero Kelvin up to the melting point.
Cp of the solid more heat capacity, divided by T dT.
We can calculate delta S for heating something up, right.
Plus, now we've got the heat of fusion to melt the stuff, so it's just delta H naught of fusion, divided by Tm right.
We saw that last time.
In other words, remember, we're just looking at q reversible over T to get delta S, and it's just given by the heat of fusion.
All right, then let's go from the melting point to the boiling point.
So it's Cp now it's the heat capacity, the molar heat capacity of the liquid, divided by T dT.
We're heating up the liquid.
And then there's vaporization.
Delta H of vaporization over T at the boiling point.
Then we can go from the boiling point to our final temperature T. Now it's the molar heat capacity of the gas over T dT minus R log p over p naught.
OK, so that's everything, and these are all things that we know how to do.
Just about.
OK, this one we're going to have to think about, but all the changes we know how to calculate, right.
So if we plot this, S, and let's just do this as a function of temperature.
I don't have pressure in here explicitly.
Well, it's going to change as I warm up the solid, soon we're really starting at zero Kelvin.
This stuff is all positive, right, so the change in entropy is going to be positive.
Entropy is going to increase as this happens, and then there's a change right at some fixed temperature as the material melts.
So here is step one.
Here is step two, right, this must be the melting temperature.
And then there's another heating step.
Well, strictly speaking, I'm going to run out of space here if I'm not careful, so I'm going to be a little more careful here.
One, two, three.
I'm heating it up a little more.
Entropy is still increasing, right.
So I've done this.
I've done this.
Now I've heated up the liquid.
Now, I'm going to boil the liquid, so it's going to have some change in entropy.
This must be my boiling point, and now there's some further change in the gas, and that gets me to whatever my final temperature is, right, that I'm going to reach.
Four and five, great.
So there is monotonic increase in the entropy.
OK, so we're there, except for this value.
That one stinking little number -- S naught at zero Kelvin.
That's the only thing we don't know so far.
So, for this we need some additional input.
We got some input of the sort that we need in 1905 from Nernst.
Nernst deduced that as you go down from zero Kelvin for any process, the change in entropy gets smaller and smaller.
It approaches zero.
Now, that actually was certainly an important advance, but it was superseded by such an important advance that I'm not even going to reward it by placing it on the blackboard.
Forget highlight, color, forget it.
Because Planck, six years later, in 1911, deduced a stronger statement which is extremely useful, and it's the following.
What he showed is that as temperature approaches zero Kelvin, for a pure substance in it's crystalline state, S is zero.
A much stronger statement, right.
A stronger statement than the idea that changes in S get very small as you approach zero Kelvin.
No, he's saying we can make a statement about that absolute number S goes to zero as temperature goes to zero.
For a pure substance in it's crystalline state.
So that is monumentally important.
So as T goes to zero Kelvin, S goes to zero, for every pure substance in its, and I'll sort of interject here, perfect crystalline state.
That's really an amazing result.
So what it's saying is I'm down at zero Kelvin.
Minimally, I've somehow cooled it as much as I possibly could, and I've got my perfect crystal lattice.
It could be an atomic crystal like this, or, you know, it could be molecules.
But they're all exactly where they belong in their locations in the crystal, and the absolute entropy is something I can define and it's zero.
So S equals zero.
Perfect, pure crystal, all right.
OK, well this came out of a microscopic description of entropy that I briefly alluded to last lecture, and again we'll go into in more detail in a few lectures hence.
But the result that I mentioned, the general result, was that S was R over Na Avogadro's number, times the log of this omega number of microscopic states available to the system that I'm considering.
Now normally for a macroscopic system, I've got just an astronomical number of microscopic states.
You know, that could mean in a liquid, different little configurations of the molecules around each other.
They're all different states.
Huge amounts of possible states, and the gas even more.
But if I go to zero Kelvin, and I've got a perfect crystal, every atom, every molecule is exactly in its place.
How many possible different states is that?
It's one.
There aren't any more possible states.
I've localized every identical lateral molecule in its particular place, and it's done.
And you know, if I start worrying about the various things that would matter under ordinary conditions, right, you know, maybe at higher temperature, I'd have some molecules in excited vibration levels or maybe electronic levels.
Maybe if it's hydrogen, maybe everything isn't in the ground state.
It's not all in the 1s orbital but in higher levels.
Then there's be lots of states available, right, even of only one atom in the whole crystal is excited, well there's one state that would have it be this atom.
A different one would have it be this atom, and so forth.
Already, there'd be an enormous number of states, but at zero Kelvin, there's no thermal lexitation of any of that stuff.
Things are in the lowest states, and they're in their proper positions.
There's only one state for the whole system, so that's why the entropy is zero in a perfect crystal.
At zero degrees Kelvin.
Now, there are things that may appear to violate that.
Now of course you can make measurements of entropy, right, so it can be verified that this is the case.
There are some things that would appear to violate that result.
You know, you can make measurements of entropies, and for example let's take carbon monoxide crystal, CO.
So let's say this is a crystal lattice, it's diatomic molecules.
It's carbon oxygen carbon oxygen carbon oxygen.
All there in perfect place in the crystal lattice.
Well it turns out when you form the crystal every now and then -- you know, let's put it in color.
Let's put it in the color that we'll use to signify something in some way evil.
No insult to people who like that color.
I kind of like it in fact.
OK, so you know, you're making the crystal, cooling it, started out maybe in the gas phase, start cooling it, starts crystallizing.
Gets colder and colder, but you know, carbon monoxide is pretty easy for those things to flip sides.
And even in the crystalline state, even though it's a crystal, so the molecules center of mass are all where they belong, still at ordinary temperatures they will be able to rotate a bit.
So even in the crystalline state, when it's originally formed, not at zero Kelvin, there's thermal energy around.
These things can to get knocked around, and the orientations can change.
Now you start cooling it, and you know, by and large they'll all go into the proper orientation.
Right, that's the lowest energy state, but you know, there are all sorts of kinetic things involved.
There it takes time for the flipping, depending on how long, how slowly it was cooled, and so forth.
May never happen, and then it's cooled, and then anything that's left in the other orientation is frozen in there.
There's no thermal energy anymore.
It can't find a way to reorient.
That's it.
All right, let's say we're down to zero Kelvin, and out of the whole crystal, we've got a mole of molecules.
One of them is in the wrong orientation.
Now how many states do we have like that that would be possible?
We'd have a mole of states, right.
It could be this one.
It could be that one, right.
Or in general, of course really there's a whole distribution of them, and they could be anywhere, and pairs of them could be, and it doesn't take long to get to really large numbers.
So the entropy won't be zero.
Entropy of a perfect crystal would mean it's perfectly ordered.
So that's zero.
But things like that not withstanding, and of course it's the same if you have a mixed crystal.
In a sense I've described a mixed crystal where the mixture is a mixture of carbon monoxide pointing this way, and carbon monoxide pointing this way.
But a real mixed crystal with two different constituents, well of course, then you have all the possible configurations where, you know, they could be here and here and here, and then you could move one of them around and so forth, there are zillions and zillions of states.
But for a pure crystal, in perfect form, you really have only one configuration, and your entropy is therefore zero.
OK, so now, we can go back and we can do this.
And at least in principle, even for things that don't form perfect crystals, we could calculate the change in entropy going from the perfect ordered crystal to something else with some degree of disorder and keep going and change the temperature and do all these things.
So the real point is that this is extremely powerful because given this, we really can calculate absolute numbers for the entropies of substances, at ordinary, not just at zero Kelvin, but using this which, you know, this is a really very straightforward procedure.
And in fact these things are really quite easy to measure.
You know, you do calorimetry, you can measure those delta H of fusions, right, delta H of vaporization.
You can measure the heat capacities, the things in the calorimeter.
You'll see how much heat is needed to raise the temperature a degree.
That give you your heat capacity for the gas or the liquid or the solid.
So in fact, it's extremely practical to make all those measurements, and you can easily find those values of the heat capacities and the delta H's tabulated for a huge number of substances.
So this is, in fact, the practical procedure then of protocol for calculating absolute entropies of all sorts of materials at ordinary temperatures and pressures.
Very important.
Now, one of those corollaries to this law is that in fact it's impossible to reduce the temperature of any substance, any system, all the way to absolute, exact, no approximations, zero Kelvin.
Because you can't quite get down to zero Kelvin.
And there are various ways that you can see that this must be the case.
But here's one way to think about it.
So, let's just write that first.
All right, can't get quite down to zero Kelvin.
All right, let's consider a mole of an ideal gas.
So p is RT over V. And let's start at T1 and V1, and now let's bring it down to some lower temperature, T2 in some spontaneous process.
We'll make it adiabatic so it's like an irreversible expansion.
I just want to calculate what delta S would be there in terms of T and V. So, well, du is T dS minus p dV.
dS is du over T plus p over T dV.
But we can also write du is Cv dT in this case, right?
So that says that p over T, that's R over V, and we can write, dS is Cv dT over T plus R dV over V. It's Cv, I'll write it as d(log T) plus R d(log V), and so delta S, Cv log of T2 minus log of T1, if T2 is my final temperature, plus R log of V2 minus log of V1, okay.
Or Cv log of T2 over T1 plus R log of V2 over V1.
Well, what does it mean when T2 is zero?
Well, I don't know what it means.
This turns into negative infinity.
So we're going to write it again as our either evil or at least unattainable color.
What's going to happen?
I mean you could say, well, we can counteract it by having this go to plus infinity, right.
Make the volume infinite.
In other words, have the expansion be in through an infinite volume.
Of course that's impossible.
That would be the only way to counteract the divergence of this term.
In practice, you really cannot get to absolute zero, but it is possible to get extremely close.
That's doable experimentally.
It's possible to get down to some micro or nano Kelvin temperatures, right.
In fact, our MIT physicist, Wolfgang Ketterle, by bringing atoms and molecules down to extremely low temperatures, was able to see them all reach the very lowest possible quantum state available to them.
All sorts of unusual properties emerge in that kind of state, where the atoms and molecules behave coherently.
You can make matter waves, right, and see interferences among them because of the fact that they're all in this lowest quantum state.
So it is possible to get extremely low temperatures, but never absolute zero.
Here's another way to think about it.
You could consider what happens to the absolute entropy, starting at T equals zero, right.
So we already saw that, you know, that we can go, the first step will go from zero to some temperature of Cp over T for the solid dT if we start at zero Kelvin and warm up.
Now, already we've got a problem.
If this initial temperature really is zero, what happens?
This diverge, right.
Well, in fact, what that suggests is as you approach zero Kelvin, the heat capacity also approaches zero.
So, does this go to infinity as T approaches zero Kelvin?
Well, not if Cp of the solid approaches zero as T approaches zero Kelvin.
And in fact, we can measure Cp, heat capacities at very low temperatures, and what we find is, they do.
They do go to zero as you approach zero Kelvin.
So in fact, Cp of T approaches zero as T approaches zero Kelvin.
Good!
But one thing that this says is, remember, that dT is dq at the heat in divided by Cp.
And this is getting really, really small.
It's going to zero as temperature goes to zero, right.
Which means that this is enormous.
What it says is even the tiniest amount the heat input leads to a significant change increase in the temperature.
So, this is another way of understanding why it just becomes impossible to lower the temperature of a system to absolute zero, because any kind of contact, and I mean any kind, like let's say you've got the system in some cryostat.
Of course the walls of the cryostat aren't at zero Kelvin, but somewhere in here it's at zero Kelvin.
You can say, okay I won't won't make it in contact with the walls.
You can try, but actually light, photons that emit from the walls go into the sample.
You wouldn't think that heats something up very much, and it doesn't heat it up very much, but we're not talking about very much.
It does heat it up by enough that you can't get the absolute zero.
In other words, somehow it will be in contact with stuff around that is not at absolute zero Kelvin.
And even that kind of radiative contact is enough, in fact, to make it not reach absolute zero.
So the point is, one way or another, you can easily see that it becomes impossible to keep pulling heat out of something and keep it down at the temperature that's right there at zero, but again it's possible to get very close.
All right, what we're going to be able to do next time is take what we've seen so far, and develop the conditions for reaching equilibrium.
So in a general sense, we'll be able to tell which way the processes go left unto themselves to move toward equilibrium.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, over the last few lectures we've worked and struggled so formulate the second and third laws of thermodynamics in addition to the first.
Last time we reach the third law which is telling us that we can't quite get to zero degrees Kelvin, but that as the temperature approaches zero degrees Kelvin, the absolute entropy of a pure substance in perfect crystalline form is zero.
And what that corresponds to, if you recall, is the idea that in a perfect crystal at zero degrees Kelvin then you have no disorder at all.
You have a perfectly ordered system.
And in that case, the entropy is absolutely zero.
And the bigger lesson from that is that entropy, unlike energy u or enthalpy H, we could define an absolutely number for it.
The zero of it wasn't arbitrary.
Unlike the case for energy like you've seen in lots and lots of disciplines, where you can arbitrarily set the zero in a way that makes it convenient for you.
Entropy really is not like that.
There is an absolute zero of entropy, and that's really what we learn.
So now, now that we've got all three of the major laws of thermodynamics, what I want to start in on is a discussion of what happens spontaneously.
We've seen that in just one specialized case so far, but we should in a more general way be able to tell when is a system at equilibrium, where there's no net change taking place, and when is it still undergoing spontaneous change towards some other state, presumably toward an equilibrium state.
How do we tell?
So that's the topic that I'd like to address today.
So, that's the big question, right?
So let's say that we've got our stuff in some state.
A, whatever it is.
And there's some other possible state, B, whatever it is.
And maybe if it some well-defined temperature and pressure.
What do we do to tell whether that change will happen spontaneously?
Do we calculate, you know, delta S, delta u, delta H?
What tells us whether or not the change happens?
Certainly in principle we know how to calculate this and other stuff for a change in state of this sort, for lots of changes of state.
But calculating it alone doesn't necessarily tell us whether or not it will just happened of its own accord.
And that's the issue that we'd like to be able to address.
Now, we have addressed this in some cases.
So, for example, we know that if we take, you know, gas A and gas B with a barrier between them, and we remove the barrier, they're going to mix.
And we saw that and went through it somewhat carefully and saw that if the system is isolated, for that case we do have a criterion that tells us whether change happens spontaneously.
Namely, it's delta S is greater than zero.
That tells us whether the change is spontaneous.
And we saw that in fact in this case delta S of mixing, we calculated it, saw that it is positive.
So, clearly, if we remove this barrier mixing takes place, and obviously you know that that happens from lots of experience.
In general, the second law gave us the Clausius inequality for spontaneous change.
Namely dS is greater than dq over T. I suppose we could specify the surroundings temperature.
We saw that in general dS is greater than or equal to dq over T. if it's a reversible process then the equality holds, but if it's irreversible, which means it happens spontaneously, then dS is greater than this.
Just to illustrate the kind of issues that we're up against here, let me just consider a few different chemical reactions, all of which happen spontaneously.
So, I just want to write a few examples down with a few values for delta u or delta H or delta S, and see whether we can get any clues from what we see.
So here are some spontaneous chemical changes.
Here's one.
If we take hydrogen peroxide in the liquid state, it can break down to form water and oxygen.
If we look at the thermodynamic quantities, the enthalpy and the entropy of the reaction, what we find is delta H is minus 209 kiloJoules and delta S is plus 132 joules per Kelvin.
So it seems like there's a favorable change in entropy going this way.
That is, you've got lower energy on the right and also higher entropy.
Higher entropy basically because you're forming molecules of gas where there weren't any before, and there's more disorder in the gas phase than in the liquid.
That is, the gas phase molecules have more freedom to roam.
Okay, that's one example.
Here's another.
Let's just take hydrogen and nitrogen in the gas phase and form ammonia.
Well, here we get, we find that delta H is negative 92 kiloJoules.
Delta S is negative 198 joules per Kelvin.
This one also happens spontaneously.
One thing that makes it pretty clear is that certainly delta S or the sign of it alone is not dictating the outcome here.
All right, here's a third example.
Let's take salt, solid, and dissolve it in a bunch of liquid water.
Hopefully you've got experience saying that this happens, of course it does.
If we measure the thermodynamics, we discover that delta H is 4 kiloJoules, plus 4 kiloJoules.
Delta S is 45 joules per Kelvin.
So now we have a different sign for delta H and it still happens spontaneously.
So clearly, we've got signs and magnitudes of delta H and delta S, and if we wanted to put delta u there, similar things would happen.
They're all over the map.
And yet, these things are all spontaneous processes.
And I didn't specify the conditions, but if we were to do this under ordinary chemical conditions of some, you'd say room temperature and pressure, right, they all happen spontaneously.
OK, clearly we'd be much better off if we had some systematic quantitative way to tell whether something would happen spontaneously.
In other words, we need criteria for equilibrium under more general conditions than the ones that we've dealt with so far, than the one set of conditions that we've dealt with so far, which is isolated system.
Most chemical changes, most physical changes don't happen in isolated systems.
So let's start by writing down our definition of equilibrium.
It's very simple.
The equilibrium state is the one, and it's just one, in which there are no spontaneous changes that can take place to any other state.
Now that's under whatever constraints there are.
There's a box around it.
The temperature or the pressure are fixed, what have you.
But the point is, no spontaneous changes can occur to any other state.
So for example, when we remove the barrier and the gases mix, you know it's over.
Once the gases are mixed, there's not going to be any further net change in the system.
It's at equilibrium, under the new condition, that is with the barrier removed.
OK, so now let's try to formulate how to describe the equilibrium state and what dictates spontaneity.
So what we're going to do is consider the first and second laws.
So our first law, du is dq plus dw.
And our second law, dS is greater than dq over the temperature of surroundings for a change that happens spontaneously.
And now, I want to combine these two, which I of course can do.
I can substitute dq from this expression in here.
I also want to assume for our present purposes that there's only pressure volume work going on, which is to say I want to put p dV in here minus p dV for dw.
So for combining for p V work, what we see then is du has to be less than T surroundings dS minus p external dV.
Or we can rewrite this as du plus external pressure dV minus T surroundings dS is less than zero.
Now this is fundamentally important, and as you know that means that it warrants the exalted distinction of being put up in colored chalk.
And, in fact, since we're going to reuse this again and again during today's lecture, I'm going to put it over here and leave it sacrosanct for our further use.
du plus p external dV minus T surroundings dS is less than zero.
This Is our condition for spontaneous change.
Now this is a really quite useful expression.
For one thing what we have here are all functions of state and parameters that we can control like temperature and pressure.
So that's a big help.
And equilibrium happens when there isn't any possible change of state that would satisfy this.
In other words, you've got your system in some state.
You know it's in some state A, there are some other states around.
Here is what we calculate to tell whether it happens spontaneously.
So we now have a real usable criterion to help guide our understanding of whether things happen by themselves of their own accord or not.
Now, this is still a little bit cumbersome, in part because of the variables involved, including S. That is, most processes that we're concerned with, they'll happen with something held constant like pressure or temperature or maybe volume.
So this isn't the most useful form that we can have, but what we'll see shortly is that from this, we can then derive further criteria for essentially any set of variables or any set of external constraints, like constant temperature or pressure or volume and so forth that we might set.
And so that's what I now want to do.
So I just want to use that again and again, starting from that, for various different sorts of conditions and derive the criterion for equilibrium in each set of conditions.
So first, let's start with the one that we already know, and make sure that it still works, starting from here, mainly our isolated system.
So remember what that means?
It means no heat, no work.
Delta V is zero.
Delta u is zero.
So looking at this, du is zero.
dV is zero.
So all that's left is negative T dS is less than zero.
In other words, T surrounding dS has to be greater than zero, and of course temperature is always positive.
So dS for u and V fixed is greater than zero.
All right, so that's sounds right.
That's what we saw before.
When we have an isolated system, the criterion that determines whether something happens spontaneously is the entropy has to increase.
Now, what this means too is if we imagine a bunch of different states, and this is the entropy of them, so this could be any sort of variable but the point is there are a bunch of possible states around, whichever one has the maximum entropy, that's the equilibrium state.
In other words, you know we've got the two gases on either side of that partition.
We remove the partition, and they mix.
Well, the equilibrium state is the one with the gases completely mixed.
Of course there are lots of states that would have maybe local pockets of one substance in excess and another substance in excess somewhere else.
In other words, there would be lots of states nearby to the equilibrium state.
That is, the chain, they could they could be approached with very little change from the equilibrium state.
They aren't the equilibrium state.
The entropy in all of those states will be lower than the entropy of the fully mixed state.
So the point is, once you're at equilibrium none of the other states, they're accessible, the system could rearrange itself to form them, but there is no accessible state that has higher entropy than the equilibrium state.
OK, that's our familiar isolated system.
Now let's try moving to unfamiliar territory and extending what we know.
So, let's try constant entropy and volume.
And the motivation for choosing a pair like that is easy to see, if we look at our condition for spontaneous change or general condition.
Well, entropy and volume constant means dV and dS are equal to zero.
What does that say?
It means du is less than zero.
That's our condition.
So we immediately get du at constant S and V is less than zero.
That's our condition for spontaneous change.
In other words, if we don't have to worry about entropy or volume equilibrium is achieved when energy is at a minimum.
Now this is what you learn in elementary physics and in mechanics, right.
You're not worried about entropy.
So, you know, if you've got a hill or valley and there's a cart on wheels, it's going to go down to the bottom.
The spontaneous change lowers the potential energy in that case.
So this is a simple condition that's very familiar.
Now, the reason this condition always holds in ordinary mechanics is because you're never, in that case, concerned with a huge statistical population of particles where the disorder among them is an issue.
We're not worrying then about the fact that, well like in the case of gas molecules mixing, the macroscopic state of the whole thing, all those molecules, how many different microscopic configurations are there?
Remember, I mentioned then we'll go further later on into this, that entropy can be related to the extent of disorder.
That is, how many different possible configurations of all those molecules there would be for a particular state.
The reason the entropy of the mixed gases is the highest is because that has the most possible configurations.
If you start segregating the gases, there are fewer possible configurations that the whole system can be in because you're forcing a particular set of circumstances.
When you don't have to worry about criteria like that, ordinary, mechanical energy rules supreme, and that's dictating where equilibrium lies.
But of course, most chemical and biological systems aren't that simple precisely because you have to worry about many particles and their statistics and the way they might order or disorder.
So, and of course, you know, keeping entropy as a fixed variable for a system like that is extremely cumbersome.
As soon as you allow anything to mix, like you might if you want to do any chemistry, entropy changes.
If you change the temperature entropy changes and so on.
So let's go on.
Let's consider a few other examples.
Let's hang on for a little while longer to a set of conditions where we will maintain constant entropy, namely constant entropy and pressure.
So, the dS term is zero, but the other two are not.
So, du plus p dV is less than zero.
I can write this as d(u + pV) less than zero.
Normally I couldn't do that because this term would have p dV plus V dp, but we've specified the pressure is constant, so the dp part is zero.
And this is a quantity that you know, right?
What's u plus pV?
d
dH less than zero, criterion for spontaneous change.
In the case where S and p are held constant.
Now, once again, like I illustrated for entropy, and I could have done the same for energy here, you know, if we again look at a bunch of different states, and look at their enthalpy, well, like before, they'll be invariably lots of possible states.
And now, what is this saying, the equilibrium state is the one with the lowest possible enthalpy.
In the case here, that I just illustrated with the little cart going down the valley, would be exactly the same with regular energy, the equilibrium state is one of lowest energy, right.
And of course there are lots of nearby states.
The cart could be a little ways up the hill, and in this case, it's enthalpy, but again, there would be lots of accessible states.
But if the system is in equilibrium, none of those states has lower enthalpy.
It's already in the lowest enthalpy state.
That is the equilibrium state.
OK, well, now, let's get to the big ones.
That is, in real life, the variables that you'd normally control aren't some combination of entropy and these variables, but really their temperature, volume and pressure, any couple of those, might be what you'd really have under experimental control.
So now let's go to them.
Let's control T and V. So, all right, so now we're getting serious.
All right, well, there's our equilibrium criterion.
We're still going back to it.
It still holds.
So we've now got the dV part equal to zero.
So what this says is that du minus T dS is less than zero, and we can combine those to say d(u - TS) is less than zero.
And again, just like before, we can do that although this normally would say this has T dS and also minus S dT.
T is fixed.
So that part is zero.
So this is really the equivalent of this.
So when we did this here, we conveniently found that that quantity u plus pV is something we know and love, and we're familiar with it.
It's our enthalpy H. So we could write that criterion as dH less than zero.
So here, let's combine these to define a new quantity.
It obviously has importance because what it's going to say is that that's the quantity that defines equilibrium, that tells us about equilibrium, under the very important practical constraints of having fixed temperature and volume.
Realistically, the more likely constraints than either of those.
So let's -- we'll go to a brand new color, define A as u minus TS.
it's called the Helmholtz free energy.
OK, and then our criterion for equilibrium under these conditions is dA, V and T equal to the temperature of the surroundings, is less than zero.
OK, and once again, you know if we wanted to look at a bunch of states that could be accessed, well, we would find lots of states near by, in character to the equilibrium state.
The one that is at equilibrium, there is only one macroscopic state at equilibrium.
It has the lowest A.
In some sense, that's one reason to associate this as a kind of energy, just like mechanical energy u or enthalpy H, it's the minimum free energy state that is the equilibrium state under the relevant conditions.
Now, let's take the step to the biggest set of conditions of all.
What is it when you run a chemical reaction under ordinary circumstances, what's constant?
[UNINTELLIGIBLE]
A little louder
Pressure and temperature
Pressure and temperature, right.
You're running, you're shaking a beaker up here at room temperature.
So let's look at that set of conditions.
All right, there it is.
This is the condition for really the lion's share of chemistry, biology, and other kinds of changes we'll be concerned with.
So, there's our condition for equilibrium.
We don't get to set any of them to zero, right?
So, okay and we can handle that.
du plus p dV minus T dS is less than zero, but we do get to simplify in writing this as d(u + pV - TS) is less than zero, and just like we've seen before, yes, this has p dV and V dp, but the dp is zero because we're at constant pressure.
This has minus T dS minus S dT, but the dT part is zero because we're at constant temperature.
So the result is we can combine all of these as a single differential, and just like we've seen before, what that suggests is that we define another new quantity given by this expression.
And that is the last one we're going to describe.
And that is G, u plus pV minus TS.
The Gibbs free energy.
Notice, we could also write, let's rewrite that.
G is u plus the pV minus TS, but u plus pV is H. So we also can write this as H minus TS and u minus TS is what we just defined a minute ago as A.
So we can also write this as A plus pV.
And the main thing of crucial importance is what, by defining this in the way we have, what that's saying is that dG at constant p and T is less than zero.
There's our condition for equilibrium at constant temperature and pressure.
Boy, is that going to be important for the whole rest of the course.
So, and of course, I hardly need to emphasize further, but we could do the exact same consideration that we have for H and A, there's G. There's our equilibrium state.
It's the state that has the lowest Gibbs free energy.
All these things though are incredibly practical, useful criteria.
This is only defined in terms of state functions.
And just like we saw before for the case of entropy in an isolated system, now we have something we can calculate.
It's a state function, so we're at constant temperature and pressure, and now we want to consider some chemical change or a phase transition or you name it.
Does it happen of its own accord?
Well now we know what needs to be calculated in order to determine that.
So this one is so uniquely pervasive, let's just really explicitly write it all out.
For constant pressure and temperature delta G is less than zero, means A going to B is, all right, let's consider some process state A and state B.
If delta G is less than zero, it happens spontaneously.
If delta G equals zero, then we're already in equilibrium.
And if delta G is greater than zero, then it goes spontaneously in the other direction.
Any questions about any of this?
Let me just give a couple of examples.
If we go back to any of those chemical reactions that I wrote on the board before, right, well certainly we can calculate what delta G would be for each one of them.
Because we know how to calculate all the parts of it.
It's state functions, it's composed of state functions that we know how to calculate.
So we could tell.
When delta G is zero, you know, it doesn't mean that you've got all of one side, all reactants and zero products or all products and zero reactants.
There is some mixture of them.
What this will tell us is what mixture.
You know the stuff is in there in equilibrium, you know the hydrogen and nitrogen that will form ammonia.
And in the end, when it's at equilibrium, and you look and you'd make a measurement, right, you could do spectroscopy.
You could easily see how much of each thing is there.
It doesn't go all the way to absolutely 100 percent ammonia, zero hydrogen zero nitrogen if they were mixed together with the right ratios.
Doesn't happen.
There would be some of the reactants and some of the products.
In the biochemical reactions that are taking place in your body, there is equilibrium between a whole myriad of reactants and products, and thank heavens that gets maintained.
So that's what this will guide us through, and of course that's incredibly, incredibly important.
Here's another thing that's worth thinking about.
There's a balance here between ordinary energy or enthalpy and entropy.
Energy means, you know, chemical reactions happen, and you end up with something that might be exothermic, that is, the products are more stable then the reactants.
You burn methane, and it combines with oxygen to form water, to form CO2.
And if you work out the energetics as we've gone with thermochemistry, you discover there's a huge negative delta H. In other words, the bonds are much stronger.
CO2 is really a stable molecule.
Methane, there are certainly some solid bonds there, but breaking those to form CO2 and water, well it's worth it, right, energetically.
Still, the actual equilibrium depends on entropy also, not only on the energy.
And that's why, when I put up those three different reactions, and we saw the signs could vary.
It's because there's a balance between the two.
Energy may be favoring reaction in one direction, toward let's say products that have lower energy.
But at the same time, entropy is going to be favoring whichever side has higher entropy, has more disorder, and there's a balance that's achieved.
And that's why all those reactions, first of all, in some sense what I put up was kind of a trivial statement in actual fact saying they all happen spontaneously, because I didn't specify what we were starting with exactly, what concentrations we were starting with.
Even something quite unfavorable might happen at least a little bit spontaneously.
You'll have equilibrium.
In those cases, though, you'd have quite a reasonable equilibrium, spontaneously, that is there would be a lot of reaction that went if you simply started under practical conditions and let it go.
Even though the signs of the enthalpy changed, and the signs of the entropy changed because it's a combination of the two that matters.
Here's a really simple example.
Mixing of oil and water.
You know from experience if you've ever mixed them to make salad dressing, they don't mix too well.
And you may know that if you heat them up, they mix much better.
Why?
You know, we've done a bunch of thermochemistry, and we've kind of seen that the energy of mixing, your energetics don't change too much as a function of temperature.
What's changing?
Why does it mix better when you warm it up?
But, you know, looking at our definition of Gibbs free energy, here it is, right, or here.
Let me just say, actually if you calculated delta S for the mixing as a function of temperature, it doesn't change all that much.
You know, the amount of disorder upon mixing is not really sensitive to temperature.
What does change though?
T, and the entropy is weighted by the temperature, so the entropy matters more and more the hotter it gets.
And that's consistent with other things that we've seen, right?
Remember the whole thing about the perfect crystal at zero degrees Kelvin has zero entropy.
It's completely ordered.
Entropy doesn't matter anymore.
It'll go to the lowest energy state.
Raise the temperature, and now entropy plays a bigger role.
So the point is, this balance between energy that you could think of as say bond energies in chemical reactions, and entropy that you can think of in terms of disorder, how many different possible combinations or configurations of something wrong, will dictate where the equilibrium lies.
And knowing now how to calculate these free energies especially the Helmholtz and the Gibbs free energies, that's what's going to guide us in really calculating quantitatively, OK, where will equilibrium lie.
And before long, we'll start in on discussing chemical equilibrium, does deriving where they lie phase equilibrium?
Does stuff change phase to go from liquid to solid and so forth, right?
And where does that happen, at what temperature and pressure and so forth.
And it's always going to come down to calculating the appropriate free energy, and how it changes in the process.
So this is going to be a guide for us for essentially all that we're going to do in the rest of the term.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Alright.
Well, we've been looking in the last couple lectures at a really important topic in thermodynamics.
Which is, how do you tell what's going to happen.
Which way does a process want to go?
Which way will it go spontaneously?
And if it goes in one direction or another, where does is lead?
In other words, what is the equilibrium state?
And this is just an incredibly important area that thermodynamics allows us to speak to.
So we started to see this.
Sort of direction of spontaneous change.
And where the equilibrium lies.
So what we did is, remember we started with the second law.
Right?
That dS is greater than dq over T. And for the spontaneous change which happens irreversibly That means that'll be dq irreversible.
It would be equal for the reversible case.
And we combine this with first law, which for the case of pressure volume changes we write as this.
And so what this gave us was a very, very useful general criterion for determining whether something happened spontaneously.
Namely, du plus p external dV minus T for the surroundings dS, is greater than zero.
Sorry.
It's less than zero.
And this is for any spontaneous change.
If it equals zero, then we're at equilibrium.
And if it's greater than zero, then the process goes the other way.
We would write the process in the reverse to have it be less than 0 and it would go spontaneously.
And based on this one result, we then looked under various constraints and said OK, what about looking at our variables, volume, pressure, temperature, other things, entropy, if we constrain those, what's the condition for equilibrium?
And that's what led us to a number of results to determine what quantities we even need to be looking at.
To figure out equilibrium.
And what the conditions were.
And so what we discovered were the following.
This one, which we already had seen, which is dS, is greater than zero.
Change in entropy is greater than zero, for an isolated system.
We also saw that dS for constant H and p was greater than zero.
du, regular energy, at constant entropy and volume is less then zero.
And u is minimized at equilibrium.
And this is the familiar result from ordinary mechanics, where you're not worrying about something like entropy for a whole collection of particles.
That is, you minimize potential energy and you see things falling under the force of gravity and so forth, going to potential energy minima in conformance with this result.
dH, S and p is less than zero.
So our H is u plus pV, as you know.
And H is minimized at equilibrium.
And this is, of course, with constant S V. This is constant S and p. But of course, the need to have entropy constrained is never going to be the most convenient one experimental.
There may be circumstances under which it's the case, but it's often difficult to control.
On the other hand, temperature, volume and pressure are variables that are much easier in the lab to keep constant.
To keep control over.
And so that led us to the definitions of other energy quantities, the Helmholtz and Gibbs free energy.
We discovered that the quantity dA, under conditions of constant volume and temperature, is less than zero.
And A is u minus TS.
And A is minimized at equilibrium, under conditions of constant T and V. And finally, and in many cases the most important of the results, because of the conditions it applies to, we saw that this Gibbs free energy is less than zero, that's our condition for spontaneous change.
Where the Gibbs free energy, u plus pV minus TS is H minus TS.
Also A plus pV and G is minimized at equilibrium with constant temperature and pressure.
And that's why the Gibbs free energy is just so enormously important.
Because so much of what we do in chemistry does take place with constant temperature and pressure.
So we have this condition that's established in a quantity that we know how to calculate.
That tells us the direction of spontaneous change for ordinary processes, chemical processes, mixing and you name it, under conditions that are easy to achieve in the lab.
OK, now what we'd like to do is be able to calculate any of these quantities in terms of temperature, pressure, volume properties.
That is, in terms of equations of state.
For any material.
Then we would really be able to essentially calculate anything.
Anything thermodynamic.
About a material.
Of course, that's assuming we know the equation of state.
We may or may not.
But because in many cases we can reasonably either model or measure equations of state, collect data for a material for its temperature, pressure, volume relations, then in fact if we can relate all these quantities to those, then in fact we really can calculate essentially all of the thermodynamics.
For the material.
So let's relate the thermodynamic quantities to equation of state p, V, T data.
And we can do that by going through and deriving what we'll call the fundamental equations of thermodynamics.
that'll provide these relations.
And at this point we know enough to do this in a straightforward way.
So if we start with a relation for energy, T dS minus p dV.
Where u is written as a function of entropy and volume.
And we've seen that that's generally the case.
It comes from the fact that dq reversible is T dS, and dw reversible is minus p dV.
And of course du is the some of those.
So, this is generally true.
Since these are all state functions.
That is, this is derived in the case for reversible paths.
But since these are all simply state functions and quantities, this is generally true.
Now we can use it to derive differential relations for all of the thermodynamics quantities.
So let's just go through and do that.
So H is u plus pV.
So dH is just du plus p dV plus V dp.
And now we're just going to substitute du in here.
And the p dV terms are going to cancel.
So we have the result that dH is T dS plus V dp.
Right?
And that shows us that H is written naturally as a function of entropy and pressure.
And now let's keep going.
A is u minus TS.
dA is du minus T dS minus S dT.
We're going to do the same thing.
Substitute this for du.
This time, the T dS terms are going to cancel.
So we have dA is minus S dT minus T dS.
That can't be right.
And it isn't.
Minus S dT, that's the p dV term that's left, minus p dV.
And it shows us that A is written naturally as a function of T and V. G, we can write in any of a number of ways.
Let's write it as H minus TS.
So dG is dH minus T dS minus S dT.
Here's dH.
We'll substitute that in, and the T dS terms are going to cancel.
So dG is minus S dT plus V dp.
And this shows that G is written naturally as a function of T and p. So these, which we will exalt and celebrate by our sparingly-used colored chalk, are our fundamental equations of thermodynamics.
So what they do is, they're describing how these thermodynamic properties change, in terms of only state functions and state variables.
Very, very useful.
And that's what it means, when we say well, it's natural then, to express say, G as a function of T and p, that's what we're saying.
Is that we can express its changes in terms of these variables.
Related only through quantities that are functions of state.
I don't need to know about a specific path here.
If I know about the states involved, I just need to know what the volume was in each of them.
Now, before, of course, in the first part of the class we started out looking at u and then looking at H not as functions of S and V or S and p, but as functions of temperature, mostly.
In general, temperature and volume or pressure.
And it doesn't mean that something was somehow wrong with that.
It certainly is, it still is going to be useful to do thermochemistry.
To ask questions like how much heat is released in a chemical reaction that takes place at constant temperature.
Not one of these variables.
And we can calculate that.
So it's not that we're somehow throwing away our ability to do that.
However, the thing to remember is, when you look at heats of reaction under those conditions it's all well and good.
But it doesn't tell you, this is the direction that the reaction is going to go.
It doesn't tell you, this is the equilibrium concentration that you'll end up with.
That doesn't come out of what we calculated before in thermochemistry.
What does come out, which is very useful is, if you do run the reaction, here's how much heat comes out.
And if you want to run a furnace and provide energy, that's an extremely important thing to be able to calculate.
Because you're going to run it and you'll probably find conditions under which you can run it more or less to completion.
But it doesn't tell you, by itself, which direction things run in.
Whereas under these conditions, these quantities, if you look at free energy change, for example, at constant temperature and pressure, you can still calculate H. You can still calculate the heat that's released.
This is what will tell you under some particular conditions what will actually happen.
Where will you end up.
Very, very important, of course, to be able to understand that.
Now, it's also very useful to look at some of the relations that come out of these fundamental equations.
And they're straightforward to derive.
So, all I want to do now is look at the derivatives of the free energies with respect to temperature and volume and pressure.
So for example, if I look at A, which we now have written as the function of T and V, of course, in general I can always write dA as partial of A, with respect to T at constant volume dT, plus partial of A with respect to V, at constant temperature dV.
And I know what those turn out to be.
It's minus S dT minus p dV.
So what does that tell me?
It tells me that the partial of A with respect to T at constant V is minus S. Right?
In other words, now I know how to tell how the Helmholtz free energy changes as a function of temperature.
Or as a function of volume.
dA/dV, at constant T, must be negative p. Things that I can measure.
So I can in a very straightforward way say, OK, well, here is my Helmholtz free energy.
If I'm working under conditions of constant temperature and volume, that's very useful.
Now, if I want to change those quantities; change the temperature, change the volume, how will it change?
Well, I can, for any given case, measure the pressure, determine the entropy and I'll know what the slope of change will be.
Similarly for G as a function of temperature and pressure, I can go through the same procedure.
That is, it's easy to write down straight away that dG, with respect to temperature at constant pressure is minus S. That is, this is, dG/dT at constant pressure.
And this is dG/dp at constant temperature.
So again with the Gibbs free energy, now I see how to determine, if I change the pressure, if I change the temperature by some modest amount, how much is the Gibbs free energy going to change?
Well, it's easy to see.
These two relations involving entropy are also useful because they'll let us see how entropy depends on volume and pressure.
And let me show you how that goes.
Now, you've already seen how entropy depends on temperature.
We've already seen that, going to write dS as dq reversible over T. And it's Cv dT over T at constant volume.
It's Cp dT over T at constant pressure.
So we know that dS/dT at constant volume is Cv over T, and dS/dT at constant pressure is Cp, over T. And we've seen that on a number of occasions.
So that tells us what to do to know the entropy as the temperature changes.
But now, what happens if, instead we look at what happens when we go to some state one to some other state two and it's the pressure.
Or the volume, that changes.
And by the way, just to be explicit about this, let's take this example, it means that delta S, if we undergo a change from, say, T1 to T2.
There's Cp over T dT.
So it's Cp log of T2 over T1, and we saw this before.
So now, instead, let's look at some process.
State one goes to state two.
Let's have constant T. And look at what happens if pressure goes from p1 to p2.
Or volume goes from V1 to V2.
And see what happens there.
We looked at pressure change before, actually, in discussing the third law, the fact that the entropy goes to zero as the absolute temperature goes to zero for a pure, perfect crystal.
But, actually, we didn't do that in a general way.
We just treated the one case of an ideal gas as the temperature is reduced.
But we can do this, generally, by using what are called Maxwell relations.
And all this is, is saying that when you take a mixed second derivative, it doesn't matter in which order you take the two derivatives.
So, let's, we're going to use this relationship.
And we're going to use these two.
So, using those, now, what happens if we take the second derivative of A, the mixed derivative, partial with respect to T and the partial with respect to V. So let's leave these off for a moment, and now let's try that.
And the point is that the second derivative of A, with respect to V and T in this order is the same as the second derivative of a with respect to T and V in this order.
It doesn't matter which order.
But that turns out to be useful.
So let's do this explicitly.
Which means we're going to take the derivative with respect to volume of dA/dT.
Now, the dA/dT isn't constant volume.
The derivative we're taking with respect to volume, when we take that it's at constant temperature.
But what is it?
Well, we already know what dA/dT at constant V is.
It's negative S. So this is negative dS/dV.
At constant temperature.
Now let's take it in the other order.
So d/dT of dA/dV, just like this.
The dA/dV is calculated at constant temperature.
We know it.
Then we can take the derivative of that quantity, when we vary the temperature, holding the volume constant.
But again, dA/dV dT, there it is.
It's negative p. So this is just negative dp/dT at constant volume.
These things have to be equal to each other.
Because these mixed second derivatives are the same thing.
But that's very useful.
Because this is what comes directly out of an equation of state, right?
You know how pressure changes with temperature at constant volume if you know the equation of state.
It relates the pressure, volume, and temperature together.
So from measured equation of state data, or from a model like the ideal gas or the van der Waal's gas or another equation of state, you know this.
Can determine how entropy is going to behave as the volume changes.
If we try that for an ideal gas, pV is nRT.
So dp/dT at constant volume, it's just nR over V. And that, now, we know must equal dS/dV, with a positive sign.
At constant temperature.
So now let's try looking at something where are V1 goes to V2.
The volume is going to change, and we can see how the entropy changes.
So, if one goes to two and V1 goes to V2, and it's constant temperature, just what we've specified there.
Delta S is S(T, V2) minus S(T, V1), T's staying the same.
So it's just the integral from V1 to V2 of dS/dV At constant temperature dV.
And now we know what that is.
So it's nR integral from V1 to V2 dV over V. So it's nR log V2 over V1.
There's our delta S. So we know how to calculate it.
Make sense?
Now, we can do the same procedure for the pressure change.
And all we do is, I'll just outline this, I think.
I won't write it all on the board.
But, of course, it's going to come from the fact that these second derivatives are also equal.
So d squared G dT dp is equal to d squared G dp dT.
In other words, the order of taking the derivatives with respect to pressure and temperature doesn't matter.
And what this will show is that dS/dp at constant temperature, here we saw how entropy varies with volume, this is going to show us how it varies with pressure.
Is equal to minus dV/dT at constant pressure.
And again, this is something that comes from an equation of state.
We know how the volume and temperature vary with respect to each other at constant pressure.
That's what the equation of state tells us.
So, again, I can just use that result.
So, if we do a process where one goes to two at constant temperature, and now the pressure, p1, goes to p2, well then delta S is just the integral from p1 to p2 of dS/dp times dS, so it's just this.
And so of course it's still pV equals nRT.
So now we just have nR over p dp.
Right?
So we're going to see the same story.
It's nR log of p2 over p1 for the process where there's a pressure change.
Any questions about this part?
So what we've done is take one step further.
We've used the fundamental equations that are hiding down here, out of sight but never out of mind.
And what we've done is look at the derivatives of the new free energies that we've just recently introduced, A and G. And then, the only thing we've done beyond that is say, OK, well now let's just take the mixed second derivatives, they have to be equal to each other.
And what's fallen out when we do that, because in each case, one of the first derivatives gives us the entropy.
Then the second derivative gives the change in entropy with respect to the variable that we're differentiating, with respect to which is either pressure or volume.
And the useful outcome of all that is that we get to see how entropy changes with one of those variables in terms of only V, T, and p, which come out of some equation of state.
And all we did, further, is take that second derivative.
That mixed second derivative.
And, of course, see that either way we do that we'll have an equality.
Now, let's go back to our older friends u and H. Which we've expressed now in terms of S and V, S and p. So, so far we don't have a way to just write off, relate them to equation of state data.
Which also would be very useful.
Here, A and G, we've already got as functions of these easily controlled, conveniently controlled state variables.
Let's look at those quantities.
u and H. And look at, for example, the V dependence of u.
The volume dependence.
And in particular let's look at, for example, du/dV at constant temperature.
Now, we can immediately see what du/dV at constant entropy is.
Experimentally, though, that's not such an easy situation to arrange.
Of course, this is a much more practical one.
But it doesn't just fall out immediately from the one fundamental equation for du.
But we can start there.
So, du is T dS minus p dV.
And I can take this derivative.
du/dV at constant T. And so, what is it?
Well, it's not just p because there's some dS/dV at constant T. This isn't zero.
There's some variation, dS/dV, at constant temperature.
That's going to matter.
This part, of course, is just minus p. But we just figured out what dS/dV at constant T is.
This is dp/dT at constant V. So that's neat.
So in other words, we can write this as T, dp/dT at constant V, minus p. Let's just check T, p, T, V, p. Right?
In other words, we just have pressure, temperature and volume.
Again, if we know the equation of state, we know all this stuff.
So again, we can measure equation of state data.
Or, if we know the equation of state from a model, ideal gas, van der Waal's gas, whatever, now we can determine u.
From equation of state data.
Terrific, right?
So let's take our one model that we keep going back to.
Equation of state, and just see how it works.
That is, ideal gas.
And see how it works with that.
Now, we saw before, or really I should say we accepted before, that for an ideal gas, u was a function of temperature only.
Well, now let's try it.
So, dp/dT, for our ideal gas, at constant volume, remember pV is nRT.
So this nR over V. And then, using the relation again, we can just write this as p over T. In other words, we're taking advantage of the fact that we now know that quantity.
In the case of the ideal gas, we just have a simple model for it.
More generally, we could measure it.
We could just collect a bunch of data.
For a material.
What's the volume it occupies at some pressure and temperature?
Now let's change the pressure and temperature and sweep through a whole range of pressures and temperatures and measure the volume in every one of them.
Well, then, we could just use that for our equation of state.
One way or another, we can determine this quantity.
For the ideal gas it's this.
So now our du/dV, at constant T is just T times dp/dT, which is just p over T minus p, it's zero.
Remember the Joule expansion.
And we saw that, you saw that the Joule coefficient for an ideal gas was zero.
So that you could see that for the ideal gas, u would not be a function of volume, but only of temperature.
But actually, when you saw that before, you weren't given any proof of that.
It was just that when the good Mr. Joule made the measurements, to the precision that he could measure, he discovered that for some gases it was extremely small.
At least, smaller than anything he could detect.
So it sure seemed like it was going to zero, under ideal gas conditions.
And that was the result that we came to accept.
Here, though, you can just derive straight away.
That for an ideal gas it has to be the case that there's no volume dependence of the energy.
Only a temperature dependence.
It's the same for H. Just like u, we'd like to be able to express it in a way that allows us to calculate what happens only from equation of state data.
But, again, our fundamental equations show us how it changes as a function of entropy and pressure.
So, dH is T dS plus V dp.
So let's look at dH/dp.
We know how to get it immediately if we keep entropy constant.
But we'd like to relate it to what happens if we keep the temperature constant.
So then, just like we saw, analogous to what saw just before, it's T dS/dp at constant T. Plus V. But now we've seen from the Maxwell relations that dS/dp is minus dV/dT, for constant p. Again, this is this quantity, one of these quantities that again we can determine from equation of state data.
Only V, p and T appear.
So it's minus T dV/dT at constant p, plus V. And so, again, this can come from equation of state data.
And if you do this again for an ideal gas, let me see.
So we have pV is nRT.
So dV/dT at constant pressure is just nR over p. But we can plug that in again just like we did before.
It's just equal to V over T. And so dH/dp under our condition of constant temperature is just minus T times V over T plus V, everything cancels, and that's zero.
That's our Joule - Thompson expansion.
That was a constant enthalpy change.
And again there, too, you saw an experimental result you were presented with that says, well at least to the extent that it could be measured, it was obviously getting very small.
For gases that approach ideal gas conditions.
Well, there you can see it.
Sure better have gotten small because in fact it has to be zero.
Now let's take just one somewhat more complicated case.
Let's look at a van der Waal's gas.
Let's try it with a different equation of state, that isn't quite as simple as the ideal gas case.
So, then p plus a over molar volume squared times V minus b molar volume V minus b is equal to RT, remember?
This was back from the first or second lecture in the course.
So, we can separate out p. It's RT over molar volume minus b minus a over molar volume V squared.
And then we can take the derivative with respect to temperature, it's just R over molar volume minus b.
So it's dp/dT at constant V is just R over V bar minus b.
Well, let's now look, given this, let's now look in that case, at what happens to u as a function of V. For the ideal gas, we know that u is volume independent.
It only depends on the temperature.
But for the van der Waal's gas, now it's going to be different.
And that's because this is different from what it is in the ideal gas case.
Namely, now du/dV at constant T, for the van der Waal's gas.
So it's this.
So it's RT over molar volume minus b.
Minus p, right?
But in fact, if you go back to the van der Waal's equation of state, here's RT over v minus b.
If we put it as minus b, that's just equal to a over V squared.
Equals a over molar volume squared.
But the point is, the main point is, it's not zero.
It's some number.
a over the molar volume squared.
a is a positive number in the van der Waal's equation of state.
So this is greater than zero.
In other words, u is a function of T and V. If we don't have an ideal gas.
By the way, just to think about it a little bit, it's a positive number.
What that means is, I've got my ideal gas in some container.
There's some energy, some internal energy.
Now I make the volume bigger.
I allow it to expand.
And the energy changes, it goes up.
In some sense it's less favorable energetically.
What's happening there, that a term in the van der Waal's equation of state, is describing interactions, favorable attractions, between gas molecules.
The b is describing repulsions.
Effectively, the volume changes.
The molar volume is being changed a little bit by the fact that if you're really trying to make things collide with each other, they can't occupy the same volume.
And that is being expressed here.
The a, though is expressing attraction between molecules at somewhat longer range.
So now, if you make the volume bigger, those attractions die out.
Because the molecules are farther apart from each other.
So the energy goes up.
Now, you might ask, well why does it do that, right?
I mean, if the energy is lower to occupy a smaller volume, then if I have this room and a bunch of molecules of oxygen, and nitrogen and what have you in the air, and there are weak attractions between them, why don't they all just sort of glum together and find whatever volume they like.
So that the attractive forces can exert themselves a little bit.
Not too close, right?
Not so close that the repulsions dominate.
Why don't they do that?
What else matters besides any of those considerations?
What else matters that I haven't considered in this little discussion?
Yeah.
What about entropy, right?
If I only worry about minimizing the energy, it's true.
They'll stick together a little bit.
But entropy also matters.
And there's disorder achieved by occupying the full available volume.
Many more states possible.
And that will end up winning out at basically any realistic temperature where the stuff really is a gas.
OK. Next time, what you're going to see is the following.
It turns out we can express all these functions in terms of G, we wouldn't need to choose G, but it's a very useful function to choose.
Because of its natural expression in terms of T and p. So we can write any of these functions in terms of G. Which means that we can really calculate all the thermodynamics in terms of only g. It's not necessary to do that, but it can be quite convenient.
And then we can say, OK, if we have many constituents, what if we have a mixture of stuff?
We can take the derivative of G with respect to how much material there is.
With respect to n, the number of moles.
And if there are one, and two, and three constituents with respect to n1, and n2, and n3.
Each individual amount of stuff.
What that's going to allow us to do is, if we say, OK I have a mixture of stuff, how does the free energy change?
If I change the composition of the mixture?
If I take something away, or put something else in?
And we'll be able to determine equilibrium under those conditions.
That's very useful for things like chemical reactions, where this constituent changes to this one.
And I can calculate what happens to G under those conditions.
And that's what you'll see starting next time.
And professor Blendi will be taking over for that set of lectures.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundred of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time, you talked about the Gibbs free energy.
And the fundamental equations.
And how powerful the fundamental equations were in being able to calculate anything from pressure, volume, temperature data.
And you saw that the Gibbs free energy was especially important for everyday sort of processes.
Because of the constant pressure constraint.
And the fact that the intrinsic variables are pressure and temperature.
Well, it turns out that the Gibbs free energy is even more important than that.
And this is something that it took me a while to learn.
I had to take thermodynamics many times to really appreciate how important that was.
Even when I was doing research, at the beginning, I was a theorist, and I was trying to calculate different quantities of liquids and polymers and in these papers the first thing they did was to calculate the Gibbs free energy and I didn't quite appreciate why they were doing that.
And the reason is, is if you've got the Gibbs free energy, you got really everything you need to know.
Because you can get everything from the Gibbs free energy.
And it really becomes the fundamental quantity that you want to have.
So let me give you an example of how important that is, if you have an equation that describes the Gibbs free energy as a function of pressure and temperature, number of moles of different things.
Different things you can calculate.
So let's just start from the fundamental equation for the Gibbs free energy, dG is minus S dT plus V dp.
And let's say that you've gotten some expression, G, as a function of temperature and pressure for your system.
It could be, as we're going to see today that we're going to increase the number of variables here, by making the system more complicated.
So what I'm saying, now what I'm going to say now, is going to be more general than just these two variables here.
So you've gotten this.
So you have this.
You've got the fundamental equation.
You've got all the other fundamental equations, and from there you can calculate all these quantities.
For instance, you can calculate an expression for S. Because you know that S, from the fundamental equation, is just the derivative of G, with respect to T, keeping the pressure fixed.
So you've got your equation for G. That translates into an equation for S. You can get volume, volume is not one of the variables.
Temperature and pressure are the two knobs.
But you can get volume out, because volume is the derivative of G, with respect to pressure.
Keeping the temperature constant.
And you've got S now, you got volume.
Do you know where you, how did you define G in the first place?
We define G as H minus dS.
One of the many definitions of g. Reverse that, you've got now H as a function of G and temperature and entropy.
Well, you've got an expression for G, we just calculated, we just showed we could get an expression for S, which is sitting right here.
Temperature is a variable here, so now we have an expression for H. And you can go on like that with every variable that you've learned in this class already.
For instance, u is H minus pV.
Well, there's the H here.
We have that equation for H. We have an equation for V, coming from here.
So there's nothing unknown here.
If we have an equation for G here in terms of temperature and pressure.
Same thing for the Helmholtz free energy.
You can even get the heat capacities out.
Every single one of these interesting, useful quantities that one would want to calculate falls out from the Gibbs free energy here.
Any questions on that important step?
And, really, I can't believe how clueless I was when I started doing research.
Because I would go through the process of calculating G, and getting G, and et cetera, et cetera.
I wrote papers, you know, G equals blah blah blah.
And I didn't realize that that's why people wanted to know G. Anyway, I know better now.
So, there are a few things we can say about G that are fairly easy to calculate.
For instance, if I look at liquids or solids.
And I want to know how G changes with pressure.
So, I know that the volume here dG/dp, that dG/dp is the volume here.
So if I look under constant temperature, I pick my fundamental equation under constant temperature, and I want to know how G is changing, I integrate.
So I have dG is equal to V dp.
So if I change my, and I look at per mole, and if I change my pressure from p1 to p2, I integrate from p1 to p2, p1 to p2 here, final state minus the initial state is equal to the integral from p1 to p2, V dp per mole.
And what can I say, for a liquid or a solid, the volume per mole, over a liquid or a solid, is small and it doesn't change very much.
So V is small.
And usually these solids and liquids, you can assume to be incompressible.
Meaning, as you change the pressure, the volume doesn't change.
It's a good approximation.
So when you do your integral here, you get that G at the new pressure is G at the old pressure, then if this isn't changing very much with pressure, or not at all, then you can take it out.
It's just a constant.
Plus V times p2 minus p1.
And so, this is the incompressible part, you take it out.
The fact that it's small means that you can assume that this is zero, this whole thing is zero, that it's small enough.
And then you see that G, approximately doesn't change with pressure.
Tells you that G, for a liquid or solid, most of the time you can assume that it's just a function of temperature.
Just like we saw for an ideal gas, that the energy and the enthalpy were just functions of temperature.
And that's a useful approximation.
It's useful, but it's not completely true.
And if it were true, then there would not be any pressure dependents to phase transitions.
And we know that's not the case.
We know that if you press on water when it's close to the water liquid-solid transition, that you can lower the melting point of ice.
You press on ice, and you press hard enough, and ice will melt, the temperature is closer to melting point.
And we'll go through that.
So, that means that there has to be some sort of pressure dependence, eventually.
And we'll see that.
This is just an approximation.
What else can we do?
We can calculate, also, for an ideal gas.
Liquid and solid, we can do an ideal gas.
So for an ideal gas, again, starting from the fundamental equation, we have dG equals V dp.
We can do it per mole.
So integrate both sides, G(T, p2) is equal to G at the initial pressure, plus the integral from p1 to p2, the volume.
So instead of putting the volume, this is an ideal gas now, we can put the ideal gas law.
So V is really RT over p. RT over p dp.
We can integrate this.
Get a log term out.
G(T, p1) plus RT log p2 over p1.
And then it's very useful to reference everything to the center state.
p1 is equal to one bar, let's say.
So if you take p1 equals one bar as our reference point, and get rid of the little subscript two here, we can write G of T at some pressure p, then is G and the little naught on top here means standard state one bar plus RT log p divided by one bar.
And I put in a little dotted line here because very often you just write it without the one bar and bar.
And you know that there has to be a one bar, because inside a log you can't have something with units.
It has to be unitless.
So you know if you have something with bar here, you've got to divide with something with bar, and there happens to be one bar here.
So pressure p is G at its standard state plus RT log p. And this becomes a very useful, very useful, quantity to know.
OK, so G is so important.
And G per mole is so fundamental, that we're going to give it a special name.
Not to make your life more complicated but just because it's just so important.
We're going to call it the chemical potential.
So G per mole, we're going to call mu.
And that's going to be a chemical potential.
We're going to do a lot more with the chemical potential today.
And the reason why we call potential is because we already saw that if you've got something under constant pressure, temperature, that you want to use G as the variable to tell you whether something is going to be spontaneous or not.
So you want G to go downhill.
And so, we're going to be talking about chemical species.
And instead of having a car up and down mountains, trying to go down to the valleys, we're going to have chemical species trying to find the valleys.
The potential valleys.
To get to equilibrium.
And so we're going to be looking at the Gibbs free energy, or the Gibbs free energy per mole at that particular species, and it's going to want to be as small as possible.
We're going to want to minimize the chemical potentials.
And that's why it's called potential.
It's like an energy.
So, that's the end of the one component, thermodynamic background, before we get to multi-components.
So it's a good time to stop again and see if there's any questions.
Any issues.
OK.
So, so far we've done everything with one species.
One ideal gas, one liquid, one solid.
We haven't done anything with mixtures, except for maybe looking at the entropy of mixing.
We saw the entropy of mixing was really important, because it drove processes where energy was constant.
But most of what we care about in chemistry, at least in chemical reactions, species change.
They get destroyed.
New species get created.
There are mixtures.
It's pretty complicated.
For instance, if I take a reaction of hydrogen gas plus chlorine gas to form two moles of HCl gas, I'm destroying hydrogen, I'm destroying chlorine, I'm making HCl in the gas phase.
I get a big mixture at the end.
I get three different kinds of species at the end.
So the fundamental equations that I've been talking about, that we've been talking about, they're too simple for such a system.
Because they all care about one species.
Even more complicated, for instance, if I take hydrogen gas and oxygen gas and I mix them together to make water, liquid, for instance, not only do I have species that are changing, that are getting destroyed or created, in this case here the total number of moles is changing.
And the phase is changing.
Got all sorts of changes going on here.
And so if I want to understand equilibrium, if I want to understand the direction of time for these more complicated processes, I have to be able to take into account, in an easy way, these mixing processes, these phase changes, these changes in the number of moles.
And that's what we're going to talk about today.
We're going to try to change our fundamental equations to make them a little bit more complicated so that we can deal with these sorts of problems.
Because those are the real problems we need to keep track of.
And the ultimate goal, then, of changing our fundamental questions is to derive equilibrium from first principles.
To really understand chemical equilibrium, which you've all seen before.
You've all used the chemical equilibrium constant K, you've done problems.
But you've been given, basically, the equilibrium constant, and not really derived it, understood where it came from.
OK, another simple example here, which is actually the one that we're going to be looking at in the first case.
Where there's a change going on, is just to look at a phase change.
H2O liquid going to H2O solid.
There's a phase change, you can think of it as one species, the H2O liquid sort of changing into an H2O a solid.
It's the same chemical in this case here, there's no change in the molecules.
But it's still a change that we have to account for.
Another example that's also simple like this, that you all, I'm sure, have seen before, suppose I take a cell.
My cell here, full of water.
And then I put my cell, let's say I take a human cell.
My skin or something.
And I take it and I put it in distilled water.
What's going to happen to the cell?
Is it going to be happy?
What's going to happen to it?
It's going to burst, right?
Why is it going to burst?
Anybody have an idea why it's going to burst?
Yes
[INAUDIBLE]
That's right.
So the water wants to go from, you're completely right.
But let me rephrase it in a thermodynamic language here.
The water is going to go from a place of high chemical potential to low chemical potential.
And the cell can't take all that water in there.
The membrane's going to try to swell.
And eventually burst, right?
Same thing if you if you take a, go fishing, go to the Atlantic Ocean and then get a nice cod or something.
Bring it back and on your sailboat, you dump it in a tub of fresh water.
Is that cod going to be happy?
It's not going to be happy at all, right, because its biology is geared towards living in salt water.
And turns out that the chemical potential of water, in salt water, is lower than the chemical potential of pure water.
And so when you put the cod in there, the chemical potential of the water and the cod, is lower than the chemical potential of the fresh water you have on the outside.
And the fresh water wants to be at a lower chemical potential.
It rushes into the cod, and well, the cod does what the cell does, when you put it in distilled water.
It sort of bloats.
It isn't very happy.
OK, so but all these things are basically the same idea here.
Where you have a complicated change, where species are mixing, and things like this.
And it turns out the chemical potential is going to tell us all about how to think about that.
That's why the chemical potential is so important.
So we're going to go back to these two examples here many times.
So let's take the simplest example here.
Let's go back and derive some equations.
Let's take our simplest example that's not a one species system, but has two species.
Species 1 and 2.
And n1 and n2 are the number of moles of species 1 and 2.
And then we're going to see if I make a perturbation in my system.
I change the number of moles of 1, or the number of moles of 2.
How does this affect the Gibbs free energy?
That's the question we're going to post.
And our goal is to find a new fundamental equation for G that includes the number of moles of the different species as they change.
Because in chemistry they're going to be changing.
They're not going to be fixed.
So what we want is just purely mathematically formally, take the differential of the Gibbs free energy, which we know is dG/dT, keeping pressure, the number of moles of 1, the number of 2 constant, dT.
That is, dG/dp constant temperature, n1 and n2 dp.
And then we have our two more variables now, dG/dn1, remember this is just a formal statement keeping temperature and pressure and n2 constant.
dn1 plus dG/dn2, dn2 keeping temperature and pressure and n1 constant here.
I'm not writing anything new here, I'm just telling you what the definition of the differential is here, for G. We already know what some of these quantities are.
We know that this is the entropy, minus the entropy.
This here is the volume.
And I know the answer already.
But I'm going to define it anyways.
And we're going to prove it.
I'm going to define this as the chemical potential for species 1.
I'm going to define this, I'm going to give it a symbol, chemical potential mu, for species 2.
So that I can write my new fundamental equation as dG as minus S dT.
Plus V dp plus, and if I have more than two species present, the sum of all species in my mixture times the chemical potential of that species, dni.
The change in the number of moles of that species.
So, this quantity mu, that I've just defined, dG/dni, keeping the temperature, the pressure and all the n's, except for the i'th one constant, that is an intensive quantity.
Because G scales with size, scales, with size of system.
G is intensive.
n, obviously, scales with the size of the system.
Also intensive, and you take the ratio of two extensive variables, you get an intensive variable which doesn't care about the size of the system.
Which is a good thing.
For what we've been talking about.
Intensive.
If I'm talking about putting a freshwater fish and dumping it in the, putting it in the Atlantic Ocean, the chemical potential of the water in the Atlantic ocean better not care whether the Atlantic Ocean is huge or even huger.
It just cares about the local environment.
Just cares that it that wants to be in that freshwater fish.
So the chemical potential is intensive.
Just as we've written it down here for a single species system.
It's the Gibbs energy, free energy per mole.
Which we haven't proven yet.
We haven't proven yet here.
We've just defined it this way and we're going to prove that in fact the mu's are the Gibbs free energies per mole for each of the species in our system.
OK, so this is now our first new fundamental equation.
All we did was to add the sum here.
Now, we started the lecture by saying that if you have the Gibbs free energy, you've got everything.
And we wrote equations that are covered, were we can get S, we can get V, we can get H, we can get u, we can get A.
So now that we have a fundamental question for G, we've got our new fundamental equations for everything else.
Without really thinking too much.
We go back to our definitions.
Enthalpy is G minus TS.
So dH is dG minus d of TS.
dG, we've got our new fundamental equation for G. We plug it in here.
Expand things out with a T dS here, we can write immediately a fundamental equation for H. dH is T dS.
The beginning is going to look just like what you've seen before.
Plus V dp.
Plus an extra term, which is exactly the same extra term that we had in the fundamental equation for G. Exactly the same.
And every one of the other fundamental questions can be derived in the similar way from G. And they're going to be what you had before minus S dT plus minus p dV plus the sum of the mu i's, dni, and du is T dS minus p dV plus i mu i dni.
So immediately we can see that this mu, this quantity mu that we've defined as the derivative of G with respect to the n's, we can write many other equations for mu.
There are many other ways to derive it.
Because this is the differential for H. This is the first derivative of H with respect to entropy.
This is the derivative of H with respect to pressure.
And this is the derivative of H with respect to n. Just formally, that's what it is.
When you write a differential.
So formally, this is also mu i, is dH/dni, keeping, now, we have to be very careful, keeping the entropy and the pressure constant.
Because those are the variables.
Keeping the entropy, and the pressure, and all the other n's, constant.
We can also write it as dA/dni, keeping the temperature and the volume constant.
And all the other n's, or we can write it as du/dni, keeping that this entropy and the volume, and all the other n's, constant.
So we have many ways to write the chemical potential.
They give you all the same result.
So this is the formal.
Sort of the formal part of the chemical potential.
Now, what we really want to show is that the chemical potential really is connected to the Gibbs free energy per mole.
That's going to be the useful part.
Let me get rid of this here.
So I said earlier, at the beginning of the lecture, that the Gibbs free energy per mole was so important, we were going to call it the chemical potential.
And I said that here.
And then I said, well, we're going to define this here, the chemical.
But I haven't equated the two yet.
I haven't proven to you that in fact this quantity here, which we've formally defined as the derivative of G with respect to n, is the Gibbs free energy per mole, for this species.
In fact, what we want to show is that if I take the sum of all the chemical potentials, times the number of moles per species, that that is the total Gibbs free energy.
In other words, that the chemical potential for one species in the mixture is the Gibbs free energy per mole for that species.
Once we have that idea, then we'll be able to talk about the concept of chemical potential as this thing that we can use to look at equilibrium.
To look at going downhill for the species.
To see why the cell bursts and all these things.
Because now we understand that Gibbs free energy is so important for equilibrium.
We don't understand that quite yet, with the chemical potential.
So we got to make that relation here.
We need to go from the formal definition to a relation that we can understand better, because it includes the Gibbs free energy.
OK, so that's our goal now.
So let's see.
Let's formally do this now.
Let's define, let's derive this.
OK.
So remember, our goal in this derivation is to show that this is true.
Or that this is true, here.
And again, we're going to start with the simplest system possible.
We're going to start with a two component system.
And we can easily generalize to multi-component.
And in our derivation, what we're going to be after is, we're going to start with the Gibbs free energy, because that's where we always start with.
And we're going to remember that by definition, mu i is dG/dni, So if somehow in our derivation dG/dni falls out, that would be great.
Because we'll be able to replace this derivative with the chemical potential.
So the goal was to find something where this falls out, so we can replace it with the chemical potential.
We're going to start with the fact that G is an extensive variable.
So if I take G at a temperature and pressure times some scaling factor for the size of my system, lambda, number of moles of n1, lambda times the number of moles of n2, if I double the size of the system, lambda is equal to 2.
If I half it, lambda is equal to 1/2.
Because it's extensive, this is the same thing as lambda times G of temperature and pressure, n1, n2.
Just rewriting the fact that Gibbs free energy is an extensive property.
And lambda is an arbitrary number here.
Arbitrary variable.
Now I'm going to take the derivative of both sides with respect to lambda.
So I'm going to take d d lambda of this side here.
And d d lambda of that side here.
Now, lambda here is inside the variable here.
So I'm going to have to use the chain rule.
To do this properly.
So this is going to be dG/d lambda n1, there's lambda sitting in the variable lambda n1 here, times d lambda n1 d lambda.
Plus dG/d lambda n2 times d lambda n2 d lambda.
Using the chain rule.
And on this side here, lambda's sitting straight out here.
So this is very easy.
This is G(T, p, n1, n2).
Now, this is good.
Because this is what I'm looking for.
I'm looking for the derivative of g with respect to the number of moles.
Because that's the chemical potential.
That was my goal up here, to make sure in the derivation, somehow, this came out.
And so it's coming out right there.
Right here and right here.
Since the number of moles is lambda n1, that first derivative here is just the chemical potential of species 1 there.
So mu 1, then we have d lambda n1 d lambda.
Well, d lambda n1 d lambda, that's just n1.
It's lambda times the number n1 that doesn't have anything to do with lambda.
So this is n1.
This is the chemical potential of species 2.
Again, the derivative of lambda n2 with respect to lambda is just n2.
And there is G here.
It's a fairly simple derivation, but it gets us exactly what we want.
An association between this formal definition of mu, up here, directly from taking the differential.
How much more formal can you be, mathematically, here?
To associating this formal definition to the Gibbs free energy.
Number of moles times mu 1, number of moles times mu 2, this is the Gibbs free energy per mole of species 1.
Gibbs free energy per mole of species 2.
The sum of all the species of the Gibbs free energy per mole of species i times the number of moles of species i is G. Or, mu i. Voila, we've done it.
This is what we wanted.
The chemical potential is the Gibbs free energy per mole.
And in the mixture, it's the Gibbs free energy per mole of the individual species in that mixture.
And if you want to know what the total Gibbs free energy is, because if you have an equilibrium, what you care about is the total Gibbs free energy.
It's not the Gibbs free energy for one particular species.
What's going to tell you whether you have a minimum or not in your system, whether you're at equilibrium, where you're at the lowest state possible, is the total Gibbs free energy.
Now we'll be able to manipulate chemical potentials, of the individual species, to get at this number here.
Any questions?
This is really, we're going to see this over and over again now.
This chemical potential.
This idea.
And it's not an easy concept.
OK, let me give you an example, then, of the water melting.
And how the chemical potential comes in, now, instead of using the chemical potential.
Instead of the Gibbs free energy.
This is the phase transition.
But it's not very different than the cell bursting when you put it in distilled water.
So, we take a glass of water with an ice cube in it.
H2O liquid.
This is H2O solid.
And I'm looking at the melting process.
I'm looking at a process where I take a small number of molecules of water from the solid phase.
And I bring them to the liquid phase.
And I want to know, is this process spontaneous or in equilibrium, or not possible?
Is this process going to go on?
Is the direction of time that this is melting.
And I want to do this formally thermodynamically.
In terms of the chemical potentials.
That's going to be what we're going to be talking about.
So, formally then, what's going on is, I'm taking nl moles of liquid water, H2O liquid, which is in here.
Plus ns moles of solid water.
And I'm, this is my initial state.
My final state is nl, plus a small number of moles, dn, of H2O liquids, H2O liquid, plus ns minus dn, there's a conservation of the number of molecules here.
Whatever I add to the liquid has to come from the solid here.
Of H2O solid, of ice.
And to know if this is spontaneous or not, if this is done under constant temperature and pressure, what variable should we look at?
G, right.
We want to look at the Gibbs free energy.
So what is G doing during this process here?
What is delta G here?
Well, we have a way of doing it now, in terms of the chemical potentials.
Because we've just shown that this is the case here.
So G is the sum of the chemical potentials times the number of moles in the species.
Therefore, delta G is going to be equal to mu for the number of moles of l. Liquid, dn, number of moles of liquid, plus mu s dns.
So the change in G is going to be equal to the chemical potential times the change in the species, which is in a liquid form, plus the chemical potential of the solid.
Times the change in the species in the solid form.
Now, dns is equal to minus dnl.
This is what we did right here.
You take a certain number of moles from the solid form.
You put it to the liquid form.
That the dn up here.
And you've got to have the negative of it, up here.
So dns is minus dnl.
Which is minus dn.
Delta G is dn times mu l minus mu solid.
So now we can rephrase this, it's all rephrasing.
It's all basically the same thing.
But, we can rephrase this process by asking the question, is the chemical potential of the liquid greater than, equal to, or less than the chemical potential of the solid?
Of the water in the solid.
So the chemical potential of the water in the liquid phase is greater than the chemical potential of the water in the solid phase, mu l is greater than mu s, then delta G becomes positive.
In that case, delta G is greater than zero.
And that's not going to happen.
On the other hand, if the chemical potential of the water molecules in the liquid phase is smaller than the chemical potential of the water in the solid phase, mu s is bigger than mu l, this becomes a negative number.
Delta G is less than zero.
And this will happen spontaneously.
So that illustrates this idea, that the chemical potential of a species will want to go, so the species will want to go, where it can minimize its chemical potential.
So in this case here, when we have the spontaneous process of the water, of the ice cube, melting, you can think of it as these water molecules that are in the ice phase looking around.
They know what their chemical potential here is in the ice phase.
They're looking around, they're looking to see the water phase.
And they see that in the water phase, those water molecules have a smaller chemical potential.
They're happier.
And so these solid water molecules are jealous.
And they want to go in the water phase.
And the ice cube's going to melt.
And it all has to do with this difference in chemical potentials for the water.
And the same thing happens for the water molecules that are inside or outside of that cell that you put in the distilled water.
The water molecules in the distilled water have a chemical potential which is higher than the water molecules inside the cell.
And they don't want to be like that.
They want to change, the system wants to change, until the water molecules couldn't care less whether they're in the water phase, or outside or inside the cell.
The system is going to change until the water molecules have the same chemical potential everywhere.
Where they don't have to choose one place or the other.
And so that gives us, immediately, what we're going to need when we talk about equilibrium.
Equilibrium, chemical equilibrium, is going to be where the chemical potential of a species is the same everywhere in the system.
So at 0 degrees Celsius, one bar, which is the melting point of water, the chemical potential of a molecule of water in the ice phase and in the liquid phase is the same.
That's the definition of the melting point.
It doesn't care.
It could go either way.
It's an equilibrium.
You take an ice cube, water, liquid water, 0 degrees Celsius, one bar.
You come back three days later.
It's the same.
Come back a week later, it's the same.
It's an equilibrium.
Chemical potential of the water species is the same everywhere.
It's an equilibrium.
And I'm just repeating that because this is so important.
OK, any questions?
The last thing we're going to do is to illustrate also the importance of mixing to the chemical potential.
So I'm going to set up sort of an arbitrary system here.
This is kind of like the cell, or the fish, also, idea.
I'm going to put a system where on one side I have a pure gas, A.
On the other side, I have a mixture of A and B.
And here is going to be a membrane that only allows A to go through.
Only A can go through that membrane.
They're going to be partial pressures in here, p prime B, and p prime A.
For the gas pressures on that side, and pressure on that side is p sub A.
And my goal in this example here is to show that if I compare the chemical potential of a species in a mixture, where the temperature and the pressure total are T and p, and I compare that to the chemical potential of the same species when it's pure, when it's not mixed with anything else, under the same temperature and pressure conditions, that, in fact that that equals sign is not there.
That's not what I'm trying to show.
I'm trying to show that there's a less-than sign right here.
To show that if you take, again, this is the cell idea.
If you take the water and the distilled water, under constant temperature and pressure conditions, it's pure.
And it's looking at the cell.
And inside the cell is there, boy, is there a mixture of things.
There's salts, there are proteins, there are all sorts of things, right?
It's a mixed system.
The water in that mixed system, under the same temperature and pressure conditions, the chemical potential of that water molecule is less.
And that's just an entropy thing.
Entropy wants to increase.
It just wants to be in a place with high energy.
It's the Gibbs free energy.
Gibbs free energy has enthalpy and entropy incorporated into it.
The enthalpy's not doing anything.
It's all driven by entropy.
So this is what we're going to try to show.
And I'm not going to get to it today.
We'll start with it on Wednesday.
And I'll let your ruminate on this for the next few days.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's start with some of the things we learned last time.
So there are two things that were important.
We learned that the chemical potential for a species is the Gibbs free energy for that species divided by the number of moles, this is Gibbs free energy per mole.
We learned that the pressure dependence of the Gibbs free energy gives you the pressure dependence for the chemical potential.
That it's equal to the chemical potential at one bar for an ideal gas plus RT log p. We also learned that a species will want to go to minimize its chemical potential, and we saw that for the cell bursting in salt, in distilled water.
Or an ice cube melting at a temperature greater than 0 degrees.
And what we want to prove right now is that if I take a species A, in a mixture, some temperature T, some pressure p, and I compare its chemical potential to the same species, A, when it's pure, the same temperature, the same pressure, what I want to argue is that the chemical potential in the mixture is always less than the chemical potential when it's pure.
The same conditions of pressure and temperature.
And, so we can do a little thought experiment.
Let's do a little thought experiment.
Let's make a box, and in our box we're going to have a partition.
And a flexible membrane here.
And on one side of the partition we're going to have a gas, B.
A gas, A, on this side here.
And gas A on this side here.
So let me just red chalk for A, And I don't think I have any other colors listed.
Yellow chalk for B here.
And everything's one bar.
Everything's one bar.
So one bar B here.
One bar A here, one bar A here .
And this membrane here only lets A through.
And this membrane is deformable, but it's elastic.
You can't deform it forever.
It has some strength to it, right?
So if you push on, it'll push back.
There'll be pressure associated with that.
So the next thing I do, then, in my experiment, shouldn't have done it here.
Next thing I do in my experiment is to break this partition here.
I'm going to break the partition.
And this will cause A and B to mix.
So now in my box I have my partition, my membrane here.
I've got A at one bar here, total pressure of one bar.
And on the other side I have A plus B, with a total pressure of one bar.
That's my initial point now.
What's going to happen?
What's going to happen is that molecules of A here are going to want to go through the membrane to go in this area here.
And there's two ways to look at it.
You can look at it the thermodynamic way, which is the way that we're going to want to be looking at it.
Which is that, from what we're going to prove, is that the chemical potential in the mixture is always less than for the pure substance.
Here we have a mixture.
One bar.
Here we have the pure substance at one bar.
So these molecules are going to look around and say hey, you know, I'm much happier here.
And they're all going to want to go in this area here.
As a result, the volume of this area, if you want to keep the same pressure on both sides, you're going to deform the membrane.
The membrane's going to get deformed.
It's going to bloat on that side here.
It's going to cause an increase in pressure on that side.
And the increase in pressure from the membrane sort of deforming and pushing back, is going to increase until the partial pressure of A, here, equals the pressure of A here.
At which point the flow of A from either side is going to be the same and you're going to be in equilibrium.
And at that point the chemical potentials of both sides are going to be the same.
So I sort of gave you the other way of looking at it.
Which is just purely in terms of pressures.
At equilibrium, the partial pressure of A here has to be the same as the pressure on this side here.
So that the flow of A on either side of that membrane, going from right to left or left to right is the same.
And that's going to cause this mixing to happen.
If you look at it from partial pressure perspective.
But really, it's a chemical potential idea.
Now, the chemical potential, as we saw, was the Gibbs free energy.
And the Gibbs free energy, you can write it as H minus TS.
So basically, in this process that I described, the enthalpy's not doing anything.
These are ideal gases.
They're not interacting with each other.
The only thing that's changing, that's driving the chemical potential, which is basically Gibbs free energy per mole, the only thing that's driving the chemical potential to be lower on this side here, is that entropy term.
It's the entropy of mixing.
So entropy of mixing is really super important.
When we're talking about systems where you have multiple components.
That's going to drive a lot of things.
And in fact that's going to drive equilibrium, as we're going to see a little bit later today.
Alright, let's quickly go through the math and prove this here.
So our goal, then, is to have a mixture, chemical potential of the mixture on one side, and the chemical potential of the pure material on the other side.
And so we're going to start by sort of a similar thing here.
We're going to have a box, let me redo my box here, we're going to have a box with, let me get rid of these one bars here.
We're going to put our membrane in the box.
And we're going to have a pressure, pA, on this side here.
And we're going to have a pressure pA prime for the partial pressure of A.
And pB prime for the partial pressure of B.
And the p total is going to be pA prime plus pB prime.
And I'm going to see at equilibrium, I'm going to write everything I know about equilibrium.
At equilibrium, I know that the partial pressure of A on that side here has to be equal to the pressure of A here.
The partial pressure, the pressure is basically the force of these molecules hitting that membrane per unit time, times the number of molecules hitting.
So we have the same flux of molecules going this way, is equal to the flux of molecules going the other way.
So at equilibrium, pA prime equals pA.
So.
What else can I say?
At equilibrium, in terms of the chemical potentials, I know that the chemical potential of the mixture, mu A in the mixture, temperature under pressure p total.
Is equal to the chemical potential of the pure system, same temperature under pressure is p sub a on that side.
These are the two things that I know at equilibrium.
So let's start to turn the crank.
And see if we can come up with, well, we basically already have this.
If we can massage the right side of our equation so that the pressure term, pA, is p total.
And then we'll have an equation that will compare the chemical potential of the mixture under the same conditions as the chemical potential of A in the pure state.
So, what can we use?
We can use Dalton's law here, which tells us that pA prime is equal to xA p total.
That's from Dalton.
And now, pA prime is equal to pA.
So this is also just pA. And we can plug that in here, and suddenly we've got p total included in here.
Let's just pass these out.
If you want to, thank you.
Thank you.
So, this is equal to mu A pure temperature xA p total.
So what else do I know?
I know that I've written something here, that mu is equal to mu naught, to temp, so this is at one bar.
RT log p. So I'm going to rewrite this as mu A pure temperature T at one bar.
Plus RT log xA p total.
Now, the log of xA times p total is the log of xA plus the log of p total.
It's equal to plus RT log p total plus RT log xA.
So I can lump these two things together.
I have mu A pure T plus RT log p. Well, that's just the chemical potential of A, in the pure state, at temperature T and pressure pT.
That's the equation here to relate pressure, at some variable pressure p, to what it would be at one bar.
Which is that.
Then we have the plus RT log xA sitting here.
Plus RT log xA.
So we're done.
We're done because on that side here I have mu A, the chemical potential of A in the mixture, temperature T, pressure p total.
I've got mu A pure temperature T pressure p total, plus a term.
Plus RT log xA.
Can xA be bigger than one?
xA's the mole fraction.
xA's always less than one.
Log xA is always less than zero.
This term here is always less than or equal to zero.
Therefore, this term is always less than that term.
Therefore the chemical potential in the mixture is always less than the chemical potential inside the pure material.
This is going to be important for the next part of the presentation.
Any questions?
This is what drives the death of saltwater fish in fresh water, right?
Because osmotic pressure is basically given by this basic idea here.
And it's all driven by entropy of mixing.
Sure you don't have any questions?
Speak up.
So now we have all the tools we need to look at chemical equilibrium.
When we have a mixture, and we're going to start with ideal gases.
But everything I'm going to say about equilibrium and ideal gases is valid for solutions.
An ideal gas, and we're going to be talking about ideal solutions.
And molecules in an ideal solution, an ideal solvent, are not very different than molecules in an ideal gas.
They don't interact with each other.
You use concentration instead of partial pressures, but pretty much all the ideas are the same.
All the concepts are the same.
The equations are basically the same.
You just do a little bit of replacement of variables.
But it pretty much is the same thing.
It's easier to think about it, to learn first in terms of the ideal gas, but it applies equally well to what you're more likely to use.
Which is solutions.
So let's look at the prototypical gas phase reaction that everybody writes down when they first do this sort of problems.
The Haber process.
Gas, temperature T, you take nitrogen.
You react it with hydrogen, gas, temperature T, p. And you make ammonia.
NH3 gas T, p. And we're going to ask the question, so I take nitrogen, I take some, hydrogen, I mix them together in a container.
And I make ammonia.
And I'm going to ask, after I reach equilibrium, what is the partial pressure of nitrogen hydrogen and ammonia?
Standard equilibrium problem.
You all, I'm sure you've all seen equations about equilibrium.
Equilibrium constant K, log delta G of the reaction, et cetera, et cetera.
But you probably don't have a really good intuition as to why it is what it is.
So the point of the class here is not to relearn log K is equal to minus delta G, reaction divided by RT.
The point is to learn how we get there.
How we get to that equation.
And what parts are important.
Specifically, how entropy of mixing really becomes key to equilibrium.
But before we get there, let me give you a little bit of history as to why this is such an important reaction.
This reaction, which you see in every chemistry class, was developed by Mr. Haber and Mr. Bosch.
Mr. Bosch made it large-scale around 1910.
And it's the reaction that you use to make fertilizer.
Ammonia is the feed stock to make nitrogen-based fertilizers.
So today, there are a hundred million tons of fertilizer, of nitrogen fertilizer made, using, essentially, the Haber process.
It's a huge, huge, commodity.
1% of the world's energy is taken up to make this reaction.
Almost 1% is about 3/4 of the world's energy is used on this.
In World War 1, Germany was making explosives out of nitrogen feed stock.
And it was getting its feed stock from Chile.
From saltpeter mines in Chile.
Chile was under British hands at the time.
Well, the British didn't let Chile sell saltpeter to Germany.
Germany had to find another way to make ammunitions.
And the Haber process, which had just been invented by Bosch-Haber, became the way that Germany made explosives.
Without this, Germany would have stopped the war in 1916.
Or even before then.
Way before then.
And this process was basically, and Haber and Bosch got the Nobel Prize, essentially, for this process.
For showing how to take chemistry, and doing chemistry and large scale processes under high pressure and high temperature conditions.
Haber got the Nobel Prize in 1918 and Bosch got it in 1931.
This process, arguably, you could say, was the birth of the dominance of Germany as a chemical industry.
Based on that, and the fact that they had to make liquid fuels out of coal.
The syn-gas, or the syn-fuel process.
Also invented during World War 1, where they had to use, they couldn't find any oil and they had to use their coal mines.
And obviously this process, the syn-fuel process, is coming back in vogue with the energy crisis around now.
So they developed a lot of knowledge about how to do large-scale chemical reactions.
And that was the birth of, or the explosion of, companies like Merck and Bayer and Bosch, and BASF, and all these German companies that dominate, basically, the chemical industry.
And when Germany lost the war, the US government confiscated the American divisions of these German companies.
And I'm sure you've heard of Merck as the company.
And you think of Merck as an American company that makes drugs.
Well, that's true.
It's a very large American company that makes drugs.
But there's another Merck, which is the German Merck.
Which also makes drugs.
But it makes liquid crystals.
And liquid crystals is where it makes all of its money right now.
And it's very confusing because they're both Merck.
And they both came from the Merck family from the 1600s that were pharmacists.
But after World War 1, Merck Germany was allowed to use the name Merck everywhere in the world except for the US.
In the US it's called EMD.
Merck USA is allowed to use the name Merck in the US and outside of the US it's called, let me remind myself, it's called MSD.
It's very confusing.
Bayer is the same way.
So Bayer was split up.
And there was Buyer in Germany and there was Bayer in the US.
You've got Bayer Aspirin from the US term Bayer.
And you've got all the other pharmaceuticals from Bayer Germany.
All the chemicals.
And a few years ago, Buyer bought Bayer, and now we have one Buyer-Bayer, depending on where you live.
And so it's very interesting to see the history of all these big chemical companies.
And that's why Switzerland and Germany are still the home of, it's all due to that reaction.
That's why you always see that reaction.
Fertilizers, at the birth of the chemical company, and it's a great example for equilibrium.
So that's the example.
Let's generalize it.
We're actually not going to work on this until the very end.
And we'll see it again.
But let's generalize.
Let's just take a mixture of gases with stoichiometries nu A, a gas, pressure, temperature plus nu B, B, gas, pressure, temperature, goes to nu sub C moles of C, it's a gas, with a temperature and pressure, plus nu sub D moles of D, which is a gas, temperature, pressure.
All ideal gases.
For now.
So this the setup here.
Now, when we write that reaction up here, what we're really writing is, in terms of the process, is take a container that contains A, plus a container that contains B, and go to a container that contains C, plus a container that contains D. And when we write delta G for that reaction here, the process that we're talking about is taking the reactions separately from each other.
That's the initial state.
And the final state is the product.
Separated in their own containers from each other.
And when we say delta G is less than zero, for this process, which means it's spontaneous, we mean the process to go from the separated reactions to the separated products.
But in reality, that's not what you do when you do an experiment.
In reality, you take these two containers and you mix them together.
A plus B.
And you let that react.
And at the end of the day, you have a big container with A plus B plus C plus D inside, all mixed up together in equilibrium.
And this is not the same as that.
It's not the same as that.
Because you've got entropy of mixing happening in all of this.
Forget about A and B interacting with each other.
Entropy of mixing is going to dominate equilibrium.
You've got this problem to deal with.
So that means that we're going to have to worry about, if we're going to want to know at which state the process is in equilibrium, you're going to have to worry about this issue right here.
It's not enough to know what delta G of the reaction is.
So for instance, if I plot, as function of the reaction, I've got the reactants on that side here.
And the products on this side here.
And I want to plot delta G as a function of the reaction.
Well, my initial delta G that I would write, to calculate delta G of the reaction, which is the delta G of the products, minus the delta G of the reactants, so initially I have delta G of the reactants.
That's, in those two boxes, that aren't mixed.
So I'm up here somewhere.
Delta G of the reactants.
And at the end of the process, in these two boxes, when I calculate this delta G naught reaction, I'm sitting somewhere here.
Let's say it's a downhill process.
Delta G naught of the products.
Well, the first thing that happens, when I take these two boxes and mix them together, is delta G naught of the reaction's going to go up or going to go down, or stay the same.
Anybody want to guess?
When I go from here to here, before there's any reaction that happens.
Shall we vote?
How many say that it's going to go down?
How many say that it's going to go up?
How many say that it's going to stay the same?
We have about three people that said it's going to go up.
And a lot of people that say it's going to go down.
And a few people that are abstaining.
OK. We're mixing things.
Anybody want to change their votes?
Something is happening here.
And this entropy term here, when A and B come together and start mixing up.
You're changing your vote.
Alright.
It's going to go down.
Now we have near-unanimity.
So the first thing that happens is, you're going to go down here.
You're going to have delta G naught of the reactants in the mixture.
And if it were to go all the way, if A and B were to disappear, it still would be mixed.
And so the end product here would actually be down here.
It wouldn't be up in the products.
It would be the mixture.
So that's the first thing we know.
That this delta G naught reaction is not the full story.
And along the way, here I have two species to begin with.
I've got two species to end with.
But I've got three different species here in the middle.
So as soon as I form a little bit of products, in this case here, or if I start from products and go the other way in the reaction and form the reactants, the first thing that's going to happen is I'm going to decrease the delta G. Just from the entropy of mixing.
And so if I plot my delta G as a function of reaction conditions, I'm going to get a bowing curve like that.
If the entropy wasn't there, then it would just be a straight line from one to the other.
The entropy of mixing of reactants and products wasn't there.
Actually, let me put it up here.
If entropy of mixing wasn't there, I would start from up here.
And as my stoichiometry changed, I would have a linear curve from here to there as a function of the process of reaction.
But the entropy of mixing is causing my initial state and my final state to go down.
And it's causing a bow in this here.
Because if I have equal amounts of A, B, C, and D, that's a lot of entropy of mixing there.
So equilibrium is actually somewhere down here.
It's where delta G of the mixture is at its lowest.
Delta G of the mixture is at its lowest.
Any questions?
Now we going to do the math.
We're going to see how that comes about.
Let me do it here.
So the question we're going to ask is, suppose that I've got my mixture here.
And I'm sitting somewhere on this curve here.
So I've got pA for partial pressure of A, partial pressure of B, partial pressure of C, and partial pressure of D in my mixture.
And I want to know, I've got this mixture of reactants and products.
Which way is the reaction going to go?
Is it going to go towards the products?
Is it going to go towards the reactants?
Or is it at equilibrium?
And to answer that question, I'm going to let the reaction react a little bit more to create a little bit more products.
Remove a little bit of reactants.
And see what the sign of delta G is for that process.
So I'm going to go from a moles of A, b moles of B, c moles of C and d moles of D in my mixture, which is that point on my graph here.
Partial pressure A, partial pressure of B, partial pressure of C, and partial pressure D, and I'm going to react it a little bit more.
a plus, a minus epsilon times nu A, where epsilon is a very small number.
Of A, b minus epsilon times nu B times B.
And I'm going to have, now, products being formed.
c moles plus epsilon times nu C, I've going to have the stoichiometry in there.
For every nu A moles of A that I lose, I create mu C moles of C. Times epsilon as the scaling factor.
And d plus epsilon times nu D moles of D. That's going to be my new mixture.
And I'm going to ask, as I go from this initial state to that final state, I'm sitting on that curve, which sign is delta G. How do I know what the sign of delta G is?
What is delta G for this process?
Not the reaction with the isolated reactants and products, but the reaction where everything is mixed together.
Where I'm going to have to worry about the chemical potentials of mixtures.
And I'm going to go from one mixture to another mixture, and that's going to be the key to telling me if I'm in equilibrium or not.
But we know how to do this.
We know how to do this.
So I want to calculate delta G for the process is delta G after minus delta G before.
That's delta G after minus delta G before.
And I know that delta G, rather, let's call it G after minus G before, the Gibbs free energy after minus the Gibbs free energy before.
And I know the Gibbs free energy is just the sum of the chemical potentials, right?
You take all the species, and you take all the chemical potentials.
You add it up together times the stoichiometry.
And that gives you the Gibbs free energy.
That's what we learned last time.
So if I want to find what the Gibbs free energy at the end here is, I look at the chemical potentials of A, B, and C and D times their number of moles.
So I look at a minus epsilon times nu A times the chemical potential of A, plus b minus epsilon times nu B times the chemical potential of B plus c, plus epsilon times nu C, the chemical potential at C, plus d, plus epsilon times nu D, the chemical potential of D. And then I subtract what G was before.
This infinitesimally small process.
Was a times mu A plus b times mu B plus c times mu C, plus d times mu D. And we're assuming that my small change, the small amount of A and B that get destroyed to form C and D is small enough that the chemical potential basically stays the same during this infinitesimally small change.
That's why I can use the same chemical potentials before and after.
OK, a lot of things drop out.
This term drops out from that term.
This term drops out from this term.
This term drops out from this term.
This is a minus, no, this is all plus.
That's fine.
There's the minus sign right here.
So then this term drops out from that term.
And I am left with epsilon times nu C mu C, the chemical potentials of the products minus the sum of the chemical potentials of the reactants, nu A, mu A plus nu B mu B.
That's the delta G for this small change.
Now, you remember way back, maybe from even just last semester if you've taken 5.112 last semester, or from last year, or from high school, that the partial pressure is going to the equilibrium constant.
Somehow we're going to have to get partial pressures in there.
But we know now how to go from chemical potentials to partial pressures.
It's written right here.
The chemical potentials of the, and we also know how to go from the chemical potential in the mixed species, in the, mixture to the chemical potential in a pure.
We saw that mu A in the mixture, temperature, pressure was equal to mu A pure temperature, pressure plus RT log xA.
So those are things that we're going to be using, to go from something that has chemical potential to something where we'll be able to get back pressures, partial pressures, and delta G of the reaction, because we're going to need the chemical potential of the pure stuff.
So let me go forward just a little bit.
Remind you, if I look at the delta G of the reaction, delta G of the reaction, in terms of chemical potentials.
Delta G naught of the reaction.
This is the delta G of the products minus the delta G of the reactants when they're pure, not when they're mixed.
So delta G naught of the reaction is nu C mu C at one bar pure plus nu D mu D pure.
Minus nu A mu A pure minus nu B mu B in the pure state.
That's what delta G naught of the reaction is in terms of the chemical potentials of all these species.
Everything's at one bar, and everything is pure.
Somehow this is going to have to come out of this.
So let's keep going.
And see how it falls out.
So for each one of these chemical potentials, I'm going to write it in terms of the pressure.
What I just covered up here.
Mu(T, p) is mu naught of T times RT log p. So now, delta G is going to be equal to epsilon, and I'm going to do a little massaging quickly.
And I'll let you take this, go home and see how I went from one state to the other.
The secret is to put in here the pressure dependence.
Nu C mu C naught, plus nu D mu D naught, minus nu A mu A naught, plus nu B mu B naught, plus RT log pC to the nu C, pD to the nu D, divided by pA to the nu A, pB to the nu B.
These log partial pressures all come from expanding out the chemical potential as mu naught plus RT log p. And then we recognize that this part right here, this part right here is delta G naught reaction.
So I have delta G now, for this process of taking reactants to products just a little bit, let me take it as a function of epsilon here.
Delta G naught of the reaction plus RT log of this ratio of partial pressures.
I'm going to call that Q. I'm going to call this thing here Q, which you've seen before.
The reaction quotient.
And this tells me that for this very small process, epsilon, very small, if delta G naught, delta G, of this process, is less than zero, then the reaction will keep going forward.
It means that I'll be on the side of the curve here.
I'm going to go down a little bit.
delta G naught is going to be great, it's going to be spontaneous.
I'm not in equilibrium.
I'm going to go towards the products.
If delta G is equal to zero, then I'm at the bottom of this curve here.
I'm here.
If I go forward a little bit, the slope is zero.
Delta G is zero, I'm in equilibrium.
And if delta G is greater than zero, then I'm going to back to reactants, and I'm sitting, then, on that part of the curve here.
And if I try to make more products I'm just going uphill a little bit.
So now, I've done this all for epsilon is equal to very small.
What you usually find written in books is where the epsilon equals to one.
Basically taking one mole of this process.
So you'll see it per-mole of this reaction.
But it's the same idea.
It's the same thing.
And in fact it doesn't really matter, because the quantity that matters is this delta G naught reactions plus RT log Q.
It's the sign of this quantity here.
This epsilon is just an arbitrary number.
I can pick it, really whatever I want.
So it's the sign of this thing that's important.
Let me go back.
So what you'll see, then, is delta G is equal to delta G naught reaction plus RT log Q.
Basically, taking epsilon is going to zero.
As determining where the equilibrium is going to be.
OK, any questions?
All driven by entropy of mixing.
Without entropy of mixing, we would be sitting on this curve here.
Delta G naught of the reaction would tell us that everything should go to completion.
Things don't go to completion, and that's a good thing.
Otherwise there would not be life on Earth.
We're basically a set of equilibria in a big membrane.
Which is our skin, right?
All these biochemical cycles are in equilibrium with each other.
And it's a complicated process.
And it's all driven by entropy.
Ultimately.
And other things, but entropy is very important.
Alright, so equilibrium now.
Equilibrium is when we have this delta G equal to zero.
That's when delta G naught of the reaction equals RT log Q.
And at that point, we replace Q with to equilibrium, and we call that the equilibrium constant.
And we're going to put a little p here, because it's in terms of the partial pressures.
And it's equal to the partial pressures of the products raised to their stoichiometry.
And divided by the partial pressures of the reactants, raised to the power of their stoichiometry.
And this, you've seen before, I'm sure.
At equilibrium.
And there's a minus sign somewhere here.
Because it's this plus that that's equal to zero, and there's the minus sign that I forgot to write down.
So we define, then, equilibrium constant this way.
And one of the things to note is that Kp as written here is actually unitless.
It doesn't look like it.
The way that I've written it.
Because I did a shorthand way of writing the pressures, when I wrote RT log p here.
There's the assumption, when you have the log of the pressure, that there's always one bar sitting behind there.
Underneath.
It's always referenced to a reference pressure.
There's always a reference pressure.
p naught dividing it by, because you don't want to have any units inside the log.
And this reference pressure, we took as one bar.
And it's pretty common to just ignore the fact that you've got one bar sitting in the denominator.
And so, actually, all these pressures here are divided by p reference divided by p reference, divided by p reference, divided by p reference, which happens to be 1 bar.
But it's there.
And that means that the bars on top and the bars on the bottom cancel out.
Which means that K sub p doesn't have any units.
It is unitless.
It's a number.
Straight number.
Very common mistake to make, to write it and forget that there's one bar sitting on the bottom here.
And you take all the bars to the new powers on top, and the bars to the new powers on the bottom.
And make units there.
And get Kp is equal to some number, to the, times bar to some power.
That would be wrong.
I know I've made that mistake before.
But you are not going to make that mistake, right?
Because you've been warned.
That this is a common mistake.
This is unitless.
Now, you can invert this to get Kp as a function of delta G. e to the minus delta G naught of the reaction divided by RT.
And those are the things that you use to go back and forth between the thermodynamic quantities, like G that you calculate, to equilibrium quantities like the K, to finding equilibria between something like the Haber process.
Now, there's other equilibrium constant that is used a lot.
Which is the form of the equilibrium constant not in terms of the partial pressures, but in terms of the mole fraction.
That's also an important one when you're looking at solution cases.
Because we can rewrite the partial pressures using Dalton's law.
pC is equal to the mole fraction of species C times the total pressure, I'll call it p. So if I replace every one of these partial pressures using Dalton's law, I get K sub p is equal to xC pC times p to the nu C, times xD pD to the nu D, divided by xA pA - no, not pA, total pressure.
Dalton's law.
To the nu A, xB p to the nu B. OK, so I've got p, p, p, p. They all come out.
And that's p to the minus delta nu.
Where delta nu is the difference in the number of moles of reactants minus the number of moles of products.
So delta nu is nu C plus nu D minus nu A minus nu B.
It's the moles of products minus moles of reactants.
And then I have xA to the nu A, xC to the nu C, xD to the nu D, xA to the nu A, xB to the nu B.
And that ratio, which is in terms of mole fractions, we call K sub x.
The mole fractions.
Because p, to the minus delta nu, times Kx.
Now, this was unitless.
This pressure to the minus delta nu sitting here, this has bars to the minus delta nu.
Kx has units.
It may not look like it because it's a bunch of mole fractions, and it certainly doesn't look like it has any units.
Mole fractions or ratio, but it's got units.
So let's rewrite Kx in terms of Kp.
Rewrite Kx as equal to p to the minus delta nu, K sub p. And the units are in terms of bars, let's say, bar to the minus delta nu.
Always want to check your units at the end of the day.
If your units don't work out after you've done a calculation, you're in trouble.
Any questions?
Because we're going to end today, at here.
Yes
[INAUDIBLE]
In the notes it says Kp and Kx are both unitless.
That is a mistake.
I'm going to fix that, and put it on the Web.
So it may be that there's a special case where the number of moles before and after are the same.
But generally that is not true.
Where did I say that?
Oh, yeah.
Both unitless.
Well, that is actually true.
Let me think through this
[INAUDIBLE]
About one bar, one bar, one bar.
That is actually true.
They're both, that is true.
They are both unitless.
Well, this is a big boob on my part here.
Because you are absolutely right.
Because there's one bar sitting here.
one bar, one bar, one bar, one bar, and the bars cancel out.
Good catch.
OK, the notes are right.
My notes are right.
Alright.
Next time we'll talk about the temperature dependence and the pressure dependence of equilibrium constants.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to remind and clarify as needed the equilibrium constant Kp for gas phase reaction was the ratios of the partial pressures referenced to some reference pressure, which we usually take as one, one bar.
To the stoichiometry , pD divided by p naught, to the mu D where species C and D are products.
And the reactants are on the bottom.
And usually we don't write p naught, but it's important to remember that it's there.
And then we can also write this in terms of the Gibbs free energy for the reaction.
The standard Gibbs free energy, minus delta G naught of the reaction, divided by RT.
And what this tells us is that this is a number.
This is a number, there's no pre-factor here that has units.
That's a unitless number.
And it doesn't depend on the total pressure.
And the other thing to remember is that delta G naught for the reaction is the process of taking the reactants separated in separate boxes, separate containers, and the final product, the final step, is the products separated in individual containers.
That's what we write when we write delta G naught for the reaction.
We also looked at K in terms of mole fractions.
So if you replace all the partial pressures with the mole fraction times the total pressure, you get an expression for K sub x, which we define as the mole fraction of the products to the stoichiometric powers.
Which is also unitless.
This is true, it's unitless.
But, if you write it in terms of K sub p, the total pressure comes in here.
p total, divided by the reference pressure to the minus delta nu, where this is the change in the number of moles, in going from reactants to products.
And there's K sub p sitting here.
So unlike K sub p, which doesn't depend on the total pressure, K sub x does depend on the total pressure through this term right here.
So when we look at problems where we change the pressure, the total pressure of the system, this is going to stay the same.
Because it only cares about delta G naught for the reaction.
But this K sub x will depend on the total pressure.
And that's often a source of confusion in doing problems.
OK, any questions?
We're going to do an example where we change the pressure first.
So there are examples in the notes, and I'm going to skip the first one.
I'm going to go to the second one, which is the effect of the total pressure on the reaction.
And Le Chatelier's principle, for pressure.
And the example we're going to take is a fairly standard example, also.
Which is the reaction of N2O4, which is a gas, to 2 NO2, which is a gas, the kind of reaction that happens when you have smog and, fairly common in big cities.
The question we're going to ask is, what happens when we change the total pressure in this reaction here.
Which way does the equilibrium go?
Does it go to the right, does it go to the left?
Does it go to the products or the reactants.
And so, in order to answer that question, we're going to ask a slightly different question.
We're going to ask what is the molar ratio, what is the fraction, what is fraction of the reactant, the N2O4 that's reacted.
That has reacted.
And we're going to call that alpha.
It's the number of moles that have reacted divided by number of moles initially.
So we're going to need to find at equilibrium what is the number of moles of N2O4 that has reacted.
So we have to set up the problem.
And so the way that, the standard way of setting up the problem is to write the equilibrium, 2 NO2 gas.
And then on this line here, we write the initial conditions before we set up the equilibrium.
And let's say that we have n moles of N2O4 initially in the box.
And zero moles of the NO2.
So we have n moles here.
And zero moles here.
At equilibrium, let's write the number of moles.
A certain number of moles of N2O4 will have reacted, let's call that x.
So n minus x moles left.
For every x moles of N2O4 that's reacted, we create two moles of NO2.
So we have 2x here.
And then we're going to need the total number of moles, because we're going to be doing mole ratios, mole fractions.
So the total number of moles at any time is the sum of these two, n minus x plus 2x.
It's n plus x.
So if we're going to be writing our equilibrium constant in terms of mole fractions, we're going to need mole fractions.
So the mole fraction at any time is n minus x divided by the total number of moles, which we just calculated as n plus x.
And this is 2x divided by the total number of moles, n plus x.
And what we want is this ratio here.
We want the ratio of the number of moles reacted, which is x, that's the number of moles that's gone.
That have reacted.
Divided by the number of moles initially, which is n. That's what we want.
We want to see how that is going to change with pressure.
So we're going to deal first with Kp, because Kp doesn't depend on total pressure.
We're going to write that down.
Then we're going to go to Kx, somehow.
And that's going to depend on pressure.
So let's see what Kp is here.
So K sub p, you've got the products on top.
So it's the partial pressure of NO2 to the second power, divided by the partial pressure of N2O4 to the one power.
And everything is referenced to one bar, everything's in bar.
And in terms of the molar fractions, it's the total pressure squared, times the mole fraction of NO2 squared, divided by the total pressure to the first power.
So the square root on top gets divided by one factor of pressure.
So we have total pressure in front.
Divided by x to the N2O4, and that's p times Kx.
So let's plug in what these mole fractions are from our table here.
The mole fraction of NO2 is 2x divided by n plus x.
2x divided by n plus x to the square power.
Mole fraction of N2O4, n minus x over n plus x. n minus x over n plus x.
Multiply, square the top, 4x squared.
Divided by n plus x squared.
Things sort of cancel out here.
Rearrange.
4x squared divided by n squared minus x squared.
What we're really interested in is, we're not interested in x.
We're interested in x divided by n. So let's divide both the top and the bottom by n squared.
And we're going to get alpha come up.
So this is then p times 4 alpha squared divided by one minus alpha squared.
This is not, there's no total pressure here.
The only way, the only place, where the total pressure comes in, is right here.
This is just a number that doesn't care what the total pressure is.
Which is why we're using it.
And not K sub x, which cares what the total pressure is.
So now we can solve for alpha.
We can solve for alpha by rearranging this equation.
This is just a number.
And this is where the total pressure comes in.
So you rearrange that, and you get alpha is equal to 1 plus 4p divided by Kp, to the minus 1/2 power.
And if I rewrite that slightly to make it a little bit easier to see what's going to happen, when I change the pressure, 1 plus 4p divided by Kp, to the 1/2 power.
And that's what I'm after.
This tells me what happens at equilibrium to the amount of NO2 as I change the total pressure.
This is the only place where it comes in.
So now I can see that if I raise the pressure in my container, raise the pressure, this is in the denominator, so this fraction gets smaller.
Alpha gets smaller.
I raise the pressure, the fraction of material that reacts gets smaller.
Therefore, the reaction goes towards the reactants.
If I decrease the pressure, this is a smaller number here.
The fraction gets bigger.
Alpha goes up.
If I decrease the pressure and I compare what happens to the number of moles of reactants that react, more of it reacts.
Equilibrium shifts towards the product.
And this is Le Chatelier that you already know.
Le Chatelier's principle, for pressure.
The way it works is that Le Chatelier's principle states that, this chemical system wants to stay as close to what it was before.
It doesn't like change.
It doesn't want to have any change happen.
So if you increase the pressure, the chemical system says, hey, you know I'm not so happy that you're increasing the pressure on me.
I'd like to go back to a smaller pressure.
It doesn't like change.
Very conservative.
And the way to decrease the pressure is to decrease the number of moles in the container.
How does it decrease the number of moles?
Goes back to where there are fewer moles.
And that's on the reactant side.
How many of you know Lenz's law?
In magnetism.
The diamagnetic materials.
Right.
At least one person knows it.
It's the same idea.
You take a diamagnetic material in the absence of a magnetic field, and you slowly move it into a place where there's high magnetic field, what does the diamagnetic material do?
It orients its magnetic moment to reverse the field.
So that there's no field inside.
It starts out with no field.
Doesn't like change.
So Lenz's law says it's going to do whatever it can so that it retains no field inside.
Le Chatelier's principle is basically the same thing.
Equilibrium systems are very unhappy if you try to change them.
And that's what happens for Le Chatelier here with pressure.
Any questions?
So before we go to Le Chatelier's with temperature, and the van 't Hoff equation.
Let's do a little detour here and talk about equilibrium in solution, which is really as important, or if not important for a lot of you, then gas phase equilibrium.
Although gas phase equilibrium was where everything started.
And still a huge deal.
OK, so in equilibrium now, when we talk about equilibrium in solution, we still have to, still going to be the chemical potential.
It's still going to be looking at how chemical potential likes to go downhill.
And we're going to have to write chemical potential for a species, A, let's say, which is in solution.
At some concentration c sub A in solution.
And the concentration could be given in moles per liter.
Or it could be in grams per liter.
Or it could be in grams per 1000 grams.
Whatever your favorite unit of concentration is.
Use it.
Stick to it.
And in order to do equilibrium, we're going to have to reference it to, so this would be the species at some arbitrary concentration.
We're going to have to reference it to some reference concentration.
Just like we referenced everything to one bar before, as our standard pressure.
And we're going to take, usually you take one mole per liter, or one gram per liter, or one whatever.
One as your reference concentration.
And the reference concentration is going to disappear from the equation.
It's just like the reference pressure.
Disappear from the equation.
So we're going to reference this to some standard state chemical potential.
Where the naught refers now to the standard concentration.
And instead of having RT log p, now we're going to have RT log cA.
It's kind of like considering the molecules in the solution to act like an ideal gas.
Knowing fully well that behind, that underneath the cA, is this reference concentration of one.
One whatever is your favorite units.
Now, it's a little bit more complicated than for the ideal gas.
Because your solution may contain other things than your reactants and your products.
Especially if you're doing biology.
It could be a buffer.
It could be a buffer, it could have salt.
It could have a pH that's not equal to seven, whatever.
And so this reference, chemical potential, now needs to be referenced to a particular pH or salt concentration.
Or whatever the properties of your solvent are.
Or your solution are.
And that's the big difference.
In an ideal gas, it's reference to vacuum, basically.
There's nothing there.
In here, in solution you have all these molecules of solvent, molecules of salt, molecules of acid, or whatever, that are going to be around to buffer the pH.
And that's going to change what the chemical potential of a species is.
And if I change the pH and I've got a molecule that I'm interested in, it may not have an acidic moiety on it, but it could still care what the pH is, slightly.
And that would change what the reference potential is, chemical potential is.
So this is really important to remember.
And there are textbooks that are written on how to do this the right way.
We're not going to do that here.
We're just going to remember this is, we're going to assume that this is done correctly.
Once you take that as a given, that you have a way to have a reference chemical potential at a properly referenced pH and salt concentration, then you can go through the same analysis that we went to for partial pressures.
This looks just like the ideal gas, where the concentration replaces the partial pressure.
Or the pressure of chemical A.
And you can go through, then the same argument, where you take your reaction.
And you initially have some delta G for the reactants.
And you have some delta G for the products.
And the difference is the delta G for the reaction, delta G naught for the reaction, and then on this side here, you have a solution of A, so the reaction would be nu A times A, which is in a solution.
Temperature and pressure.
Plus nu B of reactant B, in a solution, temperature and pressure.
Going to a nu C, C solution, temperature and pressure plus nu D, D in a solution, constant temperature and pressure.
So this is taking a solution of A, in one container, a solution of B in another the container.
That's the initial point.
Mix them together.
Let them react.
Then you take the product, you put them in separate containers.
And that gets you the stuff that you have the reaction.
So when you mix A and B, you're going to have the same entropy of mixing.
You're going to lower the delta G. Of the solution.
And you're going to have the same curve that goes down like this.
To the mixture of products.
And just like for the gases, where we wanted to know what is the bottom of this curve which gives us the equilibrium, we can do the same thing.
Exactly the same thing, for solutions.
And so we start out with a mixture of the A and B in solution and C and D, reactants and products together.
And we let the reaction proceed a little bit.
And we look at the change in delta G, going from, say this point here through that point here.
This will be delta G of epsilon.
We ask, is this positive, negative, or zero.
And if it's zero, that means that we're in equilibrium, that we're actually sitting down here.
And that gives us the equilibrium constant.
So, just for the sake of completeness, let me just write down what we would do.
We would react it for a small amount.
And then we'd end up with nu C. So you would have the chemical potentials of the products minus the chemical potentials of the reactants, nu C mu C, plus nu D mu D, minus nu A mu A, minus nu B mu B.
And instead of these chemical potentials, you would write them in terms of the pure chemical potentials times their concentrations.
And then you'd end up with epsilon times delta G naught of the reaction, plus RT log, and then the concentrations.
And then you write them in a different way.
So if it's moles per liter, you usually write that with these brackets here.
That means concentration of A in moles per liter, I'd say.
So C to the nu C power, D to the nu D power, A to the nu A power, and B to the nu B power, in these concentrations.
Where this ratio of logs comes from expanding out the chemical potential here.
And there's the log term here.
Just like an ideal gas.
Then at equilibrium, this is equal to zero, you're at the bottom of that curve.
And you set these two things equal to each other.
And you get the chemical, you get your equation that you know well.
For the equilibrium constant.
And this time it's not K sub p, it's just K. And that's what you know from doing solution equilibrium.
And it's just like V. So the thing to remember, which is the slightly more advanced part, which you'll have to worry about at some point if you stay in some sort of biochemistry oriented field, is that you've got to reference your initial solution properly.
To get to the right equilibrium constant.
OK, any questions?
Alright, then now we can do the temperature dependence of the equilibrium constant in a general way.
Whether it be a gas or a solution.
It doesn't matter.
It's going to be the same thing.
So the question that we ask now is, suppose that I change the temperature of my equilibrium, which way is the equilibrium going to shift?
And you all know the answer already, probably.
But let's derive it out.
So we're going to want to know basically, we want to know what is dKp/dT.
How does equilibrium constant, or dK/dT, if you're doing solution, how does the equilibrium change with temperature?
What's the slope?
Is it positive, negative?
If we have this, we can integrate it out.
We can do an integral over temperature.
And get an actual change.
So that's our goal.
To find how the equilibrium constant changes with temperature.
What do we know?
Well, we know how Kp depends on temperature, through the Gibbs free energy of the reaction.
The Gibbs free energy, delta G naught, has a temperature dependence.
And then there's an RT sitting on the bottom.
There's another temperature dependence here.
Well, dKp/dT is sort of like, we could also ask what's d log Kp dT.
That might be an easier question.
It's basically the same question.
Especially since we have something which is log K, is equal to something.
So let's ask this question instead.
Let's ask, what is d log Kp dT?
Alright, so let's differentiate both sides.
d/dT, d/dT here.
Got to use the chain rule now.
Because we've got temperature as part of delta G Write it out.
So let's take the derivative with respect to the temperature on the bottom first.
We have delta G naught, which is a function of temperature, divided by RT squared.
The minus sign here disappears when you take the derivative on the bottom.
Minus one over RT, d/dT of delta G naught.
So this is a derivative of delta G, where zero means here one bar.
Fixed at one bar.
So really, d/dT, with delta G naught fixed on one bar, is the same thing as the partial derivative of delta G with respect to temperature, keeping p is equal to constant at one bar.
It's the same thing, just different notation.
And we know what this is.
In terms of other things that we can find in books, like delta H, or delta S. Because we can go to the fundamental equations.
To find out how delta G depends on temperature.
And our goal is to get rid of delta G, which clearly has a nice temperature dependence through the entropy term.
And to replace delta G with delta H, if we can.
Because delta h is going to be much less sensitive to temperature and it's, delta H is going to be over small temperature ranges, is going to be independent of temperature.
And we know the temperature dependence of delta H, because it's through the heat capacities.
So our goal is to get rid of this delta G here.
And to try to replace it with delta H, if at all possible.
So we go to the fundamental equation for G, dG is equal to minus S dT plus V dp.
And sitting right here is dG/dT at constant p. Which is what we have here.
So we get rid of this derivative of G. And replace it with S. So now, now we have d log Kp dT, and I mentioned already that I want to get rid of G. Because it has a strong temperature dependence.
And I want to somehow get H in there, which is not going to have a strong temperature dependence.
And delta G naught, I can write in terms of H and S, and T. Delta H naught minus T delta S naught divided by RT squared.
And then my derivative here, I have minus one over RT.
Partial of G with respect to T. p is equal to one bar.
Well, that's just delta S. p is equal to one bar, well, that's just delta S naught.
Times delta S naught.
And that's great, because now there's minus T divided by RT squared.
That's one over RT.
And somewhere I've lost a sign.
And there's my sign that I lost, right there.
This minus sign here.
d/dT of delta G naught is minus S. It actually includes this minus sign right here.
Which is great, because now things work out, because this becomes a plus sign.
And this and this cancel out.
And this becomes delta H naught over RT squared.
Great, so we have what we want.
We have how the equilibrium constant depends on temperature in a way which is very clear.
Where the top part is only very weakly dependent on temperature, usually.
And this is called the van 't Hoff equation.
And this will tell us what happens to equilibrium when we change the temperature.
So if you want to do a finite temperature change, now what you need to do is, you need to integrate.
You integrate both sides here.
From some T1 to T2.
From T1 to T2, and that tells you, then, that the log of the equilibrium constant at the new temperature is equal to the log of the equilibrium constant of the old temperature, T1, plus the integral from T1 to T2 of delta H over RT delta H naught, over RT squared.
And this could be slightly temperature dependent.
dT.
And this is the integrated van 't Hoff equation.
And if you're going to be designing a chemical plant where you have high temperatures and high pressures around, you better use that.
Because there is some temperature dependence in delta H, through the heat capacities of the reactants and the products.
And that could make the difference between your plant running nice and smoothly or your plant exploding.
And you don't want to have exploding plants around.
So for heavy-duty uses of this equation, you've got to do the integral properly.
But for most normal applications, like if you're doing biology, where the temperature changes by a few degrees, like today I have a little bit of a cold.
I don't have a fever, but I could have a fever.
So my biochemistry would change if I had a fever.
The equilibrium constant of all my reactions would change a little bit.
It's a small change in temperature.
I'm not going to explode.
And so you can then take the approximation.
In that case, the delta H naught, is independent of T. And this is fine over small temperature ranges.
And that's the one that you're most used to.
Is this approximation here, this approximate van 't Hoff equation.
Which is really fine for most cases that you're going to be dealing with.
So then, if that's the case then you can take your delta H. Ignore the temperature dependence and take it outside of the integral.
And now you can do the integral fine.
And then you have an analytic expression for the change in the equilibrium constant with temperature.
Log Kp at a new temperature, T2, is log Kp temperature T1.
And I'm carrying this little p around everywhere.
But really, it doesn't have to be there.
This could be solution.
I shouldn't really have written this for the specific case of partial pressures.
But it's equally valid for solutions.
Then we have delta H naught over R. And then we have the integral from T1 to T2, over one of RT squared.
And if you do that the right way, you get T2 minus T1 over T1 T2.
And that gives you the approximate van 't Hoff equation.
Which is fine.
And you'll know that it's fine in problem sets or exam, because we'll say assume that delta H is temperature - yes
You said that the [INAUDIBLE]
For K, if K is solution
Right
Yeah
So then, can you also use Kx?
Can you use Kx?
Well, as long as you keep the pressure, the total pressure, constant, then you should be able to use Kx.
Let me think about this.
Yeah, here you would have p, you have a log c so, you can use Kx, fine.
Because then you would have log p to the minus delta nu times Kx, log p to the delta nu minus Kx.
And the log of the multiplication is the sum of the logs.
And the logs will just fall out.
So it could be any K that you want.
Doesn't matter.
As long as you keep the pressure constant.
If you change the pressure, then you're in trouble.
So now we can see what happens when you do change the temperature.
If I have some equilibrium, and it's all going to depend on the sign of delta H. Whether the reaction is exothermic or endothermic.
And it's the same thing as Le Chatelier's for pressure, or Lenz's law.
The system doesn't want to have change happening.
So if you have something that's, delta H is less than zero, it's exothermic.
Exothermic, that means that it's putting out heat.
it wants to heat up its environment.
And if I take temperature and I raise the temperature, the system's not going to like that very much.
It doesn't want to get hotter, and going from reactants to products makes things hotter.
And if you go from products to reactants, that's the opposite.
Go from reactants to products, that becomes endothermic.
It sucks in heat.
You raise the temperature.
The system said no, no, no, I'm happy where I am at my original temperature.
I'm going to start sucking in heat, to try to get the temperature down.
And it's going to try to make more product.
More reactants.
So, equilibrium K is going to go down.
and the reaction goes towards the reactants.
And the opposite if you have something that's endothermic to begin with.
OK, delta H is positive here.
Delta H naught is positive.
You raise the temperature, delta H naught is positive, T2's bigger.
This is a positive number.
K becomes larger at higher temperature.
K goes up.
The reaction goes to products.
So if you think of it in terms of the system, the system is at some temperature.
You raise the temperature.
System doesn't like it.
Says, I want to go back to my original temperature, what can I do.
I can try to suck in heat that you're trying to put in the environment.
That's great because if I make more products, that's endothermic.
And I'm just going to make more products until I try to lower my temperature.
So I move the equilibrium to the products.
OK, Le Chatelier for temperature.
Any questions?
On equilibrium.
OK, let's do a quick example.
Because this was the example that we started out with, talking about, the Haber process.
This important industrial reaction that started the chemical industry, essentially.
That uses up 1%, or close to 1%, of the world's energy.
If you think about it, that's an amazing number.
1% of all energy produced in the world goes to one reaction.
One industrial reaction.
Just shows how important it is.
OK, why does it take so much energy?
We're going to find out.
We're going to find out why it takes so much energy to run this here reaction.
Alright, let's look at this Haber process.
Take some nitrogen gas.
Plus some hydrogen gas.
And this is usually done over catalysts, like an iron oxide catalyst or something.
To try to speed it up.
It doesn't change the thermodynamics.
As you know, and you'll hear again in this class, catalysts just affect the kinetics.
They don't change the thermodynamics.
So this is usually done over some catalyst to try to speed things up.
To make ammonia.
And ammonia becomes the feedstock for fertilizers, for almost anything that contains an amine in it, or a nitrogen in it.
If you're going to make proteins or whatever.
You've got to have ammonia somewhere in the process.
OK, delta H naught of the reaction, we're given all these numbers.
At 298 degrees Kelvin.
And they're in your notes, so I'm not going to go through them in detail.
Delta G naught for the reaction, we're given that number.
At 298 degrees Kelvin, that's minus 16, roughly minus 16 kilojoules per mole.
And we want to know, what is the equilibrium constant.
Room temperature.
So you know how to calculate that.
Minus RT log Kp, log K is equal to minus RT.
Minus delta G naught over RT.
So you put that in there.
You get Kp is equal to 860.
A number, no units.
It's a big number.
It's a big number, you've got the products divided by the reactants.
It means that the products are favored.
This is great.
What a wonderful reaction.
Shouldn't take energy to for us to do that, right?
It's a room temperature reaction.
Thermodynamics is great.
But even over a catalyst, this is a really, really, really slow reaction.
We'd still be waiting here for Mr. Haber to produce his first mole of amine, if you were doing it, or ammonia if we were doing it at room temperature.
It's just so slow.
Thus, not at all practical.
We're not going to run the world on room temperature Haber process.
But it turns out, if you raise the temperature, kinetics is wonderful in terms of the temperature dependence.
It's exponential.
Arrhenius rate law.
Great thing, you raise the temperature by a little bit.
Rates speed up, things go faster.
So if you were to raise the temperature from 298 degrees Kelvin to 800 degrees Kelvin, the rate speeds up.
You're going to need some energy.
As input here, to feed that.
Hence the 1% energy use.
Rate speeds up, that's great.
Things happen faster.
It becomes practical.
But, what happens if you raise the temperature?
Let's see.
This is an endothermic, or exothermic, negative sign.
And negative sign's exothermic.
I raise the temperature, K goes down.
I know how to calculate it here.
And if I want to be super careful, because it's a fairly large temperature range, I can even use the exact form of the van 't Hoff equation.
And what I find, if I do that, and putting the heat capacities for all these gases, I find that Kp does go down.
In fact, it goes down quite a bit.
It becomes 0.0007.
Two zero's.
Still really small.
That's not practical.
Not practical at all.
No good.
We can't run an industrial plant with this kind of yield.
There's just no way it's going to work.
So, you're a chemical engineer.
Or a chemist, like Haber and Bosch were.
And you're trying, you know by Le Chatelier, you know that it went in the wrong direction for you here.
And then you look at your reaction and you say, how many moles of reactants do I have?
3/2 plus 1/2, that's two moles of reactants.
And I've got one mole of product.
Two moles reactants, one mole of product.
Two moles reactant... What happens if I change the pressure?
If I change the pressure, if I increase the pressure, the system is going to say, no, I don't want the pressure increased.
It's going to go to where there's less moles.
And the less moles in the product area.
It's going to go to my product.
That's great.
I've got to increase the pressure.
Wonderful.
Let's start increasing the pressure.
Again, we need some energy to do that.
We're going to go from one bar to some higher pressure.
It's going to make our lives more complicated.
The plant might explode now.
If the pressure's too high.
All sorts of problems going to come into play.
But, let's do it.
Let's increase the pressure.
So, you increase the pressure from one bar to 100 bar.
And you calculate Kx.
Which is really what you want.
So Kx, in this case here, is equal to p times Kp.
Kp doesn't change.
Kp doesn't care what the total pressure is.
It's Kx that cares.
At one bar, Kx is equal Kp.
Kx is p to the minus delta nu.
Number of moles of products minus the number of reactants.
If I go from one bar to 100 bars, Kx goes from 0.007 to 100 times 0.007, which is equal to 0.7.
That's a lot better.
Kx is the mole fraction or the, of products divided by reactants.
And if I go to p is equal to 300 bars, then Kx goes to 2.1. three times 0.7.
This is great.
Now I'm really starting to make good products.
But I've got to go to 300 bar.
I've got to go to 300 bar, and 800 degrees Kelvin.
That is incredibly energy-intensive.
But it works.
That's why Haber and Bosch made this work.
And why Germany stayed in the war longer than after 1916.
The first world war.
Ended in 1918.
Nobel Prizes.
Merck, Bayer, all these german companies.
Because they figured how to do this at high pressure and high temperature.
Without blowing everything up.
OK. Any questions?
Alright.
The last topic is, so far we've seen equilibria where you had things that were well mixed.
equilibria of ideal gases.
Or in solutions, where your solutes, your solute molecules are mixing around.
And the entropy of mixing was really super important.
Our curve, our going down for delta G, was all because of the entropy of mixing.
Now, suppose that I have a heterogeneous mixture.
I've got some solids or some pure liquids that are refusing to share their environment.
And staying as pure materials.
So, for instance, if I have a beaker with some solid reactant on the bottom here.
And the products are in solution.
How do I deal with that equilibrium?
Well, you know the answer, but let's just do it out again.
So we're going to have nu A moles of A, of solid.
Not mixed in the solution.
Let's say we have multiple phases here.
Nu B moles of B, which is a gas.
Instead of a solution, let's do a gas phase reaction.
Nu C moles of C, which is a pure liquid.
And nu D moles of D, which is a gas.
So these two gases can mix.
But the pure solid and the pure liquid can't mix.
So let's think again, where does the equilibrium constant come from?
It comes from looking at this delta G of the mixture and letting it react a little bit more.
And taking out these chemical potentials for the species and the mixture.
Expanding it out in terms of log p or log concentration.
So we need to have this delta G in.
Let's take epsilon equal to one, to make our life simpler here.
So now we have nu C mu C of the pure.
The pure solid.
Plus nu D mu C of the gas.
Which is in the mixture.
Minus nu A mu A of the pure liquid, minus nu B mu B of the gas.
Which is in the mixture.
That's what this delta G is, when we allow the reaction to proceed for a little bit more.
We add a little bit of chemical potentials from the products.
Subtract a little bit of chemical potential from the reactants.
And then we expand it out in terms of the standard chemical potentials for everything being pure.
Entropy of mixing comes in here.
Comes in here.
Delta G of mixing comes in here.
And we end up with something that looks like nu C mu C naught, plus nu D mu D naught, minus nu A mu A naught, minus nu B mu B naught.
Plus RT log, and the only place where we have these log p's, or log concentration coming in, is for those species that were not pure.
And those are only these two guys here.
The ones that are in the gas phase.
D and B.
So we end up with partial pressure of D to the nu D power.
Partial pressure of B, to the nu B power.
The other two species don't come in there.
Because they started out as pure.
There's no mixing going on.
And there's no expansion of the log for them.
And so now, when we look at the Q for the reaction, the reaction quotient, it doesn't contain any of the pure species.
It only contains those species that are allowed to mix.
Those that are in the gas phase or in solution.
And so K, then, for this reaction only takes in those products like D. Or reactants like B, which are in the gas phase.
The pure solids or pure liquids don't come in.
They come in for delta G naught.
There's delta G naught sitting right here.
Delta G naught for the reaction is sitting right there.
So when you write your log K is equal to minus delta G naught over RT, the delta G naught has everything in it.
The pure stuff, the solution stuff, the gas phase stuff.
But the K only has the gas phase and solution stuff.
Alright, any questions?
Good.
Next time we'll do an example.
And then we'll go on phase transitions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to do an example of using some of the equilibrium concepts that we've learned to drug design.
And it's going to be an example that's out in the literature.
And potentially a very big deal if you can do drug design this way.
And it's an example of how, remember how I told you that if you have the Gibbs free energy, you have everything and how clueless I was as a graduate student, and when I look back I think how silly it was for me not to realize how important it was.
Well, this is an example where people calculate delta G's or Gibbs free energy changes for binding of proteins to receptors on cells, or ligands.
And from that, design drugs that work much better than the real thing.
And it's all about calculating delta G. We're going to see how that is, and how that comes in with equilibrium.
So the paper that this is based on was published in 2002.
And since then there have been larger-scale trials, animal trials.
And I believe there have been human trials of this concept, of this drug that they have designed.
You've got the reference in the notes.
And we're going to have to do a little bit of review of biology first, to figure out what's going on.
And this particular process, this particular drug, is called the, I have it right here, granulocyte stimulating factor.
Where was it in here?
To give you the right name for it.
There it is.
It's a protein drug called GCSF.
Granulocyte Colony Stimulating Factor.
And it's there's wild tie for natural version of this protein.
That is generated by normal tissue.
And this protein goes to the blood, to the bone marrow.
And it binds the receptors on cells that are in the blood and the bone marrow, and stimulates the growth of white blood cells, and stem cells, and also acts to tell the bone marrow to pump out these stem cells into the blood.
And the problem is that if you have a chemotherapy patient, their bone marrow cells are largely destroyed through the chemotherapy.
And so they have a problem with the white blood cell counts, and with stem cell counts and all that stuff.
So one of the ways that you can try to reverse this problem is by stimulating, artificially stimulating the growth or the proliferation of these white blood cells.
And one of the ways you could do that is by giving the patient a lot of this protein, granulocyte colony stimulating factor, to stimulate the output of white blood cells from the bone marrow.
Whatever is left of the bone marrow.
To rebuild it up.
And so that's done with wild type, with protein.
You can make the wild type protein.
But if you had something that was basically the same thing but somehow mutated, where you made a few changes to the amino acid sequence, so that it worked a little bit better, then you might have a lot of people, a lot of patients.
And that's what the basis of this paper is, that's referenced here.
Is how to go about mutating this protein here to make it a drug that would be more effective then the natural protein.
So that you can give it to patients.
So now let me go back and tell you a little bit about this equilibrium that we're talking about.
So we've been talking about equilibria of molecules coming together.
And then creating new molecules, products and reactants.
Well, when you're talking about ligands and receptors on cells, the same sort of ideas come in.
The same equilibrium concepts come in.
So if you have a cell membrane, the cell membrane could have a receptor in it.
Which is a protein that extends through the membrane that's got some sort of pocket outside here.
The membrane keeps on going.
And you've got the blood on the outside here, or some of the tissue of the cells, and you've got the inside of the cell here.
And this receptor is waiting for some ligand.
A ligand being another protein that's floating around.
That's specific for binding to this receptor here.
And there's the ligand here.
There's the receptor here.
And every now and then one of these ligands will find a receptor and bind to it.
So we write, then, in equilibrium, ligand plus receptor goes to complex.
This is happening all the time on cells.
There are millions of different kinds of receptors, millions, billions of kinds of ligands.
Wild types, fake ones, et cetera et cetera.
When you've got a binding event, the cell knows that's something has bound, usually.
And that usually triggers a cascade of other things.
It signals a cell to do a lot of things.
So, for instance, in the case of the GCSF, you've got the receptor on the cell.
And you've got this granulocyte colony signaling factor, that would be the ligand here.
It binds to the receptor on the cell.
That triggers the bone marrow cells to produce more proteins, which signals more things to happen, and eventually down the way, after a bunch of signaling processes, out come a burst of stem cells or white blood cells.
It's a complicated process, and every step of the way needs to be done correctly.
And for this process at least, this protein here, this ligand protein, triggers the cascade.
And so this binding and unbinding of this protein then triggers the whole sequence of reactions.
So this equilibrium becomes super-important then.
OK. What else should we review about, I should have used a different board than this, but.
This is going to get covered up.
What else can we review about biology that would be interesting?
So, other things that we will need to remember, need to know, is that these receptors on the surface of the cells aren't static.
The cells recycle their receptors.
Things can happen.
Things get degraded.
And so the cell is constantly taking these receptors, bringing them back inside the cell, merging them with lysosomes, where the pH is five, or 5.5 compared to the outside of the blood, which is 7.4.
It breaks down the proteins into their amino acids, the cell can use the amino acids again to make more proteins.
Make better, make new receptors.
And that's how the cell recycles the receptors.
So basically, you end up with the cell taking the receptor.
Making a small sort of cavity around the receptor.
So there's the cell sitting here.
There's the inside of the cell.
There's the receptor being engulfed by the cell, now.
And it could have the ligand in on it, also.
There's the ligand sitting right here.
It's called an endocytotic event, or it's endocytosis of the receptor.
Into the cell.
And then once you're inside the cell, let's put the nucleus in the middle here, you've got things called lysosomes that are sitting nearby.
There's a lysosome here.
Lysosome.
And there's your little vesicle that contains your receptor.
Merges towards the lysosome, these two get together.
Then you have your ligand, I'm running out of colors here.
So there's the receptor and the ligand together in here.
And then the pH here becomes 5.5, things break apart.
And you've got to find another receptor, another ligand, to continue the process.
Everybody knows this biology, or you know it well enough.
OK, good.
So let's go do this example.
So this is the process of binding.
We can have a delta G associated with this, delta G a, delta G0 is minus RT log K sub a.
This is the association equilibrium.
And you can have the reverse process, where the complex gets broken up into receptor plus a ligand.
And then you have a delta G for this process here.
So this is delta Ga, let's call it.
This would be delta GD, which is the negative minus RT log K sub D. This is the dissociation process.
And the dissociation process, K sub D, is equal to R, the concentration of the receptors times the concentration of the ligand, divided by the concentration of the complex.
And the lower K sub D is, the smaller this number is, small means that you're mostly on this side here, mostly in the complex, the tighter the binding.
So small KD means tight binding.
So if I want, in principle, then, if I want to design a drug that's going to signal this event here, I want K sub D for this ligand that I'm going to design to be very small.
To be small.
Small enough so that it binds strongly and does its job, so I don't need too much of it.
Now, you need to do experiments to figure out what's going on.
So how do you do these experiments?
You need to be able to measure, then these K sub D's, experimentally, to see whether or not what you've designed on the computer, when you calculate delta G's on the computer you've got to know whether or not it's working.
And this is still an experimental science.
How does it work?
So, usually you do the experiment with the ligand concentration very high.
Very large.
So that the concentration of the ligand at any time is basically the same thing as the concentration of the ligand you put in.
So you overwhelm the system with ligands.
So that L is much bigger than C or R, and so it doesn't matter which way the equilibrium is.
L stays roughly the same.
So you know, throughout the experiment, what concentration is.
Then you get a bunch of cells.
In a well or something, or a 96 well plate with a bunch of different kinds of ligands.
And you can take your ligand, you can label it radioactively.
So you can take your ligand as some long protein.
And you can take an iodine 125, let's say, radioactive label on your ligand, on your protein.
So you can keep track of it.
The nice thing about radioactive labels, and which is why people use them in biology or bio-medicine.
We use them to look at, for instance, bio-distribution of things in animals.
We want to know where things are, and whether they all left the animal, or.
Because you can use a Geiger counter and you can count the events.
The radioactive events.
And that gives you extremely quantitative analysis of where things are.
So you take those radioactive ligand.
And, L0, very large concentration.
You expose this concentration to the cells.
The cells have some receptors on the surface.
There's a bunch of receptors on the surface of the cell.
And some fraction of the receptors will have the ligand bound to them.
So you incubate.
You let it wait a while.
Then you wash.
So you get rid of all the excess ligand that's on top there.
And then you take your petri dish, or your 96 well plate.
And you read the radioactivity that's coming from the cells.
And that signal, that radioactive signals, tells you exactly how many ligand you have on these cells.
You knew what the cell concentration was initially.
So that tells you what the concentration of ligands, of complexes, was.
So this experiment then gets you, experimentally, gets you the concentration of C, the complex.
So now this is something you know.
You also know what L0 was, because that's what you put in.
And that's enough for you to find out what KD is.
And if you know what KD is, then you know what delta G0 is.
And so generally, then, let's go ahead and do that.
So we know what L is.
We know what L0, C is, let me just do it on this board here.
So we start with rewriting KD as equal to R times L divided by the complex concentration.
And now L is basically L0, so we can use that approximation.
So we have L0 sitting here.
We still have the complex on the bottom, and that's something that we've experimentally discovered.
And the concentration of receptors, this is the concentration of receptors that don't have anything bound to them.
So it's these guys right here.
That's equal to the concentration of receptors, the total number of receptors, minus those receptors that have a ligand bound to them.
Complexes.
Something we can experimentally define, or discover.
So we replace this R here with RT minus C. And then we're all at equilibrium.
So we're going to put a little equilibrium sign under these C's here.
equilibrium.
And then we can rearrange this equation so that, people like to have plots that are linear, in a way that is a linear plot, where the slope of the plot is the inverse of the equilibrium constant.
So you do some massaging of this equation here.
And you rewrite it in terms of C over L0.
There's the equilibrium concentration of the complex.
This is the total receptor concentration divided by the equilibrium constant.
And then you have the equilibrium concentration of the complex divided by KD.
And so you plot, then, this ratio.
Which is an experimental ratio.
You've just measured this concentration of complexes using this radioactivity, radioactive labeling experiment.
You know what this is, because that's what you've put in.
This is your x-axis on your plot.
This is what you've measured.
And this is going to be the slope.
It's called a Scatchard plot.
After Mr. Scatchard So you get a straight line.
So we're plotting here the equilibrium constant of the complex.
And on this here we're plotting the ratio of the complex divided by L0.
And we get this straight line like this, where the slope is one over KD.
Minus one over KD.
OK, now there's a couple of things that you can look at that are sometimes useful.
In this analysis here.
You can also rewrite this equation up here in terms of the ratio of C equilibrium divided by RT.
So that's basically the ratio of receptors that have a ligand attached to them, divided by the total concentration of receptors.
So if most of the receptors are empty, then this is a small number.
Then most of the receptors are taken up, this is a number close to one.
It can't be anywhere, it can't be ever bigger than one, because the biggest number of complexes you can get is the total number of receptors.
So if you take this equation here and you massage it a little bit, one over one plus KD over L0, and that, you can also plot that.
As a function of L0.
How much ligands you put in.
And you find this is something that saturates at one.
So this ratio here is going to saturate at one.
This is C equilibrium divided by total number of receptors.
And so if L0 is small enough, if L0 is small enough, meaning that it's smaller than KD, so if L0 is much smaller then KD, then you can rearrange this ratio here so that C equilibrium divided by RT is roughly L0 over KD.
So that's linear, with a slope of one over KD.
So this slope here is one over KD is the slope.
And as you get L0 to be quite large, bigger than KD, then this saturate to one.
And this one over something very large become zero, and basically you end up with something close to one.
So you saturate at one.
So that's another way of doing this.
But this is really what we want to concentrate here.
The fact that you can get KD out of experimentally.
If you have KD, you have delta G. So now let's go back and think about this whole process here.
If we're going to design this drug here, this protein, artificially.
So what you want, then, is you want something that's going to bind strongly enough to receptor, to stimulate the growth of granulocytes, or the colony of granulocytes.
But when the cell decides to recycle the receptor, and destroy it, chew it apart in the lysosome here, you want this drug not to be degraded.
Because you have to keep injecting in the patient.
So you want this drug to release from the receptor, before the lysosome has a chance to chew it up.
So that means that you want the equilibrium constant at pH 5.5, you want that KD, to be much larger at 5.5 than you want it at 7.4.
You want strong binding at pH 7.4, and you want weak binding at pH 5.5.
That way the drug gets recycled.
Can find, go back to the blood.
Bind to a receptor again, generate the signaling events, gets engulfed by the cell.
Releases before it has a chance to be chewed up, et cetera.
The wild type will get chewed up.
The regular kind will get chewed up, and so that's why you have to, with the patients you have to keep giving them this drug over and over again.
Because it gets chewed up by the cells.
So that's the design principle that these authors had in mind when they started their study.
And so they knew what the sequence, what the amino acid sequence was, for this wild type drug, and they started doing point mutations.
Changing one amino acid here and there.
And cranking out the calculation to calculate delta G for binding of this protein to this receptor.
At pH 7.4 and at pH 5.5, and getting differences of delta G's.
So let me then review again.
What is the motivation here.
The motivation is to try to get the ratio, KD, at 5.5 divided by KD at 7.4, pH 7.4.
And you want this to be bigger than one.
And really as large as possible.
As possible, of course within limits.
So we want this to bind at pH 7.4.
And the point of comparison is the wild types.
Which at 7.4 has a KD of 270, roughly.
And at the ratio of pH 5.5 to 7.4 is 1.7.
So it's a little bit weaker binding at 5.5.
Remember, large KD means weak binding.
Small KD means tight binding.
So the fact that KD at 5.5 is bigger than KD at 7.5 means that it's slightly weaker binding at 5.5 than 7.4.
So this is sort of the baseline that the protein designers had to deal with.
So everything gets compared to this guy here.
So they don't actually, in a calculation they don't actually measure this.
What they do is, they look at delta G's.
And so they look at differences of delta G's.
They look at delta G0 for the binding of the ligand to the receptor.
At 7.4 minus the delta G0 at pH 5.5, let's call this the delta delta G. Since you know delta G is minus RT log K, this is equal to minus the delta of log KD, which is log KD at 5.5 divided by KD at 7.4.
So then they calculate delta delta G0, and it's completely related to this ratio that you measured in the experiment.
And so for the wild type, this delta delta G is just basically the log of this number here.
Is 0.53.
So now they go ahead and do their calculation.
And they find a couple of mutants.
Where, and they focus on delta delta G0.
They want something that's bigger than this.
And then they'll worry about whether there are more problems associated with it.
So they found two mutants, let's call them D110H and D113H.
I think it's a histidine mutation, point mutation at 110 and 113, where this ratio here was 8.3 and 17.
So, huge differences here.
More than a factor of 20, or factor of 30 difference in the change in the binding efficiency, at least according to these delta delta G, between pH 5.5 and pH 7.4.
That means that this protein here with the point mutation, with this one point mutation, if it binds as strongly to the receptor on the surface, as soon as the lysosome comes in and begins to decrease pH inside the cell, this ligand is going to come off.
And it's going to be able to float away.
Hopefully, before it gets recycled.
Before it gets chewed up.
So they can be used again.
So that's good.
Alright, so experiments were done.
And on the D110H and the D113H.
And KD was measured in the experiments.
And remember, the wild type is 270.
The KD's were at 370.
And 320, with some error bar.
And the different, the ratios of KD's were measured.
There were 4.4 and 6.8.
With some error bars.
And they turned out to match reasonably well, at least as far as comparison between experiment and theory.
Still not great for these sorts of calculations, because pretty involved.
A lot of approximations go on in there.
And it gives you a rough guide.
So this 17 here is probably not quite right, because it doesn't quite translate to 6.8 here.
This difference between 4.4 and 6.8 doesn't quite match the difference that they saw here in the calculation.
But it's the right direction, right?
And that's why you still need to do experiments.
This number here is a little bit bigger than this number here.
Which means that these don't bind quite as strongly.
Because KD small means strong binding.
That means the initial binding of the ligand to the receptor is not quite as good as the wild type.
That's often the case.
It's often not so easy to find something that binds as strongly as the wild version.
But, it's good enough.
And this ratio is really what clamps the deal in this case here.
And so, these mutants, in fact these two mutants are the ones that have undergone the animal trials.
That are in the pipeline.
This is a big deal.
This is a big deal because this is a huge, huge market.
There's a very large number of people that are affected with chemotherapy.
And this is basic thermodynamics here.
It's basic thermodynamic calculation of a complicated molecule with some fairly simple equilibrium concepts.
OK, any questions on this?
OK, well, we're ended really early today.
I don't have the phase transition ready, but Keith Nelson is going to start a phase transition next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I want to start us off on a somewhat new topic.
Which is phase equilibria.
So, recently you've been looking at chemical equilibria.
What happens when substances change their chemical structure and interchange, and you've figured out how to calculate where equilibrium lies in the case of chemical reactions.
Calculate equilibrium constants, and so forth.
Now we'll introduce phase equilibria.
So this is a little bit different.
You still have a lot of molecules in one state changing to another state, but they're not changing their own molecular structure.
They're changing what phase of matter they're in.
From liquid to solid to gas.
And so the treatment is a little bit different.
And there's some new results that are somewhat different from what you've seen in the case of chemical reactions.
So we'll start with phase equilibria in just one component system.
So in other words, we're going to start with a single substance and see how it goes from solid to liquid to gas.
And we'll from there see what happens in multiple component systems.
Where, for example, then we'll be able to talk about solubility and mixtures of liquids and so forth.
And into which phase different constituents in a mixture will go.
But let's start here.
So, let's say we just have two phases at equilibrium.
And this could just look like ice and water.
You could have a little bit of ice floating in water.
So in this case you have solid and liquid.
That could be in equilibrium.
And this is going to be at some pressure and temperature.
Or, you could have liquid and gas in equilibrium.
So let's put a lid on our container.
Still have some specified pressure and temperature.
And now there's a liquid at the bottom.
And a gas at the top.
So, you could have equilibrium between liquid and gas phases.
And in some cases you have equilibrium between solid and gas phases, depending on the pressure.
And and a little piece of solid could sit at the bottom and there's equilibrium with the gas above it.
And so, to start, what quantity are we going to need to look at to tell whether the different phases are in equilibrium, or if they're not in equilibrium, in which direction are things going to go?
Are things going to change so that the ice will melt and we'll have water or, so the water will freeze and we'll have ice.
What quantity is going to tell us about all that?
Who thinks they can guess?
Give you a hint.
We're specifying pressure and temperature.
Nobody really knows?
What quantity is it that you've seen from which you can determine everything.
Yes.
The Gibbs free energy is going to tell us everything.
And we're going to frame the discussion in terms of the chemical potential.
That is, the Gibbs free energy for a mole.
And since there's only one constituent, one component at this point, we just have n. We don't have different mole numbers for different constituents, we'll see that later.
OK, so what's going to happen is, the value of the chemical potential in the different phases is going to tell us everything.
Because you know that at equilibrium, the chemical potential needs to be the same in all of these.
If not, let's say the potential energy here is lower in the solid than in the gas, nothing's going to stop the molecules in the gas phase from going into the solid phase.
That's what they'll do.
If the chemical potential of the water is lower than the chemical potential of the ice, then the ice is going to melt.
Because the molecules are going to want to go toward lowest possible accessible chemical potential.
And there's the liquid right there.
They have access to it, so they can, the ice will melt.
And you'll have just water.
So the main point here is that at equilibrium, mu is identical everywhere.
By everywhere, I mean in all phases that are at equilibrium.
And so if multiple phases are present, mu has to be the same in all the phases.
And if it isn't, then the molecules will go into the phase with the lower chemical potential.
And this is what is going to guide everything in our consideration of phase equilibrium.
So, OK, what's going to happen?
Let's consider our ice water equilibrium.
So of course, if mu of the solid is equal to mu of the liquid, which for water it should happen at zero degrees Centigrade.
And pressure of one bar, for example, not the only place it could happen.
But certainly at that condition.
Then the ice and water will be in equilibrium, they'll co-exist.
Now, what happens if you just want some ice water and you take some ice out of the freezer and put it in the cup, and you put some water in?
And you see them both there.
They're sitting there.
Maybe for a while.
And yet it's not at zero degrees Centigrade, at one bar.
It may be at one bar, you might be out in the room, but it might be room temperature.
Does that tell us that this isn't true?
What's going to happen?
The ice is going to melt.
It may take a while.
This doesn't tell us about the kinetics of it.
And it could take a while, gradually, from the surface inward, for the ice to melt.
So the fact that you might see something sitting there with two phases present, that alone of course, doesn't tell you the two phases are in equilibrium.
And it may take time for equilibrium to occur.
And what's going to happen if at some pressure and temperature, you have the mu s greater than mu of the liquid.
Then which way are things going to go?
What's going to happen?
You've got to be able to figure this out.
Of course.
The ice is going to melt.
The chemical potential of the liquid is lower.
And so molecules are going to move toward that phase.
And, of course, if mu solid less than mu liquid, then the water freezes.
All this behavior is something that we can summarize in a phase diagram.
So the phase diagram can tell us for all pressures and temperatures, what phase, or what phases, will be present in equilibrium.
And so typically, phase diagram might look like this.
Where here's our solid.
Here's our liquid.
Here's our gas phase.
So let's just look at what's happening here.
If we go to really low temperature, in general, what happens?
What phase does stuff enter when you're at the lowest accessible temperatures?
Solid, right?
Everything eventually freezes.
In fact, really I should make this, leave some room on my axis for that effect.
So of course, low temperature you get solid.
How about high pressure, what usually happens?
You squeeze something real hard.
Tends to go into the solid phase.
Now let's warm stuff up.
And depending on the pressure, you might go to the liquid phase.
Certainly you'll go to the gas phase at high enough temperature.
And that's what'll happen.
And now, if you start changing the pressure, let's say you're at high temperature and you've got just the gas present, in some big container.
Now you start squeezing it.
You just apply more and more pressure to the gas at constant temperature.
What's going to happen?
Forget the phase diagram.
You know what's going to happen.
You keep squeezing, what will happen eventually?
This isn't going to be part of your grade.
You can speak up loud, and even if you're completely wrong, nothing bad is going to happen to you.
Yeah.
It's going to liquefy, of course.
You're going to start getting liquid to form.
It's essentially Le Chatelier's principle, of course.
The liquid occupies less volume.
The density of the liquid is much higher.
So as you increase the pressure, the stuff wants to go from the gas phase to the liquid phase.
Yeah
[INAUDIBLE]
Well, with water you know that at ordinary pressure, you know where the gas-liquid coexistence would occur.
So let's say you're there.
Let's say you're at the boiling point.
And now, you already in some sense know that that's a point that's somewhere on the gas-liquid curve.
So now what's going to happen?
Well, if you from there keep that temperature constant.
You're at the boiling point.
And you could easily do this.
You've got a pot that's covered, right?
And it's boiling.
So there's liquid down here and there's gas here.
So what'll happen?
As you start squeezing, literally what'll happen is you'll squeeze out all the room for the gas.
And eventually you'll have all liquid.
And in fact, you know what'll happen at that point, if you keep squeezing and you keep the heat on.
The liquid will get hotter than the boiling point.
Normally that doesn't happen, if you're just out at atmospheric pressure.
Because it'll boil.
And instead of getting hotter, molecules will simply leave the liquid and go into the gas phase.
So that's what'll happen.
If you increase the pressure, you'll liquefy all the gas.
And obviously the opposite will also occur.
If you just allow the volume to expand and keep expanding, maybe you don't have too much liquid there, you wouldn't have to expand it too much.
You'll run out, all the liquid will vaporize.
And you'll just have gas at 100 degrees C. You could heat it more if you wanted, but even there.
So in other words, changing the pressure if you think about actually executing that in the lab, or in this case even in your kitchen, you'll get the result that you should see on the phase diagram.
So let's just look at this.
So we've already talked about boiling.
That's what's happening along this line.
The line, and I'll explain the end of this in a while, all these lines are coexistence curves.
That is, these are where you have equilibrium between two phases.
They coexist in equilibrium.
and it could happen for water, for that case, for 100 degrees C and one bar.
But that's not the only place.
It could happen it a whole bunch of temperatures and pressure.
Because as you vary the pressure, you're almost surely familiar with this, the boiling point changes.
So the boiling point up in Denver at high elevation is not the same as the boiling point here in Massachusetts right at sea level.
Because the pressure's lower up in Denver.
Let's just look at the others.
So, of course, this is going from solid to liquid.
So we're looking at melting.
And then there is solid-gas equilibria in many cases.
And this is sublimation.
And this endpoint is called the critical point.
And there's one point where all three phases are in equilibrium, called the triple point.
So let's look at the various cases.
For example, let's look at the melting line.
It could be water, it could be water and ice.
Solid going to a liquid.
Let's just look at them one at a time.
What you have there is mu of the solid at some temperature and pressure.
Is equal to mu of the liquid at the same temperature and pressure.
Now, just if we back off and look at this essentially mathematically, there's an equation here.
And there are two free variables.
Temperature and pressure.
The equation puts a constraint on them.
So there are going to be a bunch of solutions along some line or some curve.
So in other words, it's one equation in two unknowns.
Temperature and pressure.
So the solution is a line or a curve.
In other words, we could solve for pressure as a function of temperature, or temperatures as a function of pressure, along the coexistence curve.
And we'll do that shortly.
Now, if we're in one of these regions, then there aren't any equalities.
There aren't any constraints.
Because we don't have multiple chemical potentials that are equal to each other.
Right here we have three phases in chemical equilibrium.
Solid, liquid and gas.
So let's just look at that triple point.
Mu solid equals mu liquid equals mu of the gas.
We have two unknowns, or two variables, temperature and pressure.
And we have two equations.
What that means is there's only one solution, and that's what the phase diagram shows.
There's a unique solution, the triple point temperature and the triple point pressure.
So this has only one solution.
If we summarize, how many degrees of freedom, or how many free variables, we have, anywhere here, we can do that in the following way.
We can write F is 3 minus p. This is the number of degrees of freedom.
Or independent variables.
And this is the number of phases in equilibrium.
So, what's it telling us?
If we have just one phase in equilibrium, or way over in the solid or just gas or just liquid part of the phase diagram.
This is one.
We have two independent variables.
Which is exactly what we know to be the case.
In other words, in those regions, temperature and pressure can vary freely without constraints.
If we go to any of the coexistence curves, now we have two phases in equilibrium.
And then only one variable can be changed freely.
If we want to stay on the coexistence curve, let's say we've got gas and liquid in equilibrium.
It's boiling.
We could change the pressure some, but if we want to keep them in equilibrium, if we've changed the pressure, we'd better change the temperature accordingly to stay on the coexistence curve.
Otherwise everything will go up into the liquid or into the gas phase.
Because you'll go off of the coexistence curve if you were to change one of these variables and not the other.
In other words, there's a constraint.
So only one can vary freely.
And if all three phases are in equilibrium at the triple point, then you have no degrees of freedom.
Nothing can vary and there's only one place where that happens.
Now let's see if we can understand also, why does the whole thing have the overall structure that it does.
Why are some of the slopes generally steeper than others.
Turns out, we can get a qualitative understanding of phase equilibria in a pretty simple way.
So in other words, can we understand how p and T change with respect to each other along any one of these coexistence curves?
Can we understand the qualitative features of the phase diagrams?
And what we'll see is, we can get an equation for dp/dT at coexistence.
That's, in a sense, our objective here in order to get the sort of qualitative picture.
So let's say we are somewhere here.
It could be the gas-solid part.
And we have some T. And now we go to T plus dT, and out here is pressure.
How much does the pressure change when we stay on the coexistence curve?
Well, let's go through this.
Let's just consider any two phases, alpha and beta.
They could be solid, liquid, gas.
And the main condition, if we're going to be on the coexistence curve, is that the chemical potentials have to be equal.
Well, that's fine.
Now let's change the conditions a little bit.
Staying on the coexistence curve.
So, let's start again.
mu s, or mu alpha of T and p is equal to mu beta at T and p. Now let's let temperature go to T plus dT, and pressure go to p plus dp.
But since we're staying on the coexistence curve still, at the new temperature and pressure, mu alpha and beta have to be equal to each other.
So mu alpha is going to go to mu alpha plus d mu alpha.
There'll be some change in it.
Mu beta is going to go to mu beta plus d mu beta.
But we already have that mu alpha and mu beta at the beginning were equal to each other.
And these have to be equal to each other.
Because we've specified that we're staying on the coexistence curve.
So what that says is d mu alpha equals d mu beta.
And now, remember in general d mu is dG per mole, so our fundamental equation for G, it's minus S per mole dT plus molar volume dp.
So this equality is telling us that minus S alpha dT plus V alpha dp is equal to minus S beta dT plus V beta dp.
So we're just applying the fundamental relation to the changes in each of the phases.
But since the changes in mu are equal, so are these changes.
And now I'm just going to rearrange these.
In other words, S beta minus S alpha dT is equal to V theta minus V alpha dp.
And that immediately is going to give us our derivative dp/dT.
So, dp/dT, staying the coexistence curve, is just S beta minus S alpha over V beta minus V alpha.
Which is to say it's just, we can say delta S over delta V. Going from alpha to beta.
So we have a result that can guide our thinking into what the slopes of those coexistence lines are going to be.
And it's pretty useful to look at that.
By the way, we can actually express this in another way if we want.
And in some cases it can be advantageous.
And that is, we know that of course G is H minus TS.
So our mu alpha equals mu beta condition, or a d mu alpha is d mu beta condition, can be applied that way.
That is, H alpha minus TS alpha equals H beta minus TS beta.
So we can do the same rearrangements.
H beta minus H alpha equals T S beta minus S alpha.
And what that's saying, in other words, is that we can substitute for delta S, delta H over T. So we can write our relation for dp/dT in two ways.
We can also write dp/dT at coexistence is equal to delta H over T delta V. Going from alpha to beta.
These are both forms of what's called the Clapeyron equation.
And these are always valid.
We haven't made any approximations.
Yet.
Later, in going to the gas phase, we'll make some approximations about having an ideal gas.
We'll be able to treat this delta V in a simple way using the ideal gas law.
But for now, these are exact.
OK, so now let's look at what it tells us.
Let's look at the different pieces of the phase diagram and see what we can conclude.
Essentially we should be able to construct qualitatively, the features of the phase diagram using this relation.
So let's try to do that.
Here's p and T. We're going to start with an empty slate.
And now let's look at the liquid gas case.
So, let's look at our delta S delta V, going from liquid to gas.
Well, one thing you know, when you go from liquid to gas, what happens to the molar volume?
It increases.
By a little, by a lot?
By a lot, right?
The molar volume in the gas is enormous compared to the molar volume in the liquid.
So delta V is positive and very big.
Now let's think about delta S. What's it going to do, going from liquid to gas?
It's going to increase?
The disorder increases.
Now, it increases significantly.
But the liquid already is a disordered phase.
And the molecules and the liquid already have considerable freedom.
Certainly not near as much as they have in the gas phase.
But still considerable.
So, to be sure, delta S positive.
And it's not small.
So there's your condition.
It's not nearly as big as delta V is going to turn out to be.
So what's going to happen here is, if we look at the slope going from liquid to gas, it's positive.
But it's not enormous.
It's not very steep.
So dp/dT, for liquid-gas coexistence right, greater than zero, but small.
Not steep slope.
So somewhere, we're going to put our liquid-gas curve.
And I, where exactly we can worry about it.
We haven't put any numbers on this scale.
But the point is, wherever it is, it's not going to be a very steep slope.
How about solid to gas?
Well, so what's delta V going from the solid to the gas?
What happens to the molar volume?
What happens to the molar volume, going from solid?
Of course, by a lot, by a little?
By a lot.
It's an enormous increase.
Even bigger than before.
How about delta S, the entropy change?
Yeah, it increases.
And also by a lot, because now you're going from an ordered crystalline solid, usually, so you have a very, very, low entropy there.
Into a disordered phase.
Not just the liquid, but the gas phase.
So this is also very big.
Well, ok, so now we've got delta S over delta V. There's going to be a steeper slope.
Because this is bigger then for liquid to gas.
So typically, the solid-liquid part will have a steeper slope then the liquid to gas part.
Now, wait.
Sorry, sorry, this isn't right.
And that's because I've really badly misplaced things here.
That was solid to gas that we just discussed.
It has the steepest slope relative to liquid-gas.
OK, now let's think about solid to liquid.
Delta V for going from solid to a liquid.
What is it?
What sign is it?
How big is it?
Let's forgot about the sign for a minute.
How much does the volume, molar volume, change going from solid to liquid?
Yeah, not a very lot, right?
You melt the solid.
There's still a little piece of stuff down there.
Didn't increase in volume enormously.
Usually, it increases.
So generally delta V is greater than, or sort of equal to, zero.
It's not a big amount.
How about entropy?
Going from solid to liquid?
It increases, by a good amount.
You're going from an ordered to a disordered phase.
So delta S is greater than zero.
And so the result here is, now, if you look at dS/dV, or delta S over delta V, delta S is positive and you a pretty good amount.
Delta V is small.
The molar volume hardly changed.
And so what that's going to tell us is the slope is going to be steep.
And generally positive.
So now, we're going to really be on the increase.
Typical kind of phase diagram and individual coexistence curve.
And all these things, again, you could understand in terms of Le Chatelier's principle.
In terms of going from, say, solid to liquid if you know think about two pressures.
Go from p1 to p2, some higher pressure.
Where are you?
Well, if you want to stay on the coexistence curve you need to raise the temperature quite a bit in order to do that.
Because, basically, to keep on there, you know what's going to happen if you have, say, solid and liquid.
It's ice and water.
You know the freezing point, it does depend on the pressure, of course.
But since the change in the volume isn't that big, what happens is, a modest change, if you change the temperature by a little bit, maybe easier to think about it that way, you've got to change the pressure by a lot to stay on the coexistence curve.
Otherwise it'll have a great tendency to simply go into one phase or the other.
Now we've, in a sense, reconstructed the coexistence curve by our consideration of the Clapeyron equation.
There's another way we can usefully understand the qualitative features of the phase diagram.
And that's just by looking at mu itself.
Looking at the free energy itself, at different temperature.
So let's just do that.
Let's just consider mu of T at some fixed pressure.
Just qualitatively, what will happen?
So let's go back to d mu is minus S dT plus V dp.
And just look at the derivatives.
So d mu / dT at constant pressure is minus s. Minus the molar volume.
This is the one we're really going to look at.
Of course, we also could look at d mu / dp at constant temperature, and just the molar volume.
Let's just look at this.
We know that entropy is always a positive number, right?
Entropy at the lowest, in a perfect crystal, at zero Kelvin, could be what value?
Entropy of a perfect pure crystal at zero degrees Kelvin.
Zero, right?
And it's only going up from there.
So entropy is always positive.
So what that tells us is that, then, if we just look at mu versus T, it's negative S, for whatever phase.
So it's always negative.
So in some sense, you say why?
Well, it's because we're keeping pressure constant and we're raising the temperature and essentially the effect of the entropy is more important.
Is going up.
You could see it just by saying well, G is H minus TS.
And as you raise temperature, the product TS is increasing.
Which means the G is decreasing, because it's H minus TS.
So it's always a negative slope.
Let's look at it in the different phases.
So we have entropy of the gas, that's greater than the entropy of the liquid.
Which is greater than the entropy of the solid.
So that immediately tells us the relative slopes for the three phases.
So d mu gas / dT at constant pressure is minus S gas.
d mu liquid / dT is minus S of the liquid.
d mu solid / dT is minus S of the solid.
And the magnitude is much bigger in the gas than the liquid than the solid.
And so the same goes for the slope.
So now let's sketch this.
There's mu.
And there's temperature.
So what we've seen is for the gas, first of all, the slope's always negative.
And it's steepest by far, or steepest substantially, for the gas.
Next steepest for the liquid.
And least steep of all for the solid.
Now, there can be variation.
Because, of course, these do change as a function of pressure.
But this, certainly the slope, the magnitudes of the slopes are always going to be like this.
What could change as a function of pressure is where these happen to lie.
But this is certainly typical.
So what does it mean?
So, it means, remember, the stuff is always going to be in the phase with the lowest value of mu.
So at low temperature, down here, that's where the solid phase has the lowest chemical potential.
Now, let's keep going up.
Eventually they cross.
This is where, for this particular pressure, this is where the liquid and the solid are in equilibrium.
In other words, we've started, we've got the solid and we're raising the pressure.
Now it's going to melt.
Here's our melting point at this particular pressure.
And now we've got, if we keep raising the temperature, we're no longer in equilibrium, and we'll be in the one phase part of the phase diagram.
It's just the liquid.
Because that's the lowest chemical potential.
Let's keep raising the temperature.
Down here, of course, the chemical potential of the gas is way up there, right?
So it's not going to come into play now, though they're going to be in equilibrium.
Here is the boiling point.
So now let's keep raising the temperature.
And we'll be in just the gas part of the curve.
And of course that'll go on forever.
So that can just help us understand, in addition to the slopes, the positions, the relative positions, of where these equilibria occur.
Now if we start changing the pressure in particular, we could move this.
So that you could have a sublimation point where the gas and the solid are in equilibrium because this has moved over to here.
Any questions about the overall structure of the phase diagrams and these phase equilibria?
OK, let me just end with a few comments about one part of the phase diagram that we haven't talked about yet.
This.
What's going on there?
If we were to change, of course, I've got a finite amount of space on the blackboard, so I haven't drawn T and p out to infinity.
But if I could, this line would keep going forever.
There would always be some pressure and temperature at which you would have equilibrium between two different, distinct solid and liquid phases.
Turns out, that's not true for the gas and the liquid.
And there are a bunch of ways to see this.
But one is you could think of the symmetry difference.
When you go from a crystal, a crystal and solid, to a liquid or a gas, there's no question that you've changed phase.
Because there's a symmetry in the solid that's no longer maintained in the isotropic liquid or gas.
So clearly there's never going to be a point where these two phases become the same thing.
Become somehow indistinguishable.
But the gas and the liquid, I don't know, what's the difference between them?
The density, right?
The gas is more dense.
It's condensed.
Sorry, the liquid is condensed.
The gas isn't.
But that starts to change when you adjust the pressure and temperature accordingly.
And you can sort of see it coming.
What'll happen is, you change the temperature, the molar volumes of the liquid and the gas start to get equal to each other.
That's what'll end up happening.
And at that point, what's the difference between them?
There is no difference.
There's no symmetry difference.
So you can actually, instead of going through a phase transition, where there's a meniscus, you can see the top of the liquid when you heat it up.
There's boiling and then there's the gas.
You can actually go from the gas, let's go the other way, from the liquid to the gas phase, without ever boiling.
If you change the pressure as well.
So that you can actually walk all the way around the critical point.
What happens, is up here you don't have two distinguishable phases any more.
And actually, material out at this region has a bunch of very unusual behaviors.
Very useful behaviors, actually.
Chemical behavior can be really very, very, different in what's called supercritical, the supercritical region beyond the critical point.
Water behaves really unusually there.
Or it turns out, in that region, it turns out to be a terrific solvent for organic stuff.
Organic molecules which normally don't dissolve well in water.
And it doesn't dissolve salts very well.
That's pretty unusual.
Carbon dioxide, CO2, turns out to be a pretty good solvent for organics in the supercritical region.
That's what dry cleaning is.
Very nice, because know CO2, apart from its slow effect on our atmosphere, it's not poisonous.
It's not toxic.
So you can do dry cleaning without organic solvents.
All sorts of unusual behavior happens above the critical point.
Beyond the critical point.
There's also a whole range of behavior at the critical point.
Right at the critical point that's extremely unusual.
That you could think of as a consequence of the fact that the molar volumes are getting to be almost equal to each other.
That's unusual in a lot of ways.
And what ends up happening is, you can see if you go online, actually, if you just Google critical point, you can see amazing movies of stuff that's at the critical point.
Because what happens is, so you've got the liquid.
And you've got a meniscus, and you've got the gas.
And you can tell.
You look at it.
There's the liquid.
There's the gas.
But if you go real close to the critical point, the stuff doesn't really know what phase it's in.
Because it's losing the distinction between the two phases.
You start seeing globules of liquid up there in the gas phase.
And globules of the gas down in the liquid.
Because their densities - I mean, the only reason the liquid's at the bottom normally is because the density is higher.
If that stops being the case, it has no more right to occupy the bottom part than the gas.
And so they sort of start interchanging.
And if you get even closer, of course, what eventually happens is, there's just one phase.
You don't see distinct meniscus's any more.
But right near there, you see enormous fluctuations in the local density and in the location.
The physical locations of different phases.
And we'll see that come in another context later.
Because when we talk about two phase equilibria solutions.
You get this with you oil and water, you get critical mixtures also, where right at the special point, the meniscus goes away and the oil and water start interchanging.
And in those situations, a tiny amount of force can give you a really big response.
Because the two things are just teetering in the balance.
So a little bit of force pushes things one way or the other.
That's actually useful in a whole bunch of different cases.
Because that critical behavior happens in all kinds of phase transitions.
Magnetic phase transitions.
Ferroelectric phase transitions.
And in those situations, a tiny little magnetic field, for example, gives you a really big change in the magnetization.
If the alignment of spins.
And that's very useful.
You can get a very big material response and change.
You can switch something, for example, with a really small external field.
And in these cases, you can change the density really a lot by only a small change in pressure.
OK, next time we'll extend the Clapeyron equation and also go to two component systems.
So you actually, Professor Blendi will take over again on Friday.
And we'll see you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's do a one minute review, and then move onto the Clausius-Clapeyron equation and see how far we can go on that.
So, reminding you then, what you learned last time about phase transitions is, you drew a phase diagram in temperature pressure space here.
And then you had a triple point somewhere, which was a unique point.
The triple point temperature, triple point pressure.
Then there was a gas, solid, call it coexistence line.
Which kept on going ad infinitum.
And the a gas liquid coexistent line.
With a critical point.
So that allowed you to go around.
You wanted to go from liquid to gas, you could actually go around this critical point, and never actually see a phase transition.
Then there was a solid liquid coexistence line.
Which usually has a negative slope, except for the two most important substances on Earth, which are water and silicon.
So for water and silicon, we have positive slope.
H2O and silicon.
This is why there's life on Earth.
Did you hear the story of why there's life on Earth?
This is the secret of life.
Without this, this is why we're here.
Because of this property.
So, the reason why these slopes come from the, so I'll tell you the secret of life.
But first let me remind you what the coexistence curve is.
dp/dT.
Coexistence is delta S over delta V, or delta H over T delta V. And that's the Clapeyron equation.
Now, the slope of the curves points us by, look at this solid liquid line, delta S, for solid liquid.
I looked at it up here.
It's going to be S liquid minus S solid.
And you know that's greater than zero because the entropy in the liquid state is much bigger than it is in the solid state.
You have a crystal, entropy's very small.
Liquid state, much more disorder.
Got a positive sign here.
Yes.
Questions?
[INAUDIBLE]
So, this is almost everything.
It has this slope here.
Which is a negative slope.
And water has a positive slope.
Right?
Is that wrong?
It's wrong.
Let's go through the argument.
All right.
All right.
Let's go through the argument.
So, then you have delta V, and delta V is V solid minus V liquid, this is per mole.
Now, almost every substance has V solid less than V liquid.
So this is negative.
So almost every substance, you have delta S divided by delta V is negative.
A negative slope.
You're right.
So I have it backwards.
I do have it backwards.
OK. See, I was really trying to make it so that my drawing was right.
Which is why I inverted the solid and liquid here.
So I really wanted to be right, but I ended up being wrong.
Liquid is less than solid.
Well this is water.
Why do I have it backwards in my notes.
I've got to correct that.
So this is most substances, is positive.
OK so, and most substances, OK?
Except for water and silicon.
Because for water and silicon this is reversed.
Because the molar volume of solid for water is bigger than that for liquid.
So this is bigger than here.
Thank you.
OK.
Thanks for catching it.
OK, so what happens if the molar volume for the liquid is bigger than the molar volume of the solid.
The density of water which is the inverse of the mole volume, so the density, the molar density, is equal to one over the molar volume.
So you've got the molar density for liquid is then smaller than the molar density for the solid.
It's one over, right?
So that's why ice floats.
Because it's less dense than liquid water.
So what do you think would happen in the winter if ice didn't float, in the Charles River.
It would go to the bottom.
It would freeze, solid and you would have no fish.
You would have no life.
It would freeze solid and that would be the end of life.
And you know, five hundred million years ago there would be no fish.
There would be no dinosaurs.
There would be no anything, there would be no humans.
That's the secret of life.
And the fact that this is also the case is very useful for making silicon.
For processing silicon.
Another the secret of life.
Where would we be without silicon?
Our civilization would be, we'd still be using stones and things, right?
OK, so it's super super important.
Alright, so this is a Clapeyron story.
Which we ended up getting right.
Even though I insisted on getting it wrong.
So what happens now is Mr. Clausius came around and he realized that you can make some approximations that are very useful.
So the first approximation is, you can realize that the molar volume of the gas is always much bigger than the volume of the solid or the liquid.
And so as a result whenever you have these delta V's, for sublimation, or delta V for vaporization, which is the volume the gas minus the volume of the solid.
Or the volume of the gas minus the volume of the liquid, you might as well ignore the volume of the liquid or of the solid.
And this can just be equal to roughly the volume of the gas.
So this is the first approximation he made.
So now, if you go back to the to the Clapeyron equation up here, with this approximation then dp/dT, the coexistence line, is delta H divided by T V gas.
For sublimation.
Or vaporization.
You don't have the delta V down there any more.
Then the next thing that, next assumption that he realized you can make was, well, all these gases could, they're like ideal gases.
So we can, instead of having the volume of the gas here, we can use the ideal gas law.
This is all, let's do all this per mole.
So V is equal to RT over p. You can plug this back in here.
And then you end up with the Clausius-Clapeyron equation.
So this is approximation now.
We put in RT over p for here.
We have p delta H, where this is either sublimation or vaporization divided by RT squared.
dp/dT, sublimation or vaporization, and this is the Clausius-Clapeyron equation.
Two important approximations that go in here.
And it's not valid for solids or liquid.
You have to have a gas in there because of the ideal gas for the approximation that goes in here.
And once you have that, you can integrate both sides.
You've got to put the temperatures on one side.
Put the pressures on the other side.
I'll go ahead and cover this up here.
So you have one over p dp/dT is equal to delta H over RT squared.
One over p dp/dT that's just like d log p. And that's another form that you're going to see often for the Clausius-Clapeyron approximation is d log p / dT is equal to delta H over RT squared.
And I'm freely dropping the sublimation and the vaporization because we know that's what I mean here.
It's only valid for those two lines.
So that's another form that you'll see.
And that's not dT here, that's d log p, d log p dp.
Yeah, that's right, d log p / dT.
And so now you can take the dT to the other side and integrate.
From point one to point two d log p, is from point one to point two, delta H over RT squared dT.
And you make the usual approximation, which we've made before.
That delta H is roughly independent of temperature, or very slowly changing with temperature.
And that's fine as long as you're in a narrow temperature range.
Which is usually the case when you have these Clausius-Clapeyron equation problems.
So you can take the delta H out of the integral, the R out of the integral.
And you can integrate both sides.
To give you log p2 minus log p1, is delta H times R, divided by R, delta H divided by R. Delta H over R times T2 minus T1 over T1 T2.
It looks a lot like and the temperature dependence equilibrium constant.
And in fact, but it's not, right?
And some people use that equation here instead of equilibrium constant.
Temperature dependence.
That's, can give rise to problems.
Because obviously they're not the same equation.
But roughly has the same form.
So, another way that you'll also find this written is very often p1 and T1 will be some reference.
Pressures and temperature, maybe one bar, let's say.
T1, 298 degrees Kelvin.
That's your reference point and you want to find out the pressure temperature dependence in an equation.
So if you rearrange your equations so p1, T1 are now some reference point.
They become numbers.
So you have log of a reference point, T1 here is a number.
So you often see this equation rewritten then as log p is equal to delta H over RT.
And I feel like I've got a minus sign missing somewhere.
Here.
Oh yeah, when you integrate this, the one over T squared, there's a minus sign.
Oh, I took care of that by doing T2 minus T1 here.
I think there's still a minus sign problem somewhere.
I think there's a minus sign problem.
Let me check up the notes here.
Minus one over T1 minus T2 over T1 T2.
Where's my minus sign?
T2 minus T1.
OK, so I got minus T1, minus T2 over T1 T2.
T2 minus T1.
This is fine, right.
There's no minus sign problem.
Plus a constant.
OK, so you'll often see it written like that.
And that gives you a relationship between the pressure and the temperature then, for a substance where all the reference point information is contained in this constant, C. Which is where the T1's and the p's come from.
OK, any questions?
We'll do a little example here of how this could be used.
So let's switch to an example now.
Let's go to the example, let's first do the example which is at the very end of the notes.
So this is an example of RDX.
RDX is a famous explosive, as I'm sure you know.
And it's plastic explosive.
It's got a melting point of 204 degrees Celsius.
481 degrees Kelvin.
And it's a problem for, so it's a very powerful explosive and you can't see it in X-rays.
So it's a problem at airports.
You've got be able to find if somebody's carrying a little piece of RDX in their luggage.
And so, the question becomes what do you need to know to detect it?
You want to find the vapor pressure.
You've gotta have, and the machine which basically is sensitive enough to detect molecules of RDX that are in vapor at room temperature.
So the way that you do that is, you do an experiment.
Where you measure the pressure, the vapor pressure, as a function of temperature.
And room temperature, it turns out if it's not volatile at all, if the vapor pressure is very, really tiny.
And so in order to get accurate numbers, you actually do your measurements at significantly higher temperatures than room temperature.
They do it close to the melting point, so you're actually looking at the sublimation of RDX.
You do it, let's say, at 400 degrees Kelvin, where the melting point is at 481 degrees Kelvin.
So slightly below the melting point you're looking at sublimation.
And you plot.
Log p versus one over T. And according to Clausius-Clapeyron, that should give you a straight line.
And in fact, that's exactly what you see when you take RDX and do that experiment.
You end up with a bunch of data points.
That's fall nicely on this straight line.
And then you can extrapolate to room temperature.
Somewhere here, let's say, 300 degrees Kelvin.
And find out what log p is, at 300 degrees Kelvin.
And what you find is that, and that's the graph that's on the last page of the lecture notes, that the vapor pressure of RDX at room temperature is 10 to the minus 11 bar, That's ten parts per trillion.
So it tells you that if you want to detect RDX with a sniffer machine at the airport, that machine better be able to tell you, find one molecule of RDX out of a tenth of a trillion other molecules, basically.
Which is a really hard thing to do.
So this gives you a design rule for these pieces of equipment.
And why they're so expensive.
OK, before we do the next example, let's see if any questions on Clausius-Clapeyron.
Yeah
[INAUDIBLE]
So, p as some exponential here?
You know, I don't recall seeing it as an exponential form.
But that doesn't mean it's not used the exponential form.
This is an easy way to do it, because then you've got a linear relationship.
And generally we like to see straight lines.
So this gives you a nice straight line.
OK, so now let's do a slightly more complicated example.
Which is, so most of the time you don't have pure material.
You have a mixture of some sort.
For instance, if I have a glass of water on the table at room temperature.
Above the glass of water, I have air, there's my H2O here.
And then I've got some air.
Above the glass of water.
And the air's inert to the water.
It's not reacting with the water.
So it's like an inert gas sitting on top of the water.
Then I want to ask the question, what is the vapor pressure of the water?
In the presence of this inert gas, the air.
It's not really the same as the problem that we've been looking at up here.
Right?
Because this gas liquid coexistence line, this diagram, and this Clapeyron equation, is all done for a pure substance.
And at room temperature, at room temperature, we're sitting squarely in the liquid phase.
We're not on the coexistence line.
At room temperature and one bar pressure.
So instead of having this system here, I looked at the system where I had the water, H2O, with nothing on top.
With no air on top.
Except the cylinder.
Now, I'd better make the surface of my water a straight line, otherwise I'm going to get in trouble.
There's the water sitting right here.
And I put a cylinder with one bar pressure on top.
My cylinder's going to sit squarely on the stump of the surface of the water.
This is not going to be any water vapor at one bar pressure on that cylinder.
At one bar, we're way up in the liquid phase.
I would have to decrease the pressure on that cylinder down to, I don't know, whatever the water vapor pressure is at room temperature here on this coexistence curve.
0.1 bar or something.
0.05 bar.
Something pretty small.
So that's what we've been working on.
This pure system.
And now we're going to be working on this system here, where we have air.
With a cylinder on top, one bar.
And we want to know what is the vapor pressure of H2O in this system here.
Is it zero?
Anybody want to guess, if it's zero?
We know there's vapor pressure there, right?
And a good guess is that that vapor pressure here, it's pretty close to the pressure that you would guess by using this diagram with the pure water.
But the key word is, it's pretty close.
It's not exactly the same.
So, let me ask you this.
Without looking at the notes, now is p H2O in this problem here greater than, less than, or the same as p H2O in the pure Clapeyron case?
How many people think that it's the same?
How about smaller than?
Smaller than?
Few people think that it's smaller than.
How about greater than?
A bunch of people don't know what to answer.
OK.
I think that if you think about it, what the difference is between here and there, you could probably reason which way it should go.
Which way the partial pressure of the water should go under one bar pressure, room temperature.
Compared to what you might expect it to be at room temperature with a diagram.
So, I'm going to give you a little bit of time just to think about it.
Without looking at the notes, what you think the right sign is here.
Then we'll do the problem and then we'll figure out if you were right or wrong.
OK. How many people say it's greater than?
One, two.
Two brave people here.
Two.
How many people say it's smaller than?
OK.
Smaller than.
So I'm going to say a large number, OK?
Many.
How many people think it's equal?
One person thinks it's equal.
Alright, let's do it out now and see how it comes out.
And then we can reason why it came out the way it came out.
Logically, without actually doing math.
Because there's a way to reason it out that you should be able to do.
OK, so this is what we have here.
Then the system is, we have this container.
Let me redo it instead of writing air.
So we have the liquid state here.
Some substance, and it doesn't have to be water.
That's a liquid, at some pressure p. And I'm going to say P capital, that's going to be the total pressure.
And the total pressure is given by the piston here.
p total, p T. And on top here I'm going to have two substances.
I'm going to have A, in the gas phase, that's the water vapor.
Some partial pressure, p sub A, temperature T. And I'm going to have the inert gas, with some partial pressure p, inert.
Such that the sum of the partial pressure of the inert gas, plus the partial pressure of A, is the total pressure.
One bar, in our case.
With the water with the air on top.
And the question, oh, let me make a few more definitions.
We're going to define p0 as the vapor pressure of pure vapor pressure of pure A.
So that's what you would read off the diagram here.
And what we want, the question we ask is, what is the partial pressure of A as a function of the total pressure?
That's what we're asking.
As I'm increasing the total pressure, meaning I'm putting more and more inert gas in there, which way is this going?
Is it going up, down, or staying the same?
And we've got a bunch of votes up here.
Most people think that it's going to go down.
Because I put more gas in there.
OK, I know that if P, capital P, is equal to pA, then pA is equal to p0.
Right?
There's no inert gas.
So that's the one extreme case here.
There's no inert gas, then the partial pressure's going to be what you read off that graph here.
So another way of saying what is p as a function of capital P, is to ask the question what is the slope?
I've got one point here.
If I can integrate the slope then I have this equation.
So the other way to ask the question is, what is dpA/dp, at a constant temperature?
Where I'm keeping the temperature constant.
And that turns out to be easier to calculate.
These slopes are always easier to calculate than the absolute things.
So we're doing an equilibrium here.
We have an equilibrium between the liquid state and the gas phase.
And we want to know something about this equilibrium as we change a parameter.
Equilibrium, what do we always start with when we have an equilibrium.
What's the first thing we think about?
We think about chemical potential.
Chemical potentials of A, in the gas phase has to be the same as the chemical potential of A in the liquid phase.
Well, I can't think of anything else to start with.
So let me start with that.
So let's start by writing the chemical potentials being equal to each other.
Because we know that's true.
So mu A, in the gas phase, temperature and with a partial pressure p sub A.
Is equal to mu A in the liquid phase, where the pressure on the liquid is capital P. What else do we know?
Well, we, what else are we going to need?
Before we do that, since what we're after is the derivative, let's take the derivative of both sides.
We're looking at a derivative with respect to pressure here.
Let's take the derivative outside with pressure and see what happens.
So let's take the derivative of this with respect to total pressure.
Well, this isn't total pressure.
This is the partial pressure of A.
But it's a function of the total pressure.
It's a function of the total pressure.
Partial pressure of A is total pressure minus p inert.
So, we have to use the chain rule.
So on this side here, we have d mu A / dpA.
So I'm taking d/dp of both sides.
Then by chain rule, dpA/dP.
Constant temperature.
And then on the right-hand side, I have d mu A / dP, constant temperature.
Well, d mu / dP, that may or may not seem familiar to you.
I never remember these things, but I always remember that there's, that these relationships are interesting.
Especially when you're talking about something like a chemical potential, which is really nothing but the Gibbs free energy.
With respect to pressure, when we know that the variables that we use for Gibbs free energy are pressure and temperature.
Right?
So this is the variable that goes with Gibbs free energy.
So this means that they were taking the derivatives of the Gibbs free energy with respect to one of its variables is something that we should know.
So we go back to writing what the Gibbs free energy is.
So dG is equal to V dp minus S dT.
Which is the fundamental equation for Gibbs free energy.
And this is the Gibbs free energy per mole.
That's just the chemical potential.
The same thing.
And this is the derivative with respect to pressure.
And this is the derivative with respect to temperature.
So this thing here is d mu / dp.
At constant temperature.
It's just the volume.
So now we have this equation, where we started out with the chemical potentials.
And we've got these d mu A / dpA, we've got d mu A / dP here.
That's just the molar volume of the gas.
Right, because this is mu of the gas phase here.
So this is the molar volume of the gas.
And I have dpa/dP.
And, this is very nice.
Because this is actually what I'm trying to get.
Trying to see how the partial pressure of a changes with the total pressure.
That's what I'm trying to calculate here.
I know what this is.
This is just an experimental number.
And then on this side here I have d mu A / dP.
This is the liquid phase.
Derivative with respect to total pressure, that's the molar volume of the liquid.
The liquid.
Great, I'm done.
dpA/dP total, constant pressure, is the ratio of the molar volume of the liquid of A divided by the molar volume of the gas.
What's the sign?
Volumes are positive, right?
This has to be positive.
There's no choice.
It can't even be zero.
It has to be positive, right?
So that means the slope is a positive slope.
As I increase the total pressure, p is going to go up.
If I integrate now, this, starting at, so the total pressure is equal to pA, I'm going to get a curve that looks like this.
OK, if I write on this axis p inert, so when p inert is equal to zero, total pressure is pA.
I'm going to start at p0.
And I'm going to go up from there, positive slope.
So.
Who was right?
A couple people are right.
So let's think about it in a different way.
Let's think about it in a different way.
So we did the math, and we found that actually it's, and that turns out to be a very small amount.
You do a very good job by just taking this curve here.
And using the pure substance.
In fact, in the notes we have an example of how much it changes for mercury.
So for mercury, at 100 degrees C, for pure mercury, the vapor pressure is 0.27 bar.
If you add enough inert gas to make it 1 bar for the total pressure, you go from 0.27 to 0.2701.
Not much of a change.
They do a really good job of just picking this.
And if you go to 100 bar, then you increase it to 0.28.
So it takes a lot of pressure on top, a lot of extra inert gas to really make much of a difference.
But, let's think about why it would make a difference, just logically.
So in the absence of this inert gas here, I just have the pure substance.
I add some inert gas, what happens to the entropy on top?
It's increasing, right?
Do systems like to have more entropy?
Yeah.
So, I have a lot of inert gas here.
A small amount of A.
What if I had a little bit more A, in the presence of the inert gas?
Which way would the entropy go?
It would go up a little bit more, you'd have more disorder.
If I have nothing, if I have no A here, up here and just the inert gas, then there's no entropy of mixing.
I add a little bit of inert gas, there's a little bit more entropy of mixing.
Eventually, I add more and more, entropy of mixing goes up and up and up, and eventually it comes back down to where I have pure A on top.
So the entropy of mixing up here is driving the increase in the pressure of, in the partial pressure of A. Entropy is this amazingly important term.
That's what we saw.
Entropy of mixing was responsible for equilibrium.
Without entropy of mixing, then everything would go directly to the products.
You'd never have an equilibrium.
Another secret of life.
So entropy of mixing here is driving this positive slope.
That's another way of thinking about it.
And that's why you can guess ahead of time what the signs of these things ought to be.
Without doing the math.
Alright, any questions?
OK.
So there's another sample problem with the notes, which is sort of like a standard problem.
Where you're given a Clausius-Clapeyron equation for a substance in this form.
So if you look at it, log p is minus some energy divided by RT plus some number.
In this case, it's negative, the delta H is negative for the substance that you have there.
And you're given it for the liquid phase, for the vapor pressure above the liquid and the vapor pressure above the solid.
You can, because the only way it doesn't work is if you're looking at solid liquid.
So, and then you're asked to manipulate this equation to find different properties.
For instance the triple point.
It's a very common thing to get, the triple point from this.
The delta H, it's a way to get delta H. Just like we saw for the RDX, right?
The data points formed a straight line.
And the slope gave you delta H, for RDX.
Delta H of sublimation for RDX.
So this is the way that this equation is used for problems.
I'm not going to do this problem.
Instead I'm going to go ahead and forge ahead to the next topic.
And start it.
This is the topic we're going to do it after spring break.
And and it really is a topic that follows this pretty straightforwardly.
Well, it's more complicated, but it's similar to this problem here.
So we saw that in real life yes, question?
[INAUDIBLE]
The exam.
Oh there's an exam is, which is, oh I forgot about the exam.
So let's see.
There's a problem set that's due today.
The exam will be up to the problem set that's due today.
Which means that what I talked about today is not going to be on the exam.
Because it's not covered in the problem set.
So let's say it goes up to Wednesday's lecture.
Which means you might still have a phase transition problem, right?
In fact, you're very likely to get a phase transition problem.
It's very likely to be the case.
Because it's an easy problem to put together.
But you don't have to worry about Clausius-Clapeyron.
You do have to worry about Clapeyron, but not Clausius-Clapeyron, OK?
So, for the exam, we're going to put on the Web last year's exam, and you guys are going to do reviews.
The usual thing.
Yeah?
[INAUDIBLE]
So that you don't see the answers.
Yeah, absolutely this can easily be done.
We'll do that.
Any other questions, about the exam?
Now, I don't remember.
Is there a problem set that, so I'm not assigning a problem set today, right, then?
Right.
OK. Good.
Because I hadn't loaded it up yet.
And so now I won't load it up.
OK.
Anything else?
OK, so let me tell you where we're going to go then, after the break.
So after the break, we're going to be starting to talk about things that are called colligative properties.
Colligative properties are the properties like the vapor pressure, lowering when you have a mixture in a liquid, instead of having a pure substance you have a mixture of substances here.
And then the vapor pressure of both substances above gets lowered.
Anybody want to guess why they got lowered?
What's the magic word?
Entropy of mixing, right.
Right.
There's more entropy of mixing if you've got a mixture in the liquid than if you have a pure gas up there.
So usually, so let me back up.
Usually it's when you have a solute, like salt in water.
Where water is volatile and the salt is not.
Right, a salt in water solution will have a lower vapor pressure than a solution of pure water.
Because of the entropy of mixing in a salt and water solution.
The salt and water solution we have a lowering, lower melting point.
And that's significantly lower.
Which is why we use it on roads, right?
That's a colligative properties.
Osmotic pressure.
Which you've already sort of heard about.
Why saltwater fish are unhappy in fresh water.
That's a colligative property.
So we're going to go through these colligative properties.
And the mainstay of these colligative properties is that we're going to be talking about mixtures in the liquid phase.
Here we talked about a mixture in the gas phase, changing some property.
From now on, we're going to be talking about mixtures in the liquid phase, pretty much.
And so our goal is going to be able to, is going to be predict the way that properties change as you have these mixtures.
OK. so the standard mixture that we're going to have, is to have a binary liquid gas mixture.
So that means we're going to have two substances.
A and B.
And at first, we're going to make it as general as possible.
So we're going to have liquid phase of A, the liquid phase of B.
So, for instance, this could be vodka, right?
Water and ethanol.
And water and ethanol are both volatile.
So we're going to have water and ethanol in the gas phase.
There, and it's a mixture, they're perfectly miscible in the liquid phase, A and B.
And there's going to be some fraction, molar fraction, of these things.
We're going to have a molar fraction in the liquid phase for A and B.
And some molar fraction in the gas phase, for A and B.
We're going to call it yA and yB, that's the molar fraction in the gas phase.
And there's no reason why the molar fractions in the gas phrase should be the same as that in the liquid phase.
And we know that.
We know that ethanol is more volatile than water.
And so if you have a gas of vodka, then you can smell the alcohol because it's coming out.
But the water is, vapor pressure is, very slow, very small.
It's the usual, it's pretty close to the usual vapor pressure.
And our question is, what we're going to be asking is, how many, if I give you the total pressure and the temperature here, what else do I need to know to find out all these molar fractions?
So let's first see what all the variables are that we have.
Usually, we have two variables.
The temperature and pressure, and that's enough to tell us everything.
But here now we have a mixture.
So we're going to need more than that.
We're going to need four variables.
We're going to need the temperature.
The pressure, and we're going to need the molar fraction of A in the liquid phase.
And the molar fraction of A in the gas phase.
We're not going to need any more than that.
Actually we're going to need less than that.
But those are the four variables that we can easily identify here.
And we don't need yB and xB, because the sum of the molar fraction has to be equal to one.
yB is one minus yA, xB is one minus xA.
So we don't need y.
So these are a priori the independent variables that we have to work with.
Now, we have some constraints here.
We have a constraint.
Our constraint is that we are the coexistence point.
There's a coexistence between the gas phase and the liquid phase.
And what does it mean for A liquid to be coexistent with A in the gas phase?
Chemical potential.
The chemical potential of a molecule of A in the liquid phase here is the same as the chemical potential of A in the gas phase.
So we have two constraints.
We have mu A for the liquid phase is equal to mu A for the gas phase, and mu B for the liquid phase equal to mu B for the gas phase.
So we have four variables, two constraints.
That means that the total number of independent variables we have is just two.
It's the number of variables four, minus the constraints.
OK?
Or you could say six variables if you include yB and xB, and four constraints.
Whatever we accounted, you just have two degrees of freedom.
So we have two degrees of freedom.
Which means that if you know the temperature and pressure, then you still know everything.
At the coexistence point.
You still know what the molar fraction is of your two substances in both phases.
So that's really powerful.
Or if you know just the molar fraction of A, and the temperature, then you know the pressure above.
And you know everything else.
So in this very complicated mixture here, you still only need to know a couple of things to tell you everything about this mixture.
And this example here is a specific example of something called the Gibbs phase rule.
Which you've seen before.
In this diagram.
In this diagram that we wrote here.
Where, let me write down what the Gibbs phase rule is first.
The Gibbs phase rule tells you the number of independent variables given a number of constraints.
So, it tells you that the number of degrees of freedom, the number of independent variables in a mixture, in a complicated system like this, is equal to C, which is the number of components.
So in this case, it would be two components.
You have A and B.
The components is the number of independent species that you have, minus the number of phases.
So here we have two phases, the gas phase and the water phase, plus two.
So for this example here, then we would have two components, C equals two.
Two phases liquid and gas.
A and B are the components.
And then plus two.
So two minus two plus two, F equals to two.
Two degrees of freedom, two independent variables.
If I'm looking at this diagram here, in this diagram here I have C equals one.
One component, right?
If I'm sitting where there's only one phase, say, the liquid phase or the gas phase, then p equals one.
Then F is one minus one plus two, is equal to two.
Two degrees of freedom.
And so in the pure phase, I am in a plane.
The pressure and the temperature are my two degrees of freedom.
I can freely move around the plane.
At a coexistence line, then the number of phases equals to two.
Then I have one minus two is minus one, plus two is one.
The degrees of freedom is one.
That's a line.
I have one degree of freedom.
I can move in the line, in the line in a plane.
Then if I change to the triple point, where the number of phases is three, then the number of degrees of freedom is one minus three is minus two, two plus two is zero.
I don't have any degrees of freedom, I'm stuck at one point.
So this is an example of the Gibbs phase rule, specific where C happens to be equal to one.
And then, this is a more generalized example.
So next time, then, what we'll do is we'll start by deriving the Gibbs phase rule.
Which is not so hard.
And then start on our way to the colligative properties.
And then you'll have an exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Alright, so last time we started talking about more complicated mixtures.
To get into colligative properties.
And the example we gave was a binary system that has two components.
And two phases.
So we have a liquid phase on the bottom.
A gas phase on top, and two components, A and B, where compositions xA and xB are in the liquid phase.
Those are the mole fractions.
And yA and yB in the gas phase, the mole fractions in the gas phase.
And what we did last time was to begin to say how many degrees of freedom do we have, how many variables do we need to completely describe this mixture.
And it turned out that for this mixture, we only needed two variables.
Two degrees of freedom.
We started out with four variables; the temperature, the pressure, and the components in the composition in the liquid phase and the composition in the gas phase.
We need one of these, either xA or xB, and one of these, either yA or yB because they add up to 1.
So you start with four, but because you are in a mixture, an equilibrium, you have constraints.
You have the chemical potentials that have to be equal to each other.
The chemical potential of A in the gas phase has to be equal to the chemical potential of A in the liquid phase.
And the chemical potential of B in the gas phase has to B equal to the chemical potential of B in the liquid phase.
So you start with four variables.
Temperature, pressure, xA, yA, then you have the constraints that mu A, in the liquid phase has to be equal to mu A in the gas phase.
Because you're at equilibrium.
And mu B in the liquid phase has to be equal to mu B in the gas phase.
So four minus two constraints means you have two degrees of freedom.
And so if you want to know everything about this mixture, all you need is the temperature and the total pressure.
And that's enough.
It's kind of amazing.
Give me the temperature and pressure and these two components, like water and alcohol, for instance, and I can tell you everything about the compositions.
It's very powerful.
And then we said that this turned out to be a special case, or sub-case of the more general Gibbs phase rule.
where the Gibbs phase rule tells you that if you have C components, in this case here we have two, and the general case of C components with p phases, the number of degrees of freedom is C minus p plus two.
This is components.
And phases.
And where we're going to start today is by proving this.
And then we'll do more with this problem here.
So the first thing is for us to prove Gibbs' phase rule.
So Gibbs phase rule.
We start out with all the variables that we have.
And then we add the constraints.
And for the Gibbs phase rule, then we start with the two knobs that we can turn externally.
Which are the temperature and the pressure.
So we start with temperature and pressure, as two variables that we have.
And then we have a bunch of phases.
We have C phases.
And in each phase we have to describe the composition in that phase.
So, for each phase alpha, we have to describe its mole fraction.
So here we needed to describe xA for phase A.
And yA for phase A.
For component A in the gas phase.
So now we have alpha components.
And for each component we have to describe its mole fraction in that particular phase.
So we have to describe x sub i.
But we have a constraint on that.
The constraint is that the sum of all the mole factions has to be equal to one.
That's what it is, to be a mole fraction.
So the constraint is that x sub i, i from one to alpha, is equal to one.
So the constraint on the number of the components, instead of having, actually, i goes from one to C, because that's the number components.
So instead of having C different compositions that we need to take care of, we only need C minus one.
Just like here we only needed one, just xA.
We didn't need both xA and xB, because we knew the sum was equal to one.
So that means that we really need C minus one variables to describe the composition in one of the phases.
But we have p phases, so let me fix this.
We had C components.
We have p phases.
And so for each phase, we have to describe the composition.
So that means that we need p times C minus one variables to describe all the components and all the phases, plus these extra two.
So the total number variables then that we start out with before putting the fact that we're in equilibrium, is going to be two plus p times C minus one.
So that's a basic description of the system.
Then we put in the same constraints that we had up here.
In terms of the chemical potentials.
If I take any of the components, let's take the alpha component, or the i'th component.
Because it's in equilibrium, the chemical potential of that particular component has to be the same in all the phases.
So let's take the i'th component.
So yi in phase one has to be equal to yi in phase two, has to be equal to yi in phase three, has to be equal to yi in phase p. All the chemical potentials have to be the same across all the phases for that component.
And if you count the number of equations, the number of equals signs here, that's p minus one.
p minus one equals signs, constraints due to the fact that I have an equilibrium here.
So this gives me p minus one constraints.
And this is true for every single component.
Every single component has to have its chemical potential, its chemical potential equal throughout the phases.
So p minus one's constraint for one component times C components.
So for a total of C times p minus one total constraints.
My variables, my constraints.
So the number of degrees of freedom then becomes p times C minus one minus C times p minus one and then plus two.
And if you multiply this out, p times C gets rid of the p times C here.
And you get the plus C minus p plus two.
Which is the Gibbs phase rule, right?
Which I'm going to bring back up here.
C minus p plus two.
So just purely in accounting, just accounting for the variables and the constraints.
OK, any questions on how we did this?
Great.
OK.
So let's move on, then, to these ideal solutions.
Which is this case right here.
And the first thing we're going to do is look at the case where we have A is a solvent, and it's volatile.
And B is a solute, which is non-volatile.
So let's take the first case.
We have A volatile, that's a solvent.
Could be water.
And B is going to be the solute.
It's non-volatile.
And let's interchange solute or non-volatile to be consistent here.
So this could be sugar.
A sweet water.
So, Mr. Raoult looked at such solutions, and found that they could be well described by the following diagram here.
So if I plot on the x-axis here, from zero to one, I'm going to plot the partial molar fraction of the solute.
Partial molar fractions of the solute, that's going to be xB.
So when xB equals to zero, I have a pure water solution.
When xB is equal to one, I have pure sugar.
Then on the y axis, I'm going to plot the total pressure.
So when I have a pure water solution, the total pressure is going to be the same as the vapor pressure of water at a particular temperature.
Let's take temperature fixed at some value.
Temperature is fixed at some value.
I'm going to have the total pressure is going to be the partial pressure of the water.
And clearly, when I don't have any water left, all I have is the sugar, there's no pressure.
We're going to assume that it's totally non-volatile.
Obviously there's going to be a little bit of vapor pressure, even at room temperature.
Ten to the minus, I don't know, 13 torr or something.
But anyway, it's basically zero here.
And what Raoult said is that while these two points are just connected by a straight line.
Simplest way to connect two points.
Basically, he said that the pressure of, the vapor pressure of the water in this case, is equal to xA times pA star.
I could also plot this with xA, going from one to zero, since xA plus xB is equal to one.
Or I could write this as one minus xB times pA star.
Where a star, the star, means pure water.
So pure A.
And pA is the vapor pressure of the mixture.
Which in this case here is the total pressure, because I don't have anything else that's volatile.
And this is Raoult's law.
And the first thing that you can use this for is to look at your first colligative property which is vapor pressure lowering.
And let me point out another thing first.
So if a solution obeys that property here, it's called an ideal solution.
So if the water-sugar solution really did behave like a straight line like this, as you increase the amount of sugar, then it would be an ideal solution.
Many things are very close to it.
It's not such a bad approximation.
Especially in that region up here.
Around zero, when the amount of sugar is pretty small.
You expect to obey an ideal solution behavior.
Alright, so let's look now at a glass of water that has sugar in it.
We've got my glass of water, H20.
With some sugar dissolved in it.
And it's got a certain vapor pressure on top.
pA, or pH2O.
And I'm going to ask the question, what's the difference between the vapor pressure what it would be without the sugar in it.
What is pA star minus pA?
Obeys Raoult's law.
pA star, pA is xA pA star, plug that in here.
That's one minus xA pA star, one minus xA is xB.
xB pA star.
Well, the first thing I know is that this is a positive number.
xB is a positive number.
pA star is a positive number.
This is greater than zero.
That means that the act of putting some solute in my water decreases the vapor pressure of the water in the gas phase.
Pretty simple.
It's called vapor pressure law.
And we can actually take this one step further.
We can draw a phase diagram.
So our usual phase diagram here with temperature on this axis here and pressure on this axis here, and we've got our triple point sitting here.
And our gas-liquid line, with a critical point here.
So this is gas here, there's a liquid here.
And we've got our, this is going to be water.
So it's got a negative slope.
I think I've got it right this time.
And there's the solid phase here.
OK, this is for the pure stuff.
So now I can draw a similar diagram for sugared water.
And I just looked at this, assuming it obeys Raoult's law.
What we looked at here was the gas-liquid coexistence.
And we found that the gas-liquid coexistence point, the vapor pressure of the water, was less than if it were pure.
So that means that if I looked at the gas-liquid coexistence, which is this line right here.
In the presence of sugar, that whole line is going to get lowered.
The pressure, the vapor pressure, of the water is going to be lower when the sugar is there.
So the whole thing is going to get lower.
Now, if I have a solid, if the water is a solid, it's going to crystallize the water, the sugar is going to be separated from the water.
The sugar and the water are not going to really know about each other.
They're just going to have big chunks of pure water, with every now and then a molecule of sugar that's in there.
So you don't really expect to see any difference between the pure solid water, in terms of the gas-solid interface, and the pure liquid water.
There's no sugar in the gas phase.
And as far as the gas phase is concerned, that solid phase is pure water.
It's just seeing pure ice crystals on the surface.
Every now and then there's a molecule of sugar.
So you expect this to be pretty close here.
So you expect the triple point to go down.
And if I now, I go up to the solid-liquid coexistence, and go up like this, I can make a prediction about solid-liquid coexistence.
I can predict that if I'm looking at, let's put one bar here somewhere.
One bar.
Room temperature, sitting here somewhere.
One bar, room temperature, water is all liquid.
This is what it would usually, solid, liquid, that would be 273 degrees Kelvin.
That's the melting point of water.
So now if I put sugar in the water, and I follow my diagram here, what I find is that the new melting point of water is some new temperature T, T1, where T1 is less than 273 degrees Kelvin.
You all know this.
When you put something, like an impurity, a salt or sugar in the water, the melting point gets depressed.
It goes down.
Straight from here.
You can build it up straight from here.
When we get to the colligative properties, we'll attack it from the point of chemical potentials.
We'll equate chemical potentials and we'll look at the change in the chemical potential in the water, and the presence of the sugar and we'll do it rigorously.
But this is sort of the first indication that these colligative properties are all connected to each other.
They're connected to this diagram here.
OK.
So when I was a kid, and I knew that this was the case.
Everybody knows this is the case, right?
You put salt in the water and you expect all sorts of things to happen.
You expect the vapor pressure to go down.
And certainly you know that you add salt to the roads and the water melts, et cetera.
And my mother always told me that the reason why you added salt to the water was so that the temperature of the water would get higher when you try to boil pasta.
And for, I don't know, fifteen years I believed her.
Until I taught thermodynamics.
And I actually calculated the change in the temperature by adding a few pinches of salt to the water.
And you know how small that temperature change is?
You should calculate it.
It's really tiny.
It won't make any difference for the water.
Probably the difference in the temperature of the water will be the same as if I cooked here versus halfway up Mount Washington in New Hampshire or something.
Probably even less.
There's no difference.
The reason why you add salt is so that it tastes better.
That's the only reason.
It has nothing to do with thermodynamics.
Alright, any questions on this?
So we going to make it a little bit more complicated now, if there's no questions.
We're going to now have, instead of having one non-volatile solute, use we're going to have two, we're going to have this mixture.
Where both components are volatile.
Can we erase this?
Actually, I may not erase this.
Because I want to keep Raoult's law here.
OK, we're going to have a mixture of two volatile components.
For instance, water and alcohol.
Your your typical martini type situation.
Or margarita.
Whatever's your favorite drink.
So now we have xA, we have xB, we have yA, we have yB.
And we're going to assume that both obey Raoult's law with respect to each other.
So now if I draw a diagram, and I just focus on, so there's the total pressure sitting here.
And I just focus on one component and ignore the other.
Suppose I focus on A.
And I know that A in the presence of B, the partial pressure of A is going to obey Raoult's law.
And let's take A to be the more volatile component in this case here.
So pA star is bigger than pB star.
The partial pressure, pure A, had a particular temperature, T fixed is bigger than pB star, it's the more volatile of the two.
So if I were to plot pA as a function of xB, going from zero to one, so it'll be Raoult's law, I'm going to start at pA star.
And I'm going to go down linearly to xB equals one.
Now I can do the same thing for the partial pressure of B.
And assume that in the presence of A, just looking at this component B, now, and I'm looking at the partial pressure of B, and it's got its impurity in it, A.
B obeys Raoult's law with A in there.
But now if I want to keep the same x axis here, I can look at xA here, going from zero to one.
So I just basically flip the thing over.
So at xA equals zero that's pure B. pB star is less than pA star.
I'm starting here, on that side here.
pB star, and I'm going down in a straight line to zero.
pB is equal to xB pB star, straight line all the way to zero when xB is equal to zero.
So that's pB.
And that's pA. Now, the total pressure is the sum of the two.
pA plus pB.
So the total pressure in this system here, that I measure here, is just the sum of these two straight lines.
Let me do it in blue.
The total pressure, if you add up these two straight lines together, you got another straight line.
Two straight lines make a straight line.
p is pA plus pB is equal to xA pA star plus xB pB star.
Now, xA and xB are related.
xB is one minus xA.
And so really, this is only a function of one variable here.
Linearly, with respect to one variable.
OK, what does this diagram tell me?
So now, let's ignore how we built it.
And let's take a look at it.
So I have, going to take xB on the bottom here.
From zero to one.
We could choose either one, and I've got the straight line for the total pressure going from pA star to pB star.
T is fixed.
I need to tell you that T is fixed because I have two degrees of freedom, right?
I have two degrees of freedom.
My degrees of freedom are the temperature, which I'm fixing.
And the composition in the liquid phase, which I'm fixing.
So my two variables are xB and T. Those are my two degrees of freedom.
And I'm telling you what the total pressure is.
I'm giving you total pressure as a function of xB and T. By fixing T, I'm really telling you what the pressure is of the function of xB.
So the Gibbs phase rule then tells me that the pressure should be only a function of xB if T is fixed.
Should be a line.
Not necessarily straight, but it should be a line in this diagram.
To have coexistence.
So what this line is, then, this line is the line of points that tells me when I have coexistence between the gas phase and the liquid phase.
The coexistence line.
This is the gas-liquids coexistence line for the mixture.
It's not so different than this coexistence line up here.
So you can think of this as a phase diagram, kind of like you thought, you know this as a phase diagram.
It's got a coexistence line.
And then if I'm above this line, if I'm at a certain pressure here, I'm at this point here, this pressure well, let's go to a slightly higher different pressure here.
And this composition here.
I'm not on the line.
That means that I don't have coexistence between liquid and gas.
I'm above the line.
I'm at a pressure higher then where I would get coexistence.
The pressure is higher, means I probably don't have a gas any more.
Probably means that the total pressure pushing down on my mixture is too high to have a gas phase around.
It means that the top part here is a liquid phase.
So I'm in the liquid phase if I'm above this line and I have a coexistence between the gas and the liquid if I'm on the line.
Now, if I'm below this line here, well, I get in trouble.
I get in trouble because my first instinct would be, well this is just gas.
I go from liquid, and then I have coexistence.
Then I go into the gas phase.
But my x-axis here is the composition in the liquid phase.
There's no liquid around.
So I really can't say anything here.
This diagram is a little bit meaningless down here.
Because my x-axis is describing a phase which doesn't exist on the diagram.
On this diagram, here.
So when you see it, this diagram here with Raoult's law on it, you've got to remember that the part that's really interesting is above the line and at the line itself.
So you could do an experiment, then, on this line here.
T fixed.
This is the liquid phase.
So suppose I start with some high pressure up here somewhere.
At this point, let's call this point one.
We've got my container, my piston here.
p is equal to p1.
Again, I put the point in an awkward place.
Let's put it right there.
p1, and composition xB.
p1 and xB here.
And I'm going to slowly decrease the pressure.
I start in this composition.
The composition doesn't change.
I'm just decreasing the pressure.
Decrease the pressure, decrease the pressure, decrease the pressure.
And at some point, I get to that line.
Get to that line, what happens when I get to that line?
I start to see bubbles coming out of the liquid.
Vapor starts to form, because it wants to be in the coexistence line.
So I get to that line.
And I get a little bit of bubbles forming.
And a little bit of vapor.
That's the liquid here.
Little bit of gas.
And that's why this line here is called the bubble line.
So I've gone down, decreased the pressure.
So p has gone down now.
Decreased the pressure.
And if I keep on decreasing pressure, well, I'm not going to jump into this area of the diagram.
Because this area of the diagram means there's no liquid, it's a meaningless area.
If I keep decreasing the pressure, I'm still going to be in coexistence.
I'm just going to make more gas.
More vapor.
I'm going to transfer more of the liquid into the vapor phase.
I'm going to ride down that line.
I'm going to ride down the line.
I'm decreasing the pressure.
I'm not going to skip into here.
I'm going to ride down the line here.
We're going to keep decreasing the pressure even more.
p goes down even more.
Now, I make even more vapor.
There's my piston right there.
There's the gas phase.
There's the liquid phase.
Now I've transferred material from the liquid phase to the gas phase.
But I also changed the composition of my liquid phase.
And if I read my new composition, this is what I started out with.
xB.
My new composition in the liquid phase is going to be, let's call it xB prime.
There's going to be more solute in there.
Or more B in there, than what I started out with.
Now which one was the more volatile one?
A or B?
A was more volatile, right?
So as I decreased the pressure and I started bubbling off the material, which one's going to bubble off preferentially?
A is going to.
So it makes sense that I would concentrate B in there.
Decrease the pressure.
Bubbles that are rich in A come out.
Both A and B come out.
But more A comes out than B.
And what's left over is rich and then more than the less volatile material which is B.
So we're beginning to see how distillation comes about.
This is the first step in it.
In a distillation process.
So let's make this a little bit more complicated now.
Any questions, first?
Alright, now, suppose I wanted to know.
So I found out where the composition in the liquid phase is here.
And I know that should tell me immediately whether the composition in the gas phase is here.
So I have all, I have everything I need to know to calculate what the composition in the gas phase is.
So I have everything I need, then, to calculate and draw a diagram that looks just like this.
Except where my x-axis is the y's instead of the x's.
Going to make the same diagram, except now I'm going to use the gas phase as my reference point.
The composition in the gas phase.
So I need to get, so here I got p as a function of xB.
I want p, total pressure, as a function of yB.
So I can draw a diagram that looks just like this except now with the x-axis being the gas phase composition.
Let's turn the crank.
What do I know?
I know Dalton's law.
pA is yA times total pressure.
I'm trying mix up the partial pressures and the total pressure in all ways that I know how to write it.
And then hope that something's going to come out that's going to be helpful.
So that's Dalton's law.
I know Raoult's law.
pA is xA star times pA star.
And that's Raoult.
And I can write Raoult's law in a different way.
I can write as pB is equal to xB pB star, that's just Raoult's law for B.
One minus xA pB star.
And I'm looking for this.
I'm looking for the composition in the gas phase.
Or the total pressure as a function of the composition in the gas phase.
Let's start by finding the composition in the gas phase as a function of the composition in the liquid phase.
And the total pressure.
Rewrite yA is equal to pA over P. Total pressure is pA plus pB.
Sum of the two partial pressures.
pA over pA plus pB.
And I'm trying to get yA in terms of the xA's.
Or xB's, in terms of composition in the liquid phase.
And I know how to relate that to a constant.
Which is the vapor pressure of the pure material times the, and the composition of the liquid phase, Raoult's law.
This is xA pA star divided by xA pA star plus xB pB star.
And I know that xB is one minus xA.
So this is xA pA star times pB star plus pA star minus pB star times xA.
Where all I did was replace xB with one minus xA.
And rearrange my equation on the bottom.
So I've gotten the composition in the gas phase in terms of the composition in the liquid phase.
They're not independent.
I knew they weren't independent.
And this just shows me that math works.
OK.
So I've got that.
And I can invert this to get the composition in the liquid phase in terms of the composition in the gas phase.
It's not so straightforward, but you can get xA as a function of yA, as well.
You've got xA, if you reverse this you get xA is equal to yA pB star.
It looks kind of the same.
pB star plus pA star plus pB star minus pA star times yA.
xA as a function of yA.
Alright.
So now you can actually put everything together, starting from Dalton's law.
Because really what we want is the total pressure as a function of the composition in the gas phase.
So there's our total pressure here.
There's the composition in the gas phase.
There's pA. We've found composition in the gas phase in terms of composition in the liquid phase.
P is equal to yA over pA.
Which is equal to yA over xA pA star.
So if we could get rid of this here, in terms of yA, we'd be all set.
We'd get the total pressure as a function of the composition in the gas phase.
And there's what we need.
So we plug that in here.
And now we have the total pressure in terms of constants like these pure vapor pressures and the composition of A in the gas phase.
So let me find a way to write that somewhere.
Let me write it right here in this little box here.
Plug it in, you get something that's not a straight line.
Unfortunately.
Doesn't look like Raoult's law.
pA star pB star.
pA star plus pB star minus pA star times yA.
It's not a straight line.
It's a more complicated function.
What does it look like?
Looks like this.
I'm going to use the same sort of plot.
I going to plot the total pressure on this axis here, on the y axis.
I'm going to keep the T fixed.
Temperature is fixed.
So I'm going to find the total pressure as a function of the composition in the gas phase.
I'm going to put yB here.
yB going from zero to one.
So it's kind of looking like what I had before.
Total pressure is going to look the same.
If you want to do it in terms of B, you just have everywhere you see B replace it with A.
And everywhere you see A replace it with B.
You just need to change the two.
You know that if yB is equal to zero, that you have pure A.
So you know that this is going to be pA star here.
You know if you have yB equals to one in the gas phase, you have pure B in the gas phase.
You know that what you have is the vapor pressure of the pure B, in this case the pure water.
pB star, OK?
Where A is more volatile then B.
A would be the ethanol and B would be the water.
And it's not a straight line.
If it were Raoult, he would say just connect these two with a straight line.
But it's not.
It's not, we just figured out with an equation here.
And it turns out to be a line that looks like this.
It's a curved line that looks a bit like this.
OK, what does this line mean?
This line is also a coexistence line.
That's what we started out with.
We started out with Raoult.
That was part of our derivation.
Raoult's law tells you the composition at coexistence.
Tells you the pressure of, the total pressure, or partial pressure at coexistence and the composition.
So this is a coexistence line between the liquid and the gas.
Coexistence between liquid and gas.
But now, it's not as a function of the composition at the liquid phase.
It's a function of the composition in the gas phase.
Which was missing before.
So now if I do my experiment, and I'm, so what does this mean?
So if I have a point, if I have a pressure which is lower than the line here, then I have pure gas.
Pressure, low pressure, I have pure gas.
Don't have any liquids.
And that's fine, because I can, I know what that point is.
If I'm sitting below the line here somewhere, so I'm sitting here.
Some composition in the gas phase, a certain pressure.
That's fine.
It's a meaningful point.
So my container here has a piston on it.
And I'm starting out with this pressure, p1 here.
p1 pressing down.
And I got yB in this here.
And I increase the pressure, pressure goes up.
Composition doesn't change.
I'm just increasing the pressure.
Increasing the pressure, increasing the pressure.
Now, at some point the pressure becomes big enough that I start to see liquid forming.
Little drops of liquid forming in my container.
So now I have little drops of liquid that are forming in my container, that are going to start to rain down and form a little bit of liquid pooling at the bottom of the container.
And that's called the dew line.
The other one was called the bubble line, this is called the dew line.
So when I reach the dew line, I start forming liquid.
And again, just like in the previous case, if I keep increasing the pressure, I don't go into this area here.
First of all, this area's meaningless for this diagram here.
Because this area's pure liquid.
But the x-axis here doesn't tell me about the composition of pure liquid.
It's all yB.
It's all about the composition in the gas -- yes
[INAUDIBLE]
Between the dew line and the bubble line?
Yeah
This never, never world.
Where you're not allowed to inhabit.
We're going to go into that next time.
On how to use those two curves together.
OK, but the same thing happens here.
If I increase the pressure, I'm going to ride the coexistence line up.
I ride it up, and keep riding it up.
And I get liquid forming in here.
And I can find out the composition in the gas phase now.
The composition in the gas phase, as I increase the pressure, I get yB prime, there's yB that I started out with with.
With yB prime is less than yB.
And remember, B was the less volatile of the two.
As I increase the pressure, I am forming liquid.
The composition in the gas phase changes.
You decrease the mole fraction of the less volatile material in the gas phase.
The more volatile A becomes more prevalent in the gas phase.
And what's coming down is mostly, then, it's enriched in B.
You've done an enrichment of the less volatile material down in here.
And now you can put the two together, as the question was asked.
Now we can have, we mix the two together.
We mix our Raoult diagram with our dew line diagram.
We mix the bubble line and the dew line together.
In one diagram, which is going to be a powerful diagram to use.
Total pressure.
Temperature fixed.
You've got two materials, pA star is bigger than pB star.
So I'm going to have pA star here, pB star down here.
And if I choose, as my x-axis, to have xB going from zero to one, I can draw a coexistence line on this axis.
On this diagram.
That's the coexistence line in terms of xB.
And any point up here is well-described by this xB value, and the p value.
And as long as I have xB here, I'm not allowed to touch the bottom part of this diagram.
But now if I add another x-axis, I'm going to add yB.
From zero to one.
I draw a coexistence line in terms of yB.
Like this.
So this line is p as a function of yB coexistence line.
This line is total pressure of function of xB.
And as long as I'm on this line or below this line, then this axis makes sense.
That's how we build this diagram here.
It's got a bubble line on top, and the dew line on the bottom.
So I'm up here going down, I'm going to hit the bubble line.
If I'm down here and go up in pressure, I'm going to hit the dew line.
And next time -- well, next time we have an exam.
But on Friday, we'll figure out how to use this diagram.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So.
In the meantime, you've started looking at two phase equilibrium.
So now we're starting to look at mixtures.
And so now we have more than one constituent.
And we have more than one phase present.
Right?
So you've started to look at things that look like this, where you've got, let's say, two components.
Both in the gas phase.
And now to try to figure out what the phase equilibria look like.
Of course it's now a little bit more complicated than what you went through before, where you can get pressure temperature phase diagrams with just a single component.
Now we want to worry about what's the composition.
Of each of the components.
In each of the phases.
And what's the temperature and the pressure.
Total and partial pressures and all of that.
So you can really figure out everything about both phases.
And there are all sorts of important reasons to do that, obviously lots of chemistry happens in liquid mixtures.
Some in gas mixtures.
Some where they're in equilibrium.
All sorts of chemical processes.
Distillation, for example, takes advantage of the properties of liquid and gas mixtures.
Where one of them might be richer, will be richer, and the more volatile of the components.
That can be used as a basis for purification.
You mix ethanol and water together so you've got a liquid with a certain composition of each.
The gas is going to be richer and the more volatile of the two, the ethanol.
So in a distillation, where you put things up in the gas, more of the ethanol comes up.
You could then collect that gas, right?
And re-condense it, and make a new liquid.
Which is much richer in ethanol than the original liquid was.
Then you could make, then you could put some of them up into the gas phase.
Where it will be still richer in ethanol.
And then you could collect that and repeat the process.
So the point is that properties of liquid gas, two-component or multi-component mixtures like this can be exploited.
Basically, the different volatilities of the different components can be exploited for things like purification.
Also if you want to calculate chemical equilibria in the liquid and gas phase, of course, now you've seen chemical equilibrium, so the amount of reaction depends on the composition.
So of course if you want reactions to go, then this also can be exploited by looking at which phase might be richer in one reactant or another.
And thereby pushing the equilibrium toward one direction or the other.
OK.
So.
we've got some total temperature and pressure.
And we have compositions.
So in the gas phase, we've got mole fractions yA and yB.
In the liquid phase we've got mole fractions xA and xB.
So that's our system.
One of the things that you established last time is that, so there are the total number of variables including the temperature and the pressure.
And let's say the mole fraction of A in each of the liquid and gas phases, right?
But then there are constraints.
Because the chemical potentials have to be equal, right?
Chemical potential of A has to be equal in the liquid and gas.
Same with B.
Those two constraints reduce the number of independent variables.
So there'll be two in this case rather than four independent variables.
If you control those, then everything else will follow.
What that means is if you've got a, if you control, if you fix the temperature and the total pressure, everything else should be determinable.
No more free variables.
And then, what you saw is that in simple or ideal liquid mixtures, a result called Raoult's law would hold.
Which just says that the partial pressure of A is equal to the mole fraction of A in the liquid times the pressure of pure A over the liquid.
And so what this gives you is a diagram that looks like this.
If we plot this versus xB, this is mole fraction of B in the liquid going from zero to one.
Then we could construct a diagram of this sort.
So this is the total pressure of A and B.
The partial pressures are given by these lines.
So this is our pA star and pB star.
The pressures over the pure liquid A and B at the limits of mole fraction of B being zero and one.
So in this situation, for example, A is the more volatile of the components.
So it's partial pressure over its pure liquid.
At this temperature.
Is higher than the partial pressure of B over its pure liquid.
A would be the ethanol, for example and B the water in that mixture.
OK. Then you started looking at both the gas and the liquid phase in the same diagram.
So this is the mole fraction of the liquid.
If you look and see, well, OK now we should be able to determine the mole fraction in the gas as well.
Again, if we note total temperature and pressure, everything else must follow.
And so, you saw this worked out.
Relation between p and yA, for example.
The result was p is pA star times pB star over pA star plus pB star minus pA star times yA.
And the point here is that unlike this case, where you have a linear relationship, the relationship between the pressure and the liquid mole fraction isn't linear.
We can still plot it, of course.
So if we do that, then we end up with a diagram that looks like the following.
Now I'm going to keep both mole fractions, xB and yB, I've got some total pressure.
I still have my linear relationship.
And then I have a non-linear relationship between the pressure and the mole fraction in the gas phase.
So let's just fill this in.
Here is pA star still.
Here's pB star.
Of course, at the limits they're still, both mole fractions they're zero and one.
OK.
I believe this is this is where you ended up at the end of the last lecture.
But it's probably not so clear exactly how you read something like this.
And use it.
It's extremely useful.
You just have to kind of learn how to follow what happens in a diagram like this.
And that's what I want to spend some of today doing.
Is just, walking through what's happening physically, with a container with a mixture of the two.
And how does that correspond to what gets read off the diagram under different conditions.
So.
Let's just start somewhere on a phase diagram like this.
Let's start up here at some point one, so we're in the pure - well, not pure, you're in the all liquid phase.
It's still a mixture.
It's not a pure substance.
pA star, pB star.
There's the gas phase.
So, if we start at one, and now there's some total pressure.
And now we're going to reduce it.
What happens?
We start with a pure - with an all-liquid mixture.
No gas.
And now we're going to bring down the pressure.
Allowing some of the liquid to go up into the gas phase.
So, we can do that.
And once we reach point two, then we find a coexistence curve.
Now the liquid and gas are going to coexist.
So this is the liquid phase.
And that means that this must be xB.
And it's xB at one, but it's also xB at two, and I want to emphasize that.
So let's put our pressure for two.
And if we go over here, this is telling us about the mole fraction in the gas phase.
That's what these curves are, remember.
So this is the one that's showing us the mole fraction in the liquid phase.
This nonlinear one in the gas phase.
So that means just reading off it, this is xB, that's the liquid mole fraction.
Here's yB.
The gas mole fraction.
They're not the same, right, because of course the components have different volatility.
A's more volatile.
So that means that the mole fraction of B in the liquid phase is higher than the mole fraction of B in the gas phase.
Because A is the more volatile component.
So more, relatively more, of A, the mole fraction of A is going to be higher up in the gas phase.
Which means the mole fraction of B is lower in the gas phase.
So, yB less than xB if A is more volatile.
OK, so now what's happening physically?
Well, we started at a point where we only had the liquid present.
So at our initial pressure, we just have all liquid.
There's some xB at one.
That's all there is, there isn't any gas yet.
Now, what happened here?
Well, now we lowered the pressure.
So you could imagine, well, we made the box bigger.
Now, if the liquid was under pressure, being squeezed by the box, right then you could make the box a little bit bigger.
And there's still no gas.
That's moving down like this.
But then you get to a point where there's just barely any pressure on top of the liquid.
And then you keep expanding the box.
Now some gas is going to form.
So now we're going to go to our case two.
We've got a bigger box.
And now, right around where this was, this is going to be liquid.
And there's gas up here.
So up here is yB at pressure two.
Here's xB at pressure two.
Liquid and gas.
So that's where we are at point two here.
Now, what happens if we keep going?
Let's lower the pressure some more.
Well, we can lower it and do this.
But really if we want to see what's happening in each of the phases, we have to stay on the coexistence curves.
Those are what tell us what the pressures are.
What the partial pressure are going to be in each of the phases.
In each of the two, in the liquid and the gas phases.
So let's say we lower the pressure a little more.
What's going to happen is, then we'll end up somewhere over here.
In the liquid, and that'll correspond to something over here in the gas.
So here's three.
So now we're going to have, that's going to be xB at pressure three.
And over here is going to be yB at pressure three.
And all we've done, of course, is we've just expanded this further.
So now we've got a still taller box.
And the liquid is going to be a little lower because some of it has evaporated, formed the gas phase.
So here's xB at three.
Here's yB at three, here's our gas phase.
Now we could decrease even further.
And this is the sort of thing that you maybe can't do in real life.
But I can do on a blackboard.
I'm going to give myself more room on this curve, to finish this illustration.
There.
Beautiful.
So now we can lower a little bit further, and what I want to illustrate is, if we keep going down, eventually we get to a pressure where now if we look over in the gas phase, we're at the same pressure, mole fraction that we had originally in the liquid phase.
So let's make four even lower pressure.
What does that mean?
What it means is, we're running out of liquid.
So what's supposed to happen is A is the more volatile component.
So as we start opening up some room for gas to form, you get more of A in the gas phase.
But of course, and the liquid is richer in B.
But of course, eventually you run out of liquid.
You make the box pretty big, and you run out, or you have the very last drop of liquid.
So what's the mole fraction of B in the gas phase?
It has to be the same as what it started in in the liquid phase.
Because after all the total number of moles of A and B hasn't changed any.
So if you take them all from the liquid and put them all up into the gas phase, it must be the same.
So yB of four.
Once you just have the last drop.
So then yB of four is basically equal to xB of one.
Because everything's now up in the gas phase.
So in principle, there's still a tiny, tiny bit of xB at pressure four.
Well, we could keep lowering the pressure.
We could make the box a little bigger.
Then the very last of the liquid is going to be gone.
And what'll happen then is, we're all here.
There's no more liquid.
We're not going down on the coexistence curve any more.
We don't have a liquid gas coexistence any more.
We just have a gas phase.
Of course, we can continue to lower the pressure.
And then what we're doing is just going down here.
So there's five.
And five is the same as this only bigger.
And so forth.
OK, any questions about how this works?
It's really important to just gain facility in reading these things and seeing, OK, what is it that this is telling you.
And you can see it's not complicated to do it, but it takes a little bit of practice.
OK. Now, of course, we could do exactly the same thing starting from the gas phase.
And raising the pressure.
And although you may anticipate that it's kind of pedantic, I really do want to illustrate something by it.
So let me just imagine that we're going to do that.
Let's start all in the gas phase.
Up here's the liquid.
pA star, pB star.
And now let's start somewhere here.
So we're down somewhere in the gas phase with some composition.
So it's the same story, except now we're starting here.
It's all gas.
And we're going to start squeezing.
We're increasing the pressure.
And eventually here's one, will reach two, so of course here's our yB.
We started with all gas, no liquid.
So this is yB of one.
It's the same as yB of two, I'm just raising the pressure enough to just reach the coexistence curve.
And of course, out here tells us xB of two, right?
So what is it saying?
We've squeezed and started to form some liquid.
And the liquid is richer in component B.
Maybe it's ethanol water again.
And we squeeze, and now we've got more water in the liquid phase than in the gas phase.
Because water's the less volatile component.
It's what's going to condense first.
So the liquid is rich in the less volatile of the components.
Now, obviously, we can continue in doing exactly the reverse of what I showed you.
But all I want to really illustrate is, this is a strategy for purification of the less volatile component.
Once you've done this, well now you've got some liquid.
Now you could collect that liquid in a separate vessel.
So let's collect the liquid mixture with xB of two.
So it's got some mole fraction of B.
So we've purified that.
But now we're going to start, we've got pure liquid.
Now let's make the vessel big.
So it all goes into the gas phase.
Then lower p. All gas.
So we start with yB of three, which equals xB of two.
In other words, it's the same mole fraction.
So let's reconstruct that.
So here's p of two.
And now we're going to go to some new pressure.
And the point is, now we're going to start, since the mole fraction in the gas phase that we're starting from is the same number as this was.
So it's around here somewhere.
That's yB of three equals xB of two.
And we're down here.
In other words, all we've done is make the container big enough so the pressure's low and it's all in the gas phase.
That's all we have, is the gas.
But the composition is whatever the composition is that we extracted here from the liquid.
So this xB, which is the liquid mole fraction, is now yB, the gas mole fraction.
Of course, the pressure is different.
Lower than it was before.
Great.
Now let's increase.
So here's three.
And now let's increase the pressure to four.
And of course what happens, now we've got coexistence.
So here's liquid.
Here's gas.
So, now we're over here again.
There's xB at pressure four.
Pure still in component B.
We can repeat the same procedure.
Collect it.
All liquid, put it in a new vessel.
Expand it, lower the pressure, all goes back into the gas phase.
Do it all again.
And the point is, what you're doing is walking along here.
Here to here.
Then you start down here, and go from here to here.
From here to here.
And you can purify.
Now, of course, the optimal procedure, you have to think a little bit.
Because if you really do precisely what I said, you're going to have a mighty little bit of material each time you do that.
So yes it'll be the little bit you've gotten at the end is going to be really pure, but there's not a whole lot of it.
Because, remember, what we said is let's raise the pressure until we just start being on the coexistence curve.
So we've still got mostly gas.
Little bit of liquid.
Now, I could raise the pressure a bit higher.
So that in the interest of having more of the liquid, when I do that, though, the liquid that I have at this higher pressure won't be as enriched as it was down here.
Now, I could still do this procedure.
I could just do more of them.
So it takes a little bit of judiciousness to figure out how to optimize that.
In the end, though, you can continue to walk your way down through these coexistence curves and purify repeatedly the component B, the less volatile of them, and end up with some amount of it.
And there'll be some balance between the amount that you feel like you need to end up with and how pure you need it to be.
Any questions about how this works?
So purification of less volatile components.
Now, how much of each of these quantities in each of these phases?
So, pertinent to this discussion, of course we need to know that.
If you want to try to optimize a procedure like that, of course it's going to be crucial to be able to understand and calculate for any pressure that you decide to raise to, just how many moles do you have in each of the phases?
So at the end of the day, you can figure out, OK, now when I reach a certain degree of purification, here's how much of the stuff I end up with.
Well, that turns out to be reasonably straightforward to do.
And so what I'll go through is a simple mathematical derivation.
And it turns out that it allows you to just read right off the diagram how much of each material you're going to end up with.
So, here's what happens.
This is something called the lever rule.
How much of each component is there in each phase?
So let's consider a case like this.
Let me draw yet once again, just to get the numbering consistent.
With how we'll treat this.
So we're going to start here.
And I want to draw it right in the middle, so I've got plenty of room.
And we're going to go up to some pressure.
And somewhere out there, now I can go to my coexistence curves.
Liquid.
And gas.
And I can read off my values.
So this is the liquid xB.
So I'm going to go up to some point two, here's xB of two.
Here's yB of two.
Great.
Now let's get these written in.
So let's just define terms a little bit.
nA, nB.
Or just our total number of moles.
ng and n liquid, of course, total number of moles.
In the gas and liquid phases.
So let's just do the calculation for each of these two cases.
We'll start with one.
That's the easier case.
Because then we have only the gas.
So at one, all gas.
It says pure gas in the notes, but of course that isn't the pure gas.
It's the mixture of the two components.
So.
How many moles of A?
Well it's the mole fraction of A in the gas.
Times the total number of moles in the gas.
Let me put one in here.
Just to be clear.
And since we have all gas, the number of moles in the gas is just the total number of moles.
So this is just yA at one times n total.
Let's just write that in.
And of course n total is equal to nA plus nB.
So now let's look at condition two.
Now we have to look a little more carefully.
Because we have a liquid gas mixture.
So nA is equal to yA at pressure two.
Times the number of moles of gas at pressure two.
Plus xA, at pressure two, times the number of moles of liquid at pressure two.
Now, of course, these things have to be equal.
The total number of moles of A didn't change, right?
So those are equal.
Then yA of two times ng of two.
Plus xA of two times n liquid of two, that's equal to yA of one times n total.
Which is of course equal to yA of one times n gas at two plus n liquid at two.
I suppose I could be, add that equality.
Of course, it's an obvious one.
But let me do it anyway.
The total number of moles is equal to nA plus nB.
But it's also equal to n liquid plus n gas.
And that's all I'm taking advantage of here.
And now I'm just going to rearrange the terms.
So I'm going to write yA at one minus yA at two, times ng at two, is equal to, and I'm going to take the other terms, the xA term.
xA of two minus yA of one times n liquid at two.
So I've just rearranged the terms.
And I've done that because now, I think I omitted something here.
yA of one times ng.
No, I forgot a bracket, is what I did.
yA of one there.
And I did this because now I want to do is look at the ratio of liquid to gas at pressure two.
So, ratio of I'll put it gas to liquid, that's ng of two over n liquid at two.
And that's just equal to xA of two minus yA at one minus yA at one minus yA at two.
So what does it mean?
It's the ratio of these lever arms.
That's what it's telling me.
I can look, so I raise the pressure up to two.
And so here's xB at two, here's yB at two.
And I'm here somewhere.
And this little amount and this little amount, that's that difference.
And it's just telling me that ratio of those arms is the ratio of the total number of moles of gas to liquid.
And that's great.
Because now when I go back to the problem that we were just looking at, where I say, well I'm going to purify the less volatile component by raising the pressure until I'm at coexistence starting in the gas phase.
Raise the pressure, I've got some liquid.
But I also want some finite amount of liquid.
But I don't want to just, when I get the very, very first drop of liquid now collected, of course it's enriched in the less volatile component.
But there may be a minuscule amount, right?
So I'll raise the pressure a bit more.
I'll go up in pressure.
And now, of course, when I do that the amount of enrichment of the liquid isn't as big as it was if I just raised it up enough to barely have any liquid.
Then I'd be out here.
But I've got more material in the liquid phase to collect.
And that's what this allows me to calculate.
Is how much do I get in the end.
So it's very handy.
You can also see, if I go all the way to the limit where the mole fraction in the liquid at the end is equal to what it was in the gas when I started, what that says is that there's no more gas left any more.
In other words, these two things are equal.
If I go all the way to the point where I've got all the, this is the amount I started with, in the pure gas phase, now I keep raising it all the way.
Until I've got the same mole fraction in the liquid.
Of course, we know what that really means.
That means that I've gone all the way from pure gas to pure liquid.
And the mole fraction in that case has to be the same.
And what this is just telling us mathematically is, when that happens this is zero.
That means I don't have any gas left.
Yeah
[INAUDIBLE]
No.
Because, so it's the mole fraction in the gas phase.
But you've started with some amount that it's only going to go down from there
[INAUDIBLE]
Yeah.
Yeah.
Any other questions?
OK. Well, now what I want to do is just put up a slightly different kind of diagram, but different in an important way.
Namely, instead of showing the mole fractions as a function of the pressure.
And I haven't written it in, but all of these are at constant temperature, right?
I've assumed the temperature is constant in all these things.
Now let's consider the other possibility, the other simple possibility, which is, let's hold the pressure constant and vary the temperature.
Of course, you know in the lab, that's usually what's easiest to do.
Now, unfortunately, the arithmetic gets more complicated.
It's not monumentally complicated, but here in this case, where you have one linear relationship, which is very convenient.
From Raoult's law.
And then you have one non-linear relationship there for the mole fraction of the gas.
In the case of temperature, they're both, neither one is linear.
Nevertheless, we can just sketch what the diagram looks like.
And of course it's very useful to do that, and see how to read off it.
And I should say the derivation of the curves isn't particularly complicated.
It's not particularly more complicated than what I think you saw last time to derive this.
There's no complicated math involved.
But the point is, the derivation doesn't yield a linear relationship for either the gas or the liquid part of the coexistence curve.
OK, so we're going to look at temperature and mole fraction phase diagrams.
Again, a little more complicated mathematically but more practical in real use.
And this is T. And here is the, sort of, form that these things take.
So again, neither one is linear.
Up here, now, of course if you raise the temperatures, that's where you end up with gas.
If you lower the temperature, you condense and get the liquid.
So, this is TA star.
TB star.
So now I want to stick with A as the more volatile component.
At constant temperature, that meant that pA star is bigger than pB star.
In other words, the vapor pressure over pure liquid A is higher than the vapor pressure over pure liquid B.
Similarly, now I've got constant pressure and really what I'm looking at, let's say I'm at the limit where I've got the pure liquid.
Or the pure A.
And now I'm going to, let's say, raise the temperature until I'm at the liquid-gas equilibrium.
That's just the boiling point.
So if A is the more volatile component, it has the lower boiling point.
And that's what this reflects.
So higher pB star A corresponds to lower TA star A.
Which is just the boiling point of pure A.
So, this is called the bubble line.
That's called the dew line.
All that means is, let's say I'm at high temperature.
I've got all gas.
Right no coexistence, no liquid yet.
And I start to cool things off.
Just to where I just barely start to get liquid.
What you see that as is, dew starts forming.
A little bit of condensation.
If you're outside, it means on the grass a little bit of dew is forming.
Similarly, if I start at low temperature, all liquid now I start raising the temperature until I just start to boil.
I just start to see the first bubbles forming.
And so that's why these things have those names.
So now let's just follow along what happens when I do the same sort of thing that I illustrated there.
I want to start at one point in this phase diagram.
And then start changing the conditions.
So let's start here.
So I'm going to start all in the liquid phase.
That is, the temperature is low.
Here's xB.
And my original temperature.
Now I'm going to raise it.
So if I raise it a little bit, I reach a point at which I first start to boil.
Start to find some gas above the liquid.
And if I look right here, that'll be my composition.
Let me raise it a little farther, now that we've already seen the lever rule and so forth.
I'll raise it up to here.
And that means that out here, I suppose I should do here.
So, here is the liquid mole fraction at temperature two.
xB at temperature two.
This is yB at temperature two.
The gas mole fraction.
So as you should expect, what's going to happen here is that the gas, this is going to be lower in B.
A, that means that the mole fraction of A must be higher in the gas phase.
That's one minus yB.
So xA is one minus -- yA, which is one minus yB higher in gas phase.
Than xA, which is one minus xB.
In other words, the less volatile component is enriched up in the gas phase.
Now, what does that mean?
That means I could follow the same sort of procedure that I indicated before when we looked at the pressure mole fraction phase diagram.
Namely, I could do this and now I could take the gas phase.
Which has less of B.
It has more of A.
And I can collect it.
And then I can reduce the temperature.
So it liquefies.
So I can condense it, in other words.
So now I'm going to start with, let's say I lower the temperature enough so I've got basically pure liquid.
But its composition is the same as the gas here.
Because of course that's what that liquid is formed from.
I collected the gas and separated it.
So now I could start all over again.
Except instead of being here, I'll be down here.
And then I can raise the temperature again.
To some place where I choose.
I could choose here, and go all the way to hear.
A great amount of enrichment.
But I know from the lever rule that if I do that, I'm going to have precious little material over here.
So I might prefer to raise the temperature a little more.
Still get a substantial amount of enrichment.
And now I've got, in the gas phase, I'll further enriched in component A.
And again I can collect the gas.
Condense it.
Now I'm out here somewhere, I've got all liquid and I'll raise the temperature again.
And I can again keep walking my way over.
And that's what happens during an ordinary distillation.
Each step of the distillation walks along in the phase diagram at some selected point.
And of course what you're doing is, you're always condensing the gas.
And starting with fresh liquid that now is enriched in more volatile of the components.
So of course if you're really purifying, say, ethanol from an ethanol water mixture, that's how you do it.
Ethanol is the more volatile component.
So a still is set up.
It will boil the stuff and collect the gas and and condense it.
And boil it again, and so forth.
And the whole thing can be set up in a very efficient way.
So you have essentially continuous distillation.
Where you have a whole sequence of collection and condensation and reheating and so forth events.
So then, in a practical way, it's possible to walk quite far along the distillation, the coexistence curve, and distill to really a high degree of purification.
Any questions about how that works?
OK.
I'll leave till next time the discussion of the chemical potentials.
But what we'll do, just to foreshadow a little bit, what I'll do at the beginning of the next lecture is what's at the end of your notes here.
Which is just to say OK, now if we look at Raoult's law, it's straightforward to say what is the chemical potential for each of the substances in the liquid and the gas phase.
Of course, it has to be equal.
Given that, that's for an ideal solution.
We can gain some insight from that.
And then look at real solutions, non-ideal solutions, and understand a lot of their behavior as well.
Just from starting from our understanding of what the chemical potential does even in a simple ideal mixture.
So we'll look at the chemical potentials.
And then we'll look at non-ideal solution mixtures next time.
See you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And today I'll continue with both ideal and non-ideal, or real, liquid mixtures.
And let me just say the reason we're in some sense lavishing so much attention on this topic is because, after all, there's just an enormous amount of chemistry that happens in liquid mixtures.
Awful lot of biology.
Awful lot of just processing of stuff.
Distillation and everything else.
So there's so much chemistry that takes place in liquid mixtures that it is really important to have a sense of what the free energy and chemical potential of each of the species is doing in there.
Because of course this is what directly guides the chemistry that happens.
So I just want to start by finishing up something, a couple of things, from the topics from the last lecture.
One of them is, I derived an expression for the lever rule.
And I just wanted to make a little more explicit the result.
It may not have been completely clear.
So, I just want to go back to that.
We had looked at what happens if you start at some particular point, labeled one, and work your way up.
So we're raising the pressure.
So we're in the all gas region to start.
All gas region of the phase diagram.
This is xB and yB.
The liquid and gas mole fractions of B.
And the idea is that here is our initial value.
Now we're only in the gas phase.
There is no liquid.
So this is yB of one.
There is no xB at this point.
We raise the pressure.
And at some point, depending on where we end, we then look to both coexistence curves to determine the compositions of both the liquid and the gas phases.
So when we look out here, at the liquid phase.
That gives us xB.
At where I labeled point two, so there's two after we raise the pressure.
We started at one.
And then over here, we look at the gas phase coexistence curve.
so there's our yB value at two, right?
And I derived an expression for the ratio of the number of moles in the gas and liquid phases.
Because the idea here is, let's say you're working your way up on the curve.
If you just barely get to the coexistence curve, of course then you could see how much material you've got in the two phases.
But there's very little material still.
You've still got almost everything in the gas phase when you first reach here.
And as you work your way up in pressure, you'd have more and more liquid.
Of course, if you were to just keep going, you'd get into the pure liquid phase.
Typically, though, you might stop somewhere in the middle and have some reasonable amount of material in both phases, and you want to find out the composition in each phase.
And also, want to know how much material there is in each phase.
So the result that I derived was n gas at two over n liquid at two.
The ratio of total moles in the gas to the liquid.
What I derived was xA of two minus yA of one.
Over yA of one.
Minus yA of two, and I just want to, in some sense, finish up by making the obvious substitution, which is, of course these are mole fractions.
So I can just write xA is one minus xB.
And yA is one minus yB.
And I just want to put this in the terms that are written here in the phase diagram.
When I got to this result I pointed out that therefore you can use the lever rule.
And I just want to make that a little bit more explicit.
So using that relation, then we can see that ng of two over nl of two, total number of moles in the gas to the liquid is, now I'm going to substitute in to get yB of one minus xB of two over yB of two minus yB of one.
So all I've done here is substitute in these expressions.
And now I'm going to just switch both of these.
And so I'm going to multiply both sides by negative one.
So I can write xB of two minus yB of one over yB of one minus yB of two.
So that's the result that I want.
Because if I look at the two segments of this line, on either side of the pressure value I've reached, then what I see, of course, is that over here I've got xB of two minus yB of one.
That's this.
And this part is yB of one minus yB of two.
That is, it's this part.
And that's here.
And so that's the point, is that the ratio of moles in the gas and liquid phases is just given by the two segments of this on either side.
And so what this allows us to see very simply is alright, I'm going to raise the pressure at some point.
And depending on how much material I feel like I need to collect in the new phase, I can always determine that simply by reading off the phase diagram what the ratio of these two segment lengths is.
Now what I want to do.
What what we've done mostly is just go through these phase diagrams, see how to read them.
How to follow events on them as you change pressure in a diagram like this.
Or change temperature in diagrams like I also showed last time.
Now what I'd like to do is go a little bit further and just look at expressions for the chemical potential and the free energies.
So that we can, a little more quantitatively, see what's going on.
So let's just see what's happening.
So let's look at the chemical potentials.
In ideal liquid mixtures.
So everything is derived from the fact that when we have any of the constituents in both phases, the chemical potential must be equal in both phases.
Right?
So, we can write for A, mu A of the liquid at some temperature and pressure must equal chemical potential of A for the gas.
That's the partial pressure.
So if we have an ideal gas, and certainly if we're going to assume an ideal liquid mixture, we can safely assume that it's an ideal gas above it.
Then we can write mu A in the gas.
Is just mu A naught.
That's a function of the temperature plus RT log of pA over p0.
Nothing new here, this is just our expression for the chemical potential in the gas, with reference to a standard potential.
Usually one bar.
At whatever the temperature is.
And, of course, this is how it varies as the partial pressure of A in the gas phase varies.
So then we can just write our expression for the liquid.
At our temperature and pressure, it's given by mu naught in the gas phase.
Plus RT log pA over p0.
So it's the same thing.
Because I'm just taking advantage of this equality.
But of course, this is an obvious step, having written this.
But is a super important step.
That's what allows us to do this treatment in such a straightforward way.
We know a lot about the chemical potential of something in the gas phase.
Since the gas and liquid are in equilibrium, therefore we know the chemical potential in the liquid phase too.
So we can rewrite this by recognizing that if we just go to the limit where we only have pure liquid A, so pure liquid A.
Well, in that case, mu A naught or sorry, mu A in the liquid phase, mu A star is mu A naught in the gas phase.
Plus RT log pA over p0.
And so for the mixture, now all I'm going to do is just add and subtract terms.
So we can write mu A in the liquid at T and p, right?
So this is in the case of the limit of the pure liquid.
Now we're going to the mixture.
So it's just mu A naught plus RT log pA star over p0 minus RT log of pA star over p0.
Wait a minute, lost a term.
Plus RT log pA over p0.
And all I want to do now is combine terms.
To write mu A star liquid temperature and pressure.
Plus RT log of pA over pA star.
So this is a very convenient form for it.
And then, of course, we have an expression for pA, right?
From Raoult's law.
pA is just the mole fraction xA times pA star.
That's just true for the ideal mixture.
So now, finally, we can write that mu A in the liquid at T and p is just given by mu A star.
So that's for the pure liquid at that temperature and pressure.
Plus RT log of the mole fraction of A.
So that's a very simple expression for the chemical potential of A.
And of course the analogous expression will hold for any of the constituents.
In an ideal liquid mixture.
By the way, it's convenient because it looks just like the chemical potential in a mixture of ideal gases.
Except that we have liquid instead of gases, right?
Now, of course, since this is a mole fraction, it's always between zero and one.
That means this is always a negative number.
So what that means is that the chemical potential in the solution is always lower than the chemical potential of the pure liquid.
Very important result.
So it has all sorts of implications that we'll see.
One of them is osmotic pressure.
It means that if I have a biological cell, or some container with a membrane through which one of the constituents might pass.
So in other words, the let's say, component A, maybe it's just water.
It can pass freely, let's say, from the outside to the inside of the cell, or from one side to another of a membrane.
And on one side I've just got pure water.
And on the other I've got saline solution or whatever is in the cells, right?
What's the water going to do in that situation?
What's going to happen?
Anybody know?
So I've got, I just take fresh cells and plunk them into pure water solution.
What happens?
Yeah.
They burst.
Water rushes in.
Because the chemical potential is lower inside.
It's always lower in solution than outside.
So water rushes in, and the membrane expands.
Now, if the membrane is strong enough, at some point it may not burst.
And the pressure might start to go up.
Let's say the membrane is strong enough to resist and not burst.
Eventually, the water won't keep going in indefinitely.
That's because it won't be at the same temperature and pressure any more.
In particular, the pressure will have changed.
So you can build up some high pressure, what's called osmotic pressure.
Because of the fact that at the same pressure the chemical potential of the water's lower inside the cell or inside the enclosure with the membrane.
So water will keep filling.
And at some point the chemical potentials will equalize because of the change in pressure.
And at that point there'll be an equilibrium established.
We'll see that quantitatively a little bit later.
OK.
The other thing to notice is, this is familiar from the expression for a gas mixture.
What drives the gas mixture?
Remember back when we discussed mixing of gases and the fact that they would mix at all.
What makes that happen?
What's driving it?
Yeah, it's entropy, right?
And that's what's happening here too.
Of course, in the case of the ideal liquid mixture, there's no energetic interaction.
The molecules are non-interacting in this case.
But of course, entropy is going to want them to mix.
And that's what's resulting in the decrease in chemical potential.
Let's see that a little more explicitly by just calculating out the free energy change of mixing.
So, delta G of mixing.
So we're going to start with two separated liquids.
And then we'll remove the barrier.
And we'll have the two mixed together.
So of course, the free energy in either case is just the sum of the number of moles of each times the chemical potential of each.
We have expressions for that.
So starting G1 in this case.
It's n of A, number of moles of A.
Times the chemical potential.
So it's xA mu A star in the liquid.
Plus nB xB B mu B star of the liquid.
Here, G of two.
After we let them mix.
Then it's n -- wait a minute.
It's just n. I think I've got that wrong in the notes also.
Right, it's just a number of moles times the chemical potential in each case.
So it's the number of moles times xA.
Times mu A in the mixture.
Plus n xB mu B in the mixture.
But we've just figured it out our expressions for mu A and mu B, our chemical potentials of each constituent in the ideal liquid mixture.
So this is just n xA.
And here's our expression.
And of course these are going to cancel.
Let's write it out.
So it's mu A star plus RT log xA.
And then, mu B star plus RT log xB.
So our delta G of mixing is just the difference between these two.
So it's n RT xA log xA plus xB log xB.
Where have you seen that before?
Yeah.
The same thing that you had for the delta G of mixing for an ideal gas mixture.
Why?
Because we're not accounting for any interactions between the molecules in either of the phases.
And if the molecules aren't interacting, it's all entropy.
And the entropy term has the same form in either case.
In microscopic terms, it's just measuring the fact that there's more disorder in the mixture than in the pure liquids.
Just the same it was in the gas phase.
OK. We can see this even more explicitly if we just recall that G is V dp minus S dT.
So we can just explicitly calculate the entropy of mixing.
Delta S of mixing is just the partial of delta G of mixing.
Negative partial.
With respect to temperature.
At constant pressure.
So it's just minus n R xA log xA plus xB log xB.
So that's our entropy of mixing.
We can calculate our enthalpy of mixing.
Just delta G of mixing.
Plus T delta S of mixing.
But it's immediately apparent that these are just going to cancel when we multiply this by T. So there's no enthalpy of mixing.
Just as we expect when there are no energetic terms involved.
It's all entropy that's driving the mixture.
One more detail, it's straightforward to see that there's no volume change.
That is, if we take the derivative of delta G of mixing the partial derivative with respect to pressure, at constant temperature, of course, there's no explicit pressure dependence.
This is zero.
So again, in the ideal liquid mixture case, their molecules aren't interacting.
So there's no reason, when I open that barrier, that the amount of volume they occupy altogether is going to change.
Any questions, so far?
OK. Then, let's move on to non-ideal solutions.
By the way, just to return briefly to this topic of osmotic pressure, I just want to emphasize that result didn't need any kind of energy of mixing, either, right?
Just from the entropy term you would burst the cell or do whatever.
You end up with a larger pressure.
So in other words, if you have a situation where, here is A, pure liquid.
And here is A plus B liquid, so mu A star is greater than mu A.
We know that in all cases at the same pressure, the chemical potential of the mixture is lower than the chemical potential of the pure liquid.
What's going to happen?
Well, A is going to, oh, sorry, A over here is going to rush in.
It's going to get through this, if I've got a semi-permeable membrane, it's permeable to A but not to B.
Common situation, of course.
That's certainly the case for the cell membrane.
Water may flow easily in through the membrane, but all of the stuff, all the salt and everything else that's inside the cell generally won't.
So of course A then flows through this, what's called a semi-permeable membrane.
And you'll wind up with either a burst membrane, or something that looks like this.
Where now there is additional pressure here.
And you'll have some different mole fraction than you started with before.
Now let's go over to non-ideal solutions.
So let's just think microscopically for a moment about how this is going to work.
You know, normally, always, there are interactions between the molecules and the liquid.
The liquid is a condensed phase.
The molecules are in immediate proximity to each other.
So it's very different from the gas phase, where it can be a pretty realistic approximation to say, well, the molecules are essentially non-interacting.
The ideal gas law may turn out to be a very good approximation.
In the liquid, really, it's never the case that you don't have interactions.
So if we just sketch that.
If we start with a bunch of molecules in A, they're interacting.
So there's some interaction energy.
I'll call it AA, it's the interaction between two molecules of A.
In most cases it'll be less than zero.
That is, there'd be a weak attraction.
Same thing between molecules of B.
So that's my situation in the separated liquids.
I've got molecules in each container.
They're interacting.
There's some energy associated with that.
So here's the pure separated liquids.
And now, I'll open up the barrier and let the liquids mix.
And now suddenly, of course, A and B are going to interact with each other as well as other molecules of their own kind.
So now, suddenly, there's going to be some interaction energy.
uAB.
And very simplistically, we can envision that for some of the molecules, essentially, there would be exchange of this sort.
So, one neighbor of this molecule and one neighbor of this molecule will wind up exchanged for some pairs of molecules with the unlike species.
And essentially, likes interactions will be replaced by unlike interactions.
So there's some now change in energy involved.
There's a delta u.
And as outlined in this simple picture, it's just 2 uAB minus uAA plus uBB.
Real liquids are very complicated.
For liquids with relatively simple non-directional interactions, things like organic liquids that might interact van der Waal's interactions.
This might be a reasonable starting point.
And in general, if we're going to deal with relatively small deviations from the ideal liquid mixture, the ideal solution, then we can start with a model like this.
And this difference is what's going to determine how far the liquid mixture will deviate from the ideal solution case.
Of course, we could always just write a term like this.
It may or may not be easily expressed in this sort of way.
So let's just think about what the possibilities are.
Of course, simply, there are two.
The sign could be positive or negative.
And what that means is, you could have situations where it's positive because basically there's an energy of mixing now in these cases.
If that's not zero, unlike the ideal case, now unlike before, where we just had entropy driving the mixture, now there's an energy of mixing.
Could be positive or negative.
If it's positive, that means energetically speaking, the mixture is unfavorable.
In other words, the intermolecular interactions between like molecules are more favorable then the interactions between the unlike molecules, in that case.
So, delta u is greater than zero.
And our delta H of mixing is going to be close to delta u of mixing.
That is, we're going to figure that delta pV is not going to be something large.
And in that case, we can say delta G of mixing is 1/4 n. And that's only because there are 4 species involved here.
Times delta u.
In other words, I'm multiplying this molecular delta u.
This is the change in energy for this collection of four molecules.
Or of four molecular pairs.
Plus the other stuff that we've seen for the ideal case.
That is, the entropy term, of course, is still there.
xA log xA plus xB log xB.
And because we're treating the case where this is positive, that means this is bigger than in the ideal solution case.
So of course there are lots of examples of each of these.
In fact, this is the more common deviation.
So, and lots of examples.
An easy one is acetone.
And carbon disulfide.
So, if we look at that, and just see what the phase diagram is going to look like, it's the following.
So let's start with the ideal case.
So I'm going to make B the more volatile component.
That is, B is going to be CS2.
So here's p star CS2, which I'll call p star B.
And somewhere down here will be p star for A, that is, p star for acetone.
And this is going to be the mole fraction of CS2.
Or xB.
So let's start with the ideal case.
Not steep enough.
Pretty good.
OK. And now let's look at what's going to happen in the real mixture.
Well, because we've got a positive deviation, what's going to happen is the mixing is unfavorable.
The molecules aren't that happy any more about being in solution.
So what that means is, relative to the ideal solution, they'd rather go up into the gas phase.
They don't like being in the mixture.
Compared to what they would feel in the ideal solution, where there are no interactions.
Because the interactions between unlike molecules are unfavorable.
So what's going to happen, then, is the pressure is higher.
You have deviations from the ideal case, in the total pressure.
And in each of the individual partial pressures.
That's the total pressure.
pA plus pB.
In the ideal case it's this straight line.
In the non-ideal case, though, it's bigger than the straight line.
Because more stuff wants to get up into the gas phase.
And it's the same for each individual component.
So here's pA. And there's pB.
Any questions?
OK, so it's obvious just from looking at this that each of the partial pressures is bigger than it is in the ideal case.
And so, so is the total pressure.
So in other words, if I say what's the partial pressure of CS2, well, since it's not the simple ideal case, actually I don't know what it is in general.
It's not easy to calculate a priori what it's going to do all the way across the phase diagram for any mole fraction.
What I do know, though, is it's bigger than it would be for the ideal solution case.
In other words, it's bigger than x CS2 times p star CS2, the pressure over the pure liquid CS2.
Same for acetone.
It's bigger than x acetone p star acetone.
And since both partial pressures are bigger, the total pressure has to be bigger.
And of course, that's something you can read right off the diagram as well.
In other words, the total pressure is bigger than in the ideal solution case.
Any questions?
OK.
So the negative case, of course, is exactly the opposite.
So I'll just go through it quickly.
So, remember positive deviations came, let's put this back up there.
That came from the case where the unlike intermolecular interactions are not as favorable.
There's less attraction, more repulsion than in this case.
Of course, there are plenty of cases of the opposite sort.
Where the unlike intermolecular interactions are attractive.
More than the interactions between like molecules.
And in that case, you have the opposite result.
So you can have negative deviations.
In other words, delta u is less than zero.
Lots of examples.
I've got one in the notes, it's just acetone and chloroform.
So if we get rid of the carbon disulfide and put CHCl3 in there, that's more favorable.
Because there's weak hydrogen bonding between the unlike species.
And not between the like species.
So what happens is, you have acetone, it's this.
And you have chloroform.
And there's weak hydrogen bonding between them.
There's an attraction there.
That doesn't exist between either case of the like molecules.
So now there's stronger attraction between them than there is between molecules in either of the pure liquid cases.
So, of course, you know that the opposite result is going to happen.
In this case, now you mix them together, relative to the ideal solution where there are no interactions, now they want to be together.
They're clinging to each other through these hydrogen bonds.
What's that mean for the pressure up in the gas phase, in the solution?
They don't want to go there any more.
Stuff in the gas phase now wants to condense down into the liquid.
Because there they have favorable interactions.
So that's, of course, what'll happen.
And so then you just see the opposite sort of deviation from what I just illustrated before.
So then if we look at p star CHCl3, where are these lines going?
Well, may have been more artistic.
But I'm afraid I'm going to have to go with reality.
p star for acetone.
There's chloroform.
And now our deviation is going to be in the opposite direction.
So all the, both of the partial pressures and the total pressure are going to go lower than indicated here.
So p total is now less than, for the real case, is less than p total for the ideal case.
In the situation where you have attractive interactions between the unlike molecules.
Any questions?
OK. Now, in general, this is complicated.
To describe quantitatively throughout the entire phase diagram.
The interactions are complicated.
Lots of times, molecules interact with more than one neighbor.
So in general, we don't have a simple analytical expression for what the pressure is going to do as a function of mole fraction all the way from zero to one.
Of course, we have it in very simple form for the ideal solution.
Because everything's linear.
So it's very simple.
These are replaced by equalities and we know everything.
Now we don't.
But we can at least make some progress by looking at the limiting cases.
Where you have just very dilute solutions.
So in that case, you can anticipate what's going to happen.
So let's say we mix A and B.
And one of them is in very dilute concentration.
For the other one, for B, most molecules of B are just going to see themselves.
They hardly know that you've mixed any molecules of A in there.
For them, it's pretty much an ideal solution.
They think they're in an ideal solution.
Which is to say there's an entropy change in mixing.
But hardly any of them experience interactions with the unlike molecules.
For the dilute molecules, there is a change.
You can't use Raoult's law any more.
But at least we can look of the slope of the curve and say, OK, at least in the dilute solution case for a little while it's going to be linear.
We can understand it.
We can tabulate it.
So let's look at how that works.
Alright.
That limit is something that has a name, that's called the ideal dilute solution.
Not to be confused with the ideal solution.
Where both constituents follow Raoult's law, as we've seen before.
So ideal dilute solution.
We're going to just look at these limits.
So, if we start with the ideal case, so let's imagine we're going to look at carbon disulfide again.
Here's p star CS2, which is our pB star.
And this is x CS2, or xB.
OK, well, what happens if we've got almost pure carbon disulfide.
Well, what's going to happen is in this limit, like I was mentioning, up here the CS2 pretty much sees other CS2.
It thinks it's in an ideal solution.
It's going to look like that.
Now, let's go to the other limit.
So this is the limit x CS2 equals xB approaches one.
Raoult's law.
Two, the other limit.
x CS2 approaches zero.
We're down here.
So it's definitely not going to be the same.
Because now, every molecule, every CS2 molecule, is completely surrounded by acetone.
So they all feel the energy of mixing.
Because there are hardly any of them there.
So every one of them is, it's very rare for one to find a neighbor of its own kind.
And this is a case where there's a positive deviation from ideality.
That is, the interactions are less favorable between unlike species.
So, it's up here.
Doesn't like that.
It says, get me out here, I'd much rather be in the gas phase, compared to the ideal case, then surrounded by all these acetone molecules, right?
So that's what'll happen.
And now it's up to the artist.
But in some way there's going to be a set of values in between.
And the ideal dilute solution by itself isn't going to tell us about that.
It's telling us about the two limits.
Now, if we follow this, and I can see I'm going to be in trouble.
But I can do a little creative bending of the line here.
If I follow this slope up to here, it'll intercept somewhere.
That has a name.
It's called the Henry's law constant.
I can write it as K CS2, or KB.
So the point is, that at least for some range of concentrations, the variation is linear.
And what that means is, if I know this Henry's law constant, for CS2 mixed with acetone specifically, it would be different.
If it's mixed with something else, right?
Other kinds of interactions with different energies will have a different slope.
But if I know that number, it's still useful.
Because at least for that pair of constituents, I know the dependents for some range of concentrations.
So this is Henry's law.
p CS -- Oops, CS2 is x CS2 times K CS2.
Not quite as convenient as Raoult's law.
That number, I don't need to know what the other constituent is.
It's just a property of CS2.
It depends on the temperature, but not the other member of the mixture.
This one does, but still there's at least a substantial number of these things tabulated.
Especially for important mixtures.
So it becomes useful for dilute solutions.
And there's an awful lot of chemistry.
There's an awful lot of biology that happens in relatively dilute solutions.
OK.
I'll leave it at this point.
Next time we'll look at the total phase diagram, just like we did last time looking at both the liquid and gas coexistence curves, and seeing what the behavior is as a function of mole fractions.
And we'll talk about distillation and so forth in non-ideal liquids.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time you learned about the ideal solution limits with Henry's law and Raoult's law.
And I just want to start with one reminder of what this is all about.
So let's say that you have a mixture of CS2 and acetone.
So this is a review of what you saw last time.
Acetone like this, CH3, CH3.
And if you take the CS2 as the solvent and the acetone as a solute, then when you plot as a function of the amount of CS2 here as a solvent, then you have the limit where the mole fraction of CS2 goes to one, meaning that CS is the solvent.
You expect Raoult to work if it's an ideal solution.
So that the vapor pressure of CS2 is related to the mole fraction of CS2 times the vapor pressure of CS2 if it was pure.
So you're sitting here somewhere.
This is the vapor pressure of CS2 pure and CS2 goes from zero to one.
And you expect Raoult to work.
So let's do Raoult.
Like this.
So in this area, if I were to plot the real curve, I would expect it to start out just like Raoult.
And then there's going to be some deviation.
Then if I take the other limit, where the mole fraction of CS2 goes to zero.
So now the CS2 is the solute.
And the acetone is the solvent.
what happens for this particular mixture is that Henry's law is no longer valid.
Raoult's law is not valid here.
Instead, you have to look at Henry's law.
And Henry's law tells you that for the solute now, not the solvent, so the CS2 here is the solute, very small amount of it.
Very small mole fraction.
That the vapor pressure of the solute is related to the mole fraction in the solution.
But instead of having the vapor pressure of the pure solute, you have another number.
Another constant.
Which we call K. Which is a Henry's law constant.
And this constant depends on what else in the solution.
It depends on the fact that you have acetone in there.
You would have a different constant if the acetone were replaced with chloroform, let's say.
So this constant here cares what the solvent is.
And it cares what the temperature is, also.
There's a lot of stuff that's built into this constant here.
It's a phinemelogical constant.
And in the case of the mixture of CS2 and acetone, the deviation from Raoult's law is positive.
Meaning that the Henry's law constant is greater than the vapor pressure of the pure CS2.
So that means that the molecules of CS2 would rather go in the gas phase.
If there's acetone around.
That means they don't like to mix so much.
They're not happy with each other.
So the CS2 molecules would rather escape what would normally be a Raoult's law type of solution to go in the gas phase away from being close to the acetone.
And as a result you get this positive deviation.
You get a higher pressure than you expect from Raoult's law.
So that the slope here This slope here is K CS2.
And that slope here is p star CS2.
This is a positive deviation.
And you saw an example where you if you mix chloroform and acetone, you get a negative deviation.
Chloroform and acetone are happy together because of the hydrogen bonding between the oxygen here and the hydrogen in chloroform.
And you get a negative deviation.
So let's start by doing an example with numbers.
Of how this might come about.
Any questions on this first?
I always got confused with Henry's law and Raoult's law.
And I always got confused as to when I was supposed to use Henry's law and when I was supposed to use Raoult's law.
Very confusing to me, initially.
The thing to remember is that if you're talking about something which in a very small amount, just the solute, like, well I'm going to give you an example which is a bad example, like sugar and water.
Where sugar is clearly the solute, but sugar is not volatile clearly this is a bad example.
But something which in a very small concentration, then Henry's law is going to apply.
Raoult's law is only going to apply for the solvent.
When you have a lot of stuff around.
On this side here is Raoult's law.
On this side here is Henry's law.
Henry's law, the deviation's from Raoult's law, are going to happen where the molecules in the solution mostly see molecules that are not like themselves.
If they mostly see molecules like themselves, since it's an ideal solution, then they don't know that there's other stuff around.
In this case here, the CS2 molecules don't really know.
Most of them don't know that there's any acetone around.
They couldn't care less.
So it obeys an ideal solution.
Here, they don't see any other CS2 molecules.
They mostly see acetone.
And so Raoult's law is going to fall apart.
Because Raoult's law assumed that you have a single component.
Or mostly one component.
OK. Let's do an example.
So in this case here, we're going to work the example out here.
So let's take two components, A and B.
Where B is, and let's look at something which is a dilute solution and non-ideal.
So there's going to be some deviation from Raoult's law.
And what we're told is at 50 degrees Celsius, that the vapor pressure of pure A is equal to 0.67 bar.
And the vapor pressure of pure B is equal to 1.2 bar.
So B is clearly the more volatile.
And we're also told, then, that at 50 degrees C we have a molar fraction of A in the liquid phase is 0.9.
And the molar fraction of B in the liquid phase is 0.1.
And the total pressure above the solution is 0.7 bar.
And eventually what we're going to try to find out is, we're going to change these concentrations.
Are we going to try to figure out how the total pressure above the solution changes.
That's maybe the goal of the problem.
When you change this to, eventually we'll change it to 0.95 and 0.05, and we'll want to know how does that change the total pressure.
So immediately we see that this is a solvent.
This is a solute.
This is volatile.
So, we're close to where this is zero.
So if we're going to apply any, sort of, Raoult or Henry's law problem to this, this is probably going to be Henry's law for B and Raoult's law for A.
We're going to be close to this area here for B.
We're going to be close to this area for A. OK.
So ah the first question is, what's the composition in the gas phase.
What is y sub B.
And it looks like this particular set of notes doesn't have the calculations.
I'm going to go to my other set of notes here that has them.
So, because I'm going to make a mistake if I go through it without my notes, I'm going to leave it up to you to do part a.
That's a good way out of making a mess.
I'm just going to tell you the answer.
Because the actual, the interesting part, is part b of the problem.
Part a is the easy part.
So this is something that you can do on your own.
So yB is equal to 0.139.
Is going to be answer to part b, given these two molar fractions and the total pressure.
So now, what we're going to do is, we're going to change the molar fractions and we're going to find out how that changes the total pressure.
So, second part is we're still at 50 degrees Celsius.
And we're going to change the molar fraction of A to 0.95.
And the molar fraction of B to 0.05.
And we're going to ask, what is p total now.
So we write what we know.
Which is that we expect p total to be the sum of the partial pressures of A and B.
A is the solvent, B is the solute.
It's a dilute non-ideal solution.
pA is going to be proportional to Raoult's law, because it's a solvent.
It's in large quantities.
So it's going to be xA pA star.
And xB is going, and pB is going to be proportional to the mole fraction again.
But because it's a solute, it's seen most the other molecules of A around, it's not necessarily going to be obeying Raoult's law.
It's going to obey Henry's law.
It's going to be close to the zero point.
There's going to be a deviation because of the interactions between A and B.
Which dominate now.
So instead of pA star, we have KB here, where this is the Henry's law constant.
Which depends on the fact that A is around.
We get a different constant if we have a different solvent around.
So we know this.
We know this because that was given to us at the beginning.
But we don't know this here.
We don't know what the Henry's law constant here is.
But we can get it.
We can get it from part a here.
Because we have all the information we need there.
So from part a, we already have a mixture where we're given the total pressure.
So from part a, KB is pB over xB.
And we calculated what yB is over there.
On part a.
So we know what the partial pressure of B is.
It's the molar fraction of B in the gas phase times the total pressure.
That's Dalton's law.
Then we divide by xB.
And then we plug in the number 0.139 times the total pressure up there is 0.7.
Divided by 0.1.
And we get that this is 0.973 bar.
So now we can plug this in to our equation here.
We know xA, we know xB, we know Raoult's law, the 0.67 pA star.
We know KB here.
We plug in all the numbers, and we get that the total pressure is now 0.685 bar.
This is a pretty typical example of a problem that combines Raoult's law, Henry's law and Dalton's law together.
And keeping track of these laws takes a little bit of thinking.
And it's very, very common on problem sets or exams to mix these up.
So it's good to go through examples like this and really keep track of where each law applies.
Dalton's law applies in the gas phase.
It's partial pressure in the gas phase.
Henry's law is the pressure.
If you know what's happening in the liquid phase, for a solute.
Raoult's law is the partial pressure in the gas phase, if you know what's happening in the liquid phase.
So Raoult and Henry tell you something about the pressure if you know the liquid.
Dalton only deals with what's happening in the gas phase.
OK, any questions on the problem?
Alright.
So we did this problem here, and now let's draw the phase diagram for our solute here, or for B, rather.
Zero to one.
And let's take a vote here.
So if Raoult's law worked for B, for the mixture of A and B, then we would have pB star here.
And everything would be great.
But Raoult's law doesn't work.
Close to molar fraction of zero.
Instead, Henry's law.
There's a deviation, due to the interactions between the molecules A and B here.
So we have two choices for the deviation.
We can have a positive deviation, or we could have a negative deviation.
Positive deviation, if the molecules of B would rather be in the gas phase, than be in the mixture in the liquid phase.
Meaning they don't like molecules of A.
They don't like to be mixed with molecules of A.
The energy of interaction is not favorable.
The enthalpy is not favorable, for mixing A and B.
In this case here, the enthalpy is favorable for mixing A and B.
So in this problem here, which do you think it is?
The blue one or the green one.
Let's raise hands here.
How many people think it's the blue one here.
Anybody else?
The right answer, we have four people.
How many people think it's the green one?
Four.
Five.
Six.
You didn't vote twice, did you?
Vote early, vote often, right?
Mayor Daley, the old Mayor Daley in Chicago.
And if you know somebody who's dead, vote for them.
OK, so this is the, I've got about five people here.
And a bunch of people didn't vote.
So it's a tie.
It's basically a tie here.
I think you should think about this.
Maybe talk to each other a little bit.
See if you can figure out, given all the numbers that are in this problem here, you know what, the pure pressure on the solvent is.
We change concentrations here.
Got Henry's law constant out.
And Henry's law constant is going to tell you whether or not that enthalpy of interaction is favorable or not favorable.
It's going to be a positive deviation or a negative deviation.
OK, shall we vote again.
How many people say it's the blue one?
How many people say it's the green one.
Thank you.
Alright.
Good.
Alright.
We have K sub B is less than p star, therefore it's a negative deviation.
The pressure is less than you'd expect.
The molecules like each other better than you expect.
There is a favorable enthalpy of interaction between A and B.
They like to interact.
It could be hydrogen bonding.
Could be a lot of different things.
OK, so now if I put the whole diagram together, which I shouldn't have erased here.
Right, so there's my diagram here again.
There's Raoult's law.
There's the deviation from Raoult's law here.
For one substance, let's say for B.
There's Raoult's law for A.
There's a deviation for A.
It could be like this.
And if I now add these two together, instead of having a diagram for, this is a pressure xB, yB diagram, pressure xB, yB diagram, instead of having Raoult's law sitting right here.
And then the yB part down here below, your usual sort of ideal solution phase diagram with the high pressure of the liquid on top and the gas on the bottom.
Now if you add the blue curve and the red curve together, you're not going to get a straight line like you did before.
When Raoult's law applied throughout.
Instead you're going to get a deviation from the straight line.
You're going to get a curve that's always below the straight line.
Because we have the negative deviation on both sides.
So you end up with something that's curved and deviates below.
And the same thing will happen for the bubble line.
It will also deviate below.
In some way.
Maybe not as much, maybe.
So this is what you usually see.
You see deviation of that whole phase diagram, like you're pushing down on it.
Or maybe it didn't pulled up.
If you have a positive deviation, then it's the opposite.
It gets, this is negative deviation.
And if you have a positive deviation, instead of having a usual phase diagram, you're going to get something that's looking like this.
Which is the Henry's law coming in.
Yes
[INAUDIBLE]
Yes
[INAUDIBLE]
Doesn't matter.
Good question, though.
OK, so this is what it usually looks like.
And everything you've learned so far can be applied to these diagrams here.
There's one exception, though, which is an important one.
Which is, sometimes, these non-ideal diagrams have a funny spot.
Which is that if I draw them, so there's my xB, yB, on the bottom.
Zero to one.
I have pressure here.
Sometimes, there's the dew line.
And the bubble line.
Sometimes, instead of having a nice separation between the bubble and the dew line, there's a special point which for some reason those two curves meet.
Non-ideality.
There's interaction between the molecules.
In the gas phase and the liquid phase.
This special point is an interesting one.
Because, well, if I'm away from this special point, let's say I'm sitting to the left of the special point, right here.
And I can distill this thing.
I can take my gas phase here.
And increase the pressure.
Say I'm sitting here in the mixture, gas phase mixture with x prime B, y prime B.
Sorry, x prime B.
And I go up and I increase the pressure.
Start making vapor.
Start making vapor, I can find out what the, I start making liquid.
I can find out where the composition in the liquid is by going over to the blue line here.
That tells me the composition in the liquid.
And reading off the composition of the liquid down here.
So this is actually the composition in the gas phase is yB prime.
yB prime is the gas phase.
I'm starting here.
I start to increase the pressure.
I begin the distillation process.
And I get a little bit of liquid out.
And I can read off the composition in the liquid.
xB prime.
And I know from this diagram that xB prime is less then yB prime.
So I'm enriching my liquid in A. I'm purifying my solution in A, right?
I can get pure A out eventually.
The fact of doing this, I can get pure A.
So I decrease the pressure, I get the pure gas phase of my new solution here.
And I increase the pressure.
And go over and make a little bit of liquid.
And capture that liquid.
And repeat the process over and over again.
And eventually I have pure A.
Now, suppose I'm sitting here at this special point.
And I'm starting with yB And let me call it a here.
yB of a. I'm sitting right here.
And I'm increasing the pressure.
I go all the way up here.
And I start making liquid.
.
What's the composition of liquid, compared to the composition of the gas phase?
What's y?
What's xB compared to yB of this point up here?
It's the same.
So I'm making liquid.
I got the same composition.
Doesn't help me.
If I make a gas again, I still have the same composition.
I can't distill.
I can't purify it.
So if I have this special mixture here, of A and B, at this point here, I form what's called an azeotrope.
And then you can't separate it out.
You can't separate out the mixtures into A and B.
You can't distill it at that point here.
Now, you can draw the same diagram in, instead of pressure, mole fraction, you can draw the same diagram in terms of temperature.
And the same thing happens.
You can have deviations from, this is the extra figures that are on the bottom page of the notes from last time here.
That we've replaced here.
You can have a positive deviation.
With a positive azeotrope.
And your T p diagram, T x diagram.
Or you can have, depending on the interactions between the two molecules, you can have something that has a negative deviation.
And a negative azeotrope here.
So in this case here, this is a particularly awful situation.
Because there's your liquid phase down here.
There's the gas phase here.
And now, suppose that you're doing some sort of distillation.
So you start here in the liquid phase, with some composition.
You make a gas.
Let's say you're doing a fractional distillation of some sort.
Your gas now has a new mixture.
So this is the composition in the gas phase.
You take that gas phase mixture, and you condense it in your condenser.
In your fractional distillation condenser.
So now I've condensed it.
And I heat it up again.
I've condensed it.
So I'm here somewhere.
And cool it down so that I have a liquid.
And I condense it.
And slowly I move over to the azeotrope point.
And once I get there, I'm stuck.
And the same thing happens if I'm on the other side.
I'm on this point here, I keep my solution, my mixture.
I make a gas, I read off the composition of the gas phase here.
I take my gas phase composition out and I re-liquefy it.
So I'm here, down here somewhere, again.
And I heat it up again.
And and I move to the azeotrope point.
That's what happens if you have a mixture of benzene and water, for instance.
You can't distill it.
You work your way down to this point here.
You can't use fractional distillation until you've separated it out.
So this is a real life problem.
And you have to do things like change the pressure.
Do your distillation under high pressure or low pressure.
To try to get this point to separate out.
So you can get past this point.
Or to try to get the deviation to become of the opposite sign.
Where you can start to do some distillation.
Any questions on azeotropes, before we get to colligative properties?
If you're doing organic chemistry, and you're trying to purify things, this is a real pain, right?
Pain in the neck.
So you've got to know, which things form azeotropes.
So we're going to start colligative properties.
There are four colligative properties, and you already know what they are.
And you've even seen the math behind them.
And in fact, you've seen the math for one of them already pretty clearly.
And these colligative properties are properties that you use all the time in your day to day life.
At least, three of them.
OK what are the four colligative properties?
There is vapor pressure lowering.
If you have a mixture and the vapor pressure of the mixture is less than the vapor pressure of the pure material, there's the boiling point elevation.
If you add a large amount of insoluble - not insoluble, non-volatile solute to a solution, you will depress, you will increase the boiling point of that solution like the example of the salt and water that I gave.
So if you need a lot of salt to increase the boiling point of water.
There's the freezing point depression.
That's what we use to melt ice in the winter.
By adding salt to it.
Then there's the osmotic pressure.
Which is the example of the freshwater fish in salt water, or the saltwater fish in fresh water.
Or the cells bursting when you put them in.
OK, so these colligative properties all come from the fact if I have a mixture, the chemical potential of a molecule in a mixture, in the liquid phase here, is less than the chemical potential of the pure guy.
Where A is the solvent, and B is some solute.
So this is always true if your number of moles of A is much larger than number of moles of B.
Mix a little bit of B in there, and the chemical potential of A in the liquid is going to come down.
Because of the mixing process.
Because of the entropy of mixing.
And that's going to give rise to these properties.
And, to do these, one more thing before we start.
We're going to use a special concentration for these guys.
We're going to use, call something molality.
Molality.
Molality is the number of moles of solute divided by kilograms of solvent.
So it's moles per weight of solvent.
As opposed to moles per liter, which is something that we're more used to.
Moles per kilograms of solvent.
It's pretty close to moles for liter for water.
Because a gram of water is a milliliter, roughly.
The density of water's close to one.
So let's start on these things.
So we've already seen this.
You've already seen vapor pressure lowering.
It comes from Raoult's law.
If I look at the change in the vapor pressure of A, pA minus pA star.
pA is equal to xA pA star.
xA pA star, that's Raoult's law.
Minus pA star, that's equal to one minus xA.
Minus pA star.
Which is minus xB pA star.
Which is less than zero.
So, that means that the vapor pressure above the solution is lowered.
So I should have said vapor pressure lowering.
And we also drew a phase diagram to show how this vapor pressure lowering was going to imply freezing point depression and boiling point depression.
So now let's go on and look at the freezing point depression.
Number two.
We're going to find that the change in the boiling point is going to be some constant K sub b, which is not to be confused with Henry's law's constant.
It's completely different.
This is a colligative property.
Times the molality of B.
That's going to be the answer we're going to get.
Amount of temperature change depends linearly on the molality of B, with a constant that depends on what the mixture is.
And what the pressure is.
Whenever we derive anything connected to phase transitions, we usually always start with the fact that the chemical potential changes.
And it has to be equal at equilibrium.
So that's what we're going to do.
We're going to start with the chemical potential equal of the liquid phase is equal to the chemical potential of the gas phase.
For the solvent at equilibrium.
And the liquid phase obeys Raoult's law.
So it's going to be the chemical potential of the pure material.
The pure liquid.
Plus RT log xA.
So this is where the mixing of the solute comes in.
And on the gas phase side, mu A, it's going to be the same thing as the chemical potential of the pure A.
Because we're going to take B to be non-volatile.
So the only thing in the gas phase is A.
Same chemical potential as if it were pure.
Because it is pure.
So now we solve for log xA is equal to one over RT mu A star in the gas phase minus mu A star in the liquid phase.
And this is just the Gibbs free energy per mole of A in the gas phase.
The Gibbs free energy per mole of A in the liquid phase.
So we can replace this with the change in Gibbs free energy from going from gas to liquid.
Per mole, which is just the Gibbs free energy of vaporization.
We're going from liquid to gas.
And I probably have a negative sign somewhere.
No this is right, this is the final state.
This is the initial state.
Divided by RT.
OK. That's not really what we want, though.
What we want is delta T. Here I've got some complicated equation that has T in it, but not delta T. But before we do that, before we transfer and try to get a delta T out of this equation, let's see if we can do any sort of simplifications here.
I don't like to carry logs around.
There are no logs around here.
So let's see if we can get rid of that log.
What is log xA?
Log xA, well, xA is close to one.
It's connected to xB.
And everything in here is connected to B.
There's a molality of B here.
The concentration of A doesn't come into play here.
I don't want it there.
So let me replace it with B. one minus xB.
xB is a small number.
It's a solute, it's in small amount.
We're close to being a pure solvent here.
So one minus something very small.
The log of that.
Well if you've worked with Taylor's expansions.
You probably know that that's close to minus xB.
The log of one minus x where x goes to zero is close to minus x.
So that's minus xB.
Now, minus xB, in terms of the moles, is the number of moles of B divided by the number of moles of B plus the number of moles of A, the total number of moles.
The number of moles of B is very small, compared to the number of moles of A.
So I can get rid of this.
So, and there's a minus sign here.
So this is approximately minus nB divided by nA.
Now, the only I can do is, I'm trying to get at molality here.
I said we're going to use molality.
Got moles.
I don't have kilograms of solvent here.
I have moles of solvent.
I'm not so happy with that.
So, in order to get something that has kilograms out, let's multiply up and down by capital M where capital M is that total mass of A.
Basically the number of kilograms of solvent.
Total mass of A.
So now, this fraction here, that's the molality.
That's what I'm looking for.
So this is minus the molality.
Then I have the total mass of A divided by the number of moles of A.
Well, that's the molecular weight of it.
Total mass divided by the number of moles is the molecular weight.
So this is the molality of B times the molecular weight of A. OK.
So now I can plug this instead of log xA.
And I get minus nB times nA is equal to delta G of vaporization divided by RT.
Let's rearrange a little bit.
And mB is equal to to delta G vaporization divided by nA times RT, with a negative sign.
Well, I've eliminated the log at least, but I still don't have a delta T in there.
I don't have a delta T. How am I going to get a delta T in there?
Got to get the delta T. Everything's very small.
So one way to get a small change in temperature is to take a derivative with respect to temperature.
That's going to get a delta T in there.
So I'm going to take the derivative of both sides with respect to T. And delta T is going to come out that.
I'm going to take d nB / dT, constant pressure.
Minus one over mA times R d delta G over T. This is delta G of vaporization.
dT, constant pressure.
And I wish I'd made my boards a little bit better.
Because I have to cover things up.
OK, now I have this derivative of the Gibbs free energy divided by the temperature.
This turns out to be something that you've seen before.
It's called the Gibbs-Helmholtz equation.
And it relates the temperature change in the Gibbs free energy with the enthalpy change.
The Gibbs-Helmholtz equation is that the d delta G over T dT is equal to minus delta H over T. And you can get that from just writing fundamental equations about G and H. And turning the crank.
And we've done that before.
And I'll leave that as an exercise for you to go back and figure out what, you can go back to your textbook and look at, you know under the index.
Gibbs-Helmholtz, and figure out why this is the case.
So now, we have this equation here.
This is going to be nice, because it's going to get rid of the d/dT.
On the right-hand side.
And so now our equation becomes d mB / dT is equal to delta H vaporization divided by mA R T squared.
Now, I can rewrite this in a slightly different way.
This is delta m over delta T, basically.
Delta the change in number of moles.
I mean, the molality of B divided by the change in temperature.
I'm getting very close here.
There's a change in temperature here.
There's a change in molality here.
I can rewrite this as delta T, then, is equal to molecular weight of A times RT squared divided by delta H vaporization times delta mB.
Now, I'm making a very small change in B here.
Starting at zero.
Basically I'm taking pure A and I'm dumping in a little bit of B.
That's what I'm doing when I experiment.
I'm dumping in a little bit of B in my pure A, and I'm trying to figure out how the temperature is changing.
So I'm doing delta m sub B is my small amount of B that I'm adding to the solution.
Minus what I started out with, which was zero.
So delta mB is basically m sub B.
Right?
That's basically what it is.
So let's get rid of this delta here.
What else can I say?
Well, I can say that the temperature change that I'm looking at is also pretty small.
Compared to the absolute temperature.
This is a small change.
I'm only adding a small amount of B in there.
So this temperature up here is not going to change very much.
Let's take it to be the temperature of the boiling point of the pure material.
T sub B star, of the pure material.
OK And there's my delta T here.
So this is of the form delta T is equal to molecular weight of A.
Times R. Times the boiling point of the pure material squared.
Divided by delta H of vaporization of the pure solvent.
Times the molality of B.
And this combination of properties of A, the molecular weight of A, the boiling point of pure A.
The heat of vaporization of pure A, this is K sub b up here.
Which I covered up.
It's of the form delta T is equal to K sub b times mB.
And this turns out to be always greater than zero.
Now.
If you want to look at freezing point depression, it's really easy.
In your equations, it's the same thermodynamic picture.
All you need to do, in all your equations, you replace delta G of vaporization with minus delta G of freezing.
When we did a vaporization, we're going from liquid to gas.
We had delta G of vaporization.
When we're doing freezing point depression, we're going from liquid to solid.
And so we have to use minus delta G of fusion.
Minus delta H of fusion.
s to l would be the positive delta G of fusion.
So we're going from liquid to solid.
We have to replace delta G of vaporization with delta G of fusion.
We replace delta H of vaporization, where minus delta H of fusion.
Replace the boiling point with the freezing point.
And replace K sub b with K sub f. And the math is exactly identical.
It all starts from the chemical potentials being equal.
And turning the crank at equilibrium, being equal between the solid phase and the liquid phase.
And turning the crank.
The solid phase is like the gas phase.
It's a pure A, pretty much.
Because the molecules of B are stuck.
And A doesn't know that B's around, in the solid phase.
They know that they're around in the liquid phase because the fusion fast enough.
.
And then we can write, if we turn the crank, we get an equation that looks like this.
The freezing point depression is equal to minus mA R Tf star squared.
Divided by delta H of fusion, for mole.
Times the molality of B.
And this number is always less than zero.
There's the negative signs in here.
OK.
Always depressed.
Any questions about those freezing point depression and boiling point elevation.
Yes
[INAUDIBLE]
Why did I drop it here?
Because the small change that I'm making in nB is going from pure A, where nB is zero, to a very small amount of nB.
That's why I'm allowed to say this.
And then just replace delta nB with nB.
I'm comparing every, my reference point is the pure material.
I'm just adding a little bit to start with.
It's a good question.
Any other questions?
OK.
I didn't get to osmotic pressure.
You'll get to that on Friday.
And also start statistical mechanics.
And after osmotic pressure, you'll be done with thermo.
It'll be statistical mechanics and then kinetics.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time, under the tutelage of Professor Blendi, you started to see some of the consequences, the more immediate consequences, of some of the properties we've been discussing over the last few lectures of binary liquid mixtures.
And in particular he started showing you some of the colligative properties by which you see changes in the vapor pressure and the boiling point and so forth, that arise because you've got liquids in mixtures and this affects their chemical potentials.
And today I want to start by going through the fourth colligative property that's commonly seen, what called osmotic pressure.
And I mentioned this before.
Today we'll go through a quantitative calculation of its effects.
And then we'll start in on a different unit of the course.
We'll start in on statistical mechanics.
So let's continue with the colligative properties.
And osmotic pressure plays a super-important role, certainly in all of biology, because of course inside biological cells you have solutions.
Water is in there, along with a whole bunch of other components.
And so the chemical potential of the water is strongly affected by all the other constituents.
And this has profound effects on what's involved in maintaining an equilibrium between what goes on inside and outside of the cell.
And this is the case in all organisms.
And it's also exploited of course in all sorts of other contexts.
So what's happening with osmotic pressure is, you've got some situation where there's a semi-permeable membrane.
It allows one of the constituents but not others to pass through.
And typically that's water.
Although, of course, it could be anything.
So, in the lab the way this typically looks is something like this.
Here's your semi-permeable membrane.
Outside, we can imagine that there's pure liquid A.
Could be water, could be something else.
Inside, you've got A mixed with B, one or more other constituents.
And then there's some level of the liquid outside the container.
Of course, A can pass between, pass through, the membrane.
And so what ends up happening is, if you just look at the situation, what you immediately notice is that the liquid in here goes up higher than the liquid around it.
It's at a higher level.
It seems counterintuitive at first.
And in fact it can be enormously higher.
But that's what happens.
So there's extra height, h, of liquid inside the container relative to what's outside it.
And what that means is the pressure inside here must be higher, right?
Of course, the weight of this liquid is pushing down on what's down here.
So the pressure inside here is higher than the pressure outside.
But of course, from what you know already about liquid mixtures, this should in fact make sense.
Because what's happening is that if these things are at the same pressure, you know that the chemical potential of A is going to be lower in the mixture.
It's always lower in the mixture than it is in pure A.
So there's nothing to stop A from rushing in.
So A does rush in.
If the liquid level goes up, it raises the pressure down here.
But as the pressure rises, now the chemical potential changes and goes up.
So eventually, equilibrium is reestablished.
Because these aren't at the same pressure.
The additional pressure means the chemical potential of A goes up.
Of course, it goes down by virtue of being in the mixture.
And those two things will eventually reach an equilibrium where the chemical potential of A is the same everywhere, as it has to be.
So what we'll calculate now is, OK, how much does the pressure go up.
When that happens.
So let's take a look.
And so of course, there are all kinds of examples.
In a typical one, A would be water.
And B would be one or more sugars.
All sorts of other potential constituents also.
OK, so let's just consider the pressure at two places that are the same level.
The same height.
We'll label this one point alpha.
Let's put some color in here.
We'll have points alpha and beta.
And we'll make this distance l. So now let's just first calculate what the pressure is at both points.
It's going to be higher here, because of this additional liquid on top.
So, if we look at point alpha, well, let's start at point beta, actually.
That's the simpler one since it doesn't have the additional liquid.
And first of all, there's just atmospheric pressure.
So there's an external pressure.
Certainly, typically, it would just be atmosphere pressure.
It's pushing down here.
It's also there.
So that has to factor in.
So there's our external pressure.
And then there is the product of this height times the density of the solution.
Times the gravitational force constant.
And I'll work through that momentarily so you see that that indeed is the pressure.
That's exerted by the amount of liquid that's up above this level.
But before I do that, let's put the pressure at point alpha.
So p alpha.
And that's p external plus l rho g. But now, there's this additional height on top of that.
So there's also h rho g. There's an extra pressure.
We'll label this pi.
So we can write p alpha is equals to p beta plus h rho g. Which we'll say is equal to p beta plus pi.
Now, just to make sure we're all on the same page about these quantities being the pressure that's exerted by the liquid, let's just look at that.
So l times rho times g. It should be a pressure which is a force per unit area.
And in particular, this is going to be a weight per unit area.
Remember the weight is the force that a mass exerts.
And it exerts it because of the gravitational constant.
So what happens?
Well, this is just a length.
Rho is a density, right?
So it's mass per unit volume.
And of course, g is just our gravitational force constant.
So it'll be in units of meters per second squared.
So what's going to happen?
If we multiply l times rho together, this is units of length.
This is units of volume.
What we're going to get then is mass per unit area.
And that's what it is, of course.
It's distributing across the area of that surface where point alpha is.
So these two together, l times rho, give us mass per unit area.
So it could be in units of kilograms.
Per unit area, per meters squared.
And now we're going to multiply that by the gravitational force constant.
So now we're going to have all together kilograms per meter second squared.
In other words, it's force per unit area.
And so that's what we have.
And now, of course, we have the additional component.
Due to the height here.
Now, what do we know about the chemical potential in both parts?
The chemical potential has to be equal outside and inside the container.
So mu A at point alpha is just mu A of the liquid at pressure p plus pi and temperature T. And that has to be equal to mu A at point beta.
But point beta is just in the pure liquid.
So that's just mu A star of the liquid at pressure p and temperature T. So here's the chemical potential at point beta.
Here it is at point alpha.
Those two have to be equal to each other.
And now we're just going to use Raoult's law.
And what does that tell us?
We have RT log of xA plus mu A star of the liquid at pressure p plus pi and T So there is our chemical potential inside the container.
And that's equal to mu A star of the pure liquid at point beta.
And so we can just rewrite this as RT log xA.
Plus mu A star liquid at pressure p plus pi of T minus mu A star at pressure p is equal to zero.
OK., now we need the pressure dependence of the chemical potential.
But we certainly have an expression for that.
So we know that dG is S dT plus V dp.
The temperature is the same on both sides.
But we need to worry about what happens at different pressures.
So at constant T, dG is V dp.
So now if we look at the chemical potential, which is just the Gibbs free energy per mole, then d mu A star is VA star molar dp.
In other words, the difference in the chemical potential is, this changes as a function of pressure.
Is going to be given by the molar volume of a under these conditions in the pure liquid dp.
It's just the potential, the pressure dependence of G. It's not the kind of topic I would have thought would drive away a lot of people.
But, you never know.
OK. For the rest of you who are willing to bear with me, let's continue.
So now let's just integrate.
We need to know the change in mu A at a finite jump in pressure from inside to outside the container.
So we're just going to integrate.
So if I want mu A star at pressure p plus pi minus mu A star at pressure p, then I just have to integrate from p to p plus pi VA bar star dp.
And now I'm going to assume, and this is certainly a safe assumption, that this quantity this, molar volume, isn't going to change.
In other words, the molar volume of liquid A isn't going to change significantly, going from the pressure out here to the pressure in here.
On the grand scale of things it's a small pressure change.
We can assume that the liquid is incompressible over that small pressure change.
Which means that this is constant, since all we have then is simply VA bar star times pi.
So then, substituting back here, so now we've calculated this difference, it's a simple result.
So then we simply have that RT log xA plus VA bar star pi is equal to zero.
Now, you saw last time, and I'll work through this quickly again, that the log of xA can be written approximately as minus nB over nA if you remember, this came from writing that this is equal to the log of one minus xB.
But we're assuming that B is the minor constituent here.
So xB is a small number.
So that this is roughly equal to minus xB.
And that's just minus nB over nA plus nB.
And since nB is much smaller than nA, this is approximately equal to minus nB over nA.
So we're going to make this substitution.
Also, this quantity, this molar volume of A over the pure liquid, since the concentration of B is low, we can assume that this is just the molar volume of A in general.
So, in other words, we can write VA bar star.
We can just write it as VA bar.
That is, we're not going to worry about changes in the molar volume, either as a function of pressure or a function of concentration at the low concentrations that we're working.
And then, note that nA times VA per mole is just the total volume.
Oops, not bar.
Not the molar quantity.
This is the molar quantity multiplied by the number of moles.
So it just gives us VA, the total volume, occupied by A.
And using these two results, then we have that RT times negative nB over nA, that's this result.
Plus VA over nA times pi is equal to zero.
nA will cancel.
And finally, since almost all the volume is due to A, again because B is the minor constituent, we can approximate further that VA is approximately equal to V. So finally getting rid of the nA on the denominators, we're left with a simple expression.
pi times V is RT nB.
So we have a very simple expression which is called the van't Hoff expression, and look at how it resembles the ideal gas law.
This is the pressure times the volume equals the number of moles times RT.
Of course, really it's a change in pressure.
And this number of moles is the number of moles of a guest constituent in a solution.
But it has the same form that you're familiar with.
We also, sometimes it's convenient to substitute, since nB over V, that's just the concentration.
That's the number of moles over the volume.
And in that case, the expression is rewritten as pi is RT times c, where it's understood that this is the concentration of the solute.
OK, any questions?
Let's work through a numerical problem.
These things can seem a little bit, they can seem simple, but sometimes when confronted with a problem, sometimes it may seem less than obvious what to do.
And also the results can seem a little bit surprising.
So I just want to work through a simple numerical example to see how these play out.
So let's try, let's imagine we have 10 grams of unknown solid.
Dissolved in 1000 grams of water.
The vapor pressure is 25.195 torr at 27 degrees C. And the pure water vapor pressure is 25.000.
No, .200 torr at the same temperature.
So first, let's just calculate the molecular weight of the solid.
Now, this you I think, saw from last time.
So this is straightforward.
It's going to come from the vapor pressure lowering not from the osmotic pressure, which we'll do next.
So the delta p of H2O is minus xB times p star of H2O.
Alright, this is from last time.
The expression for the vapor pressure reduction.
Due to the concentration of the mole fraction of B.
And then xB is 0.005 torr over 25.2 torr.
Right?
And that's 1.98 times 10 to the minus 4.
So that's our mole fraction of B.
And then xB is nB over nA plus nB.
So, now we have this, it's 10 grams divided by the molecular weight, which is in grams per mole, over 1000 grams of water.
Divided by 18 grams per mole plus 10 grams divided by the molecular weight in grams per mole.
But since the solution is overwhelmingly water, we can get rid of this term and make the math simple.
Can you see that?
I guess barely.
So this turns out to be about 1.98 time 10 to the minus 4.
And if we solve for molecular weight, then we wind up with 907 grams per mole.
So that's using one of the colligative properties that you saw last time.
Now let's, though, finish up by calculating the additional pressure that we'll find.
So what's the osmotic pressure of the solution.
So now we're going to assume that we have that solution inside, in a situation like this where there's pure water outside that can go in.
And there's going to be excess pressure because of the water rushing into the solution.
So what happens?
And let's use a value of the density.
0.995 grams per centimeters cubed.
So, pi is RT times c. That's 0.08314 bar per Kelvin mole.
Times 300 K times c. c is nB over V. So it's 10 grams divided by 907 grams per mole, over 1010 grams over 0.995 grams per milliliter, I'm equating with centimeter cubed.
And that turns out to be 1.09 times 10 to the minus 5th mole per millileter.
Let's get that into liters.
So we'll multiply by 1000 milliliters per liter.
So it's 1.09 times 10 to the 5th moles per liter.
OK, yeah
[INAUDIBLE]
Let's see.
Oh, don't forget that, right
[INAUDIBLE]
Ah, yes, yes, yes.
And I think that's the case in the notes also.
Yep.
Thank you.
OK, so the concentration, just to calibrate us here, it's a pretty low concentration, right?
It's about 10 to the minus 5 moles per liter.
The excess pressure is 0.27 bar.
In other words, about a quarter of an atmosphere.
Uh-oh, what's happening?
[INAUDIBLE]
Where are we?
[INAUDIBLE]
Sorry, where exactly?
[INAUDIBLE]
Oh, yes.
Yes, sorry.
Yeah.
10 to the minus 2 moles per liter.
All right.
And so the pressure that results is a modest pressure, right?
It's about a quarter of an atmosphere of excess pressure.
2.7 time 10 to the 4 pascals.
So let's just calculate, given this small additional pressure, how high is the liquid going to go, above the level of the solution.
Anybody want to guess?
You know, a centimeter.
10 centimeters.
100 centimeters?
Give me an order of magnitude.
Who guesses one centimeter?
You got three choices.
One, 10 and 100, and you have to make one.
Who guesses one centimeter?
Who guesses 10 centimeters?
Who guesses 100 centimeters?
OK. Now, fortunately we don't do problem sets.
Or other scientific problems by popular vote.
Rather, we just solve them.
Whenever possible.
So, what's the height of a column of solution?
Well, that pi, remember, that's equal to rho times g times h, right?
And we know pi now.
So h is pi over rho times g. So it's 2.7 times 10 to the 4 pascal over rho, that's 0.995 grams per centimeters cubed.
Times 1 kilogram over 1000 grams times 100 centimeters per meter cubed.
I'm just getting this into units that will give me, that will be in meters, times 9.8 meter per a second squared.
And when I multiply all that out, it's 2.8 meters.
So that puny little pressure ends up being meters high.
Taller than any of us, by far.
Sorry, it's up to there.
And that's actually pretty typical.
And what that means is, it actually makes osmotic pressure a very, very sensitive measurement method.
If you want to determine a molecular weight, for example.
Because it's so high, that means, and of course you can measure that height pretty accurately.
Certainly within about a millimeter.
And so even relatively modest concentrations, it's still easy to see the effect and measure the impact.
And use that to calculate the properties of the solute.
OK, any questions?
Alright.
Now we're going to start something new.
So, so far what you've seen, and I hope come to appreciate, is the whole structure of thermodynamics is built on empirical macroscopic observation and deduction.
We observe things.
We formulate these broad thermodynamic laws.
All macroscopic.
It doesn't depend on any microscopic model at all.
And in fact much of it was formulated before there was a good microscopic model of matter.
Before the atomic theory of matter, and molecules, and so forth was well worked out.
And it's incredible how powerful that whole formalism is, right?
In some sense, although it may seem like neglecting what we now know to be an important part of nature, actually part of its power is its empiricism.
Right?
There are these very small number of fundamental laws from which everything else follows.
And at this point, of course, there's just enormous confidence in that small number of laws.
Because of their being verified in so many context.
But, at the same time, we do know about the atomic theory of matter.
We know there are atoms and molecules.
So it ought to be possible, at least in principle, to start from a purely microscopic approach to nature.
And just based on figuring out the microscopic properties.
And then saying, well, OK, my macroscopic stuff is just a collection of those microscopic entities.
I should also be able to figure out macroscopic thermodynamics.
I should be able to start from my microscopic picture and get to macroscopic thermodynamic results.
And in fact, that is possible.
And the theoretical formulation for it is what's called statistical mechanics.
Called that because, of course, you won't be surprised to learn that it's going to require a statistical treatment.
We're going to be dealing with moles of material.
But now we're going to be trying to think about their microscopic properties.
And so we'll be dealing not with n number of moles, but with 10 to the 24th number of molecules or atoms.
And you know we won't be able to keep track of every one of their individual states, all the time.
We may be able to do a terrific calculation of the quantum mechanics or the classical mechanics that describe the states that they could be in.
But there's no way in practice that we're either going to experimentally or theoretically keep track of all of that.
So we're going to, at some point, have to introduce statistics.
To take what we know about the microscopic properties and try to go from there to the macroscopic results.
The thermodynamics that we've seen so far.
And that's what statistical mechanics is all about.
So let's try to introduce a little bit of statistical mechanics.
Where we're going to go from what we know about microscopic properties all the way to macroscopic thermodynamics.
That's our objective.
So let's start by just trying to calculate energies of individual molecules, or individual particles.
And what's the probability that some molecule, one of the oxygen molecules somewhere in this room, is in a certain energy state.
Right?
And what our strategy is going to be, is to determine what are the probabilities that molecules are in certain states with certain energies.
And if we can determine that for all the possible states, then we can average over those.
So without keeping track of every individual molecule, we could then calculate, an average energy, which is what you would measure thermodynamically when you look at the whole collection and measure what we call u, right?
Of course, we're really averaging over disparate energies of lots of different atoms or molecules.
They don't all have the same molecular energy.
And we don't try to measure their individual energies.
So that's what we'd like to calculate.
And so we'd like to be able to know what are all these probabilities of different energy states.
And then from that, statistically averaging, what are the macroscopic average energies.
And other macroscopic quantities.
So, let's start there.
Probability that a molecule, must be specific as possible is in state i with energy Ei.
And right now we're not even going to specify the nature of the state.
We could be worrying about translational energy.
What state it is, how fast it's whizzing around the room.
We could worry about its vibrational or rotational energy, or electronic state.
For now, let's not even specify it.
Let's just say the only thing we're really specifying about the state is it has some energy that we presumably know.
Now, what we'd like is to know a functional form for the probability.
And we don't know one a priori.
But let's just think about two molecules.
And the probability that one of them has energy i, and another one has energy Ej.
It also, let me say, would be sufficient to think about even one molecule and say, we'll let's think about independent parts of its energy.
Let's say, translational energy in this direction.
And in an orthogonal direction.
What matters is, we're thinking about two independent energies.
Could be separate molecules that aren't interacting.
Or independent degrees of freedom on the same molecule.
But let's just consider it.
So a molecule is in state i, and that another molecule is in state j, with energy Ej.
So what we're going to calculate is the joint probability that the two molecules are in those states.
So we want a probability.
This one, we'll call Pi(Ei).
This would be Pj of Ej.
And this probability together will be Pij(Ei + Ej).
I could just write Ei comma Ej, but the energy's add.
And that's crucial.
Now, since the molecules are completely independent, this should just be the product of the separate probabilities.
Two completely independent events, or things whose probability we want to determine together.
So, that suggests a really simple functional form.
Because what we're saying here is we want a function of a plus b, which is equal to the same sort of function of a times the same sort of function of b.
In other words, when we multiply these functions, we get a function of the sum of those arguments.
What functional form does that?
Exponential.
Sure.
So in other words, e to the a plus b power equals e to the a times e to the b.
So it suggests that there's an exponential probability.
Suggests it.
Now, there's also some physical insight that we can apply to the problem.
First of all, we expect that states with higher energy in general are going to be less probable than states with lower energy.
If it's cold, not much thermal energy around, and you say, how much energy does a particle in an equilibrium collection of particles have, at thermal equilibrium, if there's not much thermal energy around, you don't expect it to be very likely that a particle has a huge amount of energy.
That one individual particle has it.
Could happen, right?
From some random collisions that just happened to bulk up the energy of that one particle.
But on a balance it's not going to be very probable.
Lower energies will be more probable.
And the other thing is, even in thinking about something like that, temperature comes in.
Immediately.
Right?
If you raise the temperature a lot, surely we have to expect many more molecules will start to become energized.
Will have more energy.
So, lower probability for higher energy.
But the probability of higher energy should go up if the temperature goes up.
In other words, the ratio of energy to temperature should be involved here.
So, P of Ei should go down as Ei goes up.
And should depend on Ei over T. Right?
So, we can start with this functional form.
And by the way, there's no reason we can't have, in general we would have some constants here, right?
We haven't changed anything to do that.
And now let's just use the little bit of physical intuition that we're thinking about here, to refine this just a little bit.
Let's write, we expect that Pi of Ei is some exponential to the minus C Ei over T, where C is some constant greater than zero.
So this will have the property, then, that as energy goes up, the probability goes down.
But it's scaled by temperature.
If I raise the temperature, that makes the higher energy states get more and more likely.
Well, this is our functional form for probability of a molecule being in a state with energy Ei.
And the only difference between this and what's written conventionally is the way the constant is labeled.
So really what we have is Pi of Ei is proportional to e to the minus Ei over kB T, or just k T, where kB is called the Boltzmann constant.
And it's just equal to R over Avogadro's number.
It's the gas constant per molecule, rather than per mole.
One way to try to rationalize this is, you've probably seen, you've all seen Arrhenius kinetics, right?
Arrhenius rate laws?
If you remember what that looks like, you get this rate constant.
Arrhenius rate constant is some constant A times e to the minus Ea over RT.
Remember that?
You've all seen that before?
And so what's happening here, Ea is what is an activation energy.
Remember, the idea is that you've got reactants and products, and there's some barrier you've got to get over.
The activation energy, before you can have reaction.
So the rate depends on surmounting that activation barrier.
This is really coming from the same idea as this, which is the probability of one of the molecules having this much energy.
Depends on e to the minus energy over RT.
This is per mole, so this is per mole.
The exact same relation.
So the idea in the context of kinetics is that the rate depends on how many molecules can get up here.
And what the probability is of their getting enough energy to go over the barrier.
And that energy, that probability, is given by this expression.
So in this form you've seen this kind of dependence before in a very explicit way.
Let me just take it one small but important step farther.
Which is that proportionality constant.
Couldn't we do better?
Couldn't we say, couldn't we figure out what exactly it is, not just what it's proportional to?
Well, let's try.
So as we've written it, we've got Pi of Ei is proportional to e to the minus Ei over kT.
We could write that as equals A e to the minus Ei over kT.
This is just some proportionality constant.
But we can determine what that constant is, because we know that if we sum over all the possible states, the molecule has to be in some state, right?
So that sum of all those probabilities has to equal one.
So sum over i of Pi of Ei, is equal to one.
Because the molecule must be in one of the states.
OK, now this is our expression for all those p i's.
So let's just write that way.
a is just a constant times the sum over all the i's of e to the minus Ei over kT.
And so there it is.
a is equal to one over the sum over i, e to the minus Ei over kT.
So now we can rewrite Pi of Ei is equal to e to the minus Ei over kT over, the sum over i, e to the minus Ei over kT.
Now, just so you're not confused, this i matters.
That's the i we're talking about.
What's the probability of it being in some particular state i with energy e i.
This i is just a dummy variable.
So just to be explicit, we could write this just as well as e to the minus Ei over kT times the sum over j, e to the minus Ej over kT.
It could be confusing, but this is usually the way that it's written.
Even though this dummy variable here has nothing to do with the particular choice that's made here.
So now we know.
And we can calculate what the probability is for a molecule to be in any particular state if we know the energy of that state.
Now, notice we need to know all the energies of all the other states, too.
Because this thing, in other words, this a, right, depends on summing over all of those.
But if we know those, then we can do it.
We can do the whole calculation.
And for a good number of cases, we realistically do know it all.
So from the proportionality relationship alone, of course, we can tell the ratio of chances that you're in one state or another.
Even without this a constant, we already could've said well, if what's the ratio of, the probability of being in state i with energy Ei to state j with energy Ej.
Well, it's just e to the minus Ei over kT over e to the minus Ej over kT.
It's e to the minus Ei minus Ej over kT.
But apart from just being able to get the ratio, we can get the absolute number.
What's the absolute probability that a molecule is in any particular state of a given energy.
Next time what you'll see is what can be calculated based on the results that you've seen, just the results you've seen so far.
From these quantities alone, it turns out you'll be able to calculate every single macroscopic thermodynamic quantity.
So you'll see some of that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, last time we started in on a discussion of a new topic, with was statistical mechanics.
So what we're trying to do now is revisit the thermodynamics that we've spent most of the term deriving and trying to use in a microscopic approach.
So our hope is to be able to start with a microscopic model of matter, starting with atoms and molecules that we know are out there.
And formulate thermodynamics starting from that microscopic point of view.
In contrast to the way it was first formulated historically, and the way we've presented it also, which is as an entirely empirical subject based on macroscopic observation and deduction from that.
So, we looked at the probabilities that states with different energies would be occupied and we inferred that there would be a simple way to describe the distribution of atoms or molecules in states of different levels.
First of all, although I didn't state it explicitly, essentially assumed there is that if I've got a bunch of molecules, and I've got states that they could be in of equal energy, then the probability that they would be in one or another of those states is the same.
If the energies of the states are equal, the probabilities that those states will be occupied.
That they'll be populated by molecules, will be equal.
And then we deduced what's called the Boltzmann probability distribution.
Which says that the probability that a molecular state, i, will be occupied is proportional to e to the minus Ei over kT, where little Ei is the energy of that molecular state.
And then realizing that the probability that some state out there has to be occupied, we saw that of course if we sum over all these probabilities that sum has to equal one, right?
In other words, the sum over all of i of P of i is equal to one.
So that means that we could write not just that this is proportional to this, but we could write that Pi is equal to e to the minus Ei over kT divided by the sum of all such terms.
So now we know how to figure out how likely it is that a certain state is occupied.
And what it means is, let's say we have a whole bunch of states whose energy is pretty low compared to kT.
kT, the Boltzmann constant times temperature is energy units.
So if it's pretty warm, maybe it's room temperature, maybe it's warmer.
And there are a whole lot of states that are accessible because the energies are less than kT.
What does that mean?
That means that you could put a bunch of different i's here whose energies are all quite low compared to kT.
And they'll all have significant probabilities.
Now let's decrease the energy.
Go to cold temperature.
So kT becomes really small.
So that e to the minus energy over kT starts to, if this gets to be bigger than kT by a lot, then this whole thing is a very small number.
Suddenly, you get very few states accessible.
So if we just plot this distribution, of course it's easy to do, it's just a decaying exponential.
So Pi, which is a function of Ei.
It just looks like this.
And here are a bunch of states.
And if we have a classical mechanics picture of matter, then there would just be continuous states.
And if we have a quantum mechanical picture of matter, there might be individual states with gaps in between the energies.
Either way, what happens, what this is saying is, if we go out to higher and higher energies, then you have a smaller and smaller probability to be in a state like that.
And if you go to low energies, the probability gets bigger.
And it depends on temperature.
Because it's the ratio between the energy and kT that dictates the size of that term.
So let's say this is moderate temperature.
If we go to low temperature, it might look more like this.
Hardly any states can be occupied because even states of rather moderate energies, suddenly now those energies are much bigger than kT.
In other words, kT is measuring thermal energy.
It's saying there's not enough thermal energy to populate, to knock things into states whose energy is much higher than that.
So you have a very precipitous decay.
If you go to the other extreme, of very high temperature, then this will tend to flatten out.
Eventually it'll decay, but it may take a long time.
Because now kT is enormous.
So the energy has to get to be very big.
Before it's bigger than kT.
And as long as it isn't, then this exponential term is not small.
So there be a whole bunch of states that may be occupied.
In other words, a whole bunch of states that are thermally accessible at equilibrium.
Systems at thermal equilibrium, molecules are getting knocked around with whatever thermal energy is available.
Crashing into each other or into the walls of a vessel.
And there'll be a distribution of molecular energies.
And that distribution is skewed either low or high depending on the temperature.
So that's what that distribution, the Boltzmann distribution, is telling us.
This is often called a Boltzmann factor, because it's telling what the population of some particular state is.
OK. Now, this is just dealing with individual molecule states.
Then we said, OK, what about the whole system?
Well, the same kind of relation holds there, too.
In other words, if these are individual molecule energies, now if I look at the entire system, right well, still, there's no reason that same distribution doesn't hold.
And it does.
So in other words, Pi of Ei, that's the whole system energy, is e to the minus Ei over kT, over the sum over i, Ei over kT.
Now, this i doesn't refer to a single molecule state.
We're talking about a whole system.
It might be a mole of atoms or molecules in the gas phase, or what have you.
It refers to a system state where the energy, the state of every one of those atoms or molecules is specified.
So this is a system microstate.
Every molecular state is specified.
So for a mole of stuff, that means there might be 10 to the 24 or so molecular states that this single subscript is indicating.
But the point is, those states exist.
They have a total energy.
What's the probability that the whole system will be in such a state?
Still going to be proportional to the total system energy.
It turns out that these summations end up taking on an enormous importance in statistical mechanics.
And the reason is, as we'll see shortly, it turns out that every single macroscopic thermodynamic function can be derived by knowing just that.
Just these sums, what are called partition functions.
So of course they take on enormous importance.
So, we call them partition functions because what they're doing is, they're indicating how the molecules are partitioned among the different available levels.
The molecules or a whole system.
So the molecular partition function is labeled little q.
And the system partition function is labeled big Q.
It's called the canonical partition function.
And because they are going to take on such special importance, let's just look at some of their properties for different kinds of systems and in general.
OK, first of all, let's talk about units and values.
They are unitless.
This is an exponential function.
Here are units of energy and energy.
But this is a unitless was number.
It's just some number, right?
Could be 1.
Could be 10.
Could be 50.
Whatever, right?
Could be 10 to the 24.
Its magnitude tells you about more or less how many states are thermally accessible.
Because, look at it.
And then go back to the example that I showed you if lots of these terms are big, are significant because this is really a big number, right?
It's really hot.
So lots of states have energies lower than kT, which means this is not too small, for lots and lots of values of i, then this just keeps adding up.
And of course, same here.
So the number might be very large.
But if we're in the low temperature limit, maybe we're in such low temperature that only the lowest possible state is occupied.
And everything else, it's just too cold.
There's not enough thermal energy to occupy anything but the very lowest state available.
Well, in that case the lowest state, this would be one.
Essentially we could label this zero.
We could put the energy, the zero of energy there.
Everything else is really big compared to kT, which means this exponential gets to be a really small number for every state except one.
And this is just equal to one.
In other words, the magnitude of these numbers tells us about how many states are accessible, thermally accessible, to molecules or to a whole system.
So let's just go through a couple of specific examples to try to make that a little more concrete.
One is, let's start in the simplest case.
Which it'll turn out I just alluded to.
Let's consider a perfect atomic crystal at essentially zero degrees Kelvin.
Zero Kelvin.
So every atom.
Or even if it's a molecular crystal, every molecule, they're all in the ground state.
There's no excess thermal energy.
Every molecule is in its proper place.
Every atom, if it's an atomic lattice.
In other words, it's in the ground state.
That's it.
So again, we can place the zero where we like.
We'll place it there.
That one, for that one state, this will be equal to one.
And for everything else it'll be zero.
So Q is just the sum of e to the minus Ei over kT.
It's equal to e to the minus zero over kT plus e to the minus E1 over kT plus e to the minus E2 over kT.
But remember, T is really tiny.
It's almost zero degrees Kelvin.
So all these things are much bigger than that.
So this is vanishingly small.
This is vanishingly small.
The whole thing is approximately equal to one.
That's it.
Also, if we say OK, now what's the probability of the system being in a particular state.
Well, we have an expression for that.
So let's look at P0.
It's e to the minus E0 over kT over the sum.
Which we've just seen.
So it's e to the minus E0 over to kT divided by e to the minus E0 over kT plus e to the minus E1 over kT, and so on.
This is the only term that's significant.
So it's approximately equal to one.
Now, while I've got this written here, let's just make sure we've got something clear.
What if we hadn't arbitrarily set the zero of energy equal to zero?
I mean, it's arbitrary, right?
We can put the zero of energy anywhere.
And you might think, well, gee that's going to have a big effect on everything.
Well, it would have an effect on the actual number that we get for Q.
But what you'll see is that any measurable quantity that we calculate won't be affected.
It's only the zero of the energy scale that's going to be affected.
So for example, let's look at this probability.
It doesn't matter where we put the zero of energy.
This term is still going to be enormously bigger then the next one.
And the next one.
And the next one.
So in this sum only this term is going to matter.
It's going to cancel with this term.
So whether or not these individual terms are equal to one, which happens if we set this to zero, or whether they're equal to some other number.
Still, the probability that the system is in the lowest state is one, right?
That doesn't depend on where we arbitrarily put the zero of the energy scale.
The state is still going to be in the ground state, at essentially zero degrees Kelvin.
And it'll turn out to be that way with any property that we can measure.
Only the zero of this scale moves if we arbitrarily move it somewhere.
But the observable, measurable quantities that we calculate, they won't change.
Other than that scale.
So that's one simple example.
Now let's look at another example.
Let's consider a mole of atoms, roaming around in the gas phase at room temperature.
OK, so now what I want to do is just have a simple model for their translational motion.
And of course, we could do that either quantum mechanically or classically solve for that.
We're going to use an even simpler model.
And this model is going to be very, very useful for a lot of the things that we'll treat.
It's called a lattice model.
All it means is, we're going to divide up the available volume.
This room, for example, into zillions of tiny little elements.
Little volume elements.
Each one about the size of an atom.
The idea being, we're going to specify the state of the atom by saying where is it.
Is it in this lattice sight, in this one, in this one, in this one.
So it's a lattice model.
Might be an atom there.
Might be one there.
So we're going to divide the volume up.
So let's call our atomic volume little v. Our total volume big V. And an atomic volume, it's going to be on the order of 1 angstroms cubed.
Or 10 to the minus 30 meters cubed.
And our room, our volume macroscopic one, might be on the order of one meter cubed.
Ordinary sort of size.
So now let's figure out our molecular partition function.
Now, implicit in this, we're basically saying that the energy, the translational energy, is basically zero.
In other words, all these states have the same energy.
They're just located in different positions.
At any given instant of time.
And what that means is all these terms, all these Boltzmann factors, we just set them equal to one.
We'll set the zero of energy there, be done with it.
It's a simple model.
But it's going to give us the right order of magnitude that we're after.
So, how many states are there that are accessible?
Well, on the order of 10 to the 30th, right?
So q, little q, we'll call it little q translational, it's just discussing where things are.
Is on the order of 10 to the 30th.
And if we do a more careful treatment, if we treat the translational energy of atoms, either classically or quantum mechanically and solve it.
We'll still get, we still do get, about the same order of magnitude.
Now let's treat the whole system.
So, what happens?
What are our total possible states?
Because we have to add this up for every possible state.
Well, let's start with the first atom.
It has to be somewhere.
It has 10 to the 30th possibilities for where it can be.
Let's start with the second atom.
And put it somewhere.
Well, it has 10 to the 30th minus one, which is still pretty close to 10 to the 30th.
Let's let's go to the third atom.
And the fourth.
If we have about a mole of atoms, let's say 10 to the 24th atoms, that's still going to occupy a very small fraction of the site.
Only one in a million.
So we don't have to keep careful track.
Every one of them, we can just say, look, there are 10 to the 30th available sites.
Because we're not going to worry about a change in one in a millionth.
So what that means is, capital Q, trans for the system, is 10 to the 30th times 10 to the 30th.
In other words, let's now take the whole state.
Well, I could put atom one here.
I have 10 to the 30th possibilities.
Atom two, I can put anywhere else.
So the joint probability of that particular state, for just the two atoms, is 10th to the 30th times 10 to the 30th.
It's 10 to the 30th squared.
So this is going to keep going.
It's going to be 10 to the 30th to the Nth power.
Where N is the number of atoms, which is 10 of the 24th.
Huge number, right?
It is a huge number.
And that, too, if we treat the whole thing classically or quantum mechanically and work it all out.
We'll get that number.
Or something on that order.
Because you know there really is a simply astronomical number of states accessible to the whole bunch of atoms or molecules in this room.
It really is that big.
So in other words, capital Q is just an astronomical number.
And it is the case.
OK.
There's an important sort of nuance that we need to introduce.
And it's the following.
Turns out, it is an astronomical number.
But a tiny bit less astronomical than what I've treated so far would indicate.
So let's look a little more carefully.
What I've said is that Q translational is little q translational, that is, that 10 to the 30th number.
To the Nth power.
But there's one failing here.
Which is, when I got that, when I decided that if I have just the first two atoms I've got 10th to the 30th times 10 to the 30th possible states, the trouble with that is then if I keep counting all the possible atoms starting with each one, I'll double-count it, right?
In other words, what if I interchange those two atoms?
Well, those states are identical, right?
Indistinguishable.
And it's only distinguishable states that count.
When you specify these things, these are indicating distinct states.
In some way, at least in principle, measurably different.
If the atom's are identical, in other words, if it's a mole of the same stuff, I don't have a way of distinguishing between those two.
I have to correct for that.
And when I come to the third atom, I have to correct for all the possible interchanges.
Of course, it's three factorial.
And in general, it's N factorial.
So this result needs to be modified.
This is true for distinguishable particles.
In other words, if I had all different atoms, so I could label them all, then I don't have that correction.
Because then there's really a difference between that state and the state with the two atoms interchanged.
But if it's a mole of identical atoms, that's no longer the case.
So then, Q translational is little q translational to the N power divided by N factorial.
It's still going to be an absolutely enormous number.
But it's going to be a little less absolutely enormous than it was a minute ago.
Now finally, I just want to introduce a handy approximation to N factorial that's going to turn out to be very useful again and again.
And that's called Stirling's approximation.
For the log of a big number, ln N factorial is N log N minus N. If we take e to those, both sides, then we find that N factorial is equal to e to the minus N N to the nth power.
So this, then, is approximately equal to q translational to the Nth power.
Over N to the N times e to the minus N. Now let's put the numbers back in.
There's our 10 to the 30th to the 10 to the 24th power.
And now we're going to diminish that just a bit.
It's N to the Nth power.
So it's 10 to the 24th to the 10 to the 24th power.
Times e to the minus 10 to the 24th power.
So, we can cancel something here.
So we have 10 to the 6th to the 10 to the 24th power times e to the 10 to the 24th power.
e is about 10 to the 0.4th power.
So this whole thing is about 10 to the 6.4th power to the 10 to the 24th power It's still a pretty respectable number.
You could still say astronomical.
But maybe astronomical, but not quite to the edge of the universe.
Whereas the one before maybe hit the edge of the universe and then beyond.
So that's our second example.
And again, part of what I'm trying to do here is just introduce some ideas.
Things like this way of modeling positions and so forth.
But also, again, orders of magnitude.
How big are these numbers.
For different sorts of systems.
So let's do one more example.
Now, let's consider a polymer in a liquid.
And it has different configurations.
And they might be a little bit different in energy.
For example, some configurations might bring neighboring regions of the polymer into proximity where they could hydrogen bond.
And the point is that then the molecular energies involved will change a little bit.
Because of some sorts of interactions that are possible in some configurations.
And not in other configurations.
So what I'm trying to do is introduce a very simple framework through which we might be able to look at things like protein folding, or DNA hydrogen bonding.
Things like this.
And just in a simple way model how those work and what the forces are that drive them.
So we'll think about polymer configurations.
So let's look at a few configurations.
Here is going to be one.
I'm going to label this one here.
And then here are a few others.
So I've labeled them this way so that you can see how they are distinct from each other.
This one would have a possible interaction.
So we'll label its energy.
So this is molecular state i, here's going to be our energy, Ei.
And let's call this one negative e int, for an interaction energy that's favorable.
And these will be zero.
Let's just put that there.
Zero, zero, zero.
And let's also indicate the degeneracy.
How many states, different states are there with the same energy.
And that's called gi.
And here it's one.
And here are these three.
So that's the framework of our model.
Well, so what's our molecular partition function for this configurational degree of freedom?
Well, we can label it little q configurational.
So we're going to sum over these states.
e to the minus Ei for the different configurations over kT.
So it's e to the e int over kT.
Plus three times e to the zero over kT, we'll get all those other three terms.
So that's it.
It's e to the e int over kT plus three.
Remember, the interaction energy is negative e int.
It's a favorable interaction.
e int is a positive number.
So that's our result.
Now we've described the probabilities in terms of the states that can be occupied.
That is, we've added it up.
But of course, even the way I wrote it just out of convenience, I didn't actually write out each term in the sum.
In the notes I actually did that.
But of course it's not necessary to write e to the zero over kT plus e to the zero over kT and write that three times for each of these three.
Rather, it's convenient to group them together.
And the point I'm illustrating here is that in our expression, for the partition function, we've written that in terms of the individual states.
But we doing a sum over the individual molecular states.
But we could also sum over energy levels.
Including the degeneracy.
So we could say, let's not do the sum over every state.
After all, what if there are a hundred states that had the same energy.
Rather than just three.
Gets to be kind of painful, right?
Instead let's just sum over energy levels.
They're all going to be the same factor anyway.
And then we'll have, we'll just explicitly write the degeneracy in there.
So, we can write the partition function as a sum over energy level.
So, in other words q is the sum over i, e to the minus Ei over kT.
That's individual molecular states i.
But we also could write it as the sum over i, where this now is molecular energy levels i.
And then we need to incorporate the degeneracy gi e to the minus Ei over kT.
Same thing, of course.
But again, sometimes much more convenient to write things this way.
And of course, we can write the probabilities of the occupancies in the same way, too.
That is, we could talk about the Pi in terms of individual states.
e to the minus Ei over kT over q, the whole sum.
Or Pi summing over energy levels, gi e to the minus Ei over kT, divided by q.
Here, of course, this is bigger if it's degenerate, right?
It's saying that the probability of being in this energy level is three times the probability of it being in any individual one of the states.
But sometimes it's useful to keep track of things in this way.
What it shows you too is that, remember when we looked at, did I erase it?
I guess it's gone.
When we looked at this probability distribution for the occupancies of the levels, of course, what it says is at any temperature, the lowest level is the most probable.
For intermediate temperatures.
For low temperatures, for high temperatures.
At high temperature, it might be only a little more probable than the next one over, and the next one, and so forth.
But the very lowest state always has the highest probability.
But the lowest energy doesn't always have the highest probability because of degeneracies.
There might be many states with an energy up here.
And the probability of any one of them is only a little lower than the probability of the lowest energy.
That could be the case here.
Let's say, the interaction energy isn't enormously strong.
So there is some energetic favoring of this state.
Maybe there are 10% at room temperature.
Maybe it turns out there are 10% more molecules like this than in any one of these states.
But of course, these three altogether mean that there are many more molecules at this energy than at the lowest energy.
Now of course we could do the same thing for the canonical partition function.
Not just the molecular one.
So in other words, capital Q sum over i system microstates.
e to the minus capital Ei over kT.
But we could also write it as a sum over energies.
Sum over system energies.
Ei.
And then we have to include the degeneracy.
Capital Omega i e to the minus Ei over kT.
Degeneracy of the system energy Ei.
Little g here is the degeneracy of molecular energy Ei.
Now, same form.
Just an important difference, though.
This number, this little gi, is typically a small number.
It could be one, it could be a few.
This number is usually astronomical.
That's basically, that is, what we calculated here.
In other words, how many total system states are there with a particular energy.
Well, in many, many, many cases the answer is some astronomical number.
So this number might often be between one and ten.
This number might be 10 to the 24th.
It might be 10 to the 24th to a large power.
And, of course, that has a big effect on the way things end up working.
In statistical mechanics and in thermodynamics.
A lot of thermodynamics results are the way they are because you have so many possible states with a particular energy.
That that energy can be strongly favored just by virtue of the number of states that there are.
Just the way, in a very small way, this energy might be favored just because there are more states with it then there are states here.
But again, with the system, it's not a factor of three to one.
It might be a factor of 10 to the 24th to the power of something.
It might be just enormously larger than other possibilities that'll tend to put the energy at a certain place.
And just to continue, of course, the same thing goes for the system probabilities.
Pi, right, which is e to the minus Ei over kT divided by capital Q.
If this is a sum over states, or omega i e to the minus Ei over kT over Q, let's write this over, if now we're calculating the probability of an energy level.
And it's important to make the distinction.
Because in many cases, that's what we care about.
What's the energy of the system?
And we often don't care about exactly where's that molecule and that one, and that one, and that one, right?
The individual states that might be involved that would comprise that energy.
Now, what I want to do is start deriving thermodynamics.
Like I promised, we're going to be able to derive every thermodynamic quantity if we just know the partition function.
So now I just want to show that that might really be true.
So, the point is that from Q we're going to get all of our thermodynamics.
And let's start with the energy.
Remember u, right?
That's our system energy.
It's an average energy.
It's an average of the energy that we would get by looking at the states that the system might be occupying.
So we can write it as a sum over i.
Of Pi times Ei.
In other words, it's going to be determined by the energy of any system state times the probability that the system is in that state.
Add them all up.
Well, we know what that is.
It's the sum i of Ei, e to the minus Ei over kT divided by Q.
Now, I'm going to just make a simple substitution.
I'm going to use the term beta to mean one over kT.
I'm really just using that so I don't need to use the chain rule a billion times and doing derivatives.
So I'm going to write this is sum over i Ei e to the minus beta Ei over Q.
So Q, then, in this term is just a sum over i e to the minus beta Ei.
So.
Now I do want to take some derivatives.
If I take dQ / d beta, keeping V and N constant, then I get derivative with respect to beta.
Sum over i e to the minus beta Ei.
So that's going to bring out Ei, right?
So it's going to bring out minus Ei minus the sum over i, Ei e to the minus beta Ei.
That obviously looks like a handy thing.
Because that looks like that term.
Then, our average energy, remember, it's one over Q.
Sum of i, Ei e to the minus beta Ei.
So now, it's just minus one over q, dQ / d beta.
So that's just the same as minus d log Q / d beta.
And now I'm going to use the chain rule.
So it's minus d log Q / dT times dT / d beta.
All at constant V and N. Now I can easily get dT / d beta.
Because d beta / dT is just minus one over kT squared.
It's just the derivative of one over kT with respect to T. So finally, my average E, which is u, is just kT squared times d log Q / dT constant V, N. That's terrific.
In other words, if I have an expression for Q, I know the partition function, and I can calculate it at any temperature.
I just need to take log of it, take its derivative with respect to temperature.
Multiply it by k T squared and I've got my energy.
Not very complicated, right?
So in other words, macroscopic thermodynamic properties come straight out of our microscopic model of statistical mechanics.
Statistical thermodynamics.
Now I'm just going to state the next result, just because I want to get there and it'll be followed up more next time.
But it's the following.
You can see, of course, our Q is a function of V and N and T. It's a function of V because in principle the energies that are going into all this can be a function of volume.
What thermodynamic function is naturally a function of N, V, T?
Who remembers?
Gibbs free energy?
Helmholtz free energy?
Enthalpy?
Which one?
Nobody knows.
What's a function of N, V, and T. Or, V and T, is what we really formulate it as.
N was introduced later.
The Helmholtz free energy.
Thank you.
What that suggests is that actually the simplest and most natural connection between Q and macroscopic thermodynamics is to the Helmholtz free energy.
And the result that you'll see derived next time is A is just minus kT log of Q.
What a simple result.
And you'll see the derivations in your notes.
You can see it's a couple of lines, right?
But of course, if you know A, and you know E, you know everything, right?
Because S is in there.
And then other combinations can give S and G and mu.
And p. And anything else you want.
So that's the point, is that all of macroscopic thermodynamics follows.
And you'll see that elaborated more next time.
And in addition, a very simple natural expression for the entropy in terms of the states available follows, that we've alluded to before.
And now you'll see it played out.
Right.
The following content is provided under a Creative commons license.
Your support will help MIT Open Courseware continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So you've been learning about statistical mechanics.
The microscopic underpinnings of thermodynamics.
And last time we ended up working with the canonical partition function and showing that once you have the canonical position function, you have basically every thermodynamic quantity that you've learned how to calculate this far in the course.
But before I start, on the lecture notes 25, there were a couple typos in there that actually didn't make any difference to the final result because they cancel each other out.
But it's been corrected on the web version.
And near the bottom of the page -- because we're going to be mostly using these notes today.
Near the bottom of the page, where it says A is equal to u minus TS is equal to u.
That should be a plus here.
dA/dT, volume.
And then further down the next line, blah blah blah blah blah.
Minus u over T squared.
And this should be a minus here.
One over T. dA/dT, V. And et cetera.
So luckily these two typos cancel each other out the result is correct.
But there it is.
OK.
So last time, then, you saw how from the canonical partition function, you could get something like the energy.
You wrote down an equation.
The energy is equal to k, T squared, d log Q / dT.
Under constant volume.
And number of particles.
And then you notice that the important variables are the volume, the number of particles, and the temperature.
And we know that every thermodynamic quantity has a set of natural variables.
For the Gibbs free energy it was the pressure and the temperature, the number of particles.
And for the volume of the number of particles and the temperature, we know that that's the Helmholtz energy.
So the natural variable that we would associate -- the natural thermodynamic variable we would associate with that set of constraints is the Helmholtz free energy.
So it becomes interesting, then, to figure out, how can we write the Helmholtz free energy in terms of the canonical partition function?
They seem to have the same set of natural variables.
And that's what you started doing and what we'll do today.
And that's where the typo comes in.
So let's write what we know about the Helmholtz free energy in terms of the energy u. I already wrote it up there.
u minus TS.
That's the definition of the Helmholtz free energy.
And from the fact that dA is equal to minus p dV, minus S dT, plus mu dN.
We can read out what S, here, is in terms of the Helmholtz free energy.
S is just dA/dT, constant V and N. So we can plug that in here.
u minus T dA, with a minus sign there, so that becomes a plus.
And hence the typo.
dT, constant V, and N. OK.
So now we have an equation that relates u and the partition function.
We want an equation that relates A and the partition function.
If we rearrange this slightly, we can get that u, then, is equal to A minus T, dA/dT, constant V and N. So the question that we could ask ourselves is -- is there a function of A that kind of looks like that?
And I know the answer, so I'm going to give it to you.
A function of A that looks like the sum of A minus something times the derivative, with respect to that something.
We should try to look at something like the derivative d(A/T)/dT, constant V and N. So if you take that derivative, you end up with one over T, dA/dT, minus A over T squared.
But if you look at this result, here, and you multiply by T squared, there's the A, and there's the T times dA/dT.
There's the A that -- we just have the sign wrong.
So we have -- multiply this by minus T squared.
Minus T squared, times minus T squared.
And you have the same thing as here.
So that tells us then, that u can be written as minus T squared, d(A/T)/dT, constant volume.
It's a nice way to relate those two energies.
And we have an expression for u in terms of the canonical function.
And we can then replace it in here.
And that gets us, then, that minus T squared, d(A/T)/dT, constant number and volume is equal to u.
And u, we saw, was equal to k T squared d log Q / dT.
V and N fixed.
And then we can start -- The t squareds disappear here.
And then we have d/dT on this side and d/dT.
Let's just take the integral.
You take the integral of both sides, and that gets us that then A over T is equal to k, log Q, plus the constant of integration.
And we can take that constant of integration to be whatever we want.
Energy is all relative to some reference point.
We can take it to be zero.
A is equal to k T log Q.
That's a pretty neat result.
There's the microscopic underpinning of things.
Where we know about atoms, and energies, and states, and even quantum mechanics, and all sorts of things goes into here.
All the microscopic information goes in here.
And there's a thermodynamic variable that only cares about the macroscopic state of matter.
It doesn't care that there are atoms there.
It just cares that you know the pressure, the volume, the temperature, or any couple variables.
And you can get a direct equality without any derivatives or anything.
Between the macroscopic and the microscopic.
So this is really pretty remarkable.
And once you have A, you have everything.
Just like before, we said once you have G, that Gibbs free energy, when we were talking about things that depend on pressure and temperature.
You have everything.
It's the same thing here.
We've got A, we've got u.
We have everything.
We can calculate every single thermodynamic variable from then on.
For instance, if we want to have the entropy, S is equal to minus A over T, minus u over T. Where did I get that?
I got that from way up here.
A is equal to u minus TS.
I solved for S in terms of A and u. I've got expressions in terms of the canonical function for A and u. Plug that in there.
Get k log Q, plus k T, d log Q / dT, constant number and volume.
et cetera.
You can get the pressure.
You can get the pressure from the fact that the pressure up here is a derivative of A with respect to volume.
So you take the derivative of A with respect to volume here.
You get the pressure.
If you want the chemical potential.
The chemical potential from the fundamental equation up here is the derivative of A with respect to the number of particles.
You take the derivative of A with respect to the number of particles, you get the chemical potential in terms of the canonical partition function.
So you've got everything.
You've got u.
You've got A.
You've got p. You've got S. You've got H. You've got G. Name your valuable, you've got it.
You want the heat capacity?
It can get you the heat capacity in terms of the partition function.
Any questions?
Alright so let's go on.
Let's look a little bit closer at the entropy, in terms of the microscopic theory here.
So let's start with S is A minus u over T. And let's see how far we can go.
Alright.
Let's write it in terms of -- let's write A and u in terms of things that we know.
And let me just go to the end to tell you where I'm going, and why I'm going to make certain changes in my math here.
So what we're going to get at the end is that S is this very nice quantity, which is minus k, pi log pi.
Where the p's are the microstate probabilities.
The probability that your state is in a particular -- yes?
[UNINTELLIGIBLE]
S is u minus A.
Yes.
u minus A.
Thank you.
I should read my notes.
OK.
So we're going to equate S here in terms of the probabilities of microstates.
And that's going to be -- remember how we talked about S is related to concepts of randomness, or order, or disorder.
So the number of possible microstates is related to the number of amount of disorder that you might have.
If you have a pure crystal, and every atom is in its place, then the number of microstates at zero degree Kelvin is one.
So the probability of being in that microstate is one.
And the probability of being in every other microstate is zero.
Alright, so there's a relationship that we're going to have here, which is going to be interesting.
Let's derive it.
That means that we're going to want to have this somehow pop out of the equation.
We're going to want to have this pop out of the equation.
If you remember, pi is the -- pi is e to the minus the Ei over kT, divided by the partition function Q.
So somehow we are going to have to get this to come out.
We're going to have to have these e to the minus Ei over kT's come out.
OK.
So let's try to get them to come out right away.
We know u.
A way to write u is the average energy.
Which means let's take one over kT out here.
And there's a -- let's divided by k here.
Let's get the k here.
kT here.
We're going to want to have a k come out here.
There's the k here.
So that's one way of getting it to come out.
And then the u is going to be the average energy.
e to the minus Ei, minus kT.
That's just writing the average energy.
So the energy times the probability of having that energy divided by the normalization.
Plus, well, A is just log Q.
A over kT is just log Q.
So we've managed to extract this guy out here.
OK. Now, this sum, here, this is sum over i. I'd really like to have this sum come all the way out.
So I've got to find a way to do that.
And it would be nice if I could find a sum here.
Maybe if I multiply by one.
If I write one in a funny way.
Get a log here.
You know, if I've got a couple logs here, maybe I can combine them to get a ratio.
So let's rewrite this Ei in a funny way, here.
Ei is -- I'm just rewriting it, but in a strange way.
Log e to the minus Ei over kT.
So if I take the log of e to the minus the Ei over kT.
I get minus Ei over kT.
The kT's disappear.
So I just get Ei is equal to Ei.
I'm just writing something that's pretty obvious here.
And then we're going to take that expression, and put it in here.
So now I'm going to be able to write S over k is minus -- So the kT here cancels out this kT here.
Sum over i. e to the minus Ei over kT.
Over Q.
That's that term right here.
And then I have the Ei, which is log e to the minus Ei over kT.
This whole thing is in this parenthesis here.
And then I have the plus log Q.
Plus log Q. OK.
So I have this nice thing here.
e to the minus Ei over kT divided by Q.
Well that's looking an awful lot like this pi here.
Which is what I'm trying to get out.
I'm trying to get these pi's coming out.
So that's a nice thing to have here.
Now if I could only have a pi coming out here, somehow, that would be great too.
And if I also got a sum here, that's a sum over i I could sort of combine everything together.
So I'm going to write one in a funny way.
One is equal to the sum of all probabilities.
That's obvious.
And I'm going to write this pi here in a form that looks like this.
Sum over all i, e to the minus Ei over kT, divided by Q.
Just writing one.
The sum of all probability is equal to one.
And I'm going to take this one, here, and I'm going to put it right in here.
Now log Q doesn't care on i, so it's just a number.
So that allows me to rewrite -- S over k is minus the sum over i. e to the minus Ei over kT, divided by the partition function.
Times the log of e to the minus Ei over kT.
Plus sum over i, e to the minus Ei over kT over Q.
Times log Q. OK good.
I'm going to take these summations.
Now everything is over the sum of Ei.
This is great.
And there's this factor here.
E to the minus Ei over kT divided by the partition function.
That appears in both.
I can factor that out.
and.
Then I have these two logs.
Log of this and log of that.
That I can also combine together.
This is the log of this.
And then there's a minus sign.
So if I take the -- it's going to be the log of this minus the log of that,.
It's going to end up with a ratio.
S over k is equal to minus Ei.
Taking the summation out.
e to the minus Ei over kT, over Q.
That's this term.
That term.
And then I have the logs.
Log of e to the minus Ei over kT.
This is a minus sign.
This is a plus sign.
That means I divide here by Q.
This is great.
Look.
This e to the minus Ei over kT over Q. e to the minus Ei over kT over Q.
What is that?
That's just pi.
This is pi.
That's the probability of microstate i.
And this is equal to minus sum over i, pi log pi.
There's the k, here.
S is equal to minus k, log sum over i, pi, log pi.
Another great result.
Now if you system is isolated.
If you have an isolated system, that means that the energy -- You've got your boundary.
The boundary doesn't let energy go in and out.
Doesn't let the number of particles go in and out.
Every single microstate is going to have the same energy.
If this system is isolated.
The only thing you're going to change is the positions of the particles.
Or their vibrational energy.
Or something.
But let's just stick with translation.
You're just going to change the positions.
You're not going to change the energy.
So, if the system is isolated, then the degeneracy of your energy is just a number of ways that you can flip the positions around.
Indistinguishable ways.
So the probability is just one over the number of possible ways of switching positions around for your particles.
So for an isolated system, all microstates have the same energy.
We can set that equal to zero as our reference point.
And the probability of being in any one microstate is just one over the number of possible ways of rearranging things.
So the probability of been in any one microstate is one over the number of my microstates.
They all have the same energy.
Where this is the degeneracy.
So now when you plug that in here, S is minus k. Sum over all microstates from one to blah, one over blah, log of blah.
This is a number.
It can come out.
The sum of one to omega.
Of one over omega.
Is omega times omega.
It's one.
The one over omega -- the log of one over omega is minus the log of omega.
The negative signs cancel out.
k log capital omega.
For an isolated system.
You've seen this before, probably.
This is called the Boltzmann equation.
And that is what is on his tombstone.
If you go to - I think it's in Germany somewhere.
Is it in Germany?
Do you guys know?
Austria
Austria, thank you.
I knew it was that part of the world.
If you go to Austria, to some famous cemetery, and go look for the tombstone that says S is equal to log omega.
That's where Boltzmann is buried.
OK.
So this picture of -- yes question?
Can you repeat the argument for making that one over omega go away?
Making the one over omega go away here?
Because you're taking the sum of all states from one, i equals one to omega.
So it's one over our omega plus one over omega plus one over omega plus -- omega times.
And this is just a number.
So it comes out.
It's not in the sum.
So the sum, here, is all by itself.
Any other questions?
OK. That's why we talk about entropy as being this fundamental property that tells you about the number of available states.
That's what it is.
You've got this connection now between this variable, which is sort of hard to really intuitively understand, when you're talking about thermodynamics.
And this is much easier to understand here, in terms of the available ways of distributing your energy, or your particles, in this case here.
In different bins.
OK any questions?
So the next topic is we're going to work a little bit with the partition functions.
And see how when you have systems that have multiple degrees of freedom, where each degree of freedom has a different kind of energy.
Let's say translation, rotation, vibration.
Then you can have a partition function for each of these degrees of freedom.
And whereas the energies of the degrees of freedom add up, the partition functions get multiplied.
So it's the separation of the partition functions into subsystem partition functions.
So so far, we've written for translation partition function.
That the system partition function is the molecular partition function to the Nth power.
If you have distinguishable particles.
And you have to divide by N factorial.
If you can swap particles without knowing the difference.
This is the number of ways of swapping N particles with each other, so it's indistinguishable particles.
OK.
So now let's say that -- Let's just make sure that -- Let's say that your energy of your system -- yes?
It says that when system's not isolated, [UNINTELLIGIBLE]
Where does it say that?
In the notes?
[UNINTELLIGIBLE]
Let me see the notes here.
What does it say?
So that sentence has to do with this guy here.
Basically it says that if you've got a huge number particles, the average energy is a given number.
And the fluctuations around that average are very small.
And so, the system behaves as if it's isolated.
So when you have a system which is not isolated, then energy can come in and out of the system.
So in principle, over time, you could have huge energy fluctuations.
As energy comes out, or energy comes in.
And if you have a countable number of molecules in your system, then if one molecule suddenly captures a lot of energy, then the whole system energy will go up a lot.
But if you have ten to the 24th molecules, if one molecule suddenly gains a lot of energy, the system energy doesn't care.
So small fluctuations -- or big fluctuations in the small number of molecules doesn't make any difference to the total energy.
And so you can still use this then.
It's good enough
So how long is that good enough [UNINTELLIGIBLE]
Well if it's accountable, a handful of things, and it's not valid.
If it's ten to the 24th and it's valid, and somewhere in between it breaks down.
Then I don't know what the answer is.
But usually if you have a thermodynamic system, then it's big enough.
That's what thermodynamics is about.
Where you don't really care that you have atoms there.
You don't even know you have atoms there.
So it's big enough.
Good question.
Alright, so now let's take our microstate energy here.
And our microstate energy is the sum of all the molecular energies Ei.
So it's the sum of all energies E sub, over all the atoms.
And each one of these energies, if it's a molecular energies, can be indexed by a quantum number of some sort.
So it would be the sum over all energies.
So quantum number for particle one, n1 is some sort of quantum number.
n2 is some sort of quantum number.
n3 is some sort of quantum number.
And then you have all the molecules.
En1 plus En2 plus En3, et cetera.
So this is the energy from molecule one, energy from molecule two, energy from molecule three, energy from molecule four.
And that little n tells you which energy state that molecule is.
And the sum of all these energies is your microstate energy.
As long as you can write this this way, then you're allowed to write this this way.
So that basically means that they're not interacting with each other.
They're independent from each other.
In this case here.
So now if I write Q in terms of the sum over all microstates Ei.
e to the minus Ei over kT.
I'm going to replace this Ei here with the sum over all these energies here.
And so the sum over all microstates, then, becomes the sum over all possible combinations of quantum numbers.
n1, n2, n3, n4, et cetera.
All the possible ways of getting molecule one in some state.
All the possible ways of getting molecule two in some quantum number state.
And then e to the minus -- and instead of capital Ei, I'm going to write the molecular energies.
En1, plus En2, plus epsilon n3 plus et cetera.
And then divide by kT.
Basically I'm going to prove that this is a fine statement to make, as long as you can write the energy as a sum of component energies.
OK so now this term here, e to the minus En1, only cares about this sum here.
En2, that's molecule number two, only cares about this sum here.
Molecule number three only cares about the sum over all possible quantum numbers connected to molecule number three.
So I can factor out all these sums into a factor of sums.
Is equal to the sum over quantum number n1.
e to the minus epsilon of n1, divided by kT.
Times n2 e to the minus epsilon n2 over kT, times n3 e to the minus epsilon n3 divided by kT, et cetera.
And now each one of these is basically the molecular partition function.
These are all the possible energies of that molecule.
And the sum over all possible energies times e to the minus E over kT is the partition function for the molecule.
So we have q for molecule one times q for molecule two.
And they're all the same.
There are N of them.
Plus q to the N. So just, in a way, clarified that the reason why we're able to write this system partition function, in terms of the molecular partition functions, with N of them, to the Nth power, is because we were able to separate out the energy here.
In terms of the independent molecular energies.
Where this is saying the molecules don't interact with each other.
And are independent from each other.
And then the one over N factorial comes in, so that you don't over count, for translation, the positions.
They're indistinguishable.
OK.
So now we can have -- Actually we're not -- This basic concept of the partition function multiplying each other, if the energies add, is not limited to going from the molecular partition functions to the system partition function.
You can also look at the molecular partition function itself.
And if the energy, the molecular energy, can be written in terms of a sum of energies of different degrees of freedom, for instance, the energy of a molecule could be the energy of the vibration, plus the energy of the translation, plus the energy of the rotation, plus the energy of the magnetic field, plus the energy of the electric field.
et cetera.
You have many energies that can add up with each other to create the molecular energy.
And what we're going to be able to write, then, is that this molecular partition function itself can be written in terms of a product of partition functions for the sub parts of the molecular energy.
So let me clarify that statement.
So if I can write my molecular energy, epsilon, is equal to a translational energy, plus a vibrational energy, plus a rotational energy, plus every other little energies that you can think of that are independent of each other.
Then using the same argument we used to show that Q is the multiplication of these molecular partition function, we can write that the molecular partition function, little q, is just the multiplication of the degree of freedom partition functions -- molecular partition functions.
The translational partition function times the vibrational partition function, times the rotational partition function, et cetera.
If the energies add, then the partition functions multiply each other.
And that's going to be powerful because when we look at something like a polymer or DNA or protein or something, in solution.
And we're going to be looking at the configurations possible for that polymer or that biopolymer, then we'll know that the energy of that polymer in solution is going to be -- we'll be able to approximate it as the energy of the configuration for that polymer.
The different ways that you can fold the protein, for instance.
Plus everything else.
The energy of everything else.
OK.
So if the configurational energy can be separated from the sum of all vibration energies of all the bonds in that polymer.
The way that polymer interacts with -- The way that the solution itself interacts with itself.
Then if we can do this and we can do this approximation most of the time, then we'll be able to take the partition function for the polymer, and write it as the configurational partition function times the partition function for everything else.
And we'll find that this part here will tend to factor out.
We won't have to worry about it.
And that this will carry all the important information that we'll need to know to see about changes in the system.
Changes in Gibbs free energy, changes in the chemical potential.
Everything will be related to this partition function.
This subsystem.
And because of the fact that you can factor them out, then this thing will end up dropping out.
And this will become the important factor.
OK. Let's do a quick example.
OK.
So this is the example of having a very very, short polymer.
Containing three monomers.
Which can be in two configurations.
And the energies are the same for these two configurations.
So the configurational partition function, which you would generally write as the sum of e to the minus Ei for that configuration, divided by kT, plus e to the -- So we would usually write it as e to the minus epsilon one over kT, plus e to the minus epsilon two over kT, plus et cetera.
They're all the same energy.
And there are two configurations.
The degeneracy is two.
So you can write this as the degeneracy of the configuration, times the energy of the configuration.
And you can set your energy reference to be zero.
You can choose whatever you want it to be.
And zero is a good number.
So that e to the zero is equal to one.
So that configuration partition function is just the degeneracy, which is equal to two, in this case here.
So now let's calculate the molecular and canonical partition functions for an ideal gas of these molecules here.
And it's usually interesting to use a lattice model as a guide.
And so in this lattice model here, you would divide space up into little cells.
Pieces in two dimensions, but in reality it would be three dimensions.
And then you place your molecules in lattice sites.
Something like this.
And then you end up counting the number of ways of arranging the molecules on the lattice.
And let's say that we have N molecules that are in the gas phase.
And the molecular volume, i.e.
the size of the lattice site is v. That's the molecular volume.
And N times v is the total volume.
That's the total volume occupied by the particles.
The total volume is the number of lattice sites.
V is the total volume.
And we're going to assume that all particles, all molecules have the same translation energy.
It's an adequate approximation.
We're going to set that equal to zero.
So all molecules at any position here has the same E translation.
And we're going to set that equal to zero.
So the translational partition function.
The molecular transitional partition function is -- Well there's only one energy.
It's zero.
So we only care about the degeneracy of that molecule.
That molecule could be in here, or it could be here, it could be here, it could be here, it could be anywhere, right?
The number of ways of putting that molecule on the lattice is the number of lattice sites available.
Which is basically the molecular volume.
Which is the total volume divided by the molecular volume.
The total volume is -- right.
So the total volume is the number of lattice sites times the volume of each lattice site.
So the total volume divided by the small volume is the total number of lattice sites.
And the number of choices of putting that one molecule is anywhere on the lattice.
That's your degeneracy.
So now if I look at the total molecular partition function, it's going to be the multiplication of the configurational partition function and the translational partition function.
At each site, the molecule could have two configurations.
So q, for the molecule, that's q translational.
I'm going to ignore all vibrations, rotation, et cetera.
I'm going to assume that there are two degrees of freedom.
Translational one, which is basically the positional one.
And then the configurational one, which is internal to the molecule.
So this one is V over v. That's the degeneracy.
Capital V over little v. The degeneracy of placing the molecule on the lattice.
And the configuration is the degeneracy of how the molecule folds.
Yes
[UNINTELLIGIBLE]
The notes are wrong.
So usually capital V is large and little v is small.
So in the notes, if we have it in reverse, we should fix that.
This is lecture 25, right?
So we have q.
No the notes seem to be right.
Total volume is capital V, molecular volume is little v. Where is it that wrong in the notes?
[UNINTELLIGIBLE]
Oh look at that.
My notes are different than yours.
My notes are right.
OK well it's obviously right on the web.
Because this is the latest version.
Alright so flip those big V's and little v's then.
Huh.
I thought they were from the same pile.
OK.
So this is your molecular partition function.
And then when you look at the system, the system partition function can also be separated into a translation and the configuration for the system.
We know what you need to do is take all the molecular partition functions, the transitional ones, and -- to the N factor.
The number of particles.
But now you have degeneracy.
You've over counted.
So you need to divide this N factorial.
You also have the system partition function for the configurations.
And that's q configuration to the N. Except here we don't need to divide by one over N factorial, because we're not over counting here.
The over counting only happens when you're placing identical particles in a lattice, and you can swap them without making a difference.
Here we talking about configurations.
When we're talking about configurations, we're not talking about placing the identical particles in different spots.
We're just looking at these two configurations here.
And then the next particle is two configurations.
The next particle is two configurations.
So this is really important.
That this N over factorial only comes into play when you're talking about the translational degree of freedom.
Not the other degrees of freedom.
And now the total system partition function is the multiplication of these two.
It ends up being capital V over v, to the N power, over N factorial, times two to the N. OK.
In general you could extend this analysis to include vibrations, rotations, energy in a magnetical field, electric field, et cetera.
Any questions?
Alright the next thing that you're going to do then is to use this concept, as a sort of example, as a way to begin to calculate things that you've already calculated before.
For instance, if you look at an expansion of an ideal gas, can we now calculate the entropy change.
Not based on thermodynamics, but based on the statistical mechanics.
On the microscopic description that we've just gone through.
And it turns out that that's what happens.
That you can do that.
You get the same answer.
Thank god you get the same answer.
Otherwise we'd be in big trouble.
So I'm just going to set up the problem, because I won't have time to do it.
And then you can do it next time, when Keith comes back.
So the problem is going to be the usual problem of having a volume V1 of a gas on one side, and a vaccuum expanding to volume V2, gas.
And asking what is the entropy change.
And you know that from thermo, that delta S in this case here is nR log V2 over V1, when the temperature is constant.
That's going to be our answer.
It has to be our answer.
But this time, instead of knowing the answer, we're going to calculate it from microscopics.
So what you do is you start out with your initial state.
You ask what is -- you write down the molecular volume V. The total volume here is V1.
You assume that all molecules have the same translational energy.
You set that equal to zero.
The system translational energy is equal to zero.
And so the entropy for this gas here is just the number of ways of placing the molecule in the lattice.
This model that we have of space being separated into little cells.
And so S is k log omega, where omega is the number of ways of placing the molecules in the lattice.
Which is basically k log V, over v. Where this is the number of lattice sites.
OK. And what you're going to do next time, then, is start from here.
Calculate what it is before.
Calculate what it is after, and turn the crank, and get to the right answer.
Then you're going to do the same thing for liquids.
And that'll be it for this simple statistical mechanics.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And last time, you got to see how you can derive macroscopic thermodynamic results from the microscopic point of view of statistical mechanics.
So in contrast to the way we'd gone through the course from the beginning, starting from empirical macroscopic observation and ideas through macroscopic laws and then working out the consequences of them, in statistical mechanics you start from a microscopic model, build up the energy levels that are going to give you the terms in the partition function, determine the partition function, and what you saw last time is that given that, you can calculate everything.
You can calculate all the macroscopic properties that ordinarily come from the thermodynamic laws that were based on empirical macroscopic observation.
So today, I just want to continue doing some statistical thermodynamics.
Basically going through a few examples, just to see how it plays out when we calculate thermodynamic quantities, based on our microscopic picture.
So let's just look at a few cases.
And I'll start with a couple of examples that are entirely entropy driven.
And we've seen them before from the macroscopic point of view.
What we should hope, of course, is that we can derive those results from the partition functions that are appropriate here.
So, we'll look at entropically driven processes.
And the first one, the simplest one, maybe, to cover, is just free expansion of a gas.
So what happens if we've got particles in a gas that are enclosed in a certain volume.
And now we let the volume expand into vacuum on the other side.
Of course, you know what'll happen.
The volume will be filled and we've derived the thermodynamics for it before.
So let's see what happens in a statistical mechanical treatment.
So let's say, here's V1, there's our gas.
There's vacuum.
And now we'll open it up and just have the bigger volume, V2, and let the gas fill it.
So I think last time you got introduced to basically a lattice model for translational motion.
And I started in on this just a little bit the time before also.
So this is a simple way to describe the statistical mechanics of filling an open volume.
Filling a space with molecules.
And all it does is say that we're going to divide up our available volume into little bits that are basically the size of an atom or a molecule, or whatever the particle is in the gas phase.
And say, OK, there are maybe 10 of the 30th of those little volumes.
The little volume elements are going to be on the order of an angstrom cubed if it's an atom, a little bit bigger if it's a molecule.
And the available volume is on the order of meters cubed.
So that works out, an angstrom is 10 to the minus 10 meters.
And then the cube of that is 10 to the minus 30th.
So if you look at the total number of little volume elements of this sort, it's on the order of 10 to the 30th.
So, the total volume, capital V, let's make that distinction clear.
And in this case, all the states of the system have equal energies.
So in other words, it doesn't matter we're dividing our volume up into imagined little volume elements.
And for both the molecular and the system states, the energy is the same.
It doesn't matter whether the molecules are here, here, here and so forth.
They're not interacting in this picture at all.
So it doesn't matter how close or far they are from each other.
They can't be in the same volume.
So all the energies are the same.
And what that means is, in the partition function, which is a sum over all these terms with these Boltzmann factors.
That have e to the minus energy over kT.
But energy's all the same in every one of the terms.
So in order to determine the partition function, to determine Q, all we have to do is count up the total number of states that are available.
So, our energy, our molecular translational energy, we'll just set it to zero.
Same with our system translational energy.
We'll set that to zero.
All the microscopic available states, that is, if I take an individual particle and I say where it can be, all those states have the same energy.
If I take a microstate of the system, you know, the whole collection of the particles, all of those states also have the same energy.
So what that means is that little q is just equal to capital V over little v, right?
How many states are there?
Well, I've got my little volume, and then however many individual cells there are, that's the number of available states in this model.
So it's big V over little v, on the order of 10 to the 30th.
And then, big Q, the canonical partition function for the whole system, it's something that we've been through before.
It's this process of saying, well, you take the first atom or molecule, and it has any one of these possible states.
So it's on the order of 10 to the 30th possibilities.
Then where does the second one go?
Well, there are basically 10 to the 30th more possibilities.
And the third one has 10 to the 30th.
Since there are so many fewer atoms or molecules then there are volume elements, when we're dealing with the gas phase, we don't have to worry about the fact that, well the first million of the atoms filled some of the sites.
So the next ones have fewer sites available to them.
It's true, but it's such a small fraction that are ever filled that we don't need to worry about it.
So Q is just little q to the capital N power, the number of particles.
And then we've seen you have to divide by N factorial to avoid the overcounting of configurations that are in fact not distinguishable.
The whole idea that maybe there's an atom here and another one here.
That configuration is indistinguishable from the configuration with those two atoms reversed if we're dealing with a mole of identical atoms.
So, OK.
There's our capital Q.
And that's also our system degeneracy.
So the degeneracy, the little g for the molecular degeneracy, how many molecular states are there of a certain energy.
Well, it's the same thing as q.
And it's the same thing here for the system states.
The capital Omega is just equal to that.
Well, now let's look at what happens when we do this.
When we expand from volume V1 to volume V2.
So that capital V is going to change.
So capital V1 goes to capital V2.
And we know it's entropically driven.
Let's calculate the change in entropy.
So delta S is just k log capital Omega 2 minus k log capital Omega 1.
Now, in the process of just seeing how the development goes, how you can derive all these thermodynamic quantities from the partition function, one of the really most central results concerns the entropy.
How you can describe the entropy in terms of the probabilities that the different states are occupied.
And how, in the case like this, where the states all have the same probability of being occupied, then you have this very simplified form for the entropy.
Just, if S is k log capital Omega, where capital Omega is the degeneracy.
The number of system states of that energy.
An amazing result.
You told them about Boltzmann's tombstone and so forth.
So it's on there.
In fact, the ill acceptance of that result kind of led to the tombstone being erected when it did.
Boltzmann, depressed over the lack of acceptance of his theories, put himself to an early death.
And this was off a big part of the reason.
But we, generations later, have come to accept the results that concerned him so deeply.
So this is our change in entropy, when we just have this expansion of gas from V1 to V2.
So it's just k log omega 2, omega 1.
So now let's just put in our results for the volumes.
It's k log V2 over v, I'm exaggerating the size there.
To the N, over N factorial, over V1 over v to the N divided by N factorial.
So this is going to turn out to be a relatively simple case, because the factorials are going to cancel.
So, then we just have N k, log V2 over V1.
Terrific, right?
Now, remember, k, the Boltzmann constant, is just the ideal gas constant per molecule rather than per mole.
So this is the same thing as little N, the number of moles, times R, times log V2 over V1.
And that should be a familiar result.
That's the change in entropy in expansion, free expansion of a gas, from V1 to V2.
So now we've been able to derive that just based on this really simple molecular picture.
Based on that plus the result that Boltzmann fought and paid so dearly for, so that we would have it and understand it.
As before, of course note it's greater than zero.
If the volume, if we're expanding into a bigger volume than before.
The entropy goes up.
Also notice one of the other results that you've seen is that you could relate the partition function to A, right?
The Helmholtz free energy is minus k T log Q.
And S is negative dA/dT.
And so this would immediately give us the same result, because omega's going to go in here.
So we could get this result from another pathway, too, but a more simple and direct way is just to start directly from the Boltzmann result for entropy.
Any questions?
Alright, let's go one step more complex.
Let's now look at the entropy of mixing.
So now we've got two species.
Rather than one expanding into a free volume, and we're going to open a barrier between them and see them mix.
And again, it's still entropically driven.
And we should be able to calculate the entropy change that we saw before from a macroscopic perspective.
So, let's take a look.
So now, we have NA molecules of A in some volume VA. And NB molecules of B, in some volume VB.
And then we're going to open it up.
Then we've got capital N, NA plus NB.
And capital V. VA plus VB, right?
And the whole thing is happening at constant pressure.
Well, then let's look at the initial and final expressions for the entropy.
So, S1, at the beginning.
It's just k log omega A plus k log omega B.
And that's k log VA over little v to the NA power.
Divided by NA factorial times the same thing for B. VB over little v to the NB power over NB factorial.
So that's our initial entropy.
The sum of the two entropy contributions, from the two sides.
And S2, the final one, is just k log omega for the whole shebang.
And that's k log.
And now we have capital V over little v to the N power.
And now we have a combinatorics result that I think probably you're all familiar with from one context or another.
That is, now we have to divide by NA factorial times NB factorial.
In other words, the amount that we need to divide by to avoid overcounting is the product of those two factorials.
Just in case it's been a while since you've seen stuff like that, I've got a couple of pages further on the notes at the bottom of the page.
I've just broken that out for a really simple example where there are just two molecules of A, and two molecules of B represented by different little balls in lattice sites.
And I just worked it out for the total where there are only ten total in the mixture.
The point is, now what happens is, when you start filling a lattice, but it's not all the same, right?
So now you have A here.
And A here.
And B here.
And B here.
So, of course if you interchange A and B here, you don't have an identical state.
An indistinguishable state.
That's distinguishable and needs to be counted.
It's only interchanging B with B that you need to correct for, and interchanging A with A.
So of course you need to account for that in a way that's different from how it turns out if every one of the molecules is identical.
So it's the product of these factorials.
In the numerator.
So now delta S is then just this minus this.
So it's k log V over little v to the NA plus NB over VA divided by little v to the NA power VB divided by little v to the NB power.
And, let's see, maybe I'd better back up.
Let me know, that's all.
And what's happening is, again, luckily, these factorials are canceling.
So we just have k log V to the NA, V to the NB over VA to the NA VB to the NB.
And now we're going to go back and use the fact that the pressures are the same, all the way through.
And what that means is that the volumes must be in the same ratio as the number of molecules.
And what that means is that, for example, VA over V is the same as NA over N. And that's just equal to XA, the mole fraction of A.
And the same for B.
So we can substitute that.
And know our delta S just becomes k log, let's see, let me break this out from the beginning.
And just take minus k log on XA to the NA power minus k log XB to the NB power.
All I've done make these substitutions here.
For the ratios of the volume.
So it's just minus N k XA log XA plus XB log XB.
Look familiar?
Same thing we saw macroscopically before.
So, again, we can still use our microscopic model.
And continue to derive the macroscopic entropy changes.
And, of course, for many of these we can still get our delta G and so forth.
Of course, it's only entropy that's going to contribute.
OK, let's look at just one more entropy driven problem.
And that is, it's a little bit different from these.
Let's look at the entropy mixing in a liquid.
And the difference between the gas and the liquid is that in the case of the liquid, every single cell is filled.
It's not like with the gas.
And that makes a difference in how the states need to be counted.
Because here, remember how I said when we were filling this lattice.
Well, the first molecules goes somewhere.
And there are 10 to the 30th possibilities.
And the second molecule goes somewhere and there are 10 to the 30th possibilities.
And there are always 10 to the 30th possibilities because there's so few molecules that you never have to worry about the fact that it's getting filled up, so there would become fewer possibilities for that last molecule.
And the reason again is because there are so few molecules that essentially, all the cells are open and available even to the last molecule.
Because maybe there's a mole of molecules.
Maybe there are 10 to the 24 molecules.
And there are 10 to the 30th lattice sites.
So there might be one in a million lattice sites occupied, even at the very end of the procedure.
But of course for the liquid, that's definitely not the case.
All the available volume is filled.
So by the time you get to that last molecule, it has one and only one space it can go into.
So you have to count for the diminishment of the available sites as molecules are placed in them.
OK.
So it's a different kind of combinatorics problem that results.
So, OK. Let's look at liquid mixture.
So now, it's going to look like, and they're all filled up.
I sure used fewer sites maybe, but that'll be easier here.
We've got open circles.
So these two are going to mix.
And now we're going to have filled mixture.
So we're going to have a bigger volume, but it's still going to be filled and all the positions are random.
So we have to consider all the possible configurations that we have.
OK.
So we still should go through the same part of the procedure.
So let's call this A.
And this, B.
So at first, already the situation is different.
If we say what's the entropy of system A.
Well, in this simple model the entropy of the pure liquid is zero.
Now, of course, that's not realistic.
But it's going to turn out to be suitable.
Because the change in entropy that we go from here to the mixture.
The other contributions to entropy are going to more or less be the same from beginning to end.
What's going to be significantly different is simply the fact that in the mixture you have the random positions that can be occupied.
So that's going to be the term that matters, the contribution that matters.
So entropy is just k log Omega A.
There's only one system state.
All of the lattice sites are filled, and they're all filled with A.
So there's only one way to do it.
And of course, it doesn't matter if you interchange them.
They're indistinguishable.
So this is zero.
This is one.
There's only one state.
Same, of course, for B.
So that's our initial entropy in this simple model.
Well then, delta S of the mixture is just S of the mixture.
We don't need to worry about the original parts.
Now, this is going to be k log of N factorial over NA factorial NB factorial, like we've seen before.
That's not different from before.
Except that unlike before, we don't have the convenient and simple cancellation of the factorials.
That happened before, because they were there.
In other words, they were there in the entropy contribution in the gas phase as well, in the pure gas.
They're only there here in the liquid.
And that means we have to deal with them.
But it turns out it's not too bad to deal with them.
And we're going to use this Stirling's approximation that I introduced before.
So the log of N factorial is approximately equal to N log N minus N. So let's just break that out then, and use the Stirling's approximation for each of the factorial terms.
So then we have S for the mixture is N k log N minus N k minus.
And now we'll do the bottom, the numerator.
NA k log NA minus NA times k, that's this part.
And now we'll do this one.
So it's plus NB k log NB minus NB times k. But NA and NB, of course, are just N. So these will cancel.
So then we're left with NA plus NB times k log N minus NA -- And I just want to write this this way so that I can then separate the terms.
Minus NA k log NA minus NB k log NB.
And now I just want to combine the easily combined terms.
So it's NA k log N over NA plus NB k log N over NB.
Looks like we're home.
Now we just have N k XA log XA plus XB log XB right?
This NA over N is just XA.
NB over is XB.
And then I've divided the same thing to make XA's over here.
And take the total number of moles out.
Alright.
Look familiar?
Again, the same thing derived now in a way that's a bit different from what we did in the gas phase.
So now I've got the entropy of mixing even in a condensed phase the liquid.
And again, the reason this simple model works is because although a real liquid, certainly a pure liquid has a finite entropy, a substantial amount of disorder, that's present, that kind of disorder.
In other words, the fact that you don't really have the molecules sitting in sites on a lattice but there's disorder.
There's also rotational disorder if it's a molecule.
There are various contributions to entropy.
But those are all comparable in the starting and final states.
The thing that's changed is simply the interchanging of A and B molecules.
That's introduced a new kind of disorder that didn't used to be present.
A very big difference in the number of states at the end from the beginning.
And that's what we've calculated and captured here.
And that's why such a simple model still works OK. Any questions about these entropy driven cases?
OK. Now let's move on and talk about the cases which are more commonly encountered, where the states aren't all the same energy.
Of course, here in the liquid, just like in the gas, we haven't treated interactions between the molecules.
We've assumed that they're equal between A and B, as they are between A and A and B and B.
So in this case then all the energy to the states are the same and it's just a counting problem.
How many states are there available.
As soon as the energies are different, then of course we need to account for all those Boltzmann factors.
Those e to the minus energy over kT factors become part of the, they weight the counting.
And we have to do that.
So, what I want to do is go back to this simple polymer configurational model that we introduced before.
And this sort of model, in one form or another, we're going to see a few times in the rest of this treatment.
The reason, really, is because it's a simple and also realistic way of portraying a system that has a finite number of well-defined energies.
If you don't have that, of course you could do the treatment.
In a, sort of, classical mechanic sense, a continuum of states, all with different energies.
It's hard.
Because then those sum in the partition function become integrals over all the different configurations and possibilities.
It's much more complicated to do.
It's much more straightforward if you can identify discrete states and add over them.
Now, someday you'll take quantum mechanics.
And then you'll see that the states are discrete, even if we're talking about translation or rotation and so forth.
So you'll have discrete quantum mechanical energy levels and so forth.
You haven't had that yet.
And so that's not the starting point that we're going to use.
Instead, we're going to use a starting point that goes through the exact same formalism, which is just the discrete states available in molecules that have multiple configurations.
So we're going to have unequal energy states.
And what we're envisioning is molecular or polymer configurations.
So here we have our states.
And just like before, the idea is that if you have two sub-units that are in proximity, there's some kind of favorable interaction.
It could be hydrogen bonding.
Could be different.
But the point is, there's some sort of interaction there that reduces the energy.
And then there are other configurations that just don't have that.
Because the sub-units aren't in proximity to each other.
And it's not hard to work out that in this very simple model, these are the only configurations that are available.
The only distinct configurations.
So then, our molecular energy, E, we can define as zero here.
Before we defined it as minus E int, but I'm going to, it's a little more convenient to make this the zero of energy, and then this is plus the interaction for each of these states.
Of course, we can put the zero of energy wherever we prefer.
And then we have the degeneracy is one.
And in this case it's three.
So now we can just write out the configurational partition function for the molecules and also the canonical partition function for the system.
So q configurational.
And we're just going to sum over the states.
So it's e to the zero over kT, for the lowest state.
And then there are three states that are going to have this term, e to the minus E interaction over kT, where E int is a positive number.
In other words, the probability of any one of these states is a little bit lower than this state.
Because this state has lower energy.
Remember what I mentioned earlier, which is although the probability of any one of these states is lower than the probability of this state, the probability of this energy is likely to be higher than the probability of this energy.
Because there's only one of these states and the degeneracy here is three.
So there are three possibilities in which the molecule could have this energy, and only one this.
So if, basically, kT is bigger than E int, so in other words, if this term isn't very much smaller than one, then of course this will be bigger than this.
So of course this is one plus three e to the minus E int over kT.
And now we have our capital Q, our canonical configurational partition function.
And that's just little q configurational to the Nth power.
And like you've seen so far, in various cases where the molecules are separate non-interacting molecules, this is just the molecular partition function taken capital N times.
If all the molecules are behaving independently, then you sum over all of those states.
And you see that each one of these is just taken again and again.
As we've seen implicitly in all these treatments so far as well.
Now, in the translational case where you interchange particles.
Like particles, you have to divide by N factorial.
.
But the different configurations don't do that.
If I've got a system where a molecule over here is in this configuration and a molecule somewhere else is in this one, and now they change configurations, that's a distinguishable state.
So there's no N factorial involved here.
In the configurational partition function.
So, fine, then it's just one plus three e to the minus E int over kT to the Nth power.
That's Q.
And once we know Q, as you've seen, we know everything.
And so we can immediately start in deriving all of the thermodynamics, right?
And the place to start is A, Helmholtz free energy, or configurational free energy.
Because that's the simplest relation, just as minus kT log of capital Q configurational.
So it's minus N kT one plus three e to the minus E int over kT.
I'm going to rewrite A configurational in terms of beta rather than kT, just because there are going to be a lot of these factors.
So it's minus N kT one plus three e to the minus beta E interaction.
And that's our A.
Everything's going to follow from that.
Before I write some of the other results, one thing to notice is, it has N in it.
In other words, there's a free energy per molecule.
The total free energy is just something times N. And that's because all the molecules in this model are independently behaving.
So whatever their average free energy is, that's going to add up and give us the total.
They're all acting independently.
And in fact, we could have gotten this directly from minus kT log little q configurational.
And if we look at energy, regular energy, u, configurational, it's minus d log capital Q with respect to beta.
V and N. And I'll just write the result.
It's N times three E int e to the minus beta E int over one plus three e to the minus beta E int.
And once again, the feature I want to point out is that there's a factor of N here.
Again, there's an energy per molecule.
And so if we want, we can also just write the average E, little E, configurational.
It's just u configurational over N. It's the same as this without the factor of N. Not only that, again, we could get this directly from the molecular partition function up there.
From little q configurational.
We'd have the same result.
Exactly as it should be.
So if we wrote E as we've seen in the past, it's just the sum over that energy times the probability for each state.
And that's just the sum of Ei, e to the minus beta Ei over little q.
And if you put in the terms, you immediately get this.
Here's our little q, and this has the interaction energy brought out, multiplied here.
So the point is, in a case like this, where you have a bunch of independently behaving particles, the totals for these, for quantities like energy, are simply for the system energy, it's just the average particle energy times the number of particles.
The number of molecules.
And remember, before, I spoke a little bit about the fact that, well, if you look at one individual molecule and another and another, the energy will fluctuate.
It'll vary considerably.
Of course, this is a simple case with only two possible energies.
But in a case where there may be many more possible energies, the molecular energies may vary quite widely.
Still, there will be a well-defined average.
And then the system energy would simply be the total number of molecules times that.
Where they're all independent.
Where all the energies are independent of each other.
And, of course, the system energy fluctuates a great deal less than the individual molecule energies.
OK, so that's our energy term.
And I think I won't write out the results.
They're in your notes for entropy.
For chemical potential.
They're all there.
And again, the same point would hold.
They all scale with N. But I want to talk a little bit about the heat capacity.
The expression for it isn't so simple.
So let me just right Cv configurational.
It's du configurational / dT, at constant V and N. And again, the details are worked out in the notes.
So I won't write out the whole derivation of it.
And even writing out the results, I'm almost reluctant to do it.
But I'll put it up.
Three E int over k T squared, because there are some things I want to point out about it.
N, and then one plus three e to the minus beta E int minus E int e to the minus beta E int.
Minus e to the minus beta, I have to get rid of this.
Minus e to the minus beta E int times negative three E int e to the minus beta E int.
All over one plus three e to the minus beta E int, that whole thing squared.
And this is just, of course, the kind of messy results that comes from taking this derivative with respect to temperature and doing the chain rule to get it with respect to beta and so forth.
So it looks like a little bit of an intractable, or at least a little bit of a complicated, function.
And the detailed functional form is complicated.
But what I want to emphasize is that it has simple limits that are very easy to understand physically.
And that are important to understand for lots of systems.
So I just want to look at the limits of the heat capacity at low and high temperature.
And this is something that recurs in statistical mechanics, in an enormous number of systems where you have simplified limits.
And they're really important.
Because what's going to matter is this.
Maybe I shouldn't have covered up those configurations.
So there's a big difference in what happens when kT is much bigger than this energy.
Where you know that under those conditions - let's say it's hot, and this is a small energy difference.
Then the probabilities are essentially equal.
That the molecules are in this state or in any of these states.
Because the energy is so tiny compared to the thermal energy.
The molecules are continually being kicked around between all the states, among all those states.
So the high temperature limit, physically, is one that's simple to understand.
Where now you're really just going to revert to the cases that we've treated before, where all the energies are effectively the same.
Because compared to kT, they are.
And in the low temperature limit, now go to the opposite limit.
Let's say kT is much, much, smaller than the interaction energy.
So that now this term is really small.
Because this is much bigger than this.
So, what it means physically is all the molecules are in the ground state.
The probability of this is basically one.
The probability of being in any of these states is zero.
And that's also a simple result.
And there are lots and lots of cases where one of those limits really obtains.
If you look at molecules moving around in the gas phase or in a liquid.
And you say, well, OK, let's think about their rotational motion.
Rotational energies are small.
At room temperature, kT's much bigger than them.
You immediately go to the high temperature limit.
Now let's go to a small molecule and look at the vibrational energy.
In most cases, the vibrational frequency is pretty, molecules are pretty stiff.
And in many cases you can just say, look, forget it.
All the molecules are in the ground state.
And again, in the opposite, but also simple limit ends up holding.
Or molecular electronic states, right?
If you've got benzene or hydrogen atoms in, say, at room temperature, how many hydrogen atoms thermally are going to be up in the n equals two state, the 2p state or the 2s state?
Forget it.
There's not nearly enough thermal energy to do that.
So these simpler limiting cases play a huge role in simplifying statistical mechanics and the calculations from them generally.
OK, so let's just see what happens.
So, this we want, so this we can move down.
So our limiting cases, when T goes to zero, not going to work out what happens to all the betas, beta gets big in that case.
But the result is that Cv configurational goes to zero.
That's the limit where, like I've described, here is E interaction, or E int.
Here is zero.
And your kT is barely above zero.
So when all the molecules are here, none of them has enough thermal energy to be up here.
So why should the heat capacity be zero?
The heat capacity is du/dT.
It's zero because, here's kT.
Let's say I increment kT up a little bit.
I just heat the system a tiny, tiny bit.
An infinitesimal amount, to look at the derivative.
Once I do this, how many molecules are in this higher level now?
Still zero, right?
There's still not nearly enough thermal energy to have any molecules up here.
So what, was the derivative of the energy with respect to the temperature?
I changed the temperature.
The energy didn't change a bit.
And that means the heat capacity is zero.
Now let's look at the other limit.
High temperature limit.
Really, it doesn't need to be infinity.
It's really just T greater than, much bigger than, kT is much bigger than E interaction.
And in this case, kT is much less than the interaction energy.
So it doesn't need to be nearly that extreme.
Well, what you find out is the heat capacity is zero.
Now, it's zero.
Because we're in the following limit.
Now kT is way up here.
Compared to kT, E interaction is way down here.
So, what happens?
It means that this is so hot, that term, The e to the minus E int over kT, forget it, it's one.
And so the probability, in other words, that the molecules are in this state or this state are essentially equal.
So now let's say I raise the temperature a little bit higher.
What happens to those probabilities?
What changes?
Right.
Nothing changes.
So what happened to the energy when I raised the temperature?
Of course, nothing happened to it.
Which means the heat capacity is zero.
Now, the first limit that I described, this one is almost universal.
For any system where you have quantized level, you can always eventually get to a low enough temperature that you're in the first limit.
Where the kT is lower than, by far, than the first excited level.
Everything's in the ground state.
And so you reach this limit.
This one, it's only cases where you have a finite number of levels.
Then you have the same high temperature limit to it.
In other words, the reason this result happens is because there aren't a bunch of other levels up here that eventually get to be comparable to kT.
And not all kinds of systems or degrees of freedom are like that.
This one is, because there's a finite number, four in this case, of configurations.
So there's just nothing above there.
There's another really important kind of degree of freedom like that.
And that's spin.
Think of proton spins.
It's plus 1/2 or it's minus 1/2, and that's it.
You can't put any more spin energy into it.
Just like you can't put any more configurational energy into the system than to be in this state.
That's it.
So in other words, the maximum possible energy is finite.
Of course, lots of other degrees of freedom are different from that.
If you think a molecule's rotating, they could always spin faster.
Vibrating, they can always vibrate harder, translate faster.
Those degrees of freedom won't have this limit.
They also will have a simple high temperature limit, but not zero, because, of course, if there are always more levels, and I keep increasing kT, then I'll have thermal energy to go into those higher and higher levels.
It'll still go up.
But for systems with a finite number of possible levels, and a finite amount of total energy, for degrees of freedom like that, once I get to a temperature higher than any of that stuff, then forget it.
You can't change the energy anymore thermally.
So your heat capacity is zero.
You can change the temperature and nothing further happens.
OK, next time we'll see what happens when you do have continuing basically unbounded possible energies.
The following content is provided under a Creative creative commons license.
Your support will help MIT Open Courseware continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last few lectures, and we started to see some of the consequences and looked at simple changes, transformations of expansion and mixtures of liquids and gases.
And saw how in the framework of statistical mechanics, we could derive the thermodynamic results that you saw before, based on an empirical framework.
On the thermodynamic framework we've been working with all term.
But you saw them derived from a microscopic perspective.
Now what I want to do is move to examples that are more commonly encountered in chemistry.
One that we did last time actually was very common, or is at least a prototype for something common.
That is we looked at what would be a simple model for a polymer with different configurations available to it.
And just tried to understand what the basic thermodynamics is that it would exhibit.
What the heat capacities would be in various limiting cases.
High and low temperature.
I want to continue that today.
But for an even more common case.
So the model that I want to lay out is indicated on your notes for today.
And essentially, you could look at it as double stranded DNA.
Any kind of polymer with a whole variety of configurations.
Ultimately what's going to matter is what is the set of energy levels available to the system.
And so here's what it looks like.
The way I've tried to depict it.
I'm imagining something like, covalently bonded chains that have links between them.
So of course, it's reminiscent of DNA with hydrogen bonding between the strands.
And the idea in this simple model is that you can have various numbers of these hydrogen bonds.
So this is the case that would be the most energetically stable.
That is, all the available hydrogen bonds are formed between neighboring pairs.
So this would be our lowest energy.
Now let's starts breaking some hydrogen bonds.
And what I'm imagining is they break starting at one end and working their way down.
So the next highest energy state would be like this.
So now these are still hydrogen bonded.
This is not.
And there's a cost in energy for breaking that hydrogen bond.
We'll call it epsilon zero.
It's a positive number.
So that this is higher energy than this.
And now we'll go to the next one, which could have two broken bonds.
Broken hydrogen bonds.
And so on.
And if we imagine that this is very long, there may be many many, members of the chain.
Then of course this sequence of structures with corresponding energies might go on for a very long way.
So this is a simple construction that gives us a set of states with distinct energies.
And so we have a series of configurational energy levels.
They're evenly spaced in this model.
So there's zero, E zero, two E zero.
And so on.
So those are available energy states.
And now, just based on that simple model, we should be able to figure out all the thermodynamics.
We should should be able to figure out what the equilibrium energy is at a particular temperature.
What the average energy is.
And so forth.
And we can do it in a straightforward way.
So we'll start with our molecular partition function, as always.
So it's q configurational.
And it's just a sum over our available states with their associated Boltzmann factors.
So sum over n of e to the minus En over kT.
Now the way I've modeled this, there are no degeneracies.
Each energy level has just one microscopic state of the molecule corresponding to it.
So there are no molecular degeneracies.
Turns out we'll be able to put this into a simple form.
So the first thing is, let's approximate that we could take the sum -- not to some finite level, which would be the case if there's a finite number of elements in the chain.
But all the way through infinity.
And the idea here is the following.
We'll assume that the chain at least has a great many members.
Even if it's finite.
And then at some point, this becomes a really small number.
Even if epsilon naught, the energy of one of the bonds, is less than kT.
Once we multiply it by a very large number n, it becomes very much greater than kT.
So the underlying assumption here is, well there will not be a significant number of molecules in the highest energy state anyway.
In other words, the terms higher than a finite but large value of n between there and infinity are going to be so small that those terms will contribute negligibly to the sum anyway.
And on that basis, we could make this approximation.
And the incentive to make this approximation and carry the sum to infinity is that then we can put this in what turns out to be a very simple closed form.
So now if we just write out a few of the individual terms in the sum, it's one plus e to the minus epsilon zero over kT.
Plus e to the minus epsilon zero over kT.
I'll write this as squared.
e to the minus two epsilon zero over k t. And so on.
So this has the form of a sum that looks like one plus x, plus x squared, and so on.
And because the sum goes to infinity, we can simply write it as one over one minus x.
Sort of a very simple result.
And now putting this back into x, it's one over one minus e to the minus epsilon zero over kT.
That's our q zero.
Our molecular partition function.
So this model yields a particularly simple result for q zero.
And then everything else follows from there.
So then we can write the canonical partition function capital Qn.
It's just q configurational to the capital Nth power, where capital N is the total number of these molecules.
So it's just one over one minus e to the minus epsilon over kT to the Nth power.
And then we can start writing out the results for the various thermodynamic properties.
So A configurational is minus NkT, log q configurational.
That is, remember, A is minus kT log capital Q.
And this is just going to come straight out.
So it's minus NkT log of little q.
Minus NkT, log of one over one minus e to the minus epsilon over kT.
Or positive NkT, log of one minus e to the minus epsilon over kT.
So a pretty simple form for the Helmholtz free energy.
And I'm not going to write out all of the individuals thermodynamic terms, but I'll write a few of them.
So mu, the chemical potential for these configurations, is just dA/dN.
With T and V constant.
And so the only capital N dependence, the only dependence on the number of molecules is multiplicative.
Right here.
So this just gives us kT log of one minus e to the minus epsilon naught over kT.
And the important point to realize here that we mentioned last time too, in the examples we considered then, especially the last one, is that because the only place that N figures in is as a multiplicative factor.
What this is telling us is that we just have a chemical potential, of Helmholtz free energy per molecule.
The molecules are all independent.
The total energy doesn't depend on any interaction between this molecule and its neighbor somewhere else.
So terms like this are simply additive.
So if we work out the chemical potential, it's just one over N times A.
And I'll just write the result for u configurational.
Because I do want to look at the heat capacity.
So it's Nk T squared, d log of little q configurational / dT.
With N and V held constant.
And it turns out to just be N epsilon zero.
One over E zero over kT minus one.
So now we can look at the heat capacity.
Cv configurational.
So it's du configurational / dT.
With constant N and V. And taking this derivative with respect to temperature.
Gives us an expression of the following form.
It's Nk epsilon zero over k T squared.
e to the E zero over kT, over e to the E zero over kT minus one squared.
Just taking the derivative in the usual way.
Now, this looks kind of complicated for the heat capacity.
But let's take a look, just like we did last time for the simpler case that we treated then, let's take a look at the high and low temperature limits of what happens to the heat capacity.
So at high temperature, well we can start by just looking at the energy.
That has a form that's fairly simple.
So when temperature is large, this is small, and we can Taylor expand it.
So then we're just going to have one plus epsilon zero over kT minus one.
Which means the ones will cancel.
And we end up with a fairly simple result.
So u configurational, in that case, is just NkT.
The epsilon zeroes are going to cancel also.
And what that says is that the heat capacity -- and of course we could take the limit of this, but we can just take the derivative of this with respect to temperature more easily.
So Cv configurational.
It's just N times k. In other words, the heat capacity in the high temperature limit is a constant.
Last time we treated a simpler case, where there were only four configurations altogether available, to this sort of simple polymer model that we drew.
Unlike this present model where we're saying there are essentially infinite number of configurations and different energies available.
So many that the highest ones we're never even going to access, because they'll be much higher than kT.
Here it's different, so now we have -- You know before, when we had a limited number of total states, remember what happened in the heat capacity at high temperature.
What was the limiting case?
Finite number of states.
What's the heat capacity at high temperature?
This is going to be on the exam.
What's the heat capacity at high temperature, if there's a finite number of states available?
Zero
It's zero.
What was the low temperature limit of the heat capacity?
Zero
It was also zero.
Right.
And the idea was for a system with a finite number of states.
The idea was -- so let's say, the way we had it before, there was one state with energy zero.
And we had three states with some energy epsilon zero.
And that was it.
Those were all the molecular states available.
So we looked at the two cases.
In one case, when the high temperature limit where kT is much bigger than any of this stuff.
In that limit, the molecules are just equally likely to be in any of these states.
Because there's much, much more thermal energy than this energy difference.
And the molecules are constantly getting kicked around by the available thermal energy.
So if you raise the temperature a little bit more, it doesn't make any difference.
The molecules already are evenly distributed among the states.
There's no additional configurational energy.
So du/dT is zero.
It can't increase any more.
So in that case, the high temperature limiting heat capacity is zero.
And in the other case -- In the low temperature limit, now let's say kT is really low.
It's much less than epsilon zero.
So now we'll redraw this as zero.
And put the epsilon zero states up here.
And kT is here.
Well when it's like this, the temperature is so low that there's not nearly enough thermal energy to populate any of these states.
And if you change the temperature by a little bit, it's still not enough thermal energy to populate any of these states.
So once again, du/dT is zero.
You change the temperature and the configurational energy doesn't change.
So the heat capacity is zero again.
This is why it's so informative to measure heat capacities.
Because you can learn an awful lot about the intrinsic structure of the material.
What are the energy levels available to it?
What do they do?
You can learn a tremendous amount about that by making measurements of the heat capacity.
Well, now we're in a different case.
We have a whole set of evenly spaced energy levels.
Never ends.
So now let's look at the high temperature limit.
But we're never in a limit that's higher than the highest level, because we're assuming it goes on forever.
So it's up here somewhere.
There's kT.
So what happens?
Well if you raise the temperature, there still are higher lying levels that can be populated, and that will get populated.
So du/dT isn't going to be zero in the high temperature limit, in this case.
But it stops changing at some point.
Because nothing's very different about this than having kT be, let's say, up here or up here.
So in the high temperature limit, yes, the energy does change with temperature.
But it changes in the same way at any temperature.
In other words, the energy is just linearly varying with temperature.
And the heat capacity is a constant.
du/dT doesn't change anymore, once you're in the high temperature limit.
Now without me writing any expression on the board, what's the low temperature limit of the heat capacity going to be in this case?
Going to be on the exam
Zero?
Zero, that's still going to be the same, right?
Put kT way down here.
That's just like this, right?
Not near enough energy to populate even though lowest, you know, the first level above the ground state.
Change the temperature a little bit, still not enough thermal energy to get up there.
So the energy doesn't change with temperature in that low temperature limit.
So you can immediately see what's going to happen at low temperature.
Any questions?
Yeah?
[UNINTELLIGIBLE] will still be n k t, and then it's just -- [UNINTELLIGIBLE]
So, of course, that's a limiting case here, right?
That happened because in the limit of high temperature, then this exponent is really small, right?
So then you can Taylor expand it.
In the limit of low temperature, this is really big.
This is nearly zero.
So e epsilon zero is much bigger than kT in that case.
This is big.
This is negligible, right?
Oh.
Wait a minute.
Something's making me real unhappy.
I can't have this right.
Ah.
Yes.
It needs to be zero, is what it needs to be.
What am I thinking?
Of course.
It's one over a huge number.
It's zero.
OK.
So of course you can't expanded it.
It's just, this is much bigger than this.
This is an enormous number at that point.
So in other words, what it's saying is the configurational energy is zero, because everything is stuck in the ground state.
And it stays zero if you vary the temperature.
So you get the zero limiting value for the heat capacity, and the energy itself is also zero.
Other questions?
OK. Let me just make a few comments about the entropy.
So I didn't write it out before.
And I'm tempted not to do it again, but I suppose I'll do it.
So it turns out to be Nk, minus log of one minus e to the minus epsilon zero over kT.
Plus epsilon zero over kT.
Over e to the epsilon zero over kT, minus one.
And this comes from combining the terms for A and u.
It comes from minus A over T. Plus u over T. And what I want to do is look at its limiting cases also.
In particular, what happens here in the limit of high temperature.
And what happens turns out to be k log kT over epsilon zero to the Nth power.
Or put the N over here.
OK. And what this is telling us about is the number of available states.
Roughly, how many states are there that are accessible to the system at some temperature.
So in other words, think of it as -- let's put the N back there.
k log of kT over epsilon zero to the Nth power.
And think of it as k log capital omega.
Where that would be the degeneracy.
Now all the states are in equal energy, but remember for the whole system, remember we discussed this before.
How you have a very, very narrow distribution of system energy states at equilibrium.
So you can think of this as the degeneracy of the system states that are actually going to exist at a particular temperature.
So if we look at the limiting value of the partition function, it's just kT over omega.
Or the same thing for capital Q. kT over epsilon zero, sorry.
To the Nth power.
So you have a very simple expression.
And so again, what this is doing, is it's giving us a measure of about how many states are available.
And it's particularly informative to look at that for the molecular partition function.
What it's telling us is, if I look at kT over epsilon.
And these levels, remember, are evenly spaced.
So here's epsilon zero, two epsilon zero, three epsilon zero, and so on.
It says, you know, if kT is about ten times bigger than epsilon zero.
So this is ten.
It's telling us, roughly how many states does the system have thermal access to.
The molecular state.
It has about ten states.
So going over to our picture of the set of structures, you could have anywhere up to about ten bonds broken.
Now, the individual molecules are going to be in a whole range of states.
Some of them will have fewer than that, and some of them will have more than that number of bonds broken.
But on average, it's going to be about that number.
And then, if you look at the whole system, the number of states available, of course it's astronomical.
Because you have to take each molecule, and say, well it could be in any one of something on the order of ten states.
And then whole set of N other molecules can be in the states they might be in.
Change the first one, and do it again.
So you have this astronomical number of system states.
But remember, like we discussed once before, it'll turn out that although the individual molecule states vary considerably with energy, the system states, which are averaging over some astronomical number of molecules, where capital N is something like ten to the 24th.
Once you average over that many individual molecules, you find that there's very, very little fluctuation in the energy.
In the system energy.
The individual molecule energies vary considerably.
Realistically the variation of the molecular energies -- that variation is going to be comparable to the energy itself.
To the average energy.
So if the average energy is roughly you know, the ten epsilon zero, you say okay, how much might it vary?
Well, there are going to be some molecules that have only a couple of bonds broken, and some that might have 20 bonds broken.
The variation will be on the same order of magnitude as the average itself.
But then you average over capital N of them.
And then you immediately discover that there's very, very, very little variation.
And in particular, what happens then, is, you know, for molecular average energy, epsilon zero, and variance, or standard deviation about the same magnitude.
System average energy is u, and it's going to be capital N times.
Well let's say that -- average -- Oh sorry, let me not put the zero There.
It's just average energy.
And the system average energy is just N times the molecular energy.
But the system variance is going to be on the order of the square root of N times epsilon.
If you've done statistics, then you've seen that sort of result.
You do a bunch of samplings a capital N number of samplings, and the variation looks like the square root of that number.
So what ends up happening then, if you look at the relative variation, you might look and then say, well it's pretty big.
This is ten to the 12, right?
It's still a huge variance.
But let's compare it to the average.
So this is ten to the 24th.
This is ten to the 12th.
It's on the order of ten to the minus 12.
An incredibly tiny fractional variation in the system energy.
You could never -- There would be no practical way to measure it.
And of course that is consistent with what you would expect.
If you say let's measure the configurational energy of a bunch of molecules in a liquid solution, or molecules in a gas floating around.
And it's a mole of them, that total average energy is not going to fluctuate significantly, even though individual molecules that you pick out of that whole system might have quite widely varying energies.
And that's the point.
So it's an incredibly small variation.
Now you can derive this.
I didn't derive this, of course.
I asserted that this is the case.
And it's probably familiar to some of you if you've seen some statistics.
But the way you would derive it is, you know in addition to calculating the average energy, the average of E, you can also calculate the average of the energy squared.
And you can calculate the standard deviation that way.
You calculate the average of E squared.
And then you minus the average of E, the quantity squared.
Take the square root.
That's your root mean square.
And that's what leads to this result.
So it's pretty straight forward to calculate it also.
Alright.
So any questions about just the extent of variation of the individual molecule energies or the system energies?
Okay.
Then, now what I'd like to do is look at another kind of energy that's going to turn out to have exactly the same set of levels that we just derived from this simple model.
And some of you may have seen this before.
Many of you may not have.
And for right now, I'll just assert it.
But it will illustrate why this is so useful.
It turns out that if I've got molecular vibrations -- you know, there's nitrogen and oxygen in the air.
If I look at those nitrogen vibrational energy levels.
Or oxygen energy levels.
They look like this.
Evenly spaced, non degenerate energy levels.
So the model that we've constructed here, based on this simple two-chain polymer, actually gives us a set of energy levels that maps directly onto the vibrational energy levels of a molecule.
So all the results that we've just seen apply, not just for conformations of a polymer, but for vibrations of a molecule.
So everywhere where it says configurational, you can just write in vibrational.
And you'll still be right.
because of course, what matters is what are the states that are available to the system, and what are their energies?
After that, the formalism of statistical mechanics takes over, and calculates partition functions and thermodynamic functions.
The input into that is states.
And their energies.
Well, we have the exact same set of states and energies.
So we immediately arrive at a super important case and its results for molecular vibrations.
And not only molecular vibrations, but vibrations of a crystal lattice.
Acoustic vibrations of a glass.
Or even a liquid.
So there's an enormously wide ranging set of results that we've derived by starting at this simple picture.
And when you take quantum mechanics, you'll see that indeed you do get this set of evenly spaced energy levels.
You may well have seen the results before, and you'll see it derived there.
OK.
Given that, then I want to talk a little bit further about the heat capacity.
So we've seen the limiting cases for the heat capacity.
Namely, the low temperature limit is zero.
That's the case whenever you have quantized levels.
You can always get kT lower by far than the lowest excited state.
So that everything is stuck in the ground state.
How low that is depends on how far apart the states are spaced.
But there's got to be some temperature somewhere that's down there.
So we've seen the heat capacity limiting cases.
Let me just sort of draw them again.
So here's kT much less than epsilon zero.
And up here, kT is much bigger than epsilon zero.
Now I'm just going to sketch the full temperature dependence.
In other words, I'm going to connect those two limits that we've seen.
So here's what that looks like.
And I'll write it as vibrational.
So we know it's got to be zero at the low temperature limit.
And we know that it's this constant value in the high temperature limit.
It's Nk.
And in some way, it smoothly connects.
And if you look at the actual scale, here -- so here's kT over epsilon zero.
Here's the limit of low temperature.
Actually, you don't have to be very high above epsilon zero to be in in this high temperature limit.
So if you look at the plot that's on your notes, this is already -- it's not quite leveling off precisely.
But it's already pretty close.
When kT is just two times epsilon zero.
It's already almost at the limiting case.
Now this has tremendous significance.
So a long time before quantum mechanics was developed, people made measurements of the heat capacities of materials.
And they were familiar with the fact that when you got to ordinary temperature -- room temperature, the heat capacity was temperature independent.
And that was understandable for reasons we'll discuss shortly.
Basically, in that case, the heat capacity -- it's not just Nk for one vibrational mode.
Of course, that's what I describe for a single vibrational mode of, say, a molecule.
If you have a crystal lattice with N atoms in it.
Let's say it's an atomic crystal.
Each atom has three degrees of freedom.
It can move in each of three independent directions.
And there are N of those atoms.
There are 3N total degrees of freedom.
That's how many vibrations the lattice has.
So in fact, you have to multiply this by 3N.
So what would be seen is, you'd have 3R, the gas constant, per mole, in other words.
And if you looked at Cv over 3R, then this value would be one.
Very useful to see that.
To figure that out.
And people understood it.
What people didn't understand is this stuff.
Why did it do that?
That was a great mystery.
Why did the heat capacity go to zero at low temperature?
And the reason they didn't understand it is because the model for vibration was really simple.
The lattice is a bunch of masses and springs.
I know the vibrational energy.
And I can calculate it.
That is, I know the classical vibrational energy.
Remember, the reason it goes to zero is this.
It's all because of the fact that the energy levels are discrete, quantized levels, with gaps in between them.
If I've got classical mechanics, then that means it's vibrational energy.
The energy just gets bigger if the amplitude get bigger.
It's not discrete, it's continuous.
There are always energies in there.
In that case, there's never a situation like this.
Where kT is lower than the first available excited level, and everything's in the ground state.
That never happens in classical mechanics, because there are always levels there.
But it does happen in quantum mechanics.
And Einstein recognized that this was a way to explain this low temperature limiting heat capacity.
So actually if you look at early development of quantum mechanics, really it was all predicated on statistical mechanics.
And the idea that, well, that you could then do the statistical mechanics with quantized levels, just the way we've done it.
And what you immediately discover is, gee, it all makes sense.
You go to low temperature, everything's stunk down here.
And suddenly you've got zero heat capacity, because you changed the temperature a little bit, and everything is still stuck back here.
So that's the vibrational heat capacity of a solid.
That's what it looks like.
And at moderate temperature, this doesn't have to be very much bigger than epsilon zero.
You're already in, essentially, the limit.
The high temperature limit.
And now for molecules, that limit isn't usually reached at room temperature.
Here's a kind of calibration.
kT at room temperature is about equal to 200 wave numbers.
So molecular vibrations, you know, you've taken IR spectra they're typically on the order of 1,000 wave numbers or so.
kT isn't bigger than the vibrational energy.
But a crystal lattice, you know the vibrations are the acoustics vibrations.
And those are much lower in frequency.
If you have an atomic crystal, it just has the sound vibrations at all the different wavelengths that are available.
They're never very high.
So it's easy to get into the high temperature limit, in that case.
Where you basically see a temperature independent heat capacity.
OK. By the way, this was actually used commonly to determine the molecular weights of molecular crystals.
Because you've got a factor of the number of moles in there.
If you ask how big is the heat capacity?
Well it does depend on how many moles of material you have.
Because it depends on how many atoms.
That control how many modes there are in the crystal.
How many vibrational modes.
Because each atom has 3N degrees of freedom.
So if you just weigh the whole crystal, and then you measure the heat capacity, you know how many moles there are, and now you know the weight, you can figure out the molecular weight.
OK.
So that's the low temperature limit.
And now I want to talk a little bit further about the high temperature limit.
And in particular, I want to talk about both the heat capacity and the energy that we've seen.
Namely, this thing.
Of course it's really the same result for the energy and the heat capacity.
They're obviously intimately connected.
So these are the high temperature limits.
Now it turns out that that high temperature limit doesn't just obtain for vibrations.
But it's also the case for molecular rotations, translations.
All these low energy degrees of freedom.
So let's see why.
It has a name, that result.
It's called the equipartition of energy.
Sometimes it's called the classical equipartition of energy theorem.
And what it says is one half kT per degree of freedom equals energy in the high T limit.
And in particular, for translation, so E translational is 3/2.
Well, NkT for N atoms.
Or molecules.
E rotational is, now it depends how many rotational degrees of freedom there are.
If I've got a linear molecule, there are only two.
It can rotate this way, and it can rotate this way.
If I've got a non linear molecule, parts all over the place, it has three unique axes.
It can also spin this way.
And that's a rotational degree of freedom.
Of course the linear molecule wasn't like that.
Nothing is moving when you do that.
So it's either NkT, linear.
Or 3/2 NkT.
Non linear.
Great.
And then E vibrational is NkT per vibrational mode.
That's because vibrational energy is potential and kinetic energy, and it's 1/2 kT each.
Why does all that stuff happen?
Turns out, you can see why pretty easily.
All those degrees of freedom.
Those classical degrees of freedom.
If you say, let's write out an expression for the energy.
Classical.
You know it's 1/2 m v squared.
Or it's 1/2 k x squared.
Or for rotational energy it's 1/2 I omega squared.
You see a functional form emerging here?
That looks similar in all these cases?
You know, 1/2 times some constant, times some variable squared.
Well, now let's look at our expressions for the energy.
Average energy.
We're going to sum over all the energies of epsilon i, e to the minus Ei over kT.
Over sum over i, e to the minus Ei over kT.
That's just how we originally derived the average of energy.
In other words, it's the probability of each state times the energy of that state.
Summed up over all the states.
Well, now we have an expression for the energy.
It's one of these things.
It's something times a variable squared.
So, we'll use some general functional form, a y squared.
And now, let's assume we're in the high temperature limit.
And we've seen what that means.
It means kT is big, compared to the separation between the energies.
When that's the case, there are lots of these terms in the sum.
We can convert them to integrals.
We can forget about the fact that the energies are discrete.
We can say look, they're so close together, compared to kT.
That we can turn the sums into integrals.
So then we have integrals instead of sums.
And here's our energy.
It's a y squared.
Whatever that is.
It could be this, it could be this, it could be this.
And it's not going to matter.
e to the minus a y squared over kT over, integral over e to the minus a y squared over kT.
dy, dy.
OK. And here's what's going to happen.
If you do this integral by parts.
This one.
What ends up happening is it gives you this integral times a certain number.
This is going to come out of the integral.
And so it will turn out.
And it's straightforward to do it.
It's in the notes.
It goes through.
But I'll just write the result here.
The result is that you get, in this high temperature limit where you've gone to the integral form, you get 1/2 kT.
And it will always be the case.
All you need to know is the form of the energy.
As long as that's the case -- Remember y is just a variable of integration here.
That's not going to be preserved.
This comes out because of what happens here.
So what that's telling you is whenever you have an energy of this form, and you're in the high temperature limit, then you get this classical equipartition of energy result.
So, for translation, of course there are three separate degrees of freedom.
For velocity in the x, y, or z direction.
For rotation, if it's a linear molecule, let's say there are two separate degrees of freedom.
You have to keep track of how many degrees of freedom there are.
But that's all you have to do.
That's enormously powerful.
It means that without doing anything, I know the average translational energy of the molecules in this room.
Because of course I'm certainly in the high temperature limit.
With respect to translational energy levels, they're really closely spaced.
Same with rotations.
Now at room temperature vibrations, forget it.
I have essentially no vibrational energy.
I'm at the low temperature limit for the molecular vibrations of nitrogen or oxygen.
Those are high in frequency.
So much higher than 200 wavenumbers.
But for each molecule, I have 3/2 kT of translational energy.
Linear molecules, I have kT of rotational energy.
Without doing any work at all.
And since that applies to most molecules at room temperature, it's an incredibly useful, very, very general result.
OK. Next time we'll do a little bit of chemistry, and look at phase transformations.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, the first thing that we did is look at a simple polymer model not too different from polymer models that we've seen before.
That is, models for molecular configurations.
The difference in the case that we treated last time, though, was that instead of having just a handful of possible configurations with their just, designated energies, we had essentially an infinite sequence of levels.
All of evenly spaced energy.
And we went through that and saw what the thermodynamics worked out to be.
And also with the high and low temperature limits of the thermodynamics turned out to be.
And the interesting consequence of that is that that particular choice of configurations and energies associated with them maps exactly onto the vibrational levels of an ordinary molecule.
So, of course, when we do the statistical mechanics in the end, everything depends on the partition function.
Right?
So we're always writing some molecular partition function.
And we've got a sum over our energies for all the states.
And so the only input information we need to know is, what are the levels and what are their energies.
So, although we started yesterday with a model of polymer confirmations, once we decided what the energies were, which was this sequence of energies starting at zero and going with E, and 2E, and 3E, I think we labeled them E0, and so on, that's all that mattered as far as the statistical mechanics was concerned.
And so, since we started, We formulated this by imagining different confirmations for a polymer, but in fact these are the vibrational energies of any molecule too.
Of any vibrational mode.
So the results are very generally applicable.
And so we saw what the thermodynamics worked out to be.
And what the limiting cases were.
And a couple of them, that I'll just review, are, one that's very important in lots of settings, is that the low temperature limit when you're down it basically zero degrees Kelvin, that's where you've got kT way down here.
So there's not nearly enough thermal energy for anything to get out of even the lowest level.
And so in that case, everything is in the ground state.
And then the heat capacity, which is this often a very simple thing to measure, you change the temperature.
And you see that there's no change in the total vibrational energy.
In other words, the limiting low temperature heat capacity is zero.
So we saw that for Cv for vibrations.
And the limit that T goes to zero was zero.
And that's because we were in this limit, where everything is in the ground state.
Which is to say that u vibrational energy in that limit is zero.
Unlike the case we treated earlier, where we had a limited number of levels.
Because we treated a simple four-unit polymer with only four possible confirmations, in this case we have essentially an infinite number.
So even in the high temperature limit, we don't have them, there's not a maximum total energy.
If the temperature keeps going up, you'll get more and more energy.
Because you can keep populating higher and higher levels.
But the way that happens stops changing.
Because the levels are all evenly spaced.
So if the temperature, instead of being down here, is somewhere up here, now for sure if we change the temperature, the energy will continue to change.
But what we've found is that in the high temperature limit, we had the equipartition of energy result.
And that is, u vibrational in the high temperature limit, was just, for N molecules, it was NkT.
1/2 kT for each kinetic energy degree of freedom.
1/2 kT for each potential energy degree of freedom.
And since vibrations have each of those, potential and kinetic energy, it's kT for each molecule, for each degree of freedom.
That's an incredibly simple, useful thing.
That means, if I've got a molecule in the gas phase and it's in the high temperature limit for translation, I know each translational degree of freedom will contribute for each molecule 1/2 kT.
Or 3/2 kT, in all three dimensions.
So I know the translational energy of a mole of molecules in the gas phase at room temperature without doing anything all.
It's 3/2 NkT.
Rotations, if it's a diatomic molecule, will be two different degrees of freedom.
For rotation.
In two orthogonal planes.
And I'll have 1/2 kT for each.
1/2 NkT for N molecules.
For vibrations, if I'm in a high temperature limit, then it'll be kT for each vibrational mode.
For molecules, the vibrational frequencies are usually high, so that you're not in the high temperature limit.
But for materials where you have vibrations, acoustic vibrations, those are low frequency.
You can be in the high temperature limit.
And easily are.
So the result for energy for vibrational energy in the high temperature limit, was NkT.
And so that means that the heat capacity in that same limit at high temperature was just Nk, derivative with respect to temperature.
Very simple thing to measure, again.
So that's what we saw last time in the case of both the conformational model that we treated and the vibrational energies of molecules onto which that same model maps.
Now what I want to do, is look a little further.
And let me actually write the partition functions too.
Also an important result was that the partition function itself, q vibrational, in the high temperature limit, was just kT over E0.
Also very simple result.
E0 is h nu 0 for vibrational mode with frequency nu 0.
And of course, given the partition function you can calculate everything.
So in the high temperature limit we can easily calculate things.
In the low temperature limit, it's just one, right?
There's only one allowed possible state in the low temperature limit.
Everything is in the ground state when it's cold enough.
So the sum of our states just gives us exactly one term that's of any reasonable value.
That is, the term with this being the lowest energy.
Everything else, the energy is much bigger than kT.
This is a vanishingly small number then.
So just write q, vibrational, in the low temperature limit, is just one.
OK, that was the case for vibration.
And remember, these low temperature limiting cases, that's a common case for nearly any degree of freedom.
As long as it's quantized.
At some point, you'll get this situation.
Where the temperature is low compared to even the lowest excited level.
Of course, what'll happen in the high temperature limit might vary, depending on the structure of the energy levels.
In the case of vibrations, it's like this, it'll turn out rotations do the same thing.
But it's not always as simple as this.
OK, now what I want to do is just go through the next step of what we can treat, given the statistical mechanics that we know.
Which is chemical equilibrium.
Why can't we just calculate equilibrium constants based on what we've seen so far.
If we can calculate the partition functions for molecules, and they undergo reactions and they're in equilibrium, we should be able to calculate the equilibrium constants.
From first principles, just based on the statistical math we've seen so far.
So let's try to do that.
So it just means that, again, just as always, if we know all the energy levels and what all the possible states are, there's no reason we shouldn't be able to set our sights on a calculation of that sort.
So let's just sketch out what the levels are going to look like for simple chemical events.
So I just want to draw an energy diagram, here's energy.
And, like we often do with chemical equilibria, I'm going to set the zero of energy at the separated atoms, and I'm imagining I've got reactants and products.
So let's make this our products.
Then over here we'll have our reactants and make the energies different.
Alright.
So there's some amount of binding energy, right?
There's a bond dissociation energy going from the lowest available level in each molecule to the dissociation limit where we pulled the atoms apart.
So now I'm going to draw vibrational energy levels inside the molecule.
Let's imagine, it wouldn't need to be this, but let's imagine it's just diatomic molecules.
So there's one vibrational mode in each.
Just the stretching mode.
And we've already seen the levels are evenly spaced.
So there's going to be a bunch of evenly spaced levels.
Actually, it stopped being quite evenly spaced once this stops being a simple harmonic oscillator, but for all the low lying levels it's pretty close to that.
So those are the available levels.
And there's a dissociation energy.
So minus E, and I'll call this for the products, Ep.
And it's just minus D0 for the products.
Right, in language, terminology that we've seen before.
And it's the same thing for the reactants.
And really, if this were more than a diatomic molecule, maybe there would be a bunch of vibrational modes.
But it wouldn't matter.
This would just represent all the vibrational energies.
There's some lowest state available.
So now we've got the dissociation energy for the reactants.
So those are the energies it takes to separate the molecule.
But those aren't the only states available to molecules.
Of course, they could have extra vibrational energy.
They're not always in their ground states.
And we're going to calculate partition functions.
In general we would sum over all the available states.
And then just calculate the probability of being in the state.
One way to look at this, since if there's chemical equilibrium between the species, it means the molecules can interconvert.
What that really means is that any of the molecule has access to any of these states.
Of course, we like to group all these states over here.
Because they correspond to a particular chemical structure.
And we like to group these states over here, because they correspond to this different chemical structure.
From a statistical mechanics point of view, it's just states and levels.
And we could just calculate the probability of any molecule being in any one of the states.
But it's useful to keep them separated like this.
So let's do that.
And let's look at the difference here, the difference in dissociation energy.
Delta D0.
Let's not put a double arrow.
Let's define the sign of it.
This way, going from products to reactants.
OK, now let's just look at how equilibrium should work.
So now, let's just take a generic reaction.
Little a, our number, our stoichiometric number, A plus b, and B goes to c C, d D. And we know that delta G0 is minus RT log of Kp.
And that delta G0 is just the free energy of the products minus the free energy of the reactants.
So it's c times G C0 plus d G D0 minus a G A0 minus b G equals B0.
So we need to know the free energy of each one of the species.
And now we know how to calculate that from first principles, through statistical mechanics.
So we know that G is A plus pV.
A is minus kT log of capital Q.
And I'm going to assume that we're in the gas phase.
And it's an ideal gas, so I'm going to replace pV by NkT.
And what's Q?
Well, we know what Q is.
It's q, little q, to the N power translational over N factorial.
For atoms, this is all it would be.
But we have molecules.
So we have individual other degrees of freedom besides translation.
So let me just label those internal, q internal.
And that's also to the N power.
And q internal, if you say, what are those degrees of freedom, well it's the electronic energy.
it's the vibrational energy.
It's the rotational energy.
And we'll deal with those shortly, but let's just separate it this way for now.
Just we can keep track of where the N factorial belongs.
And now we're going to use Stirling's approximation for the N factorial.
So our log of Q, which we need up here, is just N log of q trans q internal minus log of N factorial.
And that's just equal to N log q trans q internal, minus N log N. Plus N. And of course, this I'm going to put down here momentarily.
And now our expression for G is up there.
So it's minus kT log of Q plus NkT.
So these factors of N are going to cancel.
So when I take minus kT log of Q, that's going to be minus NkT over here.
That's going to cancel the plus NkT here.
So all I'm going to have left is this term and this term.
Which I can combine.
So it's just minus NkT log of q translational q internal over N. Now these, really, are just the same as q, it's just minus NkT log of little q over N. I didn't need to separate that into this product.
I only wanted to do it to be clear where we were getting this factor of N factorial from.
So now we have an expression for G. And if we know G for all the species involved in a chemical reaction, we should be able to calculate the chemical equilibrium.
So let's do it.
So now to do it, let's look a little more closely at what these internal degrees of freedom are.
Because that's where these important details are going to come in.
Obviously the equilibrium is going to depend on the energetics.
How much different are the bonding energies or the dissociation energies and the molecules involved.
And also, how much different or the other molecular energy levels.
The vibrations, rotations, and so forth.
So, q internal is the product of rotational, vibrational, and electronic partition functions.
Remember how we saw that if you can write the energy as a sum of energies, then the partition functions are multiplied.
Because, of course, if this is the sum of a rotational plus vibrational plus electronic energy, then of course I can just separate out these things.
These are in the exponent.
I can write it as a product.
Same with translations.
I already have separated that.
So now let's look at how these things behave.
Well, for the electronic case, there's really only one electronic state of interest in general.
And that's the lowest state.
In cases you're extremely familiar with, if it were the hydrogen atom, it would be down in the 1s orbital.
And it would take a huge amount of energy to get up into the 2s or 2p orbital.
At ordinary temperatures you never have that.
All the atoms would be in the ground state.
Now, for most molecules, it don't take as much energy as for the hydrogen atom.
If you have benzene, for example, the ground electronic state, the lowest electronic state, is quite far below the first excited state.
Not as much as the hydrogen atom going from 1s to 2s to 2p, but still by much more than ordinary thermal energies at room temperature.
And what that means is, again, only the ground state term is going to matter.
Now, sometimes we would define that is this as the zero of energy.
But since we're doing chemical equilibria, the zero of energy is up here.
So that epsilon, that energy term, is this amount.
The dissociation energy.
This is minus dissociation energy.
So we're going to have a positive number there.
So q electronic is just e to the D0 over kT.
So that's easy enough.
We also know what the vibrational partition function is.
We saw it last time.
It's this thing we could write as a simple form.
One over one minus e to the minus E, I'll call it E vibe, over kT.
It's h nu 0, where nu is the vibrational frequency.
Again, in the case of many molecules, the vibrational energy is pretty high compared to kT.
So even this often simplifies to be just one.
In other words, all the molecules, in many cases, are basically all of them are in the ground vibrational state at room temperature.
Certainly if you look at molecules of high frequencies, if you look at the hydrogen molecule, H2, the vibrational frequency is about 4,000 wave numbers.
Remember I mentioned last time, kT at room temperature, T is 300 Kelvin.
Then kT corresponds to 200 wave numbers.
It's a factor of 2/3.
Much less than 4 wave numbers.
In other words, the vibrational energy is much lower than kT.
So, again, everything would be in the lowest level.
And this just simplifies to one.
In many cases, it's reasonably close to one.
For high vibrational frequencies, nu 0.
Now, we haven't talked about rotation.
But you probably have just an intuitive feeling that at ordinary temperatures, if I do this, if I wave my hand in the air, molecules that I happen to intersect are going to start spinning faster.
In other words, ordinary thermal energies do populate some number of rotational levels of molecules.
Molecules probably aren't going to start getting squished together and vibrate harder when I do something like this.
So molecules might generally still be in the ground vibrational levels, thermal energy isn't enough to raise them in many cases.
But rotation, for sure.
They're not all in the lowest level.
Turns out that, in fact, the energy separation between rotational levels is very small compared to kT at room temperature.
So just like we saw for the high temperature limit for vibrations, it turns out that for q rotation in the high temperature limit, we have the same situation where we have kT over, now it's a rotational energy.
And it's typically on the order of about one to ten wave numbers for small to medium sized molecules.
Remember, too, last time when we talked about in the vibrational levels and said, well, if you're at kT and it's this big, it's basically telling you roughly how many levels do you have thermal access to.
Because we saw the same results for vibrations.
It's not so different for rotation.
And it turns out that at ordinary temperatures, you might have access to a few tens to a few hundreds of rotational levels, depending on whether they're closer to one or ten wave numbers or a little higher.
Again, room temperature is 200 wave numbers.
So, that means that, remember q is a unitless number.
You're counting the states, weighted by the energy.
Weighted by the Boltzmann factor.
And for rotations, it's a number that might be on the order of 100 or so.
Order of magnitude estimate, that's all.
And of course we've seen the translational partition function is on the order of 10 to the 30th, right?
Enormous number.
In other words, in the simple lattice model that we've use to describe translation, just breaking up the available volume into little pieces.
Counting.
Well, OK, you have something on the order of 10 to the 30th possible locations.
And if you treat this properly, quantum mechanically, for the translations, there's actually a magnitude of the number is similar.
Now I just want to go through a very simple example.
Just treating a particular generic reaction.
And look at what the equilibrium constant is.
Working it through, given what we've seen so far.
So I just want to simplify it by having all the stoichiometric coefficients equal to one.
It's not a great complication, if we don't do that.
But it's a little bit simpler.
So let's take a molecule that's A-B plus C-D. And now let's break bonds and reform them.
So we get A-C plus B-D, right?
So we're going to do something simple.
And since I've got all the stoichiometric coefficients equal to one, we've got the same number of molecules as reactants and products, that'll make things a little bit easier.
Just in the sense that if we look at the contribution of the translational energies at room temperature, they're going to be the same.
For the reactants and the products.
Any of the molecules is going to have a translational partition function on the order of 10 to the 30th, at room temperature and an ordinary volume.
And nothing's going to change from reactants to products.
And for vibrations, let's assume, as is often the case, that the vibrational frequencies are fairly high.
So all the partition functions for the vibrations are equal to one.
Like we've got written up there.
For all four of the molecules.
So then nothing is going to change in the vibrations.
Going from reactants to products.
Finally, the rotations.
So, for the rotations our partition function is something on the order of, I think I meant to write 100, not ten here.
As our order of magnitude.
That's what it's going to be.
It's a number on that order.
And if we assume that the masses of the atoms involved are comparable, then we can cheat a little bit and say that's also, those numbers are also, going to be comparable for the reactants and the products.
We don't have to do that.
We could put it in, and rather easily calculate it.
But I'm going to make life simple in this way and just work through how to carry through the calculation.
And I think it'll be clear how to put in different partition functions for those quantities if they are different from each other.
OK, so let's try it.
Then, our delta G0, let's go back over here.
Here's delta G, let me just put the relationship up.
So we've got an expression for G. Which is, let's start from back there.
It's minus.
And now for each one of the substances, it's Ni kT log qi over Ni.
I'm just going to put this in molar terms.
So it's minus little ni RT.
Log of qi over capital Ni.
And I only want to do that because I want to express this in free energy.
This is for substance i.
Free energy per mole.
So it's just minus RT log of qi over Ni.
That's G per mole.
That's Gi naught per mole.
Plus RT log pi over p0.
Where p0 is an atmosphere, basically.
One bar.
So that means Gi0 bar is minus RT.
And I'm going to take this minus this and combine them.
So it's log of qi over Ni, that's this part.
And now I've got log of pi over p, and I'm just going to use the ideal gas law.
So I'm going to use pV is nRT.
So then I've got Ni kT over my factor of p0, times the volume.
So that's Gi0 for each one of the substances.
The Ni's cancel.
And I've got minus RT log of qi kT over p0 times V. Now, qi is just the total molecular partition function.
It's this product of q trans times q rotational times q vibrational times q electronic.
And now I need delta G. So delta G0, is just minus RT.
And now it's just, I'm going to take this for each one of the substances, for the products, minus the reactants.
And I'm going to combine the log terms.
This is the same, and I'm going to have this in every term.
So what's going to happen?
Well, all this stuff is going to cancel.
Now, that doesn't necessarily happen.
That happens because of the stoichiometric numbers being the same.
Otherwise I'd have to take them to that the power of the stoichiometric number.
But in this case they're just going to simply cancel.
So I'm going to have left is the ratio of my partition function for molecule AC.
Partition function for molecule BD.
Partition function for molecule AB, and for molecule CD.
Products, reactants.
Pretty simple, because I know how to calculate all this stuff.
And again I'd suggested some simplifying assumptions for what those partition functions are.
But it wouldn't be hard to calculate each one of them if I just had all the relevant energy levels.
To make this simple, we're going to assume that the rotational energies are the same for all the molecules.
It wouldn't need to be, it wouldn't be hard, to plug in different values.
We're going to assume we're in the low temperature limit for vibrations.
So q vibrational for each one of these things is equal to one.
The translational contributions, are equal for all of them.
So the only thing that's different is the electronic contribution.
And that's different.
Because, of course, you have different electronic energies.
The binding energies for the products and the reactants aren't in general going to be equal.
And in most cases, it is the energetics that dominate the equilibrium constants.
Of course, that is, it's the electronic energies that usually dominate.
Of course, the other things do matter.
But it's not unusual for the electronic energies to dominate.
So here's delta G0.
Of course, this is minus RT log of Kp.
So there's our expression for our equilibrium constants.
So it's just this product.
So let's just put in the partition functions for the electronic part.
It's e to the D0 over kT for molecule AC.
Times e to the D0 for molecule BD over kT.
Divided by e to the D0 for AB over kT, times e to the D0 for molecule CD over kT.
So the whole thing is just e to the D0 AC, plus D0 BD minus D0 AB minus D0 CD over kT.
Or in other words, it's e to the delta D0 over kT.
That's the whole story.
So it's a super-simple calculation to execute.
And for lots of molecules, we know the dissociation energies.
These are measured, determined, either thermochemically or spectroscopically for a great many molecules.
So in fact, we have the information we need to do that calculation.
And if we want to worry about the vibrational and rotational levels, typically we have that information, too, spectroscopically.
So we know what the relevant energy levels are to do the whole calculation.
So what it means is, we can calculate equilibrium constants for ordinary chemical reactions just from first principles.
Knowing the energy levels of the available states of the molecules.
Any questions?
In your notes I've included another example.
It's a little bit of an extra example.
You can go through it if you'd like.
It turns out to have a kind of attractive closed form solution.
Which is the case of a chemical reaction like an isomerization inside a crystalline solid.
So you can imagine you've got a crystal of some molecule, many crystals, thermally over time, they might decompose.
They might have reaction processes that can take place.
So I've imagined you've got uni-molecular reactions that can occur.
And in that case you start with a pure crystal of one species.
And you end up with a mixture of some number of the original species, and some number of new species.
And there's an equilibrium constant that'll say where that should be.
So again, it'll depend on the energies.
On the dissociation energies of the different species.
And there's also a kind of mixing term, because effectively now you've got the different species located at different places in the crystal.
Those are distinguishable states, in the solid.
And you need to count those.
But the one other example I want to work through is a little bit different from chemical equilibria.
It's phase equilibria.
Why shouldn't we be able to calculate phase diagrams?
So in the earlier part of the course, you went through and saw the macroscopic thermodynamic treatment.
The equilibrium constants and chemical equilibria.
So of course you saw the whole delta G0 as minus RT log Kp and so forth.
And you've seen now we can actually calculate all that from first principles.
What about phase equilibria?
What you saw before, we just drew phase diagrams.
And they had boiling points or lines of boiling.
Or in other words, liquid-solid and liquid-gas and solid-gas equilibria.
And what we did is, given where those lines fell, you can calculate things.
But where do the lines fall?
What's the boiling point or the sublimation temperature of some material at a particular pressure?
Well, we didn't offer any prescription for calculating that.
You had to take that from measurement and then given that, you could use the Clausius-Clapeyron equation and so forth and look at the way things behaved if you moved along a line in a phase diagram.
But there was no prescription offered to calculate where exactly would that line be on the phase diagram.
But again, if you know all the energies of the possible states, in the solid, in the liquid and the gas, statistical mechanics shows us that we can calculate the equilibrium between those.
Which is to say we know we can calculate where those lines belong.
So I want to just go through maybe one of a couple of relatively simple cases.
I want to go through the case of a solid-solid equilibrium.
Let's imagine we have two solid phases and an equilibrium between them.
And this problem was on the problem set.
How many of you did it?
Everybody did it.
Did you all get to the end of it?
Who got to the end of it?
Some of you got to the end of it.
If you did, congratulations, it's not so trivial to work on for the first time.
So I'll just briefly go through that problem.
And show how it works.
So, it's similar, of course, to the case that we're doing now.
To the case we just did, of chemical equilibria.
The difference, though, is that it's cooperative.
The solid, at least in the way I formulated that problem, you either have the crystal in phase one or phase alpha, or in phase beta.
And there's nothing in between.
There are lots of systems like that.
Not just crystals, you could go from different forms of DNA.
Act like that.
Where you have super coiled DNA and regular DNA.
And you actually can have some of the DNA in each.
But it's actually very cooperative.
So there's a huge tendency to either be all in one, or be all in the other.
Or at least very very, large pieces of it like that.
Lots of other systems act like that.
In other words, unlike the case where we're thinking about chemical equilibrium among molecules in the gas phase, these two molecules over here crash into each other and they react.
Nothing else cares.
The other whole mole of molecules does whatever they were doing.
And a little later some other pair of molecules happen to crash into each other and maybe react or maybe don't.
In other words, there's no cooperativity at all.
What happens to some pair of reactants and products, everything else is completely independent of it.
That's not the case when you have, in many cases, a phase transition.
You can see it by eye, often.
If you do something like super cool water.
This is a kind of fun experiment.
If you take dry ice, and you've got a little bit of water.
Right or a little pot of water.
And you put some dry ice into it.
So you can actually get the water to be below the freezing temperature and it won't freeze yet.
Then you drop a crystal in there and boom, it all freezes.
Very cooperatively.
Lots of phase transitions behave like that.
So let's just see how it works.
In this case, it's really similar to what we just did.
We can treat it, here's phase alpha.
Here's phase beta.
And the crystal, the interactions between molecules or the atoms in the crystal are different in the two phases.
So effectively, the binding energy is different.
In other words, if you say, now let's think of the energy it would take to evaporate all the atoms or molecules and let them loose in the gas phase.
That's the analog of dissociation for the molecule.
You're pulling everything apart from the crystal and separating them all.
So that's what we'll call these energies.
This'll be minus E alpha.
And this will be minus E beta.
And then there are vibrational energies.
So the lattice has a bunch of vibrational energies.
We can assume they're evenly spaced.
And they're going to be different.
They're not the same in the two different crystalline forms.
That's typically the case.
You have a phase transition from one crystal to another.
The lattice vibrational frequencies aren't the same any more.
Speed of sound is different.
Things are different.
And that's all.
That's the only thing that's different.
So these are binding energies per atom.
So now, let's just try to calculate Q.
And what we are supposed to get out of this is, what's the phase transition temperature?
That's the thing that we didn't have any prescription for calculating before.
We just said, here it is.
Here's the boiling point.
Here's the melting point, or whatever the phase transition was.
Or if we drew a phase diagram, here's the line.
But now we're going to try to calculate where that is.
Or what those temperatures are.
So, Q for either phase it's just e to the E over kT.
Just like we saw before for the dissociation energies of molecules.
To the Nth power.
No N factorial any more.
There's no translation.
Times the vibrational part.
This is the electronic partition function.
And then there's the vibrational part.
And that's one over one minus e to the minus h nu, what's called E over kT, for the lattice I'm assuming it's just one frequency.
So this energy is h nu E. And of course, it's different for the alpha and beta phases.
And I'm going to assume we're in the high temperature limit.
Which is often the case for lattice vibrations.
So it's e to the, little E over kT.
For our electronic part, to the Nth power.
And we've seen what the high temperature limit is.
It's kT over h nu E. And that's also, that to the 3N power.
I should have written this here.
In other words if you say, how many vibrations are there in the lattice, well if there are N atoms, each atom in the gas phase would have three degrees of freedom.
Translational degrees of freedom.
In the crystal, it can't freely translate.
But those degrees of freedom are still there.
The atoms can all move.
So now, those are the lattice vibrations.
When the atoms try to move, they vibrate against each other.
So how many different modes are there?
They may be degenerate.
They may all have the same energy.
But those modes are all still there.
Actually, they're the acoustic modes of the crystal.
And there are 3N of them.
One for each translational degree or freedom that each atom had.
OK, so that's it.
A, minus kT log of Q, so it's just minus NE, and the kT cancels.
Minus 3NkT log of kT over h nu E. Second part.
And then we can calculate the chemical potential.
It's just the dA/dN, d constant T and V. So this goes away.
It's just minus little E. This goes away, it's minus 3kT log of kT over h nu E. Well, basically we just finished.
At the phase transition temperature for the two phases, those things have to be equal.
And that's all there is to it.
So, at, we'll call it Tc, the phase transition temperature, I guess I called it T1 here, mu alpha is equal to mu beta.
So this stuff for alpha is equal to this for beta.
That's it.
So what does it say?
It says E beta minus E alpha, these terms, equals 3kT1 log of nu E beta over nu E alpha.
Solve for T1.
That's it.
It's E beta minus E alpha over 3k log nu E beta over nu E alpha.
There is our phase transition temperature.
That's the whole story.
So if we know the electronic energy that binds the crystal together, usually something that can be measured easily, so it's known for many materials.
And if we know the lattice vibrational frequency.
Also something that's pretty routinely measured, we're done.
So again, from a very simple first principles approach, we can calculate that phase transition temperature.
We could do it for a solid-gas equilibria too, if we said OK, let's think of sublimation of the solid.
And now we know how to calculate the chemical potential in the gas phase.
We can have those be equal.
That's actually worked through in your notes.
Again, it's an extra problem.
If you would like to take a look at it, it's a straightforward calculation.
The point is, we can calculate those phase equilibria.
Liquids, I will say, are harder.
Just because it's harder to define and know all the energies available.
Doable, approximately.
But the point is, we can actually now locate all those lines that we sort of arbitrarily drew on the phase diagrams.
And make sense of what the temperatures and what the pressures are, for that matter, where they occur.
Any questions?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, now we'll start on the last of the main topics in the course.
So we've finished most of our macroscopic thermodynamics, and our microscopic approach to it through statistical mechanics.
And now, our final topic is kinetics.
Kinetics is really a very different kind of topic.
Because unlike thermodynamics, thermodynamics tells you all about equilibrium properties.
A huge part of the work of this term has been to figure out what equilibrium is.
What is the equilibrium state, given some situation.
You've got some phases present.
Different then, you could go from solid to liquid to gas.
Or you've got different chemical constituents together that can react and go back and forth.
What are the equilibrium concentrations?
But we haven't worried at all about how long it might take to get there.
And that's what kinetics does.
Kinetics is concerned with rates of reactions, primarily.
How long it takes to reach equilibrium.
And of course it's super-important.
Because if you look at that window glass, it's not in equilibrium.
It's silicon dioxide, the equilibrium state would be a crystal.
It would be crystal and quartz.
Nevertheless, none of us is very worried that on the moment it's likely to suddenly, spontaneously turn into a crystal and be opaque and scatter light and do all that sort of stuff.
And in lots of other situations, certainly if you look at any living biological system, including yourselves or your friends or any other one, it's certainly far from equilibrium.
And you probably would hope for it to stay that way.
So there's an enormous amount of chemistry and processes we're concerned with.
Which depend intimately on kinetics in order to work the way they work.
They also do depend on thermodynamics and where equilibrium states are.
But that doesn't mean they necessarily reach equilibrium states.
So we're going to go through kinetics, starting with the simplest examples and working our way up to more complex cases.
And just see how we can describe elementary chemical reaction rates and processes.
So that's our concern now, is dynamics.
How long things take to get to equilibrium.
And actually just like macroscopic thermodynamics, kinetics does take an empirical approach to the topic.
In other words, it's based on experimental observation.
Of macroscopic rates.
How long it takes a collection of stuff, a mole of stuff to change chemically.
And undergo reaction.
And so forth.
We often infer molecular mechanisms based on kinetics.
And it's hugely important and valuable to do that.
But it's also important to recognize that what kinetics can do is show us how we can formulate microscopic mechanisms that might be consistent with our macroscopic kinetics models.
But the kinetics models by themselves don't prove the mechanism.
And there are all sorts of examples where mechanisms that were proposed and accepted because they were consistent with macroscopic kinetics results turned out to fail.
There are other ways to prove mechanisms.
You might be able to design direct spectroscopic observations of intermediates and so forth.
And in that case, it often becomes possible to distinguish between different mechanisms.
Which might all satisfy the macroscopic kinetics equation.
So we use kinetics to infer a mechanism but not necessarily to prove it.
Not generally to prove it.
We also use kinetics to describe an enormous range of time scales.
So, the fastest things that we're sometimes concerned with that might take place on femtosecond time scales, 10 to the minus 13 seconds or so, or 10 to the minus 15 seconds, even.
So, if we look at the range of time scales we might be concerned with, might go anywhere from about 10 to the minus 15 seconds at the fastest, and might go to enormous scales on the other end.
All the way out to maybe 10 to the 10 seconds, which is on the thousands of years.
Could be longer than that, sometimes even millions of years.
And the formalism that we'll set up applies equally to the full range of time scales.
So it can describe an enormous amount of chemical activity.
More commonly, for stuff that we're going to compare to, you might measure in the lab.
Common time scales range from roughly 10 to the minus 6 seconds out to about 10 to the 5th seconds.
That is, microseconds to about a day.
But there are plenty of examples of going either faster or slower, depending on the need and the experimental equipment available.
Let's define a few terms.
Let's talk about how we're going to just formulate chemical reaction rates.
So, let's just imagine any simple reaction of this sort.
So A plus B are going to go to C. There's some rate at which it happens.
Note, by the way, it's an arrow in one direction.
It's not an equals sign like we've seen before, or a double arrow.
So the point I'm emphasizing here is when we talk about reaction rates, unlike equilibria, we're talking about a particular direction.
Later on, we will talk about reversible reactions.
But there, too, the arrow going this way refers only to one of the chemical reactions that can take place.
Namely, in this case, the changing of what was the product now, would be the reactant, C, back into A plus B and it has nothing to do with the rate this way.
Measured independently and so on.
So the rate, we can look at the rate of disappearance of A.
So it's just negative d[A]/dt, where the brackets indicate concentration.
Usually in moles per liter.
If it's in a gas, then it would be pA, of pressure.
So negative d[A]/dt is our rate.
And note that that's generally a positive number.
We're looking at reactions going, if it's a reaction going in this direction.
A is gradually disappearing.
So we're going to define our rate this way.
To be a positive number.
The rate for C is going to be plus d[C]/dt.
It's also going to be defined in a way that makes it positive.
Because in this case, since the reaction's going this way, we're looking at the appearance of C. Now, by stoichiometry , in this case, of course whenever a molecule or a mole of A disappears, a molecule or a mole of C appears.
So because of that, in this case, the stoichiometry tells us that that d[C]/dt is equal to negative d[A]/dt.
And also equal to negative d[B]/dt.
And any of those can be used to define the rate of reaction.
Now, this is a particularly simple case because I've chosen the case where all of the stoichiometric coefficients are equal to one.
So now let's just look at any case that's different.
Let's look at 2A plus B goes to something else, 3C plus D. And just look at the reaction rates that we might see there.
So here now, the appearance of C is going to be three times as fast as the appearance of D, for example.
And also three times as fast as the disappearance of B.
So if we write negative d[B]/dt, we expect that's going to be negative 1/2 d[A]/dt.
In other words, A is going to disappear twice as fast as B.
Every time a molecule of B reacts, two molecules of A do.
And that's going to be plus 1/3 d[C]/dt, and every time that happens three molecules of C get formed.
And it's going to be plus d[D]/dt.
One molecule of D gets formed.
And so, the reaction rate could be defined in terms of any of these.
But the important thing is to keep track of stoichiometry so that the rate as it pertains to each constituent is accounted for correctly.
So to generalize, if I have little a of A, and little b of B, going to little c of C, and little d of D, then the rate of reaction can be written as minus one over a d[A]/dt, or minus one over b d[B]/dt, or one over c d[C]/dt, or one over d d[D]/dt.
So, experimentally, lots of measurements of reaction rates have been made.
And now to start on what's seen empirically, basically the following result is extremely common.
The rate is equal to some constant times the concentration of A to some power alpha, times the concentration of B to some power beta, and so on.
For all reactants.
Multiplied together, each concentration taken to some power.
Notice no products.
Again, we're only looking at a reaction going one way.
And if we look at the rate, this is typically what's found.
Alpha is called the order of reaction, with respect to A.
Beta order with respect to B.
And so on.
And little k is our rate constant which, just to make clear, since we've been using it extensively in the past few lectures, is completely not equal to the Boltzmann constant.
Absolutely no connection between them.
Now, alpha and beta, what they are, what they turn out to be, is typically what's determined as a result of kinetic measurement.
Typically they're small integers.
So, just sort of a typical example, if you look at the reaction of NO and O2, to make NO2, simple reaction, and you look at minus d[O2]/dt, use that as a measure of the reaction rate.
What's found is that it's a constant times NO concentration squared, times O2.
Simple, right?
One of the exponents is two, the other's one.
Doesn't seem so surprising mechanistically.
But it's not always the case.
Even when you have small integers, it's not always the case that the most obvious mechanism you would infer is the real mechanism.
And sometimes those exponents don't turn out even to be integers.
But here's another example.
If you just look it CH3CHO going to methane and carbon monoxide, seems like it would be a pretty straightforward thing, too.
So if you measure, for example, the rate of appearance of methane, what you discover is that it's equal to a constant times the concentration of the starting material to the 3/2 power.
Not obvious mechanistically why that should be the case.
Usually it's telling us something.
It's telling us that the reaction mechanism is complicated.
It's a multi-step process.
Sometimes there could be chain reactions and so forth.
So again, seeing things like this certainly helps us to infer molecular mechanisms.
Again, they don't prove molecular mechanisms.
But they certainly can be very helpful in suggesting them.
And then other means can be used to try to prove them.
Including, above all, direct observation of the intermediates that you would expect on the basis of one mechanism or another.
Now let's just go through some elementary examples of kinetics one at a time.
So let's start with the simplest possible case.
Actually, a very rare case, but one that'll help us just set up the formalism of, and the mechanism for us to proceed.
So let's talk about zero order reactions.
Actually very rare.
So, what this means is something like A goes over to products.
Make a measurement of d[A]/dt, and discover that it's k time A to the power of zero, that is, it's just k. There's no dependence on the concentration of A.
Or anything else.
And you have some rate of its disappearance.
So here's an example of it.
You could have oxalic acid, and it just turns into hydrogen carbon dioxide and carbon dioxide.
Things break apart, forms these products.
Make a measurement of the disappearance.
Seems to have nothing to do with the concentration of it.
At least under certain conditions.
Now, turns out that there's another element that helps understand this.
Turns out that light is needed, it's a photochemical reaction.
And then it's easy to see how this can happen.
Let's say we have an abundance of the starting material, and not very much light.
So every now and then photons bleed in, and every now and then when they're absorbed, it leads to dissociation.
In that situation, you'll be limited by the photons, but not by the concentration of the molecules.
And in fact, strictly speaking although this is zero order in terms of the chemical constituents, it's not zero order in the photons.
So in some sense, in this sort of situation, the photons should be considered one of the reactants.
You could write this as this plus h nu plus one photon, goes over to these products.
And then if you measure the rate and things are under circumstances like I described, you would discover that in fact yes you'd be photon limited.
The rate would depend on how many both photons are coming in.
Still, ordinarily, chemical rate equations aren't formulated in those terms.
So in the usual formulation, this would still have the appearance of a zero order reaction.
Now, how do we write and formulate a solution?
So it's simple.
We've we've written a differential equation here.
It's a pretty straightforward one.
Minus d[A]/dt is just a constant.
So we can solve it.
And typically we'll solve it by rewriting in integral form and then doing the integration as long as we can do the integration.
So from here we can write the integral from starting concentration [A]0 to some other concentration, [A], d[A] is equal to minus k integral from zero to t dt.
So all we've done is integrate on both sides.
And we've assumed a starting concentration.
We've assumed an initial condition.
And we've also assumed an initial time, which usually we can just call zero.
And this is, of course, something we can solve for straight away.
So we just have that [A] minus [A]0 is negative kt minus zero, which is minus kt.
So [A] is minus kt.
Plus [A]0.
In other words, the concentration of [A] at any time is given by the initial concentration, minus the rate constant times time.
It decays linearly in time.
So we can sketch that.
This is our initial concentration.
And then it's just going to decline with time after that.
And there's our solution.
In lots of cases, it's useful to define what's called a half-life.
It's just useful.
Because it provides some timeframe, a single number that's a timeframe on which the reaction occurs.
So, the half-life is just the time that it takes for half of the reactants to disappear.
t 1/2 time to react half the reactants.
OK, so in this case it's straightforward to see when that happens.
In other words, that's the time at which this concentration of A is just equal to [A]0 over two.
So we have [A]0 over two is minus kt 1/2 plus [A]0 or t 1/2 is equal to [A] over 2k.
[A]0 over 2k.
So there's our half-life.
So if we go over here, we can put that in.
[A]0 over two, and this time is our half-life.
So that's zero order reactions.
And, again, zero order reactions are rare.
But the procedure we're going to used to solve for kinetics is outlined in this way.
And we'll use that again and again.
Let's look at first-order kinetics.
Let's go over here.
Now, first order reactions are quite common.
Much, much more common than zero order.
So here, you have A goes to products.
That's the simplest case.
But this time if we measure d[A]/dt, we discover that it's equal to a constant times the concentration of A.
There's an important point to note here.
What are the units of k, in this case.
What do they have to be?
Yeah, reciprocal seconds right?
The equation has to work.
This is concentration per second.
This is concentration.
This better be per second.
Let's just, before we move on completely, look at this.
What are its units?
Here's the equation.
What are the units of k?
No, not unitless, because, look at this.
This is some concentration unit, typically moles per liter.
So this is moles per liter per second, right?
It's disappearance of some concentration for time.
That's got to be here.
All that is here.
Moles per liter per second.
So in every case, the units of k, the rate constant, have to be figured out on the basis of the specific rate equation.
Doesn't have the same units.
When the kinetics are of different order.
Now, let's solve this using the same approach as before.
Namely, this is still a pretty straightforward differential equation.
So let's just integrate both sides.
So that is going to tell us the integral from [A]0 to [A].
But now we have [A] on this side.
So here we just had a constant.
And effectively, I didn't write it out.
But we effectively wrote this, rewrote this, as d[A] equals minus k dt, and then went from here to the integral.
We're going to do the same thing here, except now there's this.
So really we're going to have d[A] over [A] is minus k dt.
That's what we're going to integrate on both side.
We need to have the variables distinct on each side.
So we have d[A] over [A].
Equal to minus k integral from zero to t dt.
And so, of course, you know how to do this integral.
It's going to look like log of [A].
And again, it's taken at [A] or, at [A]0.
So we have log of [A] over [A]0.
We're going to have log of [A] minus log of [A]0 coming out.
And that's equal to minus kt.
So now the kinetics are quite different.
We have [A] is equal to [A]0, e to the minus kt.
Very important, very common sort of result.
It's saying we start with a certain amount of material, and there's an exponential decay of it.
So very different from kinetics here, which are just linear.
And again the kinetics here, if you imagine that situation where you've got starting material and bleeding in gradually are photons, presumably at the same rate, then sure that material you're going to see the disappearance of it linearly with time.
Just depending on the rate at which the photons are coming in.
Here it's very different because, presumably, A is required in order to do this reaction.
It'll depend on how much.
Because of course if there's more of it, you'll just have more at any given time decaying over to products.
So you have an exponential decay.
So let's plot that.
Here's [A], let's make that [A]0.
There it is.
Now, let's look at what happens to the product.
So let's imagine that it's A going to B, so of course minus d[A]/dt is just equal to d[B]/dt, and the rate of appearance of B has to match the rate of disappearance of A.
Let's assume that we don't have any of B present at first.
So let's make [B]0 equal to zero.
Well then, [B] just has to equal [A]0 minus [A], right?
All the stuff that's left, all of the A that has disappeared, that's given by this difference.
Is just equal to B.
So that's just [A]0 minus concentration of A, but that's just given by that.
Which is [A]0 e to the minus kt.
Or in other words, [B] is just equal to [A]0 times one minus e to the minus kt.
So at t equals zero, this is zero and this is one.
In other words, this is going to be zero at first.
And then it's going to grow in with the same exponential form that this decayed.
So, [B] is going to do the exact opposite.
It's going to be like this.
So this is [B] of t and this is [A] of t equals [A]0 e to the minus kt.
Now, it's always useful to, whenever possible, to plot these things linearly.
Find a way to plot these as straight lines.
And of course in this case it's straightforward to do that as a log plot.
So if we take the log of both sides, we know of course that the log of [A] is just minus kt, plus the log of [A]0, so now let's look at that.
Make this the log of [A]0 and this is the log of [A] on the axis.
That'll be time.
So there's just some linear decay now.
And the slope is minus k. So experimentally, of course, this'll be done typically as a simple way of determining it.
Now, we also can usefully look at the half life, in this case, in the case of first order kinetics.
So we have an expression in general for [A].
So if we let [A] equal [A]0 over two, at t equals t 1/2, that tells us that log of [A]0 over two divided by [A]0 is minus kt to the 1/2.
But this is just the log of two, or log of two is over k is t to the 1/2.
And so t to the 1/2 is just 0.693 over k. So we can write that just generally, completely independent of whatever [A] is, also independent of what [A]0 is, and that makes sense.
So the point is, if you have something that just spontaneously decays into products, maybe it's just gradual chemical decomposition of something.
Spontaneously, without the participation of other constituents.
There's a general half-life that can be associated with that.
That'll just be related directly to the rate of the decay.
So it's knowable and measurable in a simple form.
And again, the half-life is always useful.
Because it just gives a simple, one-number measure of the rough timescale for things to change.
So if we go back to this plot, and then once again here's our half concentration.
And here's our t 1/2, just a useful way of summarizing in a simple way, what happens.
Yeah?
[INAUDIBLE]
Ooh.
Well, I didn't think about it.
But it isn't necessary that that'll happen.
Sure seems like it must be.
When one is half decayed, the other must be half formed as long as there wasn't any B present at first.
So yeah, thank you.
Well, given that, something needs to move.
But let's pretend like I got it right at this intersection.
Even though it doesn't look that good.
So really it should be there.
Thank you.
So the single most common example of first order kinetics of this form is radioactive decay.
You've got some radioactive isotopic that can spontaneously decay into some other nucleus.
And of course, this is measured and half-lives have been tabulated for lots of cases of this sort.
So a simple example.
Let's look at carbon-14.
It's got a nuclear charge of six.
It can decay into nitrogen-14 through the loss of an electron.
Happens.
First order kinetics.
Now, in the atmosphere, what happens is this will end up, the 14C ends up getting replenished.. Because from cosmic rays, what can happen is in the atmosphere, you can have your nitrogen plus a neutron will come and form 14C plus a hydrogen atom.
So in fact, the overall concentration in the atmosphere of 14C tends to be constant over time.
But stuff that's formed down here on Earth, with carbon, its content of 14C decays over time, and it doesn't get replenished.
So, let's think back some long time ago.
Here's a tree.
we can make it pretty.
So it's got some concentration of 14C that came from carbon dioxide in the atmosphere.
That's what it started with.
That's our starting point.
At some point later, through natural occurrence or human intervention, that tree became horizontal.
Then, and although we're looking back here, let's call this our t equals zero.
And our 14C concentration at the time is our concentration at zero.
Now let's say, shortly after that, either right after because of deliberate action, or shortly after because it was just discovered, somebody decides to build something using that tree.
So, early human craft.
And then let's say lots later, depending on your definition of lots, modern human, which can be denoted by this style of hat, makes exciting discovery.
Terrific.
And would like to know how old is it.
How long ago did all that stuff happen.
Let's assume this is also approximately t equals zero.
Well, so a log of [14C] over [14C]0.
So this carbon dating, all it's really doing is measuring that.
It gives a number for it.
And we know this because we're assuming it hasn't changed any in all those years.
In the atmosphere.
It's different down here, because the tree or the boat wasn't replenished.
Not nearly as many cosmic rays fell on it.
So that's minus log of t. And it turns out the t 1/2 is 5,760 years.
Amazing, that this can be known down to ten years.
But it is.
So that says, k his 0.693 divided by t to the 1/2, which is one over 8,312 years.
So there's our answer, then.
The time, how long ago that happened, is just minus 8 years times the log of [14C] in the artifact.
Divided by the log of, by [14C] in the air.
And the assumption again is that this is the same now as it was back then.
And there's a bigger number than this, so it's a positive number overall.
So in a fairly straightforward way, we'll use first order kinetics to determine the lifetime of something like this.
Because we know the rate constant.
And everything else follows.
Let's see.
Next there's going to be second order kinetics.
But let me just stop here and say just a word about the exam on Wednesday.
So I've handed out an information sheet about it.
But I don't have anything very different to say this time than I've had to say about previous exams.
Solve lots of problems.
Go over the homework.
Go over practice problems.
Try last year's exam as a sample exam when you're ready to do it.
Try it under test conditions.
There's nothing on the exam that you're going to look at and say oh my God, how was I supposed to know that we ought to study that.
There won't be any surprises.
Most of the exam questions so far, not all have been easy, but I think they've been more or less plain vanilla in the sense that they're right more or less down the middle of what we're trying to teach.
And not very many are taking off little peripheral elements of the class.
And it's not going to be any different here.
So if you can just do the problem solving and get familiar enough with it that you're just good at it, so that you can do it with a reasonable speed, you're going to be fine.
Also try to work on just understanding the underpinnings.
Especially of statistical mechanics.
That's where when you have these true-false or multiple choice questions, these sort of thought exercises.
Those are really hard.
Because they go a little bit beyond problem solving.
If you could do the problem solving, you're going to do fine.
But it's always useful if you can also formulate things.
And for that, it's never easy to just tell you here's what you have to do, to get good at this.
Because you have to think about it.
And it's always hard.
But to be sure, if you can just try when you review the statistical mechanics, especially, and you review the expressions that you use and how you solve problems with them, the next step in studying is to try to think, OK, do I really understand where that expression came from.
And why it makes sense physically.
And if you can do that, then so much the better.
Then you'll be in an even stronger position.
The info sheet gives a handful of equations that we do expect you to come in with those on your fingertips.
There are not very many.
Mostly we'll provide expressions that you'll need, and the important thing is that you know, understand, where they apply.
And how to use them.
So, good luck on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So last time you started kinetics which is a completely different topic from thermodynamics.
They're related and we'll see a relationship at some point.
And you did first order kinetics.
And today we're going to move on and go ahead and do more complicated kinetics and hopefully get to some interesting stuff in a couple lectures.
Right now we just have to do the review of stuff that you've probably seen before.
And so we're going to go reasonably fast.
So you saw first order reactions.
Today we're going to do second order reactions.
At least begin with second order reactions.
Second order kinetics.
Of the form, and then there are two kinds.
There's first order, rather, second order, in one reactant of the form A goes to products.
And then you have first order in two reactants first order in two reactants.
So I'll have the form A plus B goes of products.
Where that's first order in A and first order in B, and this is second order in A.
So you can think of this as A plus A goes to products if you want.
And there's some rate constant k, associated with this reaction.
And we do the rate analysis.
We write the rate of this process.
Minus dA/dt.
And for the purpose of writing on the board, and you might want to do this also in your homework, when you're tired of writing.
I'm going to skip the brackets.
The little brackets that we usually put for concentration.
I'm going to skip those, because it's just too much work to write them.
And you can understand that A is a concentration of A.
So this is equal to k A squared.
Second order on A.
And the units for k, it's important to keep track of your units, at the end of the calculations, often you want to make sure your units work out.
So the units for k are going to be of this, A is in moles per liter.
The units for k, you're going to have to be able to match the units on this side here.
So the units for k are going to be liters squared per mole squared per second.
To make the units match.
Then you integrate this, on both sides, from zero to t, or from A0, the initial rate, to A.
Or from zero to t and this doesn't go like this.
So integrate with A0 to A.
You've got dA over A squared, you put all the A's on one side, all the t's on the other side, from zero to t dt with a minus k on this side here.
And then you get your rate equation, integrated rate equation for this.
Which gives you one over A is equal to kt plus one over A0.
So this gives you A as a function of time.
And this is a convenient way to write it, because it's linear in time.
So you always try to get things to be linear in time.
Because then you can plot them as a straight line.
Plot on this axis here you plot one over A. on this axis here you plot t as a function of time.
And then you get a straight line where the slope is k. Gives you the rate, and the intercept is one over A0.
Yes
[INAUDIBLE]
k is moles per liter squared per second.
You're right, liters per mole per second, yes.
That is correct.
because it has to work.
Otherwise it doesn't work.
So I need to turn on my brain.
OK, think.
Liters per mole per second.
Thank you.
And the other thing that you want to know is the half-life.
What is the half-life.
So you set 2 A0.
So A0 over two, you look for the time where you get to A0 over two, so you put A0 over two in here.
And that's going to be equal to k times t 1/2 plus one over A0.
So you solve for the half-life and you get a half-life of one over k A0.
So the half-life is inversely proportional to the amount of stuff you started out with.
Unlike the first order reaction, where the half-life was independent of the amount that you started out with.
So this was the easy one.
The next one is a little bit more complicated.
Which is when your first order in each of two reactants.
Then your rate, your differential rate equation, looks like this.
Because the k times A times B, and now you don't know what to do with B, a priori.
So you want to rewrite this equation a little bit differently in terms of the amount of A that's used up.
So you define A, x is equal to A0 minus A, this is the amount of A that's used up.
This is what you started out with, this is what you're left with.
And so the difference is what's being used up.
And dx/dt is minus dA/dt.
And by stoichiometry, what you've used up, of A, is also what you've used up of B.
Because for every A that reacts, you have to use up one mole of B.
For every mole of A that reacts, you use up one mole of B.
And so you also have x to B0 minus B.
So, you can plug this in here.
And get a differential equation which is purely in terms of one variable, which is x.
Right here, it looks like it's in terms of two variables, which makes it complicated.
By doing this change of variables, you see that A and B actually related.
Because of the reaction stoichiometry.
So you can rewrite that as dx/dt is equal to k times A0 minus x times B0 minus x.
And now you have a differential equation in one variable, which with some tricks you can solve.
So we want to have the integrated equation.
So we take an integral of both sides.
We put in all the x's on one side.
All the times on the other side.
We'll go from x equals zero to x, dx, A0 minus x times B0 minus x is equal to k from zero to t dt.
And now you have to dig back into the last time you took integration calculus, which for me was about two centuries ago.
And figure out how to do this integral here.
And the trick for doing this integral is to use partial fractions.
So you use partial fractions to do this integral here.
Which means that if you take this ratio, one over A0 minus x times B0 minus x and rewrite it as some number, n1 divided by A0 minus x, plus some number n2 divided by B0 minus x, you solve for n1 and n2, and you find that n1 here is equal to one over B0 minus A0 and n2 is minus one over B0 minus A0.
And so you plug, now, this in here.
And instead of having this complicated denominator, you have a sum of two integrals that you know how to do.
Because they're basically of the form one over x.
And in doing this, you also realize that you have to be careful because when A0 is equal to B0, when you have the same amount of A and B, then things blow up and you're in trouble.
So that's going to be a special case.
So always look out for special cases for these things.
So you assume, then, that B0 is not the same as A0.
And then you can go forward with solving it.
So you integrate, and at the end of the process, I'm not going to go through it, it's really complicated, you get something that looks like this.
A0 minus B0 log of A B0 over A0 B.
And you have your equation here.
Now, we don't really have a good way to plot it against one variable.
And the usual thing is look at specific cases.
And limiting cases.
And there's one limiting case we already brought up.
Which is when you started with the same amount of material.
What does it look like if you have the same amount of material to begin with?
So if you have A0 equal to B0, you start out with the same amount of stuff, but if you start out with the same amount of stuff, then this doesn't look so different from, at least mathematically, from this one right here.
If you start out with A0 is equal to B0, and for every mole of A that you use up you use a mole of B, then throughout the whole reaction, the concentration of A and the concentration of B are going to be the same.
So for the whole reaction, if you started with this, A is equal to B for all times.
That makes it easy.
Because now you can go back and instead of writing A times B, you can write A times A, which is A squared, which is what we had here.
And then you have the whole thing solved.
So in that case here you just have minus dA/dt is equal to k A squared, and one over A is equal to kt plus one over A0.
You don't even have to do any math, you just look at it.
So that's one case.
Another case is if one of the reactants is in much higher concentration than the other reactant.
And that's called flooding.
You basically flood the system with one reactant.
And that's something that we'll use again, hopefully by the end of class today.
Let's say that we take, so this is another limiting case.
Let's say we take A0 to be much bigger than B0.
So we flood the system with A0.
As a result, the concentration of A doesn't change very much in my pot.
It's hugely concentrated in A, there's a little bit of B around.
Again, with the process.
If I use all of the B up, the difference in A is going to be very small.
So at the end of the process, I'm basically still going to have A0 left in the pot.
So during the whole process, during the whole time period, I might as well assume that A is equal to A0.
And that makes my life much easier.
Because now if I write my differential equation in terms of B instead of A, so the rate of destruction of B, k A times B, instead of writing A here, it's pretty much constant for the whole time.
I'm just going to write A0.
So now, if k times A0 is a constant, and this looks awfully like a first order reaction.
So I can solve for it.
And I get that, so I can just write the answer because I've done this already.
I don't have to do it again.
The concentration of B then goes like B0 e to the minus k prime t, that's the first order reaction.
Where k prime here is this new rate constant, this new number, which is k, times A0.
So that's easy to solve also.
So always go to the limiting cases, because they tend to easy.
And if you were to go to the full solution and put in this limiting case, then you'd find that you get the right answer this way as well.
You can directly go to the easy way of doing it, or you can go through the whole process of solving it and putting the approximation in there.
And do the cancellations and get this.
But this is much easier.
Just writing the answer down is always much easier.
So it's a pseudo first order reaction.
We call this a pseudo first order reaction.
So we're done with the simple stuff now.
Any questions on first order and second order reactions?
So the next step is, you've got a reaction.
It could be a gas phase reaction, it could be a solution phase reaction.
There's some quantity, some property, of the solution that's going to change.
That's going to allow you to follow it as a function of time.
And that property could be many things.
It could be spectroscopic.
It could be that there's an absorption in the visible that changes as the concentration of one of your reactants changes.
Or one of the products could have an absorption band that you could follow in time.
Or you could using infrared spectroscopy to follow it in time.
Or if you have a reaction in the gas phase, and you have more products or less products than the reactants, and the pressure is going to change in time if you have a finite volume.
So there's usually some quantity that you can use to follow the reaction in time to extract out data.
And then from that data, that you want to know what are the kinetics of this reaction.
Because eventually you're going to try to find a mechanism.
You're going to try to find a mechanism that's consistent with the data.
So you get data.
Then you want extract out of the rate constants and orders.
So let's assume that you've found a way to get data.
And now you've got to analyze your data.
And suppose that you've found a way to measure the reactant concentration as a function of time.
And let's say that in the first case, the simplest case is that you have one reactant.
So you have one reactant, A.
So A goes to products.
And you've managed to extract A as a function of time.
Well, the obvious thing to do is to plot A versus time and see what it fits like.
So you take A versus time, and you plot it.
And you know that if you plot log A versus time and it's a straight line, well, that's going to be a first order process.
Plot log A versus time in a straight line, you know that's going to be first order.
If it doesn't go to a straight line you know it's not first order.
So then you go ahead and plot one over A versus time.
And if it's a straight line, you know it's second order.
And if it's not second order, it's not a straight line.
It's not a straight line, you look for some other order.
So that's one way to do it.
And you've got to have enough points on your graph, because if I were to plot a, let's say this is my data point, if I have a second order process, at the beginning it's going to look an awful lot like a first order process.
It's not until after a while that it's going to start to deviate.
So you've got to have enough points down in time to make sure that you can differentiate between a straight line and a line that's not straight.
And usually, that's often a mistake that experimentalists make.
They look at the beginning.
They say, oh, it's a straight line, work is done.
Go home.
But usually you need to have a good amount of the reactant consumed before you can tell the difference between first and second order.
So you'll have an opportunity to do this on the homework.
This kind of exercise of extracting the order of a simple reaction.
Another way to do it if you have a simple reaction is to look at half-lives.
That would be the half-life method.
If you can measure A as a function of time, then you know when you've gotten A over two.
So, you know that if I look at the half-life versus the concentration, the initial concentration, of my reactant, that tells me something about the order.
Because we saw that for our first order, t 1/2 was independent of the initial concentration.
And that for a second order, t 1/2 was proportional to one over the inverse of the initial concentration.
So if you plot t 1/2 versus A0, or have a few A0 versus t 1/2s, then you can tell the difference between first order and a second order reaction, and see which one fits.
Sometimes to get even more solid numbers, because from here you can also extract k. If you have a bunch of points, of t 1/2 versus A0, you can extract k, the rate constant.
Not just the order, but also the rate constant out of this data.
You can use multiple lifetimes if you have enough data.
Multiple lifetimes.
So you can define, you can define a t 3/4.
Which is the amount of time it takes for the concentration of A to be 1/4 of what you started out with.
So 3/4 is gone.
And then you can put that into your first order rate law.
So when you have log of A over A0 is equal to minus kt, you get that t 3/4, we can solve for t 3/4, and you get that that's equal to two log two, over k. And then you can do the same thing for a second order process.
So this is first order.
You plug in t 3/4 and A is equal to 1/4 A0, and you solve for t 3/4 for a second order process.
And you get that this is equal to three over A0 times k. So it's the same functional form as these two.
But there's a pre-factor that's different here.
So here there's a two that comes in there.
And here there's there's a three that comes in there.
And so there's an obvious way to tell, then, if you have the t 1/2 and the t 3/4 signs.
Basically, you follow, instead of having many reactions, instead of having many reactions to do, with different A0's here you can do one reaction.
If you do one reaction and you watch the reactant go away, and you time it.
When 1/2 of it is gone, that's one time, then you keep going, like 3/4 is gone, that's another time.
Then you can take the ratio of those two times.
Of t 3/4 versus t 1/2.
t 3/4 versus t 1/2.
And if it's a first order process, the ratio here is just two.
And if you take t 3/4 over t 1/2 and it's a second order process, the ratio is three.
So with one experiment, then, you can extract out the order.
You can't extract out, well, you can extract out the rate constant.
If you know the order then you know which equation fits, and you can extract out the the rate constant, with a big error bar.
You're always better off doing many multiple lifetimes of different A0's or many of these just to get more statistics in the result.
So this is the simple process.
And you always try, if you have something complicated, you always try to bring it back to a one component process by doing something like flooding.
So if you have five different reactants, if you make four of them in very large quantities and keep one of them in very small quantities, then all the four that are in large quantities are basically constant over the process.
And you basically are looking at only one reactant going away.
And then you can use these methods to figure out what the order is for that one reactant.
So, questions about simple one reactant sort of processes.
It's pretty straightforward.
So now, let's say we have more complicated reactions.
There are two ways that we can deal with that.
The first, I already mentioned, which is the flooding which we'll get back to.
And another way is called, so we have some complicated reactions.
Complex reactions, with multiple reactants, that's A plus B plus C goes to products.
And there could be some stoichiometry in front of there.
So one of the ways to deal with that is called the initial rate method.
We want to find out orders and rate constants.
Initial rate method.
So if I look at minus dA/dt, one of the reactants or the rate of the reaction near time t equals zero.
That's the initial rate.
Right as the reaction starts.
I mix everything together and I, just as after I mix it together, I watch the process of A disappearing or B disappearing or C disappearing.
And so in reality what I'm doing is minus delta A / delta t near t equals zero, where delta t is a small interval.
So there's not much change in delta A. Experimentally that's what I do.
And that's pretty much, essentially getting this number out.
And we're going to call that the initial rate.
And the initial rate is k, and at the beginning I haven't used up anything.
So all the initial concentrations are there A0 to the alpha, B0 to the beta, C0 to the gamma.
Et cetera if you have more reactants.
So you measure this.
You measure this R0, and then you repeat the same process with a new concentration of one of those three reactants.
So with A0 prime, let's say.
And then you get a new R0, R0 prime.
Then you take the ratios of these R0 and R0 primes, R0 divided by R0 prime.
So R0 is k A0 to the alpha, B0 to the beta, C0 to the gamma, then you have k A0 prime to the alpha.
B0 to the beta, C0 to the gamma, and the k's you disappear.
The B0's disappear.
The C0's disappear.
The only thing that we've changed is the concentration of A0 to begin with.
So the ratios of these initial rates is the ratios of the A0's to the alpha power, to the order in terms of A0.
So if you're clever about your choice of ratios, then you can get alpha pretty easily.
So if you choose, so if you now choose A0 prime to be equal to 1/2 A0, then you measure R0 over R0 prime.
And if you get one, then you know that that alpha's zero.
Alpha has to be zero here.
Then you know alpha is zero, that gives you the order.
It's a zero order reaction.
If you get that R0 over R0 prime is square root of two, or rather 1/2, then a square root of two, square root of 2, then you know that alpha is 1/2.
And that's a half order.
We haven't seen any half order reactions yet, but we will.
Those are indicative of a complicated mechanisms.
But that's what you would get out of this experiment.
If you get that it's equal to two, if you get that this ratio is equal to two, then you know that alpha is equal to one, et cetera.
So it's a pretty easy way to get the order.
Then you repeat the experiment.
Now instead of changing A0, you keep A0 constant and you change B0.
Or, you can use flooding or isolation, which is the next process to get the other orders.
And eventually you get the rate constant.
Once you have all the orders, and you have R0, then you have the rate constant.
OK, so that's one way of doing it.
And a second way of doing it is the way we already mentioned.
To solve the second order reaction in two components.
Which is to flood the reaction with everything except for one.
So this is called flooding or isolation.
Basically, you isolate one reactant and watch it.
You're trying to get back to a system, which is a simple system.
Which is a system of one reaction.
So let's say you take A0 to be much smaller than all the other species.
You flood with B and C, and you isolate A0.
And then your rate minus dA/dt, it's going to be A to the alpha.
And instead of B, well, during that process B's going to say pretty much constant.
Because it's hugely concentrated in B.
You can replace B with B0.
You can replace C with C0.
So now you have an effective constant, an effective rate constant, and then you have a process which is effectively, or pseudo, one reactant.
Then you can use these methods here, you can plot A versus time, you can find path lines, et cetera, to gather the order for it.
Then you can get alpha and k prime.
And if you can change B0 and C0, then you get k out of this.
So this is basically all fairly straightforward, just tedious experimentation to get all these numbers out.
OK, any questions on this?
You'll get experience on the homework.
There's likely to be a question on the final where you're given data and asked to extract out orders and rate constants.
So let's move on now.
So, so far we've looked at first and second order elementary processes, and we've looked at taking data and extracting out rate and rate constant.
And the next step is to build mechanisms.
So a mechanism is when you take a complicated reaction, like A plus B plus C goes to D plus E. And you break it up into elementary steps.
What's an elementary step?
An elementary step is a step which happens in a single reaction.
So I could hypothesize that this complicated reaction happens in three steps, where I need to have a molecule of A and a molecule of B collide with each other to first form an intermediate F. Then I want a molecule of F plus a molecule of B to collide to form intermediate G plus a product D. Then have the intermediate G plus reactant C collide together to form the product E. So this set of elementary steps, where at each step you have a collision of two or three molecules together, three is not so common but two is very common, those elementary steps are called the steps of the mechanisms.
And these elementary steps you can define something called molecularity, which is the number of species that you need to collide with each other in one of these elementary steps.
So the molecularity here would be two, you need two molecules to react.
Here it's two, here it's two.
If I have an elementary step which is a zero order in one reactant, then the molecularity would be one.
Or I could have A plus A, the same molecules have to collide with each other.
Molecularity would be two.
And the molecularity and the order of the reaction are connected.
So if you have something which is a molecularity of one, then it's going to be a first order reaction.
One reactant is just sitting by itself, falls apart, like in radioactive decay.
Molecularity of one, that's a first order of process.
If I need to have two molecules come together, then it's a second order process.
If I have to have three molecules collide at the same time together, molecularity of three, then it's going to depend on the concentration of all three at the same time.
That's called a ternary reaction, and those are really quite rare.
Termolecular reactions, you need to have your concentrations very, very high to statistically get an event happening where all three molecules collide together.
So three body reaction is hard and anything higher than three body is essentially impossible.
So that limits your choices, which is nice.
So that's the mechanism.
And so what we're going to do next is go through some mechanisms.
Some simple mechanisms and build up the complexity.
Any questions about mechanisms here?
What we're doing here is, we're formulating a framework.
Where we can go back and look at things that are more complicated, like chain reactions or explosions or enzymatic reactions, and know when to apply approximations and et cetera.
So we basically, here, are just laying down the ground rules.
So let's go to our first example of a mechanism, a more complicated reaction.
And what we're going to do is we're going to extract out integrated rate laws out of all these mechanisms.
And see what it looks like, as a function of time.
So the first one we're going to do is called parallel reactions.
Simple mechanism.
In this case here I have one reactant, and that reactant has a choice.
You can think of it as a radioactive decay, an atom decaying in two different channels.
So it can decay into B, or it can decay into C. There are two rate constants, k1 and k2.
So you can write it like this, or you can write it as A goes to B plus C. This is how you would write a reaction.
And this is how you would write your mechanism.
A goes to B and A goes to C. Each elementary step, these are the elementary steps and this is the complex reaction, each elementary step is unimolecular.
It's a first order process.
So in all of these examples, the first thing you do is you write your rate law.
The rate at which A gets created or destroyed.
And there are two paths.
It gets destroyed and into B, with a rate which is proportional to the concentration of A, and it gets destroyed into C, proportional to the concentration of A.
So you write down all the ways that A can get destroyed.
There are two ways here.
Two channels.
This one happens to be fairly easy to solve.
k1 plus k2 times A, and you've seen this before.
It's minus dA/dt as a constant times A.
That's a first order process.
So you can just write down the answer.
You don't need to do any math here.
You recognize that we just call this one k prime.
And that the rate is A as a function of time.
This is A0 e to the minus k1 plus k2 times t. And everything you've learned about plotting first order processes et cetera, is applicable here, where the rate constant is the sum of these two.
So that's for the reactant.
The products are also interesting to plot as a function of time, to see how they are related to each other in terms of their concentrations.
So let me go through this also.
When things get more complicated we'll quickly go and make approximations.
But for now, we can still do everything exactly.
So you write down your rate law for the product.
dB/dt is equal to k1 A. dC/dt is equal to k2 times A.
The formation of B depends linearly on A.
The formation of C depends linearly on A, because they're both first order processes.
To make B and C. And you integrate.
You integrate here from zero to B, dB.
Is equal from zero to t, A dt, k1.
A is a function of time.
And we've already solved for that.
It's this exponential up there.
So you plug in here A of time.
And you turn the crank and you integrate, and it's an exponential.
So it's not so hard to integrate.
And you get that B is a function of time is k1 times A0 over k1 plus k2 times one minus e to the minus k1 plus k2 times the time.
Things are already starting to get a little bit more messy in the math.
And then to get C, you actually don't need to do anything.
Because you notice that the only difference between B and C here is replacing k2 with k1.
So don't worry about doing any math.
Just write down the answer.
k2 A0, you interchange k1 and k2 at every step.
One minus e to the minus k1 plus k2 times the time.
The only difference is up here in the k2 term.
And those are your equations for k1 and k2.
And what you find, is the ratio of B to C is a constant.
If I divide B by C, everything cancels out except for the k1 and the k2 here.
Is equal to k1 over k2.
And that is called the branching ratio.
The branching ratio, because there are two branches out of the reactions.
And this gives you the ratio of which one is more likely to happen than the other one.
And so if k1 is much bigger than k2, the rate is per unit time.
The units of k1 are per second or per minute or per hour.
So if this is big, if k1 is big, then mostly you're going from A to B, and only a little bit of C is formed.
And the ratio of B and C is always constant.
And so you can plot, then, you can plot the result.
You can sketch out the result.
So you know that A is going to come down exponentially.
Time on this axis here, concentrations on this axis here.
So this is A as a function of time.
That's that equation up here, in exponential decay.
And the quantity of A.
And B and C are going to come up in time, also, with this exponential format here.
B is going to saturate at this ratio right here, k1 A0 divided by k1 plus k2.
C is going to saturate at this quantity here.
So they're going to start both at zero.
And B is eventually going to go to k1 A0 over k1, plus k2.
And C, eventually, will go to k2 A0 over k1 plus k2.
And this, and the ratio of these two lines at every point is k1 over k2.
So, a simple question, for instance, that you might be of the type that you might be asked to look at is, suppose that k1 is 1/10 of k2.
Which would you expect?
Would you expect to have, let's see.
So C is in green here.
C, B.
Or do you expect, so A comes down.
B comes up.
C comes up like this, or do you expect the last choice, I'm going to put the last choice here, so this is, let's call this choice number one.
Choice number two.
Choice number three.
So k1, the rate k1 is 1/10 of the rate k2.
That tells you something about the branching ratio.
So do you expect this one here to be the right one?
How many people think this is the right one?
What about this one here?
How many people think this is the right one?
One person.
What about this one here?
So the branching ratio is the ratio of the two.
It's 1/10.
So this is approximately, in my sketch, poor sketch, granted, but this is approximately 10 times bigger than this.
So that's the ratio that you'd expect.
And it's the right, here k1 is the rate into B.
It's slower than the rate into C. So, you got it right.
So for more complicated, we have a more complicated process, we're going to ask you the same sort of stuff.
And it won't be as straightforward.
Any questions on this beginning here?
Next time we're going to finish with the parallel first and second order processes.
And hopefully we'll get done with the complex reactions and mechanisms and move on to, I forgot what's next on the list.
But some explosions or chain reactions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, any questions from last time?
We started doing reaction mechanisms.
We did first order parallel reactions.
So one of the things that we'll be stressing is that the mechanisms that we write down are ways to try to understand a complicated chemical process.
And you take data, you can find intermediates sometimes.
And you get rate laws from the data.
You infer rate laws.
And from these rate laws, you think up of a reaction mechanism and you make sure that what you measure is consistent with your mechanism that you make up to explain the complicated chemical process.
And one really important thing to remember is that when you come up with a mechanism and you've got a bunch of data that supports it, it's great.
But it doesn't mean that you've solve the problem.
It doesn't mean that you've proven that the way that the chemistry proceeds is really the way it proceeds.
Because there's always the possibility that there's an intermediate somewhere in there that you haven't been able to measure.
And a lot of people got tripped up over the course of the last century, of making up mechanisms, getting a lot of data that supports the mechanism and missing something really important.
So it's a common mistake, that people think they've proven a particular chemical mechanism just because they have data that supports it.
It's up to a certain point that you know what you have.
And we'll see examples of that today, probably.
And then certainly next time when we start making approximations in our kinetics.
And see where things can really go wrong if you don't have enough data.
If you don't try to fish out really, really, tiny intermediates somewhere along the way.
So last time we did then parallel first order reactions.
And today we're going to march through and do parallel first and second order.
So one is first order, the other one is second order.
Things are going to get more complicated.
And what you're going to get is a flavor of how to solve the problems.
Setting up the problems.
We're not going to do a lot of the algebra here on the board, because otherwise we would spend the next three weeks doing algebra.
And that would be no fun at all.
That doesn't mean that you're not going to be doing any algebra on the homework.
Because part of it is figuring how to do it.
So, we have a reaction where A goes to B plus C. And the first thing we did was, the first mechanism we wrote was a parallel mechanism where we had A goes B and A goes to C. That's the mechanism.
And one other way to write it is A goes to B or C, in a way that is more, sort of describes the branching process, that when we talk about a branching ratio, the ratio of amount of B to C, this looks like the branching out of two different paths.
And this time we're going to do where this is first order and this is second order.
With the rate k1 and a rate k2.
And the way that you do all these problems, you have to be very systematic about it.
The first thing you do is you have to write down all your rate laws.
And make sure that you include everything.
So you start by writing the rate law for the destruction of A.
And then the rate law for the appearance of B and for C. So for a, we have dA/dt.
And again, I'm dropping the brackets around the A, because I don't want to carry around all these extra symbols.
So we destroy A by making B.
So that's going to be first order.
And then we also destroy A by making C, and that's second order.
So we have two paths out of A.
To get rid of A.
Then we write down the appearance of B, dB/dt.
It's only through one path, first order in A, and then the appearance of C, only one path which is second order.
And these are all our differential equations.
And now the trick is, how do you solve these three differential equations in a way that you get meaningful information out.
Well, the first thing to do is to, so these two here depend on A.
So the appearance of B depends on A, the appearance of C depends on A.
But this one here only depends on A by itself.
So this is the first one you're going to start out with.
Because you can put all of the A's on one side and all the time on the other side and integrate.
So then you solve, and you can rewrite this as minus dA/dt is equal to k1 times A times one plus k2 over k1 times A.
And then you put all the A's on one side, all the t's on the other side.
And integrate from A0 to A minus dA over A times one plus k2 over k1 times A.
That is equal to k1 from zero to t dt.
And then you have to solve this integral here.
And there's a reason why I factored out the A here.
It's because this is of the form you can use to use the trick of partial fractions, which we mentioned this before.
So you use partial fractions to solve this.
And I'm not going to do it on the board.
You turn the crank, you basically write one over A times one plus k2 over k1 A is equal to n1 of A plus n2 over this part here, and you solve for n1 and n2 and you plug it back in, and you redo your integral, et cetera.
And you get your answer.
Which I'm going to write down because I'm going to use it later.
k1 A0 over e to the k1 times t, k1 plus k2 A0.
So one of the things we immediately see is even though this mechanism isn't very complicated, just two paths, one first order, one second order, the solutions start to get not so simple pretty quickly.
So it depends, there's an exponential on the bottom here.
Depends on the initial concentrations and both rate constants are in there.
The first thing you want to do is you want to look at limiting cases.
Just like before.
We did, and the first limiting case we're going to look at, so interesting to look at is this guy right here.
k1 and k2 A here.
If one is bigger than the other, then things will cancel out.
And we can get some intuition.
So the first limiting case we're going to look at is where k2 A0, this term right here, is much smaller than k1.
Now, we want to make sure that in these limiting cases that when I write something like this, that the units actually make sense.
That I'm not saying, three apples are less than four oranges.
So k1, it's first order, so the units are one over second.
k2, it's a second order rate constant, so the units are one over second molar, times moles per liter.
So the moles per liters cancel out and I have one over our second is less than one over second, so things are great.
It I'm making the right kind of approximation here.
What else is this approximation saying?
This is saying that the rate into B, so this one is the faster one.
k1, the rate into B, is the rate into, is k1, is faster than k2 times A, which is basically the rate into C. So this is saying that the rate into B, to form B, is faster than into C. So we can write down a sketch.
We can sketch what we expect this approximation to be, then.
And we know it's going to look something like this.
We know that A is going to come down, exponentially.
Without any structure to it.
It's going to go either into B or into C, in some branching fractions.
And we know that the rate into B is faster than into C, so B is going to come up like this.
And C is going to come up, but slower.
And there's going to be a ratio between these two guys.
So we know the slope here is going to be slower then the slope for B.
And we can check that out, within our approximation.
We can write, at t equals zero, find out what these initial slopes are.
For the creation of B and C. So dB/dt is the slope of the creation of B at the beginning.
At t equals zero.
And that's just the rate law.
dB/dt at t equals zero is k1 A0.
And dC/dt, the initial slope, t equals zero, is k1 A0 squared.
Or k2 A0 squared, rather.
So I'm going to write it as k2 A0 squared, like this.
We said k1 is much bigger than k2 times A0.
There's the same A0 here.
So we see, just by writing this down, that our intuition that because the rate into B is much faster than into A, that it should look like this.
Well it's borne out just like this very simple sort of looking at the initial rate where the slope here is much smaller than the slope here.
And then we can take the approximation in here.
And see what it implies.
So let me do that here, let me just use my green chalk here.
So k2 A0 is much less than k1.
So this is going to go away.
k2 A0 is much less than k1.
Well, this is building up from zero.
So this is always bigger than one here.
So I can get rid of this as well.
Because there's the k1 sitting here.
And k1 is much bigger than k2 A0.
Once I got rid of all these two guys, then the k1's cancel out.
And this is looking very nice.
Because now I can take that e to the k1 t and put it upstairs.
A is equal to approximately A0 e to the minus k1 times time in this approximation.
It looks like it's first order coming down, with a rate constant k1.
It looks like basically the branching into C is nonexistent as far as, at least in this approximation, as far A is concerned.
It's going to look like this thing here.
It's going to look like first order A goes to B with rate constant k1.
So if you didn't know, if you didn't know that there was another substance, C, that was being formed along the way, if you didn't measure it, if you didn't do the analytical chemistry and sort of fish it out, and you just looked at the two major components, A and B, in this case here, you'd measure a rate out of A, it would look like it's first order.
It's great.
Got my mechanism.
A goes to B, period.
And you would know there was a minor component that was being formed at the same time.
OK, let's do another approximation.
Let's do the other case, where k2 A0 is much bigger than k1.
And, in this case here, you have to be a little bit more careful.
You have to add that you're going to look at this at early times.
And we're going to see why early times is important here.
In just five minutes.
And early times, and how you define early times.
What does it mean for things to be close to time equals zero?
Well, we have to have a reference time scale.
The reactions, I'm going at a certain rate.
And one second may be very slow.
Or it may be very fast.
Depending on the rate of the reaction.
So k1 here defines the time scale of the problem.
It's one over second, unit of one over second.
And early times means that the times that I'm looking at compare to this rate, it's very small.
So k1 t is less than one.
That means that the time, during the time period that I'm looking at, hardly anything has happened to the branching of A to B.
That that's what I mean by early time.
So B is hardly being built up yet.
k1 is our reference time.
k1 is units of one over second.
This has units of seconds.
So it's all working out fine.
In other words, I would have no idea how to define early times.
I could say, it's in a year, it could be, if you're looking at plutonium decay, it's a 100,000 year half-life.
A year would be an early time.
So it would be really fast, compared to the process.
But if you're looking at a reaction that takes a few nanoseconds than a year would be very, very long.
So you really need your reference time somewhere in there.
So this approximation means that essentially, no B is being created.
While we're looking at the process.
And so we might then very well expect that if we look at A and C, that the answer is going to be that we're going to see something that's second order.
So we expect that the form of A, the way to write it is going to be one over A equals k1 t plus A0.
So let's go ahead and take our complete solution and write it in the way that we might expect the answer to be.
In terms of one over A rather than A here.
So let's just invert it.
e to the k1 times time, k1 plus k2 A0, minus k2 A0 over k1 A0.
This should be a minus sign here.
There should be a minus sign here.
It didn't matter for our approximation because we got rid of it anyway, but it would be a proper sign here is minus.
And so now we need to make our approximation and figure out what this here.
So if we take the approximation that k1 t is small, less than one, then what that should trigger in your mind if that if you have e to the something that small you should use a Taylor series.
And expand this into a Taylor series.
So we're going to see this multiple times.
So we take e to the k1 t and expand it out as one plus k1 t plus et cetera.
We're going to keep the first order terms in there.
So we're going to plug this approximation into here.
That's this approximation there.
And then we're going to expand it out.
So plug it in here, and, keeping it to first order in time, and multiplying it out, and I'm not going to do the intermediate steps, I'm just going to give you the stuff after doing the multiplications, we get k1 t squared.
So that's basically k1 times one plus k1 t, and then you have plus k1 k2 A0 times t. That's this term here.
k1 k2 A0 times t times that.
And the k2 A0 times one gets subtracted out from this minus k2 A0 here.
So that's all we have on top.
And then there are terms of higher order in time.
But we're not going to worry about this.
We're going to stick to first order in time.
Divided by k1 A0.
And now you do your cancellations.
We have our approximation at k1 times time is much less than one.
So here we have k1 times k1 times time.
So this term here means it's much less than this term here, because of our approximations.
So we can get rid of this term here.
Get rid of that.
There's no reason for us to get rid of this one here.
Because we've got k2 A0 there, which is much larger than k1.
And so when you expand this out, all the k1's cancel out here.
So so k1 cancels out that k1, we have one plus one.
So this is basically one over A0.
And then we have plus k2 A0 time over A0, so this ends up being equal to one over A0 plus k2 times time.
Exactly what we expected.
That if you put in the right approximation in the math, it behaves as your intuition would tell you.
Which is that the branching into B nonexistent.
And that you're looking at everything going to C. It looks second order in C. But in this case here, it's only early times.
So if you keep going, what happens?
If you keep going in time, beyond the time scale of this first order rate constant, and you wait long enough, then the amount of A keeps decreasing.
Keeps getting smaller and smaller.
So if you start the clock again, a little bit later, where A is much smaller, then k2 times the amount of A you have there, is no longer going to be much greater than k1.
At some point along the way, k2 times A is going to be much less than k1.
So at some point along the way, it's not going to be this approximation any more.
It's going to be the previous one that we did.
The one up there.
So at early times we start with something that may look second order for A.
Time goes on, time goes on, time goes on, A gets depleted.
A gets depleted, and the rate k2 times A, which is sitting right here, this part here becomes smaller and smaller.
And pretty soon this part wins.
And it becomes first order.
So what you'd expect to see, then, is if you were to plot as a function of A, you expect to see something that starts out as first order and then as A gets depleted, switches over to second order.
Something that's second order here.
And first order.
This is the concentration of A.
So if you were to plot it, here you're plotting the concentration of A.
If you were to plot the log, of the concentration of A, at long times you'd expect it to be first order.
So something linear.
But at early times you'd expect it to be second order.
So you'd expect to see something that's nonlinear, then switching to something linear.
And if you were to plot it as one over A, then you'd expect to start off with something that's linear, at second order, early times.
But instead of keeping, being second order, as you deplete the amount of A, the branching into B becomes important.
And you become second order.
So this is second order at the beginning.
And this becomes first order.
So, any questions here?
The importance of doing this problem here was to lay out basically a systematic way of looking at the problem.
You lay out your rate equations.
Like this.
You solve what you can, in this case you solve for A.
And it's complicated.
So you want to learn something about what the data might look like.
And you look at limiting cases.
Two obvious limiting cases.
One rate is faster than the other.
You pick the first one first, then go through it.
And then use your intuition.
Every point along the way, your intuition can tell you what you expect the result to be.
So in this case here, our intuition told us that if we took the rate into B to be much faster than that into C, than we expected to see something that was largely first order.
And when you we the approximation in our exact solution, in fact it does look like it's first order.
So always use your intuition.
Because the math is going to get pretty hairy.
The algebra might make it complicated.
And it's really easy to make a plus sign into a minus sign somewhere along the way.
Just like I did here.
So if I hadn't fixed my mistake here, and I had gone and put in my Taylor series here, I would have gotten in trouble.
Because this minus sign would have been a plus.
And these k2 A0's would not have canceled out.
And I wouldn't have gotten the right result, which my intuition had told me should be a second order process.
And so if you have a problem, let's say you're on an exam and you're doing algebra, you're trying to, and you end up with a result that doesn't match what your intuition tells you, and you don't have time to fix it.
Tell us.
You know, I know this is wrong because I know it's supposed to be second order.
But I don't know where I went wrong.
And that's really important.
Just tell us.
That means that you're thinking.
Let's get a little bit more complicated.
Any questions?
Consecutive series reactions.
These were parallel reactions and now.
Basically what we're doing here is we're building up a toolkit of simple mechanisms.
And then we'll be able to put these mechanisms together to make something more complicated.
So the next kind of mechanism, series mechanism, series reactions, so we have our reaction.
Which is A goes to C. And the mechanism that's been thought for this reaction is that there's an intermediate.
That first you have A goes to B, some intermediate, which gets used up to form the final product, C, with a rate, k2, here.
And we can either write the mechanism in two steps.
Or we can write it as A goes to B goes to C, like this.
And see both ways of writing it.
And we want to solve this.
And we want to look at special cases, and we just want to understand the implications of this mechanism.
So the first thing to do, just like we did before, is to write all the rate laws.
Before we got going solving anything.
So we start with A, minus dA/dt.
There's only one way that gets used up.
Through the formation of B in a first order process.
Easy.
This is easy to solve.
We know the answer is going to be exponential in it.
For B, dB/dt is equal to, well it can be formed through my first order in A, so that's k1 times A.
But it also could get destroyed by forming C. So we've got to get all the paths out of A, here on the right side of this equation.
So it can be destroyed in a process, which is first order in B.
So we're going to take both k1 and k2 to be first order initially.
Then we'll make one of them second order and things will become very complicated and we'll throw up our hands.
But for now, let's not throw up our hands quite yet.
OK and for dC/dt, there's only one way it can be formed is first order by destroying B.
And we have a couple of differential equations here.
This differential equation for C, depends on B here.
The differential for B depends on A here.
So that means that things are going to get a little bit more complicated.
The easy one to solve is always the first one here.
That's first order.
So we just write down the answer.
A is equal to A0, e to the minus k1 t. We know the answer to that one.
Then we have to work on B.
So we want to find out integrated rate laws for every one of these chemical species.
And always there are tricks involved.
Partial fractions, whatever.
So we write down B now.
dB/dt plus, let me rearrange it a little bit, plus k2 times B, putting the minus k2 B on the other side here.
Is equal to k1 times A.
But I'm not going to keep this A there, because now I know what A is in terms of time.
And a constant.
k1 times A0 e to the minus k1 times time.
OK, that's my differential equation for B.
Got to solve this thing.
Well, the last time I did differential equations in college was a century ago.
And I wasn't kidding.
It really was last century.
In fact, it was last millennium.
That was a long time ago.
So how do you do this?
Well, the trick here, there's a trick.
And the trick is to multiply the whole thing, both sides, by e to the k2 t. And you multiply this side by e to the k2 t and you multiply this side by e to the k2 t, to solve this equation.
So once you do that, then you have e to the k2 t times dB/dt plus k2 times B.
This side here.
And then you have is equal to k1 times A0 e to the minus, e to the k2 minus k1 times time.
OK, so the reason why this trick works is because this guy right here can be nicely written as d/dt of B times e to the k2 t. It's in nice simple form, so when you integrate, it'll be very easy to integrate.
And then the other side, you still have k1 A0 e to the k2 minus k1 times the time.
So now you integrate both sides with respect to time, The integral of something d/dt with respect to time, it's just going to be itself right here.
So it's very simple.
So you integrate both sides from zero to time, dt.
Zero to t, dt on both sides.
And on this side here, you end up with B, e to the k2 times time minus B, e to the k2 times zero.
Just taking the upper limit minus the lower limit.
And that's just equal to B0, at time to equal zero.
And so, and at time to equal zero, B0 is equal to zero because we haven't made any intermediates.
So this goes away.
And then we have an equals sign.
And on the other side, we have k1 A0 over k2 minus k1, it's just the integral of this thing, here with a k2 minus k1 goes in the denominator.
e to the k2 minus k1 times time minus one.
So now you have your integral form for the amount of B that gets created at any time.
And I want to keep that on the board.
Because I'm going to use it later.
k1 A0 over k1 minus k2 times e to the minus k1 times time minus e to the minus k2 times time.
OK, we're two thirds of the way through.
Now we've got to find C. And for the last component, you don't want to be doing this kind of stuff.
You don't want to waste your time solving differential equations, because stoichiometry can help you out.
At the end of the day, you know that everything that started out as A, in terms of quantities, is going to end up at C. You know that if you end up with a concentration A0 at the beginning, at the end of the process the concentration of C, at infinite time, is also going to be A0.
If you know that there's a correlation, there's a stoichiometry that relates A and C together.
And B.
You have conservation of mass here, basically.
Conservation of atoms.
That has to come into play So, for C, we're just going to do algebra instead of calculus.
So let's write down the stoichiometry here.
The amount of C at any time is what it's going to be at the end.
Which is A0, very end of the process.
It's going to be A0.
But before we get to the end, there's still stuff that's left in A and B.
And that's going to decrease the amount of C. So minus whatever's still in A and B.
That's the stoichiometry.
That's one way of writing it.
You could also write it in a different way.
Which is that the amount of C is equal to the amount of A that you have used up.
A0 is what you started out with.
A is what's left over.
So this is the amount of A used up.
It's not all quite in C yet.
Some of it is stuck in B.
So it's the amount of A used up, minus the amount that's stuck in B.
Those two are the same.
I hope.
A0 minus A, yeah, that's right.
Minus B, yeah, that's right.
So this is used up.
And then stuck in B.
That's the hard part.
Putting down the stoichiometry.
Once you've done the hard part, then if you solve the other two then it's easy.
You just plug in.
It's just algebra.
And you get something complicated.
C is equal to complicated.
It's in the notes, I'm not going to write it down.
So we've solved the problem exactly.
And now you're going to do an experiment.
And the experiment you're going to do is not going to follow this whole thing, exactly.
It's going to look at limited cases.
That's what you can do.
And so the first thing to look at is to look at the initial times.
The very beginning of the process, how are things appearing.
And then we'll look at the late times.
So initial times.
Near t equals zero.
Well, every one of these things has an exponential in it.
And whenever we see exponentials and we look at something at the early times, times that are faster than one over k1, or faster than one over k2, it tells us use a Taylor series.
So use the approximation e to the minus kt is approximately equal to one minus kt plus kt squared over two plus et cetera.
And keep as many terms as you need in the Taylor series to get something sensible.
So if everything comes out of zero at the end, you know you haven't kept enough terms in t. Things have canceled out too much.
And you want something that is time dependent.
Because you've got a change in time, right?
It's a very common mistake to say only keep one here and then everything cancels out you get zero, and you say, well, B is equal to zero at all times.
That's nonsensical.
And I don't know a priori how much time to keep.
So let's go to second order.
So if you plug this Taylor series into A, and get something that's time dependent, you only need to go to first order.
One minus k1 times time plus et cetera.
Plug it into b, plug your Taylor series into these two guys here, get k1 A0 over k2 minus k1, you only need it to go to first order.
The one's cancel out but the times don't cancel out.
Minus k1 plus k2, that is, unless k1 and k2 are the same.
So we're making the approximation that k1 is different than k2.
Times the time.
And when we look at C, this complicated thing that I didn't write down, if you were just to stop at first order in C, you get C equals zero.
You know that can't be right.
You know that C is going to increase in time.
It can't be equal to zero at all times.
So we have to go to second order for C0.
And it's approximately equal to A0 times k1 k2 over two times the time squared.
Let me rewrite this a little bit closer in, because I need some room here.
So c is approximately equal to A0 k1 is k2 over two times time squared.
OK, and then we can start plotting out what this is going to look like then.
So we have the concentrations time on the axis here.
Let's start with the concentrating of A.
A is dropping down linearly in time.
It's dropping down linearly in time here.
B is increasing linearly in time, at the beginning.
So B is going up linearly in time, like this.
And C is increasing quadratically in time.
So it's pretty flat, coming up first.
And then it starts to curve up.
It's our first approximation.
Then, we go to late times.
t equals infinity.
So t equals infinity where am I going to write this?
Have to move this up.
Let's go, t goes to infinity.
Well, here I know that I'm allowed to go to zero.
Or allowed to go to constant, because infinity is forever.
So at t infinity, I know that I'm going to use up all of A.
So I know that A is going to be equal to 0.
I know I'm going to use up all of B.
So I know B is going to go to zero.
Somehow.
And I know that C is going to go to A0.
So C is going to go to A0 at infinity.
So now on my graph here, I know that at infinity, A is going to go down to zero like this, somehow.
Probably exponentially, because it is exponential in time.
So it's going to go to zero exponentially.
B is going to go down to zero exponentially.
There are the exponentials right here.
So B is going to go down to zero exponentially.
Something like this.
And C is going to saturate to A0.
Exponentially rise up to A0.
So this was A0 right here.
It's going to go like this.
So then I connect the dots.
And I know that C is going to start quadratically.
And then it's going to rise up an exponent like this.
I know that B is going to start off linearly, and then it has to go to zero.
So it has to go through some sort of maximum.
And A is going to go down exponentially, exponentially like this.
So the interesting part of this diagram is that B goes to a maximum.
There is some point, there's some time, that we can call t sub B max, where B is a maxima.
And that's an interesting point.
That's something that we can try to figure out experimentally what it is.
And we could get some parameters, we could maybe extract some information.
There's a maximum B, concentration of B, that gets formed.
And how do you solve that?
You set dB/dt equal to zero.
So you have your equation for B right here.
You set it equal to zero.
and you solve.
You get your maximum time.
It's made up of rate constant.
You get your maximum concentration, which is also made up of rate constants.
And you can get rate constants out of this.
Question
[INAUDIBLE]
Yes
[INAUDIBLE]
That is a coincidence.
That is my poor sketching.
It should probably be somewhere like, in the middle here.
Actually, it completely depends on the rates.
And we're going to see limiting cases where this maximum can occur here, or it can occur somewhere down here.
So this is the complete case.
So, limiting cases.
the important limiting cases.
We've made an approximation along the way that k1 is different than k2.
So that has to be a case that needs to be worked out.
And it's likely to be easier than the full solution.
So one of the limiting cases is k1 is equal to k2 and I'll leave that for homework.
You can do that yourselves.
It's good exercise.
What about another limiting case?
Well, you have two other limiting cases.
One, you've got two rates.
So, one is bigger than the other.
No information on this board here.
You start off with one case.
So k1 greater than k2.
k1 greater than k2.
What does it mean?
k1 greater than k2.
It means that the rate determining step, or the limiting step, in this reaction, is the second step.
k2 is very slow compared to k1.
And sometimes I like to think of these things as pipes.
And connecting vessels.
So let's say we have the amount of liquid in the top vessel is the concentration of A.
The amount of liquid in the middle vessel, let me get my color scheme to be consistent here.
Then we have a middle vessel, which is the concentration of B, and a final vessel, which is the concentration of C. And we start out with some sort of amount of A in here.
Started with a lot of A, and then these vessels are connecting with pipes.
So there's a very thick, big pipe that connects A and B together.
The rate is fast, going from A to B.
And the rate going from B to C is slow, so there's a skinny pipe connecting B and C together.
Then I turn on the system, I set time to equal zero, poof, what happens?
There's the big pipe connecting A and B.
The whole amount of a here gets transferred to B, suddenly.
Then it gets stuck here.
And then it dribbles out from B to C. So what do we expect, if we were to plot our quantities as a function of time.
What we'd expect then, then we're going to make sure that the math works out, is that A is going to decrease really fast.
It's going to go, poof.
First order in time with a very fast rate.
And all of this is going to go straight into B.
So we expect B to go linearly.
Remember, at the beginning it's linear.
Linear goes up.
Almost all the way up to A0.
Because it can't hardly get out.
Reaches its maximum, then it goes from B to C through this skinny, skinny pipe.
First order rate, with a rate k2, so we're exponentially going down, slowly, slowly, slowly, with a rate k2.
And then C starts quadratically.
And then very quickly, once everything's into B, A is forgotten.
A doesn't matter any more.
Basically what you're looking at is a transfer of B into C through this little pipe.
And you know that's going to be a first order of rate.
First order of process, B to C's first order of process with rate k2.
So you expect, then, C to go up in the first order process to A0 with rate k2.
There's the rate k2 here.
And the rate k1 here.
So when you turn the crank on your approximation in the math, what you'd better see is that at the end, B as a function of time should look like a first order process with rate k2.
C as a function of time should look like a first order process with rate k2, going to A0.
And you can write the answer, you should be able to write the answer, for C, it's going to be C is approximately, it's going to go to A0, there's no other choice.
Everything that was in A gets transferred to B0, so at time is infinity it's going to be A0, and it's going to be a first order process with rate k2.
One minus e to the minus k2 times times.
I don't even have to do the math.
I know that's the answer.
If you don't believe me, then do the math.
You should do it.
But that's the answer.
It has to be the answer.
And if you look at B, the answer has to be that it's approximately looking like B is disappearing with a rate constant that's k2, e to the minus k2, and the maximum here is very close to A0.
So I'm going to be putting A0 here.
Close enough.
So as long as I ignore the very initial time where A suddenly dumps into B, then everything after that very early time here is just looking like B goes to C. This is not zero, this is t. And A just disappears quickly.
And we can write it as A is equal to A0 e to the minus k1 times the time.
That's an easy approximation.
And you've just got to make sure that this fits.
So the other limiting case, which I'm also going to leave as a homework, because it's so straightforward, is to have k2 greater than k1, and you should make sure that you can predict it.
and that the math works out.
OK, any questions?
Next time we'll do reversible reactions.
And some more approximations.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time we were starting more complicated mechanisms and we looked at series reactions.
A goes B, which is an intermediate.
Goes to C with rates k1 and k2.
And we solved the problem exactly.
Which turned out to be a little bit complicated.
And then we were looking at special cases.
And I left the special case of k1 equals k2 to do as a homework problem.
And then on the board we did k1 much greater than k2.
Which looks like, if k1 is much greater than k2, the rate into k1 is much greater than k2, so using the analogy of buckets and pipes, it's a big pipe between k1 and between A and B.
And a little pipe between B and C and you started with a bunch of liquid on top.
And you basically dump it into here.
And then you slowly drip into C here.
So the first thing that happens is that A gets dumped, it becomes B very quickly.
With a rate constant k1.
In a first order fashion.
So A is A0 e to the minus k1 times time.
Then B comes up sharply.
And then drops down with rate constant k2 slowly.
So B is approximately A0 e to the minus k2 times time.
And then C has a very small quadratic rise, followed by the exponential rise to saturation that you'd expect.
So C is approximately a0 times one minus e to the minus k2 times time.
The rate constant k2 dominates the long times.
And everything it's looking like it's pseudo first order at long times.
So you can do this either by thinking about it, solve the problem by thinking about it, which is perfectly fine.
Or you can do it by solving it exactly, as we have done last time.
And then putting in the right approximations.
Which in this case was Taylor's approximation, if I remember right.
And know just by looking at the equation and putting the right approximation, you get it out.
And a third approximation, which is the opposite one, k2 much less than k1, so k1 much less than k2, the way that that's going to look, we can do it by inspection.
So basically if we think of buckets and pipes, that's a very thin pipe connecting the bucket for A and B, so we drip from A to B, then we have this very fat pipe connecting B and C. So as soon as a little B gets in there, it immediately becomes C. So, the rate determining step is the first one, A goes to B.
And so k1, we expect that to be the rate that dominates the long time.
And we never expect to get any significant amount of B to pile up in the middle bucket, because this pipe is much bigger than that one here.
Therefore, what we expect to see is a slow decay in A, with rate constant k1 dominating the long times, A is approximately A0 e to the minus k1 times the time.
We don't expect B to amount to very much at all.
It'll rise up a little bit at the very early times, and it'll pretty much stay very, very small.
Not quite constant.
And in fact if you look at the full result, and you put in your approximations, and I'll let you do that, you would find that B is essentially related to A through a constant term where k1 is very small compared to k2.
So this is a very small number here.
B is equal to a very small number times A.
So it basically follows A, but at a very very, low level.
And then you would find that C rises up in approximately e to the minus k1 times t, so with rate constant k1.
One minus e to the minus k1 times the time.
So the dominant rate is k1, and in this case here this is an approximation that we're going to see again.
We're going to revisit this approximation here when we get to more complicated mechanisms, it's going to become this steady state approximation.
Where the amount of the intermediate, or the concentration of intermediate, is low at all times.
And doesn't change very fast.
The slope here is very slow.
Because k1 is so slow coming down, and B is related to A, which means it's related to this slow decrease in A, the amount of B is small and the rate of change in B is also very small.
And hopefully we'll get to that very soon in this lecture.
OK, so that's a summary of what we did last time.
Any questions?
So the next thing now is to go to the next mechanism.
Which is, so far we've looked at reactions that proceed all the way.
We haven't looked at equilibrium yet.
But we know thermodynamics is all about equilibrium.
So now we're going to connect kinetics and thermodynamics.
We're going to look at a mechanism which is an equilibrium mechanism.
Equilibrium or reversible reactions.
Where you have A goes to B, with summary constant k1, but you can have the reverse process B goes back to A with the rate constant k minus one.
Or you can write is as A k1, k minus one to B, where k1 could be k forward and k minus one could be k backwards.
Also written as k sub f, and this is also written as k sub b.
So now we have more information than we had before.
Because now we know this is an equilibrium problem.
And we already know about equilibrium from thermodynamics.
And we know that at equilibrium, in addition to all the rate laws that we're going to write down to solve the problem, we also know that at equilibrium, the equilibrium constant is related to the equilibrium concentrations.
To the ratio of the equilibrium concentrations.
This is something additional that we have.
So let's write down everything we know.
And our goal is to find out the time, the dynamics of this problem.
Suppose you start at some time t equals zero, at a certain amount of A0 in your pot, a certain amount of B0, how do you get to equilibrium.
What's the rate, what does it look like?
So the first thing is to write down the rates.
So the forward rate minus dA/dt, which is the forward rate, R sub forward is equal to k1 times A.
And the backwards rate, minus dB/dt, which we can write as R sub backwards, that's k minus one times B.
At equilibrium, we have no dynamics.
Or no ensemble dynamics.
The pot looks constant.
It doesn't look like anything's going on in there.
The concentration of A doesn't change.
The concentration of B doesn't change.
They're constant at the equilibrium concentration.
So that implies that the rate of formation of B must be equal to the rate of destruction of B.
Or, in other words, that the rate forward at equilibrium has to be equal the rate backwards.
Otherwise, there'd be changes in concentrations.
So if that's true, then that means that at equilibrium, k1 A equilibrium has to be equal to k minus one B equilibrium.
Which means that we can get this ratio of B to A.
And see that this is equal to k1 over k minus one.
So we've just, this is pretty major here.
Looks pretty simple, but we've taken an equilibrium property here.
And related to kinetics.
To a rate property.
So here we have kinetics.
Here we have thermo.
And this is going to turn out to be, this kind of ratio between forward rates and backwards rates, related to equilibrium constants is going to be valid, not just for this very simple mechanism we have here with just two species.
But it's going to be valid for more complicated.
And we have two species reacting to form another species, or bimolecular stuff, et cetera.
OK, so now we've laid the groundwork, let's look at the dynamics.
Let's look at the time evolution.
So that means that we need to take, well this is not quite right.
I shouldn't write minus dA/dt.
This is minus dA/dt rate forward in the absence of reverse process.
It says just the destruction of A and not the creation of A.
So if you look at the full kinetics, the full mechanism, this also, the rate backwards is minus dB/dt, in the absence of forming B back through the reverse process.
So both of these are in the absence of their reverse process.
The full kinetics, for the destruction of A.
A is the rate backwards, or the rate forwards, which is k1 times A, and then the rate backwards, which is the creation of A.
That's the two rates together.
Minus k minus one times B.
There's the formation of A here, this is the destruction of A.
This is the full differential equation for the full mechanism.
Not just one part of it.
This one here is just for the first step, and that one here is for the second step.
This is what we need to solve.
But now we have a lot of help by what we've already done.
So, what we can do is well, the problem here is that's it looks hard because there's B included in here.
This is not just A, there's B in here.
But there's only two species.
And stoichiometry is going to help us out.
Remember last time, or a few times ago, we said the last species is the easy one.
Because it's always related to the concentration of all the other species by stoichiometry.
And the last one here is B.
So we know that B is related to A.
B is whatever you started out with.
Plus whatever you used up for A.
That's one way of writing it.
So you start with this amount, and you create this amount.
By having A react.
This is what you started with, A, and this is what's left.
The difference has to go into B.
So stroke stoichiometry give you a relationship, which you can use to plug into your differential equation here.
And now, after rearranging your differential equation, putting B in there, you can rewrite this as k1 plus k minus one times A minus k minus one times B0 plus A0.
Something which depends on A, something which is constant here.
OK, and what else do we know?
We know something about equilibrium.
We know something about when we reach a state where the rate forward is equal to the rate backward.
When the rate forward is equal to the rate backwards, the rate of change of A is zero.
A is not changing.
The concentration of A is not changing.
So at equilibrium, dA/dt is equal to zero.
And just like in this example here, this is going to turn out to be related to an approximation called the steady state approximation, which we're going to see later this morning.
Writing at equilibrium that the change in the concentration of one of the species is equal to zero, we're going to use that as an approximation also later, when we get to more complicated mechanisms.
This is going to be the equilibrium approximation.
And this is going to be the steady state approximation.
Let's keep going with our, we're solving this guy here.
So at equilibrium, dA/dt is equal to zero.
And then we can replace our concentrations here with, and so if you write equal to zero here, then we can write it as equilibrium right here.
And we'll be able to solve for A equilibrium in terms of k1, k minus one, B0, and A0.
And then if you do that, you get A equilibrium is equal to k minus one over k1 plus k minus one times B0 plus A0.
And we're going to need that.
Because now we're going to be able to plug this, so there's A0 plus B0 sitting here.
And we're going to replace A0 plus B0 in terms of A equilibrium.
There's a minus sign here.
And between this A and this A equilibrium here, so by rewriting, by plugging in A B0 A equilibrium instead of B0 plus A0, we can rewrite that equation as dA/dt is equal to k1 plus k minus one times A minus A equilibrium.
This is nice because, this quantity, A minus A equilibrium, describes the difference between where are you are and where you want to be.
It's a nice variable to have.
It's going to change in time and at infinite time, this is going to go to zero.
A is going to go to equilibrium.
A equilibrium is a constant.
Now we have a differential equation that relates A to this difference.
But A equilibrium is a constant.
So there's nothing that forbids us from just writing minus A equilibrium here.
d of a constant dt is zero.
So I'm just basically subtracting zero here.
So and I have something of the form dx/dt is equal to constant times x.
And I know how to write that.
That looks just like a first order process.
I know the solution is A minus A equilibrium is equal to my initial A, my initial state.
A0 times e to the minus k1 plus k minus one times time.
And I've solved the problem.
I've described how, as a function of time, how A, the concentration of A, gets equilibrium.
And if you want to know B, it's just the same thing.
You replace B here and B0 here.
You put k minus one here, k1 here, but it's the same thing.
So if you were to sketch it out as a function of time, there's, eventually you'll get to the concentration A equilibrium.
If you're above it, you're going to decay in a first order fashion, with a rate constant k1.
A minus A equilibrium is A0 minus A equilibrium e to the minus k prime times t, where k prime is k1 plus k minus one.
And this is when A is greater than A0, A equilibrium.
A0 is greater than equilibrium.
And if you start below, you're going to come up like this.
A0 is less than A equilibrium.
And both rates come into here, in a very simple way.
Just the sum of the two.
So experimental, if you want to find out these rates, you can measure their equilibrium concentrations.
Measure k equilibrium by obtaining the equilibrium concentrations of B and A.
And that gives you the ratio of k1 plus k minus one.
And then you can just start the process with some concentration of A.
Which is different in equilibrium.
Then watching it in time, extract out, observe and measure k1 plus k minus one.
In this kinetic equation.
At kinetic relation.
And then you have two results, two pieces of data.
You've measured k prime, which gives you the sum of the two.
And you've measured K equilibrium, which gives you the ratio of the two.
And you've got your two rate constants out.
OK, any questions for the equilibrium problem?
We're slowly -- yes
[INAUDIBLE]
There is not yet a relationship between k1 and k minus one.
It depends on the problem.
And we're going to study the, when we get to potential barriers, we look at, we'll look at this issue.
OK, it's a good question.
We're going to do that probably next time.
So let's get going, then, with reversible reactions.
So now we have, let's make it a little bit more complicated.
Let's, instead of two species, let's make it three species.
So we have A plus B goes to C, where the rate constant k1, and then C goes to A plus to B with a rate constant k minus one that you can rewrite as A plus B goes to C, k1, k minus one, and then you write down your, you want to solve this.
So the first thing you do is your write down your differential equations.
dA/dt is equal to, put all the rates in there.
k1 A B minus k minus one times C. And at equilibrium, we set that equal to zero.
So you can write equilibrium here, equilibrium here, equilibrium here.
And when you bring this term over to the other side, you find, then that at equilibrium the ratio of the forward rate to the backward rates is equal to C equilibrium divided by A equilibrium.
The equilibrium, which, you know is the equilibrium constant.
So again, for this slightly more complicated problem, the ratio of the forward and backward rates are related to the equilibrium constant.
So now you want to solve this.
Well, we're not even going to try.
Because it's going to be too complicated.
So as soon as you get away from first order kinetics and go to second order kinetics with multiple steps, it's a mess.
So instead you try to find approximations.
In this case here there's one we can use.
Or a limiting case, at least.
One we can use, which is an obvious one, which is flooding.
As a limiting case.
If we want to isolate a small part of the problem, we can use flooding.
Because we can overwhelm the system with either A or B.
Take B0 much greater than A0 and C0.
So over the course of the reaction of the process, the concentration of B hardly changes.
And so when we write our kinetic equation, our rate law, k1 A B minus k minus one C, we could put a little naught here, because we know the concentration of B is not changing.
And so now we're left with the problem we had before.
Which was a reversible first order process which we have just solved.
And you go through the experiment and you extract out k1 times B0 and k minus one.
You do the experiment again with a different concentration of B0 to start out with.
And you can extract out k1.
So this is a process of getting the rates by using a simple approximation here.
Simple limiting case.
Any questions?
So we've just finished putting all the building blocks together now.
Let me remind you what these building blocks are.
We have a bunch of approximations under our belt.
Flooding, we've looked at rates being much faster than others.
And we have three simple mechanisms, we've looked at parallel reactions.
We looked at series reactions.
And we've looked at reversible reactions.
So a complicated mechanism will basically put these three building blocks together in a series.
And they'll all be happening at the same time somehow.
And obviously, since it was, since we threw up our hands, just with a simple reversible process like here, it's clear that we're going to throw up our hands a lot.
When we write down these mechanisms.
So we're going to need approximations.
We're going to need something that we'll automatically go to, and say this is going to be too hard.
I'm not even going to try.
Let's do an approximation.
And two approximations that we're going to talk about are ones I already mentioned at the beginning.
Which is the steady state approximation, where one rate, the rate to go into the intermediate is very slow, or the rate to get out of the intermediate is very fast.
And the equilibrium approximation where the system sets up a very fast equilibrium and you're allowed to use thermodynamics to help you out in solving the problem.
These are the two approximations that we'll use when we put these things together.
OK, so let's look at the first, simple, more complicated mechanism.
To see where these approximations come in.
So the first one is going to be a series.
We're going to start putting these things together.
First one is a series reversible.
Those two together.
Series reversible.
So we have first, a reversible process.
Everything is first order here.
k minus one goes to B.
And then B goes to C with some rate constant k2, all first order.
If we were to turn the crank, we'd say, oh, I've got to write down all my rate laws here.
Minus dA/dt is all the ways that I destroy and create B, so there's a k1 times A minus k minus one times B. dB/dt, all the ways that create and destroy B is that you create it through destruction of A. I destroy it through the backwards rate to make A.
And I destroy it by making the product, C. There's the intermediate.
And then for C, that's pretty simple.
There's only one channel into C, and that's the destruction of B to form C, k2 B.
All the rate laws.
I write everything I know here.
Couple of differential equations.
We know it's going to be hard to solve.
So, let's remind ourselves of what I just erased.
Which was the case where the intermediate concentration was always very small and didn't change very fast.
And the process was dominated by k1 here.
That was the rate limiting step.
Remember that our first example this morning.
So this is where the concentration of B is roughly constant over time.
It's small.
And over any small time period, it's roughly constant.
Which means that dB/dt is roughly equal to zero.
If you look at a long term, obviously it's going to change as you go from this point to that point.
In fact, there's a relationship between A which is changing in time and B.
But A is also changing in time very slowly.
Because the rate k1 is very small.
So dB/dt, which is related to this rate k1, is going to be very slowly changing in time.
We can approximate it at zero.
What are the ways in terms of matching these rate constants that we can get to the approximation.
To a diagram that looks like that?
But we don't want B to pile up.
That means we have to have the rates out of the intermediate to be much faster, at least one of the rates out of the intermediate to be much faster than the rate that creates the intermediate.
So the different ways of creating that approximation are if, and the length of the arrows now is going to be proportional to the rate.
We want it to be very hard to make B.
And as soon as we make B, we want it to go away.
So one of the ways to do that is to have the reverse rate much faster than the initial rate.
And we could have a slow rate into C, that's fine.
We could have, again, a very slow rate into B.
We could have a slow reverse rate and a fast rate into C. Perfectly fine.
We could have both.
Fast rate out of B through A, or out of B through C. As long as this first rate into the B is small compared to one of those two rates, we're never going to pile up B.
It's going to go away as soon as we make it.
So in all these cases, k2 plus k minus one is much bigger than k1.
When we have that situation, then we can make the approximation up here.
That the rate of change in B is very small.
Almost zero, which means it's basically a constant.
And instead of writing B in this case here, we're going to write it as B steady state.
Which is basically a constant in terms of solving the differential equations.
And that's going to make our life much easier.
Because instead of having to solve these coupled differential equations, we're just going to have to solve coupled algebraic equations.
Which is really messy, but less hard.
If you're a bean-counter than this is heaven.
So now we're going to put steady state, this is going to be a constant.
And this is going to be equal to zero.
And then here this is going to be steady state here.
This is going to be steady state here.
And the first thing to do now is that we've eliminated this differential equation.
We now have an algebraic equation where we can solve for B steady state.
So if you recognize that you're in this situation here, forget about trying to solve.
Immediately go to the process of putting in a constant for B.
Setting dB/dt for the intermediate equal to zero.
And then starting to turn the crank on the algebra.
So let me go ahead and turn the crank here.
Go through the steps, which is basically typical of these problems.
So you turn the crank.
You solve for the steady state concentration.
You get, in terms of A, over k minus one, plus k2.
Then you plug this back in here.
And you get a new differential equation, minus dA/dt is equal to k1 times A minus k minus one times B steady state, so we plug this in here.
And you get, so there's A sitting here.
A sitting here.
It's going to be of the form, effective rate times A.
It's going to be basically a first order form.
Which is what we'd expect from sketching the diagram just like that.
So you get minus dA/dt is an effective rate.
k1 times k2 over k1 plus k2.
Times A.
This is k prime.
So if you were to do an experiment under these conditions, you'd find that A behaves, it's basically a pseudo first order problem.
With a funny rate that contains all the elementary rate constants as part of it.
And when you do the same thing for C, dC/dt, you find that depends on A.
It's k1, you plug these steady states in here.
k1 k2 divided by k1 plus k2 times A.
This is k prime again.
So the problem, and it's first order.
So the problem looks like effectively you're going from A to C. Forget about the intermediate.
With an effective rate k prime.
Where k prime contains these rates.
So this is steady state approximation.
It's the prototypical problem.
Questions on steady state, before we go to the next approximation.
Alright.
So now, as promised, the next approximation is that where are set up the fast equilibrium.
And you can use thermo to help you out.
Equilibrium approximation A goes to B.
This is a fast process.
And then k2 out of B is a slow process.
Little arrow here, and two big arrows here.
You put your A in your flask.
Immediately you set up the equilibrium.
And you slowly dribble out of that.
If you want to do it as a function of buckets and, so you have a big pipe connecting two buckets that are just offset from each other.
So there's, in this case here B is favored over A.
Because it's a little bit lower than this guy here.
And then there's a little bucket, there's a little tube, comes out of here.
So there's a little dribble into C over time.
First thing that happens is you set up your equilibrium.
And then you slowly extract stuff through a little tube into C. So what do you expect this to look like?
Well, you expect A to really slowly come out.
Because the rate limiting step is the rate from B to C. So k2 is going to be the way that A is going to come out.
You expect it to slowly go away.
You expect B to get created very fast, to some equilibrium amount.
And then also to follow the same rate k2, slow k2, to disappear.
And you expect C to come up.
You're basically in a first order process to saturation of A0.
So when you expect the dominant rate to be k2, and the fast dynamics to happen at very, very early times, and then everything to follow first order kinetics.
So now let's do the math and make sure that it agrees with what we've just assumed that it's going to look like.
So equilibrium is happening.
You can assume that at any time after the initial very fast process of getting equilibrium.
So after some initial time, the ratio of B over A is always going to be a constant.
The dribble out of B here is not going to be fast enough to change that ratio.
As soon as you get a little bit of B out here, immediately the ratios, the amounts rearrange to keep the ratio constant.
So now when I look at the rate of the reaction, the rate of formation of C, k2 times B, well, in terms of A, it's k2 times K equilibrium.
Times A.
Which is k2 times k1 over k minus one times A.
And you can immediately see that C is behaving, or the reaction is behaving, like a first order process.
With an effective rate which contains all three rates in this ratio here.
So it looks like A goes to C, with some k prime where this is k prime.
It looks like A goes to C with an effective rate constant k prime.
OK.
Questions?
We're going to do some examples.
And then we're going to do chain reactions next time.
We're one lecture behind.
Alright.
Let's do some reactions.
Some examples.
I'm going to skip the first example, which is the example with the chaperone, in the notes.
And I'm going to go directly to the gas decomposition.
The two examples are basically very similar in their use of the steady state approximation.
So I'll let you read over the first example.
So this example here is going to give us the Lindemann mechanism, for which Mr. Lindemann got a Nobel Prize many many, years ago.
Basically, it's looking at decomposition of a gas phase molecule.
Where you have a molecule, A, that breaks down into products.
And the observation before Mr. Lindemann got around, was that it looked like a first order process.
It looked like A is just falling apart on its own into these products.
Mr. Lindemann got around and said, well, why would A just want to fall apart.
There's something a little bit odd about that.
Stable molecule, why would it want to fall apart.
And so he hypothesized a mechanism.
And it went like this.
That A actually collides with another molecule, which could be a bystander.
Could be a chaperone molecule that just happens to be there.
Or it could be another molecule of A.
There's a collision.
There's a collision that creates a vibrationally excited version of A.
Of A sitting there.
It collides.
It suddenly starts to be really vibrationally excited.
Bonds vibrate all over the place.
Atoms wiggle.
There's the collision partner that goes away.
So kinetic energy is transferred from M and A to the vibrational loads of A.
In a process which is reversible.
k1, k minus one, because this excited A, this vibrationally excited A, could also collide with a molecule and cool down.
The energy in the vibrations could be transferred back to kinetic energy for a reverse process.
But if you wait long enough, or if this vibrationally excited A, with these atoms widely moving around, that has more of a chance of falling apart than this guy here.
So A star could fall apart into products.
With a rate k2.
So he said, this is actually an important step.
It's not that A just suddenly falls apart.
But this has to happen.
So all the observations that supported a first order process, which basically only supported one elementary reactions, might not be right.
So let's assume his mechanism, and let's look at what we need to do as an approximation.
And where that leads us in terms of trying to experimentally confirm that this mechanism is plausible.
Not prove it, but just to make sure that it's consistent with, the data's consistent with the hypothesis.
So let's compare our rates here.
So we have a collision.
A and M collide, and there's a certain probability on the collision that kinetic energy will be transferred to vibrational energy.
That turns out to be a moderate probability.
So k1 is reasonably fast.
The reverse process, you have something that's vibrationally excited colliding with a molecule.
There's a probability that that excitation is going to get turned back into kinetic energy.
And then these two molecules will fly apart with high velocity.
And that tends to be much more highly probable then the first process.
So this is faster.
Then the excited molecule's waiting around.
And there's a probability that there's it's going to just fall apart.
And that turns out to be really slow.
So we're going to assume that the last step is a slow step.
So what does it look like?
Well, we have the creation is fast.
So the intermediate is fast.
But the destruction through the reverse process is much faster.
We have at least one step, out of the intermediate, which is faster than the step into the intermediate.
That's all we care about.
It doesn't really matter that this is slow in terms of using the steady state approximation.
We just want the step getting out of the intermediate, a backwards step in this case here, to be faster than the state into the intermediate.
So this allows this, this fact that this is faster, that allows us to use a steady state approximation.
So now we can go ahead and solve the problem.
So we write down all the rates minus dA/dt.
It's k1 A.
Let's start with the rate of the action.
Let's take a product.
Let's take C. Let's say this goes to product C here.
So dC/dt is equal to k2 times A, that's the rate of the reaction.
And we're going to look at this rate of reaction.
And see how it changes.
What it looks like under limiting conditions.
The rate of reaction.
Let's look at the intermediate.
Because this is what we're going to solve first, because this is going to turn out to be an algebraic equation.
Because we're going to apply the steady state approximations.
Intermediate d A star / dt is, we can create it through the forward process.
We destroy it through the backwards process.
A star times M. And we destroy it through the final process.
A star.
We apply the steady state approximation, steady state, steady state.
We solve for A steady state algebraically.
k1 A M over M times k minus 1 plus k2.
And then we plug that back into the rate of the reaction.
The rate of appearance of C. Which is what we're measuring experimentally.
We're looking at the products.
Measuring the appearance of products.
So they're destruction.
This is also equal to the destruction of A.
And so we plug this into the, we write our rate of reaction.
Which we measure, experimentally.
We find this is k1 times k2.
Times, we plug in for, and here, what we do here, we solve A in terms of A steady state.
A is equal to something A steady state.
Well, let's see, A is equal to M k minus one plus k2 times, well this is actually, I have this wrong up here, this is d A star.
So we don't need to do this here.
The appearance of C is, A star here.
So we plug in A steady state here.
A star steady state, and this is A star steady state.
The intermediate is what we're solving a steady state for.
So this is k1 k2 A M over M k minus one plus k2.
We just have the extra k2 that appears when we multiply A steady state star with the k2 there.
And then we can look at the limiting cases.
There are two limiting cases, which we can tell by looking at the denominator.
If we have one term bigger than the other.
Only two choices here.
So the first case is where M k minus one is much bigger than k2.
Experimentally, what does that mean?
It means that this in the gas phase now.
It means that regardless of what these rates are here, we've managed to make the concentration of M high enough so that this becomes true.
There's always an M that's going to be high enough so that the multiplication of these two is going to be bigger than that.
What does that mean?
It means that the pressure is high.
Concentration of M is high.
This is a high pressure case.
And then the other one is M k minus one is much less than k2.
M is very small.
Low concentration, low pressure.
The partial pressure of M is very low.
This could be A. M could be A, or it could be some other molecule that's in the mix.
And by high pressure we mean something on the order of one bar, let's say.
That by low pressure we mean something around 10 to the minus 4 bar.
So now we can go and put these limiting cases in our equation here.
k minus one is much bigger than k2, so we can ignore k2 here.
And then we have something that looks like the M's disappear.
We have something that looks like k1 k2 over k minus one times A is the rate of reaction.
First order.
Looks like it's first order in A.
This is what people had observed.
Great.
The problem is that they didn't know to go to low pressure.
They were doing all their experiments at atmospheric pressure.
They were getting a first order rate.
They thought they had solved the problem.
It's just A falling apart by itself.
Until you go to low pressure, where now this term dominates the denominator.
Or rather, this one dominates the denominator.
And now you have something of the form k1 A times M. This dominates, this term goes away.
The k2 goes away and you have k1 times A times M left over as the rate of the reaction.
Second order.
There are two species here.
A times M. M could be A or it could be some chaperone.
A squared.
Second order process.
So if you're at low pressure, you see something at second order.
You've discovered something about the mechanisms you didn't expect.
And then you get a Nobel Prize.
Alright, next time we'll do chain reactions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to start directly today and look at chain reactions.
And explosions.
So chain reactions, how many people have heard of chain reactions?
Lot of people have heard of chain reactions.
Have seen chain reactions done, in kinetics.
No.
Alright, well, then I'll do chain reactions.
Chain reactions.
Chain reactions are reactions that feed on themselves.
And there are three parts to a chain reaction.
There's an initiation step, which is where your reactant makes an intermediate.
And that's usually a slow step.
There's a propagation step, where the intermediate reacts with your reactant, becomes a feedstock.
And then there's some intermediate plus maybe a first product.
Then there's that second intermediate, can further react to re-form the initial intermediate.
And that's the loop.
That's the loop of the chain reaction.
Plus maybe a second product.
And then there's a termination step.
Which is where when your intermediate, I1, becomes a stable product.
Or I2 becomes a stable product.
Basically, where the intermediates come out of the loop.
So if you look at this loop here, you have your reactant that comes into the loop.
And then go around the loop.
You create I1, and maybe a product comes out.
Maybe I1 gets terminated, but most likely it keeps going around the loop.
Creating I2, which reacts with, the reactant at some point gets included in the circle.
And I1 comes again, and I2, and I1 comes, and I2.
And every time you form I1 or I2, you may form some of the product.
So this is the initial feed.
Then you go in circles.
Eating up R, spitting out products.
Keep going in circles.
And at some point the termination's going to win, and the circle stops, and the chain has stopped.
And there are two kinds of chain reactions.
There's a stable chain and an unstable chain, where you get your explosion.
Stable chain, or also called stationary, chain reaction, is where the amount of these intermediates, I1 and I2, are constant in time.
So you don't build up any intermediate.
So I1 constant in time, which is why it's called stationary.
Then you have the unstable, or non-stationary chain.
Where in that case your intermediate increases in time.
So in the example I gave here, you destroy I1 in the first step, you create it in the second step.
Comes back, you destroy it, you create it, you destroy it you create it.
For every I1 that you start out with, you end up with an I1.
There's no change in the concentration of I1.
But if instead I put a two here, than my reactant creates an intermediate I1.
I destroy it.
Create a new I2, the I2 decomposes, creates two I1's.
So after the first cycle, let's look at the concentration of P2 here, after the first cycle here I have one I1, and make one P2.
But I make two I2's.
So now I have two I1's coming in here.
So I have twice this reaction.
I make two I2's.
These two I2's form two products.
And now four I1's, two times two.
These four I1's go through the process.
I get four P2's.
So third cycle, I get four P2's.
And I get two times four, eight I1's.
These eight I1's go through, and I get eight here.
For cycle, et cetera.
So I end up with something that goes like, so this is two to the zero power.
This is two to the first, two to the second, two to the third, et cetera.
Eventually you get two to the 25 or whatever after so many cycles.
And if these are gas phase products, so if there's an exothermic process where heat gets created at any of these steps here, then you end up with a problem.
The thing goes out of control.
Sometimes that's a good thing.
If that's what you want to do.
And sometimes it's a bad thing.
Depends on what you want to do.
OK, so we're going to show two examples of these chain reactions.
One is stable chain and the other one is unstable chain.
And go through the process of how you solve for them.
So the first example is taking, so let's do a stable chain.
Take acetaldehyde, CH3CHO, that decomposers to form methane and carbon monoxide.
That's the reaction.
And here are the observations.
There are two observations.
The first one is that methane and carbon dioxide are not the only products that are formed.
In fact, there's a trace amount of ethane and hydrogen, dihydrogen are also formed.
That's a clue to something funny is going on here.
That there's a mechanism that's more complicated than just this decomposing in a first order process.
And the other thing is that the rate, if you do an experiment and you measure, somehow you do a half time experiment, or an initial rate experiment, or one of the ways that we talked about early on, you measure the rate of the reaction and you find that it's proportional to something to the 3/2 power.
That's pretty weird.
It's not a simple mechanism.
It's pretty weird.
And so those two observations are clues, usually, that you've got a chain reaction.
The fact that you have a trace amount of extra things, that's usually due to a termination reaction.
It's not a major product, it's a minor product that's the termination of the chain.
And then this funny power here, in the kinetics.
Of the rate of reaction, that's also usually a chain reaction.
So now we have to figure out if there's a mechanism that supports those observations.
And I'm going to tell you possible mechanism that supports those observations.
And you can do experiments to show that you can fish out certain intermediates.
It's not proved that that's the ultimate mechanism, but it's at least consistent with these observations.
More hidden chemistry.
OK, so this is the mechanism that's proposed.
And then we'll figure out whether this proposed mechanism fits the data.
So, the proposed mechanism is that the initiation step consists of the acetaldehyde, CH3CHO, decomposing into a radical.
Chain reactions often have radical processes also.
Because the radicals love to make chains.
To the methyl radical, and then the CHO radical.
So this is the first step.
It's the decomposition step, it's very slow.
That molecule doesn't really want to fall apart.
Then there's a propagation step.
And that's the methyl radical plus the acetaldehyde, your reactant, forming with the system rate k2.
Forming CH3CO radical, plus methane.
So that is their product right here, coming out of the first step of the propagation.
And then this new intermediate here, so there's this one here would be I1.
The methyl radical.
And the CH3CO radical here, this is I2.
And you have to use that up in the second step.
CH3CO.
That decomposes on its own.
Rate k3, to form the first intermediate back, the methyl radical, plus the second product, CO. And then this intermediate then feeds the first step back.
And you've got your cycle.
And every point you spit out your two products.
And this could go on forever.
But it doesn't, because at some point those two methyl radicals, two of those methyl radicals can get along, can collide.
And in a very easy way.
So we've got a few termination steps.
You can form two times CH3 radical ethane, C2H6.
Rate k4, let's call this k4.
And then there are other side reactions that can happen.
For instance, this radical here, which we haven't used up.
That can also react.
It's just sitting around, slowly building up as this thing, very slowly building up, because the first step here is very slow.
You don't need very much of this methyl radical to start the chain.
Then you start chewing up the reactant.
But you're still building up a little bit of this guy here.
This is a side reaction.
It's not a termination reaction, because that radical's not involved in the chain propagation step.
But that radical can react with some other species, M, and it could be any of the other species.
Because it's just a chaperone.
It's just a, to help this thing to decompose.
CO plus H radical plus the chaperone back.
And then that hydrogen radical here can itself react with your reactant.
CH3 acetaldehyde, CH3CHO, with a rate k6.
To form hydrogen gas plus your intermediate, CH3CO.
And this intermediate can feed the step right here.
OK, so in terms of the observations, then, we got the right products.
Methane, and carbon monoxide.
We've got the right amount of trace stuff, the ethane coming out from the termination step.
And hydrogen gas coming out from the side reaction.
And I should probably have written CO also forming from the side reaction - well, CO is a real product, so it's happening both from the side reactions as well as from the main reaction.
So we've got all the species taken into account.
And now we've got to show that this mechanism here, in fact, gives rise to this 3/2 power.
And I know I won't be able to do it on one board.
I want to keep all up, so let me use up.
So the first thing, as we've been doing in kinetics this whole time is, you've got to write down all your rates.
Once you've got that down, and you haven't made a mistake, then you basically have the problem solved.
So you write down all your rate laws.
Eventually, what we're interested in showing is the rate of the reaction, is that power.
So that's write down what the rate of the reaction is.
The rate of reaction is the rate of formation of one of the products.
Let's say the methane.
So we write down that rate.
How was it formed?
It's formed through the k2 step.
It's formed through this step right here.
Which is a second order in CH3 radical times CH3CHO.
acetaldehyde.
We're going to keep that in the back of our mind.
Because we're going to need this again.
Because we want to show, at the end, that in fact this is, this rate is proportional to this funny power.
There's the intermediate sitting here.
So what's the rate for this intermediate here is CH3, dt.
There are many places that it's coming from.
It's being created in the first step, and so the hard part here is bean-counting.
Making sure that you've found all the places where the species is being created.
k1 times acetaldehyde, CH3CHO, and then it's destroyed through the second step, minus k2 CH3, better get your signs right also.
So there's the destruction, there's the formation.
Times acetaldehyde, CH3CHO.
Then it gets created again, through the second step of the propagation, rate k3.
Radical CH3CO.
And then it finally gets destroyed in the termination step.
We've got to put that in, even though it may be a very small amount compared to these other amounts there.
So the termination step is k4.
And it's a second order process, CH3 squared.
We've got another intermediate.
We want to write down all our intermediates here.
Because they're all related, it's the intermediates sitting here.
So there are going to be a couple of differential equations.
So we've got to write down the equation for that CH3CO radical.
And again, you look at your mechanism.
And you find out where it's coming from.
It's being created in this step.
Destroyed in this step.
It's also being created in this step.
But we're going to ignore that side reaction for now.
Just to make our lives a little bit easier.
k2 CH3.
You've got to put the termination step in, but the side reactions are not so important.
CH3.
Because if you don't put the termination step in, then your reaction's just going to go on forever.
That's just not physical.
Minus k3 CH3CO.
Creation and destruction.
OK, so now what do we do?
Well, we know it's a chain reaction.
A stable chain reaction.
Because we don't create more intermediates than we started out with.
The first step is always very slow.
Initiation at this step is slow.
So we're not making very much of this methyl radical.
We're not making very much of it.
And we're not making any more of it here.
So during this whole time that the chain reaction is happening, this methyl radical here is in small quantities.
Small quantities and it's not going to change very much at all.
It's pretty much stationary.
Magic word is stationary.
Stationary means it's not changing.
Not changing, so which approximation does that belong to?
That's a steady state approximation.
We've got a steady state here.
Chain, which is going blah, blah, blah, blah, stationary.
Steady state.
All sounds the same.
So that's the clue here.
This is going to be a steady state approximation problem.
So we're going to put a steady state here.
We're going to put a steady state here.
We're going to set this equal to zero.
We're going to put a steady state right here.
We're going to put a steady state here.
We're going to set this equal to zero.
And suddenly the problem becomes a tractable algebraic problem.
Now we have, to solve this, let's do it in one more place.
Let's put it right here.
So now we have two equations, two algebraic equations.
Two unknowns.
These two intermediates.
The concentration of these two intermediates are the two unknowns.
Two equations, two unknowns, we can solve it.
It can be messy.
But not necessarily tricky.
Just turn the crank.
And computers could do this.
So, let me just write the answer.
Because I don't want to go through the algebra.
After you turn the crank, what you want is to get this out.
You've got your two equations, two unknowns.
You turn the crank.
And you get that form for the concentration of the methyl radical two k4.
CH3CHO turns out to be to the 1/2 power.
When you do that.
There's the funny power coming up.
That's going to be here.
And then this is what you want to put into here.
So, and there's the acetaldehyde, sitting right here.
There's one acetaldehyde to the 1/2 power, times acetaldehyde to the one power, that's the whole thing to a 3/2 power.
So in fact, the rate is proportion to the 3/2, according to our mechanism.
And all these rate constants go into it as well.
And what we find is that the rate of reaction is equal to k2 times square root of k1 over two k4.
CH2CHO to the 3/2 power.
Basically, k prime to an effective rate times that.
It's a typical problem.
Any questions on this one right here?
Yes
: [INAUDIBLE]
Here?
Right here
: [INAUDIBLE]
You know, according to my notes I don't have it, yes, you're absolutely right.
There's a two here.
Thank you very much.
It's arbitrary.
Where the two comes from, I didn't want to get into this.
Sometimes it's a matter of convention.
When you have a second order reaction, A plus B going to C, you write your rate of the reaction, as let's write it here, k times A times B.
Sometimes we have 2A goes to C. You have a rate k2, and the rate of the reaction is, you can write it as dC/dt, which is minus k2 times A squared.
But if you write dA/dt, because of the stoichiometry.
dA/dt concentration for, so you have that C is equal to A0 minus A, twice.
So dC/dt is equal to minus two dA/dt.
So minus 1/2 dC/dt is equal to dA/dt.
So dA/dt is equal to minus 1/2 k2 A squared.
And instead of having the 1/2 here, what we do is we redefine our rate constant.
So that's k2 prime is equal to twice k2.
You put a two here, a prime here.
Get rid of this here.
And there's the two sitting right here.
Purely convention.
Not something to really worry about.
You've just got to have it right at the beginning, and keep that two in, if you're going to put it in there.
Does that make sense?
It's the stoichiometry, it's because of this two sitting right here.
Then you have a choice.
You can either say the rate of the reaction is the rate of formation of this thing.
Or you can say the rate of reaction is the rate of destruction of A.
And then if you say that this is the rate, and you don't want to have any factors in here, then the two shows up here.
If you want to say this is the rate, you don't want any factors in here, then you're going to have a different rate constant.
Purely arbitrary choice.
Good question, though.
Any other questions?
OK.
So, there's another thing about the chain reactions, a bit of nomenclature that's interesting to talk about.
It's the chain length.
And that nomenclature is due to the fact that many of these chain reactions are used for polymer reactions.
Where you use radical polymerization and then you have, basically, in a polymer reaction you'll have an initiator and then you'll add a monomer through a radical process.
And another monomer will add through a radical process.
Another monomer will add, another monomer will add, another monomer will add, and then you're going to terminate.
And then you end up with a chain.
A polymer chain.
So your product, instead of being molecular products, the product is adding a monomer to the chain of the growing polymer.
And then it's meaningful to talk about chain length.
So that's where the nomenclature comes from.
It's a number of cycles of the reaction before you get a termination step.
So we can define the chain length as the number of propagation steps per initiation step.
That gives you the chain.
Each one of these is a propagation step.
And there's the initiation step here.
And this could also be termination step.
In in a stable chain, or termination step.
There's a termination step here.
For every initiation step in a stable chain, you've got a termination step.
Because you're not building up intermediates.
So you create it.
At the end of the chain you terminate it.
You have as many initiation as termination steps.
So you have a choice here, in the algebra.
You can use either one.
So another way, instead of doing numbers here, you can also rate.
This is also the rate of propagation divided by the rate of initiation.
You just take the derivative of this with respect to time of the top and bottom.
Same thing.
So we can also define it as rate of propagation divided by rate of, let's say, termination.
Or initiation, whichever is your favorite one to use to make it simple.
So, for instance, in our example here, the rate of propagation is the rate, also the rate of product formation.
Since we have a product formation.
Rate of propagation is the rate of forming this methane here.
So in our example here, the rate of forming the methane is the rate reaction here.
This is this right here.
So you can plug this here on top.
I may not write it down because I don't want to spend time writing too much down.
So this guy goes right in there.
And then the rate of termination is going to be, the rate of termination is, well, let's not use termination because that has intermediates in it it.
Let's use initiation.
Since we have a choice.
Rate of initiation.
Either one works fine.
Initiation is k1 times the acetaldehyde.
k1 times CH3CHO.
So there's the 3/2 power, which, where is it?
There's 1/2 plus, so 1/2 plus one is the 3/2 power here.
So there's CH3CHO to the 3/2 power.
CH3CHO to the 3/2 power, then some effective rate k prime.
And you end up with chain length going to the 1/2 power.
So there's some k double prime, so effective rate times CH3CHO to the 1/2 power.
That's the length of chain.
It depends on the concentration of the reactant.
And the typical chain's on the order of a couple hundred or something like this, usually.
You always have a termination step at at some point.
Maybe a few thousand if you're doing polymer chemistry and sometimes millions, if you're very good at doing polymer chemistry.
OK, any questions on the stable chain?
OK, let's do the explosion now.
And so this is the typical example.
Of an unstable chain.
It's the formation of water from hydrogen and oxygen.
2 H2O.
That's the reaction.
Gas phase reaction.
And the observation is that that, well, you know the observation.
You put a spark in a balloon that contains hydrogen and oxygen and get an explosion.
So we're going to have to account for that.
Sometimes you don't get an explosion.
Sometimes it kind of peters out.
You have a drop of water coming out.
Not bad.
And there's a temperature dependence to it also.
So, whatever mechanism we put in there needs to account for all the pressure dependence, the stoichiometry dependence, and the temperature dependence.
So this is the mechanism that's proposed.
Initiation.
You have your hydrogen molecule.
Spark, or something slow that decomposes it to two hydrogen radicals.
This could be a spark plug, and every time you make a little bit of the radical.
Then you have a branching step.
Where your radical reacts with the oxygen molecule to form a hydroxyl radical.
Plus an oxygen.
Oxygen atom plus your hydrogen gas, your reactant, reacts to form more hydroxyl radical plus the initial intermediate.
And then the hydroxyl radical, I don't know why I have parentheses here, hydroxyl radical reacts with the reactant again, the hydrogen gas, k3, to form more of the initial intermediate plus water, your product.
So there are three steps to the branching here.
You get one intermediate, one of the initial intermediates here coming in.
Forms a second intermediate.
OH radical.
Two OH radicals.
And the OH radicals form the hydrogen.
There's also radical hydrogen that's formed right here.
So you have this formation.
So because you're forming two hydroxyl radicals here, in this step here you're actually forming the equivalent of two hydrogen radicals.
So, one hydrogen atom here.
After going through the branching, you end up with three hydrogen radicals.
So you go from one to three in one cycle.
And the second time around, you go from three to nine.
The next time, you go from nine to 27.
Pretty soon you've got a couple of billion hydrogen radicals.
And that's the problem right there.
Or the opportunity.
The opportunity to get something that goes out of control.
And there are termination steps.
That we need to take into account.
The hydrogen radical can hit the wall.
Before it finds something to react with.
And gets stuck.
So if the pressure is low enough, you know that's going to happen.
You're going to make a hydrogen radical.
Hardly any gas molecules around.
It's going to find a wall, it's going to get stuck.
It's very reactive.
So that's going to terminate the process.
Hydrogen radical could re-find an oxygen and another chaperone, that's a termolecular process.
That's an unlikely process.
But nevertheless, it could happen.
With another termination step, to form the HO2 radical plus M. And then that HO2 too radical can react in a side reaction to form some products, but not be part of the chain.
Yes
: [INAUDIBLE]
How do we get three hydrogens?
Because there are two hydroxyl radicals here.
And so in this step here, you actually have two hydroxyl radicals to form two, so I could multiply this by two.
But you usually don't do that.
So this already tells you what conditions you're going to get, with something where the termination is fast, compared to the propagation.
Because you have low pressure.
If you can't get that hydrogen radical to react with something before hitting the wall, you're going to be OK. You're not getting an explosion.
This tells you that a very high pressure, where the probability of having a three-body collision becomes high enough, then you're going to terminate the chain as well.
Those hydrogen radicals are going to find oxygen molecules and some other molecule and form this radical before you propagate the chain.
At least some reasonably high pressure.
So that's the clue here.
What's the strategy?
The strategy is to assume that everything's going to be fine.
That is, stable.
That's going to be the strategy.
And then we're going to do a proof by contradiction, basically.
We can assume that the chain is stable.
That we can use the steady state approximation.
We're going to apply the steady state approximation.
We're going to find where that breaks down.
And where it breaks down means that it wasn't right.
That the intermediate concentration was too high.
And that means that we're going to get an explosion.
That's the logic.
So strategy is a proof by contradiction, basically.
Proof by contradiction.
So we're going to assume that the propagation, or the intermediates, are in small quantities.
We're going to assume that we can use the steady state approximation, the d[O]/dt steady state is equal to zero.
That the hydroxyl radical, d[OH]/dt, steady state is equal to zero.
And that d[H]/dt steady state is equal to zero.
These are going to be our assumptions.
That all the intermediates are in small quantities.
And they don't change very much.
And now we're going to write down, you write down the rates that correspond to these guys.
And then you solve.
So I'm only going to do this first one here.
So you write dO/dt, you see where it gets formed.
It gets formed in the first step here.
So it's going to be k1 times H radical times O2, and it gets destroyed in the second step here, minus k2 times O times H2.
You set the steady state approximation, you get a steady state.
Steady state here, you set that equal to zero.
You solve for O, radical O, steady state.
And you find that it's k1 times H times O2 times k2 H2.
Then you do the same thing for the other radicals.
You solve for OH steady state.
And you get some expression which is in the notes.
I'm not going to write it down, it's going to get too messy.
But the expression that I will write down is the final expression, which we're going to be taking into account.
So, this expression here, for the oxygen atom.
It has the radical in there.
So it's not, we'd like something that's just in terms of the rates and either products or reactants.
And this hydroxyl radical, the concentration turns out, it will also be a function of the concentration of the hydrogen radical.
So what we really want, I don't want to cover this up, let me go on this board here.
So when you solve for this, the hydrogen atom is where you get something that's free of intermediates.
And has just rates in it.
There's a termination rate here going in the denominator.
Another termination rate in the denominator.
Times O2 times M, where M could be anything, really.
Minus, and there's the creation rate of the first propagation step.
O2 here.
So now we have to see whether or not our approximation was valid.
We made an approximation.
We said the steady state approximation was valid.
If it is valid, then this concentration, this pressure here, needs to be small.
Needs to be small at all times.
If it's not small, then steady state is not valid, we have a branching non-stable chain and we've got an explosion.
So let's look at the different possibilities here.
What are the possibilities?
Well, the first possibility is that as we saw intuitively, when we wrote down this equation here, the first possibility is that we're at low pressure.
Where at low pressure, then, k1 times oxygen is very small.
Because the concentration of oxygen is small, the pressure is small.
k5 times the pressure of oxygen times the pressure of M is small also.
So k5 T O M, O2 times M is small.
And they're both, then, much smaller than k4 times the time.
k4 termination.
The hydrogen radical hits the wall before finding anything else.
End of chain.
So in the equation here, these two guys are small compared to here.
RI, the initiation rate, is very small.
You control that by your spark.
Once a second, once a minute.
Could be really, slow.
Steady state is good.
We don't have an unstable chain.
No explosion here.
Medium pressure.
You've got a medium pressure where two k1 times oxygen concentration is approximately.
You're building up your concentration.
This is getting bigger here.
This is a constant.
This is getting bigger.
This is also getting bigger, but you've got this rate here and this rate here.
At some point, at some point, this rate here, two k1 times oxygen concentration, is going to be on the order of k4 T plus k5 T times oxygen times M. At some point it's going to be like that.
And then you're in big trouble.
You're in big trouble, because you've got a denominator here that's really tiny.
Could be even, if you hit it right on the nail, you get zero in the denominator and then you're in real trouble from the math perspective.
But from the physical science perspective, it's not being in trouble.
Is just that your approximation is wrong.
The approximation that you were in a stationary state was wrong.
And so that means that in this case here, if you want to have an explosion, you've got it.
So you adjust everything until you're in a situation.
And then finally, you keep increasing the pressure.
And then if you increase it high enough, then this guy here, which goes as the square of the pressures, is going to overwhelm this.
Which goes linearly with pressure.
And k5, the termination step, O2 times M, is going to be bigger than two k1 times the pressure of O2.
And then you've got something small again.
This is where that's hydrogen radical here is able to undergo a termolecular process before finding a reactant.
Then steady state is valid.
OK. At very high pressures, you get in trouble again, because this hydroxyl radical, this HO2 radical, so at very high pressure.
then what happens is that you have this side reaction, HO2 radical plus hydrogen goes to H2O plus hydroxyl radical.
So there's a probability that this radical here finds a hydrogen molecule.
And then feeds into the propagation through this hydroxyl radical that's the result.
And then that keeps the chain propagating.
So at very high pressure, you're in trouble again.
And you get an explosion again.
So you can plot this as a function of the pressure.
And we're going to put pressure, actually, on the y axis here.
Let's put pressure here.
Put temperature here.
And so for a given temperature, let's call it temperature T prime, at low pressure there's no explosion.
Then there's a range of pressures.
Where there's an explosion.
Medium pressures.
And then a high pressure termination wins again.
And then if it's really high you get explosion again.
You do this for, as a function of temperature.
And what you find that is a diagram, looks like a phase diagram, that looks something like this.
Where at high temperature, you get an explosion at low temperatures, and this steady state is valid.
OK, any questions on this?
Don't try this in your kitchen.
Go to the desert if you want to do this.
Or don't do it either.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, let me start here with the temperature dependence of k. And this turns out to be extremely important.
And it's due to Arrhenius in 1889.
And Mr. Arrhenius is famous for many reasons.
Not just for his rate law.
So I learned recently, turns out that he was also one of the first people to calculate the potential effect of carbon dioxide in the atmosphere.
And and its potential effect on global warming.
This was the Industrial Revolution and people were starting to use fossil fuels at increasing amounts.
And back then, already, some people started to get worried.
And so he did the calculation.
It was a very crude calculation.
And he extrapolated, assuming that the rate to fossil fuel use would keep going, he got his calculation pretty much right.
What he didn't get right was the amount of fossil fuel that people would be using.
And so when he did his calculation, which is again a very crude calculation, he got that we would get in trouble at about 2,000 years from the time of his calculation.
So he said, no problem, 2 years, we've got a lot of time.
And ever since then people have redone these calculations and more and more sophisticated.
But his crude calculation was good enough.
And as people redo the calculations, the time that they say we're in trouble to the time of the calculation gets closer and closer together.
And the reason for that isn't that their calculations are wrong, it's that the rate at which we put out carbon dioxide just keeps getting faster and faster and faster.
So it's interesting to go back to these calculations through the last century and a half and see that.
Anyway, so Mr. Arrhenius predicted global warming.
And he wrote his rate equation, k is equal to e to the minus Ea over RT.
Famous Arrhenius rate equation.
If you plot it as log of k is equal to log A minus Ea over RT, you find that it looks like a straight line.
With an intercept at log A, where A is the pre-factor here and Ea is going to be the activation energy.
And the slope is Ea, minus Ea over RT.
R is the gas constant.
T is the temperature.
And if you put in some typical numbers, activation energies typically are on the order of a few tens to hundreds of kilojoules per mole.
Let's say 50 to 300 kilojoules per mole.
And typical activation factors, typical pre-exponential factors, are depending if it's a first order or second order.
Let's say, first order 10 to the 12th, 10 to the 15th, per second, so the A carries the units for the rate here.
So if you have A, so this is for first order, if you have a second order, then the units of A will be different.
And typically they're going to be on the order of 10 to the 11th or so, 10 to the 10th, 10 to the 12th, one over molar per second, for second order.
And it's interesting to do sort of a back of the envelope example to figure out how important this rate is.
And one of the interesting things to look at is, suppose that I'm at some temperature and I want to know how do I change the temperature.
How much do I need to change the temperature to double the rate.
And let's say my temperature's at, say, T1 is on the order of 300 degrees Kelvin, room temperature.
Room temperature, and let's take a typical Ea on the order of 100 kilojoules per mole.
Room temperature, or body temperature are roughly the same, right?
These things are.
And so the question is, what does T2 need to be to double the rate connected with an activation energy which is fairly typical of 100 kilojoules per mole.
So we want to know, what is, if I have k2 over k1 is equal to two, what's the new T2.
Alright so A e to the minus Ea over R T2 divided by A e to the minus Ea over R T1, we want that equal to two, the A's cancel out.
So log k2 over k1, which is equal to log two, is equal to Ea over R, one over T1 minus one over T2.
We know what this is, we know what this is, we solve for T2 and we find that T2 is 305 degrees kelvin.
It's a pretty small change to double the rate.
So it's pretty important that your body temperature doesn't change very much.
If you have a fever and your temperature goes up, rates start going up.
In your body, you could cause a lot of problems if the rates of some reactions go up by a factor of two.
So, very sensitive to temperature.
Rates are very sensitive to temperature.
What are these things physically?
Ea and and the pre-exponent factor.
Let's take a look at that.
So physically what's going on is, you have your two, let's say it's a bimolecular reaction, A plus B goes to C. So mechanistically what we think is happening is that the molecules come together and collide, right?
You have A and B getting together and colliding.
And the hypothesis is that when they collide they form a complex.
They sort of bind together momentarily.
And form this complex that has all the kinetic energy, or part of the kinetic energy of this collision as part of it.
So A and B are bound together in some highly excited complex.
Which then falls apart to give the product by rearranging its atoms around.
And if you plot the energy of this process as a function of the reaction, we call this the reaction coordinate, then we're going to start at some delta G, or some energy.
For the reactants.
We're going to end up with some energy for the products.
And it could be higher or lower.
Let me make them higher here, just because I don't have enough room on the board.
Products.
So if this is an endothermic reaction, that's the energy for C, this is the energy for A plus B, and then along the way we have this complex that captures some of the energy of the collision.
Up here somewhere.
And this is the energy, then, at A B star, which we call the activated complex.
So for the reaction to happen, then, we have to go over the barrier and then back to the product.
And this distance from the energy of the reactants to the top of the barrier, that's Ea.
That's the energy of activation.
How much extra energy you have to put in there to go over the barrier.
So it's very clear that as you increase the temperature and you increase the amount of energy that the reactants thermally have, or in terms of their kinetic energy, as you raise the temperature you get more and more kinetic energy, you're going up higher and higher in the Boltzmann distribution.
And the number of reactants that can make it over the barrier clearly goes up exponentially.
As the temperature goes up.
So this is Ea for the forward reaction.
Clearly there's going to be an equivalent activation energy for the backward reaction.
The two are related.
The difference between the forward and backward reactions, so E backwards minus Ea forward gives you the delta G for the reaction itself.
So, Ea backwards minus Ea forwards gives you delta E for the reaction.
Where E could be enthalpy or it could be free energy.
So that's the physical origin of this Ea, this activation energy.
Now, A, the pre-exponential factor, that's the rate of attempt for the molecules to try to go over the barrier.
So it has units that make sense for that.
And this A is different than this molecule here.
Rate of attempt.
How many times per unit time, or how many times per second, do these two molecules try to collide?
They never collide, they never try, they'll never make it over.
So it's one over second, or one over mole second.
So this is the rate of attempt.
Then e to the minus Ea over RT is the probability of success, of making it over the barrier.
So the rate of attempt times the probability of success gives you the rate per unit time of making it over the barrier.
Any questions on the dissection of this Arrhenius rate law?
Alright, so here's some examples, a few examples that you know of.
Let's look at OH minus plus methyl bromide.
Displacement reaction going to this OH minus attacks the carbon right here.
To form an activated complex H, H, H, with the OH coming in like this.
Let me go like this.
OK, there's the intermediate right here.
The activated complex in your reaction.
Which then falls apart.
It can fall apart two ways.
The OH can be spit out again, forming back to the reactants, or the bromine can be spit out, forming the product.
Which in this case here is methanol.
H, H, H, OH, plus Br minus.
Typical example.
So, given this picture here, which you're probably already somewhat familiar with, we can then move on to talk about how to affect this barrier.
How to change the rate.
Oh, I should say a couple more things.
One more thing, too.
Now, the rate of, so clearly the Ea's are related.
Because the difference of the two Ea's is the delta E for the reaction.
But the pre-exponential factors, these A's, for the forward reaction and the backward reactions aren't necessarily related.
You can imagine that in one case you have, first of all they don't even have to have the same units.
You could have a bimolecular reaction one way.
And a unimolecular reaction the other way.
So you can't really say anything about the forward versus the backward rate this way.
All you know is about the energies.
OK, catalysis.
So now we have this barrier we have to overcome.
And, so suppose that you have two motivators.
Suppose you have an equilibrium case which is slow.
And a reaction which is slow in both directions, k1, k minus one.
And you don't want to wait for this to happen.
And this could be in a biological environment.
Or it could be an industrial process, like the Haber process.
You don't want to wait.
And so one of the ways that you can speed it up, you know is by changing the temperature.
You change the temperature, the rate goes up.
You change it by five degrees, the rate goes up by a factor of two.
You can make a huge change by changing the temperature.
But, so if you raise T, speeds up.
But, K equilibrium also changes.
And we saw that when we did the Haber process.
We raised the temperature.
The rate speeds up, but the equilibrium switches to the reactants.
That's no good.
So changing the temperature is not always the best thing to do if you want to change the rate.
Instead, what you can do is use a catalyst.
if you find one.
A molecule that reacts with your reactant to form a product, that's B, spitting back that molecule C again without using it up.
And now with the activation energy, we can understand what the catalyst does.
What the catalyst does is speeds up those rates, k2 and k minus one, by changing the activation energy.
By making the E smaller.
So now I have a catalyst, and I can make this energy smaller.
So this would be A plus A C activated, getting ready to spit out B.
So in my example here where I have A plus B goes to products, so I would have A, some combination of A, B, and C together, to give out the products.
In this case, the difference in energies doesn't change.
All that you're changing is that the hump, in both ways.
Equilibrium constant doesn't change.
Just the rate changes through the Arrhenius rate law.
And this is extremely powerful.
Especially in biology.
So let me give you some examples here.
This is sort of a typical example of increasing rates.
Let's say you have the reaction, hydrogen peroxide, goes to water plus oxygen.
If you take a little bit of hydrogen peroxide and you put it on your skin, it starts to bubble.
But if you let it sit on the bottle nothing happens to it.
Put it on your hair, your hair turns white.
Blond.
But again, if you just let it sit in the bottle very little happens.
Well, it happens but very, very slowly over time.
So if you look at the rate of this reaction here, if the rate, moles per second, with no catalyst at all, the rate is 10 to the minus 8 molar per second.
Which is very slow.
And the activation energy in kilojoules per mole, in this case here is 71 kilojoules per mole.
Now we can start adding catalysts.
We can start adding inorganic catalysts like hydrogen bromide.
Increase the rate at 10 to the minus 4.
So this creates a complex with the hydrogen peroxide, which lowers the barrier to 50 kilojoules per mole, a small amount of lowering.
But because the energy is in the exponent there, it makes a big change in the rate.
Yes
[INAUDIBLE]
The rate is independent of the catalyst concentration
[INAUDIBLE]
The rate would have to change.
You're right.
That's a good question and I'm not prepared to answer it.
I'm going to have to think about this.
Let's pretend now that we are at per unit concentration of the catalyst.
And then, and I'll look into it.
OK, so this is what happens with an inorganic catalyst.
And instead if you use a generic biological catalyst, an enzyme catalase, it's a sort ubiquitous enzyme which is why you have it on your skin, et cetera, then this rate become 10 to the 7th.
And the activation energy drops to eight kilojoules per mole.
So your question really has to do with the units of A.
Of the pre-factor right there.
OK, and so there are all these examples of reactions.
That are very important biologically.
Where with an inorganic catalyst, an organic catalyst, you can change the rate by maybe a few orders of magnitude.
But as soon as you put in an enzyme, the rate changes by ten orders of magnitude.
Or in this case eight plus seven, fifteen orders of magnitude.
Humongous change in the rate.
So you've probably done some enzyme catalysis before.
But it's probably a good idea to quickly do it again.
Because it's just so important.
And it ties together our approximations that we've learned about.
So enzyme catalysis, so enzymes can be either, could be heterogeneous catalysis, can be homogeneous catalysis.
Enzymes are ubiquitous in the biological environment.
They serve to regulate the cellular activity in very complicated ways.
The cell will up-regulate or down-regulate the concentration of enzymes as it needs to make more or less products.
And there's whole cascades of events that happen in this way.
And they're in very small concentrations.
But they play an extremely important role.
Because they're also extremely specific.
So you can have an enzyme that will only act on one part of the biological cycle.
And affect it by 10 orders of magnitude or 15 orders of magnitude.
But not affect any other protein that's around.
And that's amazing.
And it turns out that a lot of the diseases of old age like, what I'm about to face, or am facing already, you, not yet, but, are the result of some of these enzyme up-regulation, down-regulation, beginning to break down.
So we have all these reactions going on that need to be essentially perfect.
When you have DNA replication or protein folding, or things like this.
And errors are made.
Errors are made all the time in these processes.
And errors cause diseases.
And so, even as a baby, your biological processes make errors.
But you have these processes, these enzymes especially based on enzymes, that can go in there and sort of fix things.
Fix things and make sure that you don't end up getting Alzheimer's at age six months.
But as we get older, for some reason, these repair processes lose their bearings.
Just like we lose our bearings.
And and they can't repair things any more.
They don't do it very well.
And that causes all sorts of diseases.
Cancer is probably one of the diseases, due to the lack of being able to repair problems.
Alzheimer's.
All sorts of dementias.
MS. Just name a chronic disease and it's probably due to a problem with up-regulation or down-regulation of some proteins, some enzymes, that are due to a repair process.
So anyway, these enzymes are big proteins.
10 to the 4th, 10 to the 6th molecular weight proteins.
On that order or so.
They tend to be fairly large in size.
On the order of nanometers.
Let's say ten nanometers to 100 nanometers.
That could be, that's a little bit on the big side.
Probably closer to ten nanometers.
Ten nanometers is kind of big, for any sort of biological molecule.
And they always end with their name ase.
Like catalase, uriase, rnase.
Esterase, clips ester bonds.
Your liver is full of esterases.
Because it likes to break things down into smaller and smaller pieces and lots of ester bonds and things that are not very biologically interesting.
And that's one way of the liver breaking things down.
So the way it works is that you have your reactants, which in the biological language are called your substrate, come in, into an enzyme.
Gets bound up in a pocket of, there's the substrate, reactant.
There's the enzyme here.
Forms a complex.
And then product gets spit out.
And the product floats off.
And does its thing.
It probably gets bound to another enzyme, which makes another product.
Et cetera, and the cascade goes on.
The signaling cascade goes on.
And this enzyme is in very small concentration.
The product goes away, so that's going to stay as a very small concentration as well.
So let's observe experimentally, then.
Experimentally, what's seen is that the substrate makes products in the presence of the enzyme.
With a rate dP/dt, that's, at t equals zero this is the initial rate is proportional to the concentration of the enzyme.
This is initial rate.
And in the language of enzymatic kinetics, this is called the velocity.
dP/dt is also called the velocity.
Velocity, moles per unit time.
And this would be called, then, the initial velocity of v initial.
v initial's proportional to the enzyme concentration.
And what else is seen?
For fixed, if I fix my concentration of enzyme, I look at the velocity over time, let's say minus dS/dt, which is dP/dt, I find that that's proportional to S. For small S. Small concentration of substrate.
And then at large concentration, it's a constant.
So it's not a straight line.
In fact, I don't want to do it on this board here.
I'll do it on this board here.
Plot the concentration of substrate on this axis, and I plot the velocity, or the rate, of the reaction on that axis here.
What I find is that it's a constant.
So it's going to saturate, the rate is going to saturate to some number.
And it's going to start linear with substrate at the beginning.
So it's going to be a straight line at the beginning.
And eventually it will saturate.
That sort of slope.
And the saturation point, that's the maximum velocity, or maximum rate that it can have.
So we call this v max.
And this part here, the velocity is proportional to S. And somehow we have to find, explain this.
Using what we know from kinetics.
So we have to come up with a mechanism, and then solve the mechanism.
And make sure that it reproduces the data.
So we're not the first ones to do this, obviously.
Michaelis and Menton did this many years ago.
For this mechanism.
And the idea is, you have the enzyme plus a substrate react, k1, k minus one, to form a complex.
Enzyme substrate complex.
But unlike what we've drawn before in terms of this hump, where there's an activated complex which is not stable, in this case here this enzyme substrate combination lasts a long enough time that it's basically a stable complex.
So we're going to write this as a real intermediate that lasts long enough for you to be able to fish it out.
And characterize it.
And then eventually, that, k2, k minus two, goes to product plus enzyme.
So the enzyme is a catalyst that forms a long-lived complex.
And if you were to draw this, then, in our energy diagram, where we have the reaction coordinate here, you start out with your enzyme plus substrate.
Go up, and you form your intermediate up here.
ES.
And I put a little dimple in there, because it's stable enough that it's not on the top of the hump, but it lives long enough.
And it comes back down to form the product.
Plus the enzyme.
Without the enzyme in there, this hump would be way, way up there.
Would be maybe a factor of ten higher.
So this really lowers it a lot.
Now we can start to solve this mechanism.
And I forgot one arrow.
There which is the arrow going back.
Which you usually don't see, but we might as well keep it there.
For the sake of completeness.
So what do we know?
We know that this intermediate concentration is very small.
The enzyme concentration itself is very small.
Intermediate is very small.
And it doesn't change very much.
Not changing much.
So that means that we need to use a steady state approximation.
So let's write down the rate for this intermediate, d[ES]/dt.
It gets formed through the forward process.
I'm going to put my brackets back in because E and ES would look the same otherwise.
Gets destroyed.
Through the backward ways.
And I'm going to use a steady state approximation.
So I'm already going to start adding steady state here every time I see an intermediate.
Get it destroyed to make products.
It gets recreated through the backwards reaction from the products.
And I'm going to set that equal to zero for the steady state approximation.
Now, we don't really want to have this E floating around here.
Because this is something that is very hard to measure.
It's much easier to measure the initial concentration of the enzyme.
What you put in there, instead of the amount that's not being bound up.
So we're going to solve, instead, in terms of [E] is equal to [E]0 minus [ES], where this is the initial concentration and this is the amount that's binding substrate.
And this is the amount of free enzyme here then.
And when you do that, and you plug in here, and you plug in here, you get your result.
Which is that [E] steady state, [ES] steady state, is this ratio.
k1 times [S] plus k minus two times the product divided by k minus one plus k2 plus k1 times [S] plus k minus two times the product, times proportional to the initial concentration of enzyme.
And then you can take your steady state approximation for the intermediate and plug it back into your rate equation.
So the velocity, we defined as the rate which is minus d[S]/dt, which is k1 times [E] times [S], this is the destruction of the substrate minus k minus one, times [ES] steady state, the backwards process.
So we put in, instead of this [E] we put in [E] is equal to [E]0 minus [ES] steady state.
And then we put in for [ES] steady state, we put in what we found here.
We turn the crank on the algebra.
And we find that the velocity, then, is k1 k2 times the substrate concentration.
Minus k minus one times k minus two times the product concentration.
The whole thing times the initial enzyme concentration.
And then k minus one plus k2 on the bottom.
Plus k1 [S] plus k minus two times the product.
So, as I mentioned, this product here usually just floats away.
And so locally, where you're doing the, and then it gets used by the next step in the cycle.
So this is pretty much zero.
We can pretty much ignore this.
We can ignore this guy here too.
Because it's not, you don't have any steady state or an equilibrium.
You have a steady state but not an equilibrium situation.
And so in that case here, you can rewrite this then as k1 k2 times the substrate times [E]0 divided by k minus one plus k2 plus k1 times the substrate.
And now we can look at these experimental observations and see whether they match our mechanism.
If we can understand something.
So let's look at the initial rate.
Initial rate is supposed to be proportional to the substrate.
Initial rate is supposed to be proportional to the substrate.
So the initial rate, k1 k2 plus k1, if at early times, plus one.
So the initial rate is going to be, somehow I've got the product missing here.
No, I got it backwards here.
This is not the initial rate.
This is the rate where [S] is small.
The initial rate is when you don't have any products made.
Or when the product concentration is very low.
And because we're making the assumption that the product concentration is basically equal to zero here, this is basically the initial rate here.
So by taking the product equal to zero here, we're also saying that this is the same thing as the initial rate.
So this is what it looks like here..
The initial rate here.
And this is, this, you rewrite as v initial is equal to k2 times [S] times [E]0.
You divide by k1 up and down.
And you have this k minus one plus k2 over k1 and then plus [S] sitting down there.
And you define this ratio of rates, k minus one plus k2 over k1 as the KM, the Michaelis constant, by definition.
And this is an interesting ratio.
This is the rate, k minus one is the rate of destroying the enzyme substrate complex by going back to the reactants.
k2 is the destruction of the complex by going to the product.
And k1's the creation of the complex.
So this is the rate of destruction of the complex divided by the rate of creation of the complex.
So if the rate of destruction of the complex is much faster than the rate of creation, meaning that this is a large number, then you're not going to pile up any complex.
It's going to be destroyed as soon as you create it.
So if KM is large, then the concentration, [ES], is going to be very small.
Compared to [E]0.
But if the rate of destruction of the complex is small compared to the rate of creation, you create complexes, you create complexes but you don't keep up in terms of destroying them, in terms of making products, or going back to reactants.
And so you end up saturating your enzyme.
Every enzyme ends up having substrate bound to it.
So when KM is very small, then [ES] goes to saturation.
Basically, when KM is very small, then you're limited by the rate, the second rate in the process, of the enzyme falling apart.
To form the product.
You have to wait until that happens.
Because then the product just floats away.
And that becomes your rate limiting step.
Another way that you also will see this written is as this, then, is equal to, we'll define k cat is equal to k2.
k cat times the enzyme concentration times the substrate, [E]0, divided by KM plus but the substrate concentration.
So let's look at a few limiting cases then.
First limiting case is, suppose, that [S] is large.
Let's take [S] to be much larger than KM.
Because you look at the denominator, and you see it's this ratio of rates but there's this concentration that's important here.
So in one case KM is going to dominate, and in the other limiting case the substrate concentration is going to dominate.
So let's say that the substrate concentration dominates.
Meaning K sub M is small.
In which case we already saw.
If K sub M is small then you reach saturation.
And in the equation, then, the velocity, KM is small, [S] is large, the [S]'s cancel out and the velocity is equal to k cat times the initial substrate concentration.
Both of these are constants.
The velocity is constant.
And the rate is constant, it's that limit up here.
That's experimentally seen.
And the rate is depending on the initial substrate concentration.
Initial enzyme concentration.
All the enzymes have a substrate in there.
The more enzymes you have to begin with, the more intermediates you're going to have, the faster you going to make products.
And then it's going to depend on this on the rate, k cat, which is just k2, which is the rate of formation of products.
That becomes the rate limiting step here.
The other special case is if substrate concentration is very small compared to KM.
And in that case here, and the other thing that we're going to do is, we're going to, because we now understand this as this maximum rate up there, we're going to call this v max.
k cat times [E]0, we're going to call it v max.
And so when [S] is very small, v, then, is equal to v max, which is k cat times [E]0.
This is v max here now.
So we can rewrite this as v max times substrate divided by KM plus the substrate concentration.
Another way of writing it.
Capital K. v max times the substrate concentration.
And we have KM plus [S].
But [S] is very small.
So we drop it.
And this is then proportional to the substrate concentration.
And then that's small substrate concentration compared to KM, we're sitting right here.
Where it's linear, where the rate is linear.
And there's a third place on the graph which is interesting.
Which is when the substrate concentration is equal to KM.
In that case there, you plug [S] equal to KM, and you end up with the velocity then is equal to v max divided by two.
So when [S] is equal to KM, you are halfway up.
There's v max over two and there's KM sitting here.
When [S] is equal to KM, you're at v max over two.
And so enzymes, then, are labeled by their KM's.
Because then it becomes very important to know how strongly they bind the substrate.
Sometimes you want the enzyme to bind it very strongly.
Sometimes you don't, you want it to be fleeting.
Depends on the role that the enzyme plays.
Now there's a way to plot this that extracts out these important numbers.
k cat and KM.
And that's the Lineweaver-Burk plot.. Lineweaver-Burk plot.. And I just looked up this morning to see if Mr. Lineweaver was still alive.
And as far as I can tell he's still alive.
He was 97, in 2003.
So as of 2007 he was still alive.
He's getting up there.
One of the most cited papers that you have in your notes is the in Jack's, was the paper that showed how to go from this curved line to a straight line by plotting one over v versus [S], instead of v versus [S].
So if you take your equation and massage it, one over v KM over v max times [S], plus one over v max, we haven't done anything except rewrite the equation in terms of one of v versus one over [S].
So it becomes linear.
In one over [S], and there's one over v sitting here.
And you get a straight line.
With an intercept here that's one over v max.
And if you keep going, extrapolate out, you get this point here to be minus 1 over KM and the slope is KM over v max.
And v max was equal to k cat times [E]0, so you get k cat out of this.
So it turned out to be a very useful plot.
It's very easy to plot a straight line, especially before computers.
In the age of computers.
And the referees, there were six referees that got this paper and pretty much turned it down because they didn't think there was any new chemistry in it.
Which is true, there's no new chemistry.
It's just a way of rewriting the plot.
But it was very important nevertheless.
OK, any questions on catalysis?
Enzymes?
Arrhenius?
Alright, next time we'll oscillating reactions and recap the course.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time, there was a question that was asked about the order of magnitude rates.
You asked the question about the concentration of the catalysts.
So the rates that I gave you, which were the rates of reactions are basically order of magnitude.
They're estimates and odd numbers that, they obviously depend, as you pointed out, depend on the concentration of the catalyst.
Depends on the concentration of the reactants.
Depends on where you are on those concentrations.
As we saw when we did enzyme catalysis, is that depending on KM, the Michaelis constant, and the substrate concentration, if your substrate's concentration is much higher than KM, you are in the maximum velocity limit, where the rate depends on k2 times the initial concentration of enzyme.
In that case the rate of the reaction depends just on the concentration of enzyme, and not on the concentration of substrate.
But if you are at the small concentration of substrate relative to KM, then your rate depends linearly with the concentration of substrate.
And the same thing happens with the non-enzymatic catalyst, that there will be some sort of mechanism that goes along with it.
And most likely it's going to be second order in the catalyst.
And the reactant if it's just the two body process.
But it could be more complicated.
So definitely those concentrations will go in there.
But a priori, it's not clear how they will go in there.
But the orders of magnitude are roughly right.
And the basic idea is that you change your rate of reactions by 15, 18 orders of magnitude with the right catalyst, especially if it's a biological catalyst.
One concept that I didn't go through last time, which is also quite useful, it's called the turnover number.
Turnover number.
And this is useful for any kind of catalyst.
But for enzymatic, or enzyme catalysts, the turnover number is k cat.
The number per second.
It's a rate per second.
How many times does a reactant get, the number of times that a reactant turns into a product per second, per catalyst.
So let me write it out.
So it's the number of product formed per second, per second, per enzyme, per molecule of enzyme.
OK, so if one enzyme is going to be there, there's going to be some substrate going in.
Coming in, the pocket coming out.
Coming in, the pocket coming out, so the number of cycles of this process per second.
is the turnover number.
And so if you are where the concentration of substrate is very much larger than the Michaelis constant, where are you are at the saturation limit, then this turnover number doesn't depend on the substrate concentration.
It's just k cat.
So that's why k cat is important.
As this turnover number in the maximum velocity limit.
Because in the maximum velocity limit, then the process that's dominant is the second process, ES goes to enzyme plus product.
The first process here is not the rate limiting step.
This is the rate limiting step.
Because the substrate concentration is very high.
So this first part goes very high, goes very fast.
And the number of product formed per second, per molecule of enzyme, the number of molecules formed for second, that's the rate of product formation.
Rate of product formation per second.
Divided by the concentration of the enzyme.
If you normalize by the number of enzyme.
So this becomes rate of product formation in moles per second, divided by moles of enzyme.
It's the same thing as what we're saying here in words.
And so the rate of product formation is dP/dt, and you divide that by the concentration of enzyme, which you can write as E0.
dp/dt, and the maximum rate limit is k max.
It's a constant divided by E0.
And k max is k cat times E0 divided by E0.
The E0's come out.
And this is k cat.
Which is basically k2.
So that's the turnover number.
Any questions on this turnover number?
You see that a lot in catalyst literature and especially the enzymes.
I have to get out these lecture notes.
Send these back.
OK, so the last topic I want to talk about is oscillating reactions.
It's sort of like an interesting topic.
And it's applicable not just to just chemistry, but it's basically playing with the differential equations.
And you'll see this sort of stuff later if you keep doing science that involves coupled differential equations.
It's just all over the place.
So normally we have an equilibrium process.
A goes to B, B goes back to A.
And if you start out with something which is out of equilibrium, then you will eventually reach equilibrium at a rate which is dependent on those two rates.
k1 plus k minus one.
So B will to come up to some B equilibrium.
And A will, if you start out with a lot of A, and a little bit of B, and concentration of A, will come down smoothly, monotonically, to the equilibrium state.
But sometimes, if you're out of equilibrium, and this doesn't have to be chemistry.
It could be anything that's out of equilibrium, your emotion's out of equilibrium.
Stock market is out of equilibrium.
Population dynamics are out of equilibrium.
The weather.
Whatever.
So, you're out of equilibrium, let's say this is equilibrium for A, this is equilibrium for B, let me try to keep them somewhat the same as here, I messed up.
You're out of equilibrium, you start down here for A.
And you, instead of going linearly down or monotonically down, exponentially down to the equilibrium, instead you overshoot.
And you keep going back and forth.
Like a spring.
And then B does the same thing.
Overshoots, comes down.
Overshoots, comes down.
Like a pendulum.
A lot of things work like that.
The heart is something that works like that, right?
It beats.
It's not in equilibrium.
Good thing it's not in equilibrium.
So the heart is one example of something that oscillates, back and forth, back and forth.
And so there are actually many examples of complicated processes that involve chemistry, that aren't at equilibrium, that instead go back and forth, in and out, of on one side or the other of equilibrium.
Now, there's an important feature of, if you want to build a chemical process that looks like this.
It's often very useful to have a particular step in the mechanism that's called an autocatalysis step.
And that provides feedback.
It's like, if my microphone here, the speaker which is here, I were to stand right in front of the speaker and my microphone would pick up the volume from the speaker, it would get amplified, and I get feedback.
And that would be a bad thing.
So this is a form of feedback here.
You need some sort of feedback that the reaction knows that you're building up, that this is going too fast.
And then there's a feedback process that brings it back up.
And a feedback process that goes back and forth.
And the way you get this feedback is by having a step in the mechanism that has both a molecule as part of the reactants and as part of the product.
It's not the same as a chain reaction, where we talked about an intermediate being recycled and building up.
This is an actual reactant and an actual product that's part of this step here.
So this is an autocatalytic step.
And let's just see what that step looks like.
Suppose we just have that step.
So we want to find out, as a function of time, if you start out with some A and B, what happens to the concentration of B as a function of time.
The time dependence of that.
Clearly it's going to build up.
But how is it going to build up?
So we need to solve for B as a function of time.
So let's go ahead and write down our rates.
The rate of the reaction is d[A]/dt, is, let's put a rate constant here.
k, k times [A] times [B].
Now, [B] and [A] are related to each other through stoichiometry.
The concentration of B is the concentration of B that he started out with, [B]0, then every time you destroy an A, you create 2B.
So it's plus two times the amount of A that you've destroyed, [A]0 minus [A], [A]0 is what you started out with.
This is what's left over.
So the difference is what you've destroyed.
But every time you destroy an A to form 2B, you also destroy a b.
So you have to subtract away the B that you've destroyed.
Which is [A]0 minus [A].
This is the 2B that you created by destroying A, and this is the B that you destroy by having to destroy A, for the reaction.
You've got to bean-count correctly.
Keep track of everything.
So [B], then, is just, I'm going to drop those brackets.
B0 plus A0 minus A.
So you can plug that in here.
And is equal to k times A, times B0 plus B0 minus A.
And now you have a differential equation that only contains A and time.
B0 is a constant.
And the way you solve that is by partial fractions.
Use partial fractions, solve for this.
And I'm not going to go through the math of solving for it.
It's not that interesting.
Let me give you the answer.
You end up with [B] as a function of time, is this function A0 plus B0 over one plus A0 over B0 times e to the minus k times A0 plus B0 plus time.
And that's where the time component comes in.
Whoops, I'm sorry.
Usually I don't have this on, let's turn this off.
That's where the time dependence comes in.
And clearly, at t is equal to infinity, t equals infinity just goes to zero.
So you have B as A0, plus B0, everything goes through to the product.
t equals to zero, this is equal to one.
And B is equal to just B0.
Alright, and so the curve looks something like this.
This is what it turns out to look like.
We're starting with B0 here.
There's an induction period where very little happens.
It's like the dormant stage.
So, locusts that are sitting in the ground for seventeen years waiting for something to happen.
Dormant stage, the induction period.
And suddenly something starts to happen.
There's an inflection point.
And you get to A0 plus B0.
The locusts wake up.
And go to maximum concentration.
So there's an s-like shape that has an inflection point and an induction period.
That's pretty typical of an auto-catalytic reaction.
OK so let's apply it to a process now.
And again, this is going to be coupled differential equations.
Broadly, broadly applicable, the example that I'm going to give you.
This mechanism has been used for all sorts of things.
So what do we have here?
We're going to have an island.
In the ocean.
Island here.
And it's going to be sunny, so let's get some yellow chalk here.
There's the sun.
And every now and then it's going to rain.
We need some white chalk for the rain.
There's a cloud that comes in.
That's going to be one of the reactants.
And then some seeds blow in from the continent, and grass starts to grow on the island.
Grass.
And then there's a shipwreck and a couple of rabbits get on the island.
And now we have rabbits.
Rabbits.
A little tail on the back.
I hate rabbits.
I have a little vegetable garden, a little tiny vegetable garden.
And there are rabbits.
And I have to fence in my vegetable garden and make it impermeable impenetrable to rabbits.
And rabbits are very clever.
And my vegetable garden looks like a fortress.
It's got a green fence.
Heavy duty things.
It's terrible.
Rabbits are awful.
They're very cute but they're awful.
Alright, so now what happens.
Well, given the rain and the grass, it turns out that this island can only support 25 rabbits.
If there are more rabbits than that, then they all eat too much.
If there are less rabbits then that's fine.
But 25 is about the maximum that can support on this island.
The rabbits, unfortunately, are not that smart.
They don't know that.
They don't know that only 25 rabbits can live there.
So, what happens is that the Year one, Year one there are ten rabbits.
And the food condition is great.
Lots of food, number of rabbits.
Lots of food.
So the bunnies, the rabbits make bunnies.
Lots of rabbits.
Year two, they kind of went overboard.
Now there are 50 rabbits.
The food is lousy.
Terrible.
Well, there's a feedback here.
Feedback and rabbits start to die off.
Don't reproduce as much.
Year three, we're back to ten.
Great food.
Rabbits don't notice.
They multiply.
Food is terrible.
And they just don't learn.
Humans are a little bit like that.
Except our cycle isn't one year, it's usually on the order of 30 years.
It's like the memory of a generation disappearing.
And we have to relearn the mistakes of the previous generation.
Science is like that too.
Science goes in cycles.
20, 30-year cycles.
Topics that were out of fashion 20 years ago, 25 years ago, come back.
And suddenly everybody's excited about them.
And all the ideas, of course we make progress.
We go a little bit further than we did 20 years ago.
But all the ideas that were out of fashion 20 years ago come back.
People get really excited, and they relearn all the mistakes that people made at the beginning.
Technology has improved, we know more, and therefore we go a little bit further this cycle than we did the previous cycle.
But it's really interesting to look at history of science and see this sort of cycle.
So let's write a mechanism.
Let's write a mechanism here.
So we have rain plus grass makes more grass.
At the rate k1.
That's autocatalytic.
It provides some feedback.
Then we have grass plus rabbits make more rabbits.
OK, let's do more because I don't know how much, more, more.
Also autocatalytic.
Then we have rabbits.
Eventually, unfortunately, rabbits become dead rabbits.
Rate k2 k3, and that's the termination of the process.
OK, so now let's put some chemistry on this.
Let's assume that instead of having objects, live objects here, we have chemical molecules.
So A plus B goes to 2B.
That's the first step.
Then we have B plus C, C is the rabbits.
And then C goes to some sort of products.
Soil.
This is a very famous mechanism.
It's called the Lotka-Volterra mechanism.
It's also called the predator-prey mechanism.
In my case here, I made it sort of warm and fuzzy.
I didn't have a predator in there.
But you could change this.
Instead of having rabbits and grass, we could have rabbits and foxes.
Where A is the grass, if you take A to be the grass.
B to be the rabbits.
And C you be the foxes.
Works the same.
You have grass plus rabbits makes more rabbits.
Rabbit plus foxes make more foxes.
Eventually the foxes die off.
Same idea.
So now, let's solve this.
Let's assume that A, the rain here, it's constant.
There's a steady supply of rain.
It doesn't go away.
That makes sense in my example here.
So the concentration of A is kept constant.
In order to get oscillations to keep going, that turns out to be important.
To have one of the reactants just keeping being reintroduced in the system.
Because it gets used up.
And when all the reactant gets used up, then you're done.
So in the case of the heart, we have to keep feeding ourselves to produce energy.
Otherwise the heart would stop.
OK, now we want to know what is the concentration of B and C as a function of time.
So we write down our kinetic equations.
dB/dt gets produced through the first step, k1 A times B, gets destroyed in the second step.
k2 B times C. dC/dt, C gets produced in the second step, k2 B times C. Gets destroyed in the third step.
k3 times C, got to do your bean-counting correctly.
And the strategy here that we're going to use to solve the problem is to assume that we're at steady state.
We're going to find the solution at steady state.
Then we're going to perturb away from steady state a little bit.
And see whether this creates an oscillation.
First we need to find out what the steady state solution is.
So first, let's find steady state.
Then perturb away from steady state.
We're looking at a harmonic response, if you've heard that term.
When it's going to look like a spring, close to its equilibrium state.
That means that we're going to set it these two things equal to zero, for this steady state.
Things are not changing.
Concentration of B and C are not changing.
A is not changing on purpose here.
And so all these guys here are then steady state concentrations.
If we set it equal to zero.
We solve for A in this.
For C in this process here.
The B's cancel out here.
We divide by B.
And you solve for C as a function of A.
So this one here gives you C steady state is equal to k1 over k2 times the concentration of A.
And this one here, now you put in the C and you solve for B, B steady state, is equal to k3 over k2.
So the next step is to perturb away from equilibrium.
Or from the steady state, rather.
This is not equilibrium, it's not equilibrium because we keep adding A.
We have to put some input into the system.
So here we are.
We've got some concentration of B here at the steady state.
We have some concentration of C at the steady state.
And now we're going to add some delta B to the system, or delta C. Or both.
Delta C, we're going to ask the question, given our system here, how is it going to respond?
It has a number of choices.
It could respond by going back to the steady state in a monotonic fashion.
Without overshooting.
This is kind of like an overdamped system.
It's like having a shock absorber on your car.
You go over a bump, the car doesn't oscillate up and down.
It's sort of damped.
It's a damped oscillator.
It goes back to the steady state.
Or, you could, there we go, there's B, C. Or you could perturb, and it could just stay where it is.
That could happen too.
That's an inelastic response.
Kind of like the supply and demand with oil.
The price of oil goes from $10 a barrel to $120 a barrel.
The use of oil doesn't seem to be affected very much by the increasing price.
You're still using just as much oil.
There's an inelastic response.
What we're looking for is something that has an elastic response, or a harmonic response.
We perturb it, it tries to get back to steady state.
But because of feedback, and not being over-damped, if it's an oscillator, it goes up and down.
Like a scale that is not well, well, like a car that has bad shock absorbers, all it has is the springs.
Up and down.
So what we want is, we want to solve for dB/dt as a function of time.
We want, well, I'm going to do everything in green.
Since I lost my white chalk.
We want delta B, delta C as a function of time.
So, that means that we start with B is B steady state plus delta B, C is C steady state plus delta C. And we put that in our equations.
For dB/dt and dC/dt.
So now dB/dt is the same thing as dB steady state plus delta B dt, which is d delta B / dt.
Because this is a constant here.
And that's equal to k1 times A, times B steady state plus delta B.
Minus k2 times B steady state plus delta B times C steady state plus delta C. And then you have the same thing for dC/dt, it's going to look exactly identical.
d delta C / dt.
Thank you much, this is magic.
k2 times B steady state plus delta B, times C steady state plus delta C, minus k3 times C steady state plus delta C. So these are two differential equations.
And there are coupled differential equations.
Because delta B is occurring here, here, and here.
So the time derivative of delta C depends on delta B.
And the time derivative of delta B depends on delta C. It's a coupled system.
And if you actually expand this out and take away all the terms that cancel out, you end up with something that looks much simpler but is just as hard.
d delta B / dt is equal to minus k3 delta C and d delta C / dt is equal to k1 A delta B.
And then you really see that it's coupled, because the time derivative of delta B depends on delta C. And the time derivative of delta C depends on delta B.
And there's the feedback sitting right here.
And that's what you do.
You learn how to solve in 18.03.
So I'm not going to solve it here, I'm just going to give you the solutions.
Because that's what we're interested in.
After all, you're not expected to learn how to solve this equation.
Although you should be able to put the solutions in and make sure that they do work out.
And so the solutions are harmonic solutions.
Delta B is a function of time is equal to delta B0.
The amount of the perturbation times the cosine of omega t, where omega is the frequency of the oscillation.
Minus some pre-factor here, k3 over k1 A to the 1/2 power.
Times delta C0 sine of omega t. And then delta C looks the same.
The same frequency.
Delta C0, cosine of omega t. Plus k1 over A times k3 to the 1/2 power.
Delta B0 sine of omega t, where omega, the frequency of the oscillation, is these rates.
k1, k3 and A.
So obviously, keeping the concentration of a constant is important because we don't want a time dependence here.
The rain is constant.
And then if you do that, then you have this oscillation.
Up and down and up and down with this frequency here.
You get exactly what we're trying to get.
Which is this process right here.
So let me give you an example of this.
And the best way to do is to actually go and see a movie of a reaction that looks like this.
This is the reaction that was optimized and created for demonstration purposes.
So it's a pretty complicated reaction.
There are ten steps to the mechanism.
And if we gave you that mechanism to solve for the final exam, you would still be here probably two weeks form now.
It's pretty complicated.
You'd need a computer for sure.
So, ten steps, well, that's what's known.
There are probably more than that.
It's probably more complicated than what we really know in terms of fishing out intermediates.
And the basic reaction is, you start out with an oxide of iodine, which is clear.
You create I2, which is going to look gold.
In solution, and then it creates Ii minus, which is going to look deep blue.
And then it'll feed back and start again.
So we're going to have this cycle of products.
And amongst those ten steps, or more than ten steps, there are a few autocatalytic steps, which provide the feedback.
So now I have to start it.
Let's go ahead and read.
Alright.
So here we go.
So, there are two solutions, A and solution C. Actually, we mix three solutions together.
The potassium iodide is what is going to give rise to the different colors.
And now we're in the gold phase of the reaction.
It's important to stir, otherwise it wouldn't, and then it turns deep blue.
And then there's an induction period, where you wait for a while.
And then eventually it should go clear.
So it goes clear.
And then it turns gold again.
And then deep blue, and there's the oscillation.
And then a long induction period, and then it goes through that cycle over and over again.
And the recipe can be found pretty easily.
And if you ever are in a lab, have access to doing this, it's a pretty easy reaction to put together.
The only problem is that unless you use fresh hydrogen peroxide, it doesn't work.
Because hydrogen peroxide tends to go bad over time.
If you're going to do this, buy your hydrogen peroxide immediately before you try to do the experiment.
And then it'll work.
So what happens is that as long as there is a steady supply of hydrogen peroxide in here, it'll keep oscillating.
But as you've used up the hydrogen peroxide, this is going to start to die.
And eventually it'll have this pale blue solution.
And it's like the heart stopping.
If you want to start it up again, add more hydrogen peroxide and it'll start up again.
OK, any questions?
There's like an infinite number of these oscillating reactions that people have discovered.
And optimized, that you can find in different places.
Questions?
Questions about the final.
I didn't get to do a review, but you'll get that done with the TAs.
Yes
[INAUDIBLE]
Because there's enough of the equivalent of A, of the rain here.
So it'll oscillate and there'll be some damping.
So what will happen in this reaction is that the induction period will get longer, and longer, and longer.
Because the w, this term right here, the frequency which depends on the concentration of A, that well get slower and slower.
And eventually it'll just die.
OK?
Well, active learning is the idea that students, to really learn something, to really understand something have to be actively involved and that just sitting passively and listening to a lecture really doesn't help students develop the higher order cognitive processes that they need to really, really understand something.
So you can listen to something, you can watch a movie, you can watch TV, and you can generally get the plot.
But if you're asked to recall specific details or to even explain a particular nuance associated with the TV show or movie, you can't really do it.
And that's what happens often in a lecture, is that students will sit in the lecture, they'll write down what's being said, but they're not really engaged with the material.
So active learning is this idea of, people say minds on, always hands on sometimes.
So students have to be actively with their mind thinking about the material, applying what's being said, and given opportunities within the lecture to apply what's being taught or the topic at hand.
And then active learning, strictly speaking means that just one particular individual is active.
Interactive, we tend to parse that a little bit and say interactive learning would mean the student has been active in his or her own mind in thinking about the material, but then is also interacting with others, peers or potentially the faculty member or TA in order to further develop understanding, construct meaning for the topic.
I always start maybe the second session of the class.
The second class meeting is a discussion of what we know about how people learn.
So a discussion of the literature and the research on human cognition and learning.
And if you take a constructivist point of view or a constructionist point of view, which really says that as I said before, to understand, people have to make meaning of a topic, they have to construct their own meaning.
And we show the research that really shows that this is true.
For higher level cognitive processes, people have to be actively engaged.
And there's research to show that.
We also show the classroom-based research, so [?
Freeman's ?]
2014 paper that was a meta-analysis of this 225 other studies that showed that in courses, in college-level courses where active learning was used, there was a 12% decrease in the failure rate.
And they normalized it to all of the important factors that they should be normalized to-- the experience of the instructor, the size of the class, the type of the institution, the position that the class is situated within the larger curriculum.
And across the board it was shown that there was a 12% decrease on average of the failure rate.
And they make a comment in the paper that if that had been a clinical trial of a drug and 12% of the people on the drug had [?
shown ?]
marked improvement, they would have had to stop the trial and give everyone the drug.
So this idea that there's a 12% decrease in the failure rate in courses that use active learning, to me is pretty compelling that we should all be using active learning.
So whenever possible, because our students are MIT students, we use data, we use the research, and we try to find really good research, solid research that shows the way people learn and then how to support that with specific classroom practices.
So many of the students haven't had the experience of being in a class where active learning was used, so they don't really understand it.
So when we start to talk about it as a way of teaching, they may not really get it.
So throughout the course, from the first class all the way through, I try to use several different types of active learning exercises each class.
So the students themselves are actively engaged with the material from the first day.
So I may have them break into pairs and discuss a particular topic or identify something they didn't understand from the pre-class readings.
And then after three minutes, they can either share their comments with someone else or maybe we just report out to the larger group.
If they just report, if they just write down and then report back, that's a pretty good example of active learning.
It's pretty simple, it's what you might call low-hanging fruit in terms of what it takes for a faculty member to do that.
So I'll do that, I'll have them do that.
And then I step back and say, OK, why did I ask you to think about it?
Why did I ask you to take three minutes before we had this discussion?
So I try to deconstruct the exercise for them, showing them the advantages for the learner.
The beach ball strategy is a way to get students who might not want to answer questions or be a little less outgoing in their answering of questions to answer questions.
So there's a few different ways to use the beach ball, actually.
And one is you just pose a question that has enough variation in the types of responses.
So you wouldn't say OK, what's 2 plus 2 as an example for the beach ball question.
You would have to have something that was a little bit richer.
And then you throw the ball at one student or up into the lecture hall, if it's a big lecture hall.
And whoever catches it has to respond.
And that response, you share it with the group, comment on it if you want to comment on it.
And then that person who gave the first response throws the ball to somebody else.
And then whoever catches the ball that time responds, et cetera.
You can use it coupled with a picture prompt or a graph.
So if you show a graph or an image and you say, tell me something you observe about this image, and then you throw the ball and the person catches it first says one thing they observe, and they throw it and the next person says something that they observed about the picture prompt.
So that gives a foundation and it opens up space for lots of responses.
The beach ball does a few things.
One is that as the instructor, you don't have to be the one that calls on the students all the time, so the students don't feel like you're picking on them, that you're the one that's cold calling on them.
And in fact you can throw on the beach ball completely blind the first time, close your eyes, throw the ball, and so nobody really feels that you've called on them.
And then subsequently, it's other students that are throwing the ball, so there's no issue that you're the one that's picking on the students.
The other thing is it's kind of fun, you're throwing the ball around, so it's hard to get upset, it's hard to get as a student nervous that you're going to have to answer the question.
And in fact, you could dodge the ball if you really didn't want to answer the question.
So it's a great way to get students talking.
And in general, it's pretty fun.
And at first I thought oh, I can't really use this with grad students, it's too goofy, it's too baby or silly.
But actually, every time I use it with grad students, postdocs, everybody seems perfectly fine with it.
I think it's more to do with the nature of the question.
If you just ask stupid questions and throw the ball around, it'll be stupid.
But if you ask good questions and it's a way for students to give you their answers, then it seems to work totally fine.
That's the general way.
Sometimes I'll actually write questions on the beach ball and then throw the ball into the class.
And then students are loosely supposed to answer the question that's in front of them when they catch the ball.
So that's another way to do it.
But it's a nice, inexpensive, fairly flexible technique.
The technique of debate is a really interesting and useful technique.
I make sure to arbitrarily assign students the side that they'll be taking in the debate because it's pretty easy to debate something you feel passionate about.
It's not so easy to debate something that you don't feel passionate about, or where you feel the other side is correct.
In terms of learning, I think it's a great exercise because it forces people to, at least superficially, understand the stated arguments for the side they're taking and then find a way to at least be semi convincing about those arguments.
And then I give them a period of time to prepare, to sort of prepare their arguments about how they're going to defend their positions.
And then I have them voice their arguments for the particular side that they've been assigned.
So in crafting the arguments, pro if you're a con, you really have to start to understand what goes behind those arguments a little bit better, and so that's where the learning takes place I think.
The other thing is that the students get a little bit competitive, so they want to construct a good argument.
They want to sound convincing.
And so that extra little bit means that they really do put in the effort to construct a good argument and to make it believable.
If I were making recommendations to someone who had never used a debate before I would say that you should, first of all, don't pick anything that's really emotional to start out with.
So people can have a pretty strong opinion about it, but it's not something that's really deep.
So I would stay away from something too touchy.
The other thing is that it should be something where it's really not a slam dunk one way or the other.
So you want to make sure that the topic that you choose really does have two reasonable positions.
You don't want to give one team just a terrible thing to have to argue.
The other thing you want to make sure is that you give students enough time to craft their arguments and to debate the points.
So that if you cut the debate short after three minutes, it's just not going to help.
So you have to set aside a pretty good chunk of time.
Another active learning technique that can be very, very effective in supporting student learning is the jigsaw.
And it gets its name from the activity involved when you do a jigsaw puzzle.
And usually, most people when they do a jigsaw will look for the-- they'll find all the edge pieces, or they'll find all the blue pieces, or they'll find all the pieces that have a certain image on them.
You do the homogeneous aspects of the exercise.
So you get all the edge pieces, the blue pieces, the particular image pizzas.
But eventually, you have to take those pieces and put them together.
You have to link them up with pieces that don't look quite like those pieces, so that's the heterogeneous component of it.
And so what you do in a jigsaw is you ask students to become experts, if you will, homogeneous experts, in a particular area, on a particular topic.
And you can do this by giving them a reading in class, or you can do this by asking them to reflect on something that you've addressed in a previous class or that they've addressed in a previous homework.
But they sit together, or they group together in this, what I call, homogeneous group, where everyone is talking about the same particular topic.
And they just discuss that topic and they become experts in that homogeneous topic.
And then you can have three or four different topics, whatever, depending on the size of the class.
And you have these local groups of experts, like the edge pieces, and the blue pieces, and the picture pieces.
And then after giving them a period of time, maybe five minutes, 10 minutes, depending on the class, you break them up into heterogeneous groups.
So you get one with one representative from Group A, goes with one representative from Group B, and one representative from Group C, and one representative from group D. And now you have a new heterogeneous group that has an A, a B, a C, and a D in it.
Or if we use the puzzle analogy, you have a group that has some person that's the edge expert, some person that's the picture expert, and some person that's the blue piece expert.
And they're all in a group.
And now they share what they know individually with each other and then they combine that expertise, they synthesize that expertise, to make something bigger to come up with a more global understanding of the problem.
And then each group does that in their own way.
So the heterogeneous groups then form a more holistic, or a more heterogeneous, solution to the problem, as if they're putting together the different kinds of puzzle pieces.
And then again, as with the other activities, we ask them to then report out.
Before that I usually will circulate in the room the same way I do with the pair share to hear what each of the groups are saying, to see if any groups are having particular issues, or if they have some particular breakthroughs that I want to make sure they share with the class as a whole.
There are some logistical issues.
One, you want to make sure that the students really can become these homogeneous experts.
So that if you have a group of four people, that they've all really come up to speed on the topic.
So ideally, you would give a pre-class assignment and you'd say, yes.
I need everybody to read this article and to be able to explain this point, this point, and this point, and you would assign that to that particular group of students.
And then you'd do the same with a few other topics.
You're assuming then that the students have read the paper, or have read the article, whatever it is you want them to read, and that they understand it to the level that you hope they understood it.
Sometimes you don't have time for that or it just doesn't work out, so you give them something to read within class.
And again, that can be tricky.
Students may not have the same level of understanding, so groups may be a little bit spotty.
The other thing that can happen with the jigsaw, which is a really silly problem but sometimes it just trips you up, is if you have, depending on the number of students, you have to kind of think on your feet about how many students are in the homogeneous group versus how many students are in the heterogeneous group.
If it's a perfect square, it works fine.
So if you've got nine students, or 16 students, or 25 students, you know you have five groups in one situation and then five groups in the other, that means you have to have the same number of groups as heterogeneous topics.
So if I went A, B, C, D, I have to then have four groups because I had to have four A's, four B's, four C's, and four D's.
So sometimes just the number of topics and the number of students that you have in your class makes it kind of hard to make groups that don't have extra people or aren't short a particular expert.
And that's just something you want to plan beforehand.
And if somebody doesn't come to class, if you know you have 25 students in your class and someone doesn't show up, then you have to kind of deal with it on the fly.
But that can be the trickiest thing with the jigsaw, is just getting the number of students right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, great.
I looked over some of the Mud cards we had from last time.
And there were some really good questions, some really good points, and one lingering one from the previous time that I forgot to address.
Someone said, could we please just move the desks into an arrangement where we could see each other.
It was a fabulous suggestion.
Today we can't do it.
But I'm going to try to do it in the future whenever the logistics allow, because I think you're absolutely right.
A little bit more of a U, or a circle, would be great for this class.
So thank you for that suggestion.
Another one was that we didn't get to motivation theory.
And I think I mentioned this last time, the idea that there's a huge amount of stuff that's been written and published on motivation and theories of motivation.
And I'd be more than happy to discuss this as the semester goes on.
And it might be a great topic for one of those lunches that somebody was going to organize, one of those informal lunches that someone was going to organize.
Or maybe we can carve out some space at the end of the semester and pop it in, take something out and pop it in, because it's really fascinating.
It's really interesting, but it doesn't quite fit anywhere in particular.
So thank you for bringing that up.
And keep bugging me about it.
The other thing is someone asked if all those learning theories that we discussed were equally valid.
And yes, they are equally valid.
I mean, there are situations where being behaviorist could help.
There are situations where being constructivist could help.
It depends on what you're trying to have students learn.
So you want to make sure that you tailor the way that you're teaching to support what it is you want students to learn.
And we'll get into that in a lot more detail today when we talk about developing learning outcomes, and then also when we talk about tailoring your instructional activities to support those learning outcomes.
So this class, this particular class today is a balance between developing course design, developing intended learning outcomes, which for me is at the heart of it.
If you can come up with some good intended learning outcomes, you basically have designed your course.
And I'll show you how.
I mean, you can't just ignore it, but that's the hard work, and then the act of writing constructing a syllabus.
So once you've got those learning outcomes, you have an incredible anchor for the rest of your course.
And there was just a very timely haiku.
I put it on the wiki.
But there's this Twitter feed, academic haikus, which if you want to waste any time and read them, it's a way to waste time.
But there was one here, "that question you have?
I know where the answer is-- on the syllabus.
So it was, like I said, it came out yesterday.
I thought it was pretty timely.
And it's true.
If you write a good syllabus, it should really clear up a lot of, not the content, not the understanding level issues students have, but the issues with respect to when's the test, do we have to know this, how will this be graded, what's important?
All those things should be in your syllabus.
So that's why that's there.
So here's a little slide, a graph that I saw a few years ago, and I thought it's a really good way to start this discussion.
So on the y-axis we have sort of an assessment, a proclamation of students understanding.
They got it, they didn't get it.
And I think the word in French is [FRENCH].
So for some reason I find it very cool to say [FRENCH].
Got it.
I don't know.
So whether they get it, like whether after the tests, you've given a test, and you go, man, that student really got it.
That student really gets this material, and whether they passed the exam or failed the exam.
So the size of the circle is more or less the number of people that fall into those quadrants.
So anybody surprised by this plot?
Yes, Rachel
[INAUDIBLE]
OK. All right.
Well, there's the segue.
So we want, we don't want the green, but we more or less expect the green, right?
If you get it, you pass the exam.
If you pass the exam, you get it.
That's that, and we like to have lots of people have that happen too.
And then we also acknowledge that somebody isn't going to get it, and they're not and they're not going to pass the exam.
And that kind of maybe makes us sad, but it's not surprising.
And then anybody can kind of have a bad day or kind of a fluke.
So sometimes, sure, somebody will fail the exam, and actually really understand the material.
Generally speaking, that doesn't happen, and then there's often a reason for it.
But it's that red circle that Rachel mentioned that is troublesome, at the least, troublesome.
So why might that happen?
Why so many red circles?
Yeah, Dave
Because the test was set up for basic memorization, or you could figure out with a lucky guess with multiple choice.
So you could easily pass just with good test-taking skills or a good memory
Excellent.
So you pass with sort of good test taking skills.
Now I want us to hang on to that.
There's an underlying assumption in Dave's comment.
So I'd like to hang on to that thought, but let's hear a few other conjectures and see.
Rachel, why do you think you might have passed it?
So often I feel like TAs are either really good or really terrible when it comes to the exam reviews, and they're mainly good.
They often told us too much of what was going to be on the exam.
And so then you kind of just knew what the test was going to be if you could well on it if you [INAUDIBLE]
OK. That's interesting.
Yes, Gordon
Yeah, it's possible that [INAUDIBLE] tests [INAUDIBLE].
When it comes to true life situations and something that [INAUDIBLE] knowledge that was gained from that material, then [INAUDIBLE]
OK.
So Gordon is sort of kind of flirting with this idea of you as the instructor, this real life knowledge, this is what it means.
They have to have this practical working knowledge for us as the instructors to say they got it, right?
But the exam isn't measuring it.
And that's exactly what, that's what Dave was saying as well.
This idea that the exam is measuring one thing, but it's not really measuring whether they got it.
When we go home and think what does it mean to be a good material scientist, or what does it mean to be a good chemist, we don't think, oh, it means that you can define what a mitochondria is, or you can balance a simple equation.
And yet, we put that on the exam.
It's also a bit of the idea that students are good at taking tests.
They're good at gaming systems.
They're good at listening to TAs or TAs may have not given out appropriate information.
So it comes from both sides.
It comes from the students and it comes from the professors.
But you have control over this.
You shouldn't be writing an exam that people can get a good grade on, but then when you go home and say, oh, they got a good grade on this test, but they didn't get it.
You need to define what it means to get it in your field, to get this subject, whatever it is that the test is about.
And then you need to write a test that measures that.
And too many times we don't do it.
We're kind of rushed, or we don't really know.
We haven't really thought about how to measure whether somebody gets it.
So the first step in all this is to define what it is that means that they got it.
And that's the first step, and that's the hardest step.
But once that's done, the rest is easier, and it will save you from this.
OK?
OK.
So that's the topic of today's class.
And if you needed a little more motivation, I'd like you to read this paragraph, and I'd like you to tell me what it's talking about.
But I'm just going to give you about 2 and 1/2 minutes to read it.
And then we'll discuss what it means.
What do you think this is talking about?
Yes, David
Yeah, I think it stresses about how we can arrange by say, maybe schedules or activities that put them into groups.
Then maybe if we form a group, how we plan [INAUDIBLE]
OK.
So arranging schedules or activities.
OK. OK. Something else?
When you're planning, how to go about teaching a module, [INAUDIBLE]
Planning teaching.
What else?
Anything?
It's like a mailroom operation or something
Mailroom.
Something from the back, maybe?
You know you're smiling
I couldn't think of anything
All right.
Well, then don't say anything
The first is the course content, the course outline, and what you are going to teach the students, [INAUDIBLE]
OK.
So students, grouping students-- Adam, right?
Adam.
OK.
So content, arranging content.
Anybody in the back?
Flyer?
Yes, Gordon
I think you talked about sorting materials
Sorting materials.
So we'll go with maybe sorting.
So if I tell you-- so these are all great guesses.
They're as good as what I'm going to tell you the answer is.
So this was written to describe the process of doing your laundry.
Like doing the wash. You take the clothes and you put, maybe you put the dark clothes in one bin and the white clothes in another bin.
And then you fold them and you put them away.
It's impossible.
This happens almost every time I show this little piece of writing.
People don't generally get it.
They get the fact that it has to do with sorting, and some planning, some arranging things, but they don't get that it's about the laundry.
If you skim it-- you don't have to totally reread it-- but if you skim it with the idea that it's about doing the laundry, you probably can understand it better, maybe.
So now I'm going to ask why did I show you this now, today, before the third, the beginning of the third class?
Yes
I think it's sort of about arrangement of course content [INAUDIBLE]
So it could be about how you arrange the content in the syllabus.
[?
Hina.
?]
I think it's easy to describe methodology as a person more in a different field.
But if there's no context for the audience about what this is, it doesn't matter that you've laid out everything.
No one knows [INAUDIBLE]
Right.
It's very detailed at a certain level, but it's still really hard to understand, right?
Any other ideas?
Yes
I think it's interesting.
There's a lot of value judgements in it.
Like, it's really easy.
And one can never tell with comments like that.
Because I'm trained to figure out what they're saying, it's actually just frustrating me more, because I don't know what they're talking about.
And it's really easy--
All right.
So that's a great point.
So we're talking about writing learning outcomes, about organizing the course around writing learning outcomes.
If you're using phrases, or you're using language that's just, there's just like throwaway words in there or value judgments, it's not doing your students any good, right?
And also, that if you have this beautiful course that's beautifully organized even, wonderfully organized, but if you don't tell them what it is you're trying to do with it all-- what it is they're going to be able to do with it all, whatever it is you expect them to do-- they're not going to be able to follow you.
So imagine a procedure as fairly straightforward as doing the laundry.
Now imagine, OK, you're going to be able to select the right, the appropriate method to integrate equations, so methods of integration, by separation, by card or substitution or whatever.
So if you don't tell them what it is you're expecting them to do, even if you're very good at explaining what it is they're going to do, they won't get it.
So this is my sell for why when you write a learning outcome, you have to write a learning outcome that's clear and that's understandable to the students, that doesn't have a lot of extra words and that tells them exactly what they're supposed to do.
So I have some learning outcomes for today and I hope that they're clear.
You'll be able to describe the components of the constructive alignment and backward design processes.
And those, constructive alignment anyway, was in the pre-class reading.
You'll be able to classify the content of a course that you might teach.
So what do I expect students to do with this content, to be able to do with this content?
You'll be able to create measurable, specific, and realistic learning outcomes for a class you will teach, and you'll be able to state the components of a syllabus.
So I feel that those are pretty understandable.
You may disagree with me, or maybe you want to wait.
So constructive alignment, it was in the pre-class reading, right?
Anybody want to just give a brief summary of it?
I mean it's kind of up here, but in your own--
I think constructive alignment is about aligning assessment, method of assessment, with course content.
[INAUDIBLE] assess [INAUDIBLE] type of assessment so that what the teacher has in mind, the course content or the content I want them to get, to test it with the assessment process.
Then we would also do the assessment, and then come up with the intended outcome
Right.
They'll achieve the learning outcomes.
And if you do this right, you get rid of that red circle on that graph.
You get rid of students who did well on the assessment but don't really know what's going on, because you have aligned the assessment with what it was you wanted them to know or be able to do.
So that's important.
I just think this is a very, very compelling image to keep in your head, that these three things have to be linked.
And as I said before, this, you will see that the top part, writing the learning outcomes is the tricky part, it's the hard part.
It's the part that requires a lot of thinking and work.
When you get a good learning outcome, the other two things almost take care of themselves.
OK?
Did anybody have any, does anybody have any questions about something from the reading associated with constructive alignment or some other idea from the reading?
Yes, Gordon
I just wanted to make a contribution.
I think that this constructive alignment is one of the things that the intended outcome should measure.
So I think maybe that's confusing [INAUDIBLE] and also how we can test and be sure and confident
Right.
Right.
I think the measurability part for scientists and engineers is kind of intuitive.
Like why would you say something if you couldn't tell whether or not it was true, or not from a scientific point of view.
What's the point of saying, making a proclamation about what somebody should be able to do if you can't measure it in the end?
So it's comforting to say, OK, I'm going to make learning outcomes that are measurable.
But it also can be a bit challenging.
But we'll do that.
We're going to do that today.
Some other comment?
Nina
I guess as far as [INAUDIBLE] Do people write learning outcomes that have no constructive alignment process, like choosing to write learning outcomes, are you going through the prcoess?
It seems decoupled a little bit in the reading, but it seems like you can't have one without the other
You owe it to yourself-- and I don't know what people do, and I guarantee you people write learning outcomes without thinking about whether they can assess them or measure them, I guarantee that happens in real life-- you owe it to yourself to try very hard not to do that.
OK?
And I think the important thing to do is, often we say, oh, my god, this is the best assignment.
This is the greatest question.
It's this project, and they're going to do this, and they're going to whatever.
They're going to build a race car.
And if you sit down and say, well, what do I want them to get out of that?
If you can't articulate it in a way that's consistent with your class, consistent with the course as it sits in this series of their courses, or as it sits in the institution, then you either need to rethink the assignment, rethink the project, or think a lot harder about what learning outcomes that activity advances.
That's a very good point.
It's really kind of iterative.
And I think it might be easier to see with this diagram.
This is another-- so John Biggs is from the UK.
And so there's sort of the UK camp of people and they like John Biggs.
And then there's Wiggins and McTighe.
Grant Wiggins and Jay McTighe are from the US.
They wrote primarily for the K through 12 audience, but it's completely applicable.
And they're from the US, so they have kind of a US following.
It's just how it works.
It's essentially the same thing.
It's identify the results, so the intended learning outcomes.
Figure out how you're going to get them there.
So the black is Wiggins and McTighe and the red is Biggs.
So plan the learning experiences that are going to support them, achieving the learning outcomes, and really help you measure whether they got them or not.
And then determine what evidence will be that acceptable evidence that they achieved the learning outcomes and what are the assessments.
So real often we do have this project that we want.
Oh, this is a great problem.
Often in thermo-- I taught a lot of thermo-- there's like, oh, they're going to do this double loop, double reactor cycle with the refrigerant and a heating cycle.
And it's going to be coupled and they're going to use the output from one for the input in the other and all this crazy stuff.
Like, this is a great assignment.
And you kind of, I know this is a great assignment.
If I can't back up from this assignment-- whether or not I put it here or here-- if I can't back up and say after a successful completion of this assignment, you will be able to describe the components of the Brayton cycle, you will be able to define the increase in entropy for a system with x boundaries, If I can't articulate those things, then I need to either change the assignment, or, well, then I need to change the assignment.
OK?
But you can start with the activity, and then back up and say, well, what learning outcomes does that support, and then, am I OK with that?
Is that relevant to this class?
So it's often a very iterative, the arrows kind of go this way, but it's often iterative.
So I had asked that you come to class with some topics from a class you're likely to teach.
That was on the assignment page.
If not, start thinking now.
And so you want to have, for most classes, most undergraduate classes, there's a bunch of topics.
You might have 10 topics, 20 topics.
And I don't care right now about the scale of the topics, whether they're really big.
You probably don't want to go with like first law, second law.
That's probably a little bit too big, but on the order of 5 to 15 topics.
Jot down the topics.
And then I have a chart here.
So the chart has two sides, and the columns are Bloom's Taxonomy, which you don't have to worry about that specifically, but you can look at the words that are associated with it.
And then the second column is what students should be able to do with each topic.
So let's say my topic is first law for closed systems.
So I might think about, OK, do I want them to be able to define, to just state the first law for closed systems?
Do I want them to be able to use the first law for closed systems in order to solve certain kinds of equations?
Do I want them to be able to derive the first law?
Do I want them to be able, whatever it is.
So you're going to look at the columns, the categories on the left side, and you're going to think about your topic.
And you're just going to decide which box in the middle column to put the topics into.
You don't have to worry about writing anything.
You're just going to distribute the topics to the level of, it's really cognitive processes that are associated with the things on the left.
Remember, understand, apply, analyze, evaluate, and create.
So we'll take about five minutes.
And you just do that by yourself.
And just fill in that middle column of the table with the topics from your course.
Do you want more explanation, Rachel?
It's starting to make you think about this part of the beginning circle.
But you don't want to write any sentences yet.
But it's starting to help you think about what are they going to do with these topics.
What do I want them to do?
Is it remember, or is it create something using these things?
Where is it?
Those are two ends of the spectrum.
And in your chart, the things that at the top are the more remember, right?
It's more about kind of a recall information.
And then the things in the back of the chart tend to be more higher order cognitive processes, like synthesize, apply, create.
Yes
[INAUDIBLE]
So don't worry about the third column.
Don't worry about the last column yet.
We're going to do that later.
So all you have to do is just say where each topic falls loosely.
OK.
Does everybody have a chart?
Do you want us to write as many topics as possible?
Write as many topics as possible, but distribute them through the chart.
Right?
Not everything is going to be remember, right?
So some things will be remember, some things will be apply, some things will be analyze, some things will be create.
OK?
If you have any questions, just raise your hand.
All right.
So what we're going to do is I'm going to ask you to just keep those with you.
We're going to do another activity later on using those as a starting point, but I want you to just keep that sorting with you.
Any comments or questions about the experience, about what you just did?
Was it easy?
No.
It can sometimes be a challenge.
Well, what the heck you were supposed to do with this anyway?
What do I really want them to do with it?
This is the beginning of that process whereby you define exactly what it is you want students to do, know or be able to do.
And then you can start to think about how you're going to assess whether or not they can do it or not.
But it's very different if I were to say I want everybody to be able to state the first law of thermodynamics versus I want you to be able to use the first law of thermodynamics to define chemical, mechanical, and thermal equilibrium in a simple composite system.
That's different.
That's very different.
So that's what we're starting to do.
So now we have kind of the seeds.
We thought about which topics kind of we want to have, what we want to do with each of the topics.
But now we really have to turn them into learning outcomes, into real statements.
By the end of this class, you will be able to, by the end of this course, you will be able to.
And just as an aside, remember that sometimes you'll notice that for every class, every session we have, I have learning outcomes.
But there's also overarching learning outcomes for the course.
So I will use them interchangeably, but they'll be at a slightly different scale.
The ones for the class are a little bit more fine grained.
As was mentioned in the reading, and as Gordon mentioned, they have to be specific, measurable, and realistic.
If they don't have those properties they're not going to be particularly useful, or as useful.
All right.
And we'll see some examples of that in just a sec.
So here's what they're not.
They're not topics.
So you came with topics.
You will leave today with learning outcomes.
But topics are not learning outcomes.
The first law of thermodynamics is not a learning outcome.
You need a verb with it.
What do you want me to do, what do you want your students to do with the first law of thermodynamics?
What do you want them to know about the first law of thermodynamics?
So they're not topics.
They're not things that you will do.
It's easy to say, I could have a list a mile long.
I will talk about the first law of thermodynamics.
I can check that off pretty easily.
I mean I just did.
I just talked about the first law of thermodynamics.
It says nothing about what you learned about it, what your students learned about it.
So it's not about you.
Get over it.
That was a joke.
It's not value statements.
As Adrian pointed out, some of these think this is easy, this is good, this is useful.
Any of those things don't belong in a learning outcome.
And it's not your hopes and dreams for your students.
I mean, ultimately, it is.
You hope that they will be able to use the first law of thermodynamics, da, da, da.
But it's really about what they're doing, about what they're going to do.
So, no.
Here's a little example.
This is just from like an earth and planetary course.
Students will understand plate tectonics.
How's that?
Is that OK for a learning outcome?
Why not?
It's too broad
It's not very specific.
What the heck about plate tectonics?
What else?
Measurable
Thank you.
OK, the measurable part.
This word understand, strike it from your brains.
It's a property that we love.
We love understand.
But you can't flippin' measure it.
You can't go in there and measure it.
You can measure lots of things that tell you whether students understand, but you can't measure understanding.
And I think we'll see that with some more examples.
I don't think anybody is earth and planetary here.
So how about this one?
Students will be able to interpret unfamiliar tectonic settings based on information on volcanic activity and seismicity.
Again, I don't think any of us are geologists.
But the idea that I've told, or the person has told us what we're going to use to interpret these situations, and then we're going to make predictions based on those interpretations, I mean, that's an assumption.
So it's specific and now it's measurable.
Even though we're not geologists, I suspect we can think of how we could write a test question that would ask students to do this.
And then we could see whether they did it or not.
So that's measurable.
And probably, by doing this, by having students do this, we get a pretty good measurement about whether they understand plate tectonics.
But it's much more specific and it's much more measurable.
From here, this is from the University of Minnesota, which has a nice website on learning outcomes.
They're not from your point of view, as we mentioned.
They're not what you're going to do.
I'm going to introduce students to the major turning points and processes in North American history.
You can say anything you want, but it doesn't mean the students learned anything.
So it's not about you.
It's what they're going to do.
They're going to list and describe the turning points and processes in North American history.
I'm going to create an understanding of the formal constructs of physical design.
That one is like, this one is, it's like, I don't even know what it means, right?
Create an understanding of the formal constructs of physical design.
I don't even know what it means.
And so it's like that laundry example.
Your students aren't going to know what it means, and it's not going to help them.
It's not going to help them access the material.
So the ones on the left are all teacher-centered and they're not very useful.
The ones on the right are student-centered and they are generally useful.
But there's some things wrong with the things on the right
Understand
Understand, yes.
So what would you write instead of understand the formal constructs of physical design, whatever the heck that means?
What might you use?
Identify
Identify.
OK. That might be like a list, right, or identify or list, pick it out of the lineup kind of thing.
This is a nice one.
Explain to an intelligent non-expert, an INE.
And so explain this to somebody who's not in the field.
If you could do that, it probably means that you get it.
Somebody I know used to say, explain this to my four-year-old brother.
Explain-- what was it-- angular momentum, to my four-year-old brother.
I would argue if you can explain it to a four-year-old kid, you probably understand it.
So you can use that.
Use formal constructs to design something.
I mean, that's kind of an inversion.
How about the one on the bottom.
Understand gender, race, ethnicity, socioeconomic class, understand how they have shaped Americans' lives.
Completely unmeasurable, right?
So what would you write instead?
Describe
Describe how, sure.
Anything else?
[INAUDIBLE] it seems like you're trying to get [INAUDIBLE]
Right.
So that's a great point, is that we don't know where this faculty member put this on their table.
We don't know whether it really just states some ways that, list some ways that these things have affected Americans, North Americans' lives, or Americans' lives-- which is kind of a remember thing, just make a list-- or whether it's do some analysis on some situation.
We don't know that, but you as the instructors get to decide that for your classes.
So that's an important thing.
You get to call all the shots.
You get to make the decision.
We can only guess about what this person meant, and therefore, his or her students can only guess about what he or she means.
So you have on your table this hierarchy of Bloom's Taxonomy.
It's often shown in this triangle, this pyramid.
There's a handout which has exactly the same strata on it.
And it just has some handy dandy verbs associated with it.
But Benjamin Bloom came up with this classification of cognitive processes way back when, in the 1950s.
And it is this idea that it is this pyramid, that at the base of the pyramid are these ideas like remember.
And those are basic cognitive processes, remember, list.
And then it moves up the pyramid to things that require a higher level of cognitive processing.
Rather than just pulling facts out of storage and stating them, you're going to do something with those facts.
So as you go higher and higher up the pyramid, note that he chose-- really unfortunately-- to call the second level understand.
However, there are words that can help you.
If it really is understand that you want to measure, there's words that describe that a little bit better.
They're in your table, and they're also in that second table that I handed out.
So arrange, list, label.
For understand-- describe, relate, recognize, explain, those things.
Gordon
Just a little [INAUDIBLE].
When it comes to understand, maybe if you put it like [INAUDIBLE]
Understand to what?
Understand to an [INAUDIBLE].
Maybe that's what we can measure.
Understand to this level or to this [INAUDIBLE]
But how, can you give me a specific example?
I wish I could go back to the last slide [INAUDIBLE] can describe [INAUDIBLE]
OK.
So I see where you're going.
Does someone have a suggestion?
So think about that laundry.
Think about that description of doing a laundry.
Can somebody take what Gordon said and get where he wants to go, but get there in perhaps a more direct way.
Katherine
[INAUDIBLE]
Exactly.
So if you're going to do that, and I see where you're coming from, understand so that you can describe, in order to describe, or to the level of describe.
Just forget the understand part, and just say describe, or define, or whatever it is you want.
But yes, I mean, that first step is saying, what do I mean by understand?
And when you say that, then it generally rewrites itself.
And so the more direct you can be, generally, the better.
And keep that laundry example in your head.
Yes
You're asking a student [INAUDIBLE]?
So I would say, well, let's just open that up.
What do you think about that, of asking students?
So do you understand this stuff about learning outcomes?
I think as teachers, when you, in the front of the class, you're teaching students and you say do you understand, they say yes.
But you see from psychology you know who doesn't understand
So David says as I ask you that, do you understand?
Most people are going to say, hey, I understand, sure.
Don't think I'm an idiot.
I understand.
Right?
So I completely agree.
Students are not likely to say they don't understand.
What else?
That's difficult for the teacher to [INAUDIBLE]
Well, so do you want to say something about that?
Yeah, I agree.
I think the part that we do understand as nonspecific is the learning understanding [INAUDIBLE].
As an instructor you have to know, understand what you're trying to get at, which I think is where the clicker question is beautiful, because you know already what you're trying to evaluate that they're catching in class in real time, if you can see who's getting at the nuance of what you're teaching
Yes.
I think that's an excellent, excellent point, that many of these activities that we do, these small group activities, or the clicker questions or other activities that students are doing in class are helping them learn.
But they're also formative assessments of whether or not they get it or not, whether or not they understand.
And you as the instructor can see it.
If I walk around, if I listen, if I look at who answered what question on the clickers, or on the raising your hand, multiple choice questions, whatever methods I used here, I get a better, a much better sense of who understands or doesn't understand.
If all I'm doing is lecturing I can't tell.
And I can ask do you understand, but they're not likely to tell me.
Which is kind of the other point is that sometimes-- and I think you were alluding to this, Gordon-- that sometimes students don't know whether they understand or not.
So we need to give them the opportunities to find out whether they understand or not.
And arguably, if you're just lecturing and you're a great lecturer-- you're eloquent, you don't stumble, the writing on the board is awesome, all this stuff-- students will think they understand maybe when they don't, because they're not having to confront any misunderstanding, and it all seems very nice.
So these activities where you actually have students do something, say something, where you ask them a specific question, that's when you can determine whether or not they understand or not.
And because they are not likely to say they don't understand, either because they just don't want to admit it, or because they really don't know whether they don't understand.
So you need to give them measurements to find out whether they get it or not.
Yes
Sometimes, for example lecture and the activity has [INAUDIBLE].
For example, we have lectures and the teacher just [INAUDIBLE] and then we have expectations as we do exercises.
So do you recommend that we also do some activities during the lecture or just [INAUDIBLE]?
No, do them during lecture.
I mean, the model that we're trying to do here is the model that you should do even if you have a hundred or 200 people in your class.
You can use clickers.
You can use pair share discussions.
You can do all sorts of things to break it up, and it's effective.
There's a paper coming up for next week, a meta-analysis, active learning strategies, how they actually, students learn more in a lecture if you actually ask them to talk to each other, break up, share, et cetera.
And I think when we talked about misconceptions, remember that example with the coin?
How many students got it, could draw the free body diagram before and after?
I mean, before and after in a straight lecture, there's hardly any learning gain.
But if you incorporate active learning, the learning gains go way up.
So that's Bloom's Taxonomy.
And an interesting take on Bloom's Taxonomy, there's a woman, Kathy Schrock, who-- it's a nice website and the slides we posted so you have the link-- but she decided, she has this theory that it's not really so pyramidal.
That everything really is dependent, these cognitive processes are dependent on each other at a certain level.
And so she's drawn it as a gear, where everything sort of supports the act of creating, which is up at the top.
But the other ones are sort of around the side, but they all contribute to the ultimate cognitive process of creating.
So that might be a nice way to think about it.
And then she's taken the specific words, like analyzing, and said, OK, what's an instantiation of analyzing, outlining, deconstructing, organizing, structuring, surveying, whatever?
So it's the same idea, it's just a different graphical representation.
So what you've done is you've taken your topics, put them in those cells in the table.
And I just want to point out kind of how as the instructor you have a lot of power about where you're putting those, where you've decided to put those.
And it may be different for a first year undergraduate class, or a second or third year graduate course.
It could be the same topic, but you're going to put them in a different category.
So that's why when I asked you to bring a topic, it's for a specific class.
It's for a specific group of students.
So let's say we take the idea of interstitial sites.
So I have a crystalline structure, it's got an order of periodicity of atoms.
And then I have spaces between the atoms, those are the interstitial sites.
And so I could say you'll be able to identify the interstitial sites.
So I could say where's the tetrahedral site in a face-centered cubic or something, whatever?
And students just really have to find it.
I could say calculate the maximum size of an interstitial atom that could occupy that space, an ion that could occupy that space in a particular crystal structure.
So those are two very different levels, but it's the same topic.
X-ray diffraction, I could say apply Bragg's Law to calculate d-spacing.
So that's very much a turn the crank kind of thing.
I know the equation for Bragg's Law, I stick things in, I solve for the unknown.
I could then also ask you to index unknown diffraction patterns, meaning identify an unknown crystal structure based on the diffraction pattern.
That's a completely different activity, completely different set of cognitive processes than just using Bragg's Law.
But you have to decide what it is.
And arguably, if all you've had students do during the class is sort of problems that they applied Bragg's Law and they solved for d-spacing-- so they solved for the incident angle or they solved for the wavelength or whatever it is-- you really can't throw this at them on the exam, right?
Or conversely, if you spent the whole time, they've indexed, they've been in the lab indexing unknown diffraction patterns, and then this is the question you ask them on the exam.
Well, then you have people that perhaps get it without really, perhaps pass the exam without really getting it.
So that's just something to keep in mind.
The other thing to note is let's say this is my intended learning outcome-- calculate the maximum size of the interstitial atoms in a variety of crystal structures.
How do I know whether students get that or not?
What's that?
It's easy
It's very easy.
I give them a crystal structure and I say calculate the maximum size of the ion that can fit in the space.
So I have figured out how, the measurement is easy, it's cake.
It's totally straightforward.
Now I haven't told them, OK, I'm going to give you this crystal structure and I'm going to ask you this spacing, this ion size.
I haven't given it away, but I have told them what's expected of them.
And my colleague wrote these for VSEPR theory.
It's the same topic, and it's different.
Identify the common geometric shapes found in simple molecules.
Explain the assumptions of this theory.
Apply the theory to predict 3D structures.
Compare and contrast the geometry of a certain molecule as predicted by two different theories.
Evaluate the accuracy of each theory for a particular set of compounds, and then create some recommendations.
So it's the same topic, but you decide what it is that students should be able to do.
OK?
So I have some, just a little exercise for us.
So they should be specific, measurable, realistic.
I'm going to put them up here and we're going to say, you're going to tell me what's wrong with them.
So number one, t-tests are like a statistical analysis.
So is anybody a t-test kind of person?
We can skip that one.
How about gaining appreciation for the use of linearization techniques?
What's wrong with it?
It's not specific
It's not specific
It's not measurable
It's totally not measurable, exactly.
I mean, how do I know that you appreciate it?
Don't do it.
All right.
Great.
Have an intuition for the most effective method of integration for a given problem
It's vague.
It's not measurable
So I've heard it's not specific, it's not measurable, and it's not vague, I mean, and it's vague.
So right, so have an intuition, I cannot measure that.
I have not figured out a way to measure intuition.
I can tell whether you can do it, right, but I can't tell you whether you have an intuition for it.
Well, how would you rewrite it?
Actually, how would you rewrite 2?
Use [INAUDIBLE]
So it could be use the techniques.
Another suggestion?
Make something using the evaluation techniques
Calculate something.
Make a first pass estimate using a particular linearization technique
Describe when you would use a linearization technique
Describe when you would use it, describe why you would use it, all of those things
List those areas where [INAUDIBLE]
List the areas where it's useful, yes.
Great.
All right.
For number 3, the gain an intuition, we decided it's vague, it's not measurable.
So how would you rewrite it?
Compare.
Compare the areas integration technique [INAUDIBLE] Solve a particular problem using different kind of [INAUDIBLE]
OK.
So solve a problem, and then we need a little more.
Did you have something?
Identify the most effective method of--
Let's hear what Rachel had to say.
Rachel
Yeah.
Learn how to identify the simplest [INAUDIBLE]
Right.
So you probably don't want to say learn how.
You would say be able to identify.
Right?
Gordon
[INAUDIBLE]
Right.
I mean, you could just say integrate--
[INAUDIBLE]
Integrate a problem using whatever.
Select a technique to integrate a particular equation.
And again, those things end up being more measurable.
So 4, provide problem solving tools and strategies.
What's wrong with that one?
It's too wide
It's too wide
It's instructor centered
It's instructor centered.
What else?
But it's measurable
Well, I can measure that I provided it.
Here, I provided these handouts, right?
OK.
Check.
It happened.
Whether or not you could do anything with them, I don't know at this point, right?
That's the problem
It doesn't tell us what kind of problem and what kind of tools we're going to use.
Just who gets to choose the strategies
Right.
Right.
It's very, very general, totally not specific.
So what could we do?
I mean, we don't know what this instructor had in mind, but what could you do if this was your class?
What would you say?
What about develop instead of provide?
Develop.
And again, it would have to be student centered, so the student will be able to develop problem solving tools and strategies.
That's for x, exactly.
We want to make it a little more specific and a little more focused, because that's pretty broad.
I mean, I have developed a problem solving strategy that lets me get out the door without tripping.
Does that mean I satisfied your learning outcomes?
Probably not.
So in order to do something-- Use thermodynamics to solve engineering problems
That's not specific as well
It's not specific at all
Engineering is [INAUDIBLE]
Yeah.
I mean, it's crazy broad on a number of fronts, right?
I mean, it's like engineering is huge.
Thermodynamics is pretty darn big, too.
So which part of thermodynamics, which part of engineering?
Build an SAE race car
[INAUDIBLE]
What?
It's not realistic
Right.
It's pretty specific.
Build a functioning SAE race car.
It's very specific, and I can totally think of the metric of how I would measure whether students could do it.
I would just ask them to build the race car.
But they're never going to do it in a class, in one class period, in one course.
It's unrealistic, exactly.
Learn to use Laplace transforms to solve differential equations
[INAUDIBLE]
It's funny.
I mean, you can imagine the instructor saying you will learn to use that.
But let's go, Katherine, work your magic with this one
[INAUDIBLE]
Right.
You're my go-to person for when you want to get rid of all the extra words.
I'm going to call on you.
Right.
Use Laplace transforms.
That you can measure.
You can give them an equation and see if they can use Laplace transforms to solve it.
And yes, you might want to be a little more specific about the kinds of differential equations they use, and know how to upper diagonalize a matrix.
Katherine
.
Upper diagonalize a matrix
Upper diagonalize a matrix.
That's all you have to say, right?
Forget the extra words.
And that is really, I have just written the assessment question for that.
I know it's totally measurable.
I give you a matrix and I ask you to upper diagonalize it.
I haven't given it away.
I haven't told you you will upper diagonalize this matrix, right?
Memorize these equations that you're going to need to write down in order to upper diagonalize it.
But I have told you that that's what's expected of you
I have a comment on it.
It's pretty useful when a student is trying to evaluate the use of the class.
For example, the career fair tomorrow [INAUDIBLE].
There's a million [INAUDIBLE] to get jobs.
OK, how are they going to evaluate me?
And if you could come in and say, this happened and this is what we did, as opposed to having, or have them evaluate me on some random knowledge.
And then I don't know that I could respond, but I do know stuff.
So this is really helpful in kind of a broader sense
Right.
Great.
No, I'm glad.
And I think it's so true.
And people often think, well, you're giving it away.
But you're not giving anything away.
You're just telling them the expectations.
You're not giving them the problems, you're not teaching to the test, but you're just telling them what they're going to be able to do at the kind of high level, or medium level.
You're telling what they're going to be able to do.
I will say-- so ABET, the Accreditation Board for Engineering Teaching-- every accredited engineering program has to list, they have A through K outcomes.
And if you look it up, ABET-- and I'm sure you see us as accredited-- if you look up, they have to, they used to do a lot more bean counting.
But what they would do is the outcomes would be critical thinking and problem solving skills, and they were a little bit broad.
But every department that got accredited had to show how various courses contributed to the outcomes.
So somewhere in your department should be a pretty detailed list of how 6 double 00 whatever, did this.
Or 6 yada, yada-- I know your grad students-- but did this.
The problem is they don't have to do it for grad student courses.
But it might be helpful if you looked at that, because I would imagine that sometimes the recruiters would be looking at that information, so that might be a commonality.
But every program in the US that's accredited has to have, show at least how they've addressed those outcomes.
So you might want to take a look at that.
But yeah, if it was a little more explicit it would certainly be better.
And I'm glad it's helpful.
So I think we can see, you can start to see how there are better and worse learning outcomes, that if they're not measurable they're not so useful.
If they are measurable, they're quite useful.
So now what I'd like to do is revisit the worksheet.
And what we can do is we're going to take about five minutes where you just sit quietly and write out specific intended learning outcomes.
So you'll be able to upper diagonalize a matrix, you will be able to use Laplace transforms to solve this type of differential equation, whatever you've written in this column.
And note that if you put it in this column, let's say you put it in the apply column, then when you write the learning outcome it's going to be you will be able to interpret data using blah, blah, blah.
You will be able to model the vibration, model a car suspension system using linearization techniques.
They're going to be whatever, wherever you put it in the box.
These are some of the verbs, these are the verbs you're going to use in your learning outcome.
Does that make sense?
And I have even more verbs for you if that's helpful.
Did I pass those out?
You got those, right?
Yeah
So there's more verbs.
It's the same thing.
I mean, it's consistent.
So take five minutes by yourself and do that.
And then you're going to get into pairs and you're going to discuss.
You're going to trade worksheets and give each other feedback.
All right?
So five minutes of quiet and then we'll pair up.
So if you can pair up, you don't necessarily have to pair up with people that are sitting next to you.
You might want to get a different perspective, so feel free to get up and move around.
You can talk to somebody that wasn't in your group.
So maybe if you go back and talk with Alex, maybe.
We need one group of three.
Oh, are you a group of three?
So now just trade sheets of paper with your partner, or if you're a threesome you can figure out how to do it.
And then review the learning outcomes critically.
Guys, just one sec.
So just review your partner's learning outcomes critically.
Make sure that they're measurable, realistic, and specific.
If your partner has used the word understand, help him or her work through that to get a better word.
So you're going to be a critical friend here.
And then you guys can discuss how to improve it.
And then I'm going to pass around some flip chart paper and each one of you will write, maybe just write your name at the top, and write two or three learning outcomes on the flip chart paper.
And then we can stick them up around the room, and then everybody can see everybody else's, all learning outcomes.
[INTERPOSING VOICES]
Was there a question?
Oh, no.
I just think--
I think [INAUDIBLE]
How are you guys doing?
Doing OK?
[INTERPOSING VOICES]
So you can use this zone if you need it.
As you're looking at other people's learning outcomes, think about how clear, how easy it would be, or not easy, to come up with an exam question, or a project, or an assessment or measurement of whether or not the student reached, attained the learning outcome.
You can put them up here, Ina and Shau.
OK. And as I said, make sure that you've read everybody else's.
I'm going to say a few words about them.
I think we may have a record.
This is like the first time ever that no one has used the word understand, which I think you're all to be commended for.
So David, make sure you don't write understand.
You build me up just to knock me down
[INAUDIBLE]
You can, or you don't have to.
It doesn't matter.
Does anyone have any comments, either about the exercise itself, or about somebody's learning outcomes, that you see around?
Again, the measurement, the thing you want to look at when you look at your own learning outcomes and those of others, is are they specific, measurable, and realistic?
And would you know if a student achieved those learning outcomes?
So I would say about 90% of these are very specific, which is great.
I mean, there's some incredibly specific ones here.
The more specific, the easier it is to measure.
So that's one thing.
That's a useful thing.
You want to make sure you don't get so specific that there's only like one question you could ask.
So that's the balance.
Sometimes you can't tell that unless you're in the discipline.
Do we have other comments about any of these learning outcomes, something you don't get, something that looks particularly interesting?
Yes, Gordon.
Perhaps you can sit down if you're done reading
I'm looking around and I'm seeing [INAUDIBLE].
I don't know how it's going to be.
How are we going to test that?
Well, I guess you have to come up with a reaction that you're confident they haven't seen before.
So if you can do that, then you can ask them to propose a mechanism for it, I suppose.
If the person that wrote it wants to-- I know who wrote it-- but if the person that wrote it wants to talk a little bit about it, that would be great.
If not--
OK, So a lot of people are going to just try to memorize each specific outcome.
But it's easier to just identify your activity group and find patterns within them.
So if you can try [INAUDIBLE] They can't just [INAUDIBLE]
Right.
And this is a great point.
I mentioned it to one of the groups.
But sometimes we'll say explain how blah, blah, blah, works.
And that's a great thought, that, oh, they're going to be able to explain it, right?
But many times they can fall back on a memorized explanation or sort of kind of a canned procedure.
So if you really want them to be at the level of explain, you will have to kind of make some things up to get them out of, to get them away from the ability to just pull something completely from memory.
So this is an example of how you might do that.
Other observations or questions?
I also [INAUDIBLE] something that all [INAUDIBLE]
So perhaps the person that wrote it wants to comment on it
OK.
So [INAUDIBLE].
I think what we have, just [INAUDIBLE].
So we write and compile.
So you write a code and then you compile it
Right.
And this is a great example of how having people that aren't in the field read them can help clarify things for your students, who are by default not experts.
So perhaps to an expert, it totally makes sense, of course, code.
But to somebody that's not an expert, maybe that's confusing.
Maybe the use of code twice was confusing.
To an expert, it's totally readable.
It's totally understandable.
Perhaps to a novice it isn't.
You're laughing
It's funny, because we're both [INAUDIBLE]
Yeah, that was kind of random.
Well, that's a perfect example, right?
It's a perfect example of how getting maybe other people that aren't in your field to look at them can help make them more accessible to your students.
OK.
This is great.
Oh, Hina
I had, it's on a different topic.
But really, I'm talking about buzz words in learning outcomes.
I think with engineering, you're really excited about telling people about real world problems.
I guess that's the reason why.
I guess it's more of an opinion question.
As you write your learning outcomes I think in some ways it's better to be specific about the real world problems that they're going to attack.
Because I've seen that on syllabuses all the time.
I'm very excited about it.
I think it's a way to teach people in your class.
It's going to help them know what tools they're getting
Right.
I mean, real world problems in particular really do kind of encapsulate a whole pile of hopes and dreams.
So it may help to be a little more specific.
You can use it in your learning outcomes, but then maybe describe it in a little more detail in your syllabus, the kinds of real world problems, or why we care about real world problems.
They're messy.
They're ill-defined.
What about them is so important?
What about them is so important?
Try to minimize the buzz words.
But at some level, they may not know about something until they're completely through with the course, and you may have to use that word.
So there's a bit of a balance.
What I'm going to do before, I'll probably bring all these back to my office, take pictures of them, and put them up on the wiki so you'll have a record of them.
I wanted to just as we go out here, my point is really that the hard part is these learning outcomes.
And that once you write the learning outcomes, as we said before, you kind of know how you're going to test whether students got them, and you know what you're going to do to help them get them.
The syllabus itself is just an articulation of that.
It's a description of the course, an articulation of the learning outcomes stated, like the laundry example, with the title.
This is what you're going to do.
By the end of the class this is what you'll be able to do.
It's kind of a promise that that's what you're going to do.
It motivates students to take the class, to stay in the class, to engage with the class.
It keeps you on track.
So at the beginning of the semester maybe I'll be very clear, but four weeks in, and you're a little bit fuzzy, because you've been dealing with all sorts of stuff.
And you kind of go, what the heck.
What's this in here for?
Well, if you've lined everything up at the beginning using your learning outcomes as the anchor, then you don't have to worry about that.
You just follow the map.
So it really, really helps you and the students get through the semester in a logical way.
And it tells the students what they can expect of you, what you can expect of them.
So it's motivational, structural, and it's evidence, hopefully of what they did, but certainly of what you hope to do with them.
So for the post-session assignment, you'll have to write up more learning outcomes from your course.
You can use these as the basis for sure.
Hopefully, that table will help you as you write more of them.
And I will go in and give you feedback.
I want to make sure that everybody knows that you should read each other's postings.
So a lot of you had great examples from the first class about how you would use these learning theories in your courses and how you teach.
They were fabulous examples.
Make sure you read those postings of others.
That's one reason why it's up on a wiki and not just you handing it in to me.
So you have the Mud cards.
Fill them out if you have any more information that you want.
And otherwise, we'll see next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
A general comment is by and large, I thought you guys did a great job.
Your learning outcomes were for the most part specific, measurable, and realistic.
The biggest problem I think I had was with-- the biggest problem that you had on average was the idea of the learning outcomes really have to be measurable.
So if you use a word like appreciate, it's not measurable.
So that's something to remember.
If you use words like discuss, it's potentially measurable.
But you need to make sure that you set up a venue for students to discuss.
You can't just say you'll be able to discuss that and assume that they'll be able to discuss it, That If that's in your learning outcome, you have to set up some venue for them to discuss and you to know what they're discussing.
And then the two biggest things that come up, or the two most problematic words that come up-- and I compliment you all, because you did a great job.
Nobody used understand.
So that was awesome.
But the other thing, there's words like explain and describe, which are perfectly good words to use in a learning outcome.
But you have to make sure that when you say, students will explain the ideal gas law.
They'll be able to explain the ideal gas law.
Well do you know that they're really explaining it in the true sense of that word that they actually internalized what it means, and they're really explaining it?
Or are they just spitting it back, spitting back, maybe, what they read in a book, a definition that they read in the book?
So you have to make sure that you've set up a situation wherein they really are explaining.
And they're not just remembering.
So it's a great word to use.
But you have to make sure that we're using in the right context.
Same thing for describe.
You can imagine that has the same issues.
Describe da, da da, da, da.
Well are you sure they're doing the describing?
Or are they just remembering somebody else's description and writing it down?
One good way around that is to tweak the parameters a little bit.
To say, in a world where there is no oxygen, what do you expect this organism to look like, given that it has to run really fast or something?
So come up with alternate realities that that actually force students to transfer and then re-explain.
That can be helpful.
That can be really, really helpful.
I remember once in a kinetics class, we had been deriving the critical nucleus size for growth.
And it's all based on a radius, a critical radius.
And then on the exam, the guy said OK, they don't grow as sphere's.
They grow as rectangular boxes.
So then all of a sudden, we had to look at the ratio of the length to the height of the box or something.
So that really wasn't just we were just remembering a solution or remembering a method.
We had to actually think about what it meant and then reapply it.
So just think about how you can get students to really describe, explain, derive even, because you can memorize a derivation.
But if you really want them to derive it, what does that mean?
How are you going to test that?
So those are generally the biggest problems people have with learning outcomes.
But by and large, you all did a great job.
If you came in late, I'll give you your learning outcomes at the end to class.
If you looked at the Wiki-- and this is a test to see who looked at the Wiki-- I had suggested you bring a web enabled device if you had one.
And that's because I wanted to set up something called Back Channel The topic today is on active learning.
And so this is just one method that might help your students, especially if you have a big class.
So we don't really need it in this class, but I just wanted to give you the opportunity to play with it a little bit.
It was designed for conferences actually.
But it works pretty well, I think, in classes.
So if you login to here, this is the URL.
If you go to our Wiki page bookmarked, there's a link on the Wiki page, if you just want to link to it that way.
And then at any point in the class, you can type a question in.
And if you're on the page, you can see other people's questions.
You can upvote and downvote their questions.
And I will take a break in the middle, when you're doing some other things, and read what people have posted and then get back to you.
So it's an opportunity for students to interact who may be a little more shy, who may not want to.
I don't think we need that in this class.
I don't think that there's any one of you that isn't willing to share your ideas, which is great.
But this may be something that's of interest to you.
Before we jump into the topic today, active learning, there were a couple of mud card questions that I just wanted to talk about.
One was sometimes students may nod off during class.
And if it's a big class, there's probably not a lot you can do about it.
If you do enough active learning, though, that will keep them-- that will make them wake up.
Especially pair activities or activities where they have to talk to one or two other people, or two or three other people, it's pretty hard to be asleep during those.
And you'll see.
We're going to do something called a lightning round later on.
And it's virtually impossible to be asleep during that.
So you want to make sure that it's worth the student's time to actually be in class and be awake for it.
And active learning can play a huge role in that.
Sometimes people are tired.
They stayed up too late for whatever reason.
And you can't control everybody in your class.
But you want to make sure that you're doing everything you can do to keep them engaged.
And if it's just one student every now and then, you probably don't want to take it personally.
It's probably not you.
If the whole class is falling asleep, then it's probably a good opportunity to rethink your strategies.
Somebody asked if there were good times of day to have a lecture.
And part of it is you.
Part of is your schedule.
Sometimes you're allowed.
You can pick that.
And sometimes you really don't have much control over when you choose to teach your class.
People do say that students, undergrads, people between the ages of 18 and 21 are not particularly interested in being up at 8:00 or 9:00 in the morning.
But again you may not have any control over that.
Generally speaking, that's a pretty low time for people.
So if you want to take that into consideration, you might.
But it depends as much on you as the students.
If you're going to be really tired at 4:00 but they're just waking up, then don't teach it at 4:00.
Maybe don't teach at 4:00.
And the last one was guiding readings.
If you just put a lot of readings up, students might not know where to focus their attention until after the class.
So there are reading questions, reading guides that you can post.
Read such and such and try and think about how you might explain this, that, or the other thing.
And then there's also pre-class questions you can ask that they might derive, that they may have to pull from the readings to answer.
So the concept of just in time teaching, the idea that students do a reading and then they actually have to answer questions on the reading before they come to class and submit their answers.
So there's a whole procedure around that.
It can be very, very effective.
You can do it for a large class.
There's strategies and techniques to streamline the amount of time you put into it.
But it does force them to do the reading.
And it does guide, at some level, what they read.
And that's a great great approach.
And I'm happy to talk more about that at another time or later or offline.
I did go to the Teaching with Technology conference, the Teaching Professor Technology Conference.
And they were giving out little ribbons to stack on your badge.
And this was a prominent one.
It's in the syllabus, which I thought was just crazy timely because a couple of weeks ago.
So if you want one of these to stick somewhere.
And then they all said when I tweet, which was sort of weird.
But anyway I brought those back.
Anybody want some.
Sometimes my computer doesn't wake up.
So let's pretend that instead of the class active learning today, I decided I was going to teach you how to juggle.
And I said, here's the equation for how to juggle.
And here's the variables, F, D, V N, and H, which are ball in the air, ball in an hand, time hand is vacant, number of balls, number of hands.
And this is this handy dandy equation, which if you balance it out, it describes the juggling process.
And for those of you that are visual learners, I have a graph here, a yellow ball, a blue ball, and a pink ball.
And then you can see that just shows the path of the balls.
If you remember from physics, there's the equations of motion in the x direction in the y direction.
And so I think I probably did a pretty good job of teaching you how to juggle.
No, did a terrible job of teaching you how to juggle.
I didn't even come close to teaching you how to juggle.
And if I were to give you a test, let's say, and I were to measure whether or not you could juggle, I guarantee that some of you would be able to juggle.
And I could say, I did a great job teaching how to juggle.
But I didn't.
If you knew how to juggle coming in, presumably you still know how to juggle.
I did no harm, I think.
However if you didn't know how to juggle coming in, you don't know how to juggle now.
So why is this here now?
Why did I bother to do this stupid thought experiment right now?
Because this is clearly how most classes are taught.
Obviously it would be easier to have a ball and show us how to juggle, active learning as opposed to just showing the theory behind it and ending there
And I think a lot of times we really do, in science and engineering classes, we really put up a lot of equations.
And then it's not like we expect somebody to do something physically with them.
But we expect them to be able to manipulate the equations, transfer the equations, apply the equations.
And we never really talked about that.
We talked about the equations.
So it's a real call in my mind, for us to think about what it is we really want students to be able to do.
And then make sure that they're doing stuff to teach them that helps them learn how to do that.
And I know you all read the readings and that they talk a lot about active learning.
There's a few things about techniques for active learning.
There's a few things about the why's of active learning.
There's some data for why active learning works.
But I'm guessing that there's a range of buy in at this point from you guys.
Eh, I don't think it's worth it.
Or I don't really know.
So for the first 20 minutes, 25 minutes of class, what I'd like you to do is I'd like you to pretend that you were part of a department, the STEM department.
Call it whatever field you want.
And there is a proposal that your department should commit funds to train professors and develop resources, the resources necessary for the adoption and ongoing use of active learning techniques in all courses.
And the target date is 2017.
You've got a year to get all courses up to speed with active learning stuff introduced into these classes and with a plan for how to do that.
So what I'm going to do is I'm going to just count off one, two, three, one, two, three.
And I'm going to ask you to form a group.
If you're a one, go with the one group.
Two go with the two group.
A three go with the three group.
And you're going to have to physically move.
So just remember your number.
One
Two
Three
One
Two
Three
One
Two
Three
One
Two
Three
One
It doesn't matter that they're not the same size.
Can we get all the ones, let's say all the ones up here?
All the twos here.
And all the threes there.
And I'll tell you what to do in just a sec.
So if you're group number one, you have to adopt the position that you are pro.
You are going to push for the adoption of this resolution.
And you have to come up with a set of arguments and a set of reasons why it's a good thing.
I don't care what you think personally.
I don't care if you think active learning is the worst idea ever.
For the purposes of this debate, you need to pretend you're totally into it.
And you need to come up with some good arguments why.
Two are against it.
Again, I don't care how you feel personally.
But come up with some good solid arguments why you should not do this
We are against it?
Against, you do you not want it.
No.
Group number three, you got a little bit of time.
But get to know each other.
And you're going to hear the arguments.
You're going to hear a case presented by the pro people, the con people.
And then you are going to deliberate and decide what's going to happen in your department.
You guys are the deciders.
It's an American political joke.
But we'll let that go.
And you guys get this lovely microphone.
So I'm going to give you 10 minutes for this.
So be ready to share your views.
You've got about, maybe, two or three minutes of a spiel of proof.
And then we'll do one exchange of debate.
And then the deciders will decide.
[SIDE CONVERSATIONS]
I'm going to ask someone from each of the groups to make a statement that puts forth the side that you're supposed to defend, if that is the side you're supposed to defend.
So we'll start with the pros.
So someone from your group ready to talk pros?
All right, Michelle
So we are pro active learning.
And part of the basis for thinking that active learning's important is that we believe that without active learning, the professor usually just resorts to lecturing.
And we think that lectures are not effective.
Students don't learn simply because the material is presented to them.
Lecturing wastes resources, because you could be lecturing to 1,000.
But only 15 students may be reached.
And if you use active learning instead of lecturing, it can make sure the students are paying attention.
You can check that they're understanding.
You can know what they are getting and what they're not getting.
We also feel that active learning is a life skill.
And that if students learn to learn by do things and they learn by talking to each other and by going through a process and solving problems, that they can take this skill to outside of the classroom later in society.
Then that the society that you have students who are used to being active learners and who know how to solve problems that way.
And we also think that the end, the big point that we have is that you remember things that you use.
If you learn something, if it goes in one ear and out the other, if you don't actually use it, you're not going to remember it.
If you're not forced to actually work through a problem and really think about it and use it in a classroom, that you're much less likely to actually hold on to that information.
So these are all reasons why we think that we should promote active learning in our department
OK, thank you very much.
The con, or I guess I should say the not whatever.
Anyway, group two
Do you want to do it?
OK. We start with a basic question.
The question is, why do we want to change what is working?
We are training our students.
They are getting job.
The professor are doing well.
They are collecting grants.
The original [INAUDIBLE] is OK.
So why do you want to change what is working?
So that is the first question.
Now the first thing is that, if you want to go about this, it's going to consume a lot of money.
It's going to take a lot of money.
And then it cuts into your funds.
The one funds for other such, want to buy equipment for the laboratory, want to go hire scientists, want to go hire post doc.
So why do we want to change?
Where do we get these funds?
We divert funds for the training of professors.
And they're going to be little bit of no funds for all this such work.
And secondly, the efforts to train professors is likely to be very huge.
Professors are used to this method of teaching.
And as you saw, they will not want to change.
There's going to be a lot of time changing them.
And we're not even sure, if we try to change them, whether they're going to be able to do it at the end of the day.
Thirdly, we are not sure that the stakeholder would be happy.
What about the alumni that are very much used to the method of teaching?
What about the parents?
are they now going to go against us?
Will they not see that the professor no longer, just come to the class, give them group work like this without teaching, without writing?
Then the professor is no longer teaching.
And they are just going to collect their salary.
And then, there's not to be constrained on time to cover syllabus.
When I saw I spend a lot of money being in group work, individual, or this and that.
So how do you complete your syllabus?
You have about 20 topics to cover this semester.
And by the time you start active learning, how are going you cover about 50% of them?
Then you push all of that up [INAUDIBLE].
So it's not going to work.
We should just drop the idea.
What have we value very last class?
You have your class of 500.
How do you want active learning to be effective?
How do you want to treat them?
It's good here, because we are just about 15 or less than 20.
By the time you have 500 students in a class, it's going to be very difficult for you to have active learning, difficult for the professor to control classes, and of course, difficult for you to really get something concrete out of the whole thing.
So my final guys and professors, I think it's not necessary.
the.
System is working.
Let it continue to work this way.
Thank you
So now what I'm going to do is I'm going to let each group ask one question or a set of clustered questions to the other group.
The other group can answer them.
And then the deciders can ask a couple questions.
And then we'll let them deliberate.
So group number one, like to ask some questions to group number two or make some comments?
Go ahead
Yeah, I listened to you speak.
And I want to ask you.
You are saying that the system is changing.
It is changing because the status quo isn't delivering.
So that is why we are adopting active learning.
And this is proven from a research perspective.
There is actually letting more students learn more materials faster and more effectively.
So what do you think about that?
I'll answer it by saying the different research shows us that the student that is good is good.
[INAUDIBLE] So why can't we just concentrate on getting those students?
And what your presentation didn't say about funds, was the whole idea is to fund the proposal.
And if we divert funds to this new program, and that's a very big issue.
You have to concentrate on the question that is the diversion of funds away from research.
So if we can get the new students and they can learn by [INAUDIBLE], and it's working great-- go ahead
So you want to throw away the baby with the bath water?
If you have 20 students in a class, and you have five active learners, and then you want to discover 15 that wouldn't.
This program is geared towards each one of the students in the class to participate.
And I think it's more cost effective.
If you could get all the 20 students to learn and to learn actively, I think that would pay itself off at the end of the--
No, the problem there is that you have to face the proposal.
This is 2017.
It's a lot of change in a very short time.
The proposal itself is already against--
Our department could be three people big.
We don't know how much the department is.
[INTERPOSING VOICES]
But then the issue there is that you're going to retrain professors.
Professors are really stuck in their ways.
And you spend all this money.
And it's very short time.
There's lots of people from the labs, and money from the labs in this very short time to train them on something we don't know if they're ready to buy into it
So I think your assumption is that-- you're like, we're deferring funds from labs and resort.
But you never once said you're diverting funds from anything related to the students.
You're thinking of it from the professor perspective.
But really professors are here to teach their students.
And you're like, oh, we're taking money away from the research, from the lab equipment, from all these different things that sound great to the professor but have literally nothing to do with the students learning experience or the student's experience on campus.
And from the student's perspective, that's really frustrating, when that's all the professors care about.
And that's what makes a difference between a good professor and a crap professor is one who cares.
And I think, if you want a professor to care, oftentimes they're willing to make the effort that is needed to change their teaching style or do whatever it is that's necessary to have their kid interested and engaged in the same material that this professor loves.
So you keep talking about diverting funds.
But you never once mentioned diverting these funds away from this one teaching initiative to a different teaching initiative.
So just keep that in mind
So the problem is that when we are looking at the proposal itself, the time, 2017, this is already the end of 2015.
So that means you are going to do this, by 2017, that means you're going to do it by 2016.
We're not against active learning in the [INAUDIBLE].
[LAUGHTER] [INTERPOSING VOICES]
--the proposal.
We are against-- we need these two group and against this proposal.
So for this proposal, we are against this proposal, because you know how much it's about the funds and the resources.
And that is why we are against the proposal.
And also you pick whatever [INAUDIBLE], we're not even sure that it is going to work
I'm going to take, because you guys actually did ask a question, so I'm going to count that to you Did you have another separate one?
AJ, you look like you might have some questions
Yeah actually I was just curious, so if this initiative is really great, it looks like a lot of this research has been done since '80s.
Why isn't it adopted?
Why are we having this conversation now in 2015?
I think we should look back at this issue of the lecturing system that we are using now.
It's static.
When there was no printing, lecturers then, they get the material.
They reach the students.
So lecturing is a Latin word that means reading.
Now since then, now we are printing.
We have information readily available that people can get on at their home and their own and develop to something.
Too So there is a need for us to change to a new system, because the old system that more facilitated lecturing has changed.
So then we have to change
I'm going to turn it over to the deciders who now have to discuss.
They've been resting for many minutes.
So now they have to.
So you have questions for each of the groups?
Yeah
Yeah, thank you.
Great arguments on both sides.
I found it interesting, when you talked about the larger classes, so my first question is for this group.
So how would active learning apply to a large class?
The proposal is that it's for all courses.
And so are there courses where this doesn't lend itself to?
Even in the classes with 700, 800 students here, they still do active learning for the freshmen, i.e.
Clicker questions.
Oftentimes professors have to talk to your partners.
So they'll pose a question.
Everyone will click on their answer.
And then they'll be like, now go back and talk to the person sitting next to you.
And then you'll vote again in two minutes.
So I think that's just one example of how you can engage a class of a lot of students
And to add to that, actually it's also been found out that when peers speak to themselves, they even learn the material better than when the expert, who may have lost touch to what it could be like to learn from a student's perspective.
So peers speak to themselves and teach themselves even faster than the expert who is in front of them
Yeah I have comment for the against group.
You mentioned that you don't require to create or active learning, because some students are very smart.
They get it right.
But don't you think that education is a right of every child?
And they deserve to have a good shot at it.
So why do you want to pull one child out of here, because he's not smart enough?
I just to answer, I think we are pushing for the best education.
We believe that is the traditional way.
We think that active learning can happen outside of the classroom.
This is the way it's worked for 300 plus years over at Harvard, one of the best universities, MIT for 100 plus years.
We think that essentially it's working.
And the reason we're having this conversation right now is because it has worked.
So we want to continue that legacy.
We think that every system is going to have problems.
And if you analyze any system, you're going to find issues.
And there are certainly some smaller inefficiencies within the lecture.
But by and large, it's trusted.
And it works
I want to add on what adding.
So I think we are not pushing those less smart kids away, because they can actually learn from each other outside of class.
We just want to say that doing the lecture is more efficient if the professor lecturing at them, giving all the information.
As a footnote to that-- sorry-- if professors come to the class and just give theory and go away, and the smart kids get it, then how do you measure the professor's performance?
How do you think about, say, something like students information on educational quality?
How do you measure that?
You don't really need the professor to be extremely active and do active teaching in the class.
The full student can actually get what they need.
You have a bunch of homeworks.
You have a bunch of test questions, exam questions, that student, whether they like it or not, they will surely be forced to read those things.
And by the time they come back, they come back learning.
Lot of graduate from MIT, they are l really doing well.
About 25% start up their own business.
Probably like 70% get jobs when they finish.
What are you talking about?
The system is working.
And That's what we're saying.
And their value is a great.
The evaluation is done on the type of textbook, the mode of lecture, the [INAUDIBLE] lecture.
So these are the things that I try to evaluate in the evaluation process.
And it has been great.
And we want it to continue
So we addressed the measuring impact.
And that goes for both.
But for the active learning, how do we quantify how much active learning needs to go into these different classrooms in the department?
So how will we ensure the appropriate amount has been implemented in different classrooms?
Basically, how much active learning needs to go into a classroom to create change or to strengthen our student learning?
How much has to go in or how you're going to make sure that the professors are doing it?
Both, I guess, is the question.
If there's a professor that's more on the lecture side that might throw one thing pair share in and call it a day.
So how do we ensure the appropriate active learning techniques and the quantity are there for the classrooms?
I think we'd have to go back to the research and see how much, whether it's 30% of the time needs to be spent on active learning activities, or to see what has been proven, because active learning has been proven to be effective.
But we would have to go back and make sure that we set a standard that, if it is 30% of the time or 70% of the time.
And I think oversight from the department would be important, in that the leadership in the department will have to sit in on classes.
And as part of the training of the professors, there has to be training.
But then there has to be someone to observe them in the class, give feedback.
The training has to be ongoing.
It's not just a three hour seminar on this is with active learning is.
And this is what you should do.
I think it's going to be a long term investment in the professor training that someone's going to have to check up and give them feedback.
And then they can adjust from there
I'm sure that we could [INAUDIBLE] all day.
But I'm going to give the deciders two and a half minutes, three minutes, to talk to each other and come up with a decision.
All right, the committee has met.
And now they're going to determine your fate in this department
Does this mic work?
Is this on?
Do I need to use the mic.
You can hear?
Well thank you again for your arguments.
Certainly the panel felt that the pro arguments in terms of the benefits of active learning are great.
And we definitely support that.
However, given the time frame, and this proposal is very aggressive, it would be extremely disruptive.
Also the amount of funds that we think that it would take to implement this would be very high without any proven way to measure success.
So what we are willing to do is commit funds to a pilot program, where we can then measure what the success will be and then make it a decision next year as to whether or not we invest for the rest of the departments.
[INTERPOSING VOICES]
So I hope you found this exercise useful.
I just want to debrief on the exercise itself, why we bothered to do it.
It took probably 45 minutes of time to do it.
But I made a decision that would be useful.
So what did we get out of this exercise?
What was the point?
Whether you think that's it's worth it or not, it's really important time.
Also, it seems this confrontation is happening a lot in the department, especially [INAUDIBLE] as they go through the program.
I assumed this test would be debated quite a bit, internally and externally
Other reasons we engaged in this?
I think it gives everybody the opportunity to think deeply about the subject matter.
And it also, even if you are against or for something, you have to have a deeper knowledge to be able to bring out the good points, certain issues that might actually make see [INAUDIBLE]
[INAUDIBLE], you want to add to that?
The topic is active learning.
And what we just demonstrated is active learning in practice
Yes, exactly.
And the idea to Gordon's point, the idea that it can be very, very useful for you to have students take an arbitrary position and argue it, because they will learn things, they will think about things, that they would not have thought about otherwise.
And it's usually not a good idea to say OK everybody that thinks this, go over here.
Everybody that thinks this, go over here, because all that does is reinforce what people think already.
And maybe it's not completely correct.
So this is often a very good technique, when there's two clear viewpoints or two clear sides.
And you want students to really engage with that.
And exactly, we could have had a discussion.
Or I could have had PowerPoint slides that said, here are the good things.
Here are the pros with active learning, bum, bum, bum, bum, bum.
Here are the cons of active learning, bum, bum, bum, bum, bum.
And that would've taken-- just to come out of my neutral position here-- that would have taken probably four minutes.
Instead we took 35 or so, 40 minutes.
But I think you probably engaged with some of the ideas and concepts a little bit deeper, because you had to come up with it yourself.
So yes, all those reasons.
And it does come up often.
People are discussing it.
And it's not so clear cut.
There's not a right answer.
And you're going to want to say, OK, well we'll do it by 2019.
Or we'll do it.
But we won't do that.
There's going to be deals and all those kinds of things.
And that's OK.
It's OK that there's not a right answer.
But that was the point of this exercise.
So that served as our discussion for active learning.
But you guys did it.
I didn't do anything
And that buttresses the point that active learning is the way to go
Thank you.
Yes, this is very meta.
This is very meta here.
Yeah and I do have to feel a little bit bad when you're like, what?
Then we're going to do all the work.
And the lecturer's just going to sit back and not do anything.
I took it personally.
But no, no, it's OK. One thing I wanted to bring up is if you haven't read the Scott Freeman article, it's in the reading list for today.
It shows-- let me just bump to it-- it shows essentially these results.
He did a meta analysis of 225 studies on active learning or on classes that that used active learning.
And they showed a 12% decrease in the failure rate in classes that used active learning.
And so I was telling the pro group that what say in the paper is that if it had been a clinical drug trial, and 12% of the people on the drug were having a better outcome than the people on the placebo, they would have had-- or the people on the control-- they would have had to stop the trial and give everyone the drug.
So in the sense of why we're still lecturing, that's to me one of the biggest compelling reasons for why we should be doing more active learning.
12% of the students are going to pass the class, on average, normalized.
And they actually considered both experienced lecturers, who were just lecturing, people that were considered to be good lecturers that were just lecturing, as well as graduate students who were first time teachers but were using active learning.
So they looked at all sorts of-- they mixed up all sorts of outcomes.
And active learning appears to, the data's pretty conclusive.
And this isn't the only study that shows it.
So there are definitely issues.
There are definitely implementation issues.
They're definitely sticking points.
But it's something that I think we can't ignore
I just want to add that it's not only that we have 12% decrease in the failure rate.
We also have big shifts in the letter grade students get.
For instance, more people shift from grade C to grade B or from B to A.
And B to see a test like that.
So that's really significant also
So I just want not have learning outcomes.
But I did want to have the discussion before I even introduced them.
So I think you'll be able to explain the impact of active learning exercises in the classroom.
You've already started to do that.
And then you'll be able to identify and develop active learning exercises that you'll want to use.
And you'll be able to think about how you might use them.
I think, pretty attainable learning outcomes.
I'm going to show the slide a lot throughout the rest of the time.
But remember, it's constructive alignment.
We've seen this diagram before.
But the beauty of some of these active learning techniques is that their formative assessments of students' understanding, of students' learning.
And so they are actually activities that happen in the class.
But they're also measures, informative, formative measures of whether students are learning or not.
So if somebody had come up with a really terrible argument, or it was clear that nobody had read any of the papers on the active learning research, then I would know that from this debate.
I know what you've read and what you haven't read, and what you're thinking about and what you're not thinking about.
And I get a good sense for whether or not it you know it and how much and how well you know it.
And so do you.
And we'll see that a little bit more in these other situations, where if you get a clicker question wrong or you get a multiple choice question wrong or that you can't participate in the lightning round, then you know that you've got to step up to the plate.
Or you need to try something different.
Or you're not really getting it.
So you get information.
The learner gets information.
And the teacher gets information
I just thought that was really powerful in the readings, as well.
They mentioned that, that students who, when they get in the group sections, if it's clear they really are getting a lot of things wrong, they know they need to step up.
And that's in a different way, I think, than getting a bad grade on a paper.
And you know the average distribution or whatever.
Or the professor says you're wrong.
But when you know that the people sitting right next to you are getting it and you're not, that really, I think, will push the student forward to do something
It's news you can use.
It's information.
And then you're not being penalized.
You're not in trouble.
It's not affecting your grade if you act on it, if you take the information and use it.
So that's why, I think that's the beauty of it.
There are many, many active learning techniques.
And if you looked at the website from University of South Florida has just, I think, it's close to 200 techniques for what you might do.
I'm doing a bit of a-- I call it the foie gras method.
You know how they make foie gras?
They force feed the goose.
So I'm going to give a lot.
And we're going to do a lot of different active learning things in this session.
It's a little bit comical.
I know it's kind of funny how many we're trying to cram into two hours.
But I'm going to try to do that.
So we'll do some of these.
And we can talk about any one of them in particular.
But a nice one to get students in the frame of your class is to they come in.
And they have five minutes before you do anything to write down what they thought was the most interesting part of the reading or answer a certain question related to the reading or solve a problem or whatever it is.
You give them a prompt.
You sit down.
You focus them.
And then they're off.
I do want to just focus on the word active for just a second.
It may help you to think about active and interactive learning.
There's all sorts of ways to chop it up.
And at some level, it's semantics.
But at some level, it isn't.
So students can be active if they're sitting by themselves thinking about something and writing something down or solving a problem.
That is active learning.
They don't have to be talking to each other.
You can do active learning in a class of 3,000 people.
Stop and think.
Answer this question.
That's active learning.
That's way more than happens in a lot of classes.
Interactive learning generally implies that students are talking to each other.
That's the most general categorizations you can get, active, interactive.
So we're going to do a bunch of stuff with clickers or plickers, these things.
But there's some other techniques you can do.
You can put up a graph or an image.
And you could say, OK, somebody come up and show me the green boundary in this picture.
Show me a triple junction.
Show me where the building, the crack in the building started.
Show me whatever.
And you pass the pointer.
And then you have the student who has the pointer come up with another question and pass it to somebody else.
So that's another nice technique that takes the responsibility off of the teacher.
And so you're not picking on students.
We've been doing mud cards, think, pair, share.
We're going to do a lightning round.
I'll talk a little bit more about jigsaw later.
And if time allows, I'll show you a demo I did on diffusion using the students as the atoms.
Sometimes if you're trying to ease in active learning, slowly into your class-- you don't want to do too much-- you want to think about the short time things that don't require huge disruptions.
If you want to go back to your seats, you can.
Maybe I should let you guys do that.
So you know what we'll do?
While they're taking a break, I'm going to demo the beach ball.
So this is nice.
This is the beach ball technique.
So when you catch the ball, I'd like you to tell me if you've had any experiences with active learning in classes that you've taken.
And you can't count this one
I had active learning in my electricity and magnetism class.
And that was called TEAL.
And it's very polarized.
The people for and against it, especially the students.
So they basically tables of nine.
And then within that table, you had groups of three.
And each of those would be a problem solving group.
So you'd have four problems to solve per two hour class period.
And that would be going up to the board look and doing it by themselves.
And then the teacher at the end would reveal the answers and tell us how to go through it.
And, for my group at least, there was always-- or my table-- there was always this one person who would write every single time--
3,000 People.
Stop and think.
Answer this question.
That's active learning.
That's way more that happens in a lot of classes.
Interactive learning generally implies that students are talking to each other.
And that's the most general categorizations you can get active, interactive.
So we're going to do a bunch of stuff with clickers, or plickers, these things.
But there's some other techniques you can do.
You can put up a graph or an image.
And you could say, OK, somebody come up and show me the green boundary in this picture.
Show me a triple junction.
Show me where the building, the crack in the building started.
Show me whatever.
And you pass the pointer.
And then you have the student who has the pointer come up with another question and pass it to somebody else.
So that's another nice technique that takes the responsibility off of the teacher.
And so you're not picking on students.
We've been doing mud cards, think, pair, share.
We're going to do a lightning round.
I'll talk a little bit more about jigsaw later on.
And if time allows, I'll show you a demo I did on diffusion using the students as the atoms.
Sometimes if you're trying to ease in active learning, slowly into your class-- you don't want to do too much-- you want to think about the short time things that don't require huge disruptions.
If you want to go back to your seats, you can.
Maybe I should let you guys do that.
So you know what we'll do?
While they're taking a break, I'm going to demo the beach ball.
So this is nice.
This is the beach ball technique.
So when you catch the ball, I'd like you to tell me if you've had any experiences with active learning in classes that you've taken.
And you can't count this one
I had active learning in my electricity and magnetism class.
And that was called TEAL.
It's very polarized, the people for and against it, especially the students.
So we basically had tables of nine.
And then within that table, we had groups of three.
And each of those would be a problem solving group.
So you'd have four problems to solve per two hour class period.
And that would be going up to the board and doing it by themselves.
and then the teacher at the end would reveal the answer and telling us how to go through it.
And for my group at least-- or my table-- there was always this one who would write every single time.
And I feel like that detracted from active learning.
I think it worked better for other people
It's definitely polarized.
Students really are not-- I think we talked a little bit about this when we did some of the learning theories-- but students really are polarized about whether they like it or not.
The data that there's been some pretty good analysis of learning, measuring how much students learn, and then how much they retain and all that kind of stuff.
And when you go through that, students seem to learn more and retain more in the active learning format, even though they really at least half of them don't like it at all.
So it's an interesting commentary That's a good great story.
So why don't you give the ball a toss?
There you go.
No, you have to answer a question.
Have you ever experienced active learning in a classroom besides this class?
And how did it work or not work?
I would say yes to that.
It's definitely the traditional method, sitting in front, face to face lecture
Fair enough.
Give a toss, toss to anyone.
Are we going to get the same answer from all of you?
If it's just--
Is it the same question?
Yeah, same question
Yeah, I have.
Because I recently studied in Sweden.
And that's the way, most of the time that way, they teach in Sweden.
So for some classes, generally in higher level classes, according with d-level classes, they use active learning
Great and students buy into it?
They're OK with it?
They work at it?
Culturally, they tend to veer towards active learning
And that's a very interesting point, the idea that it can be a cultural thing.
If everybody else is doing it, or it's expected, then that's what you do right.
So some of these changes, there's an incubation period before they actually take off and they're widely accepted
In fact in Denmark, I think that's the only way they each
It's pretty active.
I have a colleague who's in, or friend who works at the University of Copenhagen, and all new faculty have to take a yearlong course on teaching, no matter what department, physics, math, a year long, the faculty.
And here you could never force the faculty to do anything.
So it's pretty interesting that they've been doing it for so long
I have a question about that.
Is it the way that professors, their role as actual teachers is viewed?
[INAUDIBLE] But seriously, people [INAUDIBLE] or something, I think definitely trying to enforce a teaching class would be a lot more receptive than if you tried to enforce that here, where teaching isn't necessarily the focus
But there, it's a good research university.
But on the teaching side, that's the norm.
That's the expectation
But what about requiring-- I know it's hard when you have professors that have been doing what they're doing for 25, 30, 40 years.
It's hard to get them to adopt new techniques.
But I think that requiring things-- the faculty that you can require things of are the new faculty, who have no power yet
Yes and no.
If the tenured system, the requirements for tenure don't change, then asking them to take on new-- oh, you guys should try active learning.
Revamp this course.
And then they don't get tenure.
So it really has to be a whole systemic change, where the department buys in.
The school buys in.
The university buys in and says, OK, these are the criteria for tenure.
And in that is you have to use active learning.
Or you have to-- that's where it would have to get change.
Otherwise you can't expect people that aren't tenured to do it.
You shouldn't.
Let's give the ball one more toss
So it's been a long time since I've been in a classroom.
I would say, in a regular classroom, I don't that I've been in a class that does active learning.
But I have taken courses that are largely lab courses, like a mouse embryology course, where you have 15 minutes of lecture.
And then you go into the lab And you do it.
And the professor will walk around and help you with the technique.
You're actually there with the mice.
And you're actually doing something.
So does that count?
Lab courses can be their own genre.
But there has been this movement to take that's a little bit of what TEAL does, at a more simple level.
And there's a little bit of lecture.
And then students are supposed to answer some questions or do something.
And then they come back together.
And Studio Labs at WPI has that same format, where the classrooms are built with lab benches.
And students sit at lab benches and see a lecture.
And then they do some experiments.
They're mini experiments.
It's not whole mouse dissection.
But they do a mini experiment.
And then they report back.
And there's an interchange like that.
So there are some blending of the two.
Often what people do in labs is different.
And usually a good lab is fabulous.
A good lab is fabulous, which is another argument for why active learning is good, because everybody thinks back about, oh, that lab course was so great.
I learned so much and did so much.
And yet that's the lab course.
And then we have the lecture.
So it's an interesting comment.
So that was the beach ball.
And you see you guys got to throw the ball at other students.
So I didn't have to pick on every-- I didn't have to call on people or pick on people.
I had a little demo here about why it's important, just if you ask a question or you ask students if there are any questions or what questions they have.
And that's generally a better phrase.
What questions do you have?
It's more welcoming, opposed to saying OK, who's stupid enough.
Who's not following me?
Say, OK, what are the questions?
It's a more welcoming phrase.
But the idea of waiting.
So here, if I click on this little guy, you should hear 30 seconds of music.
So imagine you ask a question, 30 seconds for people to think about it and formulate an answer.
This is what 30 seconds feels like of dead time.
Well, there'll be music.
[MUSIC PLAYING] That's not part of it.
So did that feel like a long time?
I know we had a little break there.
But that feels like a lot.
Especially when you're standing up here, it feels like a long time.
But 30 seconds, give them 30 seconds.
The point is, if you want to give students 30 seconds, you're going to have to time it.
You totally have to time it, because you will not estimate 30 seconds effectively.
You will totally underestimate it.
And then another technique that you can piggyback on that is saying, OK, take 30 seconds.
I'm not going to take any answers for 30 seconds.
So you may have an answer right away.
But I don't want to hear it for 30 seconds, because there's always that guy in the front.
I had this guy in my high school math class.
And he used to sit on his-- this little short guy-- and he used to sit on his foot on the chair.
He's on his chair.
And he's sitting on his foot.
And he had his hand like this.
And he was in the front row.
And whenever the teacher asked the question, he would go.
And he would come up off of it.
And so he was always the guy that answer the questions.
But that left some of us who took a little more time not having the opportunity, kind of like having the one person write all the time in the TEAL group.
So that's a good way to go about that.
We talk a little bit about using clicker questions to solicit feedback.
And we've used the plickers in this class.
We'll use them again.
You can get dedicated clickers, these dedicated punch in things.
They use them for TEAL, I think.
Students may have to purchase them.
That may cost money.
The department may have to purchase them.
That's another source of money
They make us buy everything
So when I was at Brown, actually, the department would buy them.
And they'd leave them in the library.
And then you could check them out with your student ID.
And then you got that.
So you got it for the semester.
But it was linked to your ID.
And if you didn't bring it back, they'd charge you for it.
But that's another way to do it, a fairer way to do it.
We'll look at something called Socrative later on.
I'll use it later on.
But it's a text.
It's an online format.
So students need a smartphone or a laptop.
But they don't need a separate clicker.
There are issues with that, because not everybody is going to have a smartphone.
And are you biasing your class against people that don't happen to have the technology?
This is why I like plickers, because you don't need to buy anything.
I give you the card.
And I need to have the smartphone.
But you don't.
And then you could always use index cards or do the whole, put one finger up if you think it's A and two if it's B and three if it's C. That gets a little tough in a bigger class, because it's hard to see.
It's hard to count.
I like the fact that the clickers, plickers, and the Socrative or Poll Everywhere give you a histogram.
So you get a really quick idea about who's getting it and who isn't getting it.
So we're going to do a couple of these.
Remember-- this is from the label of the free body diagram-- that these misconceptions always linger in a lecture class.
We talked about this free diagram the other day.
So let's just try this one here, give you another little warm up on the picker's.
So you have the card.
And I'm just going to ask you to look at the question.
And when you're ready to answer, just hold up your card.
And if you can keep your finger away from the pattern, that would be great, because it makes it hard to read it.
And then however you want to--
Does this work for larger classrooms?
It does.
I have a colleague who uses it at Brown.
And he just started teaching this semester.
And he showed me his-- or a friend-- he showed me his plickers.
And he printed them out on 8 by 10, like a sheet of paper, cardstock.
And they're enormous.
They're like this big
How do you not get [INAUDIBLE]?
Well Well they have to hold them up.
You have to.
And you can walk around a little bit.
But I'll show you this.
So I have the histogram.
And I measured everybody.
So now I know what you think about this question.
I know who's got it right.
I don't know who's got it right.
But I know that most of you have chosen the correct answer.
But there were a few of you picked up one of the other answers.
And so now I know that it's a non-trivial number of you that picked an incorrect answer.
So now I know exactly what I need to do.
It's given me incredibly good information that there is some out there that have a pretty strong misunderstanding of the literature.
And that's that some of you picked A.
There's a lot of data to support that active learning, active technique supports student learning.
I call your attention to the Freeman paper that's in the reading for today, also to the [?
Haik ?]
article, which was referenced.
So those are two of the more compelling readings.
But if you go through, you'll see quite a few studies that unambiguously show that active learning has an impact.
So the incorrect answer is that it always involves group work.
That's not true.
It doesn't always involve work.
As we said, you could just ask students to sit and think and then do something with their thoughts.
And that would be active.
So Rachel, you're smiling.
Is there a--
Sometimes my face just does that
I didn't know whether it was a smirk or whatever.
And did I show you what this looks like?
So you can see with the histogram looks like.
I did?
[INTERPOSING VOICES] You weren't here, though, when I did.
The time I used it before.
I'll show you offline.
So the histogram is really, really useful.
So it really can ferret out some misunderstanding.
We don't have to go through all these.
But this is another way to check for misunderstanding.
I can ask this kind of question.
It's a concept question.
It's a conceptual question.
And then I can ask you to pick an answer.
And then I can see where your misconceptions lie about this topic.
So it can be very, very-- it should be very, very focused.
It's not just a busywork question it's a focused question with focus distractors.
The distractors are the incorrect answers
Can we do this?
I want to see what the spread of answers is
Sure.
Let's do it
They asked this question-- I laughed, because they asked a very similar question at Harvard graduation
Exactly.
What was it called, a world of their own?
Yeah one of those
Two times, check.
I've got to clear my responses.
Let's go.
So I'm getting you folks in the back without a problem.
I got one.
I'm missing one of you.
Wow.
So there's any number of things I could ask.
We're going to do another pair, share around this kind of thing.
But maybe not for this one.
But this is amazing.
Basically it's A got three votes.
B got three votes.
C got four votes.
And D got three votes.
So it's pretty amazing
What is the right answer?
Biologists?
The answer is B.
And it's the most-- maybe not surprising-- but the idea that it's taking in things from the air.
They're taking in gases.
But there's chemical reactions which manifest themselves in a weight gain.
[INTERPOSING VOICES] It's not a biology class, which is why-- so that's just super telling.
That would be incredible.
If this were a biology class and that was the spread, that would just be really, really informative for the instructor
Really worrying
Well yeah, that too.
But it's biology class.
So don't worry.
But you do want to think very, very carefully about the wrong answers, because you want the wrong answers to tell you something about what students understand or don't understand.
Generally speaking in the adoption of clickers, plickers, whatever, takes a three step process.
One is that people generally start with these simple fact questions, and then more challenging conceptual questions, which I would put the one we just did at.
That students, they might-- arguing with each other could be useful.
And then the third is you've just totally revamped the lecture.
And it's all centered around these questions.
And depending on how students answer, that's how you navigate through the lecture or through the course, class period.
I would never recommend that anybody start with this one.
I think that we're all capable of starting with two actually.
And I think starting with three may be a waste of time, unless you really just want to check and see if people did the reading or something.
It's a big class.
And you just want to see if they've done the reading.
But in general, I would say put your energy into coming up with some really good questions.
And maybe don't ask as many.
And then as I keep saying, I hope you can start to see how that's really, really linked.
It's a measure of whether they get it.
But it's also an activity to help them get it.
So I've been minoring the Back Channel.
Nobody's saying anything except for Dave.
And I answered him.
And because I gave you a break before, I may scoot on, continue on.
So if you want take a second and login, that's fine.
But I may not--
That's the link for last year's
I entered it, and I put entered into last year's.
And I got really confused
Uh oh.
So the one for this year is back here.
Hold on
It's like four something, 4125 instead of 3969
Oh OK, thank you.
So while you're doing that, I'm going to set up something which is called lightning round.
And we're doing these two.
So I'm going to just, sitting in your seat, if you can think about this question, just the question on the left.
Based on research at NASA, what was the approximate net global change in temperature between 1880 and 1975?
It's not a plicker question.
It's just a question.
Try to think about what you think the answer is
What is Celsius to Fahrenheit?
To get from Celsius to Fahrenheit?
So you have to-- well I always just double it and add 30.
So it's 8/5 plus 32.
[INTERPOSING VOICES] 8/56 plus 32 is the real equation.
Is it 8/5 or 9/5?
9
9/5 That's why I just double it.
Close enough.
So what I'm going to ask is that-- let's see-- David, Gordon, and Julie.
And if you can move down, down, just to the end.
And then if Adam, [INAUDIBLE], Dave can come around?
Just keep going.
Keep going.
Keep going.
That's fine.
One, two, three, four-- no, no.
You have to stand up, guys.
No relaxing.
Yes, but that's not-- I can tell a story.
Please stand up and move over.
Move over.
And just keep coming around.
Everybody's up.
Everybody's doing this.
Michelle, so I need six, Michelle come on over here next to [INAUDIBLE].
That's fine.
Now [INAUDIBLE], you come around here please, and go opposite David, please.
And everybody else just file in and go opposite someone.
Sarah, do you want to participate?
So here's what happens.
I'm going to tell you to go.
And you have two minutes.
The person in this line takes some amount of time and says what their answer is and why they chose it.
And the person in that line can ask questions or whatever, and then say what they thought and say why they thought that.
So you're going to have a really quick conversation about your choices.
And after two minutes, I'm going to tell you to please stop.
And then you have to get a new person.
And I'll show you how to do that.
Everybody understand what we're doing?
You're having a two minute exchange of ideas.
And it's going to be loud.
Believe me.
And it's the person opposite you.
Are you ready?
Everyone ready?
On your mark, get set, go.
[INTERPOSING VOICES] Let me ask a question.
Did you encounter people that had different choices than you?
Yes
Yes.
So that generally means it's a pretty good question, because you get to hear lots of different perspectives.
Now if this were the central part of the class, this topic, you might let the pairings go even longer, even more pairings.
So did anybody change their mind based on what they heard a partner say?
OK, a couple of you.
Generally, what happens then is you would want to have a de-brief, where we talk about why you picked the answer you picked and what you picked.
And most of the time, the choices are a little been informative.
If you found, as the instructor, that everybody picked a negative number, well you're in a very different place than if people are arguing over how much positive it is.
So you're going to use that information to find out about where the misconceptions are.
What's the advantage of this method?
Gordon
It's very interesting, I thought.
One I found out is that the students talk to each other.
We never talk to each other.
And it's easier for them as to [INAUDIBLE].
So at the beginning, they are aware of conflicts.
But as we rotated, there was a convergence.
It was very interesting
[INAUDIBLE] have your hand up
Yes, one time I was laughing just because it was very effective.
There was so much [INAUDIBLE].
And when we started, one resisted some passionated idea about this is how it should be.
And by the time it went to all the people, people can get all the idea.
You try to look for the superior idea.
Then you decide, oh, this is better.
When we try to switch again, you listen to a better argument.
So from which one decides without [INAUDIBLE] the first one?
A couple of points.
One is that it's impossible not to-- when you pair people up, it's pretty hard for somebody not to participate.
So if you have somebody that's resistant, they might still be resistant.
But it's virtually impossible for them not to participate.
And also it's a little less scary.
One person, it's really noisy.
You know you only have to talk to them for a minute or two.
So it's a little less scary.
Everybody's doing it.
The other thing is that you definitely want to have some sort of de-brief.
You can't just stop and go, OK, let's go on.
You're going to have to find out what people's answers are.
And you're going to have to address it or at least reveal the right answer, because there's many places many, many places for misconception.
And also, then, if you do it enough, you can maybe-- students can find arguments or people's explanations that resonate with them.
And then you get exposed to different kinds of arguments or different kinds of ways of explaining the problem.
So in this case, the answer is actually positive 0.4 degrees.
So B is the correct answer, just so you know
Yes, I just want to make a comment that sometimes this method could be confusing as well.
For instance, if you listen to two superior arguments, one against.
For instance, I talked to someone who was actually in support of negative.
And she brought an argument and so forth like that.
And somebody thought of positive.
And there was a strong argument also for that.
Sometimes if you just be sitting on the fence, I don't know where to get off
Well I think it's OK to be confused for a while.
That's fine.
As I said, if this were a course on global warming, climate change, whatever it is, we would have to make sure that we heard all of those explanations or a sampling of those explanations and said, oh, well that's a good.
I can see why you think that.
But you've forgotten about this.
Or I can see why, so that you can process it all so that people don't leave confused.
But it's good to hear the wrong answer, to be convinced of the wrong answer, maybe, but then to see why it's wrong.
That's an important learning activity
But this method is only applicable for where you have cases where you can debate.
Because if you have very clear cases, for example, if I ask, what is the color of the sky?
It's a definite.
But when it's fuzzy, then this can be--
Yes, or when it's a concept you think people might have difficulty with.
Yeah, it's not going to work if it's a real clear cut.
I want to demo one more method.
This is think, pair, share.
And I know we've been doing those forever.
But I'm going to couple it with a clicker question.
And I think we can do it in seven minutes.
So I have two blocks here, because this is just too fun, really.
I have two blocks.
They're the same width and the same.
Length but they are different thickness.
And I have this thing.
And I'm going to tap on the thin block.
Makes more or less-- it's not really a note.
But it's some sound.
And I'm going to ask you, when I do the same thing, the same force, the same place on the thick block, will the note be higher, lower, or the same as the thin block?
And this should be A, B, and C, not one two and three.
So but if it's A, B, and C go ahead and vote with your plickers.
Do you know that's what they auto correct to?
That's what it, if you're typing and you write plickers, it autocorrects to pluckers, which can be pretty embarrassing
So high is for A.
You've not clicked on the thicker one
I know.
That's why you have to tell me what you think it is
Is this a A, B, and C?
A B and C. So there's no D. There's a reason for this.
If I told you the answer, then there wouldn't be much fun in voting.
And I have everybody.
So I have a distribution.
I'm not going to tell you the distribution.
I'm not going to share that with you I'd like you to take-- just cluster.
It could be groups of three or four, whenever you have nearby, and talk about why you picked the answer you picked.
And then I'm going to give you the opportunity to vote again.
So in the interest of time, two minutes to discuss this.
[SIDE CONVERSATIONS] I wanted to just call attention to a couple things I did.
I walked around and tried to eavesdrop on your conversations so that I could tell what you were thinking about, but without putting you on the spot, without asking you to say, in front of everybody, what you thought.
So that's one thing to do.
Now I have a sense for your mental models, how you're thinking about the problem.
So I have some really good information about why you think the answers you think.
What I'm going to do is I'm going to give you the chance to re-vote.
If you want to change your vote, you may.
Or you don't have to.
Some of the clicker software actually lets you save consecutive histograms, which can be useful.
You can compare if there's change.
This doesn't let you do it.
So I just have to--
Voting twice there?
I'm voting for [INAUDIBLE].
He's not present, so--
Voting in Chicago, early and often.
We have exactly the same distribution, which flies in the face of some other data I was going to show you.
But that's OK.
I'll just in the interest of-- so one thing I could do is I could call on you.
And I could say, OK, if you said it was lower, tell me why you thought it was lower.
And I could get some arguments for why you thought it was lower on the board.
So some people think of cellos make lower sounds than violins.
And some people think that doors on mansions have a low sound.
And other people talked about xylophones and the length and the wavelength of the xylophone, and the longer wavelengths for the longer pieces.
So I know a lot about how you're thinking about this problem.
The other thing I know is that I am not telling you the answer yet.
So there's a little bit of suspense in here.
There's a little bit to try to keep you interested and wanting to know, wanting to understand the answer
Janet?
Yes?
So you said we had the same distribution.
But can we have a show of hands for how many people changed their answer?
That's a great question.
Yes.
Raise your hand if you changed your answer.
It could've been amazing
So sometimes we've done that in my physics class.
When they ask, who changed their answer?
And everyone raises their hand.
It's the same exact distribution.
Or everyone goes from A and B to C and D. And even C and D originally only had three people, you'll see the entire shift.
And it turns out A and B was right.
When we vote the second time, we all picked another answer
That's very funny.
That could have happened.
Sometimes a nice follow up question is how confident are you in your answer.
So then that's a really good-- because you may have 80% of the people h the right answer but if you say how confident and hardly anybody is very confident then, you do need to talk about it some more.
So it gives you some really, really good information.
So here we go.
Here's this one.
It's higher.
Exact same material
So when you play the marimba and the xylophone, you have bigger bars.
That they get really thin when they're on the bass.
The bass notes are really thin.
And the really small high notes, they're really, really thick
So that's one thing.
The other thing is that on a xylophone, it's a standing wave, because you fixed the endpoints.
And the wave just vibrates based on the length of the pins, the separation of the pins or the mounts.
So in this case, it's a free vibration.
So that's another different misconception.
And then if you think-- so the easiest way to think about it is a diving board.
If you had a thin diving board, and you went and jumped on it, it would go way down and come back up again.
That's a long wavelength.
And that's a low note.
And then if you think about a stiffer diving board that you'd be kind of like, mmm.
That's a shorter wavelength.
So it's different.
You have different-- and then any kind of a cavity, like a cello or a door of a house, if it's a big cavity, it's a bigger standing wave.
It's still a standing wave.
So those are very different.
They are different models.
So this technique is really great, because it really gives you incredible insight into what students' misconceptions are.
And then you can really make sure to address those misconceptions to deal with their faulty models.
So for the post-class assignment, I've asked you to think about what active learning techniques you might adopt in your own teaching, in your own classes.
And then think about how you would adapt them, anticipate any issues or whatever.
So hopefully you've gotten a good smattering.
And I'll post the slides, which have some links and things in there if you want to look at videos.
And also check out that list from University of South Florida.
That has some amazing techniques that you might find useful.
I'm going to send everybody off.
But if you'd like to comment on the mud cards-- I know we're over time-- I have them here, if you want to make a comment.
But otherwise, I will see you next time.
And I'd like the plickers back.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And before we get into the topic, which is educational teaching with educational technology, I just want to go back to some of the previous assignments, a little bit of logistics, et cetera.
So you should have feedback on all the assignments that you can be given feedback on.
The assignment where you gave a presentation to a friend, I can't really give you feedback on.
I looked over it.
You all made nice observations, good pointed observations, and I hope that you can see how-- and I think I made this comment on the wiki-- that you can see how just your presence, and the choice of words that you make, can have a big effect.
And that was the whole point of that assignment.
So that's my feedback on that assignment.
The previous one, the last one that I feel like I can give you substantive feedback on is the active learning, how to incorporate active learning.
As I mentioned last time, you guys did such a great job of incorporating active learning in the first time you'd submitted that, where it was just creating a course outline, that there wasn't a lot extra to add.
But I added comments whenever possible.
One thing that I commented on at least two people's homework, the one about active learning, was the idea-- you mentioned you doing demos.
So there's a very interesting piece of work out of Eric Mazur's group at Harvard about the use of demos.
And that paper is called "The Crouch and Mazur Paper."
And it's a optional reading in, I believe, it's the active learning session.
So this is the active learning class, which is class 5, right?
Am I right about that?
Or it was 6?
It was 6
6?
Yes, 6
6
[INAUDIBLE]
Oh, OK.
So it's 5.
Yeah, so that's a really interesting paper.
Because I think sometimes we say, oh, I'm going to do a demo.
Apriori, a demo isn't active.
I mean, it's active for you.
You're doing the demo.
But it's not necessarily active for students.
And if we enter into the enterprise of doing a demo thinking we're being-- well, thinking they're being active-- they might not be.
And Croutch and Mazur did an interesting study where they looked at-- they were doing a freshmen mechanics class.
And they had amasses on an air track, when you have like a massive 2M, and you slam into it with a mass of M. What happens to the mass of 2M?
What happens to this mass of M that was coming in, that kind of stuff.
And so that it would show the students these demos.
Oh, look what happens.
Here comes a mass of M at a velocity, da, da, da.
And then when they asked students in the same class at the end of the same class period to say what happens when a mass of M collides with a mass of 2M, a mass of M going x miles an hour collides with the mass 2M, what happens, students got it wrong.
They had just seen the demo.
They had just seen the demo.
And they still got it wrong.
And I think that's a really profound finding.
Because as experts, we think, of course, they're going to see this.
It's going to be emblazoned into their brain.
They'll never forget it.
But they never learn it, basically.
So it's more evidence for active learning for sure.
But what they found out was they forced students to make a prediction, to discuss that prediction with somebody else before they saw the demo.
And when they did that, the scores on these sort of post-demo quizzes, if you will, went way up.
So it's a small tweak, but it's a profound one.
Because you might really just be wasting your time if you show a lot of demos and you don't ask students to make these predictions beforehand.
This goes back to the mathematician George Polya, who wrote in the '50s about asking people to make predictions before you gave them the answer.
I'm not sure he had any data for it.
But this was his thing.
This is pretty good data suggesting that you really should ask students to make predictions before showing the demo.
So a couple people brought that up.
And I wanted to make sure everybody heard that.
OK?
If you can't find this reference, let me know.
It might be hiding somewhere else.
I'm pretty sure it's in this-- it is?
OK. OK, so that was that.
But overall, a great job on that assignment.
And then I briefly looked over the assignment it was due today or whenever creating prompts at various levels of Bloom's Taxonomy or a taxonomy of your choosing.
Again, I thought you did a great job.
I'd like to hear a little bit of reactions to that assignment.
Was it difficult?
Was it easy?
Do you see how it could be useful?
Does it seem like a waste of time?
I think it was very, very useful.
Yeah, it lets you see what the students will go to when you [INAUDIBLE].
Otherwise, because it's like you are solving too.
So you kind of feel and see what level to take them to.
It's very good
Great
Otherwise, you don't know.
You just [INAUDIBLE].
And then you don't know how [INAUDIBLE]
Exactly.
And I mean, it's happened to me when I first started teaching intro materials.
[INAUDIBLE] was like, OK, they'll calculate the stress on this plane, and blah, blah, blah.
And you're like, well, why are they doing that?
What is that getting them?
And it's just this reality check for you, so that you don't go off into the weeds sort of.
So, yeah, [INAUDIBLE]?
I think [INAUDIBLE] most of very good question that [INAUDIBLE] They're [?
setting ?]
very good questions and [INAUDIBLE] more than just one, [INAUDIBLE] three at the same time.
So I think it's really very good.
It's a good experience
Right.
And also to make sure you're aware of that, and then also decide, well, if they don't remember the value of the universal gas constant in these units, but they do the apply part fine, they just use the-- so it's important, we always have to think about that when we grade questions.
But I think by parsing it out like that, it makes you more cognizant of those issues that might come up, in terms of how the students respond to the question
I thought it was actually difficult to make a question that only satisfied one of-- like applied [INAUDIBLE] Because everything I was asking [INAUDIBLE] to try to apply, was also analyzed.
Because I tried to-- I guess i could have just made one that did both.
But I was sitting and I was like, no, OK, not this one.
But really, I almost left it blank and said, I'm stuck.
I finally come with something that I think just answered the [INAUDIBLE].
But it was interesting to think about
Right, right.
And you know, in real life, you don't have to just have one.
You can embed them together, but you know, recall and apply, or whatever.
And I think for some topics, it might be easier to add to silo-ize it a little bit.
Given the data, construct a whatever, so that's more of an apply.
But there may be classes of subjects where you can't really do that quite the same way.
Other reactions?
I was also thinking of it in terms of grading.
Like if you have a rubric or you have assigning the number of points based on which level you were asking.
Like as opposed to say, OK, five points for each question, to say, OK, you get one point for the remembering and then you get 10 points for the creative
Right.
Right.
I would be careful-- I mean, I think that's a fine idea.
I would be careful to make sure that students know that, so they don't kill themselves memorizing pi out to 25 digits or something, right?
So if you do have these sort of, this is what I think is important, you'll need to specify that.
I mean, and a rubric could incorporate that for sure.
But you do want to make sure students know what your logic is
I think also there are some questions you said that you give the formulas to the equation.
Just to make sure that the student [INAUDIBLE]
Yes, yes.
Right.
And in most classes, that's totally reasonable.
You know the joke I make about it's rare to real life that somebody comes in and says, I need this answer right now!
Don't look up anything.
I mean, it's kind of a weird pretense.
So it's usually reasonable to just put the equations at the bottom, or write it in the problem statement.
Anything else on that?
OK, so today we're going to talk about teaching with educational technology.
I had written on the website to bring a web-enabled device.
If you didn't, I have an extra laptop.
And we can share.
It's shouldn't be a problem.
As part of this, I hope that you'll be able to identify appropriate educational technologies to advance an intended learning outcome.
And I want to get back to that in just a second.
And maybe describe best practices and potential pitfalls of particular educational technology, so somewhat modest goals.
I think it's a really interesting thing to think about technology-- and I didn't make this up.
I heard this at a conference.
So they said, when you walk into Home Depot or one of these big home improvement stores, if you happen to be blessed by having an employee come up to you-- because sometimes it's hard to find an employee in those stores-- what do they ask you?
Is everything OK?
Is there anything I can help you with today?
Right.
OK, is there anything I can help you with?
If and you say, yeah.
Yeah, you can.
And then they say?
Usually, it's, what are you working on?
What are you trying to do?
They don't say, what tool do you want?
I mean, they might.
But if they're good, they don't.
And I think that's a word the wise when we think about educational technology.
It's not, what looks really cool and fun, and what tool do I want?
It's, what am I trying to build?
What do I want my students to build?
What am I trying to build?
What am I trying to accomplish?
And then I can step back and say, which of these technologies is going to help me do it?
And it's easy to get wrapped up-- I mean, we're all kind of techie, geeky.
I mean that is a compliment.
But it's kind of fun.
Like, oh, cool, look what this can you.
And then we go walking around looking for something to do with it.
And that might be OK, but you have to check your assumption.
You have to check your assumptions about why you're doing it.
Are you doing it just to be cool?
Or are you doing it because it's really going to help student learning, or it's going to help advance the learning outcome?
We can use educational technology as some kind of a scapegoat, but that's true for anything.
It's true for that amazing exam question that you thought up in the shower.
Like, I know what I'll do.
They'll do this, and do that, and blah, blah.
Maybe it's cool, but maybe it's not going to get you anywhere.
Maybe it's not consistent with your learning outcomes.
So it doesn't have to be technology.
But often, technology is kind of pretty shiny.
And we tend to gravitate toward it.
But it's remembering that we're using it to advance some particular learning outcome, OK?
So you guys did the readings.
What are educational technologies?
What does it mean?
Can we come up with a definition?
Yeah.
I think from what I can understand, educational technology is a tool which, perhaps, the student and also have the teacher.
So that means something that can enable inter-communication between the teacher and the student.
And then the student and the student
OK, so we have a lot of ideas in that statement.
So it's a tool.
You said, it enables communication
Yeah, students connect.
And then the teacher [INAUDIBLE]
And I think you had something else in there.
Did you?
Or no?
Somebody else?
[INAUDIBLE]?
Education technology is like a tool to enhance students' learning-- enhancement of student's learning.
And enhancement of instructors' teaching as well
The technology part is like a electronic device
So it has to have electricity?
It has to involve electrons?
No, that's everything.
Wait, let's let Michelle-- what should we say about that?
I think that's kind of correct to say it has to be an electronic device, right?
Like if it's a clicker or if it's a-- [INTERPOSING VOICES]
It must be electronic.
That sounds so like 1950s.
I think we had-- yeah, Gordon?
I would just adjust it a little bit and add in front of the two [INAUDIBLE]
A kind of tool that's created [INAUDIBLE] by time or place
So it lends itself to asynchronicity?
Yes, something like that
Most-- Anything else?
I think it's also enhancer, easier use of materials
Facilitates--
Yeah, materials like.
It has to [INAUDIBLE]
Right, a distribution, aids distribution?
[INAUDIBLE]
Did I hear something else?
[INAUDIBLE]
Efficient, that's a loaded word.
Anybody have anything else?
Or does somebody want to make a comment on this collection of attributes?
[INAUDIBLE]
I don't think I have any cynics in this-- well, the cynics haven't said anything
I think all the comments are positive things.
Like if you're doing it well what should happen.
But they aren't necessarily what actually happens.
Because most of the time, it's probably not done well.
Using technology is not optimal
OK, that's one observation.
It's sort of, you know, that rhyme, there was a little girl who had a little curl right in the middle her forehead.
And when she was good, she was very, very good.
And when she was bad, she was horrid.
So it's a little bit like this, right?
When it's good, it's very, very good.
And when it's bad, maybe it's even worse than if you hadn't bothered.
But-- Alex, you had something to say?
No?
Well, on the side of the cynics, I thought that students are often disinterested when they don't see the point, or don't see them using it later.
It's not necessarily a definition of it
Right, right.
No, but that's a very good point.
And somebody had quoted this idea of an app, that if an app doesn't load in something like seven seconds, people stop using it.
Like if you try to get an app to open on your phone, and you're like, what's going on here?
Come on, 1, 2, 3, 4-- OK, forget it.
And that's a similar idea, that if it doesn't look like it's going to help you-- and whatever that means, help, it's self-defined, help.
But if it doesn't look like it's going to help you, you're not going to buy into it.
Some other thoughts about this list?
Personally, I don't think that to be an educational technology it has meet all these criteria at once.
That's an important thing to note, that it might improve productivity.
The old school learning management systems that you put up problem sets, and you put up-- the grade book was in one place, and that kind of thing.
That might help your productivity.
You're not looking around for stuff.
But it's not necessarily promoting learning at any level.
It's a delivery system, a materials delivery system.
So it kind of does this.
And it does this.
But it might not really be part of the learning enterprise.
It's a store.
It's a repository.
And that's fine.
There's nothing wrong with that.
But that's one type.
If you're saying that it enables communication, well, that might be part of this.
But it might be something else.
If I say, all right, it's a wiki.
And I want you guys to read what everybody else wrote, and to make a comment on at least one other persons' posting, well, I have facilitated communication between you in an asynchronous way.
I didn't tell everybody to do it at noon on the 29th of October.
But it does facilitate communication.
I think this one is a loaded one, for sure.
And it gets to Michelle's comment about if you did it right, it would do that.
But if you did it wrong, it's likely to not do that at all.
So these are attributes that you could use to describe some educational technologies.
I do have a definition from the reading book.
But before I put that up, any other thing you want to add?
It's interesting to think about it, like does it have to be electronic?
So I have had classes in the past where people argue, OK, well, a blackboard is an educational technology.
I mean, you could do that without some sort of human-made product.
And without it, I would have to be walking around speaking very slowly and hoping you took notes, or walking around with-- I don't know-- a piece of paper that I wrote on, and sharing it with you or something.
So a blackboard does fit a lot of these categories, right?
It's an efficient delivery system.
At some level, it enables communication.
It's a distribution of learning materials.
So we want to think about whether it means it has to be electronic.
And you may have guessed by now, there is not a single definition, so that's the other issue.
Comments?
Yes, go ahead
Just a comment.
What about if it's not well-moderated [INAUDIBLE]?
Yeah, that's the efficient one, that we've all gotten sucked into websites, or sucked into, perhaps, an online discussion, or commenting on something.
And it might be interesting.
You might learn.
But it may not be efficient at all.
Or you might just be on the screed end of things, where people are just either ranting or they're reinforcing misconceptions or whatever.
So yeah, you do have to kind of make sure people are using it carefully.
But that's true for virtually anything you do.
Any other comments?
And I think, as well, that the fact that these tools are valuable.
A student can decide to stay home in the bad weather since he knows this is available.
Then he can go [INAUDIBLE].
That could be good and bad.
Because the student would continue to learn on his own and become a good student.
And then he could miss out on some critical points as well
Right, right.
But that implies some sort of like web-housed delivered repository of content, or of video lectures, or whatever.
You could have a course that used clickers, let's say, that didn't even have a website, but they used clickers.
So that course might be using educational technology, maybe in a very, very good way in the classroom, but they don't even bother with a website.
So in that case, the student has to come to class if they want to experience that particular use of technology.
I think you're right, that if you do have a website, or you do have lectures online, or you do have rich content online, then it does sort of, perhaps, make being in the classroom a little bit less necessary.
I will say, if you employ active learning methods in class, group discussions, et cetera, that being in the classroom is, certainly, I would argue, is value added for the student.
So it's not a replacement experience, but it might be better than nothing, let's say.
It might not.
Yes, Gordon
I found something written very interesting when it said that if you use clickers-- I would love to do that.
To take attendance of the students, that it might [INAUDIBLE] the students from using technology.
But it's only [INAUDIBLE].
It might actually be good to have technology they can use non-evasively to find out if the student is actually coming to your class
What if they gave their clicker to their friend?
Yeah, it's true
One person will come with five friend's clickers, the clicker won't--
If you have like RFIB technology or something like that that can use to [INAUDIBLE] come to the class to know that, OK, this person is the one using [INAUDIBLE] [INTERPOSING VOICES] [INAUDIBLE]
So in certain courses you have to do that
Yeah, you must do that
You have to show up?
Yeah, you should show up.
But if you decide it's difficult to take attendance, and if you have this wonderful way of getting the students to [INAUDIBLE]
[INAUDIBLE]
Well, I don't know if there's an answer to that.
For some students, it's never a problem.
For some students, it's awesome.
For some students, it's just more information.
And it's just another path.
And they're learning.
And they're learning.
And they're learning.
And it's supplementing what they're learning in the class.
It kind of depends on--
Taking attendance that Gordon is talking about, so is it [INAUDIBLE]?
Well, yeah.
And actually that's an important point is that I think I showed these slides about clicker use in 511 1.
I showed when we did active learning.
And the first year they used them, they used them more or less to take attendance.
And students figured that out pretty quickly.
They did all sorts of things, sharing clickers, clicking and then leaving, or coming in at the end and clicking, but not being there the whole time.
And if you ask students, well, what did you think about the clickers, they're like, fu.
This is crappy.
This didn't help me.
This is terrible.
And probably it didn't.
There's an attitudinal thing that says at some point, if you think it's not helping, it's not helping.
So the next year, they made it a point to really incorporate it into the class, to ask good questions, rich questions, questions that had interesting and informative distractors.
So the wrong answers told the instructors what the problems and misconceptions were.
And this was made clear to the students.
And in that way, if you ask students what they thought about the clickers, they were not at all resentful of the use of clickers.
And they did not see them as a way of just taking attendance.
They didn't think they were sort of baby or whatever.
And they were therefore more effective.
[?
Rachel, ?]
you had a comment
Yeah, I was just going to say going to 802
That's the E and M, Electricity and Magnetism freshman year
The class I took it in [INAUDIBLE] So we did all the online [?
courseware ?]
and then the idea was that you come to class and you just learn for two hours from the teacher using group discussions, and all these other things.
And it was one of the worst classes I've ever [INAUDIBLE].
Because the professor wasn't prepared to do these two hour discussions, like group discussions, and group work, and all these different things that you could tell he wasn't used to doing.
And it was just a giant mess.
So I think they're really useful tools.
But they also require the professor to really put something into it for the students to get something out of it
Right, right.
And that's a consideration.
For some things, you can adopt them.
There's kind of a low bar for adoption and a low bar for solid use, maybe not innovative, maybe not groundbreaking, but a solid pedagogical use.
For others, if you're going to use them effectively in the classroom, you're going to have to spend some time engaging with the technology, making sure you understand it, making sure you understand what you can do with it.
Otherwise, it's not going to work.
A couple stories, one is the idea of-- well, maybe I'll hold that one.
I'll hold that one.
I'll bring it up later.
Remind me.
Any other comments about this?
So this is a definition that I got, I think from one of the [?
reasons, ?]
so technological processes and resources that are used, created, or managed for learning and/or improving performance.
So that's crazy broad.
And then in that definition, there's this technological, which gets around the electronic, but brings back the idea of well, a blackboard is probably a technology at some level, or at least it was at some point.
So it's admittedly an incredibly difficult concept to pin down.
And maybe at some point, you don't necessarily have to.
You have to decide which slice of it you want to deal with.
And so the other important thing is that it's-- and it doesn't have to be, but it can be-- an integral part.
Or it should be well-integrated within the course, and or it could be a supplemental learning aid.
So it could be a way of delivering content outside of class, or the idea of an enrichment of the material.
And it could be something like hardware, really just like hardware.
I'm going to come back to how we chop this up in just a second.
Remember that we don't want to do any thing-- this is the idea about walking into Home Depot.
We don't want to do anything without thinking first about our intended learning outcomes.
What intended learning outcome is the technology going to advance, not, what am I going to do with this technology?
And so in terms of categorization, this is something that I came up with just after grappling with things.
So this is just my thing.
And we're going to talk more about different ways to think about it.
So this is just one slice of it.
You could think about technologies that you use in the classroom, so that would be clickers, that would be using some sort of projection system, using a smart board, using some sort of simulations that you show students, et cetera, like virtual demos.
It could be a way for students to engage with the content and with other students outside of the classroom.
So it could be some sort of a wiki, some other collaboration tool, and/or it could be something that delivers the content to the student outside of class.
And there may be an overlap.
You may show a simulation in class, and then decide students should have access to it outside of class.
So they're not mutually exclusive.
And so the engagement with the content can include these collaboration tools, extra materials, content enrichment materials.
And then the third one might be student assignments.
So the idea, maybe you're going to say to students, OK, I want you to create a website for a company that you're going to create, and this parameters.
And you want to make sure you're explicit about what the company sells, and how you deal with customers, or whatever.
The assignment involves the technology.
Go out and make an audio recording of people using a particular technology.
Or if you're going to design something for use in some community, interview the people in the community, and submit this audio recording.
So it could be that technology could be used in the student assignments.
And you could back that all the way up to online problems as they use in MITx.
And then for assessment and feedback, the idea that you can either use clicker questions, which would be for the formative feedback.
Or you can use it as some sort of an online quiz, or some other way of giving feedback to the students.
So that gets to the summative and the formative idea.
And then the idea of just using a learning management system, just to administrate the course.
That's one slice.
There's another couple other very interesting ways to slice this up.
And I have a handout here, because these are woefully tiny, these schematics.
So if you just take one and pass it along.
One is called the SECTIONS framework.
And the other is called the backward Ed Tech Tool Flowchart.
So these are different.
These are not mine.
And these are different ways of thinking about technology.
So the first one is this Ed Tech chart.
And I know you can't read that here.
So that's why you have the handout.
And I like it, because it kind of starts with the activity that you want students to use.
So it starts with this, what do you want students to do?
And that's what the purple box are, the student tasks.
And then it kind of creates these sort of guiding questions that sort of reroute you.
And then it gives you a suggestion for a tech tool.
So I'm going to give you a few minutes just to parse that, just to engage with it a little bit
It's very beautiful
I like this one.
I like this one.
I think it's very useful.
Any other comments or thoughts about it?
Yeah, Gordon?
I think it's [INAUDIBLE]
OK, Gordon doesn't get it.
Somebody want to-- maybe nobody gets it.
But do you want to explain it a little bit?
Out loud, David, please
Yeah, [INAUDIBLE]
But can you explain to everybody?
[INAUDIBLE]
Oh, sure
OK, [INAUDIBLE]
It's certainly not exclusive.
I mean, it doesn't have everything on here.
But it kind of gives you, I think, a way of thinking about, OK, I start with what I want students to do, which most of us have a pretty good handle on.
And then it helps you just think backwards to the technology.
[INAUDIBLE]?
Yes, I just want to add that it's only about what you want to do, and how can you did it
Yes
That's what it's all about.
And secondly, [INAUDIBLE] but what you want them to do, and how do you have them doing it, that's what [INAUDIBLE]
Yeah, Yeah.
No, it is great.
Other thoughts?
And yeah, we'll get to get back to the idea of the taxonomy shortly.
Other thoughts?
Let's see, there's a second one, which is on the other side of your handout.
And so these are just-- I know you can't see this.
That's why you have the handout.
These are just ideas.
Again, there's no rules really.
But depending on what you're trying to do, it just may help you start to think about how to get the right tool for the job.
So it's called sections, because each of those boxes with the red outline, S-E-C-T-I-O-N-S.
The S is student characteristics.
The E is ease of use.
C is cost, teaching and learning, interaction to promote active learning, organization, novelty, and speed.
And this chart, I think-- I'm going to let you engage with it for just a minute.
But it brings up the idea of the learning curve for the instructor, which you don't want to lose sight of, that you want to factor that into your choices about what technologies to use.
So again, I'm going to shut up for a minute or so, so you can look at this
[INAUDIBLE]
Right, right.
But what's the difference between this one-- I mean, there's many differences, obviously.
But what's kind of a utility difference?
One's a flow chart that points you to the answer.
So it asks the yes or no questions.
The other just gives you the-- then you have to select from a-- it doesn't guide you directly
Right.
Exactly.
It kind of helps you think about it.
Oh, my gosh, I have no money.
So that's certainly going to rule out a whole set of options for you.
But it doesn't say, oh, you have no money, think about these.
It's not a flowchart.
It doesn't point you to an end result.
But if you went through this exercise and answered these questions, I think you'd be in a better place, with respect to choosing the particular technology.
Gordon?
Yeah, also I think that second one is not [INAUDIBLE] if you want to use technology and you have the money, [INAUDIBLE] the things you can think about.
Because if you don't think of everything [INAUDIBLE]
Right.
It's some things to just keep in your head when you're making these choices.
It's not necessarily a formalism for making a choice.
So one other slice of all this technology is kind of a classic, I would say classic, model, which is called the SAMR model, S-A-M-R.
I know I'm throwing a lot of models at you.
But you have to kind of find the slice that works for you.
The SAMR model, I think, is very useful to keep in mind when you're deciding why you're gravitating to a particular technology.
And it's this idea of, is the technology just a substitution?
So back in the day, when everybody said, oh, courses need web pages.
You have to have a course web page.
Well, what was the course web page?
It was an electronic version of your syllabus, basically.
It had the name of the course.
It had some picture.
And then there was a link.
And you could click on it, and the students got the syllabus.
They got a text version of the syllabus, a soft copy of the syllabus.
But then they printed it out and brought to class.
So it was a one-to-one substitution.
Instead of seeing the syllabus in a piece of paper, they see it online.
It was complete substitution.
It didn't do anything extra, except maybe made it a little easier for students not to lose the syllabus.
But it didn't really have any functional difference.
Then the next level-- that's the S. The A is that there's an augmentation.
So there's a substitute, but with a little bit of an improvement.
So maybe students could hand the homework in online.
They're still handing homework in.
They're still probably writing their homework out, scanning it, and handing it in.
But it's a little easier.
It's a little bit easier.
Maybe you don't lose their homework, that kind of thing.
So that's the augmentation.
The next level will be a modification, so some sort of serious redesign of a task that you couldn't do before that technology.
And then the final would be it just redefines the tasks that you do, the tasks that students do, in a way that's completely different.
So I'm going to ask you to think about that for your own courses in about two minutes.
There's a little bit of a schematic here that might help.
These are sort a little bit baby.
But the idea of substitution, so the idea of using a word processor instead of a typewriter, that was a substitutional technology.
So then augmentation, you get spell check.
Once you're in a computer, you get to have spell check, and formatting, and cutting and pasting.
And that's different than what you can do on a typewriter.
And then so modification, well, there was like an email, or you could put graphs and images right into the document.
It changed sort of the way we viewed a whole document.
And then the idea of having hypertext, or some sort of a web page that students navigated in a nonlinear, or a non-prescribed way, that would the redefinition, let's say.
So that's some sort of baby examples of that.
And then this next one, which is pretty fun, is there's this woman, Kathy Schrock.
And if you Google Kathy Schrock, she comes up.
She's got a ton of stuff.
And this to get back to Bloom's Taxonomy there, is that she's taking Bloom's Taxonomy, remember, understand, apply, analyze, evaluate, create, and then she said, OK, well, here's some apps that can facilitate those things, facilitate the act of remembering, or facilitate the act of evaluating, or creating.
So it's not mutually exclusive.
It's not exhaustive.
But it is important to remember that some of these tools are supporting remembering, or they're supporting some other level, understanding.
But they're not necessarily at the highest level.
Or they're not at the lowest level.
It's good to be aware of that.
Before I charge you with your assignment, I did want to demo a few other technologies, just so you can take a look at them.
And there was that list in the reading, which I hope you checked out.
I invited you to this-- well, I thought I invited you to-- oh, wait a minute.
Huh.
Hang on.
Well, let's just start up a new one.
So I can create this picture here.
And I can share this, then, with you.
And I can invite the board.
And so this is the link.
And if you have a device and want to log in there, you can check it out.
And then you can write on it.
Oh, somebody wrote on it.
Somebody erased my picture.
I don't know who that was.
Oh
Is that a [INAUDIBLE]?
U-Q-P-
So that's different from the link that was in the wiki
I know.
I just created a new one, because I didn't want to go look for the wiki link
But this is free?
This is free, yeah.
So we're working a little backwards here.
I'm showing you the tool.
And now I'm asking you what you might want to do with it, which is not necessarily best practice.
But I have a use that I-- you can change the color.
This pen here changes the color.
No, something changes the color.
Maybe not.
Oh, somebody got blue.
When I taught thermodynamics, we would often do sketches.
I'd say, OK, sketch the Gibbs free energy versus temperature, or Gibbs free energy versus pressure.
Tell me what the first derivative of g, with respect to t, is, things like that.
Sketch the slope, sketch the curvature, et cetera.
And so this kind of application is you could have students log in, do that kind of a sketch.
And then people can evaluate it.
It's anonymous, you can see.
I can't tell who said what it, except David, who wrote his name.
But I can't see who did what, so that it could work pretty well in a small class.
If you had a class of 400 people, I'd be afraid.
I'd be very afraid.
But in a small class, you can come up with some interesting uses of it.
The good thing about it is that you don't have students coming up to the board.
It minimizes that motion, the sort of getting everybody unsettled and settled again.
It's not a cure.
I wouldn't say this redefines anything.
In terms of the SAMR model, I'd say it's probably a modification, maybe an augmentation, maybe.
I don't know.
But it's certainly not and R. So I wanted to just demo that, because it's kind of fun.
The other thing I wanted to do is, I think you guys saw this list?
Did you see this list from the readings?
This is just some things that I've compiled.
It's kind of a mishmash.
But there's some interesting things on here that you might want to explore, depending on your learning outcomes.
And we've used Backchannel.
Today's meet is pretty similar to Backchannel.
Backchannel chat is pretty similar Backchannel.
So those are some things that are pretty similar.
We just looked at the web app.
We've used Plickers.
We've talked about Socrative, which is just a cell phone-based clicker.
But there's a lot of interesting uses.
And you might want to just play around with them.
And I'll augment this list.
So if you click it again from the wiki, you'll get a augmented list of things, a few things I want to add to it.
So now, given your experience in life, and the readings, et cetera, is that I'd like for you to think about a technology that you haven't used for-- actually, what you want to first is think about a learning outcome.
But you want to think about a learning outcome.
You want to think about a technology that might advance that learning outcome.
And then just jot down why you chose this technology, where does it fit in the SAMR model, how you would use it in your class, any kind of difficulties.
And then there's a Google doc.
I'm pretty sure everybody can access it.
We can only hope.
And if you can type your responses into that.
So we can give you about five, seven minutes for that activity.
And then we're going to do a lightning round based on it.
So use whatever.
Use the handouts.
Use your knowledge, use, et cetera.
And then somebody can tell me really quickly if they can get into the Google doc.
Yeah, you guys are good.
I was going to say, write your name down.
But it doesn't really matter whether your name is up or not.
So I want to make sure that you take a look at everybody else's some of the learning outcomes.
We're going to do this lightning round where you're going to be talking to other people about the technology they chose, their learning outcome, they chose it, et cetera.
And you're going to be trying to help your partner troubleshoot, or perhaps, implement the technology in a better way, solve some of their problems with the implementation process, or perhaps, think about a different technology altogether given their desired learning outcomes.
So let's just take a minute, and either read it on your own device or read it up here, so that you kind of get a sense for what you're going to be talking about.
And if you haven't read over it, please do that now.
So there's some really interesting ideas up here, I think, and a broad spectrum of ambition.
We had some pretty small-- like one assignment's worth of technology, perhaps.
And then we also have kind of a whole course design.
So that's another thing we didn't really talk about.
I didn't narrow the scope in any way, or define the scope in any way.
So it could be for a whole class.
Or it could be for one particular assignment, or one particular class activity.
And that's fine.
That's OK.
So did everybody get a chance to look over the other posts?
I'll say, I've have had [INAUDIBLE]
So it gives a follow up of all of these things is that when the AWW where everyone was drawing, I could draw here.
But it was never showing up there
Interesting
Oh, yeah, my thing I drew never showed up
Yeah And then the other thing with this now, with the Google doc, I tried to edit it.
And it doesn't let me click on it.
It says, do you want to use the app?
So then I had to download the app.
And now I say, yes, use the app.
And then it says something about the server not being recognized.
But then if I go back and say, no, thanks, and I try to use it without the app, then I can't put anything in
Interesting.
OK, so who had trouble?
Yeah, I downloaded the app and used it instead
All right, did anybody that was using a smartphone not have trouble?
I didn't have trouble
You didn't have trouble?
No, I downloaded and use it
Oh, OK. And were you an Android, or Android?
[INAUDIBLE]
Android, iPhone, iPhone.
Who else had trouble?
It was more than just the two of you with this one?
With this one
With this particular one?
Yeah, [?
Rachel ?]
had problems with it
So for now, if you wouldn't mind, just-- let's see.
We don't have to complete this right now.
But maybe if there's a lull, just come on over here.
And you can type your app in, just so we have a complete record-- your technology in.
But that's interesting.
It's a good thing to keep in mind that access might be an issue.
I did try this out on two computers.
But I tried it out on two computers.
And I didn't try phone
[INAUDIBLE] Couldn't do
[INAUDIBLE]
Yeah, after is fine.
So if we could get up.
And anybody else that wasn't able to input their ideas, if you will do it later, that's fine.
So we're going to do the lightning round.
And if I could get you gentlemen to come around and fill in the back.
Just fill in up to here.
Great, so what we're going to do is I'm going to ask the row in the back, this row, to share their technology, their intended learning outcome, their concern.
And then this row will give their advice.
And it's two minutes for the whole process.
And then we'll switch.
OK?
So what do I do?
So you just have to tell me more or less what you wrote on the screen.
OK, but hold on.
I got to set the timer.
OK, go.
[INTERPOSING VOICES]
Is going to share their ideas with the other side.
[INTERPOSING VOICES]
It's moment you've been waiting for.
So we're going to do this two more times, this time and one more time, OK?
All right.
[INTERPOSING VOICES]
Yeah?
For the course which I want to teach using educational technology, so it was a big challenge, because the head of department wanted to say no.
But then it continued because many of the professor already preferred it
Wow, so does [INAUDIBLE].
[INTERPOSING VOICES]
Anybody can do anything.
And you can reconfigure it
Alex, if you come around.
And then, [INAUDIBLE], if you come around, please.
Everybody slide down
We already went that way
Oh, you went that way
We already saw those people
So, [INAUDIBLE], if you come on over.
[INTERPOSING VOICES]
So I know in my conversations, I heard some really interesting ideas, some fascinating uses, and some incredible obstacles.
I'll just point out David's obstacle is on the spreadsheet.
But he wants to use his piece of software.
And it costs $12,000 a year.
So it sounds fabulous.
It sounds really fabulous.
And your department actually bought it for you?
Yeah.
Have it for all faculty [INAUDIBLE]
Yeah, so that's great.
But that's a huge hurdle.
So that's an example of trade-offs that you might not always be able to make.
So that was my interesting-- but I did hear some other wonderful ones too.
So I hope that you take a look at it.
You'll note that I always give you time after this lightning round.
Because it's a very noisy kind of little crazy exercise.
And I always like to give you a few minutes of silence afterwards to sort of process it a little bit.
Because it can be a bit of an overload.
So that's sort of the part of the design or the planning for this exercise.
Anything anybody want to share about something they learned, or some issue or problem that was solved, or still unsolved?
All right, I see some people are still typing.
I'm going to go on.
Feel free to keep adding to this.
And again, please take a look at it.
Because we only got so many pairings.
And you can see virtually everything from the Google doc, assuming everybody gets in.
So what I wanted to do is flash back a little bit to what we talked about the second class that we met.
We talked about all these different learning theories, about behaviorism, cognitivism, constructivism.
I think we've done some other exercise with these three learning theories before.
But it's a good chance for you to reflect back, to do a little backward transfer, and to reinforce some of the concepts from the beginning of the semester, and to also think about how technologies might be-- so we talked about how technologies might fit at different levels of Bloom's, or supports different levels of Bloom's, but also, different technologies-- different anything you do in the classroom.
But use of different technologies can actually be more behaviorist, cognitivist, or constructivist in nature.
So I think it's useful for you to think about that just as a way of just getting a bit of a marker.
About again, it's this idea about, what are we trying to do with this tool?
And so I'm going to divide you up.
And we have nine people left, so three groups of three, which works absolutely perfectly.
Yes, three groups of three.
And we're going to have the behaviorist group, the cognitivist group, and the constructivist group.
What I want you to do is think about a particular technology that sort of would be consistent with this-- ooh, OK, one group will have four-- a particular technology that's consistent with that particular learning theory.
And so this will take about maybe four minutes.
And think about, perhaps, even think about a learning outcome that that tool advances, but at that level, so the behaviorist level, let's say.
And then what I'm going to do is the activity is called a jigsaw.
And so first, you're going to be in these homogeneous groups.
So one group talks about constructivism.
One group talks about behaviorism.
And one group talks about cognitivism.
And then I'm going to mix you up.
So each group then has one cognitivist, one constructivist, and one behaviorist in it.
So this is an active learning technique.
And I wanted to model it for you.
So let's just create some groups.
[INTERPOSING VOICES]
All right, folks.
I don't know how far you got here.
But now each one of you is going to be expert in your particular field.
So I'm going to take-- and this is the constructivists.
So here's three constructivists.
I'd like one cognitivist and one behaviorist to find a constructivist, and to make a new group.
All right, so now each one of you is the content expert for your particular theory.
And just sort of explain what you came up with to your peers.
So you'll just go around the table and talk about it.
[INTERPOSING VOICES]
They're still learning.
But they're having an active roll in participating in when they're going to learn about, rather than just being passively taught that
And that's?
Constructivist
OK, so what technology [INAUDIBLE]
Well, I think that a cool way to do it is to have computers set up throughout the room to have internet access
So the class is officially over.
I mean, it's five of.
I don't know if people have to go other places or not.
So we can take this up beginning of class next time if there's things you want to finish up.
I'd like to hear some of what you came up with for sure.
So can we just plan to do that, and just tie up the exercise?
Not the best of the best pedagogical strategy, to make you wait a week, but it is what it is, so.
OK, thanks very much.
And I look forward to seeing your posts on the assignment.
The lightning round is a nice interactive technique that I use in my class.
And what we do is I divide the group into usually two teams, two sides, and I have them face each other.
So you can have them face each other in two parallel lines, or you can have an inner circle and an outer circle, as long as one person is facing only one other person.
And then you can do it on a topic similar to the debate where there's really two sides and you ask the one side to take one side and the other side to take another side.
But the difference with this is that the communication is only between the student and the student that he or she is facing, and they only have one or two minutes for the interaction.
So one student gets to speak for say, it's a minute, and the other student gets to respond for a minute.
And then after that's over, I ask one line to shift, or one circle to take a step to the right or the left, and now each person has a new partner that they speak with.
Generally speaking, if you had one row be the pro and the other row be the con, then for the next iteration, you would switch roles.
So there would be the con and the pro.
The advantages of this technique are that the students get to hear a lot of different opinions from a lot of different students.
The fact that it takes only two minutes at most for each interaction means that you can do it several times, four times, five times.
If you do it five times, it's only about 10 minutes, maybe 13 minutes with some time to translate.
So it's a pretty quick exercise.
And by doing this lightning round, students can get many different perspectives from other students.
The other kind of question you can ask with a lightning round is you can ask a question, maybe you give students a few minutes beforehand to do a calculation or to solve a particular kind of problem.
And then you say, OK, you have one minute to explain how you did the problem to your partner.
And your partner has one minute to explain how he or she did the problem to you.
And again, that lets students see other students' approaches, and it lets them think about their own approach in the context of the other person's approach.
You also tell them they're allowed to change the way they think about the problem, or to think about the response, based on what they've heard in their previous pairing.
So it lets students kind of iterate and develop their opinions kind of as they go.
A couple of disadvantages of the technique are that it's very, very noisy.
So students are speaking with each other.
They're really only about a foot apart, and everyone is speaking at the same time
I originally thought there was going to be an increase, but then I learned that in the 1870's, there was like, a mini ice age
So depending on the class, depending on the personalities, depending on the people in the class, it may not be such a great technique.
But the advantage is that it really gets everybody up.
It gets everybody moving, even people that are really shy.
There's virtually nobody that will stand there for a minute and not say anything.
Even people that are extremely shy are generally willing to talk with one other person.
And since you can't hear anything about what's going on in other groups or even the instructor can't hear what's going on, it's a fairly safe conversational situation.
So that is an advantage of it.
The thing I like about it the most is that it gets students up.
It gives them the opportunity to formulate an argument and to listen to the argument of others and then put their opinion, or argument, or answer in the context of other answers.
And it lets them hear the opinions and responses of a lot of their peers in a very short period of time.
After I finish the lightning round, I always give students a minute, or two minutes, three minutes to sort of sit down and reflect on the exercise, what they learned, if their answer changed, how it changed, why it changed.
And then we usually do a large group report out to say, I thought this going in, but I heard from so-and-so and this is what I think now.
So we do a little bit of a report out.
Also because of the fact that as the instructor, since you can't hear with students are saying, you want to know if there's some big misconception that's been propagated during the activity, and you would want to address that.
So the report out for that exercise can be particularly important.
I'm Janet Rankin.
I'm the Interim Director of the Teaching and Learning Lab here at MIT.
And for the past four years, I've been teaching 595J.
It has a lot of other numbers, but it's Teaching College-Level Science and Engineering.
Well, before I came to MIT, I was on the faculty at Brown University, and I taught in the engineering department there.
And it was very interesting to me how you could either do a really good job teaching, or not so good a job teaching.
So I started to get interested in some of the theory and some of the best practices associated with teaching specifically to engineering, but teaching in general.
And I worked in the Teaching and Learning Center at Brown for a while.
And so when there was the opportunity to come here and work at MIT in the Teaching and Learning Lab, I thought it would be great-- a great opportunity to come and work with other scientists and engineers who are also interested in improving teaching, or using research-based best practices in teaching.
One active learning strategy that I use in 595 are called mud cards.
And they are a really easy activity to use.
Anyone can implement them, and it's a really low barrier.
So I like to introduce that early in the class so that students can see that it doesn't necessarily disrupt the class flow.
It doesn't really take a lot of prep.
It doesn't take a lot of resources.
So a mud card is, at minimum, is an index card.
And at the end of every class, I ask them to write down on the index card what they might want to know more about, what they're still confused about, or something that they found particularly compelling or interesting.
I ask them to specify that so that I know if they just say, active learning, I don't know whether they thought it was really interesting or whether they thought it was confusing.
So I asked them to specify.
But we use we use that because-- and then I also talk to them about why it's so effective.
They're able to anonymously identify what they understood or what they're still confused about.
I can go back to my office afterwards, I can look through the cards.
I can sort them out really quickly, even in a large class, and I've used them in a class of 100 people, even though 595 is just around 15.
And generally speaking, the cards fall in three categories.
You get three main problems.
And so you know that maybe you didn't do a very good job of helping students understand a particular topic, or that students are really interested and want more information about something.
I know that really easily, really quickly.
So I can make a decision to either get more resources, write something up that I can post on our course website, or I can come to class the next time and say, hey, looks like most of you were confused about X.
Let's talk a little bit more about that.
And I can prepare ahead of time and be totally ready to go in with multiple explanations or multiple examples of whatever it is people are confused about.
I looked over some of the mud cards we had from last time and there were some really good questions, some really good points.
Someone asked if all those learning theories that we discussed were equally valid.
They get they get targeted feedback based on what they don't understand.
And it's really easy.
It takes two minutes at the end of class.
I have a colleague who calls them tickets to leave, meaning he doesn't let people leave the class until they've handed him an index card with something on it.
But they're very, very effective.
I believe they were first used in the Aero-Astro department here at MIT, but they're used all over the place now and they're extremely, extremely effective.
For instructors that are thinking about using mud cards in their classes, I would say the biggest issue or the biggest concern would be to make sure that you use them early in the semester and that you use them often.
If you use them one day and then you don't use them for another two weeks or even another week, students really won't get in the pattern of filling them out and filling them out thoughtfully.
And they won't see the utility, because you really need to come back the next time with useful information saying, I looked at these cards.
I understand you don't get this.
Let me help you.
And that act really encourages students to keep filling them out, and it's just a win-win for everyone.
A common active learning technique that people often use, or maybe the first technique that they use, is personal response systems.
So that's an opportunity for students to respond to a question.
And that can be anything.
It's usually a multiple choice question, but it doesn't have to be.
And it can be anything from a show of hands-- who thinks the answer is A, who thinks the answer is B, who thinks the answer is C-- to dedicated devices and applications that allow students to respond to questions.
So there's pros and cons for each of the different techniques, but the essence of all the techniques is that students are given an opportunity to think about the answer that they think is correct, and then respond to that.
And there is evidence to show, research that shows, that when students make a claim, when they're asked to make a public response, a public claim as to what they think the correct answer is, that makes them more committed to the answer.
And then they're more likely to remember or to even notice if their answer is incorrect.
It primes the student for learning.
And that's the biggest advantage for any of these personal response systems.
The disadvantage with having students raise their hands is that in a big lecture hall, it can be hard to count, right?
It can be hard to say, OK, how many people raised their hand when I said A?
How many when I said B, etc.
So that can take up time and it can be pretty inaccurate.
The other thing that people do sometimes is they'll use like, just a sheet of paper.
They'll put on letters and then the students can fold the paper up.
And depending on how they want to vote, they just hold it up so that really only you can see it.
Point of fact is other students can kind of tell what the other students answered.
And if I know that a friend of mine always gets the answer right, then I might just look and see what he or she said and then answer the same.
So it might not give us a real accurate measurement of what every student thinks.
The other problem with that is this can be hard to read in a big lecture hall.
Sort of the first kind of digital or electronic personal response systems were these dedicated clickers, is what they're called.
And students literally, they log into the system.
They turn their devices on.
And you ask a question, and then the students respond A, B, C, D using the touch pad.
Some clickers lets students actually type answers, so open response.
They can actually type a word, or a number, or a set of numbers.
So that can give you a little bit more flexibility.
And then the software on the instructor's computer will create a histogram of responses.
So you know right away the distribution of responses.
If you have more than 70% of the students get the question right, you probably just want to state the correct answer and move on.
You can also say, if you didn't get it right, here's some extra resources or go to the TA office hours, or whatever.
But in general, if more than 70% of the students get it right, you can move on.
One thing I like to do is if I really have a resounding, say 75%, get it right, I like to ask a follow-up clicker question that says, how confident were you in your answer?
Because sometimes students will just guess.
And if you give them the opportunity to say, yeah, I got it right, but I don't really know why-- that might be the situation sometimes, that a lot of people get it right, but not that many people know what's going on.
So you want to know that too.
If between 30 and 70 get it correct, then I like to move into the pair share.
So I like to say, OK. Let's divide up.
Looks like we have a pretty good split of answers.
So let's talk to each other and we can do some peer-peer learning where peers can try to argue their point to whoever is sitting next to them.
And then I ask them to vote again.
Now, when things are working well, people actually converge to the correct answer.
And generally speaking-- I think it was Eric Mazur has a preliminary research study that shows that when you put two students with the incorrect answer together, they actually tend to converge to the correct answer because they argue each other out of their respective wrong answers.
So that can be pretty effective.
You do have to have a pretty good distribution of incorrect answers for that to work.
If two students had the same incorrect answer, generally, speaking they stay embedded with the incorrect answer.
If less than 30% of students get the correct answer, you probably just want to stop and reformulate your explanation and really go over everything again, because that's really not a lot of people that know what's going on.
So that's the general ballpark for how to proceed with the clicker questions.
In between the pieces of paper and the clickers are some newer technologies, new applications.
One is there's a few applications for web-enabled devices, like Socrative, which is a software that you can download.
The instructor downloads it and then students log into a room using their web-enabled device.
So they log in and then no one has to buy anything extra, but you are assuming that every student has a web-enabled device, and you are saying you're happy if they take them out in your class, which may not be true for everyone.
So that's the issue with that, access and also whether you want the smart phones out.
An application that I think is a really great compromise is something called plickers, with a P, P L I C K E R S. And with that, the instructor downloads an application on his or her smartphone or web-enabled device, it has to be a device with a camera.
And then the instructor downloads these PDFs, which are just pieces of paper with these patterns on them.
They're all different, and I believe you get 40 patterns for free and they're reusable.
And so each one is unique and you give them to the students.
And students then vote A, B, C or D. There's four sides to the shape and four letters, A, B, C, D. And if they think the answer is A, they hold it with the A at the top.
If they think it's B, they hold it with the B at the top, et cetera.
And you can use your web-enabled device, and you scan the crowd with the software, and that actually picks up the responses and creates a histogram.
So I think that's a really nice compromise, because it gives you the histogram, it has student anonymity, and students don't need anything but the piece of paper that you gave them.
And you can actually reuse the piece of paper.
We printed sets on card stock, which are pretty durable.
And we can use them over and over again, so the students don't need any extra technology.
For educators that are thinking about using some kind of personal response system for their students, there's a couple different levels of how far in to get and how fast.
The first step would be to construct some really good, thoughtful questions that students can answer in a multiple choice way.
That would let you use the clickers, or a show of hands, or any of the other methods.
It is important to construct the clicker questions in a way that students know they're really meaningful, that you're not just asking kind of silly questions.
You want to ask them questions that are a bit rich and that also lend themselves, as I said to, if students, you know if a decent number don't get them right, that lend themselves to that peer-peer interaction, that sort of debate with each other.
So want to think carefully about your learning outcomes, what you want students to know or be able to do by the end of the class.
And you want to construct questions that help them get there and test whether they're there or not.
So you might use one, or two, or three questions in a class.
And that might be it, that might be all you do.
But you do want to make sure that you construct the questions thoughtfully.
The next-- sort of if you want to go further in, is to think about what it is, really think critically about, your learning outcome.
What do I want students to get out of this class, this particular class?
And then construct a series of clicker questions that really help students understand whether they get it or not.
Help them argue their point with others or encourage them to argue their opinion or point with others.
And then ultimately, come to a realization or come to attain the learning outcomes really by engagement with the clicker questions and by engagement in a subsequent discussion.
So where the clicker questions are really the core of your class and the learning happens sort of threaded among those questions.
The think-pair-share strategy is one of the classic active learning strategies that one can use in one's classes.
And it has, as you might guess, three parts-- the think, the pair, and the share.
Often there's a collapsing of the pair and the share.
But in general, you first give students time to think about a question or a situation or some other scenario that you want them to think about.
And you give them time to think actively but alone about that.
And often you can ask them to write down their thoughts.
But then at the end of that short period of time, whether it's three minutes or five minutes, you have them pair up.
And if it's a really big class or the numbers of students in your class warrant it, you can have them triple up, it doesn't have to be a pair.
And they discuss whatever it is they came up with.
It's just an opportunity for them to just say here, this is what I thought of.
What did you think about it?
What did you think about what I said?
So it's just an interactive opportunity for the students to hear from their peers.
And then the key part when they're talking to each other is that the instructor should walk around the room and kind of eavesdrop on what the students are saying.
So you want to understand the kinds of things they're talking about, maybe not the specific details, but the kinds of things they're talking about.
And pretty quickly you can learn if students are on the right track or if there's a big misconception that maybe they might be reinforcing each other's misconceptions.
So you want to know that as the instructor.
So that's why if possible, to walk around the room and just overhear what people are saying, that can be a really important part of that pair step.
And then for the share, you have them report out.
So this does two things.
One, it lets you bring to light any specific comments that you overheard during the pair section.
You can bring it up with the whole class, because there can be some really important, wonderful observations the students make within the pair.
And if you don't share that, or if you don't create a situation where that can be shared with the bigger class, then everybody misses out except for those three people.
So it gives you an opportunity to do that.
It also gives you an opportunity to globally clear up any misconceptions that you've heard that have bubbled up from this pair exercise.
The other thing it does is if there's a group of two or three or even four, but I think three is probably optimal, the group reports as a group.
It doesn't report as an individual.
So if a student is a little insecure or may not really want to share what he or she thinks at a personal level or at an individual level, being able to share out as a group is a bit safer, is a bit more comfortable for many students.
So that's what the share part does.
And then it just gets the whole group together, and it lets you kind of sum up the smaller conversations with the larger group.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'm Sanjoy Mahajan.
This course is Teaching College-Level Science and Engineering.
The course is intended for graduate students some towards the end of their PhD who are intending to go onto academic teaching careers.
But all graduate students are welcome, and even undergraduates who have a strong interest in teaching.
The goal of the course is to give some preparation and background into the principles of teaching, so graduate students and in fact as undergraduates too, all get the same for their specific research area.
For example, in physics, they'll learn the principles of conservation of momentum and energy.
But in teaching, everyone is basically just thrown to the wolves and figured, well, if you know your subject, you can teach or given very little preparation.
So the idea is to actually show that teaching and thinking about teaching is a very serious intellectual activity.
And there's general ideas and principles that can be used to make teaching much more successful.
And therefore, students learn much more and enjoy what they're learning.
Many departments encourage their students who they know are going on to teaching careers.
They say, well, you might actually find this course interesting.
It's paired also with another new program at MIT, called the TA certificate program, which is a non-credit option where students do about six or seven seminars of an hour and a half each and do work for each seminar.
Here, this is the sort of four-credit cousin of that course.
And it's for students who want to do everything in one semester and want something on their transcript as well.
So at the end of each lecture, I give out a short sheet that has three questions on it.
One is what was most confusing today or what do you wonder about most?
What example or teaching technique worked well or not well?
And any other comments?
And so at the end of each session, the students have one or two minutes to fill that out.
And at the end of the next session, I answer all the questions.
I was just amazed at the thoughtfulness of the questions and how right on they were and how many useful suggestions I got for when I teach the class the next time.
Many students have told me they think everyone who teaches should take it.
And some told me also that they wish some of their professors had taken those classes, their professors from when they were undergraduates.
But on the other hand, if it were required, then it wouldn't be such a lively atmosphere as it is now, where we have students who really are fascinated by teaching and are therefore much more motivated to contribute.
But the theme that I got from many, many students was, wow, this has really helped them thinking about their teaching and said oh, finally, this may be the only preparation I get in teaching and I'm really glad I had that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Due to technical difficulties, only a portion of lecture one is available for viewing
Welcome to Teaching College-Level Science and engineering.
The title contains the word teaching, which may spark some questions in your mind.
For example, is teaching just an art?
Or is it something that's just something you're born with?
In which case, either you have it or you don't have it.
Well, obviously, I don't believe that, or I wouldn't be teaching a course on it.
What would be the point?
Or is it purely a science where there's a set of equations and procedures to learn, and then all of a sudden, you'll be an excellent teacher?
Well, actually, it's neither.
And it's both.
It's things that we're all born with, on the one hand.
And there're also procedures and techniques and ways of thinking that will improve how you teach and that we can all learn.
So it's a happy mix, my favorite mix, an art and a science.
So another example that's an art and the science is book design.
So compared to, for example, just pure art-- painting, say modern painting-- very unconstrained, versus say biology procedures in the laboratory, very, very closely specified.
It's somewhere in between.
There's an art-- all the arts of color, space-- but they all have to be used together to achieve a particular purpose.
So again, in there, there's some beautifully designed books and some not so beautifully designed books.
And there're principles behind that that we can use to design good books.
And similarly, there are principles we can use to design good teaching.
So the whole point of this semester is to design good teaching and how you do that.
And rather than me give you a big long theory about it, because actually there isn't really theory so much in the equivalent to, say, Einstein's theory of relativity.
But there's principles to learn.
The best way to learn those principles is with an example.
So what we're going to do today is I'm going to do an example of teaching with you.
We're going to do a sort of maybe slightly sped-up version of what we would normally do, say, if we were actually using this example in a class.
Then we're going to analyze why it was done that way.
And from that analysis, general principles of teaching will come out that will be addressed throughout the semester.
And they'll be addressed in the context of particular tasks.
Like for example, how to make slides that are useful for teaching, how to use a blackboard, how to teach equations, how to design a whole course, how to make problems.
So all of those tasks will be the week by week subjects.
And in each task, all of the principles that we're going to talk about now will show up in those tasks.
And you'll see the principles illustrated repeatedly.
OK, so the problem-- one of my favorites.
So these are two cones.
One has twice the dimensions of the other cone.
So let me show you how I made the cones.
So I printed out a circle and just cut out one quarter of the circle.
And then I taped this edge to that edge.
Or in mathematician speak, I identified the edges, which now I know means I taped the edges together.
And then you get a cone, like that.
So this cone and the other cone were cut out of the same sheet of paper except this one has twice the linear dimensions in its circle.
So I think this circle was 7 centimeters in radius, and this is 3 and 1/2 centimeters in radius.
So other than that, they're the same.
And the question is, which one has the higher terminal velocity?
Or are they about comparable?
So in particular, the question is this.
So I'm going to drop them.
And the question is, what's their ratio of their terminal velocities?
So the ratio of the big cone's terminal velocity to the small cone's terminal velocity is equal to what.
And you get choices along this axis.
So here is-- OK, so those are the five regions to choose from.
So you have five choices for the ratio, basically roughly 1/4.
This is some range here, because nothing's exact.
And we're certainly not going to do an exact experiment.
Roughly 1/2, roughly 1, roughly 2, or roughly 4.
So does everyone understand the question?
Because you're going to get to try it yourself.
Question about the question?
If you could restate it-- actually, there was a sign-up sheet going around which distracted me.
So I sort of lost the--
Sure.
Could I restate the question?
No problem.
So I'm going to drop them, just like this.
No tricks-- I'm not going to flip this one around or anything.
And the question is, what's the ratio of their terminal speed?
So right away, as soon as you let go of them, they come to a steady speed, which is their terminal velocity.
And the question is, how do the terminal speeds of the big one and the small one differ?
OK, so in particular, the question is, what's the ratio?
And there's five choices for them.
Does that help?
Yeah.
So what were the dimension of them again?
So this guy is-- he was cut out of a circle who was 7 centimeters in radius.
And this guy was cut out of a circle who was 3 and 1/2 centimeters in radius.
And then I was also very careful to use-- let's see.
Did I do this right?
Yeah, I used to half the with of tape on the small guy as I did on the big guy, just to get it really very perfectly scale invariant.
OK any questions about the question?
OK, so think to yourself for about 30 seconds or so, just to induct yourself into the problem.
And then we'll take a vote.
And then you'll have a chance to discuss it with each other.
OK, let's just take a vote.
So I recognize I haven't given you at all enough time to come up with an exact answer or calculate everything.
And that's by design.
So let's just get a straw poll.
And then you'll have a chance to argue with your neighbor about it.
So who votes for 1/4, which is that the-- so let's see-- 1, 2, 3, 4, 5, 6.
OK, 6. Who votes for 1/2?
1, 2, 3, 4, 5, 6, 7-- 12. Who votes for C?
22. Who votes for D?
No takers.
No takers for D. OK, how about E?
OK.
So now find a neighbor or two, one or two neighbors.
Introduce yourself to your neighbor.
And also, by the way, unless you're taking notes on your laptop and really insist on doing that, if you could close your laptop, that would be very helpful for the purpose of discussion in this whole course.
So find a neighbor or two.
Introduce yourself.
You'll be getting a chance to meet graduate students from across the institute.
This is a great opportunity.
And try to convince them of your answer, especially if you have a different answer.
Or if you happen to share an answer, well, see if you can figure out why you're sure of it.
Or if you're not sure of it, see if you can settle that.
OK, so discussion time.
And if you have any questions that come up as you're discussing, raise your hand.
And I'll come and wander over.
Meanwhile I also handed out feedback sheets for the end of the session, which I'll ask you to just spend a minute on at the end.
But you'll notice one of the questions is, what's the most confusing thing?
So if anything really confusing comes up during the whole session, you can just put it right then.
You don't have to wait till the end.
Or if there's something you really liked or hated-- that's question two-- you can put that whenever it happens to come up.
But vote number two, and then we'll take some reasons.
So 1/4?
1, 2, 3.
OK, 4.
1/2-- the halves don't have it.
Oh, no, there's 1.
Oh, there's 2.
OK great-- 3, 4, 5, 6.
Equal?
OK, so that's about-- let's call it 30.
2?
And 4.
OK so thanks for the votes.
Let's take reasons for any of them.
I'll take reasons for any of them, and I'll put them up here.
You don't even have to agree with the reason.
It's just something you guys discussed and thought maybe was plausible
C seems to have [INAUDIBLE].
When you do this kind of activities, there is always some guys who have seen it before.
And he may choose to spoil it.
So I may want to know what you would do in that kind of situation
So you're-- hm.
I'm not sure how to phrase this.
Let me just take other comments.
I'll come to it afterwards.
Other comments for any of the reasons?
So again, it doesn't have to be things you necessarily believe, but things that are plausible.
And that's actually more instructive than what you think is for sure right.
Because now you're trying to figure out what might be true.
And you're expanding the ways you're thinking.
Yeah?
C, because they have identical mass to surface area ratio
OK. C, so mass to area ratio is the same.
OK, can people think of plausible reasons against that argument?
Yes
So I have no idea what the actual formula is for calculating it.
If there was a square in there, then that would [INAUDIBLE]
Right.
So I'll call this Not C. So suppose the formula actually depended on the square root of A or square of A or something like that.
It's, say, one chance out of three that it has A to the first power here.
It could have A to the 1/2 or A to the 2.
So could be m over A to the k for k not equal to 1.
Others for or against C, intuitive reasons, or for any of the others?
So hopefully that's-- oh, yes.
Go ahead
[INAUDIBLE] The air resistance scales with the area.
And the gravitational force would normally scale with the volume, [?
except that it's ?]
in the surface, [INAUDIBLE]
OK, so let's see.
F drag proportional to area and weight proportional to area.
So that's an argument for which choice?
For
For C, OK. How do you know that the drag scales with the area?
Maybe it scales with square root of area.
Any arguments pro or con?
OK, yeah
You could argue that it scales with the area because then you just break it up into decimal pieces.
Each one has the same force intensity
OK, so for this, let's say there's a thought experiment of subdividing.
So I'll just note that as Subdividing Thought Experiment.
OK, yeah
But then if you have some weird shape, the air has to flow through hitting some pieces and then flow through the rest of the pieces.
So that [INAUDIBLE] may not work.
[INAUDIBLE]
So it may depend on the geometry.
So I'll put that up here as Geometry.
What else might depend on?
For example, is air resistance, say, always proportional to area?
Hm.
Yep?
It could depend on the material, surface roughness or something
OK, so it might depend on the material.
And it certainly does, which is actually one reason I was very careful to construct them out of the same piece of paper.
So let me put this is the material.
So the surface roughness.
Yeah?
Well, this may not be relevant given how you said you dropped them.
But it also might depend on [?
exactly how they ?]
fall
Right, OK, so whether they fall vertically or downward.
Yeah, that's true.
So it might depend on the way I drop them.
So actually, yeah, to make us not have to worry about that, I'll just drop them simultaneously pointing downwards.
Yeah, so the fall configuration.
So there's all these other variables.
So let's do the experiment.
Then I'll come back to your question.
So let's do the experiment this way, which is that I will stand on the table and pray that I have matching socks on, which is sort of 80% these days.
It's increased.
And I will drop them on the count of three.
Are they both about-- the points about the same level?
They look sort of to me, but my depth perception is actually quite bad.
OK, so is that about equal?
OK, one, two, three.
Simultaneous.
OK, so there you have choice C. So now the interesting consequence of that-- so what that shows is that drag, in this case, is proportional to area.
Now, it turns out that that's not always the case.
So drag very often-- well, not very often in everyday life.
But very easily, drag can be proportional to size.
And you don't know ahead of time which one it's going to be.
So it turns out at slow speeds, low Reynolds number, this is true.
Turns out at high Reynolds number, this is true.
And this is the simplest experiment to show that.
So what this shows is that drag is proportional to area.
So that with the same velocity, the extra weight is balanced by the extra drag force by exactly 4 to 1.
And what that shows, now-- the consequence-- is that-- I'm going to replace the proportional with a twiddle.
So it has an area in it.
So I'm just going to get something with the correct units now.
So it has an area in it.
And now you have left to play with density, speed, and viscosity.
So now let's actually construct the drag force as a result of that.
So we know by the experiment it's proportional to area.
And now among these-- so this here is the kinematic viscosity, which is the one you may be more familiar with divided by rho, the density.
So we have to put some of these guys in, some of these guys in, and some of these guys in, and make the units come out as a force.
Well, one of them we can do right away.
There's how many powers of mass over on this side?
In a force-- just one.
And there's none here, yet.
So we need to get one power of mass on this side.
Now, among all these guys, which of them have mass in them?
Not this one, because you divided out.
Not velocity.
Only density, and density has one power of mass.
So you have to put one density.
Question
I don't know what the units of drag are
So this is a force.
Good question.
So drag as a force.
So this is just newtons or mass, length per time squared.
Does that help?
So it's just newtons, in SI units.
Or in general, mass length per time squared, so a mass times an acceleration.
So now, we've matched the units of mass.
But we haven't matched the units of time yet.
So let's sort out the time.
There's no time here.
There's no time there.
There's t to the minus 2 there.
Well, what can we do about that?
We have to throw in some v and some nu.
And the problem is we don't know how much.
So the time doesn't help us enough.
Turns out to make the time and the length work, the simultaneous constraint, the only way to do it is that.
You make the same argument to get the masses to match, the lengths to match, and the times to match.
This is the only way to do it.
So you don't have any viscosity.
So actually, that's the simplest experiment I know to show that the drag at high speed-- in other words, most flows are actually high speed, high Reynolds number-- is independent of viscosity.
So it's rho av squared.
And that is a great result, because it tells you all kinds of stuff about everyday flows in everyday life.
Like for example, why did people reduce the speed limit on the highway back in the '70s to preserve gas?
Well, on the highway, you're burning gasoline to fight drag.
So if you reduce the speed, you reduce the drag, you reduce the amount of gasoline.
So in particular, if you reduce the speed by 20%, you reduce v squared by 40%, which reduces the drag by 40%.
So you just decreased the gas consumption by about 40%.
So you can see all these things right away just from this simple formula, which is a immediate consequence of this experiment.
Now, it turns out this-- how do you get that to work?
This is the low Reynolds number limit.
You can't deduce it from this experiment.
But if you know that this is true, you can make the same argument and figure out how the drag force varies for low Reynolds numbers.
OK, so now let's just check whether this formula here that we've deduced works at all.
So the follow-up question is the following, which is that I have-- one, two, three, four.
Here on this side, I have four small cones.
They're all identical to this small cone.
So I have one small cone, two small cones, three small cones, four small cones.
So I'm going to stack all four small cones into a thick small cone.
I'm going to race it against one small cone.
So the question is, what's the ratio of these guys' terminal speeds?
So let's call it v4 and v1.
So v4-- OK, so what is the ratio of their terminal speeds?
1/4, 1/2, 1, 2, or 4.
So talk to your neighbor for just a minute.
I'll take a quick vote.
We will do the experiment.
OK, so let's take a vote.
And then we'll do the experiment.
1/4-- no votes for 1/4 ratio.
Who votes for 1/2?
1?
2?
That's about 35.
4?
About 10.
So let's do the experiment.
One, two, three, four of them.
OK, so now let me drop them like that.
Well, it's kind of hard to tell it, isn't it?
So that was actually not a well-designed experiment, because you have to actually get out a timer and decide which one is going faster and measure how long they took.
It would be nice if we had a way that was just like the other experiment.
What was nice about the other experiment is when I dropped them, you got the answer by the fact that they hit simultaneously.
So if we could make them hit simultaneously, then that would be nice.
Now, what do I have to do to do that?
Well, I have to switch their heights 4 to 1 or 2 to 1.
So let's try 4 to 1.
OK. Is that sort of 4 to 1?
No
No?
What do I have to do?
That has to [?
go-- ?]
This guy's got to go down.
Yeah, see, this is where my depth perception really fails me.
So I only have monocular vision.
I can see with both eyes, but I don't binocular fuse, so I can't tell depth
Lower them both
[INAUDIBLE]
Oh, that's true
[INAUDIBLE]
Lower them both
--corner of the desk
Pardon?
If you go in the corner of the desk, then you drop one on the desk and the other one [INAUDIBLE]
Oh, but then you wouldn't be able to see it so well.
So I want to do it on the desk just so everyone can see.
OK, so if I lower them both like this, is that about right?
Yeah.
[INAUDIBLE]
Lower?
Wow.
OK. OK. One, two, three, go.
No, not very simultaneous.
So let's try 2 to 1.
OK, I'll overcompensate in my mind.
[?
And it's ?]
probably about 2 to 1.
Is that about 2 to 1?
This one?
[INAUDIBLE]
[?
Socks.
?]
Every time.
OK, 1, one thousand, 2, one thousand, 3, one thousand.
Simultaneous.
And what's always amazing to me is I always listen to the class every time I do this.
And I never believe them when they say it's really 2 to 1 in height.
And I was like, OK, I'll trust you guys.
And it always comes out simultaneous.
So people haven't lied to me yet.
So 2 to 1-- why 2?
Well, it had 4 times the weight.
Same surface area, same density-- you couldn't change the air.
So you have to change the v. How much do you change the v by?
Factor of 2, because you square it.
And you make up for the weight increase by factor of 4.
So factor of 2.
So there you have an application of it.
So now, let me get to your question about what do you do if someone tries to spoil the experiment.
So the first thing to do is-- well, let me say what were my reactions as soon as you did that.
So my first reaction was, god damn, why did he do that?
That was a really nasty thing to do.
But then the second reaction is-- whoop.
So now I've developed a habit, which is a good habit to learn, of just pausing before you react to things.
Because you're much less likely to cause bad interactions if you pause.
So I paused to myself.
And I thought, there's actually no point in saying how annoyed I am right there.
And this will happen.
Suppose someone tries to wreck your experiment.
You will be annoyed.
So it doesn't help you to actually say how annoyed you are and really make that person feel terrible.
Because then what you've done is you've polarized the audience against yourself besides making that person feel bad.
So it's much better to do what I did, which was say, OK, tell you what.
I'll answer your question later.
But also what I tried to do is answer it implicitly.
I don't know if you noticed, but what I did was when I came to ask people, I figured, OK, so people sort of have an idea that it should be C just because of somebody said, well, I've seen the demonstration before.
But what I wanted to know was not why you think C is right but why you think any of them could be plausible.
So that was always part of the question, but I emphasized that part more.
And it has a good educational purpose, which is that you're trying to predict not just what the quote "right answer" is, but you're trying to expand your repertoire of options.
So you're actually making yourself more creative by thinking, what could be plausible?
Because it's that way of thinking that you're able to use in other problems.
So I tried to turn it around a bit and push it in a direction that it would still be a useful discussion for people.
OK, maybe not as useful as if people didn't know ahead of time that the answer is C, but still a useful one.
Does that help answer your question?
OK, so now what we've done-- we've actually gone through a whole teaching example.
Yes, question
So [INAUDIBLE] talk about when you're writing down the number of votes, if you should leave them actually like [INAUDIBLE]
I didn't see.
That's a good point
And so that way, those people would be like, oh my God you didn't see me
This is a good point.
And I didn't.
And partly, it's because the lighting in this room is ghastly.
Partly my eyes are not very good.
But all of those reasons support exactly what you said, which is that, especially if you have a few people in the class, you can always get away with adding one.
And another thing you can do is you can always add one for yourself.
You think, well, I actually have a plausible reason for one of these.
Well, I'm going to vote for one of them.
And every once in a while, you vote for a correct answer-- you know, one out of five times.
So the class learns.
But generally, you vote sort of randomly just to give support to the various answers.
So you're right.
The question was, is it better to not use zip ever?
And generally, it's true.
Unless you're dead sure no one voted for anything.
And it's hard to see in a large class.
That's one of the advantages of the clickers.
There's disadvantages and advantages of clickers.
And we can talk about them.
But you're right.
One way to mitigate that-- just throw in an extra vote.
Other questions?
So as I promised, what we're going to do with this example is now-- because this is actually not an aerodynamics class.
You may be surprised to find out now.
So you may have thought the purpose was to actually learn the air resistance formula.
If you do learn that, that's really useful.
And I'd be really happy, because it's a great result which is very, very, very hard to derive in pretty much any other way.
You certainly can't derive it from scratch from the Navier-Stokes equations.
At least no one with current mathematical knowledge knows how to do that.
So experiment really is one of the main ways of doing that.
So it's a great result for that.
But what I want to use it for is deconstructing it, looking at the way it was taught and why all the things were done that way.
So in that discussion will come out general principles that we'll use throughout the semester in all of our examples of how to do teaching, like for example, how to make a lecture, how to design a course.
So as always, those principles are not very effective if I just say, well, here are the principles, and we're going to use them.
It's much better if we actually try them out in example and see where did they show up by constructing them.
So the question for the next portion of this session is to identify all of the teaching elements in this.
So for example, what's a teaching element?
Well, for example, there was a vote or there was a second vote.
So just deconstruct it to as fine a granularity as seems reasonable, and say, OK, this was done, or this happened.
And then, why?
So two questions-- do a two-column table.
What and why?
And there'll be a pretty long list because there was quite a lot of things done.
So find a neighbor or two again.
We'll discuss the what and why.
And then I'll clump the whys into principles.
So any questions about that question?
Due to technical difficulties, the rest of lecture one is unavailable.
What follows is a summary of the material presented in the remaining part of the lecture
Think back to the question about the cones-- which fell faster, or were they the same speed, the big cone or the small cone, and how that question was asked.
What were all the elements in the way the question was asked?
What was discussed?
What did you do?
So it was scripted very carefully.
And there were reasons for the elements of the script.
So we've had a chance to think about that.
And let me list many of these scripting elements and their reasons for them.
So on the left side, I'll list the elements.
For example, there was lots of interaction.
You were talking to each other.
You were asking questions.
In fact, you weren't just asking questions.
You were voting.
So twice you voted.
So voting twice, discussing.
So what are the reasons for that?
What are the educational principles behind designing a question that allows lots of interaction, voting, discussing?
Well, first of all-- the why-- it gives the teacher, it gives me, an idea of how the class is thinking.
If, for example, I ask a question, and in the first vote almost everyone gets it right, I know, OK, everyone pretty much understands this idea or this concept.
I can spend very little extra time on it and move on to something that may be more confusing or more problematic.
Whereas if most people get it wrong, I know, OK, either the book was confusing or what I explained already was confusing.
And we need to spend more time discussing the core idea here.
So it gives me instant feedback about where the class is.
I don't have to wait till the end of the term to find out on the final exam, oh, no one understood that.
I know right there when I have a chance to do something about it.
The voting also has another benefit.
So the first benefit is it tells me.
Also, it makes this students form a public commitment.
And that public commitment has a strong benefit, which is that the students become much more involved in learning what the outcome of the discussion is.
Because they've actually made a public commitment to one choice or another choice.
So it increases the level of engagement.
Similarly, it creates tension.
People want to know what's going to happen.
So when the demonstration is actually done-- the final decider is the experiment-- everyone is paying very, very close attention.
So I'll say not just tension, but I'll say it creates dramatic tension.
Furthermore, the chance for lots of interaction increases the amount of interaction in the whole class.
For example, students will ask questions much more easily if they've already had a chance to discuss their ideas in small groups.
So that's one set of elements and the educational reasons for building those in to how one asks questions in class.
Another one is that I used a physical object, in an actual demonstration, so these cones.
And the physical object is also cheap.
Anyone can make those.
You just cut circles out of paper, cut a quarter away, and tape it together.
You have a cone.
What are the benefits of that?
Well, first, it's much more convincing than if I just do a derivation from the Navier-Stokes equations or however one gets to the result.
And say, OK, well, therefore the drag is proportional to the square of the speed.
Or therefore, they all fall at the same speed whatever their size.
If I just say that, or even if I prove it with equations, that's not nearly as convincing as seeing it actually happen.
And that's where in terms of being convincing, it's also important that the demonstration be cheap.
If, for example, to see it actually happen requires $100,000 of equipment-- well, who knows what happens in that $100,000 of equipment and sensors and test devices and whatnot.
Whereas if it's a cheap piece of paper you can cut out for $0.02, well, then anyone can try it themself, too.
And it's not so mediated by lots of sensory devices.
It's much more immediate.
It's much more convincing.
So I'll say here in parentheses, a cheap physical object-- ideally, if possible.
Furthermore, the physical object actually connects the mathematical and physical and engineering ideas to the physical world.
In the end, we want to try to understand how the world works, maybe put it back together in ways of our choosing.
So purely living in the formal world, we're living in the clouds.
By bringing in a physical object, we actually put our feet on the ground as well.
Another part of the demonstration of the script was various things that might have seemed extraneous.
For example, I mentioned that I hope my socks were matched well when I stood on the table.
I also discussed-- when I was standing on the table, I said, OK, I need the class's help here because I don't have depth perception.
So I can't tell when I'm dropping them-- when I was dropping one cone versus four cones, I can't tell when I've got this one twice as high as this one.
And so you need to tell me.
So I talked about my failing in the depth perception department.
So both of these might seem extraneous to engineering and science.
But actually, they have a sound educational reason.
So the socks joke-- well, that makes the teacher seem much more human.
Similarly, for the lack of depth perception-- oh, well, the teacher has flaws, too.
They're not this perfect person who seems to know everything.
And in fact, that one, specifically because I didn't have depth perception, I needed the help of the class.
So that, again, is an excuse to involve the whole class.
So no depth perception makes a teacher human is also an excuse for more engagement.
And related to discussing another element of the script, I want to amplify this one up there, the discussing.
And I'll put it back down here.
The discussion happened first by yourself, then in a small group, and then in the whole class together where people were suggesting reasons.
So I'll say discussions at several scales-- self, group, and class.
So what's the educational purpose of that?
Well, I mentioned one purpose up there, which is it increases questions.
And that's certainly true.
Another one is by allowing people to think for themselves first, you're giving space for what's called the introverts.
Some people like lots of time to think for themselves, and only then do they want to share with other people.
Because otherwise, their thoughts get derailed by what everybody else is saying.
Some people don't know what they're thinking till they talk to other people.
Let's call them the extroverts.
So by allowing space for self talk and group talk, you're teaching to the introverts and the extroverts.
So you're matching to the learning styles of different students in the class.
And then by scaling it up slowly from self to group to class, you make it safe.
You make it not a huge transition to start to go to the next stage.
Whereas if I just ask the question of the class, OK, whole class, now, you tell me what you think about this question and I skipped these parts, only the usual suspects would be offering me ideas here.
The people who always raise their hand-- maybe they sit in the front row.
Whereas by allowing people to think for themselves, then discuss in a group the ideas that they thought for themselves, check out what they've done there.
If they still think they don't understand something and no one in the group understood it, then they feel safe asking the whole class.
We're actually building a bridge for people to participate in the large group.
So again, that point's up there.
But it's so important I'm putting it down here, too.
So I'll call it builds a bridge to collective participation.
And that participation can be people contributing ideas or questions which are themselves ideas.
Another feature of how the whole thing was done was that I walked around a fair amount.
I used a lot of the space.
For example, even some of the vertical space, standing on the table.
Moving over here, moving over there to make sure I heard the question from that side of the class.
So filling the lecture space.
Well, what's the educational reason for that?
Well, in a way it's the opposite of the socks joke.
In one way, it's the opposite.
In another way, it's the same.
It's showing that the teacher is human and that teaching is a human activity.
This is what actors do on stage.
The stage, the whole stage is there for a reason.
And the actors will use all of it.
They won't just stand in the middle the whole time, except for rare comedy monologues or monologues.
But otherwise, all the pieces of the stage are there for a reason.
And the actors use it.
They make that space their own space.
They fill the lecture room space.
So as a teacher, you want to do that because that shows that this is your space.
You're comfortable there.
And it increases your credibility with the students.
And similar to this, it brings in students from all parts of the classroom, especially if it's a big classroom.
They all feel somehow connected to what's going on.
Because you haven't just stood in one corner of the room or just paid attention to one section of the class.
So in one way, it's the opposite of the socks joke.
Because in that way, the teacher is being a bit self-deprecating.
Here, the lecturer is acting very confident by filling the space.
But in another way, it's the same.
They're both playing on the human part of teaching, which is inescapable.
And if it's neglected, the teaching falls flat.
And finally, there was one other point which may seem like a mechanical point.
But it's so useful that I'll put it here, too.
And I'll write it with big chalk.
Point is, I wrote everything on the board using big chalk.
And why is that important?
Well, in a small classroom of maybe 10 or 15 people, it probably doesn't matter.
Everyone can read everything.
But if it's at all a large classroom, people can't read the writing way in the back.
By using big chalk, you increase the signal to noise ratio.
Now, that may not seem so important when you first think about.
The reason being is that you already know what you're writing.
So even if you see really thin lines, something that's not very clearly written, you still know what it should say.
And you just read it correctly.
But the students, on the other hand-- they don't have the knowledge that you have.
So they can't error-correct any-- fill in the missing gaps in the lines.
They need as much signal as possible to really copy down anything correctly, to make sure that they're not adding noise onto the signal that you're trying to get across.
So in some ways, a small point, but in other ways, an important point.
Because if you don't do it, you'll throw away most of your other benefits because the students will actually be incorporating wrong information, just errors, just by mistake without even realizing.
And the big chalk can actually minimize that.
So just a mechanical point.
I always keep pieces of big chalk in my backpack.
And wherever I happen to be teaching, I can always pull them out and use them.
Because you will find many classrooms only have standard chalk provided.
And the big chalk is something you have to do yourself.
So just carry a few pieces with you as you need.
So those are some of the many scripting elements in this example.
And you'll see a similar example in a subsequent lecture, in the lecture on teaching interactively, which is lecture six.
And you'll see some of these points come up again and some of these reasons discussed again in a different example.
So you'll have two different engineering examples, but done in a similar way.
And you can see the scripting elements and how they're different and how they're the same in both examples.
And see which ones you can use in your own teaching.
So now, from these scripting elements, we've extracted teaching principles.
So credibility, signal to noise ratio-- just remember those.
And then here's a bunch more.
I want to summarize them, extract the main highlights from that so that you can come away with a clean picture of what to watch for throughout this semester.
So the main themes of all of these are-- I would say they fall into three categories.
Let me list them.
And then I'll break them into three categories.
There's ways to promote questions and discussion.
There's using stories and humor.
And demos and, related to that, things you can see-- visual examples.
And another pair of environments-- credibility-- filling the lecture space-- and what I haven't directly said yet as such, but we'll call it a safe environment for learning.
Creating a safe environment for learning.
That's what discussion at several scales enables.
You create a safe environment for people to think for themselves, which then makes it safe for them to think in a group and discuss in group, which then makes it safe for them to think in a whole class.
And so that makes safe bridge, or two bridges, towards collective participation.
So you've made a safe learning and questioning environment.
Without this environment, that won't happen.
So I put these in three columns because I think they fall into sort of three pieces of the brain.
Questions and discussion-- sort of left brain.
Stories, humor, demos, and visual-- that's right brain.
And credibility and a safe environment-- I'll call that the emotional brain, the amygdala.
Maybe you could put humor over there, as well.
But the reason I put preamble stories with the right brain is it's a different kind of learning than we think of when we think of left-brain learning, for example, learning an analytical method or the sequence of steps or a recipe.
The learning that happens from stories-- for example, about the depth perception-- is a different kind.
It's kind of implicit in a way, more perceptive and the holistic.
So I put it in the right-brain category.
So these are main themes in creating your own classroom teaching environment and structuring how you're going to teach.
And sure, there's also all the questions of content.
But the reason I'm focusing on all of these is that the questions of content-- those have been done so many, so many times.
You can find all the content in a gazillion different textbooks.
But the question is, how do you integrate all of that content into something that makes for good student learning?
And to do that, you need to take account of all this, all of these features, all these aspects of the brain.
Because humans are not computers.
And they can't just learn by being programmed.
So from all these themes, extract one giant theme, which is that teaching is not necessarily equal to learning.
Just standing up there and teaching stuff is no guarantee that students will learn it.
You need to pay attention to all these features, all of the human interaction to make sure that learning actually happens.
So that's, if anything, the main theme.
Or phrased another way is that meaning and knowledge must be constructed by the learner.
And all these ideas up here, these scripting elements, these principles here are ways to help that construction happen.
Just telling them doesn't do it.
You have to enable, facilitate, that construction.
So what we're going to do for the rest of the semester is we're going to see these principles, these principles, these boiled-down themes, this idea, this large idea-- the big idea of the entire course exemplified in several topics and areas that are central to teaching science and engineering and mathematics at the college level.
And so the outline for the rest of the semester is as follows.
Today was lecture one.
And that was general principles of teaching.
The next one will be on how to teach equations.
Equations are central in all science and mathematics and engineering teaching.
So how do you teach equations-- in other words, some of the central content-- taking account of all of these ideas?
Can you tell stories when you teach equations?
How can you do that?
Well, we'll see.
The third lecture, the third topic, is taking account of students' misconceptions and thereby avoiding rote learning.
So you can see already where some of the ideas that we've been talking about here are going to help here.
For example, having lots of interaction and voting-- you find out where the students are.
You'll find out if they have funny ways of thinking about stuff, maybe even predictable ways, but ways that don't help them actually solve problems in the real world.
Well, how do you take account of that?
There's one way right there already.
Then designing homework and exam problems.
And the fifth one is course design.
How do you structure a whole course?
Sixth one is teaching interactively.
As an example, you've already seen one example, which is this kind of question.
Well, what are the general principles behind such questions?
Can you make longer versions, shorter ones?
Are there short ways of building in interaction and questions into learning?
Then what do you actually do when you come into the classroom?
Lecture planning and performing-- how do you plan one lecture?
So this is the overall structure of the whole course.
How do you plan and perform one lecture?
Topic eight is teaching with blackboards and slides.
So what are the advantages, generally speaking, of blackboards?
But what are situations where slides are actually a really helpful way to teach?
Slides meaning-- well, it could be old-style photographic slides, but pre-prepared PDF files that projected on the screen with an LCD projector is generally how it's done now.
And then how do you plan blackboard work?
How do you construct slides so that they're effective in creating learning?
Ninth is sort of a result of all these.
Well, if you're going to apply all these principles, you'll find yourself wanting to change a lot about how the course is done or how it was done previously.
Well, what are the barriers to that?
In particular, what are the political barriers to educational change?
So that by being aware of the barriers, you're more able to go around them, surmount them, take account of them, plan for doing it bit by bit by bit.
But eventually, hopefully designing courses in ways that you think are good for promoting learning.
And then the 10th topic will be a summary of everything, using the examples we've done the whole semester and a chance for-- well, you'd ask questions about the whole course.
OK, so that's the themes of this entire semester of teaching college-level science and engineering one topic at a time.
See everyone next week
Answers from lecture two to questions generated in lecture one
What I'm going to do is answer your questions from last time on the feedback sheets.
And I'll pass out a new set of feedback sheets in a moment.
So I should say that the questions were, as always, excellent.
And I learned a lot by thinking about them.
And I hope that you'll learn a lot as you hear the results of everyone else's questions together, and that they reflect concerns that maybe all of you had at different levels and the question that you may have put down may not have been all of your concerns.
But collectively, hopefully it captures most of them.
So I'll categorize the first section of questions as in reality.
I've typed in all the questions.
I'm going to put them all on the course website.
So you can see the questions.
And the answers will be here, but you'll see the questions.
So in reality-- one question along that line is, what about grading?
How do you actually implement your grading policy as an assistant professor in a traditional department?
So my grading policy is generally that, especially in a pass/fail course, that the grading be very minimal.
And in general, I think that's a good policy for grading overall.
And we can talk throughout the semester about why and the research behind that.
But how would you do that if you're starting out new?
And the general rule is there's no right answer.
For example, one solution-- it's not quite the answer to the question-- is to take a position at a place where you're the department head.
So actually, I was offered a teaching position like that after I finished graduate school.
And I thought long and hard about it.
So I would have been the head of the physics and math department.
Now, that was the good side.
The bad side was that I was the only member of both departments.
And so on the one hand, I was a department head.
But also, I was the only one doing all the teaching.
And I would have only me to boss around.
But in that situation, you do have a huge amount of freedom.
And I could have done pretty much whatever I wanted.
Now, I decided not to take that position because I really wasn't sure exactly what I wanted to do in terms of improving teaching.
And I thought, well, I really need some more experience.
And I decided to do a post-doc.
But that is one solution to that problem.
It's not a very generalizable solution.
There are not many positions like that.
Another solution is to what I call the Clark Kent solution.
So you go around as a traditional faculty member, in the Clark Kent outfit.
And then as soon as you get tenure, foom, off comes the suit.
And everyone realizes that oh, actually, we hired someone else completely, who actually has all these interesting ideas about teaching.
So that is generally legally protected to some extent.
And you won't easily be fired, although a colleague of mine in Canada-- the university is trying to fire him for giving all As in a course.
So this does speak to this question.
And it shows it is a tricky issue.
So he believes-- and I agree with him-- that grades generally just produce obedient people.
One of their main functions is to produce obedient people and obedient students, rather than students who love learning and want to question.
So he wanted to teach one of his courses-- he's a tenured professor of physics.
He wanted to teach one of his courses pass/fail.
And the university said, no.
By university regulations, that course is graded only.
It has to be taught with grades.
So he said, OK, fine.
On the first day of class, he just told everyone they were going to get an A+.
And according to the TA-- at least, this was then reported in Canada's Globe and Mail, their sort of New York Times-- one of the teaching assistants for the course said, yeah, there were a few students who took advantage of that fact.
But most actually did more work and were very interested in the material because of it.
So I would say, basically, the thing was a success.
And that's generally what I find, that the less emphasis you put on grades-- especially at a place like MIT-- the students' natural interest in the material comes out.
And they're more and more willing to work and learn.
And that produces long-lasting learning.
Because they're doing it for their own reasons, not because you stood over them with a grade and beat them over the head.
So that was good.
The bad part is the university basically said, well, you're defying us, and you're going against the purpose of the university.
It's not really clear what the purpose of the university is, except maybe to produce obedient people.
So they said, we're going to institute proceedings for your dismissal, to strip you of your tenure and fire you, which is a big step for a tenured professor.
And meanwhile, they banned him from campus.
So he's now not allowed on his own university campus.
So then he came to campus for a film screening, because he runs a film series on the campus.
So he came for that, and the university police arrested him.
So he was arrested for going to his own university.
So then it got into the national press in Canada, which doesn't make the university look so good.
But that's all by way of saying that grading is a very touchy issue with universities.
And it's really not clear what to do about it, because it's so embedded in the system.
So I can't really offer you any solutions to that question except to say that it's a very tricky negotiation and you want to plan your activities carefully.
If you're going to use a lighter grading system, don't do it all at once.
Get people around you to accept it slowly, slowly.
Get students on your side slowly.
Do things slowly.
Don't make any sudden moves that get yourself fired.
Work with other people of like minds.
Work together collectively.
So it's not a long-term instant solution.
But any kind of social problem-- and I think grading is a social problem-- doesn't have any instant solutions.
So I wanted to answer that one first, to basically tell you that I don't have all the answers to all the questions.
And I want to give you honest answers as far as I know them.
OK, next question about reality.
You've obviously done this many times before.
As someone who needs to figure out a logical order for the material in addition to balancing preparation time and research, how do we juggle all this in addition to simply focusing on style?
So I want to dispel one misconception was that last time, I wasn't saying that you can substitute style for preparation and really knowing the material.
It's really that I wanted to up the importance that people give to what people would call style by saying that the way you present, how you present, is as important as what you present.
In fact, it is, to some extent, what you present.
If, for example, you're very interested in the material, and you present it in that way, well, the students are very naturally going to become interested in the material.
As a test of this, you can actually do little home experiments on this principle.
So just go around to somebody.
You can do these little like Candid Camera experiments.
Just go up to them and smile, just like this, and then watch what they do.
And a whole bunch of people actually just smiled back at me for no reason at all, just because I smiled.
Now, not no reason at all.
It's because we're social creatures.
We're programmed from millions of years of evolution to pick up other people's emotions and ways of being.
So if you're enthusiastic about the material, if you're interested in it, and you present it like it's important, well, that actually gets transmitted very effectively and picked up at the other end very effectively.
So actually, that is part of the content.
So I wouldn't say style is separate from the content.
How you present, how you think about the material is a crucially important part of the material.
Because that's when students are going to be taking away.
And that's going to be affecting how they think about the material.
So it's not just focusing on style-- that sort of gives it too little importance.
Instead, yeah, how do you think about all those issues as well as surviving your research grant?
And again, there I would say the way to do it is to start slow.
When you're an assistant professor, you need to focus on your research to get tenure.
So focus on your research and just try small teaching experiments bit by bit.
Maybe in your first year, just try putting in one small conceptual question per lecture or per week.
And if that goes OK, maybe try longer ones.
Try more the next semester or the next year.
So just try small things.
And as they work, grow them.
So that's one way, so you don't have to sacrifice your research and hose yourself.
So preparation time, as well-- so try to borrow notes from colleagues.
Try to get as much from other people as you can share.
Talent invents, genius steals.
So steal all the good stuff around you.
So another one is how does this work in the real world, which is basically the same thing.
So what I'm trying to do in this class is show you general principles that you can use always to guide yourself in as far as you want to go in your teaching.
So now how far you go in your particular situation depends on your situation and how much importance you put on doing new teaching methods, new experiments right then.
But I'm not saying you have to start everything tomorrow, like the first day you're a TA, all of sudden throw out all the regular stuff and do everything like I'm demonstrating now.
I'm showing you this as examples to illustrate principles for directions you could go and for threads you can pick up as you see best.
Syllabus and logistics for this class-- any exams or problem sets?
Yeah, there's problem sets, which you've seen one already.
No exams.
And so how are you being evaluated?
Basically again, very light touch.
Basically, just make a reasonable effort on the problem sets, and you'll be fine.
I'm hoping basically that the material's interesting enough that you don't need a stick beating you, saying, oh, you must do this, you must do that, you must do this.
It's not a required course.
People are taking it really because they're interested.
So I'm not going to try to ruin that with a whole bunch of grading sticks.
Does it matter which subject number we're assigned for?
No, and I don't even remember what the subject numbers are.
I know 595, and there's a whole bunch of them.
So I just write et cetera.
No, it doesn't matter, as far as I know.
Could we have a break after one hour?
Yeah, I think that's a good idea.
So I'll give everyone like a two- or three-minute break at the hour just so you can stand and get your blood flowing again.
Biology examples-- can we have biology examples?
A fair point-- so actually, today our first example of equations is a biology example, amazingly.
OK, so now, interactive teaching-- how do you teach a class interactively if you have a lot of material to get through?
This is a very interesting question.
It comes up quite often.
And the answer is that you don't get through it in the lecture.
So the lecture is a terrible place to get through lots of material.
In fact, even the word "get through," "cover"-- those all have the wrong sense to them.
And so Victor Weisskopf, MIT physics professor for many years, he said-- and I think it's a great saying-- instead of covering a lot of material, I prefer to uncover a few key ideas.
So uncover is the right verb.
Cover has the sense of smother.
You can't see it anymore.
And that is what happens when you cover too much material.
You want to uncover a few ideas and let them bloom.
So a lecture is something that's paced at human scale.
You talk at maybe 150 words a minute.
People can't write that fast.
They can write maybe 15 words a minute.
Reading is much faster.
Reading-- people can read 300, maybe 400 words a minute.
So if you have lots of material to get through, don't fall into the trap of putting it all on the blackboard or on the overhead slides.
And we'll talk about that in the session on making good slides and using the blackboard.
You want key ideas to be there.
And you can leave lots of the details for reading.
So that's one answer.
The other answer is that sometimes, actually, I think one of the fundamental problems with so much teaching is that we try to cover too much material.
And one of the goals should be for good teaching is to reduce how much material one covers.
And there's a simple estimation argument to show why that's important.
The estimation argument is that basically, suppose you cover 100% of the material.
But because it's so much material, students remember only 10%, and really learn only 10%, which I think is actually a reasonable estimate.
Because they're just so overloaded.
They have no time to make connections, and it just flitters away.
So 10% is all they retain.
Well, you could cover, say, half the material.
And let's say that they still don't remember everything, even though you've done half.
Maybe they remember half of that, because they have time to struggle with it, make connections.
So now, they're at half of half, which is 25% retention instead of 10%.
So that's a plausible model.
I'm not claiming those figures are actually based on exact research or anything.
But just a plausible model for how reducing the amount of stuff you cover can actually increase learning.
So try to reduce material when possible.
Almost always, there's too much material.
OK, so again, how do you pace lectures?
If you do too many demos and you have less material, advanced students might get bored.
Actually, I find advanced students are actually quite interested in demos and questions like this that we used last time.
They want to really understand what's going on.
And it's those kind of questions that help them.
Covering lots of material-- again, they can read it in a book much faster than you can write it on the board.
So put it there.
Can these principles be applied to upper-level graduate classes?
For sure.
You can apply them all the way from kindergarten through graduate school because they're really based on how people think.
And they're based on common ways that people perceive the world and reason.
And that doesn't change-- I mean, life does change when you become a graduate student.
But your whole way of looking at the world doesn't change that much.
So how do you deal with classes where someone dominates a discussion or the class does not like to participate?
Sort of two ends of the same coin.
So what I like to do is if someone keeps trying to say stuff, I just say, oh, well, actually I want everybody to participate.
Could I hear from someone I haven't heard from?
Just with a light touch.
You don't have to make a big confrontation in class about it.
If people don't like to participate-- for example, suppose you ask them to discuss with each other and they don't.
So here's a trick I learned actually from one of the teaching seminars when I was in England, which is that you just turn your back.
And why do you do that?
Well, your back is not as interesting as your front.
So students are actually taught with, say, 12 years of teaching to watch the teacher, watch the teacher.
So one of the reluctances to talk to each other is they're watching the teacher, figuring out what the teacher's saying and doing.
Kind of like Clever Hans was watching his trainer-- do I do more clips on my foot?
Is that 42?
6 times 7-- OK.
So students are actually taught, basically, after 12 years of conditioning, do that very automatically.
So you just have to break that conditioning a bit.
If you just turn around, there's nothing really that interesting to look at, unless you're maybe-- I don't know, some fashion model or a GQ model, which I'm not claiming to be.
So then the students will just naturally be more likely to talk to each other.
So you create the conditions for that.
Another one-- and I often use this trick, because it's actually not a trick-- is sometimes I'm thirsty.
I just walk out of the room and go get a drink of water.
So if it's an interesting enough question, the students will be talking to each other about it.
And there's definitely not you around there, watching them or worrying them, for them to focus on.
So that's one very useful trick.
Generally, though, the overall point is if the class doesn't want to participate, you just make it more and more safe to participate.
You make sure you never criticize people if they say something that wasn't right.
You make sure you value the wrong answers and you explain why.
So all of the things I talked about last time about safety means you just have to do them double if people aren't participating.
How do you cope with a variety of confidence levels among students?
Also interesting.
So there, I find actually that most students aren't very confident.
They have different levels-- they're all generally much lower than you think.
So they do have some different levels, but it's generally overall lower than you think.
So I spend more of my effort making sure everyone feels comfortable.
But again, if some students are much more confident than others, I try to mix up the level of questions that I ask.
I ask some questions that I know most people are not going to be able to answer and some questions that I know most people will answer, just so everyone has something.
And then on the problem sets as well-- warm-up problems, regular problems, really hard problems.
Like in Donald Knuth's textbooks, the fifth set of, kind of, problem at the end of the chapters is unsolved research problems.
And you never know.
Someone might actually solve one of them.
So you can put something for everyone in there.
What about the left brain?
Surely humor and demos can't take you all the way.
Yeah, and that's true.
And so my answer to what about the left brain is that the left brain is what basically most teaching has focused on for most people's experience over the last 15 or 16 years.
So I'm not saying, forget about the left brain.
I'm saying that you basically already know how to do the left-brain side of teaching.
You have so many models of that.
It is pretty much what we think of as teaching.
You go up to the blackboard, and you write down a series of symbols one after another, write down a series of equations.
So you already have models for that.
And we'll talk about how to improve those.
But I'm not so worried about that part.
What I'm worried about is the neglected part.
So this you could think of as the balance.
And then you'll be able to find ways of integrating the left brain.
And we'll do left-brain things, equations.
But how do you integrate that with all the quote "right brain," the humor and the demos?
How do you project confidence?
Fake it till you make it?
Yeah, that's basically right.
So one way is to actually make sure your voice is clear.
So to do that, my piano teacher in graduate school-- actually, she was a voice teacher, as well.
She gave me a really useful exercise, which is-- I do this for like 30 seconds before I come to lecture.
So the exercise is you first just close your lips.
And you mmm, you hum.
So we'll all do this together.
So you hum until you can feel it vibrating in your nose.
That's step one.
So let's all do that together.
[HUMMING] And if you have hay fever, you can tell if you're doing it right in the spring.
Because you'll dislodge pollen grains and you'll probably sneeze.
That's what I find in the spring.
When I do that, I actually start sneezing.
My theory is that I'm dislodging pollen grains and then I'm sneezing as a result of them.
So mmm-- so now you've got a resonance up here.
And now what you want to do is with the same mmm sound, now do it in the back of your teeth with your mouth closed.
So you feel behind your teeth.
So we'll do step one and then step two.
Mmm and then the teeth.
Mmm, teeth.
[HUMMING] And you should feel in your lips vibrating, kind of tickling.
OK, so that's step two.
Step three is to then make the vowel sound that is most likely to make your mouth resonate and be really wide and open.
That's E. So we'll go hmm and them mmm and eee.
OK, so nose-- [HUMMING] lips-- [HUMMING] eeeeeee.
OK, that's step three.
And then the final step is you just add some words.
You modulate it with words.
All right, so I'll give you a demonstration.
Then we'll do all of it together.
[HUMMING] Eeeee-- (SPEAKING ON SAME NOTE) how are youuuuuu.
OK, so now, that sounds a bit strange.
But you find after you do that, your voice is much more resonant.
Actually, I can hear it right now.
To my own ears, my voice sounds much more resonant and clear.
So let's do that.
[HUMMING] Eeeee-- (SPEAKING ON SAME NOTE) how are youuuuuu.
So you'll find, actually, what you've done is you've shifted your voice from here, where it just makes your throat tight, to the front.
And just by doing that, you will project.
And by projecting, you actually sound more confident.
And you'll feel more confident.
So that's one way.
But another way is to be confident of your material.
And another way is, as you said in the question, just fake it till you make it.
Now, there's a few more questions.
I'll answer some of them on the website.
But I wanted to answer one more.
How long did it take you to become such a good teacher?
So I'm glad people think I'm a good teacher, or at least one person thinks so.
But there's a general answer to that question, which is actually going to be one of the topics today, which is, how do you become an expert in anything?
And the first example I'm going to show you-- it's biology equations.
I'm going to use that.
And then after that, when I talk about expertise in general, not just in teaching, and where that comes from.
And I'll show you the results of some studies.
And then I'll come back to that question.
And I'll hopefully answer it then.
So those are I would say 2/3 of the questions.
The ones I haven't answered I'll try to answer online.
And I'll pass out these in just a moment.
But right now, what we're going to do is we're going to do the first example of equations, which is from biology.
Now, before I do that, any questions that have occurred to you meanwhile?
Did I create any new questions?
Yes
Very germane, but if this is P/D/F, are we supposed to be registered as P/D/F?
I think it's automatic.
I don't think there's any way not to register as P/D/
OK, thank you
So the question was, if this is pass/fail, do you have to specifically register as that?
I don't think so.
I think it's automatic.
So yeah, basically, don't worry about your grades.
Everyone's going to P.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Today, teaching equations and how to teach equations in a way that promotes long lasting learning and understanding.
OK.
So the first example-- I'm going to give you two choices for starting the example.
So this is example one for teaching equations.
This is Hardy-Weinberg equilibrium.
So here's one way you could start.
So if a locust has n alleles and the organism is polyploid-- so that means it has, let's say, C copies of chromosomes.
OK.
So there's n alleles.
So the allele frequencies are P1 through Pn.
Then we're going to deduce the genotype frequencies.
So this is the multinomial middle coefficient C. Choose K1, K2, K3, all the way to Kn.
And K1 through Kn are the number of copies of each allele with K1 plus blah, blah, blah, to Kn, all equal to C. So you could actually state that and then prove it.
That's option A.
So now I'll give you option B for starting the same topic.
So option B is from Hardy.
So in Hardy's Mathematician's Apology, he writes, I have never done anything useful.
No discovery of mine has made or is likely to make, directly or indirectly, for good or ill, the least difference to the amenity of the world.
That's a very interesting, very bold statement.
And amazingly, for such a brilliant mathematician, it's wrong.
And today, what we're going to do is we're going to look at an example of the discovery by Hardy-- the Hardy-Weinberg equilibrium-- that does make a difference to the amenity of the world and helps you actually understand genetics.
OK.
So the option B is to quote Hardy and then point out the contradiction between what he said and what we're going to do.
Now, also, in option B, you can go a bit farther.
You can say, well, why would Hardy have said something like that?
And indeed, why would he have said something like that in his Apology?
Well, it's actually probably hard to appreciate on this side of the Atlantic, because since the Civil War, we've never had a war that just basically devastated entire countries on our own soil.
Whereas in Europe, the memories of World War I are very strong.
So if anyone's from Europe, you know that.
In every English chapel, there's names on the wall of all the people who died in the, quote, Great War.
So World War I had a very strong effect on European society.
And one of the effects was on European science and people's attitude towards science.
So poison gas was invented partly by German chemists.
Hopper was one of them.
And in the end, he committed suicide, partly maybe because of what he had done.
No, sorry.
His wife committed suicide.
I forget if-- I should check that.
But basically, people in the family were so unhappy about what had happened just there that there was a suicide.
And furthermore, science after that was considered-- World War I was considered the chemists' war.
So there was a wish to distance oneself from those kinds of horrible effects.
And the quote from Hardy is actually a reflection of that.
He himself was very strongly anti-war.
He left Cambridge because Cambridge fired Bertrand Russell for protesting World War I.
So Hardy left Cambridge for a professorship in Oxford and only came back basically 12 or 13 years later to Cambridge.
So that statement of his partly a wish that he's hoping nothing he's done has any effect on the world because in his mind, maybe most effects are bad.
Well, but actually, he did have an effect.
The effect is Hardy-Weinberg.
Even if you discount everything you did in number theory, there's Hardy-Weinberg, and it has an effect and we're going to look at it.
So let me ask you rhetorical question.
If you're a student, which would you find more engaging, more inviting?
This one or this one?
Who votes for that one?
OK. Who votes for that one?
Yeah.
Now, why?
So now think of-- so the question I'm going to ask you to think about just for a minute with one or two of your neighbors is why.
So yeah, this is definitely more engaging.
In terms of the principles we talked about last time, what makes this way of introducing it more engaging, more likely to bring students in, more likely to make them want to learn about Hardy-Weinberg.
OK.
So find a neighbor or two we think about the principles behind option B. OK.
So I'll rudely interrupt you.
And good thing I did the voice exercise so I can project all the way into the back.
So what reasons for option B or against option A, which is the same thing?
Yes.
Can you tell me your name?
Brian
Brian.
I'm going to try to learn people's names.
So go ahead
The reason against option A-- when you start out that way, you are activating the ability that students have to transfer material from the blackboard to their notes and then pass it to their brains
Right-- through the wave.
Yeah.
I'm activating the ability of the students to basically take dictation.
Or as someone said, teaching is an excellent way to transfer material from the notes of the teacher to the notes of the student without it passing through the minds of either.
So I'll call that not A.
So that's the not symbol.
Not A is-- so I'm basically asking to do dedication which doesn't necessarily lead to any kind of learning.
They could, for example, copy all of that down but not really understand of it.
OK. Other-- yes, can you tell me your name?
Susannah
Susannah
This is right brain?
So it's easier to remember and ingest it.
And maybe even identify the theorems
OK.
Right.
And so B is the story.
I'll give you an example of how stories can be so powerful, and just even the word.
So a big commercial publisher-- this was several years ago-- they wanted me to write a freshman physics textbook.
I had actually just put up a proposal on the web saying, we should write a freshman physics textbook that is based on the history of science.
And they saw it and they thought, oh, that's great.
So they flew me from England to California to talk to them.
And they liked it mostly.
But they said, history-- so there was the word.
They liked the idea as I talked about it, but the word history really frightened them.
And then, I'm still amazed-- I had this amazing insight of just two letters.
I thought, oh, wait.
I can actually explain to them what I mean.
I said, well, actually, if you just take away that part, what I'm really talking about is that.
And it was interesting.
As soon as they saw the word story and they saw that history doesn't have to be names, dates, facts-- I think that's what they were seeing it as-- they saw it as story, which is actually its origin in French or Latin-- [FRENCH].
They all of a sudden were totally convinced.
They said, oh, yeah.
That's exactly how we should do the textbook.
For various reasons, basically because I wanted to be a freely licensed book, we didn't actually sign a contract.
But it's an example of how powerful story is and how people who actually spend their lives thinking about teaching and reaching students-- in other words, this educational publisher knew the importance of story.
But if you present it as just dry history, it's not so interesting.
So story-- yeah.
So let's see.
Who haven't I heard from?
I haven't heard from Adrian.
Yeah, and then you're next
So the fact that it's a contradiction creates some sort of tension from linear thought.
And it's more of a exploratory
OK.
So there's a contradiction.
And the contradiction creates some kind of tension.
So the tension creates interest.
So every good story needs some kind of tension.
So the tension here, the contradiction, is that Hardy wanted to do nothing that could harm people, basically.
And so he was like, OK, well, anything could be used, even if it's for good, for ill.
So let me just back off from all of that and say I'm not going to do anything that has an effect, an application.
I'm a pure mathematician.
His most famous book is A Course of Pure Mathematics.
Well, maybe his most famous book is that book that I quoted from, A Mathematician's Apology.
His most famous math book is probably A Course of Pure Mathematics.
And that's what he wanted to be known for.
And that's in contradiction with the fact and no, even Hardy couldn't help an application-- an actually very, very common one.
It's taught in every single introductory biology course, probably.
OK.
So you need tension.
Now, that's a general principle of learning there-- the idea of story and tension and paradox.
So the one way that, as a undergraduate in physics, I learned a ton of physics, me and my friends were doing problem sets together.
And a lot of problems were just grinding through math.
So we didn't learn a hell of a lot from that.
But we were in the library late at night, the physics library, ordering pizza, and trying to do a problem set, and we just got to making up physics paradoxes-- perpetual motion machines.
We'd invent perpetual motion machines and try to get the other person to figure out what was wrong with it.
Sometimes, we didn't even know what was wrong with it and we'd both try to figure it out.
So from that kind of tension-- tension is almost, in a way, self teaching, because as long as the tension's there, you know you're not at the end of the story.
You know there's more to do.
So the same thing-- as long as there was perpetual motion going on and we hadn't found the reason, we knew we weren't done with the problem.
We didn't have to ask a teacher to say, well, is this right, because it was basically self teaching.
As long as the tension was there, we knew that we weren't finished.
So yeah-- another general principle, stories and tension.
Let's see.
There was a comment over-- yes.
Tell me your name?
Wing Ho
Wing Ho?
Yeah.
So this along the line that the teacher should make themselves more human, because Hardy-Weinberg sounds like such a big name you would tattoo.
That student may be driven to believe in the equation just by the matter of the fact that it's authority.
But then making a story makes--
OK.
So the teacher makes himself more human and makes one of the people who invented the Hardy-Weinberg equilibrium more human.
So it's actually something much more easy for the students to connect to.
Yes, can you tell me your name?
Gregor
Gregor
Yeah.
So my problem is that two different options are very, very different.
And the amount of legwork it is, it's maybe more tense to me, because you can discuss it without legwork, but I think also that option A is pretty interesting, because you can approach this biological problem from more of a qualitative approach.
So depending on what kind of learner you have and what kind of question you choose, A would be also very interesting.
But at that point, with the background I have right now, it's very difficult to pursue that
OK.
So Gregor's point, which is a good one, is that B doesn't have the same quantitative depth as A.
And actually, what I'm going to show you is a way to get to this starting from here and using the principles that we're talking about so that by the time you get here, or to something like this, it actually makes sense to the students-- so that you can have both.
So that's a promise.
And hopefully, I'll deliver on it for you.
But does that address what you're talking about?
Yeah
--that B doesn't have the quantitative depth-- and that's true
Actually, I mean, you can talk about one hour on B.
But if you really want to deliver an idea or a concept, that is as quantitative as option A, then you have to get to A at some point
So the point made is that if you want to get here, you do have to say this at some point.
You can't just say the story for now.
Well, it depends on the purpose of the class.
If the purpose is the history of biology, maybe you'd continue with the story or the history of science and war.
But if you want students to be able to solve problems in genetics, maybe you need to go here.
Or what we'll find is I'll continue with option B moving towards A.
And I'm going to show how you can get there and still have the opening of B preserved.
OK so let's see.
There was-- yes.
Can you tell me your name?
Roger.
I think one of the reasons we like B is because we see A all the time.
So I don't know to what extent that's [INAUDIBLE]
OK.
So B is new.
Right.
So against A, A is very familiar and common.
So in fact, yeah, I chose this-- I didn't really make a straw man here.
This is how a lot of things are introduced.
For example, in mathematics class, theorem proof-- so the theorem will be introduced without any of the struggle, the wondering that led to the theorem.
Why would anyone even care about such a thing?
Or if they cared about it, why would they come up with something like that?
How can you see that?
So yeah, A is seems very familiar and therefore, maybe not as interesting.
Although, it may also be intrinsically less interesting for the other reasons.
But I take your point, too, that it could be just familiarity.
Yes.
Can you tell me your name?
Yeah, I'm Meg
Meg
It seems that you could really incorporate the quantitative side into B if you wanted to by having the way the proof occurred [INAUDIBLE].
And then you wind up reversing the traditional order, where you have a proof and then a theorem, because the way it happened in reality and the way it happens when you're going to do research, and the way it happens to students when they're trying to work it out themselves.
And so I feel like that's much more of a natural flow [INAUDIBLE]
OK.
So your point, which is an excellent one as well, is that B doesn't preclude the quantitative.
What you could do is you could talk about the history, start with the history of Hardy saying that, which is not chronological, because he said that in 1940.
But that's OK.
It doesn't have to chronological just because it's history.
Go to 1940 and then backtrack to well, how did this problem actually come to Hardy's attention, which actually turns out to be quite an interesting story.
And then talk about how they solved it and what was the publication history.
And all the quantitative ideas would come out in that way.
And it would be almost backwards from the usual way.
The usual way here is completely general.
And that's probably not the way it was first figured out.
And the advantage of telling about it that way is that you're preparing students themselves for a research career, because nobody comes up with theorems full blown like Athena out of the head of Zeus.
You come to them from struggle and wondering.
Hm, I wonder.
Everything has some historical background to it.
So it's called the, you could say, the genetic approach.
So in biology, there's a slight, not misconception, but a saying that ontology recapitulates phylogeny-- in other words, that the organism, as it develops, say, in the womb, goes through all the evolutionary stages that it went through over the last 200 million years or whatever to become, say, a person.
You're at first a fish and then maybe you're a monkey and then you're a person.
So roughly speaking-- it's not exactly true.
But there's a lot of truth to that for learning ideas-- that you have to recapitulate the history of ideas to really understand where we are now, the ideas today.
So a great example of that is Newton's Second Law of Motion, the idea that force and acceleration are connected.
So for thousands of years, millions of years, people thought force and velocity were connected.
And it's actually force and acceleration.
So actually, you can guide people to the understanding of force and acceleration being connected by showing them the history of how people thought it was force and velocity, because for those same reasons, those are the reasons that students will think it, too.
So you're actually helping them overcome their misconceptions.
So the history actually, generally speaking, helps overcome misconceptions.
So I'll put that here.
And the history is actually quite interesting.
I think it was Punnett from Punnett squares.
He played cricket with Hardy and Hardy loved cricket.
And he just asked him about this problem.
And Hardy said, oh.
Yeah, no, it's just this, and sent off a paper to Science or Nature.
So basically, it's because they were cricket playing colleagues in Cambridge, that's how a mathematician ended up interested in a biology problem.
Punnett was a biologist.
OK.
So any other reasons?
Yes
I'm going to play devil's advocate for a second here
Sure.
Can you tell me your name?
Paul
Paul
So when I had biochemistry the first time, the first month of the class, the teacher would tell the history about the people developing quantum mechanics.
And that was the lectures.
We didn't touch the material one bit.
And because his philosophy was the material was in the book, you don't need to teach it.
You just go home and read it.
But I remember being in that class-- and maybe that's because I normally don't see things taught like that-- but everyone was outraged, because we didn't feel we learned anything
Right.
OK.
So the comment is that you were in an actual class that was purely taught about the history.
People were outraged.
They felt they weren't learning actual content.
And there's two answers to that.
One is that actually, I don't recommend if, say, you're teaching a physics class, that you teach it purely historically.
But to the extent that you do teach the history, you can actually teach the content through the history.
So it's not that the history and the equations or the ideas are separate.
It's that you can use the history as a means of making the equations come alive with a deeper understanding.
So it's possible that the teacher you're talking about actually didn't do that and just talked about the people who did this and did that.
But he didn't really know what they did or why.
So you're not forced to do that.
OK.
So I'll give you an example.
I'll continue with this in just a moment after I take any other questions, showing you how you can start with the history, and then you can show the content.
You can actually show people exactly why that would be true.
And I'll show you that in one second.
OK.
There was a question there.
Yes.
Sharon.
Oh, sorry.
Not Sharon.
Can you tell me your name?
Me?
No, in front
Cecilia
Cecilia.
Yes
I was wondering that Option B-- it's like targeting another audience.
I come to a class and I'm not very interested in the class.
I need to be motivated here, Genetics or something.
If I already signed up for genetics-- because if I were being in a class interested in that, I actually want to learn things.
I want to know the rules.
So I don't know if I want to be distracted by some story that is-- when you first said that, I was like, oh my god, if this is going to be some other big story, he's going to have the equation at the end.
I'm not going to be understanding what's going on or whatever you say.
And I think that could be a little bit of a turn off more, because-- try to make it look easy.
You have avoided writing the equation on the board
Yeah, I'm going to do that next
OK. And also, I guess there are two kinds of stories you can tell.
I don't know what story you're going to tell exactly, but this seems to be removed from the actual equation.
This is more Hardy's philosophy.
It's not really--
It's not directly about that
Something you did a second ago was more about the equation and more about the concepts involved with the equation
That's true.
So one question is how related should the stories be?
And there's a lot of freedom in that.
I would say the answer is that you want it-- if it does this, if it creates interest, then it's already done something.
Now, your other point was that it depends on the audience.
Suppose the audience is all graduate students in genetics or people who want a crash course in genetics.
Maybe actually, they want to just know the formula.
And they would actually be offended if you started telling them stuff that wasn't the formula, because they feel like they're time's being wasted-- a little bit like what you were saying.
So now, did you have a comment about that?
No
OK. Then I'll come to you one second.
So one comment about that is that actually, most of the students, at least in a big class in genetics, are going to be students who are-- for example, at MIT-- are there not because they love genetics and are going to continue in genetics.
They're doing it because they're required to take introductory biology.
And your job as a teacher is to actually show them that this is a really fascinating subject.
It's the same in physics.
It's often taught as if the only students in a physics class were the physics majors.
Now, when I was an undergraduate in Standford, there were 1,600 undergraduates and 12 physics majors every year.
So 1% of the student body was physics majors.
So they were actually teaching in a way, generally speaking, for 1% of the student body.
What about the other 99%?
It was really important for them to reach them as well-- important, also, for the health and growth of the field, because the field needed people who, even if they weren't professional physicists, saw the value in the field and the value in physics.
So generally speaking, it is actually a wise way to reach all of them.
Now, suppose you have people who just want this.
The next thing I'm going to do will show you a way of teaching to them that they'll actually learn this better, because I guarantee you, almost everyone, if you just tell them this and they take dictation, now they go away, and you say, OK, everybody, what did I write down?
Most people can't reconstruct it.
They'll say, well, was this a C or an N?
In fact, if you look in Wikipedia, the Wikipedia entry is incorrect.
Actually, I think it has an N over here.
It seems plausible, but if you actually understand the equation, you think, that can't possibly be right.
And I think it has an N over here.
So they actually do not really have a command of the equation.
So even for them, you want to not-- the story isn't the proof of it for them, but the continuation will be valuable for them.
So again, I'll promise that.
Now, there was a question over there.
Yes
The story, you can basically use to-- and if you see you lose your students when you go on this quantitative approach, then it's maybe helpful to loosen up a little bit and basically bought you interest to see what's a big context.
And the other point was--
So you're first point was that you can add stories as needed
Yeah.
So depending on what it's like.
And the other thing is when I make my comments related to the problem, I see it more from a various student point of view.
So you point out that, depending on what the audience is, you teach differently
Yeah, that's true.
Yeah
So when you give your examples--
Oh, I should tell you who the audience is
--interesting to know at a certain point what kind of audience it was
Yeah, OK. Good point
So I'm undergrad.
So I don't know how it is to teach first year university.
I've never had students.
So how much basics or how much story you have to put in compared to how much math.
So it would be very helpful to see what our target is
OK.
Your comment is that it would be helpful when I'll give you an example like this to say, OK, for the particular audience, what would you do?
Yeah.
And I'll try to do that.
That's a good point.
Now, there was another question-- behind Cecilia.
Did you still have a comment?
Oh, I just wanted to mention that in choice A, there's a lot of jargon
OK.
So there's a lot of jargon in choice A, which makes it much harder to understand.
You have almost-- for example, the multinomial coefficient-- you have to keep that in your head, as well as all these subscripts, keep it all in your hand and try to manipulate that object-- very difficult.
One of the points about that related is the chunking.
So the chunking paper-- the idea is that these, for a student, each of these things is going to fill up one of the chunk slots, almost, because it's all new to them.
You've flooded the chunking system.
You've overflowed it.
And they can't actually manipulate this as one object anymore, because it's far too many chunks.
Now, you, as a professional in the field, are like, well, of course.
It's a multinomial coefficient.
What else could it be?
For you, that's one chunk.
For the student, that's is it C or N?
K1?
K?
Why K?
What the hell is K here?
Is it commas or spaces?
Parentheses?
What about brackets?
I've never seen something like that.
Shouldn't there be as many things up here as here?
So the way they look at it is completely different.
Every symbol is almost a chunk.
So this is just massively overflowing with chunking.
So this is sort of related to the jargon.
Yes
Scott
Scott
I have a meta comment.
In this example, if you don't know how you're actually going to complete these, it's created a lot of tension.
I'm dying to know how you're going to do it
OK. Fair enough.
Would you be very offended if I gave everyone a break for a few minutes and then finished it?
OK--
OK.
So finish your comment
My question is in a situation like this where you have people who are making so many comments, and the other half's just dying to know, how do you know when to stop?
Good question.
So how do I know when to stop?
Because I don't want to exhaust people's patience, but I do want to take questions.
And in this case, there's the American teaching phrase saved by the bell, which is that I don't have to make the decision too hard, because it's probably time for a break anyway.
But how do I make the decision?
Partly, I listen to people's comments.
For example, suppose I got yet another comment saying, well, we still don't know the equation.
And I'm saying for the third time, oh, wait.
I'll show you a way to get to the equation.
I think, you know, it's probably time for me to do the equation and then I'll take more questions later.
So listen for the tension in people's voice, which is also a good reason to have everyone do the voice exercise first and free the tension so that if the tension creeps back in, you know it's something you've done.
OK.
So I'll take more questions after the break.
But it's 10:02 by that clock.
10:05, we'll start again.
You can jump up and down, do jumping jacks or whatever it takes to get the blood flowing.
I'll take a couple more questions and then I'll show you how to continue it to get towards that in a perceptive way.
OK.
So let me, as promised, continue along the lines-- not continuing the story, but continuing that approach, the alternative mirror image approach of A, which is to say, well, what would I do after telling the story?
So first of all, I would try to fix some of the problems in just basically blasting people with that.
So I try to make it as clear and unjargony as possible-- so as you notice, jargon up there.
So imagine a gene with two flavors-- sickle cell or not.
And two chromosomes-- so just like people have two copies of each chromosome.
OK.
So now, before I continue, what have I done just by doing that?
Well, first of all, I've made something concrete.
It's sickle cell or not.
So right away, you can imagine it.
So that's continuing the idea of a story.
It's much easier to imagine a concrete situation.
Either you have sickle cell anemia or you don't.
Or you have a gene for it or a gene that doesn't cause it.
And the two chromosomes-- so rather than having C chromosomes, I'm just restricting it to two.
Now, you might think, well, that's a terrible restriction.
That's a specialization.
Therefore, it's bad.
No.
Actually, it's good for that reason, because it makes it possible to understand, in the next step, the idea behind that equation.
And once you understand the idea, then that equation is not so mysterious.
And furthermore, what's nice about two chromosomes?
Yeah, it's a specialization, but--
Same two
Yeah, we all have two.
There's some plants that have more and some plants have less, I think.
But people have two.
So it's already interesting to us just for that reason.
OK.
So then the question is how frequent are the combinations?
So we want to answer how frequent are the three combinations sickle-- lowercase S is not sickle.
So this is S. This is lowercase S. What happened to the fourth combination?
Why are there only three?
Yeah.
It's the same.
S, S-- this and this are the same.
OK So I would actually ask that, too.
Any time there's something interesting-- it should have been four.
Plausibly, it could have been four, but it's actually three.
Why three?
OK, it's because we're actually lumping these two together.
OK.
So how frequent are those three combinations?
Well, that's provided by this formula.
P squared, q squared, 2pq-- so p and q are the frequencies of S and little S. So this is the-- OK.
So now, because you've asked them about the three versus four, this two makes sense, already somehow.
But how could make sense of the p squared, pq, and q squared?
And so that's where I would continue with the following, which is this.
So here is 0 to 1.
And this is p. And that's q.
So this is the frequency of sickle cell.
This is the frequency of not sickle cell.
So right away, you have p plus q equals 1.
I'll draw this up here too.
So I make a square.
And I have four regions.
So it's not magic.
Why am I making a square?
What about the problem tells me square?
So I would ask the class that.
I'm looking for area.
And why an area?
Why two dimensions?
Because there are two chromosomes?
Because there are two chromosomes.
So this is your C. The number of dimensions is eventually going to become this thing, so you can see how we're going to get there eventually.
So I'm looking for an area, because I have two chromosomes.
So now, let's just look at these various areas.
Here's a p squared.
What's this area?
There's q by p. So that's a pq.
That's also pq.
And that's q squared.
OK. Let's look at all our pieces.
Oh, we have a p squared there.
We have a q squared there.
Oh, and pq, pq-- 2pq.
So in fact, this picture explains the entire set of frequencies here.
And if you see this picture, there's nothing really new to understand here.
In fact, all this picture is showing is that-- it's a picture of p plus q squared equals 1.
So if p plus q is 1, its square is also equal to 1.
And you break this up into the four terms, group two of them together, and you get three terms, three different kinds of terms.
You get three different frequencies.
So now, this square is actually very easy to use.
Suppose someone tells you that sickle cell anemia, people who have both genes, are 1% in the population.
Well, what fraction of the population has no sickle cell gene that all?
OK. Well, we can just look at the square.
The information was that this is 1%.
So that means this 0.1.
And that's 0.9.
That's 0.9.
So this here is people with no sickle cell at all.
It is 0.81.
And so the square actually makes everything really easy to understand.
So now, the question is how do you generalize it?
Well, what are some of the-- the first generalization I'll do is I'll say, suppose that there are-- well, I'll ask you.
Should I generalize it to more dimensions or more different copies of the gene?
What's easier to draw?
More copies, because more dimensions-- I don't know how to draw in three dimensions.
But I can draw three copies.
That's pretty easy.
So now, take a piece of paper and just on your own, draw this same figure for three copies of the gene with frequencies p, q, and r. So people have a picture?
Does anyone want to describe their picture to me?
Yeah
I did a three by three square with p squared and q squared and r squared diagonal [INAUDIBLE]
OK.
So I do the same.
I make a p, a q, and an r. And then I do the same here-- qr.
OK.
So there's a p squared, q squared, r squared.
Now, how many guys-- so we have nine square, sub squares.
Three of them are this.
There's six more.
Now, which of them look similar?
So this is a pq.
Are there any other pq's?
Yeah, one other, right?
pq over here.
.
And here is a pr.
Is there a pr?
Yea, another pr over here.
And here's a qr and a qr.
So we have p squared.
And that's that guy.
We have q squared, r squared, and then 2pr, 2qr, and 2pq.
So that's the generalization to three copies of the gene.
So it turns out that all these coefficients-- 1, 1, 1, 2, 2, 2-- those are binomial, multinomial coefficients.
So now, let's generalize one more.
So what have we done?
There, before, we've written out this.
And those are the nine terms grouped into six.
So this was our C. So now, we know how to generalize.
p plus q plus r to the C equals 1.
If this is two chromosomes-- this is C chromosomes.
So this is C chromosomes with three copies.
Well, what happens if we have n copies?
p1 plus p2 plus one of those plus pn to the C equals 1.
And all you have to do is expand that out.
You can use a square in higher dimensions.
Or you can actually use formulas for math.
And that's where these come from-- p1 to the 1 power, p2 to the next power, all the way to pn to the next power.
And these guys are the coefficients that count the multiplicity.
Yes
Why do you have to add up to 1?
Oh.
They have to add up to 1 because p plus q plus r, if you only have three flavors of the gene-- either you have sickle cell A, B, or C, let's say-- then these are probabilities, probability p
Oh, so parentheses says how often it's done?
Yes.
Frequency and probability-- so p plus q plus r equals 1.
So if you square it, you still get 1.
So you start from the idea that p plus q equals 1, which is what we did over here.
So p plus q is equal to 1.
Then you square it.
It's still equal to 1.
So now, you just another way of writing 1.
So it turns out that Hardy-Weinberg is all just fancy ways of writing 1.
Starting from this picture, we've successively complicated it.
This is C equals 2, n equals 2.
There's C equals 2, n equals 3.
Here is the same thing again.
Here is n equals 3, general C. Here's general n, general C. OK so what we've done is we've basically got here-- by stages of successive approach, one step at a time.
So at every stage, it's clear what is going on.
And what's the core idea?
The core idea is the one you just asked a question about, which is that the frequencies add up to 1-- p plus q equals 1-- so if you square the frequencies, you still get 1.
And there's nothing more to Hardy-Weinberg than that.
Yes
I'm Julie
Julie
My question has to do with the original way you presented the problem-- using the word flavors instead of alleles.
And I've always been fond of teaching that you should always use the probable vocabulary of your students
To switch
So you got the same thing as the--
Yeah, good question.
So I used flavors instead of alleles-- so right here, imagine a gene with two flavors.
So actually, probably the best way to do it is to combine the two.
So you say flavors, because-- so this is a question of transmitting information to the student without noise on the channel.
So if you say the word allele-- so this is again related to chunking.
If you say the word allele, the problem is that now, you're expecting them to try to understand this new idea as well as this new item for taking up one of their chunks that they have available.
So when I initially presented it, I would use flavors.
And then I'd say, OK, now we understand the whole idea.
The thing has kind of seeped-- it's not really part of short term memory anymore.
It's connected to something else that they know-- for example, this.
p plus q equals 1, so p plus q squared equals 1.
Now, it's not taking up so many chunks anymore.
Now, they're ready to hear the word allele.
So I'll say, OK. Colloquially, we could say flavors, but actually, the word in the literature is alleles.
So at no point, you're overloading the system.
So again, it's philosophy based on, say, history and chunking.
Yes
When you presented this whole second way, you made, actually, use of the equation that you wrote down in the--
Oh, over there?
Yeah
Oh, no.
I was just saying that for your benefit
But I thought, actually, for me, it was very useful to see what the kind of equation was.
And then as you were going through the argument, I could reason out myself--
Good question, good point
--where everything was coming from
Yeah OK. That's a good point.
So it actually was helpful to actually see the final goal to know where you're going.
Actually, that's a good suggestion.
It's actually not bad to say, OK, this is what we're going to try to understand.
I don't expect to understand it now.
It's full of all kinds of squigglies.
So for example, maybe a really good way to do the whole thing would be to start with the Hardy story and say, we're going to talk about Hardy-Weinberg.
Say, what does Hardy-Weinberg say?
Well, in its full generality, it says, blah.
I don't expect you to make any sense of that right now.
Let's talk about what the core ideas of it are.
With this, you approach it, creep up on it bit by bit, successive generalization, and then you get there.
So actually, it's a good idea.
Leave that on the board the whole time so you have a context and a goal, like a mountain peak you're trying to scale.
Thanks for the suggestion.
Yes.
Tell me your name?
Scott
Scott
Last week, you did present an example where you wrote down this complicated physics--
The Navier-Stokes equation
Yeah, you wrote down the Navier-Stokes equation, and then you said, yeah, I sometimes do that just to create this tension in the students [INAUDIBLE].
So given that when you complete this method-- you actually didn't use the story at all.
It seems to me that if you had presented that and then you stood back and made a joke about [INAUDIBLE], it would have been perfectly satisfactory
I think that's true, too.
So a lot of teaching is preference of the instructor.
I, for example, happen to really like that story because I taught in Cambridge for a long time and I can imagine Hardy running into Punnett at the cricket ground.
So it's interesting to me, so I can tell it with enthusiasm.
And I think it says something about the relation of science on society, which I think is important for students to learn.
So I might use that story.
But an instructor with different purpose might do it, as you say, which is make a joke about how impenetrable this is and say, it seems really cryptic, but we'll actually understand this by the end.
Don't worry.
And that's a good way of-- actually, I think you're right.
It's a good way of creating and releasing tension.
Yes
Eric
Eric
So what level of background do we assume the audience has?
Because you use sickle cell as an example, and it's obviously not critical to the example that they know what sickle cell is.
Do you assume that they know that?
Right.
So what level of background?
So if they don't know what sickle cell is, or maybe I'd use cystic fibrosis, which they're maybe even less likely to know
--presentation
Yeah.
I could be adding noise.
So it's a flip side.
It's a two edged sword.
So are you adding noise by saying sickle cell to someone who doesn't know what it is?
Yeah.
It would add noise.
So maybe it's actually worth saying just one sentence.
What is sickle cell anemia?
It's a mutation of the red blood cells that makes them take a different shape and makes you unable to transfer oxygen as effectively and makes you more resistant to malaria.
So actually, that part, I wouldn't say at the beginning.
I'd say, well, if it's so bad-- can't transport oxygen so well-- how come it's still around?
Just let people think about that for the day and come back the next time and tell them the answer.
But yeah.
So I'd probably say one sentence if the people haven't heard of sickle cell.
When I was saying it, I was assuming mentally that they know what it is.
But I think you're right.
Many people wouldn't know what sickle cell anemia is, and it's worth saying just one sentence.
So let me just see if there's anyone who hasn't made a comment or question.
Yes.
Can you tell me your name?
[INAUDIBLE]
[INAUDIBLE]
Could you point to the last little bit, because I'm curious to see [INAUDIBLE]
Oh, how do you get this piece?
Yeah
OK.
So how do you get this whole thing?
OK.
So let me do that by analogy.
So for example, suppose-- let me do C equals 3.
I'm running out of board.
Let me erase this one.
And in fact, let me just use n equals 2, because it doesn't illustrate any new ideas to crank n up.
But the cranking the c up, the number of copies, actually makes it important to see why you need to add them all up.
So let's do p plus q cubed.
So this is an organism with three chromosomes.
But it's either sickle cell or not at each spot.
OK. Well, let's actually just write this whole thing.
You could do it by looking at a cube and seeing all the chunks, because you could actually just do it with algebra, too.
So there's a p cubed.
There's a ppq.
So there's a ppq, pqp, and qpp.
So those are all contributing to p squared q, and there's three of them.
OK. Now, the three isn't what we're talking about.
We're talking about this exponent here.
Why is K1 all the way up to Kn equal to C?
Well, let me just put that 1 in there.
What does this equal?
The sum of those guys is 3.
That's p to the cubed q to the 0.
That's also 3.
What's the next term?
Well, there's a 3 p cubed squared, and there's a q cubed.
Well, that's q cubed p to the 0.
That's 0 plus 3.
This is 1 plus 2.
It's always three.
And it's always the C. Why is that?
Well, you only have three products-- p plus q.
So you have three factors.
And you get to choose, when you're writing out all the terms-- there are eight of them and we've combined them into four groups.
There's three here, three here, one here, and one there.
You get one from each of the factors.
So you get one exponent from each factor.
So the total of all the exponents has to be 3.
So once you understand it for 3, then this is just for C in general.
Does that help?
So that's a good example, actually.
Someone probably would ask that question or should ask that question.
And that's how I would answer it.
OK. Now, Cecilia, you had another question.
Yes.
Did you have another question?
What are we proving?
For example--
Yeah.
You'd think, on one hand, what are we proving?
Well, we just proved p plus q plus r squared equals 1.
So it looks like there's no content
[INAUDIBLE]
How has that done anything?
The key fact is how you prove that the 1, the 3, the 3, and the 1 in each [INAUDIBLE].
For example, are we supposed to prove that today or break even?
Well, I would say there's no less information-- well, what we've learned is that, for example, we've learned this.
Why are there various products here?
And then what this thing is-- we've learned intuitively what this thing does.
This thing counts for the number of copies.
So in the original-- n equals 2, C equals 2-- it was either there's two copies or one copy of each of these guys.
So it adjusted for the number of copies.
When you have p plus q plus r, there's one copy, one copy, one copy.
There's two copies of that one, two of that one, two of that one.
So it's that factor.
It's the number of copies factor.
And then in probability course, you learn how to count those factors in terms of factorials.
And that has a definition in terms of factorials.
But I wouldn't focus, for example, in a biology course on why it's factorials, necessarily.
I'd want them to understand that this thing counts for the number of factors.
But actually calculating the number of factors for general C and n wouldn't be my first goal in the course
So if I didn't have that in mind, why would I-- so the general purpose is teaching the terms in the multinomial expression
Right.
So it's terms in a multinomial expansion-- so basically, it's from writing out 1 equals 1.
So it seems like, oh my god, we've done nothing.
What have we learned?
We've just learned 1 equals 1, which we already knew.
But actually, by splitting up 1 on the other side, you've actually given meaning to each of the terms on the other side.
So this is the frequency of probability of having three copies of gene type A and none of the other type.
This is the probability of having two copies of gene A and one of gene B.
And this is the probability of having one of gene A and two of gene B.
So there are three, because there's different ways of doing it
So do you have these chromosomes with different flavors [INAUDIBLE]?
That's right.
Because these organisms we're talking about come with multiple copies of chromosomes.
Yes
[INAUDIBLE]
What is that?
Yeah.
So what's the phenotype in the end?
And you're right.
So with the sickle cell-- so actually, that is an advantage.
So the point is you need to link this phenotype and the genotype.
And that's true, actually.
I've erased the sickle cell, but actually, the sickle cell is a good example for doing that.
If you have no copies of the sickle cell gene, then you're perfectly healthy.
If you have two copies of the sickle cell gene, you have sickle cell anemia.
What happens if you have one copy of the sickle cell gene?
Well, you do have some symptoms of it.
So the question is why does that gene survive?
Well, it's because you're actually more malaria resistant.
At least, that's one of the theories.
So somehow, I guess the malaria parasites can't eat those red blood cells as well because they have a different shape or can't invade it.
So it actually gives you some advantage and some disadvantage, but they're sort of balanced.
And yeah, if you have full blown sickle cell with two sickle cell genes, then you're in trouble.
But that's much rarer than the 2pq, as long as q is small enough.
So then you can actually continue that example.
You can look at the frequency of sickle cell gene in different populations and say, OK, well, is it higher in areas where there's malaria?
And test that theory.
Yes
I noticed that the first one that you did was to stay mathematical with the number line.
When I've, in the past, taught [INAUDIBLE], I always stick with conceptual-- p is this allele, and then you've got a phenotype in the end.
I'm just wondering if you chose this because you know that you're teaching mostly to people with math rather than scientists.
And if you were teaching to, say, English majors--
Good question.
Yeah, so I think you're right.
I did this kind of mathematically, compared to maybe how you've taught it to biology students.
And yeah, I guess I'm implicitly assuming-- but I didn't say, and I should say-- that this is MIT students.
I'm just, in the back of mind, thinking MIT students.
But I didn't make that assumption explicit.
So MIT students are perfectly happy with squaring binomials and trinomials-- trinomials, usually.
And to the C power-- sort of stretching it, but it's OK.
But generally, that way of doing things for them is good.
If it's people, English majors, yeah, you're right.
I would try to actually do even more conceptually.
But one thing that's good for all of them is the picture, because once you see the picture, then you actually understand the idea in one grasp.
It's really just one chunk now.
Oh.
It's just a square in four pieces.
Oh, OK.
It's just a different way of writing 1.
This is one area 1 and there's four pieces.
So that I would keep, no matter who I would talk to.
And the question is how much would lead up to it?
Yes
So I think I just had an idea here that I think highlights why you can't just go in with this, because there's a disconnect between what these terms mean at the end
Why you can't go in with
Right.
So I think you have to draw a connection between, say, the areas of the boxes in the diagram and the phenotype frequency
You're right.
So what you're saying is that you can't just launch in with this, because even though you could give-- say you gave really exact definitions of what all these things are.
It's not clear that exact definitions are computationally productive for a student.
Yeah, they may have the exact definitions, but they can't actually use it.
For example, let's do an exact definition of chess.
I'll tell you all the rules of chess.
And that's enough of an exact definition to be able to decide what the best first move in chess it.
But it's computationally useless.
I still don't know what the best first move is, even if I know the rules of chess.
So just telling the student all the rules that, say, define what a genotype is and what's polyploidy, doesn't mean they can actually use it in any problem.
So if you want to transfer, all these things have to have meaning for them.
And that's what the goal of this approach is.
Now, I think the approach has been improved from your suggestions.
I just wanted to show you a direction to go.
But I think you've actually taken it farther and extended it and improved upon it.
And the general principles among all of them are that you want to-- I would say one of the key ones is chunking.
You want to not overload the chunk system.
You want to somehow bring people in so they'll even listen to you.
If they don't listen to you, if they don't care, the learning is going to be so much less.
So all those are for that.
So the dictation and jargon oppose chunking.
And these all go together.
OK.
So now, what I want to do is give you a short answer to one of the questions that was raised earlier.
How you become a good teacher?
The reason I want to do that is that is exactly the same thing-- if you understand how you become a good teacher, you understand how you become good at anything-- how you become good at chess, how you become good at biology, how you become good at solving physics problems, how you become good at playing concert piano.
So if that's what you want to teach your students-- to be good at those things-- well, you want to understand that in a context, say, that you're working on-- say, being a good teacher.
And to do that, there is the following set of experiments.
So projectors-- OK.
So the way I'm going to illustrate this is I'm going to show you a chess position.
And the goal is to try to remember the chess position.
I'm going to give you two more seconds.
OK.
So everyone got the position?
Now, I'm going to ask you instead of to remember exactly, to reconstruct it.
How many pawns were there?
That would be E. So who votes for A?
OK. Who votes for B, nine pawns?
Who votes for C, 11 pawns?
Who votes for D, 13 pawns?
Who votes for E, none of the above?
OK.
So let me show you.
Then I'll explain why I asked you this particular question.
So there are actually 11.
Now, it's a very hard task.
So this actually, this very tasks-- not counting the number of pawns-- I made that slight variation.
But the so-called reconstruction task was given to chess players of various abilities.
Grandmaster slash master is one group.
What are called experts-- experts are people who are not quite chess masters, but close, in chess lingo.
And then class A players, which is pretty strong tournament players.
So they were given the task of looking at a position for four to five seconds.
The position was knocked down, and they were asked to reconstruct it as accurately as they could.
OK.
So it's even harder than the task of counting the number of pawns.
So the results are very striking.
So by level of chess player-- so class A is the strong tournament players, experts or grand master or master.
So the percent correct-- 51% of the pieces correct, 72% or 93%.
So the grand masters and the masters were amazing.
And in fact, for the number of pawns, I don't think they ever make mistakes on that, because pawns are one of things that you just know as a really strong chess player.
So 93% percent correct-- and that's amazing.
So there's an related story, which is about the memory of chess players, which is the Bobby Fischer-- yes
So I'm thinking the positions, the positions wouldn't work.
It wasn't possible.
They wouldn't do it as well
I'll come to that in one second.
So Bobby Fischer was in a tournament and some strong master was playing in it.
And Bobby Fischer went to the bathroom.
And as he went to the bathroom, he happened to see that master playing a game and just continued walking.
And then about six months later, he ran into him at another tournament.
Fischer said to him, oh, in that position in that tournament, did you play blah?
And the guy said, well, actually, I had no idea what the position was, even.
Fischer said, oh, and he set up the board and said, well, see, this was the thing that you really needed to do.
And by then, he sort of remembered.
But Bobby Fischer remembered at one glance six months later.
So what's a natural conclusion from this data?
You'd say, well, naturally, grand masters and masters-- they were born with better visual memory.
But in fact, the crucial experiment was then done in 1973.
So this experiment was done in 1948 by de Groot.
What Chase and Simon did in 1973 was that they showed positions that were random.
So they redid the experiment-- they confirmed these results.
And then, as you suggested, they showed just positions where the pieces were scattered arbitrarily over the board.
And then everyone was basically at 12%, give or take 1% or 2%, just random variations.
So what does that show?
It's not that the-- maybe Bobby Fischer was an exception.
But for almost everybody else, even the very strongest players, it's not that they're born with a better visual memory.
It's that they've learned somehow a way of looking at chess positions that there's less to remember for them.
OK.
So now let's compare.
Here, when the student sees this, there's a ton for the student to remember, just like when we look at a chess position, every piece is separate.
But what does a chess player see?
When a chess master looks at a chess position-- so I'll put the position back and I'll show you what the chess master sees.
The chess masters see something very different.
They see groups of related pieces together.
So for example, here, the chess master sees-- this king here is not a surprising thing for this chess master.
That's when you castle your king.
That's where it goes.
Then the rook goes next to it, and then you usually move your rook into the middle.
So that's not surprising.
This rook is also not surprising, the second one, because it usually comes from this corner into the middle.
So all of this almost contains no information for the chess master.
Here, these three pawns are very common, with the castled king on that side.
But then it looks kind of strange.
There's some new information there, because maybe this king castled, but then the rook went all the way to the corner.
So actually, maybe black never castled.
And his kind just sort of wandered into this area.
What does that suggest?
This suggests that the black king is really vulnerable.
Maybe it's time for an attack.
And in fact, I'm pretty sure this is a position from one of Garry Kasparov's games.
And in fact, that is the right conclusion.
The right conclusion is that it's now time to sacrifice your knight and take this pawn and draw the king out.
And he actually won using that-- by sacrificing his knight.
So the chess master looks at it completely differently than the novice.
I'm a novice when I play chess.
To me, every piece is a new bit of information.
I'm way overloaded past my chunk threshold.
I can't hardly remember the board at all.
So your students are in exactly the same position when they're learning material that, for you, you're the chess master.
So you're teaching Hardy-Weinberg.
Well, clearly, you've been appointed to teach Hardy-Weinberg because you have a Ph.D.
In biology.
You know a lot of biology.
You're the biology master.
So this doesn't surprise you that much.
But for the student, every single almost letter in there is news to them.
So what you want to do is you want to find ways of thinking about it that you can group the ideas into chunks.
So here is almost one chunk-- for example, the idea that really, it's just p plus q squared.
And there's a picture for it.
And once you understand that, there's another chunk.
There's another idea which is well, you could actually have three kinds of flavors instead of two.
OK. p plus q plus 4-- oh, I can transfer it there.
Oh, and then once you understand that, you can transfer it to n flavors.
Now, you can increase the number of copies of the chromosome-- so into two dimensions, three dimensions, four dimensional picture.
So then, you can actually make sense of all this.
You can have a way of understanding the position.
And not too long ago, there's, I think, a not well enough known paper-- I'll put the reference on the website-- which shows the relative importance of symbolic calculation versus perception.
So this is a perceptual mode.
So much of our teaching is, let's say, left brain-- very, very symbolic.
Well, there was this really interesting study done of chess grandmasters-- in fact, of the strongest chess grandmaster today, Garry Kasparov.
What's the relative importance of perception versus analysis in his really strong chess playing?
So the way they tested that-- there was a really good experiment-- is he would play simultaneous exhibitions all the time.
So the way you do a simultaneous exhibition is there's, for example, let's say, 10 opponents.
And you would just go around one component after another.
They have the full time, say, three minutes, to think until you come back.
But as soon as you get to a board, you just think for about five or six seconds, maybe 10, and make a move, and go to the next board, so that by the time you come around back to that same opponent, they've had their couple minutes to think.
So now, he played simultaneous exhibitions against very strong grandmasters.
And you can then measure his performance there.
So why is that a good experiment?
Well, he's now not able to do all of his calculation that he does normally.
He's normally able to think for three minutes, maybe five minutes, and do a whole bunch of analysis, symbolic computation.
But when he has five seconds, 10 seconds to think, mostly it's perception.
Well, his chess rating effectively dropped by maybe 50 or 60 points.
So 50 or 60 points, to give you an idea-- his chest rating is the highest in the world.
It dropped to a level which only five people in the world are higher.
So it's a very small drop.
So he still plays incredibly strong chess, better than almost every other grandmaster on the planet.
So almost purely with perception-- so what that shows is that the way Kasparov has become so good and in general, experts have become so good is it they look at the world different.
Their perception is different.
So how do you do that as a teacher?
That's one question.
How do you promote that in your students?
Well, you want to give the ways of looking at the world that change their perception.
That's why I'm so focused on the story, the tension, the human, the right brain, the pictures, the chunking, because it's those that are actually producing long term expertise, whereas this is producing what would be the equivalent of programming a chess computer.
But that doesn't work.
That may work for chess computers to play good chess.
But it doesn't actually work for people to be able to use the knowledge later.
So now, what produces that?
So there's one short answer which is that for teaching, you want to change your perception of how students think.
If you have a correct, new, good perception of how students are thinking, then you're actually able to make teaching judgments on the fly.
You can plan chess move, your lecture, like a chess game.
Your intuition is right.
So you want to tune your intuition.
Well, that is why I do this.
I've found the single most important thing that has improved my teaching, and I highly recommend, is the feedback sheet, because for example, I learned what was confusing in question one.
And in question two, I learned what works and what doesn't work.
So as I see what works and doesn't work, I start to build up a more and more accurate model of you.
And I start to be able to plan and reason about how to teach you and, in general, how to teach students.
So I'll talk about that more.
That's the idea of deliberate practice and expertise.
I'm going to talk about that more in the subsequent sections in more detail and show you some of the studies around that.
But the general rules is you want reflective, quick feedback on what you're doing in order to become an expert.
And that's true whether you're in teaching, concert piano, physics problems, whatever it may be.
So with that said, if you can fill out the sheets so that I can become a better teacher, that will be very helpful.
And one announcement, which is that next week is a Tuesday.
MIT is open on Tuesday, except it's Monday's schedule of classes.
So we don't have a class next week.
The week after that, I'm a witness in an administrative law trial.
So I'm not here.
So there's no class for the next two weeks.
So we'll meet again in three weeks.
And I'll post some readings and a short problem set for you to work on-- some readings growing out of what we've done today.
OK.
So if you could bring up your sheet and your index card to separate piles, that would be very helpful.
And there's going to be another class coming in-- a big class, I think.
So I'll just go outside and answer any questions that people I have right outside so that the new class can come in
Answers from Lecture 3 to questions generated in Lecture 2
I'm going to first answer questions from before, since there are lots of questions, and all interesting.
And I'm also going to do another equation example.
There was lots of requests for another equation example to see how it plays out in a different field and a different way of approaching equations, not just from the entry point of a story.
So I'll show you that.
And then we're going to look at misconceptions in various fields and the fundamental importance of understanding that so that you can understand how to change your teaching and how to reach the students.
Basically, if you can't understand where they are, you can't come to them.
So questions from before-- one comment was that I don't often enough summarize the end result with the transferable lessons for later.
So thanks for that suggestion.
I'll make sure to do that.
Another question was graduate versus undergraduate classes.
We talked a fair amount about audience a bit last time in response to questions.
So how do you change your teaching in response to questions, in response to the change in audience?
So the particular question here is graduate verses undergraduate classes.
And the sense I got from some of the questions was that somehow, it's harder to do what we were talking about last time, which is teaching equations in an intuitive way, in a graduate class than a undergraduate class.
Actually, in some ways, it's the opposite.
It's true that generally, in graduate classes, people just put up a ton of equations.
For example, in quantum field theory, you just get a gazillion integrals with epsilons floating all over the place and then path integrals.
You integrate this.
And there are some 2 pis and you take a bunch of limits.
And it seems like a whole bunch of methods gymnastics.
But A, it doesn't have to be that way.
And also there's another characteristic of graduate students which you don't have so much with undergraduates, which is that graduate students know how to read.
Now, this may seem like a bizarre statement, because surely, everyone knows how to read by, say, age three or four or five or whenever they teach reading in school these days.
But what I mean is that undergraduates-- generally, they have no experience on how to read a textbook because they've had so much experience with us telling them stuff everything on the board.
So they have no incentive to actually read the textbook.
And they don't learn how to do that.
They think textbooks are read the way you read Jane Austen novels.
You just read for plot.
Something happened to some equation, then something else happened to some equation.
And you just carry on, paragraph by paragraph, as you would a novel.
That way of reading is completely hopeless for technical material.
But graduate students-- not always, but generally-- have much more maturity about this.
So graduate students actually can or often are closer to being able to read technical material with skill.
For example, graduate students often read papers in their own field.
And you know you have to read a paper differently than you would a Jane Austen novel.
So because of that, you can actually teach equations very differently.
What you do is you give all the long, messy, yucky parts-- you leave that for the notes, for the book, somewhere where everything is printed in a very easy to read format, rather than copying long, long, long, long strings of symbols off the board.
So that connects back so what we talked about last time, which is chunks.
So if you put long equations up on the board, generally, you overflow the chunk system.
And once that happens, people start making mistakes.
So you want to avoid doing that as much as possible.
And with graduate students, it's even easier to do, because you leave all that for a type set, professionally published book or type set by yourself, but somehow printed in a clean way with no mistakes.
And you can then, in class, discuss the meaning of the terms.
What are the terms?
Where do they come from?
Why would you expect that kind of term?
So I'll give you an example of doing that with an equation today.
But generally speaking, all of what I was talking about last time applies perfectly well to graduate classes, even if people at first think that it doesn't.
So I should say if any questions occur to you as I'm answering questions-- basically, questioning beginning questioning-- please raise your hand and ask them right now.
OK.
I found that the square diagram muddied the development of the Hardy-Weinberg equation.
So the square diagram was this one.
So the conclusion from that was the question-- which is shouldn't college students be comfortable with expanding p plus q squared?
Why do they need a diagram?
And the answer isn't that people aren't comfortable with expanding p plus q squared, although you will find people for whom this and this is just a symbol replacement strategy.
In other words, it's something like you program in a computer whenever you see this pattern do this.
But the terms don't actually have meaning for people.
They don't know why those terms are that way.
And if they'd mismemorize and put cubes here, they would write that down too.
So the picture actually makes it clear what the meaning of the term is.
So it gives, actually, meaning to what people might be otherwise comfortable with or having done from lots of practice.
So I should say there's a difference between just rote matching between here and here versus this kind of understanding.
And that's illustrated by the following research, which is about people's ability to multiply and add when people have brain damage.
So the technical term is brain lesions.
The people with brain lesions in the-- let's see if I can say this right.
Some people with brain lesions in the arithmetic areas lose their ability to add, but they can multiply fine.
And people with brain lesions in the verbal areas, they lose their ability to multiply, but they can add fine.
Now, this seems kind of strange, right?
So why would multiplication and addition to not go together when you get brain damage in the arithmetic area?
So if the damage is either an arithmetic or verbal, here's what you lose.
So arithmetic area damage, you lose addition.
Here, you lose multiplication.
And that is very bizarre, because you'd think you should lose multiplication here, too.
And the reason that it doesn't work that way is because of the way multiplication is generally learned and taught.
So for example, in England the way it's taught is-- this is research from Brian Butterworth in England.
The way multiplication is taught in England is in the multiplication table, same is here.
And people memorize it as six 9's is 54.
So six 9's is 54.
Here, I think I learned multiplication both places.
So 6 times 9 is 54.
Either way, that's a purely linguistic string.
It's no surprise that when you lose verbal ability, you lose the ability to memorize linguistic strings.
So the multiplication table went.
So what that tells you is that most people-- and I don't know if this is true for everyone, but my guess is it depends on how you learned the multiplication table.
For most people, they learned the multiplication table in a way that is not meaningful.
They've learned the multiplication table purely linguistically.
And that is the problem with just going down this path only, which is that-- again, this is where it's important to know where the students come from.
If you know students basically have just linguistically memorized this transformation to that transformation, like the rock bands in other countries that sing in English.
They know when they see English words.
They know what to say.
But they don't necessarily understand any of the words.
That is actually very common.
And this is another linguistic transformation.
There's no meaning underneath it.
So here to give it some meaning underneath it, some picture, it actually incorporates another brain area into the understanding, which is where addition actually has some meaning to people.
So that brings up a related question, which is if people have learned multiplication in this linguistic way, how could you teach it in a non linguistic way so that people actually understand it?
There are several ways.
I don't know of any studies that show that after doing it this way and then brain damage, it doesn't get lost.
But my speculation is that if you learn it, actually, pretty much the way I learned it, you wouldn't actually lose it.
And the way is, for example, suppose you have to multiply 6 times 9.
Rather than memorizing that as 54, you think about what it should be.
And you reason your way to it.
And we think, oh, well, that's slower.
Yeah, it's slower in the beginning.
But in the end, you get to the same place.
But you've put it in a different part of the brain.
So the way you could do this one, for example, you say 6 times 9.
That's 6 times 10, which is really easy, minus 6.
So 6 times 10 you know not by memorizing but from understanding the number system.
So you know that that's 60 because of the way the number system works.
You don't have to memorize that.
That's the kind of example where I think calculators should be programmed to self destruct or at least not work for about a week if you type in a problem like that.
They should just freeze as an incentive to actually think about these things before you put them into the calculator.
So minus 6, and you get 54.
So now, that's a bit longer the first time you do it.
It's longer than memorizing it.
But after you do multiplication a bunch of times like this, you actually reinforce the meaning of the number system, and you come up with the same answer.
And eventually, you will memorize it, but you've memorized it in a different way.
Here's another example.
From the 12 times tables-- 8 times 12.
Now, should you memorize that as 96?
No.
You should write that as 10 minus 2 times 10 plus 2.
So that's equal to 10 squared minus 2 squared equals 96.
And you can even draw a picture for this and show what happens to squares and areas.
So again, you've come up with the same answer, but you've done it in a meaningful way.
So that's one answer to why it's worth showing pictures, even if people can do the algebra.
It reinforces the algebra.
Is determining what constitutes a chunk simply a matter of my intuition about students' level?
Well, to some extent, it is.
But in the chess playing research I talked about, they actually had more objective ways of determining what a chunk was.
And what they did is they put eye trackers on people.
And they had the chess masters and the non chess masters and the experts look at the board, and they tracked their eyes to see what they did.
So the non chess masters, their eyes wandered all over.
But the chess masters looked at pieces in groups.
They would go here, here, here, here, and then here, here, here, here.
And the assumption was that that's a chunk and then that's a chunk.
Or if they went here then to there, that somehow, these two chunks were related.
So there was ways in the chess playing problem of measuring chunks.
And then how do you apply that to, say, teaching physics?
Well, that's a matter, partly, of intuition.
And the idea is to help the students build up the chunks.
So you have to be on the watch for what chunks you use.
So you have to introspect.
OK. How do you keep things interesting and reveal material in a time appropriate way?
So the comment was that it took a while to do the Hardy-Weinberg with the story and then building up to it.
So this is a question of how do you plan lecture time.
What's worth doing in lecture?
Is it worth spending a bunch of time on understanding the concepts?
And there's two extremes to this.
I'm basically towards one extreme, which is that if you don't understand the concepts, it's not even worth learning the thing, because as a student, if you don't understand the concept, you might as well just forget the material right now, because you're going to forget it soon anyway.
So the only benefit from taking the course if you don't understand the material, is you just get a great on a final exam and you pass a requirement.
But in terms of actually changing how you see the world, it ha no value.
So there was a study done, I think, at Carnegie Mellon.
Yeah, this was at Carnegie Mellon.
They studied freshman physics students.
So they had students who took freshman physics and students who didn't take freshman physics.
And then a year later, they gave them the freshman physics final exam-- I think it was the same one, pretty much, as the students who took freshman physics took, except with numbers changed, but otherwise, it was same problems-- to see whether taking freshman physics had any effect on whether you did well on a freshman physics final a year later.
And the conclusion was that it had no effect a year later.
So yeah, sure, if you're give them the final the next day after the final exam that they took, they would have done pretty well, maybe 50%.
Who knows what the time constant is.
But certainly, by a year, it was gone.
So what that tells me is that that regular way of teaching material actually did no good to the students except for passing a requirement, but no intrinsic good for their way of analyzing the world.
So what that says is then you really need to find something different.
And yeah, if it means it takes extra time in lecture, a bunch of time in lecture, so be it.
At least people will understand something and they will change how they see the world.
And that's the motive for today, which is to really understand misconceptions, because if you don't understand the misconceptions, you're not going to be able to teach in a way that produces long lasting learning.
OK.
So another one was how do I apply this to something really abstract?
The way of approaching equations, like the proof of-- oh, this is actually one of my favorite infinite series.
So the question is how do you apply it to something really abstract like this?
So this is a famous infinite series.
What's the sum of that?
And it turns out to be pi squared over 6.
Well, even that, you think, well, how can I apply it to that?
And it turns out there's great stories about that one, too.
I think this is a story about this problem, which is that no one knew how to do this sum from 1 to infinity.
It's quite a hard sum.
And so it was set as a problem, basically, for the mathematicians and physicists of Europe.
And then someone produced a solution.
I think someone produced a solution anonymously.
And everyone basically figured out who had done it, because it had their handy mark.
So the solution was by Euler.
And it involved a whole bunch of trickery with polynomials and infinite degree polynomials.
And it was a really sly method.
So if I was going to teach this equation, I would actually teach the history of it-- how was really hard, how you could actually guess this.
What are ways you could guess this?
Well, you could actually approximate the sum, get a number that's one or two or three digits accurate.
And then you feed it into-- does everyone know this guy?
If you Google for that, you should probably find it.
It's called the inverse symbolic calculator.
It's a fantastic thing.
I do not know how it works.
And I would love to know how it works.
But what it is is you feed in a number, and it will tell you all the ways of producing something really close to that.
So for example, if you put in 3.141, it'll say a bunch of numbers that get near here.
And one of them is pi.
If you put in this to one or two or three digits, it'll probably guess for you pi squared over 6.
So that's one way of getting at an answer.
So part of the way of teaching it is to say, well, let's somehow get an answer with ways we can do that aren't too abstract.
And then let's see if we can justify that answer.
So even there, you're not lost.
There's always stuff you can do.
Oh, another question, which was readings-- how do you incorporate readings into a course so that students do it?
So that was asked twice, actually, because I didn't answer it the first time.
So one way to do readings is something called reading memos.
And it's an MIT invention by Edwin Taylor, who's recently retired from the physics department.
And what a reading memo is-- I'll put up the handouts.
So I have often done this in my classes.
I'll put up the handout for you to use that I give out and you can just copy it or do whatever with it.
So a reading memo is a request to the students to write you a short memo about something that you've asked them to read.
It could be the draft notes for your textbook that you're working on, which is what I often do it with.
Or it could be the textbook someone else wrote and you ask the students to read chapter.
And what it's not is it's not a summary of the text, because you already know what the text says.
There's no point in getting a summary.
What it is is students' reactions to it.
So anything that questions, things that puzzle them-- oh, I didn't understand why you did this or the author did this.
And then maybe three pages later, oh, now, I see-- which, if you're the author of those notes, you know that you explain the two things out of order and you should connect them.
But what that does is it teaches students how to read actively, because again, I like I talked about, people just do Jane Austen's approach to reading technical material.
And by getting students to read actively and formulate questions, by doing that, students learn a different way of reading, a way necessary for reading technical material.
And also by writing their questions down and you seeing the questions, you actually get a view into how the students are thinking.
So it's actually a way of understanding what their misconceptions are, their conceptions of the field are, and tuning your teaching.
Just automatically, you'll find your teaching will impedance match to where the students are just by reading the reading memos.
It has a further benefit, which is that it inverts the normal power relationship between teacher and student.
So for example, most assignments, problem sets, there's the correct answer, which you know.
And you're seeing whether they know the correct answer.
So they're now writing an answer, worried whether they're correct or not.
And then you're judging them.
So normally, p set, the power hierarchy is you and then the student down here.
And the student is looking up to you for validation.
So now, this is not a good thing to teach.
And maybe it's hard to avoid with problem sets.
You have to do problems sets somehow when you teach people to do problems.
But you want to minimize this as much as possible, because it's not a transferable way of dealing with the world.
They can't use that when they go elsewhere.
And it teaches bad habits of deference to authority.
So that's normal.
How does a reading memo work?
Well, it's the other way around.
If the student says, this is confusing, by definition, they are correct.
It's confusing.
They are the expert on what's confusing or not.
So it inverts the hierarchy to this.
And you become very interested in what the students are saying.
They are the experts now.
And I've had very good results with doing reading memos.
And my explanation is that it's because of this inversion of power hierarchy.
Now, what I mean also by good results-- two things.
One is that I get fantastic feedback on the things I'm writing.
The other is that I find students actually want to do reading memos after the class finishes.
They say, oh, if you have more notes, can we just do some more reading memos?
Great.
Let's do that.
And it's because it's actually-- if you write the problem sets, they'll often say, can we do more problem sets?
But that requires a fair amount of work to minimize the hierarchy.
But it's automatically here in the correct hierarchy.
So students actually enjoy doing that and want to continue.
So that's one excellent way of incorporating reading into class.
So now, the problem is, what you do when you have 50 people in a class and you get 50 memos?
So I've had this problem.
And one thing I do is I just feel overwhelmed and I just flip through them but I don't know what to do.
But another is I revise my notes based on it.
But the I think correct solution is an online system.
So what you want is an online system where you can post a PDF file, and then people make comments on the PDF file.
So everyone gets to see an image of the page, and they can just click and make a comment.
And everyone gets to see their own comments, and then when they submit them, they get to see everyone else's comments.
So I actually wrote half of that system.
And there's a graduate student in EECS who I think has now written a whole system independently of me.
So I'm going to try it out and see how it works and try it in some of my classes this semester.
So the benefit of that is that you can then see all the comments at once, rather than flipping through 50 sets of reading memos with page numbers on them.
OK. How do I come up with intuition examples?
How do I know if what builds intuition for me will also build intuition for the students?
It's a very good question.
Is it just my personal opinion or is it just the teacher's personal opinion?
One of the whole themes about this class is yes, teaching does have a fair amount of art and there is a fair amount of personal opinion in it.
But there's also a fair amount of science and things you can do to make it more objective.
And one of them is actually to do reading memos.
Any way you can to learn how students think will make it so that your intuition about the students actually matches how the students really think.
That's the whole purpose of today about talking about misconceptions.
Reading memos are way of understanding what students think.
Oh, there they are-- the reading memos.
So once you understand what students think, it's much easier to realize, just intuitively choose things that you know are going to work for them.
The other way is the feedback sheet.
So every time the students tell you, oh, this really helps me or this really didn't help me at all, you now have one more piece of feedback about what works for them and what doesn't work for them.
So then, you can actually choose intuition examples.
How do you invent them from scratch?
Well, there are some general principles.
One is use pictures whenever you can.
Generally, that speaks to people's intuition, just because people have much more hardware for pictures than they have for equations.
So I try to put myself in the position of the student, and I say, well, for example, here.
I say, yeah, all these equations may well be true, but I want a way that makes me see it instantly.
And that just forces me to start looking for pictures.
And that tunes me, actually, towards what students need.
So you can do the same.
I think that was most of the questions.
The other questions were similar to that.
And I'll answer any that were new that I haven't answered-- but I think most of them I have answered-- at the beginning of the next lecture.
So any questions that were generated by the questions?
Yes, question
So your story about the physics retension issue made me question the idea of the survey courses or topics in class, where it's like, I don't really need to learn this.
I just want you to-- been exposed to it so that they remember enough to know to go back to it if you need it someday.
That research, that that has any prayer of working
Right.
So the comment was what I'd said about even a year of intense freshman physics-- maybe it was a semester.
I forget if it was just mechanics.
But it was either one semester or a whole year freshman physics did no good towards long term understanding and change of understanding of freshman physics.
Well, what does that say about these big survey classes, where you're not expected to understand?
So in freshman physics, at least they have the expectation that you're supposed to understand everything.
Now, what about the courses where they start with the expectation that you're not going to understand most of it?
That one is going to be totally hopeless.
Then I think that is basically true.
And maybe you could justify back in the day-- let's say, 400 years ago, even when books were around but there was no web.
People need to know what books are out there.
So this is a traditional thing in Cambridge, for example.
They just give people big huge reading lists.
So then you go back.
Later, you're like OK, these are the key books in the area.
So you know one place to go for a reading list.
That's kind of obsolete now with the web.
If you want to find out something-- for example, suppose there was an equation I didn't know about.
Let's say the Black-Scholes equation.
Would I say I wonder if I took any classes about Black-Sholes-- maybe, let me go flip through all my course notes?
No.
You just type it into some web search and see what shows up.
And that is much more likely to be more relevant than some notes that you might have had or not had.
So yeah, I think the survey courses are completely pointless.
Now, that doesn't mean that introduction to a field is pointless.
But it means that the way to do the introduction has to be very different.
You can't just scatter a bunch of topics at people.
What you have to do is figure out-- and we're going to talk about this when we talk about course design.
You have to figure out what are the core reasoning ideas special that that field has to offer to the world?
For example, history-- what's special about history?
Well, historians have a sense of how to evaluate the validity and reliability of evidence and contradictory evidence.
That's something you don't get in many other fields.
For example, it's in between a science and a pure literature field.
In just straight literature, reading novels, there's historical evidence and things, but generally, you're reading in a different way.
In math, it's hard to know where contradictory evidence comes in, although there are ways of teaching math which I like which do that.
But generally, history has something new to offer, which is it's a messy world.
You have noisy evidence.
What do you do?
Well, that's something that an intro survey course could actually teach.
And that, somebody could transfer, even if they don't remember when did the Magyars invade Europe and all the random stuff that would be in a survey course.
So those would be grist for the mill hung off big ideas.
So I'm going to talk about that when we talk about course design.
But yeah, you're right.
That course design is completely hopeless the general big survey.
Other questions?
Yes.
Yes.
Could you tell me your name?
Meg
Meg.
And what was your name?
Amy
Amy.
Thank you
I was [INAUDIBLE].
I was just thinking about how for me, even if I understand something really well at the time, and I know that I'll use it again, it takes me-- having a test in front of me will not actually reflect whether or not I'm going to understand it, given the short period of time to remind myself.
And so I'm wondering, if testing people out of the blue a year later is actually capturing whether the people who take this and perform faster [INAUDIBLE]
Yeah.
That's a good question.
So maybe it was a slightly unfair test, because they were just tested out of the blue
If they had been using it all along-- if, in that period of time, they had taken courses that built on that material-- then, they would be reinforcing it all the time if they can
Right.
So if they had taken courses that used freshman physics throughout, maybe they would have remembered the freshman physics better.
And I'm sure that that's true.
So I can tell you one story from my graduate time.
So I did a Ph.D.
In physics.
And I had to do the qualifying exam.
And to do the qualifying exam, you have to study a whole bunch of undergraduate physics and then take the exams on it.
And then you take a bunch of courses in various fields.
Now, the only thing, basically, I remember from all the electromagnetism is one thing, which is I understand pretty well the index of refraction.
And why is that?
That's because I was really pissed off-- sorry for the camera.
I was really annoyed about the following thing.
So this actually goes back to what I was talking about about contradictions, which is that you're always told in relativity that speed of light equals C. And that's the great postulate of relativity-- that the speed of light does not change, dammit.
And that's what Einstein said.
And there's all these thought experiments with trains and lightning bolts and people throwing rocks from the train at different speeds.
And it's C. It's C. It's C. And then, somewhere later in an electromagnetism course, they say, the speed of light in a medium with an index of refraction N is C over N, where N is typically around 1, maybe a little bigger like 1.001 for air maybe, 1.33 for class.
And how do those fit together?
So that really annoyed me.
And I wanted to get to the root of it and say, well, how could it be that you could have a speed of light that's always C, yet it looks like the speed is some lower number, V?
So I worked out a whole bunch of stuff about how electrons scatter radiation and all the scattered radiation adds up and makes it seem like it's slowing down the light.
And because of that, I actually understand this.
And also because of that, which may be not so good, every time there's an electromagnetism problem I have to do, I always try to fit into a scattering problem.
And if I can't do that, then I just can't do it at all.
And that's despite taking two years of electromagnetism-- two years as an undergrad and one year as reviewing as a grad student.
So the point I'm trying to make by that is that most stuff disappears.
And the way to really make stuff stay is you really have to struggle with something.
And that's the most efficient way to make something stay.
And that's not what happens in your traditional class, and even less in a survey class
I'm just also asking-- it's almost impossible sometimes when you see material for the first time-- you might have to see it three or four times before anything comes, you're really going to understand it.
And if the class is set up so that you will only see it one time because that's all the time you have, then you have to take more classes on it
Right.
So maybe you need to see something a few times to really understand it.
So if a class doesn't give you that chance and then you have to take Thermo Two and then Thermo Three
Hopefully the class gives you a chance.
Say that it's building on this
Right.
So you should try to build that into the class.
So there's a name for that, which is called the spiral curriculum.
And there's a lot of sense to that.
So I'll just put a quick-- so you show the idea in its crude form.
And then you spiral back to it in a more sophisticated way.
But you'd like to do that soon, before the connection is gone, before this is actually wafted away.
You want to spiral back to it.
And so you'd like your class to do that.
So what that shows is that you should start with the big ideas and then you should refine them, because if you start with all the little details here, you'll just flood the chunking system.
And actually, there will be no memory of it here and then you have to start over from scratch here.
You had a question.
Could you tell me your name?
Roderigo
Rodrigo, yeah
Regarding the reading memos, do you think good to also ask the students questions about the readings?
The reason why I am asking is because I know of a class that's actually implemented that PDF annotation system.
And a lot of the comments that they get are too trivial, so to speak.
They don't ask those type of questions.
[INAUDIBLE]
So the comment is that a class that actually implemented the online annotation system, they found that the comments are too micro level and not broad enough about what's really confusing or interesting.
And so I haven't tried the online system myself yet.
What I do know is that on the paper, if you do it on paper, you get really insightful, detailed comments.
Now, I don't know what variables are different.
One might be that the online system just encourages-- because everyone does things quick online-- it encourages quick clicking.
So it may be that it encourages less depth of thinking, whereas writing it on paper, I actually find I get a mix of a whole bunch of not trivial comments, but small comments like, some typos, that equation isn't quite right.
But then I get things like, I don't see the picture here or I don't understand why you did this now.
And that, to my mind, is it useful comment-- or a question like that can't be right because of the following counter argument.
So on paper, I get a lot of interesting things.
So it may be that online isn't as good.
And that's one reason I want to try it and see.
So it may be that they need to go back to paper despite it being harder.
Another is also what is the material?
If the material is really boring, you're going to get really micro comments.
So it helps to actually have written interesting material or give people interesting stuff to read that people are likely to think about and make comments.
Question.
Can you tell me your name?
Brian
Brian.
Yes
I notice when you talk about equations, you like to give them the name that they're most commonly [INAUDIBLE].
Do you find that students get more understanding of concepts like this when they're given a name based on discovery versus terminology based on use of the equation?
I think, from my mind, if you want to describe the relationship between stress and strain in material, do you want to call it Hooke's law?
Or do you just want to call it the constitutive equation for solid material?
Right.
OK.
So the question is what about naming equations?
Should you name them by who made them or by how they're used or what they are?
So another example of that is, for example, the fluid mechanics equation.
Should you call them Navier-Stokes or should you call them fundamental fluid equations?
And there's a tension there.
First of all, I'll say that you do want to give a name.
The big win is giving the thing a name, because that makes a unit of thought for the students.
So that's the first order bit, the first order term.
The second order term is how should you name it?
And there, there's something which is that you want a name that's common.
So if, for example, people look it up elsewhere, they're likely to find more stuff about it.
On the other hand, you want a name that's intuitively meaningful.
So there's a tension.
There's not often a right answer to that.
And you can go either way.
So for example, I wouldn't probably use constitutive equation for the solid, because I have to think, what the hell does constitutive mean?
So myself, it doesn't mean anything to me.
There's another word.
I'm trying to remember what it is.
Epistemology-- people just use it like it's just a plain, obvious word.
But every time I hear it, I have think, what the hell does that mean?
And then I put the translation into the sentence they're saying.
And then I can sort of parse what they're saying.
So constitutive equation, to my mind, that's a word that is meaningful to the experts and not so much to the students.
So I would maybe call it the ideal spring equation, because it is the ideal spring equation.
It's just the proportionality is slightly general because you have tensors.
But otherwise, it is the ideal spring equation.
So it connects to something.
And you can say, OK, who discovered?
Hooke.
So we often called it Hooke's Law.
So then they have both And there's no harm in doing that.
Question
Because you mentioned a little bit before [INAUDIBLE].
If you're trying to teach something many times, then maybe it's better to tell the end, and then--
Yeah, the spiral
Yeah.
And I was just thinking because of the whole work-- so there are these really topics like the ideal gas law.
And is it OK if I were to lie or say to students just for the sake of simplicity, for example, do you want to say whether you can actually use that equation in some cases or is it not logical always, and then you go--
Could you tell me your name?
Cecilia
Cecilia.
Yeah, thank you.
That's an excellent question as well.
So the question is, how much should you lie, if at all?
For example, if I'm recommending teaching the big ideas in the overall approach first, that's almost necessarily going to involve some amount of lying, because the truth is complex and messy, the full truth.
And basically, you do want to lie.
Some people hate it.
But there's actually a really good book that follows this principle.
And even if you're not interested in the typesetting system, you can see how it's played out in this book.
It's called The Tech Book.
So that's the manual for the tech typesetting system, which I use and many people in math and physics use.
Now, the reason it's interesting is that Knuth actually tells you in the preface, I'm going to lie to you.
So what he does is he has three levels of statements.
There's statements that aren't marked with a-- so there's that sign.
On the road, it means slippery, icy, something like your car might fish tail basically, danger.
So there are statements without one of these, which may have some lies in it, not the full truth so that you just get the idea-- what are the fundamental concepts that tech uses.
When he starts to get into some gory details, but not super gory.
He puts one of those bends.
And when he has a super gory details, there's two of those bends.
And then he says, look, don't read any of these things unless you've been working with tech for a year and are pretty competent with it.
Don't worry about that.
You'll be able to do what you need to do just by reading this and maybe the single bend sections.
So that's an example where the lying was put to really good use.
And yeah, you should lie.
In fact, you have to lie.
There's no way to avoid it.
And in fact, everything is a lie.
It has to be, because to understand the universe, our brains are a constituent of the universe.
So there's no way to understand the full universe, because that would involved packing more than our brain capacity into our brain.
So just there is a pigeon hole principle proof that you have to lie to understand the universe.
So you have to say some stuff that isn't quite true.
And where the art is is in choosing what is a useful lie.
So what you want to do is develop the art of skillful lying.
And that's a mark of a really, really good teacher-- skillful lying.
Yes
How do we know that you're not lying to us now?
I probably am.
The question is how do you know that I'm not lying to you now?
And I probably am, the reason being that I've now practiced lying so much, I don't even know when I'm lying and when I'm not.
So no, I'll give you an example.
I am definitely lying to some extent.
For example, there probably are situations where you don't want to any lies at all-- for example, teaching people how to manipulate the machines in the intensive care unit.
Maybe the first thing you need-- if you have a half an hour to teach them, you'd better teach them, memorize these damn things and don't mess it up or you'll kill somebody.
Maybe there are situations where lying is less important and lying is more important.
And I haven't talked about those.
So right away, I have lied to you, just because that.
And I've skipped details because I wanted to get the big idea across.
So lying actually comes very naturally to me because I think the most important thing is the big details.
So just by saying big details first, you're automatically lying.
And I'm recommending it highly to you, too.
Rodrigo
I have a comment on the lying.
As a student, I have students, a lot of friends, who told me that they don't like it when teachers lie.
But what I think the differentiating factor is whether they tell you that they're actually lying.
And if they do, then it's totally fine.
And if they don't, and a couple months from there, they just told you that everything was not really true, the students might get upset
Right.
So the comment is that students often don't like when they find out that they've been lied to.
But it's OK if you tell them that you're lying to them.
And there's a general principle there which you can use in all your teaching, which is that whenever you do anything slightly nontraditional-- whenever you do anything like that-- it's really important to tell the students what you're doing and why, because they will go with you.
They'll go along with you if you explain to them the motive for it.
So you tell them, look, this is a really complex subject.
There is no way to understand the whole subject at its first glance.
You need to develop high level structures first.
And then you can put in the details underneath that.
So I'm going to tell you just the high level structures first.
And there will be some untruths in that, but I'm not going to tell you things that are completely false that you have unlearn.
I'm going to tell you things that you have to refine your understanding of or that aren't the whole truth.
So together with the idea of lying, you want to minimize stuff that you tell them that they have to unlearn and make it so that you're telling them stuff that they can keep using.
It's just not the full story.
OK.
So what I'm going to do is I'm going to give you just a quick example of another equation and how you could teach it.
Then we'll take a short break and then we'll do misconceptions.
OK.
So the other equation that I want to explain is this one.
So I did a biology example before, so I chose a physics equation this time, which is the wave equation.
OK. Now, also to vary it, with the biology example, I introduced it with a bit of history.
With the wave equation, I'm going to actually introduce it with a different approach, which is not to talk about the history but actually to get the students to try to construct the equation.
OK.
So the question is what the hell is the wave equation?
Well, the wave equation describes-- so here's some string.
And here's your coordinate x.
And you want to know how does the height of this piece here, the height being f of x as a function of time as well, change with position and with time?
So you want to figure out an equation for that behavior of that string, that stretch between two points.
For example, this might be a guitar string, and these are the two ends of the guitar string.
And it' under tension.
And you want to know, how does it move?
OK.
So we're going to construct the equation.
So let's say this is a differential equations class for people in physics who are learning mathematical methods.
And they want to learn how to construct differential equations.
So maybe they're engineers, physicists.
And they're, say, juniors.
So they have some mathematical sophistication and some knowledge of forces in physics and some knowledge of differential equations.
So I'm going to write down the rough form of the equation, and we're going to try to figure out all the pieces and fill in the missing pieces.
So I'm going to give the general form of the thing.
So there's some derivative of f with respect to time-- one or two derivatives, we're not sure-- is equal to something here.
And then there's some constant here or maybe here.
So there's a whole bunch of question marks to fill in.
And what we're going to do is reason about what they are.
OK.
So the first question is to figure out this guy.
How many time derivatives do we need?
So what this equation describes is the motion of this point.
Now, why does the point move?
Well, you could ask the class, and eventually, they'll come up with, well, there's some forces on it, because the string's under tension, so there's forces on the point.
So here's our point.
And here's, say, the string going through it.
This is a blow up now of this region.
OK.
So now, what are the forces on this guy?
Well, there's a force from that piece of the string and a force from that piece of the string.
OK. Because there's force, what's going to happen to the thing?
Is it going to have a velocity or an acceleration?
Anyone?
Acceleration
Acceleration.
OK. How many derivatives does acceleration have?
Two.
OK.
So we're going to put two derivatives here.
So you need two derivatives.
So this, and now, we're going to say this is some kind of acceleration.
And maybe there's masses in there and stuff, which would all be slurped into these constants.
So we have an acceleration.
And now, we have to decide what generates acceleration.
This side is generating acceleration or the force.
So we want to decide, for example, one or two derivatives.
Generally, most equations either have one derivative or two.
Some really nasty ones have four.
But generally, it's one or two.
So we'll just choose.
Is this one or two?
OK. And then the last thing we're going to do is to figure out the constant.
OK.
So now, one or two derivatives-- one way to decide that is to make something that has just one derivative in it.
So if here is my string and here is the point-- so f has a non-zero-- so it has a non-zero df/dx.
But the second derivative is 0, because it's a straight line.
OK.
So we know the second derivative is 0.
Let's see what we can figure out what the force or the acceleration should be.
OK. Well, here is a point.
There's going to be a force on it from that end and that end.
And what's the net result of these two forces?
0.
So when the second derivative is 0, in this case at least, we would like the force to be 0, which means that force has to be connected to the second derivative of position.
So we got that.
Now, the next problem is to work out what goes here.
The first thing is what's the sine.
Should it be plus or minus?
OK.
So I'm going to ask you that.
So find a reason whether it should be plus or minus here and a reason.
So find a neighbor or two, and we'll take a vote-- plus or minus, intuitive reason for it.
I can't use this example to decide.
Let's vote.
Everyone have their votes ready?
Who votes for plus?
So about-- who votes for minus?
OK. That's great.
So we have a diversity of opinion.
So right away, what that shows you is that it's worth actually discussing that point in class, because if you just tell people, they'll write down something you tell them, but they won't have actually internalized it.
It'll just be something that maybe contradicted what they said, thought, or not.
And then they have to remember, was it what I thought or was what I not thought?
Or was it what I not thought or what I thought?
So they don't actually understand why.
So it's actually worth going through the discussion.
So what example could you use to decide?
Who haven't I heard from?
Yeah.
Could you tell me your name?
Mike
Mike
Well, if you're at the top of the arc, and you don't it's going to want to be pulled down.
It's coming down
OK.
So let's use an arc like this.
We can't use this one, because it's 0, and 0 doesn't have a sign.
So it doesn't help us.
So the next most complicated thing is this-- so an arc like that.
Here's my point.
So the force is downwards, or the acceleration is downwards.
So we should have negative acceleration.
And what's the second derivative with respect to position here?
That's also downwards, because the arc is like that.
So the derivative has the same direction-- the space derivative and the acceleration.
So it should be plus.
So we got the sign right.
And to emphasize the importance of getting the signs, there's an interesting comment from Feynman back in the '60s at some conference about quantum electrodynamics.
He was commenting about how crazy the whole process that he invented for solving quantum electrodynamics is.
And his reason that is so crazy-- he said, well, we do the first order term, and then we calculate the second order term and add it to the first order term.
But it's very worrying that when we calculate the second order term after the first order term, we don't know whether the second order term is positive or negative.
We just have to calculate it and see what it's going to be.
But we can't predict ahead of time whether it's positive or negative.
And what that speaks to is the importance that physicists attach to knowing the sign of an effect.
Is a plus or is it a minus?
And so that's fundamentally important.
You want to make sure you get that right.
Now, why did I do the sign of the effect after this, after the order of the derivatives?
Because the most important thing is what do the terms mean.
So this is a curvature.
This is an acceleration.
So here, let me write that-- curvature.
Until you know what the terms mean, there's no hope of figuring out what the sign that connects them is.
But the sign is the very next thing you do.
And it's really important.
It's a plus.
And now, we need to put in one more thing, because the dimensions are totally bogus.
This is a position divided by a times squared.
This is a position divided by a length squared.
So we need to actually multiply by something to make the units come out correct.
So we need something that puts a time squared here and a position squared over here.
So then the units will work out-- or a position squared over time squared, which is C squared, which is some speed squared.
So speed squared is position squared over time squared.
And that makes all the units work out.
So there you go.
There you have the wave equation.
So now, if you're going to actually formally derive it, that's all fine.
But I would do this first so that people know where every single term in the equation comes from.
And now, I've been a bit sloppy.
These are really partial derivatives.
But that's, again, an example of lying.
I wouldn't worry about whether it's a partial or total derivative at the beginning, because that's not the fundamental idea.
The fundamental idea is that it's curvature on this side connecting to acceleration on this side, and they're connected by a positive sign and by something that has dimensions of speed squared, which turns out to be the speed at which the wave moves.
So that's a way to introduce yet another equation, not related to the history, but actually connecting to the intuitive approach.
Any questions about that?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So misconceptions.
So misconceptions, alternative conceptions, understanding the student view of the world, why is that important?
Well one of the best ways to introduce the importance of the idea is a story.
Now I'm not 100% sure the story is true, but it's so good that it ought to be true even if it isn't.
I think it is actually true.
So this was two, let's say physics, TAs, physics grad students, who were in a philosophy class.
And the teacher in the philosophy class said well-- they were talking about things being relative, and morals being relatives, and relativity in philosophy-- as an example of relativity, and things being relative depending on the situation, if you drop a pencil on the Earth, it will fall to the ground.
But if you drop the pencil on the moon, it won't fall to the ground.
It will just float.
And so most of the people in the class just nodded, basically thinking, oh that's a good example.
But the two physicists in the class were a bit shocked and said well wait a minute, wait a minute.
That can't be true.
Surely, something is wrong here.
So they raised their hand.
They said, well wait a minute.
If you drop a pencil on the moon, surely it's going to fall to the ground, to the ground of the moon.
And the other people in the class said, no, no, that's not right.
That's not right.
So they thought, OK, well we have some way of convincing them that the pencil's not going to float.
And they said, OK well you all saw the pictures of the astronauts walking on the moon, right?
And they said, oh yeah.
Well, when the astronauts walk, did they just float away, or did they stay on the moon?
Oh, they stayed on the moon.
Well, how'd that happen?
They thought, oh we have them now.
And what was the response?
The response was, oh, well that was because the astronauts were wearing heavy boots.
So here was a chance to do a bit of educational research.
So they thought, oh my god.
Maybe is this just this class?
Let's actually do a survey.
So they took out the university phone directory and just randomly dialed people.
And they called them up and said, OK if you drop a pencil on the moon, will it float away or will fall to the ground?
And most of the people said, it's going to float away.
So then they said, oh yeah, but what about the astronauts?
And they gave them the whole thing about the astronauts again.
And about half of the people said, oh yeah, that's because the astronauts were wearing heavy boots.
So again, they were shocked and flabbergasted.
And I think that reaction is a very healthy reaction.
But what it shows is that the students-- so in this case they weren't officially their students-- but students-- they could easily be your students-- can have a very different view of the world, of the physical world, than you have.
And if that's true, that means that as you're teaching them, what you're teaching is actually being interpreted in this completely different way.
I think Goethe said that-- he said that mathematicians, or maybe it was Frenchmen.
It was either Frenchmen or mathematicians.
Maybe it was mathematicians.
They have their own language, and everything you tell them, they first translate into their own language, and only then do they understand it.
So that's the same thing.
The students are translating the things you're saying into their own language.
But to understand what they're translating into, you have to know what their language is.
So in other words, you have to understand not just the misconceptions, but in general the conceptions that students have, so that you can actually plan your teaching.
And here is a diagram to illustrate that.
So this is a model of teaching to see where all the pieces fit in.
So here, this is the system, which is how students think.
This is the output, which is what students can do after they go through our teaching.
And this is the input, which is what we do, what kind of classroom structure we set up, what assignments we do.
And by this, I mean how students think individually and collectively, how students work together, as well.
So generally speaking the way-- so this is the system.
And call this the result.
And this is the teaching.
Our problem is trying to figure out the teaching.
And how do you figure out the teaching?
Well you start with figuring out what result you want.
What do you want students to be able to do?
What is your goal for the class?
And then you have to run it back through the system.
Right so you figure out the result, and you run it through the inverse of s. And then there's your teaching.
That's the teaching equation let's call it.
Now those of you from engineering fields know that basically the only thing you can do anything with is linear systems.
And unfortunately anything that's even slightly non linear basically is a bag of tricks, and there's nothing you can do.
And you're just hosed.
Well unfortunately this system is incredibly non-linear.
So it's probably kind of hard to work out s inverse, even if you can work out r. But it's hard, but it doesn't mean you can't start.
And a necessary condition for even doing anything in this direction is you have to know what is in s, how do students think.
Another necessary condition is you need to work out r. And that's actually something-- what is the result you want?
So that we're going to talk about in the session on course design.
That's something that's often forgotten in teaching, and it's fundamentally important.
That if you don't plan your goals, if you don't plan where you want to go, your chance of getting there is pretty low.
So assume you've done that.
The next thing you want to do is you want to understand how students think, so that you can plan your teaching to reach them, in other words, so you have some chance of working out s inverse.
Now you can't actually write down some equation for s inverse, but you can invert it mentally in your head.
And a necessary condition for doing that is to understand s. So you want to understand where students are coming from, so that you can work out t. So that's the signals and systems model of teaching.
Now, an example again to show the importance of doing that and what happens if you don't actually understand where the student's are is yet another example from freshman physics.
I'll leave this here.
So this is one of the examples from the readings.
So not everyone may have read that reading.
It's the reading by Reif.
And it's this problem.
So this is a pendulum at five points in its arc.
So these are the endpoints.
And the question people were asked was to draw the acceleration vector at these five locations.
So what's the acceleration right here, here, here, here, and here?
So I'm not so interested in what the actual answer is as so much as how people did on this.
So this is actually really shocking.
So of students in intro physics who've had acceleration-- so they'd done acceleration-- zero out of 124 students could do that problem.
So that's 0%.
So of grad TAs, it was 15%.
So these are grad TAs in physics who TA that course.
And PhD students-- so was PhD students on their qualifying exam.
It went up to 22%.
OK so now this is a fundamentally important idea, acceleration.
But you'd expect PhD students on their qualifying exam to be at 100%.
And you'd like the TAs in the course to be at 100%.
Because there's no chance of them teaching the students an idea that they themselves don't understand.
As Polya said, there's no method of teaching yet invented that will allow the students to understand what their teacher does not.
And that's true.
There's no amount of sort of trickery with style and things you can do to convey an understanding that you don't have.
So this is a very shocking number to me.
And--
Who said that?
Pardon?
Who said that?
George Polya.
So Polya was a great mathematics teacher from Hungary and then eventually at Stanford.
And he's written-- I'll put references to his books on the website.
But they are some of the best books on math teaching ever written.
And so he said that in one of his books.
And it's I think very true.
So what this example shows is that, a, the teachers didn't understand it.
But that really fundamental ideas can be completely skipped over for years and years and years.
Yes, [INAUDIBLE]
It's a special example of some vague, conceptually difficult problem.
I'm just wondering-- so I read the article.
Did they focus on how to figure the answer out?
Because a lot of physics majors don't
OK, so the question is, how difficult is this problem?
It's an interesting problem, because it's difficult if you try to do it by just blasting out the equations.
So they actually asked a bunch of faculty members the same question.
And one of them just got completely tangled in knots, physics faculty, trying to write out the differential equation and figure out what the acceleration is by brute force.
So yeah, it is hard if you do it that way.
But if you actually understand-- I'll show you-- what the acceleration means, then it's no problem.
So for example, here, the thing isn't moving.
So it has v equals zero.
But, what velocity is it going to have just a bit later.
Well it's going to be going that way.
So that means acceleration is that way
But were they given enough time to do this, to do this thinking problem?
Yeah, so they were given quite a while.
I think it was-- when the study on the faculty members, I think it was a big long think aloud protocol.
So they were given like half an hour.
In fact, the time is what hurt them.
Because they all thought, oh I have so much time.
I'm going to just blast it out with equations.
And if you were only given one minute, you might have actually had a chance.
Because you realized, oh I have to think about this intuitively.
So here, same thing by symmetry.
And then this one is hard.
And this one is hard.
But this one right at the center is much easier.
So this one catches a lot of people.
But here it's moving in a circle, and it's at its maximum speed.
So here it's moving at its maximum speed.
So it's not going to be going faster that way or that way.
So its speed is actually at a max, which means there's no change in speed.
So you're not going to get any acceleration in that direction.
But because it's moving in a circle, somebody's yanking it upwards.
So it has to be accelerating upwards to move in a circle, because it was going this way.
And then later, it's going that way.
So it has to be accelerating upwards.
So there's an acceleration vector upwards, like that.
So this is a bit exaggerated.
It's generally not that big.
And then the question is, with these guys, well it's just interpolation.
It's somewhere between that and that.
So it's going to be something like that and something like that, depending on where exactly they are.
But this is the one that tripped up a lot of people.
And in fact, a lot of people got tripped up because they memorize the following thing, which is an object in simple harmonic motion has no acceleration at the equilibrium point.
And that's true, except this isn't a simple harmonic motion.
It's almost.
But it's a pendulum.
It's not a pure spring.
So it has circular motion.
So if you reason about it from the motion, it's actually very quick.
But if you try to blast it out with equations, you're basically hosed.
So I wanted to check also, how deep are these misconceptions?
So I actually did a survey of students in Cambridge in the physics major.
And so one of the questions I asked them was the following-- so I made a survey of nine questions.
And I wanted to see how they reasoned intuitively on these nine questions.
And to force intuitive reasoning, I only gave a few minutes to get around the problem.
So they knew that basically writing out a bunch of equations wasn't going to help, because you had no time to actually write them all out.
So the question was, which is going to go faster down the plane?
So these are two identical planes, inclined planes with angle alpha.
And there is either going to be a disk like a coin rolling down the plane, or a ring, like somebody's wedding ring rolling down the plane.
And the question is, which goes faster, or which has a bigger acceleration.
So this we'll call the hoop, and this is the disk.
And they're going down the same angle
The mass is the same
I didn't say that.
The question is, is the mass the same?
So I'm going to give you that choice.
So the question is, which goes faster in acceleration or velocity?
So the choices were the hoop-- So to answer the question about mass, one of the choices is that it depends on mass.
Depends upon radius or depends on alpha.
So it could depend on the mass, the radius, or on alpha.
It could be the same.
The disk could be faster.
Or the hoop could be faster.
OK, so just for fun-- and I realize this is not a physics class-- but it's fun to actually try to guess the answer to this, even if you're not a physicist.
So choose A, B, C, D, E, or F in cooperation with one or two neighbors.
And then I'll talk about my analysis of this problem.
OK, take another 30 seconds and collect a vote.
Again, it doesn't really matter whether you get it right or not.
I just want you to think about it, so you realize it is a subtle question.
OK, so who votes for the hoop being faster?
Who votes for the disk being faster?
Who votes that they're the same?
That it depends on the mass, you'd like to know the mass?
Fair enough.
That you'd like to know the radius?
Maybe 12.
That you'd like to know the inclined plane angle?
Nobody like that one, OK. And now who's sure of their answer.
OK, so that's a much lower number.
So now I can show you what numbers came up when I asked the students this.
So this was 11%.
35%.
13%.
This one is 11% as well.
Radius was 26%.
And 3%.
So kind of similar, kind of similar to your percentages.
And again, only about a third of the people were sure of their answer.
OK, so why is this interesting?
Well first of all, all the students had the technical knowledge to do the problem.
These were all physics majors in various years, one through four, so freshman through senior.
But the freshman had seen the material to actually do this and calculate it if they need to.
So now this problem illustrates well first of all that even an intuitive question like this can actually trip people up, even if they've done all the calculations.
So the question is, how are they reasoning?
Now in Cambridge I had the advantage of also doing tutorials, because that's how half the teaching goes.
So in tutorials I actually I set this problem for my students, and I got a chance to talk to them about it.
So I got to figure out, how are they reasoning about it?
So I asked them, well what do you think is going to happen?
And then please explain why, more than you can get just from multiple choice.
And the answers were very revealing.
What they said was, many of them said, well, just as heavier objects fall faster, so should heavier objects roll faster.
So that was basically to justify they wanted to know either the radius or the mass, because that would affect the mass, and that would also affect the mass.
So, let me put that up again, because that's a really interesting statement.
So actually what I call this, this is like the American theory of the British accent.
Some of you know my explanation of that.
So the American theory of the British accent is that if you go to an English person and you step on their feet in the middle of the nigh, they'll actually cry out in an American accent.
Which is to say that the American, sort of folk theory of the British accent is that the British accent is a fake that people just put on, and in moments of stress they forget to do the put on act.
OK, so now why do I bring that up?
Well because it's very similar.
So if you ask the students-- suppose you didn't tell them this question about inclined planes.
You just say, do heavier objects fall faster?
You don't know about air resistance or anything.
They say, oh no definitely not.
Galileo showed that.
But what this shows is that in a new situation, in other words where they don't have their automatic reactions, all of a sudden the folk theory comes out again.
They really do think heavier objects fall faster.
Right, so how can you reconcile that with if you ask them, they'll say no they don't?
Well that's because they had so much experience with every time they said that, someone said, whack.
Heavier objects fall at the same speed.
Galileo showed that, Leaning Tower of Pisa.
So it becomes this linguistic string which hasn't actually been incorporated into their way of thinking about how the world works.
They think force is related to velocity, heavier objects fall faster.
And to get that to come out, you have to, for example, equivalent to stepping on them in the middle of a night, surprising them.
So you surprise them with a completely new context, which is the rolling.
Rolling, they haven't thought much about rolling.
So all of a sudden, they're now thinking about rolling.
And they're not thinking about all the policing rules that they were told about what happens to heavier objects.
And so now their true way of looking at the world comes out, which is that heavier objects fall faster.
So again, if you don't realize that that's what they're doing, your teaching is going to be completely pointless.
And what's going to happen is the following, which is that here are two models.
The students will develop two models of the world in their mind.
One let's call it the school model.
And another is their intuitive model.
And let's say they start out a bit different from each other.
And now you do a whole bunch more teaching, and you don't take account of the fact that this belongs in their intuitive model.
And you teach a bunch of stuff about rolling down the plane.
You do it with a bunch of equations.
By the way, I should tell you what the right answer is.
The right answer is this one.
It actually doesn't matter how heavy they are, because it's a gravitation problem.
But suppose you just calculated it out.
Well what have you done?
You've overlaid on top of this, you've put in-- so this is their intuitive model.
You've done a whole bunch of calculations which contradict it.
And the two models diverge farther and farther from each other.
So now next time they have to solve a problem, they have to decide whether I'm going to use the school model or the intuitive model.
And they learn, OK, well the intuitive model doesn't work very well.
Let me use the school model.
So the school model basically keeps diverging.
And the intuitive model and the school model never meet again, which means that they can't use their intuition for solving any of the problems you give them, or that they're going to solve later.
So that is an educational tragedy.
So the divergence, and in my view, that is the fundamental reason why most teaching produces no results a year later.
Because what you've done is you've developed the school model, the symbolic crunching, whatever it may be.
But you haven't actually yanked the intuitive model and the school model together.
You need to actually work on bringing them together.
So now the natural question is, how do you do that?
So let me give you another example of a similar misconception and a way of yanking the two together.
Yes, question
Would you mind explaining why the disk rolls faster?
Oh, yeah, why is it the disk?
Yeah, sure.
So first of all, why doesn't it depend on let's say the mass?
So that's one thing to clear up right away.
And one way to reason about that, maybe the best way in a physics class is to do it by dimensions.
But another way is the following thought experiment.
Which is, suppose I have two disks-- let's see, I might be poor today.
I only have one disk in my pocket.
Well here's two disks.
The chalk is also a disk.
So now, suppose I have one piece of chalk, and I want to roll it down the incline plane.
And now I ask about a second piece of chalk.
Let's say they're identical pieces of chalk.
They're both going to go at the same speed.
OK, so now suppose I stick a bit of glue on this chalk and that chalk, and I stick them together.
They're going to go, again, at the same speed, as if the glue weren't there.
Because the glue isn't being stressed.
They're just moving identically.
So what I've shown by that experiment is that a heavy piece of chalk moves just the same as a light piece of chalk.
So it's quite plausible from that reasoning that mass doesn't matter.
Why doesn't radius matter?
Well you could do a couple of scaling arguments to show that the extra torque you get from being bigger is exactly canceled out by the extra mass because you're bigger.
So the radius doesn't matter.
Mass and radius don't matter.
So then it's just a question of shape, which is a dimensionalist thing.
And what it is intuitively is how concentrated is the mass near the center versus near the edge.
So suppose I actually made a extreme version of the hoop.
And actually I did this for the students in Cambridge.
We manufactured-- and you could actually imagine what it is.
So imagine this were hollow inside.
That would be your ring or your hoop.
Well can you make anything that would move even slower?
Because I'm planning the hoop moves slower than the disk.
Yes, what you do is you put giant ears on the thing.
So it rolls like those dumbbell things you do for weightlifting.
So it rolls on this radius, but it has these giant ears.
Well it's just going to move so slowly, because it's just being yanked by this really weak force.
And it has to accelerate this giant mass.
No chance.
So it moves really slow.
In fact, that thing it can take 20 seconds to go down a meter long inclined plane.
So that's an extreme version.
So it rolls on this, and this is these big ears.
So that's really slow.
That's medium slow.
And that's faster.
And if you put all the mass right at the center, it would be even faster, because the rolling wouldn't suck up any of the energy.
So that's the intuitive explanation.
OK so now an example of a question showing a kind of similar misconception, a related misconception.
But I'm going to show it to you to illustrate what you can do about this.
So what can you do about it?
So the question is the following.
So that's a steel ball, which I have in my hand, and I drop on a steel table, say from a meter up high.
Forget about air resistance.
It bounces off the steel table.
And the question is I'm interested in the forces on it.
So what are the forces.
So here's my table.
So while it's falling, I'm interested in the forces on it at three different points in its flight.
First while it's falling.
I've let it go, and it hasn't yet hit.
At the instant that it's stationary during its bounce.
And while it's rising.
So when you ask the students this-- and I've done this several times-- mostly they know it's just gravity while it's falling, mg.
So if you actually use this question, don't ask-- this is chronologically next, but don't ask that next.
Ask this one, when it's rising.
And what you'll find-- what do you think many students say, not all but many?
What are the forces on it while it's rising?
Yeah, [?
Adrian.
?]
So of them will say the force goes up
Right, some will say the force goes up.
They'll say mg up.
I see people laughing, and that's a good reaction, because that shows you're learning something about how students think.
That's kind of amazing.
Why would they possibly say that?
Well let's talk about this one, and you'll see that it's actually a similar mode of reasoning to this.
So what you'll find is even students who get this correct-- so this isn't right.
It's mg down.
But there's a deep seated misconception that if something's moving upwards, it has to have a force pushing it upwards.
And then the force somehow gets used up, and then it stops.
OK, well what about here?
So now they'll agree that there's weight pushing down.
So that's the mg. And then the question is, what's the other force?
And they call it by different names.
In America, it's called the normal force, which is a terrible name.
And in England it's called the reaction force, which is a terrible name for another reason.
Which is that it implies that it's the result, somehow a reaction.
So it's a terrible name.
But let's ask the students, how big is this force?
OK, so it's upwards.
Everyone agrees that it's upwards.
But how big, r or n, depending on which country you're in?
So take a minute or two and for yourself figure out how big the reaction force or the normal force is, in comparison with the weight.
So this is a small steel ball dropped from say a meter and bouncing off a steel table.
So two questions, first figure out what you think.
And second, figure out what you think students are going to say.
OK so you're doing both, trying to get into the student mind.
Let's do a quick vote.
So what's the ratio between that normal force and the weight?
So who votes for equal?
First we'll do what you think, and then we'll do what students think.
So who thinks they're equal?
Two to one?
Greater than 10, but not quite 1,000?
Greater than 1,000 to 1?
OK now what do you think students say?
How many say one to one?
How many say two to one?
How many say greater than 10?
Greater than 1,000?
And how many are not sure what the students are going to say?
So actually when you ask the students it depends on the population.
But generally, what I find is it's 90% for one to one, And about 10% for two to one.
And no one says anything else.
OK so now why would students say one to one?
What's the reasoning going on in their mind?
Has no velocity.
So that is exactly the same reasoning as this, with mg upwards.
But what's interesting is even the students who know it's mg downwards still say, well the velocity is zero.
So v equals zero.
Therefore, it's in equilibrium.
Therefore, there's no forces acting on it.
Therefore, the reaction force, or the normal force, and the weight have to cancel.
N equals mg because it's stopped.
So that what I call that is I call that the f equals mv theory of physics.
Now maybe like in the linguistic books, I'll put a star by that saying don't say that.
That's not how you translate that sentence into real life.
But why f equals mv?
Why is it so deep seeded?
Well we have a gazillion hours of experience with it.
Suppose I move a object, and I push it.
Stop pushing it, and it stops moving.
If I have a light object, I just push it a little less and it moves.
And I stop pushing, and it stops moving.
So f is related to m, and f is related to v. This seems to explain most of our world.
It's hard to get around that.
So the question is, how do you actually try to force the students to see that that's actually completely junk what they said, and that it can't be equal?
And the way I want to do it is I want to do it in a way that recombines their intuitive model and their school model.
I want to meld them together.
So the way I do that is the following, which is I get a rock.
So let's call this a rock.
And I ask one of the students, could I please borrow your hand?
And so somebody gives me their hand.
And I say, OK, I'm going to put the rock in your hand.
Now, what's the weight of the rock?
mg. OK, great.
So what's the force of the rock on your hand?
mg. Great.
Now I hold the rock about the same place I'm going to drop the steel ball, and I say, OK on the count of three, I'm going to drop the rock on your hand.
One, two, and then I'm ready.
What are they going to do?
Like that.
And soon as they try to do that, I grab their hand and say, wait a minute.
Let's reason about this with physics.
Why are you moving your hand?
You just told me that when the thing bounces, it's still mg.
The reaction force and the normal force is still mg. And you just showed me that mg on your hand didn't hurt your hand.
You could hold the rock on your hand no problem.
So why are you moving your hand?
So then we'll have a discussion, and many of them will then change to two to one.
They'll say, oh, well it's actually two to one.
And I say OK, two to one, well I have an answer for that.
Two rocks.
So I have two rocks, and I put them on their hand.
So now compared to the first rock, mg, 2mg.
So now I take the first rock, and I'm going to drop it on your hand.
And one, two, three.
And they try to move.
And I say wait, wait.
You just told me 2mg was fine.
This felt fine to you, right?
And they say, I know it's going to hurt like hell.
So now what they're doing is they now have some kind of contradiction that's evident between the school model and the intuitive model.
And now is your opportunity to introduce and explain, yeah you're right.
Your intuitive model is actually correct in this case.
And your school model reasoning is incorrect.
So actually what we're going to do is try to move the school model towards a piece of your intuitive model that is correct.
We're trying to get away from the f equals mv part of the intuitive model and emphasize the part that does know.
And the part that says, I don't want to have that rock in my hand is correct.
So then they're very ready to do a school model calculation of how big the force is.
And it turns out to be something like 100,000 or 10,000 times the weight.
So it's huge because the impact time is so short.
And then to make that plausible to them, we can actually calculate the impact time or estimate it.
But then you can actually say, look you already know this.
You intuitively understand that.
Because if you-- again I'll stand on the table.
Suppose you're going to jump off an object like this, what do you do when you land?
What does everyone do?
Bend your knees.
Right, so you do that.
And what does that do?
That increases the contact time.
And why are you increasing the contact time?
Because that decreases the acceleration.
The acceleration is a change in velocity divided by the contact time.
So you can't affect the change in velocity very much.
That's either v or 2v or something like that.
But the content time you can change.
So with the steel ball, contact time is actually very short.
Because the speed of sound is so fast.
So the impact force is really high.
So yes, your intuitive model was correct.
And now we're yanking the school model towards it.
So you actually bring the two together.
And you fight the divergence.
So you want to look for examples like that whenever you can.
And to summarize the lesson-- because it was pointed out that I should summarize more often-- the summary of the entire theme about misconceptions is that you need to understand that so that you can teach properly.
Because there's no way to deduce this without doing that, which we're going to talk about later.
But even knowing that, if you don't know this, you can't run the system backwards and figure out what goes here.
And so then, how do you actually learn about this?
Well you've done one way.
You've done one way already.
Let me write down two or three ways for you.
So the question is, how to learn about s, the students' system.
Well one way is to read.
So in many fields, there's lots of research about what particular misconceptions students have.
And I've given you some of those readings on the website.
But there's a whole vast body of literature, and many fields have that.
So you can look at that.
Another way, and this is-- so this is shut your mouth.
Now why is that?
Well there's no way to really understand what students are thinking unless you ask them.
And if you don't let them talk, you're not going to actually hear what they're thinking.
You'll actually put words into their mouth.
Because you won't imagine that they think f equals mv.
You never would imagine that.
So you have to shut your mouth and listen.
So office hours is a chance for that.
And related to that is the feedback sheet.
If students get in the habit of asking you questions about what was confusing, you start to build a model of the students' system, what worked and what didn't work and what was confusing for them.
So all of these ways are ways to build models of s, which is absolutely necessary for deducing t. Any questions?
OK so if you could do two things.
One is to spend one minute filling out the feedback sheet.
And the other is everyone who has a homework equation treatment, please raise your hand.
OK that's great.
So now just find one person near you to swap with.
Question?
Yeah, OK.
So that's great.
So just find one person who has one near you to swap with and give each other comments.
And then just write your name over the next week and bring it in next time with you.
So basically this is a way-- and if there's no one near you, or they've already swapped, just make a group of three.
It's no big deal.
Does anyone not have someone to swap with?
OK great.
And so introduce yourself to the other person.
It doesn't matter if you're in different fields.
It often can be an advantage, because you'll get a fresh set of eyes on it.
And I'll give you some directions by email about what to look for and put it on the website.
But generally just give each other suggestions and work together on what you thought was a good and what wasn't good.
And just write your name on the person you commented on.
And then you'll bring them both in.
You'll bring the commented sheet from the other person in the next time.
Or you'll just give it back to them and bring it.
This is not a big police exercise.
I just want you to meet somebody else and have practice discussing these things and learn from each other.
So questions about that?
So if you could finish filling out the sheets and bring the sheets either here or here, I'll pack up.
And meanwhile we'll have any questions that people want to talk to me about individually outside, because I think there's another class that 's going to come streaming in with like 150 people.
So remember, learn about the student system.
It's probably the most important thing to do to make your teaching stand out and have long lasting, good effects
Answers from lecture four to questions generated in lecture three
So first, questions from before.
One of the comments-- well, let's see, two of them were that could we have less time for questions, but also they're very helpful.
So there's always a tension there.
So what I'm going to try to do is shorten some of the total answers to the questions and then maybe put some answers to the questions towards the end of the lecture as well.
I do like to try to answer most of the questions.
And the reason being is that it provides also pacing.
And you'll find that too.
If you find yourself generating four times as many questions as you have time to answer the next time, it probably means you did way too much stuff the previous time.
So it gives you feedback signal on how much material to discuss in the class time.
And so that's one reason I'm reluctant to just put all the answers on the web.
Because that's sort of like teaching with slides where you can just flip through all the equations really fast, and it seems like you got through a lot of equations.
But actually no one really understood the equations.
So it's sort of a symptomatic treatment for a more fundamental problem which may be that I'm doing too many things and not allowing enough time for questions in the class itself.
A general question, which is, how do you evaluate your own teaching?
When you feel good or bad about how class just went?
And what metrics do I use popularity, covering what I set out to?
How do I know?
So one major way is I use these.
It's not really that I say, oh did people hate it or like it?
I mean, I want to know if people hated it or liked it.
And I just I don't just add up the likes and the hates and say OK, 10 likes minus 3 hates equals 7.
That's great.
You're batting 80%.
But I try to get a sense from reading the comments of what people were confused about, what was interesting to people, and sort of integrate that up and imagine the whole class as one giant student mind.
And I want to know how I connect the material to the collective student mind.
So I evaluate it that way, sort of intuitively from the comments that people make on the sheet.
Which is why I hand out the sheet every time, and I think the sheet is so valuable.
Now another way, covering the materials I set out to.
Well I'm probably sort of on one extreme of that, which I don't really care how much material I covered.
But let me explain why I'm on that extreme.
So some people only care about how much material is covered.
And some people are more on my end.
So I'll give you the reason.
And most people I would say on this edge.
So I don't need to really justify that, because you've heard lots of reasons for that.
But it's the other side, why do I not care?
Well I don't care so much just because I think class time and a lecture course is such a small part of a student's life that if you haven't connected to the students, if you haven't made it click for them and got them interested, made them want to continue studying, then you've lost most of the effect of the teaching.
The effect of the teaching isn't really going to be so much in the class.
Because what you can do is you can kindle interest, kindle a way of looking at the world.
But it's really up to the students to carry that on afterwards.
And if you've actually bored them, or they've decided that this whole material is useless, they're not going to do that.
And then it's all just going to go away anyway.
So actually what I'm much more interested in is how much I've kindled those interests.
And that's what I try to judge using the sheets.
And I think that's actually what produces the long term change in ways of thinking.
Another comment was that the rock experiment rocked, which I thought was a good pun.
If you're not willing to write out detailed derivations on the board, is it more important to have more detail published lecture notes?
And yeah I think that's right.
In fact, that's generally necessary.
But it doesn't have to be your own published notes.
It could just be a book you refer people to.
So we've sort of gone backwards from Gutenberg.
In the days before Gutenberg, everyone read out their notes.
That was what monasteries did.
You read out the bible.
And at the end of a year or two of doing that, you had 50 bibles, because everyone took notes on what you read.
That was the old photocopier, back in say the year 1300.
And then when Gutenberg came along the universities then did the same thing.
That's what universities were.
They basically grew out of the monasteries.
Then Gutenberg came along and said, oh actually books are cheap now, well much cheaper than before.
Books used to cost roughly say $5,000 a book in today's dollars.
But then with Gutenberg and automatic printing, they could be much cheaper.
So you can-- and we've developed printing technology much farther.
So actually for a while until word processing came along and typesetting on your computer, people would actually just refer people to books and say, OK you're a student.
You are expected to read these books.
But now that people can make their own notes, people have sort of forgot how to tell people to use books.
And instead people, the lecturers, just tend to write their own notes out.
And the students actually seem for some crazy reason to basically except notes specifically for that topic and don't want to read books.
So it gets really strange.
You get in these bizarre cycles where you make notes.
And as long as you have notes that seem like they're key to the lecture, that's great.
Students will read them.
And then you assemble all the notes for the very same lectures into a book, and now suppose the book gets published.
The students will say, well now we want notes again.
You can't say, well there's the book which was the notes.
Because it's not specifically targeted.
So part of the cure for that is reading memos.
Or any way of teaching students to do reading is to shift back to taking advantage of Gutenberg.
And then once we take advantage of Gutenberg, we can worry about taking advantage of all the web technology and word processing technology that we have.
But yeah, it is important, to answer the question directly, to have the details somewhere for the students.
And the best place generally is in printed material, whether in a book or in something you write.
Because writing it on the board is very noisy.
By which I mean that it's very easy to mis-copy.
It's easy for you to make mistakes.
It's easier for the student to make mistakes when they're taking notes.
It's best to have it all in print, and also it's best to put the high level stuff in lecture, and have all the details with the students in their own comfortable space when they have time to go back and forth.
OK, so then there were several questions about how do you meld the intuitive model and the school model?
Are their general principles?
So I talked about how for example with the rock example that the school model is that when the rock isn't moving, the force is zero because it's in equilibrium.
So the net force is zero.
So the reaction force equals the weight.
So that's the school model.
The intuitive model, somewhere in there is an intuitive model that you don't want the rock to fall on your hand.
So what's a general ideas behind melding those two models?
So that demonstration I showed of borrowing someone's hand and dropping a rock on it, well not quite dropping the rock on it, but threatening to, Is one way.
But what are the general principles behind that?
So the general principles are that you want to connect to where all the mental hardware is.
So generally symbolic processing it's a sort of surface phenomenon in the brain.
It's linguistic.
It's only about 100,000 years old, whereas perceptual reasoning, visual reasoning, visceral reasoning, those are hundreds of millions of years old.
Organisms have been seeing things for hundreds of millions of years.
So there's a huge difference in the amount of hardware that we have.
So symbolic hardware say has evolved for 10 to the five years.
And perceptual, rough estimates 10 to the five versus 10 to eight years.
So there's about 1,000 times more evolutionary help for these things.
So if you want to meld-- and this is basically where the intuitive model, the internal model lives.
So if you want to change it, you have to connect to where the model lives.
You have to do things that are perceptual.
So visual arguments, visceral arguments, auditory demonstrations, things they create conflict.
Somehow some kind of conflict is also another general principle, where you expose some kind of obvious contradiction.
And that's so beneficial, because that's self teaching.
As long as there's a contradiction, the students know that the problem isn't done.
So in the rock example, there's a contradiction between, I don't want the rock dropping on my hand on the one hand-- that's the perceptual view-- with the symbolic one, which is net force is zero so the reaction force equals mg.
So there's a contradiction.
As long as you can't get the same answer by two different ways, you know you still have more to learn.
So it's self-teaching.
You don't have to wait for the teacher to say, oh wait a minute.
That's not the right answer.
You know on your own.
So know on your own.
You internally feel that there's something wrong.
So you're tapping in, I would say contradictions and puzzles.
They're somewhat symbolic, but they're also kind of visceral.
You feel them like oh, my god that can't be right.
So if you can tap into that as well that's another way of trying to meld the models.
It's interesting.
I think it's a fundamentally important point about teaching these two models.
And I've never seen it discussed anywhere.
So you're the sort of first people to hear it officially as far as I can tell.
So any questions actually about that model?
Because I think it is so important.
Yes
I'm just curious, what if you're in a field that doesn't lend itself to perceptual-- like if you're in a field that tends to be more symbolic.
And physics is really nice because it has this physical aspect to it.
So I'm just curious if you've thought about other--
So the question was, what happens if you're in a field that doesn't lend itself to perceptual reasoning so well?
I guess I wouldn't [INAUDIBLE]
Yeah there's visual in everything.
And I think it's hard to imagine a field that doesn't have it.
Can you give an example?
Well, some computer science.
And if you spend a lot of time with computers, then you start to develop it.
But you could think, well I know that it's not possible that it could just randomly decide to give a wrong answer.
There has to be a reason.
How the program develops, you can develop intuition about where the bugs might be.
But when you're just starting out, that's not something that we necessarily have built in to us in an intuitive way
That's true
[INAUDIBLE]
Yeah so computers and programming and computer science was offered as an example.
Yeah I agree with you.
Some fields are harder.
And what makes computer science harder is that computers are symbol processing devices.
So it pushes them over here.
But even there I think you can do stuff.
Like for example, what you could do?
Well there are some things like for example, loop invariants, and you can actually turn those into perceptions.
So once you have get a way of looking at programs.
You say, oh that can't be right because it has a fencepost error in it or it keeps keeps decrementing the counter, but there's nothing to stop it from having a floor.
So you start to see problems like that.
And you have tools like different kinds of magnifying glasses to look at programs.
So you could actually shift it that way.
And really good programmers actually do look at programs perceptually
See, that's encouragement to build up [INAUDIBLE]
Yeah, so you want to encourage that.
And it's true-- pardon?
It's not built in?
I would say that the field itself pushes things towards symbolic.
Just because it is a symbol manipulation device.
So that's what makes computers so much better at what they do compared to humans.
That's why we use computers.
Because we're so bad at symbol manipulation, and computers are so good at it.
So our ways of solving problems are going to be very different than computers.
But the problem that we have when we're using computers, we have to solve with human hardware.
So when the computer isn't doing what we want, we have to try to find some way of understanding what's going on.
And it can't be just purely symbolic, because we're bad at that.
So you can't-- I mean in desperation sometimes you'll trace through every step of a program and see what went wrong.
But generally that just overflows you with tons of data.
It's much better to actually try to take a step back, break the program into parts, see big chunks, and make sure the big chunks work.
And experts do that, not sort of automatically because they've been trained as experts.
So in any field the experts see differently than the novices, because their perception is different.
So yeah, when you're teaching computer science to students, you want to teach them ways that are perceptual.
You want to teach them visceral feelings.
So one way to do that for example is running time of algorithms.
Different sorting algorithms take different times.
So suppose you have-- for all the non CS people, a huge area of study in CS is sorting.
Suppose you have a giant list of numbers or list of words you want to alphabetize.
Text indexing uses it.
All kinds of stuff using sorting.
So how fast can you sort?
Well the naive sort of simplest thing to program is called bubble sort.
And if you have n numbers, it takes something times n squared, so flip operations.
So time is of order n squared.
But then there's merge sort, which is a divide and conquer algorithm, which takes n log n. Now you could just say that.
But it's also very helpful for students to have a feel, because generally students don't have an understanding of functions is what I found.
They don't have a good feel for what functions do and mean.
So what's the difference between n squared and n log n?
Well one way is to actually program a sort algorithm and then try it.
Try both algorithms and make your list bigger and bigger.
When do you get fed up waiting for the damn thing to finish?
So that's a visceral feeling.
And actually that will tell you that there's a huge difference between n log n and n squared.
Because this guy, basically when n is around thousand-- well, actually now computers are much faster, let's let's say 10,000-- you're pretty much going to be fed up with that.
Because that's 100 million sort operations.
It's going to take five or 10 seconds, maybe even bit longer if it's written in a higher level language.
So maybe you're fed up around n equals 10,000.
Here 10,000 times log n. Well, the log of anything is never bigger than 10, as a rough rule of thumb The reason being that if the log of the thing were bigger than 10, then n itself will be so big your computer couldn't even fit it.
So you're not even dealing with numbers whose logs are bigger than 10.
It depends on the base, but roughly speaking in base 10 logarithm, it's never going to be bigger than 10.
So this is say at most 10.
And so if you have a 10,000 only times a 10 here.
Whereas here you have 10,000 times 10,000.
Here you can go up easily to a million before you get fed up.
So the fed up is a connection to a visceral reasoning.
And actually it then makes the difference between these really, really apparent.
So there's ways you can do it even there.
Question?
I think that in the science fields that, if you really understood what you're teaching, then you should be able to basically put it into a simplification.
Because then it's basically summarizing everything, and it's just so much better and closer.
I mean, if you write for example a term paper or a proposal or whatever, you can write a one page text basically, or you can make a figure or a nice sketch.
It's the same information
Right so the comment was that if you really understand something, you can make a really compact pictorial representation.
That's generally true.
And that's why it takes really long time to make those.
So when you're making talks, for example, when we talk about this later in the term, when you're making talks and slides, a really good model for making slides is-- here's a slide.
This is the four to three ratio sort of.
So here is a sentence.
So this is some assertion, some message you want to get across.
And here is the evidence.
The body of the slide is the evidence.
And that's a picture.
The best is to have some kind of visual evidence.
But what you find when you do this-- and it is, I think, the best way to make slides-- it takes a long time to really find the best picture.
And even to find a good picture because it's just so tempting to write a bunch of words out.
But actually if you look for that picture and you really try to capture the main idea in a picture, you will be much more successful at conveying the messages that you want to convey.
So in general, try to talk to the perceptual system.
Now there was a question related to that which is that maybe I'm too harsh or too negative about the symbolic ways of doing things.
And the particular example was so if you remember last time, I gave this example.
So this is a pendulum, and this is the extreme of the pendulum motion.
And the question was what is the acceleration vector at these various points in motion.
And I said, well the way to do it is to reason intuitively about it, to figure out what it is here and here, understand what acceleration is, and do this one by interpolation.
And the comment was well, even if you can't do that, you can still use the Lagrangian.
If I know the Lagrangian for the system, which for the non-physicist is sort of the elephant gun.
You can solve pretty much anything with the Lagrangian.
So you can solve it with the Lagrangian and work out the forces everywhere.
And that's true.
But I still maintain that it's better to have-- I'm not saying it's bad to have a Lagrangian analysis.
The more ways of doing things is better.
But there's a fundamentally important reason why it's important to have the intuitive way of doing it.
And that is a search argument.
So if you remember from before, we talked about chunking.
The experts have big chunks, and the novices have small chunks.
Well we looked at the effect of that of perception, which is that one of the proxies for it was the chess masters could remember whole positions, and the novices couldn't.
Because their chunks were too small, and if you remember only seven chunks, which is typical, you can't remember a whole position.
Whereas the chess master's chunks are like three, four, five pieces.
And they can easily fit a whole position in seven chunks.
Well there's another, perhaps even more fundamental consequence of the different chunk size, and that is the exponential explosion in search space.
So suppose you're here.
This is your state now, and you're looking for a solution to the problem.
So suppose the solution is here.
But you don't know that yet.
You're looking for the solution.
Now if your chunk size is really big-- well let's say suppose your chunk size is really big-- you'll try a few possible chunks.
OK, do any of those get me closer?
Well this one looks a bit likely.
Let me try expanding that one, maybe that one.
Try this one.
And then eventually you get your way over here.
So here suppose there's four possible approaches you could try, and because the chunks are big, there's only three levels.
So this gives you four to the three items to search.
OK, now suppose you make your chunks really, really small.
Suppose your chunks are only half the size.
So you now at every half branch you have to search for possibilities.
So this is the large chunk.
So here we had one, two, three levels.
Here we're going to have six levels, in other words, four to the six possibilities to search.
So the difference doesn't show up in simple problems.
Because you may have only one level to go through versus two levels to go through, and either method works.
But when you're trying to solve interesting new problems, you want your chunks to be as large as possible so your search space collapses to something manageable.
So you don't want to have to reason with the Lagrangian as the default mode.
It's fine for that particular problem.
But if that problem is embedded in a bigger problem, you want to just know what the answer's going to be roughly.
So you can continue making progress and decide should I go this route or this route.
So that's why I'm not negative on the standard ways.
But I think the standard ways are actually genuinely bad for problem solving in new domains.
And you want to augment your perception so that you guide your search in the right direction with big chunks.
So that's fundamentally important.
So I think that was sort of the main themes in the questions, and I'll look over them and see which ones to answer probably at the beginning of next time.
I'll try to leave more time for questions today, so we don't generate so many questions next time to overflow it.
OK, so before I continue, any questions about these general principles of thinking about your teaching?
Yes?
So the approach is trying to focus more on the perceptual way to understand thing, but on the other hand, the symbolic manipulations are useful.
So how do you make sure that the students will themselves engage in the symbolic.
How do you encourage them to do the symbolic reasoning themselves without scaring them?
So how do you encourage the students to do the symbolic reasoning?
Because it is important.
And I agree it is important.
Just pure perception isn't quite enough.
I think most courses I would say, and most teaching, is of 99% symbolic and 1% perceptual.
And it depends on the field, but a rough estimate is I think it should be say something like 20% symbolic and maybe 80% perceptual initially.
And then as you go to more and more advanced levels, if you've done it mostly perceptual earlier on, and you've developed the intuition, then you can increase the amount of symbolic.
So I think the way you do it is you build it into the structure of the curriculum.
So one particular course might just have one mix.
And the more advanced students who got the perceptual material earlier are now ready for the symbolic.
So there's actually a graph that I show for this.
So that's the fraction of symbolic.
And that's the fraction of perceptual material as you sort of go on in the major.
So I think this is the ideal structure for a major.
Because this is going beyond course design to whole design of a major in a curriculum.
So early on, say for the freshmen, it should be mostly perceptual.
And there's many reasons for that.
First of all their symbolic capacities-- we're all bad at symbolic capacities to start with.
As freshmen, they're worse than the seniors.
They haven't had time for example get sophisticated understanding of differential equations.
Their algebra's rusty.
It's not really practiced much.
So don't rely on those modes of learning.
Rely on the perceptual modes.
And the perceptual modes give you the big chunks.
And so then once you have the big chunks, well then sure they're ready for the symbolic more and more.
Because the symbolic fits into the big chunks and helps guide the search.
It helps actually do the search.
I have to calculate this.
Then I calculate this.
Well each calculation you do you need the symbolic.
So by the time they're graduate students, hopefully the perceptual stuff they've had it up here.
It's really solid.
And now they're ready for partial differential equations and things like that.
But earlier on, you don't want to avoid partial differential equations.
You want to solve them in intuitive ways.
And an example of that is early on, when you're doing fluid mechanics, you would solve the cones using dimensional analysis, as we did before.
You wouldn't solve the Navier-Stokes equations.
And later on, for example, in a graduate class on computational fluid mechanics, you'd actually try to simulate that and figure out what the drag coefficient is numerically.
So that's the mix between perception and symbolic that I think is a very good way to construct a whole curriculum.
Does that answer your question?
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, welcome everyone.
It's 8:05, well, 9:05.
So today's lecture and discussion is about how to make good problems, whether it's on homework problems, exams, even problems in class.
Its general principles that you can use constructing questions that you ask, so problems broadly conceived.
Now why problems?
So problems are, I think, one of the central ways of thinking about teaching.
And I'll show you how problems fit into the grand scheme of this course, of rethinking all the aspects of teaching.
So one way that I like to think about teaching is called backward design.
So this is a way of designing a whole course, could even be for a curriculum.
And doing problems is one aspect of that.
So why is it called backward design?
Well, it's backward from the usual way.
So I'll show you what backward is and then show you what the usual way is.
So in backward design the highest-level thing you figure out-- not necessarily the first thing you figure out, but the highest level thing-- is your course goals.
And that then feeds to OK, if those are your course goals how do you know whether students have reached them?
And that's problems.
Problems, tasks, projects, whatever it may be, things that students are going to do.
And so generally speaking, things to do.
And then given that you've operationalized the goals in these problems what are you going to do in class, so that students actually are able to do these kind of things?
OK, so that is backward design because the usual way is the following.
You say, OK, well, you have a bunch of lecture notes.
Where do you have those?
Well, either the last person who taught the course gives you their lecture notes.
Or you just take the standard book in the subject and you just do the lectures one per chapter or one week per chapter, something like that.
And you just go through in the order.
So in the usual way you start with this.
And then you think oh, whoops, oh yeah, I've got to make some problem sets.
And oh, some exams.
Well, what have we done this last couple weeks?
OK, that's what we're going to do.
So the usual way is you start with here.
And then you get here.
And you somehow never get here.
The course goals are, basically, to get through the book.
Right?
That's the implicit course goal.
But it's never explicitly planned.
So that's a terrible way, but the normal way of designing a course.
So this is backward design.
So this is due to Wiggins and McTighe.
OK, so here we're going to be discussing this today.
So we're doing this first.
And then the next session is this one.
And I think the session after that is, what do you do in class, interactive teaching and things like that.
So why am I doing it in this order?
Well, in the levels of abstraction these are the most abstract.
And this is the most concrete, OK, what question I'm going to ask students in class now.
And this is things like goals.
I want them to really understand conservation.
They're really high level.
So I think if you start here, by trying to figure out course goals, I haven't found that works very well for me because it's too high level.
I can't think about course goals in absence of actual problems and things I want students to be able to do.
So this has the right level between concrete and abstract.
So it's actually where I start my thinking.
Now, you may actually find you want to start somewhere else on the continuum.
But this is what I found works quite well.
You start here, you operationalize what your goals are in the problems that you think are interesting.
And then you look at the problems.
And that sort of gives you an idea of the course goals.
And then you think about the course goals.
And that helps you think of problems.
But I start here.
And then, once I have things like that, then class time is actually much easier to plan.
Because class often is going to be problems like this, related to this, maybe shorter.
But it's easier to plan here after that.
Whereas we start here, you don't know, you're back in the old way.
So I don't want to start here.
This is too abstract, so I start here.
So that's why we're doing this now.
And then next week we're going to do this one.
Question?
So when you say that you're setting goals for your class--
Pardon?
So if you say that, basically, setting the goals for your class, or for your own course.
So when you design your course, basically you write out some kind of goals you want to reach?
So how do you then do the problem?
Do you go in detail before you've even started the whole class?
Or do you do it that on a week-to-week basis?
Oh, what do I tell the students about the course goals?
Yeah, I mean, if you say, OK, basically, you have your goals and then you design problems so you can reach the goals?
But then do you also design these problems before you even start the whole lecture?
The whole semester?
Oh, well, it's pretty rough.
I mean, usually the first time you teach the thing it's too abstract to design-- I find-- everything in advance.
Maybe that's my extrovert nature, which is that I find it hard to design things for people I don't know.
If I don't know the audience I don't even know what to say.
So I want to actually teach the course one time at least.
So then the audience becomes concrete for me.
So really, the first time through the course nothing is ever ideal.
And you're doing everything on the fly.
But the second time through the course, you can actually, basically, make all the problems ahead of time.
So the second time through the course you actually refine your goals and your problems together.
So that shows you the structure of where we are and where we're going to go and then later.
So this is two weeks from now.
Today, here.
So how do you make problems?
Well, the fundamental principle is, again, deduced by induction from what happens with regular problems.
So if you remember, I gave you an example earlier of the results of standard problem solving.
And I'll just remind you that we're putting up the percentages again.
OK, so after doing lots and lots of standard multiplication problems, students were-- if you remember-- unable to do the following problem.
So these 13-year-Olds and 17-year-Olds were asked to estimate 3.04 times 5.3.
So here were the answers.
So they were given four possible answers.
And they could also not answer.
So this is no answer.
So that's the age 13.
So of the four possible answers-- or let's say no answer is possible as well-- so five, they're 1% over 1/5 will get the correct answer.
So that's a bit depressing.
So you think, oh, OK, the 17-year-olds are going to be better.
And they are, but they only have 37% correct.
OK, so here is a serious problem, which is rote learning.
And that's the fundamental thing to avoid when doing problems.
So these students-- especially the 17-year-olds-- were actually perfectly capable of doing exactly multiplication.
So on the same test they were asked questions like multiply 2.7 by 8.32 and given just empty space on the page to multiply it out.
About 80% did it right.
So it's not that they don't know how to multiply.
They just don't know what multiplication means, which is a far more serious problem.
I'd rather they knew what multiplication meant and didn't really understand, didn't really know how to multiply.
Because the algorithm you can teach them later if they understand what it means.
But if they don't even understand what it means that basically it's 3 times 5, so it's got to be somewhere around 16, they don't understand what multiplication means or the number system means.
That's a far more serious problem.
And that's produced by the traditional kind of problems.
The traditional kinds of problems are, generally speaking, too low level and can be solved, to easily, by rote.
Even the complicated traditional problems can eventually be turned into rote learning, and that's the danger.
So the main goal in making good problems is-- I would say-- how to make problems that fight rote learning.
OK?
That produce long-lasting actual conceptual understanding along with whatever else you're trying to produce and teach.
But really, fight the rote learning because that is the fundamental problem.
So a way of thinking about this is to categorize the levels of thinking that you're expecting of students when you're asking them to do a problem.
And for that there is work from 50 years ago now-- which is still useful-- which is Bloom's Taxonomy.
So Bloom's Taxonomy-- let's see, when was it?
I think it was 1956.
I think it was '56.
I'll put a handout on the website for you which summarizes in one page.
But I'll summarize it even shorter here.
So what is it a taxonomy of?
Well, when you read the title the book, it's Bloom's Taxonomy of Educational Objectives.
And it's not really clear if that's going to be useful for anything at all.
Because you think, what the hell is that?
So the shortest way of understanding it is that it's six levels.
And each level is higher than the one previous to it in terms of the level of thought required of a student.
So it's not really a strict order.
You can have really hard questions that seem to be low level just because they address misconceptions.
But generally speaking, it's a rough order.
So the order is knowledge questions.
Those are things like define, name, state, list, You know, it's basically almost rote.
I mean, these are the kind of problems that everyone would agree are rote, state the definition of Newton's Second Law.
The second is comprehension.
So that's sort of like knowledge but it's a higher level.
So that's being able to grasp the meaning of the material, not just recite the facts.
But generally, that doesn't include the idea of being able to transfer it to new situations.
That's where application comes.
So can you apply the material to something?
For example, often compute, demonstrate, show that, those are typical verbs that are used when you have application questions.
The next level is analysis.
So that's really trying to understand the structure of the material.
So things like diagram, differentiate, infer, outline, point out, identify, distinguish, discriminate, all those kinds of verbs com into analysis kinds of questions.
Synthesis-- so this is breaking things apart.
This is the ability to put things back together in new ways.
OK?
So design, explain, categorize, create, revise, rewrite, reorganize, devise.
And the highest level is evaluation.
So judging the value of material, for what purposes is this good versus that good?
Explain, contrast, summarize, support, recommend, evaluate, criticize, appraise, so generally quite high-level skills.
So this is the taxonomy and this is a higher level.
OK, so a useful exercise-- whenever you're thinking of problems-- is to try to place them on this.
Now it's not that you want no-knowledge kind of questions and you want only to ask these kind of questions.
Because if you just ask these kind of questions people will not be ready for them.
And they're not prepared.
It doesn't actually teach anything.
You want a mix of them.
But generally speaking, what happens to often, is that most of the questions that happen are usually in this zone, maybe a few of those.
And then only later in people's careers do they get over here.
For example, in engineering curricula you learn a ton and ton of application, maybe comprehension, Maxwell's equations, differentiation, integration.
Then we start to synthesize stuff.
You know, in your senior year you do design and project classes.
But you don't have to wait all the way until then.
You can actually start putting stuff in much earlier.
And it doesn't have to be an entire class.
You can have tasks that are like that in a short time, on a problem set in the middle of lecture time.
OK?
So questions about Bloom's Taxonomy?
OK, so what I'll do, is I'll give you an example of questions that in freshman physics mechanics-- on the topic of Newton's laws-- that fit into each of those levels.
OK, so just to give you an example, to make it concrete, to see what those kinds of questions are.
And some of the categorizations I'm going to give you are debatable.
But the rough idea is that they go from high to low.
So this is on the subject of Newton's second and third law.
So a knowledge question, state Newton's second and third law.
Newton's second law, state Newton's third law, so you can see, just because people can state it doesn't mean they can actually do anything with it.
That's why this is way up just at the knowledge end.
Computers are actually really good at this.
Right?
You can look it up on Google.
But your computer can tell you what Newton's second and third law are but you can't actually do anything with them.
So comprehension-- so give an example of Newton's Second Law, using Newton's Second Law.
OK, this next one is an application.
It turns out to be, actually, quite a hard application.
But it's still an application.
So it's to apply Newton's second and third law to show the following, which the following is suppose I have a piece of chalk in my hand and I stand on a weighing scale.
I want the students to show that the weight that the scale reads is equal to the sum of the two individual weights.
So I weigh myself.
And then I put the chalk on the scale.
And I add up those two weights.
And that should be the same as when I hold the chalk in my hand.
OK?
So I'll diagram that as-- OK, so that's me plus chalk equals me plus chalk.
So this is a scale.
OK?
So there is the piece of chalk.
So now this is an example of where that's quite a hard application question, even though it's kind of high up on the Bloom's list, so towards the lower end.
Because this is a serious misconception that students have.
Newton's second and Newton's third law, people are completely confused about, you'll find.
They don't know when to use which.
And they think any time two forces are equal it's sort of 50-50, whether it's because of Newton's third law or Newton's second law.
So this-- if you're addressing misconceptions-- makes the problem, actually, all the much harder.
So at analysis-- so the analysis comes out-- oh, question sorry
What are Newton's second and third laws?
Ah, fair question.
So Newton's Second Law says force equals mass times acceleration.
Third law is-- the way it's usually stated, which it's terrible-- action equals reaction.
Or for every action there's an equal and opposite reaction.
Oh, which reminds me, one of the questions from before, which is what name would I give?
Because I said I slagged off the reaction force as the name and the normal force as a name.
People ask me, what should you call it?
And I actually thought about that.
And I think the answer is contact force.
Because that tells you what the thing is.
It's a force of contact.
So Newton's third law says action equals reaction.
So now, the problem is that students have, is that whenever they see two forces are equal they assume it's because the action equals reaction.
Or they think Newton's third law is the reason for it.
So for example, the bouncing-ball question that we talked about last time.
So in the bouncing ball the question is, when it's stationary on the ground what are the forces on it?
And they'll often say, well, there's a weight downwards.
And there's the force upwards.
And they'll often say, well, the force upwards equals the weight downwards.
These are both forces on the ball because action equals reaction.
And they won't realize that action and reaction have to be on different objects.
OK?
So they're fundamentally confused about the meaning of action and reaction, which you can bring out with the following analysis question-- When two forces are equal in magnitude how do you tell if it's Newton's two or Newton three?
So how do you know if it's Newton two or Newton three?
So just ask them that.
Yes, question?
This might be a little bit off subject, but should we use Newton's third law ever?
It seems like it's a natural consequence from saying a equals 0
Ah, oh, OK, interesting.
So OK, let me actually explain that one.
I'll put it over there because I didn't do the boards right over here.
So that is exactly the issue brought out by this question.
And let me show you the difference.
And then there was another question.
That reminds me of another whole set of questions from the sheets, which was how do you develop your physics intuition?
So let me answer that in a moment.
After this and then we'll have a break.
OK, so the bouncing-ball question-- actually let's not do the bouncing ball.
Let's just do an object sitting on the ground.
Oh, great, can you pass them around?
Oh, Leeann's just coming around with a handout on Bloom's Taxonomy.
And let's see, I'll do these after the break.
OK, suppose you have this object sitting on the ground and it has Mass m. OK, let's look at what the forces on it are.
Well, there's mg down-- and actually, in this case, the contact force, our normal force is equal to mg. OK, so now the question is, are these two equal because of Newton's Second Law or Newton's third law?
That's what this question asks.
So it's actually worth understanding the difference here.
So these guys are equal because a equals 0.
So this thing's just sitting on the ground, a equals 0.
So the net force is 0, so these two forces have to cancel.
OK, so that's Newton's Second Law.
So then the question is, where does Newton's Third Law show up in this?
Ah, Newton's Third Law shows up here.
Here is the Earth.
And here is your object.
Let's just separate them just so you can see them separate, although this is sitting on there.
So this is mg. Well, this is the mg because of gravity, right?
The earth gravity is attracted to mass, m. Well, the mass is also attracted to the earth.
So what's the force on the earth from the mass?
mg
mg, it has to be.
Now this mg equals that mg.
Call this f2 and this is f1.
So f1 equals f2 by Newton's third law.
So that's a fundamentally different set of forces.
So are equal and opposite.
They're an action/reaction pair.
So what Newton's third law really says-- when you understand it-- is that all forces in the world come in pairs.
So if you ever see one force you have to hunt around for his other half.
And the other half here is this.
So actually, there's another question you could use for Newton's third law.
So I don't have it on the list here.
But I'll ask it right now, which is suppose someone comes to you and proposes the following gravitational law-- f gravity equals g m1 squared m2 over r squared when the g is the proper units.
So m1 squared times m2, is that a legal force law for gravity?
No, because when you switch m1 and m2 as you're doing here-- you're switching the mass and the earth-- you actually change the force.
So then this would violate Newton's third law.
OK?
So Newton's third law, it actually is a constraint on the laws of physics.
It's actually conservation of momentum.
And for momentum to be conserved you can't have any arbitrary law of physics that you want.
Does that help clarify that?
Come up with a better set of taxonomy for describing it that doesn't lead people-- it leads people into thinking about that
I know.
I agree.
So I think the terms are very important.
So that's why don't like this form, action equals reaction.
I actually would prefer it were stated all forces come in pairs that are equal and opposite.
Right, and language does have a powerful effect.
Yes?
The technical [?
route's ?]
obvious, just in calling the novel force the contact force, but that's not refined enough.
Because both friction and novel are contact force?
Yeah
[INAUDIBLE] constraint force because it could constrain into the [INAUDIBLE]
Yeah, that's probably a better name.
Yeah, I agree with you.
So the comment was that contact force isn't quite right because there's friction force as well.
And that's also a contact force.
And maybe a better name is constraint force.
I think that is a better name.
OK, so let me finish this list of two more examples of Newton's Second and third law.
And then we'll have a break.
OK, synthesis.
So creating a problem using Newton's second and third law.
So for example, this is a problem like that.
But here I'm asking the students to make one up.
And here is a very hard evaluate question, which is what, if any, are the limits of validity of Newton's Second and third law?
So that's, actually, a very interesting historical question.
Interesting historically because Einstein thought about that question.
He wanted to know whether Newtonian gravitation and special relativity were compatible.
And basically, they can't be because of Newton's third law.
Because gravity should propagate with some speed, not the speed of light.
So you can't actually conserve momentum right away.
Suppose the sun does something strange.
Well, it takes a while for the earth to actually respond to that.
And it's in that intervening time Newton's third law seems to be violated.
So Einstein thought very carefully about that question and came up with his theory of gravitation.
So this is quite a high-level question.
But it's to show the whole range of things you can ask about the second and Newton's third law as you climb Bloom's Taxonomy.
OK, so what we're going to do in the second part is we're going to actually practice rewriting a couple of questions.
And I'll show you a couple more examples.
And seeing how we can make questions worse and better, because I find that's actually quite a good way to get practice with designing questions is taking good questions, making them bad, taking bad questions and making them good.
Both ways are useful.
OK, so we'll do that.
Let's say it's 9:04, 9:10 let's say.
So it's 904 on that clock.
So see everyone, sorry it's 10:04 on that adjusted clock.
So see everyone at 10:10 or 10:11.
And meanwhile, here are the feedback sheets.
So I'll just put a bunch in the two back things.
So when you come back just grab those.
OK, so I'll just answer a couple questions which I promised to answer.
So the comment made by several people about last week's examples, say, of the rock and the rolling was that oh no, that showed me that my physics intuition is quite off, and bummer.
And so then if you're going to teach it, how do you make sure, well, A, how do you learn to think intuitively about these things?
And B, if you're going to teach it how do you make sure-- related to that-- you actually have the intuition before you teach it.
There's two answers to that.
One is that it's very rare that anyone ever achieved a goal that they didn't set out to do.
And then just happened upon it.
So I would say that's true, also, with developing intuition in these areas.
You have to make it your goal.
So it's not going to just happen by chance.
So then questions, OK, if it is your goal how do you go about it?
One way is anytime you do a calculation, at the end of the calculation ask yourself this question.
So John Wheeler-- who was Feynman's adviser-- he recommended the following question, which I think is a fantastic question.
So the question is if you could time travel back and talk to your earlier self what one or two sentences would you tell your earlier self before they started solving the problem that would make the problem just sort of flow smoothly?
OK?
So what sentence would you tell your earlier, pre-problem solved self?
So this is after you solved the problem.
So that forces you to give a insightful summary of what you learned from the problem.
So it forces you towards the intuitive way of reasoning.
And you'll find it's quite hard at first.
Because you don't have the intuitive library.
But at least it points you in the direction to be going.
OK, now what else?
Well, in chemistry there's various books that help develop intuition.
They have all kinds of intuitive reasoning questions in them.
And some are quite good.
Some of the books are quite good.
So the one in chemistry that I like is called Voyages in Conceptual Chemistry.
And in physics one of the ones I like is called Thinking Physics by Epstein.
I think Epstein's like that.
So what that is, is that's basically, one question per page.
And it's multiple-choice intuitive reasoning question.
And they're about all kinds of interesting physics, like why do tea kettles whistle?
If you shake bottle of soda up, why does it explode?
And so it's filled with really, really fascinating puzzles and ways of thinking.
And then with explanations.
So just by thinking about them, reading the explanation, trying to understand, OK, then asking yourself Wheeler's question at the end of reading the explanation, even if you didn't solve it, you can develop a lot of intuition.
I don't know a good one for math or biology.
And if anyone does, I definitely appreciate any suggestions.
OK?
But there are resources in some fields.
And the other is just whenever you find paradoxes go after them and share them with everyone.
And ask everyone you know,do you know any paradoxes I can work on?
That'll also developed intuition.
OK, and then related to that, one comment was I've noticed that teaching is the best way of learning.
How can you incorporate that into a course?
So yeah, that's true.
So sometimes you just have to accept that the first time you teach the thing you're going to be learning a lot.
So that's fine.
And the second time you teach it, it'll be much better.
So you'll actually develop a lot of intuition the first time you teach a course.
So for example, I found when I was a graduate student, I was a teaching assistant for Physics 1A at Caltech, which was like 801.
And I found I had a huge number of misconceptions about tension.
I just had no idea what tension was.
For example, I thought tension was a force.
And now that seems ridiculous to me.
And it doesn't seem ridiculous to you that's because you're in exactly the same state I was in before I was taught physics 1A.
And I've got myself all in a twist when I was trying to explain it.
And I finally tracked it down to actually, that particular misconception.
So then I developed a bunch of questions for myself, which I also asked students about tension.
So teaching is learning.
And then how do you incorporate that into a class for the students?
Well, you can ask them to teach each other and argue with each other.
So we'll talk about that in the session on interactive teaching.
But yeah, it is the best way of learning.
And so if you can get the students to do that as well you'll be increasing their learning.
And then the last question for the moment was what's my response to the following article-- so it was in the globe on February 15, 2009.
And it said, don't open with a joke.
So it was summarizing a paper called "Increased Interestingness of Extraneous Details in a Multimedia Science Presentation Leads to Decreased Learning."
Journal of Experimental Psychology, December, 2008.
So they summarized that.
So basically, what they found is that if you start with a joke that's extraneous to the material it actually decreases the learning.
And that, I think, is quite plausible.
So I recommend that you start with something interesting in the beginning.
But it should be related to the material.
So the problem wasn't the joke.
The problem was that the joke wasn't related to the material.
So the thing should be interesting.
Because you want to draw people in.
You don't want to conclude from that article that you shouldn't start with some interesting.
But you should start with something interesting that's related to the material.
OK, so redesigning questions.
So here is an actual question that we will redesign.
And I've left up the Bloom's Taxonomy there.
So actually, I have a bunch of questions.
We'll do a couple of them.
And we'll review, probably, one of the questions we did earlier, which is about the cones.
OK, so the question to start with-- and we're going to go that way and that way-- is find the eigenvalues of this matrix.
So for those whose linear algebra is rusty, the eigenvalues of a matrix so a matrix, m, times some eigenvector e is equal to lambda times e. So if there's some vector that the matrix just brings back to itself with some constant, that's the eigenvalue.
And this is the eigenvector.
So eigen meaning own in German.
So somehow this vector belongs to the matrix.
It's said to be an eigenvector of the matrix.
It's preserved when you hit it with the matrix.
You get back the thing itself.
And in that way it belongs to the matrix.
OK, so there are recipes for computing it.
And so here the question is, I'm asking students to use the recipe for computing the eigenvalue.
OK, so now I'm going to ask you the following question, which is to design-- well, first of all, where would you say that guy goes?
It's sort of comprehension.
You know, it's basically computing.
Maybe it's a bit of application.
But it's basically comprehension.
OK, so can we go lower on the taxonomy towards knowledge?
Can we go higher towards the higher-level reasoning?
Let's try to construct questions on this end.
OK, so everyone understand the thing to work out?
I'm going to ask you to try to think of stuff that could be put here and here.
So this is lower level and this is higher.
Does anyone need me to explain-- or want me to explain-- more about eigenvalues or eigenvectors?
OK, so find a neighbor or two.
And try to think of stuff that can go in this box or this box.
And generally, the higher-level questions requires more knowledge about the material, which is one aspect of their being higher level.
You need a bigger margin of safety when you're trying to design them.
But don't let that bother you.
See what you can construct.
And you can use the levels on Bloom's Taxonomy to try to create some kind of questions around it.
By the way, the answer to this question, by the way, is i and minus i, just in case you are curious.
OK, so let's see what examples you've come up with, so related to this question.
Obviously, you'll change it around some.
Let's say either side, this side or this side.
Someone I haven't heard from?
Yes, can you tell me your name?
Mike
Mike, yeah?
What is an eigenvalue?
OK, so what is an eigenvalue?
Where would you put that?
Lower.
[INAUDIBLE]?
Oh, yeah, good question.
It depends where you put this one.
This one is computing, which I roughly put that somewhere between comprehension and application.
So right now I would say, we're somewhere around there.
So your question is to define eigenvalue?
OK.
I'm just short-handing it as defined lambda.
OK, so that's one question.
OK, another question.
Can you tell me your name?
Wendy
Wendy, yes?
So related to that, is that you could put that question higher.
If you asked them, what does an eigenvalue mean?
If you word it differently instead of [INAUDIBLE]
OK, which may have been what you meant as well and I may have defined lambda.
OK, so you're saying, well, you can actually ask them to say, what does eigenvalue mean?
OK, so how would you if they know what it means?
So what would you look for in something like that?
If they could describe it to you, or maybe tell you a problem in which it'd be useful, in using it?
OK, so yeah, let me write that too.
Create a problem that would benefit from lambda, that would need lambda.
OK, so for example here, create.
That's gone pretty far down towards synthesis.
I really should have flipped the list around.
But synthesis is quite high on Bloom's taxonomy.
So you're actually going to create a problem.
It's like the Newton's law example.
Yes?
[INAUDIBLE] instead of saying, again, what does lambda mean?
You could say what do the results from finding lambda?
What are the results of that solution?
OK, so how does it help you to know the eigenvalues?
Is that the question?
No, I'm just saying instead of the general asking for the definition, what does lambda mean?
What is an eigenvalue?
What does it mean?
But after having solved the problem saying, what does your answer mean?
Ah, OK, so even if you ask them this first you can say, OK, interpret your answer.
Yeah, and that's a-- generally-- very good policy.
So interpret.
and where would I put that?
Eh, interpret is somewhere like analysis, maybe a bit of synthesis.
But yeah, it's quite high on the list.
So interpret, that's another really key word.
In fact, it's often neglected, right?
We just ask students to solve the problem.
And wander, wander, wander.
They get the thing that looks like an answer.
They put a box around it and they're done.
But actually, you can massively increase the level of a problem just by adding to the problem, interpret your answer.
What does it mean, that that's an eigenvalue?
And maybe they would say something like this.
But even that would be better than just having done the computational procedure and got an answer out of it.
Yes, can you tell me your name?
Ben
Ben, yes?
I might ask, does that matrix have any eigenvalues?
OK, so yeah, so let's put it over here.
Right, so there what you're doing is you're not presuming.
You're not key-wording what they should do.
When you do this-- especially if it's a linear algebra course-- they've probably seen a bunch of things like this.
Like in a calculus course, integrate these things by parts.
They just know I have to integrate by parts, whereas here you're at least one step higher.
Because they don't know whether there's an answer to the problem or not.
And they have to think, well, how do I know if they're eigenvalues?
So there's actually one intervening level of sophistication.
Other suggestions?
Yes, Adrian?
Just give them two numbers and say write a matrix that has these as the eigenvalues?
OK, so let me continue the list over here.
OK, so create a matrix which has the following eigenvalues.
So say lambda equals 2 comma 3.
So create a matrix, create a meaning.
We're already high up in the taxonomy toward synthesis.
And actually, I would even use a related one, which is-- so create a real matrix with eigenvalues i comma minus i.
So that one, is quite related to this.
And this is the answer.
So it's quite a hard question.
Even if people can do this and find i comma minus i, this one is very hard because you're asking to apply a constraint that the thing be real.
So it's not just any matrix.
But it's that the thing be real.
And then you have to really understand quite a lot about matrices to do that.
Yes, can you tell me your name?
Scott
Scott
So, I was thinking of a synthesis question which would be is after multiplying the item the eigenvector by the matrix how is it rotated?
OK, so what is the geometric effect on the eigenvector of the matrix on the eigenvector?
Right, but the question is if they understand what an eigenvalue is
They should be able to answer that right away.
Right, but it really tests for understanding of what the meaning is.
So it's another way of approaching the question of do they understand eigenvalue means?
What is the geometric effect of the matrix on the eigenvector?
And then hopefully, they'll be able to say well, actually, it doesn't rotate it.
It just scales it.
And the eigenvalue is the amount of scaling.
OK, and that demonstrates quite a high level of understanding too.
And This is one reason that oral exams, generally, are quite good ways of evaluating.
Because you can ask these kind of questions.
And as soon as you see that they understand you move on to something else.
Which of the flip side is that's why oral exams are very disconcerting for the students.
Because you spend no time on the things they understand.
Right?
So as soon as you know they understand it, you move on to something else.
So the student just feels like, what the hell happened to me?
I was doing fine.
And they asked me about something else.
Yeah, that's exactly right.
That's why they asked you about something else.
They knew there was nothing more here to find out.
You understood it.
But that's why oral exams are such a good way-- so efficient-- for evaluating.
So in Cambridge we used to do oral exam interviews for admission for undergraduate.
So people would be admitted to the major right away.
So we had two 20-minute interviews.
And we found those were much more reliable than all the exams that they did in high school, the equivalent of AP exams.
Because you could really ask questions that really probed understanding and right away see what people could do and couldn't do.
So geometric effective of m on e creating a matrix, so you're asking, so what's the geometric effect?
It's not just comprehension.
You're asking them to apply it and maybe, in a bit in a new area.
They hadn't really thought of eigenvalues and eigenvectors as geometric at all.
And now, all of sudden, you're asking them that.
You're asking them to transfer their knowledge, which is one of the fundamental goals.
Yes?
Once you give them two matrices that has the same eigenvalues you can ask them to see if they are related in some ways
Right, so for example, let's do this one.
So how are those two matrices related?
So they both have eigenvalues i and minus i.
This one obviously does because those are the diagonal values and there's nothing else.
And this one, you have to do it by the magic procedure.
You don't have to.
And so how are they related?
So that is some kind of analysis question.
And it's quite a difficult analysis question.
It tests for really, do they understand what matrices mean?
So that's why I've underlined this word.
The questions all in this column are all somehow testing that they understand what things mean.
So that ties back to the point I made earlier, which is that the whole fundamental point of asking good questions is to avoid rote learning.
Is that by asking good questions you're fighting rote learning.
And rote learning is the bane of most education.
Right?
Rote learning is too often the result of education.
And it's the-- in my view-- fundamental problem with education.
So by asking questions that are different you're actually starting to fight that and break the habit.
So for example, how does rote learning show up?
Well, in the bouncing-ball question, so the force is-- in the instant that it's stationary, while it's bouncing-- people say mg.
While this mg is right.
The gravity, but the contact-- let's say the constraint force-- in this case it is a very full-contact force is mg. One of the reasons they say that is the misconception that they think a equals 0 because v equals 0.
So that's one reason.
But the other reason is that they've done so many problems with books resting on tables, balls resting on tables, where the force is mg.
Right?
So they've now induced a new rule of physics, let's call it Newton's fourth law.
That anytime you see a contact force it is mg.
So contact force equals mg maybe times cosine of theta.
[LAUGHTER]
Right?
or sine theta.
I guess it's mg-- well, it depends how you measure theta.
But theta is the tilt of the plane.
So if the plane is tilted 0 degrees just to ground, well it's then mg.
So that kind of learning is the result of rote learning because they've solved problems without understanding what they mean.
So instead of figuring out that oh, actually, you could understand it from Newton's Second Law and understanding acceleration, they think oh, there's a new pattern here.
Right?
Oh, a new pattern is whenever I see contact forces it's mg times maybe cos theta or sine theta.
Yes?
What's the exact question with that system and what's the normal [INAUDIBLE]?
Yeah, so the question was so you drop the ball, say from here.
And when it bounces off the steel table there's one instant when it's stationary on steel table.
And what are the forces on it at that instant?
I remember from last week
Yeah, no, fair enough.
I should have stated the question out again.
So this is stationary for an instant.
All right, so actually the ball is quite compressed here.
Yes?
Isn't it impossible to give an exact answer for the force with the [INAUDIBLE] if you don't have the duration?
It is, right.
So it is impossible to give the exact answer.
But what you ask them is, well, how big is it roughly?
So I always protect myself by saying roughly.
And then you can ask anything you want.
And because, really, all I'm interested in is do they think it's 1 times mg or do they realize it's 10 to the 4 times mg?
And those are so far apart that I don't care about factors of two or three or even 10.
Now, how can you know that it's 10 to the 4 times mg?
Well, that is quite a hard question.
That's way down there.
Even though you'll see some lists of Bloom's Taxonomy say estimate is a low-level skill, not that high in Bloom's Taxonomy.
It's almost like computing somewhere in comprehension.
I don't agree with that.
Because as you saw from the example where students couldn't estimate 3.04 times 5.3, they're terrible at estimation.
So estimation actually hits one of the big misconceptions and reveals a lot of the misconceptions.
So I put estimation almost at the level of synthesis because it forces you to really synthesize your understanding.
You can't hide behind formulas.
So the next thing I'll do is say, OK, let's estimate this and see if you can do that.
And if they really understand the system they can estimate it.
But it's quite hard.
And I find almost no students can do it on their own.
But the method of estimation is, as you say, you do need to know the contact time.
So the acceleration is not 0 as you say.
But it's delta v over delta t, where delta t is the contact time.
Well, it's going v and then it's pretty much going v up, so that's about 2v.
But I don't care about factors of 2.
And then, what's the contact time?
Well, there's a couple methods for doing the contact time.
One is you listen for the pitch.
And whatever the highest frequency in the pitch is-- suppose it's 10 to the 4 hertz-- well, then 1 over that time, 1 over the frequency is roughly a time.
So it's probably the contact time.
So that's one approach.
The other approach is to say well, actually, physically what's happening?
The ball is hitting the bottom.
And this end compressing.
But this end keeps going.
Because it has no knowledge that the bottom of the ball hit the ground.
How's it going to know about that?
Well, the bottom of the ball has to tell it.
How's it going to tell it?
Well, it has to send some kind of signal.
How does it send signal?
Sound.
So it sends a sound wave upward.
And the sound wave travels in steel about 5 or 10 kilometers per second.
So you suppose it's a 5-centimeter ball, then it's about 10 to the minus 5 seconds is your contact time.
So this is really short.
So suppose v is something like a meter per second, and divide it by 10 to the 5 seconds.
You're talking about something like 10 to the 5 meters per second squared, which is 10 to the 4 times the acceleration due to gravity.
OK?
So I went through because I think it's interesting but also to show that estimating actually requires generally, very high-level skills.
And it's quite hard for the students.
Therefore you should do it.
So there are some people who say, well, it's very hard for the students.
Don't do it.
But again, there's another rule of thumb in teaching, which is nobody ever learned anything that you didn't actually try to help them with.
Well, that's not quite true.
But if you never teach something they're not very likely to learn it.
So if it's something hard but valuable you have to start teaching at some point, which is why I put up that graph earlier.
Oh, it's gone now.
Where in the earlier stages of teaching you do things more approximately.
And then in the later stages you do it more exactly.
So what I've told you now is a slight lie.
So this brings up the question from last time, what about lying?
Is it useful?
Yes, it's essential because this argument is simple enough that once you understand it you can just keep it in your head.
And it's a bit of a lie, the reason being that the ball doesn't have a uniform cross section.
So as the ball compresses the contact area changes, whereas if the ball were cube or a rod the contact area would stay the same as it compressed.
So there's a correction factor because the contact area changes.
Now, that correction factor is not very big.
It turns out to be the fifth root of some dimensionless thing.
And so this sound-wave argument you put in to the 4/5 power and then you put in the other thing to the 1/5 power.
So it's almost correct for this spherical ball.
So in a later of course it actually adjusts for that factor.
And then in a much later course, probably, what I would do is calculate the exact 2 pi over whatever factor that shows up as well, by way of increasing the symbolic complexity.
But even this estimate, doing that is conceptually hard.
So I would say start that early
I think the point I was trying to make was that this-- I suspect that the students answered that wrong because of real problem solving, and that they expect there to be a solution given the data that you [INAUDIBLE]
Well, if you've worked with them a bit on estimating they know that they're going to have to do some thinking.
If you say compute the force I agree with you for sure.
They're going to think they have to be able to do it from the data there.
And they'll just say it's got to be mg because that's the only other force I know.
But if you say estimate then there is a bit of a queue that they have to include some other information which might not put in the problem.
And so that mitigates the rote learning.
But what I find, even then, is they usually still can't do it because their misconceptions about acceleration are so deep and about equilibrium and force.
They think f equals mv, basically.
OK, so let's just look at what we've done here.
So I would say you can define lambda.
You can define eigenvector.
You can even say state the computational procedure.
I'm not sure where I would put that.
Depending on how people learn it, this could be purely a knowledge question.
Because people could state the procedure but not really understand it, not be able to do anything.
Or actually, you could say program the procedure.
Well, that would be something over here probably.
Because to program it you probably have to understand what you're doing pretty well.
So stating the procedure, it depends how you ask them to state it, whether it's a low-level or a high-level question.
So I'll put program over here.
OK, so we've gone down and up.
And again, I want to remind you, I'm not saying all your questions need to be here.
But generally, the questions are too much over here and not enough over here.
So this gives you a recipe for thinking of new questions.
Yes?
So throughout the course do you basically, maybe start on the first three in the beginning and then you move it more through until you reach the end of the course?
More fractal.
The question was as the course moves on do you move more towards synthesis?
I actually try to make it fractal.
So within each unit and each day even, I try to maybe have something, knowledge or comprehension just to make sure everyone's tracking.
But try to have something interesting as well, all the time.
So something synthesis, analysis, maybe some evaluate.
And so each problem set-- for example-- I'll put, generally, warm-up questions.
So basically they're marked warm up so that people know OK, those are going to be knowledge and comprehension.
And you need that.
Because otherwise there's no point trying to do synthesis if you don't have the base knowledge.
But just make sure you understand those, just in your sleep.
And then regular problems, which tend to be somewhere in the middle, say application or analysis.
And then bonus problems which are just optional.
But they're for people who either find all the other problems easy or are really curious.
For them I put either synthesis or evaluation.
For example, in the approximation class the last problem set had design an email indexing system.
So that was the bonus problem.
And there was a bunch of warm-up and regular problems which were involving Unix text processing.
And then for the people who are really curious actually, put it all together and make a really fast email-indexing system so you can just find any email that you want just by doing keyword search.
Sort of like a web search but just on your email inbox.
Because I actually use that myself.
Because I find folders are just hopeless.
It just takes all my time.
I have to figure out what folder things are in.
And that could be many.
So I just keep it all in one giant inbox with, I think, 150,000 messages in it.
And I just search through that.
And I have an index that's rebuilt every week or so.
And so I can search through it really fast.
So it's actually very practical problem and it gives you an example of the levels of things you can ask.
So try to do it on a fractal scale.
Each class should have something, mix the levels each homework, the exam.
So the general rule is to mix them up all the way.
OK?
So hopefully, from that, you feel confident that you have tools that you can make interesting questions.
And this does have lots of payoffs, which one of them is that questions that are enjoyable and educational, students love them.
So when I practiced what I preached and made these kind of problems, mix of problems for my classes, often students have said, oh, can you please have more problems?
Which is very rare.
They said, oh, these are some of the most interesting problems I'm likely to get at MIT.
And I used to read them out in my living of my independent-living group just so that other people could hear the interesting problems and we'd try to solve them together.
And then I've had several emails after class finished, saying, oh, you promised to post the optional problem set.
Where is it?
So it's because the problems, actually, somehow connect to things that people find real and interesting.
So that's a mix of the perceptual.
Don't make it just purely symbol manipulation.
Try to connect to ways people look at the world.
But also, try to connect to higher-level reasoning skills.
Question?
Do you always use bonus problems?
I do.
I find that works really well.
Because it's like a pressure-escape valve.
It caters for the diversity of backgrounds in the class.
The people who are feeling very worried by the material, they can just skip them, no problem.
And they don't feel like they have to do them.
And they don't feel like, oh my god, I can't do them.
I'm going to get points off.
And the people who find the other stuff easy, they find something for them too.
So yeah, I try to always use bonus problems.
And I learned that from Knuth's book, called Concrete Mathematics.
So actually, I think they have five levels of problems at the end of every chapter.
There's warm ups.
There's homework problems.
There's exam problems, bonus problems, and research problems.
So the research problems were the ones that are not yet solved.
So at least you know they're not yet solved.
So you're going to be spending a while on them.
And then in the second edition I think some of them were actually solved.
OK, so if everyone could just take a one minute and fill out the feedback sheet?
Oh yeah, and let me just say, why I'm doing this now at 10:50, of the questions on the sheets before was the feedback sheet, the person said, I've been using them in my class and they've been really useful.
But I find that the response rate has been dropping.
What do you do about that?
And I have to say, that I found the same thing.
This year I found the same thing.
Not in this class but in my other class, whereas I didn't find that last year.
And I was trying to figure out why.
And theory I have is that I was trying to put in too much material in the class and not leaving the students time before the five-minute mark to fill out the sheet.
So one thing, I've been trying to mend my ways and make sure I end class at 9:50 or 10 minutes to the hour so the students have at least one or two minutes to fill out the sheet without feeling pressured to get to their new class.
So I'll let you know how that goes over the next week.
So take a minute now and then that'll maybe make you think of some questions which we can answer in the next two minutes.
And then while you're doing that, just a couple announcements.
So sorry I didn't say anything before about what to do with the equation treatments that you shared with each other.
So just discuss them with each other.
You have a collaborator.
And then revise your treatment based on what that person told you.
You don't have to turn in your revised treatment.
You guys are all graduate students, or almost all of you.
You're all taking the class because you're interested.
I'm not trying to police everyone.
But the goal here, and what you should do, is take the benefit of the collaboration, revise your equation treatment, and share it with each other.
And then just bring in your original one next week.
And then I'll also have a couple of readings for you to do for next week.
And also a short problem to work on, things to think about to bring in next week as well.
OK, so sheets and then questions.
Question?
So I tried doing these sheets for a class that I was teaching yesterday.
In fact, I found that I got a response rate of 15%.
If your response rate from the beginning's very low and [INAUDIBLE] to increase that rate?
Right, so 15%, yeah, I find I'm getting that around now too.
And I think what's happened is that I've got people in the habit of not filling them out because I wasn't giving them enough time.
So I may be hosed because now they're in the habit.
So I'm trying to mend it.
And I'll let you know if that fixes it by giving people actual time before they feel rushed.
But it may be too late.
And I worry that it is too late.
Because last year I was actually more diligent about making sure they had time.
And I found the response rate stayed about 60%, 70%.
But even I find the 15% very useful.
And I found that-- people even in the past, when the response rate was 30% in another class people said, I didn't have a question all the time.
But I knew when I did I could ask it.
So even the 15% isn't bad.
It's still useful.
But yeah, I would like it to be higher.
And I'll tell you how that goes.
Question?
[INAUDIBLE]?
Wiggins and--
McTighe, yeah
Oh, McTighe.
Where are they based?
So where are they?
They're in New Jersey, and they actually consult on educational course design and run workshops and courses.
And we actually invited Wiggins last year to come and speak.
So I'll put a short reading selection from their book called Understanding By Design for optional reading for people so then you can learn more about that.
I find the basic form that they say very useful.
I find applying it the way they describe, sometimes I just can't square it.
But it may resonate with you quite well.
And the overall approach resonates very strongly with me.
So hopefully you will like it too.
So I'll put that on the website and some readings for next time about course design.
Yes?
Just a comment, actually on your web page, where the response sheet [INAUDIBLE] and situation where only a few people were very dislike the current [INAUDIBLE].
They don't like it this way and so they want to change it.
But the 80% who don't feel [INAUDIBLE] is actually OK with it.
[INAUDIBLE] the change.
It makes most people unhappy
Unhappy, right, so that can happen.
And so that's the silent majority problem.
So what I do when I get comments like people saying, well, I wasn't happy about this.
Sometimes what I do is-- especially if it doesn't resonate with my intuition about how things were going, and I thought they were going well-- I'll say, well, there were a few comments about this issue.
What do people think?
And just take an informal straw pole before I make big changes
I think it helps to have a positive question on [INAUDIBLE], like, what did you take away from this lecture?
Or what did you really understand?
Right, I think that's right.
It helps to have a positive question.
So that's why I say, either what you take away, or what helped or hurt?
So there's an opportunity to say something helpful.
And any other comments is also interpreted in a positive way.
Answers from Lecture 5 to questions generated in Lecture 4
OK, and questions from last time.
So they grouped into several ones.
One is how should you choose your levels of your problems-- say, in class or in homeworks-- among the different levels of Bloom's Taxonomy?
Should it be, for example all towards the evaluation?
So your side would be here, sort of the high level.
A Mix?
A flat mix?
Should it be peaked towards the middle with some tails?
So that depends, partly, on your course design.
So to answer that question, you can't really answer that question until you figure out what your course goals are.
So in some ways there's an argument for doing course design first and then working on what problems to do and how to design problems.
But actually, I like the other way around.
I like doing course design second.
Because actually, doing problems is concrete enough to do something.
And then it leads to questions about course design, which we'll answer how to mix the levels.
OK, another one was several people wanted to know, why is tension not a force?
Or in fact, maybe is a force.
So let me explain that.
Because that actually gives you another example of sorting out misconceptions.
So it's always useful.
So I'll show you the sequence of questions I used with myself when I finally convinced that tension wasn't a force.
And here's how I did it.
And then I used the same sequence of questions with students to fight the misconception that's been induced by many pictures of-- for example-- a pulley.
And here's a rope and there's tension.
Sorry, this is actually done the other way.
So here's the rope.
And there's a t labeled there.
And then on the mass there's a force labeled as t upward.
So when you see that you think, oh, tension's a force.
So you see that a whole bunch of times and you're pretty sure of it.
So the sequence of questions is as follows-- say OK, here's a tree.
And I'm going to tie a rope to the tree, rope or a string, it's a mass-less thing.
And I put a force on it.
OK, so I'm going to pull that tree.
The tree's not going to go anywhere.
We're going to pull on the rope with 100 newtons of force.
OK, so you can try this yourself too.
What's the tension in the string?
OK, so take 30 seconds and I'll take a tension.
OK, a tension, anyone?
100 newtons
OK, so in fact that is 100 newtons.
And most people will say 100 newtons.
OK, so now the next question is this-- here is a river and here's a rope.
And me and a friend each pull on the rope with 100 newtons.
OK?
OK, so discuss with your neighbor-- a, b, or c, what's the tension in the rope?
And we'll take a vote.
OK, so take 10 seconds, get your vote ready.
We'll take a quick straw pole.
Who votes for 0 newtons?
Who votes for 100 newtons?
Who votes for 200 newtons?
OK, so now here, how do you know what it's going to be?
Well, you make a force diagram.
And there's a way to convince yourself that there's no doubt about what it should be.
And we just draw the forces on the rope here.
Oh, we've already done that.
Let's draw the forces on the rope up there.
Well, what are the forces on the rope here?
Well, it's 100 newtons from you.
But there must also be 100 newtons from the tree.
All right?
So in fact, these two situations are identical.
So yeah, 100 newtons seems pretty clear here.
So it has to be 100 newtons up there too.
OK, now that-- so far-- isn't the hard part.
But now is when you actually create the contradiction with the idea of it being a force.
It can't be a force because if it were a force you would just add up vectors.
Right?
So there's a couple answers you get if it was a force.
If you're a bit sloppy you'd just add up the magnitudes and get 200 newtons.
If it's really a vector-- you're putting two vectors on it, two forces-- you should get 0 because they're two opposite forces.
But actually you get 100.
The only way to get 100 is that the tension has to be something completely different from a force.
So that's how you know it's not a force.
But how do what it is?
Well, it turns out to be a tensor.
So it's not a vector.
Forces are vectors.
It adds differently than vectors do.
It adds like a tensor.
So tension is more like a pressure.
It doesn't have a particular direction.
Pressure is in all directions.
So things like that, they can't be described by vectors.
So you need something else.
And that's why we introduced tensors.
So now, in freshman physics I wouldn't give them a whole story about tensors.
But I do actually teach this much of it so that they know that is a different object.
And then what do I do about this?
Well, I won't write t unless it's for myself.
But for the students I think it's sloppy and it increases misconception.
It's much better to write the force sub t, which is the force due to tension.
OK?
So even a slight change like that can mitigate the rote learning, where people think oh, tension's a force.
Because you always see it drawn like every other force.
There's mg down, There's t up, must be a force too
[INAUDIBLE]?
What are the two objects you hit it with to make a scalar?
It's the two directions that you're interested in, basically.
So it would be the x direction or the direction along the string and the direction along the string
So it's the two direction then?
You can use the same direction twice.
So it's the direction along the string and along the string.
There's no cross direction.
So it has no shear.
So there's no cross direction.
So it's some component, tension is one component of the stress tensor.
So now, again, freshman don't need to know all the details.
This is back to the subject of lying they do need to not be told fundamental lies.
Like for example, tension is a force.
OK, so now, how do you deal with misconceptions that show up on problem sets?
This is a good question.
So the point was made well, in class you can sort it out right then.
Suppose you ask a question, say, about the Ideal Gas Law.
And you find that people have confused the Ideal Gas Law and the Adiabatic Gas Law.
Well, you can talk about it right then, which is a good reason to do those things in class.
What happens on the problem set if you find there's a bunch of misconceptions?
Well, one thing is, the first time you do the problem set, yeah it's a bit of a problem.
Because you don't know what the misconceptions are unless you've researched ahead of time.
But the second time you do the class you know what they are.
So you can address the misconceptions in the solution set.
So when I'm not totally sleep deprived I always try to get the solution set out the exact same day that the problem set is turned in.
So the students can right away see.
They can turn in their problem set, pick up a solution set or look at it online.
So they can right away get quick feedback.
And they can sort out the misconceptions that way.
So that's a good question.
So it is harder.
Yes, question?
What do you do about the fact that a lot of people will just ignore the solution sets?
Or do you just consider that their problem?
So the question is, what do I do about people who just ignore the solution sets?
Because a lot of students do that.
Yeah, partly I consider it their problem.
Because look, that's there for them to learn.
But partly, to the extent that they're ignoring the solution set because they only cared about the grade.
So one cause of that is they really only care about the grade.
So they just want to wait for the grader to give them back their solutions, their problem set with a mark on it.
And they don't care what's right and wrong.
That, I think, partly, is our responsibility that we've chosen boring problems.
So to fix that, I think, one of the key areas is to use more of the Bloom's Taxonomy and make interesting problems.
And the flip side is to deemphasize the grading.
So actually, I like the PDF scale on homework.
I think we are too micromanaging, generally, about homework, you know, grading it on a percentage scale, on A, B, C, D. P, you did a reasonable.
And D, you blew it off.
You did something but it was a joke.
And F, you didn't turn it in.
I think that scale puts the right emphasis on the grading.
Look, we just want you to do the thing and try it.
And if the problems are interesting that allows people to actually get involved in it.
So there's lots of studies that show that if you reward people for things they already enjoy-- so you start paying them more for doing things they really like-- they actually start liking them less.
So I'll talk about that as one of the political barriers to educational change.
Because that's completely against the conventional psychology of this society, where everything has to be rewarded, merit-based pay and this and that.
So how do you go against that?
It's quite difficult.
But one way, in general, in those problem sets is to deemphasize the grading.
Related method-- which I haven't tried yet but I'm planning to-- is reading memos on the solution set.
So I talked about reading memos before.
And maybe one of you will try it before me and can tell me how it works.
A reading memo is-- I talked about-- where you have the students take a reading and mark-- either online or on a piece of paper-- things that were confusing or things that were interesting and things they noticed.
Well, you can do that with the solution set too.
You say, OK, your assignment, homework 3A is to look at the solution set for homework two and mark anything that's confusing.
So that forces people to read the solution set.
And often what you'll find is that they'll find mistakes in the solution set, which is good to know because you can correct them right away.
But you'll see what parts are confusing to students or not.
Did that help answer your question?
Great.
Do I ever asked Wheeler's question explicitly?
Yes, sometimes.
I should do it more in class in lecture.
But I do it a lot to myself.
Whenever I solve a problem I think oh, what was the key idea?
And then once I see the key idea happen many times I think oh, that's a very key idea.
That should be taught explicitly in class.
So I do that.
And I'm going to talk about that principle of course design today, basically organizing courses around large themes and principles, which is one of the main [?
cures.
?]
OK, so what's wrong with tension as a force?
How far should I go down the taxonomy?
Basically, where in the course should I be?
Really low in the taxonomy early and higher later?
Well, I talked about that a bit last time.
The fractal picture is not bad.
So yeah, you have a few low-level examples and higher-level examples.
But you do that throughout the course.
So you don't wait until the end of the course to get the reward of the really interesting problems.
And then related to that, what about exams and homework?
Should they be the same or different?
My view is they should be the same given the constraint that maybe the exam is done in a shorter time.
So if anything, the exam should be easier than the homework.
Generally, homework people will spend six or seven hours on.
When I was an undergraduate we had take-home exams.
And we would spend 24 hours on them.
You pick them up, you signed it out.
You'd turned it in in 24 hours.
So that was misery.
And it was interesting.
You learnt a lot.
But it was kind of miserable because it just wiped out a whole day from your quarter.
And you had that for four classes.
So four days from your quarter were just gone.
So generally, exams are three hours, maybe two hours.
And so that's much less time than a homework set.
So if anything, exams should be easier, definitely not harder.
Student loath when the exam does different things than the problem sets.
And that is a general principle of course design, that if you want to prepare them for being able to do stuff-- for example on the exam-- you have to prepare them with things.
You have to "teach to the test."
That's actually good.
You want your homeworks to be like the exam.
And that's fine if the test is good.
If the test tests real-world, interesting skills sure, then it's fine to teach the test because they'll come out being able to do that, no problem.
So the exam and homework-- as a rough rule of function-- will be pretty much the same.
Or if anything, the exam should be easier.
Oh yeah, the first time you teach a class.
So all this Bloom's Taxonomy and designing questions, it takes a long time.
And it's really hard to make good questions.
It's easy to make define XYZ questions.
But what about the higher-level questions?
What do you do to mitigate that problem of a huge amount of time?
Well, one of the cures is to cooperate with other teachers.
So you steal their good problems.
So I gave you the T.S.
Eliot saying before, that talent invents and genius steals.
So be a genius.
Whenever you see a good problem just use it.
And whenever someone wants a good problem, if you have one, share it.
And now, should you guard your problems?
I think those days of guarding problems are gone.
They've gone by because of things like OpenCourseWare, putting problem sets online.
It's just not possible anymore to guard problems.
I'm not sure it was ever a good idea.
But now it's not even practical.
So everything is online, all from the previous year.
So the problems can't be guarded.
And it's not practical to make up a whole set of new good problems for the next year.
You'll never sleep.
So I just trust the students not to cheat by looking at the old solution sets.
And if you're grading, that's yet another reason to minimize how strict the grading is.
So if the grading is really, just that they make an effort and try to learn something, then there's no benefit to cheating and looking at the old solution sets.
So you actually make the incentive structure so that they don't need to do all that.
Question?
Do you have to change your problem to a certain extent in terms of plagiarism?
So the question is, do I have to change the problems to a certain extent to avoid plagiarism?
I don't even care.
I don't even change them.
I think, again, it's sort of what Adrian has said, is it their problem if they do it?
And to some extent it is their problem.
If they're going to cheat and look at the old solution set and not acknowledge it.
There's only so much you can do.
And you then get in an arms race with them.
You change the problem to some extent.
But then someone else makes it.
So they can always subvert that.
One person can solve the new problem and give it to everybody else.
So you have no hope of actually winning that war, even if it were a moral war to try to win.
So I don't even change them.
And I say look, just acknowledge, just like you would in science.
I think it's Larry Lessig, the copyright scholar, professor of law.
He used to be at Harvard.
Now it's Stanford.
He founded Creative Commons and various fantastic copyright projects.
So he said, well, there's a couple ways to get-- not compliance-- people to do things.
One is you can do it through rules.
For example, this is illegal.
You just can't do this.
We'll punish you if you do this.
The other one is through norm.
So this is just not done.
In our community we just don't do things like that.
And generally, this kind of-- not exhortation-- but push works much better than the punishment kind.
So I just tell them, look, in science when you grow up and become a scientist of course you acknowledge your peers and people you learn from.
Because that's just polite.
It's just how we are.
We say, please.
We don't say gimme, gimme, gimme.
I'm working on teaching my daughter that right now.
She says, daddy read.
Daddy read.
Mommy read.
Elsa read.
I'm like OK, I'm fine with the Elsa read.
But daddy read, mommy read, that's nice.
And we love reading.
But how do you ask?
And now she says mommy read please, daddy read please.
And so it's the start.
So people already learnt that from young.
So you can tap into that.
Look, how do you act as a member of the community?
We're social animals.
So you don't have to put too much pressure in that direction.
Whereas if you try to do it through a punishment you have to work much harder.
The punishment one-- underneath-- has the idea that we don't trust the students.
Right?
And in that way it's common with changing the numbers on the problems set.
It's that we don't trust you so we're changing the numbers.
So I try to avoid those things as much as possible and try to go more towards this.
Yes?
Has anyone tried to have problem sets without solutions?
So then there's no grading.
And then the problem sets are just for practicing
Yeah, have problem sets for practice with no solutions or no grading on the problem sets?
Have people tried that?
So there's actually whole educational systems based on that.
Not so much in America, but in England, the Oxford and Cambridge tutorial system.
You're given problems to work on.
So I was on both sides of that system.
You're given problems to work with your tutor.
And they're not graded.
I mean, your tutor helps you make sure that you can do the problems.
But everything that's graded happens at the end.
Well, when I was an undergraduate it happened at the end of your entire degree.
There was eight three-hour exams that counted for everything.
And nothing else counted.
So it had some craziness.
That time at the end was pretty stressful.
But before that your tutor was actually your friend.
And your tutor wasn't the person doing the exams.
So you would do problems because you wanted to learn and your tutor would help you so that you would do well on the exam that other people were setting.
So you had a nice alliance with your tutor
Does that work well?
I would say it works well.
At MIT it would work great.
Given [INAUDIBLE] that people are too overloaded.
And they would say, oh well, if there's not going to be any grading at all, I don't have to do anything, while I'd really like to do this problem set.
But I have four other classes that are making me work 20 hours a week.
And I don't have any time.
So then because of the other pressures they might fall away.
But they would like to do it.
So that's why I compromise by just saying, you have to do the thing and make an effort.
And then I trust that once they start trying it they'll enjoy doing it.
So I tried to make it as light as I think you can get away with at MIT, given the pressures from all the other teachers.
But in the ideal world I think that's what we would do here.
There wouldn't be that pressure
[INAUDIBLE]?
Yeah, in Oxford and Cambridge it works very well, provided you have a good tutor.
And there's many students who are there just because it's a thing to do, to go to Oxford and Cambridge.
But if you factor out those students who are just there for the stamp of the place, for the other students it works fantastically well.
I mean, the tutorials are a great discussion time.
You can really kindle people's curiosity because it's really not based on grading, it's based on helping
[INAUDIBLE]?
It's so hard to do.
I mean, I don't know of any comparative studies within one culture.
And I don't think you could really do them well between, say, the English universities and the American just because there's so many other social variables that are so different like the population that goes to university, which universities are talking about the exam system itself, so I don't know of any studies that have been done across cultures and that I would trust.
And within I don't know of any.
But it's a good question.
I know studies in general about rewards, which is that the more you reward people for things that they like the less they like doing them.
So I might give you some readings for that for very last penultimate session.
And for the reading that I'm going to give you for the next session is going to be about tutoring, actually.
So I'll put that on the website today.
OK, questions?
Yes?
[INAUDIBLE] of a question.
So I was wondering, if you do exams or homework problems throughout the semester, you can, maybe, also better monitor how the students develop compared to having one big exam on the end.
And then maybe using [INAUDIBLE] lectures
Right, so having continuous assessment is one phrase for it.
It allows you to monitor, make sure things are going OK.
So all of a sudden you don't find, oh my god, the person didn't know anything.
They got a third, which is like straight D's in your whole undergraduate.
So in Oxford and Cambridge things are graded on first, second, third.
And all of a sudden you had had no idea the student was going to get a third, or they were sick or something like that.
So I think, yeah.
So homework problems and exams have a really good purpose in assessment and helping students get feedback on how they're doing as well.
So for that purpose, grading is useful.
And one solution to that dilemma-- because I don't like to grade exactly because it stresses them out with the grade-- is to give them the solution set right away.
So they can get feedback from the solution set whether they understood things, not whether they had a 90 or and 80, but whether it made sense to them.
And that's, I think, a much more important thing.
But then the other part of grading, which is that it's used publicly-- as part of your record and written to other people-- that part of grading I think is not right.
I don't think it's morally right to, basically, reveal information about students to other people without their permission.
I don't think that should be the purpose of the university.
But the other purpose of grading, helping students know where they are.
And helping us know where they are so we can help them more, I think, is a completely valid purpose.
OK, so there was another suggestion which was to use as many non-physics and math examples as possible.
So I'll try to reform my ways for the non physicists and mathematicians.
It's just those come most naturally for me.
OK, so I think that was the main-- yes, question?
[INAUDIBLE] for evaluating the students or for teaching the students
Pardon?
Homeworks?
Yes, for evaluating versus for teaching?
Yes, because that's one thing to evaluate, right?
[INAUDIBLE]?
It's both.
So there's two, yeah.
So the other purpose is that they learn and they do things.
That's for sure.
So you want them to do that.
And you can do that in class and homework.
And the take-home exams that we had when I was an undergraduate had that purpose too.
You spent 24 hours really learning stuff.
So that purpose is very valid too.
But it's also very valid to know where the students are.
So for example, that basically closes the feedback loop.
You're doing all this stuff.
And now you're seeing the result of it.
And you want to know, well, did any of it take?
Did any of it stick?
And if none of it did, well, maybe you want to change what you're doing
Oh, but you cannot evaluate a person every week, right?
And that's what drives students to copy solutions because they care about the grade.
[INAUDIBLE]
That's why distinguish two senses of evaluate.
There's evaluate for the purpose of helping the students.
And there's evaluate for the purpose of publishing their grades elsewhere.
And it's the publishing their grades elsewhere that worries them.
So you want to minimize that part.
And you want to do as much evaluation towards helping them as possible.
So a solution set-- for example, one thing you can do is say OK, here's the solution set.
Ask me about anything that's confusing.
And then you can put some of the responsibility for this evaluation onto them so it's self evaluation.
But you're still using the homework for evaluation.
It's just you're not trying to say look, that's the main thing.
I'm going to use that to tell everyone else about it.
That's what worries them and produces copying.
So you want to minimize that as much as possible.
And it's hard.
And that's the subject of the political obstacles to educational change session that'll be in the second-to-last lecture.
OK, so yeah, I don't pretend any of these have easy answers, unfortunately.
Oh, there was another suggestion, which is that I make a web forum where people can ask questions afterwards.
Because it's hard sometimes think of a question on the fly.
And that's true.
And the problem is it's not easy to make web forms on the MIT.edu site.
So I'll try to figure out a way to do that.
But for security reasons you can't really easily make web forms that just append to a file or do something like that, although I'm going to look into it and see what workarounds I can find.
Yes?
[INAUDIBLE] scripts?
Yeah, you can use scripts at MIT.edu.
So I have to figure out how to use that.
But yeah, I had that in the back of my mind.
But thanks for pointing that out.
Yeah?
What about forms?
Pardon?
Forms?
Forms?
[INAUDIBLE]?
Yeah, so then you get email.
So the only problem with that is that you get a ton of spam.
So there are all these spam bots that float around and just fill out forms and send junk emails all the time.
So maybe that can be filtered out and it's not so bad.
So but yeah, that's one of the few things you can do on MIT.edu, directly on the web.MIT.edu.
So there's a very solutions to it.
I'll try to think of something around that because it is a good suggestion.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to Course Design today, how to plan a course and avoid the pitfalls that many courses fall into.
In the past few sessions we've seen some of the results, for example the misconceptions, the rote learning, the lack of transfer.
And how do you design a course so that your course doesn't commit more of that and contribute to that problem, but instead perhaps mitigates it?
And it's a difficult problem.
So related to that, sort of flowing almost directly from course design is that making large changes to courses is politically very difficult.
And several people have asked me questions about that.
So actually the penultimate session will be political barriers to educational change.
So for example, the Benezet experiment, what happened to Benezet's experiment?
Why isn't it done more?
So actually, I went to the Manchester School Board, Manchester, New Hampshire, and looked at all the school board minutes from the 1930s to see what happened.
So I'll talk a bit about that, talk about what are the structural obstacles to change so that you know what you're up against so that you can make strategic, sly choices and not get yourself shot at too much.
But that's sort of a consequence of good course design.
And that we'll do in the penultimate session.
This time we'll talk about if you had your ideal world, what would you do?
Sort of a process of successive approximation.
In the ideal world what would you do?
And then in the real world, how do you get as close to that as you can?
So today, course design, which is how do you design a course so that you avoid the main problems with the traditional courses?
So now before I say how do you design a course like that, let me review a bit.
We've seen some of it in the previous sessions.
But what are the problems with the traditional courses?
So the problems in the traditional courses in broad outlines are, one, rote learning.
I think that's the most fundamental problem.
So we've seen many examples of that.
And I'll just remind you of a few.
And I'll show you a couple.
So I'll remind you of a few just to summarize it in one second.
And related to rote learning is that there's no transfer.
So what are the causes?
Well, we'll come to that in a second.
But for example, what do I mean by rote learning?
So rote learning is the fundamental problem.
And it actually causes this one.
If you learn by pattern matching, you're not going to be able to transfer to a new situation unless you happen to have by luck seen that pattern before.
But any really new situation will be completely unfamiliar and you won't be able to do it.
So the examples-- so the canonical example, because I find this actually very helpful to have a canonical example of what is rote learning, I think this is the canonical example.
It's that.
So I have that picture in my mind and the whole story behind that picture is this one.
So this is from Wertheimer again.
So students are learning to find the area of this parallelogram.
And they move that triangle over there, make a rectangle.
They know how to find the area of the rectangle.
So rote learning, you can test for rote learning by giving them this picture.
And the students who didn't understand but it only seemed like they understand, they take the triangle and they don't know what to do with it.
So they can't find the area anymore, because what this reveals is that they didn't actually understand what they were doing.
They didn't understand that the purpose of this procedure was to make a rectangle because you know how to find the area of a rectangle.
They thought what you do is you chop off the left triangle, move it over there, and then you do some w times h. So operationally, it looks almost exactly the same.
But cognitively, underneath, what's going on under the hood, is something completely different, you find out, when students have done rote learning, because they can't do this problem.
So this one actually has all aspects of rote learning.
So if they can't even do this problem, they haven't even done any learning.
But if they can do this one but not this one, they've done rote learning.
OK.
So now if they haven't done any learning, that's a completely separate problem, generally not the problem we have.
We have problems with students who have done way too much learning, but it's all in the wrong direction.
So that's one example.
Have I shown you folks-- probably some of you, the Army bus problem?
The Army bus problem is very amusing.
So there are 1,128 soldiers.
And they have to be bused from one base to another.
So this actually, again, was another National Assessment of Educational Progress problem.
So it was given to a sample of about 50,000 high school students.
So it was a huge study.
And there's 36 per bus that you can take.
So the question is, how many buses?
So it's not a rocket science question.
They're just supposed to figure out how many buses to use and maybe one is the right answer, maybe.
But you just keep shuttling back and forth.
Let's assume that's not what was looked for.
So I'll just give you this.
So 1,128 over 36 is 31 and 1/3.
So now I'm giving you that basically to simulate the state of mind of the students, because it turns out about 70% of the students did this division correctly.
So now there's another 30% who didn't even learn division correctly.
Let's leave those people aside.
Of the 70%, in other words most of them, who could do the division correctly, what did they say?
OK.
So take a minute.
Find a neighbor and see if you can predict what the students say and what the distribution of answers is.
So this is practice understanding where students are coming from.
This is American high school students.
OK.
So can someone suggest one answer that people gave?
Yes
Actually 31 [INAUDIBLE] buses
Right.
So that was one of the answers.
So actually, the way that in America at least when I was in school you were taught to write it as 31 R 12.
So that means 31 remainder 12, which I think is a completely broken way of doing things.
But that's what we learned in school.
And that's what some fraction of students wrote.
OK. Another answer
31
31
31 because they round off
31 to round off.
And another one?
32
32 would be nice.
Right.
So these in fact were the three answers.
And they were all equally chosen, except for this was sort of slightly preferred to the others.
So they were all roughly 25% of-- this was, say, 75%.
These were all about 25%.
I think this was 28%.
So this one was the winning one.
Now just stare at that for a minute.
What does that tell you?
It tells you that-- what it tells me is that the problem had no meaning for the students.
Again, they'd done learning.
So of the rite learning, they'd done learning.
There's no question about that.
They'd learn how to divide.
They all could divide.
But they didn't know what the hell they'd done.
So if you wrote, oh, yeah, if you filled out on the requisition form for the Army I need 31 R 12 buses, where would you be?
You'd probably be peeling potatoes for the next week, on KP duty.
So this shows no meaning.
And what it shows is that for most students, there was actually no meaning.
Here, again this also shows no meaning, but in a more sophisticated way than this.
At least they thought about what to do with the R 12.
And they came up with the wrong answer, but they thought about it.
And here, we'll hope they actually understood.
So of the 100%, 30% couldn't do the division.
Roughly another 20%, it had no meaning.
Another 30%, no meaning.
So now we're at 80%.
And only about 20% really understood what was going on in a really fundamentally simple problem.
So that is a striking example to me of how deep the rote learning is.
It's the same as that problem I showed you last time, which was 3.04 times 5.3, I think it was.
You had to multiply.
And students had to choose-- they had to estimate it.
They didn't actually have to do the calculation.
They had to choose between 1.6, 16, 160, one of these four.
And it was just chance, basically.
So again, the number system didn't mean anything to people.
So it's really fundamental.
It's really terrible.
And it's been studied a lot.
So the problem is not just in American teaching.
It's across the world.
It shows up in physics.
So the physics example I showed you was this.
So this is the bouncing ball.
So it's stationary just for an instant while it's bouncing.
And what everyone drew was that there was-- normal force equals mg and there's mg. Why?
Well, v equals zero.
So there's several reasons.
Here, partly there was rote learning involved, because n was always equal to mg in all these other problems.
So they just triggered off of that.
The other one is that there is indeed misconception.
They actually think f equals mv.
So that's the third.
So here, there's deep rote learning.
There's no transfer.
What are the causes?
So there you see one of the causes
Sorry.
[INAUDIBLE].
Couldn't it also be a problem of multiple choice thing, because [INAUDIBLE]?
It wasn't multiple choice.
They were just given the problem just like this.
No.
I don't think anyone who made a multiple choice would ever have dreamt to put that one on there
Well, I mean this is [INAUDIBLE]
Yeah.
But I think it was phrased like how many buses should you order, something like that.
I mean, they should come up with some integer
They would write in an answer that they would get
Yeah.
No, it was a blank, not a multiple choice.
It was a short answer.
So that's how they could see that they'd done the division correctly or not.
And so all of the ones who had done the division correctly, this is what you got.
Another question?
No.
OK.
Question
So I have another question.
You've shown us many studies that prove that there's more rote learning going on.
And all those studies come from the '30s and the '60s.
So why is it still a problem today?
Oh.
So this one was '84, I believe, '83 or '84.
And let's see, when were some of the others?
So Wertheimer was the '50s or the '40s.
Those examples are the '40s, so yeah, you're right
Why is it still a problem?
Well, every time I ask this one, I get the same results.
So this I've asked between 1998 and 2009, and always got the same results.
What are some of the others?
So another one is quel age est le capitaine?
So that's a French study.
So they asked students-- the question was, how do we know it's still a problem today?
So they've asked students, there are 26 sheep and 10 goats on a ship.
How old is the captain?
It's a good question.
The problem is the answer.
The answer is that people answer 36, because they have to stir the numbers around somehow and get something.
So that was done in the '80s, I believe.
Yes?
That's not quite what I'm asking.
I'm asking, if there is so much evidence that supports the existence of the problem, why isn't it solved?
Oh.
OK. Good question.
OK.
Sorry.
I thought your question was, wasn't it all done long ago so it's not a problem now.
Oh, yeah.
If there's so much evidence to support the existence of a problem, why isn't it solved?
So I would say there's two parts to that answer.
One part is that, a, lots of people don't know about this.
So normally when you teach-- so there's sort of an implicit contract, which is that if I don't ask you questions like that, you won't reveal to me that I'm not teaching you any of these important things.
So for example, if you just take a regular course, it's actually possible to do all the problems by pattern-matching, mostly.
So as an example in physics, you'll have-- say you're doing circular motion.
And someone will say, well, there's a car going around a turn and the radius of the turn is this.
And it's going at 30 miles an hour.
What's the force?
Or the car has a mass, m, what's the force that the road has to exert to keep it in the turn?
Well, you don't have to know anything about circular motion.
All you have to know is that there must be some formula which has-- so let's look at the variables that I just said.
There is F is the unknown.
m is given.
v is given.
And r is given.
So you just flip through the book until you find a formula with all of those guys, which happens to be F equals mv squared over r. And then you just plug everything in.
So you can do all those problems without understanding anything about centrifugal and centripetal.
For example, 95% of students who can find this formula with no problem, if you ask them, well, can you draw me a force diagram, 50/50 on whether the force is outwards or inwards.
You can ask them things like, OK, a car is accelerating.
Where does the force come from?
Or a car is decelerating.
Where does the force come from?
Oh, you know, the road.
That seems really strange.
But it's true.
It's in fact from the road.
So they have huge misconceptions even about linear forces, let alone circular forces.
So the regular way of teaching actually doesn't expose those.
Yes?
Do you think if the standardized exams had higher-level questions on them.
like Bloom's Taxonomy, would that help [INAUDIBLE]?
Yeah.
That would help, actually.
So I can say something about how-- maybe I should've said this last time, about how-- so the question was, if standardized exams were better and had higher-level questions, would that help?
I think it would help.
And you can still even do it multiple choice and still have higher-level questions.
So here's actually a pretty striking example.
Let's see if I can remember exactly.
Oh, yeah.
So you ask students-- these are young kids, but it applies generally.
And it's a question about how to design tests.
So you ask them, I have a piece of red cellophane and blue cellophane.
And I put them together.
What color do I get?
OK.
So now the question isn't so much what is the right answer?
But what do you think students say when you ask them that?
And these are young kids.
So takes a minute and see if you come up with what they say as their answer, their most common answer.
So you ask this as an open-ended question.
You don't give them multiple choice.
So talk to your neighbor about that.
OK.
So I think it's actually very difficult to predict what students actually say.
And I'm willing to eat a chalk if anyone-- well, I don't know if I should say that.
That's a bit risky.
But let's just-- chalk is probably safe.
But let's say I'm willing to eat a chalk, maybe with salt and pepper, if anyone can actually figure out what the students actually said, the main thing they say, because it's really, really counterintuitive.
Yes?
Oh, go ahead
Reddish-blue
OK.
So reddish-blue is one thing.
Yeah.
So that's one answer they give.
Yeah.
Yes?
Brown
Brown.
Yes, they probably do say that
Reddish-blue is like violet?
Reddish-blue is like violet.
Yeah.
But if they are young enough, they might not know of the color violet.
They might, or purple.
Yeah
I'll go with saying violet-purple
OK.
So violet-purple.
Yes?
Yellow
Yellow.
I'm still safe.
So they do say all these.
I don't know about all of them in particular, but they say most of these things.
I'm so far safe from my chalk wager.
Yes?
Blue
Blue.
OK.
Yes?
Black
Black.
That would be nice.
Yeah.
Yes?
Would they say it depends which one is on top?
Uh-oh.
I've got to eat my chalk.
Yes.
So does it matter which one's-- yeah.
In particular, they say, which one's on top?
Can I have salt with that?
So that is in fact the main question.
So now why do I bring this question up?
Well, the normal way you'd make a multiple choice question is you'd ask the question yourself.
You'd think, well, what are the plausible answers?
You'd come up with many of the choices here.
And you'd put them down in the multiple choice.
And you'd never think of putting this one there.
So one way to actually design the questions is that you first do it with a pilot group with a short answer.
And then you make your multiple choices based on the main misconceptions that they have.
OK.
So now what's the obstacle when you do this?
The obstacle when you do this is you get this very funny item response curve.
So the item response curve is the following.
So the idea response curve you get, so it-- OK.
So on this axis is the student's overall grade.
And this is the-- so you have a test say, of 50 questions or whatever.
And this is their overall grade on this axis.
On this axis is their probability of getting one particular question correct.
So now, a usual, a normal question is something like that.
If they don't know anything, they have no chance of getting it right.
Maybe they have some monkey line level over here.
So let's call it something like that.
OK.
But these kind of questions actually have a very different shape.
Their shape is shaped like a J.
So what happens is that-- so this is the good questions.
I think they are much better questions.
But they get this shape.
So what happens is when the students are just guessing, they're somewhere down here.
But as they start to think about it and learn more, then they fall into the misconceptions.
So they actually go down.
So they actually can even be below chance.
And then only when they really understand things do they get up here.
So now what happens, for example, if the SAT ever contains a question like this, they just throw it out.
They say, oh, this question does not measure how good a student we have.
It's actually skewed.
It's all messed up.
So you throw it out.
So these normal, standard ways of making standardized tests actually throw out all the good questions.
And all you're left with is these kind of questions.
So I actually saw an example of this in Cambridge, which was-- so you remember the rolling down the plain question that I showed you last week or the week before?
So the rolling down the plain question is an intuitive question.
It's qualitative.
It's hard.
If you try to calculate it, you'll get yourself in a knot unless you know exactly what you're doing.
But if you reason about it intuitively, you can do it quite well.
OK.
So that question was then put on the end-of-year exam for the sophomores.
And the examiner said, well, actually this question was not a good question, because it didn't measure who the good students were.
So what they meant was that the good students actually sometimes did worse on that question than the not-so-good students.
So they said, OK. We're just going to throw that question out.
I mean, they didn't throw it that time.
But we're not going to use questions like that next time.
So again, here you had a good qualitative question.
And it didn't measure the quote, "good" students.
So it didn't give them what they wanted.
So they threw it out.
So if you design a test like that, you're going to fill it with questions that are just sort of low on Bloom's Taxonomy.
But it doesn't mean you have to do that it way.
So if you want to make a test that is higher on Bloom's Taxonomy and tests misconception and qualitative reasoning and intuitive reasoning, the way you do it is you do a short answer first, put the main misconceptions down.
And you just have to be willing to live with this kind of curve
But when you say you don't evaluate the good student, but the question is how do you in the first place evaluate the good students if you have a normal kind of system?
How do you basically evaluate them?
How do you tell who the good students are?
Yeah
Well, it's a reification that's going on.
I mean, there's this idea that there's such a thing as a good student.
And just because you call it that doesn't mean there is such a thing.
So you need some definitions.
So like for example, what is IQ?
The definition is it's what IQ tests measure.
I mean, that's the only, I would say, valid definition of IQ tests.
So here, good student meant doing well on the standard problems.
Now, I don't think that's a good definition of a good student.
But that is the implicit definition in most courses.
So to answer your question, it's the standard way things are done, the implicit definition that it makes it so hard to change the rote learning, but also so hard to even see that it's going on.
And then the other half of the answer is that-- and when you try to change it, you find all these political obstacles.
And that's what we're going to talk about in the penultimate session.
But for example, people downstream.
And one question was raised about this in the questions.
The next courses will say, well, you didn't teach them X.
Suppose you say, oh my god.
I'm teaching them sophomore mechanics and they don't know Newton's three laws.
Well, I could just do oiler's top and the herpolhodes and the polhodes, and all of that stuff and gyroscopes and all this complicated stuff, which is sort of a nonsense if they don't understand Newton's laws.
Or I could actually sort out their misconceptions on Newton's laws.
Well, suppose you take time to do that.
Well, then they haven't learned what the herpolhode is.
So who knows what the herpolhode is?
So I sort of remember a while ago.
But even I taught the course and I hardly remember it.
But the next downstream course will say, they didn't know the herpolhode because you spent time on Newton's laws.
So that's the political problem.
So you have to then somehow get the people downstream to see that this is a serious problem, that we can only build sand castles in the air for so high before they collapse because the air can't support that much.
And that's what I'm going to talk about in the penultimate session.
Question?
[INAUDIBLE] problem, was the purpose of the test, because if the purpose of the test is to evaluate the students, you don't want to throw in tricky questions
Well, so the question is which question you use, does it depend on the purpose of the test?
It does.
But I think you absolutely need these kind of questions, or you're not actually evaluating the right thing.
So if you only put these kind of questions in, you're not exposing any misconceptions.
So you're actually not evaluating some of the important questions about what the students understand.
Because, for example, if you just give them colors on this question, you'll never realize that that's what they're thinking about.
So actually you're just evaluating can they give you one of the pre-prescribed color answers.
But that actually doesn't necessarily mean that they understand.
So you're not actually evaluating their understanding.
So to evaluate understanding you do need questions like this.
And should you punish them for being down here?
No.
That's the other part of it.
If they're down here, that means they're thinking about it.
So if you then grade them and they came down here and you say, oh, you did really badly, no.
Actually you're down here, that's because you're thinking about it and you got caught by the misconception.
But that's the only way to sort out the misconception.
You have to go beyond it.
OK. Other questions?
And then we'll take a break.
OK.
So it is 10:05, actually, according to that clock, plus an hour.
So 9:15, according to that clock.
So 10:15, see everyone back here.
Take a short, 10-minute break, just as if it were two 1-hour classes with the 10 minute break.
And then I'll put the feedback sheets in the corners.
So when you come back, just make sure you grab a feedback sheet.
OK.
So before, we talked about the problems with what most courses do and how rote learning is a primary product.
And as a consequence of that, there's no transfer to new situations.
So the knowledge is very brittle.
So what are the causes?
And what can you do about it?
So just briefly, the causes partly are no account is taken of the misconceptions students have.
So new rote learning is piled on old misconceptions, because by definition learning is going to turn into rote learning if there's no understanding and meaning given to it, just like the 1,128 divided by 36 is 31 R 12.
So no account is taken of what students have rote learned and what they've really understood, because the way the course is normally taught, those ideas, those common misconceptions aren't exposed in the normal run of the way the course happens.
And another is-- so that's directly related to the problem.
Another one is that the applications are very narrow.
So because the applications in the class are so confined to the same area, it's very easy to produce rote learning.
And so I have a-- I keep saying how pictures are the way to understand everything.
So I have a picture for that.
So I show everyone this picture.
And if you keep this picture in your mind, it'll give you a guide for how you should make problems and what examples to choose.
So suppose there is some idea you'd like to teach.
We'll talk in a bit about why you'd like teach an idea at all, but some idea you want to teach.
And you have a clear conception of this idea.
For example, maybe the idea is something like DNA is the fundamental unit of information in living systems.
OK, well that means something to you.
But to the students, it doesn't actually mean anything.
You need to give them examples.
So you're giving them an example.
A related one is you have some idea, which is dimensional analysis.
And you want to give them an example of doing it, because just stating the Buckingham pi theorem for them doesn't really help them.
So you do an example.
But now picture yourself back in the student mind.
What do the students see?
Well, they don't have this idea as a separate unit yet, because they're still students.
And the purpose of the course is to teach that.
So they haven't got yet.
So they have this merger of idea and example together.
So they see this big package and have trouble making that division between idea and example, in other words, between transferable idea and thing that was particular to the situation.
So of course what do you do?
You give them another example.
So the normal way it's done is here's your second example.
So there's the second example.
But now, again from the student's point of view, there's still quite a big overlap between idea plus example one and idea plus example two.
So that overlap is-- if they intersect things mentally in their head and see what's common, they still get a lot of stuff that's beyond just the idea.
For example, if you do a bunch of dimensional analysis examples related to entropy, then they'll think, oh, dimensional analysis is a way of solving entropy problems.
So here, the overlap is too big.
So what do you have to do?
More examples
Pardon?
More examples
More examples.
And where should they be in this space?
Yeah, way around, right?
So you should make an example over here.
So this example has very little intersection with the other example.
So here, this one and this one and this one, they all have very little overlap.
So now the overlap among all three is pretty much just the idea.
But think how different that is from the way most courses are designed.
Most courses, the examples, you'll do 18 problems with the ideal gas law.
You'll do 18 integration by parts.
You'll do 18 solutions of first order differential equations.
You'll solve 18 ion channel problems with a Nernst equation.
All of those actually don't teach you the fundamental idea to be able to transfer it.
So that's another cause, see, that the applications are too narrow.
So you have to choose your applications as widely as possible.
And this makes teaching very hard, because it means you can't really-- yes, question?
So it can't just be like also the structure involved with it.
it's defined as having a class and having problems.
So obviously, do something in written.
But you never really communicate that.
And if you would have like let's say in physics, you would have [INAUDIBLE].
And maybe you have an exam, an oral exam where you basically can evaluate the students and really can find out if the students understood the main idea.
And then he has to be transferred, transfer the skills.
So I mean in terms of like broadening the way of how you question the students will help basically get the idea across
I think that's right.
So the suggestion was that oral exams are actually-- in general, it's true, are a much better way of evaluating whether the student understands or not.
It's very disconcerting for the students, because, I think I said this last time, basically in an oral exam, as soon as the student understands something, you ask something else.
So you're like, OK.
They know that.
Let me ask them about something else.
So the student feels like they're never talk about something they know.
But that's the purpose of it.
So yeah, an oral exam, like a PhD qualifying exam, forces transfer.
That's what you're checking for in the PhD qualifying exam.
So if you do things like that in the undergraduate course, that's setting a goal of transfer, that this is one of the goals of the course, that you be able to use these skills in new situations I'm going to ask about.
But of course, to be fair, you have to give them practice in doing that.
You can't just give them regular problems and all of a sudden give them an oral exam where you say now transfer it to a new thing.
They'll just hit a brick wall.
But it helps you in designing the course.
So I'll put the backward design diagram up again, to show that process.
So here, you want to choose from as wide a set as possible.
And generally, that's not done.
So I wanted to read a couple of selections from the Eric experiment about what happens in class.
So I know you had a choice of reading.
So who chose the Eric experiment?
I thought they were all fascinating.
So if you remember, there was a couple quotes from Eric himself.
So for those who didn't read it, Eric was a non-physics student who took a summer session physics course freshman, physics course, and recorded-- I think he was a philosophy major.
And he recorded his impressions of what it was like to be in a physics class.
So that's actually-- you get quite a different view of what a physics class should be when someone from outside comes, sort of like this diagram all over again.
He said the class consisted basically of problem solving and not of any interesting or inspiring exchange of ideas.
The professor spent the first 15 minutes defining terms.
And apparently that was all the new information we were going to get on kinematics.
Then he spent 50 minutes doing problems from Chapter 1.
He was not particularly good at explaining why he did what he did to solve the problems, nor did he have any real patience for people who wanted explanations.
You know, the why, the what were the ideas, why would you do it this way rather than that way, those weren't generally part of the course.
So by not making them part of the course, you're actually pushing towards their rote learning.
And that doesn't mean that there are not all these interesting questions.
So later on in this selection, he says, well, actually, you know, I wonder about all the names that physicists give.
But there was no space to ask those questions in class.
He says action/reaction, that's Newton's third law, that presupposes a cause and effect relationship, which implies duration.
But in physics, actually action-reaction happen simultaneously.
So those are the kind of the questions that he's asking himself, Eric, the student, that get away from rote learning.
But the structure of the class doesn't encourage anything like that.
And the only reason he has time to think about that is he's not actually trying to just survive the class.
He's sort of going in as an experimental student, whereas the students who are just trying to survive the class, if any thoughts like that occur to them, they just say, forget it.
No time to think about that.
i have to solve the 10 problems on the end of Chapter 3 right now.
So those are the causes.
What do you do about it?
Well, there are several principles of design.
So last time I talked about two organizational principles, the signals and systems model of how to teach a course.
So this was the teaching equation.
So this is the output, what students can do.
So you choose that.
You also have to understand students', students' thinking.
And then you figure out your teaching given what Given you want them to be able to do and how students think, the misconceptions, how people learn, visual, by examples, diagrams like that, how should I teach?
So that gives you an overall structure for course design.
The other structure, which is quite similar, is backward design.
So backward design, you invert the normal order.
So the highest level in backward design is-- that's the and over there.
So this is a hierarchical structure.
Basically, you choose your course goals.
You find ways to make them operational.
So in other words, how would you measure, for example, being able to solve Newton's laws problems in designed bridges, for example, might be one of your goals.
Well, you would actually give them some bridges to design, things like that.
And then given that you want them to be able to do that, what do you do in class?
So this was last time.
That's this time.
And this is next time.
And as I said before, I start here, just because I find this is at the right level, balanced between concrete and abstract.
So we did that last time.
But now we're here.
How do you choose course goals?
And that, the paper by Middlebrook-- so who read Middlebrook's paper?
A few people.
OK.
So if you haven't read it, I didn't want to overload you with reading, but I think it's also a brilliant paper.
And I know Middlebrook because he was my teacher at Caltech.
And I took the course that he describes in the paper.
And he identified a fundamental problem and a fundamental solution to it.
In other words, how do you choose the goals?
And I myself have actually come to pretty much the same conclusion through years of hard experience.
So this is all to answer, what do you do about it?
Which is, you choose course goals such that the course itself will produce transfer.
So let me explain the problem that Middlebrook identified and what his solution is.
So his particular class is an analog circuit design class.
And generally, the way analog circuit design is taught is you have a circuit.
And you ask the students to analyze it.
For example, you give them a circuit that filters out high frequency noise, for example, in an audio recorder in a CD recording studio.
And you analyze it.
And you see that its behavior does do-- what is its behavior?
OK.
It throws away high frequencies.
OK.
So now that's the general run of the mill.
You'll just get tons and tons and tons of problems like that.
And you learn a wave of analysis that allows you to go from here to there.
But then what happens when the student graduates?
What happens is what Middlebrook calls a fall off a cliff.
So either they go into graduate school or into a company.
But either way, what happens is they're actually given that.
Somebody says, well, actually, I need a circuit that doesn't cost too much but is quite exact and cuts off all frequencies beyond 22 kilohertz and doesn't mess up the frequencies below 20 kilohertz.
This is actually a real problem.
That's how you need to do that to record.
The people who developed the CD standard had to make something like this.
So basically you're making a, quote, "antialias" filter.
So I want that.
Now here, the problem is that the student has learned methods of analysis that cannot be inverted.
For example, one method of analysis is you have a circuit and you just simulate the thing.
You just feed it to your computer, which does all the analysis for you.
An analogy for that is if someone gives you a function to graph, you can use your graphing calculator to see what the graph looks like.
But suppose someone says, well, I want a graph that looks like this.
The graphing calculator can't answer that question.
So they've learned methods of analysis that are not invertible.
So they're actually useless, because the real world problem is exactly opposite of the school problem.
The school problem, the circuit is the start.
In the real world, the specification is the start.
So to have any hope of going from there to there-- this is a one-way arrow, it's like a diet-- you need a different method.
You need to learn invertible methods of analysis.
So if you have invertible methods of analysis, you can go both ways.
And this problem I think is central to many, many, many, many courses, not just to this particular analog circuit design course.
So then, what Middlebrook slowly evolved to was not just teaching invertible methods of analysis, but he made those the foreground of the course.
As he described, "I found in teaching this stuff for a long time, I've actually had to put names and words to ideas for the students to get hold of them.
This came about because for a long time I taught by example.
I used a lot of methods, saying we're going to look into the properties of a certain circuit.
And I would use some shortcut method or trick that I thought was a neater way to do it.
And I just showed them this without actually emphasizing the method.
That was fine.
But when they did the homework problems and exams, their minds just automatically reset to the old way of doing it.
So that didn't work very well.
And I finally realized I had to give names to these things."
And those are the names that he introduces in the paper.
So those names, some of those are specific to analog circuits.
Some are general.
But it's the giving of names that is the central [INAUDIBLE].
So you give names to the big ideas in the course.
And you don't just use them in passing, but you make them the focus of the course.
So let's see.
So in answer to what should the course goals be, roughly speaking-- OK.
Questions about that, because it's very different from how it's normally done?
So if you look, for example, at a biology course, it might be organized by, say in cell biology, organized by the organ to organelle, the little system inside the cell.
But those aren't necessarily transferable to anything except that particular subject.
A physics class might be organized by a whole bunch of kinematics problems and dynamics problems.
But a big organizational principle that is implicit there but isn't said is that, well, actually what we're doing is we're approximating down to something simple enough that we can do.
That's an organizational principle that you can transfer.
But that's never said.
It's just implicit.
If you look at Chapter 1, the systems are simpler.
And then Chapter 2, they get harder.
Maybe they have two dimensions in them.
In Chapter 5, they have two particles in them.
In Chapter 7, they have n particles in them.
So it's this process of successive complication.
And the beginning part of it, where we simplified real-world things into just one particle moving in one dimension at constant acceleration, that was never told to the students.
So the transferable principle just went straight by them, because we never made it explicit.
So you have to, if you ever want to have any hope of teaching these ideas, you have to make them explicit and give them names.
Giving things names is absolutely essential to have any chance of learning them.
And this was known in all the old legends.
The gods did not want the mortals to have names for the gods, because it would give the mortals too much power over the gods.
So some gods were He-who-has-no-name.
And maybe only the high priest knew the name.
Well, you don't want to be the high priest.
You want to share the names with the students.
And that involves a lot and a lot and a lot of thinking.
That's where most of the work in course design goes, is figuring out those main ideas, those transferable goals and ways of reasoning that students can use no matter what they do.
And then to teach them, you need to have it from as wide a set of examples as possible.
So for example, to give you an illustration, when I teach dimensional analysis, which is the idea, and I want them to be able to use it wherever, the first example-- I forget whether it's the first or second.
i sort of flip it around.
The first example is in economics, because you can identify a lot of bogus stuff in economics, because it has the wrong dimensions.
So I want them to see it there.
The second example is solid geometry, volumes of tetrahedrons and cones and things.
And then we do sort of a more standard physics one.
So I want them to see the broad sweep of this idea so that the idea comes clear.
So the thing to do is to identify large ideas.
So these are the-- OK.
So to practice, choose right now a course in your own mind that either you are interested in teaching or teaching now or would like to teach.
And see if you can come up with some big ideas for that course, just to practice what some big ideas are.
And I will show you as an example the large ideas for my-- just to give you an idea of what they are.
I'll show you the tree for my approximation course just so you can see what level these ideas kind of live at.
So the whole course is designed around how do you deal with complex problems and how do you handle the fact that the world is really messy?
And I broke that down into either you manage it or you throw it away.
Those are still too high-level to actually do anything with.
But I wanted to organize the big ideas as well.
So you can manage it by divide and conquer reasoning, so subdividing into smaller problems.
Or you can make abstractions.
Abstractions is basically the process of giving names to things, which is what we're talking about now.
To throw away complexity, you can either do it losslessly-- so this is methods of reasoning that find complexity that isn't there but look like it is there and get rid of it.
So symmetry is number one.
And examples of symmetry reasoning are proportional reasoning and dimensional analysis.
So those are all lossless ways.
They don't actually cost you anything.
If they work, they make your life so much easier.
And you haven't thrown any information away.
Or you can make lossy compression.
So you can do extreme cases, look at only the simple cases of things so you're throwing out information.
Or you can make spring models.
So even though things aren't-- like chemical bonds, you can just say, well, they're just like springs.
So that's a very useful but lossy way.
Or the final one is you can lump.
So you can lump changing processes into one.
For example, a curve, you want to find the area under the curve, well, replace it by one rectangle.
So that general pattern-- those are what I've found are the most useful ways of dealing with complex problems.
So these are the big ideas in that course.
And the whole course is organized that way.
And I show this tree once a week usually, just so people know where they are.
So now, depending on the course, what stuff you have in your big ideas will be different.
But I just want to give you an idea of what some large ideas, some structuring principles are.
And then the course itself is actually organized Unit 1, Unit 2, 3, 4, 5, 6, 7, 8.
So what I've found is, just like Middlebrook found, that not only do you have to give names to the things, but I found that I had to actually organize the course around them, rather than organizing them around the topics that they apply to, if I really wanted the techniques, the ways of reasoning, to transfer.
OK.
So questions about what these large ideas, these reasoning principles, are?
What kind of things to look for?
Yes
What's the topic of the course?
The course is called The Art of Approximation in Science and Engineering.
So actually, it's carte blanche to talk about anything.
So that's why these are probably higher-level objects than in most courses which will be narrower.
But even there, like for example in physics, in the Intro Physics, one of the core ideas is successive approximation.
And that's not brought out normally.
But you could bring it out.
You could say, look, we're using successive approximation again.
Or here's the structure of the course, so that it's plain for the student.
Does that help?
OK.
So pick that course in your mind.
And talk to your neighbor.
And see if you come up with, say, two or three large ideas.
And now what we're going to do is we're going to put them on the board, the reason being that you'll find that people's big ideas and large ideas from other courses you'll be able to use in your course, not every single one, but many of them.
That's almost by definition what makes something a large idea, that it's not just limited to that course.
So it helps to see what other people's are.
So I'll make some space on the board to draw those up.
So I'll give you a chance to develop your list farther in the homework.
Oh, I should say something about the homework, which is that you guys are all grad students, or almost all of you.
You're all taking the course for fun.
It's kind of pointless of me to police whether you're doing stuff or not.
But all the same, I will give you an assignment to do-- and I'll just trust that you'll do it-- which is to work with each other on making large ideas for a course and then figuring out examples for those large ideas, so basically a sketch of a course design And the reason to do this, starting now and also later, is that basically that is the hardest work of course design.
Once you've done that, everything else sort of flows.
The syllabus writing, all that stuff, just flows.
But if you haven't done this, you're just kind of spinning around.
And you don't actually know what you're doing.
It's really hard to actually figure out how you should start and what you should do.
So, big ideas.
Any big idea in any course?
Separation of scale
OK.
Separation of scale, so that behavior at one scale and another scale, you can often separate them and analyze them differently and without interfering with each other
[INAUDIBLE] past problems and then [INAUDIBLE]
OK.
Separation of scale.
So for example, I've seen that done in neurobiology.
It's done in physics all the time.
So in neurobiology, the calcium flux, the calcium concentration changes slowly, so you do the calcium concentration separate from, say, the sodium and the potassium, which is done with the action potential spikes.
So separation scales, very important principle across many fields.
Yes?
Equilibrium
Equilibrium.
Is that right?
So equilibrium, right away you can see if you really want students to understand the value of equilibrium, you have a whole set of examples to use, to steal from.
You can use chemistry, physics, mechanical engineering.
And they all have slightly different senses to equilibrium, statistical mechanics.
But there's some core idea that's the same.
And so it also is an argument for team teaching, or actually talking to colleagues from different fields to steal examples from them.
Yes?
I don't know how to describe this, but it describes a process which we all do research, and that is how to deal with scientific problems.
[INAUDIBLE]
OK.
So identifying a scientific question and how to start studying it
[INAUDIBLE]
Yeah.
Identifying and studying.
Right.
And there's many things that are common across many fields.
And there's many things that are different in each field.
And you can actually have a very fruitful-- I could imagine a really fruitful interesting-- you could almost teach a whole class on this.
Actually, that sounds kind of fun.
You could teach a history of science class on scientific questions that were identified, how they were studied, what principles still are done today.
You could have people read papers from across many fields and read the intro and see how they would study that question, see what questions there are.
Yes.
Sharon
Resonance
Pardon?
Resonance
Resonance.
Right.
Resonance you can have a field day with.
It's in electric circuits.
It's in mechanical systems, the earth-moon system, the tides are driven almost on resonance by the moon and the sun.
Resonance is everywhere.
It's in mechanical systems.
There's beautiful films you can show of the Tacoma Narrows Bridge.
I think the London Millennium Footbridge as well got hit by resonance.
So resonance, another very transferable idea.
Yes?
How to develop 3D models
OK.
So how to build models.
Right.
And that's something that there are general patterns to it.
And some of it's specific to a particular course.
But there are general patterns.
And that's something the students can use no matter what they do later.
Whenever you go to a new field-- so there's a Nobel Prize winner at Harvard, Walter Gilbert.
So he's worked in I think chemistry, physics, and biology, many different fields.
And he said, oh, yeah.
Whenever you go to a new field, there's only a few key ideas in that new field.
And the hard work is figuring out what those are.
But that's what you have to do.
So that is one of the transferable skills as well.
Others?
Yes
Problem solving requires communication
OK.
So communication.
Right.
If you can't communicate, you haven't solved the problem, which reminds me of my Caltech students, when I had them keep a journal.
And then I told them I wanted them to explain-- the journal they liked.
But then I said, on the homeworks, I want you to explain your reasoning so that it reads.
It's not just hence and thus and therefore.
And they said, well, why we have to do that?
We're physics students.
I said, well, later on, you're going to have to communicate your answers to someone.
They said, oh, we'll just hire someone who does that.
So then I said to them, "well, how are you going to explain it to that person?
How will they know?"
I said, "oh you're going--" "oh, I'll explain it to them."
I said, "oh, so you are explaining something?"
They said "oh, no."
Somebody said, "oh, well, actually that person will understand it already."
So then I said to them, "well, so that person understands it already and can communicate it.
And you just understand it.
Who are people going to turn? "
So only then did they agree that it was a good thing to actually write problem sets so they could communicate.
So that's something you can teach in any field and across fields.
You could look at scientific writing.
You could look at writing scientific papers.
You can look at giving presentations.
Doing it on problem sets so they communicate, you can have students exchange problem sets with each other.
If they can't understand the other problem set, clearly it wasn't communicated.
Other transferable ideas?
Yes
Coordinates
Coordinates.
Yes.
So coordinates and-- it's also fun to make this kind of list, because you start to see commonalities and they start to suggest other ones.
So when you say coordinates, I also think, oh, yeah.
That's a general example of representations.
So there's different-- this is probably what you meant is coordinate representations, what you were thinking of.
But you can also think of the general idea of choosing good representations for solving particular problems.
Like for some problems, a symbolic representation may be right.
And some, there's a pictorial representation.
You want to use in circuits, the time domain representation, or the frequency domain representation.
So coordinates, I'll put dot, dot, dot, to say that almost leads to the next one, which is representation in general.
Others?
Yes
Symmetry
Symmetry.
Yes.
Symmetry, incredibly powerful.
Yes.
Adrian, and then you're next.
Yeah?
Then you have approximation [INAUDIBLE]
OK.
So linearization.
And that is of course also huge.
I mean, symmetry is used in almost every field.
Linearization, basically you can't do anything that's nonlinear.
So every field does linearization.
So that core idea would be beautiful to convey that.
Yes?
Error
OK.
Errors .
How do you deal with errors?
There are common principles, probability theory, statistics, that deal with errors in any field.
What are those?
Well, that's what you want to understand so you can move to any other field.
I'll put a dot, dot here, so symmetry, another one that's related to symmetry is conservation.
Noether's theorem is that every conservation law corresponds to a symmetry and vice versa.
I think I've stated that correctly.
But they flow from each other.
So actually I do conservation here as part of symmetry.
So let me stop the list right there, just to say that right away-- so let me put the other one which I talked about before, which is successive approximation, that you don't want to solve a hard problem all at once.
It's sort of like divide and conquer.
But you want to try to solve it approximately and then less approximately, then less less approximately, so you get closer and closer, as close as you need.
That's a general principle of engineering design as well.
So these, just to show you that all of a sudden you thought of a whole bunch of examples.
And probably you can use that in any course you take.
So now you all have like 11.
Right now I'm not saying you have to use all 11 in your course.
But you can start to restructure courses around those ideas and make them explicit and choose which you think are the most important in your field in the ways of thinking and illustrate them with things from your field but nearby as well.
So this evening I'll post a reading for next week.
And I'll post a short assignment basically to continue this line of thinking, just to give you some more practice in course design, fleshing it out farther.
But what you've done is the key step, the hardest step to course design.
So if you could fill out the question sheet, as always very helpful.
And also there was a question before.
Could I make sure to list on the website things I mentioned in class?
Yes.
I should do that.
So if I could request people to, if you have in your notes any of the books that I mentioned in class but didn't put on the website, if you could email me those, I will put them on the website.
So as a sort of divide and conquer approach, I'm sure actually collectively you have all of them.
OK.
So just when you're done, you can put the sheets up here.
And then outside of that door, we'll have office hours as usual so that the next class doesn't get delayed by us being in here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, interactive teaching.
Here is a famous graph produced by Benjamin bloom and his students.
So here on this axis is student achievement.
So after a course is over, students are tested on the material.
And here's their score.
And this is the percentage who attained each score.
So here, they compared three methods of teaching, but I'll just compare two.
So one method is standard one-group instruction, say one-to-30, or today, maybe one-to-90.
And here is the histogram, the distribution of students' scores, nicely spread out.
Now, by itself, it doesn't say much.
But what's really interesting is the distribution base after the same amount of teaching, but done in a tutorial form, with one-to-one or maybe one-to-two.
You can put some quantitative figures on this from Bloom's results.
The average-- so this is one-to-one tutoring, one-to-30.
The average tutorial student is two sigmas better than the group students.
In other words, the average tutorial student is at the 98th percentile of the ordinary teaching group.
So the question is-- the natural question is, how can we make group instruction as effective as tutorial instruction?
As Benjamin Bloom says, he says, finding such methods would be an educational contribution of the greatest magnitude.
Because, for example, it would change our notions about human potential.
We'd actually realize, oh, actually, there's not that much innate in what people can do.
Most of it is how well they've learned, how well they've been taught.
So the great question-- and it's the title of Bloom's paper-- is the search for methods of group instruction as effective as one-to-one tutoring.
To that end, let's think about the following question, which is, what makes tutoring so effective?
So I'm going to ask you that, to consider that question in groups of two and three, whatever's sort of convenient in your region of the room.
So, what makes tutoring so effective?
After you discuss, we'll take reasons, put them on the board, and then I'll show you methods of group instruction that may not quite get us all the way to tutoring, but that answer many of the good characteristics of tutoring and bring them into group instruction.
OK, so please find one or two neighbors.
Introduce yourselves.
You never know.
You might find a teaching colleague.
And think about what makes tutoring so effective.
OK, so take 30 seconds and collect your thoughts together.
OK, who would like to suggest something that your group thought of?
You don't necessarily have to believe it, just something you discussed and think we should share.
Yes?
Could you tell me your name?
Elena
Elena
For one-on-one tutoring, I imagine there's less time for misconceptions to get ingrained in your brain
OK, you can nip misconceptions in the bud.
Whereas in, for example, group instruction, you may not even know the students have a misconception until the final exam.
Then what do you do?
OK, let's see.
This group, could you tell me your name?
So we thought--
Could you tell me your name?
Eric
Eric
We thought in a group, there's probably a lot of just dissemination of information.
And in one-on-one tutoring, it's probably not one person just telling person two a bunch of stuff.
It's going through exam problems
Right, OK.
So the contrast in this is what does the teaching situation look like?
In a typical lecture, someone may just talk at you for 50 minutes, whereas it's hard to imagine a one-on-one tutorial situation where you walk in for your tutorial, and they just talk at you for 50 minutes.
That would seem so odd and so counter to all conventions of human discussion.
It would take a really, really disciplined and well-trained person to sit still for 50 minutes for that kind of treatment, so not initially a good kind of training either.
OK, so I'll say you cannot yammer at the audience for 50 minutes straight.
Yes, can you tell me your name?
Sorry?
Could you tell me your name?
I'm Whitney
Whitney
I think it follows from this discussion that arises between two people that the student experiences validation of what they've understood, and it helps build their confidence
OK, because they're talking in a small group, so they can get validation in a safe setting.
Is that?
OK, so it's safe.
It's a safe setting for raising ideas maybe that you're not sure about.
OK, there was one more.
Yeah, can you tell me your name?
Martin
Martin
Yeah.
With tutoring, it's much easier to identify exactly what the student has trouble understanding, and then tailor the teaching according to that
OK, so you can tailor the teaching according to what you find out.
So sort of like the new electronic SATs.
According to what you don't know or do know, in each question, they give you questions based on your previous answers at the right level of difficulty.
So you can tailor.
Yeah, can you tell me your name?
Azrael
Azrael, yeah
I think related to this, it's very valuable to be able to tailor your teaching style to the student [INAUDIBLE] in general, not just specific points [INAUDIBLE]
OK, you can tailor the teaching style to the student.
If, for example, the student is-- for example, suppose you're teaching math to somebody who's an architect.
Well, you can use lots of examples that are particular to something they're totally fascinated by.
You can tailor a teaching method.
OK, let me get something from somewhere back there, in case I've been neglecting people way in the back.
Oh, go ahead.
You guys had your chance.
I'll be back.
Can you tell me your name?
Graham
Graham
Just the level of engagement, it is impossible to sit there in a one-on-one setting and think about the music you like [INAUDIBLE] the subject matter at hand
Right.
OK, so the level of engagement is much higher in a tutorial.
So that's the complement of this one.
So the student engagement is much higher.
Yeah, just as it's hard for a teacher in a tutorial setting to just talk for 50 minutes straight, it's also hard for a student in a tutorial session to surf the web for 50 minutes straight, or use their BlackBerry or read the newspaper or whatever it may be that the standard methods of lecture distraction are.
Yes, can you tell me your name?
Question about the conditions in these experiments.
So normally in a university setting in this country, tutoring is something that students seek out for themselves.
It's not something that's provided in lieu of lecture.
Was that the case in this?
Was it really just that the same material was covered in the same semester and they just did it two different ways, and everybody had to go to tutoring?
Yeah, so what they did was-- I don't think these were university students.
I think these were high school and even grade school students.
Yeah.
But what they did was they took the same number of teaching hours, and they either taught them in lecture or they taught them in tutorials, one-on-one, or I think sometimes one-on-two
So in both cases, they were required to go to class, in a sense
Yeah.
I mean, I don't know if they actually went to the big class.
But I'm sure they went to the tutorial class.
That's partly of one of the differences.
That's number six.
Yes, way in the back
From experience, I've found that you know students don't understand something.
You don't why until you sit down.
Usually you find there's some gaping hole that's quite often easily plugged
OK. And how do you find that out?
You start asking them, what class did you have?
What did you not understand?
And sometimes they'll just bubble [?
right out, ?]
I never got this one key concept
OK, so you can ask them questions, specific questions to figure out what they don't know, to diagnose what they're confused about.
OK, let's take one more, and then I'll show you some methods that hopefully address what these characteristics-- these missing characteristics of group instruction.
Yes?
On one-on-one tutoring, for the student who might not be as into the effort [INAUDIBLE] added pressure, just feeling that there are other people [INAUDIBLE] student, there's no feeling of why I need to learn much more, [INAUDIBLE].
You don't really prepare yourself for anyone else
Right, so the method of teaching automatically adjusts to whatever level the student is.
So they're not comparing to other people.
They're are actually just comparing to what they don't know right then.
So let's just call it intrinsic comparison with the knowledge, rather than extrinsic with other students.
OK, so there's less emotional churn as a result of that.
OK, so we could continue this list, but this is an excellent start.
So now, if we look through that list, there's several themes that come out.
I mean, it's hard in a group setting to figure out what students don't know.
It's hard to ask them questions.
It's hard to keep people engaged.
It's easy to just talk at people as a teacher.
So what can we do in group teaching to capture as many characteristics of this as we can?
Well, I'll show you methods that I divide into four time scales, ranging from short to long.
But before I do that, let me ask if there are any questions on anything so far.
OK. You were first
How is student achievement measured?
Is it just test scores?
Yeah, that was a test score.
So what they did is, actually they gave them a test.
Like, if it was algebra, they gave them an algebra test, and then tested them at the end, and at the beginning too, just to see where they were to start with
So were the lectures more tailored to just teaching them algebra, and the tutoring was specifically tailored to doing problems in algebra?
Well, the tutoring was whatever they needed to do well on the test.
And the lecturing was hopefully designed that way too.
But it had all the flaws of, you want to teach so students will do well on your final exam.
But what do you do when you have 30 diverse students in there?
You had a question too
I was just wondering, do we know which of these particular suggestions are the most important, and which ones might be good?
We have some idea.
Good question.
The question is which of these is most important, and which is less important.
Actually, they're all getting at something similar.
But there's a middle group in there, actually, that they studied, which was mastery learning.
So mastery learning, it's just like a programmed instruction.
So the idea is that-- Keller Plan is another word for it.
So mastery learning is midway between tutoring and group instruction.
There was one sigma better, so 84th percentile.
So mastery learning, you don't go on to the new topics until you pass the mastery exam on the previous topic.
So you're always building on a solid foundation.
And so you work until you can pass the exam, and then you continue on.
And just by adding that to group instruction, you get one sigma better.
So right away, diagnosing misconceptions, you can see, is quite important.
But there's still another one sigma to get.
And so that's, I think, what all the rest of these items are about.
And we do know a lot of stuff about how people learn.
One of it is questions.
Questions are essential.
So I'm going to talk about that.
But let me just ask again, any questions?
OK, good.
This time it worked.
So now, what did I do?
So this is actually the first method.
So how long did I wait after I said, any questions?
Half a second?
One minute?
No, five seconds.
So the first time, I didn't wait five seconds, because actually a question came up a little before five seconds.
But this time, the second time, I waited five seconds.
Now, why is that?
So that is the first method.
When you ask a question-- for example, "any questions" or "what do you think about this"-- wait five seconds.
And I'll explain why in a second.
But, methods shortest to longest.
The reason I divide them by shortest to longest is the shortest ones are the easiest to incorporate into your teaching.
So anytime you teach, if you ask a question, wait five seconds, so the five-second time scale.
OK, so why do I wait five seconds, and why not two seconds?
Well, the reason is, think about your audience.
Suppose you're talking away at them, and all the sudden you ask a question.
Now, what are they hearing?
So everyone's seen Charlie Brown?
So what does Charlie Brown hear when he goes to school?
Wah wah wah wah wah wah wah wah wah wah wah wah, and all of a sudden that changes and there's a question.
It takes about a second to realize that the wah wah wah has stopped, and you're being asked a question.
OK, so one second's gone.
Let's call it that, the thumb-- one second.
Now, it takes about another second to even figure out what the question is.
Now you say, oh yeah, there's a question being asked.
What did they say?
OK, maybe now let's say it's one and a half seconds.
So now we're at somewhere between two and three seconds.
Now it takes maybe one or two more seconds to figure out what you're going to say about that.
Do I have any thoughts about it?
Do I actually have any questions?
Do I have a thought about the thing they're asking me?
Do I know what the band gap is in silicon, whatever the question may be?
So now we're somewhere between four, maybe near five seconds.
And then it maybe takes another second to get the courage up to raise your hand and be ready with a sentence.
So now we're at least five seconds.
So if you don't wait those five seconds, what you'll get is, you'll be getting a few questions, but you'll get questions from people who already thought about the material, in other words, the people who are already at the 98th percentile.
But if you want to bring everybody in, and get everybody thinking about it, you really need to wait, so you don't get questions just from the usual suspects.
So wait five seconds.
Now, the mechanism is quite important.
If you just say, OK I'm going to wait five seconds, you'll go, any questions.
Oh, done, five seconds is up.
And you'll think five seconds is up in about one second.
And why is that?
Well, you're up in front of a whole bunch of people, and the adrenaline will be flowing.
And adrenaline speeds up your time sense.
All those stories about people who-- like mothers who saw their kid about to get hit by the car, they race out and grab the kid and the car keeps going.
Maybe it screeches to a stop.
They've managed to rescue their kid.
Well, they could act much faster because adrenaline makes the time sense much sharper.
So your idea of five seconds will be more like one second.
There's another reason, which is there's this mistaken idea that if you're not talking, you're not teaching.
So you're being paid to teach.
And oh my god, you're not saying anything.
That means you must not know something.
So you feel insecure, and that increases the adrenaline and only speeds up the time problem more.
So to wait five seconds, you actually have to count it -- one 1,000, two 1,000, three 1,000, four 1,000, five 1,000.
And I actually sometimes just do it behind my back with my hand-- one 1,000, two 1,000, three 1,000, four 1,000, five 1,000.
And it's not like you have to be watching the audience for the whole five seconds.
It's actually better to wait the whole five seconds, and then choose one of the hands that has gone up rather than the first hand that goes up at two seconds.
You say, oh, thank god, somebody asked a question.
OK, now my waiting is over.
Well, again, you'll be back in the usual suspect soup.
So, first time scale for bringing in questions, in other words, spreading the engagement through the class.
Because when people are asking questions, they're by definition engaged.
They're struggling with the material.
They're trying to make it their own.
Wait five seconds.
OK, the next method.
So the next method is short time scale, one minute.
And that is to use a feedback sheet at the end of every teaching session.
So you actually have an example of a feedback sheet that we'll ask you to fill out at the end of this session, which we handed out at the beginning.
And the blank versions of that sheet will also be on the TLL website, so you can actually download it, print it, and use it in your own teaching, as well as my notes on how to use it.
But the main point is that at the end of the class, you give the students one or two minutes to fill out a feedback sheet.
And what I've converged on is the following three questions on my feedback sheet, which is, what was most confusing?
Basically, what question you have that still wasn't answered.
Second question is, what was the most useful or most useless example or demonstration or teaching method that happened today?
And the third question is, any other comments?
So now let me explain why I use those three questions.
The first question, these sheets are all anonymous.
And it's a chance for someone, if they felt a little bit shy asking questions in front of people-- so here in the tutorial, one of the characteristics that was mentioned was that you don't feel nervous about asking a question, because there's no one there, or maybe there's one other person there.
Whereas in a class of 30 or 80 or 20, you may feel a bit nervous.
Well, if you feel a bit nervous, you just put your question on the feedback sheet.
And so then at the beginning of the next session, I answer all the questions on the feedback sheets.
So it takes me more than one minute, but actually not that much longer.
Because you could flip through 100 sheets in about 10 minutes.
You just glance through.
You'll see a bunch of questions are repeated.
And you'll say, oh yeah, I do really need to explain that.
And at the beginning in the next session, just explain all the important points.
And that has another benefit, which is that it links back to the previous lecture, or the previous teaching session.
So now you've made a thread between the previous session and the current one, and you got everyone up to speed into the space where you want them to be.
So that's why I use the first question.
The second question is, what worked?
What didn't work?
And I also had to add, and tell me whether it worked or didn't work.
Because I used to get examples saying, that example.
And I didn't know whether they loved it or hated it.
So then I said, please tell me the sign of the effect as well.
And then I would know.
I would start to build a model of how students think and what works for them.
So every time, basically, I learned that I brought in a demonstration, there was universal love on the second question.
So I said, OK, that really connects with students.
And I would make sure to do more of that.
So it was reinforcement for me and practice for me at being a better teacher.
And what you'll find is that as you use the sheet more and more, you get better and better at predicting how students think.
So for example, you're doing some discussion on the board, some examples on derivation.
And the first time you do it, maybe you don't know it's not going to go well.
And then you read the sheet and you see, oh my god, I was a disaster.
30 questions out of 80 all said, I didn't understand x.
So now you understand that students didn't understand the way you did it.
Well, now you've built a better model of how students think.
So next time you're doing a derivation, what you'll find is-- at least what I've found is in the middle of the derivation, I'll realize, oh, this is not going well.
I'm going to get a lot of questions about this on the sheet.
Oops, but it's to late.
I'm in the middle of the soup.
I can't un-drink the soup.
So I just finish the soup and just wait for the questions.
But then after maybe another week of using the sheet, you actually have quite a good model of the students.
And now before you start the derivation or when you're planning the lecture, you realize, oh yeah, that's not going to go well.
Let me do it the other way, because I know that works better.
So you'll find your model of the students developing more and more refinements and more and more accuracy.
In other words, you're becoming a more expert teacher.
And then the third question on the sheet is, any other comments?
The reason for that is, it's just a free-form space.
And I've actually had students say, oh, actually, when you explained that thing, you actually explained it wrong.
I was like, oh that's good to know.
So then in the beginning of the-- because we did it properly in this other class, and here's the exact reference that you maybe didn't know about.
I was like, oh great.
So at the beginning of the next class, I say, hey, one of you guys pointed out a mistake I made, and here is the corrected version.
And people might not be willing to say that personally, because they say, oh my god, I don't want to correct the professor and make them feel stupid.
But it's much easier to do it on an anonymous sheet.
So I get really useful information on that.
And sometimes you can figure out who did it, so you could send an email saying, oh thanks, that was useful.
All right.
[LAUGHTER]
And it is.
So you may think, oh no, you don't want to be corrected.
But actually, after you tell the students the following enough, so it's again back to this.
In a group, students feel like, oh my god, I don't want to make a mistake.
So I give them the following speech quite often.
Because if you don't make mistakes-- I tell them, look, how much did you know when you were born?
Not a hell of a lot.
You couldn't talk.
You couldn't hardly walk.
Well, how do you learn stuff?
Did you just all the sudden start walking and talking?
No.
You mangled your grammar.
You put "-ed" at the end of every past participle, even though some don't have that.
And eventually you sorted it all out and you stood up straight and you walked.
So basically, how did you learn?
By making mistakes.
So the only way to learn is to make mistakes.
So if you don't make any mistakes, you will learn-- you'll know as much at the start of your class as at the end of the class.
So you want to make mistakes.
So I tell them that a bunch of times.
But if then I'm really embarrassed about making mistakes, then I've defeated the whole purpose of telling students that.
So I actually thank them whenever they point out something that I didn't do right.
OK, so that's the second time scale.
Now, the third time scale is not that much longer.
It's about, let's say, four or five minutes.
And it's a question which I'll call a conceptual multiple choice question.
And I'll give you an example.
OK, so here is the question that I'll use to illustrate it.
Here is a pyramid with a square base.
And it's B by B.
The length of one side of the base is B.
And the perpendicular height of the pyramid is H. OK, so what's the volume?
OK, so does everyone understand the question?
OK. Give it a try.
Find your group or one or two.
Any questions-- any confusions about the question?
Don't give me any answers just yet, but any confusions about the question?
OK, so find your groups of one or two other people, and we'll discuss that question in a couple minutes.
Anyone want to say anything about this?
Is there a section I haven't heard from?
Yeah, go ahead.
Can you just say your name?
Brian
Brian, OK.
So you like D but not the others?
OK, can you tell me, for example, why you don't like A?
Well, because all of them just have the wrong proportionality.
If B increases, then the square is going to increase in area as B squared--
Ah, so you would like that to B squared
Right
Whereas it's just a single B and a single H. OK, so you don't like that because the proportionality is wrong.
It doesn't scale correctly.
So let me put here, bad scaling.
What was another problem?
It also has the wrong units
It has bogus units.
OK, so it has bogus dimensions
[INAUDIBLE] dimensions
No, they're pretty much interchangeable.
I use units to mean meters or feet or something, and dimensions to be things like length.
But they're pretty much interchangeable in usage.
So bad scaling, bad dimensions.
OK, anyone have any comments about B?
Same
Same problem.
In fact, it's even more horrendous.
Well, it starts out OK, I mean, at least as far as dimensions are concerned.
But then what does it do?
This has the wrong dimensions.
That has the right dimensions, and then you add them.
That's terrible.
OK, so the dimensions are just bogus.
Ah, OK. Can anyone say anything about choice C?
Yeah?
If you look at the extremes, like if you make H very small--
So if you stop on this pyramid and step on it until H goes to 0, what happens to the volume here?
It goes to infinity.
Yeah, it's terrible.
So the extreme cases are just bogus.
So it fails that test.
OK, now we're back to D. So who likes D?
So we have a few people who like D. [STUDENT CALLS OUT] [LAUGHTER]
OK, go ahead.
You don't like
No, I don't like
OK, why not?
[LAUGHTER]
It's the volume of the box, which has a size [INAUDIBLE]
So this is the volume of a box, in other words, if I had vertical sides.
So let me draw vertical sides here.
Right, so whereas the pyramid has actually got to be less.
Now, it's kind of mysterious what this factor here should be, but it's definitely less than 1.
So this is comparison with a box.
So it has a good properties, and it's a good answer.
It's just not correct.
But that's OK, because it's important to realize what's good about it and what's not good about it.
So it satisfies the extreme cases.
It satisfies dimensions.
It scales correctly.
Like you pointed out, you want B squared, because it's the area down there.
So it's good.
It's not all good.
Let's call it good mostly.
But as they say, a miss is as good as a mile except in horseshoes and atom bombs.
So good mostly, but box comparison.
OK, so let me cross this guy out and this guy out and that guy out.
So what do you conclude from that?
They're all wrong.
So here's another multiple choice question.
[LAUGHTER]
OK, so who votes for A?
One of these is definitely correct.
[LAUGHTER]
No votes for A?
Who votes for B?
Oh, I'm hurt.
I'm hurt.
Well, B is in fact correct.
[LAUGHTER]
So every once in a while, I throw the students a curve ball and make all the choices wrong.
Now, why would I do that, such an evil, dastardly thing?
Well, it's good every once in a while to do that.
In fact, I found that out experimentally, because the first time I used this question with the students, I was already using the feedback sheets.
And on the feedback sheets, I got back tons of love on the comment sheets.
The students said, oh, we love that, having all the answers wrong.
Don't do it too often, but do it every once in a while, maybe once a month or so.
And that way, they never know when it's coming, so they always have to check all the answers.
They can't just do everything by elimination.
So it is a very useful thing to do.
There are other reasons for it too, and if you're curious, ask me in the questions, and I'll explain more reasons to do it.
But that's already a sufficient reason.
OK, so that's method three.
And you see you get a bit of discussion.
You get multiple choices.
And what does that allow you to do right away?
Well, what was one of the problems?
Is, you can find out what misconception students have early.
Point one up there, in a tutorial, you can ask students questions.
Well, why not ask some questions in a big lecture too?
So that's what we just did.
So you can find out.
If 90% of the students get it like that, you know you don't have to say much more about the topic.
But if the answers are distributed at the monkey line, 25% for all the answers, then you know there's going to be trouble.
And it's worth actually expanding on this point and the theme that question gets across right then in lecture.
So you could make it much more tailored to the group of students you have.
And the students also-- what happens in their small groups?
Right, so remember we mentioned that in a tutorial, that it's a safe setting for raising questions, whereas a big group might not be.
Well, what have we done by letting everyone work in groups of two or three?
We've created a whole bunch of safe settings for questions.
So the students are free to discuss in their groups of two or three.
And if they can't resolve it there, well then they feel pretty confident that it's worthwhile sharing it with the whole class.
Maybe the whole class can learn something from that as well.
And that has another benefit, which is that it gets around the gender imbalance.
So to illustrate this problem, I was once in a bookstore in New York.
And I was standing in the computer section and this couple walked up the stairs.
And the woman asked the man, oh, there's some big fat book on Java.
And she said, what's Java?
And within a millisecond, not batting an eyelid, he said, oh, it's the fastest search engine on the internet, which is completely rubbish.
[LAUGHTER]
It's a programming language.
So it has absolutely nothing to do with a search engine.
I'm sure no search engine is even written in Java.
So it has nothing to do with search engines.
So it's a complete lie.
But he said it so fast.
So there's a dichotomy there, which is-- generally speaking, it's called actually male answer syndrome.
And the women-- [LAUGHTER]
I see all the women laughing, or I hear all the women laughing.
Well, what you'll find is generally, men are very willing to talk about stuff they know nothing about, speaking crudely in extreme caricatures, whereas women are reluctant to talk about stuff they even know about.
So if you just ask a question to a group just flat out, even with the five-second wait, you're more likely to get responses from the guys than from the girls.
So the way to try to mitigate that imbalance is to give people, first, a safe setting for asking questions.
And then people actually calibrate to the correct level about, well, is this a reasonable idea?
Is this not a reasonable idea?
Could the people around me have good ideas about it?
Is it resolved now?
If Not, well, it's probably safe to raise in a bigger setting.
So you'll actually fix the gender imbalance to a large extent, and balance it out.
So it has lots of benefits that you get from tutorial teaching.
So now I'll show you the last one, which we won't actually do for half an hour.
But I want to sketch out how you could do a question for half an hour.
I'll show you the kind of question that works in that time frame.
So this could be multiple choice.
Doesn't have to be, but some kind of conceptual question with lots of discussion.
And often, it may have a demonstration.
So let me show you that particular question.
So I'm going to use-- I have these guys too, but I also have the wood blocks which I like so much.
So here are two wood blocks cut for me by the machine shop of the Cambridge physics department when I taught there.
So they cut them out of the same piece of pine, same grain, direction, everything.
The only difference is this is 2 centimeters thick, and that's 1 centimeter thick.
But They're both a foot long and 5 centimeters wide.
So what I'm going to do is I'm going to tap this block, and you'll hear a note.
[TAPS A NOTE ON WOOD BLOCK]
Did everyone hear the note?
Great.
OK, so now the question is, when I tap this block, is the note going to be higher, the same frequency, or lower?
OK, so does everyone understand the question?
OK, so discuss with your neighbors.
We'll take a vote in about a couple minutes.
I won't give you the 10 minutes I would normally give people to discuss and argue.
We'll take a couple reasons, then we'll do the experiment.
OK, time to vote.
So, just to sketch out how this could actually turn into a 30-minute question, normally I give students about a couple minutes to think on their own, then take a vote, and then set everyone to argue with each other.
I say, in particular, find someone who has a different answer than you, and try to convince them of your answer.
So then you get a big argument going.
And so then I let that go for about 10 minutes.
And then we take another vote.
And then we discuss.
So let's just imagine we've skipped forward to the end of the group discussion, and we're going to take the second vote.
So who thinks the thick block will have a higher frequency than the thin block?
About maybe 20 people.
Who thinks they'll be equal?
About 25. Who thinks the thick block will have a lower frequency?
About 50.
OK, so we have a good spread of opinion there.
So now, again, in the scenario of the 30-minute question, I would ask for reasons, just like I did here, and I would just be the scribe.
I would just write down people's reasons for A, B, or C, for greater, equal, or less than.
Just make sure I understand the reason and then write it down.
I don't have to agree with the reason.
Just make sure I understand it, and the person who says it says, yeah, that's what I'm trying to say.
Write it on the board.
Then give people the opportunity to say a reason for any of the others or against any of them, or comment on any reason already on the board.
And that can go for another 10 minutes or so, sometimes even longer.
And then I ask people, OK, the test of science is experiment.
Should we do the experiment?
Should we do the experiment?
Yes
OK.
So just to remind you of the earlier tone.
[TAPS A NOTE ON WOOD BLOCK] [TAPS A HIGHER NOTE ON WOOD BLOCK] [LAUGHTER]
So the ayes have it.
OK, so it turns out to be roughly almost a factor of two higher in frequency, almost a full octave.
It's not quite.
If they were infinitely thin, and this were twice infinitely thin, it would be exactly a factor of two.
But because they have a finite thickness, it's not quite true.
But that's how xylophones work.
In fact, many people have reasoned their way to the correct answer by thinking about xylophones.
And many people have reasoned their way to the incorrect answer by thinking about guitar strings, which brings up a very interesting discussion you can have.
So what did you notice about here?
When I was about to tap the second one, could you hear a pin drop?
[LAUGHTER]
Right.
Student engagement is much higher, because people are involved.
But the act of voting commits people to a public view.
And they want to know, is this right or is this not right?
And the act of voting has another benefit, which is that you get a much better error signal.
Whereas, suppose you didn't vote.
And suppose you thought, well, it could be higher because it's stiffer.
On the other hand, it's bigger wavelengths, so it's lower.
And you sort of both reasons in your head.
You're in a superposition state.
And then the experiment happens-- bing bing bing-- and it's higher, and you say, oh yeah, I knew that.
It was higher because it's thicker and stiffer.
So you think actually you understood everything, but you didn't actually figure out, well, which of these two reasons is right and why?
Whereas if you have to publicly commit to one answer, then you get a clearly defined signal back about whether that answer was right or not.
So it's very important to make sure people vote.
And at the same time, it's very important to make sure people are not punished for voting.
So does it matter in my classes whether people get the vote right or not?
I just want them to vote.
And so then I have to give my speech about, it's better to be wrong than right.
Question?
So I've been in lots of classes where the professor asks a question and waits a few seconds for people to raise their hands.
And by the time he's done or she's done, there's a third of the class raising their hand, and those the people who had their hands up first.
So how do you actually take that step?
Sorry, could you tell me what they're doing?
They ask a question
They ask a question--
Do they let the whole class discuss it?
Not necessarily.
So that's what you're saying is key here?
Yeah, it's really key to let the whole class discuss-- so the normal structure for a long question like this one I'll do is I'll make sure everyone understands the question.
And then I'll let people think by themselves for a minute or two, and take a vote after that, a straw poll, I call it, so people don't feel it's too weighty.
The reason for that is that people have different learning styles.
And some people want to think longer for themself, and some people want to just start talking right away, so I want to give something for everybody.
And then after I take that vote, that gives me an idea where the class is, and it gives the class an idea of where they are, and who has different views.
And then they can start arguing with each other.
And so now they've had a lot of time to actually check what they all think.
So people will have had arguments in their group that they can't resolve, so they're very eager to actually share the argument.
Whereas if you just ask the question without giving people time to struggle with it, you're basically wasting the effect of the question
So how do you decide what level of difficulty for discussion?
This is a hard question.
But it's so rich.
It has so many things.
I don't expect most people to get the question right.
In fact, when I ask this to faculty in physics department, when I give physics teaching seminars, most people get it wrong.
I won't say all the places where that's been true, but it's been at many universities around the world.
So I don't expect people to get it right, and I don't care.
The reason for it is that it introduces so many ideas-- about, for example, spring models of solids.
We can use musical instruments as analogies-- that I think it's worth spending even a whole class on it.
And in fact, one time somebody said, well, but a xylophone, all the things are the same thickness and they're just longer.
How does that work?
And I said, oh, I'm glad you asked that.
And I pulled out my daughter's xylophone-- she has a baby xylophone-- from my backpack.
And so we analyzed the xylophone right there.
So it leads to so many interesting things that I'm willing to spend a whole class on it.
So that's how I decide how much time to allocate to something.
If it's a really core idea, spend more time on it.
And if it's peripheral, tend to spend less time on it or do a really short question on it, if at all.
And then the difficulty-- I like to get questions, ideally-- I mean, this one's different just because it's so rich-- but ideally where students are sort of at 60% correct thinking by themself, maybe 65%, even 70%.
Because even though they may be right, that sounds like oh, they're mostly fine, but they may not know why they're right.
So then in the discussion they can solidify their reasons and maybe get up to 80% or 90%.
Whereas if you've got a question where they're 30% correct to start with, and that doesn't have other redeeming qualities like the richness of it, you'll just propagate noise throughout the system, and just basically wrong ideas will just spread all over and you'll spend all your time correcting those.
So I gauge the difficulty based on trying to hit that mark.
OK, so hopefully you've seen, first of all, that tutorial teaching-- the main points from today-- is much more effective than regular teaching as normally practiced because of these characteristics-- generally questioning, feedback, instant feedback and corrective feedback right away.
But through a series of methods, you can bring many of the benefits of tutorial teaching to your own large group teaching, which is the kind of teaching that most of us will do, just for cost reasons.
So I wish you best of luck in all your teaching
Can I just [?
get ?]
one interesting statement?
Sure
It's something to think about, and faculty often have difficulty with this.
There's probably a pretty high correlation between the amount of information that you want to get through to students, and the time that it takes to do it.
The more questions you ask, the less you can get through the material.
And this constant questioning--
This constant battle
What should I do?
Should I just try to get this much information because I want to ask lots of questions?
Or do I want to get that much information through, then I better not spend too much time asking questions?
It's a conundrum.
I think asking questions is just a phenomenal way of learning, but [INAUDIBLE]
Yeah, basically we teach too much stuff.
And that's the fundamental problem.
And so one solution to that is to say, look, all the details, get it from a book.
And in class, we're going to do what you can't easily do from the book, which is struggle, argue, reason, and correct those reasons.
That's hard to do from a book.
So you should use the class time for something that's value added that you can't do elsewhere.
So good luck with that.
Please take a minute to fill out the feedback sheets.
And as long as no one kicks us out, I'll be here as long as people have questions.
[?
Answers ?]
from lecture seven to questions generated in lecture six
OK, so questions from before.
What are your suggestions for someone teaching a course that they are not expert in at all, you know, for the first time?
So I'm a practical man.
I know most people, when they're teaching a course, teaching is not going to be their life.
I mean, it is for me, but that's unusual.
And if you love teaching, that's great.
But probably, your job will be dependent on how much research you do or other activities like department committees.
And so you don't want to destroy your whole career on your first course.
By which I mean, just try to survive your first course.
Because what you could do the first time you're teaching it-- so the answer is, you need to somehow become an expert.
If you want to teach it differently and creatively and interestingly, you somehow need to become an expert.
Now, you could do that by spending the whole summer before really figuring out the material from scratch.
And that probably in the long term is a good thing.
You'd probably find useful results in your research from it.
But probably you don't have that summer to do that.
And so what you need to do is kill two birds with one stone and become an expert by teaching it, in which case the best plan I would recommend is just to teach it pretty much the regular way, with some interactive changes in lecture.
But just follow the regular plan, as much as it sticks in one's craw.
Because in doing that, first of all, you will be rocking the boat.
So people will accept it even if it's not great.
And you'll become an expert by the end.
And then the next time you teach it, you'll be OK. Then you you've done that summer of thinking, but you've done it during the class.
So it meant the first class didn't go as well as it could.
But it also meant you survived and you could order material for your lab and all the other things that you'd do in the summer.
How do you-- so I like it when you emphasize that it's OK to get the wrong answer.
It's like with the wood blocks, I mentioned how, mostly when I ask this of physics professors too, people get it mostly wrong.
And so I had another example of that in Colorado, because I gave a talk using the wood blocks.
So how can you incorporate that "it's OK to be wrong, as long as you learn" attitude into class?
One of the best ways is to minimize the amount of grading, and the emphasis given to grading.
Grading, basically, as a first approximation-- so if I just add two letters, it's hard to avoid that.
So grading, when you mark someone and you tell them, bad, A B, C, D, well, the people who are getting B's, C's, and D's, whatever the quote bad grades are, you are actually shaming people.
And in some ways, you are degrading their psyche.
So it's not healthy.
So that is one of the political barriers to educational change is the importance that we give to grading.
And I will talk about that in the final session.
You may think, oh what about the people who get A's?
Well, if you're giving A, B, C's, and D's, the people who get A's.
They may have struggled and were thinking, oh my god, will I get an A?
Will I get an A?
Then they get an A.
So they go, phew.
But still, they were nervous about their learning the whole time instead of enjoying it as much as they could.
And second, there's also studies, many studies, which show that when you reward people for things they already enjoy-- this is totally fascinating-- so when you reward people for things they already enjoy doing, their enjoyment drops.
So if you start paying people for things that they would like doing anyway, then they don't like it as much.
And you [INAUDIBLE] everyday examples of this.
You know if you know anyone who's a symphony orchestra player, they say, oh yeah, the best way to start hating music is to be a professional musician.
They love music, and then they had to do it for pay, in all of a sudden they don't like as much anymore, not all of a sudden, but slowly.
So there are many, many studies about that-- motivation.
So what you want to do really is to find intrinsic motivation.
And so the difference between extrinsic and intrinsic motivation is fundamental.
And what you want to try to do is create the environment always where the motivation is intrinsic, so that the questions are interesting.
So what that means is you don't want to use bad questions that people don't have an interest in, and then force them to be interested because you give them extrinsic motivation.
That's a fundamental bad choice, and then it's being solved with the fundamental wrong approach.
What do you need to do is rather than do both of those, you say, OK, let me find interesting questions that people care about.
And then the intrinsic motivation carries people through, whether they're right or wrong.
So that's why I like questions like the wood block so much, because people are just actually interested.
They want to know.
They say, oh, that's a good question.
I should know that.
Oh, maybe I don't.
OK, well maybe I'll learn something.
So the other part of that is, if you notice, I didn't use clickers when we did this question.
Partly, it's because clickers are expensive and you have a bunch of logistics to do, and then you need probably some ludicrous Windows program to receive the signals, and I haven't written the Linux version for it.
So there's all of those issues.
Those are, I would say, not the fundamental issue.
The fundamental issue with clickers, why I don't like them, is that they make people think they're being spied on.
So even especially if you live in a surveillance society, where actually all your phone calls are being spied on, people might think, well when you have clickers, you're being spied on to know whether you got the right answer or not, and it's going to count against you at some point.
So I'm very happy with people to just raise their hands.
And that's, in a way, more anonymous than a clicker that belongs to you.
So there are ways of getting around that with clickers.
You can to have people trade clickers so they know that the clickers are random.
So you can do things like that.
But still, I like the informal feel of, yeah, just raise your hand.
Don't worry.
It's a safe space in the classroom.
And I have to say that a few times.
But just raise your hand, and don't worry.
And I'm not spying on you.
I don't care whether you get the right answer or not.
For example, with the wood blocks, I expect most people to get the wrong answer.
That's OK. We'll learn something about it afterwards.
So you do have to work at it, but there are things you can do to keep the class [?
safe.
?]
There are also things to do, which is that never-- it's hard to remember how much power you have as the instructor, and just as someone in a position of authority.
So I might have told you the story from when I was a graduate student, but it's worth telling again, if I haven't, which is that I was giving a talk in a group meeting when I was a graduate student.
My adviser got frustrated with me because I wasn't understanding some fundamental point, actually about springs.
So to demonstrate it, he kicked the table at me and said, is this acting like a spring?
And I was so shocked.
I was just mortified for like two days.
I just couldn't believe my adviser had done that in front of everybody else, and thought that I must be idiot and showed it.
So then eventually my fellow grad students said, well, actually he does that only when you really cares.
So that's better than people just switching off.
So I sort of consoled myself with that.
But it did make a real strong impression on me, how much even a small push from someone in authority can do.
So it's the same thing in lecture, if when someone says something wrong, if you say, oh, why would you have said that?
Even the slightest hint of that, people magnify and will pick up and be very, very sensitive to.
So you have to be really, really, really careful not to let that happen.
So one way is the poker face.
So I practiced the poker face.
So when people are making suggestions, for example, about the wood block, why they would act one way or another, I just make sure I get in a different mental state.
The mental state is transcription.
So I just want to make sure I understand what people are saying.
And then I just write that down.
Now, it's even there you have to be careful.
Because you'll understand the correct answer quicker than you will the incorrect answer.
So you have to make sure when people are explaining the correct answer, you don't complete it for them.
And then in contrast, when people are explaining a wrong answer, you ask lots of questions, because that too is a signal.
So the poker face is tricky to do, but once you master it, that's one good way of giving yourself a pause and not criticizing people for their answers.
So there are various ways, but you just have to be careful.
And if you do that regularly, you'll find in a week or two, you've overcome the presumption people have that it's dangerous to ask questions.
But you do have to work at it.
Is it possible to incorporate mastery learning at the college level?
So actually, there is a very interesting experiment on that.
And it was done at MIT in the 1970s, I believe.
It was called-- so it was called the Keller Plan.
I don't know if they did it for all courses, but the physics courses were taught this way.
I don't know if only that way, or that was an option.
So it was in the 1970s, which was a time of-- as people who were here then said, it was a very turbulent time.
And the turbulence actually enabled lots of educational experiments to happen, basically the '60s and '70s.
So if you look at, for example, the Feynman lectures were in Caltech in the '60s, the Berkeley physics course, 1960s, things like the Keller Plan, the free schools at the school level-- tons of experiments happened in the '60s and '70s.
So the Keller Plan was one of them.
It was mastery learning.
Students worked on problems, and when they were ready for the next piece, they continued.
And from what I can tell, it worked pretty well.
But then eventually it got killed off by basically faculty who said, no, I'd rather teach and do lectures.
Because what's the point of the faculty if the students can learn really well from the books?
So I think the Keller Plan was a really good idea.
In school, I learned for five years of high school math in three or four weeks using the Keller Plan, compared to the normal pace in school where you spend a whole year doing trigonometry.
So the Keller Plan is a really, really excellent way, if you have reasonable books.
So now, what are the full reasons it died?
I'm not really sure.
I need to check into that.
But people who were around then, some of them said, yeah, just people who wanted to perform in lecture were really sad about the Keller Plan and wanted to go back to the old way.
And so when the political currents changed, so did the Keller Plan.
How do I come up with these rich and interesting examples like the wood blocks, and how often do you incorporate demos?
Well, we'll talk about that in Walter Lewin's lecture.
So if you saw Walter Lewin's lecture, the one on the pendulum, he must have had four or five different demonstrations.
They were short, but he had many demonstrations.
And questions like this, if I had 20 of these, I would use 20 different ones.
So usually I have four or five of them that are a good long time demo that are suitable for, say, half a lecture or longer.
And I use them.
So wherever I can find them, I use them.
But not every single demo is worth spending 30 to 40 minutes on.
So you have to think about whether it teaches core ideas, how it fits with the goals of your course.
So for example, let me talk through why I use this one.
Well, one of the goals that I want is I want students to be able to make physical models themself.
I don't want them to just be able to solve equations that I've put in front of them.
So normally what you do is you'd teach them the bending beam equation, and then maybe you'd show them this, and then you'd do this in two or three minutes.
But actually a much more interesting question is not, what's the solution to the bending beam equation when the beam's twice as thick.
The question is, what physical model is responsible for making sound?
When you hear sound, where is it coming from?
When you tap this guy-- [TAPS A NOTE ON WOOD BLOCK]-- why is it making sound, long before you're even discussing the bending beam equation.
So that question is, to my mind, much more interesting.
And so I'm very willing to spend more time on that, and having students develop that feel, and have them argue and realize that there's different physical models.
It could be a standing wave in the block.
It could be that the block is bending.
How do you decide, and how do you decide which one you're going to hear more?
And then for each one, how do you figure out how the frequency's going to change?
So that, because it fits so well with the course goals, spend more time on it.
Why is the stiffness a factor of eight bigger?
So that one, anyone who wants to know, I will explain after lecture.
Just come to office hours.
So there's a simple picture using wood atoms.
Don't ask what wood atoms are.
How can you incorporate aspects of mastery learning into regular teaching?
Ah, so that's what interactive teaching is all about.
So the reason mastery learning works so well is that you're not building castles of knowledge on sand, roughly speaking.
So normally what happens-- so the extreme case is Cambridge in England.
So the teaching system there is you have an eight-week term, and then you have a six-week break, and then an eight-week term, six-week break, eight-week term, and then 16-week break.
So the eight-week term, about what we would do in a semester is compressed into eight weeks, just basically straight dictation and lecture.
And the students have a hard time really understanding everything at that pace, so they just take dictation.
And then you're supposed to figure everything else out, integrate it in the six-week break.
So the problem with that is that it's very inefficient.
It's very inefficient because you haven't understood A.
And now B uses A, but A hasn't really been understood.
So you're not going to understand B.
So now you have A and B not really understood, so you just write them down.
Now C comes along using A and B, but A and B aren't understood, so the problem compounds.
Whereas with mastery learning, only when you understand A do you go onto B.
And also when A and B are solid, then you go onto C. So your knowledge castles are robust and strong.
And that's not true with most teaching.
So how do you remedy that problem?
Well one way is you ask questions in lecture.
So here, plan interactivity and build it into your lecture.
So by asking students questions you figure out, oh, if only 20% or 30% or maybe even 40% of the students knew the correct answer, and even after discussion, you think, oh, OK, well this is a fundamental concept.
Presumably that's why you asked the question about it.
But it wasn't mastered.
So better fix it, before you go on.
So that builds mastery learning into the regular lecturing process.
And that's one reason that interactive teaching is so effective.
So one of the other questions later in the sheets is, is there evidence that interactive teaching is good and helps people learn?
There is actually good evidence.
Maybe I should put some of those papers online.
One is by Richard Hake.
He's now retired.
He's physics professor from Indiana University.
So what he did is he did a sample of physics courses across the country, intro physics courses that included a total of 6,000 students.
And he asked the instructors to describe how they did everything.
And then everything was grouped into either interactive teaching or regular teaching.
And just the scores that students got on one particular exam, which was a force concept inventory, before and after-- they were given the force concept inventory before the class started and at the end-- they saw how much the students improved.
OK, so if you improved 100%-- in other words, suppose you started at 20% correct and you got to 100%-- that was considered a 1.0 gain.
If you went from 20% percent to 60%, that mean you got halfway to the maximum possible score you could have gotten, so half of the maximum improvement, that was a gain of 0.5.
So he measured the gain for everybody.
So this is the gain on the force concept inventory.
And this is the number of courses in each area.
So it was roughly like that.
So this was the regular.
So the regular teaching, by the way, included fantastic teachers, people who were rated highest lecturer in the university and had won lecturing awards.
And the interactive teaching was something like this.
I'll put the paper online so you can see the exact graph.
But basically, the worst interactive teaching was better than most of the regular teaching.
And the regular teaching hardly got better than 0.2 or 0.3.
So if you wanted any more gain than that, you really needed to do interactive methods.
And I think that is because you've incorporated mastery learning into the curriculum.
OK, what do you do when the students are really quite and won't answer your questions?
Or flip side is, what if only the quote 5% usual suspects are the ones answering the questions?
What do you do?
Well, so that is related to the question of, are there bad ways of incorporating interactivity?
And there are.
The worst is the following, which is-- I'll sort of parody it.
So we're doing relativity, or nuclear physics.
E equals MC-- [LAUGHTER]
Squared, yeah.
OK.
So you'll find a lot of people who normally wouldn't teach interactively are now, they're told or they hear that it's good to teach interactively and ask students questions.
But if you don't have any practice in how to do that, the easiest questions to ask are just things like this.
And the problem with that is not that they're asking a question.
It's good to ask a question.
But the problem is that this thing, this two, is just a memorized fact.
So it's possible to answer it just by memory, whereas if there was actually some debate about what it should be, you could ask the students the question.
But now, if you just ask people question like that, and you give people no time to think, yes, you will get answers just from the quote top 5%, the usual suspects.
So to avoid that, what you need to do is give people a chance to think on their own.
So that's one of the interactive teaching techniques, the idea of give people a minute or even 30 seconds to think on their own, so that you'll get answers from a wider spectrum of people, because they've checked it out with their neighbor.
And they feel safer in volunteering and answer or asking a question about it.
So that's how you avoid two things, two problems at once.
You avoid the quiet students, because you've given-- those students are being quiet because you've given people a chance to think.
And you also avoid the problem of just the usual people answering, because you've given everybody a chance to think.
OK, if we don't really understand until we teach, should we have students teach each other in presentations, or is that not worth the time?
My feeling is that huge big presentations and having students present to each other is not so much worth the time.
I mean, it's worth the time of the one person who's doing the presentation.
But that's the main person who learns in that teaching session.
So there are several graduate seminars, many I took as a graduate student, that are run the following way.
The professor just sits back and each time-- they assign a list of topics, and each student volunteers for one topic to present to everybody else.
Now, that is a fantastic way to learn one topic in the entire course.
And it does work for that.
And part of my Ph.D.
Came out of one of the topics that I learned from one of those courses, because I really wanted to figure it out.
So it can work.
And it does work in some ways.
But it's not a very efficient way, because all the other people are just basically passive.
Yeah, they read the stuff.
But now they're being taught by someone who's just barely mastering the field and is not likely to teach interactively with a large margin of safety.
So it's much better to have students teach each other by having them argue with each other on small time scale.
So for example, if you remember in the wood blocks, I said, OK, are you with your neighbor about your answer?
Well, now the students are trying to teach each other.
Some students think, oh yeah, it's due to the bending.
OK, well that turns out to be correct.
But can you convince your neighbor?
Well, in trying to convince your neighbor, you are doing teaching.
You're trying to figure out, what's a way that you can do it without writing down 50 equations.
That's exactly what you want to do in teaching.
So I think it's a much better way, is to build the interactivity into the class for everybody, rather than having it as a rotating seminar where just one person at a time learns.
In organizing big ideas into a tree, there are often multiple ways-- for example, you can do current understanding versus history, or experiment versus theory-- and it's often good for students to know more than one of these organizations.
How do you do that in a course design?
Yeah, so unfortunately there's only one time dimension.
There's three space dimensions, but only one time dimension.
So the course has to evolve in one dimension somehow.
So you need at least one organization.
But you're right.
You need other organizations as well.
So the way I square that circle is as follows.
So let's say time goes that way.
So this is unit one.
Let's say it's one technique, one mode of reasoning, dimensional analysis.
Let's say this is another mode of reasoning, conservation, and so on.
So these are units.
And now you have topics.
Well, what I like to do is I want to show other organizations.
And why is that good?
Well, what you're doing is you're connecting.
You're building threads across.
And you're right.
You definitely want to do that.
So what I'll do is I'll find problems that can be solved multiple ways.
Suppose this is, for example, a problem about a pendulum.
Well, you can solve a pendulum with dimensional analysis.
You can also solve a pendulum with conservation laws.
So I'll do it again.
So this way, this method is sufficient to solve it mostly.
This method is sufficient, but now we're revisiting the old problem.
So I try to find problems that thread through, and maybe then reappear over here.
So that gives you a cross cutting organization.
So you can do that not just with problems, but with, for example, other principles.
For example, the history-- suppose you want to incorporate history versus current understanding.
Well, every time you mention history and the importance of history, you say, oh look, there's yet another example where we learned from history.
We learned it here.
We've learned it in this unit.
So now the students have yet another structure to bind the course together.
This is hard to do the first time you teach the course.
What you'll do is this first time you teach your course, well, you'll probably do it the regular way.
The second time, you'll maybe reorganize it this way.
And the third time you do it, you start building all the bridges across it.
Because that's the only time you actually see them too.
What are my thoughts on grade inflation?
I wish every grade was an A.
So I think there isn't enough of it, because of the studies I said before.
So I'll talk about that in the penultimate lecture.
I think most of these, yeah, I think I've now answered with other comments or questions that are there, except for one, which I'll save for next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Welcome back, everyone, and me too.
I just came back from Boulder where I gave eight talks.
I was on seven panels and gave one separate talk, most on education.
And one of the talks was on teaching how to think or what to think.
So there were three panelists, one professor from Berkeley in the astronomy department, who is also a renowned teacher, and then an investigative journalist, and then I spoke.
And we all said something for about 10 minutes.
I talked about the history of education and why it's difficult to change educational institutions and educational practices and what the original purposes of the schools were, so some of this material that I'll talk about in the second to last lecture on political barriers to educational change.
And I then also, before this whole session, I described to the reporter for the Daily Camera, which is the paper in Boulder, a bit about what I was going to say.
And I talked about some of the material I've told you already about multiplication table.
So if you remember, the multiplication table, when people get brain damage in the arithmetic area, they can't do addition anymore, but they can still do the multiplication table.
And when people get brain damage in the language area, they can still do addition, but they can't do multiplication anymore.
And that's because multiplication, generally, is learned as-- So when you see it, you think of symbols and math, but actually the way most people learn it is 6 times 9 is 54.
Then you have to memorize yet another fact.
9 times 6 is 54.
Oh.
And then you have to learn yet a new fact, that these two facts are related, when, actually, if you understood the meaning, there would be just one fact.
So now you have to memorize all these linguistic sentences.
And no surprise, when you get damage in the linguistic area, that kind of multiplication table goes away.
So I said to the reporter, yeah, what we really need to do is teach multiplication and math in a whole different way.
So the reporter, she understood everything and said, oh, that's really interesting.
I'm actually studying education myself, and I have children.
And I'll try to apply that with my children and in the teaching that I do.
I said, oh, great.
So then, I gave my talk, and then the moderator asked everybody, well, does anyone on the panel want to say anything to anybody else?
And so Alex Filippenko from Berkeley, he stood up and he said, well, I agreed with everything that Sanjoy said, except he was quoted in the paper this morning as saying we shouldn't teach the multiplication table.
So that whole discussion, which seemed to be completely understood, just got condensed into one sentence.
So now I didn't actually have to say very much in response, because the second panelist, as I mentioned, was an investigative journalist.
And what she was talking about is how you have to be very skeptical of what's in the media.
So all I had to do was look at her for a bit, and then everybody started laughing because the point was clear.
But then I explained to everybody else what I'd explain to the reporter.
That's by way of saying that education is a complicated subject, but the audience understands a lot more than the media does, which is one of the barriers to political change is that there's people who understand lots of individual points, but how do you assemble it into a coherent plan for change?
Well, you need forums for discussion.
And if the reports in the media are, oh, don't teach the multiplication table, or do teach it but do it with lots of drill, well, it's hard to get in a discussion of, well, actually you'd like to teach it in a different way, where instead of memorizing this you say, well, 6 times 9 is like 6 times 10, which is 60, minus 6, which is 54.
So there's no space for putting things like that into public discussion.
And one of the purposes of classes like this is to do that.
OK. Today, we're going to talk about lecture planning and performing.
What we've done so far is the flow of the ideas in this class has gone as follows.
So this is a logical flow.
We did it slightly out of order, because it's the preparation flow, slightly different than the logical flow.
You work out your course goals, or even your curriculum goals.
And then you decide, how are you going to tell whether you've reached those goals?
Let me just write the goal a little clearer.
And with problems, tests, that's the way you operationalize your goals.
And then given that you have those kind of problems, well, now you have a pretty concrete idea of what it is you're trying for, because you have concrete examples of it.
Well, lectures, recitations, so I'll put et cetera here.
What do you do with your contact time?
This is the overall structure.
This is at the level where it makes concrete sense.
And this is, what do you do when the rubber hits the road, what do you do in lecture?
Last time, what we did is we talked about interactive teaching.
And there was, basically, several time scales over which you can make the class interactive.
The longest time scale was the wood blocked example, and I'll answer a question about that in just a moment.
So now, continuing the theme of where the rubber hits the road, today-- Now that you have some techniques, how do you assemble it into a lecture?
And I use both words advisedly and deliberately.
So plan, it's essential to plan.
And performance is essential.
People forget that.
They forget that lecturing and teaching is a public act, and when they forget that, the lectures are flat.
We're going to talk about the techniques for doing that, starting with the Walter Lewin lecture that you watched for today.
Because he actually has many, many examples, I think mostly good, of performance, and some that I think he could improve on.
And we'll discuss that and work out and induce general principles that you can use.
OK. Those are the questions from last time.
There was a slight review of interactive teaching as well and things like this.
Now the question is, what do you do in lecture?
How do you plan, and how do you perform?
And a good way to start discussing that was actually to look at a master of the art and see what they do.
Now no master is perfect.
But I think Walter Lewin did so many things well it's really worth understanding what he did and looking at it.
So what did you notice from your homework problem?
Basically, what did you notice that he did?
And I have a bunch of things as well.
And we'll just write them down.
Because what we'll do, as before-- we've done this several times now-- by looking at examples, we can induce general principles.
Actually, I'll leave this here, and I'll erase these guys.
While I'm erasing, just pull out your homework and the things you've noted down.
And actually, take a minute and check in with your neighbor and see if your lists overlap.
And sort of vet your lists with each other.
And if you didn't think about anything that could be improved, see if you can remember anything that you would have done differently.
So let's share all the points.
So I can see that you've come up with a lot of examples, and I actually had a bunch too, because it's such a rich lecture.
So let's see what things that were done and what was either that you would keep or you would change.
And I'll put them in one or the other side, according to how you categorize them.
Any one example?
Yeah, Wendy
He awards students when something might not be intuitive for them or something.
So when he was doing analysis of error under the square root, and he was saying, OK, this might not be something you would know right now.
I might not hold you responsible, but it's interesting [INAUDIBLE]
OK.
So he tell students, look, this is harder or not intuitive.
He gives them a gauge and tells them, look, you're don't have to worry about it before he says something.
He gives the difficulty of something.
OK. Why is that helpful?
What does that do for students?
Yeah
It sort of removes the tension of thinking, oh, man, I don't know this while they're trying to pay attention to the other [INAUDIBLE]
Right.
Right.
So it differentiates the lecture.
So I'll put in other color the reasons.
Yeah, so it gives some kind of differentiation to the lecture.
It gives a structure to it.
It gives you a way of evaluating so you know what are you supposed to be doing here and trying to learn from it.
You don't all of a sudden trigger your reflex fear.
So it minimizes fear.
And it gives a clear expectation.
So in the particular lecture, he didn't say anything was you absolutely have to know this.
So there wasn't a flip example.
But if it had been a flip example, it would have the same benefit.
If he said, oh, this is really fundamental.
You really, really, really, really need to know this, well, that also punctuate the lecture.
So it minimize fear and sets expectations.
To expand on this point, which is really fundamental, think about the difference between the way a novice perceives something and the way an expert perceives something.
If you want to know how a novice perceives something, try to remember from the days when you were not a chess Grandmaster.
That shouldn't be too hard for most of us, certainly not hard for me.
You look at a chessboard, all the moves look kind of similar.
Oh, I could move that pawn.
You think about that for a bit.
You analyze some moves.
He takes, I take, he takes, I take.
Hmm, OK, let me think about-- I could move the rook.
I could castle.
He takes, I take, or I'll move the bishop.
And you sort of say, OK, I'll move the bishop.
That's how the novice or medium chess player thinks, speaking from personal experience.
The way the master or the Grandmaster looks at the chessboard, the chessboard has meaning.
All the different pieces of it are kind of talking to each other.
Hmm, that pawn structures is really full of holes.
You really want to do something about that.
So the undifferentiated mass of pieces that the notice sees is completely different from what the expert sees, which is why the expert can remember the chessboard and the novice can't.
I noticed this exact same thing happened in my freshman year.
I learned to play bridge I'd never played bridge before, and all my friends, who I don't know if they did me a favor or not, taught me to play bridge so I could play bridge with them.
When I first was playing with them, at the end of the hand, everyone does the post mortem.
You have four people, you play all the tricks, and everyone talks about what happened and it didn't happen.
And they would say, Oh, yeah, why didn't you play the queen then?
I was like queen of what?
What queen?
Did I have a queen?
I didn't even remember the cards I had.
I didn't even remember how many cards I had of each thing.
I just sort of remembered the last card, maybe.
Or more like, oh, phew, the hand's over.
OK.
I can't make anymore mistakes.
That's how I would see it compared to how they would see it.
And I was like, how could they ever even remember who had the queen of what?
And by the end of, I don't know, six months, I would say, uh, why didn't you play the five then?
Because all of a sudden, now I was doing exactly the same thing.
The hands had meaning.
The cards had meaning.
So by doing this, by giving clues to the difficulty or ease of things, you're actually giving meaning.
You're transferring some of that knowledge that you have as the expert to the student, who otherwise wouldn't know.
There was another example of that in the lecture where he said, this is the most important equation in all of physics.
He wrote that, and he boxed it, and he said, that's probably the most important equation in all of physics.
So again, he's giving something that the expert knows.
Because the expert knows, oh, yeah, when you see this, you're home free, and you know you can model so many physical systems with this.
Let's tell the students how important it is so they have some way of thinking about it.
OK So that's really fundamental.
Others?
Other points?
Yes
He uses big ideas and terms for them, like uncertainty and restoring [INAUDIBLE]
OK.
So he uses clear terms for big ideas, and he uses the same term each time.
He's careful to do that, so you don't wonder why it changed.
How does that help students, to give things names and repeatable names?
Yeah
Gives them fewer facts to [INAUDIBLE]
Yeah.
It gives a structure to everything.
It's gone now.
But it gives a structure, repeatable abstractions that you can reuse over and over again.
Structures.
In fact, it's another example of this.
It structures the knowledge.
So it's structured in his mind.
He's the expert.
He has a really good, sound flexible structure.
And part of the job of really good teaching is to help the students build that structure.
Yes
When he's characterizing visions, he really emphasizes building intuition for why the vision is the way it is and why it's [INAUDIBLE]
OK.
So he emphasizes the why of the equation.
So why is it plausible that it, for example, has-- Can you give an example of that?
He talked a bit about why [INAUDIBLE]
OK. Yeah.
There was an example where the frequency was derived, so emphasizes the physical meaning and interpretations also.
And interpretation.
So the example he gave, which I think is a good one, is in a pendulum, why is the period independent of the mass?
For a pendulum-- OK. And the argument for that was, oh, well, if you double the mass, you double the force, but you double the mass, so you haven't changed the acceleration.
So everything stays the same.
And he contrasted that.
He didn't just do this one.
He contrasted that with what happens in a spring.
OK.
So how does this help the students?
What's the benefit?
I think it helps them because it makes them feel like they could potentially derive it themselves and then afterwards check their answer
OK.
Right.
He's modeling, actually, what you would do.
You, yourself, as a physicist, and hopefully as a student, you want to be able to check your answer by making sense of it.
It promotes sense-making is a shorthand for it.
Let me just push the other board up
Well, and it's also [INAUDIBLE]
Oh, sorry
--which is probably the most difficult thing to teach.
I thought that was the best part of the lecture.
One thing is to give a list of facts so you learn and then you can use
Right
That seems to be the closest to teaching how to think, which is the most difficult thing you can ever teach
OK.
It teaches how to think, because it's a transferable skill.
You may never have to analyze a pendulum again as a student outside of the class.
But whatever formula you get, you can use that same method to say, does the formula make sense?
So that's teaching a way of thinking.
It's a transferable skill.
Yeah, Adrian
He's very, very explicit about his notation
OK.
Explicit about notation.
Can you give an example?
Well, even when [INAUDIBLE] method of a string, he says, this is the method of a string.
And he'll label an equation with the whole, writes it out.
In [INAUDIBLE] terms
Right.
He'll write everything out.
So he's very explicit about notation.
How does that help the students?
What does that do for them?
Yes
It makes it clearer and it makes mistakes hard to make
Right.
It makes it hard to make mistakes.
Again, so this is hard to make mistakes.
Harder-- OK. And what makes it harder, and why is this so important?
Because, as Adrian said, yeah, it may be intuitively obvious to the expert physicist that, of course, L is the length.
That's why you chose L, because length begins with L. But for the student, everything is new.
The chess example, the knocking down the chessboard problem and seeing how well you can reconstruct it, explains so much.
The novice, they can't reconstruct the chessboard.
That's because each piece on the chessboard is not intuitively obvious.
Whereas, for the chess master, it's so easy to reconstruct the board, because, to them, it is intuitively obvious that, of course, you have three pawns in front of your king.
Unless you have some really crazy opening, of course you've castled your kind and you have three pawns sitting in front of it protecting it.
And it's just obvious from the configuration of the pawns that, of course, with that pawn configuration you did this opening.
And of course, the king is defended in this way.
So what's obvious to the expert is completely obscure to the non-expert.
You want them to not have to worry and fill up their short-term memory slots with things that really are obvious, so you put that right on the diagram.
He wrote M, I think, and then there was an L there.
So right away, you just look at the diagram.
You know what L is.
OK. Other things that were done?
There are so many
He asked the students to help [INAUDIBLE] passively but actively
OK.
He asked for help.
That increased participation.
How did he ask for help?
What were some examples of that?
[INAUDIBLE] the gun spray toward the [INAUDIBLE] of the harmonic motion, he asked a student for help holding the end of the paper
Right
And then, when he drew the big pendulum where he--
Oh, the final pendulum.
Right.
Several times, actually-- let's see.
I think I have it here-- Lewin says, I want to hear you loud, because he didn't hear them.
He wanted people to really participate, and then they really started counting.
So they were really involved in what's going to happen by the end of 10 swings.
It involved them, so it's a form of interactive teaching.
He's getting people more involved.
Yes, Adrian
On the other hand, I thought he pretty much never asked questions
OK.
He didn't ask them questions.
Right.
Yeah, it's good that he asked for help, and that is one way of involving them.
So it made them interested, but it didn't necessarily-- It increased their motivation to learn about the subject, but it didn't necessarily actually help them learn right then about something.
So he would change.
OK. Can anyone else think the spots where he could have asked questions?
Yeah
I think there were natural pauses, natural breaks [INAUDIBLE].
For instance, when the air tracks through the pendulum, he didn't stop there.
He never stopped to let students think about what they [INAUDIBLE]
OK.
He didn't pause at the natural break points
He didn't pause at the natural--
Yeah
[INAUDIBLE]
Right.
One example that I remember of that, because I was also watching for that, was he wanted to show Hooke's law and he plotted out-- I'll draw an example here.
He plotted force verses extension for a bunch of extensions.
And forces was the weight of those springs.
And he got a bunch of points here, and he drew this.
Then he pulled one of the springs, and he stretched the hell out of it.
And then he let go of it, and it came back much longer than it started.
And then he drew what happened on the graph.
So one thing he could have done, rather than drawing it straight, is the following.
You say, OK, how do we describe what happened to that spring as I stretched it?
Was it this curve, that curve, or that curve?
A, B, C. You don't have to spend long on it, just 10 seconds, or even 30 seconds.
But instead, he just told them which it was.
So actually, which was it, A, B, or

C. Right.
It was C. And just the act of thinking about that forces you to connect the physics and the mathematics once again, even this small amount.
Yes, Lat
Wouldn't the same apply to you?
I've never thought about asking a question.
Maybe you answered it immediately.
[INAUDIBLE]
Right.
OK.
He didn't pause at natural break points, and he didn't pause at--
[INAUDIBLE] smaller time scale?
Yes.
Not the smaller time scale.
Right.
No five-second pause.
So the questions are almost rhetorical.
I would say it's one of the dangers of being such a good performer, which he is.
He's fantastic, I think.
And one of the things he does so well in his performance is connecting-- which you mentioned over here-- the physics and the mathematics, explaining the physical meaning and interpretation, that top item.
He does that so well.
But one of the dangers of being such a good performer is that you just steamroller over the students, and you become, in some ways, more important than the students.
Yeah.
You'd be able to do less in lecture, but the five-second pause is really important to have everyone tracking and following with you.
Other items?
Let's see.
Someone I haven't-- Yeah, Tsu
Sometimes, when he says, this is totally non-intuitive, he kind of just accepted that.
And I think it would have been nice if he had given a reason for why this had to be, instead of just saying--
Yeah.
OK.
When he said things were non-intuitive, he sometimes didn't explain why it was non-intuitive.
So the one I'm thinking of is he said, no matter what the amplitude is, the period is the same.
And he said it isn't intuitive.
So that was good, in that he did tell people-- Where was that?
Yeah, it's here.
He gives a difficulty.
He says, for example, this isn't intuitive.
You should be surprised that the period is independent of the amplitude.
But he doesn't give them another way of seeing why it should be.
They do the derivation, and you find that it is independent of the amplitude.
But why?
And you can explain that sort of intuitively just by saying-- I'll give you a quick explanation of that.
So it is, actually, an amazing fact.
I'll give you a quick explanation of the unintuitive fact that the period is independent of the amplitude.
OK?
This is a scaling argument.
E, G. OK. Let's estimate how long it takes a mass to move a distance, A.
Well, it has some acceleration, a, small a.
This is the amplitude, and that's the period.
So the mass has some acceleration, little a.
And in some time, t, the period, or some fraction of the period, let's say a quarter of the period, it goes this distance, give or take factors of 2 and whatnot.
So this is roughly the amplitude.
Now we want to figure out, what is this A here?
Well, A is the force over the mass.
And the typical force, that's coming from the spring.
So it's a spring constant times the extension.
The typical extension is the amplitude.
OK?
k times A is your force, divided by m, so we'll put that in there.
OK. Oh, look at that.
Hmm.
And in fact, there you have t is proportional to square root of m/k.
So you've got not only that it's independent of the amplitude, but you've got the whole story, except for the 2 pi.
So intuitively, what it's saying is that, yeah, if you move yourself farther out, you have father to go.
But you're pulled harder by just enough to make the time the same.
And so not only is it an intuitive argument, it's very plausible.
And Galileo extended it to the pendulum, and he thought he found a proof to show that the pendulum actually has a period independent of amplitude.
In the pendulum, the amplitude is related to the restoring force, so he thought the period of the pendulum was independent of amplitude, not just for a regular spring.
And that proof turned out to have a slight error in it.
But as Walter Lewin pointed out, that's only true at large angles, and at small angles, you can't tell the difference.
So you can actually explain these non-intuitive pieces in short ways that don't require doing a full calculation.
And that is important to do, I think.
OK.
So why don't we take a 10-minute break, and then we'll-- Oh, do you have a quick comment?
Go ahead
One other thing that I thought he should have done
Yeah?
Which is actually motivate why this is so important.
He mentioned [INAUDIBLE], as you said, that it's the most important thing in physics.
But actually, I thought it would have been nice to give some real examples of how [INAUDIBLE]
OK. Yeah.
Answer the "Who cares?"
question.
Right.
And that's a good point.
I teach physics myself, so actually that one slipped me.
Because this is, again, the expert blind spot.
So he put that equation there, and I was so glad he said, most important equation in physics.
And to myself, I thought, yeah, right on.
You use it in electromagnetism.
The whole theory of quantum electrodynamics depends on it.
You use it in modeling materials.
It's everywhere.
So all those thoughts went through my mind right away.
And I thought, wow, that's equivalent to him saying it.
But actually, it's not, because the students, of course, have none of those thoughts.
And you really do need to say all that.
Why?
OK. We'll continue after our 10-minute break.
And then we'll induce our lessons for lecture planning and performance, and I'll give you a structure that you can use when you plan and perform your own lectures.
OK?
There's several more points to come out.
So we'll take some more points about Lewin's lecture at-- It's 10:05 by that clock.
--10:15 by that clock.
And if you didn't get a comment sheet, just grab one on your way in or on your way out.
And I'll be here if people have any questions during the break.
So a few other points.
Let me give you one point from Lewin that I noticed and take some more points from you too.
One I thought was very interesting, and which is not normally done in most physics lectures on springs.
The way it's normally done in the physics class, you would do simple harmonic motion, you'd study a whole bunch about the spring, and then maybe a few lectures later, you might get to the pendulum.
But Lewin actually did both in the same lecture, which I thought was very interesting and a really excellent choice.
It had some negative effects, which is that there's less time to talk about things like this.
And maybe it feels like there's more pressure to not wait five seconds.
So it has maybe subtle pressures in the wrong direction, which maybe could be mitigated by planning.
But it had a really great advantage, which is that it's this figure.
So if there's some idea you want to teach, and the idea you wanted to teach is simple harmonic motion.
He called it Simple Harmonic Oscillation, SHM is the standard jargon in most of the physics books.
If you want to teach that idea, just showing it in one example is not good enough for the students, because they can't separate the example from the idea.
It's just one big morass, and they can't draw this dividing line.
So you have to give them another example.
But if you give them a second spring example, then it's still too hard.
Suppose you do this and yet another spring example.
Well, the problem is it still the overlap here is too big.
So you need an example that illustrates the core idea but is quite different.
And the pendulum falls into that pretty well.
It's different, in that-- and the order was correct too-- he first did the spring, because the spring really is pure simple harmonic motion.
The pendulum isn't, but you have to approximate to get it there.
So it shows how you get to simple harmonic motion, because you've seen it now.
And so the piece you're getting to is the common thing between the two examples.
And furthermore, it's different enough that you can see what's common and what's not.
What's common is the structure of the equation.
The structure of the equation is-- So that's common.
And so you have a really particular example here of inducing the core idea, which is if you just did simple harmonic motion with the example of a spring, students may think simple harmonic motion is this.
They think whenever you have k/m you have simple harmonic motion.
Whenever you don't have k/m, you don't have simple harmonic motion.
They may think that actually it depends on exactly what's here.
But because the second example has something completely different-- The second example, the pendulum had g/l after approximating it.
You get the same structure of the equation, but a new physical phenomena.
And it shares the common feature of simple harmonic motion.
So it's a really excellent way of bringing out this idea and borrowing most of the derivation that he did before.
For example, he didn't have to do resolve this equation with these new parameters.
Once he'd solved this equation, all he had to do is substitute this into the solution and get the new solution instantly.
OK?
Yes
He created emphasis by using both gestures and also modulating his pace and pitch
OK.
I'll try to squeeze it in here.
So he modulated his pace and pitch, and his gestures were very bold.
He talked about the spring.
He stretched the spring.
He actually almost broke the spring.
Whenever he was describing something, you could feel him feeling it in his own body.
His gestures were bold, and his pace was varied.
OK. How does that help?
Yeah
Conveys enthusiasm
Yeah.
It's exciting.
It conveys enthusiasm.
It punctuates.
There are several languages, I think, which have no vowels and-- Not no spaces.
Well, German approximately has no spaces.
That's not quite true.
But that don't have vowels or punctuation.
In the old days, for example, there was no punctuation.
People just ran all the words one after another.
You didn't put a period at the end of sentences.
Without punctuation, you have to work really hard to figure out what are the units of thought.
So by varying your pace and pitch, you convey enthusiasm.
You convey enthusiasm and structure.
So the pieces that are more important get more importance, because they're emphasized accordingly.
That's actually fundamentally important in writing.
I don't know if you know the following recipe, but this is the first-order term in good clear writing.
Let's call it not fiction writing but expository writing is the right word.
The first-order term in punctuating writing and giving structured to your writing is this.
A sentence starts with old information that links to something before.
It continues to-- This is how the sentence ends.
Whatever you put at the end of the sentence, readers expect that to be the thing that you want to emphasize.
That's the new information.
There's maybe a fair amount of new information in the sentence, but this is the new interesting information.
This is what readers assume has emphasis.
And in the beginning of a sentence, they look for things that connect to what has already happened.
It could connect to the last emphasized thing in the last sentence, the subject of the last sentence, the thing that was the topic, or the topic of the paragraph.
It somehow links to stuff before, and then you're going to say something new about it.
So if you don't use that order, it gets very confusing for readers, because they expect this to be emphasized, but you're mentally emphasizing something else, and then they have to redo their expectations each time.
So punctuation, the period actually is a way of saying, here came the most important information.
And if you follow that structure, you'll tap into much better how your readers understand.
If you want to know more about that, I'll put some references on the website.
This was partly developed by Joseph Williams, a linguistic at the University of Chicago and George Gopen, who's an English professor and law professor at Duke.
And he's written a really interesting article called, him and Judith Swan-- it's called "The Science of Science Writing"-- where he gives a whole bunch of examples of restructuring prose along these lines and how much clearer it becomes.
And he's written two books, actually, one on teaching this approach to writing and one on doing it yourself.
It's a textbook for such a class.
One's called The Sense of Structure.
I forget whether that's the teaching book or the textbook.
But I'll put both titles on the website So I highly recommend that.
And what you'll see from this is the huge importance of punctuation.
And here, it's the same thing.
By varying the pitch, by giving periods, semicolons, and commas in the lecture, even though you can't say, OK, here's the period.
You don't say that, but you do it by varying your pitch and your volume and your tone.
You give structure.
And fundamentally important, you also convey enthusiasm.
Yes
On the other hand, I think in that lecture he offered [INAUDIBLE] [LOUDLY] he always speaks like this, [NORMAL VOLUME] which intimidated students a bit
Yeah.
Yeah.
He maybe does it too much.
In fact, he doesn't vary enough.
It's sort of like the radio in Spinal Tap.
It goes to 11, and it starts at 10.
Because it's always loud.
The amplifier's always loud.
Yeah, I think he could actually have more dynamic range.
So let's see.
More-- In his defense-- Oh, yes.
Go ahead
Finish your thought
OK.
In his defense, you could say, well, when do you want to be really quiet?
Well, that's when you're doing something that's less important.
And he's actually such a good lecturer that he planned the lecture, I would say, quite well.
He took out all the stuff that normally you would find in a lecture on this that's kind of dull.
For example, one thing that people do that's really kind of dull is they may derive the sinusoid as a solution.
And how did he derive the sinusoid?
Spray paint.
They did an experiment.
He made it visible.
He had a spring oscillating up and down with spray paint, so it was spraying on a big piece of paper.
And then he pulled the piece of paper that way.
So as the spring oscillated up and down and sprayed on the sheet of paper, it made a track, which you could see developing.
And then he just showed the class.
He said, what does this look like?
And they said, oh, it looks like a sinusoid.
There you go.
Now you skipped a whole bunch of painful math, otherwise, to make the guess.
And so either you guess it, or you do a bunch of derivation.
He avoided both of those problems by actually just having the students see for themselves.
In fact, he used the word see.
I want to make you see what it is.
So he got that.
He's taken out a lot of stuff where you would normally just try to pretend you weren't doing and speak really quietly.
So that's maybe one explanation for why his dynamic range wasn't as big as it should be.
But yeah, I think, generally, the point is right that he's so forceful and loud that it would be better if he just had a twice as much range
Just an extra comment
Yeah
And this maybe be necessary in the old days when you don't have microphones in lectures.
But when you have microphones, you can speak colloquially but still very clear to the students
That's true.
You have amplification.
Yeah.
And he did have a microphone, because he was being taped.
But you're right.
The habits of thought in presentation come from when we didn't have microphones.
Yes
The one thing I would change is at the very beginning he was talking about the test results
Yeah
And he said the test results were better, on average, than in previous years.
And he said, oh, you're either a smart class or the test was too easy.
Just generally, calling the students smart seems pretty alienating for the students who did poorly.
And then even the kids who did well, there's pretty good evidence that saying "you're smart" rather than "you're working hard", the latter's much more effective
Right.
That's a really good point.
Yeah, the test results discussion at the beginning.
I always knew this lecture was a classic one, and this is the first time I watched it straight through, rather than just watching the sort of demonstration parts and the famous parts.
And so it started off with this discussion about test results where he says all kinds of interesting things.
"You did very well on your first exam."
That's word one in the lecture.
"I was hoping for an average of about 75."
So hoping, that's not a word I would have used.
I would hope for an average of 100.
That's my hope.
Maybe not what I expect, but my hope is 100.
So by saying hope, he's saying, actually, it would be good if people got a lot of stuff wrong, like I'm hoping that people get 25% wrong.
That could be interpreted as hoping that the students don't do great.
But then, on the other hand, the class average was 89.
OK.
So that's good news.
Now, as you say, he says, "either you are very smart, this is an exceptional class, or the exam was too easy."
And he eventually concludes, no, the exam was taken by the instructor, so you guys are smart class.
But even that he takes away by saying, "but time will tell whether you are indeed exceptionally smart or whether the exam was too easy."
So even the good news is tempered and taken partly back.
And then even the good news itself, as you point out correctly, is not necessarily good for helping the students learn.
So there's very interesting studies-- I wish I could remember the name.
She's a psychology professor at Stanford, and she's done studies with kids.
What you do is you have the kids do a problem, and then you tell them either that they worked hard at it or that they were very smart.
That's why they solved it.
You tell Group A that, oh, yeah, you worked really hard at it.
That's why you solved it.
Group B, you say, oh, yeah, you solved it.
You must be really smart.
And then you see what kind of problems they try later.
Let's do Group B first, who are told you did it because you were smart.
Later-- this is making an extreme of it, but this is the trend-- they don't want to solve hard problems.
Because the only thing you can get out of solving a hard problem is you don't solve it, which then proves that you're actually not as smart as people though.
So they want to solve easy problems.
The group that was told, oh, yeah, you did that because you worked really hard, they enjoy solving the problems.
And they want to solve hard problems that they can't necessarily solve, because they can work on them.
And some of them even say things like, oh, I love it when a problem's hard, which is a completely different attitude than the attitude of the people who are like, oh, I don't want to try anything.
Because I might fail, and then the truth about me will come out.
By making comments about people's intrinsic characteristics-- I made distinction earlier between intrinsic motivation and extrinsic motivation and how you really want to use intrinsic motivation.
But there's the flip one, which is you don't want to make intrinsic comments about people, because that's very hard to change and very fixed.
And that's what people are doing way too much of.
So whenever people can't solve things, it's because you didn't work hard enough, the conditions weren't right, you can call it making excuses, but it isn't.
It's an optimistic way of looking at the world.
And it's that optimism that people learn when they have control over the things that lead to problem-solving successfully or not.
Yeah, I thought the opening was actually very bad in terms of that and actually set a kind of competitive, hierarchical tone, which I would have definitely avoided.
Other comments?
Other things you noticed?
Yeah
He kind of oozed with passion about the subject.
Usually, [INAUDIBLE] 5,000, he still seemed amazed that they worked
Right.
Right.
Let me erase this guy.
Right.
He's shedding passion.
He's oozing passion.
I'm sure he's done the demo many, many times for every semester, no doubt.
Yet he was still so excited.
For example, the ending, what did he say?
They did 10 oscillations with Walter Lewin as the pendulum bob.
And then they counted up to 10.
So then he got to 10t with Walter Lewin, 45.6, plus or minus 0.1 seconds.
And the prediction was 45.7, plus or minus 0.1, so it was right on the nose.
He said, "Physics works, I'm telling you.
See you Monday."
So he ends with enthusiasm.
He is enthusiasm throughout, all the way through.
Yeah, it is contagious.
People did applaud, and I'm sure it wasn't just because it was being taped.
They really were very pleased and amazed at the close agreement, even when he changed the pendulum bob from just the regular pendulum bob of a few kilograms to Walter Lewin of 70 kilograms or-- he's probably more-- probably 80 kilograms.
And throughout as well, doing the demonstrations, he was also very enthusiastic.
And even sometimes he got lucky, and he even said so.
But he said, fantastic agreement between theory and experiment.
Yeah, so the enthusiasm, it transfers.
And in fact, our system, our brain is actually much better at picking up this kind of stuff than it is at picking up the Navier-Stokes equation and things like that.
Let's call it an emotional highway.
It goes in the emotional highway.
So I mentioned the experiment that you can do to test this.
The experiment you do is you just go somewhere and you just smile at people.
And now 30%, 40%, 50% of the class is smiling back already.
You can't help it.
Even though you know and I'm telling you I'm just doing this-- There's nothing actually funny that just happened right now.
I'm just smiling, just because I'm a nice guy and I'm smiling, and I just wanted you to see if the experiment works-- you can't help it.
So that tells you how powerful that emotional highway is.
It's not just for smiling.
It's the oozing passion.
Neurobiologically, that mechanism is starting to be understood.
I'll write the mechanism down.
We have what are called mirror neurons.
What mirror neurons do is they recreate the state of mind of other people in us.
So that's why they're called mirror neurons.
And they're innate.
It seems that monkeys have them.
We have them.
There's nothing you can do about them.
And they are actually probably the mechanism by which we become social creatures and which makes us social creatures.
So if people around us are unhappy, we feel unhappy.
If people around us are happy, we feel happy.
And these mirror neurons, their firing tendencies are partly set in young childhood.
So if the people around you are happy in childhood, you'll have a much easier time just automatically being happy later.
And if the people around you in childhood are really unhappy and your childhood's unhappy, you have to work much harder later to overcome that.
So that's the importance of childhood and child upbringing.
But now these are there, and your students all have them, being people.
So how you act and how you perform is going to affect them much more strongly than any equation you say.
It's fundamentally important, again.
OK. Any other points?
Yes
Yeah.
I was thinking about the use of the spray--
The spray paint can?
Yeah
Yes.
And it wasn't until now that I actually understood why he did that.
Because when I saw it, I was sort of thinking, why is he doing that?
Or if he pulled the lever faster, it's going to look the same.
I didn't get it at all.
And now I see it was to guess the solution [INAUDIBLE].
And then I'm thinking, when I was taught the harmonic oscillator for the first time, that's an opportunity the professor used to actually teach how to solve, OK, this is the differential equation, and this is the solution for it.
And now you know for the rest of your life.
Because solving a differential equation by [INAUDIBLE], you're not always going to have an experiment to solve--
To show it
Right.
It's kind of cool.
But I'm not quite sure what's the purpose
OK.
So what's the purpose?
It could go here or here, the spray paint example.
The normal way it's done is that you either just guess it outright, you say, well, we've seen that equation before-- we being the royal we-- the instructor's seen the equation before and says, use the sinusoid.
The students have never seen it before and just have to take it on faith.
Then you put in the sinusoid, and then you show that it solves the equation.
The other way you do it is you separate the equation.
There's many techniques for solving that equation.
One is you write it in terms of operators and factor the operators, but the freshman aren't really ready for that.
So you can't really do that.
You're basically reduced to guessing, at least in the American curriculum.
Now what he's done is he's done an improvement on that.
Instead of saying just by bold assertion, just outright authority saying, please guess this, he's saying, look, this is what it did.
What do you think it is?
And then the students say, oh, it's a sinusoid.
He says, OK, good.
That justifies the guess, basically, that I was going to make anyway.
So it improves on the standard method of guessing and just plugging in.
But you're right.
Not always can you do that.
So maybe the thing I would have done is, if I were him, I would have said, ah, this is going to avoid us just either guessing the solution or doing a whole big, long calculation.
And he does say that elsewhere when he talks about the zoo, when he wrote down the differential equations for the x motion and the y motion.
He said, oh, that has a coupled nonlinear equation.
It's hopeless.
It looks like a zoo, and it is a zoo.
So then he approximated it.
He was quite good at saying, look, there's some things you just don't want to do.
And this would have been, I think, another spot to do that.
He could have explained why he was going to do the experiment.
OK. Just one other point, I think, that wasn't mentioned so far.
Oh, yeah.
Yeah, he never erases what he's using.
We'll talk about that more the next time when we talk about using chalk and slides.
If you notice, he never erased anything he needed.
Now you could just call that chance.
So an example of when he had stuff that he was using before.
He had the differential equation, and then that was way up on some board.
And then later, after doing the experiment with the spray paint, he came to x equals-- So now he had to calculate x and x dot, the derivative, and x double dot and then put it in.
Well, as he was doing that, fortunately this equation was still there the whole time.
It was just high up.
So he never erased anything he needed later.
And I was watching pretty carefully through the whole lecture.
I think there was no time when that had happened.
And you could call that chance.
Maybe if it happened once it would just be chance.
But it's actually preparation.
So that's part of lecture preparation.
What am I going to need?
And how do I make sure that it's still visible?
Or you can write it again.
That's also fine.
But how do I make it so that students don't have to remember it, flip back in their notes?
Because to them, this is a new equation.
It's not just the equivalent of one pawn, one chunk.
It's six or seven chunks.
So you don't want to fill up their short-term memory trying to remember that.
You want it visible somewhere.
And that was also planning.
Yes
He calls tension a force
Yeah, he did call tension a force.
I was most upset about that.
That is the standard way it's done, and all the physics courses I took as an undergraduate did the same thing.
And I'm sure that's how I got confused about it.
I'm sure most people are confused about it because of that.
And until they do a few thought experiments or have to teach it and really sort through it, they won't figure out that it's actually not a force.
Now I'm sure Walter Lewin doesn't internally think of it as a force.
He knows it behaves differently.
But it's so easy to use this shorthand, and then the students, they don't have any other structure for it.
They'll just think it does act like a force.
It add vectors, but it doesn't.
Yeah.
So that was, I thought, a dubious point.
But it's standard, unfortunately.
It's a good lesson that just because something's standard doesn't mean it's a good thing to do.
Yep
He used the big chalk and colors
Yeah, big yeah.
Right.
Actually, if you need to order this chalk, it's called railroad crayon.
You want to be careful to get railroad crayon, because there is actually big chalk you can buy from, I don't know, variety stores.
And that's for, for example, writing messages on sidewalks.
And the problem with that is that has wax in it so that it stays against the rain.
But that's not so good for the blackboard.
It's much harder to get off.
Railroad crayon is designed to be removed easily.
If you order railroad crayon, unfortunately it seems to come in boxes of 144.
Maybe you can get a box of 72.
So share it around.
If you find a source with smaller amounts, let me know.
But the ones we found were 144, a gross, one dozen dozen.
But yeah, he used railroad crayon, so it was very easy to see what he was doing.
And he used colors.
For example, forces were always read.
And he used a consistent naming convention for that, so he wasn't throwing noise onto the system.
Yeah
This was mentioned before, but I think one important thing he did is that he created suspense by taking a risk.
Because he repeated the full experiment in class.
Because [INAUDIBLE]
He created suspense by taking a risk.
So there was actually tension, not this kind of tension, but the good kind attention where people were involved in interested.
Because he redid the experiment, and it could have come out not quite right.
And amazingly, it did come out right.
And when it does, then the point he's trying to make it even more strongly reinforced about physics works.
And that was his goal.
That was one of his overall goals for the implicit.
I don't know if he ever stated it, but one of his implicit goals, you can see, is to convince people that physics works.
It's a way of understanding the world.
And if you really believe that, well, you should be willing to put your money where your mouth is and do it yourself.
There's an another famous experiment-- I don't know if it's in Lewin's lectures-- where you want to show that energy is conserved.
And the way you measure energy conservation is how far a pendulum swings.
So now the way you do the experiment, you have some giant pendulum with a bowling ball or a cannon ball at the end hanging from the top of the physics lecture hall, which is about this high.
And you release it from your nose, and you just stand there.
So if energy's really conserved, it's going to come just to where you released it and no farther, which is good because your nose is right there.
A little farther would hurt.
The problem-- this still doesn't show physics doesn't work, but it looks like it-- is that it's so tempting to give it a little push when you release it.
And it's just hard not to do, because whenever you let go of things, you push a ball, you throw something in the laundry basket.
So now you give it a little bit of push, and physics works.
So it comes back with exactly the same push, which wouldn't be a problem if it was a wiffleball or a tennis ball.
But because it's a giant cannon ball and at the other end of it's your nose, it gives you a nice crunch right there.
I would only do that expect if I'd practiced it and make sure I didn't push it.
But even if you do push it, it too shows physics works.
But yeah, he created suspense.
Yes
So how can people do this in other disciplines?
In physics, you have weights and swinging pendulums and blocks
How can you do it in other fields?
Yeah, it's a good question.
If you remember, the wood blocks-- this is physics, mechanical engineering-- there was also suspense.
Because when I tap this one, and then everyone was dead silent before we tapped this one-- just for review-- because everyone wanted to know.
So suspense was created.
And it's just like any sort of magic trick or performance.
You have to prepare it.
So it doesn't have to be that you bring something in.
It helps.
But, let's see, what else could you do it in?
Chemistry.
In chemistry-- it helps to bring stuff in, no question-- you could bring in chemicals.
What do you think is going to happen when I mix these two?
Well, what do we know about this?
What do we know about this?
What's going to happen when I drop a ball through this?
Is this viscous?
Is it not?
What clues did you have?
OK. Let's see if now that's consistent with how the thing behaves.
So it doesn't have to be pure physics.
Though it does help to bring things in, and so you do want to make everything you have as visceral and as visualizable as possible.
So that is a general transferable principal.
Let me write down four general principles that you can use in all lecture performance, which is basically everything you said here.
The first one I'm going to write down is very neglected, but you've just mentioned it, which is the suspense and the timing.
If you want evidence that timing really matters, just go watch Romeo and Juliet again, or read Romeo and Juliet.
Just because they were slightly off in their timing, they all committed suicide, basically.
That's probably the very, very, very, very CliffsNotes plot summary.
And another part about timing, if you want more evidence that timing really matters, try telling a joke and tell the punchline first.
See how well that does.
And even a small example of that, for stories too.
Stories, you want to tell the important part later, and you want to give the introduction first.
Again, remember the sentence structure here, which is that the thing with emphasis comes right at the end.
If the thing you want to emphasize happened at the beginning, then your timing is off, and things will fall flat.
Yes
OK.
But that's exactly contradictory to the way some people want their information laid out, which is tell me the most important thing, and then back up with all the details
Yeah.
OK.
So that's a good comment, which is that it's opposite of how people say they want things.
They want the most important stuff first and then the details.
The way I square that circle is I don't want to give away punchlines.
Oh, those are gone.
So suppose I'm starting a new unit, and we're going to do a unit on extreme cases.
I don't want to give away the punchline, which is, oh, this problem is hard until you use extreme cases.
So I say, forget the organization of the course that I told you about before, and we're just going to do an example and discuss it.
Then we're going to look at what makes it really easy, and then the punchline is extreme cases.
OK. Now that we know the punchline, because we've done the example, now I'm very happy to say extreme cases.
Let's use extreme cases here.
And this whole unit is going to be about extreme cases.
So now people still have the structure, but the very beginning of it, they're a little confused.
And I like the confusion.
So that's part of this.
In a play or in a movie, or you go to a movie, you're not even sure who are the main characters and what are they up to.
And then it comes into focus soon but not right away.
And part of that is to draw you in.
Yeah.
I wouldn't begin a lecture by telling people everything that they need to know, because that's spoiling the timing.
The beginning of the lecture is this kind of sacred moment when you want to draw everybody into the lecture.
And the beginning of a course is doubly that much.
You want to draw people in, not tell them, oh, yeah, here are the rules on homework collaboration and blah, blah, blah, blah.
You want to emphasize the important things first, and you want to use this to pull people in so that you can come to a punchline.
The punchline doesn't have to be way at the end of the lecture or the end of the course.
It can be soon, and you can have a lot of soon punchlines.
But for each one, you need to build to it.
Yes
[INAUDIBLE].
When I did my referencing thing, the first slide had a list of goals that I wanted to make, because I thought that that was what we should be talking about in [INAUDIBLE] teaching.
And then you asked, would you have included those when you started the real lecture?
Right
And I said, I wouldn't.
I would just plunge right into the material, because I was thinking along the line of what you're saying now.
And then you said, no, I think it's actually good for the students to see it outlined what they're going to learn.
And so that's actually [INAUDIBLE]
Yeah.
That's right.
I think what I would do is I would show them that goals-- Goals are OK, as long as you don't begin the lecture by saying, here are my goals, because that's that secret time you're taking away from.
I would put the goals, say, on a side board, so they can see them all the time, or write the, just after you've done your punchline.
Then put them all up
So it's the second or third slide
Yeah.
That's right.
Yeah.
The very first slide is something interesting that doesn't necessarily seem like it's related to the class.
And that's already a punchline.
Oh, wow, this?
We can understand this from the stuff in class?
I want to know.
That's a drawing in.
And then you get to the punchline, which is, yes, you can.
And the goals are to see this all over and see these three principles that we just used and really understand them.
OK Another one be bold, so bold gestures.
Bold gestures, voice, you'll see that all throughout Walter Lewin.
Another one, we've actually done, I would say, a lot more with it than you saw in the lecture, which is-- And the third one is you want to involve the audience by asking them questions, making your whole lecture interactive.
So we've discussed, actually, many ways where the Lewin lecture could be improved in that direction.
And perhaps underlying everything, none of this really works unless there's principal four, which is that you have to care.
You have to really, really love what you're doing and transmit that to the students.
And then, all of these things become so much easier when you care.
So those are the fundamental principles of lecture performing and, basically, planning around things like that.
And if I just had written those down in the beginning, it wouldn't have made sense.
But now you've seen many, many, many examples, and we've had many discussions about it that I wish you success in applying good timing, good bold gestures, strong gestures, and involving the audience, not just emotionally or cognitively but both, so that you provide long-lasting learning and interest in the subject that you love.
OK.
If you could fill out the sheets for just one minute.
And then we will have office hours.
Actually, what we'll do is we'll meet right outside the door, and then we'll just go sit in the cafe area.
And anyone who wants to get a bagel can also get a bagel, if you're interested.
Answers from Lecture 8 to questions generated in Lecture 7
So first question was, a while ago I mentioned there was research saying that if you pay people for things they like doing then they start enjoying it less.
Can I give you a reference for that?
The reference for that is I actually put on the readings for next week, which is the political barriers to educational change session, one of the five readings-- they're all basically short-- is by Alfie Kohn, and it's about competition.
And that's one of the areas he researches a lot.
I gave you one of the short articles by him to read.
But his whole website as lots of different articles and references.
He has a whole book on the whole subject called No Contest.
I think the subtitle is The Case Against Competition.
So he goes through, I don't know, 100 or 200 studies about the effects of rewards on people's performance and interests.
It's a totally fascinating book.
And I think it just came out, or recently it has the second edition.
But both editions are completely fine, so whichever one you can find, use that.
And on his website, you'll find many articles on that theme, and I've given you one to read for next week.
OK. What is the Berkeley Physics Course?
The Berkeley Physics Course-- This is interesting, because in the '60s there was a whole bunch of change.
It was change in political systems and protests and legal systems, the Civil Rights Act.
And there was also change in education.
And that's probably not coincidence, that both changes were coupled or happened together.
There was the rise of the Free School Movement.
There was also, in the university level, lots of different new curricula started.
The Feynman Lectures on physics are a famous one that happened way over on the other side of the country and Caltech.
And the Berkeley Physics Course happened also on the other side of the country.
There are several volumes in it.
Probably the best known is Ed Purcell's Electricity and Magnetism.
That's one of the classics of the Berkeley Physics Course.
There's a less well-known volume, which is, I think, even better, which is by Frank Crawford, and it's called Waves.
The edition I have came in the back with a whole bunch of polarizers and toys you could use to actually do home experiments.
And at the end of every chapter, there are home experiments.
I think they're called "Kitchen Sink Experiments, and there were toys in the back in the little pocket that you could use for it.
So if you find a used copy, those toys might not be there, but the book is still wonderful.
So those were two pieces of the Berkeley Physics Course, and they were about six volumes.
And MIT had a whole set of books as well, I think by Tony French.
So it was a whole time of ferment, and lots of changes happened, many of them good.
OK. How can you apply these lecture techniques that we talked about last time to laboratory or practical courses?
This is actually an important topic, because many of us will do that?
What's the right balance between teaching and lecturing and allowing students to solve the lab problems on their own, in other words, giving them enough rope to hang themselves?
Yeah.
You do want to do that.
You want to give people enough rope, but you also want to guide them, because you do know more than them.
So I don't look at laboratory teaching, practical teaching as that different than lecture teaching.
In some ways, it's easier, because one of the goals in lecture teaching is to show people that all the theory you're talking about actually has any relevance at all.
And you have to try to work hard to bring stuff into the classroom and really show, yeah, look this really means something, the way Walter Lewin stood on the pendulum or sat on the pendulum himself and was the pendulum to really show, yes, this applies here.
In a laboratory practical class, if the practicals are at all interesting, that's automatically done for you.
The last practical class I taught was the electronics practical for physics majors.
At the end, they built an AM radio, a crystal radio.
So that actually had a nice finale, and experiments before that built up to it.
The relevance was already done for me, but the problems the students had were the same as in regular lecture.
They didn't understand the concepts.
For example, what does a capacitor do?
What does an inductor do?
How do they work together?
What is resonance?
So what I did for them is I gave them short little mini-lectures, two minutes three minutes on a conceptual question, and then sent them off working.
Some of the other problems they had were, for example, they were really scared of oscilloscopes.
They just got to the oscilloscope, and they thought, what the hell do I do with this thing?
So do I used an exercise that was done to me when I took the same class as an undergraduate, which was in your laboratory pair one person turns around and the other person randomizes the oscilloscope settings.
They just turn all of dials to some random setting, and then the first person has to turn back and try to get some signal on the screen.
So they have to figure out, OK, this voltage setting is crazy and get it back to a setting.
And once you do that, you're no longer afraid of the oscilloscope.
Again, you can do in class problems that people work on together.
You can give short little conceptual questions.
So in a way, laboratory teaching isn't that different, provided that the laboratory isn't just a cookbook in the sense that, OK, do this, now measure that, now measure that, now measure that, now measure that.
OK, write it all in your report.
That's like lecturing just by saying now copy this down, now copy that down, now copy that down.
So if you get away from that in lecturing and that in practical, then you get to the similarities between the two, and you can apply pretty much every lesson from here.
OK. Oh, yeah.
Many of you pointed this out.
The distillation of examples into principles that we did.
So we looked at Walter Lewin's lecture number 10, and then we distilled from our discussion various principles of lecturing and performing.
And many of you said, oh, yeah, that should have gone longer.
You liked the principles, but we should have done that for longer, which I think is a great sign.
I think it's good that you guys are figuring out what could be improved.
And I agree with you.
I just got so carried away with all the interesting things you were finding that I didn't allocate enough time for the principles.
So in my next life.
But also, the point is everyone did appreciate having principles, so that's something to transfer for your own teaching.
OK. Let's see.
What kind of interactive examples and demos can you do for a math class?
I brought an example.
They're in here.
This was actually an example I did last week in my approximation class.
And I enjoyed it so much I thought I'd show it to you too.
The principal I was teaching was the principle of easy cases, also known as extreme cases often.
But I like this name better.
And then I wanted to show that this principle applies across all kinds of areas.
You can use it in math.
You can use it in physics.
You can use it in engineering.
And what we were trying to derive is the volume of a pyramid.
So the original problem was this.
You take a pyramid with a square base, and then you lop off some piece of the top.
So the pyramid is B by B on the base and A by A square on the top, and it has height h. So what's the volume of that thing?
Well, you can guess a lot of it from easy cases.
I won't go through all of that.
But at some point, you need to work out in part of this guess the easy cases, use the easy case that A equals 0.
In other words, you just have a regular square pyramid, like one of the Egyptian pyramids of Gaza.
Giza?
Giza, maybe.
So then what's the volume?
OK. Well, volume's going to be proportional to the height.
And then, it's got to have two more lengths here, so it's got to have B squared here.
There's no A left.
But then what goes here?
What's the constant?
Well, this is actually where you can, yet again, use easy cases.
So if you could find a special pyramid that could be combined into a shape whose volume you know, then you could actually figure out this constant.
Because you don't know the volume of one pyramid, but if you could replicate it and then assemble them into a nice shape, you're home free.
What nice shape could make this into?
And what pyramid shape do you need to be able to do that?
So take a minute and think about that.
Anyone have an idea?
What shape would you like to make?
Could you first make a rectangular cube and then cut it into six pieces?
OK.
So you want to make a cube.
Let's talk about how you're going to make this cube.
You'd like to make the cube.
Now what part of the pyramid is going to become the face of the cube?
Well, you have a square base, right?
So it seems reasonable to expect that each pyramid's going to provide one face the cube.
So you need six pyramids to make this cube.
Well, what shape should the pyramid have so that you can fit them into a cube?
Hmm.
How about that shape?
So what's special about that shape?
I'll put one here, and here's another one of them.
Put that there.
And here's another one.
OK.
So here we go.
Here's a cube with six of those pyramids.
So what's special about the shape of the pyramid?
What do you have to set in it?
Yeah
The height is half of the base line
OK.
The height is half the base.
How do you know that?
Because the tips of the pyramids are [INAUDIBLE]
Right.
So the tips touch like this.
And this width has to equal either side of the square.
That means, let's say, if h equals 1 and B equals 2, six times the volume is equal to the volume of the cube, which is 8.
So the volume of one pyramid is 8/6 or 4/3.
h times b squared is 4, so you have to divide it by 3, and you're home free.
So there's an example of a demonstration in math class that you can use.
Now the point of it is that it's possible to imagine it in your head, but it is actually useful to see these things built into a cube.
So even in math class you can do demonstrations.
What's my blackboard philosophy?
I write big and sometimes place things fairly randomly.
Is there a specific reason for this?
But I like what you write down.
Yeah.
My blackboard philosophy is not ideal, because of exactly that.
I write large, which is good, but I do place things kind of randomly sometimes.
And that's not ideal, so I try to improve that.
I don't necessarily have a reason for that.
Don't actually copy that part of my blackboard style.
But it's good that you're observing that and trying to criticize it.
OK. Let's see.
How to practice blackboard work.
We'll talk about that more today.
But the basic simplest way that I've found is at the end of your session you just go to the back of the room and look at the blackboard and you just see was it organized, did it make sense, did I write a bunch of stuff in between, and just take a few notes down on that.
And then you'll automatically be getting feedback that you can use to improve your blackboard work.
Oh, yeah.
Several people pointed out they watched a different lecture than Walter Lewin's Lecture 10, because I said, oh, watch that one or another one.
So next year, I actually am going to ask everyone to watch the same lecture, Lecture 10.
That's a good suggestion.
OK. We talked about that.
Oh, yeah, if you're an excellent lecturer and you are aware of the common problems, because you've taught the course many times before, and you try to address all the confusing parts, can you get away with not allowing students to ask questions in class and not asking students questions yourself?
That's a very interesting question.
Basically, can you move away from the interactive model I've been describing and recommending once you've done it enough and you know where the students' misconceptions are.
I thought about that, and the answer is really no.
Because then you're reverting back to telling.
Now you're telling them things they really do need to know.
So instead of telling them a big long song and dance about the Atwood machine, which they don't really need to know, you're telling them about their misconceptions about F equals ma.
But the problem is just telling them that F equals ma, F equals mv, doesn't help them realize that.
They really have to have some kind of cognitive conflict and struggle with it.
So questioning, either you question them or them asking questions of you and formulating questions, is really about the only way to get them to struggle with it.
Even then, you still have to do it.
And the experience you have from teaching it many times means you'll be doing questions about useful, important material.
That was quite an interesting question.
OK. What do I think of group assignments, balancing the advantages of collaboration with the fact that each individual may not encounter all the topics?
I generally don't like them that much.
I think if you design a course and you think there's a coherent set of material, everyone really should try to master that coherent set of material and not palm off half of it on one person in the group and the other person in the group.
The extreme of this kind is the graduate seminar where each graduate student gives a talk on one topic.
And then they really learn that one topic, but they don't necessarily learn the other topics very well.
So I don't like that very much.
I think it's better to allow people to collaborate, but make sure everyone does their own assignment.
And yeah, collaborate in groups, by all means, but please write up your own work as well.
What's the use of experiments in Walter Lewin's lecture?
Was it to show the predictive power physics or its experimental character?
It's not clear.
And in that case, wouldn't it be better if the students performed the experiments on their own or in a lab class?
Aren't these demonstrations a waste of time that could be used to teach more theory?
And aren't shows more appropriate for high school classes than for physics at MIT?
Well, I think, at the time, this was 1999, there was no laboratory with the introductory physics course.
I need to get my history right, but I'm pretty sure that's right.
There was no other way the students would ever see any application.
But the other main point of it is that it showed that the physics is real.
Now I think it could be improved, as you suggested, by Lewin asking students questions about the experiments, trying to predict what would happen beforehand, for example.
But the idea of bringing in objects is really a sound one.
I had a teacher at Caltech.
He taught the waves and the nonlinear waves course.
He always used to bring in a wave guide, which is, basically, a piece of metal in a tube-like shape.
and he said he always did that because, otherwise, the mathematicians would think it's just a vessel function, because that's how it is in all the problems.
You see wave guide, you just start solving Laplace's equation with all these boundary conditions, you get Bessel function.
And wave guide is just trigger for do Laplace's equation, get Bessel function.
It has no physical meaning.
So he would always bring in the wave guide so that you knew that it was a real object that you were analyzing.
And so in that way, Lewin, using the pendulum, showing that these things actually exist, avoids the trap that many students fall into, which is that symbols just have no meaning on their own.
They're just stuff that goes on on the screen.
So I think is really important that he does that.
OK.
Working hard versus smart.
How do you respond to people who tell you that they worked really hard but they still did poorly?
I think, basically, the reason is that they didn't work smart.
So what do students think working hard means?
They generally think it means reading the book over and over again or trying really hard to do the problems but not reflecting.
The equivalent in playing chess is working hard at chess, people think, oh, that just means playing lots of chess games.
But actually, that doesn't improve your chess ability.
You have to analyze your chess games, reflect on them.
And that's what we need to teach students how to do.
So whenever they do say that, I say, OK, well, what are you doing?
Let's look at what you're doing and see if there are ways we can make it more reflective and less just going through the same stuff over and over again.
And then, final comment-- there are some more questions, but I'll leave it there for now-- while watching the 801 Walter Lewin lecture, I was surprised how much I was able to find wrong with it, even though he's a great lecturer.
That goes to show how much we've learned so far in this class, which I thought was a very nice comment.
I'm glad.
And I hope people did draw that conclusion.
From all this stuff that you found to improve that lecture means you have actually learned a lot, either in this class or elsewhere, and you could actually make really excellent lecturers, really powerful, constructive lecturers yourself.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK today, teaching with blackboards and slides.
And also several questions from the last time.
And, related to that, a handout, which I've put online, which is on how to make a lesson plan.
So I'll do all that through the questions.
So now just a quick bit about lesson planning, because a couple of you pointed out that you'd have liked to know more about lesson planning.
And reminded me that when I taught the chemistry TA workshops, I actually gave everyone a handout on how to plan a lesson.
So I've put that handout online, and I'll just show you what it looks like here.
So this is basically the sheet I use for planning any kind of lecture recitation.
At the top, you need some kind of course, and date.
And then the objective.
So that's quite important.
You'll see the exact sheet.
And there's a PDF file, you can just use it yourself.
There's a course objective for that particular session.
And that relates to your overall goals for the course.
So, for example, in this problem, my objective might be something like, to show that easy cases-- in fact, this was my objective-- I wanted to show that easy cases-- extreme cases-- are useful not just for checking formulas, but also for generating formulas.
OK, so I write that on the top.
So I remind myself, why am I here?
What am I doing?
Then there's a three-column table.
Basically you just plan items in your lecture.
The first column is minutes-- how long you expect each thing to take.
And then the middle column, most of the space is, what's your goal in this local part of the lecture?
And what are you going to do for that?
So here's my overall objective.
And my local goal may be, just this example-- getting that one third.
And then here in the third column is props.
So there's actually a piece here, too.
So props are anything that I need to bring.
So I just stick that in the margin.
OK, bring the cones, or the pyramids, whatever they may be.
And so then, when you're filling this piece in, about what to bring, you just scan down this last column, and just write everything here.
OK, cones.
Homework three, solution set two-- whatever it may be.
So now, at a glance, before you go to your session, you review these two things.
Make sure you have everything, your know why you're here.
But then here, you're going to follow your script.
Which is a loose script.
It's not exactly a word-for-word script.
It's, maybe, a few equations.
Suppose there's a question.
What is this constant?
OK, so that's my overall, what I'm going to do for, say, ten minutes.
And then I break it into sub-questions.
OK. What is H and B?
How many pyramids do I need?
What's my goal shape?
So these are questions that I ask students.
And for each one, I write down the minutes.
Maybe in parentheses, put the total up here.
So now, when I've done my two or three pages-- and by the way I find, just at the level of detail I use, if I ever go beyond three pages, I never get to that material.
So it's just sort of for a security blanket by the time I'm beyond three pages.
It depends how detailed you write these things, but I've generally found that's where I am.
And then, you estimate the time.
And what you'll find is, the first time you do stuff, you'll massively underestimate the time.
You'll be off by a factor of two.
So, in other words, if you think it takes five minutes, it'll really take ten minutes.
And that's true even after you take account of this rule.
Sort of.
OK, so then you put down all your minutes on this page, and the next page.
You add them up, and you make sure that you're not over 50, or whatever the amount is.
And then what you'll find is what you thought was 50 really is going to take you 130.
So the next time, when you evaluate minutes, you'll have a better idea.
And you'll find yourself actually tuning your time sense pretty well.
So now I've got my time sense that I don't really need to put the minutes down, because I can just do it by number of pages.
Three pages, too much.
And so you'll find your own writing level, and detail level, how many pages to use.
Now the other point is, where do these questions come from?
OK, well this is a way you can turn any regular lecture into an interactive one.
So suppose you have a long derivation that you're going to do.
Or, for example, suppose the first way I planned this, was I wanted to show people that this was 1/3 over here, this is three here.
And the way I was going to do it, was I was going to draw six pyramids, show them in a cube, and show that 6V, and do all this showing.
Well, first draft, write it all out like that, with times and everything, without questions.
Then, any time you come to something interesting-- so here.
So before you have [?
tel.
?]
So now you look at your sheet, and you say, hmm, where did something interesting happen?
Hopefully, there is at least one point.
Because if there isn't, maybe you shouldn't be giving the lecture at all.
So now, let's just say, by construction, you found some interesting things.
OK, so that's interesting.
Oh, it's interesting that it makes a cube-- it's not obvious.
So just think, where does something require thought, and that you're short-circuiting the thought by telling.
So then, what I do is I have a green pen, and I turn it into a question.
So I'll circle it in green, and write, "ask" next to it.
OK, or you can just rewrite your sheet.
Telling something, then asking, and then continuing.
OK so that's how you can turn any regular lecture into an interactive one.
And that's your sheet that you walk into class with.
And what you'll find, is that the first time you do it that way, there will be a bunch more spots you realize were actually subtle.
Because, for example, at the end of class session, when you collect your feedback sheets, people will have questions about different parts.
You'll realize, oh, there was actually something interesting that happened that wasn't obvious in one of these telling points.
And you'll be able to turn that into a question, as well.
The master sheet with all that formatted for you is online, on the course website.
And you're welcome to use that in your own teaching, and distribute it to everyone you know Any questions about that?
Yes
When you run over time, then you have to shift some of the things to the next lecture.
So how do you do that?
What happens when you run over time.
The first time you teach the course, you basically find out there's just twice as much material in the course as there should be.
And you'll always be running over time if you try to cover every single thing.
So, the first time you run out of time-- there's two approaches to it.
One is just to slow the entire pace down, and cover half of the amount of material.
So that's quite a reasonable approach.
The other is to sort of keep rhythm, but not cover every single thing.
So my piano teacher, a while ago, she said, when you're sight-reading new piano music, the most important thing is, keep rhythm.
So don't just stop, and then think for like ten seconds about one measure, and then continue in this herk and jerk.
Just keep playing at speed, but skip some of the notes, and do whatever you have to do to keep rhythm.
So you can try that approach too.
And then, what you do is you say, OK look I'm not going to cover every single thing in lecture.
A lot of stuff is in the book.
And that's good to do anyway.
So a mix of the two approaches is one way to deal with running over time
[INAUDIBLE] So each lecture you have some objective.
And then, if you run over time, then in the first lecture, you only finish half of it.
And then, the second one seems to start from the middle of [INAUDIBLE]
Right.
OK.
So that's one reason it's worth writing the objective first.
Because then you know what the main goal is.
So one way to do it, which I do like, is to make sure that-- right away-- the objective is reached.
So you do it as sort of a layer cake.
So the first example-- maybe the first two examples-- just from those, if everybody just doesn't do anything else but the first two examples, which you expect will only take 15 minutes-- but it takes 40 minutes, it's okay.
Because the first two examples reach your objective.
Now they don't reach the objective 100%, but they give you 80% of the objective.
So you want to plan your lecture structure like a layer cake, or like JPEG.
People know how JPEG compression works?
So JPEG, the way you do it, is the low-frequency, quote, "most important" terms, come first.
And then the higher-frequency, say, less important terms, come later.
So sometimes you see things rendered in your browser step-by-step, and they just sort of take focus.
And it gets better and better as you wait longer and longer.
So you want to plan your lecture a bit that way.
So that it's robust to time shortage.
So that if, for example, there's a fire alarm halfway through your lecture, still the main point got across.
Now it didn't get across in all the detail, and the beautiful glory of that you wanted, but the main point still got across.
So then, if you run out of time, it's OK. You say, look, there's two more examples of this which we didn't do, but they further illustrate the main point.
See the notes.
And then you can keep to your plan.
Assuming that your plan is a reasonable one.
If your plan is that, I'm going to do f equals ma today, and I'm going to do rigid body rotation tomorrow, that's probably not a reasonable plan.
And the fact that you ran out of time on day one is probably a good sign that you should actually spend a little more time on f equals ma.
Does that answer your question?
Other questions?
Yes
I have a question about [INAUDIBLE]?
A lot of people have told me over the years that it's good to write really big.
And I know you're constrained by that the people have to be able to see.
But, given that you're writing large enough for them to see, what's the advantage of writing any bigger than that?
As opposed to having the [INAUDIBLE]?
OK, so that's a good question.
Actually, why don't we save that.
We'll talk about that today.
So the question was, how big should you write?
Just enough so people can see, or bigger?
So that you can keep the full story there.
OK, so, any other questions on the lecture planning?
And blackboard, and slides, we'll come to starting right now.
Teaching with slides and blackboards.
What are the advantages, and disadvantages of each.
I'll tell you my bottom line, which is that my zero-th order term, so this is my objective-- if the lecture ends after the next two minutes, at least you'll have the zero-th order term.
The zero-th order term is that, if you can't help yourself, use slides, but otherwise, use blackboard.
For 90% of things, blackboard is much better for teaching than slides.
There are some cases where slides are useful, and you can make an argument for it.
There's some cases where maybe the slide is as good as a blackboard, few cases where it's better than a blackboard.
But the default is, blackboards are better than slides.
Now let me show you an example of why that's true.
Suppose you're teaching the-- an example I've done with you before, slightly, is the Navier-Stokes equation.
Let me write it down, to start with.
Actually, I'll put it on a separate blackboard.
So there's our topic.
So we always know what our topic is, because it's up there, and it's just going to live there for a bit.
And in fact, in some classrooms-- you can't do it so easily here, but my favorite classroom is 4-265 because it has pretty much this much blackboards in the front, and it has blackboards all around the side, and on the back.
So I don't use the back ones so much, but the side ones are really, really useful.
Because you can, for example, put the topic, like I've done here, on the side blackboard, and just let it stay there the entire time.
You can put an outline of the lecture.
OK, so let's say we have a lecture on Navier-Stokes equation.
We want to look at them, try to understand them.
So here are the equation.
Well "are," "is," the plural is a bit ambiguous.
This is actually three equations, one because it's a vector, vector, vector.
Already there, you see something that's harder to do with slides.
You'd have to work harder to do that in slides.
You can just go back, and annotate as you go.
And here, you can see the entire topic while you're looking at this.
So, for example, the student has now got overwhelmed by all the little symbols here-- the del squared, the partial derivative, the v dot grad-- that part always scared me when I was a physics student.
I knew what del dot v is, but what's v dot del?
So you get scared by all that.
And then people forget, they panic.
Because their short-term term memory got filled with every single, little, chunk here.
Their chunks are, remember, very small.
And they forgot, what are they doing?
Navier-Stokes equation fluid mechanics.
OK. And now, you can even improve that first blackboard by talking about, what does this apply to?
Drag, turbulence, river flow, airplanes.
So here, you can have a reminder of why it's important.
So now that just stays up there the whole time.
And people can re-center themselves every time they get confused of what they're doing, just by that.
And then, you can continue.
You can say, OK, let me try to explain the meaning of these terms.
So let's do the terms one at a time.
What's this?
So this term here is Dvdt.
We've seen things like that-- that's sort of like an acceleration.
You can annotate the equation right there.
That's also hard to do on a slide.
I mean, you can do it, you can work at it, but it's very natural on a chalkboard.
Now, there's some things that are common to chalkboards and slides, which is the use of color.
So now this, unfortunately, is blue, which is not the ideal color.
Unfortunately, I don't have my-- orange is generally better, but anyway I have blue.
Can you see the blue?
So marginally?
Maybe this orange, even though it's not as big, will be better than the blue.
OK, well this shows you an example of color, and the importance of choosing the right color.
There used to be a green one here, but they're isn't green anymore.
Blue is not ideal, I won't use the blue.
Let's try this.
Is that better?
OK.
So let me use this orange one, and I'll try to write large, even though it's not as bold.
OK, so the use of color.
That's important in both slides and blackboard.
What's the use of color here?
Is it just because it's nice to have a pretty picture?
No.
The reason for using the color is that you get layering.
So again, remember the students' dilemma.
Pretty much all students' dilemma come from the following problem, which is that their chunk size is one symbol.
You've been teaching the Navier-Stokes equation for 20 years, your chunk size is the entire equation.
Right?
So that's one chunk to you, you just write it down, like I just did.
But for the student, every single, little thing is one symbol.
So now, if you then write all of this stuff in the same color, as your rest of your equation, it just becomes more stuff.
They don't know how to separate it into two different objects.
But, if you write your labels in a different color, it pops out in a different layer.
They know that lives in a different space, so you're already helping them chunk.
So what is this?
This is also a change in v. So it's sort of like an acceleration.
Well, it's part of the acceleration, actually.
So this is another acceleration.
It's a change in v, but it's a change in v because you've moved to a new place.
So this is a change in v because you waited-- you move to a new time.
This is a change in v because a little piece of fluid moved to new place, where the fluid velocity is different.
So these two things together are two different contributions to the change in v. So we'll actually call this whole thing D of vdt, which is actually its actual name.
So this is an acceleration.
These are the two pieces of it-- this is the acceleration.
What's on the other side?
Well, this guy, pressure-- p is pressure-- grading of pressure.
So that's the differences in pressure.
So that's going to do some kind of thing, like a force.
And row is density.
So you're taking a force-- I'll put it in quotes, because it's not quite a force, it's sort of force per area.
But now you divide by density, and you get f over m. So f over m. So we have, here this guy-- acceleration is equal to f over m. Do you recognize that equation before?
Yes, Newton's Second Law.
So this whole thing, so far here, is just this is a equals f over m. So this is the pressure forces, and this must be the-- and this is the viscous forces divided by the mass.
So these are the two contributors to the forces-- pressure forces, viscous forces-- and those produce the acceleration.
So the Navier-Stokes equation comes out just as Newton's Second Law all over again.
So now, this kind of drawing is quite difficult to do in the slide.
Yes
So I actually would argue with you.
You could prepare a set of slides with annotations, and then you could do everything.
You could put all the information [INAUDIBLE] in a slide, or in multiple slides.
[INAUDIBLE] animation
Yeah.
So you could put them on multiple slides, and then flip to it, one by one That's true.
So you can do it, which is why I said you can do it, but it's hard.
Because it is a bit of a pain, and you can do it.
OK but then the problem is, you want to now leave that on.
So now you want to have not only this, but you also want to have this, and leave it on for the entire lecture.
Because this is the meaning of the terms.
And now you're going to leave it there, and talk about each term, one by one.
Yes
But you could also give them a handout, with all the slides there, so they would have everything in front of them.
And if they ever want to go back, they can look back on the previous slide.
[INAUDIBLE] but I think so far this is not a strong argument against the slides.
Maybe the question's about interactivity.
If somebody asks you a question, what about the third term?
If you don't have a slide for that, then--
Right.
And this is the point I'm coming to.
So there's that.
And I agree with you, you can do this with slides, you can give people a handout.
Though the problem with the handout-- suppose you put all of this on the handout for them, so that now, when you erase this screen, and that screen, you go to another thing they still have something to refer back to you.
The problem is that you want it all in one visual field.
And you don't have that when it's on a handout.
So you have a problem of split attention.
So now the student's attention is switching between the screen, and the handout-- screen and handout.
And when they look at the screen, they're trying to remember what they're confused about.
And now they look at the handout.
This is the analogy.
When I was in Prague, I would look at a street sign, and I would read the word, and look at all the letters.
[INAUDIBLE] or something like that.
And I would look away.
Two seconds later, I could not remember more than one letter that I saw on the street sign-- on the street name.
The reason is, because each of the letters was random to me.
So, yes, someone who's Czech, just says, oh yeah, that's Main Street, of course.
No big deal.
What's there to remember?
It's Main Street, it's in the Main Square.
Whereas me, as the foreigner, has no clue.
And it's all new data to me.
So it's the same way for the students.
This is all new to them.
So when they're looking at something else, you're talking about-- you're going to discuss, what's the dimensions of this term?
They look at this thing, and now you have a big analysis of the dimensions of this, you want to check that all the dimensions are right, and your goal is eventually to work out the dimensions of this-- viscosity, let's say.
They might be frightened of the partial.
So they look at this, and they say, oh, yeah, where does the Dvdt come from?
And then they look back at their handout, and they've already forgotten pretty much what this was.
Because they are the foreigner.
They are the tourist in this land
Yes.
I was just going say that I would have said something very similar.
The annotation part of it is to [INAUDIBLE] slides at all, because if I was making slides for this lecture, I would've done exactly what you did.
But it would all be nicely spaced, and there would be no handwriting issues.
So if anything, slides would be better for that.
But I think that the argument for slides is [INAUDIBLE] having them linger on, but also pacing.
Like, the fact that it doesn't flip up so quickly, because the [INAUDIBLE] that I've taken where they use slides and use [INAUDIBLE], it goes by so quickly that you have no chance to internalize.
The fact that you have to write it out by hand forces you to slow down, in a way that is really helpful for the listener.
And so that's something that's really important [INAUDIBLE] difference
Right.
And that's a fundamentally important point about pacing.
And this partly addresses your question about how big should you write.
So one of the reasons for writing big, is that you're writing slower.
And writing slower, generally forces your pace to match the pace that is absorbable by the students.
So I'll come back to that point in just one second.
So let me just finish the point about the visual field.
The good thing about the blackboard is, you could call it cognitive offloading.
In one glance, everything that the students need, can be there.
So they don't really have to remember much.
And the short-term memory isn't overfull.
They're registers.
They're a low-register machine.
Or rather, they have the same number of registers as us, but they don't store as much information in their registers.
So their registers don't get overfull, because everything's right there.
Whereas there is a much bigger danger of register overfilling with slides.
Then, as you point out, there's the issue of interactivity.
So this is partly a performance question.
Slides, not only are they much more pre-prepared-- I mean, you can prepare your blackboards, as well, but you actually do have to re-perform it each time.
Whereas a slide, as you say, you just click to Return, or Next, and there you are, at the next slide.
So there's much less performance in it.
And it seems much more pre-scripted.
Now, what does that signal to the students?
The pre-scripting.
It says to the students, look, this is all one, tight thing.
It's sort of like a play on a stage, that you're only supposed to applaud at particular times.
A modern-day play.
Back in Elizabethan times, in Shakespeare's time, people cheered, and screamed, and cat-called, and it was a much more lively thing on the stage.
But now, everyone sits really quiet, God help you if you cough, you try to wait, and you think about your coughing, and then you don't think about the play, and you try to prevent your coughing.
So all that transfers a bit.
That same mentality comes when you're using slides.
People think, oh my God, this is a pre-scripted thing-- how dare I intervene, and break into that.
So if your goal is to encourage students to question, it's harder to do that with slides.
You still can do it.
You can, for example, put questions on the slides-- and you definitely should do that you're going to use slides-- but the random, spontaneous character of your session-- the good side of that, is going to be minimized by the slides.
So this one was just to continue saying, OK, now we're going to leave these two on, and we're going to do the dimensions of each term.
I have one, two, three, four, let's say-- four terms I want to talk about.
I'll do the dimensions of two terms on this board, dimensions of two terms on that board, now I'll have all the dimensions there.
And now I can start, for example, doing dimensional analysis on this side of the room.
And we'll eventually get to the Reynolds number, which is vr over nu.
Once we know this guy's dimensions.
And v, we know his dimensions, r the size of your sphere or something.
So this is dimensionless.
Now you've got the Reynolds number, you've also got, say, drag coefficient, which is f over 1/2 [INAUDIBLE] v squared times some area.
So this is also dimensionless.
So now, you have two dimensionless things.
And then finally, the finale is-- So now, let's say by the end of the lecture, you get to the point where you've plotted the drag coefficient, a dimensionless thing, versus the Reynolds number, on a log-log scale.
And then you can talk about this, you can go back and you say, OK, well what is this?
This is really interesting.
It looks like it was constant.
And then what happened?
Well, you can go back and talk about that.
And as you talk about it, the entire graph is visible to everyone, as well as the source of all the pieces of the graph, which is all the-- why is everything dimensionless?
Well, you have it on these two boards.
And what are the terms that you used?
Well, those are all on there.
So all of that is visible in one glance.
So that's an example of something that is hard to do in the slides.
Now, it's not easy to get it right on the blackboard, either.
You do have to plan.
You have to plan by saying, what am I going to put on each board?
And how am I going to structure the boards?
But you can do that on your lesson plan, for example, on the back of the lesson plan.
Say, OK, how many boards do I have?
And what am I going to use?
And then, if you want to take advantage of the fact that everything is visible all the time, you generally don't want to use the up-and-down nature of the board.
You just want to use two boards on each of the regions.
If you push this up, and start writing on this guy, you're now OK, still.
But then, when you push this up, and you write on here, you've covered this guy up.
And yes, you've lost some of that advantage.
Again, you can plan it out, and do a trade off.
But that is harder to do with slides.
Now, your question about how big you should write so that everyone can see, versus telling the whole story.
I think it is important to try to tell the whole story.
It doesn't have to be the entire lecture sits on the set of blackboards, but a coherent chunk of the lecture should sit on the blackboards.
You have two things you could control.
One you can control is the writing size, and the other is what you write down.
So this is a similarity between slides and blackboards, which is you don't want to write everything down.
Because if you write everything down, then you say, well for example, did I write every single thing I said here?
No, because I want the main ideas to come across.
Every little detail that I say, that should be either in the notes, or in some book.
But the main point should be on the blackboard at a level of detail that people can absorb.
It's the same thing with slides.
You don't want to put everything on the slides.
So, you can still write big, if you just cut down how much you write.
And generally, that is more suited to what students need.
Because, again, they are living in the dirt of what is this, and what is this.
And you want to try to bring them up higher.
So you don't want to spend a whole bunch of time on all these little guys.
You want to try to bring them up higher, and put some high-level chunks onto the little bits that they have.
So use a big chalk, write, large, just don't write everything.
And try to fit a coherent piece onto all the blackboards.
It doesn't have to be a whole lecture.
Although Feynman, at Caltech, when he gave the Feynman Lectures-- well, what later became the Feynman Lectures-- he did actually plan it so that he would start at one corner, and he would end all the way here.
And I find that's too hard to do if you're also going to make the lecture at all interactive.
For example, if you're going to have questions where you're going to have the students give answers, and you're going to write down the answers.
Well, you're not necessarily going to keep all the answers for the rest of the class.
So the goal of corner-to-corner, I think, is a bit too strict.
But the general idea is sound.
Any questions about that before we take a break, and then go onto slides, and slide design?
Yes
I just wanted to comment that also, a lot of students, if there are slides, and they're available online, they will just print them out, and not come to lectures
That's true.
A lot of students will just print the lecture slides, and then just think they can thumb through them, and understand the lecture.
So that's partly-- you could call it the optimism of youth.
That they don't realize how much they're actually not seeing, because they didn't actually come.
They think if they just follow each slide, they've understood everything.
But there's no way to put everything in a slide.
If you packed a slide with everything you said that was worth remembering, or that was actually part of the structure of the argument, then it would become not a slide, but a book chapter.
And so, yeah.
I think there is a reasonable argument that if there's a book chapter-- maybe they don't want to come to lecture if all you're doing is reading the book chapter.
But slide, and lecture, are so different.
It's sort of like, if you just take a few photographs of the blackboard, could you replace the lecture?
No, but you're right, students do think that.
So actually I don't like giving them handouts of the slides, if I do happen to use slides.
Because it's really not a substitute for a lecture.
It's an aid to the lecture, just like the blackboard is an aide to the lecture.
But what's important about the lecture is, what are you having them do in the lecture?
And that's the reason they should come into lecture.
Because in lecture, they're actually struggling with some stuff, and you've structured it so that they struggle, and learn.
And somehow you have to convince them that that's the value.
And it's hard, because of what you say.
Other comments or questions?
One second, in the back
[INAUDIBLE] But at the same time, I've probably taken at least half a dozen classes where every single word that the lecturer spoke, was present on the slide.
And then it felt kind of worthless going, because you show up, you pick up the slide handouts, and you can read the slides during the lecture.
You pay no attention to the lecture, because he's not saying anything that isn't there
Right.
So that's sort of the worst of both worlds.
So now, what they've done, is they've taken the constraints of the slide format-- because there's only so many words you can put on the slide, and make it even at all readable-- and often they don't make it readable, and I'll show you an example when we come back.
And so then they've torqued the lecture to be basically reading out the slide.
So now they messed up the lecture, and the slide still isn't really a good substitute.
It's a huge problem.
I think the answer to that, is that if you're going to use slides, they're really an aide, but the students really need to learn how to read a book, or your notes.
Or something that's actually a coherent presentation of the material.
Yes
So when you write something on the board to respond to a comment, and it [INAUDIBLE] some of your space that you could otherwise use to plan for the usual stuff.
So when do you erase that comment?
That's a good question.
So how temporary should stuff be?
One way to do it is to think about the number of minutes each thing deserves.
So that-- I like things like that to be on the board the whole time, if possible.
And now it may be that, halfway through, I might reclaim that board if I don't have a choice.
But I'd like that to be there for a while, at least, because that's really the main reason we're doing all this stuff.
And what are we doing.
So ideally, I'd put that on the side blackboard, and just leave it there forever.
Something like this, I would like it to stay there the whole time, so we remember.
So this, I would mark as, "don't erase," if at all possible.
So when I'm writing stuff down, I would know, how perishable is the material?
That's the word I'm looking for.
So suppose I ask people a question, like the wood blocks.
OK, what's the frequency going to be?
And we get a bunch of reasons.
I think the reasons are quite perishable.
After we write them down, and we talk about them, and then we reconcile everything, I'm perfectly happy to erase all the reasons, and just have the reconciled picture on the board.
But while I'm reconciling, I want them all visible.
So then, what I would do, is I would try to have all those reasons go, say, here.
I might cover that up temporarily.
What I would do, is I would write the reasons here, and then do the reconciliation here.
And then-- so let's call this Reconciliation-- so this is actually the picture that I would use for the wood blocks, which is this.
And I would draw a bunch of arrows, and stuff, dot, dot, dots, and spring [?
bogs ?]
OK. And now, I need to reclaim this board with perishable material on it.
So now I reclaim the board.
So this had reasons before.
Lots of reasons.
OK, and then, reclaim this.
And I still have that reconciliation up there, and I have this board back
So you think it is a good idea to separate the perishable things on one board, and then the more permanent things on the other board?
Yes.
I think it is a good idea.
That's a good point.
I hadn't thought of it that way, but that is a good idea.
Because that way you can reclaim the entire board back, and then replace it with more permanent things.
Or, if something perishable is going to go back on it, put it here.
And then, when it's done, to erase it, until a permanent thing goes on there.
And then you move onto the next board.
So it does require some board planning.
But what you find, basically, is the first time you teach the course, you wish you'd planned it more in the first lecture.
But then you start to be more automatic about it.
Yes
I have a comment on the performance aspect of the board.
So when you were writing these equations, and then as you went back in the vector symbols, and also when you're circling things, because if you're doing it right, the students are engaged.
Their focus is [INAUDIBLE] chalk
That's a good point
It's as much of an interactive activity as they can with you writing on the board.
Whereas with a slide, certainly [INAUDIBLE] a physical pointer, you might circle something, but then there's nothing there.
A laser pointer's even worse, because you're not actually touching anything.
Even if you have an animation drawing, you're still separate from it.
So I think that's a big advantage of the board.
If their attention is at your chalk tip, then that's some degree of engagement with that action
Yeah that's a really good point.
So the point is that when I'm, for example, circling this, it directs people's attention, and then there's a permanent trace left.
So you can, kind of, do it on a slide.
If you knew you were going to do that ahead of time, you can make an animation that goes, circle.
So you can do that, but it's hard.
So one of the cures for that is this tablet, a kind of PC.
So you can actually write on the tablet as you go
Yeah.
But even so, the thing is that you're still over there [INAUDIBLE]
I know
[INAUDIBLE]
I think that's true.
That, actually, we are together, doing this, on the object
Yeah that's exactly what I was not saying very well earlier, was you can do all this, what you're saying, on a slide, just as well.
When I do [INAUDIBLE] presentations, I do actually animate the circle.
I find that that's the most intuitive way for me to do something.
But it's not the same as actually drawing it yourself.
It's not the same level of activity.
And obviously you kind of have to use slides for [INAUDIBLE], but in a classroom you don't have to
Right.
OK, so that's what we'll talk about when we come back-- the difference being technical presentation, and teaching.
And then we'll actually look at an example of slides, and we'll critique them, and I'll show you how to redraw them.
OK so break for 10 minutes.
We'll start at 10:18 sharp, by that clock.
So you can set your clocks accordingly.
And I will erase all this, and put up a slide that we're going to critique.
And then we'll talk about technical presentations versus teaching.
Because there's a lot of misconceptions that come from technical presentations.
And therefore, people think we should use slides for teaching.
OK, so see in 10 minutes, or if people have questions during the break.
OK, so our next task is to figure out, if you are going to use slides, what should you do.
And related to that point is, when are slides appropriate.
People have the idea the slides are always appropriate for teaching, because they see them used all the time for seminars.
So if you go, and give a talk at a conference, you go to a conference, most-- almost all the talks-- or at the department seminars, basically people now come with some form of slides.
When I was an undergraduate, it was sort of transitional-- people came with overhead projector foils.
And it was sort of shifting, from seminars used to be blackboard seminars, and then they shifted to overhead projectors, at least in physics.
And now it's pretty much slides all the way through.
But there's a fundamental difference between seminar presentation, and teaching.
And seminar presentation-- basically you're talking to people who are sort of experts.
They may not be experts in that particular area you've understood.
Probably not, otherwise they would have invited one of them to talk, instead of you.
So now, you're the expert locally, on that local area.
But in the broader area, you're talking to basically people who are already interested in, and are professionals in the field.
And maybe some apprentices in the field, who are going to soon be professionals, like graduate students.
So your goal is not to slowly, necessarily, uncover one or two ideas, especially if it's a 10-minute conference presentation.
Your goal is something like, here are the core, main ideas-- one or two points-- and here's something to show you that really interesting stuff is going on.
And you should pay more attention to me is part of the goal of the seminar.
Now that's very different in teaching.
In teaching, your goal really is to kindle some new thoughts in the student.
And who are very different from a seminar audience.
They are not professionals.
Maybe they would like to eventually be, but part of your job is to kindle that interest so they would like to become professionals.
So you have a very, very different audience, and generally much different time constraint.
In a seminar, maybe it's 10 minutes, maybe 15 minutes.
Rarely, once you start getting well known, maybe you get invited to give a 50-minute seminar.
But, teaching, you generally will have 50 minutes.
So your time is different, your audience is different.
So just because slides are used so much for seminars, does not mean they should be used so much for teaching.
You should really use them when you don't really have a choice, and when it seems like it's the optimal thing to do.
So for example, art history class.
Now, an art history class, it would be fantastic to go to Sienna, and look at all the tapestries, and take the whole class with you.
Now that's just too expensive.
So, instead, you show slides.
That's exactly why they're called slides-- they originally were 35 millimeter slides.
35 millimeter?
Yeah, 35 millimeter slides.
So for something like that, yes, slides are ideal.
And drawing the tapestries of Sienna on the blackboard is not even close to showing the actual pictures.
So, in that case, 100% go with slides.
Other cases generally go towards blackboards, but there are cases where, for example, there are no blackboards in the room, and you have to teach with slides.
There's no whiteboard, either.
Whiteboards are not as good as blackboards, but they're a reasonable substitute.
But there's neither, and it's just slides.
Or, yes--
Why aren't whiteboards as good as blackboards?
I find the markers always dry out, whereas chalk always works.
So blackboard markers never quite work as well.
And the other problem, just me personally, is that I'm sensitive to the chemicals in them.
So I eventually just get a bit dizzy using them.
So I don't like him for that reason.
And there's quite a few people.
I'm sort of the canary, so I'm more sensitive, maybe, than many people, because I have lots of allergies.
But I think lots of people are bit sensitive to them.
But, generally, I find the markers just don't work as well.
And, also, there's very rare room has lots of whiteboards.
Whereas many, many, classrooms have this many blackboards.
When they replace them with whiteboards, usually it'll be one whiteboard.
So then you're back to many of the disadvantages of slides, which is that you can't get a big field of view.
But now let's say you are going to use slides.
How do we make good slides?
Well, one way to figure out what constitutes good slides, is to look at what constitutes bad slides.
So this, here, is a horrid slide.
So what we're going to do is, I'm going to show you how to re-write the slide.
But first, let's figure out what are some of the problems with this slide.
So, take a couple minutes with your neighbors.
You can all see the slide.
What are the things that just don't work about it?
And there's many.
It's a very competitive field.
And I'll write some stuff down on here.
By the way, one thing is that the JIT, that I just actually shortened.
It actually was, "Just in Time Learning".
I've just shortened it to improve the typesetting, assuming that the audience knows what it means.
So the original slide didn't have that problem.
So I see you found lots of trouble.
Give me one thing that's wrong.
Adrian
There's no message
What do you mean by message?
What is the person trying to tell you here.
You have to infer what the person is trying to tell you by analysing it yourself
OK, no message.
So, the reader, the viewer, the student-- this was actually used in a class, never mind where, but it's not here.
Yeah, what is the teacher trying to communicate?
So Jean-Luc Doumont, if you saw him during IAP-- he has a great way of talking about what's a message, versus what's information.
So the message-- so information, that's what that slide has.
It has lots of information.
That's the "what."
Message is the "so what."
His native language is not English.
I wish I could come up with things like that in other languages.
I think it's fantastic, because there's lots of "what."
Well, 12% said Just in Time Learning-- JIT Learning-- was a bad idea.
But your question is, so what?
And there's no answer to that question.
So there's no message.
Yes
So a counterpoint to that would be, this is documentation, this isn't propaganda.
Right?
You don't want to say, this is what you should all think about this data, instead of, look at the data yourself, and make your own conclusion.
Presenting the data in an accessible manner is a different issue.
But I don't know if I agree with this, "so what" idea
And some of that is that, yeah, definitely in a seminar, you lean much more towards the "so what."
And you would put in the "so what," really first.
And in teaching, maybe you start with the "what," and then you lead to the "so what."
So, for example, the wood blocks.
Here's the "what--" what do you think is going to happen?
And I haven't said, "so what," really.
And then we talk about a whole bunch of ways of analyzing it, and then we finally lead up to the "so what.".
But at some point, you do want a "so what."
So, yeah, without seeing the entire lecture, it's hard to know if they ever gave a "so what."
But I can tell you that the succeeding slides just went on to other topics.
So it wasn't that they were using that to develop the point-- what do you think about Just in Time Learning, or how should we analyze Just in Time Learning.
They just moved on.
Yes
Going back to your art history example as a good use of slides, that doesn't have anything in it that gives you "so what."
It gives the presentation of information, and then the discussion of it is what gives that.
So I don't necessarily think there's something inherently wrong with giving information on a slide, and having not answered the discussion in the slide
It's not inherently wrong, and that's a good point.
So the art history one, it's sort of a different use of the slide.
So there, it's really like you're bringing in a prop.
So the cones, these pyramids themselves, didn't have a "so what."
They only got a "so what" as we used them.
So it's the same way with the art history slide.
It would get its "so what" in how you discussed the slide.
But here, the slide is really the alternative to the lecture-- to the blackboard.
If you've gone to slide talks, or slide teaching, which aren't art history.
I think art history is really a separate example.
It's, I'm telling you stuff, I'm telling you stuff, I'm just telling you in a way that you don't really know what I'm telling.
Like, I may be telling it to you verbally, but the slide doesn't match what I'm telling you.
Sorry, you're next.
Go ahead first
You may have already said this, but is this slide supposed to be for the students, or is this for a faculty meeting?
No, this was for a course about-- what was the course about?
It was a computer science course somewhere.
Maybe it was a discussion of a-- no, it was a computer science course, or a teaching course.
I forget exactly what course it was, but it was from a particular course.
So it's a slide used in teaching.
Yes
That partially answers what I was going to say.
This material doesn't belong in the lecture in any course, unless it's a course on teaching.
The surveys are filled out, so they benefitted the lecturer so that they know what to do when planning and running the course.
The students don't need to know what the results are
I think it wasn't for the course itself, it was for the students in a teaching course, is what I think it was for.
And either way, whether it's for a seminar, or for a teaching course, it has the same problem, which is you don't even know what to think about it.
You just have a lot of data.
And there's no integration.
OK, so that's the no message-- the "what," versus "so what."
What else is wrong?
Yeah, Wendy?
The data is just terribly organized.
[INAUDIBLE] One thing is that there's bullets under bullets.
By the time I get to [INAUDIBLE], I'd rather skip, I've forgotten [INAUDIBLE], I don't remember what we were talking about
Right, so the organization is ghastly
And another good comment, which is that the numbers of the percentages are not organized in any way.
They're not going from highest to lowest, or lowest to highest.
So you've seen, maybe 52, you're like, I assume this is going in descending order, and then things get flipped around
Right.
So the percentages are kind of random.
It's sort of like the "what."
To organize it towards the "so what," you would put some theme in it.
But there isn't any, or none that's obvious.
And the typesetting, yes, is related to the organization.
The typesetting is a bit off.
Yeah
So related to that, [INAUDIBLE] the numbers and the questions are given completely equal footing, because the numbers are organized, [INAUDIBLE].
And so the numbers just follow better to worse.
I'm sorry, the answers to the questions are organized from better to worse.
[INAUDIBLE]
OK, so then it comes out
And then it comes out, but that's not [INAUDIBLE]
So the Likert scale isn't obvious.
The Likert scale is the better-to-worse scale of how much you agree with this statement.
Right.
So you're saying, also, the answers have equal prominence to the questions
Equal prominence to the percentages.
You have no clue what's organizing the information
Right OK.
So the emphasis is random.
Yes
So [INAUDIBLE] so she suggested [INAUDIBLE] or [INAUDIBLE]
OK, so it's not visual.
A pie chart, or a histogram, or something would help you see much more.
So that's one of our principles, right, that your perceptual system is so much smarter than your symbolic processing system.
So you're actually slowing everyone down, and making them much dumber, by forcing them to try to extract-- basically build a mental picture which you could build for them.
Yes
I'd say there's too much
Too much.
So what do you think is too much?
All at once, you're getting lots of text, and it's all these different things.
Too much [?
given ?]
Yeah, it's too many chunks.
It just overloads you.
You don't relate it to the lack of messages.
If there are messages in there, there are too many messages.
You don't know what's important, and what to pay attention to.
Other problems?
Yes
Format switches midstream
Where?
At the bottom, the last question
Oh yeah.
You're right.
OK So the format
[INAUDIBLE]
Right.
It's sort of related to too much.
Right, I tried to put too much on my slide, and I really tried, really hard.
So the format changes at the end, right, so they could fit everything onto one slide.
Yes
Extraneous use of bullets, colons, [INAUDIBLE].
Not helping
So a random use of colons and bullets.
And it's not clear how those actually help.
Other problems.
Yeah
Is that all bold?
Yeah.
I think it is all bold.
So everything's equally emphasized.
So all bold.
So again, it's presenting all the information, right?
It's just you don't know what's important-- what's the "so what."
Yep
The numbers are very far away from the text, so by the time you get over there, you don't know
Right, it's sort of like a table of contents.
And that's OK, for a table of contents.
It's not ideal, but there are ways of doing table of contents that don't have that problem.
For a table of contents it's OK. You don't need to sort of get the entire table at a glance, right?
You just want to know where chapter five begins, and you just scan across, and you go there.
Whereas here, you can't match the things.
You want to try to match it all together, and you can't, because the numbers are so far away from the text
I don't know what PI means
Sorry, that was my fault, too.
I just tried to shift it.
Peer instruction.
Yeah, you're right, so there's a lot of use of acronyms.
Yeah, so the JIT, and the PI were my contributions to it.
So maybe I made it a little bit worse.
Yeah
So how would you do this on the blackboard, instead, if you had a chance?
So this one, actually, if I were going to present this, I would either draw the thing I'm about to show you, or I would use a slide for it.
Either way
A slide with what?
I'll show you what the redrawn slide is.
OK, so now let's redraw the slide.
So I think the slide that I redrew-- I'll show it to you-- basically answers most of these objections, though maybe not all.
I think most of them.
So what I'm going to show you is what Michael Alley calls the Assertion-Evidence design for slides.
So Michael Alley, I'll write his name down.
So Michael Alley, and his book is called The Craft of Scientific Presentation, I think.
And this is to get around many of these problems.
So the idea is, first of all, so related to what Jean-luc says, talking about messages versus information, each slide should have one message-- one assertion.
So that's the assertion part of the slide.
And the body of the slide is the evidence.
And it should ideally be visual evidence, because of all the reasons we talked about for visual evidence.
So here is an example of that.
There.
Is this the entire slide?
No.
This is just one, basically, third of it.
But you can redraw the rest of the slide, with two more slides, this way.
So this is one slide, with one message-- most students like in-class concept tests.
If that's the point you're trying to make.
The problem is, it's hard to redraw slides with no messages.
Because you don't actually know what the message was, so you have to sort of infer it.
But hopefully the author could actually infer it, and redraw it themself.
So here you have to play author, and I guessed the message-- most students like in-class concept tests.
The presentation is here, with a pie chart.
And you notice all of the tags, the "I like them", the "please skip them," "I dislike them."
They're all right next to their percentages.
And then, what's color used for?
There, color is used to emphasize the like.
So the green, and the light green, are both categories of like.
As asserted by the author.
I mean, you could say, well, no "enjoy and learn from them" is the only category that matters, "like" isn't strong enough, or really you shouldn't lump them together.
You can have those debates.
But what the author-- the teacher-- is trying to say, is that most students like them.
And you can see it just at a glance, because most of the pie chart is colored in.
So this solves most of the problems.
First of all, you don't have the ghastly typesetting that you have here.
The typesetting here is just horrendous, this is sort of typical PowerPoint typesetting.
The alignment is nonexistent.
There's random alignment lines everywhere.
If you look, for example, at the way top, the yellow bar doesn't align with anything else.
The left edge of it just doesn't align with the bullets.
The open bullets are just random.
It's just a big mess, as you pointed out.
So here, just get rid of all that.
And you can do this with any program.
You can make your figure however you want.
You can put one assertion at the top.
So now assertion, another synonym for that is "sentence headlines."
So you want your title of your slide to be a sentence.
If you're not writing a sentence, you're probably writing a topic.
You're just mentioning something-- you're giving them "what."
To give "so what," you really need a sentence.
So fit a sentence of one or two lines there, and then, give visual evidence.
Yes
To sort of play devil's advocate, I mean, I agree [INAUDIBLE], but the advantage of having all the information on one slide is you can see the different things, and see the different conclusions
If you could actually see them.
The problem is, you can't actually see all the conclusions.
I agree it's nice to see all the conclusions at once.
So to do that, what I would do is, I would make-- I think there are really three messages here-- one for each of the questions.
So I made a slide for the first one, which is about the concept test.
You'd make another slide for the Just in Time learning, another one for, do lectures cover too much material.
So now you have three.
And now, basically you want an overall message of, overall these instructional changes were well liked.
Then you could have one summary slide, which just lists those three points.
And with maybe three small pie charts, to remind people of each pie chart, to fit it all together.
But to try to pull it out of that, the people won't actually get the three messages.
So you have to guide them towards it.
Hopefully you can use this for your seminars as well, not just for your teaching.
This works for teaching, as well seminars.
Now, how would you actually put this into practice, in a teaching example?
So that's what comes up next.
I'll show you a mathematical example, which you could do on the blackboard, or I sort of optimized it so you could do it on slides too.
And we can see how that goes.
OK, so here is just the comparison, side by side, of the two slides.
Just at a glance.
I don't actually mean for people to be able to read the words, just to see that one has enough information that you can get a point, but doesn't overload you, and the other is just a rat's nest.
OK, so then the mathematical example-- actually, a statistical mechanics example as well-- is the log of n factorial.
So we're going to approximate the log of n factorial using pictures.
Now why n factorial?
Well, what is n factorial?
5 factorial is 5 times 4 times 3 times 2 times 1.
Why do we care about it?
It's the most important function in statistical mechanics when you're counting objects.
And it shows up in probability theory all the time.
And statistical mechanics has a lot of particles, so the n is big.
So we'd like to approximate how big n factorial is, because it's so big we take the logarithm.
OK, let's see if we can approximate the logarithm using pictures.
So log of n factorial is the area of those rectangles.
You can see rectangle for log 2, log 3, log 4, log 5, log 6, and log 7.
And log of n factorial is just the sum of all of those.
Say, 3 factorial is 3 times 2 times 1, so log of 3 factorial is log of 3, plus log of 2, plus log of 1.
Log of 1 is 0, so all you have left is those rectangles to add up.
OK, so now let's add them up, approximately.
So the area under the curve is your first approximation.
So the curve is just log k. So let's see what that is.
Well, OK, that's just the integral of log k from 1 to n, from the lower limit to the upper limit.
So that we can just do symbolically, you get n log n minus n plus 1.
OK, now the error.
Where does the error come from?
Well, it comes from the protrusions beyond the log n curve.
So there you have the protrusions.
so we'd like to add up all those protrusions.
Well, how are we going to do that?
Each piece is almost a triangle.
So let's just straighten out each little piece of log k curve, and make triangles.
And now, it turns out they're much easier to add up if you double them, and make every triangle into a rectangle.
OK, so now I'm going to stop here, and see if you can figure out how to add up all of those pieces.
What's the sum of all of those shaded regions?
Take a minute and check with your neighbor.
OK, when you see it, raise your hand.
Find someone around who sees it, if you haven't seen it, and check with them.
Because it is fun to see.
I don't want to spoil it unnecessarily.
Here's a little hint.
So what you do, is you hold your hand at the right page, and you whack all of the rectangles, and they slide across, and go psssssh, and stack, and form the last one.
So that means the sum of all the corrections, times 2, is the last rectangle.
So log 7, or log n, is twice your error, roughly.
So now you just combine the integral, which is the piece under the curve, with the approximated protrusion, and you get that for log of n factorial.
So let's see.
So that is actually very close to Stirling's formula.
Let's see how close it is.
Well where's the error?
Well, the error comes from when we straightened out the pieces of the log k curve, and made triangles instead of the funny-shaped region.
So the error is those shaded guys, which is not very big.
And most of the error happens low down, so if you just corrected for that, pretty much it would all go So here is the total error.
Here's an example.
The picture method, so we get 7 factorial, gives you 8.594.
The exact answer for the logs of 7 factorial, 8.525.
So it's an error only 0.07.
So all of this just adds up to 0.07.
Which ends up being a 7% error in seven factorial.
So the moral of the story is that pictures can help you approximate log of n factorial, and here is your approximation.
So one of the most important functions in statistical mechanics.
95% with pictures, and one tiny integral.
OK, now.
That's an example or something you could do if you wanted to teach it using slides.
You could easily do that on the blackboard, and normally I teach it on the blackboard.
But I made this as an exercise for myself to say, OK, well suppose I have to teach it with slides, what would I do?
And you notice here.
So when I first drafted it, this slide was missing.
And I went from here to there directly.
And then I thought, oh, actually-- so this is about the interactivity question-- oh, actually I'm depriving people of a chance to think about something, and realize it.
On the board, it's really easy to pause.
You just put the picture up, and you wait.
And you say, OK, what do you think?
And then you start drawing stuff.
So I thought, OK, well you can actually simulate that on slides, too.
So I put the intervening slide in there, so that people had a chance to stop here, and think about the result before we continued on to the solution, where it adds up to the last guy.
So that's an example of basically somewhat interactive slide teaching of a mathematical idea.
Yes
So why didn't you animate a slide [INAUDIBLE] with boxes to take advantage?
That's a good question.
Why didn't I?
Actually, because, A, I didn't think of it, but it's a good idea.
I should have probably done it.
The problem is that I'm showing it as a PDF file, rather than through any program.
So PDF animations-- I don't know if PDF has animations in it.
It probably doesn't
[INAUDIBLE]
Yes, that's true, it's sort of against the purpose of PDF, which is a stable format.
Yes
I actually thought this entire presentation was really clear, and would be just as good as if you did it on a blackboard.
Because it was interactive, slow, had pacing, the suspense of having us ask the questions.
So I think one of the major reasons why I would be opposed [?
to teaching ?]
in this way, is just because it takes you more time to create the slide.
From my own experience, it takes a long time to make really good slides, just fiddling around with [INAUDIBLE].
So I think if you could spend all the energy we're spending on slides instead on [INAUDIBLE]--
That's a good point
--or whatever, I think that might be [INAUDIBLE]
I think that is a really good point.
And I hadn't thought of it until you mentioned it, but you're right.
It took me a long time to make the slides right.
To do slides well, you have to really think about the visual presentation, make everything visual, you're making a lot of diagrams.
There's all kinds of little tweaks.
Like, I made sure that all the diagrams are exactly the same size, so that when you flip from slide to slide, you don't get a shift.
So all those things, you don't have to worry about on the chalkboard.
And yes, I would advise people, just as you do, to spend that time thinking about your teaching, and where you're going to put the interactivity in.
And do it on the blackboard.
So this is something where, with a lot of effort, the slide reaches to the level of the blackboard.
But with colored chalk, and lots of blackboard, the blackboard is just as good, and quicker.
And the blackboard has the advantage that this could be two blackboards in your class.
Say it's a class about visual representations, and you have two blackboards for this thing.
The middle two blackboards for yet another example, the end two blackboards for yet another example.
And you have all of them at once.
Yes
Last time I taught a class where I used all slides, and when I put the amount of time that, things like you put into this, to make them all so they perfectly line up, and the animations were really helpful and everything, just creating the slides probably took half the overall time that I put into the class.
But I think the one advantage was I was able to use printouts of those handouts as class notes.
Which I handed out after the class.
They didn't have them during class.
So that's the only advantage, is you can reuse all of these [INAUDIBLE]
And that's exactly true.
So all these nice figures, we did it for this class, and for a seminar on making slides.
And then I used them verbatim in my Street-Fighting Mathematics textbook.
So basically, to make good slides requires making something pretty much publication quality.
And then, you do have something publication quality, but maybe the first time you teach it, that's not where you want to be spending your time.
There was one other point that was raised in an email beforehand, which is, in Jean-luc's paper, he mentioned that he uses less popular alternatives to the Microsoft software for making slides.
So, yeah, I highly do not recommend using Microsoft software, because it's not free software.
And it's, I think, unethical to use non-free software.
Free software meaning freely licensed, so you change the source code, view the source code, make your own versions, do whatever you want.
And it's none of that.
Second, it's all proprietary formats.
So you're encouraging other people to use proprietary formats.
No one has fully debugged, and decoded, all the formats that Microsoft uses.
So you should use open formats at least.
Now, you could use Open Office, but the problem is Open Office, the whole model of the way it does slides is kind of yuck, because it's just copied from the Microsoft way.
So actually I prefer-- maybe it's because of my computer science math background-- what I prefer is to program all the slides.
So let's see.
[?
Log.tech.
?]
Where are the-- yeah.
OK.
So I actually write in [?
Tech, ?]
or version of [?
Tech.
?]
And here is your title of the slide.
And I can change the formatting later
Maybe that's what took you so long
Maybe.
Maybe.
But the thing is, you can't make publication-quality stuff otherwise.
So if you're going to actually do a good slide, basically you want to do that.
Otherwise, I don't think people should bother.
And mathematical typesetting is just ghastly unless you use [?
Tech ?]
[INAUDIBLE]
Oh.
And I don't think you can make it look as nice.
And I'm speaking here about Open Office, and it's probably a clone of rubbish that it replaces.
The mathematical typesetting is just not as good.
And so for technical presentations, it's just not as good.
So now, making the figures, I have the same philosophy.
So the figures I actually program as well.
Where is that fig [INAUDIBLE].
So here are the figures.
So this is a function that draws logarithm graphs.
And then I tell it which parts of the graph to put in.
So that's how I do the-- you draw all the lines, you drew all the pieces.
And then here I call it with different arguments, and I get different figures.
So I'll put all the source code up for you guys, so you can see an alternative way of doing it.
And some links, so you can try it yourself.
And some templates.
But it's not for everyone.
Though it is a public format.
It produces PDF, which is a public format.
So I actually highly recommend it for those who have at all some programming affinity
[INAUDIBLE] programs that ends up as PDF format
That has the advantage of PDF, which is good.
So I do recommend presenting things as PDF because you can view it on anything.
It's just, you have the problem of, do you have good mathematical typesetting.
And that's hard to find, except [?
in Tech, ?]
basically.
That's the only thing that does it right.
OK, so if you could just take one minute, fill out the sheets.
I'll answer the questions next time.
I'll have office hours as usual.
We'll meet outside that door.
And then we'll go to the cafe area, and have a chat at the tables, as long as people have questions.
OK, so, the moral the story is use the blackboard if you can.
Use the big visual field.
But if you're going to use slides, Assertion-Evidence.
Sentence headline.
Visual evidence.
One message per slide.
Answers from lecture nine, to questions generated in lecture eight.
Audio quality for this video is poor due to technical difficulties
[INAUDIBLE] is I'd like to go through the questions from before.
There's one question from the time before, which is, could I get the reference for the works that said, there's a difference between telling people that they have intrinsic abilities-- saying, oh, you did that because you are really smart, versus, oh you did that because you tried really hard.
And how those two ways of talking to people-- and children especially-- produced different interests, and confidence in trying new problems.
I mentioned that that work was done a lot by a Professor at Stanford, in the Psychology Department.
Could I get a reference for it?
The next update of the website will have the reference, with the reference is Carol Dweck, D-W-E-C-K. And she's written a ton of books, and papers.
But the one that [INAUDIBLE] popular, [INAUDIBLE] is called Mindset.
Which is a really, really fascinating read.
So that will be on the website.
OK. First question was, you mentioned many times, problems that are PDF-- pass, D, or F, where P is made a decent effort, D is made an indecent effort, and F is didn't make an effort.
That's great, but then how do you balance that with the fact that exams become high-stakes, hit-or-miss performances that completely determine grades.
I don't use the exams that way.
I give exams, they're balanced with the problem sets.
The problem sets are PDF, and if you do P on all the problem sets, that contributes a large part of your grade, towards getting an A.
So there's no necessity to make the exams all or nothing.
Problem sets still account for quite a bit.
[INAUDIBLE] It's just you're not forcing the students to get every last T, and dot every last I, which often means going through [INAUDIBLE] to try to do all that.
Instead, you're saying, look, I want you to try to learn something big, important.
If you take responsibility, then, you get a reward for that, by saying, OK, that'll help your grade, too.
About slide talks.
Were there any questions about slides talks versus teaching, versus blackboards.
So one of the papers says that the best usage of slides is as a completed form that's understandable without the talk that comes with it.
Is that for a research talk?
Yes, for a research seminar, that's what you want.
You want two channels.
You want the auditory channel, and the slide channel, or the visual channel-- whatever's on the blackboard, however you do it.
You want them to stand independently.
So if people are deaf, and they just see the slides, they get something out of the talk.
And if they're blind, and they just hear your speech, they get something out of the talk.
That's, I would say, less true for teaching.
The slides tend not to be so, message, message, message.
You may develop a point, build up a bit of suspense, and then give a final message.
So there's more interaction that happens from the lecture, generally, when you're teaching, than in a research seminar.
So those are somewhat different.
But generally, you do want your slides, or your blackboard work, to somehow stand independently.
Although I've just violated that principle right there.
This is meaningless unless you've heard what I said.
But as I said, it's not an absolute rule.
It's just that there's tendencies that are different between [INAUDIBLE] talks, and teaching.
What do you do if you're assigned to a classroom with a bad blackboard situation.
In other words, not like this.
This is fantastic.
It would be even more ideal if there was a slide [INAUDIBLE].
But what do you do if you go to a classroom with, for example, just one whiteboard, or just one or two blackboards?
Then you have to work harder.
Then you have to plan even more carefully, how are you going to use the blackboard.
And be more telegraphic.
Put coarser details on the boards, not every single detail.
Use the notes more.
So you optimize the constraints that you have.
Should you only use the front half of the room.
Suppose you're in a room which has blackboards all around.
Should you only use the front half of the room?
Yeah, I generally try to use only the front half of the room, because people can look at that, and glance out of the side of their eye at the side, and then, the front as well.
The back forces people to turn around, so they're much less likely to use it.
So it's sort of like putting something on the ceiling.
[INAUDIBLE] never see it.
So generally, yeah, I use the front of the room, and [INAUDIBLE] the side.
Should all slides have a "so what" element, or is that only for research talks?
Definitely for a research talk, all the slides should have a "so what," so people know what the point is.
And ideally, for teaching, you should have two.
Now, last time, I didn't mention one of the fundamental benefits of putting a "so what," an assertion, in every slide.
So that was the idea.
So the slide title is the "so what.".
And then here is the evidence, maybe a picture, or a graph.
Some kind of, ideally visual, evidence.
So what's the advantage of doing that?
It's not just for the audience-- it's for you, as well.
What you do, after you make your series of slides, with your sentences, your "so what."
You extract just the sentences, which is easy to do if you're using a program [?
like Tech.
?]
You just [INAUDIBLE], just the titles.
However you do it.
You make a list of sentences.
And then you see whether the sentences flow.
With a sentence, you can tell, very well, because whether your presentation, your teaching, your seminar is well organized, if it herks and jerks, if sentence one and sentence two seem to have nothing to do with each other, there's no transition between them, well, maybe you need a sentence one and a half, a slide that goes in between.
Or maybe sentence two belongs down here, next to something that relates better.
So you can see not just at the level of the individual slides, but at the level of the entire presentation, whether there's a thread that goes through.
And if you remember, one of the principles of good teaching is having a good story.
So this helps you make a good story.
When you're teaching a class that has expectations for teaching lots of material, how do you reconcile your desire to teach the important things really well, and keep a reasonable pace?
Some things aren't really reconcilable.
Classes that, for example, say, OK, we're going to go through the 1,600-page textbook, there's really not much you can do to help that.
And students are just hosed.
But try to avoid that, mitigate it.
Sometimes it seems like, just out of habit, everyone does that 1,600-page textbook.
But you could find out, what are the actual requirements?
What do people need for [INAUDIBLE]?
If may be a lot less.
It may be that a bunch of stuff was just put in there historically, and could be taken out.
So you might have some freedom to reduce that.
The other thing is you put more stuff in the notes, and you give the higher-level ideas in the lecture.
Can I put my slides on the course web page?
Yes, I'll do that, with the reference, on the next update.
What about having slides that scroll rather than flip?
This was a suggestion to avoid the problem I mentioned of signal overload, where you fill up the short-term memory with a slide, and then you've forgotten what's on the old slide.
And so, [INAUDIBLE] too small to remember the whole slide.
So what about having slides that scroll, instead of flip.
And this may be a critique of the PDF format, which organizes things in [?
pages.
?]
I think it's actually not a critique of the PDF format, it's just a critique of the PDF viewers.
So if you have a PDF viewer that can scroll, which many can, you could actually use that.
The problem is that screens are not big enough.
That's really the fundamental issue.
So if this slide is taking up the entire screen space, when you scroll it, stuff goes off these screens, and gets replaced by new stuff.
So that problem is really hard to work around.
What you really need is multiple screens, and multiple projectors.
Many questions asked, what do you do if you're in a room with a bad blackboard situation, where you just have one or two blackboards.
Well, almost every room has a bad slide situation, because you only have one slide projector.
So you only use a small part of the room.
So that's the really fundamental problem.
And if you could get around that, it wouldn't really matter so much whether the slides scrolled or flipped.
You'd put one over there, one over there, maybe one over there.
Of course now [INAUDIBLE].
My guess is that the t prepare is probably proportional to n cubed, or something like that.
Depending on the number of slide projectors you have to deal with.
Because you have to do all the coupling of [INAUDIBLE] n squared [INAUDIBLE].
You'd have to do all the coupling between all the slides, and it will probably take you twelve hours for every lecture that you give, of one hour.
Which is kind of [INAUDIBLE].
Do I think the video camera for OCW impacts the style of my lectures, blackboards, pacing, et cetera?
If anything, it probably improves it.
Because I'm a little more conscious that I write clearly, and maybe it improves my dress code a bit, because I make sure to wear a tie.
But other than that, it doesn't really affect, too much, what I do.
And I usually forget that it's even there.
It's much easier to write equations [?
in Tech, ?]
and then copy and paste them into a presentation, than doing the entire presentation [?
in Tech.
?]
Actually, I disagree.
This is a case of start-up costs, versus running costs.
So there's many examples of that, here at issue.
It's actually a general principle worth knowing, because you can analyze lots of stuff.
Another example of this principle is, suppose you see a book that you really like, that's available in print, from publisher n, and there's also a PDF file online.
Well, you could just print it out and replace your printer.
Or, you could buy the thing from the publisher.
Now, unless the publisher's a total thief -- unfortunately, many of them are, but many of them aren't-- it's usually much better to buy it at the bookstore, because it's much cheaper, than to print it on your laser printer, get all the paper trimmed to the right size, and have it all bound.
So the running cost for you to do one book-- you have no start-up cost, you just do it.
There's no fixed amount you have to pay, no matter how many books you do.
But there's a running cost, and your running cost is pretty high.
Whereas the publisher, they have a big start-up cost.
What they do, is they make all these plates, they take them to a PDF file, [INAUDIBLE], they make all these metal plates, and they ink the metal plates, stamp it onto the paper.
So they do all that.
That's a big start-up cost.
But their running cost is much lower.
They're much more efficient printing, per book.
But it requires all that start-up cost.
So there's a contrast between book publishers and you.
So it's the same thing, actually, with doing [?
Tech, ?]
versus doing, say, regular word processing.
With [?
Tech, ?]
there's a lot of start-up cost.
No doubt about that.
It's not what most people are used to, you end up programming your documents.
And so, if you're only making one document, if you only write one letter in your life, no question it's faster to write the letter.
Your have lower running costs.
You don't have to worry about the start-up costs if you write it using a word processing program-- Open Office, for example.
You just type, and you make it look like you want, and just spit it out.
But, if you have to write 50 letters that all have a roughly similar format, it's much better to get the template.
You pay the start-up cost once, learn [?
Tech, ?]
learn some kind of program template.
And then, you just enter the blanks into the template.
The text, dear who, the address, and all that.
[INAUDIBLE] formatting, that's all dealt with once.
So you pay start-up costs, in return for lower running costs.
And that's basically true of lectures, too.
Once you have all your templates ready, it's much faster to do everything [?
in Tech, ?]
or in some kind of programming, text-processing system, rather than in a word processor.
The problem is that, because of the start-up costs, [INAUDIBLE] lecture the day before, or two days before.
Because of the start-up costs, you always say, well, I could learn [?
Tech, ?
], and do this properly, and it would save me a lot of time later, but I don't have the time right now.
And then you end up paying the higher running cost to do it some other way.
But each time, before each lecture, it happens again, over and over and over again And you would have been better off [INAUDIBLE] actually paying the start-up cost.
So, to that end, I will put my templates up, so other people can use them.
So you don't have to pay so much start-up cost, and you can just take them, and use them for your own [INAUDIBLE].
So I think that was the main questions for this time.
Any questions that have been created since then?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Audio quality for this video is poor due to technical difficulties
OK, today, political barriers to educational change.
"One had to cram all this stuff into one's mind, whether one liked it or not.
This coercion had such a determined effect that after I had passed the final examination I found the consideration of any scientific problems distasteful to me for an entire year."
Who said that?
Einstein, so one of the greatest scientists in the last 300 or 400 years.
So that was the effect of regular school and education on Einstein.
And it's not just Einstein.
It's a general pattern.
So here, from Manchester, New Hampshire, in the 1930's is how Bruce [?
Betiset ?]
superintendent describes the children's approach to learning and to reading and interest in knowledge.
"In the traditional fourth grades, when I asked children to tell me what they'd been reading, they were hesitant, embarrassed, and diffident.
In one fourth grade I cannot find a single child who would admit that he had committed the sin of reading.
I did not have single volunteer and when I tried to draft them, the children stood up, shook their heads, sat down.
In the four experimental fourth grades the children fairly fought to tell me what they had been reading.
The hour closed-- in each case-- with a dozen hands waved in the and little faces crestfallen because we had not got around to hear what they had to tell."
So the contrast is pretty strong.
And the question is why do experiments like what [?
Betiset ?]
did not survive?
Whereas why does the education system tend to have an effect such as Einstein described, where he found the consideration of any scientific problems distasteful an entire year?
So reading is a particular example-- which I'll give you another one [INAUDIBLE]-- of the change and the effect of the formal educational system.
So in revolutionary America-- so this is late 1770's-- Thomas Paine's Common Sense sold 120,000 copies.
And their population then was three million.
OK, so that's 1/100th of the population now.
So the equivalent of 120,000 copies then was about 10 million or 12 million copies today.
Now if you actually look at Thomas Paine's Common Sense, it's written in a language, today, that we would call very biblical.
When we read it you use English and we speak English but you wouldn't expect 12 million people to buy a book of that complexity and subtlety.
Now that expectation's is confirmed by many a study about literacy and proficiency in analyzing documents conducted across the United States.
So the latest one, it was in 2003, National Assessment of Adult Literacy.
So this was published in 2005 by the US government.
So they America in 2003 and they tested three kinds of proficiencies.
Prose proficiency, which is comparing viewpoints and two editorials as an example of something where if you have prose proficiency you have document proficiency, which is interpreting a table about blood pressure and any physical activity.
So taking information from doctors and [INAUDIBLE].
And quantitative proficiency, computing and comparing the cost and analysis of food items.
But none of these are very advanced skills.
OK now, the percentage of adults who are proficient in each of these categories?
13%.
OK, now, 13% is a very, very low number especially considering the level of proficiency that's required, the threshold is pretty low.
Whereas back in 1770's America 10 million, 12 million people equivalent today bought Thomas Paine's Common Sense.
And probably many of them were not actually literate in the sense that they couldn't read.
So that book was read to them.
So there was a great hunger for words and ideas and learning, which in many ways we had not matched, despite or because of the vast extent of the schooling system and educational system.
So suppose you want to change this.
You want to improve the situation.
You'd like to make education better.
What are some of the obstacles?
So I'm going to discuss the obstacles in ascending order of difficulty in overcoming.
So another way to phrase, ascending order of how local they are.
So the first obstacle I'm going to discuss is one that's local to personal psychology.
It's local to how people individually act.
And then I'll move to obstacles that are broader in scope.
So the first obstacle is what I'll call cognitive dissonance.
So the way to see this in action is suppose you go to someone and you say, well you know, the way that that course is done could really be improved if you do it this way.
And you think you've made a perfectly reasonable point, perfectly valid point.
But somehow your point isn't taken any notice of.
Or it's met with resistance and maybe even hostility.
Now, why might that be?
Well, there are many reasons.
But one of them is you might have phrased it badly.
You might have cursed out somebody's mother while you were doing it.
There are all kinds of things that you have a lot of control over and don't necessarily have to do.
But there are some reasons that are harder to control.
And that's in the psychology of the person who's hearing what you're saying.
So one of the obstacles is this idea of cognitive dissonance.
So let's see how that works.
So cognitive dissonance was a concept invented in the 1950's by Festinger, Leon Festinger, psychology at Stanford.
And he did the following experiments.
The experiments were, he had people come and do this task where they basically just put round pegs into round holes and-- he's a college student-- put square pegs into square holes and triangle pegs into triangle holes.
So after they did the experiment they were then split into two groups.
Group A was just sent away.
You know, they did the experiment and they left and nothing more was heard from them until a year later.
Now Group B was told, oh, we actually need people to help us recruit more volunteers for this experiment.
And obviously they might have problems recruiting people to do the experiment because it was so dull.
All you do is put square pegs into square holes.
So they said would you be willing to be-- in the experiment-- our assistant in helping recruit people?
So Group A, they just went away.
And Group B, they were offered a position as experimenter's assistant and were paid $20 for that.
And Group C-- the same thing.
They were offered a position as experiment's assistant but they were only paid $1.
This was in 1950-ish.
So roughly speaking in today's dollars, let me just translate those.
So here's a way of estimating the equation.
So in 1950 we still have the gold standard.
And the gold standard was $35 an ounce.
So the world price of gold was fixed by Fiat from Bretton Woods at $35 an ounce.
Now it's maybe $70 an ounce.
So inflation is about 25%.
So this is about 400 today.
And this is about $20.
So then what happened was these students then did their work, nothing, this, or that, being an assistant.
And then they were tested a year later about their opinion of the experiment.
So that was actually the real experiment.
So these students thought-- like all psychology experiments-- they were actually helping recruit people for an experiment.
When actually, they were the experiment.
They were really the subject.
So a year later they were asked their opinion of the experiment, did you think it was interesting?
So these people, they obviously thought it was boring because it wasn't worth it.
So a year later these people were asked what they thought.
And these people said boring.
OK, now, what about Groups B and C?
What did they think?
So take a minute or two, try to predict what they were opinions about the experiment were one year later.
Question, yes?
So did they have a choice to participate?
Were they forced to do this?
No, I don't think they were forced.
It was the Group B who agreed, or the Group C people who agreed.
I'm pretty sure that's right.
And I'm sure they weren't forced.
So yeah, find a neighbor or two and see if you can predict, what did Groups B and C think after a year or so?
OK, so, what do you think they thought?
Yeah?
Well, maybe Group C liked the experiment better because they were only trying to recruit, I think they wanted to but [INAUDIBLE]
OK, so Group C maybe liked it more.
So the question is, who liked it more?
So maybe Group C liked it more.
So they weren't getting enough incentive in cash and they must have liked it.
Any other thoughts, comments?
Yeah?
Well, it's possible they didn't like it originally, but they convinced themselves they liked it because they were doing this work recruiting for not very much money
OK.
So they must have convinced themselves because they were getting enough cash to be happy
I actually have a question
Yeah?
So did [?
all of them ?]
take over?
Or are we talking only about those who did it?
That I don't remember.
That's a good question.
I think we're just talking about the people who took the offer
OK, so it was self selection?
Yeah, there's some self selection
But if a group came from that, you probably should ask the question immediately after the experiment before asking them to help assist and then you set the control.
[INAUDIBLE]
And they might have done that.
I don't remember all the details of it.
So they might have done that.
And [?
some wonder ?]
if the control was the Group A.
But you're right, that's another good control.
And they might have done that.
Other comments or questions?
Predictions?
Yeah?
Is this a flat rate or is this a per person?
Oh, per person they recruited?
No, this was their salary.
Yeah, that'd be pretty lucrative, $20 a pop.
Yeah?
I was thinking B probably wouldn't have liked it as much as C because they probably would have thought, if they have to pay me $400 it's really, really boring
OK, so B liked it less on the theory that they were doing it just for the money.
They were thinking to themselves--
Or also if the reason they really have to pay them this much to get the subjects it must really suck
Yeah?
Well, and you can look at it another way.
You can say, they're paying me this much money.
There must be some intrinsic importance to the work.
So you could convince yourself--
it could go both ways
Right, exactly
They pay me a lot so it must be really important work.
They must have tons of government funding for people
Yeah
All right.
So it could be [INAUDIBLE].
Oh, comment?
Well, I have to say, I think the process of recruiting people would get you to like it more.
Because you'd have to be selling it.
And then the question is whether you put more effort into that if you're getting paid more money or not
OK, but the common point is recruiting can help you like it?
Yeah
OK, so I would say to me, actually, you were right, actually, what happened is that these people still found this boring.
And these people liked it.
And basically, at the time this was counter intuitive.
And the whole idea of cognitive dissonance was built up to explain this counter-intuitive result, which is why would you like it more when you're paid less?
It is sort of counter intuitive.
And it's exactly like you say, which is that you reason to yourself well, if they're giving me 400, yeah, sure, I'd sell my grandmother for 400.
You know, that doesn't mean that I don't like my grandmother.
But $400 is $400.
But if they only give you $20-- that's option C-- no, of course you wouldn't do that.
There must be some reason that you're doing that.
So if you're participating in the activity we'll say by doing but don't like it, or at least you try to justify it to yourself.
You need some kind of justification.
And the justification of Group B is well, I'm doing it because it pays a lot of money.
The justification of Group C can't be that because there isn't enough money.
So it must be that it's an intrinsically worthwhile activity.
So the opinion shifts.
Now if you remember, I mentioned cognitive dissonance as one of reasons to have people vote when you're originally doing multiple-choice questions.
Give a muiltiple-choice question to the students, say A, B, C, or D. The importance of having them vote and make a public stand is that it commits them to the question and to the answer.
Now they have some public appearance in the one of the A, B, C, D categories.
And they want to know, is this right?
So the political organizers know this.
And these kind of ideas are old hat now.
We've seen things that consciousness and psychology in marketing.
So people will sometimes-- for example-- come around and ask you well, would you mind putting a sign in your front yard for this political campaign?
Well, at first they'll say, would you like to give $100 to the campaign?
And you say, no.
I don't really know who XYZ is.
I sort of maybe support him $100.
[INAUDIBLE].
So you say no.
They say, OK, well, would you mind putting a sign in your front yard?
So now, what have you done?
Made a public commitment.
All right, everybody who drives by or walks by sees that sign for the candidate.
So now studies are completely consistent with this-- you now do it-- you now support that candidate more.
So turns out, what's the key thing in changing your view?
Is that you feel you had a choice.
So here, if you put a gun to someone's head and say, put a sign in your front yard.
And they put a sign in their front yard, that doesn't change their view of the candidate.
The person could refuse or not.
It was up to them.
If they chose to do it, well, that opens a door to further commitment.
So here, this is sort of like putting a gun to their head.
This is sort of like putting a gun to their head.
You don't really feel like you have a choice.
You're like, OK, whether it was boring or not, who cares?
Whereas here, you took the choice.
And the ones that did it changed their view.
So cognitive dissonance is very, very powerful.
Now, how does that relate to peoples' response when you say, you'd like to change the course?
OK, and we'll come to that.
Yes, question?
Yeah, so again, Group A then was never offered a choice?
Yeah, they were sent away.
And they were asked a year later to see what they thought of it.
And they said, yeah, it was boring.
They're sort of control.
I mean, you expect them to say, it's boring
[INAUDIBLE] boring?
Yeah, just to check so no one could say, well, maybe people found it really fascinating.
You know, because they didn't.
So now what happens when you say to someone, oh, we really should change the way this course is done.
So now suppose you're at some weak university like MIT or Cambridge or something like that.
Now, where do a lot of the people who teach at these universities come from?
Either that same university-- especially in England, that was really notorious in the united justification [?
for it ?
], which was that our system's so wacky and different, only people who've been through that particular system can possibly teach here.
So how are the justifications?
Either the people come from that same place or come from a similar background, similar university background.
So now you're not criticizing a system that they A, selected them.
Right?
The system selected them from a whole bunch of people.
And so by definition, it must be a good system.
Also, they've now had many opportunities to make public commitments to the system of the way things are taught, the way things are done.
They've taught them in classes, they've participated in curriculum discussions.
They've selected the students that come, recommended the students that would be better or worse according to that system.
So the result is that because of years of cognitive dissonance working on people, they are now very, very convinced that the way things are done right now is great.
And although it's called cognitive dissonance, it's not necessarily a cognitive belief.
If you ask them, can you explain why it's great they wouldn't necessarily give you great reasons.
And this can be very misleading and can lead you into hot water.
So for example, when you say, well, really we should teach this course differently and put in interactive questions so students learn them.
You'll get responses that are almost nonsense.
They'll be things like, for example, well, if we don't put everything on the chalkboard how will students know what's going to be on the exam?
And yeah, it's a valid English sentence.
But it's not a good argument because there are other ways to communicate with students.
You can put stuff on notes.
You can say, read the book.
There's tons of answers to that problem.
But these kind of things can be offered as just a complete trump card.
OK, these trumps have been played, because no one can do anything interactive.
Now, that level of thought that's behind it, no way people would put that level of thought into their research and be successful with research.
And how come that's OK and can be sort of convincing in this area?
Well, because it's really not a cognitive response to be made.
It's a belief response.
You threaten and challenge a person's core beliefs and they're going to come out and react very strongly.
And if you try to argue it just on the rational level you'll, actually, pick up angry people.
And you won't anger yourself because you wonder why you hate the progress.
So one way to think about those kinds of conversations is A, if they're difficult conversations-- and actually, the reason I write that there is there's an excellent book called Difficult Conversations.
written by-- I'm pretty sure-- it's from the Harvard Negotiation Project.
And they said, look, every interaction-- and I think this is right-- has three levels.
So that would be the factual level, which is just the ideas being exchanged, of should we teach interactively or not?
Is there enough time?
There's the emotional level, which is how we feel about these facts.
And then there's the meaning.
So what does this discussion mean for people?
So as an example, [INAUDIBLE], the problem with a student who's worried about their grade.
Well, you can tell them, factually, look-- about 3% on that problem set is going to be 0.03% of your grade's total points, don't even worry about it.
All right, so that's the factual level.
You may often find that just the factual discussion is just the tip of the iceberg because there's all this stuff going on.
So what could some of this stuff be?
Well, the emotional level is they just feel really anxious about grades.
Now, why might that be?
Well, that's where the meaning comes in.
The meaning may be that they feel that the grade measures how good they are.
And they've always thought they got into MIT by mistake.
And now people are going to find them out.
If they ever get a bad grade their cover will be blown.
And everyone will know that they don't belong.
So it may have a really strong meaning for them, which is rather emotional.
Which means the fact [INAUDIBLE] discussion [INAUDIBLE].
So it's a similar thing with the discussion of change in education, is that people will have a factual discussion that seems pointless and it's just frustrating to you.
Because actually there's emotional investment in the way things are done.
And meaning, it means a lot to people.
This system, it chose them.
It's a system they're committed to.
And it's being criticized by you.
Therefore, you're going to get a whole bunch of unrealized anger [INAUDIBLE].
Question?
So when it comes to changing a single class and adding some interactivity, what are the downfalls avoiding the difficult conversations by asking for forgiveness rather than permission?
Nate, that's a good question.
So asking for forgiveness rather than permission, one of the downsides is-- I think, in small and moderate doses that is a good idea.
Because first of all, no one's going to notice.
Because they're too busy with their own [INAUDIBLE] teaching that that's rather busybody over what you're doing.
So you have, definitely, something right away that you might as well use.
But if you want to make any larger changes, like for example, take out something that you think isn't so important in the curriculums-- and make room for doing this-- and somebody downstream really wants that.
Well, it's important to have done it beforehand, discuss it beforehand.
Because otherwise, if there's emotional and meaning investment in the way things are done, they won't be able to listen and then you surprise them with a factual change.
They won't be able to listen to your factual arguments then, because you activated the emotional and meaning responses.
So you want to try to arrange, [INAUDIBLE] those things beforehand and get all those out in the open first.
So I would say, for smaller changes, yeah.
For larger changes, it's important to have people jointly figure out what to do, which is true of any political change.
And that's why when I deliberately chose the title Political Barriers to Educational Change.
Because political barriers has the implication that it's a political problem.
It's a social problem.
It's a collective problem.
And any collective problems has these issues of what you do by yourself versus what we do in a group or [INAUDIBLE].
Question, yeah?
So I'm just a little bit confused about the idea of cognitive dissonance.
So is that where the [INAUDIBLE] between the responses of groups B and C?
Mm-hmm.
Yes, it's that.
Oh, so where does the dissonance come into it?
I should've explained that.
The dissonance is what you try to reduce by changing your view.
So in Group B there's really no dissonance because the experiment's boring.
And they did the recruiting because they were paid.
There's no dissonance.
Here, the experiment is boring, but they don't have the resolution that oh, I'm being paid to do it.
So they have to have some way of reducing that dissonance.
And they're going out and recruiting people.
So the way they do it is they reduce the dissonance between the internal view and external performance by changing their internal view.
Thanks
Yeah
So I'm wondering whether-- do you know how people talk about how sometimes if you're trying to convince people to do something you try to get them to feel like it was their idea?
And it seems like that would be the opposite approach to what you were just saying, which was that you try to get people on board in advance maybe even so subtly that they don't even realize it.
Because I think if you make people feel like they're being threatened and furthermore that like you're deceiving them, but they'll claim to be always kind of on purpose because you felt like you were backed into a corner.
And even if they're not committed to it, because it produces them as [INAUDIBLE], or just because people don't like to be fooled, that they don't like to feel like they're taken advantage of
Right.
So that's why I would say, yeah.
Take advantage of the freedom that you have that everyone sort of [INAUDIBLE] to make smaller changes just on your own
Do you have ideas for how you might go about [INAUDIBLE], if you see people with [INAUDIBLE]?
Yeah.
And not so much convincing things that it's their idea it's-- yeah, I don't want to phrase it as people think this as a dishonest [INAUDIBLE]
Yeah, yeah
It's really you're trying to find collective solutions to problems that you both think are happening, or you all think are happening.
So one way to go about that is to say, well, look.
Here is a problem I see.
What do you think you could do about it?
They might agree with you on the problem.
By creating one problem, pretty much generally people will say, yeah, the students are fine with problem set.
But when they have to go do something with it, they can't do it.
So you'll get a much wider grievant on that statement than on what to do about it.
But you can start there.
Say, OK, well, what do you think we should do about it?
They may have some ideas.
They may accept the idea that, well, this is a serious problem.
And it's not clear what to do about it.
But that right away gives you some license to try to [?
external?]
[INAUDIBLE].
So you say, OK, well, here's one thing I'm thinking of trying.
Let's see how it works.
So you're not saying, it must be done this way, and this is the only way to do it, and you're bad to not do it.
Say, well I'm going to try this.
Let's see how it works.
And I'm happy to share the results with you.
And you can use that, too.
And so there's another related point to that, which is the-- I don't know what percentage to give it.
So let's call it a 10%, 10%, 8%.
So 10%, 10%, 8% equals this.
So 10% of people will be early adopters.
They're really curious about new things.
They want to try new things.
Let's call them [INAUDIBLE].
There will be, say, 10% who no matter what you do are just going to say, this is terrible.
So, the never adopters.
And the 8% who are in the middle who will wait to see and will sort of sit on the fence but [INAUDIBLE], but open.
They have other priorities, but they're open to these kind of things.
So my rough rule of thumb is, when you're talking to someone, could be figure out which category they're in.
If they're in- this is true of political discussions, right?
You probably have some relatives in the family where you just agree we're not going to talk about politics.
Right?
So, I have them.
I'm sure I'm in some people's category of that , too.
We just say, we're just not going to talk about that.
Because one or the two of us is not going to adopt the view of the other no matter what is said because you just think it's absurd.
So if someone's in that category, don't even spend any time.
Just spend your time [INAUDIBLE] category and say, OK, I'm going to move on.
If they're in this category, you usually can figure that out pretty quickly, too.
Work with them.
And if they're in this category, that's the people you can reach if you do things in a good way.
OK, now this is actually pretty large.
I think in the '70s this was called the silent majority, was the code for this [?
this word.
?]
Yeah
Are you ready to be done with this?
Yeah
Could you talk a little bit-- maybe a little later-- about what sort of scale changes you're talking about?
And also along with that, how much power each individual professor has to design their own course.
If you figure out what the prerequisites are for the next course, then I don't see exactly what the problem is doing whatever you want in your course as long as you meet those-- as long as you teach them
So the question is what size scale change am I talking about?
Well, it depends.
How much freedom you have depends on the change you want to do.
For example, suppose you took to heart some of the readings that were for today and you thought that grading was really terrible and you want to just dispense with grading completely.
Well, even if you're teaching all the right stuff, that will raise a lot of of ire.
And people will come after you for that even if you're teaching everything right and you're doing all the prereq.
Because it will trigger all kinds of factual, emotional, meaning responses in other people.
And it'll go against many, many-- and that's what I'm going to talk about next is-- features of American society to think we must do things that way.
So you have less freedom to do something like that, even if you're meeting all the prereqs.
Now, if you want to, for example, use different examples to teach the same ideas, Everyone wants complete freedom.
So it really depends what you want to do.
Then there's many scales where you can make change.
So you may find that there isn't any way to do what I want to do and meet all the prereqs for the next course, because it just means doing way too much stuff.
OK, well now you have a problem of a bigger scale change.
How do you get the whole group of faculty or group of people in charge of that area or course to change what was going on and changes the prerequisites, change the overall curriculum.
So now, you're definitely going to be working with lots of other people.
There's another sort of dichotomy, which is is the course required or not?
If the course isn't required, you have much more freedom.
[INAUDIBLE] so your course.
If people don't like it, they don't have to take it.
And even if other people don't like your course or don't think it's that important, they're perfectly happy, generally, to let you teach the course that they don't think is that important.
Because they want to teach their course, which you may think is not that important.
But now the required courses that every single undergrad that majors in the fields take, those generally are guarded more with jealousy.
And so changes in those are harder and generally require more consent from other people just because it's considered a shared core of the field.
So that's the meaning.
And by changing that, you're complicitly perhaps threatening to share core of the field.
Does that answer your question?
OK, so what I'm going to talk about after the break-- so this was the idea of cognitive dissonance, and what role-- and maybe just a few, your local group, maybe your department, maybe your school-- difficulties and obstacles and issues that come up when you're trying to make some kind of educational change.
What I'm going to talk about after the break is society-wide features.
All right.
So American society or say, European-American society in general, that impose really large obstacles in educational change.
In some ways, even bigger than this one.
And you definitely want to take your time with this one.
But it's also important to take your time with even larger scale ones.
And those will be Behaviorism, Taylorism, and the history and the purpose of those schools.
And we'll talk about those after the break.
It's 9:58 right now.
So we'll start at 10:08 according to that clock.
The first barrier, cognitive dissonancy is a-- to a personal psychology.
But now we're going to look at factors that span the individual.
I mean, this spans the individual in the sense that it's the common feature of our psychology, maybe of this [INAUDIBLE] psychology.
But let's see it in individual people.
But they're are also features that are common across-- you could call them social psychology or social tendencies.
The first one is Behaviorism.
Behaviorism is also a school of psychology, which, basically in America, grew out of Pavlov's work on conditioning dogs and working on teaching patience by reward and punishment.
And in psychology, what it was is it was a focus on outputs.
So it basically denied the importance of studying what's going on inside the human mind.
It was correlated with what was called the logical positivism in the Vienna school.
So in Europe, around the same time and soon after, was the rise of quantum mechanics.
So if you look at quantum mechanics, quantum mechanics was created partly by the Vienna logical positivists in math in [INAUDIBLE].
In quantum mechanics what's one of the fundamental ideas is what's called an observer.
You know, where it says, look.
Don't talk about what's not observable.
The important thing is what's observable.
Everything is sort of rigid around observance.
All right, so behaviorist psychology was almost in the same way.
It says, look, don't talk about the mind.
We don't know what the mind is.
It has no separate existence.
Let's just talk about behaviors.
How does what you do effect what the organism does?
It's a fine way of training pigeons, but it's not a great way of understanding people.
But you can see that it's actually dominant in lots of American education.
So Behaviorism was actually strongest in America, It was eventually, I would say, demolished as an intellectual movement basically in this space.
I guess the old linguistic department was here.
And Noam Chomsky in 1957, '59 wrote the review of Skinner's book on Behaviorism.
And that's considered the coup de grace of Behaviorism.
But even then, the idea lives on.
And it underlies so much of what goes on in this school.
And you can see that with actually the lively examples of that, given the core.
I'll give you one more of that [INAUDIBLE].
So this was a study.
This was fourth, fifth, and sixth grade.
And they were asked what's six times three.
Well, they weren't having a problem with that, It's 18.
They were asked to make a story problem for 6 times 3 equals 18.
80% of the students could not do it.
They couldn't make a valid story problem.
Now, what I mean by story problem is a word problem.
So they could actually do the multiplication.
That was no problem.
But to make a problem which used that multiplication, that was difficult for them.
Most of the problems they came up with were along this line.
On Monday, Johnny buys six apples.
It's always Johnny, not John.
I don't know why.
On Tuesday, Johnny buys three apples.
So 6 times 3 equals 18.
There's was a fundamental lack of understanding of what's going on here behind the equation.
So the performance was fine.
He said, what's 6 times 3?
18.
And so the observable was [INAUDIBLE].
So by the [INAUDIBLE] standards that was fine.
There was no problem.
And so much of it was on the school.
Bacon just looks at the outreach.
Whereas, if you actually look into the [INAUDIBLE], that Behaviorists say you shouldn't really do, and ask the tougher [?
question, you'll ?]
find it actually has no meat.
But it's just like tissue
Is the problem here me, or is it we are looking for the wrong outputs?
I don't see, per se, how asking students to make a word problem is looking into the mind any deeper than asking them 6 times 3 equals 18 because it's an output that's closer to what we want students to actually be able to do
The thing is if they this, if they produce a correct story as a formula too, then you wouldn't be happy either.
So in some way you want to know does this have meaning for them.
So you could then say, well, 6 times 3 is 18.
OK, they understood that.
Now, I want them to give me the right output of a story problem.
Well, I have them memorize a correct story problem.
Whenever they see multiplication, do this kind of story problem.
And then, if you ask them the question give me a story problem, they would produce the right output.
But you'd still be unhappy.
You'd have to ask a different question to figure out if it actually had any meat.
Maybe for example you'd say, well, multiplication's pretty similar to division, so let me see if they could make a story problem for division.
Or let me give them a different question, which is this one.
Suppose they hadn't had this question before and hadn't been pigeon trained with it.
So you could give them that question and see if they understood what really multiplication means.
If they just write down 73, you would say, OK, they understood that.
And it would be kind of pointless to try and have them memorize all questions like this.
But after they've seen this kind of question, yeah, they might produce the right output.
If you just tell them, OK, whenever you see this, one, two, three, four guys here and four there, produce 73 here.
Well, that's different from really in understanding multiplication.
So I would say output is not sufficient.
So you have to look behind the output.
Did that help answer your question?
Well, but I feel like you're still looking [INAUDIBLE] but the difference is the process you used to get the students to the point where they can generate that
No.
Well, you're looking at the output, because that's all you can look at.
But you're not saying if the output is fine, it's-- you're making inferences about what's going behind that.
And that's what behaviorists don't want.
You're saying, oh, there's understanding, there's meaning, there's reasoning going on beyond the output.
And that's what the Behaviorists don't want to talk about.
They just want to talk about just the outputs.
And that's a fundamental problem.
Because real understanding-- focus on just outputs leads to rote learning.
And I think that is one explanation for why there's so much rote learning produced in the American education system
Does anyone-- [INAUDIBLE] [?
location ?]
[INAUDIBLE] making the contributions [INAUDIBLE] like understanding a problem, [INAUDIBLE].
So what's the operation for that that's different from making up problem.
Making up a problem requires more understanding, right?
Yeah.
Making up a problem does require a lot of understanding
[INAUDIBLE] So society wise, you only need a person to solve the problem not make up a problem, right?
Well, that's not the purpose of-- so do mean prompt people to just solve problems or make up problems?
That can't be the premise on [INAUDIBLE] democratic society
Right.
OK.
I agree with that.
But why should [INAUDIBLE]?
Pardon?
Who's happy to choose the computation?
They know that 6 times 3 is 18.
[INAUDIBLE]
Well, so it depends.
So on the [INAUDIBLE] in many schools there's tests that just test things like that.
[INAUDIBLE] .
So the No Child Left Behind Act has pretty much mandated that children be tested pretty regularly on their basic skills.
And so there's a ton of questions and things like that on these basic skills tests.
So some people care.
And these kind of tests are an impediment to making any kind of educational change.
Because if you start spending time on things like that, well, then you have less time to practice three-digit by five-digit multiplication
Now this [INAUDIBLE].
We should say [INAUDIBLE] point first, and then from there break it down to the [INAUDIBLE] much more [INAUDIBLE] than [INAUDIBLE]
That may be true.
So the comment was that maybe it's better to try something not so hard as making up a story problem.
Identify--
[INAUDIBLE] the [INAUDIBLE] that we need
It shouldn't be that hard
I mean, we're on a skill, right?
It shouldn't be that hard, but it is hard if you've just done drill
Yeah.
If you start so low--
Yeah.
I agree.
So it's not saying that all of a sudden now-- I'm not saying all of a sudden now you should ask what we're doing with this question how to make a story problem.
But this is used as an example to show how far away we are from where we'd like to be.
So Behaviorism is, I would say branded reward and punishment, pretty much rampant in American society.
In discussions of merit pay for teachers, OK, should we give teachers higher pay if their students do better on tests from No Child Left Behind?
So that is the assumption that you want more output from the teacher in that [INAUDIBLE].
You just pay more money.
You give them more pigeon food.
But there actually maybe other reasons going on, there may be completely other factors besides input output.
For example, maybe teachers think we shouldn't be teaching the test this way.
Maybe there's all kinds of social consideration which aren't included in just paying people quote "meritly."
So you could see how rampant it is because when you say, oh no, we shouldn't do that to teachers, people think you're crazy.
They say, well, of course we should-- and then they make a really, really long argument for why we shouldn't.
So the default is, yeah, everything should be paid according to results and rewards, and rewarding according to result.
Yeah
I'm curious about what you think-- because I do think that there is no question that in our society things in which we give respect to and value as a society do get paid more and [INAUDIBLE] to have more perspective as we [INAUDIBLE] our educational system is that we don't pay our teachers anything and don't respect them.
And that's why very few people actually consider that as a real career.
[INAUDIBLE] you know, [INAUDIBLE] to actually consider teaching at my school.
Very [INAUDIBLE] rate.
And that's because it's not respected.
It's not considered a challenging [INAUDIBLE].
But don't you think that the amount of money that people get paid actually does in some sense request our respect for them as a society?
That I agree with.
So there's the meaning [INAUDIBLE].
Yeah, there's meaning associated with the salary.
So the problem, what's wrong with merit pay is that the meaning is that you're competing with other teachers.
I'll give some money to the schools and say, now dole it out among your teachers.
And the one's who do best---
Yeah.
No, I think that's [INAUDIBLE]
But I agree with you about increasing the overall salaries for teachers.
And it's exactly right what you say that it's not respected as a profession in this country.
And as you say, people who get a Ph.D from MIT are not likely to teach in high school, which is, I think, sad.
But it doesn't have to be that way.
So in Europe-- so George Collier, one of the greatest mathematicians of the century, he taught at a Swiss high school.
It was a considered a very respectable position and profession.
It's a serious profession.
It's really important.
Whereas, that's pretty much unheard of here.
And [INAUDIBLE].
So it's a combination of respect and salary.
And related to that, autonomy.
So teachers here in America generally have the least autonomy compared to, say, Western European countries.
So one way of measuring that is just how much free time teachers have to do curriculum [INAUDIBLE].
So think back to your own high school.
How many classes did your teacher teach?
Typically, six a day.
I mean, I'm exhausted if I teach two classes today.
Because it really takes energy to teach a class really well, think about it, prepare, answer questions, follow up with what happened, and make references.
Teaching six times a day is absurd.
And all you can do is just try to survive.
So that's, again, another barrier to educational change.
So that is barrier number two which is-- the second we'll call the [INAUDIBLE].
So it's the focus on outputs and reward and punishment.
And then the next barrier is another social wide factor called Taylorism.
So Taylor-- it's related a bit to Behaviorism.
So Frederick Taylor was a mechanical engineer.
Let's see, when did he live?
About 1850ish to about 1915 or 1920.
1856 to 1915.
So what he did-- he's the originator of time and motion studies.
So time and motion studies watch how someone, for example, is riveting bolts onto a car.
And you watch every single movement and you optimize it.
And you say, oh, actually if you have the screwdriver on this side, well then when the car door comes down this way you can do this with much shorter movement than trying to do that.
So they said you can break every task down with a lots of little little fragments you can optimize.
Now, this basically took the whole country by storm in the 1910s or so.
Now, how did that happen?
Well, this was a time of industrial unrest.
And unions were striking demanded higher wages.
And there was the Interstate Commerce Commission, which regulated the railroad fares.
And so the railroad attorney, Louis Brandeis, later Supreme Court Justice, argued the in front of the Interstate Commerce Commission, well, actually we don't have to raise fares.
And we can actually give the unions some of what they want without doing any of that provided we implement Taylor's system.
So the idea of Taylor's system was a way of basically trying to prevent the unions from striking, but not distribute any of the wealth for management, but just make things more efficient.
So it was what we now call the Cult of Efficiency.
And this Cult of Efficiency I would say probably could only happen in a frontier society, like America.
So there's a completely fascinating book, which I highly recommend reading, The Great Frontier, by Walter Prescott Webb.
So what he looks at is he looks at the effect on European society of the European discovery of the Americas, so around 1500 BC.
Columbus sailed and all the explorers were going a sail.
So what happened?
Well, before 1500 Europe was extremely crowded, lots of people, very few resources, very little land, pretty much deforested, not much fuel.
And so it was a hard place to live.
With the discovery of the new world, that switched.
There was now a ton of land and not enough people, especially after the Indians all got small pox and died.
There was a ton of natural resources-- wood, minerals, eventually oil.
So around 1500, so the world society changed.
At least the European society.
And with it, changed conceptions of how society should work.
And in the frontier societies, basically everything was there for the taking.
You just had to work to get it.
So in a frontier society, basically work was a religion.
That was the religion of a frontier society.
And you can still see that in America.
How many weeks of holidays do people have completely guaranteed?
I think zero.
And whereas in-- for example, England, I think it was four weeks, maybe statutory minimum.
In Germany, it's five or six weeks.
So in the frontier society, work will be your religion.
And it's likely to be placed where ideas such as Taylorism will grow, and sprout, and thrive.
So Taylorism spread throughout the entire country, including to the schools.
And you can still see it in school.
School is considered a factory.
And the inputs are the students.
And we want to do that as efficiently as possible.
Think class size, giant buildings to [INAUDIBLE] to produce more product at the end.
So this is one of the early examples of a Taylor school.
The Platoon School.
So this was an elementary school.
What they did was they-- this is absurd.
They enrolled 10% to 70% more children than there were seats for in the classrooms.
So [INAUDIBLE] efficiency.
This is done by dumping the 10% to 70% on the playground, into the auditorium or basement.
So by keeping all the children rotating from room to room, teacher to teacher, so that the same 10% to 70% [INAUDIBLE] way longer than 30 consecutive minutes to get into classrooms.
So even the 6-year-old children have 6 or 7 teachers a day and as many as 12 or 13 in some cities.
And teachers handle as many as 400 people through the day and 1,000 a week.
So the terrorists came to the school and said, hey, look, these schools-- it's all very relevant to have small schools for people, but really, it's not efficient.
Look, there's all this inefficiency you could squeeze out of the system.
And so you can see probably MIT's going to be going through something like that with the budget cuts, all the [INAUDIBLE].
And trying to squeeze the life-- I mean, the inefficiencies out of the place.
So what they said was they said, oh, look, there are all these rooms that aren't being used.
You have the gym that's not used all the time.
There's the lunch room that's only used at lunch time.
That's crazy.
There's an empty room there.
There's the playground that's only used at recess.
What we should do is we should rotate the kids all the way through.
And you'll actually see that in schools now.
Many of them have lunch period that starts at 10:00 AM, and the lunch period goes till almost 2:00, because it's a much more efficient use of the lunch room.
But ironically, they had to cancel recess, because recess is inefficient.
Recess is not efficient for doing standardized test problems, so you guys will see that as well.
So again, they called it efficiency.
That was back in the '20s [INAUDIBLE].
It's from Raymond Callahan's book called Education and the Cult of Efficiency: A Study of the Social Forces that have Shaped the Administration of Public School.
It's about Taylorism in the public schools.
But it continues to today.
In No Child Left Behind, you can see is a perfect example of it.
[INAUDIBLE] wrote, he said, "The long lines of marching children that look to be like nothing so much as the lines of uncompleted Ford cars in the factory, moving always on with a screw put on or a [INAUDIBLE] tighten as they passed-- standardized, mechanical, pitiful.
When the factory system carried into the public school, which needs only the closing time whistle to make complete [INAUDIBLE] with giant, great industrial plants."
So there you have another fundamentally important social factor.
[INAUDIBLE].
So behaviorism and Taylorism are sort of lumped together.
Which is the joining of the school and the factory and the educational system and the social order.
So that I'll call the social history of education.
So education-- and America was one of the leading countries in this, in making schools free and required-- compulsory free education.
[INAUDIBLE] public school.
So this-- in Massachusetts, actually-- our state was the leading state in American [INAUDIBLE].
And so Horace Mann in the 1840s-- he was one of the leading educational reformers.
And he was arguing to businessmen.
OK, you really should be willing to have higher taxes to make schools free and required.
Now, why would businesses want to part with their profits to pay for schools?
But he says, look, there's all these benefits from schools.
The schooled workers are more docile and quick in applying themselves to work.
They had better domestic and social habits, you know, punctuality and fidelity.
They have higher punctuality and fidelity in the performance of their duties.
So basically, he was arguing for school as a method of social control.
Now, there was a strange feature in my high school, which I never understood till I actually read about this.
History of Horace Mann called the School of America, which was that-- so I experienced this first-hand.
One day in my high school, this racist student attacked me and called me nasty names.
And so when I fought back-- and we sort of fought to a draw, which was good because he was trying to do serious damage.
As a result, the school policy was that whoever was in the fight was suspended for three days.
So we were both automatically suspended, and then had to go to court to figure out who was really at fault.
But by then, of course, I'd already been suspended for two days.
So I was suspended for three days and I got to find out, what are the policies when you miss school?
So it was very interesting.
So first of all, if you're late to school three times in a quarter, then you fail for that quarter.
If you fail in the second quarter of a semester, you fail the semester.
If you fail in the second semester of a course, you fail the entire year.
And if you failed a required course, like English-- math or English required-- then you failed the entire year and had to repeat the whole grade.
OK, so they were very, very, very sticklers about punctuality.
But they didn't really care what your grades were that much.
You could get C's and D's on tests and you'd do much better than if you missed four days of school.
Now, my teachers were nice in this case.
Said, no, don't worry about it.
Good thing you fought and stood up for yourself.
We won't do any of that stuff.
So it was always puzzling to me why punctuality was so much more important in this public school, which was supposed to be one of the leading public schools [INAUDIBLE], than the actual grades you got and how much you learned.
When I read the history of Horace Mann, I thought, oh, yeah.
That actually made perfect sense.
The purpose of the school wasn't necessarily to teach people to think and really understand.
The social history behind it was to produce people who are basically docile workers for the factory system, which was just coming around in the 1840s.
So there were many comments along those lines.
One of the arguments also for the school was similar.
It was a resolution that the best police for our cities, the lowest insurance of our houses, the permanent security for our banks, the most effective means of preventing pauperism, vice, and crime, and the only sure defense of our country are our common schools.
They mean the public, free schools that are required.
So again, it's seen as a defense of property, basically.
This was true-- so when I say America, it's not just America, actually.
England in 1870 passed an education act that required public schooling.
[INAUDIBLE].
And the arguments in England were almost exactly the same.
"Would not the policemen be better employed in assisting the work of the school master by collecting [INAUDIBLE] for school attendance the willfully neglected children who are [INAUDIBLE] recruits for the great army of crime."
We wanted to prevent the working classes from engaging in those vain strikes which end in 99 out of 100 leaving them in worse condition than before.
We wanted to keep them from habits of waste and improvidence that some knowledge of the succession of events in life, such as education could supply.
So in other words, education has a class role.
So that history of education, it still permeates it.
And people were very open about this.
These weren't secret documents that people wrote in their conspiratory council.
These were openly discussed, saying this is why you should pay for schools.
I would say it was even quite a soph-- reached quite a sophisticated level.
So William Whewell, he was master of Trinity College, Cambridge, in the 1860s, I think.
So when he was a university professor, before becoming master at Trinity, he was commenting on the difference between the European and the, say, English and American educational system.
And he said, well, actually, they had [INAUDIBLE] problem in Europe.
And the problem [INAUDIBLE].
So he was recommending the teaching of mathematics.
He said, "The mathematical doctrines were fixed and permanent.
No new system of geometry can supersede the old.
Not only so, but even the old books remain in use."
So now contrasting that, "The fixed and constant nature of mathematics was not likely to promote the critical spirit [INAUDIBLE] France and Germany, where too much attention to the more provisional and contentious observational sciences had been partly responsible for the imminent general hostility to the existing institutions in their country."
So in other words, we should teach mathematics because it has one answer.
[INAUDIBLE] conclude about society, too.
It has the one answer, which is the society we have.
Whereas, if you teach observational sciences, social sciences, people might start questioning way too much.
So there was not just schools, but what should we teach in our schools?
So that history is very, very hard to overcome because it's so deeply embedded into the foundation of a school.
We will find its purpose is like that.
Such purposes keep creeping up whenever you try to change things.
So there are all kinds of things that are crazy in the schools that we do that you don't necessarily need to do in the school.
For example, grading.
Why do schools have to grade?
People think, oh, well, how will other people know?
If other people want to know what students can do, let them worry about it.
Why does the school have to do that?
Sure, it's school.
It could tell students, look you're not learning this idea very well, you could spend more time on that.
You're learning this idea really good.
You don't need to spend any more time on that.
That kind of evaluation, for the student's benefit, is quite useful.
But why should that be publicized to other people?
Well, those are two separate questions, but they're automatically conflated because-- partly, the purpose of the school is to get people to accept their place in the society.
There was a very interesting experiment along those lines.
I don't know if you know it.
It was blues eyes, brown eyes.
So this is related to tracking and grading.
So I don't think you could do this experiment today.
But what they was they took-- I think it was third graders.
And they came into the class and they said, OK, you third graders.
We've just found out that the blue-eyed children are more intelligent than brown-eyed children.
And so we're going to have to teach them separately.
So they then separated into two classrooms and they taught them exactly the same stuff.
And then they tested them a while later and they found the blue-eyed children did much, much better than the brown-eyed children.
So sure, the test results must have been correct.
Actually what they did is afterwards they said, actually, no we fucked up-- sorry.
[LAUGHTER] We screwed up.
And actually, we got it backwards.
It was the brown-eyed children who were more intelligent than the blue-eyed children.
So we had to switch the classrooms.
So they switched the classrooms.
And of course, they taught them exactly the same stuff.
And what do you think happened [INAUDIBLE]?
They flipped.
That's an example of the effect of tracking and grading.
So what's the result of that is that, well, students then accept their place.
They think, oh, yeah.
I am the dumb student.
I am the bad reader.
Oh, I'm just not so good at math.
It's my fault.
Rather than-- well, it's the way it's taught.
It's that I came from a family where we just didn't have much money, even for food.
And the society just didn't provide that, so I was hungry every day.
So there's all kinds of social explanations which you're taught not to look at.
And even the very problems that are done in school tend to reinforce the social order that we have.
So an example of that, to really show how strong that tendency is, is look at the problems that are done about percentages.
The percentage problems are always of the following form.
Johnny has $1,000 which he lends to Jane at 6% interest.
How much interest does Jane pay after two years?
Well, there's many interesting features about that.
First of all, Johnny makes the money.
The guy has the money and not the woman.
The woman is the borrower.
So now, with feminism, we've made it so that Jane can have the money.
But you haven't changed the fundamental problem, which is that the problem is reinforcing the idea of money lending.
[INAUDIBLE].
What about the following problem, which you never see, which is also [INAUDIBLE] interesting percentage problem?
The hourly minimum wage is $7 an hour.
If somebody works 40 hours a week, what's their salary?
OK, it's this much.
It's roughly $14,000 a year.
What percentage of the federal poverty line is $14,000 a year?
That's an equally interesting percentage problem, but it has a different social message.
So the schools have chosen, basically [INAUDIBLE] social pressures that push towards one kind of problem, one kind of grading, one kind of system.
And this is despite what people know about these things.
So blue eyes, brown eyes experiment shows what's the detrimental effect [INAUDIBLE].
Many people say, well, actually, [INAUDIBLE].
Another example is tracking.
So tracking where people know that it's harmful.
So a friend of mine in New York, he went to an elite private school.
It's a small, elite private school.
And one year they decided to introduce tracking.
So tracking is advanced math, median math, OK math, and remedial math.
And after a year, there was a parental revolt and they abolished it.
The parents said, this is outrageous.
We're not paying $25,000 a year for you to tell our kid that our kid is not intelligent.
Get rid of it.
Whereas, it's considered perfectly acceptable to do that to kids in public schools.
So people know, yet the thing goes on.
So whenever you see something like that happening, you can bet there's some really deeply embedded social history-- the purposes of the schools.
And related to the social history is the social history of our society.
So these tendencies, they're not immutable.
All social factors have changed in the past and all of them can change again.
But in order to change them, It's important to be aware.
You have a question?
Yeah.
I just had a question with the whole concepts of getting the grades, which I can see is a benefit in some ways.
But I feel like [INAUDIBLE] to just stop grading children, I feel like at least at the level of college acceptance, there's still going to be some level of evaluating the children.
And eventually, it's just going to be that some other form of evaluation is leading to this separation, even if it's not just by numbers.
Saying [INAUDIBLE].
And is there any way you could really deal with it without having grades?
Well, I think you've identified the fundamental problem, which is the competition.
So it's sort of like the water bed.
If you sit on one side of the water bed, the other side goes up.
So if you reduce the competition in one place, it might increase in other ways.
So for example, you have no grades in high schools, but then you have a really, really high stakes university entrance exam, which is not necessarily the ideal solution.
I think the ideal solution is that you make good universities accessible to all people.
Or, for example, if you make it so that you don't need a university degree to earn a decent salary
But is that ever going to be possible?
Well, I always quote Margaret Mead on that, "Never underestimate the power of a small number of people [INAUDIBLE] to make social change."
And not just a small number, but people in general.
So all things that we now think are absurd and just horrible were once normal.
Slavery was considered absolutely normal.
And in fact, the view of the inferior races [INAUDIBLE].
But all these things that we think are horrible now-- torture-- are now considered just beyond the pale because of social [INAUDIBLE].
So my main point here is that these are structural, fundamental, large issues.
And they require structural and large society-wide changes to fundamentally change them.
But it goes both ways.
The schools themselves produce people who reinforce the social structures.
So if people had come through the blue eyes, brown eyes experiment and they feel themselves that they are not worthy, they're less likely to try and make a social change to make a just society.
So it goes both ways.
Improving the schools and the educational system improves the society.
Improving the society improves the schools
I was wondering if you could take advantage of that blue-eyed, brown-eyed thing, like to say-- one could say something like, oh, people in that class always did really well in their future physics classes.
And then everyone could just be like, oh, great.
[INAUDIBLE]
It's a great point.
So the point was, could you take advantage of the blue eyes, brown eyes experiment to just tell people, look, everyone in my physics class does great.
And so just, actually, make people do well?
There is actually a fair amount of data that does work.
So [INAUDIBLE] wasn't phrased in exactly that way, but the example I'm thinking of is women's colleges.
So a large portion of women in science Ph.D's come from all-women's colleges.
And I think that's partly that effect.
In the regular society, women are generally told, look, [INAUDIBLE] the Barbie doll that was taken off, [INAUDIBLE] Barbie doll that for a while said, oh, math.
Math is hard.
I don't do math, or something like that.
That's what it taught.
It's really amazing that they thought they could do that.
So there was huge outcry and they had to take it-- some people did spoofs.
They put that as a Ken doll and they put a big, deep voice in the Barbie doll just to play on that.
But that social attitude does permeate the society.
And it's much less in the all-women's colleges.
So places like Swarthmore, Bryn Mawr, and Smith where women are-- it's just expected.
Of course you're going to do well in science.
Everyone around here does well in science.
It's not considered anything special.
It's considered a good thing, but it's not [INAUDIBLE] to women.
So they already took away the blue eyes or the brown eyes and just put everyone [INAUDIBLE].
So I think that is a reason for not criticizing kids.
When you're raising young children or teaching them things, don't criticize them.
And same with grades at a university.
Grades should be so the students can figure out what they need to learn more of, but not as a way of making them feel bad.
That's so hard to do when people have been graded or degraded for so long.
Question, yeah
So I wanted to go back to your earlier point about competition, how it takes it away from one place and it ends up another place
I don't say it's conserved.
But you have to think about the whole system.
Yeah, so I agree with the question
[INAUDIBLE].
Not only it doesn't scale because of limited resources.
You can't keep everybody in teacher one on one.
It would be great if it took, but the system won't work that way.
We won't have enough teachers.
So there's always going to be competition for resources.
I just want to--
Well, OK, so the question of competition.
So let me close with one quote, and then I'll come to your question.
OK, so it's a good question.
So to continue the rest of what Einstein said.
So remember, before Einstein said that one had to cram all this stuff into one's mind, whether one liked it or not.
"This [INAUDIBLE] effect that after I passed the final exam, I found the consideration of any scientific problems distasteful to me for the entire year.
It is, in fact, nothing short of a miracle that the modern methods of instruction have not entirely strangled the [?
whole new ?]
curiosity required.
For this delicate little plant, aside from stimulation, stands mainly in need of freedom.
Without this, it goes to wrack and ruin without fail."
So I hope that when you're out in the world, when you're teaching and raising your children, that you find ways to make more freedom in education.
So that when the students actually that you produce improve the society, and they then, can help you improve the schools and the rest of the education system.
OK, your question about competition.
It is a fundamental one.
And about competition for resources, you have to really-- so make a radical analysis.
So radical in Latin means to the root.
So you have to get to the root of the problem.
So the root of the problem, you can see it in the extreme case of India where there are so many people and so few comfortable jobs, and the route to the comfortable job is in education.
That's there's intense competition for tutors and all this.
Not because people want to learn, but because that's the means to actually not starve on the street.
So what that shows is that not everyone wants to learn with a tutor differential equations.
But a lot of people are learning differential equations with a tutor because that means they'll get a degree, which means they can work in investment bank, which means-- well, it used to mean that they would be comfortable.
So part of the issue is of redistributing the wealth in the society, so that you're comfortable whether or not you had a tutorial in differential equations or one-on-one tutorial.
Then, when the fears are taken away, people are much more likely to do things for intrinsic reasons.
So the people who really are interested in that, then do that.
The people who are really interested in teaching that will teach that.
And you'll have a redirection of social resources in much more productive way.
But that requires, for example, the value of, say, guaranteed income, guaranteed health care.
Maybe even guaranteed basic schooling for everyone, so that those things are features of competition.
Yeah
But the system based on competition is quite self-regulating, right?
I don't know about that.
So a good example-- is the system based on competition self-regulating?
I'm not sure about that.
I mean, the mortgage derivative market is a great example that-- actually, what happened was-- you can think of it in electrical engineering terms.
You had-- let's do it under continuous time.
So you had some sort of slow, pokey banking system with feedback loops and the poles were [INAUDIBLE].
[INAUDIBLE], nice decay constant.
And then, the competition to make more money quickly-- well, one way to make more money was to trade [INAUDIBLE].
What did that do?
It increased the gaming of feedback loops and [INAUDIBLE] the poles this way.
And eventually, some of the poles crossed the axis and you got huge nonlinear oscillations, which grew to-- huge oscillations grew to nonlinear [INAUDIBLE] and broke a whole bunch of the world banking system.
So it's actually not necessarily true that competition is self-regulating, because there are all these nonlinearities and feedbacks and delays in the system, there's no guarantee of that.
And one of the harms of the competition is it puts people against each other.
And what'd you much rather have is people working together and helping each other.
Because as studies quoted by [INAUDIBLE] showed, that actually-- if you want it to go a high level, [INAUDIBLE].
That's how it happens.
Not through competition and dog-eat-dog, but people cooperating and sharing knowledge.
And what we wanted to do, I would say in our educational system, is shift it more towards there and free people [INAUDIBLE] for creating the [INAUDIBLE].
Instead of [INAUDIBLE]
So we don't determine the pay of different people in the society based on the competition, what do you propose?
Do you propose that somebody else decides [INAUDIBLE]?
Well, we already have all kinds of ways.
I mean, you can have regulations.
You could, for example.
Say, I mean, income tax is a social way of deciding what's worthwhile and what's not.
You could say, OK, we're going to tax peoples' income the 99%, beyond 200,000.
[INAUDIBLE].
And we're going to use that money to pay teachers more.
That's a social choice because saying the teachers' salaries are this.
If you think it's a worthwhile thing to do we could say look, we think bridge building is really important it revitalizes the [INAUDIBLE].
We're going to pay people to do that.
Those are social choices, just like anything.
It doesn't have to be permissive.
All right?
So we can change our educational system that way.
The barrier to it is that our thoughts are limited by the educational system that we went through.
So in one way that's bad news, because there was this giant feedback loop keeping things the way they are.
On the other hand, because there's a feedback loop, the educational system affects the society and vice versa.
Well, it means you can work anywhere on the feedback loop you want that you think is interesting.
You could try to improve the society directly.
You could try to improve the educational system in [INAUDIBLE].
You can try to skip the feedback wherever you see it.
Insert something good in here.
And you improve things for everybody that way.
OK, so if you could fill out the question sheet.
[INAUDIBLE] put on the website who [INAUDIBLE] as well as referenced [INAUDIBLE].
And next time we'll have a summary of all of the second [INAUDIBLE] as well as questions from you.
Because I know I have a few questions that I have to answer plus [INAUDIBLE].
So we'll try to answer all your questions [INAUDIBLE].
Answers from lecture 10 to questions generated in lecture 9
OK, lots of questions from last time.
So I won't answer all of them because otherwise we'll probably spend the entire time just on the questions from last time.
Which maybe I should have anticipated since it's a central topic and one that connects to not just education, but directly to society, the topic on political barriers to educational change.
So I'll answer some of those.
But then I wanted to also give you a chance to ask any questions that have been unresolved that have come up in your teaching or that you think might come up in your teaching.
One of my old teachers, Don [?
Knuth, ?]
always reserved the last session for student questions.
And I think it's a really good philosophy to tie everything together, tie all the loose ends up.
So I call it your turn.
OK, but part of your turn was answering some of your questions from before.
Is social education a bad thing?
And so the comment was, I'm still not convinced social education is a bad thing.
Obviously it gives a lot of control to whoever gets to make the decision of what constitutes a good citizen.
But some of the social agendas in public schooling were good.
For example, diversity, recycling.
And that's true.
So I don't want to be misinterpreted as saying society is a bad thing.
I think society naturally educates people.
You pick up your values from people around you.
Your parents give you ethical norms.
Schools give you ethical norms.
And I think it's exactly what the question said, it said who gets to decide?
Is it some group of elites who decide or is it the whole society collectively deciding what is good for everybody.
And what's good for everybody may not be one thing.
It may be the decision is well, different people need or want different things.
And we think a good society is based on the idea that people have freedom to follow their interests.
And also, some required work that everyone need to learn, for example, reading.
But then what you read may be up to you.
So yeah, I don't think all social education is a bad thing at all.
I think all education is social.
There's no way to avoid that.
Inherently we're social animals.
And it's the question of what's in that education?
And it's the same thing, political barriers educational change.
It's not that politics is inherently a bad thing.
The question is, what kind of politics do we have?
And how can we make that better and more humane?
So for example, I think recycling is a good example of making things more humane.
OK, if competition demotivates some people does it motivate other people?
Does eliminating competition reduce maximum performance?
Other than the studies referred to by Alfie Kohn, I'm not sure if the answer to that is known.
I mean, there's certainly studies on how, for example, competition leads to a lot of waste.
The classic example in economics is suppose you-- and this was actually a real example-- give people 160 acres of land-- I think that was the a 1/4 plot-- or 640 acres of land way out in the west.
And the rule is whoever gets there first gets to claim that piece of land.
And you get there and you stick posts up and you put a fence there.
That's roughly how the Homestead Act worked.
Well, the problem with that is that it does cause a lot of waste.
Which is that tons of people might run out and chase after that land.
And make a whole bunch of effort buying train tickets, go on a wagon with their whole family out west.
But only one person gets the land.
So a whole bunch of people actually put in a lot of wasted effort.
So that's a classic example of how competition can be quite wasteful.
Other examples, where exclusive rights do that, or patents.
So as is the case, a patent's only given to one person-- or the inventor-- and you have 15 people or 30 people trying to invent something.
So the more the competition, in this case, the more waste is going to happen.
Because all these people are going to invest funds.
But only one person is going to get the exclusive right.
And the other people are just left high and dry.
So there's many examples where eliminating competition will actually increase the amount of social wealth contributed towards useful ends.
But to the direct question of whether it reduces maximum forms, I don't know if that's known.
The studies in Alfie Kohn's article pointed to how competition reduces high-level performance.
For example, complex cognitive tasks.
Maximum performance, probably, one would need to define that pretty closely.
And then one could do studies on it.
OK, so there was an interesting question about the understanding multiplication.
So the understanding multiplication example I gave, where people could multiply 6 by 3 and get 18 but then couldn't come up with a word problem that used 6 by 3.
Or they could, but the word problem was Johnny buys 6 apples on Monday.
He buys 3 apples on Tuesday.
So 6 times 3 is 18.
So that was, maybe, a bit unfair to ask students.
Because writing a story draws on skills beyond mathematical understanding such as imagination and language.
So it's true, it does draw on those skills.
But I don't think those are separate from mathematical understanding.
If you can't explain the mathematics that you in you, you don't really understand it.
When I was an undergraduate a friend of mine and I used to torment the math grad students.
It was sort of unfair.
And we were really curious about math.
We were fascinated by math.
And we wanted to know what people do in graduate school.
And only later did I realize we were actually tormenting them.
We weren't doing it intentionally.
We would ask them, OK, what are you working on?
Sometimes we had classes with them or we'd see them in the math lounge.
And they would say, oh, blah, blah, the theorem.
And we were undergraduates and young.
And we didn't really know what the theorem was.
So we'd say, well, can you give us an example?
And that produced just deer in the headlights.
Example?
Example?
That was completely foreign and not at all what they were thinking about.
It was completely out of their space.
And later I actually realized that that was actually a very fair question.
And if you wanted to test whether people understood what they were working on.
And I found that things I can give examples on I understand.
The things I can't, I don't understand.
Another example that is Feynman.
He said, I have always found that I don't understand anything unless I can give a freshman lecture on it.
Or equivalently, people asked him to give a lecture on something.
He said, no, I don't understand.
I don't understand that well enough.
I'll have to first work out a freshman lecture on it.
Then I can explain it better.
And so there's famous lectures of his where he's explained anti particles and spin.
And he finally got it to a level where he thinks it was actually suitable for a freshman, so first-year college students.
So it's a brilliant lecture.
If you search for the Dirac Memorial Lectures by Feynman you'll find that lecture there.
And it's fascinating.
So it shows the level of how hard work it is to actually understand something of that level.
And almost essential to that process is examples.
So examples, explaining, understanding, I think those are intrinsic to mathematical understanding.
So it wasn't an unfair test.
It was actually a test of high-level mathematical understanding.
One point was made, Swarthmore was not a women's college.
Yeah, that's true.
I mentioned that women's colleges were much more successful in producing women Ph.D. students in science.
And I listed Bryn Mawr and Smith.
And then I listed Swarthmore, which was not correct.
That was crosstalk between Bryn Mawr is and Swarthmore.
I grouped them all, Bryn Mawr, Haverford, and Swarthmore together.
So Swarthmore is a coeducational college.
And I meant Bryn Mawr and Smith.
And I was thinking of Wellesley as well.
So Wellesley is an all-women's college, unless they've changed lately?
No, OK.
I mean every once in a while colleges think of changing.
But they're still all women.
Now, people asked for references and further information about that.
So I'll put a link on the website.
But it's Barbara Whitten at the Colorado College physics department, she's done the most extensive studies on not just what makes for women being successful in science, but in general, what makes for students in say, physics majors.
She was the chair of the Colorado College physics department when I interviewed there.
What makes for a successful undergraduate major program?
So this is quite interesting for people interested in teaching.
Not just revising one course, but revising an entire department curriculum.
So she's done cross-institutional studies of the culture of departments and what facets of the culture are responsible for students continuing on in science.
And what she's found is that the same features that help women continue in science help everybody.
So I highly recommend her work.
And I'll put a link to her web page on our course web page.
Barbara Whitten.
I think it's W-I-T-T-E-N.
But it might be W-H-I-T-T-E-N, the Colorado College physics department.
OK, it's clear that I don't like No Child Left Behind.
But would I do if I were the Secretary of Education in this country?
Yeah, well, Hell would probably freeze over before that happened.
There's a long vetting process before people become that.
And you have to show your loyalty to pretty much every aspect of the current system before that happens, before you're appointed to such a post.
But if Hell froze over what would I do?
I think one of the fundamental problems with education in America is the vast inequalities in funding, in resources.
So the wealthy school districts have two, three, four times as much funds per student as the poor school districts.
And there's a good part of that.
I mean, the disparity is bad.
But there's a reason for the disparity.
And the reason isn't all bad.
The reason is that America education is a local affair.
It's controlled by local authorities.
So the constitution reserves a bunch of powers for the federal government, printing money, making war, various things like that.
Then it says, OK, then the states can do a bunch of stuff like, for example, regulate voting.
And then it says anything not explicitly given to the national or state governments is given to the people.
Or in fact I think it says-- this is the 10th or 11th commandment-- especially devolution.
It says any powers not given to the federal government go to the state.
If they're given to the state.
And if not given to the state they go to the people.
And so education is in that category.
It's nowhere mentioned in the constitution, education.
So it's not given to the states.
It's not given to the federal government.
So in America it's given to the local authorities.
Now the problem is that different communities have different wealth bases.
Because there's some very rich suburbs.
And for a while the cities were wealthy.
But then people fled.
All the wealthy people fled the cities in the 1950s during suburbanization.
So the centers of cities became very poor.
So there's vast disparities in wealth there.
And I think one thing that the federal government could do is actually just correct those.
So there've been lawsuits along those lines for forcing state governments to try to correct that because there is some role for the states in education.
I'm not sure where that comes from in the constitution.
But there are state boards of education.
And I think it's Vermont and New Jersey, there've been lawsuits in state courts to force the states to equalize funding across school districts.
So somehow mitigating the differential in property-tax revenues.
And I think the federal government could do that as well.
So those lawsuits are based on-- I'm pretty sure-- the 14th Amendment, equal protection of the laws.
Now, there's no reason the federal government couldn't do that as well.
So I would do that as step number one.
No strings attached, here, let's just equalize the resources.
And then we can worry about what we're going to do after that.
Whereas No Child Left Behind, what it does, it makes the resources actually, more unequal.
Because it tests all the schools.
And the schools that are going to do badly are the ones with less resources.
And then they're going to get punished for doing badly.
So I would take the opposite approach.
Is there a particular country's educational system that I particularly admire?
That's a good question.
Again, I think it's local.
There's local educational experiments that I admire a lot and occasionally larger-scale ones.
A whole country, not so much.
I mean, there's states in India, for example, for awhile Kerala-- I don't know if this is still true-- but had an incredibly high literacy rate, I think 98%, Which was much higher than anywhere else in India and matches literacy levels in the West.
And this has many benefits.
So for example, one study about population is that population growth is slowed down the most by educating women.
And Kerala actually, that 98% included women not just men.
So I think that's an educational system worth looking into.
And places that I admire, there are local pockets.
The free schools in the 1960s.
There's a few places like that still in America.
There's the Sudbury Valley School.
Maybe it's 50 miles west of here.
It's in the Sudbury Valley.
And that's modeled on Summerhill, one of the earliest free schools in England.
So actually, it has its good and bad.
So Summerhill, there's no required classes.
And there's no required curriculum.
And the students can go to class or not and can play outside.
So that would work really well.
I think that works really well if all the teachers are really inspiring.
Because then the students would be automatically drawn in to learn stuff.
But if the teachers aren't very skilled then you have problems with that.
So I think it worked mixed there.
But the potential is good and the freedom is really good.
And it teaches people to be responsible.
But it has its downsides.
So I have a personal experience with that.
Which is when I was three.
So I was born in England.
And we lived there for a few years.
And then we moved to New Jersey.
And I guess I was four maybe, my parents thought about sending me to England to go to Summerhill because this was 1973.
So it was in the middle of lots of political change in America.
Free schools were a big part of the cultural discussion.
And my parents believed in those values, which I'm glad they did.
And they thought, OK, let's see about sending Sanjay to Summerhill.
So we took a trip back to England.
And they visited.
And the way they tell me what happened, was that everything was fine except then they were wandering around with the teacher on the tour.
And they saw these marks on the door.
And they said, what are those marks on the door?
Because they believed in the philosophy of letting children do what they want to do and follow their interests.
But they said, oh, what are those marks on the door there?
And they said, oh, that's Johnny.
He likes to stand over there and throw knives at the door.
So then my parents thought, well, maybe that's just a little too unsupervised.
Especially if the child is across the Atlantic from us.
And it's not really easy to see how things are going and intervene if things aren't going so well.
So they decided not to send me to Summerhill.
But they sent me to progressive nursery schools locally as well.
And I have good memories of those too.
So yeah, I would say there's a fundamental point in there, which is there's a difference between freedom and license.
And so this is a distinction not often made.
So license is the bad pole and freedom is the good.
This is how I'm splitting the two parts.
So license is just do whatever you want.
Take drugs.
Be a prostitute.
Do whatever you want.
You're free.
Well, that's not really freedom.
You're licensed.
Whereas freedom-- in my mind-- has some intrinsic regulation on it.
You're free.
You're a free creature.
You have responsibility to use that freedom well.
And license doesn't have any of that responsibility.
So I think when one is designing curricula like that you have to try to make the norm such that freedom is promoted and not license.
Now, to some extent, you can't avoid a bit of license.
Because, for example, the Summerhill teachers say that when students come from regular schools where they've been basically competing with each other, basically feeling beaten down, some of them come to Summerhill and just do nothing for a year.
They just sit outside and play on the swings and in the creek, just for a year.
It takes them a year of detoxification before they are ready to go back into the classroom and learn things formally.
So you could say that's a bit of license.
But maybe that's a bit of necessary license.
Or maybe they're actually just discovering how to use freedom.
And it takes them a year because they never had any practice with that.
So I think it's important to keep those two pieces separate.
OK, so now there's so many questions.
OK so there's two that I think are particularly important to discuss.
So there was a comment that one of the Alfie Kohn readings was off putting because he made comments about people with different political views than his and he seemed like a left-wing nut.
And that, therefore, that reading shouldn't be assigned.
So yeah, I agree that if you have quite a different political view than Alfie Kohn it can be a bit off putting.
But actually, I think that's a good thing.
Now why is that?
It's because it's fundamentally important to find views that challenge you and read them anyway.
So as you probably guessed, most of my views are not mainstream.
So I do that all the time whether I like it or not, just when I open the paper, or if I happen to watch television, which I actually don't hardly at all.
But if I open the newspaper or read things, just right next to my office there was a teaching magazine.
I forget what is called.
Prism maybe?
I forget the name of the magazine.
But it's on the magazine display.
And I just was walking by.
And turned my head this way to read the title on the front cover.
And it said cash incentives are proving successful for promoting student learning.
So this was against everything I believed in.
So if I have the time I'll actually read the article.
But I'm surrounded by things I disagree with.
And I think that's actually pretty healthy.
So if you disagree with the article read it all the way through.
And the law professor Cass Sonstein from Chicago-- who's talked about as being a possible Supreme Court nominee by Obama-- he's written eloquently about the dangers of not doing that.
Because it wasn't so possible 50 years ago.
There was large-circulation newspapers and maybe two, three in each town.
And people read those.
There was a common discourse through that.
But now, everyone can get their own particular news on the internet.
Right?
You can get your own views from this small journal or from that particular newspaper or this person writing an article that you happen to agree with.
And the danger is of people just collecting into small little micro communities, where each community just agrees with itself and only reads things and discusses things that they agree with.
And that's very unhealthy for democracy he points out, which I think is right.
So I think it's very important to actually seek out things you disagree with.
And so if I happen to give you a reading that at first you disagreed with it's not such a bad thing.
And it belongs, for sure, in an academic curriculum.
Because that's one of the purposes of an academic curriculum.
The word university comes from-- it has Uni in it, one.
So the idea is it's one universe.
All ideas are open discussion, even ones we disagree with.
OK, so let's see.
OK, then the related one is, I find it off putting when I put my personal biases and opinions as absolute.
For example, programming-- and this is related to the last point-- programming LaTeX may be easier and more convenient for me.
But it's possible to develop familiarity with other software and systems that makes them close to equivalent, if not equivalent.
That may be true.
Though I think there's fundamental reasons to wonder about it.
Just because By not using a system such as LaTeX or Tex.
What's common about all of them is that you program them.
And by not using a system like that, you give up the ability to program.
And that is a huge loss.
You can see it, how much can computers do?
The way computers are so powerful is because we program them.
And the fundamental recipe for programming is-- at least when I teach it-- this-- So this is how the fundamental idea of programming I think.
So you do something and then you do it again.
And when you're doing it again, you're like, oh, I just did this.
Maybe I should write a program.
Nah, I'm too lazy.
But then the third time it happens again.
And you're like, OK, bloody hell, this time write a program.
OK, so basically the recipe is the third time you're about to do something, don't do it.
Write a program to do it.
So now you have a little mini program that you can reuse.
Now it's a new module.
And you can use that module to build still new programs.
You can build a huge power abstraction.
So for example, in the figure-drawing program you can actually write programs that generate trees.
And then you can write a five-line script.
And you say, OK, feed that through my tree-generating program.
And it'll make a tree that you can use in your figures, in your talks, or in your books.
So I'll post a few examples of that.
The input as five-line tree and the output PDF.
So the first time you write that program it takes longer than doing it graphically, no question.
Which is why this second part happens.
Because you do it the slightly hard way the first time.
You don't realize the second time you're like, oh, there's something in common here.
[BUZZER SOUND] But it's too late.
I just have to do it the slightly hard way.
The third time you think, OK, finally I'm going to do it with the program.
So I think there are fundamental reasons to think that if you can program your documents you'll also be more efficient.
Now, that actually may end up not being true.
I suspect it is.
But it may not be.
But even if it isn't take it as not an absolute.
But as a suggestion to try in your own work and in your own writing.
And I don't mean it as an absolute.
OK, so now there's 10 more questions.
But I'm going to leave them for now.
And if at the end we haven't used up all the time with questions I'll continue with some of those questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
What I want to do now is basically summarize the theme of each of the sessions that we've had, partly as a way of reminding you of all the things we did and also jogging your memory or any question that you might have about that.
And then it'll be your chance to ask questions.
So I'm going to do it, so I'll think aloud here.
I'll do it in a way that I can leave them up the whole time.
So let me do it here.
I'll do it on these two guys.
Yeah, I'll do it here.
So our sessions and the summaries of them-- so session number one was the introduction.
But I'll start with session two, was teaching equations.
Now the fundamental idea I wanted people to learn in this one is to help students learn equations in big chunks.
So doing derivations is generally antithetical to that.
Because students will just see the little symbols, and they'll see a ton and ton and ton and ton of symbols.
And they're chunk size is much smaller than you, so you will overload their short term memory.
And they won't really be understanding, they'll be just trying to remember and write as fast as they can.
So is that visible, the orange?
Yeah?
Great.
Oh yeah, you're a good test.
So teaching equations, you want to try and chunk the equations, because that gives meaning for the students.
So, for example, what does each term mean?
Why would you expect that term to show up in the equation?
Those are the kinds of things that a book generally does not do very well.
And that's something that you, as a teacher, actually have a lot of value to add.
Now I just learned about a book that does try to do this, which is called A Student's Guide to Maxwell's Equations.
Does anybody know that book?
Yeah, you know it?
So, it looks fantastic.
I'm number seven or something in the queue to get a copy from the MIT library, which shows how popular it is.
But the way it's organized-- well, first of all, it's sold an immense number of copies for a technical textbook, something like 7,000 or something like that.
So it's a physics best seller.
It's not an Agatha Christie novel, but it's really well done.
And each chapter talks about one of the equations.
So there's four, roughly, Maxwell's equations.
And there's four chapters, one per equation.
So it's helping the students chunk just by the very structure of the books.
But most books don't do that, aren't that well or that interestingly constructed, and that thoughtfully constructed, so it's something that you, as a teacher, really should do.
The next one-- misconceptions.
So why did we spend a whole session on misconceptions?
Well, it's fundamentally important, misconceptions, because if you don't know where your students are, you have no chance of bringing them to where you want them to go.
The example that's classic, that's [?
probably ?]
started this whole line of research about misconceptions, was students' understanding of force and motion.
And basically, what you find is that most peoples' intuitive understanding of force and motion is f equals mv.
If you have an object, you stop pushing it, it stops moving.
Force and velocity are connected.
If you have a heavier object, you have to push it harder.
If you have a lighter object, you push it less.
So force and mass are connected, so it looks like f equals mv.
Whereas, we would like to teach them f equals ma, Newton's Second Law, force equals mass times acceleration.
So there's a fundamental difference there, and if you don't take account of that, you'll end up producing just rote knowledge.
They'll say OK, well whenever I happen to be in physics class, I'll use f equals ma, and I'll just solve problems symbolically using that procedure you've told me.
But I don't really believe it, or understand it, or use it in my own reasoning, so my intuition is no longer available for reasoning.
So by taking account of students' misconceptions and helping them come to an intuitive understanding of how things really work, you're actually making them much smarter.
So this is knowing-- so that's knowing how students think.
So this is pretty much opposite to behaviorism.
Because you're saying, well, I really want to peer into how people are reflecting, and how their cognition works, what's going on under the hood.
And behaviorism says, well, no, all you need to do is look at the behavior.
If they're solving problems correctly, that's fine.
Whereas actually you want to know, well, why did they solve it that way?
Is it because they really understood it or not?
Did they just say 6 times 3 is 18, because they were told, or they memorized it?
Or was it because they really understood it, in which case they could make a word problem related to it?
So knowing how students think, you're actually part of the cognitive revolution.
And you're actually connecting to students, directly and making them much smarter.
Homework and exam questions-- so constructing homework and exam questions, the main theme of that was Bloom's Taxonomy.
So this was a hierarchy of goals that you could have.
So the full name is A Taxonomy of Educational Objectives.
And the ones we're talking about, if you look at the full title, it's the cognitive domain.
There's the cognitive and affective.
Affective means related to feelings and emotions.
This is the cognitive domain ones I'm talking about here.
So are you asking for just comprehension?
Are you asking for students to analyze?
Are you asking them to synthesize knowledge?
What is the level of cognitive activity that you're asking for?
And generally, you want to mix levels.
So you don't want to just push the students off a cliff and say-- the first day in thermodynamics class, discuss how the Second Law of Thermodynamics applies to the entropy of the universe?
That's just too hard.
You've pushed them way off the cliff, and you jumped way too high in Bloom's Taxonomy.
You also don't want to just give them pure comprehension problems, because they remember the rule that you're trying to teach.
Rather, you want to mix them, and Bloom's Taxonomy gives you a structured way of seeing what level your questions are at and making sure you mix them, so that you prepare students comprehension questions, so that they can later do evaluation questions.
Course design-- so for course design, the big idea is big ideas.
So you want to organize your course around some kind of large ideas.
And ideally, those large ideas will transfer outside of the course too.
And if a whole curriculum is organized around a few large ideas, for example, waves and oscillatory motion, you could also organize a whole bunch of a physics major around ideas like that, or conservation.
By giving structure to the curriculum, you turn it away from just a series of unconnected facts, like we did this topic, that topic, that topic, that topic, actually you give it form and organization.
And then it's much more likely to be built into long lasting chunks in the students' minds.
So the next one was interactive teaching.
The sixth session was interactive teaching.
And the main theme is that questioning and reflecting lead to long lasting learning.
So I showed you three or four different time scales on which you could be interactive.
A short one is when you ask a question, you wait 5 seconds.
Or actually some of you suggested seven, which is probably better.
It's just harder to do, but it's probably better.
So you wait a certain amount of time to allow people to actually formulate questions.
So that promotes interaction on a short scale.
You can ask short questions, one or two minutes, two or three minutes, with a multiple choice right in the middle of lecture.
It gives you feedback.
So it's related to the misconceptions one.
You're learning how students think, just as they answer the questions.
Or on the same time scale, you can ask the feedback questions from this sheet at the end of the lecture.
So that's also promoting a space for questions.
And you can actually see how successful that is.
There's so many questions, actually, we could spend two more sessions answered all the questions, which I'm glad about.
Because questioning and reflecting-- that's how people make knowledge their own.
So the formal name for that is constructivism.
But what you're doing is you're helping people construct knowledge.
And you can do this on a long scale.
Like for example, for the whole lecture, you can use a question like the word blocks, which I showed you earlier.
You can actually do that for an entire lecture.
And actually, we just did that for entire lecture in my Art of Approximation class.
Because the word blocks leads to xylophones, and I brought in a xylophone.
And we can talk about how the pitches of the thick and thin wood blocks relate to the pitches of shorter and longer xylophone slats.
So rich examples like that-- one doesn't have that many of them.
But when you do have them, you can actually build an entire session around them.
And it produces all of these good things.
Seven-- lecture planning and performing.
So for the planning of the lecture, just like for the course, the planning of a course, you want to organize it around some kind of big idea, some kind of thing that gives it structure.
The whole lecture, you want an objective.
For example, by the end of the lecture, I want students to be able to explain the origin of the terms in the Navier-Stokes equation.
So then you would choose your activities in lecture around this goal.
But that goal, actually just like the longer scale one for the whole course, gives structure to the lecture.
And you know what to put in and what not to put it.
And the lecture planning blank sheets, or the templates I gave you, I posted, you can use those to that end.
And that's for the planning part.
And for the performing, I would say the main thing in performing that people don't normally think about when they think about teaching is timing.
So if you want to create little bits of suspense and interest, little bits of tension, that are then released.
And you want to do that on multiple scales-- short and long.
The interactive teaching questions from here help do that, especially if you have a demonstration.
Because people want to know, is the thick wood block going to be higher pitch or is it going to be a lower pitch?
And you'll find that when you do things like that, the entire room is dead silent, because people really want to know.
And the way you structure the questions can promote that.
By having people vote, they've now made a public commitment, which then activates the cognitive dissonance part of people's mind.
By making a public commitment, they've now made their internal state more towards the public state, which is oh, I want to know what's going on.
And they really do want to know what's going on.
So timing-- you can get a lot of timing ideas from the interactive teaching methods.
But timing is really important for performing.
Imagine jokes where people tell you the punchline first, and then they tell you the rest of the joke.
They don't go over too well, and most comics, at least most successful comics, don't do that.
But we usually do that with our teaching.
Blackboards and slides for teaching-- so the main concepts to think about there are the chunk size that students have and how many spots they have in short term memory.
So with the blackboard, you have many advantages for free, which is that on the blackboard, you don't have to use too many short term memory slots.
You can put the entire session all on one giant set of blackboards in the front.
And you can offload a lot of the work onto something that's in the visual field and present for everyone, all the time.
Because remember, their chunk size is much smaller than yours.
So you might see the entire lecture as one chunk.
But obviously, they don't see it that way, or they wouldn't be in the lecture.
They're the learners.
And then when you're doing the slides, if you're going to use slides for teaching, to mitigate some of this problem of not having enough short term memory slots, generally not enough space, well, you could try to get four slide projectors or three slide projectors.
But it's pretty hard to do.
And it triples or quintuple or [?
nontuples ?]
the time to prepare the slides, because you have to synchronize three different slide presentations.
So at least use assertions at the title and then visual evidence.
Visual evidence is much easier to remember.
And the assertions help people know what to look for.
So they're not spending all their cycles, which they don't have that many of, because their short term memory's been filled up.
They're spending their cycles trying to understand the visual evidence.
And I posted a set of example slides with the [?
text ?]
source.
There was a request for the [?
text ?]
source, so I posted that, as well, showing this method of presentation for the factorial, the logarithm of the factorial.
And then the last one was [INAUDIBLE].
So maybe the summary could be-- the comment that was on one of the sheets was is there any hope?
[LAUGHTER]
And I think, well, that question-- it's coupled to another question.
So the main theme from that is that social and educational change are coupled.
So if there's no hope for education, there's no hope for society.
Or alternatively, if there's hope for society, there's hope for education, and vice versa.
So the main positive lesson from this is that by improving society, you improve education, and also, by improving education, you improve society.
So that's one really good reason to want to be a teacher and to make teaching a part of your career, as I'm sure many of you are thinking about doing.
So those are the short summaries of the nine sessions.
And what I'd like to do now is give you a chance to ask any questions you have.
We'll, of course, have our-- let's see, in 10 minutes, at 10 o'clock, we'll take a 10 minute break.
But before that, I want to start the question session.
So the way we'll do that is if everyone could just look at your index card that you brought with a question, or it doesn't have to be an index card, it could be a sheet of paper, and spend one minute thinking about something that puzzles you, you wonder about, you're interested in, you'd like to know more about, about anything related to teaching, and check in with a neighbor or two.
And then we'll just start doing questions.
And then we'll have our break at 10:00.
I'll leave all these guys up here, and I'll mark your questions.
[INTERPOSING VOICES]
Take another 10 or 20 seconds to formulate a question or formulate your thoughts, and we'll start
So--
[INAUDIBLE] yeah?
Implicitly throughout this semester, we've been thinking that our audiences, like the college age audience-- so if you're teaching older people, [INAUDIBLE] things that you have to do differently?
Good question.
So let me repeat the question, and then also, that will give me a chance to bring in one of the other comments from the question sheets.
So the question was what about teaching older learners, so people who aren't college age, so middle aged people?
Do you have to do things differently?
And now, before I answer that, let me say why I repeat the question.
So their comment was that it's a good idea to repeat the question, because OCW asks that I repeat the question.
Because the pick up, the microphone pick up, I guess, is here, and it picks up me much better than the audience.
So it's good for that, but it's also good, as the commenter pointed out.
It's good, because it shows the students who are asking the question that I actually value their questions.
Because I'm actually telling everybody, look, this is something we should all be thinking about.
So it brings the whole class together around it, and it also gives me time to think about the question.
Although, in this case, I didn't actually use any of that time to think about the question, so let me think about the question now.
So I'm not sure it actually gives you that much time.
It gives you one or two seconds.
But actually, if you want one or two seconds, it's best to just sit there still and think for one or two seconds.
So that's what I'm going to do now.
[LAUGHTER]
So now I waited actually four seconds just to show that it's possible to do that, though difficult.
[LAUGHTER]
So there are some things that are different.
And I would say one of the main differences is not just the misconceptions, but the conceptions of learning that people have.
So for example, you might be teaching people who just failed out of math in school and never did math again, and now you want to help them learn math.
Or you want to help them learn reading, and they always thought of themselves as terrible readers.
So now they've imbibed the lesson of the blue eyes brown eyes experiment, which is that they're bad at this-- for whatever reason, it's their fault.
And we're very good at that.
We're very good at passing blame.
So why didn't people learn physics?
Well, physics is a very hard subject.
It's not that we teach it terribly, it's a very hard subject.
So basically, we're saying, blame the victim.
I know that's very convenient.
But after a while of that, the victim actually starts to get Stockholm Syndrome.
So they start to identify with their captors.
And they agree, yes, physics is a really hard subject.
I never had a head for physics.
So as an experiment for that, next time you go to a party, ask people what you do-- I mean, obviously not a party of other graduate students in science and engineering, because that test wouldn't work here.
But just for example, my office mate, when I was a graduate student, he used to live in Hollywood, in Hollywood, itself.
So he used to go to parties, and his roommate was a script rewriter.
So his roommate would actually-- scripts that were considered too bad even to make movies out of, but they really thought they should make a movie out of it, because they thought they could make a lot of money if they just rewrote the script a bit, so like D-grade scripts that needed to be made C-grade, his roommate rewrote them.
So he had a lot of Hollywood connections.
So he had a lot of parties with people who weren't from Caltech, which was good.
So go to a party like that, and when people say, oh what do you do?
Say, oh I'm a chemistry graduate [?
student, ?]
I teach physics, or I'm a TA for this class, or I'm working on a Ph.D. in mathematics.
And just watch how fast people run.
[LAUGHTER]
So if they're not very polite, they'll say oh, what do you do?
Right?
But if they're more polite, they'll actually say, oh, yeah, I always was terrible with that.
That was never my thing.
And this is so prevalent.
So now, what do you have to do in that case?
So that's their misconception.
And so number three up there, you have to really take account of that.
Their way of thinking about math and physics, the way that it was taught to them completely did not work, and it just produced fear and phobia.
So now what you have to do is you have to go around that.
And so one of the ways actually Brian Butterworth, who actually studies this-- he's a professor of neurophysiology, in London.
And I mentioned his book a couple times, I think, it's called The Mathematical Brain or What Counts, depending on whether it's the English or American title.
So math phobia a lot caused by people having a fear that there's only one way to get the right answer.
So they think they're walking across a tight rope, and if you make it one misstep, then you fall into the abyss.
And so, of course, if you have that conception of mathematical reasoning and problem solving, you're going to have a lot of fear.
So what you have to do is you have to show people, no, actually-- right over here.
So if this was their model, that if that's the one way to get to the solution, actually you want to show them not that, and show them even dead ends.
Not every method of solution will work out.
Some might just end up there, and some might end up there.
And some connect to other methods and get to a solution.
And even maybe, that's one solution, but maybe there's two.
So you want to show them that there's robust paths across the river, let's say.
You're not trying to, for example, jump Niagara Falls, where one misstep and down you go.
So that's a way of reducing the math phobia.
Now that's a particular example, but that's an example of how you have to think slightly differently for the adult learner.
The reason is that they've had longer time to internalize the misconception.
The high school student, for example, they haven't necessarily formed their own view of them self.
And it's less true at the college level, that people are starting to form their view of self more and more, but it's still changeable, especially in America.
And that's one really excellent part of the American university system.
So you may not realize it if you come from America, but in almost every other country in the world, when you go to university, you pick your subject and that's what you do for the rest of the university time.
So England, for example, it's even before that.
If you want to do physics at Cambridge, you have to have done, for the last two years of high school-- basically everyone who did physics at Cambridge did physics, math, chemistry, and further maths, I should say.
So further maths are just more maths-- basically like DE calculus.
so they did, for the last two years of school, only math and physics and chemistry courses.
And then they go on to university, and they do just physics.
And Cambridge had a radical innovation on that, which was that in the first year, you didn't do just physics, you did physics, math, chemistry, and maybe a fourth subject.
So that was considered really very progressive.
And it was compared to the rest of the educational system.
So in Europe, in general, there isn't that freedom.
So in America, you still have it.
So the students in college are still going to be quite different from the people who've basically fixed their view of them self as bad at math or bad at whatever.
And that's one of the things that you change when you're thinking about adult learners.
I'll take one question back there, and then we'll have our break
So towards the beginning of the class, we talked about the experiment in math teaching in New Hampshire
Benezet, the experiment?
Yeah, the Benezet experiment.
I don't think we ever really got to figuring out what happened to that.
[INAUDIBLE]
So what happened to the Benezet experiment in New Hampshire?
So Louis Benezet was superintendent of schools in Manchester, New Hampshire from 1924 to 1938.
And so what happened?
If it was so great, where is it now?
So I might have mentioned that I actually went and did research on what happened to it.
And I went to the Manchester, New Hampshire school board and just spent a whole day in their archives reading all the minutes of all the school board discussions about it and all the votes about Benezet and whether to reappoint him as superintendent or what to do about the curriculum.
And basically, what happened was two things.
So first of all, there was extensive studies done showing that the curriculum was very successful.
So one study was by-- so Etta Berman, she was a teacher in Manchester in the program.
And then she did her master's in education I'm pretty sure at Boston University.
Yes, I'm sure it is, because you can find her thesis in the Boston University library.
So her education thesis was an assessment of how the students did in that experimental program versus the regular program.
And the students in the experimental program were just much better.
It was just totally clear.
So now what happened?
Well, what happened, partly, I think a lot of it is what Alfie Kohn article about "Not for my Kid: How Privileged Parents Undermine School Reform."
So there was huge opposition from the privileged parents.
And so I interviewed some of the people who had those privileged parents and who had taken that curriculum 50 or 60 years ago.
And one of them was later mayor of Manchester, and he said his parents just hated the curriculum.
So generally, the people whose parents were foreign, for example, French Quebecois who had come to New Hampshire, had to work in the mills, the non-English speakers, the lower middle class, it was OK, and it was fine.
And the ones I talked to from that background loved the curriculum.
And then the few who really hated it were actually socially quite powerful, and they came from the predominately English speaking families.
And they hated that he wasn't, for example, giving homework.
Because he said, if you have eight hours of school, what more homework do you need, which I think is actually true.
If you can't teach people in eight hours a day, that's ridiculous.
That's 40 hours a week.
So why are you giving them still more homework?
So he didn't have homework, he had them actually learn in school, which is one of the benefits of interactive teaching.
If your lecture is no longer dictation, you are actually learning, you don't have to do a lot of stuff out of school.
So there was opposition, there.
And what happened was the votes in the school board went something like this.
So this is t. So this is 1938 and 1924.
And this is the votes in favor, minus against.
They went roughly like this.
So the school board, I think it had 15 people on it, or 14 people.
It depended, sometimes one person wasn't on the school board.
So the first votes were something like 13 to 1, so in other words, plus 12.
And then, over here, 1924, when he was first appointed, and then the votes went like this, and then it started going down like that.
And by 1938-- so there was votes taken every three or four years, or two years, whether to renew the appointment, because that was the term.
And by here, the vote was 6 to 6 right here.
Yeah, that's right-- it was 6 to 6 in 1937.
And then what happened in 1938 was they didn't vote to not take him on, but what they did is they voted to use a standard textbook.
So they voted to use a standard textbook in the curriculum, and then the curriculum was then declared incompatible with the standard textbook.
Of course, the standard textbook was pretty terrible.
Well, I don't know the textbook, personally.
But I'm sure, given the state of standard textbooks, then and now.
And it certainly wasn't the progressive, understand the algorithms approach that Benezet had.
So then the textbook was instituted, and the curriculum was basically killed off, like that.
So then Benezet left for a professorship of education at Dartmouth.
And then, after Benezet left-- so this is Benezet-- Benezet left, the new superintendent put in a, basically, drill and kill curriculum with lots of testing and the standard now.
There was No Child Left Behind 60 years before or 70 years before its time.
But the curriculum was very successful, and it was basically killed off by opposition from wealthy parents and, to be fair, Benezet didn't help his case.
So a couple of the students told me that actually when the students wouldn't answer things in class, he would sometimes make fun of them.
[LAUGHTER]
Now that's terrible.
I mean it's terrible just on moral grounds, but also it's terrible political strategy.
Because now, the kids will tell their parents, and if the parents needed ammunition, at all, there they have it.
And so Benezet was just arming the enemy too.
Yeah?
If, presumably, most or all of us would be teaching college level courses, we don't have to worry about parents.
[LAUGHTER] [INTERPOSING VOICES]
Good question
[INAUDIBLE]
Hey
Is it a comparable force to the parents [INAUDIBLE]?
Yes, there is.
Good question.
So is there a comparable force in college education to parents with privilege?
Yeah, and unfortunately it's one's colleagues on the faculty.
The reason is because they were the ones who did well by the old system.
And so now a change is likely to be misinterpreted or maybe rightly interpreted as well, this isn't for the benefit of the top 5%.
So we can't do it.
So for example, interactive teaching is often criticized as well, that's really useful for the people who aren't learning anything, but the top 5%, it will just slow them down.
So we can't possibly do it.
So the wars about interactive teaching turn on a lot of that question.
And that is a very similar force.
Because people are saying well, it's those people who are just like me, because they're the ones who are going to become future faculty.
And those ones aren't going to be benefited by the system.
Now I think, empirically, that's not true.
I think they are actually benefited, as well.
And Benezet found that too.
It wasn't that some people did worse.
They were all doing better.
But it's the intensity of the response that you get to things sometimes in that vein shows that its an underlying social force and not just purely a cognitive reason.
Is that what you were thinking about?
[INAUDIBLE]
OK so 10:08, so 10:18, we'll start again with more questions.
[INTERPOSING VOICES]
So I know there are several more questions, because I already have two in the queue.
Go ahead
My question is--
And then [?
Lortis.
?]
Yeah?
What do you do when you're confronted with a question you don't know the answer to
What do you-- I don't know?
No, sorry.
[LAUGHTER]
No, no, no.
The question was, what do you do when you're confronted with a question that you don't know the answer to?
And my answer was I have no idea.
No, there's good answers to that question, and hopefully, I'll give you one of them.
So I would say the number one thing is not to bullshit.
That can only end-- at best, it won't end in disaster.
But it can end in disaster.
The first thing-- so if you don't know the answer to it, but you have to think of what's going to get triggered in you?
So if it's something, for example, really related to the material, you're more likely to get triggered and think-- [GASPS] Excuse me.
Oh, I should have known the answer that, I must be a lousy teacher, how will they ever respect me again?
So if you remember the discussion we had last time in political barriers about the three levels at which conversations are carried on, so this was from difficult conversations, there's the factual level, the emotional level, and the meaning level.
So the fact level is that someone just asked you a question, and you don't know the answer right away.
The emotional reaction, and the emotional transmission, is oh my God, I feel worried and nervous.
And then the meaning level is oh, what does that mean for me?
Oh, it means that I'm a bad teacher.
They found me out.
I'm really not supposed to be here.
[LAUGHTER]
So that same thing happens with students when you ask them a question, and they don't know.
So you want to create structures where that bad meaning, isn't triggered.
So you want to just pause, first of all.
The reason is, if you don't pause, then you're more likely to act out, based on the emotion.
So I've seen this happen several times when I was a graduate student.
So this is sometimes when the person didn't know, but in general, when they were somehow challenged by a question, so either way, whether it's because you don't know the answer, or because the student's tone has some kind of cheekiness to it, and you feel a bit of insolence, and then you think, I'm the teacher God dammit, how dare you talk to me like that?
So that emotional reaction, or that challenge, that worry, that anxiety about your authority will produce a bad response, in general.
So if you just, right away, respond, you are likely to do bad things.
And this is a thing I've seen happen a few times, is where you put the student down, you compete with the student, you say things like, well, how could you not know that?
And you dismiss the question.
So if you pause for just one or two seconds, even an instant after you practice it, just the pausing to realize oh, wait a minute, I just had an emotional reaction happen, I felt in my body, I got a bit sweaty, my palms sweated, whatever it may be that your particular reaction is.
I feel it often in my face-- my face gets warm.
If you remember, way back when we were doing-- was it the wood blocks or the cones?
No, it was the cones.
I was dropping the cones.
And there was a question when it was time for questions and discussion.
One person said, oh, the answer is block, and I've seen you do it before, and I just remember just feeling flushed.
I was like, this is not the time to answer that question.
So it's the same thing.
So that's lesson number one is to pause.
And then you can think and regroup and have a reflective response.
The reflective response may be, oh, that's a really interesting question, I'd never thought about that, but I'm going to think about that tonight, and I'll tell you the next time.
That's perfectly fine.
Another one is you can try to figure out the answer together.
You say, well I'm not sure, but let's see if we can figure out the answer in the next minute or two.
If not, I'll work on it in the evening or work on it tomorrow and tell you.
And then you can try it together, and students have the advantage of seeing how you would reason about a question.
And it helps if you do that intentionally.
Because there's a general rule of thumb which is that one's facility with equations declines with distance to the blackboard.
So the closer to the blackboard, the lower the facility of the equation.
So if you just leap to the blackboard and start writing, you're likely to feel too nervous and actually just mess it up just because of that.
So if you say, well, let's actually think about doing this together, think about what you might do, and think about some approaches, then go to the blackboard and maybe start working.
And when you have your out, it doesn't work in two minutes, you say look, it's a good question.
I clearly need to think about this some more, and I'll do that.
And I'll come back to you.
Does that help answer your question?
Yes, because we were just talking about just having these discussion with our students.
And I feel like perhaps there would be somebody who would point out a different way of doing something.
And then you aren't really prepared to talk about that with them.
[INAUDIBLE]
Right.
So if you're going to do interactive teaching, you're going to have lots of discussion.
So people think, oh, interactive teaching means you just let the students do everything, you kick back.
No, it's actually much harder.
Because you've changed it back from basically playing a prerecorded tape, which is what a lot of lecturing is-- you just get out and read the book, that's what a lot of my lecturers did-- to now it's a stage performance-- a stage performance in the old days of Elizabethan theater where the audience interacted with the stage.
So that's much harder.
You have to, quote, "Know your lines."
Not that you're going to say the same lines no matter what-- you have to really feel the lines.
You have to really understand the feel.
So it is harder.
And you're going to be producing lots of discussions.
And at a place like MIT, the students are very curious.
And they're going to think, oh, well, what about this?
Well, that's fine.
So you want to use Aikido.
So if someone suggests something you haven't thought of, don't dismiss it.
You say, oh, I hadn't thought of that, that's a really good suggestion, which then, usually, maybe, I'll use that next year when I teach the course, thank you.
So if you just think about the students as your allies, then your reaction that you'll intrinsically produce will be much less confrontational and much less worried.
So I'll give you one example of where I was very, very nervous and what I did.
So I was lecturing thermodynamics, and there was 200 students.
And I was doing the Carnot cycle, so this was the last lecture.
So we were doing the Carnot cycle, and I drew it up on the board.
And I'd encouraged questions throughout the whole lecture, throughout the whole lecture course.
So, of course, someone raised their hand and said, oh, excuse me, I think that this box is not right.
They didn't know what was wrong, but they said, oh, that box isn't right.
So I looked at it, and I thought, oh my God, they're right.
So I'd now just drawn rubbish on the board for 200 people, and I had a bunch of my colleagues that day who happened to decide that was the day to come visit my lecture.
[LAUGHTER]
So I think that was probably why I was a bit nervous to begin with.
And that probably contributed to me writing down the wrong diagram.
So I realized I needed some time to actually think, so I just said, you know, you're right.
Let me just think.
And I just turned my back, for a whole minute, and I sorted out what was wrong.
And then everything was fine after that.
But I still remember that.
I still remember the feeling in my body, just that sinking feeling of oh my God, disaster has happened.
I've just been exposed.
And really, I should've been one of the students in the class.
So pausing is immensely valuable.
[?
Lordis?
?]
Follow up to the question that was said before, how do you effectively argue against [INAUDIBLE] to the top students?
Right, so often when you want to do so something more interactive in your teaching, the counter arguments are well, you're going to harm the top 5%.
Well, one counter argument-- so first of all, it doesn't often help to counter argue cognitive reasons if there's an emotional underpinning.
So generally, I try to step aside from those arguments, unless the people have a sense of humor, I find.
If the people have a sense of humor, then you can present things in a humorous way that challenge their view, and they can take it on board.
But if they don't have a sense of humor, or you don't have a good way of presenting it that way, I find it's just wiser to step aside and see if you can find some way around it.
Maybe try half of what you wanted to do, or say, well, yeah, I worry about that too, a bit.
So let's leave the main course alone that way.
Let's try an experiment and see how it goes.
Maybe we can agree on some questions that all the students should be able to do to try to make it cooperative.
So that's one thing.
But if you do want to go directly into cognitive discussions and reasons, one is, and this has scope for making humor out of, [?
I've ?]
said, what's a top student?
The top students, the top, quote, "5%" are the top 5% by the current system of testing and homeworks.
So they're are often the top 5% in regurgitating canned formulas.
Or by the time they get higher and higher and more selections happened, they're good at pattern matching and solving problems they don't even understand.
I mean certainly, the final exam in Cambridge that people took in their final year-- if I just sat down, I would not have passed it in the physics department.
And so there were students who did really, really well on it.
But I'm sure I understood more physics than them.
And it's because if you train for those kind of questions, even if you don't really understand what you're doing, you can actually answer them.
So now the top 5% at that, well is that so valuable to preserve?
Maybe actually we should change the rules of the game.
So now, if you just say it as directly as I've said it, then you'll trigger the meaning of-- well, because a lot of times, those 5% became the people you're talking to.
So you have to maybe do it slightly differently, and say, look, what is the thing you're most concerned about, about the students not knowing, whether they're the top 5% or not?
And a lot of times, they'll just give you a big, long list.
On their own, before you even say it, they'll say, oh, they can never use anything outside of the class.
So people will just outright concede that, not even concede it, they'll say, yeah, it's so terrible.
And people will talk about this all the time, and say, look, we all agree that.
Well, let's see, is there a way that we can make people really good at that?
Because even the top 5%, I find, are not like that.
So now you've opened a door.
And you can have a shared discussion based on something like that.
So that's how you can reach the 80% who are not sure but are willing to listen.
Yeah?
I was wondering what you think of the surveys that they have at MIT by which the students evaluate the professors and teachers, and also, how to interpret the results of those?
Oh good, I'm glad you said that for several reasons.
So one of the other reasons is that this course has one too.
[LAUGHTER]
So I'll put it on the course website just after class.
So 595 is one of the course numbers.
But it's also 6.92 or something, so all course six classes have an online evaluation system.
So for everybody, we're doing it that way.
So everybody who is registered for the class is listed as a possible survey filler-outer.
So I encourage everyone-- I'll put the link on the website.
Just click on the link, and if you have your MIT certificates installed, it'll just take you to the online survey.
So I think the surveys are very useful.
Now the numbers are a ballpark-- well, were things working or not?
But I think the most useful part of the surveys is the comments people make.
So again, I encourage you to make comments in the comment boxes, wherever they are.
For example, there's one that says, what would you suggest for people next year?
So part of that I'm going to learn by reading these.
So I'll crib good ideas from how you would do this course from your homeworks.
But the survey will actually help with that.
So I think the surveys are really useful, if used properly.
I don't think they're very useful for saying, you get tenure, you don't get tenure.
Because people just use the numbers for that.
I don't think the numbers are super reliable.
I mean seven is probably significantly better than four or five, but there are a lot of ways to get six and a half, and six or five.
How did that happen?
Well, you want to really look into the course and see that.
And the course survey doesn't really address that super well.
But the comments give you space, if you really listen closely, to understand it.
So I think they're quite useful.
But what I'd really like to see-- because those are surveys right after the class, and my goal, what I've been trying to stress this whole time, is how can you construct teaching for a long lasting learning, questioning and reflecting, ideally for a long lasting learning?
Well, if it really lasts long, people should remember something a year later.
So I think actually the policy should be changed to do feedback sheets at the end of each lecture, so you get the quick feedback you need as the lecturer.
And then by the end of this course, you have a lot of feedback, and you can really improve things next time.
And then do the survey a year after the course is finished.
And if people don't remember anything from the course right away, that's a pretty important piece of feedback.
So I fear though that the results from that kind of survey would be quite distressing.
Because you'd ask things like, well, which of the ideas from the course do you use?
And people have done surveys like that in physics majors.
So they're surveys of physics majors in the job, whether their in research or in industry or finance-- well, before that ended-- what do you use from your physics degree?
And very rarely do you find people saying, if you just average across all the jobs, Maxwell's equations.
It's just not on the list-- is very low on the list.
It's things like general problem solving, quantitative skills, working in groups.
Now it's not necessarily true that therefore we should just dump all our classes and teach people how to work in groups.
That's not what I'm saying.
But it's useful to know what people are doing with the things you teach them, so that you can take that into account.
So that would be my ideal-- surveys one year, five years, 10 years, later.
And there are a few of those at MIT, alumni surveys.
[?
Meki, ?]
I know, has done a few of them.
Other questions?
Yes?
So let's say you're a new teacher, and you're giving a course.
And you design it, and you really think about how you want to do it, and the labs and the homeworks, et cetera.
And you refine that over a number of years
Yeah, two or three years
So five or six years, you've got it down pretty well.
Does it become boring?
Good question.
Yeah, so the question is you make a new course, two or three years you've refined it, after five years, does it become boring?
My guess is probably yeah, unless you keep changing it.
So I've taught Art of Approximation for almost nine years now.
And every three years or four years, I completely figure out a new way of doing it.
And now it's finally converging, I would say.
I've finally realized, oh the main thing-- because I didn't know this when I first started, because I was a new teacher-- so to organize it around large ideas, main themes, transferable technique.
So I've finally converged to that.
And now I'm changing the examples.
And so I would say the [INAUDIBLE] is probably converging.
But I could probably keep doing that for another three or four years.
But if you just do it, refine it for two or three years, and then just keep doing it, I would say yeah, after another couple years, it's not necessarily that it will get boring, because when you teach this way, the teaching always stays interesting, but it's that you find yourself maybe just thinking about other courses.
And I already have a sign that that's happening, because I'm now thinking, I'd like to make a Physics of Music class.
And they may be related to the other class.
Because we just did the wood blocks and the xylophone in my approximation class.
I'm thinking, oh, a whole class of the physics of music would be really a lot of fun to make-- organize it around demonstrations, and then you could bring in the physics, as needed, for the demonstrations, and you choose all the demonstrations so that all the main physical ideas in acoustics, and electromagnetism, and mechanics, and sound are all combined.
Oh that'll be a really nice design problem.
So maybe I'm converging with the Art of Approximation one after nine years.
So it depends how often you keep changing things.
But yeah, and that's OK, because you'll find that that's the natural life cycle.
So for example, if you're a faculty member say in almost any university in the country, after six years, you get a sabbatical.
So in the sabbatical, you'd be very well advised-- I mean, you could teach if you wanted-- but you're very well advised to not show up on any committees, not teach, and generally, you're pretty much required not to and then just leave.
Because otherwise, you'll be dealing with all the normal, run of the mill things.
You wouldn't be able to have a block of time to really think.
So if you've developed this course, and it's your course, after five or six years, it's automatically going to have to be given to somebody else.
And that's fine.
Because that's around the time that you'll be thinking about other stuff.
And the sabbatical is a time to reorient yourself towards other stuff.
So looking ahead towards that, what you want to do is write up what you do.
So the first year, it's pretty hard, you're just barely scrambling, you're trying to breathe.
It's like having four children, or us having quadruplets.
Well, maybe, I don't know, I haven't had that happen to me.
[LAUGHTER]
But the first year you do a brand new course, especially a three days a week one, it's pretty hard.
So I can tell you, this is two hours a week, but it's the first time I've taught this course.
So it's a lot of work.
And that will be true.
So the first year, yeah, you just want to survive.
The second year, you start to figure out what to do, what to write, and just start writing stuff up bit by bit-- lecture notes, problem sets that are type set, things that other people can use pretty easily.
So then when the course is handed off to somebody else, they can continue that and refine it, based on what you've done.
And everything you did doesn't just vanish.
Yeah?
So you mentioned in classes you really like to [?
do a lot of ?]
teaching and [?
carve ?]
the main ideas and then put some of the mechanistic--
The grunge
[INAUDIBLE].
So do you have anything illuminating to say about how to choose those textbook [?
readings, ?]
other than [INAUDIBLE]?
Yeah, so the question is-- so I like to use lecture time for the larger ideas, for the concepts, for the chunks.
So for example, in the teaching equation one, and leave the derivations and detailed manipulations, the mechanics of stuff, the grunge let's say, for the textbook.
Do I have any suggestions for how to choose a good textbook?
Well, it is important to choose a good textbook.
But it becomes less important if the classroom is focusing on concepts in chunks.
Because that's the differentiator, generally, between the good and the bad textbooks, is that the good textbooks do more of that.
So if you're focusing on that in the classroom, then you have more freedom-- not freedom to use bad textbooks, but the bad textbooks don't do you as much harm.
Because the students don't need so much out of the textbook.
They need to basically have a place where the derivations are correct.
And that's an important part, and it has the derivations you need, sure.
So right away, you can relax about choosing textbooks, slightly.
But how do you find good ones?
Actually now, I look at the reviews online and see what students say about them.
So I do a web search for books, and I see what books people are using, what people are commenting on, look at the online bookstores, the various reviews of the books.
That helps a lot.
And then, nowadays, often people put the first chapter and the Table of Contents online-- the publishers do.
And ideally, they would put the whole book online.
Some are doing that too.
So have a look at that.
From the Table of Contents, you can really tell a lot about a textbook.
You can say, is this organized right?
Does this have the right philosophy that I'm looking for?
So look, right away, at the preface, where the author talks about why this book.
And if they have no good reason for why this book, it's probably not the book for you.
Sometimes you don't have a choice about the book.
People just say, well, this is the book that we always use.
Or it's the first time you've taught the course.
For example, you're hired say in March, you start your job in July, and in August, they come to you and say, oh yeah, by the way, Professor Blah is on sabbatical teaching X, you're up, the course book is already there.
So you can't do much about that.
So sometimes you don't have freedom, but then when you do, you want to do it.
So look at the preface and the online reviews.
They're quite helpful.
But don't worry as much if your classroom is interactive.
That mitigates a lot of textbook problems.
Yeah?
So you [?
focused ?]
on the fact that in Europe, in your undegrad, you take more specific classes towards [INAUDIBLE]
Mm-hmm
And I wonder actually if there is some [?
standard ?]
or analysis of what is more effective.
Because in general, I goes that if you have more variety, than you have a higher tendency to do more disciplinary work.
[INAUDIBLE] that would be the advantage.
But the advantage of having taken only chemistry, math, and physics during your undergrad is that then you know a lot about that
Right
And that also goes a little bit-- I don't know, I keep hearing, although I have never seen proof that math skills in the US are not as good as the people who come from outside.
And also, [?
because ?]
we were talking about doing a class on music and--
Physics and music, yeah
And then I wonder if I'll ever get an undergrad that took that class instead of taking a physics hardcore math class.
And I'd be like, oh, I wish you had taken this class that would give you the tools to work, rather than some class that you would enjoy, but that I'd rather have you taking, later on, at the end of your grad school
OK, so let me unpack those.
So there were several questions in there.
So first, are there any studies on the difference between the results produced by the European's, say, educational system versus the American, or the European where people just do their one subject to first order, where the American one where you take courses across many different subjects?
I'm not sure about that.
But also, the notion of effectiveness is not clear, so what's the goal?
So that the American one, I think the historical reasons why it's different is that education in America, early on, back in colonial times, and just post-colonial or the early 1800s, was seen as a democratic thing.
The idea was that this is a democracy, and all the citizens are expected to participate in that.
And so they had a narrow definition of citizens-- just white males-- but within that definition, the idea was that everybody had to participate or was going to participate.
So to that end, they needed a broad education.
They couldn't just specialize in one thing.
Because they would be too narrow to actually help govern society.
Because society is multi-farious and multi-disciplinary, intrinsically.
Whereas in Europe, that idea of democracy came hundreds of years later, if at all, maybe at the end of World War I.
So the European educational system was much more oriented towards producing people who would fit into a particular slot in society.
So now if you want to measure how effective each was at its historical goals, I think that's a fair question.
To measure how effective each is at producing people who are trained for a particular slot in society-- I don't think that's a very useful analysis.
Because the historical origins of the two systems are different.
It's much more fruitful, I would say, to think about the goals.
Which goals do we agree with?
Can we get the benefits of the American goal with some of the benefits of the specialization in Europe?
And I think at places like MIT, you do do that.
The students who come out of MIT-- they have a broader education than the students who come out of European universities, and they actually know a lot.
Now I would like it to be much more conceptual and long lasting.
But that's true of the European ones too.
So I would take the question back from just studying the end result to think carefully, as a society, about their purposes.
Now about the Physics of Music, I would say, actually, I could imagine teaching an intro physics course that's Physics of Music, where students would learn all the fundamental ideas of say first semester physics, but through the physics of music.
And so they wouldn't be deprived of the essential knowledge of physics.
They would actually be contextualized.
So it might have longer lasting value
[INAUDIBLE]
OK, other-- yes?
So you talk about emotional barriers, and I think one of the big ones that I've seen, especially in students at MIT, through TA'ing or just through hearing people talk is I don't belong here.
I must have gotten here by accident
Yeah
And I think so many people say that to themselves, at one time or another, or multiple times while they're here, that it can't possibly be true, because everybody is saying it
Right
Because if they have that barrier up, I don't think there's much to do to combat that.
So how do you get through that kind of barrier?
Because no matter how clear your teaching is, if they're worried that they're not going to get it, because they don't belong--
Yeah, if they're in the math phobic mode or phobic mode
Yeah
So this is a really interesting question.
So the comment was that a lot of people at MIT, a lot of students, one of the emotional barriers they have is they feel-- I would call that a meaning barrier.
Let me rephrase it in the three levels of conversation.
So their emotion is that they're very tense and anxious.
And the meaning is that when something doesn't go right for them, they get something wrong on a problem set or in class, they think that that's exposing how they really don't belong here.
And that creates a lot of anxiety and tension and misery.
So at Caltech, there was a saying that, oh yeah, Caltech is a terrible place for the bottom half of the 99th percentile, which captures that pretty well.
All these people, all the undergrads who went to Caltech-- they were probably valedictorians in their high school and two sigmas above in doing science fair projects than everybody else around them.
And now they come to Caltech, and they feel terrible.
What are we doing that produces that?
It's crazy.
So how can your teaching-- can you take account of that?
So that, I would say, also relates partly to the misconceptions.
They have this misconception or this conception about themselves.
So what can you do?
Well phobias are very difficult.
You have to work around them.
So one thing is, you don't want to trigger them.
So every time you trigger them, you reinforce them.
So any neurobiology people in here?
I forget.
[?
PCS?
?]
Well the fundamental rule of neurobiology of brain neurons-- so this is-- of course, I teach Art of Approximation, so forgive me for approximating the whole brain with one sentence-- is neurons that fire together wire together.
So wiring together, that means they activate, they increase their coupling.
And so for example, you create a classroom environment or something happened in the classroom that they get something wrong.
And now that is still wired, somewhat, to the bad feeling and their thoughts of oh my God, I don't belong here.
So now you've reinforced that, they're going to wire together more.
It reinforces that pathway.
So what you have to do is find a way to actually prevent those guys from wiring together.
One thing is create an environment where whenever people get things wrong, it's actually great.
So oh, that wasn't right, great, now think how much you're going to learn.
Or when you get things wrong, you want to model that too.
You say, oh you're right, oh thanks, I've learned something from that.
So try not to be defensive.
Because if you're defensive, then that triggers that wiring in them.
And that firing together builds up the wiring together.
So creating a classroom environment where feeling bad and competing against other people isn't the goal is one way to minimize it.
So minimizing the importance of defined distinctions on homework-- did I get a 95?
Or oh, that person's getting a 95, I only got a 90.
So that's one reason MIT was pass/fail.
For a whole year, originally, was pass/fail.
And that was to mitigate the pressure that students were coming in, feeling like, oh my God, now I don't belong here.
[?
It was ?]
look, we don't care about your grades in the first year.
You're really here to adjust to university, to a different way of learning, and now, it was actually done in a way that somehow the competition was partly the students reinforcing the culture, the upperclassmen, as well.
If you could actually try to mitigate that, you'd go a long way towards mitigating the wiring together.
But you can do it in your classroom too by deemphasizing the things that trigger that.
And so when you call on people-- so for example, for that reason at MIT, I almost never what's called cold calling.
If you've ever seen that old movie The Paper Chase-- so The Paper Chase is about the what we could call today the terroristic Socratic method.
So the law schools all use the Socratic method, where they read a case, and you ask them about things-- things about it.
Mr. Jones, what do you think, what were the findings of law?
Mr. Smith or Mrs. d'Arbeloff, what do you think about that, do you agree with that, or is that pure rubbish?
So people were just put on the spot repeatedly, and they just produced terror in the students, and that was considered part of the hazing ritual, basically.
So at MIT, I just don't do that at all.
Because I think it just triggers that.
And I'd rather just create a structure whereby students feel ready to participate, by, for example, in the interactive teaching, giving them time to think to each other and then say something.
So you can do stuff, and it's hard, and it's an important problem.
Yeah?
Sure
It's worthwile to actually confront it head on, is just say, a lot of you may be feeling this way, and please know that it's normal, and in fact, I felt this way most of my life.
Because I had a professor in my undergrad who actually came in and said to a group of us-- like, it was in a meeting.
And she said, how many of you have felt like, basically, you're an imposter and it's just a matter of time [INAUDIBLE] discovers you?
[INAUDIBLE] all of us raised our hands.
And she said, I wanted you to know that it wasn't until I won my fourth major award in the field that I stopped feeling that way.
So It's normal.
And that's run in my head for years.
So even someone who was amazing as she was, and she was an amazing individual, has felt that way, and to know that you're not the only one [INAUDIBLE].
So do you think it's actually worth saying?
Yes, that's a really good point.
So the point was that it's worthwhile making it explicit for the students and really naming the feeling that they're going through.
Like look, you may be feeling really terrible and feeling like you're an imposter.
Well, let it be known that I felt the same way too for a long time, and describe it to them, and talk to them about why they may be feeling that.
Say look, there's a culture of competition in the society and the university, and it's actually harmful for your learning.
And share with them some readings about that.
Help them see that yeah, this is something they're going to have to struggle with.
But there's help to do that, and they're not alone in that.
So yeah, definitely be explicit about things like that.
I think more question, yeah?
Just in a follow-up, do you think that contextualizing the course is a way to fight that?
Good question.
Do I think conceptualizing the course is a way to fight that?
I think it is.
Because when the course is not contextualized, it generally tends to be really abstractly focused.
And the interest and the orientation towards doing things first abstractly is very rare.
I don't have it, and I've gone really far in math and physics.
My office mate had it in graduate school.
He was the only person I've ever met who would rather have a proof than a picture.
And now he's a professor at Caltech, and he's also doing well.
But it's pretty rare.
So by contextualizing the course, for almost everybody, actually I would say even for my friend and office mate, you're actually connecting to a much larger part of their mind and their experience.
So you're actually making people much more intelligent and much more equal in that way.
So that's one reason I would actually like to teach the Physics in Music class
It made me think [INAUDIBLE] they teach drugs in the brain [INAUDIBLE] biology.
So if you're a physicist and you're like, I can't do biology, you can take drugs in the brain [INAUDIBLE].
And they teach you biology by teaching [INAUDIBLE]
Interesting.
Yeah, so it somehow gets around the phobia you might have
[INAUDIBLE]
Pardon?
They do that at [INAUDIBLE]
Interesting.
So I was there 10 years ago, and that wasn't true then.
So the comment was that at Caltech now the required classes all have a contextual version, as well.
And I think that's a great idea.
It's also, from your point of view, it's really fun as a future teacher, because you get to think of all the interesting contexts in which your field is useful.
It connects you back to the reason you did the field, originally.
And that's good to remember.
Because you'll be going along, and you'll be absorbed in your research, trying to get tenure, whatever it may be, and there will be all these layers that are created on top, and you'll forget why you came into the field to begin with.
And if you can capture that feeling, actually if you can connect to that feeling in you, then you'll actually be able to transmit that feeling to your students.
And they'll have it, as well, and you'll actually be doing good for the field, good for you, and good for your students.
So with that, I'll wish you lots of success in your teaching.
I hope that the principles and examples that we've done throughout the term you find useful, at least in one way.
And I look forward to hearing from you later how you use or not use or any suggestions that you may have.
So good luck.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to 5.111, and today what we're going to do is introduce you to the course and the people teaching the course.
And we're also going to let you know that you were going to be part of the great web exercise that is OCW, OpenCourseWare.
So, this course is being videotaped this year, and this is the announcement that I have to make.
So, the videotape is in the back, and if you want to come up front and participate in the class, you know that you'll be videotaped -- if you want to hide your face or whatever, you can do that but please pay attention to the lectures anyway.
So, this course will be available on the OCW site in the future, I'm not sure exactly what that date is going to be.
So, today we're going to introduce the chemistry topics, which we will cover in 5.111, and give you general information, practical information about the course number of points you need, when the exams are, that kind of thing, policies, and introduce you to the teaching staff.
I am, again, Professor Cathy Drennan, and I'm one of the lecturers in this course.
So, because this is MIT, we're going to start to quiz.
OK, not a quiz for points or anything, don't freak out, but I do want you to tell me who these people are.
So, what about this person?
That's me -- this is my college yearbook photo.
OK, what about this person over here?
You?
It's not me again.
Not Elizabeth Taylor.
It is Lisa Kudrow, known as Phoebe on "Friends."
So, we both went to college at the same time, we went to the same college.
Does anybody know what college that was?
Vassar
Vassar College, very good.
And we graduated the same year -- now no one has to say what year that was, even if you know, but we did graduate the same year.
All right.
So, given what you know about us, what do you think Lisa went to college to study?
Computer?
Computers?
No
Theatre?
Theatre?
Surprisingly, no
Nuclear Engineering?
Nuclear Engineering at Vassar, no for a variety of reasons.
Any other guesses?
English?
English, no.
Biology -- I heard it.
Biology.
What do you think I went to college to study?
Theatre
Theatre, correct!
And/or I hadn't made up my mind exactly -- biopsychology or drama.
So, biopsychology was what they called sort of brain and cognitive sciences in those days.
So, those were the two things I was thinking about.
What do you think Lisa ended up majoring in college?
Biopsychology
Not biopsychology
Biology
Biology, yes.
What do you think I majored in college?
This should be a bit easier
Chemistry
Chemistry, And, of course, our professions, actress and chemistry professor.
So, let me ask what happened here?
My understanding about Lisa Kudrow is that she came from a Hollywood family.
She went to college and said here's my opportunity to study the thing that I find most interesting, and that was biology, and then she went back and participated in the family business, which was of course the acting profession.
For me, what happened?
Well, I have to say, I did not like chemistry in high school, so I did not think about going to college to study chemistry.
So, why did I not like chemistry in high school?
I think it was because of images such as this one.
You spend a lot of time talking about the transition between alchemy and modern chemistry.
I wasn't very interested in that kind of thing, and there's nothing in these photographs that really appealed to me personally.
I mean, Avogadro -- I'm fond of his number, and he is, in fact, an interesting, if not frightening looking man -- this just didn't connect with me.
But then I got to college and they said, "Well, if you're thinking about anything bio -- biopsychology, biology -- you have to take chemistry."
And I said to my advisor, "No, no.
I have taken chemistry in high school, and I can assure you that chemistry has no relevance whatsoever to the life sciences."
And they said, "Well, I'm sorry you feel that way, it's incorrect, and you have to take it anyway."
So, I, like some of you in this room, took freshman chemistry, because we had to, not because we wanted to.
And I, like hopefully some of you in this room, discovered that chemistry was actually a lot of fun, and that the chemistry I got in college was pretty much nothing like the chemistry I had seen in high school.
So, let me introduce you to some of the topics we are going to be covering in chemistry this semester.
So, there's more detail on your syllabus -- a detail of what we'll cover every day, but these are the kind of basic things that were covering, and you don't need to write this down, you'll become familiar with it as the semester goes on.
We start out with some really basic principles.
So, up here, atomic theory, periodic table, bonding, structures and molecules.
And there will be a little bit of history in there, but this is mostly modern chemistry and represents the basic properties of matter, and it's basic properties of all matter, including living matter, which was what really interested me, that connection between chemistry and biology.
So, then we go to thermodynamics and chemical equilibrium, and this is really about chemical reactions -- weather a reaction will go, will it be spontaneous, if there an equilibrium, what direction will the reaction be shifted in.
And then, of course, not just whether the reaction will occur but, how fast it occurs is really important.
So, that's kinetics -- how fast a reaction will go, and from the perspective of someone who's a biochemist, I'm interested in kinetics and enzyme kinetics, and thinking about molecules that catalyze reactions in the body.
And then, there's acid base equilibrium, and also oxidation reduction reactions, and what is true is that most reactions that occur are either catalyzed by either some kind of acid base catalysis or involve some kind of oxidation reduction reaction, and so, this sort of represents a lot of the basic way reactions go -- now, whether that's a reaction in your body or a reaction in a test tube -- it doesn't matter, a lot of the same principles are involved.
And then, we also cover transition metals, which is something that you often don't see in high school.
And transition metals, those all medals in the middle of your periodic table, have some really unique properties, which are exploited again in reactions that occur in your body, and also are utilized in industry, for example, so we'll talk about some of those unique properties.
And if we put all of that together, we get the real fundamentals that you need to go on and study -- any kind of curriculum that involves chemistry.
So, these are all the fundamentals that are involved in chemistry that relate to physical chemistry, organic chemistry, inorganic chemistry, biological chemistry, and are a solid foundation for studying any kind of life science.
So, I congratulate you of being here in this class, this is really good solid foundation for whatever you go on to do, here at MIT.
So, normally at this point, we do actually start class with a little bit of history from alchemy to modern chemistry, but I decided to skip that this year.
If you are interested in that, it's never required on any test, it never has been, but if you're interested in that there is an OCW lecture, which you can listen to that's an excellent lecture by Professor Sylvia Ceyer on that.
But today instead, I thought I would give you some examples of modern chemistry -- why people now need to know chemistry, what they're doing with chemistry, what is chemistry research here at MIT, and how does it utilize these basic principles, which we'll be talking about in the course.
So, I'll start with my colleague, Professor Joanne Stubbe.
She studies molecules, in particular she studies biological molecules.
And, so one of the things she's very interested in is how this anti-cancer drug, gemcitabine, works in the body.
So, it inhibits an enzyme, and she's interested in knowing how that really works.
So, enzymes are made up of amino acids, you have long chains of amino acids that form together into a protein molecules, protein molecules in your body often act as enzymes, catalyzing reactions.
So, she is interested in how this molecule, gemcitabine, inhibits an enzyme.
So, to do those studies, she needs to know a lot of the stuff on this list.
Of course, she needs to know the basic principles, but she's also talking about it an enzyme, so she needs to know about enzyme catalysis.
She needs to know this enzyme works by both acid base chemistry, and oxidation reduction.
It has two irons that are involved in doing the chemistry, so it includes transition metals.
She thinks about how things bind, how the natural reactants binds, how the inhibitor binds, and so she needs to know what happens to the chemical equilibrium, she needs to know about the thermodynamics of those binding events, and, of course, everything, all the basic principles, are required here.
So, to do this biochemistry research, she needs to know all of these things, and she has really made tremendous progress in understanding how gemcitabine works, and it is not so toxic, so it's a really good thing to have in chemotherapy.
So, in addition to studying molecules, chemists often want to make molecules, such as Tim Jamison, who's an organic chemist.
So, you will hear, probably, hopefully, in this presidential debate about the environment and about why saving the environment is important.
And one of the things you often hear about in this discussion is about our oceans and about rainforests, and part of the reason why people want to protect those areas is because you find a lot of natural products in those regions.
So, a natural product is something that is made by nature, and often natural products, whether it comes from a plant or a marine organism have some really good, useful properties.
And so, one particular compound has anti-tumor properties.
So, again, along this line of cancer research.
So, Tim Jamison's lab figured out how to make this thing.
And often that's really important, because you can't get enough of the organism that naturally makes it, to be able to grind that organism up and have enough that you can actually use as a medicine.
So, you have to make more of it, because nature doesn't make enough.
So, it's very important to figure out how to do that.
So, in doing that, Tim Jamison's lab needs a lot of these things.
So, he needs a lot of knowledge of bonding, he wants to form bonds in making this.
He needs to know about the structures of the molecules, because if the structure is wrong it's not going to work.
And often, if you want to make a lot of it, you have to think about the thermodynamics of the system, how fast the reactions will go and kinetics, and then whether they'll go, the thermodynamics, and sometimes then you need to adjust the reactions, maybe use a transition metal to make it go better.
So, these are all the things that Tim Jamison needs to know to do organic chemistry.
So, you'll be learning in this class a great preparation for 512, which is organic chemistry.
In addition to studying molecules and making molecules, some chemists want to detect molecules, and a chemist who likes to detect molecules is Tim Swager.
So, Tim Swager's lab has designed sensors that detect vapors, and so they will detect TNT, for example.
And so, he has put this chemistry to use in this robotic arm and they call it Fido, because often dogs are the creatures that have to go out and detect these things, and it's not a great job if you're a dog to be sent out to see whether there was an explosive and discover yes, there was, a little bit too late.
So, this is a much nicer way to detect chemicals with this robotic arm, and here's a picture of it in use in Iraq.
So, in doing this, if you go down to kind of a basic principles that Tim needed to know about, oxidation reduction was really key in developing this technology, so we'll talk about that.
So, my final example is from Alan Davidson's lab, and Alan is an inorganic chemist -- he loved those transition metals and they're unique properties, and he designed this compound, it's called Cardiolite, and it's used in heart imaging.
So, many people have relatives they know of that have had to have their heart imaged -- heart disease is a major problem in the United States, and there's a good chance that they had Cardiolite given to them to help in that imaging process.
So, this again, takes advantage of those great unique properties of transition metals, which we'll talk about in this course.
So, again, all together, this is the basis for modern chemistry, and examples I just gave you, are some of the things that modern chemists are working on -- some of the issues that our country faces and our world faces, and how chemistry is involved in that.
So, not only will you have the fundamental knowledge to go on and take more courses in chemistry, you will also have the fundamental knowledge to go on and do undergraduate research here, and here are some of the 5.111 undergraduate researchers that have come through my lab, in particular, from this class.
So, it's a really nice solid foundation.
So, I want to encourage you to set some of your likes and dislikes from high school aside when you come to MIT, because at MIT you often see disciplines taught and emphasize a very different fashion than what you've seen before.
And you may discover that the thing you came here to study is not the thing that you really want to study after all.
One other thing that I'll say to you is that I said these words at one point, it's true, I said, when I was in high school, I said, "I hate chemistry."
And now, I do chemistry every day and will for the rest my life.
I love chemistry now.
Be very careful what you say.
Have any of you made that statement about hating a subject?
Tell me later what it is you're going to be doing for the rest of your life.
So, at MIT things are very different, and keep an open mind, explore new areas -- take advantage of being at this amazing place for science and technology and you may surprise yourself in what you really enjoy learning about.
So, that's a little bit about the chemistry that we're going to cover in this class, and now I'm going to talk a little bit about some of the policies and procedures.
But first I need to introduce my co-instructor for this class, and let me just put up her picture, you'll see her in a minute.
So, Dr. Beth Vogel Taylor.
So, all chemistry courses are team taught, so you have a different lecturer for the first half than the second half, and Dr. Taylor will be doing most of the first half lectures, and I'll be doing most of the second half lectures.
So, Dr. Taylor will take you from atomic theory through thermodynamics, and I'll start up with chemical equilibrium, talk to about kinetics, acid base, oxidation reduction and transition metals.
So, you will have both of us as lecturers in this class.
Now, in the past, sometimes students have found this whole thing a little frustrating, that they just get used to one lecture style, and then all of a sudden there's another lecture style, and that can be true.
I mean sometimes the styles of the two professors couldn't be more different -- think McCain/Palin, odd couples.
Sometimes they're more similar, and when I first, about a year and a half ago, got to know Dr. Taylor, we sort of realized that we had very similar styles, and we got very excited about the idea that we could teach together, so that there would be much more continuity throughout the semester.
And so, Dr. Taylor had been teaching the first half of the material in the Spring, and I had been teaching the second half of the material in the Fall, and we thought wouldn't it be great if we got together and taught in the Fall.
So, this was actually, for a variety of reasons, a very complicated thing to request and do, and so we started a campaign and campaigned for a year and a half that we should be allowed to do this course together, and finally just a few weeks ago in August -- we really didn't know up until almost when this course started -- that permission was granted.
So, I have to say, I am very excited now to introduce you to Dr. Taylor, who I would be teaching with this semester -- limited engagement -- who will tell you about some of the course policies
Okay, so before we get to some of these course policies, I think I'll tell you a little bit about my path to chemistry as well.
Professor Drennan explained that not everyone that ends up as a chemist started off that way on their first class freshman year, for example, in chemistry.
And in fact, if you talk to a lot of chemists, if you talk to some of the graduate students, maybe your TA, you'll find that that phrase, "I hate chemistry," has maybe been uttered by more than one us at some point in our lives before we realized, and once it happens you don't go back, that actually you love chemistry and it's hard to even remember a point where you didn't see all of these connections that it provided for you.
To give a little background of where I was, sitting where maybe you are today on the first day of chemistry, when I left high school, I had no interest in chemistry whatsoever.
And I have only one strong memory from high school chemistry, and that memory is shown right here, and that is the common ions.
Did you guys have to learn the common ions?
Does anyone have that in their brain somewhere for ready use?
I don't, in fact, so it's actually okay if you don't know all your common ions, if you missed that part.
This is the strongest memory I have, and I remembered a) that I didn't learn them, and that was really bad because it kept coming up, but the other thing I remember is that I had no idea why they were important.
I didn't really understand what any of these molecules were.
I certainly didn't understand how they even connected really to chemical reactions, much less other disciplines that I was interested in.
I couldn't have told you, for example, if we look at a phosphate group, that that's going to be incredibly important in DNA, that it's also an incredibly important group when you're dealing with proteins and whether you're turning the function of a protein on or off.
So really, I just had no context for the chemistry.
So, when I started in college, that wasn't even an option for me and I was interested in a lot of things, chemistry not being one.
But one that I was very interested in was biology, and the reason was we did a lot of cool labs in high school, I loved doing the dissections -- it was very interesting to me to think about how different organs worked, how the heart could be a pump, how the lungs worked.
And then when we got to more of a cellular level, it was even more interesting to see that we could actually understand how our body worked as low of a level as thinking about cells.
And so, that was a clear major for me to pick -- I actually also was considering English and ended up being a minor in English.
But, I think what most of you, actually having come to MIT, have probably realized is sometimes it's nice to major in a science, because you can't just pick up a reaction and do it in your kitchen on the weekend, where as you can sometimes join a book group and do that.
So, it's kind of nice to major in the thing that you're going to get to have the opportunity to do for the rest of your life.
So, I actually also started pre-med.
Is anyone else pre-med here?
Okay, so a pretty good showing.
So maybe you can relate to some of the reasons I wanted to be pretty pre-med -- part of it was the interest in the science and the biology.
Also, I wanted to help people -- it seemed like a really clear way that I could have a career that was challenging and involved in science, but also helping others.
So, it seemed like a good start for me, pre-med/bio, and I signed up for my bio class -- I found out, as Professor Drennan did, that I had to take chemistry as well.
I wasn't as upset, I was sort of a neutral chemistry person at this point, but I thought it was pretty smart to get it over with on the first semester, so that's what I did.
And my plan was going along fine until something happened, and what happened was that chemistry was just way more interesting than I anticipated.
So, my perfect pre-med/bio plan was getting a little shaken right from the start, and the reason that it was getting taken was because I would learn this new principal in chemistry and because I was taking bio with the same time, I could see the connections.
And at one point I realized, "Oh, my gosh, chemistry is just biology, it's just looking at one level deeper."
So actually, all of my interest in biology was quickly transferred to saying, "Wow, now I can think about things on the molecular level."
And one of the molecules that caught my attention first, and I can't remember if this was freshman or sophomore year in high school, was the first time I actually took meaning in looking at a chemical structure, and that was with the structure of penicillin here, and I know that all of you are familiar with penicillin, whether or not you know the structure or not, but the most important part of this structure is the four-membered ring here, the beta-lactam, and this was the first time I thought I could actually understand how a molecule worked because I knew something about chemistry.
So, for example with penicillin what it does is it inhibits an enzyme that builds the cell wall in bacteria, the bacterial cell wall, and if I thought about what I'd learned in chemistry -- some of you know this from high school, some of you will be very familiar with this soon, is that this carbon here, for example, is bonded to three things.
Does anyone know what angle those would like to be at?
120.
They want to get as far away from each other possible, the ideal angle is 120.
But what we have here is a four-membered ring, so what angle does that have to be, that bond?
90 degrees.
So, we have a problem here if we're thinking about keeping things at the lowest energy, so there's a lot of ring strain in the system.
And I was incredibly excited that I could look at that and realize it and say "Wow, that's why it's so reactive, that's why it's such a good medication," because when it comes into contact with these bacterial cell wall building enzyme, the enzyme can actually react with this four-membered ring and open up the ring and relieve that ring strain.
So, now the angles can open up all the way to 120 if it wants to, and there's no way it's going to form that ring again, right, because it's not going to back to those 90 degree angles, if it can help it.
So now, the enzyme is locked up with the penicillin molecule, no more bacterial cell wall being built, and the penicillin has effectively killed the bacteria.
So, that, for me, was kind of the first connection that what went, "Woah, wait a second, I want to be thinking about these molecules all the way down to the level of individual atoms."
So, at this point, kept the pre-med, just switched the major to chemistry.
The next problem came up when I went and took organic chemistry.
So, if you're dead set on staying with bio, maybe, I guess you have to take organic, so this might happen to you, just to warn you.
We started looking at all sorts of other kinds of molecules that became very interesting to me.
I especially love thinking about vitamins and drugs, because I do have that interest in medicine and human health.
These are actually all examples that we'll talk about in freshman chemistry at some point, as an example of a connection between a chemical principle we learn, and what we can know about how it functions.
But what happened here was I thought "Oh, my gosh, now I could actually, using my chemical knowledge, think about synthesizing these molecules, or maybe coming up with new ways to synthesize them better or synthesize different molecules.
And the real clincher was when I started doing some undergraduate research.
Any potential UROPs out there -- anyone planning to do a research at some point?
Excellent.
Okay.
So, just to be warned, you might fall in love with the subject you do your UROP in.
This is one of our summer students from this past summer, who is also premed.
She's continuing to be pre-med, which is fantastic.
That didn't happen to me -- once I got into the lab, I didn't want to leave.
So, I thought, "You know what, I think I'll change the medical school plans and now I'm going to go all the way -- chemistry major, chemistry grad school."
And the reason I was able to do that and keep with what my original intentions were was to have a career that was the fulfilling, in terms of helping people and being engaged in science, is all of a sudden I realized, as chemists, we can think about better ways to build molecules that are important for making medications.
Another thing we can do is we can use our chemistry to understand biological systems, so we can help illucinate pathways, maybe, that are implicated in disease.
So, the combination of these two things had made my decision and I ended up coming here for graduate school, actually, and working in Professor Imperiali's lab doing bio-organic chemistry, which means that I synthesize molecules, which I loved, and used them to study biological systems.
So, really I'm pretty happy with what I've gotten to do, and I just want to say we're not trying to convert all of them you pre-med people, by any means.
My roommate for many years was going to medical school as I was going to graduate school, and we found we had so many interesting conversations about chemistry -- her from the context of practicing and using medications and talking about how they worked on a molecular level, and me talking about my research.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, our first question here is about limiting reactants.
So, that's something you will encounter in your review reading for the sections, that kind of review -- what we hope you have picked up from high school or will pick up quickly by doing some review.
So, how about we have everyone take ten more seconds on the clicker question, get your final answer in here.
All right.
So, let's see what we have.
All right, so it looks like we weren't showing the percentages here, but it looks like hopefully most of you were able to get the correct answer of H2 being the limiting reactant.
It looks like we're still figuring out -- this room was just renovated, we're still working out exactly how the electronics work.
So, normally we'll see a percentage of how many of you got it, but I'm going to say it was probably about 95% got the answer right.
So, good job there.
If you didn't get the answer right -- we'll send these questions to your TA, so any time you get a clicker question wrong and you're confused, bring it up in the next recitation section and you'll be able to discuss it there.
So, starting in, we can switch over to the to the notes now.
When we left off on Wednesday, what we had really been doing is trying to give you an overview of all of the different topics that we're going to be going over this semester.
And also, to make a couple of those connections between the principles we're learning, and some of the exciting research that's going on at MIT in the Chemistry Department, and also, to give you the idea that we are going to be trying to make these connections between the chemistry and things like human health or medicine.
So, now we get to actually take a step back and start at the beginning, because before we can talk about some of the more complex issues, which involve interactions between molecules reacting or even when we're talking about individual molecules -- the bonds that form between individual atoms -- before any of that we actually need to establish a way that we're going to describe and think about how an individual atom behaves.
And the way that we'll do this is starting with talking about the discovery of the electron and the nucleus here.
Once we go through that, we will be able to talk about describing an atom using classical physics.
So, once we have an atom and a nucleus, what we'll try to do is apply the classical mechanics to explain how that behaves.
What we'll find is that this fails, and once this fails we're going to need another option.
Luckily for us, we have quantum mechanics, which we'll be talking about for the next few lectures, and we'll dive into that.
We might get a chance to introduce it today, but certainly in next class we'll be introducing this new kind of mechanics that's going to allow to describe the behavior of atoms.
So, I want to point out that it makes a lot of sense for us to start with the discovery of the electron and the nucleus, because it really highlights one of the big issues that comes up in all chemistry research that you do, and that is how do we actually study, or in this case, how do we discover atoms or sub-particles that we actually can't see at all.
And there are lots of solutions that chemists come up with -- there's always new techniques that allow us to do this, and these are just some of the first, and we'll go through them in a little bit of detail here.
So, this all starts, in terms of putting it in its historical context at the turn of the Century, we said we'd start right in on the 20th Century of where chemistry was.
And where we where at the start of the 20th Century in the late 1890's is that we were at a place where there was great confidence in our understanding of the universe, and our understanding of how all matter worked.
So, people in the chemistry community and in the physics community had this general feeling that the theoretical structure of the entire universe was pretty well understood.
And they had this feeling because there had just been this huge boon of discovery, of scientific advances that included Newtonian mechanics, it included Dalton's atomic theory of matter, also thermodynamics and classical electromagnetism.
So, you can understand they really felt quite confident at this time that we could explain everything that was going on, and in fact, a really telling quote from the time was said by a professor at the University of Chicago, and what he said is, "Our future discoveries must be looked for in the sixth decimal place."
So, basically what he's saying here is we pretty much understand what's going on, there's nothing new to really discover, all we need to do is measure things more precisely.
So, that's not exactly the case, and we're going to start in at the point where right around this time of great confidence of feeling all has been conquered, there are some observations and discoveries that are made that completely break down these theories.
For example, in terms of the atomic theory of matter, at the time at the turn of the Century, the understanding was that atoms were the most basic constituent of matter, meaning you couldn't break atoms up into anything smaller -- that was it, you're done.
And with using Newtonian mechanics, it was assumed since this type of mechanics worked so well to describe everything we could see, it could even describe the universe and planets, that, of course, we could use Newtonian mechanics to describe how an electron -- actually, we didn't even know about an electron here, but how atoms behaved, and it turns out this is not the case, and the first step in discovering this is not the case, was accomplished by J.J. Thomson, and J.J. Thomson is credited for discovering the electron.
He was a physicist in England, and what his laboratory was studying is something called cathode rays, and cathode rays are simply rays that are emitted when you have a high voltage difference between two electrodes.
So, if you look at this set up, what he did when he was studying these rays is he had an evacuated tube, which is schematically shown here, where it's evacuated of all it's air and filled instead just with hydrogen gas, and he had this high voltage difference between an anode and a cathode, and he actually put a little hole in the anode here, so these cathode rays that were produced could shoot out of the cathode and actually could be detected as this luminescent spot on a detector screen.
So, lots of people were studying cathode rays at the time -- one reason is they actually gave off this bright glow -- if you put them in an evacuated glass tube, you got these crazy patterns and glowing colors.
So, for that reason it was a very hot issue in terms of research.
But also, no one really knew what these were and Thomson was seeking to figure out some more properties of them, and he had the theory that maybe they were actually charged particles of some sort, and others had proposed this in the past, but they didn't really have an experimental set up to test it.
And that's what Thomson did.
And what we did was he put two detection plates on either side of these cathode rays, and when he put a voltage difference between these two plates, he wanted to see if he could actually bend the rays and test if they're actually charged or not.
So, when the voltage difference between the plates is zero, or when we just don't have the plates there at all, the cathode rays are not bent, they just go right in a straight line, and they can be detected on this screen.
When he actually cranked up the voltage between these two plates, what he saw was really amazing to him, which is that he actually was able to bend these rays -- this had never been observed before in any capacity, and he was able to detect on his screen that there was this deflection, and he could even measure the degree of the deflection that he had.
So, we know now that we have charged particles.
Are these negatively or positively charged, based on this evidence?
Negatively
Yeah, that's right.
So, what we have here, cathode rays we now know are negatively charged particles.
And, in fact, he named these negatively charged particles.
Does anyone know what he named them?
No, not electrons -- very good guess.
He named them corpuscles.
Has anyone heard of corpuscles?
A little bit.
Yeah, so it was later named that these particles were, in fact, electrons, and that's what they are.
J.J. Thomson continued to call them corpuscles for many, many, many years after everyone else called them electrons, but I'm sure no one minded because he did, in fact, discover them.
And he was actually able to find out more than just that these were charged.
From classical electromagnetism, he could actually relate the degree of deflection that he saw to the charge and the mass of the particles.
So, using that he could say that delta x, and we'll put sub-negative, because we know now that these are negative particles, is proportional to the charge on that particle over m, which is the mass.
So, we have e being equal to the charge of the negative particles, and m, of course, is equal to the mass of those particles.
So, Thomson didn't stop here, he actually continued experimenting with different voltages.
And what he found was if he really, really ramped the voltage up between those two plates, he could actually detect something else.
And what he could detect here is that there was this little spot of luminescence that he could see on the screen that was barely deflected at all -- certainly in comparison to how strongly this first particle was deflected.
The second particle was deflected almost not at all.
But what he could tell from the fact that there was a second particle at all, and the fact that it was in this direction, is that in addition to his negative particle, he also, of course, had a positive particle that was within this stream of rays that were coming out.
So, of course, he can use the same relationship for the positive particle, so delta x now of the positive is proportional to the charge on the positive particle all over the mass of the positive particle.
So, this is interesting for several reasons.
What did he manage to pull out information-wise from using these two relationships?
And actually to do this, he made a few more observations.
The first, which I just stated, is that the deflection of that negative particle was just far and away more extreme, much, much larger than that of the positive particle.
The other assumption that he made here is that the charge on the two particles was equal.
So, how could he know that the charge on the two particles was equal?
And actually he couldn't exactly know it -- it was a very good assumption that he made, and he could make the assumption because he, in fact, did know that what he started with was this hydrogen gas.
So, he was starting with hydrogen.
If some negative particle was popping out from the hydrogen, then what he must be left with is h-plus, and since hydrogen itself is neutral, the h-plus and the electron had to add up to be a neutral charge.
So, that means the charges of the two pieces, the positive and negative particle, must be equal in terms of absolute charge.
So, using this relationship, he could then actually figure out by knowing, which he knows how much each of them were deflected, he could now try to think about whether or not he could make some relationship between the masses -- between the mass of the positive and the negative particle.
So, this relationship he was looking at was starting with the deflection, and the absolute distance that the particles were deflected.
So, what he could set that equal to is he knows what x is proportional to in terms of the negative particle, so that's just the absolute value of the charge over the mass of the negative particle.
He could divide all of that by the absolute value of the charge of the positive particle, all over the mass of the positive particle.
And as we said, he made the assumption that those two charges were equal, so we can go ahead and cross those right out.
So, what that told him was if he knew the relationship between how far they were each displaced, he could also know something about the relationship of the two masses.
So essentially, there was an inversely proportional relationship between how far the particles were displaced, and what the mass of the two particles turned out to be.
So, because he, of course, observed that the negative particle travelled -- it was deflected much, much further by those plates, what he could also assume and make the conclusion of is that the mass of that negative particle is actually larger or smaller?
Smaller
Much, much smaller, exactly, then the mass of the positive particle.
So essentially, what he found here is the relationship between the mass of an electron and the mass of the rest of the atom, the rest of the hydrogen atom there, which is an ion in this case.
And, in fact, it's so much smaller, it's close to 2 times smaller, that we can make the assumption that essentially the electrons take up no mass.
I mean they take up a teeny bit, but essentially, when we're thinking about the set up of the atom, we don't have to account for them as using up a lot of the mass we're discussing.
So, Thomson came up with a model for the atom due to this, and this is called the Plum Pudding model of the atom, and he was, as we said, English, so plum pudding is kind of a British food.
Has anyone here ever had plum pudding?
A couple of people.
Okay.
I've never even seen it, so that's good -- you must be better travelled than I.
So, the idea that he had here was he treated the whole of the atom as sort of this positive pudding, so the majority of the atom was just kind of this goopy, positive stuff that you could think about, and within the pudding, he had all these negative charges, which were the electrons, and they were the raisins or the plums that were in the pudding.
So this was a revolutionary model of an atom when we thought of the fact that before this experiment, the understanding was an atom could not be divisible into smaller parts, and now here we are with subatomic particles with electrons, and this wonderful Plum Pudding model.
So, for those of you that haven't actually had plum pudding, which myself is included, I threw a picture up here.
This was my first glance at plum pudding, and I guess you can see that this must be that positive part -- most of the plums are within that, and you can see all these little raisins or plums in here, that would be that negative charge.
So, that already was a big advancement from where the understanding was at the time.
We already moved way forward and completely revolutionizing the understanding of an atom in that there's something in an atom -- it's not the smallest thing there is.
However, as you know, we didn't stop at the plum pudding model, which is good, because it's a little goofy, so it's nice to move on from that and move on we did.
About 10 to 15 years later, another physicist, Ernest Rutherford, actually put this plum pudding model to test, and he did it through studies that he'd been doing on radiation that was emitting something called alpha particles.
So, Rutherford, some of you may recognize that name, is a very famous physicist who made a lot of contributions in terms of radioactivity.
When he was studying these alpha particles, he was actually the first person to identify the difference between different types of particles that radioactive materials emit.
And he got this particular material that he was studying, radium bromide from his good friend, Marie Curie, who, obviously, also was a leader, really the leader in figuring out much of how radioactive materials work.
She has two Nobel Prizes for her work in radioactive materials.
And something that maybe many of you think, which I know I always think when I hear about radioactivity studies in the early 1900's, is oh, my gosh, this sounds really dangerous, right, they're using radium bromide, and this is pretty dangerous radioactive material.
So, for those of you that don't know radium is extremely radioactive, even in the range of radioactivity, and one of the major problems with it is that if it does get in your body, the radium is treated as calcium in your body.
So, you can imagine what happens as it gets deposited into your bones, which is not the ideal situation after a long day in the lab.
So, this is really a pretty dangerous situation that's always interesting to point out.
He got this from Marie Curie -- you can imagine they used the postal service, I'm not sure how else they would have transferred it to each other.
So, it really brings up some issues.
The first thing I did when I heard that is actually look up to see how, in fact, Ernest Rutherford did die in 1937, and you'll be happy to know, it actually wasn't from radiation poisoning or from bone cancer, so that's really good that that worked out okay for him, and that he did get to, sort of safely, at least end his life before the radiation ended it.
But it's really interesting the studies that he did do with radium bromide, and he was studying the alpha particles.
And what was known about alpha particles at the time is that they were these charged particles and that they were very heavy.
Does anyone know more than what Rutherford knew at the time, what alpha particles actually are?
Yeah, good.
So, they're actually helium atoms, helium ions.
And this wasn't really important for the studies, it didn't matter that didn't know what they are, but it's nice to kind of know now -- that we do know what they were using.
And he was doing quite a few studies with them.
One experiment that he was doing is detecting the number of particles that were being emitted by this radium bromide as a rate, so he would measure the number of particles per minute that the radium bromide was emitting.
And what he used was a detector here, so he here could detect how many particles were hitting this detector.
He had actually developed this detector with a postdoc by the name of Hans Geiger.
Does that name ring a bell?
Geiger counter
Um-hmm, a Geiger counter.
So, this, in fact, is my very schematic representation of a Geiger counter.
For those of you don't know what that is, it's simply an instrument that counts radioactive particles in the air, and now that you're at MIT, you'll all have a chance to see one first hand, if you're ever in any of the labs, especially in the chemistry or bio labs.
As carefully as people work with radioactivity here, and using often much, much safer radioactive materials than radium bromide, and using them, and special hoods, and having special procedures, they still do a lot of checking with these Geiger counters to make sure everything's safe.
You'll actually see a man walking around with one, sometimes in the halls, just kind of like this -- you hear that click, click, click.
That's a good sound, it means low levels of radiation.
He'll walk by your hood, so click by your hood -- I always get a little nervous when he walked by my hood, I don't know why, I never worked with radioactive material.
But I was convinced I'd hear the click-click-click-click-click, which is what tells you you're in trouble.
So, I've never heard the click-click-click-click-click, and we might bring a Geiger counter in here some time later in the semester so we can check all of you out, and hopefully we won't hear any when we do that either.
So, one thing that he discovered with this detector initially, and he was the first to discover this, is that radioactive material, including radium bromide, have a characteristic rate that they emit, radioactive decay.
So basically they're decaying at a constant rate, which means, of course, that you can figure out how old things are by seeing how much they've decayed.
So, he was really the first person to discover you could do this, which was used to make the first somewhat close approximation of the age of the earth.
So that's a pretty exciting set of experiments he did.
But one thing that he wanted to do specific to understanding the atom, and using these alpha particles, these heavy-charged particles, was to test if this Plum Pudding model actually fit what he could observe.
So, what he did was he first recorded the count rate of radium bromide before it's going through any kind of a plum pudding atom, and he found that it had a count rate of 132,000 alpha particles per minute were being detected by this Geiger counter.
He then set up a situation where he put a very, very thin piece of gold foil right in what would be in the stream of the alpha particles.
So, it was only 10 to the negative, 9 meters thick, so about one nanometer, so that's really thin, it's thinner than a strand of hair.
So you can imagine, we actually don't need to think of it as a piece of gold foil, it might be easier to think of it as a couple of layers of atoms.
So basically he's trying to put some atoms in the way of the alpha particle.
And what he would expect is if this Plum Pudding model is true, nothing's really going to happen to the particles, right, they should go straight through, because if they hit an electron, those are so small.
We figured out the mass is so tiny that it shouldn't really deflect them very much.
And, of course, all that's left is this positive pudding.
So that's not going to do anything either.
And what he found when he did this experiment, was that the count rate with still 132,000 counts per minute.
So, what he could conclude thus far was that this was really consistent with the Plum Pudding model.
All of his heavily-charged alpha particles were going right through this thin layer of gold atoms.
So, you might think that he would stop his experiments here, and maybe he would have, but as I mentioned, he did have a postdoc working with him by the name of Geiger.
He also had an undergraduate, we could say maybe even a UROP working with him, and this was by the name of Marsden was the name of this UROP.
And Rutherford realized, you know I have these two people that are very excited to work on this project, I don't need to spend time doing it.
Maybe it's not the best way for me to spend time looking to see if I can find any bounced-back particles since all the particles are accounted for.
But, you know, this undergraduate's very eager to do it, let's let him have a try.
And something you might find in your UROP experience is you have a unique advantage as an undergraduate, which is that there's not a lot of pressure to actually make a huge discovery or necessarily accomplish a great amount.
You have a little more pressure in grad school, but sometimes that means when you're an undergrad your advisor will decide to put you on projects that maybe when you look at them seem a little bit silly.
So this project was, let's see if we can detect any alpha particles by making a detector that swings around.
So, some people might say, why are we doing this?
We know we started with a 132,000 alpha particles.
We detected a 132 alpha particles.
What are we even looking for?
We have to build this whole new detector, is this really the best use of my time?
As an undergrad, you don't have to worry about it, you're just worried about learning.
You can take these big risks of time, and if at the end of the day there's nothing to detect, you still know how to build a detector.
So, keep that in mind if you're not over-the-top excited about the prospects of some of your research.
You might be surprised at what you find out.
And this is exactly what happened with Marsden who discovered that when he shot the alpha particles at the gold foil, he detected something on his detector that click, click, click went a little bit faster.
So, what he detected was that there were 20 alpha particles per minute.
Does that sound significant?
It depends, right?
So hopefully, the first experiment he did, which I know that they certainly did do was maybe it's just background noise, right?
So, they took away that gold foil and said is just the alpha particles hitting it some other way?
And no, it wasn't.
When he took away the gold foil, the count rate went down to zero.
If he switched from gold to let's say iron, he also tried platinum, a number of different foils, he found that they count rate, it still was 20 alpha particles per minute.
So, this is an absolutely outstanding discovery, even though, if we think about it, what is the probability that this happened, how often did this happen?
It actually almost happened not at all.
We can figure out exactly what the probability of this backscattering was just by dividing the count rate of the number particle that were backscattered divided by the count rate of the incident particles.
So essentially, we just have 20, and our 20 is divided by 132,000.
So, we end up with a not so large probability of 2 times 10 to the negative 4.
But still, we can't even overstate how exciting this discovery was.
Rutherford, the advisor here, he had a lot of good things happen in his life, as I mentioned.
He was the person responsible for being able to first date the age of our earth.
That's a pretty nice thing.
He was also married, he had a child, which I hear is very nice, very exciting, also.
But yet, when he saw this one single experiment from this undergraduate, he described this as the most incredible event that had ever happened to him in his life.
So, this was a pretty big deal.
We won't tell his daughter.
And he gave a very good analogy in saying, "It was almost as incredible as if you'd fired a 15 inch shell at a piece of tissue paper, and it came back and hit you."
And that really illustrates what's happening here, because if we think of the Plum Pudding model, it's essentially this very thin film, right, there's nothing that should hit if we send alpha particles through it.
But what we actually have is that something's bouncing back.
So, what happened is Rutherford needed to come up with a new model for the atom with several interpretations that came out of these experiments, and some of these interpretations were that, of course, we now know that these gold atoms, they must be mostly empty, and the reason that we know that they must be mostly empty is because all but 20 of these 132 particles went all the way through.
So they weren't hitting anything, we're dealing with mostly empty space.
But he also realized that when they did hit something, what they hit what unbelievably massive, but also that that mass was concentrated into this very, very small space.
So eventually, this is what we have come to call the nucleus of an atom.
And the nucleus name was used as an analogy to the nucleus of a cell, so in some ways that makes it easier to see the connection, but I think it can also be a little bit confusing for maybe 7th graders that are learning both at the same time, that this nucleus acts very different from a nucleus in a cell, although, of course, there many of them in the nucleus of a cell.
There are some other things that Rutherford was able to figure out.
One is the diameter of the nucleus, and that turns out to be 10 to the negative 14 meters.
If we think about the size of a typical cell -- excuse me, now I'm getting confused about nuclei.
If we think of the size of a typical atom, we would say that would be about 10 to the negative 10 meters.
So, we can see the diameter of a nucleus is absolutely smaller, really concentrating that mass into a very small space.
So, you might be asking how did he actually figure that out?
We'll do the calculation ourselves.
In fact, we'll do the whole experiment ourselves, minus the radioactivity in just a minute, so we'll be able to answer that question for you.
He also figured out that the charge of the nucleus was a plus ze.
This makes sense intuitively as well, because z is just the atomic number.
So, let's say we have an atomic number of 3, that means we have 3 electrons, so we better hope to get our neutral atom that we have a charge of plus 3 in the nucleus.
So, I mentioned at the beginning, while he was working with this radium bromide, that I was very relieved to see that it did not kill him to do these experiments.
However, I think I will share with you that the cause of his death was, in fact, related to his research here, even though it was a little more tangled up.
So, what happened, of course, after he discovered the nucleus, not surprising, he won a Nobel Prize for this -- I would hope that he would.
And in addition to winning a Nobel Prize, he was also knighted, which was a nice bonus for someone born in England, that's a great thing to happen to them.
The problem that he ran into is at some point a little bit later in his life, he had a hernia which was a pretty standard case, but what he was going to need was an operation on it.
And the glitch came that at least at the time, if you were a knight, you could only be operated on by a doctor that was also titled.
So, Rutherford had a little bit of waiting to do for that doctor to show up, and it turns out the wait was too long, and he actually passed away because he discovered the nucleus and got a Noble Prize and became knighted.
So, it's still dangerous.
If that opportunity comes up for you, maybe you want to check into the policies of how that works with the doctor situation now.
Hopefully they've cleared it up a little bit.
So, what we want to do now is see if we can understand how this backscattering experiment worked.
So, we will do our own backscattering experiment.
And we'll ask you to imagine a few things.
First is that we have this mono layer of gold particles.
So let's see if Professor Drennan is able to help us out here.
Oh, great.
So, that is her daughter, Sam that you see strapped to the chest, and Dr. Patti Christie helping us out here.
All right.
So, we'll move this up to the front in just a minute, but I'm going to explain how this experiment works, and we'll do the calculation first before the excitement breaks out.
But I'm sure you can easily see how these styrofoam balls could, in fact, be a mono layer of gold nuclei.
We have 266, as some of you might know who saw me counting ping-pong balls the other day in office hours.
We have 266 ping-pong balls, and we need someone, hopefully you, to be some radioactive material that are going to be emitting these ping-pong balls.
And when the time comes, in just a minute, I'll ask the TAs to come down and hand these out very quickly to you, so we can do this experiment.
But first, let's go through how we're going to determine what Rutherford determined, which was he was interested in knowing, which we said what the diameter of the nuclei were.
So, we're going to do the same thing and figure out the diameter of these styrofoam balls here, and we can do it by using the relationship of how many backscatter.
So, if we think about the probability of backscattering, which is the exact same thing that we saw Rutherford calculate, using the 20 divided by 132,000.
But in our case, the probability of backscattering is going to be the number of balls that backscatter, and that's going to be divided by the total number of ping-pong balls.
So, do you remember what that was?
266
266.
Good information retention.
All right.
So, we have the probability here.
So, in terms of the number of balls scattered over the total, we can also relate the probability to the area of all of those nuclei divided by the total area that the atoms take up.
Right, this makes a lot of sense, because if the entire atom was made up of nuclei, then we would have 100% probability of hitting one of these nuclei and having things bounce back.
So, here we have the area of the nuclei we'll figure out adding those all together versus the space of all of the atoms put together.
So, not only did Professor Sayer, who's in the Chemistry Department who put together this contraption for all of you, not only did she magnify the size of these gold nuclei, but she actually had to smoosh all of these atoms closer together then they normally would be.
If, in fact, a gold nucleus was this size here, we would need to use another lecture hall in order to find a place to put this nucleus right here.
This is a little bit of a tricky experiment, so we decided we'll just smoosh it all in, and we'll actually be able to account for it, because we'll take into account the area of all of those atoms.
I think this board does not like to go by itself.
All right.
So we can figure out what that is, the area of all of the nuclei is going to be the number of nuclei times the area per nucleus, and we're going to talk about the cross-section here to keep it simple.
And all of that is divided by the area of the atoms, which is 1 .
39 meters squared, measuring that space there.
So, the number of nuclei, if we were to sit and count these as well, is 119.
So, we'll multiply that by just pi, r squared, to get that cross-section, and divide all of that by 1 .
39 meters squared.
So, what we have here is a relationship that can tell us what the probability of backscattering is, but what we want to pull out, since we can experimentally measure what the probability is, what we need to pull out is the radius or the diameter of these nuclei, so we can just, instead of solving for p, we can switch it around and solve for the radius.
So, that's going to be equal to the probability raised to the 1/2 times 6 .
098 times 10 to the negative 2 meters.
So, actually, just for discussion sake, it makes a little more sense for us to talk about the diameter, so that's just twice the radius.
So, once we figure out what our probability of backscattering is, we'll just raise that to the 1/2, and we'll multiply that by 12 .
20 centimeters.
All right.
So now all we have to do is figure out this probability of backscattering.
We know we need to divide by 266, but what we need you to help us with is to figure out this top number here and see how many particles are going to backscatter.
So, if the TAs can come up and quickly hand out 1 particle to everyone.
And a few people will need to throw 2, if you feel like you have particularly good aim
So, as you're getting your ping-pong balls -- do not throw them yet.
Let me explain to you what constitutes a backscatter event.
So, it'll be considered a backscatter event if your ping-pong ball hits one of the nuclei.
It's not going to be a backscatter event if your ping-pong ball hits the frame or these strings, or the top part.
So, in a few minutes, not now, we're going to ask you to stand up, and you can kind of come over more toward the center of the room if you want, and aim your ping-pong ball at the lattice here, follow the ping-pong ball with your eye, and discover, watching it, whether it's a backscatter -- it hits one of the nuclei and bounces back towards you, or if it goes through, and also if your ping-pong ball doesn't land anywhere in the vicinity of this at all, then keep that in mind.
And then at the end of the experiment we'll ask you what happened to your ping-pong ball, and you'll let us know, and we can calculate the number of backscatter events.
Are there any questions before we get started?
Raise your hand if you don't if you don't have a ping-pong ball yet.
Any questions before we get started?
All right.
So, we'll come around and get ping-pong balls to the rest of you.
Those of you who have your ping-pong balls can now begin the experiment.
[EXPERIMENTING]
All right.
Any last shots?
There we go.
All right.
So, it looks like we were a little bit successful, I saw some backbouncing.
We were going to have a clicker slide on how many bounced back, but it looks like we're having a little technical difficulty with that.
So, what I'll ask is can you stand up if you had your particle bounce back?
All right, so let's count how many we have here.
So, 13 backscattered.
TAs, if you can maybe pick up these ping-pong balls for me.
I'm sure it would be very amusing if I fell, but I'd rather not.
All right, so, we have 13 divided by 266.
All right, MIT students, who has a calculator on them?
Actually, I should probably do it as well, so I know I'm hearing correctly.
So, are you getting 0 .
0489 or so?
All right.
So, we've got our probability.
We can go ahead and plug that in, take the square root of it, multiply it by 12 .
2.
What are you getting for your diameters?
Yup, that's what I got, too.
All right.
So, we have 2 .
70 for our diameter, and that's in centimeters.
So, we actually did a pretty good job here.
It turns out that the diameter is actually 2 .
5 centimeters.
So, good job, experiment well done, plus we were not exposed to radioactivity, which is a bonus.
So, this is exactly how Rutherford did discover that these particles were present and made this new model for the atom that we now know has both a nucleus, and we know the size, and also has electrons.
So, to finish up today, we won't get through all of it.
But the next thing we can actually talk about is now that we know we have an atom that has a nucleus, let's say somewhere in the center, and it has electrons around it, thinking on our most simple example, which is hydrogen, we have a nucleus and an electron that have to hang together in the atom in some way, and we need to think about well how can we describe how atoms behave, and specifically, how do we describe how any single atom stays together where the two are associated, but at the same time they don't immediately collapse into themselves.
So, what we can do is try using the classical description of the atom and see where this takes us.
So, if we think about the force that occurs between a positively and a negatively charged particle, what we have is essentially a Coulomb force, so we can describe this as a force of attraction.
We can use the Coulomb force law to explain this where we can describe the force as a function of r. So, let's think about what we're saying here.
We're describing the force that's holding these two particles together, and it's related to the charge of each of the particles, where e is the absolute value of an electron's charge.
So, an electron has a charge of negative e, we've written here, and the nucleus has a charge of positive e. And then we have r, which is simply the distance between the two charges.
And what we see is that the force is inversely related to the distance between the two charges.
And we can simplify this expression as saying negative e squared over 4 pi, epsilon nought r squared.
Epsilon nought is a constant, it's something you might see in physics as well.
Essentially for our purposes here, you can just think of it as a conversion factor.
What we need to do is get rid of the Coulomb tag that we have -- that's how we measure our electron charges -- charge, and so we use this epsilon nought quite often, this permativity constant of a vacuum to make that conversion.
And I'll just point out here also, this is a conversion factor you'll use quite frequently -- many of you, quite on accident, will memorize it as you use it over and over again.
But I do want to point out that you don't have to memorize it for any exams in this class, we will give you a sheet that has all the needed constants that you're going to use on there, so save up that brain space for other information.
ah So, we can use Coulomb's force law, and we can think about these different scenarios.
So, when you come in on Monday, we're going to start off, you can think for the weekend -- you probably only need to think for a second about what happens when r goes to infinity, but that's where we'll start on Monday.
And let me just suggest to all of you also, that you get those problem sets started this weekend.
You should absolutely finish, at least through part a this weekend, and save part b for next week.
So, have a great weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And this is a question based on where we left off on Wednesday -- we were talking about Coulomb's force law to describe the interaction between two particles, and good job, most of you got this correct.
So, what we're looking at here is the force when we have two charged particles, one positive, one negative -- here, the nucleus and an electron.
So, I know this is a simple example and I can see everyone pretty much got it right, and probably those that didn't actually made some sort of clicker error is my guess.
But I wanted to use this to point out that in this class in general, any time you see an equation to explain a certain phenomenon, such as here looking at force, it's a good idea to check yourself by first plugging it into the actual equation, so you can plug in infinity and this equation here, and what you would see is, of course, the force, if you just solve the math problem goes to zero.
But you can also look at it qualitatively, so, if you think about the force between the electron and the proton, you could just qualitatively think about what's happening.
If they're close together there's a certain force -- they're attracted because they have opposite charges, but as that gets further and further away, that force is going to get smaller and smaller, and eventually the force is going to approach zero.
So, it's a good kind of mental check as we go through this course to remember every time there's an equation, usually there's a very good reason for that equation, and you can go ahead and just use your qualitative knowledge, you don't have to just always stick with the math to check and justify your answers.
So, we can get started with today's lecture notes.
And, as I mentioned, we left off and as we started back here to describe the atom and how the atom holds together the nucleus and the electron using classical mechanics.
And today we'll finish that discussion, and, of course, point out actually the failure of classical mechanics to appropriately describe what's going on in an atom.
So, then we'll get to turn to a new kind of mechanics or quantum mechanics, which will in fact be able to describe what's happening on this very, very small size scale -- so on the atomic size scale on the order of nanometers or angstroms, very small particles.
And the reason that quantum mechanics is going to work where classical mechanics fails is that classical mechanics did not take into account the fact that matter has both wave-like and particle-like properties, and light has both wave-like and particle-like properties.
So, we'll take a little bit of a step back after we introduce quantum mechanics, and talk about light as a wave, and the characteristic of waves, and then light as a particle.
And one example of this is in the photoelectric effect.
So, we just talked about the force law to describe the interaction between a proton and an electron.
You told me that when the distance went to infinity, the force went to zero.
What happens instead when the distance goes to zero?
What happens to the force?
Yeah.
So, the force actually goes to infinity, and specifically it goes to negative infinity.
Infinity is the force when we're thinking about it and our brains, negative infinity is when we actually plug it into the equation here, and the reason is the convention that the negative sign is just telling us the direction that the force is coming together instead of pushing apart.
So, we can use Coulomb's force law to think about the force between these two particles -- and it does that, it tells us the force is a function of that distance.
But what it does not tell us, which if we're trying to describe an atom we really want to know, is what happens to the distance as time passes?
So, r is a function of time.
But luckily for us, there's a classical equation of motion that will, in fact, describe how the electron and nucleus change position or change their radius as a function of time.
So, that's -- does anyone know which classical law of motion that would be?
Yup, so it's going to be Newton's second law, force equals mass times acceleration -- those of you that are quick page-turners, have a little one-up on answering that.
And that tells us force as a function of acceleration, we want to know it though as a function of radius, so we can just take the first derivative and get ourselves to velocity.
So, force is equal to mass times dv /dt.
But, of course, we want to go all the way to distance, so we take the second derivative and we have this equation for force here.
And what we can do in order to bring the two equations together, is to plug in the Coulomb force law right here.
So, now we have our Coulomb force law all plugged in here, and we have this differential equation that we could solve, if we wanted to figure out what the force was at different times t, or at different positions of r. So, all you will have the opportunity to solve differential equations in your math courses here.
We won't do it in this chemistry course.
In later chemistry courses, you'll also get to solve differential equations.
But instead in this chemistry course, I will just tell you the solutions to differential equations.
And what we can do is we can start with some initial value of r, and here I write r being ten angstroms.
That's a good approximation when we're talking about atoms, because that's about the size of and atom.
So, let's say we start off at the distance being ten angstroms.
We can plug that into this differential equation that we'll have and solve it, and what we find out is that r actually goes to zero at a time that's equal to 10 to the negative 10 seconds.
So, let's think qualitatively for a second about what that means or what the real meaning of that is.
What that is telling us is that according to Newtonian mechanics and Coulomb's force law, is that the electron should actually plummet into the nucleus in 0.1 nanoseconds.
So, we have a little bit of a problem here.
And the problem that we have is that what we're figuring out mathematically is not exactly matching up with what we're observing experimentally.
And, in fact, it's often kind of difficult to experimentally test your mathematical predictions -- a lot of people spend many, many years testing one single mathematical prediction.
But, I think all of us right now can probably test this prediction right here, and we're observing that, in fact, all of us and all the atoms we can see are not immediately collapsing in less than a nanosecond.
So, just, if you can take what I'm saying for a moment right now that in fact this should collapse in this very small time frame, we have to see that there's a problem with one of these two things, either the Coulomb force law or Newtonian mechanics.
So, what do you guys think is probably the issue here?
So, it's Newtonian mechanics, and the reason for this is because Newtonian mechanics does not work on this very, very small size scale.
As we said, Newtonian mechanics does work in most cases, it does work when we're discussing things that we can see, it does work even on things that are too small to measure.
But once we got to the atomic size scale, what happens is we need to be taking into account the fact that matter has these wave-like properties, and we'll learn more about that later, but essentially classical mechanics does not take that into account at all.
So, we need a new kind of mechanics, which is quantum mechanics, which will accurately explain the behavior of molecules on this small scale.
So, as I mentioned, the real key to quantum mechanics is that it's treating matter not just like it's a particle, which is what we were just doing, but also like it's a wave, and it treats light that way, too.
The second important point to quantum mechanics is that it actually considers the fact that light consists of these discrete packets or particle-like pieces of energy, which are called photons.
And if you think about what's actually happening here, this second point that light consists of photons is actually the same thing as saying that light shows particle-like properties, but that's such an important point that I put it separately, and we'll cover that separately as we go along.
So, we now have this new way of thinking about how a nucleus and an electron can hang together, and this is quantum mechanics, and we can use this to come up with a new way to describe our atom and the behavior of atoms.
But the problem is before we do this, it makes sense to take a little bit of a step back and actually make sure we're all on the same page and understanding why quantum mechanics is so important and how it works, and specifically understanding what we mean when we say that light is both a particle and a wave, and that matter is both a particle and a wave.
So, we'll move on to this discussion of light as a wave, and we really won't pick up into going back to applying quantum mechanics to the atom until Friday, but in the meantime, we'll really get to understand the wave particle duality of light and of matter.
So, we'll start with thinking about some properties of waves that are going to be applicable to all waves that we're talking about, including light waves.
The easiest kind of waves for us to picture are ocean waves or water waves, because we can, in fact, see them, but they have similar properties to all waves.
And those properties include that you have this periodic variation of some property.
So, when we're talking about water waves, the property we're discussing is just the water level.
So, for example, we have this average level, and then it can go high where we have the peak, or it can go very low.
We can also discuss sound waves, so again it's just the periodic variation of some property -- in this case we're talking about density, so we have high density areas and low density areas.
So, regardless of the type of wave that we're talking about, there's some common definitions that we want to make sure that we're all able to use, and the first is amplitude.
And when we're talking about the amplitude of the wave, we're talking about the deviation from that average level.
So, if we define the average level as zero, you can have either a positive amplitude or a negative amplitude.
So, sometimes people get confused when they're solving problems and call the amplitude this distance all the way from the max to the min, but it's only half of that because we're only going back to the average level.
So, what we really want to talk about here is light waves, and light waves have the same properties as these other kind of waves in that they're the periodic variation of some property.
So, when we're discussing light waves, what we're talking about is actually light or electromagnetic radiation, is what we'll be calling it throughout the course.
And that's the periodic variation of an electric field.
So, instead of having the periodic variation of water, or the periodic variation of air density, here we're talking about an electric field.
We know what an electric field is, it's just a space through which a Coulomb force operates.
And the important thing to think about when you're talking about the fact that it's a periodic variation, is if you put a charged particle somewhere into an electric field, it will, of course, go in a certain direction toward the charge it's attracted to.
But you need to think about the difference, if you have a particle here on your wave, it will go in one direction.
But remember, waves don't just have magnitude, they also do have direction.
So, if instead you put your particle somewhere down here on the electric field, or on the wave, the electric field will now be in the other direction, so your particle will be pushed the other way.
And from physics you know that, of course, if we have a propagating electric field, we also have a perpendicular magnetic field that's going back and forth.
But in terms of worrying about using the concepts of a wave to solve chemistry problems in this course, we can actually put aside the fact, and only focus on the electric field part of things, because that's what's going to be interacting with our charged particles, such as our electrons.
So, other properties of waves that you probably are all familiar with but I just want to review is the idea of a wavelength.
If we're talking about the wavelength of a wave, we're just talking about the distance that there is between successive maxima, or of course, we can also be talking about the distance between successive minima.
Basically, we can take any point on the wave, and it's the distance to that same point later on in the wave.
So, that's what we call one wavelength.
We also commonly discuss the frequency of a wave, and the frequency is just the number of cycles that that wave goes through per unit time.
So, by a cycle we'd basically mean how many times we cycle through a complete wavelength.
So, if something cycles through five wavelengths in a single second, we would just say that the frequency of that wave is five per second.
We can also mathematically describe what's going on here other than just graphing it.
So, if we want to look at the mathematical equation of a wave, we want to describe -- again as I mention, what we're describing is the electric field, we're not worrying about the magnetic field here, as a function of x and t that's equal to a cosine [ 2 pi x over wavelength, minus 2 pi nu t ].
And note this is the Greek letter nu.
This is not a v. Where we have E, which is equal to the electric field, what is x?
Position
Yup, the position of the wave.
And what about t?
Yeah, so we're talking about both position and time.
So what we can do if we're talking about a wave is think of it both in terms of position time, but if we're trying to visualize this -- for example if we're actually to graph this out, the easiest thing to do is keep one of these two variables constant, either the x or the t, and then just consider the other variable.
So, for example, if we're to hold the time constant, this makes it a lot simpler of an equation, because what we can end up doing is actually crossing out this whole term here.
So what we're left with is just that the electric field as a function of distance is a times cosine of the argument there, which is now just 2 pi x over wavelength.
So, what we want to be able to do, either when we're looking at the graph or looking at the equation up there, is to think about different properties of the wave.
For example, to think about at what point do we have the wave where it's at its maximum amplitude?
So, if we think about that, we need to have a point where we're making this argument of the cosine such that the cosine is going to all be equal to one, so all we're left with is that a term.
So, we can do that basically any time that we have an integer variable that is either zero or an integer variable of the wavelength.
So, for example, negative wavelength or positive wavelength are two times the wavelength, because that lets us cross out the term with the wavelength here, and we're left with some integer multiple of just pi.
So, that's sort of the mathematically how we get to a, but we can also just look at the graph here, because every time we go one wavelength, we can see that we're back in a maximum.
So, I mentioned we should be able to figure out where the maximum amplitude is.
You should also just looking at an equation, immediately be able to figure out what that maximum amplitude is in terms of the height of it just by looking at that a-term, here we should also be able to know the intensity of any light wave, because intensity is just the amplitude squared.
So, we should immediately be able to know how bright or how intense a light is just looking at the wave equation, or just by looking at a graph.
We can also do a similar thing, and I'll keep my distance from the board, but we can instead be holding x constant, for example, putting x to be equal to zero, and then all we're doing is considering the electric field as a function of t. So, in this case we're crossing out the first term there, and we're left with amplitude times the cosine of 2 pi nu times t. And, of course, we can do the same thing again, we can think about when the amplitude is going to be at its maximum, and it's going to be any time cosine of this term now is equal to one.
So that will be at, for example, negative 1 over nu, or 0, or 1 over nu.
And again, we can just look at our graph to figure that out, that's exactly where we're at a maximum.
So, 1 over nu is another term we use and we call it the period of a wave, and the period is just the inverse of the frequency.
And if we think about frequency, that's number of cycles per unit time.
So, for example, number of cycles per second, whereas the period is how much time it takes for one cycle to occur.
And when we talk about units of frequency, in almost every case, you'll be talking about number of cycles per second.
So, you can just write inverse second, the cycle part is assumed.
But you'll also frequently see it called Hertz, so, Hz here.
So, if you're talking about five cycles per second, you can write five per second, or you can write five Hertz.
The one thing you want to keep in mind though is that Hertz does not actually mean inverse seconds, it means cycles per second.
So, if you're talking about a car going so many meters per second, you can't say it's going meter Hertz, you have to say meters per second.
So, this really just means for frequency, it's a frequency label.
Alright.
So, since we have these terms defined, we know the frequency and the wavelength, it turns out we can also think about the speed of the wave, and specifically of a light wave, and speed and is just equal to the distance that's traveled divided by the time the elapsed.
And because we've defined these terms, we have ways to describe these things.
So, we can describe the distance that's traveled, it's just a wavelength here.
And we can think about how long it takes for a wave, because waves are, we know not just changing in position, but the whole wave is moving forward with time, we can think about how long it takes for wave to go one wavelength.
So, one distance that's equal to lambda.
So, how much time would that take, does anyone know?
So, would it take, for example, the same amount of time as the frequency?
The period, that's right.
So, it's going to take one period to move that long.
And another way we can say period is just 1 over nu or 1 over the frequency So, now we know both the distance traveled and the time the elapsed.
So, we can just plug it in.
Speed is equal to the distance traveled, which is lambda over the time elapsed, which is 1 over nu.
so, we can re-write that as speed is equal to lambda times nu, and it turns out typically this is reported in meters per second or nanometers per second.
So, now we have an equation where we know the relationship between speed and wavelength and frequency, and it turns out that we could take any wave, and as long as we know the frequency and the wavelength, we'll be able to figure out the speed.
But, of course, there's something very special about electromagnetic waves, electromagnetic radiation and the speed.
And it's not really surprising for me to tell you that electromagnetic radiation has a constant speed, and that speed is what we call the speed of light, and typically we abbreviate that as c, and that's from the Latin term celeritas, which means speed in Latin.
That's one of four or five Latin words I remember from four years of high school Latin, but it comes in handy to remember speed of light.
And some of you may have memorized what the speed of light is in high school -- it's about 3 times 10 to the 8 meters per second.
This is another example of a constant that you will accidentally memorize in this course as you use it over and over again.
But again, that we will supply for you on the exam just in case you forget it at that moment.
And this is a very fast speed, of course, it's about 700 million miles per hour.
So, one way to put that in perspective is to think about how long it takes for a light beam to get from earth to the moon.
Does anyone have any guesses?
Eight seconds, that sounds good.
Anyone else?
These are all really good guesses, so it actually takes 1.2 seconds for light to travel from the earth to the moon.
So, we're talking pretty fast, so that's nice to appreciate in itself.
But other than that point, we can also think about the fact that frequency and wavelength are related in a way that now since we know the speed of light, if we know one we can tell the other.
So, you can go ahead and switch us to our clicker question here.
So, we should be able to look at different types of waves and be able to figure out something about both their frequency and their wavelength, and know the relationship between the two.
So, it's up on this screen here now, so we'll work on the other one.
If you can identify which of these statements is correct based on what you know about the relationship between frequency and wavelength and also just looking at the waves.
Alright.
So, let's give ten more seconds on that.
So, ten seconds on that.
Alright.
So, good job.
So, most people could recognize that light wave a has the shorter wavelength.
We can see that just by looking at the graph itself -- we can see, certainly, this is shorter from maxima to maxima.
This we can't even see the next maxima, so it's much longer.
And then, we also know that means that it has the higher frequency, because our relationship between wavelength and frequency are inversely related.
And also, we know the speed of light.
So, if we think about if it's a shorter wavelength, we'll be able to get a lot more wavelengths in, in a given time, than we would for a longer wavelength.
So, we can switch back to the notes and think about what this means, and what this means when we're talking about all the different kinds of light waves we have, and I've shown a bunch here, is that if we have the wavelength, we also know the frequency of these wavelengths.
So, for example, radio waves, which have very long wavelengths have very low frequencies.
Whereas where we go to waves that have very short wavelengths, such a x-rays or cosmic rays, they, in turn, have very high frequencies.
So, it's important to get a little bit of a sense of what all these different kinds of lights do.
You're absolutely not responsible to memorize what the wavelengths of the different types of lights are, but you do want to be able to know the general order of them.
So, if someone tells you they're using UV light versus x-ray light, you know that the x-ray light is, in fact, at a higher frequency.
So that's the important take-away message from this slide.
If we think about these different types of lights, microwave light, if it's absorbed by a molecule, is a sufficient amount of frequency and energy to get those molecules to rotate.
That, of course, generates heat, so that's how your microwaves work.
If we talk about infrared light, which is at a higher frequency here and a shorter wavelength, infrared light when it's absorbed by molecules actually is enough to cause molecules now to vibrate.
If we move up to the more high-frequency and divisible light and all the way into UV light, if you shine UV light at certain molecules, it's going to have enough energy to actually pop those electrons in that molecule up to a higher energy level, which will make more sense once we talk about energy levels in atoms, but that's what UV light can do.
And actually, that's responsible for fluorescence and phosphorescence that you see where typically UV light comes in.
So, if you use a black lamp or something and you excite something up to a higher energy level and then it relaxes back down to its lower energy state, it's going to emit a new wavelength of light, which is going to be visible to you.
X-rays are at even a higher frequency, and those are sufficient to actually be absorbed by a molecule and pop an electron all the way out of that molecule.
You can see how that would be damaging to the integrity of that molecule, that's why x-rays are so damaging -- you don't want to have electrons disappearing for no good reason from your molecules that can cause the kind of mutations we don't want to be seeing in ourselves.
And then also as we go higher, we have gamma rays and cosmic rays.
Within the visible range of what we can see, you also want to know this relative order that's pretty easy -- most of us have memorized that in kindergarten, so that should be fine.
Just remembering that violet is the end that actually has the shortest wavelength, which means that it also has, of course, the highest frequency.
So, just an interesting fact about this set of light, which we're most familiar with, if we think about our vision, it turns out that our vision's actually logarithmic and it's centered around this green frequency.
So, if instead of a red laser pointer here, I had a green one, you'd actually, to our eyes, it would seem like the green one was brighter, even if the intensity was the same, and that's just because our eyes are centered and logarithmic around this green frequency set.
So, using the relationship between frequency and wavelength, we can actually understand a lot about what's going on, and pretty soon we'll also draw the relationship very soon to energy, so it will be even more informative then.
But I just want to point out one of the many, many groups at MIT that works with different fluorescing types of molecules, and this is Professor Bawendi's laboratory at MIT, and he works with quantum dots.
And quantum dots are these just very tiny, tiny crystals of semiconductor material.
They're on the order of one to ten nanometers, and these can be shined on with UV light -- they have a lot of different interesting properties, but one I'll mention is that if you excite them with UV light, they will have some of the electrons move to a higher energy state, and when they drop back down, they actually emit light with a wavelength that corresponds with the size of the actual quantum dot.
So, from what we know so far, we should be able to look at any of these quantum dots, which are depicted as a cartoon here, but here we have an actual picture of the quantum dots suspended in some sort of solution and shone on with UV light, and you can see that you can achieve this whole beautiful range of colors just by modulating the size of the different dots.
And we should be able to know if we're looking at a red dot -- is a red dot, it's going to have a longer wavelength, so is this a higher or lower frequency?
Yeah, and similarly, if someone tells us that their dot is blue-shifted, that should automatically in our heads tell us, oh it shifted to a higher frequency.
And these dots are really interesting in that you can, I'm sure by looking at this picture, already imagine just a whole slew of different biological or sensing applications that you could think of.
For example, if you were trying to study different protein interactions, you could think about labeling them with different colored dots, or there's also a bunch of different fluorescent techniques that you could apply using these dots, or you could think of in-vivo sensing, how useful these could be if you could think of a way to get them into your body without being too toxic, for example.
These are all things that the Bawendi group is working on.
What they are real experts in is synthesizing many different kinds of these dots, and they have a synthetic scheme that's used by research groups around the world.
The Bawendi group also collaborates with people, both at different schools and at MIT.
One example, on some of their biochemistry applications is with another Professor at MIT, Alice Ting and her lab.
So really what I want to point out here is as we get more into describing quantum mechanics, these quantum dots are one really good example where a lot of the properties of quantum mechanics apply directly.
So, if you're interested, I put the Bawendi lab research website onto your notes.
And also, Professor Bawendi recently did an interview with "The Tech."
Did anyone see that interview in the paper?
So, three or four -- a few of you read the paper last week.
So, you can either pick up an old issue or I put the link on the website, too.
And that's not just about his research, it's also about some of his memories as a student and advice to all of you.
So, it's interesting to read and get to know some of these Professors at MIT a little bit better.
So, one property that was important we talked about with waves is the relationship between frequency and wavelength.
Another very important property of waves that's true of all waves, is that you can have superposition or interference between two waves.
So, if we're looking at waves and they're in-phase, and when I talk about in-phase, what I mean is that they're lined up, so that the maxima are in the same position and the minima are in the same position, what we can have a something called constructive interference.
And all we mean by constructive interference is that literally those two waves add together, such as the maxima are now twice as high, and the minima are now twice as low.
So, you can also imagine a situation where instead of being perfectly lined up, now we have the minima being lined up with the maxima here.
So, if we switch over to a clicker question maybe on this screen -- okay, can it be done up there to switch?
So, we're still settling in with the renovations here in this room.
So, why don't you all go ahead and tell me what happens if you combine these two waves, which are now out of phase?
So, let's -- okay, so, why don't you all think about would happen -- we'll start with the thought exercise.
You can switch back to my lecture notes then if this isn't going.
Alright.
So, hopefully what everyone came up with is the straight line, is that what you answered?
Yeah
OK, very good.
And I didn't make you try to draw the added, the superimposed positive construction in your notes, but I think everyone can handle drawing a straight line.
So, you can go ahead and draw what happens when we have destructive interference.
And destructive interference, of course, is the extreme, but you can picture also a case where you have waves that are not quite lined up, but they're also not completely out of phase.
So in that case, you're either going to have the wave get a little bigger, but not twice as big or a little bit smaller.
So, I think the easiest way to think about interference is not actually with light, but sometimes it's easiest to think about with sound, especially when you're dealing with times where you have destructive interference.
Has anyone here ever been in a concert hall where they feel like they're kind of in a dead spot, or you don't quite hear as well, and if you move down just two seats all of a sudden it's just blasting at you -- hopefully not in this room.
But have people experienced that before?
Yeah, I definitely experienced it, too.
And really, all you're experiencing there is destructive interference in a very bad way.
Halls, they try to design halls such that that doesn't happen, and I show an example of a concert hall here -- this is Symphony Hall in Boston, and I can pretty much guarantee you if you do go to this Symphony Hall, you will not experience a bad seat or a dead seat.
This is described as actually one of the top two or three acoustic concert halls in the whole world.
So, it's very well designed such that they've minimized any of these destructive interference dead sounds.
So, it's nice, on a student budget you can go and get the worst seat in the house and you can hear just as well as they can hear up front, even if you can't actually see what's going on.
So, another example of destructive interference is just with the Bose headphones.
I've never actually tried these on, but you see people with them, and what happens here is it's supposed to be those noise cancellation headphones.
All they do is they take in the ambient noise that's around it, and there's actually battery in the headphones, that then produces waves that are going to destructively interfere with that ambient noise.
And that's how it actually gets to be so quiet when you have on, supposedly, these quite expensive headphones.
So, that's light as a wave, and the reason -- well, that was sound as a wave, but light as a wave is the same idea.
And it was really established by the early 1900s that, in fact, light behaved as a wave.
And the reason that it was so certain that light was a wave was because we could observe these things -- we could see, for example, that light defracted, and we could see that light constructively or destructively could interfere with other light waves, and this was all confirmed and visualized.
But also, around the time that Thomson was discovering the electron, there were some other observations that were going on, and the most disturbing to kind of the understanding of the universe was the fact that there were some observations about light that didn't make sense with this idea that light is a particle.
And the photoelectric effect is maybe the most clear example of this.
So, the photoelectric effect is the effect that if you have some metal, and you can pick essentially any metal you want, and you shine light of a certain frequency onto that metal, you can actually pop off an electron, and you can go ahead and measure what the kinetic energy of that electron that comes off is, because we can measure the velocity and we know that kinetic energy equals 1/2 m b squared, and thanks to Thomson we also know the mass of an electron.
So, this is an interesting observation, and in itself not too disturbing, yet but the important thing to point out is that there's this threshold frequency that is of the metal, and each metal has a different threshold frequency, such as if you shine light on the metal where the frequency of the light is less than the threshold frequency, nothing will happen -- no electron will pop off of that metal.
However, if you shine a light with a frequency that's greater than the threshold frequency, you will be able to pop off an electron.
So, people were making this observation, but this wasn't making any sense at all because there was nothing in classical physics that described any sort of relationship between the frequency of light and the energy, much less the energy of an electron that would get popped off of a metal that would basically come off only when we're hitting this threshold frequency.
So, what they could do was actually graph what was happening here, so we can also graph what was happening, and what they found was that if we were at any point below the threshold frequency and we were counting the numbers of electrons that were popping off of our metal, we weren't seeing anything at all.
But if you go up the threshold frequency, suddenly you see that there's some number of electrons that comes off, and amazingly, the number of electrons actually had no relationship at all to the frequency of the light.
And this didn't make a lot of sense to people at the time because they thought that the frequency should be related to the number of electrons that are coming off, because you have more frequency coming in, you'd expect more electrons that are coming off -- this wasn't what people were seeing.
So, what they decided to do is just study absolutely everything they could about the photoelectric effect and hope, at some point, someone would piece something together that could explain what's going on or shed some light on this effect.
So, one thing they did, because it was so easy to measure kinetic energy of electrons, is plot the frequency of the light against the kinetic energy of the electron that's coming off here.
And in your notes and on these slides here, just for your reference, I'm just pointing out what's going to be predicted from classical physics.
You're not responsible for that and we won't really discuss it, but it just gives you the contrast of the surprise that comes up when people make these observations.
And the first observation was that the frequency of the light had a linear relationship to the kinetic energy of the electrons that are ejected here.
This made no sense at all to people, and again they saw this effect where if you were below that threshold frequency, you saw nothing at all.
So, that was frequency with kinetic energy.
The next thing that they wanted to look at was the actual intensity of the light and see what the relationship of intensity to kinetic energy is.
So, what we would expect is that there is a relationship between intensity in kinetic energy, because it was understood that however intense the light was, if you had a more intense light, it was a higher energy light beam.
So that should mean that the energy that's transferred to the electron should be greater, but that's not what you saw at all, and what you saw is that if you kept the frequency constant, there was absolutely no change in the kinetic energy of the electrons, no matter how high up you had the intensity of the light go.
You could keep increasing the intensity and nothing was going to happen.
So, we could also plot the number of electrons that are ejected as a relationship to the intensity, so that was yet another experiment they could do.
And this is what they had expected that there would be no relationship, but instead here they saw that there was a linear relationship not to the intensity and the kinetic energy of the electrons, but to the intensity and the number of electrons.
So, none of these observations made sense to any scientists at the time, and really all of these observations were made and somewhat put aside for several years before someone that could kind of process everything that was going on at once came along, and that person was Einstein, conveniently enough -- if anyone could put it together, we would hope that he could, and he did.
And what he did in a way that made sense when all of us look at it, is he plotted all of these different metals on the same graph and made some observations.
So, for example, here we're showing rubidium and potassium and sodium plotted where we're plotting the frequency -- that's the frequency of that light that's coming into the metal versus the kinetic energy of the electron that's ejected from the surface of the metal.
And what he found here, which is what you can see and we can all see pretty clearly, is the slope of all of these lines is the same regardless of what the type of metal is.
So, he fit all these to the equation of the line, and what he noticed was the slope was specifically this number, 6.626 times 10 to the negative 34, joules times seconds.
And he also found that the y intercept for each one of these metals was equal to basically this number here, which was the slope times the minimum frequency required of each specific metal, so that's of the threshold frequency.
And he actually knew that this number had popped up before, and a lot of you are familiar with this number also, and this is Planck's constant.
Planck had observed this number as a fitting constant years earlier when he looked at some phenomena, and you can read about in your book, such as black body radiation.
And what he found was he needed this constant to fit his data to what was observed.
And this is the same thing that Einstein was observing, that he needed this fitting constant, that this constant was just falling right out of, for example, this slope and also the y intercept.
So he decided to go ahead and define exactly what it is, this line, in terms of these new constants, this constant he's calling h, which is Planck's constant.
So, on the y axis we have kinetic energy, so we can plug that in.
If we talk about what the x axis is, that's just the frequency of the light that's coming in.
We know what m is, m is equal to h. And then we can plug in what b is, the y intercept, because that's just the negative of h times that threshold frequency.
So we have this new equation here when we're considering this photoelectric effect, which is that the kinetic energy is equal to h nu minus h nu threshold of the metal.
And what Einstein concluded and observed is that well, kinetic energy, of course, that's an energy term, and h times nu, well that has to be energy also, because energy has to be equal to energy -- there's no other way about it.
And this worked out with units as well because we're talking about joules for kinetic energy, and when we're talking about h times nu, we're talking about joules times second times inverse seconds.
So, the very important conclusion that Einstein made here is that energy is equal to h times nu, or that h times nu is an actual energy term.
And this kind of went along with two observations.
The first is that energy of a photon is proportional to its frequency.
So this was never recognized before that if we know the frequency of a photon or a wave of light, we can know the energy of that light.
So, since we know that there's relationship also between frequency and wavelength, we can do the same thing -- if we know the wavelength, we can know the energy of the light.
And I use the term photon here, and that's because he also concluded that light must be made up of these energy packets, and each packet has that h, that Planck's constant's worth of energy in it, so that's why you have to multiply Planck's constant times the frequency.
Any frequency can't have an energy, you have to -- you don't have a continuum of frequencies that are of a certain energy, it's actually punctuated into these packets that are called photons.
And, as you know, Einstein made many, many, many very important contributions to science and relativity, but he called this his one single most important contribution to science, the relationship between energy and frequency and the idea of photons.
So this means we now have a new way of thinking about the photoelectric effect, and that is the idea that h times nu is actually an energy.
So, it's the energy of an incident photon if we're talking about nu where we're talking about the energy of the photon going in, so we can abbreviate that as e sub i, energy of the incident photon.
We can talk about also h times nu nought, which is that threshold frequency.
So this is a term we're going to see a lot, especially in your problem sets, it's called the work function, and the work function is the same thing as the threshold frequency of a metal, except, of course, that it's multiplied by Planck's constant.
So, it's the minimum energy that a certain metal requires in order to pop a photon out of it -- in order to eject an electron from the surface of that metal.
So this is our new kind of schematic way that we can think about looking at the photoelectric effect, so if this is the total amount of energy that we put into the system, where here we have the energy of a free electron.
We have this much energy going in, the metal itself requires this much energy, the work function, in order to eject an electron.
So that much energy is going to be used up just ejecting it.
And what we have left over is this amount of energy here, which is going to be the kinetic energy of the ejected electron.
So, therefore, we can rewrite our equation in two ways.
One is just talking about it in terms only of energy where our kinetic energy here is going to be equal to the total energy going in -- the energy initial minus this energy of the work function here.
We can also talk about it in terms of if we want to solve, if we, for example, we want to find out what that initial energy was, we can just rearrange our equation, or we can look at this here where the initial energy is equal to kinetic energy plus the work function.
So before we go we'll try to see if we can do a clicker question for you on this, and we can, very good.
So, everyone take those clickers back out and tell me, if a beam of light with a certain energy, and we're going to say four electron volts strikes a gold surface, and here we're saying that the gold surface has a work function of 5.1 electron volts, what is the maximum kinetic energy of the electron that is ejected?
So why don't you go ahead and take ten seconds on that.
And if you don't know, that's okay, just type in an answer and give it your best shot.
And let's see what we come up with here.
Alright.
So, it looks like some of you were tricked, but many of you were not, so no electrons will be ejected.
The reason for that is because this is the minimum amount of energy -- hold off a sec on the packing up, so in case someone doesn't understand -- this is the minimum amount of energy that's required from the energy going in in order to eject an electron.
So if the incident energy is less than the energy that's required, absolutely nothing will happen.
That's the same thing we were talking about with threshold frequency.
All right, now you can pack up and we'll see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Let's get started.
Why doesn't everyone go ahead and take ten more seconds on the clicker question.
All right, and let's see how we did.
Alright, excellent job, 86% of you, that's right.
What we had just done a clicker question on is discussing light as a particle and the photoelectric effect, so we're going to finish up with a few points about the photoelectric effect today.
And then we're going to try a demo to see if we can convince ourselves that the kind of calculations we make work out perfectly, and we'll do a test up here about half way through class.
And we'll also talk about photon momentum as another example of light behaving up as a particle.
After that, we'll move on to matter as a wave, and then the Schrodinger equation, which is actually a wave equation that describes the behavior of particles by taking into account the fact that matter also has these wave-like properties.
So, starting back with the photoelectric effect -- yes
[INAUDIBLE] class last time [INAUDIBLE] notes
Oh, sure.
Can one of the TAs maybe come up and hand around anyone that didn't get notes?
We have not yet perfected the art of entering and exiting this classroom yet, we're still working on that.
Raise your hand if you need notes and we'll make sure we get those to you.
All right, so where we left off with the photoelectric effect was when we first introduced the effect, we were talking about it in terms of frequencies.
So, for example, we were talking about a threshold frequency as in a minimum frequency of light that you need in order to eject an electron from a metal surface.
What Einstein then clarified for us was that we could also be talking about energies, and he described the relationship between frequency and energy that they're proportional, if you want to know the energy, you just multiply the frequency by Planck's constant.
So, now we can talk about it in different terms, for example, talking about e sub i, which is the incident energy or the energy of the light that comes in, or talking about work function here, and that's just another way to say threshold energy.
So, the work function's the minimum amount of energy that's required in order to eject an electron, and most of you understand this relationship here, which is a little bit cut off, but it is all the way on in your notes, and that is what you saw the clicker question on -- how you can figure out, for example, the kinetic energy of the ejected electron by looking at the difference between how much energy you put in and how much energy is required to eject that electron in the first place.
So, in this class we'll be talking about energy a lot, and it's often useful to draw some sort of energy diagram to visualize the differences in energy that we're discussing.
So, we do this here for the photoelectric effect, and in terms of the photoelectric effect, what we know the important point is that the incoming photon has to be equal or greater in energy then the work function of the metal.
So here we have energy increasing on the y-axis, and you see this straight line at the bottom here is lower down on the graph, and that's the energy of a bound electron, so that's going to be a low stable energy.
But we see if we have a free electron, as we do in this dotted line here, that's going to be a higher energy that's less stable.
So, if we want to go from that stable state to that less stable state, we need to put in a certain amount of energy to our system, and that's what we define as the work function here -- that difference between the free electron and the electron bound to the metal.
So, the most basic case to understand, which is what we just saw is a case where we have the incident energy coming in, and that incident energy is greater than the work function, and in that case what we see is that we have an electron that is ejected.
That makes sense and it also makes sense that this little extra bit here, that's the amount of energy that we have that goes into the kinetic energy of the electrons.
So, that's how we could also graph figuring out the kinetic energy.
So, in the second case what we have is what happens if we have the incident energy at some amount that's less than the work function, and in this case we're showing 1/2 of the work function.
So in this case, we don't have enough energy to eject an electron, so, an electron is not ejected.
And that's pretty clear, too, and the question I want to pose to you is instead the third case here.
So in the third case what I'm showing is that we have -- now we're not just talking about 1 photon, we're talking about 3 photons -- let's say we shoot them all at the same time at our metal, each of them having some energy that's let's say 1/2 the work function.
So, just to take a little bit of an informal survey, who thinks here that we will have an electron that is ejected in this case?
So a couple hands, all right.
And what about who thinks that we will not have enough energy here?
All right.
We've got a big majority, and both are logical ways of thinking, but it turns out that the majority is correct, which is not always the case, but the electron is not ejected in this case.
And the reason for this, and this is a very important point about the photoelectric effect, and the point here is that the electrons here are acting as particles, you can't just add those energies together.
One individual particle is being absorbed by the metal and exciting an electron.
So, having other particles around that have the same energy that you could technically add up if you were adding them up like a wave, you can't do the same thing with particles, they're all separate.
So, the take-home message is whether you have three photons or 3,000,000 photons that you're shooting at your metal, if you're not at that minimum frequency or that minimum energy that you need, nothing is going to happen.
So, you might ask then well what is the significance of shooting different amounts of photons at a metal?
Is there any significance at all, for example, in the number of photons that are hitting the metal or being absorbed by the metal.
And there is a relationship here, and that is that the number of photons absorbed by the metal are related to the number of electrons ejected from the metal.
So, in this figure here what I'm actually showing is these little sunshines, which let's say are each one individual photon.
So we had six photons going in, so the maximum number of electrons that we're going to have coming out is also six because the maximum scenario that we could have that would maximize the number of electrons is that each one of those photons comes in, excites an electron, ejects it from the surface of the metal.
It's important to note, of course though, it's not just the number, it's really important that the energy of each one of these individual photons is, of course, greater than the work function of the metal.
So, that's, in fact, it's that number of photons that we're talking about when we refer to the intensity of light, and the intensity of light is proportional to that number, because when we talk about intensity, really we're talking about the amount of energy that a stream of particles, a stream of photons, has per second.
So, if we have a high intensity, we're talking about having more photons per second, and it's important to know also what that does not mean.
So it does not mean that we have more energy per photon.
This is a really important difference.
Intensity, if we increase the intensity, we're not increasing the energy in each photon, we're just increasing the number of photons that we're shooting out of our laser, whatever our light source is.
And when we talk about intensity in terms of units, we usually talk about watts, so if you change your lightbulb, usually you see the intensity in terms of watts.
But in terms of SI units, which become much more useful if you're actually trying to use intensity in a problem and cancel out your units, we're just talking about joules per second is what intensity is.
So at this point, you should be able to have all the background you need on the photoelectric effect to solve any type of problem that we throw at you, and you see three on this problem set, and we'll probably give you one more on your next problem set, and the reason we ask you so many questions about the photoelectric effect is because it actually is very similar to ionization energy that we'll talk about later, also problems dealing with photoelectron spectroscopy.
So, we want to make sure that this is something the entire class is 100% solid on.
Sometimes the questions are worded quite differently, so I just want to sum up here the different ways they could be worded.
For example, if we talk about photons, of course, we also just mean light, sometimes we refer to this as electromagnetic radiation, and there's several ways that you might be asked this in a problem or that you might be asked to answer.
Sometimes we might just directly tell you the energy of the photon -- that's probably the easiest scenario, because when we think about work functions those are usually reported in energy.
So since that's the easiest scenario, you can probably be sure it's not going to be too frequently that you're just given the energy, right, that might be too easy.
So really what we'll probably do is instead either give you the wavelength or the frequency and you'll go ahead and calculate the energy from there.
In terms of talking about the electrons, I wanted to point out that in the book and other places you might see electrons referred to as photoelectrons.
That's sometimes confusing for people, because it seems like okay, is it a photon or is it an electron.
I just want to clarify that it is an electron.
It's called this just because it's an electron that results when an electron absorbs a photon's worth of energy, so thus it's a photoelectron.
And if we talk about electrons or photoelectrons, again we can describe it in terms of energy, we can talk about velocity, and from there, of course, you can figure out the energy from 1/2 m v squared, and actually we can also describe the electron in terms of wavelength.
So you don't actually know this yet from this class, you'll know it by the end of the class that electrons can, in fact, have a wavelength.
So once we cover it, it will then be fair game to ask these photoelectron spectroscopy or these photoelectric effect questions using the wavelength of the electron.
Also to point out, a lot of times you'll see electron volts instead of joules, this is the conversion factor here just so you all have it in your notes.
All right.
So let's test what we, in fact, know about the photoelectric effect, and before we do that actually, we're going to calculate what we would predict, so when we do the demo it will be meaningful and we can tell whether we're successful or not.
So hopefully we will be successful.
And as I point this out, we now know how to do any kind of photoelectric effect problem, also this means you should be able to go back to Monday's notes where we filled in all those graphs, which were what different scientists were observing when they were measuring either the frequency or the intensity of light that was irradiating different types of metals, and also the number of electrons ejected, and the kinetic energy of those electrons ejected.
You should be able to maybe print out a blank copy of those notes from the website and fill in all those graphs -- not for memorizing them, but now just understanding how the photoelectric effect works.
All right.
So let's do an in-class problem, and this will be done with zinc.
We have a zinc plate up here, and we're going to -- in a minute I'll describe how we can probe if electrons are coming off of it.
But we're going to irradiate it with two different light sources.
We have a UV lamp right here, which is centered at a wavelength of 254 nanometers.
And then since we have my red laser pointer, we will also try with the red laser pointer, which is centered at wavelength of 700 nanometers.
So, there are a few questions that we need to answer first.
So we want to see, do we expect to eject electrons off of this metal surface, or do we expect that we don't have enough energy?
So that means we're going to need to figure out what is the energy per photon that's emitted by that UV light.
Also, what's the energy per photon of this red laser pointer, and then it's also worth trying a calculation dealing with intensity.
So let's also try calculating the numbers of photons that would be emitted by this laser pointer, if, for example, we were to use it for 60 seconds and this were a one milliwatt laser.
So, let's do some of these calculations starting first with what is the energy per photon, and let's start with the UV lamp.
So we know that energy is equal to Planck's constant times nu, but what we know about the lamp is its wavelength, or the light that's emitted.
We know that nu is equal to c over wavelength.
So we can figure out the energy of each photon emitted by our UV lamp by saying e is equal to h c over wavelength.
So let's just plug in these numbers here.
That means our energy is equal to 6.626 times 10 to the -34 joules times seconds.
And then we have c, the speed of light, 2.998 times 10 to the 8 meters per second.
And we want to divide all of that by our wavelength, and to keep our units the same we'll do meters.
So that's 254 times 10 to the -9 meters.
So hopefully if some of you have your calculators with you, you can confirm the answer that I got, which is that the energy is 7.82 times 10 to the -19 joules.
So, remember what we're talking about here is the amount of energy that's in each photon.
So if we think about the work function for zinc, and the work function for zinc is 6.9 times 10 to the -19 joules, do we expect that when we shine our UV light on the zinc, we'll be able to eject electrons?
What do you think?
Yes.
Good.
OK, anyone disagree?
No, OK and that's correct, because each photon of light actually has more energy than is needed to eject an electron.
So, we would expect to see electrons ejected with the UV light source.
So let's now think about using instead the amount of energy per photon in that red laser pointer.
So again, we know that energy is equal to h c divided by wavelength, and energy is equal to -- you have written out in your notes what the actual value for h c is, but now our wavelength is 700 times 10 to the -9 meters.
And what we end up with for the energy then is 2.84 times 10 to the -19 joules.
All right.
So please raise your hand now if you think there'll be sufficient energy to eject electrons from the metal surface?
And raise your hands if you think there won't be.
OK. Good hand raising technique.
Yes.
In fact, there is not enough energy in a single photon to go ahead and eject an electron from this zinc surface.
So our last question we ask is what's the total number of photons emitted if we give this given intensity for 60 seconds?
So, keep in mind that one milliwatt is just the same as saying 1 times 10 to the -3 joules per second.
So we have 1 times 10 to the -3 joules per second, and we want to multiply that by -- or cancel out how much energy we have per photon, first of all, so how much energy do we have per photon if we're talking about the red laser pointer?
Right.
So this value right here.
So for every photon we have 2.84 times 10 to the -19 joules.
We're saying let's do this for 60 seconds.
So what we end up with for the number of photons in this laser beam of light is 2.1 times 10 to the 17 photons.
So this gives you a little bit of an idea of just how many individual photons there are in a laser beam of light.
This is a huge number of photons.
So the question is does this matter?
How about if we shoot this many photons?
Does it make any difference at all in terms of whether we can eject an electron?
No, it actually doesn't.
It is an impressive number, it is very, very large, but it doesn't make a difference.
So we see that we do not eject electrons in the case of the laser pointer, even if we have this intensity, even for 60 seconds -- it is still not related to the energy of an individual photon, so we won't see an effect.
All right.
So let's hope that we can confirm our predictions here by actually doing it, and Professor Drennen well help me out by loading up our device with electrons, and I'll explain exactly what our set up here is as she does that.
So basically what we have is this zinc plate here.
So that's what we want to load up with electrons, and then see if we can remove some.
But that's a little bit hard, we aren't all that good at seeing electrons with our eyes, so we need to think of a way to do this.
So what she's going to do is start loading up the electrons, and you see this wand here move slowly, and it takes a while to do it, start become perpendicular.
The reason for that is because all of this is connected, so we're moving electrons everywhere in the system.
And since we have two bars that are together like this, once they're both loaded up with electrons there's going to be negative charges that repel, so the electrons will want to get as far away as possible, and they're on their slow way to doing that, to getting as far away from each other as possible.
And if we do, in fact, hit it with light to get the electrons off, it will go back to the straight up in position, or if it gets knocked hard enough it does that, too.
Sometimes it's easier actually not touch it to the metal, I should have--
It's hard to see if it's moving or not
So, our technology TA is also our paper TA.
Darcy will hold up the yellow paper.
Right, there we go, now we're making a little progress
It was moving before, you just couldn't see it
So, does anyone have any questions about the set up here, does it make sense what we're looking for the bar to go back once we make some progress.
This demo works wonderfully in the winter months in Boston when we will all be full of static at all times.
We're still close enough to the summer that the air is not just filling us up with extra static electricity, so it's a little more challenging here.
We'll try to make this happen only once.
I think that's probably, if we can get one more.
So, it works, I think it's just getting too much [INAUDIBLE].
Sometimes it helps to not actually hit the metal, just put it next to the -- there we go.
I wonder if there's some UV light out of this new lighting set up in our classroom here.
That would be a little tricky.
All right.
I think this -- if this sticks.
Yeah, it's the pressure of the paper.
I think that's good enough, we'll be able to see.
If you can keep showing that, though, Darcy, we'll try different scenarios and I'll try not to put laser in your eye.
Actually you can look down as well as an added precaution.
OK, let's try it with that.
That's enough then.
So, the first thing we're going to try is with the red laser pointer, because that we are expecting not to have an effect, and that will prevent Professor Drennen from having to charge up our apparatus again.
So, Darcy will look down at this moment and we will hit this with the laser pointer, and what we see is nothing is happening at all.
OK, good.
Control one working.
So now very carefully take our UV light source -- Darcy again will divert her eyes and her skin.
Let me make sure this is actually on.
OK, so we've got UV light here, and let's see what we can see, and we lose electrons, if that's what's happening.
And it often doesn't go all the way, because actually this device gets stuck right there.
So let's charge it up again and see if we can check again.
But did you see movement?
Are you buying our story here?
This is actually very representative of when you do research in the laboratory, you will find often things do not work quite exactly as they worked 20 minutes ago when you just checked it in your office, for example.
And sometimes it's a matter of factors that you need to figure out what it is, and maybe it's that there's extra light in the room we don't know about.
It might just be -- so, we did get it back to the starting position.
Next time maybe we'll charge it up before class.
All right.
So we kind of saw what was happening here, you saw it move a little bit.
They'll keep trying to get it going, but maybe we should move on with our lives here while this is happening, and we'll click it back at the end, and if we have a nice set up at any point, I'll just stop and we'll go back and we'll look at it again.
Since, I think that's just not going to happen right now.
So let's switch, actually, back to our notes.
So, ideally what we did see was, in fact, it does have enough energy with the UV lamp, it wasn't a dramatic shift you saw because we didn't start very high and then it went to that stuck point.
But luckily we had the control of the red laser pointer where nothing moved at all.
So hopefully you're convinced that your predictions worked well and you are able to predict what's going on when you're looking at the photoelectric effect.
So, it turns out that the photoelectric effect is not the only evidence for the fact that light has these particle-like characteristics.
And one thing that Einstein put forth is he figured if well, what we're saying is that light is, in fact, a stream of particles, each one of those particles or photons must, therefore, have a momentum.
And that's really neat to think about, because photons, of course, are massless particles, they have no mass, so it's neat to think about something that has no mass, but that actually does have a momentum.
And the relationship that he put forth is that the momentum is equal to Planck's constant times nu divided by the speed of light, or it's often more useful for us to think about it in terms of wavelength.
So, since the speed of light equals lambda nu, we can say that momentum is equal to h divided by lambda.
And there was experimental evidence that came along that supported this, and this is called the Compton scattering experiment, and this was done by Arthur Compton, and basically what he did was he took x-ray light, which had some frequency, which was a very high frequency because it was x-rays, and he shot it at a stationary electron.
And what he was able to observe was that the electrons scattered and now had some momentum, and that both the frequency, and therefore, the momentum of the light that he shot in, went down once it was scattered.
So what he's showing here is, first of all, that the light has some momentum and when it hits an electron it can actually transfer some of that momentum to the electron.
So the transfer of momentum from a photon to an electron is what was being observed, and it was seen as completely separate evidence to the photoelectric effect that, yes, in fact, light is behaving in these particle-like ways.
So up to this point, before it was really established that yes, light is like a particle sometimes, there was this very strong distinction between what is light and what is matter.
And the distinction was when we're talking about light, light is a wave, and when we're talking about matter, well, matter is a particle.
And these behave completely separate, they don't overlap at all in terms of behavior, but then, of course, with the photoelectric effect with Compton's scattering, what we see is that, oh actually, sometimes photons behave as if they're particles.
So now this relationship's beginning to get a little bit fuzzy in terms of what is the difference between how we treat light and matter.
And actually, this was taken a step further by Louis de Broglie who in his PhD thesis, as part of his work as a graduate student, put forth the idea that, OK, Einstein says, and everyone agrees that, in fact, light is particle-like at times, and light, in fact, of course has a wavelength, and if it has a wavelength we're saying that it can have momentum.
And what de Broglie said is well, if it's true that light, which has a wavelength can have momentum, then it must also be true that matter, which has momentum, also has a wavelength.
And you can look at this in two different ways.
One is that he's just re-arranged an equation here and gotten both his PhD thesis and a Nobel Prize, but I think the more representative way to think about this is the real revolutionary idea that he put forth, which is that matter can actually behave as a wave.
And in terms of equations that we use, it's sometimes easier to plug in the fact, since momentum is equal to mass times velocity.
We can know the wavelength of any matter -- and he's not limiting this, for example, to electrons.
What de Broglie is saying we can know the wavelength of any matter at all, as long as we know its mass and it's velocity.
And Einstein credited de Broglie, which is a fair statement of lifting a corner of the great veil, because really there was this fundamental misunderstanding about what the difference was between matter and light, and the reality is that they can both be like-particles and they can both show characteristics of waves.
So I mentioned, however, that in terms of de Broglie's work.
This was Nobel Prize worthy, absolutely, but it was also his PhD thesis.
So, we can think about what would happen if we're on his thesis defense, we're on his thesis committee, we would need to think of some pretty mean, hard, nasty questions to be asking de Broglie about this theory -- that's what happens when you defend your thesis.
This is necessary, it's hard to find holes in a Nobel Prize worthy idea.
But let's just try maybe one of the basic questions they could ask, and they can say, all right, de Broglie, so you say that all matter, absolutely all matter has wave-like behavior.
Why is it that we're never observing this, for example, why is it the table doesn't defract as we bring it through the door?
Why don't we see the influence of the wave-like behavior on every day matter?
So it turns out that he could have picked anything to explain this, and hopefully done out the calculation, and we'll do this ourselves.
And the example we'll pick is considering, for example, a Matsuzaka fastball.
So, many of you are new to the Boston area, now I still realize, and I want to let you know it's not required that you be a Red Sox fan to be at MIT.
We do encourage it, however, and in general, I find you don't have to give up that old team, you can keep your old team, even if it's teams I won't name, just keep them to the side.
And you can join on to the Red Sox nation on top of that, and part of being a good Red Sox fan is knowing the statistics of your team.
For example, if we're talking about a pitcher, like Matsuzaka, we might want to know the speed of his average fastball.
We might want to know his ERA.
If you're really into it and you're at MIT, maybe you want to know the wavelength of these average fastballs.
So, let's go ahead and look at that.
So, if we're trying to figure out the wavelength of a Matsuzaka fastball, we need to consider the velocity first, which is 42 miles per hour.
We don't usually do our chemistry calculations in miles per hour, so let's switch that to 42 meters per second, so it's -- sorry, it's 94 miles per hour.
And we can use the de Broglie relationship that wavelength should be equal to h over mass times volume.
And we can put up here Planck's constant -- and I want to make note that instead of writing joules per second, I actually wrote out with a joule is.
A joule is a kilogram meter squared per second squared.
Occasionally, you'll find you need to cancel out units, because, of course, you're always doing unit analysis as you solve your problems, and sometimes you'll need to convert joules to kilogram meters square per second squared.
We divide that by the mass, so 0.12 kilograms, that's the mass of a regulation baseball for the major leagues, and the velocity of the baseball is 42 meters per second.
So, we can cross out our units doing our unit analysis.
The seconds cross out, the kilograms cross out, one of the meters crosses out from the top, so we're left with an answer in meters.
It's always good when we're looking for a wavelength that our answer is in a unit of length, that's a good sign already.
And what we find out is the wavelength of a Matsuzaka fastball is 1.1 times 10 to the -31 meters.
So, this is really small, this is undetectably small.
And especially when we consider it, what tends to be important is the size of wavelength in relationship to its environment.
So 1.1 times 10 to the -31 meters is not, in fact, a significant number when we're comparing it, for example, to the length of a ball, or the size of the baseball field.
So that would probably be de Broglie's answer for why, in fact, we're not observing the wavelength behavior of material on a day-to-day life.
So, that's for Matsuzaka, and even if you don't memorize all the wavelengths for all the pitchers.
I would expect, whether you're a Red Sox fan or not, you to be able to look at a list of different pitchers and their average velocity for their fastball, and tell me who has the longest or the shortest wavelength.
You should all be able to know that relationship.
So why don't we go to a clicker question here, and see if you can tell us this.
So we have 4 different pitchers we're showing here -- they all have different strengths.
It's not always how fast you throw the fastball, sometimes it's your different styles or the different ways that you decide when to throw what.
So, first we have Matsuzaka at 94 miles per hour.
So, click one if you think that he's going to have the longest wavelength.
Tim Wakefield on the DL right now throws a lot slower, because he has that tricky knuckle ball, he doesn't need to throw as fast.
Then we have Beckett who can get up 96 just on a regular old day.
And Timlin who is about 91 miles per hour, one of our relievers.
So, why don't you take ten seconds to do that.
If you can't decide, Timlin is my favorite ever, so that would be a good back up choice if you forgot the relationship between wavelength and the relationship between speed.
It looks like, in fact, people did not forget that relationship, and only 1% of you humored me.
So, let's see what the correct answer is, and it is, in fact, Wakefield, right, because there's an inverse relationship between how fast a particle is going and what its wavelength is.
So, in terms of wavelength, Wakefield has the largest wavelength, but in terms of being significant, we're still not even close.
It's still undetectably small.
Yes
Why doesn't wavelength go to infinity as it stops, like a standing [INAUDIBLE]
As it stops.
So, let's think.
I would think that it would approach inifinity, and I would need to think about it and get back to you in terms of why we don't actually hit it and see something with an infinite wavelength.
I'm sure there's some upper limit as there are to most things, like if we think of wavelengths and different types of light, there is so large that you can get, but you would be approaching that level.
All right.
So we can switch back actually to our notes here -- oh, do we have--?
OK. We're going to just try this one more time just so you can see it.
It'll still likely get stuck in that spot, but we'll just show you one more time the effects of the UV light, and actually we'll throw in an extra trick here, too.
We know that UV light gets absorbed by glass, so it shouldn't be able to go through the glass.
So first if Professor Drennen can try it through the glass, and we see nothing's happening.
Let's move the glass away.
All right.
[APPLAUSE]
All right.
Good.
So we can fully believe what our calculations were now, which is a nice thing to do.
Let's go back to considering the wavelengths of different objects.
We considered a baseball, but let's also think about now an electron.
And an electron is something where, in fact, we might be able to, if we calculate it and see how that works out, actually observe some of its wave-like properties.
So, if we do this calculation for an electron, saying it moves at 10 to the 5 meters per second, then what we end up with for a wavelength is 7 times 10 to the -9 meters.
A lot of times we talk about these kind of distances either in nanometers or in angstroms so we can say this is 70 angstroms.
So this is, first of all, even just on an absolute scale, this is way, way larger than the wavelengths we're talking about for baseball.
In addition, if we compare this to the diameter of an atom, which is on the order of somewhere between one and ten angstroms, now we're seeing that, in fact, this wavelength is significantly larger than its environment.
So certainly we would expect to see that it has an effect in terms of seeing its wave-like properties.
And this was experimentally validated, hopefully, even more clearly than our experiment here.
And at first this was done by Davidson and Germer, and they were American scientists who tried defracting electrons from a nickel crystal.
They did this in Bell Laboratories, and they found that, in fact, the electronis did defract.
And G.P.
Thompson showed a similar thing.
What he did was he defracted electrons through a very thin gold foil, and this is a picture -- oops, that is not.
OK.
It is a picture from your book here showing the defraction pattern of an electron going through that gold foil.
So, you can see that, in fact, it's confirmed that an electron can have both wavelength and particle-like behavior.
And it turns out that Davidson and Thompson shared a Nobel Prize for this discovery of seeing the wave-like behavior of electrons.
So, this is actually kind of neat to point out, because we all remember J.J. Thomson from our second lecture, and J.J. Thomson got a Nobel Prize in 1906 for showing that electrons exist in that they are particles.
And it turns out that G.P.
Thompson, well, that's his son, so we can actually think of this -- and I'm sure this wasn't the case, but I like to think of it as a little bit of child rebelling against the father.
So, the father gets a Nobel Prize for showing that an electron is a particle, and the son says, well, what can I do to top that?
I'm going to show the exact opposite.
I'm going to say that an electron's a wave no matter how much my father says differently, and I'm going to get a Nobel Prize for that, and he does.
But the nice part of the story is, it turns out they're both right.
An electron is a particle, but an electron's also a wave.
So, father and son, happy ending, they both have their Noble prizes.
So, what happens now that we, in fact, know that matter is a wave?
Well, this allows us to try to go back and explain some phenomena that over the years, mounting evidence was building that couldn't be explained.
So, for example, when people, and we'll talk about this next class, were looking at different characteristics spectra of different atoms, what they were seeing is that it appeared to be these very discreet lines that were allowed or not allowed for the different atoms to emit, but they had no way to explain this using classical physics.
And it turns out that the Schrodinger equation is an equation of motion in which you're describing a particle by describing it as a wave.
So you're basically having a wave equation for a particle, and for our purposes we're talking about a very particular particle.
What we're interested in is the electron.
So basically describing electrons by their wave-like properties.
And this is Erwin Schrodinger, and this is the equation that he put forth where we have h hat psi being equal to e psi.
So, let's explain what these are.
So this symbol here is actually what we call a wave function.
That doesn't mean a whole lot in itself, it will mean more in about two lectures from now.
But right now, what I want you to be thinking of a wave function as is just some representation of an electron.
So, it's some way of describing an electron.
Specifically, we'll talk more about this, it's talking about different orbitals, it's the spatial part of an orbital.
But before we get to that, in terms of thinking just think, OK, this is representing my particle, this is representing my electron that's what the wave function is.
This e term here is the energy, or in our case when we talk about an electron in a hydrogen atom, for example, the binding energy of that electron to the nucleus.
So, e is binding energy.
And h with the carrot or the hat here, well, that carrot or hat tell us it must be an operator, and this is called the Hamiltonian operator.
So when you operate on the wave function, what you end up with is getting the binding energy of the electron, and the wave function back out.
When we need to describe the wave function term a little bit more specifically so we can describe, for example, the position of the electron, and I just want to mention that we do have two choices if we're trying to describe this, we could use cartesian coordinates, or we could use polar coordinates where we're either talking about x y z or r theta and phi.
So, I just want to point out that when you look at wave functions, we are going to be using those spherical polar coordinates, and the reason is because a very important interaction here is the interaction between the electron and the nucleus, which we want to describe the distance of in terms of r. So, you can see, it's much easier to describe that as one term, r here, instead of using both y and z.
Another reason I wanted to point this out in terms of the polar coordinates that we're using, is I think they're actually flipped from what you're used to seeing in physics.
Sometimes different disciplines have different conventions, which can be very confusing because the whole point of what's happening now is there's so much interplay between different disciplines, but still I think this might be one remaining one where in our case theta is that distance from z, that angle there, where phi is this distance or angle from the x-axis.
So just keep it in mind that it's flipped.
It turns out we won't really using it, needing to identify it on the graph so much in chemistry.
We'll be using the solutions, so you shouldn't have a problem, but I wanted to point it out so it does not look too strange to you.
In terms of the Schrodinger equation, we now can write it in terms of our polar coordinates here.
So we have the operation on the wave function in terms of r, theta, and phi and remember this e is just our binding energy for the electron, and we get back out this wave function.
So, you might ask, this looks pretty simple up here, right, just with that h hat.
It turns out, we can write it out fully.
It's three different second derivatives in terms of the three different parameters.
It's a little bit complicated.
You won't have to solve it in this class, you can wait till you get to 18.03 to start solving these types of differential equations, and hopefully, you'll all want the pleasure of actually solving the Schrodinger equation at some point.
So, just keep taking chemistry, you'll already have had 18.03 by that point and you'll have the opportunity to do that.
What I want to point out also is that this h hat, the Hamiltonian operator written out for the simplest case we can even imagine, which is a hydrogen atom where we only have one electron that we're dealing with, and of course, one nucleus.
So you can imagine it's just going to get more and more complicated as we get to other types of atoms, and of course, molecules from there.
So, we just want to appreciate that what we'll be using in this class is, in fact, the solutions to the Schrodinger equation, and just so you can be fully thankful for not having to necessarily solve these as we jump into the solutions and just knowing that they're out there and you'll get to solve it at some point, hopefully, in your careers.
So, we'll pick up with that, with some of the solutions and starting to talk about energies on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
As you're settling in, why don't you take 10 more seconds to answer the clicker question.
This is the last question we'll see in class on the photoelectric effect, so hopefully we can have a very high success rate here to show we are all ready to move on with our lives here.
OK, good.
So, most of you did get the answer correct.
For those of you that didn't, you, of course, can ask your TA's about this in recitation, they'll always have a copy of these slides.
But just to point out the confusion can be we've actually switched what the question is here.
What the information we gave was the work function, which is what we've been giving before, but now we gave you the kinetic energy of the ejected electron, so you just need to rearrange your equation so now you're solving for the incoming energy, which would mean that you need to add those two energies together.
So, hopefully everyone that didn't get this right, can look at it again and think about asking it's just asking the question in a different kind of a way.
All right.
So, we can switch over to the class notes.
So today, we're going to start talking about the hydrogen atom.
Now that we have our Schrodinger equation for the hydrogen atom, we can talk about it very specifically in terms of binding energies and also in terms of orbitals.
And we talked about on Wednesday the conditions that allowed us to use quantum mechanics, which then enable us to have the Schrodinger equation, which we can apply, and part of that is the wave particle duality of light and matter.
So there was a good question in Wednesday's class about the de Broglie wavelength and if it can actually go to infinity.
So I just wanted to address that quickly before we move on, and actually address another thing about dealing with wavelengths of particles that sometimes comes up.
So, the question that we had in last class, was if we have actually a macroscopic particle, and the velocity let's say starts to approach zero, shouldn't we have the wavelength go to infinity, even if we have a magic board, and even if the mass is really large.
So, in most cases, you would think that as the velocity gets very tiny, the mass is still going to be large enough to cancel it out and still make it such that the wavelength is going to be pretty small, right.
Because if we think about the h, Planck's constant, here that's measured in 10 to the negative 34 joules per second.
So, we would actually need a really, really, really tiny velocity here to actually overcome the size of the mass, if we're talking about macroscopic particles, to have a wavelength that's going to be on the order.
So, let's say we're talking about the baseball, have a wavelength of the baseball that's on the order of the baseball.
So, if we kind of think about the numbers we would need, we would actually need a velocity that approached something that's about 10 to the negative 30 meters per second.
So first of all, that's pretty slow here.
It's going to be hard to measure anyway, and in fact, if we're talking about something going 10 to the negative 30, and we're going to observe it using our eyes, so you using visible light to observe something going this slow, we're actually not going to be able to do it because we're limited by the wavelength of light to see how precise we can measure where the actual position is.
So let's say we have the wavelength of light somewhere on the order of 10 to the negative 5 meters, being the wavelength of light, we're only going to be able to measure the velocity because of the uncertainty principle to a certain degree.
And it turns out it's three orders of magnitude that -- the uncertainty is three orders of magnitude bigger than the velocity that we're actually trying to observe to get to a point where we could see the wavelength, for example, for even a baseball that is moving this slow?
So that the more complete answer to the question is that no, we're never going to be able to observe that because of the uncertainty principle it's not possible to observe a velocity that's this slow for a macroscopic object.
So, hopefully that kind of clears up that question.
And, of course, when the velocity actually is zero, this equation that the de Broglie has put forth is valid for anything that has momentum, so if something does not have any velocity at all, it actually does not have momentum, so you can't apply that equation anyway.
And another thing that came up, and it came up in remembering as I was writing your problem set for this week, which will be posted sometime this afternoon, your problem set 2, is when we're talking about wavelengths of particles, and for specifically for electrons sometimes, you're asked to calculate what the energy is.
And I just want to remind everyone, so this is a separate thought here, that we often use the energy where energy is equal to h c divided by wavelength.
So if we're talking about, for example, we know the wavelength of an electron and we're trying to find the energy or vice versa, is this an equation we can use to do that?
What do you think?
No.
Hopefully you're going to say no.
And the reason is, and this will come up on the problems and a lot of students end up using this equation, which is why I want to head it off and mention it ahead of time, we can't use an equation because this equation is very specific for light.
We know it's very specific for light because in this equation is c, the speed of light.
So any time you go to use this equation, if you're trying to use it for an electron, just ask yourself first, does an electron travel at the speed of light?
And if your answer is no, your answer will be no, then you just know you can't use this equation here.
So instead you'd have to maybe if you start with wavelength, go over there, and then figure out velocity and do something more like kinetic energy equals 1/2 n b squared to get there.
So this is just a heads up for as you start your next problem set.
All right.
So jumping in to having established that, yes, particles have wave-like behavior, even though no, they're not actually photons, we can't use that equation.
But we can use equations that describe waves to describe matter, and that's what we're going to be doing today.
We're going to be looking at the solutions to the Schrodinger equation for a hydrogen atom, and specifically we'll be looking at the binding energy of the electron to the nucleus.
So we'll be looking at the solution to this part of the Schrodinger equation where we're finding e. Then we'll go on, after we've made all of our predictions for what the energy should be, we can actually confirm whether or not we're correct, and we'll do this by looking at photon emission and photon absorption for hydrogen atoms, and we'll actually do a demo with that, too, so we can confirm it ourselves, as well as matching it with the observation of others.
And if we have time, we'll move on also to talking about the other part of the solution to the Schrodinger equation, which is psi or this wave function here.
And remember I said that wave function is just a representation of the particle, particularly when we're talking about electrons -- we're familiar with the term orbitals.
psi is just a description of the orbital.
So, we'll start with an introduction to that if we got to it at the end.
So, to remind you, when we look at the Schrodinger equation here, we have two parts to it, so when we solve the Schrodinger equation, we're either finding psi, which as I said, is a wave function or an orbital.
And in addition to finding psi, we can also solve to find e or to find the energy for any given psi, and these are the binding energies of the electron to the nucleus.
And the most important thing about using the Schrodinger equation and getting out our solutions for potential orbitals and potential energies for an electron with the nucleus, is that what we find is that quantum mechanics, and quantum mechanics allowing us to get to the Schrodinger equation, allows us to correctly predict and confirm our observations for what we can actually measure are indeed the energy levels.
Here we're talking about a hydrogen atom and that's what we'll focus on today.
And it's incredibly precise and we're able to make the predictions and match them with experiment.
Also, when we're looking at the Schrodinger equation, it allows us to explain a stable hydrogen atom, which is something that classical mechanics did not allow us to do.
So here's the solution for a hydrogen atom, where we have the e term here is equal to everything written in green.
We've got a lot of constants in this solution to the hydrogen atom, and we know what most of these mean.
But remember that this whole term in green here is what is going to be equal to that binding energy between the nucleus of a hydrogen atom and the electron.
So, let's go ahead and define our variables here, they should be familiar to us.
We have the mass, first of all, m is equal to m e, so that's the electron mass.
We also have e, which is going to be the charge on the electron.
In addition to that, we have that epsilon nought value, remember that's the permittivity constant in a vacuum, and basically that is what we use as a conversion factor to get from units.
We don't want namely coulombs to units, we want that will allow us to cancel out in this equation.
And finally we have Planck's constant here, which we're all familiar with.
So, what actually happens when people work with the solution to the Schrodinger equation for a hydrogen atom is that they don't always want to deal with all these constants here, so we can actually group them together and use them as a single new constant, and this new constant is the Rydberg constant.
And the Rydberg constant is actually equal to 2 .
1 8 times 10 to the negative 18 joules.
So when we pull out all of those constants and instead use the Rydberg constant, what it allows us to do is really simplify our energy equation.
So now we have that energy is equal to the negative of the Rydberg constant divided by n squared.
So, what we have left in our equation is only one part that we haven't explained yet, and that is that n value.
And it turns out that when you solve the Schrodinger equation, you find that there are only certain allowed values of this integer n. And those allowed values range anywhere from n equals 1, you can have n equal that 2, 3, and it goes all the way up to infinity.
But the important part is that there are only certain allowed values, so for example, you can't have 1 .
5 or 2 .
3, there are only these interger numbers.
And this n here is what we call the principle quantum number.
And what we find is when we apply n and plug it in to our energy equation, is that what we see is now we don't just have one distinct answer, we don't just have one possible binding energy of the electron to the nucleus.
We're going to find that we actually have a whole bunch of possible, in fact, an infinite number of possible energy levels, and that's easier to see on this energy diagram here.
So, let's start with n equals 1, since that's, of course, the simplest case.
So, if we have n equals 1, we can plug it into our energy equation here, and find that the binding energy, the e sub n, for n equals 1, it's just going to be equal to the negative of the Rydberg constant, so we can actually graph that on an energy diagram here, and it's going to be down low at the bottom because that's going to be, in fact, the lowest or most negative energy when n equals 1.
But we saw from our equation that there's more than just one possible value for n, so we could, for example, have n equals 2, n equals 3, all the way up to n equaling infinity.
So what this tells us here is that this is not necessarily the binding energy of the electron in a hydrogen atom, it's also possible that it could, for example, have this energy, it could have this energy up here, it could have some energy way up here.
So we have this infinite number of possible binding energies.
But the really important point here is that they're quantized.
So it's not a continuum of energy that we can have, it's only these punctuated points of energy that are possible.
So as I tried to say on the board, we can have n equals 1, but since we can't have n equals 1/2, we actually can't have a binding energy that's anywhere in between these levels that are indicated here.
And that's a really important point for something that comes out of solving the Schrodinger equation is this quantization of energy levels.
And thanks to our equation simplified here, it's very easy for us to figure out what actually the allowed energy levels are.
So for n equals 2, what would the binding energy be?
Someone shout it out.
Yup.
So, I think the compilation of the voices that I heard was negative r h over 2 squared.
We can do the same thing for 3, negative r h over 3 squared is going to be our binding energy.
For 4, we can go all the way up to infinity, and actually when we get to the point where it's infinity, what we find is the binding energy at that point is going to be zero.
And when we get to infinity, what that means is that we now have a free electron, so now the electron has totally separated from the atom.
And that makes sense because we're at the point where there's no binding energy keeping it stable.
You'll also know that all of these binding energies here are negative, so the negative sign indicates that it's low.
It's a more negative energy, it's a lower energy state.
So whenever we're thinking about energy states, it's always more stable to be more low in an energy well, so that's why it makes sense that it's favorable, in fact, to have an electron interacting with the nucleus that stabilizes and lowers the energy of that electron by doing so.
So, we actually term this n equals 1 state gets a special name, which we call the ground state, and it's called the ground state because it is, in fact, the lowest to the ground that we can get.
It's the most negative and most stable energy level that we have.
And when we think about kind of in a more out practical sense what we mean by all of these binding energies, another way that we can put it is to give it some physical significance, and the physical significance of binding energies is that they're equal to the negative of the ionization energies.
So, for example, in a hydrogen atom, if you take the binding energy, the negative of that is going to be how much energy you have to put in to ionize the hydrogen atom.
So, if, for example, we were looking at a hydrogen atom in the case where we have the n equals 1 state, so the electron is in that ground state, the ionization energy, it makes sense, is going to be the difference between the ground state and the energy it takes to be a free electron.
When we graph that on our chart here, it becomes clear that yes, in fact, the ionization energy is just the negative of the binding energy, so we can just look over here and figure out what our ionization energy is.
So when we're talking about the ground state of a hydrogen atom, our ionization energy is just the negative of the Rydberg constant, so that easy, it's 2 .
1 8 times 10 to the negative 18 joules.
So, that should make a lot of sense intuitively, because it makes sense that if we need to ionize an atom, we need to put energy into the atom in order to eject that electron, and that energy we need to put in better be the difference between where we are now and where we have to be to be a free electron.
So in most cases when we talk about ionization energy, if we don't say anything specific to the state we're talking about, you should always assume that we are, in fact, talking about the ground state.
So, oftentimes you'll just be asked about ionization energy.
If it doesn't say anything else we do mean n equals 1.
But, in fact, we can also talk about the ionization energy of different states of the hydrogen atom or of any atom.
So, for example, we could talk about the n equals 2 state, so that's this state here, and it's also what we could call the first excited state.
So we have the ground state, and if we excite an electron into the next closest state, we're at the first excited state, or the n equals 2 state.
So, we can now calculate the ionization energy here.
it's an easy calculation -- we're just taking the negative of the binding energy, again that makes sense, because it's this difference in energy here.
So what we get is that the binding energy, when it's negative, the ionization energy is 5 .
4 5 times 10 to the negative 19 joules.
So we should be able to think about these binding energies and figure out the ionization energy for any state that were asked about.
So if we can switch over to a clicker question here and we'll let you do that.
And what we're asking you to do is now tell us what the ionization energy is of a hydrogen atom that is in its third excited state.
All right.
Let's take 10 more seconds on that.
OK.
Interesting.
Usually the majority is correct, but actually what you did was illustrate a point that I really wanted to stress and there's no better way to stress it then to get it incorrect, especially when it doesn't count, it doesn't hurt so bad.
So, if you want to switch back over to the notes, we'll explain why, in fact, the correct answer is 4.
So the key word here is that we asked you to identify the third excited state.
So, what white is n equal to for the third excited state?
4 OK.
So that explains probably most of the confusion here and you just want to be careful when you're reading the problems that that's what you read correctly.
I think everyone would now get the clicker question correct.
So, the third excited state, is n equal to 4, because n equals 2 is first excited, 3 is second excited, 4 is third excited state.
So now we can just take the negative of that binding energy here, and I've just rounded up here or 1 .
4 times 10 to the negative 19 joules.
So, I noticed that a few, a very, very small proportion of you, did type in selections that were negative ionization energies.
And I'll just say it right now you can absolutely never have a negative ionization energy, so that's good to remember as well.
And intuitively, it should make sense, right, because ionization energy is the amount of energy you need to put in to eject an electron from an atom.
So you don't want to put in a negative energy, that's not going to help you out, you need to put in positive energy to get an electron out of the system.
So that's why you'll find binding energies are always negative, and ionization energies are always going to be positive, or you could look at the equation and see it from there as well.
All right.
So, using the equation we'd initially discussed, the negative r sub h over n squared, we could figure out all of the different ionization energies and binding energies for a hydrogen atom, and it turns out if we change the equation only slightly to add a negative z squared in there, so, negative z squared times the Rydberg constant over n squared, now let's us calculate energy levels for absolutely any atom as long as this one important stipulation, it only has 1 electron in it.
So basically we're dealing with hydrogen atoms and then we're going to be dealing with ions.
So, for example, a helium plus 1 ion has 1 electron at z equal 2.
A lithium 2 plus ion has 1 electron, it has z equals 3, so if we were to plug in, we would just do z squared up here, or 3 squared.
Terbium 64 plus, another 1 electron atom.
What is z for that?
Yup.
65.
So again, terbium 64 plus, not an ion we probably will run into, but if we did, we could, in fact, calculate all of the energy levels for it using this equation here.
And the difference between the equation, the reason that that z squared comes in there is because if you go back to your notes from Wednesday, and you look at the long written out form of the Schrodinger equation for a hydrogen atom, or any 1 electron atom, you see the last term there is a coulomb potential energy between the electron and the nucleus.
So, of course, when we have a charge on the nucleus equal to 1, as we do in a hydrogen atom, the z is equal to 1, so it drops out there, but normally we would have to include the full charge on the nucleus, which is equal to z or the atomic number times the electron.
So even if we strip an atom of all of its electrons, we still have that same amount of positive charge in the nucleus.
So, this allows us to look at a bunch of different atoms, of course, limited to the fact that it has to be a 1 electron atom.
So, now that we can calculate the binding energies, we can think about is this, in fact, what matches up with what's been observed, or, in fact, could we predict what we will observe in different kinds of situations now that we know how to use the binding energy, and hopefully we can and we will.
So one thing we could do is we could look at the different wavelengths of light that are emitted by hydrogen atoms that are excited to a higher state.
So what we'll do in a few minutes here is try this with hydrogen.
So we'll take h 2 and we'll run -- or actually we'll have h 2 filled in an evacuated glass tube.
When we increase the potential between the 2 electrodes that we have in the tube -- we actually split the h 2 into the individual hydrogen atoms, and not only do that, but also excite the atoms.
So when you just run across an atom in the street, you can assume it's going to be in its most stable ground state, that's where the electron would be, but when we add energy to the system, we can actually excite it up into all different sorts of higher states -- n equals 6, n equals 10, any of those higher states.
But that only happens momentarily, because, of course, if you have an energy in a higher energy level, it's going to want to drop back down to that lower or more stable level.
And when it does that it's going to give off some energy equal to the difference between those two levels.
And that will be associated with a wavelength if it releases the energy in terms of a photon.
So that's what we'll look at in a few minutes.
There's some important things to point out about what is happening here.
Just to visualize exactly what we're saying, what we're saying is when we have an energy in a higher energy level, so let's say energy level, this initial level high up here, and it drops down to a lower final level, what we find is that the photon that is going to be emitted is going to be emitted with the exact energy, and the important term here is the exact.
That is the difference between these two energy states.
That makes sense because we're losing energy, we're going to a level lower level, so we can give off that extra in the form of light.
And we can actually write the equation for what we would expect the energy for the light to be.
So this delta energy here is very simply the energy of the initial state minus the energy of the final state.
This is a little bit generic, we're not actually specifying the states here, but we could, we know we can calculate the energy from any of the states.
So, for example, let's say we excited the hydrogen atom such that the electron was starting in the n equals 6 state, so that's our n initial.
And we drop down to the n equals 2 state, or the first excited state.
Then we would be able to change our equation to make it a little bit more specific and say that delta energy here is equal to energy of n equals 6, minus the energy of the n equals 2 state.
When we talk about that frequency of light that's going to be emitted, it's not too commonly that we'll actually talk about it in terms of energy.
A lot of times we talk about light in terms of its frequency or it's wavelength, but that's OK, because we know how to convert from energy to frequency, so we can do that here as well, where our frequency is just our energy divided by Planck's constant, and since here we're talking about a delta energy, we're going to talk about the frequency as being equal to delta energy over h. Or we can say the frequency is going to be equal to the energy initial minus the energy final all over Planck's constant.
So this means that we can go directly from the energy between two levels to the frequency of the photon that's emitted when you go between those levels.
What we can also do is say something about the wavelength as well because we know the relationship between energy and frequency and wavelength.
So, in the first case here, let's say we go from a high level to a low level, let's say we go from five to one.
If we have a large energy difference here, are we going to have a high or low frequency?
Good.
A high frequency.
If we have a high frequency, what about the wavelength, long or short?
All right.
Good.
So we should always be able to keep these relationships in mind.
So, similarly in a case where instead we have a small energy difference, we're going to have a low frequency, which means that we're going to have a long wavelength here.
So now we can go ahead and try observing some of this ourselves.
So what we're actually going to do is this experiment that I explained here where we're going to excite hydrogen atoms such that they're electrons go into these higher energy levels, and then we're going to see if we can actually see individual wavelengths that come out of that that correspond with the energy difference.
So, our detection devices are a little bit limited here today, we're actually only going to be using our eyes, so that means that we need to stick with the visible range of the electromagnetic spectrum.
Actually that simplifies things, because that really cuts down on the number of wavelengths that we're going to be trying to observe here.
So it turns out that in the visible range, when you figure out the differences between energy levels, in hydrogen atoms, there's only 4 wavelengths that fall in the visible range of the spectrum.
So hopefully, when we turn out the lights, we're going to turn on this lamp here, which has hydrogen in it and we're going to excite that hydrogen.
You'll see light coming out, but it's, of course, going to be bulk light -- you're not going to be able to tell the individual wavelengths.
So what our TAs, actually if they can come down now, are going to pass out to you is these either defraction goggles, or just a little plate, and you're going to be splitting that light into its individual wavelengths.
And the glasses, there aren't enough to go around for all of you, so that's why there's plates.
And though glasses do look way cooler, the plates work a little better, so either you or your neighbor should try to have one of the plates in case one of you can't see all the lines.
So our TAs will pass these around for us.
And I also want to point out, it's guaranteed pretty much you'll be able to see these three here in the visible range -- you may or may not be able to see, sometimes it's hard to see that one that's getting near the UV end of our visible spectrum.
So we won't worry if we can't see that.
I'll take one actually.
Can you raise your hand if you if you still need one?
All right, so TA's walk carefully now, I'm going to shut the lights down here.
All right, so we do still have some little bits of ambient light, so you might see a slight amount of the full continuum.
But if you look through your plate, and actually especially if you kind of look off to the side, hopefully you'll be able to see the individual lines of the spectrum.
Is everyone seeing that?
Yeah, pretty much.
OK. Can anyone not see it?
Does anyone need -- actually I can't even tell if you raise your hand.
So ask your neighbor if you can't see it and get one of the plates if you're having trouble seeing with the glasses.
So this should match up with the spectrum that we saw.
And actually keep those glasses with you.
We'll turn the light back on for a second here.
And hydrogen atom is what we're learning about, so that's the most relevant here.
But just to show you that each atom does have its own set of spectral lines, just for fun we'll look at neon also so you can have a comparison point.
All right.
So, this is probably a familiar color having seen many neon lights around everywhere.
So you see with the neon is there's just a lot more lines in that orange part of the spectrum then compared to the hydrogen, and that's really what gives you that neon color in the neon signs.
That's the true color of a neon being excited, sometimes neon signs are painted with other compounds.
All right, does everyone have their fill of seeing the neon lines?
No
All right, let's take two more seconds to look at neon then.
All right.
So our special effects portion of the class is over.
And what you see when we see it with our eye, which is all the wavelengths, of course, mixed together, is whichever those wavelengths is most intense.
So, when we looked at the individual neon lines, it was the orange colors that was most intense, which is why we were seeing kind of a general orange glow with the neon, which is different from what we see with the hydrogen.
All right, so we can, in fact, observe individual lines.
There's nothing more exciting to see with your glasses on, while you look nice.
You can take those off if you wish to, or you can try to just be splitting the light in the room until the TAs grab your glasses, either is fine.
It turns out that we are far from the first people, although it felt exciting, we did not discover this for the first time here today.
In fact, J.J. Balmer, who was a school teacher in the 1800s, was the first to describe these lines that could be seen from hydrogen.
And he saw the same lines that we saw here today, and although he could not explain, even start to explain why you saw only these specific lines and not a whole continuum of the light.
Right, we already have an idea because we just talked about energy levels, we know there's only certain allowed energy levels, but at the time there's no reason J.J. Balmer should have known this, and in fact he didn't, but he still came up with a quantitative way to describe what was going on.
He came up with this equation here where what he found was that he could explain the wavelengths of these different lines by multiplying 1 over 4 minus 1 over some integer n, and multiplying it by this number, 3 .
29 times 10 to the 15 Hertz, and he found that this was true where n was some integer value -- 3, 4, 5, or 6.
So he could explain it quantitatively in terms of putting an equation with it, but he couldn't explain what was actually going on.
But we, having learned about energy levels, having had the Schrodinger equation solved for us to understand what's going on, can, in fact, explain what happened when we saw these different colors.
So, what we know is happening is that were having transitions from some excited states to a more relaxed lower, more stable state in the hydrogen atom.
And it turns out what we can detect visibly with our eyes is in the visible range, and that means that the final state is n equals 2.
Because you see how n equals 1 is so much further away, and actually that's not to scale, it's actually much, much further down in the energy well, such that the energy of the light is so great that it's going to be in ultraviolet very high energy, high frequency range.
So we can't actually see any of that, it's too high energy for us to see.
So everything we see is going to be where we have the final energy state being n equals 2.
So if we think about, for example, this red line here, which energy state or which principle quantum number do you think that our electron started in?
Three
Good.
So, it's going to be in 3, because that's the shortest energy difference we can have, and the red is the longest wavelength we can see -- those 2 are inversely related, so it must be n equals 3.
What about the kind of cyan-ish, blue-green one?
Yup.
so n equals 4 for that one.
Similarly we can go on, match up the others.
So n equals 5 for the bluish-purple and the violet is n equals 6.
And again, that matches up, because the violet, or getting really close to the UV range here has the longest energy, so the highest frequency, and that's going to be the shortest wavelength and we can see here it is, in fact, the shortest wavelength that we can actually see.
So, we can see if we can come up with the same equation that J.J. Balmer came up with by actually starting with what we know and working our way that way instead of coming up from the other direction, which he did, which was just to explain what he saw.
So, if we start instead with talking about the energy levels, we can relate these to frequency, because we already said that frequency is related to, or it's equal to the initial energy level here minus the final energy level there over Planck's constant to get us to frequency.
And we also have the equation that comes out of Schrodinger equation that tell us exactly what that binding energy is, that binding energy is just equal to the negative of the Rydberg constant over n squared.
So that means that our frequency is going to equal, if we plug in e n into the initial and final energy here, 1 over Planck's constant times negative r h over n initial squared, minus negative r h over n final squared.
So we have an equation that should relate how we can actually calculate the frequency to what J.J. Balmer observed.
We can simplify this equation by pulling up the r h and getting rid of some of these negatives here by saying the frequency is going to be equal to the Rydberg constant divided by Planck's constant all times 1 over n final squared -- this is just to switch the signs around and get rid of some negatives -- minus 1 over n initial squared.
We can plug this in further when we're talking about the visible part of the light spectrum, because we know that for n final equals 2, then that would mean we plug in 2 squared here, so what we get is 1 over 4.
So this is our final equation, and this is actually called the Balmer series, which was named after Balmer, and this tells us the frequency of any of the lights where we start with an electron in some higher energy level and we drop down to an n final that's equal to 2.
So it's a more specific version of the equation where we have the n final equal to 2.
And it turns out that actually we find that this matches up perfectly with Balmer's equation, because the value of r h, the Rydberg constant divided by the Planck's constant is actually -- it's also another constant, so we can write it as this kind of strange looking cursive r here.
Unfortunately, this is also call the Rydberg constant, so it's a little bit confusing.
But really it means the Rydberg constant divided by h, and that's equal to 3 .
29 times 10 to the 15 per second.
So if you remember what the equation Balmer found was this number multiplied by this here.
So, we found the exact same equation, but just now starting from understanding the difference between energy levels.
So, sometimes you'll find the Rydberg constant in different forms, but just make sure you pay attention to units because then you won't mess them up, because this is in inverse seconds here, the other Rydberg constant is in joules, so you'll be able to use what you need depending on how you're using that constant.
So in talking about the hydrogen atom, they actually have different names for different series, which means in terms of different n values that we end in.
So we talked about what we could see with visible light, we said that's actually the Balmer series.
So anything that goes from a higher energy level to 2 is going to be falling within the Balmer series, which is in the visible range of the spectrum.
We can think about the Lyman series, which is where n equals 1.
We know that that's going to be a higher energy difference, so that means that we're going to be in the UV range.
We can also go in the opposite direction.
So, for example, when n equals 3, that's called the Paschen series, and these are named after basically the people that first discovered these different lines and characterized them, and this is in the near IR range.
And again, the n equals 4 is the Brackett series, and that's an IR range.
I think there's names for even a few more levels up.
You don't need to know those, but just because it's a special case with the hydrogen atom, they do tend to be named -- the most important, of course, tends to be the Balmer series because that's what we can actually see being emitted from the hydrogen atom.
So, now we should be able to relate what we know about different frequencies and different wavelengths, so Darcy can you switch us over to a clicker question here?
And we can also talk about the difference between what's happening when we have emission, and we're going to switch over to absorption.
So, we just talked about emission, so before we head into absorption, if you can answer this clicker question in terms of what do you think absorption means having just discussed hydrogen emission here?
So we have four choices in terms of initial and final energy levels, and also what it means in terms of the electron -- whether it's gaining energy or whether it's going to be emitting energy?
So, why do you take 10 seconds on that, we'll make it a quick one.
All right, great.
So already just from knowing the emission part, we can figure out what absorption probably means.
Absorption is just the opposite of emission, so instead of starting at a high energy level and dropping down, when we absorb light we start low and we absorb energy to bring ourselves up to an n final that's higher.
And instead of having the electron giving off energy as a photon, instead now the electron is going to take in energy from light and move up to that higher level.
So now we're going to be talking briefly about photon absorption here.
So again, this is just stating the same thing, and it could take in a long wavelength light, which would give it just a little bit of energy, maybe just enough to head up one energy level or two, or we could take in a high energy photon, and that means that the electron is going to get to move up to an even higher energy level.
And again, we can talk about the same relationship here, so it's a very similar equation to the Rydberg equation that we saw earlier, except now what you see is the n initial and the n final are swapped places.
So instead now we have r h over Planck's constant times 1 over n initial squared minus n final squared.
And what you want to keep in mind is that whatever you're dealing with, whether it's absorption or emission, the frequency of the light is always going to be a positive number, so you always want to make sure what's inside these brackets here turns out to be positive.
So that's just a little bit of a check for yourself, and it should make sense because what you're doing is you're calculating the difference between energy levels, so you just need to flip around which you put first to end up with a positive number here, and that's a little bit of a check that you can do what yourself.
So let's do a sample calculation now using this Rydberg formula, and we'll switch back to emission, and the reason that we'll do that is because it would be nice to actually approve what we just saw here and calculate the frequency of one of our lines in the wavelength of one of the lines we saw.
So what we'll do is this problem here, which is let's calculate out what the wavelength of radiation would be emitted from a hydrogen atom if we start at the n equals 3 level and we go down to the n equals 2 level.
So what we need to do here is use the Rydberg formula, and actually you'll be given the Rydberg formulas in both forms, both or absorption and emission on the exams.
if you don't want to use that, you can also derive it as we did every time, it should intuitively make sense how we got there.
But the exams are pretty short, so we don't want you doing that every time, so we'll save the 2 minutes and give you the equations directly, but it's still important to know how to use them.
So, we can get from these energy differences to frequency by frequency is equal to r sub h over Planck's constant times 1 over n final squared minus 1 over n initial squared.
So let's actually just simplify this to the other version of the Rydberg constant, since we can use that here.
So kind of that strange cursive r, and our n final is 2, so 1 over 2 squared minus n initial, so 1 over 3 squared.
So, our frequency of light is going to be equal to r times 5 over 36.
But when we were actually looking at our different wavelengths, what we associate mostly with color is the wave length of the light and not the frequency of the light, so let's look at wavelength instead.
We know that wavelength is equal to c over nu.
We can plug in what we have for nu, so we have 36 c divided by 5, and that cursivey Rydberg constant, and that gives us 36 times the speed of light, 2 .
998 times 10 the 8 meters per second, all over 5 times 3 .
29 times 10 to the 15 per second.
So, what we end up getting when we do this calculation is the wavelength of light, which is equal to 6 .
57 times 10 to the negative 7 meters, or if we convert that to nanometers, we have 657 nanometers.
So does anyone remember what range of light 657 nanometers falls in?
What color?
Red
Yeah, it's in the red range.
So that's promising.
We did, in fact, see red in our spectrum, and it turns out that that's exactly the wavelength that we see is that we're at 657 nanometers.
So it turns out that we can, in fact, use the energy levels to predict, and we could if we wanted to do them for all of the different wavelengths of light that we observed, and also all the different wavelengths of light that can be detected, even if we can't observe them.
All right, so that's what we're going to cover in terms of the energy portion of the Schrodinger equation.
I mentioned that we can also solve for psi here, which is the wave function, and we're running a little short on time, so we'll start on Monday with solving for the wave function.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
As everyone finishes getting settled in, why don't you take 10 more seconds on the clicker question here, and let's see how you did on that this, this is very similar to the clicker question that we had on Friday.
OK, so let's get started here.
It looks like we are doing a lot better.
We now have 77% getting the correct answer, we only had about 30-something percent on Friday for a very similar question.
So if you're not in this 77%, let's quickly go over why, in fact, this is the correct answer, 0 .
9 times 10 to the negative 18 joules.
So I'm using the same kind of tricky language that we'd used before, not to trick you, but so that you're not tricked in the future.
So if we're talking about the fourth excited state, and we talk instead about principle quantum numbers, what principle quantum number corresponds to the fourth excited state of a hydrogen atom
Five
Five.
OK.
So, hopefully that cleared up for some of you why you got the wrong answer.
So we know that we're in the n equals 5 state, so we can find what the binding energy is here.
The ionization energy, of course, is just the negative of the binding energy.
We know that binding energy is always negative, we know that ionization energy is always positive.
So hopefully, putting all those things together, if you looked at this question again we'd get 100% on it, that our only option here is 0 .
9, and that it's not the negative, it's the positive version, because we're talking about how much energy we have to put into the system in order to eject an electron.
All right.
And today we're going to mostly be talking about wave functions of electrons, but before we get to that, I wanted to review one last thing that's back on to Friday's topic, which was when we were solving the Schrodinger equation, or in fact, using the solution to the Schrodinger equation for the energy, the binding energy between an electron and a nucleus.
And when we talked about that, what we found was that we could actually validate our predicted binding energies by looking at the emission spectra of the hydrogen atom, which is what we did as the demo, or we could think about the absorption spectra as well.
And what we predict as an energy difference between two levels, we know should correspond to the energy of light that's either emitted, if we're giving off a photon, or that's absorbed if we're going to take on a photon and jump from a lower to a higher energy level.
So we came up with two formulas, which are similar to the two that I'm showing here.
The formula tells us the frequency of the light that's emitted or absorbed based on the energy difference between the two levels that we're going between, that the electron is transitioning between.
You'll notice that there's a little bit of a difference in these equations here from the ones from the other day, which is that you have this z squared value in there.
So these are both called Rydberg formulas for figuring out the frequency of light emitted or absorbed, and before we were looking at the Rydberg formula specifically for the hydrogen atom, and now that we have this z squared term in the formula here, we're now talking about absolutely any one electron atom.
And it should make sense where we got this from, because we know that the binding energy, if we're talking about a hydrogen atom, what is the binding energy equal to?
Negative Rydberg over what?
Yes.
So, it's negative Rydberg constant over n squared.
But if we're talking more generally about any one electron atom, now we have a more general equation for the binding energy, which has this z squared term out in front of it, right, so it's negative z squared times the Rydberg constant all over n squared.
So, essentially when we're talking about these equations up here, all we're doing is talking about the regular Rydberg formulas, but instead we could go back and re-derive the equation for any one electron atom, which would just mean that we put that z squared term in the front.
So when you solve certain types of problems, such as problems later on in the second half of your p-set, if you need to talk about the frequency of light emitted or absorbed for a one electron atom, such as lithium plus 2, for example, then you would need to plug in z, and remember the z value for lithium would just be 3.
The z value for hydrogen, of course, is 1, and that's why this term falls out of that equation when we're talking specifically about the hydrogen atom.
So, just to finish our review of what we talked about on Friday, when we're thinking about transitions between two different states, and we're talking about a situation where the final state, the n final, is greater than n initial, in this case, are we talking about absorption or are we talking about emission?
Hearing a little bit of a mix here.
In fact, we're talking about absorption when n final is greater than n initial.
We start at this lower energy state and go up -- that means we need to absorb a photon, we have to take in energy.
Specifically, we have to take in this exact amount of energy in order to bump the electron up to the higher energy level.
So that means that when instead we start high and go low, we're dealing with emission where we have excess energy that the electron's giving off, and that energy is going to be equal the energy of the photon that is released and, of course, through our equations we know how to get from energy to frequency or to wavelength of the photon that we're talking about.
All right.
So that's all I'm going to say today in terms of solving the energy part of the Schrodinger equation, so what we're really going to focus on is the other part of the Schrodinger equation today, which is solving for psi.
So we're going to for psi, and before that, we're going to figure out that instead of just that one quantum number n, we're going to have a few other quantum numbers that fall out of solving the Schrodinger equation for what psi is.
We're also going to talk more about what psi actually means.
When we first introduced the Schrodinger equation, what I told you was think of psi as being some representation of what an electron is.
We'll get more specific here, more specific even than just saying you can think of it as an orbital.
We'll really think about what psi means.
And in doing that, we'll also talk about the shapes of h atom wave functions, specifically the shapes of orbitals, and then something called radial probability distribution, which will make sense when we get to it.
But, as I said before that, we have some more quantum numbers to take care of, because it turns out that when you solve the Schrodinger equation for psi, these other quantum numbers have to be the defined.
When we talked about binding energy, we just had one quantum number that came out of it.
And that quantum number was n, which was our principle quantum number, and we know that n could be equal to any integer value, so, 1, 2, 3, all the way up to infinity.
And this quantization that comes out of having n is what gives us the quantization of different energy levels.
That's why we can't have a continuum of energy, we actually have those quantized points.
So, it turns out that n is not the only quantum number needed to describe a wave function, however.
There's two more that you can see come out of it.
And the first is l, and l is our angular momentum quantum number, and it's called that because it actually dictates the angular momentum that our electron has in our atom.
And when we talk about l it is a quantum number, so because it's a quantum number, we know that it can only have discreet values, it can't just be any value we want, it's very specific values.
And unlike n, l can start all the way down at 0, and it increases by integer value, so we go 1, 2, 3, and all the way up.
But also unlike n, l cannot have just any value, we can't go into infinity.
L is limited such that the highest value of l is n minus 1.
We can't get any higher than that.
So, it would be a good question to ask why are we limited -- clearly there's this relationship between l and n, and we can't get any higher than n equals one.
We can actually think about why that is, and the reason is because l is our angular momentum.
It describes the angular momentum of the electron.
So another way to think about that is just the rotational kinetic energy of our electron.
And we know that n describes the total energy, that total binding energy of the electron, so the total energy is going to be equal to potential energy plus kinetic energy.
So if we say that l is just talking about our kinetic energy part, our rotational kinetic energy, and we know that electrons have potential energy, then it makes sense that l, in fact, can never go higher than n. And, in fact, it can't even reach n, because then we would have no potential energy at all in our electron, which is not correct.
So, that's the second quantum number.
And the third one is called m, it's also m sub l. This is what we call the magnetic quantum number, and we won't deal with the fact of its being the magnetic quantum number here -- that kind of tells us the shape of the orbital or the way that the electron will behave in a magnetic field, but what's more relevant to thinking about the limits of this number is that it's also the z component of the angular momentum.
So since it's a component of the angular momentum, that means that it's never going to be able to go higher than l is, so it makes sense that, for example, it could start at 0 and then go all the way up to l. But since it is a component it can have a direction, too, so can go up between negative l and positive l. So the allowed values for m sub l are going to be negative l, all the way up to 0, and then up to positive l. So, if we think of just an example, we could say that 4 l equals 2, what would be our lowest value of m sub l?
Yup.
So m sub l could equal negative 2, negative 1, 0, 1 or 2.
So we could have five different values of m sub l. So, those are our three quantum numbers.
So if, in fact, we want to describe a wave function, we know that we need to describe it in terms of all three quantum numbers, and also as a function of our three positional factors, which are r, the radius, plus the two angles, theta and phi.
So, we have now a complete description of a wave function that we can talk about.
So, we can think about what is it that we would call the ground state wave function.
We knew from Friday, when we talked about energy, that ground state was that n equals 1 value, that was the lowest energy, that was the most stable place for the electron to be.
But now we need to talk about l and m as well.
So now when we talk about a ground state in terms of wave function, we need to talk about the wave function of 1, 0, 0, and again, as a function of r, theta and phi.
So this is our complete description of the ground state wave function.
So, a lot of you talked about different types of orbitals in high school, I'm sure, or in previous courses, and it might be less common that you actually talked about a wave function that was labeled like this.
We're used to labelling orbitals as an s, or a p, or a d, for example, but it turns out that these correlate to those letters that we're more used to seeing.
Does anyone know what the 1, 0, 0 orbital is also called?
Yeah.
And specfically it's the 1 s, so not just the s, but the 1 s orbital.
So, using the terminology of chemists, which is a good thing to do, because in this course we are all chemists, we want to make sure that we're not using just the physical description of the numbers, but that we can correlate it to what we understand as orbitals, and instead of 1, 0, 0, we call this the 1 s orbital.
The reason that we do this is because this is another way to completely describe it.
The n designates the shell, so that's what this number is here, we're in the first shell.
The l is what we call the sub shell.
And instead of having a 0 there, what we have here is an s. So, if we look at what the other sub shells are called, essentially we're just converting the number to a letter.
L equals 0 is s, what is l equals 1?
Um-hmm, it's the p. What about 2?
d, and 3?
Yup, so 3 is f. So these names, they don't really make any sense if we're looking at them why they're called past s p and f, and it turns out that it comes from spectroscopy terms that are pre-quantum mechanics where, for example, this is called the sharp line, I think the principle, the diffuse, and the fundamental.
It doesn't even make sense now, they're not used in spectroscopy anymore, but this is where the names originally came from and they did stick.
So, we being chemists, we'll call that 1 s instead of 1, 0.
In addition to having another name to denote l, we also have another name for the m designation here.
So, for example, when l is equal to 0, we're going to find that we have to call -- we have to specify what m is as well.
All right.
So, when we have, for example, l equal to 1, what kind of orbital is this?
The p orbital.
And for example, we could also in this case, have m is equal to 0.
If m is equal to 0, in this case we would call it the p z orbital, so we would have the subscript z here.
Similarly, if m is equal to either plus 1 or minus 1, we would in turn call it the p y orbital, or the p x orbital.
So you should know that any time m is equal to zero when we are talking about p orbitals, that it's the p z.
The p y and the p x are actually a bit more complicated, they're linear combinations of the m plus 1, and the m minus 1 orbital, where 1 is the positive linear combination, and 1 is the negative linear combination.
You're not responsible for that, you're not responsible for correlating plus 1 to y, minus 1 to x.
Just know that you have plus or minus 1, for our class, you can call it either x or y, either is fine, because it's a little bit more complicated than just the 1:1 translation between, for example, m equals 0 and having a p z orbital.
All right.
So let's look at some of these wave functions and make sure that we know how to name all of them in terms of orbitals and not just in terms of their numbers.
Once we can do that we can go on and say okay, what actually is a wave function, but first we need to know how to describe which ones were talking about.
So we saw that our lowest, our ground state wave function is 1, 0, 0.
We can call that psi 1, 0, 0 is how we write it as a wave function.
We said that's the 1 s orbital.
We also know how to figure out the energy of this orbital, and we know how to figure out the energy using this formula here, which was the binding energy, which is negative r h, and instead of n, we can plug it in because n equals 1, so over 1 squared, and the actual energy is here.
So, our next level up that we can go is going to be the n equals 2 energy level, but we also have an l and an m value, so our lowest l is going to be a 0 there.
So we'll call that psi 2, 0, 0 wave function.
What will we call that in terms of orbitals?
Yup, so that's the 2 s orbital.
So something I actually wanted to point out that I forgot to here is you'll notice that there's no subscript to the s. We said we have a subscript to the p, for example, that describes what m is equal to.
The reason that we have no subscript to the s, is because the only possibility for m when you have an s orbital is that m has to be equal to 0.
So we just assume it, you don't actually have to write it because there is, in fact, only one possibility.
We can also figure out the energy of this orbital here, and the energy is equal to the Rydberg constant.
The negative of the Rydberg constant now divided by 2 squared.
So we can go on and do this for any orbital or any state function that we would like to.
So, for example, if we talk about the 2, 1, 1 state label, that's just psi 2, 1, 1.
What, in this case, would be our orbital?
2 p what?
OK, good, I heard mixed answers, which is correct.
So you can either write 2 p x or 2 p y, whichever one you want is fine.
And again, you'll notice that our energy is absolutely the same for an electron in that 2 p x orbital and in the 2 s orbital.
So that's true for a hydrogen atom, it doesn't matter if you're in a p or an s orbital, their energies are the same.
Then we can also talk about the 2, 1, 0 state function, which would be psi 2, 1, 0.
What is this orbital?
Yup.
And there's only one correct answer here, which is to 2 p z.
Is the energy going to be the same or different as up here?
It's going to be the same energy.
Again, the reason for that is because the energy only depends on the n value here, it doesn't depend on l or on m. So finally, if we talk about our last example of when n is going to be equal 2, we can have 2, 1 for l and then minus 1 for m. We can re-write this as psi 2 1 negative 1.
And then our orbital is going to be just the opposite of whatever we said it was up here.
So if you said 2 p x the first time, say 2 p y this time.
And again, our energy is going to be the same where we again only depend on the n value.
All right.
So hopefully we're pretty comfortable naming any type of wave function using the chemist terminology.
Let's switch to a clicker question and just confirm that that is, in fact, true.
So what's the corresponding orbital if we talk about this state, 5, 1, 0?
And you can go ahead and give 10 seconds on that.
OK. All right, 77%.
So, that's OK, you don't have to memorize things as I speak, you just need to go back and look at this and make sure you understand how to name it and that you'll be able to, for example, by next class, get a similar clicker question correct, and good job to the 77% that did get it.
So I think we're safe to move on here.
And I just want to point out that now we have these three quantum numbers.
The reason there are three quantum numbers is we're describing an orbital in three dimensions, so it makes sense that we would need to describe in terms of three different quantum numbers.
And the complete description, as I said, is from n l and m. And when you talk about n for an orbital, it's talking about the shell -- that shell is kind of what you picture when you think of a classical picture of an atom where you have 1 energy level, the next one is further out, the next one's further away.
That's kind of your shell that we're discussing.
L is the sub shell here, and then we have m, which is finally the complete description of the orbital.
And what you can see is that for any n that has an l equals 0, you can see here how there's only one possibility for and orbital description, and that's why we don't need to include the m when we're talking about and s orbital.
The other thing that we know, which is what we were just discussing when we were going through the table is how this all relates to energy.
And I want to really highlight here we're talking about for a hydrogen atom -- orbitals with the same n value have the same energy.
Some of you might be saying in your heads, wait a second, I happen to know, I happen to remember from high school, that p orbitals have different energies then, for example, s orbitals.
And that is not true for one electron atoms.
We're going to get to more complicated atoms eventually where we're going to have more than one electron in it, but when we're talking about a single electron atom, we know that the binding energy is equal to the negative of the Rydberg constant over n squared, so it's only depends on n. So, for example, if we're talking about the n equals 2 state, all of these four orbital descriptions are going to have the same energy.
And we can generalize to figure out, based on any principle quantum number n, how many orbitals we have of the same energy, and what we can say is that for any shell n, there are n squared degenerate orbitals.
And the word degenerate simply means same energy, so you have n squared orbitals that are of equal energy when they're degenerate.
So, let's look at where this comes from with an energy level diagram here.
So what you can see is again, we've got this ground state.
So if we go to the ground state, what you see is we're at that lowest energy level, and we only have one possibility for an orbital, because when n equals 1, that's all we can do.
So that's the 1 s orbital -- we have n squared or 1 degenerate orbitals.
When we talk about the n equals 2 state, we now have 2 squared or 4 degenerate same energy orbitals, and those are the 2 s orbital.
And then we also have the l being equal to 1 orbital, so those are going to be the 2 p x, the 2 p z, and the 2 p y orbital.
All four of these orbitals have the same energy, they're degenerate.
And as we go up the next energy level, which is based on n equals 3 principle quantum number, well now we have again the s, so we have the 3 s orbital, we're going to have three 3 p orbitals, right, so we'll have 3 p x, 3 p z, and 3 p y, and now we're actually also going to have five different possible l equals 2 orbitals.
Does anyone remember the l equals 2?
Yes, everyone remembers.
Good.
So we have five possible d orbitals.
We'll call these here the 3 d x y, as the subscript, the 3 d y z, the 3 d z squared, the 3 d x z, and the 3 d x squared minus y squared.
So, what do you need to know here?
What you need to know is that when m equals 0, it's 3 d z squared.
That's it.
Again, these other p -- or the d x y, d y z, those are going to be those more complicated linear combinations, you don't need to worry about them.
Eventually you will, at least, need to know the labels and know a little bit more about them.
And in the second half of this course, Professor Drennen's going to talk to us about transition metals in depth, and that's when we'll really delve into d orbitals.
For right now, you can kind of put the d orbitals in the back of your head.
You need to know how to think about them in the same way we think about s and p orbitals, but for example, you don't yet need to know what all of the names are except for this 3 d z squared here.
So we'll wait on that until we start talking more specifically about atoms where the d orbital becomes very significant.
So, what we can see is this degeneracy.
So what we know now is we can start thinking about the next step because we can fully describe the energy of orbitals, and we can fully describe a complete orbital in terms of its three quantum numbers, and its three positional variables, r, theta, and phi.
So next we can think about okay, what is actually a wave function, and for example, what might the shape of different wave functions be.
So essentially, what we're asking for here is the physical interpretation of psi, of the value of psi for an electron.
And it turns out that the answer to can we have this physical interpretation of thinking about what psi means, the answer is really no, that we can't.
There's no classical way to think about what a wave function is.
There's no classical analogy that explains oh, this is what you can kind of picture when you picture a wave function.
And that's somewhat inconvenient because we're working with wave functions, but it's a reality that comes out of quantum mechanics often, which is that we're describing a world that is so much different from the world that we observe on a day-to-day basis, that we're not always going to be able to make those one-to-one analogies.
But luckily we don't have to worry about how we're going to picture all this, now that I said that, because even though there's no physical interpretation for what a wave function is, there is a physical interpretation for what a wave function squared means.
So when we talk about a wave function squared, we're taking the square of the wave function, any one that we specify between n, l and m, at any position that we specify based on r, theta, and phi.
And if we go ahead and square that, then what we get is a probability density, and specifically it's the probability of finding an electron in a certain small defined volume away from the nucleus.
So it's a probability density.
The important point here is it's not just a probability, it's a density, so we know that it's a probability divided by volume.
And the person we have to thank for actually giving us this more concrete way to think about what a wave function squared is is Max Born here.
And actually after the Schrodinger equation first was put forth, people had a lot of discussions about how is it that we can actually interpret what this wave function means, and a lot of ideas were put forth, and none of them worked out to match up with observations until Max Born here came up with the idea that we just square the wave function, and that's the probability density of finding an electron in a certain defined volume.
And it's very helpful because it gives us a way to think about it.
We can't actually go ahead and derive this equation of the wave function squared, because no one ever derived it, it's just an interpretation, but it's an interpretation that works essentially perfectly.
Ever since this was first proposed, there has never been any observations that do not coincide with the idea, that did not match the fact that the probability density is equal to the wave function squared.
So, also about Max Born, just to give you a little bit of a trivial pursuit type knowledge, he not only gave us this relationship between wave function squared, he also gave us Olivia Newton-John.
This is her grandfather, I don't know if you can see from the eyes, I feel like there's a little bit of a resemblance there.
So, I don't know what she grew up hearing about when she went to her grandparents' house, but it might have been wave function squared.
So, a little tidbit of knowledge for you that's somewhat trivial.
Then back to the non-trivial knowledge that is not trivial at all, in fact, is OK, how do we think about this probability density now that we have a little bit more of an idea.
We know that it's a density, it's not an actual probability.
So, one way we could look at it is by looking at this density dot diagram, where the density of the dots correlates to the probability density.
So, what you see is near the nucleus, the density is the strongest, the dots are closest together.
As you get far away from the nucleus, the dots get farther and farther apart, meaning the probability density at those volumes far away from the nucleus is going to be quite low, eventually going to almost zero, although it turns out that it never goes to exactly zero, so if we're talking about any orbital or any atom, it never actually ends, it never goes to zerio.
But it turns out the probability is only significant within one angstrom.
So you can either say that electrons are very, very tiny or that they're never ending, and both are pretty accurate ways to think about what an atom is.
So, that's probability density, but in terms of thinking about it in terms of actual solutions to the wave function, let's take a little bit of a step back here.
I have yet to show you the solution to a wave function for the hydrogen atom, so let me do that here, and then we'll build back up to probability densities, and it turns out that if we're talking about any wave function, we can actually break it up into two components, which are called the radial wave function and angular wave function.
So, essentially we're just breaking it up into two parts that can be separated, and the part that is only dealing with the radius, so it's only a function of the radius of the electron from the nucleus.
And we abbreviate that by calling it r, which is specified by two quantum numbers, and an l as a function of little r, radius.
And we have the angular wave function, which is specified by l and m, and it's a function of the two angles when we're describing the position of the electron, so theta and phi.
So, let's look at what this actually is for what we're showing here is the 1 s hydrogen atom.
If you look in your book there's a whole table of different solutions to the Schrodinger equation for several different wave functions.
So this is the 1 s, you can look it up if you're interested for the 2 s, or 3 s, or 5 s, or whatever you're curious about.
But what I'm going to show you here is the 1 s solution.
So you can see there's this radial part here, and you have the angular part, you can combine the two parts to get the total wave function.
And what you can see is we have this new constant that we haven't seen before.
So what do you see in there that is new?
Yeah.
This a sub nought.
That's a new constant for us in this course.
This is what's called the Bohr radius, and we'll explain -- hopefully we'll get to it today where this Bohr radius name comes from, but for now what you need to know is just that it's a constant, just treat it like a constant, and it turns out to be equal to 52 .
9 pekameters or about 1/2 an angstrom.
The more important thing that I want you to notice when you're looking at this wave equation for a 1 s h atom, is the fact that if you look at the angular component of the wave function, you'll notice that it's a constant.
It doesn't depend on theta, it doesn't depend on phi.
No matter where you specify your electron is in terms of those two angles, it doesn't matter the angular part of your wave function is going to be the same.
So, what does that mean for us?
Well, essentially what that tells is that these s orbitals are spherically symmetrical.
That should make sense, right, because they're only dependent on r. How far you are away from the nucleus in terms of a radius, they don't depend at all on those two angles, they're independent of theta and they're independent of phi.
So, what I'm showing in this picture here is just an electron cloud that you can see.
Think of it as a probability density plot.
And what here is just a graph of the 1 s wave function going across some radius defined this way, and you can see that the probability -- well, this is the wave function, so we would have to square it and think about the probability.
So this squared at the origin is going to be a very high probability, and it decays off as you get farther and farther away from the nucleus or from the center, and that's independent of the angle.
So, let's look at these probability plots of different s orbitals here, and up top here, we have the probability density plot and what you can see is what I just said, a very high probability density in the nucleus, decays as you go out.
And what is plotted below is the actual wave function, so you can see it starts very high and then the decays down.
More interesting is to look at the 2 s wave function.
So, if we look at the bottom here and the actual plot of the wave function, we see it starts high, very positive, and it goes down and it eventually hits zero, and goes through zero and then becomes negative and then never quite hits zero again, although it approaches zero.
So, at this place where it hits zero, that means that the square of the wave function is also going to be zero, right.
So we can see if we look at the probability density plot, we can see there's a place where the probability density of finding an electron anywhere there is actually going to be zero.
So we can think of a third case where we have the 3 s orbital, and in the 3 s orbital we see something similar, we start high, we go through zero, where there will now be zero probability density, as we can see in the in the density plot graph.
Then we go negative and we go through zero again, which correlates to the second area of zero, that shows up also in our probability density plot, and then we're positive again and approach zero as we go to infinity for r. So, what this means is that when we're looking at an actual wave function, we're treating it as a wave, right, so waves can have both magnitude, but they can also have a direction, so they can be either positive or negative.
So, for example, if we were looking at the actual wave function, we would say that these parts here have a positive amplitude, and in here we have a negative amplitude.
And when we're looking at the probability density graphs, it doesn't make a difference, it's okay, It has no meaning for our actual plot there, because we're squaring it, so it doesn't matter whether it's negative or positive, all that matters is the magnitude.
But when we're thinking about actual wave behavior of electrons, it's just important to keep in the back of our head that some areas have positive amplitude and some have negative.
So we'll talk about this more we get into p orbitals and bonding is where it's going to become an issue.
So I just want to kind of introduce that idea here.
Because if we think about wave behavior of electrons and we're forming bonds, then what we have to do is have constructive interference of 2 different electrons, right, to form a bond, we want to and together those probabilities.
So we want to have constructive interference to form a bond, whereas if we had destructive interference, we would not be forming a bond.
So that's where you have to think about whether it's positive or negative.
You don't have to think about it right now, but you might have heard in high school talking about p orbitals, the phase, sometimes you mark a p orbital as being a plus sign or negative sign.
Did any of you do that in high school at all?
A little bit, yeah.
So, that's having to do with the actual wave function.
So, that'll become more relevant later, bonding actually, a couple lectures down the road.
But I just want to introduce it here while we do, in fact, have the wave function plots up here.
But a real key in looking at these plots is where we, in fact, did go through zer and have this zero probability density.
We call that a node, and a node, more specifically, is any value of either r, the radius, or the two angles for which the wave function, and that also means the wave function squared or the probability density, is going to be equal to zero.
So, we can see in our 1 s orbital, how many nodes do we have?
There's no nodes, yeah.
It looks like we hit zero, but we actually don't -- remember that we never go all the way to zero, so there's these little points if we were to look really carefully at an accurate probability density plot, it would never actually hit zero.
And then, for example, how many nodes do we have in the 3 s orbital?
two.
That's correct.
So we have two nodes in the 3 s orbital.
We can actually specify where those nodes are, which is written on your notes.
For the 2 s orbital, at 2 a nought, so it's just 2 times that constant a nought, which is the Bohr radius.
And for the 3 s, we have one at 1 .
9 a nought, and one at 7 .
1 a nought.
We can also specify what kind of node we're talking about.
We'll introduce in the next course angular nodes, but today we're just going to be talking about radial nodes, and a radial node is a value for r at which psi, and therefore, also the probability psi squared is going to be equal to zero.
So, when we're talking about an s orbital, since there is no angular dependence, and it only depends on r, every single one of our nodes is actually going to specifically be a radial node, right, because these are, for example, this 2 a nought is a value of r, a value of the radius, no matter which way you go around at which there's going to be a node at which there is zero probability density of finding an electron there.
So, it's very easy to calculate, however, the number of radial nodes, and this works not just for s orbitals, but also for p orbitals, or d orbitals, or whatever kind of work of orbitals you want to discuss.
And that's just to take the principle quantum number and subtract it by 1, and then also subtract from that your l quantum number.
So what you can do for a 1 s is just take 1 minus 1 and then l is equal to 0, so you have zero radial nodes.
And that matches up with what we saw.
If we try this for the 2 s, we have 2 minus 1 minus 0.
So what we should expect to see is one radial node, and that is what we see here in the probability density plot.
And then if we think about 3 s, we want to start with 3, we subtract 1, again l is equal to 0, so minus 0 and we have two radial nodes.
So, this should be pretty straight forward, let's see if we can get close to a 100% on this one, which is how many radial nodes does a 4 p orbital have?
And let's give 10 seconds on that, make you think fast here.
OK, so most people were correct, or well, the majority, at least, were correct.
And seeing that it's a 4 p has two nodes -- let's just write this out since not everyone did get it correct.
So, if we're talking about a 4 p orbital, and our equation is n minus 1 minus l, the principle quantum number is 4, 1 is 1 -- what is l for a p orbital?
1
1.
So, I tricked you a little, I guess I didn't put an s up there and that's what we had been talking about, so that was probably the issue.
But what we find is that we have two radial nodes.
All right.
So we can switch back to our notes here.
So, doing those probability density dot graphs, we can get an idea of the shape of those orbitals, we know that they're spherically symmetrical.
We're not going to talk about p orbitals today, we're going to talk about p orbitals exclusively on Friday, and as I said, d orbitals you'll get to with Professor Drennen.
But we can also think when we're talking about wave function squared, what we're really talking about is the probability density, right, the probability in some volume.
But there's also a way to get rid of the volume part and actually talk about the probability of finding an electron at some certain area within the atom, and this is what we do using radial probability distribution graphs.
And what that is the probability of finding an electron in some shell where we define the thickness as d r, some distance, r, from the nucleus.
So, think about what we're saying here.
We're saying the probability of finding an electron at some distance from the nucleus in some very thin shell that we describe by d r. So if you think of a shell, you can actually just think of an egg shell, that's probably the easiest way to think of it, where the yolk, if you really maybe make it a lot smaller might be the nucleus.
And let's also make our egg perfectly symmetric and perfectly round.
But still, when we're talking about the radial probability distribution, what we actually want to think about is what's the probability of finding the electron in that shell?
Think of it as that egg shell part.
So, we can do that by using this equation, which is for s orbitals where the radial probability distribution is going to be equal to 4 pi r squared times the wave function squared, d r. That should make sense to us, because when we talk about a wave function, we're talking about a probability divided by a volume, because we're talking about a probability density.
So if we actually go ahead and multiply it by the volume of our shell, then we end up just with probability, which is kind of a nicer term to be thinking about here.
So, of course, if we're talking about a perfectly spherical shell at some distance, thickness, d r, we talk about it as 4 pi r squared d r, so we just multiply that by the probability density.
We can graph out what this is where we're graphing the radial probability density as a function of the radius.
And what you see is that at zero, you start at zero.
And so, the radial probability density at the nucleus is going to be zero, even though we know the probability density at the nucleus is very high, that's actually where is the highest.
The reason in our radial probability distributions we start -- the reason, if you look at the zero point on the radius that we start at zero is because we're multiplying the probability density by some volume, and when we're not anywhere from the nucleus, that volume is defined as zero.
So, it's a little bit artificial that we're seeing that zero point there.
So, actually I want you to go ahead in your notes and circle that zero point and write "not a node."
This is not a node because a node is where we actually have no probability density.
So this, where we start at zero is not a node, is the first thing to point out.
And as we get further and further from the radius, the volume we're multiplying it by actually gets bigger and bigger, because you can see how the volume of that little thin shell is going to get larger and larger as you get further away.
So there's some distance where the probability of actually finding an electron there is going to be your maximum probability.
And that's what we label as r sub m p, or your most probable radius.
This is the point at which your probability is highest for finding an electron.
This is equal to a sub nought for a hydrogen atom, and we remember that that's just our Bohr radius, which is 0 .
5 2 9 angstroms.
And basically, what that means is you can actually find an electron anywhere going away from the nucleus, but you're most likely to find that you have the highest probability at a distance of a sub nought, or the Bohr radius.
So, I said I'd tell you a little bit more about where this Bohr radius came from, and it came from a model of the atom that pre-dated quantum mechanics, and Neils Bohr is who came up with the idea of the Bohr radius, and here is hanging out with Einstein, so he had some pretty good company that he kept.
And what you need to remember when we're thinking about this model of the atom is that in 1911 it had already been discovered that we have an electron, and we have a nucleus, and there needs to be some way that those two hang together, but it was not for another 15 years that we actually had the Schrodinger equation that allowed us to understand the interaction fully between the electron and the nucleus.
So all that Bohr, for example, had to go on at this point was a more classical picture of the atom, as you can see on the left side of the screen there, which is the idea that the electrons actually somehow just orbiting the nucleus.
And even though he could figure out that this wasn't possible, he still used this as a starting point, and what he did know was that these energy levels that were within hydrogen atom were quantized.
and he knew this the same way that we saw it in the last class, which is when we viewed the difference spectra coming out from the hydrogen, and we also did it for neon, but we saw in the hydrogen atom that it was very discreet energy levels that we could observe.
He knew the same thing that had been observed by that point.
So, what he did was kind of impose a quantum mechanical model, not a full one, just the idea that those energy levels were quantized on to the classical picture of an atom that has a discreet orbit.
And what he came out with when he did some calculations is that there's the radius that he could calculate was equal to this number a sub nought, which is what we call the Bohr radius, and it turns out that the Bohr radius happens to be the radius most probable for a hydrogen atom.
And the reason we won't talk any more about this Bohr model is because, of course, it's not correct.
So we're not going to spend too much time on it here.
But we can see, for example, one reason or one way in which is not correct.
Because what it tells is that we can figure out exactly what the radius of an electron and a nucleus are in a hydrogen atom.
That's a deterministic way of doing things, that's what you get from classical mechanics.
But the reality that we know from our quantum mechanical model, is that we can't know exactly what the radius is, all we can say is what the probability is of the radius being at certain different points.
so, that's a more complete quantum mechanical picture of what is going on here.
So if we superimpose our radial probability distribution onto the Bohr radius, we see it's much more complicated than just having a discreet radius.
We can actually have any radius, but some radii just have much, much smaller probabilities of actually being significant or not.
So, I think we're a little bit out of time today, but we'll start next class with thinking about drawing radial probability distributions of more than just the 1 s orbital.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So, let's get started.
Why doesn't everyone take 10 more seconds to answer this clicker question here.
All right.
So, good.
It looks like just about everyone is able to go from the name of an orbital to the state function.
That's important.
And we're actually going to get a little bit deeper in our clicker questions here, since when you do your problem-set it won't be quite this straight forward that you'll be answering this kind of question, but actually you'll be thinking about how many different orbitals can have certain state functions or certain orbital names.
So let's go to a second clicker question here and try one more.
So why don't you tell me how many possible orbitals you can have in a single atom that have the following two quantum numbers?
So let's say we have n equals 4, and n sub l equalling negative 2.
How many different orbitals can you have that have those two quantum numbers in them?
And this should look kind of familiar to some of the problems you may have seen on the problem-set if you started it this weekend.
So this should be something you can do pretty quickly, so let's take 10 more seconds on that.
All right.
OK, looks like we got the majority, which is a good start, but we having discrepancy on what people are thinking.
So, let's go through this one.
So, what we're saying is that we have n equals to 4, and m sub I being equal to negative 2.
If we have n equals 4, what is the highest value of l that we can have?
3
OK. We can have n 4, l 3, and then, sure, we can have m sub l equal negative 2 if l equals 3 What's the second value of l that we can have?
2.
OK.
So we can have this orbital here.
What about l equals 1, can we have this?
No, we can't.
Because if l equals 1, we can not have m sub l equal negative 2, right, because the magnetic quantum number only goes from negative l to positive l here.
So that means it's not possible, if we've made these stipulations in the first place, to have a case where l equals 1.
So this means we can only have 2 different values of l. We already know our value of m. So now we're just counting up our orbitals, an orbital is completely described by the 3 quantum numbers.
So we end up having 2 orbitals here.
All right, so hopefully if you see any other combination of quantum numbers, for example, if it doesn't quickly come to you how many orbitals you have, you can actually try to write out all the possible orbitals and that should get you started.
All right.
So today we're going to finish up our discussion of the hydrogen atom.
We'd started on Monday talking about radial probability distributions for the s orbitals.
We'll finish that up, and then we're going to move on to talking about the p orbitals.
We'll start with talking about the shape, just like we did with the s orbitals, and then move on to those radial probability distributions and compare the radial probability at different radius for p orbital versus an s orbital.
And once we do that, we're actually going to move on to multi-electron atoms.
So, you might have noticed that we will have spent about 6 and 1/2 lectures just getting to the point where we have only one electron, so we're only up hydrogen so far.
And you might have kind of been projecting ahead and thinking if we keep up at this pace, pretty much we would only get to carbon by the end of the semester.
So I will assure you that we will not be spending 6 lectures per atom as we move forward, and in fact, what we're going to find is that by taking all the principles we've learned about the hydrogen atom and applying that to multi-electron atoms, but making a few of changes and making a few modifications to take into the fact that we're going to have electron-electron repulsions going on, we'll be able to think about any multi-electron atom using these same general ideas, and the Schrodinger equation ideas that we came up with and have looked at for the hydrogen atom.
And this is really good news because it's good to get passed carbon.
I'm an organic chemist, so I love carbon, it's one of my favorite atoms to talk about, but it would be nice to get to the point of bonding and even reactions to talk about all the exciting things we can think about once we're at that point.
So, one we finish our discussion of how we think about multi-electron atoms, we can go right on and start talking about these other things.
All right.
So, let's pick up where we left off, first of all we're still on the hydrogen atom from Monday.
And on Monday what we were discussing was the solution to the Schrodinger equation for the wave function.
And we also, when we solved or we looked at the solution to that Schrodinger equation, what we saw was that we actually needed three different quantum numbers to fully describe the wave function of a hydrogen atom or to fully describe an orbital.
We didn't just need that n, not just the principle quantum number that we needed to discuss the energy, but we also need to talk about l and m, as we did in our clicker question up here.
We also talked about well, what is that when we say wave function, what does that actually mean?
And first we discussed the fact that well, in terms of a classical analogy, we don't really have one for wave function, we can't really think of a way to picture wave function thinking in classical terms.
But we do have an interpretation for wave function squared.
And when we take the wave function and square it, that's going to be equal to the probability density of finding an electron at some point in your atom.
And that's useful in terms of seeing a general shape, but if we're actually interested in thinking about how far away that electron is from the nucleus, you can see that instead of talking about probability density, which is the probability per volume, instead it would be much more useful to talk about something called radial probability distribution, or in other words talking about the probability of finding the electron at some distance, which we define as r, from the nucleus in a spherical shell that is just infinitesimally small, infinitesimally thin at distance or at a thickness that we'll call a d r here.
So, basically what we're saying is if we take any shell that's at some distance away from the nucleus, we can think about what the probability is of finding an electron at that radius, and that's the definition we gave to the radial probability distribution.
And we can look at the formula that got us here.
This is the radial probability distribution formula for an s orbital, which is, of course, dealing with something that's spherically symmetrical.
It's somewhat different when we're talking about the p or the d orbitals, and we won't go into the equation there, but this will give you an idea of what we're really talking about with this radial probability distribution.
So, what we can do to actually get a probability instead of a probability density that we're talking about is to take the wave function squared, which we know is probability density, and multiply it by the volume of that very, very thin spherical shell that we're talking about at distance r. So if we want to talk about the volume of that, we just talk about the surface area, which is 4 pi r squared, and we multiply that by the thickness d r. So if we take this term, which is a volume term, and multiply it by probability over volume, what we're going to end up with is an actual probability of finding our electron at that distance, r, from the nucleus.
So, the example that we took on Monday and that we ended with when we ended class, was looking at the 1 s orbital for hydrogen atom, and what we could do is we could graph the radial probability as a function of radius here.
And when we do that we can see this curve, this probability curve, where we have a maximum probability of finding the electron this far away from the nucleus.
And we call that most probable radius r sub m p, or most probable radius.
And what is discussed is that for a 1 s hydrogen atom, that falls at an a nought distance away from the nucleus.
And remember, a nought, that's just the Bohr radius, it's a constant -- that's all we need to worry about.
We talked about the Bohr model and how that told us an exact distance.
It was a classical model, right, so we could say the electron is exactly this far away from the nucleus.
We can not do that with quantum mechanics, the more true picture is the best we can get to is talk about what the probability is of finding the electron at any given nucleus.
And the most probable one here is that a nought.
The other thing that we looked at, which I want to stress again and I'll stress it as many times as I can fit it into lecture, because this is something that confuses students when they're trying to identify, for example, different nodes or areas of no probability.
In an orbital is remember that this area right here at r equals zerio, that is not a node.
We will always have r equals zero in these radial probability distribution graphs, and we can think about why that is.
At first it might be counter-intuitive because we know the probability density at the nucleus is the greatest.
So the probability of having an electron at the nucleus in terms of probability per volume is very, very high.
But remember that we need to multiply it by the volume here, the volume of some sphere we've defined.
And when we define that as r being equal to zero, essentially we're multiplying the probability density by zero.
So that's why we have this zero point here, and just to point out again and again and again, it's not a radial node, it's just a point where we're starting our graph, because we're multiplying it by r equals zero.
So, we can look at other radial probability distributions of other wave functions that we talked about.
We talked about the wave function for a 2 s orbital, and also for a 3 s orbital.
So, let's go ahead and think about drawing what that would look like in terms of the radial probability distribution.
So what we're graphing here is the radius as a function of radial probability.
And for a 2 s orbital, you get a graph that's going to look something like this.
So, again, we're starting at zero.
We have one node here, and we can again define that most probable radius.
And it turns out that for a 2 s orbital, that's equal to 6 times a nought.
So when we think about what it is that this radial probability distribution is telling us, it's telling us that it is most likely that an electron in a 2 s orbital of hydrogen is six times further away from the nucleus than it is in a 1 s orbital.
So another way to say that is, in a sense, if we're thinking about the excited state of a hydrogen atom, the first excited state, or the n equals 2 state, what we're saying is that it's actually bigger than the ground state, or the 1 s state of a hydrogen atom.
And when we say bigger, remember this is not a classical description we're talking about.
We are talking about probability, but what we're saying is that most probable radius is further away from the nucleus.
So we can also look at this in terms of the 3 s orbital.
And in this case, we have a graph that looks something like this.
So you can draw that into your notes.
And again, we can define what that most probable radius is, that distance at which we're most likely to find an electron.
And in the case of the 3 s orbital, that's going to be equal to 11 .
5 times a nought.
So again, what we're saying here is that it is most likely in the 3 s orbital that we would find the electron 11 and 1/2 times further away from the nucleus than we would in a around state hydrogen atom.
And I just want to point out here in terms of things that you're responsible for, you should know that the most probable radius for a 1 s hydrogen atom is equal a nought.
And you should know that a 2 s is larger than that, and a 3 s is even larger, and of course, hopefully as we go to 4 and 5, you would be able to guess that those are going to get even larger.
But you're not responsible for knowing specifically that it's 11 .
5 times greater.
You just need to know the trend there.
Another thing to point out in these two graphs is that we do have nodes, and we figured out last time, we calculated how many nodes we should have in a 2 s orbital.
And in terms of radial nodes, we expect to see one node.
And how many nodes do you see in the 3 s orbital?
two, good.
I'm glad to hear that no one counted this r equal zero as a node.
So we expect to see two nodes right here in the 3 s orbital.
And we can calculate that with the formula that we used, which was just n minus l minus 1 equals the number of nodes.
Or we could just look at the radial probability distribution itself and see how many nodes there are.
So if we're looking at these two situations here and we're actually thinking of them from a more classical standpoint, which is natural for us to do because we live our lives in the every day world, not thinking about things on the atomic size scale all the time, most of us.
So, for example, if we were to look at this 3 s orbital here, you might have the question of how this can be, because we're saying that, for example, we have probability of having an electron here, an electron can also be way out at this radius here.
But what we're saying is there's a node here, so that there's no probability of finding an electron between those two points.
So you can think of it, if we were to just think of it as a straight line that we were going across, essentially what we're saying is that we're getting from point a to point c without ever getting through point b.
So, that can be a little bit confusing for us to think about, and when it's a very good question you might, in fact, say well, maybe there's not zero probability here, maybe it's just this teeny, teeny, tiny number, and in fact, sometimes an electron can get through, it's just very low probability so that's why we never really see it.
And in fact, that's not the answer.
The answer is, in fact, there is zero, absolutely zero probability of finding a electron here.
So basically we're saying yes, we can go from point a to point c without ever going through point b.
That might seem confusing if you're thinking about particles, but remember we're talking about the wave-like nature of electrons.
So, the quantum mechanical interpretation is that we can, in fact, have probability density here and probability density there, without having any probability of having the electron in the space between.
And you can think about that if you think about a standing wave, for example, where you can have amplitude at many different values of x, so an amplitude at many different distances, but you also have areas where there is a 0 amplitude.
So, remember this makes sense if you just think of it as a wave and forget the particle part of it for right now, because that would be very upsetting to think about and that's not, in fact, what's going on, we're talking about quantum mechanics here.
Yes
[INAUDIBLE]
Oh, I'm sorry.
So it's n minus l minus 1.
So here we have 3 minus l equals 0, because it's an s orbital, minus 1, so we have two radial nodes here.
OK.
So let's actually go to a clicker question now on radial probability distributions.
So I mentioned you should be able to identify both how many nodes you have and what a graph might look like of different radial probability distributions.
So here, what I'd like you to do is identify the correct radial probability distribution plot for a 5 s orbital, and also make sure that it matches up with the right number of radial nodes that you would expect.
All right.
Let's take 10 more seconds on that, this should be a quick identification for us to do.
All right.
So it looks like 82% got the correct answer here.
So, you should know that there's four radial nodes, right, we have 5 minus 1 minus l -- is there a question?
[INAUDIBLE]
It is very difficult for me to draw graphs on the computer.
That's a good point, I'm sorry.
This was my best attempt at hitting zero and not having the graph go down there.
I'm not the most gifted at drawing on the computer.
So yes, it should be zero at zero, but I made the line too thick.
So, assuming -- if anyone got it wrong because of that, that's my apologies, that's my fault.
But you should see that there are four radial nodes here since we have a 5 s orbital.
And also that we know that the zero does not count as a node, if per se I actually had managed to hit zero in drawing that, so the correct answer would be the bottom one there.
So, you should be able to generally identify and draw the general form of these radial probability distributions.
Obviously we don't expect you to know exactly what the distances are, but you should be able to compare them relatively.
Yes?
[INAUDIBLE]
No, they actually don't, and when you graph it all out.
You can see this if you look at some examples in your book, actually.
So this doesn't fall, for example, at 6 a nought, but that's a really good question.
And the trend always is that the probability gets smaller with each of the peaks as you're drawing them.
All right.
So we can switch back to our notes.
So we got our clicker question set there.
And so now we can move on to thinking about p orbitals, we now have two ways to talk about p orbitals.
We can talk about the wave function squared, the probability density, or we can talk about the radial probability distribution.
So when we talk about p orbitals, it's similar to talking about s orbitals, and the difference lies, and now we have a different value for l, so l equals 1 for a p orbital, and we know if we have l equal 1, we can have three different total orbitals that have sub-shell of l equalling 1.
So we can have, if we have the final quantum number m equal plus 1 or minus 1, we're dealing with a p x or a p y orbital.
Remember, we don't do a one-to-one correlation, because p x and p y are some linear combination of the m plus 1 and m minus 1 orbital.
And if we talk about m equals 0, we're looking at the p z orbital.
And the significant difference between s orbitals and p orbitals that comes from the fact that we do have angular momentum here in these p orbitals, is that p orbital wave functions do, in fact, have theta and phi dependence.
So they do have an angular dependence that we're talking about.
And what I'm showing here is not on your notes, if you're interested you can look it up in your book.
This is a table that's directly from your book, and what it's just showing is the wave function for a bunch of different orbitals.
I mentioned last time that there was this list in your book.
And what I want to point out here is this angular dependence for the p orbitals for the l equals 1 orbital.
So, first, if I point out when l equals 0, when we have an s orbital, what you see is that angular part of the wave function is equal to a constant.
So, remember we can break up the total wave function into the radial part and the angular part.
When we look at this angular part, we see that it's always the square root of 1 over 4 pi, it doesn't matter what the angle is, it's not dependent on the angle.
In contrast when we're looking at a p orbital, so any time l is equal to 1, and you look at angular part of the wave function here, what you see is the wave function either depends on theta or is dependent on both theta and phi.
So we do, in fact, have a dependence on what the angle is of the electron as we define it in the orbital.
So what this means is that unlike s orbitals, p orbitals are not spherically symmetrical -- they don't have the exact same shape at any radius from the nucleus.
And these shapes of p orbitals probably do look familiar to you, most of you, I'm sure, have seen some sort of picture of p orbitals before.
So what I want to point out about them is that they're made up of two nodes, and what you can see is that nodes are shown in different colors here and those are different phases.
Sometimes you see this written when you see p orbitals, one is written as plus, one is written in minus.
That's not a positive and negative charge, that's actually a phase, and that arises from the wave equation.
Remember when we have waves we can have positive or a negative amplitude.
When we talk about p orbitals the phase of the orbital becomes important once we talk about bonding, which hopefully you were happy to hear at the beginning of class we will get to soon.
And it turns out that when you constructively have two p orbitals interfere, and when I say constructively, I mean they're both either positive or they're both the negative lobes, that's when you got bonding.
Whereas if the phases where mismatched, you would not get bonding.
So, that's going to be important later when we get to bonding, but just take note of it now, we have two nodes, each with a separate phase -- or we have two lobes, excuse me, each with a separate phase.
And when we look at this, it's actually split by what's called a nodal plane, which is pointed out in light orange here on this picture, but what we just mean is that there is this whole plane that separates the two lobes where there is absolutely no electron density.
So, the wave function at all of these points in this plane is equal to zero, so therefore, also the wave function squared is going to be equal to zero.
So, if we say that in this entire plane we have zero probability of finding a p electron anywhere in the plane, the plane goes directly through the nucleus in every case but a p orbital, so what we can also say is that there is zero probability of finding a p electron at the nucleus.
So, again we can use these probability density plots, which are just a plot of psi squared, where the density of the dots is proportional to the density, the probability density, at that point.
So what we can say is look at each of these separately, so if we start with looking at the 2 p z orbital, the highest probability of finding an electron in the 2 p z orbital, is going to be along this z-axis.
We can see that right here.
And in terms of thinking about the phase of this p orbital, the phase is going to be positive anywhere where z is positive.
So we would say we have a positive phase here and a negative phase there.
Remember, that's going to become important when we talk about bonding, we don't need to worry about it too much right now.
We can also think about where the nodal plane is in this p z orbital, so how would we define the nodal plane here?
What would the nodal plane be?
So, it's the x-y plane, you can see there's no electron density anywhere there.
And similarly, actually, if we're looking at our polar coordinates here, what we see is it's any place where theta is equal to 0 is what's going to put up on the x-y plane.
So another way to define the nodal plane is where theta is equal to 90 degrees.
So let's look now at the 2 p x orbital.
This is the probability density map, so we're talking about the square here.
The highest probability now is going to be along the x-axis, so that means we're going to have a positive wave function every place where x is positive.
What is the nodal plane in this case?
Um-hmm.
So, it's going to be the y z nodal plane, or in other words, we can say it's any place where phi is equal to 90 degrees.
So you can see if you take phi, and you move it over 90 degrees, we're right here in the y z plane.
Anywhere where that's the case we're going to have no probability density of finding an electron.
And finally, we can look at the 2 p y, so the highest probability is going to be along the y-axis.
It's going to be positive in terms of its wave function or in terms of its phase anywhere where y is positive.
And the nodal plane's going to be in the x z plane, or again, anywhere where phi is going to be equal to 0, that takes us to the x z plane.
So, let me get a little bit more specific about what we mean by nodal plane and where the idea of nodal plane comes from, and nodal planes arise from any place you have angular nodes.
So we talked about radial nodes when we're doing these radial probability density diagrams here.
You can also have angular notes, and when we talk about an anglar node, what we're talking about is values of theta or values of phi at which the wave function, and therefore, the wave function squared, or the probability density are going to be equal to zero.
So, you remember from last time radial nodes are values of r at which the wave function and wave function squared are zero, so the difference is now we're just talking about the angular part of the wave function.
And, in fact, these are the only two types of nodes that we're going to be describing, so we can actually calculate both the total number of notes and the number of each type of node we should expect to see in any type of orbital.
And our equation for total nodes is just the principle quantum number minus 1.
And when we talk about angular nodes, the number of angular nodes we have in an orbital is going to be equal to l. So that's why we saw, for example, in the p orbitals we had one angular node in each p orbital, because l is equal to 1 there.
And we talked about the equation you can use for radial nodes last time, and that's just n minus 1 minus l. You can go ahead and use that equation, or you could figure it out every time, because if you know the total number of nodes, and you know the angular node number, then you know how many nodes you're going to have left.
So you don't really have to memorize that.
So, let's go ahead and just do a few of these.
They're pretty straight forward to do and it gives us an idea what kind of nodal structure we can expect it an orbital.
So for a 2 s orbital, how many total nodes will we have?
Yup, I heard one, so 2 minus 1, one total node.
Angular nodes, we're not going to have any of those, we'll have zero, l equals 0, so we have zero angular nodes.
And in terms of radial nodes, we have 2 minus 1 minus 0, so what we have is one radial node.
So, what you find with the s orbital, and this is general for all s orbitals is that all of your nodes end up being radial nodes.
That has to be the case because l equals 0 for s orbitals.
Let's look now at a p orbital, so how many total nodes do we have here?
Yup, so one total node, 2 minus 1 is 1, and that means since l is equal to 1, we have one angular nodes, and that leaves us with how many radial nodes?
Yup, zero radial nodes.
So, for a 2 p orbital, all the nodes actually turn out to be angular nodes.
So, let's have you try one more, if we can switch over and talk about a 3 d orbital.
So, I'm asking very specifically about radial nodes here, how many radial nodes does a hydrogen atom 3 d orbital have?
So, you can go ahead and take 10 seconds on that.
All right.
So most of you got that, though there is this little sub-set we have thinking that we have one, so let's actually write this out here.
So if we have a 3 d orbital, we're talking about n minus l minus 1, what is n equal to?
What is l equal to?
OK. and 1 is equal to 1.
So, it turns out that we have zero nodes that we're dealing with when we're talking about a 3 d orbital.
OK.
So we should be able to figure this out for any orbital that we're discussing, and when we can figure out especially radial nodes, we have a good head start on going ahead and thinking about drawing radial probability distributions.
We did it for the s orbitals, we can also do it for the p, we can do it for the d. All we have to figure out is how many nodes we're dealing with and then we can get the general shape of the graph here.
So, let's actually compare the radial probability distribution of p orbitals to what we've already looked at, which are s orbitals, and we'll find that we can get some information out of comparing these graphs.
So if we draw the 2 p orbital, what we just figured out was there should be zero radial nodes, so that's what we see here.
The other thing that I want you to notice, is if you look at the most probable radius, for the 2 s orbital it's actually out further away from the nucleus than it is for the 2 p orbital.
So what we can say here is that the 2 p is less than or smaller than the 2 s orbital.
So think about what that means, we're, of course, not talking about this in classical terms, so what it means if we have an electron in the 2 p orbital, it's more likely, the probability is that will be closer to the nucleus than it would be if it were in the 2 s orbital.
We can also take a look the 3 s, which we have looked at before, and we figured out that that should have two radial nodes.
We can look at the 2 p, which should have one radial node, and we just figured it out for the, excuse me, for the 3 p has one radial node, and for the 3 d here, we should have zero radial nodes, we just calculated that.
So again, what we see is the same pattern where the most probable radius, if we talk about it in terms of the d, that's going to be smaller then for the p, and the 3 p most probable radius is going to be closer to the nucleus than it is for the 3 s most probable radius that we're looking at.
So, there are 2 different things that we can compare when we're comparing graphs of radial probability distribution, and the first thing we can do is think about well, how does the radius change, or the most probable radius change when we're increasing n, when we're increasing the principle quantum number here?
So, from going from the shell of n equals 2, let's say, to the shell of n equals 3.
And what we find is we're going from about or exactly a 6 a nought here, to almost three times that when we're going from 2 s to 3 s. So we say if n increases, the orbital size is also going to increase.
And when we talk about size, I'm again just going to say the stipulation that we're talking about, probability -- we're not talking about an absolute classical concept here, but in general we're going to picture it being much further away from the nucleus as we move up in terms of n. The other thing that we took note as is what happens as l increases, and specifically as l increases for any given the principle quantum number.
So if we're keeping n the same, we look and what we saw was that size actually decreases as we increase the value of l. So, I want to contrast that with another concept that seemed to be opposing ideas, and that is thinking about not how far away the most probable radius is, but thinking about how close an electron can get to the nucleus if it's actually in that orbital.
And what we'll find is that we actually see the opposite.
So if we compare l increasing here, so a 3 s to a 3 p to a 3 d, what we find is that it's only in the s orbital that we have a significant probability of actually getting very close to the nucleus.
So, if I kind of circle where the probability gets somewhat substantial here, you can see we're much closer to the nucleus at the s orbital than we are for the p, then when we are for the d. So, the size still for an s orbital is larger than for a d orbital, but what we say is that an s electron can actually penetrate closer to the nucleus.
There's some probability that it can get very, very close the nucleus, and that probability is actually substantial.
A kind of consequence of this is if we're thinking about a multi-electron atom, which we'll get to in a minute where electrons can shield each other from the pull of the nucleus, we're going to say that the electrons in the s orbitals are actually the least shielded.
And the reason that they're the least sheilded is because they can get closest to the nucleus, so we can think of them as not getting blocked by a bunch of other electron, because there's some probability that they can actually work their way all the way in to the nucleus.
So, this is a concept that's going to become really important.
Soon when we're talking about multi-electron atoms, and I just want to introduce it here, that it is sort of opposing ideas that even though the s is the biggest and it's most likely that the electron's going to be furthest away from the nucleus, that's also the orbital in which the electron can, in fact, penetrate closest.
All right.
So I think we are, in fact, ready to move on to multi-electron atoms, and what happens is when we solved the relativistic version of the Schrodinger equation and we're discussing more than one electron, we actually have a fourth quantum number that falls out and that we need to deal with and this is called the electron spin quantum number.
And I promise, this is the last quantum number that we'll be introducing.
And this spin magnetic quantum number we abbreviate as m sub s, so that's to differentiate from m sub l. And when you solved the relativistic form of the Schrodinger equation, what you end up with is that you can have two possible values for the magnetic spin quantum number.
You can have it equal to plus 1/2, and that's what we call spin up, or you can have it equal to minus 1/2, which is what we call spin down.
So, there's two kind of cartoons shown here that give you a little bit of an idea of what this quantum number tells us.
And this spin is an intrinsic quality of the electron, it's a property that is intrinsic in all particles, just like we would say mass is intrinsic or charge is intrinsic.
Spin is also an intrinsic property.
One way to think about it, if we want to use a classical analogy, which often helps to give us an idea of what's going on, is the spin of an electron, we can picture it rotating on its own axis.
So that's kind of what's shown in these pictures here.
So you can see, if it's spinning on its own axis in this direction we'd call it spin up, where as this way it would be what we call spin down.
So, it turns out there's not actually a good classical analogy for spin, we can't really think of it like that, but if that helps give you an idea of what's going on here then it's valuable maybe to consider it spinning on its own axis, even though that's not technically what's exactly happening here.
But the reason that I like that analogy is that it points out a very important part of spin, and that's the idea that it's a description of the electron.
It is not dependent on the actual orbital.
So we can completely describe an orbital with just using three quantum numbers, but we have this fourth quantum number that describes something about the electron that's required for now a complete description of the electron, and that's the idea of spin.
So, we need to actually add on this fourth quantum number, and it's either going to be plus 1/2 or negative 1/2.
So, we can talk a little bit actually, because it's an interesting story about where the idea of spin came from, and it was actually first proposed by two very young scientists at the time, George Uhlenbeck shown here, and Samuel Goudsmit who's here, and they're with a friend, and I can't remember who that is, but he did not have anything to do with discovering spin.
And what you can see in this picture is that these are actually, they're pretty young guys in this picture.
I think this is taken about two years after they discovered the fourth quantum number.
So hopefully, you can picture yourself at this age in a similar situation with an anonymous friend and think this is something, kind of observations maybe you can make as well.
And what they were doing when they discovered that there must be this fourth quantum number is they were looking at the emission spectrum of sodium, and, so specifically, they were looking at the frequencies, and if we think about the frequencies of sodium, it was already known at this time that you could calculate what those would be based on the difference in the energy levels -- this happened in about 1925.
So they actually knew exactly what they were expecting.
So, let's say they were expecting to see one certain frequency or one line in the spectrum at this point here.
It turns out that what they actually observed, so this is the actual of what they saw, is if they were expecting their line at some given frequency, which I'll show by this dotted line here, what they actually saw was two lines, and one was just the teeniest tiniest bit of a higher frequency than what was expected, and one was just, just below what they expected.
And if we're talking about things in spectroscopy terms here, this is what we call a doublet.
So it's centered at this frequency that was expected, but it's actually split into two different frequencies.
And this was an amazing observation that they made.
They were totally surprised and excited, and they were thinking how could this happen, where did we got this split doublet from.
And what they could come up with, what they reasoned, is that there must be some intrinsic property within the electron, because we know that this describes the complete energy of the orbital should give us one single frequency.
And that the fact that it split into two was telling them that there must be some new property to the electron, and what we call that now is either being spin up or spin down.
But at the time, they didn't have a well-formed name for it, they were just saying OK, there's this fourth quantum number, there's this intrinsic property in the electron.
So, where the story gets kind of unfortunate, but also a little bit more interesting is the fact that well, they did publish what they observed, and they did write that up.
They put it in a pretty low impact paper.
I think it was in French, so it wasn't really hitting all the scientific community of the time in any major way.
And they didn't put their explanation of what they thought was going on, it just sort of was observing what they saw.
These were very young scientists, of course, so what you would expect that they would do, which makes sense, is go to someone more established in their field, because they have the completely radical revolutionary idea, let's just run it by someone before we go ahead and publish this paper that makes this huge statement about this fourth quantum number.
So the person they chose to talk to, and I think it was just Goudsmit that went to him and discussed it is Wolfgang Pauli, which is shown here.
So how many before five minutes ago had heard the name Goudsmit?
All right, a couple.
OK, cool.
How about Pauli, like the Pauli exclusion principle?
Hmm, OK.
So, Pauli seems to be getting a little bit of fame that you'll see in a second here.
So, it turns out they go and they discuss their idea with Pauli, and what Pauli tells them is that this idea is ridiculous, that it's rubbish, if they go ahead and try to publish this their scientific careers are ruined.
They can pretty much pack it up and go home, because everyone's going to think they're ridiculous, no one will believe what they say, and it's a stupid idea anyway, is basically the gist of this conversation.
And in chemistry, just like in any discipline, you have all types of scientists, but also all types of personalities, and unfortunately Pauli had a personality that was known for, first of all, being very arrogant, but also the very unfortunate trait of taking other people's scientific ideas as his own.
And as the story goes, as Goudsmit was leaving and the door with slamming, Wolfgang Pauli was already writing down this idea into a scientific paper of the idea of a fourth quantum number.
And, in fact, he did make some more strides, he was a brilliant thinker, maybe he put it more articulately than those two younger scientists could have.
But now, it has come to light that they are the ones that do get credit for first really coming up with this idea of a spin quantum number, and it's interesting to think about how the politics work in different discoveries, as well as the discoveries themselves.
You see that a lot with Nobel prizes, there's usually a nice little scandal, a nice little interesting story behind who else was responsible for the Nobel Prize-worthy discovery.
So, here, Pauli came out on top, we say, and he's known for the Pauli exclusion principle, which tells us that no two electrons in the same atom can have the same four quantum numbers.
So let's just think exactly what this means, and that means that if we take away function and we define it in terms of n, l and m sub l, what we're defining here is the complete description of an orbital.
In contrast, if we're taking the wave function and describing it in terms of n, l, m sub l, and now also, the spin, what are we describing here?
An electron.
So now we have the complete description of an electron within an orbital.
So, that's an important distinction to make -- what three quantum numbers tell us, versus what the fourth quantum number can fill in for us in terms of information.
So what that means is that we're limited in any atom to having two electrons per orbital, right, because for any orbital we can either have a spin up electron, a spin down electron, or both.
So, if we look at neon just as an example, neon has ten electron in it, and once we look at all the orbitals written out here, this is probably a familiar thing for you to look at, but it's important to think about why, in fact, we don't just put all ten electrons, why wouldn't they just want to go in that ground state, that lowest state, right?
That should be the most stable, the lowest energy orbital for it to be in, and the reason they can't do that is because of the Pauli exclusion principle -- the idea that all of the electrons have to have a different set of four quantum numbers, so only two of them can have the same set of three quantum numbers here, because for m sub s, we're only left with two options.
So let's try a clicker question and thinking about the Pauli exclusion principle.
It might look a little bit similar to a question we just saw, but hopefully you'll find that it is, in fact, not the same question.
So you can go ahead and take 10 seconds on that.
OK, great.
So, most of you recognize that there are four different possibilities of there's four different electrons that can have those two quantum numbers.
Actually the easiest way is probably to bring this down here.
And the next highest percentage of you thought that we still only had two.
So, remember we solved this problem earlier in the class, but we were talking about orbitals.
So there's two different orbitals that can have these three quantum numbers, but if we're talking about electrons, we can also talk about m sub s, so if we have two orbitals, how many electrons can we have total?
Yeah.
So we have two orbitals, or four electrons that can have that set of quantum numbers.
So you'll notice in your problem-set, sometimes you're asked for a number of orbitals with a set of quantum numbers, sometimes you're asked for a number of electrons for a set of quantum numbers.
So make sure first that you read the question carefully, and realize the difference that is between the two.
So, that's where we'll end today.
So on Friday, we'll start with talking about the wave functions for the multi-electron atoms.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. As you're settling into your seats, why don't we take 10 more seconds on the clicker question here.
All right, so this is a question that you saw on your problem-set, so this is how many electrons would we expect to see in a single atom in the 2 p state.
So, let's see what you said here.
Six.
And the correct answer is, in fact, six.
And most of you got that, about 75% of you got that right.
So, let's consider some people got it wrong, however, and let's see where that wrong answer might have come from, or actually, more importantly, let's see how we can all get to the correct answer.
So if we say that we have a 2 p orbital here, that means that we can have how many different complete orbitals have a 2 for an n, and a p as its l value?
three.
So, we can have the 2 p x, 2 p y, and 2 p z orbitals.
Each of these orbitals can have two electrons in them, so we get two electrons here, here, and here.
So, we end up with a total of six electrons that are possible that have that 2 p orbital value.
So this is a question that, hopefully, if we see another one like this we'll get a 100% on, because you've already seen this in your problem-set as much as you're going to see it, and you're seen it in class as much as you're going to see it.
So if you're still having trouble with this, this is something you want to bring up in your recitation.
And the idea behind this, of course, is that we know that every electron has to have its own distinct set of four quantum numbers.
So that means that if we have three orbitals, we can only have six electrons in those complete three orbitals.
All right, so today we're going to fully have our discussion focused on multi-electron atoms.
We started talking about these on Wednesday, and what we're going to start with is considering specifically the wave functions for multi-electron atoms.
So, the wave functions for multi-electron atoms.
Then we'll move on to talking about the binding energies, and we'll specifically talk about how that differs from the binding energies we saw of hydrogen atoms.
We talked about that quite in depth, but there are some differences now that we have more than one electron in the atom.
Then something that you probably have a lot of experience with is talking about electron configuration and writing out the electron configuration.
But we'll go over that, particularly some exceptions, when we're filling in electron configurations, and how we would go about doing that for positive ions, which follow a little bit of a different procedure.
And if we have time today, we'll start in on the photo-electron spectroscopy, if not, that's where we'll start when we come back on Wednesday.
So, what we saw just on Wednesday, in particular, but also as we have been discussing the Schrodinger equation for the hydrogen atom, is that this equation can be used to correctly predict the atomic structure of hydrogen, and also all of the energy levels of the different orbitals in hydrogen, which matched up with what we observed, for example, when we looked at the hydrogen atom emission spectra.
And what we can do is we can also use the Schrodinger equation to make these accurate predictions for any other atom that we want to talk about in the periodic table.
The one problem that we run into is as we go to more and more atoms on the table, as we add on electrons, the Schrodinger equation is going to get more complicated.
So here I've written for the hydrogen atom that deceptively simple form of the Schrodinger equation, where we don't actually write out the Hamiltonian operator, but you remember that's a series of second derivatives, so we have a differential equation that were actually dealing with.
If you think about what happens when we go from hydrogen to helium, now instead of one electron, so three position variables, we have to describe two electrons, so now we have six position variables that we need to plug into our Schrodinger equation.
So similarly, as we now move up only one more atom in the table, so to an atomic number of three or lithium, now we're going from six variables all the way to nine variables.
So you can see that we're starting to have a very complicated equation, and it turns out that it's mathematically impossible to even solve the exact Schrodinger equation as we move up to higher numbers of electrons.
So, what we say here is we need to take a step back here and come up with an approximation that's going to allow us to think about using the Schrodinger equation when we're not just talking about hydrogen or one electron, but when we have these multi-electron atoms.
The most straightforward way to do this is to make what's called a one electron orbital approximation, and when you do you get out what are called Hartree orbitals, and what this means is that instead of considering the wave function as a function, for example, for helium as six different variables, what we do is we break it up and treat each electron has a separate wave function and say that our assumption is that the total wave function is equal to the product of the two individual wave functions.
So, for example, for helium, we can break it up into wave function for it the r, theta, and phi value for electron one, and multiply that by the wave function for the r, theta, and phi value for electron number two.
So essentially what we're saying is we have a wave function for electron one, and a wave function for electron two.
We know how to write that in terms of the state numbers, so it would be 1, 0, 0, because we're talking about the ground state.
We're always talking about the ground state unless we specify that we're talking about an excited state.
And we have the spin quantum number as plus 1/2 for electron one, and minus 1/2 for the electron two.
It's arbitrary which one I assigned to which, but we know that we have to have each of those two magnetic spin quantum numbers in order to have the distinct four letter description of an electron.
We know that it's not enough just to describe the orbital by three quantum numbers, we need that fourth number to fully describe an electron.
And when we describe this in terms of talking about chemistry terminology, we would call the first one the 1 s, and 1 is in parentheses because we're talking about the first electron there, and we would multiply it by the wave function for the second one, which is also 1 s, but now we are talking about that second electron.
We can do the exact same thing when we talk about lithium, but now instead of breaking it up into two wave functions, we're breaking it up into three wave functions because we have three electrons.
So, the first again is the 1 s 1 electron.
We then have the 1 s 2 electron, and what is our third electron going to be?
Yeah.
So it's going to be the 2 s 1 electron.
So we can do this essentially for any atom we want, we just have more and more wave functions that we're breaking it up to as we get to more and more electrons.
And we can also write this in an even simpler form, which is what's called electron configuration, and this is just a shorthand notation for these electron wave functions.
So, for example, again we see hydrogen is 1 s 1, helium we say is 1 s 2, or 1 s squared, so instead of writing out the 1 s 1 and the 1 s 2, we just combine it as 1 s squared, lithium is 1 s 2, 2 s 1.
So writing out electron configurations I realize is something that a lot of you had experience with in high school, you're probably -- many of you are very comfortable doing it, especially for the more straightforward atoms.
But what's neat to kind of think about is if you think about what a question might have been in high school, which is please write the electron configuration for lithium, now we can also answer what sounds like a much more impressive, and a much more complicated question, which would be write the shorthand notation for the one electron orbital approximation to solve the Schrodinger equation for lithium.
So essentially, that is the exact same thing.
The electronic configuration, all it is is the shorthand notation for that one electron approximation for the Schrodinger equation for lithium.
So, if you're at hanging your exam from high school on the fridge and you want to make it look more impressive, you could just rewrite the question as that, and essentially you're answering the same thing.
But now, hopefully, we understand where that comes from, why it is that we use the shorthand notation.
So, let's write this one electron orbital approximation for berylium, that sounds like a pretty complicated question, but hopefully we know that it's not at all, it's just 1 s 2, and then 2 s 2.
And we can go on and on down the table.
So, for example, for boron, now we're dealing with 1 s 2, then 2 s 2, and now we have to move into the p orbital so we go to 2 p 1.
So that's a little bit of an introduction into electron configuration.
We'll get into some spots where it gets a little trickier, a little bit more complicated later in class.
But that's an idea of what it actually means to talk about electron configuration.
So now that we can do this, we can compare and think about, we know how to consider wave functions for individual electrons in multi-electron atoms using those Hartree orbitals or the one electron wave approximations.
So let's compare what some of the similarities and differences are between hydrogen atom orbitals, which we spent a lot of time studying, and now these one electron orbital approximations for these multi-electron atoms.
So, as an example, let's take argon, I've written up the electron configuration here, and let's think about what some of the similarities might be between wave functions in argon and wave functions for hydrogen.
So the first is that the orbitals are similar in shape.
So for example, if you know how to draw an s orbital for a hydrogen atom, then you already know how to draw the shape of an s orbital or a p orbital for argon.
Similarly, if we were to look at the radial probability distributions, what we would find is that there's an identical nodal structure.
So, for example, if we look at the 2 s orbital of argon, it's going to have the same amount of nodes and the same type of nodes that the 2 s orbital for hydrogen has.
So how many nodes does the 2 s orbital for hydrogen have?
It has one node, right, because if we're talking about nodes it's just n minus 1 is total nodes, so you would just say 2 minus 1 equals 1 node for the 2 s orbital.
And how many of those nodes are angular nodes?
zero.
L equals 0, so we have zero angular nodes, that means that they're all radial nodes.
So what we end up with is one radial node for the 2 s orbital of hydrogen, and we can apply that for argon or any other multi-electron atom here, we also have one radial node for the 2 s orbital of argon.
But there's also some differences that we need to keep in mind, and that will be the focus of a lot of the lecture today.
One of the main difference is is that when you're talking about multi-electron orbitals, they're actually smaller than the corresponding orbital for the hydrogen atom.
We can think about why that would be.
Let's consider again an s orbital for argon, so let's say we're looking at the 1 s orbital for argon.
What is the pull from the nucleus from argon going to be equal to?
What is the charge of the nucleus?
Does anyone know, it's a quick addition problem here.
Yeah, so it's 18.
So z equals 18, so the nucleus is going to be pulling at the electron with a Coulombic attraction that has a charge of plus 18, if we're talking about the 1 s electron or the 1 s orbital in argon.
It turns out, and we're going to get the idea of shielding, so it's not going to actually feel that full plus 18, but it'll feel a whole lot more than it will just feel in terms of a hydrogen atom where we only have a nuclear charge of one.
So because we're feeling a stronger attractive force from the nucleus, we're actually pulling that electron in closer, which means that the probability squared of where the electron is going to be is actually a smaller radius.
So when we talk about the size of multi-electron orbitals, they're actually going to be smaller because they're being pulled in closer to the nucleus because of that stronger attraction because of the higher charge of the nucleus in a multi-electron atom compared to a hydrogen atom.
The other main difference that we're really going to get to today is that in multi-electron atoms, orbital energies depend not just on the shell, which is what we saw before, not just on the value of n, but also on the angular momentum quantum number.
So they also depend on the sub-shell or l. And we'll really get to see a picture of that, and I'll be repeating that again and again today, because this is something I really want everyone to get firmly into their heads.
So, let's now take a look at the energies.
We looked at the wave functions, we know the other part of solving the Schrodinger equation is to solve for the binding energy of electrons to the nucleus, so let's take a look at those.
And there again is another difference between multi-electron atom and the hydrogen atoms.
So when we talk about orbitals in multi-electron atoms, they're actually lower in energy than the corresponding h atom orbitals.
And when we say lower in energy, of course, what we mean is more negative.
Right, because when we think of an energy diagram, that lowest spot there is going to have the lowest value of the binding energy or the most negative value of binding.
So, let's take a look here at an example of an energy diagram for the hydrogen atom, and we can also look at a energy diagram for a multi-electron atom, and this is just a generic one here, so I haven't actually listed energy numbers, but I want you to see the trend.
So for example, if you look at the 1 s orbital here, you can see that actually it is lower in the case of the multi-electron atom than it is for the hydrogen atom.
You see the same thing regardless of which orbital you're looking at.
For example, for the 2 s, again what you see is that the multi-electron atom, its 2 s orbital is lower in energy than it is for the hydrogen.
The same thing we see for the 2 p. Again the 2 p orbitals for the multi-electron atom, lower in energy than for the hydrogen atom.
But there's something you'll note here also when I point out the case of the 2 s versus the 2 p, which is what I mentioned that I would be saying again and again, which is when we look at the hydrogen atom, the energy of all of the n equals 2 orbitals are exactly the same.
That's what we call degenerate orbitals, they're the same energy.
But when we get to the multi-electron atoms, we see that actually the p orbitals are higher in energy than the s orbitals.
So we'll see specifically why it is that the s orbitals are lower in energy.
We'll get to discussing that, but what I want to point out here again is the fact that instead of just being dependent on n, the energy level is dependent on both n and l. And is no longer that sole determining factor for energy, energy also depends both on n and on l. And we can look at precisely why that is by looking at the equations for the energy levels for a hydrogen atom versus the multi-electron atom.
So, for a hydrogen atom, and actually for any one electron atom at all, this is our energy or our binding energy.
This is what came out of solving the Schrodinger equation, we've seen this several times before that the energy is equal to negative z squared times the Rydberg constant over n squared.
Remember the z squared, that's just the atomic number or the charge on the nucleus, and we can figure that out for any one electron atom at all.
And an important thing to note is in terms of what that physically means, so physically the binding energy is just the negative of the ionization energy.
So if we can figure out the binding energy, we can also figure out how much energy we have to put into our atom in order to a eject or ionize an electron.
We can also look at the energy equation now for a multi-electron atom.
And the big difference is right here in this term.
So instead of being equal to negative z squared, now we're equal to negative z effective squared times r h all over n squared.
So when we say z effective, what we're talking about is instead of z, the charge on the nucleus, we're talking about the effective charge on a nucleus.
So for an example, even if a nucleus has a charge of 7, but the electron we're interested in only feels the charge as if it were a 5, then what we would say is that the z effective for the nucleus is 5 for that electron.
And we'll talk about this more, so if this is not completely intuitive, we'll see why in a second.
So the main idea here is z effective is not z, so don't try to plug one in for the other, they're absolutely different quantities in any case when we're not talking about a 1 electron atom.
And the point that I also want to make is the way that they differ, z effective actually differs from the total charge in the nucleus due to an idea called shielding.
So, shielding happens when you have more than one electron in an atom, and the reason that it's happening is because you're actually canceling out some of that positive charge from the nucleus or that attractive force with a repulsive force between two electrons.
So if you have some charge in the nucleus, but you also have repulsion with another electron, the net attractive charge that a given electron going to feel is actually less than that total charge in the nucleus.
And shielding is a little bit of a misnomer because it's not actually that's the electron's blocking the charge from another electron, it's more like you're canceling out a positive attractive force with a negative repulsive force.
But shielding is a good way to think about it, and actually, that's what we'll use in this class to sort of visualize what's happening when we have many electrons in an atom and they're shielding each other.
Shielding is the term that's used, it brings up a certain image in our mind, and even though that's not precisely what's going on, it's a very good way to visualize what we're trying to think about here.
So let's take two cases of shielding if we're talking about, for example, the helium, a helium nucleus or a helium atom.
So what is the charge on a helium nucleus?
What is z?
Yup, so it's plus 2.
So the charge is actually just equal to z, we can write plus 2, or you can write plus 2 e, e just means the absolute value of the charge on an electron.
When we plug it into equations we just use the number, the e is assumed there.
So, let's think of what we could have if we have two electrons in a helium atom that are shielded in two extreme ways.
So, in the first extreme way, let's consider that our first electron is at some distance very far away from the nucleus, we'll call this electron one, and our second electron is, in fact, much, much closer to the nucleus, and let's think of the idea of shielding in more of the classical sense where we're actually blocking some of that positive charge.
So if we have total and complete shielding where that can actually negate a full positive charge, because remember our nucleus is plus 2, one of the electrons is minus 1, so if it totally blocks it, all we would have left from the nucleus is an effective charge of plus 1.
So in our first case, our first extreme case, would be that the z effective that is felt by electron number 1, is going to be plus 1.
So, what we can do is figure out what we would expect the binding energy of that electron to be in the case of this total shielding.
And remember again, the binding energy physically is the negative of the ionization energy, and that's actually how you can experimentally check to see if this is actually correct.
And that's going to be equal to negative z effective squared times r h over n squared.
So, let's plug in these values and see what we would expect to see for the energy.
So it would be negative 1 squared times r h all over 1 squared, since our z effective we're saying is 1, and n is also equal to 1, because we're in the ground state here so we're talking about a 1 s orbital.
So if we have a look at what the answer would be, this looks very familiar.
We would expect our binding energy to be a negative 2 .
1 8 times 10 to the negative 18 joules.
This is actually what the binding energy is for hydrogen atom, and in fact, that makes sense because in our extreme case where we have total shielding by the second electron of the electron of interest, it's essentially seeing the same nuclear force that an electron in a hydrogen atom would see.
All right.
Let's consider now the second extreme case, or extreme case b, for our helium atom.
Again we have the charge of the nucleus on plus 2, but let's say this time the electron now is going to be very, very close to the nucleus.
And let's say our second electron now is really far away, such that it's actually not going to shield any of the nuclear charge at all from that first electron.
So what we end up saying is that the z effective or the effective charge that that first electron feels is now going to be plus 2.
Again, we can just plug this into our equation, so if we write in our numbers now saying that z effective is equal to 2, we find that we get negative 2 squared r h, all divided again by 1 squared -- we're still talking about a 1 s orbital here.
And if we do that calculation, what we find out is that the binding energy, in this case where we have no shielding, is negative 8 .
7 2 times 10 to the negative 18 joules.
So, let's compare what we've just seen as our two extremes.
So in extreme case a, we saw that z effective was 1.
This is what we call total shielding.
The electron completely canceled out it's equivalent of charge from the nucleus, such that we only saw in a z effective of 1.
In an extreme case b, we had a z effective of 2, so essentially what we had was no shielding at all.
We said that that second electron was so far out of the picture, that it had absolutely no affect on what the charge was felt by that first electron.
So, we can actually think about now, we know the extreme cases, but what is the reality, and the reality is if we think about the ionization energy, and we measure it experimentally, we find that it's 3 .
9 4 times 10 to the negative 18 joules, and what you can see is that falls right in the middle between the two ionization energies that we would expect for the extreme cases.
And this is absolutely confirming that what is happening is what we would expect to happen, because we would expect the case of reality is that, in fact, some shielding is going on, but it's not going to be total shielding, but at the same time it's not going to be no shielding at all.
And if we experimentally know what the ionization energy is, we actually have a way to find out what the z effective will be equal to.
And we can use this equation here, this is just the equation for the ionization energy, which is the same thing as saying the negative of the binding energy that's equal to z effective squared r h over n squared.
So, what we can do instead of talking about the ionization energy, because that's one of our known quantities, is we can instead solve so that we can find z effective.
So, if we just rearrange this equation, what we find is that z effective is equal to n squared times the ionization energy, all over the Rydberg constant and the square root of this.
So the square root of n squared r e over r h. So what's our value for n here?
one.
Yup, that's right.
And then what's our value for ionization energy?
Yup.
So it's just that ionization energy that we have experimentally measured, 3 .
9 4 times 10 to the negative 18 joules.
We put all of this over the Rydberg constant, which is 2 .
1 8 times 10 to the negative 18 joules, and we want to raise this all to the 1/2.
So what we end up seeing is that the z effective is equal to positive 1 .
3 4.
So, this is what we find the actual z effective is for an electron in the helium atom.
Does this seem like a reasonable number?
Yeah?
Who says yes, raise your hand if this seems reasonable.
Does anyone think this seems not reasonable?
OK. How can we check, for example, if it does or if it doesn't seem reasonable.
Well, the reason, the way that we can check it is just to see if it's in between our two extreme cases.
We know that it has to be more than 1, because even if we had total shielding, we would at least feel is the effective of 1.
We know that it has to be equal to less than 2, because even if we had absolutely no shielding at all, the highest z effective we could have is 2, so it makes perfect sense that we have a z effective that falls somewhere in the middle of those two.
So, let's look at another example of thinking about whether we get an answer out that's reasonable.
So we should be able to calculate a z effective for any atom that we want to talk about, as long as we know what that ionization energy is.
And I'm not expecting you to do that calculation here, because it involves the calculator, among maybe a piece of paper as well.
But what you should be able to do is take a look at a list of answers for what we're saying z effective might be, and determining which ones are possible versus which ones are not possible.
So, why don't you take a look at this and tell me which are possible for a 2 s electron in a lithium atom where z is going to be equal to three?
Let's do 10 more seconds on that.
OK, great.
So, the majority of you got it right.
There are some people that are a little bit confused still on where this make sense, so, let's just think about this a little bit more.
So now we're saying that z is equal to 3, so if, for example, we had total shielding by the other two electrons, if they totally canceled out one unit of positive charge each in the nucleus, what we would end up with is we started with 3 and then we would subtract a charge of 2, so we would end up with a plus 1 z effective from the nucleus.
So our minimum that we're going to see is that the smallest we can have for a z effective is going to be equal to 1.
So any of the answers that said a z effective of .
3 9 or .
8 7 are possible, they actually aren't possible because even if we saw a total shielding, the minimum z effective we would see is 1.
And then I think it looks like most people understood that four was not a possibility.
Of course, if we saw no shielding at all what we would end up with is a z effective of 3.
So again, when we check these, what we want to see is that our z effective falls in between the two extreme cases that we could envision for shielding.
And again, just go back and look at this and think about this, this should make sense if you kind of look at those two extreme examples, so even if it doesn't make entire sense in the 10 seconds you have to answer a clicker question right now, make sure this weekend you can go over it and be able to predict if you saw a list of answers or if you calculate your own answer on the p-set, whether or not it's right or it's wrong, you should be able to qualitatively confirm whether you have a reasonable or a not reasonable answer after you do the calculation part.
All right.
So now that we have a general idea of what we're talking about with shielding, we can now go back and think about why it is that the orbitals are ordered in the order that they are.
We know that the orbitals for multi-electron atoms depend both on n and on l. But we haven't yet addressed why, for example, a 2 s orbital is lower in energy than the 2 p orbital, or why, for example, a 3 s orbital is lower in energy than a 3 p, which in turn is lower than a 3 d orbital.
So let's think about shielding in trying to answer why, in fact, it's those s orbitals that are the lowest in energy.
And when we make these comparisons, one thing I want to point out is that we need to keep the constant principle quantum number constant, so we're talking about a certain state, so we could talk about the n equals 2 state, or the n equals 3 state.
And when we're talking about orbitals in the same state, what we find is that orbitals that have lower values of l can actually penetrate closer to the nucleus.
This is an idea we introduced on Wednesday when we were looking at the radial probability distributions of p orbitals versus s orbitals versus d orbitals.
But now it's going to make more sense because in that case we were just talking about single electron atoms, and now we're talking about a case where we actually can see shielding.
So what is actually going to matter is how closely that electron can penetrate to the nucleus, and what I mean by penetrate to the nucleus is is there probability density a decent amount that's very close to the nucleus.
So, if we superimpose, for example, the 2 s radial probability distribution over the 2 p, what we see is there's this little bit of probability density in the 2 s, but it is significant, and that's closer to the nucleus than it is for the 2 p. And remember, this is in complete opposition to what we call the size of the orbitals, because we know that the 2 p is actually a smaller orbital.
For example, when we're talking about radial probability distributions, the most probable radius is closer into the nucleus than it is for the s orbital.
But what's important is not where that most probable radius is when we're talking about the z effective it feels, what's more important is how close the electron actually can get the nucleus.
And for the s electron, since it can get closer, what we're going to see is that s electrons are actually less shielded than the corresponding p electrons.
They're less shielded because they're closer to the nucleus, they feel a greater z effective.
We can see the same thing when we compare p electrons to d electrons, or p and d orbitals.
I've drawn the 3 p and the 3 d orbital here, and again, what you can see is that the p electron are going to be able to penetrate closer to the nucleus because of the fact that there's this bit of probability density that's in significantly closer to the nucleus than it is for the 3 d orbital.
And if we go ahead and superimpose the 3 s on top of the 3 p, you can see that the 3 s actually has some bit of probability density that gets even closer to the nucleus than the 3 p did.
So that's where that trend comes from where the s orbital is lower than the d orbital, which is lower than the d orbital.
So now that we have this idea of shielding and we can talk about the differences in the radial probability distributions, we can consider more completely why, for example, if we're talking about lithium, we write the electron configuration as 1 s 2, 2 s 1, and we don't instead jump from the 1 s 2 all the way to a p orbital.
So the most basic answer that doesn't explain why is just to say well, the s orbital is lower in energy than the p orbital, but we now have a more complete answer, so we can actually describe why that is.
And what we're actually talking about again is the z effective.
So that z effective felt by the 2 p is going to be less than the z effective felt by the 2 s. And another way to say this, I think it's easiest to look at just the fact that there's some probability density very close the nucleus, but what we can actually do is average the z effective over this entire radial probability distribution, and when we find that, we find that it does turn out that the average of the z effective over the 2 p is going to be less than that of the 2 s. So we know that we can relate to z effective to the actual energy level of each of those orbitals, and we can do that using this equation here where it's negative z effective squared r h over n squared, we're going to see that again and again.
And it turns out that if we have a, for example, for s, a very large z effective or larger z effective than for 2 p, and we plug in a large value here in the numerator, that means we're going to end up with a very large negative number.
So in other words a very low energy is what we're going to have when we talk about the orbitals -- the energy of the 2 s orbital is going to be less than the energy of the 2 p orbital.
Another way to say that it's going to be less, so you don't get confused with that the fact this is in the numerator here, there is that negative sign so it's less energy but it's a bigger negative number that gives us that less energy there.
All right, so let's go back to electrons configurations now that we have an idea of why the orbitals are listed in the energy that they are listed under, why, for example, the 2 s is lower than the 2 p. So now we can go back and think about filling in these electron configurations for any atom.
I think most and you are familiar with the Aufbau or the building up principle, you probably have seen it quite a bit in high school, and this is the idea that we're filling up our energy states, again, which depend on both n and l, one electron at a time starting with that lowest energy and then working our way up into higher and higher orbitals.
And when we follow the Aufbau principle, we have to follow two other rules.
One is the Pauli exclusion principal, we discussed this on Wednesday.
So this is just the idea that the most electrons that you can have in a single orbital is two electrons.
That makes sense because we know that every single electron has to have its own distinct set of four quantum numbers, the only way that we can do that is to have a maximum of two spins in any single orbital or two electrons per orbital.
We also need to follow Hund's rule, this is that a single electron enters each state before it enters a second state.
And by state we just mean orbital, so if we're looking at the p orbitals here, that means that a single electron goes in x, and then it will go in the z orbital before a second one goes in the x orbital.
This intuitively should make a lot of sense, because we know we're trying to minimize electron repulsions to keep things in as low an energy state as possible, so it makes sense that we would put one electron in each orbital first before we double up in any orbital.
And the third fact that we need to keep in mind is that spins remain parallel prior to adding a second electron in any of the orbitals.
So by parallel we mean they're either both spin up or they're both spin down -- remember that's our spin quantum number, that fourth quantum number.
And the reason for this comes out of solving the relativistic version of the Schrodinger equation, so unfortunately it's not as intuitive as knowing that we want to fill separate before we double up a degenerate orbital, but you just need to keep this in mind and you need to just memorize the fact that you need to be parallel before you double up in the orbital.
So, we'll see how this works in a second.
So let's do this considering, for example, what it would look like if we were to write out the electron configuration for oxygen where z is going to be equal to 8.
So what we're doing is filling in those eight electrons following the Aufbau principle, so our first electron is going to go in the 1 s, and then we have no other options for other orbitals that are at that same energy, so we put the second electron in the 1 s as well.
Then we go up to the 2 s, and we have two electrons that we can fill in the 2 s. And now we get the p orbitals, remember we want to fill up 1 orbital at a time before we double up, so we'll put one in the 2 p x, then one in the 2 p z, and then one in the 2 p y.
At this point, we have no other choice but to double up before going to the next energy level, so we'll put a second one in the 2 p x.
And I arbitrarily chose to put it in the 2 p x, we also could have put it in the 2 p y or the 2 p z, it doesn't matter where you double up, they're all the same energy.
So if we think about what we would do to actually write out this configuration, we just write the energy levels that we see here or the orbital approximations.
So if we're talking about oxygen, we would say that it's 1 s 2, then we have 2 s 2, and then we have 2 p, and our total number of electrons in the p orbitals are four.
So it's OK to not specify.
I want to point out, whether you're in the p x, the p y, or the p z, unless a question specifically asks you to specify the m sub l, which occasionally will happen, but if it doesn't happen you just write it like this.
But if, in fact, you are asked to specify the m sub l's, then we would have to write it out more completely, which would be the 1 s 2, the 2 s 2, and then we would say 2 p x 2, 2 p z 1, and 2 p y 1.
So again, in general, just go ahead and write it out like this, but if we do ask you to specify you should be able to know that the p orbital separates into these three -- the p sub-shell separates into these three orbitals.
So let's do a clicker question on assigning electron configurations using the Aufbau principle.
So why don't you go ahead and identify the correct electron configuration for carbon, and I'll tell you that z is equal to 6 here.
And in terms of doing this for your homework, I actually want to mention that in the back page of your notes I attached a periodic table that does not have electron configurations on them.
It's better to practice doing electron configurations when you cannot actually see the electron configurations.
And this is the same periodic table that you're going to get in your exams, so it's good to practice doing your problem-sets with that periodic table so you're not relying on having the double check right there of seeing what the electron configuration is.
So, let's do 10 seconds on this problem here.
OK, great.
So this might be our best clicker question yet.
Most people were able to identify the correct electron configuration here.
Some people, the next most popular answer with 5%, which is a nice low number, wanted to put two in the 2 p x before they moved on.
Remember we have to put one in each degenerate orbital before we double up on any orbital, so just keep that rule in mind that we would fill one in each p orbital before we a to the second one.
But it looks like you guys are all experts here on doing these electron configurations.
So, let's move on to some more complicated electron configurations.
So, for example, we can move to the next periods in the periodic table.
When we talk about a period, we're just talking about that principle quantum number, so period 2 means that we're talking about starting with the 2 s orbitals, period 3 starts with, what we're now filling into the 3 s orbitals here.
So if we're talking about the third period, that starts with sodium and it goes all the way up to argon.
So if we write the electron configuration for sodium, which you can try later -- hopefully you would all get it correctly -- you see that this is the electron configuration here, 1 s 2, 2 s 2, 2 p 6, and now we're going into that third shell, 3 s 1.
And I want to point out the difference between core electrons and valence electrons here.
If we look at this configuration, what we say is all of the electrons in these inner shells are what we call core electrons.
The core electrons tend not to be involved in much chemistry in bonding or in reactions.
They're very deep and held very tightly to the nucleus, so we can often lump them together, and instead of writing them all out separately, we can just write the equivalent noble gas that has that configuration.
So, for example, for sodium, we can instead write neon and then 3 s 1.
So the 3 s 1, or any of the other electrons that are in the outer-most shell, those are what we call our valence electrons, and those are where all the excitement happens.
That's what we see are involved in bonding.
It makes sense, right, because they're the furthest away from the nucleus, they're the ones that are most willing to be involved in some chemistry or in some bonding, or those are the orbitals that are most likely to accept an electron from another atom, for example.
So the valence electrons, those are the exciting ones.
We want to make sure we have a full picture of what's going on there.
So, no matter whether or not you write out the full form here, or the noble gas configuration where you write ne first or whatever the corresponding noble gas is to the core electrons, we always write out the valence electrons here.
So for sodium, again, we can write n e and then 3 s 1.
We can go all the way down, magnesium, aluminum, all the way to this noble gas, argon, which would be n e and then 3 s 2, 3 p 6.
Now we can think about the fourth period, and the fourth period is where we start to run into some exceptions, so this is where things get a teeny bit more complicated, but you just need to remember the exceptions and then you should be OK, no matter what you're asked to write.
So for the fourth period, now we're into the 4 s 1 for potassium here.
And what we notice when we get to the third element in and the fourth period is that we go 4 s 2 and then we're back to the 3 d's.
So if you look at the energy diagram, what we see is that the 4 s orbitals are actually just a teeny bit lower in energy -- they're just ever so slightly lower in energy than the 3 d orbitals.
You can see that as you fill up your periodic table, it's very clear.
But also we'll tell you a pneumonic device to keep that in mind, so you always remember and get the orbital energy straight.
But it just turns out that the 4 s is so low in energy that it actually surpasses the 3 d, because we know the 3 d is going to be pretty high in terms of the three shell, and the 4 s is going to be the lowest in terms of the 4 shell, and it turns out that we need to fill up the 4 s before we fill in the 3 d. And we can do that just going along, 3 d 1, 2 3, and the problem comes when we get to chromium here, which is instead of what we would expect, we might expect to see 4 s 2, 3 d 4.
What we see is that instead it's 4 s 1, and 3 d 5.
So this is the first exception that you need to the Aufbau principle.
The reason this an exception is because it turns out that half filled d orbitals are more stable than we could even predict.
You wouldn't be expected to be able to guess that this would happen, because using any kind of simple theory, we would, in fact, predict that this would not be the case, but what we find experimentally is that it's more stable to have half filled d orbital than to have a 4 s 2, and a 3 d 4.
So you're going to need to remember, so this is an exception, you have to memorize.
Another exception in the fourth period is in copper here, we see that again, we have 4 s 1 instead of 4 s 2.
This is 4 s 1, 3 d 10, we might expect 4 s 2, 3 d 9, but again, this exception comes out of experimental observation, which is the fact that full d orbitals also are lower in energy then we could theoretically predict using simple calculations.
So again, you need to memorize these two exceptions, and the exception in general is that filled d 10, or half-filled d 5 orbitals are lower in energy than would be expected, so we got this flip-flip where if we can get to that half filled orbital by only removing one s electron, then we're going to do it, and the same with the filled d orbital.
And actually, when we get to the fifth period of the periodic table, that again takes place, so when you get to a half filled, or a filled d orbital, again you want to do it, so those exceptions would be with molybdenum and silver would be the corresponding elements in the fifth period where you're going to see the same case here where it's lower in energy to have the half filled or the filled d orbitals.
So here's the pneumonic I mentioned for writing the electron configuration and getting those orbital energies in the right order.
All you do is just write out all the orbitals, the 1 s, then the 2 s 2 p 3, 3 s 3 p d, just write them in a straight line like this, and then if you draw diagonals down them, what you'll get is the correct order in terms of orbital energies.
So if we go down the diagonal, we start with 1 s, then we get 2 s, then 2 p and 3 s, then 3 p, and 4 s, and then that's how we see here that 4 s is actually lower in energy than 3 d, then 4 p, 5 s and so on.
So if you want to on an exam, you can just write this down quickly at the beginning and refer to it as you're filling up your electron configurations, but also if you look at the periodic table it's very clear as you try to fill it up that way that the same order comes out of that.
So, whichever works best for you can do in terms of figuring out electron configurations.
So the last thing I want to mention today is how we can think about electron configurations for ions.
It turns out that it's going to be a little bit different when we're talking about positive ions here.
We need to change our rules just slightly.
So what we know is that these 3 d orbitals are higher in energy than 4 s orbitals, so I've written the energy of the orbital here for potassium and for calcium.
But what happens is that once a d orbital is filled, I said the two are very close in energy, and once a d orbital is filled, it actually drops to become lower in energy than the 4 s orbital.
So once we move past, we fill the 4 s first, but once we fill in the d orbital, now that's going to be lower in energy.
So that doesn't make a difference for us when we're talking about neutral atoms, because we would fill up the 4 s first, because that's lower in energy until we fill it, and then we just keep going with the d orbitals.
So, for example, if we needed to figure out the electron configuration for titanium, it would just be argon then 4 s 2, and then we would fill in the 3 d 2.
So, actually we don't have to worry about this fact any time we're dealing with neutrals.
The problem comes when instead we're dealing with ions.
So what I want to point out is what we said now is that the 3 d 2 is actually lower in energy, so if we were to rewrite this in terms of what the actual energy order is, we should instead write it 3 d 2, 4 s 2.
So you might ask in terms of when you're writing electron configurations, which way should you write it.
And we'll absolutely accept both answers for a neutral atom.
They're both correct.
In one case you decided to order in terms of energy and in one case you decided to order in terms of how it fills up.
I don't care how you do it on exams or on problem sets, but you do need to be aware that the 3 d once filled is lower in energy than the 4 s, and the reason you need to be aware of that is if you're asked for the electron configuration now of the titanium ion.
So, let's say we're asked for the plus two ion.
So a plus two ion means that we're removing two electrons from the atom and the electrons that we're going to remove are always going to be the highest energy electrons.
So it's good to write it like this because this illustrates the fact that in fact the 4 s electrons are the ones that are higher in energy.
So the correct answer for titanium plus two is going to be argon 3 d 2, whereas if we did not rearrange our order here we might have been tempted to write as 4 s 2 so keep that in mind when you're doing the positive ions of corresponding atoms.
Alright, so we'll pick up with photoelectron spectroscopy on Wednesday.
Have a great weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Let's get started.
Can you go ahead and take 10 more seconds on this first clicker question here?
OK.
So it looks like most of you got that the electron configuration that we're writing here is for copper.
So I'm actually going to give the benefit of the doubt that the people that didn't get it right were rushing to get out their clickers and didn't have time to think it all the way through.
Remember that when we're talking about 4 s 1, 3 d 10, that's one of those exceptions where a completely filled d orbital is more stable than we would expect.
So, that's actually the electron configuration we have when we're talking about copper and some other exceptions in the periodic table that you're going to be looking at.
So, hopefully, if you were to go back and look you could see that this is, in fact, copper.
We're actually going to do one more clicker question to get started with today, and as we do, I'll explain something we're going to be trying today, which is a little bit of a friendly competition in terms of answering the clicker questions correctly.
So we've tagged each of your numbers to your actual recitation, so we can see today which recitation actually is going to be doing the best in terms of clicker questions, who's going to get the most correct today.
So, you may or may not know this about your TAs, but this is a pretty competitive group of TAs we have this year, and they like to brag about how smart their recitation is, how good questions they're getting in the recitation section.
So, do your TA proud today and see if you can be part of the recitation that gets the most correct in terms of a percentage.
And at the end of class we'll announce which recitation that is, we'll also make sure to give you a little bit of a prize if you are, in fact, in that recitation.
So we have extra incentive to get these clicker questions right.
So, in this one we're selecting the correct electronic configuration for an ion.
So, why don't you go ahead and take 10 more seconds on this second clicker question for our intro.
OK.
So, it looks like we have a little bit of a mixed consensus here.
Let's go over this question.
And I know there's a lot to talk about about this competition, but let's just get into listening mode here and talk about how we can figure out what the correct electron configuration is for this ion.
Remember, ions are a little bit different.
The first thing we need to do is write the electron configuration for the atom itself, and then we need to take an electron away.
So here we're talking about v plus 1, so if we were to write it just for the neutral electron itself, we would find that the electron configuration is argon, that's the filled shell in front of it.
Then 4 s 2 and 3 d 3.
So this would be for the actual filled, the completely neutral atom.
But remember what we said, which was when we talked about, this is at the end of class on Friday, we said that it turns out that even though 3 d is higher in energy when it's not filled, once we fill it with an electron, these 2 orbitals actually switch place in terms of energy.
So if we were to write this in terms of energy, we would actually have to rewrite it has 3 d 3, and then 4 s 2.
So, which orbital would we take an electron out of if we were ionizing this atom here?
The s. So, we would actually take an electron out of the s, which gives us 3 d 3 and then 4 s 1.
So, it's a little bit of a trick when you're dealing with ions.
The best suggestion is just to write it out completely for the neutral atom, and then you want to take an electron out of the highest orbital.
It makes sense that it's going to come out of the highest occupied atomic orbital, because that's going to be the lowest amount of energy that's required to actually eject an electron.
All right.
So let's go to today's notes.
And actually before we start into today's topics, I want to remind everyone and hopefully you all do remember that our first exam is coming up and it's coming up in exactly a week, so it'll be a week from today, next Wednesday.
And on Friday in class, at the beginning of class, I'll go over just in all the detail you could possibly imagine everything you need to know logistically for the exam -- where it is, what you do, what kind of calculators you can bring, which by the way are any calculator.
So you'll get all of that information on Friday.
So don't worry if you have some questions right now.
I just want to let you know that.
The other thing I want to let you know is that instead of having a new problem-set that you'll be assigned this Friday, what we'll do instead is we'll give you some practice problems, and these will be just more of the same type of problems that you saw before but that's another chance to try them out more.
These won't be graded, you don't have to turn them in, it's just to give you some extra practice if you want while you're studying for the exam.
We'll also post an exam from a previous year so you can actually see exactly what the format's going to look like.
So when you go into the exam a week from today, it'll all look really familiar, you'll be comfortable with the format and you can just dive right in and start answering the questions.
So you'll have all that information and we'll get it to you on Friday.
The other quick thing I want to say is that I do have office hours today from 3 to 5, so feel free to stop by if you have questions about problem-set 3 that you're finishing up.
And also, for those of you that did sign up for the pizza forum tonight, that's going to be at 5 o'clock, it's in room 56-502, so we'll see some of you tonight for that as well.
All right.
So, let's move on to what we're talking about today.
What we're going to start with is discussing photoelectron spectroscopy, which is a spectroscopy technique that will give us some information about energy levels in multielectron atoms.
We'll then take a turn to talking about the periodic table, we'll look at a bunch of periodic trends, including ionization energy, electron affinity, electronegativity and atomic radius.
And then, if we have time at the end, we'll introduce one last topic, which is isoelectronic atoms and ions.
I also want to note that the end of the material today, so this last topic here, that's the end of the material that's going to be on this first exam.
So whether we finish it today, or more likely when we finish it up on Friday, once we get passed isoelectronic atoms, that's it, that's all you need to study for this first exam.
So from that point on it'll be exam 2 material, so depending on how you like to come compartmentalize your information, you can separate that in your brain in terms of what you're trying to learn right now versus what you can put off until a little bit later.
So, let's start with talking about photoelectron spectroscopy.
This actually relates very closely to what we discussed in class on Friday before the long weekend, and what we were talking about is the energy levels of multielectron atoms.
So what we'll start with today is talking about the technique that's primarily used to actually experimentally figure out what these different energy levels are.
And this is called photoelectron spectroscopy, and essentially what it is is very similar conceptually to what we were talking about way back in the first couple lectures when we were talking about the photoelectric effect.
Because here what we have is some atom that we're studying, in the case, it's going to be a gas, and we hit it with a photon that has some incident energy.
So e sub i, some energy that the photon comes in with, and if it has sufficient energy to eject an electron, it will do that, and our electron will be ejected with a certain kinetic energy, which is going to be whatever energy is left over from the initial energy we put in minus what was taken up in order to actually ionize or eject the electron.
So, you can see how this can directly give us different ionization energies for any atom that we're interested in studying.
For example, with neon we can think about all of the different orbital energies we could be looking at.
In the first case, so here is the electron configuration of neon.
So we can think about what is our most loosely-bound electron, what's that highest energy orbital, and it's going to be the 2 p orbital, that's going to be what's highest in energy.
So if we're going to eject an electron using a minimum amount of energy, that's where it's going to come from.
So, you can imagine, that we'll actually probably have a lot of kinetic energy left over if we put a lot of energy in in the first place.
We're only using up a little bit to eject the electron, then we'll have a lot left over.
So, one difference between photoelectron spectroscopy and, for example, the photoelectric effect is that in this case, we're not just looking at one energy level, which is what we were looking at from the surface of a metal, now we're talking about this gaseous atom.
So we can actually pop an electron or eject an electron from any single orbital that is occupied within the atom.
So, for example, it's not just the 2 p that we could actually take an electron from, we could also think about ejecting an electron from the 2 s orbital.
Now this, of course, is going to take more energy because a 2 s is lower, it has has a more negative binding energy than the 2 p, but that's OK as long as we put in enough energy, but what we're going to find is the kinetic energy coming out with the electron is actually going to be a little bit less, right, because we had to use up more energy to eject the electron, so we don't have as much left over.
There's actually one more orbital that we could talk about if we're talking about this sample case of neon, which is a 1 s orbital, and if we're talking about a 1 s orbital, now we're going to be even lower in energy still, so that means the minimum energy required to eject an electron is going to be at its highest, so that means the energy that we have left over that turns into kinetic energy for the electron, is now going to be really quite small.
And what happens when you irradiate one of these atoms that you're studying with this light is in photoelectron spectroscopy, you want to make sure that you put in enough energy to actually ionize any single electron that you have in the atom.
So the way that we really make sure this is done is that we use x-rays.
So you know that x-rays are higher frequency than UV light, for example, that means it's also higher energy than UV light, and if you think back to our photoelectric effect experiments, do you remember what type of light we were usually using for those?
Does anyone remember?
Yeah.
It was UV light that we used.
Well, we can't guarantee with UV light we'll have enough energy to eject every single electron, so that's why when we use x-rays, they're higher energy, you can pretty much be guaranteed we're going to eject all of those electrons there.
So I said that this technique was used to experimentally determine what the different binding energies or the different ionization energies are for the different states in a multielectron atom.
Another way to say states is to talk about different orbitals.
So we can do this directly as long as we have certain types of information.
The first that we need to know the energy of the photon that's incident on our gaseous atom.
The second piece of information we need to know is what actually the kinetic energy is of the ejected electron, and that's something we can just measure by measuring its velocity.
So we can use an equation to relate the incident energy and the kinetic energy to the ionization energy, or the energy that's required to eject an electron.
This should all sound incredibly familiar, like I'm just repeating myself in terms of photoelectric effect, because essentially that's what I'm doing, and that's one reason we spent so much time and did so many problem-set problems on the photoelectric effect.
So what we're saying here is the incident energy, so the energy coming in, is just equal to the minimum energy that's required to a eject an electron.
When we talked about the photoelectric effect, that was called the work function.
In this case, it's called the ionization energy, plus whatever kinetic energy we have left over in the electron.
So if we want to solve for ionization energy, we can just rearrange this equation.
Our ionization energy is going to be equal to the incident energy coming in, minus the kinetic energy of the electron.
So, let's take a look at the different kinetic energies that would be observed in a spectrum for neon where we had this incident energy here.
And it turns out that the first kinetic energy that we would see or the highest kinetic energy, would be 12 32 electron volts.
So if that's the case doing a quick little calculation, what would the ionization energy be for a 2 p electron in neon?
Yup, 22.
So, basically all we did was take 12 54, subtract 12 32, and we got 22 electron volts.
We can do the same thing for the other observed kinetic energy.
So, for example, in the second case, we say that we see 12 06 in terms of the kinetic energy.
Same sort of subtraction problem, what do we have for the ionization energy of the 2 s electron?
Good, quick math.
All right, so 48 electron volts.
And let's look at the final kinetic energy that we'd observe in this spectrum, which is 384 electron volts, so what is that third corresponding ionization energy?
I couldn't quite hear, but I have a feeling everyone said 870 electron volts.
So, we can actually kind of visualize what we would see if we were looking at a photoelectron spectrum.
And what we would see if we were graphing, for example, increasing kinetic energy, is we would see 1 line corresponding to each of these energies of electrons that we see coming out.
And, of course, each of those electrons correspond to an electron coming out of a particular orbital.
So in the case of 12 32, that is our highest kinetic energy, that means it's our lowest ionization energy -- it's the smallest amount of energy it takes to pop an electron out of that orbital.
So that's why we see the 2 p here, the 2 s is 12 06, and it makes sense that what we see as the greatest ionization energy, which is also the smallest kinetic energy is that 1 s orbital.
Remember, because that 1 s orbital is all the way down in terms of if we're thinking about an energy diagram, we're all the way down here, so we have a huge amount of energy we have to put into the system in order to eject an electron.
So what I want to point out when you're kind of looking at these numbers here, what the significance is, look at that huge difference between what the ionization energies are for what we call those valence electrons, those outer shell electrons, versus the ionization energy for the 1 s orbital -- those are core electrons there.
So we can think about something I mentioned last time, which is when we're thinking about chemistry and what's really interesting in terms of chemical reactions, it's mostly valence electrons we're talking about, those are the ones that tend to be involved in chemical reactions.
It makes a lot of sense when we look at it energetically, because if we think about a 1 s core electron, that's going to be held really, really tightly to the nucleus.
We see that we have to put this huge energy in to actually get a 1 s electron ejected, so it makes a lot of sense that we wouldn't want to pay that energy cost in a normal chemical reaction.
And we don't -- we very rarely would see these core electrons actually being involved in any type of a reaction.
All right, so one thing that I want to point out, which I said many, many times on Friday, and this is perhaps the last time I'll say it, but one last time is we can think about why we only see a line for the 2 p orbital, versus we don't see separate lines for a 2 p x, a 2 p y, and a 2 p z.
Remember, we need those three quantum numbers to completely describe the orbital.
Why do we just see one for all the p's?
And the reason is that the energy of the orbitals, depend on two quantum numbers, and that's quantum number n, and quantum number l. M does not actually have an effect, in this case, on the energy of the orbital.
So that's why we're not seeing separate lines in this spectrum.
All right.
So let's go ahead and try an example here in thinking about photoelectron spectroscopy.
So, let's say we're looking at an element and we have an emission spectra, and we know that it has five distinct different kinetic energies in that spectrum.
We might be asked, for example, to determine what all of the different elements could be that would produce a spectrum that gave us 5 different lines.
So the first thing that we want to do, if we're thinking about something like this, is just to determine exactly what orbitals are causing the five different lines that we're seeing in the spectrum.
So, if we're talking about five different orbitals and we're talking about a ground state atom, we know that we just need to start at the bottom and work our way out up.
So, our first orbital that an electron must be coming from is the 1 s. What comes after that?
2 s. All right, then what?
2 p. After that?
3 s. Next?
3 p, and that's 1, 2, 3, 4 -- that gives us five different options, five different orbitals, five different energies right there.
So, then all we need to do to determine which elements that corresponds to is take a look at our periodic table.
So we want to look at any element that has a 3 p orbital filled, but that does not then go on and have a 4 s, because if it had the 4 s filled then we would actually see six lines in the spectrum.
So that is relevant for all of these atoms here, so we actually have several different possibilities.
It could be aluminum, silicone, phosphorous, sulfur, chlorine or argon.
Any one of these different elements could actually produce a photoelectron spectroscopy spectrum that has five distinct lines.
If I went on and told you what the different incident light was, and what the electrons were ejected with, and then you could look up the ionization energy for the particular different elements, you should be able to actually determine exactly which element it is, but just with the information given, we can only narrow it down to these choices here.
So let's actually let you try another example of solving a problem that has to do with one of the spectrums.
So, let's turn to another clicker question here.
Remember, your answer holds great weight in terms of the state of the TA bragging for next week.
So, how many distinct, so again, we're talking about distinct kinetic energies, would be displayed if you're talking about a spectrum for the element hafnium, and I'll tell you here that it has a z of 72, so you don't have to spend two minutes searching your periodic table.
The period of table's on the back page of your notes if you don't see that there.
All right.
It looks like a lot of you are done, so let's take 10 more seconds here.
Part of the challenge is speed, too, how quickly you can get these answers in terms of getting them in on time.
So let's see what we say.
All right.
So I think I can safely say that most people had the right idea and were counting quickly, though I have a feeling that some people who wrote 13 might have forgotten about those 4 f, the 4 f electrons.
So, remember when you're looking at your periodic table, don't forget about the lanthinides, sometimes they come into play.
So it's actually 14, and the way that we got that answer was we just wrote out or just looked at your period table, figured out all of the different orbitals that you could have in terms of the principle quantum number, and then the l quantum number, and then write them all down -- it turns out to be 14, so that's what the answer is.
So, it looks like this is good, because we'll have some separation in terms of not everyone's going to get 100% in terms of recitations here, which is what we're going for.
All right.
So let's turn our attention to a new topic, which is thinking a little bit about the periodic table, and also talking about periodic trends.
And there's a lot we can explain by talking about what we see in the periodic table in terms of what different trends are in grouping different elements in different spots within the periodic table.
So, here we have a picture of Dmitri Mendeleev, who is one of the scientists responsible for first compiling the periodic table.
You'll notice I have what's a very flattering picture of him up here, and if you haven't done the reading yet you might not think this is particularly flattering, but if you look at the picture of him in the book, you'll notice I chose a very flattering picture of Dmitri up here, and here he's pondering putting these elements together in a periodic table.
And he actually did this in the late 1800's, back before even all of the elements that we know today were discovered, really only about 60% or so, 70% were discovered then that we now know today.
But still, he was able to put together a periodic table.
And what he did what he actually grouped things in terms of their chemical properties.
So the way that we like to think of things now is in terms of electron configurations, right, but at the time that wasn't really understood.
So, instead, it was amazing he was able to group things in terms of the properties that he saw.
So, for example, if he was talking about the group one metals, lithium, sodium, potassium -- he noticed these were all very soft reactive metals, those were grouped together.
Versus looking at, for example, helium or neon or argon, these are all inert gases, inert meaning essentially do not react, those were grouped together in the periodic table.
So basically, at the time he was just going on size and then traits, but what we actually know today is that we can also order things in the periodic table by electron configuration.
In fact, that is the most logical way for us to look at it now.
So, for example, if we're actually thinking about electron configuration and we look at lithium, sodium and potassium, these all have one valence electron.
So basically, that means one electron in an s orbital in their outer-most most shell.
So that explains why they're so reactive, they're all very willing to give up that 1 s orbital and then drop to a lower energy level.
In contrast, helium, neon, and argon all have filled shells.
That also explains why they're very stable.
They're not going to want to add on another electron, because then it'll have to jump a very large energy level and start filling in another shell -- go from n equals 2, to n equals 3, and n equals 4, and so on.
So it turns out that we can really know a lot just by looking at the periodic table.
You will never in this class have to memorize anything about the periodic table.
Depending on what kind of chemistry you go in to, you might accidentally memorize parts of the table, which is fine, but what you really want to know how to do is know how to use the periodic table.
But you actually need to keep a few caveats in mind as you do this, which is the fact that trends predict a lot of chemical properties, but they can't predict everything in terms of biological properties.
And after the periodic table was developed in the late 1800's, people didn't understand this quite as well, they took things a little more literally.
They thought, for example, if you could do something with one element, if you looked at an element very close to it, it would be similar enough that you could maybe replace it with that.
Today we know, for example, if you can put one certain kind of element in your mouth or eat that, it doesn't necessarily mean you want to put the element next to it and your mouth as well, that might not be safe.
But this is things we've learned as the years have gone past.
So, let's just take a quick example to show how not completely you can use these periodic trends, that there are limits.
So if we consider lithium, potassium, and sodium, they're all together in the same group on the periodic table, knowing what we do about biology we can immediately think of sodium and potassium, or even just knowing what you know about table salt, for example, that these are two elements that we find, and particularly in the ion form in very high concentrations in our body.
For example, sodium in our blood plasma is almost to the point sometimes of 100 millimol or that's very, very concentrated.
Similarly, we find it in table salt, we're taking it in all the time, the same with potassium, think of bananas, were always eating potassium.
Not so with lithium.
I don't think too many people and here are probably taking lithium.
It turns out there's actually no natural function known in the body for lithium.
So there's nothing naturally going on unless we were to introduce it ourselves in our body that we know of, at least, that involves lithium.
But this did not stop people, for example, in the late 1800's, early 1900's, and, in fact, in 1927 a new soft drink was put on to the market and they wanted to make a lemon-lime soft drink, these were very popular in the early 1900's, and to get sort of that lemony flavor, they decided to use citric acid, so that's a good idea, that gives that soury taste.
And they wanted to use a soluble salt of citric acid, so they could have used sodium, they could have used potassium.
But, you know why not do something a little special, little different, and they decided instead to use lithium.
So, here we have this soda with lithium citrate, some of you might be familiar with this, soda is called 7-Up.
So, 7-Up no longer has lithium in it, but from 1927 to 1950 it did, and, in fact, not only did they not try to hide the fact that there's lithium in the soda, this they used as a really special marketing technique, they really pointed out this is something that stands out about our soda, this is something special.
There's a lot of good things about lithium.
I don't know if you can see, probably not, what's written on here, so let me point out to you a few things.
Lithium, slenderizing, that's great to see in a soda.
Other nice things about lithium in your soda, it dispells hangovers, takes the ouch out of grouch.
That's very nice.
So basically, you get a lot of benefit supposedly from this 7-Up soda from the 1920's or so.
And this went on and was unregulated for some time, but at some point the Food and Drug Administration did take a step in, so here's a case where they did do something important -- that's not what I mean at all -- where they did take the step, they do many things that are important, often not quickly enough.
Here's a -- actually here it did take 25 years, but they did, they did eventually step on before we started drinking 7-Up.
And what they said was, look, you can't put this in, we're starting to notice it does some strange things.
Because it was in the 1950's or so, maybe the late 1940's, that people started to discover lithium, even though it had no natural function, it did do something in our bodies.
Does anyone know what was lithium's used for?
Yeah, it's an anti-psychotic drug, so, for example, some people with bipolar disorder even today still take it, it works really well for some people, for other people it doesn't work so well.
But anyway, this isn't really something you want to have in your soda, so they did take it out eventually.
Another side effect if you take too much lithium is death, so that's no good to have in sodas either, and it might not have been as big a deal back in the 1920's, but you can imagine with supersizing today, this might be a bigger problem.
So anyway, when we talk about periodic trends, it doesn't always match up.
This was eventually taken out, and actually just for your interest, there was no overlap between the time when cocaine was in Coca Cola and lithium was in 7-Up, so there was a few years difference between those two times, but it's amazing to think about what does go into processed foods.
And the other thing to point out, which I don't know if this is true or not, but does anyone know -- well that's part's true, does anyone know what the atomic mass of lithium is?
Yes, it's 7.
So, I don't know if this is true or not, but I wonder if that's where the actual name 7-Up came from.
So, even though we don't have the lithium anymore, we still keep that atomic number 7 around.
All right.
So that is an anti-example of using periodic trends.
So let's go to some actual real examples, which might come more in handy for this class.
So it's going to keep in mind the limitations, so let's start off with talking about ionization energy.
Now this is a good place to start, because we are very familiar with ionization energy, we've been talking about it in a lot of different forms for quite a while -- it's that minimum energy required to remove an electron from an atom.
And specifically, when we talk about ionization energy, it's assumed that what we mean is actually the first ionization energy.
So, you can imagine, we could talk about any of the different electrons, or we could talk about taking out an electron and taking out second electron.
Whenever you hear the term ionization energy, make sure you keep in mind that unless we say otherwise, we're talking about that first ionization energy.
And we know what that's equal to, this is something we've been over and over, ionization energy is simply equal to the negative of the binding energy.
So negative e, which is sub n l, because it's a function of n and l in terms of quantum numbers.
So, let's think about kind of differentiating, however, between first ionization energy or just ionization energy, and other types such as second or third ionization energy, and let's take boron as an example here.
So, if we want to think about what the first ionization energy is of boron, what you want to do is write out the electron configuration, because then you can think about where it is that the electron's coming out of.
The electron's going to come out of that highest occupied atomic orbital, that one that's the highest in energy, because that's going to be the at least amount of energy it needs to eject something.
So what we'll end up with is boron plus, 1 s 2, 2 s 2, and what we say is the delta energy or the change in energy as the same thing as saying the energy of the products minus the energy of our reactant here, and we just call that the ionization energy -- that's how much energy we have to put into the system to eject an electron.
And again, this is just the negative, the binding energy, when we're talking about the 2 p orbital.
So, this is first ionization energy, let's think about second ionization energy.
So, second ionization energy simply means you've already taken one electron out, now how much energy does it take for you to take a second electron out.
So in the case of boron here, what we're starting with is the ion, boron 1 s 2, 2 s 2, and now we're going to pull one more electron out.
The highest occupied orbital is now the 2 s orbital, so we're going to end up with boron 2 plus 1 s 2, 2 s 1, plus the electron coming out of there.
And what we say when we talk about the delta energy is that this is going to be equal to i e 2, or the second ionization energy, or we could say the negative of the binding energy of a 2 s electron in b plus.
so it's important to note that it's not in b, now we're talking about b plus, because we've already taken an electron out here.
So, similarly if we start talking about our third ionization energy, this is going to be going from b plus 2, to 1 s 2, 2 s 1.
Now we're going to pull that second electron out of the 2 s, so we end up with boron plus 3, and then the configuration is just 1 s 2, plus our extra electron here.
So, what we call this is the third ionization energy, or the negative of the binding energy, again of the 2 s orbital, but now it's in boron plus 2 to we're starting with.
So, this raises kind of an interesting question in terms of what the difference is between these two cases, and we're talking about numbers of energy.
So let's address this by considering another example, which should clarify what the difference is between these ionization energies.
So let's think about the energy required now to remove a 2 s electron, let's say we're removing it from boron plus 1 versus neutral boron.
So, in the case of boron plus 1, what we are starting with is the ion, so we're starting with a 2 s electron, and then we're going to 2 s 1 here.
And what we call the binding energy is negative 2 s in b plus -- this is what we saw on the last slide.
And the second case here looks a lot more like what we saw when we were talking about photoelectron spectroscopy, because here we want to remove a 2 s electron, but it's actually not the highest occupied orbital, so that's not the one that would naturally come out first, but let's say we're hitting it with high energy light sufficient to knock out all the different electrons, and one that we end up knocking out is this 2 s here.
So if we think about what that delta energy is, we call that the ionization of the 2 s, that's different from saying second ionization energy.
And that's going to be equal to the negative the binding energy of 2 s in b, in neutral boron.
So, my question to you is are these two energies equal?
No.
All right, good answer.
So, we can think about why is it that these are not equal.
In both cases we're taking an electron out of the 2 s orbital.
And it turns out that if we're talking about a 2 s orbital in an ion, that means it doesn't have as many electrons in it, so what we're going to see is less sheilding.
There are fewer electrons around to shield some of that nuclear charge.
So what we're going to see is less sheilding, which means that it will actually feel a higher z effective.
So even though they're both 2 s electrons, in one case it's going to think its feeling more pull from the nucleus, and it, in fact, will be, than in the other case, and if its feeling a higher z effective, then it's actually going to require more energy to remove that electron, right, it's being pulled in closer and more tightly to the nucleus, you have to put in more energy to rip it away from that very close interaction.
So, that's the difference in thinking about different types of ionization energy, so it can get a little bit confusing with terminology if you're just looking at something quickly, so make sure you look really carefully about what we're discussing here.
If you see a problem that asks you, for example, the third ionization energy versus taking a second electron out of the 2 s in a photoelectron spectroscopy experiment, those are two very different things.
So, let's make sure everyone kind of has this down, let's do another clicker question here.
And in this case we're going to look at silicone, and we'll say if you can point out to me which requires the least amount of energy.
So, which has the smallest energy that you have to put in in order to eject this electron?
Will it be if you take a 3 s electron from neutral silicone, if you take a 3 p electron from the neutral atom, or if you take a 3 p from the ion?
So this you should be able to know pretty quickly, so let's just take 10 seconds here.
All right, great.
So most of you see that, in fact, the energy that's going to be the least that we need to put in is in case 2 here.
Let's compare case 2 and 3, since this where some people seem to have gotten confused.
In case 2, we're taking it out of -- oh, it's kind of hard to compare case 2 and 3 when we can't see it anymore.
In case 2, we're taking the 3 p out of the neutral atom, whereas in case 3, we're taking it out of the ion.
Remember in the ion, we're going to have less electrons around to counteract the pull from the nucleus.
So we're going to feel a higher z effective in the case of the ion compared to the neutral atom.
If we have a higher z effective, it's pulled in tighter, we have to put in more energy in order to eject an electron, so it turns out that that's why case 2 is actually the lowest energy that we need to put in.
The z effective is lower, so we have to put less energy in to get an ion out.
So, let's take a look at this in terms of periodic trends -- that's our topic here, we're talking about periodic trends.
So as we go across the row, and this is my beautiful picture of a periodic table here.
As we go across the row what happens is that the ionization energy actually increases, and we can think about logically why it is that that's happening.
As we go across the row, what we have happening is that the z or the atomic charge -- I'm not talking about z effective here, I'm just talking about z -- the z is increasing as we go across a row, that's easy to see.
But we're still in the same shell, so we still have the same n value as we go all the way across a row in the periodic table.
So, in general what we're going to see is that what happens to z effective if we have z increasing but we're in the same shell here.
Would it increase or decrease z effective?
All right.
Kind of mixed thoughts here.
So it turns out that it increases, and the reason is because the predominant thing that's going on here is that z is increasing.
So the z is increasing, and what goes along with it is that the z effective is increasing, because it turns out that while we're in the same n, even though we know that energy depends on both the n and the l in terms of quantum numbers, while we're in the same n, the distance from the nucleus, it's pretty close, it's not hugely different.
So the factor that predominates is that we're actually increasing z.
That's why we see z effective increase, and that's why we see ionization energy increase.
As we go down a column, what happens is that the ionization energy decreases.
We can also think about this in terms of z effective.
This is because even though z, the atomic number is still increasing, we are also getting further away from the nucleus.
So, remember when we talk about Coulomb force, what's holding the nucleus and the electron together, there's 2 things we need to think about.
The first is this the z effective, or how much charge is actually in the nucleus that's felt, or the I guess we would say the z, how much the charge is on the nucleus that holds it close together.
But the second factor is how far away we are from the nucleus.
So, if we're really close to the nucleus, that's when z effective is high, but if we get really far away, then z effective is going to get low, because even though we have the same charge in the nucleus, the z effective gets lower.
So this is not even thinking about the other electron shielding, just if we think of the fact, all we need to think about is that the effect of going to a further away n actually dominates as we go down the table.
We're getting further away from the nucleus because we're jumping, for example, from the n equals 2 to the n equals 3 shell, or from the n equals 3 to the n equals 4 shell.
And when you're switching n's, you're actually getting quite a bit farther away.
That's why in the earlier models of the atom, they're not horrible to sometimes think about just each n value as a little ring around.
It's not complete and it's not accurate, but it's OK to kind of think of it in terms of how far we're getting away from the nucleus.
So, as we go down a column, we see ionization energy's going to decrease.
So, this means we have the general trends down, so we should be able to look at actual atoms in our periodic table and graph them and see that they match up with our trends.
So here we have that graphed here, we have atomic number z graphed against ionization energy, so, let's fill in what the actual atoms are here, and we can see in general, yes, we're following the trend.
For row one, we're increasing, as we should, across the row.
Let's look at row two also.
Well, we're generally increasing here, we'll talk about that more in a minute.
And also in a row three, yeah, we're generally increasing, there's some glitches here, but the general trend holds true.
Similarly we see as we go down the table, so as we're going from one row to the next row, so, for example, between helium and lithium, we see a drop; the same with neon to sodium, we see a drop here.
So it looks like we're generally following our trend, that's a good thing.
But hopefully, you will not be satisfied to just make a general statement here when we do have these glitches.
So you can see, for example, we have several places where instead of going up as we go across the row, we actually go down in ionization energy a little bit.
So between b e, and b, between n and o, magesium and aluminum, and then phosphorous and sulfur, what we see here is that we're kind of going down, or quite specifically, we are going down.
So, let's take a look at one of these rows in more detail to think about why this might be happening, and it turns out the reason that these glitches occur are because the sub shell structure predominates in certain instances, and that's where these glitches take place.
So I've sort of just spread out what we have as the second row here, graphed against the ionization energy.
So, let's consider specifically where these glitches are taking place.
So, let's look at the first one between beryllium and boron here.
And the glitch that doesn't make sense just through periodic trends, is that it turns out that the ionization energy of boron is actually less than the ionization energy up beryllium.
So I put the electron configurations and actually drew it on an energy diagram here, so we can actually think about why this might be happening.
So what is this, which element did I chart here?
Which one is that, the boron or the beryllium?
I couldn't tell what you said, sorry.
So, I'm going to assume that was beryllium, and then it turns out that if that's beryllium, the other one must be boron.
So, we have beryllium in the first case here, it has four electrons, that's how we know it's beryllium, boron has five electrons.
So, just looking at putting in the electrons, filling up the energy diagram here, we should be able to see a little bit why this is happening.
And the reason is simply because the energy that we gain in terms of moving up in z, so from going to z equals 4 to z equals 5, is actually outweighed by the energy it takes to go into this new shell, to go into this new sub shell.
So to jump from the 2 s to the 2 p, takes more energy than we can actually compensate with by increasing the pull from the nucleus.
So, it turns out that in this case, and any time that we see we're going from a 2 s to 2 p, filling in of electrons, we actually see that little bit of glitch in ionization energy.
So it's shown here for the second row, but it's actually also going to be true for the third row.
The same thing when you're going from filling in the 2 s to putting that first electron into the 2 p. So that explains one of our glitches here, but we have another glitch, and that second glitch comes between nitrogen and oxygen.
So, these sound more different, so I think I'll be able to distinguish.
Which element is shown here?
Yeah, nitrogen.
So, nitrogen is shown here, we know that because it has 7 electrons.
In this case, we're talking about 8 electrons, which is oxygen.
So if we're comparing the difference between these 2 now, what you'll notice is that in nitrogen we have all half-filled 2 p orbitals, and now, once we move into oxygen, we actually have to add 1 more electron into 1 of the 2 p orbitals.
There's no more 2 p orbitals to put it into, so we're going to actually have to double up.
So now we're putting 2 electrons into the same p orbital, that's not a problem, we can do it, it's not a huge energy cost to do that.
But actually there is a little bit of an energy cost into doubling up into a single orbital, because, of course, it takes energy when you create more electron repulsion, that's not something we want to do, but we have to do it here, and it turns out that that effect predominates over, again, the energy that we gain by increasing the atomic number by one.
So, our two glitches we see when we go from the 2 p, or from 2 s to start filling the 2 p, and then we also get another glitch when we've half-filled the 2 p, and now we're adding and having to double up in one of those p orbitals.
Again, we see the same effect as we go into different rows as well.
So let's talk about another periodic trend, this one is called electron affinity.
Electron affinity is actually the ability of an atom, or we could also talk about an ion to gain electrons.
So it's the affinity it has for electrons, it's how much it likes to get an electron.
We can write out what it is for any certain atom or ion x, so it's just x plus an electron gives us x minus.
We have the minus because we're adding a negative charge from the electron.
So, basically any time we have a really high positive number of electron affinity, it means that that atom or ion really wants to gain another electron, and it will be very stable and happy if it does so.
So let's look at an example of chlorine here.
So chlorine, if we talk about it in terms of electron affinity, we would be writing that we're actually gaining an electron here, and getting the ion, c l minus.
And the change in energy for this reaction is negative 349 kilojoules per mole.
So if we have a negative change in energy for any reaction as it's written, what that actually means is we're giving off energy as the reaction proceeds.
So, in other words, this c l minus is actually lower in energy than the reactants were.
So that's why we're giving off extra energy.
We saw a similar thing as we saw electrons move from different levels.
We can think of it in the same type of way when we're talking about actual reactions happening.
So, if we have energy that's released, would you say that the chlorine ion is more or less stable than the chlorine atom?
Who thinks it's more stable, show of hands?
All right, who thinks it's less stable?
Very good.
So it turns out it is, in fact, more stable.
It's more stable because you actually -- this happens spontaneously, you actually get energy out of the reaction as this happens.
And we haven't talked about reactions at all yet, so you don't need to worry about the specifics of that exactly, but just that if you have this negative change in energy, you have a more stable product than you do reactant.
So, we were talking, however, about energy in terms of electron affinity, so we can actually relate electron affinity to any reaction by saying if we have this reaction written as here where we're gaining an electron, we say that electron affinity is just equal to the negative of that change in energy.
So, for example, for the chlorine case, we would say that the electron affinity for chlorine is actually positive 349 kilojoules per mole.
That's a very large number, it's all relative, so you don't necessarily know it's large without me telling you or giving you other ions to compare to, but chlorine does have a very large affinity, meaning it's really likes getting an electron and becoming a chlorine ion.
One major difference between electron affinity and ionization energy is that when we talked about ionization energy, remember ionization energy always has to be positive.
We will never have a case where ionization energy is negative.
Electron affinity, however, can be either negative or it can be positive.
So let's look at a case where it's actually going to be negative.
So, if we took the case of nitrogen, if we add an electron to nitrogen and go to n minus, we find that the change in energy is 7 kilojoules per mole.
This means in order to do that we actually have to put 7 kilojoules per mole of energy into the reaction to make it happen.
So this is not going to be a favorable process, we're going to find that the electron affinity is actually a negative 7 kilojoules per mole for nitrogen.
So this means nitrogen has low electron affinity, it doesn't actually want to gain an electron.
So, that also tells us that the n minus ion is less stable than the neutral atom itself.
So, we can think about trends in electron affinity just like we did for ionization energy, and what we see is a similar trend.
Electron affinity increases as we go across a row in the periodic table, and it decreases as we go down a column.
I left out the noble gases here because they do something a little bit special, and actually, I'm going to give you one last clicker question today to see if you can tell me what you think noble gases do.
To answer this question you just really want to think about what does electron affinity means.
It means how much a certain atom actually wants to get an electron.
So do you think noble gases would have a high positive electron affinity, a low positive, or negative electron affinity?
So, let's take 10 seconds on that.
All right.
Great.
So most of you recognize, if we switch back to the notes, that they do have a negative electron affinity.
This should make sense to you, because they don't, in fact, want to gain another electron, because that would mean that electron would have to go into a new value of n, a new shell, and that's really going to increase the energy of the system.
So they'd much rather just stay the way they are and not have another electron come on, and it turns out that halogens have the highest electron affinities.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
It's 12:05, so why don't you go ahead and take 10 more seconds on the clicker question today.
This is about the periodic trends that we discussed on Wednesday.
So specifically, what we're asking here is as we go across the periodic table, we want to consider which has the smaller ionization energy.
All right.
So, let's focus our attention up here now, whether it's between aluminum or whether it's between phosphorous.
And I also wanted you to identify why it's also important to understand why we have these trends, not just to memorize the trend itself.
So, it turns out that the majority of you got the correct answer, which is that it's aluminum.
The reason it's aluminum is because aluminum has a lower z effective, so it's not being pulled in as tightly by the nucleus, and if it's not being pulled in as tightly, you're going to have to put in less energy in order to ionize it, so that's why it's actually going to have the smaller ionization energy.
So it looks like not too many more than half of you got this correct, so make sure you can look at your periodic table and figure out how to think about ionization energy in terms of z effective, not just in terms of memorizing what that trend is.
All right, so we can go to today's notes, and in terms of the notes, what we're going to start with is finishing material that's going to be relevant for exam 1, and I told you on Wednesday that actually I'd give you some information today in terms of what you need to do to prepare for exam 1.
So you should have gotten two handouts as you came in, and if you didn't, please raise your hand and a TA will come to you and get you that second handout.
But the second one says "exam instructions and logistics."
So, if everyone can pull that out.
So this is going to tell you pretty much everything you need to know in terms of getting ready for the exam, which is next Wednesday.
So when you go home today or some time this weekend, make sure you read this page in detail.
I'm just going to go over a few of the main points here.
So, the first is that what we're going over notes today and also on Monday, the exam material ends at the end lecture notes from lecture 9.
So that was Wednesday's class -- not at the end of Wednesday's class, but at the end of the lecture notes.
So we're going to finish up with those today.
I'll be really, really clear when we get through them, and that's where you can stop in terms of studying for this.
Also, on everything that was on problem-sets 1 through 3.
So, you turned in p-set 3 today, but we'll have the answers posted for you this afternoon, so you can start studying from p-set 3, even as early as tonight, if you want to, because those answers will be there for you.
So, in terms of what it is that you need to prepare and bring with you for the exam, you need to bring your MIT ID, especially if you haven't been showing up regularly to your recitations and you aren't 100% sure if your TA knows exactly who you are, you need to make sure you have your MIT ID with you.
You can't take the exam unless you're registered for the class, and we need to make sure we can verify that.
You also need to bring a calculator, of course, because we'll be solving problems that involve calculations.
You can bring any calculator you want, we don't actually have restrictions for calculator types here, but what you can't do, is you can't program any relevant chemical or information about constants in there.
It's OK to use certain fundamental constants that come in a lot of calculators, so there's nothing we can do about that.
That's OK.
If you're wondering what's OK or what's not OK, it's very clearly written out in this handout, so make sure you read through it, because it is your responsibility to make sure that your calculator does not have anything extra programmed in it.
And if you have your calculator all set up as you love and you don't want to change it, then maybe you should just go and get an $8.00 scientific calculator that doesn't have any of the graphing functions, because you don't actually need them, so that's a better option for you, you can do that as well.
And I had mentioned several times that you do not need to memorize the majority of the equations and you don't need to memorize any physical constants.
So if you flip the info page over on the back here, what you'll see is the periodic table, this is the same one that I've handed out in the last two lectures -- the periodic table without any electron configurations.
This is exactly the sheet here, it's exactly what you'll get on exam day.
You'll also see that they have all the physical constants that you're going to need, and also a bunch of the actual equations that we've been using in the first couple weeks here.
So you don't need to memorize any of this, you're actually going to be handed this.
There are a few equations that you need to memorize -- those are the very simple -- very, very simple equations, such as e equals h times nu -- hopefully you don't have to sit down and try to memorize that, hopefully we all know that already.
But just to be really clear, I've written out exactly which equations you do have to memorize on the front here, so long as you know those, the rest you can just look up.
And in terms of using these equations in solving problems on the exam, and also using these constants, make sure if you think there might be any chance you're going to get any little part of a problem wrong or do a calculation inaccurately, you need to write out every single step of your thinking as you write out these problems.
We can't give you any partial credit whatsoever if we can't see your thought process.
So it's important to write out the equation you use, you need to write out the constants that you use to fill in that equation.
And that we need to see your work to get full credit, and then especially if you get things wrong, we need to know where it went wrong, because we do try to give as much partial credit as possible in these exams, since there are a lot of places where small mistakes can result in the wrong answer.
So also, along those lines in terms of test taking, make sure you also box your answers and that you keep track of significant figures and that you also remember to include your units.
These are just little things that can add up, so you just want to make sure you're on top of those.
And your TAs on Tuesday are going to share a lot of other types of sort of exam strategies in thinking about how you can approach an exam when we're in a time situation like we are, so they'll share some of their experience with you in terms of taking these timed exams.
So, in terms of practicing this weekend, I mentioned that instead of getting a problem-set today, what I am going to be posting is optional extra problems.
So they're optional, but they're very, very, very highly encouraged that you do these, because this is going to give you practice for the types of problems that are going to be on the exam.
We're also posting a practice exam for you to take, so after you're completely done your studying, it's good to have everything done before you take the practice exam, and then sit down just with this sheet here and your calculator, and ideally a timer, and make sure you can do the practice exam in the allotted amount of time.
So that way you can have an idea if, oh, I do really understand this but I'm a little bit slow, maybe I need to practice this one type of problem a little bit longer so I can get up to speed so I'm going to be able to get through all this in terms of the exam time.
All right.
So let's move on to today's topics.
So, as I said, we're finishing up with what we left off with yesterday -- or excuse me, on Wednesday.
This includes atomic radius and the idea of isoelectronic atoms.
So that's going to be the end of the exam 1 material, and then we'll move on to exam 2 material, which is kind of exciting, because we've been talking about just individual atoms and ions up to this point, and now we can talk about molecules, so we're going to start talking about bonding.
So for some you that are less interested in maybe the physical structure of an individual atom, now some more exciting material for you might be coming up if you like to think about how, instead, molecules behave, either within bonding, within themselves, or with other molecules, that's what we're going to be heading to in this next unit.
So, we need to finish up with periodic trends.
And first, on your lecture notes, I start with atomic radius.
I was so proud of myself getting the lecture notes finished early and handing them in to CopyTech, and then I realized we didn't do electronegativity on Wednesday.
So, if you can flip your lecture notes over and just write on the blank space, we're going to cover electronegativity first here, and specifically, you can go back and fill this in to your lecture 9 notes, if you want to stay organized, but I just suggest just writing it on lecture 10 notes now and going back.
You can still keep organized, which hopefully most of you like to do, and get it in the right place in the notes.
So, when we're talking about the idea of electronegativity, essentially what we're talking about is the ability for an atom to attract electron density from another atom.
So it's just a measure of how much does one given atom want to pull away electron density from, let's say, an adjacent atom.
So it's actually very related to what we're talking about when we said electron affinity, and it's also related to ionization energy, and we can call electronegativity by symbol here, and it turns out that it's going to be proportional to 1/2 of the electron affinity of a given atom, plus the ionization energy.
So, in other words, we can just think of electronegativity as being the average of that ionization energy and the electron affinity.
This should make sense, because if an atom has a very high electron affinity, that means it's really happy taking an electron from another atom, or taking a free electron -- that that's very favorable.
If something has a high ionization energy, it means that it really, really, really does not want to give up an electron.
So you can think about how these 2 things combined are going to be electronegativity, which is a measure of how much an atom wants to pull electron density away from another atom.
So, if we think about electronegativity as a periodic trend, we can just draw our nice periodic table here, and let's separate it into quadrants.
So if we think about the upper right hand part of the quadrant, well, this is where we're going to have high electron affinity and high ionization energy, so we're also going to see high electronegativity here.
And in contrast, in the lower left hand part of the periodic table, these 2 quantities are low, so also what we're going to see is low electronegativity.
And if we talk about what's going on in areas, or with atoms that have high electronegativity, and we think about whether they're electron donors or electron acceptors, what would you expect for an atom that has high electronegativity?
Is it going to be an electron donor or acceptor?
Great.
Yup, it's going to be an electron acceptor, it wants to accept electrons, it wants to accept electron density.
So, in contrast, if it has a low electronegativity, this then is going to be an electron donor.
All right, so it's very common to talk about electronegativity of different atoms, and you can look up tables of these.
Often what you'll see is not a table based on this definition, but something that's called the Pauling definition of electronegativity, but it's exactly the same idea and the same trend as this more numerical way to think about what the meaning of electronegativity is.
All right, so now we can move on to the start of today's notes, which is atomic radius.
So this, in fact, is going to be our last principle that we're going to talk about in terms of periodic trends.
So this is actually the most straightforward, so sometimes it's nice to end with the easiest concept, and that's what we're doing here.
And if we're talking about atomic radius, essentially we're talking about atomic size.
And immediately it should probably come into your head that we don't actually have an atomic radius that we can talk about, right?
What I just spent many lectures discussing is the fact that we can not know how far away an electron is from the nucleus, so we can't actually know the radius of a certain atom.
And that's true.
An atom's not a defined sphere, for example.
We can't define it as an exact radius in terms of the definition we might think of classically.
So, keep that in mind when we're talking about atomic radius, I'm not suddenly changing my story and saying, yes, we do have a distinct radius.
Instead, what people have done is come up with different ways to think about how they can define a radius.
And one common way to think about it, is to think about the value of r, or the radius, below which 90% of that electron density is going to be contained.
So we're not saying it's all the electron density, it's just 90%.
Because we know as we go to infinity, even though the density gets smaller and smaller and smaller, we still have electron density very far away from the nucleus.
So, what we're going to define is just let's just capture 90% of that electron density.
So, that's one way to think about it, and there's also another way, and this is the way that your book presents it.
If you, in fact, have two of the same atom right next to each other, let's say you have a crystal, or let's say you're talking about a metal, what you can do is just look at the distance between the two nuclei, and split that in 1/2, and take the atomic radius that way.
So, these are two different definitions of how to think about atomic radius, but really what you find when these are measured is they come up with almost the identical values, so there are tables you can look up of atomic radii and see these values, and you can trust them that, yes, they work for both this definition and for this definition here, in most cases.
And what we've been talking about with all of these properties are, of course, how can we figure out what that is for a certain atom by looking at the periodic table, so we want to think about the periodic trend for atomic radius.
And we know as we go across a row in the periodic table, what's happening is that z effective or the effective pull on the nucleus is increasing.
So would you expect, therefore, as we go across a row for the atomic radius, to increase or to decrease?
Good.
OK, yes.
We are expecting to see that it decreases because it's feeling a stronger pull, all the electrons are being pulled in closer to the nucleus, so that atomic size is going to get smaller.
This is in contrast to what's happening as we go down a periodic table.
So as we go down we're now adding electrons to further and further away shells, so what we're going to see is that the atomic radius is going to increase as we're going down the periodic table.
And we can look at an example here.
If we start in the upper left hand corner of the periodic table with lithium, you can see that as we go down the table, what you're seeing is that that atomic radius is actually increasing, as we would expect.
Whereas, if we go across a row, what we see is that the atomic radius is decreasing.
So, again, this is one of the more straightforward trends.
You just need to remember what's happening to z effective, which really tells us what's happening with all the trends, and once you know z effective, you can figure out, for example, what direction the atomic radius should be going into.
So, that's it for periodic trends.
We have talked about four different ones.
We talked about ionization energy, electron affinity, we talked about electronegativity, which is just kind of a combination of the first two, and then ended with atomic radius here.
And what you might have noted is although we described how to make predictions about these properties, I didn't talk too much about what it actually means, what the ramifications of these different properties are.
And the reason we didn't do that is because we're actually going to spend much of the rest of the course relating these different properties to the properties of molecules in terms of bonding, and also in terms of chemical reactions.
So, for example, if we have a very electronegative atom within a certain molecule, what you'll actually find is that it does affect how the molecule is going to take part in different chemical or biological reactions.
And this will become more and more clear as we actually talk about these reactions and talk about bonding.
But you need to be able to predict what kind of properties a certain atom's going to have within a molecule, whether you're talking about something, for example, that's very electronegative, or something that is not electronegative at all, it is going to make a difference in terms of thinking about how molecules are structured and also how they interact with other molecules.
However, I can give you at least one example while we're still on just talking about atoms.
So we haven't gotten to molecules yet, we're just talking about single atoms or single ions, but what's nice is just talking about this very straightforward principle of atomic radius.
We can already use that in terms of single ions to think about a really complex biological issue, which is to talk about ion channels.
So, that is just a quick example for some of you, you might be very familiar with ion channels, others might not know what these are, so I'll just tell you quite briefly that ion channels are these very massive transmembrane proteins.
Essentially, what they are is it's a protein that spans the membrane of a cell.
And what they do is they regulate the influx of ions across that cell.
So the influx of ions from the outside of the cell to the inside of the cell, for example.
And you can think of ion channels as being gated, by gated it means the gate can be closed and no ions are going through, as in this case here.
Or you can talk about the gate being open, and in this case, you can see that you will have an influx of ions.
So, ion channels are important for maintaining a voltage difference between the inside of the cell and outside of the cell, and they're found in all sorts of cell types in your body, but if you think about where they're most prevalent, it turns out that they're most prevalent in muscle cells and also in nerve cells, so in your neurons.
And essentially, what they do in neurons is they underlie those nerve impulses, or those, essentially what we call electrical signaling between neurons -- you might also call that the action potential of the neurons.
So essentially, they regulate this action potential, and they do so by helping to establish and then control the voltage gradient within the cells.
so, essentially, they're establishing or controlling or changing the difference between the charge inside the cell and the charge outside cell.
And when we talk about any type of ion channel, there are just tons of different kinds of ion channels, and you can characterize them in a few different ways.
So, for example, you could characterize them in terms of how they're gated, and basically how they open or close -- that's one way to talk about different types.
Another way to talk about different types is to think about which ion they're selected for.
And all ion channels are selective for a single type of ion, and we can think about how that selectivity takes place, and that's where this idea of atomic radius is going to become very important.
So, for example, if we look at sodium channels, and sodium channels are some of the particularly prevalent ones when we're talking about neurons, if you think about the cell membrane, and this little green cartoon is me trying to show a sodium channel here, and in this case, you can see that it's closed, such that no ions are getting through.
However, when that gate is opened, the sodium channel is now going to be incredibly selective and only let through sodium ions and no other type of ion.
And this is really interesting to think about because you can imagine in our body we have concentrations of all types of ions, and specifically, some seem very, very similar to each other.
So we could think about comparing the potassium ion to a sodium ion.
They have the same charge of plus one.
The only thing that's different is that they're one down on the periodic table, potassium is down one row, so it's going to be a little bigger, but when we're thinking about size, it maybe does not seem that significant to talk about the size.
But what we find out is that it is.
So, what happens, this is another view of a sodium channel, so this is actually looking a little bit more at the protein structure.
What all of these channels have is what's called a selectivity filter, so this filter filters out the type of ion that's going to be allowed through.
And there's two parts of the filter.
First we need to select for actual charge.
So the way that it does this is the filter is actually lined with all of this negative charge, and for those of you that are more into biology or biochemistry, that's because of negatively charged amino acid residues, but all you need to think about is that it has negative charge in the inside of this pore, and what happens then is that if something has a positive charge, it's going to be stabilized to enter this pore, whereas any negative ions are going to be repelled.
So that's the first step in being selective, but now how do we differentiate between these sodium and potassium ions.
And the answer is just really beautifully simple, and it's just that the pore gets really, really tiny, to the point that it gets so small that all that can fit through this pore is one a single ion, one single sodium ion, solvated by one single water molecule.
And that's all that's big enough to pass through or small enough to pass through.
And if we go up even just one row on the periodic table to potassium, what we actually see is now that it's going to be too large, and, in fact, a potassium solvated with one water molecule won't go through our channel.
So, this is just one example of how these properties can already, even our understanding just talking about single atoms, can already make an impact in these biological systems.
And actually, a question that might come up, I just explained, the sodium channel, you might say, well, how do potassium channels work then, because I can understand how you can filter something big out, but how do you filter out something small.
And it turns out that size exclusion is also the principle that's in play with potassium channels as well, but in this case it's a little more complex, because what happens is these negative residues the are in the pore need to stabilize the potassium as it goes through, and the potassium is large enough to make all the contacts it needs, but the sodium, which you can picture being smaller, actually can't reach all of the stabilizing charges that it needs to to get through the pore.
So, again, it is based on size, it's a little bit less intuitive than the idea of just straining out all of the potassium ions.
But again, many of these ion channels have this size exclusion pore, it's a very important part of them.
All right, so that was a quick aside on thinking about how these properties can, in fact, relate to something in our body.
Let's move on to the last topic in terms of this first exam, which is thinking about the idea of isoelectronic atoms, or isoelectronic ions.
And isoelectronic is very straightforward, it just means having the same electron configuration.
The easiest way to look at this is just to do an example.
So let's take the example of neon.
This has the electron configuration of 1 s 2, 2 s 2, and 2 p 6.
It looks like we're cut off the screen a little bit here, but you can see I've just circled it there.
So we can go ahead and think about, well, are there any other atoms that are going to have the same electron configuration?
The answer to that is definitely no -- if they had the same electron configuration, they would, in fact, be neon.
But we can think about different ions that have this electron configuration.
So for example, if we think about fluorine, that has an electron configuration of 1 s 2, 2 s 2, 2 p 5, so all we would need to do is add one more electron to get the same configuration as for neon.
So if we want to write out what that would be, it would just be to say that f minus is isoelectronic with neon.
So, we can say that -- if we have neon here and we want to think about what's isoelectronic, f minus would be isoelectronic.
We also have oxygen -- what would the charge on oxygen be?
Um-hmm, right.
2 minus.
Then also, nitrogen, 3 minus -- these are all going to be isoelectronic with neon.
We can go in the other direction, so let's go to sodium, but we would need to take away an electron to make it isoelectronic.
So we would say sodium plus, or magnesium 2 plus, we can just keep going -- aluminum 3 plus, silicone 4 plus, and we can go on and on in either direction all the way across and down the periodic table.
So, that's the idea of isoelectronic ions.
These are all isoelectronic, they all have the same electron configuration.
And we can also think about going back to atomic size for a second.
What the relationship is between these ions and their parent atoms.
So, for example, if we think of the fluorine minus case, would you expect fluorine minus to be larger or smaller than neutral fluorine?
Okay.
I heard mostly larger, but a little bit of a mix in there, and it turns out that larger is correct.
And we can think about why -- essentially we have fluorine and now we're adding another electron.
So you can picture that fluorine is going to get larger in this case.
And that would be true for all of the negatively charged ions.
So, by the same logic, that means that all of our positively charged ions are, in fact, going to be smaller in terms of radius, compared to their neutral parents.
Not only are we taking away an electron here, but we're also going to decrease shielding, so the electrons that are already in there are going to feel a higher z effective and will be pulling and the atom will be getting smaller.
And this is just a picture showing some of these sizes with their parent.
So, for example, a lithium here, you can see how lithium plus is smaller than the actual lithium atom in its neutral state.
Whereas for fluorine, fluorine is smaller than f minus is the one that's the outer shell shown here.
So, let's do a clicker question on isoelectronic atoms.
And now we're asking you to look at krypton, so the atomic mass is 36.
You can actually just grab that handout, the second handout on the exam and look at the periodic table there.
So, which of the following ions listed is isoelectronic with krypton?
OK, let's take 10 seconds on that.
OK, good.
This might be our all-time high, 89% got this right.
This is great.
So, selenium 2 minus is what's going to be isoelectronic, because if you add two electrons to selenium, you'll get the same electron configuration that you have for krypton here.
OK, I think we can safely go back to notes.
So, I said I would announce it, that's the end of exam 1 material.
So, if you compartmentalize things in your brain in certain ways, put that off into the end of the exam 1 part of your brain, and now we're going to move on to exam 2.
Remember, for exam 2, you still need to know and understand everything you learned in exam 1, but you can put off learning it completely until we get through, at least, next Wednesday before we start maybe spending time on these concepts outside of class.
So, we're going to start with talking about bonding, and any time we have a chemical bond, basically what we're talking about is having two atoms where the arrangement of their nuclei and they're electrons are such that the bonded atoms results in a lower energy than for the separate atoms.
So we know we always want to have our systems in as low an energy as possible, so it makes sense that a bond would happen any time we got a lower energy when we combine two atoms, versus when we keep them separate.
So specifically, today we're going to talk about covalent bonds.
A covalent bond is any time we have a pair of electrons that is shared between two different atoms.
And the key word for covalent bonds is the idea of being shared.
The two electrons, for example, we see in the h 2 molecule, they don't belong to one or the other atom, they're actually shared.
And what we'll see later, is that the sharing is not always equal -- in the case of h 2, it is completely equal sharing.
In some cases, because of things like electronegativity, one atom will take away more of the electron density than the other atom, but they're still shared, even if they're not always evenly shared.
So, in talking about covalent bonds, we should be able to still apply a more general definition of a chemical bond, which should tell us that the h 2 molecule is going to be lower in energy than if we looked at 2 separate hydrogen atom molecules.
So, let's see if that's actually the case.
So if I tell you that the energy for single hydrogen atom is negative 13 12 kilojoules per mole.
If we want to talk about two hydrogen atoms, then we just need to double that, so that's going to be negative 2 6 2 4 kilojoules per mole that we're talking about in terms of a single hydrogen atom.
So, let's compare this to the energy of the h 2 molecule, and we find that that's negative 3,048 kilojoules per mole.
So, in fact, yes, we did confirm that these covalent bond, at least in the case of hydrogen, we have confirmed by the numbers that we are at a lower energy state when we talk about the bonded atom versus the individual atom.
And when we talk about covalent bonds, there's 2 properties that we'll mostly focus on, and that's going to be thinking about the bond strength or the energy by which it stabilized when it bonds.
And we can also talk about the bond length, so we might be interested in what the bond length is, what the distance between these two nuclei are.
And we can actually better visualize this if we plot how that energy changes as a function of internuclear distance.
And when I say internuclear distance, we actually call this r here.
It's kind of ironic that we put this in the same lecture as we talk about atomic radii, which we also call r, but they're two different r's, so you need to keep them separated in terms of what you're talking about.
When we're talking about r for internuclear distance, we're talking about the distance between two different nuclei in a bond, in a covalent bond.
So, if we look at this graph where what we're charting is the internuclear distance, so the distance between these two hydrogen atoms, as a function of energy, what we are going to see is a curve that looks like this -- this is the general curve that you'll see for any covalent bond, and we'll explain where that comes from in a minute.
I want to point out that the zero energy is defined as when you have a naked proton where the electron has popped out -- that's what we've defined as zero energy up to this point when we're talking about single atoms.
So, for starters we'll keep that as our zero energy, we're going to change it soon to make something that makes more sense in terms of bonding, but we'll keep that as zero for now.
So, we see that the two h atoms separate have a certain energy that's lower than when the electron's not with the atom.
And then even lower down, we have our bonded hydrogen molecule.
So, we can think about the different kinds of interactions that are taking place.
I said what hold the bonds together, what holds two atoms together is the attractive force we have between each electron and the other nucleus.
That's the huge force that we're talking about in terms of making a bond stable, but there are also repulsive forces, so you can imagine we're going to have electron-electron repulsion between the two electrons if we're bringing them closer together.
And the real killer is if we get too close we're even going to have nuclear-nuclear repulsion between the nuclei of the two atoms.
So, this makes this chart shown in pink make a lot more sense, because if we're way out at very far distances, essentially what we have here is we're talking about two separate atoms.
They're not interacting at all so that's why the energy is the same as that for two individual atoms, that's what we're dealing with.
As we get closer together, we start get lower and lower in energy.
The reason is because the predominant force at this point is going to be the attraction that's being felt between the nuclei and the electrons in each of the atoms.
At some point you're going to hit a well here, which is the point where it's most stabilized or at it's lowest energy.
So, when we think about a bond length, this is going to be the length of our bond here, that makes sense because it's going to want to be at that distance that minimizes the energy.
But as we keep getting closer, even though as we get closer, the attraction is going to get stronger between the two nuclei and the electrons.
We're also going to start to have the repulsive forces become more prominent here, and, in fact, they take over at some point, becoming the more prevalent of the forces, so as you get closer, the electron-electron repulsions, and eventually the nucleus-nucleus repulsion is going to mean that your energy is just absolutely skyrocketing, so it just keeps going up and up as you get closer to zero here.
So, when we want to talk about the information that we can get out of looking at a chart like this, well, the first thing I did tell you was that this is going to be the bond length, so the distance r where the energy is lowest, but we can also talk about something called dissociation energy, that's going to be this distance right here or the energy that is this value.
And the dissociation energy is very intuitive in terms of what it means, it means how much energy you need to put into the molecule in order to disassociate it into its individual atoms.
And so we can actually think about how do we calculate what the dissociation energy should be for h 2, so let's go ahead and do this.
So, if we talk about dissociating h 2, we're going from the h 2 molecule, and breaking this bond right in half, so we now have two individual hydrogen atoms here.
So we need to take the energy for the two atoms, which we know is -- so let's take our dissociation energy is going to be equal to negative 2 6 2 4 kilojoules per mole, and we want to subtract the energy of the hydrogen molecule itself, so that's going to be negative 3 0 4 8 kilojoules per mole.
So, what we get for the disassociation energy for a hydrogen atom is 424 kilojoules per mole.
So what that means is that's how much energy we would have to put in to a hydrogen molecule in order to get it to split apart into its two atoms.
So, another way to talk about dissociation energy is simply to call it bond strength, it's the same thing, they're equal to each other.
If we know that this is it the dissociation energy for a hydrogen atom, we can also say the bond strength for hydrogen molecule is 424.
So, there's actually another way to graph it where we can directly graph the dissociation energy or the bond strengths.
So I said before when we were talking about single atoms, we always define the zero energy as when an electron was actually ejected, but now, when we talk about chemical reactions taking place, it's very, very rare that we're actually going to be talking about anything that gets to this point here.
It's much more relevant to set our zero point energy as the separation of a bond in terms of talking about the reactions that we'll usually be dealing with here.
So, let's change our graph where we now have this zero point set as the two individuals hydrogen atoms, and then we see that our h 2 molecule is at the negative of the dissociation energy, or the negative what that bond strength is.
So we know what that number would be, it would be negative 424 kilojoules per mole that we see here.
So, what this let's us do now is directly compare, for example, the strength of a bond in terms of a hydrogen atom and hydrogen molecule, compared to any kind of molecule that we want to graph on top of it.
So, let's, for example, look at nitrogen.
So n 2, we can do the chart here in green, so it's the green dotted line, and what we see is that we have now defined this energy as where the 2 nitrogen atoms are separated.
So what we can actually directly compare is the dissociation energy or the bond strength of nitrogen versus hydrogen.
So, if we think about this, which would you say has a stronger bond?
Is it going to be hydrogen or nitrogen?
Yup, it's going to be nitrogen.
And the reason we can see that by looking at this graph is that we see that nitrogen when it's bonded is in an even lower well than we saw for hydrogen.
It's going to be a stronger bond because it's more stabilized when it when it comes together as a molecule.
We can also think about the distance, the bond distance.
So, which would you say is going to be shorter in this case?
Is a hydrogen bond shorter, or is a nitrogen-nitrogen triple bond going to be shorter?
Um-hmm, again, we can get this information directly from our graph.
We see that the radius is shorter, so that means that the nitrogen-nitrogen bond is going to be shorter.
We can know this information even if we just knew that the bond was stronger, we wouldn't need to look at a graph here, because it turns out that if you have a stronger bond, that also means that you have a shorter bond -- those ywo are correlated.
And something that we'll see later on is that triple bonds, for example, are going to be stronger than a corresponding double bond or a corresponding single bond.
So, if we talked about a nitrogen-nitrogen single versus double versus triple bond, the triple bond will be the shortest and it will be the strongest.
So, that's basically the idea of how we are going to be thinking about covalent bonds.
It's also important, once we start talking about molecules, to have a way to represent them, and also to be able to look at a shorthand notation for a certain molecule and understand what the bond is.
So, for example, down here I wrote that it was n 2 and that it was h 2, but when I re-wrote the molecules up here, you saw that it's an h h single bond where it's a nitrogen-nitrogen triple bond.
So any chemist should be able to just look at n 2 and know that it's a triple bond, but that's not something that we've learned how did to do yet, so let's go ahead and start a new topic that's going to allow us to have some sort of sense of what the valence electron configuration, which includes whether something's a single or double or a triple bond can be figured out for any given molecule.
So, to do this, what I'm going to do is introduce the topic of Lewis structures.
We're going to really get into this next class, but I just want to introduce it to you to give us a start, and many of you have used Lewis structures in high school, but we'll be doing some much more challenging Lewis structures, I can assure you, in this class here.
So, a Lewis structure is basically an organizing property of bonding, of molecules, which is the idea that when we're thinking about bonding, the key is to achieve a full valence shell in each of the individual atoms.
So we want to have in an h h bond, for example, a full shell for each of the hydrogen atoms.
And G.N.
Lewis is the scientist that is credited, and who did, in fact, come up with this idea for the way to represent this, so the other parts of this idea, another way to phrase it is that the electrons are going to be distributed in such a way that we have what are called full octets for each of the atoms, and basically that's the same thing as saying we have a full valence shell, and this is something that Lewis was able to recognize very, very early, way before we had quantum mechanics to describe what these orbitals were, but it makes sense a full valence shell means for most atoms that we have a full s orbital plus a full p orbital, so we're going to have a total of four orbitals that are each filled with eight electrons, so that's why we see that we need an octet here.
And the idea is that when you do these Lewis dot structures, we're representing electrons with dots, which we'll see in a minute, and each dot is going to represent a valence electron.
So, hopefully, you remember what we mean by valence electrons versus core electrons.
Core electrons are all those electrons held in really tight with the nucleus in the inner shells, whereas the valence electrons are only those electrons that are in the outer-most shell, or at your highest value of n of the principal quantum number.
So, Lewis structures are really a model for a way to think about what the valence electron configuration is, and as I said, it's not based on quantum mechanics, it's something that Lewis observed far, far before quantum mechanics were discovered.
So he came up with the ideas that led to the idea of Lewis structures in the very early 1900's.
So you might ask well, why are we using this model if it clearly doesn't take into account quantum mechanics?
And the reason that we use it is that it is incredibly accurate, and allows us to very, very quickly predict and to predict accurately, in most cases, what the electron configuration of molecules are going to be.
So this is really useful.
We don't always want to go and solve the Schrodinger equation, and in fact, once we start talking about molecules, I can imagine none of you, as much as you love math or physics, want to be trying to solve this Schrodinger equation in that case either.
So, what Lewis structures allow us to do is over 90% of the time be correct in terms of figuring out what the electron configuration is.
And we won't just use them in this class.
If you actually go to any of the chemistry labs at MIT, if you go over to building 18 and look in the organic labs where they're synthesizing new molecules or making up new reactions, what you'll see if you open anyone's notebook, their lab notebook, assuming they keep a nice lab notebook, is that they will have Lewis structures drawn in there that explain the reactions that they're going to be doing for that day.
And I mean this means way past all the chemistry they've taken, they're now graduate students or they're now professors, and they're still writing out Lewis structures.
Now they're writing a more abbreviated form, which you'll probably get to if you take organic chemistry, but really it's the exact same idea.
And this goes all the way back to 1902.
In fact, Lewis was an American scientist, so he was trained in America, and he actually was a professor here at MIT from 1905 all the way to about 1911 or 1912, and these are some notes from 1902, and you can't see them very well, but this was essentially an early form of Lewis structures, and this was called the cubicle atom.
So, basically what he's showing in these cubes is that there are eight spaces that need to be filled up to have a full cube.
So in order to fill them, he would have to have eight electrons or an octet around the cubes.
So what we're seeing is this is notes from 1902 -- he actually didn't publish any of this work or these ideas that led to Lewis structures until 1916, but his early class notes were used as evidence about how long ago he actually came up with the idea of it.
So it's really neat to think that your counterparts 100 years ago right here at MIT could have been sitting in a class where they had Lewis as their lecturer, and he's putting forth these ideas -- these are actually his lecture notes, even though it wasn't even published yet, and giving this idea of Lewis structure, which is exactly what we keep using today in order to make a lot of these predictions.
So, let's see how some of this works, and hopefully your counterparts from 100 years ago would also be able to think about how this works, even if they don't have the quantum mechanics behind the individual electron configurations for atoms.
So I said that we want to be talking about valence electrons here, so that means if we're talking about, for example, the octet rule for an f f molecule where we have two fluorine atoms, we need to write the valence electrons as dots around them.
So let's do a quick clicker question, and you tell me how many valence electrons does fluorine have?
Remember, valence electrons are different from core, they're only the outer-most electrons in the outer-most shell.
So, 10 seconds on this, this should be fast.
OK, great.
Good job on the clicker questions today.
So we have seven valence electrons.
So, let's go back to the notes, and let's fill these in, seven electrons.
Another way you could have known them was to look at Lewis' notes here, where if look at this box carefully you see there are seven dots around the cube, so there are his seven valence electrons.
So, we see is when we use the octet rule to look at fluorine molecule, we're combining two fluorine atoms, and what we end up with is an f f molecule where they're sharing two electrons, so making that covalent bond.
But that each individual fluorine atom has eight electrons, or full octet around it.
We can think about where those electrons came from, so we got seven from the blue electrons here, seven as shown in green here, but each individual fluorine atom has eight, even though two of those are being shared between both of them.
So, the octet rule is a general rule that you'll for all of the atoms.
There are some exceptions, which we'll get to later, but the only a big exception here is with hydrogen, which has a special stability that's associated with two electrons.
This should make a lot of sense, because we know that a hydrogen has 1 s as it's outer-most or valence orbital, so it can be filled up just with two 1 s electrons.
And we give different names, depending on what kind of electrons we're dealing with, so, for example, with h c l here, we can talk about having bonded versus lone pair electrons.
So, in terms of the c l atom, we need to talk about each atom individually.
How many bonding electrons does c l have?
All right.
Let's see, we've got a mixed response here, it turns out it has two bonding electrons.
I heard some people say one, and that's a good guess, remember they're actually sharing.
So these two electrons, they belong to chlorine, they also belong to hydrogen, but they do, in fact, belong to chlorine as well.
There's no one person owning them, so they both have two electrons here that are bonding.
So how many lone pair electrons do we have?
OK.
I hear six and three, so both are sort of right, we have 6 lone pair electrons, which means that we have three lone pairs.
So, in terms of thinking about how to draw a Lewis structure, I won't go through this today or any day in terms of just reading through the rules, you can read that yourself.
But what we'll do is go through each of these rules in terms of an example.
So, what will start with on Monday is doing the most simple example of methane using these Lewis structure rules.
So, don't forget to study this weekend and get those extra practice problems from the course website.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
Could everyone take 10 more seconds on the clicker question.
And as a reminder, hopefully I don't need to remind any of you, but exam 1 is on Wednesday, so rather than our clicker question being on something from last class, which is exam 2 material, let's just make sure everyone remembers some small topic from exam 1 material, which is the idea of angular nodes.
So I was hoping to see more like 99% on this, but you do still have two more days before the exam.
Remember, we're talking about angular nodes here, so you need to read the question carefully.
For an angular node, we're just talking about what the l value is, so whatever l is equal to is equal to the number of angular nodes you have.
For an f orbital, what is the quantum number l equal to?
Three.
Good, so everyone that recognized that probably got the right answer of three angular nodes here.
So let's switch to today's notes.
Two more quick things about the exam, the first is that just remember on Wednesday, don't come here, the exam is not here, don't come here.
It's in Walker, so make sure you go to Walker.
And also, keep in mind that I have office hours today from 3 to 5.
All your TA's have either already have their extra office hours, or there are some that will be going on tonight or tomorrow, so keep those in mind as your finishing up your studying for the exam.
So, today we're moving on, we're talking about Lewis structures.
This is a really good topic to do the class before an exam because it's a little bit of a lighter topic.
We remember that Lewis structures are an idea that are pre-quantum mechanics.
So that means that we don't have to worry about things like wave functions when we're talking about Lewis structures, but because they're so simple to use and because they so often predict the electron configuration of molecules accurately, we end up using them all the time in chemistry, so it's very valuable to know how to draw them correctly and to know how to work with them.
So we'll talk specifically about drawing Lewis structures and then about formal charge and resonance, which are within Lewis structures.
So remember, that when we talked about Lewis structure, the organizing principle behind Lewis structures is the idea that within the molecule the atoms are going to arrange their valence electrons, such that each atom within the molecule has a complete octet or full outer shell.
So this is the idea of the octet rule that Lewis came up with way back in 1902.
So, at the end of the class on Friday, we saw that list of eight steps that you always need to go through when you draw a Lewis structure.
Once you're doing this on your own, especially, for example, on exam 2, which is a ways down the road, you won't be able to look at those steps, so you need to make sure that you can go through them without looking at them, but for now we can look at them as we are actually learning how to draw the Lewis structures, and rather just go through them step-by-step, it's more interesting to do it with an example.
So let's hydrogen cyanide as our first example.
So we have h c n. So, in this example here, we start with the first step -- the first step in any Lewis structure is drawing the skeletal structure.
So, essentially drawing how the atoms are arranged within our molecule.
And in this case we have three choices here in terms of what's going to be in the middle, so we need to decide that first.
In terms of where different atoms are in a molecule, if you have a hydrogen atom or a fluorine atom, you can pretty much guarantee they're always going to be terminal atoms.
By terminal I mean they're only bonded to one thing.
So, for example, hydrogen or fluorine they'll never be in the middle, they'll always be on the end of a molecule.
So that takes care of the hydrogen, what about between the carbon and the nitrogen?
In terms of picking a Lewis structure that's going to be the lowest energy, what you want to do is put the atom with the lowest ionization energy in the center of your atom.
This should make sense because if something has a low ionization energy, that means it's not very electronegative, which means it's going to be a lot happier giving up electron density, which is essentially what you're doing when you're forming covalent bonds is you're sharing some of your electron density.
So, we keep the atoms with the lowest ionization energy in the center.
So, why don't you go ahead and tell me, keeping that in mind, which atom in terms of h c or n would you expect to be in the center of hydrogen cyanide?
And I put a periodic table up there, just the part you might need to look at.
So this should be fast, so let's take 10 seconds.
All right, good job everyone.
So most of you saw that carbon should be in the center.
Carbon should be in the center because it has the lowest ionization energy.
We know that ionization energy is going to increase as we go across the periodic table, so that means carbon has a lower ionization energy than nitrogen, which is right next to us.
So as I just said, we want to put that one in the middle.
You got an extra hint here in terms of the order, so even if you had just forgotten what I said, sometimes it's not a terrible idea just to put it in the order it's written, that can give you a lot of clues as well.
So, either of those ways of figuring this out is the first guess of what goes in the middle will work pretty well.
So, let's go ahead and draw our Lewis structure based on the rest of the rules now that we have a skeleton.
So our skeleton tells us that carbon is in the middle, so we'll put the h on one side, and the n on the other side there.
So, our second step, as we go through our Lewis structure rules, is to figure out how many valence electrons we have in our entire molecule.
So if we talk about hydrogen, how many valence electrons are we talking about?
1.
What about carbon?
I heard 4 and 6.
And remember, we're only talking about valence electrons, so the outer-most shells.
So we're talking about four valence electrons for carbon.
And then for nitrogen?
Lots of options I have to choose from from these answers, but it's 5.
So, if you can't immediately know, and you don't all have periodic tables in front of you, so that's fine, but if you have a periodic table in front of you, you need to be able to count valence electrons, so work on that if it doesn't come naturally to you in terms of figuring that out.
So then in order to figure out the complete number of valence electrons in our molecule, we just add 5 plus 4 plus 1.
So we end up having 10 valence electrons.
Step three in our Lewis structure rules is to figure out how many electronis we would need in order for every single atom in our molecule to have a full valence shell.
So, if we're talking about hydrogen, that's our one exception so far to the octet rule.
So we actually only need two electrons to fill up the valence shell of hydrogen, remember that's because all we need to fill up is the 1 s. However, for carbon and nitrogen we need 8 each.
So in terms of total numbers that we would need to complete our octets and fill our valence shells, we would need 18 electrons.
All right.
So let's bring down our middle slide here.
So we have 18 electrons, and the next thing that we need to figure out is how many bonding electrons we have.
So to figure out bonding electrons, what we take is that number 18, which is our total number of electrons we need to fill valence shells, and we subtract it from our number of valence electrons, which is 10.
And what we find that we have is 8 bonding electrons.
And hopefully on your paper, you can actually reach your h c n skeleton -- I think I should probably re-draw mine here.
Because step five is that we need to fill in our bonding electrons, and we start it with filling in two electrons per bond.
So I'm just going to re-draw my skeleton.
So, the first thing we do is put two electrons between h and c, and then two electrons between c and n. Remember, every time we have two electrons that are being shared, that's a single bond.
The next thing that we want to do is figure out do we have any bonding electrons left.
So let's see, we started with 8 bonding electrons, and we used up only 4, so the answer is yes, we have 4 bonding electrons left.
So what we need to do is fill in those extra bonding electrons into our bonds.
Should I put an extra pair of electrons here, does anyone think?
No.
The reason is because we already have a full valence shell for our hydrogen, it doesn't want anymore electrons.
What about between the carbon and nitrogen?
Yes.
Definitely, because both of these are not anywhere near filling up their octets yet.
So we can put actually all 4 of our extra electrons in between the carbon and the nitrogen.
Now we have 6 things around the nitrogen, and we have 8 around the carbon.
So, what we do as our seventh step is then figure out if we have any extra valence electrons left at all.
So we started with 10 valence electrons, we used up 8 of those electrons in terms of making bonds.
So it turns out that we have 2 valence electrons left.
So we need to add those 2 valence electrons left as lone pair electrons in our structure.
So, which atom is in need of those lone pair electrons?
The nitrogen.
The reason being that's the only one that didn't have a full octet yet.
So now we're done, actually there is one more step, which is to determine the formal charge.
This is a good way to actually check if your Lewis structure is correct or not.
We haven't actually learned how to calculate the formal charge yet, we'll learn it soon.
So we won't do it for this molecule, but we'll go back and do it for some of our other examples, and you can go back and do it for this one.
The other thing is that we can re-write our h c n in terms of bonds.
So we know every time we have two electrons, that's a bond.
So we have h, then we can draw our bond as a line.
And then we have a triple bond there because we have 3 pairs of electrons.
So it looks a lot less messy if we just draw our Lewis structure like this for h c n, where we have h bonded to c triple, bonded to n, and then a lone pair on the nitrogen there.
All right, so this is the same procedure that we're going to go through, regardless of what kind of Lewis structure we're going to draw.
What you'll actually find in terms of asking your TAs about the Lewis structure rules is that sometimes they won't be as good at them as you are, and the reason is once you've drawn enough of these structures, you start to get a lot of chemical intuition about what's right or what's not right -- it just looks wrong to you if it's wrong.
So your TA might take a minute, so be patient with them if they see your structure and they say oh, no, no, no, no that's wrong, that's terrible, and they don't immediately know why.
They might need to go through the rules with you, you might need to remind them.
Hopefully, they'll all study them again, so this will be an issue.
But what really happens is as you go on in chemistry, you draw so many of these you can just draw them without following the rules.
Some of you might get almost to that point, or you might be at that point now, but I recommend this for you and for me and for the TAs, go through the rules because there'll be cases where it's a little bit tricky and it's always much faster to have gone through step-by-step, than to try to just kind of hit or miss figure out what's going to be right or wrong.
So, let's try another example here, and let's try a case now where instead of dealing with a neutral molecule we have an ion, so we have c n minus.
And what I'll mention to you just in terms of the fact that we're finally dealing with real molecules, which is -- or molecules that are made up of more than one atom, which is kind of exciting for me and maybe for some other of you that like to move into thinking about what some of the consequences of these molecules reacting might be.
A lot of the examples that we're going to give you in terms of trying out your Lewis structures will be molecule that are used in organic synthesis, or maybe they're molecules that react in interesting ways with biomolecules in your body or proteins in your body.
So, you already will have a head start when you get on to later classes, like organic chemistry or if you're thinking about biochemistry where being able to draw the Lewis structure allows you to think about, eventually, the reactivity of the molecule, which becomes very interesting in thinking about how you're going to synthesize a more complex molecule, or how that molecule is going to interact with an active site in a protein in the body.
So, for example, just talking about hydrogen cyanide or the cyanide anion, these are both molecules which are used in organic synthesis, so particularly the cyanide, anion and salts of the cyanide anion.
So either a potassium cyanide or sodium cyanide, these are used in synthesis in terms of making carbon-carbon bonds.
So if you're trying to make a more complicated organic molecule, carbon-carbon bonds are one of the most difficult things to make an organic chemistry, and it turns out that c n minus is a very reactive molecule, so it's a good way, even though we'll go over some drawbacks in a second, it is a good way to make carbon-carbon bonds.
It's very reactive.
And because, of course, we have this carbon here what you end up doing is adding a carbon to your molecule.
So, when you think about cyanide, you might not think about organic reagents.
Does anyone have something else they think about when they think about cyanide?
Death
Death -- that's a good thought.
Yes, cyanide and death, very closely related as well.
Cyanide is incredibly toxic, it's a poison.
That might be how you're more familiar with cyanide.
So if you're working with cyanide in the lab as potassium cyanide or sodium cyanide, those are what are called p h s's, or particularly hazardous substances -- it's a rating for different kinds of chemicals.
And what that means it's there's all sorts of precautions and procedures you take that are special when you deal with these.
They're kept away from other chemicals.
You handle them very special in terms of being extra careful in a very high ventilation area, in hoods is how you handle them.
So, yes, they're very poisonous, and in fact, there are areas where you find this toxic compound, cyanide.
Other than just in poisons and in organic synthesis shells, you might also find them in some things we're more familiar with, such as almonds.
I don't know how many know that there are trace , trace amounts, of cyanide in almonds.
I don't know if there any big almond eaters out there.
You don't have to worry, we're definitely talking about trace, trace amounts.
It's not going to hurt you.
And actually, what we usually eat are what are called sweet almonds, and there aren't actual cyanide in the sweet almonds we eat, there's precursors to cyanide, which might not make you more comfortable.
But the fact is there are trace, trace amounts.
This is nothing that we need to worry in our food supply.
However, some people aren't so lucky.
I don't know how many of you are familiar with the Cassava plant, which is a kind of woody shrub that's first been cultivated in South America, but it's grown throughout Africa, the Caribbean, South America still, many places around the world, and this is a major source of carbohydrates for much of the world, because Cassava root is very, very rich in carbohydrates.
It's not the best form of food in that they're actually very poor in protein, and unfortunately very, very rich in cyanide.
So, these roots can be very dangerous.
There's different types of the root that you can get called the bitter and the sweet.
Hopefully you would all choose the sweet if two are put in front of you.
The bitter, of course, are the ones that are very high in cyanide.
If you eat these raw, which they do in many places around the world, if you eat a bitter one, you could, in fact, get enough cyanide to kill you.
And there are ways to prepare these, so it's important -- this kind of thinking more along the food science idea.
There's a way to actually grind down and prepare the flower, so that you promote the enzymes within the plant to breakdown the cyanide precursors.
And if you put this in the well-ventilated area, if you prepare this outside, the h c n gas will actually be released into the air, so you're safe, you can eat it later.
About 80% of the cyanide at that point is gone, so it does render the root much more safe.
But you do, in fact, have to worry about long-term exposure, cyanide poisoning in terms of long-term effects in certain populations that do get the bulk of their carbohydrates from this root, from the root of the Cassava plant.
But in terms of us going to the grocery store and thinking about things, probably we're all breathing sighs of relief.
I just told you almonds are not a problem, no worries there.
We probably don't find any forms of the Cassava plant ever in the U.S. that are going to have that high cyanide content, so we should all be relieved.
Unless, of course, you're a smoker, or you're thinking of becoming a smoker, and then maybe you should worry, because this is one of the advertisements that was airing in terms of anti-smoking campaign.
Hydrogen cyanide is found in cigarettes.
So, if you're looking for another reason to quit, if you're looking for a reason not to start smoking, here's another good one.
As I said, it's a particularly hazardous substance, this is worked with in fume hoods.
You don't want to inhale it, it's definitely not recommended.
The way, in the simplest terms that cyanide can kill you, is it basically out-competes your oxygen for the heme in your blood.
So instead of carrying oxygen to your cells, you're carrying cyanide to your cells.
Obviously, the amounts that are in cigarettes are not enough that people are dropping dead of cyanide poisoning, but still it's not a good idea if you can avoid eating or inhaling cyanide -- you definitely want to minimize your exposure.
And in terms of thinking about it for organic chemistry or if you're interested in thinking about the mechanism maybe by which it is toxic, a first step would be to draw its Lewis structure.
So, let's go ahead and make sure we can draw that, if we have interest either in the area of organic chemistry or biochemistry or biology here.
So in terms of the first step of skeletal structure, this is actually going to be easier because we don't have a central atom, we just have carbon and nitrogen here.
Our next step is thinking about valence electrons.
So we have 4 plus 5, but we're actually not done yet, because it's c n minus, so if we have minus, we actually have an extra electron in our molecule.
So we need to add 1 more.
If instead we had a positive ion, a cation, what we would have to do is subtract 1.
But here we're going to add 1, so again, we have 10 valence electrons.
And if we go on to step three where we figure out how many we would need for full octets, it's just going to be 2 times 8, so we have 16.
And step four is going to have us figure out how many bonding electrons we have, so we have 16 minus 10, is going to be 6 bonding electrons.
So, step five tells us to add 2 electrons between each atom, so we add two there.
And step six asks us, well, do we have any bonding electrons left?
So how many bonding electrons do we have left?
Yup, so we do, we have 4 left.
We started with 6, we only used 2.
This is very easy molecule because we know exactly where to put them without even having to think, we only have one option, and we'll make a triple bond between the carbon and the nitrogen.
So, seven asks us if we have any valence electrons left, and how many valence electrons do we have left?
Yeah, so also 4.
We started with 10 valence electrons, we used up 6 of those as bonding electrons, so we have 4 left, which will be lone pair electrons.
So, in order to fill our octet, what we do is put two on the nitrogen and two on the carbon.
So, in terms of finishing our Lewis structure, we're actually not done yet here, even though we have full octets, and we've used up all of our valence electrons, and the reason is because it's c n minus, so we need to make sure that that's reflected in our Lewis structure, so let's put it in brackets here, and put a minus 1.
And also I wanted to mention in terms of checking your Lewis structures, regardless of what they are, you should always go back and say how many valence electrons did I have -- I had 10, and then count 2, 4, 6, 8, 10, because you always need to make sure you have the same number of valence electrons that you calculated in your actual structure.
That'll catch a lot of just silly mistakes for you if you go back and see it and you don't have all of that.
Let's re-draw this, so it looks a little bit neater, where we have a triple bond in the middle instead, and again, we need our negative 1 charge there.
And our eigth step in the process, again, is formal charge, which we will talk about very soon.
All right.
So let's try one more example of drawing Lewis structures before we talk about formal charge.
And the last example that we're going to talk about is thionyl chloride, so it's s o c l 2.
This is another good step forward, because now we actually have four different atoms in our molecule.
I'll tell you a little about thionyl chloride as well.
This is another organic chemistry reagent, it's also used extensively in the pharmaceutical industry.
And what it's used is to convert one type of group, what's called a carboxylic acid into another type of very reactive intermediate, which is called an acid chloride.
So I show that here, so in green, you have what's called a carboxcylic acid group, a c o o h, which gets converted by s o c l 2 to a c double bond o c l or an acid chloride.
This is the very reactive intermediate.
You'll learn a lot more about this if you take organic chemistry, but, In fact, you can then go on and make a bunch of other different kinds of very interesting molecules.
So, for example, this is the synthesis of novacaine.
This is what's used in industry to actually make novacaine.
Has anyone had a novacaine procedure?
Yes.
I've had it also many times, you usually get novacaine for cavities.
There's some alternatives that are used now as well.
It's also used as a local anesthetic for other types of small procedures.
So, this is, in fact, what's used to make novacaine in industry.
You'll notice that a lot of different kinds medications do you have chlorine in them, you'll see that c l group.
So, for example, you might be familiar with Wellbutrin here, this is a type of anti-depressant that a lot of people use right now that are taking anti-depressants.
It's on the market, very popular in terms of your choices right now as an option as an anti-depressant.
Also, Lunesta.
This was very big in an ad campaign at least last year, I'm not sure if it still is, with the little butterfly.
This is the structure of Lunesta, and you see the c l in it as well.
I just wanted to point out that although you see these chlorine atoms in these drugs, what you almost never see is an acid chloride -- in fact, I don't think I've ever seen an acid chloride in a final pharmaceutical product or drug that we take, and the reason is because they're so reactive that you wouldn't want to have that in something you digest.
So just keep in mind when you do see the chlorine in these drugs, it's very different from the acid chloride.
So, for example, Wellbutrin, it is very unlikely that it would have thionyl chloride in order to make it, and if thionyl chloride was used at some point in the synthesis, it was not to put that chlorine atom on, it was to put something else on.
But in terms of drugs that don't look like maybe this compound was used in the synthesis, many of them might have used thionyl chloride, because it generates such a nice reactive intermediate that you can go on and make a bunch of different compounds from that intermediate.
All right.
So let's think about how to draw the Lewis structure for thionyl chloride -- oh, actually, let me let you tell me how we should start this Lewis structure.
So, which atom would you expect to be in the center of a Lewis structure for thionyl chloride?
All right.
Let's take 10 seconds on that.
Looks like we have some fast thinking here, a lot of last minute answers coming in.
OK. We have a split decision, so -- you know what, actually, let's think about this for a second.
So hopefully, it was a time issue in terms of looking at the periodic table, because let's have you tell me what are we looking for here?
Yeah.
OK. We're looking for the lowest ionization energy.
So, this one can be tricky because oxygen looks like it's in the middle because of the way it's written, but we need to start by looking at the lowest ionization energy.
So, if we look on the periodic table, comparing, for example, s to o, if we have s it's below o, what happens to ionization energy as we go down a table?
It decreases.
If you're still not completely up on the periodic trends, that is stuff that's going to be on the first exam, so make sure that you're able to do this without taking too much time to think about it.
We would expect the ionization energy to decrease, that means that sulfur has our lowest ionization energy.
All right.
So, let's go ahead and draw our Lewis structure here with sulfur in the middle.
So, we can put our sulfur in the middle, and then it doesn't really matter how we draw the rest of it, where we put our c l's versus where we put our oxygens.
We'll just put them in some way around our sulfur atom.
So that's our step one.
For our step two, what we need is number of valence electrons.
So we have 2 for each of the chlorine.
How many valence electrons are in chlorine?
All right.
So it's 7 that are in chlorine, it's the same as fluorine or any of the others in that row or in that group rather.
2 times 7, plus we have 6 in the sulfur, and oxygen is right above sulfur, so that also has 6.
So we end up having 26 valence electrons that we're dealing with here.
Our step three is to figure out how many bonding electrons that we need, or excuse me, how many total electrons that we need to fill up our octets, so that's just going to be 4 times 8, which is 32.
And then we take 32 minus 26.
So what we end up with in terms of our bonding electrons is going to be 6 bonding electrons.
So we can go right ahead and fill these in.
1 2, 3 4, 5 and 6.
And that was step five.
Step six is thinking about do we have any bonding electrons left?
Nope, we used them all up.
So we don't need to put any more bonds in there.
And step seven is how many electrons do we have left over that are going to go into lone pairs?
How many?
20.
26 minus 6, so that tells us 20, and these are all going to be lone pairs.
Well, we're talking about a pretty high number here, so to make counting easier, we'll just say 10 lone pairs, because 20 lone pair electrons is the same thing as 10 lone pairs.
And all we need to do is go over here now and fill up our octets.
So oxygen gets 3 pairs, and each chlorine gets 3 pairs, so now we're up to 9 pairs.
And what we have left here is the sulfur, which will also get a pair.
So, if you look at all of these, we have full octets for all of them, and if we count up all of the valence electrons, it's going to be equal to our number 26 here.
And the last thing we do for any of our structures to check them and figure out are these valid or not valid, are these good Lewis structures is to check the formal charge.
So now that we have enough practice drawing Lewis structures, let's talk about actually figuring out this formal charge.
So when we talk about formal charge, basically formal charge is the measure of the extent to which an individual atom within your molecule has either gained or lost an electron.
So as we said when we first introduced covalent bonds, it's a sharing of electrons, but it's not always an equal sharing.
Sometimes we have a very electronegative atom that's going to take more of its equal share of electron density.
So for example, that might have a formal charge of negative 1, because to some extent it has gained that much electron density that it now has a formal charge that's negative.
So, when we think about any type of formal charges, we have to assign these based on a formula here, which is very easy to follow.
Formal charge equals v minus l minus 1/2 s. It's even easier to follow if we know what all of those stand for.
So, f c, I think you all know is formal charge.
Does anyone have a guess for v?
Everyone has a guess, great.
Valence electrons.
What about l?
Lone pairs.
So, lone pair electrons, actually, not lone pairs themselves.
And then s?
Good.
That's a tricky one, shared electrons.
All right.
So this means we can actually calculate this for any molecule that we've drawn the Lewis structure for, because we actually do need to draw the Lewis structure before we know, for example, how many of each of these we have, or at least go through the rules.
And what's important to keep in mind about formal charge is if we have a neutral atom, such as we did in thionyl chloride here, the sum of the individual formal charges on individual atoms within the molecule have to equal 0.
So if we add them all up, there should be no net charge on the molecule, if the molecule is neutral.
So, if we think about the second case here where we have c n minus, now we're talking about a molecule with a net charge of negative 1.
So that means if we add up all of the formal charges within the molecule, what we would expect to see is that they sum up to give a net charge of negative 1.
So we can do this for any final charge we have, if we a molecule that has a charge of plus 2, then all of the formal charges should add up to plus 2 and so on.
So, let's just figure this out for some of the examples we did, so for the cyanide anion.
So, if we want to figure out the formal charge on the carbon, we need to take the number of valence electrons, so that's 4.
We need to subtract the lone pair, what number is that?
It's 2.
And then 1/2 of the number of shared electrons.
So, shared electrons are the ones that are shared between the carbon and the nitrogen, so we have 6 shared electrons, and we want to take 1/2 of that.
So we end up with a formal charge on carbon of negative 1.
We can do the same thing for nitrogen.
So in terms of nitrogen that starts off with a valence number of 5, again we have 2 lone pair electrons in the nitrogen, and again, we have 6 electrons that are shared.
So what we see is that the formal charge on the nitrogen is 0.
Also, formal charges can be checked, as I just said.
Negative 1 plus 0 should add up to negative 1, if in fact, we're correct for the c n anion.
And it does, so we know that we're probably on target in terms of calculating our formal charge.
So, let's think about our second example -- actually our third, but the second one we're going to talk about in terms of formal charge, which is thionyl chloride.
So why don't you tell me what the formal charge should be on the sulfur atom of thionyl chloride?
All right.
Let's take 10 more seconds.
OK, so the majority got it.
So hopefully next time we do a formal charge question, we'll get everyone back up to speed.
But we've just introduced it, so let's go back to the class notes and explain why this is the correct answer.
So if we look at sulfur, what we need to do is take the valence electrons in sulfur, and there are 6.
By looking at the periodic table it's right underneath oxygen, so those both have 6 valence electrons.
There are 2 lone pair electrons on sulfur -- we only have 2 lone pairs -- or 1 lone pair, 2 lone pair electrons.
And then we end up having 6 shared electrons, 2 from each of the bonds, so we end up with a formal charge on sulfur of plus 1.
If we go to the oxygen atom, now we're talking about starting with 6 in terms of valence electrons again, but instead of 2, you can see we have 6 lone pair electrons around the oxygen minus 1/2 of 2, so we have minus 1 is our formal charge.
And if we talk about chlorine, and both of the chlorines are the same in this case, we start with a valence number of 7 for chlorine, and then we subtract 6, because it had 6 lone pair electrons around each of the chlorine atoms.
Then minus 1/2 of 2, because we only have one bond or 2 electrons in a bond.
So again, we should be able to check all of our formal charges and make sure they add up to 0, which they do, and that makes sense, because we have a neutral atom in terms of thionyl chloride.
Another thing to mention in terms of thinking about if you have a good or a bad Lewis structure, is that when you figure out the formal charge on each of the atoms, it's the more electronegative atoms that you would expect to have that negative charge, and that should make sense to you because electronegative atoms want to have electron density, they want to pull in electron density to them, so it would make sense that they have more of it, which would give them a negative charge.
So, if we compare the sulfur to the oxygen, the oxygen it turns out is more electronegative and that is what holds the negative charge in this molecule.
Another thing I want to point out, and for some of you just ignore what I'm saying, if you haven't thought about oxidation number, if you haven't heard of that before, don't worry we'll get to it in the second half with Professor Drennan.
But for those of you from high school that have learned about oxidation number and maybe are starting to think about it when you look at these molecules, formal charge is -- it's not the same thing as oxidation number, so separate those two things in your head.
We'll get to oxidation number in the second half of this course, but it's not in any way the same idea as formal charge.
All right.
So formal charge can actually help us out when we're trying to decide between several Lewis structures that look like they might be comparable in terms of which might be the lower energy or the more stable structure.
The examples we've done so far have been pretty straightforward, so we haven't needed to use formal charge to make this kind of decision.
But we could, for example, look at a case where we have several different structures that look pretty good, and the one we want to determine as being the lowest energy structure is the one in which the absolute values of the formal charges are going to be lower, so essentially that they have less charge separation.
Those are going to be the more stable or the lower energy structures.
So, for example, let's look at thiocyanate ion, we have c s and n. What we've learned so far is as a first approximation, what we want to do is put the atom with the lowest ionization energy in the middle here.
So let's compare those ionization energies.
We have 10 90 for carbon, 1,000 for sulfur, and 1,400 for nitrogen.
So, thinking about ionization energy, which atom would you put in the middle here?
So, a lot of people I hear are saying sulfur, and that's right.
So, in terms of ionization energy, we would expect to see sulfur in the middle.
So if we went through and drew out our Lewis structure following each of our steps, what we would get is this as our Lewis structure here, and we could figure out all of the formal charges.
But what I'm going to tell you already is this is a case where, in fact, it's an exception to the idea that the lowest energy structure has the lowest ionization energy in the middle, and we can figure this out when we look at formal charge.
It's always a good first approximation, because you need to start somewhere in terms of drawing Lewis structures, but then if you go and figure out the formal charge and you just have lots of charge separation or very high charges, like a plus 2 and a minus 2 and a minus 1 all different places in the atom, what it should tell you is maybe there's a better structure.
So, let's think of all of the combinations that we could have in terms of this molecule.
So in one case, we could actually put carbon in the middle, in one place, we could put sulfur in the middle, and in one case we could put nitrogen.
And then we can go ahead and let's quickly write out what the formal charges for all of these will be.
So in our first structure, we would find for the nitrogen we have a formal charge 5 minus 4 minus 2, because we're starting with 5 valence electrons, so that is a formal charge of minus 1.
For the carbon, we start with 4 valence electrons, we have 0 lone pair electrons minus 4, and we end up with a formal charge of 0.
For the sulfur, we start off with 6 valence electrons, minus 4 lone pair electrons, minus 2, taking in account our bonding electrons, so we end up with a formal charge of 0.
All right, so this looks pretty good.
We don't actually have much charge separation in this case here.
Let's take a look at the lowest ionization energy in the center case.
And what you find out if you do these calculations, is that you have a negative 1 for your formal charge on nitrogen, you have a negative 2 for your formal charge on carbon, and you have a positive 2 for your formal charge on sulfur.
If we look at our last structure here where we have nitrogen the middle, we can also figure out all those formal charges, and in this case we have plus 1 on the nitrogen, we have minus 2 on the carbon, and then we end up with a 0 on the sulfur there.
So let's go to a clicker question.
If we call this structure a, b, and c -- you can look on your notes, it also says structure a, b, and c on your notes, so you didn't need to have just memorized that slide.
But I want you to tell me in terms of thinking about formal charge, which Lewis structure would you predict to be the most stable?
And this should be fast, so let's take 10 seconds on this.
Okay, great.
So let's go back to our notes here.
And the reason that this should be so fast is we already did all the calculations for the formal charges.
So what we see is that structure a is the most stable because we have the least separation of charge in the case of structure a.
And you can do this any time if you have Lewis structures that you're choosing between.
You won't always draw out every single possibility that you have to start with.
Often a good thing to start with is to put the lowest ionization energy atom in the middle, and if you don't have charge separation then go with that structure, but if you do find you have a lot of separation, such as the case in negative 2, positive 2, and minus 1, then you want to say wait a second, this is really bad in terms of formal charge, let me go ahead and see what other options I have here.
So, we can also get into a case where we have similar values in terms of absolute values of formal charge between two different molecules we're deciding between in their Lewis structures.
And in this case, the tie-breaker goes to the molecule in which the negative charge is on the most electronegative atom.
So, let's look at an example of this here.
So we could say, for example, this molecule here, this -- now we're dealing with a lot of different atoms in the molecule, much more complicated than the initial case of the cyanide ion where we only had two.
So, how do we figure out first how to draw the skeletal structure of this molecule here?
And one thing I want to tell you to start out with is something about this c h 3 group here.
Any time you see a c h 3, this means a methyl group.
And if you draw out what a methyl group is -- hopefully you won't have to spell it -- then what you have is a carbon in the middle with three hydrogens around it, and then it can only be bonded to one other thing.
So any time you see c h 3 here, remember that that's methyl and that's going to be a terminal group.
So, you already have a hint that methyl groups are never in the middle, they always have to be on the outside.
So that's a good start for us putting together a skeletal structure for this compound here.
The other tip I'm going to give you is any time you see a chain molecule, by chain I just mean many different atoms written out in a row.
A convention is that typically you will put a terminal atom.
We know that h is always terminal, it's always on the end, never in the center.
Right after the molecule that it's attached to.
So, for instance, this would suggest to us by the way it's written, that the hydrogen is attached to the nitrogen and not the oxygen.
So, if we use those two tips to try to figure out a structure, a skeletal structure, we would get this structure here if we write out the full Lewis structure.
We could I think well, maybe this isn't written out in terms of that convention, which sometimes it's not, so let's also try writing it, such that we have the hydrogen and the oxygen atom there.
So that would give us this structure here.
So notice a difference in these structures, is this has an n h bond whereas this has an o h bond.
And if we were to think about which one of these is better, it turns out that it's the same in terms of formal charges, so that doesn't help us out.
So we need to go to this second case where we're instead going to think about electronegativity, and we want to think about which atom is the most electronegative.
So, in this case, we see that our formal charge is negative on the nitrogen, in this case it's negative on oxygen.
Which of those two is more electronegative?
The oxygen.
So you should be able to look at your periodic table and see this, or also I've written the trend here.
So that means that the more stable molecule is going to be this molecule here, which actually puts the negative charge on be more electronegative atom.
So this is our lower energy structure.
So, these are the different ways that we can actually go ahead and use formal charge when we're choosing between different types of Lewis structures.
So one last concept that I want introduces is this idea of resonance.
And resonance is the idea that sometimes one single Lewis structure does not adequately describe the electron configuration around a given molecule, so instead you need to draw two different Lewis structures to describe that more appropriately.
So, let's quickly go through the Lewis structure for ozone.
I have the skeletal structure written up there, I've written it twice and you'll see why in a minute.
It's easy to write the skeletal structure, because it's all oxygen, we don't have to worry about what's going to go in the middle.
In this case, we're going to have 3 times 6 for valence electrons -- 6 valence electrons for each oxygen, so we have 18 total valence electrons.
So, in order to fill up our shell, what we need is 3 times 8 or 24 electrons.
This leaves us with 24 minus 18, or 6 bonding electrons left.
So what we can do is fill that in here.
Clearly, we put 2 for each bond, and now we end up having 2 remaining bonding electrons left.
So here is where our question comes, because where do we put it?
There's absolutely nothing that tells us which atoms we should put it between, because they're both oxygen-oxygen.
So, let's just arbitrarily put it between these two in this case here, but actually there's no reason we couldn't also put it between oxygen b and c, so I'm going to draw another structure where we have it here.
So in terms of remaining valence electrons we have 12, so we can finish off each of our Lewis structures, so that's our first structure there, and our second structure there.
We could also figure out the formal charges, and obviously the formal charges between these two atoms, they're going to be identical, we're only dealing with oxygen atoms here.
So, we need to think about what this means -- which is the more stable structure, because we have two different structures here.
In this case we have a double bond between a and b, and in this case we have it between b and c. So, presumably, if we follow our rules so far only one of these should be correct.
And again, if we figure out the formal charges, let's go through this quickly, we get 0 plus 1 and negative 1 for structure 1.
We get negative 1 plus 1 and 0 for structure 2.
So as I said, they're going to be identical in terms of making the decision that way.
And what it turns out is experimental evidence tells us that these two structures are equivalent.
And by that what we mean is that they're absolutely identical, and it turns out that this here is not a double bond, it's not a single bond either, it's actually something in between.
So if we look at its length, it's actually shorter than a single bond, but longer than a double bond.
Or if we look at how strong it is, it's actually stronger than a single bond, but weaker than a double bond.
And we find the same thing for these two atoms here, it's not actually a double bond, it's somewhere between a single bond and a double bond.
So this is a case where we have resonance structures, or we call this a resonance hybrid.
So the reality of the situation is that it's a combination between these 2 structures.
And an important thing to remember when we talk about resonance hybrids is that the structure it's not 1/2 the time this structure, and 1/2 of the time this structure, it's actually some combination or some average between the two structures.
But since in drawing Lewis structures we don't have a way to represent that -- actually, in some cases you do, you can draw a dotted line that means a 1 and 1/2 bond, but most in most cases, we just draw out both resonance structures, and the way that we say it's a resonance structure is that we put it in the brackets and we put an arrow between it.
So, when you think about resonance structures, some students tend to just get confused and be picturing this flickering back and forth.
A good example to keep in mind is the idea of a mule.
So most of you know, hopefully, that a mule is a combination of a donkey and a horse.
A mule is not spending 1/2 of its time as a donkey, and 1/2 of its time as a horse, we don't see it flickering back and forth between the two, that's not what we see.
Instead what we see is it's an average, it's part like a horse, it's part like a donkey.
So if we want to put that in chemical terms, we want to make sure we put these in brackets here, and remember, this is the resonance arrow, it's not a reaction arrow, it's a resonance arrow, so make sure you mark it up correctly like that.
When we talk about resonance structures, the key word is that the electrons are de-localized.
So they're not just between two atoms here.
Now, for example, in our structure with ozone it's between all three atoms.
And the final point I want to make today, and this is very, very important, so make sure that you do understand this.
When we talk about resonance structures, we're talking about cases that have the same arrangement of atoms, the key is the atoms are the same, and the thing that is different is the arrangement of electrons here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, let's get started here.
Go ahead and take 10 more seconds on the clicker question, which probably looks all too familiar at this point, if you went to recitation yesterday.
All right, and let's see how we do here.
OK.
So, let's talk about this for one second.
So what we're asking here, if we can settle down and listen up, is which equations can be used if we're talking about converting wavelength to energy for an electron.
Remember, the key word here is electron.
This might look familiar to the first part of problem one on the exam, and problem one on the exam is what tended to be the huge problem on the exam.
I think over 2/3 of you decided on the exam to use this first equation, e equals h c over wavelength.
So I just want to reiterate one more time, why can we not use this equation if we're talking about an electron?
C. OK, good, good, I'm hearing it.
So the answer is c. What you need to do is you need to ask yourself if you're trying to convert from wavelength to energy for an electron, and you are tempted, because we are all tempted to use this equation, and if you were tempted, say, does an electron travel at the speed of light?
And if the answer is no, an electron does not travel at the speed of light, light travels at the speed of light, then you want to stay away from using this equation.
And I know how tempting it is to do that, but we have other equations we can use -- the DeBroglie wavelength, and this is just a combination of energy equals 1/2 m v squared, and the definition of momentum, so we can combine those things to get it.
You might be wondering why I'm telling you this now, you've already -- if you've lost points on that, lost the points on it, and what I'm saying to you is if there are parts of exam 1 that you did not do well on, you will have a chance to show us again that you now understand that material on the final.
One quarter of the final is going to be exam 1 material, and what that means is when we look at your grade at the end of the semester, and we take a look at what you got on exam 1, and you're right at that borderline, and we say well, what happened, did they understand more at the end of the semester, did the concepts kind of solidify over the semester?
And if they did and if you showed us that they did, then you're going to get bumped up into that next grade category.
So keep that in mind as you're reviewing the exam, sometimes if things don't go as well as you want them to, the temptation is just to put that exam away forever and ever.
But the reality is that new material builds on that material, and specifically exam 1 a, question 1 a that deals with converting wavelength to energy for an electron.
I really want you guys know this and to understand it, so I can guarantee you that you will see this on the final.
Specifically, question 1, part a.
You will see something very, very similar to this on the final.
If you are thinking about 1 thing to go back and study on exam 1, 1 a is a really good choice for that.
This is important to me, so you're going to see it on the final.
So if you have friends that aren't here, you might want to mention it to them, or maybe not, maybe this is your reward for coming to class, which is fine with me as well.
All right.
So I want to talk a little bit about exam 1.
I know most you picked up your examine in recitation.
If you didn't, any extra exams can be picked up in the Chemistry Education office, that's room 2204.
So, the class average for the exam was a 68%, which is actually a strong, solid average for an exam 1 grade in the fall semester of 511-1.
What we typically see is something right in this range, either ranging from the 50's for an exam 1 average to occasionally getting into the 70's, but most commonly what we've seen for exam 1 averages is 60, 61 -- those low 60's.
So in many ways, seeing this 68 here, this is a great sign that we are off to a good start for this semester.
And I do want to address, because I know many of you, this is only your second exam at MIT, and perhaps you've never gotten an exam back that didn't start with a 90 or start with an 80 in terms of the grades.
So one thing you need to keep in mind is don't just look at the number grade.
The reason that we give you these letters grade categories is that you can understand what it actually means, what your exam score actually says in terms of how we perceive you as understanding the material.
So, for example, and this is the same categories that were shared in recitation, so I apologize for repeating, but I know sometimes when you get an exam back, no more information comes into your head except obsessing over the exam, so I'm just going to say it one more time, and that is between 88 and 100, so that's 20% of you got an A.
This is just absolutely fantastic, you really nailed this very hard material and these hard questions on the exam where you had to both use equations and solve problems, but also understand the concept in order to get yourself started on solving the problem.
The same with the B, the B range was between 69 and 87 -- anywhere in between those ranges, you've got a B, some sort of B on the exam.
So again, if you're in the A or the B category here, this is really something to be proud of, you really earned these grades.
You know these exams, our 511-1 exams, we're not giving you points here, there are no give me, easy points, you earned every single one of these points.
So, A and B here, these are refrigerator-worthy grades, hang those up in your dorm.
This is something to feel good about.
All right.
So, for those of you that got between a 51 and a 68, this is somewhere in the C range.
For some people, they feel comfortable being in the C range, other people really do not like being in this range.
We understand that, there is plenty of room up there with the A's and the B's.
You are welcome to come up to these higher ranges starting with the next exam.
And what I want to tell you if you are in the C range, and this is not a place that you want to be in, anyone that's got below the class average, so below a 68 -- or a 68 or below, is eligible for free tutoring, and I put the website on the front page of your notes.
This means you get a one-on-one tutor paid for by the Chemistry Department to help you if it's concepts you're not quite up on, if it's exam strategy that you need to work on more.
Whatever it is that you need to work on, we want to help you get there.
So, if you have a grade that you're not happy with, that you're feeling upset or discouraged about, please, I'm happy to talk to all of you about your grades individually.
You can come talk to me, bring your exam, and we'll go over what the strategy should be in terms of you succeeding on the next exam.
You can do the same thing with all of your TAs are more than happy to meet with each and every one of you.
And then in addition to that, we can set you up with a tutor if you are in the C range or below, in terms of this first exam.
All right.
So 44 to 50, this is going to be in the D range.
And then anything below a 44 is going to be failing on this exam.
And also keep in mind, for those of you that are freshman, you need at least a C to pass the class.
So, if you did get a D or an F on the first exam, you are going to need to really evaluate why that happened and make some changes, and we're absolutely here to help you do that.
So the real key is identifying where the problem is -- is it with understanding the concepts, are you in a study group that's dragging you along but you're not understanding?
Do you kind of panic when you get in the exam?
There are all sorts of scenarios we can talk about and we want to talk about them with you.
Seriously, even if we had a huge range in this exam from 17 to 100, if you're sitting there and you're the 17, and actually there's more than 1 so don't feel alone, if you're a 17 or you're a 20, it's not time to give up, it's not time to drop the class and say I'm no good at chemistry, I can't do this.
You still can, this is your first couple of exams, certainly your first in this class, potentially one of your first at MIT, so there's tons of room to improve from here on out.
This is only 100 points out of 750.
So, the same thing goes if you did really well, you still have 650 other points that you need to deal with.
So, make sure you don't just rest on your high score from this first exam.
So, OK, so that's pretty much what I wanted to say about the exam, and in terms of there's tons of resources if things didn't work out quite as you wanted.
If you feel upset in any way, please come and talk to me.
We want you to love chemistry and feel good about your ability to do it.
Nobody get into MIT by mistake, so you all deserve to be sitting here, and you all can pass this class and do well in it, so we can help you get there no matter what.
You all absolutely can do this.
And then one more time, to reiterate, in case anyone missed it, 1 a, make sure you understand that, I feel like that's important.
And actually all of 1 -- I really feel like the photoelectric effect is important for understanding all of these energy concepts.
So, as you go on in this class, make sure you don't go on before you go back and make sure you understand that problem.
All right, so let's move on to material for exam 2 now, and we're already three lectures into exam 2 material.
And I do want to say that in terms of 511-1, what tends to happen is the exam scores go up and up and up, in terms of as we go from exam 1, to exam 2, to exam 3.
One of these reasons is we are building on material, the other reason is you'll be shocked at how much better you are at taking an exam just a few weeks from now.
So this will be on, starting with the Lewis structures, so go back in your notes -- if this doesn't sound familiar, if you spent too much time -- or not too much time, spent a lot of time studying exam 1 and didn't move on here.
Today we're going to talk about the breakdown of the octet rule.
Cases where we don't have eight electrons around our Lewis structures, then we'll move on to talking about ionic bonds.
We had already talked about covalent bonds, and then we talked about Lewis structures, which describe the electron configuration in covalent bonds.
So now let's think about the other extreme of ionic bonds, and then we'll talk about polar covalent bonds to end, if we get there or will start with that in class on Monday.
Also, public service announcement for all of you, voter registration in Massachusetts, which is where we are, is on Monday, the deadline if you want to register to vote.
There's some websites up there that can guide you through registering and also can guide you through, if you need an absentee ballot for your home state.
And I actually saw, and I saw a 5.111 student manning, there's some booths around MIT that will register you or get you an absentee ballot.
So, the deadline's coming soon, so patriotic duty, I need to remind you of that as your chemistry teacher -- chemistry issues are important in politics as well.
So make sure you get registered to vote.
I just remembered one more announcement, too, that I did want to mention, some of you may have friends in 511-2 and have heard their class average for exam 1.
And I want to tell you, this happens every year, their average was 15 points higher than our average.
Last year, their average was 15 points higher than our average.
This is for exam 1.
This is what tends to happen to 511-2 grades as the exam goes on.
This is what happens to 511-1.
You guys are in a good spot.
Also, I want to point out that what's not important is just that number grade, but also the letter that goes with it.
So, for example, if you got a 69 in this class on this exam, that's a B minus.
If you got a 69 on your exam in 511-2, that's a D, you didn't pass the exam.
So keep that in mind when your friend might have gotten a higher number grade than you and you know you understand the similar material just as well.
Similarly, an 80 in this class on the exam was a B plus, a very high B.
An 80 in that class is going to be a C. So, just don't worry so much about exactly where that average lies, you really want to think about what the letter grade means.
OK, I've said enough.
I just -- I hate to see people discouraged, and I know that a few people have been feeling discouraged, so that's my long-winded explanation of exam 1 grades.
All right.
So, let's move on with life though, so talking about the breakdown of the octet rule.
The first example where we're going to see a breakdown is any time we have an odd number of valence electrons.
This is probably the easiest to explain and to think about, because if we have an odd number that means that we can't have our octet rule, because our octet rule works by pairing electrons.
And if we have an odd number, we automatically have an odd electron out.
So, if we look at an example, the methyl radical, we can first think about how we draw the Lewis structure -- we draw the skeletal structure here.
And then what we're going to do is add up our valence electrons -- we have 3 times 1 for the hydrogen atoms, carbon has 4 valence electrons, so we have a total of 7.
If we want to fill all of our valence shells in each of these atoms, we're going to need a total of 14 electrons.
So, what we see we're left with is that we have 7 bonding electrons.
So we can fill in 6 of those straightforward here, because we know that we need to make 3 different bonds.
And now we're left over with 1 electron, we can't make a bond.
So, what we'll do is carbon does not have an octet yet.
We can't get it one, but we can do the best we can and help it out with adding that extra electron onto the carbon atom, so that at least we're getting as close as possible to filling our octets.
This is what we call a radical species or a free radical.
Free radical or radical species is essentially any type of a molecule that has this unpaired electron on one of the atoms.
This might look really strange, we're used to seeing octets.
But you'll realize, if you calculate the formal charge on this molecule, that it's not the worst situation ever for carbon.
At least it's formal charge is zero, even if it doesn't have -- it would rather have an extra bond and have a full octet.
But it's not the worst scenario that we can imagine.
But still, radicals tend to be incredibly reactive because they do want to fill that octet.
So, what happens when you have a radical is it tends to react with the first thing that it runs into, especially highly reactive radicals that are not stabilized in some other way, which you'll tend to talk about it organic chemistry -- how you can stabilize radicals.
So the term free radical should sound familiar to you, whether you've heard it in chemistry before, or you haven't heard it in chemistry, but maybe have heard it, I don't know, commercials for facial products or other things.
People like to talk about free radicals, and they're sort of the hero that gets rid of free radicals, which are antioxidants.
So you hear in a lot of different creams or products or vitamins that they have antioxidants in them, which get rid of free radicals.
The reason you would want to get rid of free radicals is that free radicals can damage DNA, so they're incredibly reactive.
It makes sense that if they hit a strand of DNA, they're going to react with the DNA, you end up breaking the strands of DNA and causing DNA damage.
So, this is actually what happens in aging because we have a lot of free radicals in our body.
We can introduce them artificially, for example, cigarette smoke has a lot of really dangerous free radicals that get into the cells in your lungs, which damage your lung DNA, which can cause lung cancer.
But also, all of us are living and breathing, which means we're having metabolism go on in our body, which means that as we use oxygen and as we metabolize our food, we are actually producing free radicals as well.
So it's kind of a paradox because we need them because they are a natural by-product of these important processes, but then they can go on and damage cells, which is what kind of is causing aging and can lead to cancer.
We have enzymes in our body that repair damage that is done by free radicals, that will put the strands of DNA back together.
And we also have antioxidants in our body.
So, you might know that, for example, very brightly colored fruit is full of antioxidants, they're full of chemicals that will neutralize free radicals.
Lots of vitamins are also antioxidants, so we have vitamin A on the top there and vitamin E. So, the most common thing we think of when we think of free radicals is very reactive, bad for your body, causes DNA damage.
But the reality is that free radicals are also essential for life.
So this is kind of interesting to think about.
And, for example, certain enzymes or proteins actually use free radicals in order to carry out the reactions that they carry out in your body.
So, for example, this is a picture or a snapshot of a protein, this is a crystal structure of ribonucleotide reductase is what it's called.
It's an enzyme that catalyzes the reaction of an essential step in both DNA synthesis and also DNA repair, and it requires having radicals within its active site in order to carry out the chemistry.
So, this is kind of a neat paradox, because radicals damage DNA, but in order to repair your DNA, you need certain enzymes, and those enzymes require different types of free radicals.
So, free radicals are definitely very interesting, and once we get -- or hopefully you will get into organic chemistry at some point and get to really think about what they do in terms of a radical mechanism.
We can think about radicals that are also more stable, so let's do another example with the molecule nitric acid.
So we can again, draw the skeleton here, and just by looking at it we might not know it's a radical, but as we start to count valence electrons, we should be able to figure it out very quickly, because what we have is 11 valence electrons.
We need 16 electrons to have full octets.
So, we're left with 5 bonding electrons.
We put a double bond in between our nitrogen and our oxygen, so what we're left over with is this single bonding electron, and we'll put that on the nitrogen here.
And I'll explain why we put it on the nitrogen and not the oxygen in just a minute.
But what we find is then once we fill in the rest of the valence electrons in terms of lone pairs, this is the structure that we get.
And if you add up all of the formal charges on the nitrogen and on the oxygen, what you'll see is they're both 0.
So if you happen to try drawing this structure and you put the lone pair on oxygen and then you figured out the formal charge and saw that you had a split charge, a plus 1 and a minus 1, the first thing you might want to try is putting it on the other atom, and once you did that you'd see that you had a better structure with no formal charge.
I have to mention what nitric oxide does, because it's a very interesting molecule.
Don't get it confused with nitrous oxide, which is happy gas, that's n o 2.
This is nitric oxide, and it's actually much more interesting than nitrous oxide.
It's a signaling molecule in your body, it's one of the very few signaling molecules that is a gas, and obviously, it's also a radical.
What happens with n o is that it's produced in the endothelium of your blood vessels, so the inner lining of your blood vessels, and it signals for smooth muscle that line your blood vessels to relax, which causes vasodilation , and by vasodilation, I just mean a widening of the blood vessels.
So, n o signals for your blood vessels to get wider and allow more blood to flow through.
And if you think about what consequences this could have, in terms of places where they have high altitude, so they have lower oxygen levels, do you think that they produce more or less and n o their body?
More?
Yeah, it turns out they do produce more.
The reason they produce more is that they want to have more blood flowing through their veins so that they can get more oxygenated blood into different parts of their body.
N o is also a target in the pharmaceutical industry.
A very famous one that became famous I guess over 10 years ago now, and this is from a drug that actually targets one of n o's receptors, and this drug has the net effect of vasodilation or widening of blood vessels in a certain area in the body.
So this is viagra, some of you may be familiar, I think everyone's heard of viagra.
Now you know how viagra works.
Viagra breaks down, or it inhibits the breakdown of n o's binding partner in just certain areas, not everywhere in your body.
So, in those areas, what happens is you get more n o signaling, you get more vasodilation, you get increased blood flow.
So that's a little bit of pharmacology for you here today.
All right, so let's talk about one more example in terms of the breakdown of the octet rule with radicals.
Let's think about molecular oxygen.
So let's go ahead and quickly draw this Lewis structure.
We have o 2.
The second thing we need to do is figure out valence electrons.
6 plus 6, so we would expect to see 12.
For a complete octet we would need 8 electrons each, so 16.
So in terms of bonding electrons, what we have is 4 bonding electrons.
So, we can go ahead and fill those in as a double bond between the two oxygens.
So, what we end up having left, and this would be step six then because five was just filling in that, is 12 minus 4, so we have 8 lone pair electrons left.
So we can just fill it in to our oxygens like this.
All right, so using everything we've learned about Lewis structures, we here have the structure of molecular oxygen.
And I just want to point out for anyone that gets confused, when we talk about oxygen as an atom, that's o, but molecular oxygen is actually o 2, the same for molecular hydrogen, for example.
All right, so let's look at what the actual Lewis structure is for molecular oxygen, and it turns out that actually we don't have a double bond, we have a single bond, and we have two radicals.
And any time we have two radicals, we talk about what's called a biradical.
And while using this exception to the Lewis structure rule, to the octet rule for odd numbers of valence electrons can clue us into the fact that we have a radical, there's really no way for us to use Lewis structures to predict when we have a biradical, right, because we would just predict that we would get this Lewis structure here.
So, when I first introduced Lewis structures, I said these are great, they're really easy to use and they work about 90% of the time.
This falls into that 10% that Lewis structures don't work for us.
It turns out, in order to understand that this is the electron configuration for o 2, we need to use something called molecular orbital theory, and just wait till next Wednesday and we will tell you what that is, and we will, in fact, use it for oxygen.
But until that point, I'll just tell you that molecular orbital theory takes into account quantum mechanics, which Lewis theory does not.
So that's why, in fact, there are those 10% of cases that Lewis structures don't work for.
All right, the second case of exceptions to the octet rule are when we have octet deficient molecules.
So basically, this means we're going to have a molecule that's stable, even though it doesn't have a complete octet.
And these tend to happen in group 13 molecules, and actually happen almost exclusively in group 13 molecules, specifically with boron and aluminum.
So, any time you see a Lewis structure with boron or aluminum, you want to just remember that I should look out to make sure that these might have an incomplete octet, so look out for that when you see those atoms.
So, let's look at b f 3 as our example here.
And what we see for b f 3 is the number of valence electrons that we have are 24, because the valence number of electrons for boron is 3, and then 3 times 7 for each fluorine.
For total filled octets we need 32, so that means we need 8 bonding electrons.
So, let's assign two to each bond here, and then we're going to have two extra bonding electrons, so let's just arbitrarily pick a fluorine to give a double bond to.
And then we can fill in the lone pair electrons, we have 16 left over.
So thinking about what the formal charge is, if we want to figure out the formal charge for the boron here, what we're talking about is the valence number for boron, which is 3, minus 0 because there are no lone pairs, minus 1/2 of 8 because there are eight shared electrons.
We get a formal charge of minus 1.
What is our formal charge since we learned this on Monday for thinking about the double bonded fluorine in boron?
So, look at your notes and look at the fluorine that has a double bond with it, and I want you to go ahead and tell me what that formal charge should be.
All right, let's take 10 more seconds on that.
OK, so 49%.
So, let's go look back at the notes, we'll talk about why about 50% of you are right, and 50% need to review, which I totally understand you haven't had time to do yet, your formal charge rules from Monday's class, there were other things going on.
But let's talk about how we figure out formal charge.
Formal charge is just the number of valence electrons you have.
So fluorine has 7.
You should be able to look at a periodic table and see that fluorine has seven.
What we subtract from that is the number of lone pair electrons, and there are four lone pair electrons on this double bonded fluorine, so it's minus 4.
Then we subtract 1/2 of the shared electrons.
Well we have a double bond with boron here, so we have a total of 4 shared electrons.
And when we do the subtraction here, what we end up with is a formal charge plus 1 on the double bonded fluorine.
Without even doing a calculation, what do you think that the formal charge should be on you single bonded fluorines?
Good.
OK, it should be 0 and it is 0.
The reason it's zero in terms of calculating it is 7 minus 6 lone pair electrons minus 1/2 half of 2 shared electrons is 0.
The reason that you all told me, I think, and I hope, is that you know that the formal charge on individual atoms has to equal the total charge on the molecule.
So if we already have a minus 1 and a plus 1, and we know we have no charge in the molecule, and we only have one type of atom left to talk about, that formal charge had better be 0.
OK.
So this looks pretty good in terms of a Lewis structure, we figured out our formal charges.
These also look pretty good, too, we don't have too much charge separation.
But what actually it turns out is that if you experimentally look at what type of bonds you have, it turns out that all three of the b f bonds are equal in length, and they all have a length that would correspond to a single bond.
So, experimentally, we know we have to throw out this Lewis structure here, we have some more information, let's think about how this could happen.
So this could happen, for example, is if we take this two of the electrons that are in the b f double bond and we put it right on to the fluorine here, so now we have all single bonds.
And let's think about what the formal charge situation would be in this case here.
What happens here is now we would have a formal charge of 0 on the boron, we'd have a formal charge of 0 on all of the fluorine molecules as well.
So, it turns out that actually looking at formal charge, even though the first case didn't look too bad, this case actually looks a lot better.
We have absolutely no formal charge separation whatsoever.
It turns out again, boron and aluminum, those are the two that you want to look out for.
They can be perfectly happy without a full octet, they're perfectly happy with 6 instead of 8 in terms of electrons in their valence shell.
So that is our exception the number two.
We have one more exception and this is a valence shell expansion, and this can be the hardest to look out for, students tend to forget to look for this one, but it's very important as well, because there are a lot of structures that are affected for this .
And this is only applicable if we're talking about a central atom that has an n value or a principle quantum number that's equal to or greater than three.
What happens when we have n that's equal to or greater to three, is that now, in addition to s orbitals and p orbitals, what else do we have available to us?
D orbitals, great.
So what we see is we have some empty d orbitals, which means that we can have more than eight electrons that fit around that central atom.
If you're looking to see if this is going to happen, do you think this would happen with a large or small central atom?
So think of it in terms of just fitting.
We've got to fit more than 8 electrons around here.
Yeah, so it's going to be, we need to have a large central atom in order for this to take place.
Literally, we just need to fit everything around is probably the easiest way to think about it.
And what happens is it also tends to have small atoms that it's bonded to.
Again, just think of it in terms of all fitting in there.
So, let's take an example p c l 5.
The first example is the more straightforward example, because let's start to draw the Lewis structure, and what we see is that phosphorous has five chlorines around it.
So we already know if we want to form five bonds we've broken our octet rule.
But let's go through and figure this out and see how that happens.
What we know is we need 40 valence electrons, we have those -- 5 from the phosphorous, and we have 7 from each of the chlorine atoms.
If we were to fill out all of those octets, that would be 48 electrons.
So what we end up with when we do our Lewis structure calculation is that we only have 8 bonding electrons available to us.
So we can fill those in between the phosphorous and the chlorine, those 8 bonding electrons.
So, this is obviously a problem.
To make 5 p c l bonds we need 10 shared electrons, and we know that that's the situation because it's called p c l 5 and not p c l 4, so we can go right ahead and add in that extra electron pair.
So we've used up 10 for bonding, so that means what we have left is 30 lone pair electrons, and I would not recommend filling all of these in your notes right now, you can go back and do that, but just know the rest end up filling up the octets for all of the chlorines.
So, in this first case where you actually need to make more than for bonds, you will immediately know you need to use this exception to the Lewis structure octet rule, but sometimes it won't be as obvious.
So, let's look at c r o 4, the 2 minus version here, so a chromate ion, and if we draw the skeletal structure, we have four things that the chromate needs to bond to.
So, let's do the Lewis structure again.
When we figure out the valence electrons, we have total, we have 6 from the chromium, we have 6 from each of the different oxygens, and where did this 2 come from?
Yup, the negative charge.
So, remember, we have 2 extra electrons hanging out in our molecule, so we need to include those.
We have a total of 32.
40 are needed to fill up octets.
So again, we have 8 bonding electrons available, so we can go ahead and fill these in between each of the bonds.
What happens is that we then have 24 lone pair electrons left, and we can fill those in like this.
And the problem comes now when we figure out the formal charge.
So, when we do that what we find is that the chromium has a formal charge of plus 1, and that each of the oxygens has a total charge of minus 1.
So we actually have a bit of charge separation here.
Without even doing a calculation, what is the total charge of these that are added up?
OK, it's minus 2, that's right.
We know that the total charge of each of the formal charges has to add up to minus 2, because that's the charge in our molecule.
We can also just calculate it -- the chromate gives us a plus 2, then we have 4 times minus 1 for each of the oxygens, so we have a minus 2.
So, we have some charge separation here, and in some cases, if we're not at n equals 3 or higher, there's really nothing we can do about it, this would be the best structure we can do.
But since we have these d orbitals available, we can use them, and it turns out that experimentally this is what's found, that the length and the strength are not single bonds, but they're actually something between a single bond and a double bond.
So how do we get a 1 and 1/2 bond, for example, what's the term that let's us do that?
Resonance.
That's right.
So that's exactly what's happening here.
So, if we went ahead and drew this structure here where we have now two double bonds and two single bonds, that would be in resonance with another structure where we have two double bonds instead to these two oxygens, and now, single bonds to these two oxygens.
We can actually also have several other resonance structures as well.
Remember, the definition of a resonance structure is where all the atoms stay the same, but what we can do is move around the electrons -- we're moving around those extra two electrons that can be in double bonds.
So, why don't you tell me how many other resonance structures you would expect to see for this chromate ion?
All right, let's take 10 more seconds on this.
All right.
This is good.
I know this is a real split response, but the right answer is the one that is indicated in the graph here that it's four.
This takes a little bit of time to get used to thinking about all the different Lewis structures you can have.
So, you guys should all go back home if you can't see it immediately right now and try drawing out those four other Lewis structures, for chromate, there are four others.
You'll probably get a chance to literally do this example in recitation where you draw out all four, but it's even better to make sure you understand it before you get to that point.
So, we can go back to the class notes.
So it turns out there's four other Lewis structures, so basically just think about all the other different combinations where you can have single and double bonds, and when you draw those out, you end up with four.
So, for every single one of these Lewis structures, we could figure out what the formal charges are, and what we would find is that it's 0 on the chromium, it's 0 for the double bonded oxygens, and it's going to be negative 1 for the single bonded oxygens.
So, what you can see is that in this situation, we end up having less formal charge separation, and that's what we're looking for, that's the more stable structure.
So any time you can have an expanded octet -- an expanded valence shell, where you have n is equal to or greater than 3, and by expanding and adding more electrons into that valence shell, you lower the charge separation, you want to do that.
I also want to point out, I basically said there's 6 different ways we can draw this in terms of drawing all the resonance structures.
You might be wondering if you have to figure out the formal charge for each structure individually, and the answer is no, you can pick any single structure and the formal charges will work out the same.
So, for example, if you pick this structure and your friend picks this structure, you'll both get the right answer that there's just the negative 1 on the oxygens and no other formal charges in the molecule.
All right.
So those are the end of our exceptions to the octet rule for Lewis structures, that's everything we're going to say about Lewis structures.
And remember, that when we talk about Lewis structures, what they tell us is the electron configuration in covalent bonds, so that valence shell electron configuration.
So we talked a lot about covalent bonds before we got into Lewis structures, and then how to represent covalent bonds by Lewis structures.
So now I'll say a little bit about ionic bonds, which are the other extreme, and when you have an ionic bond, what you have now is a complete transfer of either one or many electrons between two atoms.
So the key word for covalent bond was electron sharing, the key word for ionic bonds is electron transfer.
And the bonding between the two atoms ends up resulting from an attraction that we're very familiar with, which is the Coulomb or the electrostatic attraction between the negatively charged and the positively charged ions.
So let's take an example.
The easiest one to think about is where we have a negative 1 and a positive 1 ion.
So this is salt, n a c l -- actually lots of things are call salt, but this is what we think of a table salt.
So, let's think about what we have to do if we want the form sodium chloride from the neutral sodium and chlorine atoms.
So, the first thing that we're going to need to do is we need to convert sodium into sodium plus.
What does this process look like to you?
Is this one of those periodic trends, perhaps?
Can anyone name what we're looking at here?
Exactly, ionization energy.
So, if we're going to talk about the energy difference here, what we're going to be talking about is the ionization energy, or the energy it takes to rip off an electron from sodium in order to form the sodium plus ion.
So, we can just put right here, that's 494 kilojoules per mole.
The next thing that we want to look at is chlorine, so in terms of chlorine we need to go to chlorine minus, so we actually need to add an electron.
This is actually the reverse of one of the periodic trends we talked about.
Which trend is that this is the reverse of?
Electron affinity, right.
Because if we go backwards we're saying how badly does chlorine want to grab an electron?
Chlorine wants to do this very badly, and it turns out the electron affinity for chlorine is huge, it's 349 kilojoules per mole, but remember, we're going in reverse, so we need to talk about it as negative 349 kilojoules per mole.
So if we talk about the sum of what's happening here, what we need to do is think about going from the neutrals to the ions, so we can just add those two energies together, and what we end up with is plus 145 kilojoules per mole, in order to go from neutral sodium in chlorine to the ions.
So, the problem here is that we have to actually put energy into our system, so this doesn't seem favorable, right.
What's favorable is when we actually get energy out and our energy gets lower, but what we're saying here is that we actually need to put in energy.
So another way to say this is this process actually requires energy.
It does not emit energy, it does not give off excess energy, it requires energy.
So, we need to think about how can we solve this problem in terms of thinking about ionic bonds, and the answer is Coulomb attraction.
So there's one more force that we need to talk about, and that is when we talk about the attraction between the negatively and the positively charged ions, such that we form sodium chloride.
So this process here has a delta energy, a change in energy of negative 589 kilojoules per mole.
So that's huge, we're giving off a lot of energy by this attraction.
So if we add up the net energy for all of this process, all we need to do is add negative 589 to plus 145.
So what we end up getting is the net energy change is going to be negative 444 kilojoules per mole, so you can see that, in fact, it is very favorable for neutral sodium and neutral chloride to form sodium chloride in an ionic bond.
And the net increase then, is a decrease in energy.
So, I just gave you the number in terms of what that Coulomb potential would be in attraction, but we can I easily calculate it as well using this equation here where the energy is equal to the charge on each of the ions, and this is just multiplied by the value of charge for an electron divided by 4 pi epsilon nought times r, are r is just the distance in terms of the bond length we could talk about.
So, let's calculate and make sure that I didn't tell you a false number here.
Let's say we do the calculation with the bond length that we've looked up, which is 2 .
3 6 angstroms for the bond length between sodium and chloride.
So we should be able to figure out the Coulombic attraction for this.
So, if we talk about the energy of attraction, we need to multiply plus 1, that's the charge on the sodium, times minus 1, the charge on the chlorine, times the charge in an electron, 1 .
6 0 2 times 10 the negative 19 Coulombs, and that's all divided by 4 pi, and then I've written out epsilon nought in your notes, so I won't write it on the board.
And then r, so r is going to be 2 .
3 6 and times -- what is angstrom, everyone?
Yup, 10 to the negative 10.
So 10 to the negative 10 meters.
So, if we do this calculation here, what we end up with is negative 9.774 times 10 to the negative 19 joules.
So that's what we have in terms of our energy.
That does not look the same as what we saw -- yup, do you have a question?
[INAUDIBLE]
OK. Luckily, although, I did not write it in my own notes, I did it when I put in my calculator, thank you.
So you need to square this value here and then you should get this value right here, negative 9.77.
All right, so what we need to do though is convert from joules into kilojoules per mole, because that's what we were using.
So if we multiply that number there by kilojoules per mole -- or excuse me, first kilojoules per joule, so we have 1 joules in every kilojoule.
And then we multiply that by Avagadro's number, 6.022 times 10 to the 23 per mole.
What we end up with is negative 589 kilojoules per mole.
So this is that same Coulombic attraction that we saw in the first place.
So, notice that you will naturally get out a negative charge here, remember negative means an attractive force in this case, because you have the plus and the minus 1 in here.
So we should be able to easily do that calculation, and what we end up getting matches up with what I just told you, luckily, and thank you for catching the square, that's an important part in getting the right answer.
So, experimentally then, what we find is that the change in energy for this reaction is negative 444 kilojoules per mole.
If we look experimentally what we see, it's actually a little bit different, it's negative 411 kilojoules per mole.
So, in terms of this class, this is the method that we're going to use, and we're going to say this gets us close enough such that we can make comparisons and have a meaningful conversations about different types of ionic bonds and the attraction between them.
But let's think about where this discrepancy comes from, and before I do that I want to point out, one term we use a lot is change in energy for a reaction where, for example, you break a bond.
Remember that the negative of the change in energy is what's called delta e sub d. We first saw this when we first introduced the idea of covalent bonds.
Do you remember what this term here means, delta e sub d?
A little bit and some no's, which this was pre-exam, I understand, you still need to review those notes, it's dissociation energy.
So you get a negative energy out by breaking the bond.
The dissociation energy means how much energy that bond is worth in terms of strength, so it's the opposite of the energy you get out of breaking the bond -- or excuse me, the energy that you get out of forming the bond.
It's the amount of energy you need to put in to break the bond is dissociation energy.
It takes this much energy to dissociate your bond, excuse me.
All right.
So, let's take a look here at our predictions, so I just put them both ways so we don't get confused.
The dissociation energy is 444.
The change in energy for forming the bond is negative 444.
We made the following approximations, which explain why, in fact, we got a different experimental energy, if we look at that.
The first thing is that we ignored any repulsive interactions.
If you think about salt, it's not just two single atoms that you are talking about.
It's actually in a whole network or whole lattice of other molecules, so you actually have some other chlorines around that are going to be having repulsive interactions with our chlorine that we're talking about.
We're going to ignore those, make the approximation that those don't matter, at this point, in these calculations.
And the result for that is that we end up with a larger dissociation energy than the experimental value.
That's because the bond is going to be a little bit more broken than it was in our calculation, because we do have these repulsive interactions.
The other thing that we did is that we treated both sodium and the chlorine as point charges.
And this is what actually allowed us to make this calculation and calculate the Coulomb potential so easily, we just treated them as if they're point charges.
We're ignoring quantum mechanics in this -- this is sort of the class where we ignore quantum mechanics, we ignored it for Lewis structures, we're ignoring it here.
We will be back to paying a lot of attention to quantum mechanics in lecture 14 when we talk about MO theory, but for now, these are approximations, these are models where we don't take it into consideration.
And I think you'll agree that we come reasonably close such that we'll be able to make comparisons between different kinds of ionic bonds.
All right.
So, the last thing I want to introduce today is talking about polar covalent bonds.
We've now covered the two extremes.
One extreme is complete total electron sharing -- if we have a perfectly covalent bond, we have perfect sharing.
The other is electron transfer in terms of ionic bonds.
So when we talk about a polar covalent bond, what we're now talking about is an unequal sharing of electrons between two atoms.
So, this is essentially something we've seen before, we just never formally talked about what we would call it.
This is any time you have a bond forming between two non-metals that have different electronegativities, so, for example, hydrogen choride, h c l. The electronegativity for hydrogen is 2.2, for chlorine it's 3.2.
And in general, what we say is we consider a difference in terms of a first approximation if the difference in electronegativity is more than 0.
5, so this is on the Pauling electronegativity scale.
So what we end up having is we sort of have a kind of, and what we call it is a partial negative charge on the chlorine, and a partial positive charge in the hydrogen.
The reason we have that is because the chlorine's more electronegative, it wants to pull more of that shared electron density to itself.
If it has more electron density, it's going to have a little bit of a negative charge and the hydrogen's going to be left with a little bit of a positive charge.
So, we can compare this, for example to, molecular hydrogen where they're going to have that complete sharing, so there's not going to be a delta plus or a delta minus, delta is going to be equal to zero on each of the atoms.
They are completely sharing their electrons.
And we can also explain this in another way by talking about a dipole moment where we have a charged distribution that results in this dipole, this electric dipole.
And we talk about this using the term mu, which is a measurement of what the dipole is.
A dipole is always written in terms of writing an arrow from the positive charge to the negative charge.
In chemistry, we are always incredibly interested in what the electrons are doing, so we tend to pay attention to them in terms of arrows.
Oh, the electrons are going over to the chlorine, so we're going to draw our arrow toward the chlorine atom.
So, we measure this here, so mu is equal to q times r, the distance between the two.
And q, that charge is just equal to the partial negative or the partial positive times the charge on the electron.
So this is measured in Coulomb meters, you won't ever see a measurement of electronegativity in Coulomb meters -- we tend to talk about it in terms of debye or 1 d, or sometimes there's no units at all, so the d is just assumed, and it's because 1 debye is just equal to this very tiny number of Coulomb meters and it's a lot easier to work with debye's here.
So, when we talk about polar molecules, we can actually extend our idea of talking about polar bonds to talking about polar molecules.
So, actually let's start with that on Monday.
So everyone have a great weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
-- 10 seconds to answer the first clicker question.
As you can see we're having another competition today, so see if you can beat out Justin's recitation and get the most correct today.
So, this is a topic from Friday, which is asking us which of the following molecules are free radical species.
OK.
So it looks like most and you got it, that it's o h. Let's quickly go over why that is just in case you weren't one of that 74%.
So, if we're looking at o h versus c o, what are we looking at to decide if we have a radical here?
Valence electrons.
So we have 6 plus 1 in terms of o h, that's 7, versus having 4 plus 6, which is 10.
Specifically, what do we look for in terms of valence electrons?
Odd numbers.
So, we have a radical here with o h, whereas we have an even number of valence electrons for c o, so that's not a radical, in terms of what we know how to think about so far, which is using our Lewis structures to determine that.
All right.
So that's a little review from Friday, let's move on to today's lecture.
In terms of what we're going to be talking about today, we're going to finish up with talking about polar covalent bonds, and finish up with discussing the idea of polar molecules.
And then we'll move on to start talking about well, we know how to draw our Lewis structures, how can we use those Lewis structures to think about the actual shapes of molecules.
So, first thinking about our polar covalent bonds.
What we had established is if we have a case where we have a covalent bond, but there's unequal sharing of electrons between the two atoms, namely because they have different electronegativities, what we see is we have a polar bond or a polar covalent bond where one of the atoms is going to pull away more of the electron density from that bond, giving it that partial negative charge where the other atom is going to end up with a partial positive charge.
In terms of thinking about well, what do we call a covalent bond versus a polar covalent bond versus a purely ionic bond, it turns out that the distinction is a little bit fuzzy, it's not a clear line, but in terms of talking about things in general, and especially in thinking about problems for this course, what we're going to say is if you have an electronegativity or chi difference between the two atoms that is greater than 0 .
4, and less than 1 .
7, and we're talking about the Pauling electronegativity scale here, which is what's used throughout your book.
If you have a difference in electronegativity anywhere in that range, we'll have what we call a polar covalent bond between those two atoms.
So, for example, if we compare the chi values for hydrogen versus carbon, we see they only have a difference of 0.4 .
0.4 is not greater than 0.4 .
So we would not call this a c h bond, we would not say it's polar covalent, we would just call it covalent.
In contrast, if we talk about a carbon oxygen bond, now we have a chi difference of 0.8 , so that is greater than 0.4 , so we'll call carbon oxygen bond polar covalent.
We can extend this idea of talking about these polar covalent bonds to thinking about an entire molecule.
I'm sure you've all heard of the term a polar molecule versus a non-polar molecule, and essentially, when we talk about having a polar molecule, what we're saying is that there's is a net non-zero dipole moment within the whole molecule.
So essentially, what we have to do is combine all of the individual bonds to think about the molecule as a whole.
So, let's take an example of carbon dioxide here, or c o 2.
Very soon in the lecture you'll learn how to predict shapes based on Lewis structures.
So you will quickly see that this is a linear molecule.
We can think about the 2 bonds within it.
There's two carbon oxygen bonds, and we know that there is a dipole in that bond, which is going toward the oxygen.
Remember, in chemistry we always draw the arrow, we're always interested in what those electrons are doing.
So if we draw in the bond dipoles, and you can draw these into your notes, you want to draw an arrow towards the oxygen in each case.
But we're not just interested here in thinking about do we have a polar bond, we actually want to know in general do we have a polar molecule.
So, looking at c o 2 as a whole, do you think we have a polar or a non-polar molecule here?
Non-polar.
OK, very good for all of you that said non-polar.
The reason it's non-polar is we simply have two equal vectors in opposite directions, so they cancel each other out.
The net effect is that we have a 0 dipole moment in the c o molecule.
In contrast, we can look at h 2 o. H 2 o actually has this bent shape, and again, we'll see very soon and how to predict that h 2 o has a bent shape, that water is bent.
And again, we do have dipole moments -- we have bonds where there is a polar bond and we should point our arrow toward the oxygen, because electron density is being pulled from the hydrogen atoms to the oxygen.
And again, we can cancel some of these vectors out, but we still have a net dipole moment that is going to be going up if we think about combining these 2 vectors here.
So, what we would say, which is what we know from everything we've always heard, which is that water is a polar molecule.
So, this is a great way to think about molecules in general.
It's very easy to think about if those vectors cancel each other out or if they're additive when we're talking about just single bonds, so we have two atoms or if we have just three atoms, as we do in these cases here.
But it turns out, a lot of the molecules we consider are actually much, much, much larger than just a couple of atoms.
So we can't often think about canceling all the different vectors out.
And in general, when we're talking about these really large molecules, whether they're large organic molecules or we're talking about proteins, which can have thousands of atoms in them, instead of adding up all those individual polar bonds, what we do is we talk about the number of polar groups that it will have in the molecule.
So we can think about a protein, for example, as having a lot of different polar groups or one that has not too many polar side groups.
That's one way you hear people talking about proteins, and that has a lot to do with their solubility in water, and also about how the protein's going to fold.
But let's look at a little bit about less complicated example than a protein with thousands of different atoms.
Let's instead consider a couple of vitamins here, which instead have a few dozen different atoms in them.
Specifically, let's look at vitamin A and vitamin B9.
Does anyone know another name for vitamin B9?
I don't think I hear it -- does anyone say folic acid.
It's also sometimes called vitamin M. It's one of the vitamin B9 vitamins, there's actually other forms of vitamin B9, but this is one of the big ones that you hear about.
Hopefully you're all very familiar with folic acid, because it's an important vitamin to take, especially if you're a woman, especially if there's any chance you ever might become pregnant in any kind of near future time, by accident or on purpose.
And the reason for this is because folic acid deficiency in pregnant women leads to neural tube defects in babies, and your brain develops very, very, very early in pregnancy, the brain of an embryo.
So it turns out that a lot of women don't realize they're pregnant while the fetus' brain is developing, and if these women are not getting enough folic acid, you can end up with spina bifida, which is an absolutely devastating and very preventable disease or other neural tube defects.
So hopefully, if you're not familiar with folic acid, you'll become familiar at least within the next 10 or 15 years or so before you're thinking about maybe, on purpose, becoming pregnant at some time.
Vitamin A you probably all heard about growing up, getting enough of your carrots so you can see at night.
Vitamin A is important for eye health.
But just looking at the structure of these two molecules, knowing what we know so far about general chemistry, we can already say a lot about how these are going to function the body or how they'll be treated in the body.
And specifically, let's take a look at which of these two molecules has more polar bonds and see what that tells us.
So take a look at your two structures and tell me which has, between folic acid and vitamin A, more polar groups within the vitamin.
And this should go pretty quickly since you don't need to have an exact number, we're just looking and glancing and seeing which has more.
So let's take 10 seconds on this.
All right, great.
So most you said folic acid or vitamin B9, so let's switch back to our notes and take a look at why.
So vitamin B9 has more polar bonds.
It's very easy to see, once I highlight, that it's not even a close call at all.
We have a bunch of different polar bonds in B9 versus just one in vitamin A.
Remember, the c h bond we're not calling polar covalent because it only has an electronegativity difference of 0.4 between the two atoms.
So what we know is that vitamin B9, folic acid, is more polar.
Remember, we also just said that water is polar, so would we say that B9 is water or fat soluble?
It's going to be water soluble -- everyone knows the saying, "like dissolves like."
B9 is going to be very water soluble.
This actually is very important if you think about taking your vitamins.
This means if it's water soluble, any vitamins that are water soluble, and now you should to be able to look at the structure of any vitamin and tell me, for example, that vitamin C is water soluble, that folic acid is water soluble.
If you take these vitamins, what happens is that they're very quickly and easily excreted into your urine.
So, for example, some people like to take mega doses of vitamin C. What really happens is that you have a mega dose of vitamin C in your urine, it doesn't stick around long in your body.
So it doesn't help and just take all of your vitamin C at once.
That's why it's important to eat balanced meals throughout the day, you need to be getting a constant supply of these water soluble vitamins.
The same is for folic acid, you can't just take it once a month, you need to be taking it regularly in order that you keep the stores up in your body, otherwise you're going to excrete it, you're going to get rid of it very quickly.
In contrast, if we think about vitamin A, is this going to be water soluble or fat soluble?
Yup, so this is the fat soluble vitamin.
Vitamin E is another big one that's fat soluble that gets a lot of press in terms of being an important vitamin to take.
We can think about what it means when a vitamin is fat soluble instead of water soluble.
Well, now it's not going to get excreted out of our body so quickly, so we actually can build up amounts of vitamin A vitamin E, for example, but that can also pose a problem if you think about biologically what's happening.
And up here I just show two different supplements that I found on the internet.
This is One A Day vitamin, it has about 100% of what you need of everything.
And then this vitamin is one that I found that's supposed to really help you with your eye health, if you have bad vision instead of glasses, they suggest that you try this vitamin here.
And what you can see is that it has five times your daily value of vitamin A, and 13 times what you need of vitamin E. Is this a good idea?
No.
Basically, you are just building up more and more and more of these fat soluble vitamins into your body.
And there have been studies that have come out recently trying to look at the health benefits of vitamin E, and in some of these studies they give these mega doses, and instead what they find is increased bleeding in these patients, an increase in overall different types of death that can be happening.
You want to take your vitamins, it's very important, but you don't want it just build up, and you want to think about is the vitamin that I'm going and getting 800 times what I need a day, is that a water soluble vitamin or a fat soluble vitamin, and using your chemistry, you should be able to very quickly take a quick look at that structure and figure out what kind of a vitamin that is.
All right.
So that's just one reason we would want to be able to think about polarity is thinking about whether something is very water soluble or not.
Let's think about some other things that have to do with polarity and then can tell us a lot of other information.
So let's move on to talking about the shapes of molecules.
And the shapes of molecules is very important for a number of different properties when we're thinking about chemical reactions and reactions that take place in the body.
When we talk about shapes of molecules, we're talking about the geometry of that molecule.
And the geometry influences all of its different properties, including things like melting point or boiling point, it's reactivity.
We just saw when talking about polar molecules, that it influences whether or not the molecule itself is polar or apolar.
It's also really important when we talk about biology.
Shape is particularly important when you think about enzymes having an active site where a molecule needs to fit perfectly into the active site, it does that because of its shape.
A quick example that we can think about is with sucrose.
Does anyone know what sucrose is?
It's just table sugar -- sucrose is that crystal in sugar that we hopefully use in most of our sugar intake as sweeteners.
A lot of times we use corn syrup, which is often not sucrose and which tends to be not always as good for us, although anything in too much excess is obviously a non-ideal situation.
But in order for us to use the energy from sucrose, sucrose is made up of two individual sugar monomers.
It's made up of a monomer of glucose and one of fructose, a 6 and a 5 membered ring, so in order for our body to use it, we need to break it into with its monomers.
And if we that, we can do it by what's called hydrolysis -- sucrose and water breaks down into its two monomers, now our body can use it.
The problem is this process takes on the order of 10 or maybe 100 years in order for us to get enough of the sugar broken down.
That's why we need an enzyme molecule, and the enzyme in our body is called sucrase.
Sucrase breaks down sucrose, it catalyzes that reaction, so it happens very quickly.
And the key is, and here's a ball and stick shape of what sucrose looks like.
It needs to fit exactly into that active site.
When it does, it combines into the enzyme, the enzyme can then catalyze the hydrolysis or the breaking apart of those two individual monomers into its separate pieces.
And then it let's go of the glucose and the fructose, and we get our enzyme back again, and something else combined into that active site.
So, this is one very simple example of why molecular shape is very important.
A lot of times you're thinking about small molecules interacting with proteins or interacting with other molecules, and you want to think about the shape of that molecule to think about how that interaction is going to take place.
So what we're going to use to think about molecular shape or molecular geometry is what's called valence shell electron repulsion or vsper theory.
And this theory is very convenient for us to be thinking about because we are now masters of drawing Lewis structures, and vesper theory is based on Lewis structures, and also the principle that when we have valence electron pairs, they're going to repel each other.
This make sense any time you have negatively-charged electrons, you want to get them as far away from each other as possible because they're all negatively charged.
And the other principle is that the geometry around that central atom, which you've identified as the central atom in the Lewis structure, is going to be such that it minimizes the repulsion between those either bonding electrons or the electron pairs.
So when we talk about vsper there's a special nomenclature that we do use, and in vsper we have a -- does anyone know what a means in vsper theory?
It's a central atom.
What about x?
So this is actually the bonding, any bonding atom you call x.
And then we have e, which is equal to lone pairs.
So e is a lone pair -- e is not a lone pair electron, so if you have two electrons, that's one lone pair.
All right.
So there are some guidelines that we use when we're coming up with these vsper geometries.
The first thing we talk about is the steric number, which we use to predict what that geometry will be.
When we're talking about steric number, all we're talking about is adding together the number we have of bonded atoms, plus the number of lone pairs.
So, essentially were just adding x to e and that gives us our steric number.
So, for example, if we look at this molecule, which is a x 2 e, what is the steric number here?
Yeah, it's 3, so we have a steric number of 3.
Something that I want to point out is that we could have a different molecule, which is also a x 2 e, but in this case we have a double bond between the central atom and one of the x atoms here, and what I want to point out is in vsper theory we treat bonds as bonds, we don't worry about if they're single or they're double or they're triple.
So again, we call this a x 2 e, this again, has a steric number of 3.
So what's important in terms of vsper theory is the number of atoms bonded to a central atom.
What's not important is the types of bonds that we're dealing with.
A few other guidelines I want to mention is first of all, to think about resonance structures.
So on Friday we were talking about the chromate anion, and we said that here are two of its resonance structures here, but that we could actually draw four more resonance structures, and I just want to point out when you take a molecule that has many different resonance structures and you want to draw its vsper, it's Lewis structure and its vsper geometry, you can take any single one of those resonance structures, it doesn't matter, you'll all end up with the same geometry.
And lastly, if we're talking about a molecule that has more than one central atom, which is what we're very, very often doing, you need to deal with each one separately.
So for example, with methanol here, we have a carbon, which is a central atom, and we also have an oxygen, which is a central atom, you need to talk about the geometry separately, we don't talk about the geometry of the entire molecule.
So, let's go ahead and look at some of these vsper examples, and Professor Drennan is going to help demonstrate what some of these are with some models here.
So the first case we're going to talk about is without any lone pairs -- this is the most straightforward case.
And our first case that we can have is a x 3, which is going to have a linear shape, and that will have a bond angle of a 180 degrees.
The next case that we can talk about is trigonal planar or a x 3.
Now, you do need to know these geometry, you need to know the names.
A lot of them are very easy to remember.
I expect not too many of you will get linear incorrect.
Also, trigonal planar, pretty easy.
Trigonal, it has 3 atoms around the central atom and the molecule is planar
So, what is the bond angle for this geometry?
120
All right.
Great.
So, next we can think about a x 4, and this is what's called a tetrahedral geometry.
If you're trying to remember this name you can think of each of those bonding atoms as being in the corner of tetrahedron.
One thing I want to point out is you're seeing some notation we haven't used before in this class where you have a wedge coming out at you, and you also have a dashed line to one of those bonds.
Any time you see a wedge in a Lewis structure, it means that it's coming out at you -- it's either coming out of the screen or it's coming out of your paper.
And any time you see a dashed line here, which might be easier to see in your notes, that means that the bond is actually going into the page or into the screen or into the blackboard
And what are the angles for a tetrahedral?
All right, so they're 1 0 9 .
5.
Pretty close.
So that's as far away as those bonds can get from each other would be 109 .
5 degrees.
The next case we have is a x 5.
That is what we call trigonal bipyramidal.
Again, we have trigonal, because we have those 3 bonds in the center, and if you can picture putting walls between all those bonds, you can see there's a pyramid on the top and a pyramid on the bottom, so bipyramidal
And there are 2 sets of angles here, what are the angles for equatorial atoms?
120.
And then for axial?
90
And again, axial, those are just the ones in the axis, and equatorial if you put your globe around the equator.
So the last case we have is a x 6.
A x 6 is what we call octahedral -- you can picture each of those bonded atoms as a corner of an octahedron
And what are the angles in this, this one set of angles?
90
Great.
So, 90 degrees.
All right.
So this is all straightforward when we're just thinking about different molecular shapes that don't have any lone pairs in them, but once we start having lone pairs, we now need to think about how those lone pairs are going to affect the geometry.
But before we do that, let's talk about some examples of molecules without lone pairs.
So the first case is what we saw at the beginning of class, carbon dioxide, and we said that that was linear, and now we know why it's linear -- it has a formula of a x 2, and an s n number of 2.
And that's a bond angle of 180 degrees.
So, you can tell us about borane, borane is a x 3
And what is the geometry?
And the angle?
120.
Feel free to yell out very loudly, we want people in OpenCourseWare to hear the answers coming from the room
All right.
So the next case we're going to look at is c h 4 or methane, and let's do a clicker question here to make sure everyone is remembering these geometries.
And this should be very quick, and try to not turn the page back one, and tell us in 10 seconds here what the geometry is.
98%, that is a new record.
Very good.
Tetrahedral
All right, while I'm holding carbon dioxide in one hand and methane in the other hand, I just want to do an additional plug for the energy debate tonight.
So there will be representatives from both presidential campaigns that are going to be there to answer questions.
And I'm not sure of the exact format, I'm not sure whether all audience members are going to be able to ask questions or not.
I'm not sure if I'm going to be there, I'm supposed to be giving a talk actually, ironically, about energy at the same time.
But if I can't make it and you're allowed to ask questions, I'd like you to ask a question for me.
So, my lab looks at enzymes that remove carbon monoxide and carbon dioxide from the environment.
And I have an energy initiative grant to do this, because MIT recognizes that an important part of any energy initiative is thinking about lowering pollutants and greenhouse gases.
So, I heard on Thursday at the debate when Governor Sarah Palin was asked directly if she really doesn't think there's any connection between global warming and any man-made activities, that she said, well, you know, it could be just regular temperature fluctuations, and she doesn't want to point any fingers.
So, she's not quite sure.
I also learned on Thursday that she will head the energy initiative in a McCain/Palin White House.
So, if you go to this debate, I would like you to ask what is your plan for sequestering carbon dioxide and other greenhouse gases.
You can ask both candidates, and please let me know what the answer is, because I think that's a really important question -- here we have carbon dioxide, here we have methane, we need to be thinking about greenhouse gases
You can also throw in the question that you do know this is tetrahedral and 109 .
5 degrees -- show your chemistry knowledge here.
All right, and let's keep showing that we know our geometries, let's look at p c l 5
And what is geometry here?
Yup.
And the angles?
120 and 90
So, 120 equatorial, and 90 axial.
And last we have s f 6
So, what is the geometry here?
Octahedral.
And angle?
90
All right, so there's our set of examples, one for each for those shapes or geometries that have no lone pairs.
And now let's think a little bit about once we do have molecules with lone pairs.
And the biggest point to keep in mind when we're comparing lone pair electrons with bonding electrons or electrons in bonds, is that when you have electrons in bonds they have less spatial distribution than lone pairs.
So that's just a way of saying that electrons and bonds, they take up less space.
So, when we think about lone pair electrons, they're taking up more space and this means that they're going to experience more repulsion in thinking about the lone pair electrons with either other lone pair electrons or with bonding electrons.
So let's think a second about the order that this will be in.
The biggest repulsion we would feel is if we have two different lone pair electrons or lone pairs.
They're going to have the most repulsion.
In the middle is if we're talking about a lone pair with a bonding pair.
And then the least repulsion is going to be between two bonds or two bonding pairs of electrons.
So, let's look at an example of where this comes up.
So the first example I'm going to talk about is a molecule that has the geometry of a seesaw shape, and once we get to having Professor Drennan actually show you that shape, you will never forget.
See-saw, that's going to be one of the easy geometries to remember.
But the first thing that we need to consider in terms of the shape, which starts, if you picture first of all, and these models sometimes don't quite stay together, we could actually have two different possibilities.
The first is the idea that we could have an axial lone pair, and the second is the possibility that we could have an equatorial lone pair.
And we could consider both, and we should be able to use our vsper principles to think about which one is actually going to happen.
So if we think about having an axial lone pair here, that would mean that these lone pair electrons are going to be within 90 degrees of three different loan pairs, and actually we'll see later, it will be more than 90 because they're actually going to push them down.
But in terms of considering how many bonding electron pairs they'll repel strongly, what we care about is anything within 90 degrees initially.
So, what we would see with the axial lone pair is that we have three lone pairs that we're going to very strongly repel.
Now would you tell me, if we think about an equatorial lone pair, how many different bonding electrons or a bonding pairs that will strongly repel if we have an equatorial lone pair?
So you might need to look at your notes to actually compare these two before you submit your answer up here.
All right.
Let's take 10 seconds on that.
OK, strong showing with the clicker questions today.
It's going to be a tight race for clicker competition.
It's correct that the equatorial is, in fact, has two, only two that it strongly repels.
So, actually it's probably easier to look at Professor Drennan's model here to see that.
So it's where the equatorial lone pair is that is the actual seesaw geometry
So, how many of you had seesaws in playgrounds when you were growing up?
Oh, a lot of people.
I know that they're not considered totally safe anymore, so some of them are going away.
So this is seesaw.
Now you should never forget it
All right, so seesaw, we've got seesaw.
Just to point out a few more shapes.
We took one out, we replaced one bond with a lone pair in terms of seesaw.
If we have a x 3 e 2 now we have two equatorial lone pairs.
This is called the T-shaped
T-shaped
See, these aren't too hard to remember the geometries, the geometry names.
And we also can think about if we have a x 4 e 2.
So now in order to get our lone pairs as far away from each other as possible, they're going to be on the axial position, and this is called square planer
So, this is easy to remember, square, also planar
All right.
So, let's talk a little bit now about what the actual angles are between bonds when now we have these lone pairs present.
And remember, what we said is when we have a lone pair of electrons, they actually are going to have more repulsion than if we just have a c h bond or a bonded pair of a electrons.
So essentially what that means is that in molecules that have lone pair electrons, so for example, if we look at n h 3 or ammonia versus methane here, and Professor Drennan is showing those models to you here.
What you end up seeing is actually that the bonding angle in n h 3 between the n h bonds is smaller.
And you can't see it with these models because these are not actual lone pairs.
But if they were actually repelling each other, they would be pushing those bonds as far away as possible, and instead of being 109 .
5, you'd see that the angle is now 106 .
7.
So it's a smaller angle between bonds, because you have more repulsion from those lone pairs pushing those bonds down.
We can also think about the influence of atomic size in terms of this effect.
So first of all, what happens to atomic size as you go down the periodic table?
Good, it increases.
So we have size increasing as we go down the periodic table.
Phosphorous is right underneath nitrogen on the periodic table, so phosphorous is going to be bigger than nitrogen.
In terms of picturing what happens with those lone pairs, when we have a larger atom, the orbitals are also going to be larger, they can take up more space.
That means these electrons, this lone pair electrons are going to take up more space.
This means they're going to push away those bonding electrons even more.
So would you expect the bond to be larger or smaller for p h 3?
[INAUDIBLE]
It's going to be smaller.
So, what we see is that angles between bonded atoms actually decrease as you go down a column on the periodic table.
So the actual angles for p h 3 are now going to be really quite a bit smaller, they're 93 .
3 degrees.
All right.
So let's create a list for ourselves in terms of all the geometries that we can have now that we're dealing with lone pairs.
One thing I want to point out in terms of remembering the names of the different geometries, when you're naming a geometry, the geometry name, for example, when we looked at square planar, we're only actually naming where the bonds are, where actual atoms are.
The name doesn't really depend on the lone pairs.
And also, when you draw a geometry, you don't always have to draw the lone pairs in, but you have to remember that the lone pairs are very much affecting the angles within your molecule and also the actual shape.
So, for example, let's start talking about different types that have lone pairs in it.
First of all, thinking about a x 2 e, so we have one lone pair, and the geometry here is bent, and I want you, thinking about lone pair repulsion, to tell us what you think the angle between these bonds are going to be?
And let's take 10 seconds on that.
OK, good.
75%, that's not bad.
Let's think about why.
It's going to be less than 120 degrees, because we know that normally in a trigonal planar situation, we have an angle of 120 degrees.
And since lone pairs are going to cause more repulsion, we're actually pushing down these two bonds here closer together, so what we end up seeing is that they're going to be less than 120 degrees.
And it depends on the actual molecule what the exact angles are going to be, so you never have to learn the exact angles in terms of lone pair electrons, you just need to be able to tell us if the bond angle is going to be less than 120 degrees.
So let's look at another example here, a x 3 with one lone pair, e. This is called trigonal pyramidal.
Again, you have trigonal because there's three atoms bonded to the central atom, and this looks like a pyramid here
So, what would the angle of this be?
[INAUDIBLE]
I think I heard it
I think so, too.
Less than 109 .
5.
Let's take another example, a x 2 e 2.
This is also bent.
Tell us what the geometry -- we told you the geometry, tell us what the bond angle is going to be between these bonds.
So, let's take 10 seconds on this.
Let's re-poll F 4.
So 10 seconds again.
All right.
In about 10 seconds, Darcy, in 10 seconds we'll hit the next side.
OK, so it's less than 109 .
5 degrees now.
So some of you wrote that it was less than 120 degrees, and we can think about if we switch back to the class notes the difference here.
So even though, so this is now going to be less than 109 .
5, if we looked at the case we had bent in the first case, that started with 120 and one of those bonds was replaced with a lone pair, but now we have two lone pairs here, so now what we're going to see is that the two bents are not, in fact, equal.
The bent where you started with the tetrahedral shape is actually going to be less than 109 .
5, and Professor Drennan can maybe show us that with the models here
It's a little had to see, actually, between the two, but I think it's easier to see on this, that if you consider the sort of starting place for the two, if you look at the bottom set and you put orbitals, so it also depends on your starting geometry.
It can be bent if take off these bonds on the top, but it depends on what the starting geometries were.
And so you will go with your starting geometry, if it was 109 .
5, it'll be less than that.
If it's 120, then it's less than that
So, this is a good case of where, even when you have the geometry the same, you need to think about how many lone pairs you're dealing with in your molecule, and think about what would the geometry be if I just pictured those lone pairs in there as bonds and then push them even closer together so that it's less than that actual angle.
OK, let's talk about a x 4 e, and that is what we call the seesaw shape
And so, what difference in angles are you going to have?
[INAUDIBLE]
Yup.
PROFESSOR 2: So, two sets again, one is less than 120, and one is going to be less than 90.
So, let's look at a x 3 e 2, you'll notice there's a lot more combinations we can get to once we're talking about lone pairs.
This is called T-shaped, and we had briefly discussed that but not mention what the angles would be
So, what angle would you have here?
Yup
All right, great.
Less than 90 degrees.
So, we'll start a new page here and talk about a x 2 e 3, and we'll let you tell us what this geometry is.
Yup, so I hear a lot of linear out there.
And you need to keep in mind we give the names based on where the actual bonds are, not where the lone pairs are, even though the lone pairs, of course, affect the structure in the geometry
The angle?
180
Yup, 180
So, it's exactly 180 here, not less than
So, for a x 5 e, what we have here is called square pyramidal.
This is square because of the four square at the bottom of your pyramid, then you can picture the pyramid as you go to the top
And so what angles would you have here?
Yup, less than 90
So, less than 90, great.
All right, there are more.
So, our next combination that we can think about is if we have four bonded atoms and two lone pairs, a x 4 e 2, that's going to be called square planar
So, again, we have square and it's planar.
And so what are your angles here?
Yup, exactly 90
So next let's look at a x 3 e 3.
This is what we call T-shape, this is also T-shape
And angle here?
Yup, I think we heard less than, it's less than 90 degrees.
And let's look at a x 2 e 4, and again, you guys can tell us what this geometry is, first of all.
Yup, it's linear.
So, we have lots of different ways we can get to a linear molecule, some of which have lots of lone pairs, some of which have no lone pairs at all.
This is linear and a 180 degrees.
All right.
So we'll do just a few examples to show you some actual molecules that have these geometries.
We won't go through all of them, because as we saw, there's lots of different combinations we can have once we start getting into lone pairs.
The first we'll look at is water here, and water is a good one to look at because we had seen that at the beginning of class here when we were talking about all polar molecules.
When we talk about the formula type of water, what when you say the formula type is?
[INAUDIBLE]
A x 2 e 2, that's correct.
And what is the geometry of water?
It's bent.
So what we saw at the beginning of class is water is bent and now you can see why and should be able to predict that yourself.
I just want to mention that if you look, and depending on the edition of the book you have, in one edition it's called bent, in the other edition it's called angular.
We'll call it bent, but just remember that bent and angular, they're the same geometry.
Let's look at another example, s f 4.
And here it is drawn here.
What is the formula type for s f 4?
I think I heard it, a x 4 e. And the geometry?
Seesaw, good.
All right, we never get seesaw wrong, that's the easy give-away one.
All right, what about b r f 3?
This is the shape here.
What is the geometry for b f 3 -- the original geometry, not the new geometry here?
What was that?
T-shaped, that's right.
B f 3 is T-shaped.
All right, so let's look at xenon f 2.
That's going to have a formula type of a x 2 e 3, if we actually look at the Lewis structure here.
So if you think about what the geometry is, and you should just be able to look at this and see, what would you say the geometry of xenon f 2 is?
Linear
Linear, good.
All right.
And let's try one more here, which is a x 4 e 2, or a xenon f 4 -- very explosive, but a good example
Square planar
Square planar, all right, great.
So, in general, when we think about vsper theory, even though it doesn't talk about or tell us anything about the energies of these different shapes, it's very useful in making a first approximation and coming very close to thinking about what the actual shapes of molecules are.
So one thing I want to point out in terms of what you're responsible for is you should be able to fill out one of these charts with only seeing the labels up here -- tell us the formula type, tell us the steric number, tell us the geometry, you should be able to draw the Lewis structure, you already know how to do that, and also talk about the angles.
So, as you finish up your problem-set, you can, I think now, get through just about all of it.
You can do now this part with the geometry.
So, we'll see you on Wednesday -- we're going to get out a little bit early today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
-- that this is not a quick clicker question, so you actually need to maybe write something down to figure out the answer.
So if you haven't clicked in yet, or if you want to change your answer, keep in mind that you might need to jot down, for example, a Lewis structure before you can answer this question.
-- and get in your final answer to the clicker question.
For those of you just walking in now, you might not have a chance to get all of the thought process that you need in on this clicker question, because it is based on a Lewis structure, so we will go over it.
But for those of you that have been here for 30 seconds or longer, see if you can get the right answer in here.
All right, so OK, it looks like we have a very mixed response in terms of the answer to the clicker question today.
Raise your hand if you didn't have time to figure out the Lewis structure.
OK, so that accounts for some of you.
Let's go over the correct answer this question.
So, who got the right answer -- 31% of us.
This is not the kind of percentages we're looking for, so let's go over this.
In class on Monday, we did go over the geometries, and the geometries themselves are very straightforward, once you know what the Lewis structure is, but remember, you can't just always look at a molecule and automatically know the Lewis structure.
We actually need to think about those valence electrons.
So, here we're dealing with selenium hydride, so s e h 2.
I told you that there's 6 valence electrons in selenium, so we have 6 there, plus 1 for each of the hydrogen.
So what we should have is 8 valence electrons in our Lewis structure.
How many electrons do we need to have full valence shells?
[INAUDIBLE]
12, that's right.
We need 8 plus 4 is 12 for full shells.
That means that we need 12 minus 8, or 4 bonding electrons in our structure.
So I tried to give you one that you could draw pretty quickly, it just has hydrogens in It.
So we have s e h 2.
We can do our 4 bonding electrons.
How many valence electrons do we have left?
[INAUDIBLE]
4, that's right.
So, we need to fill our octet for selenium, so 1, 2, 3, 4.
So, this is our Lewis structure here, hopefully you can see why it's not linear.
If it were linear, which 32% of you seem have thought, that would have meant that our Lewis structure had no lone pairs in it, right, and that's not the case.
We have two lone pairs, so if we thought about what the bonds were everywhere, it would be 109 .
5, but it's bent because we're only looking at the bonds, we're not counting the Lewis structures in naming our geometry, but they do affect the angles.
And it turns out that it's actually less than 109 .
5, because those lone pairs are pushing the bonds even further away.
So, does this make sense to everyone if you think about it this way?
Pretty much, okay.
So, we'll try another clicker question like this later.
Some of these questions that look very straightforward just naming the geometry, you have to remember to do the first step before you jump in and go ahead with a naming.
So you'll got plenty of practice with this on your problem-set if you haven't already.
All right.
So before we start in with today's notes, I do want to mention that this morning the Nobel Prize in chemistry was announced.
This is an exciting week in science in general because we got to hear another Nobel Prize every morning pretty much.
So, let's settle down for a second and start listening.
All right.
So, today we're going to be talking about molecular orbital theory, but first I wanted to just mention, in case some of you didn't hear what the Nobel Prize was this morning, and this was in chemistry, it went to three different chemists.
Osamu Shimomura, who's a Japanese chemist, and then Martin Chalfi who's at Columbia, and Robert Chen who's at U. C. San Diego.
The three of these chemists split the Nobel Prize this morning for their discovery and/or their application of using green fluorescent protein, which is also called GFP.
How many of you have heard of GFP before?
Oh, that's so great.
OK.
Many of you have heard of GFP.
For some of you that haven't I'll just say that it's a protein, it's 238 amino acids, which means that it's about 1,000, actually more than 1 atoms in size, and this protein is fluorescent.
The protein was first discovered and first isolated from this jellyfish here.
This was done by Shimomura, and he did this in the 60's and 70's.
He actually recognized and, of course, many people recognized that this jellyfish was fluorescent, and he isolated the actual protein, and determined and proved that this protein was sufficient to cause this fluorescence.
So, in terms of thinking about applications of why it's so exciting that you have this fluorescent protein, other than it's always really fun to look at things that are fluorescent, we can think about in terms of biology why it's so exciting, you can actually tag this protein to any other protein that you're studying and now you have a visual handle on what's going on.
So, for example, if you were interested in some protein involved in cancer, you could tag it with GFP.
You could watch where it localizes in the cell, there are fluorescent assays you could use to determine what other proteins it interacts with.
You could see, for example, when it's expressed in a cancer cell, and where it's expressed.
There are all sorts of things you can do once you can visualize something with fluorescence.
The reason it is so exciting that it's a protein, and it's a protein, this is the structure here, it's a ribbon structure so you can kind of see what it looks like, it's made up of all natural amino acids.
So this means we can code for it in DNA, you don't have to worry how am I going to get into the cell.
All you have to do is mutate the DNA, which is very straightforward to do in molecular biology, and now you can tag absolutely any protein that you're interested in.
So, as I said, this was first discovered and isolated from the jellyfish.
This was done by Shimomura -- that was kind of the first step in this process of having it become such a useful tool.
And then, many years later, not until 1994 did Martin Chalfi, at Columbia, show that yes, I can, in fact, take the DNA and put it into a different organism, and he put it into e coli, a bacteria.
And what he could show was that it could be expressed, this is a picture from his 1994 science paper, in that e coli, and it is going to fluoresce green.
Now, the first application tends to not be quite as exciting as, for example, all the other organisms people have put it in since then -- you can have flies, you see transgenic mice that are glowing green with this GFP.
Of course, that's not the useful application for it, it's more of a proof of principle, but it does show you that you can put it in for studies in any organisms.
And the field was really pushed forward by the discoveries of Robert Chen at UC San Diego, and what he did was he actually figured out how it was that this protein fluoresced, what caused the actual fluorescence.
And once he did that, both he and many other scientists, could then, once they understood what caused the fluorescence, make little changes to the actual protein, and tune what the properties of that fluorescent protein were.
So now, for example, there are a whole range, just a whole rainbow of fluorescent proteins that can be used.
And I'm sure you can imagine that if you want to label onw protein green and one red and one yellow, now you can start looking at really complex biological processes.
So, it's pretty rare that chemistry makes the every day news.
So hopefully you'll all look in the normal papers today, not just the scientific journals, and get to read something about chemistry.
It's always fun to see how its described in The New York Times or in The Boston Globe.
The other thing I wanted to mention and I'm not sure if the exhibit's still there.
But there was an exhibit of jellyfish, I know at least until last year, at the Boston Museum of Science, and all of you guys can get in there free.
And it's neat to see the glowing jellyfish and think about the fluorescent protein that's in them.
So, I encourage you to do that the next time you have some free time on your hands, maybe at IAP or some time like that.
All right, so let's move into today's notes.
Today we're talking about molecular orbital theory.
This is a shift, this is a new topic that we're starting.
So far we've exclusively been using Lewis structures any time we've tried to describe bonding within molecules.
Lewis structures are really useful, we use them all the time in chemistry.
And they're useful because, first of all, they're easy to depict, they're easy to draw -- relatively easy once we get all the rules down.
And also they're accurate over 90% of the time.
But they're not accurate all the time in predicting bonding within molecules, and the reason for this is because Lewis structures are not, in fact, based on quantum mechanics.
So, molecular orbital theory, on the other hand, is based on quantum mechanics.
And specifically, MO theory is the quantum mechanical description of wave functions within molecules.
So, saying wave functions within molecules might sound a little confusing, but remember we spent a lot of time talking about wave functions within atoms, and we know how to describe that, we know that a wave function just means an atomic orbital.
It's the same thing with molecules -- a molecular wave function just means a molecular orbital.
So, we'll start today talking about the two kinds of molecular orbitals, we can talk about bonding or anti-bonding orbitals.
Then we're going to actually use MO theory to describe bonding within these molecules, and we'll start with homonuclear diatomic molecules.
Diatomic mean it's di atomic, it's made up of two atoms, and homonuclear means that those two are the same atoms.
Then at the end, we'll look at an example with a heteronuclear diatomic molecules.
So, again, the same thing, but now two different atoms.
So, I will point out, in terms of MO theory, because it rigorously does take into account quantum mechanics, it starts to become complicated once we go beyond diatomic molecules.
So we're going to limit in our discussion in 511-1 for molecular orbital theory to diatomic molecules.
However, on Friday we will use a different approach so we can talk about bonding within atoms that have more than two atoms, molecules with more than two atoms.
All right, so one thing that I first want to point out about MO theory that is a big difference from Lewis structures, is that in MO theory valence electrons are de-localized over the entire molecule.
So, when we talked about Lewis structures, we actually assigned electrons to individual atoms or to individual bonds.
Whereas in molecular orbital theory, what I'm telling you is instead we understand that the electrons are spread all over the molecule, they're not just associated with a single atom or a single bond.
So specifically, what we do associate them instead is within molecular orbitals, and what we say is that they can be either in bonding or anti-bonding orbitals.
And again, I want to point out that a molecular orbital, we can also call that a wave function, they're the same thing.
And these orbitals arise from the combination of individual atomic orbital.
So, if we have two atomic orbitals coming together from two different atoms and they combine, what we end up forming is a molecular orbital.
The reason that we can talk about this is remember that we're talking about wave functions, we're talking about waves, so we can have constructive interference in which two different orbitals can constructively interfere, we can also have destructive interference.
So, we'll start by taking a look at constructive interference, and another way to explain this is just to say again, molecular orbitals are a linear combination of atomic orbitals.
So, let's start our discussion of a bonding orbital.
Our simplest case that we can look at would be if we had two 1 s orbitals coming together.
So let's say, for example, in a hydrogen atom.
So in hydrogen atom a, I'll depict that here where the nucleus is this dot, and then the circle is what I'm depicting as the wave function.
It makes sense to draw the wave function as a circle, because we do know that 1 s orbitals are spherically symmetric.
So, we can say that a circle is a good approximation for a 1 s wave function.
Similarly, with the second hydrogen atom, we've got the nucleus in the middle, and the 1 s b wave function around it.
So these are atomic orbitals.
What we're going to do in forming a molecule is just bring these two orbitals close together such that now we have their nucleus, the two nuclei, at a distance apart that's equal to the bond length.
And what we end up forming is a molecular orbital, because as we bring these two atomic orbitals close together, the part between them, that wave function, constructively interferes such that in our molecular orbital, we actually have a lot of wave function in between the two nuclei.
So we can go ahead and name our molecular orbital, just like we know how to name our atomic orbitals.
And I'm going to name this sigma 1 s. The 1 s just comes from the fact that the molecular orbital is a combination of two 1 s atomic orbitals.
And the sigma tells us something about the symmetry of this molecular orbital, specifically that it's cylindrically symmetric about the bond axis.
So that is the bond axis -- it's just the axis between the two nuclei.
Sometimes it's also called the internuclear axis.
So any time you have two atoms bonding, the bond axis is just the axis that they're bonding along.
Another thing I want to point out about every sigma orbital that you see, and it will make more sense when we contrast it with pi orbitals later.
But in sigma orbitals, you have no nodal planes along the bond axis, so if we had a nodal plane here, we'd see an area where the wave function was equal to zero.
We don't see that.
It will make more sense when we can show you one where it does have that area.
But keep in mind sigma orbitals have no nodal planes along the bond axis.
All right.
So, let's look at this in another way, sometimes it's hard to picture these waves combining.
So let's think of them a little bit more by graphing the amplitude of the wave, and seeing how we can have this constructive interference.
So again, if we think of a graph of the wave function, we had the wave function is at its highest amplitude when it's lined up with the nucleus, and then as we got further away from the nucleus, the amplitude of the wave function ends up tapering off until -- it never hits zero exactly, but it goes down very low.
So we can draw that for 1 s a, we can also draw it for 1 s b, and what I'm saying for the molecular wave function is that we have the interference between the two, and we have a constructive interference, so we end up adding these two wave functions together.
So, we're talking about wave functions and we know that means orbitals, but this is -- probably the better way to think about is the physical interpretation of the wave function.
So what is the wave function squared going to be equal to?
[INAUDIBLE]
Probability density, yes.
Probability density of finding an electron within that molecule in some given volume.
So let's think about that instead, let's think about probability density.
So if we're talking about probability density that's the wave function squared.
But now we're talking not about an atomic wave function, we're talking about a molecular wave function.
So to talk about it's squared, we're going to say it's sigma 1 s squared.
Oh, and actually before you skip your page in the notes, I realized I should write out for you what the addition is to start with.
So, when we're combining two waves, what we have is 1 s a that we're adding together with 1 s for b, the second atom.
And what we end up for our molecular wave function is sigma 1 s. So this is what we call our molecular orbital.
All right, so that will now allow you to turn the page, I think, and we can take a look at the probability.
So the probability again, that's just the orbital squared, the wave function squared.
So when we write that out, we just write sigma 1 s squared, or we can break it up into its individual parts, there's no reason we can't do that as well.
So just to say that it's 1 s squared plus 1 s b, all of that together squared.
So if we write out every term individually, what we end up with is essentially just the probability density for the first atom, then the probability density for the second atom, and then we have this last term here, and this is what ends up being the interference term.
So in this case where we're adding it together, this is going to be constructive interference.
So in this case the cross term represents constructive interference between the two 1 s atomic wave functions.
And this again is what we're going to call a bonding orbital.
So it can often make a lot more sense if we think about things in terms of energy.
We've been discussing energy diagrams a lot in this class, it's a very good way to visualize exactly what's going on.
So, let's think of the energy of interaction when we're comparing atomic orbitals to molecular bonding orbitals.
And what you find is when you have a bonding orbital, the energy decreases compared to the atomic orbitals.
So you can see here in this slide we have the atomic orbitals for the two hydrogen atoms, each of them have one electron in them, hydrogen has one electron in a 1 s orbital.
So, if we look at the molecular orbital, that's actually going to be lower in energy than either of the two atomic orbitals.
So it's going to be favorable for the electrons instead to go to that lower energy state and be within the molecular orbital.
So, let's draw in our electrons there, so we have our two electrons now in the molecular orbital.
So any time that you're drawing these molecular orbital diagrams, you want to keep in mind that the number of electrons that you have in atomic orbitals, you need to add those together and put that many electrons into your molecule.
Right, we had one from each atom, so that means we need a total of two in our molecular orbital.
And what you can see directly from looking at this energy level diagram, is that the molecule that we have is now more stable in the individual atoms.
That makes sense because we're lower in energy, the electrons are now lower in energy.
So that's the idea of a bonding molecular orbital.
Since we're talking about wave functions, since we're talking about the properties of waves, we don't only have constructive interference, we can also imagine a case where we would have destructive interference.
Just like we see destructive interference with water waves or with light waves, we can also see destructive interference with orbitals.
So, let's think about what that would look like.
So in this case we would have 1 s a and 1 s b, and instead we would subtract one from the other, and what we would see is that instead of having additional, more wave function in the middle here, we've actually cancelled out the wave function and we end up with a node.
So we can also name this orbital, and this orbital we're going to call sigma 1 s star.
So if we name this orbital, this is an anti-bonding molecular orbital.
So we had bonding and now we're talking about anti-bonding.
When we talk about anti-bonding, essentially we're taking 1 s a and now we're subtracting 1 s b, and what we end up with again is sigma 1 s, and the important thing to remember is to write this star here.
So any time you see a star that means an anti-bonding orbital.
Again we can look at this in terms of thinking about a picture this way, in terms of drawing the wave function out on an axis.
So we have 1 s a, and we're drawing this as having a positive amplitude, but since we have destructive interference we're going to draw 1 s b as having the opposite sign, so we have a plus and a minus in terms of signs.
So that should make it very easy to picture that this is being cancelled out in the middle.
If we overlay what the actual molecular orbital is on top of it, what you see is that in the center you end up cancelling out the wave function entirely.
So this is the 1 s star, sigma 1 s star orbital, and what you have in the center here is a node, right in the center between the two nuclei.
So again if we look at this in terms of its physical interpretation or probability density, what we need to do is square the wave function.
So if we square sigma 1 s star, we flip the amplitude so it's all positive now, but again we still have this node right in the middle.
So if we talk about the probability density and we write that in, it's going to be sigma 1 s star squared, so now we're talking about 1 s a minus 1 s b, all of that being squared.
And again, if we write out what all the terms are, we again have 1 s a squared plus 1 s b squared, but now what we're doing is we're actually subtracting the interference term.
So if we're subtracting the interference term, what we have here now is destructive interference.
So let's think about the energy of interaction here.
When we were talking about constructive interference, we had more electron density in between the 2 nuclei.
So that lowered the energy of the molecular orbital.
So, bonding orbitals are down here.
But when we think about where anti-bonding orbitals should be, it should be higher in energy.
It's increased compared to the atomic orbitals.
So we would label our anti-bonding orbital higher in energy than our 1 s atomic orbitals.
So, let's think a little bit about what this means.
First of all, again to repeat, any time we see the star, sigma 1 s star, that's an anti-bonding orbital.
When we talk about that, basically what we're saying, and you can see that because of that negative interference, we actually have less electron density between the nuclei than we did when they were two separate atoms.
So you can see that this is non-bonding, this is even worse than non-bonding, it's anti-bonding, because we're actually getting rid of electron density between the two nuclei.
And we know that it's electron density between the nuclei that holds two atoms together in a bond.
So what I want to point out is that it creates an effect that is exactly opposite of a bond.
You might have thought before we started talking about molecular orbital theory that non-bonding was the opposite of bonding, it's not, anti-bonding is the opposite of bonding, and anti-bonding is not non-bonding.
We can see that if we just look at this picture here.
Here is bonding, and here is non-bonding.
Anti-bonding is even higher in energy than non-bonding.
And the other thing to point out is that the energy that an anti-bonding orbital is raised by, is the same amount as a bonding orbital is lowered by.
So any time I draw these molecular orbitals, I do my best, and I'm not always perfect, yet trying to make this energy different exactly the same for the anti-bonding orbital being raised, versus the bonding orbital being lowered.
So those should be raised and lowered by the same energy.
So now let's take a look at some of -- is there a question up there?
Why would two atoms decide to [INAUDIBLE]
Well, they don't want to.
It's a higher energy situation.
So we'll start to look at molecules and we'll see if we take two atoms and we fill in our molecular orbital and it turns out that they have more anti-bonding orbitals than bonding, that's -- a diatomic molecule we'll never see.
So it helps us predict, will we see this, for example, h 2, which we're going to be about to do, we'll see is stabilized because it has more bonding than anti-bonding.
So we'll predict, yes, there's a bond here.
That's a really good question.
So, let's go ahead and do this and take a look at some of the actual atoms that we can think about and think about them in molecules.
So our simplest case that we started talking about was molecular hydrogen.
I want to finish this discussion by including the anti-bonding orbital, and this is a tip for you when you're drawing your molecular orbital diagrams, any time you draw a bonding orbital, there is also an anti-bonding orbital that exists.
It might not have any electrons in it, but it still exists, so you need to draw these into your molecular orbital diagram.
So I wanted to make sure you have a complete set for hydrogen in your notes.
So let's take a look at this.
Hydrogen, we can first draw in our atomic electrons.
So there's one electron in each hydrogen atom.
And then this means we'll have a total of two electrons in our hydrogen molecule, so we can fill both of those into the sigma 1 s orbital, the bonding orbital.
We don't have to put anything into the anti-bonding orbital, so that's great.
What we've seen is we have a net lowering of energy of the molecule versus the individual atoms.
So let's draw the electron configuration of hydrogen, the molecule, molecular hydrogen.
What you saw, what we've done a lot of is drawing the electron configurations for different atoms, we can do the same thing for a molecule.
So, if we take h 2, and we want to draw the electron configuration, it's very short.
All it is sigma 1 s, and then we have two electrons in it, so it's sigma 1 s squared.
So this is our electron configuration.
Let's take a look at another example.
Let's draw the molecular diagram for h e 2 now.
So again, we can fill in our atomic orbitals here, there's going to be two electrons in each of our atomic orbitals.
So now let's go ahead and fill in our molecular orbitals.
We need to fill in a total of four electrons.
So we have two electrons in our bonding orbital, but because we use the same rules to fill up molecular orbitals as we do atomic orbitals, so the Pauli exclusion principle tells us we can't have more than two electrons per orbital, so we have to go up to our anti-bonding orbital here.
So this means that we have two of the electrons are lowered in energy, but two are raised in energy.
So would this be a stabilized molecule then?
[INAUDIBLE]
No.
So, compared to the atoms, it should be somewhat the same energy, we shouldn't get any extra stabilization from forming the molecule.
So lets go ahead and write what the electron configuration would be of h e 2.
And again, we're just filling in the different orbitals, so we have sigma 1 s, that's going to be squared, and now we have sigma 1 s star squared.
So we can compare the two electron configurations, and we can actually think about -- what we figure out from them, we see that two are lowered in energy, two electrons are raised in energy, so we have no net gain or no net loss in energy for h e 2.
And there's actually a way that we can make predictions here, and what I'll tell you is molecular orbital theory predicts that h e 2 does not exist because it's not stabilized in terms of forming the molecule.
The way that we can figure this out is using something called bond order, and bond order is equal to 1/2 times the number of bonding electrons, minus the number of anti-bonding electrons.
And the bond order you get out will either be, for example, zero, which would mean that you have no bond, or you could have 1, a single bond, 1 .
5, a 1 and 1/2 bond, 2, a double bond, and so on.
So let's figure out the bond order for our two molecules here that we figured out the electron configuration for.
So I guess we'll start with helium 2.
So the bond order is going to be equal to 1/2, and then it will be 2 minus 2.
So our bond order for h e 2 is going to be equal to 0.
So it has a 0 bond, there's no bond in h e 2.
Let's look for hydrogen.
For hydrogen our bond order is going to equal 1/2, 2 minus 0.
So we would predict a bond order of 1.
What kind of a bond is a bond order of 1?
Yeah, we'd expect to see a single bond in hydrogen.
So what actually turns out the reality is that h e 2 does exist, but it exists as the weakest chemical bond known, and it wasn't, in fact, even found to exist until 1993, so I can assure you this is not a bond that you see very often in nature, and it is a very, very weak bond.
It only has a dissociation energy of 0 .
1 kilojoules per mole.
So that should make sense, because we saw no energy difference between the actual atoms and the molecules.
Molecular orbital theory, even at this very basic level, allowed us to predict that no, we're not going to see a true bond here, a strong bond.
In contrast, the dissociation energy of a bond for hydrogen, and molecular hydrogen is everywhere around us, we see 432 kilojoules per mole.
All right, so we can now see a little bit of what the power of molecular orbital theory is in predicting what kind of bonds we're going to see in molecules, or whether or not we'll see this bonding occur at all.
So let's look at another example, let's take lithium 2 and see what we can figure out here.
In lithium 2, we have two atoms of lithium, each have three electrons in them.
So now we have to include both the 1 s orbitals and also the 2 s orbitals.
So any time in a molecular orbital diagram you draw in orbitals, you need to draw the corresponding molecular orbitals.
So, this means we need to have sigma 1 s, sigma 1 s star, and now sigma 2 s and sigma 2 s star.
Something I'll also point out as you see these dashed line that tell you where the individual molecular orbitals are arising from, as you get to higher and higher atomic numbers of molecules that you're making, it makes a lot more sense to look at a diagram when you draw these dotted lines in, because they can start to get a little bit confusing.
So when you go ahead and draw these on your problem-sets or on your exams, it's a good idea to put these dashed lines in, both for you and for people reading it to see exactly where your molecular orbitals are coming from.
So, this means we have a total of six electrons that we need to put into molecular orbitals.
So again we just start filling those up -- we have two in the 1 s, two in the sigma 1 s star, and then we have two in the sigma 2 s. So we should be able to also calculate the bond order, just like we did for hydrogen and helium.
First we can do that by knowing the electron configuration, we can write it out just by going up the table here, up the energy levels.
And you can go ahead and tell me what you think the bond order is going to be for this molecule.
All right.
Let's take 10 more seconds on this.
OK, good, we're back on track a little bit with our clicker answers.
So it's selection three or one is the bond order.
So let's switch back to our class notes and look at what this means.
So we know that it's 1, because we have 1, 2, 3, 4 bonding, minus 2 anti-bonding, and 1/2 of that is a bond order of 1.
We would predict to see a single bond between lithium, and it turns out that's what we see.
And so you have for a reference, the dissociation energy of lithium 2 is 105 kilojoules per mole.
So let's keep moving along the periodic table and keep applying our molecular orbitals.
For example now, with b e 2, so beryllium 2 has four electrons in terms of each atom.
So you can start by filling those in, and now we can fill in our molecular orbitals as well.
This means we need a total of eight electrons in our molecular orbitals.
So we have two in 1 s, two in the sigma 1 s star, two in the sigma 2 s, and two in the sigma 2 s star.
So let's go ahead and figure out the bonding order for beryllium here.
So when we figure it out for beryllium -- let's see if I wrote in your notes what the actual -- electron configuration, OK that's already in your notes for you.
So let's go right to the bond order for beryllium.
So for the bond order we want to take 1/2 of the total number of bonding electrons, so that's going to be 4 minus anti-bonding is 4, so we end up getting a bond order that's equal to 0.
So what I want to point out with this case in beryllium is that you don't have to use all of the electrons to figure out the bond order, and in fact, once you get to molecules that are from atoms with atomic numbers of 8 or 10, you're not going to want to maybe draw out the full molecular orbital diagram.
So what I want to tell you is we also always get the same bond order if we instead only deal with the valence electrons.
So let's just prove that to ourselves and figure out the bond order just using valence electrons.
So this would mean the bond order is equal to 1/2, and in terms of valence electrons, how many bonding valence electrons do we have?
[INAUDIBLE]
All right, what about anti-bonding?
[INAUDIBLE]
Two.
OK, good.
So again, we're going to see that we have a bonding order of 0.
So we would not predict to see a b e 2 bond.
So what we see is a bond order of 0, and again, the bond is very, very weak.
Essentially we're not going to see this, it's 9 kilojoules per mole.
All right.
So, so far we've looked only at molecules that involve atoms that have only s orbitals in them.
I'm sure you're thinking well, what do we do in the case of p orbitals, and, in fact, we can do the same thing.
Again, we're going to take the linear combination of those p atomic orbitals and make what are called pi or some more sigma molecular orbitals.
So let's look at the first case where we have either the 2 p x or 2 p y type of orbitals that we're combining.
So they're the same shape, this is the shape of the orbital or the shape of the wave function, and we can call this either 2 p x a being combined with 2 p x b, or we could say since it's the same shape, it's 2 p y a being combined with 2 p y b.
And in either case if we first talk about constructive interference, what again we're going to see is that where these two orbitals come together, we're going to see increased wave function in that area, so we saw constructive interference.
So again, we can name these molecular orbitals and these we're going to call -- also to point out there is now a bond axis along this nodal plane, which is something we didn't see before when we were combining the s orbitals.
So when we go ahead and name these, we're going to call these pi orbitals.
We'll call it either pi 2 p x, if we're combining the x orbitals, or pi 2 p y.
The reason that I wanted to point out this nodal plane here is because this is why it is called a pi orbital.
Pi orbitals are a molecular orbital that have a nodal plane through the bond axis.
Remember this is our bond axis here, and you can see there is this area where the wave function is equal to zero all along that plane, that's a nodal plane.
So that's why these are pi orbitals instead of sigma orbitals.
So again, we can think about the probability density in terms of squaring the wave function.
So now what it is that we're squaring is if we're talking about x orbital, it's pi 2 p x squared, and this is just equal to the 2 p x a plus the 2 p x b all squared, or if we write out all of the terms we have 2 p x a squared plus 2 p x b squared, and then this term here, and again, this is our interference term.
In this case is it constructive or destructive interference?
[INAUDIBLE]
Constructive interference.
We're seeing that the wave function's adding together and giving us more wave function in the center here.
All right.
So we see constructive interference, of course, we can also see destructive interference.
So I changed the colors here to show that these are 2 p orbitals with an opposite phase or an opposite sign.
So what happens when we add a 2 p a and we subtract from it a 2 p x b, or the same with a 2 p y a subtracting a 2 p y b, is that we're actually going to cancel out the wave function in the center, so we now have 2 nodal planes.
So again, this is an anti-bonding orbital, and what you see is that there is now less electron density between the two nuclei than there was when you had non-bonding.
So we're going to call this the sigma 2 p x star, or if we're talking about the 2 p y orbitals we'll call this the pi 2 p x star, and the pi 2 p y star.
And the pi star orbitals result from any time you have destructive interference from 2 p orbitals that are either the p x or the p y.
So now we can move on to an example where we do, in fact, have to use some p orbitals, so this would be b 2.
How many electrons are in boron?
Five.
I see some hands going up.
There's five electrons So what you'll notice here is that I only filled in 3 electrons.
What do these electrons represent?
Valence
Valence electrons.
OK, sometimes you're going to be asked to draw a molecular orbital diagram where you're asked to include all electrons, and sometimes it will specifically say only include valence electrons.
That happens because of space issues that you were asked to do that, because you can always assume that all of the core orbitals are already going to be filled.
So, in this case, we're just drawing the molecular orbital diagram for the valence electrons, so we have three for each.
And what we see here is now when we're combining the p, we have our 2 p x and our 2 p y orbitals that are lower in energy, and then our pi anti-bonding orbitals that are higher in energy.
You might be asking where the 2 p z orbital is and we'll get to that soon once we need it.
Let's just first fill in this for the b 2 case.
So we can start at the bottom, two electrons in sigma 2 s, two electrons in sigma 2 s star.
Now we need to jump up to using these pi orbitals, and what we're going to do is put one electron into each of our pi 2 p x and 2 p y orbitals.
So again you can see as we're filling up our molecular orbitals, we're using the exact same principle we used to fill up atomic orbitals.
So let's think about what the valence electron configuration is here.
So now we're looking at the case of b 2.
And what we're looking at is a valence electron molecular orbital diagram, so let's just draw the electron configuration for the valence orbitals, so that will be sigma 2 s 2, sigma 2 s star 2, and now we start in with our pi 2 p x 1, and our pi 2 p y 1.
So this is our valence electron configuration for b 2.
All right.
So what would you expect the bonding order for b 2 to be?
Shout it out if you know
one
And I didn't write up there but it is one, and we can see that it's 1, because it's 1/2 of 2, 4 minus 2, so 1/2 of 2, the bonding order is going to be equal to one.
So let's move on to another example, let's talk about carbon here.
Again, we're just talking about the valence electrons.
So carbon has four valence electrons, so if we talk about c 2, again we're going to start filling in our molecular orbitals, and now we're going to have eight electrons to fill into our molecular orbitals.
So, we'll put two in the sigma 2 s, two in the sigma 2 s star, and now we're going to fill one and one into each of our pi 2 p x and 2 p y, but we still have two electrons left, so what we're going to do is double up in terms of our 2 p x and our 2 p y.
So let's think about what this valence electron configuration is for c 2.
And again, I want you to have practiced drawing these out in the form -- you always need to start with the sigma and then write the number of the orbital.
So, sigma 2 s has two electrons, sigma 2 s star with two electrons, and now we have sigma 2 p x -- how many electrons here?
Two
Two.
And sigma 2 p y, two electrons here.
Oh excuse me, pi 2 p y, thank you -- pi 2 p x and pi 2 p y.
So let's talk about what the bonding order is going to be for c 2.
So what's the bonding order for c 2?
Two
two?
OK, so we have 2, 4, 6 minus 2, so we have 1/2 of 6 minus 2, so that's 1/2 half 4, so we have a bonding order of two for carbon 2.
So we would expect to see a double bond for a c 2 where we would expect to see a -- double bond for c 2 and a single bond for b 2.
And that is, in fact, what we can surmise if we look at the different dissociation energies for the two bonds.
So for b 2, which is a single bond, that's 289 kilojoules per mole to break it, and it takes us more energy to break this double bond for carbon, which is 599 kilojoules per mole.
So, in general what we see, and this is always true if we're comparing the same atom, and in general, if we're comparing different types of molecules, but we know that a single bond is always weaker than a double bond, which is weaker than a triple bond.
And obviously, no bond is the weakest of all is not bonding.
All right.
So let's look now at the case where we do have 2 p z orbitals that we're talking about.
So again, what we're talking about is the linear combination of atomic 2 p orbitals, and now we're talking about 2 p z.
So if we have constructive interference between the two, what we're going to see is our molecular orbital looks something like this.
Do you predict that this will be a sigma or a pi orbital?
[INAUDIBLE]
All right, I'm hearing a little of both, but I'm very encouraged to hear quite a few people saying sigma.
This is, in fact, a sigma 2 p z orbital is what this orbital is called.
The reason that it's sigma is if you look at the bonding axis here, is that there is no nodal plane along the bonding axis.
Also, it is cylindrically symmetric around the bonding axis, so this is how we know that it's a sigma orbital.
So some p orbitals form pi molecular orbitals, and some form sigma p orbitals.
Specifically, it's always the z that forms the sigma orbital, and the reason is at least at a minimum for this class we always define the internuclear axis as the z axis, so this is always the z axis, so it's always going to be the 2 p z's that are coming together head-on.
The reason that there is increased electron density here is you can see that these two orbitals come together and constructively interfere.
We can also talk about anti-bonding orbitals where we have destructive interference.
So instead, these would be canceling out wave functions between the two, so we would end up with a nodal plane down the center.
Would this be a sigma or a pi?
It's still sigma.
So even though we see a nodal plane down the center, I just want to really point out that it's only when we have a nodal plane in the internuclear or the bond axis that we're calling that a pi orbital.
So this is still a sigma -- it's a sigma 2 p z star orbital.
We have destructive interference here.
So now let's look at an example where we talk about using these 2 p z orbitals, so let's look at oxygen.
So the first thing I want to point out is that the 2 p z, the sigma 2 p z is even lower in energy than the pi orbitals here, and the anti-bonding sigma orbital is going to be higher in energy than the pi 2 p orbitals that we have here.
So in oxygen again, this is just showing the valence electrons, so we end up having six valence electrons from each oxygen atom.
We can fill right up our table just like we did before, but now we have included our 2 p z orbital here.
So we have two electrons in sigma 2 s, two in sigma 2 s star, in sigma 2 p z we have two -- those are filled first, and now we're going to put one into pi 2 p x, and one into pi 2 p y.
So we can write out what the electron configuration is here, and I think that I have already written that out for you in your notes.
What is the bond order of o 2?
It's two.
Oh excuse me, I didn't fill in all of my electrons.
All right.
So we had a total of 2, 4, 6, 8.
So we had a total of 12.
So we filled in four here -- we need to keep going.
All right, so I did this not at all purposely, but this can point out for you that you need to make sure that the number of electrons that you have in your molecular orbital does match up with the total number that you have in your atomic orbitals.
So I did not do any counting up here, you should make sure you do counting, I apologize.
So we need to fill all the way up to the pi 2 p x, and the pi 2 p y.
All right, so the bonding order, you're correct, should be 2, if we subtract the number of bonding minus anti-bonding electrons and take that in 1/2.
And what I want to point out that we just figured out for molecular orbital theory, is that o 2 is a biradical, because remember, the definition of a radical is when we have an unpaired electron.
You can see that we have two unpaired electrons in this molecule here -- one in the pi 2 p x star, and one in the pi 2 p y star orbital.
This was something we could not predict using Lewis structures, but we can predict using MO theory that we have a radical species here.
So that's a really important type of an application that we can use MO theory for that we weren't able to do with our Lewis structures.
All right, I want to do one more at homonuclear example here, and this is n 2.
The first thing that I need to point out is you can actually see an n 2 versus o 2 that we flip-flopped the energy of the sigma and the pi 2 p orbitals.
So this is a glitch, just like sometimes we had glitches in filling up our atomic orbitals.
This is something that you need to remember.
And what you need to remember is if the z is equal to eight or greater, such as oxygen being the cut-off point, this sigma 2 p orbital is actually lower in energy than the pi 2 p orbitals, the molecular orbitals.
But for anything 7 or less, so what is the atomic number for nitrogen?
Five
five -- there's five valence electrons, but the atomic number is actually seven.
So z equals 7 -- this is the cut-off where, in fact, the sigma orbital is going to be higher in energy than the pi 2 p orbitals.
This is something that you just need to remember.
I wrote it down your notes, if you can put a big star next to it so you don't forget this.
This is something you're going to be responsible for in drawing out your molecular orbitals.
So let's fill it out in this way, keeping in mind that we're going to fill out the pi 2 p's before the sigma.
So we have a total of 2, 4, 6, 8, 10 valence electrons, so I'll make sure I count to 10 as we fill up our molecular orbitals here.
We have two, then we have four.
Now we're going to start in with that pi 2 p orbitals, which gives us 1 each, and then two each in those, and then after that, we'll go up to our sigma 2 p z orbital.
So this is going to be our molecular orbital diagram.
And again, I've written for you, but you can figure out what the electron configuration is just by writing up in this order here.
And what is the bond order going to be n 2?
[INAUDIBLE]
It's going to be three.
So, you can go ahead and calculate that, if you can't see that right away.
So we'll end here, we'll finish up with the heteronuclear example on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, I want to take 10 more seconds now the clicker slide.
This is giving us one more try on the vsper geometries, because it didn't go so well on Wednesday.
All right, excellent.
So that is a very good job.
Let's quickly go over why.
We have p h 3.
We told you that phosphorous has 5 valence electrons plus 3 from each of the hydrogens, so we have a total of 8 valence electrons.
How many do we need to get full valence shells everywhere?
[INAUDIBLE]
14.
So, we need 14 minus 8.
That leaves us with 6 bonding electrons.
And if we put that in our bond here, we have 1, 2, 3 bonds, plus we have one lone pair left over.
So this is our Lewis structure here.
If these bonds were all completely of equal distance apart, whether is was a lone pair or bonding electrons, the angles would be 109 .
5 degrees.
But because there's this lone pair here, it's pushing down on the other bonds, so we end up with an angle of less than 109 .
5 degrees.
All right, so let's switch over to notes for today.
So we're going to finish talking about molecular orbital theory, and then we'll switch over to discussing bonding in larger molecules, even larger than diatomic, so we'll move on to talking about valence bond theory and hybridization.
So, clearly you don't have your notes in front of you yet, so you can just listen, take it all in.
What I'll do is I'll post the notes filled in to the point where you actually get your class notes today.
So, this will be a little bit more like a seminar to start with, and a little bit less like a lecture in class.
But let's go ahead and start our discussion in terms of molecular orbital theory.
So where we had left off with was we'd fully discussed up to the point of considering homonuclear diatomic molecules, so molecules that both have the same nucleus.
And where we had left off was we were going to start one example of thinking about now where we have a heteronuclear diatomic molecules, so two different atoms in terms of forming the molecule.
But first, I just want to remind you when we're talking about molecular orbital theory, this is treating electrons as waves, so what we're actually able to do is either constructively or destructively combine atomic orbitals to form molecular orbitals.
So you should remember that any time we combine 2 s orbitals, what we're going to find is if we constructively interfere those two orbitals, we're going to form a bonding orbital.
And that's going to be lower in energy than the two individual atomic orbitals.
And we call that, for this case, our sigma 2 s orbital.
In contrast, if we have destructive interference, what we're going to form is a sigma 2 s star, and what does the star designate?
[INAUDIBLE]
Anti-bonding, yup.
So it's an Anti-bonding orbital.
It's going to be higher in energy than the individual atomic orbitals.
All right, great.
So, I think we have these molecular orbital energies down, so let's move on to talking about more complex molecules.
And to do this we're going to introduce valence bond theory, and the idea of hybridization of orbitals.
So the idea behind valence bond theory is very easy to understand.
Essentially what you have is bonds resulting from the pairing of unpaired electrons.
So the simplest case we can think of is with h 2 where we have two unpaired electrons, each in a 1 s orbital of a separate h atom.
And if we picture those two coming together, we form the h 2 molecule.
And again, we have the pairing of the unpaired electrons, and we have two orbitals coming together.
So in molecular orbital theory, what we did was we named orbitals based on their symmetry.
In valence bond theory, the focus is on discussing the bonds, but it should look very familiar to you, because there's two types of bonds that we want to discuss here.
We want to discuss sigma bonds and pi bonds.
So this is very similar to what we saw in terms of sigma orbitals and pi orbitals.
So in this first case here, what we're seeing is a sigma bond.
And a sigma bond forms any time you have two orbitals coming together and interacting on that internuclear axis.
So we talk about a sigma bond as being cylindrically symmetric about the bond axis, and it's important to point out that it has no nodal plane across this bond axis.
This is in direct contrast to when we're thinking about pi bonds.
So pi bonds have electron density both above and below the bond axis, but they actually have a nodal plane at this z, this bond axis here.
And remember for this class, we always define z as the internuclear or the bond axis.
So it might look like here, if you don't understand about p orbitals, which I know all you do, but if someone else was just looking and seeing, it kind of looks like there's two bonds here.
There's not two bonds, that's one pi bond, and the reason is because it's 2 p orbitals coming together, and remember p orbitals have electron density above and below the axis, so when they come together, it kind of looks like one bonds, but essentially what we have here is one pi bond.
So let's think about how we can classify single and double and triple bonds, which is what we're really used to dealing with in terms of these sigma bonds and these pi bonds.
So, if we take a look at what a single bond is, and let me grab some molecules here.
If we're talking about a single bond, we're talking about 2 orbitals overlapping in the internuclear axis.
So if we have a single bond here, would you consider that a sigma bond or a pi bond?
[INAUDIBLE]
Right, it's a sigma bond.
Essentially what we're seeing is overlapping in this z axis here.
In contrast, if we talk about a double bond, what we're now talking about is having both a sigma bond and also one pi bond.
And I apologize, I intended to set this up right before class, but that didn't happen today.
All right, so what we see here is we have our sigma bond that's along the internuclear axis here, but we also have a pi bond, because each of these atoms now has electrons in it's in a p orbital, so we're going to overlap of electron density above and below the bond.
So that's exactly what our definition of a pi bond is, so we have one sigma bond, and one pi bond.
So now let's think about a triple bond.
A triple bond, again is going to have one sigma bond on the internuclear axis.
How many pi bonds would you expect?
[INAUDIBLE]
Two, great.
So, we're going to see two pi bonds.
The first one will be above and below the bond axis is where we'll see the electron density, and the second will be perpendicular to that, so it will be a density in front of and behind the bond axis.
So we can kind of flip it this way -- this will be one pi bond, this will be another interacting between these p orbitals.
All right, so that's really all there is to thinking about valence bond theory in terms of the most simple explanation here.
But what we're going is we're going to start trying to apply it to a molecule, and I actually picked a molecule that it's not going to work for, even though it would work even just at this level for many, many molecules.
And I picked looking at methane so we could see if there are other factors that we're not considering, that we need to maybe tweak our model a little bit, and I think we'll find that we do if we take a look at a polyatomic molecule, methane, so c h 4.
So let's think about methane using valence bond theory.
So, using our simple valence bond theory, what we would expect is that we want to pair up any unpaired electrons in methane with unpaired electrons from hydrogen and form bonds.
But what we see we have is that we only have two unpaired electrons here.
Because we have paired set in a 2 s orbital, so all we're left essentially is two electrons that are available for bonding.
So this should immediately look like a problem because we know, in fact, that methane is tetravalent, and this is telling us it's only divalent.
Essentially it would only allow for us to bond to two hydrogen atoms.
So if it did this, it now looks like, from looking at the paired electrons that we have a stable structure here, and our structure is not c h 4, it's a stable structure of c h 2, and it will actually predict, also, what this h c h bond angle it is.
So according to this model what is that bond angle?
[INAUDIBLE]
One more time.
OK, I hear a mix.
So, according to this model, what we're seeing is a bond angle of 90 degrees.
What do you know the bond angle should be?
It's 109 .
5 is what we would expect for methane because it's tetravalent, but here we're just seeing something that's divalent, and they're both in p orbitals that are perpendicular to each other.
So what we're predicting is a bond angle of 90 degrees.
This is totally wrong, this is the wrong picture altogether.
If you had your notes, you could do some fun scribbling right now, so you can do that at home.
We're going to need to tweak our explanation here, and take into account another factor, and that factor is the fact that we know that we must have four unpaired electrons in carbon if we're going to form four bonds.
So the way that we can explain this is through something called electron promotion and hybridization of atomic orbitals.
So let's take a look at what we mean by this.
So if we take our carbon atom here, which has two electrons in the 2 s orbital, and we promote one of these electrons into a 2 p orbital, what we see now is that yes, we do, we have four unpaired electrons.
So, looking at this, this might not look so good for you.
What we're proposing here is that you take a nice low energy s electron and move it into a higher energy p orbital.
And the truth is that yes, this costs energy, we're going up to a higher energy state.
But it doesn't actually cost as much energy as you might think, because in this s orbital here we have a paired electron situation where we're moving up to a p orbital where the electron is no longer paired, so it won't feel quite as much electron repulsion, but nonetheless, this is going to cost us energy.
So we'll have to think about where that energy is going to come from and we'll see that in just a minute.
But let's assume that this is, in fact, going to happen.
So now what we have is four unpaired electrons.
That's great, but it's still not quite the picture we need, because actually, all the electrons are not in equal orbitals -- one's in an s orbital, and 3 are in p. But what we need to remember is the fact that we're talking about electrons which are waves.
When we're talking about orbitals, we're talking about wave functions.
So we can actually constructively and destructively combine these waves, these atomic orbitals to make a hybrid.
So if we go ahead and hybridize our p orbitals and our s orbitals, we'll switch from having these original orbitals to having something called hybrid orbitals.
And hybrid orbitals are all going to be completely equal, and you'll notice that they're higher in energy than the s orbital, and lower in energy than the p orbital.
That should make sense because they come from combining s orbitals and p orbitals.
And specifically, when we give them a name it's very clear exactly which orbitals they come from combining, we're calling these s p 3 orbitals -- that's because they come from combining 1 s orbital and 3 p orbitals.
You should never get the names of hybrid orbitals wrong because it's very straightforward.
If it has 1 s and 3 p's, we call it s p 3.
Naming doesn't always make sense in chemistry, so I like to point out this is a place where naming does make a lot of sense.
All right, so let's consider our methane situation now that we have our hybrid orbitals.
So I want to mention also, these are exactly equivalent, they're equivalent in energy, they're equivalent in shape.
The only thing that is different about these orbitals is their orientation in space.
So actually, first let's take a look at how we got these orbitals.
We got them from combining again, 1 s orbital and the 3 p orbitals.
If we hybridize these, what we end up seeing are these four hybrid orbitals.
You'll notice the shape is the same, all that's different is their orientation.
So essentially, each of these orbitals come from linear combinations of all of the original orbitals, and it's hard to picture exactly how that happens, but one that you can at least start to get an idea is if you think about combining the 2 s and the 2 p z here, which is not quite accurate because of course, we're combining all of them.
But it would just give you a little bit of an idea of shape.
You can see if we combine the s with the top lobe of the p, they're going to constructively interfere because they have the same sign.
So you see in the hybrid orbital we actually have a larger lobe on top where they constructively interfered.
If you compare the s orbital with the bottom lobe, these have a different sign so they're going to destructively interfere.
So what you see is actually a diminished lobe on the back part of this s p 3 orbital.
So s p 3 orbitals always have one huge lobe and one really little lobe.
A lot of times when people draw them, they even only draw the big lobe just to keep their paper looking nicer, but there is that little tiny lobe on the other side.
All right, so in terms of s p 3 hybrid orbitals, let's combine all four together on one axis, because this is what's going to happen in an s p 3 carbon atom.
So in this case, what would you say that the angle is here?
[INAUDIBLE]
Right, great.
So, we've achieved the angle that we observed, which is good, which is a 109 .
5.
So we can think about now doing bonding, and now we have four equal orbitals with one electronic each.
So we can bring in four hydrogen atoms, which will each contribute another unpaired electron.
So now what we have is four bonds.
And we can think about where we did get that energy for electron promotion that I mentioned before where we moved the electron from the 2 s to the 2 p. We get that from bonding.
We're going to release a lot of energy for bonding, it's going to more than make up for the fact that we actually had to spend some energy to promote that electron.
So, we can think about now how do we describe this bond in valence bond theory.
So the way that you describe a bond is you describe the orbitals that the bond comes from, and also the symmetry of the bond.
So would you expect this to be a pi bond or a sigma bond here?
[INAUDIBLE]
OK, so I'm hearing some mixed answers.
It turns out that it's a sigma bond.
The reason that it's a sigma bond is because the s p 3 hybrid orbital is directly interacting with the 1 s orbital of the hydrogen atom, and that's going to happen on the internuclear axis, they're just coming together.
Any time two orbitals come straight on together in that internuclear axis, you're going to have a sigma bond.
So if we go ahead and name this bond, what we're going to name it is sigma, because that's the -- basically the shape of the bond or that's how our bond is coming together.
And then we're going to name the atomic orbitals that make it up, and it's being made up of a carbon 2 s p 3 orbital, and a hydrogen 1 s orbital.
All right, so let's think of a case now that's getting a little bit more complicated.
We were talking about methane, which has only one central atom.
We can also talk about atoms that have two or more central atoms.
So let's talk about ethane now, which is c h 2.
So let's take our carbon s p 3 hybridized carbon and just move it around here so we can make the z inter- bonding axis between the two carbons right here.
So if we still have an angle of a 109 .
5 degrees, and again, we still have four unpaired electrons available for bonding, we can make one of those bonds with another s p 3 hybridized carbon, so we're going to make up one pair here.
If we think about that, that's a sigma bond, right, they're coming together along the nuclear axis.
We also have six spots available to form hydrogen bonds, so we can go ahead and fill in those electrons as well.
So in terms of thinking about ethane, we actually have two bond types that we're going to be describing just in terms of the carbon-carbon bond and then the carbon h bonds.
So let's talk about ethane and how we would actually write these bonds.
If we have the molecule ethane, then what we're going to have first is our sigma bond that we described between the two carbons.
So it's going to be carbon, and then what's the hybridization here?
[INAUDIBLE]
All right, start again, what's the hybridization of the carbon atom?
[INAUDIBLE]
OK, so it's 2 s p 3, and our second carbon is also 2 s p 3.
All right, so this is our first type of bond here.
Our second bond is going to be between the carbon and the hydrogen atoms.
Is that a sigma or a pi bond?
[INAUDIBLE]
Sigma, good.
So again, our carbon is going to be 2 s p 3.
And what will our hydrogen be?
1 s -- we don't have to hybridize it, it already has only one unpaired electron in a 1 s orbital.
All right, so that's how we describe ethane.
We don't have to just stick with carbon, we can think about describing other types of atoms as well using this hybridization.
For example, we can talk about nitrogen, and nitrogen has five valence electrons shown here.
Would you expect to see electron promotion in nitrogen where we pull one of these 2 s electrons into one of the 2 p orbitals?
[INAUDIBLE]
No, good.
So, electron promotion does not happen in terms of nitrogen, because it would not increased our number of unpaired electrons.
No matter what we do in terms of promotion, we're always going to have three unpaired electrons.
We can still hybridize all these orbitals, however, so we can still form four hybrid orbitals, which are again, 2 s p 3 hybrid orbitals.
So if we take a look at nitrogen here, what you'll notice is we have thre available for bonding, and we already have our lone pair -- one of our orbitals is already filled up.
So we can add three hydrogen atoms here, and fill in our other orbitals right here.
So if we do this and we form the molecule ammonia, let's switch to a clicker question, and have you tell me what the bond angle is going to be in ammonia -- the h n h bond angle.
Actually, let me draw it on the board as you look -- actually, can you put the class notes on, since you don't actually have your notes to refer to.
So there's the class notes there.
All right, this should be a pretty quick thing for you to figure out, so let's just take 10 seconds on this.
OK, great.
Even thinking quickly, most of you got it correct.
So what we see is on ammonia here, we know that it's less than a 109 .
5, it's actually 107, so it's less than a 109 .
5, because of that lone pair pushing down in the bonding electrons.
And what is the shape, for one more clicker question on ammonia?
Let's take 10 seconds again, this should be pretty quick.
All right, pretty good.
So, 70% of you.
We'd like to get this up higher.
The shape is actually trigonal pyramidal.
And you need to just remember your shapes.
If they're not obvious to you what they're called, you need to just study them and learn them.
So it's trigonal because we have these three atoms that are bound to the central atom here, and if you picture it, it's actually shaped like a pyramid.
So it's trigonal pyramidal.
That's what we call when we have three bonding atoms and one lone pair.
All right.
So we can switch all the way back to our notes here.
And the last thing we can think about is how do we name this n h bond, and again, we just name it based on it symmetry.
It's a sigma bond, and it's going to be -- no.
OK, it's going to be nitrogen 2 s p 3, because it's a nitrogen atom, and then hydrogen 1 s. So, I don't even have to worry because you're not writing this down, so I can just fix it when I post the notes and no one will ever know, except that this is not OpenCourseWare.
So let's switch to thinking about oxygen hybridization here.
So in oxygen we have a similar situation where, in fact, we are not going to promote any of the electrons because we have two lone pair electrons no matter what we do.
So when we hybridize our orbitals, we're going to end up with again, four hybrid orbitals, 4 s p 3 orbitals, and what we'll see is that two of these are already going to be filled up with a paired electrons, so we're only going to have 2 orbitals with an unpaired electron available for bonding.
So let's think about water here as our simplest example with oxygen.
So we can have our two hydrogen atoms come in here, and what we will find is now that we have all of our orbitals filled up -- so thinking about what this angle is here, would you expect it to be less than or greater than what we saw for ammonia before?
Less than
Good, good, it's going to be less than, and it's going to be less than because now we have two lone pairs.
So since we have two lone pairs, we're going to be pushing down even further on the bonding electrons, so we're going to smoosh those bonds even closer together.
The bond, it turns out, is 104 .
5 degrees, that h o h bond.
So in terms of naming our o h bond, good, it's right here.
So it's going to be a sigma bond, and we have oxygen 2 s p 3 and hydrogen 1 s. And the geometry, which I didn't ask you, is going to be bent for this molecule.
All right, so that's s p 3 hybridization, but those aren't the only type of hybrid orbitals that we can form.
Let's take a look at what happens if instead of combining all four orbitals, we just combine three of those orbitals, and what we'll end up with is s p 2 hybridization.
So in s p 2 hybridization, instead of combining all four, we're just combining two of the p orbitals with the s orbital.
So what we're going to end up with now is three hybrid orbitals.
And what happens to this last p orbital is nothing at all, we just get it back.
So we end up with 1 p orbital completely untouched, and three hybrid s p 2 orbitals.
So again, we can think of an example here.
So let's take boron, for example, and this has -- it starts off with three valence electrons.
Would you expect to see electron promotion for boron?
Yes
Yeah, absolutely.
if we move up one of our electrons into an empty p orbital, what were going to see is now we have three unpaired electrons that are ready for bonding.
So, if we hybridize just these three orbitals, what we're going to end up with is our s p 2 hybrid orbitals.
Again, the name is very straightforward, it comes from 1 s and 2 p orbital, so it will be s p 2.
And again, you might be thinking well, why didn't we actually hybridize this 2 p y orbital.
It doesn't actually have an electron in it, so we don't have to worry about whether it's very high in energy or not, we don't care that it's high in energy.
What we do care about is the energy of our orbitals that have electrons in them, and if we combined all four of the orbitals, then our hybrid orbitals would have more p character to them, so they'd actually be higher in energy.
So if we don't have to hybridize one of the p orbitals, we can actually end up with a lower energy situation, because now these s p 2 orbitals are 1/3 s character, and only 2/3 p character, instead of 3/4.
So we end up with 3 s p 2 hybrid orbitals, so we can think about what would happen here in terms of bonding, and if we think about how to get our bonds as far away as possible from each other, what we're going to have is the trigonal planer situation.
So if you picture, for example, b h 3, it's going to look like this.
All of our electrons are in our bonds, we want to got them a 120 degrees away from each other, that's as far away as we can get them.
Keep in mind we do have this p orbital here and it's coming right out at us.
And this p orbital is here, but it's empty, it doesn't have any electrons in it, that's why we don't have to worry about it in terms of getting our electrons as far away from each other as possible.
So what we'll have here is a trigonal planar case, and you can see that we only have three electrons that are set for bonding, so we'll add three hydrogens, and for b h 3, we'll get a stable structure here.
So, remember, boron was one of those exceptions to our Lewis structure rules where it was perfectly happy not having a full octet.
So this can tell you why it's so happy with only having six electrons around it.
All right, so if we think about b h bond here, again, it's the sigma bond, and we're going to say it's a boron 2 s p 2 hybrid orbital interacting with a hydrogen 1 s orbital.
So let's take a look at another case where we have s p 2 hybridization, we can actually also have it happen in carbon.
So if we think about having it happen in carbon, we're starting with the situation where we've already promoted our electron into a 2 p orbital here, and what we're going to do is just combine the s and two of the p's, so we'll end up with electrons in one of each three s p 2 hybrid orbitals.
But unlike the case with boron where we had an empty p orbital, we're actually going to have an electron in the p orbital of carbon as well.
So again, if we think about that shape of that carbon atom, it's going to be trigonal planar, it's going to have bond angles of 120 degrees, because we have this set up of having three hybrid orbitals.
So let's take a look at what actually happens if we're talking about a carbon-carbon double bond, such as in ethene, c 2 h 4, we're going to have a double bond.
If we have a double bond, we know we need to have only one sigma bond, and we're also going to have one pi bond.
So it already should make sense why we have that p orbital there, in order to form a pi bond, we're going to need a p orbital.
So if you picture this as our s p 2 carbon atom where we have three hybrid orbitals, and then one p y orbital coming right out at us.
So again, we picture the same thing as we pictured with the boron there.
If we have, coming along this z axis, another carbon atom, we can actually form one bond between the two carbon atoms there.
So if we picture how this happens, what we have here if these are our 2 s p 2 carbon atoms -- so here we have s p 2 hybrid carbon, and here we have s p 2 hybrid carbon atom.
These 2 are going to come together like this, and the first bond that we're going to form is going to be a sigma bond, right, so we see that here.
If we're looking head on, we see they form a sigma bond.
We can also look at them coming in from the side, and that's what I tried to depict here where you can actually see in pink is the p orbital.
So we can also show them coming together this way, so now you're looking at it where you can see the p orbital, and maybe just see well one of the hydrogen atoms.
So we can have four total hydrogens bonding here, and we can think about how to describe these carbon-carbon bonds.
So in the first case of this first bond here that I've put in a square, what type of a bond is this, is the sigma or pi?
Sigma
Yup, it's a sigma bond.
We're having two orbitals coming together on the bond axis.
So we'll call this sigma, and it's between two s p 2 hybrid carbon atoms.
So it's stigma carbon s p 2, carbon s p 2.
What about this second bond here where we're going to have interaction of 2 p orbitals, is that sigma or pi?
Pi
Pi, great.
So our second bond is going to be a pi bond.
And again, this is between the p orbitals, these are not hybrid orbitals, so when we name this bond we're going to name it as a pi bond here, because it's between two p orbitals, and it's going to be between the carbon 2 p y orbital, and the other carbon 2 p y orbital.
Remember, we didn't hybridize the 2 p y orbital, so that's what we have left over to form these pi bonds.
All right.
So in addition to having these two carbon bonds, we actually also have four carbon hydrogen bonds in addition to our carbon-carbon bonds.
So why don't you tell me what the valence bond description would be of these carbon hydrogen bonds?
So let's take 10 seconds on that.
OK, great.
So most and you got it, so we can switch to the notes and let's talk about this here.
So in terms of the carbon hydrogen bond, it's a sigma bond, because we define it -- any time we are bonding to an atom, we have to keep redefining our bond axis to whatever two atoms we're talking about.
So it's along the bond axis and it's between a carbon s p 2 hybrid, and then the hydrogen is just a 1 s orbital that we're combining here.
So those are our three types of bonds in ethene.
One thing that I want to mention that is really important is once you have double bonds, what happens between those two atoms in the molecule is they can no longer rotate in relation to each other.
So you can think about why that is.
When we have just a single bond in them molecule, you have all the free rotation you want, you can just spin it around, there's nothing keeping it in place.
But once you have a double bond here, we have our pi bond, as well as our sigma bond.
So there's electron density above the bond and below the bond.
So if I try to rotate my 2 atoms, you see that I have to break that pi bond, because they need to be lined up so that the electron density can overlap.
So in order to rotate a double bond, you have to actually break the pi bond, so essentially what you're doing is breaking the double bond.
So really, you can not ever rotate a double bond, it makes your molecule very rigid.
This is incredibly important because if you picture having a double bond in a very large molecule, you could have all sorts of other atoms off this way and all sorts of other atoms off this way, and you can picture the shape would be very different if you have one confirmation versus another confirmation.
So it's very important that the double bond locks it in a particular conformation.
This completely could change if you were to flip from one to the other conformation which can happen in chemical reactions.
If you were to make that change you would find that the molecule now has completely different biological and chemical properties.
So it's very important to be keeping in mind that any time you see a double bond, you have a pi bond there, so you're not going to see any rotation around the bond axis.
All right, so let's think of a more complicated example of having a double bond, and maybe a more interesting example, and this is talking about benzene.
I think most and you have talked a little bit about benzene over this past week in recitation.
Benzene is a ring that's made up of six carbon atoms and six hydrogen atoms.
So let's picture what this looks like here, and we'll start with four and we'll add in our last two.
So essentially, we have two ethene or ethylene molecules here to start with where these blue are our 2 s p 2 hybrid orbitals, so you can see that for each carbon atom, one is already used up binding to another carbon atom.
If we think about bringing in those last two carbons, what you can see is that for every carbon, two of its hybrid orbitals are being used to bond to other carbons.
So that leaves each carbon with only one hybrid orbital left.
And if we think about the six hydrogens, now each of those are going to bind by combining one of the carbon hybrid orbitals to a 1 s orbital of hydrogen.
So, if we think about what bonds are in this molecule, we actually have six of these sigma carbon s p 2, carbon s p 2 bonds.
We also have carbon s p 2 hydrogen 1 s bonds.
How many of those do we have?
Yup, we also have six of these, because we have six carbon hydrogen bonds.
So that's two of our types of bonds in benzene, and we have one type left, and that's going to actually be the double bond or the pi bond that forms between some of these p orbitals.
So we can have one bond here between this carbon's p orbital and this carbon's p orbital.
So let's have a clicker question here on how many total pi bonds do you expect to see in benzene?
Oh good, so it's left up -- the notes are left up on this screen right now.
All right, so let's take 10 seconds on that.
All right, great.
So, most of you saw that what we would expect to see is a three bond, some of you thought six.
So let's take a look at why three is correct.
So, what we end up having is three of these pi -- 2 p y 2 p y bonds, we can have one between these two carbons here.
Can we have one between these two carbons here if we have one here?
No
No, we can't.
We're already using it up in this pi bond here, so that means we're limited to only two other spots on the molecule, so we have three.
But, of course, what you saw in recitation, and hopefully, what you can now think very quickly by looking at this, is that this is not the only configuration of pi bonds that we could have in benzene.
There's absolutely no reason I couldn't have switched it around and said that instead the pi orbitals form between these atoms instead of those first atoms I showed.
So, let's look at this in a more simple structure here where we have the two possible forms of benzene, and the reality is is that it's going to be some combination of the two.
This is resonance, this is a case of a resonance structure.
So what we see is that those six pi electrons are actually going to be de-localized around all six of those atoms.
So if you think about any one of these carbon-carbon bonds, what type of a bond would you expect that to be?
What would the bond order be for this bond?
[INAUDIBLE]
Yup, it's going to be a 1 and 1/2 bond.
It's a 1 and 1/2 because it's halfway between a double bond and a single bond.
So, of course, this is resonance so we can go ahead and put our resonance notation in there to indicate that benzene is a resonance structure.
All right.
So let's quickly talk about our last type of hybridization that we're going to discuss today, which is s p hybridization.
So s p hybridization comes now from when we're combining an s orbital now with only one p orbital.
So let's take a look at this with carbon.
And if we hybridize these orbitals in carbon, what we end up with is having two hybrid orbitals, and then we're going to be left with two of our p orbitals that are each going to have an electron associated in them.
So again, looking at the shapes, now we're just combining two, we've got these two equal hybrid orbitals plus these 2 p orbitals here.
So let's take the case of acetylene where we have two carbon atoms that are going to be triple bonded to each other, each are bonded to a carbon and then to one hydrogen.
So this is a little bit trickier to look at and see what it means, but essentially we have two hybrid orbitals, which are shown in blue here, and then we have one p orbital that's left alone that's going up and down on the page.
And then that second p orbital's actually coming right out at you, it's coming out of the screen at you.
So, if we think about this z bonding axis between the two carbon atoms, we can picture overlap of those s p hybrid orbitals, and then we can also picture bonding to hydrogen.
So, in a settling, what is the bond angle here?
This is the easiest question all day, what is the bond angle between all of these?
Great, so it's 180 degrees.
So, if we think about the bonds that are forming -- oh I see our TAs are here, so you can start handing them out, because we have two minutes left to go.
So, as they're very quietly handing out your class notes, let's think about what this bond is here, this boxed bond, is it a pi bond or a sigma bond?
It's going to be a sigma bond.
So, we have sigma 2 s p, carbon 2 s p. So they're two s p bonds combining.
Now let's think about this first pi bond, which will be above and below the bonding axis.
Is this pi or sigma?
This is pi.
So we're talking about pi carbon 2 p x, because it's the x axes combining to carbon 2 p x.
And the last bond that we have here is a carbon-carbon bond, and this is our last p orbitals that are coming together.
These are the ones that are coming right out at you, so this is going to be on a second pi orbital.
So this will be pi carbon 2 p y, carbon 2 p y.
All right.
So, we'll stop here today.
Just stay in your seats for another 30 seconds as they're handing out your notes.
I want to mention that you're going to get problem-set five is posted today, and I'll write which ones you can already do so far, because you don't have class on Monday.
But remember that you do have recitation on Tuesday, so that could be very helpful with the problem-set, so be sure to go to recitation on Tuesday, and have a great long weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, let's get started.
Everyone can go ahead and take 10 more seconds on today's clicker question.
This is about hybridization, which we will go over more to start today, but just to see where we are as a starting point.
All right, great.
So most of you got this.
The question is what was the hybridization of an atom if we know that it has two hybrid orbitals.
This is really, really important, so I'm just going to write this out for that 20% of you that didn't see this right away.
So we know that when we have hybrid orbitals that come from s and p orbitals, these are our atomic orbitals we have to choose from to hybridize.
So if we know, for example, that we need two hybrid orbitals, that means we needed to have started with two atomic orbitals.
So that means what we would form are two s p hybrid orbitals and we'd be left with our other two p orbitals left over.
So any time you figure out that you're going to need two hybrid orbitals, and we'll go over when you would find this out in just a minute, but any time that happens you know it has to come from an s and a p, so that's why you'll end up with s p hybrid orbitals there.
And exam 2 is happening in just one week from today, so next Wednesday.
The semester moves really fast.
So I can't believe we're already up to exam 2 talk, but we are.
So, you should have all received two handouts when you walked in today, your notes and then an exam 2 info sheet.
Did everybody get that?
If you didn't get it, just raise your hand and a TA will come to you with that.
I see there's a few around, if one of the TA's can jump up and fill those in.
All right, so basically, as with exam 1, all the information you need to know about what's going to be on exam 2 is on this handout.
Specifically exam 2 is going to cover lectures 10 through 17.
Today is lecture 16, so that means it's going to cover today's lecture and then a bit into Friday's lecture as well, so I'll be really clear on Friday where exam 2 material ends, where you can kind of switch off the exam 2 part of your studying.
And it will also be on problem-sets 4 and 5.
So it's going to be on two problem-sets instead of three problem-sets, but as you know, there are a lot of different concepts covered in these two problem-sets.
So it's still a lot to be thinking about and you need to make sure that you really understand all of these different concepts.
So this is explicitly listed in this sheet here, so make sure you look it over.
But I just want to say is well, the concepts that we've covered since exam 1 have been talking about covalent bonds, going along with that are all the Lewis structures that we've done.
Then we talked about ionic bonds, so you need to make sure you can solve those ionic bond problems.
And then we started talking about different types of bonding in terms of thinking about MO theory, and also vesper theory and hybridization.
And then today, and a little bit on Friday, you'll be responsible for some of this new material, which is going to be a little bit of thermochemistry.
Not too much on exam 2, most of that will be saved for exam 3 material, but an intro to thermochemistry, that will also be on the exam, just as far as these two lectures.
So, basically this is all written out here, and I also want to mention in terms of your MO diagrams and MO theory, I sent out an email about this, but I also wrote it out here so you have it written down in terms of what you are responsible for in terms of ordering the energy levels of molecular orbitals.
So that's all written down explicitly right here, this is what you need to know for the exam.
Also it's helpful for the problem-set as well.
And also, in terms of MO diagrams, I think that you will already gone over this pretty explicitly either last week in recitation or yesterday, in terms of what you need to do to get full credit when you're actually drawing an MO diagram.
Just in case you don't have that information or you didn't write it down, I wrote this down for you here as well in terms of things like making sure to draw your energy arrows, or remembering to label all of your atomic orbitals.
We don't want anyone getting points off on the exam for doing something silly where they do understand what's going on, but they're just not writing it in the correct form.
So all this information is on your sheet here.
And also on this sheet mentions that you will be getting optional extra problems on Friday.
This is just like for exam 1.
The way that you should approach these extra problems, you don't need to turn them in, you're not required to do them, but if you want to do well on the exam we really, really encourage that you do these problems.
And in terms of having it be worth your time when you're solving these problems, what you want to do is make sure you're at the point where you can put your notes away, where you can sit alone without your friends, and really approach these extra problems as if they're exam problems.
This way if there's some conceptual leap that you haven't quite made yet, but you don't realize it because you've been using your notes or using the conversations with your friends, you want to make that leap and run into those problems alone in your room when you have two hours to figure it out.
You don't want to do this leap on an exam.
That's not the ideal situation.
So especially with Lewis structures where sometimes you can run into confusions that you need to work through and make sure it all makes sense in terms of your head how to do this, make sure you take the time to do that before the exam and don't have to end up struggling with that on the exam in a time situation.
All right, so the extra problem solutions will be posted on Sunday.
And in terms of office hours for next week, my office hours are going to be on Monday from 2 to 4, I'm moving them up, and your TAs will move up their office hours as well, so check with them in recitation or on the website.
So that's pretty much it for exam 2.
Are there any questions about what you're responsible for or anything regarding exam 2?
OK, and in terms of the room it's in Walker again, so at least this part should be familiar.
You'll be used to going to your row, it'll be the same row, it's all written here again, but you can just do exactly what you did for exam 1.
All right, so one last class announcement, and this is that if you've been looking at our first sheet of the whole year, sort of the course overview sheet, you know that our class in terms of grading is out of 750 points, and 50 of those points are for attendance and something called occasional in-class quizzes.
So what we're going to do is start filling in some of those points because we only have 36 classes worth of attendance.
So we have a couple of points to make up in terms of quizzes, and these are going to be clicker quizzes, and let me explain how this is going to work.
Basically, in at least for my part of the class, what we're going to do, because what I really care about is that people are answering the clicker questions, it's really valuable for me.
I don't necessarily care if you get them right or wrong during class, because this is the point at which you're still learning them.
So these quiz points, which could be up to 2 points per class, will be in addition to attendance points, and you'll get full credit if you answer the quiz question.
So you need to be answering the questions because the quizzes will not be announced.
So this is another reward for people that are always participating.
These clicker questions are really valuable to us in terms of the feedback, it really gives me an idea of where the class is as a whole.
So really, it's important that you are answering these questions and this is a way to get your quiz points up, so in addition to attendance, you'll get 2 quiz points.
We'll have our first unannounced quiz today, so you won't know what problem it is beforehand, so I suggest you make sure you're answering all the clicker questions, which most of you do anyway.
And I will just say I recognize that some of you are just scurrying across campus to get here on time.
So I'll never make that first question a quiz question, because I know how hard it is to get from one place to another on this campus, and I know most of you do a great job of sneaking in right at the last minute when you have that long run.
So we won't make the first question a quiz question, but be on the look out for a quiz question today.
All right.
So let's get to today's topic.
We're going to finish talking about valence bond theory and hybridization.
So what we've already done is really cover all the theory that we're going to cover behind how hybridization works, and behind how valence bond theory works.
But what we haven't covered is how do we actually solve problems doing this in the really quick way.
So we're going to do that.
This is going to be a more practical lesson in hybridization.
And once we do that, we'll move on to now talking about energies and enthalpies of chemical reactions.
So this is a real shift.
We spent a while talking about single atoms, and then we've spent most of the material, since exam 1, talking about bonding.
So now we're taking it one step further and we're going to actually get to talk about some chemical reactions.
So we'll do that once we finish up with our hybridization unit here.
All right, so how do we go about determining hybridization in complex molecules?
It's actually incredibly simple, and all that you need to do is think about for the given atom that you're looking at, what you want to do is add up the number of atoms that are bonded to your central atom, or the atom you're considering, and add to that the number of lone pairs.
And what you end up with is the number of hybrid orbitals that you need.
All right, so the theory behind hybridization and picturing everything, that can be pretty complicated, and I do want you to understand that.
But if, for example, on an exam situation, you don't necessarily have to think through everything, you can just use this very quick way to figure out the number of hybrid orbitals that you need.
And what we just said is that any time we need two hybrid orbitals, we know that the hybridization of that atom is going to be s p, because it's made up of one s and one p orbital.
So let's say we need three hybrid orbitals.
What is our hybridization of our atom in this case?
Yeah, it's s p 2.
We want to take three atomic orbitals to make three hybrid orbitals, so it's going to be s p 2.
So what if we need four hybrid orbitals, what's the hybridization going to be in this case?
S p 3
S p 3.
All right, great.
So, really that's it.
Hopefully hybridization just got a lot simpler for any of you that have been struggling a little bit with it.
But let's go ahead and do an example to make sure that we can all do this.
And let's take a look -- first let me mention an exception here.
The one exception to thinking about how many hybrid orbitals that you have are in the case where you have single bonded terminal atoms.
So in the case of a terminal atom that is a single bond, you're not going to hybridize it.
So in that case, just don't even change anything it's just going to be one of the p orbitals, or unless you're talking about hydrogen in which case will be an s orbital that overlaps.
All right, so we can illustrate this with an example of formal chloride.
So this is a good example because we're just dealing with our carbon as our central atom here, and we have three terminal atoms, two of which are single bonded, so we're not going to hybridize those, and one which is double bonded, the oxygen, so we will hybridize that one.
So in terms of thinking about the carbon, what is a hybridization around the carbon atom?
[INAUDIBLE]
All right, I'm hearing some mixed answers.
So let's think.
So the carbon atom is bonded to three different atoms, and it has no lone pairs.
So what's the hybridization?
S p 2
Good s p 2.
So, if we're talking about the c h bond, we're going to say that it's carbon, it's a sigma bond because it's a single bond, and then it's carbon 2 s p 2 bonded to a hydrogen 1 s orbital.
All right, so let's take a look at the carbon chlorine bond here.
Again, carbon is still 2 s p 2, it's the same carbon atom, and it's going to be a sigma bond because it's a single bond, but what about the chlorine, what atomic orbital is going to be bonding in the chlorine?
[INAUDIBLE]
All right.
I'm hearing a little bit of mixes, they're all p's, which is a good start.
It turns out that it's going to be the chlorine 3 p z.
So the reason that it's p is because we're not hybridizing it, and the reason that I specifically say that it's the p z instead of the p x or the p y, is remember that the z-axis, that's our bonding axis, that's our internuclear axis.
So any time we have a p orbital that's involved in a sigma bond, it has to be the z orbital because that's the only one that has the right orientation to overlap along that z-axis.
So we're going to say it's carbon s p 2, and then chlorine 3 p z.
All right, so let's look at the last bond here, which is a carbon oxygen double bond.
We know that any time we have a double bond it's made up of one sigma bond plus one pi bond.
So, when we look at the carbon again, that's 2 s p 2.
But what about the oxygen, what's the hybridization of this oxygen atom?
S p 2.
So we're going to have carbon 2 s p 2, and then oxygen 2 s p 2.
It's s p 2 because the oxygen is bonded to one atom plus two lone pairs, so we're going to have a total of three hybrid orbitals.
All right, so this is not our only bond, we have a double bond, so we also need to talk about the pi bond.
So if we talk about the pi bond, we can say that's carbon 2 p y, then the oxygen 2 p y.
We could alternatively say, and be correct, that it was the carbon 2 p x and the oxygen 2 p x.
Either one of those is fine.
All right, so that's an example of how we can assign very quickly and very easily what the hybridization is of a given atom.
And then also fully describe the symmetry and the atomic or hybrid orbitals that make up bonds.
So this is a lot of your problems on the p-set, so if you haven't finished that section, hopefully you'll be able to get through that section pretty quickly when you go back to it.
And let's take a look at a little bit of a more complex molecule here, which is ascorbic acid or vitamin C. And this is a good example because it's starting to look a little more complicated, but not too much more complicated.
But the reality is if you know how to do assigning the hybridization this way, even if I gave you a 1,000 atom protein, you should still be able to get it completely correct.
Vitamin C is a really important molecule to actually think about in terms of thinking about its shape and its hybridization.
Vitamin C is an antoxidant.
So we might remember that from when we were talking about free radicals.
Free radicals can cause oxidative damage.
These antioxidants, some of our vitamins, including vitamin C are antioxidants.
So that's something that a lot of people in the general population know if they're into their health.
But another really important thing about ascorbic acid is that it's an enzyme cofactor.
And a cofactor just means that it's some type of molecule or atom, in this case it's a molecule, that's required by an enzyme in order for the enzyme to carry out its chemistry.
And the chemistry that the enzymes that vitamin C are a cofactor for are responsible for putting oxygen, o h groups, onto collagen molecules in order to form what is the collagen triple helix.
And collagen is the main constituent of bones, of joints, of connective tissue, of many important structural parts of our bodies.
So you can imagine that vitamin C is very important because we need to keep our collagen intact.
Collogen is actually one of the most prevalent proteins in our entire body, and it makes up a large part of our cells.
So you can imagine if we don't have collagen that is in this triple helix, we're going to run into some problems.
Does anyone, or I'm sure many of you do know, but can someone tell me what the name is of the disease if you have a deficiency in vitamin C?
Scurvy
Yeah, scurvy.
So scurvy, most often associated with ships because of the 1500's and 1600's and 1700's when boating technology was far above biochemistry technology, so people could make ships and go on these long journeys, but people did not yet understand they needed to keep certain vitamins in their bodies, such as vitamin C. Maybe also, they didn't know about -- well they didn't even have the structure, so they couldn't have realized that vitamin C is polar and won't stay in their bodies for very long, it will get excreted out through their urine.
So they were not thinking about how quickly they could run into trouble in the long sea voyages.
And I don't know how many of you had a unit on explorers in elementary school -- I did in fifth grade, and one that always sticks in my head is Magellan's trip around the world, who led the first fleet of ships around the globe.
What they did not mention to us in fifth grade was that 90% of his men died on that journey.
And I understand we were young kids, I didn't want to know that at the time, but it's interesting and it's really important, and the reason for many of these deaths was scurvy.
And it's totally preventable.
And, in fact, the cure for scurvy, which is as simple as taking vitamin C in any form, was known for -- since the fifth century people started figuring this out, but the problem was there was so much false information as well that it was not a general practice on ships to be treating men that were suffering with vitamin C or using it as any kind of preventative measure.
The other somewhat interesting thing I want to say about vitamin C is that it was part of the first ever published clinical study.
And this was a controlled clinical study that was done in the 1700's, the first one that's ever been reported, and not surprisingly it was done on a ship and it was done by James Lind who was a Scottish naval surgeon.
And basically, he took 12 of his men, so this was not a very large clinical study like they do today, but he took 12 men that were suffering from scurvy, and he gave them a diet mostly of just carbohydrate mush, because when you have scurvy, one of the first signs is bleeding gums and teeth falling out, they couldn't really eat too much.
But he supplemented their diet.
He put them in pairs of two and supplemented with 1 of 6 things.
So two of them got a pint of cider a day to see if that helped.
Two of them got half a pint of seawater -- maybe not the best group to be in.
Even worse group to be in is two of them got diluted sulfuric acid.
Tuned out that didn't help so well.
Other things where some got spices, not so bad.
And then two of them actually got citrus fruit.
So no big surprise ending to this clinical study, the people with citrus fruit were completely cured and back to duty within a week, and I'm not sure exactly what happened to everyone else, but let's hope they got switched over as well.
But it's just neat to think about even back then, people were doing these controlled scientific studies, this was one of the first cases.
Unfortunately, it was another 40 years between this study being published and the British Navy requiring supplements of citric acid.
So, really, we've come a long way in terms of disseminating scientific knowledge to the community, and that's a really important part of being scientists -- not just making these discoveries, but also passing along, for example, to these ships full of people that need to know it.
So, in terms of scurvy, we don't see too much of it today, but who should be concerned about suffering from scurvy?
Do we have to worry about our cats or dogs at home, are they getting enough of their vegetables and vitamin C?
It turns out we don't have to worry.
Primates are who should be concerned.
So we need to be concerned, other primates need to be concerned.
It turns out that most other mammals actually biosynthesize vitamin C, so we don't have to worry about most of our pets at home.
Unless you have a guinea pig and they don't, in fact, biosynthesize vitamin C, so I think they supplement most guinea pig pellets with vitamin C, but maybe keep an eye on your guinea pig.
We primates and guinea pigs are the only types of mammals that don't actually biosynthesize their vitamin C. So again, vitamin C is a cofactor, that means it needs to bind to an enzyme, it means that the shape of vitamin C and the hybridization are going to be really important.
So let's take a look and talk about some of these carbon atoms here.
And you can look at your structure of vitamin C in your class notes while you're doing this, and first I want you to tell me what the hybridization is of that carbon a in the vitamin C molecule.
And let's take 10 more seconds on that.
OK, 78%.
We'll do another question like this in a minute.
So if we can switch back to the notes we'll take a look at that.
I think we can get better, I think we can get to the 90's with these hybridization, we just need to follow the rules.
So if we're thinking about vitamin C and we're talking about carbon a, how many things is carbon a bonded to?
[INAUDIBLE]
four things.
We have two hydrogen atoms, an oxygen, and then another carbon.
So that means we need, if it's bonded to four things, we need four different hybrid orbitals, so it needs to be s p 3 hybridized.
All right, let's try this again but just shouting out, what is the hybridization of carbon b?
Good.
OK, I'm going to say that was about 90% of you I heard, which is excellent.
S p 3.
What about carbon c here?
Yup, s p 3 again.
Bonded to four things it's s p 3.
What about carbon d?
S p 2, great.
Carbon e?
S p 2
All right, and this last one, the last 10% of you can join in.
Carbon f?
Great, s p 2.
All right, so we can also think about shape once we know hybridization and how many atoms a carbon atom is bonded to.
What would we say that the shape is of these s p 3 orbitals -- or the shape of the carbon atom.
Yeah, it's tetrahedral.
So this is going to be tetrahedral.
And what about carbon d, e, and f?
What is the geometry of carbon d, e, and f?
Good, trigonal planar.
All right.
So let's actually take this one step further from just talking about the hybridization of the individual atom.
Let's talk about a couple of these bonds.
I won't go through all of them.
I think you can get the idea with just a few, but you can go back and try all of them later.
So let's start with thinking about carbon b hydrogen bond.
Is this going to be a sigma or a pi bond?
Sigma.
We know it's sigma, because any time we have a single bond it's a sigma bond.
So carbon b, we just said, is carbon, it's going to be 2 s p 3.
And then hydrogen, what's the atomic orbital we're talking about here?
1 s. All right, great.
So let's take a look at another one, let's look again at carbon b, but now let's talk about it bonding to its oxygen there.
So again, are we going to be talking about a sigma or a pi bond?
A sigma bond, because again, it's a single bond here.
So what we'll say again is carbon 2 s p 3, and let's take a look at this oxygen that it's bound to.
Is this oxygen -- what is the hybridization of this oxygen right here?
What is it?
OK, good, it's s p 3.
It's bound to two atoms, plus it has two lone pairs, that's a total of four things.
So it needs to have three different hybrid orbitals, so we'll say it's oxygen 2 s p 3.
So let's skip to a different carbon now, let's talk about carbon d, specifically the carbon d oxygen bond.
So let's do a clicker question for this one.
Let's get into the 90's, if you don't mind, this would be very good to do.
It'll make me feel better about you finishing your problem-sets tonight.
So if you talk about the symmetry and the hybrid or atomic orbitals that contribute, we're talking about the c d oxygen bond.
So go ahead and look at your notes and see what that bond is.
We already identified the hybridization of carbon d, so we really just need to think about the oxygen here.
All right, let's take 10 more seconds.
OK, I'll take that, 84%.
That's not bad.
So again, we're looking at a sigma bond here.
It looks like the one people got it confused with was talking about oxygen s p 2 versus s p 3.
Remember, you just need to look at what it is actually bound to, so for carbon d, the oxygen's bound to two atoms plus two lone pairs, so it's going to be s p 3 hybridized.
So we'll call this a sigma bond where we have carbon 2 s p, now it's s p 2, and oxygen 2 s p 3.
All right, let's take a look at just one more here.
So, let's look at the carbon d bond with carbon e. So if we can switch back to the class notes just to see that, though you guys can see it on your notes as well.
So in terms of that, we're talking about a double bond now, so we know that we have to have one sigma bond and one pi bond to completely describe our double bond.
So for the sigma bond, again, it's going to be carbon 2 s p 2.
And now we're talking about the other carbon.
What is the hybridization of the other carbon?
Yup, s p 2.
So carbon 2 s p 2.
All right, but we're not done there.
We also need to talk about the pi bond.
So for the pi bond, we'll talk about carbon 2 p y, and then the second carbon 2 p y.
All right, so I filled in a few more in your notes, so you can maybe cover those up and make sure that you get all of those correct is a good self test before you finish off any in your problem-set.
Does anyone have any questions about figuring out hybridization?
Yes?
[INAUDIBLE]
I'm sorry, what is that?
In this bond here?
Oh, OK, that's a good question.
So the question was why is there not hybridization in these p orbitals here.
So in order to form a double bond, we need to have a pi bond forms, and a pi bond forms from two unhybridized p orbitals.
And if you can kind of picture those p orbitals, if our molecules are this way and they're coming up here, we need to have electron density above the bond and below the bond.
So if they're hybrid, then they're going to be spread out into that tetrahedral geometry.
We want to have them parallel to each other and they need to be the p bonds, they need to be the p orbitals.
Does that makes sense?
So any time we have a double bond, the pi part of the bond is going to be p orbitals.
Yes?
[INAUDIBLE]
No, absolutely not.
It would be absolutely correct to put 2 p x and 2 p x here, you just can't put z, because that's the one that's going to be involved in the sigma bond.
And the other important thing is if you do put x for one, you have to make sure you put x in the other, because they do need to be able to interact with each other.
You can't have one x and one y, but sure, you can have both x or both y. OK, so let's shift gears a little bit and start talking about bonding in terms of chemical reactions.
So we're going to start today with talking about bond energies, and also something called bond enthalpies.
And the first point is just to bring us back to something we're very familiar with.
When we talked about covalent bonding, a concept that we have been discussing is bond dissociation energy.
That's just the energy that's required to put into a molecule in order to break that bond.
This is something that we saw when we were talking just at the very beginning of our unit on covalent bonds.
That energy difference is the energy difference between when we have the c h 4, if we're talking about methane, versus one of those hydrogen bonds breaking, one of those c h bonds breaking.
So we've talked in the past about bond energy, but what I want to introduce to you today is a very related concept, which is called bond enthalpy.
So that's delta h here is what we call bond enthalpy.
So, this is talking about instead of the change of energy accompanied with a bond breaking, we're talking about a change in heat accompanied with a bond breaking.
So whether when that bond breaks it requires heat, or whether it gives off heat when you break that bond or talking about a reaction.
That's what we're talking about when we're talking about enthalpy.
It turns out that enthalpy is, in fact, very related to energy, and we can relate it with this equation here that bond enthalpy is equal to bond energy plus the change in pressure times volume.
So we could, in fact, go back and forth between bond energies and bond enthalpies, but the reality is if we're talking about gases, which we are in many cases, then what we find is the difference between bond enthalpy and bond energy is only 1% to 2%.
So it's not very significant.
And, in fact, if we're talking about solids or liquids, now we can say that the difference between bond energy and bond enthalpy is going to be negligible.
So, for the rest of this class today, we're going to stop talking about bond energy, which I did just for a moment, to orient you back to a discussion that was familiar, we're going to switch our discussion to bond enthalpies.
One reason that we like to talk about enthalpies is unlike energy, which can be a little bit more tricky, bond enthalpies are easy to measure.
It's easy to measure how much heat a reaction gives off or how much heat a reaction takes in.
So we can also talk about something, which is called the standard bond enthalpy, so any time you see this symbol here, which looks like a knot, so it looks like a delta h knot, that refers to the standard bond enthalpy.
Any time we we're talking about a standard, whether it's enthalpy or, as we'll see in the next lecture, standard entropy or a standard free energy, we're just saying that the molecules that we're talking about are in their standard states, which means they're in their pure form.
If they're a gas it means they're at one bar.
Usually this is referring to room temperature, so 298 kelvin.
It's often that when you look up tables of different bond enthalpies, you'll see them in their standard state and it will be at room temperature.
So if we talk about standard bond enthalpy, let's talk about what this is for these c h bonds, which we just saw in methane.
So if we're going from c h 4 to breaking one of these c h bonds, what we see is that the bond enthalpy is 438 kilojoules per mole.
The fact that it's positive tells us that we need to put in that much heat into the system in order to break that bond.
So we now know the enthalpy of a c h bond, but, of course, methane is not the only kind of c h bond that you can envision.
We could talk about the c h bond, for example, in ethane or trifluoromethane or trichloro or tribromomethane, and what you'll find is that the amount of enthalpy that that reaction is associated with depends on the exact type of c h bond that you have.
So, for example, we can see that we have slightly different, not vastly different, but slightly different changes in enthalpy depending on which kind of c h bond that were breaking.
And what I want to point out also is that all of these delta h's, all of these standard bond enthalpies are positive.
Any time we have a positive delta h, we call this an endothermic reaction.
And again, endothermic just means that the reaction takes in energy.
It's very similar to what when we're talking about energy whether something's taken in or released, whether it's positive or negative.
If you have positive delta h, it's endothermic, it takes in heat in order for the reaction to go.
So we can think about instead of talking about all of these individual c h bonds, instead we can talk about what the average value would be for all c h bonds.
So we can say that the average of any c h bond is 412 kilojoules per mole.
So you can imagine if you're looking these things up in terms of a reference table or maybe in the appendix of your textbook, it would be an impossibly huge number of pages to write every single different kinds of c h bond.
So that's why instead they talk about the average bond enthalpies.
So for c h, again, that's 412, and all of the enthalpies we just saw fall within about 8% of that.
And in your book you can look up any type of bond that's listed here.
So we saw c h is 412.
You can look up a c c bond, you can look up a c c double bond, a c c triple bond, and so on.
And you'll note that what these are are mean bond enthalpies, so they're the average of all different types of c h bonds or c c bonds that you can imagine.
So, you might be asking what's so important about being able to look up and think about these different bond enthalpies.
And the reason that it's important is because if you're looking at a reaction, no matter how complicated that reaction is, you can actually figure out what the enthalpy of the entire reaction is by adding up all the individual mean bond enthalpies of the products, and all the individual mean bond enthalpies of their reactants and thinking about the difference between those two.
So we'll do that in just a second, and the reaction that we'll do it with is the oxidation of glucose here.
So, c 6 h 12 o 6, one mole of glucose plus six moles of oxygen, gives us six moles of carbon dioxide and six moles of water.
So what we can find out, and hopefully what we will match up when we look at using the different bond enthalpies, is that the enthalpy of this entire reaction is negative 2816 kilojoules per mole.
So in this case we're saying that delta h is negative.
Does anyone know what it's called when delta h is negative?
Everyone knows, great.
So it's exothermic.
This is an exothermic reaction, the reaction releases heat.
So I just want to mention before we go on, if you look at almost any freshman chemistry textbook, what you'll find is this oxidation of glucose reaction is used a lot in talking about thermochemistry.
And one reason it's talked about is because it's very convenient to talk about something where we start with one mole of glucose and end up with 12 moles of products.
That's going to be helpful what we're doing a practice problem to see exactly how you deal with it when there's different numbers of moles.
But the other reason you always see this reaction is because it's an incredibly important reaction.
The oxidation of glucose is going on all the time in our body, this is our main source of energy for all animals.
So let's think a little bit about why this reaction is so important and so prevalent.
It turns out that if we're talking about plants, plants do the reverse reaction.
So plants take carbon dioxide and water and they turn it into glucose or energy for us, energy stored in the bonds of glucose plus oxygen.
So if we do the reverse reaction, this is actually going to require energy.
Where do plants get this energy?
Yeah, this is just photosynthesis here.
This is the photosynthesis where plants are turning carbon dioxide and water into sugar and into oxygen.
So what happens when we eat the plants or when we eat animals that have eaten the plants is that we perform the reverse reaction now, which is what I just showed you, the oxidation of glucose.
And even though it's not the products of the reaction that are particularly valuable to us, we just breathe out the c o 2, and actually we'd rather have less of that than more in our environment, but what's important here is instead the energy or the enthalpy that's given often in this reaction.
So as I said, this reaction has a negative enthalpy of 2816 kilojoules per mole.
That's a lot of enthalpy and a lot of energy.
So we actually end up using that energy to fuel most of what is going on in our bodies.
So instead of storing it as sugar, once we oxidize the sugar, now we just store it as ATP, and as you know, ATP is the currency of energy in the cells.
So this is why you see this reaction again and again and again in just about any chemistry textbook that you open up as sort of a general reaction.
The reason it's used is it's just so important and so prevalent in terms of thinking about our bodies and how we're staying alive and using energy.
All right, so let's go ahead and use this as an example of what we just said, which is using the bond enthalpies of the products and the reactants to figure out the enthalpy of the entire reaction.
So the way that we do this is we add up all of the individual mean bond enthalpies of the reactants, and we subtract from that all of the individual bond enthalpies of the products.
So we can think about what this will tell us.
If you think about the fact if the bonds are stronger in the products than they were in the reactants, you can go ahead and click in and tell me if you think we'll have a negative or a positive delta h here.
All right, let's take 10 more seconds on this.
OK, great, so negative is correct and some people got totally mixed up.
Didn't just get one thing -- so let's just focus on the right answer to start with.
The correct answer is that it's negative, it's an exothermic reaction.
And actually, lets switch to our class notes to explain why.
And also, this was the quiz question for today, so whether you got the answer correct or incorrect, you get full quiz points if you did, in fact, answer.
But let's still focus on the right answer here and see why it's correct.
So if we're talking about the bonds being stronger in the products, that basically means that we ended up releasing a lot of heat when we made those bonds and the products, and we didn't have to use up too much of that heat to break all the bonds and the reactants.
So that's why we say that delta h is going to be negative.
You could also just do it not conceptually, just plugging it into the equation, but it's better to kind of understand exactly why that is.
It's because it takes more energy -- you gain more energy forming the products than you take breaking up the reactants.
So if we have the opposite case here, if the bonds are stronger in the reactants, now what we're going to find is that the delta h of the reaction is positive and what you're dealing with is an endothermic reaction.
So let's go ahead and do this with our example of the oxidation of glucose.
So, I've just written out the glucose molecule so you can see all of its individual bonds here since we're going to be using that information.
So let's start by talking about the bonds that are broken in terms of thinking about all of the reactants here.
So if we're talking about the sugar molecule itself, what we have is we have seven c h bonds.
And if we add up all the o h bonds, we have five of those.
How many c o single bonds do we have?
What do people think?
Yeah.
five c o single bonds.
What about c c single bonds?
five of those as well.
C o double bonds?
Just one c o double bond.
And then we have our oxygen molecule to worry about and we have six o o double bonds.
So if we add up all of the bonds are broken, and we subtract from that the bonds that are formed, those strengths, the c double bond o, we have 12 of those.
And how many o h bonds do we have?
Right, 12 as well.
So if we add up all the bonds broken, what we end up with is 12,452 kilojoules per mole, and it's talking about per mole of glucose that's oxidized.
And in terms of bonds formed, what we see is 15 kilojoules per mole.
So all we need to do to figure out the change in enthalpy, and when it has this subscript r here, I meant to mention, that means the enthalpy for the reaction.
We just subtract these bonds here from the ones that we ended up forming.
So basically, what we're saying -- excuse me, these are the ones that we broke.
It's 12,452 minus 15,192, which we formed.
So we end up with an enthalpy of reaction of negative 2 kilojoules per mole of glucose that's oxidized.
All right, so if you remember the number that I told you before, it doesn't exactly match up with what we had said is the exact number.
The exact number is 2,816.
It is within 3% though, that's pretty good.
Because remember, we're not using exact bond enthalpies here, what we were using is average bond enthalpies.
So it makes sense that we're going to be a little bit off.
But if all you have in front of you is the information on bond enthalpies, mean bond enthalpies, this is a great way to figure out the enthalpy of an overall reaction.
We can think of a different way, however, to think about the enthalpy of an overall reaction, and this is talking about the heat of formation.
And the heat of formation or delta h formation here is equal to the heat of -- or the enthalpy of reaction, if we're talking about forming one mole of compound from its pure elements, which are in their standard states.
So basically, there's a table that you can look up which tells you the enthalpy of formation for any compound that you're interested in, and this is actually an appendix to of your textbooks, and you will need to use this to solve your problems on p-set.
But let me say that, for example, if we're talking about water here, that's formed from hydrogen and oxygen, if we're talking about the elements in their pure forms at one bar, standard states, and room temperature.
And if we look this up in our textbook, we would find that the delta h of formation for water is negative 286 kilojoules per mole.
Similarly we can look up the same thing, for example, for carbon dioxide, and what we'll find here is that our delta h of formation that we look up is negative 393 .
5 kilojoules per mole.
We can also look up or think about what our delta h of formation would be for oxygen.
Does anyone have a guess here?
Yup, it's actually going to be zero.
Oxygen already is in its most stable state, so any time we're talking about an element in their most stable state, it's going to be zero.
That's going to be the delta h of formation, there is none, it's zero.
We can also look up what it was for sugar, for glucose, c 6 h 12 o 6.
So that coming from its most stable forms, it was the most pure form of the elements, is going to be negative 1260 kilojoules per mole.
All of this information has been tabulated and is in tables, you can refer to them, and they're also in the back of your textbook, so you can refer to those in terms of your problem-set.
So let's go ahead and solve a problem actually using the delta h's of formation.
Let's see if we can do any better to coming close to the reality for the oxidation of glucose.
So if we're going to calculate the delta h for the reaction for the oxidation of glucose, or actually for any reaction at all, what we want to do is take the delta h of formation of the products and subtract from that the delta h of formation of the reactants.
So let's go ahead and do that.
And in terms of talking about the oxidation of glucose, if we talk about delta h of this reaction, what we need to take is 6 times delta h of formation of c o 2, since we're forming, that's in our products, we're forming six moles of c o 2.
Plus 6 times delta h formation of h 2 o, since we're forming six moles of h 2 o.
And we subtract from that the heat of formation of all of our reactants.
So we only have one mole, so we just say delta h formation of c 6 h 12 o 6, and then in addition to that we have six moles delta h formation of oxygen, of o 2.
All right, so this is our equation here and at this point what we would do is we would look up what all of the delta h formation values are for c o 2, h 2 o, sugar, and oxygen.
So what we would find is that we end up having 6 times negative 393 .
5 -- that's what we had just looked up and told you for c o 2.
Plus 6 times negative 285 .
8 for the water.
Minus 1 times 1260, so it's negative 1260 for our sugar here.
And then what we're going to end up with is having minus, and what is it for oxygen again?
So minus 0.
So what we end up with in terms of the delta h for the entire reaction here, is we end up with negative 2816, and it's going to be kilojoules per mole of glucose.
All right, so let's see how this matches up, and hopefully you can actually remember that this matches up actually perfectly here.
So what we are going to see is that, in fact, what we calculated versus what is experimental is dead-on the same.
So there's actually one more way to figure out the enthalpies of a reaction, I'm going to go over it just very briefly.
And that's based on the fact that enthalpy is a state function, and by state function what I mean is that it doesn't matter how you got from point a to point b, all that matters is the difference between the two.
So another example of a state function is altitude, for example, on a mountain.
So if you're talking about altitude, you go from point a to point b on the mountain, and it doesn't matter how you got there -- you could have climbed all the way up the top of the mountain, then went back down and eventually landed on point b.
Or you could have gone straight from point a to point b.
The change in altitude between point a and point b do not depend on the path, they're independent of path.
So this difference right here does not matter on how you got there, it does not make a difference how you got there, altitude is a state function.
So similarly, we can say that enthalpy is a state function as well, if we're talking about part reaction a here, which has our reactants going to our products in 2 b here.
It doesn't matter how we got there.
We can actually calculate the change in enthalpy at all different points here.
All that we actually have to worry about at the end of the day is the difference between a and b.
So that's what we mean by state function.
Actually, we're not going to have a chance to get to fully explaining the consequence of this, which is Hess's law, which allows us to add and subtract different reactions.
So what I'm going to say is the last two problems on your problem-set you won't have to do because we're not going to get a chance to cover in class on Friday.
So I'll send out an email about this as well, so that'll be pushed back.
So, it will be on the exam, so you want to do them at some point, but you don't need to actually turn those in.
So those are the last two problems, I believe.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So again, this is one more question on hybridization.
This is the last question you'll get on hybridization before you see the exam.
So now we're talking about hybridization of two different atoms in a molecule.
So we're looking at this carbon atom here and this nitrogen.
So using the rules that we've learned in terms of doing this quickly, let's see if you can get these answers in quick.
So let's go ahead and just take 10 more seconds on this question here.
Your parents are free to help if they're here with you.
OK.
Excellent.
excellent job, 98% of you.
You're a lot louder with the parents here, so let's make sure we're still listening.
OK, so let's go over what we just we just established here.
Carbon a is going to be s p 3 hybridized.
What is the geometry, what's the vsper geometry there?
Good, it's tetrahedral.
And this nitrogen b here, that's s p 3 hybrid as well.
What's the geometry around that nitrogen?
Trigonal pyramidal, right.
We have to take into account the fact that there's that lone pair there.
All right, so let's go ahead and switch over to the lecture notes and see what we just did here.
And essentially actually what we just did was I had you identify two of four components that make up what's called the morphine rule.
And the morphine rule is a set of structural elements that are responsible for the biological activity of morphine and other morphine-like drugs that have a similar pharmacological activity in terms of how they're active.
So specifically, the morphine rule is a set of four components, which start with the phenyl ring here, so, an aromatic ring.
The second thing you identified, which is an s p 3 hybrid carbon.
And then following that we have a c h 2 c h 2 group here.
And the last part of the rule is this s p 3 hybridized nitrogen.
So, in terms of thinking about what this means, I said this is responsible for the biological activity of morphine.
If we take a look at the morphine molecule here, what you can see is that it's a little bit more complicated than this structure that we see right here.
But if I go ahead and highlight the residues in purple, you can see that it, in fact, does follow the morphine rule.
And what this means is this is what allows morphine to actually bind into its receptor, which is a pain receptor and block the feeling of pain.
So that's a very important thing in terms of thinking about hybridization in geometry, because we've actually established a very important structure and shape, which is this morphine rule here, that if it's found in a molecule, as long as there aren't other parts of the molecule that are messing up that interaction, we can actually form a very tight binder into these pain receptors.
So morphine, as you know, is a very potent pain killer.
But you also, I'm sure, know that it's also very addictive, so it's only used sparingly in terms of treating pain in a general sense, and it's always used in very closely-monitored situations in terms of hospital care.
But what's interesting to take note of is that the bioaction of morphine is very similar to endorphins that we naturally biosynthesize.
Endorphins have a structure that's very similar to this structure up here -- endorphins are small peptide molecules that we biosynthesize and have in very low concentrations in our brain.
They bind to pain receptors and block those pain receptors.
So, when someone talks about an endorphin high, which you sometimes get -- one example would be the runners' high, if you run for a long way running fast, eventually you'll hit that runner's high where you get this burst of endorphins.
All of a sudden your feet don't hurt, your shins don't hurt, the pain goes away and you got this feeling of eurphoria.
It's the exact same interaction and it's because of that structure there.
It's not exactly the morphine rule in endorphins but it's very similar.
So, if we look at other derivatives of morphine, such as codeine and diacetylmorphine, these also you can see have this morphin rule within them, this structure that you've identified using your vsper rules and thinking about hybridization.
Codeine, as you may know, is less of a potent pain killer compared to morphine, but it's also less addictive.
So for that reason it's prescribed with slightly less oversight.
So if you get your wisdom teeth removed or something like that, some of you might have had taken codeine before, and now you can think about by looking at the structure, a little bit more about how that worked in terms of blocking your pain.
One that I just want to mention because it's so interesting is this derivative, which is diacetylmorphine.
The only difference between morphine and diacetylmorphine is the changing of an alcohol group, or two actually, two acetyl groups here.
This was synthesized by the company Bayer, and you probably know Bayer from Bayer's aspirin.
Bayer had a huge success with aspirin where they took salicylic acid and replaced an o h group with an acetyl group here, and made acetylsalicylic acid.
Does anyone know the common name for that?
Yup.
So that's aspirin, and we often think of Bayer's aspirin.
We associate it really closely.
They worked really hard in marketing to make that really close association and with copywriting, so that when we think about aspirin we think about Bayers.
Bayers also made this change here, and they did find that diacetylmorphine is much, much more potent than morphine is.
They could use just a tiny, tiny amount of this compound and get the same pain-killing properties that they saw with morphine.
The problem is that this masked some of the side effects that they didn't realize initially, which was the extreme, extreme ability for you to become addicted to diacetylmorphine.
And this was first thought of as a hero drug because it was such a strong pain killer, and this was called a Bayer's heroin.
We don't usually associate the name Bayers with heroin, like we did with aspirin -- that's because Bayers did not work so hard to keep that association there.
But it's an interesting story from the history of drugs.
One more example I want to show you is demerol, which is used clinically quite often.
Demerol, if we look at the structure here, actually looks nothing like morphine at all.
So to an untrained eye, you might not see the relationship there.
But because you all know some of these basic principles of chemistry, you should be able to pick out that morphine rule right in the middle of demerol.
And, in fact, demerol is a pain killer, it's an alternative to using morphine.
Again, not quite as strong as morphine in terms of pain killing abilities, but also not as addictive either, and there are some other side effects, such as nausea that are limited with demerol compared to morphine.
Demerol has its own set of problems, so it's not the perfect solution to a different alternative to morphine, but you do see it used in certain cases.
All right, so that's thinking about vsper theory, and that's thinking about hybridization.
We're now going to shift gears to talking about some new topics today.
So, we ended in Wednesday's lecture with giving that set of rules to very quickly determine hybridization.
And then after we'd established that, we moved on to starting to talk about a little bit of thermochemistry.
Specifically what we were talking about is the enthalpies of chemical reactions, so either the amount of heat that's released, or the amount of heat that's required in order to have a chemical reaction go.
And we'd established so far two ways to think about how we can measure what the overall enthalpy of a reaction is.
And I just wanted to introduced one more technique that we can use to do that.
And this relies on the fact that when we talk about enthalpy or we talk about the heat change of a reaction, this is a state function, and any time we're talking about state function, it's independent of path.
So essentially, all that we're worried about is the state that we started in.
So, for example, when we were talking about the oxidation of glucose, the state that we start in are the reactants, so it's glucose plus oxygen.
And what we ended up with were six moles each of carbon dioxide and water.
So, we don't care how we get there in terms of making this calculation, all we're interested in is the difference between those two states, to think about the overall change in enthalpy.
So, for example, when we talked about how we could calculate this, one way that we can do it, and we'll look at these numbers even more specifically in a second, is thinking about instead of going straight down here, which we might not have the numbers available to us to calculate directly, we can think about well, what happens if instead we break apart this glucose molecule and decompose it into its elements, because using tables in the back of our book, for example, we can figure out what the change in enthalpy is of this reaction here.
That doesn't quite get us to where we need to go though, so then we can think about the enthalpy of formation of six carbon dioxide molecules.
And then we can go one step further and think about the enthalpy of formation for six water molecules.
So this is just a round about way of going from here to here.
Instead we went up and then down and then down, but because it's a state function, we're absolutely allowed to do this.
And to formalize this, we can talk about what's called Hess's law.
And Hess's law just tell us that if we have two or three or any number of chemical reactions that we can add together in order to get the chemical equation that we're actually interested in, we can figure out the change in enthalpy of that reaction that we're interested in, just by adding together all of the other enthalpies of reaction for every other reaction that we had to add together in order to get there.
So, let's take a look at what we're talking about here.
So when I just showed you that example graphically, I'm just writing it out more like an equation here, because the nice thing about Hess's law, is it allows you to think about chemical reactions kind of as if they're algebraic expressions.
So we can just add all of these different expressions together, and if we're thinking about algebra, we can just think about crossing things out that are in the reactants versus in the products.
So if we do this, what we end up getting is the reaction that we're interested in, the oxidation of glucose to form carbon dioxide in water.
So let's look pretty carefully at exactly the fact that these all cancel out to give us the reaction I say we get.
So, for example, we can think about the oxygen molecules -- we can cancel out 6 here and 6 here, as long as we multiply this whole reaction by 6.
Again, we can get rid of those last 3 oxygen molecules there.
Similarly, we can cross out our 6 hydrogens from the products with 6 molecular hydrogens from the reactants.
And then the last thing we can cancel out are our carbon atoms -- we have 6 in the products, and we have 6 in the reactants here.
So you'll see that the only things that are not canceled out is what's left in our bottom reaction.
So we can go ahead and just add up all of these different reaction enthalpies.
In the first case what we're going to have is an enthalpy of 1260, and that's in kilojoules per mole of glucose.
And the reason that this is positive is because it's essentially the reverse reaction of the change in enthalpy of formation.
So it's going to be a positive number there.
We add to that the enthalpy for this reaction here, which is negative 393 .
5, but we need to remember to multiply that by 6, because we're actually adding together six of those individual reactions.
And lastly, we also add together six of this final reaction here, which has an enthalpy change of negative 285 .
8.
So if we go ahead and just add up all of our individual enthalpy changes, what we're going to end up with for our entire reaction here, is a change in enthalpy of negative 2816, and that's kilojoules per mole of glucose.
Does anyone remember from Wednesday's class, does this match what we had seen experimentally?
[INAUDIBLE]
Yeah, so this is the exact number that we calculated when we used heats of formation, and it's also the exact number that's seen in terms of experimental.
All right.
So there's actually three ways we now have to calculate heats of the change in enthalpy of an overall reaction.
So, we have these in our notes from last time and from finishing the end of this unit right now, but I'm just going to summarize them for you on the board, because there are a couple things that can get confusing that I want to make sure no one is confusing, particularly in the upcoming exam.
So what was the first way that we learned to think about calculating enthalpies of reaction?
[INAUDIBLE]
Um-hmm.
So, it's talking about bond enthalpies.
And when we talk about bond enthalpy, the symbol that we see for a bond enthalpy is usually delta h. And this is where all the confusion starts, because everything's delta h, it just depends on the subscript here, right, what kind of delta h we're talking about.
In the case of bond enthalpy often you'll see no subscript at all.
But sometimes you do see a subscript, which would then just be delta h sub b here.
So if we're trying to do this calculation based on bond enthalpies, for the delta h for an overall reaction, what we want to do is take the sum of all of those bond enthalpies of our bonds that are broken, and what we want to do is subtract from that the sum of all our bond enthalpies of our bonds that are formed.
All right, so let's think a little bit more about what this means.
So, if we're talking about bonds broken, are we talking about reactants or products?
[INAUDIBLE]
.
Reactants.
So, essentially we're summing up the bond enthalpy of reactants and subtracting it from the products.
All right, great.
So that's our first strategy there.
The second strategy that we learned is thinking about bond enthalpies of reaction by using the enthalpy change in terms of the enthalpy of formation.
So this one is sometimes a little bit more intuitive, because we're talking about the enthalpy change it takes in order to form a given molecule.
So, if we're trying to do a calculation using enthalpies of formation, what we find is that delta h of the reaction is equal to the sum of the delta h of formation of products or reactants?
Products
Products, good.
Of products minus the delta h of formation, and this is the sum again of reactants.
All right, and our third strategy is what we just talked about, which is Hess's law, and in fact, both one and two are applications of Hess's law -- specific applications, but any application of Hess's law you can use any way of adding up different equations to get the final enthalpy of the reaction.
The reason I wanted to put these two strategies, however, on the board right next to each other, is so we can just confront a point of confusion that happens a lot with students, which is why in the first case do we have reactants minus product, and in the second case we have products minus reactants.
So, you need to keep these two straight when you're doing your calculations in terms of enthalpies, and the reason is because the definition of a bond enthalpy when we're talking about bonds, is the enthalpy that it takes in order to break the bond, the amount of heat that goes in to breaking a bond.
So it's typically a positive number, because if you have a stable bond you have to put heat into it in order to break it.
So you end up having a positive bond enthalpy here.
In contrast, when we're talking about enthalpies of formation, now we're talking about is the change of enthalpy in order to form a molecule.
So, for example, if we have a nice stable molecule with strong bonds in it, what we're going to find is that the delta h of formation, would that be positive or negative?
So it's negative.
So you'd end up with a negative delta h of formation.
So because you're talking about a term that's negative here where it would be positive in the first case, you end up flipping signs to have it work out.
So, if this doesn't make sense as I'm saying it right now, just go back in your notes and look through and make sure that you aren't going to get mixed up when you're using bond enthalpies versus enthalpies of formation here.
All right, so that's pretty much all we're going to say on enthalpy.
So I really just want to stress, this is going to be the end of exam 2 material here, I said I'd be clear about it in class today.
This is it.
It's also written in your notes when you go ahead to study.
So remember, in terms of exam 2, which I'll remind you is on Wednesday, it's going to be all the way from the material on lecture 10 up to this point here.
So only through enthalpy, and it's also going to be on problem-sets four and five, and I'll mention that this afternoon, as I said, I'll post a bunch of extra practice problems, which are more review of these same concepts, so you can get even more practice this weekend and early next week to prepare.
All right, so that's enthalpy, that's the end of exam 2 material.
But there's actually another really important concept that we need to talk about -- we really don't want to stop with enthalpy, luckily it's exam 2 and not the end of the class, because a really important concept to think about is spontaneous change.
So thinking about whether a reaction is spontaneous or non-spontaneous.
And when we think about a spontaneous reaction, I think this is a term many of you are familiar with, spontaneous just means that the reactions going to proceed in the forward direction without any kind of outside intervention.
So thinking of a spontaneous process, can talk about a chemical reaction, but you could just picture, for example, putting a round rock on the top of a hill -- it will spontaneously just roll right down that hill.
So a spontaneous process also has direction, right, because the rock won't spontaneously roll back up the hill without putting in work.
So essentially we're talking about a spontaneous process when something's going to happen without actually having to do anything else to force it to happen.
So let's think about in terms of chemistry what we're talking about is a spontaneous reaction -- that's the specific type of spontaneous process that we're interested in.
So let's think about a few different types of spontaneous reactions, and see if we can come up with some idea of what's going to cause them to be spontaneous.
So one spontaneous reaction is written here, so this is just the oxidation of iron, or the formation of rust.
This turns out to also be an exothermic reaction, it has a negative delta h of 824 kilojoules per mole.
Another spontaneous reaction is written here, the combination of an acid and a base, which neutralizes each other.
So we have a hydronium ion and a hydroxide ion interacting to form water.
This is spontaneous, and again, it's exothermic.
So, its delta h is negative 55 .
9 kilojoules per mole.
Let's think about some really relevant reactions in our bodies.
So one incredibly important reaction, of course, is ATP hydrolysis where we have adenosine here, a triphosphate, so we have three phosphate groups here.
That's called ATP.
It has a total charge of minus four.
So if we hydrolize this, one of the phosphate bonds here and lose a phosphate, we end up with adenosine diphosphate or ADP and that has a charge of minus three.
Yup?
On our notes, this equation [INAUDIBLE]
Oh, no.
OK, thank you.
Which equation are we talking about?
[INAUDIBLE]
I'm sorry, what page are you on?
Page two.
So the one with h 3 -- OK. All right, let's go back to that.
OK, so if you can fix this reaction in your notes here.
I'll also fix it on the website, so if you don't want to fix it now I'll just re-post the notes.
So it should be four irons plus three oxygens is two f e 2 o 3 solid.
Thanks for pointing that out.
All right, does everyone have that down?
I know you had to flip a page there.
All right, so I'll post it in the notes if you didn't get it there.
Let's go back to ATP hydrolysis here.
So we're going from ATP to ADP, and that is a spontaneous process, and it's an exothermic reaction as well.
So we find that it's negative 24 kilojoules per mole in terms of the change in enthalpy there.
All right, so at this point I just showed you three reactions where we have something that's spontaneous and it's also exothermic.
And I could show you a countless number of other reactions where it's the same thing.
It's actually quite common if you have an exothermic reaction that it's also spontaneous at room temperature.
So we might start to draw the conclusion that, in fact, it's enthalpy that's responsible for whether or not a reaction is spontaneous or non-spontaneous.
So it's very easy to get that impression, but let me show you a few more reactions before we draw any conclusions.
Let me show you some more spontaneous reactions.
So, for example, the conversion of solid h 2 o to liquid h 2 o at room temperature, I think we all know this is spontaneous.
Ice melts at room temperature, but it turns out that the enthalpy change is positive, it's 7 kilojoules per mole.
Similarly, if we look at this reaction here, which is ammonium nitrate.
This is a very commonly used fertilizer, a very commonly used nitrogen source in agriculture.
If we think about solvating this and forming the two ions, ammonium ion and nitrate ion.
Again, this is a spontaneous reaction, but what we find here is that delta h again is positive, it's also endothermic.
So thinking about all of this, would you say that enthalpy is the key to spontaneity?
No
No, it's definitely not the key to spontaneity.
It tends to correlate in many cases at room temperature, but it is not the key to spontaneity.
And the key to spontaneity is instead something called Gibbs free energy or delta g. And if we think about what delta g is, we can relate it to enthalpy, and this will sort of show us why there's often this correlation, because delta g is equal delta h minus this term here, which is temperature, and that's temperature times a change in entropy.
And we'll talk very in-depth in just a few minutes about what entropy actually is, but for now I just want you to think about the fact that in addition to thinking about enthalpy there's this other term here that comes into play.
So when we're talking about what the sign about delta g is or free energy is, some of you might already be familiar with this, when we're talking about a negative delta g, is this spontaneous or non-spontaneous.
So, maybe not so familiar, which is totally fine.
So in terms of thinking about a negative delta g, that's going to be a spontaneous process.
Any time you see a negative free energy, which just like when we talk about entropy, it's a similar idea, it means that free energy is released as the reaction progresses.
That's going to be a spontaneous process.
So that means that if delta g is greater than zero, then what we're going to see a non-spontaneous process.
And finally in the case where delta g is equal to zero, at this point we're at equilibrium, which basically means that there's no net change in either the forward or the reverse reaction in terms of thinking about whether something's spontaneous or not spontaneous, it's just at equilibrium we see no net change.
And something I want to point out is that this reaction is valid only when where at constant temperature and pressure, which we will be throughout the discussions in this class in terms of using this equation.
And also, it turns out that when you do most chemical equations, you are, in fact, at a pretty constant temperature, and a constant pressure in terms of most things are done in the open atmosphere, so that's really not going to change.
All right, so let's just talk very briefly about why it is that free energy is what determines spontaneity, and not this enthalpy here, not what the change in enthalpy is.
And it really comes in terms remembering it in terms of the definition.
When we think about delta g, what that is is that it's energy that's released or used in the reaction, but if it is released, that it can directly be used to do work.
It's what we call free energy -- it's free to do other things.
Whereas when we were talking about enthalpy here, well, this is the amount of heat that's released when we break and form the bonds in the reaction.
But some of it actually gets stuck in the molecules, and this is just a way to think about it, this term here is one way we can think about it, it's just some energy that's getting stuck.
So, for example, molecules, which we don't really go into in this class, have vibrational movements, they have rotational movements, there's energy associated with that that can sort of get stuck, as we could say, some of the enthalpy that's released in terms of forming and breaking the bonds in a reaction.
So that's our case where we could see that we have a negative delta h, but we still end up with a positive delta g. And again, I haven't really gone into what entropy is here yet, and we will just in a few minutes.
But first, I want to take a look at one of the examples that we talked about, which is the conversion of ammonium nitrate when its solvated, breaking up into its ions.
So what I had told you is that, in fact, this is a positive delta h, but still that the reaction is spontaneous.
So let's just do this calculation and see if we can confirm that in terms of delta g. So the reaction again that we're going to use is delta g equals delta h minus t delta s. So if we're talking about room temperature here, so we're going to say that delta g in this case is going to be equal to 28 kilojoules per mole, minus -- we're at room temperature or 298 kelvin -- in this reaction, you always put temperature into kelvin.
And then we need to multiply it by the entropy, which is 109 joules per kelvin mole.
I want to point out that entropy tends to be a much smaller value than enthalpy, so it's reported in joules instead of kilojoules.
When you solve these problems, it's really important to convert it into kilojoules so that your units work out and you don't get a completely crazy answer.
So we want to convert this to 0 .
109 kilojoules per kelvin mole.
So we can just re-write this, so we have 28 kilojoules per mole, then this term here will be 32 .
5 kilojoules per mole.
So what we end up with our delta g for this reaction is going to be negative 4 kilojoules per mole.
All right, so talking about this as a negative delta g, would you say this is a spontaneous or non-spontaneous reaction.
Yeah, it's spontaneous.
So what we're going to see is delta g is negative 4, so this reaction is spontaneous even though our enthalpy change was positive.
Let's take a look at one more reaction since we spent so much time discussing the oxidation of glucose.
So again, what we know about this reaction, we've already calculated the delta h is negative 2816 kilojoules per mole.
And you could look up and we'll figure out how to calculate it soon, the change in entropy, which is plus 233 joules per k mole.
So before we actually do this calculation, let's go to a clicker question and I want you to think about what's happening here.
So if you're thinking about the oxidation of glucose and what the sign is of the change in enthalpy and the change in entropy, which of these statements is true.
Is it going to be spontaneous at all temperatures, non-spontaneous at all temperatures, or will it depend on the temperature whether this reaction is spontaneous or not spontaneous?
So, remember this is our relationship here.
So let's go ahead and just take 10 seconds on this one.
OK, so we have a pretty mixed response here.
So this could have made or broke the chances of your team on the competition today, but they'll be a few more left to redeem yourselves if you got it incorrect.
So what we're going to find is that it's spontaneous at all temperatures, and we can actually just look at this equation here to figure out why that is.
If our reaction is spontaneous, that means that delta g is negative.
So in terms of delta h being negative, well, that's going to always contribute to delta g being negative no matter what the temperature is.
And if delta s is positive, since it's minus t delta s, and our temperature is always positive because we're on the kelvin scale, so it starts at zero there, then this component here is always going to be negative as well.
So we're always going to have two negative numbers so that our combination, our delta g's always going to be negative, it's always going to be spontaneous.
All right, so let's make sure that's what we do see when we calculate this for the oxidation of glucose here.
So if we do this and we plug in our numbers, we see that delta g is going to be equal to negative 2816 times the temperature, room temperature, times again, remember to put 0.
233, because we need to convert from joules to kilojoules for our entropy term.
So what we find out is that this reaction has a delta g of negative 2885 kilojoules per mole.
This is a spontaneous reaction.
All right, so let's get back to that entropy term.
I introduced it without explaining at all what it really is.
So let's take a few moments and think about entropy.
Entropy is actually a very easy concept to think about, it's a measure of disorder.
I think most of us have a very good concept of how things tend to go to disorder, so it's something we can conceptualize just in an infinite number of ways.
And entropy is simply a measure of the disorder of a system.
And when we're talking about chemical reactions, what we're talking about is a change in entropy.
So whether, as a reaction goes forward, are we becoming more ordered or are we becoming more disordered?
So just like we saw for enthalpy, entropy also is a state function, so it doesn't matter in terms of path how we got from point a to point b.
All we need to worry about is the current state of our system, what the current entropy is and take the difference.
So, an example that I like to use in terms of conceptually thinking about what's going on in terms of disorder, and which is especially relevant in New England is thinking about stone wall.
So you can think about a stone wall in terms of being very, very ordered, right.
If you just built your stone wall, every stone is in place, it has a high degree of order.
So in other words, the disorder is very low.
So it has a low entropy, a low level of disorder.
But if we're looking at a New England stone wall, which are typically found if you're hiking in the woods you see ones that are just ancient and are completely crumbling down, you'll find that the stone wall is no longer quite so ordered as it was.
In fact, let's say there's a few stones that have fallen off, our disorder has increased, so we end up increasing the entropy.
And I do just want to point out with this analogy that it is a state function.
So for thinking about the change in entropy, which is this distance right here, we can either calculate it directly from here to here.
but it also doesn't matter, let's say the wall crumbled down completely and we ended up having a very high degree of disorder, if then someone put most of the stones back in place and we went back down to this degree of disorder here, it doesn't matter, it's a state function, all we care about is the actual difference between our starting point and our ending point.
And I just can't resist putting in a little quote, "something there is that doesn't love a wall."
Does anybody know where this quote is from -- very famous poem, very famous poet.
Good guesses, parents can help with this.
Yes, excellent.
I've never heard the correct answer this question before.
Excellent.
So this is Robert Frost in The Mending Wall.
He talks about, it sounds like a lot of you did know this, this makes me full of joy.
"Something there is that doesn't love a wall," there's many interpretations for this poem, but maybe it's about entropy.
Entropy doesn't love a wall, we're going from order to disorder, that wall wants to come down.
All right.
So let's formalize these thoughts on entropy and in terms of what we're talking about.
Again, what I said on that's chart there is as we increase entropy, we're increasing disorder.
So if you saw a positive change in entropy in a reaction, would you think about that as being more or less ordered?
[INAUDIBLE]
Yeah, so it's actually going to be less ordered, and, in fact, I should stop saying order, I should be just saying disorder because disorder's actually what we're measuring, and there's an increase in disorder here when we have a positive change in enthalpy.
So if we have a negative change in enthalpy, we're going to see a decrease in disorder.
So in terms of considering different types of states that we can be in, whether we're in a gas state or a liquid state or a solid state for any given molecule or any given compound, we can think about how much order or disorder these states have.
So which of these three states would you call the most disordered?
Gas
Yeah, it's the gas state.
So it turns out that the disorder gas is greater than liquid., which is greater than solid.
This makes sense and the gas molecules are free to bounce around, go wherever they want.
Once you're in a liquid, they have a little bit more limitation, they can't go anywhere.
But the molecules can still slide all around and pass each other.
Once you're in a solid state, this molecule is not going to flip places with the other molecules there.
It's going to be kind of stuck in its place within the solid.
So, just understanding this explanation of entropy that we're going to have an increase in entropy if we have an increase in disorder, allows us to make predictions about reactions even without doing any calculations, and we always like when that happens, because then once we do calculations, we can check our calculations and make sure they make sense with what we would be predicting.
So before we talk about how to actually do a calculation, let's go to a clicker question here and make sure everyone is on the same page in terms of thinking about changes in entropy or changes or delta s for the reaction.
So let's say we're talking about the decomposition of hydrogen peroxide into water and oxygen.
I want you to tell me if you think it's going to have a positive delta s, a negative, or zero, or maybe this will change with temperature as well.
All right, so let's go ahead and take 10 seconds on this one.
OK, excellent.
Great job with the questions today.
So we are actually going to see a positive delta s. this should be very clear because we're going from two moles of liquid, to now having two moles of liquid, plus a mole of gas, so we're going to be increasing the disorder of our system here.
All right, so let's think about how we actually can calculate, though, delta s. So far I've just been giving you the delta s's for the reactions, but, as I'm sure you can guess, you won't be given that information, for example, on upcoming problem-sets, you'll actually need to calculate the entropy.
So what we can do is we can actually calculate the entropy of a reaction from absolute entropies of individual molecules that we're discussing.
So this is very similar to how we calculated the enthalpy of a reaction by taking the change of enthalpy of formation, except we don't even have to worry about a change in entropy because we can talk about entropy as an absolute value.
There's an absolute zero in terms of entropy where there is no disorder at all.
And this is described for any molecule as being its perfect crystal at absolute zero.
So this is what we're going to say there's absolute complete order, there's no disorder at all in this system.
So what we can take to calculate the entropy in a reaction, is just to take the sum of the entropy of all the products, and subtract from that the entropy from all of the reactants.
So let's try this for the example that you just did on the clicker question, and make sure that our calculation matches up with our prediction here.
So again, we're just going to take the sum of the entropy of the products minus out of the reactants.
So if we talk about this, we're starting with our product, so our first product is water.
And what I want to point out is you need to make sure you multiply this by 2, because, in fact, we have two moles of water that are formed here
[INAUDIBLE]
OK, I'm so sorry.
Yeah, so yikes, this should be water.
You know what, let's just write this on the board.
So we're going to talk about delta s of the reaction.
So that's to be sum of the entropy of the products.
So we're going to be talking about 2 times entropy of water, and we'll add to that our other product, which is molecular oxygen here, and we don't need to put a number out here because it's just one mole.
So that's of o 2, and we'll subtract from that, the entropy of hydrogen peroxide here.
And do we need to put anything out here for this?
2
So, yeah, because there's two moles there.
So water plus oxygen minus hydrogen peroxide.
Let's hope I at least got the values right.
Yeah, and actually I think this is right, I believe this is the entropy of water.
So OK, so the actual numbers are right here for what is written on the board and hopefully in your notes.
So what we find out is that the entropy of this reaction here, the change in entropy, the delta s, is 125 joules per k per mole.
All right, so we can also -- well, first, we already thought about why delta s is positive.
Delta s is positive because we're going from a liquid to a liquid and a gas -- we're increasing our disorder.
So let's think about whether this reaction is spontaneous or not, and to do that, we actually need to go ahead and completely calculate delta g. So to do this, we can take our values here, and delta g is equal to delta h minus t delta s. So if we plug all of this in, we have our delta h is negative 196 kilojoules per mole, minus our temperature, and I wanted you to write this again, because I want to make sure you get in the habit of converting our joules to kilojoules.
It's 0 .
125 kilojoules per kelvin mole.
So we end up with a delta g of negative 233 kilojoules per mole.
Is that spontaneous or non-spontaneous?
[INAUDIBLE]
Great.
So the reaction here is spontaneous.
So actually, this is probably a reaction that you're familiar with because a lot of us do have hydrogen peroxide just in our medicine cabinet.
And if you go and look at the expiration date on your hydrogen peroxide, you will see that it does, in fact, expire.
And you might think oh no, what happens when it expires, when it goes bad?
All that's happening is it's turning into water.
So I wouldn't drink it after its expiration date, but probably there's more water than hydrogen peroxide at that point.
But it also actually brings out a really important point about thermodynamics, which is that thermodynamics, in fact, tells us if a reaction happening in the forward direction is spontaneous or non-spontaneous.
So, for example, we see that yes, it is a spontaneous reaction.
But you'll note that thermodynamics tells us absolutely nothing about the timeframe of that reaction.
So some reactions that are spontaneous take place in a matter of seconds or microseconds or less.
Other reactions, such as this, to have an appreciable amount of that reaction go forward, it actually takes years.
You'll notice that it's not just one day before that hydrogen peroxide expires.
It's takes a really long time to get a significant amount of this reaction to go.
So just make sure, we will get to talking about kinetics, which does tell us about the timeframe of chemical reactions.
But don't confuse thermochemistry or thermodynamics with kinetics.
It doesn't matter how huge our delta g is in terms of being if it's negative two or if it's negative two million, that still doesn't tell us how fast the reaction's going to go.
All right.
So let's think about one more example here in terms of thinking about entropy.
So another example we talked about was the melting of ice at room temperature.
I think all of us agree that this is spontaneous, but that the enthalpy change is positive, it's an endothermic reaction.
So we can think about calculating the entropy here, and I'm happy to see that I did, in fact, put the products h 2 o here as a liquid, subtracting the reactant, which is h 2 o as a solid.
If we go ahead and plug in those values here, what we end up with is a delta s of this reaction of 28 .
59 joules.
This makes sense because we're increasing the disorder of the system, we're seeing a positive delta s here.
And I just answered that question for you.
We're going why is delta s positive?
We're going from a solid to liquid, we're increasing disorder.
So, we can calculate delta g as well, and if we do this we plug in these numbers here.
What we end up with is that delta g equals negative 1 .
57 kilojoules per mole.
So it is spontaneous, delta g is just barely negative.
And what actually is the case, which we'll talk about more on Monday and thinking more and more about how temperature actually affects these reactions, but what we know about melting of ice is that it is, in fact, temperature dependent, right.
Ice doesn't melt at just any old temperature.
So that's where temperature actually comes into play here.
All right.
So we have a case -- spontaneous, even though delta h is positive.
All right, so let's think also about how we can go about calculating the free energy of formation.
So we know that we can use the equation that we saw in terms of the free energy the reaction, but what if we're actually thinking about the free energy of the formation of a particular molecule?
So that's actually completely analogous to thinking about the enthalpy of formation, or it's basically the standard Gibbs free energy of formation.
So this is talking about forming a mole of a compound from its most stable elements, from its elements in their most stable states at pressure of one bar and at room temperature.
So we know that we can tabulate it must like we tabulated delta h formation, but we can also tabulate it from this reaction here, which it looks like I already told you this reaction, which I did but I told you a more general form, which was for the free energy of a reaction.
This is the free energy of a reaction, which is the formation of a certain compound, and this is the free energy of formation equals the enthalpy of formation minus t delta s. So let's think about why it is important to be thinking about these changes of free energy in terms of forming a molecule, and it's important because the delta g information is actually a measure of how stable or unstable a molecule is relative to its element.
So, for example, if something is really stable relative to its elements, we can think about whether delta g of formation should be negative or positive.
So let's have you do one last clicker question here, and tell me what do you think, if delta g of formation, of actually forming a molecule is going to be negative, would you say that the compound is going to be stable or unstable relative to the elements that that compound is made up of?
All right, let's take 10 seconds on this.
OK, interesting.
So this is going to be another tie breaker for us.
So let's go back to the notes and think about this.
So it turns out that it's going to be stable, relative to its -- the compound is stable relative to its elements.
And the reason for this is if you think of the reaction of the elements forming the compound, if the delta g of that reaction is negative, that means it's going to spontaneously form the compound.
It's going to go -- release energy when it actually forms that compound.
So that's another way of telling us that the compound is more stable than its elements.
So conversely, if we have delta g of formation being greater than zero, so delta g of formation is just delta g of the reaction where the compound is formed, then what we find is that it's thermodynamically unstable relative to its elements, because instead of releasing energy in that reaction, you actually have to put energy into that reaction in order to make it happen.
So, delta g of formation can give us an indication of whether we have a stable or an unstable compound.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
OK, let's get started.
Why doesn't everyone go ahead and take 10 more seconds on this clicker question.
It should look familiar.
You were pretty split on this question on Friday, so we're hoping after learning a little bit more about delta g of formation, we have at least one direction that wins out here.
OK, great.
So now we're up to 85% of you, and hopefully in just a minute we'll be up to a 100%.
But if delta g of formation here is less than zero, that means we're talking about a spontaneous reaction when we form the compound.
So if it spontaneously forms the compound, that must mean that the compound is going to be stable relative to its elements.
So that's kind of the last thing we went over in terms of topics on Friday, and we'll pick up right there today.
All right.
Here we have our notes for today, so reminder exam 2 is on Wednesday -- I don't think I need to remind anyone of that.
But just please, don't come to this room, make sure that you go to Walker to take the exam.
And also if you have any questions or you feel like there's anything you don't understand, I have office hours today from 2 to 4 in my office.
Your TAs have also all moved their office hours so they fall before the exam.
So make sure you get any of your questions addressed before you're trying to sit there and figure it out in exam time.
All right, so today we're going to pick up where we left off on thermodynamics, so that was talking about free energy of formation.
After that we're going to talk a little bit more about the effect of temperature on spontaneity.
We touched upon this on Friday, but we're going to formalize exactly in which cases temperature can or can not affect whether a reaction is spontaneous or non-spontaneous.
Then we're going to look a little bit into thermodynamics and biological systems.
Two examples that I wanted to talk about were ATP coupled reactions -- those are very important in biology.
And also, thinking about the idea of hydrogen bonding, we're going to combine our thoughts on bond enthalpies, and also our ideas of bonding that we had from thinking about covalent and ionic bonds.
All right, so let's finish up first with free energy of formation.
So as 85% of you just told me, when we have a case where delta g of formation is less than zero, what we're talking about here is a compound that is thermodynamically stable relative to its elements.
So that means we know the inverse as well, which is when we're talking about a case where delta g is now greater than zero, the compound is going to be thermodynamically unstable relative to its elements.
So any time you're looking at delta g of formation, just remember to think about this is just the delta g of a reaction where you're forming a compound.
So it should make sense that if it's negative, it's going to be spontaneous and you're going to have a stable compound here.
So we could look at any number of examples, really we could pick any compound to look at, but up on the screen here I'm just putting the compound, the formation of benzene, we've looked at benzene a lot.
So benzene has a delta g of formation of 124 kilojoules per mole.
Is benzene stable or unstable compared to its elements?
I can't really tell the difference between stable and unstable when you say it, so everyone start at the same time, go
Unstable
OK, excellent.
Benzene is unstable compared to its elements.
So that means the reverse reaction here is going to be stable -- or the reverse reaction is going to be spontaneous where we actually have the decomposition of benzene.
So when we're seeing this, when we're seeing that the reverse reaction is spontaneous, a question that might immediately come to us is well, why did we just spend all the time talking about benzene because clearly benzene's just going to break down, right.
Why do we form benzene and not have it immediately decompose into its elements.
Thermodynamically that is what should happen and it is what does happen, but the reality is that this reaction, this decomposition of benzene is actually very, very slow.
It's so slow that, you know we use benzene all the time in organic reactions, we don't see it break down even when we heat it up, is because even though this is thermodynamically a non-spontaneous reaction to form benzene, it's very slow for the actual decomposition of benzene to occur.
So this is just another case, and I'll keep saying this, and when Professor Drennan starts talking about kinetics, she'll keep repeating this as well.
What we want to keep in mind is that delta g tells us whether a reaction will happen or whether it won't happen.
It tells us absolutely nothing about how long it takes for that reaction to happen.
It tells us nothing about the rate of the reaction.
We'll keep seeing examples of this, so hopefully no one will be confused by the time we do get to kinetics.
So if we're talking about calculating delta g for any reaction, now we actually have several ways to do it.
The first way is very analogous to thinking about delta h for the reaction.
We can just look up a table where we have delta g as a formation.
So we can take the sum of the delta g of formation of the products, and subtract from it the delta g of formation of the reactants.
We also have another way if maybe we don't have that information available to us.
We can also take a look at using this reaction or this equation right here, which is telling us that delta g of a reaction is equal to the change in enthalpy minus t delta s. So that tends to be very helpful, especially when we want to take into consideration temperature.
So let's take into consideration temperature.
We've done this a little bit so far, but let's really take a look at some reactions where temperature's going to make a big difference.
So the reaction we're going to look at here is the decomposition of sodium bicarbonate, or sodium bicarb here, and it decomposes it into sodium carbonate, plus c o 2, carbon dioxide, and water.
So we can think about calculating that delta g of this reaction.
I'll tell you that the change in enthalpy, this is actually an endothermic reaction.
It requires heat.
It's plus 135 .
6 kilojoules per mole.
So let's go to a clicker question and I want you to pick out from several choices which of the changes in entropy seem reasonable to you here.
So what would you predict the delta s for this reaction to be?
All right, let's do 10 more seconds on that.
OK, so we've got the majority, but not everyone.
So let's take a look at why this is the correct answer, that it should be plus 0.334 .
If we're going from 2 moles of solid, to 1 mole of solid, plus 2 moles of gas, are we increasing or decreasing the disorder?
[INAUDIBLE]
We're increasing the disorder.
If we're getting to a more disordered state, then we're going to have a positive change in entropy.
We're going to increase the disorder.
The only one with a positive delta s is this choice here.
So if we switch back to our notes, we can see that, in fact, so what we see is that it's 0.334 kilojoules per k per mole is our delta s, so we can go ahead and calculate our delta g for the reaction, so we're just plugging in our delta h. So our delta h is 135 .
6.
And we're talking about to start with let's talk about room temperature.
So 298 k times 0.334 .
So what we end up having for the delta g of our reaction is that its 36 .
1 kilojoules per mole.
So this is at room temperature.
So is our reaction spontaneous or non-spontaneous at room temperature?
Non-spontaneous
Non-spontaneous.
All right, but let's take a look at a different temperature.
For example, let's look at baking temperature.
So if we think about baking cookies, we maybe bake them at 350 degrees fahrenheit, so that would be 450 kelvin -- our ovens are usually set to fahrenheit and not kelvin.
So if we think about this reaction that this temperature, first of all let me point out why we would be talking about baking cookies for this particular reaction here.
Does anyone know what another name for sodium bicarb is?
[INAUDIBLE]
Yeah, it's just baking soda.
So this is the reaction that causes your cookies or your cakes to actually rise.
So we're producing gas here, and when we're producing that gas when this sodium bicarb decomposes, we're forming these pockets of gas in our baked goods.
So if we bake our cookies at room temperature, obviously, they don't bake and they also don't rise because the sodium bicarb, clearly it was non-spontaneous at room temperature, this reaction just doesn't happen.
But if we take a look at baking temperature now, we're going to plug in this new temperature, which is 450 k, and plug this into our delta g reaction.
What we find is now our delta g for the reaction is negative 14 .
7 kilojoules per mole.
So in this case, we are dealing with a spontaneous reaction, which is good, because this means that when we put our cookies in and we turn it to 350 fahrenheit or 450 k, the baking soda will decompose and we'll got our cookies to rise a little bit.
All right, so one thing that I want you to notice when we were talking about the case with the decomposition of sodium bicarb is that the delta h for that reaction and the delta s, they both had the same sign.
And something that you can keep in mind in general is any time that both delta h and delta s have the same sign, it's actually possible to switch from spontaneous to non-spontaneous or vice versa just by changing the temperature of the reaction.
And we can think about this graphically.
If we assume that delta h and delta s are independent of temperature, which is a good assumption to a first approximation, in this case we find that the delta g of the reaction is the linear function of the temperature.
So that means we can go ahead and graph what we saw for the case of baking soda.
So our first point here was that we saw at room temperature, about 298 k, the delta g was positive, it was 36 kilojoules per mole.
We also saw that once we heated it up to baking temperature, we actually had the delta g now at negative 15 kilojoules per mole.
So since this is linear, we can actually draw a straight line right through here, and we can think about the fact that we have this temperature here where anything below this temperature has a positive delta g, and anything above this temperature is going to have a negative delta g. And let's actually think about the fact that this is a line and see what our slope and our y-intercept is going to mean.
We can say that delta g is equal to negative delta s t plus delta h, if we want to put it in the formation of the equation for line.
So what this tells us is that our slope is going to be negative delta s here, and if we think about our y-intercept, that's going to be the change in enthalpy for the reaction.
So again, what I want to point out is that any time we're at a temperature that's lower than this change in temperature here, we're going to find that delta g is greater than zero, and we're going to be dealing with a non-spontaneous reaction.
However, if we move our temperature up and up and up so we go across the graph this way, eventually we'll hit a point where if we're above that temperature, we'll find that delta g is less than zero, and now we have a spontaneous reaction.
So we can actually think about what this temperature is.
We can call this T star.
This is the temperature at which our reaction switches, whether it's spontaneous or non-spontaneous.
And if you're thinking about trying to get a reaction to go, it's very important to be able to calculate what this temperature is.
A lot of times in organic chemistry laboratories, they need to heat up reactions -- part of why they do that is kinetics, but the other part is sometimes they need to make a reaction go from being non-spontaneous to spontaneous.
So let's think about how to calculate T star or this change in temperature.
So we're talking about this threshold temperature, so we're talking about where delta g is going to be equal to zero, because if we set delta g equal to zero, we know that anything on one side of that temperature is going to be spontaneous, and the other side is going to be non-spontaneous.
So if we do this, we can just rewrite our reaction, delta g equals delta h minus t delta s, and let's plug in our zero for delta g there, and now rearrange our reaction so that we're talking about this threshold temperature.
So that T star is just going to be equal to the change in enthalpy divided by the change in entropy.
So it's very easy for us to calculate, so let's go ahead and do this for the case with baking soda.
And for baking soda what we saw was that delta h was 136 kilojoules per mole, and the change in entropy was 0.334 kilojoules per k mole.
And that means if we do that simple division, then what we end up as our temperature star is 406 kelvin.
So basically, what this tells us is if we tried to bake our cookies below 406 kelvin they would not rise, if we bake them above 406 kelvin they will rise because this reaction is now spontaneous.
All right, so this was a case where we had seen that the delta h and the delta s both had a positive value.
But let's take a look at what happens when now delta h and delta s are both negative.
So we can think about this just by plotting it on our graph again -- we don't have actual values, but we can think about what the sign should be.
So if we talk about our zero point where temperature is absolute zero, if we have a positive, or excuse me, if we have a negative delta h now and we're at temperature equals zero, what is delta g going to be?
Negative or positive?
Yeah, it's going to be negative.
So if we have negative delta h and we're at temperature of zero, our s term completely falls out, so we're definitely going to start with a negative delta g. As we increase the temperature higher and higher, at some point that delta s term is going to become greater than our delta h term, so at some point we're going to flip to where the reaction has now a positive delta g. So you can draw this into your graphs in your notes where we're going to start, in this case if delta g is negative where delta g starts negative, and then when it hits that threshold temperature, anything above that temperature is going to be positive.
So if we had actual numbers we could plug those into our graph, but we should be able to understand what the general trend is going to be even in a hypothetical case where we're just dealing with is it positive or is it negative.
So what we see is that below this temperature, we have a spontaneous reaction, and above this temperature we have a non-spontaneous reaction.
That was flipped from the case where we had delta h and delta s both be positive.
All right, so let's actually summarize what all of our four different scenarios could be if we're dealing with delta h's and delta s's.
So to start with, why don't you tell me what you think if we have a reaction where we have a negative delta h and we have a positive delta s, do you think that this will be a reaction that's never spontaneous, always spontaneous, or will this be one of these cases where the spontaneity depends on temperature?
All right, let's take 10 more seconds on this.
OK, great.
So most of you got it.
So let's go back to the slides and think a little bit about why this is.
So, this case is always spontaneous, because in this case we have a negative delta h, which contributes to a negative delta g, and a positive delta s, but since the equation says minus t delta s, that means a positive delta s is also going to contribute to a negative delta g. So regardless of what the temperature is, we're going to have a spontaneous reaction.
So we say this is always spontaneous -- delta g is less than zero at all temperatures.
All right.
So let's look at the reverse case here where delta h is now greater than zero, and delta s is less than zero.
Is this always, never, or sometimes spontaneous?
[INAUDIBLE]
Nope.
So, this is going to be never spontaneous in this case.
So for the same reason, because delta h is greater than zero, that contributes to a positive delta g, and delta s being negative also contributes to a positive delta g. So we'll say again, that this delta g is greater than zero at all temperatures.
All right, so now we have a case where delta h is greater than zero and delta s is greater than zero as well.
Is this going to be always, never, or sometimes spontaneous?
Sometimes
Sometimes, good.
So, even before thinking, you can just remember if the signs are the same, we can have a dependence on temperature here.
So what we find is that this is sometimes spontaneous.
So we can think about when this happens.
Would it be when the actual temperature is greater or less than that threshold temperature?
[INAUDIBLE]
All right, so I heard a lot of people say greater than.
It's greater than, because when it's greater than that threshold temperature, that means that the delta s term is the one that's going to actually sort of take over, it's going to be at some point greater than the delta h term.
The delta h is going to be making the delta g positive.
That means it would make it non-spontaneous, but once delta s gets large enough, that's going to override, and now we're going to have a negative delta g only when the temperature is above that threshold temperature.
So similarly, when we see that delta h is negative and delta s is negative, this is also going to be sometimes spontaneous, and specifically, it will be spontaneous or delta g will be less than 0 when the temperature is less than that threshold temperature.
All right, so you should be able to look at any situation where you have or you figure out or you calculate the enthalpy and the enthropy change in the reaction, and you should right away, before doing any calculations, be able to know is this always spontaneous, is this never spontaneous, or is this something where I'm going to need to really take temperature into consideration.
And if I do, you can actually calculate what that temperature is going to be where you flip from spontaneous to non-spontaneous or vice versa.
All right, so shifting gears a little bit, let's take a look at a few examples where it's important to think about thermodynamics and biological systems.
So there's two things I particularly want to focus on.
So the first is the idea of ATP coupled reactions.
So this is really important, because when we're talking about biological reactions, a lot of the reactions that take place in our body are actually non-spontaneous, so energetically we figure that the delta g for this reaction is actually positive.
So I just show a schematic biological reaction here where we have some molecule that's broken up into two building blocks.
So let's say, for example, this has a delta g that's greater than zero.
We need to think about how it is that our body can actually make this reaction happen.
And the way that it does this is that it takes the hydrolysis of ATP, and the hydrolysis of ATP is what's called that it's coupled to this non-spontaneous reaction.
And when you have a spontaneous reaction coupled to a non-spontaneous reaction, you can potentially drive the reaction in the forward direction.
Remember the hydrolysis of ATP is spontaneous, so what we're doing in this case is we're taking a spontaneous reaction where we give off energy, and we're coupling it to a non-spontaneous reaction that requires energy.
So what we will hope is when we add up the total energy changes between these two reactions, if the sum of those two is a negative delta g total, now we can have this whole process move in the forward direction.
So let's take a look at thinking about what we can do in terms of a coupled reaction.
The first thing we need to do if we're thinking about using the hydrolysis of ATP is actually calculate what the delta g for the reaction of ATP hydrolysis is, and I picked 310 kelvin because that's the temperature of our bodies.
So again, this reaction is taking adenosine triphosphate that has three phosphate groups and hydrolyzing it, so reacting it with water to form ADP plus phosphate plus acid here.
So I'll tell you that the delta h of this reaction is negative 24 kilojoules per mole.
This is what we actually calculated in class on Friday.
And that the delta s is plus 22 joules per k, per mole, so we can go ahead and calculate what this delta g is here.
So the delta g of this overall reaction is just going to be, of course, delta h minus t delta s, and our delta h is minus 24 kilojoules per mole, and we'll subtract 310 kelvin times 0 .
022 kilojoules over k mole.
So if we do this, what we end up getting is a delta g of negative 31 kilojoules per mole of ATP.
All right, so again, this is good, we got a negative number, that's what we were expecting.
So that means that we will have energy available to use if we hydrolyize ATP and couple it to another reaction here.
So what we saw for our delta g, negative 31 kilojoules per mole.
All right, so let's talk about a reaction that is a ATP coupled reaction, and there's just tons of examples of this in our body.
One I picked because it has to do with glucose, which we have talked so much about, is the conversion of glucose to glucose 6 p. So 6 p -- p just stands for a phosphate group here, so what we're doing is taking, if we numbered the glucose carbons, this would be carbon number six, and we're putting a phosphate group on carbon number six.
So first of all, why would our body need to do this reaction?
It turns out that glucose, we know that we use glucose for energy.
Glucose is somewhat apolar, so it can actually move in and out of our cells, because our cell walls, those are very greasy.
So what we're actually doing here is we're putting a charged molecule onto the glucose.
There's two negative charges in a phosphate group, this is going to make sure that our glucose is now it's very polar molecule.
And now that we have a very polar molecule, we're not going to be able to have the glucose move in and out of the cell.
So we keep it in our cell by putting this phosphate group on it.
That's why we want to do this reaction, but let's think about the energy of actually doing it.
And it turns out that this requires energy.
The delta g for this reaction is 17 kilojoules per mole.
All right, so that could be a problem if our body had not come up with a way to solve it, which it has, and what it actually does is it couples this reaction here with the conversion of ATP into ADP.
And we just calculated that that has a delta g of negative 31 kilojoules per mole.
So this means we can think about delta g total, and by total we mean of this overall process here.
So that's just equal to 17, and we're adding it to the other delta g, which is negative 31.
So we get an overall delta g for this process of negative 14 kilojoules per mole.
All right, so that's one really simple example just to illustrate to you how these ATP coupled reactions work.
Some ATP coupled reactions require one molar equivalent of ATP, some require a lot more.
But essentially, the idea is the same.
We have this reaction that's energetically unfavorable that we couple with an energetically favorable reaction.
So, quite literally, this is what we mean when we talk about ATP as being the energy currency for the cell.
We spend some ATP in order to get these non-spontaneous reactions to go.
All right, so I also want to talk to a little bit about hydrogen bonding.
This also is a topic that deals with the thermodynamics, and it also is related to the ideas that we talked about before in terms of thinking about different types of bonds.
So if we talk about a hydrogen bond, a hydrogen bond, first I just want to be really clear, is not a covalent bond.
So this h x bond here is a covalent bond, this is not a hydrogen bond.
But a hydrogen bond can form when you have a partial positive on a hydrogen, and you have what's called a hydrogen bond donor atom, which has a partial negative on it, and also has a lone pair.
Typically this y atom will be on a separate molecule.
And what happens is you have that Coulombic attraction between the partial positive on the hydrogen and the partial negative on this hydrogen bond donor atom.
So what you form between them is a hydrogen bond, and you'll notice that I drew it as a dashed line.
H bonds or hydrogen bonds are drawn either as dashed lines or as dotted lines, and they're done so to differentiate them from covalent bonds, which we draw with this straight line here.
So let's think about what can form hydrogen bonds.
First of all, there's a requirement for what this h x bond can be.
It has to be a really electronegative atom in this x here, because we need to we need to form that partial positive on the hydrogen, which means that the atom that it's covalentally bonded to needs to be pulling away some of that electron density from the hydrogen, such that we have a delta positive on a hydrogen atom.
Similarly, the same atoms are what can be the y here, so it can either be a nitrogen or an oxygen or a fluorine.
So these are the only hydrogen bond donors that can form h bonds.
And we can think about the reason for this and the reason is pretty straightforward.
We need to have a small atom, but it also needs to be really electronegative.
This should make sense because we need to have this partial negative charge here to have that Coulomb attraction.
And the other requirement is that we definitely need to have that lone pair of electrons so it can actually interact in this hydrogen bond here.
So let's take a look at an example of a hydrogen bond, and that's between two water molecules.
So let's go ahead and re-draw our water molecules -- they're just kind of randomly oriented there.
But let's re-draw them as if they were going to have a hydrogen bond between them.
And one thing I want to point out about hydrogen bonds is that they're strongest when you actually have all three atoms in a straight line like this, because it keeps the dipoles the strongest, so the partial positive here and the partial negative here can interact.
So let's re-draw our water molecules like that.
So we have our first water molecule here, and let's say we're going to be talking about this hydrogen in terms of hydrogen bonding.
Obviously, we have four hydrogens up there that can take place or take part in h bonding, but we're just going to focus on this one here.
So that means we want our straight line to be something like this to our second oxygen.
So let's draw the hydrogens on this water molecule as well.
All right, so thinking about what the Lewis structure of water is, how many lone pairs do we have on this oxygen here?
[INAUDIBLE]
Two lone pairs, great.
So let's draw those in, and let's draw these lone pairs in as well.
All right, and we're going to be talking about these two atoms in terms of our hydrogen bond, so does this have a delta plus or a delta negative on it?
Delta plus.
So this is a polar bond here, this o h bond, so we have a delta plus on our hydrogen, and we have a delta minus on our oxygen.
So these also have delta plusses, of course, and this also has a delta minus.
So because these two can interact and they're lined up to do so, what we can do is we can draw a hydrogen bond right in between this hydrogen and this oxygen atom here.
So again, I pointed out that, in fact, hydrogen bonds are not as strong as covalent bonds.
Since they're not as strong, if you remember thinking about the relationship between bond strength and bond length, would you expect a hydrogen bond to be longer or shorter than a covalent bond?
Longer
Good, OK, it should be longer here.
So this is our longer bond because it's our weaker bond.
They're not held together as tightly.
All right, so this is an example of an intermolecular hydrogen bond, it's between two different molecules.
Hopefully, you can also see that if we're focusing on any single one water molecule, we could also form hydrogen bonds between other water molecules with either of these two hydrogens as well, and also with our other lone pair here.
So any water molecule can actually form four different hydrogen bonds.
All right, so let's talk a little bit about these hydrogen bonds in terms of their bond enthalpies here.
I said that they were weaker than covalent bonds, so let's look at a few comparisons here.
So we can look at an h o hydrogen bond versus an h o covalent bond.
And what we find is that the bond enthalpies in kilojoules per mole, it's 20 kilojoules per mole for a hydrogen bond versus 463 kilojoules per mole for a covalent bond.
So we're not talking about just a little bit weaker, we're talking about much, much weaker for the hydrogen bonds here.
Similarly, if we look at nitrogen, for nitrogen hydrogen bonds, if we have an o h bound to a nitrogen, that's 29 kilojoules per mole.
If it's an n h hydrogen bound to a nitrogen, that's 14 kilojoules per mole, and if we're talking about an h n covalent bond, now we're talking about 388 kilojoules per mole.
All right, so again what we see in these cases is that the hydrogen bonds are much, much weaker.
They tend to be as low as 5% of what the covalent bond is, so this is actually much weaker than any kinds of bonds we have within molecules, but it's also the strongest type of bond that we can have between two different molecules.
And one thing that you find as in the case of water, is when you have lots of hydrogen bonds between molecules it changes the property of, for example, water here.
It makes the boiling point much, much higher than you might expect if you looked at the other properties, because you have to actually break apart all of these individual hydrogen bonds, and even though it's not that much energy for one individual hydrogen bond, once you get a huge number of hydrogen bonds, you're talking about huge energies here.
So let's look at some examples of where we see hydrogen bonds in biological processes.
First thinking about proteins, they're absolutely hugely important in proteins.
And in proteins, we're talking about a molecule that's so large that you don't just see h bonds in between two different molecules, what you actually see is what's called intramolecular hydrogen bonding, so hydrogen bonding within the protein molecule itself.
And hydrogen bonding is incredibly important, it, in fact, is the shape of a protein, which I'm just showing an example here, histone deacetylase, and the histone deacetylase molecule actually has just countless hydrogen bonds in here, and the hydrogen bonds largely govern the shape of the molecule.
So, if you're thinking about any protein that has a shape, so for example, we see these ribbon structures here, they're called alpha helices, you can see some arrows, which are just like a sheet structure or beta sheets, that actual shape is stabilized and formed in the first place by just many, many hydrogen bonds within the protein being in a correct orientation that allows for this particular shape to be stabilized.
And we know that shape is so important when we're talking about proteins, and that's because the actual shape of the protein governs the interaction with other proteins or other small molecules.
The shape of the protein is very important in terms of positioning the specific atoms that are involved in the chemistry that the enzyme carries out being in the correct position.
So hydrogen bonds are very important in terms of proteins.
We can also think of them in terms of sugars or polysaccharides.
This is a nice dramatic example, thinking about the structure of a protein is a very microscopic way of thinking about hydrogen bonding.
If we want to take it to the macroscopic level, we can talk about h bonding in trees.
So if we're talking about trees, trees are made up -- a major part of a tree is the polysaccharide cellulose.
So that's shown right here.
Cellulose is a chain of glucose molecules linked together.
It can be as many as 1,000 or more glucose molecules linked together covalentally.
But what actually happens to those individual chains is that if you look at this picture here, all those dotted lines are actually hydrogen bonds.
So the individual molecules are in a much larger chain of hydrogen bonded molecules and they're incredibly rigid.
You have so many hydrogen bonds that now it takes a huge amount of energy to break all of these.
So that accounts for why wood is such a hard, solid material.
And also, I mean for any plant, and tree is the most dramatic, the structure of a tree, the macroscopic structure, can be explained by hydrogen bonding.
It's hydrogen bonds that keep all of those molecules together in the forest and in trees.
All right.
So, that's proteins, that thinking about sugars.
We can also talk about the importance of hydrogen bonding in DNA.
So if you think about the characteristic DNA double helix, hydrogen bonds, we have in a double helix we have two strands of DNA that you can see are intertwined here.
And those two strands are actually held together by hydrogen bonding.
So specifically, it's hydrogen bonding between what are called complimentary bases within the DNA.
So, for example, if we look at guanine and cytosine here, you can see that there's several places, when they're lined up like this, where we can have hydrogen bonds.
So in terms of thinking about how these two molecules are lined up, how many h bonds would you expect between these two bases?
Yeah, so it's lined up that there are three that can form here.
So we have an o h hydrogen bond, an h n, and then an h o hydrogen bond that can form.
So the other side of complimentary based pairs are AT base pairs, and if you look at the way these are lined up here, how many hydrogen bonds can form between A and T base pairs?
Yup, so it it's two that we can form here.
We have one between the h o, one between the n h, and you see that we've lost this nitrogen, this n h group here, and first of all this would be too far apart anyway for hydrogen bond, but we can't have a hydrogen bond form when we have a carbon h. Remember, it has to either be a nitrogen or an oxygen or a fluorine because those are the atoms that are going to pull away enough electron density from the hydrogen to give it a partial positive charge that it needs.
So in terms of thinking about DNA, this is a really neat case to consider the actual thermodynamics or the bond enthalpies of the hydrogen bonds, because, of course, does not always stay in it's double helix -- when we're talking about transcription, we actually need to unzip or separate this helix into its two strands, so each individual strand can be copied.
So it's really important that hydrogen bonds are strong enough to hold the DNA double strand together, but that they're not so strong that when you actually pull the hydrogen bonds apart to open up the double strand, that you actually, you don't want to break all of a covalent bonds in DNA as well.
All right, so that's all we're going to say in terms of thinking about thermodynamics and biological systems.
Hold on a sec, we have plenty of time left.
I want to go over a few clicker questions.
I didn't want to give you too long set of notes, because we're switching over to Professor Drennan's going to start lecturing on Friday, so I thought maybe I shouldn't have to have her finish up with my notes.
So, Professor Drennan will continue talking about thermodynamics and thinking about equilibrium, and then she'll transition that into talking about kinetics.
So that's actually what's going to start happening after the exam on Wednesday.
But let's take a look, since we do you have a little bit of time left, at a few clicker questions just to make sure everyone has caught what we've gone over so far.
And a few reviews for the exam, and I will have this clicker question or one of the ones following, be a clicker question quiz, so make sure you are still answering these questions here.
So the first one covers what we went over today, so I want you to tell me and try to not look back in your notes for this, for a reaction where you have a positive enthalpy and a negative entropy change, would you expect this reaction to be never, always, or sometimes spontaneous?
So let's take 10 seconds on that, that should be pretty fast.
OK, excellent.
90% is very good.
So this should never be spontaneous, because we both have the entropy and the enthalpy term contributing to a positive delta g. All right, let's shift gears to a couple of questions that are review for the exam.
So hopefully, these will all be very straightforward for you now.
So I want you to think about bond lengths here, and tell me which molecule contains the shorter nitrogen oxygen bond.
So, we're comparing n o minus 1, and n o 2 minus 1.
So if you wanted to get started on this problem, what's the first thing you should do?
Yeah, draw some Lewis structures.
So that might be a good place to start in thinking about this.
So I'll do that up here as well, but don't look if you want to try it on your own.
Remember, the quiz points come for answering, not for getting it right.
So try doing it on your own here.
All right, let's go ahead and take 10 more seconds on this one.
OK, hold on.
Don't show the correct answer.
You know we're going to re-poll and give you a little bit more time to see if we come to a consensus here.
So let's take another 30 seconds or so on this while I finish drawing these up here.
OK, good.
So we've got 81% have it correct that in the n o minus 1 bond here, this is going to be a double bond
What if [INAUDIBLE]
No.
It would be a double bond here.
So, if you follow just the Lewis structure rules, and you go ahead, you'll find that we end up having four bonding electrons available.
So you can just go ahead and plug those in, and you end up with this many left.
If we had a triple bond, then we would end up having more bonding -- or we would end up using more electrons than we have available for bonding here.
All right, so we have a double bond in the case of n o.
What is this bond that we have here?
Yup, so it's actually a 1 .
5 bond.
What is this bond that we have here?
[INAUDIBLE]
All right, so we have two 1 .
5 bonds in this case, how come these are 1.
5 and not, for example, a double bond?
[INAUDIBLE]
Great, so that's the key.
Even though we have double bonds in each of these structures, in this case here, we have resonance, so it's turns out to be, in fact, two 1 .
5 bonds.
So since this is the double bond, it's going to be the stronger bond.
Since it's the stronger bond, it's also going to be the shorter bond.
So make sure you can make those relationships between bond strength and bond length and thinking about if you have resonance or you don't have resonance in a particular situation.
All right, let's try one more clicker question here.
We'll have this be the last one for you.
So this is one that hopefully you'll all get right, because we've gone over this again and again both in class and recitation.
So let's talk about the hybridization of the specific carbon and oxygen atoms in ATP, so I want you to go ahead and tell me what these hyrbridizations are for these two atoms.
OK, let's take 10 more seconds here, get those final answers in.
OK, good, so 83% of you got this.
Let's take a look at why.
So carbon a is bonded to three things, but it is bonded to no lone pairs, so we need to have it be three hybrid orbitals, so it's s p 2.
And oxygen is bonded to two atoms plus two lone paris.
So we need four hybrid orbitals or s p 3.
All right, so this was your quiz question, so as long as you answered it, you got your quiz points for today.
And you can get going a little bit early today and finish studying for this exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're pretty much ready to get started.
Let's settle down and take a look at the clicker question.
So take 10 more seconds.
Very good.
Most people had this right, and the trick here is just to think about the sign of delta g, whether it's negative and positive, and think about the equation and the influence of temperature on that equation.
So whether it's going to make delta s a bigger factor or a smaller factor and how that will play out.
So you can look at the equation and figure out what the signs have to be and then the influence of temperature.
All right, so let's move to the first slide that I have today, which is welcome to the middle of the semester.
So I'm Kathy Drennan and this is lecture 19 of 36, which means that you are halfway through the course.
And so what you heard on Monday before the exam was about thermodynamics, and if you don't have all your delta g's and delta h's and if you don't have entropy full in mind, that's OK, we are not really leaving thermodynamics, we're going into chemical equilibrium, which is all about thermodynamics.
So you're going to hear a lot more about delta g, delta h, temperature, and about our friend, entropy as we go along in the second half.
So it's the middle of the semester.
That means that you've had two of the sort of four hour exams -- remember the fourth hour exam is actually combined with the final exam.
So the final exam is 200 points of cumulative material, and 100 points of new material, so you've done two now of those four sort of hour exams.
Just to remind you of some of the topics you have seen and the topics coming ahead, this is on the syllabus, and we're actually where we're supposed to be on the syllabus, so if you want to take a look at that ever to see where we're going, we're actually on track.
So the first half of the course are a lot of basic principles, and we're moving today into chemical equilibrium, so a lot more delta g's, delta h's coming on.
And then we're going to move into acid base, which is acid base equilibrium.
So we're not going to be leaving equilibrium or thermodynamics.
Then we're going to go into oxidation reduction, which is also about equilibrium, and then transition metals, and end with kenetics.
And so, these altogether represent the fundamentals needed for the study of biochemistry, organic chemistry, any kind of chemistry, biology, the life sciences, many things -- so you're getting all the fundamental principles of chemistry in this class.
So along those lines, I just thought I would share something that happened to me on Wednesday when I was riding the T, the Silver Line, in particular.
So I don't know if any of you have had this kind of experience yet where you're on public transportation and someone next to you says, "So, are you a student?"
Maybe you have a notebook out or a book out or something.
And you say, "Yes," and they said, "Oh, what are you studying?"
And then you say "Well, you know, chemistry, math, physics."
And they're like "Oh."
And then they go busy themselves doing something else.
Or they tell you "I didn't really like those subjects in school."
And then they stop talking to you.
So how many people have had that experience so far, that you scare the person next to you by what you study?
OK, a few people.
If I ask that question four years from now, I think it'll be like half the class, eight years from now it'll be everybody.
So this is kind of common.
But what happened to me on the Silver Line was actually pretty exciting, because I've had that other experience many times.
So, I was wearing my keys around my neck, because women's professional clothes, no pockets, with a little MIT cord, and the person next to be on the Silver Line said "So, do you go to the Georgia Tech of the east?"
And I said "Well, I am a professor at MIT."
And he said "Well, I'm a mechanical engineer and I went to Georgia Tech."
And so he said, "So, what do you teach?"
I said, "Chemistry."
And he said, "Chemistry -- I really wish I had paid more attention in chemistry."
I said, "Well, what do you do now?"
And he said "Well, I work for the army, and I'm part of a team that has mechanical engineers, electrical engineers, and chemists, and we're trying to figure out ways to detect explosives."
So I said, "Oh, well have you heard of the head of the Chemistry Department at MIT, Tim Swager who works in the area, and he said, "Yes," he knew the name.
And so he said that one of his porblems on the team, and in talking to the chemist, his chemistry language is not really good enough, and so he was really struggling, and that the best results of this team effort would be if everyone could really talk to each other.
So he wished that he had paid more attention in his chemistry class.
And so I, of course, my connection with chemistry was by wanting to understand biology, but for everyone it might be a little bit different.
So, as I mentioned in the beginning of the class, one challenge that you have this semester is figure out what your connection to chemistry is.
What are you going to use chemistry for?
How does this fit into what you want to do?
And maybe for some of you, you're not going to know that yet.
Hopefully it won't take till you have a job doing something to realize that you're lacking something in your education.
And so, probably if you stay in science and engineering, need chemistry, good to learn it all now.
And from exam 2, we see that you are learning it now and that's really great.
So, some day maybe someone in this room will be involved in national security, and if you're not a chemist, you'll be able to talk to the chemists and make really good progress toward that work.
So, chemistry -- it's important in medicine, national security, the economy, energy initiatives, a lot of the big things going on now.
It's a fundamental, and you will get in this course, the fundamentals.
So if you have any good tea or bus or airplane conversations, let me know -- I'll catalog those for future reference.
It's always a touch of how what we do here connects with the real world.
All right, so we're not going far from thermodynamics, we're going into chemical equilibrium.
And we're going to be talking a lot about delta g. So if delta g is not your friend yet, don't worry, you can still bond with delta g. All right, so chemical reactions, chemical reactions can go into a state of equilibrium.
And it's a dynamic equilibrium, the reaction is still happening, but if a reaction is an equilibrium, the rate of the forward reaction will equal the rate of the reverse reaction, so there'll be no net change in composition.
So let's take a look at an example.
So let's look at a reaction in which we have nitrogen gas, and we have hydrogen gas, and they are reacting to form ammonia.
Suppose we just start with nitrogen gas and hydrogen gas and we don't have any ammonia left.
So if we consider concentrations versus time.
So say we start with, we have some nitrogen gas, some concentration of nitrogen gas.
As it reacts with hydrogen, the concentration will decrease and then level off.
We'll start with some amount of hydrogen gas, and its concentration will also decrease and level off.
And in the beginning we won't have any of the product, any of the ammonia, so its concentration will increase and then level off.
So when these reactions level off, you're reaching equilibrium.
The reaction is still happening, but the rate of the forward reaction is equal to the rate of the reverse reaction, so there's no net change.
So the concentrations are staying the same, but there's still -- the reaction is still going forward.
So let's think about the case when we have pure reactants when we haven't formed enough products yet to reach equilibrium.
So if we have pure reactants, the reaction is going to be spontaneous in the forward direction.
So we'll have a reaction that is spontaneous in the forward direction.
And what will that mean about our friend delta g?
Is it going to be greater or less than zero?
So it'll be less than zero -- so delta g or the forward reaction will be less than zero.
So when delta g is negative, the reaction is spontaneous in the forward direction.
So what about when you have pure products, then the reaction should be spontaneous in the reverse direction.
So you have spontaneous in the reverse direction, and what does that mean about the sign of delta g?
Greater or less than zero?
Greater, so it'll be positive.
So, we can think about that in terms of a plot.
We can think about free energy versus the progress of the reaction.
So, the progress of the reaction is going this direction as you go along.
So in the beginning if you have pure reactants, delta g is going to be less than zero.
And you're going to proceed along in the forward direction spontaneously.
If, on the other hand you have pure products, delta g will be positive, so you'll be spontaneous in the reverse direction, and the reaction will go in the reverse direction until what happens?
Till you reach equilibrium.
And what would delta g be at equilibrium?
Zero.
So you see, there is a great relationship between delta g and equilibrium, so we have not left delta g behind.
So, delta g is going to change as the components of the reaction change.
As you have more products or more reactants, you're going to have a different delta g. So let's look at some equations.
So delta g is the change in free energy, the difference in free energy, at some point in the reaction at any time with amount of composition.
We also have delta g nought, which talks about delta g under particular conditions, so it's sort of the standard free energy.
We have a term called q, which is the reaction quotient, which tells you about products and reactants.
We have our friend r, which is the gas constant, and this depends on temperature.
So temperature is a term involved here.
So the delta g at any point in the reaction is going to depend on the delta g nought for that reaction, and the reaction quotient, products and reactants, and we'll define this in a minute, and then the temperature as well.
So we need to know more about what q is, what this reaction quotient is.
So this slide looks a little bit scary, but it's going to be fine, because a lot of the terms are going to cancel out.
So we're going to talk about q, this reaction quotient, and we're going to have different types of problems -- some where we're talking about gases, and others we're talking about solutions.
So you can see two different kinds of q's, one that depends on partial pressure of the gas, and another that depends on concentration.
So here is our equation again that we just saw -- delta g equals delta g nought plus r t natural log of q, and here we've expanded the term for q.
So p to the sub x is the partial pressure of a particular gas, and so in this equation, we have a plus b going to c plus d, the top of the line or the products.
So we have the partial pressure of gas c, and this is over a reference, partial pressure, raised to the power c, the coefficient, and then also we have d, the partial pressure of gas d, over a reference raised to the small letter d. On the bottom of the terms for the reactants, partial pressure of gas a over a reference raised to the coefficient a, partial pressure of gas b over the reference raised to the coefficient b.
Now what's great is that the partial pressure is one bar, and so that basically cancels out as far as we're concerned.
And so the term for q is much simplified.
You won't see problems that have the reference in them, so you can just think about q in terms of products over reactants.
And you just have to remember that the coefficients in the reactions do matter.
If you're talking about solutions, the only difference is that we'll be talking about molar, so we have one molar, so the reference term here also cancels out.
When you see something in brackets, like c in brackets, here that telling you it's a concentration term.
So here, q is the concentration of c raised to its coefficiency.
The concentration of d raised to its coefficient d over reactants.
Concentration of a to a, concentration of b raised to b.
So the thing in x indicates it's a concentration term.
So q is just products over reactants, considering the stoichiometry of the particular reaction.
So what about the equilibrium constant k?
So at equilibrium you told me that delta g equals zero, and at equilibrium q, the reaction quotient equals k, the equilibrium constant.
So we can consider that in terms of this expression, if we're talking about this expression at equilibrium, delta g is going to equal zero.
And so, we just set this whole term equal to zero.
We can rearrange the equation bringing delta g nought to the other side.
And so we have delta g at nought equals minus r t natural log of K, because at equilibrium, k is equal to q.
So now we have another term to solve for delta g, and an equation that relates delta g nought to be equilibrium constant k. So k, the equilibrium's constant has the same form as q, but we're only talking about the concentrations or the partial pressures of things at equilibrium.
So it's the same, it's product over reactants, same expressions as q, but in the corner you say at equilibrium.
So you're only talking about the concentrations of things at equilibrium if you're solving for k. If you're solving for q, it's the concentration or the partial pressure at any time in that particular reaction.
Important thing, products over reactants.
All right, so, we can rewrite the equation one more way.
So I just told you that delta g nought is equal to minus r t natural log of K. So we can substitute into the expression minus r t natural log of k, and now we can rearrange this equation.
And so, if we rearrange it, we have delta g, the delta g at any particular point in the reaction equals r t natural log of q over k. And this equation is helpful for people if they're thinking about what is the equilibrium constant, what concentrations do I have now -- we're not at equilibrium, what concentrations do I have now, what is q now in the reaction, how does that compare to k?
And when you know what those values are, you'll know something about the direction of the reaction because you'll know if delta g is positive or negative.
It'll to be spontaneous in the forward direction or in the reverse direction.
So this is a handy equation for thinking about the relationship q k delta g. So let's think about that relationship for a minute.
If q is less than k, what is the sign of delta g?
So it would be negative, which means the forward direction of the reaction will occur.
So if you think about it, you can think about in terms of products and reactants.
So at equilibrium then, if q is less than k at equilibrium, there are more products than there are right now, so you need to make more products.
So you would have delta g would be negative, you'll see that mathematically, and you can think about it in terms of whether you're going to make more products or less products.
If q is greater than k, what is delta g?
So it would be positive and the reverse direction would occur.
So you think that at q, if it's larger than k, it has more products in its terms, and at equilibrium there are less products, so you need to go in a direction that will get rid of some of those products so you'll reach equilibrium again.
So this equation is very helpful in thinking about the direction of the reaction -- which direction will it be spontaneous.
So let's look at an example.
K is given in this example, and then we have a bunch of partial pressures.
And we're asked which direction the reaction will go.
So what do I need to do to answer this question, what do I need to calculate?
So if I'm given k, and a bunch of partial pressures, what do I first need to calculate?
Q, right.
So let's calculate q.
So q, we're going to talk about products over reactants, so we're going to talk about be partial pressure of the ammonia, and there are two of them being formed.
Our reactants, we want to talk about the partial pressure of the nitrogen, and the partial pressure of the hydrogen gas, and again include the stoichiometry, so we have three there.
So now I can plug in my values, so I'm told I have 1 .
1 bar, and down here I have 5 .
5 and 2 .
2 to the 3, again, considering the stoichiometry.
And if you do the math, you get 2 .
1 times 10 to the minus 2.
That's your q value.
Now given your k value, which the problem states, and this q value, let's do a clicker question, tell me which direction the reaction will go.
All right, 10 seconds.
So, 77%, pretty good.
So, q is greater than k here.
And if q is greater than k, what will be true about delta g?
I have another clicker question for that -- yell out the answer.
What is it?
Positive, right.
So you're going to shift towards reactants, so you're going to go in the reverse direction.
So you think about whether there are more products or less products at equilibrium, and so there are more products, so we have to think about which direction it'll go, and here, ammonium will dissociate until equilibrium is reached again.
OK.
So let's think more about what k is going to tell us.
So k tells us about the mixture of products and reactants at equilibrium, whether we can expect low or high concentration of reactants at equilibrium.
So let's look at another example.
So when you have k that's greater than one, so more products than reactants at equilibrium, you can think about this in terms of higher products at equilibrium.
When you have k less than one, we're going to have lower products.
So again, think about k in terms of products over reactants at equilibrium.
So if k is greater than one, there are more products than reactants.
If it's less than one, there will be less reactants than products.
So let's look at an example of that.
Let's look at when k is greater than one.
And I have the equation up there and I'll write it here as well.
So we have 2 n o 2, and two double arrows, and n 2 o 4.
So we have a k value here of 6 .
84, so that's greater than one value.
So let's think about this reaction.
So over here instead of concentration, we're going to talk about partial pressure because we're talking about gas, and we have time.
So initially we have a reactant.
And the reactant starts at some concentration and decreases and then reaches a straight line, so reaches equilibrium.
So we have our reactant here.
Originally we have no product, and so product is going to go up and be formed and then it's going to level off as you reach equilibrium.
So initially, what is true about q and k?
So with no products, what is true about q and k?
Q is less than k. And so what's true about delta g?
Less than zero, it'll be negative, so it'll be spontaneous in the forward direction.
So you're going to be spontaneous in the forward direction and you're going to make your product.
So now let's calculate what the concentrations are going to be at equilibrium.
So initially, so you have your initial pressure for the reaction, 2 n o 2 going to n 2 o 2, and our initial concentrations are given as one bar, and we have no product.
Now we talk about the change as we go toward equilibrium, how much does the reactant change?
What do I write here?
What change?
Minus x minus something x?
Minus 2 x.
So again, we're considering the stoichiometry, and what's over here?
So just plus x, and then at equilibrium we now have 1 minus 2 x and x.
So we're talking about equilibrium concentrations, so we're talking about k, so k equals 6 .
84, which is going to be equal to the partial pressure of the product over the partial pressure of the reactant squared.
So it's going to be equal to x over 1 minus 2 x squared.
So x, if you calculate it out, should equal point 381 bar.
And then if we do 1 minus 2 times 0.381 bar, we get 0 .
238 bar.
So if we go back over here, our products at equilibrium -- oh, I guess I should write what -- so x is our product and this is our reactant.
So the product we have 0 .
381, and our reactant at equilibrium is going to be 0.238 bar.
So we have more product than reactant at equilibrium, which is consistent with the value of k being greater than one.
So, when you know something about the equilibrium constant, you know something about the reaction and whether you would expect more products or reactants at equilibrium.
So again, you can think about this in terms of q and k. All right, so now we are going to go toward our next clicker question.
So if we can rewrite the expression for delta g equals minus r t natural log of k, and express it in terms of k, and now we can think about then what is that relationship.
If you have a large value for k, what do you expect to be true about delta g nought?
All right, let's give that 10 seconds.
OK, why don't you discuss for a minute with your friends whether you agree with that 78% or not.
All right, now we're going to re-poll, so click in again.
Now give the right answer.
Interesting.
It actually usually goes the other direction, that after there's a discussion, more people come to the same conclusion.
So I guess it's a matter of if the professor asks that, you assume that that must have been the wrong answer.
Was that people's logic?
So what are the points of doing this, and we're going to actually do this kind of thing a few more times in class, is that collectively, and I was actually just at an education meeting about science down at the Howard Hughes Medical Institute, that statistics show that if you have a group where everyone in the group has the wrong answer and they're allowed to discuss it, there's a good chance they'll come up with the right answer.
So that it's not just about one person in the group having the right answer, convincing everyone else that they're right, that the act of discussing often leads to new answers.
So now that you know that it's not a trick on my part to tell you you got it wrong, we'll see next time whether this, in fact, holds, that the act of discussing helps give the right answer.
Anyway, so if k is large, it is true that delta g nought would tend to be negative and more -- it would be negative and more on the large side.
So one can think about if there's more products over reactants, that's going to indicate something about the delta g nought for the reaction, and the k -- the k, if it's greater than 1, if it's a big number, then there are more products than reactants, and delta g nought would be negative and would tend to be a large number.
All right, so this is more actually kind of simple bookkeeping involved that if you know steps in the reactions and you know equilibrium constants, you can calculate an overall equilibrium constant for that reaction.
So you can write a reaction as the sum of different components.
And so up here, we are going to try to add these first two equations to get this net equation.
So we have 2 gas p 3 c l 2 going to 2 p l c 3, and that's equilibrium 1.
And so then in the next reaction, some of that is being consumed reacting with another c l 2, giving you p c l 5, and the net reaction we're interested in has 2 p 5 c l 2's going to 2 p c l 5.
All right, what do I have to do before I can add these together effectively and have things cancel out?
I need to multiply what?
Second equation by 2, yeah.
So to get that to work I would need to have 2's there, then this is going to cancel out and these will add up, so we have 5 and then we have two of the main products.
So if I do that, and I have the equilibrium constant for one and for two, how am I going to get equilibrium constant for three?
I'm going to multiply k 1 by k 2 and k 2.
So I'm going to have to multiply k 2 in there twice, because there's two of those -- we'd multiply that up.
So if you have different parts of reactions and you can sum them together, then you can multiply out the individual k's to get the new value of k. So that's something that's just useful that you'll run into in doing these types of problems.
All right.
Now we're going to think about how equilibriums respond to stress.
And I always feel like I need to pause.
MIT students, you guys are some of the smartest, most talented scientists in the world.
I don't know if you fully appreciate how smart you all are.
But this concept is tough for MIT students.
So Le Chatelier's principle says that a system in equilibrium that is subjected to with stress tends to react in such a way to minimize that stress.
I have advisees coming to my office saying, "I'm double-majoring in this and that and I'm taking five classes and I have UROP and I have this lab exercise -- I don't know what's going on, so I've been thinking a lot about it and I think I should add a third major."
Le Chatelier would be very unhappy with that.
Le Chatelier would say that the appropriate response is to drop one of the majors, to minimize the stress.
So in doing these problems, what I want to encourage you to do is think the opposite of what you would do.
Minimize the stress.
If you think about that you'll be all set.
All right.
So Le Chatelier's principle actually is very useful.
If you think about minimizing the stress, you'll be able to predict the direction that the reaction will shift.
So the reaction's going to shift in a way to minimize the stress, so you can predict it.
If you're thinking along these lines, you say, oh, that system was stressed, and then you can think about how that reaction or that system is going to respond.
So let's give some examples.
So, we have a system in equilibrium -- we started out, we had our nitrogen and our hydrogen and we had no product.
We reacted the hydrogen and the nitrogen, their forming products, and eventually they reach equilibrium, their delta g equals zero there's still the reaction going on, but there's no net change.
All right, now the system is going to be stressed.
So we're going to add more of a reactant.
How will the system react to minimize that stress?
What is it going to do?
It now has too much of one of the reactants.
You'll form more products, and that's going to use up some of your other reactant, which will go down, and you're going to form more product until equilibrium is reached again.
And one thing that I'll mention that the ratio has to be the same.
The equilibrium constant is a constant given the same temperature, but you're not necessarily always going to have the same concentrations, but you should have the same ratios, the same value of k. All right, so what about if you add more product, what's going to happen?
What direction will the reaction shift?
Right, it's going to shift toward the reactants, so you're going to make more of each of the two reactants until again, you find your equilibrium again.
And then the reaction's still going but there's no net change anymore.
All right, so let's think about this -- let's think about this in terms of the math as well.
If you want to stick with the math, that's OK, you can think about calculating delta g's here.
So if you're a system in equilibrium and you add more hydrogen, the system will respond to minimize that increase.
So it's going to make more product that will minimize the increase, it shifts to the right.
And this can be explained in terms of q and k. So, you can think about, again, do you have more products now or did you have more products at equilibrium, and if you're adding more reactants, momentarily q will become less than k. And so, you would get a negative delta g, which would make it spontaneous in the forward direction.
So some people like to think about this in terms of delta g equals r t natural log of q over k. So you think about if q is less than k, what sign you have for delta g, and that's tells you whether the reaction is spontaneous in the forward or the reverse direction.
And let me just give you one hint for taking exams in this unit.
That use of the arrow is really good, or saying "toward product" or "reactant."
I can't tell you how many people write -- know what the answer is and write left when mean right, or write right when they mean left.
But if you draw an arrow you never really get it wrong, and when you say products or reactants it's a lot harder to make that mistake.
So if you're not good with right or left, which let me tell you a large fraction of people are not, hedge your bets, you can write everything, more products, arrows, and then write if you want, then you're pretty sure that you have it all in there.
OK.
So again, we can explain this in terms of q or k. All right.
So let's just quickly talk about adding more products.
We did this already.
If more products are added.
then q is going to be greater than k momentarily, and you would shift toward reactants, shift toward the left, and again, we saw that down there.
So tell me a final clicker question and then we're done.
What happens if you remove products and why?
OK, 10 seconds.
See if we can get in the 90's.
No, no 90's today, we'll have to do it next time, but we got 73 right.
So we're going to make more products as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, it's time to get started.
Pay attention to the clicker questions.
All right, I'll give you 10 more seconds.
Not bad, we seem to be in the 70's.
All right.
So if you didn't have time to click in, let's consider what's going on here.
What's true about the relationship of q and k here?
So is q less than or greater than k?
Less than.
And so you have to think about what that means in terms of where we are now in the reaction and where the reaction is at equilibrium, and I like to think about it in terms of products -- are there more products now, or are there more products at equilibrium.
So if q is less than k, then there are more products at equilibrium, and so the reaction will shift toward products.
So it's going to move in the direction to reach equilibrium, since k is a bigger number than q, and both of these terms are products over reactants, there'd mean there's more product at equilibrium then there are now, and so the reaction is going to shift toward products.
All right, so we're at 73%.
Let's see if by the end of today, we can get up to 90 on these kinds of questions.
All right, so we're going to continue to consider external effects on k in today's class.
I also wanted to mention that on each of the handouts for the lectures, I have the corresponding reading material listed on the handout, and I've listed it as section numbers in the chapters rather than page numbers, so that's a little bit of a difference.
And the reason why I've done that is that we've had many different versions of the book in this class, and they don't seem to change the section titles or the section numbers, but they do change the page numbers associated with them.
So if I give you section numbers then you should be able to use of whichever version of the book is available to you.
So that's how I have it listed now, and the reading assignment is also listed on each problem-set as well.
So when you're going over the handouts later studying for the exam, you can see where in the book this particular lecture, the material is, so you can go and do that reading.
All right, so we talked at the end of the class last time about the Le Chatelier's principle, and this is a principle that you can use to predict the direction of change of a reaction -- whether it'll shift to the right or shift to the left.
And simply stated, that systems tend to respond in such a way to minimize the stress.
So if something is added to the reaction, the reaction will shift in a way to minimize that stress caused by the thing that's added to the system.
So, we talked about adding reagents and removing products last time, and now we're going to go on and talk about what happens when you change the volume.
So if you have a system at equilibrium and you change the volume of it, and here in particular we're talking about a gaseous system, what's going to happen?
So what do we know about volumes when it comes to gases?
We say that a decrease in volume of the gaseous system causes an increase in total pressure.
What equation leaps to your mind when I say those words?
Exactly, p v equals n r t. Yes.
So there's a relationship between pressure and volume -- n is the number of moles, r is our gas constant, and t is our temperature.
This is one of the things that usually sticks from high school, and if you haven't seen it before, it's pretty easy to get up to speed with this equation.
So Le Chatelier's principle then predicts that the system will respond if possible in a way to reduce the total pressure.
So if you decrease the volume and you cause an increase in total pressure, Le Chatelier says wait a minute, the system wants to respond to minimize that stress, so that it's going to respond in such a way to reduce that total pressure.
So let's look at some examples of this.
So we have an example of a reaction where we have two moles of p 2 gas going to one mole going to p 4 gas.
And I have a little cartoon here to show this.
So we have our p 4 gas up here, and our p 2 gas.
All right, so what's going to happen if we're going to change the volume of this system?
So if we're going to decrease the volume of the system, then the reaction should shift toward products.
So let's think about that.
So here in the middle we have our sort of system, and then on one side what happens when you decrease the volume and on the other side we're expanding the volume.
So first, let's consider our system when we're going to compress that volume, decrease that volume.
So what would be true is that the system's going to respond in such a way to minimize that stress that has been put forward by decreasing the volume, and you're going to try to compensate for that, and in this case you can compensate by a shift toward products.
Why is that?
Well, it all has to do with the stoichiometry of this equation up here.
So for every two moles of reactants, you get one mole of products.
So if you shift from the two to the one, that will cause a decrease in pressure, and so that will help compensate for the increase in pressure caused by the switch in volume.
So you're going to respond in such a way to compensate for that stress.
So a shift to the reaction to the right decreases the total pressure.
So now let's think more about why this is true.
Let's think about it from a math perspective and also just this qualitative perspective.
So let's consider it in terms of q and k. So suppose our volume is decreased by a factor of 2, let's just make it easy, and we have constant temperature here.
So this change will increase the partial pressure of both gases, the reacting gas and the product gas, and it's going to increase them both by the same amount, by 2 initially.
So let's look at our equation for q, the reaction quotient.
We have partial pressure of products over the partial pressures of the reactants, a raise to the coefficient to the equation.
So initially you're going to increase the pressure, partial pressure of both gases by 2 -- then your product by 2, the reactant by 2 -- but here it's 2 squared.
So overall, q becomes 1/2.
So q decreases by a factor of 2, and so q becomes less then k. And so, in the clicker question, we talked about what happens when q becomes less than k, and what happens there is you're going to shift toward products until q equals k again.
So you can think about this in terms of q and k, and you can also just think about it how many moles of gas are on one side, how many mold of gas on the other side, and how are you going to correspond, how are you going to decrease or minimize the stress on the system.
All right, so let's think about what happens if we increase the volume, so here is our system and now we're going to expand it.
If we're going to increase the volume, what's going to happen to the total pressure?
I'm hearing it, but I want everyone to answer.
Thank you.
All right, so it will decrease the pressure.
And what that's going to do is shift toward reactants, because we want to compensate for that decrease in pressure, and to do that, we can increase the pressure by switching or shifting the reaction toward the reactants.
So again, we're going from one mole of product on one side to two of the reactants.
So here, we see a shift toward the reactants.
And so, this shift to the left is going to increase the total pressure to compensate for the decrease in pressure that was a result of this applied force to the system.
And we could do this again in terms of q and k. Now we're going to get a little trickier.
What happens if we add an inert gas to a system increasing the total pressure at constant temperature?
So we're not adding one of the reactants or one of the products, we're adding an inert gas, and we're increasing the total pressure.
What's going to happen?
There's a couple of options, what do you think?
How about nothing.
Why would nothing happen?
Well, q depends on the partial pressure of the reactant gas and the product gas.
And in this particular example, the partial pressures are not changing.
We are changing the total pressure of the system by adding an inert gas, but we're not changing the partial pressure.
So let's take this opportunity to review partial pressures, or if you haven't seen it before, I'm going to tell you everything you need to know about partial pressures.
So here's a little review, or for the first time, description of partial pressure.
So what is the definition?
The partial pressure is the pressure that each gas would exert if it were present alone in the container.
So in this example, we have oxygen in a container, and we have one atmosphere of pressure.
In the second one there's nitrogen in the container with one atomosphere of pressure.
If you put this amount of oxygen and this amount of nitrogen together in a container, then you would have two atmospheres of total pressure, but you're still going to have one atmosphere of partial pressure for oxygen, and one atmosphere of partial pressure for nitrogen.
So it's as if the gas is alone in the container.
That's the definition of partial pressure.
So let's look at some equations.
So the partial pressure of a gas, p to sub a, is equal to the number of moles of that gas, and we also have in this equation, r, our gas constant, t, our temperature, and v, our volume.
The total pressure on the system, which is what this two atmospheres is, is equal to the partial pressure of gas a, which would be the partial pressure of the oxygen at one atmosphere, the partial pressure of the nitrogen at one atmosphere, that's all we have here.
So the partial pressure of those two, of one plus one, we have two for a total.
And that's equal to the total number of the moles.
So in these problems, we're going to be considering partial pressure, and the question you're going to be asking yourself in all of these different wordings of the question is is the partial pressure of the gas changing?
That's a very important question to ask because that'll help you answer the questions correctly.
So let's go back to the original question, what happens if an inert gas is added to the container, increasing the total pressure at constant temperature?
And the answer is nothing, because q is not affected, because q depends on the partial pressures, and the partial pressure isn't changing.
We aren't changing the number of moles of the gas in question, we're just changing the total pressure of the system, we're adding an inert gas.
And because partial pressures don't change, q doesn't change, and if q doesn't change, there's no response from the system.
So a lot of the questions in this unit, they're not actually very difficult, but the wording can be tricky.
And so when you're doing these problems, you have to be thinking about what is changing and what isn't changing to be able to get these questions correct.
And if you do that, then these are some nice questions for you to get right on the exams -- they're short, they're good questions to go after.
All right.
Let's see how you're doing with this.
What happens if an inert gas is added to the container, but the total pressure is now kept constant, temperature is also constant.
So let's think about this.
OK, let's take 10 more seconds.
OK, we're going down into the 60's -- again, we're heading toward the 90's by the end, but this is actually new material, so that's fine.
All right, so let's think about it.
So the reaction's going to shift toward reactants.
Let's think about why, let's break it down.
And what was important is that hint that for the pressure to be kept constant, the volume of the container must have increased.
So let's take a look at that.
So if the total pressure was kept constant, if the total pressure didn't change when you added an inert gas, the volume must have changed.
Let's look at this.
Say oxygen is our gas of interest, and n 2 is an inert gas.
If we added an inert gas to our system, the total pressure should go up.
If it doesn't, then something else must have changed, and what must have changed is the volume.
Otherwise the total pressure would not have stayed the same.
So the volume of the container must have increased if the pressure is the same.
So then we asked the question, if you increase the volume, what happens to the reaction, how does it shift?
If you increase the volume, if you expand the volume, then you're going to shift from, in this case where you have one mole to two moles toward reactants, you will, as the volume increases, then you're going to have a change in the partial pressure of the gas -- all of a sudden the gas has a lot more room, and its pressure is going to decrease, the partial pressure will decrease.
It has a lot more space for itself -- it's like it's in there all by itself.
Partial pressure, you can think of it as selfish gas molecules, they don't care what else is there.
They're only thinking about how they're fairing in this environment.
So then we want to shift in a way that compensates for that stress that increases the pressure, so we move from one molecule to two molecules.
So it's all about the partial pressure q and k. So again in these questions you have to dissect what's going on.
What has changed?
The questions are often phrased in such a way, something's kept constant, but if something's kept constant, you have to ask the question, to keep it constant, did something have to change for that to be true.
So again, the wordings of these can be tricky.
Do you have a question?
[INAUDIBLE]
So, the question is is it necessary for it to try to increase pressure.
This is just the predictive tool of Le Chatelier.
Le Chatelier would predict that if the system has changed in such a way, such as volume increases so the partial pressure decreases, that the system will shift in a way to minimize that stress.
And to minimize that stress, you would increase the partial pressure by doing the shift from one mole to two.
So again, this is Le Chatelier's predictive, how you would predict the direction of the shift based on the simple idea that a system where a stress is applied, the system will respond in a way to minimize the stress.
And again, I talked about last time that this concept of minimizing stress is difficult for some MIT students, but you just sort of have to memorize that in this case we're predicting how a system will shift to minimize that stress.
OK.
So if you need a review of partial pressure, go over this.
This part, it looks a little challenging at first, and some of the questions you really have to sit down and dissect what's going on, and for most people, once they get there, that part, those are good questions, you look forward to them on the exam.
So now, we've talked about adding reagents, we've talked about removing reagents, removing products, adding products, shifts in volume, and now we're going to talk about changing the temperature.
So here, Le Chatelier's principle, it's a little bit more fuzzy, but it still basically works.
So if you talk about raising the temperature of a mixture at equilibrium, then Le Chatelier's principle would suggest that the system is going to respond in such a way to minimize that stress.
So if you're adding heat, the system wants to respond in a way to absorb some of the heat, to counterbalance that change to the system.
So let's take a look at this.
So let's think about raising the temperature of an exothermic reaction.
So what's going to happen here?
Raising the temperature of an exothermic reaction, favors the formation of reactants, would shift toward reactants.
Well, why would this be true.
Well, you can think about this very simplistically that in an exothermic reaction, you're going to be producing heat in the forward direction and absorbing heat in the reverse direction.
So if it's an exothermic reaction, that means it's exothermic in the forward direction, which would mean it was endothermic in the reverse direction.
So if you raise the temperature of an exothermic reaction, you're adding heat, the system wants to respond in such a way to absorb that heat, so it would shift toward the endothermic direction of the reaction, which is the reverse direction toward reactants.
So add heat, system wants to shift to absorb some of that heat.
So in this part you think about the direction of the reaction, whether it's exothermic or endothermic, and then think about compensating for that stress.
Endothermic reaction, again, they say it's an endothermic reaction, it means it's endothermic in the forward direction.
So heat is being absorbed in the forward direction -- again this is just sort of a very simplistic way to think about it.
So if you raise the temperature then, you want to absorb that heat, according to Le Chatelier's principle, and so that would favor a shift toward products.
So let's look some more at this.
The predictive tool here is delta h. So delta h, whether the reaction is exothermic or endothermic, is going to be a predictive tool in thinking about the direction of change, when heat is added to a system or heat is removed from a system.
So let's try this out.
Heat is added to a system.
You're given information about your predictive tool, delta h. Which direction will the reaction go?
All right, 10 seconds.
Why don't you discuss with your friends and vote again.
All right, 10 more seconds.
Yes!
I knew we could get into the 90's.
All right, so what type of reaction is this, endo or exothermic?
Exothermic.
And so we're adding heat to an exothermic reaction, so it wants to shift to absorb that heat, so it wants to go in endothermic direction, which is the reverse direction or toward the reactants.
And that's how it's supposed to work too, more people get the right answer after the group discusses.
All right.
So we've been talking about the equilibrium constant so far in terms of it being a constant, it's called the equilibrium constant, but it's only constant at a given temperature.
So the equilibrium constant changes with temperature.
It's also true that rates of reaction change with temperature.
And kinetics is our last unit in this course, and we'll be discussing that quite a bit.
So how does the equilibrium constant change with temperature?
Let's consider that.
So let's look at some of the equations that we know that have delta h in them -- delta h, again, is our predictive factor when we're talking about changes in temperature.
We know that delta g nought is minus r t natural log of k, so the relationship with our standard free energy, our gas constant temperature, and our equilibrium constant.
We also know that delta g nought equals delta h nought minus t delta s nought.
If those equations don't look familiar, then you should go back and review the material on thermodynamics because you're going to need it in this unit, the next unit, the unit after that, and then the next two units, and they we're done with the class.
All right, so good to get familiar with this now.
All right, so we can rearrange these in terms of solving for our equilibrium constant.
And so we see the natural log of the equilibrium constant minus delta h nought over r t, plus delta s nought over r. And so, it's reasonable to assume that for the things we're talking about, that delta h nought and delta s nought are pretty much independent of temperatures.
That would be true for pretty much any temperature we would be talking about.
So that means that k, or the natural log of k, is going to change depending on the temperature.
It won't change -- these other ones are constant at all the temperatures, so there's a relationship between equilibrium constant and temperature.
So you can consider an equilibrium constant at one temperature, and an equilibrium constant at another temperature.
So we can talk about the natural log of a k 2, of an equilibrium constant 2, equal to minus delta h over r t temperature 2, plus delta s nought over r. We can do the same thing for another temperature, another temperature 1, equilibrium 1, and if we subtract these two, delta s is going to cancel out, and we're going to get this equation, which you know is important because it has a name, so this is the van't Hoff equation.
And at some point later in the course, I'll be asking you to come up with this name, so I always get very excited when people can tell me it's a van't Hoff equation later on, not that that's on a test, but there are very few equations that have names.
So here we go.
So what does this do?
Well, we can talk about the natural log of one equilibrium constant over another, an equilibrium constant k 2, which is the equilibrium at temperature 2, equilibrium constant k 1, which is the equilibrium constant at temperature 1, is going to be equal to minus delta h nought over r, and then it depends on the temperature.
So you bracket 1 over temperature 2 minus 1 over temperature 1.
And this just comes from subtracting those 2 equations.
So again, you can think about how the equilibrium constant's going to change with temperature -- that's what this equation does for you.
So let's look at some of the things that fall out of this particular equation.
Let's think first about a case where this delta h nought is less than zero.
What type of reaction is that?
Exothermic.
All right, so now let's consider a case for this exothermic reaction where we're going to increase the temperature, and so that means that t 2 will be greater than t 1 -- we've increased the temperature.
So now let's think about what sign the natural log of k 2 over k 1 will have if these things are true.
So there's a minus in the equation, let's put the minus down here.
Delta h nought is negative, it's exothermic reaction, so that's another negative sign.
And if we're increasing the temperature, and t 2 is greater than t 1, then this temperature term will also have and negative sign.
So if you have negative times negative times negative, you're going to get a negative for your net result over here.
And what that will mean in terms of k 2 and k 1, is that k 1 will be greater than k 2.
So you can think about this mathematically by running through this argument.
You can also sort of think about it in terms of what you would expect if you increase the temperature in terms of what's the relative ratio of products to reactants at equilibrium for an exothermic reaction.
So if you're increasing the temperature of an exothermic reaction, the reaction would want to shift in the direction to absorb that heat in the endothermic direction.
So you would expect that the equilibrium constant that was at the lower temperature would be larger than the higher temperature one, that there'd be less products at this new higher temperature.
So again, you can think about it in terms of Le Chatelier's principle, or you can run through with this equation and do the math and think about the relative size of k 1 and k 2.
So now let's think about what happens when you decrease the temperature where t 2 is less than t 1.
So we can do the same thing, we have a negative sign here, we also have a negative sign for delta h, it's an exothermic reaction.
But now we have a positive sign for this temperature term.
So overall we're going to have a positive sign for the natural log of k 2 over k 1, which mathematically is going to mean that k 1 is less than k 2 or that you would expect more products in the equilibrium constant at higher temperature.
All right, so that's an exothermic reaction.
So if you're ever having problems with Le Chatelier, you can go and use the van't Hoff equation to think about what you would expect in terms of this shift, what you would expect if you change the temperature of an exothermic reaction in terms of the magnitude of the new equilibrium constants.
All right, now you can do the same for me in terms of this delta h nought is greater than zero.
Which of the following are going to be true, and notice there's a possibility that all of them are true or that just some of them are true or that one of them is true.
All right, take 10 more seconds.
See, actually people did very well, because e is also true, all of those are true, but d was also true.
So if we add the 30 and the 58 together, I'm very happy.
All right, so let's look at this -- we'll keep the questions up and let's consider all of them in order.
So the first one is that the reaction was endothermic, I think everybody saw that as being correct.
OK, the second one talked about the equilibrium constant is higher at higher temperatures.
So let's look at the first one.
We increased the temperature here, and we see that if you work out the math or just think about it, k 2 is larger than k 1, so the second equilibrium constant is greater than the first when you increase the temperature.
So that would favor then, in an endothermic direction, if you're increasing the temperature, it favors the endothermic direction of the reaction, so you could think about the ratio having more products at that new equilibrium constant at that higher temperature.
So that one was also true.
Then let's think about when you decrease the temperature, when temperature 1 is greater than temperature 2.
And so this would favor, then, the exothermic direction.
So you would expect less products at this newer temperature.
So, this part asked about k 1 versus k 2, and we see k 1 is greater than k 2.
And then d says there's fewer products and equilibrium when the temperature is decreased, and that's just the word way of saying the same thing.
So all of those are true.
But again, people did very, very well with this question.
All right.
So now we're going to combine what we learned on Friday with what we learned today, and think about ways that that can be applied.
So why should you care about Le Chatelier's principle?
Well, for two reasons -- one I think Le Chatelier had sort of a good life plan that when stress is applied to the system, one should respond in a way to minimize that stress.
I think that is an important life lesson.
But also, it's very useful in terms of thinking about maximizing a yield of a reaction.
So if you were going to create an industrial process and make lots of money and give some of it back to MIT to improve things in terms of teaching chemistry at MIT, then you would want to think about these principles, because you would want to maximize your yield.
If you're going to be making a product, you want to make a lot of that product, and so you want to be thinking about these things.
So we've talked about the reaction with nitrogen and hydrogen making ammonia last time.
And this is an exothermic reaction.
So, there are a lot of people who want to make ammonia -- it's for fertilizer, and as some of you have heard in terms of terrorism, it also can be used as an explosive, so there are a lot of people who want to make a lot of this.
So how do you maximize this yield?
It's an exothermic reaction.
So you can think about temperature.
So here, low temperature would favor product, which is good.
We haven't talked about kinetics yet, but low temperature also slows the rate, which is bad.
Because you not only want to make a lot of your product, you want to make it in a reasonable time frame and sell it and make lots of money and retire early.
So you care about how fast the reaction is going as well.
So for an exothermic reaction then, you have to balance out what's thermodynamically favorable and what's kinetically favorable.
So thermodynamically, you would want a low temperature, but kinetically in terms of the rate, that's not so good.
So they use a compromised temperature, which is 500 degrees c, so that's pretty high actually for this reaction.
So what are other things that you could do to drive this reaction toward products?
So yeah, what's something else you could do?
Increase the volume
Right.
So you could think about volume here.
So here we have four molecules of gas on one side, and two molecules of gas on the other side.
So you can think about changing that to favor your products.
What is something else you could do?
Enzymes
Enzymes, we'll talk about that in a minute.
If you can't use the enzymes, what's something else that you could do?
You could remove ammonia from the system
Yup, so you could remove ammonia from the system, which would also shift the direction.
So let's just put those two things down.
Remove products.
So if you remove ammonia from the system, and they actually do this in the industrial process, so they'll stop -- they'll, at a certain time points, they will remove the product and they'll say hey, let's shift the reaction to make more.
You just changed q versus k, you removed your product, so now we want to shift to minimize that stress and make more product.
And the first response, compress the volume, so we go from four molecules to two molecules, and that would be favor -- again, the system would want to respond, you compressed the volume, you increased the pressure, so you want to respond in such a way that compensates for that increased pressure, which is decrease the pressure by going from the four molecules to two.
So both of those are great, and both of those are, in fact, used.
Sometimes when you study chemistry you'll realize that some of the -- people made a lot of money on this, and that some of the principles are actually very simple.
You are learning principles in this class that, applied correctly, could make you a lot of money.
Some of the scientific discoveries are actually not all that complicated.
All right.
So someone said use enzymes.
I love that answer, I want to use enzymes to do a lot of those things, and people are trying to do this.
Now, why would you want to use an enzyme?
Well, there is a lot of nitrogen in the air, but there's not a lot of usable nitrogen, because n 2 gas is in the air, but it's very difficult to break it apart to form ammonia.
And when you think about bonding, you can probably tell me why it's hard to break n 2 apart, and if you consider how many bonds would be between those two nitrogens.
So it's there but it's hard to do.
So the industrial process that's used now pretty much is the same industrial process that's been used for quite a while, the Haber-Bosch process, which does not use enzymes.
And here are some pictures of these folks.
This resulted in two Noble prizes, the development of the industrial process here, and it's still being used.
On a historic note, these were German scientists, and so, working out this process was important to the Germans during World War II.
It's still the process going on today.
So what about enzymes.
Maybe enzymes could do a bit better than this process.
It's not true yet.
Still the Haber-Bosch industrial process is the one that's being used, but enzymes can do it.
So, some of the problems of a lot of the common industrial reactions are that you use high temperatures -- remember we said the compromised temperature was 500 degrees celsius.
We were talking about compressing the volume, so you want to put in energy to compress that volume.
And so that is expensive.
And you might want to think about a way, could you do that reaction at normal ambient temperature without applying energy to the system.
Well, enzymes can do this reaction at ambient temperatures and they don't have to have high pressure associated with them, and the way the enzymes do this, and here's a little cartoon of a space-filling model of a particular enzyme called nitrogenase.
So for nitrogen, and the ase means it's an enzyme.
And the secret to this enzyme, how it can do what Haber and Bosch got two Nobel prizes for -- this enzyme has received no Nobel prizes, and it can do the same thing.
Its secret are metals.
It uses these clusters of metals in the context of the protein environment to do this chemistry, and they have iron, and they have also inorganic sulfide, and molybdenum, and so these combinations of metals can do this chemistry.
And a number of scientists have been working for decades now trying to understand how the enzyme works.
It's actually quite complicated and it's been pretty controversial.
So let me just kind of show you at the heart of the enzyme, here all the atoms of the enzyme, but if you rotate the enzyme around and go in, the secret was that those combinations of metals.
And one of the units that we're going to have second to last in this courses is about transition metals, that metals can do a lot of really important things in biological systems, and also in industrial processes and other things as well.
So I do also like to mention research is going on here at MIT.
I've mentioned Dick Schrock before, he won a Nobel Prize in chemistry a few years back, but not for his work on sort of nitrogenase systems, but he was able to design a small catalyst that can do this chemistry, that can catalyze this reduction of nitrogen to ammonia at a defined molybdenum center.
Again, we'll talk about transition metals later in the course.
And so his laboratory is very interested in coming up with better ways to split nitrogen than the industrial processes that are currently used.
Some people are studying enzymes, other people are trying to use some of the tools that come out the enzymes, thinking about the enzymes and come up with other catalysts as well.
So this work is going on here at MIT.
All right, so let's give one more example of Le Chatelier from a biological perspective, and this has to do with the following reaction.
The combination of oxygen with hemoglobin in your blood.
And the reaction here, we have hemoglobin plus oxygen has oxyhemoglobin, so the oxygen is now bound to be hemoglobin.
This reaction is very important for you.
None of us would be alive if this was not happening.
We need to get the oxygen from our lungs to our cells.
So what happens if you decide to climb Mount Everest, for example.
You may be thinking how hard can that be after a first semester freshman year at MIT.
Really, I can think more broadly about it.
All right, how many of you have climbed a mountain so far?
Woah, a lot of people.
OK, something about going to MIT and climbing big mountains seems to be connected.
How many people feel like after surviving the first year, why not, go ahead and climb a mountain?
Any more people?
OK, no.
So you gotta -- you've been through the hard part now, you don't want to risk anything anymore.
All right, so this slide seems to be cutting off some stuff, so let's look over here.
So what happens to this reaction?
Well, as you go up higher and higher, the partial pressure of oxygen changes.
So you're going to drop in partial pressure 0.2 to 0.14 .
So what happens to this reaction if this value decreases?
What happens, what direction does it shift?
For products or reactants?
Right.
So, if this is going down, then some of your oxygenated hemoglobin is going to a release its oxygen.
So it'll respond in such a way to compensate for that added stress.
So what can you do to switch it back the other way?
Again, we want our blood to be oxygenated.
What happens to compensate for this if you're going to climb a mountain?
What happens in the body?
Yeah
The body can produce more hemoglobin
The body can produce more hemoglobin.
So you often go to another altitude and hang out for a while before you start going way up.
And what's happening during that time is your body realizes there is a problem, it knows about the Le Chatelier's principle, and so it's going to make more hemoglobin.
And if you add more hemoglobin to this reaction, what direction does it go?
Toward products, right.
OK.
So I wanted to mention significant figures, and there is -- I have to make a confession that when I first started teaching this course, I had never really paid attention to the rules of significant figures for logs.
In this unit, the next unit, the unit after, you get the idea, there are going to be a lot of logs, and they have special rules for significant figures.
So in the bottom of your handout, these rules are explained, it's also in your book.
Pay attention to this.
It's going to help you on all the problem-sets and in the course.
So I just wanted to point out special rules about significant figures for logs.
All right, that's it for today, see you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so today we're going to start talking about acids and bases, and this is acid-base equilibrium, so you can't forget anything that you've learned from the last two lectures about equilibrium.
And then we're going to talk about, after this, oxidation reduction equilibrium, and so there's a lot of equilibrium going on.
So today we're going to give you some definitions.
We're going to talk about autoionization of water.
We're going to talk about the p h function, which most people are familiar with.
We may think about what the p h's are of some commonly found ingredients around campus.
And talk about the strengths of those acids and bases.
And then if we have time, we'll start thinking about how to work a problem associated with a weak acid.
So actually, I guess, do we want to do that other clicker question, maybe, before we get started?
At some point, we're going to ask you a question about when you want forums.
So, I'm not sure if we're going to have that for you today or not, but we need to -- we want to have these pizza forums where you can come, and the time isn't always working, so we thought we could use the clickers to figure out what the best time is for people.
So we may have that for you later.
So this is the narrowest definition of an acid and a base.
So the narrowest definition of an acid and a base is that an acid is a substance when you dissolve it in water, it increases the concentration of hydrogen ions, or h plus.
Whereas a base is a substance that when dissolved in water, increases the concentration of hydroxide ions, or o h minus.
So that's pretty narrow.
We can be broader.
We can talk about Bronsted-Lowry, and here, an acid is described as a substance that can donate a hydrogen ion.
And a base is described as a substance that can accept a hydrogen ion.
So let's look at some examples using that definition.
So let's look at an example, so c h 3, c o o h plus water, and I guess I should put that in aqueous, going to hydronium ion plus c h 3, c o o minus aqueous.
All right, so what is going on here?
So if we look at the things on one side of the equation and the other, you can see that this is an acid that has lost its hydrogen ion, so the hydrogen is gone, whereas the water molecule has gained a hydrogen ion, and so now it's h 3 o plus.
So we have an acid here, it's acting as an acid, and acid acts as a substance that gives up a hydrogen ion, and the water is acting as a base, it's accepting that hydrogen ion.
And when it accepts the hydrogen ion, it becomes an acid in the reverse direction, whereas when the acid gives off the hydrogen ion, it becomes a base in the reverse direction.
So in the reverse direction, hydronium ions is giving off a hydrogen ion to this base, reforming the acid, and after a hydronium ion gives off its hydrogen ion, it forms water again.
So that would be an example of Bronsted-Lowry talking about substances as acids and bases whether they accept or donate a hydrogen ion.
And this is the definition we're going to be using mostly throughout this unit.
So, let's look at a little movie of this going on.
So, in this movie, we have our water molecules with red and the white dots here are their hydrogen atoms, and now we're going to come in and have an acid come in, there's the acid, it has the hydrogen ion on it in white.
It meets up with a water molecule, and now you formed hydronium ion.
And that forms another water molecule and it passes it along, so there's a different molecule of hydronium ion.
So that's what's going on in this definition.
All right.
So this brings us to another term, which is conjugate acid base pairs, so you can talk about something being a conjugate base of a particular acid, and so a conjugate base of an acid is a base that's formed after the acid has donated its hydrogen ion.
A conjugate acid of a base is the acids that its formed when the base accepts the hydrogen ion.
So you can look at those examples here.
So we have a pair that we've drawn here with this red line, an acid base pair, and the other pair is the water and the hydronium ion.
All right, let's look at a couple more examples to get the sense of this and figure out what are the acid base pairs.
So, one more example.
So now let's look at h c o 3 minus an aqueous solution and water, going to, again, hydronium ion, and c o 3 minus 2, also an aqueous solution.
So, what is h c o 3 minus acting as here?
As an acid.
And so what does that make water?
A base.
And so the conjugate acid of that base, again, is the hydronium ion concentration.
And the conjugate base of the acid is c o 3 minus 2, over here.
And so in the reverse direction, this base will be accepting a hydrogen ion from the acid, forming the conjugates on the other side.
OK, now you can do an example.
Let's have a clicker question.
So identify what the acid and the base pairs are here.
All right, let's take 10 seconds.
That's quite good actually.
So, that's correct and you can write it in your notes that here we see that there's a little bit of change, water's doing something different.
So instead of the conjugate of water being the hydronium ion, we see it's a hydroxide.
So here, the water is acting as an acid giving off a hydrogen ion to this h c o 3 minus.
And so now we have a second hydrogen ion over here, we have the h 2 species, and that the conjugate of water is o h. So this is acting as a base, it's accepting a hydrogen ion, this is donating it, it's an acid, this is the conjugate acid of that base, and this is the conjugate base of that acid.
So in the reverse direction, this is an acid giving off a hydrogen ion to the hydroxide forming water.
So one thing that you'll notice about these examples that we've written up is that when you see water in the equation, you don't really know what it's going to be doing until you actually look at what the products are and then you can figure it out.
So water can act as either an acid or a base in these equations.
And if we go to the lecture notes, the term is amphoteric, which is a molecule that can act as either an acid or a base, depending on the reaction conditions.
So depending on if it's mixed with something that's a stronger acid or a stronger base than it is.
And an example, one of the most common examples is water.
So now let's consider a broader example of acids and bases, and these are Lewis acids and bases, and we're going to actually come back to this around Thanksgiving time when we talk about transition metals.
And so here it's really broad -- we're not actually even going to talk about a hydrogen ion at all.
So in this case, we're talking about a Lewis base as a species that donates lone pair electrons, and a Lewis acid is a species that accepts such electrons.
So here would be an example.
So we can think about forming a complex, and which thing is going to act as an acid or a base.
One will be donating its lone pair electrons and the other will be accepting.
So this is a very broad, a much broader definition, and so when you talk about acid base here, so always say as a Lewis acid or Lewis base to make it clear what's going on.
So again we have our base donating its lone pair electrons and the acid accepting.
All right, so those are our definitions of acids and bases.
So now let's come back to this issue of water and how water can act as an acid or a base.
So if it can act as an acid or a base, it seems like it can react with itself to do some chemistry, and it can.
So up here, you could have one water molecule acting as an acid, giving up its hydrogen ion to another water acting as a base, forming hydronium ion and also forming hydroxide ion.
So then you can ask the question, well, how much h 2 o is in a typical glass of water.
How much, so you don't like the idea that I'm drinking hydroxide ions, how much hydronium ion and how much hydroxide ion are in this glass of water, how much h 2 o is in a glass of water?
So that's the question.
So here's the equation again, and we can think about how to calculate it.
What do we really want to know in this question?
What are we really asking?
How much, at an equilibrium situation, how much are products and how much reactants do you have?
What can you tell me about ratios of products and reactants at equilibrium?
K. And how are some ways you can calculate k's?
Different terms, but, right, it's what about -- what is k?
So we have the equilibrium constant, and there a couple different ways one might calculate k -- you might be given concentrations at equilibrium, or you might be given information about delta g. So you can calculate k's from delta g nought, and this is an equation that you'll use pretty often -- delta g nought equals minus r t natural log of k. So if we want to know k, we need to find out what delta g nought is.
What are ways to calculate delta g?
So you've seen some of those -- oh, you're probably recognizing these already, temperature and our gas constant.
So we can solve for k in terms of delta g. And how do we calculate that delta g. Well, there are a couple of ways.
So you can think about the delta g's of formations, and we can think about one of my personal favorites, which is relationship between delta h and t delta s. So you can think about your enthalpies and your entropies at a certain temperature, and you can calculate delta g, and from that you can get the equilibrium constant.
So this is just a little review showing the relevance of material you've learned before to the material we're covering now.
So we're going to pick one and just calculate the delta g. So we can look up these values of formation for our products and our reactants and plug them in, and we get a value for delta g nought of plus 79 .
89 kilojoules per mole.
So we have a positive value here.
So without doing any more math, which we'll do it a minute, do you expect a large or small value for k for the equilibrium constant if delta g nought is positive 79 .
89 kilojoules per mole
Small
We would expect it to be small.
And you probably already knew that that there's a lot of h 2 o in a glass of water.
So again, from that perspective, we'd also expect it to be small.
So now we can plug those values in, we calculated this delta g, we know the gas constant, we're at room temperature, and so we get a value of k as 1 .
0 times 10 to the minus 14 at room temperature, and that's a small number.
So, the very small value indicates that only a small percentage all your h 2 0 has ionized, and that mostly, there's h 2 o in a glass of water, not so many ions.
Not many of the molecules have ionized, because k is a small number, not a lot of products at this equilibrium.
So there's a lot of h 2 o in a glass of water.
So, this particular k has a special name, and it's k w, w for water.
And this term and this number, if you haven't memorized it in high school, you probably will by the time you're done with problem-sets.
This is a very valuable number, you'll be using it a lot in calculating acid base problems, and you will end up memorizing it whether you want to or not.
So, then k w equals your hydronium ion concentration times your hydroxide ion concentration.
Now for a minute, let's consider why that's true.
So our reaction, it's products over reactants for an equilibrium constant, but now, all of a sudden, I don't have my reactants going on in here.
So the k w is expressed in terms of the concentration a hydronium ions times the concentration of hydroxide, and it doesn't have this water term at the bottom.
And that'll be true for any problem in which the water is a solvent.
And so, it's really not going to be changing very much.
A solvent is nearly pure, and when you have in a nearly pure solvent or solid, it's not included in the equilibrium expression.
So we'll see other examples of this as we go along.
So you always want to ask yourself is this the solvent.
If so, we're talking about very dilute things going on in solvent, the solvent concentration isn't changing very much, so it drops out of our term.
So, because k w is an equilibrium constant, the products are always going to be equal to the same thing at the same temperature.
So, at room temperature, or 298 kelvin, it's always going to be equal to 1 .
0 times 10 to the minus 14, and that's why it's such a valuable number.
And when we're talking about acid base problems, you're almost always going to be at room temperature, just to not make life more complicated for you.
So, you can pretty much assume, it should be in big bold letters if the temperature is not room temperature, so that you can use these values.
All right, so let's look at the p h function now.
So what does p h equal?
So p h is equal to the minus the log of the hydronium ion concentration, and let's also talk about p o h, and that's equal to minus log of the hydroxide ion concentration.
I just told you that k w is equal to the concentration of hydronium ions times the concentration of hydroxide ions.
And now we can express this in another very useful way for you.
If we take the log of all sides, actually let's take the minus log of everything, so minus log of k w equals the minus log of hydronium ion concentration, minus the log of hydroxide ion concentration, and we end up with terms of p k w being equal to p h, because minus log of the hydronium ion concentration is p h, and minus the log of the hydroxide concentration is p o h. So plus p o h. And we know that this term at room temperature is 1 .
0 times 10 to the minus 14.
So this term at room temperature is 14 .
0 0, again at 25 degrees c or 298 kelvin.
So this is also a useful expression.
If you know the p h, you can calculate the p o h if you're at room temperature, remembering this number of 14.
So these are things that you will be doing a lot in the problems and you will start remembering all of these numbers really well.
So p h and p o h, what does p h do for you?
Well, the p h tells you about the strength of the acid.
So the p h of pure water should be neutral, which is 7.
And now, tell me what the p h of an acid is and the p h of a base.
Let's just do 10 seconds on this, this is pretty straightforward.
So this tests previous knowledge on this topic, and it is very good.
People know about what the p h's are.
So that's right.
So the p h of an acid solution is less than 7, and the p h of a base solution is greater than 7.
And the EPA defines corrosive as something where the p h is lower than 3 or greater than 12 .
5.
So, if here is our scale of p h, we're neutral at 7, we're acidic below 7, and we're corrosive below 3.
We're basic above 7, and corrosive above 12 .
5.
So now what I want to do is ask Dr. Taylor to come up and we're going to measure some p h's of things, so we're going to be interested in knowing how much danger you're in around MIT.
So we're going to have you measure them.
We have these little strips on it, and so someone will come around and help you read it, so you can read off the strip of what you have.
Should we start with water?
This is random MIT water.
Let's start there.
So, just pick a volunteer
OK, so what we're going to do is have the TA's go in and ask you to read off of a p h strip, what the p h of various things are, and actually Marcus, if you could write on the board what these are.
So we'll start with MIT water.
So, we know if it's 7 it's neutral, if it's below 7 we're talking about acidic, and above that it's basic.
We actually, also for you to be able to visualize as well, what I did is I just boiled up some cabbage last night and brought in the extract with me.
And cabbage actually has anthocyanins in it, which is a color indicator, and it changes color based on whether it's in an acidic or a basic solution.
So we'll let you see this here.
It looks like MIT water, pretty safe to drink, which is good news.
We can go ahead and -- so it looks like if we add MIT water to cabbage solution, what do you think's going to happen -- this is neutral right now.
Hopefully not much.
We either have invalid strips or we'll see nothing happen here.
All right, so you can see we still have the purple color for MIT water.
Two confirmations that it's safe to drink right out of the tap when you get home.
So, the next thing is vinegar did someone take the, the strip?
We have a 2 and a 1/2
OK, so are we drinking vinegar?
No
Probably not straight.
All right, so we've got our cabbage extract here, it's purple.
Does anyone have a guess as to what color it's going to turn if we pour in vinegar, very acidic.
All right, a couple guesses I hear, blue and pink.
They're both good guesses because different color indicators turn different colors.
But, all right, looks like a dramatic difference here.
What do we have next out there?
Baking soda.
Seven for the baking soda.
I'm going to guess we did not pour in enough or it is not dissolved here.
All right, so let's do our secondary test here and see what happens with the baking soda
[UNINTELLIGIBLE] the water we added it to was also not neutral
All right, so we're actually pretty basic with the baking soda.
We won't give it a number because it was just dissolved in water.
But we'll remember baking soda, blue here is basic.
So the next thing we're going to test is soda that you drink all the time.
I brought Sprite.
Coke probably would have been a good pick as well or Diet Coke.
So, we'll test what that is.
Hopefully it comes out neutral, right?
So while we're waiting, we'll start taking a little bit of a look here.
All right, we've got a 3.
Soda, corrosive.
It's not just the sugar that's bad for your teeth.
Luckily we see here it is not quite as bad as vinegar in terms of how acidic it is, but we definitely have a color change here
Has anyone used soda for something other than drinking?
What did you use it for?
Cleaning pennies
Cleaning pennies, what else?
When you have stuff all over your car battery you can use Coke
And some of those other uses make sense.
[UNINTELLIGIBLE] What else?
Taking the galvanization off of a steel wire
OK, so cleaning steel wire.
How many of you still drink soda knowing this information?
All right, so the next thing we put out there was aspirin dissolved in water, and it's going to depend what concentration we did, but we put aspirin in water and we got a 3 here.
So, aspirin sometimes gives you an upset stomach, and that's why Tylenol's an improvement in some ways -- obviously, that has its own drawbacks, too.
But you can see what your stomach on aspirin might be feeling like here.
So if you're having an upset stomach, something you might do is take Tums or Mylanta or some other kind of a -- yeah, let's measure the p h here.
So if you decide to take some Milk of Magnesia after an upset stomach, are you hoping it will be acidic or basic here?
Basic
All right, let's see what we get.
All right, so this is kind of thicker.
Let's see how this works.
I think you can start to see the green at the bottom.
So, it's white in here, so this is not just color.
So we'll let that slowly mix in.
What did we get for a p h, if you can read it on there.
This might be another no-go
It says 7, but--
It's probably blue--
So, how many people have had lunch yet today?
How many are going to have lunch soon?
How many are reconsidering what they're going to eat for lunch based on this demo?
Al right, we'll take a look at one last thing that you might be consuming.
Lemon juice, or actually this is lime juice here.
All right, we're probably ending with an easy one here.
What do you think, acidic or basic for the lime juice.
So, really, the question is probably just the shade that we're going to get from going from a purple here.
All right, so what are we reading for the lime juice?
A 2,
OK, so that's the end of our demo here, providing you with some tips about what is corrosive and what is not corrosive around MIT.
And I think the title of this lecture on the syllabus is "is it safe to drink the water at MIT," and the answer is a lot safer than drinking soda.
So let's talk about some acids in water some more, and introduce something you'll use a lot, which is an acid ionization constant.
So let's look at an example of an acid in water.
So say we have c h 3, c o o h aqueous, so it's in water, which is our solvent.
It's acting as an acid, so it's giving off a hydrogen ion to water forming hydronium ion, and forming its conjugate, which is missing its hydrogen ion.
All right, so here we have an equation, and now we're going to introduce something which is the acid ionization constant, or k a, and you'll be using k a a lot in this course, we're also going to have some k b's for bases.
So the acid ionization constant.
And it's an equilibrium constant, so you all know how to write equilibrium constants.
So the equilibrium constant is going to be products, which in this case is hydronium ions times the concentration of the conjugate base of the acid over the conjugate acid.
And there is no water in that equation, because here the water is pretty much pure.
It's the solvent so its concentration is not going to change very much, so it is left off.
And I can tell you that the ionization constant here is 1 .
76 times 10 to the minus 5.
Again, that's temperature dependent, so that's at 25 degrees.
This is a small number, so that tells us this is not a very strong acid.
So it's not ionizing very much in solution.
That is a definition of a weak acid, something that doesn't ionize very much.
The definition of a strong acid is something that does ionize quite a bit.
So here then, we can write equations generically for acids and bases.
You can write an equation generically, h a being as your acid, plus water, goes to hydronium ions, and your conjugate of the acid, which is a minus.
So this is an acid, h a, in water.
We can also write it as h b plus as an acid in water going to hydronium ion concentrations and the conjugate, which is base, it's lost its hydrogen ion as well.
A strong acid is something that has a k a greater than one.
More products than reactant at equilibrium.
So that means it ionizes almost completely, so it goes far toward products, ionizes almost completely.
A weak acid is something with a k a of less than one, which means that when you put this acid in water, it doesn't ionize very much, when you have equilibrium.
So you can tell if something is a strong acid or not by looking at it's k a value, or alternatively you can consider something called p k a.
So the p k a is minus log of the k a, and if you have a low value of k a, you'll have a higher p k a, and the higher the p k a then, the weaker the acid.
So you can look it k a or you can think about p k a in terms of whether something is a strong acid or not.
And so we'll just finish up with this slide.
So, up here we have some very strong acids.
These k a's are much greater than one, and you have extremely low values for p k a.
And if you keep going from strong acids, again, a strong acid has a k a greater than one, so these are all strong.
And so, then weak acids are less than one.
And so down here you'd have small numbers for k a, and so -- up here we have big k a's, here we have smaller k a's, and the corresponding p k a's are going up, and if we keep going, this table is very long, we're going to get some very high p k a values when you have some very, very small numbers.
OK, we'll stop there for today and continue acid base next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
-- All the things would be relevant, so here are some examples from a couple of lectures ago.
And so here we're thinking about what happens when we add an inert gas.
And remember, it's all about the partial pressure.
So, you always have to ask yourself did the partial pressure change, and partial pressure's going to change if there's a change in volume.
So, the secret in this problem is realizing that if inert gas is added, and the total pressure is kept constant, what had to have happened?
Yup, the volume would have had to increase, and so the system is put at stress, and it responds in a way to minimize the stress.
So it's going to respond in a way to go from fewer numbers of molecules to more molecules.
So, on one side of the equation reactants there are three, and on products there are two, so it's going to shift toward reactants.
All right.
So today we're going to have another clicker competition, and because it is Halloween, this is the prize for the recitation that has the most correct answers.
So, let's see how we do today.
All right.
So we're going to continue where we left off on Wednesday, and so these are your notes -- I've added them to today's handout and they're also in the handout from the last class.
And I want to make a note that it's a good idea to start this problem-set early.
You don't know everything you need to know to do the problem-set, but you do know a number of them, so there's some questions on thermodynamics and equilibrium, and Le Chatelier's principle, you can do all those problems.
So next week -- today we're going to talk about bases and buffers, and then we're going to move into acid-base titrations on Monday.
And so, the problem-set looks like it's not that long, it's not that many questions, but the acid-base titrations have many parts, and each part is actually quite long.
So it's a very deceptive problem-set.
So don't be fooled by the total number of questions.
All right, so we were talking about acid and water and base and water and p h, and so we're going to continue with base and water right now.
So here we have a base in water, and so in this case, the water is acting as an acid.
It's giving up a hydrogen ion or proton to the n h 3, causing it to form its conjugate, n h 4 plus, ammonia ion, and also hydroxide ion.
So here we have a base in water, and when we're talking about a base in water, we're going to talk about base ionization constant, or k b.
So at the end of last class we talked about the acid ionization constant, or k a, and when you're talking about bases, you're going to talk about k b's.
So, k b, it's an equilibrium constant for this reaction of a base in water, and so it'll be equal to the products in h 4 plus and hydroxide ion over the reactant, n h 3.
The water is the solvent here, and since this is all pretty dilute, it's mostly pure and its concentration isn't going to change, so it's not included in that equilibrium constant.
So we have a k b when we're talking about a base in water.
And we were talking about a base in water, the equation should work that you have hydroxide ions on one side of it.
So here, the k b is 1 .
8 times 10 to the minus 5 at 25 degrees, so that's a fairly small number.
And so the small value tells us that only a little bit of the n h 3 is going to ionize when it's in solution.
So only a little bit it's going to form n h 4 plus and hydroxide ions.
So that's what that small number tells us.
So that tells us that it's going to be a weak base.
So a strong base is something that's going to react pretty much completely to go to hydroxide ion concentration.
A weak base only ionizes a little bit in water.
And you can tell about whether something's strong or weak by its k b value, or if it's an acid, it's k a value or p k a value.
So here are some general ways to write these equations.
We have a base in water and so the water's going to act as the acid, base is going to accept that proton or hydrogen ion forming a base, h plus, and hydroxide ion.
So the base is just written as b.
You could also write the base as a minus, something a minus in water going to h a plus hydroxide ion.
So sometimes you might see that when you're talking about a conjugate base of a weak acid.
So these are two expressions that you'll see that are fairly generic that expresses what happens when you have bases in water.
Now remember, you know it's a base in water, you better have hydroxide ions on the other side, because a base in water is going to be forming hydroxide ions, and acid in water would be forming hydronium ions.
So, a strong base, again, almost completely ionizes to o h when in water, and here we can know what's strong or weak by the k b, so the larger the k b the stronger the base.
And like there is the p k a, there's also a term p k b. P k b is minus log of the k b.
And the larger the p k b, the weaker the base.
Now you won't see p k b very much.
It's not used very much, most things are converted to a p k a.
So you'll see p k a's quite a bit, and you will see p k a's if you take organic chemistry, if you take biochemistry, if you take biology, you'll be hearing a lot about p k a's as we go along.
Not so much about p k a's, but p k b's.
And so what I want you to do when you hear about p k a's, is remember that you've learned about it.
Because I have been confronted by some of my colleagues who teach in advanced levels, and they said our students tell us that you never talked about p k a's in freshman chemistry.
And I assure them that I did.
So I'll be emphasizing this.
And so I want to make sure that by the end of this unit you're really familiar with p k a's because you'll need them later on, and I want you to really impress my colleagues in later classes, and they'll say oh, that's a 511-1 student, of course, they know what a p k a is.
Even my six month old daughter who's over there, she's like what, people didn't know what a p k a is?
You don't want to get her upset.
OK.
So all of these things are related with the acids and the bases, because for every acid it has a conjugate base, every base is a conjugate acid.
And so, if you have a stronger acid, the stronger the acid, the weaker its conjugate base.
And the stronger the base, the weaker its conjugate acid.
And this becomes very important in doing these problems.
So here's a little table that emphasizes that fact.
So we talk about a strong acid.
Most people are familiar with h c l, hydrochloric acid.
So it's a very strong acid.
And its conjugate base, c l minus, is not really a base, it's completely ineffectual as being a base.
It doesn't really do anything at all.
A strong acid really drives you all the way to hydronium ion concentrations.
It doesn't go back the other way, it's not really equilibrium, it's just going to completion there.
So the conjugate is really, really weak, basically not a base at all.
Then we get into this middle range and here things that are moderately weak or very weak acids also have their conjugates in the weak range.
But if you get to something that is a very strong base down here, its conjugate is going to be also ineffective as an acid.
So if something is very strong, its conjugate is pretty much non-existent in those properties, but when you have weak-weak, then you can start talking about buffers, which we're going to get into later in today's class.
OK.
So let's prove that, in fact, it has to be true that there's a relationship between the conjugate acid and its conjugate base or conjugate base and its conjugate acid, that they both can't be strong.
One has to be -- you have to be weak-weak, strong or ineffectual.
All right, so let's look at the first one up here.
So first let's look at what is this acting as?
What is n h 3 acting as, an acid or a base in this equation?
So it's acting as a base.
And that means water is acting as an acid.
The water gives up a proton or hydrogen ion to the n h 3 forming the conjugate acid of that base.
And then the conjugate base is the hydroxide ion.
All right, so now let's write term for k, and so we're talking about a base in water, so we're talking about k b.
So k b is going to equal what -- what do I put up here?
Tell me one thing to put up there.
Yup.
OK, n h 4 plus, and hydroxide ion over n h 3.
OK, so we don't have water in there.
All right, so let's look at the next reaction.
So what is n h 4 plus acting as?
It's acting as an acid here, so it's giving up its proton or hydrogen ion to the water, which is going to act as a base and accept that hydrogen atom.
And when this gives up its hydrogen ion or proton, it forms its conjugate base, and the water is conjugate here, is an acid, hydronium ion.
So we have our conjugate acid base pairs here.
So now am I talking about a k a or a k b?
I'm talking about a k a, so I'm talking about an acid in water, and we know this is an acid in water if we look at what's happening over here.
So we have an acid in water, and so we'll put our concentration of hydronium ions and n h 3 over n h 4 plus.
So now we have k a's and k b's written for the conjugates, the conjugate acid of n h 3 and n h 3 itself.
So now, we can think about what happens if we take these k's and we multiply them out together.
All right, so we have a k a and we have a k b.
All right, so we have a k a and a k b, so if we take our k a and times our k b, we're going to just multiply these out together.
So I'll do this one first, k a -- just copy from above -- times k b, hydroxide ion, over n h 3.
And things are going to cancel out, and so I'm left with hydronium ion and hydroxide ion.
What is this, when you have hydronium ion times hydroxide ion?
What is that called?
It's another k. K w. So we just showed that k a times k b equals k w. So we can take the logs of all our k terms here, and if we take the log of k a plus the log of k b equal the log of k w, or p k a plus p k b equals p k w equals 14.
So there's this relationship with a conjugate acid and its base between its k a and its k b.
So if one is really big, the other has to be small or they can both be sort of in the middle.
But they're always going to add up in terms of the p k a and the p k b to 14.
And the thing about these problems is if you're given a k a for an acid, you can calculate the k b for its conjugate base, and you'll be doing that a lot in titration problems that are coming up.
All right, so there's this relationship between the strength of an acid and the strength of its conjugate base, and let's just think for a minute again about this concept of strong and weak, because this is really important for the next unit.
So if we have a strong acid, h a, in water, it's going to go pretty much completely over to hydronium ion and the conjugate, and this conjugate is going to be really ineffective as a conjugate base, as a base at all.
So it's going to really go all to that hydronium ion concentration.
And so, in talking about a strong base, you don't really have to worry about an equilibrium situation.
Just remember it goes pretty much to completion, and so you can do complete subtractions when you're doing this.
And the same is true for a strong base.
So, for a strong base, any b, in water, it's going all the way down, it's driving the reaction all the way over here, and you're forming, you can consider it that however much strong base you added is how much hydroxide ion you have here.
How much strong acid you add is equal to how much hydronium ion concentration you have here.
So however much of a strong acid or a strong base, you think they go all the way to completion, but for a weak acid we're going to have equilibrium, and so you'll have to set up equilibrium tables to figure out if you added this much weak base, how much did it ionize.
So remember that -- people get worried about the strong acid, and you just assume it goes right to completion.
And you can tell again by the k a's and the k b's what's going to be strong or not.
And so our definition for strong acids is that you have a k a greater than one, strong base pretty much, the only problems you use, people are adding sodium hydroxide or potassium hydroxide, there are not a lot of options for strong bases.
But for strong acids, people are always worrying about whether they've identified those correctly or not.
OK.
So let's look at this relative strength of acid problem and do an example here.
So in this equation we have acid in a base going on one side, another different acid on the other side.
So we can look at whether the reaction is favored toward this direction, toward the right or the left, depending on which acid is stronger.
Will the reaction ride to the right or left.
So if this acid is stronger, then it should drive the reaction this way.
If this is the acid on the other side, if it's stronger, then you would expect to drive the reaction the other way.
So we can take a look at that.
So we can consider the k for the overall reaction, again, just products over reactants here.
And we can also consider the reaction from just each acid alone in water.
So we can consider it separately as well.
So first we can take a look at one acid.
So if we take a look at this acid alone in water, it'll form h 3 o plus and n h 3 minus.
So when it gives up its hydrogen ion to the water and then it forms its conjugate over here.
So now are we going to look at a k a or a k b?
K a?
So we're going to have our products over our reactants, and the number is quite large, 20, for that k a.
And now we can look at the other reaction as well.
So we can look at this acid in water here and form your hydronium ion concentrations and your conjugates -- again, we're looking at that acid in water, so it's a k a, and we have our products over reactants.
Now we have a number of 5 .
6 times 10 to the minus 10.
So, we can consider those two equations back together, and this time we're going to be subtracting the equations from each other to get our sum equation.
And because we are subtracting, we're going to end up to dividing the equilibrium constants.
So when we add the equilibrium constants together, we multiply things, and if we're subtracting we divide.
So the k in this case is going to be equal to the k a of the first acid over the second acid, and you can prove this to yourself, you write up the k a here, and the k a here, and then some of your terms, the hydronium ions cancel, and you get the k overall equilibrium constant that we wrote in the beginning, again products over reactants.
We know the value for k a for one acid, we know the value of k a for the other acid, and we can divide those to get the k for the overall reaction.
And then you can tell me what that k means in terms of which is the stronger acid of the two, and which side of the equation does the reaction lie to, the right or the left?
All right, let's give 10 more seconds.
Yup, people did pretty well on this.
So you could have thought about it in terms of the overall k or of the individual k a's.
So, the stronger acid is the one with the larger number.
And h n o 3 had a number of 20, so that was pretty big, so that's a really strong acid.
And because it's a strong acid, it'll lie to the right, so it's going to push toward products -- it's a strong acid, so it wants to disassociate a lot, so would push it that direction.
And you can see that also in terms of the equilibrium.
Overall equilibrium constant, if we go back to the slides for a minute.
So this number overall, k, is also quite large, very large, so that means a lot more products than reactants at equilibrium.
So again, those are what you can determine if you're given a table of k a values, which on the test you will get a table of k a values, you can tell me a lot about different reactions knowing that information about k a's.
All right.
So in this unit there are different types of acid base problems, and sometimes it feels for people like there's an infinite number of different types of acid base problems.
But, in fact, there are really only five.
And so one of the things I strongly recommend in this unit and working problems is figuring out which type of problem it is, and that will help you a lot in solving it.
So you can either have a weak acid in water, a weak base in water, and sometimes you can be fooled and say oh, it's a salt and water problem, but a salt and water problem actually breaks down to a weak acid and water problem and a weak base and water problem.
So it's really not a different kind of problem, and we'll see that in a few minutes.
And you can have a strong acid in water and a strong base in water, and then you can have my good friend the buffer type of problem.
So those are the type of problems and being able to recognize them is key to doing well in this unit.
So let's work a problem in the first type of a weak acid in water problem.
All right, so what's a weak acid?
Well, vitamin C is a weak acid.
And so sometimes when you're taking your vitamins you get a bad taste in your mouth.
And if you did take a vitamin C tablet here, which is 500 milligrams in this vitamin C tablet, and dissolved it in water -- this is not scientifically measured -- but dissolved it in water, so you would say taking your vitamin with a lot of water, and it was starting to dissolve and being pretty unpleasant, then we could calculate at equilibrium what kind of p h we would have in that mixture.
Now, these vitamins, this Nature's Bounty, they do a really good job of isolating the vitamin, so it is pretty much impossible for it to dissolve.
So they have a nice coating around it that's highly protective, at least at normal water p h's, so it doesn't really dissolve.
And one year I thought I would do the actual experiment, we could talk about significant figures, but I could not get the tablet to dissolve -- I heated it, I stirred it, I did everything at neutral p h, at room temperature.
I want to do it at room temperature, it just didn't work, even the high temperature didn't work.
So if you buy Nature's Bounty, it will not dissolve in the water that you're taking, so just for that little bit of information.
But if you had, say, an inferior brand of vitamin C that readily, that didn't have a nice coating around it, then you could do this experiment.
So let's take a look at that.
So the first thing that we have to do is calculate the molarity of the acid that we've added.
So here, just have to make sure that your units are going to be correct.
So we have grams, we're converting it with a molecular weight to moles, and then we have the number of moles in the amount of water, and we can calculate the molarity of that solution.
And one of the mistakes that people often make in doing these problems, they forget to do all of the conversions that are necessary, sometimes they stop at moles, and you're talking about concentrations here, so don't forget about your friend the volume.
All right, so then you can write and equation, and I highly recommend that people do this on the test because it helps them figure out what type of problem it is and it voids people making silly mistakes.
So if we're talking about an acid in water, you should make sure that your equation reflects an acid in water.
If you have hydroxide ion on the other side, something is very wrong that's going on, and acid and water is going to be giving hydronium ion concentrations and a conjugate base.
So then we can set up an equilibrium table here.
We calculated the initial molarity.
And in the beginning there's nothing over on this side, so we've just added our weak acid to water.
And so then the change, there's going to be some amount of this that ionizes minus x, some amount of this that's formed, and some amount out of the conjugate that's formed.
So we have 0.0284 minus x plus x plus x.
Now we're talking about a weak acid in water, so what term am I going to want to use next?
K a.
So I'm going to want to use k a next -- k a value is here, 8 .
0 times 10 to minus 5, we have products over our reactant here, and we have x squared over 0.0284 minus x.
Now you can make an assumption when you're working on these problems and check it later.
So you can make the assumption that x is really kind of small compared to this o.084, and you can just drop this x out of the term here.
And then check later and see if that worked or not.
So that makes the math easier, and so now we can just solve for x, and x comes out to be 0.00151.
The really two significant figures, but we're going to carry an extra one for the moment, so they're just two figures right here, two significant figures here.
Now we can check and see if x was really small, if that assumption worked right.
So, is 0.0284 minus 0.00151 really the same as 0.0248, and we let you make the assumption that it is, we say it's OK if it's less than 5% of the value.
So, in this case, it's actually not, it's 5 .
3, so that violates our policy.
So it's more than 5%, so then you have to use the quadratic equation to solve the problem.
I just want to note that this term, this percentage, can be called sometimes percent ionized or percent deprotonated, so that you're not thinking that's some kind of bizarre term if you see that.
And if you use the quadratic equation, you get an answer of 0.00147, again, that's really two significant figures.
So it's not a whole lot different, actually, than the number you got making the approximation.
So, once you know what x is, x is the hydronium ion concentration in this problem, and so we can plug that in, p h is minus log of this, so that's 2 .
83.
And so we had really two significant figures here, and so we are going to have two significant figures after the decimal point here.
And if you haven't reviewed your sig fig rules and need more help if that seems wrong, then you should definitely review it before the next test.
All right, so now we'll continue with today's lecture notes, and we're just going to continue right on and we're going to talk about weak bases, and well, we're start working on our way through, we're also going to try to get to buffers today.
So, we've done a problem for a weak acid in water.
So now let's talk about a weak base in water, and you can start us off.
So in this problem we're given a molarity, so you didn't have to calculate that, and now you can help me fill out the table so we know what to do here.
OK, let's just do 10 more seconds.
Very good.
So you're going to be losing some of the amount -- some of the amount of the weak base you have in, it's going to ionize, and so then you'll be forming the conjugate acid plus x, and you're going to be forming hydroxide ions plus x.
And so the one with the minus sign is important, and sometimes there will be 2's involved, and that depends on the stoichiometry of the reaction.
All right, so we can use that information now and go on and look at -- actually you could leave that clicker question up for a minute, and we're going to talk about the k b.
So the k b is going to be equal to our products, n h 4 plus and our hydroxide ion concentration over n h 3.
And so now I can fill in the values that you told me.
So we have, on the top we're going to have x squared, on the bottom we're going to have 0.15 minus x.
And now we can make an approximation here that x is going to be small compared to 0.15, and so we can say that's just going to be equal to x squared over 0.15, and the k b value that was given was 1 .
8 times 10 to the minus 5.
So now we can solve for x using this approximation, and using this approximation x comes out to be 0 .
00164, and we can look at whether that approximation was OK.
So is this number less than 5% of 0.15, so we can say 0.00164 over 0.15 times 100, and that comes out to be 1.1%, so that's OK, that's less than 5%.
So that's good, we don't have to use the quadratic equation here.
So now, we want to calculate the p h. So can I just plug that number for x into my p h equation?
What is x?
What is x equal to here?
It's equal to two different things.
What's one of them?
Hydroxide ion concentration.
So what we can do is calculate a p o h. So, p o h is minus log of the hydroxide ion concentration, or minus log of 0 .
001647, and we really only have two significant figures here, and that is going to come out to 2 .
79.
And so, we would have two significant figures after the decimal point, because this number had two significant figures in it.
But I'm not done.
I've calculated p o h and the problem wanted p h. So how do I go from p o h to p h?
14 minus, yup.
So 14 .
00 at room temperature minus 2 .
79 is going to be equal to 11 .
21.
And so that makes sense.
Now in doing these problems always consider, sometimes you're rushing and you get done you say OK, my p h is 2.
But go back and think about the type of problem you're doing.
It's a base in water problem.
Would it make sense that the p h was 2 if it was a base in water problem?
No.
And so then you realize oh, I have to do another step.
So that kind of thinking can save you a lot of points on the exam, to remember what it is you're trying to calculate and go back and make sure that your answer makes sense.
And sometimes people run into weird math problems and some they'll write and say this p h should be above 7, it's 2, I don't know what I did wrong, clearly I did something wrong.
I know that's wrong but I don't have time to figure out what I did wrong.
That will get you points.
So just recognizing that if something makes sense or not tells us you know what's going on, and sometimes math issues can get you into a place that you can't get out of quickly.
So just thinking about whether the problem makes sense is a big step.
All right.
So now, we're going to talk about salt problems and I'm going to try to convince you that salts are actually the same as the weak acid and weak bases that we just did.
So a salt is formed when you mix an acid and a base together.
So, for example, if you have h c l and sodium hydroxide, you're going to get table salt and a c l and water.
So the p h of a salt in water is not always neutral.
Sometimes it's neutral, sometimes it's not neutral.
Well, when would it not be neutral?
Well, if a salt contained a conjugate acid of a weak base, then that conjugate acid is going to make it weakly acidic.
Salts that contain things like iron 3 plus also may be acidic.
So when you're drinking water and you measure the p h of that water and it's not neutral, this could be part of the reason that there's some salt in the water, some different ions in the water.
So a general rule from the periodic table group one and group two metals, so lithium, calcium, sodium, those are all going to be neutral in solution, so you can just remember that.
And if a salt contains a conjugate base of a weak acid, then it'll form a basic solution.
So it's all about whether the salt derived from a weak acid or a weak base that's going to give you a clue as whether it's an acid or a basic solution.
If it derived, say, from a strong acid mixed with a strong base, then it's going to give you a salt that's neutral.
So let's look at some examples.
So here we have n h 4 c l. So we can break this down and think about where a salt like this would have come from.
So it would have come from n h 4 plus, and it would have come from c l minus.
So n h 4 plus, let's think about where this came from.
What is it?
So, we want to ask the question is n h 4 plus a conjugate acid of a weak base?
And what is its conjugate base?
Well, it's conjugate base is n h 3.
If you lose a hydrogen ion or proton from n h 4 plus, you get n h 3.
And so you're really asking, if that's a weak base, then its conjugate is also going to be weak.
So you need to know about these guys to see what would happen if you have n h 4 plus in solution.
So how do you know about this.
Well, you know about things being weak or strong based on their k a's and their k b's.
So, is ammonia a weak base?
It has a k b of 1 .
8 times 10 to the minus 5, so yeah, that's a small number, so that's a weak base.
And so it's actually in this table, so if you have something that's weak over here, then its conjugate is also going to be weak over there.
These are totally lined up right.
And so the conjugate over here is also going to be weak.
And if you weren't really sure you could always look it up, so here you have ammonium ion and it has a k a of 5 .
6 times 10 to the minus 10.
Yup, that's a very small number, that is a weak acid.
So yes, the conjugate is weak, the base is weak, and the conjugate acid of that weak base is also weak.
So, n h 4 plus does have acidic properties.
It's not a strong acid, it's a weak acid, but it will make things acidic.
So this should be acidic.
What about c l minus?
Do you think it's going to do anything useful for you?
Where do you think c l minus came from?
From h c l. So we could ask again, is a c l minus a conjugate base of a weak acid.
The acid is h c l, is that a weak acid?
No.
So we can look it up if you didn't remember it, 10 to the 7, definitely not weak -- very, very, very strong acid, and if something is strong acid, its conjugate is ineffective as a base.
So c l minus is ineffective as a base so it's going to be neutral here.
So overall, you have something that's going to be acidic with something that's going to be neutral.
So overall it'll be acidic.
So this particular salt and water is going to be acidic because the things that were mixed together to get it, one of them included a weak base, and so that's going to form a weak conjugate acid, so it'll be acidic in solution.
All right, so let's look at another one.
I'm giving you the k a value in this problem, and knowing that particular k a value, tell me what you think is going to be true about this particular salt in solution, whether it'll be acidic, neutral, or basic.
OK, 10 more seconds.
So, some people were a little fooled by the information I gave you, so let's take a look at this.
So if we go to my presentation here.
So we can break this up into n a plus and c h 3 c o minus.
N a plus, is that a conjugate acid of a weak base?
Is that going to be acidic?
What do we know about things in group one, a column of the periodic table.
They're going to be neutral.
Where do you think that this came from, n a plus, where might it have come from?
It might have come from n a o h, that might have been a base that was added.
So it's not going to do anything for you -- things in group one and group two are going to be neutral.
All right, so we can ask the question about a c h 3 c o o minus.
Is it a conjugate base of a weak acid?
So then we can say is the acid that it came from a weak acid?
Is its conjugate a weak acid?
Well, how do we know about that?
Well, we know about that from the k a value that I gave you.
So is this a weak acid?
Yes.
So is its conjugate going to be a weak base?
Yes.
So, given that information we can say yes.
So its conjugate acid is weak, so then it would be a weak base.
So if it's a weak base then it'll be basic in solution, and we have something that's neutral plus basic, so overall you get basic.
All right.
So let's look at a general example now of this as well.
So here we can talk about a general rule.
So if you have compound x y, we can talk about x plus and y minus.
And for x plus you're going to be asking about is it a conjugate acid of a weak base.
For y minus you're going to be asking is it a conjugate base of a weak acid.
So for the first part you're asking about if it's a conjugate acid, in the second half you're asking about if it's a conjugate base.
So if something is a conjugate acid of a weak base, and that's yes -- if you know that's a weak base or you know that it's a weak acid, then it's going to be acidic, if it's not, it'll be neutral.
Same thing is true over here.
You might know about that something is a weak base, you might know that its conjugate is a weak acid, and if you have a conjugate base of something that's a weak acid, then it is a base.
If it's a strong acid it would be ineffectual, but if it's a conjugate of a weak acid, it'll be basic.
No, it's neutral.
So overall, you can have three possibilities.
Acidic plus neutral is acidic, basic plus neutral is basic, and neutral plus neutral is neutral.
Now some people might come up with another option here.
What other thing am I leaving off of this overall?
Acidic plus basic.
That's because I'm never going to ask you that when it comes to a salt.
Because pretty much, salts are formed when you're doing a titration, and you're going to be either titrating a strong acid, a strong base, you're going to be titrating a weak acid with a strong base, or you're going to be titrating a strong acid with a weak base.
You are never going to titrate a weak acid with a weak base.
That would yield no interesting results of any kind.
So you're not going to be forming salts that are conjugates of both those things.
So, if you want to think about it that way that's fine, or you could just remember that that is not what I'm going to ask you.
These are great little short answer questions on an exam, so if you're good at thinking about this, it'll definitely give you a couple of points on one of the exams.
All right, so now and last, just a couple minutes, I just want to introduce very briefly buffers.
So a buffer is something that maintains the p h of a solution, so it's going to buffer that solution.
So if you add a little bit of strong acid or a little bit of strong base, it doesn't matter the p h is going to stay the same.
So there are two kinds of buffers.
You have an acid buffer, which is going to buffer/maintain the p h on the acidic side of neutral.
And a basic buffer, which will maintain the p h on the basic end of the p h scale.
So let me just give you a brief example of a buffer and just get you thinking about buffers.
So here, in about a buffer problem you're going to mix an acid with its conjugate base.
So, you have acetate, and then probably the acetate salt of the acetic acid.
So over here, you have the acetic acid, on this side you have its conjugate base usually added in the form of a salt.
And then you have an equilibrium.
So, what's going to happen if you add a strong acid to this solution?
If you add strong acid, if you add more h 3 o plus, what happens if you add more?
What direction will the reaction shift?
So you'll get the back reaction, it'll try to minimize that stress and move the other way, and it'll use up some of that acid and maintain the p h. Then you can think, so these amount of acid added is effectively removed and the p h stays the same.
What about if you add a strong base?
Well, that strong base will react with the acid, it will remove protons from this acid, or the hydrogen ion here, forming this water and its conjugate.
So you'll make more of these and the p h will also stay the same.
So the base is going to be removed by reacting.
So they're effectively removed and the p h stays the same.
So in this you have a weak acid, h a, it'll transfer protons to o h minus supplied by the strong base.
The conjugate of that weak acid, which would be a weak base, is going to accept protons from any acid that is going to be supplied.
So, in this way, you maintain the p h. And so I just to emphasize that in a buffer solution you have h a, acidic buffer solution, you have h a, and you have its conjugate.
And if you only have one or the other, it's not going to make a good buffer.
So, I want you to remember that in buffers you have both conjugates -- one alone is not going to work.
And people forget this in the class, and so you can remember that for Halloween, your chemistry professor dressed up as a buffer to help you remember that in a buffer you're going to have both.
You've got to have the conjugate acid base set, otherwise it will not buffer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so we're going to leave off where we were last time talking about buffers.
And I'm a big fan of buffers, actually, and buffers are extremely useful in my research because they keep the p h of things constant.
So if you're going to do any kind of biology research or biochemistry research, you need to know about buffers.
And as I'll mentioned later, also your body needs to be buffered appropriately, you don't want a large changes in p h in your body or things won't function properly.
So buffering is very important.
So let's go over buffers.
So if we talk about an acid buffer, that's something that's buffering on the acidic side of the p h range.
And in that, you want to have play on both sides.
So if you have a strong acid, it can be neutralized, the p h stays the same.
If you had a strong base, that would be neutralized so the p h stays the same.
So what happens, you have to have an acid and its conjugate base in the mixture to have a good buffer.
So what the weak acid is going to do is it will transfer protons or hydrogen ions to hydroxide ions that are supplied by the strong base to neutralize that strong base that was added.
So you need to have an acid in there that's capable of giving up a proton to neutralize the hydroxide ion that was added.
You also need to have a conjugate base.
The conjugate base is going to accept protons if an acid is added, thereby neutralizing the effect of that added acid.
So again, you need a weak acid and its conjugate base, you need both to have a good buffer.
So I've been talking about weak acids and weak conjugate bases, so why would a strong acid and the salt of its conjugate base not make a good buffer?
What do strong acids do?
They dissociate almost completely, and that's not going to do you any good.
You want to be able to shift that equation both directions.
So a strong acid goes to completion is going to drive it to the right, and for a good buffer you want to be able to switch things around that if you add a strong acid, then you neutralize that, you add a strong base, you neutralize it.
So you need the conjugate to be effective as a base, and a strong acid has an ineffective conjugate base.
It's not good as a base at all, so that won't work.
So buffers are made up of weak acids and their conjugate bases, or weak bases and their conjugate acids.
They need to be in the weak range or there's no buffering capacity.
So let's look at a base buffer example.
The only difference here is that a basic buffer is going to buffer on the basic end of the p h range.
So here's an equation.
We have a base in water forming a conjugate of that weak base and hydroxide ions.
So let's think about what happens if a strong acid is added.
Well if a strong acid is added, then this base, n h 3, can accept protons from the incoming acid and make more n h 4 plus, thereby removing the strong acid from the solution and neutralizing the p h. If on the other hand a strong base is added, n h 4 plus or ammonium ions can donate the proton forming its conjugate base in water, and again, the p h remains about the same.
So the base and its conjugate acid can both react, neutralizing the p h. So in the base buffer acid, a weak base, b, will accept protons supplied by the strong acid, removing a strong acid from the solution.
The conjugate acid of that weak base, which again, would be a weak acid, can transfer protons to the hydroxide ions added by the strong base, again, neutralizing the effect there.
So a buffer is a mixture of a weak acid base conjugate pair that will stabilize the p h serving as a source or a sink for the added protons.
So that's how a buffer works.
So, I mentioned buffering is very important.
It's very important in your blood.
Your blood is buffered in the range of p h 7 .
35 to 7 .
45, and you have a buffering pair and a conjugate acid base pair that's in your blood that does to work.
So nature has its own buffering system design to keep the p h of your blood constant.
Well what happens if the p h of your blood changes?
So let me just tell you about one disease that can change the p h of your blood.
This is from a vitamin B12 dependent enzyme called methylmalonic coa mutase, and this enzyme is your friend.
I think that most of you are excited about the idea of having all your fat go to energy, so you should be all fan of this enzyme.
So it breaks down from fatty acids -- you get methylmalonyl coa, which is converted to succinyl coa and goes into the citric acid cycle.
But if this step is blocked, if there's some kind of genetic disease associated with that particular enzyme, you get a condition called methylmalonic aciduria.
So this is excreted in the body, it can't be converted to the succinyl coa, so it has to be excreted from the body, and it has an acidic p h. And so when it's excreted, it will actually, there's a large amount of it, and it can change the p h of blood.
So even though you have this buffering system, it can overwhelm the buffering system and cause a change in the p h, which can lead to neurological disorders and sometimes death.
So they can look for this in newborns, they can do a test, and I was -- one of the tests is -- it's not in every state, but it is in Massachusetts, so my daughter, Samantha, had this test.
But they can simply look at the urine of a newborn and look at the p h and whether any of this is being excreted.
And some infants that have this condition can excrete like a gram of this acid a day.
So it's really large quantities of acid that are excreted and it goes beyond the buffering capacity.
So, I'm going to tell you a little story and it sort of emphasizes some of the importance of genetic research.
So before the human genome project, scientists were trying to find the gene to figure out what was wrong with these patients that had this condition.
And here are all the amino acids translated from the gene.
This is not a small protein, that's a lot of amino acids.
And they found that the problem was right here, or one of the main problems is right here.
There's a glycine residue, which glycine is the smallest amino acid and that had been changed to something else and that led to the disease.
Now I want you to imagine if you were parents of a small child and you found out your child had this condition and then you had heard that there were advances, they now know what caused it, and so, they went to the doctor and the doctor said, yes, your child, the DNA is wrong, and so let's substitute a different amino acid for glycine here.
OK. What does that do for you?
Like why is that a problem?
Well, they didn't know why it was a problem, they just knew that it had the wrong amino acid in there.
And so, the next step was to try to understand what was actually going on.
So what happened in this case was that the vitamin B12 cofactor had changed its conformation when it bound to the protein.
And instead of having this sort of loop structure here, it had more of a tail.
So this base here had moved down, and so there was a different shape to the vitamin than had been expected.
So when the first structure was solved of one of these enzymes, it found that here's kind of what the vitamin B12 looked like, it had this extended region which no one was expecting, and so that had to fit into the protein.
So this is the domain of the protein that binds the vitamin, and it had a large hole, and you put these together and that's great, it fits and you have happy enzyme.
Well, how do you create a hole -- nature supposedly abhors a vacuum, so how do you have this hole?
Well, what nature did was it put the smallest amino acids there are, glycines, right along here to make room for this to fit together.
So, what was happening in this genetic case is that the switch of a glycine to an argenine did this to the hole.
So, what people were trying to do is treat these patients with extra B12, you give them injections of it, you have a lot more B12, and you want to the B12 to bind and get the activity back going again.
But really, no matter how much vitamin B12 you added to it, it wasn't really going to do much good, it just wasn't going to fit.
So instead, there was a suggestion that instead of giving the whole vitamin, perhaps you could just give a truncated version of the vitamin, which was commercially available, and that that might bind and restore activity of the protein.
So this is what was suggested once you knew what the problem actually was.
So this is just an example of a genetic disease, and it's not that common, but as I said, it is tested for here in Massachusetts.
And how knowing something about what the problem is can lead to a potential solution.
But for most of you, your blood is doing just fine, you don't have a genetic condition which is pumping too much acid into your bloodstream, so that nature's buffering capacity is working quite well.
So buffers are important.
It's great to have my important statements emphasized up there.
We've been rehearsing at home.
OK.
So let's do a sample buffer problem.
All right, suppose we have an acid, 0.1 moles of an acid, and 0.5 moles of its conjugate base added in the form of a salt.
And so they're put into water and diluted to 1.0 liter.
And you're given information about the k a of the acid, 1 .
77 times 10 to the minus 4, and you're asked to calculate the p h. So what do you do?
All right, first, it's always good to write the equation that you're talking about.
And so you have acid in water forming hydronium ions and the conjugate base.
So here, the acid is giving up a proton to the water, forming h 3 o plus and it's conjugate base.
Now we want to think about what happens at equilibrium.
So we know how many moles we have and we know what the total volume is.
And in this table we should be using molarity, but the math here is pretty easy because we have 1 mole and 1 liter or 1, and 0.5 moles in 1 liter or 0.5 for molarity.
And now at equilibrium what's going to happen, well, when this equilibrates some of this will go away and more of these are going to be formed.
So we have minus x over here, plus x and plus x over there, and we add it all together you get 1 minus x plus x and 0.5 plus x.
So the important thing to remember is in buffering problems, you have the acid and the conjugate.
So we're used to seeing these tables where we only have something over here and it's all zero on the other side.
But in a buffering problem, you have things on both sides from the very beginning.
And that's a really important point, and that alone, if you can remember that, will get you a long way through this unit.
So you remember that in a buffer you got to have both an acid in its conjugate base or a base in its conjugate acid.
So here we're talking about an acid in water, it's an acidic buffer, and so we can use our k a value -- we want to know what the concentrations are at equilibrium to calculate our p h. So we can use our k a and we can set it up, so we have on the top products over reactants, and we we're not including water in this because it's dilute in solution and so it's not changing very much.
And now we can plug in our values, so we have 0.5 plus x, x over 1 minus x.
All right, now this is just written again from the last slide, we can try an approximation, which is it that x is small compared to 1 molar or 0.5 molar.
And so we can get rid of the plus x and the minus x down here and just have one x term, which makes it simpler to solve, but we'll have to go back and check that approximation in a few minutes.
So making that approximation, we can calculate x as 3 .
54 times 10 to the minus 4 molar, and now we need to check that assumption.
Well, you can probably guess it's going to be OK, because something times 10 to the minus 4 compared to 0.5 and 1, that's probably going to be OK that it's small.
But our assumption here is that it needs to be less than 5%, and here it's 0.1%.
So you could just take this value of x, divide it by the smaller of the two, 0.5 , times 100, and calculate the percent ionization.
If it's less than 5% you're OK, if it's more you need to use the quadratic equation to solve the problem.
So x here is our concentration of hydronium ion, which is great, because then we can easily calculate the p h. So p h again is minus log of the hydronium ion concentration, and so the p h here is 3 .
45.
And again, significant figures, we'll be talking about this as we go through.
The volume had two, it was 1 .
0 liters, and so we're going to have two significant figures after the decimal point, because the volume was limiting insignificant figures.
All right, now let's consider what happens if some strong acid had been added in the solution.
So the volume is still going to be 1 liter, because the acid was added in before we went up to the 1 liter mark.
So strong acids, they go to completion.
So, if we added 0.1 moles of that strong acid, it will react with equal number of moles of the conjugate base to form the conjugate acid.
So we can just do subtractions here.
We don't have to worry about setting up any kind of equilibrium, we assume it goes completely.
So for the conjugate base we had 0.5 moles before, and it's going to be reacting then with a strong acid, so 0.1 of those will react, leaving us with 0.4 moles of the conjugate, and that's in 1 liter.
So now we have 0.4 molar.
For the acid, we had 1 mole to begin with, but now we formed more as they reaction has taken place, and so we have 0.1 more.
Now we have 1 .
1 moles again in 1 liter.
So now we can do the same thing again.
So after this has happened, after we have added the strong acid, then a new equilibrium is going to be reached.
And so we can plug in this table as well.
So we have 1 .
1 now over here, and 0.4 on the other side.
As equilibrium is approached, you lose some of the acid as it reacts with water and ionizes and you form more of the h 3 o plus and more of the conjugate.
So we have 1 .
1 minus x, x, and 0.4 plus x.
So again, the trick here is just to remember that there is a reaction with a strong acid that has been added, and you need to figure out the new molarities, and then go back to your equilibrium table with those new molarities.
So here we can set up with a k a again, and do the same problem that we did before.
And so, we can make the assumption again that x is small, try it out and see if it works.
X turns out to be 4 .
87 times 10 to the minus 4 molars -- so again, it's a small number.
And so, it turns out to be 1 .
-- less than 1% of 0.4 .
If it's less than 5% of the smaller number, you don't have to worry about the bigger number, and so your assumption is OK, you don't need to use the quadratic equation.
And so, then we can calculate at the end that the p h is now 3 .
31, again, two significant figures after the decimal place because of the volume.
So addition of 0.1 moles of a strong acid changed our p h from 3 .
45 to 3 .
31.
So we buffered pretty well.
There was a change in p h, we added acid and the p h did go down, but not by an enormous amount.
And so, that's because this turned out to be a pretty decent buffer here, so we didn't have a really huge effect.
So, those are -- that's an example of a buffer problem.
So, let's think for a minute about designing a buffer.
Suppose you wanted to create a buffer at a particular p h, what do you need to think about?
Well, you need to think about the ratio of your acid to the conjugate, and you need to think about the p k a of the acid, and the p h that's desired.
So here is a generic equation for an acid in water, so we have our acid, h a, in water forming hydronium ions and our conjugate base.
And now we're going to do a little derivation to come up with an equation that would be useful to you in thinking about buffers and designing buffers.
So, we can write a generic term for k a. K a equals products over reactants, and so we have our hydronium ions are conjugate base over our acid.
And now we can take this term and rearrange it.
So we can pull a hydronium ion concentration over to one side.
And now we're going to take the logs of both sides, so we get the log of hydronium ion concentration, the log of k a, and the log of h a concentration over a minus concentration, and now we're going to multiply everything by negative sign, so we have minus logs of things.
And I'm just going to move this now up to the top of the screen, and we have this equation.
So what is minus log of hydronium ion concentration?
P h. What's minus log of k a?
P k a.
So, we have p h equals p k a minus the log of a concentration of your acid over the concentration of your conjugate base.
And these are equilibrium concentrations for that, because remember, we derived it from our equilibrium expression for k a.
But a lot of times when you're working buffer problems, you know the concentrations that you added of these things, not necessarily the concentrations of equilibrium.
And if we rewrite this expression in terms of the initial concentrations or original or o concentrations, then we have p h is approximately equal to p k a minus log of the initial concentrations of your acid over your conjugate base.
And this is known as a Henderson Hasselbalch equation.
And people love the Henderson Hasselbalch equation, and it can be incredibly useful to you in solving these problems.
But I'm going to emphasize when it's OK to use it and when it's not OK to use it, because people try to apply it to everything, and it's for buffer problems, and people apply it to all sorts of things that are not buffer problems.
So I want to make sure that it's clear when you can use it and when you can't use it.
And it's fine to use it and you should use it when it's acceptable, but not at other times.
It's one of the few equations in acid base, so people get very excited by it.
All right, so remember that it's really equal when they're at equilibrium concentrations, and this is just an approximation that we're saying that those are initial concentrations.
So when is it OK to say well, the equilibrium concentration is more or less the same as the initial concentration.
And that's true when x, which is in this particular example your hydronium ion concentration, is small compared to your initial concentration of the acid and the conjugate base that you added to the buffer.
So, and when x is small, then pretty much, the equilibrium concentration equals the initial concentration.
So there's not really very much change if x is a really small number.
So, a lot of the time this is true.
Remember, we're making buffers from weak acids and weak conjugate bases, and the definition of a weak acid is that it only loses a tiny fraction of its protons.
It only ionizes slightly in water.
And a weak base typically only accepts a fraction of the protons it can accept.
It's a weak base, so it's not doing a whole lot of chemistry there.
So this approximation is good a lot of the time.
So just you know when you can and can not use it, we're going to give a rule and say it's the same rule as before.
When we say when x is small or your hydronium ion concentration is small compared to the acid or the conjugate base, when it's less than 5%, that's what we're going to call small, so the same rule we've been using all along.
When x is small, this approximation works pretty well.
And you can plug in your concentrations right there -- your initial concentrations not your equilibrium concentrations.
All right, so let's use Henderson Hasselbalch and design a buffer system if we want a p h of 4 .
6.
And a good rule to remember is that a buffering solution is most effective when it's plus or minus 1 away from the p k a.
So that's a good thing too I keep in mind for research as well.
If you're really far away from the p k a, it's not going to be a good buffering system, and this is a mistake the people make in the lab sometimes, so you may run into that in some of your laboratory courses.
So we want a buffer about p h 4 .
6.
So we can look at our ionization constants of acid, and of course, we're always doing everything at room temperature in this unit, and so we can look at what p k a's are of some weak acids, and acetate, acetate is an easy buffer to get your hands on, acetate is quite available, and it has a good p k a 4 .
75.
So that's great, we can use that in designing our buffer.
So, acetic acid has the right p k a, and we're all set, we can prepare a buffer.
But we'll need to add the acid and the conjugate, and we need to know how much acid and how much of the conjugate to add.
So we can use Henderson Hasselbalch equation for this.
We're creating a buffer, and so this is an equation we can use for buffers.
We know what p h we want, we know the p k a of the thing we're going to use, and we want to figure out how much of the acid to use and how much of the conjugate to use.
We need to know the ratio of one to the other to set up our buffer ideally.
So we can plug our numbers in and calculate that.
So, we can rearrange this if you want so that the unknown is on one side and we have to p k a and the p h on the other side.
And so, we have here our p k a minus our p h and we get 0 .
15.
Now we need to inverse log and come up with the ratio, and the ratio that we want is 1 .
4.
So we want to have a ratio from our acid to the conjugate of 1 .
4.
Well how much exactly are we going to add them, we know the ratio now, how much are we going to add?
Well, you could use 1 .
4 molar and 1 molar, for example, that would have the correct ratio.
But how do you know if that's going to be good?
Well, the ratio is more important than the exact amounts, however, the amounts get to this issue of sort of the capacity of the buffer -- how resistant it is to changes in the p h. So if you use too low concentrations of both, it won't be all that resistant.
So if you need to have a better buffering capacity, the higher concentrations that you should use.
And you can use Henderson Hasselbalch then to calculate what sort of the minimum amount you use, the minimum amount to have that 5% rule work.
So in the example I told you about this condition, the buffer in the blood was pretty good, but the acid with so much that it overwhelmed buffering capacity.
So if there had been larger concentrations of the buffer, that would have certainly helped in that case.
So the higher the concentrations you use, the more resistant to change.
And that's the idea of buffering capacity.
So, and if you use too low concentrations, then your Henderson Hasselbalch equation won't even be valid.
So we can go back and calculate what is sort of the minimum concentrations we need to use for Henderson Hasselbalch to be valid to meet that 5% rule.
So for a p h of 4 .
6, a hydronium ion concentration is a 2 .
5 times 10 to the minus 5.
And so then if we work our equation backwards, we want to be less than 5%, so we have this over the concentration of either one of those times 100% gives you 5%.
So the concentrations need to be greater than 5 times 10 to the minus 4 moles or you won't meet this 5% rule.
So at least to be greater than that and they need to be in the right ratio.
Other than that you're somewhat free to decide what you're going to do with that, and there may be other considerations involved if you're working with a certain concentration of protein, you might not want your buffer concentration to be too high, it might start interfering with enzyme assay, for example.
And often buffers are somewhere around 100 millimolar, that's a pretty common concentration for buffers.
All right, so this slide we're doing really well.
So we talked about weak acids in water, weak bases in water, and I'm trying to convince you that these are the same as the salt and water problems.
So we're going to talk much more about salt and water in a few minutes.
And I told you about buffers, so we only have 2 more things to go.
We need to talk about strong acids in water, and strong bases in water, and then you'll have all of the 5 types of problems to do the problem-set.
And as I mentioned last time, it looks deceivingly short, the number of questions on them, but the titration problems are long.
So don't leave those to the last minute or there will not be a whole lot of sleep involved.
So just a word of warning from the past.
Everyone looks and go, "Oh, this is an easy problem-set, there aren't many problems."
So, titration problems do take a lot of time.
So let's talk about titration, so that as soon as class is over today, you can go work on those questions on the problem-set.
All right.
So acid base titrations.
How many of you in high school titrated an acid with a base?
Large fraction of you, OK.
So usually what titration problems are meant to do, that you have something known about an acid or a base, base say of known concentration, and and acid of unknown concentration or maybe unknown molecular weight, and you're titrating them out, titrating them together to find missing information.
So you can determine concentration, sometimes you can determine a molecular weight.
All right, so here's what a plot looks like and some of the key terms involved in acid base titrations.
So you have a p h on one side, and then volume of either base or acid added on the other side.
So p h versus a volume.
And the actual experiment here, you have a base that you're dripping, one drip at a time supposedly into your strong acid.
And then you're looking to figure out when you're going to reach the either equivalence point or the end point, these are basically the same terms.
It's also called the stoichiometric point, and often equivalence point is used as a more theoretical amount of volume is added, where as end point is the experimentally measured.
So we're going to be talking a lot about equivalence points, because we have no lab associated with this course, so everything will be theorectical in terms of these, but they should be the same.
So, many times you will measure p h with the p h meter -- this just shows a p h meter.
And for those of you who have done these experiments, this should look familiar.
And for those of you who haven't, often what you're doing when you're dripping something in, you're waiting for an indicator dye to change color as an indication that you have reached the end point, and often you're adding, and it's clear, clear, clear for what seems to be forever, and just as you get incredibly impatient, you start adding it faster and you go all the way to this dark color.
And what you want is this very, very, very light changed color that indicates the end point.
So you usually go too slow and then go too fast and then have to do the experiment over again.
Those of you who know how to calculate theoretical values could sit down and do a calculation first and then go really fast right until you get around this point and add it really slowly, and be done with lab sooner.
So, we'll talk about how you do the theoretical value, which could save you a lot of time if you ever run across these labs in any future class.
So, here are the two curves that you would see for either a strong acid with a strong base, or strong base with a strong acid.
So if you're titrating a strong acid with a strong base, you're going to start down being very acidic, because all you have is your strong acid.
And then as you add in base, the p h will go up.
You'll reach this point -- s is for the stoichiometric point, we also call this, again, the equivalence point, and then it'll continue to go up and then start to level off a bit more up here.
If you're going the other direction, you're starting with strong base, your p h is going to be high, and the p h will decrease as you add the acid.
And then you get to the stoichiometric or equivalence point in the middle of this curve here.
And then as you go down farther, more acidic, it starts to level off.
So those are what the titration curves look like.
So let's do an example.
We're going to have a strong base being titrated with a strong acid first.
And our strong base is n a o h, and our strong acid is h c l. So let's calculate the p h at the equivalence point.
Well, let's just calculate the p h at 5 mils -- I don't know whether it's equivalence point, actually.
5 mils of this have been added to the amount of base.
So we have 25 mils of our strong base at 0.15 molar concentration, and we're adding 5 mils of our 0.34 molar acid.
So what we want to do is figure out how many moles of base we had to start with.
So how many moles of o h minus -- again, n a o h is a strong base, so however much n a o h you add, is the amount of o h minus you get, and so we just put in the number of liters times the concentration to get moles.
We don't need any equilibrium table here.
So then we want to figure out the amount of acid added when you're adding 5 mils of it.
So again, h c l is a strong acid, so the amount of h c l added equals the amount of h 3 o plus, that's formed, goes to completion, and so we added 5 mils, the concentration was 0.34 molar, so we have 1 .
7 times 10 to the minus 3 moles.
So now, the strong acid is going to react with the strong base, and we want to figure out how much of the base is left, how much hydroxide ion is left after it reacts with the hydronium ions.
So it'll react 1:1, so we had 6 .
25 times 10 to the minus 3 moles, and we added 1 .
7 times 10 to the minus 3 moles of the acid, so we're going to have 4 .
55 times 10 the minus 3 moles left.
And now we can calculate the molarity here, so we have the number of moles and the new volume, so we had 25 mils to the base to begin with.
We've added 5 mils to the acid, so our new volume is 30, and so we have our new concentration.
And now we can calculate p h. First we'll calculate p o h, and then use 14 minus to get the new p h. So if you were making your own little titration curve, we could have volume of acid on one side and p h on the other side, and you have a point of 13 .
18.
That's how much we have after you have 5 mils added.
So we didn't, in this curve, we have not measured the zero point, but we know what one point after 5 mils of the acid has been added.
So now let's go right to the equivalence point -- so the way I draw it should indicate that was not an equivalence point.
So we want to figure out how much we need to add of the acid to reach the equivalence point.
So the equivalence point or the stoichiometric point means that you add the same amount of moles of your titrate as you had, and so they're going to be equal to each other.
So if you're adding acid to a strong base, you've added enough acid that you now have equal number of moles to the number of moles of base that you had to begin with.
So we know we have 6 .
25 times 10 to the minus 3 moles of the base were present.
So at the equivalence point, you need to have that exact same number of moles of acid.
So we know the concentration of the acid we have and the number of moles we need, and so we can calculate the volume, which is 18 .
4 millimeters, or 0.0184 liters.
So what would the p h then be of the equivalence point?
We're titrating a strong acid with a strong base, what should the p h be?
The p h should be 7.
So somewhere around here we're going to be at 18, so you can just try to draw a curve, and here we have a p h of 7 when we've gone to about 18 .
4 milliliters added.
So when you titrate a strong acid with a strong base, you form a salt that's neutral.
Because the conjugates of a strong acid and a strong base are not basic or acidic.
They're ineffective as acids or bases, so you're going to get a neutral salt.
So if you're doing a problem with a strong acid and strong base, it's not a lot of calculations that need to be done at this point, you just need to recognize that your salt should be neutral.
So now, we can talk about what happens if you add an extra milliliter of your acid, after you've reached that equivalence point, which is something that people probably have done in doing these titrations not meaning to, gone well beyond the equivalence point.
So first we can find out the number of moles of the strong acid that we've added extra, so we added one mil of the strong acid, and so again, it goes to completion.
The concentration, the acid, is 0.34 molar times our 1 mil, so we've added 3 .
4 times 10 to the minus 4 extra moles of our acid, and so let's calculate then what the molarity is that we have added that was extra.
And even if you don't have a calculator, we've helped you out.
Let's just take 10 more seconds.
I think this is a first, isn't it?
Did I mention it's a good idea to start the problem-set early.
So the trick here was about the volume.
So we had 25 mils to begin with, then we added 18 .
4 to get to the equivalence point, and then we're 1 mil beyond the equivalence point.
And the tricks to these parts of the problems are to remember all the things that have been added to get to the volume.
So the problem itself is not very tricky of having just an acid in water, but the one trick is in calculating the molarity, remembering all of the things that were added to get to this point.
And so by the end of the problem-set this should be pretty familiar to you, but it's also something that you want to make sure that you check on an exam that you're not making any volume mistakes.
So again, here are the things that you want to remember to add in in calculating this.
And then, if you calculate the p h of that, you get a p h of 2 .
1106, so that's down somewhere in here.
We've added 1 more mill, and we're at a p h of 2 .
1106, and this can be a check as well.
If you forgot to add some of your volume, you might get a p h that doesn't make a lot of sense.
So that can be a double check.
Always ask yourself when you have a strong base and you're adding acid, before you've added very much, your p h should be basic.
At the equivalence point of a strong acid, strong base titration, it'll be 7.
And if you continue to add acid, then you should have a pretty low p h, you'll notice the curve drops off pretty fast.
So it should be a dramatic change in p h by adding extra of your acid.
So always check your work.
OK, so let's stop there, and we're going to weak acid, weak base titrations next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Please settle down and take a look at this question.
OK, let's take 10 seconds.
I think that it's a simple math mistake is between one and two at least.
So, the trick here is you know the p h and the p k a and you want to find the ratio so you can subtract and do the log.
So maybe we'll have this question later or something similar and we can try this one again.
So we're going to talk about buffers again today.
I just feel the need to take a moment and reflect on the historic events of the last 24 hours, and talk about how it will affect chemistry.
So some of you may have voted for the first time.
Some of you may have worked on a campaign for the first time.
Some of you may have been very active in a campaign for the first time, either for Obama or McCain, that you got involved.
And I thought just to put this election in a little bit of the historic perspective in terms of about being an undergraduate student or a student and working on a political campaign or being part of a political movement.
So, my father was very active as a political student activist.
But the difference between some of you and my father was that he was a political activist at the University of Hamberg in Germany in the 1930's in Hitler's Germany.
So he was the leader of the left wing student organization.
That was something that put one's life at risk to take on that role at that time.
So, things were heating up a little bit and the gestapo were discussing some of the activities with the left wing student leaders at college campuses in Germany.
And some of them, after the discussions, no one knew where they went, they seemed to disappear.
Now my father was very concerned about this and he decided to lay low for a while, and so he thought I'll do a semester at another university.
And he told his parents that if the gestapo came looking for him, that they should send him a telegram saying "Your Aunt Millie is sick."
Since he did not have an Aunt Milly, he knew that that would mean get out now.
So he went to another university and he was doing a semester there, and someone he knew told him you really need to go into hiding.
But he didn't really trust this person, so we packed a bag with a few clothes and some toiletries, but he didn't actually leave.
Then the next day he came home and there was a telegram under his door.
So, you can guess what the telegram said.
He grabbed the bag that was already packed and headed down the stairs.
The gestapo was coming up the stairs.
My father's name was Heinz Leopold Lushinski and the gestapo said to him, "Do you know Herr Lushinski?
And my father said, "Yes, of course, he lives on the top floor."
The gestapo went up, my father went down, and he didn't go back to Germany for 30 years.
So he came to the United States as a political refugee and became a citizen.
He voted in every election, every possibility, he was very, very active.
My family was very, very active in politics.
He gave money every year to the American Civil Liberties Union to protect civil liberties, and he also gave money to the American Rifle Association.
He always liked to have a plan b.
So, it was sometimes a little humbling to be the only child of this man.
He was in his 50's when I was born, and I thought how can I live up to something like this?
Am I ever going to risk my life for what I believe in.
If given that choice would I do the right thing?
And I don't know if I'll ever get an answer to that question, but I talked to my father about this and he said all I need to do is work hard, find something that I love doing, some way that I can contribute, and that's what's really important -- contributing is really important.
So I was drawn to teaching, and I love teaching here at MIT because you all are so talented and smart, and it is really an honor and a privilege to be involved in your education.
But I feel that in the last 24 hours, we have all received an additional call to service.
That president elect Obama said in the campaign that his top priorities are going to be scientific research, coming up with clean energy technologies, and improving healthcare.
He called to scientists and engineers.
And last night the American people said yes, we like that vision, and they elected him president.
So we have been called, you have been called, he has reached out to students and said, students of science and engineering, you need to contribute.
And it's been a while since any president has really called to action, scientists and engineers.
And last time that happened, a man went on the moon.
So let's see what we can do this time.
The next challenge is clean energy, healthcare.
It's going to be really important for sciences and engineers to get involved, and at the core an energy technologies, and at the core of medicine is chemistry.
So you are in the right place right now.
You are going to be the generation that needs to solve these problems, because if you don't solve the energy problem and don't come up with clean alternatives, there isn't going to be much of a planet left for another generation to try to solve those problems.
So it's going to be your job and your job is starting right now with the education that you can get at MIT.
So, it's actually somewhat interesting that today, the day after this election, we are going to talk about one of the units that students in this class have had the most difficulty with over the years, acid based titrations.
This has been the undoing of some chemistry individuals.
It has been the undoing of some grades of A.
It has been the undoing, perhaps, of some interest in chemistry.
But I would like to say today, at this moment, it will not be your undoing, it will be your triumph.
Every year I challenge students to do the best job on acid based titration ever, and people have been doing well.
This might be the last time I teach in the fall.
You have actually had the highest grades so far in this class, in the history of the class that I know of, and so this is the challenge.
So right after this election, your challenge is to conquer chemistry starting one acid and one base at a time.
So, ready to do some acid based titrations?
Who are the naysayers in this crowd?
Just a few people up there.
All right.
I have to tell you that what I'm going to tell you about acid based titrations will seem like it makes pretty good sense as I'm saying it.
But often, people inform me that when they actually go to work the problems on the test, it's a little less clear on what they're supposed to be doing.
So the key to acid based titrations is really to work problems.
And so we have, for your benefit, assigned problems for the problem-set due Friday.
And so after today, you should be set to do all of the problems on the problem-set.
And in terms of acid based titration, you will need a lot of this knowledge again in organic chemistry, biochemistry, if you go to medical school -- I used to TA medical students, they didn't know how to do this.
And I said "Who taught you freshmen chemistry?"
So it's good to learn to this now here today, work problems, take the next test, and guaranteed it'll be on the final again.
So you'll learn it now, you'll get lots of points, both on the final and the third exam.
All right, so acid based titrations, they're not that hard, but there are not a lot of equations to use, and I think that people in chemistry are used to what equation do I use.
No, it's really about thinking about what's going on in the problem, and as the problem proceeds, as more, say, strong base is added, the problem changes.
So it's figuring out where you are in the titration and knowing what sort of steps to apply.
So here are some titration curves, and one thing you may be asked to do is draw a titration curve, so you should be familiar with what they look like.
So we talked last time about strong acids and strong bases.
So if you have a strong base, you're going to have a basic p h, and then as you add the strong acid, you will go to the equivalence point, equivalence point when you've added the same amount of moles of acid as there is base or base as there is acid, equal number of moles.
And when you mix a strong acid in a strong base, you form a salt, and the salt is neutral in p h, because the conjugate of a strong acid or a strong base, is ineffectual, it doesn't affect the p h, it's neutral.
So we have p h 7, and then you continue to add, in this case, a more strong acid, and the p h goes down.
So for the other titration it's pretty much the same, except you start at acidic p h's, go up to neutral p h, and then go basic.
So we talked about these last time and we worked a couple of problems, but now we're going to move into the slightly more difficult type of problem, which has to do with when you have a weak acid or a weak base being titrated.
So let's look at the difference of the curve to start off with.
So here we have the strong acid and the strong base, and here we have a weak acid and a strong base.
One thing you may notice right off is that the equivalence point has a different p h. So, a strong acid and strong base again, mix, you form a salt that's neutral, p h 7.
But if you're titrating a weak acid in a strong base, the conjugate of the strong base will be ineffective, but the conjugate of the weak acid will act as a base.
So the p h then, at the equivalence point, when you've added equal number of moles of your strong base as you had weak acid, then you'll have the conjugate base around, and the p h will be greater than 7.
So in working the problems, if you get an answer with this type of titration problem that's different than that for p h at the equivalence point, you're going to know that you did something wrong, you need to go back and check your math.
Another big difference has to do with the curve shape down here, and so you notice a difference over here than over there.
And in a titration that involves a weak acid in a strong base, you have a part of the curve that's known as a buffering region, and the p h is fairly flat in this buffering region as shown down here.
So that's in contrast, there's no such buffering region on this side.
So here the p h will go up, it'll level off, and then go up again.
And this, for some of you, is probably the frustration in doing acid based titrations in lab, because you're adding and nothing's happening and nothing's happening and nothing's happening, and you're in this region, then all of a sudden you add just a little more and you're up here.
So notice how steep that is over here.
So sometimes when you're in the buffering region, it seems like you're never going to reach the end of the titration and then it'll happen all too quickly.
So buffering region, remember a buffer is something that has a conjugate, weak acid and weak base pair, and then in a buffering region, the p h pretty much stays fairly constant in that region.
It acts as a buffer, neutralizing the p h, maintaining the p h by being a source or sink of protons, and so here the p h then is staying constant in that buffering region.
So those are some of the differences between the type of curves.
Another point that I will mention or term I will mention that has to do with weak acid in strong base or a weak base in strong acid is this 1/2 equivalence point concept.
So 1/2 equivalence point you've added 1/2 of the amount of strong base that you need to get to the equivalence point, and that's right in the middle of that buffering region.
So that's another point where you'll be asked to calculate the p h. So now let's look at different points in a titration.
So, first let's walk through and just think about what is happening.
So when we start in this titration of a weak acid in a strong base, before we've added any of the strong base, all we have is a weak acid.
So it is a weak acid in water type problem.
And so here I've drawn our acid, and the acid has its proton, which is going to give up when you start doing the titration.
So that's what we have at zero volume.
Then we start adding our strong base, and the strong base is going to react with the acid, one-to-one stoichiometry, it's a strong base.
It'll pull off protons off the same number of moles of the strong acid as the number of moles of the strong base that were added.
And so then, you'll start to have a mixture of your conjugates, your weak acid and your conjugate base.
So the base is a minus here.
And so if you have a mixture of a weak acid in its conjugate base, that's a buffer, and so you'll move in to the buffering region here.
So that's at any volume that is greater than zero and less than the equivalence point is going to be around in that region.
Then we have a special category of the buffering region, which is when you've added the volume to get to the 1/2 equivalence point.
And when you've done that, you will have converted 1/2 of the weak acid it its conjugate base, so you'll have equal number of moles of your weak acid as moles of the conjugate base -- 1/2 has been converted.
And so that's a special category right there.
Then you get to the equivalence point.
At the equivalence point, you've added the same number of moles of strong base as the number of moles of weak acid you have, so you've converted all of your weak acid to it's conjugate base.
So all you have is conjugate base now, and so that's controlling the p h, so the p h should be greater than 7.
So that's a weak base in water problem.
And if you keep going, then you're going to end up with a strong base in water problem.
The weak base will still be around, but it will be negligibly affecting the p h compared to the fact that you're dumping strong acid into your titration.
And so that's this part of the curve.
So you see that in one type of problem, one titration problem, you actually have a lot of sub problems, or sub types of problems, you'll have weak acid buffer, special category of buffer, a conjugate base or a salt issue, and then a strong base.
And this is one of the things that people have trouble with in the titrations, because we may not ask you to do all the points, we may just sort of jump in somewhere, and say okay, what is the p h at the equivalenced point, and you need to think about what's happened to get to the equivalence point.
Or we may jump in and ask you about a region that would be in the buffering region, and you have to remember that at that point you should have some of the weak acid and also some of the conjugate bases being formed.
So, it seems like there are a lot of different things, but there are only five types of problems.
But in a titration curve, you run into a lot of those different types at different points in the problem.
So now let's go the other direction and consider titration of a weak base with a strong acid.
So here's what that curve would look like.
You're going to start basic, of course, because you're starting with a weak base, you haven't added any strong acid yet.
As you add strong acid, the p h will decrease.
Because it is a weak base, you will be forming some of its conjugate as you add the strong acid, and so you'll go through a buffering region again where the curve would be flat, where the p h will be pretty much the same for region of time.
Then the curve will drop again and you'll get to the equivalence point.
At the equivalence point, you've added the same amount of moles of strong acid as you had weak base, so all of your weak base is converted to its conjugate acid, and so you should be acidic at the equivalence point, and then the curve goes down.
So again, we can think about this in terms of what is happening.
In the beginning it's just a weak base in water problem, but as you add strong acid, you were pronating some of your base and forming its conjugate acid here, and you're in the going to be in the buffering region.
Then at the 1/2 equivalence point, you've added enough moles of strong acid to convert 1/2 of the weak base to its conjugate, so those are going to be equal to each other -- the number of moles of the weak base and the number of moles of its conjugate acid.
At the equivalence point, you've converted all of the weak base you started with to its conjugate acid, so it'll be a weak acid in water problem, and then at the end it's strong acid.
So the trick is to recognizing what type of problem you're being asked to do, and a lot of times if people get a question and they just write down OK, at this point in the titration curve, it's going to be a weak base in water problem.
And just writing that down, most of the time if you get that far, you do the rest of the problem correctly.
So just identifying the type, there are only 5, of problems gets you a long way to getting the right answer.
So let's do an example.
We're going to titrate a weak acid with a strong base.
We have 25 mils of 0.1 molar acid with 0.15 moles of a strong base, n a o h, we're given the k a for the acid.
First we start with 0 mils of the strong base added.
So what type of problem is this?
It's a weak acid problem.
So we know how to write the equation for a weak acid or for an acid in water.
We have the acid in water going to hydronium ions and a conjugate base.
So weak acid.
For weak acid, we're going to use our k a, and we're going to set up our equilibrium expression.
So here we have 0.1 molar of our acid.
We're going to have some of that go away in the equilibrium, forming hydronium ion and some conjugate base, and so we know we have expressions for the concentrations at equilibrium.
And we can use our k a, k a for acid, it's a weak acid problem, and we can look at products over reactants.
So, see, now we're doing a titration problem, but you already know how to do this problem because we've seen a weak acid in water problem before.
So we have x squared over 0.10 minus x here.
We can assume x is small, and get rid of this minus x, and then later go back and check it, so that just makes the math a little bit easier.
And we can solve for x and then we can check -- we can take this value, 0.00421 over 0.1 and see whether that's less than 5%, it's close but it is.
So that assumption is OK.
If it wasn't, what would we have to do?
Quadratic equation.
All right, so now, here's a sig fig question.
Tell me how many sig figs this p h actually has.
OK, 10 seconds.
So, in the first part of the problem we had a concentration that had 2 significant figures, the 0.10 molar.
Sometimes later, people have extra significant figures that they're carrying along, but we had those 2, and so we're going to have 2 after the decimal point then in the answer of the p h. So again, the number of significant figures that are limiting are going to be the number after the decimal point.
All right, so we have one p h value, and now we're going to move on.
So let me just put our one p h value down.
We have volume of strong base, and p h over here, and we're starting here with zero moles added.
We have a p h of 2 .
38.
It's a weak acid, so it should be an acidic p h, which it is.
All right, so now let's move into the titration problem, and now 5 .
0 mils of the strong base have been added, and we need to find what the p h is now.
So it's a strong base, so it's going to react almost completely, that's our assumption.
If it's strong, it goes completely.
And so, the number of moles of the strong base that we add will convert all of the same number of moles of our acid over to its conjugate.
So we can just do a subtraction then.
So first, we need to know the initial moles of the acid that we had.
We had 25 mils, 0.10 molar.
We calculate the number of moles for the hydroxide added, we added 5 mils, it was 0.15 molar, and so we can calculate the number of moles of the strong base that were added.
So the strong base will react completely with the same number of moles of the weak acid.
And we're going to do then -- we have the moles of the weak acid here, minus the number of moles of the strong base we've added, and so we're going to have 1 .
75 times 10 to the minus 3 moles of the weak acid left.
So, then how many moles of the conjugate base will be formed by this reaction?
What do you think?
Same number.
So 0.75 times 10 to the minus 3.
So always remember that in these titration problems, nothing has been added yet, you're at zero mils added.
Some amount of some subtractions are going to have to occur because something has happened.
You've converted something, things are different than when you started.
All right, so now we have weak acid and we have moles of its conjugate, what type of problem is this?
If you have a weak acid and its conjugate base -- buffer, right.
So we're going to do a buffer problem and we need to know the molarity first.
So we have moles over volume -- again, the volume, you had 25 mils to begin with, you added 5 more.
So you have to have the total volume 30 mils, and we can calculate then the concentrations of both.
Now we can set up our equilibrium table, and this looks like a buffer problem because it is, and by looking like a buffer problem you something over here, you have your weak acid over here, but you have something over here now, it's not zero now, we're starting with some conjugate base.
So we have 0.0583 minus x on one side, and we 0.025 molar plus x on the other side.
We can use k a again.
This is set up as an acid in water going to hydronium ions and conjugate base, so we can use our k a, set things up, and we can always say let's see if x is small, make an assumption, check it later.
That'll simplify the math.
So we get rid of the plus x and the minus x.
Again, we're saying that if x is small, the initial concentrations are going to be more or less the same as the concentrations after the equilibration occurs.
And we can calculate 4 .
13 times 10 to the minus 4, as x, that is a pretty small number.
And we have to check it, and yup, it's small enough, it's under 5%, so that's OK.
So now we can plug this in.
X is our hydronium ion concentration minus log of the hydronium ion concentration is p h, and we can calculate p h to 3 .
38 -- again, we're limited by two significant figures in the concentration.
So now we've added 5 mils down here, and our p h has gone up a little bit, it's now at 3 .
38 over here.
There's another option for a buffer problem.
What's the one equation in this unit?
Our friend Henderson Hasselbalch.
And yes, you can use that here too, assuming that you check the assumption and it's OK.
Most people will prefer to do this because it is a bit easier.
So, you weren't given, though, the p k a in this problem, you were given the k a, so pretty easy to calculate -- minus log of the k a is the p k a.
So you can calculate that, put that in.
You have your concentrations and it should be concentrations, but you may notice that if you actually had moles the volume would cancel here.
So here are the concentrations, but with the same volume, the volume term does cancel.
It makes this a little faster and it gives the same answer, which is great.
To use Henderson Hasselbalch you also need the 5% rule to be true, because Henderson Hasselbalch is assuming that x is small.
It's assuming that the initial concentrations and the concentrations after equilibrium are about the same.
So we can check the assumption.
We can back-calculate the hydronium ion concentration, which would be x, and see if it's small, we already know it is, so it's OK.
So there are 2 options for buffer problems, but do not use the Henderson Hasselbalch equation when it isn't in the buffering region, it doesn't hold then.
So again, you check the assumption, and if it's OK, it's fine.
If not, you need to use option one and you need to use the quadratic equation.
All right, so buffering region.
Now we're at the special kind of problem in the buffering region, the 1/2 equivalence point.
So here you've added 1/2 the number of moles of the strong base to convert 1/2 the moles of the weak acid to its conjugate.
So at this point, the concentration of h a equals the concentration of a minus -- equal number of moles in the same volume, those are equal.
You can use Henderson Hasselbalch here, and find that if they're equal, you're talking about minus log of 1, so the p h is going to equal the p k a.
And you're done with this type of problem.
I have been known to put 1/2 equivalence problems on an exam, because exams are often long, you have only 50 minutes, there's lots of different type of problems, and this problem should not take you a long amount of time.
You do not have to prove to me that this is true.
All you need to remember, 1/2 equivalence point, p h equals p k a, and if you calculate the p k a, you're done.
So this is a short type of problem.
If you remember the definition of 1/2 equivalence point, it's easy to do.
So now we have another number, so 3 .
75, and we're working on our curve.
Now let's move to the equivalence point.
At the equivalence point, you've added the same number of moles of your strong base as you had weak acid.
So you've converted all of your weak acid to its conjugate base.
So the p h should be greater than 7.
Now all you have is conjugate base, that's basic, p h should be greater than 7.
So when you are doing this titration, you have your weak acid and your strong base.
You're going to be forming a salt here, and a salt problem, you can tell me about salts.
And so, just remind me, what does the n a plus contribute to the p h here.
It's going to be neutral.
And what about this guy down here?
Yeah, so it's going to be basic.
So, the sodium, anything group 1, group 2, no effect on p h, they're neutral.
But if you have a conjugate base of a weak acid, that's going to be basic.
Salt problems, really just part of what you already know about.
So always check your work.
If your p h doesn't make sense from what you know, you might have made a math mistake.
So let's calculate the actual p h at the equivalence point.
We know that it should be basic, but what is it going to be?
So first, we need to know how much of the strong base we had to add, because we need to know about all the moles.
So how much of this did we need to add.
So we needed to add enough of the strong base that you converted all of the moles of the weak acid to its conjugate.
So we had 2 .
5 times 10 to the minus 3 moles of our weak acid.
So that's all going to be converted to the moles of the conjugate base, and so that's going to be equal to the number of moles we needed to do it.
So we needed 2 .
5 times 10 to the minus 3 moles of our strong base to do that complete conversion.
We know the concentration of the base was 0.15 .
So we would have needed 1 .
67 times 10 to the minus 2 liters of this concentration added to reach the equivalence point.
So then the total volume that we're going to have at the equivalence point is the 25 mils that we had to begin with, plus this 16 .
7 mils to make this final, total volume.
And remember, you always need to think, what is the total volume, how much has been added to get to this point in the titration curve.
Then we can calculate molarity, so we know how many moles of conjugate base have been formed, and we know the new volume, so we can calculate the concentration of the conjugate base.
So now, you can help me solve this problem.
Set up an equation for me to solve it.
Let's take 10 seconds.
That's the best score we've had today.
Yup.
So now we're talking about a conjugate base.
So we have converted all of the weak acid to the conjugate base, and so it's a weak base in water problem, so we're going to talk about a k b.
If you were only given the k a for this problem, how would you find k b -- what interconnects k a and k b?
K w, right.
So you can calculate, here it's given to you, but you could calculate it if you had a calculator, and you would find that this is true.
Now it's a weak base in water problem.
We're not in the buffering region anymore.
We've converted all of our weak acid to the conjugate.
So it's a weak base in water problem.
So we have x squared, 0.06, that was the concentration we calculated, minus x.
So again, think about what type of problem it is.
So again, weak base in water problem -- x squared over 0.06 minus x.
And we can assume that x is small, and calculate a value for x, which is 0.83 times 10 to the minus 6, and then we're going to calculate p o h, because now x is the hydroxide ion concentration.
Because in a weak base in water problem, here in this type of problem, the base, and here is your acid -- the conjugate of this acid is the base, hydroxide, and the conjugate of this weak base is its conjugate acid over here, so now when we are solving for x, we're solving for hydroxide ion in concentration, so we're calculating a p o h, which then we can calculate a p h from.
So we can take 14 minus 5 .
74 and get our value.
And it's bigger than neutral, it's 8, it's basic, and that makes sense, it is a weak base in water problem.
So, let's see, it's 8 .
26, so now we're up here in our curve, and we're at 8 .
26, and that's going to be greater than 7 for this type of problem.
So that makes sense, it's good.
Greater than 7 is what we want to see.
So now, you've gone too far -- you've passed the equivalence point, and you keep adding your strong base in.
Now you still have some of the weak conjugate base around.
So you still have this around, but you only have 1 .
83 times 10 to the minus 6 molar of it.
So very little amount -- x is small.
So your p h is going to be dictated by the amount of extra strong base you're adding.
So this is similar, then, to a strong acid or strong base in water problem.
So if you're 5 mils past the equivalence point, 5 mils times your concentration of a strong base, so you have extra, 7 .
5 times 10 to the minus 4 moles extra.
So then you need to calculate a concentration of that, and so you remember the whole volume -- you're 5 mils past, you had 25 miles to start with, and you had to add 16 .
7 mils to get to the equivalence point.
And you have, that's your total volume, you get a concentration, that's your concentration of hydroxide, it reacts completely, you don't have to do any equilibrium table here.
It's going complete, it's a strong base.
You could try adding that value of your other weak base to this, but remember, that's times 10 to the minus 6, so it's not going to be significant with significant figures.
So you can just use this value -- plug it in to p o h, calculate it, and then calculate p h. And so now we're somewhere up here at p h 12 .
21, 5 mils past.
And there we've worked a titration problem.
So let's review what we saw.
In the beginning, zero mils of the strong base, we have a weak acid in water problem.
We moved into the buffering region where we had our weak acid and the conjugate base of that weak acid.
At the equivalence point, we've converted all of the weak acid to the conjugate base, so it's a weak base problem.
And then beyond the equivalence point, it's a strong base problem.
That's what we've just worked.
So, we can check these all off now.
You know how to do all of these types of problems.
And there are not that many, you just need to figure out where to apply what.
And if you can do that, you're all set, this unit will be easy for you, and you can go through and make me very happy on the exam.
There's nothing -- well, there are few things in life as beautiful to me as a perfectly worked titration problem.
It really, it brings me joy, and I've had people write on the exam sometimes, "I hope that my solution to this brings you joy."
And I will often write, "Yes, it does," and put a smiley face.
Because it really is nice to see these beautifully worked.
I know, I'm a little nerdy and geeky, but after yesterday, being smart and a nerd and a geek is cool again.
All right, so let me just tell you where we're going.
We have five more minutes, and actually that's perfect, because I can get through some rules in those 5 minutes.
So let's do 5 minutes of rules.
Oxidation reduction doesn't have a lot of rules, so five minutes is actually all we need to do that.
Oxidation reduction involves equilibrium, it involves thermodynamics.
I like it because it's really important for reactions occurring in the body, and acid bases as well -- p k a's are really important to that.
And so, between acid base and oxidation reduction, you cover the way a lot of enzymes work.
So let me give you five minutes of rules, and that will serve you well in this unit.
Some of these are pretty simple.
For free elements, each atom has an oxidation number of 0, so this would be 0.
So, oxidation number of 0 in a free element.
For ions that are composed of one atom, the oxidation number is equal to the charge of the atom, so lithium plus 1 ions would have an oxidation number of plus 1.
Again, pretty straightforward.
Group one and group two make your lives easy.
They seem to have a lot of consistent rules.
Group one metals in the periodic table have oxidation numbers of 1.
Group two metals have oxidation numbers of plus 2.
Aluminum is plus 3 in all its compounds.
Pretty simple.
Now we get to things that are a little more complicated but still useful, oxygen.
Oxygen is mostly minus 2, but there are exceptions to that, such as in peroxides where it can have an oxidation number of minus 1, and if it's with a group one metal, it can be minus 1.
Remember, group one, and actually group two here, that's plus 1, always plus 1, always plus 1, always plus 2, and so hydrogen has to accommodate that.
So usually plus 1, except when it's in a binary complex with these particular metals that are in group one or group two.
Fluorine, almost always minus 1, or always minus 1 -- other halogens, a chloride, bromide, iodide, also usually negatives, but if they're with oxygen, then it changes.
So, here is an example.
And in neutral molecules, the sum of the oxidation numbers must be 0.
When the molecule has a charge, the sum of the oxidation numbers must be equal to that charge.
So, let's do a quick example.
Hydrogen, in this case, is going to be what?
Plus 1, so it's not with a group one, group two metal here.
So what does that leave for nitrogen?
And that makes the sum, plus 1, which is equal to the sum of that molecule, so that works.
So we might not have known nitrogen, but we can figure it out if we know the rules for hydrogen and we know what it all has to equal up to.
And so, this unit is sometimes a relief after oxidation reduction, because it's all about simple adding and subtracting, it's not so bad.
OK, oxidation numbers do not have to be integers.
Example here, you have superoxide, what would its oxidation number be?
Minus 1/2.
And those are the rules, and then on Friday, we'll come back and we'll look at some examples.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, you have now 10 seconds, go ahead and click in your answer.
OK. We have a little bit of a mixed response here.
So for those of you coming in late, I'm going to give you a second chance, and we say now what if the p k a was 4.
Everything else is the same, but the p k a is now 4 -- what would the answer be?
OK, go ahead and click in your response for a p k a of 4, everything else being the same.
OK, let's do 10 seconds.
Interesting.
More people got that right.
So it's the same answer for a p k a of 3 as a p k a of 4, but it gave people the opportunity to think about that a little more and see what their neighbors had voted and make a decision based on that.
And so more people came up with the right answer.
So let's consider now for a minute why this is true.
So if you can switch to my lecture notes, I have a couple of slides on this that are not in your handout but that are related to things we've talked about.
So just a brief sort of reminder of where we were with acid based titrations.
So, we last time talked about what's happening at different points of the p h curve, and introduced this concept of 1/2 equivalence point where the p h equals the p k a, where you have equal number of moles of molecules that are pronated as depronated.
And another way to sort of think about this whole thing is that in the beginning at p h's that are lower than the p k a, you have more pronated molecules.
And if p h is much above the p k a, you have more depronated molecules.
So, we can think about things that way.
So let's think about the last question that I asked where are have a p k a of 4.
So, if you gave someone a sample of molecules where the p h was equal to the p k a of the molecule, you'd be giving them a sample where there would be equal amounts of pronated and depronated.
But instead, if you gave a sample where the p h is much above the p k a, then you're going to be giving them a sample that's mostly depronated.
And I emphasize about using, in terms of titrations, using Henderson Hasselbalch for buffer, but it also can be applied to thinking about this type of problem.
And so you can be thinking about what sort of ratio are you going to get of your pronated to depronated when you have given a particular p k a and a particular p h. And so, if the p h is really very far above the p k a, then you're going to be largely depronated, and here's the math for these two numbers.
If you're at a p h that's really below the p k a, then your molecules are going to be pronated.
And so, you're going to be thinking about this kind of thing when you get to organic chemistry, and also biochemistry.
Biochemists spend a lot of time thinking about the p k a's of amino acid side chains, and there are often mechanisms that people propose of how an enzyme works, and they're proposing that some residue is going to play a role as a catalytic acid or a base in an enzyme mechanism.
And often people are proposing things that really just don't make much sense in terms of the p k a of that molecule, that they're proposing, that it's going to be giving off a proton, but given the p h of the enzyme in the body and given the p k a, it wouldn't be pronated.
So how is it going to give off something it's not going to have.
So these are the kinds of questions people talk about in biochemistry, and if you're in a biochemistry seminar and someone's talking about something like this, there will probably be a hand go up in the audience and say "What do you think the p k a is of that amino acid that you're proposing that role in this enzyme mechanism?"
And there have been people who are pretty high profile who've gotten themselves in trouble over p k a issues.
So now, you should all be ready to raise your hand in those seminars and say "What do you think the p k a is of that residue?"
So, you'll be hearing about p k a's later, and the thing that makes me very excited is that some other people who are taking chemistry here, may not be as aware of the sort of biological role of p k a's.
Now you know something, you've had some of these in your problem-sets, and so when you go into those advanced classes and there's a discussion of p k a, you'll raise your hand and impress my colleagues with your tremendous knowledge of p k a's.
So I'm very excited about that.
And please send me an email when you get extra points because of knowledge of p k a's.
I want to collect that information and use it for evil purposes.
No.
But anyway, you can think about p k a's of molecules now, which will be very handy to you later on.
So it's not just -- some people tell me, if I promise never to titrate a weak acid with a strong base, do I have to take that part of the test?
Well, you know it's not just about acid based titrations.
Some of the things you learn in that are actually relevant to other things that most of you will see later in your career.
All right.
So, I love enzymes, enzymes are great, I'm a biochemist.
Acid base is very important in biochemistry.
And the other thing that is very important in biochemistry is oxidation reduction.
So, in these two units that we're doing now that'll be on the third exam, you're learning a lot of the basic principles that apply to how enzymes work.
And so we started last time talking about some rules.
Rules of assigning oxidation number.
So this is sort of the very basic knowledge that you need to know to go on and do oxidation reduction problems.
And here are those rules, and now let's see look at some examples of how we're going to apply these rules.
So, first let's look at a compound that has lithium and oxygen in it.
All right.
What do I know about lithium's oxidation number?
What's it going to be?
Plus 1.
So, things in group 1 are going to be plus 1, and I have 2 of them.
So we're going to be using one of those rules up there to assign it.
What about oxygen?
What's oxygen going to be in this molecule?
Minus 2.
So it's not in a peroxide here, so it'll be minus 2.
And then we have plus 2 minus 2 is equal to 0, and that's good, because there's no charge on this particular molecule, so all of the oxidation numbers should add up to 0.
That's also one of our rules.
All right, so let's look at p c l 5.
So, what we know about oxidation number of chloride?
What are you guessing?
Minus 1.
What's the exception to that?
When it's with what?
When it's with oxygen it can be different.
All right, so there's 5 of those.
I haven't told you anything about phosphorous, but what can you guess it's going to be?
5
I love that enthusiasm, good.
So, it will be 5, and so overall, we're going to add up to 0.
So you won't always have a rule about everything in your molecule, but you'll be given something that you can get a handle on and then predict what the other thing is going to be.
So let's do a couple more.
So, what about h n o 3?
So let's start with oxygen -- you told me about oxygen, what's oxygen going to be here?
Negative 2
Negative 2 and there's 3 of those.
And we need to have this all add up to 0 again.
So, what about hydrogen?
Plus 1
Plus 1.
What's the exception to that rule?
When its with a metal.
So what does that leave for nitrogen?
Plus 5.
And that will all add up.
All right, so you're very good at this, so let's bring in that clicker competition, and you can tell me about this molecule.
All right, 10 seconds.
Excellent -- not necessarily for the competition since most people got it right, but I love seeing people get it right.
So, 2 times plus 1 minus 2 is 0, so we know our states here.
And I'll just tell you another thing you can do in doing these problems, if you recognize that something is often seen has a unit here, you can also think about this as h plus and n o 3 minus, and you would get the same answers to the problem.
So, if it's a little complicated and you want to break it apart, you can do that too, you'll get the same answers there.
All right, so as I said, in this unit it's a lot of adding and subtracting, so this part is not too complicated, but you just have to be paying attention and doing your math, simple math, correctly.
All right, so we looked at some examples.
So now we're going to give some definitions.
What is oxidation?
What happens when something is oxidized?
Yup.
So, when you oxidize something, electrons are lost.
What happens when you reduce something?
Gain electrons
You gain electrons.
Most people are good with those definitions.
Here are some that give people a little bit more trouble.
Oxidizing agent.
So it is an agent of oxidation, so it accepts -- an oxidation agent is something that wants to oxidize something else, so it gets reduced itself.
So it's an agent of oxidation.
It runs around trying to oxidize other things, but it itself is going to get reduced, and then you can probably figure out what a reducing agent is, it's an agent of reduction.
And so it itself will be oxidized.
It'll try to reduce something else.
It'll run around trying to reduce something, trying to give off its electrons, donate its electrons so that it can be oxidized.
So, keep these definitions in mind, because you're going to be using them a lot in this unit.
So now we're going to use them, we're going to look at a reaction, and we're going to think about what's being oxidized and what is being reduced.
And this particular type of reaction, a disproportionation reaction, the same element can be both oxidized and reduced.
So let's break this down into two equations.
In every reaction there's going to be an oxidation and a reduction, and so you can write those separately.
And poor sodium, sodium is getting kind of a jip.
A lot in these last few units in acid base, and here again, it's a spectator ion here, so it doesn't even make it into the equation.
And it was ineffective as a conjugate acid, it hasn't really been doing very much recently.
But that's OK. All right, so let's look at what's going on here and try to figure out what's happening in terms of what's being reduced and what's being oxidized.
So let's start over here, let's think about what the oxidation numbers are in this particular molecule.
So what would be true about oxygen here and chloride.
What do we know about this combination of things in terms of oxidation numbers?
So let's start with chloride, what's that going to be here?
So, we have a minus 2 here.
And the whole thing is going to be minus 1, and so what is that about chloride?
Plus 1, right.
So this is one of the exceptions.
Chloride is usually minus 1, except when it's with oxygen, and here we have an overall charge of the molecule of minus 1, so it all has to add up to minus 1 and it does.
All right, so now let's do that side.
So what's going on here?
Are they going to be similar or different?
Let's start with the oxygen, what's that going to be?
So we have three negative 2's.
It's not a peroxide, so it's negative 2 for oxygen.
The overall has to be equal to minus 1, so what is chloride here?
Plus 5, right.
So that's unusual, but that's what it is.
So we use our rules.
Chloride is usually minus 1, except when it's with oxygen, and then it can be something different.
So, here chloride's going from plus 1 to plus 5, so what's happening to it?
Is it being oxidized or reduced?
Yup.
So it's going from plus 1 to plus 5, so we have an oxidation going on.
So to tell whether something is an oxidation or not, you need to figure out what the oxidation numbers are and then see what's changing in the course of that reaction.
All right, so down here, we've already done this one, so we can put that down here.
So we have minus 2 for oxygen, overall minus 1, and so chloride is plus 1 again.
And what's the oxidation number on the other side for chlorine?
Minus 1.
I just have to look, and so that's minus 1.
So here we're going from plus 1 to minus 1, so what is that?
That's a reduction.
And if we had figured out that they're both oxidations and something we would have done incorrectly, because in these reactions you're going to have oxidations and reductions.
So, here, and this is disproportionation.
So, you have chloride in one state, and in another, it's undergoing a reduction.
OK, so that's how you sort of think about what's happening in these types of reactions.
So now we need to balance reactions.
And this is very important in getting the correct answer to the later problems, and so we'll go through an example of how you're going to balance.
You need to think about whether it's an acidic solution or basic solution, and we'll talk about that at the end.
So, first here, we can look at the two 1/2 reactions going on, we have iron, we have chromium, and so we're going to look at those separately.
So here is the first one, and let's think about what's happening over here.
So, what would our oxidation number of the oxygen be here?
What is happening to it?
So, figure out what the oxidation numbers are of the chromium on one side, you know what it is on the other side, and then tell me what's happening to it.
I hear some murmuring.
People in one recitation aren't helping out people in the other recitation, are they?
All right.
Do you need more times?
You good?
You clicked in?
Let's do 10 seconds.
OK. People did pretty well on that one.
Let's look at the answer to that.
So, let's go back to my Powerpoint.
And so, oxidation number for oxygen here, minus 2.
The overall charge on that is minus 2.
So you have to figure out what the math equals there.
And so, if you run through the math, then you see that you're going from a plus 6 to a plus 3 state, so we have a reduction.
OK, 75% did that, got that, good.
So, most of you should be able to get this one now, iron plus 2 to iron plus 3.
Just yell it out, what is that?
Yeah.
So here we have our oxidation.
Now let's balance this.
And I left some nice blanks in your notes, but not so many that you won't be able to keep up and you should feel free to yell out the answers and we'll go through balancing this pretty quickly.
So there's a couple of different rules.
I'll just say some books have things differently.
If you can get the right answer, you can use whatever procedure you want.
I have found in the past that this particular procedure a lot of people find to be the easiest.
So I'll teach you this one, but you're free to use whichever ones work well for you.
All right, so the first thing we want to do is balance all elements that are not oxygen or hydrogen.
To make it equal on both sides, we're going to do oxygen and hydrogen later.
But what do we need to do up here to balance our non-oxygens?
We need to add a what?
A 2.
So we need to add a 2 over here, two chromiums here, two on the other side.
What about for iron?
Nothing.
All right, so that was pretty simple.
Now, in this procedure you add water to balance the oxygens.
So, what are we going to do up here.
How many waters do we need to add?
So we need to add 7 waters.
And the bottom one, bottom one's pretty easy so far, we don't have to do anything.
So, go ahead and write in your 7 waters.
Next step is we're going to balance the hydrogens that we just added, and we can balance here with h plus, and I say here's one place that books are different.
Some of them balance in the more sort of technically correct way with hydroium ions, but then your oxygens get unbalanced again, so it's OK to use the simpler approach, and just balance with h plus.
So how many h plusses do we need to add to balance the top part?
Yup, so we need to add 14 over here.
Again, the bottom one, nothing to do.
Pretty simple.
Now we need to balance the charge, so we just added h plusses, so we just added some charge to this, and now we have to balance the overall charge, so you want the charge on one side of the equation to be equal to the charge on the other side of the equation.
So how many electrons are we going to have to add to this top equation to balance the charge?
I heard the answer.
6.
So we have to add 6.
I told you, this unit involves adding and subtracting in your head, and you always want to check your work when you're doing this on an exam, because it's really sad when you lose points for something that is adding and subtracting.
So you want to make sure that you don't lose points on these on an exam.
So what about the bottom -- finally we get to do something with the bottom one, what do we do?
So, we add how many electrons?
1, right.
OK, so now our charge is balanced.
So now, we want to multiply up one of the 1/2 reactions so that the electrons are going to cancel, and so what do we have to multiply by the bottom equation so that the electrons cancel with the first?
6.
So we have 6, 6, and 6.
And now, we're going to add those together and make the appropriate cancellations.
So here is our overall equation -- we have the 6 electrons, the 14 protons, we have our chromium oxide compound, we have the 6 iron 2 plusses, on the other side, the 2 chromium 3 plusses, the 7 waters, the 6 iron 3 plusses, and the 6 electrons.
So, we should be able to cancel the electrons, so we cancel those out.
And now we want to double check that, in fact, it's balanced.
Again, this is very important to do on the test, it's really easy to make some kind of math mistake, but you should be able to figure it out at this point.
It won't be balanced if you've made a math mistake.
So there should be 14 hydrogens over here, 14 over there, 2 chromiums, 2 chromiums, you have 7 oxygens here, 7 oxygens there, 6 irons, 6 irons.
And the charge also should be balanced on both sides.
So you can double check that and if it's good, then you're done.
And this was in acidic solution, so we should have again this number and a charge of plus 24 on each side, double check, make sure it's correct.
So, that was acidic solution and we ended up with an equation that had h plusses in it.
You can also be asked to do it in basic solution, and again, books have different approaches here.
What I like to do is the simplest thing, which is to use the same steps all the way up to here to get your same answer that you had for the acidic solution, and then neutralize it at this point.
So, adjust the p h in quotes by adding hydroxide ions to both sides, and so, if we have 14 h plusses here, we can add 14 o h's on one side, and 14 o h minus on the other side.
And then we can add those guys together.
So we have now 14 waters over here.
And we have still our 7 waters on this side, and then the 14 hydroxides on this side.
And now, we should be able to cancel out some of those waters.
So, we had 14 over here, 7 over here, so we can just have 7 on this side.
And now we have an equation that looks like it's in basic solution.
So instead of having h plus, which you would have in acid, you have hydroxide for basic solution.
So you can follow the same rules.
Again, some books do it differently.
I think this is the simplest way to do it, the least likely to make those kind of adding and subtracting errors.
So those are sort of the fundamentals you need for this unit.
You need to figure out oxidation numbers, looking at the composition of a molecule, and you need to be able to balance equations.
When you can do that, you can go on to do other things.
So, at this point -- oh, let me just say, those are the two answers there, so you can see acid and then in base.
So, at this point then, I want to try to do a little demo to show you that when you do do oxidation reduction reactions, cool things can happen.
So, it's more exciting, perhaps, in real life when oxidation happens than on paper.
So I'm going to turn it over now to Dr. Taylor and Dr. Patti Christie who's here to help us with this demo.
And they will -- go for it
OK, so this is mostly visual, there's not too much we'll need to say.
Basically what we're going to do for you is oxidize magnesium.
So we have a source of oxidation, which is going to be carbon dioxide, which is dry ice, and what we put inside the dry ice is solid magnesium.
So we're going to set the magnesium on fire, and then put the oxidating agent on top of it, and you can see the rest of what's going to happen there.
And you can actually also determine if this is an exothermic or an endothermic reaction while you're at it.
[EXPERIMENTING] [APPLAUSE]
Thanks to my helpers.
All right.
So oxidation reduction reactions can be quite interesting, and don't try that at home.
So, we're going to continue on now, you know the basics and we're going to talk about electric chemical cells.
We're going to introduce Faraday's law, and a thing I love to do, is come back to my friend Gibb's free energy.
So, thermodynamics is all over the place in chemistry and you can't get very far away from it.
So hopefully, by the end of the class today, we'll come back to free energy.
All right, so what is an electric chemical cell.
It's any device in which electric current, which is a flow of electrons through a circuit, is either produced by a spontaneous reaction, or used to bring about a non-spontaneous reaction.
And so a battery is technically a collection of cells in a series, so that the voltage that each cell produces is the sum, the battery has the sum of the voltages of each cell.
So, let's take a look at what some of these might look like.
Here's a little cartoon of a simple version.
We have a beaker -- 2 beakers with different solutions in them, 2 electrodes, a salt bridge across, and as electrons transfer through this, you can read a current on an amp meter here, you can read some kind of voltage coming off.
And so, I like to talk about how good you guys have it at MIT with the Web and handouts and everything and how students in the old days at MIT, if they're going to do their problem-sets at night, first they had to build a battery to get electricity to be able to see.
So you guys have it easy, you have these electric lights and all this fancy stuff now here.
So, let's sort of break apart sort of a components.
Here is my beautiful picture that I drew of this system, that's why I show you a better cartoon first.
So this is a beaker that you -- here's one beaker with one solution on this side, here is the other beaker.
So we have two electrodes put in, we have a salt bridge, and then we have a wire across where we can measure voltage.
So, let's think about what is happening on each side.
So in one beaker you're going to have an oxidation reduction, and in the other beaker you're going to have a reduction.
So if we think about what's happening over here, you could have a zinc system, is oxidized from zinc 0 to zinc plus 2.
And so, this is sort of a view of what would be happening at the electrode, you have zinc solid or zinc 0, and you've also have zinc plus 2 in solution.
And so, as you take a zinc solid atom, going into zinc plus 2, you can have an oxidation reaction.
And here would be the equation going down.
So zinc solid to zinc plus 2 with two electrons.
Those electrons can go through to our other beaker on this other side, and so here we have a copper solid electrode or a cathode in this case, and we have copper plus 2 in solution, which is being reduced to copper solid, and so we're plating on to our electrode on this side.
And so, here we have the reduction reaction, copper 2 with two electrons going to copper solid.
So our oxidation is happening at an electrode called the anode.
The reduction is happening in an electrode called the cathode.
And as these reactions occur, you're changing the charge on either side.
So, you have this salt bridge, and so to neutralize the change in charge, you would have negative ions come down here, because we're producing more plus 2.
And on the other side potassium is going in as we're going from copper plus 2 to copper solid, again, to balance this change in charge that's occurring.
So those are the basic components of a simple electrochemical cell.
So, we could talk about the cell -- here's some nomenclature that represents the cell.
So, the one that I just showed you, you have one electrode, a zinc solid, and we also have zinc plus 2 in solution.
You have a single line between the solid and the zinc plus 2 ions in solution, that indicates there's a phase boundary, so you're changing phase from solid to aqueous.
Then if you see two little lines that tells you, okay, one part of the reaction's in one beaker, the other part is in the other beaker, that represents the salt bridge which separates out the two 1/2 reactions.
And then on this side, we're going from copper 2 in solution, single line which indicates the phase boundary, to a copper solid over here.
So the amount of charge that goes through the system, it depends -- and how much of the zinc that is consumed or the copper that is deposited, is proportional to that charge, the number of electrons that go through the system, and this is called Faraday's law.
So let's look at what is happening a little bit here, so we have a little movie that shows the oxidation reaction.
So in green is the electrode, and then here are the water molecules, and let's look at what happens when there's this oxidation.
So here things are floating around, and there go the electrons.
And this pops off into solutions, so if this is zinc, so then as this is happening, we have a zinc plus 2 now, it's aqueous.
There go electrons, we have our oxidation.
Here comes off zinc plus 2 floating around, there's another zinc plus 2 that just came off.
And so our electrode here is being consumed as this oxidation occurs.
I think it's really cool, the electrons, you can see them as this green going along in this movie.
So, this is a little thing to think about what's happening.
So what's happening at the cathode, we have a reduction.
So if we have our copper ions are going to plate onto our electrode.
So here in blue is the electrode, and again the water molecules, here's a little copper 2 in solution.
And when it gets electrons and gets reduced, it's going to start plating, so let's take a look at that.
So, here's -- oh, there comes the electrons.
And so now it becomes part of the electrode there, the electrons come in, and so you're building up, you're adding, your plating on to your electrode.
Now it got its electrons, it got reduced, got reduced again, and it becomes solid, and plates onto the electrode.
And we see the electrons coming in.
So again, the amount of current that flows, the number of electrons that flow are going to be proportional to the chemistry that's happening at the two electrodes.
So you can figure out how much zinc would be consumed, how much of the zinc solid electrode could be consumed, and how much copper would be deposited on the copper electrode for a particular amount of current.
So say we have 1 amp of current for 1 hour, what are we going to do to our system?
So we can calculate this out, so we use this equation to find out how much charge is going to pass through the system, we have this term q, the magnitude of charge and it's in Coulombs, those are our units, and that's equal to the current in amps, and just for unit conversion, amps are Coulombs per second, so that's very convenient because we're going to take the current and times second, so times time.
And so if we work that out, we had 1 amp and we had 1 hour, so we're going to have 3 Coulombs that are our magnitude of our charge.
So once we know that, we can convert, using Faraday's constant from charge into moles of electrons that had to pass through the system to generate that kind of current.
And here is our friend Faraday's constant, and so that's in Coulombs per mole.
And so, we can we can convert the number of Coulombs into the number of moles of electrons that were passed through the system with that amount of current, 1 amp for 1 hour.
Now, how is this going to relate to the zinc consumed or the copper deposited?
So we know the total number of moles that passed through the system.
And so, then we have to think about for every 1 mole of zinc that was consumed, how many moles of electrons went through the system?
So why don't you yell out and tell me what you think that is.
What do you think?
2, right.
So we are going from zinc solid to zinc plus 2.
So, we need to do -- to consume one, we need two electrons.
So then we can look up the atomic weight of zinc and calculate the number of grams.
We can do the same with copper.
So for one mole of copper deposited, how many moles of electrons needed to pass through the system?
two, right.
And then we look up the atomic weight, which is very similar, and actually with significant figures, it's the same amount, which won't be true for most things.
All right, so these problems are not too complicated.
So let me just tell you a little bit more about types of electrodes that can be used.
It's not always true that you have to use electrodes that are going to be consumed or have species deposited on them.
You can use an inert electrode, such as a platinum electorate, and here is an example of that.
So here, this cell has a platinum electrode, and the reaction that's happening, you have actually two things in solution instead of going from a solid to a solution.
So this is just another example of a type of cell.
So let's think about, then, what equations we would write for this chemistry and what's going on.
So over here at the cathode, what happens at a cathode, oxidation or reduction?
I'm hearing somewhat of a consensus.
That's correct.
Reduction.
That's the thing you're going to need to memorize, it'll be important in doing problems later on.
So, what is the reduction reaction that you imagine would be happening if you have copper plus 2 and copper solid?
So, if you're reducing it, you're going to be reducing copper plus 2 with two electrons to copper solid.
So, same reaction we saw before.
So what about over here.
This is the anode, which has the oxidation reaction going on, and what oxidation reaction can you imagine happening with chromium plus 3 and chromium plus 2.
What's going to what?
So, you're going from plus 2 to plus 3 with one electron here.
So that would be the oxidation reaction.
So let's think about how we would actually write this down then.
So the notation for this type of cell, we're going to have a platinum that indicates the electrode, a single line that indicates a phase boundary.
And in solution, you have the chromium plus 2, chromium plus 3, those are both aqueous, so they have a comma between them.
So this is on one side in one of the beakers, so that's the reaction at the anode.
And then we have a bar here that indicates a salt bridge.
And then in the other beaker, we would have the copper plus 2 and the copper solid, separated by a single line, because there's a phase boundary between the aqueous and the solid.
And then here are our two equations.
At the anode, we have copper plus 2.
Aqueous going to copper plus 3 aqueous in an electron at the cathode.
We have copper plus 2 aqueous and two electrons going to copper solid.
So those are our two equations and how we would write that down.
Another example of an electrode is a hydrogen electrode.
And this is very common.
In fact, most potential standard reduction potentials, or as they're also known, oxidation reduction potentials are measured against a standard hydrogen electrode, and it'll be abbreviated s h e. And so, if you see it was measured against s h e, you will now know what that means.
So that's a standard hydrogen electrode.
So there's a couple of different variations.
It can be used at the cathode or at the anode.
And so, if you're using it at the cathode, h plus is reduced, and at the anode, h 2 is oxidized.
And often you have a platinum system there as well, for the standard hydrogen, so platinum is commonly used in a standard hydrogen electrode.
So, let's just sort of look at a little picture of this.
So here, we may have like a glass cylinder here, we're pumping in h 2 gas.
And here you can see it sort of bubbling down.
We may have a solution of hydrochloric acid, which would have a lot of h plus in it, and so this would be a hydrogen electrode on one side, the other side may have something more common or something that we saw before with the zinc solid and the zinc in solution.
So since on this side we have the cathode, of we can write about the h plus aqueous, a bar for the phase transition to h 2 gas, and then we indicate that there's also a platinum electrode going on here.
And the other side is what we saw before, the zinc solid and the zinc plus 2.
So you're just introducing different types of electrodes that you may be seeing in this particular unit.
And again, you want to remember what reaction's going on at the cathode, and at the anode.
All right, so just very briefly, I will mention cell potential, which has many different names -- cell voltage, EMF it's often called.
And so the flow of electrons generates a potential difference between the electrodes in the current.
And this can be related back to our friend, delta g. So in this equation you have this potential difference that's generated by the flow of the electrons, and that times Faraday's constant times the number moles of electrons, and if you have a negative sign here, that's equal to the free energy of the cell.
So we can relate that back.
And a little bit of information.
We have the standard terms as well.
We can have the delta e nought, the cell potential, cell voltage, you'll see many names for this.
And that's when the products and reactants are in their standard states.
The units we're talking about here are in volts, sometimes you'll see things reported in millivolts as well.
And let's end with one final clicker question, and then we can announce our clicker winner for today.
OK, 10 seconds.
Most people should get this right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so we're going to try to finish up oxidation reduction today.
The next unit we're heading into is transition metals.
So, in the last class we were talking about delta e knots of cells, so the standard potential of a particular electric chemical cell.
And as we ended class last time, you had a clicker question, which you told me what the reaction was at the anode and the reaction at the cathode for this particular electric chemical cell.
So, now we're going to consider, we're going to calculate what this delta e nought is for this cell.
So we have an equation that we can use to calculate delta e nought of a cell, and that is that the delta e or the potential of the cell, is equal to the standard reduction potential for the reaction that's occurring at the cathode, minus the standard reduction potential for the reaction that's occurring at the anode.
So you told me which reaction was occurring at the anode and which at the cathode last time.
So we can just go ahead and use that information in here.
So the reaction that's occurring at the cathode is the copper reaction, and the reaction and the anode is the zinc reaction, and note here, I have these written as reduction potentials, because we're going to be plugging in the standard reduction potential into this equation.
So, where do we get our standard reduction potentials?
Well, your textbook has tables of many standard reduction potentials.
And, of course, on an exam you would be given that information.
And the standard reduction potentials were measured against the standard hydrogen electrode.
So, now we talked last time about what some of these abbreviations mean.
So we can look up the values and they're all going to be listed as reduction reactions, because they are standard reduction potentials.
So all the reactions, if you're looking for ones written as oxidation, you'll be out of luck, they'll all be written as reductions.
So here, written as a reduction, zinc plus 2, two electrons to zinc solid, copper plus 2 plus two electrons to copper solid, and you can look up those standard reduction potentials.
And then you can plug them in to your equation.
Again, the equation is the standard reduction potential for the couple at the cathode minus the standard reduction potential for the reaction at the anode.
So the reaction at the cathode is the copper reaction, and so we have -- we put in here 0 .
3 4 0 2, and then minus, and the zinc reduction potential's negative, so it's minus a minus 0 .
7 6 2 8, and then if we add that together we get a positive value of 1 .
1 0 3.
And so in doing these, this is a pretty easy thing to do, but I'm sort of emphasizing it because a lot of people out-clever themselves in doing this.
They say, well, one thing's being oxidized in this reaction, one thing's being reduced, so I'm looking up a standard reduction potential, I'm going to change the sign, and then plug it in and change the sign again.
And so they change the sign so many times and they come up with the wrong answer.
So just a hint in doing these problems, if you always think that the equation is asking you for the standard reduction potential, look that value up and put it in.
Don't do anything fancy with signs, just use this equation as written, plugging in the standard reduction potentials, and you'll be all set and you won't have any problems getting it wrong.
So try not to out-clever yourself and flip signs around many times in doing these problems.
All right, so we have a value, a positive value then for our potential for the cell.
So we can ask the question, in this type of cell, would the flow of electrons be spontaneous?
What tells us if something is spontaneous?
Delta g tells us if something is going to be spontaneous.
And we mentioned last time a connection between delta g and delta e. So, delta g tells us if something will be spontaneous, and we talked last time of this equation that delta g nought for the cell is equal to minus n, n here being moles of electrons times Faraday's constant, times the delta e nought for the cell.
So here again, we're relating back to thermodynamics, back to delta g, and we're thinking about whether things are going to be spontaneous or not.
So, if we have a positive value for delta e nought of the cell, what's going to be true then for delta g?
It'll be negative, and so will it be spontaneous?
Yes.
So, if delta e nought is positive, delta g will be negative, and if delta g is negative, the reaction will, in fact, be spontaneous.
So the answer, then, is yes.
So now let me introduce some more terms to you.
Galvanic cell is an electric chemical cell in which a spontaneous chemical reaction is used to generate an electric current.
So on a problem, which you may have on your problem-set and have already looked at this, it'll say something about for a galvanic cell.
Well, that wasn't just kind of random information they're throwing out, they're telling you that the reaction's going to be spontaneous in that cell.
So that will often tell you what reaction had to be happening at the anode, and what reaction had to be happening at the cathode.
Because you need to have it be spontaneous, you need a value for delta e nought, then it's positive.
So, the information that it's a galvanic cell tells you a lot about the problem.
In contrast, we have electrolitic cell, and in this case, we can put in energy to provide, to be able to drive a non-spontaneous reaction.
So you can generate a current to then force and non-spontaneous reaction to go.
So these are 2 different kinds of cells.
So again, whether something's spontaneous or not comes back to our friend delta g. So if a cell is operating spontaneously, that means you're going to have a delta e nought of the cell that's positive, which means that the delta g of the cell will be negative.
And we can calculate these delta e noughts of the cell from the standard reduction potentials, which some nice person measured for us against the standard hydrogen electrode.
And so we can look up those values and then we can calculate delta e nought of a cell, we'll know something about whether it will be a spontaneous reaction or not.
So now let's think about the size and the sign of standard reduction potentials and what they tell us about a particular reaction.
So the meaning of the standard reduction potential that you can look up in your book.
So first let's think about what happens or what would be true if we had a large positive delta e nought.
And that's going to mean that the element is easy to reduce.
So let's look at an example.
At the top of your table, you're going to see this particular reduction with this particular reduction potential.
So we have f two gas plus 2 electrons going to 2 f minus.
And the standard reduction potential for this is measured at plus 2 .
8 7 volts.
So, as written, that's the value of the standard reduction potential.
That's a large positive number, so that's going to mean that it's easy to add electrons to f 2.
The delta g nought would be favorable for that.
So then, you can tell me, does that make f 2 a good oxidizing agent or not and why?
OK, let's do 10 seconds.
Good.
Yes, it is easy to reduce.
So it's easy to add electrons, which makes it a good oxidizing agent.
So let's go back to the slides.
So, a good oxidizing agent is something that oxidizes other elements and gets reduced itself.
So it goes around oxidizing things, it's an agent of oxidation.
So something that easy to reduce is going to be a good oxidizing agent.
And something that's easy to reduce is going to have a large positive standard reduction potential.
So, one way to sort of remember this is for a particular couple.
If something has a large positive delta e nought, the oxidized species, the oxidized species here is the f 2, it's, of these two things, it's the one that's oxidized.
So the oxidized species is very oxidizing.
So that's one way to remember it.
Large positive delta e nought, oxidized species is very oxidizing.
So, here is a list that is similar to what you would see in the back of your book, and we just talked about this couple between fluoride gas and a fluoride minus up here with this large positive standard reduction potential, and as it says up here, oxidized form is strongly oxidizing.
Now, here we have positive numbers, and now we start to go to small negative numbers, and by the time we get down here, we have a large negative number for the standard reduction potential, and this is between lithium plus and lithium solid.
So, let's consider what would be true down on the other end of the table.
So a large negative delta e nought means that the element is hard to reduce.
So let's look at this reaction with lithium.
So we have lithium plus with lithium plus 1.
When you add one electron to it, you get lithium solid.
And the standard reduction potential for doing that reaction is minus 3 .
0 4 5 volts.
So, that's hard to add electrons to lithium plus 1 -- lithium plus 1 is very happy being lithium plus 1, it doesn't want that electron back, and so that would be a non-spontaneous, unfavorable reaction.
So, is lithium plus 1 a good oxidizing agent?
No, it's not a good oxidizing agent, but something around here is going to be a good agent.
Lithium solid is a good reducing agent.
So, lithium solid likes to reduce other things, lithium solid likes to be oxidized, lithium prefers to be lithium plus 1.
So it's very happy to be lithium plus 1, so the solid form is a good reducing agent.
So, if we think about what's true at the bottom of the table, then if we have a couple of plus 1 to the solid, then if you have a large negative standard reduction potential, the reduced species is very reducing.
So the reduced species here is the lithium solid, it is a good reducing agent.
So, large negative reduced species is reducing.
And if we go back to work table then, up here we have the big positive numbers.
The oxidized form of this first couple is very oxidizing, at the bottom you have the large negative numbers, and the reduced form of that couple is very reducing.
And you're going to be asked questions, given different elements in problem-sets or on exams and saying which of these is the better oxidizing agent, which of these is the better reducing agent, and be able to compare, and you need to think about where it is, which are the bigger positive numbers, bigger negative numbers, and you can make those comparisons, and if you remember on the top, big positive, oxidized form very oxidizing, big negative, reduced form very reducing, you'll be all set to answer those questions.
And some of this should be sort of intuitive of the things that we've talked about already in this course.
So we think about the periodic table for a minute, and here are some of the potentials, lithium is easy to oxidize, it's a good reducing agent.
If it's lithium plus 1, then it has a noble gas configuration, it's very happy there.
Whereas fluoride can get its noble gas configuration if it's f minus, so it's easy to reduce and that makes it a good oxidizing agent.
So you can think about this in terms of it's sort of intuitive if you think about what you know about those elements going into this unit.
So now, let's think about calculating a standard reduction potential for this particular electric chemical cell.
So we're given equations and we want to calculate a standard potential for this.
So, in doing this then, we're going to use our standard reduction potentials on this table, it's a little hard to see, so I'll go back over there.
So now we have to figure out if we see this equation, what's the reaction that's going on at the cathode?
First tell me what's happening at the cathode -- is it an oxidation or reduction happening at the cathode?
The reduction is happening at the cathode.
Now look at that equation up there and tell me what is being reduced -- which element is being reduced in that equation?
The iron is being reduced.
So we're going to write the 1/2 reaction, that's balance, so there were two iron 3's.
And on the other side there's two iron plus 2's, and how many electrons am I talking about here?
two electrons.
So we have the reduction reaction.
Then at the anode, the anode has an oxidation going on, and we only have one choice left of what's being oxidized.
So, we have -- and again balanced.
We have 2 i minus, and this is an aqueous solution going to i 2 solid plus two electrons.
So we have now our two 1/2 reactions written out, and we can look at the potentials for the standard reduction potentials, which we'll need to calculate the e for the cell.
So now we want to calculate e nought for the cell, and that's going to be equal to the e for the cathode reaction.
And that particular couple at the cathode, we're looking then at iron 3 plus going to iron 2 plus -- that's the reduction potential that we're going to be looking up.
And then we also need another standard reduction potential, the one for the couple at the anode, and the couple at the anode that we're looking up is i 2 to i minus.
So, if you can see that up there, it's also in your handout, then we can plug in those values.
So, for iron, we have plus 0 .
7 7 0 volts minus plus 0 .
5 3 5 volts.
And that's going to equal 0 .
2 3 5 volts, which is a positive number.
So what would be true about this reaction in terms of it being spontaneous or non-spontaneous?
Right, so it would be spontaneous, because you have a positive value for delta e, so we'd have a negative value for delta g. All right, so now, let's think about what the good oxidizing and reducing agents are here, and let's have a clicker question for that.
So, which is the better oxidizing agent, iron plus 3 or i 2, and which is the better reducing agent, i minus or iron plus 2?
So think about that one.
All right, let's take 10 seconds.
Very good.
So the trick here is to, again, think about these two different potentials.
Now, unlike the example we had before, we had a big positive and a big negative value, these are both positive values.
But one of them is bigger than the other, and so the better oxidizing one is going to be the one of these two, these are both the oxidized form.
So the one with the bigger positive number, the oxidized species will be better at oxidizing, and so that would be the iron plus 3, that's the bigger positive number.
Whereas here, when we're considering which is the better reducing agent, we're looking at the two reduced species here, and here the thing with the smaller a positive number, the reduced form will be a better reducing agent, more reducing.
So again, you can look at where those are in the table and the size difference between them to get the right answer.
So, very good.
OK, so now, I want to consider a biological example for a minute, and I'm going to ask a question that we we'll then answer at the end of class.
So in cells, things have reduction potentials.
Vitamin B12 has a reduction potential.
In fact, it has one of the largest negative reduction potentials of any biologically occurring molecule.
And so it has to be reduced to be active in the body.
So how can something with a very large negative reduction potential be reduced?
That's the question.
Why should you care?
Well, because vitamin B12 needs to be reduced to be active.
And the proper functioning of enzymes, there's only actually two that you have in humans that require B12 -- one that requires B12 and folic acid is thought to be important in preventing heart disease, birth defects, and B12 hs recently been linked to mental health.
In particular, a lack of B12 has been linked to dementia.
So, these are all pretty significant things.
So, one thing that I think is kind of interesting here when they talk about heart disease, how many people have heard of cholesterol?
How many people have heard of homocysteine?
Would you be surprised to know that homocysteine is a better indicator of whether you'll have heart trouble than cholesterol?
Yes.
You would think that you would have heard of the one that was a better link.
Well, homocysteine is actually a better link to indicate whether you might have heart problems, but the thing is that there's not a whole lot of money to be made there, whereas there's a lot of money to be made in drugs that lower cholesterol.
So you may realize, if you think about where you get your medical information, it's often from commercials where someone's trying to sell you something.
And so if there isn't much money to be made, say, the treatment for a condition might be taking vitamins, which there's not a whole lot of money to be made there, you don't hear as much about it.
So, it's important to consider the source of information about your health, and as scientists, you can all evaluate information about your health now.
All right, so vitamin B12 is very important, you really need vitamin B12 to be a healthy person.
And most people at MIT are maybe at this stage are not that worried of heart disease.
Many of you are probably not worried yet about having children with birth defects, but maybe some of you are worried about mental recognition of particular facts for exams coming up, and so you may be concerned about B12 to keep sharp mentally.
So, where do you get vitamin B12 and folic acid in your diet and how is it reduced?
So let's first consider where you get it in your diet.
So, does anyone know where you get vitamin B12 in your diet?
I heard vegetables.
Anyone agree with vegetables?
I didn't say if anyone liked vegetables, does anyone agree that vitamin B12 come from vegetables.
It doesn't.
Where does vitamin B12 come from?
So, red meat is a good source of vitamin B12, all meat is a good source the vitamin B12.
Plants do not use vitamin B12 at all.
So, people who are vegan are in trouble in terms of how much vitamin B12 they get, but luckily there are vitamin tablets that can help take care of this.
So meat is really the best source of vitamin B12.
So, I'm always looking for good references to vitamin B12, and I saw one recently.
Has anyone seen the HBO series, True Blood.
One person.
OK, if you watch that, you will notice that if you date a vampire, you have to make sure you get extra vitamin B12, and that Sookie, of True Blood is taking vitamin B12 tablets just to be on the safe side.
So, vitamin B12.
What about folic acid?
Anyone know where you get folic acid?
In the fall, what happens, people say let's go outside and look at the trees or look at the pretty foliage.
Any suggestion of where folic acid comes from?
Oh, wait a minute.
I was at a meeting once where there was a whole lecture about how Norwegian beer was the best source of folic acid, and it was, perhaps, not coincidentally reported by Norwegian scientists.
They would not vouch for any other kinds, but some -- but it does come from barleys, vegetables, this kind of thing.
So, here is a secret for healthy diet.
Have some of you seen the orange juice commercial that says "Drink orange juice, it's good for your heart."
That's because of this, so that actually is potentially true.
So you get a lot of folic acid in orange juice, and folic acid is, in fact, good for your heart.
So, who would have guessed, some of the claims are actually true.
All right, but we still have a problem, we still need to know how its reduced in the body.
And so we're going to come back to that at the end of the lecture and answer that if we get that far.
All right, so we need to do a couple more things first so everyone can finish their problem-sets, and the things we're going to do is we're going to look adding and subtracting 1/2 cell reactions, and then get to one of my favorite things, which is the Nernst equation.
All right.
So, some of you may have encountered this problem on the problem-set already.
Suppose you need you know a standard reduction potential and it's not given to you in the back of the book, but other things are given to you in the back of the book.
Can you calculate the thing you need from related equations?
So suppose you really want to know the reduction couple for copper 2 going to copper 1.
But that's not given.
You find copper 2 with two electrons going to copper 0, and you find copper 0 going to copper plus 1 with one electron, and you'll realize that if you combine these equations, you'll get the desired one.
So if you add these together, you have copper 2 with one electron, one electron cancels out here.
The copper solids cancel out and you're left with copper plus 1.
So you add these together, and you know the potentials for these.
How do you get the standard potential for the thing that you've come up with, for the sum of those?
So, as someone actually asked me before class, you have to go back to free energy, and you do, but I'm going to drive an equation so that you don't have to go all the way back, but this is, in fact, how you do the problem, you think about the different free energies.
So, the new reaction, the new delta g for that new reaction will be equal to the delta g knot for the reduction minus the oxidation reaction here.
But we can substitute for delta g this minus n, Faraday's constant times our reduction potential, and put it in to this equation here.
And so we have it for the new reaction, and then the 2 reactions that we're adding together.
So we have Faraday's constant in common, so that's going to cancel out.
And we can also move n 3 to the other side, because we want to solve for this new e value, this new standard reduction potential.
And so that's going to be equal to the number of moles that are involved in the reaction that's the reduction times its reduction potential minus the number of moles involved in the oxidation reaction with its reduction potential, and then the number of moles for the reaction that -- this new reaction in question.
So we can use, then, this equation to look for a 1/2 cell reaction that is the sum of two other reactions.
So let's go and use that equation, then.
So here we have the known values -- we know the couple for copper 2 to copper 0, we know the couple for copper 1 to copper solid, and this is what we want to know.
So we can use this equation.
Which of these reactions is the reduction, what value am I going to put in here?
Which of these goes into the reduction?
Which of these is a reduction reaction?
Yeah.
And how many moles of electrons are involved?
So, we put in 2 times 0 .
3 4 0 volts, and then over here in the oxidation, there's one electron involved, and we put in our other potential.
So 1 times 0 .
5 2 2.
And what are the number of moles of electrons for our desired, final reaction?
1.
And so, then we can do the math and come up with an answer of 0 .
1 5 8 volts.
And so now we've just come up with a new reduction potential, we've calculated a new reduction potential for this 1/2 cell reaction right here.
So, this equation will be given to you on an exam and all you need to do is know how to use it.
So we've calculated now the standard reduction potential for copper 2 to copper plus 1.
So, just a little note about when you're going to use this and when you're going to use the other equation that I showed you.
So, if we're talking about a whole electric chemical cell, the number of moles that are released at the anode are going to equal the number of moles taken up at the cathode and be the number of moles involved in the overall equation.
So this equation is not necessary, and for a full electric chemical cell, we're going to use the equation that I gave you before for this delta e nought for the cell, so we just use this.
But if it's not a whole electric chemical cell you're talking about, if you're talking about calculating a 1/2 cell potential, then you need to use this equation.
Both of these equations will be given to you on sheets for the exam.
This one's for an electric chemical cell, this one's for calculating a 1/2 cell potential.
So again, if it's 1/2 cell potential, you want to use this equation.
OK, so now the Nernst equation.
So, some of you may have encountered in your life where you go to turn something on and you discover that the battery is dead.
So, an exhausted battery is a sign that your chemical reaction has reached equilibrium.
At equilibrium, you're going to have a zero difference potential across your electrodes, and the battery will be not useful to you at that point.
So, if you -- instead of getting annoyed next time you have a dead battery, you can think about all, ah, it's finally reached equilibrium.
So, we need to think about, then, to think about how these cell reactions are happening, how the potential is going to change with the composition of the ingredients in those electrochemical cells.
So, we know something about equilibrium and components of reactions again.
So this is a nice lecture to give coming up with a week before the exam, because now we're going back and reviewing the first material on the exam from chemical equilibrium.
So, we know that delta g is going to change as the components change until equilibrium is reached.
And when equilibrium is reached our delta g is going to be equal to zero.
Before equilibrium is reached, delta g will depend on the delta g nought for the equation, r t and the natural log of q, where q is what?
What's q?
The reaction quotient.
So we saw this before and we're back to using it again.
So what do we know about the relationship between delta g nought and delta e?
We know that delta g nought is equal to minus n, moles of electrons times Faraday's constant times delta e nought.
So, we can combine some things around.
So we can take our equation that we saw in chemical equilibrium and substitute in values that are related to standard reduction potentials.
So we can put in, for this delta g, minus n Faraday's constant times the delta e. And for delta g nought, we put in the same thing but now we have delta e nought.
And we have r t, gas constant times temperature times the natural log of q, our reaction quotient.
Now we can divide both sides by n and Faraday's constant and we come up with the Nernst equation.
So the Nernst equation tells us the potential for a cell at any given time, at any given component of ingredients in the cell, any amounts of, say, your zinc plus 2, compared to the standard potential for that cell, which you're going to calculate from your standard reduction potentials in the table, and then you have this term, gas constant times temperature, number of moles of electrons involved, Faraday's constant, and you need to know q, the particular composition of the cell at that given time.
So, just a hint in doing Nernst equation problems, if you're given a problem and it's giving you concentrations of things at a particular time, that should be a clue that the Nernst equation is going to be what you're going to want to use here.
So, let let's look at an example.
Say we have a -- we want to calculate the cell potential at a given time at 25 degrees, and we know our zinc plus 2 ions or 0 .
1 0 molar and copper 2 ions are 0 .
0 0 1 0 molar, and this is the equation for that cell.
So we can look up, again, our values in the table, and the first thing we're going to do, if we're going to use to Nernst equation, is to calculate the standard potential for that particular cell, and here are the values from the table, and why don't you go ahead and calculate that for me.
All right, so let's just take 10 more seconds on that.
OK, so let's see if we can get into the 90's pretty soon, because we should be able to do that for this particular one.
All right, let's go back to the presentation here.
So it was actually, this was pretty easy.
I didn't actually mean for these two things to show up, to help you out of what the reaction at the cathode and the anode was, so that was an easier question than I had intended.
So, first you need to think about if you're given an equation here, what's happening at the cathode, what's happening at the anode.
As it's written, you can look and see so copper plus 2 is going to copper solid, zinc solid's going to zinc plus 2, so you can think about which is the cathode reaction, which is the anode, which has a reduction, which has an oxidation going on, and then once you know which couples you're talking about, then you can plug in your values.
So, at the cathode, we're going from copper plus 2 to copper solid.
We put in our standard reduction potential, using the equation we have a minus, and then the standard reduction potential at the anode, so we have the oxidation from zinc solid to zinc plus 2, and we put in this value and we calculate the number here.
So then, we have that, we also need to know q.
So why don't you tell me what q is now?
OK, let's just do 10 more seconds on this.
OK.
So here, if we go back, q is going to be equal to products over reactants, except things that are solids are not are not changing during the equilibrium so we leave those out.
So it's going to be concentration of zinc plus 2 over the concentration of copper, which gives you a value of 1 .
0 times 10 to the 2.
So, just a review of q, if you need help calculating q's, we're going to have to extra problems for the exam coming up to be posted on Friday.
So, this'll be used in several units.
So then you need to know n, and then we have everything to go back to the Nernst equation.
So how many moles of electrons are involved in this?
2.
Sometimes it's not so obvious, so this can trick people, so make sure that you pay attention to this.
This one is a pretty obvious one.
All right, now we have everything that we can put in.
You calculated the standard potential for the cell, you calculated q, and you told me how many moles of electrons there are.
So we can go and put this in.
So we calculated positive 1 .
0 3 volts minus the gas constant times the temperature, it was at room temperature, natural log of q.
We had two moles of electrons and Faraday's constant here.
And now if we do the math, this whole term comes out to 0 .
0 5 9 2 volts, and we get our answer, which is a positive number there.
So, just a note about units and constants.
Where did volts come from here?
Well, the moles canceled out, and the kelvin canceled out, and we are left with joules per Coulomb, and conveniently for us, joules per Coulomb is a volt.
So, all our units add up here.
And just a note about a significant figures in doing these problems, the Nernst equation, boy, significant figure fun.
If you want to make sure you know significant figures, here you go.
Significant figure rules for logs, we have significant figure rules for multiplication and division, and then significant figure rules for subtraction.
So, one equation gives you every type of significant figure rule.
So, that can be a lot of fun as well.
All right, but I'm going to try to make it easier for you on an exam and avoid some math mistakes.
All the problems I'm going to give you are at room temperature.
And so, the gas constant is a constant, temperature for these problems is going to be a constant, always room temperature, Faraday's constant is a constant.
So, all of these, I'm going to give you the value that they all add up to, and if you use log instead of natural log, there's this value as well.
So, these will be given to you on an exam, so you'll see these equations on the exam as well and you can use them.
So, this is for natural log, this is for log, so we are not going to test your ability to multiply room temperature times the gas constant divided by Faraday's constant.
So that's going to make it a little easier in doing these problems.
All right, so what about at equilibrium.
What does q equal at equilibrium?
So, q equals k at equilibrium.
What does delta g equal at equilibrium?
Zero.
And so that means that we knew before, from this equation, when this is equal to zero, then delta g knot equals minus r t, natural log of k, so q equals k at equilibrium.
And so we had this equation that we used before.
We now have this equation, so here it related delta g knot to the equilibrium constant, here we relate delta g nought to delta e nought And so, I think you can see what's coming.
We are going to relate these two together and come up with an expression where are you can calculate equilibrium constants from standard reduction potentials.
So, you may be asked to do this as well.
So everything in these units are, in fact, related to each other.
And the only thing we have left today, the answer to this question about how B12 is reduced in the body.
So, you're just going to have to wait to find out, don't worry you won't get heart disease between now and Friday, and I'll let you know how it all works out on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, we have the clicker question up.
We're going to have another clicker competition today.
The defending champs, there's Darcy's recitation.
So, OK, let's just take 10 more seconds on this.
OK, not too bad, 74%.
So, in this problem, the trick was just to look at the equation and figure out what's going on -- which element is being reduced, and which one is being oxidized.
So this is something that you will be seeing on the upcoming exam.
So we're going to talk about that in a few minutes, but first I want to answer the question from last time.
So, first, let's just, I know you've all been wondering about how vitamin B12 is reduced in the body.
So, let's take a look at this now.
So, if everyone can quiet down, let's get started.
So, vitamin B12 is reduced by a protein that's called flavodoxin.
And flavodoxin is a protein that has a cofactor which is a flavin, and that's also a vitamin B, as it turns out, they're both vitamin B's.
So, vitamin B12 has a redox potential or a standard reduction potential of minus 0 .
5 2 6 volts, and that is a very low number for a biological system.
And flavodoxin has a potential of minus 0.
2 3 volts.
So, which of these things is a better reducing agent?
So, which thing, you want to think about reducing agent, which thing would prefer to be oxidized and reduce something else?
So, with the low negative number, B12 is a better reducing agent than flavodoxin, and just based on these numbers, vitamin B12 should be reducing flavodoxin, not the other way around.
So, how does this work then?
So, in your body right now, you have a version of a flavodoxin protein, and you of vitamin B12 attached to an enzyme, methionine synthase, and for that to be activated, it needs to be reduced.
So how is this happening?
So, let's consider whether the reduction of B12 by flavodoxin is spontaneous.
It happens in your body so one might think that it is.
But let's look at that.
So, we can look at it the same way you've looked at all the other systems, we're talking about batteries, electric chemical cells -- you can use the same equations if it's a biological system as you can for any other system.
So we've seen this equation before.
We've calculated changes in the standard reduction potential for a cell, and we talked about e nought for reduction minus the e nought for oxidation, and we can use that same equation.
So we can put it in in the biological context.
The vitamin B12 is the thing that's being reduced, and the flavodoxin is the thing that's being oxidized, that's the reaction that happens.
So, we can plug those values in.
So we have minus 0 .
5 2 6 volts, minus a negative 0 .
2 3 volts, and if you put those together, you get minus 0 .
2 9 6 volts.
So is that going to be a spontaneous reaction?
No, it's not going to be.
Your value for the e is negative, which means what is true about delta g?
Positive, so it won't be spontaneous.
So, let's figure out how not spontaneous it's going to be.
How much trouble are we all in?
So we can use again, the same equation that we used for batteries that we've used in this course -- just because it's a biological system, doesn't mean the equation doesn't hold.
So our equation for delta g nought minus n, the number of moles of electrons times Faraday's constant times the change in the standard potential.
So we can plug it in.
I can tell you it's a one electron process -- flavodoxin puts one electron into vitamin B12, so we have minus 1 times Faraday's constant times the cell potential difference you just measured, which is minus 0 .
2 9 6, and if you multiply those out, then you get 28 .
6 kilojoules per mole positive.
That is a very big value -- that's not spontaneous and it's not a small number for a biological system.
So, why don't we all have heart disease and megaloblastic anemia?
Those are problems associated when this particular enzyme is not functioning.
So, what happens in this system is what happens in a number of biological systems.
How do you drive something forward that is not spontaneous?
And what you can do is put energy into the system to drive that non-spontaneous reaction.
And in this case, the energy that's put into the system is from a molecule called s adenosylmethionine.
And the cleavage of s adenosylmethionine has a delta g nought of negative 37 .
6 kilojoules per mole.
So it's more favorable than the reduction of B12 is unfavorable, and so this drives this system.
So, s adenosylmethionine is your friend, it helps in the body for B12 to be reduced so that you can function and be healthy.
So, many biological systems work like this.
So what have we been calling something, a cell, in which an unfavorable reaction is driven by applying some kind of energy or current, what do we call that?
Two types of cells we mentioned.
One that has a favorable, it has a spontaneous reaction.
A cell that has a spontaneous reaction is called what?
Galvanic.
And the other kind is called?
Yup.
So, electrolitic cell.
So this is sort of the biological equivalent of an electrolitic cell where you have s adenosylmethionine cleavage coupled to an unfavorable reaction to drive that reduction of B12 by flavodoxin.
And B12 has such a low potential, that there really isn't anything else that can reduce it.
It has one of the lowest potentials known in a biological system, so nature said, OK, well, we're not going to make something with a lower potential to do this chemistry, we'll have something else with a higher potential, but will drive the reaction because it's going to be non-spontaneous.
So that's how this works.
So, today there's a long list of topics, all of these are pretty short and basically constitute the introductory material in this unit.
And we've jumped ahead, this is in chapter 16.
So, I really like transition metals, because I am a fan of metals that are involved in biological systems.
So, I just thought as part of chemistry, I just want to sort of review the kind of basic areas of chemistry for a minute.
This is not in your handout, but just so you sort of think about what are all parts of chemistry.
There is organic chemistry -- does anyone know what organic chemistry concerns itself with?
Carbon.
And then what we have known as inorganic chemistry.
Anyone know what that is?
Not carbon.
So, a lot of people who are inorganic chemists study transition metals, but it's basically other things besides carbon.
And then one of the areas that I like is bioinorganic chemistry, and these are people who study metals in biology.
So, people who study metals in biology, and we're also kind of referred to as, we're sort of in a club, which we call the MIB.
So, some of you -- and Will Smith did a disservice -- people now associate this with hunting aliens, but in fact, people who are in MIB are associated with hunting for metal ions in biological cells.
So this is sort of the true MIB.
And in an honor of this discussion today, I am wearing a teeshirt from one of our meetings.
This is from the International Congress on Bioinorganic Chemistry, which we refer to as ICBIC.
And you notice that the clever people who made this teeshirt used the B from the bioinorganic to make B12, which is a very popular vitamin in the bioinorganic community.
So, metals in biology.
Carbon, carbon's a good thing, amino acids are good, I like proteins.
But often, when you take a metal and attach it to a protein, that protein can do really cool chemistry -- really need oxidation reduction chemistry.
So, here is part of the periodic table that includes many of these transition metals that we're going to be talking about.
And things that are in orange here are metals that are very important biologically.
Some of the things that are in grey here are metals that are used as probes or as drugs in biological systems, and some could be both.
So, when you add a metal to a protein you can do cool stuff.
What can you do?
Well, you can split nitrogen.
You learned that the triple bond of nitrogen is pretty hard to break, but metals in proteins can do it.
You've heard of hydrogen fuel cells and things like that.
There's a protein called hydrogenase that uses metals to do chemistry with hydrogen.
You could you radical based chemistry.
You can do a lot of really great things when you have a protein that has a metal in it.
So, let me introduce you to some of the concepts and terminology involved in this, so we're going to have a metal and the metal's going to be bound to stuff.
So, one of the great features of transition metals is their ability to form complexes with small molecules or ions.
They also, this applies to a protein system in the case that instead of small molecules, you have amino acids of proteins it can also form complexes with.
So, the way that they do this is metals have often, they can attract electron density, usually a lone pair of electrons from another atom, and so they can form what are called these coordination complexes.
So, now we're going to review for a minute, donor atoms, they're called ligands, and we can think about something that we learned when we talked about acids and bases, our more broad definition.
So, donor atoms are ligands, and ligands are what?
Lewis acids or Lewis bases, and what do they do?
OK, let's just take 10 more seconds.
So most of you thought donor atoms -- the answers that had donate electrons did better, so that was good reading of the question, but they are Lewis bases.
So Lewis bases donate electron pairs.
So, here's some examples that you will see of ligands that we'll be talking about in this unit.
You will become very familiar with some of these as you work problems.
So they're typically donating one lone pair of electrons.
So, acceptor atoms are the transition metals themselves, and so the transition metals are acting as Lewis acids, and so then they would be accepting those lone pairs of electrons.
So, you can think about coordination complexes as Lewis acids and Lewis bases, or acceptor atoms and donor ligands or donor atoms.
So here are some examples of transition metals that we're going to see, so it's that sort of d-block of the periodic table that we'll become very familiar with in this unit.
So, then when you take your acceptor atom and your donor atom and you put them together, you get what is known as coordination complexes.
So, coordination complex is just the metal or the Lewis acid surrounded by the ligands, or the Lewis bases or the donor atoms.
And here is an example of a coordination complex.
We have metal, we have cobalt in the middle, and it's surrounded by a series of ligands, and these are n h 3 ligands.
And so then our cobalt in the center, our metal, is the Lewis acid, and it's going to be the acceptor atom.
And the n h 3 groups are our Lewis bases, they're donor atoms, see it's written with those two electrons there, that they're sharing their electrons with the cobalt forming this coordination complex.
And in this scheme, we have the bonds are just these straight lines are in the plane, the thick bonds coming out are coming out toward you, and the dashed lines going back are going back into the screen here.
So, let's talk about a couple of definitions here.
We have something called coordination number or CN number, and this is simply the number of ligands bonded to the metal.
So the number is six, there is six ligands.
Typically, the numbers will range for these coordination complexes to two to 12, with six being the most common.
So, here is some notation.
If you see this picture, you should be able to write a notation for it.
And within a bracket, you would write cobalt, and then you have your n h 3, six of them, so you put that in a bracket and indicate that there's six n h 3 groups.
Then you have this overall bracket here with a charge up above.
And so over here, we have this little bracket plus 3, that means that this whole coordination complex has a charge of plus 3.
Because it has a positive charge, sometimes coordination complexes are associated with counter ions.
So, you might see three chloride minus 1 ions around to counter that charge.
And if you did, you would see the three chlorides on the outside of the bracket.
So things inside of the bracket are actually coordinated to the metal, things outside the bracket are counter ions.
And so that would be how you would translate that particular notation.
All right, so, we're back to geometry again.
As I said, everything you learn in this course, we're going to use again.
So, if you're forgetting some material from units 1 and 2, this would be a good time to review those.
So, if we have this coordination number of six things around, what kind of geometry are we going to have?
All right, 10 seconds.
Yup, so we have octahedral geometry.
And here is an example of our octahedral geometry.
So, let's just quickly run through the rest of these.
You can yell out the answers, either looking at your handout or not looking at your handout.
The next one over here, what is it?
Right, trigonal bipyramidal.
For a c n number of five, we have another option shown over here, what's that one called?
Square pyramidal.
For c n numbers of four, there's two things that you'll commonly see, what's the first one called?
Square planar.
The second one?
Tetrahedral.
For c n number of three, what's that geometry called?
Trigonal planar.
And c n of two, only one option.
Linear.
All right, so let's just review the angles as well, those are not in your handouts, but you can yell those out as well.
So what angle do we have in an octahedral system?
90.
There are two angles involved if we have trigonal bipyramidal.
What's the angle from the axial to the equatorial?
90
And what's the angle from one equatorial atom to another equatorial atom?
So, we have 90 and 120.
What about in our square pyramidal system, what are the angles here?
90.
What about square planar?
90.
Tetrahedral?
Let's try that again with more enthusiasm.
Awesome, 109 .
5.
Trigonal planar?
120.
And finally, linear.
180.
So, we're going to use this information again in this unit, and of course, it'll be on the final when we're talking about Lewis structures and hybridization and other things as well.
So this comes back in several times, vsper theory, you have this several times.
OK.
So, another term that you hear a lot when people are talking about coordination complexes is what's called the chelate effect.
So, ligands that bind to a metal at one point are called unidentate or monodentate, and that's from dent, which is tooth, so one tooth.
Now, if a ligand attaches with two or more points of attachment, it can be called a chelating ligand.
And this comes from the Greek, chele is claw.
So, if it's attaching with more than one point of attachment, if the ligand is grabbing on, it's like a claw and that's called a chelate.
So, if you have two points of attachment, that's called bidentate.
And even if you've never had a unit on transition metals before, I bet that you can answer this without me even telling you.
What do you think tridentate might be?
Three.
Tetradentate?
Four.
Hexadentate?
Six.
So sometimes this would be a little question on the final.
Please don't get this wrong, this is my gift to you, right, you knew it even before I taught it.
So, this should be something that you can definitely get a couple extra points on on the final.
All right, so, chelates bind with more than one point of attachment, and metal chelates are unusually stable.
And this property is due to favorable and tropic factor, so we're back to entropy, we're back to thermodynamics.
And this has to do with the fact that when a chelate binds a metal, it releases a lot of water, and that makes the chelate pretty stable.
So, let me give you some examples of chelates, and then we're going to come back and think about the chelate effect again.
Of course, you knew it, vitamin B12 has a chelating ligand associated with it.
So, cobalt is in the middle of vitamin B12, and it has a ring system around it.
And the ring system has these nitrogens, the nitrogens are the donor ligands to the cobalt, and so this ring system is attaching at four points, so it's a tetradentate ligand.
There's also two other ligands attached to vitamin B12, there's an upper ligand shown here, which is 5 prime deoxyadenosine, and a bottom ligand, which is a dimethylbenzamitisol ligand.
So, let's take a look, this is the cartoon, but let's look at another model for this here.
So, we have a ring system, we have this upper ligand, and then a lower ligand down here.
And so, the middle ring system is a tetradentate chelate.
Overall, at the cobalt metal, what is the geometry of this system?
It's octahedral, right.
So we have this sort of square, the middle part is square planar, but the two upper and lower ligands make the whole thing an octahedral system.
So this is an example of a naturally occurring chelated complex.
So, this structure is actually fairly complicated for a vitamin.
It's one of the most complex vitamins that's known, and I thought I'd just mention just a moment of history.
So, this structure was determined by Dorothy Hodgkin who was a crystalographer in England.
And she started working on this in the late 1940's, and people told her she was crazy, that something this large could never be solved by these x-ray defraction techniques.
And, of course, now people are solving today's structures of things like the ribosomes.
So, we've come a long way with crystallography.
But she was one of the first true believers that x-ray crystallography was a powerful technique.
And so, she went ahead, despite what everyone said, and determined this structure.
And for this work and for some other things, she was awarded a Nobel Prize.
So, this was a really significant contribution in the early 1950's and late 1940's.
She had graduate students that she inspired.
Some of them went on to become famous crystalographers.
Other ones were not so successful -- crystallography's hard and it's not for everyone.
She was actually known also, Dorothy was, for her work in the third world and her liberal politics, but one of her graduate students did not agree with her politics, but they were still friends.
So this is just a message that as you go through your career, Margaret Thatcher who is a graduate student of Dorothy Hodgkins, couldn't quite cut it as a crystalographer, but for some people, you haven't found your true thing, and she was able to find another job, so I'm told.
So, just remember that if you're not good at one thing, there's something else out there that you might be good at.
So, let me give you a second example of a chelate.
This molecule is known as EDTA.
So this ligand has six points that it can attach to a metal.
So, it is six atoms the can serve as these donor ligands.
So, we have one up here, have the little electrons and the oxygen, another oxygen down here, nitrogen, nitrogen, another oxygen, another oxygen.
So all of those six points can coordinate to a metal.
So here again is the free EDTA, and here's a structure of EDTA bound to a metal, the metal is abbreviated m. So, why don't you tell me what the geometry is of that metal?
All right, let's take only 10 more seconds.
Hoping for 90, we're getting close, maybe by the end of today.
All right, yeah, so it's octahedral geometry.
So we have upper ligands and a lower ligand, two ligands coming out, two ligands going back, and it's also a hexadentate complex.
So we see we're coordinated with the oxygen in red, nitrogen in blue, around to the other nitrogen in blue, another nitrogen in purple, another nitrogen up here in green and in blue.
So, EDTA, when EDTA binds to a metal, it forms a very stable metal complex.
And the reason for this is that chelate effect, and it's entropy.
So if we look, metals are often coordinated by waters if they're just is solution.
But if you bind one molecule of EDTA to the metal, it'll take all of those six sites, displacing all of these six waters.
So here, on this side, you have one thing free in solution, and over here you have six things that are free.
So that's entrotropically favorable, you have more entropy when you have lots of more free things floating around.
So this has greater entropy.
So, the chelate effect, then, has to do with entropic effect, that when you're binding a metal with multiple points of attachments, it's releasing water.
So these are very stable.
And so, because of this stability, they have a lot of important uses.
What is one use you can think of that you might have for such a thing?
When might you want to chelate a metal out -- get metal out of something pretty quickly.
Yeah.
Yeah, you can purify out a metal out of water with this.
What about rushing to the emergency room with something?
Yeah, lead poisoning, yup.
So, sometimes you might want to just purify your water, you might be happy with that, other times you might really want to have that metal out.
So, every emergency room in the United States, and probably mostly around the world, will have EDTA on hand in case someone comes in with lead poisoning.
Do you know who's at the most risk for lead poisoning, at least in this country?
Kids.
And what do they do they gives them lead poisoning?
Lead paint, yeah.
So, I don't know, most of you are probably living on campus, but if any of you move off campus, and some people who are living maybe across the river in fraternity houses, those buildings are old and they have had lead paint in at some point or another.
So this is a big problem, actually, in the Boston area.
So, if you have any toddlers visiting you, you might want to make sure they're not eating paint chips off any window sill or anything like that.
Or if you do, have some EDTA on hand.
All right, one other thing, some of you, once you start studying chemistry, it's always fun/scary to start reading the ingredients on food packages.
But you will discover that some food that you eat will say as an ingredient EDTA added for freshness.
So, bacteria and fungi and things like that need metals for growth, metals are very important for life, and so you add a little EDTA and it prevents things from growing on your food.
So that's added for freshness.
So you'll see it in food.
Another use that occasionally I find MIT students are not as familiar with is in cleaning bathtubs, so you want to chelate out the calcium in tub scum, and that's a good use for EDTA.
So, I always like to mention how freshman chemistry information can make you a lot of money.
So, I thought I'd tell you a little story about a man named Robert Black.
So one day, I think it might have been a little before Thanksgiving, Mrs. Robert Black, I don't know her first name, said to her husband, "We're having company, how about you go clean the bathtub?"
Apparently, Robert Black's wife had never mentioned this to him before, and he had, in fact, never cleaned a bathtub.
Also, I think that Robert Black's wife was tired of cleaning the bathtub, and perhaps, had not done it for, say, a very long time.
So, Robert Black went in there and tried to scrub off the scum, and it was actually really challenging and he was very frustrating, and he said, "I never want to do that again."
So, he went and developed a product, a shower cleaner, that had all the ingredients of the shower cleaners he was using, except that he advertised it slightly differently, and he said "Every time you shower, just spray a little bit on and you'll never have to really scrub again."
because this will take care of it and avoid scrubbing in the future, so a little bit of spraying after each shower avoids these problems.
And the ingredients he had in were all typical ingredients, including chelating agents.
You need a surfactant to break up the water beads, and alcohol to get rid of the oily stuff, and the chelating agent for the calcium in the tub scum.
And the one he actually used is EDTA.
So, as a result of a wife saying to her husband, it's your turn to clean the tub, they now make $70 million dollars annually.
So, I think this is a lesson that we can learn in many ways that cleaning the tub is something that everyone should do at some point in their life, and that the simple things you learn in freshman chemistry can, if applied correctly, make you a lot of money.
And I just want to encourage you, again, that anything that I teach you that you use to make a lot of money, I'll remind you that I am perfectly willing to have a cut of any of that.
So, just something to keep in mind.
So, one more thing that you might have heard for EDTA.
Have any of you heard, perhaps, maybe watching television or a movie, about a use for EDTA?
Have any of you, perhaps, seen a movie called Blade?
How many of you have seen this?
Not very many people.
Hmm, I think we're going to have a separate course in vampire movies and TV.
Anyway, Blade fights vampires, and they have a special gun, and the special gun has cartridges -- cartridges are shown here.
And the cartridges have in them -- anyone want to guess?
EDTA, yes.
And the idea behind it, if you shoot at a vampire, the vampire turns instantaneously to dust, and the idea behind it is that vampires drink blood, blood has what in it?
Iron.
What chelates iron?
EDTA.
And, therefore, if you chelate the iron out of blood, and a vampire's mostly blood, it just instantaneously turns to dust.
So, actually, as chemistry goes, it's kind of cool, yeah.
So, that's the final use that I am aware of for EDTA.
Some of you may encounter some more.
All right, so coordination complexes.
Some of them can have isomers.
They can have geometric isomers.
And geometric isomers can have vastly different properties.
And if you're interested in biochemistry, this is something that you'll be interested in, because this can be very important in biological systems.
So, let me tell you about a coordination complex that has platinum at the center, it has two ammonia ligands and two chloride ligands, and it has two ways that you can do it.
So, you can either put the ammonia ligands on different sides or you can put them on the same side.
So, here we have two nitrogen ligands over here, and I have these here.
So, on one side, you have the two nitrogen ligands, on the other side two chlorides, or we can have a transarrangement where they're across from each other.
So you see, there's 2 different ways to do it, on the same side or sort of across.
So, cis or trans.
Cisplatin is a potent anti-cancer drug, transplatin has no use that anyone's been able to detect.
So it's the exact same composition, but a different arrangement of the atoms coordinated to the central metal.
So, why should it make a difference, they have the same ingredients, why should one be a potent anti-cancer drug and the other one be useless?
Well, it's because of how it interacts in the body.
And so, one of the people who have spent a lot of their career studying cisplatin is Professor Steve Lippard, who's in the Chemistry Department here, and he has determined x-ray structures looking at this, and done a number of other studies.
And so, here is a little cartoon of cisplatin bound to DNA, so the chlorides are displaced when it binds, and so they need to be on the same side, otherwise it's not going to work.
And when cisplatin binds to DNA, then this will act to kill the cancer cells.
So, the chlorides both need to be on the same side so a combined DNA.
If they're on other sides, that's not going to interact with DNA, and so it doesn't have any known function.
So, geometric isomers, same composition, but in different arrangements can have vastly different properties.
So you can have what are called optical isomers or enantiomers, and so these can our mirror images of each other, but they're not superimposable.
And so, when you have a mirror image like this, it's called a chiral molecule, and this is a term that you'll hear a lot when you take organic chemistry.
So, I have two mirror images up here, so these are mirror images, so the mirror plane is between these molecules.
But these molecules may look the same in the mirror, but they're non-superimposable.
So, they are, in fact, different molecules, and they can have different properties in a chiral environment.
What do I mean by a chiral environment?
Well, the human body, for example, is a chiral environment.
So that's why a lot of drugs, people are wanting to make just one enantimer of the drug and not the other, one enantiomer can have good properties, the other one may not.
All right, so let's do some d-electron counting.
This is the final thing we need to talk about in coordination complexes.
So we're going to refer to a thing called the d-electron count of the metal, which just has to do with a group number from the periodic table minus the oxidation number of the metal, is going to tell us the d-electron count.
So, if you haven't learned how to do oxidation numbers yet, you need to know that for exam 3, and you need to know that for this unit.
So, let's look at a few examples of this, and we'll put up our friend, the periodic table.
So let's look at the complex that we saw on the first page where we had -- in the beginning of today's lecture where we had cobalts with six n h 3 ligands in a coordination complex that had a charge of plus 3.
So here, we need to figure out the oxidation number of the cobalt, and for this particular group, the overall charge of that is 0.
We have three hydrogens at plus 1, and we also have a nitrogen at minus 1.
And so, overall that charge is 0.
So we have six things of 0, and what does that leave for our cobalt charge?
Plus 3.
Because overall, it now has to equal plus 3.
So, then our d count for our electrons is going to be the group number.
And for cobalt, what's the group number?
9.
So minus 3 is 6, so this cobalt in this complex is what's called a d 6 system.
All right, let's look at a couple other examples here.
Let's look at nickel with carbon monoxide ligands, four of them.
What would nickel be in this case?
Someone said zero.
That's right.
So this is zero, and nickel is zero, the overall charge is zero.
So, our oxidation number here is 0.
So, our d count here, what's the group number for nickel?
10.
10 minus 0 is 10.
So we have a d 10 system.
So, I'm going to write another example down and you're going to tell me the answer to this one as a clicker question.
So we have cobalt, two waters -- three waters -- that's actually a mistake.
It should be two.
Two waters, because it's going to be six things -- this is not some kind of bizzaro seven complex.
So here's the correct, cobalt, two waters, one ammonia, and three chlorides with an overall charge of minus 1.
All right, let's just take 10 seconds, since we're running out of time.
So that's right, we should have a charge of plus 2 here.
This is zero, 0 minus 3, overall minus 1, so we have a plus 2 state.
We have 9 minus 2, which is 7, and it's a d 7 system.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so we started talking about transition metals, and at the end of last time we'd given an introduction to all the sort of nomenclature and things you need to know, and we had just gotten up to d orbitals, and when you're talking about transition metals you're talking about d orbitals.
So, we're going to start with a little review of d orbitals, some of you have seen this before, maybe some of you have not.
But here is what you're going to need to know about the d orbitals.
You need to know the names of all the d orbitals, you should be able to draw the shapes of the d orbitals, and so the bar is not too high for this.
You see the drawings that are in your handouts, you should be able to do about that well, it doesn't have to be super fancy.
But you should be able to draw their shapes, and you should also be able to recognize which d orbital it is -- if you have a picture of different d orbitals, you should be able to name that d orbital.
So let's just review what the d orbitals look like.
And in this class, we're always going to have the same reference frame, so we're always going to have the z-axis up and down, the y-axis is horizontal, and the x-axis is coming out of the board and going into the board.
So, we'll always use that same description.
It'll often be given, but if it's not, you can assume that's what we're talking about.
So, the first d orbital we'll consider is d z squared.
It has its maximum amplitude along the z-axis, and it also has a little donut in the x y plane.
So, d x squared minus y squared has its maximum amplitude along the x and the y axis, so directly on-axis for this particular d orbital.
The next sets of d orbitals have their maximum amplitudes off-axis, so they don't correspond directly to the axes that I just mentioned, they're, in fact, 45 degrees off-axis.
So we have the d y z shown here, d x z shown here, so maximum amplitude 45 degrees off z and x axes, and then z -- these pictures are a little bit complicated to see -- d x y, so it's 45 degrees off of the x and the y.
And so, here I tried to draw them all in the same orientation of axes, which is a little bit difficult.
So now let's look at them in terms of where they're drawn so you can kind of see them a little bit better, and so why don't you try to learn to recognize all of these.
So, what is this one called?
d z squared.
What is this one, so its maximum amplitude is along x and y?
d x squared minus y squared.
I think this picture might be a little -- you might have somewhat aversion later on, but this is just good practice for you in recognizing them.
So here we have one 45 degrees off-axis, which one is this?
Yup.
And this one over here?
Yup.
So, y z.
And this last one here?
Yup, we have the x z, so it's going up and down for the for z-axis.
So, a little more practice now.
To show what these look like again, you want to think in three dimensions, and on paper and in most the time in Powerpoint you're not in three dimension, so here's a little movie in three dimensions.
Here you can really see that donut, so this is d x squared -- see the maximum amplitude along z-axis here, and down here, and the little donut in the x y plane.
So this one is d x squared minus y squared.
The maximum amplitudes are right directly along axis, so that allows you to distinguish it from d x y.
So when it's on-axis here, it's the x squared minus y squared.
So moving along here, so this is d x y, so you see that it's in that axis but it's not directly on-axis -- the maximum amplitude is 45 degrees off, so the orbitals are in between the axes there.
Now we're looking at one that has a z in it, and it looks like it's x z, so that's where our maximum amplitude is between the x and the z-axes, 45 degrees off.
And our last one, we have y and z here.
Again, 45 degrees off-axis between the y-axis and z-axis.
So, hopefully these little movies will help cement in your brain, what the shapes of these d orbitals are.
All right, so that's d orbitals, and we're going to be mentioning d orbitals in every lecture in this, and you have to be thinking about what the shapes of the d orbitals are to talk about today's topic, which is crystal field theory.
So, there are two types of theories that you may hear of and that your book mentions -- crystal field theory, and ligand field theory, and like most things that you learn about in freshman chemistry, the theories were developed to explain experimental information.
So there are special properties of coordination complexes, so that's where you have a transition metal in the middle and you have ligands all around it, so you have these coordination complexes and they have special properties.
And so people wanted to try to rationalize these special properties and they came up with these two theories.
So, the basic idea behind these theories is that when you place a metal ion with the particular oxidation number in the center of a coordination sphere, and you have all these ligands, these donor ligands, all surrounding them, that the energy of the d orbitals is going to be altered by the position of those ligands.
So it's all about the d orbitals, and the d orbitals are going to experience some influence from these ligands, these donor ligands that are surrounding the metal.
So then, between these two theories that are used to explain how these d orbitals are being affected.
The crystal field theory is based on an ionic description, so it considers the ligands as negative point charges.
It's a very simplified model, whereas as the ligand field theory considers covalent, as well as ionic aspects of coordination.
It's more powerful it's more useful, but it's also a bit more complex, and so we don't cover it in this of course, and if you go on and take the first level of inorganic chemistry, which is 503, then you'll hear about this.
But for this course, we're just going to talk about crystal field theory.
Even though it's very much of a simplified model, it actually works very well.
You can explain quite a few properties of coordination complexes just using this simplified method.
So, crystal field theory, again, very simple.
It's just considering the ionic interactions, it considers the ligands as negative point charges.
And so, the basic idea is that ligands, as negative point charges, are going to have repulsive effects if they get close to the d orbitals.
So here is a drawing of a metal, and so this is metal abbreviated m, its oxidation number is m plus here, and it has ligands all around it.
What is the geometry here?
It's octahedral geometry.
And so we have ligands up and down along z, ligands along y, and a ligand going back along x, and a ligand coming out along x.
And so here's another picture of the same thing, the metal is in the middle, and the ligands -- in this case, you have these ammonia ligands or these little negative point charges, which are all along the axes.
You have four along the equatorial, and one up and one down, so this is the octahedral geometry.
And so you can just think about each of these ligands as negative point charges.
And so, if the negative point charge is pointing right toward a d orbital, that'll be very repulsive.
If it's not it's less repulsive.
That's the whole idea behind this crystal field theory.
So, here again, is just another little picture, so you can kind of get the idea that we're going to be thinking about the all the shapes of the d orbitals, and we're going to think about where the ligands are.
Today we're going to talk about octahedral geometry, but we're also going to go on and talk about tetrahedral geometry later.
So here in octahedral geometry, you can think about the positions of all of these negative point charges surrounding your d orbitals.
And when the d orbitals are on axis, like the ligands, there's going to be more repulsion, so you can see here that would be quite repulsive -- you have a negative point charge by that d orbital.
When the d orbitals are off-axis and the ligands are on-axis, that's less repulsive.
And that's the basic idea.
So, let's look at each one of these orbitals now in detail and think about how a ligands that's pointing directly toward it is going to be affected.
So we have the ligands, l, as these point charges directed toward the d z squared, and the d x squared minus y squared orbitals, and these would result in quite a bit of repulsion.
So if you had a ligand right up here along z, and so that would be a very close interaction.
In this case, you're going to have ligands along x and y, again pointing directly toward the orbitals, that would be quite repulsive.
And I'll just mention, we'll come back to this later, that one can think about the case where you have the octahedral geometry where the ligands are in a definite position, and you can also think about this sort of hypothetical case where you have a metal in the middle and you have the ligands, here are the little ligands, and they're everywhere, there's ligands everywhere all around.
And so, in this case where you have ligands everywhere all around your metal, then all your d orbitals would have the same energy, but if you take the ligands and you isolate them in particular positions, then you can consider how the different shapes of the d orbitals will be affected.
We'll come back to that in a minute.
All right, so here we have a case where our ligands are on-axis, our orbitals on-axis, this is a large repulsion.
So, I will tell you that d x squared and d z squared and d x squared minus y squared orbitals are destabilized, and they are destabilized by the same amount.
So there's repulsion now, and so they're destabilized, and they're destabilized by the same amount of energy.
So what's it called when orbitals are of the same energy?
Yup.
So, d z squared and d x squared minus y squared are degenerate.
So, d z squared and d x squared minus y squared orbitals are destabilized more than the other three orbitals, and let's consider now why that is true.
So here are our other sets of orbitals, and remember, here the maximum amplitude of these orbitals are 45 degrees off-axis, whereas our ligands are all on-axis.
So, the ligand negative charges are directed in between these orbitals, not directly toward them.
So that is stabilized compared to this hypothetical case where the ligands are everywhere, so some of them will be pointing toward them, and also stabilized compared to the other sets of orbitals where the ligands are now pointing directly at them.
So these three sets of orbitals are stabilized relative to the d z squared and the d x squared minus y squared orbitals, and they're stabilized by the same amount.
So these three orbitals are also degenerate with respect to each other.
So then to sort of summarize this set of orbitals, we have for d z squared and d x squared minus y squared, we have large repulsions by those negative point charges, they're pointing directly at the orbitals, and so they're destabilized, higher in energy than the other -- the d x y, d y z and d x z.
For the d y z, d x z and d x y, they're smaller repulsion, because these orbitals are off-axis, and so the negative point charges aren't pointing directly at them.
So they're stabilized relative to these guys up here.
So that's the whole idea behind an octahedral case of crystal field theory.
And we can look at this just one other way, if pictures help you.
Here it's a little clearer that those negative point charges are pointing directly toward the orbitals, here I think you can see that the negative point charges are not directly pointing toward any of the orbitals.
So I'll show you a bunch of different figures, this all shows you the same thing, but some might help you see this relationship better.
OK, so now we're going to draw some diagrams.
I'm going to start over here.
So we're going to draw what's called a crystal field splitting diagram, and this is for an octahedral case.
And the diagrams are going to look different depending on what the geometry is.
So when we draw this diagram, energy is going up, and we're going to start with our 5 d orbitals, and so this is going to be the average energy, the average energy of our d orbitals.
And so, this is then with a spherical crystal field.
So that's where the ligands are distributed around uniformly.
So it's all spherical, they aren't set up in the octahedral case yet, our octahedral diagram's going to be over here, but this is the case that this represents.
If you have all your ligands spherically distributed around your metal, then the energy of all the d orbitals are identical, because every d orbital has the same amount of ligands, it's uniform, it's symmetrical all around the metal.
And I just want to tell you that this is was very exciting to me when I saw this.
I've been teaching this class for a while, and I never had a real spherical crystal field around my metal before.
And then I walked into Walgreens one day, and I was very excited to see that Walgreens sold spherical crystal fields.
I mean you never know what you're going to get.
I'm a big fan of Walgreens, I've found a lot of good stuff, toys for my dog, etcetera, but this was really amazing.
So I asked the cashier on the way out whether they knew they were selling spherical crystal fields, and they did not actually.
So, you just never know what you're going to get.
OK, so in that case, where the ligands are uniform all around, the energy is the same.
But now, if we have an octahedral crystal field over here, so we have our octahedral crystal field, then we get some splitting.
So some of our orbitals are going to be destabilized, and they'll be higher in energy here.
So we have the d x squared minus y squared, and d z squared over here are going to be higher in energy.
And we're going to have three that lower in energy, so we'll have our d x y, our d y z, and our d x z over here, will be lower in energy.
This difference is called the octahedral field splitting energy, because it's the amount of energy that the octahedral field is split.
So over here, we can put this is for the octahedral case, crystal field splitting energy.
And again, some of the orbitals go up in energy, some of the orbitals go down in energy, and the overall energy needs to be conserved.
So, if two orbitals go up in energy, and three go down in energy, then to have everything add up, you can say that three go up in energy by 3/5, and two, these three orbitals are going to go down by 2/5.
So overall, the energy of the system is maintained.
OK, so that's a crystal field splitting diagram for an octahedral case, and now let's look at some examples of this.
So let's look at an example, and we're going to have a chromium system that has three n h 3 ligands and three b r ligands.
Now, you tell me what the d count of that is.
Let's just take 10 more seconds.
They're not as high overall, but still more people got the right answer.
So, let's take a look at this.
What's the oxidation number of bromium?
What is it?
Bromium?
What's 1 b r minus?
What's its oxidation number.
Minus 1.
There are three of them.
What about ammonia?
0.
So, 3 times 0.
And the overall charge of this is 0, so there's nothing up there.
So what does that mean about chromium?
Plus 3.
All right, so now we have to figure out the d count.
So the d count is going to equal the -- and the periodic table, the group number, so if you switch to my slides, we can see what that is.
So what is that for chromium?
6.
And then we have 6 minus 3, because our oxidation number is 3, and so we have a d 3 system.
So, did some of you get this wrong because you stopped too early?
Here, the answer, if you had 3, you could have stopped with oxidation number and still gotten it correct.
So that's a d 3 system.
So, we're going to worry about three d electrons, and we're going to put three d electrons into our splitting diagram.
OK, so if you had a hypothetical spherical crystal field, you would have your one's in here, but now let's consider what happens in the octahedral case.
So we can come down here.
Am I going to put my first electron down here or up here?
Down.
Oh, I just realized that I didn't put two things on this diagram, so these are in your notes, but these diagrams have little abbreviations in them for the orbital levels.
So we have an e g and t 2 g, and that's an abbreviation for the names of the ts of orbitals, which you'll see later is very convenient in terms of writing things out.
All right, so we're going to put them down here.
Am I going to put two of them together with the spins up in the first orbital?
No.
So you know that that is not good, give you the same four quantum numbers, you don't want to do that.
So, we can put them in.
What about just putting in a paired set over here yet?
No.
They have the same energies here, so we're going to put them in all singly, and then we're done.
So we put three electrons in these three orbitals.
Now we can introduce a couple other terms, which is where I realized I forgot to put the labels on.
So, you'll often be asked for -- OK, you'll often be asked for something called the d n electron configuration.
And so here you can use the abbreviations for the orbitals, so we point three electrons in to the t 2 g orbitals, so we can just say that's t 2 g raised to the three.
So that let's someone know that you have three electron in the set of orbitals that are stabilized in an octahedral crystal field.
Then we can consider something else that's called c f s e, and I think I should have -- OK, so I'll write that out.
So, this is the crystal field stabilization energy.
So it's not the crystal field splitting energy, it's the stabilization energy, which indicates how much those electrons are stabilized by being in an octahedral field, rather than this hypothetical spherical crystal field.
And so what we can do there is you see that you have three electrons in those lower sets of orbitals, and those orbitals are stabilized by 2/5 times the octahedral crystal field splitting energy.
And so that gives an answer of minus 6/5 times the octahedral crystal field splitting energy.
So that's how much those electrons are stabilized.
So they are lower in energy -- see, the average energy is much, much higher in this hypothetical case, but because of having this octahedral geometry, and there are only three electrons to consider, they all go into the stabilized energy, and so they're stabilized by minus 6/5 times whatever the octahedral crystal field splitting energy is for this particular case.
So, now let's look at another example.
So let's look at an example of a coordination complex -- you have manganese and you have six water ligands and some chlorides hanging around.
So why don't you tell me what the oxidation number is for this now -- not the d count, but the oxidation number?
OK, so let's just take 10 more seconds.
OK, so let's just look at that one for a minute, people did very well on that.
So, what's the overall charge in this coordination complex?
So we can write this out here.
Our six ligands and we say that here it's plus 3 overall, because we have three chloride ions hanging around with a negative charge.
So this tells us that the overall charge on the coordination complex had to be plus 3.
And this again is 0, so that means that is plus 3.
So, people did very well on that.
All right, so then what is the d count?
What is it?
4, right.
So we have 7 minus 3 is 4, so we have a d 4 system.
All right, so now, if you look up there, we have to make a decision about that fourth electron.
three electrons were easy, four makes it complicated.
Do we put the fourth one down in the lower set or do we go up?
So there are two possibilities here.
So I have two diagrams drawn over here.
And you might be in a case where you have a small crystal field splitting energy, or you might be in a case where you have a large crystal field splitting energy.
And so, there are two different ways the electrons can go.
So over here where you have a small crystal field splitting energy, that's called a weak field.
And when you have a big splitting energy, that's a strong field.
So, with the weak field there's not that much of an energy difference between them.
And so, when you're putting in, you do your first three electrons, that's always going to be the same.
But then the fourth electron, in this case, if there's not a big difference in the energy, if it's pretty small, if it's a weak field, you can put that fourth one up there.
Because it takes energy to pair the electrons up, and so you're asking the question, does it take more energy to pair them, or does it take more energy to put one in the upper set?
And for a weak field you say that the crystal field splitting energy is smaller than the pairing energy or p e, so p e is the pairing energy.
And so it takes more energy to pair than it does to bump one electron up to the higher level.
So that's what a weak field situation would look like.
Now in a strong field situation, boy, there's a big splitting difference, a big energy difference here.
So in this case, the crystal field splitting is much larger than p e, that pairing energy for the electron.
So it's better to pair, then to put one up.
I can't even reach those, that's really far up.
So I'm not going to do that, I'm just going to put them in here.
I'll put the first three in, and then the fourth one is going to go down where I can reach it.
I don't have the energy to put it up there.
So that's a strong field.
Weak field I can handle, strong field I'm going to try pair them all up.
So, now we can write the different d n electron configurations for these two.
So, in this case, if we have our d n electron configuration, so we have three in the t 2 g, we have three.
And in the e g set we have one, so that is our electron configuration for this weak field case.
And for the strong field case, we didn't put any up in the e g, so we just have t 2 g, four electrons in that set of orbitals.
OK, so let's just put up what we've done here, and introduce another term, which are high spin and low spin.
So we have these two cases here, and again, we're considering how big is this octahedral crystal field splitting energy compared to a pairing energy.
The energy involved in pairing electrons together.
In a weak field, the splitting energy is small, so electrons are placed singly with spins parallel to the fullest extent in all the sets of orbitals.
In this other case with the strong field, the pairing energy is smaller than the splitting energy -- strong field you have a big splitting energy.
And so, in that case, with the splitting energy is large, you're going to put all your electrons in and pair them up in t 2 g and don't put any electrons in the e g sets of orbitals until you completely filled your t 2 g set.
So, the net result of this is for a weak field, you have the maximum number of unpaired electrons, so see, you have a maximum number, you have four electrons, all four are unpaired.
So that's the maximum number of unpaired electrons, and this is referred to as high spin.
And in the other case, you have the minimum number of unpaired electrons that you can have, and so you have this one set that's paired here, so that would be considered a low spin case.
We can also talk about the stabilization energy of these two cases, and so we have the crystal field stabilization energy.
So why don't you go ahead and tell me for a weak field case, what is our stabilization energy?
Let's just take 10 more seconds.
OK, pretty good.
So let's work this out.
So, first we consider how many electrons we have in the lower set of orbitals, so we have three.
Those three are stabilized by 2/5.
And then we have one in the upper set, so we have one at 3/5 times the octahedral crystal field splitting energy.
So this ends up with minus 3/5 times the octahedral crystal field splitting energy.
And notice why a is not correct -- you don't have the symbol for the octahedral crystal field splitting energy.
On a test, you need to make sure that you remember to write this term, so this was a little test so that you hopefully emphasize that you want to have that there.
All right, so that's for this one.
Let's look at the low spin case, the strong field case, and do our crystal field splitting energy for that.
So in this case we're going to have four electrons times minus 2/5 times the octahedral crystal field splitting energy, so that's equal to minus 8/5 times the octahedral crystal field splitting energy, and some books will also indicate, and they'll say plus p e to indicate that there's pairing energy that's associated there.
So it's not quite as beneficial as one might think.
You do have a lot of electrons in these in lower energy orbitals, but you did have to pair some of them, so you had one, and if two of them are paired, you can see sometimes 2 p e. So, some books will do that, some books will not.
So I just wanted to mention that both are OK. And whether you're asked to write that or not, and the question will say include the pairing energy, so you know whether you're supposed to do that or not, that there is that energy associated with pairing in this strong field case.
OK, so let's look at another example, and then you should be all set to do these problems for an octahedral field.
So we'll take our electrons down.
All right, so let's at a case of cobalt plus 2.
So let's consider how many we're going to put in, the oxidation number is?
Plus 2.
And what is the d count then?
What group number?
9 minus 2 is 7, so we're doing a d 7 system.
All right, so now for the weak field case here, why don't you tell me what is the correct electron distribution.
OK, let's just take 10 more seconds.
OK, very good.
This is the weak field case, so in this case, we are going to fill up singly all of the orbitals, because the splitting energy is less than the pairing energy, so we're going to put them to the fullest extent possible before we have to pair.
So we have 7, so we're going to do 1, 2, 3, 4, 5, and now we have to pair.
We have no -- we've used up all our orbitals, so we're going to start pairing down here in the lower energy, so that would be 6, 7.
So we had no choice, we had to pair them, but because it was a weak field, we filled them all up singly first before we paired.
So now let's consider what we're going to do down here.
Now remember, this is a strong field, so there's a big splitting energy, so it takes less energy to pair them than to reach this higher level.
So first we put them in the same way -- 1, 2, 3, but now that we have the 3 in, it's better to pair than to bring them up here.
So we'll do 1, 2, 3, we'll pair them all up.
Now it's all filled, we have no other choice but to go up here.
So, we have these very different cases depending on the splitting energy.
So what controls the splitting energy?
Well, what controls the splitting energy is nature of the ligands, so we're going to be talking about that next time, and you'll recognize for certain kinds of ligands you're going to have a strong field, and other kinds you'll have a weak field.
But right now we're not talking about that, we're just showing those two possibilities.
So, for this system, is this going to be a high spin or a low spin system?
So, this will be high spin, because we have the maximum amount of unpaired electrons.
And over here we're going to have a low spin system.
We have the minimum number possible of the unpaired electrons.
All right, so let's finish this up now, and do our d n electron configurations and our crystal field splitting energies.
So, for this case, we're going to have in our t 2 g system, how many?
5.
And in e g?
2.
And for our splitting energy, we have 5 times minus 2/5 times the octahedral crystal field splitting energy plus 2 times plus 3/5, and what does that end up equaling?
Minus 4/5 times the octahedral crystal field splitting energy.
And we could also optionally put 2 p e because we have two sets paired.
All right, so let's look at our strong field system.
How many do we have in our e 2 g set?
6.
What about e g?
1.
And for our splitting energy then, we have 6 times minus 2/5 times the octahedral crystal field splitting energy plus 1 times 3/5, and what is that going to equal?
Minus what?
9/5.
And how many pairing energies?
three pairing energies, great.
I think you have that part down.
Next time we get more complicated, we're going to talk about types of ligands, we're going to talk about tetrahedral, we're going to talk about square planar, and that's, of course, after the exam on Wednesday.
So, good luck, everyone, with the exam on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, let's just take 10 more seconds on the clicker question.
OK, 76, I think that says, %, which is not bad, but we should be at 100%.
So, when you're past the equivalence point, so you've converted all of your weak, in this case, acid to its conjugate base, and because it was a weak acid, the conjugate base is going to be a weak based and so it's not contributing a whole lot it'll make the solution basic, but it's nothing compared to adding strong base in there.
So even though you have the weak base around, at this point it's really a strong base problem.
So you would calculate this by looking at how many mils of the strong base you've added past, and figure out the number of moles that there are, and divide by the total volume.
So this was like one of the problems on the exam, and one thing that I thought was interesting on the exam is that more people seemed to get the hard problem right than this, which was the easy problem.
So we'll see on the final, there will be an acid based titration problem on the final, at least one.
So let's see if we can get, then, the easy and the hard ones right.
So you've mastered the hard ones and let's see if you can learn how to do the easy ones as well for the final exam.
OK, so we're going to continue with transition metals.
We were talking about crystal field theory and magnetism, and you should have a handout for today, and you should also have some equipment to make models of orbitals and coordination complexes -- these are not snacks.
They can be snacks later, right now they're a model kit.
All right, so I'm going to introduce you to some terms that we're going to come back you at the end of today's lecture, and then we're going to talk about the shapes of coordination complexes.
So, magnetism.
So we talked last time, before the exam, if you remember, about high spin and low spin, unpaired electrons and paired electrons.
Well, compounds that have unpaired electrons are paramagnetic, they're attracted by a magnetic field, and those where the electrons are paired are diamagnetic are repelled by a magnetic field.
So you can tell whether a coordination complex is paramagnetic or diamagnetic, you can test the magnetism, and that'll give you some information about the electron configuration of the d orbitals in that coordination complex.
And that can tell you about the geometry.
And so you'll see that by the end we're going to talk about different types of energy orbitals when you have different geometries.
So why might you care about the geometry of a metal center.
Well, people who study proteins that have metal centers care a lot about the geometry of them.
So let me just give you one example.
We talked a lot about energy in the course this semester, so we need catalysts for removing carbon monoxide and carbon dioxide from the environment.
And nature has some of these -- they have metal cofactors and proteins that can do this, and people have been interested in mimicking that chemistry to remove these gases from the environment.
So let me tell you these enzymes are organisms.
And this is pretty amazing, some of these microorganisms.
So, over here there's one -- it basically lives on carbon monoxide.
I mean that's -- you know alternative sources of energy are one thing, but that's really quite a crazy thing that this guy does.
So, you can grow it up in these big vats and pump in carbon monoxide and it's like oh, food, and they grow and multiply, and they're very, very happy in this carbon monoxide environment.
There are also microorganisms that live on carbon dioxide as their energy and a carbon source.
And so these organisms have enzymes in them that have metal centers, and those metal centers are responsible for the ability of these organisms to live on these kind of bizarre greenhouse gases and pollutants.
So people would like to understand how this works.
So microbes have been estimated to remove hundred, a million tons of carbon monoxide from the environment every year, producing about one trillion kilograms of acetate from these greenhouse gases.
And so, what do these catalysts look like and these enzymes, what do these metal clusters look like that do this chemistry.
And this was sort of a rough model of what they look like, and they thought it had iron and sulfur and then a nickel in some geometry, but they had no idea sort of where the nickel was and how it was coordinated.
And so before there was any kind of three dimensional information, they used spectroscopy, and they considered whether it was paramagnetic or diamagnetic to get a sense of what the geometry around the metal was.
So we're going to talk about different coordination geometries and how many unpaired or paired electrons you would expect, depending on those geometries today.
And so, crystal field theory, again, can help you help explain/rationalize the properties of these transition metal complexes or coordination complexes.
So, to help us think about geometry, I always find for myself that it's helpful to have models.
So not everyone can have such large models as these, but you can all have your own little models of these geometries.
So, what we have available to you are some mini marshmallows, which, of course, as we all know, are representative of d orbitals, and jelly beans, which we all know are useful for making coordination complexes.
So, what you can do with your mini marshmallows is you can put together to make your different sets.
And so, over here we have -- oh, actually it says gum drops -- you don't have gum drops this year, I changed up here, I forgot to change it down here.
We have mini marshmallows.
Dr. Taylor went out and tried to purchase enough gum drops to do this experiment, and discovered that Cambridge only had 300 gum drops, so we have mini marshmallows instead today.
But this gives you the idea.
You can take one toothpick and you can make d z squared, putting on your orbitals, you have your donut in the middle, and then your two lobes, which run along the z-axis.
And then for your other sets of orbitals, you can take these two toothpicks and put on these sets of mini marshmallows, and handily, you can just have one for all of the other d orbitals, because depending on how you hold it, it can represent all of the other d orbitals just very well.
So, you can just have one of these for all the others and then your d z squared.
So what we're going to do when we have our orbitals set up, then we can think about how ligands in particular positions, in particular geometries would clash with our orbitals -- where there'd be big repulsions or small repulsions.
So, any other people missing their jelly beans or their marshmallows?
Please, raise your hand, we have extras.
So, those of you who have them, go ahead and start making your d orbitals.
All right, so if you're finished with your two d orbitals, you can start making an octahedral complex.
So in your geometries set, you'll have a big gum which can be your center metal -- you'll have a big jelly bean -- sorry, big jelly beans and small jelly beans are our ligands, or our negative point charges, and you can set up and make an octahedral geometry here.
OK, so as you're finishing this up, I'm going to review what we talked about before the exam -- so this isn't in today's lecture handouts, it was in last time, which we already went over.
But sometimes I've discovered that when there's an exam in the middle, there needs to be a bit of a refresher, it's hard to remember what happened before the exam, and you have your models to think about this.
So, before the exam, we had talked about the octahedral case, and how compared to a spherical situation where the ligands are everywhere distributed around the metals where all d orbitals would be affected/repulsed by the ligands in a symmetric fashion equally, when you have them put as particular positions in geometry, then they're going to affect the different d orbitals differently.
And so, if you have your d z squared made, and you have your octahedral made, you can sort of hold these up and realize that you would have repulsion from your ligands along the z-axis directly toward your orbitals from d z squared.
So that would be highly repulsive.
The ligands are along the z-axis, the d orbitals are along the z-axis, so the ligands, the negative point charge ligands are going to be pointing right toward your orbitals.
And if you hold up this as a d x squared y squared orbital where the orbitals are right along the x-axis and right along the y-axis and you hold that up, remember, your ligands are right along the x-axis and right along the y-axis.
So, you should also have significant repulsion for d x squared minus y squared, and octahedrally oriented ligands.
In contrast, the ligands set that are 45 degrees off-axis, so d y z, d x z, and d x y, they're all 45 degrees off.
Your ligands are along the axis, but your orbitals are 45 degrees off-axis.
So if you look at that together, you'll see that whichever one you look at, the ligands are not going to be pointing directly toward those d orbitals.
The orbitals are off-axis, ligands are on-axis.
So there will be much smaller repulsions there.
And that we talked about the fact that for d x squared minus y squared and d z squared, they're both have experienced large repulsions, they're both degenerate in energy, they go up in energy, whereas these three d orbitals, smaller repulsion, and they're also degenerate with respect to each other, and they're stabilized compared to these guys up here.
So you can try to hold those up and convince yourself that that's true for the octahedral case.
So, that's what we talked about last time, and now we want to -- oh, and I'll just remind you we looked at these splitting diagrams as well.
We looked at the average energy of the d orbitals -- d z squared and d x squared minus y squared go up in energy, and then the other three d orbitals go down in energy.
So now we want to consider what happens with different geometries.
So now you can turn your octahedral case into a square planar case, and how am I going to do that?
Yeah, so we can just take off the top and the bottom and we have our nice square planar case, and try to make a tetrahedral complex as well.
And here's an example of a tetrahedral one.
Again, you can take a jelly bean in the middle, and big jelly bean, and then the smaller ones on the outside.
So what angles am I going for here in the tetrahedral case?
109 .
5.
So you can go ahead and make your tetrahedral complex, and don't worry so much about the 0 .
5, but we'll see if people can do a good job with the 109.
OK, how are your tetrahedral complexes coming?
Do they look like this sort of?
So let me define for you how we're going to consider the tetrahedral case.
So, in the tetrahedral case, we're going to have the x-axis comes out of the plane, the y-axis is this way, z-axis again, up and down.
We're going to have one ligand coming out here, another going back, and then these two are pretty much in the plane of the screen.
So this is sort of how I'm holding the tetrahedral complex with respect to the x, z, and y coordinate system.
So, there is a splitting, energy splitting, associated with tetrahedral, and it's going to be smaller than octahedral because none of these ligands will be pointing directly toward the orbitals.
But let's consider which orbitals are going to be most affected by a tetrahedral case.
So, let's consider d z squared.
What do you think?
Is that going to be particularly -- are the ligands pointing toward d z squared?
No.
And d x squared minus y squared, we can think of, what about that one?
No, not really.
What about d x y, d y z, and d x y?
Moreso.
So, if you try holding up your tetrahedral in our coordinate system, and then hold your d orbitals 45 degrees off-axis, it's not perfect, they're not pointing directly toward them, but it's a little closer than for the d orbitals that are directly on-axis.
So, if we look at this, we see that the orbitals are going to be split in the exact opposite way of the octahedral system.
In the octahedral system, the ligands are on-axis, so the orbitals that are on-axis, d x squared minus y squared and d z squared are going to be the most affected.
But with tetrahedral, the ligands are off-axis, so the d orbitals that are also off-axis are going to be the most affected.
But they're not going to be as dramatically affected, so the splitting is actually smaller in this case.
So here, with tetrahedral, you have the opposite of the octahedral system.
And you can keep these and try to convince yourself of that later if you have trouble visualizing it.
So, you'll have more repulsion between the ligands as negative point charges, and the d orbitals that are 45 degrees off-axis than you do with the two d orbitals that are on-axis.
So here, d x squared minus y squared and d z squared have the same energy with respect to each other, they're degenerate.
And we have our d y z, x z, and x y have the same energy with respect to each other, they are also degenerate.
So it's the same sets that are degenerate as with octahedral, but they're all affected differently.
So now let's look at the energy diagrams and compare the octahedral system with the tetrahedral system.
Remember an octahedral, we had the two orbitals going up and three going down.
The splitting, the energy difference between them was abbreviated.
The octahedral crystal field splitting energy, with a little o for octahedral.
We now have a t for tetrahedral, so we have a different name.
And so here is now our tetrahedral set.
You notice it's the opposite of octahedral, so the orbitals that were most destabilized in the octahedral case are now more stabilized down here, so we've moved down in energy.
And the orbitals that are off-axis, 45 degrees off-axis, which were stabilized in the octahedral system, because none of ligands were pointing right toward them, now those ligands are a bit closer so they jump up in energy, and so we have this swap between the two.
So, we have some new labels as well.
So, we had e g up here as an abbreviation for these sets of orbitals, and now that's just referred to as e. Notice the book in one place has an e 2, but uses e in all the other places, so just use e, the e 2 was a mistake in the book.
And then we have t 2 g becomes t 2 up here.
So we have this slightly different nomenclature and we have this flip in direction.
So, the other thing that is important to emphasize is that the tetrahedral splitting energy is smaller, because none of those ligands are pointing directly toward any of the d orbitals.
So here there is a much larger difference, here there is a smaller difference, so that's why it's written much closer together, so that's smaller.
And because of that, many tetrahedral complexes are high spin, and in this course, you can assume that they're all high spin.
So that means there's a weak field, there's not a big energy difference between those orbital sets.
And again, we're going to -- since we're going to consider how much they go up and down in energy, the overall energy is maintained.
So here we had two orbitals going up by 3/5, three orbitals going down by 2/5.
So here, we have three orbitals going up, so they'll go up in energy by 2/5, two orbitals go down, so they'll be going down in energy by 3/5.
So again, it's the opposite of the octahedral system.
It's opposite pretty much in every way except that the splitting energy is much smaller, it's not as large for the tetrahedral complex.
All right, so let's look at an example, and we're going to consider a chromium, and like we did before, we have to first figure out the d count, so we have chromium plus 3.
So what is our d count here?
You know where chromium is, what its group number -- here is a periodic table.
So what is the d count?
3.
So we have 6 minus 3, 3 -- a d 3 system.
And now, why don't you tell me how you would fill in those three electrons in a tetrahedral case.
Have a clicker question there.
So, notice that in addition to having electron configurations that are different, the d orbitals are labelled differently.
OK, 10 more seconds.
OK, very good, 80%.
So, let's take a look at that.
So down here, we're going to have then our d x squared minus y squared, d z squared orbitals up in the top, we have x y and x z and y z.
Again, the orbitals that are on-axis are repelled a little less than the orbitals that are off-axis in a tetrahedral case.
And then we put in our electrons, we start down here.
And then one of the questions is do we keep down here and pair up or go up here, and the answer is that you would go up here.
Does someone want to tell me why they think that's true?
Yeah
[INAUDIBLE]
Right, because it has a smaller splitting energy.
So, the way that we were deciding before with the weak field and the strong field, if it's a weak field, it doesn't take much energy to put it up there.
So you go they don't want to be paired, there's energy associated with pairing.
But if there's a really huge splitting energy, then it takes less energy to pair them up before you go that big distance up there.
But in tetrahedral cases, the splitting energy's always small, so you're just going to always fill them up singly to the fullest extent possible before you pair.
So this is like a weak field case for the octahedral system, and all tetrahedral complexes are sort of the equivalent of the weak field, because the splitting energy is always small in an octahedral case, because none of the ligands' negative point charges are really pointing toward any of those orbitals that much, so it's not that big a difference.
So, here we have this and now we can practice writing our d to the n electron configuration.
So what do I put here?
What do I put first?
So we put the e and then what?
Yup.
There are two electrons in the e set of orbitals, and in the t 2 orbitals, there's one.
So that is our d n electron configuration.
And then we're also asked how many unpaired electrons.
Unpaired electrons and that is three.
All right.
So that's not too bad, that's the tetrahedral case.
The hardest part is probably making your tetrahedral complex.
Now square planar.
So again, with the square planar set you have your square planar model -- we have a bigger one down here.
And the axes is defined such that we have ligands right along x -- one coming out at you and one going back, and also ligands right along the y-axis.
So as defined then, we've gotten rid of our ligands along the z-axis.
So, what do you predict?
Which two of these will be the most destabilized now?
What would be the most destabilized, what do you guess?
You can hold up your little sets here.
What's the most destabilized, what's going to go up the most in energy here?
Yeah, d z squared minus y squared.
What do you predict might be next, in terms of most unfavorable?
Yeah, the x y one.
So these two now are going to be the most destabilized, with d x squared minus y squared being a lot more destabilized than just the x y, because again, those d orbitals are on-axis and these ligands are on-axis.
So, let's take a look at all of these again.
So in the octahedral case, these were degenerate.
That's no longer true, because there are no ligands along the z-axis anymore.
So we took those off in going from the octahedral to the square planar, so you have much less repulsion, but with the d x squared minus y squared, you still have a lot repulsion.
so then if we start building up our case, and this diagram is, I think, on the next page of your handout, but I'm going to start building it all up together.
So now d x squared, y squared is really high up, it's very much more destabilized than anybody else.
D z squared, on the other hand, is down.
It's not -- it would be stabilized compared -- it's not nearly as destabilized as the other system.
So then we go back and look at these.
You told me that d x y would probably be next, and that's a very good guess.
You see you have more repulsion than in the other two, because the other orbitals have some z component in them.
So you have less repulsion than d x squared minus y squared, because it's 45 degrees off, but still that one is probably going to be up a little bit more in energy than the other set.
These two here are stabilized compared to the others, so they're somewhere down here.
Now the exact sort of arrangement can vary a little bit, but the important points are that the d x squared minus y squared is the most destabilized, d x y would be next, and the other are much lower in energy.
And we're not going to do this how much up and down thing, like the 3/5 and the 2/5 because it's more complicated in this case.
So just the basic rationale you need to know here, not the exact energy differences in this particular case.
OK, so now we've thought about three different kinds of geometries -- octahedral, tetrahedral, and the square planar.
You should be able to rationalize, for any geometry that I give you, what would be true.
If I tell you the geometry and how it compares with our frame, with our axis frame of where the z-axis is, you should be able to tell me which orbital sets would be the most destabilized.
And to give you practice, why don't you try this one right here.
So we have a square pyramidal case as drawn here with the axes labeled z, y and x, coming in and coming out.
Tell me which of the following statements are true.
And if you want, you can take your square planar and turn it into the geometry to help you out.
Let's just take 10 more seconds.
All right.
That was good.
People did well on that question.
So, if we consider that we had the top two are correct.
So, if we consider the d z squared, now we've put a ligand along z, so that is going to cause that to be more destabilized for this geometry rather than square planar, which doesn't have anything in the z direction.
ah And then in terms, also, other orbitals that have a component along z are going to be affected a little bit by that, but our other one here is not going to be true, so we just have all of the above is not correct, so we have this one.
So if we had up those, that's actually a pretty good score.
And so you could think about, say, what would be true of a complex that was linear along z, what would be the most stabilized, for example.
So these are the kinds of questions you can get, and I think there are a few on the problem-set.
All right, so let's come back together now and talk about magnetism again.
So, we said in the beginning that magnetism can be used to figure out geometry in, say, a metal cluster in an enzyme, and let's give an example of how that could be true.
So, suppose you have a nickel plus 2 system, so that would be a d 8 system, so we have group 10 minus 2 or d 8, and it was found to be diamagnetic.
And from that, we may be able to guess, using these kinds of diagrams, whether it has square planar geometry, tetrahedral geometry, or octahedral geometry.
We can predict the geometry based on that information.
Let's think about how that's true.
We have a d 8 system.
Think about octahedral for a minute.
Are there two options for how this might look in this case?
Is there going to be a difference in electron configurations if it's a weak field or a strong field?
So, write it out on your handout and tell me whether it would be true, think about it both ways.
Is there a difference?
So, you would end up getting the same thing in this particular case.
So if it's a weak field and you put in 1, 2, 3, then jump up here, 4, 5, and then you have to come back, 6, 7, 8.
Or you could pair up all the ones on the bottom first and then go up there, but you actually get the same result no matter which way you put them in, the diagram looks the same.
So it doesn't matter in this case if it is a weak or strong field, you end up with those number of electrons with the exact same configuration.
So, we know what that looks like.
Well, what about square planar.
So let's put our electrons in there.
We'll start at the bottom, we'll just put them in.
I'm not going to worry too much about whether we can jump up or not, we'll just go and pair them up as we go down here, and then go up here, and now we've put in our eight electrons.
So, how close these are, we're just going to put them all in.
We're just going to be very careful not to bump up any electrons there unless we absolutely have to, because d x squared minus y squared is very much more destabilized in the square planar system, so we're going to want to pair all our electrons up in those lower energy orbitals.
So even if we sort of did it a different way, that's what we would get.
So we're going to want to pair everything up before we go up to that top one there.
So there's our square planar.
Well, what about tetrahedral.
How are we going to fill these up?
Do we want to pair first, or we do want to put them to the full extent possible singly?
Single, right, it's going to be a weak field, there's not a big splitting here between these, so we'll put them in, there's 1, 2, 3, 4, 5, 6, 7, 8.
All right, so now we can consider which of these will be paramagnetic and which will be diamagnetic.
What's octahedral?
It's paramagnetic, we have unpaired electrons.
What about square planar?
Square planar's diamagnetic.
And what about tetrahedral?
Paramagnetic.
So, if the experimental data told us that a nickel center in an enzyme was diamagnetic, and we were trying to decide between those three geometries, it really seems like square planar is going to be our best guess.
And so, let me show you an example of a square planar system.
And so this particular nickel is in a square planar system.
It has four ligands that are all in the same plane, and it is a square planar center for a nickel, so that's one example.
And this is a cluster that's involved in life on carbon dioxide.
All right, so that's different geometries, you're set with that.
Monday we're going to talk about colors of coordination complexes, which all have to do with the different geometries, paired and unpaired electrons, high field, low spin, strong field, weak field.
Have a nice weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, everyone, pay attention to the clicker question.
If you haven't responded, now's a good time to click in your response.
All right, let's just take 10 more seconds.
OK, we can do better than this.
So, tetrahedral complexes.
Do you recall tetrahedral complexes with angles of what between the ligands?
109 .
5.
The ligands' negative point charges aren't facing any of the d orbitals perfectly.
They're a little bit closer to the orbitals that are 45 degrees off-axis, so those three are the most repelled.
But they're not really directly hitting any of them.
So that's in contrast with the octahedral system or square planar where the ligands negative point charges are headed directly toward some of the d orbitals.
So because the ligands in a tetrahedral case are not headed directly toward any of the d orbitals, there is not a huge amount of crystal fields splitting, so that's small.
And so, when the splitting is small, then you tend to have high spin systems.
So you put in all of the electrons singly to the fullest extent of the orbital as possible before you pair.
And so, since you put them in singly for the fullest extent possible before you pair them up, that will lead to a high spin system, which is the maximum number of unpaired electrons.
So today is then the last lecture on transition metals, and we've been talking about crystal field theory, and today we're going to talk about colors and crystal field theory.
So, colors, there are a lot of beautiful colors in nature, and some of the beautiful colors you find in nature have to do with transition metals or other liganded states.
So, we're going to start with an example of how colors can change, how a molecule's color can depend on its oxidation state, how a molecule's color can depend on its liganded state.
So, Dr. Taylor is going to be doing this for you -- should we do it the demo and then look at the questions or do the questions first?
Demo first?
So, here are some of the reactions up on the Powerpoint that you're going to be looking at, and as the reaction proceeds, there's going to be changes in oxidation state, and also changes in ligand in state, and that will lead to changes in color.
[DEMONSTRATION]
So, how would you describe that first color?
Any predictions?
Have any of you seen this before, what's going to happen next?
[DEMONSTRATION]
So this is called an oscillating clock reaction, so as it runs through, it cycles between the different colors.
So, that should happily keep going, and we can consider what's happening.
So here's the overall reaction here and it can be divided into two components.
So, why don't you tell me, what is happening to iodide in that first reaction, and there it is again.
All right, so let's take 10 more seconds.
Very good.
So, if you looked up here, you see that you have minus 2 for the oxygen, three of them, minus 6, and it needs to equal minus 1, so you have plus 5.
And then over here, we again have plus 1 minus 2, and so that has to be plus 1.
So that was actually a quiz question, but you score three points if you answered, even if you got it wrong, but we could have had that four points, most people got that right anyway.
But if you answered, you get an extra few points on that one.
All right, so very good.
That's what's happening to iodide.
Now you notice that in this reaction h o i is being produced, and then in the second step it's being consumed.
This later reaction can be divided, then, into two additional reactions.
And one of the reactions, you can tell me again what is happening, what is being oxidized and what is being reduced in this part of the reaction.
OK, let's take 10 more seconds.
Excellent.
Even a little higher than before.
All right, so you figured out that this was the plus 1, this is minus 1, and they're both going to 0.
So you have oxidation and reduction going on all involving iodide in that reaction.
OK, so as the reaction was proceeding, you could see that it started out with clear and then it went to sort of amber color, and that was the i 2, the clear was i minus, and the sort of darker blue color that you saw is the complex with starch in the reaction.
So that both of these guys have colors independent, but then when they're liganded to something else, the color is different.
And that's very true with coordination complexes that the metal by itself will be strongly influenced by what the ligands are.
And depending on what type of ligands it has, it can have a really entirely different color than it had before.
So, today we're going to talk about why that's true, and how you can predict colors based on what type of ligand is bound to a transition metal.
So, a lot of transition metals have really beautiful colors, and my laboratory studies metals bound to proteins, and often the proteins will have really beautiful colors because of the metal cofactor involved, and that's one of the things that I liked about that particular area of study is just how beautiful these proteins can be.
So, the color given off depends on the nature of the metal, and depends on the nature of the ligand.
And so we can use crystal field theory, which again, is a very simplified theory, to try to predict or explain the observed colors.
And so again, this is not always very precise, but given, if you're told information out of color, you can rationalize why that would be true, and you can also predict at least a range of color that you would have under certain circumstances.
So, let's take a look at this more.
So the ligands again, have the ability to split those d orbitals.
And when we're talking about for metals, it's all about the d orbitals.
And we talked already about strong field ligands and weak field ligands, and we're going to talk more about that today, but this time in terms of color.
So a strong field ligand, as we discussed, creates a big splitting in the d orbital energy, whereas weak field ligand, like the first question you had today with tetrahedral complexes, there's usually a weak field there, so we have a small energy separation between the d orbitals.
So here is something that you actually have to memorize -- there's not much memorization in this course, but you do need to memorize these six ligands in terms of their ability to split d orbitals.
So, on the side, we have three that are strong field ligands, cyanide, c o, and ammonia, and so those are going to be strong field, so they're going to have big splitting energy, and so they'll tend to be low spin.
Then we have three that are sort of in between that are intermediate field -- water, hydroxide and f minus.
And so, in comparing, those are intermediate, so you're going to be asked questions such as how does that compare to a weak field, how does that compare to a strong field.
And then our weak field ligands or a lot of our halides down here, i minus, b r minus, c l minus.
And so those are weak field ligands, so you'll have a small splitting, so they'll tend to be in high spin complexes.
So let's take a look at some examples.
So we talked about iron before in complexes, and now we can consider two cases where we have iron plus 3, so the same metal in the same oxidation state, but it has different ligands.
It So in one case you have a high spin system with six water ligands, and then the other in a low spin system with six cyanide ligands.
So first, before you do anything else with this, you always have to think about what the d count is.
So, to do the d count, we're going to look at where iron is in the periodic table, and we're going to see it's in group 8.
And then, we'll have 8 minus 3, the oxidation number is d 5 system.
So now we have two diagrams here, one has a big splitting, one has a small splitting, and why don't you fill in for me in a clicker question, what the high spin system would look like.
OK, let's just take 10 more seconds.
Very good.
I think that's one of our highest numbers in a while.
That's right, so we're going to fill to the fullest extent possible before we pair any of the electrons.
So again, here we put in electrons down here, and then go up here, because the splitting is small, so it doesn't take that much energy to put an electron in the upper orbitals, it takes more energy to pair the electrons for this weak field system.
And so, this is a high spin case, we have a maximum number of unpaired electrons.
So, over here when we have a much bigger splitting energy, it's going to take a lot more energy to put electrons up there, and so we're going to fill up all of these orbitals down here until we have to put an electron up there.
So, if we do that, put in the three, and now we're going to pair, because it takes less energy to pair than it does to put an electron up there, so we do four and five.
And so then here is our system where we have a strong field, and so here's a weak field and it's going to be high spin, maximum number of unpaired electrons, here we have the strong field and it'll be low spin, the minimum number of unpaired electrons.
And we're doing this, again, because we know that cyanide is a strong field ligand, whereas water is an intermediate field ligand, it's a lot weaker than cyanide.
OK, now we can continue doing some of the things that we've done before.
So just to review this material, we can write the d n electron configurations.
What are the orbitals called that are down here?
Yup, t 2 g. And how many electrons do we have there?
three.
And then the e g system up here with two electrons.
And over here what do we have?
Yup, t 2 g to the 5.
So, a review, electron configurations, these are just shorthand notations, which tell people what these diagrams look like.
Then, we also talked about this before.
So, what does this term stand for?
Crystal field stabilization energy, right.
So now why don't you tell me what it is for the high spin system.
OK, let's just take 10 more seconds.
Yup, zero.
So if we look at that, we have three electrons down here, so that's minus 2/5, two electrons up here, which is plus 3/5, so 3 times minus 2/5 is minus 6/5, plus 6/5 gives you 0.
So here is a case where you really don't have much stabilization.
It would be equivalent then, so there's zero stabilization because you have three electrons down and two up.
All right, so what about the low spin system now?
So if we can look at that, what are we going to have for this system?
So, we have 5 times minus 2/5 or minus 10/5, and we also have two pairing energy terms there, which we can write in, because there are two sets that are paired.
So this is mostly review on what we've had before, and we haven't talked about some of these things in a little while, so we go over it again, and now we're going to take it a next step and think about what sort of wavelength would be absorbed if you're going to promote some of these electrons to unfilled orbitals.
So what about the light absorbed by these octahedral coordination complexes?
So, if you remember back to the beginning of the course, a physics course or high school, a substance absorbs photons of light if the energy of the photons match the energy required to excite those electrons to a higher energy level.
And so now we are doing some review from the first part of the course, which is always good.
As I mentioned, everything kind of comes together and we need to go over everything for the final, but there's also connections between all the different parts of the semester.
So this should look familiar to you.
So, the energy of the absorbed light equals Planck's constant times the frequency of that light.
But now we can make that equal to another term, a term we've been talking about in this unit, and that is our octahedral crystal field splitting energy.
Because the energy that's going to be required to bump an electron from here to here is that energy, that splitting energy.
So that's going to be equal to this term here.
OK, so what does this mean in terms of the wavelengths of lights absorbed by different coordination complexes.
So we can think about that.
So if you have high frequency of light is absorbed, the wavelength of the absorbed light is going to be short.
And we know that relationship -- again, think back to the beginning of the course and also probably to physics and to high school, we know a very handy equation for telling us about the relationship of frequency and wavelength of light, so we have the speed of light equals the wavelength times the frequency.
So if you have a high frequency of light absorbed, the absorbed wavelength is going to be short.
So, let's look at a couple of examples, the example here, going back to our example.
So we had this high spin system with water, and now I'm telling you that the splitting energy is 171 kilojoules per mole, and when you have cyanide as your ligand, your splitting energy is 392 kilojoules per mole.
Again, this was a stronger field ligand, so we have a bigger splitting energy.
And this was an intermediate field ligand, certainly weaker than cyanide, so this has a smaller splitting energy.
So, from these values now, we can calculate the wavelength of absorbed light.
So for the high spin system first, we can rearrange these equations, which you know well, to come up with the rearranged equation, the wavelength equals Planck's constant times the speed of light, and divided by e, and this time our e is that crystal field splitting energy.
So we can put in these terms.
So we have Planck's constant times the speed of light over our octahedral field splitting energy, oh, but then we have some other terms here.
Now one thing that you have to pay attention to in this unit is your units.
So, splitting energies are often given in kilojoules per mole, whereas you often see Planck's constant in joules, so we want to make sure that we convert one or the other, and here it's set up to convert the kilojoules to joules.
And also, we want our final unit, we're talking about wavelengths, in meters or nanometers, so we need to get rid of this mole term, and we use Avagadro's number to do that.
So now we should be able to cancel our units and get the correct units.
So we should be able to cancel the seconds over here, and we should also be able to go in and cancel our moles.
We should be able to cancel the joules and the kilojoules out, and that should leave us just with this over here, which is meters.
So this is 7 .
0 0 times 10 to the minus 7 meters.
Does that make sense in terms of a wavelength of light?
Because that would convert to what nanometers?
700 nanometers.
So if you do something strange and you forget Avagadro's number, you're going to come up with a very interesting wavelength.
So that's a good way to check to make sure that you've done the problem correctly.
So, 700 nanometers, anyone remember what light that is, what color that corresponds to?
Red, so it's absorbing red light.
All right, so now let's just do the same thing for the low spin system with the cyanide ligand, and we're going to plug in our 392 here, and we get 305 over here.
And so, that is a much shorter wavelength of light.
So again, light absorbed for the compound with water, 700 nanometers, and the compound of iron with cyanide, 305, and so we're absorbing a red light over with the water compound, and sort of purple or violet light is being absorbed in the cyanide complex.
And we're talk in a minute about the light that is being transmitted, which is complementary to the color of the light absorbed.
So by knowing something about splitting energies, by knowing something about the types of ligands, then you can know something about colors.
So now, for another example, we're going to look at the different colors of two chromium complexes.
So first, what is the oxidation number of chromium in the water complex here?
What is it?
And what about over here with n h 3 ligands.
Plus 3.
So, plus 3 and plus 3.
And what is our d count?
You know where chromium is, in what group?
Six.
6 minus 3 is 3, so we have a d 3 system.
What does c n mean again?
Coordination number, so what is that for both of these?
Six.
So there's Six things coordinated to the chromium, and so we have, again, an octahedral system.
And what type of ligand is water?
Intermediate.
And what about n h 3?
Strong.
So this is strong, and water is and intermediate ligand, and certainly, it is weaker than n h 3.
So, we expect one system over here for a strong ligand, and then something that's weaker than that strong ligand.
All right, so here are two diagrams, one with a big splitting energy, and one with a smaller splitting energy.
Are the diagrams going to look same or different?
The same.
Because we only have three electrons, so they're going to be the same.
In both cases, we put in the three electrons in the lowest orbitals, and then there's no decision to be made, because there isn't that fourth electron, so we don't have to decide which place to put this.
So these diagrams are going to look the same, and before when we were doing this in this unit, we said OK, we're done, they look the same.
But now, we realize that these are really not the same compounds and that they're going to have different properties, even though their diagrams are going to look the same.
Because the energy that it's going to take to excite an electron here is much smaller than over here, and that's going to result in a different wavelength of light being absorbed in these two different cases, which will mean a different wavelength of light being transmitted.
So, again, here we have a weaker field and here we have a stronger field.
So again, we can go through and think about these two cases.
So when we have a smaller term here, that means a lower energy -- this is a splitting energy, and so we'll have a lower frequency.
When we have a larger case or higher energy, we have a higher frequency.
Again, if you have a lower frequency absorbed, we have a longer wavelength absorbed, and in this case, the higher frequency translates into a shorter wavelength absorbed.
The color of the transmitted light is complementary to the color of the absorbed light.
So now we can think about -- I always want to ask, when do people learn about complementary colors?
Is that 6th grade, earlier?
I don't really remember, but I think it's pretty early on.
And people always ask me, are you going to have that on the equation sheet, or do I have to actually remember my complementary colors?
And I tell people that I will put some version of this on, so you don't have to review your kindergarten notes for this class, if that's when you learned it.
It's pretty early, I don't know when it is exactly.
So, the color of the light is going to be complementary about -- this is very approximate.
So here, if the transmitted light is shorter because we have this weaker splitting, then we are expecting we're going to have this shorter wavelength, the transmitted light, and experimentally, if you make this compound you'll see it's violet.
In this other case, we have a stronger field ligand, and so you have a larger energy, higher frequency absorbed, shorter wavelength absorbed, then you're going to have a longer wavelength from your transmitted light.
And that this compound, if you make it, is actually yellow.
So if we go back to our colors for a minute, you see that when you have the shorter wavelength, we have a violet, and that is a short wavelength.
And in this case for the strong field ligand, we are going to have transmitted light of a longer wavelength and it's yellow.
So it's the same oxidation state of chromium, it's the same -- it's an octahedral complex, six ligands in both cases, same octahedral crystal field diagrams, but yet one compound has a violet color, and the other one has a yellow color.
All right, so one can also be asked to calculate a crystal field splitting energy in kilojoules per mole, given the appropriate information.
So we've looked at when a splitting energy is given, and we've been asked to calculate wavelength absorbed, you can also be asked to go in the other direction.
And so here we have another chromium complex to work with.
And we're told that the wavelength of the most intensely absorbed light is 740, and so what would you predict the color of this to be?
It would be greenish.
So that would be what you would predict.
Again, chemistry is an experimental science, but based on having a complementary color to the one absorbed, that would be a guess.
All right, so we can actually calculate the frequency of the light absorbed.
So we were given the wavelength, and use speed of light, and plug in your wavelength and you can come up with a frequency, 4.05 times 10 to the 14 per second.
Then we can calculate from that the crystal field splitting energy, and so we use Planck's constant, and we have our frequency, and we calculate 2 .
6 8 times 10 to the minus 19 joules.
Am I done with the problem?
What does the problem ask for?
It asks for it in kilojoules per mole, and so, we're not done, we need to convert to kilojoules per mole.
And I'm making this point, because often this is where people lose points on the final exam, and that's not where you want to lose points.
You want to lose them on a really hard problem, not on something like this.
So, most of the time you're asked for kilojoules per mole, so make sure that if that's what asked for, that's what you provide.
So here, we can just do the conversion of units, and then we're going to use Avagadro's number to give us that per mole.
And so, this translates into a 160 kilojoules per mole, which you might recognize is more similar to the other numbers that you saw for octahedral crystal field splitting energy.
All right.
Sadly, there are some coordination complexes that do not have colors.
Why would that be?
Why would something not have a color?
It has d orbitals, it's a transition metal.
So what would be true about all of the d orbitals?
Yeah, so one example, is if they're all full, and that is the most common thing that we see, so it's not possible to have a d to d transition in the visible range.
So there are a number of examples of metals who have this situation.
Zinc and cadmium are two of the most common that give you problems in biological systems.
So why is this?
Well, that's because they're over here in group twelve.
But their most common oxidation states are plus 2.
So, if you have 12 minus 2 you have a d 10 system, and that's the case for both of these systems here.
And so all the d orbitals are filled.
Now, zinc is a really important metal in biological systems, and because it has all these d orbitals filled, it doesn't have a color.
And so it's very hard to tell if an enzyme molecule has zinc.
And I think one of the problems that you have on this problem-set talks about how zinc is important in a biological system by altering the p k a of a residue that coordinates to it.
And that's often its job, and so biochemists are often faced with the problem of trying to figure out if their protein has zinc, but they have no color of the protein, also they might try to look for a paramagnetic or diamagnetic system.
They're not -- you know if they see a paramagnetic system, they say oh, unpaired electrons, we know we must have metal involved, but there's no sort of spectroscopic probe for zinc.
And so, often someone will determine a crystal structure, and it'll be a huge surprise that there's zinc associated with this protein.
Do you think there's a lot of proteins that use cadmium as of part of their mechanism?
What do you know about cadmium?
Yeah, cadmium is poisonous.
Old barbecue grills were sometimes, they used to coat things with cadmium on a barbecue grill.
Yeah, that was not very smart.
So, cadmium poisoning is a problem, and people have been trying to figure out the mechanism of that, but again, it's hard to study cadmium because it has no spectroscopic signal.
All right, so getting back now to just kind of review over what we've talked about in terms of colors.
So we have our weak field ligands, again, you need to memorize what they are.
You have your intermediate field ligands, which you need to memorize, and also your strong field ligands.
So these weak field ligands are going to have a small splitting energy, and that means that in terms of how the complex absorbs, low energy, low frequency, long wavelength, and that the color transmitted will be complementary.
And usually what this means is that it'll be sort of in the end of the spectra, so it's often hard to say well, red will definitely be green.
So it's not a perfect agreement, but you can usually say well, it's probably going to be in the blue-violet or green end.
So in sort of one part of the spectra.
Strong field ligands, again, have a huge splitting energy.
So you're going to have big energy, high frequency, short wavelength, and so it's going to transmit, then, in the complementary, so it should be in the yellow, orange or red end.
And you will be asked in some of the problems, in again, problem-set 9 due Wednesday, and you should be able to finish that up pretty quickly tonight after this lecture, and there are some problems on this.
So, for cobalt complexes, you get pretty much the entire range of colors.
So I'm just going to end with one more biological example.
And here are some pictures of actual colors, and so this is cobalt coordinated to vitamin B12.
So one ligand gives you this brilliant red color, another gives you an orange color, and a third ligand gives you a pink color.
So you can tell the oxidation state of vitamin B12 by the colors of the molecule.
All right.
Now you have all the information to finish your problem-set, and that's the end of transition metals.
On Wednesday we start kinetics.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So today we're starting the last unit of this class, which is kinetics.
So, we're moving toward the end of the semester, and today will just be an introductory lecture on kinetics.
So, again, kinetics are the rates of chemical reactions, and so we're going to talk about, we're going to introduce you to rate expressions and rate laws.
So, when you're considering a chemical reaction, we've been asking so far in this class whether the reaction will go spontaneously.
So we've been talking a lot about thermodynamics, we've been talking a lot about delta g. But we also need to consider how fast a reaction goes, and that's kinetics.
So, a kinetic experiment is one in which you measure the rate at which something's happening.
The rate at which a concentration is disappearing or whether the rate at which a concentration is forming.
You're measuring some kind of change of a reaction, change in composition versus time.
So, I'm going to give, for an example, of why kinetics are important talking about the oxidation of glucose.
And I'm going to ask -- we're going to do a little experiment, and I'm going to ask the TA's to help me.
Everyone needs to have something to do an experiment with, something that might have some glucose in it to do this experiment.
So, if the TA's could help me hand out the ingredients for this experiment.
So, don't open your experimental apparatus until I give the OK.
So, just collect it and let me set up the experiment for you while we're handing things out.
So you're all familiar with this reaction, so we have glucose and oxygen go to c o 2 and water.
And so we've talked a lot about delta g for this reaction, and delta g again, is equal to delta h minus t delta s. So for this particular reaction, we have a negative delta h nought, so what does negative delta h mean about this reaction?
It's exothermic.
Delta s nought is positive, what does that mean, if delta s nought is positive?
What's increasing?
Entropy or disorder of the system, which is favorable.
So, our delta g nought is a big negative number.
So, in terms of thermodynamics, what does this mean?
Is this reaction spontaneous?
Yes, it's very spontaneous, it's very thermodynamically favorable.
Now, when glucose in products are wrapped, they're often wrapped under a nitrogen environment, so, oxygen is not sealed up in this container, in the wrapper, and they do that to prevent bacterial contamination and things like that.
So, the glucose that would be in a wrapped candy is not exposed to oxygen.
And so if you look at this thermodynamic information up here, one might imagine that if you ripped this open, and oxygen came in, that there might be some kind of level of explosion with c o 2 coming out and water coming out.
Now, you know, you might say well, if you have one sort of little thing it might not, but now if everyone has one, and we all open it at the same time, what do you expect to happen?
Let's try it.
Oh, we need some more down here.
Anybody else need any?
All right, so I'm trying mine, and I'm not really noticing anything happening.
So the situation is the following, It is a spontaneous reaction, but it's slow.
So here, kinetics are very important.
So, how many of you are having Thanksgiving with a small child around, younger siblings or something like that?
When you see them eat a lot of sugar, which can happen over Thanksgiving, it sometimes seems like an explosion has occurred, that there's all of a sudden hyper energy running around.
But that could be due to this reaction biochemically, getting a lot of energy into the system, but probably you will not see c o 2 and water coming off of the small child as they run around.
So, let me introduce you to a couple of terms, if you haven't heard these already.
So people are often talking about compounds being stable or unstable.
When they're doing this, they're talking about the thermodynamics of a system, the tendency to decompose, whether the reaction is spontaneous or not.
Then you can also hear people talking about whether the compound is labile or non-labile or inert, and this refers to the rate at which that tendency is realized.
So you might have a very unstable compound, thermodynamically it wants to decompose, but it also might be very non-labile, it might be fairly inert, that kinetically that decomposition is going to take such a long amount of time, that you're not going to notice the tendency to decompose.
So, these two can play off each other.
So, you can have materials that are very unstable thermodynamically, but you won't really see them decompose because the rate is just way, way too slow.
So, you know for this particular reaction, this is how we get energy -- we make ATP's of energy for the body.
So it's slow, but in the body we have to do something about that because we need energy to keep going, and so to be a useful energy source, the oxidation must be fast enough.
So, does anyone know in the body what happens to make that reaction faster?
Enzymes, right.
So, enzymes are catalysts and they will speed up the reaction.
So, let me just give one more example.
Some of you may have heard the commercial from deBeers, "a diamond is forever."
If you haven't, you will probably hear that quite often between now and the Christmas season.
This is a very popular time, I guess, for people to buy each other diamonds.
But if you look at that the thermodynamics of this, graphite is actually much more stable than diamonds by 2,900 joules.
So, one could argue that graphite is forever, not that diamonds are forever.
Of course, here kinetics are on the side of the diamond, they're relatively kinetically inert, it's a huge activation energy barrier, which we're going to be talking about in this unit for the conversion.
So, diamonds do, in fact, stay around for a long time.
One could still argue that, perhaps, a better gift would be a graphite ring than a diamond ring, but I have a feeling that probably even if the person receiving said ring was a chemist, they might still not a 100% appreciate that gesture.
So, it's important, the thermodynamics are important, but kinetics are also really important to understand what kind of chemical reactions are going to occur.
So, let's think about what are some factors that would affect rates of reactions.
So let's write down, and you can think about what are some factors affecting rates.
What's one thing people can think of that might affect the rate of a reaction?
Temperature
Temperature, definitely.
And this is used quite often in cooking to get things to occur.
What else affects the rate of a reaction?
OK, yeah, so pressure could -- if you're changing things, applying pressure to sort of switch things around.
That will depend, say, for pressure on the nature of the material and things like that of what you're talking about.
So let's make that a bigger category and put nature of the material.
What type of material it is.
And also, along with that, if you're thinking about a particular reaction and how it's going to go and how you might get it to sort of push one way or the other, you're also thinking about the mechanism of that reaction.
So what's reacting with what will make a difference.
I think I also heard someone talk about concentration, how much of it you have.
And you told me the last one that we're going to talk about in this unit, when we were talking about how the body gets the oxidation of glucose to go, what was necessary there?
A catalyst, right.
So, those are all the things we're going to talk about in factors affecting rates of reaction in this next unit.
So now, chemistry is an experimental science, and so a lot of kinetics are involved with measuring the rates of reactions.
So let's talk about how one might measure the rate of a reaction.
So here's a particular reaction, we have n o 2 plus carbon monoxide, going to n o plus carbon dioxide, and one could measure the rate of decrease of the reactants or the amount of increase of the products.
So let's just look at one of the products.
So one might be plotting a change in the concentration of one of the products versus time.
And you may find that it goes up, and then maybe starts to level off a little bit over time.
And now, how are you going to measure a rate out of this, the rate at which this reaction is going?
Well, one could measure an average rate, which would be some change in concentration over some change in the amount of time.
And you could express that as the change in concentration over the change in time.
So, you could pick a particular interval, say, we want to measure the average rate from time is 50 seconds to time equals 150 seconds, and we'll look at how much the concentration has changed over that time interval.
So, we can do a little calculation of the average rate and get a number, 1 .
2 8 times 10 to the minus 4 molar per second, and that's an average rate.
But if we had picked at different interval, we would have gotten a different number.
So the average rate depends on the time interval, so that's not always ideal.
You don't always want to know an average rate, which might be different depending on which particular unit you pick.
So often, when you're talking about rates, you talk about instantaneous rates.
So the rate at a particular instance of time.
And so let's look at instantaneous rate.
So, we have the same reaction, same plot, but now we're going to be considering instead of the average rate, the rate at a limit when your time interval is going to zero -- at a very, very small time interval, so the rate at the particular instance.
And so this can be expressed as d times the concentration of a product, n o, over d t. So, as delta t approaches zero, the rate becomes the slope of the line tangent to the point, that particular time point that you're interested in.
So, let's find an instantaneous rate at 150 seconds.
What is the rate of this reaction at 150 seconds?
So we can look for 150 seconds, we can have a point on the curve at 150 seconds.
And so as our time interval approaches zero, the rate will approach the slope of this line.
So we can draw a slope that's tangent to the curve at the time t, it's time 150 seconds, and then we can calculate the slope of that line.
And so we can do that math, calculating the slope, and here we find that at an instantaneous rate that time equals 150 seconds, change in -- this is the slope of the line -- the change in concentration over change in time, and that gives you 7 .
7 times 10 to the minus 5 molar per second.
So, that's instantaneous rate at a particular time.
What do you think the instantaneous rate is called at time equals zero?
Any guess?
Initial rate, yeah.
So, initial rate, instantaneous rate at time equals zero seconds.
So that's instantaneous rate.
So now, let's talk about rate expressions, and then we're going to talk about rate laws.
So, same equation.
Again, you can monitor how much your reactants are disappearing, you can monitor the amounts of your products being formed, and you can express this in the following way.
So we could say that the rate is going to be equal to the decrease in one of the reactants, so minus d times the concentration of n o 2 over d t. We could also express it in terms of the second reactant, so minus d times change in the concentration of the other reactant over time.
Or we can express this in terms of the products being formed, so here there's no negative sign, so we have d concentration of n o over d t, or our last product, d times the concentration of c o 2 d t. So this would be a rate expression, and these would all be equal to each other, if we make the following assumption.
We're going to make the assumption here that there's no intermediate species that's being formed, or that if there are intermediate species, that their change in concentration is independent of time.
So if there were some other really complicated thing going on here, then those rates may not be equal and we might have something else in the mechanism that would cause one thing to disappear a lot faster than something else that's being formed.
But these should all be equal if you assume no intermediate species, or make assumptions about those intermediate species.
So, then the general expression for a rate expression, if we have an equation, a plus b going to c plus d, where we have coefficients of the reaction, small a, coefficient small b, coefficient small c, and coefficient small d. So we could express the overall rate then, minus 1 over a times d a t t minus 1 over b, d b d t, 1 over c, d c e t, 1 over d d d d d t. That was not easy.
All right, so that's the general form for the rate expression.
So just to make sure that everyone's on the same page, why don't you do a rate expression for me for this one.
OK, just 10 more seconds.
Excellent.
So you just have to pay attention to your minus signs, your stoichiometry.
So, it's minus for the disappearance, stoichiometry 1 over 2, and then no negatives for your products of things that are appearing.
Very good.
Rate expressions, not that complicated.
All right, so now we're going to talk about rate laws, which are slightly more complicated than rate expressions.
So, a rate law is -- you come up with a rate law experimentally, and it's the relationship between the rate and the concentration.
So we have -- we're going to introduce a term called the rate constant, which is the small letter k. And so the rate constant is going to tell you about the relationship between the rate and the concentration of your reactants in a reaction.
So, if we had that same reaction here, we could also write that the rate is equal to a rate constant times the concentration of your reactant, a, raised to a power m, and b raised to a power n. So here, m and n are the order of the reaction with respect to a and b respectively, so m is the order of the reaction in a, and n is the order of the reaction in b, and our small letter k here is the rate constant.
So that would be an expression for rate law.
And so, now I'm going to tell you a lot of things that are true about rate laws.
So, a rate law, again, comes from experiment, so you can't just look at the stoichiometry of the reaction and predict the rate law.
So m is not the stoichiometry of the reaction in terms of the a, that is, unless the reaction is what we call an elementary reaction, or a step in a reaction, and we're going to talk more about that next week -- then you can, if it's an elementary reaction, then you can use stoichiometry to predict.
But for other reactions you can't, and that'll become more clear next week when we talk about this in detail.
So, rate laws are not limited to reactants, sometimes a product will show up.
It's not that common, but it's possible, and again, it's experimentally determined.
So the experiment would have to tell you whether that term is going to be there or not.
So, occasionally you'll see a product term in a rate law.
So, in terms of m and n, the orders of the reaction, m and n can be integers, they can be whole numbers, or fractions, negative or positive, lots of options for m and n. And let me tell you about all the options for -- we're going to use m here.
So, you have this table in your notes, most of it is blank, some of it is filled in, and we're going to fill in the parts that are not filled in right now.
So we're going to start in the middle where the order of the reaction is one here, m equals 1, and that's called a first order reaction.
So these names are pretty much intuitive when you look at them.
So here, the rate law for a first order reaction would be our rate constant, k, times the concentration of a.
So let's think about what that rate law would mean.
Say you double the concentration of a, what would happen to the rate of the reaction?
How many people say double, raise your hand.
Good, that's what happens.
All right, so it doubled the rate of the reaction.
So now let's think about m equals 2.
See on your handout that that's called second order -- again these names make sense.
The rate law for this would be k times the concentration of a squared, so m equals 2, and that's shown over here.
If you double the concentration of a, what would happen?
So you should quadruple the rate.
So what about if you tripled the concentration?
Why don't you go ahead and tell me if you triple it for m equals 2, what would happen to the rate?
OK, let's just take 10 more seconds.
Excellent.
That's right, 9 times.
So, you're definitely getting the hang of this.
All right, so let's move down and talk about m equals minus 1.
If you think of a good name for this let me know, because no one ever calls that anything, so we can leave that blank.
And go ahead and talk about what the rate would be there and how we would write the rate law.
And so, the rate law, then, would be k, our rate constant, times the concentration of a raised to the minus 1.
All right, if we double the concentration here, just yell out what you think would happen.
You would 1/2 the rate of the reaction, or you can think about as 2 to the minus 1, 1/2 the rate of the reaction.
And you will have problems on problem-set 10, which will be due a week from Friday where you're going to be given experimental data, and you have to look at it and see, OK, what happened to the rate, and then what does that mean about the order of the reaction.
So this is a lot of what the problems are like in this unit.
All right, also there is no name that I'm aware of for when n equals minus 1/2, but we can write the rate law.
So that's just going to be k times the concentration of a raised to the minus 1/2 -- again, for the order of reaction, they can be integers, they can be fractions, they can be negative, and they can be positive.
So, tell me for doubling the concentration here of this one, what's going to happen to the rate?
OK, let's just take 10 more seconds.
Yup, so most people got that right.
So, if we go back here, 0 .
7 times the rate, you could think about this as 2 raised to the minus 1/2.
These get a little more complicated and are some of the harder ones on the problem-set to recognize the relationship.
So if you remember all the possibilities, it's going to help you think about what's going on when you see the experimental data.
All right, so let's go up to m equals 1/2 here, and this one sometimes does have a name -- anyone want to guess what that one might be called?
Half order, very good.
So, half order, and so here our rate, then, is going to be equal to k times a to the 1/2.
So, if we double the concentration here, what happens to the rate?
Someone want to yell it out?
So, 1 .
4 times the rate.
And m equals 0 -- any guess of what that's called?
Zero order.
And the rate law here, the rate is going to be equal to what?
K, and that's it.
So the rate equals k. So what does that mean -- if you double the concentration, what happens to the rate?
Yup, no effect on rate.
So that's zero order -- the concentration term, it doesn't matter what the concentration is, and there will be some that you'll see on a problem-set like that, so if there's no effect on rate, you have a zero order.
So those are the possibilities that you will see for the order of reactions, and again, you'll be given experimental data and have to figure out the order of the reactions.
And often, it's not just as simple as one thing, there'll probably be two things in there, so you'll have to figure out the order with respect to both of those two things.
So that makes it a little more complicated.
All right, a couple more things that are true about rate laws.
So the overall reaction order is this sum of the exponents in the rate law.
So then, if you had this rate law, rate equals k times a to the 2, second order, and b 1, the overall order for this would be?
3.
So, we have third order.
Sum of 2 plus 1.
So for this particular one, your second order with respect to a, first order with respect to b, third order overall.
Units.
Units for k are a lot of fun.
So it depends on what the reaction is, and often, for a problem, you'll have to figure out what the units for k are, depending on are we talking about molar per second or what's going on.
So just pay attention to the units for k, they can vary.
And sometimes one of the problems will be what are the units for this particular k, so you'll have to figure that out per problem.
So now, again, kinetics is experimental -- we determine rate laws experimentally, and sometimes this is not easy, because sometimes there'll be very small changes that you're looking at, or those changes will happen really quickly, so the interval of time is very short.
And sometimes when you're trying to measure these changes, the reaction's happening faster than your equipment can record those changes.
So this can be technically very difficult, and so one thing that scientists do is they use integrated rate laws.
So in this way, you can express concentrations directly as a function of time and you don't have to worry so much now about measuring those small amounts of changes that occur are very, very quickly.
So let's talk about integrated rate laws.
And here, we're going to start a derivation of an integrated rate law.
So here's -- we're told this is a first order reaction, and so we're going to do the first order integrated rate law.
And in this reaction, a is going to b -- we can write a rate expression for this.
so we can talk about the disappearance minus d a over d t. We can also now write the rate law for a first order reaction, and we know that that is our rate constant, small k, times the concentration of a.
So we know we can write the rate expression, and we can write a rate law for this first order reaction.
So now, we can do a derivation.
So, in this derivation, we're going to separate our concentration terms and our time terms.
So we're going to move everything with concentration to one side, and things with time to the other side.
So, on one side, then, we can have 1 over the concentration of a, we're going to bring that down here.
We have d a over here, and then we have our negative sign, and our k over here, and move d t also over to the other side.
So now we have concentration terms on one side and time terms on the other side.
And now, as you might have guessed from the name integrated rate laws, we can integrate, and so we can look at the interval from the initial concentration or original concentration of a, to concentration of a at some time, t. And we can also look at from some time, 0, to the time, t, in question.
So there's that expression again.
Now you can write this expression also in terms of natural log.
So we can re-write this in terms of the natural log of the concentration of a at time, t, minus the natural log of the concentration of a at our original time, or the original concentration, equals minus k t. And you can also express it in this term, so you can just bring this guy over to the other side of the equation, and this is one expression for the integrated rate law that you will see.
There's also another expression that you'll see, and let's show you what that is.
So you can also take the natural log and bring these two terms together.
So natural log of your concentration of a at time t, over your original concentration of a equals minus k t. And now you can take the inverse natural log of both sides, so just your concentration at time t over your initial concentration equals e to the minus k t. And that expression is often written in your book or on equation sheets as a concentration at a particular time equals the concentration of the original material, e to the minus k t. And these are the two expressions that you'll see the most often.
This one is often referred to as your integrated first order rate law, whereas this one is the equation for a straight line.
So these two are the ones that you will see the most often for integrated first order rate laws.
So one was an equation for a straight line, so let's come up with a straight line.
So if you plot your data, if you've measured your concentration of a at various times, you can take the natural log of those concentrations that you measured and plot them against time.
And if you do this, and it is, in fact, a first order reaction, you should get a straight line.
So here, we can look at this in terms of a straight line, so on y-axis, we have the natural log of the concentration of a at a particular time, and that's plotted against time, over here in seconds.
And so, what does that mean in terms of what is this, what is the intercept here?
Yup.
So the natural log of your initial original concentration of a.
And what is the slope of this line?
The slope is negative k, and so the slope is really what people are after.
So often what you want to do is measure rate constants for particular reactions, and if you can then measure the concentration of a at particular times and plot it this way, you can come out with your rate constant.
And that's what you want to know, and those are some of the problems that you're going to see.
Are you going to see experimental data for what happens to rate at particular concentrations, and you can come up with values for your rate constants.
So, let's talk about one other thing, which is half life.
So people are often very concerned with half life.
When do you hear about half life a lot?
Does anyone know?
Yeah, when we talk about radioactivity, which we'll be talking about next class.
They're very concerned about the half life, which is the time it takes for the full amount of your original material to be reduced by a 1/2.
So we can look at an equation that we had up above for our integrated first order half life, and we can think about this in terms of a half life expression.
So here, t has a particular special meaning -- we have t, 1/2, which is the abbreviation for half life.
And so half life by definition is the amount of time it takes for the original concentration to be reduced by 1/2.
So here we have our final concentration, concentration of a at a time t. So when we're talking about half life, we're going to have 1/2 as much as we had when we started.
So our a t is going to be our original divided by 2, 1/2 the amount we had in the beginning.
And so, now t has a special name, it's t to the 1/2 here.
So now we can simplify this expression to come up with an expression for half life for a first order reaction.
So, the a to the 0, our original concentrations cancel out, and so we have the natural log of 1/2 equals minus k, our rate constant, times t 1/2.
The natural log of 1/2 is minus 0 .
6 9 3 1.
We can get rid of our negative signs and solve for t 1/2, because this is a half life expression, so t 1/2 equals 0 .
6 9 3 1 over k. So that's an expression just for a first order half life.
And notice that half life does not depend on concentration here.
That term dropped out.
So half life depends on our rate constant, k, and k depends on the material in question.
So, for radioactive material, some radioactive materials have short half lifes, so their k's are very different from each other, and so that can be very important, especially when you're thinking about storing radioactive waste.
So, using that value, tell me, for the same material then, does it take longer to go from 1 ton to a 1/2 ton, or from 1 gram to 1/2 gram?
Let's just take 10 more seconds.
Yup, it takes the same amount of time.
And then just to finish up here, we have one more thing to do and then we're done.
So here, in a plot for a first order half life, the concentration of the material at the first half life has dropped by how much?
1/2.
At the second half life is what?
1/4
And the third half life?
1/8
Yup, so that's first order half life.
All right, everybody, have a happy Thanksgiving.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so kinetics we're continuing with today.
We talked on Wednesday about first order kinetics and we'll do a brief review of some of that.
And we're going to talk about second order kinetics.
Today we're going to come back up and talk about chemical equilibrium, which is something I love to do to review things that we've talked about before.
And we're going to start in on reaction mechanisms.
So, I thought I would mention, some of you have may have seen the activity in the Infinite Corridor, but today is world AIDS day, and today a lot of the compounds that are being used to treat HIV were actually designed based on making inhibitors to enzymes.
And so, to design those pharmaceuticals, people had to understand the reaction mechanism of the enzyme, and enzymes, of course, are catalysts in the body.
So, knowledge of what medical individuals needed to know to design these inhibitors to treat HIV are actually a lot from this unit that we're going to be talking about.
So, we'll be talking about reaction mechanisms, and we're also going to be talking about enzyme catalysis, which were key points in being able to come up with some of the current treatments against HIV.
All right, so just a little review from last Wednesday.
We talked about first order half life.
We talked about first order kinetics, we came up with an integrated first order rate law, and we also talked about half life.
And you told me last time that an example of first order half life is radioactivity, which we're going to be talking about today.
So, just a little review from last time, you have your first order integrated rate law, and the half life is defined as the time it takes for half of the original material to go away, and half life is abbreviated t 1/2, that's the symbol for half life, so the time for half of the original material to go away.
If you plug original material divided by 2 in there, then the original material a to the o for original, drops out and you come up with this equation of the natural log of 1/2 equals minus k t 1/2, and k is, of course, our rate constant.
And so, then we can take the natural log of a 1/2 and we get a value.
Rearranging that, you get half life equals 0 .
6 9 3 1 over k. And so you told me last time for this plot for first order half life, each half life, half of the original material goes away.
So, one example of a first order half life process is radioactive decay.
And the reason why this is a first order process is because the decay of the nucleus is independent of the number of surrounding nuclei that has been decayed.
So it's independent of the original starting concentration, and since it's independent, notice that's a blank in your notes, since it's independent, then that makes it a first order process.
So, we can apply first order integrated rate laws to radioactive decay.
So here were some of the equations we had last time.
We had, this is a different expression of the first order rate law where the material, concentration material of material a, at some particular time equals the original concentration, e to the minus k t, where you have your rate constant and the time that has elapsed, and we also just talked about first order half lifes with this equation here.
So, we can use those same equations, but often you don't see it in terms of concentration of a, you usually see these expressions in terms of either the number of nuclei or a different a, which is a for activity.
So instead of concentration, we're talking about the number of nuclei that have decay or capital N. So, we can write this same expression, but now using N instead of concentration of a, same thing, number of nuclei equal the original number of nuclei, e, to the minus k t, where k is our rate constant, or in this case decay constant, and t is time.
So, with chemical kinetics, we're usually talking about the change in concentration of things over time, but with nuclear kinetics, we're talking about the number of decay events, the number of nuclei that have decayed.
And so, here with nuclear kinetics, we measure these events using a Geiger counter.
So this can measure radiation, and I'm going to come around and just check the room.
And so, the gasket's ionized and then you hear different clicks.
See if you can hear the clicks as I come around.
So let's just see if we have any problems over here.
Oh, maybe a little bit.
No, this is fine.
There's always a little bit of radioactivity, it's all fine.
So, this is a Geiger counter, which will measure nuclear kinetics, it will measure radioactivity.
And my lab has this particular one, because we use x-rays in our experiments.
OK, we'll leave this on low just to check things out as we go along.
All right, so we do have a term, A, that we talk about in this unit.
Instead of concentration of A, it's activity.
And so, activity here, sort of the decay rate, is also called activity, capital A, and so this is equal to the change in the number of nuclei or our decay constant times the number of nuclei.
And people will often talk about the activity of particular radioactive compounds.
So, because activity is proportional to the number of nuclei, you can also take this expression and write it as this expression.
So you can have either the number of nuclei equal the original number of nuclei, e to the k t, or you can do it in terms of activity -- that the activity at some time is equal to the original activity, e to the minus k t. So, all of these equations can be re-written in this way.
So, let's talk a minute about units.
All right, so the activity for units, the new activity is Bq, Becquerel.
and that, actually is named after a French person.
Henry was his first name, and my French pronunciation is not very good.
This is the current unit.
It's equal to one radioactive disintegration per second.
The older unit, which you may be familiar with, is called the Curie, and that is 3 .
7 times 10 to the 10 disintegrations per second.
Does anyone want to guess, the Curie unit, who that was named after?
Marie Curie?
No, it was named after her husband actually, Pierre Curie.
And I actually always assume, because Marie is actually more famous than her husband.
But she, Marie Curie won two Nobel prizes, so we shouldn't feel too sorry for her.
Her husband shared the first Nobel Prize with her in, I think it was 1903, but then in 1906 he was killed in a road accident, run over by something that was crossing the street.
So he did not share the second Nobel Prize, because by the time that came around about 1911, he had passed away.
So, at first, we had the Curie, but then that turned out to be a really big number.
And so, when you were talking about sort of safe units for workers to be exposed to, if they were being exposed to things 10 to the 10, that really isn't very healthy.
So they wanted to have sort of a much smaller unit.
And so I guess that Marie Curie at that point talked about how her husband would feel about having the Curie not being the standard unit, but I guess she was OK with it, because if we had kept that same unit, then people would have been using it and it would of had to of been a really, really small number, because it was sort of picked to be set up to something that was too large, and she didn't want her husband's name apparently associated with a sort of an infinitesimally small quantity of something.
So, the Curie was sort of done away with.
And Henry Becquerel, who was one of the people who discovered radioactivity and shared that first Nobel Prize, had the unit named after him.
And I always ask the freshman chemistry class that as they go through MIT, if they ever discover a unit that is named after a female scientist, to please come back and let me know.
This was the one I thought was named after a female scientist, but as it turns out, it was actually Pierre Curie.
So, if you hear of any, please let me know for future reference.
So, the current unit you'll be using is Bq here for radioactivity.
So, you're not responsible for knowing all the different types of radioactivity.
When you're working problems, you can always get this information.
I'll just mention that a number of different kinds of radioactivity, some involve a mass change, some do not involve a mass change.
So, alpha decay, this isn't actually in your notes, there's a reference to where the table is.
You're not responsible for memorizing it, so I didn't put it in the notes.
An alpha decay is equivalent to a helium 4 nucleus, so you lose two protons, two neutrons, so that's a big mass change.
Whereas say a beta decay involves a loss of electrons, so there's no mass change associated with that.
So just to be aware that there are these differences in different types of radiation.
There's also really big differences in terms of half lives of radioactive isotopes, and again, this information would be given to you on a test or a problem-set, so you don't have to memorize it.
So, this table is similar to one in your book, and the point here is just how different half lifes can be.
So the abbreviation a here is year, d is day.
So you see some of these half lifes are in multiple years, some of them are days, so there are big differences in terms of the half life of some of these radioactive isotopes.
Some of them stay around for a really, really, really, really long time.
So, I thought I would share with you a poem about half lifes today.
And this was written by a former graduate student at MIT, Mala Radhakrishnan, and she is now a professor at Wellesley college right here in Wellesley, Massachusetts.
So, her poem entitled "Days of our half lives," is from her collection of chemistry poetry, "Chemistry for the Couch Potato."
And this particular poem involves the uranium 238 decay series.
So, here we go.
"Days of our half lives.
My dearest love, I writing you to tell you all that I've been through.
I've changed my whole identity, but loved, I can not pretend to be.
When I was uranium 238, you were on my case to start losing weight.
For 5 billion years I'd hoped and I prayed, and finally I had an alpha decay.
Two protons, two neutrons went right out the door, and now I was thorium 234.
But my nucleus was still unfit for your eyes, not positive enough for it's large size.
But this time my half life was really not very long, because my will to change was quite strong.
It took just a month, not even a millennium, to beta decay into protactinium.
But you still rejected me right off the bat, protactinium, who's heard of that?
So, beta decay, I did much more to become uranium 234.
Myself again, but a new isotope, you still weren't satisfied, but I still had hope.
Three alpha decays, it was hard, but I stayed on through thorium, through radium, and then radon.
I thought I would finally please you, my mass was a healthy 222, but you said, although I like your mass, I do not want to be with a noble gas.
You had a point, I wasn't reactive, so in order to please you, I stayed proactive.
A few days later I found you and said, two more alpha decays and now I am lead.
You shook your head, you were not too keen on my mass number of 214.
I had a bad experience with that mass before, an unstable acitone walked right out the door.
So in order to change, I went away, but all I could do was just beta decay.
My hopes and my dreams started to go under, because beta decays don't change a mass number.
To bismuth, then polonium, I hoped and I beckoned, my half life was 164 micro seconds.
And then finally I alpha decayed and then I was lead with the prize worthy mass of 210.
I've got to admit I was getting quite tired, my patience with you had nearly expired.
You were more demanding than any I dated, and much of my energy had already been liberated.
But you still weren't happy, but you had a fix, I really like the number of 206, So I waited for years until the day, which began with another beta decay, and then one more, and finally in the end I alpha-ed to lead 206, my friend.
To change any further I wouldn't be able, no longer active, but happily stable.
It took me billions of years to do, and look how I've changed and all just for you.
And wait, what did you say?
You've gotten so old that I'd rather be with a young lass of gold?
Well, I give up, we're through, my pumpkin.
Shouldn't all my effort be counting for something?
Well, you won't be able to rule me any more, because I'm leaving you, not for one atom, but four.
That's right, when you were away defusing, I met some chlorines that I found quite amusing.
So we're going to form lead c l 4, and you won't be hearing from me any more.
See, over the years I've grown quite wise, I've learned that love is about compromise.
You still have half of your half lives to live, so now you go out there, it's your turn to give."
And that is "The days of our half lives."
So, Mala takes great effort to make sure that all her poetry not only rhymes, but it is chemically correct.
So, it's a good way to review material to read "Chemistry from the Couch Potato."
All right, so let's do an example now and think about how things will change over time.
So we have an example, we want to know the original activity, and the activity after 17 years of a sample of plutonium.
So let's take a look at how we'll do this problem.
So first, given the information up there, the first thing we want to do is find the original number of nuclei.
So first, capital N o, the original number of nuclei.
So, we're given information about grams, so we have 0 .
5 grams.
And now if we want to know the number of nuclei, what's the first thing I have to do?
Convert from grams to what?
Moles
Moles, right.
And here we want to use the molecular mass that's given to us in the form of that isotope.
So here, we are given information about 239, and so that's the number we want to use in our conversions.
So we can convert that over, but that's going to give us moles, so how do we go from moles to molecules?
Avagadro's number, 6.022 times 10 to the 23.
This time we're going to talk about it in terms of nuclei per mole, and so that's going to give us 1.3 times 10 to the 21 nuclei.
OK, so now we know the original number of nuclei.
The next thing we're going to want to do is find k. And k is our rate constant for decay or our decay constant.
So what do we know about k for a first order process?
We know the equation for what?
For first order half life, right.
So that's o .
6 9 3 1 over t 1/2.
And in this problem we were given, the half life, and often you will be given the half life or you can look it up, and so we can put that in, so we have 0 .
6 9 3 1 over 7 .
6 times 10 to the 11 seconds.
And we can calculate our constant, which is 9 .
1 times 10 to the minus 13 per second.
So now we were asked to find the original activity and the activity after 17 years.
So, first we'll find the original activity, and the original activity is going to be equal to our rate constant times our original number of nuclei.
So we've just solved for both of these, so we can plug these in, so we had 9 .
1 times 10 the minus 13 per second times the number of nuclei, 1 .
3 times 10 to the 21 nuclei, equals 1 .
2 times 10 to the 9, and what are the units here?
It's just like a hum, it's hard to understand
Nuclei per second
Nuclei per second, which is the same as what?
That's equal to something else.
Yup, so that's the same as the Becquerel or the Bq, so it's defined as nuclei per second, or number of disintegrations per second.
All right, so let's do the last one.
OK, so now after 17 years, so now we can say that the activity at some time is equal to the original activity, e to the minus k t, and we can put in the activity that we just found, which is 1 .
2 times 10 to the 9 Bq, times e to the minus k, which is 9 .
1 times 10 to the minus 13 per second times 17 years, which in seconds is 5 .
4 times 10 to the 8 second.
So here, we want to make sure that our units are going to cancel, and this is where people often run into problems.
They'll plug in 17 years, and then a rate constant, which was calculated in seconds, and things will not cancel appropriately.
So make sure that you get your units consistent so that your seconds are going to cancel.
And so this term, if we do the math out here with the number significant figures, we find that that equals 1.2 times 10 to 9 Bq.
That term is insignificant in our problem.
So, the original radioactivity and the activity after 17 years are the same in terms of the significant figures.
And so, I choose this problem to emphasize a problem that we have, and that is radioactive waste.
It takes a very long time for some compounds to decay.
And so you have to think about storing radioactive waste, and think about a container that will outlast that radioactive waste.
And how do you know that the container is going to outlast the radioactive waste.
You can't really do an experiment because the time involved in doing the experiment, anyone who designs the container won't be alive by the time you're concerned about whether the container is going to be stable or not.
So taking radioactivity is an issue.
You heard some in the presidential campaign about whether both candidates believe in nuclear energy or not, and I think that both of them said, it needs to be considered, we need to have everything on the table.
If we're going to have a real uniform energy policy, we need to think about everything.
So, issues of radioactive waste and how to handle radioactivity safely are going to come back as being current, important topics.
And so these may be topics that you will, in your lifetime, have to deal with, either as a scientist trying to come up with new technologies, or as a citizen deciding whether having a radioactive plant in your hometown is a good idea or not.
A lot of people are happy about nuclear energy, as long as the power plants are nowhere located near them.
But, these are things that you'll have to face, and you probably will be voting on this in the future, if not dealing with it directly.
So that's how you do a problem in this.
All right, so let's talk about a medical use of radioactivity.
Radioactivity can definitely be our friend, as well as something to be concerned about.
And I think I mentioned this in the first day of class, one of the ways that the Chemistry Department has moved to being the number one ranked Chemistry Department of U.S. News and World Report over the years, is a little extra money that came in from the work of -- a patent from Professor Alan Davidson that we were able to do some pretty exciting things with that money over the years.
So I always like to mention all the great money-making discoveries that occurred using 511-1 material, and this is another example.
So, he used an isotope of technetium, and it's being used organ scanning, bone scans, it's one of the leading ones for heart imaging.
It's also been used recently in breast cancer.
It's estimated 7 million uses annually in the U.S. And so, this was patented as cardiolite, and it's really just very simple chemistry.
So you're using a d block metal, an isotope of a d block metal, which has your exciting d orbitals.
And what did he do, he made a coordination complex with that metal, an isotope of it, and he found ligands, cyanide ligands, those are pretty common ligands.
You've seen a lot of coordination complexes with cyanide ligands, and he tried different ligands to get the desired properties of stability and solubility, and that's all it was.
So he used some knowledge of radioactivity, knowledge of inorganic chemistry -- he was an inorganic chemist, he's retired now.
And simple coordination chemistry, and made an enormous amount of money for MIT, and particularly, the Chemistry Department, and also, this has saved a lot of lives.
So, imaging is something that chemists do a lot of, actually.
Not just imaging for cancer or imaging of organs, but also imaging of live cells to try to understand how the cell works when it's healthy.
And so recently, Professor Alice Ting, in the Chemistry Department, received an NIH pioneer award, NIH is National Institutes of Health, and started giving these pioneer awards for people coming up with very innovative ideas, the kind of innovative ideas that most people would not want to fund, because there's a good chance it might not work, but if it did, it would be spectacular.
So she received one of these awards for trying to develop technology to image protein-protein interactions in living cells, which is something that people would really, really love to be able to do.
And so she is involved in developing technology.
So developing of imaging tools is something that a lot of chemists do, it's a very popular area in chemistry.
And if it's something that you're interested in, there's definitely a lot of people around that you could think about working with for a UROP position.
OK, so that is first order.
And now let's go on and talk about second order integrated rate laws.
And we're going to have a little derivation for you, I always like to warn people that it's coming, because all of a sudden equations are coming in and out, and you just want to know where these equations are coming from.
So, as we talked about last time, this is an expression for rate law.
You have your rate constant, your concentration of something, a, and it's raised to a coefficient, and here that coefficient is 2, indicating it's a second order process.
So if there's nothing up there, that's 1.
And then 2, and again, the order of the reaction can be positive, negative, it can be integers, it can be fractions.
But this is second order, so we have 2.
Now, as we did with the first order expression, we're going to separate our concentration terms and our time terms.
So we're going to bring our concentration term over to one side, another concentration term here, and we're going to have our rate constant and our time term on the other side.
And now we're going to integrate, because it is an integrated rate law.
So we can integrate from the original concentration of a to the concentration of a at some time, t, and then we'll also integrate from zero time to that time, t, on the other side.
Now, I'm going to take this expression and just bring it up to the top of the page, so that's the exact same expression, nothing has happened.
And now we're going to solve that integral.
So we can solve that integral, and if you want to look at these -- the back of your textbook has all of these conversions, if you want to look at them.
So, we're going to solve that integral, now we have minus parentheses 1 over the concentration of a at some time, t, minus 1 over the original concentration equals minus k t. We can get rid of some of these minus signs.
So we're going to bring the concentration of time, t, over on this side, we have our k t, and now we have this other term, the original concentration term is on the other side, and this is expressed in a certain way that gives you the equation for a straight line.
And again, kinetics, you need experimental data for kinetics, and so when you measure your data, you plot your data, and so there's a lot of equations for straight lines that you have, because it's all about trying to plot data, and figure out what the order is experimentally.
So, here's an equation for a straight line, and we can plot this, and you can tell me what the intercept of this line is.
OK, let's just take 10 more seconds.
Yup, so all of you know how to analyze the equation for a straight line.
So, here we have 1 over the original concentration, and then our slope is equal to what here?
k. So, you can plot your data as your concentration of a changes with time.
You can plot the data, and if the data, if it's plotted as 1 over the concentration of a versus time and it gives you a straight line, that's consistent with it being a second order process.
So, in terms of second order half life, we talked about first order half life, and for any half life, it's just the time it takes for half of the original material to go away.
So we can rewrite this and take a to the t, and substitute in the original concentration divided by 2, and then we can have t have a special name t 1/2, so that's the half life.
And now we can just simplify this expression.
We can bring the 2 up here, and now we can combine our concentration terms on the side.
So we take 2, we bring this over, minus 1.
And that simplifies 1 over the concentration of the original material here.
And now we can solve for it in terms of the half life.
So the half life for a second order process equals 1 over k times the original concentration of the material.
So, a second order half life depends on the starting concentration.
So that's very different from a first order half life process where concentration term cancels out entirely.
So for a first order process, the concentration of the original material does not affect the half life, or for radioactive decay, the original number of nuclei -- it's independent of how many nuclei were around at the time.
But for second order process, the starting concentration does matter.
So again, chemistry is experimental.
And so what you would be doing in a lab, you would be trying to figure out what the order of the reaction is, and so you could try out your data.
You say I don't know if it's first or second order, so for a first order plot, you're going to be plotting the natural log of your concentrations versus time, and if second order, you plot 1 over the concentration versus time.
And so you could plot your data and see that oh, look at this, it fits a straight line really well if I plot it as 1 over the concentration.
If I plot it as the natural log versus time, the data doesn't fit a straight line at all.
So this is not a first order process, this is much more consistent with a second order process.
So again, figuring out where something is first order or second order is done experimentally.
All right.
Now we're going to talk about kinetics and equilibrium constants.
So, I always get very excited, as you know, when we come back to equilibrium constants, so am always very happy at this time in the course when we can relate kinetics and equilibrium constants.
So, at equilibrium, another way to think about what's happening at equilibrium, is that the rate of the forward reaction and the rate of the reverse reaction are equal to each other.
So, we can now talk about big letter K, which is our equilibrium constant again, and our little letter k's, which our rate constant.
So the equilibrium constant for a chemical reaction a plus b equals c plus d is going to be equal to what, what do I put on the top?
Concentration of?
c, and concentration of d, right.
So our products, and at the bottom we put our concentration of our reactants, or a and b.
Now, we can also think about this reaction in terms of little rate constants.
So we have small letter k 1 on top, and small letter k to the minus 1 on the bottom.
So, the forward reaction, the rate of forward reaction is going to be equal to k 1 times the concentration of a and the concentration of b.
And on the bottom, our rate is going to be equal to the little rate constant, so for the reverse reaction it's the reverse rate constant, k minus 1, and in the reverse direction, our reactants are the products for the forward direction, or c and d. So, here we have these rates, and at equilibrium, those rates are going to be equal.
So at equilibrium, little k 1, a times b is going to be equal to little k minus 1 times c times d. And at equilibrium we had c d over a b is equal to then k 1 over k minus 1, so if we just rearrange this expression and move the rate constants to one side and concentration terms to the other side, this expression is the same as this expression, and we also know what this expression is equal to, which is our big K. So, therefore, our equilibrium constant equals the rate constant for the forward reaction over the rate constant for the reverse direction.
And so here is an expression that compares equilibrium constants with rate constants.
So now, let's think about what is true about this.
So our equilibrium constant, then, is the ratio or the forward rate over the reverse rate for these elementary reactions.
And if we think about rate constants in kinetics terms, if k is greater than 1, if there are more products than reactants at equilibrium, what's true about k 1 and k minus 1?
Is k 1 greater than or less than k minus 1?
Right.
So the forward rate constant is greater than the reverse rate constant.
And if K, big equilibrium constant, K is less than 1, if at equilibrium there are more reactants than products, what is true about this relationship?
It would be less than.
So, you can think about equilibrium constants now in terms of rate constants, which we'll be doing a lot on Wednesday, too.
All right.
So let me introduce you to a couple of more terms in the last few minutes.
So reactions don't usually occur in one step, but occur in a series of steps.
Each step is called an elementary reaction.
So, the overall reaction, the order and the rate law, can be derived from the stoichiometry for an overall reaction, you can't use the stoichiometry, but for an elementary reaction you can.
So, for an elementary reaction, say one step in the reaction mechanism, that step occurs exactly as written so you can use stoichiometry.
And that's going to be handy in coming up with mechanisms.
So, let's just look at one example very briefly.
So here we have the decomposition of ozone, which is another environmental issue that you will be faced with in your lifetime.
So this is the overall reaction, and you can't use the stoiciomery to figure out the order of the reaction, but if you divide it up into elementary reaction steps, then you can use the stoiciomery to write the rate law for each step of that reaction.
So, the first step here is a unimolecular step.
You have one thing going to two things, and molecularity is the number of reactant molecules the come together to form product.
So, unimolecular you just have one thing that's forming some kind of product.
What do you think it's called if you have two things forming a product?
Bimolecular.
These are good little one or two point questions on a test, they're not very hard, they should be pretty easy to think about.
So, we have unimolecular, examples would be some kind of decomposition or radioactive decay.
Bimolecular, two reactants coming together to form products.
And termolecular is three reactants coming together to form a product, and that's rare.
And you can remember that it's rare if you think about how you would hold three tennis balls in your hand and have them all come together at the same time to form product, that is a difficult thing to do.
Two things coming together is easy, bimolecular is very common.
Termolecular not very common, that they'd all come together at the same time to form a product.
All right, so we can write rate laws for each step here.
For the first step here, the rate would be equal to a k -- I don't have the k written up here, but there's always going to be a little k over the arrow.
So, k times the reactant would be here.
For bimolecular, again, assume a k over there.
What's that rate going to be equal to?
OK, let's just take 10 more seconds since class is almost over.
Yup, so it's this one right down here.
So we have the rate is equal to k times these two reactants.
You can sum up the steps and get the overall reactants.
Notice that o is an intermediate, it's formed here, decayed here.
It goes away, and so o doesn't appear in the overall expression.
So we're going to talk a lot about reaction intermediates next time.
And also, remember that you can't prove reaction mechanisms to be correct, they're just consistent with the data that you have.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Please note that this is a clicker competition, perhaps the last of the year, before you put in your vote for this particular clicker question.
And the defending champs are recitation three, and I believe that no recitation has won more than once.
Is that true?
Is there a recitation that has two victories?
So, this might be your last chance if your recitation has not won, or if you want to be decisive victors in the clicker competition to have a repeat victory, this is your chance -- no pressure.
But go ahead and put in your clicker response.
OK, let's just take 10 more seconds to click in.
OK, the answer here is b, and the trick was to recognize which is the equation for first order half life, and which is the equation for second order half life.
Most people got this right.
And we picked this clicker question to demonstrate a point that will, for the final exam, which is that you will have an equation sheet of all the equations.
There'll be a few you have to memorize, we'll let you know what they are, but that's a lot of equations.
So, so far on each hour exam, you've had an equation sheet that's been related to that particular material.
On the final, you of an equation sheet that has the equations from exam 1, exam 2, exam 3, and also the fourth exam material.
And so, it doesn't say on them, oh, first order half life equation, colon, and then the equation, it just has all of the equations.
So one thing that you have to do is recognize which equation can be applied where.
And this is something that people, occasionally that's part of knowing the material is to know/recognize the equations.
You don't have to memorize them, but you do have to be able to recognize them and know when to apply them.
So that's something you should be thinking about as you're reviewing the material for the exam.
And we're just going to at jump right in and talk about reaction mechanisms.
So this is the last type of question that's on problem-set 10.
So, when you're investigating reaction mechanisms, you often will have an experimentally determined rate law, and then you want to try to come up with a mechanism that's consistent with that rate law.
So, for this particular reaction, we have 2 n o plus o 2 going to 2 n o 2 gas.
And experimental rate laws determined there was a rate constant called k obs, for k observed, the observed rate constant that was calculated.
And we know that it's the order of the reactions here.
Overall, what is the overall order at this particular rate law?
Three.
So we have second order in n o and first order in o 2.
So if it's overall order of three, is it likely to occur in one step?
No.
What would it be called if it did occur in one step -- if three things came together at the same time to form a product?
Yeah, thermonuclear.
And we learned last time that thermonuclear reactions are rare.
So it's unlikely that those three things, the two n o's and the one o 2 are going to come together at the same time to form product, so it probably has more than one step.
So, on the board here and on your notes, we have a two-step proposed mechanism, and let's talk about these two steps and come up with a rate law that's consistent with this mechanism, and then see if that agrees with the experimental rate law that was determined.
So, in the first step of this reaction, we have 2 n o's coming together.
There's a little rate constant, k 1, above the first arrow, and for the reverse direction, we have k minus 1, so this is written as a reversible reaction, and it's forming an intermediate n 2 o 2.
In the second step, we have the o 2 coming in, reacting with that intermediate with a rate constant, k 2, and it's forming two molecules of n o 2.
So now we can write the rates for each of these individual steps.
We can write the rate for the foward reaction here, and so that's going to be equal to k 1.
And we have n o, two n o's, so n o squared.
And again, this is a step or an elementary reaction so we can write the reaction exactly as it occurs.
We can write a rate law for that step using the stiochiometry of the reaction.
For an overall reaction, you can't do that, it has to be experimentally determined.
But for an elementary reaction or a step in a mechanism, you can do that.
So we write it exactly as it occurs.
So, what would be the order of this particular reaction as written here?
Two.
And what about molecularity?
It would be bimolecular.
So we're just getting you used to some of these terms.
All right, so then for the reverse direction, the rate of the reverse direction would be equal to what rate constant?
k minus 1.
Times what?
Yeah, n 2 o 2.
So what's the order here?
One.
And what's this called?
Yeah, so that's called unimolecular.
All right, what about step 2?
What would be the rate here, and that is a clicker question.
OK, just 10 more seconds.
Yup, so we write it exactly as written, so we have a k 2 times the concentration of o 2 times the concentration of n 2 o 2.
All right, so what is the overall order here?
Two.
And again, that would be the bimolecular.
All right, so we have our two steps, and we've written rates for all of those particular steps.
Now we're interested in writing the rate of the overall reaction that forms n o 2.
So, let's think about what the rate of n o 2 formation is.
So, in the last step, we're forming n o 2, and we can just use that last step to write this out.
We're forming two molecules of n o 2.
So we're going to put a 2 in there, because the rate at which the concentration of this will increase twice as much as will decrease this amount.
And the book is somewhat inconsistent in its use of 2's in these equations.
If you're forming two molecules in your last step, there should be a 2, but 2 is not always in the answer key.
So what I've done in the past is if you put a 2 and there should be a 2, great, but if you don't put it, that's OK, too, so I have not taken off for this because the book is not been completely consistent in its use of that.
But, if you see a two, and 2 things are being formed, you know where that comes from.
So, 2 times k 2, that's the rate constant of that last step, and that we're talking about the concentration of o 2 and the concentration of n 2 o 2.
So we're writing out just that last out there with the 2, because two molecules of n o 2 are being formed.
But we're not done with this, we can't use this, because it has an intermediate term in there.
And if we're going to write the rate of a reaction, the rate of product formation, we can't have any intermediates in our equation.
We need to solve it in terms of products and reactants.
So, we need to solve for n 2 o 2.
So we need to solve for the concentration of n 2 o 2, and substitute that into our equation.
So we need to solve for it in terms of products and reactants.
So, we need to think about how this intermediate is formed, and we need to think about how the intermediate is decomposed, and we need to think about how this intermediate is consumed.
So, the net formation of n 2 o 2 is going to be equal to the rate at which it's formed.
So the rate at which it's formed.
What step is it formed in, the intermediate formed in?
It's in step one in the forward direction, so the rate at which its formed is k 1 times n o squared.
So that's the forward rate of the first step.
Then we need to think about the rate at which its decomposed.
What step does the intermediate decompose in?
Right, the reverse of the first step.
So we're going to put a minus sign here, and k minus 1 times the concentration of the intermediate.
And then we also want to think about the rate at which it's consumed.
And it's consumed in the second step, so that's k 2 times the concentration of the intermediate times the concentration of o 2.
So that's the net formation.
And now we can use something called the steady state approximation, and the steady state approximation is used in all sorts of kinetics, including enzyme kinetics, and it can be phrased two different ways, which are equivalent.
You can say that the steady state approximation is that the net formation of the intermediate equals zero -- i.e.
this is the net formation of the intermediate, so this entire equation equals zero.
Another way to say it is that the rate of formation of the intermediate equals the rate of decay.
So that's saying the same thing, that this term is going to be equal to those two terms.
That's saying the same thing.
So, if we can set this whole equation equal to zero, then we can solve for the concentrations of the intermediates in terms of the rate constants and the concentrations of the reactants and the products, which is what we need to do.
So, let's do that.
So we can rearrange this expression now that we have set equal to zero, and so we can bring n 2 o 2 over to one side, so we have our intermediate on one side.
And then we are going to have k minus 1, and our little k 2 times the concentration of o 2.
And so, that is the rate in which the intermediate is decomposed and consumed, and that's going to be equal to the rate at which it's formed, which is another way of expressing the steady state approximation.
So we've just moved the decomposed and consumed to one side, and we have the rate at which it's formed on the other side, and now we can easily solve for the concentration of the n 2 o 2, so we're going to do that over here.
So that's now going to be equal to our k 1 times n o squared over k minus 1 plus k 2 times the concentration of o 2.
So now we've just solved for n 2 o 2, our intermediate, in terms of our rate constants and in terms of our reactants.
So that's good.
Now we need to take this and substitute it back into here.
And if we substitute it back into here, let's do that over here, so now we'll have k, 2 times k 2, we'll also have a k 1 term.
We're going to have, we have n o squared.
We also have an o 2 term from over here, and this is all going to be over k minus 1 plus k 2 times our concentration of o 2.
So we've just substituted that term in here, and now we have a new expression for the rate of our product formation, and that's lovely.
Except that it doesn't agree with the experiment.
So we have a problem.
In the experiment, there's n o squared and o 2, but there is no o 2 in the bottom, so something is going on.
So that means that our reaction as written, which didn't have any fast steps or any slow steps is inconsistent if you write a mechanism for this, if you write a rate law for this, it's inconsistent with the experiment.
So we need to do more to our proposed mechanism.
We need to add fast steps and slow steps until we can write a mechanism that agrees with the experiment.
And so, if we want to do this, we're going to assume, now we can try that the first step is fast and the second step is slow.
So let's talk about this slow step and what happens if you have a slow step.
So I'm going to introduce this term of rate determining step.
So the slowest step in an elementary reaction is going to determine the overall rate.
If it's slow compared to the other steps.
And so, the overall rate, it can't be any faster than that slow step, and that slow step is going to govern how fast the overall reaction is.
All right, so let me give you an example of this.
I know they're you're all very anxious to complete problem-set 10, the last problem-set that will be graded in this course.
And you know, because I told you at the beginning of class, that after today's lecture, you will know all the material that will allow you to complete that problem-set.
So some of you are going to get really antsy at the end of class thinking OK, I can go and do that problem-set.
So, you're going to be ready, you may have already started packing up your backpack as the class is winding down, you see we're getting close to the end of the handouts saying, all right, I've got to get out of here, I've got to get to the library and finish the rest of problem-set 10.
All right, so let's say it takes you five seconds to pack up and be out the door.
Or maybe there's some people coming in, OK, maybe ten seconds to get out the door.
Then you're running down toward the library and you're moving pretty fast, you're jumping over people that are stopping and talking to their friends, you're moving very quickly, you might slide down the banisters to get to the first floor, and you get to the library, but you were too slow.
Even though you did it really fast, all the tables are busy, everyone else is completing problem-set 10.
So you go up and down, you're looking for a table, there's no tables, they're all busy, everyone has their chemistry textbooks out and they're going.
All right.
So, then you have to leave the library, you go to building 2, that's close by, they're classrooms, they're not in use, the first couple you check there's recitations going on.
Finally you find the one that's empty.
When you find it that's empty, your backpack is off, your books are out, your calculator's out and you're going.
Right, maybe another ten seconds.
But it took you about 20 minutes to find that free table.
So, maybe ten seconds to get out of this classroom, ten seconds to get your books out of your backpack, but then 20 minutes overall to find that free table.
So that's the rate determining step, that 20 minutes to find that table is going to control the overall rate of the reaction.
And so, many of you have probably experienced rate determining steps in your life, some of you may have friends that are always your rate determining step, and so you know about this.
You know about this.
And so rate determining steps allow us to come up with different kinds of mechanisms and simplify different expressions.
All right, so let's look at what this does for us here.
If we make assumptions now about a step being fast and a step being slow.
All right, so we're going to be able to -- this expression here, we're going to be able to simplify.
All right, so we're going to ask the question then, here, we're assuming that we're going to say, all right, let's just say the first step is fast and the second step is slow.
We're going to see whether that gives us an answer that's consistent with the experimentally determined rate law.
So here, if we say this is fast and this is slow, basically we're asking the question, is the decomposition of the intermediate or it's consumption faster?
So as we wrote this, then, we're saying decomposition is fast.
The first step is fast and reversible, so the decomposition is fast.
The second step is slow, the consumption of that intermediate is slow.
So that allows us to change our rate.
So, what we're then saying in terms of the here's the words, here's sort of the equations, we're saying that this term here, this k minus 1 times the intermediate term, is a lot bigger, because this is faster than this.
This is slow, the rate of consumption is slow.
So k minus 1 is going to be a bigger number than k 2 times o 2.
So, if we say that, if we say this is fast, that's a big number, this is a fast step, k 2 is a small number, it's slow, if this is a lot smaller than k minus 1, and these are both appearing in the bottom of the equation here, that allows you to get rid of this term.
So if you have something really big and something pretty small, on the bottom of this equation, the really small thing is kind of insignificant.
And so we can get rid of that here.
So we can get rid of this term.
So that allows us to simplify this expression.
Now we have k 1 times n o squared over k minus 1.
We can also rearrange that and bring our concentration terms on one side, so we have our intermediate over our n o squared and have that now is going to be equal to k 1 over k minus 1.
What is little rate constant k 1 over rate constant k minus 1?
Big k, right.
So that's another way of saying the equilibrium expression.
So it's the equilibrium constant for the first step is equal to k 1 over k minus 1.
So basically what we're saying then, if we have the first step being fast and reversible, and the second step being slow, we're saying that the first step is really an equilibrium.
So what allows us to say that?
Well, if you have a fast reversible step followed by a slow step, that means that not much of your intermediate is being siphoned off by the second step, allowing you to reach equilibrium.
So you can think about that in terms of this plot here.
If you have reactants going to intermediate, and this is fast and reversible, so they're going back and forth, fast, reversible, and very little of that intermediate is getting siphoned off, this is very slow.
So very, very little of this is being siphoned off to products.
So that allows that first step to really reach equilibrium conditions, because this doesn't really factor in, and so you reach an equilibrium here.
And that allows you to simplify your expressions.
So now, we can go back with our simplified expression, and we can express it as rate constant k 1 over rate constant k minus 1 times n o squared, or we can write it as the equilibrium constant 1 times n o squared, and we can plug that in solving for our intermediate.
And so, if we put that now back into this expression, we have the 2 k 1, k 2, we have our oxygen concentration and our n o squared over k minus 1, or we can have equilibrium constant 1 with our k 2 in our 2.
And now, we're consistent.
If we realize that the k observed that was given in the experimental rate law is a combination of all of our k terms.
So it's a combination, you can't, at least in this experiment, you weren't able to come up with the individual rate constants, they just came up with overall observed rate, and so that's all of our terms in there.
And so if we put that in here, we realize that this is consistent with the experiment.
And in doing these problems, it's OK to leave your little rate constants, it's OK to use equilibrium constants in there, you should show all of your work, so that if you have a k obs in your final answer, we can look back and see what that was equal to.
So it's hard not to show all your work as you're writing through all of these on a test, but you can end your answer, it would be correct end your answer here or here or here.
So I don't care which k is in your final answer, but definitely show all your work.
So that agrees with the experiment.
All right, so let's look at another example now.
And now we have, again, a 2-step reaction, and the first step is fast and reversible, and the second step is slow.
So, with those predictions of the mechanism, let's just go right through and try to solve for the rate here without going through all of the steps we did the first time and see if we can do this more efficiently.
And again, this is a reaction with ozone, and depletion of ozone is a major problem that will be facing us, that has faced us, and will continue to face us in the future.
All right.
So lets first think about the rates of the individual steps.
So the rate of the forward reaction here is going to be what?
Just yell it out.
Yup, k 1 times the concentration of o 3.
What about for the reverse step?
Yup, it's like clear and then it gets mumbly.
k minus 1 times the concentration of o 2 times this concentration of o, which is an intermediate.
All right, so let's look at the second step here.
So the rate here is going to be k 2 times this intermediate -- concentration of this intermediate, o times the concentration of o 3.
All right, so now we know that this second step is going to be controlling the rate, it's a slow step, at least that's our prediction that we're going to write a rate expression based on.
And so, that's going to be our slow step, our rate determining step.
And again, we're forming two molecules of o 2 in this step.
So we can have a two in there.
And that the rate of formation of o 2 in this last step, 2 times k 2 times our intermediate o times o 3.
Are we done?
Was it that easy?
Are we done?
No.
Why isn't this finished?
There's an intermediate.
Intermediates will not be your friend in doing these problems.
Yes, we have an intermediate so we're not complete.
We need to solve for o in terms of products, reactants, and rate constants.
So we're not done.
So we need to get rid of the intermediate.
But now, we can do that more simply because we've set this up that we have a fast reversible step followed by a slow step.
So we can solve for the concentration of that intermediate in terms of equilibrium expressions.
So, if you haven't learned how to write an equilibrium expression yet, remember you need it for the unit on chemical equilibrium, the unit of acid base, the unit of oxidation reduction, and here you need it again in kinetics, so definitely want to be able to write those for the final.
So we can go ahead and write those.
So, for the first step, again, it's products over reactants.
You can also express that in terms of rate constant k 1 over rate constant k minus 1, and that's all equal to our big k, our equilibrium constant, k 1.
And so now we can solve in terms of o, our intermediate, and so here, we pull that out, here we've solved for it in terms of k 1 times o 3 over k minus 1 times o 2, so we just brought those to the other side.
You could have also used the equilibrium constant here instead of the rate constants, either one is OK.
So, now we've solved for it here, and we can put it down into this expression here.
So we are going back to our overall expression that we wrote.
So the expression that we wrote based on this was 2, k 2 times the concentration of our intermediate, times the concentration of o 3.
Now we substitute in for the intermediate, so we have 2 times k 2, k 1, we have o 3 and an o 3, so that squared, over k minus 1 times o 2.
And this can also be expressed in terms of k observed, putting those terms together, o 3 squared over o 2.
And that would be the answer to what the rate is for that particular proposed mechanism.
So, let's think about if that's true, what we should expect if we do some experiments, what we should expect about the order of the reactions, and what would happen if you double concentrations of things and look for the effect.
So if you're going to test this proposal, what would we get?
So, what is the overall order of this rate law here.
Oh, sorry, the order first let's do of o 3.
2.
So if we double the concentration of o 3, what would you expect to see in terms of the rate?
It will what?
Yup, quadruple.
What is the overall order of o 2, or order of o 2, and that is a clicker question, actually.
So you know what some people think.
And not only tell me the order, but tell me the effect of doubling.
OK, let's just take 10 more seconds.
Very good.
All right, so the overall minus 1, that's what it means if it's on the bottom of that equation there.
And so if we double it then it should half.
So overall then, what is the order of the reaction?
It's 1.
And so, remember you can sum up the order of the individual ones to get the overall order, so the overall order is 1.
And what about if you double both things, what's going to happen to the rate?
OK, let's just take 10 more seconds.
OK, people are doing quite well on this.
All right, so you double both and you double.
So, everyone's in very good shape on these types of problems, which is excellent because this is worth a lot on the final.
All right, so let's do a final example, and in this case we're given an experimental rate law, which is up here, k observed times the concentration of n o and the concentration of b r 2, and we want to figure out from this, we'll look at if we have one step is being fast and the other step is being fast, and see which would be consistent with this experimentally determined rate law.
So these are some of the types of problems you have, your given experimental rate law and say tell me which step has to be fast and which has to be slow to have a rate that's consistent with the observed rate.
So, in this reaction we have 2 n o plus b r 2 going 2 n o b r. So if we write this in the first step, we have n o plus b r 2 going to an intermediate, n o b r 2 with forward rate constant, k 1, and reverse rate constant, k minus 1.
And in the second step we have the intermediate interacting with another molecule of n o forming two molecules of n o b r, our product.
So let's consider what the rate of the forward reaction is here, and so that would be -- you can yell it out.
k 1 times -- yes, excellent.
And for the reverse reaction then we're going to have our k minus 1 times our intermediate concentration.
Now we can look at step 2 where our intermediate is being consumed, and the rate would equal then k 2 times the concentration of the intermediate, n o b r 2 times the concentration of n o.
All right, so we know nothing now about fast or slow steps, so we can write the formation of our product just in terms of this second step.
And again, there are two molecules being formed, so we have a 2, and 2 times this rate for this second step, k 2 times intermediate times n o.
So again, now we have an intermediate that we have to get rid of.
We can't have an intermediate in our final expression.
But we can't use the equilibrium constant right now because we don't know anything about what are the fast or the slow steps.
So we're going to write it the long way with no assumption about fast or slow steps.
So to do this, so we have our intermediate, and so we're going to solve for it in terms of products and reactants.
So we're going to write about the change in the concentration of that intermediate.
So, the intermediate's being formed in the first step.
So we first write the rate at which it's being formed.
So it's being formed here, so that's k 1 times n o times the concentration of b r 2, so this is how it's being formed.
And then we're going to consider how it's being decomposed, and you told me before it gets decomposed in the reverse of the first step.
So we have a minus sign, because this is a negative change in concentration, it's being decomposed, and it's being decomposed with a rate constant of k minus 1 times the concentration of that intermediate.
And the intermediate's also being consumed, so that's decreasing the concentration of the intermediate, and it's being consumed in this second step.
So it's being consumed here with the rate constant of k 2 times the intermediate times n o.
So that second step.
So this is how we write it if we don't know anything about the rate at which -- what's slow and what's fast.
So this will always, always work to write things this way.
So that's the overall change in the concentration of the intermediate, how it's being formed, decomposed, and consumed.
Now we can use the steady state approximation, and you can always use the steady state approximation in these problems, and that says that this net change is going to be equal to zero.
So the steady state approximation, then, let's you set this entire term equal to zero.
And sometimes, a question you might get is to say what the steady state approximation is.
So you should be able to articulate that in words as well as use it on a problem.
So now we can set this all equal to zero, and we can solve for our concentration of our intermediate.
So, rearranging this then, bringing our intermediate terms on one side of the expression, and again, here's the rate at which it's decomposed and consumed being equal to the rate at which it's being formed.
And we can pull out our intermediate, so then that's times k minus 1, k 2 times n o, and that's equal to the rate at which it's being formed.
And so now we can take that, divide by that term, and we get an expression for the concentration of the intermediate in terms of our reactants and our rate constants.
So now we need to plug that back in to the original equation that we had.
So this is what we solved for, and now we can put it into here, and if you rearrange that, we have the 2, we have the k 1, we have the k 2, we have an n o here, we have an n o there, so it's squared, b r 2 here, and then on the bottom we have k minus 1 plus k 2 times n o. OK, so that's great, but that's not consistent with the experimentally determined rate law, so something has to be fast and something has to be slow.
So let's first consider if the first step is slow and the second step is fast, which would mean that the second step is bigger, it's faster, that's k 2 times n o would be bigger than k minus 1.
So the second step then is fast, faster than this.
So now why don't you go ahead and tell me how this simplifies if that is true.
All right, let's just take 10 more seconds.
Yup.
All right, let's look at why.
All right, so that's the answer and let's consider why.
So that means if this is much bigger than this term, that cancels out.
If that cancels out, what else cancels?
k 2 also cancels, and what else cancels?
one n o cancels giving you this expression.
And that is consistent with the answer up there.
So you could also see that as the k observed.
So that's consistent.
And let's just quick -- oh, first tell me the overall order.
2.
And now let's just look at what would have been true if we had done it the other way around.
So if we did it the other way around, we'd say that the first step is fast so that's a lot bigger than this.
If this is a lot bigger than that, we'd cancel out this second term here.
Can anything else cancel?
No.
So, we're left with this expression, and that is not consistent with this up here, because we have this squared term.
So even if you write it as k observed, that's not consistent.
And so, the overall order for this would have been 3, which is not consistent with the experiment.
All right, so now you know everything you need to know to finish problem-set 10.
Don't let anything be your rate determining step.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, let's finish up new material today, new material Monday, and then we're done with the kinetics unit.
So, we're going to talk about temperature, collision theory, and activated complex theory today.
So on the first day, we talked about kinetics, we talked about factors affecting the rates, and this was in your handout on that first lecture, but just to kind of review for a minute what we have talked about so far and where we're going.
So these are some of the things we came up with are factors affecting the rates of reaction.
People said mechanism affects the rate of reaction, so we talked about mechanism.
Concentration of material, nature of material -- that pretty much fits in in every lecture.
We've been talking some about that, say, if it's first order or second order, if you have a concentration term in there, or if you don't have a concentration term.
The nature of the material, we're going to talk more about that today.
Temperature is going to be the number one topic for today, and on Monday the number one topic is the use of catalysts to speed up reactions.
So, by Monday we'll be done with this list.
So, what about temperature.
So temperature, when I said what affects the rate of reaction, temperature was one of the things that you yelled out at me, and that's very much true, it has one of the biggest effects on the rate of reactions.
So when we were talking about the gas phase, there was an observation made that the reaction rates tend to increase as the temperature increases.
And most of you can think about this and are aware of it, and today we're going to talk more about the quantitative affect of this, how much does the rate of a reaction increase when the temperature increases, how do you know what equation do you use.
So, this idea has been around for awhile.
So in 1889, Arrhenius plotted rate constants versus temperature, and he found that if he plotted the natural log of those rate constants versus inverse temperature, then he would get a straight line.
So let's look at the plot that he had.
So, he found that if he plotted natural log of the rate constants versus 1 over temperature, so units or kelvin to the minus 1, that he got a straight line.
Which means that the rate constants are varying exponentially with inverse temperature.
So you have to use a natural log to get a straight line, if you don't use natural log, you wouldn't get a straight line here.
So, let's look at some of these terms that we have here, and here's our equation for the straight line, the natural log of k equals minus e a -- e a is activation energy, which we're going to talk a lot about today -- over r t, r is our friend the gas constant, and t is temperature, plus natural log of a.
So what is a?
A is called factor a or sometimes it's called pre-exponential factor, and had has the same units as k, k being the rate constant.
So, let's think about this equation and this plot, and about factor A and this other term, activation energy.
So, factor A and activation energy depend on the reaction being studied.
So, it depends on the nature of the materials involved.
So I said we'd talk again about nature of the material.
So one kind of reaction's going to have one activation energy, another one will have a different one.
So, let's think about this term, factor A.
What is it exactly?
Do you think it would be temperature dependent?
What do you think?
How many people think yes, it would be temperature dependent?
How many people think no?
Some people are not going to commit.
The answer is no.
So, let's think about what factor A is.
So, factor A is the rate constant at some really, really large temperature.
So, if we look at this plot, the natural log of k equals the natural log of A, or k equals A if you're along this axis here.
If you're along that axis here, that's at zero, so that's when 1 over temperature equals zero, then the rate constant equals factor A.
What would be true about the temperature for 1 over temperature to equal zero?
Very big.
So, at the biggest, hugest temperature you can imagine, the rate constant's going to equal A.
And so, factor A depends on the nature of the material being studied and you would have to determine what that value is.
But it's definitely not temperature dependent.
It's the value of the rate constant at an enormous temperature.
So, what about activation energy?
Do you think that's temperature dependent?
Why don't you tell me what you think.
All right, let's take 10 more seconds.
All right, so I'd like you to discuss this amongst yourselves and see whether you're happy with this answer.
All right, go ahead and vote again.
All right, 10 more seconds.
And the correct answer is?
So, I think most of you will remember this answer.
So the activation energy is not temperature dependent.
You can calculate what the activation energy is by plotting the rate constants versus 1 over temperature, and then you get it from the slope of the line.
So, activation energy depends on the nature of the material, but it isn't temperature dependent.
So, we're going to talk a lot more about that today.
Before we leave this, I just want to say one thing about a clicker question.
So, at the end of last time, we had our first repeat winner for a clicker competition.
So, we thought that we might have an opportunity for recitations to see if they can -- if some other recitation can get a second win.
So on Monday we're going to have our final clicker competition, and if somebody else does tie, one section, that we'll have a final one-time clicker question that'll be a run-off.
And whatever recitation wins, we have special prizes for the members of that recitation.
So that's on Monday.
So you may want to review catalysis on Monday, if you feel that you're in the running for the grand champion clicker recitation.
All right, so let's go back to this.
So we found out that no, activation energy is not temperature dependent.
All right, so what are you going to see on your equation sheet on the final, you may see a couple of different forms that are all equivalent to each other.
So here is the plot of a straight line.
It will often be written with just natural log of a and e a over r t terms reversed.
So that's as a straight line, and this is often what you see for the Araneus equation.
You can also get rid of the natural logs and have the term here exponential, so you have k equals factor A times e to the minus, activation energy divided by r t. So, those are the equations, they're all equivalent to each other that you might see on your equation sheet.
So, it's also true that non-gases can exhibit this kind of behavior, and let me just give you one example of a non-gas that exhibits this kind of behavior and you tell me what this is.
What is that?
Crickets.
So crickets exhibit Arrhenius behavior.
They will chirp faster as the temperature gets hotter.
So if you're out camping, sometimes it can be in the summer, it can be actually be quite deafening, and you were very happy to go back to the city where you only have ambulances and cars going, and you don't have this incredible racket at night.
You can actually calculate what the temperature is by counting the number of chirps of the crickets and using a little equation.
I think it's you count for 14 seconds and add 40 and that's the temperature in fahrenheit or something like that.
So, not only gases do this kind of behavior.
All right, so activation energy, we're going to be talking a lot about this today.
So, what is it exactly.
So let's think about two molecules coming together.
So when two molecules come together, you have this bimolecular process going on, but every time two molecules come together, they're not going to go on and form a product.
You're only going to form a product when those molecules have a critical amount of energy.
When they have the energy which allows them to react, that activation energy.
If they have enough energy when they come together, they will go on and form products.
So, that's what activation energy is, it's this critical amount of energy that they need to react with each other.
All right, so let's just think about what affects that critical amount of energy, and of course, temperature is going to be involved.
So let's think about that.
So if we have fraction of molecules on one side, and we have kinetic energy down here, let's think about how temperature is involved in this.
So, at a low temperature, the fraction of molecules that are going to have enough energy to react is going to be less.
And let's think about at a higher temperature, go like this, so this is high temperature, and we have low temperature up here.
And then over here we would have our activation energy, the energy needed for a reaction.
And you see, if you're at low temperature, only a small number of molecules are going to have enough energy to react, but if you're at higher temperature, a large number of molecules are going to have that critical energy to react.
So, at low temperature, not many can react, at higher temperature, many more will have that energy -- will be able to overcome that activation energy, will have it and they can react.
So, temperature plays a big role here.
So we can use this idea of activation energy to predict a rate constant.
Let's look at an example.
So, often people have sucrose in their diet, and the hydrolysis of sucrose will form glucose and fructose as part of this digestive process.
And we can think about the rates at which that would happen in the body.
So normally, we're at 37 degrees, that's normal body temperature, and we have the observed rate constant for that is 1 .
0 times 10 to the minus 3 per molar per second.
But what happens if your body temperature is lowered, what happens if it was at 35 degrees, and to be able to answer that question, you need to know what the activation energy is for this process, and here it's 108 kilojoules per mole.
So, we want to ask the question, what is the new rate constant at 35 degrees?
So, we can take our Arrhenius equations, and we can put a 1 by the rate constant for temperature 1, and a 2 for the rate constant at temperature 2.
Now we can combine these equations.
Natural log of a cancels out, it's not temperature dependent so it doesn't stay in here.
And so we can solve, we can subtract these two, or that's equivalent to dividing them, and so natural log of rate constant 2 over rate constant 1 equals minus the activation energy over the gas constant times this temperature term.
Does this look at all familiar?
Does this look like some other equation you saw once upon a time?
Do you remember what that equation was called?
Van't Hoff equation, right.
And there we were comparing what instead of rate constants?
Equilibrium constants, and instead of e a, what term did we have here?
Delta h, right.
And it's good you remember that because we're going to come back to that at the end of today's class.
So, we can use this equation and plug in the values.
We put in our activation energy, and remember to pay attention to units, because if you're going to cancel your joules with a gas constant, you want to make sure you've converted your kilojoules to joules, and then we can plug everything in and solve for k 2.
So, here k 2 is 7 .
6 times 10 to the minus 4 per molar per second, so it's slower.
And this is one reason why it's really nice to have your body temperature stay the normal temperature.
If you get too cold, your body processes, your enzymes are not functioning, everything is slowed down.
And if you get too hot, that's not good either.
So you really want to maintain it, and so for some of you who come to MIT from warmer climates, let me introduce you to the L.L.
Bean catalog, they sell coats, they sell boots, and all sorts of things.
So, this winter, when you come back for IAP, or for the term, you're well-prepared, and you don't have to prove that it's not good if your body temperature goes below 37 degrees.
All right, let's think about what else this equation tells us, and the other thing it tells us is that if you have a very large activation energy, if this e a is a very, very big number, that means that your rate constants will be very sensitive to temperature.
So if this is really, really big, there's going to be a big change in your rate constants as temperature changes.
And keep that in mind, we're going to come back to that later.
So, what do you think happens to the rate of an enzymatic reaction at liquid nitrogen temperatures?
We looked at going from 37 degrees to 35, liquid nitrogen is pretty cold.
So, not a whole lot happens at liquid nitrogen temperatures as far as enzymes go, and I'll just mention, so it slows way down, that this is a trick that I use in my research.
So we have crystals of enzymes, and we can try to get structures in particular states by taking the crystals that have enzyme in it, and starting a reaction, and then dunking the crystals in liquid nitrogen to kind of stop it at a particular stage, and then you look at what the structure looks like.
So that's one use.
So, we're going to also take a look at other reactions, we don't have any enzymes here, but some other things, and look at what happens when we get things to be very cold.
[EXPERIMENTING] So, Dr. Taylor is pouring out some liquid nitrogen
All right, so what we're going to look at is a reaction that we can see pretty easily here.
Have any of you used glow sticks before, maybe trick or treating or some other point.
So basically, you may or may not know how they work.
There's two compartments in glow sticks that have two different chemicals in them, and they're trade secrets so we can't put them on the board.
But basically, what we have here is a reaction.
A lot of reactions we know give off heat, or they give infrared light.
Here we have a reaction that gives off energy as visible light.
So would you call this an endothermic or an exothermic reaction here?
Yeah, so this is an exothermic reaction.
So just as Professor Drennan was talking about with slowing down enzymatic reactions we can think about if we can slow down this reaction.
So, we're just going to put it in the liquid nitrogen -- so, keep an eye on this, we'll do several controls of different colors here.
So, see if an orange glow stick works the same.
What we're looking to see is if we can slow down this reaction.
What would we see if we slowed it down or even if we stopped it?
Yeah, we're not going to see anymore color.
So are you starting to see color loss in this first reaction here?
So, we'll try green, and yellow.
So, it looks like this first one might have stopped already, you see there's no color anymore here.
What would you expect if this heated back up to room temperature?
Yeah, so hopefully, if you just keep your eye on this, we'll continue on with the lecturing, because I don't know how long it will take for it to warm back up the room temperature.
But keep an eye and we'll see if we get the temperature back high enough to see the glow again.
And since we do have a liquid nitrogen here, it's too hard to resist freezing a flower.
This has nothing to do with kinetics -- we can't even try that connection.
But we will be freeze a flower.
OK, she will try making -- good catharsis pre-exam
So, has anyone had liquid nitrogen applied to them?
It's used in doctor's offices, if you want a little something removed perhaps, put a little liquid nitrogen on and burn it off.
Good premed training for you.
I think we're pretty frozen here.
So it looks the same, but as you can see -- [INAUDIBLE] -- So I think that's all we can do [LAUGHTER] [APPLAUSE]
So I actually heard something interesting on NPR about liquid nitrogen removing warts and things like that, and they were talking about how there was something that was actually better than liquid nitrogen for doing that.
Did anyone hear the story about what the thing was that was better than liquid nitrogen?
It was duct tape.
And so some scientists had looked at what all the uses of duct tape are.
And since most of you are planning on being scientists or engineers, duct tape will probably be an important part of your life in the future.
And so, duct tape worked better than liquid nitrogen for removing warts.
And they found that duct tape worked really well for a lot of things.
There was only one thing they tried that it did not work well for.
Anyone want to guess what that was?
Repairing ducts, yes.
Not really good -- there were many, many, many better ways to repair ducts than with duct tape, but I guess they decided that calling it wart removal tape was just not quite as catchy as duct tape, so it's still referred to as duct tape.
OK, so when you lower the temperature, things tend to slow down.
But if molecules are going to react, they need to have enough energy, they need to have a high enough temperature to overcome this activation energy.
So that critical amount.
So again, when the molecules come together, and let's just look at these for a minute, when they're coming together to react, and if they're going to react, there needs to be some energy associated with this, because you're probably going to have to break a bond, and that's going to take something, and then you may have to form a new bond, for example.
So there needs to be a critical amount of energy, you need to be able to overcome that activation energy barrier for the molecules to react.
So it always takes some energy for things to react and so you need to have enough energy.
And so, if those molecules have that energy, they'll come together, react, and you'll form products.
If they don't have that energy, they're just going to go back to what they were, unchanged reactants.
So you need to have sufficient energy to overcome that activation energy barrier.
So, this is just a little movie that shows two molecules coming together, and if they have enough energy to react, you will see a spark, and then the molecules will react.
So, here we go.
Molecule in red, in green, they're checking each other out.
Do they have enough -- they had enough energy, and they reacted, and went on to product.
All right, so let's talk about this activation energy barrier and these activated complexes.
OK, so in this example, you have n o 2 plus c o, and they can come together and form n o plus c o 2.
And that's going to take some amount of energy to react.
And so, I'm drawing something that's called an activated energy diagram, and we have potential energy on one axis, and on the other we have what's called a reaction coordinate.
And so, the reactants are going to have a certain amount of energy, so our reactants are going to have some amount of energy, and our products will have some amount of energy.
But even though in this case the change, the products are lower in energy, and you have a delta e for the difference between the reactants and the products, they can't go directly to products.
They have to overcome an activation energy barrier.
So they have to overcome some kind of barrier before they can react.
So, only ones that can overcome that barrier, that have enough energy to overcome this activation energy barrier, so the activation energy for the forward reaction, only those will be able to react.
There's also an activation energy barrier for the reverse reaction on this side.
So, if you go from products to reactants, you have to overcome that activation energy barrier.
And up here, this is called the activated complex, so you have some kind of activated complex or transition state, so the molecules will come together, they'll form some kind of transition state up here, and then go down into products.
OK, so most of you are sort of familiar with the concept, I think, of this activation energy, we've been talking about it, but the idea of an activation energy barrier, I think is something that probably all of you can personally connect with.
So, for me, one of the things that I find really hard to do is get started writing a long National Institutes of Health grant.
They're about 25 pages long, they're single spaced, font 11, and they have point 5 margins on every side of the page, and it's really dense, and it takes a long time to sort of get going on that.
And so, there is deadlines, and MIT is very particular, you need to have it to the Office of Sponsored Research five full business days before it's due at the National Institutes of Health, and then the Department needs to sign off on it.
And I'll be looking at my calendar and I have those dates marked, and checking how many days I have left, and eventually, just like getting started, it's like oh, there's so much to do, I have to read the literature, the new stuff that's come out on my topic.
And I have to think about what projects I'm going to do in the future, and I have to write about the progress that I've made so far, which is not really what I want it to be.
And so, I think a lot about how -- it's just overwhelming.
But then eventually something happens.
Either it's tremendous fear that there's so few days left and you just have to do it.
Sometimes it's going and getting an enormous cup of coffee and sitting down.
You know, people have been known to sort of chain themselves to their desk, like they're not going to get to get up until they've written the introduction to the grant.
So, a lot you can connect with this.
That any new thing you start, there's some barrier that you have to overcome to get started with it.
And often once you're started it's not that bad, and some of you may be thinking, exam 1 was a long time ago, I recall there was a lot of material on exam 1.
And it seems really scary.
But then the fact that I mention the final exam over and over in class is helping you get that energy that you need to overcome that activation energy barrier and start studying, because once you start studying you go, Oh yeah, I remember this, this wasn't so bad.
So you just need to get over that activation energy barrier and you're all set.
So, molecules have to do the same thing, and the ones that have higher temperature have an easier time getting over that barrier.
So, here we can talk about this general process.
We can look at the individual numbers involved.
So in this particular case, there's an activation energy for the forward reaction of 132 kilojoules per mole, and there is an activation energy for the reverse reaction of 358 kilojoules per mole.
And there's also a delta e for the reaction, which is this line, from reactions to products, which in this case is minus 226 kilojoules per mole.
So do you think this reaction is endothermic or exothermic?
What do you think, exothermic or endothermic?
It's exothermic, and if you look back in your notes, we talked a little bit about the relationship between delta h and delta e. And they're actually pretty similar.
A delta h usually equals delta e plus a change in p v. So, for gases it's about 1% or 2% difference, and for solids, there's really negligible difference between delta e and delta h. So, they're pretty similar types of values.
So, we can we can think about what delta e really is, and so delta e in terms of activation energy is going to be equal to the activation energy for the forward reaction minus the activation energy for the reverse reaction.
And in this particular case, we have 226 kilojoules per mole is our delta e, and for our forward reaction, we have 132 kilojoules per mole, and for the reverse reaction, the activation energy for the reverse reaction is 358 kilojoules per mole, and so these should all equal up to each other.
And so, if you know two of these values, you can calculate the third.
And this is one of the equations that you have to memorize for the final, because it has, its sort of a conceptual thing that you need to understand what this diagram says, that this plus that is equal to that, that these all add up to each other.
And if you have a negative value here, that means it's an exothermic reaction.
So, this delta e is a change in internal energy of the system, and you can determine that value experimentally, say, with a calorimetry experiment.
OK, so let's keep this in mind and go on and take a look at how this connects back with some other things that we have already talked about in this course.
So, for an elementary reaction, and I think for all of us, there's always some activation energy barrier to overcome.
There's always some positive activation energy to overcome.
And because there's always this activation energy to overcome, increasing the temperature is always going to increase the rate of an elementary reaction.
It's always going to make it easier to get over that activation energy barrier.
But for an overall reaction, increasing the temperature may not increase the rate of the reaction.
So let's consider why that would be true.
So, here is a reaction that we've talked about before, we talked about this proposed mechanism where we have a fast reversible step and a slow second step.
So we learned last time that we can write the rate of product formation from the second step, there are two molecules of n o 2 being formed, so we have 2 times k 2, the concentration of n 2 o 2, and the concentration of o 2.
But this is an intermediate, so we need to solve for that intermediate.
So, in this case, we have a fast reversible first reaction and a slow second reaction.
So this intermediate is going to build up, and it's going to be more or less an equilibrium with the reactants, because this is very fast, and only a little bit of this is siphoned off to make product, and so this creates an equilibrium type situation.
So why don't you solve this intermediate for me, this is a review from the last class.
OK, let's just take 10 more seconds.
Very good.
So, we can solve for this, equilibrium constant for the first step, products over reactants, and then if you solved for this, the intermediate here, which is the product in the first step, then it would be equal to k 1 times n o squared.
We can take that term and we can we can plug it in, so over here, we can substitute it into this equation, and so we have 2 k 2, big K 1 times n o squared times o 2.
All right, so here is our rate then.
And if you missed some of this, this was in the notes from before.
So now let's think about the effect of temperature.
So, k 2 is an elementary rate constant, and so its temperature -- if you increase the temperature, its rate will increase.
So here again is our equation, the activation energy is always positive, there's always positive, there's always some barrier to overcome.
So if you increase the temperature, you're always going to increase the elementary rate.
Well, what about equilibrium constant?
So we've talked about this back in chemical equilibrium that the effect of temperature on the equilibrium constant depends on whether the reaction is endothermic or exothermic.
And you told me before, the equation, and that's the Van't Hoff equation shown here.
And so look how similar those equations are.
So for an elementary rate constant, we had e a, and for equilibrium constant, we're talking about delta h. So, if you have, here the reaction is exothermic, and if you increase the temperature of an exothermic reaction, what happens to k?
It decreases.
So, again, it would shift, then, to the endothermic direction, decreasing k. So let's look at this then.
So we have in this k obs term, we have an elementary rate constant and an equilibrium constant, so if you if you increase the temperature, the rate constant increases, but the equilibrium constant is going to decrease.
So, the magnitude of the increase or decrease depends on the size of the activation energy or the size of delta h. So for this particular example, there's no way that you would know this so I'm telling you, that the activation energy is a small number, or you might be able to look it up in your book, and delta h is a very big number and it's negative, it's an exothermic reaction.
So if you have a very small number for e a, that means that the rate constant will increase only a little bit, it's not that sensitive to a change in temperature, because e a's a very small number.
But if delta h is a big number, then the equilibrium constant would decrease a lot with temperature because this is a big number here.
So in this particular example, increasing the temperature actually decreases the observed rate, because delta h is, in this particular example, a much bigger thing.
So if you were given either numbers or some information like that, you should be able to rationalize what might be true about the rate of the reaction.
so, a large activation energy means that the rate constant is very sensitive to changes in temperature.
A large delta h means equilibrium constant is very sensitive to changes in temperature.
And, as we've talked about, e a is always positive, so the elementary rates always increase with temperature, whereas delta h can be positive or negative, so equilibrium constants can increase or decrease with temperature.
And here, the magnitude of delta h indicates the magnitude of the change, how much k will change, will it be a big change or a small change, whereas the sign of delta h indicates the direction of the change.
So, just want to review one thing and then we'll stop for the day.
So when a stress is applied to a system, an equilibrium, the system tends to try to minimize that stress.
So we're back to LeChatelier's principle.
And so, just one more clicker question and we'll stop for the day.
So increasing the temperature is going to do what again?
Again, thinking back to LeChatelier.
And just 10 more seconds.
Very good.
So, we're going to finish up these notes on Monday.
We're going to think about LeChatelier in a new way, we're going to think about it in terms of activation energy, which is really fun, because we tie back what we learned in the middle of course to what we're seeing now in the end of the course.
All right, have a great weekend, everybody.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today is our final clicker competition.
The first question is up.
And I would just like to go over the current standing for the clicker competition as we're getting started here.
So, recitation 6 has two wins, and there are other recitations and have one win, so recitation 2, 3, 5, 7, and 10 have one win.
If any of those recitation wins today, we're going to go to a tie breaker to decide the overall recitation champion.
There are also recitations 1, 4, 9, and 11, who are looking for their first win.
This is their last chance today.
So, if they win today, they can not be overall champions, but they will have snacks, and they will assure that recitations 2, 3, 5, 7, and 10 do not win the overall competition.
So, you have a chance to foil those recitations.
And if we need to go to a tie breaker, it will not be from current material, it's going to be past material or past things that were mentioned.
So, just keep that in mind.
All right, and for the overall winner, I just would like to show you what you will win, so members of the recitation will each get their own chemistry t-shirt, and at some schools, of course, you would look at this and say oh, they're on the varsity team, but at MIT, it means that you are associated with course 5.
So, Department of Chemistry at Massachusetts Institute of Technology on the back, and we have different colors and different sizes.
All right, so time to get started with the clicker competition.
Go ahead and click in your first responses.
OK, let's just take 10 more seconds.
All right.
Big improvement from the first time we asked this question last class where it's about 33%, now we're up to 89.
And no, it is not temperature dependent.
And we're going to be talking more about activation energy today.
All right, so today we're going to finish up kinetics, and we're going to start with the material that we didn't quite get to at the end of last class.
And I love this part, because I love it when we can think about what we learned before in a slightly different way.
So, when I first started lecturing, I was talking about LeChatelier principle and, of course, encouraged students to sort of -- you know it's not too intuitive to MIT students, but LeChatelier's principle says that if you apply stress to a system, it tends to respond in such a way to minimize that stress.
So it's all about minimizing stress, which is something that's hard to kind to imagine this time of year, a little easier to imagine activation energy barriers, a little hard to imagine minimizing stress.
But we're back to talking about minimizing stress.
And we talked about adding reagents and removing products, and we talked about how reactions will shift to respond to stresses.
And one of the things we talked about is rationalizing the shift of reactions at equilibrium when temperature has changed.
So, if you increase the temperature of a particular reaction, it tends to shift in the endothermic direction to remove that added heat from the reaction.
So, that's what we learned before, but now we're going to think about this in a slightly different way.
So we're going to think about this in terms of these reaction coordinate diagrams.
So, on one axis we have p e plotted -- what does that stand for?
Potential energy, right, so we have potential energy going up on both sides.
And here we have a view of an endothermic reaction, and over here is an exothermic reaction.
So, let's just take a look at this.
For the endothermic reaction, we have reactants and products, we have a delta e between those, and we have a very large activation energy barrier for the forward direction.
And, of course, for anything to react, there always has to be some barrier, you need some amount of critical energy, because as the molecules come close to each other, and if they're going to react and form some kind of product, you have to sort of distort bonds and sort of rearrange things, and that takes a certain amount of energy.
So if they come together and they have that energy, they'll react and go on to products, if they don't they'll break apart and go back and be reactants.
And how much temperature the system has, is more likely that they'll be able to react.
So it's more likely, if there's high temperature, they could overcome this activation energy barrier.
All right, so in an endothermic reaction, you have a big one in the forward direction, and a much smaller activation energy in the reverse direction.
For exothermic reaction it's the reverse -- there's a smaller barrier for the forward direction, and a much larger barrier for the reverse direction.
So, we can think about the sign of delta e. So delta e, this difference here which is related to delta h for liquids and solids, it's pretty much the same for gases is 1% or 2% different.
So you have a positive value here, and you can get a positive delta e here if you have a big number for the forward activation energy barrier minus a small number for the reverse, so that's going to give you a positive number.
For an exothermic reaction, delta e's going to be negative, and you get a negative number if you take a small number here for the forward activation energy, and subtract it from a larger number here for the reverse activation energy.
And remember, this is one of the equations that we're not going to give you, you need to have that memorized because it's part of understanding what's going on in these diagrams.
So now let's think about what increasing the temperature is going to do.
So, it will help the molecules if the is temperatures increase, they'll have enough energy to overcome barriers.
Now if the barriers are pretty small, then it's not very hard for the molecules to get over them, regardless of the temperature.
But when the barrier is big, then temperature is going to make a big difference.
And you can sort of think about this in your own life as you have a small assignment to do, you just jump in and do it and get it done and it's fine.
But if it's a very, very large thing that you have to do, you often need to procrastinate longer before you start doing it.
So, a small barrier's not so much of a problem, but if it's a big barrier, you need to increase that temperature to help the molecules get over the bigger barriers.
So, let's think about what this means in terms of the direction that a reaction might shift.
So, if you increase a temperature over here with an endothermic reaction, it makes it easier to overcome the bigger barrier.
The forward reaction barrier is the bigger one, so increasing the temperature will help molecules get over that barrier.
They're already doing fine with the reverse barrier, it's the forward barrier that we're having trouble with.
So, if you increase the temperature, that's going to shift the equilibrium toward products, or it will shift the reaction in the endothermic direction.
That's what we had learned before with LeChatelier's principle, but now there's a new way of thinking about why that is true.
So, for an exothermic reaction, if you increase the temperature, that helps with the bigger barrier.
Again, here the bigger barrier is for the reverse reaction.
So, if you increase the temperature of an exothermic reaction, it's going to shift it toward reactants.
More molecules can overcome that reverse barrier, and so you'll form more reactants, it'll shift toward reactants or it'll shift in the endothermic direction.
Again, these are the same things that we already knew from LeChatelier, but it's a different way of thinking about why those things are true.
So, a large activation energy barrier means that rate constants are very sensitive to temperature.
And so, if you have a big activation energy barrier, increasing the temperature makes a big difference -- if it's a big barrier, then there's going to have a big difference if you have higher temperature.
If it's a small barrier, then increasing the temperature doesn't make much of a difference.
And so you can think about that in terms of these diagrams as well.
So, now we're done with temperature and we're going to talk about catalysis and the use of catalysts.
And so, if you remember there were factors affecting the reaction, and we've talked about everything on this list except for catalysts.
So today we're going to talk about catalysts.
So, a catalyst is a substance that speeds up a reaction, but it doesn't get consumed in the reaction, it doesn't undergo any permanent change itself.
It's just added to speed up the reaction.
So catalysts do not appear in the overall balanced equation.
So let's look at what a catalyst is going to do, and we're going to look at in terms of these potential energy diagrams as well.
So here we here we have a reaction, we're going from reactants down here to products.
We have the delta e for the reaction, we have a forward activation energy barrier and a reverse one.
So, up here, at the top, that's our barrier without a catalyst.
This is the transition state or the activated complex up here, so you have to overcome the barrier, you form some kind of activated complex, which then, if there's enough energy to overcome that barrier, goes on to products.
So when you add a catalyst what happens is it lowers this barrier.
So, in the blue line here is the new activation energy barrier.
This is the barrier with a catalyst, and so this would then be the new transition state with a catalyst.
So when it lowers the barrier, it's going to lower the barrier for the forward reaction, so we have a new activation energy for the forward reaction, and we're going to have a new activation energy for the reverse direction.
So catalysts work by reducing both the forward and the reverse activation energy barrier.
And another way that you can say this is that they stabilize the activated complex or the transition state.
So here you have an activated complex with a much higher potential energy without the catalysts, and by stabilizing, you lower in energy that transition state.
So that's another way of expressing what a catalyst does.
So, catalysts have no effect on the thermodynamics of the reaction, they affect the kinetics of the reaction, but they don't affect the thermodynamics of the reaction.
And so, of course, when you think of thermodynamics, you often think of delta g. So, why don't you tell me what you think, what would happen about, so delta g is a state function, it's independent of path, and so therefore, what can you tell me about the equilibrium constant in the presence of a catalyst?
OK, let's just take 10 more seconds.
Yup, very good.
So, the equilibrium constant is not changed.
The thermodynamics, which includes delta g and the equilibrium constant are not affected, the rates of the reaction are affected.
All right, so opposite of a catalyst is an inhabitor, and we'll talk more about this at the end of class, so it would -- inhibitor is slow or sometimes stop the rate of the reaction, and one way that they do this is by increasing the activation energy.
So let's consider types of catalysts now.
You can have a homogeneous catalyst, which is in the same phase as the reaction that it's catalyzing as the reactants.
An example of this is depletion of the ozone layer by chlorofluorocarbons.
And so, this was one of the big environmental challenges that the U.S. has faced, whether it should ban these chlorofluorocarbons, and there was a lot of debate on what the data really was about what how much they affected the ozone layer.
And I guess that debate is somewhat still going on, although I think most people now recognize that this is a serious problem and that legislation is really needed to help correct it.
So that's an example.
In this case they're all gases, so that's homogeneous catalyst, not a happy one.
You could also have heterogeneous catalysts, which are a different phase.
And here's another example that has to do with the environment.
So a catalytic converter is an example of a heterogeneous catalyst.
So here you have a solid metal surface, you can have palladium or platinum that will catalyze reactions with gases, and so they catalyze oxidation of hydrocarbons, carbon monoxide, also reduction of nitrogen oxide.
So this is all to reduce pollution.
So that's an example of a heterogeneous catalyst.
And let me just show you a little movie of how that might work.
So in this movie, in grey here we have a metal surface, and this metal surface can break the hydrogen bond of h 2.
And so here, it is already broken, the h 2 bond, so there's a little hydrogen there and hydrogen there, and so that activates the hydrogen to react.
And so, then we have ethene molecule come in, and oh, there goes the hydrogen, and oh, there goes the hydrogen, oh, there goes the other one and you can form ethane.
So it speeds up the reaction by breaking the h 2 bond so it's more ready to react.
That would be an example of a heterogeneous catalyst there.
All right, and, of course, you may all have guessed that my favorite kind of catalysts are enzymes.
They are the catalyst of life.
And so, enzymes are made up of protein, or mostly protein molecules -- you can have an enzyme that's actually made of RNA, but most are protein molecules.
And they're typically about 20,000 grams per mole or more, and they're capable of carrying out specific reactions.
And so, they're made up of amino acids, and just to take a quick look at that, an amino acid has an amino group and it has a carboxyl group, and it has what's called an alpha carbon that has a side chain on it, which is abbreviated r, and there are 20 different types of r. The simplest is just a hydrogen, you could also have a hydroxide, etc.
And so, this makes up the alphabet of proteins, there are 20 different ones of these that get connected via a peptide bond.
So at the end of this amino acid you'd stick on the amino group of another amino acid and form this peptide bond.
And then you put together hundreds and hundreds of these to form an enzyme complex.
And so, the long chain of amino acids folds up and forms a compact structure.
So, in this particular picture, there are about 200 amino acids in each of these colored units, so this is a tetramer, so there are four -- there's green, red, yellow and blue.
And so, this is what's called a ribbon diagram.
So, these ribbons, this is an alpha helix -- the beta strands are long.
It traces out the position of the alpha carbons.
And overall, this structure is about 90 angstroms by 70 angstroms by 50 angstroms.
Of course, 1 angstroms is 1 times 10 the minus 10 meters.
So these are fairly small.
These protein molecules are, of course, in your body.
And this particular one is an enzyme that makes an antibiotic, and that antibiotic is fosfomycin, shown here.
And so, fosfomycin is used in antibiotic combination therapies to treat staph infections and other kinds of very difficult infections to treat.
And so, I always talk about the things that I'm most concerned about.
There are a lot of things that are big threats that I don't worry too much about.
Antibiotic resistance is something I actually worry a lot about.
I guess I go to too many meetings were they talk about problems.
But the rate at which different kinds of bacteria are becoming resistant to antibiotics seems to be increasing.
So it used to take a lot longer before a resistance would appear than it does now.
And there really haven't been, I think since about 1980s or so, really new antibiotics.
So, we're still using the same antibiotics that we have been, which causes more things to become immune or resistant to those antibiotics, and I think this is really dangerous.
And there's not enough money in it for the pharmaceutical industry to really go after this.
And it's not just a problem for biological warfare, although that is a possibility, but also in hospitals, you go in for some kind of surgery, you have to worry about the fact that even if the surgery goes great, you might get an infection that could really compromise your health in the hospital.
And so, whether you have a heart problem, if you have cancer, you become immune compromised, there's a lot, a lot of cases where people end up not dying of cancer directly, they end up dying of the bacterial infection that can't be treated.
So this is really a huge problem.
So I always like to let you know all the possible problems that you can solve in your future.
This is one that I think is particularly important, and I hope some of you will focus on this, because we really need different kinds of antibiotics, or we need to change the way we do medicine in this country so that resistance doesn't become as much of a problem as it has been.
All right, so that enzyme made an antibiotic, so we like it, and let's talk about how that works.
How that particular series of amino acids folds up to do that reaction.
So now we have some new nomenclature, and sometimes when people get into the biochemistry world they get scared because there are all these words that they don't know what they mean.
So most of them are not -- they are things that you can relate back to something you already know.
So, we've been talking about reactants, and when you have a reactant with an enzyme, it's usually called a substrate.
So this is just another name for a reactant molecule.
And then substrates tend to bind in what we call the active site of the enzyme, which is the part of the enzyme that's going to do the chemistry.
So those are two terms that you'll hear about in biochemistry.
All right, so we have an enzyme, we'll call it e, it binds a substrate, we'll call that s, and then it forms an enzyme substrate complex, which we'll call an e s complex, for enzyme substrate.
And then that enzyme substrate complex will go to enzyme and product, which we'll call p. And if you look in the state-of-the-art biochemistry books, these are the abbreviations you'll see in there as well.
So, this pains me greatly to show this cartoon, because I spend my career determining 3-dimensional structures of proteins, and so to describe it as a little squiggly is painful for me.
But nonetheless, there you go.
Also, the substrate is almost about as big as the enzyme.
That is also usually not true.
But nonetheless, here's a little cartoon.
Here's our substrate binding to our enzyme, it's forming an e s complex.
And then the enzyme is going to move the ears of the substrate around to form product, and now product is released.
So, this is actually pretty simple in terms of writing a mechanism.
And so, what we can do is write a mechanism the way that we have been writing a mechanism so far in this class.
And so we're going to go through this, we're going to derive expression mostly to show you that all the things you've been learning are related to chemistry, they're also related to biochemistry.
So, you already know a lot of biochemistry from just studying freshman chemistry in 511-1.
All right, so we have 2 steps in our mechanism.
We have enzyme binding substrate to form an intermediate enzyme substrate complex, which then goes on to form enzyme plus product in step 2.
So the first step is reversible, the second step is not as drawn.
All right, so now we can come up with the rates for each of the individual steps in this overall mechanism, and since they are elementary reactions or steps in an overall mechanism, we can write the rate laws directly from what we observe.
So for the forward direction, we're going to have what rate constant?
K 1 times the concentration of what?
And?
Yeah.
So there we go.
We have k 1 times the concentration of e and the concentration of s. See, you already knew how to do biochemistry.
All right, so now at the rate of the reverse direction is -- what's our rate constant?
K minus 1 times e s. And then for step 2, we have what rate constant?
K 2 times e s. All right, so now we can talk about the rate formation of product.
And so you'll see this expressed as d concentration of p d t, so the change in product that's being formed and that's going to be equal to the second step here, k 2 times e s. But, as has been the case before, e s is an intermediate, and we need to solve our rate laws, our rate expressions, in terms of reactants or products and rate constants.
So we need to solve for the intermediate.
All right, so let's think about solving for the intermediate.
And why don't you tell me how to do that.
OK, let's just take 10 more seconds.
Very good.
All right, so let's just take a look at why that is the case.
So, when we solve for this, it's going to be equal to the formation of the intermediate, which happens in the first step with this rate that you told me, k 1 times the concentration of e s. And then we have the decomposition in the reverse of step one, so that's minus k minus 1 e s. And then we also have the consumption of the intermediate, which is -- so we have minus this step, which is k 2 times e s. So again, formation minus decomposition minus consumption.
And now, we can use the steady state approximation.
So the steady state approximation applies in biochemistry just as well as in all of your chemistry problems that you've done in this course.
And so, the steady state approximation allows us to set that whole term equal to zero.
So it says that the net rate of intermediate formation and decomposition and consumption is 0, or you can express it as the rate of formation of the intermediate equals the rates of decomposition and consumption -- those are equivalent.
So we can set all of this equal to zero.
So again, the same as what we've done before.
All right, so now we're going to have a slight change of what we've done before, and this has to do with the practical consideration.
So you can solve for these in terms of reactants and products and rate constants, but it's actually easier to solve for e s in terms of your total enzyme rather than your free enzyme.
So what we've had before is our free enzyme, but we often don't know how much enzyme is free, and by that we mean not bound to the substrate.
And so, we know how much total enzyme we put in, but we might not know how much of that is unbound.
So it's actually easier for the experimentalist to solve for things in terms of total enzyme.
And so what we're going to do then is we're going to replace our free enzyme term with the following, total enzyme minus bound, and total enzyme minus the bound enzyme is the free enzyme.
So we're going to do this change that makes it easier for the experimentalist.
So here's the term we had before, but now we want to replace this term of e with e to 0 minus e s. So we're going to put that, here's the term e up here, so we're going to replace that with e 0 minus e s. And then we're going to multiply that out, so we get k 1, e 0, s minus k 1, e s, and then the concentration of substrate here, and the rest of it is the same.
So we got rid of our term e. So now we can solve for e s. So this is what we just had, and so we're going to rearrange so the e s terms are on one side, so we have an e s term here, here, and here.
So all that is going to be on one side, and then this other term will be on the other side of the equation.
So now we have moved all the e s terms over here, so we've got rid of the negative numbers, we've added them all to the other side, and then on this side we just have this k 1 term left.
Now we're going to pull out the e s terms.
So, we'll solve for e s. We'll pull that out, that leaves us with k 1 times the concentration of s, k minus 1, and then k 2.
And now we're going to take this and divide by that term in parentheses, and so now we've solved for e s in terms of total enzyme substrate and our rate constants.
All right, now we're going to do another thing that's a bit different.
We're going to introduce a term in biochemistry, which is k m, also known as the Michaelis-Menten constant, and so k m is equal to the rate constant for the forward/reverse direction, k minus 1 plus k 2 over k 1.
So we're going to now try to use this in our expression.
So we're going to put this term in there, and so to have this term of k m in there, we need to have this divided by k 1.
So we can do that, it's OK, as long as we divide everything by k 1, so we're going to divide the top by k 1, and this by k 1, and that by k 1.
So now, we divide all by k 1, and so we have this top with k 1, this term with k 1, and this with k 1, and that's our k m value.
But now we can simplify this, we've got a lot of k 1's here, and so if we simplify this, we can cross out those k 1's over there.
And we can also get rid of these k 1's over here.
And that's going to leave us with a much simpler expression.
So that's going to leave us with the total enzyme times substrate over substrate plus k m. And that looks a whole lot better than that other term that we had before.
OK, so we can now take this and substitute it back into the expression we had earlier.
So here we had the rate of product formation, k 2 times this intermediate, we solved for the intermediate using total enzyme and using this k m term.
And now we just put all of this times k 2.
And if you put all of that times k 2, you have this expression, which is the Michaelis-Menten equation.
So the only difference of what you did before was that you solved in terms of total enzyme, and also, we have this new k m term.
And so now we have an equation that's used in a lot of biochemistry research to think about enzyme kinetics.
So let's think about enzyme kinetics for a minute.
This is what a plot often looks like -- we are looking at the product formed by an enzyme versus the concentration of substrate.
And it often goes up.
It's pretty steep, and then it starts to level off.
So let's think about what's happening here.
So if you have low substrate, so that's way down here, not much substrate is available, adding more substrate will increase this rate significantly, you'll form a lot more product, it increases the rate of product formation.
Because a lot of the enzyme is free and can react with substrate.
So it goes up, but then the rate starts to level off.
So when you got to higher substrate concentrations, adding more substrate does not affect the rate much, the rate is leveling off.
And we say that the enzyme is saturated.
All the enzyme that you have in there already has substrate going on, it's already doing the chemistry.
So if you're throwing more at it, the enzyme can't bind that substrate, it's busy with a different substrate molecule.
So all the active sites in the enzyme are full, and so not much happens.
So this kind of graph is what you often see in enzyme kinetics.
Now let's think about that in terms of our Michaelis-Menten equation.
So let's think about conditions when there's a lot of substrate, when substrate is much greater than k m. So here we have substrate and k m on the bottom of our equation, and what's going to happen if this term is much, much bigger than this k m term?
What happens to the k m term?
It goes away.
It's really small compared to this, it's kind of insignificant.
So, if that happens, then you can cancel substrate as well.
And so, that means that under conditions where substrate is much greater than k m, the rate is a much simpler term, it's just equal to k 2 times your total enzyme.
And so this has a name, this is called Vmax or the maximum velocity of the reaction.
So, under these conditions, when the substrate is much greater than that k m value, this expression simplifies and you get just k 2, your rate constant, times your total enzyme, and that'll give you the maximum velocity of the reaction.
And so, that's just written again here.
Vmax equals k 2 times total enzyme, and that equation will be given to you on an equation sheet.
So what's happening?
What's happening is that you're up here.
So when substrate concentration is much greater than k m, then you're in these conditions, and you're at some maximum velocity.
So the velocity will only go, the rate of the reaction will only go so fast, it will saturate, it'll level off up here, and so you can calculate that maximum rate of that reaction.
Now let's consider what happens when your substrate concentration equals your k m. So down here, if k m is equal to substrate, then we're going to have two substrates down here.
And so, we can write this, if they're equal, we can write this as substrate plus substrate, and then we can do some canceling.
So this will be equal to 2 substrates and then we can cancel that out, and so we get the rate of product formation under conditions where substrate equals k m of 1/2 k 2 times the total enzyme concentration.
And this is referred to the half maximal rate.
Remember, the maximal rate was k 2 times this e knot, and so half of that is half the maximal rate here.
So, the definition of k m is the concentration of substrate for which the rate is half maximal.
So if your rate is half of the maximum rate, that substrate concentration that gives you half the maximum rate, is the value for k m. Substrate concentration equals k m at the half maximal rate.
So let's just look at what that meant.
So if this is Vmax up here, half of Vmax is here, half of that value, and the substrate concentration at half Vmax is equal to k m. So you can write those in on your diagram.
Vmax is this velocity up here, half of that, when you're at half of the maximal rate, k m equals the substrate concentration.
And there will be problems that you will work in your book very shortly, and often, the problems just have to do, they'll give you the information in words and the calculations can often be pretty simple.
Sometimes it'll say, figure out what the k m for this reaction is.
At this substrate concentration, the rate is half maximal.
Well, at that substrate concentration, that's the k m. So, a lot of the problems are sort of word problems, they give you the information and you need to know what the definitions are.
But let's try an example.
So here in this example, we've talked about buffering in the blood, so the conversion of your ingredients that make your buffering agent in the blood, that can be catalyzed by an enzyme called carbonicanhydrase, and the following Michaelis-Menten constants are known, a k m is known, and a k 2 is known, and if you're doing experiment and you have a total concentration of your enzyme of 5 times 10 to the minus 6 molar, then you should be able to figure out what the maximum rate of the reaction would be.
So, why don't you go ahead and tell me what the maximum rate would be.
OK, let's just take 10 more seconds.
Very good.
Everyone's doing very good today.
All right, so all you have to do, you have remember Vmax equals your total enzyme concentration times your k 2.
And pretty much no one, hardly anyone was fooled by k m value there.
All right, very good.
So there are extra problems on enzyme kinetics posted on the extra problems posted on the Web.
So there's a few there, so you can see the type of problems, again, they're like that, they're pretty simple.
If you remember the definition of k m, and you'll be given the equation for Vmax, you should be able to handle the enzyme/kinetics problems pretty well.
Often, also I'll ask questions like explain why at high substrate concentrations, the rate does not increase very much, or why the graph levels off, or I might ask you to draw the plot and tell me about low substrate concentrations, what's true there, and high substrate concentrations, what's true.
So those are the types of questions you're going to get on enzyme kinetics, they're actually pretty simple.
All right, so let's just talk briefly about enzyme inhibition.
So the opposite of catalysis -- instead of binding a substrate, you'll bind an inhibitor.
So often, these are actually pretty simple.
An inhibitor sometimes will look a lot like a substrate or like a transition complex, and it'll blind to the enzyme.
And when you have that inhibitor stuck in the active site, substrate physically can't bind, so it occupies the place that substrate goes, substrate will come in but it can't bind anywhere.
So this is actually the mechanism by which a number of pharmaceuticals work.
So, one thing that people who were designing enzyme inhibitors think about is the fact that enzymes do tend to stabilize a transition state in the reaction.
So if you make an inhibitor that resembles a transition state, it should bind to the enzyme more tightly than either the reactants, the substrates, or the products.
So, a lot of the pharmaceutical industry likes to try to figure out what the transition state might look like, and then try to make a molecule that looks like that transition state that hopefully will blind to the enzyme active site and prevent the enzyme from doing what it's supposed to do.
So, just to kind of look back at this figure for a minute that you had earlier in your notes, so you have the transition state is often stabilized by a catalyst, and so an enzyme will also stabilize a transition state.
So if you make your drug look like a transition state, it will hopefully bind very tightly.
And so this is one of the sort of principles that's behind a lot of the pharmaceuticals that have made to treat HIV infections.
So I mentioned last week we had world AIDS day, that understanding kinetics was actually very important in HIV research.
And so, many of the pharmaceuticals that are given to HIV patients are what are called protease inhibitors.
So they inhibit enzymes that are called proteases, and if you have ase at the end of the name that means it's an enzyme, and so protease means that it's an enzyme that cleaves proteins.
It's a protein ase.
And so, how do these enzymes work.
Well, they cleave other proteins, they cleave peptide bonds, and so you often have some kind of either activated water or other hydroxide molecule that will attack here the carbonyl of a peptide bond, and it forms a tetrahedral intermediate, which then collapses and it cleaves that peptide bond.
So the peptide bond is then broken, so that's what a protease does, and a protease inhibitor prevents that cleavage.
So often, protease inhibitors look like tetrahedral intermediates, and so they'll bind to the protease active site and prevent the chemistry from occurring.
So if it's a tetrahedral intermediate, what kind of angles should some pharmaceutical company be looking for in its compounds?
109 .
5, yes.
So, molecules with this stable tetrahedral intermediate somewhere on the molecule could bind to the active site and prevent a catalysis.
So, let me just show you one example of an improved HIV drug, and there it is.
This is the tetrahedral site that binds at the active site of that enzyme, and the enzyme can't cleave this tetrahedral intermediate, and so this just binds, but the enzyme can't work on it, so it just sits there and prevents substrate from binding.
So that's how a lot of these compounds work.
And so, this is just a little picture of the enzyme active site and there's an inhibitor bound to the enzyme active site, and so a number of companies are working on trying to come up with better and better inhibitors that will sit in the HIV protease active site.
So, knowledgeable of a reaction mechanism can lead to new therapeutic treatments.
So, one question with this, with protease, there are a lot of proteases, not just the ones involved with HIV, and so one real problem is specificity and toxicity, so you want to have a drug that inhibits one enzyme and not all the enzymes in that category.
So that's a big problem in the pharmaceutical industry.
So, apparently, clicker results, we actually need to break a tie.
But we can't do it now.
OK, so Justin's section, woah!
So, apparently we're having, we have the questions ready, but apparently they're not ready to be used right at the moment.
So, we're going to use low tech here and indicate the results for today.
And then on Wednesday, last day of class -- oh, my goodness, so close.
Recitation 6, you were second.
All right, so on Wednesday then, we're going to have the tie breaker, and review and evaluations.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so let's do our first clicker question.
All right, let's just do 10 more seconds.
Interesting.
So, in this problem you were given the maximum rate, Vmax, which, by the way, you calculated in class last time.
And then said what will the substrate concentration be when you have half of that rate.
And so, when you have half of the maximum rate, that's the definition of k m. So all you have to do is if a k m is given, you just write down that value.
So that's the level of question you'll get on enzyme kinetics.
All right, so that should be a good tie breaker, too.
All right, so in this little review and we talk about the material from the second half of the course, and why I think this material is really important and cool, and that's because it all represents the basic principles of how enzymes work.
And as a biochemist, I recognize this material is really important.
So that's what interests me particularly about it.
And the point that I want to make in general is that when you're studying introductory material, sometimes it seems like you're never going to really need that material again, but that's because you haven't seen the connection between it in other topics.
So, a lot of the things you've learned will be related to things that you're going to see you later on.
So I'm going to give you examples and review examples of how the material you've learned will allow you to understand an enzyme, for example.
So, we talked last time that one reason why you care about understanding how an enzyme works is because a lot of people want to inhibit enzymes.
Well, why would you want to inhibit enzymes, and that's because inhibiting enzymes is very useful to treat a bunch of things.
A number of you will, in the next week, I'm pretty sure, be interested in ways to inhibit enzymes to prevent headaches or to decrease headaches.
And this is true of students and faculty during finals week.
And so, some of the treatments for headaches that a number of those pharmaceuticals are geared toward enzymes called prostaglandin synthesis, and they inhibit those enzyme from working and that gets rid of your headache.
Arthritis, for example, is also treated by a lot of the same medications that treat headaches.
Cancer, often chemotherapy or you're giving people inhibitors of enzymes to decrease tumor growth.
And we talked last time about inhibiting HIV proteases as part of a treatment for HIV.
So, understanding enzymes is very important for the pharmaceutical industry, and so that's one example of why you need to know about chemical equilibrium, why you need to know about acid based oxidation reduction transition metals, and also kinetics.
So, I'm going to give you a case study.
You've seen some of these examples before, but we're going to put them all together now.
And I try in every lecture, if I can, to mention vitamin B12, and here, we're going to tell you about a vitamin B12 dependent enzyme, and how knowing what you know now about these topics would allow you to understand how this enzyme works.
So, first, kinetics, this is easy, methionine synthase, it's an enzyme.
So, what does this enzyme do?
So this particular enzyme converts an amino acid called homocysteine to another amino acid, methionine.
It also converts another vitamin, a B vitamin, the methyltetrahydrofolate form to tetrahydrofolate, so this folate is one your B vitamins.
Why do you care?
Well, methionine is needed for proper formation of brain and spinal column, and it's particularly important when women are pregnant that they have enough folic acid so that they can make enough methionine.
If they do not have enough folic acid, then you can have babies born with neural tube defects.
Also, as I mentioned before, inhibiting methionine synthase raises homocysteine levels, so you can't do this conversion, and that increases risk of heart disease that homocysteine is actually a really good indicator, high homocysteine levels of whether someone's going to have heart trouble or not.
This is also a target for chemotherapy, because you need tetrahydrofolate to do DNA biosynthesis, and tumor cells that are growing, tumors that are growing, need a lot of DNA biosynthesis.
So if you inhibit this enzyme, you can inhibit DNA biosynthesis, so it's a potential chemotherapeutic target.
There is some use of this in Europe.
Actually there is a treatment that's given that's specifically supposed to inhibit methione synthase, it's not used so much in this country, but laughing gas will inhibit methione synthase.
And so, in Europe some cancer patients are treated with laughing gas as a way to decrease their tumors.
I'm not sure it's the most effective therapy, but they're at least pretty happy with it.
So that's the counterpart.
So, this is an important enzyme.
And enzymes, of course, as we know are catalysts and they catalyze reactions, so tell me what you know about catalysts.
Which is the following statements about catalysts are true?
OK, let's take 10 more seconds.
Very good.
All right, so going back to the notes, statement 2 is true.
So, catalysts work by lowering the activation energy barrier for both the forward and the reverse reaction.
So both rates would be changed.
And also, catalysts are not going to affect the thermodynamics, they're not going to shift the equilibrium toward products.
What's something that you could use to shift an equilibrium, say, to products or reactants.
Temperature, yeah.
So, just a little review of kinetics.
Again, a catalyst works by lowering the activation energy barrier, or another way to say that is by stabilizing a transition state, so it's going to affect the activation energy for the forward reaction.
And for the reverse reaction, it's going to decrease both of them by the same amount.
So if you know how much one is decreased, you also know how much the other is decreased.
And we also know that it doesn't affect the thermodynamics.
So, does it change the equilibrium constant?
No, it doesn't change the equilibrium constant.
All right, so a little review of kinetics.
So now let's talk about transition metals.
So this particular enzyme needs 2 different kinds of transition metals.
It needs cobalt in the form of vitamin B12 and it also needs zinc.
So we talked about methylcobalamin, so cobalamin is another name for vitamin B12.
And here we have a methyl ligand, c h 3 is a methyl ligand coordinated to the cobalt in the center, and we saw before that when you have a thing that forms attachments at more than one point, it's called a chelate, so the corn ring here is a chelating agent, and it's a tetradentate ligand because there are 4 points of attachment.
So we can have monodentate, bidentate, tetradentate, hexadentate, and tell you the number of points of attachment.
And again, this is a chelate, and what do you know about a chelate effect?
I want someone to tell me what the chelate effect is, and there's a special bonus prize of a teeshirt for the best answer.
Who wants to give it a try and tell me what the chelate effect is.
Any volunteers?
Bonus teeshirt.
Who wants to give it a try.
Not very brave
[INAUDIBLE]
OK, so chelate, it's something that's going to be binding to a metal at multiple points of attachment.
And what do we know about how stable or how favorable that kind of interaction is?
It makes it more stable because [INAUDIBLE]
Are you reading from your notes?
Well, you know what, I didn't say that was not allowed, so congratulations.
Right, so chelates are unusually stable due to entropic affects.
So it's an entropy affect.
So if you have a metal that's free in solution, it's going to bind to a lot of water molecules, and often, metals like to bind to thick things.
You see a lot of octahedral complexes.
So if you have a metal bound to 6 waters, and you bring in a chelate that will blind with multiple points of attachment, such as EDTA, which binds at 6 points of attachment, that 1 molecule of EDTA will displace 6 water molecules.
So that is going to be entropically favorable.
6 things floating around in solution have a lot more entropy than 1 thing floating around in solution.
And this makes chelates very stable.
And so, if you want to get -- if someone is poisoned with the metal and you want to get that metal out of your system, you give them a chelate such as EDTA, hospitals have EDTA.
It's also good, as we learned, for cleaning bath tubs.
All right, so that's the chelate effect.
All right, so what about zinc.
This enzyme has a zinc and it has zinc in the plus -- zinc plus 2 ion.
So, if you know that, you can calculate a d count.
And so again, we can go to our periodic table.
What group is zinc in?
12.
So what would be the d count?
10.
So 12 minus 2 equals 10.
So what would you think would be true about the color of this d 10 system?
All right, so let's just take 10 more seconds.
All right, so it would be colorless.
And so, all of the d orbitals are going to be full, and so there's no transition from one set of d orbitals to the other, and so it would be a colorless compound.
And that is, in fact, true that it took years and years of studying methionine synthase before people realized that it had a zinc in it, no one knew that there was a zinc, because they couldn't see it.
It didn't add a color, and so no one knew that a metal was there for a long time.
All right, so what about oxidation reduction?
So, this enzyme does a lot of different oxidation reduction reactions.
So, when it reacts with homocysteine to form methionine, and the vitamin B12, which is shown in a cartoon form down here, goes from a plus 3 state that has the methyl associated, it gives the methyl group, the c h 3 group, up to homocysteine to form methionine, and then you're in a plus 1 state of cobalt here.
But this state can lose an electron, and you can have a what's called [?
cobe ?]
2 form of the enzyme, which you need to put another electron in and reduce it to go back, and you also need to methylate it again.
And the methyl group comes from s adenosylmethionine.
So, we need to think about how we're going to reduce the vitamin to get it back in its primary turnover state.
so, we know the redox potentials for this.
The standard reduction potential for vitamin B12 is minus point 5 2 6 volts.
The potential reduction potential for flavodoxin, which is the protein that gives it the 1 electron, and it's a flavin protein, which is another B vitamin.
And that's minus point 2 3 0 volts.
So, what do we know about these?
Tell me which is the better reducing agent.
All right, let's just do 10 more seconds.
Click in your responses.
OK.
So, vitamin B12 is a better reducing agent.
If you have a large negative number, then the reduced species is very reducing.
So it wants to reduce other things and get oxidized itself.
So, that's interesting.
It seems like vitamin B12 should be reducing flavodoxin and not the other way around.
So we can look at how unfavorable this process is, and we can calculate a standard potential for this, and we can use this equation where we have reduction minus oxidation and plug the values in, and so we have a minus point 5 2 6 minus a minus of the floating potential, and that gives us a negative value for delta e. So is this reaction spontaneous?
No, and that's because if you have a negative standard reduction potential, then you're going to have a positive value for delta g, so it will not be spontaneous.
And again, you can calculate the value for delta g, so we have minus n, number of moles of electrons, Faraday's constant times the difference in potential, and it's a 1 electron process, we put in Faraday's constant and the calculated value for delta e, and we get a positive 28 point 6.
And as I mentioned before in class, the way this reaction goes is that at the same time an electron goes in, you also cleave s adenosylmethionine, which gives the methyl group to also return to the catalytic cycle, and that process is very favorable minus 37 point 6 kilojoules per mole.
So it drives the unfavorable reaction.
So in the body you will have a favorable oxidation reductions and you'll have unfavorable ones.
All right, so what do you a cell that requires energy to catalyze a non-spontaneous reaction?
So, that's an electrolitic cell.
And a cell that catalyzes a spontaneous reaction?
Galvanic, right.
So again, with oxidation reduction, it doesn't matter if you're talking about a battery, or if you're talking about a cellular process, all the same equations apply.
All right, what about acid based equilibrium.
So in this particular -- in this catalytic cycle, we are converting homocysteine to methionine, and that chemistry involves acid base.
So what is true about the reaction is that the protonation state of the substrate, homocysteine, matters.
So, you can have a protonated homocysteine, and here's the structure of homocysteine, a deprotonated homocysteine where the sulfur does not have a proton anymore.
So we have an s minus here.
And this protonated form of homocysteine can be converted to methionine where there's a methyl group attached to the sulfur here.
So, we want to know at physiological p h 7 point 4, how much of the homocysteine is deprotonated?
And here, if you're asking how much of something is in a protonated state, how much of something is in a deprotonated state, what you are asking is what is the p k a of homocysteine.
So what is the p k a of homocysteine?
That's what you need to know to figure out how much would be in the protonated state, how much would be in the deprotonated state.
So here, the p k a is 10.
So now, without doing any calculations, I want you to tell me what you expect about how much is protonated and how much is deprotonated at physiological p h if the p k a is 10.
All right, let's just take 10 more seconds.
I hope that I mentioned that acid base will be on the final exam.
So good thing we're reviewing it right now.
All right, so now let's look at the math.
So if you're given a p k a and p h and asked about a ratio of protonated to deprotonated it's OK to pull out your favorite Henderson Hasselbalch equation, which gives you a sense of the ratio, can predict the ratio, again it's approximation, but predict the ratio of protonated to deprotonated.
H a, of course, being an abbreviation for protonated, a minus for deprotonated.
And so we can calculate here that if you have a p h of 7 point 4, p k a of 10, you get a ratio of 400:1.
So that's the math, but you can also just think about it in terms of what you know.
So let's just do a brief review.
So the answer here is that free homocysteine is protonated and non-reactive at physiological p h. But let's just think about this question a little more.
So this is not a figure in today's handout, but you've seen this a few times, so let's just look at it for a minute.
So, we talked a lot about acid based titrations.
We talked about so here we have a weak acid tritrated with a strong base.
In the beginning it's a weak acid problem, at the equivalence point it's a weak base problem, because we've added enough moles of our strong acid to convert all of our weak acid to its conjugate base.
And in the middle when you've added half the number of the moles, you've converted half of your weak acid to its conjugate, then the p h is equal to the p k a at this point.
So that would be right in the middle of this buffering region.
Again, in a buffering region you have quite a bit -- you have quite a bit of your weak acid and quite a bit of a conjugate, so it can buffer.
If strong acid is added, then it can be used up, if strong base is added, it can be used up without changing the p h very much.
That's a buffer, so the p h curve is flat in here in that buffering region.
And so, when you think about this, if you're at p h's that are below the p k a, then you're going to be more protonated, and p h is above the p k a, you'll be more deprotonated.
So let's look at the figure now that's in today's handout and think about this.
So when the p h equals the p k a, you will have equal number of moles, of something that's protonated as deprotonated.
And if you're at p h's above the p k a, you'll be more deprotonated.
P h's below the p k a, you'll be more protonated.
And in the particular example I gave, we have a p k a of 10, so at 7 point 4 we're at a p h that is below the p k a, and so here they'd be more protonated.
So if you do the math it's 400:1.
But even if you aren't doing any math, you should think about the fact that that would have more of the protonated form than the deprotonated form.
All right, but the enzyme has a problem, because only the deprotonated form is active.
The p k a of the free homocysteine is 10, and you're at physiological p h, so this reaction doesn't seem like it should go.
But again, it's an enzyme, and enzymes catalyze reactions.
So they do things to make that reaction go faster.
And what this particular enzyme does is it lowers the p k a of homocysteine.
So when homocysteine is bound to the enzyme, it's p k a is no longer 10, but now its p k a is 6.
So, how does the enzyme do this?
Well, that's where the zinc comes in.
So, the zinc binds to the homocysteine and it acts as a Lewis acid as it binds to the homocysteine.
So the homocysteine is your donor ligand, and you have your metal as the Lewis acid, your acceptor.
And that alters the p k a, so it actually associates, zinc associates with this sulfur, and that changes the p k a.
So, I just wanted to mention again, I was having dinner with some of my faculty colleagues who teach organic chemistry here.
We were interviewing another job candidate, and they looked at me and said, they're teaching organic this semester and they said we asked our class about p k a's, and they all insisted that they had never heard about p k a's in their freshman chemistry course.
And I, of course, said, well, they did not take 511-1 then.
So just remember, especially when it's Barbara Imperiali asking, did you hear about p k a's, the answer is?
Yes
Thank you.
OK, so just this week.
So it alters the p k a here.
So now what's our situation?
Well, now at physiological p h, we have a p k a of 6, and so that gives us a very different ratio.
Instead of 400:1, we have 1:25.
So most of the homocysteine is deprotonated at physiological p h, which means that it can react.
So, if we go back to this now, our p k a is now 6, and so physiological p h is now above that p k a.
So when you're above the p k a, you should have more deprotonated than protonated.
So that's how you can rationalize these things.
All right, now I have to ask, all right, we do not need a tie breaker.
So at the end of the lecture, we will announce the winner.
But let's first do chemical equilibrium.
Chemical equilibrium.
So we didn't talk too much about this in chemical equilibrium, but enzymes can have alternate conformation, so the enzyme can change its shape during chemistry.
And that those conformation of the enzyme can be in equilibrium.
So the enzyme itself can be an equilibrium with different conformational states.
So here, there are a lot of states of the enzyme.
It needs to react with homocysteine, it needs to react with methyltetrahydrofolate and with s adenosylmethionine.
And when the structure was determined to this enzyme, here in green is the vitamin B12, and red is the methyl group.
You see that the methyl group is pretty buried, you can barely see it, but yet it needs to interact with homocysteine to give it a methyl group, it needs to take a methyl group off methyltetrahydrofolate, it also needs to take a methyl group off s adenosylmethionine.
But it doesn't seem like there's any room for any of them to actually get in there.
So, this was a sign that there has to be some kind of change in the structure.
So here's another picture of the structure.
Here's the vitamin B12 in green, the methyl group in red, and there's this whole protein part up here that has to get out of the way to do the chemistry.
And, in fact, we do know from experimental data that it does move so you can do the chemistry.
So that means that the enzyme has to have a number of different structures.
It's modular.
Here's the vitamin B12 binding domain, the vitamin B12, there's this region that have to move called the methyl cap.
It needs to interact, the B12 here needs to interact with a folate domain, a homocysteine domain, and an activation domain.
So you need to have at least these 4 different structures, and these 4 structures will be in equilibrium with each other.
So you need to have a structure with folate, a binding domain on top of the red B12, you need to have homocysteine in yellow on top of the B12, you need to have a resting state where those helices are on top of the B12, and you need to have an activation state where the [?
adomet ?]
domain is on top of the B12, and all of these states are going to be in equilibrium with each other, and they're going to be moving, which means that enzymes are dynamic, which means that chemistry is dynamic.
Chemistry in the body is not in the solid state.
Chemistry in the body is in solution.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseware continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseware at ocw.mit.edu
All right, I'm going to start today by talking about, so what have we been doing?
What have we actually done over the last few lectures?
And I want to suggest that what we've done is, we've outlined a lot of the basic elements of programming.
A lot of the basic elements we're going to need to write code.
And I want to just highlight it for you because we're going to come back and look at it.
So I'm going to suggest that we've looked at three different kinds of things.
We've talked about data, we've talked about operations, and we've talked about commands or statements.
All right?
Data's what we expect.
It's our way of representing fundamentally the kinds of information we want to move around.
And here, I'm going to suggest we've seen numbers, we've seen strings, and I'm going to add Booleans here as well.
They're a third kind of value that we saw when we started talking about conditions.
We saw, associated with that primitive data, we have ways of taking data in and creating new kinds of data out, or new versions of data out, so we have operations.
Things like addition and multiplication, which we saw not only apply to numbers, but we can use them on things like strings and we're going to come back to them again.
Can't use them on Booleans, they have a different set of things.
They do things like AND, and OR.
And of course there's a bunch of other ones in there, I'm not going to put them all up, but we're building up a little collection, if you like, of those operations.
And then the main thing we've done is, we've talked about commands.
So I'm going to suggest we've seen now four different things.
We've seen assignment, how to bind a name to a value.
We've seen input and output.
Print for output, for example, and raw input for input.
We've seen conditionals, or said another way, branches, ways of changing the flow of control through that sequence of instructions we're building up.
And the last thing we added were loop mechanisms.
And here we saw, while.
It's the first example we've seen.
So what've we done so far?
Now, interestingly, this set of instructions was actually quite powerful, and we're going to come back to that later on, in terms of what we can do with it, but what we've really done is, given that basis, we're now talking about, how do we write common patterns of code, how do we write things that solve particular kinds of problems.
So what I want you to do, is to keep in mind, those are the bases, we ought to be able to do a lot with that bases, but what we're really interested in is not filling out a whole bunch of other things in here, but how do we put them together into common templates.
And we're going to do that today.
Second thing we've been doing, I want to highlight for you is, we've along the way, mostly just verbally rather than writing it down, but we've been talking about good style.
Good programming style.
All right?
Things that we ought to do, as you put these pieces together, in order to give you really good code.
And you should be collecting those together.
Give you some examples.
What have we talked about?
We've talked about things like using comments to highlight what you're doing in the code, to make it easier to debug.
We talked about type discipline, the notion that you should check the types of operands before you apply operators to them, to make sure that they're what the code is expecting.
We talked about descriptive use of good variable names, as a way, in essence, of documenting your code.
The fourth one we talked about was this idea of testing all possible branches through a piece of code, if it's got conditionals in it, to make sure that every possible input is going to give you an output that you actually want to see.
So, you know, you can start writing your own, kind of, Miss Manners book, if you like, I mean, are what are good programming, you know-- I wonder what you'd call them, John, good programming hygiene?
Good programming style?
Good programming practices?-- Things that you want to do to write good code.
OK. What we're going to do today is, we're going to start now building up, beyond just these pieces, although they're valuable, to start creating two things: one, common patterns of code that tackle certain classes of problems, and secondly we're going to talk about tools you can use to help understand those pieces of things.
OK.
So last time around, we talked about, or introduced if you like, iterative programs.
And I want to generalize that for a second, because we're going to come back and use this a lot.
And I want to do a very high-level description of what goes into an iterative program, or how I would think about this, all right?
And I know if John disagrees with me he'll tell me, but this is my way of thinking about it.
If I want to try and decide how to tackle a problem in an iterative matter, here the steps I'm going to go through.
First thing I'm going to do, is I'm going to choose a variable that's going to count.
What I meant-- what in the world do I mean by that?
I'm thinking about a problem, I'm going to show you an example in a second, first thing I'm going to do is say, what is the thing that's going to change every time I run through the same set of code?
What is counting my way through this process?
Now I'm putting count in double quotes, not to make it a string, but to say, this is count generically.
It could be counting one by one through the integers, it could also be taking a collection of data and going through them one by one.
It could be doing counting in some other mechanism.
But what's the variable I want to use?
Second thing I do, I need to initialize it.
And I need to initialize it outside of the loop.
That is, where do I want to start?
And I need to make sure I have a command that sets that up.
The third thing I'm going to do, is I need to set up the right end test.
How do I know when I'm done with the loop?
And obviously, that ought to involve the variable in some way, or it's not going to make a lot of sense, so this includes the variable, since that's the thing that's changing.
All right.
The fourth thing I'm going to do, is I'm going to then construct the block of code.
And I want to remind you, that block of code is a set of instructions, the same set of instructions that are going to be done each time through the loop.
All that's going to change, is the value the variable or the value of some data structures.
And remind you that inside of here, I'd better be changing the variable.
All right, if that variable that's counting is not changing, I'm going to be stuck in an infinite loop, so I ought to [UNINTELLIGIBLE PHRASE] that , right, expect somewhere in there, a change of that variable.
All right?
And then the last thing I want to do, is just decide, you know, what do I do when I'm done.
OK.
I know.
It looks boring.
But it's a structure of the things I want to think about when I go through trying to take a problem and mapping it into a iterative program.
Those are the things I want to see if I go along.
All right.
So let me give you an example.
I'm given an integer that's a perfect square, and I want to write a little piece of code that's going to find the square root of it.
All right, so I'm cheating a little, I know it's a perfect square, somebody's given it to me, we'll come back in a second to generalizing it, so what would the steps be that I'd use to walk through it?
Well if you think about these steps, here's an easy way to do it.
Let's start at 1.
Let's call x the thing I'm trying to find the square root of.
Let's start at 1.
Square it.
If it's not greater than x, take 2.
Square it.
If it's not greater than x, take 3.
Square it.
And keep going, until the square of one of those integers is greater than or equal to-- sorry, just greater than x. OK, why am I doing that?
When I get greater than x, I've gone past the place where I want to be.
And obviously, when I get to something whose square is equal to x, I've got the answer I want, and I kick it out.
So who knows what I've done?
I've identified the thing I'm going to use to count, something some variable is going to just count the integers, I've identified the end test, which is when that square is bigger than the thing I'm looking for, I've identified basically what I want to do inside the loop, which is simply keep changing that variable, and I didn't say what I want to do when I'm done, essentially print out the answer.
OK, so how can I code this up?
Well, you might think, let's just jump in and write some code, I don't want to quite do that though, because I want to show you another tool that's valuable for thinking about how to structure the code, and that is a something called a flow chart.
Now.
People of Professor Guttag's and my age, unfortunately remember flow charts back-- as they say, on the Simpsons, back in the day, back in the 1960's, John, right?-- really good programmers had these wonderful little plastic stencils, I tried to find one, I couldn't find it It's a little stencil with little cut-out shapes on it, that you used to draw flow charts, I'm going to show you in a second, and you tucked it right in here next to your pocket protector with all your pens in it, you know, so, your belt was also about this high, and your glasses were this thick, you know, we have a few of those nerds left, we mostly keep them in the museum, but that was what you did with the flow chart.
Despite making a bad joke about it, we're going to do the same thing here.
We're going to do the same thing here, we're going to chart out a little bit of what should go into actually making this thing work.
So here's a simple flow chart that I'm going to use to capture what I just described.
And I'm going to, again, I'm actually going to do it the way they used to do it, and draw a rectangle with rounded corners, that's my starting point, and then what did I say to do?
I said I need to identify a variable, I'm going to give it a name, let's just call ANS, for answer, and I need to initialize it, so I'm going to come down, and in a square box, I'm going to initialize ANS to 0.
And now I want to run through the loop.
What's the first thing I do in a loop?
I test an end test.
So the flow chart says, and the tradition was to do this in a diamond shape, I'm going to check if ANS times ANS-- oh, which way did I want to do this-- is less than or equal to x.
Now that's a test.
There are two possibilities.
If the answer is yes, then I'm still looking for the answer, what do I want to do?
Well, I don't have to do anything other than change the counter.
So I'm going to go to ANS is ANS plus 1, and I'm going to do it again.
Eventually, if I've done this right, that test is no-- and I wonderfully ran out of room here-- in which case, I'm going to go to a print statement, which was always done in a trapezoid, and print out ANS.
I should have put a box below it that says stop.
OK?
While.
And notice what I got here.
Actually, this is a useful tool for visualizing how I'm trying to put it together, because it lets me see where the loop is, right there, it lets me see the end test, it lets me make sure that I'm in fact initializing the variable and I'm checking the right things as I go along.
And the idea of this flow chart is, if you start, you know, a little ball bearing here, it's going to roll down, setting up an assignment statement, and then, depending on here, it's like there's a pair of flippers in there, it does the test, it sets the ball this way to change it to ANS plus 1, and comes back around, eventually it's going to drop through and print out the answer.
The reason I'm going to show you this flow chart, I'm going to do one other example in a second, but I want to show you a comparison.
Remember last time, we wrote this simple piece of code to print out even or odd.
If, you know, x, it was in fact, even or odd.
So let me show you what a flow chart for that would look like, because I want to make a comparison point here.
If I were to do a flow chart for that one, I'd do the following.
It reminds you, that the test here was, we took x if that's what we were looking for, it did integer division by 2, multiplied it by 2, and we check to see if that was the same as x.
If the answer is yes, then we did a print of even.
If the answer was no, we did a print of odd, and we then carried on.
Again, while.
But there's an important point here.
Remember last time, I said that there's different kinds of complexity in our code, and I suggested for simple branching programs, the amount of time it takes to run that program is, in essence, bounded by the number of instructions, because you only execute each instruction at most once.
It didn't depend on the size of the input.
And you can see that there.
I start off, either I take this path and carry on, or I take that path and carry on, but each box, if you like, gets touched exactly once.
On the other hand, look at this one.
This depends now on the size of x.
All right?
Because what am I going to do?
I'm going to come down and say, is ANS squared less than or equal to x?
If it is, I'm going to go around, and execute that statement, check it again, and go around and execute that.
So I'm going to cycle around that loop there enough times to get to the answer, and that number of times is going to depend on the input, so as I change the input, I'm going to change the complexity of the code.
Now this happens to be what we would call a linear process, because the number of times I go around the loop is directly related to the size of the argument.
If I double the argument, I'm going to double the number of times I go around the loop.
If I increase it by five, I'm going to increase by five the number of times I go around the loop.
We'll see later on, there are classes of computation that are inherently much more complex.
We hate them, because they're costly, but they're sometimes inherently that way.
But you can see the comparison between these two.
OK. Now, having done that, let's build this code.
Yeah, if my machine will come back up, there we go.
So, I'm going to now go ahead and write a little piece of code, and I put it here and I hope you can actually see these better this time, let me uncomment that region.
All right.
So, there's basically an encapsulation of that code, right?
It says-- what, look at this, where am I, right here-- I've got some value for x initially, I'm going to set ANS to 0, just like there, and there's my loop, there's the test, which is right like that, is ANS squared less than or equal to x, if it is, there's the block corresponding to the loop, change ANS, and eventually when I'm done with all this thing, I'm just going to print ANS out.
OK. All right, let me show you one other tool that I want to use.
Which is, I've written that piece of code, I ought to check it.
Well, I could just run it, but another useful thing to do is, I'm, especially as I want to debug these things, is to simulate that code.
And I'm going to do this because, as Professor Guttag noticed to me, students seem reluctant to do this.
I guess it's not macho enough, John, to just, you know, you know, go off and do things by hand, you ought to just run them, but it's a valuable tool to get into, so let me do that here
[UNINTELLIGIBLE]
I'm doing such a great job.
I've got to say, when my, I've got two sons, now aged eighteen and twenty, they used to think I had the coolest job in the world because I came home covered in chalk.
Now they have a different opinion that you can probably figure out.
All right.
Simulate the code.
What I mean by that is, pick a simple set of values, and let's walk through it to see what happens.
And this is useful because it's going to allow me to A, make sure that I've got something that's going to terminate, it's going to let me make sure that in fact I'm doing the right kinds of updates.
I could do this, by the way, by running the code and putting print statements in various places as well, but the hand simulation is valuable, so let me just start it.
What do I have here?
I need the variable, ANS, I need x, and I need ANS times ANS, ANS times ANS.
Right.
Those are the three things that are involved in this computation.
and I pick something reasonably simple.
The ANS starts at 0.
I set up x, I think, to be 16 there.
So what does the loop say?
I can either look at my flow chart, or I can look at the code.
If I look at the flow chart, it says, I'm at this point.
Look at ANS squared.
Is it less than or equal to-- sorry, first of all, ANS squared is 0, is it less than or equal to x, yes.
So what do I do?
Change ANS.
X doesn't change.
Back around to the test.
What's ANS squared?
It's 1.
Is it less than or equal to 16?
Sure.
Run the loop again.
ANS becomes 2.
X stays 16.
ANS squared is 4.
Is that less than or equal to 16?
Yes.
Aren't you glad I didn't pick x equals 500?
All right.
ANS goes up by 0.
ANS squared is nine.
Still less than or equal to 16.
ANS goes to 4.
X stays the same.
4 squared is 16.
Is 16 less than or equal to 16?
Yes.
So ANS goes to five.
ANS squared becomes 25.
Ah!
That is now no longer true here, so I print out 5.
Right.
Sure.
Square root of 16 is 5.
It's Bush economics.
OK?
I know.
I'm not supposed to make bad jokes like that.
What happened?
Yeah
It doesn't stop at the right place
It doesn't stop at the right place.
Thank you.
Exactly.
Right?
My bug here is right there.
Ah, let me find my cursor.
I probably want that.
Right?
I want less than, rather than less than or equal to.
This is an easy bug to come up with.
But imagine, if you don't do the test, you're going to get answers that don't make any sense.
And in fact, if we just go ahead and run this now, hopefully we get out-- oops, sorry, I'm going to have to change this quickly, I still have some things uncommented at the bottom, yeah, there they are, I don't think we need that yet, all right, we will comment those out.
OK.
So.
Why did I do it?
It's a simple example, I agree, but notice what I just did.
It allowed me to highlight, is the code doing the right thing?
I spotted an error here, I could have spotted it by running it on different test sets, and using prints things, another way of doing it, but this idea of at least simulating it on simple examples lets you check a couple of important questions.
And in fact, now let me ask those two questions about this piece of code.
First question is, for what values of integers-- we're going to assume integers-- but for what values of x does this code terminate?
And the second question is, for what values of x does it give me back the right answer?
All right, first question.
What values of x does it terminate?
Again, assume x is an integer.
Well, break it down into pieces.
Suppose x is positive.
Does it terminate?
Sure.
All right?
Because ANS starts out as 0, so ANS squared is 0, and each time through the loop, ANS is increasing.
That means, at some point, in some finite number of steps, ANS squared has got to get bigger than x if x is positive.
So for positive integers, it terminates.
And it probably, I think we can deduce, returns the right answer here.
Right.
X is negative.
X is -16.
Does this code terminate?
Boy, I feel like Arnold Schwarzenegger.
Does this terminate?
Somebody
[UNINTELLIGIBLE]
Ah, thank you, so it does terminate, right?
You're sitting too far back, let me try-- oh, too far!-- Sorry.
Come get me one later if you can't find it.
Yes, it stops at the first step, right?
Let's look at it.
It says, if answer, sorry, imagine x is -16, ANS is 0, is 0 less than -16, no.
So what does it do?
It prints out 0.
Ah!
So that now answers my second question, it does terminate, but does it give me the right answer?
No.
Right?
It gives me an answer, and imagine I'm using this somewhere else, you know, it's going to go off and say, gee, the square root of -16 is 0.
Well, it really should be a, you know, an imaginary number, but this is not a valuable thing to have come back.
So that's the second thing I've just highlighted here, is that I now have the ability to check whether it does the right thing.
And those are two things that you'd like to do with every looping construct you write: you'd like to be able to assure yourself that they will always terminate, and then the second thing you'd like to do, is to assure yourself that it does give you back a reasonable answer.
We started to talk about ways to do the former.
It's looking at the end test.
It's looking at the kinds of conditions you're going to put in.
For the latter, this is a place where running test cases would do a good job of helping with that.
Nonetheless, having done that, let's look at a better way to write this.
Which is right here, it is also, I think, on your sheet, I'm going to uncomment that, and comment this one out, yeah.
All right?
So let's look at this code for a second.
Notice what this does.
Certainly the heart of it, right in here, is still the same thing.
But notice what this does.
The first thing it does is, it says, let's check and make sure x is greater than or equal to 0.
If it isn't, notice what's going to happen.
None of that block is going to get executed, and it's going to come down here and print out a useful piece of information, which says, hey, you gave me a negative number.
I don't know how to do this.
If it is, in fact, positive, then we're going to go in here, but now notice what we're doing here.
There is the basic thing we did before, right?
We're checking the end test and incrementing, actually I was going to, I commented that out for a reason you'll see in a second, but I, normally I would keep this on, which would let me, at each step, see what it's doing.
If I ran this, it would print out each step.
Which is helping me make sure that it's incrementing the right way.
OK, once it gets to the end of that, what's it going to do?
It's going to come down here and, oh.
What's that doing?
Well, I cheated when I started.
I said, somebody's giving me a perfect square, I'm looking for the square root of it.
But suppose I gave this thing 15, and asked it to run.
It'd still give me an answer.
It just would not be the answer I'm looking for.
So now, in this case, this code is going to, when we get here, check, and if you haven't seen that strange thing there, that exclamation point in computer-ese called a bang, it says if ANS star ANS is not equal to x, all right?
What's that say, it says, I've already gotten to the end of the loop, I'm now past where I wanted to be, and I'm going to check to make sure that, in fact, this really is a perfect square.
If it isn't, print out something says, ah, you gave me something that wasn't a perfect square.
And only if that is true, am I going to print out the answer.
It's the same computation.
But this is a nice way of writing it, often called defensive programming.
And I think we have lots of variations on it-- I don't know about John, what your favorite is, for the definition of defensive programming-- for me it says, make sure that I'm going through all possible paths through the code, make sure I'm printing out, or returning if you like, useful information for each style, sorry, for each path through the code, make sure that for all possible inputs there is a path through the code, or a way to get through the code, that does not cause an error or infinite loop.
What else would you add, John?
Well, we'll come back to this later in the term, and talk in some detail about particular techniques.
The basic idea of defensive programming is, to assume that A, if you're getting inputs from a user, they won't necessarily give you the input you've asked for, so if you ask for a positive number, don't count on them giving you one, and B, if you're using a piece of a program written by a programmer who is not perfect, perhaps yourself, there could be mistakes in that program, and so you write your program under the assumption that, not only might the user make a mistake, other parts of your program might make a mistake, and you just put in lots of different tests under the assumption that you'd rather catch that something has gone wrong, then have it go wrong and not know it.
And we'll talk later in the term about dozens of different tricks, but the main thing to keep in mind is the general principle that people are dumb.
And will make mistakes.
And therefore, you write your programs so that catastrophes don't occur when those mistakes are made
Good.
As John said, we're going to come back to it.
But that's what, basically the goal here.
And you saw me put my hands up when I said stupid programmer?
I've certainly written code that has this problem, I've tried to use my own code that has this problem, and good to us, right, good hygiene, I'm going to use that word again here, of getting into the habit of writing defensive code up front, it's part of that collection of things that you ought to do, is a great thing to do.
I stress it in particular because, I know you're all going to get into this stage; you've got a problem set due in a couple of hours, you're still writing the code, you don't want to waste time, and I'm going to use quotes on "waste time", doing those extra things to do the defensive programming, you just want to get the darn thing done.
It's a bad habit to get into, because when you come back to it, it may haunt you later on down the road.
So really get into that notion of trying to be defensive as you program.
OK.
The other thing I want to say here, is that this style of program we just wrote, is actually a very common one.
And we're going to give it a nice little name, often referred to as exhaustive enumeration.
What does that mean?
It says, I'm literally walking through all possible values of some parameter, some element of the computation, testing everything until I find the right answer.
All right, so it's, you know, again, I can even write that down, essentially saying, try all reasonable values until you find the solution.
And you might say, well, wait a minute, isn't that going to be really expensive?
And the answer is, yeah, I guess, if you want to search, you know, all the pages on Google, one by one, yes, probably, it's going to take a while.
But there are an awful lot of computations for which this is the right way to do it.
You just want to exhaustively go through things.
And just to give you a sense of that, let me show you an example.
I'm going to change this, all right?
Nice big number.
You know, computers are fast these days.
I can make this even bigger, it's going to do it fairly quickly, so it really is quick to do this.
It doesn't mean that exhaustive enumeration is a bad idea, it is often the right idea to use.
So we've seen one example of this, this idea of walking through all the integers looking for the square root.
Let's look at some other examples, in order to try and see other ways in which we could do it.
OK.
In particular, let's go over to here, and let me show you a second example.
And let me comment that out.
Here's another problem that I'd like to solve.
Suppose I want to find all the divisors of some integer, I want to figure out what all the divisors are that go evenly into it.
Again, same kind of reasoning says, given some value x, I happened to pick a small one here, what's an easy way to do this?
Well, let's just start at one.
That's my variable I'm going to change and check.
Does it divide evenly into x?
If it does, print it out.
Move on to the next one, print it out.
So again, I can do the same kind of thing here, you can see that, in fact, let's just run it to make sure it does the right thing, OK?
In fact, if I go back to the code, what did I decide to do here?
I say, starting with an initialization of I, there's my first step, as equal to 1, I'm going to walk through a little loop where I check, as long-- first of all, as long as I is less than x, so there's my end test, I'm going to do something.
And in this case, the something is, I'm going to look to see if I divides x evenly.
So I'll remind you of that amp-- sorry, that percent sign there, that says if x divided by I has a 0 remainder, because this gives me back the remainder, if that's equal to 0, print something out.
And there's my nice increment.
Simple little piece of code.
Notice again, exactly the same form: I picked the thing I wanted to vary, I initialized it outside the loop, I have a test to see when I'm done, and then I've got a set of instructions I'm doing every time inside the loop.
In this case, it's doing the check on the remainder and printing them out.
And when I'm done with the whole thing, before I end the increment of the variable, you know, when I'm done, I'm just not returning anything out.
OK.
So now you've seen two simple examples.
Let me generalize this.
In this case, my incrementer was just adding 1 to an integer, it's a pretty straightforward thing to do.
But you can imagine thinking about this a little differently.
If I somehow had a collection, an ordered collection of all the integers, from 1 to 10, I could imagine doing the same thing, where now what I'm doing is, I'm starting with the first element of that collection, doing something, going to the next element, doing something, going to the next element, doing something, I'm just walking through the sequence of elements.
Right?
And I haven't said yet, how do I get that collection, but you could certainly conceptualize that, if I had that collection, that would be nice thing to do.
That is a more common pattern.
That is basically saying, given some collection of data, I want to have again a looping mechanism, where now my process is, walk through this, the collection, one element at a time.
And for that, we have a particular construct, called a FOR loop.
It's going to do exactly that for us.
It's going to be more general than this, and we're going to come back to that, in fact, Professor Guttag's going to pick this up in a couple of lectures, but we can talk right now about the basic form.
The form of a FOR loop says, FOR, and I'm going to put little angle braces in here again, to say, for some variable, like a name I want to get to it, in some collection, and then I have a block of code.
And what it's saying semantically is, using that variable as my placeholder, have it walk through this collection, starting at the first thing, execute that code, then the next thing, execute that code, and so on.
One of the advantages of this is, that I don't have to worry about explicitly updating my variable.
That happens for me automatically.
And that's very nice, because this allows me to be sure that my FOR loop is going to terminate.
And because, as long as this collection is finite, this thing is just going to walk through.
All right?
So, if I show you, for example, I'm going to comment this one out in the usual manner, and let's look at uncommenting that, there is the same piece of code.
Now, I slung something by you, or snuck something by you, which is, I hadn't said how to generate the set of integers from 1 to 10.
So, range is a built-in Python function.
I'm going to come back to it in a second.
For now, just think of it as saying, it gives you all the integers from 1 up to, but not including, x. OK.
But now you can see the form.
This now says, OK, let I start as the first thing in there, which is 1, and then do exactly as I did before, the same thing, but notice I don't need to say how to increment it.
It's happening automatically for me.
OK.
In fact, if I run it, it does the same thing, which is what I would expect.
OK. Now, the advantage of the FOR, as I said, is that it has, then, if you like, a cleaner way of reading it.
I don't have to worry about, do I initialize it, did I forget to initialize it outside the loop, it happens automatically just by the syntax of it, right there, that's going to start with the first element.
I don't have to worry about, did I remember to put the incrementer in, it's going to automatically walk it's way through there.
Second advantage of the FOR is, that right now, we're thinking about it just as a sequence of integers.
We could imagine it's just counting its way through.
But we're going to see, very shortly, that in fact those collections could be arbitrary.
We're going to have other ways of building them, but it could be a collection of all the primes.
Hm.
There's an interesting thing to do.
It could be a collection of, ah, you know, I don't know, batting averages of somebody or other.
It could be arbitrary collections that you've come up with in other ways.
The FOR is, again, going to let you walk through that thing.
So it does not have to be something that could be described procedurally, such as add 1 just to the previous element.
It could be any arbitrary collection.
And if I were to use that again, I'd just put it on your handout, I could go back and rewrite that thing that I had previously for finding the square roots of the perfect squares, just using the FOR loop.
OK. What I want to do, though, is go on to-- or, sorry, go back to-- my divisor example.
[UNINTELLIGIBLE PHRASE] OK.
Try again.
I've got a number, I want to find the divisors.
Right now, what my code is doing is, it's printing them up for me, which is useful.
But imagine I actually wanted to gather them together.
I wanted to collect them, so I could do something with them.
I might want to add them up.
Might want to multiply them together.
Might want to do, I don't know, something else with them, find common divisors, of things by looking at them.
I need, in fact, a way to make explicit, what I can't do that with range, is I need a way to collect things together.
And that's going to be the first of our more compound data structures, and we have exactly such a structure, and it's called a tuple.
This is an ordered sequence of elements.
Now, I'm going to actually add something to it that's going to make sense in a little while, or in a couple of lectures, which is, it is immutable.
Meaning, I cannot change it, and we'll see why that's important later on.
But for now, tuple is this ordered sequence of structures.
OK. And how do I create them?
Well, the representation is, following a square bracket, followed by a sequence of elements, separated by commas, followed by a closed square bracket.
And that is literally what I said, it is an ordered sequence of elements, you can see where they are.
OK?
So, let me do a little example of this.
If I go back over here, let's define-- er, can't type-- I can look at the value of test, it's an ordered sequence.
I need to get elements out of it.
So again, I have a way of doing that.
In particular, I can ask for the zeroth element of test.
OK, notice, I'm putting a square bracket around it, and it gives me-- I know this sounds confusing, but this is a long tradition, it gives me-- ah, yes
[UNINTELLIGIBLE]
Sorry?
[UNINTELLIGIBLE]
I created a list here?
Ah, thank you.
I'm glad you guys are on top of it.
You're saying I want that.
Is that right, John?
Yes?
OK.
Sorry.
You're going to see why this was a mistake in a little while.
I did not want to make a list, I wanted to create a tuple thank you for catching it.
I want parens, not square brackets there.
You'll also see in a little while why both of these things would work this way, but it's not what I wanted.
OK?
So I guess I should go back, and let me do this correctly this way.
Again, I can look at test, and I guess test now if I want to get the element out-- angle bracket or square bracket?
I still want square bracket, that's what I thought-- OK. Now I can go back to where I was, which is a strange piece of history, which is, we start counting at 0.
So the-- I hate to say it this way, the first element of this tuple is at position 0, or index 0, OK?-- so I can get the zeroth one out, I can get, if I do 2, I get the third thing out, because it goes 0, 1, 2-- notice, however, if I do something that tries to go outside the length of the tuple it complains, which is right.
Tuples?
also have another nice structure, which is, I can go the other direction, which is, if I want to get the last element of that tuple I give it a negative index.
So, imagine, you think of it as, is it starting right, just before the beginning of the thing, if I give it a 0 it's going to take the first one, if I give it a 1, it's going to take the next one, but I can go the other direction, if I give it a -1, it picks up the last element of the tuple.
And again, I can go -2, go back.
So this is what we would call selection.
We can do things like foo of 0 to get out the particular element.
I can also pick up pieces of that tuple.
Again I want to show you the format here.
If I give it this strange expression, this is saying I want to get the piece of the tuple starting at index 1, it's going to be the second element, and going up to but not including index 3.
And it gives me back that piece.
Actually a copy of that piece of the tuple.
This is called slicing.
And then just to complete this.
Two other nice things you can do with slices are you can get the beginning or the end of tuple.
So, for example, if I say TEST and I don't give it a start but I give it an end, then it gives me all the elements up to that point.
And I can obviously do the other direction which is I can say skip to index 2 and all the remaining pieces.
This lets me slice out, if you like, the front part or back part or a middle part of the tuple as I go along.
What in the world does that have to do with my divisor example?
Well, actually, before I do that let me in fact fill in a piece here.
Which is remember I said range we could think of conceptually as a tuple -- or sorry as a sequence of these things.
In fact it gives me back, now I hate this, it's actually a list it's not a tuple.
But for now think of it as giving you back an explicit version of that representation of all those elements.
You'll see why I'm going to make that distinction in a couple of lectures.
All right.
What does this have to do with my divisor example?
This says I can make tuples, but imagine now going back to my divisor example and I want to gather up the elements as I go along.
I ought to be able to do that by in fact just adding the pieces in.
And that's what I'm going to do over here.
Which is, let me comment that out, let me uncomment that.
And I guess I need the same thing here, right?
I need parens not, thank you.
You can tell I'm an old time list packer.
I really do love these things.
And is that right, John?
OK, so my apologies that your handout is wrong.
I did not think to check about the difference between these things.
Nonetheless, having done that, let's look at what I'm going to do.
I now want to run a loop where I need to collect things together.
I'm going to give a name to that.
And what you see there is I'm going to call divisors initially an empty tuple, something has nothing in it.
Right here.
And then I'm going to run through the same loop as before, going through this set of things, doing the check.
Now what I'd like to do, every time I find a divisor I'd like to gather it together.
So I'm going to create a tuple of one element, the value of i.
And then, ah, cool.
Here's that addition operation that's badly overloaded.
This is why Professor Guttag likes and I don't.
Because given that this is a tuple and that's a tuple, I can just add them together.
That is concatenate them, if you like, one on the end of it.
And if I keep doing that, when I'm done divisor will be a collection of things.
So let me just run it.
All right.
This is what I get for trying to -- STUDENT There should be a comment after the i in parentheses
Thank you.
Right there.
All right, we'll try this again.
OK. And there are the set of devices.
Thank you.
Who did that?
Somebody gets, no?
Yours?
Thank you.
Nice catch too by the way.
All right, so now that you can see that I can screw up programming, which I just did.
But we fixed it on the fly.
Thank you.
What have we done?
We've now got a way of collecting things together, right?
And this is the first version of something we'd like to use.
Now that I've gotten that bound as a name, I could go in and do things with that.
I could go in and say give me the fourth divisor, give me the second through fifth divisor.
Again as I suggested if I've got two integers and I want to find common divisors I could take those two lists and walk through them.
I shouldn't say list, those two tuples, and walk through them to find the pieces that match up.
So I've got a way now of gathering data together.
The last thing I want to do is to say all right, now that we've got this idea of being able to collect things into collections, we've got the ability now to use looping structures as we did before but we can walk down then doing things to them, where else might we have this need to do things with looping structures?
And I'm going to suggest you've already seen it.
What's a string?
Well at some level it is an ordered sequence of characters.
Right?
Now it is not represented this same way.
You don't see strings inside these open parens and closed parens.
You don't see strings with commas between them, but it has the same kind of property.
It is in ordered sequence of characters.
We'd like to do the same thing with strings.
That is we'd like to be able to get pieces of them out.
We'd like to be able add them together or concatenate them together.
We'd like to be able to slice them.
And in fact we can.
So strings also support things like selection, slicing, and a set of other parameters, other properties.
And let's just look at that.
Again if I go back here, let me comment this out.
Right here are a pair of strings that I've set up, s 1 and s 2.
Let me just run these.
We can go back over here.
So I can see the value of s 1, it's a string.
I can do things like s 1 and s 2.
As we saw before, it simply concatenates them together and gives me back a longer string.
But I can also ask for parts of this.
So I can, for example, say give me the first element of string 1, s 1.
Ah, that's exactly what we would have thought if this was represented as an ordered sequence of things.
Again I should have said first, index 0, the first one.
I can similarly go in and say I'd like all the things between index 2 and index 4.
And again, remember what that does.
Index 2 says start a 0.
1, 2.
So a, b, c. And then it goes up to but not including index 4 so it gets c and d and then it stops.
I can similarly, just as I did with the tuples, I can ask for everything up to some point or I can ask for everything starting at some point and carrying on.
Now what you're seeing here then is the beginning of complex data structures.
And the nice thing is that there's a shared behavior there.
Just as I can have tuples as an ordered collection of things, strings behave as an ordered collection of things.
So I can start thinking about doing manipulation on strings.
I can concatenate them together, I can find pieces inside of them, I could actually do things with them.
And let me show you just a simple little example of something I might want to do.
Suppose I take, I better comment this one out or it's going to spit it out.
Let me comment that out.
Suppose I take a number.
I'd like to add up all the digits inside of the number.
I can use the tools I've just described in order to capture that.
So what would I want to do?
I'd like to somehow walk down each of the digits one at a time and add them up.
Ah, that's a looping mechanism, right?
I need to have some way of walking through them.
An easy way to do it would be inside of a FOR.
And what would I like to do?
Well I need to take that number and I'm going to turn it into a string.
So notice what I'm going to do right here.
I take that number and convert it into a string.
That's an example of that type conversion we did earlier on.
By doing that it makes it possible for me to treat it as an ordered sequence of characters.
And so what's the loop going to do?
It's going to say FOR c, which was my name for the character in that string.
That means starting at the first one, I'm going to do something to it.
And what am I'm going to do?
I'm going to take that character, convert it back into an integer, and add it into some digits.
And I've done a little short hand here, which is I should have said some digits is equal to some digits plus this.
But that little short hand there is doing exactly the same thing.
It is adding that value into some digits and putting it back or signing it back into some digits.
And I'll walk through that loop and when I'm done I can print out the total thing does.
And if I do that, I get out what I would expect.
So what have I done?
We've now generalized the idea of iteration into this little pattern.
Again as I said this is my version of it, but you can see, every one of the examples we've used so far has that pattern to it.
Figure out what I'm trying to walk through.
What's the collection of things I'm trying to walk through.
Figure out what I want to do at each stage.
Figure out what the end test is.
Figure out what I'm going to do at the end of it.
I can write it explicitly.
I can write it inside of a FOR loop.
And we've started to add, and we'll see a lot more of this, examples of collections of structures so that we don't just have to do something that can be easily described as walking through a set of things but can actually be a collection that you walk through.
The last thing I want to point out to you is, I started out with this list.
I haven't added anything to the list, right?
I mean I've got a different kind of looping mechanism.
I guess I should say that's not quite true.
I've added the ability to have more complex data structures here.
But I dropped a hint in the first lecture about what you could computer with things.
In fact if you think for a second about that list, you could ask what can I compute with just that set of constructs?
And the answer is basically anything.
This is an example of what is referred to frequently as being a Turing complete language.
That is to say with just those set of constructs, anything you can describe algorithmically you can compute with that set of constructs.
So there's good news and bad news.
The good news is it sounds like we're done.
Class is cancelled until final exam because this is all you need to know, right?
The bad news is of course that's not true.
The real issue is to figure out how to build constructs out of this that tackle particular problems, but the fundamental basics of computation are just captured in that set of mechanisms.
All right, we'll see you next time.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time we were talking about binary search and I sort of left a promise to you which I need to pick up.
I want to remind you, we were talking about search, which is a very fundamental thing that we do in a whole lot of applications.
We want to go find things in some data set.
And I'll remind you that we sort of a separated out two cases.
We said if we had an ordered list, we could use binary search.
And we said that was log rhythmic, took log n time where n is the size of the list.
If it was an unordered list, we were basically stuck with linear search.
Got to walk through the whole list to see if the thing is there.
So that was of order in.
And then one of the things that I suggested was that if we could figure out some way to order it, and in particular, if we could order it in n log n time, and we still haven't done that, but if we could do that, then we said the complexity changed a little bit.
But it changed in a way that I want to remind you.
And the change was, that in this case, if I'm doing a single search, I've got a choice.
I could still do the linear case, which is order n or I could say, look, take the list, let's sort it and then search it.
But in that case, we said well to sort it was going to take n log n time, assuming I can do that.
Once I have it sorted I can search it in log n time, but that's still isn't as good as just doing n. And this led to this idea of amortization, which is I need to not only factor in the cost, but how am I going to use it?
And typically, I'm not going to just search once in a list, I'm going to search multiple times.
So if I have to do k searches, then in the linear case, I got to do order n things k times.
It's order k n. Whereas in the ordered case, I need to get them sorted, which is still n log n, but then the search is only log n. I need to do k of those.
And we suggested well this is better than that.
This is certainly better than that.
m plus k all times log n is in general going to be much better than k times n. It depends on n and k but obviously as n gets big, that one is going to be better.
And that's just a way of reminding you that we want to think carefully, but what are the things we're trying to measure when we talk about complexity here?
It's both the size of the thing and how often are we going to use it?
And there are some trade offs, but I still haven't said how I'm going to get an n log n sorting algorithm, and that's what I want to do today.
One of the two things I want to do today.
To set the stage for this, let's go back just for a second to binary search.
At the end of the lecture I said binary search was an example of a divide and conquer algorithm.
Sort of an Attila the Hun kind of approach to doing things if you like.
So let me say -- boy, I could have made a really bad political joke there, which I will forego, right.
Let's say what this actually means, divide and conquer.
Divide and conquer says basically do the following: split the problem into several sub-problems of the same type.
I'll come back in a second to help binary searches matches in that, but that's what we're going to do.
For each of those sub-problems we're going to solve them independently, and then we're going to combine those solutions.
And it's called divide and conquer for the obvious reason.
I'm going to divide it up into sub-problems with the hope that those sub-problems get easier.
It's going to be easier to conquer if you like, and then I'm going to merge them back.
Now, in the binary search case, in some sense, this is a little bit trivial.
What was the divide?
The divide was breaking a big search up into half a search.
We actually threw half of the list away and we kept dividing it down, until ultimately we got something of size one to search.
That's really easy.
The combination was also sort of trivial in this case because the solution to the sub-problem was, in fact, the solution to the larger problem.
But there's the idea of divide and conquer.
I'm going to use exactly that same ideas to tackle sort.
Again, I've got an unordered list of n elements.
I want to sort it into a obviously a sorted list.
And that particular algorithm is actually a really nice algorithm called merge sort.
And it's actually a fairly old algorithm.
It was invented in 1945 by John von Neumann one of the pioneers of computer science.
And here's the idea behind merge sort, actually I'm going to back into it in a funny way.
Let's assume that I could somehow get to the stage where I've got two sorted lists.
How much work do I have to do to actually merge them together?
So let me give you an example.
Suppose I want to merge two lists, and they're sorted.
Just to give you an example, here's one list, 3121724 Here's another list, 12430.
I haven't said how I'm going to get those sorted lists, but imagine I had two sorted lists like that.
How hard is it to merge them?
Well it's pretty easy, right?
I start at the beginning of each list, and I say is one less than three?
Sure.
So that says one should be the first element in my merge list.
Now, compare the first element in each of these lists.
Two is less than three, so two ought to be the next element of the list.
And you get the idea.
What am I going to do next?
I'm going to compare three against four.
Three is the smallest one, and I'm going to compare four games twelve, which is going to give me four.
And then what do?
I have to do twelve against thirty, twelve is smaller, take that out.
Seventeen against thirty, twenty-four against thirty And by this stage I've got nothing left in this element, so I just add the rest of that list in.
Wow I can sort two lists, so I can merge two lists.
I said it poorly.
What's the point?
How many operations did it take me to do this?
Seven comparisons, right?
I've got eight elements.
It took me seven comparisons, because I can take advantage of the fact I know I only ever have to look at the first element of each sub-list.
Those are the only things I need to compare, and when I run out of one list, I just add the rest of the list in.
What's the order of complexity of merging?
I heard it somewhere very quietly
n
Sorry, and thank you.
Linear, absolutely right?
And what's n by the way here?
What's it measuring?
[UNINTELLIGIBLE]
In both lists, right.
So this is linear, order n and n is this sum of the element, or sorry, the number of elements in each list.
I said I was going to back my way into this.
That gives me a way to merge things.
So here's what merge sort would do.
Merge sort takes this idea of divide and conquer, and it does the following: it says let's divide the list in half.
There's the divide and conquer.
And let's keep dividing each of those lists in half until we get down to something that's really easy to sort.
What's the simplest thing to sort?
A list of size one, right?
So continue until we have singleton lists.
Once I got a list of size one they're sorted, and then combine them.
Combine them by doing emerge the sub-lists.
And again, you see that flavor.
I'm going to just keep dividing it up until I get something really easy, and then I'm going to combine.
And this is different than binary search now, the combine is going to have to do some work.
So, I'm giving you a piece of code that does this, and I'm going to come back to it in the second, but it's up there.
But what I'd like to do is to try you sort sort of a little simulation of how this would work.
And I was going to originally make the TAs come up here and do it, but I don't have enough t a's to do a full merge sort.
So I'm hoping, so I also have these really high-tech props.
I spent tons and tons of department money on them as you can see.
I hope you can see this because I'm going to try and simulate what a merge sort does.
I've got eight things I want to sort here, and those initially start out here at top level.
The first step is divide them in half.
All right?
I'm not sure how to mark it here, remember I need to come back there.
I'm not yet done.
What do I do?
Divide them in half again.
You know, if I had like shells and peas here I could make some more money.
What do I do?
I divide them in half one more time.
Let me cluster them because really what I have, sorry, separate them out.
I've gone from one problem size eight down to eight problems of size one.
At this stage I'm at my singleton case.
So this is easy.
What do I do?
I merge.
And the merge is, put them in order.
What do I do next?
Obvious thing, I merge these.
And that as we saw was a nice linear operation.
It's fun to do it upside down, and then one more merge which is I take the smallest elements of each one until I get to where I want.
Wow aren't you impressed.
No, don't please don't clap, not for that one.
Now let me do it a second time to show you that -- I'm saying this poorly.
Let me say it again.
That's the general idea.
What should you see out of that?
I just kept sub-dividing down until I got really easy problems, and then I combine them back.
I actually misled you slightly there or maybe a lot, because I did it in parallel.
In fact, let me just shuffle these up a little bit.
Really what's going to happen here, because this is a sequential computer, is that we're going to start off up here, at top level, we're going to divide into half, then we're going to do the complete subdivision and merge here before we ever come back and do this one.
We're going to do a division here and then a division there.
At that stage we can merge these, and then take this down, do the division merge and bring them back up.
Let me show you an example by running that.
I've got a little list I've made here called test.
Let's run merge sort on it, and then we'll look at the code.
OK, what I would like you to see is I've been printing out, as I went along, actually let's back up slightly and look at the code.
There's merge sort.
Takes in a list.
What does it say to do?
It says check to see if I'm in that base case.
It's the list of length less than two.
Is it one basically?
In which case, just return a copy the list.
That's the simple case.
Otherwise, notice what it says to do.
It's says find the mid-point and split the list in half.
Copy of the back end, sorry, copy of the left side, copy of the right side.
Run merge sort on those.
By induction, if it does the right thing, I'm going to get back two lists, and I'm going to then merge them together.
Notice what I'm going to do.
I'm going to print here the list if we go into it, and print of the when we're done and then just return that.
Merge up here.
There's a little more code there.
I'll let you just grok it but you can see it's basically doing what I did over there.
Setting up two indices for the two sub-list, it's just walking down, finding the smallest element, putting it into a new list.
When it gets to the end of one of the lists, it skips to the next part, and only one of these two pieces will get called because only one of them is going to have things leftovers.
It's going to add the other pieces in.
OK, if you look at that then, let's look at what happened when we ran this.
We started off with a call with that list.
Ah ha, split it in half.
It's going down the left side of this.
That got split in half, and that got split in half until I got to a list of one.
Here's the first list of size one.
There's the second list of size one.
So I merged them.
It's now in the right order, and that's coming from right there.
Having done that, it goes back up and picks the second sub-list, which came from there.
It's a down to base case, merges it.
When these two merges are done, we're basically at a stage in that branch where we can now merge those two together, which gives us that, and it goes through the rest of it.
A really nice algorithm.
As I said, an example of divide and conquer.
Notice here that it's different than the binary search case.
We're certainly dividing down, but the combination now actually takes some work.
I'll have to actually figure out how to put them back together.
And that's a general thing you want to keep in mind when you're thinking about designing a divide and conquer kind of algorithm.
You really want to get the power of dividing things up, but if you end up doing a ton of work at the combination stage, you may not have gained anything.
So you really want to think about that trade off.
All right, having said that, what's the complexity here?
Boy, there's a dumb question, because I've been telling you for the last two lectures the complexity is n log n, but let's see if it really is.
What's the complexity here?
If we think about it, we start off with the problem of size n. What do we do?
We split it into two problems of size n over 2.
Those get split each into two problems of size n over 4, and we keep doing that until we get down to a level in this tree where we have only singletons left over.
Once we're there, we have to do the merge.
Notice what happens here.
We said each of the merge operations was of order n. But n is different.
Right?
Down here, I've just got two things to merge, and then I've got things of size two to merge and then things of size four to merge.
But notice a trade off.
I have n operations if you like down there of size one.
Up here I have n over two operations of size two.
Up here I've got n over four operations of size four.
So I always have to do a merge of n elements.
How much time does that take?
Well, we said it, right?
Where did I put it?
Right there, order n. So I have order n operations at each level in the tree.
And then how many levels deep am I?
Well, that's the divide, right?
So how many levels do I have?
Log n, because at each stage I'm cutting the problem in half.
So I start off with n then it's n over two n over four n over eight.
So I have n operations log n times, there we go, n log n. Took us a long time to get there, but it's a nice algorithm to have.
Let me generalize this slightly.
When we get a problem, a standard tool to try and attack it with is to say, is there some way to break this problem down into simpler, I shouldn't say simpler, smaller versions of the same problem.
If I can do that, it's a good candidate for divide and conquer.
And then the things I have to ask is how much of a division do I want to do?
The obvious one is to divide it in half, but there may be cases where there are different divisions you want to have take place.
The second question I want to ask is what's the base case?
When do I get down to a problem that's small enough that it's basically trivial to solve?
Here it was lists of size one.
I could have stopped at lists of size two right.
That's an easy comparison.
Do one comparison and return one of two possible orders on it, but I need to decide that.
And the third thing I need to decide is how do I combine?
You know, point out to you in the binary search case, combination was trivial.
The answer to the final search was just the answer all the way up.
Here, a little more work, and that's why I'll come back to that idea.
If I'm basically just squeezing jello, that is, I'm trying to make the problem simpler, but the combination turns out to be really complex, I've not gained anything.
So things that are good candidates for divide and conquer are problems where it's easy to figure out how to divide down, and the combination is of little complexity.
It would be nice if it was less than linear, but linear is nice because then I'm going to get that n log in kind of behavior.
And if you ask the TAs in recitation tomorrow, they'll tell you that you see a lot of n log n algorithms in computer science.
It's a very common class of algorithms, and it's very useful one to have.
Now, one of the questions we could still ask is, right, we've got binary search, which has got this nice log behavior.
If we can sort things, you know, we get this n log n behavior, and we got a n log n behavior overall.
But can we actually do better in terms of searching.
I'm going to show you one last technique.
And in fact, we're going to put quotes around the word better, but it does better than even this kind of binary search, and that's a method called hashing.
You've actually seen hashing, you just don't know it.
Hashing is the the technique that's used in Python to represent dictionaries.
Hashing is used when you actually come in to Logan Airport and Immigration or Homeland Security checks your picture against a database.
Hashing is used every time you enter a password into a system.
So what in the world is hashing?
Well, let me start with a simple little example.
Suppose I want to represent a collection of integers.
This is an easy little example.
And I promise you that the integers are never going to be anything other than between the range of zero to nine.
OK, so it might be the collection of one and five.
It might be two, three, four, eight.
I mean some collection of integers, but I guarantee you it's between zero and nine.
Here's the trick I can play.
I can build -- I can't count -- I could build a list with spots for all of those elements, zero, one, two, three, four, five, six, seven, eight, nine.
And then when I want to create my set, I could simply put a one everywhere that that integer falls.
So if I wanted to represent, for example, this is the set two, six and eight, I put a one in those slots.
This seems a little weird, but bear with me for second, in fact, I've given you a little piece a code to do it, which is the next piece of code on the hand out.
So let's take a look at it for second.
This little set of code here from create insert and number.
What's create do?
It says, given a low and a high range, in this case it would be zero to nine.
I'm going to build a list.
Right, you can see that little loop going through there.
What am I doing?
I'm creating a list with just that special symbol none in it.
So I'm building the list.
I'm returning that as my set.
And then to create the object, I'll simply do a set of inserts.
If I want the values two, six and eight in there, I would do an insert of two into that set, an insert of six into that set, and an insert of eight into the set.
And what does it do?
It marks a one in each of those spots.
Now, what did I want to do?
I wanted to check membership.
I want to do search.
Well that's simple.
Given that representation and some value, I just say gee is it there?
What's the order complexity here?
I know I drive you nuts asking questions?
What's the order complexity here?
Quadratic, linear, log, constant?
Any takers?
I know I have the wrong glasses on the see hands up too, but
[UNINTELLIGIBLE]
Who said it?
Constant
Constant, why?
[UNINTELLIGIBLE]
Yes, thank you.
All right, it is constant.
You keep sitting back there where I can't get to you.
Thank you very much.
It has a constant.
Remember we said we design lists so that the access, no matter where it was on the list was of constant time.
That is another way of saying that looking up this thing here is constant.
So this is constant time, order one.
Come on, you know, representing sets of integers, this is pretty dumb.
Suppose I want to have a set of characters.
How could I do that?
Well the idea of a hash, in fact, what's called a hash function is to have some way of mapping any kind of data into integers.
So let's look at the second example, all right, -- I keep doing that -- this piece of code from here to here gives me a way of now creating a hash table of size 256.
Ord as a built in python representation.
There is lots of them around that takes any character and gives you back an integer.
In fact, just to show that to you, if I go down here and I type ord, sorry, I did that wrong.
Let me try again.
We'll get to exceptions in a second.
I give it some character.
It gives me back an integer representing.
It looks weird.
Why is three come back to some other thing?
That's the internal representation that python uses for this.
If I give it some other character, yeah, it would help if I could type, give it some other character.
It gives me back a representation.
So now here's the idea.
I build a list 256 elements long, and I fill it up with those special characters none.
That's what create is going to do right here.
And then hash character takes in any string or character, single character, gives me back a number.
Notice what I do.
If I want to create a set or a sequence representing these things, I simply insert into that list.
It goes through and puts ones in the right place.
And then, if I want to find out if something's there, I do the same thing.
But notice now, hash is converting the input into an integer.
So, what's the idea?
If I know what my hash function does, it maps, in this case characters into a range zero to 256, which is zero to 255, I create a list that long, and I simply mark things.
And my look up is still constant.
Characters are simple.
Suppose you want to represent sets of strings, well you basically just generalize the hash function.
I think one of the classic ones for strings is called the Rabin-Karp algorithm.
And it's simply the same idea that you have a mapping from your import into a set of integers.
Wow, OK, maybe not so wow, but this is now constant.
This is constant time access.
So I can do searching in constant time which is great.
Where's the penalty?
What did I trade off here?
Well I'm going to suggest that what I did was I really traded space for time.
It makes me sound like an astro physicist somehow right?
What do I mean by that?
I have constant time access which is great, but I paid a price, which is I had to use up some space.
In the case of integers it was easy.
In the case of characters, so I have to give up a list of 256, no big deal.
Imagine now you want to do faces.
You've got a picture of somebody's face, it's a million pixels.
Each pixel has a range of values from zero to 256.
I want to hash a face with some function into an integer.
I may not want to do the full range of this, but I may decide I have to use a lot of gigabytes of space in order to do a trade off.
The reason I'm showing you this is it that this is a gain, a common trade off in computer science.
That in many cases, I can gain efficiency if I'm willing to give up space.
Having said that though, there may still be a problem, or there ought to be a problem that may be bugging you slightly, which is how do I guarantee that my hash function takes any input into exactly one spot in the storage space?
The answer is I can't.
OK, in the simple case of integers I can, but in the case of something more complex like faces or fingerprints or passwords for that matter, it's hard to design a hash function that has completely even distribution, meaning that it takes any input into exactly one output spot.
So what you typically do and a hash case is you design your code to deal with that.
You try to design -- actually I'm going to come back to that in a second.
It's like you're trying to use a hash function that spread things out pretty evenly.
But the places you store into in those lists may have to themselves have a small list in there, and when you go to check something, you may have to do a linear search through the elements in that list.
The good news is the elements in any one spot in a hash table are likely to be a small number, three, four, five.
So the search is really easy.
You're not searching a million things.
You're searching three or four things, but nonetheless, you have to do that trade off.
The last thing I want to say about hashes are that they're actually really hard to create.
There's been a lot of work done on these over the years, but in fact, it's pretty hard to invent a good hash function.
So my advice to you is, if you want to use something was a hash, go to a library.
Look up a good hash function.
For strings, there's a classic set of them that work pretty well.
For integers, there are some real simple ones.
If there's something more complex, find a good hash function, but designing a really good hash function takes a lot of effort because you want it to have that even distribution.
You'd like it to have as few duplicates if you like in each spot in the hash table for each one of the things that you use.
Let me pull back for a second then.
What have we done over the last three or four lectures?
We've started introducing you to classes of algorithms.
Things that I'd like you to be able to see are how to do some simple complexity analysis.
Perhaps more importantly, how to recognize a kind of algorithm based on its properties and know what class it belongs to.
This is a hint.
If you like, leaning towards the next quiz, that you oughta be able to say that looks like a logarithmic algorithm because it's got a particular property.
That looks like an n log n algorithm because it has a particular property.
And the third thing we've done is we've given you now a set of sort of standard algorithms if you like.
Brute force, just walk through every possible case.
It works well if the problem sizes are small.
We've had, there are a number of variants of guess and check or hypothesize and test, where you try to guess the solution and then check it and use that to refine your search.
Successive approximation, Newton-Raphson was one nice example, but there's a whole class of things that get closer and closer, reducing your errors as you go along.
Divide and conquer and actually I guess in between there bi-section, which is really just a very difficult of successive approximation, but divide and conquer is a class of algorithm.
These are tools that you want in your tool box.
These are the kinds of algorithms that you should be able to recognize.
And what I'd like you to begin to do is to look at a problem and say, gee, which kind of algorithm is most likely to be successful on this problem, and map it into that case.
OK, starting next -- don't worry I'm not going to quit 36 minutes after -- I got one more topic for today.
But jumping ahead, I'm going to skip in a second now to talk about one last linguistic thing from Python, but I want to preface Professor Guttag is going to pick up next week, and what we're going to start doing then is taking these classes of algorithms and start looking at much more complex algorithms.
Things you're more likely to use in problems.
Things like knapsack problems as we move ahead.
But the tools you've seen so far are really the things that were going to see as we build those algorithms.
OK, I want to spend the last portion of this lecture doing one last piece of linguistics stuff.
One last little thing from Python, and that's to talk about exceptions.
OK, you've actually seen exceptions a lot, you just didn't know that's what they were, because exceptions show up everywhere in Python.
Let me give you a couple of examples.
I'm going to clear some space here.
Before I type in that expression, I get an error, right?
So it's not defined.
But in fact, what this did was it threw an exception.
An exception is called a name error exception.
It says you gave me something I didn't know how to deal.
I'm going to throw it, or raise it, to use the right term to somebody in case they can handle it, but it's a particular kind of exception.
I might do something like, remind you I have test.
If I do this, try and get the 10th element of a list that's only eight long.
I get what looks like an error, but it's actually throwing an exception.
The exception is right there.
It's an index error, that is it's trying to do something going beyond the range of what this thing could deal with.
OK, you say, come on, I've seen these all the time.
Every time I type something into my program, it does one of these things, right?
When we're just interacting with idol, with the interactive editor or sorry, interactive environment if you like, that's what you expect.
What's happening is that we're typing in something, an expression it doesn't know how to deal.
It's raising the exception, but is this simply bubbling up at the top level saying you've got a problem.
Suppose instead you're in the middle of some deep piece of code and you get one of these cases.
It's kind of annoying if it throws it all the way back up to top level for you to fix.
If it's truly a bug, that's the right thing to do.
You want to catch it.
But in many cases exceptions or things that, in fact, you as a program designer could have handled.
So I'm going to distinguish in fact between un-handled exceptions, which are the things that we saw there, and handled exceptions.
I'm going to show you in a second how to handle them, but let's look at an example.
What do I mean by a handled exception?
Well let's look at the next piece of code.
OK, it's right here.
It's called read float.
We'll look at it in a second.
Let me sort of set the stage up for this -- suppose I want to input -- I'm sorry I want you as a user to input a floating point number.
We talked about things you could do to try make sure that happens.
You could run through a little loop to say keep trying until you get one.
But one of the ways I could deal with it is what's shown here.
And what's this little loop say to do?
This little loop says I'm going to write a function or procedures that takes in two messages.
I'm going to run through a loop, and I'm going to request some input, which I'm going to read in with raw input.
I'm going to store that into val.
And as you might expect, I'm going to then try and see if I can convert that into a float.
Oh wait a minute, that's a little different than what we did last time, right?
Last time we checked the type and said if it is a float you're okay.
If not, carry on.
In this case what would happen?
Well float is going to try and do the cohersion.
It's going to try and turn it into a floating point number.
If it does, I'm great, right.
And I like just to return val.
If it doesn't, floats going to throw or raise, to use the right term, an exception.
It's going to say something like a type error.
In fact, let's try it over here.
I if I go over here, and I say float of three, it's going to do the conversion.
But if I say turn this into a float, ah it throws a value error exception.
It says it's a wrong kind of value that I've got.
So I'm going to write a little piece of code that says if it gives me a float, I'm set, But if not, I'd like to have the code handle the exception.
And that's what this funky try/except thing does.
This is a try/except block and here's the flow of control that takes place in there.
When I hit a try-block.
It's going to literally do that.
It's going to try and execute the instructions.
If it can successfully execute the instructions, it's going to skip past the except block and just carry on with the rest of the code.
If, however, it raises an exception, that exception, at least in this case where it's a pure accept with no tags on it, is going to get, be like thrown directly to the except block, and it's going to try and execute that.
So notice what's going to happen here, then.
If I give it something that can be turned into a float, I come in here, I read the input, if it can be turned into a float, I'm going to just return the value and I'm set.
If not, it's basically going to throw it to this point, in which case I'm going to print out an error message and oh yeah, I'm still in that while loop, so it's going to go around.
So in fact, if I go here and, let me un-comment this and run the code.
It says enter a float.
And if I give it something that can be -- sorry, I've got, yes, never mind the grades crap.
Where did I have that?
Let me comment that out.
Somehow it's appropriate in the middle of my lecture for it to say whoops at me but that wasn't what I intended.
And we will try this again.
OK, says it says enter a float.
I give it something that can be converted into a float, it says fine.
I'm going to go back and run it again though.
If I run it again, it says enter a float.
Ah ha, it goes into that accept portion, prints out a message, and goes back around the while loop to say try again.
And it's going to keep doing this until I give it something that does serve as a float.
Right, so an exception then has this format that I can control as a programmer.
Why would I want to use this?
Well some things I can actually expect may happen and I want to handle them.
The float example is a simple one.
I'm going to generalize in a second.
Here's a better example.
I'm writing a piece of code that wants to input a file.
I can certainly imagine something that says give me the file name, I'm going to do something with it.
I can't guarantee that the file may exist under that name, but I know that's something that might occur.
So a nice way to handle it is to write it as an exception that says, here's what I want to do if I get the file.
But just in case the file name is not there, here's what I want to do in that case to handle it.
Let me specify what the exception should do.
In the example I just wrote here, this is pretty trivial, right.
OK, I'm trying to input floats.
I could generalize this pretty nicely.
Imagine the same kind of idea where I want to simply say I want to take input of anything and try and see how to make sure I get the right kind of thing.
I want to make it polymorphic.
Well that's pretty easy to do.
That is basically the next example, right here.
In fact, let me comment this one out.
I can do exactly the same kind of thing.
Now what I'm going to try and do is read in a set of values, but I'm going to give a type of value as well as the messages.
The format is the same.
I'm going to ask for some input, and then I am going to use that procedure to check, is this the right type of value.
And I'm trying to use that to do the coercion if you like.
Same thing if it works, I'm going to skip that, if it not, it's going to throw the exception.
Why is this much nice?
Well, that's a handy piece of code.
Because imagine I've got that now, and I can now store that away in some file name, input dot p y, and import into every one of my procedure functions, pardon me, my files of procedures, because it's a standard way of now giving me the input.
OK, so far though, I've just shown you what happens inside a peace a code.
It raises an exception.
It goes to that accept clause.
We don't have to use it just inside of one place.
We can actually use it more generally.
And that gets me to the last example I wanted to show you.
Let me uncomment this.
Let's take a look at this code.
This looks like a handy piece of code to have given what we just recently did to you.
All right, get grades.
It's a little function that's going to say give me a file name, and I'm going to go off and open that up and bind it to a local variable.
And if it's successful, then I'd just like to go off and do some things like turn it into a list so I can compute average score or distributions or something else.
I don't really care what's going on here.
Notice though what I've done.
Open, it doesn't succeed is going to raise a particular kind of exception called I O error.
And so I've done a little bit different things here which is I put the accept part of the block with I O error.
What does that say?
It says if in the code up here I get an exception of that sort, I'm going to go to this place to handle it.
On the other hand, if I'm inside this procedure and some other exception is raised, it's not tagged by that one, it's going to raise it up the chain.
If that procedure was called by some other procedure it's going to say is there an exception block in there that can handle that.
If not, I am going to keep going up the chain until eventually I get to the top level.
And you can see that down here.
I'm going to run this in a second.
This is just a piece of code where I'm going to say, gee, if I can get the grades, do something, if not carry on.
And if I go ahead and run this -- now it's going to say woops, at me.
What happened?
I'm down here and try, I'm trying do get grades, which is a call to that function, which is not bound in my computer.
That says it's in here.
It's in this try-block.
It raised an exception, but it wasn't and I O error.
So it passes it back, past this exception, up to this level, which gets to that exception.
Let me say this a little bit better then.
I can write exceptions inside a piece of code.
Try this, if it doesn't work I can have an exception that catches any error at that level.
Or I can say catch only these kinds of errors at that level, otherwise pass them up the chain.
And that exception will keep getting passed up the chain of calls until it either gets to the top level, in which case it looks like what you see all the time.
It looks like an error, but it tells you what the error came from, or it gets an exception , it can deal with it.
OK, so the last thing to say about this is what's the difference between an exception and an assert?
We introduced asserts earlier on.
You've actually seen them in some pieces of code, so what's the difference between the two of them?
Well here's my way of describing it.
The goal of an assert, or an assert statement, is basically to say, look, you can make sure that my function is going to give this kind of result if you give me inputs of a particular type.
Sorry, wrong way of saying it.
If you give me inputs that satisfy some particular constraints.
That was the kind of thing you saw.
Asserts said here are some conditions to test.
If they're true, I'm going to let the rest of the code run.
If not, I'm going to throw an error.
So the assertion is basically saying we got some pre-conditions, those are the clauses inside the assert that have to be true, and there's a post condition.
and in essence, what the assert is saying is, or rather the programmer is saying using the assert is, if you give me input that satisfies the preconditions, I'm guaranteeing to you that my code is going to give you something that meets the post condition.
It's going to do the right thing.
And as a consequence, as you saw with the asserts, if the preconditions aren't true, it throws an error.
It goes back up the top level saying stop operation immediately and goes back up the top level.
Asserts in fact are nice in the sense that they let you check conditions at debugging time or testing time.
So you can actually use them to see where your code is going.
An exception, when you use an exception, basically what you're saying is, look, you can do anything you want with my function, and you can be sure that I'm going to tell you if something is going wrong.
And in many cases I'm going to handle it myself.
So as much as possible, the exceptions are going to try to handle unexpected things, actually wrong term, you expected them, but not what the user did.
It's going to try to handle conditions other than the normal ones itself.
So you can use the thing in anyway.
If it can't, it's going to try and throw it to somebody else to handle, and only if there is no handler for that unexpected condition, will it come up to top level.
So, summarizing better, assert is something you put in to say to the user, make sure you're giving me input of this type, but I'm going to guarantee you the rest of the code works correctly.
Exceptions and exception handlers are saying, here are the odd cases that I might see and here's what I'd like to do in those cases in order to try and be able to deal with them.
Last thing to say is why would you want to have exceptions?
Well, let's go back to that case of inputting a simple little floating point.
If I'm expecting mostly numbers in, I can certainly try and do the coercion.
I could have done that just doing the coercion.
The problem is, I want to know if, in fact, I've got something that's not of the form I expect.
I'm much better having an exception get handled at the time of input than to let that prop -- that value rather propagate through a whole bunch of code until eventually it hits an error 17 calls later, and you have no clue where it came from.
So the exceptions are useful when you want to have the ability to say, I expect in general this kind of behavior, but I do know there are some other things that might happen and here's what I'd like to do in each one of those cases.
But I do want to make sure that I don't let a value that I'm not expecting pass through.
That goes back to that idea of sort of discipline coding.
It's easy to have assumptions about what you think are going to come into the program when you writ it.
If you really know what they are use them as search, but if you think there's going to be some flexibility, you want to prevent the user getting trapped in a bad spot, and exceptions as a consequence are a good thing to use.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
This is 600.
So I hope any of you who thought this was a different course find where you really belong.
My name is John Guttag.
And I'll be lecturing the course all semester.
OK. Today I want to accomplish several things.
Cover some of the administrative details.
Talk about the goals of the course-- what I hope you'll learn.
And then begin talking about the conceptual material in the course.
It will seem a little bit slow, because it will be a little bit slow.
I promise starting on Thursday we're going to pick up the pace considerably.
So let's start with the strategic goals of the course.
The official introduction to course 6 is 601.
Historically, students who arrive at MIT with little or no programming experience find 601 an ordeal.
And the point of this is to prepare freshman and sophomores for entering course 6-- that's the Electrical Engineering Computer Science department-- in a gentler, kinder way.
So that 601 is not so much of a problem.
I want to help students feel justifiably-- and I want to emphasize the word justifiably-- confident in their ability to write small and medium sized programs.
So at the end of the term, you should all feel comfortable writing programs.
The real theme of the course, and what I spend most of the time on, is how to map problems into a computational framework.
There's going to be an emphasis on scientific problems, rather than, say, commercial problems.
But there will be some talk about some non-scientific problems as well.
How to take a problem that may not at first blush appear to be attackable with the program, and show you how to formulate the problem in such a way that you can use computation to get insight into the problem.
It should not take you very long.
All of the problem sets involve programming in Python programming language, which I'll say a little bit about later today.
The first problem set, basically, is getting Python installed on your own computer.
Most of the people will want to just use whatever their own laptop is to do the problem sets on.
Don't get fooled by the first two problem sets into thinking this is a gut course.
It's not.
It starts out gently to lure you in.
And then life gets pretty hard pretty quickly.
So don't be fooled.
The quizzes-- and there will be two evening quizzes and a final-- are open book open notes.
When you get to be my age, you get very sensitive about how difficult it is to remember things.
And so, we won't be asking you to memorize stuff.
The course is about solving problems, knowing how to solve problems, not how much you can remember.
For many of you who are majoring in biology or course 20, it's going to be kind of a shock that this course isn't about how much you can remember, but it's about how well you can solve problems.
And that's what the quizzes are really going to be focused on.
Probably the most unusual thing about this course is the collaboration policy, which is liberal in the extreme.
You can collaborate with anybody you want on any of the problem sets.
Not on the quizzes.
But on any of the problem sets.
You can work with each other, which is what I recommend.
You can work with your parents, if one of them happens to be a software engineer.
You can work with friends in course 6.
Whatever you want to do.
The goal of the problem sets is to help you learn.
What we've seen in the past is people who are a little, shall I say too collaborative, i.e.
they just copy the problem set from somebody else, live in a fool's paradise which comes crashing down at the first quiz.
People who don't spend enough time thinking about the problem sets themselves cannot take the quizzes successfully.
So it's a fine line.
But our goal is not to be policemen.
I tell my TAs, their job is to help you learn, not to prevent you from, quote, cheating.
So to solve that problem, we've eliminated the concept of cheating on problem sets.
There is no way to cheat on problem sets.
So just go and do them.
There's no textbook.
We will be posting readings on the web.
For the most part, these will be pointers to websites.
Occasionally we'll post readings that we wrote ourselves.
Doesn't mean you shouldn't buy a textbook.
In fact, there are a number of Python texts.
We'll recommend a few of them.
It might make sense to buy one and bring it to a quiz, because it will have an index that will let you look things up quickly, that sort of thing.
But again, a lot of students never buy a text and do just fine.
We will not be handing out class notes on a regular basis.
A lot of studies have indicated that students learn more when they take their own notes than when they are handed out.
And so, as a matter of what I think is good pedagogy, we don't hand out detailed lecture notes.
We will be using, after today, a lot of handouts with code on them, which we'll make available.
But it's not intended to make any sense outside the context of lectures.
It's not self-contained.
The main purpose of this course is to help you become skillful in making the computer do what you want it to do.
Once you acquire this skill, your first instinct, when confronted with many tasks, will be to write a program to do that task for you.
I always tell people I became a computer scientist in part because I'm lazy.
And there was a lot of stuff that I found it was easier to write a program to make the computer do it, rather than do it myself.
So I do that a lot.
If I need to do something, I say, can I just write a quick program to do it?
And I'd like you to be able to acquire that skill.
And remember, that programming is actually a lot of fun.
So I should say that, in addition to learning a lot in this course, I hope most of you will find it fun.
Kind of a strange thought.
MIT course, fun.
Maybe it's an oxymoron.
But I don't think so.
I really do think you can have a lot of fun writing programs for this course.
There are many people who believe that-- how shall I say this?
Programming is the most fun you can have with your clothes on.
It really can be a lot of fun.
So think of it that way.
All right.
So the primary knowledge you're going to take away is computational problem solving.
So to start with, we might ask the question, what is computation?
And to think about that-- this is interesting.
Ah, there's where the chalk is.
I was afraid I was going to be confronted with a sea of black boards and erasers and no chalk.
But there is chalk.
So if we think about it, there are essentially two kinds of knowledge.
Declarative-- and you're going to see, I'm not a great speller.
And imperative.
Declarative knowledge is composed of statements of fact.
For example, a good health care plan improves the quality of medical care while saving money.
As we know from doings in Washington, it's a lot easier to state that goal than to know how to achieve that goal.
So the key thing about declarative knowledge is, it says something that is true.
Otherwise it wouldn't be knowledge, it would be misinformation.
But doesn't tell you how to do it.
In a more mathematical sense, say y is the square root of x if and only if y times y equals x.
All right?
Perfectly clear statement of what it means to be the square root, but it doesn't tell you how to find the square root.
Interestingly enough, it does tell you how to test whether or not you have the answer to the square root.
And so if you had some way of generating guesses, you can at least check whether they're correct.
And in fact, starting in the next lecture, we'll talk about the fact that a lot of computational techniques involve something called guess and check.
Where you have a way to generate guesses and a way to check whether they're right.
Imperative knowledge, in contrast, tells you how to solve a problem.
How to accomplish something.
So you could think of it as like a recipe in a cookbook.
So it's one thing to say a chocolate cake is something that tastes delicious and is bad for you.
That's declarative knowledge.
But you can open a cookbook and get a recipe that tells you how to make a chocolate cake.
That's imperative knowledge.
Now, below, we have a recipe for finding not a square root necessarily, but an approximation to a square root.
And one of the themes of this course is that a lot of problems we cannot solve precisely, but we can find answers that are good enough for practical purposes.
And those are called approximation algorithms.
So here's a way-- this is a very old method for finding the square root.
In fact, it's believed that Heron was the one who did this.
Heron of Alexandria.
In the news much today, Alexandria was the capital of ancient Egypt.
He was the first one to write this method down a long time ago.
Though it's believed that even before Heron, the Babylonians know how to do it.
So you start with a guess, g. Any old guess will do.
Then you say, is g times g close enough to x?
If so, you stop.
Say OK, I've got a good enough approximation of the answer.
If it's not, you create a new guess by averaging g and x divided by g. So g new is going to be g old plus x divided by g old, over 2.
And then using this new guess, you go back to step 2.
So let's quickly run through an example of this.
We can start with this pretty easily.
We'll take a guess.
Let's say g equals 3.
So we look at three times 3.
9.
And we say, is that good enough?
Well, let's say we're looking for the root of 25.
I guess I should have started with the problem statement.
Sorry about that.
Well, 9 is probably not close enough to 25 that we're happy.
May be good enough for government work, but not for most other purposes.
So we'll reset g, and we'll set g to 3 plus 25 over 3.
All of that over 2.
Which equals 5.6666 et cetera.
All right.
So now we'll multiply that by itself.
And that gets to be about 32.04.
Close enough to 25?
Probably not.
So we'll take another step.
And we'll set g equal to-- well, when we're done with it all, I'm not going to bore you with writing the formula again.
It'll be 5.04.
If we square that, it's 25.4.
We decide that's close enough to 25, and we're done.
What we say at this point is that the algorithm-- and that's an important word.
An algorithm is a description of how to perform a computation.
We say that the algorithm has converged.
Which is a fancy way to say it's halted.
What we've got here, if you think about it, is a set of instructions.
Steps that can be executed and a flow of control.
The order in which we execute them.
So if we look at this, there's a default order of execution-- 1, 2, 3, 4.
But then there's the go back to step 2 and start over.
And there's a termination condition.
It tells us when to stop.
And of course, that's important.
I've always been amused, if you look at a shampoo bottle, you'll see an algorithm that says something like lather, rinse, repeat.
And if you follow it literally, you never get to stop.
Which I suppose make sense if you're selling shampoo, because people use a lot of it.
But really, there ought to be some termination condition there.
OK.
So now, how do we capture this idea of a recipe in a mechanical process?
One way would be to design a machine specifically to do square roots.
So if I knew how to design circuits, which I don't, I could sit down-- probably many of you could sit down-- and design a circuit that would implement this algorithm.
And in fact, that's more or less what you'll find in a cheap four function calculator that does square roots.
Not quite this algorithm, but a similar algorithm is just part of the circuitry to go compute that.
And in fact, this used to be the way that all computers worked.
So the initial computers were what are called fixed program computers.
They were designed to do very specific things, and that's what they did.
So for example, one of the very first computers, designed in 1941, by Atanasoff and Berry, solved systems of linear equations for the purpose of plotting artillery trajectories.
And that's all it did.
If you wanted to balance your bank account with this computer, you couldn't do it.
But you could figure out how to drop an artillery shell somewhere.
Also during World War II, Alan Turing built a machine specifically designed for breaking the German enigma code.
Actually a fascinating story of science how that was built.
But again, that was all it could do.
These computers were useful, but only in a very limited way.
The big breakthrough, the thing it made computation really important to society, was the invention of the stored program computer.
It took people quite a while to figure this out.
But once they did, it seems obvious.
The basic notion of a stored program computer is that the instructions are the same as data.
So now, there is no distinction between the program that implements the algorithm and the data on which that program operates.
So there's no difference between the input of 25, part of the data, and the steps of the algorithm used to do that.
Once that was possible, the machines became infinitely flexible.
You could change the program anytime you wanted.
And furthermore, programs could produce programs.
Because programs can produce data.
And if program and data are the same thing, that means programs can produce programs.
And we were off and running.
And that's really what made computers what they are today.
Once this became clear as the paradigm for computers, people began to think of the computer itself as a program.
And in particular, as a kind of program called an interpreter.
And we'll get to more on this later today.
An interpreter is a program that can execute any legal set of instructions.
And consequently, can be used to describe and accomplish anything you can do with a computer.
So roughly speaking, this is what a stored program computer looks like.
This is 6004 in 40 seconds.
It's got memory.
Lots of it today.
A control unit that basically tells it what to do.
For example, fetch some data from memory, put some data into memory, send some output to a screen, all of those kinds of things.
What for historical reasons we call the arithmetic logic unit, this is, in some sense, the brains of the computer.
The thing that actually does computations.
An accumulator, which is part of the ALU that stores results.
And a bunch of input and output devices.
The things that we actually see when we use a computer.
And that's it.
And again, the key thing to notice is, there's only one kind of memory.
There's not a memory for program and a memory for data.
There's just the memory.
The nice thing to think about here is, given a small set of instructions, you can then build any kind of program you want.
So typically, the computers have a very small number of built-in instructions.
Order of dozens, and that's it.
And by combining those instructions in very clever ways, you can do arbitrarily complex things.
In much the same way a good chef can take a very small number of ingredients, and from those, produce a variety of interesting edibles.
Alan Turing, in the 1930s-- very famous British mathematician of whom you will hear more-- showed that, in fact, there were six primitive instructions.
Each of which operated on one bit of information.
And with those six primitive instructions, you could do anything that could be done with a computer.
Kind of amazing.
It was six instructions.
There were things like read, write, plus-- I don't know, maybe minus.
I forget what they were.
And that was it.
That's all you needed.
We will not make you write programs using only six instructions.
We will give you a much larger set.
But still, it's really quite remarkable.
It's what makes programming such an amazing endeavor.
OK.
So what instructions will you be using?
Well, that's what a programming language does.
So a programming language provides a set of primitive instructions.
A set of primitive control structures.
So instructions and mechanisms for controlling the order in which they get executed.
And that's all.
And then you can do whatever you want with them.
And what distinguishes one programming language from another is what these things are.
What are your instructions?
What of your flow of control?
And how do you combine them?
What are the combining mechanisms?
And in fact, it's the combining mechanisms more than anything else that separate one language from another.
The most amazing thing about programming-- and this has its good side and its bad side, and it's something you need to remember as you do the problem sets-- is that the computer will always do exactly what you tell what to do.
It's remarkable.
You don't have any friends who will do whatever you tell them to do.
I can tell you my children certainly don't do whatever I tell them to do.
And my wife doesn't either.
Sometimes she probably thinks I do whatever she tells me to do.
But a computer will do what you tell it to do.
So that's very empowering.
It's also very annoying.
Because it means if your program doesn't work, it's your own darn fault.
You got nobody else to blame but yourself.
Because it's not the computer's fault.
You may want to curse the computer, but you shouldn't.
It's just doing what you told it to.
So be careful what you wish for.
All right.
The programming language we're going to use in 600 is Python.
It's a relatively recent addition to the universe of languages.
I want to emphasize that this course is not about learning Python.
I will spend relatively little time in the lectures telling you about Python.
It's about computational methods, is what this course is really about.
And Python is merely a teaching tool.
Once you learn to program in Python, it's easy to learn to program in another language.
It's a very easily transferable skill.
If we think about what defines any programming language, it's got a syntax, a static semantics, and a semantics.
Are any of you here linguistics majors?
Not a one.
All right.
Then I can make up whatever I want about these terms and maybe you'll believe me.
All right.
So the syntax tells us which sequences of characters and symbols constitute a well-formed string.
So it would tell us, maybe, that we could write something like x equals 3 plus 4.
And that's syntactically correct.
It's well-formed.
It might also tell us that x equals 3 blank 4 is not syntactically correct.
It's not a legal string.
So by analogy with English, the syntax describes which strings of words constitute well-formed sentences.
Well-formed.
Not necessarily meaningful.
So it would tell you that some sentence like Susan is building is syntactically well-formed.
It may not be very sensible.
The static semantics tells us which well-formed strings have a meaning.
That are which strings are meaningful.
So you can think about that as also making sense.
So in Python, it might tell us that some strings which are syntactically fine don't mean anything.
So for example, it might tell us that the string 3 divided by the character string abc is syntactically well-formed because it's value operator value.
Sort of like noun verb noun is syntactically well-formed in English.
But it would tell us that there's no real meaning to this.
Dividing a number by a string doesn't mean anything.
And so you would get an error message saying the syntax is OK, but the static semantics is broken.
So for example, in English , the sentence I are big is somehow syntactically well-formed-- noun verb noun-- but we might say it fails the static semantic test.
We don't want to assign a meaning to it.
The semantics of the language looks only at the strings that are both syntactically correct and static semantically correct, and assigns a real meaning to them.
In natural language, sentences can be ambiguous.
So one of my favorites, when I have to write a recommendation letter for a student that maybe I don't think is so good, I might say something like I cannot praise this student too highly.
Well, you can interpret that however you want.
It keeps me from getting sued, but I can also claim, well, I don't like the student at all.
And English is full of those things.
Programming languages, in contrast, are designed so that every well-formed program has exactly one meaning.
There's no ambiguity.
So you can't typically talk of a program as having a semantic error.
If it is well-formed, it means something, and that's what it means.
On the other hand, it's easy to talk about a program meaning something other than you wanted it to mean.
And you will discover in the problem sets, most of the time the programs don't mean what you want them to mean.
That is to say, when you run them, they don't give you the correct answer.
And then you will go through this process of debugging them and learning how to do it.
So what might happen when we write a program that doesn't do what we want it to do?
It might crash.
By that, we mean stop running and produce some palpable indication that it has done so.
So you've all used programs that have crashed, right?
You sat there using your email program or Word, or PowerPoint, or something.
And suddenly, it just goes away.
And you get a message on your screen and an invitation to send Apple or Microsoft a file explaining what went wrong so they can fix it.
In a properly designed computing system, when one program crashes, it does not damage the overall system.
So you'd like it to just be local.
What else might it do?
It might never stop.
Now, if you have no idea how long a program is supposed to run, this can be hard to diagnose.
But again, I'm sure you've all run into this.
I've certainly run into it.
Every once in a while I'll say, try and write a PowerPoint file.
And it'll just sit there.
Or I'll try to read a file and it'll just sit there and never finish the job.
Or I don't have enough patience.
But probably it would never have finished it.
Again, you will all write programs that do this.
It's a good idea to know how long you expect your programs to run, so that you can recognize this.
Typically, we say that these programs have in them an infinite loop.
And we'll talk about that when we get to flow of control.
Finally, a program might run to completion and produce the wrong answer.
These problems are kind of in ascending order of badness.
If it crashes, at least you know that something has gone wrong.
An infinite loop can be very annoying, because you just wait for a long time.
But the worst thing that happens is when you think everything is good and it's not.
There have been lots of examples of this.
This is the sort of thing that costs lives.
There was a radiation therapy machine that produced the wrong dosage of radiation and actually killed quite a few people, because they put in the correct input, and it would dose the patient with radiation.
And a fatal dose of radiation.
That's a really bad mistake.
There are buildings that collapse because people run programs that do the structural engineering, and the programs give the wrong answer.
Lots of bad things can happen.
So one of the things we're going to spend time on this term is, what you can do to avoid writing programs that have this rather unpleasant property.
How do you test them?
How do you write them in such a way that this is the least likely event?
That's not what you want to happen.
OK.
Some programming languages give you a lot of help in avoiding these things.
Python is kind of mediocre in that respect.
It's not the best.
It's not the worst.
It's somewhere in the middle.
Because what you'd like is a program with very rigorous static semantics, such that if you pass those tests, it has a high probability of behaving as expected.
So for example, it's a good thing that Python doesn't allow you to do this.
Because who knows what that's going to do?
Something weird.
You'd rather be told no, you can't write that.
And then you have to write something that's more obviously meaningful.
Rather than it just making up an interpretation.
As we will see going forward, Python is not, for example, as good as Java is at weeding out meaningless things.
Or things that have surprising meanings.
On the other hand, it's better than C. So kind of in the middle as these programming languages go.
Why do we use Python in this course if it's not the best in that respect?
It's got several good features.
One of them is, it's easy to learn.
It's much less complicated than, say, Java.
So the learning curve is much steeper.
That's a good thing.
You get up to speed faster.
It's very widely used today in a lot of areas of science, particularly the life sciences.
It has probably become the most popular language in biology and the other life sciences.
And therefore, for those of you who have careers in that area, it's the most useful language to know.
It's also widely used in other areas as well.
It is easier to debug than most languages.
And the reason it's easier to debug than most languages, or than many, is it's an interpreted language.
So you'll remember, I talked about a computer as an interpreter.
Something that you feed in a bunch of instructions, called the source code.
You do some checking.
And then it executes the instructions, including the flow of control instructions.
Produces some output.
The nice thing that goes on there is if something untoward happens, the interpreter can describe in the language of the source code what went wrong.
The source code is the code that you wrote.
On the other hand, the way a compiler works is, you take the source code, you check it, but then you translate it into another language called the object code.
This is a language closer to the language that the computer, the hardware, knows how to interpret.
Then the hardware interpreter interprets the compiled code, the object code, and produces output.
And the problem here is if something goes wrong, it wants to give you an error message in terms of the object code, which you've never seen in your life.
And that can make it very obscure.
So the advantage?
Why do we have compilers?
Typically, compiled languages are more efficient.
Because they go through this extra step, they take less time to run those programs.
You can compile Python as well, if you want to get an efficient version.
But it's not designed under that assumption.
And so, it works well when it's interpreted, which is why we use it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning
Good morning
Thank you.
OK.
So we're about to launch into learning some basic elements of Python today.
The elements I'm going to talk about are common to every programming language that I know, at least in concept, and of course slightly different in detail.
But as I said last time, everything you're going to learn about Python should be readily transferable.
I'll be using, for all of the examples I present, something called an integrated development environment, and in particular, one that's built for Python called IDLE.
Usually we talk about these things as IDEs.
I'm told, I don't know if it's true, that the IDE for Python is called IDLE after Eric Idle of Monty Python, which was also, I'm told, the inspiration for the name of the programming language.
So what is an integrated programming environment?
In this case, it includes a specialized text editor that provides highlighting, auto-completion, smart indent, and you'll see shortly why all that's very important, and a few other amenities that make it easier to use this to type Python than typing it into a generic text editor.
It includes something called a shell, which is the environment that actually interprets the Python code.
And the nice thing about it is it includes syntax highlighting.
So the shell gives you some information about the syntax, as does the text editor of course.
And finally, it includes an integrated debugger.
This could be useful in the unlikely event that your programs have errors when you write them.
Though truth be told, I've been programming in Python for years and I don't know that I've ever used the debugger.
It's not that I don't make mistakes, it's just that I'm kind of a Luddite, and I typically use print statements for debugging.
And in fact, almost every programmer that I know, when push comes to shove, ends up using print statements.
But the debugger is there, should you care to take a try.
All right.
So you'll see on the screen here an IDLE shell.
In the shell, we can type things.
What are we going to type?
Well the first thing to understand is that at the core of Python, and probably the most important thing to understand, are something called objects.
Everything in Python is an object.
So every kind of entity that you can create in Python is an object, and in fact, Python code itself is an object.
You'll remember, we talked about stored program computers last time, and the concept that a program is data, just like a number is data.
Each object has a type that tells us the kind of object it is, and in particular, what we can do with it.
And then there's a built-in function, called type, that can be used to find out the type of an object.
As we'll see, there are two fundamental kinds of types.
Scalar and non-scalar.
We'll start with talking about scalar types.
And the key thing to think about there is that they are indivisible.
Think of them as the atoms of the programming language.
Now, I know that some of you have studied physics and know that atoms are in principle divisible, but of course, only at great expense and with serious consequences.
And we've seen the same thing here.
You can, if you're desperate, chop up these scalar types, but it almost always leads to something bad.
All right.
Well let's look at some.
Well, the first one you'll see is used to represent integers, and that's called int.
For every type, or every built-in type, there's the notion of a literal, which is how we type it.
So for example, we can type 3, and that will now tell us it is the value 3.
You'll note it's typed it in blue.
And I can ask what the type of 3 is.
So you'll notice as I type things into the shell, it's using colors to give me a hint.
So it's this fuchsia color for the word type, telling me that's a built-in function.
And now if I ask it, it will tell me that the type of the literal 3 is "int".
So it's an integer.
And I can use other sorts of things.
All right.
There's also a type float.
So those correspond to the real numbers.
So I can do something like that.
And we'll talk about this in a second, but you'll notice if I do type of 3.2, it tells me it's a float.
And for that matter, I can do type of 3.0, and it will tell me it's a float.
So there's a difference between 3 and 3.0.
One is an int, and one is a float.
Now you'll notice something kind of weird here.
When the interpreter printed back the value of the literal 3.2, it gave me 3.2 and a bunch of zeroes, and then this funny 2 standing at the end.
In a few lectures, I'll explain why it does this, but for now, you should just take this as a warning that floats are not the same thing as real numbers.
You learned about reals, presumably in middle school or high school.
Floats are a computer scientist's approximation to reals, but they're not quite the same.
The good news is almost all the time, you can pretend that a floating point number is a real, but as we'll see in a few lectures, every once in a while it can really sit up and bite you, if you believe that.
But for now, we'll just pretend that they're reals.
There's Booleans, a nice scalar type of which there are only two values.
One of them is true, and what do you think the other Boolean value is?
False?
Thank you.
So somebody said false.
I have no idea who, but whoever it is, there's probably some candy to be had.
Oh, I managed to find the one place in the room where there was an empty.
I'm hoping that people will now scramble and fight for it, like a foul ball at a baseball game.
No, people are too polite thus far.
All right.
So we have true and false as the type Booleans.
And then we can do operations on them.
So for example, true and false is false, as you might guess.
Finally, there's this funny value, none, which doesn't print anything when I type it.
And if I look at the type of none, we'll see it's the none type.
Not very interesting.
Fundamentally, as we'll see, that gets used when you want to put in something temporary.
When you don't yet know what its value is going to be, you know it's going to eventually have one, so maybe you start out calling it none.
And then you can check, and we'll see how we might do that.
So those are the fundamental scalar types, the indivisible ones.
Interestingly enough, Python does not have what is a common scalar type in every other language called char, short for character.
Instead, what it has is strings that can be used to represent strings of characters.
So for example, I can write the string "a", and if I ask for the type of it, it tells me it's an str, short for string.
Happens to be a string of length 1, which we might usually think of as a character, but there is no type char.
So it's not a problem.
We just have to remember it.
Literals of type string can be written with single quotes or with double quotes.
There's no difference.
Just convenient that you can do it either way, and we can build strings of things.
It's worth noting that the type of, say, the string 123 is str, whereas the type of 123 without the quotes is int.
So we have to be a little bit careful sometimes as to whether we're dealing with strings or ints when we look at these literals.
You can only get so far with literals, things you can type.
So of course, Python has in it something called an expression.
Again, this shouldn't surprise anybody.
And an expression is a sequence of operands and operators.
The operands are objects.
So for example, we can write the expression 3 plus 2.
And when we type an expression into IDLE, it automatically evaluates it and prints the value of the expression.
In this case, of course it's 5.
One thing to be a little careful about is if I type the expression 3/2, slash is the divide operator, I get 1.
Whereas if I type the expression 3.0 divided by 2.0, I get 1.5.
So dividing two integers in Python 2.x gives you essentially a floor operator.
In 3.0, by the way, integer division is not allowed.
It always converts it to floats and does a floating point division.
But for many of you this will be something that will trip you up as a bug.
If you want to get real division, write floating point numbers.
Otherwise, unpleasant things may happen.
Some other interesting things I can type, just as I could type 3 plus 2, I can type a plus b.
What do you think I'll get there?
It does concatenation.
So what we see here is that the operator plus is overloaded.
So overloaded operators have a meaning that depends upon the type of the operands.
And of course, we've already seen that with the slash operator, which means one thing for ints and another things for floats.
And of course, we see the same thing with plus.
What do you think will happen here?
3 blank 3?
Any guesses?
I get a syntax error.
Remember, we talked about that on Tuesday.
It's not a valid Python expression, so we get an error.
How about this one?
That is syntactically valid.
It's got operand, operator, operand.
What do you think it will do when I hit Return here?
Somebody?
A static semantics error?
Pardon?
A static semantics error?
A static semantics error.
And because of these-- Wait, I can't see who said that.
Raise your hand?
Oh, come on.
All the way back there?
All right.
I have the most chance of carrying with one of these.
I'm going to lie.
Those of you who are watching OpenCourseWare, it was a perfect throw.
OK.
So indeed, we get a static semantic error of a particular kind, called the type error, saying you cannot concatenate an str and an int.
Type errors are actually good things.
The language does type checking in order to reduce the probability that a programmer will write a program with a meaning that will surprise its author.
So it looks at it and says, somebody might have a weird guess what this means, but just to be safe, we're going to disallow it rather than-- it could, of course, make up some funny meaning if it wanted to.
But it doesn't.
And I think you'll find type checking saves you from a lot of careless programming errors as you go on.
All right, let's continue.
Let's look at some other things.
I can write this.
Because that's just two strings, and it just concatenates them, the string a and the string 3.
Or interestingly, I can do this.
So now what we're seeing is that you can take any type name, use it as a conversion function to attempt to convert one type to another.
So this has now converted the int 3 to the str "3".
Similarly, I can do something like this.
And here, it's converted the str "3" to the int 3.
On the other hand, I could do this.
And it will tell me it's a static semantic error.
It can't convert 0.0 into an int.
Similarly, it can't convert 2.1.
Or can it?
So now I've given it the float 2.1, and I've tried to convert it to int.
Not the string 2.1, but the float.
And it succeeds.
And it succeeded by essentially truncating it.
Is this a good thing or a bad thing?
To me, it's kind of a bad thing.
If I've typed something like that or I've evaluated some expression that happened to work that way, more likely than not, I'm confused.
And I would probably have preferred to get a type error, rather than it deciding how to do it.
It's one of the things I don't like about Python.
It's too generous.
It lets me get away with stuff it shouldn't let me get away with.
Other languages, for example Java, are much stricter.
This is a design decision and it is the way it is, and we have to live with it
Professor?
Yes?
Is that the same reason that 3 divided by 2 turned into 1 up top?
Yeah.
Exactly.
If it's the same reason that that happens, this will never go that far.
[UNINTELLIGIBLE].
Yeah, exactly.
It's the same reason.
The question was, is it the same reason that 3 divided by 2 doesn't give you the answer you would get with floating point.
And it's because Python has tried to help you.
Again, Python 3.0 is a little stricter about these things.
We'll talk much more about this during the term.
This is close to the last time you'll see me typing things directly into IDLE.
For the most part, as you write programs, you'll use the text editor to produce them and then go to the shell to run them.
But you want to-- obviously, if I had a 100 line program, I wouldn't want to sit here and retype it every time I needed to change it.
So instead, I use the editor in IDLE to produce the programs, and then I can run them.
And that's what I wanted to start doing.
I should probably mention that what most people call a program, some Python programmers call a script.
Think of those two things as synonyms.
But you will see people use both of them.
I will typically call them a program.
All right.
Let's look at an example.
So the first thing to say is that things look a little bit different when they're executed from a script than when you execute them directly in the interpreter.
So I happen to have a script here.
If a line in a script starts with a sharp sign or a number sign, that makes it a comment.
So it's not executed.
So I've started here just by commenting out everything.
But now-- whoops-- what happens if I just put the number 3 here?
We saw when I typed it into IDLE, it echoed it in some sense and gave me what it was.
Or just to be clear, I'm going to put in the expression type of 3.
I'll save it, and then I'll hit F5 to run it.
And it does nothing.
Right?
You saw it move.
It didn't print anything.
So when you type an expression into the shell, it prints the value.
But when it executes a script with an expression, it evaluates the expression but does not display it on the screen.
Well, so what do we do about that?
There is something called a print command.
So I can do this, Print type of 3, and now if I run it, it will actually appear.
So whenever you want to get something to appear, you have to use the Print command.
Not a very complicated concept.
A program, or a script, is a sequence of commands.
Each one tells the interpreter to do something.
So a command is Print, for example.
OK.
So that's there.
That's kind of boring.
I'll get rid of that.
The next command is a really interesting one.
It's an assignment statement.
A key concept in almost every programming language is that of a variable.
Different languages have different notions of what a variable means.
In Python, a variable is simply a name for an object.
And what an assignment statement does in Python, is it binds the name to an object.
So the assignment stetement you see here binds the name x to the object 3.
The next statement rebinds the name x to the value of the expression x times x.
So it takes the old value of x, evaluates the expression, and then binds the name x to the new value.
So at the end of the second statement, x will be bound to 9.
By the way, these are really stupid comments I've written here.
I put them in just to show you what these statements are doing.
For goodness sake, when you write comments in your programs, assume that the reader can read Python, and you don't have to explain the programming language in your comments.
That's not to say you shouldn't write any comments.
The purpose of a comment is to make the program easier to read.
And so typically, comments are there to explain things.
Not to explain the language or its semantics, but to explain your thinking when you wrote the program.
What is the algorithm you've used?
And we'll see some useful examples of comments, probably not today, but later.
All right.
So let's execute this script.
Sure enough, it printed 9.
Just what we would have hoped.
All right.
Now let's try some other things.
Print lets us output things.
Raw input lets us input things.
Get things from the keyboard, essentially.
So this statement here is making a request to whoever is using the program to enter a number.
There are two kinds of input statements in Python 2.x.
There's raw input, which is the only one you will see me use, and input.
Raw input, by the way, is the only one that exists in 3.0.
So please, just use raw input.
The difference is, raw input always expects, interprets what the user types as a string.
So it will see here, it says, y equals float of raw input.
Enter a number.
So let's run it.
So it's taken the argument to raw input, the string enter a number asked me to enter a number.
I'll enter a number.
And then it's converted it to a float.
Suppose I get rid of that.
Suppose I do this.
That should work.
So now something has happened.
It's printed both of them as 3.0.
It looks like they're the same, but in fact, they're not.
And this is something to beware of.
What we've seen here is when it prints a string, it does not print the quotation marks.
So even though, if I were to put this in here, I'll put in two print types of y.
And I'll comment this out because I'm getting kind of tired of seeing 9.
You'll note that one is a string and the other is a float.
Again, I point this out because this is something that can confuse people when they're debugging programs.
Because you think it's a float, when in fact it's a string.
OK.
Nothing deep, but these are the things that sort of get people in trouble.
Now the kinds of programs we've been looking at so far are what are called straight line programs.
What distinguishes a straight line program is it's a sequence of commands you execute one after another.
You execute every command without making any deviations, without going back with any loops to execute a command more than once.
So in a straight line program, every command gets executed exactly once.
There is a very elegant, and even useful theory that talks about different layers of, levels of complexity of programs and says, for example, what kind of functions can you compute with straight line programs.
We'll talk more about that field, which is called complexity theory, later in this semester.
But for now, the thing to realize is that straight line programs are just dead boring.
You can't compute anything interesting with one.
Last time we talked about a recipe as an analogy for a program.
Imagine a recipe with no tests.
So every recipe, or almost every recipe I know, has some decisions.
Taste it and add salt if you need it.
Or poke at the meat and see if it's done.
Or cook it until the thermometer says some degree on it.
Those are the kinds of tests we need to make interesting programs.
The most primitive kind of test we see is what's called a conditional statement.
And those are written using the word if, and optionally as we'll see, the words else or elif, standing for else, if.
So let's look at an example here.
Where'd my mouse, oh there it is.
Yes?
Somebody has a question?
Shout it out
Sorry.
I was wondering, when the user's prompted to put in the raw input, instead of putting in a float, puts in string, could you define it as a floating integer?
How would you interpret that input?
I didn't get the question.
So this is an argument to raw input, or their response to raw input
So yeah, for the raw input where you define it as a quote--
Yeah
It usually puts in a string.
How does Python interpret that?
It will interpret it as a string containing quotation marks

So typically you don't type a string, because it interprets everything you type as if it were a string.
So don't bother typing strings.
Good question.
Thank you.
All right.
So let's look at this.
So here I'm going to get an int, or at least a string.
I'll convert it to an int.
Then I'll say, if x remainder two, that's what the percent sign is, it's a remainder or a mod operator, is equal equal zero.
That's important.
You'll notice that we used an equal sign to do assignments.
If we want to do a comparison, whether two objects have the same value, we don't write a single equal.
We write a double equal.
So whenever you're testing for equality of objects, you use double equal.
So it says, if the object x mod 2 has the same value as the object zero, print even.
Else, print odd.
And then, just for fun, I'm going to see whether or not it's divisible by three.
Why did I do that?
Just to show you that I can nest conditionals inside conditionals.
So in one of the branches of the conditionals, I'm now doing a test.
So what this does, is if comes down, it does the test.
If the value of the test is true, it executes the block of code following the if, in this case, just print.
And then it skips the else.
It does not execute the else.
So it executes one or the other.
If the test is false, it skips the block of code following the if and executes the block of code following the else.
So it does a or b, but not both.
The indentation is important.
Python is very unusual in that the way you indent things actually affects the meaning of them.
And you can tell that, if I were to type this in the editor, you'll note here it's on that line, but if I hit Return, it automatically indents it.
That's the auto indent feature I mentioned earlier in the editor of IDLE.
And this tells me how these things line up.
So the fact that this is here tells me I execute it only as part of the else clause.
The program would mean something quite different if I wrote this.
Then it would mean, if x mod 2 is zero, print even.
Otherwise, print odd.
And whether or not it was even or odd, do this test in the if statement.
So the indentation actually affects the meaning of the program.
Now a lot of other languages, almost all other languages, don't do that.
They have some punctuation.
For example, c uses set braces to designate what's called a block of code.
If you look, however, at a well-written piece of C code, or Java code, or any other language that I know, programmers are trained to use indentation to show the structure of the program.
Even though you don't need, it you could line up everything right at the left edge and just use the punctuation.
People don't do that.
And the reason they don't do that is programs are intended to be read, not just executed.
Why are they intended to be read?
Because the only reason, the only way you can debug a program is reading the code in it.
Typically, you want to write your program so that if you look at it from a distance, the visual structure of the program reflects the semantics of the program.
And that's why people use indentation when they don't need to, so that you can see the structure of the program by looking at it on your screen and not having to parse each symbol.
The authors of Python made what I think is a very good design decision.
They said, well, if that's the way you ought to write your programs, let's force people to write their programs that way and guarantee that the visual structure of the program actually matches the semantic structure.
The problem with languages like C and Java is that you can indent things and fool the reader of the program by making it look like something is under something else, when in fact it really isn't, because of the punctuation.
So here we have a guarantee that the visual structure matches the semantic structure, and I think that was one of the really good design decisions in Python.
OK, people see that?
So we could execute this program.
Let me get back to what it was before.
Control z is the go back.
And now we can enter an integer, say 14, and it will tell us it's even.
I can run it again, and now I'll put 15 in, and it will tell me it's odd.
We'll try it once more.
We'll put in 17.
It was odd and it's not divisible by three.
These kinds of programs are called branching programs.
And that's because the structure of them, as you go down you execute some statements, and then there's a branch which says execute these statements or execute those statements.
And then typically it comes back together and continues.
Of course, branches can have sub-branches.
We could do this and then join further down, as we've seen here.
Now branching programs are much more interesting than straight line programs.
We can do a lot of things with them, but fundamentally nothing really interesting.
And we can think about that by thinking about how long it takes a branching program to run.
So let's first ask the question, how long does it take a straight line program to run?
14 seconds?
No, that's not the way to think about it.
How would we think about how long it takes it to run?
What governs the length of time a straight line program can take?
[INAUDIBLE]
Exactly.
The number of statements or commands in the program.
Since it executes every command exactly once, if you have 100 command, it will have 100 steps in it.
Now there's some variation on how long each step will be.
Some commands might take longer than others, but the length of time it can take to run has nothing to do with its input.
It has to do only with the number of lines of code.
And that tells us it's not very useful because, well, we can only type so many lines in our lifetime.
Well branching programs have the same problem.
In a branching program, each command is executed at most once.
So again, the length of time it takes to execute the program is governed strictly by the size of the program.
Why isn't that good enough?
Well, think about a program, say, to compute the GPA of all the students at MIT.
Well how long is that going to take?
Think instead about a program to compute the GPA of all the students at the University of Michigan, which is probably 10 times bigger than MIT.
Well you would expect that to take longer, right?
Because you have to look at more students.
And in fact, it's true.
Most programs that are interesting, the amount of time they take to run should depend not on the length of the program, but on the size of the data that you want to evaluate using the program.
So you would argue that the amount of time taken to compute the GPA of the students at MIT should be proportional to the number of students, not proportional to the length of the program used to do it.
We'll talk a lot more about that later in the term in a much more thorough way.
But it's important to get that as something you think about.
So the fact that branching programs are not proportional in time to the input means that they're limited in what they can do.
So that gets us to the final concept we need to write every program that could ever be written, or at least to compute every function that could ever be computed.
And that's some sort of a looping construct.
Once we add loops, we get to a class of programming languages or programming constructs that's called Turing Complete.
And I mentioned this last time.
Any program that can be written, or any function that can be computed, rather, can be computed in a Turing Complete language.
So let's look at an example here.
This concept, by the way, is called iteration.
And if we look at languages with iteration, what we'll see is a more complicated flow of control.
You execute some statements, maybe you do some branching if you want.
But then you're allowed to go back and execute statements you've already executed.
Typically what you have is another branch.
One branch goes back and one continues.
So now we see we can execute a statement more than once.
Suddenly we have enormous power at our disposal.
So let's look at an example of that.
By the way, I'm skipping some of the code in your handout, but that's probably fine because it's there for you to be able to read.
And what I would recommend by the way, is that we will post the handouts on the web, but at the end of every lecture within a few hours or a few days at least, go through the handouts and make sure you understand everything in.
Because if you don't, you're probably missing something you'll need to understand to do the problem sets.
So here's a little program that finds the cube root of a perfect cube.
This, by the way, is a useful comment here, right?
Tells you what the program is intended to do.
So we get an integer.
We set the variable ans to zero.
And then while ans times ans times ans is less than the absolute value of x, we're going to set ans to ans plus 1.
We could print where we are.
I put those sort of things in as debugging statements.
If ans times ans times ans is not equal to the absolute value of x when I finish the loop, then I'll print x is not a perfect cube.
Otherwise I have to do something to deal with positive and negative values.
Now I know that this was fast and that most of you probably don't fully assimilate this program.
Do not worry.
It will be discussed in recitations tomorrow.
So tomorrow, the recitations will review the Python concepts we've discussed today, but we'll start by emphasizing how these loops work.
OK.
Thanks for coming.
Enjoy recitation tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I apologize to those of you watching on OpenCourseWare.
I forgot to turn on my microphone.
And you missed some incredibly brilliant things.
But such is life.
Let me go back to where I was, which was we we're looking at this code to find the cube root of a perfect cube.
We saw this last week, and indeed, you also saw it in recitation.
I'm not going to belabor it.
I do want to ask you the question, for what values will this program terminate?
That is to say the only input is to x.
For what values of x is this program guaranteed to always stop?
Anybody want to volunteer an answer to that?
Ask a simpler question.
Let's assume that x is a number.
In fact, let's assume it's an integer.
Will it terminate for all positive integers, all positive values of x?
Yeah.
All negative values?
As far as I can tell.
How about 0?
Yeah.
So in fact, it terminates for all values of x.
How do I know that?
And that's a key question.
I know that because I've used, and I mean by used as a mental tool, something in my head, the notion of a decrementing function.
And every time I write a loop, I think about one of these, because that explains to me why the loop is guaranteed to terminate.
We'll go over here where we have a bigger board and look at the properties that a decrementing function needs to have.
One, it will map some set of program variables to an integer.
Two, when the loop is entered for the first time or when I encountered the test of the loop for the first time, its value is non-negative.
Three, when its value gets to be less than or equal to 0, the loop terminates.
And finally four, it's decreased each time through the loop.
So what we see is if it starts to be a positive value or non-negative, and it's decreased every time I execute the body of a loop, that eventually, it's got to reach 0 or something less than 0.
And when it does, the loop stops.
If such a function exists, then the loop is guaranteed to always terminate.
Now, of course, one can count up to a value instead of down.
But there's always a trick we can use of subtracting to make it the same.
So what's the decrementing function for this loop?
How did I know it will always terminate?
Yeah?
[INAUDIBLE]
Answer equals answer plus 1.
I don't think so.
Does that satisfy all of these properties?
Remember, a function is going to map some set of program variables to an integer.
So what are the interesting program variables here?
Well, there are only two, ans and x.
Right?
At least, that's all I can see.
So what would be an interesting function?
Somebody?
Surely, there's someone who can figure this out.
Yes?
Or no, you're just scratching your head.
You fooled me.
I can't see because of the light, but I'm sure there must be dozens of hands up if I could only see them.
Actually, I don't see any hands up.
This is not so hard, guys.
It's the absolute value of x minus answer cubed.
So what does this value start at?
Let's pick a value.
Suppose that x is equal to 8.
What is the initial value of the decrementing function?
Wow.
Come on.
Let's be a little cooperative, please.
Yes?
So it's 8, answer is 0 and absolute value of x is 8
So it's 8 minus 0, which is equal to 8.
So it satisfies conditions one and conditions two.
What happens to this value every time through the loop?
Does x change?
Does answer change?
And how does it change?
It's increasing.
What does answer start at?
It starts at 0, and it increases.
So I know that answer cubed will always be positive.
Right?
So I know that every time through the loop, it will be 8.
The first time through, it'll be 8 minus 1 cubed, which is 7.
And then the next time through, it'll be 8 minus 2 cubed, which is 0.
And then, I exit the loop.
And it's that kind of reasoning that I used to convince myself that this loop terminates.
And every time I write a loop, and I hope every time you write a loop, you will think about what's the reason the loop is going to terminate.
And you will do it by thinking about what the decrementing function is.
People get me on that?
And whoever finally answered a question surely deserves to be fed.
I obviously have to bring better candy to encourage better responses.
Now, let's go back and look at the program itself.
Now that we know it stops, and you can take my word for it that it actually computes the correct answer, let's think about what kind of algorithm this is.
What's the method?
This is an example of guess and check.
And it's a particular kind called exhaustive enumeration.
Each time through the loop, I'm taking a guess at to what the value is, and I'm checking whether it's true.
But I'm enumerating the guesses in a very systematic way.
They're not just random guesses.
I'm enumerating all the possible answers.
If I get through the entire space of answers, or possible answers, and I don't find a solution, then I know that it doesn't exist, and it's not a perfect cube.
So that's why it's called exhaustive enumeration because I'm exhausting the space of possible answers.
Does that makes sense to everyone?
So let's try it.
Let's try it with say a very large value of x, because that's always an issue, of course, when we do exhaustive enumeration.
So I'm going to enter this value.
Is that a perfect cube?
Who thinks it is?
Who can tell me what it is?
What's the cube root of that?
Well, this is a question I did not expect you to answer.
But it's 1,251.
That'll be in the first quiz, so make a note of it.
Notice how quickly the computer did this.
It found the cube root of quite a large number very quickly.
And so while one might initially think that exhaustive enumeration is a silly technique because takes a lot of guesses, for an awful lot of problems, we can actually just write a pretty stupid program that's solves it by exhaustive enumeration.
We typically refer to such programs as brute force.
And brute force is often exactly the right way to solve a problem.
Why does it work?
Because computers are really fast.
How fast are computers?
Well, today, a good computer can execute in a single processor in the order of 100 million instructions a second.
How fast is that?
And now, we're going to see if Mitchell has answered the question I asked in the way in the class.
How many instructions can a computer execute between the time I say something and the time the people in the back row hear it?
Mitch thinks it's 400 million instructions.
I think that's about right.
It's hundreds of millions at any rate.
It's kind of amazing between the time I say something and the time you hear it, hundreds of millions of instructions.
It's mind boggling how fast that is.
And that's why we can often use these kind of solutions.
Next lecture, actually, even a little bit later in this lecture, I hope to get to an example of why that doesn't really get the job done, at least not always.
Before I do that, I want to look at one more programming construct.
And that's a variant on the while loop.
So if we think about what the while loop we were just looking at did or does, as the decrementing function told us, it's looking at all the possible values of answer ranging from 0 to the absolute value of x.
And at each step testing and doing something, we can abstract this process using something called a for loop.
So let's look at this code.
It's essentially exactly the same algorithm.
I got bored of typing ans times ans times ans.
So I used a Python notation for exponentiation, which you'll see is star, star.
Now, be easier to read if I get rid of that.
But other than that, the interesting thing I did was replace the while loop by a for.
So you'll see this line of code there, for ans in range 0 to abs of x plus 1.
What that says is range is a built-in function of Python that generates, in this case, a sequence of integers, something called a tuple, which we'll be looking at in a lecture or so.
But for now, it's pretty simple to think about what you get is if you look at the expression range of x to y, that gives me a sequence of values, x, x plus 1 up to y minus 1.
Notice not up to y, but up to y minus 1.
So it gives me a sequence of length y-- well, assuming that's 0.
Right?
It doesn't have to be 0.
It can be anything.
It can be another value as well.
0 in my example.
And then, the for loop executes it on this value and the next iteration on this value, and finally, at the very end on that value.
So it executes it one iteration of the loop on each value in this sequence of values generated by the range statement.
And normally, it does all of them.
However, you'll see I've added something called a break here, a command in Python.
And what break says is exit the loop.
So it exits it prematurely without executing all of the values generated by range.
You can nest loops just like you nest if statements.
And if you do that break, always executes-- always exits rather the innermost loop.
So what this does is it's generates a set of values to test.
It checks whether or not it's got the answer.
If it does, it terminates the loop.
And eventually, you exit the loop.
And then, it just checks as before whether or not it found a correct answer and does the right thing.
So you'll find, particularly when you're iterating over integers, but later we'll see when you're iterating over a lot of other kinds of things, for loops are a very convenient shorthand.
There's nothing you can't do with a while loop.
You don't need for loops.
But they do make life easy.
And over the semester, I think you'll end up writing a lot more for loops than you will while loops.
Any questions about this?
If not, I'm going to move right along.
So this is the moment.
I'm going to move right along.
So we've now got a program that does something really silly, really.
It finds cube roots of perfect cubes.
Well, that's not typically useful.
Right?
You've even got these $0.50 four function calculators that find square roots.
And they don't insist that you only give it perfect squares.
So now, let's think about how we would take this kind of program, and indeed, this kind of method of writing programs and use it to find-- for now, we'll look at the square root of any number, of any floating point number.
Well, the first question we need to ask is what do I mean?
That's kind of a subtle question.
What does it mean to find the square root of a number?
What does it mean, for example, to find the square root of 2?
Well, we know that that was an endless series of digits before we can find the square root of 2.
Right?
It does not have a nice answer.
So we can't just say we have to find something that if we multiply it by itself, it will equal 2.
Because we can't find such a thing.
So we've got to think about a different notion of what we mean.
Furthermore, even for some numbers which there is a square root, it might be a million decimal places long, and consequently, really hard to find.
So we need to think about a different kind of concept here.
And it's the concept of an approximation, finding an answer that is good enough.
So what should we do here?
How do we think about this?
Typically, what we do when we think about an approximation is we define how good an approximation we're willing to accept.
So for example, we might want to say, I want to find a square root that lies within epsilon of the true square root.
So find a y such that y times y is equal to what?
What does it mean?
How would I express it within epsilon of the perfect answer?
I don't want to say it's equal to x, because that may be impossible or too time consuming to find.
So really, what I mean is x plus or minus epsilon.
So that's what I'm asking.
Find one that's close enough.
And that's what the next piece of code I want to show you does.
Excuse me.
So I'm starting, just giving it a value for x, so I don't have to keep typing one in.
Let's say it's 25.
I'm going to take epsilon to be 0.01.
So I want it within that distance of the true answer.
I'm going to keep track of the number of guesses here, not because we need it to actually compute the answer, but because I want to then discuss how many iterations of the loop we're doing.
We're going to start by setting my first guess at 0.0.
Again, this is going to be exhaustive enumeration.
Then, I'm going to essentially encode this as a test of my while loop while the absolute value of answer squared minus x is greater than or equal to epsilon, and answer is less than equal to x.
So it's now a more complicated test.
I've got a Boolean value.
Two things have to be true to execute the body of the loop.
I'm going to increment answer by a tiny amount, increment the number of guesses just so I can keep track of it.
Maybe I'm going to comment this out for the first go around just so we don't see too many print statements and keep doing it.
And then when I'm done I'm going to see whether or not what I found is indeed a square root or close enough.
So if we think about why this loop terminates, why am I guaranteed that this loop will terminate?
What's my decrementing function here?
Somebody?
What's the decrementing function?
What am I guaranteed to reduce each time through, and when I get through, I'm done?
Yeah?
[INAUDIBLE] answer squared minus x1 times 1
No.
Close, sort of.
But I appreciate you're trying.
That's worth something just for the effort.
Somebody else.
Remember, if we look at the properties it has to have, it's going to guarantee me that when it gets to the right value, I exit the loop, which suggests it's going to certainly be part of the test of the while.
Just look at this piece over here at the end.
Answer starts at 0.
I keep incrementing it.
Eventually, answer minus x will hit a value, right?
Eventually, I'll get to the point that this condition must be true-- must be false rather.
And then, I exit the loop.
So this piece is not really the key.
It's this piece that guarantees me I'm going to get out eventually.
This piece can get me out sooner.
It's kind of an optimization, if you will.
So I'm just going to go until I find the answer.
Let's see what happens when I run it.
It tells me that 4.99, et cetera is close to the square root of 25.
So there are some things to note about this.
First, it didn't find 5, 25 happens to be a perfect square, yet I didn't find it.
Is that OK?
Yeah.
Because that wasn't what I said.
What I said is find a y that has these properties over here.
And I did.
I didn't say find the y that gets closest to the square root of x. I said find one that has these properties.
Effectively, what this is is a specification of the problem.
And I've now written a piece of code that meets the specification.
It does what I set out to do, and that's good enough.
Now, let's turn this print statement back on.
It took almost 1/2 million guesses to get there.
But it was still blindingly fast.
So once again, exhaustive enumeration seems to be OK.
Suppose, however, I choose a bigger number.
Now, first, let's choose something that doesn't have a good answer.
Let's see what it does for that.
All right.
Pretty good.
Also pretty fast.
Not too many guesses.
But now, let's try this one.
Well, it's going to wait.
It's going to get done, but it's going to take a little bit longer than maybe we want it to take.
Why is it taking so long?
There it is.
It found an answer, which I think is good.
But as you can see, it took quite a few guesses to get there.
So why?
Well, let me first ask this question.
Can we look at the code and anticipate how many guesses it's going to have to take?
We're going back to this issue of computational complexity, but here, not of the problem but of the solution.
So this is algorithmic analysis.
We're analyzing the algorithm, this exhaustive enumeration algorithm, and trying to figure out how long it's likely to take to run.
Well, what does the running time of this algorithm depend upon?
Yeah?
[INAUDIBLE]
It depends on the actual square root.
Yes.
But in particular, the distance of the actual square root from the starting point.
So that's one factor it depends on.
But that's not the only factor.
What else does it depend on?
Oh, do we have an injury?
We had a dropped pass and a deflection there.
All right.
Yes?
It depends on the level of accuracy, so how you define epsilon
It depends upon the value of epsilon.
Absolutely.
How long it takes to run
[INAUDIBLE]
Someone with a concern for safety.
it depends upon the actual value of epsilon, because if epsilon is small, we may have to take more steps to get a precise enough answer.
And it depends upon one more thing.
Yeah?
[INAUDIBLE] the increment that [INAUDIBLE]?
The increment.
Exactly.
Because the number of times we go through the loop is going to be related to how big a step we take each time through.
No applause?
Thank you.
So it depends upon all of these things.
And here, since we're trying to find a pretty [?
big ?]
square root and a sort of precise answer, but we're taking tiny steps, it's going to take a long time to execute.
So we could make it faster.
For example, suppose I change the step size to this.
Plus equal by says increment the value by whatever the right side is.
So I'm going to increment it by 1.
Wow, it was really fast.
But it didn't work.
It failed.
That's not so good.
So I can't just do that.
And of course, it's not surprising, because I ended up jumping all over the answer.
I could make epsilon smaller, but that seems like cheating.
So really, what I need to do is find a better algorithm, a better way to attack the problem.
Fortunately, I don't have to invent it.
Some people a lot smarter than I am figured out a long time ago a good method for solving this kind of problem.
And they're doing it using something called bisection search.
As we look at this particular implementation of it, we're going to use two algorithmic techniques that you'll use over and over again because they're generally useful.
So the first one related to bisection search is we'll cut the search space in half each iteration.
So with my brute force algorithm, we're trimming the search base only a little bit each step.
So if we think about it, what it looks like, we had a space of values to search for the answer.
And I started here.
And each step, I just trimmed off a tiny, tiny little bit, 0.001, leaving me a lot of space to search.
And that's why it took so long.
When I do bisection search, the basic idea is each step I want to cut the search space in half.
Get rid of half of the search space each time.
So one way I could do it is I start with a guess say in the middle.
Just pick some guess that's in the middle of my search space.
And now I say is it too high or too low?
I can easily answer that question.
I square it.
See is my result bigger than the actual square root or smaller than the actual square root?
That tells me whether my guess is too big or too small.
Once I know that, I know which side of the guess the right answer is on.
So if I knew that my guess was too big, then I know there's no point in looking over here for my next guess.
So I can get rid of this whole half in one step.
Now, what should my next guess be?
Yeah?
[INAUDIBLE]
My next guess should be half way through there.
Exactly.
And now, let's say this time my answer is too small.
Then I know I can get rid of this.
So now, I'm very quickly pruning my search space.
If I think about that, how many times am I likely to have to prune it?
It's much faster, right?
As we'll see later, it's basically log base 2.
If I have some number of values to look at-- and by the way, how many values do I have in my search space to start with?
What determines it?
Clearly, the first value and the last value, but also, how small I'm dividing it up.
Right?
So I have to think about that too.
What is the precision with which I do this.
Am I looking at every one millionth of a number or every 0.01?
That will tell me how big it is.
Once I know how big my search space is, I know that if I search it linearly looking at every value, my worst cases, I look at everything until I get to the end.
Well, my best case is I get lucky, and my first guess is right.
But if I use bisection search, my worst case is going to be log number of values in that space.
Because each time, I throw half of them away.
We'll see that in more detail later on.
Let's go back and look at the code now.
So it starts as before with a value for epsilon.
But now, I'm going to take a lower bound, here, and an upper bound on my search space.
I'm going to say my initial guess will be the upper bound plus the lower bound over 2 halfway through my search space.
And then, I'm just going to work my way through it until I get to the answer or don't find it.
So should we look at what's going on here?
Let's try this.
Well, let's first make sure it works for a small value.
Never test your program first on something big.
Always try your program in something small first.
Let's try it on that.
Got an answer.
Notice that it's different from the answer we got last time we looked for the square root of 25.
But that's OK, because it's still meets the specification.
It's still within epsilon of the actual square root.
And I didn't have to say that I wanted it below or the square root or above, just said within epsilon.
Sure enough, different algorithm, different answer, but equally good, but a lot faster.
Now let's try it for the big value.
Wow, that was a lot faster, wasn't it?
It got me an answer.
Probably, not exactly the same answer as before but pretty close.
But it did it in only 26 guesses.
Pretty cool.
And in fact, we'll see over and over again that bisection search is a really good technique for finding quick answers.
And again, why is it 26?
Well, we had some number of guesses to start with.
After 1, it was half as big, then a quarter is big, and eventually, log 2 of the size.
But what was the size?
Was it the number 12345?
No.
We already sort of talked about that.
What was it?
Let's look at the code, and let's think about what was the size of our initial search space.
It's a little bit tricky to think about this, right?
Now, we have to think a little bit harder about when we exit the loop, because in fundamentally, that's telling me the size of the search space.
So what determined the size of the search space?
Well, we talked about the upper and the lower bound.
But what's telling me roughly speaking how many divisions I have?
It's epsilon.
It's not 0.01 because when I square it, it has to be smaller than 0.01.
But that tells me roughly how many I have.
And so it's going to be roughly 12345 divided by 0.01 squared, which turns out to be 26.897 more or less.
So we could predict it.
And son of a gun, when we ran it, we actually matched the prediction.
That's the great thing about algorithmic analysis.
We can actually get accurate guesses as to how long a program is likely to take to run.
This is an important thing because sometimes we do that and we say, oh, it's going to take a year.
I better not even try.
I better find a smarter algorithm.
Or we do it and say, well, it's going to take almost no time at all.
I'm not going to waste my time looking for a smarter algorithm.
I'm going to live with the one I've got.
It's important, and again, as I said, it's a topic we're going to get back to.
Of course, whether it's 26, 27, or even 50 doesn't really matter.
What matters is it's not a billion.
Right?
Because we don't really care small differences.
Whether it takes 25 or it takes 50 will be an imperceptible difference.
It's whether it's a huge difference that matters.
And that's really the kind of things we're going after is orders of magnitude.
Now, I have a question about this program.
I've been obsessing about whether it's fast enough.
And we've shown it is.
But does it work?
Kind of more important.
It's always possible to write a really fast program that gives you the wrong answer.
The problem is to write a fast program that give you the right answer.
Does this program always work?
Well, it worked for 25.
It worked for 12345.
Is that good enough?
Probably not.
We might want to try it in some other values.
I'll ask a simpler question.
Does it always work on positive values?
All right.
I'll give you a hint.
No.
It does not.
I'm not going to, however, tell you why it doesn't, because I want you to think about it.
And I want you to tell me why it doesn't in the next lecture.
But because I'm not a complete sadist, I'll give you a hint.
When we use bisection search, or for that matter, any search method, we are depending upon the fact that the answer lies somewhere in the region we're searching.
If indeed the answer is out here or out here, then it doesn't matter how carefully I search this region.
I'm not going to find the answer.
And so this program doesn't work on some potential values of x because the actual square root of x will not lie in the region that the program is searching.
I leave it to you to think about what such values are.
And we can talk about that on the next lecture.
Suppose I want to use this program to find the cube root.
Suppose it worked, and I want it to use it to find a cube root.
What would I have to change?
How would I change it, so it found cube roots instead of square roots?
Well, I can take it up.
I could use cut and paste, and paste it into my editor and get a new program.
And how would I change that new program to make it do cube roots?
Not very hard.
Think only two places have to get changed.
That's for the simplicity, say cube roots of positive numbers.
I think you said the right thing.
All I have to do is change that two to a three and that two to a three, and I'm done.
And I should probably change the message to say cube root.
Pretty easy.
On the other hand, suppose I also want it to find the fourth root, and the fifth root, and the sixth root, however many roots.
Well, I'm going to get pretty tired of cutting, and pasting, and building a whole bunch of things.
So really, what I want to do is find a way to write the code that will find the nth root of a number for any n. To do that, I'm going to introduce a new programming concept.
And that concept is the function.
And that will be the main topic of Thursday's lecture.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
For those of you who are unaccustomed to seeing it, that blue stuff through the window is called sky.
OK.
I left you last time with a question.
I'd shown you a bisection search implementation, or I should say putative implementation of the square root, and told you that it was flawed, and asked you to think about what was wrong with it.
So first, I'd just like to take a poll.
How many of you know what the problem is, or at least what a problem is?
OK well that's a good start.
And I won't ask how many of you don't know, because I presume that's the rest of you.
So let's look at it now.
I'm not going to ask you the answer, because I want to show people how to find it.
So here's a slightly simplified version of it.
Just get rid of this stuff here, so it doesn't confuse the picture.
So this is roughly the one we looked at on Tuesday, but I just took out some print statements and things and simplified it.
And let's just run it.
And we'll run it with trying to find the square root of 0.5.
See what happens when we run it.
Well, not much happens when we run it.
Actually quite a lot is happening.
We just can't see it.
So this program is running longer than I expected it to.
So if you go to the keyboard and you hit Control and C. It will interrupt the program.
It generates what's called a keyboard interrupt, and it stops it.
And it tells us where it happened to stop it.
It happened to stop it in line seven-- the test of the while loop.
And the problem is, the program just seemed to be running forever.
So despite my, perhaps persuasive, argument last time about the decrementing function, there's clearly a flaw in my logic.
And, in fact, it does not always terminate.
So what do we do about it?
Well this is the main trick.
And one of the things I need you to understand this semester is perhaps the most important thing you'll learn is how to debug programs.
Too many people think the hard part is done when they write the code the first time.
No, that's actually the easy part.
The hard part is actually getting it to work.
So the thing you need to learn is how to debug code, and the nice thing is it's a transferable skill.
It's also fine for debugging lab experiences or family members or anything else that can be seriously awry.
So the way I do it when I program is typically with print statements.
So the fact that the program was running forever, suggests that I'm not exiting the while loop.
So clearly, I need to print something in the while loop.
And the only three variables it seems to be using are answer, low, and high.
Those are the three that change.
So let's see what the value is.
Notice, by the way, that I actually went to the trouble to type ans equals ans, low equals low, et cetera.
A lot of people would just say, OK I'm going to print and they'll ans comma low comma high.
And then when they run the code they'll forget which is which.
The most common problem that people have in debugging programs is that they are lazy.
They think they're saving themselves work, and in fact they're creating work.
So my first piece of advice to you as debuggers is don't be lazy.
Maybe you heard this from your mother at some point in life, or your father.
But now you're hearing it from me.
Try and just do it right the first time.
So let's run it and see what happens now.
Well at least it's printing some output.
And it's chugging away and chug-- uh-oh-- so now, we see we have a real problem.
We've reached a fixed point, where every time through the loop, nothing is changing.
Well the first time we go through the loop and nothing changes we know we're in trouble, because nothing's going to ever change.
And therefore, we're going to be in the loop forever.
So we see, I've gone to a stage where everything equals 0.5.
And now if I go back and look at the code, and I ask myself the question well what happens when everything is 0.5, and I can see the problem.
It's this statement here.
Yep, turn it around maybe.
But that's never going to change now, because it's 0.5 plus 0.5, divided by 2 is 0.5.
And it'll just stay there forever.
So that's my problem.
I have to somehow change my code now, so that this doesn't happen.
So what is the problem?
Now somebody can tell me, simply.
What is the problem here?
What was the flaw in my reasoning when I first set this program up?
Yeah?
[INAUDIBLE]
Louder please?
You don't have a high minus low is less than or equal to epsilon
So the comment was I don't have a high minus low is less than or equal to epsilon.
True, but that's not really the real flaw.
Yeah?
The fraction is greater than the original fraction, so the solution is not in the search space
Exactly.
So the answer is the problem was that I did a search in a region, and the answer wasn't in that region.
Because the square root of 0.5 does not lie between 0 and 0.5.
Silly me, when I thought about it, I didn't think of finding the square root of numbers less than 1.
So what's a simple fix?
Well what I can do is the following: I'll go back and say high is going to be the max of x and 1.
So now I'm going to ensure that the square root actually does lie in the region I'm searching.
I hope.
Let's run it.
Ah.
All right, well I got to some stuff at the end which you shouldn't worry about, but it found something that I guess is a good enough answer.
We'll get rid of that code I put in this morning which we'll get to this later.
OK so I've now fixed the program.
Everyone with me on that?
Any questions?
And the thing to understand is conceptually what was wrong with my reasoning, that I'm doing a search in a region where the answer doesn't lie.
So I'm not going to find it.
And the other thing to understand is my systematic way of finding the bug.
Now I confess I knew the bug was there when I wrote the code, so I kind of cheated with the debugging.
But even if I hadn't known, this is what I would have done.
I would have put in that print statement.
All right, so now we have actually a pretty good piece of code for finding square roots.
And as we looked at on Tuesday, I can use the same piece of code.
I can modify it to get cube roots, or fourth roots, or fifth roots.
And so I have a general framework for doing things.
But it's pretty unsatisfying in that sense, because let's look at it.
If I wanted to find the square root of some number other than 0.5, I have to go and edit the code, replace the assignment to x by whatever I'm trying to do.
If I want to do cube roots I have to cut and paste and edit and do things.
There's no very good way to now embed this piece of code inside a larger computation.
Imagine that I've got some 10,000 line program that needs to find the square root six or seven times, well now I'm going to have six or seven copies of this code in my program, for every time I need the square root.
Clearly not what you want to do.
In general, having more code is a bad thing.
So it's not like you're given an essay to write and someone tells you it's got to be 5,000 words, and you just sweat blood trying to figure out how to stretch it to be that long.
In code, it's the other way around.
Most of the time we want to make it shorter not longer.
And the reason we want to do that is the difficulty of getting code to work grows, maybe even grows quadratically, or worse but the size of the code.
So the more code you have, the harder it is to get it to work.
So one of the things good programmers learn to do is write less code.
And so we don't measure productivity of a programmer by the number of lines of code they produce each day, but we measure it by the amount of functionality they produce each day.
And we give them bonus points if they achieve the desired functionality with less code.
So let's talk about how we can write less code and accomplish more.
Well to do that, we're going to look at a new language mechanism-- actually not new, but new to this class-- called a function.
But before we do that, I want to pull back and talk about what it is we hope to accomplish by introducing functions into our programming language.
We want to provide a mechanism that provides for two things: decomposition and abstraction.
What decomposition does, is it creates structure.
It allows us to break our program up into something called modules.
And the module we'll focus on today is function, but later we'll see there's another important kind of module in Python called the class.
And the advantage of a module is it should be self-contained and reusable.
So it's a self-contained unit of functionality that can be used in multiple contexts.
Abstraction suppresses details.
It allows us to use a piece of code as if it were a black box.
That is, something whose interior details we can't see, don't need to see, and shouldn't even want to see.
We only need to understand what it does, not how it does it.
And that lets us use code that other people have written easily.
And, in fact, use code that we have written easily.
It's one of those few occasions where I think Thomas Gray was right, when he said, "ignorance is bliss."
Sometimes knowing less is better.
All right, so let's look at the way functions work.
The functions let us break code into reusable, coherent pieces.
Now we've already looked at similar kinds of things.
When we looked at say floating point numbers, and we wrote operations like plus or divide, whatever, we didn't worry about how they were actually implemented in the machine.
We said OK they do something, they're kind of like dividing real numbers, let's not worry about the details.
We do that with a lot of things.
We looked at strings.
We concatenated strings.
Well we didn't worry about how did Python go about doing that.
We just assumed it did it, and it had the meaning we wanted it to.
What functions let us do is extend the language in some sense by adding new primitives that we can use just the way we used the built-in primitives.
So let's look at an example here.
I've written a very simple function.
Does something that we actually did already when we looked at square roots.
It's a function called within epsilon.
And let me comment this out while I'm thinking about it, so we don't have to live with it later.
And I now want to walk you, slowly, through what this function does.
So at the start, it uses the keyword Def, short for define.
Following that is a name.
I chose the name within epsilon.
You can choose any name you want for a function.
I'm strongly encourage you to choose mnemonic names, that is to say names that have a meaning.
So in some sense you see it says within epsilon, and you know what it does already.
Following that, it has three things called formal parameters.
I'll come back to in a minute, what that means.
And then after that, it's got something called the function body.
So we see that a function has a name, it has parameters, and it has a body.
The body is the code that's part of the function.
In the body, you can write any code you want.
Plus, there's something you can't write outside of a function, called return.
That's a special command that says whoever calls me has called me to have me compute a value.
I'm going to return the value that this person would want.
And then here we see something that's very important.
This is where we get abstraction, and that's the specification of the function.
And it says here, there are two pieces to it.
One that its parameters x, y, and epsilon, are all floats.
And furthermore, epsilon is greater than 0.
You can imagine this is important, and it returns true if x is within epsilon of y.
Otherwise it will return false.
If I want to use within epsilon, I don't need to look at the code.
I look instead at the specification.
Now here where the code is one line, maybe I haven't gained a lot by looking at the specification instead of the code.
But you can imagine if the code were 1,000 lines, I'd much rather read the specification than the code.
We'll also see for other reasons later why it's in fact dangerous to look at the code.
How do I use it?
I use it by invoking it.
So I could, for example write something like print within epsilon, of two, three, one.
What's it going to print?
Pardon?
Why is it going to print an error do you think?
Because you haven't put epsilon
Ah, typed it wrong.
Thank you.
You're correct.
It would have printed an error.
Now what will it print?
[INAUDIBLE]
Sure enough.
I could also, if I chose, write something like val equals that, then if I want I could print val.
Now it's going to print false.
So within epsilon is just like plus or something else, does some computation, returns a value.
I can use that value any place I could have used an expression.
Now one more thing to look at with this.
Suppose I don't return anything.
Anyone want to guess what it's going to do now?
I point this out, because this is a very common error.
People write lots of code, calculate some wonderful value and then forget to return it.
What's it going to do now?
Well let's run it and see.
That, by the way, is a good habit to get into.
It's going to return the special value none.
Remember we looked at that earlier, meaning I don't have a value.
So if you see in your code some none popping up where you don't expect it to, it's probably because you forgot to return a value.
So just keep that in mind.
All right, now there's a big advantage of this.
Once I've written this code I can now anywhere I want call within epsilon, and I don't have to duplicate the code.
I only do it once.
As I said earlier maybe I'm not gaining much, because the body is so short.
On the other hand, I'm still gaining something.
Notice that when I look at the code down here, it's easy to read, I'm printing within epsilon two, three, and one.
And I don't have to decode this and tell me that that's what that's doing.
So if I have a function and I choose the names properly, code that uses the function is much easier to read.
And that can be a big value.
All right, let's look at another example.
So here I've got this function, f. I've chose a non-mnemonic name, because there isn't much meaning to this function.
What f does, it is a formal parameter x.
It sets x to x plus 1.
Then it prints x and returns x-- pretty boring.
So let's see what it does here.
So now I'm going to set x to three, set z, or zed if you happen to be Canadian, to f of x.
And then print the values of z and x.
Also in f, before I return x, I'm going to print it.
So let's see what happens when I run this one.
It prints four, four, and then three.
All right, what's going on?
Why did it do that?
Well it's pretty easy to see why it printed four here, because I called f of x with an x equal to 3, and then I incremented it by one, and then I printed it.
It's probably also easy to understand why z was four, because I returned the value of x here, which was four and it printed it.
But why is this x three?
And the answer is this x and that x have nothing to do with each other.
Right?
I could just as easily have chosen some other value for the formal parameter, say George, and said George is equal to George plus 1, print George, return George.
There is no relation between the name of the formal and, in this case, x defined in the calling environment.
So now let's think about that by working slowly and carefully through what happens when we call a function.
So the first thing that happens at the call, and I'll just work it through this one, is the formal parameter, x in this case, is bound-- and I'll come back to what binding means, that's a critical concept here-- to the value of the actual parameter.
So these are important terms, actual and formal, which in this case, also happens to be called x.
But what's happening here, is upon entry of a function, a new scope is created.
What's a scope?
A scope is a mapping from names to objects.
So if we look at what's going on over here, we can draw a little picture.
Well before I draw a picture, I'm going to look at a slightly more complicated example.
Well, yeah let's do that.
This one is not in your handout, but it is illustrative of, I think, what's really going on here.
Here I've got another beautifully named function, in this case f1, and inside it, I've defined another function, called g, which takes no arguments.
I've set x to abc.
Then I haven't shown you these assert statements yet, or haven't talked about them.
Assert is a command in which the keyword assert is followed by an expression that evaluates to either true or false.
If it evaluates to true, it does nothing.
It just continues.
If it evaluates to false, it stops your program dead in its tracks.
So I've just used it here as a trick to make my program stop when I run it.
In general, you'll find that I use asserts quite a lot.
So for example, in the next piece of code, which is called find root.
It takes the root, is it square, or cube, whatever, the value, and epsilon.
It assumes that powers, and int, and val, and epsilon float, in the specification.
And then you'll notice, I start by putting in an assertion here.
And what I'm asserting is that the actuals to which these formals are bound, have the properties the specification says they do.
This is what's called defensive programming.
In principle, I shouldn't have to do that, because in principle, nobody should call this with incorrect values.
But, in fact, it can happen.
Programmers occasionally make mistakes.
And so I'm protecting myself by checking that the assumptions are met, and if they're not, my program will just stop.
Then I can go hunt down the fool that called it with the wrong parameters-- probably myself.
So asserts are good for that, and I'll use them a lot for these kinds of things.
I'll also use them when I think I know what value something should be in a program at some point, and I'm not sure it really is.
I'll assert that it has the value I think it is.
I'll assert that x is six, if I think it's going to be six.
And then my program will conveniently stop for me if it's not true.
All right so that's assert.
Other than that, I think there's nothing here you haven't seen before.
So what's going to happen, we're going to step through this piece by piece.
So initially, as we look at it, we enter the main body of the program, which is not wrapped in a function.
So what IDLE will do, or the interpreter will do, is it will start by executing each def.
But executing a def doesn't do anything, but put some names in the environment.
Then it will go and start actually running and interpreting the code that's not nested inside a function.
So the first thing that will happen is the interpreter will build for me what's called the scope.
I've already mentioned, that's a mapping from names to objects.
So in the outermost scope, it will first find the name f1.
F1 it will tell me that f1 maps to an object that happens to be a function.
So it will come over here-- and I'm just going to draw some picture, we'll assume that's the memory of the computer-- and it will map to something that happens to be a bunch of code, if you will.
All right?
It will then stop.
It will then notice that it's got, at the outermost level a variable called x.
And that will map to an integer, which will initially have no value in it.
And then after the assignment, it will now be bound to the object three.
It will then create another object z, but before it can bind a value to it, it will invoke the function f1.
Now the interpreter starts to execute f1.
When it does that, it will create another scope.
So this is the main scope.
It will next create a scope called the f1 scope.
In that, it will have another name g, which will be bound to some code.
It will have a name x, which will be initially bound to the actual.
So in this case, it will be bound to the object three.
We'll then eventually do a print.
It will involve g, which will now create the g scope.
And the g scope will create a name x, which in this case will be bound to the string abc.
It will then start executing g, and it will stop.
So let's see what that looks like.
Sure enough, it got an assert false, gave an assertion error.
What I can do now is go up to this debug here, and go to what's called a stack viewer.
Each of these scopes is what's called a stack frame.
Now why are they called stack frames?
Because when we do it, we begin with the main scope.
We call f, and we get the scope.
f calls g and we get the g scope.
When g completes, which alas it doesn't because of the error, it pops the stack, and gets rid of the g scope.
And now the stack is the f and the main, and then when f completes, it will have just the main.
So it's last in, first out, which is typically called a stack in computing-- or a LIFO, if you're a course 15 major, and do accounting.
So let's look at the stack viewer.
And I apologize for the small type font, but I was unable to make it look bigger.
So it says at the top we've got an assertion error.
And then you'll note it's got three stacks: the main, the f1 stack, and the g stack.
I forgot I called it f1, not f. Then I can inspect them further.
So the g stack has local and global variables.
The local variables include x, which is equal to abc.
Globals we'll get to later.
If I look at f1, it also has a local called x, but its value is now four, not abc.
And it has a value called g, which is a function, as we discussed.
And if I look at main, it has a bunch of things, but it has everything that's available in the interpreter, which because we've looked at within epsilon it's there.
But you'll notice it's x is 3.
All right?
So the stack viewer can be very handy, to look at what you've got, when you've got a bunch of calls.
Now if we go back to our code here, and we'll take this out.
And suppose what we do is we assert false here.
Now if we look at the stack viewer, we see that we have f1 in main, but g is no longer there.
It's gone.
All those variables don't exist anymore, because I'm no longer in g. This is the nice thing, because it means if you call something 1,000 times, it doesn't use up all your memory.
Every time it's finished, it gets rid of what it no longer needs.
All right?
Does that make sense?
This is an important thing to get-- yeah, thank you, question
--the assertion
Where did I--
Where did you put this other assertion, when you just changed--
Ah, where did I put the other assertion?
If we look at the code, you'll see I put it after I call g, and g is by now returned, but before I left f1, which is why the f1 stack is still present.
That makes sense to you?
Which stacks exist, which stack frames exist, depends upon which functions are still active.
Yeah?
How come you don't need a return under the g function?
Oh, because there's not going to be anything interesting.
It's useless.
Right?
Why don't I need a return under g?
Well if I wanted it to do something useful, I would need to return something.
But I'd probably also want to pass it some arguments, rather than have it take no arguments, as well.
So it's here just to be the simplest thing I could put that created a stack frame.
But don't try and interpre it as being anything meaningful.
Yeah?
Would you run into problems assuming that g did something to x and then returned it?
Would you run into any problems that you named the variable the same?
You know, that you used x twice?
Would you want to--
No.
If an x exists, or any variable exists within a function body, when you leave that function, that variable is gone forever.
These are just names.
They have no intrinsic meaning.
So one of the ways to think about it, and we'll see this later when we get to classes-- a lot later.
You could, if you wanted, think about this as really the name g.x, and you could really think of this as the name of f1.x, and you could think of this as the name of main.x, indicating that they're really not the same.
But it would be kind of a pain to write them all that way.
OK?
So different scopes have different names available to them.
You can use the names in the scope, and you have to keep track of what they mean.
OK?
Any other questions?
These are great questions, and I really do appreciate them.
Yeah?
Does this also happen with four loops?
Like if you say 4x in range something, can you use x later?
Or is it x--
You can use x later
Okay so--
x will be available outside the loop.
This was if you said for x in something, is x available outside the loop?
Yes.
And in fact, you'll often want to test what the final value of x is, when you leave the loop.
OK, the next thing on your handout, and I'm not going to go over it, is using functions to implement something that finds roots.
There's no real point in my walking you through this code in class.
I did include it in the handout.
And by the way, the handouts are all available after lecture, online.
Is that, I think you should work through in your own, and make sure you understand it, to get a sense of how functions work.
And it's certainly related to the current problem set, which would be another good reason to work through it-- the problem set that will be posted today-- the new problem set, PS 2.
Note again how careful I am about the specifications.
And I should point out something interesting, if I type find root, open para-- well let's do this here.
Let's clear things up.
Let's get rid of things that will cause the program to halt.
Notice that when I type find root open, open paren, it's given me the values-- the names of the formal parameters, which I've chosen in such way that will remind me what their value should be.
And it's also given me part of the specification, the piece in the triple quotation marks to tell me the rules I'm supposed to be following here on these things.
So it's a very handy thing.
And as you use IDLE, you'll get used to the fact that this is a convenience.
All right, work your way through that code.
Make sure you know what it does.
Finally, today I want to switch gears, and talk about something else.
Up till now, all of the programs we've looked at have been numeric-- they've played with numbers.
And I've done that, because I assumed you guys all had some intuition about numbers.
I've use strings as a primitive data element to print things, but we haven't done anything very interesting with strings.
However strings are indeed quite interesting, in that they're the first non-scalar value we've looked at.
You'll recall non-scalar values are values that can be decomposed.
So if we now look at the code again, I've got this little piece of code called sumDigits.
So before, the for statement we looked at was for x in range.
Well you can apply it to a for statement to any type that has a way to enumerate its elements.
So for c in STR, actually of 1952, so I've taken the number 1952 and converted it to a string so it will now be quote one nine five two, I can now do something to every character in that string.
And what I'm doing is converting it back to an int, and then adding it.
So this will give me the sum of the digits.
17.
This is a very convenient mechanism, and you'll use for a lot, this way.
You'll use it in fact more for this sort of thing then you will for ints.
Now I can also select values.
So if I look at-- I don't know what's going on here.
Every once in while when you go back and forth between the editor and the shell, the shell hangs and you have to go try it again.
If I go to s equals abc, I can look at individual elements of s, for example s sub 0, which will be a. I can also look at slices of s. So for example s from 0 to one.
That's interesting.
What is it doing?
Now try and infer.
I'll give you another example.
So you'll remember when we did range from x to y, it went y minus 1.
Same kind of thing is happening here.
So that's why s from 0 to one gives me only one character, but s from 0 to two gives me the character string ab.
This is what's called slicing, and it's very common.
What a slice does, is it makes a new copy-- makes a new object, in this case-- which is a sub-string of the original string.
There are many other things I can do on strings.
I can do something like s.find, and it will tell me that b is at position number one in s. So use Google, whatever you use, to find the Python web page that describe strings, and it will give you all of the operations you can do.
And they're quite convenient.
One other scalar type that you're going to need for the problem set is tuples, and that will be discussed in recitation tomorrow.
OK, thanks a lot.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Happy Valentine's Day 11.
Actually, maybe it's a little smiley face combined with an 11.
Did any of you leave this here for me?
Or am I just stroking my ego and this was left for someone yesterday?
Stroking my ego, all right.
OK, last lecture, we looked at a program for finding roots and put in a little debugging statement that, along the way, printed various approximations to the root.
Now, suppose that instead of printing things, we actually wanted to collect the approximations.
For example, to be able to go back and look at them later and analyze them, do various kinds of things.
To do this, and this is the sort of thing we do a lot, we need some data structures that can be used for amassing collections of items.
There are 3 data structures in Python that are used to collect items.
I'm going to try and cover all of them today.
Tuples, lists, and dictionaries.
We'll start with tuples and lists.
And what they have in common is they are ordered sequences of objects.
So the key notion here is they're ordered.
It makes sense to talk about the first object, the second object, the last object, et cetera.
When we get to dictionaries, or dicts as they're spelled in Python, we'll see that they're not ordered.
All right, let's look at tuples first.
They're the simplest.
So if we look at it, there's a very simple example at the top.
I have this tuple called test.
And I just said that's the sequence of ints 1, 2, 3, 4, 5.
I can then index into it.
For example, look at the first element, which is the 0th element.
Or I could look at the next element.
And we can print them.
So let's just do that.
And you can see it prints 1 and 2, not surprisingly.
I can print the whole thing if I want.
That lets me look at the entire tuple.
I can also look at this.
And that gives me the last element without my having to know what the last element is.
I can ask about the length of a tuple.
And it tells me it's five.
Similarly, I could write something like this.
Print test.
Why was that out of range?
Yeah?
Because you're indexing from 0?
Because I'm indexing from 0.
So that's why I have this more convenient way of writing minus 1.
Otherwise, I'd have to do len minus one.
Good grab.
OK, let's look at a little example of how we might use this sort of thing.
So here, I've just written a little piece of code that finds divisors.
Going to find all of the divisor of 100, collect them into a tuple.
Notice this kind of funny piece of syntax here, i comma.
I need to do that to say it's a tuple of length one.
Why can't I just write open paren, i, comma, open paren?
Because that would just take the expression, i, and parenthesize it as we often use parentheses for grouping when we write things.
So by inserting this comma, I say, I don't just mean the-- in this case-- say the number i. I mean the tuple of length one.
So it's sort of a special case piece of syntax that you need for tuples of length one.
Then, I can print the divisors.
So let's run that.
And it now prints the tuple for me.
So I've run through.
I've computed all the divisors.
And I've collected them.
Nothing very interesting, but kind of useful.
We can also-- I've shown you how to select elements of tuples.
I can also, if I choose, get what are called slices of tuples.
So a slice gives me a range of values, or in this case, a subsequence of the tuple.
As we'll see, we can also slice lists.
So let's see.
We have a tuple called divisors here.
Yeah, OK, I'll save the source.
Oh, come on!
So if I wanted to do-- what did I call it?
I called it divisors.
So I can do something like divisors 1:3.
And you'll note that gives me those two elements at the appropriate places.
And so it's a very convenient way to take pieces of it.
All right, any questions about tuples?
Not very deep.
Lists are, I think, more useful than tuples and also, alas, more complicated.
And they're complicated because the big difference is that tuples are immutable.
And by that, I mean once you've created a tuple, you cannot change its value.
You can create a new tuple.
but you can't change the value of the old tuple.
In contrast, lists are mutable.
Once you've created a list, you can actually change it.
It's the first mutable type we've looked at because you'll recall assignment didn't actually change the value of an object.
It just changed the object to which an identifier was bound.
Mutability is the first time we've seen a way to actually change the value of an object itself.
And that's, as we'll see, both a powerful concept and an opportunity to mess yourself up by committing serious programming blunders.
All right, so let's look at an example here, first of many we'll be looking at.
So first, we won't worry too much about the mutability.
So here, I'm creating a list called techs, which happens to be, in this case, a list of strings.
Lists need not be homogeneous.
As we'll see, you can mix strings, and floats, and ints.
And most interestingly, you can have lists of list.
And then, another list called ivies.
And then, I'm going to say I'm going to have these univs, yet another list.
This list empty, containing no elements.
Then, I'm going to append techs to ivies.
Notice this syntax-- univs.append.
What that means here is that append is what Python calls a method.
As we'll see when we get to classes, methods play a very important role in Python.
But to a first approximation, it's quite safe to think of a method as an alternative syntax for writing function.
So instead of writing something like append a list and an element, I write l.append the element.
And just think of this l over here as a fancy way of denoting the first argument to the function append, the first actual parameter.
When we get to classes in a few weeks, we'll see why it's highly useful to have this specialized syntax.
But for now, just think of it as a piece of syntactic sugar, if you will.
The thing I want you to think about, though, is this is not equivalent to assigning something to l. This actually mutates the list.
And we say it has a side effect.
Up till now, since we've only been dealing with immutable types, every function we've looked at, its job was to take in a bunch of values, do some computation, return a value.
It didn't change anything.
And then, if we wanted to take advantage of what the function did, we had to assign the value it returned to some variable, and then we could manipulate it.
Or we could print the value it returned.
We had to do something with the value it returned.
Here, we invoke append.
And rather than worrying about what it returns, we're invoking it for the purpose of its side effect-- the modification it performs on the list.
So let's look at what we get when we run this.
So you'll notice what univs is now.
It's a list of length one.
And the one element in it is itself a list because I have appended a list to the end of the empty list-- not the elements of the list, but the list itself, OK?
So it's important to notice the difference between a list that contained the elements MIT and Cal Tech, and a list that contained a list which contains the elements MIT and Cal Tech.
Yes?
Previously, you added two tuples together.
And that's not like appending, right?
Because when I concatenated two tuples, in order to do something useful with that value, I had to assign it to something.
It did not modify anything.
It produced a new tuple which was the value of appending them.
And then, it assigned it to a new tuple.
So it's quite different from append, which is actually having a side effect on the list.
Does not produce a new list, it modifies the old list.
So we can look at this, draw a little picture here.
So we had techs.
And that pointed to a list with two elements in it, which I'll abbreviated as MIT and Cal Tech.
Both of these elements were strings.
Then, I created a new list called univs.
And that was initially bound to the empty list, a list with no elements in it.
I then did an append.
And the effect of the append was to modify univs so that it pointed-- now, I had one element.
And the element it was was this list.
Notice it didn't copy this list.
It actually included the list itself.
So let's look at the ramifications of doing it that way.
Whoops.
So what I'm going to do now is append another element called ivies.
I'm going to then print it.
And then, for e in univs-- so here's kind of a nice thing you can do with lists.
You can iterate over the elements in the list.
So you might think that the way to do that is, well, I'll go for i index in range 0 to length of list.
That would be equivalent.
But it's, in fact, much easier to just write this way-- for e in univs.
That will do something to every element of the list.
I'm going to print what the element is.
So let's look at that.
So now, I have a list, as we see here, of length two.
It contains two lists.
And if I print the elements, I print each of those lists.
Nothing very magical there.
Now, suppose I wanted it flattened.
You asked the question about the tuples where I did concatenation.
Well, I can do the same thing here.
I'll let flat equal techs plus ivies.
And then, I'll print flat.
And you'll note here what concatenation does is it just takes the elements of the list and creates a new list and appends it.
Not append it-- in this case, it assigns it to flat, excuse me.
So that's convenient.
Poor old plus is overloaded with yet another meaning.
All right, let's keep on trucking.
I, of course, can do this myself.
So here, I've got another list called artSchools.
It includes RISD and Harvard.
For u2 in artSchools, if u2 in flat, I'm going to remove it.
All right, so again, I'm going to iterate over everything in artSchools.
What's this going to do, do you think?
What will I get when I print it here?
No new concepts here.
This is all stuff we've seen.
Somebody up there?
Yeah
Flat without Harvard in it?
Yes.
The correct answer was flat without Harvard in it.
Wow, almost a good catch.
Just almost there.
All right, so let's confirm.
Yes, we'll save it.
Thank you.
All right, so we've now removed the art school.
All right, we'll look at one more thing.
Actually, we'll look at far more than one more thing.
But we'll look at one more thing for the moment.
So I'm going to invoke another method.
This is a built-in method of Python that works on sequence types.
And it's called sort.
So you can do-- this will have a side effect on flat.
And it will, as you might guess, put them in order.
So let's run that.
Actually, we'll comment this out for the moment.
So you'll now note that it's put them in alphabetical order.
This is something, again, that you'll find convenient throughout the term, the ability to have the side effect of sorting a list.
Now, I'm going to assign something to flat sub 1.
So let's think about what this is going to be doing.
It's going to replace the first element of flat by a new value.
So it's having, again, a side effect on flat.
So you have to be a little bit careful as you think about this, that I've written something that looks like a conventional assignment statement.
But in fact, flat sub 1 is not an identifier.
So this is not binding the name, flat sub 1, To UMass.
It's actually modifying the object that is the first element of flat
So would an identifier just be a name, flat, for example?
Well, I think the way-- the question is, would the identifier just be "flat?"
No.
Don't think of this as an assignment at all.
Because remember, what an assignment does is swing one of these pointers to point to a different object.
We'll see an example of that.
Whereas here, I'm actually modifying a piece of the object that the identifier points to.
So it's not an assignment in the sense of a re-binding.
It's actually having a side effect of modifying the object.
Let's just run it.
And then, I'll be happy with the question.
So here, you'll see I changed flat sub 1 to now be UMass.
Yeah, question?
As you've drawn on the blackboard here, you've drawn an arrow to the actual object that's techs.
So if we change techs, will we change univs?
Yes.
The question was if we change techs, will we change univs?
Now, in some sense, that's a philosophical question.
From the philosophical point of view, maybe, you could say no.
Univs is still the same object.
So the binding has not been changed.
But the object to which it has been bound is now different.
Same object, but it has a new value, all right?
So this is the key thing to keep in mind.
Assignment has to do with the binding of names to objects.
Mutation has to do with changing the value of objects.
We'll see that pretty graphically in the next example that I wanted to work my way through.
So I'm going to work through a dull example.
But I think it illustrates the points.
And I do, by the way, very much appreciate the questions, even if I forget to throw you candy in return for asking.
But it's good.
If you have questions, please do ask them.
And I'll try and remember to feed you.
So let's work through what this code is going to do.
So first, I'm going to have the list L1.
So the first thing, I'm going to create an object which is a list of length one containing the integer 2.
So L1 will be bound to the list of length one containing the int 2.
I'm then going to create another list, L2, which is going to be of length two.
And the first element will be L1.
And the second element will be L1.
So what will happen if I print L2?
What will I get?
I'll get list two, comma, list two, right?
We can look at it.
If I print L2, excuse me.
All right, just what we would have expected.
Now, I'm going to change the 0th value of L1 to be 3.
So I'm going to mutate L1.
Now, if I print L2, I will get a different value, 3, 3.
Not surprising.
It's something to keep in mind that can be useful, but can also be confusing.
Because if I'm looking at my code, it doesn't look like I changed L2 when I have a side effect on L1.
And so it can be mystifying when you're trying to debug.
You print L2.
You do a bunch of-- execute a bunch of statements, none of which apparently deals with L2.
Then, you print L2 again and get a different value.
This is both the beauty and the peril of mutation.
Now, let's see what this does.
So here, I've now mutated L2 so that its first element is no longer the list, L1, but is now the string a.
And we'll just do a bunch of these all at once here.
Now, I'm going to change L1 to be 2, length one.
So what do you think will happen, by the way, if, after this, I print L2?
Whoops.
What do you think is going to get printed here?
This is important.
You need to figure this out.
What's going to get printed here?
A volunteer, please.
Yes?
it's going to print "a" in the first slot and then L1 in the second slot
Well--
So it's "a" and then "2."
"a" and then "2" is one conjecture.
Let's find out.
"a" and then "3."
Why?
Because what happened here is, when I did the assignment to L1, what that effectively did was swing this pointer to point to the new list containing the element 2, but had no effect on this object.
I was changing the binding of the identifier.
I was not mutating this object.
And so this element still points to the same list, which was not mutated.
That makes sense now?
So you have to get your head around the way all this stuff works.
Anybody have a question about why this is what it did?
All right, if not, we'll just roar right along.
So now, we can do various things.
And we'll get some stuff.
All right, moving right along.
Here's an interesting little program not in your hand out.
Let me get rid of all this cruft.
So here's a function, copylist.
It takes a source list and a destination list.
And for e in the source list, it appends it to whatever the destination list used to be.
And then, I'm just putting in a little print statement so we'll be able to see as it runs what it's doing.
So I'm going to say L1 is equal to the empty list.
L2 is 1, 2, 3.
And I'm going to copylist L2 to L1.
Print L1 and L2.
So what am I going to get when I print those things?
This is the easy question.
Don't tell me I have you so intimidated that you think I'm playing.
This is not a trick question.
Pardon?
1, 2, 3 for both of them?
1, 2, 3 for both of them.
Who said that?
Raise your hand.
Someone back-- oh, good grief, all the way in the back.
All right.
Oh no, not even close!
Off by a row, and about four people, too.
OK. so let's try it.
1, 2, 3, 1, 2, 3.
So exactly as predicted.
Now comes the trick question.
What is this going to do?
Pardon?
Well, we'll print L1 here.
What do you think it'll do when it gets to that print statement?
Will it get to that print statement?
Let me ask that question.
Will it ever get there?
No
No.
Bingo.
Let's run it.
And you'll see LDest just gets longer and longer and longer.
Why is that happening?
Because what it's attempting to do is look at LSource and copy the remaining elements of LSource to LDest.
But every time I go through the loop, what we have is a situation where the formal LSource and LDest are now pointing to the same object.
So every time I modify LDest, I am modifying the object to which LSource points to.
And I keep finding yet another thing to copy.
This is an example of what's called an alias, one object with two names, or in general multiple names.
When you have immutable objects, aliasing is perfectly harmless.
If you have 58 different names for the object 3, it doesn't matter because you can never change what 3 means.
Here, where you have multiple names for the same mutable object, you can get massive confusion when you modify it through one name and then forget that it's being accessed through another.
So it's something to worry about.
As with tuples, you can slice lists.
You can index into lists.
You can concatenate lists.
You can do all the usual things.
I'm not going to list all of the operators.
And there are a lot of very nice operators.
But we'll post readings where you can find what operators are available.
I now want to move on to the third built-in type that collects values, and that's a dictionary.
A dictionary differs from a list in two ways.
One, the elements are not ordered.
And two, more profoundly, the indices need not be integers.
And they're not called indices.
They're called keys.
They can be any immutable type.
So we'll look at a simple example first.
So here, I'm creating a new dict.
We use set braces rather than square braces to remind ourselves that the elements are not ordered.
And I'm saying the first key is the number 1.
And that's bound to the string object "1, 1."
And then the second key is the string object deux, which is bound to the string object 2.
And the third one is the string object pi which is bound to the float 3.14159.
Now I can then index into it.
So for example, if I choose to, I can write something like print d sub pi.
I'll just stop it here with my old trick of asserting false.
And you'll see it will print the value with which the key is bound-- to which the key is bound.
So what we have here is a dict is a set of key value pairs.
And I can access it by looking at the keys.
I can do an assignment, d1 equals d, just as I can with lists.
And now, I remember this is a real assignment.
So now, I have an alias, two identifiers pointing to the same dict.
I can then print d1 sub 1, either print it or I could assign to it.
So let's do that.
First, we'll print it just to show what we get.
Then, I'm going to do an assignment, saying, OK, I want to now change the binding in d of the key one to be uno rather than one.
And we'll get something different.
Let's just run this.
So you'll note, as we would have guessed, with mutability, we see it showing up.
So far, just like lists, the difference being we have key value pairs rather than you could think of a list as being int value pairs.
The indices of a list are always ints.
Associated with each of those ints, we have a value.
Let's look at a more fun example showing what we can do with dictionaries.
So here, I have a very simple dictionary called EtoF, for English to French.
And we can do some things with it.
So I can print it.
And that's kind of interesting.
Let's see what we get when we print it.
So it's printing it.
And you'll notice the order in which it's printed, the key value pairs, is not the order in which I typed them.
That's OK.
The order in which it prints them is not defined by Python.
So there's no way to predict the order.
And that makes sense because, by specification, dictionaries are unordered.
They're sets, not sequences.
I can print keys, EtoF keys, open, close.
Keys is a method on dicts that returns the keys.
And then, just for fun, I'm going to try to print EtoF.keys without the open, close.
So you'll see when I first printed it with the open, closed, I got all of the keys in an order that was not necessarily predictable.
Certainly not the order in which I typed them.
But that's nice.
I now have this sequence of keys that I could do things with.
When I typed it without the open, close, and I wanted you to see this, it just says it's a method, a built-in method.
It does not execute the method, right?
That's fine.
But again, you have to be careful that you'll find this happening when you write code.
You'll forget to write the open, close.
And you'll wonder why your program isn't doing what you expect it to do.
Remember, it's not saying that the method, keys, has no argument.
It has one argument.
In this case, EtoF.
And the open, close is basically saying, all right, call the method rather than the method itself.
Again, this will be very important when we get to classes.
Let's get rid of some of these print statements now.
Suppose I want to delete something.
I can delete something, so delete EtoF sub 1 del is a command, much like print, that says remove something from EtoF.
So I can do that.
And now, something has disappeared.
The key value pair where the key is one is gone.
So that's how I remove things
Question
Yes?
Are the keys of dictionaries mutable?
The keys have to be immutable.
You cannot use a mutable type for a key.
We'll see the reason for that in a couple of lectures when I talk about how dictionaries are implemented in Python.
They use a very clever technique called hashing which would not work if the keys were mutable.
So in order to get an efficient implementation of dictionary look-up, we need to have immutable keys.
And so that's required.
Let's look at another example.
So here, I'm again setting a dictionary.
And now, what I'm going to show you is that I can iterate through the keys and print the values.
For key in d1.keys, nothing very fancy here.
OK?
one equals uno, pi equals 3.1459, and deux equals 2.
All right?
So nothing very dramatic happening here.
Finally, just to illustrate why this sort of thing is particularly useful.
And in fact, you'll find it quite useful in the upcoming problem set.
I'm going to do some translations.
This, by the way, is not the way Google Translate works.
It's a bit more sophisticated.
So I'm going to have two functions, translateword, which, if it finds the word in the dictionary, returns the value associated with the key.
So note, if word in dictionary says is there a key whose value is word?
If so, return the value associated with that key.
Otherwise, don't even try and translate it.
Just leave it the way it was.
And then, translatesentence will set the translation equal to the empty string and wordequals to the empty string.
Then, it'll just collect all the characters until it finds a blank, translate the word using translateword, and append it.
And when it's done, it will print it.
So we can now do these translations quite simply.
What could be easier?
You never again have to learn a foreign language.
Just use Python to do the translations for you.
All right, that's it for today.
Remember, there is a problem set you should be working on and another one that will be posted.
Take care.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
Professor Guttag has to be out of town today.
My name's Professor Grimson.
I'm going to be subbing for him.
And just to show you how much we value education here at MIT, and especially in EECS, you get the incoming chancellor as a substitute teacher.
So how cool is that?
All right, maybe not very cool.
But I'm going to talk to you anyway for the next 50 minutes.
At the end of last lecture, I think Professor Guttag introduced dictionaries to you, a really powerful type.
It's got a great capability, which is it's a tool, a data type that lets you association almost any kind of structure with a key.
So it could be that you're associating keys with numbers, you're associating keys with strings.
But you could be associating keys with other things, including dictionaries.
And I don't know if he showed this to you or not, but those keys themselves can also be really powerful.
It could be just a string.
But the key could also be a tuple, or a list, an x- and y-coordinate, or a set of names, or other things.
So it's a really great data structure to use.
Now, one of the things you could ask, though, is, gee, what if Python or whatever language you're using didn't come with dictionaries?
Could we still get the same power?
And the answer is sure.
So I want to show you a little example.
So on your handout, if we look at this little procedure up here at the top, key search, I could build it out of lists.
I could just use a list as my way of storing it.
So let's look at what this procedure.
Says it says, OK, if I've got a list, and I've got a key, K, I could write a little loop that just walks down the list, saying, is the first part of each element equal to the key I'm looking for?
And if it is, I'll return the second part of that element.
And if I get to the end of the list, I haven't found it, I return none.
So notice I'm making a choice here.
I'm assuming that I'm storing in the dictionary things that are lists too long, key and the associated value, another key and the associated value, another key and the associated value.
But this would work fine.
So if I didn't have dictionaries, I could build it.
If I wanted to make things that had a more complicated lookup, I'd have to make sure that that equality test did it.
But you could see how I might do it.
So the question then can be, so why do we bother with dictionaries, if we could just use it with lists?
Here's my question to you.
So how long in this implementation is it going to take me to figure out if something's in my dictionary?
Oh my god, he's asking questions at 10 o'clock in the morning.
This is really terrifying.
Somebody help me out.
How long?
Yeah?
Probably on average on the order the size of the list
Yeah.
On average, it's going to take me half the size of the list.
But that's the same thing as being the size of the list.
So if that list is really long, I'm toast.
If it's not in the dictionary, in fact, I'm going to have to go all the way through the list before I get to the end to decide it's not there.
So this is not as efficient an implementation as we'd like.
Now, the flip that is you say, OK, how long does it take for the dictionaries, the built-in associated retrieval that you have inside of Python?
And the interesting answer there is that that retrieval is constant.
It takes the same amount of time independent of the size of the dictionary.
And that's wonderful.
We're not going to talk about how today.
You're going to see that later on in the term.
But it is something that drives home, if you like, the point that different data structures have different costs.
And while some are easier to implement in, they may not be as efficient, some may be a little more difficult to implement in, but they are much more efficient.
And that's one of the great things about dictionaries.
The second thing I think you saw at the end of last lecture, and I want to just highlight in a slightly different way, is I believe Professor Guttag showed you a little example of a very simple translation function.
We had a little dictionary.
I'm going to give you a version right here, and un-comment it.
In fact, there's a little dictionary that simply associates English words with French words.
And yes, it's pretty simple, but it's a little way of pairing those things up.
And the idea would be if I have a sentence just a string that consists of a bunch of words, I'd like to translate that English sentence, if I can, into French.
And let's look at the code that's going to do it.
I'm going to, again, un-comment it here.
It's on your handout.
I want to walk you through it.
So first thing I'm going to do is actually write a little procedure the says, given a word in English, I want to find the corresponding word in French.
And that's pretty easy.
It says, if you give me a, word, and you give me a dictionary, I'm just going to look it up in the dictionary.
That's just that associative retrieval.
It does all the work to go into the dictionary, say, if that's a key for something in the dictionary, it's going to give it back to me.
And in particular, I can get it back by simply retrieving it out of the dictionary.
And notice I'm doing this a little more carefuly.
I'm not just directly going, and saying, give me the thing corresponding to the word in the dictionary, because it might not be there.
So that test basically says, if the word is in the dictionary, it'll return true.
And in that case, I get out the corresponding element.
If it's not in the dictionary, I'm just going to be stuck with the English words, so we'll just return the word.
Now, let's look at the more fun part of it.
I now want to translate a sentence.
And the sentence is a string of words.
I need to find where the words are.
I need to look them up.
I need to gather together a translation.
So notice what the heart of this thing does, right down here in this part of the loop that I'm going to highlight right there.
Oops, I mis-did that.
Let me try it again, right there.
What's that doing?
It's walking down the sentence.
A sentence is some big, long string.
And it says for each character C in the sentence, I need to find where the word is.
So I'm going to keep walking along until I find a place where there's a space.
So that first test there is saying, if C is not a space, just add it onto the end of the word.
Oh, yeah, I need to have inititalized word, which I did right up here.
I said, let's set word to just be an empty string.
So I'm just walking along the sentence until I find a space.
Until I do that, I'm just gathering together all of those characters.
So I'm building up the word.
When I get to the place where in fact C is a space, then here's what I'm going to do.
I'm going to translate the word using that procedure I just wrote.
It looks it up in the dictionary.
It either gives me back the French word, if it's there, and if not, it just gives me back the English word.
And, oh yeah, I'm doing a whole long sentence.
So I need to add it together.
So I'm simply going to add it to the end of what I've translated so far.
And I'm doing a slightly funky thing in there.
I'm inserting a space in between, just to keep the words separate.
And of course, I need to have initialized translation up here, which I did, to be an empty string.
So what's this do?
Walks along character by character.
When it gets to a space, it says, I've got a word.
Find the translation, add it to the end of this other internal variable, and keep going.
And I'll do that until I get to the end of that sentence.
And then I'll just return the translation.
Nice little iterative loop.
And it's simply using two pieces to make all of this happen.
I lied too you-- sorry, I didn't lie.
I misspoke to you.
Which is, what's that funky thing at the end?
What's it returning?
It's returning this strange thing this says, "Translation," starting with the first element, a copy of all of that, plus a translation of the word.
So what assumption am I making about the inputs that causes me to do this strange thing at the end?
I'm actually making two assumptions about my input.
But I'm making a particular assumption.
Why do I not just return translation when I'm done?
Why am I doing this last piece?
Anybody help me out?
Boy, I notice everybody sits way back at the back, where I can't make any eye contact.
How do I characterize words in my string?
They have to end with a space.
Ooh, is there a space at the end of my example?
So I guess I haven't shown you an example.
But I'm assuming if I give you a sentence, you don't usually have spaces at the end of sentences.
So I'm making an assumption, is that the words, except for the last one, are characterized by spaces.
So this last little piece here is to get the very last word, and translate it.
Because it won't be a space.
Let's try a couple of examples.
Let's just un-comment these, and see if this does it.
Yes, commenting them again is not a good idea.
We'll go the other direction.
Looks like it did right.
John-- we should have translated John into Jean.
I don't know what the hell Eric is in French, but John mange du pan, Eric boit du vin.
John has that right.
He's eats bread.
I like to drink wine.
And, of course, tout le monde aime 6.00.
Or in this case, everyone likes 6.00.
You just got HASS credit, by the way, for listening to me do French really badly.
OK, as I said, I made an assumption, which that the words end in spaces.
What's the other assumption I made here?
I know you're just looking at the code.
But I made another assumption inside of my code.
And it's always good to figure out what those assumptions are.
Yeah?
That the first word is a noun, so we don't have to translate it?
I didn't assume that.
I could if I was doing a more clever translation.
But I'm actually making no assumptions linguistically here.
I'll give you a hint.
How many spaces do I have between words?
Just one.
I'm sort of building that into this assumption.
It's a reasonable assumption to make.
But it points out, if I wanted to improve the code, I should think about these other cases.
And of course, to build a real translation system, I'd need something much more sophisticated.
So this shows you little example of using the dictionaries.
But I want to use this to lead into the main part of today's lecture, which is, why did I write this separate little procedure up here called translate word?
I could have buried that inside the code.
And the answer is twofold.
First one is it saves a small amount of code.
If I'm going to use it in multiple places, I don't want to rewrite the code.
And that actually is valuable, not just to save you typing.
It means I only have to debug it once.
It's one piece of code.
But the real reason I wanted to introduce it is it gives me what you might think of as modular abstraction.
Those are fancy words that basically say I am isolating that function in one place.
If I decide to change how I'm going to translate words, I don't have to search through all of my code to find all the places where I was using that.
I just need to change the definition of that procedure.
And I agree, in a simple example like this, it looks pretty simple.
But if you've got a million lines of code, you're using it 10,000 different places, you don't want to have to find all the places to change.
So this is an example of modular abstraction, isolating where that thing is.
And that is an example, in fact, of a general problem-solving principle we're going to use, called divide and conquer.
Divide and conquer is basically the idea of taking a hard problem, and breaking it up into some simpler pieces, or into smaller problems, where those smaller problems have two properties.
Small problems are easier to solve than the original one.
More importantly, the solutions to the small problems can easily be combined-- I want to stress easily-- to solve the big problem.
And we're going to look at a bunch of examples today to show that.
This is a really old idea.
Julius Caesar used it.
He did it in Latin.
My Latin is terrible, but it's something like divide et impera, which literally means divide and rule.
That's how he created the Roman Empire.
The British knew this really well.
That's how they control the Indian subcontinent brilliantly for several decades.
Ben Franklin actually knew it.
In particular, he knew how good the British were at this.
So there's a famous quote.
You may well remember-- when he signed the Declaration of Independence, he said, quote, "We must all hang together, or assuredly we will hang separately," meaning divide and conquer is going to be a real problem.
So second HASS credit for you.
We just did some history.
Why are we going to use it today?
Boy, you're a tough audience, I noticed, by the way.
That's all right.
We're going to use it today, because we're going to use it as a tool for problem-solving.
And in particular, we're going to use it with one particular example.
You're going to see divide and conquer later on in the term, when Professor Guttag talks about algorithm design.
Today, I'm going to show you one great technique for doing divide and conquer kind of algorithms.
And that is the technique of recursion.
How many people here have heard that term used before by computer scientists?
OK, not to worry.
If you've heard it, you probably think that it's a really subtle programming technique.
I'll let you in on a secret.
That's a PR job.
It's what computer scientists tell you to make you think that they're much smarter than they really are.
It's not subtle, and it's much more than a programming technique.
And we're going to talk about it.
Now, it gets used not just as a programming technique.
It gets used two other ways in computer science.
And I'm going to talk about both.
It's both a way of describing or defining problems, so it's a way of characterizing a problem independent of how we might implement it.
And it is a way of designing solutions.
So an example of divide and conquer.
Let me give you an example just to show you of why it's a way of defining a problem.
Consider the part of the legal code that defines the notion of a natural-born US citizen, and remind you, to be eligible to run for president, which I hope you all do, you have to be a natural-born US citizen.
Definition has two parts.
Part number one, anyone born within the United States is a natural-born citizen.
Part number two, anyone born outside the United States, both of whose parents are citizens of the United States, is a natural-born citizen, as long as one parent has lived in the US.
A little more complicated.
What does that actually say?
Well, it's got two parts.
And that's what a recursive definition has.
The first part is simple.
You're born here, natural-born US citizen.
Notice the second part.
It says, you weren't born here, you might still be a natural-born citizen.
But what you have to do?
You have to decide if your parents are US citizens-- which they could be by having been born here, but there are other ways to be naturalized-- and that may require you determining if your grandparents are US citizens, which may require you to-- ah.
So you may have to chain down several sets of problems.
Those two parts are what exactly we're going to use when we talk about recursive definitions.
There's what we call a base part, which typically describes the simplest version of the problem.
And then there is an inductive part that tends to describe how you reduce the problem to simpler versions of the same problem.
So in fact, let me write those both down.
We have-- it's called a base case.
This typically gives us a direct answer.
It just tells us very simply, using very simple methods, whether this is something that satisfies that recursive definition.
And there is the recursive or inductive case.
Here, you reduce to a simpler version of the same problem, plus some other simple operations.
OK, again, a bunch of words.
Let me show you some examples.
I'm going to start with three or four different examples, just to show you how quickly we do this.
What I want you to see in all of these examples is that when I describe the problem, I'm describing it in terms of what's the simplest case, and then how do I build solutions to bigger problems from solutions to smaller versions of the same problems?
OK, here's the first case I'm going to do.
Suppose I tell you I want to build a little procedure to do exponentiation, integer exponents.
I want to compute b to the n. But I tell you I've got a really cheap machine, and all I can do is multiplication.
So I can't use exp.
Mathematician-- a course 18 person would say, what's b to the n?
That's b times b times b, n times.
How could I solve this?
Well, recursively, I'd like to say, suppose I could solve smaller versions of the same problem.
And if somebody gave me a solution to that smaller version, how would I build a solution to the bigger problem?
Oh, that's easy.
That's the same as b times b to the n minus 1.
All right, I can see you're all going, well, duh.
I guess that's what chancellors know.
They're not very bright.
Of course, you know that.
But notice what I did.
I've now reduced it to a simpler version of the same problem, recursively.
This basically says I wanted to solve b to the n. It's b times b to the n minus 1.
Oh, yeah, but I've got to figure out when to unwrap this.
So there's an if there.
And that's true as long as n is greater than one-- actually, as long as n is greater than 0.
We'll do it that way.
And if n is equal to 0, I know the answer is just 1.
I know you don't believe me, but that's really cool.
Why is it really cool?
What do I have?
I have a base case.
And if it's equal to 0, I know the answer right away.
If n is bigger than 0, oh, let's assume that I've got somebody who will give me the solution to that smaller problem.
When I get it, I can just use it, do the multiplication, and I'm done.
And so, in fact, here is a simple piece of implementation of that, which I know is not in your handout, but I'm just going to show to you, because I simply want you to see the form of it.
It's an exact Python translation of what I just said.
It says, if I want to take an exponent of b to the n, if n is equal to 0, just return 1.
If not, solve a smaller version of the same problem right there.
Call to the same procedure, same function, but different argument.
And when I get that answer back, just multiply it by b, and return that.
This may look a little funky.
This is the kind of thing that your high school geometry teacher would rap your knuckles with, although they're not allowed to do that anymore.
You don't define things in terms of themselves.
This is not geometry.
This is programming.
And this is a perfectly legal recursive definition of a problem.
It will stop, because it will keep unwinding those recursive calls until it gets down to the base case.
And I'm going to show you an example of that in a second.
I want to show you a much nicer example of recursion.
And again, part of my message here is when you get a problem, don't instantly start writing code.
Think about, how do I break this down recursively?
So I have brought some very high-tech tools with me today.
This is my version of the tower of Hanoi problem.
How many people know the problem of the tower of Hanoi?
Only a few.
OK, so here's the story.
There's a temple in Hanoi staffed by a bunch of monks.
In that temple, there are three tall, jewel-encrusted spikes.
Mine aren't nearly as fancy.
And there are 64 golden disks, all of a different size.
Stack starts out on one of those spikes, and the monks move one disk at a time.
Their goal is to move the entire stack of 64 from one spike to another.
And the rules are they can it of one disk at a time, but they can never cover up a smaller disk with a larger disk.
I have to tell you, I don't know what happens when they move the entire stack.
I mean, the universe ends, or you all get A's in 600, or something equally as cool.
Question is, could we write a piece of code to help the monks, to tell them how to move them?
All right, so let's figure this out.
And I'm going to show you some examples.
So I'm going to move a stack of size one.
Well, that's not very hard.
Watch carefully, because you're going to write code to do this.
I want to move a stack of size two, so I've got to just make sure that I move the bottom one off.
That's not so hard.
Now, I want to move a stack of size three.
I've got to be a little more careful, because I can't cover up the smaller one with a larger one.
But that doesn't look very hard.
Got the solution, right?
Now, we go for stack of size four.
This one definitely takes a little bit more care, because you really can't cover it up.
But as long as you do it right, you can actually move the-- oops, and I didn't do it right.
You've got to move the pieces in the right way.
I do this for taking money off of Harvard students in Harvard Square, by the way.
Got it?
Real easy to see the solution, right?
You could write code for that right now.
I'm not going to do five, but it's really easy to see the solution.
Yeah.
I blew it, too.
I did one move I had to backtrack on.
Let's think about this recursively.
What's the recursive solution?
Break it down into a simpler problem, or a problem of a smaller size.
Ah, here's the solution.
To solve this, I've got a stack, I've got a stack I'm going to, I've got a spare stack.
What's the solution?
You take a stack of size n minus 1, move it onto the spare stack.
Now I've got a simple problem.
I can always move a stack of size one.
And then I move a stack of size n minus 1 to the target.
And, of course, how I move a stack of size n minus 1?
Well, I just unwrap it one more.
That's a really easy explanation, right?
And it's really easy to write code to do exactly that.
So let me show it to you.
Again, I know this isn't in your handout, but I just wanted to see it.
And you could write this yourself.
I'm going to write a little procedure right here called Hanoi.
What are my arguments?
Going to tell how big a stack there is.
That's m. And I'm just going to give it labels, the from stack, the to stack, and the spare stack.
Look how simple the code is.
Says, if it's a stack of size 1, just move it.
I'll just print out the instruction.
Move it from the from stack to the target stack, or the to stack.
If it's bigger than 1, what do I do?
I move a stack of size n minus 1 onto the spare stack.
I move a stack of size 1, which is what's left, onto the target stack.
And then, I move that stack over here that's on the spare stack over to the target stack.
It's what I just showed you right there.
OK?
So let's try it.
Yes, I know I still like French.
We're going to do Hanoi.
Move a stack of size 1.
And we'll just give this some labels, just from, to, and spare.
Well, duh.
You just move it there.
All right.
Let's try a stack of size 2.
It's just what I did.
I'm sure you remember that.
Let's be a little more daring here.
There's the solution to move a stack of size 5.
I'll let you check it separately, make sure it's right.
One of the things you can also see here-- I'm not going to talk about it.
You might think about it, ask your TA in recitation is, how long does it take to solve this problem?
How long is it going to take those monks to actually move a stack of size 64?
I'll give you a hint.
The answer is measured in billions of years.
This is an exponential problem.
And you can see that growth right away.
That's a separate topic.
But notice coming up with that solution on your own, maybe not so easy.
Thinking about it recursively, very easy to think about.
And that's the way we want to look at it.
OK, let me give you another example of breaking a problem down recursively, and then writing the code to do.
I want to decide if a sentence is a palindrome.
Remember what a palindrome is?
It is not an ex-governor from Alaska.
It is a string of characters-- I guess an airport in Alaska-- it's a string of characters that have the property that they read the same front to back.
The most famous one in English-- which is, of course, amusing because it's attributed to a Frenchman-- is Napoleon supposedly saying, "Able was I ere I saw Elba."
Same thing back to front.
How would I write a piece of code to decide if something is a palindrome?
I'm going to do it in a second.
You've got it on the handout.
But let's think about it for a second.
What would the base cases be-- or base case?
Somebody help me out.
What's a good base case here?
What's the shortest possible sentence I could have?
I?
A?
A?
Don't worry about whether it's a legal sense of not.
It might need a verb in there.
Base case is presumably, if I've got a string one character long, it's a palindrome.
If I've got a string zero characters long, it's probably a palindrome as well.
How would I take a longer string and break it down into a simpler version of the same problem to decide if something is a palindrome?
Anybody want to help me out?
Is that a hand up there, or are you just scratching your head?
Well-- yeah?
Maybe take the first and the last element, see if they're equal.
If they are, then cut them off, and--
Yeah.
What's a palindrome?
The easy way to start it is take the things at the end, first and last character.
If they're not the same, it doesn't matter what's happening in the middle.
This thing can't be a palindrome.
So let's check those.
And oh, yeah, if those are the same character, and I pull them off, what do I have?
I have a smaller version of the same problem.
I have a new sentence that's now two characters less.
Do the same thing.
Say, is that a palindrome?
So if these characters match, and that's a palindrome, I'm done.
How do I tell if that's a palindrome?
Check if their two end characters match, and the things in the middle.
So let's look a little piece of code to make this happen.
I'm going to do it in a couple of pieces.
Here's the first piece I'm going to write.
I'm going to walk you through it.
First thing I'm going to do is I'm going to do this outside of the recursive call is I need to convert a string that's put in to make sure that, in fact, it's in a form I want.
So I don't care about the spaces.
Able was I ere I saw Elba.
The spaces aren't in the same place.
That's OK.
It's really the characters.
And I don't really care about capitalization.
So this little procedure basically says, given a string, convert it all into lowercase-- and if you haven't seen that, I'm just importing from a module called string some built-in procedures-- and this one simply takes the string, and makes it all lowercase.
And then what do I do?
Well, just like we did before, I'm just going to walk down that string, gathering together all of the characters.
So this little loop just says, let's initialize ans to be an empty string.
And then for each character in s, if it is a lowercase character-- and that little test simply does that.
It says, if it's in the set of lowercase characters I'm going to add it to the end.
And when I'm done, I'm just going to return ans.
Going to return the answer.
A little procedure.
And, by the way, this is a nice piece of programming style, as well.
I want to separate out things that I want to do once from the things I'm going to call multiple times.
So I don't need to re-check every time that my string is all lowercase, I'm just going to convert it out.
Now, let's look at how do we test this.
Well, it's literally just a translation of what we said.
But let's look at the pieces.
It says, if I give you just a string of characters-- I've gotten rid of the spaces, I've made it all lowercase-- what does it say to do?
I've got to check for the base cases.
And here, I'm actually going to be careful.
We could have discovered this if we programmed.
There actually are two base cases here, which is, is the string of length one, or is of length zero?
Why would I end up with two base cases?
Why don't I just check for a string of length one?
Yeah?
[INAUDIBLE]
Exactly.
I can have an odd or an even number of characters.
So as I'm clipping them off the ends, I might end up with nothing in the middle, or I might have one in the middle exactly.
And we might have discovered it if we tried programming it.
But right, exactly right.
So I can capture that by just saying, if the length is less than or equal to 1, which gets both of those.
In that case, we know the palindrome will return true.
Otherwise, what do we do?
Well, we do what the gentleman over here suggested.
We take the first and the last character of the string-- again, remind you s with an index of minus 1 goes backwards 1, if you like, and gives me the last character.
If those two characters are the same, I've now reduced it to a simpler version of the same problem.
So I simply say, if that's true, and everything else is a palindrome, return true.
Now, if you've not seen this particular little funky form right here, that is taking string s, and saying, give me a copy of everything starting with the first-- which means not the 0th element-- and ending with everything up to, but not including, the last element.
We'll use this to make copies of other kinds of lists.
But that's all that's doing, is saying, give me what's in that string, throwing away the first and the last element.
And that gives me exactly the recursive call I want.
I'm now saying what?
If the first and last character are the same, and if what's left over is itself a palindrome, I'm golden.
Now, let me just wrap all of that up in a little piece here.
I'll un-comment this.
Which is simply going to say, I'm going to print out, or put a comment in it.
And I'm simply going to put the two pieces together.
Given a string, I'm going to convert it into all lowercase characters.
And then I'm going to check, is this thing a palindrome?
So again, let's try some examples.
So we'll pick on Professor Guttag.
His name is almost a palindrome.
Not quite.
So we're going to give him a name change.
It helps if I can type.
Oh, good.
Guttag is not, but Guttug, whatever that means in German, is, because the Gs, the Us, and the Ts all match up.
Oh, and let's see.
Let's try a couple of other ones here.
And I actually typed these in.
We'll check to see if Napoleon really was right when he used his palindrome.
If you can't read this last one, it says, "Are we not drawn onward, we few, drawn onward to--" and I can't read the tail end of that-- "to new era."
And if we try both of those, they are.
What's my point, you're wondering.
I solved this problem by simply breaking it down into simpler versions of the same problem.
That's the tool that you want.
If you were just trying to think about, how do I keep track of my indices as I'm walking along?
I'm going to come in from both ends, so I've got to add and subtract.
And I got to make sure I'm checking things.
You could write a nice iterative loop that would do it.
Actually, I'll take back the word nice.
You could write an iterative loop that would do it.
But it is not crisp, it's not clean, and it's really easy to screw up.
Or you could say, let's take advantage of recursion.
Let's just break it down into a simpler version of the same problem.
You'd get a very nice, simple piece of code.
Now, this may still feel a little mysterious.
I wouldn't blame you if it did.
So let's do the following.
I'm going to comment these out so that we're not constantly looking at them.
And I'm going to show you what happens if we actually look inside of this thing to see what's going on.
So just give me a second here.
We're going to comment all of those out.
And let's build a version of this that just prints out as we go along.
So I'm going to show you right here.
It's the same basic pieces.
And actually, I realized I need to leave is characters around.
Let me go find my is characters part-- two characters part.
Sorry, give me a second here.
Going to need that.
So we'll un-comment that region.
That's just doing the conversion for us.
I think this is going to work.
Let's look at what we're doing here.
This is exactly the same thing.
But I'm just going to put some print statements in it, which as I'm sure you've already heard from Professor Guttag, is a good thing to do as well.
I want you to see what happens.
So the only changes I'm making here are when I come into the thing that's doing the checking, I'm going to print out a little thing that says, what are you calling me with, so you can see how it does the work.
And then, if it's a base case, I'm going to print out a statement that says I'm in the base case.
And otherwise, I'm just going to print out something that says, here's what I'm about to return for the piece of the string I'm looking at.
So it's just instrumenting, if you like, what I'm going to do inside of here.
And the other piece I'm going to do is I'm going to have a little space called indent, so that every time I call a smaller version of the problem, I'm just going to indent over a little bit, so you can see how it unwraps it.
And if we do this, just run that through.
Let's try it.
So if I call is palindrome print with Guttag-- it would really help if I could type, wouldn't it?
Yes, it would really help if I could type.
You're all being really polite, going, ah, he missed a character there.
But we're not going to tell him, because he's going to have to figure it out for himself.
Administrators cannot possibly do this.
Ah, notice what it did.
This is how you can see the recursive unwinding.
It said-- I'm going to check this thing out.
It said way up there-- actually, I'll just do it here.
It said, I'm going to call it initially with Guttag.
Oh, the two Gs worked so that recursively-- and notice the push-in here-- said, I'm calling it again with U-T-T-A.
Oh, that one didn't work.
So I didn't have to check any further, and I pushed it out.
On the other hand, if I do this, ha, you can see all of the stages.
You can see it unwinding it.
It says, to decide if Guttug is a palindrome, well, I check the ends.
And I've got to check U-T-T-U.
I don't know what that is.
Something interesting.
Which says, I got to check that.
Which means I've got a check, oh, there's that base case of the empty one.
I can now return true-- and I'm going to open this up so you can see it-- from the base case, true for that, true for that, true for that.
Let's do one last one, assuming I can type.
I can't.
Well, you get the idea, right?
It's a little messy.
But we go up a little bit, you can start seeing-- I'm obviously just going to run out of those pieces there-- but notice what it's doing.
It's stripping off each of those characters in turn.
You can see how deep this goes in, which is why you've got all those weird spaces there as I keep going in.
But it starts off by saying, look at that, oh, look at that.
I've got to look at-- [GIBBERISH] Which says, I've got to look at [GIBBERISH] You get the idea.
But notice the key thing.
I'm simply reducing it to a simpler version of the same problem.
You can also see that to get the answer out, I've got to stack up a whole bunch of things.
All those indents are basically held operations, which is one of the properties of a recursive procedure.
You have to hold onto intermediate things.
And that can cause, in some languages, some efficiency issues.
But in terms of solving the problem, this is really easy.
I've just broken it down into those simple pieces.
And that's really nice.
So let me finish up with one last example of a recursive procedure.
Again, my goal here is to let you see, what am I doing?
I keep repeating this, but it's important.
To solve a problem, figure out how to break it down into a simpler version of the same problem.
One of the classic examples of recursion is Fibonacci.
How many people here know the Fibonacci numbers?
A few more, good.
For those of you who don't, here's the story.
Probably heard the phrase, "They breed like rabbits."
It's been used to describe a population that the speaker thinks is growing too quickly.
Works with rabbits.
It works if you put two pennies in a drawer.
There's a whole bunch of ways in which this happens.
The history of this is actually very old.
It goes back to 1202, when an Italian mathematician named Leonardo of Pisa, also known as Fibonacci, developed a formula that he thought would help him quantify the notion of how rapidly do rabbits breed.
And here was his model.
His model is not great, and we'll see that in a second.
But here's his model.
You start with a newborn pair of rabbits, one male and one female.
You put them in a pen.
You assume that rabbits are able to mate at the age of one month.
And you further assume that they have a one-month gestation period.
So they can produce offspring at the end of a month.
Finally, let's suppose that these mythical rabbits never die and that the female always produces one new pair-- that is, a male and a female every month from its second month on.
The question is, how many female rabbits will there be at the end of a year?
Hm.
OK, let's see if we can figure this out.
OK, at month 0, when it starts off, there's 1 female rabbit and 1 male rabbit.
At the end of the first month, there's still one female rabbit, but she's now pregnant.
So there's still 1.
At the end of the second month, what do we have?
Well, that initial female is still there.
And she has produced one pair.
So there is a second female.
At the end of the third month, that initial female has again been pregnant and has produced.
And these two are still there.
So there are 3.
At the end of the fourth month, both of these females have now produced offspring.
And those 3 females are now pregnant.
There are 5.
At the end of the fifth month, those 3 females have produced offspring.
Those 5 are now pregnant.
There are 8.
And at the end of the sixth month, there are 13.
What did I just describe?
I just described a nice recursive relationship.
That says, gee, there ought to be a base case.
And there ought to be a recursive case.
And, in fact, in this case, we can describe this very nicely.
The number of females at month end is the number of females there were two months earlier-- because they've all given birth, so that's that many new females-- plus the number of females there were at the previous month, who are now all in whelp.
Ah, that's a recursive relationship.
If I wanted to figure out how many females there are at month end, what do I have to do?
Well, I just have to say, what's the base case?
Because there's the recursive case.
And the base case is-- let me do it this way.
If n is equal to 0 or 1, it's 1.
And as a consequence, I should be able to figure out how quickly do rabbits breed, at least according to 12th century Italian mathematicians.
A couple of things to notice here, by the way.
Sort of similar to what I saw in the towers of Hanoi problem, I'm going to have multiple recursive calls.
To solve this problem of size n over here, I've got to solve two smaller problems, and then do the simple operation of adding them together.
That's OK.
It's going to change the complexity of my algorithm, but it's perfectly fine to have multiple recursive calls.
And as I saw in my case of the palindrome, I may have more than one base case.
That's also OK, as long as I can ground this out.
All right, so let's just look at it.
How would we write this?
Let me get rid of that, comment that out.
We can write a little procedure to compute Fibonacci.
And there's the procedure.
It's in your handout.
I've got some things up at the top here that are just making sure that I got the right kinds of arguments to pass in, which is a cleaner, crisp way of doing it.
But what do I say?
I say, if either x is 0 or x is 1, two base cases, just return the answer, which is 1.
Otherwise, solve two-- right there-- smaller sub-problems.
And what's the simple operation?
Just add the two things together, and return it out.
And if I do this, we can check to see if we actually compute Fibonacci properly.
All right, let's do a test fib of 0.
It says, ah, fib of 0 is 1.
To do a test fib of 1-- there's my other base case.
Aha, it says, there they go.
And now, let's do something a little larger.
There it is.
As it gets each computation it's doing, it says, to get fib of 3, I've got to solve fib of 0, fib of 1, and fib of 2.
And there's the answer.
And just to make sure we're really doing this well, just reproduce my table over there.
Do I expect you to get all of that code right away?
No, although it's pretty easy to understand.
What I do expect you to see, though, is notice what I did.
I took a recursive problem, and broke it down into simpler pieces.
And then I used the solutions of those pieces to give me the solution to the larger problem.
By the way, Leonardo Pisa had it wrong.
In 1859, as I'm sure many of you know, a wonderful Australian farmer named Thomas Austin imported 24 rabbits from England to use as targets for hunting.
10 years later there were 2 million rabbits being shot and trapped in Australia.
And fib, while it grows fast, doesn't go quite that fast.
So Leonardo of Pisa had it wrong.
Of course, as you probably also know, Australia, and I think New Zealand, have the wonderful property of having more sheep than people.
I don't know what that says, other than it's a great recursive problem.
Fibonacci, by the way, actually shows up in other places.
Again, it may not be a perfect model of rabbit populations, but it has some other really interesting properties.
One of them is, if you let n get close to infinity, and you look at the ratio of fib of n over fib of n minus 1-- it sounds like a strange thing to look at.
It basically tells you how quickly does fib grow.
Does anybody know the answer to this?
The golden ratio, 1 plus root 5 over 2, which Leonardo-- the other Leonardo, the really good Leonardo-- by the way, used to design buildings with.
So there's a very strange connection.
The other one thing I like about this is that Fibonacci shows up, actually, a lot in nature.
So, for example, did you know that the number of petals on most flowers is a Fibonacci number?
For example, black-eyed Susans, 13 petals.
Field daisies, 34 petals.
You might ask why.
Oh, sorry.
I'm out of time.
I won't answer it today.
But you can ask Professor Guttag next time around why it is that petals come in Fibonacci numbers.
What's the message of this lecture?
Recursion, divide and conquer.
Break a problem down into simpler versions of the same problem.
Life is really easy.
And you'll see Professor Guttag next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, I want to remind you that there's a quiz one week from today.
Yeah, I know it's soon.
Open book, open notes, no computing or communication devices allowed.
Between now and then, probably tomorrow in fact, or at least over the weekend, I'll send out a summary of what I think we've covered so far and what you'll be responsible for in the quiz.
Roughly speaking, it's anything covered in lectures, problem sets, or recitations.
I will also post some practice questions that you can work on.
And I'll tell you now that we will not be posting answers to the practice questions.
Instead, we'll be holding some quiz reviews.
OK.
I wanted to cover two different topics today.
The first topic is just a tiny bit on floating point numbers in Python.
But in fact, what I'm going to tell you is true about all programming languages-- in fact all, computers really.
And then after that we'll spend most of the lecture on the topic of debugging.
So let me start with a quick review of binary numbers.
Because you have to understand binary numbers to understand floating point.
So when you first learned about numbers, you learned about base 10.
And you learned that a decimal number is represented by some combination of the digits 0 through 9, the rightmost place is the 10 to the 0 place, and then it's the 10 to the 1 place, the 10 to the 2 place, et cetera.
So for example, the number 302 or the digits 3-0-2 represent 3 times 100, plus 0 times 10, plus 2 times 1.
Duh.
All right, binary numbers are exactly the same except we only have two digits to choose from.
Typically written as 0 and 1 and everything is represented by a sequence of those digits.
The rightmost place is 2 to the 0, the next place is 2 to the 1, 2 to the 3, 2 to the 4, et cetera.
So for example, if we look at the binary number 1-0-1, we see that's equal to 1 times 4, plus 0 times 2, plus 1 times 1, or 5.
So one of the first things we'll notice is binary numbers take a lot more digits to represent them, or take more digits than decimal numbers.
In fact, if I give you n digits, n binary digits, how many different binary numbers can I represent with those n digits?
Well, if I gave you n decimal digits, how many different numbers can I represent?
How many different values can I represent?
10 to the n
Pardon?
10 to the n
10 to the n. And so, for a binary number it's going to be 2 to the n. That's important, because we'll see as we get to talking about the complexity of various algorithms how long they take to run, or how much space they use, we'll frequently be resorting to arguments of this sort to understand them.
Now the reason floating point numbers cause problems for programmers is that people have learned to think in base 10.
Computers do everything in base 2, and that causes a cognitive dissonance sometimes.
Where people are thinking one thing, and the computer is doing something slightly different.
So why do people work in base 10?
I don't know.
Maybe it's because we have 10 fingers, but we also have 10 toes.
So why didn't we work in base 20?
We have one head, I don't know why.
But we do it, we work in base 10.
I do know why computers work in base 2.
And that's because it's easy to build switches in electronic hardware.
A switch is some physical device that has only two possible positions, on or off.
We can build very efficient switches in hardware and so it's easy to represent a number as a sequence of on and off bits, which is either on or off.
Originally they were relays, then they became transistors, now they're something altogether different.
But, what they all had in common was they were stable in the off position, they were stable in the on position, and they never had to get in between.
Hence, we represent everything in computers in binary.
So now let's think about why that causes some confusion.
And it does only for fractional numbers.
So for whole numbers binary and decimal it doesn't matter.
Ints are never confusing, they sort of do what God told us integers should do, or whoever told us integers.
All right, but now let's look at other things.
So I want to start by looking at the decimal number 0.125.
What's that as a fraction, by the way?
Happens to be one what?
1/8
1/8, we'll see why that actually matters in a minute.
So, what does it mean, in some sense, in decimal?
It's equal to 1 times 10 to the minus 1 plus 2 times 10 to the minus 2 plus 5 times 10 to the minus 3.
So it works exactly the same way that things work on the other side of, in this case, the decimal point.
Suppose we want to represent it in binary.
So instead of a decimal point, we have a binary point.
What does it look like then?
Well it's equal to what?
1 times-- if it's 1/8, what's it going to be?
1 times what?
1 times 10 to the minus 3
10 to the minus 3.
Or, 0.001.
Right?
So, so far, so good.
Not much difference between the two.
Now let's take a different decimal number.
What about the decimal 0.1?
I have to tell you that it's a decimal because it could also be a binary with just 0's and 1's.
Well, we know how to represent that in decimal.
How about in binary?
What's the equivalent?
Now that's 1/10, of course.
What does 1/10 look like in binary?
Any takers?
Well I'll give you a hint.
It's so long, that I don't want to write it on the board.
In fact, it's worse than long, it's infinite.
I guess that's kind of long.
It's this repeating binary fraction.
There is no finite combination of binary digits that represent the decimal fraction 1/10.
There's no way to do it.
And that's why things get a little hairy.
So we can stop at some finite number of bits.
And in fact that's what happens in the internal representation in Python.
It ends up representing 1/10 as something equivalent to this decimal fraction.
If I take the number of binary bits that are inside the computer, and then I translate it back to decimal, it turns out that it's using this approximation for the decimal fraction 1/10.
So for example, some of you in your problem sets -- where you were computing how much you had to pay on a credit card -- would get answers that were eventually off by a penny or something from what we expected in some, and that has to do with the fact that you were thinking in decimal.
And in fact, you were writing your program in decimal, yet internally things were happening in binary, and when you thought you were writing 1/10 for example you were actually getting something like this inside the computer.
Pretty close to 1/10, but not exactly 1/10.
Now, when we print it, we get yet something else because the print statement uses an internal function that by default rounds these things to 17 digits.
And so you end up getting something like that, or you might depending how you do it.
So let's look at an example here.
So I can do something like this, and it prints that because it's doing some rounding for me.
But if I really look at what's under there, and look at the representation, the REPR function is convenient to get a sense of what's really going on inside, it tells me that well that's a 17-digit approximation.
And now so that's what's really lurking there.
So a hint, If you think something is going funny because of the way arithmetic is working, instead of just using print, you can use print of REPR to get a better idea about what's really going on.
All right, now, does this matter?
Usually it doesn't.
Most of the time it's safe just to pretend that floating points work the way you learned about arithmetic when you were in third grade, or probably in kindergarten if you were educated in Europe or Asia.
But now let's look at an example where you can get in trouble.
So I've got a little program here.
I initialize x to 0, then I'm going to go through a loop a lot of times, where I increment x by 1/10.
And then I'm going to print x.
And because it's going to do automatic rounding, it's going to print 10,000-- or actually, it should print 100,000, right?
No 10,000, because I'm only incrementing it by 1/10, excuse me.
But then I'm going to print REPR of x, and then I'm going to do a comparison.
Now if floating point arithmetic worked the way reals work, we would think that 10.0 times x should equal the number of iterations.
Because I'm starting at 0, each time I'm incrementing it by 1/10, and so if I multiply the result by 10 at the end, I should get the same as the number of iterations.
Does that make sense to everybody?
That's what you would normally get if you did this with pencil and paper.
Of course, it would take you a really long time to do 100,000 increments.
Let's give it a shot.
And what we'll see is that if I print it, it looks OK, it's 1,000.
But if I print REPR of it, I see it's 10,000, a bunch of 0's, and then 18848.
And, of course, consequently when I compare it, I get something that says false.
And that's because if I look at REPR of 10.0 times x-- well, that's interesting, what's going on here?
It kind of looks like the same thing, doesn't it?
But it's not, because way out there are some other digits we're not seeing, something different is happening.
OK, what's the moral of this?
It's not complicated.
It's not, OK write your programs thinking deeply about what's going on in those bits way out there at the end.
It's, don't ever test whether to floating numbers are equal to each other.
Instead, do something like this.
Define a function called 'close', or whatever you want, that takes two floats and some epsilon.
And I've given here epsilon a default value.
And then just return whether the absolute value of x minus y is less than epsilon.
So whenever you're comparing two floating numbers, the question shouldn't be are they identical, but are they close enough for your purposes.
And if you do that, then you don't get tripped up by this kind of rounding and things like that.
Not a complicated story, but keeping this in mind will get you out of trouble when you're doing floating point arithmetic.
Let's run this, and see what happens.
And indeed, they're not equal but they're good enough, close enough if you will.
OK. One of the dangers, the reason this went wrong, is these little differences can accumulate if you go through a lot of iterations.
Sometimes they balance out, sometimes it rounds up, sometimes it rounds down, but not always.
So very simple answer.
Just don't get caught up in this problem of floating point numbers.
All right, any questions about that?
All right, Yes
Doesn't it change for Python 2.7?
It's only returning 0.1 and not 0.100000
In 2.7?
Yeah
Don't know, sorry.
But the moral remains the same.
Whatever is going on, don't test floating point numbers for quality because you'll have a high probability of getting false, when you should get true.
OK. You almost never get true when you should get false.
I now want to move on if there are no more questions to debugging.
I never know when to give this lecture in the term.
So what I usually do is I wait until the volume of email, and complaints, and office hours builds, and I realized people are ready to learn more about debugging.
If I do it too early, people don't pay any attention because they don't realize it's a problem.
And if I do it too late, they get irritated with me because they say well why didn't you tell me this earlier in the semester when it would've done me some good.
So, I pick a time.
And right now it looks like the need has built up enough that it's worth doing.
There's a very charming urban legend about how the process of fixing flaws in software came to be known as debugging.
It's one of those stories that's so nice that you just want it to be true.
So let's look at this story, because it's fun.
All right, what you see on the screen now is a photo of a book now at the Smithsonian Museum, of the lab book from the group working on the Mark II Aiken Relay computer at Harvard University.
Pardon?
Oh, I see it on my screen, now you see it on your screen.
Thank you.
So there it is.
It was September 9, 1947, even before I was born, it was that long ago.
And so you can see that they're running their computer, and they started to do an arctan computation, and it's kind of interesting that they started it at 8 o'clock in the morning, and it ran for two hours, and then it stopped.
Wow, to do an arctan.
Tells you something about how fast this computer was.
Then it went on.
Then they started the cosine tape, and started to do a multiple adder, and then something bad happened.
It stopped working.
Whoops.
All right, hold on a second.
And they spent a long time trying to find out why it stopped working.
And then they found out the problem.
They found a moth stuck between one of the relays.
So it had electromechanical relays for their switches, the on and off.
And they were debugging, they didn't call it debugging.
And they found the software had failed because the hardware had failed, and the hardware had failed because a bug had been stuck in one of the relays.
They debugged it, as in removed the moth, and the program ran to successful completion.
And as you can see, the comment was written in this book, first actual case of a bug being found.
Hence, we call it debugging.
This was, by the way, Grace Murray Hopper's lab book.
She is often described as the first programmer.
It's unclear if that's true.
What is true, she was the first female Admiral in the US Navy.
She was a Navy programmer who eventually rose to the rank of Admiral.
So it's a charming story that this is why we call it debugging.
Turns out it's not at all true.
That the phrase debugging had been used for a long time, and could easily be traced back to the 1800s when people were writing books about electronics and talking about debugging even in those days.
And in fact, you can go back to Shakespeare who talks about a bugbear, meaning something causing needless exercise, needless or excessive fear or anxiety.
Well that's a good description of a bug.
And he actually called it a bug when he had Hamlet [UNINTELLIGIBLE] about to bugs and goblins in my life.
All right, so I want to start now-- oh by the way, just for fun, this is what the Mark II looked like.
This was the computer the took an hour or so to do an arctan.
You see it filled-- made it's a little hard to see in this light-- but you can see it filled an entire room.
Quite amazing.
And, here's a picture of Admiral Hopper and some unidentified mail.
All right, if anyone knows who this it would be good to know so I can update my archives.
All right, so now on to some practical aspects of debugging.
The first thing I want to do is dispel some myths about debugging.
There is this myth that bugs crawl unbidden into our programs.
That we write perfect programs and somehow a bug just sneaks in, and ruins perfection.
That's not true.
In fact, if there's a bug in your program, you put it there.
So it would be almost better not to call it a bug, which sort of sounds like it's not our fault, but it's a mistake, it's a screw up.
So get that through your head.
Similarly bugs do not breed in programs.
If there are multiple bugs in your program, it's not because a couple of them got together and procreated, it's because you made a lot of mistakes.
Keep that in mind.
With that in mind, we should think about what the goal of debugging-- and it's not to eliminate one bug quickly, it is to move towards a bug-free program.
And I say this because they're not always the same strategy that you would follow for these different goals.
And I also carefully say to move towards a bug-free program because in truth be told we are hardly ever sure that we have no bugs left.
Debugging is a learned skill.
Don't despair.
Nobody does it well instinctively.
Evolution did not train us to be debuggers.
So a large part, probably the largest part in many ways, of learning to be a good programmer is learning to debug.
And what that has to do is thinking systematically and efficiently about how to move towards a bug-free program.
The good news is that it's not hard to learn, and it is a largely transferable skill.
The same skills you use to debug software, can be used to debug laboratory experiments.
I actually give lectures sometimes to physicians about how to debug patients.
How to use debugging techniques to find out what's wrong with people when they're sick.
It's a very good and useful life skill.
Now for four decades, maybe five decades, people have been building tools called the debuggers.
And you'll find that built into IDOL there is a debugger that are designed to help people find out why their programs don't work, and fix them.
Personally, I almost never use one.
The tools are not that important.
What's important is the skill of the craftsman, in this case.
And in fact, most of the experienced programmers I know rely on print statements.
So it's OK to use a debugger but I think the best debugging tool is print.
And I have to say I've been surprised-- that's a mild word here-- at how few print statements you guys seem to use.
I get these emails, or the staff gets these emails, kind of plaintiff, why doesn't my program work?
And then there's a little piece of code.
And the answer I send back-- when I reply before one of the TA's do, and they usually get there first-- is usually, put in a print statement here and see what happens.
And I'm just amazed that when the code arrives it doesn't have these statements in it.
My favorite response, was I sent an email to a student, who shall go nameless, and he-- or maybe it was a she-- and I said, insert a print statement here and see what happens.
And I got back to reply saying, no I don't need a print statement here I know what the value of this variable is.
Well, you know, my reply was that if all the values were what you thought they were, you wouldn't be sending an email saying, why doesn't my program work.
Put the darn print statement and see what happens.
And then I got a gracious email back saying, more or less, oops, I see.
But please, when you send us some code, you want some help, send us code with some print statements already in it to at least show us that you've tried to find the bug yourself.
All right, so what we're essentially doing when we insert print statements in a code is searching for the place in our program where things have gone awry.
And the key to being a good debugger is to be systematic in this search.
So you saw that when we looked at algorithms for things like exhaustive enumeration.
We said, well if we're searching for an answer, we have to search the space carefully one at a time.
And then we said, if we want to search it efficiently, maybe instead of starting at the beginning and just going to the end, we should use something like binary search.
The same techniques can be used when you're searching for bugs.
So I recommend searching for bugs using some approximation to binary search.
And we'll see an example of this as we go forward, but as we look at the example what I want you to think about is what are we searching for?
We know our program doesn't work.
So the question that I like to ask, is not why didn't it produce the answer I wanted it to?
But, how could it have done what it had done?
This is a subtly different question.
And it's usually a much easier question to answer.
Not why didn't it do the right thing, but here it did something.
So I already know what it did.
And I say, I didn't expect it to do that, so why did it do that?
Once I know why it did what it did, it's usually pretty easy to think how to fix it.
So that's the first question I ask.
I then go about it using something akin to the scientific method, which we all learned about many years ago.
And basically the scientific method is based upon studying available data.
The data you have is of course the program text itself, the test results, you ran some tests and got the wrong answer which is why you knew you had a bug.
And then you can probe it, you can change the test results by using print statements so that you have more data to study.
Keep in mind that you don't understand this program, because if you did it would work.
Once I study this, I form a hypothesis that at least I think is consistent with the data.
And then I go and design and run a repeatable experiment.
And I want to emphasize the word repeatable, here.
And again the key thing as with the scientific method, the experiment to be useful must have the potential to refute the hypothesis.
Why might repeatability to be an issue?
Well, as we'll see pretty soon, a lot of programs involve randomness.
Where you're doing something equivalent to flipping a coin, somewhere in the program which might come up heads or tails, and the program would do different things.
We'll see why that's an important programming techniques soon.
And once you do that, you can get different results with different runs.
More subtly there can be various kinds of timing errors deep down in the operating system where you have multiple activities going on at the same time.
This is usually the reason that you'll see say, Windows crash, or Word, or PowerPoint, or something else.
Because there's some timing error that occurs sometimes.
And probably most commonly, because there's human input.
Somebody typed something and they might type something different.
So one of the things you want to do when you're systematic is make sure that you can replay things.
And we'll talk more about this when we get to randomness, about how we go about doing that.
All right, now let's try and put this all together in a little program.
If you've been studying your handout, as at least one of the TA's did, you've been kind of mystified by the fact that there's a pretty crummy looking program in it.
And unlike sometimes when I make mistakes I don't know I've made, here I intentionally made some mistakes.
So let's look at this program.
I wrote a function called is_palindrome that takes in a list and is intended to return true if the list is a palindrome and false otherwise.
Then I wrote this little program called Silly that uses isPal, takes in a number, requests that the user make that many inputs, then calls isPal to find out whether or not the resultant list is a palindrome.
Not too complicated.
But now let's run it.
Do Silly of 'five'.
And it tells me 'abcde' is a palindrome.
All right, I have a bug.
Now I need to go try and find that bug.
So the first thing I need to think about when I'm looking for it is to try and find a smaller piece of input that will produce the bug.
So I want to find small input on which program fails.
Why do I want to find a smaller input?
Well, a in this case it's less typing, b if it's a real program it's probably less execution time to make it run, but c it'll be easier to debug because there are fewer kinds of problems.
So let me try it on a small piece of input say, Silly of 1.
Oh, it gets that right.
So that's no good.
Let me try something else, let's try Silly of 2, I'm sort of sneaking up.
It gets that one wrong.
All right, so I know I can test it on a small input.
So that's a good thing.
I now have a simple test.
Now in this case the code is so short, and so stupid, that you could probably look at it with your eyes and just find the bug instantly.
But the point of this exercise is not to find the bug, but to kind of show the process.
So now I wanted to go through this process of binary search to try and find the bug.
So we'll start with Silly, the top level program, and I'll look for something about halfway through, maybe here.
And try and now answer the question, that I've got a lot of code and I'm going to find a point halfway through it and try and ask is the bug above this, or below this.
So I need to find some intermediate value I can check.
And at this point in the program the only thing I have done is accumulate the input, right?
So there's nothing else to ask.
So my hypothesis is that everything is good and that the input will be 'ab'.
So let's try it.
Let's print result here every time through and see if we get what we wanted to get.
All right, that's not what I expected.
So something is wrong.
What's wrong?
Why is result always the empty list?
I can out-wait you
Because whenever it goes through the for loop it keeps coming back
Right.
So every time through the for loop, it's reinitializing-- whoa, got you.
For those of you watching on TV, I just hit a person that was heads down with a piece of candy.
Fortunately it was not a hard candy.
All right, so you're right.
Let's get that out of there.
Put it where it belongs.
Run it again.
OK, are we happy with that result?
Yeah, because I've done that before the append, right?
And now just to be sure, we'll take this print statement out here and let's put it here.
We're now searching elsewhere.
Well the good news is I now have the right result for the value of the variable, but the wrong result for the program.
It's still telling me it's a palindrome.
So the moral here is there is no such thing as the bug.
Never use the definitive article.
There is a bug.
There's a story that I've heard related to this, as far as finding a bug.
You can imagine that you're at someone's house for dinner, you're sitting at the dining room table, you can't see the kitchen, and suddenly you hear from the kitchen, [BAM].
What the heck's that?
Your hostess walks out and says, don't worry I just killed the cockroach on the turkey.
Well, your immediate reaction is the cockroach on the turkey?
Where there's one, there's likely to be more.
Every time you found a bug-- the more bugs you find, then probably the more bugs there are still left, because you've shown that you make a lot of mistakes.
All right, onward we go.
So what do we do next?
Well, we now know at least that things look OK to this point, which suggests that the problem must come below this in the program.
Well the only thing that's going on below this is the call to isPal.
So now we'll say OK, we've now isolated the bug to isPal.
That's a good thing.
Let's try and ask where things are going on there.
So we'll take a point halfway through isPal, and we'll print some things here.
So let's print-- see what we have here.
But before I do that, I've gotten really tired of typing 'a' and 'b', so I'm going to use something called a test driver, or a test harness.
And I recommend that you do this kind of thing whenever you're testing a program.
Write some code that has nothing to do with the program itself but makes it easier to test and debug the program.
The pretentious word for this is a test harness.
All this is code that helps testing.
One of the things that you see in industry is about half the code that gets written is not intended to be delivered as part of the final product, but is there merely for the purpose of testing and debugging.
It's a big deal.
So don't feel bad that you're writing code that's not part of the solution to the problem set that is there only to help you make your code work.
It seems like it's extra work, but in fact, it will save you work.
So let's call it.
We'll call isPal.
And it's going to print some things that I think it should do.
In fact, we'll look at what it does first before we look at the print statements in isPal.
So for the moment, let me just comment these out.
And what we see here is it should print false, and it prints true.
Well, should it print false the second time?
No, right.
So it should have printed true, and it did.
So this is an important lesson.
Make sure that when you put in these debugging statements, you write down as part of the print statement what you expect it to print.
So that when you look at your output you can quickly scan it and see whether the program is behaving as you thought it would.
So now, works once doesn't work the other time.
So we'll go back and turn on the print statements up here and see what we get.
So it's printed temp as 1-2-1 and x as 1-2-1.
So kind of OK that print and x are the same, we expected that.
But we thought we reversed it.
We've entered 1-2-1 and it is this.
What's going on?
What's wrong?
Well now what we can do, is let's see where it went wrong.
We'll put in another print statement here, see what value is there.
Well it was 1-2-1 before reverse, and it's 1-2-1 after reverse.
How come?
Why isn't reverse reversing temp?
Do you need parenthesis after reverse?
Exactly, I need parenthesis after reverse.
Whoa, close.
Because without the parentheses, all reverse is doing is nothing.
That's just the name of the method, not an invocation of the method, right?
All right, now let's run it.
Good news and bad news.
What's the good news?
It has indeed reversed 1-2 right, to make it 2-1 but it's also reversed x.
So naturally, since it's reversed x temp and x will be the same, and I get the wrong answer.
What's wrong now?
Yeah?
So, I think you're aliasing
I'm aliasing?
And it's reversing--
Because now remember how mutation works, now temp and x both point to the same object.
If I reverse the object, it doesn't matter whether I get to it through x or I get to through temp it will still have been reversed.
So in this case, what I'd need to do is this, clone it.
And now when I run my code, it works.
No applause?
All right, a couple more things about debugging next Tuesday, and then we'll move on to some pretty interesting topics in the next phase of the course.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, we're moving on to what will be a major unit of the course, which is the topic of efficiency.
Thus far, we focused our attention on the admittedly more important problem, getting our programs to work, i.e., to do what we want them to do.
For the next several lectures, I want to talk about how do we get them to work quickly enough to be useful.
It is in practice often a very important consideration in designing programs.
The goal is not to make you an expert in this topic.
It's hard to be an expert in this topic.
I'm certainly not an expert.
But I want to give you some intuition about how to approach the question of efficiency, how to understand why some programs take much longer to run than others, and how to go about writing programs that will finish before you die.
And we'll see that if you write things wrong, the programs could, in principle, run longer than you can.
So why is efficiency so important?
Earlier in the term, I started to spend some time talking about how really fast computers are and showing you that we can use brute force algorithms to solve fairly large problems.
The difficulty is that some of the computational problems we're confronted with are not fairly large but enormous.
So for example, in my research group where we work at the intersection of computer science and medicine, we have a big database of roughly a billion and half heart beats.
And we routinely run computations that run for two weeks on that data.
And the only reason they complete it in two weeks and not two years is we were really careful about efficiency.
So it really can matter.
And increasingly, it matters is we see the scale of problems growing.
The thing I want you to take home to remember is that efficiency is rarely about clever coding.
It's not about some little trick that saves one instruction here or two instructions there.
It's really about choosing the right algorithm.
So the take home message is that efficiency is about algorithms, not about coding details.
Clever algorithms are hard to invent.
A successful computer scientist might invent maybe one in his or her whole career.
I have to say I invented zero important algorithms in my whole career.
Therefore, we don't depend upon being able to do that.
Instead what we depend upon is problem reducing.
When confronted with a problem, we want to reduce it to a previously solved problem.
And this is really often the key to taking some problem and fitting into a useful computation.
We sit back, say, well, this looks a little bit like this other problem.
How come I transform my problem to match a problem that some clever person already knows how to solve?
Before I spend time on problem reduction, however, I want to draw back and look at the general question of how do we think about efficiency.
When we think about it, we think about it in two dimensions, space and time.
And as we'll see later in the term, we can often trade one for the other.
We can make a program run faster by using more memory or use less memory at the cost of making it run more slowly.
For now, and the next few lectures, I'm going to focus on time.
Because really, that's mostly what people worry about these days when they're dealing with complexity.
So now, suppose I ask you the question, how long does some algorithm implemented by a program take to run?
How would you go about answering that question?
Well, you could say, all right, I'm going to run it on some computer on some input and time it.
Look at my watch.
That took three minutes.
I ran this other algorithm, and it took two minutes.
It's a better algorithm.
Well, that would be really a bad way to look at it.
The reasons we don't think about computational complexity, and that's really what people call this topic in terms of how long a program takes to run on a particular computer, and it's not a stable measure.
To do that, it's influenced by the speed of the machine.
So a program that took 1 minute on my computer might take 30 seconds on yours.
It has to do with the cleverness of the Python implementation.
Maybe I have a better implementation of Python than you do, so my programs will run a little bit faster.
But most importantly, the reason we don't depend upon running programs is it depends upon the input.
So I might choose one input for which the program took 2 minutes and another seemingly similar input in which it took 1 hour.
So I need to get some way to talk about it more abstractly.
The way we do that is by counting the number of basic steps.
So we define some function, say time, which maps the natural numbers to the natural numbers.
The first n, in this case, the first natural number, the argument corresponds to the size of the input, how big an input do we want to run the program on.
And the result of the function is the number of steps that the computation will take for an input of that size.
I'll come back to this in a little bit more precise detail momentarily.
A step is an operation that takes constant time.
And that's important.
So steps are not variable, but they're constant.
So we have lots of these, for example, an assignment, a comparison, an array access, et cetera.
In looking at computational complexity in this course, we're going to use a model of the computer.
It's known as random access, a random access machine, frequently abbreviated as RAM.
In a random access machine, instructions are executed one after another, that is to say they're sequential.
Only one thing happens at a time.
And we assume constant time required to access memory.
So we can access at random any object in memory in the same amount of time as any other object.
In the early days of computers, this model was not accurate, because memory was often say, a tape.
And if you wanted to read something at the end of the tape, it took a lot longer to read than something at the beginning of the tape.
In modern computers, it's also not quite accurate.
Modern computers have what's called a memory hierarchy where you have levels of memory, the level one cache, the level two cache, the actual memory.
And it can differ by say a factor of a 100, how long it takes access data depending upon whether it's in the cache.
The cache keeps track of recently accessed objects.
Nevertheless, if we start going into that level of detail, we end up losing the forest for the trees.
So almost everybody when they actually try and analyze algorithms typically works with this model.
We also know in modern computers that some things happen in parallel.
But again, for most of us, these will be second order effects.
And the random access model is quite good for understanding algorithms.
Now when we think about how long an algorithm will take to run, there are several different ways we could look at it.
We could think of the best case.
And as we think about these things, as a concrete example, we can think about linear search.
So let's say we have an algorithm that's using linear search.
We've looked at that before to find out whether or not an element is in the list.
Well, the best case would be that the first element is three, and I'm searching for 3, and I find it right away, and I stop.
So that would be my best case complexity.
It's the minimum running time over all possible inputs.
Is the best case.
I can also look at the worst case.
What's the worst case for linear search?
It's not there.
Exactly.
So I go and I have to look at every element, and whoops, it's not there.
So the worst case is the maximum over all possible inputs of a given size.
The size here is the length of the list.
And then I can ask what's the expected or average case, what would happen most of the time.
The expected case seems, in principle, like the one we should care about.
But the truth is when we do algorithmic analysis, we almost never deal with the expected case because it's too hard.
We think about the expected case for say linear search, we can't talk about it without some detailed model of what the list itself looks like, what elements are in it, and what the distribution of queries looks like.
Are we most of the time asking for elements that are not in the list in which case the expected value is out here somewhere?
Or are we most of the time looking for things that are in the list in which case the expected value would be somewhere near halfway through the length of the list?
We don't know those things.
We have a tough time modeling expected value.
And one of the things we know is that frequently we don't -- when we release a program -- have a good sense of how people will actually use it.
And so we don't usually focus on that.
Similarly, we don't usually focus on the best case.
It would be nice.
But you could imagine that it's not really what we care about, what happens when we get really lucky.
Because we all believe in Murphy's law.
If something bad can happen, it will.
And that's why complexity analysis almost always focuses on the worst case.
What the worst case does is it provides an upper bound.
How bad can things possibly get?
What's the worst that can happen?
And that's nice because it means that there are no surprises.
You say the worst that this thing can do is look at every element of the list once.
And so if I know that the list is a million elements, I know, OK, it might have to do a million comparisons.
But it won't have to do any more than a million.
And so I won't be suddenly surprised that it takes overnight to run.
Alas, the worst case happens often.
We do frequently end up asking whether something is in a list, and it's not.
So even though it seems pessimistic to worry about the worst case, it is the right one to worry about.
All right.
Let's look at an example.
So I've got a little function here, f. You can see it here.
It's on the handout as well.
First of all, what mathematical function is f computing, just to force you to look at it for a minute?
What's it computing?
Somebody?
It is a function that should be familiar to almost all of you.
Nobody?
Pardon
Exponentiation
Exponentiation?
Don't think so.
But I appreciate you're trying.
It's worth some candy, not a lot of candy, but a little candy.
Yeah?
Factorial
Factorial.
Exactly.
It's computing factorial.
Great grab.
So let's think about how long this will take to run in terms of the number of steps.
Well, the first thing it does is it executes an assertion.
And for the sake of argument for the moment, we can assume that most instructions in Python will take one step.
Then, it does an assignment, so that's two steps.
Then, it goes through the loop.
Each time through the loop, it executes three steps, the test at the start of the loop and the two instructions inside the loop.
How many times does it go through the loop?
Somebody?
Right.
n times.
So it will be 2 plus 3 times n. And then it executes a return statement at the end.
So if I want to write down the function that characterizes the algorithm implemented by this code, I say it's 2 plus 3 times n plus 1.
Well, I could do that.
But it would be kind of silly.
Let's say n equals 3,000.
Well, if n equals 3,000, this tells me that it takes 9,000-- well, what does it take?
9,003 steps.
Right.
Well, do I care whether it's 9,000 or 9,003?
I don't really.
So in fact, when I look at complexity, I tend to-- I don't tend to I do ignore additive constants.
So the fact that there's a 2 here and a 1 here doesn't really matter.
So I say, well, if we're trying to characterize this algorithm, let's ignore those.
Because what I really care about is growth with respect to size.
How does the running time grow as the size of the input grows?
We can even go further.
Do I actually care whether it's 3,000 or 9,000?
Well, I might.
I might care whether a program take say 3 hours to run or 9 hours to run.
But in fact, as it gets bigger, and we really care about this as things get bigger, I probably don't care that much.
If I told you this was going to take 3,000 years or 9,000 years, you wouldn't care.
Or probably, even if I told you it was going to take 3,000 days or 9,000 days, you'd say, well, it's too long anyway.
So typically, we even ignore multiplicative constants and use a model of asymptotic growth that talks about how the complexity grows as you reach the limit of the sizes of the inputs.
This is typically done using a notation we call big O notation written as a single O.
So if I write order n, O(n), what this says is this algorithm, the complexity, the time grows linearly with n. Doesn't say whether it's 3 times n or 2 times n. It's linear in n is what this says.
Well, why do we call it big O?
Well, some people think it's because, oh my God, this program will never end.
But in fact, no.
This notion was introduced to computer science by Donald Knuth.
And he chose the Greek letter omicron because it was used in the 19th century by people developing calculus.
We don't typically write omicron because it's harder to types.
So we usually use the capital Latin letter O, hence, life gets simple.
What this does is it gives us an upper bound for the asymptotic growth of the function.
So formerly, we would write something like f of x, where f is some function of the input x, is order, let's say x squared.
That would say it's quadratic in the size of x.
Formally what this means is that the function f-- I should probably write this down-- the function f grows no faster than the quadratic polynomial x squared.
So let's look at what this means.
I wrote a little program that talks about some of the-- I should say probably most popular values we see.
So some of the most popular orders we would write down, we often see order 1.
And what that means is constant.
The time required is independent of the size of the input.
It doesn't say it's one step.
But it's independent of the input.
It's constant.
We often see order log n, logarithmic growth.
Order n, linear.
One we'll see later this week is nlog(n).
This is called log linear.
And we'll see why that occurs surprisingly often.
Order n to the c where c is some constant, this is polynomial.
A common polynomial would be squared as in quadratic.
And then, if we're terribly unlucky, you run into things that are order c to the n exponential in the size of the input.
To give you an idea of what these classes actually mean, I wrote a little program that produces some plots.
Don't worry about what the code looks like.
In a few weeks, you'll be able to write such programs yourself.
Not only will you be able to, you'll be forced to.
So I'm just going to run this and produce some plots showing different orders of growth.
All right.
This is producing these blocks.
Excuse me.
I see.
So let's look at the plots.
So here, I've plotted linear growth versus logarithmic growth.
And as you can see, it's quite a difference.
If we can manage to get a logarithmic algorithm, it grows much more slowly than a linear algorithm.
And we saw this when we looked at the graded advantage of binary search as opposed to linear search.
Actually, this is linear versus log linear.
What happened to figure one?
Well, we'll come back to it.
So you'll see here that log linear is much worse than linear.
So this factor of nlog(n) actually makes a considerable difference in running time.
Now, I'm going to compare a log linear to quadratic, a small degree polynomial.
As you can see, it almost looks like log linear is not growing at all.
So as bad as log linear looked when we compared it to linear, we see that compared to quadratic, it's pretty great.
And what this tells us is that in practice, even a quadratic algorithm is often impractically slow, and we really can't use them.
And so in practice, we worked very hard to avoid even quadratic, which somehow doesn't seem like it should be so bad.
But in fact, as you can see, it gets bad quickly.
Yeah, this was the log versus linear, not surprising.
And now, if we look at quadratic versus exponential, we can see hardly anything.
And that's because exponential is growing so quickly.
So instead, what we're going to do is I'm going to plot the y-axis logarithmically just so we can actually see something.
And as you can see on input of size 1,000, an exponential algorithm is roughly order 10 to the 286th.
That's an unimaginably large number.
Right?
I don't know what it compares to, the number of atoms in the universe, or something ridiculous, or maybe more.
But we can't possibly think of running an algorithm that's going to take this long.
It's just not even conceivable.
So exponential, we sort of throw up our hands and say we're dead.
We can't do it.
And so nobody uses exponential algorithms for everything, yet for anything.
Yet as we'll see, there are problems that we care about that, in principle, can only be solved by exponential algorithms.
So what do we do?
As we'll see, well, we usually don't try and solve those problems.
We try and solve some approximation to those problems.
Or we use some other tricks to say, well, we know the worst case will be terrible, but here's how we're going to avoid the worst case.
We'll see a lot of that towards the end of the term.
The moral is try not to do anything that's worse than log linear if you possibly can.
Now some truth in advertising, some caveats.
If I look at my definition of what big O means, I said it grows no faster than.
So in principle, I could say, well, what the heck, I'll just write 2 to the x here.
And it's still true.
It's not faster than that.
It's not what we actually want to do.
What we actually want is a type bound.
We'd like to say it's no faster than this, but it's no slower than this either, to try and characterize the worst cases precisely as we can.
Formally speaking, a theorist used something called big Theta notation for this.
They write a theta instead of an O.
However, most of the time in practice, when somebody writes something like f of x is order x squared, what they mean is the worst case is really about x squared.
And that's the way we're going to use it here.
We're not going to try and get too formal.
We're going to do what people actually do in practice when they talk about complexity.
All right, let's look at another example now.
Here, I've written factorial recursively.
Didn't even try to disguise what it was.
So let's think about how we would analyze the complexity of this.
Well, we know that we can ignore the first two lines of code because those are just the additives pieces.
We don't care about that-- the first line, and the if, and the return.
So what's the piece we care about?
We care about the number of times the factorial is called.
In the first implementation of factorial, we cared about the number of iterations of a loop.
Now instead of using a loop, you use recursion to do more or less the same thing.
And so we care about the number of times fact is called.
How many times will that happen?
Well, let's think about why I know this doesn't run forever, because that's always the way we really think about complexity in some sense.
I know it doesn't run forever because each time I call factorial, I call it on a number one smaller than the number before.
So how many times can I do that if I start with a number n?
n times, right?
So once again, it's order n. So the interesting thing we see here is that essentially, I've given you the same algorithm recursively and iteratively.
Not surprisingly, even though I've coded it differently, it's the same complexity.
Now in practice, the recursive one might take a little longer to run, because there's a certain overhead to function calls that we don't have with while loops.
But we don't actually care about that.
Its overhead is one of those multiplicative constants I said we're going to ignore.
And in fact, it's a very small multiplicative constant.
It really doesn't make much of a difference.
So how do I decide whether to use recursion or iteration has nothing to do with efficiency, it's whichever is more convenient to code.
In this case, I kind of like the fact that recursive factorial is a little neater.
So that's what I would use and not worry about the efficiency.
All right.
Let's look at another example.
How about g?
What's the complexity of g?
Well, I can ignore the first statement.
But now I've got two nested loops.
How do I go and think about this?
The way I do it is I start by finding the inner loop.
How many times do I go through the inner loop?
I go through the inner loop n times, right?
So it executes the inner for statement is going to be order n. The next question I ask is how many times do I start the inner loop up again?
That's also order n times.
So what's the complexity of this?
Somebody?
n-squared
Yes.
I think I heard the right answer.
It's order n-squared.
Because I execute the inner loop n times, or each time around is n, then I multiply it by n because I'm doing the outer loop n times.
So the inner loop is order n-squared.
That makes sense?
So typically, whenever I have nested loops, I have to do this kind of reasoning.
Same thing if I have recursion inside a loop or nested recursions.
I start at the inside and work my way out is the way I do the analysis.
Let's look at another example.
It's kind of a different take.
How about h?
What's the complexity of h?
First of all, what's h doing?
Kind of always a good way to start.
What is answer going to be?
Yeah?
The sum of the [UNINTELLIGIBLE]
Right.
Exactly.
It's going to be the sum of the digits.
Spring training is already under way, so sum of the digits.
And what's the complexity?
Well, we can analyze it.
Right away, we know we can ignore everything except the loop.
So how many times do I go through this loop?
It depends upon the number of digits in the string representation of the int, right?
Now, if I were careless, I would write something like order n, where n is the number of digits in s. But really, I'm not allowed to do that.
Why not?
Because I have to express the complexity in terms of the inputs to the program.
And s is not an input.
s is a local variable.
So somehow, I'm going to have to express the complexity, not in terms of s, but in terms of what?
x.
So that's no go.
So what is in terms of x?
How many digits?
Yeah?
Is it constant?
It's not constant.
No.
Because I'll have more digits in a billion than I will in four.
Right.
Log -- in this case, base 10 of x.
The number of decimal digits required to express an integer is the log of the magnitude of that integer.
You think about we looked at binary numbers and decimal numbers last lecture, that was exactly what we were doing.
So that's the way I have to express this.
Now, what's the moral here?
The thing I really care about is not that this is how you talk about the number of digits in an int.
What I care about is that you always have to be very careful.
People often think that they're done when they write something like order n. But they're not until they tell you what n means, Because that can be pretty subtle.
Order x would have been wrong because it's not the magnitude of the integer x.
It's this is what controls the growth.
So whenever you're looking at complexity, you have to be very careful what you mean, what the variables are.
This is particularly true now when you look at functions say with multiple inputs.
Ok. Let's look at some more examples.
So we've looked before at search.
So this is code you've seen before, really.
Here's a linear search and a binary search.
And in fact, informally, we've looked at the complexity of these things before.
And we can run them, and we can see how they will grow.
But it won't surprise you.
So if we look at the linear search-- whoops, I'm printing the values here, just shows it works.
But now, the binary search, what we're going to look at is how it grows.
This is exactly the search we looked at before.
And the thing I want you to notice, and as we looked at before, we saw it was logarithmic is that as the size of the list grows, doubles, I only need one more step to do the search.
This is the beauty of a logarithmic algorithm.
So as I go from a 100, which takes 7 steps, to 200, it only takes 8.
1,600 takes 11.
And when I'm all the way up to some very big number, and I'm not even sure what that number is, it took 23 steps.
But very slow growth.
So that's a good thing.
What's the order of this?
It's order n where n is what?
Order log n where n is what?
Let's just try and write it carefully.
Well, it's order length of the list.
We don't care what the actual members of the list are.
Now, that's an interesting question to ask.
Let's look at the code for a minute.
Is that a valid assumption?
Well, it seems to be when we look at my test.
But let's look at what I'm doing here.
So a couple of things I want to point out.
One is I used a very common trick when dealing with these kinds of algorithms.
You'll notice that I have something called bsearch and something called search.
All search does is called bsearch.
Why did I even bother with search?
Why didn't I just with my code down here call bsearch with some initial values?
The answer is really, I started with this search.
And a user of search shouldn't have to worry that I got clever and went from this linear search to a binary search or maybe some more complex search yet.
I need to have a consistent interface for the search function.
And the interface is what it looks like to the caller.
And it says when I call it, it just takes a list and an element.
It shouldn't have to take the high bound and the lower bound as arguments.
Because that really is not intrinsic to the meaning of search.
So I typically will organize my program by having this search look exactly like this search to a caller.
And then, it does whatever it needs to do to call binary search.
So that's usually the way you do these things.
It's very common with recursive algorithms, various things where you need some initial value that's only there for the initial call, things like that.
Let me finish-- wanted to point out is the use of this global variable.
So you'll notice down here, I define something called NumCalls.
Remember we talked about scopes.
So this is now an identifier that exists in the outermost scope of the program.
Then in bsearch, I used it.
But I said I'm going to use this global variable, this variable declared outside the scope of bsearch inside bsearch.
So it's this statement that tells me not to create a new local variable here but to use the one in the outer scope.
This is normally considered poor programming practice.
Global variables can often lead to very confusing programs.
Occasionally, they're useful.
Here it's pretty useful because I'm just trying to keep track of a number of times this thing is called.
And so I don't want a new variable generated each time it's instantiated.
Now, you had a question?
Yeah?
Just checking.
The order len L, the size of the list is the order of which search?
That's the order of the linear search.
The order of the binary search is order log base 2 of L-- sorry, not of L, right?
Doesn't make sense to take the log of a list of the length of the list.
Typically, we don't bother writing base 2.
If it's logarithmic, it doesn't really very much matter what the base is.
You'll still get that very slow growth.
Log base 10, log base 2, not that much difference.
So we typically just write log.
All right.
People with me?
Now, for this to be true, or in fact, even if we go look at the linear search, there's kind of an assumption.
I'm assuming that I can extract the elements from a list and compare them to a value in constant time.
Because remember, my model of computation says that every step takes the same amount of time, roughly.
And if I now look say a binary search, you'll see I'm doing something that apparently looks a little bit complicated up here.
I am looking at L of low and comparing it to e, and L of high and comparing it to e. How do I know that's constant time?
Maybe it takes me order length of list time to extract the last element.
So I've got to be very careful when I look at complexity, not to think I only have to look at the complexity of the program itself, that is to say, in this case, the number of recursive calls, but is there something that it's doing inside this function that might be more complex than I think.
As it happens, in this case, this rather complicated expression can be done in constant time.
And that will be the first topic of the lecture on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
In the example we looked at, we had a list of ints.
That's actually quite easy to do in constant time.
If you think about it, an int is always going to occupy the same amount of space, roughly speaking, either 32 or 64 bits, depending upon how big an int the language wants to support.
So let's just, for the sake of argument, assume an int occupies four units of memory.
And I don't care what a unit is.
Is a unit 8 bits, 16 bits?
It doesn't matter.
4 units.
How would we get to the i-th element of the list?
What is the location in memory of L-th of i?
Well, if we know the location of the start of the list-- and certainly we can know that because our identifier, say L in this case, will point to the start of the list-- then it's simply going to be the start plus 4 times i.
My list looks like this.
I point to the start.
The first element is here.
So, that's start plus 4 times 0.
Makes perfect sense.
The second element is here.
So, that's going to be start plus 4 times 1.
Sure enough, this would be location 4, relative to the start of the list, et cetera.
This is a very conventional way to implement lists.
But what does its correctness depend upon?
It depends upon the fact that each element of the list is of the same size.
In this case, it's 4.
But I don't care if it's 4.
If it's 2, it's 2 times i.
If it's 58, it's 58 times i.
It doesn't matter.
But what matters is that each element is the same size.
So this trick would work for accessing elements of lists of floats, lists of ints, anything that's of fixed size.
But that's not the way lists are in Python.
In Python, I can have a list that contains ints, and floats, and strings, and other lists, and dicts, almost anything.
So, in Python, it's not this nice picture where the lists are all homogeneous.
In many languages they are, by the way.
And those languages would implement it exactly as I've outlined it on the board here.
But what about languages where they're not, like Python?
One possibility-- and this is probably the oldest way that people used to implement lists-- is the notion of a linked list.
These were used way back in the 1960s, when Lisp was first invented.
And, effectively, there, what you do is a list.
Every element of the list is a pointer to the next element.
And then the value.
So what it looks like in memory is we have the list.
And this points to the next element, which maybe has a much bigger value field.
But that's OK.
This points to the next element.
Let's say this one, maybe, is a tiny value field.
And then at the end of the list, I might write none, saying there is no next element.
Or nil, in Lisp speak.
But what's the cost here of accessing the nth element of the list, of the i-th element of the list?
Somebody?
How many steps does it take to find element i?
i
i?
i steps, exactly.
So for a linked list, finding the i-th element is order i.
That's not very good.
That won't help me with binary search.
Because if this were the case for finding an element of a list in Python, binary search would not be log length of the list, but it would be order length of the list.
Because the worst case is I'd have to visit every element of the list, say, to discover something isn't in it.
So, this is not what you want to do.
Instead, Python uses something like the picture in your handout.
And the key idea here is one of indirection.
So in Python, what a list looks like is it is a list, a section of memory, a list of objects each of the same size.
Because now what each object is a pointer.
So we've now separated in space the values of the members of the list and the pointers to, if you will, the next one.
So now, it can be very simple.
This first element could be big.
Second element could be small.
We don't care.
Now I'm back to exactly the model we looked at here.
If, say, a pointer to someplace in memory is 4 units long, then to find l-th of i, I use that trick to find, say, the i-th pointer.
And then it takes me only one step to follow it to get to the object.
So, I can now, in constant time, access any object into a list, even though the objects in the list are of varying size.
This is the way it's done in all object-oriented programming languages.
Does that makes sense to everybody?
This concept of indirection is one of the most powerful programming techniques we have.
It gets used a lot.
My dictionary defines indirection as a lack of straightforwardness and openness and as a synonym uses deceitfulness.
And it had this pejorative term until about 1950 when computer scientists discovered it and decided it was a wonderful thing.
There's something that's often quoted at people who do algorithms.
They say quote, "all problems in computer science can be solved by another level of indirection."
So, it's sort of, whenever you're stuck, you add another level of indirection.
The caveat to this is the one problem that can't be solved by adding another level of indirection is too many levels of indirection, which can be a problem.
As you look at certain kinds of memory structures, the fact that you've separated the pointers from the value fields can lead to them being in very far apart in memory, which can disturb behaviors of caches and things like that.
So in some models of memory this can lead to surprising inefficiency.
But most of the time it's really a great implementation technique.
And I highly recommend it.
So that's how we do the trick.
Now we can convince ourselves that binary search is indeed order log n. And as we saw Tuesday, logarithmic growth is very slow.
So it means we can use binary search to search enormous lists and get the answer very quickly.
All right.
There's still one catch.
And what's the catch?
There's an assumption to binary search.
Binary search works only when what assumption is true?
It's sorted
The list is sorted because it depends on that piece of knowledge.
So, that raises the question, how did it get sorted?
Or the other question it raises, if I ask you to search for something, does it make sense to follow the algorithm of (1) sort L, (2) use binary search?
Does that make sense?
Well, what does it depend upon, whether this makes sense from an efficiency point of view?
We know that that's order log length of L. We also know if the list isn't sorted, we can do it in order L. We can always use linear search.
So, whether or not this makes a good idea depends upon whether we can do this fast enough.
It has the question is order question mark plus order log len of L less than order L?
If it's not, it doesn't make sense to sort it first in sums, right?
So what's the answer to this question?
Do we think we can sort a list fast enough?
And what would fast enough mean?
What would it have to be?
For this to be better than this, we know that we have to be able to sort a list in sublinear time.
Can we do that?
Alas, the answer is provably no.
No matter how clever we are, there is no algorithm that will sort a list in sublinear time.
And if you think of it, that makes a lot of sense.
Because, how can you get a list in ascending or descending order without looking at every element in the list at least once?
Logic says you just can't do it.
If you're going to put something in order, you're going to have to look at it.
So we know that we have a lower bound on sorting, which is order L. And we know that order L plus order log length L is the same as order L, which is not better than that.
So why do we care?
If this is true, why are we interested in things like binary search at all?
And the reason is we're often interested in something called amortized complexity.
I know that there are some course 15 students in the class who will know what amortization means.
But maybe not everybody does.
The idea here is that if we can sort the list once and end up searching it many times, the cost of the sort can be allocated, a little bit of it, to each of the searches.
And if we do enough searches, then in fact it doesn't really matter how long the sort takes.
So if we were going to search this list a million times, maybe we don't care about the one-time overhead of sorting it.
And this kind of amortized analysis is quite common and is what we really end up doing most of the time in practice.
So the real question we want to ask is, if we plan on performing k searches-- who knows how long it will take to sort it-- what we'll take is order of whatever sort of the list is, plus k times log length of L. Is that less than k times len of L?
If I don't sort it, to do k sort searches will take this much time.
If I do sort it, it will take this much time.
The answer to this question, of course, depends upon what's the complexity of that and how big is k. Does that make sense?
In practice, k is often very big.
The number of times we access, say, a student record is quite large compared to the number of times students enroll in MIT.
So if at the start of each semester we produce a sorted list, it pays off to do the searches.
In fact, we don't do a sorted list.
We do something more complex.
But you understand the concept I hope.
Now we have to say, how well can we do that?
That's what I want to spend most of the rest of today on now is talking about how do we do sorting because it is a very common operation.
First of all, let's look at a way we don't do sorting.
There was a famous computer scientist who opined on this topic.
We can look for him this way.
A well-known technique is bubble sort.
Actually, stop.
We're going to need sound for this.
Do we have sound in the booth?
Do we have somebody in the booth?
Well, we either have sound or we don't.
We'll find out shortly.
Other way.
Come on.
You should know.
Oh there.
Thank you.
[VIDEO PLAYBACK] -Now, it's hard to get a job as President.
And you're going through the rigors now.
It's also hard to get a job at Google.
We have questions, and we ask our candidates questions.
And this one is from Larry Schwimmer.
[LAUGHTER] -You guys think I'm kidding?
It's right here.
What is the most efficient way to sort a million 32-bit integers?
[LAUGHTER] -Well, uh.
-I'm Sorry, maybe that's not a-- -I think the bubble sort would be the wrong way to go.
[LAUGHTER] -Come on, who told him this?
I didn't see computer science in your background.
-We've got our spies in there.
-OK, let's ask a different interval-- [END VIDEO PLAYBACK]
All right, so as he sometimes is, the President was correct.
Bubble sort, though often discussed, is almost always the wrong answer.
So we're not going to talk about bubble sort.
I, by the way, know Larry Schwimmer and can believe he did ask that question.
But yes, I'm surprised.
Someone had obviously warned the President, actually the then future president I think.
Let's look at a different one that's often used, and that's called selection sort.
This is about as simple as it gets.
The basic idea of selection sort-- and it's not a very good way to sort, but it is a useful kind of thing to look at because it introduces some ideas.
Like many algorithms, it depends upon establishing and maintaining an invariant.
An invariant is something that's invariantly true.
The invariant we're going to maintain here is we're going to have a pointer into the list.
And that pointer is going to divide the list into a prefix and a suffix.
And the invariant that we're going to maintain is that the prefix is always sorted.
We'll start where the prefix is empty.
It contains none of the list.
And then each step through the algorithm, we'll decrease the size of the suffix by one element and increase the size of the prefix by one element while maintaining the invariant.
And we'll be done when the size of the suffix is 0, and therefore the prefix contains all the elements.
And because we've been maintaining this invariant, we know that we have now sorted the list.
So, you can think about it.
For example, if I have a list that looks like 4, 2, 3, I'll start pointing here.
And the prefix, which contains nothing, obeys the invariant.
I'll then go through the list and find the smallest element in the list and swap it with the first element.
My next step, the list will look like 2, 4, 3.
I'll now point here.
My invariant is true.
The prefix contains only one element, so it is in ascending order.
And I've increased its size by 1.
I don't have to look at this element again because I know by construction that's the smallest.
Now I move here, and I look for the smallest element in the suffix, which will be 3.
I swapped 3 and 4.
And then I'm going to be done.
Does that make sense?
It's very straightforward.
It's, in some sense, the most obvious way to sort a list.
And if you look at the code, that's exactly what it does.
I've stated the invariant here.
And I just go through and I sort it.
So we can run it.
Let's do that.
I'm going to sort the list 3, 4, 5, et cetera, 35, 45.
I'm going to call selection sort.
And I don't think this is in your handout, but just to make it obvious what's going on, each iteration of the loop I'm going to print the partially sorted list so we can see what's happening.
The first step, it finds 4 and puts that in the beginning.
It actually finds 0, puts it in the beginning, et cetera.
All right?
So, people see what's going on here?
It's essentially doing exactly what I did on the board over there.
And when we're done, we have the list completely sorted.
What's the complexity of this?
What's the complexity of selection sort?
There are two things going on.
I'm doing a bunch of comparisons.
And I'm doing a bunch of swaps.
Since I do, at most, the same number of comparisons as I do swap or swaps as I do comparisons-- I never swap without doing a comparison-- we can calculate complexity by looking at the number of comparisons I'm doing.
You can see that in the code as well.
So how many comparisons might I have to do here?
The key thing to notice is each time I look at it, each iteration, I'm looking at every element in what?
In the list?
No, every element in the suffix.
The first time through, I'm going to look at-- let's just say n equals the length of the list.
So the first time through, I'm going to look at n elements.
Then I'm going to look at n minus 1.
Then I'm going to look at n minus 2.
Until I'm done, right?
So that's how many operations I'm doing.
And what is the order of n plus n minus 1 plus n minus 2?
Exactly.
Order n. So, selection sort is order n. Is that right?
Somebody said order n. Do you believe it's n?
Is this really n?
It's not n. What is it?
Somebody raise your hand, so I can throw the candy out.
Yeah
[INAUDIBLE]
It's not n factorial
n-squared?
You said that with a question mark at the end of your voice
No, it's like the sum of the numbers is, like, n times n minus 1 over 2 or something like that
It's really exactly right.
It's a little smaller than n-squared, but it's order n-squared.
I'm doing a lot of these additions.
So I can't ignore all of these extra terms and say they don't matter.
It's almost as bad as comparing every element to every other element.
So, selection sort is order n-squared.
And you can do it by understanding that sum or you can look at the code here.
And that sort of will also tip you off.
Ok. so now, can we do better?
There was a while where people were pretty unsure whether you could do better.
But we can.
If we think about it now, it was a method invented by John von Neumann, a very famous guy.
And he, back in the '40s amazingly enough, viewed this as a kind of divide and conquer algorithm.
And we've looked at divide and conquer before.
What is the general form of divide and conquer?
A phrase you've heard me use many times, popularized, by the way, I think, by Machiavelli in The Prince, in a not very nice context.
So, what we do-- and they're all of a kind, the same-- we start with 1.
Let me get over here and get a full board for this.
First, we have to choose a threshold size.
Let's call it n0.
And that will be, essentially, the smallest problem.
So, we can keep dividing, making our problem smaller-- this is what we saw with binary search, for example-- until it's small enough that we say, oh the heck with it.
We'll stop dividing it.
Now we'll just solve it directly.
So, that's how small we need to do it, the smallest thing we'll divide things into.
The next thing we have to ask ourselves is, how many instances at each division?
We have a big problem.
We divide it into smaller problems.
How many are we going to divide it into?
We divide it into smaller problems until we reach the threshold where we can solve it directly.
And then the third and most important part is we need some algorithm to combine the sub-solutions.
It's no good solving the small problem if we don't have some way to combine them to solve the larger problem.
We saw that before, and now we're going to see it again.
And we're going to see it, in particular, in the context of merge sort.
If I use this board, can people see it or is the screen going to occlude it?
Is there anyone who cannot see this board if I write on it?
All right then, I will write on it.
Let's first look at this problem.
What von Neumann observed in 1945 is given two sorted lists-- and amazingly enough, this is still the most popular sorting algorithm or one of the two most popular I should say-- you can merge them quickly.
Let's look at an example.
I'll take the lists 1, 5, 12, 18, 19, and 20.
That's list one.
And I'll try and merge it with the list 2, 3, 4, and 17.
The way you do the merge is you start by comparing the first element to the first element.
And then you choose and say all right, 1 is smaller than 2.
So that will be the first element of the merge list.
I'm now done with 1, and I never have to look at it again.
The next thing I do is I compare 5 and 2, the head of the two remaining lists.
And I say, well, 2 is smaller than 5.
I never have to look at 2 again.
And then compare 5 and 3.
I say 3 is smaller.
I never have to look at 3 again.
I then compare 4 and 5.
4 is smaller.
I then compare 5 and 17.
5 is smaller.
Et cetera.
Now, how many comparisons am I going to do this time?
Well, let's first ask the question, how many elements am I going to copy from one of these lists to this list?
Copy each element once, right?
So, the number of copies is order len of the list.
That's pretty good.
That's linear.
That's sort of at the lower bound.
But how many comparisons?
That's a little trickier to think about
[INAUDIBLE]
Pardon?
At most, the length of the longer list
At most, the length of the longer list, which would also be, we could claim to be, order len of-- I sort of cheated using L when we have two lists.
But just think of it as the longer list.
So, you'd think that many comparisons.
You think we can do this whole thing in linear time?
And the answer is yes.
That's our merge.
That's a good thing.
Now, that takes care of this step.
But now we have to ask, how many times are we going to do a merge?
Because remember, this worked because these lists were sorted.
And so I only had to compare the front of each list.
When I think about how I'm going to do the binary or the merge sort, what I'm going to do is take the original list, break it up, break it up, break it up, break it up, until I have a list of length 1.
Well, those are all sorted, trivially sorted.
And then I'll have, at the end, a bunch of lists of length 1.
I'll merge pairs of those.
Now I'll have sorted lists of length 2.
Then I'll merge those, getting sorted lists of length 4.
Until at the end, I'll be merging two lists, each half the length of the original list.
Right.
Does that make sense to everybody?
Now I have to ask the question, how many times am I going to call merge?
Yeah
Base 2 log of one of the lists
Yes, I'm going to call merge log length of the list times.
So, if each merge is order n where n is length of the list, and I call merge log n times, what's the total complexity of the merge sort?
nlog(n)
nlog(n).
Thank you.
Let's see, I have to choose a heavy candy because they carry better.
Not well enough though.
All right, you can relay it back.
Now let's look at an implementation.
Here's the implementation of sort.
And I don't think you need to look at it in detail.
It's doing exactly what I did on the board.
Actually, you do need to look at in detail, but not in real time.
And then sort.
Now, there's a little complication here because I wanted to show another feature to you.
For the moment, we'll ignore the complication, which is-- it's, in principle, working, but it's not very bright.
I'll use the mouse.
What we see here is, whenever you do a sort, you're sorting by some ordering metric.
It could be less than.
It could be greater than.
It could be anything you want.
If you're sorting people, you could sort them by weight or you could sort them by height.
You could sort them by, God forbid, GPA, whatever you want.
So, I've written sort to take as an argument the ordering.
I've used this funny thing called lambda, which you don't actually have to be responsible for.
You're never going to, probably, need to use it in this course.
But it's a way to dynamically build a function on the fly.
The function I've built is I've said the default value of LT is x less than y. Lambda x, lambda xy says x and y are the parameters to a function.
And the body of the function is simply return the value x less than y.
All right?
Nothing very exciting there.
What is exciting is having a function as an argument.
And that is something that you'll be doing in future problem sets.
Because it's one of the very powerful and most useful features in Python, is using functional arguments.
Right.
Having got past that, what we see is we first say if the length of L is less than 2-- that's my threshold-- then I'm just going to return L, actually a copy of L. Otherwise, I'm going to find roughly the middle of L. Then I'm going to call sort recursively with the part to the left of the middle and the part to the right of the middle, and then merge them.
So I'm going to go all the way down until I get to list of length 1, and then bubble all the way back up, merging as I go.
So, we can see that the depth of the recursion will be log(n), as observed before.
This is exactly what we looked at when we looked at binary search.
How many times can you divide something in half -- log(n) times?
And each recursion we're going to call merge.
So, this is consistent with the notion that the complexity of the overall algorithm is nlog(n).
Let's run it.
And I'm going to print as we go what's getting merged.
Get rid of this one.
This was our selection sort.
We already looked at that.
Yeah.
So what we'll see here is the first example, I was just sorting a list of integers.
Maybe we'll look at that all by itself.
I didn't pass it in the second argument, so it used the default less than.
It was first merge 4 and 5.
Then it had to merge 35 with 4 and 5, then 29 with 17, 58 and 0.
And then the longer list, 1729 with 058, 04535 with 0172958.
And then we were done.
So, indeed it did a logarithmic number of merges.
The next piece of code, I'm taking advantage of the fact that this function can sort lists of different kinds.
And I'm calling it now with the list of floats.
And I am passing in the second argument, which is going to be-- well, let's for fun, I wonder what happens if I make this greater than.
Let's see what we get.
Now you note, it's sorted it in the other order.
Because I passed in the ordering that said I want to use a different comparison then less than, I want to use greater than.
So the same code did the sort the other way.
I can do more interesting things.
So, here I'm assuming I have a list of names.
And I've written two ordering functions myself, one that first compares the last names and then the first names.
And a different one that compares the first names and then the last names.
And we can look at those.
Just to avoid cluttering up the screen, let me get rid of this.
What we can see is we got-- we did the same way of dividing things initially, but now we got different orderings.
So, if we look at the first ordering I used, we start with Giselle Brady and then Tom Brady and then Chancellor Grimson, et cetera.
And if we do the second ordering, we see, among other things, you have me between Giselle and Tom.
Not a bad outcome from my perspective.
But again, a lot of flexibility.
By using this functional argument, I can define whatever functions I want, and using the same sort, get lots of different code.
And you will discover that in fact the built in sort of Python has this kind of flexibility.
You will also find, as you write your own programs, increasingly you'll want to use functions as arguments.
Because it allows you to write a lot less code to accomplish the same tasks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to finish up our little excursion into searching and sorting, a very important topic in computing.
How do you sort lists or databases?
How do you search them?
And I want to finish it up with hashing.
Hashing is how dictionaries are implemented in Python.
And this leads to a very efficient-- at least most of the time-- search.
But it comes at the cost of space.
So this is an example where one can trade space for time.
So I want to start with a simple example to explain hashing.
And I'm going to begin by assuming that what we're hashing is a set of integers.
So we want to build a set of integers and detect whether or not a particular integer is in that set.
And we want to do that quickly.
So the basic idea is we're going to take an integer-- call it i-- and we're going to hash it.
I'll tell you what that means in a minute.
And what a hash function does is it converts i to a different integer, perhaps, in some range.
So it, say, converts i to some integer in the range 0 to k, for some constant k. We're going to then use this integer to index into a list of lists.
So each of these is called a bucket.
And a bucket will itself be a list.
So this together is called a bucket.
We've already seen that we can find the i-th element of a list in constant time.
So when we ask whether some integers in this set will hash it, we'll go immediately to the correct bucket, the bucket associated with that integer.
And then we'll search the list at that bucket to see if the integer is there.
If this list is short enough, it will be very efficient.
All, right I realize that's very abstract.
But let's look at the code, which will make it much less abstract.
So the code starts with something very ugly that I'll apologize for.
But very soon, we'll see how to get rid of that.
I'm using a global variable here to say how many buckets there are going to be.
And I've very arbitrarily chosen 47.
I then have a function called create, which uses this global variable and creates a list of lists, each element of which is empty.
Because initially, we have no elements in the set.
When I want to insert an element in the set, I'll call the function insert, which will hash the element.
It doesn't actually even need to use the global numBuckets in this case, in fact.
And then append it to the correct list.
So it calls this function hashElem, which could hardly be simpler.
It just takes the remainder, the modulus of the element and the number of buckets.
So that will give me a value between 0 and numBuckets minus 1.
It gives me one of the list, and I'll just insert it at the end.
When I want to check for membership, you'll see it's quite simple.
All I do is ask the question, is i in the list associated with the correct bucket?
Remove is a little bit more complicated, but in fact, we don't need to spend much time looking at it now.
It's just some code.
And the only reason it's complicated is Insert doesn't look whether or not the element is already there.
So it may occur multiple times in the list.
So I would have to remove each one of them.
People see the basic structure of this?
Why a list of lists?
Why don't I just, say, have a list of Booleans, where I hash the integer, and it's true or false, whether or not I've seen it, depending upon the value of that bucket?
Why can't I do that?
Somebody?
Well, what's the property of this hash function?
The key issue here is the hash function -- is many-to-one.
That is to say, an infinite number of different integers will hash to the same value.
Because after all, I have a set in which I can store any integer-- or any positive integer, at least-- and there are only 47 buckets.
So it's pretty obvious that many integers will hash to the same bucket.
When two different elements hash to the same bucket, we have what is called a collision.
There are lots of different ways to handle collisions.
What I've shown you here is probably the simplest way, which is called linear rehashing.
I'm not actually rehashing.
I'm just keeping a list.
Does that makes sense?
Yes, thank you for-- I'm glad somebody has a question.
You have to ask loudly
When you take the modulus 47, what does that return?
0
So hashElem always returns 0?
Well, no, sorry.
It depends what I'm hashing.
Sorry, I thought if you were saying that if I asked 47 mod 47.
If I take 48 mod 47, I get 1.
If I take 49 mod 47-- it's the remainder.
Maybe I should have called it just the remainder.
Look up.
I'm about to throw something at you.
Ooh, I threw a curve ball.
OK. What's the complexity here of the membership test?
Kind of hard to analyze.
Roughly speaking, or exactly, it will be the length of the bucket, the size of the bucket.
Now, I don't know how many elements will be in the bucket.
But what will this depend on?
It will depend upon the number of buckets.
If I have a million buckets, I'll get a lot fewer collisions than if I have two buckets.
So let's look at an example here.
There's a small program called test.
I said numBuckets to 47 in this case.
And then I'm going to create it, create a set.
And then I'm going to put a bunch of integers in it, then a few more, just for fun, including one very big integer, just to show that it works.
Then I'm going to show you what the set looks like.
And in fact, what we'll do is we'll stop it here and see what we get.
So what we'll see here is, as you would expect with that number of buckets, each of the small numbers hashes to a separate thing.
That's just the way remainder works.
Not surprisingly-- in fact, it would be disappointing if 325 didn't have the same value both times I inserted it.
So we say we happen to have one bucket that's got two elements in it.
Happen to be the same.
But this very big number happened to hash to the same value of 30 as 34.
So here we have two elements in it.
A good hash function has the property that it will widely disperse the values you hash.
So they end up in different buckets, rather than some stupid hash function that tends to put everything in the same bucket.
Now, let's see what happens if I change the number of buckets to, say, 3.
Well, not surprisingly, we get some very big buckets, because there are relatively few choices.
So what we see here is we have a genuine trade off between time and space.
If the number of buckets is large relative to the number of elements that we insert in the table, then looking at whether or not an element in it is roughly order one.
Because these lists will be very short.
So we can actually look up something in constant time if we dedicate enough space to the hash table.
If the hash table is very small-- the reduction ad absurdium case of one bucket-- then it's order n. It's not constant time.
It's linear in the number of elements.
Typically, when people use hash tables, they make the hash tables big enough that for all intents and purposes, you can assume that looking something up is constant time, order 1.
And that, in fact, is what Python does with dictionaries.
It hashes the keys and chooses a big enough table so that looking up whether or not something is in a dictionary can be done in constant time.
If it then notices that the table is too small, because you've ended up putting a lot of elements in it, it just re-does it and gets a bigger table.
So hashing is an extremely powerful technique.
And it's used all the time for quite complicated things.
Now is it useful here only when we want to store ints?
No.
It would be kind of bad if that were the case.
In fact, any kind of immutable object can be hashed.
Now, you may have wondered why the keys in dicts have to be immutable.
And that's so that they can be hashed.
Why does it have to be immutable?
Well, imagine that you used a list as a key.
You'd hash it when you put the list in the hash table.
But then you might mutate it, and the next time you hashed it, you'd get a different value, and so you wouldn't be able to find it again.
So you need it to be a kind of object, where every time you apply the hash function, you get the same value.
I don't need to show you that this works.
You'll just believe me, I'm sure.
Let's look at a slightly more complicated hash function.
Here, I want to hash something that could be either an int or a string.
So I first check and see if it's an int.
If so, I'll set the value to be e. Then down at the bottom, I'll do the modulus operator again.
But if it's a string, I'm going to first convert e to an int.
And this is basically the trick that people typically use when they're doing hashing, is they convert whatever thing they have, to some integer.
People do this with all sorts of things.
For example, this is the way airport security systems today do face recognition.
They hash every picture of a face to an integer, and then look it up.
So we can see it here.
The way I'm doing it-- and the details don't very much matter-- is I'm going to do it a character at a time, do a shift ord of C, takes the ASCII, the internal representation bits of each character.
And again, I don't care if you understand how the code works here.
What I just want you to see is that it's not very long.
And in, fact it's typically fairly simple to hash almost any kind of an object.
Because deep down, we know that every object is represented by some string of bits in the computer's memory.
And we can always convert a string of bits to an integer.
All right, so that's hashing, a very powerful and extremely useful technique.
Yeah?
[INAUDIBLE] you're returning something, will you always find the remainder, or can you use [INAUDIBLE]?
There are many different ways of doing hash.
All the hash function has to do is convert its argument into an integer in some way or another within a fixed range.
It happens to be that remainder is a very simple way to do that and is often used.
There's a whole theory about hash functions.
You can read about them in gory detail on Wikipedia.
The math can actually be quite complicated, and there's no real need to understand it.
I typically do something like dividing by a prime number, which is known to have good properties.
But, again, you can get carried away with this.
And it's usually not worth the trouble, unless you're very deeply involved in something.
OK?
And then I've got another program that does this, but again, I don't think much would be served by running it for you.
Any questions about hashing?
Yes?
So does this only work because you said [INAUDIBLE] Python treats lists in a certain way?
You said other languages, you can't have this constant time list search.
So how would hashing work?
So the question is that, because Python gives us constant-time looking up for lists is the key to making this work -- every programming language I know has some concept that's similar.
In, say, C, it's not a list, but it's an array.
And so in C, you would use array for this purpose?
In Java, you would use the arrays for this purpose?
But yes, you can do this in any programming language, because every reasonable programming language has some way to create the equivalent of a list, in which you can get a particular index in constant time.
So it's a universally useful technique.
Good question.
Anything else?
If not, we're about to abandon searching and sorting, and in fact, abandon algorithms in general for a while.
Bounced right off his hands.
I will not sign you to a baseball contract.
Pardon?
Yes, I threw him a Butterfinger?
Oh, it's terrible.
Boy, your jokes are worse than mine.
I now want to move on to the last-- go back to Python, away from algorithms, away from computer science in general, and talk about the last three major linguistic concepts in Python, exceptions, classes, and then later iterators.
Let's start with exceptions, because we've already seen them, and they're pretty simple.
Exceptions are everywhere in Python.
And we've certainly seen plenty of them all semester already.
We get them if we set some list to, say [1,2] and then I asked for a test of 12, I get an index error.
This is an exception.
We've got exceptions when we've tried to convert things to incorrect types.
So if I go into test, I'll get a type error.
Anything that ends in the word "error" is a built-in kind of exception in Python.
We've got them when we accessed, nonexistent variables, a name error.
So there are a whole bunch of these things.
In each of these cases-- here, I'm just kind of playing around in Python.
And it printed an error message, and it stopped.
These kind of exceptions are called unhandled exceptions, where they cause the program to effectively crash.
The program just stops running.
And I suspect that some of you have written programs in which this has happened.
In fact, is there anybody here who has not written a program that crashed because of an unhandled exception?
Good.
For those of you watching on TV, no hands went up.
Almost every day, I write a program that crashes because of an unhandled exception.
On the other hand, once my program is debugged-- once your program is debugged-- this should never happen.
Because there are mechanisms in Python for handling exceptions.
And in fact, as we'll see, it's a perfectly valid flow of control concept.
You will sometimes write programs that are intended to raise exceptions.
And then you'll catch the exception and do something useful with it.
The way we use exceptions is in something called a try-except block.
So you write the word "try," and then you have some code, and then "except," and then some more code.
What the interpreter does is it starts executing this code.
If it gets through this code without raising any kind of an exception, it jumps to the code following the except block.
On the other hand, if an exception gets raised here, it immediately stops executing this code and jumps to the start of this code, the code associated with the except, and executes that.
And, then when it finishes this, it again goes to the code following the except.
Just like an if, you can nest these things.
It's just a control concept.
It's nothing more.
Let's look at an example.
So here's readVal.
This is a function.
It's a polymorphic function.
Its first argument is of type 'type'.
Remember-- and you'll hear me say this a 1,000 more times at least-- in Python, everything is an object and can be manipulated by the program.
It's one of the beauties of Python.
It's something that's not true in many programming languages.
So types are objects, just like ints or floats.
So the first argument to readVal is a type.
And then there are two strings, the request message and the error message.
It then sets a local variable, numTries, to 0.
And while numTries is less than 4, it sets val equal to raw input, printing the request message.
And then it tries to convert val to valType, whatever that happens to be.
And if it succeeds, it returns it.
On the other hand, if during this attempt to convert it an exception is raised-- for example, the user is asked to input an integer, and they type in the letter b, which can't be converted to an int-- then a type error exception will be raised, or a value error exception.
And if a value error exception is raised, it prints the error message, increases numTries by 1, and goes back to the top of the y loop.
If it goes through this y loop too many times, it leaves, raising the type error with the message that the argument "numTries exceeded."
All right.
Let's run this and see what happens.
Every once in a while, funny things happen.
When that happens, somehow, clicking in this window, and then clicking back in that window tends to make it work again.
This is a bug in Python.
So it's now asked me to enter an int.
If I enter an int, everything is good.
It just prints it.
On the other hand, if I run it, and I enter the letter A, it will give me another chance.
And now I can enter an int, and it's happy.
All right.
Now, suppose I am unhappy with the fact that if four times in a row I've failed to enter an int, something bad happens.
It comes back, and it raises an exception.
Well, I can catch that in the code that calls readVal.
So here at the top level, I'm going to try readVal.
And then I'm going to say, except if a type error is raised, print the argument returned by that exception.
So this is what's called the handler for the exception.
And so I don't have to crash when the exception is raised.
But I can actually deal with it and do something sensible.
If following the word "except," as you see over there, I have not listed any exception names, then I'll go to the except clause for all exceptions.
Doesn't matter what the exception is.
I will go there.
So I can write code that captures any exception.
OK. Usually, that's not as good a thing, because it shows that I did not anticipate what the exception might be.
But you can see that this is a pretty powerful programming paradigm.
I can write this fairly compact readVal function that's pretty robust.
It's polymorphic, it can take in what the error messages are, and it can try as many times as I want.
Yeah?
[INAUDIBLE] after type error [INAUDIBLE]?
You'll note that when I raised the exception type error after it, I had an open [UNINTELLIGIBLE] a string.
So what this basically says is that an exception can have associated with it a set of arguments, a sequence of arguments.
And I've just chosen to call the first of the arguments s, so that I could then print it.
So this is a fairly common paradigm, that you associate a message with an exception, explaining the exception.
Since after all, type error is not all that meaningful.
And this tells me why it was raised, that I tried too many times.
OK, does it make sense?
Anything else?
Yeah?
[INAUDIBLE] before you said [INAUDIBLE]?
Well, what we'll see when I execute something like assert false, it's actually raising an exception.
It's raising an assertion error exception.
And so I can actually catch that and do something with it.
I've been using these asserts just basically to stop the program.
But we've also seen that sometimes in functions, we use assertions to check the types of the arguments.
But if you don't catch that exception, then the program crashes with the wrong arguments, without doing anything useful.
Because that's just an exception, I can catch it, and then do something useful.
So it's, again, just an example of that.
We're going to try your hands once more.
I'll throw you a different kind of candy.
And this time he caught it.
Two for two.
Very powerful.
A very useful mechanism.
You can use them for sort of catching errors, as we've done here.
They are frequently used in situations where you are getting input from a user.
So for example, if you were writing a text editor, and you wanted to open up a file, and you typed in the file name, and it didn't exist, you would in Python get an error message when you try to open that file.
It would raise an exception, "File Not Found," basically, which you could then catch, and then print to the user a useful error message saying, "File Not Found."
Similarly, if you try and write to a file that already exists, you can get an exception saying, do you really want to overwrite this file?
And ask the user.
So it's very commonly used in a lot of programs as a mechanism.
And now, as we go on and see more and more code, you'll see that I'm going to start using exceptions fairly frequently as a flow of control mechanism.
It makes certain kinds of code easier to write.
All right.
That's all I have to say about exceptions.
Simple but useful mechanism.
Now, on to something that's even more useful, but not so simple.
It is probably the distinguishing thing, not only in Python, but in a whole class of modern programming languages.
And that's the notion of a class.
I'm not going to finish it today.
I'm barely going to scratch the surface of it today.
But we're going to start leading up and I will pretty much finish it on Thursday.
We've already seen the notion of a module, which has been a collection of related functions.
So, for example, we've seen code that includes something like import math.
And that provided me with access to functions like math.log.
What the module mechanism does in the import mechanism, it makes it convenient to import a lot of related things at once.
And then we use this dot notation to disambiguate, to tell us, well, which log.
Typically, there's probably only one log function.
But certainly, you might imagine that set.member and table.member would be different, and that both might exist.
And as we've said before, the dot notation avoids conflicts by disambiguating.
OK. That's a module.
What a class is, it is like a module, but it's not just a collection of functions.
A class is a collection of data and functions, functions that operate on that data.
They are bound together, so that you can pass an object from one part of a program to another.
And the part of the program to which you pass it automatically gets access to the functions associated with that type of object.
And this is really the key to what people call object-oriented programming, a very popular buzzword.
So we've already seen that kind of thing, where if we pass a list from one function to another, we can write something like L.append, some value.
The data and functions associated with an object are called that object's attributes.
So you can think of this as a way to associate attributes with objects.
Now, I've been talking about data and functions as if they're different kinds of things.
In fact, they're not really, because they're just objects.
In Python, everything is an object, including, as we'll see on Thursday, the class itself is an object.
When people talk about objects and object-oriented programming, they often use a message passing metaphor.
And I want to emphasize it's nothing more than a metaphor.
And I almost hesitate to bring it up, because it makes it all sound more complicated than it is.
But you will see this phrase in the literature.
People will use it.
So you need to know what it is.
The basic metaphor is that when I write something like L.append, I am passing the message append e to the object L. And then there's a mechanism for looking up what that object means, what that message means.
And then the object, L, executes that message, and does something.
So, for example, if I had a class called Circle, I might pass the object C -- the message area, which would cause this object to return the area of its own area, the area of the circle that is that object.
Again, nothing dramatic going on here.
If you just think of this as a fancy way of writing functions, you'll be absolutely correct.
But I did think you should hear about this metaphor.
I add by the way -- should have-- I've been using the word...
Method is a function associated with an object.
So in this case, the method area is associated with the object C. And purists would say, always refer to append as a method, rather than a function, because we use the dot notation to get to it, and it's always associated with some object of type list.
Now, just as data can have types, objects, as we know, have types.
What a class is is it's a collection of objects with identical characteristics that form a type.
So we can use classes to introduce new types into the programming environment.
So as you think about existing things-- now, I won't put that up, because it will disappear behind the screen.
We've looked at things like lists and dict.
What these are are built-in classes.
They happen to be classes that are so useful that somebody decided they should be part of the language, they should have efficient implementations, they should be built-in, people shouldn't have to reimplement them themselves.
And in fact, there are a whole bunch of interesting built-in classes in Python.
And there are a whole bunch of libraries of classes you can bring in-- and we'll look at many of those-- that extend it.
And that's the beauty of the class mechanism.
It lets you add new types to the language that are every bit as easy to use as the built-in types.
So in effect, the language can be extended to add new and useful types.
And we'll look at several examples of that starting on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning, everybody.
Let's go back to where we were.
We were talking about abstract data types and the notion of object oriented programming.
And in particular, the role of classes in all of that.
A lot of people think this notion of object oriented programming is a new notion.
In fact, it's not.
It's been around for probably at least 35 years, maybe even longer.
But it's only been widely practiced for maybe 15 years.
Even that to most of you will seem like a long time.
It started in the mid 1970s when people began to write articles explaining this advantage of the approach to programming.
And at about the same time, the language Smalltalk was developed at Xerox Park and CLU developed at MIT.
And they were the first languages that, in an elegant way, provided linguistic support for this style of programming.
But it didn't really take off in the public until the introduction of Java, considerably later.
And Java was the first popular language to support object oriented programming.
After that there was C++, which supports it in not a very elegant but a usable way.
And today probably Python is the fastest growing language supporting object oriented programming.
It's used widely, and that's one of the reasons we teach it here.
As I said, at the bottom of it all, the most fundamental notion is that of an abstract data type.
And the idea is that we can extend our programming language by adding user defined types.
And we'll shortly see why that's such a useful thing.
And that these types can be used just as easily as any of the built in types.
Why do we call them abstract data types rather than just data types?
We do that because we are essentially going to define for each type an interface.
And essentially, what the interface does is it explains what the methods do.
What do I mean by what they do?
What they do at the level of the user, not how they do it.
That, of course, is the way the built in types work.
It wasn't until Tuesday that you understood how Python made dicts do what they do.
Before then, I just explained, that you could put in a key and a value, and you could look it up and do an associative retrieval.
It was maybe not magic, but it was wonderful that you could do this and not have to bother your heads with how it was made to work.
And that was because we provided, or the designers of Python provided, an interface that lets you use it.
We'll do the same thing with the abstract data types.
The key idea here is one we've talked about before and that's a specification.
It is the specification of a type, or of a function, or of a method, that tells us what that thing does.
And we'll now until the end of the term try and maintain a very clear distinction between specifications and implementations.
Let's look at an example.
So the example should be familiar to you.
You will remember that on Tuesday at the start of the lecture, we looked at how we could use hashing to implement a set of integers.
And I explained to you that I wasn't very happy with the implementation because of the way it used this global variable.
Now we're going to see a much more elegant approach.
So I'm going to define a new abstract type called intSet.
I do that by writing the word, class, followed by the name of the class.
And then there's this funny thing saying that it is a subclass of objects.
Ignore that for now.
And I'm going to come back to it later.
But fundamentally what it's saying is that every instance of intSet is an object.
That's not very interesting because everything in Python is an object.
So from an information theoretic point of view, there's no information here.
Later we'll see that we can use this mechanism in a more interesting way.
Now let's look at the methods.
So first I tell you a comment, it's a set of integers.
Then I've got this funny method called underbar underbar init.
Whenever you see something with an underbar underbar in its name, it has a special status in Python that lets us do elegant things with the syntax.
What will happen every time I create a new object of type intSet, the underbar underbar init method, or function, of the class will be executed on that object.
What it will do, in this case, is introduce two attributes of the object.
The attributes are numBuckets which is now replacing the global variable we looked at last time.
And I've arbitrarily chosen 47.
And Vals.
This will be the hash table itself containing the values.
And then, exactly as we did on Tuesday, I'm going to initialize the values so that each element of this list is now an empty list.
Now what's going on with this funny notion of self.
Let's look at an example here.
I'm going to say s equals intSet open close.
This will create a new intSet object, execute the underbar underbar init.
So if, for example, I print self.numBuckets-- whoops, can't do that because self is not defined in this environment.
Self was a local variable to underbar underbar init.
In fact, the formal parameter.
But I can write s.numBuckets, and I'll see it's 47.
NumBuckets and Val are now attributes of s. Attributes of the instance s of the class intSet.
And I [[UNINTELLIGIBLE] what we would expect.
Yes?
You didn't put any object in between the parentheses
Yes.
So that-- the question is well, it looks like underbar underbar intSet or underbar underbar init has a formal parameter.
And I've given it no corresponding actual parameter
[INAUDIBLE]
Yes.
Let me finish.
So that's what it looks like.
And that's the magic of this syntax.
It automatically passes an implicit object, or it creates one.
It's a very special role for underbar underbar init.
Self is used to refer to the object being created.
I'm going to come back in a minute to discuss more fully the concept of self and how it's used here.
So just give me a minute to get there.
The next thing we see is hashE.
This is something we looked at again on Tuesday.
It's a private function in this case that I don't intend to be used outside the class.
So if we think of the specifications here, the interface of the class, it does not include hashE.
That's what private means here.
This is a convention.
It's not enforced by the language.
And unfortunately, as we'll see, a lot of useful things are not enforced by the language.
But nevertheless, good programmers follow the conventions.
And we expect you guys to do so as well because, of course, you're good programmers.
So this doesn't do anything very exciting.
It's just what we saw.
Insert is more interesting.
Let's look at what that does.
It apparently takes two arguments, the formals named self and E and inserts E into self.vals.
However, if we go look at the code that uses it in say testone what you'll note is I'm saying for i in range 40-- this is just like the testone we looked at Tuesday.
I'm going to say s.insert of i.
It looks like I'm calling insert with only one argument, but as we discussed last time, this s before the dot is actually the first argument to the method insert.
And so it's getting two arguments really.
And by convention, that implicit first argument is always called self in Python.
It's not enforced.
You could call it George if you preferred, or Alice, or whatever you like.
But if you do, you will confuse the heck out of anybody who ever reads your code, including any TA you ask for help.
So please do use self.
It's just a name.
It has nothing to do with what a philosopher, or a psychologist, or an ethicist might think as the concept of self, this wonderful elevated concept.
It's none of those.
It's just a name.
But stick to using that name.
So if we go back, I can insert a bunch of things.
I'm going to then print s. That's kind of interesting.
UnderBar underbar STR, underbar underbar is one of those special names as well.
The code is pretty simple.
All it is is returning a string representation of the set.
And it could be anything you want.
I chose sort of a conventional way of denoting sets.
What's interesting is that it gets automatically called by print.
So when I write the command, print s, in test one, the Python interpreter is smart enough to know oh, I better take s, convert it to a string, and then print it.
How does it know to convert it to a string?
It automatically invokes the underbar underbar STR method.
And then the other operation is member, which again is exactly what we looked at before.
But you'll notice in this code that uses intSets, I make no reference to the data attributes of the class directly, or I shouldn't.
You'll notice that I wrote evil next to s.vals.
Python will let me do it, but I shouldn't.
Why shouldn't I do it?
Why am I saying this is evil?
Well, would you be happy if you got a message saying IDLE was changed.
Please download a new version and the new version happened to have a different implementation of lists or dicts.
And it caused all of your programs to stop working.
You would be pretty unhappy.
Now why won't that happen?
Because your programs don't depend, in any way, on the way in which people chose to implement those built in types because you programmed to the specification of the types, not to the implementation.
The specification of intSet did not mention Vals or numBuckets.
Therefore, as the implementer of that class, I'm entitled to go back and change it.
So I'm not going to use a hash table at all.
I'm going to do something else.
I'm going to use a red, black tree, or some other fancy implementation.
I'm allowed to do that.
And if I make that change, and Vals disappears and numBuckets disappears, your program should continue to work so long as I still meet the specification.
The minute you go in and directly access the variables of the class, those attributes, you have used things that do not appear in this specification.
And if I change the implementation, your program might break.
So you shouldn't do that.
Does that make sense to everybody?
It's a very important concept.
The concept is known as data hiding.
It is really the most important development that makes abstract data types useful.
It gives them the same status as the built in types.
The minute you choose to ignore this, you do so at your own peril.
Some programming languages, like Java, provide a mechanism to enforce data hiding.
The designers of Python, for reasons I do not understand, chose not to.
I think it is a flaw in the language.
The things we're hiding are the instance variable.
Those are the variables associated with each instance of the class.
We should also hide class variables.
We haven't seen those yet, and we'll see them later.
The instance variables, we get a new copy of each time we created a new instance of the class, a new intSet in this case.
The class variables are associated with the class itself.
And you get only one copy of them.
Later on, we'll see an example where class variables are useful.
All right.
So let's run this just to make sure it works.
It does what we would expect.
True/False.
And then you'll see this nice string representation of the set.
And then just to show you what happens when you do the evil thing, I printed as we did last time the actual list that came out.
Let's now look at a more interesting example.
And the idea I want to convey here is how we use classes and abstract data types to design programs.
So imagine that you're writing a program to keep track of all the students, faculty, and maybe staff at MIT.
It's certainly possible to write that program without using any classes.
For each student, you might give them a family name, a given name, a home address, years, grades, et cetera.
And you could do this with some complicated combination of lists and dictionaries.
But it wouldn't be very elegant.
So what I want to do before writing that program-- and I won't actually write that program.
I'll just write some of the classes we can use-- I want to pull back and think about what abstractions would be useful.
So this style of programming in which you organize your programs around abstract data types says before we write the code in detail, we think about these types that would make it easy to write the code.
So for example, if you were a finance student, and you wanted to write some code dealing with markets, you might want to have an abstraction of a government bond, and another abstraction of an equity, and an extraction of a call option.
Whatever you want it.
But you'd say I want to think at that level of abstraction.
I don't want to think about lists, and dicts, and floats.
I just want to think about options, which have a strike price, a date, and things like that.
Similarly, as I'm working on this database for MIT, I want to think about abstractions of students, and faculty, and staff.
I'm also going to use what's called inheritance to set up a hierarchy of these.
And the reason I'm going to do that is I want to be able to share a code.
I know that there will be certain similarities between students and faculty.
Some differences too.
But I want to begin by saying what's not different?
What's similar?
What's the same?
So that I only have to implement it once and get to reuse it.
So if I pull back and say is there an abstraction that covers the shared attributes of students, and faculty, and staff, I might say it's a person.
They're all people.
And it's arguable whether every faculty member is a human being, but for now let's pretend.
And so I'm going to start with this abstraction person.
I'm going to import something called DateTime.
I'll show you what we're doing when we get there.
But it's like we've imported math before.
This is a class somebody else wrote that deals with dates and time in a fairly reasonable way.
So I'm going to import that here into person.
Probably didn't need to import it twice.
Maybe I'll just simplify it by getting rid of this one.
UnderBar underbar init here will create a person with name 'Name'.
And you'll notice what I'm doing here is introducing an extra attribute lastName -- self.lastName.
And that's because I just want to make life easy for myself.
I figure I'll want to find the last name frequently.
Let's once and for all get it and put it in something.
And I'm going to initialize birthday to none.
The next attribute, or the next method, is get lastName.
I have that here because I don't want users of this abstraction to even know that I have an attribute self.lastname.
That's part of the implementation.
And so I have this method that fetches it.
And you'll see as you build classes that you often have things called get.
And those are typically methods that return some information about an instance of the class.
You'll also frequently have set methods, for example, set birthday that gives values to instances of the class.
More interestingly, I have get age.
And that's using some of the built in operations on DateTime conveniently.
It allows me to subtract one date from another and get a number of days.
So this will allow me to return somebody's age in days.
Then I've got another underbar underbar method we haven't seen yet.
lt stands for less than.
Not a big surprise.
And I'm going to use this to order names, or order people.
Why am I using a special operator rather than just putting in the method L-E-S-S-T-H. A-N?
E-N?
I'm doing that because I want to be able to write things like p1 less than p2 in my code.
And Python will take that and turn it into the underbar underbar LT.
Better yet, if I have a list say of person's, I can use the built in sort operator on that list, and it will be smart enough to know when it's comparing two people to do the sort to use the underbar underbar LT.
Very convenient kind of thing.
Let's look at an example.
Let's look at some code.
So I'm going to set me to John Guttag, and him to Barack Hussein Obama, her to Madonna.
And we'll print some things.
So I printed him, and I printed him.getlastName.
I can now set some birthdays.
Let's look at this line.
How do you feel about him.birthday equals 8/4/61?
We'll come back to it.
But I want you to think about it.
And we see that Obama is-- well, we see their ages here.
Actually, we see that Madonna is older.
She looks really old when you look at that number of days, doesn't it.
Maybe she is.
Now what's going to happen here?
I messed up.
And I messed up because I went in and directly accessed the instance variable and assigned it what I thought was a reasonable representation of a birthdate.
But I shouldn't have because that's not even the appropriate type.
It's a string rather than something from DateTime.
So again, we see the impact of my having done this evil thing of violating the abstraction boundary and stuck in there try to directly access an instance variable.
We can do some comparisons.
And what we see is that I'm not less than Madonna.
I guess that's OK. You with me so far?
I can make a list of these things and print the list.
So it does a fairly nice job calling the underbar underbar STR operator.
And you'll note I had no trouble throwing objects of type person into the list.
No different than putting ints, or floats, or any of the built-ins.
So it all works nicely.
And then I can sort it and print that.
And now the lists come out in a different order because it's using the underbar underbar LT operator to sort the elements.
All of this is just by way of showing you how convenient it is to write code that uses a data abstraction.
If we go back and look at the code, we'll see that once again person was a subclass of object.
That's why we can do all these things we've been doing with it.
But now I'm going to start using the hierarchy.
MIT people are special.
I hate to say that because I know we have non-MIT people in the room.
But MIT people are special.
Well here's at least one of the special attributes of MIT people.
They all have an ID.
So I'm now going to say an MIT person is a special subclass of person.
So it has all of the properties of a person.
And the way we describe that is it inherits the properties of the super class.
And it adds a property.
We can now assign an ID number.
And now we're going to see the thing I promised to show you, which is a class variable.
Next ID num is not associated with an instance of MIT person, but it's associated with the class itself.
It's a class variable.
I can do that because classes are themselves objects.
And the advantage of this is every time I get a new instance of this class, I can assign a unique ID.
Similar to the way we were using global variables earlier in the term, typically once we have classes, we'd stop using global variables because we have these class variables, which can serve very much the same purpose in many cases.
And so now every time I get a new MIT person, I give them an ID and then increment the ID number so that the next person will get a different one.
So I added a property, this ID number property, which I find by get ID num.
I've also overwritten an existing property.
You'll note that I've changed the definition of underbar underbar LT.
So it's now saying we're going to compare two people not on the basis of their names but on the basis of their ID numbers.
So let's look at some code that uses this.
Get rid of the other code so it doesn't clutter up the screen.
So we'll look at some things here.
We'll get some MIT people called p1, p2, and p3.
And we'll print their ID nums by calling get ID num.
And you can see that Barbara Beaver gets 0, and the first Sue Wong gets 1, and the second Sue Wong has ID 2.
I can create even a third such person.
And now let's think about what happens here.
I'm going to print whether or not p1 is less than p2 and whether or not p3 is less than p2.
What should we get when we do this?
Somebody?
Pardon?
True and false
I heard a true false.
And indeed that's right because it's comparing IDs.
Now, suppose I want to compare names.
I could if I chose do this.
I could call person underbar underbar LT. And what this says is don't use the less than of the subclass.
Go up and use the super class one to do the comparison.
So it's going up and doing that.
I can do other things.
I can compare things for equality.
Let me just rip through all of these and see what we get.
Whoops.
Surprise!
Well, before we get to that surprise-- and it's not actually a surprise.
I shouldn't have uncommented it.
Let's look at the other ones.
We can say is p1 equal to p4?
And we discover it's not.
That's good.
And we can say is p4 less than p3?
That all works.
It's not.
But I can't say p3 less than p4.
And why can't I do that?
Why did I get an error message when I did that?
Yes?
Someone wanted to answer that?
That's because you mix [UNINTELLIGIBLE]?
Can you say that more loudly?
She's a person, not an MIT person.
So you didn't assign her an ID number
Exactly right.
The answer-- I hit somebody right on the head.
Now there's going to be a traumatic brain injury.
I'm going to get sued.
It's going to be messy.
Time to leave the country.
All right.
Because it looks at p3 less than p4, looks at the first argument, which is p3, and says OK, what's the underbar underbar LT associated with p3?
It's the one associated with an MIT person.
Let's go execute that.
And then it tries to retrieve the ID number of p4, which is not an MIT person, and it gets an error message.
All right.
Nothing subtle.
It's the same kind of thing we've seen all along when we use type errors.
In this case it's called an attribute error because we've attempted to access an attribute of an instance that doesn't exist.
And so we could catch it.
It's raised an exception.
We could catch it as we looked at Tuesday, but we're not going to do that because it's really a programming bug when that happens.
OK, let's continue.
We were interested in students.
So we're going to continue our hierarchy here.
And now I'm going to introduce a subclass of an MIT person called an UG, short for undergraduate.
Underbar underbar init is going to call MIT person underbar underbar init, which will give the UG an ID number and a name.
And it's going to introduce yet another instance attribute, or field, called the year.
And so the year, initially, is none.
Then I can set the year.
Though if I try and set it as something greater than 5, I'm going to raise an overflow error called too many.
No undergraduate should be a year greater than 5.
And I can get the year.
Let's look at what happens when we do that.
So I'll have two UG's.
Both happen to be named Jane Doe.
And then the same MIT person as before.
Let's run this code and see-- well, what's going to happen?
What's going to happen when I say print UG1?
The first thing it's going to do is going to say is there an underbar underbar STR associated with UGs?
And the answer is no.
That's OK because I know an UG is also an MIT person.
If I don't find it at the lowest level class, I'll bounce up and say all right, is there an underbar underbar STR associated with an MIT person?
No.
That's OK.
I'll go up another level and say, well, I know an MIT person happens to be a person.
And then they'll say oh, good there is an underbar underbar STR associated with person.
So I'll use that one.
So it looks at the class.
If it doesn't find it, it goes to the super class.
If it doesn't find it, it goes to the super class.
And it does that all the way up until the end, where at worst, it will use the built-in thing for printing objects because remember everything is a subclass of objects.
You don't want to do that.
You want to have something more elegant than object at location xe345, or whatever if would have printed.
Then we'll do some comparisons.
It will first look for the most local and then work its way up as needed.
OK?
Let's keep going.
We're going to introduce another kind of person called a graduate student.
And I'm going to write pass.
What that means is a G is an MIT person with no special properties, all the usual properties of an MIT person.
Does not have a year because graduate students could be here more or less forever, which goes like this when I say that.
Why did I introduce the type in the first place?
Because it lets me do type checking.
I can now check whether or not a person is a graduate student because an instance of G will have all of the properties of an MIT person, but it will have a different type.
It will be type G. And so I can now ask the question is the type of this object a G. Or is it an MIT person and get a different answer.
So I can do this.
And if I go type of G1-- well, that's interesting.
It says classmain.G.
It's going to upset its class.
And the class is defined at the outermost level, which is called main.
And the classes name is G, which is what we expect.
And so I could write something like that.
Type of G1 equals equals G. And I get True.
Ok.
So it's a handy thing to be able to do.
And in fact, I'm now going to bounce back to MIT person and add another method to it.
And this happens all the time when I'm programming that I go define a class, think I'm done, and then some time later decide that it would be convenient to add something new, another method.
In this case, I'm now adding the method is student, which returns type of self equal equal UG or type of self equal equals G. So this will let me distinguish a student from another kind of MIT person.
So for example, if I want to have a course list-- another class, this is a subclass of object.
You'll note I have an add student method, which takes self and who, maybe it should be whom.
And it says, if not who.isstudent raise type error not a student.
So I'm getting some leverage now out of this.
Now I wouldn't need to necessarily do this.
I could have said if not type of who equals G, or type of who equals UG.
But I chose not to.
The reason I chose not to is looking ahead, I might want to add some other students.
I might want to add special student, for example.
Or I might want to add cross registering student as separate types.
Now, the nice thing is I don't have to make a lot of changes.
I know there's exactly one place in my code that defines what it means to be an MIT student.
I go back and I change that method.
And even if I asked whether somebody's a student in a 100 different places, I only have to make one change to fix my code.
So I'm getting some modularity by associating the method with the class MIT person rather than every time I need to use it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. You can probably figure out, I'm not Professor Guttag.
He's away today.
And so he's asked me to cover this lecture, which I'm happy to do.
I'm asked to remind you that there is no recitation on Friday.
I was expecting at least a smile or a small cheer.
Early lead into spring break.
This ought to be good.
So my understanding is that at the end of last class, Professor Guttag was walking you through a set of classes.
And I want to remind you of where you were.
I want to look at one of those classes in a bit more detail, because we're going to show you one new piece of the language.
And then we're going to turn to a different topic, which is going to be the subject for the next several lectures.
So just to remind you where you were, Professor Guttag showed you a set of classes which started off with a person class.
There's a definition I'll put up on the screen.
This was just something that created a kind of object.
I'm not going to go through the details, but you had the standard things you'd expect.
You had a way of initializing it.
So you'd make each instance.
And it had a way of pulling out the name, so you could have a full name and a last name.
You could get out pieces of it, it set birthday, it had age, it had a bunch of stuff.
And that's fine.
The second thing that he did that we do in a lot of dealing with classes is he created specialization of that, a thing called an MIT person.
And you all know that MIT people are special people, right?
And what did an MIT person do?
Well, again, you can look at it.
It had, again, an init method.
But one of the things that was a little different here-- you may have seen it in some other places-- is that we can create, inside of that class definition, an internal variable.
In this case, it was the ID number.
So that every time we created an instance of this class, we could give that instance a special number and a unique number.
And that was exactly what we did here, right?
If you're going to create one of these things, you set the ID number to be the next number.
And then you would increment that, so that you'd be able to, next time you created an instance, give it a unique number.
And that let you do things like put things out in order, and things of that sort.
There was also a method in this class that we're going to use in a second, which is something that would say, is this person a student?
I'll just highlight for you there the definition of it, which said you'd look at the type of the instance and say, is it undergrad or grad?
And that was the other two specializations we had here.
We had undergraduates and we had graduates.
And again, I'm not going to worry about the details.
You saw this last time.
Undergrads, they have some methods like you could create an instance of it -- you'd have its year.
You could give back what the year was.
And for the graduate students, we didn't even bother with putting details in.
It could be anything you want.
Now these are things that you ought to be getting used to looking at.
So this is creating some class definitions.
I can make some instances.
There are a couple of nice variations like, for the MIT people, each one has a unique identifier, which lets us keep track of them.
I think where Professor Guttag ended up last time was actually creating a course list.
So a list of students in a class, like 6.00.
So I want to show you that, and I want to show you some highlights of that.
So there's the definition.
It's on your handout as well.
So what does a course list do?
Well, the idea should be, it's a way of collecting together all the students in 6.00.
So if you look at our init function up here, when we create an instance of a course list, what does it do?
Well, we give it a number, like 6.00.
And it's going to bind that locally inside of a definition.
And it's going to create a collection, a list.
It's going to be a list of all the students.
Now if I'm going to do that, I need to have a way of putting students into the class.
And I know nobody drops 6.00, but just in case-- drop date's coming up-- we have to have some way of removing a student from a class.
So I'm going to have a method right here-- oops, sorry, I'm going to get it from the right place, right there-- that would add a student in.
Now take a look at that for a second.
It's not just putting the student into the list.
The very bottom down here, that last line, self.students.append().
What's that doing?
Well, self, remember, refers to the instance.
.students will get me that list for that instance.
And then .append says, add on to the end of that list the person that I just added in, the person that is bound to who.
The things above it are being careful.
They're defensive programming.
They're ways of saying, look, make sure that the person is actually a student.
Huh.
Wait a minute, where'd that come from?
isStudent.
Oh, yeah, that was up here in MIT person.
So we're making an assumption here that says, we're only adding in students.
No post-docs are welcome.
We're going to only add in students.
So we're taking instances of one of these two classes, and checking using the inherited method to make sure that this person is actually a student.
And we're doing this in a careful way, which is we're going to raise an error if, in fact, it's not a student.
And then the second thing, notice, that we're going to do here is we're going to say, gee, make sure that we're not adding you twice.
So we're going to do what?
We're going to say, get out the list of students right there.
and I begin.
self is pointing to the instance.students gives me the list of students currently in that instance.
And I'm basically saying, is this person in that list?
And if they are, again, I'm going to raise an error saying, wait a minute, you're already in the class.
So these are things I'm sure you've seen before.
But they're ways of being a little careful about the programming.
The removeStudent is roughly the same.
So let's just look at some examples here.
I've created a set of people.
And just to show you what they look like, let's go down here.
So I've created m1.
Let me do that again, sorry about that.
I'm going to run this.
All right.
So if I look at m1-- it's not liking me, is it?
We'll try it one more time.
I'm going to look at m1.
It gives me back an instance of an object.
[UNINTELLIGIBLE] if I actually say a print of m1, it'll print out the name.
And the same thing for the undergraduates here.
So I've made a couple of undergraduates, I've made a couple of graduate students.
And notice what I've done down here.
I've created a course, SixHundred, by calling CourseList.
So CourseList is going to make an instance of that.
I gave it the name.
So we can go back over here.
Let's look at what SixHundred says.
OK.
I'm sure you're used to reading that gobbledygook.
It says, ah, it's an instance, and it's an instance of a CourseList.
And it gives me a pointer to where it is.
Now what if I want to see the students inside this course?
I'd like to see, who are all the students there?
So printing SixHundred-- let's see if that does the right thing.
Oops, no.
That kind of makes sense, right?
SixHundred is just pointing to an instance.
So where are the students?
They're sitting down inside of an internal variable, which is, in fact, in this case, the list of students.
So one way I could do this is I could say, take SixHundred, which is an instance, and go into that frame that corresponds to that and look up the local variable, students.
And if I do that, oh, I've got a bunch of students named Main.
Now this shouldn't surprise you, right?
This is doing what?
It's giving me back that instance of the list.
If I actually want to print out the names of the students, I have to do something a little more careful.
So here's one way I could do it.
I could say, for every student in that list, let's print out their name.
Oh, good.
You're wondering, why am I doing this?
Well, I want to show you what's going on in this class.
And then I want to highlight something for you.
So let's think about what I just said.
I want to get the names of all the students in the class.
So I took the instance, I went in and pulled out an internal variable, and then I did a for loop that walked along it, printing those out.
Is that a good idea?
Is that a not-so-good idea?
Is that a really bad idea?
Don't you hate professors who ask questions at 10 o'clock in the morning?
Well, I'm going to suggest that it's not a great idea.
And here's why.
I'm reaching into an instance and pulling out an internal variable and doing something with it.
A much cleaner way of dealing with that is to build an interface, to build a function, a method, that belongs to that instance, that would give me back the pieces I want.
And the reason I'd like that is then I don't have to worry about, if I change my mind in terms of how I represent things internally, all I have to do is think about that interface function.
Right So the last thing I want to show you here in this example is let's look, in fact, right here.
I've added a method called allStudents.
But it's a little different, I think, than things you've probably seen before.
So what's the idea?
I want allStudents.
To be a function, or a method.
When I call that, I want it to give me back all of the students.
Now, I could have had it just give me back the list, but I might want to have the method give me back not just all the students, but maybe some students, like only the undergraduates in the class.
And in fact, right below there, I've got a method that does that.
Let's look at this one.
It says, I want to get back the undergraduates in the class.
And what's it do?
It's going to do a little counting.
It's going to set index to 0.
And then it's going to run through a while loop where it basically takes each student and checks its type.
This is something you've seen before, taking a loop and walking along, taking each element of that list and saying, if it's an undergraduate, I'm going to give you back that student, and I'm incrementing index at the bottom.
Here's the funky thing.
I've used a word called yield.
And that's different than you've seen before.
I'm not directly accumulating a list.
You don't see anywhere where I'm building up a list here.
So yield is an example of a form that you're going to want to use called a generator.
Now a generator is a little bit like a return, but with one big exception.
If I had a return in that method somewhere, when it found the first thing, it's going to give me back the value and it's done.
And so any information it stored about where it was in that computation-- the phrase we use is it gets popped off the stack frame.
Yield is a little different.
And it's different because a generator-- I'm going to write this down carefully.
A generator is a function that remembers the point in the body where it was.
It remembers the point in the function body where it last returned, plus all the local variables.
So what does this mean?
It says, it's going to keep track of what was the state when I did the computation, so that if I call it a second time, it goes right back to that state and continues the computation.
If I called it a third time, it goes right back to that state and gives you a continuation on the computation.
So in fact, what yield does, what a generator does, is what you would do what you normally walk through a list.
But it gives you some control.
So it gives you a way of actually deciding, how do I want to create something I could use in a for list.
And if we go back and look at it, you can see it right there.
The method definition for UGs, undergraduates-- if I call it, it is basically going to walk through, one at a time, each element in the underlying representation and generate for me the next element of the thing I want to use.
So let me show you a couple of examples of using it.
In this case, I've buried inside of a for or a while loop, but if I go over here, I should be able to do things like, are all the students in SixHundred?
And notice the open/close paren, because that's a method.
I need to call it.
Now you're going, whoop-de-do.
That's just like the print thing I did before.
But the difference here is I'm controlling now how I get access to those.
And to show you just a slightly different way of doing this, I could now do things like say the following.
For all the students in SixHundred, and I'm going to take all the students again in SixHundred.
So notice what I'm doing, two loops.
And I'm going to create something that basically keeps track of where I am in each one of those loops using that yield function.
So take a look at what it printed out.
That outer loop is using the generator to generate each element of the list.
It hangs onto that, so that the inner loop can walk through again, a generator going through each element of that list.
And I'm printing out students squared -- first student times all other students, second student times all other students, third student times all other students.
And of course the other thing I can do is I can do the same thing-- sorry about that-- for s in SixHundred undergrads.
And in this case, it's walking through that internal representation, generating each element that satisfies the constraints.
OK.
It's a little esoteric, but you're going to find it really useful.
And it's the last piece, I think, of syntax you're going to see for a while in terms of the language.
You could do this by just manipulating the list directly.
But the idea of a generator you're going to find really useful when you want to control access to a collection.
Really think of the generator as something that keeps track of the state of the computation, so it can go back and pick it up whenever it needs it.
OK. That wraps up that piece of what Professor Guttag was doing.
So I want to now switch to a completely different topic, which we're going to do today, we're going to do Thursday, and it may well carry into after spring break.
The second topic deals with, how do we go about building computational models to solve real problems.
And I want to start by putting up here a comparison.
For much of the history of science, people focused on trying to find analytic methods.
Big word, what does it mean?
Well, an analytic model is something that lets you precisely predict the behavior of a system, just based on some initial conditions and a set of parameters.
So this is a mathematical function, if you like.
You can predict behavior given some initial conditions and some parameters.
Newtonian physics, 8.01-- analytic models.
Spring constants-- analytic models.
They're things that let you predict what a system's going to do.
For, I don't know, several centuries, this was basically what science did.
It was really useful.
It lead to things like calculus and to things like probability theory.
It lead to basically understanding the macroscopic physical world.
So it's the stuff you saw in high school.
That's nice.
But it doesn't always work.
And indeed, as the 20th century progressed, one of the things we saw was that there are places where this doesn't really satisfy what you need, and you're better off with simulation methods, which is what we're going to talk about here.
So what do I mean by simulation methods?
First of all, let me talk about why do we want to get a simulation method.
The idea was that there are going to be some places where it's really hard to build a model.
So in fact, places where this comes up-- sometimes we have systems that are not mathematically tractable.
Again, a highfalutin word.
What does it mean?
It says, there are some systems where it's really hard to build a physical model that exactly predicts what's going to happen.
Think weather forecasting.
It doesn't work well.
Actually, in Boston, it's easy.
Just assume it's going to snow, and you're fine.
All right.
That's one reason why we want to move towards simulation models.
Here's a second reason why we'd like to move towards simulation models.
There's times when, in fact, as things get more complex, we're better off just successively refining a series of simulations.
In other words, rather than spending the effort trying to build a very detailed predictive model, analytic model, just run a simulation that gives us an insight to what the problem's doing.
We'll refine the simulation, and we can keep going on like that.
The third reason you might like to do it is it's often easier to extract useful intermediate results from a simulation than it is to try and build a detailed analytic model.
And of course, the last one is pretty obvious.
The advent of computers made this possible.
I want to just give you an example of this.
Put yourself back in the 1700s and think about how you would do a computation.
You have two choices.
You can crank it out by hand.
Good luck.
Imagine how detailed a model you could actually do that way.
Or you could build really precise, detailed mechanical models.
And in fact, there are wonderful collections of these from people like Leonardo.
For example, people that wanted to understand the solar system built what were called mechanical orreries, which were these very complex pieces of clockwork that allowed you to try and predict the motions of the planets.
That's a really expensive way to try and do a simulation.
So computation suddenly makes things change a lot.
Now just to give you a sense of this, I did the following experiment this morning, to see how important simulation is.
And I haven't really said what simulation means.
So simulation means giving me an estimate, rather than a prediction.
Giving me a sense of what might happen to a system under certain conditions and doing that multiple times, actually running, if you like, a model of the system rather than trying to predict exactly what's going to happen.
So I did the following experiment this morning.
I went to Google-- I could have gone to Bing-- and I typed in "finance simulation."
I got 5 million hits, most of them probably wrong by the way, but that's OK.
I typed in "biology simulation."
I got 11 million hits.
I typed in "physics simulation."
I got 15 million hits.
And then finally, I typed in "game simulation."
How many hits do you figure I got?
North or south of 15 million?
North.
Yeah.
50 million hits.
OK, so why should you care?
A, it's fun to do.
B, it's really a common tool that we want to use here.
So let's think about what a simulation would look like, and then we're going to start building one.
So the idea of a simulation, then, is-- what we're trying to do is we want to build a model with the following property.
It's going to give useful information about the behavior of a system.
Boy, there's a statement that has no content to it, right?
Let me tell you what I mean by that.
I'm going to compare this to an analytic model.
An analytic model would exactly predict what this system's going to do.
With a simulation, we want to build a model that says, if I give you some sense of the state of the system, it will give me some information about how that system is going to behave.
It may not be exactly right, But it's going to give me some simulation of it.
Another way of thinking about it is it's going to give me an approximation to reality.
And another way of saying it is simulation models are descriptive, not prescriptive.
So what does that mean?
An analytic model is prescriptive.
You're doing 8.01 problems.
You type in the definition of the parameters of the problem, and it would tell you, at least to the accuracy of the computer, exactly what's going to happen.
With the simulation, it says, given a particular scenario, I can give you a good guess of what's going to happen.
But it might actually be the case, by the way, that for exactly the same scenario, I run the simulation multiple times and I might get slightly different answers.
Because maybe the world, as we said over here, is something that we can't model exactly mathematically.
So we want to be able to have that ability, to sort of go back and forth.
Now probably the easiest way to think about this is let's look at a model and a simulation.
So the idea here is I want to build simulations.
I want to control the fact then I may not get a precise answer.
I might not get the same answer every time.
But if I do enough simulations of a circumstance, I can get some good sense which I can refine of how this object's actually going to behave.
So here's the example I'm going to start with.
I'm going to go back, again, a couple hundred years.
In 1827, a Scottish botanist named Robert Brown observed that a pollen particle suspended in water seemed to just float at random.
You've probably heard this term.
It's called Brownian motion, named after Robert Brown.
He had no plausible explanation for this, by the way.
He just observed it.
And he made no attempt to model it mathematically, which kind of makes sense.
That's 1827.
The first really clear mathematical model of Brownian motion didn't come around until 1900.
A guy named Louis Bachelier had a doctoral thesis called "The Theory of Speculation."
Part of the problem for Bachelier, though, was that he didn't model Brownian in pollen particles.
He did it in finance markets.
And that was considered, at least at the time, completely disreputable.
And so nobody paid any attention to his thesis.
That's not going to happen to your thesis.
Your thesis is going to be reputable when you get done with this place.
So it took eighty years to get to that point.
Unfortunately for poor Bachelier, five years later, another person came along and actually built the kind of model that introduced this sort of-- what's called stochastic thinking into the world of physics.
This was a model that was almost exactly the same as Bachelier's, but it was used to confirm the existence of atoms.
Anybody know who did that model in 1905?
No physics buffs here.
If I told you he was born in Switzerland, would that help?
A minor guy named Albert Einstein.
So that was one of the first things Einstein did, was he actually built the first really good model of Brownian motion.
And that allowed, in fact, this kind of thinking to go into real-world problems.
So Brownian motion is an example of a tool we're going to use a lot.
I'm probably hiding that below the screen where you can't see it.
It's an example of what we call a random walk.
Random walks you're going to see all over the place.
They're an incredibly useful way of building a simulation.
And the essential idea of a random walk is, if I've got a system of interacting objects-- could be pollen particles-- I want to model what happens in that system under the assumption that each one of those things is going to move at each time step under some random distribution.
It's going to move in a particular direction.
I want to model what the overall system does.
OK, let me give you some examples of where this is really useful.
It's really useful in modeling physical processes.
Well, we're going to start with pollen particles in air.
But you could think about any kind of particle in water.
You could think about any kind of air particle in a larger fluid.
Ah, weather.
Modeling weather-- if we could really model the motion of all those molecules-- is just a really large random walk.
Random walks are really useful in understanding biological processes.
For example, the kinetics of displacement of RNA from heteroduplexes of DNA is a great example of a random walk.
And in fact, people interested in bioinformatics or computational biology will see random walks used all the time to try and to understand the displacements of things.
It's really useful in social processes.
Movement of the stock market is definitely a random walk, except for the day when the markets are all crashing for unfortunate reasons.
So it's something we want to use a lot.
In the example I'm going to use to motivate a random walk, we're going to build a simulation.
Here is what was the traditional view of a random walk.
So here's the motivation.
Excuse me a second.
Let's take a drunken university student.
Not an Institute student, a University student.
So this is a Harvard student, not an MIT student, because I know you're all well-behaved.
At least smile at me when I make these bad jokes.
Thank you.
All right.
You've got a drunken student.
They're out on a big field.
And they start off in the middle of the field, and every second, this student can take one step in one of the four cardinal directions.
So north, south, east, west.
After 1,000 seconds-- after 1,000 steps-- how far away is that student from where he or she started out?
So I need some help here.
Guesses.
1,000 steps.
How far away after 1,000 steps is that student-- we'll make it a he.
From where he started out.
Yeah
It depends on the probability of what direction you're going
OK, so let's assume that the steps were equally likely.
The four steps-- north, south, east, west-- they're all equally probable
[INTERPOSING VOICES]
OK.
So the suggestion over here is, where he started.
So 0 distance away.
That's not a bad guess, right?
You're just going in different directions.
It's equally likely you end up where you started.
So one possibility is 0.
You end up back where you started.
Yeah
Maybe 150 or so?
Well, I'm going to take bets here.
I've got 150.
It's an interesting number.
Not bad.
Any other guesses.
This side of the crowd.
I don't want to just do the right side.
I'm very liberal.
I want the left side of the crowd over here.
Somebody over on this side.
Help me out.
Any other guesses?
I've got 0, and I've got 150 steps.
Boy, I get no eye contact when I do this.
This is great.
Yes, please
500 root 2 steps?
500 root 2.
Can I drop the root 2 for second?
The 500 is not a bad guess, right?
Because you're about half the distance away.
The root 2 we'll come back to in a second.
Keep that in mind.
All right.
500.
So we've got 0, we've got 150, we've got 500.
Well, gee, that's the whole point of building the simulation.
Let's see if we can figure out what might happen here.
Ok. And I'm going to build the simulation in a second.
But before I do that, one of the good things to do is to try and build a simple model that we could use to get an intuition about what's going to happen.
So I'm going to start with a very simple model here.
There's my field.
The guys who mow the lawn at Fenway have been over and very nicely put x- and y-axes onto this field.
And let's assume the students starts off there.
We'll just call that the origin.
It doesn't matter what it is, but that's where the students starts off.
And what I want to do is I want to get a very rough sense of how far away the student might go after a few steps.
We won't be able to go very far, but enough steps that we can start estimating this.
So they start there.
After one step, there are the places the student could end up.
Equally likely.
So after one step, if I want to build a distance associated probability, there's probability 1 they're 1 unit away, no matter which direction they went.
OK, let's take this one here.
Let's assume the student went east.
In fact, they're all going to be roughly the same.
And the second step, where can the student go?
He can go there, he can go there, he can go there, or he could go there.
Right?
So what are the distances here?
Well, in one out of four cases, he's at a distance 0, which would be back to where we started, which is that original pretty good guess.
If he goes there or he goes there, those are both what distance away?
Root 2, right?
So it could be root 2 away, 2 out of 4 times.
And if we went east again, we're 2 units away.
Again, with probability 1 in 4.
I'm cheating slightly here, in the sense that I should do the same things if the first step was north, south, or west, but they're all symmetric to this.
So it'll all come out the same way.
So there is what happens after two steps.
Now let's do one more step.
It's going to get a little messy here.
But suppose I want to go three steps.
About three possibilities.
Well, I've got three possibilities.
If I'm in this case, which is the 0 case, I'm here, in which case no matter which step I take, I'm 1 unit away.
So this will say, distance 1.
And the probability of being here was 1/4, so the probability again is still 1/4 that one unit away.
Let's take one of these, the root 2 ones.
That says I'm either here or I'm there.
They're going to be symmetric.
If I'm here, where can I go?
I've got possibilities going to those.
So these two are 1 unit away.
So half the time I'm 1 unit away, and it was 1/2 that got me there.
Did I do that right?
Yes.
That's also with 1/4.
That was getting if I went here or I went there.
If I'm up here or I'm over there, that's root 5 away.
So let me put that over here.
So I've got a probability of 1/2 getting there, and a probability of 1/2 from here of getting that, so that's also with probability of 1/4.
And then what's the one case I have left?
The case I have left is I went all the way over to here, two steps over, in which case I'm going to go what?
There, there, there, or there.
If I just pull these together-- I'm just going to do them for you.
That says, I have a probability of being 1 away with probability 1/16, root 5 away with probability 1/8 -- and I'm running out of room here -- and 3 units away with probability of 1/16.
Don't sweat the details.
What you can see is I'm just building this up.
What I want you to see is what do the distances look like?
After one step, I'm one unit away.
After two steps, that was this.
You can kind of do the math in your head.
What's the expected distance away?
What's the average distance I'd be away?
Well, it's 0 times 1/4, plus root 2 times 1/2, plus 2 times 1/4.
So that's 1.4, that's 2.8, and that's 4.8 divided by 4.
So it's about 1.2.
Now I'm sure you can do the math for three steps in your head, right?
Yeah, right.
But if I do that, if I add those things up-- you can go do this yourself-- what you'll find is that this is about 1.4 or 1.5.
Let me make it 1.4.
It's probably a little bit closer.
All right.
Why was I doing that?
This is something you want to do as you start building a simulation, which is do a little bit of a calculation to get a sense of what you expect to happen here.
So what conclusion could I draw from that?
Admittedly on very small numbers, it looks like the more steps the Harvard student takes, on average the further away they are from the starting point.
So your 0 was a great guess, but it's probably not going to work here.
And we want to see why.
On the other hand, 500 that looks-- well, maybe we're still OK. Three steps, half of 3 would be 1 and 1/2.
So maybe the 500's OK. Let's see what happens if we do this.
So what I want to do is see how I could build a set of classes that would let me build this simulation.
And part of the design process here is I want to try and invent classes that correspond to the types of things I expect to see happening.
So in this case, what do I have?
I need to model a drunk, I need to model a field, which is where the drunk's wandering around in.
And I'll need a third thing, which may not be obvious immediately.
But the third thing I'm going to need-- so let me just put the pieces up.
I need a drunk, I need a field, and I need to keep track of where the drunk is in the field.
So I'm going to pull out another class I'm going to call a location, that's going to tell me where the drunk actually is.
OK. With that in mind, I'm going to show you some code.
I'm going to walk through it reasonably carefully, just to let you see what it looks like.
So there is my class definition for a location.
Now at this stage in the course, hopefully you can look at that and already see the main pieces of it.
So what do we have in here?
Well, we going to have an initialization.
You can kind of see that right there.
When I create a Location, I'm giving it an x and y. I'm going to make an assumption, which is they're floats.
I'll come back to that in a second.
And that's just going to store them away in some internal variables.
That's things you've done before.
So self.x gets that value, self.y gets that value.
And of course, I could get out the values right there.
I need to know where the drunk goes.
So one of the things I'd like to do is to say, given the current location of the drunk, I might like to know how far away is that from some other location, like the place the drunk started.
So that last method down here, distance from-- notice what it's going to do.
It says, if you give me another, which is another location-- remember, self will point to my instance-- what can I do?
Well, I can just do Pythagoras's theorem to figure out how far away am I. I get the x- and y-coordinates of the other location.
I get my own x- and y-coordinates.
And then I just take those distances squared and take the square root of them.
So it's just giving me distance away, which is pretty nice.
There's one other method here which is a little funky.
This is a design choice.
And that method is the one up here called move.
So the background to this is going to have a drunk at a location.
one of the things I will want is to be able to change location.
So move says, if you give me a change in x and a change in y-- I'm going to assume they're floats-- I will give you back the new location, which is to go from my x and y by some amount.
It could be in the east direction.
It could be in the north direction.
But notice what it returns.
It returns a new instance of a Location.
OK.
I made a couple of assumptions which are worth thinking about here.
So one assumption is I'm assuming that this is a 2D world.
You can't elevate.
You can't rise up.
There's no change in altitude.
That kind of makes sense.
The second assumption I made was I said, I want to build it in so that things like delta-x and delta-y are floats.
What is that doing?
Well, that's saying, I don't want to restrict myself to just moves that are only in the cardinal directions.
I'm going to start there, but initially, I'd like to have the ability that the move could be along a diagonal, or it could be some partial step.
So notice I'm making a choice there that is kind of nice.
All right.
That gives me locations.
And that's a pretty straightforward class.
So I've got locations.
That's always a good start.
I have this.
Now again, I want to model a drunk wandering around in a field.
So the second thing I need is I need to say, what's a field going to be?
And I want to show you that definition, which is right here.
And a field is basically going to be something that maps drunks to locations.
So a field is just going to be a collection of drunks, keeping track of where they are.
So let's look at what that definition says.
When I create a field, right up here at the top it sets up a variable inside of that instance, just called drunks.
And we're going to use a dictionary.
We could use other choices, but the dictionary is going to be nice, because it's going to let us keep track of the drunks.
I can add a drunk to the field.
So notice what it says.
It says, given I've got a field, I'm assuming I have a drunk and a location, because the drunk's got to be in a particular location.
Notice what it does.
First of all, it checks to make sure that I don't already have this person in my collection.
So that first check there says, if this particular drunk is already in my dictionary, I complain.
You can be drunk enough to see double, but you can't be drunk enough to be double.
Another bad joke right off the back wall.
Great.
OK, so what else do I want?
Otherwise, notice what I'm going to do here.
I want to make sure you can see these things fairly cleanly.
If I'm going to add a drunk, what do I do?
Into the dictionary, under that label of drunk, I'm going to insert a value corresponding with that label.
And the values going to be the location.
So my representation here is now a dictionary of drunks and where they are.
And that's going to be handy as I want to use this.
And again, if I want to get out the location, well, I just return exactly that.
The final piece is, when I go to move the drunk, I've got some tests here to make sure that I actually have a drunk.
But otherwise, I'm going to ask the drunk to take a step right there.
We have to say what a drunk does.
That's going to give me back a change in x and a change in y.
And then notice the funky little call below here.
This says what?
The right-hand side says, I get the dictionary of drunks.
I go in and index into it to get out the actual drunk.
That's this first part right here.
That gives me back what?
It gives me an instance.
So the dot says, for that particular drunk, get its move method, which was that call, and have it move by that amount.
And that returns for me then a location which I can store back into the list.
All right.
I made some assumptions about Field.
They're worth pointing out.
No constraints on locations.
How big is this field?
As big as you want.
So I didn't build in a real physical representation of the limits of the field with every location represented.
I just said, it's just a collection of drunks.
So there's no limits on how far the drunk can go.
The second assumption I made here is I can have multiple drunks.
And we're going to have a simulation of that in a while.
You can imagine it's going to be fun watching them collide with each other.
All right.
And the third thing is, it says nothing about the patterns in which the drunk moves.
So this is a design to keep this clean.
It says nothing about how the drunks move.
It simply says, give me a change in x and y. I'll change the location.
The last piece is we're going to capture how the drunk moves in a drunk.
And there's the definition of the drunk.
What does that say to do?
Well, we're going to give it a name, to start it off.
And the only method it has is that thing that's going to do a takeStep.
And here's where we're going to build in the choices about how they can move, what you asked about earlier.
Notice what I've done there.
In this case, I've said, I'm going to pick at random from this set, stepChoices.
And that's where I'm building in the idea that it's only going to move plus or minus 1 in the x and y direction-- north, south, east, west.
I could clearly change that.
I'm also building in here that it's equally likely, because random picks one of those equally likely.
I might decide to change that.
And I can certainly do that if I want to move to other directions.
But right now, that's where I'm going to build it in.
OK.
So now we're ready to try the simulation.
We've got what?
We've got location.
That's just representing a place.
We've got fields.
That just lets us map drunks to locations.
And then we've got the ability to move the drunk around.
So I can now think about making this actually move around.
So let me show you a quick example.
Let's create a drunk.
I'm picking the obvious person.
I'm going to create a location.
We'll call the origin just the spot at 0, 0.
And I need to create a field which I can do.
And then I'm going to add Homer to this field at the origin.
And now the last thing I want to do is I want to have Homer wander around.
I want to see how far he goes.
So I'm going to build a little function over here-- you can see right there-- called walk, which takes the field, takes a drunk in the field, and says how many steps do I want him to stagger around with?
And what's it going to do?
I'm going to set the start location to be where the drunk was initially.
It might be at the origin or it might be somewhere else.
And then I'm just going to run through that number of steps.
So range is just saying, take the number of steps, just moving the drunk.
And then I'll return how far away he is.
So if I do that here, I can do walk with that field, with my drunk, Homer.
And let's take 10 steps.
Hmm.
That's interesting.
Let's try it again.
I was getting worried there for second.
So we're getting an interesting range of values.
So here's the last thing I'd like to do.
Let's actually do a simulation of this.
And I'm going to do this reasonably quickly.
simWalks right here is simply going to do what I just did, but multiple times.
That is, it's going to take a drunk, a field, some number of steps, and it's going to run a bunch of trials, however many trials I want to get.
So I do what I just did over here, and collect those together.
And then I can just print out some statistics about how far away did the drunk get as I do that.
And that's what drunk test is going to do.
We're going to do it for 10 steps, 100 steps, 1,000 steps, and so on.
So let me simply run that.
I run drunkTest.
And we'll do it with 10 trials each.
OK.
Does that look right?
Does that look good?
You can tell this is a trick question.
So what should you be looking for there?
Look at it.
It says, if I take 10 steps, on average I'm about 2 and 1/2 steps away.
If I take 100 steps, on average I'm less than two steps away.
1,000 steps, I'm about two steps away.
10,000 steps, I'm about two steps away.
100,000 steps, on average I'm about two steps away.
Wait a minute.
Didn't we say that it looked like, over on that board there, that as I increased the number of steps, I got a little bit further away?
Hmm.
So I've done this deliberately.
This suggests what?
The first key point, think about what your simulation should return.
What are good examples to think about?
Because this doesn't look right.
OK.
So here's how I could test it.
Why don't I try a simulation on values for which I know the answer?
So I'm going to go back in here, and right here, I'm going to replace this with-- we're either going to take zero steps or one step.
Let's run that.
Oh, don't do that to me.
Oh, yes, I do have an error.
Thank you.
Get rid of it right there.
Almost out of time, so this is perfect.
And now let's try drunkStep again.
Wow.
If I take a random walk taking no steps, on average I end up 2 and 1/2 units away.
I teleported.
That's pretty amazing.
And with perfect timing, we'll figure out what's wrong with my simulation next time.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
I want to start today's lecture with, I guess one could call it, a confession.
This isn't a real 6.00 lecture.
I'm standing here on June 30th in an empty classroom dressed, for some bizarre reason, as if it's the winter.
I guess it's what the video folks would call continuity.
What happened is Professor Grimson gave a beautiful lecture 13, for which we have a lovely picture and no sound.
We decided that probably those of you watching this in OpenCourseWare would be unhappy just to watch Professor Grimson and not hear him, so we're now re-taping the lecture.
This is, actually, one of two lectures we're going to be re-taping because we had video problems.
So when I say something hilariously funny, and there is no laughter in the room, it's because the room is empty.
Do me a favor and laugh out there in video-land, so maybe at least someone will respond.
OK, the last lecture closed with a slight mystery, not a great mystery, but a little mystery.
We ran our simulation of the drunkard's walk and got a result that wasn't credible.
How did we know it wasn't credible?
Well we had worked out some details on the blackboard of what we thought would happen with a small number of steps, and the results we got didn't match that.
That told us we had something wrong.
We asked you to go think about it and come back today prepared to tell us what was wrong.
Well since there is no one here to ask, I'm not going to ask the class to fix it, but instead I have had the laborious task of fixing it myself.
So let's look at it.
The problem was in SimWalk.
And what we had is we had the wrong argument here.
We had numTrials Instead of numSteps.
And so it didn't make any sense.
I've now fixed it, and I, now, want to run it.
Well clearly, what I should do to begin with, is run it on some examples for which we already know the answer.
So I'll change drunkTest to instead of running it on large numbers of steps, run it on just a few.
And let's see what we get.
We run a lot of trials here, since it's a short trial.
And what we see is for 100 trials of 0 steps, the mean was 0, the max was 0, the min was 0.
Well that's exactly what we saw when we looked at it on the board.
It should happen.
And the same thing with 1.
Everything worked the way it was supposed to work.
It doesn't tell us the program is perfect.
It does tell us it works in at least two examples, which was better than we had last time.
All right now let's look at it on a larger set of examples.
I think I won't run 100 trials here, because it'll take a little longer than we want.
So let's run maybe 20.
Let's see what we get.
Well it's running through here.
Now we're getting examples that seem much more credible.
When we take 10 steps, the mean is 2.85, and the max is between 6 and 1, or the max is 6, the min is 1.
It's what we would hope for, that there's some dispersion there.
And as we get up higher, we see that for a 100,000 steps, the mean is 248, and there's quite a spread between the max and the min.
Finally we get to look at the question that had us writing this code in the first place-- how far should we expect this drunk to be, given a particular amount of time.
Well we can look at these numbers and try and think about them in our heads, but in fact, it's a lot easier to look at a picture.
And this will get us to a theme that we'll be getting to shortly in the course of how do we visualize data.
So here we have a visualization.
What I'm plotting, here, is the mean number of the mean distance against the number of steps for this random walk.
And what we can see is, as we knew, at 0, it's 0, and then it sort of goes up.
And I've taken it only up to a 1,000.
And at 1,000 we're somewhere, it looks like, around 25 steps.
So we can learn something.
Well what can we learn from this?
How fast is it growing?
Well it seems to grow pretty fast, and then it flattens out.
It looks to me like, roughly speaking, the distance is growing as sort of close to the square root of the number of steps.
Well I could look at it more closely.
We, actually, know this is not exactly the square root.
Right, the square root of a 1,000 is not going to be 25, but I'm not going to delve into the details of that.
It's, actually, a little bit complicated.
We could derive it, but we won't, because I want to get to what I think is a more important message.
How much should we infer from this or from the numbers I displayed before, and I want to say not too much.
Because what we saw, if we go look at what we had before, is quite a dispersion between the max and the min.
And furthermore, if we run it again, the 20 trials, we're getting different answers.
So what you can see, here, is for 10,000 steps, here, my mean was 78, and here, for 10,000 steps, my mean is 90.
Here, the mean was 279.
Here, it was 248.
The maxs and the mins are different.
So I don't want to read too much into that graph.
Now one of the issues we should have asked is, well it's the mean of how many trials.
I didn't tell you.
And to be honest, I don't quite remember.
I think it was 20.
But I don't have enough information to interpret it.
I need a lot more and that's going to be what this whole next unit of the course is about is how do we think about the results of programs when the programs themselves are stochastic.
And this is important because, as you will see, not only in this course, but as you progress in your careers, if you're involved in engineering or science, is that almost everything in the real world is stochastic.
And in order to think about it, we have to really think pretty hard about what those things mean.
So I want to, now, pull back and talk a little bit, sort of, philosophically about that, about the role of randomness in computation-- something you probably haven't seen a lot, if at all in the other courses you've taken if you're a freshman or sophomore, and that's because there's something really comforting about Newtonian mechanics.
When I first learned physics, it was really comforting.
I learn the physics of Isaac Newton.
You push down on one end of the lever, the other end of the lever goes up.
You throw a ball up in the air, it travels a parabolic path and lands.
F equals MA, that wonderful rule of physics.
Everything happened for a reason.
It was predictable.
It was a great comfort to some of us about it.
And for centuries, that's the way the world thought, from almost the beginning of the times of science to, really, if you look at history of science, to today, people believed the world was deterministic.
And they liked it.
Then along came the so-called Copenhagen Doctrine and quantum physics in the 20th century, and the comforting world of Newtonian physics disappeared.
The Doctrine, the Copenhagen Doctrine, led by the physicists Bohr and Heisenberg, argued that at its most predictable and most fundamental level, the behavior of the physical world cannot be predicted.
One can make probabilistic statements of the form x is highly likely to occur, but not statements of the form x is certain to occur-- period.
I can make the probabilistic statement that if I take this laser pointer and put it on the table, it won't fall through.
The molecules won't separate in such a nice way that it just drops through, but I can't promise it won't happen.
Truth is I'm going to make that promise anyway, because I believe in probability, and it's highly, highly probable.
But they tried to say that, in fact, the world is all stochastic.
Everything is probabilistic.
Other distinguished physicists at the time, most notably Einstein and Schrodinger, vehemently disagreed.
This debate, it's hard to believe, actually, roiled the world of physics, philosophy, and religion.
The heart of the debate was the validity of something called causal non-determinism.
The idea here-- causal means caused by previous events.
So causal non-determinism was the belief that not every event is caused by previous events.
Einstein and Schrodinger found this view philosophically unacceptable, as exemplified by Einstein's often quoted comment, "God does not play dice."
What they argued is for something called predictive non-determinism.
The concept here was that our inability to make accurate measurements about the physical world makes it impossible to make precise predictions about the future.
So this distinction was nicely summed up again by Einstein, who said, and I quote, "the essentially statistical character of contemporary theory is solely to be ascribed to the fact that this theory operates with an incomplete description of physical systems," i.e., things are not unpredictable, they just look unpredictable because we don't know enough about the initial states.
This question is still unsettled in science.
The good news is it probably doesn't matter at all what the truth is.
However you want to look at it, we have to assume that the world is non-deterministic, because we can't, actually, predict it.
So there's a little experiment I sometimes do.
I'll pretend to do it, even though there are no students here.
I take three coins-- see if there are students in the class, I ask the students to give me coins, and then I try to steal them.
But since there are no students, I'm going to have to use my own coins, and I'm going to ask, is at least one of them heads.
Well the truth is, it's completely predictable.
I know the answer.
But you don't know the answer, because you can't see these coins.
And so you might as well assume it's probabilistic, and guess well, he put 3 coins down at random, the odds of each one being a head is 1/2, so probably 1 of the 3 is a head.
And you're relying on probability, because you don't know what's going on-- predictive non-determinism.
OK, and that's the way we're going to deal with a lot of things going on for the rest of the semester.
We'll look at a lot of examples where we have to act as if things are non-deterministic.
And that gets us to this notion of what mathematicians call stochastic processes.
A process is stochastic if it's next state depends on both the previous states and some random element.
So now what I'm going to do is pick up one of these coins, flip it in the air, put it down, and ask, again, about the state of these three coins.
It depends upon the previous state, because these two coins have the same value they had in the previous state, plus a stochastic element, a probabilistic element, a random element-- the value of that coin, which I just flipped.
OK, most programming languages, including Python, include simple ways to write programs that use randomness.
As we'll see, as we've already seen in Python with our drunkard's walk, we use the function random.choice, which, given a set of values at random, chose one of those values.
That function and almost all of the other functions in Python that involve randomness are implemented using something called random.random.
This function generates a random float that's greater than 0 and no greater than 1.0, So you get one of the infinite or seemingly infinite number of floating point values that are greater than 0 and no greater than 1.
So let's go look at another example of the stochastic process.
We're going to look at throwing dice.
So I've got something called rollDie, which chooses a value between 1 and 6.
For those of you who have never gambled with dice, it's a cube.
You roll it, and it has a value between 1 and 6 that shows up at random.
And then I've got this little program called testRoll that rolls a bunch of dice and comes up with an answer.
All right, so let's see what happens.
Actually, before we do that, let me ask you-- we can look at a question.
Imagine I roll it, and I run it some large number of times, 10.
Would you expect to see the value-- more likely see that value or might more likely see a value that looks like this.
Which of these values is more likely to come up from my random rolls of the die?
Well when I take a vote-- if I take a vote-- historically in this class, this strikes people as more likely to happen than this.
But it's a trick question, because as it happens, they're equally likely.
And the reason they're equally likely is each roll is independent of the previous rolls.
And as we'll see in our excursion in probability and randomness, independence is a very important assumption.
In a stochastic process, two events are independent if the outcome of one event has no influence on the outcome of the other.
The events are independent if the outcome of one event has no influence on the outcome of the other.
So it's a bit easier to think about this, maybe, if we simplify the situation for the moment to think about flipping coins, which have either heads or tails, and I'll look at the value 0 or 1 as the examples there.
That means I can use-- I have a binary die for some reason.
So as we've seen before, if I flip a coin ten times, how many different possibilities, sequences of 0 and 1 are there?
Well we've seen this kind of thing a lot of times already this semester.
There are 2 to the 10 binary numbers of 10 digits, so we know that there are 2 to the 10 possibilities.
Each of these 2 to the 10 possibilities are equally likely.
So the number in which I have all 0's is no more likely than the number of all 1's, is no more likely than some seemingly random combinations of 0 and 1.
So it's a very small probability.
So what's the probability of getting all 1's?
It's 1 out of 2 to the 10.
What's the probability of getting all 0's?
1 out of 2 to the 10.
What's the probability of any combination you would happen to pick of 0's and 1's?
1 over 2 to the 10.
I know I'm belaboring this point, but the point I want to make is that when we talk about some result having a particular probability, we are asking, essentially, the question, what fraction of the possible results have the property we're testing for.
So I ask what property are all 1's.
I'm saying what fraction are all 1's.
If I say well, the properties are exactly four 1's.
What fraction of these numbers have exactly four 1's in them?
Whatever I want.
So probabilities will always be fractions.
That's important because it means that when we talk about the probability of some event occurring, we know it has to be somewhere between 0 and 1.
Probabilities are never less than 0.
They're never more than 1.
Cannot happen, guaranteed to happen, usually somewhere in between.
All right, that's the key thing to remember when thinking about probabilities.
Suppose I want to ask, what's the probability of getting some sequence of coin flips other than all 1's.
Well I know the probability of getting all 1's is 1 out of 2 to the 10th.
I know the probability of getting some sequence of flips is 1.
It's certain I'll get one of those numbers.
So the answer is the probability of not getting all 1's is 1 minus 1 over 2 to the 10th.
This is an important trick to remember.
Typically we have two ways of computing probabilities.
We can either compute it directly, as I did when I computed the probability of getting all 1's, or we can compute the probability of something not happening by subtracting one probability from another, this 1 minus trick.
And so you'll see me using this formulation a lot of times, And we'll talk as we go forward about when you do it which way.
All right so let's go back, finally, to our six-sided die.
How many sequences are there of length 10 for that?
2 to the 10th?
No.
6 to the 10th.
Because unlike the coin where we only had 2 possibilities, we now have 6 possibilities.
So there are 6 to the 10th different sequences of rolls I could get, quite a few.
So the probability of getting 10 consecutive 1's is 1 over 6 to the 10th.
And of course, the probability of getting this sequence, here, is also 1 over 6 to the 10th, so we see that they are equal.
OK, we're going to spend a lot more time on probability and randomized and stochastic algorithms.
But before I do that, I want to take a brief digression and return to the topic we looked at a little earlier this morning, which was data visualization, plotting.
I want to do this for two reasons.
One, it's really important.
It's something that all of us do a lot of in the course of our work, but it also will just make it a lot easier for me to talk about probability and stochastics when I can draw some pretty pictures to illustrate what's going on.
Now many people, most of us, probably, when we're writing code to do something, focus on writing programs that perform some complicated analysis of the data, and then print something.
We don't spend enough time, I think, worrying about how the results of our analyses are presented so that somebody else can make sense of them, or in fact, we can understand them better ourselves.
Sometimes text is the best way, but sometimes there's a lot of truth to the Chinese proverb that a picture's meaning can express 10,000 words.
Now most of us, sort of, believe this.
Why don't we do it?
Well because in most programming languages, it's hard to draw pretty pictures.
One of the reasons we use Python in this class is because in Python, it's easy to draw pretty pictures or, at least, to make plots.
Why is that?
It's because somebody-- not me-- went to the trouble of building something called PyLab.
PyLab is a Python library that provides many of the facilities of something called MATLAB.
If you're an MIT student, the probability of your graduating without using MATLAB is very low.
It is something people use a lot.
It's not my favorite programming language.
It has its utility.
I like Python, because it brings-- a lot of the features of MATLAB are easy to use in a programming language that I find much more convivial than MATLAB.
All right I'm not going to give you a complete tutorial for PyLab here.
It would take a long time, and it would be boring.
Instead I'm going to give you a few examples, and in fact, focus primarily on the plotting capabilities.
And the good news is the plotting capabilities in PyLab are almost identical to those in MATLAB, so if you learn how to do it here, you'll already know how to do it here.
For details you should take a look at-- let me make sure I write this correctly-- this website, matplotlib.sourceforge.net, and it's a very nicely put together website that will give you all of the capabilities of plotting.
Also you'll find in the class website, I've written a little chapter of a book about how to do this sort of thing, and you may also find that helpful.
I should point out that PyLab is not part of the standard Python distribution.
It has to be installed on your computer, and again, there are instructions about how to do this posted on the class website.
All right let's start with something very simple.
We'll look at that, here.
I'm beginning by importing.
I'll import PyLab.
You have to import it to use it.
And then I'm going to plot two things.
So what I want to observe, here, is I'm going to plot two vectors-- the vector 1, 2, 3, 4 and the vector 1, 2, 3, 4.
These are the x-coordinates.
These are the y-coordinates.
So we'll get a two-dimensional plot, x versus y.
And it's very important that these two vectors be of the same length.
Doubtless when you're using this in your problem sets, you will screw up and you'll get an error message, which you will have a hard time interpreting, but what it probably is going to boil down to is you've done something wrong, and you're plotting things that are not the same length.
After I plot these two things, I'm going to type "pyLab.show," which will put the plots up for us to look at.
So let's do that now.
So the first plot in our straight line-- 1 versus 1, 2 versus 2, 3 versus 3, 4 versus 4-- got that.
Then it plotted this rather funny looking zigzag we saw, kind of just randomly chosen in the other one.
And the plots will always look something like this.
I should mention, if we go look at the code, pyLab.show-- if I haven't said that, the plot would not have appeared on my screen.
PyLab would have produced it, but not displayed it.
You may think that's silly, but in fact it's useful, because most of the time when I'm writing programs that do plotting, I don't want to see them on my screen.
I'm producing a whole bunch of plots, and I'm going to write them to files that I will then look at later or include in a paper or a lecture or something, so it makes me say that I want to see it.
I should point out, depending upon your operating system, it can be pretty annoying, because if you try and do this twice in your code, something bad can happen.
The code can hang.
Therefore, you should only execute pyLab.show once per program.
And it should always be the last thing you executed, because once you execute pyLab.show, the program will stop running, essentially.
It's annoying.
I wish it weren't that way, but it is.
So live with it.
All right let's go look back at our graph, our plot.
At the top is a title.
This is the default title.
It says figure 1.
Later we'll see that I could have given it a much better title than that.
Then it's got the values of the x and y-axes and a bunch of things down at the bottom that we could point out.
You can zoom in on the plots, using that or zoom out.
You can use this funny icon.
It happens to be a floppy disk icon, something that probably most of you have never seen.
Congratulations if you've never seen a floppy disk.
Your life is better than it would be had you had to deal with them.
But that's used for saving them to a file.
You can move around in it.
You can get back to what the original figure was, a whole bunch of useful things.
I suggest you just bring this up and play with it.
One of the things I should mention here is you'll note that when I produced this, I only use four points-- 1, 2, 3, and 4, say, for this one.
Yet it looks as if I have a continuous plot.
You know, it claims that 1.5 and 1.5 match.
This can be very deceptive, as we'll see later on, and probably it might have been better for me to plot not lines here, but points, indicating there are only four points in this graph.
It's more apparent here, where we see this funny looking zigzag, implying some complicated relationship amongst the points, which, actually, probably doesn't exist.
And again later on we'll see ways to avoid that.
OK.
It is, of course, possible to produce more than one figure.
So let's look at this.
We'll comment this piece out for now.
And we'll run this code.
So this says I want to plot something on a figure I'm calling figure 1, as before, And then I'm going to go to figure 2.
So now instead of plotting both of these on the same figure, I'm going to put them on separate figures.
And then pyLab.savefigure will, actually, create a file in the directory in which I'm running the program and save it-- called firstsaved And then this will be secondsaved.
And now I can run it.
And now I have two figures-- figure 1, as before-- well not quite as before, a different figure-- and figure 2.
Again, nothing very magical there.
And if we look in my directory, we should see, I hope, firstsaved and secondsaved, so if we look at secondsaved, we'll see, there it is.
And just to show that I'm not cheating, you can notice that the time stamp is today at 10:34 AM, which happens to be when I'm giving this particular lecture.
All right you can put that away now and continue.
Now what I want you to notice is the last one.
I just gave it one argument-- 5, 6, 7 and 10.
And if we look at what it plotted-- actually, I think I put that in figure 1, didn't I?
You'll see, here it is.
And it's made up some values for the x-axis.
If you only give it one set of values, it assumes it's the y-axis, and it finds values.
All right now what values is it going to choose for the x-axis?
Well this is Python, so surprise, surprise, the first value is 0, 1, 2, and 3.
It's how I get the four values.
Now we could look at another example, a slightly more interesting one.
Comment this out, so we don't look at the boring stuff over and over again.
I've written a little program to calculate interest.
So I'm going to start with an initial principal, here, of 1,000, an interest rate of 5%, 20 years, and just do compound interest.
This, you all would know how to write this.
And I'm going to plot it and see what we get.
All right so what we have-- something here, which, sort of, shows the kind of beautiful growth you get with compound interest, what the finance people call the magic of compounding, which will make us all rich in principle, until we see what the markets really do.
But at any rate, for now we can look at it, and it looks very pretty.
But I don't know what it means.
I look and say, "oh, its figure 1."
Well that's not very informative, and x goes from 0 to 20, but if I haven't told you, you wouldn't know what that meant.
And y from 10,000 up to 28,000, but again, you wouldn't know what that means.
We see this all the time.
It's not a good thing.
It's a bad thing, in fact.
All plots should have informative titles, and all axes should be labeled.
I can't tell you the number of times I've had a graduate student show up in my office, having worked for weeks producing some data, put a plot on my desk, and say, look at this, isn't it great.
And I say, I have no idea what it means.
And sometimes, I'll say, well what is the y-axis, and they end up scratching their head.
They're not quite sure.
You've got to label your axes.
You've got to put a title.
You've got to give the person a break.
Well how do we do that?
Well it's pretty simple.
So here's this code.
So I just want to get rid of the old graphs, because sometimes if you look at them, it causes you problems.
Because it's now hung.
It won't continue until I get rid of it.
Now my shell is back.
So pyLab.title just says, OK, I'm going to call it 5% growth, compounded annually.
Notice that I've put in that it's 5% interest rate and that I'm compounding it annually, not semi-annually or quarterly.
The x-axis is going to be the years of compounding and the y-axis, the value of the principle in dollars.
So now it's the same curve, but a far more informative picture.
All right here's where I want to stop today.
We're going to come back to this topic, in probably more detail than you want, and spend quite a lot of time talking about how do we produce beautiful plots, and more importantly, how do we produce plots that are, actually, meaningful to those reading them.
Thanks a lot.
I'll see you in the next lecture.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Today I want to spend a few more minutes on plotting, and then return to a subject that will occupy us for a couple of weeks, which is the use of randomness in solving problems.
Don't save.
All right, so let's first look at one more example of plotting.
It's simple.
It's so simple you'll find it's not in your handout.
So here it is to start with
All I'm doing here is I wrote a little program to show the effect of compound interest, nothing very sophisticated.
We start with some principal and an interest rate, and then we just apply it over and over again.
And then we're going to plot to show what the principal has become if we just keep compounding the interest.
So it is kind of what you'd expect.
Compound interest is a nice formula.
You can actually get rich applying it, and we see this nice little graph.
On the other hand, we can't really tell what it is.
And this is the sort of thing that I see all too often, including my graduate students produce it.
They come to my office, they show me a graph, and they start explaining it.
And I usually refuse to look at it if it looks like this.
There is no point in ever, and I mean ever, producing a graph that does not have a title and labeled axes.
And in particular, you have to label the axes to say what they mean.
Fortunately, it's easy enough to do.
And here, I've just done that.
So I'm going to run the same code to compute the interest, but I'm going to put a title on the graph.
You've seen this before, I just want to remind you how it works.
PyLab.title And then I'm going to label the x-axis and the y-axis.
And that gives me a much more useful graph.
Nothing magical here, it's just a reminder that you really need to do these things.
You'll notice here I've not only told you that this is the years of compounding and that this is the principal but I've measured it in dollars.
Maybe I should have been even more explicit and said, well, US dollars, whatever.
One of the things I did want to point out is you saw the two of these various icons that will let you do things like zoom in on a graph and save a graph.
Here's this icon that I think Professor Grimson mentioned, in fact, I know he did.
It's a floppy disk, just in case you've never seen one, I brought a floppy disk to show you.
This is one of the older floppy disks.
These were invented in 1971 by IBM.
They were originally 8 inches in diameter and held all of 80 kilobytes of data.
And as you can see, unlike later floppy disks, they actually flopped.
Eventually, Apple and others pioneered a non-floppy floppy disk, that was in the '80s.
The interesting thing today is I typically carry around a USB stick with me about that big that holds roughly 400,000 times more data than this floppy.
And so it's just quite incredible how things have gone along.
All right.
I now want to return to what will be the main theme for, as I said, a couple of weeks which is randomness.
And in order to talk about randomness we have to talk about probability.
And I know that Professor Grimson started down that path just before spring break, but if you're anything like me your mind kind of deteriorated a little bit over spring break, and your head isn't quite into things.
And so, I'm just going to back up a tiny bit and start over to get our heads into it, and then fairly quickly move on to new things.
So let's start by asking a simple question.
You can tell my head isn't quite yet back to things because I forgot that I needed to begin by gathering chalk.
I've now got it.
And we'll come over here and take a look at some examples.
All right.
So the first question I want to ask is, suppose I take a 6-sided die, a fair one, and I roll it 10 times, what's the probability of not getting a single 1, out of that die?
Well, how do we go about answering this?
Well, there is a wrong way to do it, which is sort of the obvious way, and many people will start down this path.
They'll say, well the probability of rolling a 1 on the-- not rolling a 1 on the first try is 1 over 6.
Right?
That's true?
That's not true.
What's the probability of not rolling a 1 the first time?
5 over 6.
All right.
What's the probability of not rolling a 1 on the second try?
5 over 6.
Well, the wrong thing to do, of course, would be to start adding them up.
We say, well, OK, we'll just add these up.
Well, one way we can tell that's wrong is if we add up 10 of these, we get more than 1.
Probabilities can never be more than 1 as we'll see.
So let's now try and think of the right way to look at this problem.
So you can think about it.
If we roll these-- a die 10 times, each time I'll get a number.
So I might get a 3, and then a 4, and then a 2.
How many possible 10-digit numbers are there?
On a 6-sided die, if I roll it 10 times?
How many?
6 to the 10th?
6 to the 10th.
Exactly Just when we look at binary numbers, if I take a 10-digit binary number, and ask how many different numbers can I represent in 10 binary digits, it's going to be 2 to the 10th.
Here we're base 6.
So it's going to be 6 to the 10th.
Pretty big number.
Now I can say, how many of those numbers don't contain a 1?
All right.
So that's really the question I'm now asking.
How many of these don't contain a 1?
So as we said, if I look at the first roll the odds of not getting a one the first time is 5 over 6 Now what's the odds of not getting 1 the first or the second time?
It's 5 over 6 times 5 over 6.
That makes sense?
Because these are independent events.
And that's a key notion here.
I'm assuming that whether I get a 1 on the second roll is independent of whether I got a 1 on the first roll.
It should be true, assuming my dice-- die is fair.
Similarly, I can do this for the third roll et cetera.
So the probability of not getting a 1 in 10 rolls is going to be (5 over 6) to the 10th.
That makes sense?
If not, speak up, because things are going to get more complicated quickly.
All right.
So that's pretty simple.
You-- you all-- are you all with me on that?
Now, suppose I ask you the inverse question.
What is the probability of getting at least one 1 if I roll the die 10 times?
So here I've given you how to compute the probability of not getting any 1's.
Suppose I asked you the probability of at least one 1?
Yeah?
[INAUDIBLE] 1 minus not having a 1?
Exactly.
Thank you.
So that would be 1 minus because we know that the probability, the sum of all the possible things that we can do when we do a probability always has to be 1.
It was a good effort.
That's it.
If you take-- if you want to get something where everything is covered, the probabilities always have to sum to 1.
And so now, there are only two possibilities here.
One possibility is I don't get any 1's.
One possibility is I get at least one 1.
So if I take all of the possibilities, and I subtract the possibilities of not getting any 1's, the result must be the probability of getting at least one 1.
This is a very common trick in computing probabilities.
Very often when I ask or somebody says, what's the probability of x?
The simplest way to compute it, is to compute the probability of not x and subtract it from 1.
OK. Again, heading down a wrong track for this, one might have said, well all right, the probability of getting a 1 on the first roll is 1 over 6.
The probability of getting a 1 on the second roll is 1 over 6.
The probability of getting a third roll is 1 over 6.
I'll just add them up, and that will give me the probability of getting at least one one.
How do I-- how can I be sure that's wrong?
Well when I'm done, I would claim the probability is something like that.
And we know that can't be true.
Because a probability always has to be less than or equal to 1.
So this is a good trick to keep in mind, whenever you're given a probability problem, try and figure out whether you have a good way to compute it directly, or whether it's simpler to compute the not of the probability, and then subtract it from 1.
Probability is really a fun field.
It's interesting, it's history, it's intimately connected with the history of gambling.
And, in fact, almost all of early probability theory owes its existence to gamblers.
People like Cardano, Pascal, Fermat, Bernoulli, de Moivre, Laplace, all famous names you've heard, were motivated by desire to understand games of chance.
Mostly, it started with dice.
I've been talking about dice.
And in fact, dice are probably the human race's oldest gambling implement.
They date at least, archaeologically, to about 600 BC, where a pair of dice was found in Egyptian tombs, actually longer than that.
Two millennia before the birth of Christ, people found dice in Egyptian tombs.
Typically, they were made from animal bones, but that doesn't matter.
Pascal's interest in it, and Pascal is really considered the founder of probability theory, came when a friend asked him to solve the following problem which I want to work out with you.
Is it profitable to bet that given 24 rolls of a pair of fair dice, you would roll a double 6?
He actually had a friend who was in the business of gambling, making these bets.
So he said, you've got a pair of dice, you roll it 24 times and ask the question, what is the probability of getting what we call today "box cars", in those days they just called two 6's.
This was considered a really hard problem in the mid-17th century.
And in fact, Pascal and Fermat, two pretty smart guys as it happens, debated this.
They exchanged letters with each other trying to figure out how to solve this problem.
It shows how math has advanced because, in fact, today, it's quite an easy problem.
So let's work it through and think how would we answer this question.
So what's the probability of rolling, of not rolling, a double 6 on the first try?
Well, the probability of not rolling a 6 on one die is a sixth -- 1 over 6.
The probability of not rolling a one with the next die is also 1 over 6.
So the probability of not getting a die in the first roll, first double 6's is-- the probability of getting a double 6 is 1/36.
So the probability of not getting a double 6 is 35/36.
Right?
So now we know that the probability of not getting it is that.
What's the probability of not getting it 24 times in a row?
It's that.
Which is approximately equal to 0.51.
So you can see why the answer was not obvious just by experience.
But there is a slight edge in betting that you will not get a double 6 in 24 times.
Again, assuming you have fair dice.
As old as dice is, people have built cheater's dice.
The excavation of Pompeii, for example, they discovered a pair of loaded dice, dice with a little weight in it so one number would come up more often than it should.
And in fact, if you look at the internet today, you will find many sites where you can, let's see, the one I found this morning says quote, "Are you on unusually unlucky when it comes to rolling dice?
Investing in a pair of dice that's more reliable might be just what you need."
And then it says, "Of course for amusement only."
Yeah, we believe that.
All right.
As much as I trust probability theory, I don't trust my ability to use it.
And so what I did is wrote a little simulation to see if Pascal was right when he did this.
So I've got the first-- just this little test roll, which rolls a dice number of times, gets the result.
Now, then I decided to check Pascal.
So I was going to run 100,000 trials, and keep track of the number of times it worked.
So what you'll see I'm doing here is for i in the range number of trials, and this is the way we'll do a lot of these simulations.
And in fact, as we deal with probability, we'll be dealing a lot with the notion of simulation, as you are doing in your current problem set.
So for i in range number of trials, for j in range 24, because that was Pascal's friend's game, I'll roll the first die, I'll roll the second die.
If they're both 6's I'll say, yes equals 1.
And I'll break and then I'll compute the probability of winning or losing.
OK?
So let's let it rip.
So now let's let it rip.
There it is.
And we can see that it actually comes out pretty close to what Pascal predicted.
Should we be surprised that it didn't come out to exactly?
Well let's see, is it exactly?
What is 35/36 to the 24th?
So that's the-- well, to 17 digits of precision, the exact answer.
And you can see we came up with something close to that.
Not exactly that, and we wouldn't expect to.
Now I only did 100,000 trials.
If I did a million trials, I'd probably come up with something even closer, but if I did 2 trials, who knows what I get-- come up with it, right?
Could be-- I could get 1, I could get lucky both times, or unlucky.
Later on, we'll talk more about the question, how do we know how many trials to run?
Now, the interesting thing is I'm sure it took me less time to write this program than it took Pascal to solve the problem.
Now the truth is, I had several hundred years of other people's work to build on.
But in general, I think one of the questions you'll find is, is it easier sometimes to write a simulation, than it is to do the probabilities?
What I often do in practice is both.
I'll scratch my head and figure out how to figure out the answer analytically, and then if it's easy, I'll write some code to simulate the problem, and expect to get roughly the same answer, giving me confidence I've done it correctly.
On the other hand, if I've done the simulation and it had come up with something totally bogus, or totally different, then I would have had to work hard to figure out which was right, the code or the math.
Same sort of thing you saw when you looked at the random walk, and the first time it was done an answer showed up that was just wrong.
But, you need to have some intuition about a problem, so that you can look at it and say, yeah, that's in the ballpark.
And if it's not, it's time to worry.
This kind of simulation that I've just done for the dice game is what's called a "Monte Carlo simulation".
It is the most popular kind of simulation named after a Casino on the Riviera, in the small principality of Monaco.
This was back in the time when it was hard to find a place you could gamble, and this happened to be one of the places you could.
The term was coined in 1949 by Stanislaw Ulam and Nicholas Metropolis, two very well-known mathematicians.
Ulam, who later became famous for designing the hydrogen bomb with Teller, invented the method in 1946, and I'm going to quote from his description of how he invented it.
"The first thoughts and attempts I made to practice the Monte Carlo method, were suggested by a question which occurred to me in 1946, as I was convalescing from an illness and playing solitaires.
The question was, what are the chances that a canfield solitaire laid out with 52 cards will come out successfully?
After spending a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than quote 'abstract thinking' end quote, might not be to lay it out, say, 100 times, and simply observe and count the number of successful plays.
This was already possible to envision with the beginning of the new era of fast computers.
And I immediately thought of problems, as you would, I'm sure, immediately thought of problems of neutron diffusion and other questions of mathematical physics.
And more generally, how to change processes described by certain differential equations into an equivalent form interpretable as a succession of random operations.
Later, I described the idea to John von Neumann, and we began to plan actual calculations."
So as early as 1946, people were thinking about the question of moving away from solving systems of equations, to using randomized techniques to simulate things and try to find out what the actual answer was that way.
Now of course "fast" is a relative term.
Ulam was probably referring to the ENIAC computer, which could perform about 10 to the 3 additions a second.
Not very many, 1,000 operations a second, and weighed approximately 25 tons.
Now today's computers, by comparison, perform 10 to the 9th additions and weigh about 10 to the minus 3 tons.
All right.
This technique was used during the Manhattan Project to predict what would happen doing-- during nuclear fission and worked.
Monte Carlo simulations are an example of what's called "inferential statistics".
In brief, and I'm going to be brief because this is not a statistics, course, inferential statistics is based upon one guiding principle.
And that principle is that a random sample tends to exhibit the same properties as the population from which it is drawn.
So if I try and sample people, say, for predicting an election, the notion is if I go and I asked a 1,000 people at random in Massachusetts who they're going to vote for, the average will be about the same as if I looked at the whole population.
So whenever we use a statistical method like this, so for example, we assumed here, is those 100,000 times I threw the pair of dice, that that would be representative of all possible throws of the dice, the infinite number of possible throws.
One always has to ask the question whether this is true, or whether one has a sampling technique that is, for example, giving you a biased sample.
Little later in the term, we'll talk about many ways in which you can get fooled here and think you're doing a fair statistical analysis, and get all the math right, and still come up with the wrong answer because this assumption doesn't actually hold.
All right.
Let's think about it now in terms of coins, a little simpler than dice, where you can flip a coin and you get either a head or a tail.
Suppose Harvey Dent, for example, flipped a coin and it came up heads.
Would you feel good inferring from that that the next time he flipped a coin it would also come up heads?
I wouldn't.
Suppose he flipped it heads and it came up heads twice, in a row.
Would you feel comfortable with the third flip would be a head?
Probably not.
But suppose he flipped it a 100 times in a row, and it was a head each time.
What would you infer?
I would infer that the coin two-headed And, in fact, every time it was going to come up heads, because it is so improbable that if it was a fair coin-- what's the probability of having a 100 heads in a row with a fair coin?
1 over what?
1 over 100-- 1 over 2 to the 100th.
Right?
A half the first time times a half times a half.
A huge number, a very small number rather, right?
So the probability and a fair coin of getting hundred heads in a row is so low with just 100 flips, that I would begin to think that the coin was not fair.
All right.
Suppose, however, I flipped it 100 times and I got 52 heads and 48 tails.
Well, I wouldn't assume anything from that.
Would I assume that the next time I flipped it a 100 times I'd get the same 52 to 48 ratio?
Probably not, right?
Your common sense tells you you wouldn't.
All right.
Probably, it tells you, you wouldn't even feel comfortable guessing that there would be more heads than tails the next time.
So when we think about these things, we have to think about the number of tests and how close the answer is to what you would get if you did things at random.
This is sort of comparing-- this is technically called comparing something to the null hypothesis.
The null hypothesis is what you would get with a random event.
And when you do a simulation, if you get something that is far from that, or when you sample a population, you get something that's distant from the null hypothesis, you can assume that maybe you're seeing something real.
All right.
Let's look at this in a little less abstract way.
So let's go look at some coin flips.
So I wrote a simple program, flip.
Just flip the coin some number of times and tells me what fraction came up heads.
So we can run it, and let's look at a-- suppose I flip a 100, I get 0.55.
If I flip 10, I get 0.4.
If I flip 10 again, I get 0.5.
Now look at that, the same thing twice in a row but now I get 0.2.
So obviously, I shouldn't infer too much from 10 flips and even from 100 where I got 0.55.
Let's see what happens if I flip 100 again, 0.41, big difference.
So this is suggesting that we can't feel very good about what happens here.
Now if I do the following, well I'm feeling a little bit better about this, well for one bad reason and one good reason.
The bad reason is, I know the answers 0.5, and these are both close to 0.5, so I feel warm and fuzzy.
But that's cheating.
I wouldn't need to write the simulation If I knew the answer.
But mostly I feel good about it because I'm getting kind of the same answer every time.
OK, and that's important.
The more I do, the more stable it gets with the larger the number of trials.
This is an example of what's called "the law of large numbers", also known as Bernoulli's Law, after one of the Bernoulli family of mathematicians, and I can't for the life of me remember which Bernoulli.
There are a whole bunch of them.
Anyway the law states, and it's important to understand this because again it underlies the inferential statistics, that in repeated independent tests, and it's important to note the word "independent", each test has to be independent of the earlier test.
In this case, the tests are flips of the coin with the same actual probability we'll call it p, often used to represent probability, of an outcome for each test, the chance that the fraction of times that outcome occurs the outcome that with probability, p, converges to p as number of trials goes to infinity.
All right.
So if I did an infinite number of trials, the fraction of heads I would get in this case would be exactly 0.5.
Of course I can't do an infinite number of trials.
But that's the law of large numbers that says the-- Now, it's worth noting that this law does not imply that if I start out with deviations from the expected behavior, those deviations are likely to be quote "evened out" by opposite deviations in the future.
So if I happen to start by getting a whole bunch of heads in a row, it does not mean that I'm more likely to get tails in a subsequent trial.
All right.
Because if I were-- if that were true, then they wouldn't be independent.
Independent means memoryless.
So if I have an independent process, what happens in the future cannot be affected by the past.
And therefore, I don't get this business of "they even out".
Now people refuse to believe this.
If you go to any gambling place, you'll discover that if people threw the roulette wheel, if black comes up 20 times in a row, they'll be a rush to bet on red.
Because everyone will say, red is do, red is do, red is do.
And every psychologist who has ever done this experiment, finds that people don't believe it.
That it's not true.
People just don't get probability, and it happens so often it's got a name called "the gambler's fallacy".
And there's been great examples of people going broke doing this.
Now notice that the law of large numbers here is about the fraction of times I get an outcome.
It does not imply for example, that the absolute difference between the number of heads and the number of tails will get smaller as I run more trials.
Right?
It doesn't say anything at all about that.
It says the ratio of head to tails will approach 1, but not that the difference between them.
All right, let's look at an example showing that off.
So what I've got here is this program called "flip plot".
This is on your hand out.
This is just going to run this business of flipping coins.
I should point out just-- I did it this way just to show you.
What I'm doing is each flip-- if random.random is less than 5, I'll call it a head, 0.5, I'll call it heads, otherwise a tails.
You'll notice that it appears that maybe I'm biasing a little bit, because I'm giving 0.5 a value.
But there are so many floating point numbers between 0 and 1, that the probability of getting exactly 0.5 is so small that I can ignore it.
It isn't going to really make a difference Random.random is the key issue, the key function that's used to implement all the other random functions that we have in that package.
All right.
So I'm going to do it, and I'm going to do it over a range of values.
The minimum exponent to the maximum exponent and for exponent in range min x to max x plus 1, I'm going to choose an x value that is 2 to that.
So this lets me go over a big range.
So I'll see what happens if I get 1 flip, and 2 flips, and 4 and 8 and 16 and 32 et cetera.
And then I'm going to just do some plots.
I'm going to plot the absolute difference between heads and tails and the ratio of heads to tails.
Let's see what happens when we run that.
Actually, probably nothing because I didn't uncomment the run part of it.
Let's do that.
So I'm going to call flip plot with 4 and 20, running from four trials 2 to the 4 to 2 to the 20.
Let's see what we get.
Now, you may get different things when you run at different times.
In fact, you will.
So here we see something kind of uninteresting.
Let's cheat and see what we got the first time I ran it, which is on your hand out, and I have a PowerPoint with it.
I was-- I knew this might happen.
Doesn't usually, but sometimes when you run it you get surprising results.
So here's what happened when I first ran it.
Here was the difference between heads and tails.
And it seems that, OK, the difference was low, it went up, it went down, it went up, it went down.
It seemed to go down dramatically.
If you remember what we just saw when I ran it, we also saw something where it went up a little bit then it went down and then shot up dramatically at the end, which was why that scale is so funny.
And if we look at the ratio, what we see is it seems to start above 1, drop below 1, and then seems to converge towards 1.
Now, I show this because I want to make a couple of points of plotting.
Let's look at this out here.
Looks like we have a pretty dramatic trend of this linear drop.
Do we?
Do we actually have a trend here?
Well let's think about it.
The default behavior of the plot command in PyLab is to connect points by lines.
How many points do I actually have out here?
Well, you saw the code.
You have the code in the hand out.
How many points do you think there are out here?
A 1,000?
A 100?
3?
2?
2 to 3.
Right?
Depending on what I mean by "out here".
So what we see here is something that happens a lot.
People plot a small number of points, connect them by a line, and mislead the audience into thinking there's a trend when, in fact, maybe all you have is an outlier.
So it's problematical here to do it that way.
So let's see what happens if we change the code.
And what I'm going to do is change it in two ways.
Well, maybe I'll change it in one way first.
Uncomment.
Uncomment.
So what I'm doing here is I am plotting in a different way.
This quote "BO" says, don't connect it by lines but just put a dot as an "O" and B says, make it blue.
I used blue because it's my favorite color.
So now if we look at these things, we'll see something pretty different.
So that's the difference between heads and tails, that's the ratio.
But now, if we look at the difference between heads and tails here, what we see is it's pretty sparse.
So yeah, maybe there's a trend, but maybe not, right?
Because way out here I'm only connecting two points, giving an illusion that there is a trend but, in fact, no reason to believe it.
So I always think if you're plotting a small number of points, you're much better off just plotting the points, than you are trying to connect them.
Now if we look at this one, again, maybe we'd feel kind of comfortable if there is a trend here, that there are several points on this line.
We can't see much of what's going on over here, which gets me to the next thing I want to do is I'm going to use logarithmic axes here.
So PyLab.semilogx says make the x-axis logarithmic.
PyLab.semilogy the y-axis.
And so in the case of the absolute difference, I'm going to make both logarithmic.
Why am I doing that?
Because both have a large range, and by making it logarithmic, I can see what's happening at the left side in this case where things are changing.
When I look at the ratios, the y-axis does not have a very large range, and so there's no need to make it logarithmic.
We'll run it.
So here, we can see the difference between heads and tails.
And now we can see what's going on at the left as we can in Figure (4).
And we can see things much more clearly.
So log scales can be enormously useful.
And in fact, I use them a lot, everyone uses them a lot but, again, it's very important to observe the fact that it's logarithmic and not get fooled.
All right.
So I talked about linear scaling, logarithmic scaling and we now have charts where I can, perhaps, actually reach some conclusion about what's going on.
The next question is, how certain can I be?
Can I really be certain that, indeed, this should be converging to 1?
Here, if I sort of look at it, it does look like there's kind of a linear trend of the absolute difference growing as the number of trials grows.
How certain can I be of that?
You can never get absolute certainty from sampling, because you could never be sure if you haven't been vastly lucky or unlucky.
That's not to say you can't get the absolute correct answer.
Maybe I could get 0.5, which is the correct answer.
But I can't know that that's the correct answer.
So now the question I want to pursue, and it's what we'll cover on Thursday, is what techniques can I use to make a statement of the form, I'm certain within the following range that I have the right answer.
That I know the right answer is highly likely to be this close to the answer my simulation is giving me.
And we'll look at how we can make those statements and actually believe them.
OK, see you on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
This is the second of two lectures that I'm retaping in the summer because we had technical difficulties with the lectures that were taped during the academic term.
I feel I need to tell you this for two reasons.
One, as I said before, the room is empty.
And so when I say something hilarious and there's no laughter, it's because the room is empty.
And if I'm not asking the students questions during the lecture, it's because there are no students.
The other important thing I want you to understand is that I do own more than one shirt and more than one pair of pants.
And the reason I'm dressed exactly the way I was for lecture 13 is I gave lecture 13 five minutes ago, even though this is lecture 15.
So again, here I am.
And I apologize for the uniformity in my clothing.
OK, on lecture 14, which came between 13 and 15, at the end of it I was talking about flipping coins and ended up with the question how can we know when it's safe to assume that the average result of some finite number of flips is representative of what we would get if we flipped the same coin many more times.
In principle, an infinite number of times.
Well, we might flip a coin twice, get 1 heads and 1 tails and conclude that the true probability of getting a head is exactly 0.5.
Turns out-- assume I have a fair coin-- this would have been the right conclusion.
Just because we have the right answer it doesn't mean our thinking is any good.
And in fact, in this case our reasoning would have been completely faulty because if I flipped it twice and had gotten 2 heads, you might have said oh, it's always heads.
But we know that wouldn't have been right.
So the question I want to pose at the start of today's lecture is quite simply how many samples do we need to believe the answer?
So how many samples do we need to look at before we can have confidence in the result.
Fortunately, there's a very solid set of mathematics that lets us answer this question in a good way.
At the root of all of it is the notion of variance.
Variance is a measure of how much spread there is in the possible outcomes.
Now in order to talk about variance, given this definition, we need to have different outcomes, which is why we always want to run multiple trials rather than say one trial with many flips.
In fact, you may have wondered why am I not-- if I could end up flipping the coin a million times, why would I do multiple trials adding up to a million rather than 1 trial of a million?
And the reason is by having multiple trials, each of which give me a different outcome, I can then look at how different the outcomes of the different trials are and get a measure of variance.
If I do 10 trials and I get the same answer each time, I can begin to believe that really is the correct answer.
If I do 10 trials and get 10 wildly different answers, then I probably shouldn't believe any one of those answers, and I probably shouldn't even think I can average those answers and believe the mean is a real answer because if I run an 11th trial maybe I'll get something totally different yet again.
We can formalize this notion of variance in a way that should be familiar to many of you.
And that's the concept of a standard deviation.
Something I, in fact, already showed you when we looked at the spread of grades on the first quiz this semester.
Informally, what the standard deviation is measuring is the fraction of values that are close to the mean.
If many values are close to the mean, the standard deviation is small.
If many values are relatively far from the mean, the standard deviation is relatively large.
If all values are the same, then the standard deviation is 0.
In the real world that essentially never happens.
We can write a formula for this.
Fortunately, it's not all about words.
And we can say the standard deviation of x, where x is a set of trials-- sigma is usually used to talk about that-- is equal to the square root of 1 over the absolute value of the length of x.
So that's 1 over the number of trials times the summation of the value of each trial, little x and big X, of x minus mu squared, where mu is the mean.
And as I said, that's the cardinality of x.
Well, so that's a formula.
And those of you are majoring in math are going to love that.
But for those of you who are more computationally oriented, I recommend you just take a look at the code.
So here's an implementation of the standard deviation.
So the standard deviation of x is equal-- start by getting the mean of x, which is by summing x and dividing it by the length of x.
Then I'm just going to sum all the values in x and do the computation.
So that code and that formula are the same thing.
All right, now we know what standard deviation means.
What are we going to do with it?
We're going to use it to look at the relationship between the number of samples we've looked at and how much confidence we should have in the answer.
So we'll do this again looking at a bunch of code.
So I've got this function flip plot, which doesn't quite fit in the screen, but that's OK.
It's not very interesting in the details.
What it does is it runs multiple trials of some number of coin flips and plots a bunch of values about the relative frequency of heads and tails and also the standard deviation of each.
So again nothing very exciting, in the code I'm just going to keep track for all these trials.
The minimum and the maximum exponent.
I'm using that so I can run a lot of trials quickly.
The mean ratios, the differences, and the standard deviations for exponent in range.
Minimum exponent to maximum exponent plus 1.
I'm going to build an x-axis.
So this is going to be the number of flips.
And then for the number of flips I'm going to run a bunch of tests and get the ratios of heads to tails and the absolute difference between heads and tails.
And then, I'm going to do a bunch of plotting.
And again, what I want you to notice is when I'm doing the plotting, I'm going to label the axes and put some titles on.
And I'm also going to use semilog because given that I'm looking at different powers, it would compress everything on the left if I would just use linear.
All right.
Let's run it.
Actually, let's comment out the code we need to run it.
So I'm going to call flip plot with a minimum exponent of 4, a maximum exponent of 20.
That's pretty high.
And I'm going to run 20 trials.
This could take a little while to run, but not too long.
And it'll give us some pretty pictures to look at.
Give me a chance to have a drink of water.
I know the suspense is killing you as to what these plots are going to look like.
Here they are.
All right.
So if we look at plot one, that's the ratio of heads to tails.
And as you can see, it bounces around in the beginning.
When we have a small number of flips, the ratio moves a bit.
But as I get to a lot of flips out here, 10 to the 5th, 10 to the 6th, what we're seeing is it begins to stabilize.
We don't get much difference.
Kind of interesting where it's stabilizing.
Maybe not where we'd expect it.
I would have guessed it would stabilize a little closer to one than it did as I got out here.
And maybe I have an unfair coin.
That's the problem with running these experiments in real time that I can't necessarily get the answer I want.
But for the most part, actually, it looks much better on my screen than it does on your screen.
In fact, in my screen, it looks like it's very close to 1.
I don't know.
I guess there's some distortion here.
Think 1.
And if we look at the standard deviation of the ratio of heads to tails, what we're seeing is that's also dropping from somewhere up here around 10 to the 0 down to 10 to the minus 3.
And it's dropping pretty steadily as I increase the number of trials.
That's really what you would hope to see and expect to see that not the number of trials, the number of flips.
Sorry.
As I flip more coins, the variance between trials should get smaller because in some sense, randomness is playing a less important role.
The more random trials you do, the more likely you are to get something that's actually representative of the truth.
And therefore, you would expect the standard deviation to drop.
All right.
Now, what we're saying here is because the standard deviation is dropping, not only are we getting something closer to the right answer but perhaps more importantly, we have better reason to believe we're seeing the right answer.
That's very important.
That's where I started this lecture.
It's not good enough to get lucky and get the correct answer.
You have to have evidence that can convince somebody that really is the correct answer.
And the evidence here is the small standard deviation.
Let's look at a couple of the other figures.
So here's Figure (3).
This is the mean of the absolute difference between heads and tails.
Not too surprisingly-- we saw this in the last lecture-- as I flip more coins, the mean difference is going to get bigger.
That's right.
We expect the ratio to get smaller, but we expected the mean difference to get bigger.
On the other hand, let's look at Figure (4).
What we're looking here is this difference in the standard deviations.
And interestingly, what we're seeing is the more coins I flip, the bigger the standard deviation is.
Well, this is kind of interesting.
I look at it, and I sort of said that when the standard deviation is small, we think that the variance is small.
And therefore, the results are credible.
When the standard deviation is large, we think the variance is large.
And therefore, the results are maybe incredible.
Well, I said that a little bit wrong.
I tried to say it right the first time.
What I have to ask is not is the standard deviation large or small, but is it relatively large a relatively small?
Relative to what?
Relative to the mean.
If the mean is a million, and the standard deviation is 20, it's a relatively small standard deviation.
If the mean is 10, and the standard deviation is 20, then it's enormous.
So it doesn't make sense.
And we saw this.
We looked at quizzes.
If the mean score on Quiz 1 were 70 and the standard deviation were 5, we'd say OK, it's pretty packed around the mean.
If the mean score were 10, which maybe is closer to the truth, and the standard deviation were 5, then we'd say it's not really packed around the mean.
So we have to always look at it relative or think about it relative to that.
Now the good news is we have, again, a mathematical formula that lets us do that.
Get rid of all those figures for the moment.
And that formula is called the coefficient of variation.
For reasons I don't fully understand, this is typically not used.
People always talk about the standard deviation.
But in many cases, it's the coefficient of variation that really is a more useful measure.
And it's simply the standard deviation divided by the mean.
So that let's us talk about the relative variance, if you will.
The nice thing about doing this is it lets us relate different datasets with different means and think about how much they vary relative to each other.
So if we think about it-- usually we argue in that if it's less than 1, we think about that as low variance.
Now there should be some warnings that come with the coefficient of variation.
And these are some of the reasons people don't use it as often because they don't want to bother giving the warning labels.
If the mean is near 0, small changes in the mean are going to lead to large changes in the coefficient of variation.
They're not necessarily very meaningful.
So when the mean is near 0, the coefficient of variation is something you need to think about with several grains of salt.
Makes sense.
You're dividing by something near 0, a small change is going to produce something big.
Perhaps more importantly, or equally importantly-- and this is something we're going to talk about later-- is that unlike the standard deviation, the coefficient of variation cannot be used to construct confidence intervals.
I know we haven't talked about confidence intervals yet, but we will shortly.
All right.
By now, you've got to be tremendously bored with flipping coins.
Nevertheless, I'm going to ask you to look at one more coin flipping simulation.
And then, I promise we'll change the topic.
And this is to show you some more aspects of the plotting facilities in PyLab.
So I'm going to just flip a bunch of coins, run a simulation.
You've seen this a zillion times.
And then, we'll make some plots.
And this is really kind of the interesting part.
What I want you to notice about this-- let's take a look at here.
So now we have been plotting curves.
Here we're going to plot a histogram.
So I'm going to give a set of values, a set of y values.
In this case the fraction of heads.
And a number of bins in which to do the histogram.
So let's look a little example first here independent of this program.
Oops.
Wrong way.
So I'm going to set l, a list, equals 1, 2, 3, 3, 3, 4.
And then, I'm just going to plot a histogram with 6 bins.
And then show it.
I've done something I'm not supposed to do.
I just know title.
There's no x-label.
No y-label.
That's because this is totally meaningless.
I just wanted to show you how histograms work.
And what you'll see here is that it's shown that I've got three instances of this value, of 3, and one of everything else.
And it's just giving me essentially a bar chart, if you will.
Again many, many plotting capabilities you'll see on the website.
This is just a simple one.
One I like to use and use fairly frequently.
Some other things I want to show you here is I'm using xlim and ylim.
So what we could do here is this is setting the limits of the x and y-axis, rather than using defaults saying the lowest value should be this thing, the variable called xmin, which I've computed up here.
And the highest ymin.
What you'll see if we go up a little bit-- so I'm getting the fraction of heads1 and computing the mean 1, and the standard deviation 1.
Then I'm going to plot a histogram of the way we looked at it.
And then what I'm going to do is say xmin and xmax is pyLab.xlim.
If you call xlim with no arguments, what it will return is the minimum x value and the minimum y value of the current plot, the current figure.
So now I stored the minimum x values and the maximum x values to the current one.
And I did the same thing for y.
And then going to plot the figure here.
Then I'm going to run it again.
I'm going to run another simulation, getting fracHeads 2, mean 2, standard deviation 2.
Going to plot the histograms.
But then I'm going to set, for the new one, the x limit of this to the previous ones that I saved from the previous figure.
Why am I doing that?
Because I want to be able to compare the two figures.
As we'll see when we have our lecture on how to lie with data, a great way to fool people with figures is to subtly change the range of one of the axes.
And then you look at things and wow, that's really different or they're really the same.
When in fact, neither conclusion is true.
It's just that they've been normalized to either look the same or look different.
So it's kind of cheating.
And then we'll do it.
So now let's run it and see what we get.
I don't need this little silly thing first.
Let's see.
It's going to take a long time, maybe.
That's one way to fill up a lecture.
Just run simulations that take a long time to run.
Much easier to prepare than actual material.
But nevertheless, shouldn't take forever.
I may have said this before.
I have two computers.
I have a fast one that sits at my desk that I use to prepare my lectures and a slower one that I use to give the lectures.
I should probably be testing all these things out on the slow computer before making you wait.
But really, it's going to stop.
I promise.
Ah.
All right.
So we'll look at these.
So Figure (1) has got 100,000 trials of 10 flips each.
And Figure (2), 100,000 trials of a 1,000 flips each.
And let's look at the two figures side by side.
Make them a little smaller so we can squeeze them both in.
So what have we got here?
Notice if we look at these two plots, the means are about the same.
0.5 and 0.499.
Not much difference.
The standard deviations are quite different.
And again, you would expect that.
A 100 flips should have a lot higher standard deviation than a 1,000 flips.
And indeed, it certainly does.
0.15 is a lot smaller than 0.05.
So that tells us something good.
It says, as we've discussed, that these results are more credible than these results.
Not to say that they're more accurate because they're not really.
But they're more believable.
And that's what's important.
Notice also the spread of outcomes is much tighter here than it is here.
Now, that's why I played with xlim.
If I used the default values, it would not have looked much tighter when I put this up on the screen because it would have said well, we don't have any values out here.
I don't need to display all of this.
And it would have then about the same visual width as this.
And therefore, potentially very deceptive when you just stared at it if you didn't look carefully at the units on the x-axis.
So what I did is since I knew I wanted to show you these things side by side and make the point about how tight the distribution is, I made both axes run the same length.
And therefore, produce comparable figures.
I also, by the way, used xlim and ylim if you look at the code, which you will have in your handout, to put this text box in a place where it would be easy to see.
You can also use the fault of best, which often puts it in the right place.
But not always.
The distribution of the results in both cases is close to something called the normal distribution.
And as we talk about things like standard deviation or a coefficient of variation, we are talking about not just the average value but the distribution of values in these trials.
The normal distribution, which is very common, has some interesting properties.
It always peaks at the mean and falls off symmetrically.
The shape of the normal distribution, so I'm told, looks something like this.
And there are people who imagine it looks like a bell.
And therefore, the normal distribution is often also called the bell curve.
That's a terrible picture.
I'm going to get rid of it.
And indeed, mathematicians will always call it this.
This is often what people use in the non-technical literature.
There was, for example, a very controversial book called "The Bell Curve," which I don't recommend reading.
OK.
So this is not a perfect normal distribution.
It's not really exactly symmetric.
We could zoom in on this one and see if it's better.
In fact, let me make that larger.
And then we'll zoom in on it.
Now that we're not comparing the two, we can just zoom in on the part we care about.
And you can see again it's not perfectly symmetric.
But it's getting there.
And in fact, the trials are not very big.
Only a 1,000 flips.
If I did 100,000 trials of a 100,000 flips each, we wouldn't finish the lecture.
It'd take too long.
But we'd get a very pretty looking curve.
And in fact, I have done that in the quiet of my office.
And it works very nicely.
And so in fact, we would be converging here on the normal distribution.
Normal distributions are frequently used in constructing probabilistic models for two reasons.
Reason one, is they have nice mathematical properties.
They're easy to reason about for reasons we'll see shortly.
That's not good enough.
The curve where every value is the same has even nicer mathematical properties but isn't very useful.
But the nice thing about normal distributions is -- many naturally occurring instances.
So let's first look at what makes them nice mathematically and then let's look at where they occur.
So the nice thing about them mathematically is they can be completely characterized by two parameters, the mean and the standard deviation.
Knowing these is the equivalent to knowing the entire distribution.
Furthermore, if we have a normal distribution, the mean and the standard deviation can be used to compute confidence intervals.
So this is a very important concept.
One that you see all the time in the popular press but maybe don't know what it actually means when you see it.
So instead of estimating an unknown parameter-- and that's, of course, all we've been doing with this whole probability business.
So you get some unknown parameter like the probability of getting a head or a tail, and we've been estimating it using the various techniques.
And typically, you've been estimating it by a single value, the mean of a set of trials.
A confidence interval instead allows us to estimate the unknown parameter by providing a range that is likely to contain the unknown value.
And a confidence that the unknown value lies within that range.
It's called the confidence level.
So for example, when you look at political polls, you might see something that would say the candidate is likely to get 52% of the vote plus or minus 4%.
So what does that mean?
Well, if somebody doesn't specify the confidence level, they usually mean 5%.
So what this says is that 95% percent of the time-- 95th confidence interval-- if the election were actually conducted, the candidate would get somewhere between 48% and 56% of the vote.
So 95% percent of the time, 95% percent of the elections, the candidate would get between 48% and 56% of the votes.
So we have two things, the range and our confidence that the value will lie within that range.
When they make those assumptions, when you see something like that in the press, they are assuming that elections are random trials that have a normal distribution.
That's an implicit assumption in the calculation that tells us this.
The nice thing here is that there is something called the empirical rule, which is used for normal distributions.
They give us a handy way to estimate confidence intervals given the mean and the standard deviation.
If we have a true normal distribution, then roughly speaking, 68% of the data are within the one standard deviation above the mean.
And 95% percent within two standard deviations.
And almost all, 99.7%, will fall within three.
These values are approximations.
They're not exactly right.
It's not exactly 68 and 95.
But they're good enough for government work.
So we can see this here.
And this is what people use when they think about these things.
Now this may raise an interesting question in your mind.
How do the pollsters go about finding the standard deviation?
Do they go out and conduct a 100 separate polls and then do some math?
Sort of what we've been doing.
You might hope so, but that's not what they do because it's expensive.
And nobody wants to do that.
So they use another trick to estimate the standard deviation.
Now, you're beginning to understand why these polls aren't always right.
And the trick they use for that is something called the standard error, which is an estimate of the standard deviation.
And you can only do this under the assumption that the errors are normally distributed and also that the sample population is small.
And I mean small, not large.
It's small relative to the actual population.
So this gets us to one of the things we like about the normal distribution that in fact, it's often an accurate model of reality.
And when people have done polls over and over again, they do discover that, indeed, the results are typically normally distributed.
So this is not a bad assumption.
Actually, it's a pretty good assumption.
So if we have p, which is equal to the percentage sample.
And we have n, which is equal to the sample size, we can say that the standard error, which I'll write SE, is equal to the formula p times 100-- because we're dealing with percentages-- minus p divided by n to the 1/2, the square root of all of this.
So if for example, a pollster were to sample 1,000 voters, and 46% of them said that they'll vote for Abraham Lincoln-- we should be so lucky that Abraham Lincoln were running for office today-- the standard error would be roughly 1.58%.
We would interpret this to mean that in 95% percent of the time, the true percentage of votes that Lincoln would get is within two standard errors of 46%.
I know that's a lot to swallow quickly.
So as always, we'll try and make sense of it by looking at some code.
By now, you've probably all figured out that I'm much more comfortable with code than I am with formulas.
So we're going to conduct a poll here.
Not really, we're going to pretend we're conducting a poll.
n and p. We'll start with no votes.
And for i in range n, if random.random is less than p over 100, the number of votes will be increased by 1.
Otherwise, it will stay where it was and will return the number of votes.
Nothing very dramatic.
And then, we'll test the error here.
So n equals 1,000, p equals 46, the percentage of votes that we think Abraham Lincoln is going to get.
We'll run 1,000 trials.
Results equal that.
For t in range number of trials results.append, I'll run the poll.
And we'll look at the standard deviation, and we'll look at the results.
And we'll print the fraction of votes and the number of polls.
All right, let's see what we get when we do this.
Well, pretty darn close to a normal distribution.
Kind of what we'd expect.
The fraction of votes peaks at 46%.
Again what we'd expect.
But every once in while, it gets all the way out here to 50 and looks like Abe might actually win an election.
Highly unlikely in our modern society.
And over here, he would lose a lot of them.
If we look here, we'll see that the standard deviation is 1.6.
So it turns out that the standard error, which you'll remember we computed using that formula to be 1.58-- you may not remember it because I said it and didn't write it down-- is pretty darn close to 1.6.
So remember the standard error is an attempt to just use a formula to estimate what the standard deviation is going to be.
And in fact, we use this formula, very simple formula, to guess what it would be.
We then ran a simulation and actually measured the standard deviation, no longer a guess.
And it came out to be 1.6.
And I hope that most of you would agree that that was a pretty good guess.
And so therefore because, if you will, the differences are normally distributed, the distribution is normal.
It turns out the standard error is a very good approximation to the actual standard deviation.
And that's what pollsters rely on and why polls are actually pretty good.
So now the next time you read a poll, you'll understand the math behind it.
In a subsequent lecture, we'll talk about some ways they go wrong that have nothing to do with getting the math wrong.
Now, of course, finding a nice tractable mathematical model, the normal distribution, is of no use if it provides an inaccurate model of the data that you care about.
Fortunately, many random variables have an approximately normal distribution.
So if for example, I were doing a real lecture and I had 100 students in this room, and I were to look at the heights of the students, we would find that they are roughly normally distributed.
Any time you take a population of people and you look at it, it's quite striking that you do end up getting a normal distribution of the heights.
You get a normal distribution of the weights.
Same thing will be true if you look at plants, all sorts of things like that.
I don't know why this is true.
It just is.
What I do know is that-- and probably this is more important-- many experimental setups, and this is what we're going to be talking about going forward, have normally distributed measurement errors.
This assumption was used first in the early 1800s by the German mathematician and physicist Carl Gauss.
You've probably heard of Gauss, who assumed a normal distribution of measurement errors in his analysis of astronomical data.
So he was measuring various things in the heavens.
He knew his measurements of where something was were not 100% accurate.
And he said, well, I'll bet it's equally likely it's to the left of where I think it is or the right as where I think it is.
And I'll bet the further I get from its true value, the less likely I am to guess that's where it is.
And he invented at that time what we now call the normal distribution.
Physicists insist today still on calling it a Gaussian distribution.
And it turned out to be a very accurate model of the measurement errors he would make.
If you guys are in a chemistry lab, or a physics lab, or a bio lab, mechanical engineering lab, any lab where you're measuring things, it's pretty likely that the mistakes you will make will be normally distributed.
And it's not just because you were sloppy in the lab.
Actually, if you were sloppy in the lab, they may not be normally distributed.
If you're not sloppy in the lab, they'll be normally distributed.
It's true of almost all measurements.
And in fact, most of science assumes normal distributions of measurement errors in reaching conclusions about the validity of their data.
And we'll see some examples of that as we go forward.
Thanks a lot.
See you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
He ended up last Thursday's lectures talking about Gaussian distributions.
As he said, one of the interesting things about a Gaussian is it can be fully characterized by its mean and its standard deviation.
And this concept of being able to take a curve and characterize it with a small number of parameters is something we'll continue to see as a very important way of looking at modeling physical systems.
And in fact, that is the part of the term that we've entered.
And it's a part of the term we'll spend a lot of time on.
And the whole issue is, how do we construct computational models that will help us understand the real world?
When we can, we love to model distributions as Gaussians, or normal, because they're so nicely characterized.
We have nice rules of thumb that tell us how close things lie to the mean, et cetera.
However, it's important to understand that if something is not actually normally distributed and we pretend it is, we can get very misleading results out of our model.
So let's think about the fact that not all distributions are normal.
So consider rolling a single die.
Each of the 6 outcomes is equally probable.
So we would not expect to see a peak, say, at 3 or 4, and a trough at 1.
A 3 or a 4 is the same probability as 1.
Similarly, if one thinks about the Massachusetts state lottery, or any fair lottery, the probability of each number coming up is the same.
So it would be a flat line.
If you had a million numbers, the probability of each number is 1 over a million.
And so if you plotted the probability of each number, again, you'd get a flat line.
Such distributions are called uniform.
Each result is equally probable.
We can fully characterize a uniform distribution with a single parameter, its range.
If I tell you it ranges over 1 to a million, that's all you need to know to know what the distribution looks like.
So they're even simpler than normal distributions.
Uniform distributions occur quite often in games devised by humans, but almost never in nature.
And typically they're not very useful for modeling complex systems.
We have to work hard to invent something that's normal.
Most things are not naturally that way.
I'm sorry, to invent things that are uniform.
Normal, as we saw last time, occurs all the time in nature.
The other thing that occurs quite frequently are exponential distributions.
They're used in a lot of different ways.
For example, people who are trying to plan things like highway systems use exponential distributions to model inter-arrival times, how much time there is between each car, say, entering the Mass Turnpike.
We'll see many other examples of them.
The key thing about them is they have the property of being memoryless.
They are in fact the only continuous distributions that are memoryless.
So let's look at an example with which some of you are more familiar than you want to be, the concentration of a drug in the human body.
For those who are watching on OpenCourseWare, it's not because all the students are drug users.
It's because they're working on a problem set that involves modeling drugs in the human body.
I don't know how many of you are drug users, all right?
Assume that at each time step, each molecule has a probability p of being cleared by the body.
The system is memoryless in the sense that at each step, the probability of a particular molecule being cleared is independent of what happened at the previous steps.
So the fact that a molecule didn't get cleared at time t has no impact on whether or not it will be cleared at time t1.
The probability doesn't go up as it doesn't get cleared.
So it's independent of the previous steps.
So at time t equals 1, what's the probability of the molecule still being in the human body?
If the probability of being cleared at each step is p, it's 1 minus p, right?
So if the probability of being cleared was 0.5 at each time step, the time that it still exists after the first time step is 1 minus 0.5-- i.e., 0.5.
So what's the probability of it still being in the human body at time t equals 2?
Well, it wasn't cleared at time 1.
Since it's memoryless, whether or not it's cleared at time 2 is also 1/p.
And so it existing still after two steps is going to be 1 minus p squared.
We saw that with independent probabilities.
And the nice thing about working with exponential distributions is we know that the probabilities are independent.
More generally, its still being in the body at time t is going to be 1 minus p to the t. So we have a nice closed-form solution that will give us the probability of each molecule surviving until time t. All right?
So now let's look at the question of, suppose that at time t equals 0, there are m0 molecules.
Now we can ask, how many molecules are there likely to be at any time t?
Well, let's write a little program to look at that.
So that's this program, clear.
We'll start with n, the number of molecules, the probability of clearing it at each step, and the number of steps.
And we'll keep track of the num remaining.
So at the beginning, we have n molecules remaining.
And then for t in range steps, we're just going to multiply n, the number we started with, times the probability of each molecule still existing.
And then we'll plot it.
Does that make sense?
So this is a tiny bit of code that basically implements that formula over on the board.
Let's run it.
And we'll run it starting with a 1,000 molecules, a probability of each being cleared of 0.01, and we'll look at 500 time steps.
All right.
This is kind of interesting.
We're getting a straight line.
That doesn't look like an exponential, does it?
Or does it?
Why do we have a straight line here?
Somebody?
Because I used a semilog axis.
So let's look at it now without that.
We now see something that really does look like exponential decay.
It drops very quickly in the beginning, and then it asymptotes towards 0.
But of course it never quite gets there in a continuous model.
If we had a discrete model, we would eventually have to get to 0, because that last molecule would either get cleared or not.
But in a continuous world-- which is in this case probably not a good model, or not a perfect model I should say, because it allows us to have a quarter of a molecule there, which we kind of know is physiologically nonsense, biochemically nonsense.
But you can see we get this exponential decay.
But as we saw previously, if we plot an exponential on a log axis, as the math would tell us, we get a straight line.
And that, in fact, is a very simple and nice way to see whether you have an exponential distribution.
Put it on a log axis, see if it's straight.
It's a good trick, and one we use a lot.
OK.
So there, I took the physical model I described and derived, through a little bit of math, what the result should be, and implemented some code to give us a plot of what that told us.
Let's look at a different way of doing it.
I could've instead written a Monte Carlo simulation to do the same kind of thing.
So here, instead of working out the probabilities, I just tried to write some code that exactly mimicked the physical process that I was talking about.
So instead of knowing that I could just look at 1 minus p to the t, at each step, I cleared some molecules.
I just used random.random.
If I came out with something less than the clear probability, I got rid of that molecule.
And I did that for each molecule, deciding whether or not it should be cleared.
For molecule m in range, looking at all the remaining molecules, I either clear one or I don't.
And then I can plot that.
So let's look what happens if I compare the two results.
So I'm going to do the original analytical model of clear, and then the simulation model of clearing, and see what I get.
Well, much to my relief, I get kind of the same curve.
Not exactly.
You'll notice that the blue curve, the analytical model, is a beautiful smooth curve, whereas the red curve has got a little bit of jaggies.
It's clearly very similar to the blue curve, but not identical.
It doesn't surprise me.
There is some randomness in there.
And in fact, I could have gotten unlucky and gotten something that didn't look like the blue curve.
But given the sample size, that would have been quite surprising.
Which of these two models do you like better?
So we've got two models.
We've got one I'll call the analytic model, and one I'll call the simulation model.
Both show exponential decay.
That is to say the number of molecules declines exponentially, quite quickly.
But they're not quite identical.
So which would we prefer?
Or which would you prefer?
There is no right answer for this.
Just for fun, I'll ask for a poll.
Who prefers the analytical model?
Who prefer the simulation?
All right.
Somebody who prefers the analytical, tell me why
It looks a lot nicer
Well, all right.
It looks a lot nicer.
That's kind of human nature, to prefer something that looks prettier.
On the other hand, what we're really interested in is the question of not aesthetics, but fidelity to the actual physical situation.
A straight line might look even nicer, but it wouldn't be accurate.
So when we think about evaluating a model, what we really should be asking, I think, are two questions.
One is fidelity.
And another way to think about that is credibility.
Typically, we're creating a model because we don't know the actual answer.
And we're trying to see what might actually happen if we, say, ran a physical experiment.
And so we have to ask the question of, do we believe the results the model are giving us.
And so that sort of boils down to not a question of mathematics, but a question of reasoning.
Can we look at the model and convince ourselves that it is accurate?
And the other question is utility.
And I can think about that as, in some sense, what questions are answerable with the model?
So the first one is pretty much a question of personal preference.
And for this particular simulation, which is pretty simple, or this particular model, it's hard to argue that one is more believable than the other.
I might argue the second is more believable, because it's a direct implementation of the physical system.
I didn't rely on my math being right.
But the math is pretty simple here.
What's, I think, more apparent is in this case there is some additional utility offered by the simulation model.
And it's often true of that, simulation models, that we can ask what-if questions, because we can easily change the model to be slightly different in ways that is usually harder for an analytic model.
For example, suppose these drug molecules had this peculiar property that every 100 time steps, they could clone themselves.
And so every molecule that was there became two molecules.
Unlikely for the drug.
Not so unlikely for, say, a bacterium or a virus, as you've seen.
Well, a little hard to figure out how to do the probabilities in the case that that happens, because we'll no longer get this beautiful, simple exponential decay.
But quite easy to think about how we would change the simulation model, which is what I have done here.
So I said here, if time is not equal to 0 and time is evenly divisible by 100, then I'm just going to double the number of molecules.
Every living molecule will clone itself.
And now we'll see what we get.
Well, we get this rather peculiar-looking sawtooth distribution.
We still have, overall, an exponential decay.
But we see every once in while it jumps up, and then it comes down.
It's not so easy to write a simple closed-form formula that describes this, but very easy to produce a simulation model that gives you some insight to what's happening here.
And that's, I think, one of the great attractions of simulation modeling, is we get to do this sort of thing.
Many, many physical systems exhibit exponential decay or exponential growth.
For example, people in Japan now are very interested in half-life of various radioactive particles.
And when we talk about half-life, we mean that there is exponential decay in radioactivity.
That's what half-life is.
So people are looking at what is the half-life of iodine, say, versus other radioactive particles.
We also see exponential growth a lot.
I used to have a swimming pool which I had to maintain, and I realized if I let the algae get out of control in the pool, it went from having very little algae to having a lot of algae very quickly, because the algae doubles every period.
And so all of a sudden, it takes off.
So exponential growth is-- exponential decay are important things.
We see them all the time.
People use the word very carelessly when they mean quick growth.
They say exponential.
But of course, it has a very specific meaning.
OK. We've now, for the moment at least, finished our short venture into probability and distributions.
We'll come back to it a little bit when we talk about how to lie with statistics.
But before we do that, before we leave probability for a while, just for fun, I want to pose to you one of these probability problems that hurts people's heads.
It's a very popular one.
How many people here have heard of the Monty Hall problem?
OK, a lot of you.
So as we play the game, those of you who know the answer, I'll ask your forbearance not to blurt it out.
So it's a wonderful problem.
It's so exciting that people have written books about it.
So here's how it works.
This is from a game show called, I think, Let's Make a Deal, with the host, Monty Hall, who did it forever.
So the way it works is you start with three doors.
Behind one of the doors is a great prize-- for example, an automobile.
Behind each of the other doors is a booby prize, typically a goat.
I don't know why people don't like goats, but apparently they don't.
So the way it works is Monty invites someone from the audience, chosen on the basis of their outlandish costumes.
And they come down and they're told what wonderful prize is behind one of the doors.
And then they're asked to choose a door.
So the person might choose a door and say, I'll choose door number one.
Monty then opens one of the other two doors.
He knows which doors have the goats and which door has the car.
He opens a door with the goat.
So now there are two doors left.
And he asks the contestant, do you want to switch.
Do you want to stick with door one or would you like to switch to door two?
And the Monty Hall problem is, what should she do?
And the audience will always shout out advice.
So I do have a simulation of that.
I'd like to run.
I need three people to volunteer to be doors.
Come on, three doors.
It's not so hard.
Come on down.
And I need one person to volunteer to be the contest.
Is anybody in a costume here?
I don't know.
Mitch is kind of in one, but-- all right.
These are the contestants.
All right, you're door number (2).
You're door number (1).
You are door number (3).
A contestant please.
There's $1 in one of these.
You can actually win something of value.
All right, we have a contestant coming down.
Oh, all right, we have two contestants coming down.
All right.
The aisle wins.
All right, choose a door.
You choose door number (2).
All right, let us open door number (1).
And let's see what's in door number (1).
Show it to the class.
It is a goat.
Now you have a choice.
You can stick with your original decision, or you can switch to door number (3).
Suggestions?
Switch
Switch
Switch
Switch
Lower one
Don't switch it
Switch
All right.
She is going to stick with door number (2).
Let us open door number (2).
She wins $1.
It is yours.
Don't spend it all at once.
Thank you, everybody.
All right, now, was she lucky or was she smart, is the question?
Does it matter?
This was a subject of enormous debate in the mathematical community.
In 1991, Parade magazine published a correct solution to the problem, and approximately 10,000 readers, including a 1,000 with PhDs in mathematics, wrote to Parade telling them they had published the wrong solution.
And the debate roiled on.
So who thinks she was lucky and who thinks it actually matters whether you switch?
Who thinks it matters, those who don't know the problem?
Who thinks it doesn't matter?
All right.
The doesn't-matters win by a small margin.
And in fact, that's what the readers of Parade thought.
But they were wrong.
It matters a lot whether you switch.
Let's do the analysis first, analytically, and then we'll do a simulation.
So the player makes a choice.
And this is some interesting ways to think about probability.
And with the probability of 1/3, the player has chosen the correct door.
All right?
Now that means that with a probability of 2 out of 3, the car lies behind one of the other two doors.
Now here's the key step.
Monty opens a door that he knows does not contain the prize.
The key thing to notice here is the choice of doors is not independent of the choice of the player, because Monty will never choose the door that the player has initially picked.
Now since the probability of the prize being behind the two remaining doors is 2 out of 3, the probability of the prize being behind one of the doors that he did not open is 2 out of 3 -- in fact, behind the other door.
In fact, you were extraordinarily lucky to win the dollar, because switching doubles the odds of winning.
Because remember, your odds of winning were 1 out of 3 when you first chose the door.
That left two doors.
The probability of the car being behind one of those two doors was 2/3.
Monty opened the one that didn't contain the car, because he knew it contained a goat.
So that must mean the probability of the car being behind the remaining door is 2 out of 3.
So you double your odds of winning.
The logic is kind of clear.
It didn't stop people from aggressively debating it for the longest of times.
And I kind of didn't believe it myself.
So I did what I usually do, is I wrote some code.
And let's look at two pieces of code here.
And again, the theme here is how we can use simulation models to understand slightly complex, or more than slightly complex, situations.
So here's the way the game works.
So I've got a simple simulation that counts the number of wins.
And the way it's done is, for t in range number of trials, the contestant picks 1, 2, or 3 at random.
I've tried to mimic exactly the game.
So the car is behind one of the doors.
The contestant guesses a door.
And then there's this 'choose' function to open one of the two.
And we're going to have 2 ways of choosing which door gets opened.
So the Monty Hall way-- Monty chooses.
He takes the guessed door and the prize door, and he opens the non-guess that contains the goat.
So if (1) is the guessed door, and (1) is not the guessed door and it's not the prize door, then he opens (1).
Same thing for (2).
And if (1) or (2) is not the choice, he opens (3).
As opposed to the random choose function, which just chooses at random between the doors that weren't guessed.
So it might open the car, at which point the contest is told, sorry, you lose, you don't even have a choice anymore.
We're then going to run the simulation with Monty choosing and random choice, and see what we get.
So you've got the code on the handout to do this.
I'm not going to go over the details.
The thing to notice about it is it's yet another example of how we can use PyLab to do some interesting plots.
This time I'm going to print a pie chart, just to show that we can do those.
And let's see what happens.
So people understand what's going on?
That I've got these two functions, montyChoose and randomChoose.
I'm using those functions as parameters, a very convenient thing, and running the simulation each way.
And let's see what happens.
All right.
So what we see here is, when I run montyChoose, sure enough, it comes out to about 2/3 of the time, you win if you change, and only 1/3 of the time if you don't, pretty close to what the math predicts.
In fact, sort of astonishingly close.
On the other hand, if Monte had been just choosing at random, then we see it really doesn't matter whether you switch or not.
So again, from a probability point of view, we see how subtle these things can be based upon whether decisions are independent of previous decisions, or not independent.
And we also see, in some sense, that we can write a very small piece of code that actually provides a simulation of a real, in this case, game, and we can have, I think, a lot of confidence.
We can look at the code and say, is it really the way the game is described?
Yes.
And then we get nice results that tell us what to do.
And in this case it tells us that, if Monty is choosing based upon what he knows, then by all means, you should switch.
And I'm sorry that it didn't work out that way when we played the game, but that's the way probabilities are, that you didn't switch and you lucked out.
So now you're a rich lady.
All right.
So that's one thing we can do.
One more thing I want to talk about, before we leave the subject of Monte Carlo simulations-- it's pretty clear that these kind of simulations are very useful for tackling problems in which predictive non-determinism plays a role.
And at first blush, you might think that, OK, that's the only time we should use a Monte Carlo simulation, when there's some inherent randomness in the problem, and therefore it's hard to model analytically, and therefore we'll use randomness in the code.
Interestingly enough, particularly in recent years, but for quite a while, people have understood the notion of using randomized algorithms to solve problems in which randomness plays no role.
And that's a little surprising, but an incredibly useful concept to put in your bag of tricks, the ability to use randomization to solve problems that are not random.
So let me talk about an example.
Consider the concept of pi.
Pi has been around for a long time.
For thousands of years, people have known that there's a constant-- called pi since about the 18th century-- associated with circles, such that the circumference of a circle is always going to be equal to pi times the diameter.
The area of a circle is always going to be pi r-squared, et cetera.
So for thousands of years, people knew that there was such a constant.
They just didn't know what it was.
And there's a long and beautiful history of people attempting to estimate pi.
About the earliest estimate I've found is from the Egyptians, in something called the Rhind Papyrus, from 1650 BC, or thereabouts.
And it estimated pi to be 4 times-- let me get this right-- 8/9 squared, which is 3.16, more or less.
That was pretty good.
About a 1,000 years later, an estimate of pi appears in the Bible.
Or at least, it's implied by the Bible in a description of one of Solomon's construction projects.
It says, "And he made a molten sea, 10 cubits from the one brim to the other.
It was round all about, and his height was five cubits.
And a line of 30 cubits did compass it round about."
So you can take that and solve for pi, because you've given the circumference, and other details, the diameter.
You can solve for pi.
And you see, if you do that, pi comes out to be exactly 3.
Not quite as accurate as the Egyptians had 1,000 years earlier.
Now perhaps the Bible is wrong.
I don't want to offend anybody.
Or perhaps the molten sea wasn't perfectly circular.
Or maybe the circumference was measured from the wall outside and the diameter from the inside.
Or maybe it was just that was a good enough number to use in construction, because a cubit was something like the length of your forearm.
Different people have different length forearms, and there was no reason to try and be more precise.
Who knows?
The best estimate of pi in ancient times was from Archimedes of Syracuse.
And he did something quite amazing for around 200 BC.
He didn't give the value of pi.
He said, I don't know what the value is, but I can give you an upper bound and a lower bound.
And he did this by carefully constructing a polygon with a huge number of tiny little straight lines that would approximate a circle, and then actually measuring things.
So he built a polygon with 96 sides, and concluded that pi was somewhere between 223 divided by 71, and 22/7.
Very sophisticated at the time, to be giving upper and lower bounds.
If we look at what the middle of that is, it's actually amazingly good.
It's 3.1418.
Not bad.
All right.
What does this have to do with Monte Carlo simulations?
Many years later-- in fact, in the 1700s, two French mathematicians invented another way of computing pi.
Buffon and Laplace.
Actually Buffon first proposed it.
He got it wrong.
Laplace corrected it.
And they said, we can find pi using a stochastic simulation.
They didn't use those words, but that's what they basically described.
And they talked about it in terms of needle-dropping.
So think about having a square, and inscribing in the square a circle.
And you'll excuse my lack of artistic ability.
So they put one of those on the floor.
And then they dropped needles, which would get carried around by the wind and land in some random place.
And they counted the number of needles that landed in the circle, and the number of needles that landed in the square but not in the circle.
And then they did a little math.
Let's assume for the sake of argument here that the radius of the circle is 1.
They observed the following equation must hold, that the needles in the circle over the needles in the square should be equal to the area of the circle divided by the area of the square.
It seems logical, if they're landing at random, that they would get distributed proportional to the area.
And then they solved for pi, knowing that the area of the circle is pi r-squared.
They could then say that pi-- in fact, in this case, since we know the radius is 1, and 1 squared is 1, that tells us that the area of the circle should be pi, right?
Pi times 1.
So they said pi is equal to the area of the circle, which is equal to the area of the square times the needles in the circle, divided by the needles in the square.
So they had that formula.
Unfortunately they couldn't drop enough needles to get a very good estimate.
So they described how to do it.
But this was an experiment.
They did the math, they had a nice formula.
They did not have an experimental apparatus that would actually let them drop enough needles to get a very good estimate.
It would take a lot of patience.
And maybe they wouldn't land at random.
Who knows what.
Fortunately, today, we have a much easier way to do that.
So we can write some code that does the simulation.
We're going to have a way to throw the needles or drop the needles.
And then what we're going to do is we're going to have a simulation that we're going to run, that's going to-- I don't know how many needles to drop.
I'm going to keep dropping needles until I get a small enough standard deviation that I can be confident that I have a bound on pi with some confidence interval.
In fact, I'm going to use 5% here, and use that rule of thumb that Mitch talked about last time, about standard deviations.
And I say, all right, I'm going to keep running the experiment until the standard deviation of trials is 5% or less.
Two standard deviations is small enough that I get my answer within some precision.
I'm going to ask here for a precision of 0.01.
And so therefore my standard deviation should be that precision divided by 4, because I'm looking for 2 standard deviations on either side of the mean, which is why I'm dividing by 4 and not by 2.
So I divide by 4 and I see what I get.
So let's run it.
And this will take a little bit of time.
It will take no time if I don't uncomment the code to actually run the experiment.
Estimate pi.
And we'll get some estimates.
So what we can see here is that my estimates change as I run experiments.
Every time I run this-- or not every time, I often get a different number of needles I need.
But you can see that my first estimate is not very good.
My estimates do get better, though not monotonically better.
But what does get monotonically better is the standard deviation gets smaller and smaller, which is what you would expect.
So there's no guarantee that by running a bigger trial, I get a more accurate result.
What there is a guarantee is that I can have more confidence in my result.
I could have gotten lucky and run a small number of needles and gotten it exactly right by chance.
But I would have been wrong to assume it was right, because let's pretend we didn't know what the value of pi was, a priori.
But what I can say here is since my standard deviation is now 0.002, and if we look at it, we'll see that these things are normally distributed, I can be pretty sure that the true value of pi is 3.1407 et cetera, plus or minus 0.00023, et cetera, with a 95% confidence.
So this is using the stuff that you saw in the last lecture to now combine that statistical background with this simulation to compute a pretty darn good estimate of pi.
And if I ran more needles, if I wanted to get more precise, I can get as precise as I want to be, as many digits of precision as I want.
So again, what we see here is that we've been able to solve a problem that had nothing to do with randomness.
The value of pi is not a random number.
And yet we used randomness to solve it.
A very common technique.
And we use some very simple statistics to know whether or not we should believe our solution.
And so those are the two lessons I want you to take home.
Now if you look at your handout, you'll see that at the bottom, I've used the same technique to do integration.
If you think about what integration means, when you ask, what is the integral of some formula, what you learned when you first looked at calculus was that that was the area under some curve, right?
That's what the integral is.
And you learned all sorts of complicated mathematics to solve complicated integrals.
Well, you can pose an integration problem exactly analogous to this.
You draw your curve.
You drops some needles.
You count how many fall under the curve, how many don't fall under the curve in some larger area.
And you can solve the integration.
And that's exactly what I've done here, where f is the function being integrated.
I won't go through the details.
Now in fact, this kind of simulation is not a good way to solve single integrals.
It's much better to use something like Simpson's rule, whatever that is.
But in fact, it is frequently used in practice for more complicated things, a double or triple integration, where the mathematics gets fairly complicated.
People will often solve those problems using a Monte Carlo simulation.
It's a practical method for tackling it.
And again, in your handout, you'll see a very simple piece of code that does a double integral.
All right.
That's all for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to go back to where I stopped at the end of Tuesday's lecture, when you let me pull a fast one on you.
I ended up with a strong statement that was effectively a lie.
I told you that when we drop a large enough number of pins, and do a large enough number of trials, we can look at the small standard deviation we get across trials and say, that means we have a good answer.
It doesn't change much.
And I said, so we can tell you that with 95% confidence, the answer lies between x and y, where we had the two standard deviations from the mean.
That's not actually true.
I was confusing the notion of a statistically sound conclusion with truth.
The utility of every statistical test rests on certain assumptions.
So we talked about independence and things like that.
But the key assumption is that our simulation is actually a model of reality.
You can recall that in designing the simulation, we looked at the Buffon-Laplace mathematics and did a little algebra from which we derived the code, wrote the code, ran the simulation, looked at the results, did the statistical results, and smiled.
Well, suppose I had made a coding error.
So, for example, instead of that 4 there-- which the algebra said we should have-- I had mistakenly typed a 2.
Not an impossible error.
Now if we run it, what we're going to see here is that it converges quite quickly, it gives me a small standard deviation, and I can feel very confident that my answer that pi is somewhere around 1.569.
Well, it isn't of course.
We know that that's nowhere close to the value of pi.
But there's nothing wrong with my statistics.
It's just that my statistics are about the simulation, not about pi itself.
So what's the moral here?
Before believing the results of any simulation, we have to have confidence that our conceptual model is correct.
And that we have correctly implemented that conceptual model.
How can we do that?
Well, one thing we can do is test our results against reality.
So if I ran this and I said pi is about 1.57, I could go draw a circle, and I could crudely measure the circumference, and I would immediately know I'm nowhere close to the right answer.
And that's the right thing to do.
And in fact, what a scientist does when they use a simulation model to derive something, they always run some experiments to see whether their derived result is actually at least plausibly correct.
Statistics are good to show that we've got the little details right at the end, but we've got to do a sanity check first.
So that's a really important moral to keep in mind.
Don't get seduced by a statistical test and confuse that with truth.
All right, I now want to move on to look at some more examples that do the same kind of thing we've been doing.
And in fact, what we're going to be looking at is the interplay between physical reality,-- some physical system, just in the real world-- some theoretical models of the physical system, and computational models.
Because this is really the way modern science and engineering is done.
We start with some physical situation-- and by physical I don't mean it has to be bricks and mortar, or physics, or biology.
The physical situation could be the stock market, if you will,-- some real situation in the world.
We use some theory to give us some insight into that, and when the theory gets too complicated or doesn't get us all the way to the answer, we use computation.
And I now want to talk about how those things relate to each other.
So imagine, for example, that you're a bright student in high school biology, chemistry, or physics-- a situation probably all of you who have been in.
You perform some experiment to the best of your ability.
But you've done the math and you know your experimental results don't actually match the theory.
What should you do?
Well I suspect you've all been in this situation.
You could just turn in the results and risk getting criticized for poor laboratory technique.
Some of you may have done this.
More likely what you've done is you've calculated the correct results and turned those in, risking some suspicion that they're too good to be true.
But being smart guys, I suspect what all of you did in high school is you calculated the correct results, looked at your experimental results, and met somewhere in between to introduce a little error, but not look too foolish.
Have any of you cheated that way in high school?
Yeah well all right.
We have about two people who would admit it.
The rest of you are either exceedingly honorable, or just don't want to admit it.
I confess, I had fudged experimental results in high school.
But no longer, I've seen the truth.
All right, to do this correctly you need to have a sense of how best to model not only reality, but also experimental errors.
Typically, the best way to model experimental errors-- and we need to do this even when we're not attempting to cheat-- is to assume some sort of random perturbation of the actual data.
And in fact, one of the key steps forward, which was really Gauss' big contribution, was to say we can typically model experimental error as normally distributed, as a Gaussian distribution.
So let's look at an example.
Let's consider a spring.
Not the current time of year, or a spring of water, but the kind of spring you looked at in 8.01.
The things you compress with some force then they expand, or you stretch, and then they contract.
Springs are great things.
We use them in our cars, our mattresses, seat belts.
We use them to launch projectiles, lots of things.
And in fact, as we'll see later, they're frequently occurring in biology as well.
I don't want to belabor this, I presume you've all taken 8.01.
Do they still do springs in 8.01?
Yes, good, all right.
So as you know, in 1676-- maybe you didn't know the date-- the British physicist, Robert Hooke, formulated Hooke's Law to explain the behavior of springs.
And the law is very simple, it's f equals minus kx.
In other words, the force, f, stored in the spring is linearly related to x, the distance the spring has been either compressed or stretched.
OK. so that's Hooke's law, you've all seen that.
The law holds true for a wide variety of materials and systems including many biological systems.
Of course, it does not hold for an arbitrarily large force.
All springs have an elastic limit and if you stretch them beyond that the law fails.
Has anyone here ever broken a Slinky that way.
Where you've just taken the spring and stretched it so much that it's no longer useful.
Well you've exceeded its elastic limit.
The proportionate of constant here, k, is called the spring constant.
And every spring has a constant, k, that explains its behavior.
If the spring is stiff like the suspension in an automobile, k is big.
If the spring is not stiff like the spring in a ballpoint pen, k is small.
The negative sign is there to indicate that the force exerted by the spring is in the opposite direction of the displacement.
If you pull a spring bring down, the force exerted by the spring is going up.
Knowing the spring constant of a spring is actually a matter of considerable practical importance.
It's used to do things like calibrate scales,-- one can use to weigh oneself, if one wants to know the truth-- atomic force microscopes, lots of kinds of things.
And in fact, recently people have started worrying about thinking that you should model DNA as a spring, and finding the spring constant for DNA turns out to be of considerable use in some biological experiments.
All right, so generations of students have learned to estimate springs using this very simple experiment.
You've probably most of you have done this.
Get a picture up here, all right.
So what you do is you take a spring and you hang it on some sort of apparatus, and then you put a weight of known mass at the bottom of the spring, and you measure how much the spring has stretched.
You then can do the math, if f equals minus kx.
We also have to know that f equals m times a, mass times acceleration.
We know that on this planet at least the acceleration due to gravity is roughly 9.81 meters per second per second, and we can just do the algebra and we can calculate k. So we hang one weight in the spring, we measure it, we say, we're done.
We now know what k is for that spring.
Not so easy, of course, to do this experiment if the spring is a strand of DNA.
So you need a slightly more complicated apparatus to do that.
This would be all well and good if we didn't have experimental error, but we do.
Any experiment we typically have errors.
So what people do instead is rather than hanging one weight on the spring, they hang different weights-- weights of different mass-- they wait for the spring to stop moving and they measure it, and now they have a series of points.
And they assume that, well I've got some errors and if we believe that our errors are normally distributed some will be positive, some will be negative.
And if we do enough experiments it will kind of all balance out and we'll be able to actually get a good estimate of the spring constant, k. I did such an experiment, put the results in a file.
This is just a format of the file.
The first line tells us what it is, it's the distance in meters and a mass in kilograms.
And then I just have the two things separated by a space, in this case.
So my first experiment, the distance I measured was 0.0865 and the weight was 0.1 kilograms.
All right, so I've now got the data, so that's the physical reality.
I've done my experiment.
I've done some theory telling me how to calculate k. And now I'm going to put them together and write some code.
So let's look at the code.
Think we'll skip over this, and I'll comment this out, so we don't see get pi get estimated over and over again.
So the first piece of code is pretty simple, it's just getting the data.
And again, this is typically the way one ought to structure these things.
I/O, input/output, is typically done in a separate function so that if the format of the data were changed, I'd only have to change this, and not the rest of my computation.
it opens the file, discards the header, and then uses a split to get the x values and the y values, all right.
So now I just get all the distances and all the masses-- not the x's and the y's yet, just distances and masses.
Then I close the file and return them.
Nothing that you haven't seen before.
Nothing that you won't get to write again, and again, and again, similar kinds of things.
Then I plot the data.
So here we see something that's a little bit different from what we've seen before.
So the first thing I do is I got my x and y by calling, GetData.
Then I do a type conversion.
What GetData is returning is a list.
I'm here going to convert a list to another type called an array.
This is a type implemented by a class supplied by PyLab which is built on top of something called NumPy, which is where it comes from.
An array is kind of like a list.
It's a sequence of things.
There's some list operations methods that are not available, like append, but it's got some other things that are extremely valuable.
For example, I can do point-wise operations on an array.
So if I multiply an array by 3, what that does is it multiplies each element by 3.
If I multiply one array by another, it does the cross products.
OK, so they're very valuable for these kinds of things.
Typically, in Python, one starts with a list, because lists are more convenient to build up incrementally than arrays, and then converts them to an array so that you can do the math on them.
For those of you who've seen MATLAB you're very familiar with the concept of what Python calls an array.
Those of you who know C or Pascal, what it calls an array has nothing to do with what Python or PyLab calls an array.
So can be a little bit confusing.
Any rate, I convert them to arrays.
And then what I'll do here, now that I have an array, I'll multiply my x values by the acceleration due to gravity, this constant 9.81.
And then I'm just going to plot them.
All right, so let's see what we get here.
So here I've now plotted the measure displacement of the spring.
Force in Newtons, that's the standard international unit for measuring force.
It's the amount of force needed to accelerate a mass of 1 kilogram at a rate of 1 meter per second per second.
So I've plotted the force in Newton's against the distance in meters.
OK. Now I can go and calculate k. Well, how am I going to do that?
Well, before I do that, I'm going to do something to see whether or not my data is sensible.
What we often do, is we have a theoretical model and the model here is that the data should fall on a line, roughly speaking, modular experimental errors.
I'm going to now find out what that line is.
Because if I know that line, I can compute k. How does k relate to that line?
So I plot a line.
And now I can look at the slope of that line, how quickly it's changing.
And k will be simply the inverse of that.
How do I get the line?
Well, I'm going to find a line that is the best approximation to the points I have.
So if, for example, I have two points, a point here and a point here, I know I can quote, fit a line to that curve-- to those points-- it will always be perfect.
It will be a line.
So this is what's called a fit.
Now if I have a bunch of points sort of scattered around, I then have to figure out, OK, what line is the closest to those points?
What fits it the best?
And I might say, OK, it's a line like this.
But in order to do that, in order to fit a line to more than two points, I need some measure of the goodness of the fit.
Because what I want to choose here is the best fit.
What line is the best approximation of the data I've actually got?
But in order to do that, I need some objective function that tells me how good is a particular fit.
It lets me compare two fits so that I can choose the best one.
OK, now if we want to look at that we have to ask, what should that be?
There are lots of possibilities.
One could say, all right let's find the line that goes through the most points, that actually touches the most points.
The problem with that is it's really hard, and may be totally irrelevant, and in fact you may not find a line that touches more than one point.
So we need something different.
And there is a standard measure that's typically used and that's called the least squares fit.
That's the objective function that's almost always used in measuring how good any curve-- or how well, excuse me, any curve fits a set of points.
What it looks like is the sum from L equals 0 to L equals the len of the observed points minus 1, just because of the way things will work in Python.
But the key thing is what we're summing is the observed at point L minus the predicted at point L-squared.
And since we're looking for the least squares fit, we want to minimize that.
The smallest difference we can get.
So there's some things to notice about this.
Once we have a quote fit, in this case a line for every x value the fit predicts a y value.
Right?
That's what our model does.
Our model in this case will take the independent variable, x, the mass, and predict the dependent variable, the displacement.
But in addition to the predicted values, we have the observed values, these guys.
And now we just measure the difference between the predicted and the observed, square it, and notice by squaring the difference we have discarded whether it's above or below the line-- because we don't care, we just care how far it's from the line.
And then we sum all of those up and the smaller we can make that, the better our fit is.
Makes sense?
So now how do we find the best fit?
Well, there's several different methods you could use.
You can actually do this using Newton's method.
Under many conditions there are analytical solutions, so you don't have to use approximation you can just compute it.
And the best news of all, it's built into PyLab So that's how you actually do it.
You call the PyLab function that does it for you.
That function is called Polyfit.
Polyfit takes three arguments.
It takes all of the observed X values, all of the observed Y values, and the degree of the polynomial.
So I've been talking about fitting lines.
As we'll see, polyfit can be used to fit polynomials of arbitrary degree to data.
So you can fit a line, you can fit a parabola, you can fit cubic.
I don't know what it's called, you can fit a 10th order polynomial, whatever you choose here.
And then it returns some values.
So if we think about it being a line, we know that it's defined by the y value is equal to ax plus b.
Some constant times the x value plus b, the y-intercept.
So now let's look at it.
We see here in fit data, what I've done is I've gotten my values as before, and now I'm going to say, a,b equals pyLab.polyfit of xVals, y values and 1.
Since I'm looking for a line, the degree is 1.
Once I've got that, I can then compute the estimated y values, a times pyLab.array.
I'm turning the x values into an array, actually I didn't need to do that since I'd already done it, that's okay-- plus b.
And now I'll plot it and, by the way now that I've got my line, I can also compute k. And let's see what we get.
All right, I fit a line, and I've got a linear fit, and I said my spring constant k is 21 point -- I've rounded it to 5 digits just so would fit nicely on my plot.
OK.
The method that's used to do this in PyLab is called a linear regression.
Now you might think it's called linear regression because I just used it to find a line, but in fact that's not why it's called linear regression.
Because we can use linear regression to find a parabola, or a cubic, or anything else.
The reason it's called linear, well let's look at an example.
So if I wanted a parabola, I would have y equals ax-squared plus bx plus c. We think of the variables, the independent variables, as x-squared and x.
And y is indeed a linear function of those variables, because we're adding terms.
Not important that you understand the details, it is important that you know that linear regression can be used to find polynomials other than lines.
All right, so we got this done.
Should we be happy?
We can look at this, we fit the best line to this data point, we computed k, are we done?
Well I'm kind of concerned, because when I look at my picture it is the best line I can fit to this, but wow it's not a very good fit in some sense, right.
I look at that line, the points are pretty far away from it.
And if it's not a good fit, then I have to be suspicious about my value of k, which is derived from having the model I get by doing this fit.
Well, all right, let's try something else.
Let's look at FitData1, which in addition to doing a linear fit, I'm going to fit a cubic -- partly to show you how to do it.
Here I'm going to say abcd equals pyLab.polyfit of xVals, yVals and 3 instead of 1.
So it's a more complex function.
Let's see what that gives us.
First let me comment that out.
So we're going to now compare visually what we get when we get a line fit versus we get a cubic fit to the same data.
Well it looks to me like a cubic is a much better description of the data, a much better model of the data, than a line.
Pretty good.
Well, should I be happy with this?
Well, let's ask ourselves in one question, why are we building the model?
We're building the model so that we can better understand the spring.
One of the things we often do with models is use them to predict values that we have not been able to run in our experiments.
So, for example, if you're building a model of a nuclear reactor you might want to know what happens when the power is turned off for some period of time.
In fact, if you read today's paper you noticed they've just done a simulation model of a nuclear reactor, in, I think, Tennessee, and discovered that if it lost power for more than two days, it would start to look like the nuclear reactors in Japan.
Not a very good thing.
But of course, that's not an experiment anyone wants to run.
No one wants to blow up this nuclear reactor just to see what happens.
So they do use a simulation model to predict what would happen in an experiment you can't run.
So let's use our model here to do some predictions.
So here I've taken the same program, I've called it FitData2, but what I've done is I've added a point.
So instead of just looking at the x values, I'm looking at something I'm calling extended x, where I've added a weight of 1 and a 1/2 kilos to the spring just to see what would happen, what the model would predict.
And other than that, everything is the same.
Oops, what's happened here?
Probably shouldn't be computing k here with a non-linear model.
All right, why is it not?
Come on, there it is.
And now we have to un-comment this out, un-comment this.
Well it fit the existing data pretty darn well, but it has a very strange prediction here.
If you think about our experiment, it's predicting not only that the spring stopped stretching, but that it goes to above where it started.
Highly unlikely in a physical world.
So what we see here is that while I can easily fit a curve to the data, it fits it beautifully, it turns out to have very bad predictive value.
What's going on here?
Well, I started this whole endeavor under an assumption that there was some theory about springs, Hooke's law, and that it should be a linear model.
Just because my data maybe didn't fit that theory, doesn't mean I should just fit an arbitrary curve and see what happens.
It is the case that if you're willing to get a high enough degree polynomial, you can get a pretty good fit to almost any data.
But that doesn't prove anything.
It's not useful.
It's one of the reasons why when I read papers I always like to see the raw data.
I hate it when I read a technical paper and it just shows me the curve that they fit to the data, rather than the data, because it's easy to get to the wrong place here.
So let's for the moment ignore the curves and look at the raw data.
What do we see here about the raw data?
Well, it looks like at the end it's flattening out.
Well, that violates Hooke's law, which says I should have a linear relationship.
Suddenly it stopped being linear.
Have we violated Hooke's law?
Have I done something so strange that maybe I should just give up on this experiment?
What's the deal here?
So, does this data contradict Hooke's law?
Let me ask that question.
Yes or no?
Who says no?
Hooke's law applies only for small displacements
Well, not necessarily small.
But only up to an elastic limit
Which is in the scheme of inifinitely small
Compared to infinity [INAUDIBLE]
Yes, sorry, up to the limit where the linearity breaks down
Exactly right.
Oh, I overthrew my hand here
I'll get it
Pick it up on your way out.
Exactly, it doesn't.
It just says, probably I exceeded the elastic limit of my spring in this experiment.
Well now, let's go back and let's go back to our original code and see what happens if I discard the last six points, where it's flattened out.
The points that seem to be where I've exceeded the limit.
So I can easily do that.
Do this little coding hack.
It's so much easier to do experiments with code than with physical objects.
Now let's see what we get.
Well, we get something that's visually a much better fit.
And we get a very different value of k. So we're a lot happier here.
And if I fit cubic to this you would find that the cubic and the line actually look a lot alike.
So this is a good thing, I guess.
On the other hand, how do we know which line is a better representation of physical reality, a better model.
After all, I could delete all the points except any two and then I would get a line that was a perfect fit, R-squared -- you know the mean squared error -- would be 0, right?
Because you can fit a line to any two points.
So again, we're seeing that we have a question here that can't be answered by statistics.
It's not just a question of how good my fit is.
I have to go back to the theory.
And what my theory tells me is that it should be linear, and I have a theoretical justification of discarding those last six points.
It's plausible that I exceeded the limit.
I don't have a theoretical justification of deleting six arbitrary points somewhere in the middle that I didn't happen to like because they didn't fit the data.
So again, the theme that I'm getting to is this interplay between physical reality,-- in this case the experiment-- the theoretical model,-- in this case Hooke's law-- and my computational model, -- the line I fit to the experimental data.
OK, let's continue down this path and I want to look at another experiment, also with a spring but this is a different spring.
Maybe I'll bring in that spring in the next lecture and show it to you.
This spring is a bow and arrow.
Actually the bow is the spring.
Anyone here ever shot a bow and arrow?
Well what you know is the bow has the limbs in it.
And when you pull back the string, you are putting force in the limbs, which are essentially a spring.
And when you release the spring goes back to the place it wants to be and fires the projectile on some trajectory.
I now am interested in looking at the trajectory followed by such a projectile.
This, by the way, is where a lot of this math came from.
People were looking at projectiles, not typically of bows, but of artillery shells, where the force there was the force of some chemical reaction.
OK, so once again I've got some data.
In a file, similar kind of format.
And I'm going to read that data in and plot it.
So let's do that.
So I'm going to get my trajectory data.
The way I did this, by the way, is I actually did this experiment.
I fired four arrows from different distances and measured the mean height of the four.
So I'm getting at heights 1, 2, 3, and 4.
Again, don't worry about this.
And then I'm going to try some fits.
And let's see what we get here.
So I got my data inches from launch point, and inches above launch point.
And then I fit a line to it.
And you can see there's a little point way down here in the corner.
The launch point and the target were at actually the same height for this experiment.
And not surprisingly, the bow was angled up, I guess, the arrow went up, and then it came down, and ended up in the target.
I fit a line to it.
That's the best line I can fit to these points.
Well, it's not real good.
So let's pretend I didn't know anything about projectiles.
I can now use computation to try and understand the theory.
Assume I didn't know the theory.
And what the theory tells me here, or what the computation tells me, the theory that the arrow travels in a straight line is not a very good one.
All right, this does not actually conform at all to the data, I probably should reject this theory that says the arrow goes straight.
If you looked at the arrows, by the way, in a short distance it would kind of look to your eyes like it was actually going straight.
But in fact, physics tells us it can't and the model tells us it didn't.
All right let's try a different one.
Let's compare the linear fit to a quadratic fit.
So now I'm using polyfit with a degree of 2.
See what we get here.
Well our eyes tell us it's not a perfect fit, but it's a lot better fit, right.
So this is suggesting that maybe the arrow is traveling in a parabola, rather than a straight line.
The next question is, our eyes tell us it's better.
How much better?
How do we go about measuring which fit is better?
Recall that we started by saying what polyfit is doing is minimizing the mean square error.
So one way to compare two fits would be to say what's the mean square error of the line?
What's the mean square error of the parabola?
Well, pretty clear it's going to be smaller for the parabola.
So that would tell us OK it is a better fit.
And in fact computing the mean square error is a good way to compare the fit of two different curves.
On the other hand, it's not particularly useful for telling us the goodness of the fit in absolute terms.
So I can tell you that the parabola is better than the line, but in some sense mean square error can't be used to tell me how good it is in an absolute sense.
Why is that so?
It's because mean square error -- there's a lower bound 0, but there's no upper bound.
It can go arbitrarily high.
And that is not so good for something where we're trying to measure things.
So instead, what we typically use is something called the coefficient of determination.
Usually written, for reasons you'll see shortly, as r squared.
So the coefficient of determination, R-squared, is equal to 1 minus the estimated error EE over MV, which is the variance in the measured data.
So we're comparing the ratio of the estimated error, our best estimate of the error, and a measurement of how variable the data is to start with.
As we'll see, this value is always less than 1, less than or equal to 1, and therefore R-squared is always going to be between 0 and 1.
Which gives us a nice way of thinking about it in an absolute sense.
All right, so where are these values?
How do we compute them?
Well, I'm going to explain it the easiest way I know, which is by showing you the code.
So I have the measured values and the estimated values.
The estimated error is going to be-- I take estimated value, the value given me by the model, subtract the measured value, and square it and then I just sum them.
All right, this is like what we looked at for the mean square error, but I'm not computing the mean, right.
I'm getting the total of the estimated errors.
I can then get the measured mean, which is the measured sum, divided by the length of the measurement.
That gives me the mean of the measured data.
And then my measured variance is going to be the mean of the measured data minus each point of the measured data squared, and then summing that.
So just as we looked at before when we looked at the coefficient of variation, and standard deviation, by comparing how far things stray from the mean, that tells us how much variance there is in the data.
And then I'll return 1 minus that.
OK, Tuesday we'll go look at this in more detail.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning, everybody.
All right, I ended up the last lecture talking about how to calculate the absolute goodness of fit using something called the coefficient of determination.
It's usually spelled as R-squared, or R2.
And the formula was quite simple.
We measure the goodness of the fit as R-squared equals 1 minus the estimated error divided by the measured variance.
As I observed, R-squared always lies between 0 and 1.
If R-squared equals 1, that means that the model that we constructed, the predicted values if you will, explains all of the variability in the data so that any change in the data is explained perfectly by the model.
We don't usually get 1.
In fact, if I ever got 1, I would think somebody cheated me.
R-squared equals 0, or conversely, it means there's no linear relationship at all between the values predicted by the model and the actual data.
That is to say, the model is totally worthless.
So we have code here, the top of the screen, showing how easy it is to compute R-squared.
And for those of you who have a little trouble interpreting the formula, because maybe you're not quite sure what EE and MV mean, this will give you a very straightforward way to understand it.
So now, we can run it.
We can get some answers.
So if we look at it, you'll remember last time, we looked at two different fits.
We looked at a quadratic fit and a linear fit for the trajectory of an arrow fired from my bow.
And we can now compare the two.
And not surprisingly, given what we know about the physics of projectiles, we see it is exactly what we'd expect, that the linear fit has an R-quared of 0.0177, showing that, in fact, it explains almost none of the data.
Whereas the quadratic fit has a really astonishingly good R-squared of 0.98, saying that almost all of the changes in the values of the variables, that is to say the way the y value changes with respect to the x value, is explained by the model.
i.e., we have a really good model of the physical situation.
Very comforting.
Essentially, it's telling us at less than 2% of the variation is explained by the linear model, 98% by the quadratic model.
Presumably the other 2% is experimental error.
Well, now that we know that we have a really good model of the data, we can ask the question, why do we care?
We have the data itself.
What's the point of building a model of the data?
And that, of course, is what we're getting when we run polyfit to get this curve.
The whole purpose of creating a model, or an important purpose of creating a model, is to be able to answer questions about the actual physical situation.
So one of the questions one might ask, for example, about firing an arrow is, how fast is it going?
That's kind of a useful thing to know if you're worried about whether it will penetrate a target and kill somebody on the other side, for example.
We can't answer that question directly looking at the data points.
You look at the data.
Well, I don't know.
But we can use the model to answer the question.
And that's an exercise I want to go through now to show you the interplay between models, and theory, and computation, and how we can use the three to answer relevant questions about data.
No, I do not want to check for new software, thank you.
In fact, let's make sure it won't do that anymore.
So let's look at the PowerPoint.
So here, we'll see how I'm using a little bit of theory, not very much, to be able to understand how to use the model to compute the speed of the arrow.
So what we see is we know by our model, and by the good fit, that the trajectory is given by y equals ax-squared plus bx plus c. We know that.
We also know from looking at this equation that the highest point, which I'll call yPeak, of the arrow must occur at xMid, the middle of the x-axis.
So if we look at a parabola, and it doesn't matter what the parabola is, we always know that the vertical peak is halfway along the x-axis.
The math tells us that from the equation.
So we can say yPeak is x times xMid squared plus b times xMid plus c. So now, we have a model that we can tell how high the arrow can get.
The next question I'll ask is if I fired the arrow from here, and it hits the target here-- I've exaggerated by it this way.
It's nowhere near this steep.
How long does it take to get from here to here?
We don't have anything about time in our data.
Yet, I claim we have enough information to go from the distance here and the distance here to how long it's going to take the arrow to get from here to the target.
Why do I know that?
What determines how long it's going to take to get from here to here?
It's going to be how long it takes it to fall that far.
It's going to be gravity.
Because we know that gravity, at least on this planet, is a constant or close enough to it.
Unless maybe the arrow were going a million miles.
And it's going to be gravity that tells me how long it takes to get from here to here.
And when it gets to the bottom, it's going to be here, So again, I can use some very simple math and say that the time will be the square root of 2 times the yPeak divided by the gravitational constant.
Because I know that however long it takes to get from this height to this height is going to be the same time it takes to get from this point to this point.
And that will therefore let me compute the average speed from here to here.
And once I know that, I'm done.
Now again, this is assuming no drag and things like that.
The thing that we always have to understand about a model is no model is actually ever correct.
On the other hand, many models are very useful and they're close enough to correct.
So I left out things like gravity, wind shear and stuff like that.
But in fact, the answer we get here will turn out to be very close to correct.
We can now go back and look at some code.
Get rid of this.
And so now, you'll see this on the handout.
I'm going to just write a little bit of code that just goes through the math I just showed you to compute the average x velocity.
Got a print statement here I use to debug it.
And I'm going to return it.
And then, we'll just be able to run it and see what we get.
Well, all right, that we looked at before.
I forgot to close the previous figure.
So now, I'm sure this is a problem you've all seen.
And we'll fix it the way we always fix things, just start over.
I'll bet you guys have also seen this happen.
What this is suggesting, as we've seen before, is that the process, the old process, still exists.
Not a good thing.
Again, I'm sure you've all seen these.
Let's make sure we don't have anything running here that looks like IDLE.
We don't.
Just takes it a little time.
There it is, all right.
All right, now we'll go back.
All of this happened because I forgot to close the figure and executed pyLab.show twice, which we know can lead to bad things.
So let's get rid of this.
Now, we'll run it.
And now, we have our figure just using the quadratic fit.
And we see that the speed is 136.25 feet per second.
Do I believe 136.25?
Not really.
I know it's the ballpark.
I confused precision with accuracy here by giving you it to two decimal places.
I can compute it as precisely as I want.
But that doesn't mean it's actually accurate.
Probably, I should have just said it's about 135 or something like that.
But it's pretty good.
And for those of who don't know how to do this arithmetic in your head like me, this is about 93 miles per hour.
And for comparison, the speed of sound, instead of 136 feet per second, is 1,100 feet per second.
So it's traveling pretty fast.
Well, what's the point of this?
I don't really care if you know how fast an arrow travels.
I don't expect you'll ever need to compute that.
But I wanted to show you this as an example of a pattern that we use a lot.
So what we did is we started with an experiment.
You didn't see this, but I actually stood in my backyard and shot a bunch of arrows and measured them, got real data out of that.
And this gave me some data about the behavior of a physical system.
That's what I get for wearing a tie.
Maybe it'll be quieter if I put it in my shirt.
Actually, it looks silly.
Excuse me.
I hope none of you will mind if I take my tie off?
It seems to be making noises in the microphone.
Maybe we should write a computation.
All right, so ends my experiment with trying to look dignified.
Not something I'm good at.
OK, had an experiment.
That gave us some data.
We then use computation to both find and very importantly evaluate a model.
It's no good just to find the model.
You need to do some evaluation to convince yourself that it's a good model of the actual physical system.
And then, finally, we use some theory and analysis and computation to derive the consequence of the model.
And then, since we believe the accuracy of the model, we assume this consequence was also a true fact about the physical system we started with.
This is a pattern that we see over and over again these days in all branches of science and engineering.
And it's just the kind of thing that you should get used to doing.
It is what you will do if you go onto a career in science or engineering.
OK, that's all I want to say now about the topic of data and experiments and analysis.
We will return to this topic of interpretation of data later in the semester near the end when we start talking about machine learning and clustering.
But for now, I want to pull back and start down a new track that will, I'm sure you'll be pleased to hear, dovetail nicely with the next few problem sets that you're going to have to work on.
What I want to talk about is the topic of optimization.
Not so much optimization in the sense of how do you make a program fast, though we will talk a little about that, but what people refer to as optimization problems.
How do we write programs to find optimal solutions to problems that occur in real life?
Every optimization problem we'll look at is going to have two parts.
There's going to be (1) an objective function that will either be maximized or minimized.
So for example, I might want to find the minimal air fare between Boston and Istanbul.
Or more likely, the minimum bus fare between Boston and New York.
So there's an objective function.
Sometimes, you find the least.
Sometimes, you find the most.
Maybe I want to maximize my income.
And (2) a set of constraints that have to be satisfied.
So maybe I want to find the minimum transportation, minimum cost transportation between Boston and New York subject to the constraint that it not take more than eight hours or some such thing.
So the objective function that you're minimizing or maximizing, and some set of constraints that must be obeyed.
A vast number of problems of practical importance can be formulated this way.
Once we've formulated in this systematic way, we can then think about how to attack them with a computation that will help us solve the problem.
You guys do this all the time.
I heard a talk yesterday by Jeremy Wertheimer, an MIT graduate, who founded a company called ITA.
If you ever use Kayak, for example, or many of these systems to find an airline fare, they use some of Jeremy's code and algorithms to solve these various optimization problems like this.
If you've ever used Google or Bing, they solve optimization problems to decide what pages to show you.
They're all over the place.
There are a lot of classic optimization problems that people have worked on for decades.
What we often do when confronted with a new problem, and it's something you'll get some experience on in problem sets, is take a seemingly new problem and map it onto a classic problem, and then use one of the classic solutions.
So we'll go through this section of the course.
And we'll look at a number of classic optimization problems.
And then, you can think about how you would map other problems onto those.
This is the process known as problem reduction, where we take a problem and map it onto an existing problem that we already know how to solve.
I'm not going to go through a list of classic optimization problems right now.
But we'll see a bunch of them as we go forward.
Now, an important thing to think about when we think about optimization problems is how long, how hard, they are to solve.
So far, we have looked at problems that, for the most part, have pretty fast solutions, often sub-linear, binary search, sometimes linear, and at worst case, low-order polynomials.
Optimization problems, as we'll see, are typically much worse than that.
In fact, what we'll see is there is often no computationally efficient way to solve them.
And so we end up dealing with approximate solutions to them, or what people might call best effort solutions.
And we see that as an increasing trend in tackling problems.
All right, enough of this abstract stuff.
Let's look at an example.
So one of the classic optimization problems is called the knapsack problem.
People know what a knapsack is?
Sort of an archaic term.
Today, people would use the word backpack.
But in the old days, they called them knapsacks when they started looking at these things.
And the problem is also discussed in the context of a burglar or various kinds of thieves.
So it's not easy being a burglar, by the way.
I don't know if any of you ever tried it.
You've got some of the obvious problems, like making sure the house is empty and picking locks, circumventing alarms, et cetera.
But one of the really hard problems a burglar has to deal with is deciding what to steal.
Because you break into the typical luxury home-- and why would you break into a poor person's house if you were a burglar-- there's usually far more to steal than you can carry away.
And so the problem is formulated in terms of the burglar having a backpack.
They can put a certain amount of stuff in it.
And they have to maximize the value of what they steal subject to the constraint of how much weight they can actually carry.
So it's a classic optimization problem.
And people have worked for years at how to solve it, not so much because they want to be burglars.
But as you'll see, these kinds of optimization problems are actually quite common.
So let's look at an example.
You break into the house.
And among other things, you have a choice of what to steal.
You have a rather strange looking clock, some artwork, a book, a Velvet Elvis in case you lean in that direction, all sorts of things.
And for some reason, the owner was nice enough to leave you information about how much everything cost and how much it weighed.
So you find this piece of paper.
And now, you're trying to decide what to steal based upon this in a way to maximize your value.
How do we go about doing it?
Oh, I should show you, by the way.
There's a picture of a typical knapsack.
All right, it's almost Easter, after all.
Well, the simplest solution is probably a greedy algorithm.
And we'll talk a lot about greedy algorithms because they are very popular and often the right way to tackle a hard problem.
So the notion of a greedy algorithm is it's iterative.
And at each step, you pick the locally optimal solution.
So you make the best choice, put that item in the knapsack.
Ask if you have room, if you're out of weight.
If not, you make the best choice of the remaining ones.
Ask the same question.
You do that until you can't fit anything else in.
Now of course, to do that, that assumes that we know at each stage what we mean by locally optimal.
And of course, we have choices here.
We're trying to figure out, in some sense, what greedy algorithm, what approach to being greedy, will give us the best result.
So one could, for example, say, all right.
At each step, I'll choose the most valuable item and put that in my knapsack.
And I'll do that till I run out of valuable items.
Or, you could, at each step, say, well, what I'm really going to choose is the one that weights the least.
That will give me the most items.
And maybe that will give me the most total value when I'm done.
Or maybe, at each step, you could say, well, let me choose the one that has the best value to weight ratio and put that in.
And maybe that will give me the best solution.
As we will see, in this case, none of those is guaranteed to give you the best solution all the time.
In fact, as we'll see, none of them is guaranteed to be better than any of the others all the time.
And that's one of the issues with greedy algorithms.
I should point out, by the way, that this version of the knapsack problem that we're talking about is typically called the 0/1 knapsack problem.
And that's because we either have to take the entire item or none of the item.
We're not allowed to cut the Velvet Elvis in half and take half of it.
This is in contrast to the continuous knapsack problem.
If you imagine you break into the house and you see a barrel of gold dust, and a barrel of silver dust, and a barrel of raisins, what you would do is you would fill your knapsack with as much gold as you could carry, or until you ran out of gold.
And then, you would fill it with as much silver as you could carry.
And then, if there's any room left, you'd put in the raisins.
For the continuous knapsack problem, a greedy algorithm provides an optimal solution.
Unfortunately, most of the problems we actually encounter in life, as we'll see, are 0/1 knapsack problems.
You either take something or you don't.
And that's more complicated.
All right, let's look at some code.
So I'm going to formulate it.
I'm first going to start by putting in a class, just so the rest of my code is simpler.
This is something we've been talking about, that increasingly people want to start by putting in some useful data abstractions.
So I've got a class item where I can put in the item.
I can get its name.
I get its value.
I can get its weight.
And I can print it.
Kind of a boring class, but useful to have.
Then, I'm going to use this class to build items.
And in this case, I'm going to build the items based upon what we just looked at, the table that-- I think it's in your hand out.
And it's also on this slide.
Later, if we want, we can have a randomized program to build up a much bigger choice of items.
But here, we'll just try the clock, the painting, the radio, the vase, the book, and the computer.
Now comes the interesting part.
I've written a function, greedy, that takes three arguments-- the set of items that I have to choose from, makes sense, the maximum weight the burglar can carry.
And there's something called key function, which is defining essentially what I mean by locally optimal.
Then, it's quite simple.
I'm going to sort the items using the key function.
Remember, sort has this optional argument that says, what's the ordering?
So maybe I'll order it by value.
Maybe I'll order it by density.
Maybe I'll order it by weight.
I'm going to reverse it, because I want the most valuable first, not the least valuable, for example.
And then, I'm going to just take the first thing on my list until I run out of weight, and then I'm done.
And I'll return the result and the total value.
To make life simple, I'm going to define some functions.
These are the functions that I can use for the ordering.
Value, which is just return the value of the item.
The inverse of the weight, because I'm thinking, as a greedy algorithm, I'll take the lightest, not the heaviest.
And since I'm reversing it, I want to do the inverse.
And the density, which is just the value divided by the weight.
OK, make sense to everybody?
You with me?
Speak now, or not.
And then, we'll test it.
So again, kind of a theme of this part of the course.
As we write these more complex programs, we tend to have to worry about our test harnesses.
So I've got a function that tests the greedy algorithm, and then another function that tests all three greedy approaches-- the one algorithm with different functions-- and looks at what our results are.
So let's run it.
See what we get.
Oh, you know what I did?
Just the same thing I did last time.
But this time, I'm going to be smarter.
We're going to get rid of this figure and comment out the code that generated it.
And now, we'll test the greedy algorithms.
So we see the items we had to choose from, which I printed using the string function and items.
And if I use greedy by value to fill a knapsack of size 20, we see that I end up getting just the computer if I do greedy by value.
This is for the nerd burglar.
If I use weight, I get a different-- I get more things, not surprisingly -- but lower value.
And if I use density, I also get four things, but four different things, and I get a higher value.
So I see that I can run these greedy algorithms.
I can get an answer.
But it's not always the same answer.
As I said earlier, greedy by density happens to work best here.
But you shouldn't assume that will always be the case.
I'm sure you can all imagine a different assignment of weights and values that would make greedy by density give you a bad answer.
All right, before we talk about how good these answers are-- and we will come back to that as, in particular, suppose I want the best answer-- I want to stop for a minute and talk about the algorithmic efficiency of the greedy algorithm.
So let's go back and look at the code.
And this is why people use greedy algorithms.
Actually, there are two reasons.
One reason is that they're easy to program, and that's always a good thing.
And the other is that they are typically highly efficient.
So what's the efficiency of this?
How would we think about the efficiency of this greedy algorithm?
What are we looking at here?
Well, the first thing we have to ask is, what's the first thing it does?
It sorts the list, right?
So one thing that governs the efficiency might be the amount of time it takes to sort the list of items.
Well, we know how long that takes.
Or we can speculate at least.
And let's assume it does something like merge sort.
So what's that term going to be?
Order what?
Len of items times what?
Log n, right?
So maybe that's going to tell us the complexity, but maybe not.
The next thing we have to do is look at the while loop and see how many times are we going through the while loop.
What's the worst case?
Somebody?
I know I didn't bring any candy today, but you could answer the question anyway.
Be a sport.
Do it for free.
Yeah?
The length of the items
Well, we know this one is bigger.
So it looks like that's the complexity, right?
So we can say, all right, pretty good.
Slightly worse than linear in the length of the items, but not bad at all.
And that's a big attraction of greedy algorithms.
They are typically order length of items, or order length of items times the log of the length.
So greedy algorithms are usually very close to linear.
And that's why we really like them.
Why we don't like them is it may be that the accumulation of a sequence of locally optimal solutions does not yield a globally optimal solution.
So now, let's ask the question, suppose that's not good enough.
I have a very demanding thief.
Or maybe the thief works for a very demanding person and needs to choose the absolute optimal set.
Let's think first about how we formulate that carefully.
And then, what the complexity of solving it would be.
And then, algorithms that might be useful.
Again, the important step here, I think, is not the solution to the problem, but the process used to formulate the problem.
Often, it is the case that once one has done a careful formulation of a problem, it becomes obvious how to solve it, at least in a brute force way.
So now, let's look at a formalization of the 0/1 knapsack problem.
And it's a kind of formalization we'll use for a lot of problems.
So step one, we'll represent each item by a pair.
Because in fact, in deciding whether or not to take an item, we don't care what its name.
We don't care if it's a clock, or a radio, or whatever.
What matters is what's its value and what's its weight.
We'll write W as the maximum weight that the thief can carry, or that can fit in the knapsack.
So far, so good.
Nothing complicated there.
Now comes the interesting step.
We're going to represent the set of available items as a vector.
We'll call it I.
And then we'll have another vector, V, which indicates whether or not each item in I has been taken.
So V is a vector.
And if V_i is equal to 1, that implies I_i-- big I sub little i-- has been taken, is in the knapsack.
Conversely, if V_i is 0, it means I_i is not in the knapsack.
So having formulated the situation thusly, we can now go back to our notion of an optimization problem as an objective function and a set of constraints to carefully state the problem.
So for the objective function, we want to maximize the sum of V_i times I_i dot value, where i ranges over the length of the vectors.
So that's the trick of the 0/1.
If I don't take it, it's 0.
So it's 0 times the value.
If I take it, it's 1 times the value.
So this is going to give me the sum of the values of the items I've taken.
And then, I have this subject to the constraint.
And again, we'll do a summation.
And it will look very similar.
V_i times I_i, but this time, dot weight is less than or equal to W. Straightforward, but a useful kind of skill.
And people do spend a lot of time on doing that.
If you've ever used MATLAB, you know it wants everything to be a vector.
And that's often because a lot of these problems can be nicely formulated in this kind of way.
All right, now let's return to the question of complexity.
What happens if we implement this in the most straightforward way?
What would the most straightforward implementation look like?
Well, we could enumerate all possibilities and then choose the best that meets the constraint.
So this would be the obvious brute force solution to the optimization problem.
Look at all possible solutions.
Choose the best one.
I think you can see immediately that this is guaranteed to give you the optimal solution.
Actually, an optimal solution.
Maybe there's more than one best.
But in that case, you can just choose whichever one you like or whichever comes first, for example.
The question is, how long will this take to run?
Well, we can think about that by asking the question, how big will this set be?
How many possibilities are there?
Well, we can think about that in a pretty straightforward way because if we look at our formulation, we can ask ourselves, how many possible vectors are there?
How many vector V's could there be which shows which items were taken and which weren't?
And what's the answer to that?
Well, if we have n items, how long will V be?
Length n, right?
0, 1 for each.
If we have a vector of 0's and 1''s of length n, how many different values can that vector take on?
We asked this question before.
What's the answer?
Somebody shout it out.
I've got a vector of length n. Every value in the vector is either a 0 or a 1.
So maybe it looks something like this.
How many possible combinations of (0, 1)'s are there?
2 to the n?
2 to the n. Because essentially, this is a binary number, exactly.
And so if I had an n-bit binary number, I can represent 2 to the n different values.
And so we see that we have 2 to the n possible combinations to look at if we use a brute force solution.
How bad is this?
Well, if the number of items is small, it's not so bad.
And you'll see that, in fact, I can run this on the example we've looked.
2 to the 5 is not a huge number.
Suppose I have a different number.
Suppose I have 50 items to choose from.
Not a big problem.
I heard yesterday that the number of different airfares between two cities in the US is order of 500 -- 500 different airfares between, say, Boston and Chicago.
So looking at the best there might be 2 to the 500, kind of a bigger number.
Let's look at 2 to the 50.
Let's say there were 50 items to choose from in this question.
And let's say for the sake of argument, it takes a microsecond, one millionth of a second, to generate a solution.
How long will it take to solve this problem in a brute force way for 50 items?
Who thinks you can do it in under four seconds?
How about under four minutes?
Wow, skeptics.
Four hours?
That's a lot of computation.
Four hours, you're starting to get some people.
Four days?
All right.
Well, how about four years?
Still longer, just under four decades?
Looking at one choice every microsecond, it takes you roughly 36 years to evaluate all these possibilities.
Certainly for people of my age, that's not a practical solution to have to wait 36 years for an answer.
So we have to find something better.
And we'll be talking about that later.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last Tuesday, we ended up the lecture talking about knapsack problems.
We talked about the continuous knapsack problem and the fact that you could solve that optimally with a greedy algorithm.
And we looked at the 0-1 knapsack problem and discussed the fact that while we could write greedy algorithms that would solve the problem quickly, we have to be careful what we mean by "solve," and that while those algorithms would choose a set of items that we could indeed carry away, there was no guarantee that it would choose the optimal items, that is to say, one that would meet the objective function of maximizing the value.
We looked after that at a brute force algorithm on the board only for finding an optimal solution, a guaranteed optimal solution, but observed the fact that on even a moderately sized set of items, it might take a decade or so to run.
Decided that wasn't very good.
Nevertheless, I want to start today looking at some code that implements a brute force algorithm, not because I expect anyone to actually run this on a real example, but because a bit later in the term, we'll see how we could modify this to something that would be practical.
And there's some things to learn by looking at it.
So let's look at some code here.
I don't expect you to understand in real time all the details of this code.
It's more I want you to understand the basic idea behind it and then the result we get.
So you'll remember that we looked at the complexity by saying well, really, it's like binary numbers.
So the first helper subroutine I'm going to use is something that generates binary numbers.
So it takes an n, some natural number, and the number of digits and returns a binary string of that length, representing the decimal number n. Why am I giving it this number of digits?
Because I need to zero pad it.
If I want to have a vector that represents whether or not I take items, if I take only one item, say the first one, I don't want just a binary spring with one digit in it.
Because I need all those zeros to indicate that I'm not taking the other items.
And so the second argument tells me, in effect, how many zeros I'm going to need.
And there's nothing mysterious about the way it does it.
OK.
The next helper function generates the power set of the items.
What is a power set?
If you take a set, you can then ask the question what are all the subsets of the set?
What's the smallest subset of a set?
It's the empty set.
No items.
What's the largest subset of a set?
All of the items.
And then we have everything in between, the set that contains the first item, the set that contains the second item, et cetera, the set that contains the first and the second, the first and the third.
There are a lot of them.
And of course, how many is a lot?
Well, 2 to the n is a lot.
But now we're going to generate every possible subset of items.
And we're going to do this simply using the decimal to binary function to tell us whether or not we keep each one, so we can enumerate them.
We can generate them all.
And now we have the set of all possible items one might take, irrespective of whether they obey the constraint of not weighing too much.
The next function is the one that does the work.
This is the interesting one.
Choose best.
It takes a power set, the constraint, and two functions.
One, getValue-- it tells me the value of an item.
And the other getWeight-- it tells me the weight of an item.
Then it just goes through.
And it enumerates all possibilities and eventually chooses-- I won't say the best set, because it might not be unique.
There might be more than one optimal answer.
But it finds at least one optimal answer.
And then it returns that.
Again it's a very straightforward implementation of the brute force algorithm I sketched on the board.
And then we can run it with testBest, which is going to build the items using the function we looked at last time.
It's then going to get the power set of the items.
It's going to call chooseBest and then print the result.
So let's see what happens if we run it.
We get an error.
Oh dear.
I hadn't expected that.
And it says test-- oh testBest is not defined.
All right.
Let's try that again.
Sure looks like it's defined to me.
There it is.
OK. And you may recall that this is a better answer than anything that was generated by the greedy algorithm on Tuesday.
You may not recall it, but believe me it is.
It happened to have found a better solution.
And not surprisingly, that's because I contrived the example to make sure that would happen.
Why does it work better in the sense-- or why does it find a better answer?
Why might it find a better answer?
Well, because the greedy algorithm chose something that was locally optimal at each step.
But there was no guarantee that a sequence of locally optimal decisions would reach a global optimum.
What this algorithm does is it finds a global optimum by looking at all solutions.
And that's something we'll see again and again, as we go forward, that there's always a temptation to do things one step at a time, finding local optimum-- optima-- because it's fast.
It's easy.
But there's no guarantee it will work well.
Now the problem, of course, with finding the global optimum is, as we discussed, it is prohibitively expensive.
Now you could ask is it prohibitively expensive because I chose a stupid algorithm, the brute force algorithm?
Well, it is a stupid algorithm.
But in fact, this is a problem that is what we would call inherently exponential.
We've looked at this concept before.
That in addition to talking about the complexity of an algorithm, we can talk about the complexity of a problem in which we ask the question how fast can the absolute best solution, fastest solution to this problem, be?
And here you can construct a mathematical proof that says the problem is inherently exponential.
No matter what we do, we're not going to be able to find something that's guaranteed to find the optimal, that is faster than exponential.
Well, now let's be careful of that statement.
What that means is the worst case is inherently exponential.
As we will see in a couple of weeks-- it'll take us a while to get there-- there are actually algorithms that people use to solve these inherently exponential problems and solve them fast enough to be useful.
So, for example, when you go to look at airline fares on Kayak to try and find the best fare from A to B, it is an inherently exponential problem, but you get an answer pretty quickly.
And that's because there are techniques you can use.
Now, in fact, one of the reasons you get it is they don't guarantee that you actually get an optimal solution.
But there are techniques that guarantee to give you an optimal solution that almost all the time will run quickly.
And we'll look at one of those a bit later in the term.
Before we do that, however, I want to leave for a while the whole question of complexity behind and look at another class of optimization problems.
We'll look at several different kinds of optimization problems as the term goes forward.
The kind I want to look at today is probably what I would say is the most exciting branch of computer science today.
And of course I might have a bias.
And that's machine learning.
It's a word you'll hear a lot about.
And it's a technique that many of you will apply.
You might not write your own codes.
But I guarantee you were either be the beneficiary or the victim of machine learning almost every time you log on to the web these days.
I should probably start by defining what machine learning is.
But that's hard to do.
I really don't know how to do it.
Superficially, you could say that machine learning deals with the question of how to build programs that learn.
However, I think in a very real sense every program we write learns something.
If I implement Newton's method, it's learning what the roots of the polynomial is.
Certainly when we looked at curve fitting-- fitting curves to data-- we were learning a model of the data.
That's what that regression is.
Wikipedia says-- and of course, it must be true if Wikipedia says it-- that machine learning is a scientific discipline that is concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data.
I'm not sure how helpful this definition is.
But it was the best I could find.
And it doesn't really matter.
But it sort of gets at the issue that a major focus of machine learning research is to automatically learn to recognize complex patterns and make intelligent decisions based on data.
This whole process is something called inductive inference.
The basic idea is one observes-- actually one doesn't.
The program observes examples that represent an incomplete information about some statistical phenomena and then tries to generate a model, just like with curve fitting, that summarizes some statistical properties of that data and can be used to predict the future, for example, give you information about unseen data.
There are roughly speaking two distinctive approaches to machine learning called supervised learning and unsupervised learning.
Let's first talk about supervised learning.
It's a little easier to appreciate how it might work.
In supervised learning, we associate a label with each example in a training set.
So think of that as an answer to a query about an example.
If the label is discrete, we typically call it a classification problem.
So we would try and classify, for example, a transaction on a credit card as belonging to the owner of that credit card or not belonging to the owner, as i.e., with some probability, a stolen credit card.
So it's discrete.
It belongs to the owner.
It doesn't belong to the owner.
If the labels are real valued, we think of it as a regression problem.
And so indeed, when we did the curve fitting, we were doing machine learning.
And we were handling a regression problem.
Based on the examples from the training set, the goal is to build a program that can predict the answer for other cases before they were explicitly observed.
So we're trying to generalize from the statistical properties of a training set to be able to make predictions about things we haven't seen.
Let's look at an example.
So here, I've got red and blue circles.
And I'm trying to learn what makes a circle red or what's the difference between red and blue, other than the color?
Think of my information as the (x,y) values and the label as the color, red or blue.
So I've labeled each one.
And now I'm trying to learn something.
Well, it's kind of tricky.
What are the questions I need to answer to think about this?
And then we'll look at how we might do it.
So, a first question I need to ask is are the labels accurate?
And in fact, in a lot of real world examples, in most real world examples, there's no guarantee that the labels are accurate.
So you have to assume that well, maybe some of the labels are wrong.
How do we deal with that?
Perhaps the most fundamental question is the past representative of the future?
We've seen many examples where people have learned things, for example, to predict the price of housing.
And it turns out you hit some singularity which means the past is not a very good predictor of the future.
And even if all of your learning is good, you get the wrong answer.
So you sort of always have to ask that question.
Do you have enough data to generalize?
And by this, I mean enough training data.
If your training set is very small, you shouldn't have a lot of confidence in what you learn.
A big issue here is feature extraction.
As we'll see when we look at real examples, the world is a pretty complex place.
And we need to decide what features we're going to use.
If I were to ask 25% of you to come up in the front of the room and then try and separate you based upon some feature-- if I were to say, all right, I'm going to separate the good students from the bad students, but the only features I have available are the clothes you're wearing, it might not work so well.
And very importantly, how tight should the fit be?
So now let's go back to our example here.
We can look at two different ways we might generalize from this data.
And indeed, when we're looking at classification problems in supervised learning, what we're typically doing is trying to find some way of dividing our training data.
In this case, I've given you a two-dimensional projection.
As we'll see, it's not always two-dimensional.
It's not usually two-dimensional.
So I might choose this rather eccentric shape and say that's great.
And why is that great?
It's great because it minimizes training error.
So if we look at it as an optimization problem, we might say that our objective function is how many points are correctly classified in the training data as red or blue.
And this triangular shape has no training error.
Every point is perfectly classified in the training data.
If I choose this linear separator instead, I have some training error.
This red point is misclassified in the training.
Does that mean that the triangle is better than the line?
Not necessarily, right?
Because my goal is to predict future points.
And maybe that's mislabeled or an experimental error.
Maybe it's accurately labeled but an outlier, very unusual.
And this will not generalize well.
This is analogous to what we talked about as overfitting when we looked at curve fitting.
And that's-- a very big problem in machine learning is if you overfit to your training data, it might not generalize well and might give you bogus answers going forward.
OK.
So that's a very quick look at supervised learning.
We'll come back to that.
I now want to talk about unsupervised learning.
The big difference here is we have training data, but we don't have labels.
So I just give you a bunch of points.
It's as if we looked at this picture, and I didn't tell you which were the red points and which were the blue points.
They were just all points.
So what can I learn?
What typically you're learning in unsupervised learning, is you're learning about regularities of the data.
So if we looked at this and think away the red and the blue, we might well say well, at least, if I look at this, there is some structure to this data.
And maybe what I should do is divide it this way.
It gives me kind of a nice clean separation.
But maybe I should divide it this way.
Or maybe I should put a circle around each of these four groupings.
Complicated, what to do.
But what we see is there is clearly some structure here.
And the idea of unsupervised learning is to discover that structure.
Far and away, the dominant form of unsupervised learning is clustering.
And that's what I was just talking about, is finding the cluster in this data.
So we'll move forward here.
There it is, with everything the same color.
But here I've labeled the x- and y-axes as height and weight.
What does clustering mean?
It's the process of organizing the objects or the points into groups whose members are similar in some way.
A key issue is what do we mean by similar?
What's the metric we want to use?
And we can see that here.
If I tell you that, really, I want to cluster people by height-- say, people are similar if they're the same height-- then it's pretty clear how I should divide this, right, what my clusters should be.
My clusters should probably be this group of shorter people and this group of taller people.
If I tell you I'm interested in weight, then probably I want a cluster it with the divisor here between the heavier people and the lighter people.
Or if I say well, I'm interested in some combination of those two, then maybe I'll get four clusters as I discussed before.
Clustering algorithms are used all over the place.
For example, in marketing, they're used to find groups of customers with similar behavior.
Walmart is famous for using that clustering to find that.
They did a clustering to determine when people bought the same thing.
And then they would rearrange their shelves to encourage people to buy things.
And sort of the most famous example they discovered was there was a strong correlation between people between people who bought beer and people who bought diapers.
And so there was a period where if you walked in a Walmart store, you would find the beer and the diapers next to each other.
And I leave it to you to speculate on why that was true.
It just happened to be true in Walmart.
Amazon uses clustering to find people who like similar books.
So every time you buy a book on Amazon, they're running a clustering algorithm to find out who looks like you.
Said, oh, this person looks just like you.
So if they buy a book, maybe you'll get an email suggesting you buy that book or the next time you log into Amazon.
Or when you look at a book, they tell you here are some similar books.
And then they've done a clustering to group books as similar based on buying habits.
Netflix uses that to recommend movies, et cetera.
Biologists spend a lot of time these days doing clustering.
They classify plants or animals based on their features.
We'll shortly see an example of that, as in right after Patriot's Day.
But they also use it a lot in genetics.
So clustering is used to try and find genes that look like or groups of genes.
Insurance companies use that to decide how much to charge you for your automobile insurance.
They cluster drivers based upon-- and use that to predict who's going to have an accident.
Document classification on the web is used all the time.
It's used a lot in medicine.
Just used all over the place.
So what is it exactly?
Well the nice thing is we can define it very straightforwardly as an optimization problem.
And so we can ask what properties does a good clustering have?
Well, it should have low intra-cluster dissimilarity.
So in a good clustering, all of the points in the same cluster should be similar, by whatever metric you're using for similarity.
As we'll see, there are a lot of choices there.
But that's not enough.
We'd also like to have high inter-cluster dissimilarity.
So we'd like the points within a cluster to be a lot like each other.
But if points are in different clusters, we'd like them to be quite different from each other.
That tells us that we have a good cluster.
All right, let's look at it.
How might we model dissimilarity?
Well, using a concept we've already seen-- variance.
So we can talk about the variance of some cluster C as equal to the sum of all elements x in C, of the mean of C minus x squared.
Or maybe we can take the square root of it, if we want.
But it's exactly the idea we've seen before, right?
Then we say what's the average value of the cluster?
And then we look at how far is each point from the average.
We sum them.
And that tells us how much variance we have within the cluster.
Make sense?
So that's variance.
So we can use that to talk about how similar or dissimilar the elements in the cluster are.
We can use the same idea to compare points in separate clusters and compute various different ways-- and we'll look at different ways-- to look at the distance between clusters.
So combining these two things, we could get, say, a metric we'll call badness-- not a technical word.
And now I'll ask the question is the optimization problem that we're solving in clustering finding a set of clusters-- capital C-- such that badness of that set of clusters is minimized?
Is that a sufficient definition of the problem we're trying to solve?
Find a set of clusters C, such that the badness of C is minimized.
is that good enough?
No
No, why not?
Just imagine a case where you view cluster-- if you make a single cluster, every cluster has one element in it.
And the variance is 0
Exactly.
So that has a trivial solution, which is probably not the one we want, of putting each point in its own cluster.
Badness-- it won't be bad.
It'll be a perfect clustering in some sense.
But it doesn't do us any good really.
So what do we do to fix that?
What do we usually do when we formulate an optimization problem?
What's missing?
I've given you the objective function.
What have I not giving you?
Constraints
A constraint.
So we need to add some constraint here that will prevent us from finding a trivial solution.
So what kind of constraints might we look at?
There are different ways of doing it.
A couple of ones that is usual.
Sometimes you might have as a constraint, the maximum number of clusters.
Say, all right, cluster my data, but I want at most K clusters -- 10 clusters.
That would be my constraint, like the weight for the knapsack problem.
Or maybe I'll want to put something on the maximum distance between clusters.
So I don't want the distance between any two clusters to be more than something.
In general, solving this optimization problem is computationally prohibitive.
So once again, in practice, what people typically resort to is greedy algorithms.
And I want to look at two kinds of greedy algorithms, probably the two most common approaches to clustering.
One is called k-means.
In k-means clustering, you say I want exactly k clusters.
And find the best k clustering.
We'll talk about how it does that.
And again, it's not guaranteed to find the best.
And the other is hierarchical clustering.
We'll come back to that shortly.
Both are simple to understand and widely used in practice.
So let's first talk about how we do this.
Let's first look at hierarchical clustering.
So we have a set of n items to be clustered.
And let's assume we have an n by n distance matrix that tells me for each pair of items how far they are from each other.
So we can look at an example.
So here's an n by n distance matrix for the airline distance between some cities in the United States.
The distance from Boston to Boston is 0 miles.
Distance from New York is 206.
The distance from Chicago to San Francisco is 2,142, et cetera.
All right?
So I have my n by n distance matrix there.
Now let's go through how hierarchical clustering would relate these things to each other.
So we start by assigning each item to its own cluster.
So if we have n items, we now have n clusters.
All right?
That's the trivial solution that you suggested before.
The next step is to find the most similar pair of clusters and merge them.
So if we look here and we just-- we start, we'll have six clusters, one for each city.
And we would merge the two most similar, which I guess in this case is New York and Boston.
Hard to believe that those are the most similar cities.
But at least by this distance metric they're the closest.
So we would merge those two.
And then you just continue the process in principle, until all items are in one cluster.
So now you have a whole hierarchy of clusters.
And you can cut it off where you want.
If you want to have six clusters, you could look at where the hierarchy you have six.
You can look where you have two, where you have three.
Of course, you don't have to go all the way to finish it if you don't want to.
This kind of hierarchical clustering is called agglomerative.
Why?
Well, because we're combining things.
We're agglomerating them.
So this is pretty straightforward, except for two.
The complication in step (2) is we have to define what it means to find the two most similar clusters.
Now it's pretty easy when the clusters each contain one element, because, well, we have our metric-- in this case, distance-- and we can just do that as I did.
But it's not so obvious what you do when they have multiple elements.
And in fact, different metrics can be used to get different properties.
So I want to talk about some of the metrics we use for that.
These are typically called linkage criteria.
So one popular one is what's called single linkage.
It's also called connectedness, or minimum method.
In this, we consider the distance between a pair of clusters to be equal to the shortest distance from any member to any other member.
So we take the two points in each cluster that are closest to each other and say that's the distance between the two clusters.
People also use something called complete linkage-- It's also called diameter or maximum-- where we consider the distance between any two clusters to be the distance between the points that are furthest from each other.
So in one case, essentially single linkages was looking at the best case.
Complete-- in English, not French-- is looking at the worst case.
And you won't be surprised to hear that you could also look at the average case, where you take all of the distances.
So you take all of the pairwise things.
You add them up.
You take the average.
You can also take the mean, the median, if you want instead.
None of these is necessarily best.
But they do give you different answers.
And so I want to look at that now with our example here.
So let's look at it and run it.
So the first step is independent of what linkage we're using.
We get these six clusters.
All right, now let's look at the second step.
Well, also pretty simple since we only have one element in each one.
We're going to get that clustering.
All right.
Now, what about the next step?
What do I get if I'm using the minimal single linkage distance?
What gets merged here?
Somebody?
Boston, New York and Chicago
Boston, New York, and Chicago.
And it turns out we'll get the same thing if we use other linkages in this case.
Let's continue to the next step.
Now we'll end up merging San Francisco and Seattle.
Now we get a difference.
What does the red represent and what does the blue represent?
Which linkage criteria?
We're saying, we could either merge Denver with Boston, New York, and Chicago.
Or we could merge Denver with San Francisco and Seattle.
Which is which?
Which linkage criterion has put Denver in which cluster?
Well, suppose we're using single linkage.
Where are we getting it from?
Boston, New York and Chicago?
Yes.
Because it's not so far from Chicago.
Even though it's pretty far from Boston or New York.
But if we use average linkage, we see on average, it's closer to San Francisco and Seattle than it is to the average of Boston, New York, or Chicago.
So we get a different answer.
And then finally, at the last step, everything gets merged together.
So you can see, in this case, without having labels, we have used a feature to produce things and, say, if we wanted to have three clusters, we would maybe stop here.
And we'd say, all right, these things are one cluster.
This is a cluster.
And this is a cluster.
And that's not a bad geographical clustering, actually, for deciding how to relate these things to each other.
This technique is used a lot.
It does have some weaknesses.
One weakness is it's very time consuming.
It doesn't scale well.
The complexity is at least order n-squared, where n is the number of points to be clustered.
And in fact, in many implementations, it's worse than n-squared.
And of course, it doesn't necessarily find the optimal clustering, even giving these criteria.
It might never at any level have the optimal clustering, because, again, at each step, it's making a locally optimal decision, not guaranteed to find the best solution.
I should point out that a big issue in deciding to get these clusters or getting these clusters was my choice of features.
And this is something we're going to come back to in spades, because I actually think it is the most important issue in machine learning-- is if we're going to say which points are similar to each other, we need to understand our feature space.
So, for example, the feature I'm using here is distance by air.
Suppose, instead, I added distance by air and distance by road and distance by train.
Well, particularly given this sparsity of railroads in this country, we might get very different clustering, depending upon where the trains ran.
And suppose I throw in a totally different feature like population.
Well, I might get another different clustering, depending on how I use that.
What we typically need to do in these situations, dealing with multi-dimensional data-- and most data is multi-dimensional-- is we construct something called a feature vector that incorporates multiple features.
So we might have for each city-- we'll just take something like the distance.
Or let's say, instead of distance, we'll compute the distance by having for each city its GPS coordinates, where it is on the globe, and its population.
And let's say that's how we define a city.
And that would be our feature vector.
And then we would cluster it, say, using hierarchical clustering to determine which cities are most like which other cities.
Well, it's a little bit complicated.
I have to ask how do I compare feature vectors?
What distance metric do I use there?
Do I get confused that GPS coordinates and populations are essentially unrelated?
And I wouldn't like to compare those to each other.
Lots of issues there, and that's what we're going to talk about when we come back from Patriot's Day-- is how in the real world problems, we go from the large number of features associated with objects or things in the real world to feature vectors that allow us to automatically deduce which things are quote "most similar" to which other things.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to pick up with a little bit of overlap just to remind people where we were.
We had been looking at clustering, and we looked at a fairly simple example of using agglomerative hierarchical clustering to cluster cities, based upon how far apart they were from each other cities.
So, essentially, using this distance matrix, we could do a clustering that would reflect how close cities were to one another.
And we went through a agglomerative clustering, and we saw that we would get a different answer, depending upon which linkage criterion we used.
This is an important issue because as one is using clustering, one what has to be aware that it is related to these things, and you choose the wrong linkage criterion, you might get an answer other than the most useful.
All right.
I next went on and said, well, this is pretty easy, because when we're comparing the distance between two cities or the two features, we just subtract one distance from the other and we get a number.
It's very straightforward.
I then raised the question, suppose when we looked at cities, we looked at a more complicated way of looking at them than airline distance.
So the first question, I said, well, suppose in addition to the distance by air, we add the distance by road, or the average temperature.
Pick what you will.
What do we do?
Well, the answer was we start by generalizing from a feature being a single number to the notion of a feature vector, where the features used to describe the city are now represented by a vector, typically of numbers.
If the vectors are all in the same physical units, we could easily imagine how we might compare two vectors.
So we might, for example, we look at the Euclidean distance between the two just by, say, subtracting one vector from the other.
However, if we think about that, it can be pretty misleading because, for example, when we look at a city, one element of the vector represents the distance in miles from another city, or in fact this case, the distance in miles to each city.
And another represents temperatures.
Well, it's kind of funny to compare distance, which might be thousands of miles, with the temperature which might be 5 degrees.
A 5 degree difference in average temperature could be significant.
Certainly a 20 degree difference in temperature is very significant, but a 20 mile difference in location might not be very significant.
And so to equally weight a 20 degree temperature difference and at 20 miles distance difference might give us a very peculiar answer.
And so we have to think about the question of, how are we going to scale the elements of the vectors?
Even if we're in the same units, say inches, it can be an issue.
So let's look at this example.
Here I've got on the left, before scaling, something which we can say is in inches, height and width.
This is not from a person, but you could imagine if you were trying to cluster people, and you measured their height in inches and their width in inches, maybe you don't want to treat them equally.
Right?
But there's a lot more variance in height than in width, or maybe there is and maybe there isn't.
So here on the left we don't have any scaling, and we see a very natural clustering.
On the other hand, we notice on the y-axis the values range from not too far from 0 to not too far from 1.
Whereas on the x-axis, the dynamic range is much less, not too far from 0 to not too far from 1/2.
So we have twice the dynamic range here than we have here.
Therefore, not surprisingly, when we end up doing the clustering, width plays a very important role.
And we end up clustering it this way, dividing it along here.
On the other hand, if I take exactly the same data and scale it, and now the x-axis runs from 0 to 1/2 and the y-axis, roughly again, from 0 to 1, we see that suddenly when we look at it geometrically, we end up getting a very different look of clustering.
What's the moral?
The moral is you have to think hard about how to cluster your features, about how to scale your features, because it can have a dramatic influence on your answer.
We'll see some real life examples of this shortly.
But these are all the important things to think about, and they all, in some sense, tie up into the same major point.
Whenever you're doing any kind of learning, including clustering, feature, selection, and scaling is critical.
It is where most of the thinking ends up going.
And then the rest gets to be fairly mechanical.
How do we decide what features to use and how to scale them?
We do that using domain knowledge.
So we actually have to think about the objects that we're trying to learn about and what the objective of the learning process is.
So continuing, how do we do the scaling?
Most of the time, it's done using some variant of what's called the Minkowski metric.
It's not nearly as imposing as it looks.
So the distance between two vectors, X1 and X2, and then we use p to talk about, essentially, the degree we're going to be using.
So we take the absolute difference between each element of X1 and X2, raise it to the p-th power, sum them and then take the 1 over p. Not very complicated, so let's say p is 2.
That's the one you people are most familiar with.
Effectively, all we're doing is getting the Euclidean distance.
What we looked at when we looked at the mean squared distance between two things, between our errors and our measured data, between our measured data and our predicted data.
We used the mean square error.
That's essentially in Minkowski distance with p equal to 2.
That's probably the most commonly used, but an almost equally commonly used sets p equal to 1, and that's something called the Manhattan distance.
I suspect at least some of you have spent time walking around Manhattan, a small but densely populated island in New York.
And midtown Manhattan has the feature that it's laid out in a grid.
So what you have is a grid, and you have the avenues running north-south and the streets running east-west.
And if you want to walk from, say, here to here or drive from here to here, you cannot take the diagonal because there are a bunch of buildings in the way.
And so you have to move either left or right, or up, or down.
That's the Manhattan distance between two points.
This is used, in fact, for a lot of problems, typically when somebody is comparing the distance between two genes, for example, they use a Manhattan metric rather than a Euclidean metric to say how similar two things are.
Just wanted to show that because it is something that you will run across in the literature when you read about these kinds of things.
All right.
So far, we've talked about issues where things are comparable.
And we've been doing that by representing each element of the feature vector as a floating point number.
So we can run a formula like that by subtracting one from the other.
But we often, in fact, have to deal with nominal categories, things that have names rather than numbers.
So for clustering people, maybe we care about eye color, blue, brown, gray, green.
Hair color.
Well, how do you compare blue to green?
Do you subtract one from the other?
Kind of hard to do.
What does it mean to subtract green from blue?
Well, I guess we could talk about it in the frequency domain, enlighten things.
Typically, what we have to do in that case is, we convert them to a number and then have some ways to relate the numbers.
Again, this is a place where domain knowledge is critical.
So, for example, we might convert blue to 0, green to 0.5, and brown to 1, thus indicating that we think blue eyes are closer to green eyes than they are to brown eyes.
I don't know why we think that but maybe we think that.
Red hair is closer to blonde hair than it is to black hair.
I don't know.
These are the sorts of things that are not mathematical questions, typically, but judgments that people have to make.
Once we've converted things to numbers, we then have to go back to our old friend of scaling, which is often called normalization.
Very often we try and contrive to have every feature range between 0 and 1, for example, so that everything is normalized to the same dynamic range, and then we can compare.
Is that the right thing to do?
Not necessarily, because you might consider some features more important than others and want to give them a greater weight.
And, again, that's something we'll come back to and look at.
All this is a bit abstract.
I now want to look at an example.
Let's look at the example of clustering mammals.
There are, essentially, an unbounded number of features you could use, size at birth, gestation period, lifespan, length of tail, speed, eating habits.
You name it.
The choice of features and weighting will, of course, have an enormous impact on what clusters you get.
If you choose size, humans might appear in one cluster.
If you choose eating habits, they might appear in another.
How should you choose which features you want?
You have to begin by choosing, thinking about the reason you're doing the clustering in the first place.
What is it you're trying to learn about the mammals?
As an example, I'm going to choose the objective of eating habits.
I want to cluster mammals somehow based upon what they eat.
But I want to do that, and here's a very important thing about what we often see in learning without any direct information about what they eat.
Typically, when we're using machine learning, we're trying to learn about something for which we have limited or no data.
Remember when we talked about learning, I talked about learning in which it was supervised, and which we had some data, and unsupervised, in which, essentially, we don't have any labels.
So let's say we don't have any labels about what mammals eat, but we do know a lot about the mammals themselves.
And, in fact, the hypothesis I'm going to start with here is that you can infer people's or creatures' eating habits from their dental records, or their dentitian.
But over time, we have evolved, all creatures have evolved, to have teeth that are related to what they eat, we can see.
So I managed to procure a database of dentitian for various mammals.
There's the laser pointer.
So what I've got here is the number of different kinds of teeth.
So the right top incisors, the right bottom incisors, molars, et cetera, pre-molars.
Don't worry if you don't know about teeth very much.
I don't know very much.
And then for each animal, I have the number of each kind of tooth.
Actually, I don't have it for this particular mammal, but these two I do.
I don't even remember what they are.
They're cute.
All right.
So I've got that database, and now I want to try and see what happens when I cluster them.
The code to do this is not very complicated, but I should make a confession about it.
Last night, I won't say I learned it.
I was reminded of a lesson that I've often preached in 6.00, is that it's not good to get your programming done at the last minute.
So as I was debugging this code at 2:00 and 3:00 in the morning today, I was realizing how inefficient I am at debugging at that hour.
Maybe for you guys that's the shank of the day.
For me, it's too late.
I think it all works, but I was certainly not at my best as I was debugging last night.
All right.
But at the moment, I don't want you to spend time working on the code itself.
I would like you to think a little bit about the overall class structure of the code, which I've got on the first page of the handout.
So at the bottom of my hierarchy, I've got something called a point, and that's an abstraction of the things to be clustered.
And I've done it in quite a generalized way, because, as you're going to see, the code we're looking at today, I'm going to use not only for clustering mammals but for clustering all sorts of other things as well.
So I decided to take the trouble of building up a set of classes that would be useful.
And in this class, I can have the name of a point, its original attributes.
That say its original feature vector, an unscaled feature vector, and then whether or not I choose to normalize it.
I might have normalized features as well.
Again, I don't want you worry too much about the details of the code.
And then I have a distance metric, and I'm just for the moment using simple Euclidean distance.
The next element in my hierarchy, not yet a hierarchy-- it's still flat-- is a cluster.
And so what a cluster is, you can think of it as, at some abstract level, it's just going to be a set of points, the points that are in the cluster.
But I've got some other operations on it that will be useful.
I can compute the distance between two clusters, and as you'll see, I have single linkage, Mac Link, max , average, the three I talked about last week.
And also this notion of a centroid.
We'll come back to that when we get to k-means clustering.
We don't need to worry right now about what that is.
Then I'm going to have a cluster set.
That's another useful data abstraction.
And that's what you might guess from its name, just a set of clusters.
The most interesting operation there is merge.
As you saw, when we looked at hierarchical clustering last week, the key step there is merging two clusters.
And in doing that, I'm going to have a function called Find Closest, which given a metric and a cluster, finds the cluster that is most similar to that, to self, because as you, again, will recall from hierarchical clustering, that's what I merged at each step is the two most similar clusters.
And then there's some details about how it works, which again, we don't need to worry about at the moment.
And then I'm going to have a subclass of point called Mammal, in which I will represent each mammal by the dentitian as we've looked at before.
Then pretty simply, we can do a bunch of things with it.
Before we look at the other details of the code, I want to now run it and see what we get.
So I'm just going to use hierarchical clustering now to cluster the mammals based upon this feature vector, which will be a list of numbers showing how many of each kind of tooth the mammals have.
Let's see what we get.
So it's doing the merging.
So we can see the first step, it merged beavers with ground hogs and it merged grey squirrels with porcupines, wolves and bears.
Various other kinds of things, like jaguars and cougars, were a lot alike.
Eventually, it starts doing more complicated merges.
It merges a cluster containing only the river otter with one containing a Martin and a wolverine, beavers and ground hogs with squirrels and porcupines, et cetera.
And at the end, I had it stop with two clusters.
It came up with these clusters.
Now we can look at these clusters and say, all right.
What do we think?
Have we learned anything interesting?
Do we see anything in any of these-- do we think it makes sense?
Remember, our goal was to cluster mammals based upon what they might eat.
And we can ask, do we think this corresponds to that?
No.
All right.
Who-- somebody said-- Now, why no?
Go ahead
We've got-- like a deer doesn't eat similar things as a dog.
And we've got one type on the top cluster and a different kind of bat in the bottom cluster.
Seems like they would be even closer together
Well, sorry.
Yeah.
A deer doesn't eat what a dog eats, and for that matter, we have humans here, and while some human are by choice vegetarians, genetically, humans are essentially carnivores.
We know that.
We eat meat.
And here we are with a bunch of herbivores, typically.
Things are strange.
By the way, bats might end up being in ones, because some bats eat fruit, other bat eat insects, but who knows?
So I'm not very happy.
Why do you think we got this clustering that maybe isn't helping us very much?
Well, let's go look at what we did here.
Let's look at test 0.
So I said I wanted two clusters.
I don't want it to print all the steps along the way.
I'm going to print the history at the end.
And scaling is identity.
Well, let's go back and look at some of the data here.
What we can see is-- or maybe we can't see too quickly, looking at all this-- is some kinds of teeth have a relatively small range.
Other kinds of teeth have a big range.
And so, at the moment, we're not doing any normalization, and maybe what we're doing is getting something distorted where we're only looking at a certain kind of tooth because it has a larger dynamic range.
And in fact, if we look at the code, we can go back up and let's look at Build Mammal Points and Read Mammal Data.
So Build Mammal Points calls Read Mammal Data, and then builds the points.
So Read Mammal Data is the interesting piece.
And what we can see here is, as we read it in-- this is just simply reading things in, ignoring comments, keeping track of things-- and then we come down here, I might do some scaling.
So Point.Scale feature is using the scaling argument.
Where's that coming from?
If we look at Mammal Teeth, here from the mammal class, we see that there are two ways to scale it, identity, where we just multiply every element in the vector by 1.
That doesn't change anything.
Or what I've called 1 over max.
And here, I've looked at the maximum number of each kind of tooth and I'm dividing 1 by that.
So here we could have up to three of those.
Here we could have four of those.
We could have six of this kind of tooth, whatever it is.
And so we can see, by dividing by the max, I'm now putting all of the different kinds of teeth on the same scale.
I'm normalizing.
And now we'll see, does that make a difference?
Well, since we're dividing by 6 here and 3 here, it certainly could make a difference.
It's a significant scaling factor, 2X.
So let's go and change the code, or change the test.
And let's look at Test 0-- 0, not "O"-- with scale set to 1 over max.
You'll notice, by the way, that rather than using some obscure code, like scale equals 12, I use strings so I remember what they are.
It's, I think, a pretty useful programming trick.
Whoops.
Did I use the wrong name for this?
Should be scaling?
So off it's going.
Now we get a different set of things, and as far as I know, once we've scaled things, we get what I think is a much more sensible pair, where I think what we essentially have is the herbivores down here, and the carnivores up here.
Ok.
I don't care how much you know about teeth.
The point is scaling can really matter.
You have to look at it, and you have to think about what you're doing.
And the interesting thing here is that without any direct evidence about what mammals eat, we are able to use machine learning, clustering in this case, to infer a new fact that we have some mammals that are similar in what they eat, and some mammals that are also similar, some groups.
Now I can't infer from this herbivores versus carnivores because I didn't have any labels to start with.
But what I can infer is that, whatever they eat, there's something similar about these animals, and something similar about these animals.
And there's a difference between the groups in C1 and the groups in C0.
I can then go off and look at some points in each of these and then try and figure out how to label them later.
OK, let's look at a difference data set, a far more interesting one, a richer one.
Now, let's not look at that version of it.
That's too hard to read.
Let's look at the Excel spreadsheet.
So this is a database I found online of every county in the United States, and a bunch of features about that county.
So for each county in the United States, we have its name.
The first part of the name is the state it's in.
The second part of the name is the name of the county, and a bunch of things, like the average value of homes, how much poverty, its population density, its population change, how many people are over 65, et cetera.
So the thing I want you to notice, of course, is while everything is a number, the scales are very different.
It's a big difference between the percent of something, which will go between 0 and 100, and the population density, which ranges over a much larger dynamic range.
So we can immediately suspect that scaling is going to be an issue here.
So we now have a bunch of code that we can use that I've written to process this.
It uses the same clusters that we have here, except I've added a kind of Point called the County.
Looks very different from a mammal, but the good news is I got to reuse a lot of my code.
Now let's run a test.
We'll go down here to Test 3, and we'll see whether we can do hierarchical clustering of the counties.
Whoops.
Test 3 wants the name of what we're doing.
So we'll give it the name.
It's Counties.Text.
I just exported the spreadsheet as a text file.
Well, we can wait a while for this, but I'm not going to.
Let's think about what we know that hierarchical clustering and how long this is likely to take.
I'll give you a hint.
There are approximately 3,100 counties in the United States.
I'll bet none of you could have guessed that number.
How many comparisons do we have to find the two counties that are most similar to each other?
Comparing each county with every other county, how many comparisons is that going to be?
Yeah
It's 3,100 choose 2
Right.
So that will be 3,100 squared.
Thank you.
And that's just the first step in the cluster.
To perform the next merge, we'll have to do it again.
So in fact, as we've looked at last time, it's going to be a very long and tedious process, and one I'm not going to wait for.
So I'm going to interrupt and we're going to look at a smaller example.
Here I've just got only the counties in New England, a much smaller number than 3,100, and I'm going to cluster them using the exact same clustering code we used for the mammals.
It's just that the points are now counties instead of mammals.
And we got two clusters.
Middlesex County in Massachusetts happens to be the county in which MIT is located.
And all the others-- well, you know, MIT is a pretty distinctive place.
Maybe that's what did it.
I don't quite think so.
Someone got a hypothesis about why we got this rather strange clustering?
And is it because Middlesex contains MIT and Harvard both?
This really surprised me, by the way, when I first ran it.
I said, how can this be?
So I went and I started looking at the data, and what I found is that Middlesex County has about 600,000 more people than any other county in New England.
Who knew?
I would have guessed Suffolk, where Boston is, was the biggest county.
But, in fact, Middlesex is enormous relative to every other county in New England.
And it turns out that difference of 600,000, when I didn't scale things, just swamped everything else.
And so all I'm really getting here is a clustering that depends on the population.
Middlesex is big relative to everything else and, therefore, that's what I get.
And it ignores things like education level and housing prices, and all those other things because the differences are small relative to 600,000.
Well, let's turn scaling on.
To do that, I want to show you how I do this scaling.
I did not, given the number of features and number of counties, do what I did for mammals and count them by hand to see what the maximum was.
I decided it would be a lot faster even at 2:00 in the morning to write code to do it.
So I've got some code here.
I've got Build County Points, just like Build Mammal Points and Read County Data, like Read Mammal Data.
But the difference here is, along the way, as I'm reading in each county, I'm keeping track of the maximum for each feature.
And then I'm just going to just do the scaling automatically.
So exactly the one over max scaling I did for mammals' teeth, I'm going to do it for counties, but I've just written some code to automate that process because I knew I would never be able to count them.
All right, so now let's see what happens if we run it that way.
Test 3, New England, and Scale equals True.
I'm either scaling it or not, is the way I wrote this one.
And with the scaling on again, I get a very different set of clusters.
What have we got?
Where's Middlesex?
It's in one of these 2 clusters.
Oh, here it is.
It's C0.
But it's with Fairfield, Connecticut and Hartford, Connecticut and Providence, Rhode Island.
It's a different answer.
Is it a better answer?
It's not a meaningful question, right?
It depends what I'm trying to infer, what we hope to learn from the clustering, and that's a question we're going to come back to on Tuesday in some detail with the counties, and look at how, by using different kinds of scaling or different kinds of features, we can learn different things about the counties in this country.
Before I do that, however, I want to move away from New England.
Remember we're focusing on New England because it took too long to do hierarchical clustering of 3,100 counties.
But that's what I want to do.
It's no good to just say, I'm sorry.
It took too long.
I give up.
Well, the good news is there are other clustering mechanisms that are much more efficient.
We'll later see they, too, have their own faults.
But we're going to look at k-means clustering, which has the big advantage of being fast enough that we can run it on very big data sets.
In fact, it is roughly linear in the number of counties.
And as we've seen before, when n gets very large, anything that's worse than linear is likely to be ineffective.
So let's think about how k-means works.
Step one, is you choose k. k is the total number of clusters you want to have when you're done.
So you start by saying, I want to take the counties and split them into k-clusters.
2 clusters, 20 clusters, a 100 clusters, 1,000 clusters.
You have to choose k in the beginning.
And that it's one of the issues that you have with k-means clustering is, how do you choose k?
We can talk about that later.
Once I've chosen k, I choose k points as initial centroids.
You may remember earlier today we saw this centroid method in class cluster.
So what's a centroid?
You've got a cluster, and in the clusters, you've got a bunch of points scattered around.
The centroid you can think of as, quote, "the average point," the center of the cluster.
The centroid need not be any of the points in the cluster.
So, again, you need some metric.
But let's say we're using Euclidean.
It's easy to see on the board.
The centroid is kind of there.
Now let's assume that we're going to start by choosing k-point from the initial set and labeling each of them as a centroid.
We often-- in fact, quite typically-- choose these at random.
So we now have k randomly chosen points, each of which we're going to call centroid.
The next step is to assign each point to the nearest centroid.
So we've got k-centroids.
We usually choose a small k, say 50.
And now we have to compare each of the 3,100 counties to each of the 50 centroids, and put each one in the correct thing, in the closest.
So it's 50 times 3,100, which is a lot smaller number than 3,100 squared.
So now I've got a clustering.
Kind of strange, because what it looks like depends on this random choice.
So there's no reason to expect that the initial assignment will give me anything very useful.
Step (4) is, for each of the k-clusters, choose a new centroid.
Now remember, I just chose at random k-centroids.
Now I actually have a cluster with a bunch of points in it, so I could, for example, take the average of those and compute a centroid.
And I can either take the average, or I can take the point nearest the average.
It doesn't much matter.
And then step (5) is one we've looked at before, assign each point to nearest centroid.
So now I'm going to get a new clustering.
And then, (6) is repeat (4) and (5) until the change is small.
So each time I do step (5), I can keep track of how many points I've moved from one cluster to another.
Or each time I do step (4), I can say how much have I moved the centroids?
Each of those gives me a measure of how much change the new iteration has produced.
When I get to the point where the iterations are not making much of a change-- and we'll see what we might mean by that-- we stop and say, OK, we now have a good clustering.
So if we think of the complexity each iteration is order k-n, where k is the number of clusters, and n is the number of points.
And then we do that step for some number of iterations.
So if the number of iterations is small, it will converge quite quickly.
And as we'll see, typically for k-means, we don't need a lot of iterations to get an answer.
It's typically not proportional to n, in particular, which is very important.
All right.
Tuesday, we'll go over the code for k-means clustering, and then have some fun playing with counties and see what we can learn about where we live.
All right.
Thanks a lot.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
Oh, it's so nice to get a response.
Thank you.
I appreciate it.
i have a confession to make.
I stopped at my usual candy store this morning and they didn't have any.
So I am bereft of anything other than crummy little Tootsie Rolls for today.
But I promise I'll have a new supply by Thursday.
We ended up the last lecture looking at pseudocode for k-means clustering and talking a little bit about the whole idea and what it's doing.
So I want to start today by moving from the pseudocode to some real code.
This was on a previous handout but it's also on today's handout.
So let's look at it.
Not so surprisingly I've chosen to call it k-means.
And you'll notice that it's got some arguments.
The point to be clustered, k, and that's an interesting question.
Unlike hierarchical clustering, where we could run it and get what's called a dendrogram and stop at any level and see what we liked, k-means involves knowing in the very beginning how many clusters we want.
We'll talk a little bit about how we could choose k. A cutoff.
What the cutoff is doing, you may recall that in the pseudocode k-means was iterative and we keep re-clustering until the change is small enough that we feel it's stable.
That is to say, the new clusters are not that much different from the old clusters.
The cutoff is the definition of what we mean by small enough.
We'll see how that gets used.
The type of point to be clustered.
The maximum number of iterations.
There's no guarantee that things will converge.
As we'll see, they usually converge very quickly in a small number of iterations.
But it's prudent to have something like this just in case things go awry.
And to print.
Just my usual trick of being able to print some debugging information if I need it, but not getting buried in output if I don't.
All right.
Let's look at the code.
And it very much follows the outline of the pseudocode we started with last time.
We're going to start by choosing k initial centroids at random.
So I'm just going to go and take all the points I have.
And I'm assuming, by the way, I should have written this down probably, that I have at least k points.
Otherwise it doesn't make much sense.
If you have 10 points, you're not going to find 100 clusters.
So I'll take k random centroids, and those will be my initial centroids.
There are more sophisticated ways of choosing centroids, as discussed in the problem set, but most of the time people just choose them at random, because at least if you do it repetitively it guarantees against some sort of systematic error.
Whoa.
What happened?
I see.
Come back.
Thank you.
All right.
Then I'm going to say that the clusters I have initially are empty.
And then I'm going to create a bunch of singleton clusters, one for each centroid.
So all of this is just the initialization, getting things going.
I haven't had any iterations yet.
And the biggest change so far I'm just setting arbitrarily to the cutoff.
All right.
And now I'm going to iterate until the change is smaller than the cutoff while biggest change is at least the cutoff.
And just in case numIters is less than the maximum, I'm going to create a list containing k empty list.
So these are the new clusters.
And then I'm going to go through for i in range k. I'm going to append the empty cluster.
These are going to be the new ones.
And then for p and all the points I'm going to find the centroid in the existing clustering that's closest to p. That's what's going on here.
Once I've found that, I'm going to add p to the correct cluster, go and do it for the next point.
Then when I'm done, I'm going to compare the new clustering to the old clustering and get the biggest change.
And then go back and do it again.
All right?
People, understand that basic structure and even some of the details of the code.
It's not very complicated.
But if you haven't seen it before, it can be a little bit tricky.
When I'm done I'm going to just get some statistics here about the clusters, going to keep track of the number of iterations and the maximum diameter of a cluster, so the cluster in which things are least tightly grouped.
And this will give me an indication of how good a clustering I have.
OK?
Does that make sense to everybody?
Any questions about the k-means code?
Well, before we use it, let's look at how we use it.
I've written this function testOne that uses it.
Some arbitrary values for k in the cutoff.
Number of trials is kind of boring here.
I've only said one is the default and I've set print steps to false.
The thing I want you to notice here, because I'm choosing the initial clustering at random, I can get different results each time I run this.
Because of that, I might want to run it many times and choose the quote, "best clustering."
What metric am I using for best clustering?
It's a minmax metric.
I'm choosing the minimum of the maximum diameters.
So I'm finding the worst cluster and trying to make that as good as I can make it.
You could look at the average cluster.
This is like the linkage distances we talked about before.
That's the normal kind of thing.
It's like when we did Monte Carlo simulations or random walks, flipping coins.
You do a lot of trials and then you can either average over the trials, which wouldn't make sense for the clustering, or select the trial that has some property you like.
This is the way people usually use k-means.
Typically they may do 100 trials and choose the best, the one that gives them the best clustering.
Let's look at this, and let's try it for a couple of examples here.
Let's start it up.
And we'll just run test one on our old mammal teeth database.
We get some clustering.
Now we'll run it again.
We get a clustering.
I don't know, is it the same clustering?
Kind of looks like it is.
No reason to suspect it would be.
We run it again.
Well you know, this is very unfortunate.
It's supposed to give different answers here because it often does.
I think they're the same answers, though.
Aren't they?
Yes?
Anyone see a difference?
No, they're the same.
How unlucky can you be?
Every time I ran it at my desk it came up the first two times with different things.
But take my word for it, and we'll see that with other examples, it could come out with different answers.
Let's try it with some printing on.
We get some things here.
Let's try it.
What have we got out of this one?
All right.
Oh, well.
Sometimes you get lucky and sometimes you get unlucky with randomness.
All right.
So, why did we start with k-means?
Not because we needed it for the mammals' teeth.
The hierarchical worked fine, but because it was too slow when we tried to look at something big like the counties.
So now let's move on and talk about clustering the counties.
We'll use exactly the k-means code.
It's one of the reasons we're allowed to pass in the point type as an argument.
But the interesting thing will be what we do for the counties themselves.
This gets a little complicated.
In particular, what I've added to the counties is this notion of a filter.
The reason I've done this is, as we've seen before, the choice of features can make a big difference in what clustering you get.
I didn't want to do a lot of typing as we do in these examples, so what I did is I created a bunch of filters.
For example, no wealth, which says, all right, we're not going to look at home value.
We're giving that a weight of 0.
We're giving income a weight of 0, we're giving poverty level a rate of 0.
But we're giving the population a weight of 1, et cetera.
OK. What we see here is each filter supplies the weight, in this case either 0 or 1, to a feature.
This will allow me as we go forward to run some experiments with different features.
All features, everything has a weight of 1.
I made a mistake though.
That should have been a 1.
Then I have filter names, which are just a dictionary.
And that'll make it easy for me to run various kinds of tests with different filters.
Then I've got init, which takes as its arguments the things you would expect, plus the filter name.
So it takes the original attributes, the normalized attributes.
And you will recall that, why do we need to normalize attributes?
If we don't, we have something like population, which could number in the millions, and we're comparing it to percent female, which we know cannot be more than a 100.
So the small values become totally dominated by the big absolute values and when we run any clustering it ends up only looking at population or number of farm acres, or something that's big.
Has a big dynamic range.
Manhattan has no farm acres.
Some county in Iowa has a lot.
Maybe they're identical in every other respect.
Unlikely, but who knows?
Except I guess there's no baseball teams in Iowa.
But at any rate, we always scale or we try and normalize so that we don't get fooled by that.
Then I go through and, if I haven't already, this is a class variable attribute filter, which is initially set to none.
Not an instance variable, but a class variable.
And what we see here is, if that class variable is still none, this will mean it's the first time we've generated a point of type county, then what we're going to do is set up the filter to only look at the attributes we care about.
So only the attributes which have a value of 1.
And then I'm going to override distance from class point to look at the features we care about.
OK.
Does this basic structure and idea make sense to people?
It should.
I hope it does, because the current problem set requires you to understand it in which you all will be doing some experiments.
So now I want to do some experiments with it.
I'm not going to spend too much time, even though it would be fun, because I don't want to deprive you of the fun of doing your problem sets.
So let's look at an example.
I've got test, which is pretty much like testOne.
Runs k-means number of times and chooses the best.
And we can start.
Well, let's start by running some examples ourselves.
So I'm going to start by clustering on education level only.
I'm going to get 20 clusters, 20 chosen just so it wouldn't take too long to run, and we'll filter on education.
And we'll see what we get.
Well, I should have probably done more than one cluster just to make it work.
But we've got it and just for fun I'm keeping track of what cluster Middlesex County, the county in which MIT shows up.
So we can see that it's similar to a bunch of other counties.
And it happens to have an average income of $28,665, or at least it did then.
And if we look, we should also see-- no, let me go back.
I foolishly didn't uncomment pyLab.show.
So we better go back and do that.
Well, we're just going to nuke it and run it again because it's easy and I wanted to run it with a couple of trials anyway.
So, we'll first do the clustering.
We get cluster 0.
Now we're getting a second one.
It's going to choose whichever was the tightest.
And we'll see that that's what it looks like.
So we've now clustered the counties based on education level, no other features.
And we see that it's got some interesting properties.
There is a small number of counties, clusters, out here near the right side with high income.
And, in fact, we'll see that we are fortunate to be in that cluster.
One of the clusters that contains wealthy counties.
And you could look at it and see whether you recognize any of the other counties that hang out with Middlesex.
Things like Marin County, San Francisco County.
Not surprisingly.
Remember, we're clustering by education and these might be counties where you would expect the level of education to be comparable to the level of education in Middlesex.
All right.
Let me get rid of that for now.
Sure.
I ran it.
I didn't want you to have to sit through it, but I ran it on a much bigger sample size.
So here's what I got when I ran it asking for 100 clusters.
And I think it was 5 trials.
And you'll notice that this case, actually, we have a much smaller cluster containing Middlesex.
Not surprising, because I've done 100 rather than 20.
And it should be pretty tight since I chose the best-- you can see we have a distribution here.
Now, remember that the name of the game here is we're trying to see whether we can infer something interesting by clustering.
Unsupervised learning.
So one of the questions we should ask is, how different is what we're getting here from if we chose something at random?
Now, remember we did not cluster on things based on income.
I happened to plot income here just because I was curious as to how this clustering related to income.
Suppose we had just chosen at random and split the counties at random into 100 different clusters, What would you have expected this kind of graph to look like?
Do we have something that is different, obviously different, from what we might have gotten if we'd just done a random division into 100 different clusters?
Think about it.
What would you get?
A bell curve?
Pardon?
We'd get a bell curve
Well, a bell curve is a good guess because bell curves occur a lot in nature.
And as I said, I apologize for the rather miserable quality of the rewards.
It's a good guess but I think it's the wrong guess.
What would you expect?
Would you expect the different clusters-- yeah, go ahead
You probably might expect them all to average at a certain point for a very sharp bell curve?
A very sharp bell curve was one comment.
Well, someone else want to try it?
That's kind of close.
I thought you were on the right track in the beginning.
Well, take a different example.
Let's take students.
If I were to select a 100 MIT students at random and compute their GPA, would you it to be radically different from the GPA of all of MIT?
The average GPA of all MIT students?
Probably not, right?
So if I take 100 counties and put them into a cluster, the average income of that cluster is probably pretty close to the average income in the country.
So you'd actually expect it to be kind of flat, right?
That each of the randomly chosen clusters would have the same income, more or less.
Well, that's clearly not what we have here.
So we can clearly infer from the fact that this is not flat that there is some interesting correlation between level of income and education.
And for those of us who earn our living in education, we're glad to see it's positive, actually.
Not negative.
As another experiment, just for fun, I clustered by gender only.
So this looked only at the female/male ratio in the counties.
And here you'll see Middlesex is with-- remember we had about 3,000 counties to start with.
So the fact that there were so few in the cluster on education was interesting, right?
Here we have more.
And we get a very different-looking picture.
Which says, perhaps, that the female/male ratio is not unrelated to income, but it's a rather different relation than we get from education.
This is what would be called a bi-modal distribution.
A lot here and a lot here and not much in the middle.
But again the dynamic range is much smaller.
But we do have some counties where the income is pretty miserable.
All right.
We could play a lot more with this but I'm not going to.
I do want to, before we leave it, because we're about to leave machine learning, reiterate a few of the major points that I wanted to make sure were the take home messages.
So, we talked about supervised learning much less than we talked about unsupervised.
Interestingly, because unsupervised learning is probably used more often in the sciences than supervised.
And when we did supervised learning, we started with a training set that had labels.
Each point had a label.
And then we tried to infer the relationships between the features of the points and the associated labels.
Between the features and the labels.
We then looked at unsupervised learning.
The issue here was, our training set was all unlabeled data.
And what we try and infer is relationships among points.
So, rather than trying to understand how the features relate to the labels, we're just trying to understand how the points, or actually, the features related to the points, relate to one another.
Both of these, as I said earlier, are similar to what we saw when we did regression where we tried to fit curves to data.
You need to be careful and wary of over-fitting just as you did with regression.
In particular, if the training data is small, a small set of training data, you may learn things that are true of the training data that are not true of the data on which you will subsequently run the algorithm to test it.
So you need to be wary of that.
Another important lesson is that features matter.
Which features matter?
It matters whether they're normalized.
And in some cases you can even weight them if you want to make some features more important than the others.
Features need to be relevant to the kind of knowledge that you hope to acquire.
For example, when I was trying to look at the eating habits of mammals, I chose features based upon teeth, not features based upon how much hair they had or their color of the lengths of the tails.
I chose something that I had domain knowledge which would suggest that it was probably relevant to the problem at hand.
Question at hand.
And then we discovered it was.
Just as here, I said, well, maybe education has something to do with income.
We ran it and we discovered, thank goodness, that it does.
OK.
So I probably told you ten times that features matter.
If not, I should have because they do.
And it's probably the most important thing to get right in doing machine learning.
Now, our foray into machine learning is part of a much larger unit.
In fact, the largest unit of the course really, is about how to use computation to make sense of the kind of information one encounters in the world.
A big part of this is finding useful ways to abstract from the situation you're initially confronted with to create a model about which one can reason.
We saw that when we did curve fitting.
We would abstract from the points to a curve to get a model.
And we see that with machine learning, that we abstract from every detail about a county to say, the education level, to give us a model of the counties that might be useful.
I now want to talk about another kind of way to build models that's as popular a way as there is.
Probably the most common kinds of models.
Those models are graph theoretic.
There's a whole rich theory about graphs and graph theory that are used to understand these models.
Suppose, for example, you had a list of all the airline flights between every city in the United States and what each flight cost.
Suppose also, counterfactual supposition, that for all cities A, B and C, the cost of flying from A to C by way of B was the cost of A to B and the cost from B to C. We happen to know that's not true, but we can pretend it is.
So what are some of the questions you might ask if I gave you all that data?
And in fact, there's a company called ITA Software in Cambridge, recently acquired by Google for, I think, $700 million, that is built upon answering these kinds of questions about these kinds of graphs.
So you could ask, for example, what's the shortest number of hops between two cities?
If I want to fly from here to Juneau, Alaska, what's the fewest number of stops?
I could ask, what's the least expensive-- different question-- flight from here to Juneau.
I could ask what's the least expensive way involving no more than two stops, just in case I don't want to stop too many places.
I could say I have ten cities.
What's the least expensive way to visit each of them on my vacation?
All of these problems are nicely formalized as graphs.
A graph is a set of nodes.
Think of those as objects.
Nodes are also often called vertices or a vertex for one of them.
Those nodes are connected by a set of edges, often called arcs.
If the edges are uni-directional, the equivalent of a one-way street, it's called a digraph, or directed graph.
Graphs are typically used in situations in which there are interesting relationships among the parts.
The first documented use of this kind of a graph was in 1735 when the Swiss mathematician Leonard Euler used what we now call graph theory to formulate and solve the Konigberg's Bridges problem.
So this is a map of Konigsberg, which was then the capital of East Prussia, a part of what's today Germany.
And it was built at the intersection of two rivers and contained a lot of islands.
The islands were connected to each other and to the mainland by seven bridges.
For some bizarre reason which history does not record and I cannot even imagine, the residents of this city were obsessed with the question of whether it was possible to take a walk through the city that involved crossing each bridge exactly once.
Could you somehow take a walk and go over each bridge exactly once?
I don't know why they cared.
They seemed to care.
They debated it, they walked around, they did things.
It probably would be unfair for me to ask you to look at this map and answer the question.
But it's kind of complicated.
Euler's great insight was that you didn't have to actually look at the level of detail represented by this map to answer the question.
You could vastly simplify it.
And what he said is, well, let's represent each land mass by a point, and each bridge as a line.
So, in fact, his map of Konigsberg looked like that.
Considerably simpler.
This is a graph.
We have some vertices and some edges.
He said, well, we can just look at this problem and now ask the question.
Once he reformulated the problem this way it became a lot simpler to think about and he reasoned as follows.
If a walk were to traverse each bridge exactly once, it must be the case that each node, except for the first and the last, in the walk must have an even number of edges.
So if you were to go to an island and leave the island and traverse each bridge to the island unless there were an even number, you couldn't traverse each one exactly once.
If there were only one bridge, once you got to the island you were stuck.
If there were two bridges you could get there and leave.
But if there were three bridges you could get there, leave, get there and you're stuck again.
He then looked at it and said, well, none of these nodes have an even number of edges.
Therefore you can't do it.
End of story.
Stop arguing.
Kind of a nice piece of logic.
And then Euler later went on to generalize this theorem to cover a lot of other situations.
But what was important was not the fact that he solved this problem but that he thought about the notion of taking a map and formulating it as a graph.
This was the first example of that and since then everything has worked that way.
So if you take this kind of idea and now you extend it to digraphs, you can deal with one-way bridges or one-way streets.
Or suppose you want to look at our airline problem, you can extend it to include weights.
For example, the number of miles between two cities or the amount of toll you'd have to pay on some road.
So, for example, once you've done this you can easily represent the entire US highway system or any roadmap by a weighted directed graph.
Or, used more often, probably, the World Wide Web is today typically modeled as a directed graph where there's an edge from page A to page B, if there's a link on page A to page B.
And then maybe if you want to care, ask the question how often do people go from A to B, a very important question, say, to somebody like Google, who wants to know how often people click on a link to get to another place so they can charge for those clicks, you use a weighted graph, which says how often does someone go from here to there.
And so a company like Google maintains a model of what happens and uses a weighted, directed graph to essentially represent the Web, the clicks and everything else, and can do all sorts of analysis on traffic patterns and things like that.
There are also many less obvious uses of graphs.
Biologists use graphs to measure things ranging from the way proteins interact with each other to, more obviously, gene expression networks are clearly graphs.
Physicists use graphs to model phase transitions with typically the weight of the edge representing the amount of energy needed to go from one phase to another.
Those are again weighted directed graphs.
The direction is, can you get from this phase to that phase?
And the weight is how much energy does it require?
Epidemiologists use graphs to model diseases, et cetera.
We'll see an example of that in a bit.
All right.
Let's look at some code now to implement graphs.
This is also in your handout and I'm going to comment this out just so we don't run it by accident.
As you might expect, I'm going to use classes to implement graphs.
I start with a class node which is at this point pretty simple.
It's just got a name.
Now, you might say, well, why did I even bother introducing a class here?
Why don't I just use strings?
Well, because I was kind of wary that sometime later I might want to associate more properties with nodes.
So you could imagine if I'm using a graph to model the World Wide Web, I don't want more than the URL for a page.
I might want to have all the words on the page, or who knows what else about it.
So I just said, for safety, let's start with a simple class, but let's make it a class so that any code I write can be reused if at some later date I decide nodes are going to be more complicated than just strings.
Good programming practice.
An edge is only a little bit more complicated.
It's got a source, a destination, and a weight.
So you can see that I'm using the most general form of an edge so that I will be able use edges not only for graphs and digraphs, but also weighted directed graphs by having all the potential properties I might need and then some simple things to fetch things.
The next cluster in the hierarchy is a digraph.
So it's got an init, of course, I can add nodes to it.
I'm not going to allow myself to add the same node more than once.
I can add edges.
And I'm going to check to make sure that I'm only connecting nodes that are in the graph.
Then I've got children of, which gives me all the descendants of a node, and has node in string.
And then interestingly enough, maybe surprising to some of you, I've made graph a sub-class of digraph.
Maybe that seems a little odd.
After all, when I started I started talking about digraphs.
About graphs, and then said, and we can add this feature.
But now I'm going the other way around.
Why is that?
Why do you think that's the right way to structure a hierarchy?
What's the relation of graphs to digraphs?
Digraphs are more general than graphs.
A graph is a specialization of a digraph.
Just like a county is a specialization of a point.
So, typically as you design these class hierarchies, the more specialized something is, the further it has to be down.
More like a subclass.
Does that make sense?
I can't really turn this on its head.
I can specialize a digraph to get a graph.
I can't specialize a graph to get a digraph.
And that's why this hierarchy is organized the way it is.
What else is there are interesting to say about this?
A key question, probably the most important question in designing and implementation of graphs, is the choice of data structures to represent the digraph in this case.
There are two possibilities that people typically use.
They can use an adjacency matrix.
So if you have N nodes, you have an N by N matrix where each entry gives, in the case of a digraph, a weighted digraph, the weight-connecting nodes on that edge.
Or in the case of a graph it can be just true or false.
So this is, can I get from A to B?
Is that going to be sufficient?
Suppose you have a graph that looks like this.
And we'll call that Boston and this New York.
And I want to model the roads.
Well, I might have a road that looks like this and a road that looks like this from Boston to New York.
As in, I might have more than one road.
So I have to be careful when I use an adjacency matrix representation, to realize that each element of the matrix could itself be somewhat more complicated in the case that there are multiple edges connecting two nodes.
And in fact, in many graphs we will see there are multiple edges connecting the same two nodes.
It would be surprising if there weren't.
All right.
Now, the other common representation is an adjacency list.
In an adjacency list, for every node I list all of the edges emanating from that node.
Which of these is better?
Well, neither.
It depends upon your application.
An adjacency matrix is often the best choice when the connections are dense.
Everything is connected to everything else.
But is very wasteful if the connections are sparse.
If there are no roads connecting most of your cities or no airplane flights connecting most of your cities, then you don't want to have a matrix where most of the entries are empty.
Just to make sure that people follow the difference, which am I using in my implementation here?
Am I using an adjacency matrix or an adjacency list?
I heard somebody say an adjacency list and because my candy supply is so meager they didn't even bother raising in their hand so I know who said it.
But yes, it is an adjacency list.
And we can see that by looking what happens when we add an edge.
I'm associating it with the source node.
So from each node, and we can see that when we look at the children of-- here, I just return the edges of that node.
And that's the list of all the places you can get to from that node.
So it's very simple, but it's very useful.
Next lecture we'll look-- Yeah?
Thank you.
Question.
I love questions
Going back to the digraph, what makes the graph more specialized?
What makes--
The graph more specialized?
Good question.
The question is, what makes the graph more specialized?
What we'll see here when we look at graph, it's not a very efficient implementation, but every time you add an edge, I add an edge in the reverse direction.
Because if you can get from node A to node B you can get from node B to node A.
So I've removed the possibility that you have, say, one-way streets in the graph.
And therefore it's a specialization.
There are things I can not do with graphs that I can do with digraphs, but anything I can do with a graph I can do with the digraph.
It's more general.
That make sense?
That's a great question and I'm glad you asked it.
All right.
Thursday we're going to look at a bunch of classic problems that can be solved using graphs, and I think that should be fun.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK so we're going to test it.
Here's testOne.
It just says for name and range 10, I'm going to build some nodes, put in a few edges, and then I'm going to print the graph.
And all I really want to show you here, is that if we run it for digraph or we run it for graph, we'll get something different.
Yes, I'm happy to save the source.
Oh, and now the syntax.
It was valid last time I looked, what have we done wrong here?
I edit something badly?
Sort of looks valid to me, doesn't it?
We'll retype it.
Yes, we'll save.
Nope.
Well, it's one of those days, isn't it?
All right this is a good debugging exercise for us.
Let's think about how we go and find this.
I'm sure you've all had this sort of problem.
Well the first thing to do is I think I'll just comment out all of this.
Let's see if I still get an error.
I do.
All right, well that suggests that, that wasn't the problem, so I can put it back.
Now it shows a problem all the way down there.
So let's see what's going on here.
Or maybe, I'll just skip this, but doubtless I'll get in trouble if I do.
So let's see.
I must have just made a sloppy editing error somewhere this morning in commenting things out for the lecture.
Well, I think what we're going to do, for the moment, is move on and hope it goes away.
Now that seems silly.
Sorry about this everybody.
People should look with me, and someone may see it more quickly than I can.
In fact, I'm hoping someone sees it more quickly than I can.
We've now got a microphone, and the embarrassment of a code with syntax error, [UNINTELLIGIBLE]
You've got a sign.
For some reason it wasn't on the schedule, so--
Well just because we've been teaching at 10 o'clock
Yeah, I know.
Yeah.
[UNINTELLIGIBLE]
February.
There's no reason to suspect that we would teach at 10 o'clock today
I actually looked.
I double checked
Well this is embarrassing, folks.
And I wish one of you would bail me out by telling me where my syntax error is.
Well random looks OK, right?
Node looks OK. And it gets more complicated
Hey professor, can you turn on the transmitter?
No.
[LAUGHTER]
OK, I'll turn on the transmitter.
But I'm really focused on a different problem right now.
Guys help!
Where are my TAs?
Why does it keep doing that to me?
Maybe there's something just funny going on here.
Pardon?
Restart
Restarting IDLE?
You think maybe that's the issue?
We could try that.
Ha!
So it looks like IDLE was just in some ugly state.
Let's hope.
Yes, all right, so I didn't have a bug.
It was just IDLE had a bug.
All right, phew.
But we did squander ten minutes.
Oh well.
So we have the graph, and you can see when we look at a graph, we have a node from 1 to 2, an edge from 1 to 2, from 1 to 1, et cetera.
And that's the digraph.
When we look at the graph, we'll see that in fact we have a lot more nodes, because everything goes in both directions.
But that's what we expected-- nothing very interesting.
All I want you to do is notice the difference here between graphs and digraphs.
Now getting to the whole point, once we have this mechanism set up to think about graphs, we can now think about interesting problems and formulate them as graph problems.
And I want to list a few of the interesting problems, and then we'll look at how to solve some of them.
So probably the most common graph problem that people solve, is called the shortest path problem.
We talked about this briefly last time.
The notion here is for some pair of nodes, n1 and n2, we want to find the shortest sequence of edges that connects those two nodes.
All right?
So that's very straightforward.
Then there is the shortest weighted path, where instead of trying to find the shortest sequence of edges, we want to find the smallest total weight.
So it may be that we traverse a few extra edges, but since they have a shorter weights, we end up getting a shorter path.
So we might indirect to do the shortest path.
This is probably the more common problem.
So for example, that's the problem that Google Maps solves, when you ask it to give you driving directions.
And you'll notice when you use something like Google Maps or MapQuest, you can tell it to minimize the time, in which case maybe it will route you on a freeway, where you can drive at 80 miles an hour, even though you drive a few extra miles.
Or you can tell it to minimize the distance in which case it may take you on these crummy little surface roads where you have to drive slowly, but you'll cover fewer miles and use less gas.
So you get to tell it which set of weights you care about, and then it finds you the shortest path, given those weights.
We'll come back to this since we're going to look at some code to implement it.
Another slightly more complicated problem to understand is finding cliques.
So to find a clique, we're looking to find a set of nodes, such that there exists a path connecting each node in the set.
So you can think of this as similar to, say a social clique-- who your friends are.
It's a group of nodes or group of people that somehow can get to each other.
It's not saying you can't get outside the clique.
But it is guaranteeing that from any member of the clique, you can reach any other member of the clique.
And so well, we'll look at some examples of where finding a clique is useful.
And the final kind of problem I want to mention is the minimum cut problem-- often abbreviated mincut.
So the problem here, is given a graph, and given two sets of nodes, you want to find the minimum number of edges such that if those edges are removed, the two sets are disconnected.
i.e.
you can't get from a member of one set to a member of the other set.
This is often a question that gets asked.
For example, imagine that you were the government of Syria and you want to ensure that nobody can post a video on YouTube.
You would take the set of nodes in Syria, and you would take the set of nodes probably outside Syria, and ask what's the minimum number of communication links you'd have to cut to ensure that you can't get from a node in Syria to a node outside Syria.
People who do things like plan power lines worry about that.
They want to say what's the minimum number of links such that if they're cut, you can't get any electricity from this power plant to say, this city.
And they'll typically design their network with redundancy in it, so that the mincut is not too small.
And so people frequently are worried about mincut problems, and trying to see what that is.
All right, now let's look at a couple of examples, in slightly more detail.
So what we see here is a pictorial representation of a weighted graph generated by the Centers for Disease Control, CDC, in Atlanta in 2003 when they were studying an outbreak of tuberculosis in the United States-- a virulent and bad infectious disease.
Each node, and you can see these little dots are the nodes, represents a person.
And each node is labeled by a color, indicating whether the person has active tuberculosis, has tested positive for exposure, but doesn't have the disease, or tested negative for exposure, or not been tested.
So you'll remember when we looked last time at class node, and asked why did I bother creating a class for something so simple, it was because I said well maybe we would add extra properties to a node.
So now in some sense these colors would be easy to add.
So I could add to class node-- well I could attribute color, and call it red or blue or green-- or more likely an attribute saying TB state, which would indicate active not active, et cetera.
The edges, which you can see here, represent connections among pairs of people.
What I didn't bother, you can't see on these pictures, is the edges are actually weighted.
And the weights there are about how closely people are connected.
And there are really only two weights I think they used: close, someone who say lives in your house or works in the same office, or casual, a neighbor you might have encountered, but you wouldn't expect to necessarily see them every day.
So I've taken a fairly complicated set of information and represented it as a graph.
Now what are some of the interesting graph theoretic questions we might proceed to ask about this?
So an important question they typically ask when diseases break out unexpectedly is, is there an index patient?
The index patient is the patient who brought the disease into the community-- so somebody who visited some country, picked up tuberculosis, flew back to their neighborhood in the US and started spreading it.
How would we formulate that as a graph question?
Again, quite simply.
We would say does there exist a node such that node has TB, or maybe not even that, no, let's simplify it.
You might say, "or tested positive" because maybe you can communicate it without having it-- has TB and is connected to every node with TB.
Now this doesn't guarantee that the patient is an index patient.
But if there is no such patient, no such node, then you know that there's not a single source of this disease in the community.
How would we change the graph to model it in a more detailed way, and remember this is all about modeling, so that we could ask a question or more precisely?
Well we'd have to change to a more complex coloring scheme, if you will, in which we'd include the date of when somebody acquired the disease, or tested positive.
And then we could ask those kinds of questions in a little bit more detail.
But again once we've built the model, we can then go and ask a lot of interesting questions.
By the way the answer to this question, for this graph, is almost.
There is an index patient that's connected to every node in the graph, except for the nodes in this black circle.
They are not connected to any index patient.
So the CDC actually did that analysis, and they reached that conclusion that there didn't seem to be.
And then later, it came to light, in fact, that this particular graph is missing an edge.
Somebody had moved from neighborhood A to neighborhood B, and they had not kept track of that.
And if they had, they would have discovered there was a link that's missing-- an edge that's missing from this graph-- which in fact would've connected everybody to the index patient.
It was an interesting question.
They only found that, because they were puzzled about this tiny little black circle out here, and started investigating all the people in the black circle, and discovered that one of them had moved from one place to another.
What's another question you might ask once you've built this model?
Well suppose this is the current state of the world, and I want to reduce the spread of the disease, by vaccinating uninfected people so that they don't contract tuberculosis.
But I have a minimum, it's expensive to do this, I only have so much vaccine.
Who should get it?
What's the graph theory problem that I would solve to address the question of what's the best way to allocate my limited supply a vaccine?
Exactly.
I, by the way, have much better candy now.
So I think that's where the minimum cut came from.
Well, all right.
It's better for eating.
It's just worse for throwing.
That's easier to throw.
All right.
It's the minimum cut problem.
I take the people who are already infected, view them as one set of nodes.
I take the people who are not infected, and view them as another set of nodes, find the edges that I need to cut to separate them, and then vaccinated somebody on one side of the edge, so that they don't contract the disease.
So again a nice, easily formalized, problem.
All right, so that's an example.
Let's look at another example.
Let's think about the shortest path problem here.
And we'll do that by thinking about social networks.
So I suspect that at least a few of you have used Facebook, and you have friends-- some of you more, than others.
I see people laughing.
This is someone who probably has two friends, and is said.
I don't know, or 1,000 friends and is happy.
Who knows-- I don't want to know please.
And I'm not going to tell you how many friends I have either.
But you might ask the question, suppose you wanted to reach Donald Trump -- erstwhile Republican, vice presidential candidate, or presidential candidate.
Say is there a connection from you to Donald Trump?
Do you have a friend, who has a friend, who has a friend, who is a friend with Donald Trump?
Or for Barack Obama, or anyone else you'd ask.
Well what's the shortest path?
How many friends do you have to go through?
This is what's called the six degrees of separation problem.
In the 1990s, the playwright John Guare published a play called Six Degrees of Separation, under the slightly dubious premise, that everybody in the world was connected to everybody else in the world with at most six hops.
If you took all the people you knew, all the people they knew, et cetera, you could reach any person in the world in six phone calls-- say any person who has a phone.
I don't know whether that's true, but this is the whole notion of a social network.
So if we wanted to look at that in Facebook, we could either assume that the friend relation is symmetric-- if I'm your friend, you're my friend, which it is.
Or you could imagine a different model, in which it's asymmetric.
If it's symmetric you have a graph.
If it's asymmetric you have a directed graph.
And then you just ask the question.
What's the shortest path from you to whomever you care about?
And you get that.
You could imagine that Facebook already knows the answer to that question, but just won't tell you.
But they'll sell it to somebody who has enough money.
All right, So how does Facebook solve this problem?
They have a very simple piece of code, which we'll now look at which solves the shortest path.
So let's go back.
So here's a recursive implementation of shortest path.
Comment this out while I'm in the neighborhood.
It takes the graph, a start node and end node to print, and this extra argument call visited.
We'll see why that's gets used.
And we'll think about the algorithm.
This particular algorithm is what's called a depth first search algorithm.
It's a recursive depth first search.
We've seen these before, often abbreviated DFS.
So if you think about having a graph of a bunch of nodes connected to one another, just for fun we'll say it does something like this.
What depth first search does is it starts at the source node for the shortest path, let's called it this one, it first visits one child, then visits all the children of those children.
This one has no children.
Visits this child, picks one of its children, visits all of its children-- let's say it had another one here-- and goes on until it's done.
And then it back tracks, comes back and takes the next child.
Then we have to be a little bit careful about the circle.
So to summarize it, first thing we have to say is the recursion ends, when start equals end.
That is to say I've called it and I've asked is there a path from A to A, and the answer is yes, there is.
I'm already there.
Now you could argue, and in some formulations the answer is not necessarily, you'd say there's only a path if there's an edge from A to A.
But I've chosen to make the simpler assertion that if you want to get to A, and you're already there, you're done.
Kind of seems reasonable.
So then the recursive part, starts by choosing one child of the node you're currently at.
And it keeps doing that until either it reaches a node with no children, or it reaches the node you're trying to get to, or, and here's an important part, it reaches a node it's already seen.
And that's what visited is about.
Because I want to make sure that when I explore this graph, I don't go from here to here to here to here to here to here ad nauseum, because I'm stuck in what's called a cycle.
You have to avoid the cycles.
Once it's got to a node that has no children, if that's not the node it's trying to get to, it back tracks and takes the next child of the node it was at.
And in that way, it systematically explores all possible paths, and along the way, it chooses the best one.
So we can look at the code here.
I've just commented out something we'll look at later just as we try and instrument it to see how fast it's working.
I've got a debugging statement just to say whether I'm going to print what I've been asked to do, in case it's not working.
And then the real work starts.
I get the original path is just the node we're starting at, if start is end, I stop.
If I get to here, or say shortest equals none, I haven't found any paths yet.
So there is no shortest path.
And then for node in the children of start, if I haven't already visited the node-- this is to avoid cycles-- I create a visited list that contains whatever used to contain plus the node.
Notice that I'm creating a new list here, rather than mutating the old list.
And that's because when I unravel my recursion, and back track to where I was, I don't want to have think I visited something I haven't visited, right?
If I had only one list, and I mutated each time, then as I go down the recursion and back up the recursion, I'm always dealing with the same list.
By getting a new list, I'm ensuring that I don't have that problem.
Then I say the new path is whatever the shortest path is, from the node I'm currently at to the desired end node.
And I use the current set of visited nodes to indicate where I've already been at this part of the recursion.
If the new path is none, well didn't find one, I continue.
Otherwise, I found a path, and now I just want to check is it better, or worse, or the same, as the previous shortest path.
And then I'm done.
Very straightforward.
The only really tricky part was making sure that I kept track of visited properly, and didn't get stuck in cycles.
OK let's run it.
So here's testTwo -- builds the same kind of graph we've built before.
And then it tries to find the shortest path.
And I'm going to do it for the same input, essentially, the same at edge operations, but once when it's a graph and once when it's a digraph.
So you'll notice that it found two different answers.
When it was a graph, it could get from 0 to 4 in essentially one hop.
But when it was a digraph, it took longer.
It had to go from 0 to two to 3 to 4.
And that's not surprising, because the graph has more edges.
And in fact, what we saw is that in the graph there was an edge from 4 to 0, but there was no such edge in the directed graph.
So again you'll get, unsurprisingly, different answers-- but very straightforwardly.
Now let's try it on a bigger problem.
So I've called this big test.
And what this does, is rather than my sitting there and typing a bunch of at edge commands, it just generates edges at random.
So I tell it whether I want it to be a graph or digraph, and then I give it the number of nodes I want, and the number of edges.
And it just generates, at random, a graph in this case with 25 nodes and 200 edges.
So let's see what happens here.
So it's printed out the graph, and now we're waiting a little bit.
It will eventually finish, there.
I can get from 0 to 4.
It turns out there's a short path for this random graph from 0 to 14 to 4.
It's not so surprising that there's a short path.
Why is it not surprising that there's a pretty short path?
It had a lot of edges, right?
I had 200 edges in my graph.
So things are pretty densely connected.
Why did it take so long?
Well remember what it's doing is exploring all the possible paths from 0 to 4, in this case.
This is very much like what we saw when we looked at the knapsack problem, right?
Where, there when we looked at the recursive implementation, we saw that well all right, generating all possibilities, there were an exponential number of possibilities there in the number of items.
Here, depending upon the number of nodes and the number of edges, it's also large, and in fact, exponential.
We could explore a lot of different paths, but let's see what's going on when we explore those.
So what I'm going to do now, is go back to our small example.
We'll run testTwo That was the small one we looked at.
But I'm going to set to print onto true, and if you remember what that code did is they told us what each recursive call was, what the start node was and what the end node was.
So it found the same shortest path.
That's a good thing, 0 to 4.
But how did it do that?
Well it first got called with the question of starting at 0 find me a path to 4.
It visited the first child of 0, which was 1.
It said, all right see if you can find a path from 1 to 4.
It then backtracked and sort of asked the same question, can I get from 2 to 4?
From 0 to 4?
And then it said well I can get from 0 to 2, let me try 2 to 4, 3 to 4, 4 to 4, that's good.
Get to 5 to 4, and then it tried to find 4 to 4 again.
Here it tried to find 2 to 4 again.
So what you can see, is as I do that backtracking, I'm solving the same problem multiple times.
Why am I doing that?
Because there may be multiple ways to get to the same node.
So if, for example, I looked at this graph, what we would see is I would try and let's say I want to get to here, just for the sake of argument, I'd first say can I get to here from here.
I'd try this, then I'd solve here to here.
And I'd do that.
I'd also go from here to here to here, and then for the second time, I'd try and solve the problem here to here.
Now here since it's only one connection, it's a short thing.
But you can see if I have multiple ways to get to the same intermediate node, each time I get there I'm going to solve a problem I have already solved-- how to get from that intermediate node to the final destination.
So I'm doing work I've already done before.
This is obviously troublesome.
Nobody likes to solve a problem they've already solved before.
So what do you think the solution is?
How would you solve this sort of thing yourself?
What would you do?
Well what you'd-- yeah, thank you.
This guy is hungry.
Go ahead
Some way of storing information that you've already looked at
Exactly.
What you try and do, is remember what you did before, and just look it up.
This is a very common technique.
It's called memoization.
We use this to solve a lot of problems where you remember what the answer was, and rather than recalculating it, you just look it up.
And that can, of course, be much faster.
So it's a fancy way to say we're going to use a table look-up.
This concept of memoization is at the heart of a very important programming technique called dynamic programming.
In the algorithms class that's taught in this room immediately following this class, they have spent at least four lectures on the topic of dynamic programming.
But since you guys are much smarter than those guys taking that class, we're going to do it in about 20 minutes, in today and a little bit in the next lecture.
All right, so let's look at an example.
We'll look at a solution.
So I've taken the recursive implementation we had before, and rewritten it just a little bit, to call dp, dynamic programming shortest path.
And the most important thing to notice is I've given yet another argument to the function, and that's the memo, which is initially an empty dictionary.
The rest of the algorithm proceeds as before, except what happens here is when I want to get from a path, the first question I ask is I say new path is equal to the memo of node to end.
So when I get to one of these interior nodes, and I want to say what's the shortest path from here to here, the first question I ask is do I already know the answer?
Is it already in my memo?
If so, I just look it up, and I'm done.
I found it, and I proceed as before.
If it's not in the memo, well this look up will fail, and I'll enter the except clause, and I'll make a call again.
So this is a very conventional way of using try, except as a control structure.
Failing to find in the memo is not an error, it just means I haven't yet stored it away.
And as I go, I'll build up the memo, and then I'm done.
So it's very simple.
So I should ask the question.
Does anyone need me to explain this again, or does it makes sense what we're doing here with a memo?
OK, I'm assuming it makes sense.
Let's test it.
And we'll first do a very simple test.
We're just going to use the same little graph we used before.
And I'm going to run shortest path, and dp_shortest path, and at least confirm that for one search I get the same answer.
It's just fire testing it to make sure that it's not a complete disaster.
And we do.
We get 0234, 0234.
So at least for one thing, it's the same thing.
Let's see about performance, because that's really what we got interested in.
So we'll go back to our big test.
And let's go back and for both of these, I'm going to uncomment, tracking this global variable, just keeping track of the number of calls, and we'll see whether we get a substantially different amount of recursion, in one versus the other.
So it's built some random graph again.
This is the non-dynamic programming one, which as we recall, takes a bit longer.
I probably should have said-- all right, so it's pretty big difference.
They found the same path, 0, 2,3, 4.
But you'll notice the straightforward depth first search took over 800,000 recursive calls, whereas the dynamic programming one took only an order of 2000-- a huge difference.
If I ran it again, I might see a slightly smaller difference.
I might even see a considerably larger difference.
I've run this on some examples where the recursive search depth first took a million, and got through the dynamic programming in 50, 60.
But what you can see is there's a huge improvement in going from one to the other.
Dynamic programming was invented in the 1950s by someone named Richard Bellman.
Many a student has wasted a lot of time trying to understand why it's called dynamic programming.
And you or I could invent lots of theories.
Relatively recently, I found out why it was called dynamic programming, and this is a quote from Bellman.
"It was an attempt to hide what I was doing from government sponsors.
The fact that I was really doing mathematics was something not even a congressman could object to."
So he was doing this thing that was pretty evil, which was mathematics, which is what he thought this was-- the math of dynamic programming.
And he just didn't want to admit it.
So he made up a name out of nothing, and it fooled the government, and he got to do it.
Now why do I teach you dynamic programming?
And we're going to talk a little bit more about it, the next lecture.
It's because it is one of the most important algorithms that we know today.
It's used over and over again to provide practical, efficient solutions to optimization problems that on their surface appear intractable.
They appear exponential.
It says there is no good way to solve it.
In fact, if it has certain kinds of properties, it will always be amenable to solutions by dynamic programming, which will most of the time-- and I'll come back to the most of the time-- end up taking an exponential problem, and solving it really quickly, as we did here.
I could have made this graph enormous, and dynamic programming would have given us a very fast solution to it.
So when can we use dynamic programming?
Not all the time.
We can use it on problems that exhibit two properties.
The problem must have optimal substructure.
What this means is that you can find a globally optimal solution by combining locally optimal solutions.
So we can again see that with our graph problem, that we can combine the solutions from nodes at a distance from the root node to get the solution of getting there from the root node.
If I know I can get from A to B, and I can find the optimal solution from B to C, then I can use that to find the optimal solution from A to C. So it has optimal substructure.
The other thing it has to have is overlapping sub-problems.
And that's the thing I emphasized earlier-- that finding the optimal solution involves finding optimal solution to the same sub-problem multiple times.
Otherwise, we could build this memo, but we'd never successfully look up anything in it.
And so the algorithm would give us the right answer, but we'd get no speedup,.
So it's this property that we need to know that we'll get the correct answer-- that when we combine the local solutions, we'll get the right global solution.
It's this property that gives us an indication of how much of a speedup we can expect to achieve.
How many problems will we not have to solve, because we can look up the solution?
We'll come back to this.
And we'll see how it applies to another problem that you've already looked at say the knapsack problem, to give us a fast solution to that, so that if you want a answer, go back to a previous problem set, and take the full database of classes, you'll be able to solve it quickly using dynamic programming.
OK, see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We ended up the last lecture talking about dynamic programming.
And we'll spend all of today on that topic.
We showed how it could be used to provide a practical solution to the shortest path problem, a solution that would allow us to solve what was, in principle, a complex problem quickly.
And we talked about the properties that make it possible to solve that problem and, in general, the properties we need to make dynamic programming applicable.
So let's return to that topic right now.
What we looked at is we looked at two properties.
The first was optimal substructure.
And you recall what this meant is that we can construct a globally optimal solution by combining solutions to local subproblems.
So we've looked at a lot of problems already this term that have optimal substructure-- merge sort, for example, which exploits the fact that a list can be sorted by first sorting components of it and then combining those.
So merge sort exhibited optimal substructure.
The second property we looked at was overlapping subproblems.
And what this meant was that in the course of running our algorithm, we would end up solving the same problem more than once.
And that is what gave us the power of using memoization, to use table lookup instead of re-computing the solution.
Notice that merge_sort does not have overlapping subproblems.
There's no reason that we would expect, if we're sorting a list, that we would ever encounter the same sublist twice.
And so in fact, we cannot use dynamic programming to solve sorting.
Because it has one of these properties, but not both.
So how about shortest path?
Does it have these properties?
Well, let's first think about optimal substructure.
You might begin by asking the question if I knew the shortest path from A to B, and I knew the shortest path from B to C, can I necessarily combine those to get the shortest path from A to C?
No, exactly right.
Because for example, maybe there's a direct link from A to C. So we can't combine them that way.
So that gives us the question-- Oh, pardon?
[INAUDIBLE]
--Almost got it there.
So that gives us the question, what is the optimal substructure here?
Well, there is something we do know about shortest path, that if I have the shortest path from one node to another.
And I take any sub path of that, I will have-- so let's say I have a path that goes from-- let's say I know that the shortest path from A to C is A to B to D to E to C. If this is the shortest path from A to C, I know that this is the shortest path from B to E. Right?
Because otherwise, I wouldn't have bothered going through D. If there were, for example, a link directly from B to E, then the shortest path from A to C would not have included this.
So that's the optimal substructure I have, and that's the substructure that I exploited last time in showing you the dynamic programming solution to the shortest path problem.
You also recall that we looked at the shortest path.
We'd found overlapping subproblems.
Because I would end up solving the same intermediate problem multiple times, figuring out how to get from one node to another.
So indeed, we saw that the shortest path problem exhibited both of these properties and therefore, was amenable to solution by the dynamic programming.
And in fact, we ran it and we saw that it ran quite quickly.
It's not immediately obvious, but the 0-1 knapsack problem also exhibits both of these properties.
You'll recall a few lectures ago, and also in a problem set not so long ago, we looked at a recursive solution to the 0-1 knapsack problem.
It's called "solve" in this code.
It uses backtracking to implement the decision tree.
if we look at the decision tree we had there-- I gave you a small example here of the 0-1 knapsack problem.
I've got A, B, C and D. I've got some values, and I've got some weights.
We started by saying at the beginning, we looked at this node here, where we had decided not to take anything .
We still had the list A, B, C, and D to consider.
The total value of our items was 0, and for our example, I think we had 5 pounds of material left, or 5 units of weight.
We then considered a decision.
What happens if we take A?
Well, if we take A, we have B, C, and D left to consider.
We now look at this node and say, all right, our value is 6 -- the value of A.
But we only have 2 pounds left.
We then considered going depth first, left first in this case.
This is the depth- first search.
We've looked at that before.
All right, let's consider the next branch.
Well the next branch we'll consider C. Well we discover-- no we'd consider B rather.
We can't take B, because B weighs 3.
So this is a branch that we can't get to.
We can elect the other decision not to take B, in which case we still have C and D left to consider.
And still 6 and 2.
We then proceed and said right, well can we take C?
Yes we can.
And that leaves us with a value of 14 and no weight left.
So since we have no available weights, we might as well stop considering things.
So this is a terminal node.
We then back up, and said all right, let's consider the decision where we don't take C. Can we take D?
And the answer is no because it weighs too much.
And then we're done.
And so we see on this side of the tree, we've explored various possibilities.
And this is our best option so far.
And we can walk through and now we consider well, what if we don't take A?
Well then we have these decisions to make.
And eventually, we can look at the whole thing and make a decision.
By the way, I noticed this morning that there was an error on your handout.
I chose not to correct it in the slide so that you would notice it was in your handout as well.
What should node 18 be?
What's the error here?
Pardon?
[INAUDIBLE]
Right.
That should be the empty one.
Right?
Because we considered everything.
And in fact, typically all of your bottom nodes should either have 0 weights or no items left to consider.
Not very deep, just a typo but has the advantage that by looking for it, you can see whether you understand how we're constructing these decision trees.
So this is a very common way to deal with an optimization problem.
You consider all combinations of decisions in a systematic way.
And when you're done, you choose the best one.
So this is using depth-first search and backtracking.
And this is essentially what you did in your problem set.
Now what you saw in your problem set is that if the set of items is even moderately large, this algorithm can take a very long time to run.
How long?
Well, at each level of the tree we're deciding whether to keep or not keep one item.
So what's the maximum depth of the tree?
How many levels can this tree go?
If we have n items-- Pardon?
[INAUDIBLE]
n levels-- exactly.
So if we have n items, it can go n levels-- Good grab.
All right, how many nodes do we have at each level?
Well, at level 0 we have only 1 node.
But what's the down at the bottom?
Let's say we have an enormous amount of weight so that in fact we never run out of weight in our knapsack.
We'll have quite a broad tree, right?
How many nodes at level 2?
At each level, right?
Down here we have up to 4 nodes, up to 8 nodes, up to 16 nodes.
So if we had say, 39, 40 items, at level 39 we'd have 2 to the 39th nodes-- pretty darn big tree.
All right?
Not very deep, but very bushy.
So we know that for any reasonable number of items and a reasonable amount of weight, this is just not going to work.
And that's why when you wrote your problem set, and you were trying to solve the optimal set of courses to take, you could only run it on a small subset of the courses at MIT.
Because if we gave you everything-- and we gave you a lot for a test-- you noticed it effectively just wouldn't finish.
So now we have to ask the question well, can we use dynamic programming to solve this problem?
And in particular, that boils down to the question of does this solution exhibit optimal substructure and overlapping subproblems?
Well, optimal substructure is easily visible in both the decision tree and the code.
Right?
Each parent node combines the solution-- We can look at the code since you have the subtree in your handout-- or the decision tree in your handout.
And the key place in the code is where we combine decisions from lower down in the tree as we go up.
So at each parent node, we select the better of the two children.
Right?
If the left child is better than the right child, we select the left.
Otherwise, we select the right.
And we just percolate up the tree.
So there's clearly optimal substructure here, that we can solve the higher nodes with solutions to the lower nodes.
And we see that again in the code where it says choose better branch.
It's less obvious to answer the question whether or not there are overlapping subproblems.
If we go back and look at the tree-- At first glance, it may appear that there are no overlapping subproblems.
You'll notice that at each node we're solving a different problem.
Right?
The problem is described in some sense by this fortuple.
But if I've already taken A and C, and I have D left to consider, what should I do?
Where up here I've taken A, and I have B, C, and D left to consider, what should I do?
And by design of the decision tree, each node is different.
Right?
We're not considering the same thing over and over again.
So you might look at it and say, well, we're out of luck.
There are no overlapping subproblems.
Not true.
So let's think hard for a second and ask in what sense are there overlapping subproblems?
When are we potentially, at least, solving the same problem?
Well what is the problem we're actually solving?
We're solving the problem that can in some sense be stated as follows- at each node, we're saying what should we do about the nodes left to consider?
So we're solving a problem given a set of items-- what items we have at a node.
Given an available weight, which items should we take?
What's missing from this?
It says nothing about the items you've already taken.
In order to decide what to take next, we need to know how much weight we have available.
But we don't need to know why we have that much weight available, which items we have previously decided to take.
That part of the for tuple turns out not to be relevant to the problem I still have to solve.
Why is that important?
Because there may be many different sets of items I could take.
It would add up to the same total weight.
Therefore, leaving me the same problem to solve.
And that's the key observation that tells us we can use dynamic programming to solve the 0-1 knapsack problem.
Does that-- I should ask the positive question-- does that make sense to anybody?
Raise your hand if it makes sense-- a small number of hands.
Raise your hands if it doesn't make sense.
OK. Can any of you to whom it doesn't make sense formulate a question?
No, it's hard
Total weight, or the same total value?
The same total weight.
Because-- the question is are there many sets of items with the same total value or the same total weight?
There might be many with the same total value, but that doesn't really matter.
Because in choosing what to do next, as I go down the tree, I don't care what the value is above the tree.
Because I'll still try and find the best solution I can to the remaining subproblem.
If a value above is a million, or the value above is 3, it doesn't matter what I should do next
Which makes sense.
But if you use different things to get that weight, wouldn't you have a different set of items to choose from?
Right.
So the question is it makes sense if there are many different sets of items that would have the same total weight, but what I'm going to-- but, wouldn't I have different items to choose from?
Well you'll notice that I do have to look at what items I have left.
But as I go through-- let's assume here that I have a list of items to take.
As I go through from the front, say, I'll label each node of the tree.
Each item will have either a 0 or a 1, depending upon whether I decided to take it or not.
And then what I have to consider is the remaining items.
And let's say the values of the first two items-- the first four items-- were, well, let's say they were all one, just to make life simple.
Well that'll be confusing, since I'm using that for taken or not taken.
Let's say their values were all 2.
When I get here, and I'm deciding what to do with these items, I might-- so this tells me that I've used up 4 pounds of my allotment.
Right?
But if I had done this one, I would also use up 4 pounds of my allotment.
And so, which of these I take or don't take is independent of how I got here.
I do have to keep track of which items are still available to me.
But I don't care how I've used up those 4 pounds.
And obviously, there are a lot of ways to choose two that will use up 4 pounds here, right?
But the solution to this part of the problem will be independent of which two I took.
That make sense now?
Therefore, as long as there is a prefix of my list of items, such that multiple things add up to the same weight, I will have overlapping subproblems.
Now you could imagine a situation in which no combination added to the same weight, in which case dynamic programming wouldn't buy me anything.
It would still find the right answer, but it wouldn't speed anything up.
But in practice, for most 0-1 knapsack problems-- they may not be this simple-- but you could expect-- and we'll see this complexity later-- that as long as your possible weights are being chosen from a relatively small set, you'll have many things add up to the same thing-- particularly as you get further down this list and have a lot longer prefix to consider.
So indeed I do have overlapping subproblems.
That answer the question?
All right, now.
People feel better about what's going on?
OK. Maybe looking at some code will make it even clearer, and maybe not.
Anyway, I do appreciate the questions.
Since it's sometimes hard for me to appreciate what's coming across and what's not coming across.
You should get rewarded for good questions, as well as good answers.
All right, so now let's look at a dynamic programming solution.
I've just taken the previously-- the example we looked at before was solve-- made it fast solve.
And I've added a parameter of memo.
This is the same kind of trick we used last time for our shortest path problem.
I should point out that I'm using-- this calls to mind a dirty, little secret of Python, one I have previously been hiding because it's so ugly I just hate to talk about it.
But I figured at some point, honesty is the right policy.
What I'd like is the first time I call fast solve, I shouldn't have to know that there's even a memo.
Fast solve should have the same interface as solve.
The items to consider and the available weight, and that's all you should need to know.
Because the memo is part of the implementation, not part of the specification of solving the 0-1 knapsack problem.
We've made that argument several times, earlier in the semester.
And so you might think that what I should therefore do is to initialize the memo to, say the empty dictionary.
I'm going to use dictionaries for memos, as before.
And then just check and if it's the empty one, that means it's the first-- I can know whether it's the-- it'll work fine for the first call.
Well it doesn't work fine.
And here's the dirty, little secret.
In Python, when you have a parameter with a default value, that default value is evaluated once.
So the way the Python system works is it will process all of the def statements and evaluate the right-hand side here once.
And then every time the function is called, without this optional argument, it will use the value it found when it evaluated the def statement.
Now if this value is immutable, it doesn't matter.
None will be none forever.
28 will be 28 forever.
3.7 will be 3.7 forever.
But if this is a mutable value, for example a dict, then what will happen is it will create an object which will be initially an empty dictionary.
But then every time fast solve is called without this argument, it will access the same object.
So if I call fast solve to solve one problem, and in the course of that, it builds up a big dictionary.
And then I call it again to solve another problem, instead of starting with the empty dictionary, it will start with the same object it started with the first time, which is by now a dictionary filled with values.
And so I will get the wrong answer.
This is a subtlety, and it's a common kind of bug in Python.
And I confess to having been bitten by it very recently, as in yesterday, myself.
So it's worth remembering.
And there's a simple workaround, which is the default value is the immutable value.
I chose none.
And what I say when I enter it is if the memo is none, it means it's been invoked.
And now I'll initialize it to the empty dictionary.
And now every time it will get a new one, because we know what this statement says is allocate a new object of type dict, and initialize it to empty.
This will happen dynamically when the thing is invoked, rather than statically at the time Python processes the diffs.
It's a silly little problem.
I hate to bring it up.
I don't think it should work that way, but it does work that way.
So we're stuck.
Yeah.
[INAUDIBLE]
I'm sorry.
You've have to speak more loudly.
[INAUDIBLE]
Well, why doesn't my first call-- So the question is when I go to test it, say, why don't I start by calling it with fast solve in an empty dictionary?
And it's because, as we'll see when I get there, I want solve and fast solve, in this case, to have the same interface.
And in fact, you can say I want them to have the same specification.
Because in fact, imagine that you wrote a program that called solve many times.
And it was slow, and then you took 6.00-- So, I know why it's slow, because I've used a stupid solve.
Let me use dynamic programming.
I want to then be able to not name the new one fast solve, but replace the old solve by the new solve.
And have all the programs that you solve still work.
That will not happen if I insist that the new solve has an extra argument.
Now I could put what's called a wrapper in and call solve, and then have it call fast solve with the extra argument.
And that would work too.
That would be an equally good solution to address this problem.
But does that make sense?
So either one would work.
I chose this one but what wouldn't work in a practical engineering sense is to change the specification of solve.
All right?
So it's a silly little thing, but since people do get bitten by it, I figure I should tell you about it.
And I guess while I'm on the subject of silly little things, I'll tell you one more thing before we go back to this algorithm.
Down here in the place where I'm testing it, you'll note that I've imported something called sys, and then said sys.setrecursionlimit to 2000.
In Python, there's a maximum depth of recursion.
When this is exceeded, it raises an exception-- depth exceeded.
As it happens, the default value for that is some tiny number-- I forget what it is-- such that when I run this on an interesting size example, it crashes that exception.
So all I'm doing here is saying no, I know what I'm doing.
It's OK to recurse to a greater depth.
And I've set it to 2000 here, which is plenty for what I need.
I could set it to 20,000.
I could set it to 50,000.
Again, if you're writing a serious program, you'll probably find that the default value for the depth of recursion is not enough, and you might want to reset it.
I don't expect you to remember how to do this.
I expect you to remember that it's possible.
And if you need to do it, you can Google "recursion limit Python" and it will tell you how to change it.
But again, I've found in testing this, the default was too small.
Ok. Let's go look at the code now for fast solve.
It's got the items to consider, what's available, the available weight, and a memo.
And I'm going to keep track of the number of calls.
Just for pedagogical reasons, we'll want to review that later.
OK, so I'll initialize the memo if it's the first time through.
And then the first thing I'll do is I'm going to code what I have available essentially the way I've coded it here-- so what items are remaining by using an index.
So my items are going to be a set of lists.
And I'm going to just sort of keep track of where I am in the list-- just march through exactly the way I'd marched through here.
And so if the list of things to consider, the length of it-- because every time the length is the same, it will be the same list.
Since I'm systematically examining a prefix.
I'm not shuffling the list each time.
So the length can be used to tell me what I've already looked at.
So if I've looked at that sublist before, with this available weight, it will be in the memo.
And I'll just look up the solution-- result equals memo of [UNINTELLIGIBLE] to consider avail.
So those will be my keys for my dictionary.
The key will be a pair of essentially which items I have left to consider, which is coded by a number, and the amount of weight, another number.
Or if there's nothing left to consider, or there's no weight available, then I'll return 0 and the empty tuple-- no value, didn't take anything.
Otherwise, I'm now in the interesting case.
If to consider subzero, the first one here, if the weight of that is greater than what I have available, then I know I can't take it.
Right?
So I'll just lop it off and call fast solve recursively with the remaining list-- same old value of avail and the same old memo.
Otherwise, well now I have an option of taking the first element in the list.
I'll set the item to that element, and I'll consider taking it.
So I'll call fast solve with to consider without the first item.
But the amount of weight will be available now is what was available before minus the weight of that item.
This is my left branch, if you will, where I decide to take the item.
And so now I'm solving a smaller problem.
The list is smaller, and I have less weight.
The next thing I'll do is consider not taking the item.
So I'll call fast solve again with the remainder of the list.
But avail and memo are not changed.
That's the right branch.
And you'll remember our decision tree the right branch avail and weight were never changed.
Because I elected not to take that item.
Then I'll just choose the better of the two alternatives, as before.
And when I'm done, I'll update the memo with the solution to the problem I just solved.
All right?
So it's exactly the same as the recursive solution you looked at a few weeks ago.
Except I've added this notion of a memo to keep track of what we've already done.
Well let's see how well it works.
So I've got this test program.
Just so things are repeatable, I'd set the C to 0 -- doesn't matter what I set it to, this just says instead of getting a random value each time, I'll get the same C so I'll get the same sequence of pseudo-random numbers-- makes it easier to debug.
I'll have this global variable num calls.
I'm arbitrarily setting the capacity to eight times the maximum weight.
So I'm saying the maximum value of an item is 10.
The maximum weight is 10.
This just says runs slowly is false, as in don't run the slow version.
Run only the fast version.
We'll come back to that.
I set the capacity of the Knapsack to 8 times the max weight.
And then for the number of items-- and I'm just going to go through a different number of items, 4, 8 16, up to 1024-- I'm going to call build many items.
Again, we've seen this program before.
That just takes a number of items, a maximum value, and a maximum weight, and builds some set of items, choosing the values and weights at random from the ones we've offered it.
This saves me.
I wasn't going to type a set of 1024 items, I guarantee you that.
If it runs slowly, then I'm going to test it on both fast solve and solve.
Notice that these are the names of the functions, not invocations of the functions.
Because remember functions, like everything else in Python, are just objects.
Otherwise, my tests will be only the tuple fast solve.
And then for each function in tests, I'm going to set the number of calls to 0.
I'm going to load up a timer, just because I'm curious how long it takes.
And then I'll call the func.
Notice again, getting back to what we talked about before, both fast solve and solve get called with the same arguments.
I could not have done this trick if fast solve had required an extra parameter.
And then I'll just see how well it did.
So let's start with this equal to true.
So in that case we'll test both of them.
And let's see how we do.
So it's chugging along.
And we see that if I have 4 items, fast solve made 29 calls, took this much time.
The good news, by the way, you'll notice that fast solve and solve gave me the same value each time.
We would hope so, right?
Because they're both supposed to find an optimal solution.
They could find different sets of items.
That would be OK.
But they better add up to the same value.
But we seem to be stuck here.
So you'll notice at 16, Solve made 131,000 calls, fast solve 3,500-- only took a third of a second.
But we seem to be kind of stuck at-- actually, fast solve only made 1,200.
At 32, fast solve made 3,500 calls.
But solve seems to be stuck.
Should this be a shocker that going from 16 to 32 made a big difference?
Well remember here we're talking about 2 to the 16 possibilities to investigate, which is not a huge number.
But now we're at 2 to the 32, which is quite a large number.
And so we could wait quite a long time for solve to actually finish.
And in fact, we're not going to wait that long.
We'll interrupt it.
So let's see how well fast solve works.
We just call that.
So we'll set this to false.
It says don't run slowly, i.e.
Don't call solve.
So not only did we not get stuck at 32 items way up here, we didn't get stuck at 1024 items.
Huge difference, right?
Instead of only being able to process a set of 16, we can process a set of over 1,000.
And in fact, I don't know how much higher we can go.
I didn't try anything bigger.
But even this-- big difference between getting stuck here and not getting stuck here.
And in fact, not only did not get stuck, it was only two seconds.
So let's ask ourselves the question-- how fast is this growing?
Not very fast-- more or less, what we see here-- and this is not something we can guarantee.
But in this case, what we see is as we double the number of items, the number of calls kind of doubles as well.
So we're actually growing pretty slowly.
Well, OK that's good.
But suppose we want to look at it a little bit more carefully and consider the question of what does govern the running time of fast solve.
Well we know that looking things up in the dict is constant time.
So that's OK.
So what's going to govern it is every time we have to add an element to the memo, it's going to slow us down.
Because you're gonna have to solve a problem.
Every time we can just look something up in the memo, we'll get our answer back instantaneously.
So if we think about the running time, it's going to be related to how many different key value pairs might we have to compute for the memo.
Well what governs that?
So we know that the keys in the memo are pairs of to consider and avail.
And so we know that the number of possible different keys will be the number of possible values in to consider, multiplied by the number of possible values of avail.
Right?
How many possible values of to consider are there?
What governs that?
That's the easy question.
Number of items, right?
Because I'm just marching across this list, chopping off one item at a time.
At most I can chop off n items, if the list is of length n. Avail is more complicated.
What does avail depend upon?
It depends upon, well, the initial available weight.
If the initial available weight is 0, well then it's gonna end really quickly.
If it's really big, I might have to churn longer.
But it also depends-- and this is the key thing-- on the number of different weights, that sets of items can add up to.
Because you'll recall the whole secret here was that we said that different sets of items could actually have the same total weight.
And that's why we had this business of overlapping subproblems.
Now here, since I only had 10 possible weights, that tells me that I can't have very many different combinations.
Right?
That any set of say, one item has to be 0 through 10, or 0 through 9, I forget which it was.
But either way, that means that alI only have 10 different possible sums for sets of length 1.
Whereas if my weights ranged over 100, I would have 100 different possibilities for sets of length 1.
So that's an important factor in governing it.
So it's a complicated situation, but just to see what happens, try and remember how long this took for 1024.
It took 2.2 seconds.
I'm going to allow 20 different weights.
And let's see what happens.
It took roughly twice as long.
So that's sort of what we would have expected.
So again, we're going to see the running time is related to that.
So we can see that it's actually a fairly complicated thing.
Suppose I did something really nasty and went up here where I built my items-- so here you'll see, when I built my items I chose the weight between 1 and max weight.
And these were all integers-- ints.
Suppose instead of that, I do this.
So here after I choose my value between 1 and 10, I'm going to multiply it by some random number between 0 and 1.
We've all seen random.random.
So now how many possible different weights am I going to have?
A huge number, right?
The number of random numbers, real floats between 0 and 1 is close to infinite, right?
And so in fact, now I'm going to see that with reasonable probability, every single item will have a different weight.
And so, things adding up to the same could still happen, even if they're all different.
But it's not very probable.
So let's see what happens when I run it now.
It's slowing down pretty quickly.
Pretty much where the other one stopped right?
Remember we got through 16 with Slowsolve but not through 32?
Because effectively, I've now reduced the fast solve to the same as solve.
It will never find anything in the memo, so it's doing exactly the same amount of work that solve did.
And it will run very slowly.
OK, life is hard.
But this is what you'd expect, because deep down we know the 0-1 knapsack problem is exponential, inherently exponential.
And while I think dynamic programming is kind of miraculous how well it usually works, it's not miraculous in the liturgical sense, right?
It's not actually performing a miracle in solving an exponential problem in linear time.
It can in all things go bad.
The exponential, and that's what it is here.
This kind of algorithm is called pseudo-polynomial.
It kind of runs in polynomial time, but not when things are really bad.
And I won't go into the details of pseudo-polynomial, but if any of you are in Course 6, you'll hear about it in 6.006 and 6.046.
OK, we can adjourn.
I'll see you all on Thursday.
And we're not going to wait for this to finish, because it won't.
III The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
We'll back up and start again since I just turned on my microphone.
I started with the observation that for most of recorded history people thought qualitatively, not quantitatively.
They didn't know what statistics were.
They must have had some intuitive sense, for example, that if you're old, you're more likely to have bad hearing than if you're young.
If you're old you're more likely to die than if you're young.
Things like that.
But they were just anecdotes, and they had no careful way to go from statements about individuals to statements about populations or expectations.
This changed in about the middle of the 17th century, changed fairly dramatically, when an Englishman named John Graunt published something called The Natural and Political Observations Made Upon the Bills of Mortality.
This was the first work in recorded history that actually used statistics.
And what he did is he looked at the fairly comprehensive statistics of when people died in the city of London.
And from that, attempted to produce a model that could be used to predict the spread of the plague.
And it turns out, he was pretty good at it.
And that just changed the way people started to think.
And since that time, people have used statistics both to inform and, unfortunately, to mislead.
Some have willfully used statistics to mislead, others have merely been incompetent.
And that gets me to what I want to talk about today, which is statistics.
This is often attributed to Mark Twain but, in fact, he copied it from Benjamin Disraeli, who said, "There are three kinds of lies: lies, damned lies, and statistics."
And there is, unfortunately, a lot of truth in that.
More recently, in the '50s Darrell Huff wrote a wonderful book, I recommend it, called How to Lie with Statistics.
Now here's a quote from the book, "If you can't prove what you want to prove, demonstrate something else and pretend that they're the same thing.
In the daze that follows the collision of statistics with the human mind, hardly anyone will notice the difference."
Alas, that seems to be true.
So what I want to do today is talk about a few ways in which one can be fooled into drawing inappropriate conclusions from statistical data.
Now I trust that you will use this information only for good.
And it will make you a better consumer and purveyor of data rather than a better liar, but that's up to you.
All right.
So let's start with the first way out thing I want to talk about.
And it's important to always remember that no matter how good your statistics are, statistical measures don't tell the whole story.
We've seen examples of this already earlier in the term.
There are an enormous set of statistics that can be extracted from a data set.
And by carefully picking and choosing among them, it's possible to convey almost any impression you want about the same data set.
The best antidote, of course, is to look at the data itself.
In 1973, the statistician John Anscombe published a paper with this set of data.
Four different examples where he gave values for x and y, nothing very sophisticated.
The interesting thing is that in many ways, the statistics for these four data sets are very similar.
They have the same mean, the same median, the same variance for x and y both, the same correlation between x and y.
And even if we use linear regression to get a fit, we get very similar things.
So let's look at it.
I actually wrote some code to do that.
So this just reads in the data set that we just had up, and then plots some things with it.
And so let's look at it.
So what you can see is, you have these in your hand out, we have four graphs, four plots.
And I won't show you them all because they're all are identical.
What they all give me is a mean of 7.5 and a whole bunch of things, same to 17 decimal places or so, the same median and the same linear fit, y equals 0.5x plus 3.
So I might write a paper in which I would tell you that well, from a statistical sense, these data sets are all the same.
Every statistical result I ran said these things are indistinguishable.
Well, is it true that these data sets are indistinguishable?
Well, one way to look at it is to actually plot the points.
So we can run it with true, which says in addition to plotting the data, put points on them in addition to plotting the statistics.
And now we see something fairly dramatic.
So for example, figure (2) and figure (1) don't look a lot like each other, when we actually look at the data.
Figure (3) looks quite different and figure (4) is remarkably different.
Compare figure (4) to, say, figure (1).
Again, these are all in your hand out.
So the moral is pretty simple here, and it's one we looked at before, which is, don't ever ignore the data.
Don't just look at statistics about the data, try and find a way to look at the data itself.
Of course, it's easy enough to look at the data, but how do you look at it?
And the next thing to remember is that pictures can be deceiving.
There can be no doubt about the utility of graphics for quickly conveying information.
However, when used carelessly or maliciously, a plot can be highly misleading.
So let's look-- go back to the PowerPoint, and look at this plot of housing prices in the Midwest.
So we've got a plot here, and we've got years 2006, 2007 and then in 2008 and 9, we have quarters.
You may remember that there was an event in the housing market in 2008 precipitating the global financial crisis.
And if we look at this, what impression do we get about housing prices in the mid west during this period?
Well I would get the impression that they are remarkably stable.
You publish this and you say, OK, look they really haven't changed very much.
Maybe they've gone down a little, but nothing very serious.
If we compare that to this plot, which is exactly the same data, and now I ask you about housing prices in the Midwest.
Well, what you might tell me is it's-- they're remarkably unstable.
And in fact, there was clearly some sort of horrible event here.
Exactly the same data, two plots both of which are truthful, but which give a radically different impression about what happened.
The chart on the right was designed-- on the right in your handout-- was designed to show that they were highly unstable.
So what's the difference?
What trick did I use to produce these two plots?
Yeah?
In the more stable one, you used the logarithmic scale.
And then here only has selected numbers so--
So that was certainly one trick that I performed.
That in the first chart, I plotted the y-axis logarithmically, which always make things look like they are changing less than if I plot it linearly.
And in this chart, I used a linear plot.
Go ahead
You see a much narrower scale on the second plot.
So that the magnitude of the difference is much less compared to the magnitude of the whole graph of the scale
So that's the other thing I did is, if you look at it, I sort of cheated.
I had full years here and then I went to quarters.
So in part of my chart, the resolution on the x is pretty wide is a whole year, and then part of it's on a quarter.
And, not surprisingly, since we know that housing prices change seasonally, they're different in the spring than in the winter, once I start plotting quarters, even if there had not been a crash, it would have looked much less stable in the out years, because I changed the resolution on the x-axis.
I didn't lie.
You can tell reading the legend that I did that.
But I sure could have fooled a lot of people with these charts.
Here's another nice example of statistics.
So this plots-- this is from a paper by two professors, I think from Yale, shows what you can do if you're in the Ivy league, that plots initials against GPA for students at Yale.
And so you can see that if your first name starts with the letter A, I think it was first initials, your GPA is considerably higher than if it starts with C or D. And if your parents weren't nice enough to give you an A name, you could hope they at least gave you a B name.
And you certainly don't want them to give you a C or a D name.
So if you're Charlene or David you could have a real problem.
I have to say my first child was named David.
His GPA might have been somewhere in there.
All right -- clearly it matters.
Well, what tricks did I perform here?
There is very little disparity in x.
You are going from 3.32 to--
Right.
So what I did here is I made the range on the y-axis very small, ranging from 3.32 to 3.38, not a big difference.
However, because that's the whole thing, it looks like it's a big difference.
You will often see this when you, say, look at things in newspapers where, in fact, someone has manipulated one of the axes to make things look big or small.
If I had ranged this from a GPA of 0.0 to a GPA of 4.0, the highest GPA at Yale, this difference would have looked very tiny.
But I didn't -- actually it wasn't I. I actually copied this from their paper.
This was the way they presented it in their paper.
Because they were trying to argue in the paper that your name had a big influence on your life.
And they used many statistics including your grades.
And so they actually formatted it kind of like this to try and give you this impression.
Now later we'll see another statistical sin they committed in this paper, which, basically, was designed to show that in your name was destiny.
And they had many other things.
If you're a baseball player, and your name starts with K, you're more likely to strike out.
Because K is the symbol for strikeouts in a baseball score book, a lot of implausible things.
All right.
Moving right along, probably the most serious and common statistical error is the one known as Garbage In Garbage Out.
And it's so common that people typically refer to it by its acronym GIGO -- Garbage In Garbage Out.
A classic example of this occurred and, we could look at more recent examples, but I don't want to offend any of you.
In 1840, United states census showed that insanity among free blacks and mulattoes was roughly 10 times more common than insanity among enslaved blacks or mulattoes.
And the conclusion from this was obvious, I guess.
US Senator, former Vice President, and later, Secretary of State John C. Calhoun concluded from the census and I quote, "The data on sanity revealed In this census is unimpeachable.
From it, our nation must conclude that the abolition of slavery would be to the African, a curse."
Because after all, if you freed them from slavery, they would all go insane.
That's what the statistics reported, he said.
Now never mind it was soon clear that in fact that census was riddled with errors.
And John Quincy Adams, a former Vice President and Massachusetts resident responded to Calhoun and said, no that's a ridiculous conclusion.
The census is full of errors.
Calhoun, being a very patient person, explained to Adams the following, "There were so many errors, that they balanced one another out, and led to the same conclusion just as much as if they were all correct."
There were just enough errors that you could be OK. Well, what was he relying on?
What should he have said if he wanted to make this statement more mathematically precise?
What he was basically implying is that the measurement errors are unbiased and independent of each other and, therefore, almost identically distributed on either side of the mean.
I see a typo, I might as well fix it.
Might as well make it big enough to fix it.
That's interesting.
If he had made this much more precise statement, then you could have had an meaningful discussion-- assuming it was possible to have a meaningful discussion with John Calhoun, which is perhaps dubious-- about whether or not, in fact, the errors are independent.
Because if they're not, if, for example, they represent bias in the people compiling the data, then you cannot rely upon statistical methods to say that they'll balance each other out.
You remember way back in Gauss' time, Gauss talked about this when he talked about the normal distribution.
And said, well, if we take these astronomical measurements and we assume our errors are independent and normally distributed, then we can look at the mean and assume that that's close to the truth.
Well, those are important assumptions which in this case turned out to be not correct.
And, in fact, it was later shown that the errors did not balance each other out nicely.
And in fact, today you can say that no statistical conclusion can be drawn from that.
On the other hand recently, the US National Research Council, perhaps the most prestigious academic organization in the United states, published a ranking of all universities in the country.
And it was later shown that it was full of garbagy input.
And they did extensive statistical analysis and published it on data that turned out to be just wrong.
And it was very embarrassing.
Now the good news is MIT came out near the top of this analysis.
And the bad news is we can't conclude that it actually should have been near the top because who knows about the quality of the data, but kind of embarrassing.
All right.
Moving right along, another very common way to lie with statistics is to exploit what's called the cum hoc ergo propter hoc fallacy.
So anyone here study Latin?
Bunch of techy-- Oh OK. Well what does it mean?
With this therefore, because of this?
Boy, your Latin is good.
Either that or you just know statistics.
But I have to say that was the most fluent translation I've had it all the years I've asked this question.
I hit the relay man-- the relay woman on the throw.
All right.
Yes, with this, therefore because of this.
I don't know why but statisticians, like physicians and attorneys, like to show off by phrasing things in Latin.
So for example, it is a statistical fact that college students, including MIT students, who regularly attend lectures have higher GPAs than students who attend lectures only sporadically.
So that would tell us that those of you in the room are likely to have a higher GPA than the various students in 6.00 who are not in this room.
I hope it's true.
Now if you're a professor who gives these lectures, what you want to believe it's because the lectures are so incredibly informative, that we make the students who come much smarter and, therefore, they do better.
And so we'd like to assume causality.
Because I give beautiful lectures and you choose to come, you will get a better grade in 6.00.
Well, yes, there's a correlation.
It's unquestionably true, but causation is hard to jump to.
For example, maybe it's the point that students who bother to come to lecture also bother to do the problem sets, and are just more conscientious.
And whether they came to lecture or not, the fact that they're more conscientious would give them better GPAs.
There's no way I know to separate those two things, other than doing a controlled experiment, right?
Maybe kicking half of you out of lecture every day and just see how it goes.
But it's dangerous but again, you can read things like the faculty newsletter, which will talk about how important it is to come the lecture because you'll do better.
Because whoever wrote that article for the faculty newsletter didn't understand this-- this fallacy, or was just thinking wishfully.
Another nice example, one that was in the news not too long ago, has to do with the flu.
This was the cases of flu in New York State in recent years.
And you'll notice that there was a peak in 2009, and that was the famous swine flu epidemic, which I'm sure you all remember.
Now, if you look at this carefully, or even not too carefully, you'll notice a correlation between when schools are in session and when the flu occurs.
That in fact, during those months when schools are in session, there are more cases of flu than in the months when school is not in session, high schools, colleges, whatever.
Quite a strong correlation, in fact.
This led many to conclude that going to school is an important causative factor in getting the flu.
And so maybe you shouldn't have come to the lectures because you would have just gotten the flu by doing so.
And in fact because of this, you had many parents not sending their kids to school during the swine flu epidemic.
And in fact, you had many schools closing in some communities because of the swine flu epidemic.
Well, let's think about it.
Just as you could use this correlation to conclude that going to school causes the swine flu, you could have also used it to prove that the flu causes you to go to school.
Because more people are in school when the flu season is at it's height.
And therefore, it's the growth of flu that causes people to go to school.
That's an equally valid statistical assumption from this data.
Kind of a weird thing but it's true, right?
Just as we could conclude that having a high GPA causes people to come to the lecture.
You look at your GPA every morning, and if it's high enough, you come to lecture, otherwise, you don't.
You could draw that conclusion from the data as well.
The issue here that you have to think about is whether or not there is what's called a lurking variable, some other variable that's related to the other two, and maybe that's the causative one.
So for example, a lurking variable here is that the school season coincides with or the non-school season, maybe I should say, coincides with the summer.
And in fact, if you study the flu virus in a lab, you will discover that it survives longer in cold weather than in hot and humid weather.
When it's cold and dry, the flu virus will survive for a longer time on a surface than it will when it's warm and humid.
And so in fact, maybe it's the weather, not the presence of schools, that causes the flu to be more virulent during certain times of the year.
In fact, it's probably likely true.
So there is a lurking variable that we have to consider, and maybe that's the causative factor.
Now, this can actually lead to some really bad decisions in the world.
I'm particularly interested in issues related to health care and public health.
In 2002, roughly 6 million American women were taking hormone replacement therapy, in the belief that this would substantially lower their risk of cardiovascular disease.
It was argued that women over a certain age or of a certain age, if you took extra hormones, they were less likely to have a heart attack.
This belief was supported by several published studies in highly reputable journals in which they showed a strong correlation between being on hormone replacement therapy and not having cardiovascular disease.
And this data had been around a while and, as I said, by 2002 in the US, roughly 6 million women were on this therapy.
Later that year, the Journal of the American Medical Society published an article asserting that in fact being on this therapy increased women's risk of cardiovascular disease.
It made you more likely to have a heart attack.
Well, how could this have happened?
After the new study came out, people went back and reanalyzed the old study and discovered that the women in that study who'd been on hormone replacement therapy were more likely than the other women in the group to have also better diet and be on a better exercise regimen.
In fact, they were women who were more health conscious.
So there were the lurking variables of diet and exercise and other things that were, in fact, probably the causative factors in better health, not the replacement therapy.
But there was this lurking variable that had not been discovered in the initial analysis of the data.
So what we saw is that taking hormone replacement therapy and improved cardiac health were coincident effects of a common cause, that is to say being health conscious.
Kind of a strange thing but a true and sad story.
All right.
Moving right along, another thing to be cautious of is non-response bias and related problem of a non-representative sample.
You'll probably recall that when I first started talking about statistics and the use of randomness, I said that all statistical techniques are based upon the assumption that by sampling a subset of a population, we can infer things about the population as a whole.
And that's true, typically, because if random sampling is used, you can make assumptions that the distribution of results from the random sample, if it's large enough, will be the same as a distribution of results from the whole population.
And that's why we typically want to sample randomly.
And so for all the simulations we looked at, we used random sampling to try and ensure that a small number of samples would give us something representative of the population.
And then we use statistical techniques to answer the question about how many random samples we needed.
But those techniques were only valid if the samples were indeed random.
Otherwise, you can analyze it to your heart's content and any conclusions you've drawn are likely to be fallacious.
Unfortunately, many studies, particularly in the social sciences, are based on what is often called a convenience sampling.
So for example, if you look at psychological-- psychology journals, you'll find that many psychological studies use populations of undergraduates for their studies.
Why did they do this?
Is it because they believe that undergraduates are representative of the population as a whole?
No.
It's because they're captive.
They have to agree to participate, right?
It's a convenience if you happen to be at a university to do your experiments on undergraduates.
And so they do that and then they say, well, the undergraduates are just like the population as a whole.
You may have observed that at least at this institution, the undergraduates are probably not representative of the population as a whole.
A well-known example of what you can do with this occurred during World War II.
Whenever an allied plane would return from a bombing run over Germany, the plane would be inspected to see where flak had hit it.
So the planes would fly over to drop bombs, the Germans would shoot flak at the planes to try and knock them out the air.
They'd come back to England, they'd get inspected, they'd say, well the flak hit this part of the plane more often than that part of the plane on average.
And so they would reinforce the skin of those parts of the plane where they expected the flak to hit, to try and make the plane less likely to be damaged in future runs, or the planes in general.
What's wrong with this?
Yeah?
They're not getting the planes that dropped?
Exactly.
What they're not sampling is the planes that never made it back from the bombing, ooh, that never made it back from the bombing runs because they weren't there to sample.
And in fact maybe it's the case that what they were doing was reinforcing those parts of the planes where it didn't matter if you got hit by flak because it wouldn't cause a plane to crash, and not reinforcing those parts of the planes that were most vulnerable to being damaged by flak.
They did a convenient sampling, they drew conclusions, and they probably did exactly the wrong thing in what they chose to reinforce in the airplanes.
This particular error is called non-response bias where you do some sort of survey, for example, and some people don't respond and, therefore, you ignore what they would have said.
It's perhaps something we see when we do the underground guide to Course 6.
In fact I should point out that it's now online.
And it would be good if each of you would go and rate this course, rate the lectures, rate the TAs, et cetera.
We actually do read them and it makes a difference in how we teach the course in subsequent terms.
But there's clearly a bias.
You know, maybe only the people who really feel strongly about the course, either positively or negatively, bother to fill out the survey.
And we draw the conclusions that there's a bimodal distribution, and nobody thinks it's kind of mediocre, because they don't bother responding.
Or maybe only the people who hate the course respond, and we think everybody hates the course.
Who knows.
It's a big problem.
We see it, it's a big problem today with telephone polls, where you get more convenient sampling and non-representative samples, where a lot of polls are done using telephones.
By law, these pollsters cannot call cell phones, so they only call land lines.
How many of you have a land line?
Let the record show, nobody.
How many of your parents have a land line?
Let the record show, pretty much everybody.
Well, that means your parents are more likely to get sampled than you when there's a poll of, say, who should be nominated for president.
And so any of these polls that are based on telephones will be biased.
And, unfortunately, their poll may just say, a telephone sample, and people may not realize the implication of that.
That whole part of the population is under sampled.
There are lots of examples of this.
All right.
Moving along, another problem we often see is data enhancement.
It's easy to read much more into data than it actually implies, especially when viewed out of context.
So on April 29, 2009, CNN reported that quote, "Mexican health officials suspect that the swine flu outbreak has caused more than 159 deaths and roughly 2,500 illnesses."
It was pretty scary stuff at the time, and people got all worried about the swine flu.
On the other hand, how many deaths a year do you think are attributable to the conventional seasonal flu in the US?
Anyone want to hazard a guess?
36,000.
So 36,000 people a year, on average, will die from the seasonal flu, which sort of puts in prospective that 159 deaths from the swine flu maybe shouldn't be so terrifying.
But again, people typically did not report both of those.
Another great statistic, and accurate, is that most auto accidents happen within 10 miles of home.
I'm sure many of you have heard that.
So what does that mean?
Almost nothing.
Most driving is done with 10 miles-- within 10 miles of home.
And besides that, what does home mean in this context?
What home means is the registration address of the car.
So if I were to choose to register my car in Alaska, does that mean I'm less likely to have an accident driving around MIT?
I don't think so.
Again, it's a kind of a meaningless.
Another aspect of this is people often extrapolate from data.
So we can look at an example of internet usage.
This is kind of a fun one too.
So what I've plotted here is the internet usage in the United states as a percentage of population.
And I plotted of this from-- starting at 1994.
And the green line, or actually, the blue line there are the points and the green line is a linear fit.
If you looked at my code, you'd see I was using polyfit with a 1 to get a line to fit, and you can see it's a pretty darn good fit.
So people actually looked at these things and used this to extrapolate internet usage going forward.
So we can do that.
Now, we'll run the same code with the extrapolation turned on.
And so figure (1) is the same figure (1) as before, same data, same fit.
And here's figure (2).
And you'll notice that as of last year about 115% of the US population was using the internet, probably not true.
It may be possible in sports to give 110%, but in statistics it's not.
Again, you see this all the time when people are doing these projections.
They'll say, fit some data, they extrapolate into the future without understanding why maybe that isn't a good thing to do.
We saw that by when we were modeling springs, right?
We could accurately project linearly until we exceeded the constant of elasticity at which point our linear model was totally broken.
So you always need to have some reason other than just fitting the data to believe that what you're doing makes actual sense.
All right.
The final one I want to talk about is what is typically called in the literature the Texas sharpshooter fallacy.
And this is a little bit tricky to understand sometimes.
Actually, is there anyone here from Texas?
Oh good, so no one will be offended by this.
Well imagine that you're driving down some country road in Texas and that you see a barn.
And that barn has six targets painted on it, and in the dead center of each target, you find a bullet hole.
So you're driving, your pretty impressed, and you stop.
And you see the owner of the barn and you say, you must be a damn good shot.
And he says, absolutely, I never miss.
At which point the farmer's wife walks out and says, that's right there ain't a man in the state of Texas who is more accurate with a paint gun.
What did he do?
He shot six bullets into the barn, and he was a terrible shot.
They were all over the place.
Then he went and painted a target around each of them.
And it looked like he was a great shot.
Now you might think that, well that's silly, no one would do that in practice.
But in fact, it happens all of the time in practice.
A classic of this genre appeared in the magazine New Scientist, in 2001.
And it reported that a research team led by John Eagles of the Royal Cornhill Hospital in Aberdeen, had discovered that and I quote, "Anorexic women are most likely to have been born in the spring or early summer between March and June.
In fact, there were more than-- there were more than 13%-- there were 13% more anorexics born on average in those months, and 30% more anorexics, on average, in June."
Now, let's look at this worrisome statistic.
Are any of you women here born in June?
All right.
Well, I won't ask about your health history.
But maybe you should be worried, or maybe not.
So let's look at how they did this study.
You may wonder why so many of these studies are all studies about women's health.
And then, perhaps, because they're all done by male doctors.
Anyway, the team studied 446 women who had been diagnosed as anorexic.
So if you divide that by 12 what you've discovered is that, on average, there should have been 37 women born in each of those months, of the 446.
And in fact, in June, there were 48 anorexic women born.
So they said, well, how likely is this to have occurred simply by chance?
Well as I am want to do in such occasions, I checked their analysis, and I wrote a little piece of code to do that.
So trying to figure out what's the probability of 48 women being born in June, I ran a simulation in which I simulated 446 births and chose a month at random, and looked at the probability.
And let's see what it was when we run it.
Oops, well we didn't want these graphs.
The probability of at least 48 births in June was 0.042.
So in fact, pretty low.
You might say, well, what's the odds of this happening by accident?
Pretty small.
Therefore, maybe we are really on to something.
Maybe it has to do with the conditions of the birth and the weather or who knows what.
Well, what's wrong with this analysis?
Well, one way to look at it is this analysis would have been perfectly valid if the researchers had started with a hypothesis that there are more babies born in June than in any other month-- more future anorexics born in June than in any other month, and then run this experiment to test it, and validated it.
So if they had started with the hypothesis, and then from the hypothesis conducted what's called a prospective study then they would have, perhaps, valid reason to believe that the study supports the hypothesis.
But that's not what they did.
Instead what they did is they looked at the data and then chose a hypothesis that matched the data, the Texas sharpshooter fallacy.
Given that that was the experiment they performed, the right question to ask is not what is the probability that you had 48 future anorexics born in June, but what was the probability that you have 48 future anorexics born in at least one of the 12 months?
Because that's what they were really doing, right?
So therefore, we should really have run this simulation.
Similar to the previous one, again, these are in your hand out, but is there at least one month in which there were 48 births?
And if we run that we'll see that the probability is over 40%, not so impressive as 4%.
So in fact, we see that we probably shouldn't draw any conclusion.
The probability of this happening by pure accident is almost 50%.
So why should we believe that it's somehow meaningful.
Again, an example of the Texas sharpshooter fallacy that appeared in the literature and a lot of people fell for it.
And if we had more time, I would give you many more examples, but we don't.
I'll see you on Thursday, on Tuesday, rather.
Two more lectures to go.
On Tuesday, I'm going to go over some code that I'll be asking you to look at in preparation for the final exam.
And then on Thursday, we'll wrap things up.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning, everybody.
We're on the home stretch here.
Only two more lectures, no more recitations, and a small matter of a final exam.
I should remind you the final exam is on Monday of finals week.
And it will be a two-hour exam, not a three-hour exam.
Same kind of format as on the first two quizzes.
Open book, open notes.
The difference is that we'll be giving you some code, which we'll be posting today, which we're going to ask you to study in preparation for the exam.
And then we'll ask you some questions about that code.
For example, ask you to make a small modification to it on the exam.
And in fact, today's lecture, just as a special incentive to pay attention, will be about things related to the code we'll be posting for you to study for the exam.
It covers one final class of simulation that we haven't looked at at all this term, but I think is important to at least know a little bit about it.
And those are queueing network simulations.
From the time of conception until death, human beings find themselves all too often waiting for things-- waiting for events, waiting for conditions, waiting for TAs, et cetera.
Queuing networks give us a formal way to study systems in which waiting plays a fundamental role.
Some typical examples might be in a supermarket, waiting to check out.
You get in a queue, wait your turn.
Computers all the time -- you may wonder sometimes why when you hit your computer, and ask it to do something, sometimes it's fast, sometimes it's slow.
Well, it's because there are tasks running in the background that are ahead of your task in the queue, and you're waiting for them to finish.
Computer networks, waiting for arrivals and things like that.
If you look at the internal structure of the internet, as, say, an email hops from hither to yon between two sites, along the way, it enters queues and waits for its turn to get forwarded to the next hop.
A few of you have experienced this when you go to office hours, where there's a wait for a TA, and you get in a queue.
Public transit, analysis of highway systems.
All of these things depend upon understanding queues to understand how they work.
You might wonder why we have queues.
The answer is pretty simple, in that it typically makes economic sense.
As we'll see in our analysis, when resources are limited-- particularly, say, servers-- it makes economic sense to have jobs line up for service.
So if you think about a supermarket, ideally, when you go to check out, you'd like there to be no line.
But think about how many cash registers they would need to ensure that all the time when someone was ready, there was no line.
And the problem is if they did that, they would have idle servers.
And so they would not get very good resource utilization of the people they would be hiring to do the checkout.
In designing a system with queues-- and that's what the simulations help us do-- we're aiming for a balance between, essentially, customer service and resource utilization.
The organization paying for the resources would like them to be working a 100% of the time at full capacity, so there's no wasted expense.
On the other hand, it's easy to show in any reasonable system that if you have a 100% resource utilization, you're going to be providing terrible customer service.
There is always a need for excess capacity in a complex system in order to provide adequate service most of the time.
And that's because, as we'll see, the demand for service is probabilistic and unpredictable.
And in order to absorb bursts of activity, you need excess capacity, because a lot of the time, you don't see those bursts.
All right.
All that by way of introduction.
If we think about how to model a system with queues, they're are all modeled essentially the same way.
You start with some jobs, work to be done.
Those jobs enter a queue.
They wait around for a while.
They leave the queue one at a time and enter a server.
And then they depart.
So that's what it looks like.
And of course, it can be much more complicated than this.
You can have multiple streams of jobs.
You can have multiple queues.
You can have multiple servers.
But when we build an analysis of it, when we build a simulation, we typically break it down into components that look exactly like this and use that as the key.
As we do this, we have to look at a number of different things.
We start with the arrival process.
How do the jobs appear?
And there are lots of questions we can ask.
Do they arrive singly, or in groups?
If they arrive in groups, we say that it's a batch process.
So if you think about a service for, say, processing students at MIT into the database, they typically arrive in a big batch in September, rather than straggling in one at a time.
We have to ask, related to that, is how arrivals are distributed in time.
Assume for the moment that arrivals are one at a time-- for example, automobiles entering a highway.
Do they come uniformly, at evenly-spaced intervals?
Do they come at random times?
If random, is there a time of day when we see most of them?
We typically model all of that in what's called an inter-arrival time distribution.
The simplest process is where these things do arrive at regular intervals, constant intervals.
We hardly ever see this in the real world.
Most of the time, we see something that looks much like our old friend, the Poisson process.
Things arrive at a random interval.
But they are typically exponentially distributed.
We've seen this before.
Ok.
It is, as you will recall, memoryless.
It is, as you will recall, described by a single parameter, the average arrival rate.
We can look at it just briefly to see what it looks like, just to remind ourselves, really.
So I've just a small piece of code, where I've given the mean arrival.
And then I'm going to just have 2000 things arrive at Poisson intervals.
Why is it not running?
I cleverly rebooted before I got here, so it shouldn't have had any problem.
There it is.
All right, so that really should be-- anyone remember what this calls?
Right.
And so we'll see that we have exponential arrival times, a rapid decay, only a small tail out at the right in this picture, and a big cluster near the mean.
We can look at this in a different way if you look at the scatterplot.
And you'll notice that we've given it a mean of 60, the dark blue line.
But we seem to have an awful lot of points below the mean compared to a very much smaller number of points above the mean.
Why is that, in fact, inevitable in this situation?
Yeah?
Because there's less space for them to--
Because?
There's just less space for them.
We can go from 60--
Well, there's less space.
And the reason that there's less space is we don't allow things to arrive yesterday, in negative time.
So the smallest inter-arrival time is just epsilon more than 0.
So the time between jobs can never be less than 0.
And so if we want-- and we're going to have a few way out here that take a lot longer, because that's the nature of an exponential.
So we're going to have to have a bunch down here to counterbalance the few way up here.
We can only get 60 below the mean, in this case.
And you can see we here have over 500 above the mean.
So it's going to always look like that, which is why the histogram looks something like this.
So you'll observe that as you look at these things.
All right, that's the arrival process.
We also have to look at the service mechanism.
How does the server work?
Again, we can ask some question like, how long will it take to provide service?
And this is called the service time distribution.
This depends on two things-- the amount of work the job to be served entails, and the speed of the server.
So, again, you've experienced this, no doubt, at the supermarket, where if you get in the wrong line, where there's a very slow person running the cash register, even though the jobs are small, it takes forever, because there's a slow server.
Or alternatively, you've gotten behind a line, not noticing that the person in front of you has two shopping carts full of junk.
And so even though the server is fast, it'll take a long time to service that job.
The number of servers certainly matters.
An important question is the number of queues.
In particular, does each server have its own queue, which is what you see at the supermarket?
Or is there a single queue for many servers, typically what you see if you are checking in, say, in coach class at the airport, or lining up for security at the airport?
Which of those is better, do you think?
Which provides, on average, better service, multiple queues or a single queue?
Single queue, right, is much better.
In fact, it's easy to construct a proof why it's better.
Which is why we now see them, typically, at places like banks and airports.
Didn't used to be.
Initially, people didn't understand this was better, and they would have multiple queues.
Why don't we always have a single queue?
Because we may not have enough space.
So you can, again, imagine at the supermarket there'd be no place to put the single queue for all of the cash registers, maybe.
And another question we have to ask is whether or not we allow preemption.
In some queuing systems, a server can stop processing a customer to deal with another emergency customer.
In others, once a job is started, it has to run to completion.
And, again, you get very different performance, depending upon whether you allow preemption or not.
An important part of the architecture is, of course, the characteristics of the queue itself.
The fundamental question is the policy.
How from the set of customers waiting for service do we choose the customer to be served next?
So a very common policy is FIFO.
It stands for First In First Out.
That's the one we see most of the time in a lot of physical situations.
Whoever gets in line first get served first.
There are some situations where you do Last In First Out.
Whoever's arrived most recently gets served first.
A very popular one in many computing applications is what's called SRPT, which is short for Shortest Remaining Processing Time.
Take whichever job you can complete fastest.
So these are the three most popular policies.
You could also imagine a random policy, where you just chose some random member of the queue.
Then there are lots of other kinds of issues, which I won't go into in detail.
The choice of queuing discipline, as we'll see when we look at the code, can have a major effect on performance.
In particular, it can be used to reduce congestion.
We'll often see in practice this one.
Now, why might that be a very useful practical method?
Well, It's got one obvious feature.
If the available space for the queue is small, by taking the ones you can finish fastest, you reduce the average number of elements in the queue.
Because you get to process more customers by taking the customers that you can process quickly.
And so on average, there will be fewer customers in the queue.
And so it reduces congestion.
As we'll see, it also improves expected service time.
So it is an excellent method.
When we talk about queuing networks, what we're going to see-- and I hope I've implied this already-- is that probability has to play a fundamental role in understanding them, because there's randomness.
There's randomness in the inter-arrival time, and there's typically randomness in how long each job will take to process.
What are the questions we might want to ask about a queuing system to see whether we've designed it properly?
An important question is the average waiting time.
On average, how long does a job wait between the time it enters the queue and the time it leaves the queue?
Now, imagine that you're waiting for the MIT Shuttle Bus.
How long you wait before you get on the bus, on average?
You can also ask, once the job starts, how long to complete?
We can also ask, is the waiting time bounded?
So this is a different question than how long you should expect to wait on average.
It's, what's the probability of your not having to wait longer than some upper bound?
So you can imagine that if you were working for some company that, for example, sold products over the telephone, you might want to put a bound on how long a customer would wait before their call got picked up.
And you'd do some analysis, saying if people have to wait longer than three minutes, they hang up the phone.
And so we'll have enough servers that it's only rare that someone has to wait more than three minutes.
It's a different question than how long people have to wait on average.
We've already talked about the average queue length.
And by analogy, we can also ask about the bound.
on queue length.
You can imagine if you're designing a router for the internet, you need to know how much memory to allocate for the queues of jobs so that you don't run out.
It's not infinite, so there's an upper bound in how long you'll let the queues grow.
And finally, along a different dimension is something we've already talked about, server utilization.
What is the expected utilization of each server, in the sense of, what is the expected amount of time it will be fully occupied?
Remember, servers cost money, so keeping them busy is important.
If we can assign relative cost values to each of these things, we can then build a system that will let us do an analysis and build an optimal system-- decide how big the queues should be, how many servers, how fast they should be, do the analysis, and design a system that meets some optimization criteria.
So now, we need to investigate these things.
Two basic methods, as we've discussed before, are analytic-- and, indeed, you can get books on analytic queuing theory, really thick, hard-to-read books, I might add, that give you complicated formulas for deriving these kinds of things.
In practice, they're hardly ever used, because most systems are too complex to actually model analytically.
So, in fact, today, most queuing network systems are analyzed using simulations, the kind of discrete event simulations we've looked at earlier in the term.
All right.
Now, I want to look at a particular example, and see how we can build a simulation that will let us answer these kinds of questions about a particular example.
The example is a -- and again, this is the one that you'll be asked to look at -- is a simplification of the MIT Shuttle Bus system.
So as I'm sure you all know, it runs in a loop, picking up students, and dropping them off at some number of stops.
For legal reasons, each bus has a maximum capacity-- often exceeded I'm sure, in practice, but there is officially a maximum number of people allowed on a bus.
And it takes a certain amount of time to go around the loop.
The obvious question to ask is, how long should a student expect to have to wait for a bus at a particular stop to get picked up?
And what should MIT do to, depending upon whether it's feeling benevolent or not, minimize or maximize the waiting time for the bus?
All right.
Most of the code we're going to look at now is on your handout.
I should say I've deleted the obvious parts so I could fit it on the handout.
However, we'll be posting the code on the web today, a .py file that you'll be able to study, and more importantly, probably run it.
All right, let's start.
So, as usual in designing a piece of code, I began with some useful classes.
You won't be surprised that the first class I looked at was class job.
So what is a job?
A job is -- depending upon the mean arrival and the mean work involved.
And I've chosen to model arrivals as exponential.
And I've chosen to model work as a Gaussian.
Nothing magical about this.
And in fact, I played with various other models.
What would happen if the work were uniformly distributed or exponentially distributed?
And, of course, you get different answers.
I said next two attributes.
There is actually only one attribute here.
And this is going to keep track of when a job enters the queue, so that we can tell how long it had to wait.
And then there's the inter-arrival, the work, the queue, the queue time, et cetera.
And then I'm going to model a passenger as a job.
You'll note this is kind of a boring class.
All I've done is say, pass.
Because I'm not adding anything new.
I am giving it a different type.
And it gave me a place to put the comment here, that the arrival rate corresponds to passengers to arriving at a bus stop, and work is the time it takes a passenger to board the bus.
There are many other things I could have done for work.
I could have made it multi-dimensional, having to do with how long to get on the bus, how many stops, et cetera.
For simplicity, I'm just assuming that there are some agile people who just get on the bus quickly, and there's some people who are a little bit slower, who might take a longer time to get on the bus.
And that's it for jobs.
Pretty simple, but pretty typical of the kinds of things we see.
Then we come to the queues themselves.
The root class is just the job queue.
Jobs can arrive, and the queue can have a length.
You'll note that I don't have any way for jobs to leave the queue in the base class.
And that's because each subclass will be distinguished by the queueing discipline it's using.
And the queueing discipline typically has to do with how jobs leave the queue.
So there's a FIFO discipline.
Pretty simplistic.
It just removes the first item from the queue and returns it.
Raises an exception if you try and remove something from the queue when there's nothing there.
An SRPT is a little bit more involved, but not much.
I've not tried to be clever.
You could imagine a much more efficient implementation than the one I've got.
But here, each time a job needs to leave the queue, I just search the queue to find one of the jobs with the least amount of work, and I remove that.
That's why I'm searching with the index here, because pop lets me remove something from a list at a particular index.
And then finally, I have another simple class, bus stop.
And the key thing here what it's a subclass of.
Initially, I'm choosing it to be a subclass of FIFO.
Later, we'll see I could make it a subclass of SRPT, or in fact, anything else I chose.
But this gives me a convenient way to decide what queueing discipline I'm going to use at the bus stops.
So I mentioned earlier that SRPT is nice, because it allows us to reduce the size of the queue.
I also mentioned that it's typically optimal in the sense of producing the shortest average waiting time.
So let's think about why.
Imagine that we do the opposite.
This is a good intellectual tool.
Just think of the opposite.
Suppose we always took the longest job first.
What would that do to the average waiting time?
Say we had a job that took 10 minutes and a queue with 10 people in it.
Well, while we were waiting for that 1 10-minute job to get on the bus, the other 9 people would each have to wait 10 minutes.
9 times 10, 90 minutes of waiting time.
Not very good, right?
Now, suppose each of those 9 jobs only took 1 minute.
Well, the first job would wait 0, the next job would wait 1 minute, the next job would wait 2 minutes.
And even the longest job would only have to wait 9 minutes before it started.
So the total amount of time spent waiting would be much less.
So we can see that by using shortest remaining processing time, the waiting times are reduced.
Now, with the bus example, it's a little bit strange, in that, yes, you'd get on the bus, but maybe the bus wouldn't move at all while it was waiting for that last person to get on.
But most queueing disciplines-- imagine you're in the supermarket-- you don't care how long the guys behind you take.
As soon as you get to the front, you get processed, you get to leave the store.
So in fact, a supermarket would, on average, have happier customers if it followed shortest remaining processing time, and always took whoever had the fewest number of items first.
Why don't they do that?
Or why is not SRPT always used in practice?
It's got one glaring problem.
What's that problem?
It doesn't seem fair to everyone else
It's not fair.
And this is actually a technical word.
People talk about fairness of a queueing discipline.
And in particular, it's so unfair that it allows starvation, another technical term.
Imagine that you're the person with the 10-minute job, and there's a continuing stream of people arriving with 1-minute jobs.
You never to get your job processed at all.
You just sit there, and you wait, and you wait, and you wait.
And in fact, you wait forever.
In the bus situation, imagine you're the person who's in a wheelchair and takes 10 minutes to get on the bus.
And people keep arriving at the bus stop.
Well, before it gets to be your turn, the bus is full, and it leaves.
And you have to wait for the next bus.
But people arrive while you're waiting.
It arrives.
Again, you don't get on.
And you never get on.
This is technically called starvation.
And it's a problem that SRPT has.
So typically, if you're using that as your scheduling algorithm, you've got to put in some sort of a priority on top of it that gives the long jobs a chance to at least run once in a while.
And there are lots of different disciplines for doing that.
All right, let's continue looking at the code.
So now we've got bus stops, which are queues.
We got jobs.
Well, we're going to have our server.
And in this case, our server is a bus.
Every bus has got a capacity, the number of people it's allowed to hold, and a speed.
The speed has to do with how fast it goes from stop to stop.
And for simplicity, we're going to assume that this depends only on the bus itself, not on the traffic.
Not true in Cambridge and Boston, of course, but we can't get everything in a code that we get through in one lecture.
So it's got a capacity, it's got a speed.
And then, we can keep track of how many people are on the bus.
It's got an enter, which has to do with people getting on it.
And does that until it's full, and then it raises an exception.
And things can leave the bus.
And here's the interesting method.
Unload takes a self and the number to be unloaded from the bus.
And it just calls self.leave over and over again.
All right, those are our basic classes.
Nothing too sophisticated.
As is often the case with classes, the code is straightforward.
And now comes the not-so-quite-straightforward code, which is the actual simulation itself.
Probably the most important thing to understand in looking at this code is the arguments.
This will tell us what the simulation depends on.
So there's going to be a bus.
So notice we have only one bus.
So, again, I've simplified the simulation.
Instead of having multiple buses running around the loop, we're going to assume you've just got to wait for that first bus to get back.
Later, we can talk about what would be involved in having multiple buses.
The number of stops in the loop, the loop length.
And for simplicity, I'm going to assume that the stops are equally spaced along the length.
The mean arrival rate of jobs, the mean work per job, and then how long we're going to run the simulation.
Roughly speaking, I'm using the same units for the simulation time and the loop length.
So this would tell us, if we simulate for 30,000 times, and the loop is this 1,200 time units long, how many times around the loop we could get, and how many stops the bus would make.
All right.
And then comes the body of the simulation.
So the first thing-- actually, let me write up some pseudo-code here to basically show you the way it's structured.
Yeah, I think we'll put it over here.
So the first thing we do is we create the stops, and we initialize some variables then we're going to use to keep track of performance.
We then enter the main loop, which is essentially while time not expired.
That has to do with how long we're going to run the simulation.
We move the bus.
So the loop will go through one time unit each iteration.
At least, it looks that way initially.
So we'll move the bus that amount of time.
So then, while we're doing it, passengers will arrive at the stops.
Then we'll see when a bus is at a stop.
Has it driven long enough to get to the next stop?
And when it's there, we'll unload some passengers first, and then we'll load some passengers.
And then, when we're done at the very end, we'll compute some statistics.
So that's the overall structure of the simulation.
Then we can put that in something outside it that tells us how many simulations to run and get more statistics about aggregate.
All right, now working our way through the code, the initialization, while time less than simTime, advance time.
Move the bus.
Now, how far we move it depends upon how fast it is.
So we'll just add to the bus location however far it's allowed to move in one time unit.
I don't care whether the time unit is seconds, minutes, hours, days, whatever.
Just it has to be all consistent.
Then I'll see if there's a passenger waiting to enter the queue by checking the arrival time.
And passengers can arrive simultaneously at each stop.
I apologize for leaving the comments out of your handouts, but I couldn't squeeze it onto the page with the comments.
Then, here's the interesting piece.
So we do this.
We're just having passengers arrive based upon the characteristics of jobs.
See if the bus is at a stop.
If the bus location divided by the inter-stop distance is equal to 0, so it's actually at a stop, I'll do my unloading, et cetera.
Now, down here, you'll notice that up here, I had this assumption that the speed of the bus is small relative to the inter-stop distance.
This assumption is important because of this statement.
If the bus were moving too fast.
It might skip over the stops.
Now, this is kind of a clumsy trick I've used here.
It has to do with it being small.
It has to do with each movement of the bus being, in fact, an integral of the inter-stop distance so that you don't skip over the stops.
In a real simulation, I would have done something much more careful to deal with not missing stops.
But again, I wanted to make this relatively simple.
So I'm going to jigger things so that the speed of the bus evenly divides the inter-stop distance, so when I advance the bus, it never skips over a stop.
It's just a trick.
Not very elegant trick, but it keeps things working.
All right.
Then I'm going to unload some fraction of the passengers.
Now, again, in a more realistic simulation, I'd be keeping track of what's the average distance a passenger wants to travel.
And I would have, in fact, made work two-dimensional-- the amount of time to get on the bus, plus how far the passenger wanted to travel.
Again, here, I've done something simple.
I've just assumed that a fraction of all the passengers get off at each stop, and the fraction depends upon how many total stops there are.
We can still understand most of the interesting aspects of the problem, even with all of these simplifications.
And that's an important lesson.
Every simulation is a simplification of reality.
No simulation is ever a perfect model of reality.
No model is ever a perfect model of reality.
What we typically do in building the simulation is simplify it enough that we can get it to work, but not so much that we can't answer the questions we want to answer.
And so I've made a lot of these simplifications, but they will still allow us, while maybe not to get precise answers about the impact of various things, to understand how things relate to each other, the relative impact of various design decisions.
All right.
If the bus is at a stop, as I said, some passengers will get on, get off.
Passengers that are already at the stop will try and get on the bus.
We'll have to advance time, because time starts to run out as people are getting on the bus, even though the bus is moving.
But, of course, even though time is advancing, the bus is not.
So we have to model that, the impact of loading times.
And in fact, in mass transit systems, it turns out that loading time is a very important parameter.
In many urban systems, subways will spend more time at stations getting loaded than between stations, moving to the next one, if the stations are dense.
You may have noticed this on the Green Line.
So we have to make sure we model that.
And then, of course, since the bus could be at the stop for a while, passengers might arrive while it's at the stop.
Then, we can keep track of the number of people left waiting when the simulation ends, and do things like that.
All right, finally, there's this program test, which I won't go over in detail.
It's exactly like the drivers we've written for other simulations for at least half the semester, in which I run simbus multiple times, accumulate results from trials, and then plot them nicely.
All right, let's run it.
So I'm going to run it with-- I've actually written it so that I can look at combinations of speeds and capacities.
But first, I'll just look at small combinations-- i.e.
each list will be one.
So we'll test it with a capacity of 30 and a speed of 10, to 40 trials.
Then, we'll test it with a capacity of 15 and the speed of 10 and a capacity of 15 and a speed of 20.
All right.
We have some results.
So what we can tell looking at Green Line is that I have a speed of 10 and a capacity of 15.
The aggregate average wait time is long and apparently growing the longer I run the simulation.
I've only plotted here up to the first 500 times if the bus stop at a stop.
But we can imagine that if I went further, it would just get worse for a while.
Then, it will eventually plateau, but it will take it quite a while to do that.
And here we can see the interesting thing, in that the Blue and the Red Lines are not very different.
So what this tells us is there's a trade-off.
I can afford to have smaller buses if I make them faster.
Or conversely, if I want slow buses, I have to make them bigger.
And I get more or less the same average wait time and not much difference in how many people are left behind at the end.
Clearly, here I have a big number.
These are smallish numbers.
Again, we'd have to run it longer to see whether these are stable, and with different seeds, or different random numbers, to see whether or not the fact that this is 80 and this is 40 is statistically significant, or just an accident of this run.
Because we do have randomness.
But we did do 40 trials, so maybe there's a reason to believe that in fact, this is slightly worse.
You'll notice also the Blue Line doesn't quite get all the way to 500, which means that slow bus didn't get to make enough stops.
It took a long time to load up, because it had big capacity and didn't really get to all of the stops.
All right.
Let's try another experiment.
And let's go change the queueing discipline.
Go way back up here.
And let's make it SRPT and see what we get.
Well, we get something pretty different.
In particular, you'll note that the average wait time has dropped rather significantly.
It really has, as the theory predicted, led to the improved wait times.
We'll also notice that it's got some interesting shapes here, where it jumps around, goes up, almost a knee here.
And you might try and understand why it's doing that as you look at the code.
Is it random?
Is it just bad luck?
Is there something fundamental going on?
Think it all through and decide what's going on.
All right, finally-- and, again, I suggest you play this with a lot of different combinations to get a sense of what the code is doing.
So one last queuing network question to test your understanding-- different question.
Suppose you were given the job of hiring a 6.00 lab assistant, and you had a choice between hiring an MIT student at $15 an hour or two Harvard students at $7.50 an hour.
Suppose further that the MIT student could deal with twice as many problems per hour as the Harvard students.
Does it matter which you hire?
Well, you could build a simulation, and you would discover that in your queuing network, it didn't matter.
On the other hand, when you ran it in real life, you would probably discover that it mattered a lot, because the MIT student would answer questions correctly.
And the moral is, the queuing network never tells the whole story.
You've really got to think about the thing in more detail.
All right.
One more lecture.
See you on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now today's lecture.
First I want to talk a little bit about what computer scientists do.
This whole course has been about computer science.
And I hope that at least for many of you, it might have persuaded you that there's an interesting career out there for you as a computer scientist.
And then I'll finish up with an overview of what I think we covered this term.
So what do computer scientists do?
Well, they look a lot like that as you all know.
In fact, it's amazing.
There's almost nothing that computer scientists don't do.
So here I've just collected a few of the different pictures related to what computer scientists in EECS are doing these days, things like working on the software that keeps airplanes from falling out of the skies.
Quite a few people working in the movie industry these days doing animations in 3D and that sort of thing.
These days, computer scientists don't get paid as much as the actors.
But they get paid a lot to produce movies.
Robots, of course.
A lot of medical work, particularly medical imaging.
This is a picture where someone has written software to try and identify the tumors in the human brain, and then will draw little lines telling the surgeon where to cut so as to do the least damage and the most good.
A lot of work in genetics.
And of course, core things like making the internet actually not crash.
Fundamentally, what computer scientists do is they think computationally.
And I hope that you've understood that that's been the theme of 6.00, is how to formulate problems and think computationally.
This will be, I think, the most important skill used by certainly scientists and engineers but, in fact, everybody by the middle of the 21st century or sooner.
People will just do everything computationally.
And just as once upon a time maybe it's still important everyone learn to read and write and do a little arithmetic, people will think that computation is exactly as important and that you'll have to know how to formulate problems computationally to survive in society.
The process, and this is what we've been doing all term, is not an easy one.
But it's not an impossible one either.
We begin by identifying and inventing useful abstractions.
Everything we do is an abstraction of reality.
And as we've gone through the second half of the term, almost every interesting computation we've designed starts with inventing classes that give us useful data abstraction or functions that compute useful things.
And that's sort of what we start with.
We start with some existing set of abstractions, and we invent new ones that will help us think about the current problem.
We then formulate the solution to the problem as a computational experiment and then design and construct a sufficiently efficient implementation.
It doesn't have to be the most efficient one possible.
It just has to be efficient enough that we can run it and get an answer.
Of course, before we trust the answer, as with any experimental apparatus-- and it is an experimental apparatus, a program-- we need to validate the experiment, convince ourselves that when we run it, we should believe the answer.
That's the process of debugging.
And I'm sure it's a process that all of you have spent more time on this term then you might have liked.
I hope by now you feel you are a little bit better at it than you used to be.
Once we think we've got the experiment put together properly, we run it.
We then evaluate the results.
And then we repeat as needed, the classic iterative style of science.
And a lot of this, of course, is what you would have learned in your chemistry lab or your physics lab or the bio lab in designing any experiment.
You have to go through a lot of these steps.
And the difference is these are all computational experiments.
So we think of there are two ways of computational thinking.
There's abstraction.
You have to choose the right abstraction.
Typically, we operate in terms of multiple layers of abstraction simultaneously.
And this is an important part of thinking computationally.
We have to be able to not worry about, say, the details of how floating point numbers were implemented, but just assume it's a good approximation of the real numbers.
That's one level of abstraction.
And another level of abstraction, we think about bus stop queues.
And maybe we have to implement those ourselves.
And then we have to think about the relationships among the layers.
A key thing that makes computational abstractions different from so many others is we can automate them.
So we often think about abstractions as we go through life and think about how to cope with things that are too complex to understand.
We have these various abstractions.
Newton gave us some good abstractions of the physical world that let us understand how things like levers and such almost how they work.
But here we get the big advantage that we not only can invent an abstraction, we can mechanize it.
We can do this for two reasons.
One, we have a precise and exacting notation for expressing the abstractions for building our computational models.
This term, that has been Python and PyLab and NumPy.
But it could have been Java.
It could have been C++.
It could have been MATLAB.
Doesn't matter very much.
What matters is that we have a notation that's precise enough that we can actually describe a computation.
And we have some machine that can take a computation written in that notation, or a set of computations actually written in that notation, and execute them and give us results.
And that's been the big transformation that has made computation so important.
We've had the notion of computation long before we had machines that could execute them.
The Greeks, the Egyptians understood notions of computation, as we've seen before when we looked at some early algorithms.
Algorithms have been around a long time.
But nobody cared very much until we had machines and notations that would let us actually run the algorithms.
That came along in the late '40s.
And then the world changed dramatically.
So some examples of computational thinking.
How difficult is this problem, and how best can I solve it?
And that's what theoretical computer scientists spend their time thinking about.
They try to give precise meanings to these kinds of questions so that we can ask them in the context of, say, random access machines or parallel machines or, more lately, quantum machines, how do we formulate those questions precisely?
And how can we answer them?
And of course, thinking recursively is very important.
Fundamentally, it's the key in that what we try and do is we try and reformulate a seemingly difficult problem into one we already know how to solve.
This is what we talked about before, reduction to a previously solved problem.
And we've looked at different ways of doing this.
There is reduction, as I mentioned.
There's embedding, where, OK, our problem is complex.
We can't reduce it to a different problem.
But we can embed a different problem in it as at least part of the solution.
We can do some transformations, take one problem and transform it to a different problem.
Usually, a simpler problem that we know how to solve.
We've seen a lot of that.
That's, of course, what you've seen in the bus simulation you're studying, I hope as-- I hope not as I speak, but as soon as I finish speaking, you'll be studying it.
We've taken a complex problem and simplified it by transforming it into a simpler problem that we can then attack.
And we've used a lot of simulation as a mechanism for dealing with problems.
So here's a little recursive picture.
Here's one of my favorite illustrations of recursion.
I don't know how they did that, but I think it's pretty cool.
All right.
That's a very quick introduction, at a very abstract level, what computer scientists might do.
I'm gonna spend some time now on what one particular computer scientist does, and that's me.
Why did I choose me?
Well, not because I'm the most interesting, but because I'm the one I know the best.
So it's kind of easier for me to talk about my work than about somebody else's.
And the truth be told, it's not actually my work.
Like all professors, I claim credit for the work that my students do.
So all of the work you're going to see is actually work that my students have done.
Mostly graduate students, but also some very productive UROPs.
So, I won't give them as much credit as I should, but you should know in your heart that I didn't do any of this actually myself.
All right.
What do we do?
The goals of our research group is the rather modest one of helping people live longer and better quality lives.
Other than that, we don't intend to accomplish anything.
And we do this mostly in the medical area by collaborating with clinical researchers and practicing clinicians.
Along the way we have a lot of fun pushing the frontiers, the state of the art in computer science, electrical engineering, and certainly medicine.
We use a lot of the technical kind of ideas you've seen this semester and that you'll see in almost any computer science course you take.
We do a lot of machine learning and data mining.
I'll explain why shortly.
We do a lot of algorithm design because a lot of the problems we're trying to solve are too complex to solve if you're not clever about how you solve them.
We do some signal processing.
I'll explain why we do that.
And a lot of just plain-old software systems are needed.
Do we use databases?
How we build software that injects electrical signals into people's brains?
So we spend a fair amount of time trying to convince ourselves it actually works, doesn't inject the wrong signals at the wrong time.
But that might be fun too.
And these are just the logos of some of the hospitals we work with.
So what are some of the problems we try and deal with?
Probably our longest standing project has to do with medical telepresence.
These hands do not belong to a basketball player.
These are normal sized hands like yours or mine.
That is not a normal sized baby.
This is not a Photoshop photo.
This is a real photo.
This is a very small premature baby.
And we have spent a lot of time working with groups figuring out how to provide better outcomes for these tiny little, very delicate people.
A problem is that a lot of these babies are born-- if these babies are born in the right hospital, Brigham and Women's Hospital or something close to Children's Hospital or MGH, they have very good outcomes.
They tend to survive, and they tend to grow up to be relatively normal.
If these babies are born in a community hospital that does not know how to take care of these kinds of babies, they have very bad outcomes.
A lot of them die.
Many of them who survive end up being permanently disabled.
Very sad.
One of the shocking statistics is that globally there's a neonatal death every 10 seconds.
Shocking number, I think.
It's just tragic.
And many of these are unnecessary.
And they occur because there's not adequate care at the point where the baby is born.
And so we've been trying to use technology to link places where these babies might be born to places where people can advise how to take care of them.
And it turns out to be a very interesting set of computer networking communications problems.
You can't just use Skype for reasons I won't have time to go into.
But it's very exciting.
It uses a lot of computer science, a lot of medicine, but mostly computer science, in this case.
And we're actually currently running some trials in conjunction with Children's Hospital Boston where they're actually trying to look at babies born elsewhere.
Another problem we've been looking at is health-care associated infections.
Approximately 1 in 20 hospital visits results in the patient contracting an infection totally unrelated to the reason they entered the hospital.
And today it's among the top ten leading causes of death in the United States.
Somebody enters a hospital for a reason x, picks up an infection unrelated to x, and dies.
Not a very good outcome.
And it's particularly discouraging because, in principle, these infections should be preventable.
You shouldn't acquire a life-threatening infection while you're in the hospital.
But people do.
We've been trying to understand why.
We've been working on this project with Microsoft, which has provided us with data from 4.5 million different hospital visits and about 2000 pieces of information per visit, lab tests, drug tests, who the doctors were, who the nurses were, what rooms they were in.
And we're trying to study-- these are pictures of some of the most obnoxious of these infections-- and we've been trying to figure out what's causing them and what can be done not to cause them.
This has been primarily a machine learning project.
We've been struggling with it.
We've been dealing with exactly some of the issues we talked about in class.
Which features are most important?
How should you weight the different features?
How can we reduce the number of features so that we can actually finish our computations?
We've been using different kinds of clustering, different kinds of supervised learning, many different techniques.
And we're beginning to get a handle here on, in fact, what are some of the causes of these infections and what could they be done.
Some of the things are very disturbing.
Some of them, a lot of them, seem to be caused by drugs that are given in the hospital.
And maybe we can substitute one drug for another if we understand that.
At least in some cases they seem to be related to things like what room you happen to be placed in, suggesting that maybe they're not cleaning the rooms adequately.
Things like that.
All right.
Another project, probably our biggest sets of projects, have to do with extracting information from physiological signals.
We've been focusing on the heart, the brain, and the connected anatomy.
For those of you who don't happen to be biology or course 20 majors, here's where the heart and the brain are located and what they look like.
Some of the examples we're interested in are predicting adverse cardiac events.
And for that you can think about death from a heart event, heart attack.
And we spent a lot of time dealing with epilepsy.
So let me give a little bit more detail about these two examples because each of them relate to things we've covered in 6.00.
So epilepsy, an interesting disease.
Surprisingly, it affects about 1% of the world's population.
When I first heard this, I was astounded.
Because gee, 1 out of 100 people that I know have epilepsy?
And the truth is, yes.
I just didn't know it.
I've been amazed, since it's become known that I do research in this area, the number of people I've known for years who come up to me say, you know, I never mentioned this before, but I have epilepsy, or my kid has epilepsy or my parent has epilepsy, and I have some questions now that you're supposed to be an expert on this.
And I give them the usual caveats, you know, I'm not a doctor, I just play one on television.
But then I tell them go talk to one of my students.
They're not doctors either, but they're really smart.
Probably smarter than your doctor.
All right.
Why is it underestimated when we do it ourselves?
Well, because they're still some stigmas.
A few hundred years ago, they burned women in Salem, Massachusetts, who had epilepsy because they thought the seizures meant they were possessed by the devil.
Today, if you tell the Registry of Motor Vehicles that you have epilepsy, they won't let you drive an automobile.
So people tend not to announce it.
What's it characterized by is a recurrent seizure.
A seizure is abnormal electrical activity that originates and persists in the brain.
Number of causes.
It can be acquired.
It can be inherited.
There's manifest different symptoms.
Most of them do not look like what's portrayed on the Simpsons.
Slightly more realistically, and maybe more horrifying.
Here's a picture of a young girl, a movie, you're gonna see a seizure.
It's not pleasant, I'm gonna warn you.
What I want you to notice is two things.
(1) She's all happy, and the seizure seems to come out of nowhere.
She doesn't expect it.
She doesn't know she's going to have.
It just hits.
That's very important.
(2) She happened to be in the clinic at the time wearing this funny looking cap, with it has electrodes in it.
Just sits on the scalp.
And here is a record of the electrical activity at the surface of her scalp.
And what you'll see is a quick slight change on it just before the seizure hits, and then enormous changes during the seizure.
Turns out that the enormous changes during the seizure have nothing really to do with the seizure.
They have to do with muscle activity.
As you'll see once the seizure hits, there's a lot of ugly muscle activity.
All right.
Let's look at it.
Uh oh.
All right.
Let's look at it-- I thought I had put the links in here that should have done it, but who knows.
I know where to find them.
[VIDEO PLAYBACK] [SINGING IN FOREIGN LANGUAGE] [SPEAKING FOREIGN LANGUAGE] [END VIDEO PLAYBACK]
So kind of scary.
Certainly when I've been seeing them actually happen, I've found it terrifying to watch.
That kind of-- it's called a tonic-clonic seizure.
They seem- I emphasize the word 'seem' -- unpredictable.
The interesting thing about these seizures, or an interesting thing, is they're self-limiting.
In a few minutes, that young lady or girl's seizure will be over, not because anybody intervened but because the brain resets itself.
Then it will take-- in this case, probably took her about an hour to recover and get back to normal, which is not good.
But after that, everything was as before the seizure.
The difficulty is because they're unpredictable, there are huge injuries.
Imagine she'd been riding a bicycle when that hit.
Well, something catastrophic.
Or more simply, imagine she'd been going down a flight of stairs or in a bathtub.
Any one of a number of things would have resulted in this potentially catastrophic collateral damage after the seizure or during the seizure.
And indeed people who have epilepsy, you can see their scars.
It's awful.
It can result in death, about 1 per 100 patient years.
It's called the sudden unexpected death in epilepsy patients.
So what we wanted to do is see whether we could detect the seizures and give a little warning.
The notion being that even if you couldn't do anything about the seizure, if you could give somebody, say, even five seconds warning, they could sit down before they fell down.
They could get out of the tub.
They could back away from the stove.
All sorts of things.
Furthermore, if you could use technology to signal to someone else that there was about to be a seizure, help could arrive.
Also potentially a good thing.
Turns out there are now some fast-acting drugs that if you put under the tongue can ameliorate the seizure.
These have not yet been approved by the FDA, but soon.
And we were particularly interested in neural stimulation.
If we ejected electrical current into the brain at exactly the right moment, could we offset essentially the effect of the seizure and do a reset?
And maybe stop the seizure or abort it, prevent it, or at least reduce the long term recovery time.
The thing to keep in mind for the seizure is there were two distinct onset times.
What you saw if you're looking at the girl was what's called the clinical onset, when there's some clinical event.
If you were able to not look at the person and instead look at the EEG, not so easy, you would have seen that there was an electrographic onset that preceded the clinical onset.
We know, in fact, since the clinical effects are caused by the electrical activity, there will always be abnormal electrical activity prior to any abnormal clinical activity.
And so the hope was that we could detect the electrical activity early.
Now that's not so easy.
What makes it hard is that the EEG, the electroencephalograph, differs greatly across patients.
First of all, people with epilepsy have abnormal baseline EEG.
Even when they're not having a seizure, bizarre things are going on in their brains electrically, all unusual things.
And so they don't look like people who don't have epilepsy.
And it varies tremendously across patients.
So for about 35 years, people attempted to build generic detectors that would detect seizures in everybody.
And they've not worked well at all.
Every time a hospital buys an EEG machine, it comes with a detector built in.
And usually the first thing they do is they turn them off because the false alarm rate is so high that it's like the boy who cried wolf.
They're just saying seizure, seizure, seizure, and people start ignoring it.
The good news is it's pretty consistent seizure onset for a particular individual.
That suggests you should build not generic detectors, but patient specific detectors.
And we've been working on using machine learning to do that.
And in fact, it's been highly successful.
We've done several retrospective studies indicating that it works really well.
And we're now doing a substantial prospective study -- in progress at MGH.
And as part of that, we're actually working at turning on a neuro-stimulator that we hope will attenuate the effect of the seizure.
And I emphasize hope because we don't have enough data to know if it works.
It's part of what goes on in this business.
But it's certainly been an interesting project.
All right.
Heart attacks.
Let's look at that.
And I'll go to the easy way to show them.
I should warn you, you're gonna see something that's not very pleasant.
You're actually gonna see somebody die.
This was actually a person who was playing in a soccer game and died while the game was being televised.
All right.
What I want to show you is this didn't have to happen.
We're now gonna see another televised soccer game where the player had exactly the same event but with a rather different outcome.
So look at the upper center of field and you'll see the person has collapsed, much the same way it happened before.
Now watch the body convulse.
You'll see the knees kick up there.
And now comes the miracle.
He sits up.
He leaves the field and, in fact, he later asked if he can reenter the game.
The coach, to the coach's everlasting credit, said no.
Clearly the right decision.
So it's amazing really that this happens.
So what was the difference between the two?
well, we see this here.
There are things you can do.
So in acute coronary syndrome, think of it as some sort of a heart attack.
They're very common, about one and a quarter million per year in the US.
15% to 20% of these people will suffer a cardiac-related death within the next four years.
If you could figure out who were the people at highest risk of different events and choose the treatment properly, for example, implant an implantable cardiac defibrillator, you could save these lives.
And that was what we saw in our movies.
That the first patient, first person, the one who died, did not have a defibrillator.
The second person, the one who survived, did.
So he collapsed on the field.
The defibrillator sensed that his heart had stopped, gave it an electric shock.
You've all seen this in television where they put the paddles on and they say, clear.
And then there's this moment of drama where everyone stares at the EKG machine, and it goes from a flat line to suddenly up and down, and everyone goes, ah.
Well that's exactly what happened here.
And that convulsion was this huge electrical shock getting sent through the person's body which restarted the heart, saved his life.
Probably, if the other player had had an ICD, he would not have died either.
So great technology.
Well, yes and no.
This was a study in the New England Journal of Medicine not so long ago, a very good control study, where they matched patients with ICDs and patients without ICDs and over 72 months tracked which ones lived and which ones died.
Well, the disturbing news is there isn't much difference between the red line and the blue line.
So these people had what was essentially a, well, which was about a $50,000 implant and various kinds of risks and inconvenience and discomfort, et cetera, and on average it didn't save lives.
Oh, dear.
In fact, 90% of the people who got them-- and this is not just in this study.
This is in every study- receives zero, actually less than zero medical benefit from the ICD.
How do we know that?
Well, remember, it only turns on when it senses the heart is in trouble.
For 90% of the people who get it, it never energizes because the heart doesn't get in trouble, or detectable trouble.
So what we see, if we look at a little more detail in this study, is that for sudden cardiac death, the kind of death we saw in those videos, or non-death, the ICD reduces that.
Unfortunately, it increases other causes of death, for example, infections related to the surgery, et cetera, those unfortunate hospital acquired infections, for example, we talked about earlier.
So what we see here is we have a technology that if we knew whose heart was likely to stop or go into serious fibrillation, beating uncontrollably, and we only gave those people ICDs, we could save an enormous number of lives.
But currently we don't know who's in that population, and therefore we don't know who should get them.
So we use other mechanisms deciding who should get them.
And we're wrong most of the time, doing more harm than good on those patients for whom we're wrong.
OK. Well, how do the people predict it today?
The usual things.
Are you male or female?
Do you have high blood pressure?
Diabetes?
What's your cholesterol level?
BMP?
Various other kinds of things.
Electrocardiograms, which look at-- it's an echo cardiogram that looks at the activity of the heart.
Is the blood flowing through it OK?
EKGs.
Many different methods for analyzing the electrical activity, et cetera.
We played with many of these techniques.
The one I want to tell you about today, because it's the most closely related to 6.00 material, is what we think of the Tolstoy approach to risk stratification.
So Tolstoy is famous for saying that happy families are all alike, but every unhappy family is unhappy in its own way.
So we generalize that, or specialized it maybe, to say that happy hearts are all alike, but every unhappy heart is different, and then did some fairly simple work to quantify the difference between electrical activities in different people's hearts, converting it to symbols.
You don't have to worry about that.
We used dynamic programming as an important part because we were looking at roughly a billion heart beats and so we needed to make it run fast.
And then we used clustering to identify patients whose hearts we thought were similar to one another.
Here are some results.
These are people who had an acute coronary syndrome.
So the dominant group, the biggest group, the quote "people who we thought were most likely to be fine"-- because remember, most people are fine after a heart attack-- 457 patients in this particular data set.
And you can see that they did pretty well.
A very small fraction of them died.
Can't see it from this angle, but come out here.
Yeah, less than 1%.
But then we looked at people who were in these other clusters.
And in fact, we used a glomerate of hierarchical clustering to do this.
And in cluster A, which had 53 patients, well, 3.77% died.
If you happened to fall in cluster C, you were at very high risk.
And we can just see that there's a big difference here.
And so the notion is could we have used this to predict in advance who was most likely to benefit from the various kinds of treatments?
All right.
Again, very quick just to give you the details-- not the details, the overview that the kinds of things that we cover in this course are actually quite useful in solving real practical problems.
All right.
Let me wrap up the term.
So I hope most of you feel you've come a long way-- maybe you prefer this picture.
That if you think about the kinds of problems and struggles you had three months ago in getting tiny little programs to work and think about how easy those would be for you today, you should have some appreciation that you've really taught yourself a lot.
And you really taught it to yourself, by doing the problem sets, in many ways, right.
We've tried to help, but learning is very much up to the individual who's doing the learning.
But I'm certainly impressed in looking at the kinds of really pretty complicated problems you guys can now solve.
We looked at six major topics.
Clearly, you learned a notation for expressing computations, and that was Python.
And I hope you don't think that's the only notation you can use and that if you take a course that uses MATLAB, you'll say, oh, this is just the same.
It's easy.
Or Java or anything else.
Learning programming languages is easy.
Once you've learned one, the next one is much easier.
Harder was learning about the process of writing and debugging programs.
You've learned that, I think, largely through experience.
You've learned about the process of moving from a problem statement, something as vague as should shuttle buses be bigger or faster to improve service, to a computational formulation of the problem and a method for solving a problem.
And we've looked at lots of different methods.
You've learned basic recipes, algorithms, things like dynamic programming, things like depth-first search, things like decision trees, that you'll be able to use again and again.
The good news is there are only a really small number of important recipes.
And once you've mastered those, you just use them over and over again to solve new problems.
Spent a lot of time on using simulations to shed light on problems that don't easily succumb to closed form solutions.
And I think that's really the place where computation is so important.
There are a lot of problems where you can turn them into a set of differential equations, maybe solve the equations, and you're done.
Oh, we'll write programs to do those because we're lazy.
But in principle, you could solve those without a computer.
But in fact, the thing you can't do without a computer is these messy problems that are most of what goes on in the real world, where there's randomness, and things like that, and there is no simple formula that gives you the answer.
And what you have to do is write a simulation and run it and see what goes on.
And that's why we spent so much time on simulation because it really is increasingly the thing people use.
And we learned about how to use computational tools to help model and understand data, how to do [?]
plots, a small amount of statistics, machine learning, just dealing with data.
Why Python?
It's pretty easy to learn to use.
Compared to most other programming languages like, say, C++ or Java, Python is easy.
The syntax is simple.
It's interpretive, which makes debugging easier.
You don't have to worry about managing memory as you do in, say, C, right.
You get a big list or dictionary, and magically it exists.
And when you don't need it any more, it's gone.
It's modern.
It supports the currently stylish mode of object-oriented programming in a nice way with its classes and things like that.
So indeed what you would hear about in a Java class, we can cover almost all the interesting things with Python.
And it's increasingly popular.
It's used in an increasing number of subjects at MIT.
A lot of course 6 subjects, but also a lot of course 20 subjects, course 9 subjects, over and over again.
It's becoming, probably, the most popular language at MIT and at other universities.
Increasingly it's used in industry.
And as you've seen with PyLab, the libraries are amazing.
And so PyLab, Random, et cetera, there's just a lot of stuff you can get for free if you're living in the world of Python.
The main thing in writing, testing, and bugging programs, I hope you've learned, is to take it a step at a time.
Understand the problem.
Think about the overall structure and algorithms separately of how you express them in the programming language, right.
We can talk about dynamic programming conceptually without worrying about how to code it up.
Always break the problem into small parts.
Identify useful abstractions.
Code and test each unit independently.
Worry about functionality first.
Get it to work.
Get it to give you the right answer.
Then worry about making it do that more quickly.
But separate those things.
Start with some pseudo code.
Then above all, be systematic.
When debugging, think about the scientific method of forming hypotheses, forming experiments that can test the hypotheses, running the experiment, checking the results.
Don't try and do it too quickly.
Just be slow and careful.
Slow and steady will win the race in debugging.
And when your program behaves badly-- and the sad news is no matter how many years you spend at it, you will write programs that don't work the way you think they should the first time-- ask yourself why it did what it did, not why didn't it do what you wanted to do.
It's a lot easier to debug from the how come it's behaving the way it is then why isn't it behaving the way I want it to behave.
In going from problem statement to computation, break the problem into smaller problems.
Try and relate your problem to a problem that somebody else, ideally, has already solved.
For example, is it a knapsack problem?
Is it a shortest path problem?
If so, good, I know how to solve those.
Think about what kind of output you might want to see.
What should the plots be like?
I usually design the output before I design the program.
Often you can formulate things as an optimization problem.
Pull back.
Say, well, can I formulate this as finding the minimum or maximum values satisfying an objective function and some set of constraints?
If I can, it's an optimization problem, and therefore I know how to attack it.
And don't be afraid to think about approximate solutions.
Sometimes you're just gonna not actually be able to solve the problem you want to solve.
And so you'll find a solution to a simpler problem, and that will be good enough.
Sometimes you can actually solve the problem by finding a series of solutions that approaches, but may never reach, a perfect answer.
Probably the third week in the course, we looked at Newton's method as an example of that kind of problem.
And then there are algorithms.
We looked at big O notation, various kinds of algorithms, a lot of kinds.
You've already - I sent a list of specific algorithms.
And particularly, we looked at optimization problems.
We spent a lot of time on modeling the world, keeping in mind that the models are always wrong.
But nevertheless, they're often useful.
They provide abstractions of reality.
So we looked at two kinds of simulation models, Monte Carlo and queuing networks.
We looked at statistical models, for example, linear regression.
And we looked at some graph theoretic models.
Those are not the only techniques, but they're very useful techniques.
We looked at making sense of data.
We talked about statistical techniques.
How to use them well.
How to use them badly.
We looked at plotting.
And we spent some time on machine learning, supervised and unsupervised and feature vectors, and spent a lot of time talking about how do you choose the features, because fundamentally that's typically the difference between success and failure in the world of machine learning.
Throughout it all, the pervasive themes were the power of abstraction and systematic problem solving.
So what's next?
Many of you have worked really hard this semester.
We know that.
And the TAs and LAs and I all appreciate that.
And I thank you sincerely for the effort you put into the course.
Only you know your return on the investment.
I hope it's good.
Remember as you go forward in your careers that you can now write programs to solve problems you need to solve.
Don't be afraid to do it.
If you like this, there are plenty of other CS courses that you now could take.
You're actually equipped, probably, to take any one of these four courses based upon what you've learned in 6.00.
You could major in course 6 or think about the new major in computer science and molecular biology.
And you're certainly qualified to go off and to look for UROPs that involve serious programming.
All right.
Wrapping up with some famous last words, these were words that some famous people said as they were about to die.
An actor, Douglas Fairbanks senior, was asked by a family member, how are you feeling?
He said, never felt better.
And then that was the last thing he ever said.
In contrast, the last thing Luther Burbank was reported to have said, a famous scientist, was, I don't feel so good.
He was a scientist rather than actor.
He had a better appreciation of the state of the world.
Conrad Hilton, who you probably think of as Paris Hilton's grandfather, but actually is more well known in some circles for running the Hilton hotels, his last words when his family asked him-- they knew he was dying-- is there anything you want us to know about running the business?
And he said, yes, leave the shower curtain on the inside of the tub.
And that's advice, I have to confess, my wife has given me on several occasions.
Archimedes, basically before he was taken away and executed, asked, could he please finish solving the problem he was working on.
You know, I could imagine some poor 6.00 student who's about to be carted away by the police saying, can I finish my problem set first, please?
Actually, I can't imagine that.
And finally, this US Civil War general, John Sedgwick, was reputedly-- and this is, I believe, a true story-- said, talking about the snipers from the other side who were quite far away, that telling his people, don't be afraid, they couldn't get an elephant at this distance.
And if you go to the site of the battle, you'll find this plaque describing the death of John Sedgwick who was shot and killed immediately upon saying this by one of said snipers.
It says something about generals that probably is still true today.
All right, quick reminders and then we're done.
There's a final exam.
If you haven't done the underground guide, go do it.
These are things I said at the beginning of the lecture.
I'm just repeating them.
Thanks a lot.
Good luck in the final 6.00, all your finals.
And then more importantly, have a great summer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So the way that I envision recitations going is I'll start off kind of reviewing some of the points, some of the high points from lectures, might go into detail on some of the more important topics.
And this is also a time for you to ask questions.
There's only 25-30 of you here, so you should feel a little bit more comfortable asking questions.
And there are no stupid questions here.
For the most part, you're all beginners at programming.
Am I wrong?
Has anyone programmed here before?
OK.
So presumably, none of you have any experience with what we're talking about.
And so if you ask a question, no matter how basic you think it is, it's not stupid.
Because everyone's been there, including me and the professor when we started programing.
And we've made, ourselves, really stupid mistakes too.
So.
That said.
We started out by talking about the purpose of the class, which is to teach you how to take problems, real world problems, break them down, abstract them, and divvy them up so that you can solve them with a computer.
And when we talk about computer, we're talking about a very simple model where-- and this is the only time you'll see anything related to hardware in this class.
When we talk about a computer, we're talking about something very simple with a CPU and memory and maybe some input and output.
All right.
And when we talk about programs, these are sequences of instructions that are loaded into the computer's memory which can be divided up into little cells like this.
All right.
And what they might look like is-- This actually doesn't mean anything, but it's just, you know, for the sake of demonstration.
This is what a computer would see.
As the CPU starts running a program, it looks at whatever's in this memory location and says, this is an instruction I can handle, so I'm gonna go do this.
So this might be like add two numbers together and produce a result.
Something very simple, very basic.
And as it moves along in a straight line fashion, it just executes this instruction, this instruction, this instruction, et cetera, et cetera.
The problem is is that while the computer is perfectly happy to look at this, this is gibberish to us.
And most people, most sane people, if they're actually looking at computer code, can't decipher this.
So this is where programming languages come in.
We can move up a level.
We can abstract away from this, and maybe say something like this stands for -- move a number into a register or something like that.
You don't have to know what this means.
All you have to know is that this is a little bit more readable than this.
And it represents an instruction that the computer can understand.
The problem is that even at this level, this is still very atomic and very low level, and really you can't understand what's going on.
So-- just so I can fill this in.
We move into say, I'm gonna say x is equal to 1 and I'm gonna add x plus 5 and then multiply by 2.
This is a lot easier for us to understand.
And this is what we're talking about when we're talking about any sort of programming language that we'll be studying in this course.
And we're studying Python.
So there are hundreds of languages that allow you to express these concepts in a form similar to that.
And all it does is it allows us to talk to the computer in this language.
So everyone good with that?
Cool?
All right.
So when we have these programming languages, they're put together in specific ways.
So does everyone remember the term syntax?
Can anyone tell me what it meant or means in terms of programming languages?
I could guess.
It is the way that the string is structured?
Yeah.
It's the way the parts of the language are put together.
So let's say that I take a very simple statement.
I'm taking a variable plus this variable.
This is a valid piece of syntax for Python, for example, or, say, arithmetic, or a bunch of different languages.
So this is good.
I can't write.
On the other hand, if we say variable variable plus, this isn't valid in Python, for example.
It might be valid in another language, but the syntax of Python won't allow this.
So this is bad syntax.
So.
All right.
So is everyone kind of understand what syntax is?
All right.
What about static semantics?
Talked about that in class.
Can anyone give me kind of a rough view of what that is?
Can I choose someone?
All right.
When we talk about static semantics, we're talking about syntactically valid statements that mean something.
So let's say that I have variable a is equal to 5 and b is equal to 2.
A statement that's syntactically correct and is also meaningful with respect to static semantics could be something like a divided by 2, right.
Or a divided by b.
They're valid statements syntactically.
And we know that 5 divided by 2 or a divided by b, they're both numbers.
They mean something.
On the other hand, let's say that I had this.
I'm gonna say that this variable d is a string 'foo'.
All right.
This c divided by d where c is a number and d is a string, that's something that's not meaningful, right.
What does a number divided by a string mean?
Nothing.
So that's what we're talking about with static semantics.
So bad.
These two aspects of computer programs are pretty easy to check for a compiler interpreter because they're pretty explicit rules, right.
The part that we spend most of our time in is the semantic part.
And this is where the syntax checks out, the statements are meaningful in and of themselves, but the program as a whole, or the whole kind of recipe, doesn't work.
So as an example, so this, as we know is fine.
That's correct.
But let's say we have this.
So a is 6, c is 0.
I do that, what's 6 divided by 0?
It's an error.
But syntactically this is fine.
And by static semantics, this is fine.
So this is the type of thing we're talking about with the semantic aspects of a program.
Does it work properly?
And before I jump into Python, one tip to keep in mind is that when you're writing programs and you're trying to do your problem sets, your program is very explicit.
It's not ambiguous.
So a program will do what you tell it to do and no more and no less.
And a statement doesn't mean two things.
So like if I make the statement in English, I cannot say enough good things or recommend this person highly enough, if you write that out that can be considered a statement with two different meanings.
One of them is not too complimentary.
All right.
So when you're writing your programs, if it's not doing what you think it's doing, one of the skills that you need to learn throughout the semester is to be able to read the code and try to follow along in your mind what's going on.
It's not magic.
OK.
So now we're at Python.
We're past kind of the generic introductory scaffolding stuff.
Now we're onto something that's actually gonna be meaningful for you for the next 12, 13, 14 weeks.
All right.
So Python is a general purpose language.
It's used for all sorts of things, web development, small device development, desktop programs, et cetera.
It's an interpreted language, which means that if I have a program, blah, blah, blah, blah, then Python executes it for me straight away.
Some languages, compiled languages, have to go through a compiler, and then you have to run them.
All right.
It's an extra step.
The nice thing about having an interpreted language is that as you may changes to the code, you can instantly see what's going on with your program.
So.
See.
It's got a very simple syntax.
And it's also very widely used and has been getting-- more and more people have started using it, oh, for the past, say, 15 years.
So.
10 years ago when I first heard about it, it was kind of like this little cute language.
Now it's kind of blossomed into this wonderful language that everyone uses and loves.
So.
Programs in Python, and in all languages, are sequences of expressions.
So this is getting back to syntax.
And these expressions, they are composed of operands and operators and functions.
So let's say, for example, this is an expression.
This is a variable name.
This is the assignment operator, and this is a string literal.
When I say operand, what I'm referring to are things in the language.
That's what a literal and a variable are.
And then if I say operator, this is something that does something to things.
So it's-- this is an example of what we call an assignment operator.
What it does is it says, take what I'm referring to my right hand side here, create a name called myvar, and say that every time that I reference myvar, I get this value.
We'll talk more about that later on.
Um.
Let's see.
Where am I going?
So one important thing to know about Python is that everything-- so these things-- are objects.
For now, you don't have to get too familiar with what an object is just-- if you know anything, especially for the first quiz, this is something to know.
These objects, they have types, right.
So some types that we have in Python are ints.
What are examples of ints or integers?
What?
7
Yeah.
Just rattle a few off.
So 7, 0, negative 1, 2, et cetera.
Now there's another number type, floats, right.
So this is what we normally think of as real numbers.
But when you're talking about real numbers on the computer, they are kind of dicey to deal with-- and we'll actually cover that later on in the semester when we talk about kind of the inexactness of these.
But for now, just know that these are numbers with like decimal points.
All right.
OK. Where am I going next?
[INAUDIBLE]?
Actually, syntactically, yes.
So one thing that you might encounter when you're dealing with your programs-- this is kind of off on a tangent-- is when you say assign a number to a variable, when you have a literal like this number here, 0, Python infers what type you're talking about.
So in this case, it's gonna create a number, and it's gonna call it a type integer.
All right.
If it sees this, it's gonna create this variable, and it's gonna have a type of float.
Now, it might seem like something minor to you right now, but when you start doing some of the math on problem set one, if you divide by an integer, you might run into problems, right, because in computer-land 5 integer divided by 2 integer is equal to 2.
Whereas in the real world, it's equal to that.
So just be aware when you're working with numbers that you need to be aware of their type, all right, especially when you're talking about the different operations you can do on them.
So.
So for these number types, you have addition, subtraction, multiplication, division.
For integers, exponentiation, which is represented by two asterisks.
And for integers, modulo.
And then for floats plus, minus, same thing except no modulo.
All right.
So back to the list
Can I ask a quick question?
Yes
Could you explain why like maybe briefly 5 divided by 2 is 2?
You said in computer-land, like what?
Is it doing some sort of weird rounding?
Or like why, why is that different than--?
This is because the type of the variable is an integer.
And when you say something is an integer, you're talking about only these numbers, right.
So 2 and a 1/2 as an integer doesn't exist.
It only exists as a float.
But when you have an operand like this that takes two numbers on both sides, if both numbers are integers, this produces a result that is an integer.
So when you have these math operators, these produce a result, right.
So that result has a type as well.
Because I can take that result and I say c is equal to 5 and 1/2.
C is now a variable in the language.
That means that it has to have a type.
And Python says that because I did this operation with two integers, I'm gonna give it a type as an integer.
And the closest representation to 2 and 1/2 in integer terms is 2
I thought it would round up to 3.
I assumed that it would round up to [UNINTELLIGIBLE]
Oh, yeah.
You're talking about a separate issue.
It's not necessarily that it's closest, but it truncates

So when it converts-- yeah, it just truncates the decimal
What happens if you have 5 divided by 2-- [UNINTELLIGIBLE]?
So when you get into stuff like that, Python's gonna look at, see it's an integer and a float, and-- this is something that we're gonna have to actually test out when I turn on the computer.
It should convert to a float, but it might not.
So.
We'll test that out, actually.
That's one of the nice things about Python is that if you have a question like that, you can test it out instantaneously.
So remind me ints divided by float, and we'll demonstrate it.
OK. All right.
So another data type, string.
Anyone know what a string is or can tell me?
Shout it out.
Pantomime it
Sequence of characters
OK.
So it's something like this.
All right.
And you can-- does anyone mind if I erase this?
So in Python, you can specify strings a couple ways.
One is with single quotes on the sides.
And the other is with double quotes.
This is useful, for example, if you need to embed quotes inside a string.
So if I need to, say, have this as a quoted string, I would have single quotes on the outside and then double quotes on the inside, or vice versa.
I could do double quotes and then single quotes.
The point is that Python needs to know when and where the string starts and where the string ends.
We good?
All right
[UNINTELLIGIBLE]
So if you put that?
If you write this, Python's gonna call that a string.
It's a string that contains the character two, though.
All right.
I guess you're asking because of the input with the problem set?
--divide that by a string, by an integer that would [UNINTELLIGIBLE]
So in your program, there's a point where you have to enter in numbers for problem set 1.
Oh, you didn't start?
Well there is a place where you have to enter numbers.
So raw_input, for example, is gonna return a variable, and the type of this is gonna be a string.
So if you need to use it as an integer or as a float, you can convert it.
So now this is an integer, and you can do math with it.
Or you can also say-- Does that make sense?
That's something that'll be useful in your problem set.
So just FYI.
All right.
Another data type, Boolean.
Does anyone know what a Boolean is?
[UNINTELLIGIBLE]
What's that?
It's like an if-then statement?
It has to do with if-then statements, but it's a variable.
It's a type that only has two values, true or false.
OK. And we'll talk about this, all of these, when I start talking about operations on them.
Last one that you need to know of right now is the none type.
The way to think about none is that it's sort of like dark matter.
It's there.
We know it's there.
It holds a place, but we can't do anything with it.
It's there just so that you know that there's a variable that should refer to something but that something doesn't exactly exist.
OK. You'll see more of it.
But it's nothing, but it's there.
And then later on in the semester, there's other data types.
I mean, there's a lot of data types, but these are the big ones that you need to know right now.
Other data types that we might look at are lists, tuples, and dictionaries.
These are other major data types in Python.
You don't need to know them right now 'cause Professor Guttag will go over them in lecture.
But just keep them in your brain.
OK.
So we've already talked about the operations that we can do on numbers.
We can also do operations on strings.
So we can do something called concatenation.
So concatenation is just a big word for sticking two things together.
So if I have s1 and s2 and I want to concatenate them to together, I use the plus operator.
So the other thing, too, that I want to point out is that we have a plus operator for ints.
We got a plus operator for float.
We've got one for strings.
I don't think they work for bools, but they work for lists and tuples.
This is what's known as an overloaded operator.
So it changes shape or changes its behavior depending on the data types of its operands.
So that's why when you're dividing an ints by a float, some languages will convert it to a float for you.
Other languages will convert to an ints.
It's something that varies.
That's why I'm not sure of my answer right now.
So
[INAUDIBLE]?
No.
If your computer blows up in your face, then that's an issue.
That's a separate issue.
All right.
So some other operations that we have, and they relate to Booleans, are comparison operations.
What these mean are these take two operands and compare them.
So if I want to see if-- if I say a less than b and a is 2 and b is 3, then this is gonna return true.
The value of this expression becomes true.
And if I say a greater than b, obviously it's gonna be false, right.
And all of these operators work in basically the same way.
They take two operations, and they give you a Boolean value back.
All right.
Any questions on that?
So are Boolean values always true or false?
Always true or false.
And I'm actually gonna get to that now.
So Boolean values-- and this is kind of the last major one we're gonna talk about-- have three operators, AND, OR, and NOT.
Actually, these should be lowercase.
These, AND and OR, take two operands, NOT takes one.
And what they do is if you have something like a is true, b is true, c is false, if you say like a AND b, then this entire expression is gonna return true, right.
If I say a AND c, this will be false.
AND returns true, if and only if both operands are true, false otherwise.
If you say a OR c, OR returns true if both operands or one operand is true.
So if any of them are true or both are true.
If they're both false, then it returns false.
And NOT, all this does is if I say NOT a, it'll return false.
It reverses it.
Now we can combine these together.
So these are very simple expressions, but we can also combine them.
So we can say like a AND b OR c, right.
So if both a amd b are true, then this becomes true.
And then this entire expression becomes true if this part is true or this part is true.
So you can build up pretty complicated expressions.
And then the way that they relate to these logical operators is, remember, these take numbers on either side and they produce Boolean values.
So if I have-- I can say d is less than e. This is gonna give me a Boolean value, right, which I can then use the AND operator on and I can say-- So what this does is it says if d is less than e and e is less than f, then return true.
This, by the way, would check to see that these numbers are in order, so 3, 4, 5, as opposed to 5, 4, 3.
So is everyone good on all that?
Did I lose anyone?
No questions?
All right.
So the last couple of things and then I'm gonna turn on the computer and actually walk through some code with you and we'll be done for the day.
So this is kind of the crash course in basic syntax for our basic types for Python.
So there were three-- So what we have now is a way to create programs that run in a straight line, right.
So can anyone give me kind of a synopsis of what a straight line program is?
Go down line by line
What's that?
Everything one-- go down line by line
Go down line by line, do everything once.
All right.
The problem is that this doesn't allow us to do anything, right.
So we have branching.
This is implemented by something called an IF statement.
Now, the way you use an IF statement is-- all right.
This is the full version of the IF statement.
It's saying, if this condition is true, I'm gonna execute the code in this block.
If this condition is true, if this condition is false and this condition is true, then I'm gonna execute the code here.
And if none of those were true, then I execute what's here.
You don't need to have an ELIF, and you don't need to have an ELSE.
You can just have an IF or an IF and an ELSE.
So the three versions of this branching-- Now when I, just by way of explanation, when I draw like a line like that, I'm talking about a block of code.
Can anyone tell me how Python represents blocks of code?
Indents?
Yeah.
So indentation, right.
We kind of talked about that on Thursday.
So blocks of code are chunks of code that belong kind of logically together.
So what this saying is that if I execute this, all this block is gonna get executed, all this block is going to get executed, et cetera.
So if I were to represent this pictorially, then this is kind of the main part of the program and then this is the if statement and the block.
And then it goes back to whatever code is down here, right.
If I represent this pictorially, this would be the true part.
So this part of code, this would be else.
And then this part would be multiple excursions.
So it would look a little bit like a tree.
All right.
So different branches for the different bits of code.
Is everyone puzzled on that?
Or anyone puzzled on that?
OK. And there was a last bit of flow control that we talked about.
What was it?
So if we want to do something multiple times?
Iterations or loops
Iterations or loops, right.
So it's called a loop because it looks like a loop in the code, right.
And there's two variants.
There's a WHILE loop and there's a FOR loop.
A FOR loop is when you want to iterate over a finite set of elements.
So what this FOR loop does is it says I'm gonna take-- this is the range function, so we'll talk about this a little bit later.
But all it does is it gives me all the numbers from 1 to 9.
It goes 1 past.
So.
But you don't need to know it just yet.
What this is telling Python, though, is that we're gonna execute this block of code 9 times, all right.
And on each iteration through this block of code, we're gonna set i to 1, and then set it to 2, set it to 3, 4, 5, 6, 7, 8, 9, 10.
Anyone lost by that?
We'll see an example of it pretty shortly.
And then the last one is a WHILE loop.
A WHILE loop executes as long as a condition is true.
So it's useful-- don't want to do that.
It's useful when you're not necessarily iterating over this finite set of elements or you don't necessarily know how many times you need to execute a specific loop.
You just know that you need to keep executing this code while something is true.
And we'll see an example of that pretty quickly.
So I basically shotgun blasted a whole bunch of stuff at you.
Is there anything that people want me to touch on before I pull down the screen and we start looking at code?
All right.
So has everyone been able to get Python set up?
All right.
So you'll know that this is the editor window, right.
So what we use to edit longer scripts in?
And you know that this is the interactive prompt.
Before I forget, let's see what happens if I divide the integer 5 by the float 2.
So Python does what you would hope it would do.
It turns it into a float.
If I do this, though, it gives me 2.
All right.
So it's an easy way to test.
So we've got two chunks of code here that I want to go over.
One of them, you've already seen.
It's the program that tries to find the cube root of a perfect cube.
And why don't we just walk through it and read the code?
All right.
So here's that raw_input function that I told you about.
Now, it's gonna take a string and it's gonna print the string out on the screen and it's gonna say enter an integer.
And, comment this out.
So let's enter-- what's a perfect cube?
All right, the return type of raw_input is a string, right.
So that int will convert to x.
Now, here's an example of a loop, the WHILE loop.
And because we don't have knowledge of what the user's gonna input when the program is run, a while loop is an appropriate kind of control loop to use, right.
Because we can test a condition where our guess, which is what's represented by ans, if we cube it, is it still less than x.
All right.
I explained that a little bit incorrectly.
What we're doing here is we're making a guess, and we're calling it ans right now, and we're gonna set it initially to zero.
And then we're gonna enter this WHILE loop and we're gonna say we're gonna take answer and we're gonna multiply it three times.
We're gonna cube it.
And if the value of ans cubed is less than whatever input the user gave us, then we're gonna keep looping.
And on each loop, we're gonna increase the value of our guess for ans, right.
And if it turns out that x is a perfect cube, eventually, by just iterating through all the integers from 1 to whatever, whatever the cube root of x is, we will find the answer.
And we'll know we find the answer because answer cubed is going be less than x.
Now, can anyone tell me why we have abs here?
The absolute value
Right.
So let's say that the user entered in like negative 27.
It still has a cube, right.
It's still negative 3.
But the way that we set up our loop, if we were to take this out, the program would just continue executing forever.
Well, for actually a very, very long time, but-- the universe would die before this finished.
So we have this absolute value here.
Is anyone puzzled by this?
Do I need to belabor the point?
OK.
So eventually I've entered 27 and we can actually-- a good way to kind of check ourselves is to print out diagnostic input.
So if guess is 1, that's not-- 1 cubed is obviously not 27.
2 cubed is not 27.
3 cubed is 27.
And when we get to that point, we leave the loop.
And now we're down in this bit of code.
All right.
Now, because we're asking for cube roots, we need to check the condition for when the-- sorry, I'm kind of mixed up here.
Sorry.
So when we exit the WHILE loop, if we've had a number that's not a perfect cube, then we know that the value the answer stopped on is not going to equal the cube of it, by way of explanation.
So my last guess 3 cubed is 27.
Let's run it again.
Let's say that I have 20, right.
So what happens is it gets to 4.
So 4 cubed is 64, right.
That's obviously not 28.
And that's what this condition checks.
And so it knows that if it gets to that point, it's not a perfect cube.
Now, this ELIF statement here says, OK, so if this turns out to be true, that means that it wasn't a perfect cube.
But let's say that it was a perfect cube, this will be true, and Python will then start to look at this and say, well, let's look at this if condition, or actually-- I'm sorry.
This condition will be false if it's a perfect cube, right.
And Python will say, OK, look at the next part of the IF-ELIF statement and will say, well, was x less than 0?
What it's doing here, it's checking whether or not we've entered a negative number or not.
So if we enter negative 27, what it would do is it would to enter this branch, and then it would negate whatever answer it got.
Because we found the answer for a positive perfect cube, right.
Did I break it?
We are having technical difficulties.
Oh, OK.
I'm not sure what happened there.
So anyway.
So we go about trying to find the cube root of 27, which is the absolute value of 27.
And we, of course, find it.
It's 3.
But then because we've entered a negative number, Python's gonna enter this branch and negate whatever answer we got.
Because we know that in order for it to be negative, then it would have to have been a negative number, right.
Is anyone lost?
Everyone good?
Do you need an exclamation mark for the--?
Oh, this one?
Yeah
So this is one of the comparison operators.
This stands for not equal to.
So if I have-- do a little Python work here-- if I have a is equal to 5 and b is equal to 6, if I say a is double equal, that's gonna check and see if they're the same value.
Obviously, they're not, so it's gonna return false.
On the other hand, if I say not equal-- or bang equal sometimes we call-- this will return true.
OK. Any other questions?
The one that you did 28 for, shouldn't it not print the cube root of 28 is 4?
Oh.
Yeah.
So yeah.
Should it be indented?
Because if we indented here, right, then what's gonna happen?
It's only gonna print out the cube root for negative numbers.
So what-- what's that?
[INAUDIBLE]
Well, again, if we put an ELSE statement, then it won't print out the cube root for negative numbers.
What we could do, and this is kind of a hackish way, is write it like this.
But that's kind of ugly, right, because you're repeating yourself.
And one thing about computer programmers is that we are the laziest people on the face of the earth.
Like, we'll spend 20 hours writing a program to do something in five minutes that we could have originally done in 10 minutes.
It's just our nature.
So anyway.
So we're repeating ourselves.
And while I'm speaking, this is the solution I came up with.
If I were to go back and rewrite this, I'd probably make it a little bit less convoluted.
So maybe, let's try this.
So we're gonna check if we were successful, and then we're gonna check if x is less than 0.
Now we're not repeating ourselves.
And we catch all the cases.
So let's make sure that my fix works because oftentimes when you write programs, you'll introduced bugs of your own.
So we've tested when it is a cube.
Let's test it when it's not.
OK.
So there we fixed it, I think.
One thing about computer programs is that this is a very simple case, but for longer, more complicated programs, it's almost impossible to get out all of the bugs.
But in this case, pretty confident that we were successful, right.
So does anyone else have any other questions on this bit, right?
All right.
The last thing that I'm gonna go over is something called a fizzbuzz program.
This is just a silly little program.
This is the English specification.
And this is kind of a first instance of where we're gonna take English and kind of break it down into code, figure out how to break it up, chunk it up, and abstract it into something that actually works.
So the problem is to write a program the prints the numbers from 1 to 100, but for multiples of three, print fizz instead of the number.
And for multiples of 5, print buzz.
And if there are multiples are both 3 and 5, print fizzbuzz.
So if it says numbers from 1 to 100, when you see something like this when you're trying to figure out how to write your programs, the first thing that should go off in your mind is, I probably need a for loop because I'm iterating over a set of numbers.
So we happen to know the numbers 1 to 100.
And here's a range function again.
And we'll talk about range next week most likely.
And all we're gonna do is first we're gonna get the string value of this number.
So remember how earlier you asked if, or one of the students asked, if we had a number in the string, if that was a number or not?
So like this.
That's what that STR function does.
So if I have a variable a is equal to 1, I can say s is equal to STR a, and s is now gonna be that.
All right.
And then what I'm gonna do here is I'm gonna check the integer value, so i, and see if it's evenly divisible by 3 or 5.
And the way that I checked that is I use a modulo operator.
That's what that percent sign is.
What this operator does is it takes two integer values and it returns the remainder after you've divided the left integer by the right integer.
So if I have 6 modulo 3, what's that gonna be?
It should be 0, right, because you can divide 6 evenly by 3.
The other hand, if I say 5 modulo 3, then that's going to be 2, right.
Yeah, I had to think about it for a second.
OK.
So that's all this does.
And these are two expressions that return a Boolean value, right.
Because this modulo operation is gonna return an integer, and I'm gonna use the equality operator to compare it to a number, another integer, 0, and that's gonna give me a Boolean value.
This expression also gives me a Boolean value.
And then I'm gonna combine them into-- using the or operator for Boolean values-- into a larger expression.
And then if this is true, then I know that I'm gonna have to at lease print fizz, buzz, or fizzbuzz.
Because I'm not gonna print the number, right, because it's a multiple of 3 or 5.
And so all this code does is just figures out if it's evenly divisible by 3, then I know I print fizz.
So I'm gonna concatenate my final string onto it.
And then if it's evenly divisible by 5, then I know I need to print buzz.
So then I'm gonna attach buzz onto my output.
And then I'm just gonna print whatever I'm left over with.
So to see this in action because we are way out of time right now.
So 1, 2, fizz, 4, buzz, 6, fizz, 7, 8.
Then we have fizzbuzz for 15.
So it seems to work.
Everyone follow that?
We good?
So I'm done for this recitation.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
By way of review, we're going to start off with loops.
So, what are two loops that we've covered in lecture or two types of loops?
FOR and
OK. Yeah.
FOR and WHILE.
I'm going to start with WHILE loops because that's what I have first.
Can someone walk me through the syntax of a WHILE loop?
WHILE condition
WHILE condition and what type is this condition?
[INAUDIBLE]
OK. And then I have--
[INAUDIBLE]
And then a block of code, indicated by indentation.
OK. Next question is, what does this code do?
Executes the loop and returns Boolean values
Right.
That's a little bit too high level.
Specifically, what does this loop do?
Prints the even numbers from 2 to 10
Perfect.
That's exactly the kind of comment we're looking for.
When you're doing your problem sets or you're just coding in general, you want to have a comment that describes in an abstract way what's going on in the chunks of code.
You don't want to have a comment that says assigns 2 to a.
Continues looping until a is 10.
Comments like that aren't really helpful because you can get that by reading the code.
So that's exactly right.
It's perfect.
I don't have candy to pass out, though.
Next question is what's the decrementing function?
The way to think of a decrementing function is that it's a statement that moves the loop closer to termination.
In this case--
10 minus a
Yeah, 10 minus a or a plus equal 2.
As we're incrementing a, we're getting closer and closer to 10 and that's going to cause us to kick out of the loop.
Next thing.
Is a a good variable name?
The answer is no, otherwise, I wouldn't have asked it.
But a better variable name might be even_number.
Because, instead of just a being an integer, now we know what it's supposed to represent in the code.
And it becomes clearer when we go to execute it or run it.
For loops, same thing.
Could someone walk me through the syntax for a FOR loop?
And first question, does this loop do the same thing as a WHILE loop?
If this is our WHILE loop, does this loop do the same thing?
Yes
OK. Let's see.
The WHILE loop printed out 4, 6,8,10 and 12; this is going to print out, 8 and 10.
The reason is where we have the increment, right?
If we had done this, they would print out both the same.
A little aside.
That was a bug I just caught.
Someone walk me through the syntax for a FOR loop
[INAUDIBLE]
A FOR keyword and then what do we have here?
A variable of some sort.
Then we have in keyword and then we have this thing here.
We're going to get to tuples in a second, but this is a tuple literal.
And for FOR loops, a FOR loop requires something that is enumerable, which means that we can take one element after the other and assign it to the variable i.
FOR loops iterate over enumerable items.
Thank you.
That's exactly what I was talking about.
If I'm speaking too low, just jack your thumbs up.
This construction is inconvenient though.
If we wanted to list all the numbers that we wanted to print-- 2 to 10 is not too hard.
But let's say we wanted all the even numbers between 2 and 100.
I don't want to have to write out all 50 of those.
So we get to the range function.
You've seen this before.
We just haven't ever explained it.
All the range function does is it takes between one and three parameters and returns a list of integers.
So let's say that I pass it one parameter.
I'm going to give it the integer 100.
This is going to give me nothing because I didn't print it out.
This is going to give me integers 0 to 99.
If I give it two parameters, what it's going to do is give me?
The integers between 1 and 100.
In this case, it starts off at 1 and it gets to 99.
One important point is that when you give it two parameters, the first one is inclusive and the second parameter is exclusive.
Does everyone follow that?
Finally, the third form is with three parameters.
I have a start, an end, start inclusive, end exclusive, and then a step.
This will give me all the odd numbers between 1 and 100.
We can also go in reverse.
If I want to go count down from 100 to 1, I can give it a negative step.
If you want help on using the range function, there's a handy command called help, which you can type in the interactive prompt.
This gives you the syntax for using range.
You can do this for any function in Python.
It's going to tell you, here's the name.
Here's a parameter start.
Here's a parameter stop.
Here's a parameter step.
This little arrow here means this is what it returns, a list of integers.
When you type help and you see one of these square brackets that means the perimeter is optional.
A start is optional.
Step is optional.
Stop, we always have a stop.
We always know when we're stopping a range.
That's why we can pass one, two, or three parameters.
Is anyone confused on that?
If you only put stop, it will start with 0?
Right.
If you only put one parameter, then it will start from 0.
So the start is implicit.
Are we good on this?
So, we can code our FOR loop like that, which for five numbers is not too inconvenient, or is convenient.
But if we have 500 numbers, it makes it a lot easier to just change one end point than it is to type in 500 numbers, right?
Can you create a range for floats?
So let's say you want to do it by float size?
That's a good question.
We should try it.
Why don't we give it a shot?
The question was can we make a range of floats?
Let's try 1.02 to 10.0.
It'll give us an answer, right?
But it also gives us a warning.
It says that an integer argument is expected.
Let's say I do this.
Will this work?
What's it's going to do is it's going to truncate the floats.
It's going to truncate the start to 1 and it's going to truncate the end to 10.
Then it'll just return the integers as required.
So you can, but it doesn't work the way you think it would.
Moving on, you hit tuples in lecture this week.
So we're done with the old review stuff.
Can anyone tell me what a tuple is?
It's a non-scalar data type that can hold many items.
What does non-scalar mean?
It's multiple elements that you can search individually
It's a field that you can hold only one value at a time
You're both right.
A scalar can hold only one element at a time.
A non-scalar can hold more than one element at a time.
Tuples are actually the second scalar data type that we've seen.
A string is the first.
A string can have multiple characters.
Tuples are flexible.
We can have tuples of numbers.
This syntax, the parentheses with a set of elements separated by commas, this is the literal syntax for a tuple.
All we're saying is that my tuple of numbers has the approximation of pi, 2, 1, -100.
We can have tuples of strings.
We can have tuples of anything, but you'll get to that in a second
Can you mix the data types inside of that?
Yes, we're going to get to that.
The question was, can you mix data types?
The answer is yes.
We'll get to that in a second.
To access individual elements of a tuple, we do something called indexing.
We specify an index by-- if I have tuple_of_numbers, my variable name, I have a left square bracket and a right square bracket, and I have an integer in between.
This would give me the item in the tuple that exists at index 0.
Tuples are indexed starting at 0 and in increments of 1.
When I say tuple_of_numbers[0], what should this print out?
3.14159
Right.
It will print out 3.14159.
Now if I change this to 1, what will this print out?
2
Exactly.
It doesn't matter what data type is contained in tuple.
If it's string, it'll just print out whatever's here, which is 'what'.
You can also use negative indices.
Negative indices tell Python that I want to go to one past the end of the tuple.
In this case, tuple_of_ strings, I'm out here somewhere, and then walk back 1, or however many the integer is.
I have here tuple_of_strings minus 1.
Python's going to go to somewhere around here and then walk back one and give me 'name'.
Now what if I do minus 3?
What's that going to print out?
Is.
Let's see who is paying attention.
What's this going to do?
[INAUDIBLE]
My name, or name, rather?
Remember we index tuples at 0, right?
That's going to give an error
Exactly.
It's going to tell us index out of range.
We have four elements in tuple strings 0, 1, 2, 3, oops, we're off.
You can get outside of a tuple and get an error.
To avoid that-- sorry?
The thing is when you said for tuple_of_numbers, you said 1 and 3.141, number, which would technically be 0 in this case, wouldn't it?
No, I switched it
You switched it
Yeah.
Just to make you feel better--
I see it now
You got it?
OK.
It's always possible that I made an error.
In your code, in order to avoid that, you can check yourself by getting the length of tuple_of_numbers.
So there's a function len.
It's going to tell us that tuple_of_numbers has 6 elements.
We can count them: 1, 2, 3, 4, 5, 6.
What's the last index of this tuple?
5, right?
Back to your question, tuples can hold different data types.
They can hold data types that are different from each other.
I'm wording that improperly.
So, here we have a float and three strings.
This is a heterogeneous data structure.
A homogeneous data structure would be one that you would say only hold ints only holds floats.
But tuples are very flexible.
So, we have a float here and some strings.
Then you could also have tuples that contain other tuples.
In this case, how many elements does tuple_of_tuples have?
3
3.
The first element is a tuple, the second element is 'got', and the third element is 'real'.
What should this print out?
[INAUDIBLE]
Yeah.
I had to sanitize that quote
Why does it print out the strings?
Python's behavior, when it sees a tuple data type and you call it with the print statement, is it represents as a string a tuple the way you would literally write out a tuple.
'Stuff, just' is a tuple with the two strings 'stuff' and 'just'.
Python is going to print out the literal representation of that tuple.
Another way to try and explain it, let's say that I print out the entire tuple.
What should this look like?
Well, It's going to be just the literal representation of the tuple.
What does it mean for the data type to be immutable?
You can't change it.
You can't add another element to it?
You can't change it.
If I try to change the first element of tuple_of_numbers, it's going to tell me it doesn't support-- or Python's going to tell me it doesn't support item assignment.
You'll see this error.
Tuples support something called slicing, which means that if I have a tuple_of_numbers and I give it a-- where I normally put just a single integer for an index, if I give it a start index and an end index, separated by this colon, Python's going to get the item at index 1 to whatever this end element is minus 1.
It helps if I just print it out.
To make it a little bit easier to follow.
I've said I've told Python that I want a slice out of this tuple from 1 to 3.
What Python's going to go do is look into tuple_of_numbers.
This is element 0.
This is index 1; it's going to pull in 2.
This is index 2; i it's going to pull in 1.
Then 3-- in Python, we go 1 past the end of the range that we want.
It's going to return a tuple of 2, 1.
This is slicing a tuple.
Anyone confused by that?
No?
Wow.
There are many different ways that we can slice a tuple.
We can have an implicit start.
If you see this, where there's no number before the colon, that tells Python start at index 0.
Then, in complimentary fashion, if you see this, where you have an integer on the left side and nothing on the right side, it tells Python go from index 1 all the way to the end of the tuple.
What happens is I do this?
What's that telling Python to do?
[INAUDIBLE]
Yeah.
It looks redundant but it will become important to us when we get to lists and something called aliasing.
What this does is it tells Python to take a slice that is the entire tuple.
Then when we do these slices, we can also use negative indices.
This is going to tell Python to go from 0 to the last element minus 1.
It'll look like that.
Everyone following?
We good?
It was minuses from the end--
Yeah
--of the tuple?
Okay.
I wonder if this would work?
I don't know.
No, it has unexpected behavior.
I don't know why.
I'll have to look that up.
Tuples are also-- we already said earlier that they were enumerable items, right?
So we can use a FOR loop with them.
If I want to print out all the numbers in tuple_of_numbers, I have my FOR loop, my variable, in, and then my enumerable object.
OK?
I have a question now.
Who thinks this will work?
We know that tuples are immutable
Yeah.
But you have to change the inside
Right.
It's going to work.
I still have print up here.
What I'm doing is I'm taking tuple_of_numbers and I'm printing before.
Then what I'm doing here is I'm telling Python to take this tuple and this tuple and add them together and reassign it back to tuple_of_numbers.
So, It looks like I'm modifying the tuple.
But in reality, what's happening is I'm creating a new tuple.
Let's say I have ton here.
Ton is short for tuple_of_numers.
Originally, it's telling Python that some chunk of memory has a tuple of 1, 2, 3, 4.
When I say a statement like ton equal ton plus another tuple, what it's telling Python to do is create another chunk of memory that includes ton, whatever's in ton, and the other tuple.
Then this assignment statement tells Python that ton now points to this new object.
You had a question?
Does Python have a garbage collector?
It's an advanced question.
The question was, does Python have a garbage collector to discard this memory that's no longer being used.
The answer is yes.
If you don't know what a garbage collector is, don't worry about it.
You don't need to.
But to answer your question, yes, it does.
I don't want to get too far into it.
Does everyone follow that?
That also is going to be important when we get to lists and aliasing, that type of object creation modification.
We won't get to that for a while.
Everyone's good with this?
I can move on?
All right.
Python has what some might consider a wart, when it comes to tuples.
That is when you want to create a tuple with a single element in it.
People just starting out with Python would sometimes mistake the tuple literal of a single element to be this: So, parenthesis with a single integer.
The problem is that parentheses are the grouping operatorS in Python, but they're also used for making tuples.
They serve a dual purpose.
What Python will say is, oh, I've got a number between two parentheses.
Well, this person really wants this integer to have high precedence.
We're going to make an integer 50 and assign it to oopsie.
But it's not what we want.
We want a tuple with 50 as one element.
The way you do that is you have a lone comma after the first element.
If we run this and we look at what it prints out, you see that oopsie has the integer 50, which is not what we want.
We want what's in onesie.
Anyone confused by that?
We're zooming along here.
Strings, they're actually a lot like tuples.
They're immutable.
You can't change them.
They're non-scalar.
They have multiple characters within them.
If I printed out-- everyone's seen this-- I can get to individual characters.
If I want to get to the first character in name, I use 0.
If I want to get to the second character, I can use 1.
Then of course they're immutable, so that's going to tell me that I can't do that.
They also support iteration.
So, if I want to print out all the letters in name, one on each line, I can do this.
Not too useful, but it works.
You looked confused
How did that happen?
How did that happen?
So, name is a string.
Follow that?
String is just a bunch of characters.
It is enumerable, meaning that we can go one character at a time through the string.
That's what the FOR loop does
Do spaces count as characters?
Good.
The question was do spaces count as characters.
The answer is yes, they do.
If I write spaces there and I run this again, as I iterate through the string, I get a space where I'm supposed to.
Is everyone good with this?
OK. Like tuples, you can take slices.
That's going to give me 'it'.
Strings also have many functions.
This is an incomplete list of functions that you can use on string objects.
I can make everything uppercase, lowercase.
I can also find characters.
What find does is it finds the index of the left-most character.
If I want to find i, it returns 1, because i is at index 1 in the string.
I could also do something like this.
I can find an entire string.
'tch', the substring, starts at index 2
What does it return if it can't find it?
If it doesn't find it-- let's put in garbage-- you get negative 1.
We can also call a replace function.
If I want to replace m with p-- it doesn't make sense anymore-- I can do that.
The question is, how would I use this to change the string?
Think back to how we modified the tuple.
We created a new object and then we assigned it to a variable of the same name.
If I know that replace is going to return a string with m replaced, I can do that.
Does everyone follow that?
Is the version that that captures your string function?
That's a good question.
That also gives me a perfect opportunity to demonstrate another command that you'll find helpful.
If you're working with an object like string, you can use a command in the interactive editor called dir.
If you type dir str, it's going to return all of the symbols that exist within the str object.
You can also type help str, and it will give you a nicer version of this
In this case, the first character indicates what you're trying to replace and the second is what you're replacing it with?
Right.
We can do help str replace, and it'll tell us
OK.
So it did one.
And then the count in this case would be if you had multiple instances?
Yes
OK.
So like if you had your name a couple times, you could replace it a few times
Right.
If I were particularly narcissistic that day, I could replace it multiple times.
So let's demonstrate that because some people might be confused.
Let's say, I want to replace t. I'm going to replace it with r. Whoops.
Well, why didn't it replace these to t's here?
[INAUDIBLE]
Right.
In strings, you differentiate between lowercase and uppercase.
If I wanted to get the behavior that I was hoping for, I could do it this way.
Now I have something that's not even remotely looking like English.
Or, I could do something like this.
I'm going to make everything lowercase, and then replace the t
So that's pretty exact that you have to be, lower--
Remember what I said first recitation that computers will do exactly what you tell them to do and nothing more or less.
There we go
It's good though.
It figured out [INAUDIBLE]
Python is a very nice language.
It's one of my favorite languages right now.
Because, it's not as fussy as some other languages, like MATLAB or C or C++ or TCL.
Anyone of those.
Is everyone good on strings?
I can move on?
Just remember, if you need help on any of these commands, just remember you have the help command at your disposal
It'll automatically default to do everything, unless you tell it something to do like two of the letters
Yes.
We didn't demonstrate that.
Demonstrations are worth a thousand words.
If I wanted to, for whatever reason, only replace the first two t's, I could tell it to only replace two t's within the string.
It leaves the other t alone
How would you pick out the last field?
That's a good question.
I don't have a ready answer for you.
You see these functions with r in front of them?
The string object often has functions that will start from the right side as opposed to the left side.
The find command I showed you has an rfind command as well.
This guy right here.
I'm pointing on if there is the analog to replace.
Can I get back to you?
I don't want to spend all our time searching for this and then fail spectacularly.
I'll get back to you on that.
Is it possible?
Yeah.
Is there a one-liner function?
I can't tell you at the moment.
But again, that's why you have dir and help.
You can find this out on your own.
Next thing, BREAK.
We're done with strings.
We're done with tuples.
We're still working on loops.
Everyone's seen this function before, right?
This is our find-the-cube-ro ot-of-a-perfect-cube function.
If I give it 27, and I tell Python to stop printing my name, that's what this does.
As a toy example to illustrate the BREAK statement-- first of all, can someone tell me in English what the BREAK statement does?
Would it just stop the program
Well, It doesn't stop the program, but it kicks you out of the loop.
We can rewrite the cube root program to work like this.
Instead of our stopping criteria being answer cubed less than the absolute value of the number, we're just going to tell the FOR loop to go from a range 0 to the absolute value of the input plus 1.
Why do we have the plus 1 there?
So it goes to the absolute value of x
Yeah.
Because, otherwise, it would go to 1 before the absolute value of x.
Then we break out of this loop, if answer cubed is equal to absolute value of x.
As soon as we see that it's equal to x, we're going to call BREAK.
That's going to immediately kick us out to here, without executing any more of the FOR loop.
Do people understand that?
Are people good with that?
Does it break you of the innermost loop?
or [INAUDIBLE]
Yes.
Good question.
Does it break you out of the innermost loop or all the loops?
The answer is the innermost loop.
When he says innermost, what he's talking about are nested loops.
Let's say I have something like this.
I'm creating on the fly now, so excuse my inability to type.
What is this going to do?
This is an example of a nested loop.
We have an outer loop here, then we have an inner loop here.
All the outer loop is doing is it's going from the integers 0, 1, 2, 3 to 9.
And then we have an inner loop, which looks like it should go from the integers 10 to 100.
But we have the statement in here if j mod 2-- that's what the percent sign is, modulus-- is equal to 0.
What we're saying is if j is evenly divisible by 2, we're going to break.
Now, the question is-- this is obviously going to break when j is 10.
This loop is going to execute once.
The question is does it print out only one set of i,j values, or does it print out 10?
I'm getting to the answer to this question.
If BREAK statement breaks out of all the loops, then we would only see one printout of i and j.
But if it only breaks out of this inner loop, then we should see 10, followed by 'here'.
So it breaks out of the inner loop.
Long answer, but demonstration.
Is anyone confused by that?
Is so, anyone too shy to admit it?
There's office hours.
Or, you can talk to me afterwards
Now we're going to get to functions, which are triple underlined and circled over here because they're extraordinarily important for you to understand.
Can someone wing it and tell me what they think a function is?
It's a snippet of code that takes some input, does something to it, and returns some output
Perfect.
It's a bit of code.
It's named, so you can refer to it.
It takes input, does something with it, and returns something, some value.
The way that we define a function in Python is like so.
Let's say I have a function cube.
It consists of a few parts.
We've got the DEF keyword.
We've got a name, cube.
The name should be meaningful for functions.
Like variables, the name should mean something So, this would be a bad name for this function.
You want it to be meaningful.
It has a set of parameters.
In this case, it only has one parameter.
This is what we pass to the function when we call it.
We'll talk about that in another second.
It has this string.
This is called a doc string.
It's the specification for this function.
When you write functions, it's good to have this string here.
What it allows you to do is describe what the function does, what it expects for input, and what it gives you as output.
In this case, it's very simple.
This is the body of the function.
Again we denote the block by indenting.
All it does is it takes a number, which is passed into it and raises it to the third power.
This RETURN statement tells Python to send that back to whoever called the function.
As an example if this working, let's look at this line of code.
What it's doing is-- we've seen the print statement, so we know what it does.
It's going to name cube and it's going to call it.
Python knows it's calling it because it's got the name of the function with the input parameters.
So, 3 is being passed to cube to be, well, cubed.
What's going on here is that Python is breaking out of its normal flow of execution, sending 3, calling it number in the function, and then raising it to the third power.
If we run this, we see 27.
We can pass it any number that we want.
Is anyone confused by this?
The things you're running between quotation marks, doesn't that kind of--
They don't do anything.
The question was, this string here between the quotation marks, this doesn't do anything.
No, this is a comment.
This is so that you can tell yourself, six months down the road, what you were thinking.
Or so that you could tell another programmer what this function does.
It's a way of documenting your code
So cube in this case is something that is built to go around
Yeah.
This is just a toy example.
I wanted to keep it simple.
I'm illustrating concepts.
It wouldn't be too hard just to do this.
I'm going to move on.
Everyone's good with functions?
[INAUDIBLE] Can you say x equals number-- can you keep the number, and just keep the number, and just make it into a variable without the RETURN function.
Do you have to use the RETURN function on that?
I was just getting to that.
The question is, do you have to use a RETURN function.
The answer is well, it depends on what you want to do.
So, let's take a look at a new function.
Can someone tell me what this does?
First, what does the function do?
Or what is it supposed to do?
Takes the number and doubles it
You got that by reading the writing on the wall, right?
Really, it's too easy.
In the body of the function, all it does is it creates a new variable, answer, and it assigns number times 2 to it.
What's going to print out here?
Why don't we run it and see?
That's not doing what we wanted.
When you don't have a RETURN statement, Python returns implicitly none to whoever calls the function.
Ok.
In this case, this function obviously doesn't have a RETURN statement Python says, OK.
I'm just going to return none.
Whatever work it did in the function is lost in this case.
To get the right functionality, we have to add a RETURN statement.
And it works.
Was that what you were getting at?
OK.
Anyone confused?
Can I move on?
Functions have something called variable scope.
I apologize for the punning.
I was getting tired when I wrote this.
In this chunk of code, I define a global variable, all_hope.
I also define a function that takes a parameter, variables.
It says it steals all the variables, but I don't know how it does that in a computer.
It doesn't return anything.
In the body of the function, I create another variable, called my_variable, and I assign it a string.
I'm not actually returning anything right because I've said I'm not returning anything.
All this is doing is it's just printing stuff out.
It's printing out what the parameter passed into it was.
It's printing out variables.
It's printing out all_hope, which is a global variable that we define up here.
It's printing out my_variable, which is a local variable in the function.
Down here I'm defining a variable, old_meme_is_old.
I'm calling the function.
It does what we expect it to do.
What I want to illustrate, though is what happens if I do this?
As you might expect, it's going to give me an error, right?
The reason is that my_variable has local scope to this function, all _your_vars_are_belong_to_us.
Is anyone confused by this?
No one's confused by this?
OK. Let's try something else.
Let's not do that.
If I run this code, we can tell from the function that it's taking one parameter.
It's incrementing that parameter.
It's incrementing this global int that we've defined up here.
It's returning the parameter that it's just incremented.
Erase my corny humor.
The question now is, let's say that I have a variable y, and I give it a value of 10.
I'm being completely arbitrary.
If I call the variable, inc_it, on y, first of all, what's going to print?
If I print out the value of y, what's going to print?
We want to run it.
Uh-oh, I'm failing.
I'm stepping on myself here.
This is why you don't debug code on the fly.
If I have a global variable and I need to reference it, I use a global keyword.
You should never have to do this.
Ink_it is going to return 11, right?
Because it's taken y, which is 10, and it's called x equal x plus 1.
Then it's returned this x.
If I look at what y is, after I've run this, y is still 10.
This is because when we've passed in x here and we've called x equal x plus 1, it's actually shadowing itself.
It's overriding what's in the local parameter.
But it's not overriding the actual variable, y.
We'll get more into this later on.
The important thing to understand in this case is that the changes that you make to this parameter stay within the function.
We good?
All right.
I need to move pretty quickly now to gotchas.
I guarantee you someone's going to make this mistake.
Print is not RETURN.
When you call print print_is_not_return, it's going to call this function.
It will print out this string, but it's not returning the string.
What it's returning is none, because there's no return statement.
That's where this none comes from.
RETURN is not print.
So, if I say print this return value, it will print this; it will return a string.
But if I just call this, it's not going to print it again.
It's going to print nothing.
You'll just have to make the mistake.
One thing to be careful of in Python-- remember I said everything is an object?
Functions are no different.
Functions are objects.
If you just reference the function's name, cube, which is not defined now, it's going to print something out that looks like that.
This is what the object looks like to Python.
In order to call it, you have to have the parentheses with the parameters.
Python's not going to complain.
Some programming languages will complain.
Python won't.
It's possible when you're running your code, if you're trying to call a function and you forget the parentheses, and Python's just not complaining, it will merrily do what you tell it to do
[INAUDIBLE]
What's that?
[INAUDIBLE]
I'm sorry, can you speak up?
Print q open parentheses [INAUDIBLE]
OK. Is x defined?
[INAUDIBLE]
Yeah.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's start.
I mean for the first question you can answer what is tuple.
You can go to that later.
Can a tuple contain a list?
What do you think?
Can a tuple contain a list?
Yes
Yes, it can.
So we'll go through the examples later.
And let's see this first example.
We have two tuples.
The tuple B contains tuple A, as well as a list.
So it can't populate.
Let's bring those lists, too.
So for the first example we try to access the elements 0 and 2 of the tuple A.
And let me use minus 1 to access which element?
[INAUDIBLE]
OK. And 0 is the first element, of course.
And remember now we are using 0, 1.
OK, what is tuple B?
It's a two-dimensional tuple
[INAUDIBLE]
Yeah, exactly.
It has a list and a tuple inside.
So the first element is a tuple, and the last element is the list.
So now it's sort of like two-dimensional.
But if you look at the first element of it, the MIT word, that doesn't have any dimension.
Does it?
It's a string.
So you can again access, but not this way.
But still you can access.
And we are using this notation to access a portion of the list, a part of the tuple, actually, here.
It's called slicing.
So here 0 to 1 gives you which element?
Just one element right?
It's the first element.
0 to 2 gives you 1 and 2 OK?
This is an interesting usage of the slicing.
This is used to access the whole tuple.
Now you might wonder why we need them.
We could just-- the handouts are here.
So why we need to copy or why we need to access the list through the column operator like this?
We'll look at that later.
It's very essential when we have lists.
For the timing it's OK when tuples are immutable.
So it doesn't have a special usage.
But later on we'll be using it.
Then if you look at this part, this is the most interesting part.
How do iterate through a tuple?
You use a FOR loop and you call like item in tuple, the element in tuple.
The element is initialized with every element when it goes through iteration.
At every iteration it will be instantiated with the corresponding element in the tuple.
But you're using the same name anyway.
It's item.
This is equivalent to accessing the tuple like this.
Going one-by-one in the range and accessing their corresponding element in the tuple.
So this is the simpler way to access that.
Do you have any questions in that part?
Yeah?
For that part, don't, usually if you have a range you have to [INAUDIBLE] name because is not--
OK, that's correct.
.
OK.
Suppose a tuple has 3 elements, or for example in this case tuple A, how many elements are in the tuple A?
3 elements.
So what would be the length of tuple A?
3
3.
But what would be the range?
0, 1, 2, right?
So you have to type 0 to 3.
So that's why I put that, OK?
That is 3, OK?
You
So you just say a range like the tuple is, it seems like the first one--
First the 0 if you don't specify it the same way.
OK. Now let's go through the list.
So if you go through the questions, what is the difference between a list and a tuple?
What is the difference between a list and a tuple?
Anyone?
Yeah?
Tuples are immutable
OK.
Lists are mutable.
And because it's mutable it takes special functions to access the elements, add elements, and modify the list itself.
How would you add an element to a list?
Suppose you have list A.
How would you add 3 to this list?
[INAUDIBLE]
OK. Let's start our print.
OK. That's good.
OK?
How would you remove the element from the top of the list?
Or the last element, how would you remove this 3 from the list?
[INAUDIBLE]
This one?
Let's see.
You know how to check the specification of a function, right?
Sorry.
So it's not the right function, right?
Because it says it removes the first occurrence of a particular value you are parsing to that function.
OK, there's another function.
It's called list.pop.
That removes the last inserted value, so it pops from the top.
So remove can be used to remove a particular element.
So here suppose we want to remove 1.
So we could say list A, remove 1.
OK.
So in this example, actually, I have commented out this line.
This will be an error if I execute.
Why?
Why will it give you an error?
[INAUDIBLE]
It doesn't exit.
Great.
So the problem here is we can't access an element in the list unless we explicitly assign something or create something.
But suppose we want to pass a list of 10 elements to a program so that it can assign values on the fly.
And it can modify.
OK?
So how would you pass an empty list of 10 elements?
OK.
The problem is if you're passing something like this, you need to know how this list should be instantiated.
So you should know what are the initial values of these elements.
Suppose you want to create a list of ten 0's.
Then the easiest way to create the list is-- So you would create a list with ten elements.
But If you don't want to waste memory, or if you don't want to keep anything particular, you could do something like this.
Now this will create a list of empty list.
And there'll be ten such empty lists.
OK?
And you could pass this list to a program so that it will assign the elements.
OK?
You can iterate to the list the same way.
But lists are interesting.
Why?
They are mutable.
So they can be actually used to create interesting mathematical objects.
For example, like matrix.
So how would you create a matrix using lists?
Any ideas?
How would you create, say a 2 by 2 matrix?
If you consider this example, this creates a matrix 4 by 2.
So that would be like 4 rows and 2 columns.
But how can you go and access the inner element or inner matrix?
So if you suppose you want access 2, 3, that is the third row, then how would you access?
How would you call M?
M3?
M3.
M3 gives you 2, 3.
Suppose you want to access the 3 and the 2, 3.
Then we-- [INTERPOSING VOICES]
Sorry
Why does a 3 give you [INAUDIBLE]?
Oh, sorry.
My bad.
Yeah.
OK.
This is computer science, not maths.
You got it right, why it's not 3, it's 2?
Yeah.
I thought you were talking about 3 and 4
OK. That's fine.
So I'm asking now how would you access the 3 in the 2, 3 box?
So I need to do another parentheses?
Not a bracket

OK. That is all.
So you can access matrices like this.
OK. Great.
We are on time.
So this actually gives you a summary of functions.
Associated functions.
For example the pop I explained earlier.
Remove removes a particular element.
Extent, that's interesting.
Can you tell me what it does here, the method extent on list?
What does that method take?
It takes another list.
So which means it's going to merge the list you are passing with the list you're calling the method on, that is [UNINTELLIGIBLE].
So it's going to combine those lists.
Let's see.
So this corresponds to this.
So two lists are combined.
So this method is actually useful, right, because you don't have to write yourself.
Most of things you can easily do.
Now the question is, why tuple didn't have these methods?
Why?
Why we didn't have these methods for tuples?
[INAUDIBLE]
They're immutable.
So you can't extent, you can't [UNINTELLIGIBLE].
So even if you extent, what would happen?
It's going to create a new tuple.
Actually, you can do that.
But like this.
For example, you have tuple A which is 1, 2.
And suppose you have tuple B -- 3, 4.
If you want to combine them, how would you do it?
Tuple A plus tuple B. OK?
That concatenates.
This operator is all loaded for tuples.
OK?
So lists are mutable so you can do all of these things.
Let's look at a few other examples, too.
OK.
There's an interesting part here.
Lists are mutable.
That we understand.
But how these lists are actually stored in the memory, For example, suppose you have a list lst, sorry, 0, 1, apple.
How do you think it's stored in the memory?
So you have 3 elements.
But these elements are actually not stored in the list itself.
They actually in the memory.
0, 1, apple.
But you have the pointers to those lists.
Sorry, to those elements.
So actually you can modify these values without changing the list itself, because the list actually points to this part, the container.
So in this example you have list A and B.
But the list B actually contains list A. OK?
Since list A is mutable you can go and change it.
So here first you print list A and B, but you're going to change list A's 0th element.
After changing the element the list A becomes first.
Its first element changed to 88.
OK. Fine.
What do you think the list B would be?
OK, let's print.
It contains the 88, right, the modified value.
Why?
Because list B actually had a pointer to list A.
Which again had pointers to other elements.
And we modified this value to 88.
So you're still accessing list B. OK?
The pointers are not changed.
That we can see here, in the second example, the problem of aliasing.
So we have list X.
And we call it by list Y as well.
And we are now changing the list Y.
Sorry.
We are changing the list X.
But it's going to affect list X as well as list Y.
See?
Because it's just a reference.
The actual object is this.
You can call it by two different names.
It doesn't matter.
But the actual object is here.
That's where we need this operator.
So if you just point, OK, for example we had list X, if you just assign list Y to list X, it's not going to copy the elements, just going to copy the reference.
Which means list X and list Y, both would be pointing to the same object.
If you want to copy these elements and put in a new one, say, list Z, then we need to copy one-by-one every element.
For that we can use this operator.
It's called full slicing.
It's not going to modify elements.
It's going to copy the whole list of elements.
And in the same line, just another small example that tells you how to cast a list into tuple, and a tuple into list.
So you had a tuple, one for apple [UNINTELLIGIBLE] and you cast into list.
Now the type is list.
It's a very simple operation.
Can you do that yourself?
How would you do it if you want to write a function to cast, how would you do that?
You go through element by element and add it to a tuple.
And then concatentae two tuples.
Right?
So I guess you can write probably as a homework, try that.
Try how to convert a list to a tuple without explicitly calling the casting function.
OK.
Similarly you can convert a tuple back into list.
So it's pretty straightforward.
And suppose you're getting an input from the user as a tuple.
then suppose you want to make it a list.
You don't have to write your function.
You can just call this casting operator.
So it'll be-- it will save some time.
OK. Great.
Let's go to our dictionaries now.
So we have a third data type, dictionaries.
Oh, you didn't get the second one?
So we have another data type, dictionaries.
And why do we need dictionaries in the first place?
Do we need another data type?
Can we just get away with tuple and list?
Yes.
Because you can actually make a dictionary out of lists.
And Professor Grimson went through that in the lecture.
If you want, you can come and ask in the office hours.
But first try whether you can do it yourself.
But the problem is, actually not just the dictionaries, but the high-level data structures are actually available in Python as building data structures or as classes.
Just because they will have these methods on these data structures implemented.
So it would say some time for us.
Plus those methods are guaranteed to be state of art.
For example, if you want to go through all the lists, then the search function or the lookup function that you write may not be that efficient, right?
But they might use those efficient algorithms in the standard implementation.
So it's why you should always look for the standard data types.
If you can't you can write yourself one.
But it's good to use the available data types so that they would have this efficient method implemented.
OK.
So in this dictionary, we have two elements.
One is key, one is value.
OK?
So these keys actually have a special property.
What is that?
The keys should be an immutable object.
So you can actually have a tuple as a key.
You can have a string as a key, but not a list as a key.
So you need an immutable object for the keys.
What about the values?
They can be anything.
Values can be even a dictionary.
It can be a list.
But the keys must be mutable.
Do these keys need to be unique?
Yes.
No?
OK.
The problem is this.
Dictionaries have the same type, I mean structure.
This is a key.
And this key points to a place in the memory.
Suppose you call it key A. OK?
If you assign another value to key A, that will actually replace the content in the memory, right?
So that's why keys are going to be unique, It's by construction, by the construction of the dictionary the keys are going to be unique.
Because if you want to assign something different, you have to call it by a different key, call it by a different name.
Say key B.
What about the values.
Do they need to be unique?
No, of course.
Otherwise no use of key, and no use of dictionaries, right?
What is the order of the items stored in the dictionary?
Can you give a guarantee like in lists?
In lists items are stored from zero to the left, right?
Length minus 1, actually.
What about dictionaries?
Surely can't guarantee the order, but you can modify the order.
OK.
So the thing is first, if you look at this example, first we have the staff dictionary.
And when I pin the length it gives me 3, fine.
Then I'm doing three things.
First I change the address by calling its key.
And dictionaries are mutable so you can change their values.
Then also we are adding a new element.
Fine.
If you wanted a new element you call it by the key that you already assigned.
If the key already exists you will just modify the value.
Otherwise it will create a new key and add that value.
You can also check whether this particular element is in the dictionary.
But this must be the key.
Whatever you are calling here should be the key, not the value.
It won't search for the value.
It will search only for the key, [UNINTELLIGIBLE] for that particular key because you access the values through keys.
Again here if it is not in the list, actually I'm adding an element-- OK. OK. To compare in the list, suppose I have a list 0, 1, 2.
If I want to check whether 1 is in the list, how would I check?
Do you have to go through element by element?
No, there's a shortcut.
If 1 in list, right?
If I want to check whether 1 is not in the list, if I want to negate this, I would write it 1 not in list.
We just ignore the words.
It's pretty much like English.
OK?
We just remove the words.
OK.
In the dictionary we do the same thing through keys.
If key A and, OK I'll call it, say, D1.
OK?
And you actually call or you'd search for the key, not the other way.
OK.
There is an interesting part here.
If you want to modify the order-- actually I tell you that the order C earlier, we couldn't guarantee the order, right?
It was not the order we typed because it starts at [UNINTELLIGIBLE] because we added at the end actually, right?
So actually you can't guarantee the order.
But we can sort it.
But how do you sort it?
We call the keys and we sort the keys.
Because remember, every values-- if you want to access a particular value, you access it through key.
Actually, if you call the dictionary, it doesn't know where these values are.
Just you have to go to the key to access the value.
So if you want to do something in the dictionary itself, you can do that only on keys.
So if you want to sort the dictionary, you sort it by keys.
So you call the keys method for the dictionary-- so it returns the list of keys and you sort them.
This is called chaining methods.
So I have chained the methods.
I would have like two methods, right?
I call the keys.
This first part actually returns to this top key.
Then you sort them.
Yes?
It doesn't look like they're in alphabetical order?
Oh, on this?
Sorry.
Let's see.
OK. Let's see.
Ooops.
OK. Now they're sorted.
OK. What was the problem?
What was the problem there?
Why it wasn't sorted?
You have to do it in two lines
Sorry?
You have to call up the keys and then sort it in a separate line?
OK.
The problem is in the logic.
Actually, when you call this function, method, you're actually sorting the keys returned by this method.
You're not actually going to sort of the dictionary itself.
You're sorting only the list that was returned, right?
The list of keys.
Although the dictionary is mutable, it wasn't sorted.
Do you see the problem?
So that's why when they called keys equal to staff.keys I'm getting a list of keys.
Then I'm sorting that list.
And I'm printing that list.
But if you want to go in a particular order, if you want to access the dictionary in a particular order, what could you do is you could do something like keys is equal to staff.keys.
Then you can store keys.
Then for k in keys you can go and iterate now, right?
So you can say print k and staff k. OK?
So you can do this.
But actually, Python provides a way to iterate what keys and value pairs.
That you do by calling both elements?
All the method items.
The items return a list of key-value pairs.
If you call D1.items you get a list of key-value pairs.
You see that?
And we are going to iterate through individual elements.
So first we start with this and then second this.
So that's a simpler way to access every elements in the dictionary.
Great.
OK. Now we have an interesting part, recursion.
What's the principle behind recursion?
Anyone?
What's the idea of recursion?
Yes?
The cause itself is [INAUDIBLE] base case.
And it saves a lot of money
Yes.
The idea of recursion is, if you have a problem, try to express the problem in a simpler version of the same problem.
So if you want to find factorial n, try to express it in terms of factorial n minus 1.
So you could keep on doing this till you come to factorial 1 for which you know the explicit answer.
Right?
So you try to express the problem in its simpler form.
It would be useful in many cases.
Actually, in your next piece you do have the problem.
But be mindful.
What's -- what makes it possible for you to use the recursion?
Only if you can express in terms of the simpler version.
Otherwise you can't.
This is probably quite like the induction you might have studied, mathematical induction.
I don't know whether you studied it in high school, but it's quite like that.
OK.
So there are two parts in recursion.
The first one is the base case.
So for a factorial problem we can express the factorial, say nth factorial, as any n in to n minus 1 factorial.
OK?
So this would be our recursive case.
So what is a recursive case?
Suppose we want to define a function factorial A. OK?
So what will be our recursive case?
A is greater than 0?
I'm asking the recursive function.
What would be that?
[INAUDIBLE]
n into 2.
Factorial n minus 1 right?
And you just return this.
What would be your base case?
If n is equal to 0, we know factorial 0 is 1, right?
What is factorial of 1?
1.
OK?
But why in this program we didn't have that particular line.
Why we didn't have n equals 1?
Why we didn't have that?
Because it always goes to 0?
It always goes 0, right?
Because you can express 1 in terms of 0 as well, right?
So you don't need to actually write this explicitly.
Why?
Because your recursive function only depends on its previous value.
So you need only one value in advance Which means you need only one value for your base case.
So you can understand this program, right?
I'm not going to go through that.
But anyway when you write a program always check its base case.
So you have to start with factorial 0 for this case.
OK?
And check one by one.
Then you would know whether the program is running correctly or not.
So always start with a simpler case.
But remember in Fibonacci series.
OK. for Fibonacci series can you give me the recursive function.?
Yes.
What is Fibonacci series?
[INAUDIBLE]
OK. Yeah.
That's great.
So return say F of n minus 1 plus F of n minus 2.
What is the Fibonacci series?
You start from 0, you add these elements.
So 1.
You add last two elements, 2.
You add last two elements, 3.
You add last two elements, 5.
So you add last two elements.
So if you want to find F of n, you return F of n minus 1 and plus F of n minus 2.
But here since you have two elements, or you need to access two previous elements, you need to define you base case accordingly.
So for your base case, F of 0, is what?
Actually, that depends on whether you start here or here.
OK?
You could do this.
I'll go through an interesting recursive example.
OK.
It's called a recursive exponentiation.
So I actually you can do an exponentiation through recursive multiplication.
Suppose you want to find 3 to the power, say, of 10.
Then how would you do that?
You start by expressing it in terms of its simpler version, right?
So it will be 3 into 3 to the power 9.
Sorry, 3 to the power 9.
OK?
So if you want to find that nth power of number M, you would say M into M to the power n minus 1.
So now you have your recursive case.
What is the base case?
[INAUDIBLE]
Yes, if n is equal to 0.
In Python you would test this by two equal signs.
If n is equal to 0, then what?
Return 1
Return 1.
OK?
Why we have only one base case here?
[INAUDIBLE]
Sorry?
[INAUDIBLE]
Yes.
And because you're accessing only one previous series, right?
Fibonacci series.
Similarly, if you want to multiply something-- OK, here what we did is we replaced this operator by recursively using this operator.
OK?
Can you do the multiplication by recursive addition?
So can you replace this operator by this operator?
Yes.
So how would you do this, for example say 3 into 5?
So how would you do it here?
Tell me.
What would this?
3 plus 3 times 4
3 plus?
3 times 4
3 times 4.
Here?
Come on, it's simple, right?
[INAUDIBLE]
I'm insulting you.
OK. What's the base case?
That is interesting.
What is the base case if n is equal to 0?
[INAUDIBLE]
OK.
If n is equal to 0, that is 0.
But that's fine.
But there's another problem.
The problem is you don't know whether these values are positive or negative.
So you can't just keep on adding like this.
This will work if it's positive.
But if it's negative, actually you have to check.
Which is sort of tricky if you want to use the same recursive function.
Otherwise you could have an if condition here.
If say n is greater than 0, do this.
Else-if you multiply n by minus 1 you negate that so it becomes positive.
And do it the same way and then that's the answer.
So you could do that.
But if you want to do it in the same recursive case, then you had to follow this.
If you have any questions, come to the office hours on that.
The final question is, final example is, the Hanoi example.
OK.
This is very interesting.
So here we have three towers; source, target, and buffer.
Suppose you have three disks.
You want to move them from source to target.
And you can always have a smaller disk on top of a bigger disk.
Then how would you move?
OK?
So can you express this problem in a simpler version?
So now we have three disks.
You want to move them to target.
So can you express it in a simpler version?
Come on.
Just in plain English, how would you do it?
Without looking at the program.
Because Python is very robust.
So if you read the program you get it in English.
So any ideas?
OK. We have to here move the last disk to target, right?
But that's the hardest part.
So before doing that you have to move this somewhere else.
So let's move it to buffer.
Done.
How would you move this?
Now you can move it to target.
OK. Now what would you do?
[INAUDIBLE]
OK.
Sorry.
Yeah, OK?
No.
What would you do?
[INAUDIBLE]
You can only move this to the buffer, right?
[INAUDIBLE] [?
Not the target.
OK.
So we made some mistake earlier.
Remember?
The problem is because we didn't think recursively.
OK. Let's think recursively.
Source, target, buffer.
Sorry.
Suppose we have only one disk.
Then you can simply move to target.
OK?
So a condition is if there's only one disk, just move it to the target.
So if n is equal to 1 we always move from move source to target.
OK?
Suppose you have two.
Then what would you do?
You move the top one to buffer.
OK?
So move source to buffer.
Then you move the next disk from source to target.
So now it's source to target.
And finally you move it from the buffer.
All right?
Now we have a very nice recursive case.
If you have two, we move from source to buffer, then source to target, then from buffer to target.
So if you look at this example I'm checking if n is equal to 1.
If it is 1, just I'm moving from source to target.
If it is greater than n, what would I do?
And that first assert statement is just there to make sure that n is greater than 0.
Otherwise it's meaningless, right?
You need to have this.
You can't have negative this.
So what would be the next step if you have more than one disk?
Suppose you have two disks.
Then what would you do?
You move the top disk or whatever on the top to the buffer.
For this simple example it was just one disk.
But suppose you had more disks on top of the last disk?
So you would have moved all of them to buffer.
You don't have to worry how you move it.
But you have to move it anyway.
After moving that, you can leisurely move this big disk to the top, to the target.
OK?
Finally, you bring back all of them here, right?
For you first operation, for your first operation, from source to target, you have to move all of them, the [UNINTELLIGIBLE].
You could have used target as your buffer.
OK?
So that's why in this line I'm using source as source.
But the second argument, which should have been the target, is now buffer because my current target is buffer for the top n minus 1 disk.
But my buffer is now the target.
OK. Because I can use the target as buffer for the movement.
Then I moved from source to the target, the last big disk.
OK?
For that I can use buffer as my buffer.
Then finally I bring back from buffer to target using source as my buffer.
OK?
So the thing is if you want to have a recursive problem you can do it in two ways.
One thing, you can start thinking how to express this problem in its simpler version like how you thought about the Fibonacci series, for instance.
Or else, you can start from its base case, the most fundamental situation.
So here it's n equal to 1, but this doesn't give you enough context.
So you go to the next level, n is equal to 2, and you have the answer.
Right?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to pick up where we left off last Friday on recursion.
Well first of all, can anyone tell me what recursion is or what a recursive function is?
No one knows
To divide and conquer
OK.
It's a divide-and-conquer technique.
How does it do it?
It's a recursive function
You have a base case.
And usually a return function, you return the value of something.
But because you keep returning the function back
Right.
So it's a function that calls itself.
And it works by one, identifying a base case, which is the smallest sub-problem possible.
And then in the other case, the recursive case, it tries to chunk the problem down into a smaller sub-problem that it then solves by calling itself.
So we went over a few examples.
And one of the things that we wanted to talk about is that for many problems, a recursive function can be written iteratively, or actually for most problems.
So there usually is some sort of a subjective choice in how to write a function.
And it really comes down to ease of understanding.
So what we've done is we've taken a couple of the algorithms that we showed last week.
And we've written them iteratively.
We also have the recursive version.
We'll compare and contrast and see where we would want to use a recursive function.
So on the screen on the left here, we have the multiplication version, the recursive multiplication, that we showed you last week.
It's a little different because it turns out there's a couple of simplifications you can make.
Can someone walk me through how this works?
So what's my base case first?
0
0, right?
And so obviously when n is 0, if we multiply by 0, then our result is 0.
Now there are two recursive cases.
And I'm not really sure how to explain this intuitively.
But let's say that my n is positive.
So I'm multiplying by a positive n. Well, then all I'm going to do is take m and just add it to the recursive version of itself n minus 1 times.
That's how to read that.
And then analogously, if n is less than or equal to 1, then I'm going to take negative m and add the recursive result of n plus 1 times m. It is not too intuitive right?
So if we implement it iteratively though, I think it's a little easier to understand.
Now this is also a subjective judgment, so you might disagree.
You're free to.
Here's our base case again.
n is equal to 0 or m is equal to 0, return 0.
In this case though, if we don't have the base case, then we're going to initialize the result variable.
And then for n is greater than or equal to 1, we're just going to enter a while loop and keep adding m to result n times.
It's a little bit easier to understand I think than the recursive version.
And then same thing for-- Oh I have a bug here.
Same thing for n is less than equal to negative 1.
So I'm going to run the two versions of this function.
So here's the recursive version and here's the iterative function.
They both return the same exact thing.
They both work in generally the same manner.
It's just that in one case we're using recursion to solve it, which I don't find too intuitive.
And the other case, we're using WHILE loops.
All right.
So in this case in my opinion, writing this iteratively, it makes a little bit more intuitive sense.
But in other cases, let's say good old Fibonacci, we can write the recursive version and then the iterative version.
So here's recursive Fibonacci.
We have our base case or cases.
And then we have our recursive case.
You can almost rewrite the mathematical formula from this directly.
All right now.
Now what was it?
So here's the iterative version of Fibonacci.
We still have our base case.
But when we get to what was the recursive case, we have to do a lot of bookkeeping.
We have to save off the previous Fibonacci what we're currently computing.
And then we have to iterate, get the next Fibonacci and then save off the prior versions of it.
This is all stuff that in the recursive version gets done for us by virtue of just calling another function.
So this is an example of a case where your recursive version is actually a little bit easier to understand.
Doesn't mean that it's more efficient.
And later on in the class we'll actually use this to talk about complexity.
But the left version I think is easier to understand than the right version.
Are there any disagreements?
If you disagree, I'm not going to bite you.
So anyway, we can run this.
And we can see that the output's identical
What's x-range?
x-range?
Probably something I shouldn't have put in there.
x-range is like range, except that it returns what's known as a generator object that you can iterate over.
So that I don't have to explain that right now-- Well actually, we'll probably talk about it later in the semester.
The difference is efficiency.
Range will return an entire list to you.
Whereas x-range is a little bit more conservative in how it manages it's memory for these purposes.
But changing it won't make a difference in the program.
And for a program as simple as this, range is perfectly fine.
I just used x-range out of habit.
So we'll do one last example.
And then we'll move on to a different topic.
If I didn't mention it before, in problem set 4, recursion is highly recommended for the final portion of it.
So it's kind of important you understand what's going on.
Anyway, so remember we looked at bisection early on in the semester.
And we showed you an iterative version of bisection.
This shouldn't really be unfamiliar to anyone at this point.
So all this is doing is finding the square root of a number using bisection search.
And we set our low and our high, get our midpoint.
And we just keep looping until we get a value that when we square it is close enough to x.
And on each iteration we set our lows and our highs, depending on how good our guess was.
Now the recursive version looks like this.
It has a few more lines of code.
And before I launch into it, did we explain default parameters to you, for functions?
So Python has this feature where if you have a function such as rec bisection search, you can specify that certain parameters are optional or you can give default values to them.
So let's just show a simple example so I can get past this.
So if I define this function, this one's really easy.
All it's going to do is print out x. I can call it like this, in which case it's going to pass 150 in and x will be 150 when the function's executing.
See I'm not lying.
Or I can call it like this.
And it'll be 100.
So that's, in a nutshell, what default parameters do for you.
They're useful in some instances, as in this example.
So in this recursive version of bisection square root, we have a low and a high parameter that we specify.
It's exactly equivalent to the low and the high parameter in this iterative version.
This is a common idiom for recursive functions in Python.
If we're calling it for the first time, we're not going to specify in a low and a high.
So low and high will be none coming into this function.
And then we just set them as we did in this iterative version.
And then we set the midpoint.
And then we have slightly different structure here.
If the midpoint that we guess is close enough to the square root of x, then we just return the midpoint.
On the other hand, if it's too low of a guess, then we're going to recursively call ourselves with the same x, same epsilon, but we're going to use midpoint for the low parameter.
So midpoint, in this case, is here and the same high parameter.
And then if we've guessed too high, then our low parameter is low.
And then our high parameter's the midpoint.
So it's doing the exact same thing as the iterative version.
We have recursive, iterative, recursive, iterative.
Same thing, just different forms.
All right.
Before I leave recursion, does anyone have any questions, or want to ask anything, or complain?
No?
All right
Do you use a lot of recursion in your work?
Do you normally use iterative or recursion?
Or is it just case by case?
It's case by case.
It depends on the problem.
And what we are trying to show here is that there are some problems that are better expressed recursively and others that are better expressed iteratively.
And by better, it's a very subjective term.
In my mind, it means more intuitive, easier to understand.
It allows you to focus on solving the problem rather than fiddling with code.
On the other hand, sometimes efficiency comes into play.
And we're going to be talking about that pretty shortly.
And in that case, you might want to do a recursive version because it's easier to understand.
But it takes too long to run, so you write an iterative version.
Computer programming, in a lot of cases, actually in all cases, is a bunch of trade offs.
Often times you'll trade off speed for memory, elegance for efficiency, that sort of thing.
And part of the skill of becoming good computer programmers is figuring out where those balance points are.
And it's something that I think comes only comes with experience.
All right, so we've talked about floating point to death.
But we just want to really emphasize it.
Because it's something that even for experienced programmers, still trips us up.
So the thing that we want you to understand is that floating point is inexact.
So you shouldn't compare for exact equality.
So looking at the code here, I have to find a variable 10/100, which is just 10 over 100, and 1/100, which is just 1 over 100, and then 9/100, which is 9 over 100.
And so in real math, this condition would be true.
I add 1/100 and 9/100.
I should get 10/100.
So if we were not dealing in computer land, this would print out.
But because we are dealing in computer-land, we get that.
And the reason is because of Python's representation.
Now when you write print x, if x is a float variable, Python does a little bit of nice formatting for you.
It kind of saves you from its internal representation.
So here is 10/100, as you just printed out.
It's what you would expect it to be.
But this is what Python sees when it does its math.
And it's not just Python.
This applies for anything on a binary computer.
It's an inherent limitation.
And you know we can get arbitrarily close, but never exact.
And then again, we have 1/100 and 9/100, Python will show us 0.1.
So when you print these out, they'll look fine.
If you were writing debugging code and you were wondering why if you compared x to y, it wasn't exactly equal, you would naturally print out x and then print out y.
But it would look equal to you.
But the code wouldn't be working properly.
Well the reason is, is that the internal application that Python is using to compare them is that.
So what's the solution?
It's a natural question.
I don't know the answer
If they're close enough, then it would be inside variance--
Right.
We're going to say good enough.
And the traditional way of representing that is epsilon.
Epsilon, you've seen in your problem sets.
And you've seen it in code before.
And if you've come to office hours, someone's probably explained it to you.
Epsilon is the amount of error we're willing to tolerate in our calculations.
So in Python-land, you can have arbitrary precision.
Don't quote me on that though.
But for purposes of this class, if you're using an epsilon it's like 0.0001, we're not going to get too upset.
All this function does, and this is a handy function to keep around, is it just tests to see if the distance between x and y are less than epsilon.
If it is, then we say they're close enough to each other to be considered equal.
So I don't like the function named compare.
I don't think it's intuitive.
Close enough is probably better.
But it's also going to break my code.
Uh oh.
This is an actual bug.
Line 203.
What did I do to myself?
I commented out my definition is what I did.
All right.
So if we compare the two values, 10/100 and 1/100 plus 9/100 and we use our close enough, our compare function, then yeah it's within epsilon.
Again, notice here that we're using a default parameter.
So if we don't pass in something explicitly.
So I can say something like this.
Let's make epsilon really tiny.
So if I make epsilon really, really tiny, then it's going to say no.
So how you determine epsilon really depends on your specific application.
If you're doing high precision mathematics, you're modeling faults on a bridge or something, probably I want to be pretty precise.
Because if you have the wrong epsilon, then you might have cars falling of the bridge or the bridge collapsing.
And it would just be a bad day.
So are there any questions so far about floating point?
No
You normally go as close as the areas will let you
What's that?
You can't get as close anymore.
How can you make the error that small.
Because it's not going to get that close
Well yeah, that's what I mean.
So there is a limit to how close you can get.
And it depends on the language and it also depends on the hardware.
There are, and this group is getting a little bit more technical than I want, but you can define pretty precisely the smallest value that epsilon can be.
In a language like C, its defined as the minimum difference between two floating point variables that's representable on the host machine's hardware.
So yeah, there is a limit.
There are some math packages though, and we'll be using something called NumPy later on the semester, that allow you to do pretty high precision mathematics.
Keep that in the back of your mind.
But yeah you're right.
You do eventually hit a limit.
OK so the last thing that I want to cover on floating point is that even though it's inexact, it's consistent.
So let's say I define a variable 9/100 plus 1/100.
And it's exactly what it says, 9/100 plus 1/100.
Now we know that this is not going to equal 10/100, right.
We just demonstrated that ad nauseum.
And also, yeah, still defined.
The question though is, if I subtract 1/100 from this variable that I've defined, this 9/100 plus 1/100, will 9/100 now be equal to 9/100 plus 1/100?
So in other words, will this be true?
And the answer is yes.
And the reason is that if I'm adding or subtracting, even though 1/100 we know is an inexact representation, it's still the same.
And so when we do the subtraction, we're subtracting the same inexact value.
So this appeared as a quiz question at one point.
It probably won't this semester.
But it's something to keep in mind.
So any questions on floating point?
If you're a mathy type and want to, look up IEEE 754.
And this will give you all the gory details about representation and mathematical operations on floating point.
And if you don't, then don't worry about it.
It's not required for the class.
OK.
So the next topic we want to cover is pseudocode.
So can someone take a stab at defining pseudocode for me?
From what I gathered, it's basically you're writing out what you're planning on doing in just normal English
I wouldn't just say normal English.
But it's an English of sorts.
And a lot of the difficulty that programmers have with writing programs or new programs is that we don't naturally think in computer languages.
You think in English.
Or well, you think in a human language.
So what pseudocode allows us to do is to kind of be in the intermediate.
We still want to develop a step by step process for solving a problem, but we want to be able to describe it in words and not variables and syntax.
Sometimes what'll happen is programmers will get so wrapped around kind of getting the syntax right that they'll forget the problem that they're actually trying to solve.
So let's walk through an example.
Let's talk about pseudocode for Hangman.
And because you've all done this on the problem set, I don't have to explain the rules right.
So what would be a good kind of English first step for Hangman?
You have to choose a word
Right.
So let's not be too specific.
We'll just say, select random word.
OK. Now what would be another good step, next step
Display the amount of spaces maybe?
So display a masked version of the word
Exactly.
Hide the word but display it
Hide the word but display it
Well, display the amount of spaces.
You probably want to state how many letters are in the word at some point
Ah, that's a good point.
Where should that go?
That should probably be before the display
OK.
So tell how many letters.
All right.
Now what would come after this?
After display?
Yeah
See how many letters you have to choose from

First time you don't
At first time you don't.
But the nice thing about pseudocode is that we can barf things onto paper and then rearrange them as we-- It's sort of like brainstorming in a sense.
You're trying to derive the structure.
And it's easier to do like this than to try and do it in code.
But yeah, you're right.
You don't have to
You would ask the person to put a letter
OK. For a letter.
And then what would come after that?
So then you want to check if it's the --
Check if it's in the word.
All right.
And?
If it is--
If it is?
Add it to the word
Add, let's say, to correct letters guess
Yeah
OK. And if it isn't?
If it isn't, reject it
Let's say--
You want to remove it from the options
So if it's not, then we're going to remove from options.
So letters remaining.
Probably want to tell the user they're wrong too
And use up a turn
What's that?
You want to use up a turn
I'm sorry
Then you use up a turn.
If you had a set amount of turns
OK, so we're actually going get to that in a second
I sent last week.
[INAUDIBLE] game
I actually played all of your Hangman games.
It was quite fun.
So again, yeah you're right.
So we have a number of guesses that are remaining.
And the thing is that we know that the user has a certain number of terms.
So we're probably going to repeat a lot of this.
So at some point, we probably want to have a WHILE.
It'll just say, WHILE we have guesses left remaining.
By the way, the reason why I program computers is because my handwriting is horrible.
So WHILE we have guesses remaining, we're going to keep doing all this.
All right?
And then we're going to remove-- But is this the only stopping criteria?
What if they win?
So WHILE they have guesses remaining and the word is not guessed.
So this is in essence your Hangman program.
It's English language.
It's not easy to read because that's my handwriting.
But it's kind of easy to understand it at an intuitive level.
And the reason we're talking about this is because we're going to get to some more complicated programs as we move through the semester.
And a good starting off point for a lot of you, when you're trying to do your problem sets, is instead of trying to jump right into the coding portion of it, to sit down with a piece of paper, index cards, or a whiteboard and kind of sketch out a high level view of the algorithm.
So that we can see this in code form.
So let's say that I want to write a function that tests a number to see if it's prime.
First question is, what is a prime number?
One where the only factors are 1 and itself
Right.
So a number that is only divisible by itself and 1.
Are even numbers prime?
Can they ever be prime?
Really?
What about 2?
Right.
So 2 is one of our special cases.
All right.
So what would be maybe a good starting off point for pseudocode to test primality, knowing those facts
[INAUDIBLE]
All right.
Can I erase this or should I leave it up?
Because I can go over there.
It's not like I'm erasing any deep dark secrets.
There's no magic here.
All right so test number, if, what, equal to, say what?
2?
Yeah.
Why not?
And maybe 3.
Now what do I do if it is?
Are they prime?
So I'm done right.
So I'm going to return true.
Now what do I do if the number given is not 2 or 3?
[INAUDIBLE]
You're talking about the module operator.
Right.
So we will use that.
That will tell us whether or not an integer divides evenly into another integer or the remainder after an integer is divided into another integer.
Let me ask you another question.
What is the maximum value of an integer divisor for a non-prime number?
So for a composite number
So itself
What's that?
Number itself
OK, excluding the number itself.
Well let's say that I have n as the number I'm testing, square root of n, because I'm not going to have a factor that's larger than that.
And I ask that because there's a loop involved.
So how would I go about this systematically?
I'd probably start at, let's say 5.
OK. And then test if n modula, let's say 5, is equal to 0.
Now if n is evenly divisible by 5, then that must mean that n is composite, because 5 is a factor.
So if it is then return false.
Now what if it isn't?
So that means that n is not evenly divisible by 5.
Does that mean that the number's automatically prime?
So after 5, what would be a good number to test, to move to?
All right.
6?
No.
That wouldn't be it.
Because if 6 is a factor, then obviously it's not.
Whatever.
So we're going to move on to 7.
So basically we're going to test all the odd numbers.
And this is going to be the same as that.
So this repetition indicates here that I probably need a loop.
So instead of doing this, I want to say x is equal to 5 while x is less than-- We're going to test if x evenly divides n. And if it does, return false.
And if it doesn't, then we just increment x and repeat.
And what happens when x becomes greater than square root of n?
Well the WHILE loop's going to stop.
And that also means that if I've made it to that point, then I've not found any numbers between 5 and square root of n that will evenly divide n. So that means that n is prime.
So if I translate this into code, it would look something like this.
Now I see.
So first we're going to check if n is less than or equal to 3.
If it's 2 or 3, then we'll return true.
If it's not 2 or 3, then that means it's 1 or 0.
So return false.
So we've got those cases.
And then we're going to iterate-- or if n is greater than 3, we're going to iterate-- now why would you go from 2-- we're going to integrate through all the possible divisors and check for divisibility.
And if we evenly divide it, return false.
And if we make it through the loop, we'd return true
Does that RETURN stop the loop?
Yes.
Well think about what RETURN is doing.
You're in this function, test primality, And as soon as Python sees return, that's telling Python to kick out of the function and return whatever is after the return statement.
So this false here, it says return false, that means that it doesn't matter where you are, it's just going to kick out of the innermost function or the function that encloses that return and return that value.
Any questions?
All right.
So testing primality, 1 is false, 2 is true, 3 is true, 4 is false, and 5 is true.
So it looks like the program works.
And if no one has any questions, I'm going to move on to the last major topic.
Everyone's good on pseudocode?
All right
What would the main purpose of pseudocode is for yourself when you're writing a program or when you want to explain it to other people?
Both.
So the question was, is writing pseudocode useful for just understanding a program yourself or for explaining it to other people?
The answer is both.
I don't know.
It's the difference between showing someone the derivative of the function and then explaining that what you're doing is finding a function that gives you the slope of a function at that point.
So it's one is more intuitive for some people than the other.
A mathematician would understand the former pretty quickly.
An English major would understand the latter maybe.
So when I explain my research to people, I don't tell them that I mess around with Gaussian Mixture models and Hidden Markov models.
I tell them that I'm trying to figure out how people mispronounce words when they speak foreign languages.
A lot easier for people to digest.
With debugging, what are bugs?
Mistakes
Mistakes.
And if you see one bug, there are probably many more.
So when you're debugging, your goal is not to move quickly.
This is an instance where the maxim fast is slow and slow is fast comes into play.
You want to be very deliberate and systematic when you're trying to debug code.
You want to ask the question why your code is doing what it does.
And remember, the first recitation I said, that your computer's not going to do anything that you do not tell it to do.
It's not something that people do naturally.
If you watch some of the TAs and sometimes a student will say, how do you find the bug so quickly?
Well it's because I've been programming for 18 years.
Professor Guttag's been programming for longer than that.
So a lot of it is experience.
And it's just when we've debug our own programs and when we were learning to program, it was as painful for us as it was for you.
So that said, you want to start with asking, how could your code have produced the output that it did?
Then you want to figure out some experiments that are repeatable and that you have an idea of what the output should be.
So after you do that, then you want to test your code one by one on these different test cases and see what it does.
And in order to see what it does, you can use a print statement.
So when you think you found a bug and you think you have a solution to your code, you want to make as few changes as possible at a time, it's because as you're making corrections, you can still introduce bugs.
Let's see.
So a useful way to do this is to use a test harness.
So when we actually grade your problem sets, a lot of the time the TAs will put together a set of test cases for your code.
So one of the things is a lot of the times when you get one of the problems or when you look at the problems, it'll have some example input and output.
But that doesn't necessarily mean that we only test on that.
There's additional test cases that we use.
And it's not to trip you up.
It's because there's a lot of different variations.
And it's also, if you read the specification, you follow the specification, then you'll be fine.
Moving on.
So let's look at an example.
I have a function here, is palindrome.
You've seen this before, right?
Yes?
Yeah.
OK.
So it's supposed to return true if string s is a palindrome.
And so I've written this function.
And I've also written a test harness.
Now there's a lot more code in the test harness, but it's pretty simple code.
When you're writing functions, you want to think of the type of input you could receive.
And you want to think of, what are the kind of boundary cases?
So the extremes of input that you can get.
We call these boundary cases, edge cases.
For the is_palindrome function, it would be like the empty string would be one.
Or just a single character.
These are the kind of minimum we can have or we could think of.
On the opposite end of the spectrum, theoretically we could have an infinitely long string.
So we're not going to actually test for an infinitely long string.
Anyway, all we're going to do is in our test harness, we're just going to run the function on these inputs.
And we know that an empty string should be true.
We know that a single character string should be true.
We know that if I have a string that's two characters long and they're the same character, that should be true.
If they're two characters, then it should be false.
And what I'm going to do now is I'm looking at kind of expecting what we call expected input.
So after I've hit my edge cases, I'm going to look at all the strings of an even length and make sure that the function works properly.
And then I'm going to look at strings with an odd length.
And then once I get to this point where I've tested a number of different lengths, and in this case, it's just 2 through 5 or 0 through 5, if you want to include the edge cases.
Then I'm going to say, well it looks like all tests are pass.
And I think that this function works pretty good for anything we can expect it to encounter reasonably.
So the way that you use test harnesses, is every time you make a change to your program, you want to run the test harness, because that'll catch any bugs you may have introduced.
And so I'm going to finish up with this really quickly because I know my time's up.
So I got a bug.
It's telling me that one of my test cases failed.
So line 299, which is this test case.
So what we can do is now that we know that it fails, we can say, maybe printout our input and see what we have.
And instead of just running, I'm just going to run that one test case that failed.
So obviously this should be true.
And what we're seeing is that on the first call to is_palindrome, s is abba.
And then on the recursive call to it, we only get bba.
That means that we've only chopped of the front character.
So you see the bug?
Well here.
So there we go.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
There's so many things that we're going to start with today, one, is we're going to review the quiz.
And we'll be real quick on that.
Then, we're going to talk about object oriented programming, which is something that you'll probably be more interested in for your problem set.
So just going down problem (1): it's false, true, false, false, false.
Does anyone have any questions on this?
No.
Everyone good with that?
Anyone wonder why if something's true?
Or why something's false?
Why is the second one false?
Why is it false?
Why?
It's true.
Because, and this is an English thing, so, or language thing, so it can kind of trip people up.
But it's basically saying that there are some problems that you need to use recursion or iteration to solve
Why do you always have to use [INAUDIBLE]?
Well, if you have a large number of inputs or the inputs are variable, the point is that there are certain problems that you would need to use recursion or iteration to solve them.
There are some that you don't.
But this is just asking if there are problems that exist
Can you use brute force?
Well, brute force usually involves some sort of iteration.
Because you have to iterate through all the possible solutions.
Any other questions?
OK.
So the next one, this is just an exercise in code reading.
And we can actually just look at how it runs.
If I run like that.
Also, if you don't have your quiz, I have them up here, if you want to pick it up.
But that's just the output of that code.
Does anyone not see how that works?
Or want me to step through it?
Question (3) was the double recurring question.
And those are the two answers we're were looking for.
Does anyone not see how that works?
Or want to try and walk through it?
The way to tackle this problem is walk through in your head.
Let's take the first set of input.
At the top, that's your initial call.
Now, the string, s, here, is not less than 1, right.
You're going to go to this double recursive call here.
So that means you're going to get as you return.
And the way I got this at is, I'm just taking s from one to the end.
What this means is that this part is going to execute first.
And it's the same thing.
Go into the function.
This is now s. This is obviously not only one character long.
So we're going to have, again, another double recursive call.
And it's going to look like this.
Now, I've got this call to contend with.
This string is one character long.
So what does it do?
It just returns the input.
So, this function call is just going to return t. And, then, this function is going to get called with t, again.
And we already know what happens when you can pass a t. That means that this whole function here, results in t. Then, we tack on the a.
That means this function call, 'at', returns 'ta'.
And then we pass it into this outer function here.
And we can already guess what this is going to return because for this input it just reversed the characters.
It just flipped them.
For 'ta', it's going to flip them again.
The entire function call here, results in 'at'.
And then we just append 'm' to it.
The entire return for this one is 'atm'.
It's tricky, but you've just got to step through the code and step through the functions and look at each step, what the function is getting as input.
This was the coding question.
Most of the questions up until now has been, can you read code and understand what it's doing.
This one asked you to actually implement a function.
When you start with these questions you should always start from the specification.
So, this function is assuming that we have a list of words in lowercase.
lStr is a string of lowercase letters.
All the letters in lStr are unique.
And that the return of the function are going to be all the words in word list that have a one to one mapping between the letters in the word and lStr.
In English, let's say that my lStr is raft.
Ok.
Assuming that I have a fairly complete word list, I'm going to have two words that are going to meet that criteria.
I can say there's an r, a, f, t. That's pretty obvious.
And then we have a t, f, a, r. That's what we're looking at.
A lot of you did something where you iterated through all the words and then you had another loop inside that tried to find this correspondence.
The way that we solved it-- and there are multiple solutions to this.
So if your solution worked, you got full credit.
But the solution that we came up with is first, we're going to take the letters in lStr and we're going to sort them.
So we're going to have a, f, r, t. Alphabetically.
And then for each word in word list, we're going to do the same thing.
Raft becomes a, f, r, t. Fart becomes a, f, r, t, as well.
Then it becomes just a simple string comparison.
And all you have to do is iterate through the word list.
Does that make sense to everyone?
Our solution has a trick.
You don't need to use this trick.
A lot of people didn't and they got full credit.
But this is one way of doing it.
Question (5).
This was the one where we asked you to find the problem with this code.
Or, rather we asked does this code meet the specification.
When you get a question like that, first thing you should do is actually look at the specification.
Then, you should look for what this code needs to do.
Because the specification is going to tell you what the function needs to do.
If it doesn't say you need to do something then, that means it's undefined.
Right?
So In those cases, you can do whatever, as long as you meet the specification.
The first requirement is it returns a list of the pointwise sum of the elements.
That's the first requirement of the specification.
And then it gives you 2 implicit requirements.
One is this example, where it says if I'm given two vectors, this is what I expect the return to be.
In this case, the vectors are two different lengths.
It's also saying that your vectors are not always going to be the same length.
In that case, you take the pointwise sum up to the shorter of the two lists and just tack on the remainder from a longer list.
So that's the second requirement.
The third requirement is, if you have two empty lists you're going to return an empty list.
And, finally, your fourth requirement is, does not modify input.
Now, you know the four requirements from the specification.
And.
now, you need to look in the code and see if this code matches all those requirements.
The first one.
Does it return a list containing the pointwise some of the elements.
Well, this is the portion of the code that does that.
It looks like it meets that specification.
Result is going to be the longer of the two vectors.
In this case if v1 longer than v2.
We set result to v1 and other to v2.
That's the shorter.
Then, if we choose the longer, we set result to v2.
And other to the shorter of the two vectors.
Does everyone see that?
We iterate through and we get the pointwise sum.
We meet the first requirement.
The second requirement is that if we're given two vectors that are different lengths then, we're going to sum up the furthest that we can, up to the length of the shortest one an tack on the remainder.
Well, again, this for loop here, that does a pointwise sum, it only goes up to the length of the shorter list.
Second requirement met.
Third requirement.
Two empty vectors returns an empty vector.
Well, if I have an empty vector here, this is going to be 0.
This FOR loop is never going to execute.
And my result is going to be empty.
And, then, finally that leaves a fourth requirement, does not modify the input.
What's result?
Result is what we're ultimately returning.
And that's the only thing that we really modify in this function.
Result in this case, is v1 or v2.
But they're aliased.
So it's modifying the inputs and that's a violation.
So the answer to this problem is a total of six characters.
Some of you wrote entire redefinitions of the function or, copied Ryan's code from the reviews.
Perfectly acceptable, but way too much work.
Remember, programmers are lazy.
That's all we were looking for.
If you used the code from Ryan or, you had a different implementation that met the specifications, but was completely different you got full credit.
You have a question?
[INAUDIBLE] .
I'm just trying to figure out what the distinction is [INAUDIBLE] do a dot copy.
Why one is better than the other
Dot copy applies only to dictionaries
Oh
So if we try and do a dot copy on a list
Oh,
Got it.
I think we took one point for that or something.
We knew what your intent was but you didn't have your IDE there with you.
Question (6) was another exercise in code reading.
The way that I would attack this one is to figure out what the two functions do first.
Let's take the easier of the two, addUp.
Takes a dictionaries input and it has a variable result that it initially sets to 0.
And then it iterates through all the keys in the dictionary and adds them to result.
Basically, its assuming that the values in the dictionary are some sort of number, and it's summing them up, and returning the total.
And, then, this f function here -- takes the dictionary it zeroes out any of the keys it might have.
And then it iterates through all the characters in s. If the character is already in the dictionary then, it's going to add 1 to it.
So it's going to increment it.
And if the character isn't in the dictionary, then, it's going to set up to 0.
Then, it's going to return the result in dictionary.
Knowing that, the function becomes pretty easy.
f of abbc for d1, which is an empty dictionary.
Just walking through it, starting from this point, if we iterate a, it's not going to be in the dictionary.
So we're going to set d of a to 0.
Then, we move on to the next character, b. b is not in the dictionary.
So d of b becomes 0.
Now, we get one of the second b. b Is now in the dictionary.
So we increment b.
Now, d of b is 1.
Then, we move on to the final character, c, again not the dictionary.
So we set d of c to 0.
Then, we return the dictionary.
That means that in my dictionary, I have three keys a, b, and c. And they have values 0, 1, and 0 Respectively.
So add up is going to return 1.
And same process for all of these.
Question I have here is what happens when Python gets to that line?
Anyone.
It's going to be an error.
Why?
Result is a local variable
Right.
Result is a local variable to addUP.
There you go.
So, again, the approach to this problem is to figure out which each of the functions do, and then right walk through the code.
Did anyone have trouble with this or want me to actually step through it?
First, when f gets an integer, it just prints out the integer in binary.
And then, this loop here, prints out the binary representations from 0, 1, 2.
Why is the first output none in this case?
Because in that first iteration i is 0.
When f is called, n, is going to be 0.
It's just going to return nothing.
So it gives you nothing.
Now, the next question was under the assumption that the log base 2 is o of n, what is the order of the function, f?
And to figure this out, you know that this function here is o of n, because we told you.
We know that that's one of the first things that's called in f. So automatically, a run time is o of n. Now, this loop here, iterates how many times?
Log n. Well, log base 2n, to be explicit.
For this function, which is the dominating term here?
When we want to see o of n, it's just going to be that.
If you had o of n plus log n, I think we took a point.
Just because when we talk about worst case scenario, we're looking for what the dominating portion of this function is.
How does it do it?
You want to walk through the code now?
Alright.
First thing it does, is it gets something it's calling curve digit.
All that is, is you take the log base 2 of a number you're going to get the number of binary digits in it.
Think of it as like if I have three which binary is 1, 1.
If I take log base2 of this, then my curve digit is going to be 1.
I'm not sure, Python's been around too long.
Well, 1.5 but if we truncate it to an int it's going to be 1.
That's kind of like our position marker in the binary number.
Now, we're going to iterate while the current digit is greater than or equal to 0.
We're basically going to start here and move down the line in this direction.
All it does is says if my n mod 2 -- so I'm checking to see if it's odd or even -- if it's going to be 1 or 0.
In this case, it's going to be 1 to the power of the current digit, which in this case is 1.
I'm sorry, misspoke.
n is 3, The remainder is going to be 1.
In this case 1 to the power of 1 is going to be 1.
That's less than n, which it is.
Then, my ans is going to be ans plus 1.
I'm going to add 1 to the string and construct it.
Then, I'm going to subtract whatever this part is.
So 2 to the current digit, in this case to the 1.
It's just going to subtract this off.
Then, curve digit is going to be decremented and moved here.
Then, in this iteration, curve digit is going to be 0 and n is going to be 1.
So 1 mod 2 is going to be 1.
Curve digit is 0.
So that's going to be 1 less than 1.
It should print it out.
I was not prepared for that.
All right.
So.
the final question -- number (8).
Big O notation, if we match it up, does anyone know what it is?
Upper bound
Yeah.
A lot of you put the expected running time.
And the letters are messed up on this but-- a lot of you put the expected running time, but when we are talking about Big O, we're talking about worst case scenario.
So, that's the upper bound.
There is an expected bound.
If you decide to do any more algorithm analysis, you have Big O, expected, and little o.
If I plot the run time of a given function, my Big O might be this.
The worst time, my expected, might be like that.
And my absolute best case might be like that.
When we say they go that's what we're looking for.
Alright Newton's method.
What is that an example of?
[INAUDIBLE]
Yeah.
These don't look right.
You know what, I'm sorry.
This is a different version of the quiz.
Newton's method, that's an approximation.
Then the last one was recursion on your test.
The answer we were looking for was induction.
That's that.
Anyone have any actual questions
Go back to number (4)
Number (4)
Yeah.
[INAUDIBLE]
What was the part that you did not understand with number (4)?
I thought it was confusing how to join the [INAUDIBLE] together or how to go through them to see if you know exactly [INAUDIBLE]
Well, that's kind of the trick we have here.
We know we have a list of words so to iterate through the words is just a FOR loop.
So that's what that for word in wordList does.
To do the thing where you match the letters one to one, what we implemented here is, we first take lStr and we sort it, sort the characters in lStr.
Then, what we do for wordList is for each word, we sort the characters in that word.
And what that does is, it allows us to just directly compare the two strings.
And if they're equal, then we've met the criteria for adding it into the wordList, Or the return wordList.
That's the quiz.
If you don't have it, you can come pick it up
Sorry, I was just going to ask really quick.
If you're getting a [INAUDIBLE] string, you're concatenating [INAUDIBLE] strings, the empty string, why can't you just set it equal to the [INAUDIBLE] string?
Is there something about the way that operates that you couldn't do it?
Yeah, so if I say my string is 'abcdef,' and I just say sorted(s), this returns a list.
What I'm doing with join is I'm just converting it back to a string.
Got it?
OK, so on to object oriented programming.
So what can someone tell me about classes?
What do they allow you to do?
Allow you to define a custom type
Yes.
So one thing they allow you to do is to define a custom type.
Now, when you define a class, you can group your methods and data together with something called encapsulation.
So first stuff, we've actually already been using classes.
We just didn't tell you, right?
So ints, floats, dicts, et cetera, these are all types of classes.
And each of these, we've already seen them, have something called methods associated with them, right?
Methods are basically functions that are associated with a given class.
So for example, if I have the str class, which everyone's seen, then it has, say, a method, dot lower.
So if I have s equal-- well, let me write it up here on the code.
If I say something like s equal 'abcdef,' I can call the method, lower.
So we've actually already been doing object oriented programming.
You just didn't know it.
Along with that, classes have methods.
And they also have something else.
Someone help me?
What?
Parameters?
Variables?
I'm looking for something-- terminology-wise, attributes.
So they're a way of grouping methods and attributes.
So when we talk about attributes, we're talking about things that pertain to a specific instance of a class.
So let's say that I have a real world example.
I have a person class.
We've already kind of seen this guy.
A person has multiple instances.
So there's an instance of Mitch.
There's an instance of Garthi.
There's an instance of Phillipe.
We're all people, mostly human.
And we all have attributes.
So we all have an age, when we were born.
We all have a name.
Some of us have hair and other attributes, right?
We also have actions that we can take.
So I can talk.
And I could walk.
I can talk to you, you, you, you, you, and you.
So a method that I could define for a person might be talk to someone.
So that's one way of thinking about objects.
So object-oriented programming also gives us something called inheritance, right?
So I could think of-- if I'm going along with my person analogy-- I could think of sub-classing person if I'm willing to draw hard binary on the genders.
You have males and you have females, right?
And in this column, I might have Tracy.
I'm just picking on people who come to office hours.
And then, Garthi, et cetera.
Now, the inheritance portion of it is important because I've already said that one of my methods on a person is I can talk to people.
I can talk to other people.
It shouldn't matter if I'm talking to Garthi, as a male, versus Tracy, as a female.
I talk to people basically the same.
So that gets into something called polymorphism, which is we treat objects with a common super class the same as their sub-classes.
So as another example, let's say that I have dogs and-- what's another canine, foxes?
I'm not going to necessarily talk to canines the way that I talk to a person.
So they would exist with a different super-class.
And then, we could all be animals.
So really, what object-oriented programming gives you is a different way of thinking about how you're modeling your world.
And what I want to do is now, instead of just talking about these abstract things, walk through some concrete examples.
So the first thing that I want to illustrate is let's say that we want to create a person.
But we don't want to use object oriented programming.
Or, we don't want to use classes.
Professor Guttag will get angry at me if I say we're not using object oriented programming.
So let's say I have a function, makePerson.
And I'm going to represent a person as a dictionary that has name, age, height, weight as keys.
All makePerson does is it takes these things as parameters, makes a dictionary with them as values, and then returns a dictionary.
Then, I have a bunch of helper functions, like get_name of person.
All that does is it just returns whatever's in the name key.
I can also set the name in case a person decides they don't want to be known as Mitch.
They want to be known as Mitchell or something like that.
So I have a bunch of these getter and setter function.
These are called accessor and mutator functions, whatever terminology you want to use.
I can also define another function that will do something like print out the person.
So in this case, I'm just going to print out name, age, height, and weight.
And to see this in action, I'm going to make a person, Mitch, 32 70, 200.
And then, Serena, 25, 65, 130.
I don't know if these are actually correct.
So don't quote me on it.
I have a syntax error.
So if I run this, then all it's going to do is just print out what I'd expect it to print out.
I can also set my age.
So I can go back in time to 25, which is a great age.
And now, I'm 25.
Now, this is fine if you just want to do simple things.
But the reason why we kind of like object oriented programming is because we run into difficulties.
So let's say that I print out the type of Mitch.
It says I'm a dict.
But it doesn't give me any more information.
So that means that I could define any random old dict and pass it to some of my functions that I've defined to work on people.
And it will probably give me an error.
It also makes other operations kind of non-intuitive.
So let's say that I want to figure out if two people are equal.
Well, I could do that by defining a function, people_equal, or equal_people, and passing in it a person1 and person2.
And for our intents and purposes, it's just going to be if they have the same name, they're the same person.
So this is going to, of course, do what we expect it to do and return false, right?
Because Serena and Mitch aren't the same person.
But it's kind of awkward.
And if we do things using classes, it becomes a little bit more elegant.
And you get a lot more power.
So let's say I do the exact same thing with a class.
So I have the class keyword.
I have the name of my object, or my class, my new type.
And then, I have this thing called init.
All init does is it says, when I get a new person object, Python automatically calls init with whatever parameters are specified and tells the object to make attributes or to set itself up.
So in this case-- need a bigger screen-- I'm making a Mitch person, all right?
And the way that you make a new object, or an instance of person, is you have the class name.
And then, you pass it whatever parameters are specified in init.
Now, behind the scenes, Python will create a chunk of memory.
And then, it'll call this init function.
And it'll pass a reference to that chunk of memory.
So visually, let's say this is your magical memory.
Python sees us call person, goes in, grabs a chunk of memory-- this marker sucks-- grabs a chunk of memory and says, this chunk of memory is of type person.
Then, it calls init.
init says, basically, Mitch is a reference to this chunk of memory.
This is a self parameter.
And then, it has the other parameters.
And then, in the init method, all we're doing is we're creating new attributes on this chunk of memory.
So we're going to have an attribute name.
We're going to have an attribute age, height, weight.
And then, we have a bunch of accessors or getters, mutators or setters.
Same exact thing as the functions that I showed, that we had when we were trying to do this without classes.
The difference is that these are lexically scoped to person.
They are methods for an instance of type person.
Whereas before, they were just functions that worked on something we called a person.
But a person was actually a dict.
So let me just run this.
So if I run this, I'm just going to create a person, a Mitch person and a Sarina person.
I'm going to print out Mitch.
It's the same thing that I had before, right?
Except I'm using a class.
Now, if I want to use one of the accessors, so I want to get younger again, I can just take my object, Mitch, and I can call the set_age method and pass it in my new age.
And I've lost seven years.
All right, now so far, this is just a different way of doing the same thing, right?
But if we look at the type of Mitch, I'm a person.
So there's now some extra information that we didn't have before when we were using a dict to represent a person.
Before, it could've been any dict.
We didn't know that it represented a person.
The only reason that it represented a person before we used a class was because we had set certain keys to certain values.
But other than that, it was just a dict.
Here, this is marked as being a person.
And that also gives us some additional kind of power, which we'll see in a little bit.
So now, I've done something sneaky.
And I've created a way of comparing Mitch and Sarina.
Before I had that function, people_equal.
Now, I can use the double equal operator, which is something that we're used to.
We use it with all the other types.
Why don't we use it with people?
The way that we define that is that on this person class, I have to find this function underbar, underbar, EQ, underbar, underbar, right?
The underbar methods-- and underbar, underbar, init is one of them-- have a special significance in Python.
So in this case, the underbar, underbar, EQ, underbar, underbar says that if I have an object niche, and I have a double equal, and I have another object, Serena, it's going to look in the Mitch object.
And it's going to say, does it have an EQ method?
Does it have this method name?
In this case, it does.
So it says, OK, this object is capable of comparing itself to another object for equality.
And so it calls this method.
And self, in this case, is the Mitch object, is this guy.
And the other is the Serena object.
Sorry?
So when it looks for the EQ, does it look for EQ, or does it look for something that has a double equal sign, and then go off from there?
Is the underbar, underbar, EQ, underbar, underbar -- is it defined for all?
It's defined for all persons
All persons
Yeah.
OK, I see what you're saying.
Is it defined for objects other than persons, like all objects?
Yeah
Well, we can check that out.
You notice that person inherits from object.
This is where all classes that you define should inherit from.
And so object, like any other type-- remember I showed the dir command before?
If you do this, you can see everything that's inherent to an object.
So in this case, it doesn't have the underbar, underbar, EQ.
So it might be something special that Python does.
So this is where I'd actually have to look it up to answer your question.
So I'm going to punt on that
So it knows somehow to look for EQ specifically?
Yeah.
When it sees the double equals sign, it knows to look for EQ.
The double underbars signify a magical operator or magical method
So when you're calling EQ, you're not formally giving it two formal parameters in the parentheses.
So it's presumably taking the first name before the equals and the second name before the equals--
We'll get into that.
So the question is, are we actually explicitly passing two formal parameters to it?
And the answer is yes and no.
This is a bit of syntactic sugar for Python, which means that it's a nicer way of writing things.
But we could do something like this.
Which is totally awkward, but in actuality, this is what Python does in the background.
It does this sort of mangling.
And I was actually going to show that with something a little bit easier to comprehend.
So I have a method called get_age, an accessor, right?
And usually, you call a method by having a reference to the object, dot, and then whatever method name-- in this case, get_age.
You can also write it like this, which is, if you think about it, exactly what happened with the double equal operator, right?
Python took your nice, intuitive syntax-- Mitch double equal Sarina-- and totally mangled it and made it something that was completely ugly.
And this is how Python would actually see it.
So that's why, when you write these methods, you have a self parameter.
This self parameter is like this guy.
Python says, ooh, I have an object reference.
And they're calling a method, get_age?
Do I have a get_age method?
Yes, I do.
Here it is.
OK, now I'm going to take this and make it look ugly.
It's a person object.
So I'm going to call person.get_age, and I'm going to pass it Mitch, the reference to the object.
And that becomes a self parameter.
So that's where the self parameter comes from.
There's a little bit of mangling that goes on.
And it's partially hidden from you.
In other languages, it's completely hidden from you.
But in Python, you see some of the ugliness
So in this case, how did it know I was talking about Mitch self and not--
Serena?
Yeah
So the question was how did Python know that the parameter was Mitch.
Because-- let me comment this out so we explicitly what I'm talking about.
So Mitch is a reference to a person that represents me on a computer, right?
Python knows that this object is of type person because it chunked that memory and made it of type person.
So when it sees this reference and it knows it's of type person and it sees this dot and get_age, what that automatically tells Python is, hey, this instance of person is trying to call a method.
Does this class have a method, get_age?
In this case, it does.
So python says, oh, it does.
So I'm going to call this method, person.get_age, and pass it Mitch as a self parameter.
Does it make sense in some way?
So it basically allows you to put in the name of the person instead of the attribute, instead of using the type?
Yeah, so you could do this.
I don't recommend that you do this.
I'm trying to illustrate where the self parameter is coming from.
It's much cleaner to write-- than it is to write this.
Fewer keystrokes, too.
Remember, programmers are lazy.
But it'll do the same exact thing.
So can I move on?
Wow, we've only got three minutes left.
I don't think I'm going to be able to get through this.
So my intent with this code was to show inheritance with shapes, because that's the classic.
I'm going to define a base class called shape.
And I'm going to give it some methods, area, perimeter, EQ, and less than.
For my purposes, shapes are equal if they have the same area, and they're less than if the area's less than.
So when we're doing logical comparisons, we're comparing areas.
So now, I'm going to define a class, rectangle.
It inherits from shape.
It's got two sides, or the lengths of two sides.
So the area's pretty easy, just a multiplication.
And the parameter is just the sum of the four sides.
And then, I'm going to define another shape, circle.
It's got a radius.
And it does the area and the perimeter in the way that you expect a circle to do the area and the perimeter.
And then, I've got square, which is a sub-class of rectangle, because a square is just a special case, right?
And all I'm going to do is when I initialize my square, I'm just going to call the rectangle's init method, and give it the same size twice.
So what this gives us is some inheritance, some nice properties.
So first off, when Python gets to this line, what's going to happen?
s is of type shape, right?
So what is the method for shape?
[INAUDIBLE]
What's that?
What is the method for shape?
What does the area method for shape do?
Because that's what I'm calling here
It's going to throw an error
So yeah, voila.
And the reason is that we haven't implemented it here.
This method is a placeholder.
It's saying that if I have a shape, it should have an area method.
And this is something that you should see on your problem set when you're doing PS5 with trigger
So it doesn't do anything on its own.
You can ignore it and just put [INAUDIBLE] for a circle
Yeah
It just reminds you?
It reminds you, yeah
It's like commenting
What's that?
So it's kind of like commenting?
It's like commenting, except that you get a little bit nicer error checking.
So it is a way of explicitly saying that if I have something that is sub-classed from shape, then I can be guaranteed that there is an area method.
But if I sub-class off it-- so rectangle, square, and circle-- they all have concrete implementation for areas.
So this is all going to run fine.
And then, I have my equality operator and my less than operator.
Those will run fine.
So let me-- all right.
The reason why this is useful, and the big idea, is if I have a list of shapes-- circle, square, and a rectangle-- then I can treat them all exactly the same because they all have an area method, which is defined on a shape.
They all sub-class from shape.
So it'll print all those areas.
And then, if I wanted to, I could even-- because I have a list, I can call the sort method.
And because I've defined the less than operator, it'll sort the shapes in descending order by area.
And that's what this last bit of code does unless I comment out my assignment.
There we go.
So I know I rushed through the shape implementation, and I'm sorry.
But I'm actually out of time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today's focus is probability and statistics.
So let's start with probability.
Let's look at probability for binary variables.
What do you mean by a binary variable?
It can take only two outcomes.
So it can take only two values.
For example, it could be 0 or 1, head or tail, on or off.
So we are going to call this variable A, for instance.
So A could be H, or A is equal to T. But that could happen.
That event could happen with a certain probability.
So by that, I mean the probabilities, like we are expressing the belief that the particularly event could happen.
So we could assign a value to that.
That is the probability of A taking value H. So here, the values of A and B-- sorry, here, the value of A can be either H or T, which means it has only two possible outcomes.
That's why we call it a binary variable.
However, P of A is equal to H can lie anywhere from 0 and 1, including 0 and 1
They don't have to be even?
Sorry?
They don't have to be even?
Even?
Even chance, even probability, like the same
Sorry, I didn't get your question
Even though they're binary, don't you need be able to have the same probability?
OK, we'll look at that later.
Like, this particular event can take a particular probability.
And we'll look at that particular case later.
But in general, a probability will always lie between 0 and 1.
And it can take any value between 0 and 1 since the range it can take is continuous, sorry discrete.
However, the value the variable can take is going to be discrete.
It can take only H or T. So that's why you call it a binary variable.
For example, take a deck of cards.
Here, the value could be, for example if you consider only one particular suit, then it can be any one of those 13 values.
So there, this variable is not binary.
However, the probability of a particular event happening is always between 0 and 1.
Now, let's look at some probability, like what you asked earlier is whether they will be equal, whether the probably of head and tail can be equal.
So let's represent the probability of A of H. This can be between 0 and 1.
What is the probability of A not happening?
So we call it by A bar.
Given P of A, can you give me P of A bar?
1 minus P of
1 minus P of A.
If there are two events happening, for example, you're throwing two coins, then we can consider their joint probabilities.
So let's say we have a coin, A, and this coin, B.
So this coin can take two values.
And so this coin can take another two values.
Sorry.
We know A can take H with probability, say I assume it's unbiased, so it'll be 1/2.
All these are going to be 1/2.
What's the probability of HT?
So now, we are considering a joint event, P of A is equal to H and P of B is equal to T. So in probability, we represent it by something like this.
P A- do you know what is that?
P A intersection B, you want both events to happen.
That will be P of A.
And in this case, it's P of B.
So we could simply say it's 1/4.
Why is this possible?
It's because these two events are independent.
The coin A getting head doesn't affect coin B getting a tail.
So it doesn't have any influence.
That's why these two events are independent.
The dependent events are a bit complex, to analyze.
Let's skip them at the moment.
So we know all these probabilities are going to be 1/4.
So we looked at a particular condition here.
That is, A taking head and B taking tail.
What about the condition, what about the case where either A or B takes a head?
How can we represent that?
So it will be something like A is equal to H or B is equal to H. Oh, probability at least 1, so by that, I can also represent something like this.
OK, here, this is sufficient anyway.
So what are the possibility events?
So these three events could give rise to this probability.
It's better if you can represent this in a diagram.
So let's go and represent this in a diagram.
This is A and this is B getting, say, head.
In one case, both can take head.
That is, this particular condition, intersection we earlier looked at.
So what is this whole thing?
That is, either A gets H or B gets H, which is this condition.
We call it P of A union B. Ok. Is there an efficient way of finding this rather than writing down all possible cases?
Is there an efficient way of finding P of A union B?
From high school maths, probably?
No idea?
OK. P of A union B is equal to P of A plus P of B minus P of A intersection B.
Because if you consider P of A, you would have taken this full circle.
When you take P of B, you would have taken this full circle.
So which means you're counting this area twice.
So here, we deduct it once.
OK?
Great.
So this is the basics of the probability.
Now, actually we looked at two events, two joint events here.
But we should have a formal way of looking at multiple events.
So how can we do that?
The first way is doing it by trees.
Let's say we represent the outcome of the first trial by a branch.
We can represent the outcome of the second trial by another branch from these two previous branches.
So this would be H HH HT TH TT.
And we know this could happen with probability 1/2.
So we know it's, again, 1/2, 1/2, 1/2, 1/2.
So this is 1/4.
Suppose we want to do this for an outcome of throwing dice.
Then, probably we would have 6 branches here.
Which, again, forks into another 36 branches.
So there should be another easier way.
For that, we could use a second method call grid.
We could simply put that in a diagram.
So this is the first trial.
And this will be our second trial.
So now, we can represent any possible outcome on this grid.
For example, can give you me an example where you throw the same number in both the trials?
Then, what would be the layout of it in this grid?
Throwing the same number in both the trials.
Here's the first trial.
This, the second.
Then it would be the diagonal.
If you want to calculate the probability, do you know the probability is the ratio between the outcomes we expect over all possible outcomes?
So here, we know there will be 6 instances in this highlighted area.
Compare that, 36 to all possibilities.
So it'll be simpler 6/36.
How can you find the probability of getting a cumulative total of, say, 6?
Then again, it would be very simple.
It could be 1, 5; 2, 4; 3, 3; 4, 2; 1, 5.
All right?
So it'll be 5 by 36.
OK?
So either by using trees or grid, you can easily find the probabilities.
Now, let's look at a few concrete examples.
Let's see.
Suppose we are throwing three coins.
Then, what is the probability of one particular outcome in that trial, in all three trials?
What is the probability, assuming that these are unbiased coins?
What is the probability of one particular outcome?
Because how many possible outcomes are there if you are throwing three coins?
Consider this tree.
First, it splits into 2.
Then, it splits into 4.
Then?
8, all right?
OK, so there are 8 possible outcomes.
So each outcome will have the probability 1/8.
so what is the probability of heads appearing exactly twice?
How can you do that?
Of course, you can write the tree and count.
What is the easier way of doing that?
Since we know this count, since we know this probability of a particular event happening?
How can we come up with the probability of getting exactly 2 heads?
It could be head, head, or tail-- so this is by enumerating all the possible outcomes.
So it could have been head, head, tail, where me put the tail only at the end.
It could have been head, tail, head.
Or it could have been tail, head, head.
In these three cases, you're getting exactly 2 heads.
So we are enumerating all possible outcomes.
And we know each possible outcome will take the probability 1/8.
So the total probability here is 3/8.
OK?
So this is one way of handling a probability question.
You can do that only because these are independent events.
And you can sum them.
We'll come to that later.
Suppose you are rolling two four-sided dice.
And assuming they're fair, how many possible outcomes are there?
Two four-sided dice, and assuming that each of them are fair-- that means unbiased-- how many possible outcomes are there?
Consider this tree.
First, it branches into 4, OK?
In the first trial, it's a four-sided dice, so there are 4 possible outcomes.
So it branches into 4.
Then, each branch will, in turn, fork into another 4 branches.
So there are totally 16 outcomes.
So what is the probability of rolling a 2 and a 3?
What is the probability of rolling a 2 and a 3?
Not in a given order, not in the given order.
Can anyone give the answer?
OK, let's see.
So we have to roll a 2 and a 3.
So which means it could have been 2, 3, or 3, 2.
And we know the probability of each event is 1/16.
So this will be 1/16.
And this will be 1/16.
So the total probability is 1/8.
What is the probability of getting the sum of the rolls an odd number?
What is the probability of getting an odd number as sum of the rolls?
Now, this is getting a bit tricky because now it's maybe a bit harder to enumerate all possible cases.
So how can we do that?
There should be a short cut
It can either be odd or even
Sorry?
You can either get odd or even
It can be either odd or even, right?
So it will be 1/2.
OK, there's another trick we might be able to use to get the answers quickly.
What is the probability of the first roll being equal to the second roll?
In the same line, you can think.
What is the probability of getting the first roll equal to the second roll?
It's quite similar to this.
Any ideas?
It's a four-sided dice.
There are 4 possible outcomes.
This is one case where it could be 1, 1, or it could be 2, 2, or 3, 3, or 4, 4.
And if it's inside a dice, it would be n, right?
So if it's n-sided dice, there and n possible outcomes desired, and totally n by n outcomes.
So you get 1/n probability.
What is the probability of at least 1 roll equal to 4?
At least 1 roll equal to 4?
This is very interesting.
These type of questions, you'll get in that Psets, I know.
Probably in the quiz, too.
What is the probability of getting at least 1 roll equal to 4?
OK, so what are the possible outcomes?
First roll, could be a 4.
And the second roll could be anything.
Or it could be 4, and the first roll could have been anything.
Or both could have been 4, but we would have considered that here, as well.
So what we had to do is we had to calculate this probability and this probability, add them, and deduct this, because this would have been double counted.
It's quite like, this intersection.
We want to remove that, and we want to find the union OK?
So what is this probability?
Since we don't care about the second roll, we have to care only about the first roll, our first roll getting 4, which is 1/4.
And this is 1/4 similarly.
And this is 1/4 by 1/4, so 1/16.
So it'll be 1/2 minus 1/16.
And when you give the answers, if it's hard, you can just leave it like this.
So this is what we call giving the answers as formula instead of giving exact fractions.
Because sometimes it might be hard to find the fraction.
Suppose it's something like 1 over, say, 2 to the power 5 and a 3 to the 2, something like this.
Or we'll say 5.
You're not supposed to give the exact value in this amount or even the fractions.
You can give such formulas.
You can give something like this, too, to give the inverse probability of that not happening.
Let's see.
Let's move into a little bit more complicated example.
A pack of cards-- what is the probability of getting an ace?
Anyone?
1 out of 2?
1 out of 2?
out of 52
Not a particular-- an ace, yes, just ace
Is it 4 out of 52?
4/52, yes.
Or if you consider one suit, it would have been like 1/13, right?
You could have considered one suit, and out of-- OK.
It's the same analysis, right?
OK. What is the probability of getting a specific card, which means, say, the ace of hearts?
It's what she said, yeah, 1/52.
What is the probability of not getting an ace?
[INAUDIBLE]?
Sorry?
1 minus--
1/13.
OK, this is where me make you solve the inverse probability.
OK, so that will come into play very often.
OK, now let's get into two decks of playing cards.
OK, what is the sample size?
What is the sample size of drawing cards from two decks of cards?
Two cards, actually.
You're going to draw two cards from two different decks.
Sorry?
OK. What is the sample size of drawing a card from one deck?
There are 52 possible outcomes.
So for each outcome here, we have 52 outcomes there, right?
So it's 52 by 52.
It's like the tree, but here, we have 52 branches.
So eventually, you will have 52 by 52.
This is where you can't enumerate all the possible cases.
So you should have a way to find the final probability, OK?
So in this case, what is the probability of getting at least one ace?
What's the probability of getting at least one ace?
This is, again, similar to this case.
Remember this diagram.
It's called Venn diagram.
Remember this.
So what is the probability of getting at least one ace, which means you could have got the ace from the first deck, or the second deck, or both.
But if you're getting from both, you have to deduct it because otherwise, you would have double counted it.
So getting an ace from the first deck is 1/13.
Second deck, 1/13.
Getting from both is 1/52 by 52.
Sorry, 1/13 by 1/13.
Sorry
Are you adding them?
Yeah, that's what I explained earlier.
You're doing two trials.
You could have got the ace from here.
And this could have been anything.
You could have got the ace from here, and this could have been anything.
You could have got an ace from both.
So you should add these two probabilities because we need a case where at least one card is ace.
But the problem is, this could have happened here and here.
And so you will deduct it.
What is the probability of getting neither card-- what is the probability of neither card being an ace?
1 minus that?
1 minus this, exactly.
OK, you're getting comfortable with the inverse probability now.
What's the probability of two cards from the same suit?
What is the probability of getting two cards from the same suit?
Now, it's getting interesting.
Two cards from the same suit.
So how can we think about this?
Of course, you can enumerate all possible cases and count.
We don't want to do that.
OK, you're going to use the grid here to visualize this.
OK?
It could have been a spades, or hearts, or clubs, or a diamond.
So we want two cards of the same suit, right?
So it's 4/16 possible outcomes.
Do you see that?
So see, we are using all the tools available at our disposal-- trees, grids, counting, Ven diagrams, inverse probability.
Yeah, you should be able to do that to get the answers quickly because you could have actually done-- you could have done something like this, too.
But it will take more time, right?
So this will be a simpler way of visualizing things.
What is the probability of getting neither card a diamond nor a club?
Neither card is diamond nor club.
That is tricky.
But since we have this grid, we can easily visualize that.
So if neither card is diamond nor club, then it could have been only these two values, right?
Which is, again, 4/16.
So there are 4 possible cases.
OK?
So what is the summary?
What is the take home message here?
In probability, the probability of the belief, or the way of expressing the belief, of a particular event happening.
Now, there could be several possible outcomes.
Out of those possible outcomes, you have a certain number of desired outcomes.
How can you find that?
You can either enumerate all of them.
You can put them in a tree, or you can put them in a grid.
Or you can use some sort of Venn diagram and come up with some sort of analysis.
Here, we start with our belief that the coin is unbiased, or we have a fair chance of drawing any card from the deck of cards.
So we have all these unbiased beliefs, or beliefs about the characteristics of each trial.
So we start from that.
Then, we find the probability of a particular event happening in a certain number of trials.
But what if you don't have the knowledge about the coin?
What if you don't know whether it's fair or not?
What if you don't know P of A is equal to H is equal to 1/2?
Suppose you don't know that.
Suppose it's P. How can you find it?
What you could do is you could simulate this.
You can throw coin several times and count the total number of heads you get, OK?
So it could be n of heads over n trial will give you the P, right?
This is a way of finding the probabilities through a certain number of trials.
It's like simulating the experiments.
It's called Monte Carlo simulation.
And using that, we try to find a particular parameter of the model.
You know how they actually found the value of pi at the beginning, pi?
It's again using a Monte Carlo simulation.
What you could do is for a given radius, you can actually check whether it lies within a circle or not.
You can simulate the Monte Carlo simulation.
And given this radius, you can come up with a particular location at random and check whether it's within this boundary or not, OK?
So then, you know the outcome.
You know the outcomes, right?
So suppose this is n_a, And the total outcome is n_t.
This gives you the area, right?
We know this is r-squared, and this is pi r-squared.
Sorry.
When this is 4 r-squared, this is 2r, right?
So using this, you can easily calculate pi.
So now, since we are going to come up with these parameters through repeating the trials, we need to have a standardized way of finding these parameters.
We can't simply say this, right?
Take this example.
You know this MIT shuttle right?
A shuttle arriving at the right time, or the time difference between the arrival and the actual quoted time can be plotted in a graph.
So if you put that it is spread around 0, right?
Probably, or we hope so.
OK?
Now, from this, we can see that actually the mean of this simulation will give you the expected difference in the time, the expected difference in the arrival time from the actual quoted time.
And we hope this expectation to be 0.
We call that mean.
Means is taking the average.
But this distribution might actually give you some information, some extra information, as well.
That is, how well we can actually believe this, how much we can rely on this.
If the spread is greater, something like this, then probably you might actually not trust the system, right?
Although the mean is 0, it's going to come early or late, right?
Which means it's useless.
Similarly, in this case, we have a spread around mean 0.
But if you take the score, the marks you get for 600, it could be something like this.
It's not centered around 0, right?
Hopefully.
It's probably, say, 50.
Then, we actually want the spread to be small or large?
We want the spread to be large because we want to distinguish the levels, right?
The students' level of understanding.
600.
Anyway, so the spread determines what is the variation percent in their distribution of the scores?
We measure that by a variable called standard deviation.
In this case, this particular sample will be different from its mean by a particular value, right?
We can express that as x_i minus its mean.
Let's call the mean mu.
So this would be the difference.
Standard deviation is summing up all the differences.
But the problem is, when you sum up the differences, it'll be 0, right?
The total summation of the differences will be 0 if that's how you get the mean because if you expand this, it'll be something like this, right?
Which will be n mu.
Should be equal to 0.
So we have to sum, or actually take the differences into account.
So, let's square this.
So now, it will no longer be 0.
Now, this gives 0, the differences.
It's the squared sum of the differences averaged across all the samples.
We call this variance.
And the square root of variance is standard deviation.
OK?
Now, having a standard deviation-- so we know the standard deviation tells you how spread the distribution is.
But can we actually rely only on the standard deviation to determine the consistency of some event?
Can we?
Probably not.
Suppose take two examples, one is the scores, 50.
And suppose the standard deviation is minus 10, plus 10, OK?
So the standard deviation is 10 here.
Suppose it lies in this form.
Consider another example, the weight, the weight of the people, like say at MIT.
And suppose it's centered around 150.
Now, if the standard deviation is, say, 10, then the standard deviation 10 here and the standard deviation 10 here don't convey the same message, OK?
So we need to have a different way of expressing the consistency of a distribution.
So we represent it by coefficient of variation, which is equal to the standard deviation divided by mean.
Now here, it will be 10/150.
Here, it will be 10/50.
So we know this is more consistent than this.
The weights of the students at MIT, it's more consistent than the marks you might get, or you get, for 600.
It might be true.
Now, what is for the use of the standard deviation?
How can we use that?
Let's look at this graph where suppose the mean is 0 and the standard deviation is, say, 5.
Consider another example where standard deviation is 10.
It might have been like this, OK?
Now, before that, let me sort of digress a little bit so I can explain this better.
We can take the outcome of a particular event as a sample in our distribution.
So suppose you're throwing a die.
So you get an outcome.
You can represent that outcome as a distribution, OK?
So here, there's x, which can take 1 to, say, 6.
And we can represent x_i as a sample point in our distribution.
So I don't know, it might be uniform, probably, we hope.
So it's with 1/6 probability, we always take one of these values.
OK.
But this might not be the case with all events.
OK, so what I'm trying to say here is you can actually represent the outcome of the trial in the distribution.
Or you can also represent the probability of something happening in a distribution.
How does it work?
OK, in this case, we throw our dice.
We get an outcome.
We go and put it in the x-axis.
It could be between 1 and 6.
And it takes this distribution.
In addition, what you could do is you could have, say, 100 trials.
So you throw a coin.
You take 100 trials.
You get the mean, you get the probability of getting a head.
And you have that mean, right?
So probability of getting a head for 100 trials, say, 0.51.
You do another 100 trials, you got another one.
So you have now another distribution.
So there's a distribution of probabilities.
So you can have a distribution of probabilities, or you can have a distribution for the events.
We handle these two cases in the p-set.
So probably you should be able to distinguish those two.
Anyway, so here in this particular example, let's take this as our mu.
Let's take this as our standard deviation.
And for the first distribution, let's take the standard deviation to be 5.
When the standard deviation is great, it's going to be more spread.
It's going to be more distributed than the former.
So here, say the standard deviation is 10.
The standard deviation is a way of expressing how many items, how many samples are going to lie between those particular boundaries.
So for a normal distribution, we know the exact area, exact probability of things happening.
If there's no mu, we know within the first standard deviation, there will be 68% of events lie in that area.
Within two standard deviations-- OK, one standard deviation, 68%.
Two standard deviations on either side, it's going to be 95%.
Three standard deviations, it's going to be 99%.
So suppose you conducted so many trials.
And you get the values.
And in the distribution, suppose mu, mean, is 10, and the standard deviation is, say, 1.
So now, with 99% confidence, we can say then the outcome of the next trial is going to be between what?
7 and 13, right?
So this is where finding the distribution and standard deviation helps us giving a confidence interval, expressing our belief of that particular event happening.
We will look at a few examples because you might need this in your p-set.
So this particular function you have already seen in the lecture.
But we need to understand this particular part.
Suppose you have a probability of something happening.
Suppose you estimated the probability of something happening.
Suppose you're given the coin is biased, OK?
Sorry, unbiased.
So we know p of H is equal to 1/2.
How can we simulate an outcome?
How can you simulate an outcome and see whether it's a head or a tail with this particular probability?
We do that by calling this function, random.random(), which is going to give you a random value between 0 and 1.
And you're going to check whether it's below this or not.
If it's below this, we can take it as head.
If it's not, it's tail.
And this will happen with probability 1/2, because the random function is going to return a value between 0 and 1 with equal probabilities.
It's uniform probabilities.
So to simulate a head or tail, you call that function.
You write the expression like that, OK?
Then, if you consider this example, for a certain number of flips, we simulate the event.
And we count the number of heads we obtain.
And also from that, you can calculate the number of tails as well.
If you know the total flips, you know the number of tails.
Using that, we are taking two ratios.
Now, the ratio between the heads and tails, and the difference between heads and tails.
We are doing this for certain number of trials.
And we're going to take the mean and standard deviation of these trials, OK?
So here in our distribution, what are we considering?
What is going to build our distribution here?
The ratios, right?
The ratios of the events.
And we simulated certain number of trials to get those events, OK?
Only if you simulate certain number of trials, you can actually summarize the outcome of the events in mean and standard deviation.
This is exactly like the difference in the times of the bus arriving and the quoted times.
Let's check this example.
Let's plot this and see.
It's going to take a while.
OK, that's another thing I want to explain here because since you're going to be going to plot-- we are going to use PyLab extensively and plot graphs.
You'll need to put a title and labels to all the plots you're generating.
Plus, you can use this text to actually put the text in the graph.
We will show that in a while.
Plus-- here, sorry.
If you want to change the axis to log-log scale, you can call this comma at the end after calling the plot.
Because you might sometimes need to change the axis to log scale in x and y-axis.
So this is the mean, heads versus tails.
And if you can see it, the mean tends to be 1 when we have a large number of flips.
So to get the consistency, we need to simulate large number of trials.
Then only it will tend to be close to the mean, OK?
This is sort of a way of checking the evolution of the series by actually doing it for a certain number of flips at every time.
So it's quite like a scatter plot.
A scatter plot is like plotting the outcomes of our experiments.
Suppose it's x1 and x2 in a graph.
So we are going to say-- so for example, suppose you have a variable, and the variable causes an outcome-- a probability of the coin flip, so p of H. And it can result in a certain number of heads appearing, say n of H. Now, you can do a scatter plot between these two variables.
And it will be probably a spread.
But we know that if you increase the probability of heads, the number of heads is going to increase as well.
So it would be probably something like this.
From this, we can assume that it's linear or something like that.
But the scatter plot is actually representing the outcomes of the trial versus some other variable in the graph and visualize it.
And let me show the last graph, and we'll be done with that.
So this, again, we actually know, instead of putting a scatter plot, we're actually giving the distribution as a histogram and printing a text box in the graph.
This might be useful if you want to display something on your graph.
I guess we will be uploading the code to the site.
So you can check the code if you want later, OK?
Sure.
See you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So we've got three main topics to talk about.
One is distributions.
The other is Monte Carlo methods.
And one is on regression.
So for distributions, which distributions have we learned about in class?
Hmm?
Normal
OK.
So we have normal.
What's another one?
Uniform
OK. And there's one more that he's kind of mentioned, I think, in passing
Exponential?
Yes.
So exponential.
So for uniform, what would this look like if I were to plot this as a histogram, and I have endpoints A and B?
Someone clue me in?
Hmm?
[INAUDIBLE] straight line
So it's going to be a horizontal line, right?
And if we were to look at the function for this, it would be-- the probability would be 1 over b minus a for all points between a and b.
So let's look at this graphically.
So this chunk of code should not be too difficult to understand at this point, right?
All we're doing is we're using the random number generator, randint.
It's going to return us an integer, random integer from a uniform distribution between a and b.
Is there Anyone that's puzzled by that?
All right.
We're going to do that for numpoints, and then we're going to plot a histogram.
The only parameter that I don't think you've seen here is this normed=True.
What this does is, normally, when you use the hist command in Python, it's going to give you raw frequency counts on the y-axis.
What normed=True does is it gives you the proportion of the points that wound up in a particular bin.
So I can actually show you both ways.
So does that look about right?
For 100 bins, got, what, 100,000 points?
Each one has about 0.01, so it looks right, 1% in each bin.
So that was normed.
If we do it un-normed-- see how the y-axis here has changed?
Before, it was from like 0 to 0.12, or [?
0-1 ?]
Now, it's from 0 to like 1,000.
That's all that normed primer does.
But this is what we would expect.
This is for integers.
And then, of course, Python also has a way of doing it for floating point.
So here, we are going to use the uniform command.
And then when I say, show continuous uniform, going to give it the a and b, 0 and 1.0.
And it's really not going to look all that much different.
It's just that the x-axis is from 0 to 1.
Ok.
So uniform is easy.
What does a Gaussian look like, or a normal?
Like if I were to plot it, what should this look like?
Bell curve
OK, it'll be a bell curve.
Where is its peak going to be?
Exactly in the middle?
At the mean
At the mean.
Thank you.
So the peak is going to be at the mean.
We usually denote it with mu.
And then it's going to fall off asymmetrically or symmetrically off on either side?
Symmetrically.
Now, a Gaussian can be specified fully using two parameters.
What are they?
You have one here, and then you have standard deviation.
So mean and sigma.
Now, the function for this is not something you're going to have to know.
But I wanted to show it to you.
And the stats major can correct me if I'm wrong.
So it might be a little scary.
I don't know.
It intimidated me the first time I saw it.
Does that look about right to you?
Yes
All right.
So the reason why I threw that out there is because what I want to do is show you the ideal form when we plot out this function, versus a bunch of random samples we've drawn from a distribution that is Gaussian.
So, I have a function make Gaussian plot.
All it takes is the mean, standard deviation, how many points we want to draw from the distribution.
And then I have a parameter here, show ideal.
And we'll get to that in a second.
The function that we use is called dot Gauss.
And it just takes a mean and the standard deviation.
We're also going to compute the ideal points.
So if I take the mean, and I go a couple of standard deviations in either direction on the x-axis, then I can plot out what the y should be according to this function here, and then just do a histogram.
If I want to show this plot, that's what that parameter controls.
It'll plot out the function.
And if not, then it'll just plot the histogram.
So let's see what this looks like with just the histogram.
So it looks like what we would expect.
We have the nice bell shape.
It's centered at 0, and it's got a standard deviation of 1.
These are the relative frequencies of a random sampling of points from a Gaussian distribution.
And we can see that if we look at the ideal version or the actual function, it matches very closely.
And then for various shapes, standard deviation of 2, different mean, different standard deviation.
So it's pretty easy, right?
Are there any questions on Gaussian distributions or normal distributions?
Ok.
So, the last one we have--
[INAUDIBLE]
Oh -- frange is a custom function.
So we actually define it up here
[INAUDIBLE]
Was kind of hoping I could slip that past you.
It's just like range, except instead of integers, it returns a list of floating point numbers separated by step argument.
So it starts at a lower-end range start, and stops at the stop, and then increments by step, until it returns a bunch of floating point numbers.
The last one is the exponential distribution.
And I don't know-- did he really explain what the shape looked like for this at all?
So we can go really quickly through it, because it doesn't sound he actually expects you to know it too deeply.
Basically, it'll like that.
And the function is-- you don't need to know it.
It's just there for your edification.
Lambda is greater than 0.
So I'm just going to show you what it looks like, and then we'll move on.
So here, the blue are the sample points, and the red is the ideal curve.
Just different values of lambda
Does it always have a downward slope like that for it to be exponential?
Yeah, in this case.
There's another family of distributions that we're not going to touch on.
But that is that for distributions for today.
Unless anyone has any questions, I'm going to move on.
OK.
So the next big topic is Monte Carlo methods.
So can someone give me an informal definition of what a Monte Carlo method is?
Really roughly, is it based on using a random method to try to approximate something that's not random, by doing it many, many times over?
Yeah, more or less.
It's trying to arrive at a solution by repeated sampling, or random sampling.
And we've seen many different applications of this.
But we're going to review them and kind of try and get a better understanding.
So the Monty Hall problem.
This is a Monte Carlo simulation.
So, one, what's the action that a person should take?
[INAUDIBLE]
All right.
And does anyone remember what proportion of the time if they switch they won?
2/3
Two-thirds, Ok.
So I happen to know this works-- maybe.
I think my program died.
OK, so it works.
Is this code confusing to anyone or cryptic?
I tried to make it a little bit simpler than the code that was in the handout for class.
We have a number of trials.
We're going to pick a door for the prize.
The player's going to choose a door.
If they choose to stay, and the prize is in the door that they chose, then stay wins.
And if they choose to switch, and the prize door is not the door that they originally chose, then switch wins.
So it's easy.
What I wanted to try and do is look at an intuitive explanation for this.
At office hours, we were kicking around different ways of explaining this.
And we went to Wikipedia, and we found this explanation.
So the idea is let's say that the contestant chooses door One.
So there's a 1/3 probability that they've chosen the door that has the prize behind it.
And then there's a 1/3 probability that it's behind door number Two, 1/3 probability it's behind door number Three.
The key to this kind of explanation is that if you consider both Two and Three together, then there's a 2/3 probability that the prize is behind one of those two doors.
So the player chooses, and then Monty opens a door.
There's a goat behind door number Three.
This new knowledge doesn't change, though, the probability that you chose the correct door.
So you still have 1/3 chance that One was the correct door.
And there's still 2/3 chance on this side.
But you know this one is 0, because you see the goat.
So this door has to a 2/3 chance of having the prize.
Does that agree with you?
So it's one way of explaining it.
I don't know.
I had problems getting this into my head.
Does anyone want me to try again?
All right
[INAUDIBLE] two doors the probability that your goat is going to be [INAUDIBLE] behind the door you chose [INAUDIBLE], so it's basically the same [INAUDIBLE]?
Same idea, but kind of negating it, and thinking of it from the negative direction.
Another explanation that was good was if you had a million doors, and you had 999,999 goats, and you had one prize, you have a one in a million chance of choosing the right door.
So now imagine Monty walking down and open opening up 999,998 doors, each with a goat behind it.
Well, now you have your door that's still closed, and the door that's mystery also closed.
The probability that you chose the correct door is still one in a million.
So if you see 999,998 goats, and one closed door, and you know that your door only has a one in a million chance, you want to switch to the other door, because that probably has the prize.
So different ways of thinking about it.
The probability problems and statistics problems, it always helps to-- or at least, I think it does-- to have an intuitive idea of what's going on.
So with that said, let's talk about pi.
Because this is one of my favorite Monte Carlo methods.
Because it's got a nice explanation.
So does anyone need me to talk about the idea behind this, like how this method works, or to go through it?
Someone's nodding.
So the idea is we have a square.
And its side is 2r units long.
So what's the area of the square?
So Asq ... squared, right?
Now, we still have a circle that's inscribed in the square.
And it's got a radius of r. So area of circle.
If we take the ratio of the circle to the area of the square, then we find have pi over 4.
Now, let's assume that I throw darts at this.
Wakes people up.
And there's a uniform probability that the point will land somewhere in the square here.
If I throw N of these, then I can expect pi over 4 of them, times N, to wind up in the circle.
And since I find this number and this number, and I want to find pi, I can just rearrange this.
That's how we get pi.
So let's go to the code.
We just have some easy code.
It gets a random point within a square that's from minus r to r, so 2r units long.
I have a function that makes a whole bunch of points.
And then I have a function that checks if a point is within a circle of radius r and another function that looks at a bunch of points and counts how many are within the circle.
And then I have my compute pi function here.
And all it does is you can either pass at some points that are already made, or just say, I want to have 100,000 darts thrown at this square.
And it'll make a whole bunch of those random points, figure out many are in the circle.
And then we have-- this would be m and numpoints N. If we multiply it by 4, that gives us pi, more or less.
So let's look at a couple of plots.
I have a function here, runtrials.
And what it's going to do is it's going to run a number of trials for a given number of points.
So what I want to do is I'm going to run 50 trials for each number of points.
And I'm going to have a points list that goes from 10 to 10,000 in 1000-point increments.
I'm going to run the trials and get the results.
And then I'm going to plot my results.
And why don't we just throw that out there?
Ok.
So on the plot, the blue line blue, horizontal line, that's the actual value of pi, as near as a computer can approximate it.
On the x-axis, we have the number of darts that we threw at the square.
And each red dot represents the result of one trial of throwing however many darts at a board.
So when you're down here, and you're only throwing 10 darts, you tend to have a very wide spread for the estimated value of pi.
As you increase the number of darts, you get much closer-- I would say shot group, but grouping it's probably more appropriate.
And it's much closer to the actual of pi.
There's nothing really unusual about this, right?
Nothing confusing?
So another way of visualizing this is to actually, well, look at the darts that are thrown.
So I have a function here, plot pi scatter.
And this is actually just going to plot this.
And it's going to do it for 10 points, 100 points, 1,000 points, and 10,000 points.
And we'll see why we can start converging on pi.
So this is with only 10 darts thrown at the square.
The value for pi is really pretty off.
And it doesn't really look very compelling.
In fact, one of the darts actually fell outside the circle.
Nine of the darts fell inside the circle.
So you're not going to get a real good estimate there.
The blue dots there represent being in the circle.
Red is outside.
So if we do it with 100 points, it starts getting a little better.
If we do with 1,000 points, starts getting better.
If we do it with 10,000 points.
Anyone confused?
So I'm going to move on and show you how we can use the same method to do numeric integration.
So here we go.
Here's that frange function again, so it's not confusing anyone.
What we're going to do is we're going to use a Monte Carlo method to integrate a polynomial.
So let's say that I have-- what I want to find.
I'm going to do it for-- because this is a numeric method, let's say do it from negative 5 to 5.
So I want to do this.
If you haven't had calculus or anything like that, don't worry about this.
But I think a lot of people have, with a couple of exceptions.
So this is an easy function to integrate, right?
But there are also some functions that are really hard or impossible to.
So that's where a lot of software packages actually use Monte Carlo methods to do a numeric integration for you.
But the idea is the same I'm going to take a function.
And this is going to be x-squared.
And then I'm going to take an x-min and an x-max.
These become my left and right boundaries.
And then I'm going to find the minimum of the function between these limits and the maximum of the function.
So you see what I'm doing?
I'm defining a rectangle.
So again, same thing.
Same principle.
I have the area of the rectangle.
I don't have the area of this guy.
That's what I'm trying to find.
But I know that if I find the ratio, the number of points that land in the square-- or the ratio that land in this curve versus the total in the square, then I can find this area pretty easily.
So this function, find function, y-min, y-max.
Does exactly what it says.
Just goes between x-min and x-max, and then finds where the function is a minimum and where it's a maximum.
So the function I'm calling f. It's one of the few single-letter variable names I'll use that isn't an index counter.
My random point generator, it's going to take the bounds-- x-min, x-max, y-min, y-max.
So it's going to uniformly produce a point that falls within this rectangle.
My make-points -- it just makes a whole bunch of these.
Then I have this function between curve.
What this tells me is if I have a point here, it'll return true, because it's between the curve and the x-axis.
If it's up here, it's false, right?
Does anyone not understand how that works?
Ah, you're all smart.
So here is our estimate of our main function, estimate area.
You give it a function, x-min, x-max.
I'm going to tell it how many points to toss.
And optionally, we can tell it that we already have points that have been tossed.
And the first thing we do is find the y-min and the y-max.
And then if we don't have points, we make them.
And then point counter counts how many times a point wound up between the curve and the x-axis.
And we just iterate through the points.
If it's between the curve, that means it's here.
Then, if it's above the x-axis, we're going to increment the point counter.
And then if it's below the x-axis, we're going to decrement the point counter.
So we're accounting for signs here.
So if we had a function that did this, we'd be able to properly handle it.
Now we get the rectangular area.
And then all we do is we multiply the rectangular area by the ratio of the number of points between the curve and the x-axis and the total number of points thrown.
And that gives us the function area.
So here's my function, x-squared.
And this is just a plot function scatter.
All this is going to do is just do the same thing I did with the circle.
And I am going to do this for-- if I tossed 10 points, 100 points, 1,000, 10,000, or a 100,000.
So let's see what this looks like.
Assuming that Python doesn't crash.
So not too nice.
100 points, 1,000 points, 10,000 points.
And then a whole mess of points.
Oh, I crashed it.
Hm?
Can't we just [INAUDIBLE]?
I'm sorry.
Say that again?
Calculate [INAUDIBLE] split up the x-axis to a lot of points, and then multiply those by the value function [INAUDIBLE] add them up?
You're talking about doing a Riemann approximation?
Yeah, [INAUDIBLE]
Or a Riemann sum?
So his question is, why don't you do something like this?
Divide up the x-axis into very small portions, like that, and then sum up the areas of these rectangles.
Yeah, you could do that
[INAUDIBLE]?
You know, I don't have an answer for that.
I can't say which one would work better.
Do you know, Serena?
I would say that right now, whichever one you prefer.
But I'll see if there's any actual research on whether or not one is better than the other.
It might turn out that there are certain instances where doing this sort of approximation is better than doing the approximation I'm talking about.
But I don't know.
Yeah, for this problem, you could definitely use that.
Is everyone good with this?
Anyone confused?
Any questions?
Yeah?
I think my concern is that you need a fantastically large number of darts to get a reasonably good integration [INAUDIBLE]
Yeah.
That is one issue with Monte Carlo methods, is that they do rely on large numbers.
So, yeah, sometimes they can take a while
At least for the purposes of this class, we don't need to be able to quantify the error or anything like that, right?
No.
You do need to understand that there can be error.
And you should also understand stuff like confidence intervals and confidence levels.
Are you OK with that?
Mostly.
But in order to get a confidence interval, you'd have to do several trials at, say, 100,000 points, and then--
Right, exactly.
You could estimate the error.
Like you could estimate it.
But in order to really get a good sense for how much variance there is, you'd have to do repeated trials.
So yeah
What I guess I was getting at was in order to get a sense of how big the error is relative to the number of trials--
Yeah
--without sort of analytically.
But I guess that's probably [INAUDIBLE]
I'm sorry, what?
That's not something that we're going to be asked to do, at least in this course?
Yeah, no.
The purpose is we want you to understand that when you do things like this, that there is some thought that has to go into, well, how many trials do I need to do?
How many points do I need to throw?
And you have to ask yourself, how much error am I willing to tolerate?
There's the joke that mathematicians call pi pi, and then engineers call it 3.14.
OK, so if everyone's done with integration, I'm going to move on to regression.
Oh, wait, now.
There's one thing wanted to touch on.
So we kind of looked at some toy problems with Monte Carlo.
And this is, I guess, a toy problem too, because it has to do with a toy.
Is everyone familiar with the game of Monopoly?
So I don't have to explain the rules too much in depth?
OK.
So let's assume that there are no factors that modify this distribution.
If I roll the die twice, then each one of these spaces has an equal probability of being landed on.
It's about 2 and 1/2%.
But there are certain rules that distort this distribution.
So you can land on Go To Jail.
You can roll three doubles, and get sent to Jail.
You can draw a Chance card, and get sent to Jail, sent to Go, or sent anywhere on the board.
And there are 10 out of 16 Chance cards that modify this distribution.
And for Community Chest, same thing.
There's 2 out of the 16 cards that distort the distribution.
So the question is, how do you do this analytically?
And I've tried.
It's hard.
I'm actually not sure if it's possible.
Well, this is a perfect example of where you would use a Monte Carlo simulation in order to arrive at the answer.
So if you actually want to take a whack at this problem, you can go to this site called projecteuler.net.
They have a whole bunch of mathy questions on there that are meant to get people to think about math and computer programming.
And you get little rankings the more questions you answer correctly, and stuff like that.
So there's a little competition.
But the question in this particular case was, what are the top three places you'll land on with all these factors?
And if you represent them as a number that is concatenated one after the other, what is the number?
What is the six-digit number?
But that's a fun problem.
So going onto something that's less fun, regression.
So can someone tell me what purposes we would use regression for?
Take a stab
Sure.
If you have experimental data which you believe to fit some type of theoretical model.
But experiments being experiments, they're not perfect.
You can't-- the data points exactly fall in the model, so you have to find which parameters from the model to pick so that your experiment [UNINTELLIGIBLE] best fits [INAUDIBLE]
Uh-huh.
So the idea is that you have a bunch of experimental data that has error.
And you want to be able to maybe find the underlying function of those observations.
And you would do that using regression.
So we have a couple of nice cools in Python for doing that.
Actually, before I move on, another reason is you can find the function.
But you can also then, once you find that function, you can use it to predict additional values.
So say you have a gap in your data, or you want to predict values beyond the range that you collected observations for.
If you do a regression, you find the function, find the parameters for the function, then you can use it to predict those values.
And what we mainly want you to understand here are the functions that you would use to do it, and how you would tell if you have a good fit or not a good fit, and the idea of overfitting.
So we have a little bit of code that demonstrates this, so a couple of helper functions that compute various values that you've seen before.
So MSE is the sum of the residual squares.
And then you have the total sum of squares.
So these will help you compute the coefficient of termination.
And what I'm going to show is let's say I define a function here.
In this case, I have it defined as x-cubed plus 5x plus 3.
I am going to, for a certain number of x values, apply the function and get the y value.
And then to simulate observational data, I'm going to perturb it using a Gaussian distribution.
So it's going to jitter the points.
And that's what the make observations function does, is it just adds noise to the y values.
And then I'm going to-- this function here plots out the measured or observed values, the simulated.
It computes a fit for one degree.
So in this case, I have two parameters, fit degree 1 and fit degree 2, because I want to do comparisons.
So it'll compute fit using the first degree and predict some values for the curve.
And then it'll compute the residual error and the coefficient of determination and plot it out.
And then it'll do the same thing for the second degree.
Let's see what this looks like.
Let's see Python not behave badly.
There we go.
The function that we plotted was, what, x-squared something, 5x-squared?
Let me see.
x-cubed plus 5x plus 3.
And we're plotting it from negative 2 to 2.
So this is what I'm talking about with the noise.
So each of these red dots represents some observation that's been disturbed a little bit.
And then I try to fit this with a first degree polynomial, and then a second degree.
And I see-- actually, my residual error is lower for my first degree fit.
That's interesting.
So I don't know.
At this point, I'd say just stop and don't proceed further.
But we know that that's not the right function.
So let's look at what we have for a third degree fit.
It actually worse, huh.
This is the problem with random programs, is that sometimes they fail you.
I would say that these are nice pretty plots, but they're not really telling me much, other than I can fit some lines to some points
What should it look like?
What are you looking for that's not there?
So we know that the function that we made the observations on is a third degree polynomial.
So it's a little puzzling why this first degree fit is better than our third degree fit.
That's the conundrum.
So maybe-- I wonder what would happen if I expanded the x range.
So let's say I go from negative 5 to 5.
Maybe it's just too little data.
That's looking a little better.
Now I feel better.
So the issue was that we just were going from negative 2 to 2, and basically it looked linear there.
So the first degree polynomial was doing fine.
But as soon as we go out and get a little curvy in there, we see that both the first and the second degree fits, they have pretty high error.
Their R is pretty good.
But when you compare them with, say, a third degree fit, you see that the error drops down dramatically.
And it's got higher coefficient of determination.
So what we would say in this case is that this third degree fit here is a lot better than the first or second degree fit.
And then we can also look at, say, a fourth degree fit, which in this case happens to have a higher error.
So that's a good thing.
And then if we look at a fifth degree fit, it also has a higher error.
So we'd say in this case that the third degree fit is probably our best bet, and we probably have a pretty good idea of what the function is for the underlying model
Which part of this is regression?
Well, the part of this that is regression is-- the part that actually does the regression is this poly fit method here.
And what you do is you pass it in the x values, the y values, and the degree of the polynomial that you want to fit to it.
I've hit the end of my material, unless someone has questions.
Comments, fears, trepidations?
Just [INAUDIBLE] having done some stuff-- like in Excel, you can fit curves with the R-squares?
Yeah
The R-squared values are really, really high, like really, really [?
wanting ?]
these fits, even though the fits are pretty terrible
Yeah
So that's weird to me
That is puzzling.
And it's quite possible that I have a bug
I wonder whether there were different definitions for R-squared that are maybe floating around in different places?
No.
I made a correction to this earlier.
And like I said, maybe I introduced a bug.
So I'm going to have to double-check my math.
Unfortunately, I'm not perfect.
I wish I was.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Number (2) -- (2.3).
It says if the code [INAUDIBLE] 0 would be the [INAUDIBLE].
I thought it was, you're generating random values for that?
Yeah, you were but if you look at what totes 0 is collecting, so if you look at the, where it draws a random number, j is indexing in totes, right?
So when j is 0 your standard deviation, which is also being indexed by j, is going to be 0.
So you're always going to get the same value.
Any more questions?
No?
OK, that was easy.
So in lecture we were talking a lot about clustering, we've been talking about clustering the past, is it two lectures?
And we had two different types of clustering methods, what were they?
Hierarchical and--
Heirarchical and K-means.
Can someone give me a run down of what the steps are in hierarchical clustering?
Something that breaks everything down into one cluster [INAUDIBLE]
So let's say I have a bunch of data points.
What would be the first step?
You're going to first assign each point to a cluster.
So each point gets its own cluster.
And then the next step would be what?
[INAUDIBLE]
Right, so you're going to find the two clusters that are closest to each other and merge them.
So in this very contrived example, would be these guys.
And then you're going to keep doing that until you get to a certain number of clusters, right?
So you merge these two, then you might merge these two, and you might merge these two, et cetera, et cetera, right?
[INAUDIBLE]
So you're going to set the number of clusters that you want at the outset, so I guess, for the mammalian teeth example, there was stopping criteria of two clusters, if I'm not mistaken.
So let's take a look at the code here that implements a hierarchical cluster.
So this is just some infrastructure code, it builds up the number of points.
We have a cluster set class, which we'll go over in a second, and then we just add, for each point we're going to create a cluster object, and add it to the cluster set.
Let's take a look at the cluster set.
Cluster set has one attribute, the members attribute, and it just has a set of points, or a set of clusters, actually.
And the key method in here-- or the key methods are merge-1 and merge-n. Merge-n is what actually implements the clustering here.
So you give it the distance metric that you're going to use for your points, the number of clusters that you want at the end of your clustering, the history tracker, and then you also tell it if you want to print out some debugging information.
So, which apparently is not used to this method, oh, now it is, merge-1.
Anyway, so while we have more clusters than the number of clusters we desire, we're going to keep reiterating.
And on each step we're going to call this function-- or method called merge-1 and just pass at the distance metric.
And all merge-1 is going to do is it's going to, if there's only one-- if there's only one cluster here, then it's just going to return none.
If there are two clusters and its going to merge them, and if there are more than two clusters, it's going to find the closest, according to the distance metric and then merge those two.
So then the return value is going to be the two clusters it merged.
Let's look at the merge clusters code.
All it does is it takes the two clusters and for each point, in both clusters, it adds it to a new list points and it creates a new cluster from those points.
And then it removes two clusters from members and adds the newly created cluster.
So then, find closest method.
So what's this bit of code doing here?
[INAUDIBLE]
Right, so we'll get to the metric in a second.
So it initially finds, it looks at the first two members in the cluster set and it sets minDistance to be that, and it sets toMerge to be these two members.
And then it narrates through every possible pair of clusters, in this cluster set and finds the minimum distance according to the metric.
So let's look at the cluster class.
All the cluster object does or class does is it holds a set of points.
It knows the type of point that it's holding because that becomes important, when we talk about the different types of things that we want to cluster, and then it also has something called a centroid.
All a centroid is, is just the middle of the cluster, if you take all of the points and average their distances, or average their location.
So these different functions, these just compute metrics about this particular cluster, right?
So single linkage dist., all this is going to do is it finds a minimum distance between every pair of points in the cluster.
And what does max linkage distance do?
I'm sorry I'm mistaken it finds the minimum distance between this-- a point in this cluster and a point in another cluster, I misspoke.
So what does max linkage distance do?
[INAUDIBLE]
The opposite.
I have to keep you talking or you'll fall asleep.
And then, averageLinkageDist?
Same thing.
This is why having meaningful function names is important, because it helps you explain code.
So it also has this method in here called update and what update does, is it takes a new set of points and it sets the points that this cluster has, to be these new points.
And then it computes a new centroid for this cluster.
And the return value is the distance of the old centroid from the new centroid.
And this becomes important in some of the algorithms.
Then there's just some bookkeeping stuff here, like members will just give you all the points in this cluster.
You all know what yield does, right?
[INAUDIBLE]
OK.
So yield returns a generator object, which allows you to iterate over elements.
So this was asked during the quiz review.
What's the difference between range and x range?
Right.
So if I have a range of values it actually returns a different type.
So I can print out this list, right?
In this case, it will print out the type of object it is, so this is accomplished using yield.
So if I wanted to write this myself, what this is going to do is going to return something called a generator object.
And all it does is, instead of holding all the numbers in memory, it's going to return them one at a time to me.
So like when I use range here, it constructs a list and it has all of those integers in memory.
If I use xrange it's not going to hold all the integers in memory, but I can still iterate over them one at a time
So within that function [INAUDIBLE] yield a bunch of times before the function, right?
Yeah
How is that accomplished?
Does it operate [INAUDIBLE] within the way you normally have functions [INAUDIBLE]?
Right.
So what this tells Python is that when it sees a yield, it's sort of like a return, except it's telling Python that I want to come back to this location at some point.
So it a return just returns out of the function completely.
What a yield does is it takes the value that is specified after yield, and it returns to the calling place in the program that value.
But then, when it comes time to get a new value, it'll return back to where this yield exited.
So kind of a way maybe seeing this is if I iterate over my xrange.
Each time it needs new value, it's going to go back inside this function and grab it.
So it looks like a function.
But what it's actually doing is creating what's called a generator object.
And it has these special methods for getting the next value.
So it's some nice syntactic sugar.
But it's pretty neat.
But that's what's going on with this yield statement here is that instead of returning the entire list of points, or instead of doing it in some other way, all it's doing is just yielding each point one at a time so that you can iterate over them.
So what else?
So here's a method for computing the centroid.
All we are going to do is total up where each point is.
And then, take the average over all the points.
Does that makes sense?
All right.
So the example we saw was mammal teeth.
And the way that that's accomplished in this set of code is we're going to define a sub-class of a class, point, that's call mammal.
And what point does is it has a name for a given data point.
It has a set of attributes.
And then, you can also give it some normalized attributes.
If you don't give it the normalized attributes, then it'll just use the original attributes.
So it becomes important when we talk about-- when we do scaling over data, which we'll do shortly.
So there's nothing really special about it except for this distance function.
It's just defining the Euclidean distance for a given multi-dimensional point.
So everyone knows that if you have a point in two dimensions, then if it's an xy point, then it's just x-squared plus y-squared -- square root of.
It generalizes to higher dimensions if you weren't already aware.
So if I want to find the straight line distance between a point in 3D, it's just going to be x-squared plus y-squared plus z-squared -- square root.
That's all.
And then, so on and so forth.
So all this does is it sub-classes point.
And it has this function, scaleFeatures.
And what scaleFeatures does is it'll take a key.
And in this case, we have to find two ways of scaling this data, of scaling this point.
So we have the identity, which is it's just going to leave every point alone.
And then, we have this 1 over max, which is going to scale each attribute by the maximum value in this data set.
And if we look at the data set, we know that our max value is 6.
So you could do that automatically.
But in this case, we're using prior knowledge of the data set that we have.
So why don't we do a cluster?
So this is going to do a hierarchical cluster, right?
And what we're going to ask, if I just specify the default parameters, all it's going to do is it's going to look for two clusters.
It's going to use the identity.
And it's just going to print out the history, like when it's performed the different merges.
Unless I have extraneous code that I'm already running.
So what starts off first is we get a lot of merges with just these single element clusters, right?
So I have a beaver with a groundhog, so I guess they're pretty similar in terms of teeth.
We have a squirrel with a porcupine, wolf with a bear.
And so eventually, though, we start finding clusters-- a wolf and a bear, I guess, are more similar.
But they're also similar with a dog.
So we're going to start merging multi-point clusters.
So we start seeing to beaver and groundhog cluster is going to get merged with the squirrel and the porcupine cluster.
So if you were to visualize this, the reason why it's called hierarchical clustering is-- which one did I say?
Beaver, groundhog-- these guys have been merged into a cluster, right?
They started out as their own cluster.
And they've been merged into their own cluster.
And then, the grey squirrel and the porcupine, same thing.
They started off with their own clusters at the beginning.
They got merged.
And now, what this step is saying is that these two clusters get merged.
So we're building this tree, or hierarchy.
That's where the hierarchical comes from.
So we use hierarchical clustering a lot in other fields.
So in speech recognition, we can do a hierarchical clustering of speech sounds.
So if I have say different vowels, and maybe a couple of consonants, I would expect to see, say, these kind of clustered together first.
And so what I might see is these would be fricatives.
But then, I might have some stops, like "t" and "b" that get merged first.
So it's a way of making these generalized groupings at different levels.
I don't know.
Does anyone have any real questions about hierarchical clustering?
So should I move on to k-means?
All right.
So what's the general idea with k-means?
So I start off with a set of data points.
What's my first step?
Choose your total number of clusters?
Right, so I'm going to choose a k. So let's say for giggles we're going to choose k equals 3.
And then, what's my next step?
Choose k's [INAUDIBLE]?
So we're going to pick k random points from our data set.
All right, and then, what do I do?
Cluster?
Then you'd cluster.
Yeah, all right.
So after we've chosen our three centroids here, these become our clusters, right?
And we're going to look at each point.
And we're going to figure out which cluster they're closest to.
So in this case, this is going to be a pretty easy clustering.
So all these points are going to belong here.
All these points are going to belong here.
All these points are going to belong here, right?
And then, we're going to update our centroid for each of these clusters.
And there's going to be a distance that the centroid moves each time we update it.
So in this case, the centroid moved quite a bit, right?
Then, we're going to find the maximum distance that the centroid moved.
And if it's below a certain cut off value, then we're going to say, I've got a good enough clustering.
If it's above a certain cut off value, then what I'm going to say is like, this centroid moved quite a bit for this cluster, right?
So I'm going to try another iteration.
I'm going to say, for each one of these points, I'm now going to look and try to the closest cluster that it belongs to based on these new centroids.
And in this case, nothing's really going to change.
So all of the deltas, all of the centroids, are going to stay the same.
So it's going to be below the cut off value.
And it's going to stop.
So what's an advantage of k-means over hierarchical clustering?
More efficient?
Yeah.
So let's say that I have a million points.
If I were to hierarchically cluster these, that means I'd start off with a million clusters.
And now, in each iteration, I'm just going to reduce it by 1.
I don't know, let's say 3, OK?
So on each iteration, we're just reducing it by 1, which, if that's all we were doing, would not be so hard.
It doesn't take too long to count down from a million on a computer.
But on each one of these steps, we have to compute the pairwise distance between each cluster.
So it's going to be n times n minus 1 comparisons on each step which, in this first case, works out to a lot.
And it doesn't get much better.
So approximately, right?
And it doesn't get much better as we go down.
With k-means, what happens is we just have to perform, on each step, if we have a million points and we have k clusters, we just have to perform k times 1 million comparisons.
Because for each point, we need to find the closest centroid approximately.
So the upshot is that k-means winds up being a lot more efficient on each iteration, which is why if you have a large number of points, you might want to choose k-means over hierarchical clustering.
What's an advantage, though, of hierarchical clustering over k-means?
Even though it's less efficient, what's another--
[INAUDIBLE]
What's that?
More thorough
More thorough
And you can get a lot of different levels that you can look at
Yeah, you can get a lot of different levels.
So you can look at the clusterings from different perspectives.
But the key thing with--
You don't necessarily know how many clusters there actually are.
Hierarchical clustering will tell you all of the [INAUDIBLE].
You can just go down the tree and--
Right, so you could go down different levels of the tree and pick however many number of clusters you want.
But the big reason-- or one of the kind of main advantages that hierarchical clustering has over k-means is that k-means is random.
It's non-deterministic.
Hierarchical clustering is deterministic.
It's always going to give you the same result.
With k-means, because your initial starting conditions are random because you're choosing k random points, the end result will be different each time.
And so when we do k-means clustering, this necessarily means that we don't necessarily want to do it once.
Like if we choose k equals 3, we might want to do five different k-means clusterings and take the best one.
So that's one of the big points with k-means.
There's a degenerate condition with k-means.
So if my objective is to-- if my stopping criteria is that the centroid doesn't move, what's a really easy way to make the centroid not move by choosing k?
What was it?
K equals n, right?
So if I have n points and I have k equals n, then all of my points are going to be their own cluster.
And every time I update, I'm never going to move my centroid.
So in your problem set, you're going to be asked to compute a standard error for each of the clusters and a total error for the entire cluster.
So what that is, is if I have the centroids, and I had each point in the centroids-- so what I'm going to do is I'm going to take the centroid for each cluster.
And I'm going to find the distance from each point in the cluster to the centroid.
And then, I'm going to sum up all of those distances over the entire cluster.
That's going to give me my error.
Not sure if this equation's totally right.
There might be a like of division in there.
But the general idea, what I'm trying to emphasize, is that we can reduce this number by just increasing the number of k. And if we make k equal n, then this is going to be 0.
So like I was saying with statistics, you never want to trust just one number.
With k-means, you never just want to trust one clustering or one measurement of error.
You want to look at it from multiple perspectives and advantage points.
So why don't we look at the code and try to match up all of that stuff with what you'll see on your problem set?
So the big function to look at for k-means is aptly named k-means.
And it's going to take a set of points.
It's going to take a number of clusters.
It's going to take a cut off value, and a point type, and a variable named maxIters.
So first step, we get our initial centroids.
All we're going to do is we're going to sample our points randomly and choose k of them.
Our clusters, we're going to represent as a list.
And we are going to, for all the points in the initial centroids, we are going to add a cluster with just that point.
And then, we get into our loop here.
So what this is saying is, while our biggest change in centroid is greater than the cut off and we haven't exceeded the number of iterations, or the maximum number of iterations, we're going to keep trying to refine our cluster.
So that brings up a point I actually failed to mention.
Why should we have this cut off point, maxiters?
It'll go forever?
Yeah, there's a chance that, if our cut-off value is too small, or we have a point that's on a border and likes to jump between clusters and move the centroid just above the cut off point, that we'll never converge to our cut off.
And so we want to set up a secondary break.
So we have this maxIters, which defaults to 100.
So with this set up, though, there's a couple of things you have to consider.
And this is, one, you need to make sure that maxiters is not too small.
Because if it's too small, you're not going to converge.
And you don't want to make it too large, because then your algorithm will just take forever to run, right?
Likewise, you don't want to make your cut off too small, either.
So sometimes you have to play around with the algorithm to figure out what the best parameters are.
And that's, oftentimes, more of an art than a hard science.
So anyway, continuing on.
for each iteration, we're going to set up a new set of clusters.
And we're going to set them initially to have no points in them.
And then, for all the points in our data set, we are going to look for the smallest distance.
So that means we are going to look-- we're going to initially set our smallest distance to be the distance from the point to the first centroid.
And then, we're just going to interate through all the centroids, or through all the clusters, and then find the smallest distance.
Is anyone lost by that?
Make sense?
Once we find that, we're just going to add that point to the new clusters.
And then, we're going to go through our update.
We're going to iterate through each of our clusters.
We are going to update the points in the cluster.
So remember, the update method sets the points of the cluster to be this new set of points you've given it.
And it also updates the centroid, or it updates the centroid, and it returns the delta between the old centroid and the new centroid.
So that's where this change variable is coming from.
And then, we're just go look for the biggest change, right?
And if, at some point in our clustering, the centroids have stabilized and our clusters are relatively stationary, then our max change will be small.
And it'll wind up terminating the algorithm.
And all this function does is, once it's converged or it's gone through the maximum number of iterations, it's just going to find the maximum distance of a point to its centroid.
So it's going to look for a point that has a maximum distance from its corresponding centroid.
And that's going to be the coherence of this clustering.
And then, it's just going to return a tuple containing the clusters and the maximum distance.
So it's not a hard algorithm to understand.
And it's pretty simple to implement.
There any real questions about what's going on here?
So the example that he went over in lecture with k-means was this counties clustering example.
So we had a bunch of data of different counties in the US.
And we just played around with clustering them and seeing what we got.
So if we made five clusters, and we clustered on all the features, wanted to see what the distribution would be for, say, incomes, what this function, test, is going to do is it's going to take a k, a cut off, and a number of trials.
So remember, we said that because k-means is non-deterministic, we're going to want to run it a number of times to find maybe we get a bad initial set of points for our centroids or for our clusters.
And that gives us a bad clustering.
So we're going to run it a number of times and try and prevent that from happening
How do we [INAUDIBLE] multiple runs, because [INAUDIBLE] really different clustering happens [INAUDIBLE] after you run it a couple of times?
It can be tricky, to be honest.
One technique you could use would be to have a training set and a development set.
What I mean by that is, you perform a clustering on the training set.
And then, you take the development set, and you figure out which clusters they belong to.
And then, you measure the error of that development set.
So once you've assigned these development points to the clusters, you measure the distance to the centroid.
And you sum up the squared distances, and you sum up over all the clusters.
Then if you do that a number of times, what you would do is you'd choose the clustering that gave you the smallest error on your development set.
And then, you'd say, that's probably my best clustering for this data.
So that's one way of doing it.
There's multiple ways of skinning the cat.
Was trying to think of a good aphorism.
And actually, that's what's on your problem set.
One of the problems on your problem set is to cluster based on a holdout set and see what the effect of the error is on this holdout set.
So did that answer your question?
Yeah
OK. And then, choosing k, there's different methods for doing it, too.
A lot of it is you run a lot of experiments and you see what you get.
This is where the research part comes in for a lot of applications.
So you can also try some other automatic methods, like entropy or other, more complicated measurements of error.
But don't worry about those.
For our purposes, if you get below the cut off value, and you run a number of iterations, and you've minimized your error on your test set, we'll be happy.
We want you to be familiar with k-means, but not experts in it because it's a useful tool for your kit.
Anyway, so all this code is going to do is it's going to run a number of trials.
It's going to perform k-means clustering.
And it's going to look for the clustering with the smallest maximum distance.
So remember, the return value of k-means is the maximum distance from a point to its centroid.
We're going to define our best clustering as being the clustering that gives us the smallest max distance.
So that's another way you would choose your best clustering.
Yeah, and that's all we're going to do for this bit of code here.
We're going to find the average income in each cluster and draw a histogram.
So I think this is actually done.
So we have five clusters.
And what they're showing us is that the clusters-- if we take the average income of the different clusters, they're going to be centered at these average incomes.
Let's see what some other plots look like.
This set of examples is using a point type that's called county.
And county, like the mammal class, inherits from point.
And it defines a set of features that can be used, or a set of-- it calls them filters.
But basically, if you pass it one of these filter names, like allEducation, noEducational, wealthOnly, noWealth, what this is doing is it's selecting this tuple of tuples.
And so if I say wealthonly, it's going to use this tuple as a filter.
And for each element in this tuple of tuples, it has a tuple that has the name of the attribute and whether or not it should be used in the clustering.
So if it has a 1, it should be used.
If it has a 0, it shouldn't be used.
So if we look at how that's applied, it'll get a filterSpec.
And then, it'll just set an attribute called atrFilter.
And if we see where that's used, then it's going to iterate through all of the attributes.
And if the given attribute has a 1, then it's going to include it in the set of features that are used in this distance computation.
So did that make sense at all?
No?
All right.
So the idea is to illustrate that if you use different features when you're doing your clustering, you'll probably attain different clusterings.
So in that first example I showed, we used all the features.
But in this example, we are going to look at only wealth.
And that includes the features, if we look at this set of filters here, it's going to include the home value, the income, the poverty.
And then, all the other attributes, like population change, it's not include.
So those aren't going to be used in the clusterings.
So this is going to change what our clusters look like.
So if we look at what we have for our clusters-- well, that's not very clear.
Yeah, so let's see what happens.
I'm not sure that's really going to show us.
Probably better if I show them all at once, right?
So this will take a while.
Yes?
Actually I had just one question [INAUDIBLE] but there is a method showing us [INAUDIBLE] iterative additionally?
Mm-hm
[INAUDIBLE]
It's like iterValues or iterKeys, yeah
I was wondering how to go about using that in actual code
In actual code?
So you know that if you have a dictionary, d, you can do like d.keys.
So remember I was demo-ing that code, the difference between range and xrange?
Same thing.
This is going to return an actual list of all the keys, right?
What's d.iterkeys returns is a generator object that gives you, one by one by one, each key in the dictionary.
So a lot of times, you guys in your code will use something like for k in d.keys(): to iterate through all the keys in the dictionary.
What this method does, though, is it creates an actual copy of that list of keys, right?
So when you call d.keys, if you have a lot of keys in your dictionary, it's going to go one by one by one by one through each of those keys, add it to a list, and then return it to you.
What d.iterkeys does is it skips that going one by one by one and adding it to a new list.
It just gives you a generator object which, when you use it in a for loop-- when you use it in a for loop, it's going to just yield the key one at a time without creating a separate list.
That make sense?
So will it be more efficient?
Yeah, it's generally more efficient.
And then, there's also, I think, iterValues, which goes through each of the values.
And then, I think there's iterItems.
And I think, if I'm not mistaken, if you do something just like that, so you do, for k, v in d, this is going to iterate through a tuple that contains the key and the values associated with that key.
And it's equivalent to doing this.
Make sense?
Yeah
So where were we here?
Oh, maybe I shouldn't have done this all at once.
Why don't we just look at two?
Why don't we take a look at what the average incomes, what the clustering gives us for average incomes for education versus no education.
That's probably not going to be a very good comparison.
Just doing five trials and two trials for each clustering.
So k-means is the efficient one, which means that hierarchical would take a long, long time.
There we go.
So these are the average incomes if we cluster with k equals 50 on education.
And then, there should be another one.
I didn't create the new figure.
So apparently there's a bug in the code.
I wanted to show the two plots side by side so you could see the differences.
Because what you should see is-- we would see a different distribution in incomes among the clusters if we clustered based on no education versus education level.
But my code is buggy and not working the way I expected it to.
So I apologize.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
You guys have been talking about graphs in lecture, right?
So what are graphs?
So graphs are a kind of formalism that have vertices and edges.
A set of vertices is-- you can think of it-- it's like a set of things.
And then the edges are the relationships between those things.
So the set of all your friendships and your friends' friendships could be considered a graph.
So if this is me and this is all two of my friends, then an edge between us would indicate a relationship of friendship.
But there's no edge here, so there's no friendship between these two
[INAUDIBLE]
Yes.
So you have vertices, which are also called nodes, and then you have edges, and they could also be called arcs.
So if you see any of these names, these two are the same, and these two are the same.
So what kinds of graphs do we have?
[INAUDIBLE]
So there's a directed graph, and there's an undirected graph.
So up on the screen here, is a directed graph, right?
And so-- I'm not a sports fan.
I think those are the Bruins, and I don't know what the other team is.
Do you know?
OK.
Yes, I think it's a hockey team somewhere up in Montreal.
So this is a representation or a graph representation of cities, which are the nodes and vertices and then the roads that connect the cities.
And, obviously, it's not to scale or accurate, but it's an abstraction, so we're OK with that.
And the question here is what's the path to get from Boston to Montreal.
So in this case, it's a directed graph, so that means that we can only go this direction from node to node, in the direction of the arrow.
So, really, there's only one way to go, right?-- two hops.
So that's a directed graph, and then an undirected graph is basically the same thing, except we can go either direction on the edges, right?
So on the directed graph in the previous slide, you could only go in this direction, so you had to make two hops in order to get to Montreal.
In this undirected graph, you can just make one hop to get to Montreal-- so really conceptually very easy
Can you have a directed graph in a case where there's one going from Boston to New York and another one going from New York to Boston?
Yes.
So-- and actually we'll see that in the code.
So the question was, can I have New York here and then Boston here?
Can I have this sort of relationship?
And the answer is yes.
This is actually just equivalent to an undirected graph
Probably you can have a directed graph in some cases like that
Yes.
So you could have-- and we actually have-- actually, I think this might have an example of that, so, yes.
So in this case, you have-- Hartford has a path to Albany and back in a path to New York City, but there's no path directly back to Hartford.
That work for you?
[INAUDIBLE]
What's that?
[INAUDIBLE]
Well, then I won't insult Hartford then.
I didn't do these graphs.
If I had done them, I certainly wouldn't have used sports teams because I know next to nothing about sports.
So I don't know the rationale behind picking the names, other than I'm supposed to say that the reason why the Bruins want to go to Montreal is because they want to kick Canada's butt
They did
Did they?
They already have
Oh.
Well, then this is a very apropos slide then.
So what we have up here is a weighted graph, right?
So the undirected and directed graphs have been really easy so far because they're just defining the fact that there is a relationship that exists between two vertices, right?
So what a weighted graph does, though, is it says that not only is there a relationship between these two entities-- these vertices-- but it also has maybe a cost associated with it.
So if these represent the road networks, then these represent, kind of, the total costs-- the weights on these edges.
So in saying that in order to get the Hartford, I need to pay $1.00; and then to get to Albany, I need to pay $3.00; and then to get to Montreal, to pay $6.00.
So a common question-- on weighted graphs in general-- is what's the least cost path from here to here.
So can anyone tell me?
From Boston to Montreal?
Yes-- what the least cost is from Boston to Montreal
It's-- Hartford is your only choice
Right
Then New York and then Hartford
Right.
So-- and the cost for that is $9.00, right?-- because you got $1.00, $7.00, $1.00-- sum $9.00.
There are actually two paths to get from Boston to Montreal.
The other path is from Boston, Hartford, Albany, Montreal, but the cost of that path is $10.00.
So the question is how do you figure out which path is shortest, right?
So did he talk about breadth-first search and depth-first search at all?
He only talked about depth-first
He only talked about depth-first.
OK.
So we need to do breadth-first.
So before we do breadth-first, can someone define depth-first for me and maybe walk me through it a little bit?
So let's take this off the screen, and let's assume that I have a very simple graph.
I'm going to start here.
I want to end here.
And I have-- So I'm starting at v0, and I want to get to v8.
Let's call this a directed graph, so you can only go one direction.
It also doesn't have any cycles, so that makes it a little easier.
So if I'm starting here in depth-first search, what do I do?
Pick a daughter-- Pick a daughter?
So you pick one of your children, so you're going to pick v1 or v4.
Now, you go to this node.
What do you do now?
Pick a daughter
Same thing again.
So you pick another daughter and because this is a really trivial example, we just walk down the line until you find the node that you're looking for.
Or if you don't find it, and you have no more-- there are no more children to look at, then there's no path that exists.
But we can get to our goal node here, right?
So what do we do once we find this node?
[INAUDIBLE]
We save it off somewhere.
It becomes a path with just itself, and then we return that to who-- And then we say, OK, well, where did I come from?
Well, I came from v7.
So, now, I know that my shortest path from v7 to v8 is going to be v7-v8.
And then I'm back here, I'm going to add wherever I came from here.
So the idea is that I grow my shortest paths backwards.
Right?
And, actually, the shortest path is the top one here, so--
If you hit a branch in v7, can you go back to v7 would you jump to the next branch?
So, yes.
So if I had something like this, and the answer would be-- let's say that it loops around-- so now we have a cycle.
So it becomes interesting.
So let's say that I've reached my goal here, and I know that if I'm at my goal, then my shortest path is just my goal, right?
So now I return here and I say, well, what would be my shortest path in this case?
Does this child have the shortest path?
When I get here first to v7, I asked the question, what's the shortest path to v8 from either of my two children.
So I'm going to look at all the children of v7, and I'm going to find what's the shortest path from v8-- or from this node to the end node-- and then from this node to the end node.
And, obviously, this one's the shortest because it is the end node.
So now, I know that my shortest path is this plus myself.
And so that means that the shortest path from v6 to the end node is going to be this path, plus this.
And I don't know if that's getting any clearer.
So, really, if we start out at the beginning here, we're looking at this first node-- we ask the question, what's the shortest path from v1 to the end, and what's the shortest path from v4 to the end?
And we choose the shortest of those two paths as our answer, then we just add ourselves to the beginning.
And that's all we do for each of these nodes.
We ask, of the children at each of these nodes, what's the shortest path, and then we add ourselves to the beginning of that path and return that as our answer.
So if we are to look at that in code-- so you guys have all seen the graph object in class with the node and edges?
Yes or no?
OK So here is shortest path depth-first.
So there's a lot of debugging code here, but-- and some administrative stuff-- so all this is doing is just making sure that the nodes we're looking for are in the graph.
And, actually, let me backtrack.
So when we first call shortest path, we're going to call it with a graph object.
We're going to start in an end node.
And we're also going to have this parameter visited, which keeps track of the nodes that we've already seen.
And we'll get to that in a second.
So one of the first things that we do that's of any importance is, we check to make sure that the start and end nodes are actually in the graph, because you can't get from one to the other if they don't exist.
And now we're going to construct a path or a list that just contains the start node as its element.
And then we're going to check to see if start is equal to end.
So if we're already at our goal, then the shortest path is just us, right?
We don't have to go anywhere
[INAUDIBLE]
For comparison purposes.
I mean, if you look at the definition of node, it's pretty sparse-- pretty spartan.
If we wanted to make it so that we just added the node object itself, would have to add an underbar equal method-- stuff like that-- and in this case, we don't want to bother with kind of complicating it like that.
So if we're not at the end, though, now, we need to figure out what the shortest path is from each of our children to the goal node, right?
So we have a variable that we call shortest, and that's going to keep track of what our shortest path so far is, and then we're going to iterate through all the children in this node.
And if the node is not in visited-- and that's where this parameter comes in-- we're going to say, OK, well, let's take a look at it.
And then we're going to say-- we're going to add it to visited so we know that we visited this node.
It sounds kind of cyclic, which is funny because we have visited so that we avoid cycles.
Because if we've already visited a node and we figured out what its shortest path is, why would we want to visit it again?
If we're on a path, and we're saying-- let's say I have-- I'm trying to figure out the shortest path from v1 to v4-- and I'm using depth-first-- and so I decide depth-first first goes to v2, then to v3.
Now, it's got two choices on which nodes to get the shortest path for.
Let's say, for some reason, when it gets the list of children, it's going to get v1 and v4, and if it's looking at v1 before it looks at v4, then, if we didn't have that check in there, just to make sure that we are not visiting-- or looking at other nodes that we've already visited-- then the algorithm would just go here, and it would repeat itself in a cycle, like that.
So that's what that visited parameter is doing, is it's preventing that cyclic check.
So now we ask the question-- or we make a recursive call to shortest path, right?
And the only parameter that changes is the start node.
And we're going to ask it, what's the shortest path from this node to the end.
And it's going to call itself again and return an answer.
And if it doesn't return anything, then we're just going to ignore it and continue.
But if it returns something, and either we haven't found shortest path yet, or the length of this new path that it's found is shorter than the shortest path that we've already found, then we're going to record it, and say that this is our new shortest path here.
And then we're just going to keep iterating through all the children of the node until we've exhausted all possibilities.
And then once we finish going through all the children, we're going to say-- if we found the shortest path-- that means that there exists a path from one of its children to the goal node-- then we're going to add it to our existing path, which is just ourself.
So we're adding ourselves to the beginning of the shortest path that it found.
And then we will return it.
So it's kind of growing the shortest path from the back to the front, right?
Breadth-first search works in the opposite direction.
So the way that-- well, first, is anyone confused by depth-first search?
[INAUDIBLE]
So this line here-- this is-- well, this if statement first is checking to see that we've got-- that one of our children has a shortest path.
It's possible that none of our possible children leads to the goal node, so let's say that I have another kind of subgraph on here.
When v2 is my start node, I'm still going to check these two children.
And let's say that my goal node is to get to v8, right?
Well, I'm still going to check to see what the shortest path is for both of these children.
Well, once I use this is my start node, there's obviously no path to the actual goal node, so the depth-first search call, or the call to the shortest path, is going to return none in this case.
And we need to check that, so that's what that bit of code is doing there.
It's saying if there is a shortest path, then we're going to just add ourselves to the front of that shortest path, and return that as our answer.
But if there is no shortest path from one of our children-- from any of the children on the start node-- to the goal node, then we're just going to return none as our answer because we can't get to the goal node from where we are.
Did that work for you?
So why don't we take a look at how this is working.
So we're going to try DFS on undirected graph.
And the code that does this is called Test 2 here, and all it does is, it creates a graph with 10 nodes.
And, in this case, it's going to be an undirected graph.
And we're going to create a bunch of edges.
So-- is that diagram you sent out, is that the representation of it?
This is the code from lecture, Professor [INAUDIBLE] code.
So it only uses 5 nodes
So we have this graph, and what we're going to do is use depth-first search to compute the shortest path from here to here.
So this is showing the depth of the recursion, right?
So we start off on node 0, and then it starts looking for the shortest path from 1 to 4.
And at the same depth, it's going to try and find the shortest path from 2 to 4.
So it starts out here, and then it asks what's the shortest path from 1 to 4, and then what's the shortest path from 2 to 4.
And so when it's looking at 1, now it's going to ask what's the shortest path from-- I want it to do that-- Hey, Sarri?
Is there a bug in your code?
Is there?
So is this the lecture code?
None of this is mine.
I did the breadth-first search.
This is the depth-first?
Yes, because it seems like it's checking node 0 twice
It didn't do that on mine
So it's going from--
Oh, no-- because there's a directed-- is from 1 to 0, right?
Yes.
So what it's doing-- what the code does is, it says-- it does a depth-first, so first, it looks at node 0, and then it goes for child in-- for all the children nodes.
What's the first child of node 0?
It's node 1
Oh, because 0 hasn't been added to the visited list
Right.
And then--
And then it asks, what are all the children of--
Well, no.
The print statement comes before it discovers that checking node 0 is an invalid path.
I forgot to add another print statement.
If you go to the code--
OK.
So where are we at?
Yes.
So see how I have the very first at the top of the function?
See how there's the if to print?
I say that, but then there's this check here if it's not in visited.
If it is in visited, there's no print statement that says--
--that says that-- you know-- it's--
That-- yes.
So, basically, what's happened is, when we do that second check where we end up finding the second path from 0 to 4, it ends up hitting that test that says, we've already visited node 0, so we don't continue that
I got it.
I got it.
That was just a little--
There was no print statements in the code before, and I was having a really hard time figuring it out, and I did this, and this made it a lot more clear-- I thought-- to try and figure out how the depth-first search was working
No, I think so, too.
It's just maybe we should add a couple extra--
Yeah, I agree
Sorry.
I was perplexed-- because I'm like, uh-oh.
So it's working correctly.
Is there any confusion on depth-first search, in spite of that?
OK.
So let's start our breadth-first search.
So you guys covered depth-first search in lecture, so we need to do breadth-first search.
OK.
So the idea behind breadth-first search is that instead of asking the question, what's the shortest path from v1 to v4 and adding ourselves to the front of that, to get the shortest path, we're going to build the paths from the start outward-- or from the start forward-- so, in this case, we're building the paths from the goal backward.
In this case, we're going to say, I'm at v0, so my current partial path is v0.
Then I'm going to look at v1 and v4.
And so I'm going to have-- I'm going to say, now, this is my list of partial paths to the goal.
And then, for v1, I'm going to say-- I'm going to ask what its neighbors are, and we have v2.
And then, for v4, we're going to ask the same question.
So now we're building our partial paths.
So, conceptually, what we're doing is, we're just maintaining a list of all the paths that we-- or all-- the history of nodes-- or paths that we've been looking at and just kind of going out one by one by one.
So if we look at this in code, shortest path-- BFS-- has a slightly different call signature than the DFS method.
So we still get a graph, and then we get this variable called paths.
And what paths contains is the partial path, or list of the partial paths of tuples.
So the format of this is, each element has a list of the nodes in the path and then the length of that particular path.
So we also have the goal node.
And what we do-- the first thing that happens is, this pass gets shorted-- or sorted.
So it's sorted by the length of the path.
So is anyone puzzled by this lambda here?
Yes
OK.
So when you call the sorted function, you can pass it a list.
You can also pass it this key parameter.
And this key parameter is a function that takes an object, and it'll return, kind of, the value associated with that object.
So, in this case, each of these paths contain-- each of the elements and paths contains a tuple.
And what this key function does is, it says, each element in this path has got a value that is equal to the length of that path.
And this length parameter-- that's just the second element in the tuple, right?
So when sorted does its work, it's going to call this key function on each element in path.
And what this function has to do is return the value associated with that particular element
Is lambda just a key word that generates an effect?
Lambda is what's known as an anonymous function.
And he covered that in a lecture at one point
[INAUDIBLE]?
Did he just use lambda?
He blew past it, so-- this is something called lambda.
Ask your TAs
Well, since I'm your TA and I'm here-- so real briefly-- lambda is a way of, kind of, doing really simple-- not really simple functions-- but it's a way of creating anonymous function.
So let's say that I want to create a function that squares a number.
This is exactly-- well, not exactly-- but, it's equivalent to this.
I can use g as a function just as easily as I can use h. That's all.
And it's useful for situations where you have elements that don't have an order to find.
So like, sorted needs to know a value of an element in order to put it in order-- in order to do the sorting, right?
So that's what this function does, is it gives each element in that list a value so that the sorting can do its job.
So the key that we want to sort on, or the item that we want to sort on, is the length the path.
And the reason why we want to do that is, we're going to take the first partial path that exists in our paths, and for every node in the shortest-- or for every node in the-- for every child of the last node in this path, we want to check to see if it's the goal, and if it's not, then we're going to append a new partial path, which is the path that we're looking at, plus one of the children nodes.
And then the length of the path to this value called new paths.
So it's saying-- this node that I'm looking at-- let's say that this is my partial path.
What it's doing is, it's looking at this last node in this path, and it's saying what are all the children of this node?
And if none of the children are the goal node, then it's going to create a new set of paths that are composed of this path, plus all the children
So if v2 also went to v10, would the new path be easier [INAUDIBLE]?
No.
So we should probably get rid of this.
The reason why breadth-first search has its name, is that it doesn't try to go towards the end immediately.
It grows gradually.
So if you can kind of envision a graph like this.
Depth-first works by going-- by proceeding down until it finds the goal node for each of the possible paths.
Right?
What breadth-first search does is, it starts at a node, and then it builds all the paths from that node.
And then for each of these, it builds the paths.
So that's what I'm saying when I say partial paths
[INAUDIBLE]
Yes.
It's a new partial path, so--
So it would be v0, v1, v2, v4-- the next one.
And v0, v2, v [INAUDIBLE]
So let's say this is v0, v1, v2.
Let's say that I have this partial path already.
All right?
When I build a new partial-- when I'm looking at this path, which is what I'm doing when I pop it off the front of the list of paths that I already have-- I'm going to look at all its children.
So I'm going to look at these two nodes right here.
And then I'm going to say-- let's say that my goal node is here, and then I'm going to say that these aren't my goal nodes.
So I'm going to create new partial paths, and one of them is going to be v0, v1, v2, and v4.
So I already have that.
And the other partial path is going to be v0, v1, v2, and v5
You just find every single possible path to get the tuples [INAUDIBLE]
What's that?
[INAUDIBLE]
Until you reach the goal node.
So with depth-first search, you do have to-- well, no, not even with depth-first search.
There's some interesting properties about the different searches, but we just want you to be familiar with how they go about doing their task.
So the idea is that we start off with a partial path-- just this guy-- and then we find all the other partial paths.
So it'll start off with just this guy as its list of paths-- so one element-- and then it's going to pop this off the front.
It's going to say, OK, I'm looking at this partial path.
What are the additional paths that I can build off of this?
And I can build four additional paths off of this.
So now I have a set of four paths that I want to look at.
And then it takes a look at, say, this partial path of v0, v1, and it says, what are the paths that I can build off of this, and then adds it to the end.
And then it says, what are the paths that I can build off of this guy-- adds those to the end-- what are the paths that I get from this guy-- adds those to the end, and so on and so forth
So you keep track of all the paths that you [INAUDIBLE]?
Right.
Oh, OK. Is that what you meant?
You hit a node-- and there's another possible thing-- but if you don't hit a node-- if there's some paths somewhere else in the graph, then it doesn't ever reach into that.
It just doesn't account for it
Well, I mean, it is possible for breadth-first search to go find nodes that don't reach the goal, but what's going to happen in those cases is, they're going to-- so let's say that there's such a path at the front of this paths list, and it doesn't have any children.
So you can't go anywhere after you've gotten to this node.
Then what's going to happen is this for loop's not going to execute, right?-- because it has no children.
And so it's just going to be discarded as a possibility.
So I mean, the key thing is that you're generating these new partial paths for every node that you're looking at.
And you're adding those to a list or a queue of partial paths that you need to examine in the future if you don't find your goal.
If you do find a partial path that has a child that is the goal, then you can just immediately return that path because you know it's going to be the shortest path, in this case.
So-- I don't know
So what happens when you find two partial paths that both reach?
Well, then it depends on which one appears first in your list.
So you're asking-- let's do something really simple-- if I have--
Oh, OK.
So that makes sense
So at some point, you're going to have a partial path right here, or a list of partial paths that consist of v0, v1, and then v0 and v2.
Whichever path you ultimately wind up returning is going to be dependent on whether or not this one comes first in your list of partial paths to check, or this one comes first in your list of partial paths to check.
Did that kind of answer it?
So is everyone good on breadth-first search, conceptually?
Is the code flummoxing anyone?
How do you find which one is the shortest?
If you're building your partial path-- so you've popped off this path that you're examining-- and you look at all its children, and one of the children is a goal node, then you know that you've found the shortest path, right?-- because you've been building this incrementally, one by one by one.
And then if you've found the goal node, then you know to-- you know, you already built up the shortest path possible.
Because if the goal node had been on a shorter path before, you would have found it already
That would be a different scenario if that path were weighted
Well, if the paths are weighted, then it's a little different because here, we're just doing shortest path.
What you're talking about is doing a least cost path, in which case the main difference would be how it sorts its list of candidate paths, right?
[INAUDIBLE]
Hmm?
When it does sort it, it would say, keep this [UNINTELLIGIBLE] path [UNINTELLIGIBLE PHRASE]?
Yes.
So in this case, our lambda function is saying that we're going to sort these lists based on their length.
But if I wanted to say, sort them based on the sum of their weights, then I would have a function that sums up the weights on that path and uses that in the sorting in order to select the next partial path to search
Although, in that case, you'd actually have to ennumerate every single possible path, to show you found the lowest cost path
No
Because doesn't-- I might understand this-- is that you find all the paths of length one through a certain point and all the paths of length 2
Right
And so if you're worried about cost, it's possible-- in principal-- that your longest path is your lowest cost path
Wait.
You're not sorting it by length then
Right.
[INTERPOSING VOICES]
But what he's saying is that because you're growing it by one each time, then it's possible that-- let's say that I have partial paths of length 2, but my least cost path is actually of length 5, then even if one of these nodes reaches-- like, say that v2 reaches the goal before the actual least cost path, then you can't stop.
You do have to integrate through all of them
Well, you just sort it by cost.
I think that it's a better example to say that if you have one partial path that has cost 7
Well, actually, wait.
No.
You're actually right.
Because if you sort it by cost, then your least cost path is going to be in front of your queue all the time
No, I don't think that's true.
Because imagine this-- imagine you had a graph that was like-- this is your start node.
This is your end node, and you go like this, this, this, this.
And so you make a breadth-first, where you go to this node and this node, and the cost of this is 1 and the cost of this is 8.
The cost of this is 20 and the cost of this is 1.
This is your shortest path, or this is your cheapest path, but when you add them-- they're both of length 1-- so you explore this one first because it's the cheapest at a cost of 1, but the total cost is 21 to continue from this node.
But the total cost is [UNINTELLIGIBLE] that node.
So I think that he's right.
Right?
I thought you had to expand all nodes when you're doing breadth-first.
That's why you do, like, [UNINTELLIGIBLE].
But, yes.
For weights you have to expand all of them
So--
[INAUDIBLE]
Yes?
[INAUDIBLE]
Possibly, yes.
I'm blanking on the answer.
But in this case, we don't have to enumerate all paths
We're just looking for the shortest path-- we can just find the shortest path
Right.
But if you're incorporating costs, then you would use a different algorithm, so it'd be, like, actual shortest path algorithm or something like that.
So why don't we just run this
So is one better than the other, when you [INAUDIBLE]?
They have different running characteristics.
It really depends on your application.
The nice thing about depth-first search is that it's fairly memory efficient because with breadth-first search, you're storing all the possible paths-- all the possible partial paths.
With depth-first search, you only have one path and memory at a given point, right?
You have the path that you've already found from one of your children to the end node, plus 1.
But with the breadth-first search, you're going to have a list of all the partial paths that you have to search through.
So let's see.
This doesn't make sense
I don't think it matches the picture
[0, 4] is a path, isn't it?
From the graph?
Right.
Yes.
No, it's not matching the picture
[INAUDIBLE]
What's that?
[INAUDIBLE]
Is what?
It was undirected
Yes, it was an undirected graph-- that's why.
So if we run it directed, let's see what happens
0 to 4 is still in the graph
Yes.
You are absolutely right.
Oh, you know what's going on-- it's running both.
It's running the depth-first search
No, it's running both the directed and the undirected
Well, it's also running the depth-first search as well.
That was my question, because I was looking at this, and I was, like, that's not breadth-first search, that's depth-first search.
This, on the other hand, is breadth-first search.
So there we go.
So if we have a graph-- and let's see-- so it starts off with node 0, and then it builds a new list of partial paths with [0, 1], [0, 2], so it's got, in this case, this partial path and then this partial path.
And then it expands one of them.
So, in this case, I guess [0, 1] was the path that was first on the list, right?
So it's going to expand the children of 1, which is just 2 and itself.
But it's going to avoid cycles
[INAUDIBLE]
What's that?
[INAUDIBLE]
Oh, yes.
And then it's going to expand this partial path-- the [0, 2] -- because it's the shortest one, and it's going to add that new partial path to the end.
And then it's going to expand this guy, and add the expansion here, and then it's going to expand this guy.
And it turns out that one of the children of 3 is 4.
So it's found the shortest path from 0 to 4.
Make sense?
So, can you go back to [INAUDIBLE]?
Then why does it say [0, 2, 3] then?
That's the representation in the pass list, so each element in the path's list is represented as a tuple.
The first element is the list of nodes on that path, and then the second element is the length of that path.
And that's why we had that lambda function in the first place, right?
So if you read the specification here, you have path length, and then you have the nodes that are on that path.
And when we sort the path list-- because we need to make sure that we're looking at the shortest path-- then we use this lambda function in order to get that length for each tuple.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this week the last final topic we were taught-- we're really going to talk about in class is dynamic programming, so who can tell me what the key idea in dynamic programming is?
A way of [INAUDIBLE].
Instead of optimization tools, you can use [UNINTELLIGIBLE]
Sort of.
The key idea is that you don't want to repeat computations that you've already done before.
So you want to find a way to only do everything once.
So it's a form of laziness.
There are two key attributes for a problem though that need to exist for it to be solvable with dynamic programming.
Can someone tell me what those are?
It's optimal [UNINTELLIGIBLE] The local optimization has those
There's overlapping sub-properties.
[LAUGHTER] You had all the right words, just not in the right order.
An optimal substructure.
OK. Can you describe the first one, or any one?
You get to the end solution [UNINTELLIGIBLE]
So when they say overlapping sub-problems, it means that we can take the big problem and we can break it down to a slightly smaller instance of the same problem.
And then we can use the solution to that smaller sub-problem in the solution to our bigger problem.
And then optimal substructure is closely related.
It's saying that if we get an optimal solution for one of those sub-problems, then we can use that optimal solution for the optimal solution of the big problem.
Does that make sense to everyone?
[INAUDIBLE]
What's that?
It's closely related.
And so it's-- yeah, it's closely related but not exactly the same.
So to start off, we're going to do kind of a function that we're all familiar with, is Fibonacci, right.
We've seen this one a billion times before.
And so it's pretty obvious where the overlapping sub-problems are.
The solution to f of N is f of N minus 1, and f of N minus 2, right?
And so if I combine the solutions to these two instances of Fibonacci, I get the solution for the big instance of Fibonacci right?
So on the screen, you have an example function.
You've all seen this before.
And so what I'm just going to do is I'm going to run this Fibonacci function from 0 to 30, or N equals 29 actually.
And we're going to look at how many steps it takes to execute.
When we get to Fibonacci of 29, it takes about 1.6 million steps to compute the value.
And if we take a look at a plot here-- so the y-axis is semi-log, the blue dots represent the actual number of steps that the function took, so the number of times that it had to perform the computation.
And then the blue solid line represents 2 to the N. The red line is the golden ratio to the N, which is the tight bound for this version of Fibonacci.
And then the green line is a plot of a quadratic function.
So nothing really surprising there.
It's pretty inefficient, and we established that before.
So we could make this more efficient with dynamic programming, right?
Let's draw out the call tree for f of 5.
Doing a lot of writing.
OK.
So if you look at this, we see that in a lot of places, we're repeating computations.
So we repeat f of 2 three times.
We also repeat f of 3, this computation, twice.
The idea behind dynamic programming is that instead of repeating these computations over and over again, we're just going to compute them once.
So if we look at the implementation of this recursive Fibonacci, we see that the first recursive call is going down this side of the tree.
And so the upshot is that this-- all the values from f5, f4, f3, f2, f1, f0, they all get computed before any of these branches.
So what we can do is instead of recomputing f2 and f2 here, we can just compute it here.
Save off this value somewhere in some sort of a brain, and do the same thing for f3, and f4, but that's not going to matter.
So when we get to the second recursive call, say when we're calling Fibonacci of 3, we compute f of 2, get f of 1, we come back up to Fibonacci of 4, right?
The next recursive call is going to be to Fibonacci of 2.
But since we've already computed it, and we've saved it off in a brain, we can just look up this value instead of doing all the computation again.
So when you have a very small tree like this, right, it's not a huge benefit.
But if you have like a lot, like f of 100, this is going to make a huge difference.
Is anyone lost?
OK.
So why don't we take a look-- So this is an implementation of Fib that has an extra parameter called memo.
And memo is the brain I'm talking about.
This is where we're going to stash those computed values so we don't have to compute them again.
And when it initially comes in, if memo is none, that is, we haven't passed in a dictionary or if it's like the first call to this function, then it's going to set a key of 0 to 0 and key of 1 to 1.
Those correspond to f of 0 and f of 1, right, actually.
No one pointed that out?
All right.
So, now it's going to-- after it does that initialization, it's going to come down to this if statement, and it's going to check for whether or not N is in memo.
What this is actually asking is, have we seen this number before?
When we've been computing this number, the big Fibonacci number, right?
So if it's down here, and it's looking at f of 2, it's asking, have I seen Fibonacci of 2 when I've done this computation before?
And if it's in this tree, then the answer is yes because it's seen it down here.
If it hasn't seen it though, it's got to do the work.
So it's got to actually do the recursive calls.
And this is when it travels down this left branch of the tree.
Does that makes sense?
And all we're going to do is store it off here.
And then at the end, we just return whatever's in our memory.
So why don't we take a look at how this runs.
So remember, this is 1.6 million with the old implementation, and now it's only 57.
And in fact, it's linear and exactly 2 to the N minus 1, or 2 times N minus 1.
Everyone follow that?
Any questions?
There's not too much we could do
No.
It saves a lot of work for you.
Because as soon as it-- so let's say that it's computed f of 4, this entire sub-tree here.
As soon as it sees f of 3, it's going to look in it's brain, and you've already seen f of 3, and it says, I've already seen this, so I don't need to do all this computation here.
I don't need to do all these recursive calls.
I just have to return whatever's in the dictionary.
That's where your savings come in.
So let's take a look at another example.
The point is, you can have some really huge savings if you write your code right.
So let's look at a different problem.
Let's say that I have a robot.
And this robot is positioned on the grid.
Let's say that it has N rows, and N columns, or M columns.
Your robot is starting out here, and it wants to get here.
Your robot though is very stupid, and it can go only down and to the right.
The question is, how many unique paths are there from the top left square to the bottom right square, given those constraints?
If you try and do this analytically, you'll probably hurt yourself.
The easier way to do it, or at least I think so, is to realize that in order to get to g, there's only two places that it can come from.
It can come from here, it can come from here, right.
So the total number of unique paths coming into g are the total number of unique paths coming into this guy, and the total number of unique paths coming into this guy.
So there's your overlapping sub-problems, right, and also your optimal substructure.
If I can figure out these numbers, then I can figure out this number, right.
So-- and then these guys, the same condition applies.
If I know these two numbers, then I can figure out this number.
If I know these two numbers, then I can figure out this number.
So one implementation-- well, one other thing.
If I get down to this case where I have a 1 by M grid, how many different ways are there to get from here to here?
One, right?
And then same thing for M by N right?
So the first crack at this, here's a recursive function.
All we're going to do is, if we only have 1 row or 1 column, we return 1.
Otherwise, we're going to look for the number of robot paths in an N minus 1 by M matrix and then the number of paths in an N by M minus 1 matrix.
So let's take a look.
We're going to do this on a 14 by 14 grid.
And this will take a few seconds.
OK.
So there's a lot.
10 million unique paths, and it took about 20 million steps to figure it out.
So we're going to pull the same trick for this problem that we did for Fibonacci.
We're going to memorize, or memoize, the different paths or the different solutions, except our key is going to be a little different.
It's just going to be N and M. So again, if we have-- well, again, if N and M is not memo, then we need to compute it.
If either is 1, then we remember it as 1.
And if they're both greater than 1, then we're going to look at the number of paths in N minus 1 by M, and N times M minus 1.
And then we also know that the solutions are symmetric, right.
So it's going to be the same for an N by M and an M by N. And just return the solution.
So let's try this out.
We get the same answer, but it only takes 104 steps to do it.
So it's a pretty huge savings there.
So the other-- this is an example of a top-down dynamic programming solution.
We looked at the big problem, and we broke it down into two smaller problems.
We looked at those two smaller sub-problems, we broke them down into two smaller sub-problems a piece.
We can also go from the bottom up.
And we might want to do this for a reason that I'll show in a second.
So I think I needed that.
This is going to go from the bottom up.
So instead of starting back here and asking how many ways I can get from these two guys, it's going to start at the beginning and ask how many ways I can get here.
One, right.
And then here let's imagine that we only have a 2 by 2.
It's going to look at, again, the number of ways to get to the square to the left and to the square to the top, and add it here.
Add it here.
So what we're doing here-- what we're doing in this version is we're growing a grid towards a solution.
Does that make sense?
This just sets up a matrix.
The top row is going to be initialized to 1.
First column is going to be 1.
And then for everything else, we're just going to add the row above and the column to the left.
And return whatever's down in this lower right corner.
So it gets the same solution, just in a different direction.
So now you might want to do this because imagine if instead of 14, we had 1,400.
So this should crash.
Python has a maximum recursion depth.
If you have too many recursive columns, it's going to tell you, I can't go that deep, and kick you out.
So I have 1,400 rows and 1,400 columns, that's a lot of recursion.
But if we do it iteratively, where we have no recursion, that's the number of unique paths, which is a pretty sizable number.
And that's the number of columns we made, which for a 1,400 by 1,400 matrix is not too bad, right.
All right.
So that was an easy one.
So now we're going to go to little harder one.
Everyone doing all right?
OK.
This is a counting change problem.
The idea is, let's assume I have a currency with coins of a certain value.
I'm not sure where I got the $0.27 from, but that's apparently the value for the coins in our currency.
So the problem is-- let's say that I have a sum total-- the question is, how many different combinations of coins are there that equal total?
The way to break this down is-- so does everyone understand the problem?
So like if I say I want to give change $0.41 to a customer, then it's like a quarter, a dime, a nickel, and a penny, right?
We can think of the problem in two ways.
Well, we can break the problem down into two sub-problems by first considering what the total is-- what the number of combos would be if I use the largest coin in my set.
We're using at least one of the largest coins.
And then the other is the number of combinations if not using the largest coin.
OK?
So this turned, actually turns, into a nicer sub-problem, which is if I have total minus largest-- so in this case, minus $0.27, what's the number of combinations for this sub-problem?
You follow?
And then this guy can be formulated as the number-- so I still have total number of combinations.
But then I take out the largest coin, so coins is now, instead of 1, instead of having $0.27 as the largest coin, now has $0.25 as the largest coin.
Does that makes sense to people?
And so the solution here to this big problem is the solution to-- or is the sum of the solutions to this problem and In this problem.
Make sense?
All right.
Why don't we take a look at the first version of this problem or this implementation.
So we have three base cases.
So the first base case is-- well, first let me explain the function.
So, total, that's the amount that we want.
Coins-- it's the set of coins in our currency.
And I only have $0.05, $0.10, $0.25, and $0.27, so, oh well.
The first thing we do is check to see if total is 0.
That means that we're trying to figure out what combination of coins is equal to 0.
And of course, there's only one combination of coins.
There are no coins.
So that actually is 1.
This case-- this is if we're trying a particular coin that's a little too large to fit into our total.
So we wind up going below 0.
That means we can't use that coin for this particular total, right.
So there are no combinations that work for this.
And then this last case is, if we're no longer using-- if we have no more coins, and we still have stuff that we need to make up, like we still have some value in total, then that means that this particular combination that we're trying out also doesn't work, so we return 0.
So that's this case, where we've taken out everything, so our set of coins is the empty set, is nothing.
Does that make sense?
So our first recursive call is looking for the number of ways without the last coin.
That's this case.
And all we're doing is we're passing in the total we got before, and all the coins except for the largest one, so the last one.
So we're stricken one off for each recursive call.
And then the second case here is if we do use at least one of the largest coins.
In this case, we're just going to subtract that off the total.
And we're going to pass in coins as is.
Make sense?
And then we just return the sum.
OK.
So not too bad.
Everyone follow that code?
All right.
So this is without doing any memoization, it's just using recursion, breaking it down into two sub-problems and figuring out a solution.
But now we're going to use dynamic programming to optimize this a little bit.
So in this case, we have our little memoization dictionary-- you notice a pattern here?
Yeah?
Do you have an infinite amount to supply those?
Yes.
So you're assuming that you have an infinite number of $0.27 cent pieces, $0.25 cent pieces, et cetera
[UNINTELLIGIBLE] just add 1's [UNINTELLIGIBLE]
And it's possible to get below 0.
So let's see.
This is all the same, right?
No surprises there.
But then what we're going to do is we're going to memoize on the total that we're trying to find and the largest value-- largest currency piece that we have.
And if it's not in our dictionary, then we're going to compute it.
And then once we get the solution, we're going to memoize it and return it.
So let's see how this runs.
It could've been by 4.
That make sense to everyone?
No questions?
Wow.
So now let's try a different formulation.
This one's a little tricky.
Let's imagine that I have a grid.
And down the rows I have numbers up to total.
OK, starting from 0.
Across the columns, I have my different currency pieces.
So the first two implementations we're going top down, now we're going bottom up.
The way you read this is, if I have a total of 0, this is a number and my largest currency piece is $0.05, or $0.10, or $0.25, or $0.27, it's the number of coin combinations I have that will equal this total, right.
So in this first row, that's my base case, right.
And then down here, we know this is all going to be 0, because if I have no coin pieces-- So now, the bottom-up way is, we're going to fill in this table here.
So we're just going to iterate through all the totals that are possible.
And for each total, we're going to iterate through all the largest coin denominations.
Right?
And then we're just going to do two checks.
We're going to look at the current total we're looking at, so let's say that we're on row one, so our total is 1.
And we're going to subtract the largest denomination of currency that we're looking at.
We start out here.
So we're going to subtract 5 from total of 1.
And that's less than 0.
And when it's less than 0, all we're going to do is carry the number of combinations from the current total to the next smallest largest denomination.
So if we're at 5 and we know we're less than 0, that means we're going to carry this value over.
And we're now going to do the same thing for all these values.
Now let's get to an interesting case.
So now we're looking at a total of 5, OK?
When our largest coin is $0.05 and we subtract it from a total of 5, that's not less than 0, right?
So that means we're going to go into the second branch of f statement.
So again, the same thing.
We're going to look at the number of ways with the largest coin.
And to do that, we're going to use our table.
We're going to look at the current total, which is 5, minus the largest coin that we're looking at, which is $0.05.
So that's going to index us into this row because 5 minus 5 is 0.
And then we're going to look at the current coin, which is $0.05.
So that's going to be 1, right?
Well, I skipped a step.
So we're going to get this value, right, so this is the number of ways with the largest coin.
And then we're going to look at the number of ways without the last coin, so if this is the largest coin that we're looking at, then without it, this is all we have.
So that's going to be 0.
So 0 plus 1, that's 1.
And then for 10-- so again, this becomes 0.
So the next interesting one is going to be 10.
Is everyone following so far what we're doing?
This would be 0.
This is going to be 1 plus 0.
And this is going to be 1 plus 1.
Making sense?
Make an error?
No, OK. And then this is going to be 0 Everyone get the idea?
So we're just filling in this table.
What's that?
[INAUDIBLE]
Here?
There's only one way to make 10 plus 5
Because-- the smallest-- so if I had 1 here, like so I was using a one piece, then this would be 2.
But because we only have 5, there's only one way to make 10.
Two 5's.
Make sense?
[UNINTELLIGIBLE] because you can either use one 10 or two 5's
Yes, exactly
So if we're using the table that as-- so you have $0.05 as the largest coin, so there's only one way of doing it using $0.05 as the largest coin?
How come that 1 doesn't go all the way across the row?
Because if you have $0.25 as the largest coin you could use $0.05's
You know what, you're right.
Made an error.
Actually, that would be 0
[INAUDIBLE]
What's that?
Yeah, but-- what can I say?
It can be confusing.
To prove that this crazy thing works, let's run it.
So let's expand this.
Actually, I'm going to use this.
So all three versions-- what, did I make a mistake?
No, it's just how you expanded the window
Oh, you like that?
That's what we-- well, like Tracy's quote is always really, really small.
But
No worries
It's a program called Divvy, if you're interested.
I think it's like $10 on the Apps store or something, or maybe it's $5.00.
It just allows you to do those little grid things, so if you have lots of windows and stuff-- and if you need to increase your font size, that's a little bit more involved with Python.
Anyway, so they all give the same output except for the number of steps that they take, so the initial one took 855, the one with the memoization took 209 steps, and the one without memoization but from bottom up took only 337 steps, so-- it's just-- it's three different ways of attacking the same problem.
And objectively, the last two are better than the first one.
We might think of cases where we would want to use say the table-based method over their recursion method.
Like if we had like lots of combinations to explore or some kind of situation where you got into a really deep recursion that was too deep for Python to handle and we got kicked out like we did for the number of robot paths.
So that would be kind of like a criteria for which algorithm you would choose over the other.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we'll start off by going over the Quiz Two practice questions.
So, (1.1).
In Python a subclass could override methods of its superclass.
What say you all?
True?
All right.
There are some repeat people from last night, so they know these already.
Standard deviation, coefficient of variation are different names for the same thing.
True or false?
False
False.
All right.
(1.3.)
Can you go over what the coefficient of variation is?
So, coefficient of variation.
It is-- I have to write big for the camera.
Is that right?
People who haven't asked a question.
It's the other way around, right?
Coefficient of variation is the standard deviation divided by the mean.
What does this tell you?
Like, what sort of information does the coefficient of variation tell you?
Anyone?
So, it's another way of telling if your distribution is very widely spread.
So, what it's getting at is-- it's trying to help you compare your standard deviation against your mean.
Because, let's say that I have a distribution and its mean is all the way out at 1.
Let's pick a very large number.
So it's got a really big mean, but say its standard deviation is-- I don't know-- 100.
Would you say that that's widely spread out?
It's pretty tight.
If you're taking-- if this were like, I don't know, an observatory taking measurements of deep space and you've got a fluctuation of 100 for each standard deviation, you'd probably say that was pretty good.
And you wouldn't be too concerned about it, because you've got a planet out there that's billions of light years away or something.
So on the other hand, let's say that I have another measurement, and its mean is somewhere around, say, 50.
And its standard deviation is 100.
So it's a lot fatter.
You would say that this has a much wider spread, right?
So it's a way of relating the standard deviation to the mean.
And, rule of thumb is if this is less than or equal to 1, we're going to say that it's not too widespread, there's not a lot of variation.
That's a standard that Professor Guttag has.
Depending on your purposes maybe that's too large or whatever, so but that's what coefficient of variation is.
Everyone clear on it?
What's the difference between variance and standard deviation?
The difference between variance and standard deviation?
Standard deviation is the square root of variance.
All right.
Unit testing is useful for debugging.
What does the audience say?
True
True
OK. You know what unit testing is, can you define it?
Well, unit testing is when you use other code to test your code
Right.
So say you've broken your code up into nice little functions like a good programmer.
And you want to make sure that those functions are giving you the right values.
What you're doing with unit testing is, you're writing a bunch of other code.
Or a little bit of code.
Sometimes a lot, it depends on how thorough you want to be in your testing.
And what you have is known input and known output.
And you're going to feed your input to the functions you're testing, and you're going to make sure that they give you the output that you're expecting.
And if they don't give you the output that you're expecting, that indicates that there's probably a bug.
Which also kind of supports the idea that unit testing is good for debugging.
And if you are very conscientious about your code and doing unit testing, you're going to run your unit tests every time you either run your code or make a change.
Because you might have inadvertently introduced a bug into the code that you're modifying.
So it's a way of verifying, it's a way of detecting bugs, and also verifying that any changes you make in the process of writing more functions for your program or modifying functionality in your program don't introduce other bugs.
So in Python, functions cannot be used as actual parameters.
False.
Why?
[INAUDIBLE]
Well, you could just rephrase the question.
But a better answer is that functions are first class citizens.
So they are objects in and of themselves, and so you can pass them around to a function and functions can make use of them.
So we might see an example of that later on if you want.
All right.
1.5.
Increasing the size of a hash table typically increases the amount of time needed to locate a variable or value in the table.
False.
Why?
It is Constant type
Right.
The hashing function is generally constant.
OK.
So, what does this code print?
[INAUDIBLE]
Six?
11
It's 11
[INAUDIBLE]
So what is the string x actually?
Binary?
It's the binary representation of an integer.
And all this code is doing is, it's just converting it to a decimal.
And because I came prepared tonight-- sorta kind of-- I have the code here.
And we can verify it.
Any burning questions on that problem?
Moving on.
What is the expected value of g1?
Anyone?
Well, look at what the code's doing.
Did someone--
What's "gauze," that function?
We haven't used that one
Gauss is--
Oh, Gauss
--a method on the random object that draws a random value from a Gaussian distribution that has a mean at-- well, in this case, mean-- and a standard deviation of s-t-d-dev, whatever you put in there.
So maybe for this example, if I wanted to draw a random value from this distribution, which I said had a mean of 50 and a standard deviation of 100, I would say random.Gauss 50, 100.
So, in answering the question (3.1), what do people think?
What about (3.2)?
100,000.
So they're both the same.
But they both have a different standard deviation.
So what this is getting at is, it's getting you to look at this.
So this code is straight from the question.
And all I'm going to do is, I'm going to plot a histogram of what this function produces.
And also I'm going to run it a number of times and verify that I am getting 100,000 in general.
Python.
There we go.
So it's not exactly 100,000, but it's close enough.
I mean, we're not going to be too upset.
So this picture here shows the actual values that were drawn when I repeated the experiment a number of times.
So I don't know if you can see on the screen but there's this big, really thin, vertical line here.
Those are the results from the Gaussian with a 0 standard deviation.
So everything was 100,000 exactly.
And then these red blocks here, this is with the standard deviation of 20.
Any questions on this?
Yeah?
What's the difference between a normal distribution and a Gaussian distribution?
The name.
The question was, what's the difference between a normal distribution and a Gaussian distribution.
And it's just, it's a different name
[INAUDIBLE]
How I came across it?
Yeah.
How are we supposed to see that in this problem, [UNINTELLIGIBLE]?
Well, if you look at the plots, which I apparently killed-- so let me pull them up again-- All right.
So the thing that you want to know about Gaussian distributions is that, at the mean, is the value that's the most probable.
So its peak is at the mean.
And so if you're taking a 1000 numbers from this distribution, then if you know which number you're expecting to see with the highest probability-- I'm not sure if this sentence is grammatical any more--
Yeah.
So really standard deviation doesn't really have anything to do with it
What it's going to affect is, it's going to tell you that-- like, if you have a standard deviation of 10 versus 20, it's just going to tell you what the spread of the curve is.
But it's not going to change the expected value.
So, why don't I actually demonstrate that?
Because I'm a fan of visuals and not necessarily good at English tonight.
All right.
So all I did in the code was, I just changed the standard deviation of the first distribution to be 10 versus 20.
And what you can see from the plots are that the value with the highest probability is still centered at 100,000.
But the first plot, as opposed to the first time we ran this with a 0 standard deviation, is a little bit more spread out.
And the plot with 20, for the standard deviation, is even more spread out, than it.
But they both have the highest probability at 100,000 or centered at 100,000
So it's like, if you have a mean for g1 of 100, or g2 of 200, then you would expect 200,000 for g2?
Yeah.
So if instead you had a mean of, say, 200 for the second distribution, versus 100, you'd expect them in two completely separate places.
And you'd get an expected value that's 100,000 versus 200,000.
So here close to 100,000, close to 200,000 and then see what my distribution is.
All right.
We good on this?
So moving on.
Question (4).
What is the probability of the final value of num6 being 0?
9/10 to the 10
9 over 10, the whole thing to the 10th power, right?
OK.
So does anyone not see why that's the case?
So I'm going to explain that anyway.
So what this code is doing is, for 10 times its drawing an integer from 0 to 9.
A random integer.
It's a uniform distribution.
And it's counting the number of times that the integer drawn is 6.
And it's incrementing a counter.
So the question can be rephrased as, what's the probability that no 6's were drawn 10 times out of a pool of 10 integers?
And the way you can figure it out is on one pool where, say I had a 10 sided dice.
On one roll, what's the probability of not getting a 6?
It's going to be 9 over 10, right?
And if you do that 10 times.
9 over 10 times 9 over 10 times 9 over 10, that gives you 9 over 10 to the 10th power.
And because this is a code class, we have a demonstration.
So here's the answer that we hand-jammed or that we just discussed.
And now here is a function count of 6.
And it's the code that you see in problem (4).
And all I'm going to do is, I'm going to run a number of trials.
And I'm going to count the number of trials that come up 0, and increment a count numzero's.
And then I'm going to say, this is what I think the probability of getting a 0 is.
And then we'll compare the two numbers.
OK so that's the probability that we count 9/10 to the 10th power.
And then this is the estimated probability that we got from running this function 100,000 times.
So we good on this problem?
Instead of solving it the way that you guys solved it, can you solve it by saying that the problem's also-- that it's 1 minus the probability that 6 will always be picked?
That you will see at least one 6?
1 minus the probability that you'll see at least one 6?
No, never mind
You can solve it in different ways.
It's just this way is-- I think, probably, this way of thinking about it is probably more intuitive
When would you want to solve this kind of problem with [INAUDIBLE], instead of when it says it's not something, to use 1 minus the probability?
That it is [UNINTELLIGIBLE]
Well, my answer would be when it makes sense to you.
Some problems are easier viewed in the negative light, which is what you're saying as 1 minus whatever.
And some problems are better solved in a straightforward fashion.
So it's whatever tool gets the job done.
I don't have a general rule of thumb for you.
Maybe there's a course 18 person who does
So this problem, would we possibly be asked, what is the probability that it'll end up being 1?
Or does it necessarily matter which of those 10 came up 6?
And how would you show that, if we were going to be asked that question?
Well.
One, I haven't seen the quiz.
So I can't give you a definite.
But you could be asked, instead of using 6, maybe say, 1.
Is that going to really change anything?
No.
I guess I meant that one of those 10 outcomes ends up being a 6
Right
I can do it if, they say the fifth trial is a 6.
But just 1 of those 10 trials being a 6, I don't know how to write properly?
So what's the probability that you get 1 of the trials as exactly 6?
Or exactly 1 of the trials comes up as 6?
Yeah
Well, one way of thinking of it is-- so we calculated 9/10.
That's the probability that 6 doesn't come up, right?
So what's the probability that 6 will come up?
1/10.
So instead of doing 9/10 to the 10th power, it would be 9/10 to the 9th power times 1/10, right?
OK, that'll account for.
Thanks
And because I'm never very comfortable with this stuff, why don't we do a simulation?
So this is what we're saying.
Exactly one 6 comes up.
And I'm going to set this to 1
Isn't that saying that it's less likely that one 6 comes up than zero 6s come up?
Is it?
That's true?
Yeah.
Because you're looking for exactly-- you're looking for a sequence of 10 events where exactly 1 of those of events is a 6.
Which seems more unlikely than, say, getting everything that's not a 6.
So I want to verify this
Why is it 9/10 to the 9th [INAUDIBLE]
Because we're asking, if I roll this 10-sided dice 10 times, what's the probability that exactly one of those rolls is going to come up with 6?
And so what we're asking is, what is the probability that I roll the dice 9 times and it doesn't come up with 6?
And then what's the probability that on the remaining roll, that it comes up as 6?
Which is 1/10.
So let's see if I'm in the ballpark.
Did I make a mistake?
[INAUDIBLE] where if there's ten different ways-- so it would be like--
Oh, yeah.
So yeah, you're right.
That's why I don't like that stuff.
So yeah, it's a little bit more complicated than that
You only counted one case of getting--
That's right.
So in this, I'm only counting one case.
There's multiple ways that exactly one dice could come up 6.
So the answer is more complicated
So would you just multiply by 10?
Because there's 10 different places?
Yeah.
So actually, that's what we got, so-- we'll just do this.
Well, of course that's going to come up.
But the simulation gives us the same answer as what we've just come up with discussing.
OK, problem solved.
Did that work?
Yeah
Probability can be tricky
So essentially, you could just take the probability that 9 of the rolls aren't 6, and leave it at that?
Because you multiply it by 10 [INAUDIBLE]
Yeah.
You can also do it that way.
So again, there's multiple ways to think about the problem.
So she was saying, instead of multiplying by 10, we just say that-- we take the probability that we have 9 rolls that aren't 6.
And that works too.
That's what she just pointed out.
So these parts cancel each other out.
It's basically-- it is the same thing as this.
Yay.
All right.
Are we good on this question?
All right.
Problem (5).
It was matching plots.
So you have three functions here.
You have a polynomial, or two polynomials.
One's a cubic, one's a quintic.
And one's an exponential.
There are a couple ways that you could go about solving this one.
One is to plug values in and kind of match them up with the plots.
Another is to look at the axis of the plots.
So this exponential function here, if we were to have it on two linear axes, the x-axis and the y-axis for both linear, you'd see the expected curve upward sharply.
But if you notice, a couple of the plots are in a semi-log y-axis.
So if you put an exponential with a linear x-axis and a logarithmic y-axis, what would you expect the plot to look like?
It would be a straight line, right.
So what that tells us is that if we look at the bottom plot, we have something with a straight line.
So, bottom plot is the third function.
So, exponential.
And then for the remaining two functions, there are a couple methods that you would use, or you could use.
One is to plug values in and see where the points wind up and compare them with the plots.
Another way to think about it is that one of the plots is on a semi-log and another plot is on just both linear axes.
And if you plot a polynomial that's not exponential on a logarithmic y-axis, what would you expect the curve to look like?
It should look somewhat similar to a logarithm.
So with that in mind, you could then start plugging points in.
And what you would come up with is that the second function is this top plot here.
And the middle plot here is the first function.
Is there anyone who's lost on that?
Uh-oh
Wait, why?
So you have two quadratic functions, right?
One and two are quadratic?
They're polynomials.
They're not exponential
OK. And then the third one's exponential.
And the reason why the third one's linear is because when you plot an exponential--
If you plot an exponential with a linear x-axis and a logarithmic y-axis, it should be a straight line.
Right?
Or a straightish line
Why?
It's the way the math works out
[INAUDIBLE]
Any other questions?
All right, we'll move onto the next one.
What does the following code print?
16.
And then square root of [INAUDIBLE]
Close.
So I don't actually have the answer.
Well, not written down.
But I have the code, so I'm just going to run the code.
So you're close.
Shall we spend some time stepping through this code, understanding it?
All right.
So, what question (6) does is, it creates a class hierarchy.
And at the top of it, it has the shape class.
Which has just two methods, double underbar EQ and double underbar GE.
That was just defined as equals operator and the greater than or equal operator for shapes.
In terms of their area.
So what this is saying is that two shapes are equal if their areas are equal, and a shape S1 is going to be larger than another shape if its area is larger than the other shape.
So if I had-- well let's go down and we'll get to that.
Square, subclass of shape.
It is what it says it is.
It's a square.
And it has a side.
And its area is what you'd expect it to be, the side squared.
And circle is just defined-- it just has a radius, and its area is what you would expect it to be.
There is a function here, f. And it takes a parameter, l. And I -- if you look at the code here and you kind of walk through it -- l would have to be either a list or a tuple of elements.
Doesn't have to necessarily be shapes.
But this first line is just going to return none if the list is empty.
And then these next lines are going to, one it's going to set the first element equal to x, and then iterate through each element in l. And then if the element it's looking at is larger than x it's going to set x to the element.
So what this function is doing is finding the largest element in the list.
Everyone follow that?
So, this should be pretty straightforward.
As should this.
This part my trip some people up.
So, this obviously creates an empty list.
What this does is, it creates a dictionary of class names.
And what it's saying is that for a key of 0, the class is circle.
And for a key of one the class is square.
Everyone see that?
This loop here iterates for 10 times.
And it appends 10 elements.
And then each of those elements are defined by this shapes dictionary.
On each iteration, it's going to take the modulus of i, the index, or the number.
And if it's a 0, it's going to construct whatever class the dictionary of shapes returns.
So if it's an even number, it's going to be circle.
And if it's an odd number is going to be a square.
Everyone follow that?
Does no-one follow that?
OK. Or does anyone not follow that?
Got my logic messed up.
All right.
So now what we have here is a list of shapes.
And then all this does is it's going to print which shape.
Is it going to be the fourth or the fifth shape?
Fifth, right?
Because we've started indexing it at 0.
So this number is obviously even.
So this should be a circle shape, or a circle class.
And what should its radius be?
[INAUDIBLE]
Right.
And then this line here is just going call the function f, which is going to find the larger shape.
Which is determined by its area.
And in this particular instance, it's going to be a circle with radius eight.
Once you figure out what f does, this is actually not such a hard problem.
So the key to understanding this one is figuring out what f is actually doing.
And you'll probably see a problem or two on the exam that's written like this, where the function has been obfuscated.
So it doesn't have nice little variable names.
If we were writing this for real, it would be something like that
Wait, what was the radius on L?
It was--
Radius on L?
Or not radius.
Like, L4 was circled.
And was that all it was, it was just a circle?
With radius 4
With radius 4?
OK, that's what I thought
All right
How many of those special underbar functions have we seen?
Have we seen anything besides edit string and [INAUDIBLE]?
So, how many of the double underbar functions have you seen?
That's a good question.
You've seen a lot, right?
The ones that I would be familiar with definitely INIT definitely EQ, GE and definitely STR.
You might want to be familiar with this, LE.
So what is LE?
What do you think that does?
Less than or equal to.
Less than.
So it's this the opposite of GE, right?
I think there's also an LT, so less than.
And GT.
It's fairly logical.
And historically you don't need to have these memorized.
I've never seen a question where you've actually had to produce code that uses double underbar on the quiz.
You just have to understand what it's doing if you can see it.
Like, in a code reading exercise.
And the conventions are fairly straightforward to understand.
Can I move on?
Can you scroll down a little bit?
Where it says print L4, or I guess print FL.
Does it automatically just print the string function?
Yes.
So you're asking why I don't have this
Right
So in this flavor of Python, and if you don't have an explicit conversion to string, Python will look for the underbar STR function if it exists.
If it doesn't exist then you get that weird object at blah, blah, blah
Right
Any other questions?
What list are you reading up on top?
Are you confused because it's the same name?
Maybe.
I don't know.
So the third thing that [INAUDIBLE].
I understand the second thing, circle with radius 4.
But how do you get that circle of radius 8?
Well, the way we get that is, f of l-- f is the function, right?
Right, and it produces the largest element of the list.
And the list, L, contains 0 to 9?
Yes.
So we could actually print out what L contains
Why isn't it square?
Isn't 9 [INAUDIBLE]?
Why don't we do it?
All right.
So all it's going to do is, it's going to print out all the shapes in the list and their areas.
I've got stuff at the top.
So this is the first shape.
And then, as we move down the list, we have square with side 1, circle with radius 2, 3, et cetera, et cetera, et cetera.
And then, this is the area of the circle with radius 8 versus the area of a square with side 9.
So, and it depends on-- the reason why it works this way is that this is greater than or equal to operator is defined in terms of the area method of the shapes.
Does that--
So wait.
What is this doing then?
L is a list of areas?
No, L is a list of shapes.
It's a list of subclasses of shapes
So you're feeding f this list of shapes.
Oh, and to compare if S and L [?
into that list ?]
is greater.
And for equal to x, do you have to use the greater than or equal to--
Operator, right
--method, the class of shapes which compares areas?
Mm-hm.
Now, I might regret this.
I think this'll work.
Yep.
It works.
So, you see this weirdness I have here?
It's showing pretty explicitly that I'm calling this a GE operator.
It's just so-- this is really just shorthand for that.
Syntactic sugar.
It's easier to look at.
Easier to understand.
And to even drive that point further home-- whenever that GE, whenever that comparison is done, it's going to print out I'm here.
So it is actually getting called and not just blowing smoke.
Have we beaten this problem to death?
Last problem.
So, I'm actually not a huge fan of this problem.
But the way I think of it is, it's the number guessing game, right?
So I say to you, I'm thinking of a number between 0 and 100.
Or in this case, maxVal.
And your job is to guess, as quickly as possible, what number I'm thinking of.
The only thing I can tell you is that you give me a guess and I'll tell you if it's higher or lower, or if it is the answer.
So if I'm thinking of a number between 0 and 100, what's the strategy?
[UNINTELLIGIBLE]
Binary search
Binary search, yeah
Binary search, right?
So you're going to start in the middle.
You're going to start at 50, and you're going to say-- and I'm going to say, well, no, the number I'm thinking of is lower.
So now you're going to guess between 0 and 50, 25.
And I'm going to say it's higher.
Now you're going to guess between 25 and 50.
I don't know, I didn't actually think of a number.
But you get the idea, right?
So this problem is asking you to implement a binary search.
And comp.guess here is functioning as me in this problem.
It knows a number and you write a program that's going to guess it, which is find.number.
So, there are multiple ways to do this problem.
Here is my solution.
This is just an implementation of comp guess.
At the beginning, set a maximum value of 100,000.
And I'm going to guess the magic number, or I'm going to choose a random number from between-- yeah?
When you say binary search, is that the same thing as bisection search?
It's the same idea.
But the way that I differentiate binary search from bisection search is that you do a binary search on a finite list of elements, of sorted elements.
So I might have a list of names in alphabetical order, or a list of numbers in ascending order.
And I would do a binary search on that list.
With bisection search, it's not necessarily a finite list of numbers or elements that I'm looking at.
It's, I know that the answer exists within a lower and an upper bound, and I'm going to divide that search space in half successively or iteratively to find that actual value.
But it's not necessarily a finite list of elements between those bounds.
That's good?
Yes
So this is one implementation.
You can do it any number of ways.
This numSteps thing, you wouldn't need to have.
It's just for me to illustrate some stuff.
So I'm going to start at 0, and I'm going to start at maxVal.
And then I'm going to keep iterating.
While I don't have, or while I haven't reached the answer, basically.
So, I'm going to make my guess in the middle.
And then if comp.guess tells me that it's higher, or that my guess is too low, basically, then I'm going to set my upper-- sorry, if my guess is too high, then I'm going to set my upper bound to my guess.
And then I'm going to search in the middle from there.
And then if it tells me that my guess is too high, then I'm going to search in the upper region.
Otherwise, that means I've made the correct guess and I'm done.
So, very simple.
The key part of this problem that tells you that you should use binary search, though, is that it stipulates you need to have a log maxVal algorithm.
And binary search is a log N algorithm.
I mean, you could do find number in a linear search, but it would be fairly inefficient.
All right?
[INAUDIBLE] just guess one number, then go--
Yeah, start at 0--
[UNINTELLIGIBLE] guess 1, you guess 2, then guess 3
Yep.
So you just keep incrementing by one
If it didn't specify, oh, it should run log [UNINTELLIGIBLE], would it be OK if we had used the word [UNINTELLIGIBLE]?
If it didn't specify the complexity, then yeah, you could use whatever search you deemed appropriate.
But because it said it needs to be log N --
Yeah, yeah
[INAUDIBLE]
What's that?
Are you going to [UNINTELLIGIBLE]?
Not in this particular case.
Because we're dealing with integers anyway.
So all these operations are going to result in integer results
Are you going to ever be able to get to the highest number?
So if it were, like, 99,000?
Is that what you're asking?
If the number were maxVal
Well, we'll see.
I might have it wrong in my implementation, always possible.
If I do, I'd probably take one point off me.
Yeah, so I have a bug.
So I am actually--
When it divides, it rounds it down [INAUDIBLE]
Yeah.
So I have a bug.
And I'm not going to try and fix it right now, but yeah.
If I were grading this and this were my solution, I'd dock me a point or two
What's your error?
My error is that because of this the way that I get my guess here, I'm never going to get to max val.
If the computer chose maxVal as its guess, then I'd never get to that.
So yeah, you caught me
A friend of mine looked at this and was like, oh, you should define another function to solve it.
Is that kind of thing actually allowed on the exam at all, or no?
Well, why did your friend think you need to find another function?
I'm not exactly sure.
It was supposed to be [UNINTELLIGIBLE]
Did they say define a recursive function?
I think that's what I was trying to do
So I'm solving it iteratively.
You really can write a recursive function.
And in this case, because we've given you the specification for find number, you'd probably write something like find number helper, and have whatever additional parameters you needed in order to support the recursion
So you can do that or no?
Yeah.
We'd be fine with that.
The point is that you write a function that runs in log maxVal time
Because at the top it says that the magic number is in range maxVal, doesn't that mean that the number would not ever be maxVal?
Let me see.
Actually, yeah.
Because it would be 0 to 999-- or 999,999.
Well, I'm not sure if this actually constitutes a bug or not.
So I'd have to do unit testing.
I don't have any test cases.
Which is, again, back to unit testing.
You have a question or--?
Yeah.
Could you do this by saying your guess equals choose a random number between high and low?
You could.
But it would have-- the analyzing complexity would be problematic, I would think.
Doing it this way, by guessing right in the middle, you know it's going to be log maxVal complexity.
If you were to choose a random number between low and high, I'm not sure what that would do to the complexity.
It would make the analysis a little bit more intricate.
And if this were the actual quiz and you put that as your answer we'd probably take a fair number of points off for it.
There are people who do study algorithms that do that, though.
They use random numbers in order to [UNINTELLIGIBLE] the values.
So we're done with the quiz.
And now we've got a list of topics to go over.
So, let's start at the top.
The first subject that was listed-- so all you got the list of quiz topics for the quiz, I'm assuming?
Did all of you look at it?
The first topic is algorithms.
And I have big-O notation, exhaustive enumeration, guess and check, successive approximation, divide and conquer, binary search, merge sort, hashing, orders of growth, and amortized analysis.
So, first come, first serve.
What do people want to see?
Hashing
Yeah.
Hashing and amortized analysis
Hashing and amortized analysis.
OK.
So why don't I start with hashing, because that's pretty easy.
The idea behind hashing is that you have a function, and it's going to take some input.
And it's going to compute an address of some sort.
And let's imagine that I have a data structure that has a number of buckets arranged one after the other.
What hashing does is, it says I have this input s, or a key, if you will.
And I'm going to compute an address like a number of some sort.
And it's going to, say, wind up here.
And this relates to Python because this is how dictionaries are implemented.
So this s here is going to be the key that you give the dictionary.
And then it'll compute a location for this bucket where it's going to put the value.
So, the idea-- what makes these hashing functions valuable, especially with respect to dictionaries, is that you can check for the presence of a key or a value in constant time.
Because the hashing function is more or less constant.
And once you have an address into the data structure, it's very easy to compute the location where you're going to put something.
So, I'm not sure-- does that kind of help you?
You don't need to know hashing in great detail.
You're not going to need to know the hashing function for like a string, or the hashing function for a tuple or something like that.
The main points that you need to get from hashing is that it computes an address into this data structure.
It's a constant time thing.
And that's about it, actually.
If you have those two concepts down-- well, also, three that that's how dictionaries are implemented-- if you have those in your mind for hashing, you'll be good.
Anyone have any other questions for hashing?
So we never have to like write a hashing function.
It's more like [INAUDIBLE]
This is so that you know what it is, and when you see it in the future, if you see it in the future, you're not totally lost.
You have an idea of what it's doing.
You can define your own hash functions, but we're not going to require that of you in this course.
So that's hashing, basically.
And all you need to know for the quiz.
So for amortized analysis, the idea behind amortized analysis is that you have-- maybe you have a known sequence of operations.
And you're trying to find what the worst case runtime for that sequence of operations is.
And you might have a bunch of operations that are really, really low cost, or low complexity.
And one really expensive one.
And for our purposes, we're trying to find this-- find which of these operations is the dominating factor for the complexity.
So the example I would use is, let's say that I have the task of searching for an element in a big list of elements.
So we know two basic search algorithms for this, right?
We know a linear search, where we go one at a time through each of the elements.
And what's the complexity of that?
[INAUDIBLE]
What's that?
Just the length of [INAUDIBLE]
It's going to be linear in terms of the number of elements in the list.
We also know binary search.
What's the complexity of binary search?
Log
Log N. Now, there's a problem with binary search.
It has a requirement.
Will binary search work on a list that is not sorted?
No.
You require it to be sorted.
So, we've talked about sorting in class.
He covered it during the lecture.
For basic sorting that we've seen, the lower bound, the complexity of the sorting algorithms is N log N. So merge sort will be an N log N algorithm.
So, when-- let's say that we just need to do one search, right?
Do I really want to use binary search?
Why?
[INAUDIBLE]
Because I have to sort it first.
And if I just do-- if I do I have to sort and then search, and this is going to be-- whereas if I do a linear search, I just have search.
And for just a single search, for binary search, this is going to dominate my complexity.
But, let's say that I'm going to do a million searches.
Now, it might behoove me to actually sort this first.
Because I only have to sort the list once.
And then I can do as many searches as I want.
So, in the grand scheme of things, if I'm doing a million searches, then linear search is a really poor choice.
And binary search would be my best option.
And this is about as deep as we want to go with amortized analysis.
It's this idea that there's a certain point at which one algorithm becomes more important than another algorithm, because the composition of its operations kind of determines, it changes as you increase the number of elements you're working with the number of times you're going to do the operation.
That's kind of what you need to understand for amortized analysis.
And it can be used in other types of analyses, but we're not going to get into that.
All right.
What other topics do we want to go over for algorithms?
We've got big-O notation, exhaustive enumeration, divide and conquer.
Successive approximation, merge sort, and orders of growth
Can we go over merge sort?
You want to go over merge sort?
OK.
So-- and today actually, I have my code.
So merge sort is a divide and conquer algorithm.
And the idea is that I am going to take a list of elements.
And I'm going to divide the list in half.
So I'm going to get a left list and a right list, and they're unsorted.
And then I'm going to call merge.sort on them.
And I'm going to keep recursively calling merge sort until I have one element or no elements in the list.
And then I'm going to return the list.
After I do a merge sort on the left and right halves, I'm going to merge them.
And the merge operation is just going to start at the beginning of the left and right halves of the list, and it's going to append the smaller element at the beginning of each of the lists to result until it gets to the end of the lists.
So I'll try a blackboard demonstration in a second.
Or explanation in a second.
So, this before parameter, all this is doing is it's determining the order of the elements.
So it's saying that left should come before right.
It'll return true if left should come before right, and false if right should come before left.
So, intuitively, because I don't think that was a very good explanation, let's say that I have a list of elements.
And they're unsorted.
All right.
So it's an unsorted list of elements.
When merge.sort first starts, it's going to divide this list into left and right halves.
So basically it's going to take the middle here.
And then it's going to call itself, on this left half and then on this right half.
And it's going to keep doing this.
It's going to divide this half into say -- 5, 2, 1.
Now, when it gets to these little single elements, it's going to do something interesting.
It's going to merge these two.
So when it gets down to these single elements, it's just going to return the list as is, it's not going to do anything to them.
And then the code after this, division, is going to merge them.
And the way it merges them is, it's going to start at the beginning of these two lists.
And it's going to say, is this element larger or smaller than this element?
Or is it going to come before this element?
And it's going to construct a merged list.
And then return that as its result.
So this is going to come down.
So now at this point it's popping back up the stack.
It's going to return this.
It's going to merge these two lists.
Starts at the beginning of both lists.
One comes before two.
And then this list is done, so now it only has this list to do.
After each merge step, these lists are going to be sorted.
So now where are we at?
So now let's do this guy.
So this guy gets merged into 6 and 9.
And then this guy, I didn't divide up.
So, bad on me.
So we're going to merge those two single element lists.
Now these get merged.
So these guys are going to get merged.
So, this is going to become, I think my tree got messed up a little bit.
Now we're going to merge these lists.
So we're going to start at the beginning.
1 comes before 3.
First element is 1.
2 comes before 3.
Now we're here in this list.
We're still at the beginning of this list.
3 comes before 5.
5 comes before 7.
And we're here.
6 comes before 7.
Then we have 7.
And then 9 comes before 10.
Now we're done with this list.
So now we have 10, 13.
And at the end of the last merge the list is sorted.
So, I'm not sure how clear that was
What happens if there's [UNINTELLIGIBLE] to begin with?
Like all of your split equally into [UNINTELLIGIBLE]?
What if there's one more element at the top, [UNINTELLIGIBLE]?
Do you know what I mean, like, there'd be one left over?
What do you mean?
So there's an odd number of elements in the list?
So you would just get an uneven split.
So your left sub-list might have 6 elements.
Say you have 11 elements in the list.
Left element would have 6, or left list would have 6 elements and then the right list would have 5
[UNINTELLIGIBLE] then that would split with 3 and 2.
And then the 2 would split to 1 and 1, the 3 would split into 1 and 2, and then the 2 would split again?
And then the 1 would actually make a call, but it would be an empty list.
It would split, but one of them would be an empty list
Oh.
So and then it would just get--
And that's kind of what happened with my tree.
I didn't go all the way down on some of the branches of my tree.
I mean, the main idea behind merge.sort is that you split the list in half, whatever size it this.
And then you merge sort those separate halves.
And merge sort is going to do the same thing with those separate halves.
It's going to split them in half.
And sort those halves until it gets to the smallest case, which is an empty list or just a single element.
And then it's just going to start merging these lists
Right, I was just wondering what happened [UNINTELLIGIBLE]
Yeah, you'd split it into 2 and 1, and then that merge sort would be like 1 and 1.
And then this 1 would split it into a list of 1 and nothing.
That's all.
And then the merge between that's obvious, right?
So in the merge sort lecture notes, at one point, lambda is used.
Can you explain a bit how this works?
Does he actually use a lambda?
Yeah.
Why is it useful?
I guess [UNINTELLIGIBLE]
Ah.
He used lambda.
OK, Python has these functions called lambda functions.
And what they are essentially are like one-liner bits of code that return a value.
So let's see.
I can't believe he used lambda.
I'm trying to think of where I would actually use them.
Like, an example.
And I don't have one off the top of my head.
You're not going to see a lambda on the quiz.
The syntax is not too difficult.
It's basically, so, like lambda x.
And then it's going to return, like, x-squared.
This is going to create a function.
And it can be something like, I assign this lambda function to a variable square.
And I'm going to pull up a separate-- So I'm creating a function square.
But instead of writing something like this, I write it as this.
And
[INAUDIBLE]
What's that?
Do you have to save it before it lets you run it?
Yeah.
That's, again, what I'm doing.
I'm just trying to think of a witty name.
I'm actually stunned that he whipped lambda out in lecture.
So really it's a way of defining these one-liner functions.
Python's lambda functions are kind of broken, at least in the 2.5, 2.6, 2.7, 2.x versions.
3.0, 3.x versions of Python, I think, actually do away with lambda because they were broken.
You find lambda functions in a lot of other programming languages like Lisp or Scheme.
And you're never going to see this on a quiz.
But if you want to learn more about them you can look it up online
While you're running the function, what is the [INAUDIBLE] The same thing?
Oh,
Yeah.
I had it there as an illustration that the functions are-- they're functionally equivalent.
And it actually serves to illustrate another point, that functions are objects and you can sign them and pass them around and stuff.
But lambda functions on their own, you're not going to need to know that piece of linguistic Python, Pythonese.
Bad sentence.
You're not going to need to know it.
Does that work?
OK.
I'm going to have to ask him about that, because I can't believe he used lambda in lecture
In one of the lecture notes, he used something called x-range
x-range
What does that do?
Did he talk about generator objects or iterate?
Or yield?
You haven't seen those?
OK.
So what you're asking is, something like that.
He used something like that.
The difference between range and x-range is that range returns a list of integers, and x-range returns in generator object.
And I know that really meant a whole lot to you.
So let's say that I assign
[INAUDIBLE]
Yeah.
that's what I'm going to show.
So if I do print list of integers, or list of numbers, it's actually going to print the list of numbers.
Someone fall back there?
So it's actually going to print that list.
Now, if I do something like-- pretty sure this is not going to print the...
The reason is that if you look at the type of the object that's returned, it's-- well, x-range doesn't really help much either.
Why does he do this to me?
It's a way of-- so let's say that you have a list of a billion elements, or numbers, or things that you need to iterate over in a for loop.
It's probably not a good idea to bring all of those into memory at once.
So, or to necessarily keep them all around.
You want-- that's where things like x-range come in.
So, let's do this
[INAUDIBLE] for i in x-range 10 [INAUDIBLE]
Yeah.
So if I do this, it's going to be the same as if I were to do this.
It's going to have the same functional effect, but the difference comes in-- let's say that I were to do something like this.
Lot of numbers, right?
I'm hoping Python crashes on this, because this is my whole point.
Yeah, so there's too many items.
But let's see if I do it with x-range.
I should probably comment this out.
I chose a really too large number, didn't I?
I'm going to find a number that has too many list elements, but Python can still feel comfortable constructing an x-range out of...
The point is that there's a certain point, a certain number.
I don't know if it's going to do that comfortably.
So you see this little delay that's occurring?
Actually, this kind of a long delay?
It's actually hanging up on this range command.
Because it's actually constructing this list object with all these integers in it.
So, let's say I do this.
Are you going to need to know the exact difference?
No.
Is there a difference?
Yes.
Am I prepared to explain it exactly, right now?
Not really.
Previous iterations of this class have covered the yield statement in Python, which allows you to create functions that create generator objects that... You did yield?
[INAUDIBLE]
All right, then you should know about this.
So range is going to give you a list of integers.
x-range is going to give you a generator object that uses yields to produce these integers.
So let me see if can rescue my machine.
All right.
So, he covered something like that with yield?
So if I were to, say, do i in my x-range-- back to where I was.
My x-range does the exact same thing that x-range does.
Now, as I said before, if I do a list, or if I do range, that's going to be a list, right?
If I assign this to ... see that?
That's what the yield statement does for you.
It produces that generator and that's how x-range functions.
So, basically it allows you to do some interesting things.
Like, in this case, it's not too interesting because it's just returning one integer after the other.
But let's say that I wanted to do -- def my_squares.
All that's going to do is return these squares of all the numbers between 0 and max N. So if I were to do this, it should print out all the squares-- or the square root of all the numbers between 0 and 9.
Make sense?
So that's what yield gets you.
Those are generator objects.
And that's how x-range works.
That's how it's different than range.
Does that make sense to everyone?
OK.
I don't need to beat that to death any longer?
I was not expecting that question.
OK. What should we go over next?
Is everyone comfortable with orders of growth?
All right.
The next major topic area I have is linguistic issues.
So, this covers stuff like exceptions, polymorphism, classes and objects.
So people want to go over classes?
Objects?
Polymorphism?
What's polymorphism?
Yeah, we'll get to that.
So before my computer crashed-- all right.
So here are some class definitions.
So I have a shape object, and I've defined two methods, area and perimeter.
They currently do nothing except for raise not implemented errors.
And now I have to find a class hierarchy.
I have a rectangle class that inherits from shape.
So shape is a superclass of rectangle.
Rectangle is a subclass of shape.
This also says-- well, let me ask the question, is shape a rectangle?
It's not rhetorical
[INAUDIBLE]
Not necessarily.
Is rectangle a shape?
Yes
OK.
It's an easy question.
It's not a trick question.
In the rectangle class, I have an INIT method.
It just takes the length and the width.
So computing the area is really easy.
And then computing the perimeter is also very easy.
And now I have a string representation which is just rectangle with the length and width.
Ellipse inherits from shape and it has one radius-- or one, the long axis and the short axis.
And computing the area for that is pretty easy.
Computing the perimeter, on the other hand, I had to look it up online.
So if you're interested in the formula I used go take a look at this website.
Again, I have a string method.
And then now we have a square which inherits from rectangle.
Because square is just a special case of a rectangle.
And so I'm going to reuse it.
I'm just going to make both lengths the same.
That make sense to everyone?
And I don't need to override the area and perimeter because they're pretty easy to understand.
Or they're the same.
And then circle.
It just inherits from ellipse, and same thing.
I'm just going to reuse the stuff that I have in ellipse.
All right.
I've defined a bag of shapes here.
Circles, squares, rectangles.
Rectangles.
Lots of stuff.
Now, to answer your question, what polymorphism allows me to do is, it allows me to treat subclasses that share a particular superclass, or a parent class, the same.
So I've defined an area method on the shape class.
And all of my subclasses, rectangle, ellipse, circle and square, inherit from this superclass.
And some of them override area, some of them don't.
It doesn't really matter to me.
What matters to me is that if I have a bag of these shapes, I know that I have an area method.
And I don't need to know the particular or the type of the object that I'm calling the area method on.
Right?
Python is going to take care of that for me.
I can just iterate through this bag of shapes, call the area method, and I can get a sum of the areas.
That's what polymorphism gives to me, is it allows me to treat objects uniformly.
It allows the objects to change their behavior based on the particular type of object that I'm using.
But I don't need to know the particular details.
So it's pretty powerful.
So it's like, if I were to create a racing game and I wanted to drive a car, I could have different models of car.
But my game engine wouldn't need to know to drive method to call for.
Like, a Ford car versus a Chevrolet, all that stuff.
That kind of answer your question?
Yeah
So if there's not an area function that overrides it, what is it going to sum if it just raises a non-implemented error?
Yeah, if you haven't redefined
In this example, I'm using all what are known as concrete classes.
So they're not abstract.
They all have implementations of area.
But if I were to say, for some reason I decided that I wanted to make something amorphous.
This should raise an error, right?
Unless I have invalid syntax.
So when it gets to there, yeah.
Any other questions?
So what's the difference between inheritance and [UNINTELLIGIBLE]?
They're related.
But what inheritance means is that if I say that rectangle inherits from shape, that means it gets all of its methods and attributes.
But it's not necessarily polymorphic in that case.
It's just, it's like saying a rectangle is a shape so it has all these characteristics of a shape and then some.
The polymorphism comes in when I override the area method and then I can treat all shapes, whether they're rectangles, squares, circles, uniformly with the area method.
You might-- I'm trying to think of an example where it would be.
So let's say that I didn't do this in a class or entered a way or using classes.
And so I had a really naive implementation that said, let's represent them as dictionaries.
So I have two really stupid functions here.
And they say they make a circle and they make a square.
And so really all it's doing is representing these as dictionaries.
And the key is side and radius.
Now, let's say that I want to have my list of shapes.
I make a square.
I'm going to make a circle.
Another square.
OK, so now I've got a bag or shapes or a list of shapes.
What's that?
Can you [INAUDIBLE]?
Spending too much time on that one.
All right.
So if I were to take a look at this.
It's going to print out a bunch of dictionaries.
Now, the question is, what if I wanted to compute the areas of these data types?
Using polymorphism, I can just call dot area on each shape.
But if I wanted to do the same thing with this code, I would have to have square area s, return, d, side.
And then def circle, area, c, return, 3.14.
9 times c-- I'm already getting tripped up.
So even this is not so bad, because if I do something like this, I can still get the areas, right?
And I can do this because I know what's in those positions in the list.
So
You forgot to close your parentheses
Oh.
OK.
So it's going to work.
But now, if I want to print out all the areas of all the shapes in this loop here, how am I going to do that?
Because I can't just call square area of s, because what if s is a circle It's going to give me an error, right?
I could get around this by saying, if radius in s, then I know it's a circle, right?
Or, side in s. But you see how the code is getting kind of ugly now?
I have to know details about how these different objects are implemented, how square and circle are implemented in the dictionary.
I have to know what type they are before I compute their area.
And I can do it, but it's kind of ugly.
Whereas if I invest a little bit of time in a class hierarchy, I can do this much more easily.
Much more cleanly.
It's a way of abstracting from the problem.
You don't have to worry about the details of the individual shapes.
Did I kill that?
And beat it into the ground
Just for clarification.
So the point of polymorphism is just like inheritance, essentially, kind of boiled down?
The point of polymorphism is that if you have a class hierarchy of objects, you don't need to know the details of the objects, of what specific type an object is.
As long as you know that in the case of--
So long as there exists a method that you want to use
Right
Where it inherits from that class, you could use it
Yeah.
So in this particular example, shape has an area method.
And I have a bunch of concrete subclasses of shape that have all overridden the area method.
And they do it in different ways.
You compute the area differently for a rectangle versus an ellipse.
I don't have to worry about which particular shape it is.
I just call the area method, and I don't have to do all the checking that I do here.
Where I look at, is this a circle, is this a square.
I don't know, so I'm not going to compute that
How do you avoid the checking?
[INAUDIBLE]
So--
[INAUDIBLE]
See?
Oh, OK, Great.
Yeah
[INAUDIBLE] Anyone lost?
Any other questions?
So that actually touched on a couple of topics.
So we saw exceptions here.
Is there anyone who wants to see anything else on exceptions?
What's the difference between pass and continue?
Pass and continue?
OK.
So the question is what's the difference between pass and continue.
So when you have pass, this is basically a null command.
It's a do nothing command.
I can actually run this code right now.
And it will do absolutely nothing useful.
But it also won't complain.
It's a placeholder.
Sometimes-- so a lot of times you'll see in your code something like this.
Let's x equals 150.
If x mod 2 equals 0, pass else, do something useful.
So we see this a lot in your p-sets.
It's not a slam on you or anything.
But you could simplify this by saying-- but the way that you conceptualize a problem forced you to have this if statement.
And then you didn't have anything useful to put in here, so you had to put a pass in order to make Python not complain.
So, let's be a little less flip and more concrete.
It's getting late, I'm getting punchy.
All right.
So if I were to run this now, it's not going to print anything because it hit the pass command.
Now let's make this on.
So if we've verified that this code works and that I'm not blowing smoke.
Now, let's take out the pass.
Python's going to complain.
Why?
Because after this if line, here it's expecting a block of code.
But there's nothing there.
The pass keyword give you a do nothing command.
It says, I don't really have anything to say here, but you want me to put something here so I'm going to put something here.
But it doesn't do anything.
So that's pass.
so It's a placeholder, is the basic answer.
Now, continue is different.
Because continue actually has a function that is useful.
Let's say that I'm not too familiar with the range command, and I don't know that I can specify a step.
And I want to print out all the odd integers.
And I also am going to restrict myself and say that I can't use an else or something of that nature.
So I'm going to handicap myself.
This x-range -- is x-range cool?
So.
I only want to print out odd numbers.
Or we can do it like this.
Either/or, it doesn't matter.
So, comment this.
What continue does is it, if you're within a loop, as we are here with this for loop, tells Python if it sees a continue statement, do not continue beyond this point within the block of the loop.
Go instead and perform the next iteration of this loop.
Go to the top of this loop.
So, what this code should do, I'm hoping, in case I don't-- I might have a bug-- it should print out the odd integers.
So what's happening is that when the integer is even, it's hitting this continue statement.
And this is telling Python, go back up to the top of this for loop and do the next iteration
If you were to have [UNINTELLIGIBLE]?
So if I just had pass?
It'll print all the integers.
That work?
Anyone confused?
You could also have your code if not that [UNINTELLIGIBLE]
Yeah, but [INAUDIBLE].
There are some schools of thought that say that continue is something that disturbs the flow of code and should be avoided.
I don't subscribe to that.
I think that if it makes your code clearer, then use it.
Which is to say that, yeah, you can rewrite a lot of your code to not use continue.
OK, do we need to go over anything with linguistic issues?
Are we good on classes?
Can we go over exceptions, please?
So there is a question on exceptions.
So exceptions are another way of flow control.
And if I can pull up my source file, I have an example of handling exceptions.
There we go.
Exceptions give you a way of handling certain errors that occur.
So, the easiest one to come up with is division by 0.
So normally, if you would do a division by 0 -- so I'm going trying to divide 10 by 0.
The universe won't end, but Python will be very unhappy.
In programs that are more complex that the ones that we've been writing, division by 0 happens but you don't necessarily want the program to crash, right?
You want to be able to handle that error in a semi-graceful way.
And in this particular example, with a division by 0, we can do that with exceptions.
So let's say that instead of the program crashing I want to print out a nicer error message rather than everything in big red.
I say, well, you divided by 0 so maybe you should change your numbers.
The way I capture, or catch this exception, is I have a tribe keyword, a colon, and then I have this block of code.
This is the code that could potentially raise an exception where the error might occur.
This code could be-- in this case, it's just straight up assignment and division.
It could be a function that raises the exception.
It doesn't matter.
The point is that there's an exception that might occur.
If an exception occurs, then Python is going to look past this block of code here at these except statements.
And it's going to look at the type of exception it has.
In this case it's a by 0 division error, and it's going to match the exception against whatever type this is.
And you can put inside here whatever code you want in order to handle this exception.
So in this case I'm just printing divided by 0.
I could be rude and I could say-- see, can't even spell.
So I can handle this exception however I want.
But the program's not going to crash.
And this program's actually not going to really convince you of that, but you see that-- the first thing you see here is that there's no big red blaring message that says you have an exception.
It's all code that I've written.
And I could even be silent about it.
I could say, I'm not going to say anything.
I'm just going to fail silently.
Tell me it's all good.
It's not going to tell me I divided by 0.
It can come in handy if I like have an input loop.
So let's say I'm going to do a little bit more, be a little bit more creative and do-- oh
[INAUDIBLE] How does it [INAUDIBLE]?
That's an internal Python thing.
So if you want to learn about all the wonderful exceptions that Python provides you in its class library, you can go to the Python documentation.
Or you can even do a Google search.
Of course, Chrome is being very slow, and the network's probably down.
So built-in exceptions.
So these are all the exceptions that Python has.
And you can use these too.
Let's say that, let me get back to my contrived example.
I want to create a loop that says-- let's say that I want to catch, I don't want to divide even numbers for some reason.
What do we have here?
What can we find?
I think there's a value error.
There we go.
So let's say that I'm going to raise a value error.
And I am actually going to do this this way.
So I've created a function that's not going to allow me to input even numbers.
It's going to raise a value error if I enter in a 2 or a 4 or whatever.
But, if I do enter in an odd number it is just going to return that number to me.
So I'm going to use that function now.
And I'm just going to do the silly thing and divide, do a division, and print it out.
So if I comment out this code up here and I comment this out, and I run this, so-- let's do 3.
That's interesting.
Apparently Python is a little constipated right now.
We edit this during the summer, so we can take that out, I think.
All right.
So it does what we expected it to.
It's going to keep on prompting us for numbers and it's going to divide them for us and give us the answers.
So we've got a really dumb calculator here.
Except when I try to do even numbers, because my calculator is really, really dumb.
So it's going to raise a value error.
And it raises the value error in this function here.
I can handle this, though, because I know how.
So what should I type here?
If you wanted to specifically catch that error.
But you could just leave it as except, and it would catch any error?
Yes.
[INAUDIBLE]
So if any error were to be thrown, then [INAUDIBLE].
But if you wanted specifically to say, OK, the value error [UNINTELLIGIBLE]
Right.
So what she was saying is that I could do this.
I don't have to specify what error I'm expecting.
I could say-- But it's not going to give me any detail about what happened.
And what I'm going to do is, I'm going to say that if I have an exception then I'm just going to repeat my getting of the numbers.
So, let's say that I do this.
Something happened, not really specific.
Nothing happened?
OK, I divided all right
What if you put in [UNINTELLIGIBLE]?
What's that?
[INAUDIBLE] Would it also come back in case something happened?
Yeah.
That's a good question.
Something happened, but we don't know what.
We know we have a bug, because an exception was thrwn.
But we don't know what it is
Because it's never going to crash
Yeah.
It's never going to crash
So a useful example of when to use this, would that be like [INAUDIBLE], and you have the list to choose from the letters [INAUDIBLE] it would remove each of the guesses?
If you accidentally guess something that you've already guessed?
Yeah, you can raise an exception to say, already guessed this
As opposed to the program, that first one I ran, I accidentally did the same letter again, and the whole program just crashed
Yep
So you can raise an exception for whoever's [INAUDIBLE]
Yeah.
You can catch any errors that were raised.
So we've got a couple of exceptions that are raised here.
And actually this program can crash.
Oh, wait.
No, it can't.
Because [INAUDIBLE].
It thinks [INAUDIBLE].
Silly Python.
So it's not going to crash at all.
We did a really good job, right?
So now we want to get a little bit more specific so I'm going to catch a value error.
So I'm only going to handle this one type of exception.
Because-- so it still works.
A, something happened.
But now we know that it's because I did something silly.
And, now I can break the program because this keyboard interrupt, that's actually an exception.
When you hit Control-C, it's going to break you out of the program.
But because we're catching that exception it just kept going
[INAUDIBLE] make an exception [INAUDIBLE]
Yes
[INAUDIBLE]
[INAUDIBLE]
Wait, what?
[INTERPOSING VOICES]
I'm quoting the girls over there, so
Are you?
Uh-oh.
So something happened.
Oh, wow.
How did that happen?
Oh, I know how that happened.
What am I missing here?
So I caught the keyboard interrupt, right?
I didn't go to the top of the loop.
So what happened is, it tried to execute this, but num2 hadn't been defined.
So we're going to kick me back up to the top of the loop here
Oh.
It's a clingy program.
[LAUGHTER]
Yes.
It is a clingy program.
Anyway.
So, does everyone get the idea of what the exceptions are?
So I don't have to go any more?
In this except up there, there's always [INAUDIBLE]?
Yeah
But what does this [INAUDIBLE]?
So When you get an exception, when an exception is raised, it's actually an object.
So there's an object that's associated with it.
And in a lot of cases you really don't have to be concerned about it, but sometimes you want some additional information.
So these are not very interesting exceptions, but some exceptions might have some additional diagnostic information.
So if you were, say, logging the run of your program.
And you wanted to diagnose any exceptions that you caught?
Like, say, you got a generic exception or you got something that was kind of odd that weren't expecting, you could print out additional information.
And so E is the name of that object.
So, I can do this.
Well.
This is because -- if you give it a primer like that, it's going to put that message into the exception.
So now I can retrieve it when the exception is raised.
So if I'm raising the exception, I can provide additional information.
So that's the message that I've put into the exception when I raised it.
Does that make sense?
Well, how is that different-- so is it the same thing as saying print, [INAUDIBLE]?
Yeah.
So this is a really [INAUDIBLE] example.
So this is just going to print out whatever message I passed in.
So when I say this and point to my screen, I really meant this and highlighted this.
So, this line is going to print out this message that I passed in when I raised the exception.
Because what I'm actually doing is, I'm creating a value error object that has this message associated with it.
And that's what gets passed up this exception chain until I finally handle it.
here.
And E holds the reference to that value error object
So E is an actual value?
Yeah.
So if I do something like this-- and I print out the type-- you can see that?
It's pretty neat.
But did that answer your question?
[INAUDIBLE]
Could it be anything?
So let's think of something like this.
Like a list
No.
I meant in your accept value error dash E. Could you put any variable there, or does it have to be E?
Oh, any name?
Yeah
I've always used E. That's kind of a convention.
But I think you'd be fine doing that.
E is the convention that I use, but some people don't like that.
They're like, you should name it something a little bit better.
But honestly, why bother?
Ok. Are we good with exceptions?
That answer your question?
Any other questions on exceptions?
Is there a built-in amount of errors you can have?
Like value errors, [UNINTELLIGIBLE], can you make up your own?
Can you make up your own exceptions?
Yes.
You can do that.
You can say-- but-- and-- I think this is going to work.
It's been while since I've defined my own exceptions.
So and again, I can handle it.
So if you ever work on big systems, a lot of times they'll have their own exception hierarchies that define all sorts of exceptions for the specific application you're working on.
Python's built in exceptions are pretty rich, so-- there we go
[INAUDIBLE]
I don't think you do.
It's convention.
I could probably do this, I think.
Oh, wait.
The only difference-- or the only problem is, is that now-- see I expect my exception to have a message attribute.
And I don't have anything on this.
So I have to do this.
Or something similar, right?
I have to define message on this.
And I think this'll work.
Well, actually, it does guard against it.
So yes, you do have to inherit from exception.
It's one of the few things that Python seems to guard against.
So, does that answer your question?
So we're getting kind of late.
The only other topics that we have are the simulations and the distributions and that's about it.
So, are there any questions on Monte Carlo methods?
We've kind of done those to death, right?
Real quick, someone humor me, what is a Monte Carlo method?
An algorithm that uses random sampling for trials to [INAUDIBLE]
Right.
So, it's-- what's that?
[INAUDIBLE]
It computes some value or solution to a problem using a random process and repeated trials.
So--
Is it almost brute force?
No, it's not what we call brute force.
I mean, it's like we computed pi by throwing a dart at a chalkboard or something like that.
And it landed in a square, and we used some sort of analysis to figure out what pi was based on that throwing a dart at a board a million times.
So that's what a Monte Carlo method is.
Anything that uses random stuff to compute a value.
A random simulation.
Where'd my chalk go?
And we actually a couple of Monte Carlo methods when I showed the solutions for problems (3) and (4) on the quiz.
I used a random number generator in order to figure out the probabilities of not rolling a 6 and all that stuff.
And I used the Monte Carlo method to show me that I was wrong on my initial answer for rolling exactly one 6.
So, is everyone comfortable with coefficient of variation?
Because I did that first, initially.
Confidence intervals and levels?
Does anyone have any questions on that?
What is a confidence interval?
What is a confidence interval and a confidence level?
OK.
So, let's say I do a political poll.
Let's say I do a political poll and I sample 1,000 people in the population.
And I say 46% of them will vote for Candidate X.
My margin of error is plus or minus 4, and my confidence is 95.
What this is saying is that if I conduct this poll, the same poll, sampling 1,000 random people at a time, over 100 trials-- so I do 100 polls-- what I'm saying is that 95 out of 100 of those times, my level of support for Candidate X will be between 42% and 50%, right?
Basically what you need to know for confidence levels and intervals.
The other thing that you need to know for confidence levels and intervals is that if you have a normal distribution, with a standard deviation sigma, then 1 standard deviation from the mean is going to have-- is that 66 or 67?
Is it 68?
I wrote it down, 68.
Thank you.
And then 2 standard deviations is going to be 95.
And then 3 standard deviations is going to be what, 97?
Or is it 99.7?
99.7.
So, but you need to just know that for normal distributions.
So if this is one standard deviation, it's saying that 68 of the sample points are going to be within 1 standard deviation of this mean.
And then if you have 2 sigma, then that means 95 are going to be within 2 sigma of the mean.
And then if you have 3 sigma, then 99.7 are going to be within 3 standard deviations of the mean.
So I think that's all you really need to know for confidence intervals and levels, and also for normal distributions.
So we're actually kind of at the end of the review session.
So does anyone have any questions that I haven't answered?
No?
All right.
I'm done.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I wanted to give everybody a more conceptual idea of what big O notation as well as, hopefully, answer any lingering questions you might have about object-oriented programming.
So I have these notes, and I type them up, and they're pretty detailed.
So I'm just going to go through some points kind of quickly.
Who's still kind of unclear about why we even due big O notation?
So who can explain why we do big O notation quickly?
What is it?
Yeah?
[INAUDIBLE]
Right.
Exactly.
So big O notation gives us an upper bound on how long something is going to take.
Now, something that's important to remember is it's not a time bound.
So something that's often confusing is that people say, oh, this is something that'll tell us how long our programs going to run.
That's actually not the case.
Big O notation informs how many steps something's going to take.
And so why is that important?
Well, I mean I look at all you guys, a couple of you have laptops out.
Everybody's computer run something at a different speed.
Right?
But if we say something is big O of n, what we're saying here is we're saying that the worst case number of steps your program is going to take is going to be linear with respect to the size of the input.
So if my computer is five times faster than your computer, my computer will probably run it five times faster.
As the size of the input grows, I'm going to expect a linear speedup in the amount of time it's going to take.
So why is that important?
At the bottom of page one, big O notation, we are particularly concerned with the scalability of our functions.
So what the O notation does is it might not predict what's going to be the fastest for really small inputs, for an array size 10.
You guys know a little bit about graphs, right?
We have a graph of x-squared, and we have a graph of x-cubed.
There's a portion of time where the graph of x-squared is actually bigger than x-cubed.
But then all of a sudden, there's a point where x-cubed just goes, whoosh, way bigger than x-squared.
So if we're in some really small amount of input, big O notation might not tell us what's the best function.
But in big O notation, we're not concerned about small inputs.
We're concerned about really big inputs.
We're concerned about filtering the genome.
We're concerned about analyzing data from Hubble, really huge blocks of data.
So if we're looking at a program that analyzes the human genome, like three million base pairs, some segment that we're looking at, and we have two algorithms.
One runs in order n time, and one runs in order n-cubed time.
What this means is regardless of the machine that we're running on, so this is algorithm 1, this is algorithm 2, regardless of the machine that we're running on, we'd expect algorithm 2 to run approximately n-cubed over n approximately n-squared slower.
So with big O notation, you can compare two algorithms by just looking at the ratio of their big O run time.
So if I'm looking at something that has an array of size two million as its input, is it clear that this is going to be a much better choice?
Ok.
So you'll run into that, especially a lot of you guys are taking this for the purposes of scientific computing.
So you'll run into big O notation a lot.
It's important to have a grasp of what it means.
On the second page of the handout, I have some common ones that you'll see.
The first one is constant time.
We denote constant time as order 1.
But you'll notice that I have here is order 1 is equal to order 10 is equal to order 2 to the 100th.
That's unexpected to a lot of people who are learning about big O notation.
Why is this true?
That seems kind of ridiculous.
This is a really big number.
This is really small number.
Yeah?
[INAUDIBLE]
Yeah.
Exactly.
So we look at a graph of 1 and a graph of 2 to the 100th.
Ok. We'll see that even though 2 to the 100th is much higher, much bigger than 1, if this is our input size, as the size of our input grows, do we see any change in these two graphs?
No.
They're completely constant.
When you're doing big O notation, if you run across an algorithm that does not depend on the size of the input, OK, it's always going to be order 1.
Even if it's like 2 to the 100th steps, if it's a constant number of times regardless of the size of the input, it's constant time.
Other ones you'll see are logarithmic time.
Any base for logarithmic time is about the same order.
So order log base 2 of n is order log base 10 of n. This is the fastest time bound for search.
Does anybody know what type of search we'd be doing in logarithmic time?
Something maybe we--
Bisection time
Yeah.
Exactly.
Bisection search is logarithmic time.
Because we take our input.
And at every step, we cut in half, cut in half, cut in half, and that's the fastest search we can do.
The order n is linear time.
Order n log n is the fastest time bound we have for sort.
We'll be talking about sort in a couple of weeks.
And order n-squared is quadratic time.
Anything that is order n to some variable, so order n-squared, order n-cubed, order n-fourth, all of that is going to be less than order something to the power of n. So if we have something that's order 2 to the n, that's ridiculous.
That's a computationally very intensive algorithm.
So on page two, I have some questions for you.
(1), (2), (3).
Does order 100 n-squared equal order n-squared.
Who says yes?
All right.
Very good.
How about does order one quarter n-cubed equals order n-cubed?
Does order n plus order n equals order n?
The answer is yes to all of those.
In the intuitive sense behind this is that big O notation deals with the limiting behavior of function.
So I made some nifty graphs for you guys to look at.
When we're comparing order 100 n-squared to n-squared n cubed and 1/4 n-cubed, what people often think of is what I have here in the first figure.
So these are the four functions I just mentioned.
There's a legend in the top left-hand corner.
And the scale of this is up to x equals 80.
So you'll see at this scale, this line right here is 100 x-squared.
So this is, I think, often a tripping point is that when people are conceptualizing functions, they're saying, well, yeah, 100 x-squared is much bigger than x-cubed, which is a lot bigger than 1/4 x-cubed.
So for very small inputs, yes that's true.
But what we're concerned about is the behavior as the input gets very, very large.
So now, we're looking at a size of up to 1,000.
So now we see here, x-cubed, even though it's a little bit smaller than 100 x-squared in the beginning, it shoots off.
x-cubed is much bigger than either of the 2 x-squared.
And even 1/4 x-cubed is becoming bigger than 100 x-squared out of 1,000.
So that's an intuitive sense why x-cubed no matter what the coefficient is in front of it is going to dominate any term with x-squared in it, because x-cubed is just going to go, whoosh, real big like that.
And if we go out even further, let's go out to input size of 50,000, we go out to an input size of 50,000, we see that even 100 x-squared versus just x-squared, alright?
they're about the same.
The x-cubed terms now, they're way above x-squared.
So the two x-squared terms, 100 versus just 1, as far as the coefficient goes, they're about the same.
So this is the scale at which we're concerned about when we're talking about big O notation, the limiting behavior as your input size grows very large.
50,000 is not even that large, if you think about the size of the genome.
I mean does anybody here bio?
What's like the size of the human genome.
How many base pairs?
Or even one gene or one chromosome
[INAUDIBLE]
What's the biggest?
It's over 50,000
Yeah, over 50,000.
And we're talking about the amount of data that we get back from the Hubble Space Telescope.
I mean the resolution on those things are absolutely ridiculous.
And you run all sorts of algorithms on those images to try and see if there's life in the universe.
So we're very concerned about the big long term behavior of these functions.
How about page three?
One last question.
Does order 100 n-squared plus 1/4 n-cube equal order n-cubed?
Who says yes?
And so I have one more graph.
Down here, these red dots are 100 x-squared.
These blue circles are 1/4 x-cubed.
And this line is the sum.
We can see that this line is a little bit bigger than the 1/4 x-cubed term.
But really, this has no effect at this far out.
So that's why we're just going to drop any lower order terms whenever you're approached with a big O expression that has a bunch of constant factors, it has all sorts of different powers of n and stuff, you're always just going to drop all the constant factors and just pick the biggest thing.
So this line right here is order n-cubed.
Is that clear to everybody?
So now I've gotten through the basics of how we analyze this and why are we looking at this.
Let's look at some code.
So the first example, all of these things right here, in Python, we make the assumption that statements like this, x plus 1, x times y, all these mathematical operations are all constant time.
That's something that you can just assume.
So for this function down here, we have constant time, constant time, constant time, constant time operation.
So we'd say, this function bar is what?
What's its complexity?
[INAUDIBLE]
Yeah.
Constant time.
So the complexity of all these functions are just a 1, because it doesn't matter how big the input is.
It's all going to run in constant time.
For this multiplication function here, we use a for loop.
Oftentimes, when we see for loops that's just going through the input, there's a signal to us that it's going to probably contain a factor of O(n).
Why is that?
What do we do in this for loop?
We say for i in range y.
What does that mean?
How many times do we execute that for loop?
Yeah, y times.
So if y is really small, we execute that for loop just a few number of times.
But if y is really large, we execute that for loop a whole bunch of times.
So when we're analyzing this, we see this for loop and we say, ah, that for loop must be O(y).
Does that make sense to everybody?
OK, good.
Let's look at a factorial.
Can anybody tell me what the complexity of factorial is?
[INAUDIBLE]
Yeah.
Order n. Why is it order n?
Because it's self for loop
Yeah.
It's the exact same structure.
We have a for loop that's going through range 1 to n plus 1.
So that's dependent on the size of n. So this for loop is order n. And inside the for loop, we just do a constant time operation.
That's the other thing.
Just because we have this for loop doesn't mean that what's inside the for loop is going to be constant.
But in this case, if we have order n times, we do a content time operation.
Then this whole chunk of the for loop is order n. The rest of everything else is just constant time.
So we have constant time plus order n times constant time plus constant time, they're going to be order n. How about this one?
Factorial 2
[INAUDIBLE]
Yeah.
Exactly.
This is also order n. The only thing that's different in this code is that we initialize its count variable.
And inside the for loop, we also increment this count variable.
But both result times equals num and count plus equal 1, both of these are constant time operations.
So if we do n times 2 constant times operations, that's still going to be order n. So the takeaway from these two examples that I'm trying to demonstrate here is a single line of code can generate a pretty complex thing.
But a collection of lines of code might still be constant time.
So you have to look at every line of code and consider that.
I've thrown in some conditionals here.
What's the complexity of this guy?
[INAUDIBLE]
Yeah.
This is also linear.
What's going on here?
We initialize a variable count that's constant time.
We go through character in a string.
This is linear in the size of a string.
Now we say if character equal, equal t, this character equal, equal t, that's also a constant time operation.
That's just asking if this one thing equals this other thing.
So we're looking at two characters.
We're looking at two numbers.
Equal, equal or not equal is generally a constant times operation.
The exception to this might be a quality of certain types, like if you define a class and you define a quality method in your class, and the equality method of your class is not constant time, then this equal, equal check might not be constant time.
But on two strings, equal, equal is constant time.
And this is constant time as well.
So linear in the size of a string.
Something that's important when you're doing this for exams, it's a good idea to define what n is before you give the complexity bound.
So here I'm saying n is equal to the size of a string.
So now, I can say this function is order n. What I'm saying is that it's a linear with respect to the size or the length of a string.
Because sometimes, like in the one where there is the input x and y, the running time was only linear in the size of y.
So you want to define that n was equal to the size of y to say that it was order n. So always be clear.
If it's not clear, be sure to explicitly state what n is equal to.
This code's a little more tricky.
What's going on here?
[INAUDIBLE]
Yeah.
That was perfect.
So just to reiterate, the for loop we know is linear with respect to the size of a string.
We have to go through every character in a string.
Now, the second is if char in b string, when we're looking at big O notation, we're worried about the worst case complexity in upper bound.
What's the worst case?
[INAUDIBLE]
Yeah.
If the character is not in b string, we have to look at every single character in b string before we can return false.
So that is linear.
This one single line, if character in b string, that one line is linear with respect to the size of b string.
So how do we analyze the complexity of this?
I want to be able to touch the screen.
We have this for loop.
This for loop is executed.
Let's call n is the length of a string.
This for loop is executed n times.
Every time we execute this for loop, we execute this inner body.
And what's the time bound on the inner body?
Well, if we let m equal the length of b string, when we say that this check is order m every time we run it, then we run an order m operation order n times.
So the complexity is-- we use something of size m, n times
[INAUDIBLE]
Yeah.
Just order n, m. So we execute an order m check order n time, we say this function is order n, m. Does that make sense to everybody?
Because you'll see the nested for loops.
Nested for loops are very similar to this.
While loops combine the best of conditionals with the best of for loops.
Because a while loop has a chance to act like for loop, but a while loop can also have a conditional.
It's actually possible to write a while loop that has a complex conditional that also executes a number of times.
And so you could have one single line of code generating like an order n-squared complexity.
Let's look at factorial 3. Who can tell the complexity of factorial 3?
[INAUDIBLE]
Yeah.
It's also linear.
It's interesting that factorial is always linear despite its name.
We have constant time operations.
How many times does the while loop executed?
n times
Yeah, n times.
And what's inside the body of the while loop?
Constant time operations.
So we execute a bunch of constant time operation n times order n. How about this char split example?
This one's a little tricky because you're like, well, what's the complexity of len?
In Python, len's actually a constant time operation.
This example's very crafted such that all of the operations that are here are constant time.
So appending to a list is constant time.
And indexing a string is constant time.
So what's the complexity of char split?
Constant time
[INAUDIBLE]
Who would agree with constant time?
And who would say it's linear time?
OK, yeah.
Very good.
It is linear time.
That's a correct intuition.
We say while the length of the a string is not equal to the length of the result, these are two constant time operations.
Well, what do we do?
We append a value to the result, and then we add up this index.
So when is this check going to be equal?
This check's going to be equal when the length of the result is equal to the length of a string.
And that's only going to happen after we've gone through the entire a string, and we've added each of its characters to result.
So this is linear with respect to the size of a string.
Something that's important to recognize is that not all string in the list operations are constant time.
There's a website here that first off, it says C Python if you go to it.
C Python just means Python implemented in C, which is actually what you're running, C Python.
So don't worry about that.
There's often two time bound complexities.
It says the amortized time and the worst case time.
And so if you're looking for big O notation, you don't want to use the amortized time.
You want to use the worst case time.
And it's important to note that operations like slicing and copying actually aren't constant time.
If you slice a list or a string, the complexity of that operation is going to depend on how big your slice is.
Does that makes sense?
Does the way that a slice works is that walks through the list until it gets to the index, and then keeps walking until the final index, and then copies that and returns it to you.
So slicing is not constant time.
Copying is similarly not constant time.
For this little snippet of code, this is just similar to what we-- yeah?
[INAUDIBLE]
So this is what I was saying.
You want to define what n is.
So we say something like n equals the length of a string.
And then you can say it's order n. It's important to define what you're saying the complexity is related to.
So here, I'm saying if we let n equal to the size of z, can anybody tell me what the complexity of this snippet of code is?
[UNINTELLIGIBLE]
[INAUDIBLE]
Yeah, precisesly.
Order n-squared.
Why?
Well, because we execute this for i for loop here order n times.
Each time through this for loop, the body of this for loop is, in fact, another for loop.
So my approach to problems like this is just step back a minute and ignore the outer loop.
Just concentrate on the inner loop.
What's the runtime of this inner loop?
Yeah.
This is order n. We go through this.
Now, go to the outer loop.
Just ignore the body since we've already analyzed the body.
Ignore it.
What's the complexity of the outer loop?
Also order n. So now you can combine the analysis.
You can say for order n times, I execute this body.
This body takes order n times.
So if execute something that's order n order n times, that is order n squared complexity.
So we just multiply how long it takes the outer body of the loop to take the inner body of the loop.
And so in this fashion, I could give you now probably a four or five nested for loop, and you could tell me the complexity of it.
Harder sometimes to understand is recursion.
I don't know how important it is to understand this because I've never actually taught this class before.
But Mitch did tell me to go over this.
So I'd like to touch on it.
So consider recursive factorial.
What's the time complexity of this?
How can we figure out the time complexity over a recursive function?
The way we want to figure out the time complexity of a recursive function is just to figure out how many times we're executing said recursive function.
So here I have recursive factorial of n. When I make a call to this, what do I do?
I make a call to recursive factorial n minus 1.
And then what does this do?
This calls recursive factorial on a sub problem the size n minus 2.
So oftentimes, when you're dealing with recursive problems to figure out the complexity, what you need to do is you need to figure out how many times you're going to make a recursive call before a result is returned.
Intuitively, we can start to see a pattern.
We can say, I called on n, and then n minus 1, and then n minus 2, and I keep calling recursive factorial until n is less than or equal to 0.
When is n going to be less than or equal to 0?
Well, when I get n minus n. So how many calls is that?
[INAUDIBLE]
Yeah.
This is n calls.
So it's a good practice to get into being able to draw this out and work yourself through how many times you're running the recursion.
And we see we're making n calls, we can say, oh, this must be linear in time.
How about this one, this foo function?
This one's a little harder to see.
But what are we doing?
We call foo on input of size n, which then makes a call to sub problem the size n/2, which makes the call to a sub problem of size n/4 and so on until I make a call to sub problem of some size.
So this is n. This is 2 to the 1st.
This is 2-squared.
We start to see a pattern-- 2-squared, 2-cubed, 2 to the fourth.
So we're going to keep making calls on a smaller, and smaller, and smaller sub problem size.
But instead of being linear like before, we're decreasing at an exponential rate.
There's a bunch of different ways to try and work this out in your head.
I wrote up one possible description.
But when we're decreasing at this exponential rate, what's going to end up happening is that this recursive problem where we make a recursive call in the form to sub problem of size n/b, the complexity of that is always going to be log base b of n. So this is just like bisection search, where bisection search, we essentially do in bisection search.
We restrict the problem size by half every time.
And that leads to logarithmic time, actually log base 2 of n. This problem is also log base 2 of n. If we change this recursive call from n/2 to n/6, we get a cut time complexity of log base 6 of n. So try and work that through.
You can read this closer later.
Definitely ask me if you need more help on that one.
The last one is how do we deal time complexity of something like Fibonacci?
Fibonacci, fib n minus 1 plus fib n minus 2, initially, that kind of looks linear.
Right?
We just went over the recursive factorial, and it made the call to a sub problem the size n minus 1.
And that was linear.
Fibonacci's a little bit different.
If you actually draw out in a tree, you start to see like at every level of the tree, we expand the call by 2.
Now imagine this is just for Fibonacci of 6.
Whenever you're doing big O complexity, you want to imagine it and put 100,000, 50,000.
And you could imagine how big that tree grows.
Intuitively, the point to see here is that they're going to be about n levels to get down to 1 from your initial input of 6.
So to get down to 1 from an initial input of size n is going to take about n levels.
The branching factor of this tree at each level is 2.
So if we have n levels, and at each level, we increase our branching factor by another 2, we can say that a loose bound on the complexity of this is actually 2 to the n. This is something that's even less intuitive, I think, than what we did before with the logarithms.
So try and work through it again.
Play with it a little bit.
There's actually a tighter bound on this, which is like 1.62 to the n, which is a lot more complicated math that you could look up.
But for the purposes of this class, it's sufficient to say that Fibonacci is order 2 to the n. So does that roughly clear up some time complexities stuff for you guys?
OK, awesome.
Does anybody have the time?
I forgot my watch today
12:42
OK, excellent.
That gives us a little bit of time to talk about object-oriented programming.
Does anybody had any specific questions that object-oriented programming?
How about this?
How many of you guys finished the problem set and turned it in already?
Or did any of you guys not turn in the problem set yet?
I'll talk loosely about it then, not too specifically.
Does anybody have any questions from, I guess, at least the first part?
We're making some classes, making some trigger classes.
Yeah?
[INAUDIBLE]?
Self dot what?
[INAUDIBLE]
When we have like self-- we have like the getter methods.
So what's important about that?
I'll Tell you what's important about that.
So we have a class.
Let's say we have a class person.
So we define our INIT method to just take a name.
And so now, what the problems that ask you to do was to define a getter method.
Define a getter method called get_name that just returns the attribute.
So what's the point of this?
Because I can just say Sally equals person.
So here, I defined a person named Sally.
And I initialized a person with the string Sally.
If I just look at sally.name, that's going to just directly print the attribute.
So why do we need this get name function?
What's the point of this additional getter method?
Does anybody know why that is?
[INAUDIBLE]
Right.
So that's what it does.
This get_name does return the attribute name.
But we don't need this method to just look at the attribute name.
Let's actually code this up.
So we have class person.
So if we run this code, and over here in the shell, we define Sally equals person with the name Sally.
If I just print sally.name, it prints the attribute.
So why did I need to provide this getter method called get_name that does the same thing?
That's the question.
That seems sort of redundant.
But there's actually a pretty big important reason for it.
Let's say we set s name equal to the attribute sally.name.
If we look at s name, we see Sally.
Now if I say-- actually, I'm not sure if this is the correct reasoning.
This is going to be better.
Let's say Sally equals a person Sally who's taking what?
1803, 605, 11.1.
So now I can look at the attribute classes to show Sally's classes, which are weird flows.
And I can also use sally.getclasses to look at Sally's classes.
If I set a variable s classes equal to sally.classes, this binds this variable s classes to the attribute sally.classes.
Now if I say sclasses.append 1401, if I now look at the attribute sally.classes, it now has 1401 in it.
This is not safe.
This is not type safe.
Because the reason for that is if you define a class, and you access the classes' attributes directly instead of through a getter method, you can then do this.
And sometimes, it's accidental.
You'll set some variable equal to some attribute of a class.
Then later on in your code, you'll alter that variable.
But that variable is not a copy of the attribute.
Yes, you can make copies of that attribute and stuff, but the overall takeaway is that in programming, we try to do something called defensive programming.
This isn't defensive.
Because it is possible if you code it incorrectly to alter the attribute the instance of the class.
But if we use the getter method, if instead of sally.classes, instead of directly accessing the attribute here, we have set s classes equal to sally.getclasses.
And then, we had changed s classes around.
That wouldn't have happened, because the getter method, it does return self.classes.
But in the way that Python is scoped and when we return something, we're not returning the exact same thing, the reference that we're returning a copy of it.
Does that make sense?
All right.
Cool.
Other questions about classes?
We have a little class appear if there's like some basic stuff that you'd like explained again.
Now's the time
[INAUDIBLE]
So here, I'm setting just some variable s classes equal to the attribute sally classes.
it's just like setting any sort of variable equal to some other quantity
So you appending the variable, but it also appended like the attribute of Sally?
So what I did here was I set the variable s classes equal to this attribute sallly.classes.
And then, because I know this is a list, I appended another value to it.
But this is the same as when we have two lists.
If we have a list called a, and we say a is equal to 1, 2, 3, then I say b is equal to a.
What is b?
Now If I say b.append 1401, what does b look like?
What does a look like?
Because they're aliases of each other.
So what I did here, when I set s classes directly equal to the attribute sally.classes, I made s classes an alias of the attribute.
But the problem with that is that then I can change them.
And because they're aliases, the attribute itself has changed.
And we don't want to do that in object-oriented programming.
We need to find an object.
The only way you should be able to change an attribute is through some method of the class that allows you to change that attribute.
So if I want to be able to add a class to Sally's class lists, I should define a method called define add class that does self.classes.append new class.
While technically, it's possible to directly access an attribute, it's really bad practice to do so simply because this unexpected behavior can result.
And also because if you say, oh, well, it's not going to matter for this one time, I'll remember how to do the right thing.
The problem with that is it's often the case that you're not the only person using your code.
So it's a better practice to provide all the sorts of methods that you would need to do with the class in order to get an access and change attributes as methods within the class.
Does that make sense?
So yeah, this is maybe our one violation if you guys have been attending my recitation.
Our mantra of programmers are lazy.
This is less lazy than just directly accessing the attributes.
But even though we know that programmers are super, super lazy, programmers also like to be super, super safe.
So when there's a trade off between defensive programming and being lazy, always pick defensive programming.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality, educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Let's begin.
As I mentioned before, this lecture will be recorded for OCW.
Again, in future lectures, if you don't want to have the back of your head show up, just don't sit in this front area here.
First of all, wow, what a crowd, you guys.
We're finally in 26-100.
6.0001 made it big, huh?
Good afternoon and welcome to the very first class of 6.0001, and also 600, this semester.
My name is Ana Bell.
First name, Ana.
Last name, Bell.
I'm a lecturer in the EECS Department.
And I'll be giving some of the lectures for today, along with later on in the term, Professor Eric Grimson, who's sitting right down there, will be giving some of the lectures, as well.
Today we're going to go over some basic administrivia, a little bit of course information.
And then, we're going to talk a little bit about what is computation?
We'll discuss at a very high level what computers do just to make sure we're all on the same page.
And then, we're going to dive right into Python basics.
We're going to talk a little bit about mathematical operations you can do with Python.
And then, we're going to talk about Python variables and types.
As I mentioned in my introductory email, all the slides and code that I'll talk about during lectures will be up before lecture, so I highly encourage you to download them and to have them open.
We're going to go through some in-class exercises which will be available on those slides.
And it's fun to do.
And it's also great if could take notes about the code just for future reference.
It's true.
This is a really fast-paced course, and we ramp up really quickly.
We do want to position you to succeed in this course.
As I was writing this, I was trying to think about when I was first starting to program what helped me get through my very first programming course.
And this is really a good list.
The first thing was I just read the psets as soon as they came out, made sure that the terminology just sunk in.
And then, during lectures, if the lecturer was talking about something that suddenly I remembered, oh, I saw that word in the pset and I didn't know what it was.
Well, hey, now I know what it is.
Right?
So just give it a read.
You don't need to start it.
If you're new to programming, I think the key word is practice.
It's like math or reading.
The more you practice, the better you get at it.
You're not going to absorb programming by watching me write programs because I already know how to program.
You guys need to practice.
Download the code before lecture.
Follow along.
Whatever I type, you guys can type.
And I think, also, one of the big things is if you're new to programming, you're kind of afraid that you're going to break your computer.
And you can't really do that just by running Anaconda and typing in some commands.
So don't be afraid to just type some stuff in and see what it does.
Worst case, you just restart the computer.
Yeah.
That's probably the big thing right there.
I should have probably highlighted it, but don't be afraid.
Great.
So this is pretty much a roadmap of all of 6.0001 or 600 as I've just explained it.
There's three big things we want to get out of this course.
The first thing is the knowledge of concepts, which is pretty much true of any class that you'll take.
The class will teach you something through lectures.
Exams will test how much you know.
This is a class in programming.
The other thing we want you to get out of it is programming skills.
And the last thing, and I think this is what makes this class really great, is we teach you how to solve problems.
And we do that through the psets.
That's really how I feel the roadmap of this course looks like.
And underlying all of these is just practice.
You have to just type some stuff away and code a lot.
And you'll succeed in this course, I think.
OK.
So what are the things we're going to learn in this class?
I feel like the things we're going learn in this class can be divided into basically three different sections.
The first one is related to these first two items here.
It's really about learning how to program.
Learning how to program, part of it is figuring out what objects to create.
You'll learn about these later.
How do you represent knowledge with data structures?
That's sort of the broad term for that.
And then, as you're writing programs, you need to-- programs aren't just linear.
Sometimes programs jump around.
They make decisions.
There's some control flow to programs.
That's what the second line is going to be about.
The second big part of this course is a little bit more abstract, and it deals with how do you write good code, good style, code that's readable.
When you write code, you want to write it such that-- you're in big company, other people will read it, other people will use it, so it has to be readable and understandable by others.
To that end, you need to write code that's well organized, modular, easy to understand.
And not only that, not only will your code be read by other people, but next year, maybe, you'll take another course, and you'll want to look back at some of the problems that you wrote in this class.
You want to be able to reread your code.
If it's a big mess, you might not be able to understand-- or reunderstand-- what you were doing.
So writing readable code and organizing code is also a big part.
And the last section is going to deal with-- the first two are actually part of the programming in Introduction to Programming and Computer Science in Python.
And the last one deals mostly with the computer science part in Introduction to Programming and Computer Science in Python.
We're going to talk about, once you have learned how to write programs in Python, how do you compare programs in Python?
How do you know that one program is better than the other?
How do you know that one program is more efficient than the other?
How do you know that one algorithm is better than the other?
That's what we're going to talk about in the last part of the course.
OK. That's all for the administrative part of the course.
Let's start by talking at a high level what does a computer do.
Fundamentally, it does two things.
One, performs calculations.
It performs a lot of calculations.
Computers these days are really, really fast, a billion calculations per second is probably not far off.
It performs these calculations and it has to store them somewhere.
Right?
Stores them in computer memory.
So a computer also has to remember results.
And these days, it's not uncommon to find computers with hundreds of gigabytes of storage.
The kinds of calculations that computers do, there are two kinds.
One are calculations that are built into the language.
These are the very low level types of calculations, things like addition, subtraction, multiplication, and so on.
And once you have a language that has these primitive calculation types, you, as a programmer, can put these types together and then define your own calculations.
You can create new types of calculations.
And the computer will be able to perform those, as well.
I think, one thing I want to stress-- and we're going to come back to this again during this entire lecture, actually-- is computers only know what you tell them.
Computers only do what you tell them to do.
They're not magical.
They don't have a mind.
They just know how to perform calculations really, really quickly.
But you have to tell them what calculations to do.
Computers don't know anything.
All right.
We've come to that.
Let's go into the types of knowledge.
The first type of knowledge is declarative knowledge.
And those are things like statements of fact.
And this is where my email came into play.
If you read it all the way to the bottom, you would have entered a raffle.
So a statement of fact for today's lecture is, someone will win a prize before class ends.
And the prize was a Google Cardboard.
Google state-of-the-art virtual reality glasses.
And I have them right here.
Yea.
I delivered on my promise.
That's a statement of fact.
So pretend I'm a machine.
OK?
I don't know anything except what you tell me.
I don't know.
I know that you tell me this statement.
I'm like, OK.
But how is someone going to win a Google Cardboard before class ends, right?
That's where imperative knowledge comes in.
Imperative knowledge is the recipe, or the how-to, or the sequence of steps.
Sorry.
That's just my funny for that one.
So the sequence of steps is imperative knowledge.
If I'm a machine, you need to tell me how someone will win a Google Cardboard before class.
If I follow these steps, then technically, I should reach a conclusion.
Step one, I think we've already done that.
Whoever wanted to sign up has signed up.
Now I'm going to open my IDE.
I'm just basically being a machine and following the steps that you've told me.
The IDE that we're using in this class is called Anaconda.
I'm just scrolling down to the bottom.
Hopefully, you've installed it in problem set zero.
I've opened my IDE.
I'm going to follow the next set of instructions.
I'm going to choose a random number between the first and the nth responder.
Now, I'm going to actually use Python to do this .
And this is also an example of how just a really simple task in your life, you can use computers or programming to do that.
Because if I chose a random number, I might be biased because, for example, I might like the number 8.
To choose a random number, I'm going to go and say, OK, where's the list of responders?
It starts at 15.
Actually, it starts at 16 because that's me.
We're going to choose a random number between 16 and the end person 266.
Oh, we just got-- oh.
OK. OK.
I'm going to cut it off right here.
271.
OK. 16 and 271.
Perfect.
OK.
I'm going to choose a random number.
I'm going to go to my IDE.
And you don't need to know how to do this yet, but by the end of this class, you will.
I'm just going to use Python.
I'm just going to get the random number package that's going to give me a random number.
I'm going to say random.randint.
And I'm going to choose a random number between 16 and 272, OK. 75.
OK. Great.
I chose a random number.
And I'm going to find the number in the responder's sheet.
What was the number again?
Sorry.
75.
OK. Up we go.
There we go.
Lauren Z-O-V. Yeah.
Nice.
You're here.
Awesome.
All right.
That's an example of me being a machine and also, at the same time, using Python in my everyday life, just lecturing, to find a random number.
Try to use Python wherever you can.
And that just gives you practice.
That was fun.
But we're at MIT.
We're MIT students.
And we love numbers here at MIT.
Here's a numerical example that shows the difference between declarative and imperative knowledge.
An example of declarative knowledge is the square root of a number x is y such that y times y is equal to x.
That's just a statement of fact It's true.
Computers don't know what to do with that.
They don't know what to do with that statement.
But computers do know how to follow a recipe.
Here's a well-known algorithm.
To find the square root of a number x, let's say x is originally 16, if a computer follows this algorithm, it's going to start with a guess, g, let's say, 3.
We're trying to find the square root of 16.
We're going to calculate g times g is 9.
And we're going to ask is if g times g is close enough to x, then stop and say, g is the answer.
I'm not really happy with 9 being really close to 16.
So I'm going to say, I'm not stopping here.
I'm going to keep going.
If it's not close enough, then I'm going to make a new guess by averaging g and x over g. That's x over g here.
And that's the average over there.
And the new average is going to be my new guess.
And that's what it says.
And then, the last step is using the new guess, repeat the process.
Then we go back to the beginning and repeat the whole process over and over again.
And that's what the rest of the rows do.
And you keep doing this until you decide that you're close enough.
What we saw for the imperative knowledge in the previous numerical example was the recipe for how to find the square root of x.
What were the three parts of the recipe?
One was a simple sequence of steps.
There were four steps.
The other was a flow of control, so there were parts where we made decisions.
Are we close enough?
There were parts where we repeated some steps.
At the end, we said, repeat steps 1, 2, 3.
That's the flow of control.
And the last part of the recipe was a way to stop.
You don't want a program that keeps going and going.
Or for a recipe, you don't want to keep baking bread forever.
You want to stop at some point.
Like 10 breads is enough, right?
So you have to have a way of stopping.
In the previous example, the way of stopping was that we decided we were close enough.
Close enough was maybe being within .01, .001, whatever you pick.
This recipe is there for an algorithm.
In computer science speak, it's going to be an algorithm.
And that's what we're going to learn about in this class.
We're dealing with computers.
And we actually want to capture a recipe inside a computer, a computer being a mechanical process.
Historically, there were two different types of computers.
Originally, there were these things called fixed-program computers.
And I'm old enough to have used something like this, where there's just numbers and plus, minus, multiplication, divide, and equal.
But calculators these days are a lot more complicated.
But way back then, an example of a fixed-program computer is this calculator.
It only knows how to do addition, multiplication, subtraction, division.
If you want to plot something, you can't.
If you want to go on the internet, send email with it, you can't.
It can only do this one thing.
And if you wanted to create a machine that did another thing, then you'd have to create another fixed-program computer that did a completely separate test.
That's not very great.
That's when stored-program computers came into play.
And these were machines that could store a sequence of instructions.
And these machines could execute the sequence of instructions.
And you could change the sequence of instructions and execute this different sequence of instructions.
You could do different tasks in the same machine.
And that's the computer as we know it these days.
The central processing unit is where all of these decisions get made.
And these are all the peripherals.
The basic machine architecture-- at the heart of every computer there's just this basic architecture-- and it contains, I guess, four main parts.
The first is the memory.
Input and output is the other one.
The ALU is where all of the operations are done.
And the operations that the ALU can do are really primitive operations, addition, subtraction, and so on.
What the memory contains is a bunch of data and your sequence of instructions.
Interacting with the Arithmetic Logic Unit is the Control Unit.
And the Control Unit contains one program counter.
When you load a sequence of instructions, the program counter starts at the first sequence.
It starts at the sequence, at the first instruction.
It gets what the instruction is, and it sends it to the ALU.
The ALU asks, what are we doing operations on here?
What's happening?
It might get some data.
If you're adding two numbers, it might get two numbers from memory.
It might do some operations.
And it might store data back into memory.
And after it's done, the ALU is going to go back, and the program counter is going to increase by 1, which means that we're going to go to the next sequence in the instruction set.
And it just goes linearly, instruction by instruction.
There might be one particular instruction that does some sort of test.
It's going to say, is this particular value greater or equal to or the same as this other particular value?
That's a test, an example of a test.
And the test is going to either return true or false.
And depending on the result of that test, you might either go to the next instruction, or you might set the program counter to go all the way back to the beginning, and so on.
You're not just linearly stepping through all the instructions.
There might be some control flow involved, where you might skip an instruction, or start from the beginning, or so on.
And after you're done, when you finished executing the last instruction, then you might output something.
That's really the basic way that a computer works.
Just to recap, you have the stored program computer that contains these sequences of instructions.
The primitive operations that it can do are addition, subtraction, logic operations, tests-- which are something equal to something else, something less than, and so on-- and moving data, so storing data, moving data around, and things like that.
And the interpreter goes through every instruction and decides whether you're going to go to the next instruction, skip instructions, or repeat instructions, and so on.
So we've talked about primitives.
And in fact, Alan Turing, who was a really great computer scientist, he showed that you can compute anything using the six primitives.
And the six primitives are move left, move right, read, write, scan, and do nothing.
Using those six instructions and the piece of tape, he showed that you can compute anything.
And using those six instructions, programming languages came about that created a more convenient set of primitives.
You don't have to program in only these six commands.
And one interesting thing, or one really important thing, that came about from these six primitives is that if you can compute something in Python, let's say-- if you write a program that computes something in Python, then, in theory, you can write a program that computes the exact same thing in any other language.
And that's a really powerful statement.
Think about that today when you review your slides.
Think about that again.
That's really powerful.
Once you have your set of primitives for a particular language, you can start creating expressions.
And these expressions are going to be combinations of the primitives in the programming language.
And the expressions are going to have some value.
And they're going up some meaning in the programming language.
Let's do a little bit of a parallel with English just so you see what I mean.
In English, the primitive constructs are going to be words.
There's a lot of words in the English language.
Programming languages-- in Python, there are primitives, but there aren't as many of them.
There are floats, Booleans, these are numbers, strings, and simple operators, like addition, subtraction, and so on.
So we have primitive constructs.
Using these primitive constructs, we can start creating, in English, phrases, sentences, and the same in programming languages.
In English, we can say something like, "cat, dog, boy.
That, we say, is not syntactically valid.
That's bad syntax.
That's noun, noun, noun.
That doesn't make sense.
What does have good syntax in English is noun, verb, noun.
So, "cat, hugs boy" is syntactically valid.
Similarly, in a programming language, something like this-- in Python, in this case-- a word and then the number five doesn't really make sense.
It's not syntactically valid.
But something like operator, operand, operator is OK.
So once you've created these phrases, or these expressions, that are syntactically valid, you have to think about the static semantics of your phrase, or of your expression.
For example, in English, "I are hungry" is good syntax.
But it's weird to say.
We have a pronoun, a verb, and an adjective, which doesn't really make sense.
"I am hungry" is better.
This does not have good static semantics.
Similarly, in programming languages-- and you'll get the hang of this the more you do it-- something like this, "3.2 times 5, is OK.
But what does it mean?
What's the meaning to have a word added to a number?
There's no meaning behind that.
Its syntax is OK, because you have operator, operand, operator.
But it doesn't really make sense to add a number to a word, for example.
Once you have created these expressions that are syntactically correct and static, semantically correct, in English, for example, you think about the semantics.
What's the meaning of the phrase?
In English, you can actually have more than one meaning to an entire phrase.
In this case, "flying planes can be dangerous" can have two meanings.
It's the act of flying a plane is dangerous, or the plane that is in the air is dangerous.
And this might be a cuter example.
"This reading lamp hasn't uttered a word since I bought it.
What's going on?"
So that has two meanings.
It's playing on the word "reading lamp."
That's in English.
In English, you can have a sentence that has more than one meaning, that's syntactically correct and static, semantically correct.
But in programming languages, the program that you write, the set of instructions that you write, only has one meaning.
Remember, we're coming back to the fact that the computer only does what you tell it to do.
It's not going to suddenly decide to add another variable for some reason.
It's just going to execute whatever statements you've put up.
In programming languages, there's only one meaning.
But the problem that comes into play in programming languages is it's not the meaning that you might have intended, as the programmer.
That's where things can go wrong.
And there's going to be a lecture on debugging a little bit later in the course.
But this is here just to tell you that if you see an error pop up in your program, it's just some text that says, error.
For example, if we do something like this, this is syntactically correct.
Incorrect.
Syntactically incorrect.
See?
There's some angry text right here.
What is going on?
The more you program, the more you'll get the hang of reading these errors.
But this is basically telling me the line that I wrote is syntactically incorrect.
And it's pointing to the exact line and says, this is wrong, so I can go back and fix it as a programmer.
Syntax errors are actually really easily caught by Python.
That was an example of a syntax error.
Static semantic errors can also be caught by Python as long as, if your program has some decisions to make, as long as you've gone down the branch where the static semantic error happens.
And this is probably going to be the most frustrating one, especially as you're starting out.
The program might do something different than what you expected it to do.
And that's not because the program suddenly-- for example, you expected the program to give you an output of 0 for a certain test case, and the output that you got was 10.
Well, the program didn't suddenly decide to change its answer to 10.
It just executed the program that you wrote.
That's the case where the program gave you a different answer than expected.
Programs might crash, which means they stop running.
That's OK. Just go back to your code and figure out what was wrong.
And another example of a different meaning than what you intended was maybe the program won't stop.
It's also OK.
There are ways to stop it besides restarting the computer.
So then Python programs are going to be sequences of definitions and commands.
We're going to have expressions that are going to be evaluated and commands that tell the interpreter to do something.
If you've done problem set 0, you'll see that you can type commands directly in the shell here, which is the part on the right where I did some really simple things, 2 plus 4.
Or you can type commands up in here, on the left-hand side, and then run your program.
Notice that, well, we'll talk about this-- I won't talk about this now.
But these are-- on the right-hand side, typically, you write very simple commands just if you're testing something out.
And on the left-hand side here in the editor, you write more lines and more complicated programs.
Now we're going to start talking about Python.
And in Python, we're going to come back to this, everything is an object.
And Python programs manipulate these data objects.
All objects in Python are going to have a type.
And the type is going to tell Python the kinds of operations that you can do on these objects.
If an object is the number five, for example, you can add the number to another number, subtract the number, take it to the power of something, and so on.
As a more general example, for example, I am a human.
So that's my type.
And I can walk, speak English, et cetera.
Chewbacca is going to be a type Wookie.
He can walk, do that sound that I can't do.
He can do that, but I can't.
I'm not even going to try, and so on.
Once you have these Python objects, everything is an object in Python.
There are actually two types of objects.
One are scalar objects.
That means these are very basic objects in Python from which everything can be made.
These are scalar objects.
That can't be subdivided.
The other type of object is a non-scalar object.
And these are objects that have some internal structure.
For example, the number five is a scalar object because it can't be subdivided.
But a list of numbers, for example, 5, 6, 7,8, is going to be a non-scalar object because you can subdivide it.
You can subdivide it into-- you can find parts to it.
It's made up of a sequence of numbers.
Here's the list of all of the scalar objects in Python.
We have integers, for example, all of the whole numbers.
Floats, which are all of the real numbers, anything with a decimal.
Bools are Booleans.
There's only two values to Booleans.
That's True and False.
Note the capitalization, capital T and capital F. And this other thing called NoneType.
It's special.
It has only one value called None.
And it represents the absence of a type.
And it sometimes comes in handy for some programs.
If you want to find the type of an object, you can use this special command called type.
And then in the parentheses, you put down what you want to find the type of.
You can write into the shell "type of 5," and the shell will tell you, that's an integer.
If you happen to want to convert between two different types, Python allows you to do that.
And to do that, you put the type that you want to convert to right before the object that you want to convert to.
So float(3) will convert the integer 3 to the float 3.0.
And similarly, you can convert any float into an integer.
And converting to an integer just truncates.
It just takes away the decimal and whatever's after it-- it does not round-- and keeps just the integer part.
For this slide, I'm going to talk about it.
But if you'd like if you have the slides up, go to go to this exercise.
And after I'm done talking about the slide, we'll see what people think for that exercise.
One of the most important things that you can do in basically any programming, in Python also, is to print things out.
Printing out is how you interact with the user.
To print things out, you use the print command.
If you're in the shell, if you simply type "3 plus 2," you do see a value here.
Five, right?
But that's not actually printing something out.
And that becomes apparent when you actually type things into the editor.
If you just do "3 plus 2," and you run the program-- that's the green button here-- you see on the right-hand side here, it ran my program.
But it didn't actually print anything.
If you type this into the console, it does show you this value, but that's just like peeking into the value for you as a programmer.
It's not actually printing it out to anyone.
If you want to print something out, you have to use the print statement like that.
In this case, this is actually going to print this number five to the console.
That's basically what it says.
It just tells you it's an interaction within the shell only.
It's not interacting with anyone else.
And if you don't have any "Out," that means it got printed out to the console.
All right.
We talked a little bit about objects.
Once you have objects, you can combine objects and operators to form these expressions.
And each expression is going to have a value.
So an expression evaluates to a value.
The syntax for an expression is going to be object, operator, object, like that.
And these are some operators you can do on ints and floats.
There's the typical ones, addition, subtraction, multiplication, and division.
If, for the first three, the answer that you get-- the type of the answer that you get-- is going to depend on the type of your variables.
If both of the variables of the operands are integers, then the result you're going to get is of type integer.
But if at least one of them is a float, then the result you're going to get is a float.
Division is a little bit special in that no matter what the operands are, the result is always going to be a float.
The other operations you can do, and these are also useful, are the remainder, so the percent sign.
If you use the percent sign between two operands, that's going to give you the remainder when you divide i by j.
And raising something to the power of something else is using the star star operator.
And i star stars j is going to take i to the power of j.
These operations have the typical precedence that you might expect in math, for example.
And if you'd like to put precedence toward some other operations, you can use parentheses to do that.
All right.
So we have ways of creating expressions.
And we have operations we can do on objects.
But what's going to be useful is to be able to save values to some name.
And the name is going to be something that you pick.
And it should be a descriptive name.
And when you save the value to a name, you're going to be able to access that value later on in your program.
And that's very useful.
To save a value to a variable name, you use the equal sign.
And the equal sign is an assignment.
It assigns the right-hand side, which is a value, to the left-hand side, which is going to be a variable name.
In this case, I assigned the float 3.14159 to the variable pi.
And in the second line, I'm going to take this expression, 22 divided by 7, I'm going to evaluate it.
It's going to come up with some decimal number.
And I'm going to save it into the variable pi_approx.
values are stored in memory.
And this assignment in Python, we say the assignment binds the name to the value.
When you use that name later on in your program, you're going to be referring to the value in memory.
And if you ever want to refer to the value later on in your code, you just simply type the name of the variable that you've assigned it to.
So why do we want to give names to expressions?
Well, you want to reuse the names instead of the values.
And it makes your code look a lot nicer.
This is a piece of code that calculates the area of a circle.
And notice, I've assigned a variable pi to 3.14159.
I've assigned another variable called radius to be 2.2.
And then, later on in my code, I have another line that says area-- this is another variable-- is equal to-- this is an assignment-- to this expression.
And this expression is referring to these variable names, pi and radius.
And it's going look up their values in memory.
And it's going to replace these variable names with those values.
And it's going to do the calculation for me.
And in the end, this whole expression is going to be replaced by one number.
And it's going to be the float.
Here's another exercise, while I'm talking about the slide.
I do want to make a note about programming versus math.
In math, you're often presented with a problem that says, solve for x. x plus y is equal to something something.
Solve for x, for example.
That's coming back to the fact that computers don't know what to do with that.
Computers need to be told what to do.
In programming, if you want to solve for x, you need to tell the computer exactly how to solve for x.
You need to figure out what formula you need to give the computer in order to be able to solve for x.
That means always in programming the right-hand side is going to be an expression.
It's something that's going to be evaluated to a value.
And the left-hand side is always a variable.
It's going to be an assignment.
The equal sign is not like in math where you can have a lot of things to the left and a lot of things to the right of the equal sign.
There's only one thing to the left of the equal sign.
And that's going to be a variable.
An equal sign stands for an assignment.
Once we've created expressions, and we have these assignments, you can rebind variable names using new assignment statements.
Let's look at an example for that.
Let's say this is our memory.
Let's type back in the example with finding the radius.
Let's say, pi is equal to 3.14.
In memory, we're going to create this value 3.14.
We're going to bind it to the variable named pi.
Next line, radius is equal to 2.2.
In memory, we're creating this value 2.2.
And we're going to bind it to the variable named radius.
Then we have this expression here.
It's going to substitute the values for pi from memory and the value for radius from memory.
It's going to calculate the value that this expression evaluates to.
It's going to pop that into the memory.
And it's going to assign-- because we're using the equal sign-- it's going to assign that value to that variable area.
Now, let's say we rebind radius to be something else.
Radius i is bound to the value 2.2.
But when we do this line, radius is equal to radius plus 1, we're going to take away the binding to 2.2.
We're going to do this calculation.
The new value is 3.2.
And we're going to rebind that value to that same variable.
In memory, notice we're still going to have this value, 2.2, floating around.
But we've lost the handle for it.
There's no way to get it back.
It's just in memory sitting there.
At some point, it might get collected by what we call the garbage collector.
In Python, And it'll retrieve these lost values, and it'll reuse them for new values, and things like that.
But radius now points to the new value.
We can never get back 2.2.
And that's it.
The value of area-- notice, this is very important.
The value of area did not change.
And it did not change because these are all the instructions we told the computer to do.
We just told it to change radius to be radius plus 1.
We never told it to recalculate the value of area.
If I copied that line down here, then the value of area would change.
But we never told it to do that.
The computer only does what we tell it to do.
That's the last thing.
Next lecture, we're going to talk about adding control flow to our programs, so how do you tell the computer to do one thing or another?
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So the question was, if you have these two lines right here-- which I'll just put them up here just so it's bigger-- type(5), print(3.0-1), what is going to be the output, so if I run this in the editor?
So for these types of problems, you can always check yourself.
And this comes back to, if you're new to programming, don't be afraid to try it out.
Don't ask me.
Don't ask your neighbor.
Just type it into the shell and hit Run.
And then you get to see what the answer is.
So the answer to this question is going to be 2.0.
And let's see, are we right?
Yeah, great.
75% of the people got it right.
Perfect.
So if you didn't get it right, just to explain again, the reason why this doesn't get printed out is because we never actually had a print statement.
So if you want to show something to the console, you need to use a print statement.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this next exercise is which of the following is allowed in Python.
So x plus y is equal to 2.
And again, these are all things that you can just type into Anaconda, which is our IDE for the class, and see whether it works or not.
So this-- the first one is probably not right-- let's increase it a little bit-- is not right because the left side needs to be a variable.
This is not right because it's an expression.
This is not right because 2 is not a proper variable name.
And xy-- that one's actually going to be OK because it's not multiplication, it's just the variable xy.
So 50% said xy is equal to 2, and that's right.
So none of these.
So just to make sure that you guys understand, xy, this last one here, is the right one because xy is not x times y, like in math.
It's just one variable name that has two characters, x and y.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So for this last class exercise we have this following code.
So we have 46 for USA gold, 27 for UK gold, 1 for Romanian gold.
We make this variable, here, Total Gold, to be the sum of those three which I believe is 74.
Then we're going to print total gold.
OK?
Then we're going to increase the value for Romanian gold and we're going to print it again.
So as always, you can just copy this, pop it into Python, and run it to test yourself, but you should do it just in your mind first.
So notice it print 74 in 74, and that's because we never told the program to calculate the new total.
So we only calculated the total, way up here on line 5.
If we calculated the total again down here, then it would be 74.75 So let's look at the answers, hopefully you guys got that.
Perfect, majority are good to go.
If you didn't get that please try to review it, review why it didn't work.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Let's get started, everyone.
So, good afternoon.
Welcome to the second lecture of 60001 and also of 600.
So as always, if you'd like to follow along with the lectures, please go ahead and download the slides and the code that I'll provide at least an hour before class every day.
All right.
So a quick recap of what we did last time.
So last time, we talked a little bit about what a computer is.
And I think the main takeaway from the last lecture is really that a computer only does what it is told, right?
So it's not going to spontaneously make decisions on its own.
You, as the programmer, have to tell it what you want it to do by writing programs.
OK.
So we talked about simple objects.
And these objects were of different types.
So we saw integers, floats, and Booleans.
And then we did a couple of simple operations with them.
Today, we're going to look at a different-- a new type of object called a string.
And then we're going to introduce some more powerful things in our programming toolbox.
So we're going to look at how to branch within a program, and how to make things-- how to make the computer repeat certain tasks within our program.
All right.
So let's begin by looking at strings.
So strings are a new object type.
We've seen so far integers, which were whole numbers, floats, which were decimal numbers, and we have seen Booleans, which were true and false.
So strings are going to be sequences of characters.
And these characters can be anything.
They can be letters, digits, special characters, and also spaces.
And you tell Python that you're talking about a string object by enclosing it in quotation marks.
So in this case, I'm creating an object whose value is h-e-l-l-o space t-h-e-r-e. And Python knows it's a string object, because we're enclosing it in quotations.
They can be either double quotes or single quotes, but as long as you're consistent, it doesn't matter.
And this object, we're binding it to this variable named hi.
And we're using that using the equals sign, which is the assignment operator.
So from now on, whenever we refer to this variable hi, Python is going to say, oh, I know what the value is, and it's that string of characters.
So we're going to learn about two things that you can do on strings today, two operations.
One is to concatenate them.
And concatenation is really just a fancy word for using this plus operator, which means put the strings together.
So I have this original variable named hi, and I create a new variable called name.
And in it, I'm going to assign the string a-n-a to the variable name.
And when I use the plus operator in between hi and name, those two variables, Python is going to look at the values of those two, and it's going to just put them together.
OK.
I'm going to switch to Spider.
And this is just that example from the slides.
So let's see what happens.
So I have the variable hi, the variable name, and I'm just concatenating those two together.
And then I'm going to print that out.
So if I run the code, notice it prints out "hello thereana."
There's no space.
And there's no space because the concatenation operator, the plus, doesn't add any spaces implicitly.
So again, another example of just computer just doing what it's told.
If we want to add a space, we'd have to actually insert the space manually.
So that's this line here, line 8.
And in this line, we're concatenating the value of the variable hi with a space.
Notice we're putting it in quotation marks.
Just a space.
And then with name.
So if we'll go ahead and print that value, notice this was that garbage greeting there.
And now we have a proper greeting, right?
So that's the concatenation between strings.
And then the other thing we're going to look at related to strings is the star operator.
So that's this one here on line 10.
So Python allows you to use the star operator, which stands for multiplication, between a string and a number.
And when you do that, Python interprets it as repeat that string that many number of times.
So in this case, I'm creating a silly greeting, and I'm concatenating the value of hi, which is "hello there" with the space plus the name.
So notice here, I'm using parentheses to tell Python, do this operation first, and then multiply whatever the result of this is by 3.
So if I print that out, it's going to multiply the space with my name three times, and it's going to concatenate that with "hello there."
So that's exactly what it printed out there.
Last lecture, we talked a little bit about print.
Today, I'm going to talk about some nuances related to print.
So you use print to interact with the user.
It's cool to write programs that print things out to the user.
So the key word here being print.
And then you put parentheses after print.
And in the parentheses, you put in whatever you want to show the user.
So in this little program, I have-- I created a variable named x. I assigned it the value 1, and then I print 1.
Here, I'm casting.
So I'm taking the number one, the integer 1, and I'm casting it to a string.
And you'll see why in a moment.
So I want to bring to your attention a couple of things here.
So in the first print, I'm using commas everywhere here.
And in the second print, I'm using plus.
So by definition, if you-- you can use commas inside a print-- inside the parentheses of print.
And if you use a comma, Python is going to automatically add a space in between the two things that the comma is in between, the values.
So "my fav num is" is the first thing.
And the second thing is whatever's after the comma.
Let's take x.
So if you use a comma, Python is going to automatically insert a space for you.
Sometimes, you might want that, sometimes you might not.
If you don't want that, you can use the concatenation operation, the plus.
And you can add all of your little bits together to create one big string.
If you're using commas, the items, the objects in between the commas, do not all have to be strings.
That's the plus side of using commas.
But the downside is you get spaces everywhere.
If you use plus operator, the plus side is Python does exactly what you tell it to do, but everything has to be a string object.
So "my fav num is" is a string object.
You have to convert all of your numbers to string objects, and so on.
So if we look at Spider-- This is the same-- almost the same code.
So here, I don't have spaces anywhere.
So you can see that the first line here has commas everywhere.
So I'm going to have spaces in between every one of the things that I'm printing out.
This line here is sort of a combination between commas and concatenation.
So depending on where I used the comma, I'm going to have an extra space.
And this line here just has concatenation everywhere.
So if I run this, notice this very first line added spaces everywhere in between all my objects.
The second one added spaces somewhere.
And you can sort of trace through and see exactly where the spaces were added.
And the last line here didn't add spaces anywhere.
So printing things out to the console is nice, but the second part of sort of writing an interactive program is getting input from the user.
And that's the more interesting part.
So if you've done problem set 0, you might have sort of already tried to understand this on your own.
But here we are.
So the way you get input from the user is using this command function called input.
And inside the parentheses, you type in whatever you'd like to prompt the user with.
So in this case, in my example here, I have input, and then here I said "type anything."
So the user is going to see this text here, and then the program is just going to stop.
And it's going to wait for the user to type in something and hit Enter.
As soon as the user types in Enter, whatever the user types in becomes a string.
If a user types in a number, for example, that becomes the string of that number.
So everything the user types in is going to be made as a string.
In this line right here, whatever these the user types in becomes a string.
And we're going to bind that string object to this variable named text.
So now, further in my program, I could do whatever I want with this variable text.
In this case, I'm going to print 5*text.
OK.
So if the user, for example, gave me "ha," I'm going to print "ha" 5 times.
If the user gave me 5, what do you think the user is-- what do you think is going to be printed out?
25 or 5 five times?
Great.
Yes.
Exactly.
5 five times.
Oftentimes, you don't want to work with numbers as strings, right?
You want to work with numbers as numbers, right?
So you have to cast.
And we learned that last lecture.
You cast by just putting in this little bit right in front of the input.
And you can cast it to whatever type you want.
Here I cast it to an int, but you can also cast to a float if you want to work with floats.
And that converts whatever the user typed in, as long as it's some number that Python knows how to convert, into the number itself.
So in this case, if the user gives me 5, I'm going to print out 5 times 5 instead of 5 five times.
So that's the code here.
So the first bit is I'm going to get the user to type in anything, and I'm going to put 555.
And then when I type in the number, since I'm casting it, I'm going to do operations with the number.
Yeah, question
[INAUDIBLE]
Why do you want to cast to-- oh.
The question is why do you want to cast to a string?
Why do you want to cast a string to a number?
[INAUDIBLE]
Oh, so Python always-- whatever you type in, just by default, by definition of the input command, Python always makes it a string.
So if you want to work with numbers, you have to explicitly tell it, I'm going to work with a number.
So even if you give it the number 5, it's going to think it's the string 5.
Yeah.
That's just how input works.
The next thing we're going to look at is ways that you can start adding tests in your code.
And before you can start adding tests in your code, you need to be able to do the actual tests.
So this is where comparison operators come in.
So here, let's assume that i and j are variables.
The following comparisons are going to give you a Boolean.
So it's either going to say, this is true or this is false.
So that's going to be your test.
So if i and j are variables, you're allowed to compare ints with ints, floats with floats, strings with strings.
And you're allowed to compare ints and floats between themselves, but you're not allowed to compare a string with a number.
In fact, if you even try to do that in Python-- in Spider here, if I try to say, is the letter a greater than 5?
I get some angry text right here.
And this just tells me Python doesn't understand the meaning of-- how do I compare a string with a number?
OK.
So just like in math, we can do these usual comparisons.
We can say if something is greater than something, greater or equal to, less than, less than or equal to.
I'd like to bring to your attention the equality.
So the single equals sign is an assignment.
So you're taking a value, and you're assigning it to a variable.
But when you're doing the double equals sign, this is the test for equality.
Is the value of variable i the same as the value of the variable j?
And that's, again, also going to give you a Boolean either true or false.
And you can also test for inequality with the exclamation equal.
So that means, is the value of the variable i not equal to the value of the variable j?
True if yes, false if no.
OK.
So those are comparison operators on integer, floats, and strings.
On Booleans, you can do some logic operators.
And the simplest is just inverting.
So if a is a variable that has a Boolean value, not a is just going to invert it.
So if a is true, then not a is false, and vice versa.
This is a table that sort of represents what I've said here.
So you can do-- you can use and and or.
These are key words in Python.
You can use those two key words on variables, on Boolean variables.
And you get the result a and b is only true if both a and b are true.
And a or b is only false if a and b are false.
And this is the complete table just in case you need to reference it.
All right.
So now that we have ways to do logical-- question right there
[INAUDIBLE]
Yeah, great question.
So what does it mean to compare a string with a string with the greater than?
So that's just going to compare them, lexicographically.
So does it come first in the alphabet?
So we can even test that out.
We can say, is a greater than b?
And it's false.
So b comes later in the alphabet than a. OK.
So now we have ways to do the tests.
So we can add some branching to our programming toolbox now that we have ways to do tests.
This is a map of MIT.
I'm going to go through sort of a little example to motivate why we would want to do branching in our code.
And I think after this lecture, you'll be able to sort of code up this algorithm that I'm going to explain.
So most of us see MIT as a maze.
I first did when I came here.
When I first came here, obviously, I signed up for the free food mailing list.
And MIT, being a maze, I had no idea where to go, what the shortest path was to free food.
So one way to think about it is all I wanted to do was get to the free food.
A very simple algorithm to get there would be to say, OK, I'm going take my right hand, and I'm going to make sure that my right hand is always on a wall.
And I'm going to go around campus with my right hand always being at a wall.
And eventually, I'll get to where the free food is.
There might not be any left, right?
But I'll be there.
So the algorithm is as follows.
If my right hand always has to be on a wall, then I'm going to say, if there's no wall to my right side, then I'm going to go right until I get to a wall.
Then if there's a wall to my right, and I can go forward, I'm just going to keep going forward.
If I keep going forward, and there's a wall to my right and in front of me, I'm going to turn around and go left.
And then if there's a wall to my right, in front of me, and to the left, then I'm going to turn around and go back.
So with this fairly simple algorithm, I just follow the path always keeping the wall to my right.
And eventually, I would end up where I need to be.
So notice, I used, just in plain English, a few key words.
If, otherwise, things like that.
So in programming, we have those same constructs.
And those same sort of intuitive words can be used to tell Python to do something or to do something else or to choose from a different set of possibilities.
And this way, we can get the computer to make decisions for us.
And you might be thinking, well, you said that computers can't make decisions on their own.
It's not.
You, as programmers, are going to build these decisions into the program, and all the computer is going to do is going to reach the decision point and say, OK, this is a decision point, should I go left or should I go right?
Or which one do I pick?
And these sort of decisions are created by you as a programmer.
And the computer just has to make the decision and choose a path.
OK.
So in programming, there's three sort of simple ways that you can add control flow to your programs.
And that's making one decision and choosing whether to execute something or execute something else.
The first is a simple if.
And given a program that just linearly has statements that get executed, whenever I reach an if statement, you're going to check the condition.
The condition is going to be something that's going to get evaluated to either true or false.
So I've reached the condition here.
And if the condition is true, then I'm going to additionally execute this extra set of expressions.
But if the condition is false, then I'm just going to keep going through the program and not execute that extra set of instructions.
How does Python know which instructions to execute?
They're going to be inside this what we call code block.
And the code block is denoted by indentation.
So it's going to be everything that's indented is part of that if code block.
Typically, four spaces is indentation.
OK.
So that's how you write code that decides whether to execute this extra thing or not.
Now let's say I don't just want to execute an extra thing, I want to reach a point where I say, I'll either go down this path or I'll do something else.
That's this right here.
So this if else construct says this is my code, I've reached my decision point here, if the condition inside the if is true, then I'm going to execute maybe this set of statements here.
But if the condition is not true, then I'm not going to execute that set of statements, and instead I'm going to execute under whatever else is.
So using this construct, I'm either going to do one set of expressions or the other, but never both.
And after I've executed one or the other, I'm going to continue on with just the regular execution of the program.
OK.
So we're able to either choose one thing, choose one thing or another, but what if we want to have more than one choice?
So if some number is equal to zero, I want to do this.
If it's equal to 1, I want to do this.
If it's equal to 2, I want to do this, and so on.
That's where this last one comes in.
And we introduced this other key word here called elif.
So that stands for short form for else if.
So first we check if this condition is true.
So we're going through our program, we've reached our decision point, if the condition is true, we're going to execute maybe this set of instructions.
If the condition is not true, maybe we'll check-- if the condition is not true, we will check this next condition.
That's part of the elif right here.
And if that one's true, we're going to execute a different set of instructions.
You can have more than one elif.
And depending on which one's true, you're going to execute a different set of instructions.
And then this last else is sort of a catch all where if none of the previous conditions were true, then just do this last set of expressions.
So in this case, you're going to choose between one of these three-- one of these four roots, or however many you have.
And then when you're done making your choice, you're going to execute the remaining set of instructions.
So the way this works is if more than one condition is true, you're actually just going to enter one of them.
And you're going to enter the very first one that's true.
So you're never going to enter more than one of these code blocks.
You always enter one, and you enter the first one that evaluates to true.
So notice that we denoted code blocks using indentation.
And that's actually one of the things that I really like about Python.
It sort of forces you to write pretty code and nice looking code and just code that's very readable.
And that forces you to indent everything that's a code block.
So you can easily see sort of where the flow of control is and where decision making points are and things like that.
So in this particular example, we have one if statement here, and it checks if two variables are equal.
And we have an if, elif, else.
And in this example, we're going to enter either this code block or this one or this one, depending on the variables of x and y.
And we're only going into one code block.
And we'll enter the first one that's true.
Notice you can have nested conditionals.
So inside this first if, we have another if here.
And this inner if is only going to be checked when we enter the first-- this outter if.
I do want to make one point, though.
So sometimes, you might forget to do the double equals sign when you are checking for equality, and that's OK.
If you just use one equals sign, Python's going to give you an error.
And it's going to say syntax error, and it's going to highlight this line.
And then you're going to know that there's a mistake there.
And you should be using equality, because it doesn't make sense to be using-- to assign-- to be making an assignment inside the if.
So we've learned about branching.
And we know about conditionals.
Let's try to apply this to a little game.
And spoiler, we won't be able to.
We'll have to learn about a new thing.
But back in the 1980s, there was the Legend of Zelda-- cool graphics-- where there was a scene with the lost woods.
Oversimplification if anyone's a Zelda die hard fan.
But the basic idea was if you entered the woods, you entered from the left to the right.
And then as long as you kept going right, it would show you the same screen over and over again.
And the trick was you just had to go backward, and then you'd exit the woods.
So very simple.
Using what we know so far, we could sort of code this up.
And we'd say something like this.
If the user exits right, then set the background to the woods background.
Otherwise, set the background to the exit background.
Now let's say the user-- and then in the else, we're done.
Let's say the user went right.
Well, you'd show them the woods background, and now ask them again, where do they want to go?
If they exit right, set the background to the woods background.
Otherwise, set the background to the exit background, and so on.
So you notice that there's sort of no end to this, right?
How many times-- do you know how many times the user might keep going right?
They might be really persistent, right?
And they'll be like maybe if I go 1,000 times, I'll get out of the woods.
Maybe 1,001?
Maybe.
So this would probably be-- who knows how deep?
These nested ifs.
So we don't know.
So with what we know so far, we can't really code this cute little game.
But enter loops.
And specifically, a while loop.
So this code here that could be infinitely number of nested if statements deep can be rewritten using these three lines.
So we say while the user exits right, set the background to the woods background.
And with a while loop, it's going to do what we tell it to do inside the loop, and then it's going to check the condition again, and then it's going to do what we say it should do inside the code block, and it's going to check the condition again.
And then when the condition-- as long as a condition is true, it's going to keep doing that little loop there.
And as soon as the condition becomes false, it's going to stop doing the loop and do whatever's right after the while.
OK.
So that's basically how a while loop works.
We have while.
That's the key word.
The condition is something that gets evaluated to true or false.
And once again, we have a code block that's indented, and it tells Python, these are the expressions I want to do as long as the condition is true.
So the condition is true, you evaluate every expression in the code block.
When you reach the end of the expression-- end of the code block, you check the condition again.
If it's true still, you keep doing the expressions.
Check it again, and so on.
So here's a little game.
And with these lines of code, we were able-- we can code up the lost woods of Zelda.
Even worse graphics, by the way than the original Zelda is this one that I coded up here.
So I print out the following things.
"You're in the Lost Forest.
Go left or right."
And my program's going to say, "You're in the Lost Forest.
Go left or right."
It's going to get user input.
It's going to say while the user keeps typing in right, show them this text, and ask them again.
So I'm asking them again by just saying input here again.
And that's it.
That's going to just keep getting input from the user.
And if the user doesn't type in right, and maybe types in left, you're going to exit out of this loop, and print out, "You've got out of the Lost Forest."
So I have to show you this, because I spent too much time on it.
But I decided to improve on the code that's in the slides.
And I've written here ways that you guys can also improve it.
So if I run my code-- "You're in the Lost Forest.
Go left or right."
So if I say left, then yay, I got out of the Lost Forest.
But if I go right, then I'm stuck, right?
I took down some trees.
You can see there's no more trees here.
I made a table, and then I flipped it over.
So the expansion to this if you want to try it out-- I put this in the comments here-- is try to use a counter.
If the user types in right the first two times, just make that a sad face.
But if the user types in more than two times, make them cut down some trees and build a table and flip it.
That's a cute little expansion if you want to test yourself to make sure you are getting loops.
OK.
So so far, we've used while loops to ask for user input.
And that's actually somewhere where it makes sense to use while loops, because you don't actually know how many times the user is going to type in something.
You can use while loops to keep sort of a counter and to write code that counts something.
If you do that, though, there's two things you need to take care of.
The first is the first line here, which is sort of an initialization of this loop counter.
And the second is this line here, which is incrementing your loop counter.
The reason why the second one is important is because-- let's look at our condition here.
So while n is less than five.
If you didn't have this line here, you would never increment n. So every time through the loop, you just keep printing zeros.
And you would have an infinite loop.
I do want to show, though, what-- if you do have an infinite loop, it's not the end of the world.
So I can say something like-- so while true, print zero.
So this is going to give me an infinite loop in my program.
And-- whoop.
OK.
So notice it's just printing the letter p over and over again.
And if I let it go any longer, it's going to slow down the computer.
So I'm going to hit Control-C or Command-C maybe.
And it's going to stop the program from printing.
So just in case you ever enter infinite loops in your programs, just go to the console and hit Control-C, and that's going to stop it from sort of slowing down the computer.
OK.
So going back to this example, I was saying that if you're using counters-- variables in order to sort of count up inside the while loop, you have to take care to initialize a counter variable first.
And then to increment it, otherwise you'll enter an infinite loop.
That feels a little bit tedious.
And so there's a shortcut for doing that exact same thing.
So these four lines, you can rewrite those into these two lines right here using this new type of loop called a for loop.
So the for loop says, for some loop variable-- in this case, I named it n. You can name it whatever you want.
In range 5-- we're going to come back to what range means in a little bit-- print n. So every time through the loop, you're going to print out what the value of n is.
Range 5 actually creates internally a sequence of numbers starting from 0 and going to that number 5 minus 1.
So the sequence is going to be 0, 1, 2, 3, and 4.
The first time through the loop, you're going to say n is equal to 0.
Or internally, this is what happens.
N gets the value of 0.
You're going to print n. Then you're going to go back to the top.
N gets the value 1.
Then you're going to go execute whatever is inside.
So you're going to print 1.
Then you're going to increment that to the next value in the sequence.
You're going to print out 2, and so on.
So this is the general look of a for loop.
So we have for some loop variable-- again, can be named whatever you want-- in range some number.
Do a bunch of stuff.
And again, these are part of this for loop code block.
So you should indent them to tell Python that these are the things that you should do.
So when you're using range some number, you start out with variable getting the value 0.
With variable having value 0, you're going to execute all of these expressions.
After all the expressions in the code block are done, you're going to go on to the next value.
So 1.
You're going to execute all these expressions with the variable being value 1, and then so on and so on until you go to some num minus 1.
That-- so using range in that way is a little bit constraining, because you're always going to get values starting from 0 and ending at some num minus 1, whatever is in the parentheses in range.
Sometimes you might want to write programs that maybe start at a custom value.
Don't start at 0.
Maybe they start at 5.
Maybe they start at minus 10.
And sometimes you might want to write programs that don't go with-- don't expect the numbers by 1, but maybe skip every other number, go every two numbers, or every three numbers, and so on.
So you can customize range to your needs.
The one thing you do need to give it is the stop.
So if you give it only one value in the parentheses that stands for stop.
And by default, start is going to have the value 0, and step is going to have the value 1.
If you give it two things in the parentheses, you're giving it start and stop.
So the first being start, the second being stop.
And step gets this value of 1 by default.
And if you give it three things in the parentheses, you're giving it start, stop, and step in that order.
And you're always going to start at the start value and stop at-- or so you're going to start at the start value, and you're going to go until stop minus 1.
So those are the sequences of numbers.
So in this first code right here, my sum is going to get the value 0.
And you're going to have a for loop.
We're going to start from 7, because we're giving it two numbers.
And when you give it two numbers, it represents start and stop with step being 1.
So we're starting at 7.
If step is 1, the next value is 8.
What's the value after that?
If we're incrementing by 1?
9.
And since we're going until stop minus 1, we're not actually going to pick up on 10.
So this loop variable, i, the very first time through the loop is going to have the value 7.
So my sum is going to be 0 plus 7.
That's everything that's inside the code block.
The next time through the loop, i gets the value 8.
So inside the for loop, my sum gets whatever the previous value was, which was 7, plus 8.
OK.
The next time through the loop, my sum get the value 7 plus 8 plus 9.
Obviously, replacing that with the previous value.
So 15.
Since we're not going through 10, that's where we stop.
And we're going to print out my sum, which is going to be the value of 7 plus 8 plus 9.
Yeah?
OK. Yeah
[INAUDIBLE]
Do they have to be integers?
That's a great question.
We can try that out.
I'm not actually sure right off the top of my head.
So you can go on Spider and say-- let's say in this example here.
So we can say 7.1, 10.3-- yeah.
So they have to be integers.
OK.
So that's that example.
And let's erase that.
In this particular example, we have start, stop, and step.
And here, we're going every other value.
So we're starting at 5.
Tell me what the next value is supposed to be.
If we're taking every other one.
7, and then 9, and then-- are we doing 11 or not?
Excellent.
Nice.
Yeah.
So we're going to the end minus 1.
OK.
So it's possible that sometimes you write code where you might want to exit out of the loop early.
You don't want to go through all of the sequences of your numbers.
Maybe there's a condition inside there where you just want to exit the loop early.
Inside the while loop, maybe you want to exit the loop before the condition becomes false.
So that's where the break statement comes in.
So the break works like this.
It's going to-- as soon as Python sees this break statement, it's going to say, OK, I'm going to look at whatever loop I'm currently in.
I'm not evaluating any expression after it that comes within my loop.
And I'm going to immediately exit the loop.
So I'm going inside this while, this while, I'm evaluating this one expression, and I suddenly see a break.
Expression b does not get evaluated.
And break is going to immediately exit out of the innermost loop that it's in.
So this while loop that has condition 2, that's the innermost loop that the break is found in.
So we're going to exit out of this inner most loop here.
And we're evaluating expression c. And notice, we're evaluating expression c, because it's-- expression c is part of the outer while loop.
It's at the same level as this one.
And these ones are part of the inner while loop.
OK. Last thing I want to say is just a little bit of a comparison between for and while loops.
So when would you use one or the other.
This might be useful in your problem sets.
So for loops you usually use when you know the number of iterations.
While loops are very useful when, for example, you're getting user input, and user input is unpredictable.
You don't know how many times they're going to do a certain task.
For both for and while loops, you can end out of the loop early using the break.
The for loop uses this counter.
It's inherent inside the for loop.
A while loop you can use a counter in order-- you can use a while loop to count things.
But you must initialize the counter before the while loop.
And you have to remember to increment it within the loop.
Otherwise, you maybe lead to an infinite loop.
We've seen as the very first example of a for loop that the while-- the for loop could be rewritten as a while loop, but the vice versa is not necessarily true.
And the counterexample to that is just user input.
So you might not know how many times you might do a certain task.
All right.
Great.
That's all for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I have this variable name once, which is umbr-- looks like people are getting it.
We have a variable called repeat, which is ella, and I'm creating a variable u, which is the concatenation of that with the repeat multiplied by 4.
So I think people are on the right track, seeing by the 90% here.
It's going to give me this result right here.
Great.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I'm creating a variable called pset_time and I'm assigning it to be the value 15.
I'm creating a variable named sleep_time.
I'm assigning it to be the value 8.
I'm going to print the value of this expression here.
And the expression is a conditional that says, is sleep_time greater than pset_time?
So in Python, you're just going to replace these variables with their values.
So is 8 greater than 15?
And that's false, so perfect.
And then one operation on Booleans derive as true.
Drink is false.
I'm using the AND operator to figure out if I should drink and derive, and I'm going to print out false.
So, perfect, people are getting it.
Great job, guys.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we have x is going to be whatever number the user types in, y is going to be whatever number the user types in.
First thing I'm going to check is if x is equal to y.
So if we type in x is 0 and y is 5, 0 is not equal to 5, so we're not even going to consider what's in this code block.
Then we're going to check the next one-- if 0 is less than 5-- and that's true.
So this is the very first conditional that evaluates to true, so we're just going to print x is smaller.
And I think that's what's being dragged down here.
Awesome.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's look at the example, or the in-class exercise, with while loops.
Oh good, we got some varying responses here, perfect.
So the code is, "You're in the Lost Forest.
Go left or right?"
OK.
So here, the while loop is checking if the input is equal to this particular string.
OK. And if it is, type the same thing again and ask them for more input.
So my question was, what happens when you type in "R-i-g-h-t?"
And I think the majority of the class is getting it right.
And maybe the people who answered this are maybe changing their answer to something else, but you're right.
So you ask the question again, "Go left or right?"
And that's because Python, again, is very, very particular.
We're telling it that the user input must match this string exactly.
So "R-i-g-h-t" is going to-- even though it says right, it doesn't match up.
So if you wanted-- I'll just show you in this example here-- if you wanted to take care of that case, then you'd add this little-- you'd expand on this line right here, which says, "while n = = 'right' or n = = to "R-i-g-h-t" or whatever other cases you'd want to take care of.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So now we have this last exercise.
I'm creating a mysum.
I'm going to go through all of these values and you guys have already told me what values these are, 5, 7, 9, not 11.
And I'm going to add to mysum.
So I'm going to keep a running sum adding all of these values together.
So the very first time through the loop, mysum gets the value of 5.
That's this line here.
The next thing I see inside this loop is an if statement.
If mysum is equal to 5.
That's true.
So I'm going to go inside this if statement.
The next thing I see is a break.
If I see this break am I evaluating-- am I evaluating this line or not?
No, exactly.
Because whenever Python sees break, it's going to say, I'm going to stop right here, exit out of the loop that I'm currently in, and go to the statement that's immediately right after it.
So what this is going to print is mysum.
And that's the exercise we were looking at.
Let's see how the class did.
Pretty good.
Hopefully if you answered one of these, the explanation was-- was all right.
If not, just go back and try to work through it step by step.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right everyone, let's get started.
So good afternoon.
So this is the 3rd lecture of 6.0001 and 600.
As always, please download slides and code to follow along.
So a quick recap of what we did last time.
Last time, we talked about strings as a new object type, as sequences of characters.
And then we introduced two new concepts that allowed us to write slightly more complicated programs.
So we introduced branching, with these keywords, if, elif, else.
And branching allowed us to write programs that, us, as programmers, could introduce decisions into our programs.
And then we introduced two different kinds of loops, while loops and for loops.
And those also added a little bit of complexity to our programs.
Today, we're going to talk a little bit more about strings.
So we're going to see a couple of more operations that you can do on strings and string objects.
And then we're going to talk about three different algorithms, a guess and check algorithm, an approximate solution algorithm, and a bisection method algorithm.
So let's dive right in.
We'll talk a little bit about strings, first.
So strings, we thought of them as sequences of characters, case sensitive, as we saw in programs we wrote last lecture.
And strings are objects.
And we can do all of these operations on string objects, like test if they're equal, less than, greater than, and so on.
It turns out, we can do more than just concatenate two strings together or do these little tests on them.
So we're going to start introducing the idea of a function or a procedure.
And we're going to see more about functions and how you can write your own functions next lecture.
But for today, you can think of a function as sort of a procedure that does something for you.
Someone already wrote this.
So the first one we're going to look at is a pretty popular function.
And when applied on a string, this function, called len, will tell you the length of a string.
So that's going to tell you how many characters are in the string.
And characters are going to be letters, digits, special characters, spaces, and so on.
So it's just going to count how many characters are in a string.
So if I have the string s is equal to "abc"-- remember a string is in quotation marks-- then, if I do this, if I write this expression, len s, here, since it's an expression, it has a value.
So it evaluates to a certain value.
And by definition, it's going to tell me what the length of the string, which is 3 characters long.
Another thing that we can do on strings is, since they're a sequence of characters, I might want to get what character is at a certain position.
So we do this using this fancy word called indexing.
But pretty much what indexing into a string means is you're going to tell Python, I want to know the character, at this certain position or at this certain index, inside my sting.
So once again, let's use this string, s is equal to "abc."
And let's index into it.
So in computer science, we start from 0, counting by convention.
Notice, we had a problem set 0 in this class.
Python is no different.
So in Python, you start indexing at position 0.
Or you start indexing at 0.
So the first character, in your string, we say is at position 0 or at index 0.
The next character in the string is at index 1.
And the next character in the string is at index 2.
In Python, it turns out, you can also use negative numbers to index.
And if you index into the string with negative 1, for example, that means that you want the last character in the string.
So the last character in your string is always going to be at position negative 1, the second-to-last character is at negative 2, third-to-last character is at negative 3, and so on and so on.
So the way you index into a string is with these square brackets, here.
And this is the notation.
So if I want the character at position 0 or at index 0, I say s, which is the string I want to index into.
And then, inside the square brackets, I say what index I want.
So s at index 0 is going to be the value "a."
s at index 1 is going to be the value "b," and so on and so on.
And we can also do negative indexing, as well.
I added this in here.
If you do try to index into a string beyond the limits of the string-- and we can even try this out, just to show you that it's not the end of the world if we do that.
If we have s is equal to "abc," and we have s at position 20, for example, obviously, my string is only length 3, so what's at position 20?
I get an error.
I call this angry text, here, in Python.
But really, the most relevant thing to note is these last couple of lines here.
This tells you what line is problematic.
So s at position 20 has an issue.
And this last line here tells me what actual error I have.
So it's an index error, which means I'm trying to index too far into the string, because it only has three characters.
So it's nice to be able to get a single character out of my string.
But sometimes, I might want to get a substring.
So I want to start at the first character and go halfway into the string, or I want to take a few characters in between, or I want to skip every other letter or something like that in my string.
So if I want to do this slightly more complicated interaction with strings, we call that slicing, slicing into a string.
And this notation here should seem a little bit familiar, because we saw it last lecture when we did it with range.
We had a start, stop, and a step.
The notation was a little bit different, because, in range, we had open-close parentheses and commas in between.
But except for that, this sort of works the same.
The start is the index, starting from 0, from where you want to slice into this string.
The stop is the stop index.
So you're going to go up until stop minus 1 and take that index.
And then the step is how many letters you wish to take.
So this is the full notation here.
But sometimes, you can not give it a third sort of number in here.
So if you only give it two numbers, then, to Python, that represents just the start and the stop.
And by default, step is going to be 1.
And there's a lot of other things you can do with strings.
You can omit numbers and just leave colons in Python.
By definition, the way that whoever wrote slicing had decided, if you omit numbers, then it's going to be equivalent to these things here.
So we slice using square brackets, just like indexing.
Except now, we can give it two numbers.
So with this string, s, if we slice into the string s, we start from index 3 and go up until index 6.
So if we have abcdefgh, this is position 0, 1, 2, 3, 4, 5, 6, 7.
And you just count.
So s, starting from 3 and going till 6, is going to start here, 3.
So it's going to come up with-- sorry d. And then we're going to take e. And then we're going to take f. And since we're going until stop minus 1, we're not going to take g. Because this is position 6, and we're going until 6 minus 1.
The next one here, 3, 6, 2 is going every other one.
So we start at 3, and then we skip every other one, so we go d but not e, and then f, and then stop.
If you do s and then nothing inside except colons, notice that you're going to have s, and then nothing, and then colon, nothing, colon, nothing.
So nothing for start, nothing for stop, nothing for step.
And that's just going to value it to the string, itself.
It's the same as 0 to the length s going every step.
This one might actually be useful.
It reverses the string automatically for you.
So with this one little line here, you can get the inverse of your string.
And that's equivalent to that.
So the minus 1 represents starting from the end and going back every letter.
And then this one's a little bit more complicated but also not too bad.
So as we're doing these string slices, again, if you're unsure what something does, just type it into Spider.
And you might be surprised.
You might not be.
But it's a good way to check yourself, to make sure you're understanding what's happening.
One thing I want to mention, and it's good to keep this in the back of your mind.
We're going to come back to this as we start talking about slightly more complicated object types.
But strings are immutable.
So just keep this word in the back of your mind as we go through this class.
And what I mean by this is that an actual string object, once it's created, cannot be modified.
This might not mean anything right now.
But let me just draw a little something.
Let's say I have this string, s is equal to hello.
Remember, in the first lecture, we drew a diagram sort of like this.
This is my memory.
I have this object "hello."
And this object, "hello" is bound to this variable s. So now I can access the object "hello" using this variable s. Now you might think, well, since I could index into a string, I might be able to just say something like, s at position 0 is equal to y.
And that will just change the little h into a y, and I'll have a new object.
Well strings are immutable, which means, in Python, you're not actually allowed to do this.
And it gives you an error if you do try to do that.
If you want the variable s to point to the string, Y-E-L-L-O, you could just say s is equal to Y-E-L-L-O.
Or you could do string operations like this.
And this takes the y and it concatenates it to the string s, all of the elements starting from position 1, which is e, l, l, o.
So this makes Y-E-L-L-O.
Now internally, what happens when I write this line is Python says, OK, I'm going to break my bond with this original object "hello."
I'm going to bind my string variable s to the new object "yello."
and this other, old object still is in memory somewhere.
But it's an entirely different object that I've created here.
Again, it might not mean anything right now, but just keep this in the back of your mind, strings are immutable.
So the next thing I want to talk about is a little bit of recap on for loops.
And we're going to see how we can apply for loops, very easily, to write very nice, readable code when dealing with strings.
So remember that for loops had a loop variable.
My loop variable being this var, here, in this particular case.
It can be anything you want.
And this variable, in this particular case, iterates over this sequence of numbers, 0, 1, 2, 3, 4.
So the very first time through the loop, var has a value of 0.
It does the expressions in the loop.
As soon as they're done, var takes the value 1.
It does all the expressions in the loop.
And then var takes the value 2, and it does that all the way up until 0, 1, 2.
And the last time it goes around is with var is equal to 3.
And remember, we said that we can customize our range in order to start from a custom value to end at a different value and to skip certain numbers.
So, so far, we've only been using for loops over a sequence of numbers.
But actually, for loops are a lot more powerful than that.
You can use them to iterate over any sequence of values not just numbers but also strings.
So here are two pieces of code, this one and this one here.
These two pieces of code both do the exact same thing.
To me, possibly to you, this one looks a lot more readable than this one, just at a first glance.
If I were to read this one, just using the keywords and variables here, it would sound like broken English.
But you could decipher what I'm trying to say.
For a char in a string s, if the char is equal to "i" or a char is equal to "u," print "There is an i or a u."
It sounds weird, but you could probably tell what I was trying to do here.
Whereas up here, it's a little more complicated to tell what I'm doing.
You have to sort of think about it a little bit.
For some index in this range of numbers, 0 through the length of the string s, if s, at position index, is an "i" or s at position index is a "u" print, "There is an i or a u."
Both of these codes just go through the string s. And if it encounters a letter that's an i or a u, it's just going to print out this string here.
But this bottom one is a lot more pythonic.
It's an actual word created by the Python community.
And it just looks pretty, right?
You can tell what this code's supposed to do.
Whereas this one is a little bit harder to decipher.
So that's sort of an illustration of a for loop over a sequence of characters.
So char is going to be a loop variable, still.
And the loop variable, instead of iterating over a set of numbers, it's going to iterate over every character in s, directly.
And char is going to be a character.
It's going to be a letter.
So here's a more complicated example.
I wrote this code a couple of years ago.
And it was my attempt at creating robot cheerleaders , because I needed some motivation.
And then I googled, last night, "robot cheerleaders," and was not disappointed.
Created this GIF.
It looks pretty cool.
And it looks like they kind of stole my idea.
But that's fine.
So let's look at what this code's supposed to do.
I'm going to run it.
I'm going to run it, and then we'll go through it.
All right, it prints out, "I will cheer for you!
Enter a word."
You know what, I like robots, so I'll put in "ROBOTS."
How enthusiastic am I about robots?
Let's say 6.
So what this is going to print is-- it's a cheerleader, right?
"Give me an r, r." "Give me an o, o."
"Give me a b, b," and so on and so on.
"What does that spell?
ROBOTS."
And it's going to print it 6 times, because I'm 6 out of 10 enthusiastic about robots.
So that's pretty much what that code's supposed to do.
And you can write it using what we've learned so far.
Now let's go through it a little bit.
And I'm going to show you just how easy it is to convert this code using a for loop over characters.
Right now, what it does is it asks the user for input, so a word and a number.
And then it does this thing, here, right?
First, it uses a while loop.
And second, it uses indexing.
And what tips you off that it's using indexing is it's using the square bracket, here, into the word.
And obviously, it's using a while loop.
And it has to first create a counter, initialize it.
And then, down here, it's going to increment it inside the while loop.
If you remember, that's sort of what we need to do for while loops.
So it's going to start at 0, and it's just basically going to go through index i is equal to 0, 1, 2, 3 4, which is going to go all the way to the end of the word, whatever the user typed in, in this case "ROBOTS."
It's going to get the character at that position.
word at position i is going to be a character.
This line here is just for the cheerleading to make sense.
It's just to take care of letters that make sense to use an, right?
So give me a b, give me an b.
So give me an b does not make sense, right?
So that's just taking care of that.
And I'm using this in keyword to check whether the character-- so the character, r, for example, in robots-- is inside an letters.
And an letters I've defined up here, which is these are all the letters that make sense to put an an before the letter.
So give me an r for example, here, on the right.
And so if it makes sense to use an before the letter, use that, and otherwise use just an a.
And after I'm done, I say, "What does that spell?"
And then it's just a for loop that goes times many times and prints out the word and the exclamation mark.
So this code might have been a little bit more intuitive if I rewrote it or if I'd originally written it with a for loop.
So this part here, the while loop and indexing and creating my original counter, we can get rid of that.
And we can replace it with this, for char in word.
I'm originally using char, so I can use char as my loop variable again.
And simply, I'm just going to iterate over the word, itself.
So now, instead of having this mess here, I have a one-liner that says, for every character in my word, do all this stuff here.
So that remains the same.
And then I don't even need to increment a counter variable, because I'm not using while loops anymore.
I'm just using a for loop.
So the code becomes-- delete that-- for char in word.
And then delete that.
And that does the exact same thing.
And it's a lot more readable.
So this was our toolbox at the beginning of this course.
We are two and half, I guess, lectures in.
These are the things we've added to it.
We know integer, floats, Booleans.
We know a bit of string manipulation, math operations.
We added, recently, these conditionals and branching to write slightly more interesting programs.
And now we have loops, for and while loops to add interesting and more complicated programs.
So with these, the second part of this lecture is going to be looking at three different algorithms.
That's the sort of computer science part of this class, Introduction to Computer Science and Programming using Python.
Don't let the word algorithm scare you.
They're not that complicated.
You just have to sort of think a little bit about them.
And you'll be able to get them.
So we're going to look at three algorithms, all in the context of solving one problem, which is finding the cube root.
The first algorithm is guess and check, then we're going to look at an approximation algorithm, and then a bisection search.
So the first is the guess and check method.
You might have done this, in math, in high school.
The guess and check method is also sometimes called exhaustive enumeration.
And you'll see why.
So given a problem, let's say, find the cube root of a number, let's say you can guess a starting value for a solution.
The guess and check method works if you're able to check if your solution is correct.
So if your guess is originally 0, you can say, is 0 cubed equal to the cube of whatever I'm trying to find the cube root of?
So if I'm trying to find the cube root of 8, is 0 cubed equal to 8?
No.
So the solution is not correct.
If it's not correct, guess another value.
Do it systematically until you find a solution or you've guessed all the possible values, you've exhausted all of your search space.
So here's a very simple guess and check code that finds the cube root of a number.
So I'm trying to find the cube root of 8.
So my cube is 8.
I'm going to have a for loop that says, I'm going to start from 0.
And I'm going to go all the way up to-- So I'm going to start from 0 and go all the way up to 8.
For every one of these numbers, I'm going to say, is my guess to the power of 3 equal to the cube 8?
And if it is, I'm going to print out this message.
Pretty simple, however, this code is not very user friendly, right?
If the user wants to find the cube root of 9, they're not going to get any output, because we never print anything in the case of the guess not being a perfect cube.
or the cube not being a perfect cube.
So we can modify the code a little bit to add two extra features.
The first is we're going to be able to deal with negative cubes, which is kind of cool.
And the second is we're going to tell the user, if the cube is not a perfect cube, hey, this cube is not a perfect cube.
So we're not going to silently just fail, because then the user has some sort of feedback on their input.
So let's step through this code.
We have, first of all, a for loop just like before.
And we're going to go through 0 to 8 in this case.
We're using the absolute value, because we might want to find the cube root of negative numbers.
First thing we're doing is doing this check here.
Instead of guessing whether the guess to the power of 3 is equal to the cube, we're going to check if it's greater or equal to, and we're going to do that for the following reason.
So if we're trying to find the cube root of 8, for example, versus a cube root of 9-- this is 8 and this is 9-- what is this code going to do?
It's going to first guess 0.
0 cubed is not greater or equal to 8.
1 cubed is not greater or equal to 8.
2 cubed is greater or equal to 8, so here, once we've guessed 2, we're going to break.
Because we found a number that works.
And there's no need to keep looking.
Once we've found the cubed root of this number 8, there's no need to keep searching the remainder, 3, 4, 5, 6, 7, 8.
Sort of the same idea when we're trying to find the cube root of 9.
We're going to start with 0.
0 to the power of 3 is less than 9.
1 to the power of 3 is less 9.
2 to the power of 3 is less than 9.
When we get to 3 to the power of 3, that's going to be greater than 9.
So this code tells us, once we've picked a number that's beyond the reasonable number of our cubed root, of our cube, the cubed root of our cube, then we should stop.
Because, again, it doesn't make sense to keep searching.
Because if 3 to the power of 3 is already greater than 9, 4 to the power of 3 is also going to be greater than 9 and so on.
So once we break here, we either have guess being 2 or guess being 3 depending on what cube we're trying to find.
And if the guess to the power or 3 is not equal to the cube, then, obviously, the cube was not a perfect cube.
So that's this case here, if we were looking at at the cube root of 9.
And otherwise, this part here just looks at whether we should make it a positive or a negative cube.
So if our original cube was less than 0, then, obviously, the cube root of a negative number is going to be a negative number, and, otherwise, it's just our guess.
So that's the guess and check method, slightly more feature-rich program for guessing the cube root.
But that only tells us the cube root of perfect cubes and doesn't really give us anything else, any more information.
So sometimes, you might want to say, well, I don't care that 9 is not a perfect cube, just give me a close enough answer.
So that's where approximate solutions come in.
So this is where we're OK with having a good enough solution.
So in order to do that, we're going to start with a guess and then increment that guess by some small value.
Start from 0 and start incrementing by 0.001 and just go upwards from there.
And at some point, you might find a good enough solution.
In this program, we're going to keep guessing as long as we're not close enough.
And close enough is going to be given by this epsilon value in the program.
So as long as the guess cubed minus the cube-- so how far away are we from the actual answer-- is greater than some epsilon, keep guessing, because the solution is not good enough.
But once this is less than epsilon, then we've reached a good enough solution.
So two things to note with approximate solutions.
So you can get more accurate answers if your step size is really, really small.
If you're incrementing by 0.0001, you're going to get a really good approximate solution, but your program will be a lot slower.
Same sort of idea with epsilon, you can change epsilon.
If you change epsilon to be a bigger epsilon, you're sacrificing accuracy, but you're going to reach a solution a lot faster.
So here's the code for the approximate solution of a cube root.
It might look intimidating, but, look, almost half this code is just initializing variables.
So we're initializing, this is the cube we want to find the cube root of.
We pick an epsilon of this.
We start with a guess of 0.
We start with an increment of 0.0001.
And just for fun, let's keep track of the number of guesses that it takes us to get to the answer.
This is similar to the guess and check from before.
It's not similar.
Well this part is similar to the guess and check from before.
So we're going to take the guess to the power of 3 minus the cube, right?
So that's how far away are we from the actual answer?
And we're going to say, if that's not good enough-- so if we're still greater than or equal to the epsilon-- then keep guessing.
So we're going to be stuck in this loop, where we keep guessing values, until we've reached a guess that's good enough, so until we're less than epsilon.
And way we keep guessing is just with this line, right here, which says, increment my guess by increment, and increment being this really small value.
That make sense?
So I'm going to keep incrementing my guess by that small value.
Before I go on, I'm going to run the code.
And we're going to discover a small issue with it.
So with 27, we're going to run it.
Perfect, it took me 300 guesses.
But 2.99999 is close to the cube root of 27.
We can find the cube root of this guy here.
And it took me 20,000 guesses, but I figured out that 200.99999, so 201, is close to the cube root of that large number.
I should have done this.
This is going to be a giveaway, you guys.
Sorry.
Then we're going to have-- let's say I want to try cube of 10,000.
So 10,000 is not a perfect cube.
So we can run the code.
And with 8,120,601 I had already gotten an answer.
But with 10,000, I'm not getting an answer yet, right?
So I'm thinking that there might be something wrong.
So I'm going to stop my code.
So I just hit Control C, because I feel like I've entered an infinite loop.
And, in fact, I have.
So what ended up happening is this problem here.
So I'm going to draw something.
According to the code, I'm going to start from 0, and I'm going to increment my guesses, like that.
With every little increment, I'm going to make a new guess.
I'm going to take that guess to the power of 3.
I'm going to subtract the cube, and I'm going to figure out if I'm less than epsilon.
This is the epsilon that I want to be in, this little bit here.
So with every new guess, I might be, maybe-- so this is where I want to be, within this little boundary here.
With every new guess, I might be here.
With the next guess, over here, I might be here.
When I make another guess, I might be here.
So I'm getting close to being within epsilon.
But maybe with my next guess, I'm going to hop over my epsilon and have made too big of a guess.
So just because of the way the numbers were chosen in this example, just to illustrate this, using an increment of 0.01, a with finding the cube of 10,000 and epsilon of 0.1, it turns out that, as I'm doing all these calculations, I'm going to skip over this perfect sort of epsilon difference.
So first, I'm going to be too small.
And then I'm going to be too large.
And once I've become too large or too far away from epsilon, the guesses I continue to make are just going to be even farther away from epsilon.
And I'm not going to get to my answer.
And that's why I've reached an infinite loop in this code.
All I'm doing in this code is checking whether my guess cube minus cube is less than epsilon.
The only thing I need to do here is sort of add this little clause, here, that says, oh, by the way, also check that I'm less than cube.
Because this is just like we did in the very first program, when I'm checking 0, 1, 2, 3, 4, 5, 6, 7, 8, when I'm trying to find the cube root of 8.
Once I've reached 8, I'm going to stop.
And it's the same thing here.
So I just added this little clause that says, well, while I'm greater than or equal to epsilon and I'm still less than the actual cube, just keep searching.
But once I've reached the cube, then stop searching.
And with 10,000, you can see that I failed to actually find-- so that's what this part, here, does.
It tells me I've failed to find the cube root with those particular parameters.
The last thing we're going to look at is bisection search.
And to illustrate this, I'm going to need one volunteer.
And you're going to play a game with me in front of the whole class.
And there will be a prize.
There go the hands.
In the blue shirt, right there.
Cool.
So the prize is going to be, once again, this.
I promise I don't have millions of these, Google glasses.
I also don't work for Google.
I just happened to get a couple.
So the game is this.
I'm going to ask you to pick a number, a whole number, between 0 and 100.
And I'm going to try to guess it.
And you need to make it hard for me.
And you need to make it so hard for me that I cannot guess it within 10 guesses.
And if you can do that, if I cannot guess it within 10 guesses, you get this.
And I'm going to draw out what I do as we go along.
So do you have your number?
Yes?
Yeah
Perfect.
Let me erase that.
Actually, I should've probably kept that, because I'll still use it.
There's the numbers, 0 to 100.
Is your number 50?
No
50 Was my guess.
So I've made one guess.
Is your number higher or lower than 50?
Higher
Higher.
Is your number-- my next guess is going to be 75.
And the reason I'm guessing 75 is because-- what's your name?
Sophie
What's that?
Sophie
Sophie.
Sophie said, 50 was too low.
So I immediately know that it cannot be any less than 50.
So I've already eliminated half of the numbers.
So my next guess is 75.
Is your number 75?
Is your number lower or higher?
Higher
Since it's higher, I'm eliminating this half here.
Is your number-- so between 75 and 100.
Oh, boy, you're putting me on the spot.
What's that?
87
87, thank you.
87?
No
Higher or lower?
Lower
Lower.
So since it's lower, I'm eliminating that half.
Is your number 81?
Higher or lower?
Lower
So she said, lower, so I'm eliminating that half.
Is your number 78?
Oh, boy, that's really hard.
78, OK. Higher or lower?
Lower
Is your number 76?
Yeah
Yay.
All right, perfect, 76 was the number.
So how many guesses have I made?
One, two, three, four, five, six-- I made six guesses.
So I did get it under 10.
But you know what?
The game was rigged.
So you get the prize anyway, just because I rigged the game.
Here you go.
Pass it down
Thank you
Thank you.
So notice, in bisection search, what I did was I eliminated half the search space with every guess.
I said, well, she said it's higher or lower, so I definitely cannot be in the other search space, right?
If I was doing approximate solution or, in this case, guess and check, I would be asking Sophie, is your number 0, 1, 2, 3, 4, and so on?
So with guess and check, it would have taken me 76 guesses to get to the number, whereas, with this bisection search, that I just did, it only took me 6.
Isn't that cool?
So that means that the larger the space actually is, that I need to search, the better it is to use bisection search, this bisection search method.
So that's basically what I'm illustrating here.
So we have our original search space.
We're going to choose a guess halfway, eliminate half of the guesses.
Then we're going look in the remaining interval, eliminate half the guesses, and so on and so on.
So then this is the code for bisection search.
Also looks intimidating, but it's not so bad.
So we're initializing a bunch of stuff up here.
The most important couple of things we're initializing are, first of all, this high and this low boundaries.
So with the guessing game, the low boundary was 0, and the high boundary was 100.
When we're looking at the cube root, the low boundary is going to be 0, and the high boundary is going to be just my cube, because a guess to the power of 3 cannot be any greater than cube.
And then, I'm just going to do the same procedure that I did with the guessing game, which is I'm going to make my guess, be halfway in between.
So with this guessing game, I had to sort of choose, if there were four numbers in between, should I go higher or lower?
Well, when we're doing by bisection search, here, we don't care about that.
We're just going to do floating point division, because we want decimal numbers.
So I have a low boundary and a high boundary.
And I figured out my halfway point.
Then I have this while loop here.
A while loop is similar to the approximation method, where, as long as I don't have a guest that's good enough-- so this, depicted by this greater or equal to epsilon-- as long as my guess is not good enough, I'm going to keep guessing.
That's what this while loop is saying.
So if the guess to the power of 3 minus cube is not good enough, keep guessing.
And the way I keep guessing is this part, here, says, my guess was too low.
So if my guess was too low, set the low boundary to be the guess.
Because I don't care about all of the other numbers that are much lower than me.
So set the low to be the guess.
That's what that line is doing.
And otherwise, my guess was too high.
That's what this else is doing.
So set the high to be the guess, because I don't care about numbers any higher than my guess.
Once I have these new boundaries, I make another guess, again, halfway in between the new boundary points.
So essentially, I'm just halving my interval with every single guess.
And that's what the while loop is doing.
And then I print out the remaining part.
So notice the search space originally being N, we're halving it with each guess.
So the first guess divides it by 2, the second guess divides it by 4, and so on.
So by the time we get to the k-th guess, N/2k, the k-th guess, let's say that's the actual answer we're interested in.
There's only one value in that little interval.
And that's the answer we want.
So 2 to the k is equal to N. And then how many guesses did we make?
k is equal to log base 2 of N. So when we are playing the guessing game of 100, my end was 100.
Log base 2 of 100 is 6.-something, I think.
So in fact, I could have said, if I don't guess it within seven guesses, you would have won as well.
So that's why the game was rigged.
So the guess, notice, it converges on the order of log base N instead of just linearly in terms of N. So that's why it's so powerful.
One last thing I want to mention is the code I showed only works for positive cubes.
And that's because of the following.
So I have this 0 and 1.
Let's say I'm trying to find the cube root of 0.5.
When I first set my initial boundaries, my low is this one, and my high is this one.
But what's the cube root of 0.5?
Is it within this boundary or is it outside this boundary?
Outside the boundary
I heard, outside.
It's like 0.7 something.
So it's out here.
So with this particular code, I'm going to be halving my interval in between those numbers, but I'll never get to an answer.
Because the actual cube root of 0.5, or numbers less than 1, is going to be outside that boundary.
So there's a small change you can make to the program, which will fix that.
And that's in the code.
I didn't put it in, but it's a very small change, a small if statement.
So that's it.
All right, thank you.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So the question is, we have the string s is the string 600 is 6 triple 1 and 6 triple 2.
So we're going to create the string here.
It's initially an empty string.
I want to mention that the plus equals-- for example, if I say a plus equals 1.
This is equivalent to a is equal to a plus 1.
We've seen this a couple of times before-- last lecture and the last lecture before that.
So all this means here, this line here, is we're going to take the previous string that we have, and we're just going to add it to itself, plus the last letter.
So the first thing we're going to have is going to be new_str is equal to 2.
Then this line here is going to say now I'm going to add to the two that I already have, the element at position 0, which is a 6.
So I'm going to have 6.
Then I'm going to add to that, the element at index 4.
So I'm starting count from 0.
So this is 0, 1, 2, 3, 4.
So that's a space.
And I'm going to go-- notice I have 4, and then nothing in the middle.
So my stop is going to be length s by default.
And I'm going to take every 30 characters.
Now every 30 is longer than the actual string I have.
So we're just going to end at the end of the string.
So the next thing that we're going to add is a space.
And then the last thing we're doing here is we're starting at 13.
So now I'm going to have to count 13-- 0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13.
That's the 1.
I'm going to go backward, because of this minus 1.
And I'm going to go backward until 10, but right before 10-- so 1 less than 10.
So I'm going to do 100.
So this one, this 0, and then this 0.
Then I'm going to print this new string.
I think this is right.
If you paste it into Spider, it should give you 26 space 100.
So I think most of the class got it right.
I think, yes.
Perfect.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We have two strings, s1, s2.
First, we're going to check if the length of s1 is equal length 2.
Anyone, really quickly can you tell me what is the length of s1 and s2?
How many characters are in there?
10 right, because we're including spaces.
So 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, so they are equal, first of all.
So we're going to go inside this if statement.
Then we're going to go for every character in s1.
And then we're going to have a nested loop.
For every character in s2, we're going to check if the two characters are the same, and if they are, we're going to print this common letter statement and then we're going to break.
OK. Great we have some good competition here.
So let's see.
The outer for loop is first going to go through the letter M. So this is-- so first it's going to look at M, right.
For char in s1, the first character in s1 is an M. Then we're entering inside this other for loop.
For char 2 in s2.
So I'm going to say for this M, I'm going to look at I, which is the first character in s2.
If the character 1 is equal to character 2, they're not, don't do anything.
Keep going to the next character in s2.
So that-- you don't do anything there.
The next character in s2 is a space, M does not equal a space so you don't do anything.
R does not-- the next character is an R, don't do anything.
The next character is a U, don't do anything.
Then an L, don't do anything.
Then an E, don't do anything.
Space.
And then suddenly we get to an M. So we're still in the outer for loop, we're still looking at M, and we're comparing M with every other letter inside the s2.
And suddenly we reach here where we're comparing M with an M and when we say yes, they are equal.
So we're going to print out something.
Make that into a better check.
We're going to print common letter and then we're going to break.
This break says do not keep looking at the remaining letters in s2.
So once we've found one that fits-- once we found one the matches, stop right here, break, and print out the thing, and then go on to looking at I, which is the next letter in s1.
And you do the same thing, compare I with I, we have a match.
Perfect, right off the bat.
So you print out another common letter.
And then you break, which means don't keep looking at the remaining letters.
And then you look at-- since you broke out of the inner loop, you go to the outer loop.
The next letter is a T and so on and so on.
So essentially you're comparing every letter out here in s1 with every single letter in s2 until you find one that matches.
As soon as you do, you exit.
So if you actually run this program it's going to print out seven times, I think.
We can check it.
1, 2, 3, 4, 5, 6, 7 times.
Nice.
The majority of the people are right.
If you got this wrong, please go back and try to trace through your program.
That means step by step try to figure out what values of variables are just to make sure that you're on the right track and you understand exactly what's happening.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right everyone let's get started.
All right good afternoon on this rainy, rainy sad afternoon.
So-- I'm glad we're inside though-- all right so Lecture 4 of 6.0001 in 600.
Quick, quick recap of what we did last time.
So last time we did a little bit more string manipulations, and then we saw how you can use for loops over strings directly.
So instead of having for loops that iterate over range-- so 0, 1, 2, 3, 4, and so on-- you saw that it was more powerful to sometimes use for loops that iterate over string objects directly.
So that was the first half of the lecture.
In the second half, we started looking at different ways that you can implement the different implementations to the same problem.
So we saw the problem of finding the cube root, and we saw some implementations.
We saw the Guess and Check method, and the approximation method.
And then we looked at what I thought was the most powerful method, which was the bisection method.
And this one, if you remember, I played a game with someone in the audience where I guessed a number between 0 and 100.
And we saw that I was able to guess that number really, really quickly using the bisection method.
And that's the method that you're going to implement-- that you are currently implementing-- in your problem set.
OK so today-- so that sort of finishes introduction to some of the more basic mechanisms in Python.
And today we're going to talk about how to structure your programs such that you write nice, coherent code-- reusable code-- by hiding away some of the details in your code.
And to do that we're going to look at these things called functions.
All right so just stepping back and sort of getting a high-level view of how we write the code so far.
So so far the way that you've been writing code for your programs is you open a file, you type some code to solve a particular problem given, like in your problem sets, each file contains some piece of code, you have sequences of instructions that contain maybe assignments, loops, conditionals, and so on and so on.
But really you have one file that contains each code and you write everything in that particular file.
But this is OK for smaller problems that we've been seeing so far, but when you're starting to write large pieces of code it's going to get really messy, really quickly.
So think about if you want to use a for loop in one part of your code, and you find it useful to use that same for loop in another part of your code.
Some point in the future as you're debugging your code, you might want to change your original for loop, you have to figure out all the other places where you've used that type of for loop for example.
So as you're scaling your code, you'll find it harder to keep track of these details.
So this is where functions will come into play in today's lecture-- will help you out.
So if you want to be considered a good programmer, a good programming style would be to not necessarily add lots and lots of lines of code, but really to add more functionality to your programs.
So how many different things-- how many different features-- can your program do, rather than how long can your code be.
And that'll help you later on look at your code if you need it for a future class, and it'll help others if they want to look at your code later on if they find it useful.
So today we're introducing this idea of functions.
And functions are mechanisms to achieve decomposition and abstraction.
So these are two key words here that are going to pop up in today's lecture and also in future lectures.
So before I introduce decomposition and abstraction in the context of functions, let's first take a look at just sort of a real-life example.
So let's take a projector.
I'm using one right now.
Quick show of hands.
If I give you all of the electronic components that are part of a projector-- resistors, a fan, a light bulb, a lens, the casing, all of the different parts in it.
Who here would be able to build a projector?
Do I see a hand?
No?
Ooh oh yeah nice!
You can also lie.
I won't know the difference.
But if you can do that, I'd be very impressed.
All right so you can't really put together a projector right?
Another show of hands.
If I gave you a projector that's fully assembled and I gave you a computer, for example, who would be able to maybe figure out within let's say an hour how to make them work together?
Good, a fair bit of the class.
That's perfect.
That's exactly the answers I was trying to get at here.
So none of us really know how a projector works-- the internals-- but a lot more of us know how to work a projector, just given maybe a set of basic instructions or just intuitively speaking.
So you see the projector as sort of a black box.
You don't need to know how it works in order to use it.
You know maybe what inputs it might take, what's it supposed to do at a high level.
Take whatever's on my screen and put it up on the large screen there, just magnify it, but you don't know how it does it-- how the components work together.
So that's the idea of abstraction.
You don't need to know how the projector works in order to use it.
OK that's abstraction.
The other half of that was decomposition.
So let's say that now, given a projector, I want to project a very, very large image down on a very large stage.
For example, this is from one of the Olympics.
It's a stage of what, like 10 football fields, something like that?
Something massive.
You could build one projector that's able to project a very large image, but that would be really expensive and you'd have to build this one projector that's used for this one time.
So instead what you could do is you can take a bunch of smaller projectors and feed different inputs to each one of them.
And as you're feeding different inputs, each one's going to show a different output.
And then you're going to be able to have all of these different projectors working together to solve this larger problem of projecting this really cool image on a very large stage.
So that's the idea of decomposition.
You take the same projector, feed it different inputs, it does the exact same thing behind the scenes, but it will produce a different output for each one of these different inputs.
So these different devices are going to work together to achieve the same common goal, and that's the idea of decomposition.
So these is where I apply to the problem of projecting large image, or a projector in general, but we can apply these exact same concepts to programming.
So decomposition is really just the problem of creating structure in your code.
In the projector example, we have separate devices working together.
In programming, to achieve decomposition you're dividing your code into smaller modules.
These are going to be self-contained, and you can think of them as sort of little mini-programs.
You feed in some input to them, they do a little task, and then they give you something back.
They go off and do their thing and then they give back a result.
These modules can be used to break up your code, and the important thing is that they're reusable.
So you write a module once-- a little piece of code that does something once-- you debug it once, and then you can reuse it many, many times in your code with different inputs.
Benefit of this is it keeps your code organized and it keeps your code coherent.
So functions are going to be used to achieve decomposition and to create structure in our code.
We're going to see functions today in this lecture, and in a few weeks, you're going to actually see-- when we talk about object oriented programming-- how you can achieve decomposition with classes.
And with classes you can create your own object types like adding some floats.
You can create your own object types for whatever you want, but that's later.
OK so decomposition is creating structure in your code.
And abstraction is the idea of suppressing details.
So in the projector example, remember, abstraction was you didn't need to know exactly how the projector worked in order to use it.
And it's going to be the same idea in programming.
So once you write a piece of code that does a little task, you don't need to rewrite that piece of code many times.
You've written it once, and you write this thing called a function specification for it, or a docstring.
And this is a piece of text that tells anyone else who would want to use it in the future-- other people, maybe yourself-- it tells them how to use this function.
What inputs does it take?
What's the type of the inputs?
What is the function supposed to do?
And what is the output that you're going to get out of it?
So they don't need to know exactly how you implemented the function.
They just need to know inputs, what it does, what's the output.
Those three things.
OK so these functions are then reusable chunks of code.
And we'll see in a few examples in today's lecture how to write some and how to call functions.
And as we're going through today's code, I want you to sort of think about functions with two different hats on.
The first hat is from someone who's writing the function.
So in the projector example, someone had to build the first projector.
Someone had to know how to put all these components together.
So that's going to be you writing a function, so you need to know how to make the function work.
And then the other hat is you as someone-- as a programmer-- who is just using the function.
You're assuming it's already been implemented correctly, and now you're just using it to do something.
So these are some of the function characteristics and we'll see an example on the next slide.
So a function's going to have a name.
You have to call it something.
It's going to have some parameters.
These are the inputs to the function.
You can have 0 inputs or as many as you'd like.
Function should have a docstring.
This is how you achieve abstraction.
So it's optional, but highly recommended, and this is how you tell other people how to use your function.
Function has a body, which is the meat and potatoes of the function-- what it does.
And a function's going to return something.
It computes its thing and then it gives back-- spits back some answer.
OK here's an example of a function definition and a function call.
Function definition is up here.
I'll just draw it here.
This is the function definition up here.
And this is the function call down here.
So remember, someone has to write the function that does something to begin with.
So this is how you write the function.
The first is whoops-- the first is going to be this def keyword.
And def stands for-- it tells Python I'm going to define a function.
Next is the name of the function.
In this case, I'm calling the function is_even.
And the function name should really be something descriptive.
Whereas someone who is just using this function or looking at it can pretty much tell what it's supposed to do without going a lot farther than that.
They're just looking at the name.
And then in parentheses you give it any parameters, also known as arguments.
And these parameters are the inputs to the function.
And then you do colon.
OK so this is the first line of the function definition.
And after this, everything that's going to be part of the function is going to be indented.
The next part is going to be the docstring, or the specification, and this is how we achieve abstraction using functions.
Specification, or the docstring, starts with triple quotes and ends with triple quotes, and you can sort of think about this as a multi-line comment.
It's just going to be text that's going to be visible to whoever uses the function, and it should tell them the following things: What are the inputs to the function?
What is the function supposed to do generally?
And what is the function going to give back to whoever called it?
The next part is going to be the body of the function.
We'll talk about what's inside it in the next slide.
And that's it.
That's all for the function definition.
def blah, blah, blah, indented, everything inside the function.
So this is you writing the function definition.
Once the function definition's written, you can call the function.
And that's this part down here.
And here, when you call function, you just say its name, and then you give it parameters.
And you give it as many parameters as the function is expecting-- in this case, only one parameter.
So what's inside the function body?
You can put anything inside the function body.
You remember, think of a function as sort of a small procedure or a little mini-program that does something.
So you can do anything inside the function that you can do in the regular program-- print things, do mathematical operations, and so on.
The last line is the most important part of the function though.
And it's this return statement-- that's what we call it.
So it's a line of code that starts with return, which is a keyword.
And then it's going to be some value.
Notice this is an expression here-- i%2 == 0 is an expression that's going to evaluate to some value.
And as long as this part is something that evaluates some value, it can be anything you want.
And this line here return something tells Python, OK after you have finished executing everything inside the function, what value should I return?
And whoever called the function is going to get back that value, and the function call itself will be replaced by that value.
OK so let's look at an example.
I'm going to introduce the idea of scope now.
And scope just means-- is another word for environment.
So if I told you that you could think of functions as little mini-programs, the scope of a function is going to be a completely separate environment than the environment of the main program.
So as soon as you make a function call, behind the scenes what Python says is, OK I'm in the main program but I see a function call.
I'm going to step out of this main program.
I'm going to go off into this new environment.
I'm going to create entirely new set of variables that just exist within this environment.
I'm going to do some computations.
When I see the return, I'm going to take this one return value.
I'm going to exit that environment, and then I'm going to come back to the main program.
So as you're entering from one scope to another, you're sort of passing these values back and forth.
So when you're entering a scope, you're passing a variable back into the function.
And when the function's finished, you're passing a value back to whoever called it.
So once again, this top part is the function definition.
And any arguments for the function definition are called formal parameters.
And they're called formal parameters because notice they don't actually have a value yet.
In the function definition, you're sort of writing the function assuming that, in this case, x is going to have some value.
But you don't know what it is yet.
You only know what value x takes when you make a function call down here.
So this is your function definition, and then later on in your main program, you might define some variable x is equal to 3.
And then you make a function call.
f of x here is your function call.
And it says, OK I'm calling f with the value 3, because x takes the value 3, and then I'm going to map 3 into the function.
The values that are passed into the function call are called actual parameters, because they're going actually have a value.
So let's step through this program-- this small program-- and see what exactly happens behind the scenes in the scope.
And if you're just starting to program, I think it would be highly valuable if you take a piece of paper as you're doing some of these exercises and you write down something similar to what I'm going to go through here.
I think it'll help a lot, and you'll be able to see exactly step-by-step what variables take what values and which scope you're in.
So here we go.
When the program first starts, we're creating this global scope.
It's the main program scope.
In the main program scope, the first thing that Python is going to see is this part here-- def f of x and then some stuff inside.
This tells Python I have a function named x, but I don't care what's inside the code yet.
I don't care what's inside the function definition yet, because I haven't called the function yet.
So to Python it's just some code just sitting in the global scope.
So whenever you see def, you're just putting some code in there.
Then you go onto the next line-- x is equal to 3.
So in the global scope, you now have also a variable x is 3.
And then the next line-- z is equal to f of x is a function call.
As soon as you hit a function call, you create a new scope-- a new environment.
So we're temporarily leaving the global scope and sort of portaling into a new scope, where we're going to try to figure out what this function's going to do and what it's going to return.
So the first thing you do is you map the parameters.
So x here-- I'm calling f of x with 3-- so first thing I'm doing is I'm mapping every one of the parameters in the definition to their values.
So first thing I'm doing is x gets the value 3.
Next line here is x is equal to x plus 1.
So we're still inside the function call f, so x gets the value 4.
We're printing this and then we're returning x.
So in the scope of f, x is equal to 4, so we're returning that value back to whoever called it, which was this function call within the global scope.
So this part right here-- f of x, which was the function call-- gets replaced with 4.
So inside the main program, z is equal to 4.
And that's how we pass parameters into the function, and we got a parameter back from the function.
As soon as the function returns something, the scope that you were in for the function gets erased.
You forget about every variable that was created in there, delete that scope, and you're back to wherever you started calling it.
One warning though.
So what happens if there's no return statement?
I said that every function has to return something.
If you don't explicitly put a return statement, Python is going to add one for you.
You don't have to do this.
And it's going to actually have return None-- N-o-n-e. And None is the special type-- None is the value for a special type called NoneType, and it represents the absence of a value.
What's that?
Not a string.
Not a-- None is not a string.
None is not a string, exactly.
It's a special type.
OK so before we go on, I wanted to go through a small exercise in Spyder just to show you the difference that None and printing and returning makes.
So here are two functions that I wrote.
One is is_even_with_return.
That's its name, so pretty descriptive.
It's pretty much the same code we saw in the slides.
It just has this extra little print thing.
It gets the remainder when i is divided by 2.
And it returns whether the remainder is equal to 0.
So it'll either return a true or a false-- a Boolean.
OK so my function call is this: I'm saying is_even_with_return with a value 3.
When I make this function call, this 3 gets mapped into here-- this variable here-- so i is equal to 3.
I'm going to print with return, and then I'm going to say remainder is equal to 3 percent 2, which comes out to value 1, because there's a remainder 1.
And I'm going to return whether 1 is equal to 0, which is false.
So this line here returns false, but am I doing anything with the false?
Not really.
It's just sort of sitting in the code here.
So this gets evaluated to false.
I'm not printing it.
I'm not doing any operations with it.
It's just sitting there.
So it won't show up anywhere.
If I want the result to show up somewhere, then I have to print it.
So that's what this next line is doing.
So that one should be straightforward.
is_even_without_return's a little bit trickier, but not too bad.
I have print, without_return inside here, and then I'm going to get a remainder is equal to i percent 2.
And notice that I'm not-- I don't have any return.
So implicitly, Python's going to add a return None for me, like that.
You don't have to add it.
So when I make the function call here, it's going to do the same thing, except that return in this case is not going to be a Boolean.
It's going to be this special None.
So this is going to get evaluated to None.
Again I'm not printing it out.
It's just sitting there.
If I were to print out the result of that, you'd be printing out this value None, which if I run it, you'll see here it just prints it out right there.
So as you're doing your next p set, it's about functions and you're seeing these Nones popping out in some places.
Check to make sure that you've actually returned something, as opposed to just printed something inside the function like we did here.
All right so that's the difference.
And the last thing I want to mention about this is_even function is how useful it can be.
So notice this is the function as in the slides, and once you write the function once, you can use it many, many times in your code.
So here I'm using the function is_even to print the numbers between 0 and 19, including and whether the number is even or odd.
So notice this piece of code here, once I've written this function is_even, looks really, really nice right?
I have for all the numbers in this range if the number i is even, this is going to return a true or false for all the numbers 0, 1, 2, 3, 4.
If it's true, then I'm going to print out even, and otherwise I'm going to print out odd.
So if I run this, it's going to do this.
0 even, 1 odd, 2 even, and so on.
So notice using functions makes my code really nice looking.
If I wasn't using functions, I'd have to put these two lines somewhere inside here and it would look a little bit messier.
So I've said this maybe once or twice before: in Python everything is an object.
Might not have meant anything back then, but I think you're going to see what I mean using this particular example.
So if in Python everything's an object-- integers are objects, floats are objects, even functions are objects.
So as you can pass objects as parameters back and forth as function parameters, you can also pass other functions as parameters.
Let's see what this means.
So we have three function definitions here-- func_a, func_b, and func_c.
And then I have three lines of code here in my main program.
So I have one called a func_a, one called a func_b, and one call to func_c.
Let's trace through, just like in the previous example, and see what exactly happens.
First thing I create is my global scope.
And I have three function definitions.
Again I don't care what's in the code yet, because I haven't called the functions yet.
Python just knows there's these functions with these names that contain some code.
After these definitions, I come to this line here-- print func_a.
As soon as I make a function call, I'm going to create a new scope and I'm going to hop into there.
Inside func_a, I'm going to go and look at what func_a does.
It doesn't take in the parameters, it just prints out this message here.
And then it leaves; it's done.
There's no return, so we return None.
So func_a returns None to whoever called it, which was that line there, so that is going to be None.
Next line.
This one right here-- print 5 plus some function call.
Again I'm going to hop into func_b's scope and see what to do there.
So first I'm going to map my parameters.
So 2-- whoops-- 2 gets mapped to y.
So inside func_b's scope, y is going to get the value 2.
That's the very first thing I'm doing-- mapping all the parameters.
Then I'm going to print this thing here, and then I'm going to return y.
So inside func_b, y has the value 2, and I'm returning 2 back to whoever called me.
So this is the value 2 and I'm going to print 5 plus 2, which is 7.
Last one.
This is the trickiest.
Oop, that popped up.
If you think you've got it, try that exercise.
But otherwise follow along.
print func_c func_a.
So I see that I am going to enter func_c's scope.
So I'm going to look at what func_c does.
First thing I do is I'm mapping all the parameters.
Don't even worry about the fact that this is a function right now.
Just pretend it's x or something.
So you say func_a is going to get mapped to the variable z inside func_c.
So z is func_c.
Just mapping parameters from actual to formal.
Then what do we do inside func_c?
We print out inside func_c, and then we return z.
This is the cool part.
Inside func_c, z is func_a.
So if you replace z with func_a, this here becomes return func_a open close parentheses.
Look familiar?
We did that function call right there right?
So that's just another function call.
So with that being another function call, you're going to create another scope, and you're going to pop into that one.
So we're one, two, I guess two scopes deep, and we're trying to figure out where we're going.
So func_a's scope is going to be up here.
So what does func_a do?
It just prints out this, and it returns None.
So we're going to return None to whoever called us, which was func_c.
So this line here becomes return None.
And so this line here is going to return None to whoever called it, which was this line down here.
Oops, I didn't mean to cross that out.
So that line here is going to print None.
So if you just go step-by-step, it shouldn't be too bad to try to map what happens with variable names and formal parameters and actual parameters.
That's why I highly recommend pieces of paper and pens.
One last thing I want to mention about scope before we do another example.
So there are three sort of situations you might find yourself in.
The first one is probably the most typical, and this is when you define a function.
And it's using a variable named x in this case that's also defined outside of the function.
And that doesn't matter because of the idea of scopes.
So inside the global scope, you can have variables x.
When you're inside a different scope, you can have whatever variable names you want.
And when you're inside that scope, Python's going to use those variable names, so they don't interfere with each other at all.
So in this example, I've defined a variable x is equal to 1, and then I incremented, and that doesn't interfere with the fact that we have a variable x outside.
This one's a little bit trickier.
I define this function g, and all g does is access a variable x.
But notice inside g, I've never actually declared or initialized a variable x.
In this f, I said x is equal to 1.
But in here, I'm just sort of using x.
So this does not give you an error.
In fact it's OK for you to do this in Python.
Python says, OK I'm in this scope, but I don't have a variable named x, so let me just go into the scope of whoever called me.
So I'm going to just temporarily hop out of the scope and see is there variable x outside of me?
And it'll find this variable x here, and it's going to print out its values.
So that's OK.
This last example here is actually not allowed in Python-- similar to this one-- except that I'm trying to increment a value of x, but then I'm also trying to reassign it to the same value of x.
The problem with that is I never actually initialized x inside h. So if I said-- if inside h, I said x is equal to 1, and then I did x plus equals to 1, then it would be this example here-- f of y.
But I didn't do that.
I just tried to access x and then incremented and then tried to reassign it.
And that's actually not allowed in Python.
There is a way around it using global variables.
But it's actually frowned upon to use global variables, though global variables are part of the readings for this lecture.
And the reason why it's not a great idea to use global variables is because global variables sort of give you this loophole around scopes, so it allows you to write code that can become very messy.
So using global variables, you can be inside a function and then modify a variable that's defined outside of your function.
And that sort of defeats the purpose of functions and using them in writing these coherent modules that are separate.
That said, it might sometimes be useful to use global variables, as you'll see in a couple lectures from now.
OK cool.
So let's go on to the last scope example.
OK this slide is here, and notice I've bolded, underlined, and italicized the Python Tutor, because I find it extremely helpful.
So the Python Tutor-- as I've mentioned in one of the assignments-- it was actually developed by a grad student here, or post-grad student slash post-doc here.
And it allows you to go through Python, paste a code, go through it step-by-step.
Like with each iteration, it'll show you exactly what values each variable has, what scope you're in, when scopes get created, when scopes get destroyed, variables within each scope.
So pretty much every single detail you need to sort of understand functions.
As we're starting to-- you can see we've had couple questions, and these were great questions.
So if you're still trying to understand what's going on, I would highly suggest you take a piece of code and just run it in the Python Tutor and you should be able to see exactly what happens, in sort of a similar way that I've drawn my diagrams.
In all of the codes for this particular lecture, I've put links to the Python Tutor for each one of those exercises.
So you can just copy and paste those, and it'll automatically populate it with that particular example, so you just have to click, step, step, step.
OK so having made my plug for Python Tutor, let's go on.
OK so here's an example.
It's going to show couple things.
One is print versus return, and also this idea of you can nest functions.
So just like you could have nested loops, nested conditionals-- you can also nest functions within functions.
So let's draw some diagrams just like before of the scopes.
First thing we're going to do is when we have a program, we're going to create the global scope and we're going to add every variable that we have.
And then when we reach a function call, we're going to do something about that.
So the first thing in the global scope is this function definition.
Again in my global scope, I just have g as some code because I have not called it yet.
I only go inside a function when I make a function call.
So g contains some code.
So we're done with 75% of that code.
Next line is x is equal to 3.
So I'm making x be a variable inside my global scope with value 3.
And then I have this z is equal to g of x.
This is a function call.
When I see a function call, I'm going to create a new scope.
So here is the scope of g. With the scope of g, I'm mapping variables to actual parameters to formal parameters.
So the first thing I'm doing is I'm saying inside g what is the value of actual parameter x?
And x is going to be the value 3, because I've called g of x with x is equal to 3.
Next, what I see inside this function-- so this is the inside of the function-- is this bit here.
It's another function definition.
Again since I'm just defining the function and I'm not calling it, all Python sees is h is some code.
I haven't called the function h yet, because I'm just defining it here with def.
So that finishes this part here.
The next line is x is equal to x plus 1.
So inside the scope of g, I'm incrementing x to be 4.
Then I'm printing out this line.
And then I've reached here-- h. This is actually a function call, and I'm calling h. As soon as I make a function call, I'm creating another scope.
So I'm temporarily going out of the scope of g and going into the scope of h. So Python knows that h contains some code, and now I can go inside h and do whatever I need to do.
So the first-- so h doesn't have any parameters, so I don't need to populate anything like that in there.
h does define a variable called x, which is abc; it's a string.
And then that's all h does.
What does it return?
None.
I heard murmuring, but I think None was what you guys were saying.
So since there's no return statement, h is going to return None.
So h returns None.
Back to whoever called it, which was this code inside g. So that gets replaced with None-- the thing that I've-- this circled red h here.
As soon as h returns, we're going to get rid of that scope-- all the variables created within it-- and we're done with h. So now we're back into g. And we just finished executing this and this got replaced with None.
We're not printing it out, so this doesn't show up anywhere; it's just there.
So we're finished with that line.
And the next line is return x.
So x inside g is 4, so 4 gets returned back to whoever called it, which was in the global scope here.
So this gets replaced with 4.
So once we've returned x, we've completely exited out of the scope of g, and we've come back to whoever called us, which was global scope and we've replaced z is equal to g of x and that completely got replaced with 4-- the returned value.
So that's sort of showing nested functions.
All right just circling back to decomposition-abstraction.
This is the last slide.
You can see if you look at the code associated with today's lecture, there are some other examples where you can see just how powerful it is to use functions.
And you can write really clean and simple code if you define your own functions and then just use them later.
And the beauty of defining your own functions that you can use multiple times later is you only have to debug the function once right?
I know debugging is not your favorite thing, but you only have to debug this one thing once, and then you can know that it's right and it works well, and you can just use it multiple times.
All right thanks everyone.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I have two functions here.
One is add, and one is multiply.
The add function returns the sum of x plus y.
And the multiply function just prints the value of x times y, but doesn't return the value.
Instead, it's going to implicitly return none because we don't have any return statement inside mult.
So there are four lines here.
And the question was how many total lines of output will show up if you run the code?
OK, so how many lines in the console show up?
So when I do first add 1, 2, it's going to go inside this function here and say-- it says x is 1, y is 2, and return 3.
So I'm replacing this line with 3.
But notice I'm never printing it out.
So this line of code will not print out anything.
Instead, the next line is if I print add 2 plus 3, then I'm going to get 5.
And I'm going to print that out.
So that's one thing I'm printing out.
The next line says multiply 3 and 4.
So I'm going to go inside my mult function and say, x is 3, y is 4.
So I'm going to print 12 because 3 times 4 is 12.
So that's another thing that gets printed out.
And I'm not doing anything with the output-- or sorry, I'm not doing anything with the return from mult.
So that line is done.
And the last one is the trickiest and it says print mult 4 and 5.
So first I'm going to go in here.
X is 4, y is 5.
And I'm printing 20.
So that's another print.
And the return of this function is going to be none, right?
And the main thing here is I'm going to print at the return of this function.
So I'm also going to print out none.
So that's another one.
So in total, I'm going to have four different things printed out, the last one being this none.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So here's some code.
I'm defining a function named sq, and it takes in two parameters.
I'm defining a function named f, takes in one parameter, and then I'm doing these two lines.
The first one is just calling function sq, and the next one is just printing the value.
So let's first see what-- nice.
OK. Let's work through it.
So The first things we see here is two function definitions, right?
So we don't currently care about what's inside them right now because we haven't made a function call yet.
So the first thing we do is we have the function call calc equal sq f and 2.
OK?
So inside sq, we're going to have func and x.
And func is going to get mapped to f, and x is going to get mapped to 2, right?
So we're taking the variables in order and mapping them to those.
Func is f, and x is 2.
First thing the function does, sq, is create this variable y is equal to x squared.
So we're going to have y is equal to 4.
And then we're going to return func y.
So func of y, this is going to be-- we're just replacing the parameters f of 4, right?
So now this is another function call.
We know what f is.
This program knows what f is.
f is going to be this part right here, which returns x squared.
OK?
So in f, x gets mapped to whatever variable we put in, which in this case is 4.
And we're going to return to whoever called us, which is over here, 4 squared, which is 16.
OK?
So f of 4 gets replaced with 16.
And f of 4 was up here, right?
We're just popping out of scopes now.
So f of 4 was up here.
So then that line there, return func y, is going to return 16.
And whoever called us was down here, calc is equal to 16.
And then we're just printing calc.
So 16 was right.
Yay.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Quick, quick recap of what we did last time.
So last time we introduced this idea of decomposition and abstraction.
And we started putting that into our programs.
And these were sort of high level concepts, and we achieved them using these concrete things called functions in our programs.
And functions allowed us to create code that was coherent, that had some structure to it, and was reusable.
OK. And from now on in problem sets and in lectures, I'm going to be using functions a lot.
So make sure that you understand how they work and all of those details.
So today, we're going to introduce two new data types.
And they're called compound data types, because they're actually data types that are made up of other data types, particularly ints, floats, Booleans, and strings.
And actually not just these, but other data types as well.
So that's why they're called compound data types.
So we're going to look at a new data type called a tuple and a new data type called a list.
And then we're going to talk about these ideas that come about with-- specifically with lists.
All right.
So let's go right into tuples.
So if you recall strings, strings were sequences of characters.
Tuples are going to be similar to strings in that they're going to be sequences of something, except that tuples aren't just sequences of characters, they can be sequences of anything.
They're a collection of data where that data can be of any type.
So a tuple can contain elements that are integers, floats, strings, and so on.
Tuples are immutable.
And if you recall, we talked about this word a little bit when we talked about strings.
So that means once you create a tuple object, you can't modify it.
So when you created a string object, you were not allowed to modify it.
So the way we create tuples are with these open and close parentheses.
This shouldn't be confused with a function, because there's nothing-- there's no-- this isn't a function call.
It's just how you represent a tuple.
For a function call, you'd have something right before the parentheses.
This is just how we chose to represent a tuple.
And just a plain open and close parentheses represents an empty tuple.
So it's of length 0.
There's nothing in it.
You can create a tuple that contains some elements by separating each element with a comma.
So in this case, this is a tuple that can be accessed with a variable t that contains three elements.
The first is an integer, the second is a string, and the third is another integer.
Much like strings, we can index into tuples to find out values at particular indices.
So you read this as t at position 0.
So the tuple represented by a variable t at position 0 will evaluate to 2, because again, we start counting from 0 in computer science.
So that brings us-- gives us the first element.
Just like strings, we can concatenate tuples together.
That just means add them together.
So if we add these two tuples together, we just get back one larger tuple that's just those two-- the elements of those tuples just put together in one larger tuple.
Again, much like strings, we can slice into tuples.
So t sliced from index 1 until index 2.
Remember, we go until this stop minus 1.
So this only gives us one element inside the tuple.
And this is not a mistake.
This extra comma here actually represents a tuple object.
If I didn't have this comma here, then this would just be a string.
The parentheses would just-- wouldn't really make any difference.
But the comma here makes it clear to Python that this is a tuple with only one element in it.
We can slice even further to get a tuple with two elements.
And we can do the usual operations like get the length of a tuple, which says, how many elements are in my tuple?
And len of this t would evaluate to 3, because there are three elements inside the tuple.
Each element, again, separated by the comma.
And just like strings, if we try to change a value inside the tuple-- in this case, I wanted to try to change the value of the second element to 4-- Python doesn't allow that, because tuples are immutable.
So why would we want to use tuples?
Tuples are actually useful in a couple of different scenarios.
So recall a few years ago, we looked at this code where we tried to swap the values of variables x and y.
And this first code actually didn't work, because you're overwriting the value for x.
So instead, what we ended up doing was creating this temporary variable where we stored the value of x, and then we overwrote it, and then we used the temporary variable.
Well, turns out this three liner code right here can actually be written in one line using tuples.
So you say x comma y is equal to y comma x.
And Python goes in and says, what's the value of y?
And assigns it to x.
And then what's the value of x?
And assigns it to y.
Extending on that, we can actually use tuples to return more than one value from a function.
So functions, you're only allowed to return one object.
So you're not allowed to return more than one object.
However, if we use a tuple object, and if that's the thing that we return, we can actually get around this sort of rule by putting in as many values as we want inside the tuple object.
And then we can return as many values as we'd like.
So in this specific example, I'm trying to calculate the quotient and remainder when we divide x by a y.
So this is a function definition here.
And down here I'm calling the function with 4 and 5.
So when I make the function call, 4 gets assigned to x and 5 gets assigned to y.
So then q is going to be the integer division when x is divided by y.
And this double slash just means-- it's like casting the result to an integer.
It says divide it, keep the whole number part, and just delete everything else beyond the decimal point.
So when you divide 4 by 5, this q is actually going to be 0, because it's 0 point something, and I don't care about the point something.
And then the remainder is just using the percent operator.
So I divide 4 by 5, the remainder is going to be 4.
And notice that I'm going to be returning q and r, which are these two values that I calculated inside my function.
And I'm returning them in the context of this tuple.
So I'm only returning one object, which is a tuple.
It just so happens that I'm populating that object with a few different values.
So when the function returns here, this is going to say 0 comma 4.
That's the tuple it's going to return.
Q is going to be 0 and r is going to be 4.
So then this line here-- quot, rem equals 0, 4-- is basically this-- it's sort of like what we did up here.
So it assigns quot to 4-- sorry.
Quot to 0 and rem to 4.
So we can use tuples.
This is very useful.
We can use them to return more than one value from a function.
So tuples are great.
Might seem a little bit confusing at first, but they're actually pretty useful, because they hold collections of data.
So here, I wrote a function which I can apply to any set of data.
And I'll explain what this function does, and then we can apply it to some data.
And you can see that you can extract some very basic information from whatever set of data that you happen to collect.
So here's a function called get_data, and it does all of this stuff in here.
And in the actual code associated with the lecture, I actually said what the condition on a tuple was.
So it has to be a tuple of a certain-- that looks a certain way.
And this is the way it has to look.
So it's one tuple.
The outer parentheses out here represent the fact that it's a tuple.
And the elements of this tuple are actually other tuples.
So the first element is a tuple object, the second element is a tuple object, and third one is a tuple object, and so on.
And each one of these inner tuple objects are actually going to contain two elements, the first being an integer and the second being a string.
So that's sort of the precondition that this function assumes on a tuple before it can-- before it actually can work.
All right.
So given a tuple that looks like that, what's the function going to do?
It's first creating two empty tuple.
One is called nums and one is called words.
And then there's a for loop.
And notice here the for loop is going to iterate over every element inside the tuple.
Remember in strings when we were able to use for loops that iterated over the characters directly as opposed to over the indices?
Well, we're doing the same sort of thing here.
Instead of iterating over the indices, we're going to iterate over the tuple object at each position.
So first time through the loop, t here is going to be this first tuple.
The second time through the loop, t is going to be this tuple.
And the third time, it's going to be this exact tuple object.
So each time through the loop, what I'm doing is I'm going to have this nums tuple that I'm going to keep adding to.
And each time I'm going to create a new object and reassign it to this variable nums.
And each time through the loop, I'm looking at what the previous value of nums was.
So what was my previous tuple?
And I'm going to add it with this singleton tuple.
So it's a tuple of one character or one element.
This element being t at position zero.
So you have to sort of wrap your mind around how this is working.
So if t is going to be this tuple element right here, then t at position zero is going to be this blue bar here.
So it represents the integer portion of the tuple.
So as we're going through the loop, this nums is going to get populated with all of the integers from every one of my tuple-- inside tuple objects.
So that's basically what this line here is doing.
At the same time, I'm also populating this words tuple.
And the words tuple is a little bit different, because I'm not adding every single one of these string objects.
So t at position one being the string part of the inner tuple.
I'm actually adding the string part only if it's not already in my words list.
So here, I'm essentially grabbing all of the unique strings from my list.
These last sort of three lines-- three, four lines here just do a little bit of arithmetic on it saying, OK, now I have all of the numbers here, what's the minimum out of all of these, and then what's the maximum amount of all these?
And then this unique words variable tells me how many unique words do I have in my original tuple.
So this feels sort of generic, so let's run it on some data.
So here I have it-- I tested it on some test data.
And then I got some actual data.
And this actual data that I wanted to analyze was Taylor Swift data.
And representing the integer portion of the tuple representing a year and the string portion of the tuple representing the person who she wrote a song about that year.
So some real world data that we're working with here.
Very important that we know this information.
OK.
So with this data, I can run it-- I can plug it into this function that I wrote up here.
And I'm going to actually comment this out, so it doesn't get cluttered.
And if I run it-- this is the part where I'm calling my function.
I'm calling it with this data here.
tswift being this tuple of tuples.
And what I get back is-- up here, line 38-- is the return from the function being a large tuple.
And that large tuple, I'm then assigning it to my own tuple in my program.
And then I'm just writing out-- printing out some statement here.
So I'm getting the minimum year, the maximum year, and then the number of people.
So I can show you that it works if I replace one of these names with another one that I already have in here.
So instead of writing a song about five people, she would have wrote a song about four people.
Yay, it worked.
So that's tuples.
And remeber or recall-- keep in mind, tuples were immutable.
Now we're going to look at a very, very similar data structure to tuples called lists, except that instead of lists being immutable, lists are going to be mutable objects.
So much like lists, they're going to contain elements of any type or objects of any type.
You denote them with-- you denote a list with square brackets instead of parentheses.
And the difference being that they're going to be immutable objects instead of immutable.
So creating an empty list, you just do open close square brackets.
You can have a list of elements of different types, even a list of lists.
So one of the elements being a list.
As usual, you can apply length on a list, and that tells you how many elements are in it.
This is going to tell you how many elements are in your list l. So it's not going to look any further than that.
So it's going to say, this is an integer, this is a string, this is an integer, this is a list.
It's not going to say how many elements are in this list.
It's just going to look at the outer-- the shell of elements.
Indexing and slicing works the same way.
So l at position 0 gives you the value 2.
You can index into a list, and then do something with the value that you get back.
So l at position 2 says-- that's this value there and add one to it.
L at position 3, that's going to be this list here.
Notice it evaluates to another list.
You're not allowed to index outside of the length of the list.
So that's going to give you an error, because we only have four elements.
And you can also have expressions for your index.
So this-- Python just replaces i with 2 here and says, what's l at position 1?
And then grabs that from in there.
OK.
So very, very similar to the kinds of operations we've seen on strings and tuples.
The one difference, and that's what we're going to focus on for the rest of this class, is that lists are mutable objects.
So what does that mean internally?
Internally, that means let's say we have a list l, and we assign it-- sorry.
Let's say we have a variable l that's going to point to a list with three elements, 2, 1, and 3.
OK.
They're all-- each element is an integer.
When we were dealing with tuples or with strings, if we re-assign-- if we try to do this line right here, we've had an error.
But this is actually allowed with lists.
So when you execute that line, Python is going to look at that middle element, and it's going to change its value from a 1 to a 5.
And that's just due to the mutability nature of the list.
So notice that this list variable, this variable l, which originally pointed to this list, points to the exact same list.
We haven't created a new object in memory.
We're just modifying the same object in memory.
And you're going to see why this is important as we look at a few side effects that can happen when you have this.
So I've said this a couple of times before, but it'll make your life a lot easier if you try to think of-- when you want to iterate through a list if you try to think about iterating through the elements directly.
It's a lot more Pythonic.
I've used that word before.
So this is sort of a common pattern that you're going to see where you're iterating over the list elements directly.
We've done it over tuples.
We've done it over strings.
So these are identical codes.
They do the exact same thing, except on the left, you're going from-- you're going through 0, 1, 2, 3, and so on.
And then you're indexing into each one of these numbers to get the element value.
Whereas on the right, this loop variable i is going to have the element value itself.
So this code on the right is a lot cleaner.
OK.
So now let's look at some operations that we can do on lists.
So there's a lot more operations that we can do on lists, because of their mutability aspect than we can do on tuples or strings, for example.
So here's a few of them.
And they're going to take advantage of this mutability concept.
So we can add elements directly to the end of the list using this funky looking notation L.append.
And then the element we want to add to the end.
And this operation mutates the list.
So if I have L is equal to 2, 1, 3, and I append the element 5 to the end, then L-- the same L is going to point to the same object, except it's going to have an extra number at the end.
5.
But now what's this dot?
We haven't really seen this before.
And it's going to become apparent what it means in a few lectures from now.
But for the moment, you can think of this dot as an operation.
It's like applying a function, except that the function that you're applying can only work on certain types of objects.
So in this case, append, for example, is the function we're trying to apply.
And we want to apply it to whatever is before the dot, which is the object.
And append has only been defined to work with a list object, for example, which is why we're using the dot in this case.
We wouldn't be able to use append on an integer, for example, because that sort of function is not defined on the integer.
So for now, you'll sort of have to remember-- which are functions that work with a dot and which are functions like [?
ln, ?]
that aren't with a dot.
But in a couple of lectures, I promise it'll be a lot clearer.
So for now, just think of it as whatever is before the dot is the object you're applying a function to, and whatever is after the dot is the function you're applying on the object.
So we can add things to the end of our list.
We can also combine lists together using the plus operator.
The plus operator does not mutate the list.
Instead, it gives you a new list that's the sum of those two lists combined.
So in this case, if L1 is 2,1,3 and L2 is 4, 5, 6, when we add those two lists together, that's going to give us an entirely new list leaving L1 and L2 the same.
And that's why we have to assign the result of the addition to a new list.
Otherwise, the result is lost.
If you want to mutate a list directly and make it longer by the elements within another list, then you can use this extend function or extent method.
And this is going to mutate L1 directly.
So if L1 was 2,1,3, if you extend it by the list 0,6, it's just going to tack on 0,6 to L1 directly.
So notice L1 has been mutated.
So that's adding things to lists.
We can also delete things from lists.
We don't just want to keep adding to our lists, because then they become very, very big.
So let's see how we can delete some items from our list.
There's a few ways.
First one being can use this del function.
And this says delete from the list L the element at this index.
So you give it the index 0, 1, 2, or whatever you want to-- whatever index you want to delete the element at.
If you just want to delete the element at the end of the list, that's the farthest right, you do L.pop.
If you want to remove a specific element-- so you know there's somewhere in your list there's the number 5, and you want to delete it from the list-- then you say L.remove and you say what element you want to remove.
And that only removes the very first occurrence of it.
So if there's two fives in your list, then it's only going to remove the very first one.
So let's take a look at this sort of sequence of commands.
So we have first L is equl to this long list here.
And I want to mention that all of these operations are going to mutate our list, which is why I wrote this comment here that says assume that you're doing these in order.
So as you're doing these in order, you're going to be mutating your list.
And if you're mutating your list, you have to remember that you're working with this new mutated list.
So the first thing we're doing is we're removing 2 from our list.
So when you remove 2, this says look for an element with the value 2 and take it away from the list.
So that's the very first one here.
So the list we're left with is just everything after it.
Then I want to remove 3 from the list and notice there's two of them.
There's this 3 here and there's this 3 here.
So we're going to remove only the first one, which is this one here.
So the list we're left with is 1,6,3,7,0.
Then we're going to delete from the list L the element at position 1.
So starting counting from 0, the element at position 1 is this one here.
So we've removed that, and we're left with 1, 3, 7, 0.
And then when we do L.pop, that's going to delete the element furthest to the right.
So at the end of the list, which is that 0.
So then we're left with only 1, 3, and 7.
And L.pop is often useful, because it tells you the return value from L.pop is going to be the value that it removed.
So in this case, it's going to return 0.
I want to mention, though, that some of the-- so these functions all mutate the list.
You have to be careful with return values.
So these are all-- you can think of all of these as functions that operate on the list.
Except that what these functions do is they take in the list, and they modify it.
But as functions, they obviously return something back to whoever called them.
And oftentimes, they're going to return the value none.
So for example, if you are going to do L.remove 2, and you print that out, that might print out none for you.
So you can't just assign the value of this to a variable and expect it to be the mutated list.
The list got mutated.
The list that got mutated is the list that was passed into to here.
We're going look at one example in a few slides that's going to show this.
OK. Another thing that we can do, and this is often useful when you're working with data, is to convert lists to strings and then strings to lists.
Sometimes it might be useful to work with strings as opposed to a list and vice versa.
So this first line here, list s takes in a string and casts it to a list.
So much like when we cast a float to an integer, for example.
You're just casting a string to a list here.
And when you do that up this line-- so if this is your s here-- when you do list s, this is going to give you a list-- looks like this-- where every single character in s is going to be its own element.
So that means every character is going to be a string, and it's going to be separated by a comma, so including spaces.
Sometimes you don't want each character in the list to be its own element.
Sometimes you want, for example, if you're given a sentence, you might want to have everything in between spaces being its own element.
So that will give you every word in the sentence, for example.
In that case, you're going to use split.
In this case, I've split over the less than sign.
But again, if you're doing the sentence example, you might want to split on the space.
So this is going to take everything in between the sign that you're interested in-- in this case, the less than sign-- and it's going to set it as a separate element in the list.
So that's how you convert strings to lists.
And sometimes you're given a list, and you might want to convert it to a string.
So that's where this join method or function is useful.
So this is an empty string.
So it's just open close quote right away.
No space.
So this just joins every one of the elements in the list together.
So it'll return the string abc.
And then you can join on any character that you would want.
So in this case, you can join on the underscore.
So it'll put whatever characters in here in between every one of the elements in your list.
So pretty useful functions.
OK.
Couple other operations we can do on lists-- and these are also pretty useful-- is to sort lists and to reverse lists and many, many others in the Python documentation.
So sort and sorted both sort lists, but one of them mutates the list and the other one does not.
And sometimes it's useful to use one, and sometimes it's useful to use the other.
So if I have this list L is equal to 9,6,0,3, sorted-- you can think of it as giving me the sorted version of L-- gives you back the sorted version of L. So it returns a new list that's the sorted version of the input list and does not mutate L. So it keeps L the exact same way.
So this will be replaced by the sorted version of the list, which you can assign to a variable, and then do whatever you want with it.
Like L2 is equal to sorted L, for example.
And it keeps L the same.
On the other hand, if you just want to mutate L, and you don't care about getting another copy that's sorted, you just do L.sort.
And that's going to automatically sort L for you, and L now-- L is now the sorted version of L. Similarly, reverse is going to take L and reverse all the character-- all the elements in it.
So the last one is the first one, the second to last one is the second one, and so on.
So lists are mutable.
We've said that so many times this lecture.
But what exactly does that mean?
What implications does that have?
Once again, this next part of the lecture, Python tutor.
Just paste all the code in and go step by step to see exactly what's happening.
So lists are mutable.
As you have variable names-- so for example, L is equal to some list-- that L is going to be pointing to the list in memory.
And since it's a mutable object, this list, you can have more than one variable that points to the exact same object in memory.
And if you have more than one variable that points to the same object in memory, if that object in memory is changed, then when you access it through any one of these variables, they're all going to give you the changed object value.
So the key phrase to keep in mind when you're dealing with lists is what side effects could happen?
If you're mutating a list, if you're doing operations on lists, what side effects-- what variables might be affected by this change?
Let's come back down to earth for a second.
This will wake a lot of people up.
So let's do an analogy with people.
Let's say we have a person.
A person-- this case, Justin Bieber-- is going to be an object.
I'm an object.
I'm like the number three.
Bieber's an object.
He's like number five.
Different objects.
Were both of type people.
OK. Let's say a person has different attributes.
Let's say we can-- let's say he gets two attributes to begin with.
He's a singer and he's rich.
I can refer to this person object by many different names.
His full name, his stage name, all of the fan girls call him by these names, people who dislike him call him by other names that they didn't put up here.
But he's known by all these different names.
They're all aliases or nicknames that point to this same person object.
OK.
So originally, let's say I say Justin Bieber is a singer and rich.
Those are the two attributes I've originally assigned to him.
And then let's say I want to assign a different attribute to him and say Justin Bieber's a singer, rich, and a troublemaker.
I'm being kind here.
OK.
So if I say Justin Bieber has these three attributes-- so it's the same person I'm referring to-- then all of his nicknames are going to refer to this exact same person.
So all of his nicknames or aliases will refer to the same person object with these changed attributes.
Does that makes sense?
OK.
So that sort of idea arises in lists.
So a list is like a person object whose value-- whose attributes can change, for example.
And as they change, all of the different aliases for this object will point to this changed object.
So let's see a few examples.
I apologize if this is a little small, but this I basically copied and pasted from the Python tutor, which is just from the code from today's lecture.
So I have these lines of code here.
The first couple of lines really just show what happens when you're dealing with non-mutable objects.
So with non-mutable objects, you have two separate objects that get their own values, and that's it.
End of story.
With lists, however, there's something different that happens.
So I have warm is a variable.
And it's going to be equal to this list.
So warm is going to point to this list here.
Red, yellow, orange.
It contains three elements.
Hot is equal to warm.
It means I'm creating an alias for this list.
And the alias is going to be with this variable hot.
So notice warm and hot point to the exact same object.
So on line 8 when I append this string pink to my object, since both of these two variables point to the exact same object, if I'm trying to access this object through either variable, they're both going to print out the same thing.
And that's the side effect.
That's the side effect of lists mutable.
If you want to create an entirely new copy of the list, then you can clone it, which sounds really cool.
But really, it's just making a copy of the list.
And you clone it using this little notation here, which is open close square brackets with a colon.
And we've sort of seen this notation here.
And this tells Python this is 0-- sorry.
This is 0 and this is length.
Cool.
But it basically says take every element, create a new list with those exact same elements, and assign it to the variable chill.
So here, if I originally have cool is equal to blue, green, gray right here, when I clone it on line 2 with that funky notation, I'm creating a new copy of it.
And then on the next line when I'm appending another element to the copy, notice I'm just altering the copy.
The original stayed the same, because I've cloned it.
So if you don't want to have the side effects-- side effect issue, then you should clone your variable-- your list.
So let's see a slightly more complicated example where you're going to see the difference between sort and sorted in the context of this mutability and side effects issue.
OK.
So once again, let's create this warm is equal to red, yellow, orange.
So that's what warm is going to point to, this list.
And then sorted warm is equal to warm.sort.
So .sort mutates.
So as soon as I do that, that list warm is now the sorted version of it.
And notice that I've assigned the return of this to sorted warm.
And the return is none, because L.sort or .sort mutated the list.
It didn't return a sorted version of the list.
It mutated the list itself.
OK.
So when I print warm and I print sorted warm, I'm printing the mutated version and then this one here.
Sorted, on the other hand, returns-- it doesn't-- sorted does not sort the list that's given to it.
And instead, it returns a sorted version of the list.
So in this case, if cool is equal to these three colors-- gray, green, blue-- if I do sorted cool, it's going to return the sorted version of that list, which is blue, green, gray.
And it's assigned to the variable sorted cool.
So when I print them, it's going to show me the two separate lists.
One being the original unsorted one, and one being the sorted version.
Last ones a little bit more complicated, but it shows that even though you have nested-- even though you can have nested lists, you still-- you're not-- you don't escape this idea of side effects.
So first, I'm going to create warm is equal to these two colors, yellow, orange.
So warm points to these two colors.
Hot is equal to this one list-- a list with one element.
Bright colors is going to be a list.
And the element inside the list is a list itself.
So since it's a list-- this is your list, and the element inside here, which is a list itself, is actually just pointing to whatever warm is.
That object.
Then I do-- then I append hot to my bright colors.
So the next element here is going to be another list, which means it's just pointing to this other list here.
It's not creating a copy of it.
So each one of these elements here is actually just pointing to these two lists here.
So if I modified either one of these, then bright colors would also be modified.
So let's say I add pink here to my hot list.
We have red and pink.
Then notice that bright colors-- the first element points to this list, and the second element points to this list, which I've just modified.
Last thing is-- I'll let you try this as an exercise in Python Tutor-- but the idea here being you should be careful as you're writing a for loop that iterates over a list that you're modifying inside the list.
In this case, I'm trying to go through the list L1.
And if I find an item that's in L1 and L2, I want to delete it from L1.
So 1 and 2 are also in L2.
So I want to delete them from L1 and be left with 3, 4.
However, the code on the left here doesn't actually do what I think it's doing, because here I'm modifying a list as I'm iterating over it.
And behind the scenes, Python keeps this-- keeps track of the index and doesn't update the index as you're changing the list.
So it figures out the length of the list to begin with and how many indices it has.
It doesn't update it as you're removing items from the list.
So the solution to that is to make a copy of the list first, iterate over the copy, which will remain intact, and modify the list that you want to modify inside the loop.
So please run both of these in the Python Tutor, and you'll see that what ends up happening is on the left, you're going to skip over one element.
So your code-- so that's going to be the wrong code.
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, we've got some good competition here.
So let's actually work through it.
So I have a function called always sunny and it's going to take in two variables right, t1 and t2.
And I'm calling it with cloudy and cold.
So when I do my function call, t1 is going to be equal to cloudy.
These are strings but I'm not going to bother putting the quotes.
And t2 is equal to cold comma.
So remember what I said, with a comma it's a tuple, without a comma it's a string.
So t1 is actually going to be a string.
And t2 is actually a tuple.
OK.
So that's the first sort of trick to this question.
So I've made my function call and I've assigned t1 and t2 to be those two values.
So the next line is sun is equal to sunny comma sun.
So sun is going to be this tuple of two strings.
The next line is figuring out what first is.
So first is going to be-- so I'm looking at my t1 here, it's a string.
The fact that I have parentheses doesn't actually make a difference when I'm talking about strings.
Like that.
So when I'm indexing into a string, t1 at position 0 actually just gives me a C because it's a string not a tuple.
And t2 up position zero says, OK well, this is a tuple that contains only one element.
That element being at position 0, and that element is the string, cold.
So this is a tuple.
So I'm taking everything right before the first comma.
And that happens to be just the one element that's in there.
So this is just the string C cold.
And then I'm returning here a tuple.
And the tuple I'm returning is sun at position 0, so that's just sunny comma.
Just doing what's in here.
And then first, and first was this string, C cold.
So really the important thing about this example was to make sure that you understand the difference between what a string is and what a tuple is.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at owc.mit.edu
So we have a list L, originally, [life, answer, 42, 0].
In this example, if you don't know Hitchhiker's Guide to the Galaxy, that's why 42 is important.
As a scientist, if I didn't at least one example with the number 42, I'd probably hear about it.
So here it is.
OK, so we have this original list L, which contains the string "life", the string "answer", the number 42, and the number 0.
Here, I'm writing a loop that iterates over every element directly in L, right?
So originally, thing is going to be string "life", then, thing is going to be the string "answer", then, thing is going to be the number 42.
And so what am I doing inside the loop as I'm iterating?
Well, if thing is equal to 0, then I'm going to do this.
And otherwise, I'm going to do this.
So for the first two iterations when thing is equal to "life" or "answer", I'm not doing anything.
As soon as I hit 42 though, I'm saying, L, at position 1, is going to get this value here, OK?
So when I've iterated over this element here, I'm saying L, at position 1, I'm going to change its value, which says, well, I'm going to keep "life" the same, but this is going to be "everything".
And then we're keeping the rest the same, 42 and 0, because we're only modifying L at position 1.
As soon as we do that, we've finished with the loop when L was 42.
And then we go through 0.
So now, thing takes on the value 0.
And we say, if thing is equal to 0, well, we're good on that.
L, at position thing, is equal to "universe".
So L, at position 0, is equal to "universe".
So that says, OK, I'm going to replace "life" with "universe".
And then everything else stays the same-- 42 and 0, OK?
So that would be my answer, [universe, everything, 42, 0].
Great, that's the majority.
Good job.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's walk through it.
We have three lists.
L1 is equal to [re].
L2 is equal to [mi].
L3 is equal to [do].
L4 is equal to-- let's just do what it says here-- L1 plus L2.
So it's going to be the list [re, mi]-- uh, yeah, mi.
Now, extend() is going to mutate the list.
So that means L3 is going to be extended by whatever L4 is.
So L3 is going to be-- OK, what did it have originally?
It had do.
And then, it's going to be extended by all of the elements inside L4, which is re, mi, like that.
And once I've mutated it, the old version of L3 is gone, right?
This is the L3 down here that I'm going to work with.
L3.sort() is going to sort alphabetically.
So that's [do, mi, re].
Yep.
And sort() also mutates the list.
So the old version of the list I have is gone.
del is going to also mutate the list.
So it's going to look in L3, look at index 0, and it's going to delete that-- it's going to delete that element.
So it's going to mutate L3.
And it's going to be [mi, re], OK?
And once again, I've mutated the list.
So the old version's gone.
And lastly-- this is a tricky part-- I'm going to append, to my current L3, which looks like this, another list.
So to L3, I'm appending another list.
So this is going to be the list [fa, la], OK?
So [mi, re], and then the list [fa, la] is the answer, which is red-- no, sorry, blue.
Good, you had me scared.
OK.
Perfect.
So if you didn't get this, I would suggest you go through the Python tutor or you just trace it out by hand, just like I just did.
And hopefully it'll be apparent, the more exercises you do with lists, what exactly is going on.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so let's see.
So first we have L1 is equal to bacon, eggs.
I'm going to short form-- shorthand this.
L2 is equal to toast and jam.
Brunch-- I should actually make these arrows-- brunch is equal to L1.
So this is just aliasing, which means brunch is going to point to whatever L1 object points to.
And if I do L1.append juice, L1 is now going to be bacon, eggs, and juice.
OK?
This L1 has been mutated to be that.
And since brunch still points to the same object that L1 points to, brunch is now going to point to there.
OK?
So when I do brunch.extend L2, I'm going to take whatever brunch is, which is this part here, and I'm going to extend it by L2, which is toast and jam.
OK?
So it's just going to contain a large list of those five elements because of this side effect issue, where brunch was pointing to the same thing that L1 was pointing to, OK?
So it's close, but I think-- perfect.
Yes, that's the right answer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Ladies and gentlemen, I'd like to get started.
My name's Eric Grimson.
I have the privilege of serving as MIT'S chancellor for academic advancement, you can go look up what that means, and like John I'm a former head of course six.
This term, with Ana and John, I'm going to be splitting the lectures, so I'm up to date.
OK last time Ana introduced the first of the compound data types, tuples and lists.
She showed lots of ways of manipulating them, lots of built in things for manipulating those structures.
And the key difference between the two of them was that tuples were immutable, meaning you could not change them, lists were mutable, they could be changed, or mutated.
And that led to both some nice power and some opportunities for challenges.
And, in particular, she showed you things like aliasing, where you could have two names pointing to the same list structure, and because of that, you could change the contents of one, it would change the appearance of the contents of the other, and that leads to some nice challenges.
So the side effects of mutability are one of the things you're going to see, both as a plus and minus, as we go through the course.
Today we're going to take a different direction for a little while, we're going to talk about recursion.
It Is a powerful and wonderful tool for solving computational problems.
We're then going to look at another kind of compound data structure, a dictionary, which is also mutable.
And then we're going to put the two pieces together and show how together they actually give you a lot of power for solving some really neat problems very effectively.
But I want to start with recursion.
Perhaps one of the most mysterious, at least according to programmer's, concepts in computer science, one that leads to lots of really bad computer science jokes, actually all computer science jokes are bad, but these are particularly bad.
So let's start with the obvious question, what is recursion?
If you go to the ultimate source of knowledge, Wikipedia, you get something that says, in essence, recursion is the process of repeating items in a self-similar way.
Well that's really helpful, right?
But we're going to see that idea because recursion, as we're going to see in a second, is the idea of taking a problem and reducing it to a smaller version of the same problem, and using that idea to actually tackle a bunch of really interesting problems.
But recursion gets used in a lot of places.
So it's this idea of using, or repeating, the idea multiple times.
So wouldn't it be great if your 3D printer printed 3D printers?
And you could just keep doing that all the way along.
Or one that's a little more common, it's actually got a wonderful name, it's called mise en abyme, in art, sometimes referred to as the Droste effect, pictures that have inside them a picture of the picture, which has inside them a picture of the picture, and you get the idea.
And of course, one of the things you want to think about in recursion is not to have it go on infinitely.
And yes there are even light bulb jokes about recursion, if you can't read it, it says, how many twists does it take to screw in a light bulb?
And it says, if it's already screwed in, the answer is 0.
Otherwise, twist it once, ask me again, add 1 to my answer.
And that's actually a nice description of recursion.
So let's look at it more seriously.
What is recursion?
I want to describe it both abstractly, or algorithmically, and semantically or, if you like, in terms of programming.
Abstractly, this is a great instance of something often called divide-and-conquer, or sometimes called decrease-and-conquer.
And the idea of recursion is, I want to take a problem I'm trying to solve and say, how could I reduce it to a simpler version of the same problem, plus some things I know how to do?
And then that simpler version, I'm going to reduce it again and keep doing that until I get down to a simple case that I can solve directly.
That is how we're going to think about designing solutions to problems.
Semantically, this is typically going to lead to the case where a program, a definition of function, will refer to itself in its body.
It will call itself inside its body.
Now, if you remember your high school geometry teacher, she probably would wrap your knuckles, which you're not allowed to do, because in things like geometry you can't define something in terms of itself, right?
That's not allowed.
In recursion, this is OK. Our definition of a procedure can in its body call itself, so long as I have what I call a base case, a way of stopping that unwinding of the problems, when I get to something I can solve directly.
And so what we're going to do is avoid infinite recursion by ensuring that we have at least one or more base cases that are easy to solve.
And then the basic idea is I just want to solve the same problem on some simpler input with the idea of using that solution to solve the larger problem.
OK, let's look at an example, and to set the stage I'm going to go back to something you've been doing, iterative algorithms.
For loops, while loops, they naturally lead to what we would call iterative algorithms, and these algorithms can be described as being captured by a set of state variables, meaning one or more variables that tell us exactly the state of the computation.
That's a lot of words, let's look at an example.
I know it's trivial, but bear with me.
Suppose I want to do integer multiplication, multiply two integers together, and all I have available to me is addition.
So a times b is the same as adding a to itself b times.
If I'm thinking about this iteratively, I could capture this computation with two state variables.
One we'd just call the iteration number, and it would be something, for example, that starts at b, and each time through the loop reduces 1.
One.
And it will keep doing that until I've counted down b times, and I get down to 0.
And at the same time, I would have some value of the computation, I might call it result, which starts at 0, first time through adds an a, next time through adds an a, and it just keeps track of how many things have I added up, until I get done.
And yeah, I know you could just do mult, but this is trying to get this idea of, how would I do this iteratively.
So I might start off with i, saying there are b things still to add, and the result is 1.
The first time through the loop, I add an a, reduce i by 1.
Next time through the loop, I add in another a, reduce i by 1, and you get the idea.
I just walk down it until, eventually, I got to the end of this computation.
So we could write code for this, and, actually, it should be pretty straightforward.
There it is.
Going to call it mult_iter, takes in two arguments a and b, and I'm going to capture exactly that process.
So notice what I do, I set up result internally as just a little variable I'm going to use to accumulate things.
And then, there is the iteration, as long as b is greater than 0 what do I do?
Add a to result, store it away, reduce b by 1, and I'll keep doing that until b gets down to being equal to 0, in which case I just return the result.
OK, simple solution.
Now, let's think about this a different way.
A times b is just adding a to itself b times, and that's the same as a plus adding a to a itself b minus 1 times.
OK, that sounds like leisure to me, that sounds like just playing with words.
But it's really important, because what is this?
Ah, that's just a times b minus 1, by the definition of the top point.
And I know you're totally impressed, but this is actually really cool, because what have I done?
I've taken one problem, this one up here, and I've reduced it to a simpler version of the same problem, plus some things I know how to do.
And how would I solve this?
Same trick, that's a times a times b minus 2, I would just unwrap it one more time, and I would just keep doing that until I get down to something I can solve directly, a base case.
And that's easy, when b equal to 1, the answer is just a.
Or I could do when b is equal to 0 the answer is just 0.
And there's code to capture that.
Different form, wonderful compact description, what does it say?
It says, if I'm at the base case, if b is equal to 1, the answer is just a.
Otherwise, I'm going to solve the same problem with a smaller version and add it to a and return that result.
And that's nice, crisp characterization of a problem.
Recursive definition that reduces a problem to a simpler version of the same problem.
OK, let's look at another example.
Classic problem in recursion is to compute factorial, right?
n factorial, or n bang if you like, n exclamation point is n times n minus 1, all the way down to 1.
So it's the product of all the integers from 1 up to n assuming n is a positive integer.
So we can ask the same question if I wanted to solve this recursively what would the base case be?
Well, when n is equal to 1, it's just 1.
In the recursive case, will n times n minus 1 all the way down to 1, that's the same as n times n minus 1 factorial.
So I can easily write out the base case, and I've got a nice recursive solution to this problem.
OK, if you're like me and this is the first time you've seen it, it feels like I've taken your head and twisted it about 180 degrees.
I'm going to take it another 180 degrees because you might be saying, well, wait a minute, how do you know it really stops.
How do you know it really terminates the computation?
So let's look at it.
There is my definition for fact, short for factorial.
Fact of 1 is, if n is equal to 1 return 1, otherwise return n times fact of n minus 1.
And let's use the tools that Ana talked about, in terms of an environment at a scope, and think about what happens here.
So when I read that in or I evaluate that in Python, it creates a definition that binds the name fact to some code, just all of that stuff over here plus the name for the formal parameter, hasn't done anything with it yet.
And then I'm going to evaluate print a fact of 4.
Print needs a value, so it has to get the value of fact of 4, and we know what that does.
It looks up fact, there it is, it's procedure definition.
So it creates a new frame, a new environment, it calls that procedure, and inside that frame the formal parameter for fact is bound to the value passed in.
So n is bound to 4.
That frame is scoped by this global frame meaning it's going to inherit things in the global frame.
And what does it do?
It says, inside of this frame evaluate the body of fact.
OK, so it says as n equal to 1?
Nope, it's not, it's 4.
So in that case, go to the else statement and says, oh, return n times fact of n and n as 4, fact of n minus 1 says I need to return 4 times fact of 3.
4 is easy, multiplication is easy, fact of 3, ah yes, I look up fact.
Now I'm in this frame, I don't see fact there, but I go up to that frame.
There's the definition for fact, and we're going to do the rest of this a little more quickly, what does that do?
It creates a new frame called by fact.
And the argument passed in for n is n minus 1, that value, right there, of 3.
So 3 is now bound to n. Same game, evaluate the body is n equal to 1?
No, so in that case, I'm going to go to the return statement, it says return 3 times fact of 2.
And notice it's only looking at this value of n because that's the frame in which I'm in.
It never sees that value of n. OK, aren't you glad I didn't do fact of 400?
We've only got two more to go, but you get the idea.
Same thing, I need to get fact of 2 is going to call fact again with n bound to 2.
Relative that evaluates the body and is not yet equal to 1.
That says I'm going to the else clause and return 2 times fact of 1.
I call fact again, now with n bound to 1, and, fortunately, now that clause is true, and it says return 1.
Whoops, sorry, before I do, so there's the base case.
And it may seem apparent to you, but this is important, right?
I'm unwinding this till I get to something that can stop the computation.
Now I'm simply going to gather the computation up, because it says return 1. Who asked for it?
Well that call to fact of 1.
So that reduces to return 2 times 1.
And who called for that?
Fact of 2.
That reduces to return a 3 times 2, which reduces to 4 times 6, which reduces to printing out 24.
So it unwinds it down to a base case and it stops.
A couple of observations, notice how each recursive call creates its own frame, and as a consequence, there's no confusion about which value of n I'm using.
Also notice, in the other frames, n was not changed.
We did not mutate it.
So we're literally creating a local scope for that recursive call, which is exactly what we want.
Also notice how there was a sense of flow of control in computing fact of something, that reduces to returning n times fact of n minus 1, and that creates a new scope.
And that will simply keep unwinding until I get to something that can return a value and then I gather all those frames back up.
So there's a natural flow of control here.
But most importantly, there's no confusion about which variable I'm using when I'm looking for a value of n. All right, because this is often a place where things get a little confusing, I want to do one more example.
But let me first show you side by side the two different versions of factorial.
Actually, I have lied slightly, we didn't show this one earlier but there's factorial if I wanted to do it iteratively.
I'd set up some initial variable to 1, and then I'd just run through a loop.
For example, from 1 up to just below n minus 1, or 1 up to n, multiplying it and putting it back into return product.
Which one do you like more?
You can't say neither you have to pick one.
Show of hands, how many of you like this one?
Some hesitant ones, how many prefer this one?
Yeah, that's my view.
I'm biased, but I really like the recursive one.
It is crisper to look at, you can see what it's doing.
I'm reducing this problem to a simpler version of that problem.
Pick your own version but I would argue that the recursive version is more intuitive to understand.
From a programmer's perspective, it's actually often more efficient to write, because I don't have to think about interior variables.
Depending on the machine, it may not be as efficient when you call it because in the recursive version I've got it set up, that set of frames.
And some versions of these languages are actually very efficient about it, some of them a little less so.
But given the speed of computers today, who cares as long as it actually just does the computation.
Right, one more example, how do we really know our recursive code works?
Well, we just did a simulation but let's look at it one more way.
The iterative version, what can I say about it?
Well, I know it's going to terminate because b is initially positive, assuming I gave it an appropriate value.
It decreases by 1 every time around this loop, at some point it has to get less than 1, it's going to stop.
So I can conclude it's always going to terminate.
What about the recursive version?
Well, if I call it with b equal to one, I'm done.
If I call it with b greater than one, again it's going to reduce it by one on the recursive call, which means on each recursive call it's going to reduce and eventually it gets down to a place, assuming I gave it a positive integer, where b is equal to one.
So it'll stop, which just good.
What we just did was we used the great tool from math, second best department at MIT.
Wow, I didn't even get any hisses on that one, John, all right, and I'm now in trouble with the head of the math department.
So now that I got your attention, and yes, all computer science jokes are bad, and mine are really bad, but I'm tenured.
You cannot do a damn thing about it.
Let's look at mathematical induction which turns out to be a tool that lets us think about programs in a really nice way.
You haven't seen this, here's the idea of mathematical induction.
If I want to prove a statement, and we refer to it as being indexed on the integers.
In other words, it's some mathematical statement that runs over integers.
If I want to prove it's true for all values of those integers, mathematically I'd do it by simply proving it's true for the smallest value of n typically n is equal to 0 or 1, and then I do an interesting thing.
I say I need to prove that if it's true for an arbitrary value of n, I'm just going to prove that it's also then true for n plus 1.
And if I can do those two things I can then conclude for an infinite number of values of n it's always true.
Then we'll relate it back to programming in a second, but let me show you a simple example of this, one that you may have seen.
If I had the integers from 0 up to n, or even from 1 up to n, I claim that's the same as n times n plus 1 over 2.
So 1, 2, 3, that's 6, right.
And that's exactly right, 3 times 4, which is divided by 2, which gives me out 6.
How would I prove this?
Well, by induction?
I need to do the simple cases if n is equal to 0, well then this side is just 0.
And that's 0 times 1, which is 0 divided by true.
So 0 equals 0, it's true.
Now the inductive step.
I'm going to assume it's true for some k, I should have picked n, but for some k, and then what I need to show is it's true for k plus 1.
Well, there's the left hand side, and I want to show that this is equal to that.
And I'm going do it by using exactly this recursive idea, because what do I know, I know that this sum, in here, I'm assuming is true.
And so that says that the left hand side, the first portion of it, is just k times k plus 1 over 2, that's the definition of the thing I'm assuming is true.
To that I'm going to add k plus 1.
Well, you can do the algebra, right?
That's k plus 1 all times k over 2 plus 1, which is k plus 2 over 2.
Oh cool, it's exactly that.
Having done that, I can now conclude this is true for all values of n. What does it have to do with programming?
That's exactly what we're doing when we think about recursive code, right?
We're saying, show that it's true for the base case, and then what I'm essentially assuming is that, if it works for values smaller than b, then does the code return the right answer for b?
And the answer is, absolutely it does, and I'm using induction to deduce that, in fact, my code does the right thing.
Why am I torturing you with this?
Because this is the way I want you to think about recursion.
When I'm going to break a problem down into a smaller version of the same problem, I can assume that the smaller version gives the answer.
All I have to do is make sure that what I combined together gives me out the right result.
OK, you may be wondering what I'm doing with these wonderful high tech toys down here.
I want to show you another example of recursion.
So far we've seen simple things that have just had one base case, and this is a mythical story called The towers of Hanoi and this story, as I heard it, is there's a temporal somewhere in Hanoi with three tall spikes and 64 jewel-encrusted golden disks all of a different size.
They all started out on one spike with the property that they were ordered from smallest down to largest.
And there are priests in this temple who are moving the disks one at a time, one per second, and their goal is to move the entire stack from one spike to another spike.
And when they do nirvana is achieved and we all get a really great life.
We'll talk separately about how long is this going to take because there's one trick to it.
They can never cover a smaller disk with a larger disk as they're doing it, so they've got a third disk as a temporary thing.
And I want to show you how to solve this problem because you're going to write code with my help in a second, or I'm going to write code with your help in a second to solve it.
So let's look at it, so watch carefully, moving a disk of size one, well that's pretty easy, right?
Moving a disk of size two, we'll just put this one on the spare one while you move it over so you don't cover it up.
That's easy.
Moving a disk of size three, you've got be a little more careful, you can't cover up a smaller one with a larger one, so you have to really think about where you're putting it.
It would help with these things didn't juggle and there you go, you got it done.
All right, you're watching?
You've got to do four.
To do four, again, you've got to be really careful not to cover things up as you do this.
You want to get the bottom one eventually exposed, and so are you going to pull that one over there.
If you do the pattern really well, you won't notice if I make a serious mistake as I'm doing this, which I just did.
But I'm going to recover from that and do it that way to put this one over here, and that one goes there, and if I did this in Harvard Square I could make money.
There you go, right?
OK, got the solution?
See how to solve it?
Could you write code for this?
Eh, maybe not.
That's on the quiz, thanks, John, don't tell them on the quiz, damn.
All right, I want to claim though that in fact there's a beautiful recursive solution.
And here's the way to think about it recursively.
I want to move a tower of size n, I'm going to assume I can move smaller towers and then it's really easy.
What do I do, I take a stack of size n minus 1, I move it onto the spare one, I move the bottom one over, and then I move a stack of size n minus 1 to there, beautiful, recursive solution.
And how do I move the smaller stack?
Just the same way, I just unwind it, simple, and, in fact, the code follows exactly that.
OK, I do a little [INAUDIBLE] domain up here to try and get your attention, but notice by doing that what did I do?
I asked you to think about it recursively, the recursive solution, when you see it, is in fact very straightforward, and there's the code.
Dead trivial, well, that trivial is unfair, but it's very simple.
Right?
I simply write something, so let me describe it, I need to say how big of tower am I moving and I'm going to label the three stacks a from, a to, and a spare.
I have a little procedure that just prints out the move for me, and then what's the solution?
If it's just a stack of size one, just print the move, take it to from-- from from to to.
Otherwise, move a tower of size n minus 1 from the from spot to the spare spot, then move what's left of tower size one from to two, and then take that thing are stuck on spare and move it over to two, and I'm done.
In that code that we handed out, you'll see this code, you can run it.
I'm not going to print it out because, if I did, you are just going to say, OK, it looks like it does the right kind of thing.
Look at the code, nice and easy, and that's what we like you to do when you're given a problem.
We asked you to think about recursively.
How do I solve this with a smaller version of the same problem?
And then how do I use that to build the larger solution?
This case is a little different.
You could argue that this is not really a recursive call here, it's just moving the bottom one, I could have done that directly.
But I've got two recursive calls in the body here.
I have to move a smaller stack twice.
We're going to come back to that in a little bit.
Let me show you one other example of recursion that runs a little bit differently.
In this case it's going to have multiple base cases and this is another very old problem, it's called the Fibonacci numbers.
It's based on something from several centuries ago when a gentleman, named Leonardo of Pisa, also known as Fibonacci, asked the following challenge.
He said, I'm going to put a newborn pair of rabbits, one male and one female, into an enclosure, a pan of some sort.
And the rabbits have the following properties, they mate at age one month, so they take a month to mature.
After a one month gestation period, they produce another pair of rabbits, a male and a female, and he says I'm going to assume that the rabbits never die.
So each month mature females are going to produce another pair.
And his question was, how many female rabbits are there at the end of a year, or two years, or three years?
The idea is, I start off with two immature rabbits, after one month they've matured, which means after another month, they will have produced a new pair.
After another month, that mature pair has produced another pair, and the immature pair has matured.
Which means, after another month, those two mature pairs are going to produce offspring, and that immature pair has matured.
And you get the idea, and after several months, you get to Australia.
You can also see this is going to be interesting to think about how do you compute this, but what I want you to see is the recursive solution to it.
So how could we capture this?
Well here's another way of thinking about it, after the first month, and I know we're going to do this funny thing, we're going to index it 0, so call it month 0.
There is 1 female which is immature.
After the second month, that female is mature and now pregnant which means after the third month it has produced an offspring.
And more generally, that the n-th month, after we get past the first few cases, what do we have?
Any female that was there two months ago has produced an offspring.
Because it's taken at least one month to mature, if it hasn't already been mature, and then it's going to produce an offspring.
And any female that was around last month is still around because they never die off.
So this is a little different.
This is now the number of females at month n is the number of females T month n minus 1, plus the number of females and month n minus 2.
So two recursive calls, but with different arguments.
Different from towers of Hanoi, where there were two recursive calls, but with the same sized problem.
So now I need two base cases, one for when n is equal to 0, one for when n is equal to 1.
And then I've got that recursive case, so there's a nice little piece of code.
Fibonacci, I'm going to assume x is an integer greater than or equal to 0.
I'm going to return Fibonacci of x.
And you can see now it says, if either x is equal to 0 or x is equal to 1 I'm going to return 1, otherwise, reduce it to two simpler versions of the problem but with different arguments, and I add them up.
OK, and if we go look at this, we can actually run this, if I can find my code.
Which is right there, and I'm just going to, so we can, for example, check it by saying fib of 0.
I just hit a bug which I don't see.
Let me try it again.
I'll try it one more time with fib of 0.
Darn, it's wrong, let me try it.
I've got two different versions of fib in here, that's what I've got going on.
So let me do it again, let's do fib of 1.
There we go, fib of 2 which is 2, fib of 3 just three, and fib of 4 which should add the previous two, which gives me 5.
There we go.
Sorry about that, I had two versions of fib in my file, which is why it complained at me.
And which is why you should always read the error instructions because it tells you what you did wrong.
Let's go on and look at one more example of doing recursion, and we're going to do dictionaries, and then we're going to pull it all together.
So far we've been doing recursion on numerical things, we can do it on non-numerical things.
So a nice way of thinking about this is, how would I tell if a string of characters is a palindrome?
Meaning it reads the same backwards and forwards.
Probably the most famous palindrome is attributed to Napoleon "Able was I ere I saw Elba."
Given that Napoleon was French, I really doubt he said "Able was I ere I saw Elba," but it's a great palindrome.
Or another one attributed to Anne Michaels "Are we not drawn we few drawn onward to a new era," reads the same backwards and forwards.
It's fun to think about how do you create the palindromes.
I want to write code to solve this.
Again, I want to think about it recursively, so here's what I'm going to do.
I'm first going to take a string of characters, reduce them all to lowercase, and strip out spaces and punctuation.
I just want the characters.
And once I got that, I want to say, is that string, that list of characters or that collection of characters as I should say, a palindrome?
And I'm going to think about it recursively, and that's actually pretty easy.
If it's either 0 or 1 long, it's a palindrome.
Otherwise you could think about having an index at each end of this thing and sort of counting into the middle, but it's much easier to say take the two at the end, if they're the same, then check to see what's left in the middle is a palindrome, and if those two properties are true, I'm done.
And notice what I just did I nicely reduced a bigger problem to a slightly smaller problem.
It's exactly what I want to do.
OK?
So it says to check is this, I'm going to reduce it to just the string of characters, and then I'm going to check if that's a palindrome by pulling those two off and checking to see they're the same, and then checking to see if the middle is itself a palindrome.
How would I write it?
I'm going to create a procedure up here, isPalindrome.
I'm going to have inside of it two internal procedures that do the work for me.
The first one is simply going to reduce this to all lowercase with no spaces.
And notice what I can do because s is a string of characters.
I can use the built in string method lower, so there's that dot notation, s.lower.
It says.
apply the method lower to a string.
I need an open and close per end to actually call that procedure, and that will mutate s to just be all lowercase.
And then I'm going to run a little loop, I'll set up answer or ans to be an empty string, and then, for everything inside that mutated string, I'll simply say, if it's inside this string, if it's a letter, add it into answer.
If it's a space or comma or something else I'll ignore it, and when I'm done just return answer, strips it down to lowercase.
And then I'm going to pass that into isPal which simply says, if this is either 0 or 1 long, it's a palindrome, returned true.
Otherwise, check to see that the first and last element of the string are the same, notice the indexing to get into the last element, and similarly just slice into the string, ignoring the first and last element, and ask is that a palindrome.
And then just call it, and that will do it.
And again there's a nice example of that in the code I'm not going to run it, I'll let you just go look at it, but it will actually pull out something that checks, is this a palindrome.
Notice again, what I'm doing here.
I'm doing divide-and-conquer.
I'm taking a problem reducing it, I keep saying this, to a simpler version of the same problem.
Keep unwinding it till I get down to something I can solve directly, my base case and I'm done.
And that's really the heart of thinking about recursive solutions to problems.
I would hope that one of the things I remember, besides my really lousy patter up here, is the idea of Towers of Hanoi, because to me it's one of the nicest examples of a problem that would be hard to solve iteratively, but when you see the recursive solution is pretty straightforward.
Keep that in mind as you think about doing recursion.
OK, let's switch gears, and let's talk very briefly about another kind of data type called a dictionary.
And the idea of a dictionary I'm going to motivate with a simple example.
There's a quiz coming up on Thursday.
I know you don't want to hear that, but there is, which means we're going to be recording grades.
And so imagine I wanted to build a little database just to keep track of grades of students.
So one of the ways I could do it, I could create a list with the names of the students, I could create another list with their grades, and a third list with the actual subject or course from which they got that great.
I keep a separate list for each one of them, keep them of the same length, and in essence, what I'm doing here is I'm storing information at the same index in each list.
So Ana, who's going to have to take the class again, gets a B, John, who's created the class, gets an A plus, Sorry Ana, John's had a longer time at it.
All right, bad jokes aside, what I'm doing is I can imagine just creating lists.
I could create lists of lists, but a simple way is to do lists where basically at each index I've got associated information.
It's a simple way to deal with it.
Getting a grade out takes a little bit of work because if I want to get the grade associated with a particular student, what would I do?
I would go into the name list and use the method index, which you've seen before, again notice the dot notation it says, this is a list, use the index method, call it on student, and whatever the value of student is, it will find that in the list, return the index at that point, and then I can use that to go in and get the grade in the course and return something out.
Simple way to do it but a little ugly, right, because among other things, I've got things stored in different places in the list.
I've got to think about if I'm going to add something to the list I've got to put them in the same spot in the list.
I've got to remember to always index using integers which is what we know how to do with lists, at least so far.
It would be nice if I had a better way to do it, and that's exactly what a dictionary is going to provide for me.
So rather than indexing on integers I'd like to index directly on the item of interest.
I'd like to say where's Ana's record and find that in one data structure.
And so, whereas a list is indexed by integers, and has elements associated with it, a dictionary is going to combine a key, or if you like, a name of some sort, with an actual value.
And we're going to index just by the name or the label as we go into it.
So let me show you some examples.
First of all, to create a dictionary I use curly braces, open closed curly brace, so an empty dictionary would be simply that call.
If I want to create an actual dictionary, before I insert things into it, I use a little bit of a funky notation.
It is a key or a label, a colon, and then a value, in this case the string Ana and the string b, followed by a comma which separates it from the next pairing of a key and a label, or a key and a value, and so on.
So if I do this what it does in my dictionary is it creates pairings of those labels with the values I associated with them.
OK, these are pretty simple, but in fact, there's lots of nice things we can do with it.
So once we've got them indexing now is similar to a list but not done by a number, it's done by value.
So if that's my key, I can say, what's John's grade, notice the call, it's grades, which is in my dictionary, open close square brackets, with the label John.
And what it does, it goes in and finds that in the dictionary, returns the value associated with it.
If I ask for something not in the dictionary, it's going to give me a key error.
Other things we can do with dictionaries, we can add entries just like we would do with lists.
Grades as a dictionary, in open and closed square brackets, I put in a new label and a value, and that adds that to the dictionary.
I can test if something's in the dictionary by simply saying, is this label in grades, and it simply checks all of the labels or the keys for the dictionary to see if it's there, and if it's not returns false.
I can remove entries, del, something we've seen before, a very generic thing.
It will delete something, and in this case, it says, in the dictionary grades, find the entry associated with that key, sorry, Ana, you're about to be flushed, remove it.
She's only getting a b in the class and she teaches it.
We've got to do something about this, right?
So I can add things, I can delete things, I can test if things are there.
Let me show you a couple of other things about dictionaries.
I can ask for all of the keys in the dictionary.
Notice the format, there is that dot notation, grades as a dictionary, it says, use the keys method associated with this data structure dictionaries.
Open close actually calls it, and it gives me back a collection of all the keys in some arbitrary order.
I'm going to use a funny term here which I'm not certain we've seen so far.
It returns something we call an iterable, it's like range.
Think of it as giving us back the equivalent of a list, it's not actually a list, but it's something we can walk down.
Which is exactly why I can then say, is something in a dictionary, because it returns this set of keys, and I can test to see something's in there.
I can similarly get all of the values if I wanted to look at them, giving us out two iterables.
Here are the key things to keep in mind about dictionaries.
The values can be anything, any type, mutable, immutable.
They could be duplicates.
That'd actually makes sense, I could have the same value associated, for example, the same grade associated with different people, that's perfectly fine.
The values could be lists, they could be other data structures, they could even be other dictionaries.
They can be anything, which is great.
The keys, the first part of it are a little more structure.
They need to be unique.
Well duh, that make sense.
If I have that same key in two places in the dictionary, when I go to look it up, how am I going to know which one I want?
So it needs to be unique, and they also need to be immutable, which also makes sense.
If I'm storing something in a key in the dictionary, and I can go and change the value of the key, how am I going to remember what I was looking for?
So they can only be things like ints, floats, strings, tuples, Booleans.
I don't recommend using floats because you need to make sure it's exactly the same float and that's sometimes a little bit challenging, but nonetheless, you can have any immutable type as your key.
And notice that there's no order to the keys or the values.
They are simply stored arbitrarily by the Python as it puts them in.
So if I compare these two, lists or ordered sequences indexed by integers, I look them up by integer index, and the indices have to have an order as a consequence.
Dictionaries are this nice generalization, arbitrarily match keys to values.
I simply look up one item by looking up things under the appropriate key.
All I require is that the keys have to be immutable.
OK, I want to do two last things I've got seven minutes to go here.
I want to show you an example of using dictionaries, and I'm going to do this with a little bit more interesting, I hope, example.
I want to analyze song lyrics.
Now I'm going to show you, you can already tell the difference between my age and Ana's age.
She used Taylor Swift and Justin Bieber.
I'm going to use The Beatles.
That's more my generation.
Most of you have never heard of The Beatles unless you watched Shining Time Station where you saw Ringo Starr, right?
OK, what I'm going to do is, I want to write a little set of procedures that record the frequencies of words in a song lyric.
So I'm going to match strings, or words, to integers.
How many times did that word appear in the song lyric?
And then I want to ask, can I easily figure out which words occur most often, and how many times.
Then I'm going to gather them together to see what are the most common words in here.
And I'm going to do that where I'm going to let a user say, I want every word that appears more than some number of times.
It's a simple example, but I want you to see how a mutation of the dictionary gives you a really powerful tool for solving this problem.
So let's write the code to do that.
It's also in the handout, here we go.
Lyrics to frequency's, lyrics is just a list of words, strings.
So I'm going to set up an empty dictionary, there's that open close curly brace, and here's what I want to do.
I'm going to walk through all the words in lyrics.
You've seen this before, this is looping over every word in lyrics.
Ah, notice what I'm going to do.
I'm going to simply say-- so the first part is, I can easily iterate over the list, --but now I'm going to say, if the word is in the dictionary, and because the dictionary is iterable, it's simply going to give me back all of the keys, it's simply going to say, in this case, if it's in the dictionary, it's already there, I've got some value associated with it, get the value out, add 1 to it, put it back in.
If it's not already in the dictionary, this is the first time I've seen it, just store it into the dictionary.
And when I'm done just return the dictionary.
OK?
So I'm going to, if I can do this right with my Python, show you an example of this.
I have put in one of the great classic Beatles songs, you might recognize it right there.
Mostly because it's got a whole lot of repetitions of things.
So she loves you yeah, yeah, yeah, yeah.
Sorry, actually they sing it better than I just did it sarcastically.
Sorry about that, but I got she loves you there, and here's my code up here, lyrics to frequency.
So let's see what happens if we call it.
And we say lyrics to frequencies she loves you.
And it would help if I can type, all right, we'll try it one more time, lyrics to frequency's, she loves you.
Cool, this gave me back a dictionary, you can see the curly braces, and there are all the words that appear in there and the number of times that they appear.
What's the order?
You don't care.
You don't know.
What we want to do is to think about how can we analyze this, so let's go back and look at the last piece of this.
Which is, OK, I can convert lyrics to frequencies.
So here's the next thing I want to do, how do I find the most common words?
Well, here's what I'm going to do, frequencies is the dictionary, something that I just pulled out.
So I can use the values method on it which returns and iterable, as I said earlier, again notice the open close because I got to call it.
That gives me back an iterable that has all of the frequencies inside of there, because it's an iterable, I can use max on it, and it will take that editable and give me back the biggest value.
I'm going to call that best, I'm going to set up words to be an empty list, and then I'm just going to walk through all of the entries in the dictionary saying, if the value at that entry is equal to best add that entry into words, just append it onto the end of the list.
And when I'm done all of that loop, I'm just going to return a tuple of both the collections of words that period that many times and how often they appeared.
I'm going to show you an example in a second, but notice I'm simply using the properties of the dictionary.
The last thing I want to do then is say, I want to see how often the words appear.
So I'm going to give it a dictionary and a minimum number of times.
And here I'm going to set result up to be an empty list, I'm going to create a flag called false, it's going to keep track of when I'm done.
And as long as I'm not yet done, I'll call that previous procedure that's going to give me back the most common words and how often they appeared.
I check and remember it was a tuple, how often do they appear, if it's bigger than the thing I'm looking for, I'll add that into my result.
And then the best part is, I'm now going to walk through all the words that appeared that many times, and just delete them from the dictionary.
I can mutate the dictionary.
And by doing that, I can go back around and do this again, and it will pull out how many times has this appeared and keep doing it.
When I can go all the way through that, if I can't find any more, I'll set the flag to true which means it will drop out of here and return the result.
I'm going to let you run this yourself, if you do that, you'll find that it comes up with, not surprisingly, I think yeah is the most common one and she loves you, followed by loves and a few others.
What I want you to see here is how the dictionary captured the pieces we wanted to.
Very last one, there's Fibonacci, as we called it before.
It's actually incredibly inefficient, because if I call it, I have to do all the sub calls until I get down to the base case, which is OK.
But notice, every other thing I do here, I've actually computed those values.
I'm wasting measures, or wasting time, it's not so bad with fib of 5, but if this is fib of 20, almost everything on the right hand side of this tree I've already computed once.
That means fibs very inefficient.
I can improve it by using a dictionary, very handy tool.
I'm going to call fib not only with a value of n, but a dictionary which initially I'm going to initialized to the base cases.
And notice what I do, I'm going to say if I've already computed this, just return the value in the dictionary.
If I haven't, go ahead and do the computation, store it in the dictionary at that point, and return the answer.
Different way of thinking about it, and the reason this is really nice is a method called memoization, is if I call fib of 34 the standard way it takes 11 million plus recursive calls to get the answer out.
It takes a long time.
I've given you some code for it, you can try it and see how long it takes.
Using the dictionary to keep track of intermediate values, 65 calls.
And if you try it, you'll see the difference in speed as you run this.
So dictionaries are valuable, not only for just storing away data, they're valuable on procedure calls when those intermediate values are not going to change.
What you're going to see as we go along is we're going to use exactly these ideas, using dictionaries to capture information, but especially using recursion to break bigger problems down into smaller versions of the same problem, to use that as a tool for solving what turn out to be really complex things.
And with that, we'll see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, everyone.
Good afternoon.
Let's get started.
So today's lecture will be on testing, debugging, and then exceptions and assertions.
So before we begin, let's start with an analogy to sort of come back to real life for a second.
So I've made soup before.
Perhaps you've made soup before.
Let's say you're making soup in this big pot here.
And it turns out that bugs keep falling into your soup from the ceiling.
All right.
Quick question to the audience.
What do you do if you encountered this issue?
[INTERPOSING VOICES]
All right.
Hands up.
One one at a time.
Anyone have any idea?
Yeah
Eat it
Eat it.
You want to eat it Anyway OK. All right.
We're going for an analogy here with computer programming.
I don't know what you'd do if you have a buggy program, I guess you just release it to the customer and they'd complain, but.
OK. What else?
Yeah
[INAUDIBLE] Cover the soup?
Cover the soup.
That's a good suggestion.
Yeah.
So you can cover the soup, so put a lid on it.
Sometimes you'd have to open up, take the lid off, right, to check to make sure it's done.
To taste it, add things.
So bugs might fall in in between there.
But covering the soup is a good idea.
What else.
Yeah
Debug it
Debug it.
I wish I had something for that answer.
All right.
That's a good answer.
Yeah
Take all the food out of your house so there's no-- nothing for the bugs to eat
So take all the food out of your house so there's nothing for the bugs to eat.
That's sort of the equivalent of cleaning, like doing a mass cleaning of your entire house.
That's a good, that's a good one.
That's sort of eliminating the source of the bugs, right?
What else?
Yeah, John
Decide it's high protein and declare it a feature
Decide it's high protein and declare it a feature.
That's probably what a lot of people would do, right?
All right.
Cool.
So I wish computer debugging was as fun as taking bugs out of your soup.
So what did we decide?
Well we could check the soup for bugs.
Keep the lid closed, that was a good suggestion.
And cleaning your kitchen, which someone suggested.
The equivalent of cleaning their kitchen was to just throw out all the food.
I would take a mop and clean the floor, but yeah, that works too.
So we can draw some parallels for this analogy with computer programming.
So checking the soup is really equivalent to testing, right?
You have a soup you think has bugs in it.
Test it.
Make sure there's no bugs.
Continue on.
Keeping the lid closed.
It's sort of this idea of defensive programming.
So make sure that bugs don't fall in in the first place.
Sometimes you have to open the lid to make sure that the soup is tastes good or whatever.
So that's equivalent to defensive programming.
So try not to have bugs in the first place, but they might show up anyway.
Cleaning the kitchen is eliminating the source of the bugs in the first place.
This is actually really hard to do in programming.
But you can still try to do it.
OK.
So let's talk a little bit about programming so far in 60001 600.
So you expect, really, that you write a program, you maybe do a little debugging, and you run the program and it's perfect.
Right?
You just nailed it.
But in reality you write this really complex piece of code and you go to run it and it crashes.
Right?
It's happened to me many times.
It's happened to you many times.
That's the reality.
OK.
So today's lecture will go over some tips and tricks and debugging and how you can help make your life easier when you're writing programs so you don't end up like this little girl here.
Disappointed beyond belief.
All right.
So at the heart of it all is really starting with a defensive programming attitude.
OK. And this comes back to the idea of decomposition and abstraction that we talked about when we started-- when we did the lecture on functions.
Right?
So try to start out with two modularize your code, right?
If you write your code in different blocks, documenting each different block, you're more likely to understand what's happening in your code later on and you'll be able to test it and debug it a lot easier.
Speaking of testing and debugging, once you've written a program that's modular, you still have to test it.
And the process of testing is really just coming up with inputs.
Figuring out what outputs you expect.
And then running your program.
Does the output that the program give match what you expected?
If it does, great, you're done.
But if it doesn't, you have to go to this debugging step.
And the debugging step is the hardest part.
And it's really just figuring out why the program crashed, or why the program didn't give you the answer that you expected it to give.
So as I mentioned, the most important thing is to do defensive programming and to that end, you want to set yourself up for easy testing and debugging.
Which really comes down to making sure that the code you write is modular.
So write as many functions as you can.
Document what the functions do.
Document their constraints.
And it'll make your life a little bit easier later on when you have to debug it.
When do you want to start testing?
Well first you have to make sure your program runs.
So eliminate syntax errors and static semantic errors which, by the way, Python can easily catch for you.
Once you've ensured that a piece of code runs, then you want to come up with some test cases.
So this is pairs of input and output for what you expect the program to do.
Once you have your test cases and a piece of code that runs, you can start doing tests.
So there's three general classes of tests that you can do.
The first is called unit testing.
And if you've written functions, unit testings-- testing just makes sure that, for example, each function runs according to the specifications.
So you do this multiple times.
As you're testing each function, you might find a bug.
At that point, you do regression testing.
Come up with a test case that found that bug.
And run all of the different pieces of your code again to make sure that when you fix the bug, you don't re-introduce new bugs into pieces of the code that had already run.
So you do this a bunch of times.
You do a little bit of unit testing, a little bit of regression testing, and keep doing that.
At some point, you're ready to do integration testing.
Which means, test your program as a whole.
Does the overall program work?
So this is the part where you take all of the individual pieces, put them together.
And integration testing tests to make sure that the interactions between all of the different pieces works as expected.
If it does, great, you're done.
But if it doesn't, then you'll have to go back to unit testing, and regression testing, and so on.
So it's really a cycle of testing.
So what are some testing approaches?
The first, and this is probably most common with programs that involve numbers, is figuring out some natural boundaries for the numbers-- for the program, sorry.
So for example, if I have a function is_bigger, and it compares if x is bigger than y, then some natural boundary, given the specification, is if x is less than y, x is greater than y, x is equal to y.
Maybe throw in less than or equal to or greater or equal to, and so on.
So that's just sort of an intuition about the problem.
It's possible you have some problems for which there are no natural partitions.
In which case, you might do some random testing, and then the more random testing you do, the greater the likelihood that your program is correct.
But there's actually two more rigorous ways to do testing.
And one is black box testing and the other one is glass box testing.
In black box testing, you're assuming you have the specifications to a function.
So that's the docstring.
All you're looking at is the docstring and coming up with some test cases based on that.
In glass box testing, you have the code itself and you're trying to come up with some test cases that hit upon all of the possible paths through the code.
All right.
Let's look at an example for black box testing.
I'm finding the square root of x to some close enough value given by this epsilon.
And the idea here, notice I don't actually give you how this function's implemented.
The idea is that you're just figuring out test cases based on the specification.
And the great thing about black box testing is that whoever implements this function can implement it in whatever way they wish, they can use approximation method, that can use bisection method, it doesn't matter.
The test cases that you come up with for this function are going to be exactly the same.
Right?
No matter what the implementation.
So for this particular function, here's a sample set.
We check the boundary, we check perfect squares, we can check some number that's less than 1, we can check maybe irrationals, and then you do extreme tests.
So when either epsilon is really large or epsilon is really small, or x is really large or x is really small, and all the possible combinations of those.
So the important thing about black box testing is that you are doing you are creating the test cases based on the specifications only.
Glass box testing, you're using the code itself to guide your test cases.
So if you have a piece of code and you come up with a test case that goes through every single possible combination of input-- of every single possible path through the code, then that test set is called path complete.
The problem with this is when you encounter loops, for example.
Every single possible path through a loop is maybe the code not going through the loop at all, going through once, going through twice, going through three times, four times, five times, and so on.
Right?
Which could be a very, very big test.
So instead there are actually some guidelines for when you're dealing with loops and things like that.
So for branches, when you're doing glass box testing, it's important-- you should just exercise all of the parts of the conditional.
So make sure you have a test case that goes through each part of the conditional.
For for loops, make sure you have a test case where the loop is not entered at all, where the loop is entered one time, and when the loop is entered just some number more than once.
For while loops, similar to for loops, except that make sure you have test cases that cover all of the possible ways to break out of the while loop.
So if the while loop condition becomes false, or if maybe there's a break inside the while loop, and so on.
So in this example, we have the absolute value of x.
This is its specification and this is the implementation that someone decided to do for this function.
So a path complete test set means that you want to have a test that goes through each one of these branches.
So if x is less than minus 1, well, minus 2 is less than minus 1.
So that's good.
And otherwise, which means pick a number greater than minus 1.
So 2.
So 2 and minus 2 are path complete.
Yield path complete-- yields a path complete test suite.
But notice that while we've hit upon every possible path through this code, we've actually missed a test case.
Minus 1.
So this code incorrectly classifies minus 1 as returning minus 1, which is wrong.
So for glass box testing, in addition to making sure you're going through all the possible paths through the code, you also want to make sure you hit upon any boundary condition.
So in this case, for branches, minus 1 is a boundary condition.
So you've created a test suite, you've tested your program, chances are you found a bug.
What do you do now?
All right.
Quick sort of detour into a little bit of history.
The history of debugging.
So 1947, this computer was built.
And it was a computer that was very impressive for its day.
It could do things like addition in 0.1 seconds.
Things like multiplication in 0.7 seconds.
And take the log of something in five seconds.
So faster than a human, possibly.
But pretty slow for today's standards.
And a group of engineers were working on running a program that found-- that was supposed to find the trigonometric function.
And among them being this-- one of the first female scientists, Grace Hopper.
And they found that their program was not working correctly.
So they went through all of the panels and all of the relays in the computer, and they isolated a program in panel F relay 70, where they found this moth.
Just sitting in there.
I think it was dead, probably electrocuted.
But it was a moth that was impeding the calculation.
And I don't know if you can read this, but this part right here.
They made a note in their logbook that says, first actual case of bug being found.
Which I think is really cute.
So they were literally doing debugging in this computer.
Right.
All right.
So you won't be doing that sort of debugging.
You'll be doing a virtual kind of debugging in your programs.
Which, again, is not that fun.
But you still have to do it.
So debugging, as you might have noticed so far in your problem sets, has a bit of a steep learning curve.
And obviously your goal is to have a bug free program, and in order to achieve that, you have to do the debugging.
There are some tools which some of you have been using.
There are some tools built into Anaconda, or whatever ID you've been using to do debugging.
I know some of you have been using the Python tutor, which is awesome.
The print statement can also be a good debugging tool.
But over above everything else, it's really important to just be systematic as you're trying to debug your program.
I want to talk a little bit about print statements and how you can use them to debug, because I think-- Python tutor, if you don't have the internet, you might not be able to use it.
If you don't know how to use the debugger, you don't need to learn.
But print statements, you'll always have them, and you can always put them in your program.
And they're really good ways to test hypotheses.
So good places to put print statements are inside functions.
Inside loops, for example, what are the loop parameters, what are the loop values, what function-- what functions return what values.
So you can make sure that values are being passed-- the correct values are being passed between parts of your code.
I will mention that you can use the bisection method when you're debugging.
Which is interesting.
So if you take a print statement, find approximately the halfway point in your code.
Print out what values you-- print out some relevant values.
All of the possible-- print out some values at that point in your code.
If everything is as you expect it to be at that point in your code, then you're good.
That means the code so far is bug free.
That means that-- however, that means that the code beyond it has a bug, right?
So since you've put a print statement halfway in your code and you think that gave good results, then put a print statement 3/4 of the way in the code.
And see if the values are as you expect at that point.
And if they are, great.
Then put a print statement further down.
So in this way you could use the bisection method to pinpoint a line, or a set of lines, or maybe a function that that's giving you the bad results.
So the general debugging steps is to study the program code.
Don't ask what is wrong, because that's actually part of the testing.
So your test cases would have figured out what's wrong.
The debugging process is figuring out how the result took place.
And since programming is-- programming and debugging is, sort of, is a science, you can use the scientific method as well.
So look at all the data, that's your test cases.
Figure out a hypothesis.
Maybe say, oh, maybe I'm indexing from 1 instead of 0 in lists, for example.
Come up with an experiment that you can repeat.
And then pick a simple test case then you can test your hypothesis with.
So as you're debugging, you will encounter error messages.
And these error messages are actually pretty easy to figure out.
And they're really easy to fix in your code.
So for example, accessing things beyond the limits of the lists give you index errors.
Trying to convert, in this case, a list to an integer gives you type errors.
Accessing variables that you haven't created before gives you name errors.
And so on and so on.
And syntax errors are things, for things like, if you forget a parentheses, or forget a colon, or something like that.
So error messages are really easy to spot.
The Python interpreter spits these out for you and then you can pinpoint the exact line.
Logic errors are actually the hard part.
And logic errors are the ones that you will be spending the most time on.
For which I would recommend always trying to take a break.
Take a nap, go eat.
Something.
Sometimes you'd have to start all over, so throughout the code you have and just sit down with a piece of paper, try to figure out how you want to solve the problem.
And if you look up the term rubber ducky-- a lot of heads went up on that one-- rubber ducky debugging.
That is an actual term in Wikipedia.
And it's when a programmer explains their code to a rubber ducky.
That's me on the left explaining code to my rubber ducky.
You should always-- you should go buy one.
Or code to anyone else, preferably someone who doesn't really understand anything.
Because that'll force you to explain everything really, really closely.
And as you're doing that, you'll figure out your problem.
And I figured out my problem in both of these cases.
So just go back to the basics.
Quick summary of dos and don'ts of debugging and testing.
So don't write the entire program, test the entire program, and debug the entire program.
I know this is really tempting to do, and I do it all the time.
But don't do it.
Because you're going to introduce a lot of bugs and it's going to be hard to isolate which bugs are affecting other ones.
And it'll lead to a lot more stress than you need.
Instead do unit testing.
So write one function, test the function, debug the function, make sure it works, write the other function, and so on and so on.
Do a little regression testing, a little more unit testing, a little integration testing, and it's a lot more systematic way to write the program.
And it'll cut down on your debugging time immensely.
If you're changing your code, and inevitably you'll be changing your code as you're doing your problem sets, remember to back up your code.
So if you have a version that almost works, don't just modify that and maybe save a copy.
[INAUDIBLE] you've got terabytes of memory on your computer, it won't hurt to just make a quick copy of it.
Document maybe what worked and what didn't in that copy.
And then make another copy, and then you can modify your code.
So that's sort of a high level introduction to testing and debugging.
The rest of the class will be on the error messages, or on errors that you will get in your programs.
So when your functions-- when you run functions, or when you run your program, at some point, the program execution is going to stop.
Maybe it encountered an error because of some unexpected condition.
And when that happens you get an exception.
So the error is called an exception.
And it's called an exception because it was an exception to what was expected.
To what the program expected.
So all of these errors that I've talked about in the previous slides are actually examples of exceptions.
And there are actually many other types of exceptions, which you'll see as you go on in this course and also in 60002.
So how do we deal with these exceptions?
In Python, you can actually have handlers for exceptions.
So if you know that a piece of code might give you an error.
For example, here I'm dealing with inputs from users.
And users are really unpredictable.
You tell them to give you a number, they might give you their name.
Nothing you can do about that.
Or is there?
Yes there is.
So in your program you can actually put any lines of code that you think might be problematic, that might give you an error an exception, in this try block.
So you say try colon, and you put in any lines of code that you think might give you an error.
If none of these lines of code actually produce an error, then great.
Python doesn't do anything else.
It treats them as just part-- as just if they were part of a regular program.
But if an error does come up-- for example, if someone doesn't put in a number but puts their name in-- that's going to raise an error, specifically a value error.
And at that point, Python's going to say, is there an accept statement?
And if so, this except statement is going to handle the error.
And it's going say, OK, an error came up, but I know how to handle it.
I'm going to print out this message to the user.
So if we look at code-- this is the same code as in the slides-- and there's no try except block around it.
So if I run it and I say, three and four, it's going to run fine.
But if I run it and I say, [INAUDIBLE] a, it's going to give a value error.
Now if I run the same piece of code with try-- with a try except block.
I run it, if I give it regular numbers, it's fine.
But if I'm being a cheeky user, and I say three, automatically this would have raised the value error in the previous version of the program.
But in this version of the program, the programmer handled the exception or caught the exception, and printed out this nicer looking message.
So bug in user input is nicer than this whole lot here.
A lot easier to read.
So any problematic lines of code, you can put in a try block, and then handle whatever errors might come up in this except block.
This except block is going to catch any error that comes up.
And you can actually get a little bit more specific and catch specific types of errors.
In this case, I'm saying, if a value error comes up-- for example, if the user inputs a string instead of an integer-- do this, which is going to print this message.
If the user inputs a number for B such that we're doing a divided by b, so that would give a 0 division error.
In that case we're going to catch this other error here, the 0 division error, and we're going to print this other message, can't divide by 0.
So each-- so you can think of these different except blocks as sort of if else if statements, except for exceptions.
So we're going to try this.
But if there's a value error do this.
Otherwise, if there's a division error, do this.
And otherwise do this.
So this last except is actually going to be for any other error that comes up.
So if it's not a value error, nor a division error, then we're going to print, something went very wrong.
I couldn't even try to create-- I couldn't even try to make the program come up with any other error besides those two.
So a lot of the time you're just going to use try except blocks.
But there's other blocks that you can add to exceptions.
And these are more rarely used, but I'll talk about them anyway.
So you could have an else block.
And an else block is going to get executed when the code in the try block finished without raising an error.
And you can also have a finally block, which is always executed.
If the code in the try block finished without an error, if you raised an exception, if you raised a different kind of exception, if you went through the else, in any of these cases, whatever's in the finally block is always going to get executed.
And it's usually used to clean up code.
Like if you want to print, oh, the program finished, or if you want to close a file, or something like that.
So.
We've encountered errors.
We've caught them.
What else can we do with errors-- with exceptions.
Three other things.
So one is if we've caught an error, we can just fail silently.
What this means is, you've caught an error, and you just substitute whatever erroneous value the user gave you for some other value.
That's not actually a very good idea.
That's a bad idea.
Because suddenly the user thinks that they entered something, and they think everything's great, your program accepts it, but then they get some weird value as an output, which is far from what they expected.
So it's not really a good idea to just replace user's values with anything.
In the context-- so this is in the context of a function.
In the context of a function, what else can we do?
Well, if you have a function that fails, for example, let's say you're trying to do you're trying to get the square root of an even number.
And let's say the user gives you a-- sorry, you're trying to find the square root of a positive number.
And let's say the user gives you a negative number.
Well, if the user gives you a negative number, your function could return an error value, which means, well if the number inputted is less than 0, then return 0.
Or minus 1.
Or minus 100.
Just pick any value to return which represents some error value.
This is actually not a good idea either, because later on in your program, if you're using this function, now you have to do a check.
And the check is, well if the return from this function is minus 1 or minus 100, do this.
Otherwise, do this.
So you you're complicating your code because now you always have to have this check for this error value.
Which makes the code really messy.
The other thing we can do is we can signal an error condition.
So this is how you create control flow in your programs with exceptions.
So in Python, signaling an error condition means raising your own exception.
So so far we've just seen the programs crashing, which means they raise an exception and then you deal with them.
But in this last case, you're raising your own exception.
As a way to use that exception later on in the code.
So in Python, you raise your own exception using this raise keyword and then an exception.
And then some sort of description, like "user entered a negative number" or something like that.
A lot of the time we're going to raise a value error.
So if the number is less than 0, then raise a value error, something is wrong.
The key word, the name of the error, and then some sort of descriptive string.
So let's see an example of how we raise an exception.
I have this function here called get ratios.
It takes in two lists, L1 and L2.
And it's going to create a new list that's going to contain the ratio of each element in L1 divided by each element in L2.
So I have a for loop here.
For index in range length L1.
So I'm going through every single element in L1.
I'm going to try here.
I'm going to try to do this line.
So I think that this line might give me an error.
So I'm going to put it in a try block.
The error I think I'm going to get is a 0 division error, because what happens when an element and L2 is 0?
And when an element in L2 is 0 I'm going to append this not a number as a float.
So NAN, as a string, you can convert it to a float, and it stands for not a number.
So then I can continue populating the list with these not a numbers.
If an element and L2 is 0.
And otherwise, if there's no 0 division error, but there's another kind of error, then I'm going to raise my own error.
And say, for any other kind of error, just raise a value error.
Which says, "get ratios was called with a bad argument."
So here I'm sort of consolidating all errors into my one value error.
So later on in my program, I can catch this value error and do something with it.
Here's another example of exceptions.
So let's say we're were given a class list.
We have a list of lists.
Where we have the name of a student, first name and last name, and their grades in the class.
So we currently have two students.
And what I want to do is create a new list which is the same things, the same inputs here.
But I'm adding an extra-- I'm appending an extra value at the end of the list for each student, which is the average of all of their grades.
Or all of their-- yeah, all of their grades.
So let's look at the code.
This is the function that takes the class list, which is this whole list here.
I'm creating a new list inside it, initially empty.
And then I'm going for every element in the class list.
I'm appending element at 0, which is going to be this first list here.
So it's going to be the first name and the last name.
Element at 1, which is the grades.
And then the last thing I'm appending is a function call.
The function call being called with element 1, which is all of the grades, and this is my function call.
We're going to see three different function calls.
This is the first one.
It simply takes the sum of the grades and divides it by the length of the grades.
If these students are responsible, and they've taken all of the tests, then there's no problem.
Because length of grades is going to be something greater than 0.
But what if we have a student in the class who didn't show up for any tests?
Then we have no record of any of their tests.
No record of grades or anything like that.
So they're going to have an empty list.
So if we run this function, averages, on their data, we're actually going to get a 0 division error, because we're trying to divide by length of grades, which is going to be 0.
So what can we do?
Two things, two options here.
One is we can just flag the error and print the message.
So here there's a new average function, an improved one, that's going to try to do the exact same line as the previous one.
And it's going to catch the 0 division error.
And when it catches it, it's going to print this warning.
And when we run it, we're going to get, "warning, no grades data," which is fine.
And we're going to get this "none" here, for the grades.
So everyone else's grades was calculated correctly, and for this last one, we got a none.
That's because, when we entered this except statement, if this is a function, remember functions return something.
This function in this particular except statement didn't return anything.
So it returns a none.
So for the averages for this particular function, the average is going to be a "none" for this person who didn't have any grades associated with them.
And yeah, so that's basically what I said.
So that's our first option, is to just flag the error and print a message.
The other option is to actually change the policy.
So this is where you replace the data with some sort of default value.
And if you do something like this, then this should be documented inside the function.
So when you write the docstring for the function, you would say if the list is empty, then it'll will a 0.
So this is the exact same thing as before.
We have a try and an except for the 0 division error.
We also print a warning, no grades data.
And then we return the 0.
So we still flag the error, and now instead of a "none," we get a 0, because we've returned 0.0 here, as opposed to just leaving it blank.
All right.
So those are exceptions.
Last thing we're going to talk about today are these things called assertions.
And assertions are good example of defensive programming.
In that, you have assert statements at the beginning of functions, typically.
Or at the end of functions.
And assert statements are used to make sure that the assumptions on computations are exactly what the function expects them to be.
So if we have a function that says it's supposed to take in an integer greater than 0, then the assert statement will assert that the function takes in an integer that's greater than 0.
Here's an example.
This is the same average function we've seen before.
Here, instead of using exceptions, we're going to use an assert statement.
And the assert statement we're putting right at the front.
At the beginning of the function, sorry.
And the key word is assert.
The next part of the assert is what the function expects.
So we expect that the length of grades is not equal to 0.
So has to be greater than 0.
And then we have a string here, which represents what do you print out if the assertion does not hold.
So if you run the function, and you give it a list that is empty, this becomes false, so the assert is false, and we're going to print out an assertion error, no grades data.
If the assert is false, the function does not continue.
It stops right there.
Why does it behave this way?
Well, assertions are great to make sure that preconditions and post-conditions on functions are exactly as you expect.
So as soon as an assert becomes false, the function's going to immediately terminate.
This is useful because it'll prevent the program from propagating bad values.
So as soon as a precondition isn't true, for example, as you enter a function, then that means something went wrong in your program.
And the program is going to stop right there.
So instead of propagating a bad value throughout the program, and then you getting an output that you didn't expect, and then you having to trace back to the function that gave this bad value, you'll get this bad value, you'll get this assert being false a lot earlier.
So it'll be a lot easier to figure out where the bug came from.
And you won't have to trace back so many steps.
So this is basically what I said, you really want to spot the bugs as soon as they're introduced.
And exceptions are good if you want to raise them when the user supplies bad data input, but assertions are used to make sure that the types and other-- the types of inputs to functions, maybe other conditions on inputs to functions, are being held as the values are being passed in.
So the keyword here is making sure that the invariants on data structures are meant.
And that's it.
Great.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's look at this exercise.
So we have this function is even, same one as before, except now I'm giving you this implementation.
If n is positive and n divided by 2's remainder is 0, return true.
So if n is even and positive return true.
The next one, if n is negative and divisible by 2 return true.
OK, so far.
And otherwise return false.
Question being, with that implementation is this test set n is 4, n is minus 4 path complete?
And the answer is, yes.
Because 4 is a positive number and divisible by 2.
Minus 4 is a negative number and divisible by 2.
And 5 would hit upon the else.
So the answer is actually yes for that first question.
Perfect.
Second question, with that implementation, which value for n is incorrectly labeled by that program?
Well, n is very large still works, and is very small still works.
And remember, I said you have to test boundary conditions.
In this case, boundary conditions for this program being when n is equal to zero.
So I think the orange is n is equal to 0.
Perfect.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's see if you can figure out this next error.
Oh, nice.
Looks like 100% are getting it right so far.
So we have L is equal to 3 for i in range length L print i.
And this is the error message telling me the file, the line number, and the actual line that was wrong and then a description of the error type error.
OK?
And everyone's getting it right, which is, what's the problem here?
We're not allowed to call length on an integer.
OK?
If you go through the other choices, you'll see that those you can actually do.
And you can even test those out yourself before you settle on the right answer.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The code looks a little bit daunting, but it's not that bad.
Really the only part of the code that actually does something is this part right here, where we're trying to do these three lines.
The first being, we get an input from the user that says how old are you, we're converting it to an integer.
We're going to take that number that they gave us, divide by 80.
I'm assuming life expectancy is 80, and multiply by 100 to get the percent, going around it to 1.
And then I'm going to print, you've gone through that much percent of your life.
OK, so again we're dealing with user input which is unpredictable, so we're going to catch, first of all, a value error, and that's if the user doesn't enter a number.
In that case we're going to say, oops, you must enter a number.
And then we're going to also catch the zero division error which says print trying to do a division by 0.
And otherwise, we'll have a last except that, so something went very wrong.
So looks like people are getting it right.
The question being, if the user enters 20, what does the program do?
OK, so we're giving string, and strings taken in as input and then trying to convert those to an integer gives us a value error.
So it's going to go through this except block here and print that, which is great, most of you guys are getting that right.
And then the next one, perfect, I tricked some of you.
The next one says, if the user enters zero, what does the program do?
Well what does the program do?
We're going to take in zero, we're going to do 0 times 100 divide by 80, that's fine, it's always going to be zero.
We're going around that to one, and then we're going to print that out.
So no zero division error, no value error, everything went right, so that means it's going to print out this.
The first one.
So that's this one here.
So the zero division error only happens when you're trying to divide something by zero.
In this case we're not, the zero's at the top of the fraction.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right everyone.
Let's get started.
So today's lecture and Wednesday's lecture, we're going to talk about this thing called object oriented programming.
And if you haven't programmed before, I think this is a fairly tough concept to grasp.
But hopefully with many, many examples and just by looking at the code available from lectures, you'll hopefully get the hang of it quickly.
So let's talk a little bit about objects.
And we've seen objects in Python so far.
Objects are basically data in Python.
So every object that we've seen has a certain type.
OK, that we know.
Behind the scenes, though, every object has these two additional things.
One is some data representation.
So how Python represents the object just behind the scenes and what are different ways that you can interact with the object.
So for example, every one of these is a different object.
For example, this is the number 1,234.
It's a specific object that is of type integer.
The number 5 is a different object that's of type integer and so on.
We've seen floats.
We've seen strings.
We've seen lists.
Lists and dictionaries are more complicated objects.
Object types.
Sorry.
But every object has a type, some sort of way that it's represented in Python and some ways that we can interact with them.
OK.
So the idea behind object oriented programming is, first of all, everything in Python is an object.
We've said that before and in this lecture I think we'll really get at what that means.
So we've seen strings, integers, dictionaries, lists.
Those are all objects.
When we did functions, we saw that we could pass as a parameter another function.
So functions were also objects in Python.
So literally everything in Python is an object.
So what are the kinds of things we can do with objects?
Well, once you have a type, you can create a new object that is of some type.
And you can create as many objects as you'd like of that particular type, right?
An integer 5 and integer 7.
Those all work in a program.
Once you've created these new objects, you can manipulate them.
So for a list, for example, you can append an item to the end of the list, you can delete an item, remove it, concatenate two lists together.
So that's ways that you can interact with objects.
And the last thing you can do is you can destroy them.
So and with lists, we saw explicitly that you can delete elements from a list, or you can just forget about them by reassigning a variable to another value, and then at some point, Python will collect all of these dead objects and reclaim the memory.
So let's continue exploring what objects are.
So let's say I have these two separate objects.
One is a blue car.
One is a pink car.
So objects are really data abstractions.
So these two cars can be created by the same blueprint.
OK?
This is a blueprint for a car and if an object is a data abstraction, there's two things that this abstraction is going to capture.
The first is some sort of representation.
What is going to represent the car, what data represents a car object?
And the second is what are ways that we can interact with the object?
So if we think about a car blueprint, some general representation for a car could be the number of wheels it has, the number of doors it has, maybe its length, maybe its height, so this is all part of what data represents the car.
OK?
The interface for the car is what are ways that you can interact with it.
So for example, you could paint a car, right?
So you could change its color.
You could have the car make a noise and different cars might make different noises.
Or you can drive the car, right?
So these are all ways that you can interact with the car.
Whereas the representation are what makes up the car.
What data abstractions make up the car.
Let's bring it a little closer to home by looking at a list.
So we have this data type of list, right?
We've worked with lists before.
The list with elements 1, 2, 3, and 4 is a very specific object that is of type list.
Again, we think about it in terms of two things.
One is what is the data representation of the list?
So behind the scenes how does Python see lists?
And the second is, how do you interact with lists?
So what are ways that you can manipulate a list object once it's created?
So behind the scenes you have a list, L, which is going to be made up of essentially two things.
One is going to be the value at specific index.
OK?
So at index 0, it has the value 1, right, because it's the first element in the list.
And the second thing that represents a list is going to be this second part, which is a pointer.
And internally this pointer is going to tell Python where is the memory location in the computer where you can access the element index 1.
So it's just essentially going to be a chain, going from one index to the other.
And at the next memory location you have the value at index 1, and then you have another pointer that takes you to the location in memory where the index 2 is located.
And in index 2 you have the value and then the next pointer, and so on and so on.
So this is how Python internally represents a list.
OK?
How you manipulate lists, we've done this a lot, right?
You can index into a list, you can add two lists together, you can get the length, you can append to the end of a list, you can sort a list, reverse a list, and so many other things, right?
So these are all ways that you can interact with the list object as soon as you've created it.
So notice both of these, the internal representation and how you manipulate lists, you don't actually know internally how these are represented, right?
How did whoever wrote the list class decide to implement a sort.
We don't know.
You also weren't aware of how these lists were represented internally.
And you didn't need to know that.
That's the beauty of object oriented programming and having these data abstractions.
The representations are private of these objects and they are only known by what you can find out how it's done, but they only should be known by whoever implemented them.
You, as someone who uses this class, doesn't really need to know how a list is represented internally in order to be able to use it and to write cool programs with them.
OK?
So just find a motivation here before we start writing our own types of objects is the advantages of object oriented programming is really that you're able to bundle this data, bundle some internal representation, and some ways to interact with a program into these packages.
And with these packages, you can create objects and all of these objects are going to behave the exact same way.
They're going to have the same internal representation and the same way that you can interact with them.
And ultimately, this is going to contribute to the decomposition and abstraction ideas that we talked about when we talked about functions.
And that means that you're going to be able to write code that's a lot more reusable and a lot easier to read in the future.
OK.
So just like when we talked about functions, we're going to sort of separate the code that we talk about today into code where you implement a data type and code where you use an object that you create.
OK?
So remember when we talked about functions, you were thinking about it in terms of writing a function, so you had to worry about the details of how you implement a function.
And then you had to worry about just how to use a function, right?
So it's sort of the same idea today.
So when you're thinking about implementing your own data type, you do that with this thing called a class.
And when you create a class, you're basically going to figure out what name you want to give your class and you're going to find some attributes.
And attributes are going to be the data representation and ways that you can interact with your object.
So you, as the programmer of this class, are going to decide how you want people to interact with the object and what data this object is going to have.
So for example, someone wrote code that implements a list class, right, and we don't actually know how that was done.
But we can find out.
So creating the class is implementing the class and figuring out data representation and ways to interact with the class.
Once that's done, you can then use your class.
And you use the class by creating new instances of the class.
So when you create a new instance, you essentially create a new object that has the type, the name of your class.
And you can create as many objects as you'd like.
You can do all the operations that you've defined on the class.
So for example, someone wrote the code to implement list class and then you can just use the list class like this.
You can create a new list, you can get the length pf the list, you can append to the end of the list, and so on and so on.
So let's start defining our own types, OK?
So now you're going to define classes, you're going to write classes which are going to define your own types of objects.
So for today's lecture we're going to look at code that's going to be in the context of a coordinate object.
And a coordinate object is essentially going to be an object that's going to define a point in an xy plane.
So x, y is going to be a coordinate in a 2D plane.
So we're going to write code that's going to allow us to define that kind of object.
So the way we do that is we have to define a class.
So we have to tell Python, hey, I'm defining my own object type.
So you do that with this class key word.
So you say class, then you say the name of your type.
In this case, we're creating a type called coordinate.
Just like we had type list, type string, and so on.
This is going to be a type called coordinate.
And then in parentheses here, you put what the parents of the class are.
For today's lecture, the parent of the classes are going to be this thing called object, and object is the very basic type in Python.
It is the most basic type in Python.
And it implements things like being able to assign variables.
So really, really basic operations that you can do with objects.
So your coordinate is therefore going to be an object in Python.
All right.
So we've told Python we wanted to define an object.
So inside the class definition we're going to put attributes.
So what are attributes?
Attributes are going to be data and procedures that belong to the class, OK?
Data are going to be the data representations and procedures are going to be ways that we can interact with the object.
The fact that they belong to the class means that the data and the procedures that we write are only going to work with an object of this type.
OK.
If you try to use any of the data or the procedures with an object of a different type, you're going to get an error because these data and these attributes will belong to this particular class.
So the data attributes is, what is the object, right?
What is the data that makes up the object?
So for our coordinate example, it's going to be the x and y values for coordinate.
We can decide that can be ints, we can decide that we can let them be floats, but it's going to have one value for the x-coordinate and one value for the y-coordinate.
So those are data attributes.
And procedure attributes are better known as methods.
And you can think of a method as a function.
Except that it's a function that only works with this particular type of object.
So with a coordinate object, in this case.
So the methods are going to define how you can interact with the object.
So in a list, for example, we've said that you can append an item to the end of the list, we can sort a list, things like that.
So when you're defining methods, you're defining ways that people can interact with your object.
So for example, for a coordinate object, we can say that we can take the distance between two coordinate points.
OK?
And that's going to be a way that you can interact with two coordinate points.
And just to be clear, these are going to belong to this class, which means that if you try to use this distance method on two lists, for example, you're going to get an error.
Because this distance method was only defined to work with two coordinate type objects.
All right, so let's carry on and continue implementing our class.
So we've written this first line so far, class coordinate object.
So now let's define attributes.
First thing we're going to define are data attributes.
Generally you define data attributes inside this init, and this is underscore, underscore, init, underscore, underscore, and it's a special method or function in a class.
And the special method tells Python, when you implement the special method, it tells Python when you first create an object of this type, call this method or call this function.
So how do we do that?
So let's implement it.
So we say df because it's just a function.
The name is the special name, init.
And we give it some parameters, right, just like any other function.
These last two parameters are x and y, which are going to represent how you create a coordinate object.
So you give it a value for the x-coordinate and you give it a value for the y-coordinate.
The self, however, is a little bit trickier.
So the self is going to be a parameter when you define this class that represents a particular instance of the class.
So we're defining this coordinate object in sort of a general way, right?
We don't have a specific instance yet because we haven't created an object yet.
But this self is going to be sort of a placeholder for any sort of instance when you create the object.
So in the definition of the class, whenever you want to refer to attributes that belong to an instance, you have to use self dot.
So this dot notation.
And the dot is going to say look for a data attribute x that belongs to this class.
So for methods that belong to the class, the first parameter is always going to be self.
It can be named anything you want, but really by convention it's always named self.
So try to stick to that.
And then any other parameters beyond it are going to be just parameters as you would put in a normal function.
OK.
In this particular case, we're going to choose to initialize a coordinate object by two values, one for the x and one for the y.
And inside this init method, we're going to have two assignments.
The first one says, the x data attribute of a coordinate object.
I'm going to assign it to whatever was passed in.
And the y data attribute for a particular object is going to be assigned whatever y was passed in.
Questions so far about how to write this init?
Yeah, question
[INAUDIBLE]
How do you make sure that x and y are inits or floats?
So this is something that you could write in the specifications, so the docstring with the triple quotes.
So whoever uses the class would then know that if they do something outside the specification, the code might not work as expected.
Or you could put in a cert statement inside the definition of the init just to sort of force that.
Force that to be true.
Great question.
Yeah, question
[INAUDIBLE]
Does the x, does this self x and this x have to be the same name.
The answer is no.
And we're going to see in class exercise that you can have it be different.
OK. Great.
So this defines the way that we create an object.
So now we have sort of a nice class.
It's very simple, but we can start actually creating coordinate objects.
So when you create coordinate objects, you're creating instances of the class.
So this line here, C is equal to coordinate 3,4, is going to call the init method.
It's going to call the init method with x is equal to 3 and y is equal to 4.
I'm just going to go over here and I wrote this previously, because notice when we're creating an object here, we're only giving it two parameters.
But in the init method, we have actually three parameters, right?
We have these three parameters here, but when we're creating an object, we only give it two parameters.
And that's OK because implicitly, Python is going to say self is going to be this object C, so just by default, OK?
So when you're creating a coordinate object, you're passing it all the variables except for self.
So this line here is going to call the init and it's going to do every line inside the init.
So it's going to create an x data attribute for C, a y data attribute for C, and it's going to assign 3 and 4 to those respectively.
This next line here is origin equals coordinate 0, 0 creates another object.
OK?
It's another coordinate object whose value for x is 0 and whose value for y is 0.
So now we have two coordinate objects.
We can access the data attributes using this dot notation and we've seen that before, right?
When we've worked with lists we'd say something like, L dot append, right, when we create a list.
So the same dot notation can be used with your own objects in order to access data attributes.
So here, this is going to print 3 because the x value for object C is 3, and the next line, print origin x is going to print 0 because the x value for the object origin is 0.
OK.
So we've created a coordinate object.
We have to find the init method so we have a way to create objects when we use the class.
And then we can access the data attributes.
But that's kind of lame, right, because there isn't anything cool we can do with it.
There isn't ways to interact with this object.
So let's add some methods.
Remember methods are going to be procedural attributes that allow us to interact with our object.
Methods are like functions except that there's a couple of differences which you'll see in a moment.
And when you're calling methods, you're using the dot operator, like L dot append, for example, for lists.
So let's go back to defining our coordinate class and let's define a method for it.
So so far we've defined that part there, class coordinate and an init.
So we have that.
So in this slide we're going to add this method here.
So this method here is going to say I'm going to define a method called distance and I'm going to pass in two parameters.
Remember self, the first parameter, is always going to be the instance of an object that you're going to perform the operation on.
So pretty much by convention it's always named self.
And then for this particular method, I'm going to give it another parameter, and I can name this whatever I want.
I'm naming it other.
And this is going to represent the other coordinate object for which I want to find the distance from my self.
So here I'm going to just implement the Euclidean distance formula, which is x1 minus x2 squared, plus Y1 minus Y2 squared, and square root of all that.
So that's what I'm doing inside here.
Self and other are coordinate objects.
Inside this method, I have to refer to the x data attributes of each object if I want to find the difference between the 2x values from them.
So that's why I'm doing self dot x here, right.
If I just did x, I would be accessing just some variable named x in a program which actually isn't even defined.
So you always have to refer when as we're thinking about classes, you always have to refer to whose data attribute do you want to access?
In this case, I want to access the x data attribute of my self, and I want to subtract the x data attribute of this other coordinate, square that, same for y, square that, and then add those and take the square root of that.
So notice this method is pretty much like a function, right?
You have DF, some name, it takes in parameters.
It does some stuff and then it returns a value.
The only difference is the fact that you have a self here as the first thing and the fact that you always have to be conscious about whose data attributes you're accessing.
So you have to use the dot notation in order to decide whose data attributes you want access.
So we've defined the method here, distance.
So this is in the class definition.
Now how do we use it?
So let's assume that the definition of distance is up here.
I didn't include the code.
But really all you need to know is what it takes.
It takes a self and an other.
So when you want to use this method to figure out a distance between two coordinate objects, this is how you do it.
So the first line, I create one coordinate object.
Second line, I create another coordinate object.
First one is named C, the second one is named 0.
These are two separate objects.
And I'm going to find the distance.
And I want to first call it on one object, so I'm going to say C dot, so I'm using the dot notation to call the method distance on object C. So Python says this object C is of type coordinate.
It's going to look up at the class coordinate that you defined.
It's going to find this method called distance and then it's going to say what parameters does it take?
So it takes another parameter, right, for the other and then, in the parentheses, I just have to give it this other perimeter.
An easier way to see what happens is by looking at what this line here is equivalent to.
So the third line here prints C dot distance 0 is equivalent to this one on the right.
And this one on the right essentially says, what's the name of the class, dot, dot notation, what's the method you want to call, and then in parentheses you give it all of the variables including self.
OK.
So in this case you're explicitly telling Python that self is C and other is 0.
So this is a little bit easier to understand, like that.
But it's a little cumbersome because you always have to write coordinate dot, coordinate dot, coordinate dot, for every data attribute you might want to access, for every procedural attribute you might want to access.
So by convention, it's a lot easier to do the one on the left.
And as I mentioned, Python implicitly says, if you're doing the one on the left, you can call this method on a particular object and it's going to look up the type of the object and it's going to essentially convert this on the left to the one on the right.
And this is what you've been using so far.
So when you create a list, you say L is equal to 1, 2, and then you say L.append, you know, 3 or whatever.
So we've been using this notation on the left pretty much from the beginning of class.
So we have a coordinate class, we can create a coordinate object, we can get the distance between two objects.
As you're using the class, if you wanted to use this coordinate class, and you were maybe debugging at some point, a lot of you probably use print as a debug statement, right?
And maybe you want to print the value of a coordinate object.
So if you create a coordinate object, C is equal to coordinate 3, 4, right?
That's what we've done so far.
If you print C, you get this funny message.
Very uninformative, right?
It basically says, well, C is an object of type coordinate at this memory location in the computer.
Which is not what you wanted at all, right?
Maybe you wanted to know what the values for x and y were.
That would be a lot more informative.
So by default, when you create your own type, when you print the object of that type, Python tells you this sort of information which is not what you want.
So what you need to do is you need to define your own method that tells Python what to do when you call print on an object of this type.
So this is going to be a special method, just like init is, because it starts and ends with double underscores.
And the name of the method is underscore, underscore, str, underscore, underscore.
And if you define this method in your class, that tells Python, hey, when you see a print statement that's on an object of type coordinate, call this method, look what it does, and do everything that's inside it.
And you can choose to make it do whatever you want inside your definition of str.
In this case, let's say when we print a coordinate object, we're going to print its x and y values surrounded by angle brackets.
That seems reasonable, right?
So then from now on when you print coordinate objects, you're going to see things like this, which is a lot more informative.
So how do we define this?
So so far we've defined all that and the last part is going to be new.
So we define the init and the distance, and let's define this str.
So underscore, underscore, str, underscore, underscore, is a method.
It's only going to take self because you're just calling print on the object itself.
There's no other parameters to it.
Str has to return a string, and in this particular case, we're going to return the string that's the angle brackets concatenated with the x value of the object, self.x, concatenated with a comma, concatenated with the y value of this particular instance of an object, self.y, and then concatenated with the angle brackets.
So now any time you have print on an object of type coordinate, you're going to call this special method str, if it's implemented in your code.
Any questions?
OK.
So let's try to wrap our head around types and classes because we've seen a lot today.
Let's create a coordinate object, assign it 3, 4, as we have been, and assign it to variable C. We've implemented the str method, so when we print C, it's going to print out this nice three comma for our angle brackets.
If we print the type of C, this is actually going to give us class main coordinate, which tells us that C is going to be an object that is of type class coordinate.
If we look at coordinate as a class, if we print what coordinate is, coordinate is a class, right?
So this is what Python tells us, if we print coordinate, it's a class named coordinate.
And if we print the type of a coordinate, well that's just going to be a type.
So class is going to be a type.
So you're defining the type of an object.
If you'd like to figure out whether a particular object is an instance of a particular class, you use this special function called is instance.
So if you print is instance C comma coordinate, this is going to print true because C is an object that is of type coordinate.
Couple more words on these special operators.
So these special operators allow you to customize your classes which can add some cool functionality to them.
So these special operators are going to be things like addition, subtraction, using the equal equal sign, greater than, less than, length and so on and so on.
So just like str, if you implement any of these in your classes, this is going to tell Python.
So for example, if we've implemented this underscore, underscore, add, underscore, underscore in our class, this is going to tell Python when you use this plus operator between two objects of type coordinate to call this method.
If you have not implemented this method and you try to add two objects of type coordinate, you're going to get an error because Python doesn't actually know right off the bat how to add two coordinate objects, right?
You have to tell it how to do that.
And you tell it how to do that by implementing this special method.
Same with subtract.
Same with equals.
So if you want to figure out whether two objects are equal.
And when you implement these methods in your own class, you can decide exactly what you want to do.
So what happens when you add two coordinate objects?
Do you just add the x values, do you just add the y values, do you get them both together, do you do whatever you'd like to do.
And then you document what you've decided.
So let's create a fraction object.
So we've looked at coordinate, we saw sort of a higher level car object.
Let's look at a fraction object.
Fraction object is going to be, is going represent a number that's going to be a numerator slash denominator.
OK.
So that's going to be a fraction object.
So the way I've decided to internally represent a fraction object is with two numbers.
And I've decided that I will not let them be floats.
They have to be integers, hence the assert over here.
So inside the init, I've decided I'm going to represent my fracture with two numbers, one for the numerator and one for the denominator.
So when I create a fraction object, I'm going to pass in a numerator and a denominator.
And a particular instance is going to have self dot numerator and self dot denominator as its data attributes and I'm assigning those to be whatever's passed into my init.
Since I plan on debugging this code maybe possibly sometime in the future, I'm also including an str method and the str method is going to print a nice looking string that's going to represent the numerator, and then a slash, and then the denominator.
And then I've also implemented some other special methods.
How do I add two fractions?
How do I subtract two fractions?
And how do I convert a fraction to a float?
The add and subtract are almost the same, so let's look at the add for the moment.
How do we add two fractions?
We're going to take self, which is the instance of an object that I want to do the add operation on, and we're going to take other, which is the other instance of an object that I want to do the operation on, so the addition, and I'm going to figure out the new top.
So the new top of the resulting fraction.
So it's my numerator multiplied by the other denominator plus my denominator multiplied by the other numerator and then divided by the multiplication of the two denominators.
So the top is going to be that, the bottom is going to be that.
Notice that we're using self dot, right?
Once again, we're trying to access the data attributes of each different instance, right, of myself and the other object that I'm working with.
So that's why I have to use self dot here.
Once I figure out the top and the bottom of the addition, I'm going to return, and here notice I'm returning a fraction object.
It's not a number, it's not a float, it's not an integer.
It's a new object that is of the exact same type as the class that I'm implementing.
So as it's the same type of object, then on the return value I can do all of the exact same operations that I can do on a regular fraction object.
Sub is going to be the same.
I'm returning a fraction object.
Float is just going to do the division for me, so it's going to take the numerator and then divide it by the denominator, just divide the numbers.
And then I'm defining here my own method called inverse.
And this is just going to take the inverse of the instance I'm calling this method on.
And so it's going to also return a new fraction object that just has the denominator as the top part and the numerator as the bottom part.
So then we have some code here.
So that's how I implement my fraction object.
So now let's use it and see what it gives us.
A is equal to a fraction 1, 4.
This is going to be 1 over 4 for a.
And b is going to be 3 over four.
When I do C, notice I'm using the plus operator between two fraction objects, right?
A and b are fraction objects so Python's going to say, OK, is there an underscore, underscore, add, underscore, underscore, method implemented?
It is and it's just going to do whatever's inside here.
So it's going to say self dot numerator plus other dot denominator.
It's going to calculate the top and the bottom.
It's going to turn a new fraction object.
So this is going to be 4 plus 12 divided by 16, and 16 over 16.
So C as a fraction object is going to be 16 for the numerator and 16 for the denominator because it's a fraction object.
If I print C, it should print 16 over 16, so we can even run it, so print 16 over 16.
If I print floats C, so this special method float here is going to say, is there a method that converts a fraction to a float and there is.
It's this one implemented right here.
So it's just going to divide the two numbers, top and bottom, which gives me 1.
So it's this one here and here.
Notice I'm doing the exact same method call, except I'm doing it the other way where you type in the name of the class, name of the method, and then what you're calling it on, and this gives the exact same value here, 1.0.
And then here I'm calling the method inverse on object B which is going to invert 3 over 4 to be 4 over 3.
And then I'm converting it to a float and then I'm printing the value.
So it gives me 1.33.
So take a look at this code in more detail and see if you can trace through all of those different things and see if you can also write your own new fraction objects.
OK.
So last slide.
Power of object oriented programming is that you can bundle together objects that are of the exact same type.
And all of these objects are going to have the same data representation and the same methods that you can do on them.
And ultimately, you're going to be building these layers of abstraction.
So you're going to be building on a basic object type in Python, you're going to have integer objects, float objects.
On top of those, you can create lists, dictionaries.
And on top of those, you can even create your own object types as we saw in this lecture today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, which of the following is a good and valid definition for a class representing a car?
OK, most of the people I think have gotten it, which is the red, which says class car object, and that's perfect.
OK.
So this defines a function, this defines a class correctly, but A is just not descriptive name at all, and this is kind of weird, and this is the right one here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We have this definition for the class object.
I've created the init for you.
And here I'm giving it "self," which is the first thing that you have to give it, and then two parameters-- w and d. And for an instance of a car, I'm going to assign the data attribute named wheels to whatever is passed in for w. So notice that they're not the same name.
And the data attribute for "doors" is going to be the value that's passed in for d OK?
And also notice that inside init, I can do any other sort of initializations that I'd like.
So it's not just assigning variables from the parameters to variables for my objects.
So in this case I'm creating a new data attribute named "color," and I'm going to just create it to be an empty string OK, even though I didn't pass in any color to my object.
So the question says, "Using the class definition above, which line creates a new Car object with 4 wheels and 2 doors?"
OK?
So this first one is not right, because it's trying to call the class with a variable mycar for self, which isn't quite right.
So when you're creating your object, you have to give it one less parameter than what you have.
So here we're just going to have to create it with the w and the d. So we're going to create this variable mycar and we're going to assign it 4 for the number of wheels and 2 for the number of doors.
And the number of wheels is the first parameter, and the number of doors is the second one, so it should be this one here-- Car 4 comma 2.
So that's this one.
Perfect.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're given this definition for car.
We saw that on the previous slide.
I want to add a method that's going to change the color of the car, and these are my four choices.
And it looks like you guys are getting it right, which is awesome.
So to find a method to change the color of the car, which does this.
So we know that self has to be the first parameter, so we can automatically eliminate A and C and it's between B and D. And remember I said you have to be conscious about whose data attribute you're accessing, and in this case, we want to change the color of a particular instance of the car, right?
So we have to say self.color instead of just color.
If we just said color, then color would refer to just a variable inside the class, not a data attribute of a particular object.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This is my definition, including the method to paint the car, so to change its color.
And the question says you create a car with this line here.
So you're initializing it with 4 comma 2.
What's a line of code that changes the color from whatever it initially is to red?
The first one, car.paintred, is sort of attempting to do the way that we saw in the right in the slides where you calling the class name dot.
But it's missing the self.
So we don't know what object to call it on.
So that one's out of the running.
The second one is closer, but we have red here as a variable as opposed to a string.
So that one's not going to work.
The third one looks good.
And the fourth one is just weird.
I don't think that one will work.
It's attempting to put a value for self.
But you've already called mycar dot instead of the class name dot.
So here, this isn't going to work.
So since you already called the object name dot method name, you just have to give it the other parameter that it needs, which is the color.
I think the majority have gotten it right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. We have our class.
We've done pretty much everything up until here, and we're going to add this special function here.
So we're implementing this underscore underscore eq underscore underscore.
And implementing this is going to allow us to compare two types of cars.
So the way I've decided to compare two types of cars is I'm saying the two cars are going to be equal if they have the same number of wheels, they have the same color, and if they have the same number of doors.
OK?
So if all of these are equal, then return true and else return false.
Inside the actual program here, I'm creating one car, four wheels, two doors.
I'm changing its color to red.
I'm creating another car-- four wheels, two doors.
By default, this new car, or your car, is going to have the color empty string, because that's how a new car gets initialized, right?
So between my car and your car, the difference is going to be the color.
OK?
So the color is not going to be the same, but they have the same number of wheels and the same number of doors.
So because I implemented the equal method in my code, this does not throw an error.
This lets me proceed.
It compares four with four, which is good, two with two, which is good.
And then the colors don't match, so it's going to say false, which most of the guys are getting it right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, everyone, let's get started.
So today is going to be the second lecture on object-oriented programming.
So just a quick recap of last time-- on Monday, we saw-- we were introduced to this idea of object-oriented programming, and we saw these things called abstract data types.
And these abstract data types we implemented through Python classes.
And they allowed us to create our own data types that sort of abstracted a general object of our choosing, right?
So we've used lists before, for example.
But with abstract data types, we were able to create objects that were of our own types.
We saw the coordinate example.
And then at the end of the class, we saw the fraction example.
So today we're going to talk a little bit more about object-oriented programming and classes.
We're going to see a few more examples.
And we're going to talk about a few other nuances of classes, talk about information hiding and class variables.
And in the second half of the lecture, we're going to talk about the idea of inheritance.
So we're going to use object-oriented programming to simulate how real life works.
So in real life, you have inheritance.
And in object-oriented programming, you can also simulate that.
OK, so the first few slides are going to be a little bit of recap just to make sure that everyone's on the same page before I introduce a couple of new concepts related to classes.
So recall that when-- in the last lecture, we talked about writing code from two different perspectives, right?
The first was from someone who wanted to implement a class.
So implementing the class meant defining your own object type.
So you defined the object type when you defined the class.
And then you decided what data attributes you wanted to define in your object.
So what data makes up the object?
What is the object, OK?
In addition to data attributes, we also saw these things called methods.
And methods were ways to tell someone how to use your data type.
So what are ways that someone can interact with the data type, OK?
So that's from the point of view of someone who wants to write their own object type.
So you're implementing a class.
And the other perspective was to write code from the point of view of someone who wanted to use a class that was already written, OK?
So this involved creating instances of objects.
So you're using the object type.
Once you created instances of objects, you were able to do operations on them.
So you were able to see what methods whoever implemented the class added.
And then, you can use those methods in order to do operations with your instances.
So just looking at the coordinate example we saw last time, a little bit more in detail about what that meant-- so we had a class definition of an object type, which included deciding what the class name was.
And the class name basically told Python what type of an object this was, OK?
In this case, we decided we wanted to name a coordinate-- we wanted to create a Coordinate object.
And the type of this object was therefore going to be a coordinate.
We defined the class in the sort of general way, OK?
So we needed a way to be able to access data attributes of any instance.
So we use this self variable, OK?
And the self variable we used to refer to any instance-- to the data attributes of any instance in a general way without actually having a particular instance in mind, OK?
So whenever we access data attributes, we would say something like self dot to access a data attribute.
You'd access the attribute directly with self.x.
Or if you wanted to access a method, you would say self, dot, and then the method name-- for example, distance.
And really, the bottom line of the class definition is that your class defines all of the data-- so data attributes-- and all of the methods that are going to be common across all of the instances.
So any instance that you create of a particular object type, that instance is going to have this exact same structure, OK?
The difference is that every instance's values are going to be different.
So when you're creating instances of classes, you can create more than one instance of the same class.
So we can create a Coordinate object here using this syntax right here.
So you say the type, and then, whatever values it takes in.
And you can create more than one Coordinate object.
Each Coordinate object is going to have different data attributes.
Sorry, it's going to have different data attribute values, OK?
Every Coordinate object is going to have an x value and a y value.
But the x and y values among different instances are going to vary, OK?
So that's the difference between defining a class and looking at a particular instance of a class.
So instances have the structure of the class.
So for a coordinate, all instances have an x value and a y value.
But the actual values are going to vary between the different instances.
OK, so ultimately, why do we want to use object-oriented programming?
So, so far, the examples that we've seen were numerical, right-- a coordinate, a fraction.
But using object-oriented programming, you can create objects that mimic real life.
So if I wanted to create objects of-- an object that defined a cat and an object that defined a rabbit, I could do that with object-oriented programming.
I would just have to decide, as a programmer, what data and what methods I'd want to assign to these groups of objects, OK?
So using object-oriented programming, each one of these is considered a different object.
And as a different object, I can decide that a cat is going to have a name, an age, and maybe a color associated with it.
And these three here, on the right, each one of these rabbits is also an object.
And I'm going to decide that I'm going to represent a rabbit by just an age and a color, OK?
And with object-oriented programming, using these attributes, I can group these three objects together and these three objects together, OK?
So I'm grouping sets of objects that are going to have the same attributes together.
And attributes-- this is also a recap of last time-- come in two forms, right, data attributes and procedural attributes.
So the data attributes are basically things that define what the object is.
So how do you represent a cat as an object?
And it's up to you, as the programmer, to decide how you want to do that.
For a coordinate, it was pretty straightforward.
You had an x and a y value.
If we're representing something more abstract like an animal, then maybe I would say, well, I'm going to represent an animal by an age and a name, OK?
So it's really up to you to decide how you want to represent-- what data attributes you want to represent your object with.
Procedural attributes were also known as methods.
And the methods are essentially asking, what can your object do, OK?
So how can someone who wants to use your object-- how can someone interact with it?
So for a coordinate, we saw that you could find the distance between two coordinates.
Maybe for our abstract Animal object, you might have it make a sound, OK, by maybe printing to the screen or something like that.
OK, this slide's also a recap of how to create a class just to make sure everyone's on the same page before we go on.
So we defined a class using this class keyword.
And we said, class, the name of the class.
So now we're going to create a more abstract Animal class.
We're going to see, in the second half of the lecture, what it means to put something else in the parentheses.
But for now, we say that an animal is an object in Python.
So that means it's going to have all of the properties that any other object in Python has.
And as we're creating this animal, we're going to define how to create an instance of this class.
So we say def.
And this __init__ was the special method that told Python how to create an object.
Inside the parentheses, remember, we have the self, which is a variable that we use to refer to any instance of the class, OK?
We don't have a particular instance in mind, we just want to be able to refer to any instance, OK?
So we use this self variable.
And then, the second parameter here is going to represent what other data we use to initialize our object with.
So in this case, I'm going to say, I'm going to initialize an Animal object with an age, OK?
So when I create an animal, I need to give it an age.
Inside the __init__ are any initializations that I want to make.
So the first thing is, I'm going to assign an instance variable, age-- so this is going to be the data attribute age-- to be whatever is passed in.
And then, I'm also making another assignment here, where I'm assigning the data attribute name to be None originally.
Later on in the code, when I want to create an Animal object, I say the class name.
And then I pass it in whatever parameters it takes-- in this case, the age.
And I'm assigning it to this instance here, OK?
All right, so now we have this class, Animal.
We've done the first part here, which is to initialize the class, right?
So we've told Python how to create an object of this type.
There's a few other methods here that I've implemented.
Next two we call getters, and the two after that we call setters, OK?
And getters and setters are very commonly used when implementing a class.
So getters essentially return the values of any of the data attributes, OK?
So if you look carefully, get_age() is just returning self.age, and get_name() just returns self.name.
So they're very simple methods.
Similarly, set_age() and set_name()-- we're going to see what this funny equal sign is doing here in the next couple of slides.
But setters do a very similar thing where they're going to set the data attributes to whatever is passed in, OK?
So those are getters and setters.
And then, the last thing down here is this __str__ method.
And this __str__ method is used to tell Python how to print an object of this type Animal.
So if you didn't have this __str__ method, if you remember from last lecture, what ends up happening is you're going to get some message when you print your object that says, this is an object of type Animal at this memory location, which is very uninformative, right?
So you implement this method here, which tells Python how to print an object of this type, OK?
So the big point of this slide is that you should be using getters and setters-- you should be implementing getters and setters for your classes.
And we're going to see, in the next couple of slides, why exactly.
But basically, they're going to prevent bugs from coming into play later on if someone decides to change implementation.
So we saw how to-- so the previous slide, this slide here, shows the implementation of the Animal class.
And here we can see how we can create an instance of this object.
So we can say a = Animal(3).
So this is going to create a new Animal object with an age of 3.
And we can access the object through the variable a.
Dot notation, recall, is a way for you to access data attributes and methods of a class, OK?
So you can say a.age later on in your program, and that is allowed.
It'll try to access the age data attribute of this particular instance of the class, a.
So this is going to give you 3.
However, it's actually not recommended to access data attributes directly.
So this is the reason-- so you're going to see in the next slide, the reason-- why we're going to use getters and setters.
Instead, you should use the get_age() getter method to get the age of the animal.
So this is going to return, also, 3.
So these are going to do the same thing.
And the reason why you'd want to use getters and setters is this idea of information hiding, OK?
So the whole reason why we're using classes in object-oriented programming is so that you can abstract certain data from the user, OK?
One of the things you should be abstracting is these data attributes.
So users shouldn't really need to know how a class is implemented.
They should just know how to use the class, OK?
So consider the following case.
Let's say whoever wrote the Animal class wants to change the implementation.
And they've decided they don't want to call the data attribute "age" anymore, they want to call it "years," OK?
So when they initialize an animal they say self.years = age.
So an animal still gets initialized by its age.
And the age gets passed into a data attribute named "years," OK?
Since I'm implementing this class, I want to have a getter, which is going to return self.years.
So I'm not returning self.age anymore, because age is no longer the data attribute I'm using.
So with this new implementation, if someone was using this implementation and was accessing age directly as-- was accessing the data attribute age directly-- with this new implementation, they'd actually get an error, right?
Because this animal that they created using my old implementation no longer has an attribute named "age."
And so Python's going to spit out an error saying no attribute found or something like that, OK?
If they were using the getter a.get_age()-- the person who implemented the class re-implemented get_age() to work correctly, right, with their new data attribute, years, as opposed to age-- so if I was using the getter get_age(), I wouldn't have run into the bug, OK?
So things to remember-- write getters and setters for your classes.
And later on in your code, use getters and setters to prevent bugs and to promote easy to maintain code.
OK, so information hiding is great.
But having said that, Python's actually not very great at information hiding, OK?
Python allows you to do certain things that you should never be doing.
OK.
So the first, we've just seen.
The first is to access data attributes from outside of the class, OK?
So if I were to say a.age, Python allows me to do that without using a getter and setter.
Python also allows you to write to data attributes from outside the class.
So if I implemented the class Animal assuming that age was a number, an integer, and all of my methods work as long as age is an integer, but someone decided to be smart and, outside of the class, set age to be infinite as a string, that might cause the code to crash, OK?
Python allows you to do that.
But now you're breaking the fact that age has to be an integer, right?
So now the methods should probably be checking the fact that age is an integer all the time.
The other thing that you're allowed to do is to create data attributes outside of the class definition, OK?
So if I wanted to create a new data attribute called "size" for this particular instance, Python also allows me to do that.
And I can set it to whatever I want, OK?
So Python allows you to do all these things, but it's actually not good style to do any of them.
So just don't do it.
All right.
So the last thing I want to mention-- the last thing about classes before we go on to inheritance-- is this idea called default arguments.
And default arguments are passed into methods.
And since methods are functions, you can also pass in different arguments to functions.
So for example, this set_name() method had self.
And then, this new name is equal to this empty string here, OK?
We haven't seen this before.
But this is called a default argument.
And you can use the function in one of two ways.
The first way is so we can create a new instance of an Animal type object with this line here, a = Animal(3).
And then we can say a.set_name().
So this calls the setter method to set the name.
And notice, we've always said that you have to put in parameters for everything other than self, OK?
But here we have no parameters passed in.
But that's OK, because newname actually has a default argument, OK?
So that tells Python, if no parameter is passed in for this particular formal parameter, then use whatever is up here by default.
So if I haven't passed in the parameter a.get_na-- a.set_name(), sorry-- a.sett_name() is going to be setting the name to the empty string, because that's what the default parameter is.
So in the next line, when I print a.get_name(), this is just going to print the empty string, OK?
If you do want to pass in a parameter, you can do so as normal.
So you can say a = Animal(3), a.set_name(), and then pass in a parameter here.
And then, newname is going to be assigned to whatever parameter is passed in like that.
Whatever you pass in overrides the default argument, and everything is good.
So when I print a.get_name(), this is going to print out the name that you've passed in.
Questions about default?
Yeah
[INAUDIBLE]
What if you don't provide a default value for--
For newname?
For newname?
If you don't provide a default argument for newname and you do this case here, then that's going to give you an error.
So Python's going to say something like, expected one argument, got zero, or something like that.
Great question.
OK. All right, so let's move on to this idea of hierarchies, OK?
So the great thing about object-oriented programming is that it allows us to add layers of abstraction to our code, all right?
So we don't need to know how very, very low-level things are implemented in order to use them.
And we can build up our code to be more and more complex as we use up these different abstractions.
So consider every one of these things on this slide as being a separate object, all right?
Every one of these things can be considered to be an animal, OK?
According to our implementation of an animal, the one thing that an animal has is an age, OK?
And that's probably true, right?
Every one of these things has an age.
But now I want to build up on this and create separate groups, right?
And each one of these separate groups that I create on top of Animal is going to have its own functionality, right?
They're going to be a little bit more specific, a little more specialized.
So I can create these three groups now, a cat, a rabbit, and a person group.
And for example-- so they're all animals, right?
They all have an age.
But for example, maybe a person's going to have a list of friends whereas a cat and a rabbit do not.
Maybe a cat has a data attribute for the number of lives they have left, right, whereas a person and a rabbit do not, OK?
So you can think of adding these more specialized-- adding functionality to each one of these subgroups, OK?
So they're going to be more and more specialized, but all of them retaining the fact that they are animals.
So they all have an age, for example.
So on top of these, we can add another layer and say that a student is a person and is an animal, OK?
But in addition to having an age and maybe also having a list of friends, a student might also have a major or-- they're pretty, so maybe-- their favorite subject in school.
So that's the general idea of hierarchies, OK?
So we can sort of abstract the previous slide into this one and say that we have parent classes and child classes, OK?
The Animal class is like our parent class.
It's the highest-level class.
Inheriting from the Animal class, we have these child classes or subclasses, OK?
Whatever an animal can do, a person can do.
Whatever an animal can do, a cat can do.
And whatever an animal can do, a rabbit can do, OK-- that is, have an age and maybe some really basic functionality, OK?
But between person, cat, and rabbit, they're going to be varying wildly as to the kinds of things that they can do, right?
But they can all do whatever Animal can do.
So child classes inherit all of the data attributes and all of the methods, or behaviors, that their parent's classes have, OK?
But child classes can add more information.
Like for example, a person can have a list of friends whereas a general animal will not.
It can add more behavior.
Like, maybe a cat can climb trees whereas people and rabbits cannot.
Or you can also override behavior.
So in the previous one, we had animal, person, student.
So maybe we have, an animal doesn't speak at all, but a person can speak.
So that's added functionality to the person.
And maybe a person can only say hello.
But then, when we talk to a student, we can override the fact-- override the speak() method of a person and say that a student can say, you know, I have homework, or I need sleep, or something like that, OK?
So we have the same speak() method for both person and student, because they can both speak.
But student will override the fact that they say hello with something else.
OK, so let's look at some code to put this into perspective.
So we have this Animal class, which we've seen before.
This is the parent class, OK?
It inherits from object, which means that everything that a basic object can do in Python, an animal can do, which is things like binding variables, you know, very low-level things, OK?
We've seen the __init__.
We've seen the two getters, the setters, and the string method to print an object of type Animal.
All right, now, let's create a subclass of Animal.
We'll call it Cat, OK?
We create a class named Cat.
In parentheses, instead of putting "object," we now put "Animal."
And this tells Python that Cat's parent class is Animal.
So everything that an animal can do, a cat can do.
So that includes all of the attributes, which was age and name, and all of the methods.
So all the getters, the setters, the __str__, the __init__, everything that the animal had, now the cat has-- the Cat class has.
In the Cat class, we're going to add two more methods though.
The first is speak().
So speak() is going to be a method that's going to just take in the self, OK-- no other parameters.
And all it's doing is printing "meow" to the screen-- very simple, OK?
So through this speak(), we've added new functionality to the class.
So an animal couldn't speak, whereas a cat says "meow."
Additionally, through this __str__ method here, we're overriding the animal __str__, OK?
So if we go back to the previous slide, we can see that the animal's __str__ had animal, plus the name, plus the age here whereas the cat's __str__ now says "cat," name, and the age, OK?
So this is just how I chose to implement this, OK?
So here I've overridden the __str__ method of the Animal class.
Notice that this class doesn't have an __init__, and that's OK. Because Python's actually going to say, well, if there's no __init__ in this particular method-- sorry, in this particular class-- then look to my parents and say, do my parents have an __init__, OK?
And if so, use that __init__.
So that's actually true for any other methods.
So the idea here is, when you have hierarchies, you have a parent class, you have a child class, you could have a child class to that child class, and so on and so on.
So you can have multiple levels of inheritance.
What happens when you create an object that is of type something that's been-- of a type that's the child class of a child class of a child class, right?
What happens when you call a method on that object?
Well, Python's are going to say, does a method with that name exist in my current class definition?
And if so, use that.
But if not, then, look to my parents.
Do my parents know how to do that, right?
Do my parents have a method for whatever I want to do?
If so, use that.
If not, look to their parents, and so on and so on.
So you're sort of tracing back up your ancestry to figure out if you can do this method or not.
So let's look at a slightly more complicated example.
We have a class named Person.
It's going to inherit from Animal.
Inside this person, I'm going to create my own-- I'm going to create an __init__ method.
And the __init__ method is going to do something different than what the animal's __init__ method is doing.
It's going to take in self, as usual.
And it's going to take in two parameters as opposed to one, a name and an age.
First thing the __init__ method's doing is it's calling the animal's __init__ method.
Why am I doing that?
Well, I could theoretically initialize the name and the age data attributes that Animal initializes in this method.
But I'm using the fact that I've already written code that initializes those two data attributes.
So why not just use it, OK?
So here, this says, I'm going to call the class Animal.
I'm going to call its __init__ method.
And I'm going to leave it up to you to-- not you as the class, but I'm talking as the programs is running-- I'm going to leave it up to you to figure out how to initialize an animal with this particular age and what to name it.
So Python says, yep, I know how to do this, so I'm going to go ahead and do that for you.
So now it says person is an animal.
And I've initialized the age and the name for you.
The next thing I'm doing in the __init__ is I'm going to set the name to whatever name was passed in, OK?
So in the __init__, notice, I can do whatever I want, including calling methods.
And then, the last thing I'm doing here is I'm going to create a new data attribute for Person, which is a list of friends, OK?
So an animal didn't have a list of friends, but a person is going to.
The next four methods here are-- this one's a getter, so it's going to return the list of friends.
This is going to append a friend to the end of my list.
I want to make a note that I actually didn't write a method to remove friends.
So once you get a friend, they're friends for life.
But that's OK.
The next method here is speak(), which is going to print "hello" to the screen.
And the last method here is going to get the age difference between two people.
So that just basically subtracts their age and says it's a five-year age difference, or whatever it is.
And down here, I have an __str__ method, which I've overridden from the Animal, which, instead of "animal: name," it's going to say "person: name : age," OK?
So we can run this code.
So that's down here.
I have an animal person here.
So I'm going to run this code.
And what did I do?
I created a new person.
I gave it a name and an age.
I created another person, a name and an age.
And here I've just run some methods on it, which was get_name(), get_age(), get_name(), and get_age() for each of the two people.
So that printed, Jack is 30, Jill is 25.
If I print p1, this is going to use the __str__ method of Person.
So it's to print "person:", their name, and then, their age.
p1.speak() just says "hello."
And then, the age difference between p1 and p2 is just 5.
So that's just subtracting and then printing that out to the screen.
OK, so that's my person.
Let's add another class.
This class is going to be a student, and it's going to be a subclass of Person.
Since it's a subclass of Person, it's going to-- a student is going inherit all the attributes of a person, and therefore, all the attributes of an animal.
The __init__ method of a student is going to be a little different from the one of Person.
We're going to give it a name, an age, and a major.
Notice we're using default arguments here.
So if I create a student without giving it a major, the major is going to be set to None originally.
Once again, this line here, Person.
init (self, name, age), tells Python, hey, you already know how to initialize a person for me with this name and this age.
So can you just do that?
And Python says, yes, I can do that for you.
And so that saves you, maybe, like five lines of code just by calling the __init__ method that you've already written through Person, OK?
So Student has been initialized to be a person.
And additionally, we're going to set another data attribute for the student to be the major.
And we're going to set the major to be None.
The student is going to get this setter here, this setter method, which is going to change the major to whatever else they want if they want to change it.
And then, I'm going to override the speak() method.
So the speak method for the person, recall, just said "hello."
A student is going to be a little bit more complex.
I'm going to use the fact that someone created this random class, OK?
So this is where we can write more interesting code by reusing code that other people have written.
So someone wrote a random class that can do cool things with random numbers.
So if I want to use random numbers in my code, I'm going to put this "import random" at the top of my code, which essentially brings in all of the methods from the Random class, one of the methods being this random() method.
So random() is a random() method from the Random class.
And this essentially gives me a number between 0 and 1, including 0 but not including 1, OK?
So this random number I get here is going to help me write my method for speak(), where it's going to-- with 25% probability, it's either going to say, "I have homework," "I need sleep," "I should eat," or "I'm watching TV," OK?
So a student is going to say one of those four things.
And the last thing I'm doing down here is overwriting the __str__ method.
So let's look at the code.
I'm going to comment this part out, and uncomment the student, and see what we get.
OK, so here, I am creating the student.
I'm creating one student whose major is CS, name is Alice, and age is 20. s2 is going to be another student-- name-- Beth, age-- 18.
And the major is going to be None, because I didn't pass in any major here.
So by default, using the default argument, it's going to be None.
If I print s1, s2, that's going to print out these two things over here just by whatever __str__ method does.
And then I'm going to get the students to speak.
And if I run it multiple times, you can see that it's going to print different things each time.
So "I need sleep," "I have homework," "I need sleep," "I have homework," yeah.
So every time, it's going to print something different.
OK, questions about inheritance in this example?
OK. Last thing we're going to talk about in this class is an idea of-- or in this lecture, is the idea of-- a class variable, OK?
So to illustrate this, I'm going to create yet another subclass of my animal called a rabbit.
So class variables-- so so far, we've seen-- sorry, let me back up.
So so far, we've seen instance variables, right?
So things like self.name, self.age, those are all instance variables.
So they're variables that are specif-- they are common across all of the instances of the class, right?
Every instance of the class has this particular variable.
But the value of the variable is going to be different between all of the different instances.
So class variables are going to be variables whose values are shared between all of the instances in the class.
So if one instance of the class modifies this class variable, then, any other instance of the class is going to see the modified value.
So it's sort of shared among all of the different instances.
So we're going to use class variables to keep track of rabbits.
OK, so we're creating this class, Rabbit.
tag = 1.
We haven't seen something like this before.
So tag is our class variable.
Class variables are typically defined inside the class definition but outside of the __init__.
So tag is going to be a class variable, and I'm initializing it to 1.
Inside the __init__, this tells us how to create a Rabbit object.
So I'm going to give it self as usual, an age, and then two parents.
Don't worry about the two parents for now.
Inside the __init__-- sorry, inside the __init__-- I'm going to call the __init__ of the animal just to do less work.
Python already knows how to initialize an animal for me, so let's do that.
So that's going to set the two data attributes, name and age.
I'm going to set the data attributes for parent1, parent2 for a rabbit to be whatever's passed in.
And then, this is where I'm going to use this class variable.
So I'm creating this data attribute instance variable particular to a specific instance called rid, OK?
And I'm assigning this instance variable to the class variable.
And I access class variables using not self, but the class name-- so in this case, rabbit.tag.
So initially, tag is going to be 1.
And then, the __init__ is going to increment the tag by 1 here, OK?
So that means that, from now on, if I create any other instances, the other instances are going to be accessing the updated value of tag instead of being 1.
So let's do a quick drawing to show you what I mean.
So let's say I have Rabbit.tag here, OK?
So initially, tag is going to be 1, OK?
And then I'm going to create a new Rabbit object.
So this is as I'm calling the code, OK?
So let's say this is a rabbit object-- oh boy, OK-- r1.
You know, I actually googled how to draw a rabbit, but that didn't help at all.
OK, so r1 is going to be a new rabbit that we create.
Initially, what happens is, when I first create this new rabbit, it's going to access the class variable, which, it's current value is 1.
So when I create the rabbit ID-- the rabbit ID, r1.rid-- this is going to get the value 1.
And according to the code, after I set the rabbit ID to whatever tag is, I'm going to increment the tag.
So this is going to say, OK, now that I've said it, I'm going to go back up here and increment the tag to be 2.
OK.
So let's say I create another Rabbit object, OK?
All right, there-- that's a sad rabbit, r2.
The ID of r2 is going to be what?
Well, according to the way we create a new Rabbit object is it's going to access whatever the value of tag is, which is a class variable.
It was changed by the previous creation of my rabbit, so now I'm going to access that, right?
So the value is going to be 2.
And according to the code, the next thing I do after I create the instance rid is I'm going to increment tag.
So I'm incrementing the class variable to be 3, OK?
So notice that all of my instances are accessing this shared resource, this shared variable called tag.
So as I'm creating more and more rabbits, they're all going to be incrementing the value of tag, because it's shared among all of the instances.
And so this value, this tag class variable, keeps track of how many different instances of a rab-- of how many different instances of rabbits I've created throughout my entire program, OK?
So the big idea here is that class variables are shared across all the instances.
So they can all modify them.
But these rids, right, these instance variables, are only for that particular instance.
So r2 can't have access to r1's ID value, nor could change it.
But it won't change it across all of the different instances, OK?
So that's how the __init__ method works of Rabbit, OK?
So we have these tags that keep track of how many rabbits we've created.
We have a couple of getter-- we have some getters here to get all the parents.
So now let's add a somewhat more interesting function.
Oh, I just want to mention, when I'm getting the rid, I'm actually using this cool zfill() function here, or method, which actually pads the beginning of any number with however many zeros in order to get to that number here.
So the number 1 becomes 001 and so on.
So it ensures that I have this nice-looking ID type thing that's always three digits long.
So let's try to work with this Rabbit object.
Let's define what happens when you add two rabbits together, OK-- in this class, not in the real world.
OK.
So if I want to use the plus operator between two rabbit instances, I have to implement this __add__ method, OK?
So all I'm doing here is I'm returning a new Rabbit object, OK?
Whoops, sorry about that.
And let's recall the __init__ method of the rabbit, OK?
So when I'm returning a new Rabbit object, I'm returning a new Rabbit object that's going to have an age of 0.
Self-- so the Rabbit object I'm calling this method on is going to be the parent of the new rabbit.
And other is going to be the other parent of the new rabbit, OK?
So if we look at the code, and I run it, this part here, I'm creating three rabbits, r1, r2, and r3.
Notice this class variable is working as expected, because the IDs of each of my rabbits increments as I create more rabbits.
So we have 001, 002, 003.
If I print r1, and r2, and r3-- that was these three lines over here-- the parents of r1 and r2 are None, because that's just the default-- yes, the default arguments for creating a rabbit.
To add two rabbits together, I use the plus operator between two Rabbit objects.
And on the right here, I'm testing rabbit addition.
And I can print out the IDs of all my rabbits.
And notice that, when I've created this new rabbit, r4, the ID of it still kept incrementing.
So now, the ID of the fourth rabbit is 004.
And then, when I get r4's parents, they are as we want them to be, so r1 and r2.
The other thing I want to do is to compare two rabbits.
So if I want to compare two rabbits, I want to make sure that their parents are the same.
So I can compare the first parent of the first rabbit with the first parent of the second rabbit and the second parent of the first rabbit to the second parent of second rabbit or getting the combinations of those two.
So that's what these two Booleans are doing.
So these are going to tell me-- these are going to be Boolean values, either True or False.
And I'm going to return either they have the same parents of that type or the same parents criss-crossed, OK?
So here, notice that I'm actually comparing the IDs of the rabbits as opposed to the Rabbit objects directly, OK?
So if, instead of comparing the IDs in here, I was comparing the parents themselves, directly, what would end up happening is this function, this method, eq(), would get called over and over again.
Because here, we have parents that are rabbits.
And at some point, the parents of the very, very first rabbits ever created by this program are None.
And so when I try to call-- when I try to call the parent one of None, that's going to give me an error, OK, something like an attribute error where None doesn't have this parent attribute, OK?
So that's why I'm comparing IDs here, OK?
And the code in the lecture here shows you some tests about whether rabbits have the same parents.
And I've created new rabbits here, r3 and r4, the addition of those two.
And r5 and r6 are going to have the same parents down here-- True-- but r4 and r6 don't, OK?
So just to wrap it up, object-oriented programming is the idea of creating your own collections of data where you can organize the information in a very consistent manner.
So every single type of object that you create of this particular type that you create-- sorry, every object instance of a particular type is going to have the exact same data attributes and the exact same methods, OK?
So this really comes back to the idea of decomposition and abstraction in programming.
All right, thanks, everyone.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we have a class Car here, object as usual.
The init method here takes in self as usual, a w, and a d. So Car gets initialized with two parameters.
And we assign the wheels data attribute to the first one and the doors data attribute to the second one.
And then we're going to also assign the color data attribute to be the empty string.
So I'm giving you four choices here.
And the question says, which of the above is a getter method for the number of wheels?
So getter method is something that's going to get a data attribute.
A method is really just a function.
So of course, we're going to have def.
get_wheels is a good name for it.
Since it's a method for this class, we have to have self in the parameters.
So we know it's going to be between C and D. And so then what are we going to return?
The first one's going to return wheels, which, in this particular case, is just going to be a variable that we haven't defined, but it's a variable.
A getter returns a data attribute of a particular instance.
So we actually have to say self dot if we want to return a data attribute of an instance, as opposed to just a regular variable.
So the correct answer is D. Great job.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I have here-- first part is just the top part, so I have some blank line and then I'm defining a method name speak.
So write a line to replace blank that creates a class for a Dog that inherits from Animal.
So the first thing I notice is that I need to write a definition for a class whose name is Dog.
So my choices are these.
So I can tell it's either the first or the third one.
And then I want to inherit from Animal, so I don't want to inherit from object but I want to inherit from Animal, right?
So that's perfect.
Next question says, with this definition of Dog, you run a program with those three lines.
What happens?
So this is my definition and I have these three lines.
Let's go down here.
Nice.
OK.
So with these three lines, this first line says, d is equal to Dog 7.
So I'm creating a Dog object with age 7.
Is this line going to throw an error?
Well, this line actually looks for an init method.
This particular class definition doesn't have a init method, but hey, I'm inheriting from someone.
I'm inheriting from Animal.
Does Animal have an init method?
And it does, as we saw in the slides.
OK, so we're creating a new Dog with age 7 and name none.
So that line doesn't throw an error.
The next line sets the name to Ruffles.
And again, I don't have a set name method inside this particular class, but does my Animal class-- my parent class-- have a set name method?
And yes it does-- right here.
So I call that one.
So that line does not throw an error.
And the third line says d.speak, which is going to cause Python to look in this current class definition.
And it says, hey, I have a method here named speak, so I'm going to use this one.
And it's going to print ruff ruff, because it's a method, so you can just-- it's like a function.
You can just print things.
You don't need to return anything.
So, perfect, most of you are getting that right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MITOpenCourseWare@ocw.mit.edu
OK, folks.
Welcome back.
Hope you had a nice long weekend with no classes.
You got caught up on all those problem sets that have been sneaking up on you.
You enjoyed watching the Patriots and Tom Brady come back.
Oh, sorry, I'm showing my local bias.
Before we talk about today's topic, I want to take a second to set the stage.
And I want you to stop and think about what you've seen so far in this course.
We're coming up on the end of the first section of the course.
And you've already seen a lot.
You've certainly learned about fundamentals of computation.
You've seen different kinds of data structures, both mutable and immutable, so tuples and lists, dictionaries, different ways of pulling things together.
You've seen a range of algorithms from simple linear code to loops, fors and whiles.
You've seen iterative algorithms.
You've seen recursive algorithms.
You've seen classes of algorithms.
Divide and conquer.
Greedy algorithms.
Bisection search.
A range of things.
And then most recently, you start pulling things together with classes-- a way to group together data that belongs together along with methods or procedures that are designed to manipulate that data.
So you've had actually a fairly good coverage already of a lot of the fundamentals of computation.
And you're starting to get geared up to be able to tackle a pretty interesting range of problems.
Today and Monday, we're going to take a little bit of a different look at computation.
Because now that you've got the tools to start building up your own personal armamentarium of tools, we'd like to ask a couple of important questions.
The primary one of which is how efficient are my algorithms?
And by efficiency, we'll see it refers both to space and time, but primarily to time.
And we'd like to know both how fast are my algorithms going to run and how could I reason about past performance.
And that's what we're going to do with today's topics.
We're going to talk about orders of growth.
We'll define what that means in a few minutes.
We're going to talk about what's called the big O notation.
And we're going to begin to explore different classes of algorithms.
Before we do that though, let's talk about why.
And I want to suggest to you there are two reasons this is important to be considering.
First question is how could we reason about an algorithm something you write in order to predict how much time is it going to need to solve a problem of a particular size?
I might be testing my code on small scale examples.
And I want to know if I'd run it on a really big one, how long is it going to take?
Can I predict that?
Can I make guesses about how much time I'm going to need to solve this problem?
Especially if it's in a real world circumstance where time is going to be crucial.
Equally important is going the other direction.
We want you to begin to reason about the algorithms you write to be able to say how do certain choices in a design of an algorithm influence how much time it's going to take.
If I choose to do this recursively, is that going to be different than iteratively?
If I choose to do this with a particular kind of structure in my algorithm, what does that say about the amount of time I'm going to need?
And you're going to see there's a nice association between classes of algorithms and the interior structure of them.
And in particular, we want to ask some fundamental questions.
Are there fundamental limits to how much time it's going to take to solve a particular problem, no matter what kind of algorithm I design around this?
And we'll see that there are some nice challenges about that.
So that's what we're going to do over the next two days.
Before we do though, let's maybe ask the obvious question-- why should we care?
Could be on a quiz, might matter to you.
Better choice is because it actually makes a difference.
And I say that because it may not be as obvious to you as it was in an earlier generation.
So people with my gray hair or what's left of my gray hair like to tell stories.
I'll make it short.
But I started programming 41 years ago-- no, sorry, 45 years ago-- on punch cards.
You don't know what those are unless you've been to a museum on a machine that filled a half a room and that took about five minutes to execute what you can do in a fraction of a second on your phone.
Right.
This is to tell you're living in a great time, not independent of what's going to happen on November 8th.
All right.
We'll stay away from those topics as well, won't we?
My point is yeah, I tell old stories.
I'm an old guy.
But you might argue look, computers are getting so much faster.
Does it really matter?
And I want to say to you-- maybe it's obvious to you-- yes, absolutely it does.
Because in conjunction with us getting faster computers, we're increasing the sizes of the problems.
The data sets we want to analyze are getting massive.
And I'll give you an example.
I just pulled this off of Google, of course.
In 2014-- I don't have more recent numbers-- Google served-- I think I have that number right-- 30 trillion pages on the web.
It's either 30 trillionaire or 30 quadrillion.
I can't count that many zeros there.
It covered 100 million gigabytes of data.
And I suggest to you if you want to find a piece of information on the web, can you write a simple little search algorithm that's going to sequentially go through all the pages and find anything in any reasonable amount of time?
Probably not.
Right?
It's just growing way too fast.
This, by the way, is of course, why Google makes a lot of money off of their map reduced algorithm for searching the web, written by the way, or co-written by an MIT grad and the parent of a current MIT student.
So there's a nice hook in there, not that Google pays MIT royalties for that wonderful thing, by the way.
All right.
Bad jokes aside, searching Google-- ton of time.
Searching a genomics data set-- ton of time.
The data sets are growing so fast.
You're working for the US government.
You want to track terrorists using image surveillance from around the world, growing incredibly rapidly.
Pick a problem.
The data sets grow so quickly that even if the computers speed up, you still need to think about how to come up with efficient ways to solve those problems.
So I want to suggest to you while sometimes simple solutions are great, they are the easy ones to rate-- too write.
Sorry.
At times, you need to be more sophisticated.
Therefore, we want to reason about how do we measure efficiency and how do we relate algorithm design choices to the cost that's going to be associated with it?
OK.
Even when we do that, we've got a choice to make.
Because we could talk about both efficiency in terms of time or in terms of space, meaning how much storage do I have inside the computer?
And the reason that's relevant is there's actually in many cases a trade-off between those two.
And you've actually seen an example, which you may or may not remember.
You may recall when we introduced dictionaries, I showed you a variation where you could compute Fibonacci using the dictionary to keep track of intermediate values.
And we'll see in next week that it actually tremendously reduces the time complexity.
That's called a trade-off, in the sense that sometimes I can pre-compute portions of the answer, store them away, so that when I've tried to a bigger version of the answer I can just look up those portions.
So there's going to be a trade-off here.
We're going to focus, for purposes of this lecture and the next one, on time efficiency.
How much time is it going to take our algorithms to solve a problem?
OK. What are the challenges in doing that before we look at the actual tools?
And in fact, this is going to lead into the tools.
The first one is even if I've decided on an algorithm, there are lots of ways to implement that.
A while loop and a for loop might have slightly different behavior.
I could choose to do it with temporary variables or using direct substitution.
There's lots of little choices.
So an algorithm could be implemented many different ways.
How do I measure the actual efficiency of the algorithm?
Second one is I might, for a given problem, have different choices of algorithm.
A recursive solution versus an iterative one.
Using divide and conquer versus straightforward search.
We're going to see some examples of that.
So I've got to somehow separate those pieces out.
And in particular, I'd like to separate out the choice of implementation from the choice of algorithm.
I want to measure how hard is the algorithm, not can I come up with a slightly more efficient way of coming up with an implementation.
So here are three ways I might do it.
And we're going to look at each one of them very briefly.
The obvious one is we could be scientists-- time it.
Write the code, run a bunch of test case, run a timer, use that to try and come up with a way of estimating efficiency.
We'll see some challenges with that.
Slightly more abstractly, we could count operations.
We could say here are the set of fundamental operations-- mathematical operations, comparisons, setting values, retrieving values.
And simply say how many of those operations do I use in my algorithm as a function of the size of the input?
And that could be used to give us a sense of efficiency.
We're going to see both of those are flawed somewhat more in the first case than the second one.
And so we're going to abstract that second one to a more abstract notion of something we call an order of growth.
And I'll come back to that in a couple of minutes.
This is the one we're going to focus on.
It's one that computer scientists use.
It leads to what we call complexity classes.
So order of growth or big O notation is a way of abstractly describing the behavior of an algorithm, and especially the equivalences of different algorithms.
But let's look at those.
Timing.
Python provides a timer for you.
You could import the time module.
And then you can call, as you can see right down here.
I might have defined a really simple little function-- convert Celsius to Fahrenheit.
And in particular, I could invoke the clock method from the time module.
And what that does is it gives me a number as the number of some fractions of a second currently there.
Having done that I could call the function.
And then I could call the clock again, and take the difference to tell me how much time it took to execute this.
It's going to be a tiny amount of time.
And then I could certainly print out some statistics.
I could do that over a large number of runs-- different sizes of the input-- and come up with a sense of how much time does it take.
Here's the problem with that.
Not a bad idea.
But again, my goal is to evaluate algorithms.
Do different algorithms have different amounts of time associated with them?
The good news is is that if I measure running time, it will certainly vary as the algorithm changes.
Just what I want to measure.
Sorry.
But one of the problems is that it will also vary as a function of the implementation.
Right?
If I use a loop that's got a couple of more steps inside of it in one algorithm than another, it's going to change the time.
And I don't really care about that difference.
So I'm confounding or conflating implementation influence on time with algorithm influence on time.
Not so good.
Worse, timing will depend on the computer.
My Mac here is pretty old.
Well, at least for computer versions.
It's about five years old.
I'm sure some of you have much more recent Macs or other kinds of machines.
Your speeds may be different from mine.
That's not going to help me in trying to measure this.
And even if I could measure it on small sized problems, it doesn't necessarily predict what happens when I go to a really large sized problems, because of issues like the time it takes to get things out of memory and bring them back in to the computer.
So what it says is that timing does vary based on what I'd like to measure, but it varies on a lot of other factors.
And it's really not all that valuable.
OK. Got rid of the first one.
Let's abstract that.
By abstract, I'm going to make the following assumption.
I'm going to identify a set of primitive operations.
Kind of get to say what they are, but the obvious one is to say what does the machine do for me automatically.
That would be things like arithmetic or mathematical operations, multiplication, division, subtraction, comparisons, something equal to another thing, something greater than, something less than, assignments, set a name to a value, and retrieval from memory.
I'm going to assume that all of these operations take about the same amount of time inside my machine.
Nice thing here is then it doesn't matter which machine I'm using.
I'm measuring how long does the algorithm take by counting how many operations of this type are done inside of the algorithm.
And I'm going to use that count to come up with a number of operations executed as a function of the size of the input.
And if I'm lucky, that'll give me a sense of what's the efficiency of the algorithm.
So this one's pretty boring.
It's got three steps.
Right?
A multiplication, a division, and an addition-- four, if you count the return.
But if I had a little thing here that added up the integers from 0 up to x, I've got a little loop inside here.
And I could count operations.
So in this case, it's just, as I said, three operations.
Here, I've got one operation.
I'm doing an assignment.
And then inside here, in essence, there's one operation to set i to a value from that iterator.
Initially, it's going to be 0.
And then it's going to be 1.
And you get the idea.
And here, that's actually two operations.
It's nice Python shorthand.
But what is that operation?
It says take the value of total and the value of i, add them together-- it's one operation-- and then set that value, or rather, set the name total to that new value.
So a second operation.
So you can see in here, I've got three operations.
And what else do I have?
Well, I'm going to go through this loop x times.
Right?
I do it for i equals 0.
And therefore, i equal 1, and so on.
So I'm going to run through that loop x times.
And if I put that together, I get a nice little expression.
1 plus 3x.
Actually, I probably cheated here.
I shouldn't say cheated.
I probably should have counted the return as one more operation, so that would be 1 plus 3x plus 1, or 3x plus 2 operations.
Why should you care?
It's a little closer to what I'd like.
Because now I've got an expression that tells me something about how much time is this going to take as I change the size of the problem.
If x is equal to 10, it's going to take me 32 operations.
If x is equal to 100, 302 operations.
If x is equal to 1,000, 3,002 operations.
And if I wanted the actual time, I'd just multiply that by whatever that constant amount of time is for each operation.
I've got a good estimate of that.
Sounds pretty good.
Not quite what we want, but it's close.
So if I was counting operations, what could I say about it?
First of all, it certainly depends on the algorithm.
That's great.
Number of operations is going to directly relate to the algorithm I'm trying to measure, which is what I'm after.
Unfortunately, it still depends a little bit on the implementation.
Let me show you what I mean by that by backing up for a second.
Suppose I were to change this for loop to a while loop.
I'll set i equal to 0 outside of the loop.
And then while i is less than x plus 1, I'll do the things inside of that.
That would actually add one more operation inside the loop, because I both have to set the value of i and I have to test the value of i, as well as doing the other operations down here.
And so rather than getting 3x plus 1, I would get 4x plus 1.
Eh.
As the government says, what's the difference between three and for when you're talking about really big numbers?
Problem is in terms of counting, it does depend.
And I want to get rid of that in a second, so it still depends a little bit on the implementation.
I remind you, I wanted to measure impact of the algorithm.
But the other good news is the count is independent of which computer I run on.
As long as all my computers come with the same set of basic operations, I don't care what the time of my computer is versus yours to do those operations on measuring how much time it would take.
And I should say, by the way, one of the reasons I want to do it is last to know is it going to take 37.42 femtoseconds or not, but rather to say if this algorithm has a particular behavior, if I double the size of the input, does that double the amount of time I need?
Does that quadruple the amount of time I need?
Does it increase it by a factor of 10?
And here, what matters isn't the speed of the computer.
It's the number of operations.
The last one I'm not going to really worry about.
But we'd have to really think about what are the operations we want to count.
I made an assumption that the amount of time it takes to retrieve something from memory is the same as the amount of time it takes to do a numerical computation.
That may not be accurate.
But this one could probably be dealt with by just agreeing on what are the common operations and then doing the measurement.
So this is closer.
Excuse me.
And certainly, that count varies for different inputs.
And we can use it to come up with a relationship between the inputs and the count.
And for the most part, it reflects the algorithm, not the implementation.
But it's still got that last piece left there, so I need to get rid of the last piece.
So what can we say here?
Timing and counting do evaluate or reflect implementations?
I don't want that.
Timing also evaluates the machines.
What I want to do is just evaluate the algorithm.
And especially, I want to understand how does it scale?
I'm going to say what I said a few minutes ago again.
If I were to take an algorithm, and I say I know what its complexity is, my question is if I double the size of the input, what does that say to the speed?
Because that's going to tell me something about the algorithm.
I want to say what happens when I scale it?
And in particular, I want to relate that to the size of the input.
So here's what we're going to do.
We're going to introduce orders of growth.
It's a wonderful tool in computer science.
And what we're going to focus on is that idea of counting operations.
But we're not going to worry about small variations, whether it's three or four steps inside of the loop.
We're going to show that that doesn't matter.
And if you think about my statement of does it double in terms of size or speed or not-- or I'm sorry-- time or not, whether it goes from three to six or four to eight, it's still a doubling.
So I don't care about those pieces inside.
I'm going to focus on what happens when the size of the problem gets arbitrarily large.
I don't care about counting things from 0 up to x when x is 10 or 20.
What happens when it's a million?
100 million?
What's the asymptotic behavior of this?
And I want to relate that time needed against the size of the input, so I can make that comparison I suggested.
OK.
So to do that, we've got to do a couple of things.
We have to decide what are we going to measure?
And then we have to think about how do we count without worrying about implementation details.
So we're going to express efficiency in terms of size of input.
And usually, this is going to be obvious.
If I've got a procedure that takes one argument that's an integer, the size of the integer is the thing I'm going to measure things in.
If I double the size of that integer, what happens to the computation?
If I'm computing something over a list, typically the length of the list is going to be the thing I'm going to use to characterize the size of the problem.
If I've got-- and we'll see this in a second-- a function that takes more than one argument, I get to decide what's the parameter I want to use.
If I'm searching to see is this element in that list, typically, I'm going to worry about what's the size of the list, not what's the size of the element.
But we have to specify what is that we're measuring.
And we're going to see examples of that in just a second.
OK.
So now, we start thinking about that sounds great.
Certainly fun computing something numeric.
Sum of integers from 0 up to x.
It's kind of obvious x is the size of my problem.
How many steps does it take?
I can count that.
But in some cases, the amount of time the code takes is going to depend on the input.
So let's take this little piece of code here.
And I do hope by now, even though we flash up code, you're already beginning to recognize what does it do.
Not the least of which, by the clever name that we chose.
But this is obviously just a little function.
It runs through a loop-- sorry, a for loop that takes i for each element in a list L. And it's checking to see is i equal to the element I've provided.
And when it is, I'm going to return true.
If I get all the way through the loop and I didn't find it, I'm going to return false.
It's just saying is e in my input list L?
How many steps is this going to take?
Well, we can certainly count the number of steps in the loop.
Right?
We've got a set i.
We've got to compare i and potentially we've got to return.
So there's at most three steps inside the loop.
But depends on how lucky I'm feeling.
Right?
If e happens to be the first element in the list-- it goes through the loop once-- I'm done.
Great.
I'm not always that lucky.
If e is not in the list, then it will go through this entire loop until it gets all the way through the elements of L before saying false.
So this-- sort of a best case scenario.
This is the worst case scenario.
Again, if I'm assigned and say well, let's run some trials.
Let's do a bunch of examples and see how many steps does it go through.
And that would be the average case.
On average, I'm likely to look at half the elements in the list before I find it.
Right?
If I'm lucky, it's early on.
If I'm not so lucky, it's later on.
Which one do I use?
Well, we're going to focus on this one.
Because that gives you an upper bound on the amount of time it's going to take.
What happens in the worst case scenario?
We will find at times it's valuable to look at the average case to give us a rough sense of what's going to happen on average.
But usually, when we talk about complexity, we're going to focus on the worst case behavior.
So to say it in a little bit different way, let's go back to my example.
Suppose you gave it a list L of some length.
Length of L, you can call that len if you like.
Then my best case would be the minimum running type.
And in this case, it will be for the first element in the list.
And notice in that case, the number of steps I take would be independent of the length of L. That's great.
It doesn't matter how long the list is.
If I'm always going to find the first element, I'm done.
The average case would be the average over the number of steps I take, depending on the length of the list.
It's going to grow linearly with the length of the list.
It's a good practical measure.
But the one I want to focus on will be the worst case.
And here, the amount of time as we're going to see in a couple of slides, is linear in the size of the problem.
Meaning if I double the length of the list in the worst case, it's going to take me twice as much time to find that it's not there.
If I increase the length in the list by a factor of 10, in the worst case, it's going to take me 10 times as much time as it did in the earlier case to find out that the problem's not there.
And that linear relationship is what I want to capture.
So I'm going to focus on that.
What's the worst case behavior?
And we're about ready to start talking about orders of growth, but here then is what orders of growth are going to provide for me.
I want to evaluate efficiency, particularly when the input is very large.
What happens when I really scale this up?
I want to express the growth of the program's runtime as that input grows.
Not the exact runtime, but that notion of if I doubled it, how much longer does it take?
What's the relationship between increasing the size of the input and the increase in the amount of time it takes to solve it?
We're going to put an upper bound on that growth.
And if you haven't seen this in math, it basically says I want to come up with a description that is at least as big as-- sorry-- as big as or bigger than the actual amount of time it's going to take.
And I'm going to not worry about being precise.
We're going to talk about the order of rather than exact.
I don't need to know to the femtosecond how long this is going to take, or to exactly one operation how long this is going to take.
But I want to say things like this is going to grow linearly.
I double the size of the input, it doubles the amount of time.
Or this is going to grow quadratically.
I double the size of the input, it's going to take four times as much time to solve it.
Or if I'm really lucky, this is going to have constant growth.
No matter how I change the input, it's not going to take any more time.
To do that, we're going to look at the largest factors in the runtime.
Which piece of the program takes the most time?
And so in orders of growth, we are going to look for as tight as possible an upper bound on the growth as a function of the size of the input in the worst case.
Nice long definition.
Almost ready to look at some examples.
So here's the notation we're going to use.
It's called Big O notation.
I have to admit-- and John's not here today to remind me the history-- I think it comes because we used Omicron-- God knows why.
Sounds like something from Futurama.
But we used Omicron as our symbol to define this.
I'm having such good luck with bad jokes today.
You're not even wincing when I throw those things out.
But that's OK.
It's called Big O notation.
We're going to use it.
We're going to describe the rules of it.
Is this the tradition of it?
It describes the worst case, because it's often the bottleneck we're after.
And as we said, it's going to express the growth of the program relative to the input size.
OK. Let's see how we go from counting operations to getting to orders of growth.
Then we're going to define some examples of ordered growth.
And we're going to start looking at algorithms.
Here's a piece of code you've seen before.
Again, hopefully, you recognize or can see fairly quickly what it's doing.
Computing factorials the iterative way.
Basically, remember n factorial is n times n minus 1 times n minus 2 all the way down to 1.
Hopefully, assuming n is a non-negative integer.
Here, we're going to set up an internal variable called answer.
And then we're just going to run over a loop.
As long as n is bigger than 1, we're going to multiply answer by n, store it back into answer, decrease n by 1.
We'll keep doing that until we get out of the loop.
And we're going to return answer.
We'll start by counting steps.
And that's, by the way, just to remind you that in fact, there are two steps here.
So what do I have?
I've got one step up there.
Set answer to one.
I'm setting up n-- sorry, I'm not setting up n. I'm going to test n. And then I'm going to do two steps here, because I got a multiply answer by n and then set it to answer.
And now similarly, we've got two steps there because I'm subtracting 1 from n and then setting it to n. So I've got 2 plus 4 plus the test, which is 5.
I've got 1 outside here.
I got 1 outside there.
And I'm going to go through this loop n times.
So I would suggest that if I count the number of steps, it's 1 plus 5n plus 1.
Sort of what we did before.
5n plus 2 is the total number of steps that I use here.
But now, I'm interested in what's the worst case behavior?
Well, in this case, it is the worst case behavior because it doesn't have decisions anywhere in here.
But I just want to know what's the asymptotic complexity?
And I'm going to say-- oh, sorry-- that is to say I could do this different ways.
I could have done this with two steps like that.
That would have made it not just 1 plus 5n plus 1.
It would have made it 1 plus 6n plus 1 because I've got an extra step.
I put that up because I want to remind you I don't care about implementation differences.
And so I want to know what captures both of those behaviors.
And in Big O notation, I say that's order n. Grows linearly.
So I'm going to keep doing this to you until you really do wince at me.
If I were to double the size of n, whether I use this version or that version, the amount of time the number of steps is basically going to double.
Now you say, wait a minute.
5n plus 2-- if n is 10 that's 52.
And if n is 20, that's 102.
That's not quite doubling it.
And you're right.
But remember, we really care about this in the asymptotic case.
When n gets really big, those extra little pieces don't matter.
And so what we're going to do is we're going to ignore the additive constants and we're going to ignore the multiplicative constants when we talk about orders of growth.
So what does o of n measure?
Well, we're just summarizing here.
We want to describe how much time is needed to compute or how does the amount of time, rather, needed to computer problem growth as the size of the problem itself grows.
So we want an expression that counts that asymptotic behavior.
And we're going to focus as a consequence on the term that grows most rapidly.
So here are some examples.
And I know if you're following along, you can already see the answers here.
But we're going to do this to simply give you a sense of that.
If I'm counting operations and I come up with an expression that has n squared plus 2n plus 2 operations, that expression I say is order n squared.
The 2 and the 2n don't matter.
And think about what happens if you make n really big.
n squared is much more dominant than the other terms.
We say that's order n squared.
Even this expression we say is order n squared.
So in this case, for lower values of n, this term is going to be the big one in terms of number of steps.
I have no idea how I wrote such an inefficient algorithm that it took 100,000 steps to do something.
But if I had that expression for smaller values of n, this matters a lot.
This is a really big number.
But when I'm interested in the growth, then that's the term that dominates.
And you see the idea or begin to see the idea here that when I have-- sorry, let me go back there-- when I have expressions, if it's a polynomial expression, it's the highest order term.
It's the term that captures the complexity.
Both of these are quadratic.
This term is order n, because n grows faster than log of n. This funky looking term, even though that looks like the big number there and it is a big number, that expression we see is order n log n. Because again, if I plot out as how this changes as I make n really large, this term eventually takes over as the dominant term.
What about that one?
What's the big term there?
How many people think it's n to the 30th?
Show of hands.
How many people think it's 3 to the n?
Show of hands.
Thank you.
You're following along.
You're also paying attention.
How many people think I should stop asking questions?
No show of hands.
All right.
But you're right.
Exponentials are much worse than powers.
Even something like this-- again, it's going to take a big value of n before it gets there, but it does get there.
And that, by the way, is important, because we're going to see later on in the term that there are some problems where it's believed that all of the solutions are exponential.
And that's a pain, because it says it's always going to be expensive to compute.
So that's how we're going to reason about these things.
And to see it visually, here are the differences between those different classes.
Something that's constant-- the amount of time doesn't change as I change the size of the input.
Something that linear grows as a straight line, as you would expect.
Nice behavior.
Quadratic starts to grow more quickly.
The log is always better than linear because it slows down as we increase the size.
n log n or log linear is a funky term, but we're going to see it's a very common complexity for really valuable algorithms in computer science.
And it has a nice behavior, sort of between the linear and the quadratic.
And exponential blows up.
Just to remind you of that-- well, sorry-- let me show you how we're going to do the reasoning about this.
So here's how we're going to reason about it.
We've already seen some code where I started working through this process of counting operations.
Here are the tools I want you to use.
Given a piece of code, you're going to reason about each chunk of code separately.
If you've got sequential pieces of code, then the rules are called the law of addition for order of growth is that the order of growth of the combination is the combination of the order of the growth.
Say that quickly 10 times.
But let's look at an example of that.
Here are two loops.
You've already seen examples of how to reason about those loops.
For this one, it's linear in the size of n. I'm going to go through the loop n times doing a constant amount of things each time around.
But what I just showed, that's order n. This one-- again, I'm doing just a constant number of things inside the loop-- but notice, that it's n squared.
So that's order n squared.
n times n. The combination is I have to do this work and then that work.
So I write that as saying that is order of n plus order of n squared.
But by this up here, that is the same as saying what's the order of growth of n plus n squared.
Oh yeah.
We just saw that.
Says it's n squared.
So addition or the law of addition let's be reasonable about the fact that this will be an order n squared algorithm.
Second one I'm going to use is called the law of multiplication.
And this says when I have nested statements or nested loops, I need to reason about those.
And in that case, what I want to argue-- or not argue-- state is that the order of growth here is a multiplication.
That is, when I have nested things, I figure out what's the order of growth of the inner part, what's the order growth of the outer part, and I'm going to multiply-- bleh, try again-- I'm going to multiply together those orders of growth, get the overall order of growth.
If you think about it, it makes sense.
Look at my little example here.
It's a trivial little example.
But I'm looping for i from 0 up to n. For every value of i, I'm looping for j from 0 up to n. And then I'm printing out A. I'm the Fonz.
I'm saying heyyy a lot.
Oh, come on.
At least throw something, I mean, when it's that bad.
Right?
Want to make sure you're still awake.
OK. You get the idea.
But what I want to show you here is notice the order of growth.
That's order n. Right?
I'm doing that n times.
But I'm doing that for each value of i.
The outer piece here loops also n times.
For each value of i, I'm doing order n things.
So I'm doing order of n times order of n steps.
And by that law, that is the same order of n times n or n squared.
So this is a quadratic expression.
You're going to see that a lot.
Nested loops typically have that kind of behavior.
Not always, but typically have that kind of behavior.
So what you're going to see is there's a set of complexity classes.
And we're about to start filling these in.
Order one is constant.
Says amount of time it takes doesn't depend on the size of the problem.
These are really rare that you get.
They tend to be trivial pieces of code, but they're valuable.
Log n reflects logarithmic runtime.
You can sort of read the rest of them.
These are the kinds of things that we're going to deal with.
We are going to see examples here, here, and here.
And later on, we're going to come back and see these, which are really nice examples to have.
Just to remind you why these orders of growth matter-- sorry, that's just reminding you what they look like.
We've already done that.
Here is the difference between constant log, linear, log linear squared, and exponential.
When n is equal to 10, 100, 1,000 or a million.
I know you know this, but I want to drive home the difference.
Something that's constant is wonderful, no matter how big the problem is.
Takes the same amount of time.
Something that is log is pretty nice.
Increase the size of the problem by 10, it increases by a factor of 2.
From another 10, it only increases by a factor of another 50%.
It only increases a little bit.
That's a gorgeous kind of problem to have.
Linear-- not so bad.
I go from 10 to 100 to 1,000 to a million.
You can see log linear is not bad either.
Right?
A factor of 10 increase here is only a factor of 20 increase there.
A factor of 10 increase there is only a factor of 30 increase there.
So log linear doesn't grow that badly.
But look at the difference between n squared and 2 to the n. I actually did think of printing this out.
By the way, Python will compute this.
But it was taken pages and pages and pages.
I didn't want to do it.
You get the point.
Exponential-- always much worse.
Always much worse than a quadratic or a power expression.
And you really see the difference here.
All right.
The reason I put this up is as you design algorithms, your goal is to be as high up in this listing as you can.
The closer you are to the top of this list, the better off you are.
If you have a solution that's down here, bring a sleeping bag and some coffee.
You're going to be there for a while.
Right?
You really want to try and avoid that if you can.
So now what we want to do, both for the rest of today in the last 15 minutes and then next week, is start identifying common algorithms and what is their complexity.
As I said to you way back at the beginning of this lecture, which I'm sure you remember, it's not just to be able to identify the complexity.
I want you to see how choices algorithm design are going to lead to particular kinds of consequences in terms of what this is going to cost you.
That's your goal here.
All right.
We've already seen some examples.
I'm going to do one more here.
But simple iterative loop algorithms are typically linear.
Here's another version of searching.
Imagine I'll have an unsorted list.
Arbitrary order.
Here's another way of doing the linear search.
Looks a little bit faster.
I'm going to set a flag initially to false.
And then I'm going to loop for i from 0 up to the length of L. I'm going to use that to index into the list, pull out each element of the list in turn, and check to see is it the thing I'm looking for.
As soon as I find it, I'm going to send-- sorry-- set the flag to true.
OK?
So that when I return out of the loop, I can just return found.
And if I found it to be true, if I never found it, found will still be false and I'll return it.
We could count the operations here, but you've already seen examples of doing that.
This is linear, because I'm looping n times if n is the length of the list over there.
And the number of things I do inside the loop is constant.
Now, you might say, wait a minute.
This is really brain damaged, or if you're being more politically correct, computationally challenged.
OK?
In the sense of once I've found it, why bother looking at the rest of the list?
So in fact, I could just return true right here.
Does that change the order of growth of this algorithm?
No.
Changes the average time.
I'm going to stop faster.
But remember the order of growth captures what's the worst case behavior.
And the worst case behavior is the elements not in the list I got to look at everything.
So this will be an example of a linear algorithm.
And you can see, I'm looping length of L times over the loop inside of there.
It's taking the order one to test it.
So it's order n. And if I were to actually count it, there's the expression.
It's 1 plus 4n plus 1, which is 4n plus 2, which by my rule says I don't care about the additive constant.
I only care about the dominant term.
And I don't care about that multiplicative constant.
It's order n. An example of a template you're going to see a lot.
Now, order n where n is the length of the list and I need to specify that.
That's the thing I'm after.
If you think about it, I cheated.
Sorry-- I never cheat.
I'm tenure.
I never cheat.
I just mislead you badly.
Not a chance.
How do I know that accessing an element of the list takes constant time?
I made an assumption about that.
And this is a reasonable thing to ask about-- both what am I assuming about the constant operations and how do I know that's actually true?
Well, it gives me a chance to point out something that Python does very effectively.
Not all languages do.
But think about a list.
Suppose I've got a list that's all integers.
I'm going to need some amount of memory to represent each integer.
So if a byte is 8 bits, I might reserve 4 bytes or 32 bits to cover any reasonable sized integer.
When I represent a list, I could simply have each of them in turn.
So what do I need to know?
I'm going to allocate out a particular length-- say 4 bytes, 32 bits, 32 sequential elements of memory to represent each integer.
And then I just need to know where's the first part of the list, what's the address and memory of the first part of the list.
And to get to the i-th element, I take that base plus 4 bytes times i.
And I can go straight to this point without having to walk down the list.
That's nice.
OK?
It says, in fact, I can get to any element of memory-- I'm sorry-- any element of the list in constant time.
OK. Now, what if the things I'm representing aren't integers?
They're arbitrary things and they take up a big chunk of space.
Well, if the list is heterogeneous, we use a nice technique called indirection.
And that simply says we, again, have a list.
We know the address of this point.
We know the address there for the i-th element of this list.
But inside of here, we don't store the actual value.
We store a pointer to where it is in memory.
Just what these things are indicating.
So they can be arbitrary size.
But again, I can get to any element in constant time, which is exactly what I want.
So that's great.
OK. Now, suppose I tell you that the list is sorted.
It's in increasing order.
I can be more clever about my algorithm.
Because now, as I loop through it, I can say if it's the thing I'm looking for, just return true.
If the element of the list is bigger than the thing I'm looking for, I'm done.
I don't need to look at the rest of the list, because I know it can't be there because it's ordered or sorted.
I can just return false.
If I get all the way through the loop, I can return false.
So I only have to look until I get to a point where the thing in the list is bigger than what I'm looking for.
It's the order of growth here.
Again, the average time behavior will be faster.
But the order of growth is I've got to do order of length of the list to go through the loop, order of one to do the test, and in the worst case, again, I still have to go through the entire list.
So the order of growth here is the same.
It is, again, linear in the length of the list, even though the runtime will be different depending whether it's sorted or not.
I want you to hold on to that idea, because we're going to come back to the sorted list next week to see that there actually are much more efficient ways to use the fact that a list is sorted to do the search.
But both of these versions same order growth, order n. OK.
So lurching through a list-- right, sorry-- searching through a list in sequence is linear because of that loop.
There are other things that have a similar flavor.
And I'm going to do these quickly to get to the last example.
Imagine I give you a string of characters that are all soon to be composed of decimal digits.
I just want to add them all up.
This is also linear, because there's the loop.
I'm going to loop over the characters in the string.
I'm going to cast them into integers, add them in, and return the value.
This is linear in the length of the input s. Notice the pattern.
That loop-- that in-iterative loop-- it's got that linear behavior, because inside of the loop constant number of things that I'm executing.
We already looked at fact iter.
Same idea.
There's the loop I'm going to do that n times inside the loop a constant amount of things.
So looping around it is order n. There's the actual expression.
But again, the pattern I want you to see here is that this is order n. OK. Last example for today.
I know you're all secretly looking at your watches.
Standard loops, typically linear.
What about nested loops?
What about loops that have loops inside of them?
How long do they take?
I want to show you a couple of examples.
And mostly, I want to show you how to reason about them.
Suppose I gave you two lists composed of integers, and I want to know is the first list a subset of the second list.
Codes in the handbook, by the way, if you want to go run it.
But basically, the simple idea would be I'm going to loop over every element in the first list.
And for each one of those, I want to say is it in the second list?
So I'll use the same kind of trick.
I'll set up a flag that's initially false.
And then I'm going to loop over everything in the second list.
And if that thing is equal to the thing I'm looking for, I'll set match to true and break out of the loop-- the inner loop.
If I get all the way through the second list and I haven't found the thing I'm looking for, when I break out or come out of this loop, matched in that case, will still be false and all return false.
But if up here, I found something that matched, match would be true.
I break out of it.
It's not false.
Therefore, a return true.
I want you look at the code.
You should be able to look at this and realize what it's doing.
For each element in the first list, I walk through the second list to say is that element there.
And if it is, I return true.
If that's true for all of the elements in the first list, I return true overall.
OK. Order of growth.
Outer loop-- this loop I'm going to execute the length of L1 times.
Right?
I've got to walk down that first list.
If I call that n, it's going to take that n times over the outer loop.
But what about n here?
All of the earlier examples, we had a constant number of operations inside of the loop.
Here, we don't.
We've got another loop that's looping over in principle all the elements of the second list.
So in each iteration is going to execute the inner loop up to length of L2 times, where inside of this inner loop there is a constant number of operations.
Ah, nice.
That's the multiplicative law of orders of growth.
It says if this is order length L1.
And we're going to do that then order length of L2 times, the order of growth is a product.
And the most common or the worst case behavior is going to be when the lists are of the same length and none of the elements of L1 are in L2.
And in that case, we're going to get something that's order n squared quadratic, where n is the length of the list in terms of number of operations.
I don't really care about subsets.
I've got one more example.
We could similarly do intersection.
If I wanted to say what is the intersection of two lists?
What elements are on both list 1 and list 2?
Same basic idea.
Here, I've got a pair of nested loops.
I'm looping over everything in L1.
For that, I'm looping over everything in L2.
And if they are the same, I'm going to put that into a temporary variable.
Once I've done that, I need to clean things up.
So I'm going to write another loop that sets up an internal variable and then runs through everything in the list I accumulated, making sure that it's not already there.
And as long as it isn't, I'm going to put it in the result and return it.
I did it quickly.
You can look through it.
You'll see it does the right thing.
What I want it to see is what's the order of growth.
I need to look at this piece.
Then I need to look at that piece.
This piece-- well, it's order length L1 to do the outer loop.
For each version of e1, I've got to do order of length L2 things inside to accumulate them.
So that's quadratic.
What about the second loop?
Well, this one is a little more subtle.
I'm only looping over temp, which is at most going to be length L1 long.
But I'm checking to see is that element in a list?
And it depends on the implementation.
But typically, that's going to take up to the length of the list to do it.
I got to look to see is it there or not.
And so that inner loop if we assume the lists are the same size is also going to take potentially up to length L1 steps.
And so this is, again, quadratic.
It's actually two quadratics-- one for the first nested loop, one for the second one, because there's an implicit second loop right there.
But overall, it's quadratic.
So what you see in general-- this is a really dumb way to compute n squared.
When you have nested loops, typically, it's going to be quadratic behavior.
And so what we've done then is we've started to build up examples.
We've now seen simple looping mechanisms, simple iterative mechanisms, nested loops.
They tend to naturally give rise to linear and quadratic complexity.
And next time, we're going to start looking at more interesting classes.
And we'll see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time, we started talking about complexity.
And I want to quickly remind you of what we're doing, because we're going to talk some more about that today.
And when I say "complexity," it was the question of, can we estimate the amount of resources-- typically time-- that we're going to need to get an algorithm to solve a problem of a particular size?
And we talked about that both in terms of estimating, how much time is it going to take and using it to go the other direction and think about how design choices in an algorithm have implications for the cost that's going to be associated with that.
So we introduced the idea of what we call big O notation, orders of growth, a way of measuring complexity.
And we started talking about different classes of algorithms.
So today, I'm going to quickly recap those basic ideas.
And what we're going to do is we're going to see examples of standard classes of algorithms with the idea that you're going to want to begin to recognize when an algorithm is in that class.
So very quick recap-- I've already said part of this.
We want to have a mechanism, a method for being able to estimate or reason about, how much time do we think an algorithm's going to take to solve a problem of a particular size.
And especially as we increase the size of the input to the algorithm, what does that do in terms of the increase in the amount of time that we need?
Some ways, we don't care about exact versions.
In a second, we're going to see the definition of this.
But what we care about is that notion of, how does it grow as we increase the problem size.
And what we're going to focus in going, if you like, in the forward direction-- as I said, a lot of the interest is actually thinking about what you might think of as the backwards or reverse direction.
How does a choice in algorithm design impact efficiency of the algorithm?
And really, what we want you to do is to begin to recognize standard patterns, that, if you make a particular choice, this fits in a class of algorithms you've seen before.
And I know, in essence, how much time-- what the cost is going to be as I do that.
All right, so just to recap, as it says on the top, we talked about orders of growth.
And the idea is, we want to be able to evaluate a program's efficiency when the input is very big.
We talked about timing things just with a timer.
We suggested that, unfortunately, conflates the algorithm with the implementation with the particular machine.
We want to get rid of those latter two and just focus on the algorithm.
So we're going to talk about counting operations, but in a very general sense.
So we're going to express what we call the growth of the program's runtime as the input size grows very large.
And because we're only interested not in the exact number, but in, if you like, the growth of that, we're going to focus on putting an upper bound on the growth, an expression that grows at least as fast as what the cost of the algorithm is.
Now, you could just cheat and pick a really big upper bound.
That doesn't help us very much.
So in general, we're going to try and use as tight an upper bound as we can.
What's the class of algorithm it falls in?
But as we've seen before, we care about the order of growth, not being exact.
So if something grows as 2 to the n plus 5n, I don't care about the 5n.
Because when n gets big, that 2 to the n is the really big factor.
And therefore we're going to look at the largest factors when we think about that.
We've seen some examples.
We're going to do a bunch more examples today to fill those in.
One of the things that we want to now do is say, with that idea in mind, there are classes of complexity of algorithms.
So in some sense, the best ones are way up here, O of 1, order 1, constant.
It says cost doesn't change as I change the size of the input.
That's great.
It's always going to be the same cost.
Order log n says the cost grows logarithmically with the size of the input.
It's a slow growth, and I'm going to remind you of that in a second.
We saw lots of examples of linear running time-- we're going to see a few more today-- what we call log-linear, polynomial, and exponential.
And the thing I want to remind you is that, ideally-- whoops, sorry-- we'd like our algorithms to be as close to the top of this categorization as we can.
This is actually described in increasing order of complexity.
Something that takes the same amount of time no matter how big the input is, unless that amount of time is a couple of centuries, seems like a really good algorithm to have.
Something that grows linearly is not bad.
Something that grows, as we've seen down here, exponentially tends to say, this is going to be painful.
And in fact, you can see that graphically.
I'll just remind you here.
Something that's constant says, if I draw out the amount of time it takes as a function of the size of the input, it doesn't change.
Logarithmic glows-- gah, sorry-- grows very slowly.
Linear will grow, obviously, in a linear way.
And I actually misspoke last time.
I know it's rare for a professor to ever admit they misspeak, but I did.
Because I said linear is, if you double the size of the input, it's going to double the amount of time it takes.
Actually, that's an incorrect statement.
Really, what I should have said was, the increment-- if I go from, say, 10 to 100, the increase in time-- is going to be the same as the increment if I go from 100 to 1,000.
Might be more than double depending on what the constant is.
But that growth is linear.
If you want to think of it, take the derivative of time with respect to input size.
It's just going to be a constant.
It's not going to change within.
And of course, when we get down to here, things like exponential, it grows really fast.
And just as a last recap, again, I want to be towards the top of that.
There was my little chart just showing you things that grow constant, log, linear, log-linear, quadratic, and exponential.
If I go from n of 10, to 100, to 1,000, to a million, you see why I want to be at the top of that chart.
Something up here that grows logarithmically, the amount of time grows very slowly as I increase the input.
Down here, well, like it says, good luck.
It's going to grow up really quickly as I move up in that scale.
I want to be at the top of this chart if I can.
OK, with that in mind, when I'm going to do today is show you examples filling in most of this chart.
We've already seen some examples.
We've seen examples of linear.
We've seen examples of quadratic.
I'm going to just remind you of those.
What I want to do is show you how you can begin to recognize a choice of an algorithm in terms of where it lies.
So algorithms that are constant complexity, they're kind of boring.
They tend to be pretty simple.
Because this says, this code is going to run in, basically, the same amount of time independent of the size of the input.
Now, notice the bottom thing here.
It doesn't say you can't-- blah, try again.
It doesn't say you couldn't have a loop or a recursive call.
You could, it's just that that loop cannot depend on the size of the input.
So there aren't many interesting algorithms here.
We're going to see pieces of code that fit into this when we do our analysis.
But something that's constant in complexity says, independent of the size of the input.
All right, a little more interesting-- not a little, a lot more interesting-- are algorithms that are logarithmic in their complexity.
So they're going to grow with the logarithm of the size of the input.
You saw an example much earlier in the term when Ana showed you bisection search.
It was searching for a number with a particular property.
I want to show you another example, both to let you recognize the form of the algorithm, but especially, to show you how we can reason about the growth.
And that's another trick called binary search or, again, it's a version of bisection search.
Suppose I give you a list, a list of numbers, integers.
And I want to know if a particular element is in that list.
We saw, last time, you could just walk down the list, just iterate through the entire list looking to see if it's there.
In the worst case, which is what we worry about, it's going to be linear.
You're going to have to look at every element in the list till you get to the end.
So complexity was linear in that case.
And then we said, suppose we know that the list is sorted.
It's ordered from smallest to largest.
And we saw, a simple algorithm says, again, walk down the list checking to see if it's there.
But when you get to an element that's bigger than the thing you're looking at, you can just stop.
There's no reason to look at the rest of the list.
They've all got to be bigger than the thing you're searching for.
Practically, in the average case, that's going to be faster than just looking at an unsorted list.
But the complexity is still linear.
Because in the worst case, I've got to go all the way through the list before I deduce that the thing is not there.
OK, so even sequential search in an ordered list is still linear.
Can we do better?
And the answer is, sure.
So here's how we do better.
I'm going to take that list.
I'm going to assume it's sorted.
And I'm going to pick an index that divides the list in half, just pick the midpoint in the list.
And I'm going to check that value.
I'm going to ask, is the element in the list at that point the thing I'm looking for?
If it is, great, I'm done.
If I'm not that lucky, I'm then going to ask, is it larger or smaller than the thing I'm looking for?
And based on that, I'm either going to search the front half or the back half of the list.
Ooh, that's nice, OK?
Because if you think about it, in something that was just a linear algorithm, at each step, I reduced the size the problem by 1.
I went from a problem of size n to a problem of size n minus 1 to a problem of size n minus 2.
Here, I'm taking a problem of size n. I'm reducing it to n/2 in one step, because I can throw half the list away.
So this is a version of divide and conquer, things we've seen before.
I'm breaking it down into smaller versions of the problem.
So let's look at that.
And then, let's write some code.
And then, let's analyze it.
So suppose I have a list of size n, all right?
There are n elements in there.
I'm going to look at the middle one, say, is it the thing I'm looking for.
If it's not, is it bigger than or less than the thing I'm looking for?
And in this case, let's assume that, in fact, the thing I'm looking for is smaller than that element.
Great.
I'm going to throw away half the list.
Now I only have to look at the lower half of the list.
I'll do the same thing.
I'll look at the element in the middle here.
And I'll say, is it the thing I'm looking for?
If not, is it bigger than or smaller than the thing I'm looking for?
OK, and I'm down to n/2 elements.
And after I do that, I throw away half the list again.
In this case, I'm assuming that the thing I'm looking for is bigger than that middle point.
Until I find it, at each step, I'm looking at the middle element.
And I'm either throwing away the left half or the right half of that list.
So after i steps, I'm down to a list of size n over 2 to the i.
Now, what's the worst case?
The worst case is the element's not in the list.
I'm going to have to keep doing this until I get down to just a list of one element.
And at that point, if it's not the thing I'm looking for, I know I'm done, and I can stop.
Different pattern-- notice how I'm cutting down the size of the problem by 2.
So I can ask, before we look at the code, what's the complexity of this?
How many steps do I have to go through in the worst case?
And I know I'm going to be done looking at the list when n over 2 to the i is equal to 1, meaning there's only one element left that I'm still looking at.
And I can solve that.
It says I'm going to have to take, at most, i equal to log n steps, all right?
So logarithmically, I'm cutting this down.
And so the complexity of the recursion-- we haven't talked about the code yet, but in terms of the number of steps I have to do in the worst case-- is just logarithmic in the length of the list.
That's nice.
It's a lot better than looking at everything inside the list.
And in fact, you can see it, right?
I don't look at everything inside the list here.
I'm throwing half the things away at a time.
OK, so let's look at some code to do that.
Bisection search-- I'm going to give it a list of numbers.
I'm going to give it something I'm looking for.
We can walk through this code.
Hopefully it's something that you're going to be able to recognize pretty clearly.
It says if the list is empty, there's nothing there, the thing I'm looking for can't be there, I return False.
If there's exactly one element in the list, then I just check it.
If that thing's the thing I'm looking for, return True.
Otherwise, return False.
So I'm just going to return the value there.
Otherwise, find the midpoint-- notice the integer division here-- find the midpoint in that list and check it.
In particular, say, if the thing at the midpoint is bigger than the thing I'm looking for, then I'm going to return a recursive call to this function only looking at the first half of the list.
I'm just slicing into it.
Otherwise, I'll do the same thing on the second half of the list.
Nice, this is implementing exactly what I said.
We could actually try it.
I'll do that in a second if I remember.
But let's think about complexity here.
That's constant, right?
Doesn't depend on the size of the list.
That's constant, doesn't depend on the size of the list.
That's consonant.
Sounds good.
And what about that?
Well, it looks like it should be constant, right, other than the number of times I have to go through there.
Remember, I know I'm going to have order log n recursive calls.
I'm looking at what's the cost to set it up.
It looks like it should be constant.
So does that.
But I'm going to claim it's not.
Anybody see why it's not?
You can look at the slides you've already printed out.
Right there-- I'm actually copying the list, all right?
When I slice into the list like that, it makes a copy of the list.
Oh, crud.
I was about to say something different, but I won't, because this is going to cost me a little bit as I think about the work.
So let's look at that a little more carefully.
I've got order log n search calls.
We just deduced that.
I've just repeated it here, right?
On each call, I'm reducing the size of the list in half.
So it goes from n, to n/2, to n/4, to n/8, to n/16.
I'll be done, in the worst case, when I get down to having only a list of size 1.
That takes a log n steps, because n over 2 to the log n is n/n, which is 1.
But to set up the search for each cell, I've got to copy the list.
And the list starts out n long, so in principle, I've got order n work to do to set up the recursive call.
And so by the things we saw last time, I got order log n for the number of recursive calls times order n work inside of each call.
And that's order n log n. So it's not what I wanted.
Now, if you're thinking about this carefully, you'll realize, on each step, I'm not actually copying the whole list.
I'm copying half the list, and then, a quarter of the list, and then, an eighth of the list.
So if I was actually really careful-- I'm not going to do the math here-- and in fact, what we'll see-- and if you like, in your copious spare time, you can go off and work this through-- what you'll discover is that you're actually doing order n work to do the copying.
But that's still a problem, because then, I've got something that's order n plus log n. And the n is going to dominate, so this is still linear.
Sounds like I led you down a primrose path here.
Can we fix this?
Sure.
Because we could do the following.
We could say, when I want to look at that list, do I need to copy everything?
What about if, instead, I said, here's the beginning and the end of the list.
When I test the middle, I'll move one of the pointers to the middle of the list.
When I test the middle again, I'll move another pointer in.
So in other words, I can test the middle.
And based on that, I could say, I only need to search this part of the list.
Just keep track of that point and that point in the list.
And when I test the middle again, same idea.
Now I'm not actually copying the list, I am simply keeping track of, where are the pieces of the list that bound my search.
Ha, all right?
I'm still reducing the size of the problem by a factor of 2 at each step.
That's great.
All I'd need to do now, though, is just keep track of which portion of the list I'm searching.
I'm going to avoid copying the list.
So the number of recursive calls, again, will be logarithmic.
Let's see if that actually fixes my problem.
A little bit of code, not as bad as it looks-- I've got an internal function here that I'm going to come back to.
But let's look at what happens in this case.
I'm going to say, again, if there's nothing in the list, just return False.
Element can't possibly be there.
Otherwise, call this function with the list, the element for whom I'm searching, and the beginning and end of the list-- so 0 at one end, length of n l minus 1 at the other end.
It's just that idea of, I'm keeping track of the two pieces, OK?
Now let's look at what this does.
It says, here's the low part of the list, the high part of the list.
Initially, it's 0 and length of list minus 1.
It says, if they are the same, oh cool, then I've got a list of length 1.
Just test to see if it's the thing I'm looking for.
If they're not, find the midpoint.
And the midpoint's just the average of low plus high, integer division by 2.
Think about it.
If it's 0 and n, midpoint is n/2.
But if it's, for example, n/2 and n, midpoint is 3/4 n. So that mid picks the middle point.
If it's the thing I'm looking for, great, I'm done.
Otherwise, check to see, is the thing at the middle bigger than or less than the thing I'm looking for.
And based on that-- I'm going to skip this one for a second-- I'm either going to search everything from the low point up to the middle point or from the middle point up to the high point.
And the last piece here is, if, in fact, the low point and the middle point are the same, I've got a list of size 1.
There's nothing left to do.
I'm done.
OK, I know it's a lot of code.
I would invite you just to walk through it.
But I want to take you back again to just, simply, this point and say, here's what we're doing.
We're starting off with pointers at the beginning and end of the list.
We're testing the middle point.
And based on that, we're giving a call where, now, the pointer is to the beginning and the middle of the list, simply passing it down, and same as I go through all of these pieces.
So that code now gives me what I'd like.
Because here, in the previous case, I had a cost.
The cost was to copy the list.
In this case, it's constant.
Because what am I doing?
I'm passing in three values.
And what does it take to compute those values?
It's a constant amount of work, because I'm simply computing mid right there, just with an arithmetic operation.
And that means order log n steps, because I keep reducing the problem in half.
And the cost at each point is constant.
And this is, as a consequence, a really nice example of a logarithmic complexity function.
Now, if you think about it, I'm cheating slightly-- second time today.
Because we said we really don't care about the implementation.
We want to get a sense of the complexity of the algorithm.
And that's generally true.
But here is a place in which the implementation actually has an impact on that complexity.
And I want to be conscious of that as I make these decisions.
But again, logarithmic in terms of number of steps, constant work for each step, because I'm just passing in values.
And as a consequence, the overall algorithm is log.
Notice one other thing.
I said I want you to see characteristics of algorithms that tell you something about the complexity of that algorithm.
Something that's iterative and reduces the problem by size 1 each time, from n, to n minus 1, to n minus 2-- linear.
Something that reduces the size of the problem in half, or in thirds, or in quarters each time-- logarithmic, generally, unless I've got a hidden cost somewhere.
Here's another little example just to give you a sense of log.
I want to convert an integer to a string.
I know I can just call str() on it.
But how might we do that inside of the machine?
Well, here's a nice little algorithm for it.
I'm going to set up something I call digits.
It's just a string of all the digits.
If the thing I'm trying to convert is 0, I just return the string "0".
Otherwise, let's run through a little loop where I take that integer divided by 10, the remainder of that.
What is that?
Oh, that's the zeroth or the 1-order, the first order bit.
And I'm going to index into digits to find that.
And I'm going to add it on to a string that I'm [INAUDIBLE].
And I'll divide i by 10.
So this says, given an integer, I want to convert it to a string.
I divide the integer by 10, take the remainder.
That gives me the zeroth, or if you like, the ones element.
I index into the string, and I record it.
And then I add it to what I get by dividing i by 10 and doing the same thing.
So I'll just walk down each of the digits, converting it into a string.
What I care about is the order of growth here.
This is all constant.
All I want to worry about here is, how many times do I go through the loop.
And inside of the loop, this is just constant.
It doesn't depend on the size of the integer.
So how many times do I go through the loop?
Well, how many times can I divide i by 10?
And that's log of i, right?
So it's not i itself.
It's not the size of the integer.
It's the number of digits in the integer.
And here's another nice example of log.
I'll point you, again, right here.
I'm reducing the size of the problem by a constant factor-- in this case, by 10-- each time-- nice characteristic of a logarithmic algorithm.
OK, we've got constant.
We've got log.
What about linear?
We saw it last time, right?
Something like searching a list in sequence was an example of something that was linear.
In fact, most of the examples we saw last time were things with iterative loops.
So for example, fact, written intuitively-- factorial, right-- n times n minus 1 times n minus 2 all the way down to 1.
I set product to 1.
I go for a loop where i goes from 1 up to n minus 1, or just below n minus 1-- incrementally multiplying product by i and restoring that back away.
Again, we know that this loop here-- how many times do I go through it?
I go through it n times.
The cost inside the loop, there are three steps, changing i, I'm multiplying product times i, I'm storing that value back in product.
And as we saw, that constant doesn't matter.
This is linear.
So n times around the loop, constant cost each time-- order n. What about recursive?
I could write fact recursively.
I actually prefer it this way, right?
If n is less than or equal to 1, return 1.
Otherwise, multiply n by whatever I get by calling this on n minus 1.
The cost inside the loop is just constant.
I'm doing one subtraction, one multiplication.
How many times I go through it?
Again, n times, because I've got to go from n to n minus 1 to n minus 2.
So again, this is linear.
Now, if you were to time it, you'd probably see a difference.
My guess is-- I'm sure Professor Guttag will correct me if I get it wrong-- is that the factorial one probably takes a little more time, because you've got to set up the frame for the recursive call.
But in terms of what we care about, they're the same.
They're both linear.
They're order n. And so interestingly, both iterative and recursive factorial have same order of growth.
Again, I want you to notice, what's the key here.
Reducing the size of the problem by 1 is indicative, generally, of something that's going to have linear growth.
I say in general.
If it's a loop inside of a loop, as we saw, it might be a little bigger.
But this is generally linear.
Constant, log, linear, log-linear-- that is, n log n-- we're going to see this next time.
I'm certainly going to push it ahead.
It invites you to come back on Wednesday and see this.
It's actually something that's a really powerful algorithm.
It's going to be really useful.
We're going to look at something called merge sort, which is a very common sorting algorithm and has that property of being log-linear.
So we'll come back to this next time.
How about polynomial?
Well, we saw this last time as well.
This commonly occurs when we have nested loops or where we have nested recursive function calls-- nested loop meaning I'm looping over some variable, and inside of there, I've got another loop.
And what we saw is the outer loop, if it's a standard iterative thing, will be linear.
But inside of the loop, I'm doing a linear amount of work each time.
So it becomes n times n, so order n squared.
OK, exponential-- these are things-- sorry, yes, I did that right.
I'm going to go back to it.
Exponential-- these are things that we'd like to stay away from, but sometimes, we can't.
And a common characteristic here is when we've got a recursive function where there's more than one recursive call inside the problem.
Remember Towers of Hanoi, that wonderful demonstration I did.
I was tempted to bring it back, because it's always fun to get a little bit of applause when I do it.
But I won't do it this time.
But remember, we looked at that problem of solving the Towers of Hanoi.
How do I move a stack of size n of different-sized disks from one peg to another where I can only move the top disk onto another one and I can't cover up a smaller disk?
Want to remind you, we saw, there was a wonderful recursive solution to that.
It said, move a stack of size n minus 1 onto the spare peg.
Move the bottom one.
And then, move that stack over onto the thing you were headed towards, OK?
What's the complexity of that?
Well, I'm going to show you a trick for figuring that out.
It's called a recurrence relation for a very deliberate reason.
But it'll give us a little, handy way to think about, what's the order of growth here.
So I'm going to let t sub n denote the time it takes to move a tower of size n. And I want to get an expression for, how much time is that going to take.
What do I know?
I know that's 2 times t to the n minus 1, right?
I've got to move a stack of size 1 less onto the spare peg, and then, 1 to move that bottom thing over, and then, whatever it takes me to move a stack of size n minus 1 over to that peg.
OK, so how does that help me?
Well, let's play the same game.
What's t of n minus 1?
Oh, that's 2t of n minus 2 plus 1.
I'm just substituting in.
I'm using exactly the same relationship here.
All right, let's just do a little math on that.
That's 4t to the n minus 2 plus 2 plus 1.
And you're still going, OK, who cares.
Well, let's do the same thing one more time.
t of n minus 2-- that's 2t of n minus 3 plus 1.
Oh, see the pattern?
You can start to see it emerge here, right?
Each time I reduce this, I'm adding another power of 2, and I'm increasing the coefficient out front.
And so, in fact, after k steps, I'll have 1 plus 2 plus 4 all the way up to 2 to the k minus 1 plus 2 to the k times t sub n minus k. Hopefully you can see it if you just look.
This expression is capturing all of those up there.
I'm just pulling it out each time.
When am I done?
When this is size 0, when k is equal to n. And so that's when I get that expression.
If this is going by too fast, just walk it through yourself later on.
But I'm literally just using this expression to do the reduction until I see the pattern.
All right, what's that?
Well, if your Course 18 major, you've seen it before.
If you haven't, here's a nice, little trick.
Let me let a equal that sum, 2 to the n minus 1 plus 2 to the n minus 2 all the way down to 1.
Let me multiply both the left and the right side by 2.
That gives me, 2a is equal to 2 to the n plus 2 to the n minus 1 all the way down to 2.
I'm just taking each of the terms and multiplying them by 2.
Now subtract this from that.
And then on the left side, you get a.
And on the right side, you get that term.
These all cancel out minus 1-- geometric series, cool.
So that sum is just 2 to the n minus 1.
And if I plug that back in there, ah, I've got my order of growth, exponential, 2 to the n. OK, I was a math/physics undergrad.
I like these kinds of things.
But I wanted you to see how we can reason through it, because this is letting us see the growth.
What I want you to pull away from this is, notice the characteristic.
In Towers of Hanoi-- we're going to do another example in a second-- the characteristic was, at the recursive step, I had not one, but two recursive calls.
And that is characteristic of something with exponential growth, which I just saw here, 2 to the n. That, by the way, I'll remind you of the story of Towers of-- Towers of Hanoi, right?
When the priests in that temple move the entire stack from one peg to another, we all reach nirvana, and the world ends.
n is equal to 64 here.
Go figure out what 2 to the 64 is.
And if you're doing one move per second, which they will, I think we're certainly going to be here a while before the universe ends and we reach nirvana, probably several times over
I thought we were already in nirvana
We are in nirvana, we're at MIT.
You're right, John.
But we're worrying about the rest of the world.
So, OK, we'll keep moving on.
Nirvana will be next week when they do the quiz, John.
So we'll keep moving quickly.
All right.
I want to show you one more example.
It's a cool problem from math, but mostly to see that characteristic of exponential growth.
And then we're going to pull all of this together.
This is something called the power set.
So if I have a set of things-- well, let's assume I have a set of integers-- with no repeats-- so 1 through n, 1, 2, 3, 4, for example-- I want to generate the collection of all possible subsets-- so subset with no elements, with one element, with two elements, with three amounts, all the way up to n elements.
So for example, if my set is 1 through 4, then the power set would be the empty set with no elements in it, all of the instances with one element, all of them with two, all of them with three, and all of them with four.
I'd like to write code to generate this.
It's actually handy problem in number theory or in set theory.
By the way, the order doesn't matter.
I could do it this way, but this would be a perfectly reasonable way of generating it as well.
And I'm going to come back to that in a second as we think about solving this.
The question is, how would I go about finding all of these.
I'm going to use-- well, we could stop and say, you could imagine writing a big iterative loop.
You start with n, and you decide, do I include it or not.
And then you go to n minus 1.
Do I include it or not?
And you could think about writing a big loop that would generate all of these-- actually, a bunch of nested loops.
But there's a nice recursive solution.
And I want to encourage you to think that way.
So here's the way I'm going to do it.
What did we do when we said we want to think recursively?
We say, let's assume we can solve a smaller size problem.
If I want to generate the power set of all the integers from 1 to n, I'm going to assume that I can generate the power set of integers from 1 to n minus 1.
If I have that solution, then I can construct the solution to the bigger problem really easily.
Wow.
Well, all of the things that were in that solution to the smaller problem have to be part of the solution to the bigger problem.
They're all subsets of 1 to n, because they're all subsets of 1 to n minus 1.
So I'm going to add all those in.
And then I'm going to say, let's take each one of those and add n to each of those subsets.
Because that gives me all the rest of the solutions.
I've got all the ways to find solutions without n. I get all the ways to find solutions with n. That may sound like a lot of gobbledygook, but let me show you the example.
There is the power set of the empty set.
It's just the empty set.
Get the power set of 1, I include that, and I include a version of everything there with 1 added to it.
There's the power set of 1.
Now, given that, how do I get the power set of 2?
Well, both of those are certainly things I want.
And for each one of them, let me just add 2.
And if you look at that, right, that's the set of all ways of getting nothing, 1, 2, or both of them.
And you get the idea.
Now, having that solution, I can get the solution for 3, because all of those have to belong.
And I simply add 3 to each one of those.
Oh, that's cool, right?
All right, you don't have to be a math geek to admit it's cool.
It is kind of cool.
Because it says, gee, got a solution to the smaller problem.
Generating the next piece is a natural step.
And you can also see, the size of that set's doubling each time.
Because you get to 4, I'm going to add everything in to all of those pieces-- really nice recursive description.
Let's write some code.
So I'll also hand it out to you, but here's the code.
And I'm going to walk through it carefully.
And then we're going to analyze it.
But it's actually, for me, a beautiful piece of code.
I did not write it, by the way, John did.
But it's a beautiful piece of code.
I want to generate all the subsets with a power set of some list of elements.
Here's how I'm going to do it.
I'm going to set up some internal variable called res, OK?
And then, what am I going to do?
Actually, I don't know why I put res in there.
I don't need it.
But we'll come back to that.
If the list is empty, length of the list is 0, I'm going to just return that solution.
And this is not a typo.
What is that funky thing there?
It is a list of one element, which is the empty list, which I need.
Because the solution in this case is a set with nothing in it.
So there is the thing I return in the base case.
Otherwise, what do I do?
I take all the elements of the list except the last one, and I call it recursively.
I generate all of the subsets of that.
Perfect, so I'm going to call that smaller.
I then take the last element, and I make a list of just the last element.
And what did I say I need to do?
I need all of these guys, plus I need all of them where I add that element in-- oh, nice.
I'll set up new as a variable here.
And I'll loop over all of the elements from the smaller problem, where I basically add that list to that list.
And I put it into new.
That's simply taking all of the solutions of subsets of up to n minus 1 and creating a new set of subsets where n is included in every one of them.
And now I take this, and I take that.
I append them-- or concatenate them, rather.
I should say "append them"-- concatenate them together and return them.
That's a crisp piece of code.
And I'm sorry, John, I have no idea why I put res up there.
I don't think I need that anywhere in this code.
And I won't blame it on John.
It was my recopying of the code
[INAUDIBLE]
Sorry?
Maybe
Maybe, right.
Look, I know I'm flaming at you.
I get to do it.
I'm tenured, as I've said multiple times in this course.
That's a cool piece of code.
Imagine trying to write it with a bunch of loops iterating over indices.
Good luck.
You can do it.
Maybe it'll be on the quiz.
Actually, no, it won't.
That's way too hard to ask.
But it's a cool piece of code, because I can look at it and say, what's the solution, solve the smaller problem, and then, given that, take every one of the things in that smaller problem, add that element into it, and put the two pieces together.
Wonderful.
OK, with that in mind, let's see if we can figure out the complexity of this.
Up here, that's constant.
That's OK.
Right there, I've got the recursive call.
So I know, first of all, that this is going to call itself n times, right?
Because each stage reduces the size of the problem by 1.
So if I'm trying to get the power set of n, I'm going to have to do it to get n minus 1, and then, n minus 2.
So I know the recursion of genSubsets() to genSubsets().
This is going to go around n times.
That's not so bad.
But right down here, I've got to figure out, what's the cost of this, all right?
This is constant.
That's setting up as constant.
That's constant.
But there, I've got another loop.
And the loop depends on how big smaller is.
And "smaller's" a bad choice of term here, because it's going to grow on me.
But let's think about it.
By the way, I'm assuming append is constant time, which, generally, it is.
The time I need to solve this problem includes the time to solve the smaller problem.
That recursive call, I know that's going to be linear.
But I also need the time it takes to make the copy of all the things in that smaller version.
So how big is that?
Oh, crud number two-- number of things in the power set grows as a factor of 2, right?
If I've got something of, you know, 1 through 3, I've got all the things with nothing in it, all the things with one in it, all the things with two things in it, all the things with three things in it.
That's 8.
And each time around, I'm doubling the size of it.
So for a set of size k, there are 2 the k cases.
And that says that this loop right here is going to be growing exponentially.
Because I've got to go down that entire list to find all of the pieces.
So what's the overall complexity?
I'm going to play the same game.
Let's let t sub n capture the time it takes to solve a problem of size n. Just temporarily, I'm going to let s sub n denote the size of the solution for a problem of size n. How big is that thing, smaller?
And what do I know?
The amount of time it takes me to solve the problem of size n is the amount of time it takes me to solve the slightly smaller problem-- that's the recursive call to genSubsets()-- plus the amount of time it takes me to run over that loop looking at everything in smaller and adding in a new version, plus some constant c, which is just the number of constant operations, the constant steps inside that loop, OK?
And if I go back to it, t sub n is the cost here.
t sub n minus 1 is the cost there.
s sub n is the size of this.
And then I've got, one, two, three, four, five constant steps.
So c is probably 5 in this case.
So what can I say?
There's the relationship.
Because I know s of n minus 1 is 2 to the n minus 1.
There are 2 to the n minus 1 elements inside of that.
How do I deal with this?
Let's play the same game.
What's t sub n minus 1?
That's t of n minus 2 plus 2 to the n minus 2 plus c. And I could keep doing this.
You can see what the pattern's going to look like.
I'm going to have k times c constant steps.
For each reduction, I'm going to get another power of 2.
And I'm going to reduce this overall term, after k steps, to t the n minus k. When am I done?
When that's down to something of size 0.
And there's the expression.
And what you can see is what I wanted you to see, order n-- or sorry, order 2 to the n-- is exponential in the size of the problem.
What's the characteristic?
Something that has a recursive call-- sorry, multiple recursive calls at each step-- is likely to lead to exponential.
But that can also be buried inside of how I grow the size of the problem.
And that was the case here.
There's only one recursive call, but that loop grows in size each time around.
So the complexity is exponential.
I'm going to pull this together.
I said one of the things I'd like to start to recognize is, what are the characteristics of a choice in algorithm that leads to a particular complexity class.
And you now have some of them.
If the code doesn't depend on the size of the problem, that's constant.
And in fact, we've been using that as we look at pieces of the code.
If we can reduce the problem-- I said, in this case-- by half each time, by some constant factor, from n, to n/2, to n/4, to n/8, that tends to be characteristic-- unless there's a hidden cost somewhere else-- of a logarithmic algorithm.
These are really nice.
Simple things that reduce the size of the problem by 1 at each step-- an iterative call that goes from n, to n minus 1, and then to n minus 2, and then to n minus 3-- characteristic of linear algorithms.
Log-linear we're going to see next time.
Polynomial-- typically quadratic n squared when we have nested loops or nested recursive calls.
I'm looping over something.
Inside of there, I'm looping over something else on a size that depends on the size of the problem.
And then, we just saw this last one.
Multiple recursive calls at each level tends to be characteristic of exponential.
And as I said, we'd like to be as high up in this list as we can, because those are really nice algorithms to have.
Let me give you one more example of looking at this, and then we'll be done.
Fibonacci-- standard problem, right?
The nth Fibonacci number is the sum of the previous two Fibonacci numbers.
This was the example we saw of multiplying rabbits, if you like.
Here's an iterative version of Fibonacci, which says if n is 0, it's just 0.
If it's 1 is just 1.
Otherwise, I'm going to set up, initially, the two previous Fibonacci numbers.
And then I'm just going to run through a loop where I temporarily keep track of that number.
I move the second previous one into the last previous one.
I add those two.
That becomes the second previous number.
And I just keep running through that loop.
You can go run it.
You see it does the right thing.
What I want to look at is the complexity.
So that's constant.
That's constant.
That's linear, because the work inside the loop is constant, but I'm doing it n times.
So this is nice.
[INAUDIBLE] I should say, the bottom thing is constant.
The overall algorithm, the worst case is just order n. Great.
What about the recursive version?
For me, this is much nicer code.
It's nice and clean.
It says if n is equal to 0, fib is 0, 0.
If n is equal to 1, fib of 1-- or the first and second Fibonacci numbers-- are 0 and 1.
Otherwise, just return what I get by summing both of those pieces.
And you can probably already guess what the complexity is going to be here, right?
Because I've now got two recursive calls inside of this call.
So one way to think about it is, if I'm going to solve the problem up here, I've got to solve two versions of the problem below, which has got to solve two versions of the problem below.
And in general, this is going to be exponential, 2 to the n. Now you say, wait a minute.
I was paying attention when this guy was yattering on a couple of weeks ago.
Honest, I was.
And in fact, what we saw was that fib isn't balanced in terms of how it goes, right?
It's not that, on the right-hand side of the tree, I have to solve all of those portions, because the problem gets smaller.
Does that change the complexity?
Well, the answer is, it changes the base, but it's actually still exponential.
And if you want to go look this up, I'm sure you can find Wikipedia very quickly.
This actually has a very cool exponential growth.
It's the golden ratio to the nth power.
And in fact, I encourage you to go look at it in the even more copious spare time you have.
It's a very cool proof to see it.
But the bottom line is, while we can do a little bit better than 2 to the n, it still grows exponentially with n. So what do we have?
We've got big O notation as a way of talking about comparing efficiency of algorithms.
What I want you to see here is that you ought to be able to begin to reason about what's the cost of an algorithm by recognizing those common patterns.
I keep saying it, but it's going to be really valuable to you.
And you should be able to therefore work the other direction.
When you're given a new problem, how do I get this into a linear algorithm if I can?
Log-linear, if I can, would be really great.
But you know, if I can't, how do I stay away from exponential algorithms?
And finally, what we're going to show later on is that, in fact, there are some problems that, as far as we know, are fundamentally exponential.
And they're expensive to compute.
The very last thing is, you might have decided I was cheating in a different way.
So I'm using a set of built-in Python functions.
I'm not going to go through all of these.
But this is just a list, for example, for lists, of what the complexity of those built-in functions are.
And if you look through the list, they kind of make sense.
Indexing, you can go straight to that point.
Computing the length, you compute it once, you've stored it.
Comparison-- order n, because I've got to compare all the elements of the list.
Similarly, to remove something from the list, I've got to find where it is in the list and remove it.
Worst case, that's going to be order n. So you can see that these operations are typically linear in the size of a list.
These are constant.
For dictionaries, remember, dictionaries were this nice thing.
They weren't ordered.
It gave me a power in terms of storing them.
But as a consequence, some of the costs then go up.
For a list, indexing, going to a particular point, I just go to that spot and retrieve it.
Indexing into a dictionary, I have to find that point in the dictionary that has the key and get the value back.
So that's going to be linear, because I have to, in principle, walk all the way down it.
It's a slight misstatement, as we'll see later on.
A dictionary actually uses a clever indexing scheme called a hash.
But in the worst case, this is going to be linear.
So you see a trade-off.
For dictionaries, I get more power.
I get more flexibility.
But it comes as a cost.
And so these, basically, are what let me reason on top of the things I've been doing to figure out complexity.
And next time, we'll do the last piece of this when we look at sorting.
So we'll see you all on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, for the last two lectures we've been talking about analyzing algorithms, complexity, orders of growth.
How do we estimate the cost of an algorithm as the size of the input grows?
And as I've said several times, I'll say at least once more, how do we also turn it the other direction?
How do we use thoughts about choices of pieces of algorithm in terms of implications on the cost it's going to take us to compute?
We saw last time a set of examples-- constant algorithms, linear algorithms, logarithmic algorithms, linear algorithms, quadratic algorithms, exponential algorithms.
Today, what I'm going to do is fill in one more piece, a log linear algorithm-- something that's really a nice kind of algorithm to have-- and use it to talk about one last class of algorithms that are really valuable, and those are searching and sorting algorithms.
So a search algorithm.
Kind of an obvious statement.
You use them all the time when you go to Google or Bing or whatever your favorite search mechanism on the web is.
It's just a way to find an item or a group of items from a collection.
If you think about it, that collection could be either implicit or explicit.
So way back at the beginning of the term, we saw an example of a search algorithm when you were looking for square roots.
And we saw simple things like exhaustive enumeration.
We'd go through all the possibilities.
We saw our first version of bisection search there, where you would do approximations.
Newton-Raphson-- these are all examples of a search algorithm where the collection is implicit.
So all the numbers between some point that some other point.
More common is a search algorithm where the collection is explicit.
I don't know.
For example, I've got all the data records of students and I want to know how do I find a particular student, so I can record that A plus that everybody in this room is going to get next Tuesday on that exam?
That's not a promise.
Sorry.
But we'll work on it.
So could do it implicit, could do it explicit.
Today I want to focus on doing search explicitly.
And it could be on different kinds of collections, but I'm going to focus-- just as an example-- on search over lists.
And to make it a little easier, let's just do search over lists of numbers.
But it could obviously be other kinds of elements.
Now you've already seen some of this, right?
We did search where we said, we can do linear search.
Brute force.
Just walk down the list looking at everything till we either find the thing we're looking for or we get to the end of the list.
Sometimes also called British Museum algorithm or exhaustive enumeration.
I go through everything in the list.
Nice news is, the list doesn't have to be sorted.
It could be just in arbitrary order.
What we saw is that the expected-- sorry, not expected.
The worst case behavior is linear.
In the worst case, the element's not in the list.
I got to look at everything.
So it's going to be linear in terms of complexity.
And then we looked at bisection search, where we said the list needs to be sorted.
But if it is, we can actually be much more efficient because we can take advantage of the sorting to cut down the size of the problem.
And I'll remind you about both of those.
There was our simple little linear search.
Right?
Set a flag that says, I haven't yet found it.
And then just loop over the indices into the list.
I could have also just looped directly over the list itself, checking to see if the ith member of the list is the thing I'm looking for.
If it is, change the flag to true so that when I come out of all of this I'll return the flag-- either false because it was set that way initially or true because I found it.
And of course what we knew is we have to look at everything to see if it's there or not.
I could speed this up by just returning true at this point.
While that would improve the average case, doesn't improve the worst case.
And that's the thing we usually are concerned about, because in the worst case I've got to go through everything.
And just to remind you, we said this is order length of the list.
To go around this part-- the loop right here-- and inside the loop, it's constant work.
I'm doing the same number of things each time.
That's order n times order 1.
And by our rules, that's just order n. So it's linear in the size of the problem.
OK. We said we could do it on sorted lists.
But just again, we'll walk down the list.
Again, here I could loop over everything in the list, checking to see if it's the thing I want.
Return true.
And if I ever get to a point where the element of the list is bigger than the thing I'm looking for, I know it can't be in the rest of the list because all the things to the right are bigger yet.
I could just Return false and drop out.
In terms of average behavior, this is better because it's going to stop as soon as it gets to a point where it can rule everything else out.
But in terms of complexity, it's still order n. Because I still on average have-- not average.
In the worst case, I'm still going to be looking n times through the loop before I get to a point where I can decide to bail out of it.
So order n. And then finally-- last piece of recap-- bisection search.
Repeat again.
The idea here is, take the midpoint of the list.
Look at that element.
If it's the thing I'm looking for, great.
I just won the lottery.
If it isn't, decide is the thing I'm looking for bigger or less than that middle point.
If it's bigger than that, I only use the upper half of the list.
If it's less than that, I only use the lower half of the list.
And the characteristic here was, at each step, I'm reducing the size of the problem in half.
I'm throwing away half of the remaining list at each step.
And I'll just remind you of that code.
I know it's a lot here, but just to remind you.
It said, down here if I've got an empty list, it can't be there.
I'm going to Return false.
Otherwise call this little helper function with the list, the thing for which I'm searching, and the beginning and end point indices into the list.
Initially the start and the very end.
And this code up here basically says, if those two numbers are the same I'm down to a list of one.
Just check to see if it's the thing I'm looking for.
Otherwise, pick something halfway in between.
And ignore this case for the moment.
Basically then check to see, is the thing at that point bigger than e?
In which case, I'm in general going to call this only with from the low point to the midpoint.
Otherwise I'm going to call this with the midpoint to high.
And that was just this idea of, keep cutting down in half the size of the list.
Last piece of the recap-- the thing we wanted you to see here-- is there are the two recursive calls.
I'm only going to do one because I'm making a decision.
At each step, I'm cutting down the problem by half.
And that says the number of steps, the number of times I'm going to iterate through here, will be log in the length of the list.
And if that still doesn't make sense to you, it says, I need to know when 1 over 2 to the k-- where k is the number of steps-- is equal to 1.
Because in each step, I'm reducing by half.
And that's when k is log base 2 of n. So that's why it's log linear.
And so this just reminds you.
Again, that recap.
Number of calls reduced-- or, sorry.
The call gets reduced by a factor or two each time.
I'm going to have a log n work going around it.
And inside it's a constant amount of work because I'm just passing the pointers, I'm not actually copying the list.
And that's a nice state to be.
OK, so-- sounds good.
Could just use linear search.
It's going to be linear.
When you use binary search or bisection search, we can do it in log time.
That's great.
We assumed the list was sorted, but all right.
So that lens basically says, OK.
So when does it make sense to sort the list and then do the search?
Right?
Because if I can sort the list cheaply, then the search is going to be logarithmic.
That's really what I would like.
This little expression basically says, let's let sort be the cost of sorting the list.
I want to know when that cost plus something that's order log n-- which is what it's going to cost me to do this search.
When is that less than something that's order n?
Because then it's going to be better to do the sort first than do the search.
And so I can just rearrange it.
It needs to be, when does the cost of sorting-- when is it last than this expression?
Which basically says, when is sorting going to be less expensive than the linear cost?
Crud.
Actually, good news for you, right?
This is a really short lecture.
Because it says it's never true.
Ouch.
Don't worry.
We've got more to go on the lecture.
The reason it can't be true-- if you think about it just informally-- is, if I've got a collection of n elements and I want to sort it, I've got to look at each one of those elements at least once.
Right?
I have to look at them to decide where they go.
Oh, that's n elements.
So sorting must be at least order n, because I got to look at everything.
And in fact as it says there, I'm going to have to use at least linear time to do the sort.
Sounds like we're stuck, but we're not.
And the reason is, often when I want to search something I'm going to do multiple searches, but I may only want to sort the list once.
In fact, I probably only want to sort the list once.
So in that case, I'm spreading out the cost.
I'm amortizing the expense of the sort.
And now what I want to know is, if I'm going to do k searches, the cost of those k searches I know is going to be k log n-- because it's log to do the search.
And I simply need to know, is the cost of sorting plus this-- can I have something where it's less than k searches just using linear search?
And the answer is, yes.
There are going to be, for large k's, ways in which we can do the sort where the sort time becomes irrelevant, that the cost is really dominated by this search.
And so what I want to do now is look at-- all right.
How could we do the sort reasonably efficiently?
It's going to have to be at least linear.
We're going to see it's going to be a little more than linear.
But if I could do it reasonably, I'm going to be in good shape here.
So what I want to do is show you a number of ways in which we can do sorting-- take a list of elements and sort them from, in this case, smaller to higher or increasing order.
So here's my goal.
I want to efficiently sort a list.
I want to see if we can do this as efficiently as possible.
I'm going to start, you might say, with a humorous version of sort.
You're all convinced that my humor is non-existent.
You're right.
But it sets the stage for it.
This is a sort.
You can look it up.
It's called monkey sort, BOGO sort, stupid sort, slow sort, permutation sort, shotgun sort.
And here's how it works.
Anna has nicely given me a set of numbers on cards here.
Here's how you do BOGO sort.
I got to do that better.
I got to spread them out randomly, like this.
Oh good.
I'm going to have to-- sorry, Tom.
I'm not walking.
And now I pick them up, saying, is that less than this?
Which is less than-- oh, crud.
They're not sorted.
All right.
I pick them all up and I do it again.
A little brain damage, right?
Now it's intended to get your attention.
I did.
I heard a couple of chuckles.
Those are A students, by the way.
I heard a couple of chuckles here.
We could actually do this exhaustively.
Basically it's called permutation sort because you could search through all possible permutations to see if you find something that's sorted.
That, by the way-- the complexity of that is something like n factorial, which for large n is n to the nth power.
And if n's anything bigger than about 2, don't do it.
Right?
But it would be a way to think about doing this.
All right.
Now, having caught the humorous version of this, how could we do this a little bit better?
Oh sorry.
I should say, what's the complexity?
There's a nice crisp definition of BOGO sort.
Its best case is order n, because I just need to check it's sorted.
Its average case is n factorial and its worst case, if I'm just doing it randomly, is God knows.
Because I could be doing it here forever.
So we're going to move on.
Here's a second way to do it called bubble sort.
I'm going to do this with a small version of this.
I'm going to put out a set.
I'll turn these up so you can see them in a second.
The idea of bubble sort is, I'm going to start at-- I'm going to call this the front end of the list.
And I'm going to walk down, comparing elements pairwise.
And I'm always going to move the larger one over.
So I start here and I say, 1 is less than 11.
I'm OK. 11's bigger than five.
I'm going to bubble that up.
11's bigger than 6.
I'm going to bubble that up.
11's bigger than 2.
I've basically bubbled 11 to the end.
Now I go back here.
I say, 1 is less than 5.
That's good.
5 is less than 6.
That's good.
Ah, 6 is bigger than 2.
Bubble that.
6 is less than 11.
You get the idea-- comparison, comparison, and swap.
Comparison, comparison.
And now if I go back to this part and do it, you'll notice that's in the right order.
That's in the right order.
That's in the right order.
That's in the right order.
I'm done.
Small round of applause, please.
I was able to sort five elements.
Thank you.
The little video is showing the same thing.
You can see the idea here.
It's called bubble sort because you're literally bubbling things up to the end of the list.
It's pretty simple to do.
You're just swapping pairs.
And as you saw, when I get to the end of the list I go back and do it until I have a pass where I go all the way through the list and I don't do any swaps.
And in that case I know I'm done because everything's in order, and I can stop.
One of the properties of it is that the largest unsorted element is always at the end after the pass.
In other words, after the first one I know that the largest element's at the end.
After the second one, the largest thing left is going to be in the next place.
And that tells me, among other things, that this is going to take no more than n times through the list to succeed.
It might actually take fewer than that.
OK. Again let's look at some code for it.
Let's look at its complexity and let's actually run this.
So here is a little simple version of bubble sort.
I'm going to set a flag up here.
I'm going to call its swap initially to false.
That's going to let me tell when I'm done, when I've gone through everything in the list without doing a swap.
And then I'm going to loop.
As long as swap is false-- so the first time through it's going to do that loop.
I set swap initially to true, and notice what I then do.
I let j range from 1 up to the length of the list, and I look at the jth element and the previous element.
If the previous element is bigger, I'm going to flip them.
Right there.
And that's just doing that swap, what I just did down here.
And if that's the case, I'm going to set the flag to false.
Which says, I've done at least one bubble as part of this.
Which means when I come out of here and go back around to the loop, it's going to do it again.
And it will do it until all of this succeeds without this ever being true, in which case that's true, which makes that false.
And it will drop out.
OK?
Let's look at an example of this running.
It's just to give you a sense of that, assuming I can find the right place here.
So there is, again, a version of bubble sort on the side.
And I'm going to bring this down to the bottom I've got a little test list there.
And I've put a print statement in it.
So you can see each time through the loop, what's the form of the list as it starts.
And assuming I've done this right-- here you go.
There's the list the first time through.
Notice after one pass, 25's at the end of the list-- the biggest element.
Exactly what I like.
But you can also see a few other things have flipped.
Right?
Right in there, there have been some other swaps as it bubbled through.
And in fact, you can see it's-- well, you can see that idea.
You can see 25 moving through.
Notice on the next step, a whole bunch of the list is actually in the right order.
It's just because I got lucky.
All I can guarantee is that the second largest element is the second from the end of the list.
But you can see here.
Even though the list is, I think, nine long, it only took us four passes through.
So this is nice.
It says, at most, n times through the list.
And at the end, we actually get out something that's in the right form.
OK.
So let's go back to this and basically say, what's the complexity?
Well that's length n, right?
Has to be.
I'm going through the entire list.
And inside of there is just constant work.
Four operations.
I'm doing a test.
Sorry, five.
I'm doing a test.
And then depending whether that test is true or not, I'm setting a flag and doing some movement of things around.
But it's just constant.
I don't care about the five.
And there, how many times do I go around the loop?
In the worst case, n. All I can guarantee is, after the first pass the biggest thing is here.
After the second pass, the second biggest thing is there.
After the third pass-- you get the idea.
So I've got order and things inside the loop, and I'm doing that loop n times.
And I hope that looks familiar.
We've talked about this.
Right?
This is nested loops.
What's this?
Quadratic.
So it's order n squared, where n is the length of the list.
Now as you also saw, on average, it could be less than that.
But it's going to be order n squared.
OK. That's one possibility.
Here's a second nice, simple, sort algorithm.
It's called selection sort.
You can kind of think of this as going the other way.
Not completely, but going the other way.
And when I say going the other way, the idea here is that I'm going to find the smallest element in the list.
And I'm going to stick it at the front of the list when I'm done, and simply swap that place with whatever was there.
Flip them.
I might do a few other flips along the way, depending how I implement this.
Next, pass.
I'm just going to look at everything but the first element, because I know that one's done.
I'm going to do the same thing.
Find the smallest element remaining in the list, put it in the second spot, and keep doing that.
What I know is, if I implement this correctly, after i steps the first i elements of the list will be sorted.
And everything in the rest of the list has to be bigger than the largest thing in the first part of the list.
OK.
So we could build that.
Before we do it, I'm going to show you a little video starring Professor Guttag.
This is his cameo performance here.
But I want to just show you an example of this using not numbers, but people.
[VIDEO PLAYBACK] - All right.
So now we're going to do selection sort.
The idea here is that each step we're going to select the shortest person and put them next in line of the sorted group.
So we'll bring the leftmost person forward, and we will compare her to everybody else.
So one at a time, step forward.
You're still the winner.
You go back.
Please step forward
And watch the number of comparisons that go on, by the way.
We're going to come back to that.
- Next.
Still the winner.
Next.
Ah.
A new winner.
All right.
So you can take her place
So here, we're choosing to actually insert into the spot in the Line We could have put her back at the front, but either one will work.
- Now we'll compare.
Same old winner.
Same winner.
No change.
It's getting kind of boring.
Don't fall, that-- same winner.
Please
This is a tough one.
- Oh.
Close, but I think you're still shorter.
All right.
Next.
No change, which means you are the first in line.
Congratulations
So, smallest element now going to be the first slot.
- Now you step forward, and we'll compare you
I would invite you to watch the left hand of the list.
Notice how it is slowly building up at each stage to have that portion sorted.
And we deliberately admit students to be of different heights, so John can do this demo.
- You are the winner.
Take your place in line.
Next.
It's you.
And once again, we have a lovely group of students sorted in height order.
[END PLAYBACK] [APPLAUSE]
And check out-- I want you to remember number of comparisons-- 55.
Not that the [INAUDIBLE], but I want you to see a comparison as we go on in a second.
So again, selection sort.
This is this idea of, find the smallest element.
Put it at the front.
I might do a little number of flips, as you can see, here along the way.
But this is the same animation of that.
So let's first of all convince ourselves it will do the right thing, and then look at some code, and then run the code.
So to convince ourselves that this is going to do the right thing, we could talk about something that we often refer to as a loop invariant.
We're going to write a loop, but we're going to walk through this.
And the invariant here-- and we want to just demonstrate if it's true at the beginning and it's true at each step.
Therefore, by induction as we did earlier, I can conclude it's true always.
Is that if I'm given the prefix or the first part of a list from 0 up to i, and a suffix or a second part of the list from i plus 1 up to the end of the overall list-- given that, then I want to assert that the invariant is that the prefix is sorted and no element of the prefix is larger than the smallest element of the suffix.
Just what I said earlier.
It says, at any stage here-- if this is the amount of sort I've done so far-- I can guarantee, I'm going to claim, this will be sorted.
And everything here is bigger than that thing there.
How do I prove it?
Well the base case is really easy.
In the base case, the prefix is empty.
I don't have anything, so it's obviously sorted.
And everything in the suffix is bigger than anything in the prefix.
So I'm fine.
And then I just want to say, as long as I write my code so that this step is true, then I'm going to move the smallest element from the suffix-- the second part of the list-- to the end of the prefix.
Since the prefix was sorted, this is now sorted.
And everything in the suffix is still going to be bigger than everything in the prefix.
And as a consequence, by induction, this is going to give me something that says it's always going to be correct.
So here's code that would do that.
Here.
I'm just going to set a little thing called the start of suffix, or soft start.
Initially it's going to point to the beginning of the list.
And then I'm going to run a loop.
And as long as I still have things to search in the list, that that pointer doesn't point to the end of the list, what am I going to do?
I'm going to loop over everything from that point to the end of the list, comparing it to the thing at that point.
If it's less than, I'm going to do a swap because I wanted to move it up.
And you can see, by the time I get through this loop I will have found the smallest element in the remainder of the list.
And I would have put it at that spot, whatever suffix start points to.
And when I've done all of that, I just change this by one.
Having found the smallest element, I've stuck it at spot zero.
I'll do the same thing.
Having found the next smallest element, I know it's at point one.
And I'll just continue around.
One of the things you can see here is, as opposed to bubble sort, this one is going to take n times around the loop because I'm only moving this pointer by one So it starts at 0, and then 1, and then 2, all the way up to n minus 1.
You can also see in this particular implementation, while I'm certainly ensuring that the smallest element goes into that spot, I may do a few other flips along the way.
I'm going to find something I think is the smallest element, put it there and put that element here.
And then when I find another smaller element, I may do that flip.
I could have implemented this where I literally search for the smallest element and only move that.
Doesn't make any difference in terms of the complexity.
All right.
What's the complexity here?
Already said this part.
I will loop n times, because I start at 0 and then 1.
You get the idea.
Inside of the loop I'm going to walk down the remainder of the list, which is initially n. And then n minus 1, and then n minus 2 times.
But we've seen that before as well.
While they get shorter, that complexity is still quadratic.
Order n times going through this process.
Within the process, order n things that I have to compare.
And yes, n gets smaller.
But we know that that n term, if you like to dominate.
So again, this is quadratic.
OK. Before you believe that all sorting algorithms are quadratic, I want to show you the last one, the one that actually is one of the-- I think-- the prettiest algorithms around, and a great example of a more efficient algorithm.
It's called merge sort.
Merge sort takes an approach we've seen before.
We talked about divide and conquer.
Break the problem down into smaller versions of the same problem.
And once you've got those solutions, bring the answer back together.
For merge sort, that's pretty easy.
It says, if I've got a list of 0 or 1 elements, it's sorted.
Duh.
OK.
If I got a list of more than 1 element, here's my trick.
I'm going to split it into two lists.
I'm going to sort them.
And when I'm done, I'm just going to merge those two lists into one list.
And the merge is easy.
Because if I've got two lists that are sorted, I just need to look at the first element of each, take the one that's smaller.
Add it to my result.
And keep doing that until one of the lists is empty.
And then just copy the remainder of the other list.
You can probably already get a sense of what the cost is going to be here, because this is cutting the problem in half.
Now I've got two pieces.
So I need to think about both of them.
I want to give you a couple of visualizations of this.
Here's the first one.
It says, basically, I've got a big unsorted list.
I'm going to split it.
And I'm going to split it.
And I'm going to split it.
Until I get down to just lists that are either 0 or 1, which by definition are sorted.
And once I'm at that level, then I just have to merge them into a sorted list and then merge them pairwise into a sorted list.
And you get the idea.
So it's divide and conquer.
The divide is dividing it up into smaller pieces.
The conquer is merging them back together.
And we have Professor Guttag back for an encore, together with his students.
So let's show you an example of merge sort.
[VIDEO PLAYBACK] - So we're about to demonstrate merge sort.
And we're going to sort this rather motley collection of MIT students by height.
So the first thing we need to do is, we're going to ask everyone to split into a group of two.
So you split a little bit.
You two are together.
You two are together.
You two are together.
You two are together.
And you are all by yourself.
I'm sorry
Poor Anna.
- All right.
So now let's take the first group.
Take a step down.
And what we do is, we sort this group by height, with the shortest on the left.
And look at this.
We don't have to do anything.
Thank you.
Feel free to go back up.
We then sort the next pair.
Please.
And it looks to me like we need to switch.
All right.
Take a step back.
Ladies-- OK. Ladies, gentlemen-- also OK. And again,
Notice each subgroup is now sorted.
Which is great.
- And I think you're in the correct order.
Now what we do is, we take these groups and merge the groups.
So let's have these two-- going to sort these groups, have them step forward.
And now what we're doing is, we're doing a merge of the two sorted groups.
So we start by merging them.
We'll take the leftmost person in this group and compare her to the first person in this group, and decide.
She's still the shortest.
Take a step back.
Now we're going to look at you and say, you're actually taller than this fellow.
So you now step up there.
And we're good here.
Both of you take a step back.
Now we'll take these two groups and follow the same procedure.
We'll merge them.
Let's see.
We'll compare you-- the first person in this group to the first person in this group.
Now it's a little tricky.
So let's see, the two of you compare.
Let's see, back to back.
We have a winner.
Step back.
And now we need to compare the shortest person in this group to the shortest person in this group.
We have a winner.
It's you.
I'm sorry.
And now we just-- we're OK.
Please step back.
Now we'll have these two groups come forward.
We'll compare the shortest person in this group to the shortest person in that group.
I actually need you guys to get back to back here.
You are the winner.
And it's pretty clear that the shortest person in this group is shorter than the shortest person in that group.
So you go there and you step back
Notice the groups.
Now all sorted.
- And now we repeat the same process
And notice how the whole subgroup now goes up once we know that one group is empty.
- And you can see that we have a group of students sorted in order by height.
[END PLAYBACK] [APPLAUSE]
Remember the first number, right?
55, 28.
Now it's just numbers but you can see the expectation is, this is going to take less time.
And it certainly did there.
So again just to demo another way visually.
I'm sorting-- sorry.
I am splitting down until I get small things, and then just merging them up.
I may have to do multiple passes through here, but it's going to be hopefully faster than the other methods we looked at.
I'm going to show you code in a second, and then we're going to run it just to see it.
But let me stress one more time just the idea of merging.
You can see the idea.
I keep splitting down till I got something small enough.
And I want to merge them back.
The idea of merging-- you've seen it from Professor Guttag.
But I just want to highlight why this is going to be efficient.
If I've got two lists: list 1 and list 2, the things left there.
Process is very simple.
I pull out the smallest element of each.
I compare them.
And I simply put the smallest one into the result, move on in that first list.
So the 1 disappears from that left list.
And now again I pull up just the smallest element of each one, do the comparison.
Smallest one goes to the end of my result.
And I drop that element from its list.
So I've now taken 1 from list 1 and one from list 2.
You get the idea.
The reason I want to give you this visualization-- sorry.
Let me do the last step.
Once I get to a place where one of the lists is empty, just copy the rest of the list onto the end.
You can see already a hint of the code.
And that is, that I'm only going to ever look at each element of each sublist once as I do the merge.
And that's a nice property.
Having had them sorted, I don't need to do lots of interior comparisons.
I'm only comparing the ends of the list.
I only, therefore, look at each element-- the number of comparisons, rather, I should say.
I may look at each element more than once.
The number of comparisons is going to be, at most, the number of elements in both lists.
And that's going to be a nice Q as we think about how to solve it.
So here's the code to merge, and then we'll write Merge Sort.
And I know there's a lot of code here, but we can walk through it and get a good sense of it.
I'm going to set up a variable called Result that's going to hold my answer.
And I'm going to set up two indices, i and j, that are initially 0.
They're pointing to the beginning.
And remember, the input here is two lists that we know are sorted-- or should be sorted, or we screwed up in some way.
So initially, i and j are both pointing to the beginning of the left and right list.
And look at what we do.
We say, as long as there's still something in the left list and still something in the right list-- i is less than the length of left, j is less than the length of right.
Do the comparison.
If the left wants smaller, add it to the end of result.
To the end of result, right?
I'm appending it because I want it to be in that sorted order.
And increase i.
If it's not, add the right one to the end of result and increase j.
And I'll just keep doing that until I exhaust one of the lists.
And when I do I can basically say, if the right list is empty, I know if I get out of here they can't both be true.
In other words, if there's still something in the left list, just put it on the end.
Otherwise if the only things left are in the right list, just put them on the end.
So I'm just walking down the list, doing the comparison, adding the smallest element to my result.
And when I'm done, I just return result.
Complexity we can already begin to see here, right?
This says the left and right sublists are ordered, so I'm just moving the indices depending on which one holds the smaller element.
And when I get done, I'm just returning the rest of the list.
So what's the complexity here?
I'm going to do this a little more informally.
You could actually do that kind of relationship I did last time.
But what am I doing?
I'm going through the two lists, but only one time through each of those two lists.
I'm only comparing the smallest elements.
So as I already said, this says that the number of elements I copy will be everything in the left list and everything in the right list.
So that order is just the length of left plus the length of right.
And how many comparisons do I do?
The most I have to do is however many are in the longer list.
Right?
That's the maximum number I need to have.
Oh, that's nice.
That says, if the lists are of order n-- I'm doing order n copies, because order n plus order n is just 2n, which is order n-- then I'm doing order n comparisons.
So it's linear in the length of the lists.
OK.
Sounds good.
That just does the merge.
How do I do merge sort?
Well we said it.
Break the problem in half.
Keep doing it until I get sorted lists.
And then grow them back up.
So there's merge sort.
It says, if the list is either empty or of length 1, just return a copy of the list.
It's sorted.
Otherwise find the middle point-- there's that integer division-- and split.
Split the list everything up to the middle point and do merge sort on that.
Split everything in the list from the middle point on.
Do merge sort on that.
And when I get back those two sorted lists, just merge them.
Again, I hope you can see what the order of growth should be here.
Cutting the problem down in half at each step.
So the number of times I should have to go through this should be to log n the size of the original list.
And you can see why we call it divide and conquer.
I'm dividing it down into small pieces until I have a simple solution and then I'm growing that solution back up.
So there is the base case, there's the divide, and there's the nice conquer [INAUDIBLE] piece of this.
OK.
I'm going to show you an example of that.
But let's actually look at some code-- sorry about that.
Let's look at some code to do this.
And in fact I meant to do this earlier and didn't.
I also have a version of bubble sort here.
Sorry-- selection sort.
I've already done bubble sort.
There is selection sort.
Let's uncomment this.
And let's run both of those and just see the comparison between them.
Yeah, sorry-- just make that a little easier to read.
There we go.
So we saw a bubble sort.
It only went through four times, so less than n times.
There's selection sort.
And as I said to you, it has to do n passes it because it can only ever guarantee that it gets one element at the beginning.
So you can in fact see, in this case, from the first or after the initial input until the end of the first step, it looks like it didn't do anything because it determined eventually that one was in the right spot.
And similarly I think there's another one right there where it doesn't do any-- or appears not to do anything.
All it's guaranteeing is that the next smallest element is in the right spot.
As we get through to the end of it, it in fact ends up in the right place.
And then let's look at merge sort and do one more visualization of this.
Again let me remove that.
If we run it-- again, I've just put some print statements in there.
Here you can see a nice behavior.
I start off calling Merge Sort with that, which splits down into doing Merge Sort of this portion.
Eventually it's going to come back down there and do the second one.
It keeps doing it until it gets down to simple lists that it knows are sorted.
And then it merges it.
Does the smaller pieces and then merges it.
And having now 2 merged things, it can do the next level of merge.
So you can see that it gets this nice reduction of problems until it gets down to the smallest size.
So let's just look at one more visualization of that and then get the complexity.
So if I start out with this list-- sorry about that.
What I need to do is split it.
Take the first one, split it.
Keep doing that until I get down to a base case where I know what those are and I simply merge them.
Pass it back up.
Take the second piece.
Split it until I get down to base cases.
Do the merge, which is nice and linear.
Pass that back up.
Having done those two pieces, I do one more merge.
And I do the same thing.
I want you to see this, because again you can notice how many levels in this tree log.
Log in the size.
Because at each stage here, I went from a problem of 8 to two problems of 4.
Each of those went to two problems of 2, and each of those went to two problems of size 1.
All right.
So the last piece is, what's the complexity?
Here's a simple way to think about it.
At the top level, I start off with n elements.
I've got two sorted lists of size n over 2.
And to merge them together, I need to do order n work.
Because as I said I got to do at least n comparisons where n is the length of the list.
And then I've got to do n plus n copies, which is just order n. So I'm doing order n work.
At the second level, it gets a little more complicated.
Now I've got problems of size n over 4.
But how many of them do I have?
4.
Oh, that's nice.
Because what do I know about this?
I know that I have to copy each element at least once.
So not at least once.
I will copy each element exactly once.
And I'll do comparisons that are equal to the length of the longer list.
So I've got four sublists of length n over 4 that says n elements.
That's nice.
Order n. At each step, the subproblems get smaller but I have more of them.
But the total size of the problem is n. So the cost at each step is order n. How many times do I do it?
Log n. So this is log n iterations with order n work at each step.
And this is a wonderful example of a log linear algorithm.
It's n log n, where n is the length of the list.
So what you end up with, then, is-- all right, a joke version, some reasonable ways of doing sort that are quick and easy to implement but are quadratic, and then an elegant way of doing the search that's n log n. And I'll remind you I started by saying, as long as I can make the cost of sorting small enough I can amortize that cost.
And if you go back and look at last lecture's notes, you'll see n log n grows pretty slowly.
And it's actually a nice thing to do.
It makes it reasonable to do the sort.
And then I can do the search in order n time.
And here's the last punchline.
It's the fastest we can do.
I'm going to look at John again.
I don't think anybody has found a faster sort algorithm.
Right?
This is the best one can do.
Unless you do-- sorry, the best worst case.
I'm sorry.
John is absolute right.
There are better average cases.
Again, our concern is worst case.
So this is as good as we're going to do in terms of a worst case algorithm.
So there you now have sorting algorithms and searching algorithms, and you've now seen-- excuse me, sorry-- constant, log, linear, log linear, quadratic, and exponential algorithms.
I'll remind you, we want things as high up in that hierarchy as possible.
All right.
I have six minutes left.
Some of you are going to leave us.
We're going to miss you, but that's OK.
I'm sure we'll see later on.
For those of you hanging around, this isn't a bad time just to step back and say, so what have we seen?
And I want to do this just very quickly.
I'm sorry.
And I'll remind you, we started by in some sense giving you a little bit of a contract of things we were going to show you.
And I would simply suggest to you, what have we done?
We've given you a sense of how to represent knowledge with data structures, tuples, lists, dictionaries, more complicated structures.
We've shown you some good computational metaphors, iteration, and loops.
Recursion has a great way of breaking problems down into simpler versions of the same problem.
And there really are metaphors.
There are ways of thinking about problems.
We've given you abstraction, the idea of capture a computation, bury it in a procedure.
You now have a contract.
You don't need to know what happens inside the procedure as long as it delivers the answer it says it would.
Or another way of saying it, you can delegate it to somebody and trust that you're going to get what you like out of it.
We've seen classes and methods as a wonderful way to modularize systems, to capture combinations of data and things that operate on them in a nice, elegant way.
And we just spent a week and a half talking about classes of algorithms and their complexity.
If you step up a level, what we hope you've gotten out of this are a couple of things.
You've begun to learn computational modes of thinking.
How do I tackle a problem and divide and conquer?
How do I think about recursion as a tool in dealing with something?
You've begun to-- begun, I will use that word deliberately-- to master the art of computational problem solving.
How can you take a problem and turn it into an algorithm?
And especially, you've begun to have the ability to make the computer do what you want it to.
To say, if I've got a problem from biology or chemistry or math or physics or chemical engineering or mechanical engineering, how do I take that problem and say, here's how I would design an algorithm to give me a simulation and a way of evaluating what it does.
And so what we hope we've done is, we've started you down the path to being able to think and act like a computer scientist.
All right.
Don't panic.
That doesn't mean you stare at people's shoes when you talk to them.
Not all computer scientists do that, just faculty.
Sorry, John.
So what do computer scientists do?
And this is actually meant to be serious.
And I put up two of my famous historical figures of computer scientists.
They do think computationally.
They think about abstractions, about algorithms, about automated execution.
So the three A's of computational thinking.
And in the same way that traditionally you had the three R's of reading, writing, and arithmetic, computational thinking we hope is becoming a fundamental that every well-educated person is going to need.
And that says, you think about the right abstraction.
When you have a problem in your [INAUDIBLE] what's the right abstraction?
How do I pull apart the pieces?
How do I think about that in terms of decomposing things into a relationship that I can use to solve problems?
How do I automate?
How do I mechanize that abstraction?
How do I use what I know happens inside of the machine to write a sequence of steps in a language I'm using to capture that process?
And then finally, how do I turn that into an algorithm?
And that not only means I need a language for describing those automated processes, and if you like allowing the abstraction of details, but frankly also a way to communicate.
If you have to think crisply about how do I describe an algorithm, it's actually giving you a way to crystallize or clarify your thinking about a problem.
This is not to say you should talk to your friends in Python.
I don't recommend it.
But it does say you should use that thinking as a way of capturing your ideas of what you're going to do.
And that leads, then, to this idea of, how difficult is a problem?
How best can I solve it?
We've shown you these complexity classes and we've hinted at the idea that in fact some problems are inherently more difficult than others.
That's something I hope you come back to as you go along.
And especially we want you to start thinking recursively.
We want you to think about how do I take a hard problem, break it up into simpler versions of the same problem, and then construct the solution.
And that shows up lots of places.
Right?
Recursion is in all sorts of wonderful places.
So just to give you an example, I could say to you recursively, "This lecture will end when I'm done talking about this lecture, which will end when I'm done talking about this lecture, which will end when I'm done--" All right.
You don't like infinite recursion.
Good luck on the exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, and welcome to 6.01.
I'm Denny Freeman.
I'm the lecturer.
One thing you should know about today is that there's a single hand-out.
You should have picked it up on your way in.
It's available at either of the two doors.
What I want to do today in this first lecture is mostly focus on content.
But before I do that, since 6.01 is a little bit of an unusual course, I want to give you a little bit of an overview and tell you a little bit about the administration of the course.
6.01 is mostly about modes of reasoning.
What we would like you to get out of this course is ways to think about engineering.
We want to talk about how do you design, how do you build, how do you construct, how do you debug complicated systems?
That's what engineers do, and we're very good at it.
And we want to make you very good at it.
We're very good at it.
And you know that from your common, everyday experience.
Laptops are incredible.
As we go through the course, you're going to see that laptops incorporate things from the tiniest, tiniest level, things so small that you can't see them.
They're microscopic.
The individual transistors are not things that you can see.
We develop special tools for you even to be able to visualize them.
And yet, we conglomerate billions of them into a system that works relatively reliably.
Now, I realize I'm going out on a limb because when you say things like that, then things always fail.
But I'll go out on a limb and say, for the most part, the systems we construct are very reliable.
We'd like you to know how you think about making such a complicated system and making it reliable.
We want to tell you about how you would model things.
How do you gain insight?
How do you get predictability?
How do you figure out how something will work before you've built it?
If you're limited to trying out how things work by actually constructing it, you spend a lot of time constructing things that never make it.
We want to avoid that by -- where we can -- making a model, analyzing the model, making a prediction from the model, and using that prediction to build a better system on the first try.
We want to tell you about how to augment the physical behavior of a system by putting computation in it.
That's a very powerful technique that is increasingly common in anything from a microwave to a refrigerator.
We'd like you to know the principles by which to do that.
And we'd like you to be able to build systems that are robust to failure.
That's a newer idea.
It's something that people are very good at.
If we try to do something, and we make a mistake, we know how to fix it.
And often, the fix works.
We're less good at doing that in constructing artificial systems, in engineering systems.
And we'd like to talk about principles by which we can do that.
So the goal of 6.01 is, then, really to convey a distinct perspective about how we engineer systems.
Now, having said that, this is not a philosophy course.
We are not going to make lists of things to do if you want it to be robust.
We're going to learn to do things by actually making systems.
This is an introductory engineering course.
And so you're going to build things.
The idea is going to be that in constructing those things, we've written the exercises so that some of those important themes become transparent.
So the idea is -- this is introductory engineering.
You'll all make things.
You'll all get things to work, and in the process of doing that, learn something about the bigger view of how quality engineering happens.
So despite the fact that we're really about modes of reasoning, that will be grounded in content.
We selected the content very broadly from across EECS.
EECS is an enormous endeavor.
We can't possibly introduce everything about EECS in one subject.
That's ridiculous.
However, we wanted to give you a variety.
We wanted to give you a sense of the variety of tasks that you can use, that you can apply the same techniques to.
So we want to introduce modes of reasoning, and then show you explicitly how you can use those modes of reasoning in a variety of contexts.
So we've chosen four, and we've organized the course around four modules.
First module is software engineering, then signals and systems, then circuits, then probability and planning.
Even so, even having chosen just four out of the vast number of things we could have chosen, there's no way we can tell you adequately-- we can't give you an adequate introduction to any of those things either.
What we've chosen to do instead is focus on key concepts represented by the asterisks.
The idea is going to be we choose one or two things and really focus on those deeply so you get a thorough understanding not only of how that fits within, for example, the context of software engineering, but also how that concept ramifies into other areas.
Notice that I tried to choose the stars so they hit multiple circles.
That's what we're trying to do.
We're trying to not only introduce an idea to you, but also show you how it connects to other ideas.
So the idea, then, is to focus on a few, we hope, very well-chosen applications that will demonstrate a variety of powerful techniques.
Our mantra, the way we intend to go about teaching this stuff, is practice, theory, practice.
There's an enormous educational literature that says-- whether you like it or not-- people learn better when they're doing things.
You have a lot of experience with that.
You have a lot of experience on the other side, too.
I'll try to forget the other side, or at least try to wipe it from your brain momentarily to focus on your more fundamental modes of learning.
When you were a kid and you were learning your first language, you didn't learn all the rules of grammar first.
You didn't learn all the letters of the alphabet first.
You didn't learn about conjugating verbs first.
You learned a little bit about language.
You started to use it.
You ran into problems.
You learned a little more about language.
You learned to go from words like "feed me" to higher level concepts, like "Hey, what's for dinner?"
So the idea is that you learned it in an iterative process where you learned some stuff, tried it out, learned some more stuff, tried it out.
And it built up.
There's an enormous literature in education that says that's exactly how we always learn everything.
And so that's the way this course is focused.
What we will do is, for example, for today, we'll learn a little bit about software engineering.
Then, we'll do two lab sessions where you actually try to use the things we talk about.
Then, we'll come back to lecture and we'll have some more theory about how you would do programming.
And then, you go back to the lab and do some more stuff.
And the hope is that by this tangible context, you'll have a deeper appreciation of the ideas that we're trying to convey.
So let me tell you a little bit about the four modules that we've chosen.
The course is going to be organized on four modules.
Each module will take about one fourth of the course.
First thing we'll look at is software engineering.
As I said, we don't have time to focus on, or even survey, all of the big ideas in software engineering.
It's far too big.
So we're going to focus narrowly on one or two things.
We'd like you to know about abstraction and modularity because that's such an important idea in the construction of big systems.
So that's going to be our focus.
In today's lecture, we'll begin talking about modularity and abstraction at the small scale.
How does it affect the things you type as instructions to a computer?
But by next week, we're going to be talking about a whole bigger scale.
By next week, we're going to talk about constructing software modules at a much higher level.
In particular, we'll talk about something that we'll call a state machine.
A state machine is a thing that works in steps.
On every step, the state machine gets a new input.
Then, based on that input and its memory of what's come before, the state machine decides to do something.
It generates an output.
And then, the process repeats.
We will see that that kind of an abstraction -- state machines -- there's a way to think about state machines that is compositional that you can think of as a hierarchy, just as you can think of low-level hierarchies within a language.
I'll say a lot more about that today.
So the idea will be that once you've composed a state machine, you'll be able to join two state machines and have its behavior look just like one state machine.
That's a way to get a more complicated behavior by constructing two simpler behaviors.
That's what we want.
We want to learn tools that let us compose complex behaviors out of simple behaviors.
And the tangible model of that will be the robot.
We will see how to write a program that controls a robot as a state machine.
That's certainly not the only way you could control a robot.
And it's probably not the way you would first think of it if you took one course in programming and somebody said to you, go program the robot to do something.
What we will see is that it's a very powerful way to think about it for exactly this reason of modularity.
The bigger point that we will make in thinking about this first module is the idea of, how do you make systems modular?
How do you use abstraction to simplify the design task?
And in particular, we will focus on something that we'll call PCAP.
When you think about a system, we will always think about it in terms of, what are the primitives?
How do you combine them?
How do you abstract a bigger behavior from those smaller behaviors?
And what are the patterns that are important to capture?
So the bigger point is this idea of PCAP, which we will then revisit in every subsequent module.
OK, second module is on signals and systems.
That's also an enormous area.
So we only have time to do one thing.
The thing that we will do is we will think about discrete time feedback.
How do you make a system that's cognizant of what it's done so that it, in the future, can do things with awareness of how it got there?
A good example is robotic steering.
So the idea is going to be, OK, think about what you do when you're driving a car.
And think about how you would tell a robot to do that same thing.
Here's a naive driving algorithm.
I don't recommend it, but it's widely used in Boston, apparently.
[LAUGHTER] I find myself to the right of where I would like to be.
So what should I do?
Turn left.
I'm still to the right of where I'd like to be.
What should I do?
Turn left.
Oh!
I'm exactly where I should be.
What should I do?
Go straight ahead.
Oh, that's a bad idea.
And what we'll see is that perfectly innocent looking algorithms can have horrendous performance.
What we'll do is try to make an abstraction of that.
We'll try to make a model.
We'll try to capture that in math so that we don't need to build it to see the bad behavior.
We'll make a model.
We'll use the model to predict that that algorithm stinks.
But more importantly, we'll use the model to figure out an algorithm that'll work better.
In fact, we'll even be able to come up with bounds on how well such a controller could possibly work.
So the focus in this module is going to be, how do you make a model to predict behavior?
How do you analyze the model so that you can design a better system?
And then, how do you use the model and the analysis to make a well-behaved system?
The third module is on circuits.
Again, circuits is huge.
We don't have time to talk about all of circuits.
We'll do very simple things.
We'll focus our attention on how you would add a sensory capability to an already complicated system.
The idea is going to be to start with a robot-- I guess this is brighter-- start with our robots and design a head for the robot.
The robot comes from the factory with sonar sensors.
The sonar sensors are these things.
There's eight of them.
They tell you how far away something that reflects the ultrasonic wave is.
As they come from the factory, the robots can't sense light.
What you'll do is add light sensors.
The goal is to make a system to modify the robot so that the robot tracks light.
That's a very simple goal.
And the way we'll that is to augment the robot with a simple sensor here, showed a little more magnified here.
The idea is that this is a LEGO motor.
The LEGO motor will turn this relative to the attachment.
That's the robot head's neck.
So the robot will be able to do this.
And the robot will have eyes.
These are photosensors, photoresistors, actually.
So the idea is going to be that there's information available in those sensors for figuring out where light is so that you can track it.
Your job will be to build a circuit-- that that's this thing-- that connects via cables-- these red cables and yellow cables-- connects via cables over to this head.
We'll give you the head.
Your job will be to make the circuit that converts the signal from the photoresistor-- which is in proportion to light-- and figures out how to turn the motor to get the head to face the light and then ship that information down to the robot to let the robot turn its wheels to get the body.
So it's kind of like the light comes on bright over here.
The robot looks at it and says, oh, yeah, that's where I want to be.
So that's the idea in the third module is to incorporate new sensing capabilities into the robot.
The final module is on probability and planning.
And the idea there is to learn about how you make systems that are robust to uncertainty and that can implement complicated plans, that they, too, are robust to uncertainty.
So there's a number of things that we will do, including creating maps of spaces that the robot doesn't understand, telling the robot how to localize itself, how if it woke up suddenly in an environment, it could figure out where it is, how to make a plan.
And as an example, I'll show you the kind of system that we will construct.
Here, the idea is that we have a robot.
The robot knows where it is.
Imagine there's a GPS in it.
There isn't, but imagine there is.
So the robot knows where it is, and it knows where it wants to go.
That's the star.
But it has no idea what kind of obstacles are in the way.
So if you were a robotic driver in Boston, you know that you started out at home and you want to end up in MIT.
But there's these annoying obstacles, they're called people, that you should, in principle at least, miss.
So that's kind of the idea.
So I know where I am.
I'm the robot.
I know where I am.
I know where I want to be.
And I'm going to summarize that information here.
Where I am is purple.
Where I want to be is gold.
And I have a plan.
That's blue.
My plan's very simple.
I don't know anything about anything other than I'm in Waltham and I want to go to Cambridge.
So blast east.
So I imagine that the best way to do there is a straight line.
OK, so now what I'm going to do is turn on the robot.
The robot has now made one step.
And I told you before about these sonar sensors.
From the sonar sensors, the robot has learned now that there seems to be something reflecting at each of these black dots.
It got a reflection from the black dots, from the sonar sensors.
That means there's probably a wall there, or a person, or something that, in principle, I should avoid.
And the red dots represent, OK, the obstacle is so close I really can't get there.
So I'm excluded from the red spots because I'm too big.
The black spots seem to be an obstacle.
The red spots seem to be where I can't fit.
I still want to go from the where I am, purple, to where I want to be, gold.
So what I do is I compute the new plan.
OK then, I start to take a step along that plan.
And as I'm stepping along, OK, so now, I think that I can't go from where I started over to here.
I have to go around this wall that I didn't know about initially.
So now I just start driving.
And it looks fine, right?
I'm getting there, right?
Now, I know I can go straight down here.
Oh, wait a minute.
There's another wall.
OK, what do I do now?
So as the robot goes along, it didn't know when it started what kinds of obstacles it would encounter.
But as it's driving, it learned.
Oh, that didn't work.
Start over!
So the idea is that this robot is executing a very complicated plan.
The plan has, in fact, many sub-plans.
And the sub-plans all involve uncertainty.
It didn't know where the walls were when it started.
And when it's all done, it's going to have figured out where the walls were and-- provided there's a way-- presumably find the way to negotiate the maze and get to the destination.
So the idea, then, is that if you were asked to write a conventional kind of program for solving that, it might be kind of hard because of the number of contingencies involved.
What we will do is break down the problem and figure out simple and elegant ways to deal not only with uncertainty, but how do you make complex plans.
So as I said, our primary pedagogy is going to be practice, theory, practice.
And so that ramifies in how the course is organized.
So this is a quick map of some of the aspects of the course.
So we'll have weekly lectures.
It's lecture unintensive.
In total, there's only 13 lectures.
We'll meet once a week here for lecture.
There's readings.
There's voluminous readings.
There's readings about every topic that we will talk about.
And the readings were specifically designed for this course.
I highly recommend that you become familiar with the readings.
If you have a question after lecture, it's probably there.
It's probably explained.
We will do online tutor problems.
We sent you an email if you pre-registered for the course.
So you may already know about this.
The idea is going to be that there's ways that you can prepare for the course by doing computer exercises.
And we will also use those same kinds of exercises in all of the class sessions.
We will have two kinds of lab experiences.
Besides lecture, the other two events that you have to attend are a software lab and a design lab.
That's the practice part.
So after you learned a little bit about the theory by going to lecture, by doing the reading, then you go to the lab and try some things out.
We call the first lab a software lab.
It's a short lab.
It's an hour and a half.
You work individually.
You try things out.
You write little programs.
The courseware can check the program to see if it's OK. And primarily, the exercises in the software lab are due during the software lab.
But on occasion, there will be extra things due a day or two later.
The due dates are very clearly written in the tutor exercises.
Once a week, there's a design lab.
That's a three hour session in which you work with a partner.
The reason for the partner is that the intent-- the difference between the design labs and the software labs is that the design labs ask you to solve slightly more open-ended questions, the kind of question that you might have no clue what we're asking.
Open-ended, the kind of thing that you will be asked to do after you graduate.
Design the system.
What do you mean, design the system?
So the idea is that working with a partner will give you a second, immediate source of help and a little more confidence if neither of you knows the solution so that you raise your hand and say, I don't have a clue what's going on here.
So the idea is that once a week we do a software lab individually.
Once a week, we do a design lab, a little more open-ended with partners.
There's a little bit of written homework, four total.
It's not much compared to other subjects.
It's mostly practice.
There's a nano-quiz, just to help you keep pace to make sure that you don't get too far behind.
The first 15 minutes of every design lab starts with a nano-quiz.
The nano-quizzes are intended to be simple if you've caught up, if you're up to date.
So the idea is that you go to design lab, the first thing you do is a little, 15 minute nano-quiz.
The nano-quiz uses a tutor much like the homework tutor, much like the Python tutor.
And it's intended to be simple.
But it does mean please get to the design lab on time.
The nano-quizzes are administered by the software.
It starts the hour when the design lab starts.
It times out 15 minutes later.
So if you come 10 minutes late, you will have 5 minutes to do something that we planned to give you 15 minutes for.
We will also have exams and interviews.
The interviews are intended to be a one-on-one conversation about how the labs went.
And we will have two mid-terms and a final.
So that's kind of the logistics.
The idea behind the logistics is practice, theory, practice.
Come to the labs.
Try things out.
Make sure you understand.
Develop a little code.
Type it in.
See if it works.
If it works, you're on top of things.
You're ready to get the next batch of information from the lecture and readings.
OK, let's go on, and let's talk about the technical material in the first module of the course, in the software module.
We kick the course off talking about software engineering for two reasons.
We'd like you to know about software engineering.
It's an incredibly important part of our department.
It's an incredibly important part of the engineering of absolutely any system, any modern system.
But we'd also like you to know about it because it provides a very convenient way to think about-- it's a convenient language to think about the design issues, the engineering issues in all the other parts of the class.
So it's a very good place to start.
So what I will do today is talk about some of the very simplest ideas about abstraction and modularity in what I think of as the lowest level of granularity.
How do you think about abstraction and modularity at the micro scale, at the individual lines of code scale?
As I said earlier, we will, as we progress, look at modularity and abstraction at the higher scale.
But we have to start somewhere.
And we're going to start by thinking about, how do you think about abstraction and modularity at the micro scale?
Special note about programming.
So what we are trying to do is, in the first two weeks, ramp everybody up to some level of software security, where you feel comfortable.
So the first two weeks of this course is intended to make you comfortable with programming.
We don't assume you've done extensive programming before.
We want you to become comfortable that you're not behind.
And that's the focus of the first two weeks' exercises.
If you have little or no previous background, if you are uncomfortable, please do the Python tutor exercises.
If you have not -- if you do not have a lot of experience programming, if you're uncomfortable with the expectation that you can do programming, do that first.
That takes priority over all the other assignments during the first two weeks.
In particular, if you're uncomfortable, we will run a special Python help session on Sunday.
And if you attend that, you can get a free extension.
The idea is completing the tutor exercises is intended to make you feel comfortable that you have the software background to finish the rest of the course.
Do that first.
We will forgive falling behind in other things so that you feel comfortable with programming.
If, at the end of two weeks, you still feel uncomfortable, we have a deal with 6.00, the Python programming class, that they will allow you to switch your registration from 6.01 to 6.00.
But that expires Valentine's Day.
[LAUGHTER] So you have to make up your mind before Valentine's Day if you'd like to use that option.
So the idea is we'd like you to be comfortable with programming.
If you haven't programmed before, do the Python tutor exercises.
Go to software lab.
Go to design lab, but work on the tutor exercises.
The staff will help you with them.
You can go to office hours.
There's office hours listed on the home page.
You should try to become comfortable, and you should try to set as your goal -- I'm going to be comfortable before Valentine's Day.
And if you're not, talk to a staff member about that.
OK, so what do I want you to know about programming?
Well, we're going to use Python.
We selected Python because it's very simple and because it lets us illustrate some very important ideas in software engineering in a very simple context.
That's the reason.
One of the reasons that it's simple is that it's an interpreter.
After some initialization, the behavior of Python is to fall into an interpreter loop.
The interpreter loop is, ask the user what he would like me to do, read what the user types, figure out what they're talking about, and print the result, repeat-- very simple.
What that means is that you can learn by doing.
That's one of the points of today's software lab.
You can simply walk up to a computer, type the word python-- what you type is in red.
Type the word "python."
It will prompt you, so this chevron, that says, I'd like you to tell me something to do.
I have nothing to do.
If you type "2," Python tries to interpret that.
In this particular case, Python says, oh, I see.
That's a primitive data item.
That's an integer.
This person wants me to understand an integer.
And so it will echo 2, indicating that it thinks you want it to understand a simple integer.
Similarly, if you type 5.7, it says, oh, I got that.
That's a float.
The person wants me to remember a floating point number.
And it will similarly echo the float.
Now, of course, there's no exact representation for floats, right?
There's too many of them, right?
There's a lot of them.
There's even more floats than there are ints, right?
So it has an approximation.
So it will print its approximation to the float that it thinks you are interested in.
If you type a string, "Hello," it'll say, oh, primitive data structure, string.
And it'll print out that string.
So the idea is one of the features of Python that makes it easy to learn is the fact that it's interpreter based.
You can play around.
You can learn by doing.
Now, of course, if the only thing it did was simple data structures, it would not be very useful.
So the next more complex thing that it can do is think about combinations.
If you type "2 + 3," it says, oh, I got it.
This person's interested in a combination.
I should combine by the plus operator two ints, 2 and 3.
Oh, and if I do that, if I combine by the plus operator two and three, I'll get 5.
So it prints 5.
So that's a way you know that it interprets "2 + 3" as 5.
Similarly here, except I've mixed types.
"5.7 + 3," it says, oh, this user wants me to apply the plus operator on a float and an int.
OK, well I'll upgrade the int to a float.
I'll do the float version, and I'll get this, which is its representation of 8.7.
So the idea is that it will first try to interpret what you're saying as a simple data type.
If that works, it prints the result to tell you what it thinks is going on.
It then will try to interpret it as an expression.
And sometimes, the expressions won't makes sense.
In particular, if you try to add an int to a string, it's going to say, huh?
And over the course of the first two weeks, we hope that you get familiar with interpreting this kind of mess.
That's Python's attempt to tell you what it was trying to do on your behalf and can't figure out what you're talking about.
OK, so that was simple.
But it already illustrates something that's very important, and that's the idea of a composition.
So the way Python works, the fact that when you added 3 to 2 it came out 5, what we were doing was composing complicated-- well, potentially complicated (that was pretty simple) -- potentially complicated expressions and reducing them to a single data structure.
And so that means that, in some sense, this operation, 3 times 8, can be thought of as exactly the same as if the user had typed in 24.
Whenever you can substitute for a complex expression a simpler thing, we say that the system is compositional.
That's a very powerful idea.
Even though it's simple, it's a very powerful idea.
And it's an idea that you all know.
You've seen it before in algebra, in arithmetic.
So in arithmetic expressions, you can think about how the sum of two integers is an int.
That's a closure.
That's a kind of a combination that makes the system compositional and that provides a layer of hierarchical thinking so that, in your head, even though it says 3 times 8, you don't need to remember that anymore.
You can say, oh, for any purposes that follow, I might just as well think of 3 times 8 as being a single integer, 24.
It's part of many other kinds of systems, for example, natural language.
The simplest example in natural language is that you can think about "Apples are good as snacks".
"Apples" is a noun.
It's a plural noun.
Or you could substitute "Apples and oranges", and it makes complete sense within that same structure.
So "Apples and oranges are good as snacks".
The combination of "apples" and "oranges" works in every way from the point of view of the grammar in the same way that a simple noun, "apples," worked.
What we would like to do is use that idea as the starting point for a more general compositional system.
And a good way to think about that is by way of names.
What if we had some sequence of operations that we think is particularly important so that we would like to somehow canonize that so that, subsequently, we can use that sequence of operations easily?
Python provides a very simple way to do it.
Every programming language does.
It's not unique to Python.
But the idea is -- so here's an example.
"2 times 2" -- I'm squaring 2 and get 4.
"3 times 3" -- I'm squaring 3, and I'm getting 9.
"8 plus 4 times 8 plus 4", I'm squaring "8 plus 4".
"8 plus 4", well, I can think of that as 12.
I'm squaring 12, I'm getting 144.
The thing I'm trying to illustrate there is the notion of squaring.
Squaring is a sequence of operations that I would like to be able to canonize as a single entity so that, in subsequent programs, I can think of the squaring operation as a single operation just like I think of times.
The way we say that in Python is "define square of x to be return x squared".
Then, having made that definition, I can say "square of 6", and the answer is 36.
OK, this is a very small step.
But it illustrates a very important point, the idea being that Python provides a compositional facility.
And it's hierarchical.
Having defined square, I can use square just as though it were a primitive operator.
And I can use square to define higher level operations.
So for example, what if I were interested in doing lots of sums of squares?
Say I'm Pythagoreas, right?
So I might want to add the square of 2 and the square of 4 to get 20, or the square of 3 with the square of 4 to get 25.
Using that simple idea of composition, we can write a new program, sumOfSquares.
sumOfSquares takes two arguments, x and y.
And it returns the square of x and the square of y. SumOfSquares doesn't care about how you compute the square.
It trusts that square knows how to do that.
So the work is smaller.
The idea is that square takes care of squaring single numbers.
sumOfSquares doesn't have to know how to square numbers.
It just needs to know how to make a sum of squares.
So what we've done is we've broken a task, which was not very complicated, but the whole idea is hierarchical.
We've taken a problem and broken it into two pieces.
We factored the problem into how do you do a square, and how do you sum squares.
And the idea, then, is that this hierarchical structure is a way of building complex systems out of simpler parts.
So that's the idea of how you would build programs that are compositional.
Python also provides a utility for making lists, for making data structures that are compositional.
The most primitive is a list.
So in Python, you can specify a list.
Here's a list of integers.
So the list says, beginning list, end of list, elements of list.
So there's five elements in the list, the integers 1, 2, 3, 4, 5.
Python doesn't care what the elements of a list are.
We'll see in a minute that that's really important.
But for the time being, the simplest thing that you can imagine is a heterogeneous list.
It's not critical that the list contain just integers.
Here's a list that contains an int, a string, an int, and a string.
Python doesn't care.
It's a list that has four elements.
The first element's an int.
The second element's a string, et cetera.
Here's an even more complex example.
Here's a list of lists.
How many elements are in that list?
Three.
How many elements are in that list?
So the idea is that you can build more complex data structures out of simple ones.
That's the idea of compositional factoring applied to data.
Just like it was important when we were thinking about procedures, to associate names with procedures-- that's what "def" did-- we can also think about associating names with data structures.
And that's what we use something that Python calls a variable for.
So I can say "b is 3".
And that associates the data item, 3, with the label, b. I can say, "x is 5 times 2.2".
Python will figure out what I mean by the expression on the right.
It'll figure out that I'm composing by using the star operator, which is multiply, an integer and a float, which will give me a float.
The answer to that's going to be a floating point number.
And it will assign a label, x, to that floating point number.
You can have a more complicated list, a data structure, and associate the name y with it.
Then, having associated the name y, you get many of the same benefits of associating a name with a data structure that we got previously in associating a name with an operation.
So we can say, y(0).
And what that means is, what's the zero-th elements of the data structure, y?
So the zero-th element of the data structure, y, is a list, [1, 2, 3].
Python has some funky notations.
The -1 element is the last one.
So the -1th element of y is [7, 8, 9].
And it's completely hierarchical.
If I asked for the -1 element of y, I get [7, 8, 9].
But then, if I asked for the first element of that result, I get 8.
OK?
Everything is clear?
OK, just to make sure everything is clear, I want to ask you a question.
But to kick off the idea of working together, I'd like you to think about this question with your neighbor.
So before thinking about this question, everybody stand up.
Introduce yourself to your neighbor.
[AUDIENCE TALKS] So now, I'd like you to each discuss with your neighbor the list that is best represented by which of the following figures, 1, 2, 3, 4, or 5, or none of the above.
And in 30 seconds, I'm going to ask everybody to raise a hand with a number of fingers indicating the right answer.
You're allowed to talk.
That's the whole point of having a partner.
[AUDIENCE TALKS] OK.
I'd like everybody now to raise their hand.
Put up the number of fingers that show the answer.
And I want to tally.
Fantastic!
Everybody gets it.
OK, so which one do you like?
3
3 -- why do you like three.
Somebody explain this to me?
It just looks good?
Its pattern recognition.
What's good about 3?
It shows the compositional element of the list
Compositional?
What is the compositional element in the pictures?
What represents what?
OK, 'a' represents a.
That's pretty easy, right?
So that takes care of the bulk of the figures.
What's the blue lines represent?
Someone else?
I didn't quite understand
The angles represent like a list
They represent a list.
Where is the list on the figures?
The vertex?
The vertex.
The vertices are lists.
So in 3 -- at the highest level, we have a list that's composed of how many elements?
2.
The first element of that list is?
a
And the second element of that list is?
Another list
Another list.
That's the hierarchical part, right?
That second list has how many elements?
2
Fine, good, recurse.
You got it.
What is the list represented by number 2?
A single list with five elements.
Square bracket, a, comma, b, comma, c, comma, d, comma, e, square bracket, right?
What is the list represented by that one?
Not a list
Agh!
It's not a list!
What is it?
Who knows?
Looking at the variable names, it defines them.
You have variables.
You have a variable a, that defines a list that contains b, and the variable, c, that defines another list that contains d
So we could make that a variable.
If we said a is a variable that comprises b and c, then we have the problem of how we're going to associate variables and elements into this list, right?
So the weird thing about this one and, let's see, that one's weird.
This one's also kind of weird.
This one's weird because we're giving names to lists in a fashion that's not showed up here, right?
That's not to say you couldn't invent a meaning.
It's just that it doesn't map very well to that representation.
Similarly over here, we seem to be giving the name b to the element a, and then the name c to the element b.
What on earth are you talking about?
It's not clear what we're doing their either.
So the point is to get you thinking about the abstract representation of lists and how that maps into a complex data structure.
That was the whole point.
OK, so we've talked about, then, four things so far.
How do you think about operations in a hierarchical fashion.
And the idea was composition.
We think about composing simple operations to make bigger, compound operations.
That's a way of saying, there's this set of operations that I want to call foo.
So every time I do this complicated thing that has three pages of code, that's one foo.
And that's a way that we can then combined foos in some other horribly complicated way to make big foos.
So the idea is composition.
That's the first idea.
The second is associating a name with that composition.
That's what "def" does-- define name, name of a sub-routine.
So we thought about composing operations, associating names with them.
We composed data in terms of lists, and we associated names with those lists in terms of variables.
The next thing we want to think about is a higher order construct where we would like to conglomerate into one data structure both data and procedures.
Python has a concept called a class that lets us do that.
In Python, you make a new class by saying to the Python prompt, I want a new class called Student.
And then, under Student, there is this thing which we will call an attribute.
An attribute to a class is simply a data item associated with the class.
And a method-- a method is just a procedure that is associated with the class.
So there's this single item class called Student that has one piece of data, its attribute, school, and one procedure, which is the method calculateFinalGrade.
So then, this is the kind of data structure you might imagine that a registrar would have.
It's a way to associate.
So the idea here is that everybody here is a student.
They all have a school.
And they all have a way of calculating their final grade.
That's a very narrow view that maybe a registrar would have.
So classes, having defined them, we can then use the class to define an instance.
So an instance is a data structure that inherits all of the structure from the class but also provides a mechanism for having specific data associated with the instance.
So in Python, I say Mary is a student.
By mentioning the name of the class and putting parenthesis on it, I say, give me an instance of the student.
So now, Mary is a name associated with an instance of the class, Student.
John is similarly an instance of the class, Student.
So both Mary and John have schools.
In fact, they're both the same.
The school of Mary and the school of John are both MIT.
But I can extend the instance of Mary to include a new attribute, the section number, so that Mary's section number is 3 and John's section number is 4.
So this provides a way-- it's a higher-order concept.
We thought of a way to aggregate operations into complicated operation, data into complicated data.
Classes aggregate data and operations.
Classes allow us to create a structure and then generate instances.
And then the instances have access to those features that were defined in the class, but also have the ability to define their own unique attributes and methods.
You can also use a class to define a subclass.
So here, I'm defining the subclass, Student601.
All Student601s are members of the class, Student.
The reverse is not true.
So all Student601 entities inherit everything that a Student has.
But all 601 students share some other things.
Besides having a school which all students have, 601 students also have a lecture day, a lecture time, and a method for calculating tutor scores.
Not all students have a method for calculating tutor scores.
But members of the class Student601 do.
So this, again, represents a way of organizing data and operations in a way that makes it easier to compose higher, bigger, more complex structures.
The final thing that I want to talk about today is the specific, gory details for how Python manages the association between names and entities.
We've already seen two of those.
Naming operations is via "def."
And it gives rise to the name of a procedure.
Variables are ways of naming data structures.
Now, we've seen a way of naming classes.
And in fact, it's helpful if you understand.
So Python associates names and entities in a very simple, straightforward fashion.
And if you know the ground rules, it makes it very easy to deal with.
And if you don't know the ground rules, it makes it very hard to deal with.
So what's the ground rules?
Here's the gory details.
So Python associates names with values in what Python calls a binding environment.
An environment is just a list that associates a name and an entity.
So if you were to type b equals 3 what Python is actually doing is it's building this environment.
When you type b equals 3, it adds to the environment a name, b, and associates that name with the integer, 3.
When you type x equals 2.2, it adds a name, x, and associates it with the float, 2.2.
When you say foo is minus 506 times 2, it makes the name, foo, and associates it with an int, minus 1012.
Then, if you ask Python about b, the rule is look it up in the environment and type the thing that b refers to.
So when you type "b," what Python really does is it goes to the environment.
It says, do I have some entity called "b?"
Well, yes I do.
It happens to be an int, 3.
So it prints 3.
If you ask, what is "a?"
Python says, OK, in my environment, do I have some name, "a?"
It doesn't find it.
So it prints out this cryptic message that basically says, sorry, guys, I can't find something called "a" in the current environment.
That's the key to the way Python does all name bindings.
So in general, there's a global environment.
You start typing to Python.
It just starts adding and modifying the bindings in the binding environment.
So if you type a equals 3 and then type "a," it'll find 3.
If you then type "b=a+2," it evaluates the right-hand side relative to the current environment.
So it first looks here.
And it says, do I have something called "a?"
Ah, yes.
It's an integer, 3.
Substitute that.
Do I know what 2 is?
Oh yeah, that's just an int.
Do I know what plus is?
Oh yeah, that's the thing that combines two ints.
So it decides that a plus 2-- it evaluates a plus 2 in the current environment.
It gets 5.
And it says, oh, I'm trying to do a new equals, a new association, a new variable.
Make the name, b, points to this evaluated in the current environment.
So b gets associated with int 5.
Then, if I do this line, it evaluates b plus 1 in the current environment.
b is 5 in the current environment.
It adds 1.
It gets 6.
And then, it says, associate this thing, 6, with b.
So it overwrites the b, which had been bound to 5, and b is now bound to 6.
OK?
So the whole thing, the way it treats variables, the way Python associates a name with a value in a variable, is evaluate the right-hand side according to the current environment.
Then, change the current environment to reflect the new binding.
What it does in the case of sub-routines is very similar.
So here's an illustration of the local environment that is generated by this piece of code.
When I say a equals 2, it generates a name in the local environment, a.
It evaluates the right-hand side and finds 2.
So it makes a binding in the local environment where the name, a, is associated with the integer, 2.
Then, I say define square of x to be return x squared.
That's more complicated.
Python says, oh, I'm defining a new operation.
It's a procedure.
The procedure has a formal argument, x.
It has a body, return x times x. I'm going to have to remember all of that stuff.
So I'm trying to define a new procedure called square.
It's going to make a binding for square.
So in the future, if somebody says the word square, it'll find out, oh, square I remember that one.
square, it's a procedure.
Just like the binding for a variable might be an int, the binding for a procedure is the name of the procedure.
Then, in the procedure, which is some other data structure outside the environment, it's got to remember the formal parameters-- in this case, x-- and the body.
And for the purpose of resolving what do the variables mean, it needs to remember what was the binding environment in which this sub-routine was defined.
So that's this arrow.
So this sequence says, make a new binding square, points to a procedure.
The procedure has the formal argument, x.
It has the body return x times x.
And it has the binding.
It came from the environment, E1, the current environment.
OK, is everybody clear?
So the idea is that the environment associates names with things.
The thing could be a data item, or it could be a procedure.
Then, when you call a procedure, it makes a new environment.
So what happens, then, when I try to evaluate a form, square of a plus 2?
What Python does is it says, OK, I need to figure out what square is.
So it looks it up in the environment, and it finds out that square is a procedure.
Fine, I know how to deal with procedures.
So then, it figures out this procedure has a formal argument, x. Oh, OK, if I'm going to run this procedure, I'm going to have to know what x means.
So Python makes a new environment-- here, it's labelled E2, separate from the global environment, E1.
It makes a new environment that will associate x with something.
Doesn't know what it is yet, it just knows that this square is a procedure that takes a formal argument, x.
So Python makes a new environment, E2, with x pointing to something.
Then, Python evaluates the argument a plus 2 in the environment E1.
You called square of a plus 2 in the environment of E1.
So it figures out what did you mean by a plus 3.
Well, you were in the environment E1.
So it means whatever a plus 3 would have meant if he had just typed a plus 3 in that environment.
So you evaluate a plus 3 in the environment E1, and you get 5.
So then, this new environment, E2, that is set up for this procedure, square, associates 5 with x.
Now it's ready to run the body.
So now, it runs this procedure, return x times x.
But now, what it's trying to resolve its variables, it looks it up in E2.
So it says, I want to do the procedure, the body, x times x. I need to know what x is, and I need to know it twice.
Look up what x means, but I will look it up in my E2 environment that was built specifically for this procedure.
And fortunately, there's an x there.
So it finds out that x is 5.
It multiplies 5 times 5.
It gets the answer is 25.
It returns 25.
And then, it destroys this environment, E2, because it was only necessary for the time when it was running the procedure body.
Is that clear?
OK, so a slightly more difficult example illustrates what happens whenever everything is not defined in the current local environment.
What if I type define biz of a?
Well, I create a new name in the local environment that points to a procedure.
The procedure has a formal parameter, a, and a body that returns a plus b.
The procedure also was defined within the environment E1, which I'll keep track of.
Then, if I say b equals 6, that makes a new binding in the global environment, b equals 6.
Then, if I try to run biz of 2, look up biz.
Oh, that's a procedure, formal parameter, a.
Make an environment, has an a in it.
What should I put in a?
Evaluate the argument, 2.
OK, a is 2.
Put two here.
Now, I'm ready to run the body.
Run the body in the environment, E2.
When I run return a plus b in E2, I have to first figure out a.
Well, that's easy.
a is 2.
Then, I have to figure out b.
What's b?
6?
6.
So how did you get 6?
[INAUDIBLE]
So this local environment that was created for the formal parameter has, as its parent, E1 because that's where the procedure was defined.
So it doesn't find b in this local environment.
So it goes to the parent.
Do you have a "b?"
And it could, in principal, propagate up a chain of environments.
So you could construct this hierarchically.
So it will resolve bindings in the most recent environment that has that binding.
So the answer, then, is that when you run biz of 2, this b gets associated with that b, OK?
So that's how the environments work for simple procedures and simple data structures.
It's very similar for the way it works with classes.
So imagine that I had this data, and I wanted to represent that in Python.
What I might do is look at the common features.
The courses are all the same.
The rooms are all the same.
The buildings are all the same.
The ages are highly variable.
So I might want to create a class that has the common data.
So I might do this-- class Staff601.
The course is 601.
The building's 34.
The room is this.
The way Python implements a class is as an environment.
Executing this set of statements builds the class environment.
This is it.
It's a list of bindings.
Here, I'm binding the name, course, to the string, 601, et cetera.
If there were a method, I would do the same thing, except it would look like a procedure then.
So this creates the Staff601 environment.
Staff601, because I executed this class statement, that creates a binding in the local environment, Staff601, which points to the new environment.
So now, in the future, when Python encounters the name Staff601, it will discover that that's an environment.
Python implements classes as environments.
So now, when I want to access elements within a class, I use a special notation.
It's a dot notation.
Python regards dots as ways of navigating an environment.
When Python parses staff.room, it looks up Staff601 in the current environment.
If it finds an environment, it then says, oh, I know about this .room thing.
All I do is I look up the room name in the environment Staff601.
And when it does that, it gets the answer 501.
And the same sort of thing happens here.
It looks up Staff601.
It finds an environment.
It looks up coolness.
It finds out there is no such thing.
Well, no, that's not true.
So it creates coolness within 601 and assigns an integer, 11, to it.
So then, the way Python treats methods is completely analogous-- oh, excuse me, instances.
I'm doing instances first.
If I make pat be an instance of Staff601, pat is an instance of the class Staff601.
pat is implemented as an environment.
So when I make pat, pat points to a new environment-- here, E3.
The parent of E3 is the class that pat belongs to, which is, here, E2.
And when I make the instance, it's empty.
But now, if I ask what is pat.course, well, pat points to this environment.
Does this environment have something called a course?
No.
Does the parent?
Yes.
Course is bound to the string 601.
So pat.course is 601 just the same as Staff601.course had been 601. pat is an instance.
It's a new environment with the class environment as its parent.
You can add attributes to instances.
And all that does is populate the environment associated with the instance.
You can add methods to classes.
And that does the same thing.
So here, I've got the class, Staff601, which has a method, salutation, instance variables, course, building, and room.
So when I build that structure, Staff601 points to an environment that contains salutation, which is a procedure, in addition to a bunch of instance variables.
So now, all of the rules that we've talked about with regard to environments apply now to this class.
So in particular, I can say Staff601 salutation of pat.
When Python parses Staff601, it finds an environment.
It says dot salutation.
Oh, I know how to do that.
Within the environment, Staff601, look for a binding for the name salutation.
Do I find one?
Well, yeah, there it is.
It points to a procedure.
So staff dot salutation is a procedure.
Do just the same things that we would have done with a simple procedure.
The only difference here is that the procedure came from a class.
In this particular case, the sub-routine that I define has a formal parameter, self.
So then, that's going to have to build when I try to evaluate it.
That has to build a binding for self, which is set to pat.
pat was an environment.
So self gets pointed to pat.
So now, when I run Staff601.salutation on pat, it behaves as though that generic method was applied to the instance pat.
We'll do that a lot.
It's a little bit of redundancy.
We know that pat is a member of Staff601.
So we will define a special form-- or I should say, Python defines a special form-- that makes that easy to say.
This is the way we will usually say, the instance pat should run the class method salutation on itself.
This is simply a simplified notation that means precisely that, OK?
So what we covered today, then, was supposed to be the most elementary ideas in how you construct modular programs, Modularity at the small scale.
How do you make operations that are hierarchical, data structures, and classes?
What we will do for the rest of the week is practice those activities.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello.
Welcome to the second lecture in 6.01.
I hope you had a good time last week.
Last lecture, we looked at what is probably the most important theme in this course, which is, how do you organize your thoughts, your design, the things that you do, in order to manage complexity when you're trying to build a complicated system?
The mantra for this class is PCAP-- primitives, combinations, abstractions, and patterns.
And last time, we saw how at the very most elementary level, Python provides some tools with which we can achieve this goal of defining primitives, combining them into more complicated ideas, abstracting the important information, and generating and capturing new patterns.
And so, for example, we saw that Python has a defined statement that lets you associate a sequence of operations with a name-- both of those things are important, the sequence represents a combination, the name represents a way that we can abstract the behavior of the combination and treat it as though it were primitive operation.
We saw that we could do the same sort of thing for data structures.
And in particular, the list structure in Python allows us to generate hierarchical heterogeneous structures in much the same way.
Then variables allow us to associate names with those structures.
And finally, we saw that classes allow us to combine not only data, but also procedures -- all into one object of related things.
So that's PCAP at the most primitive level.
What I want to do today is talk about PCAP at higher levels.
How do you build upon that framework to continue this idea of building with abstraction and modularity?
How do you make-- how do you combine primitive operations into powerful operations?
How do you combine primitive data into powerful data?
And what I want to do is think about the next level above the most rudimentary level in Python.
So we'll look first at some programming styles, and how that affects your ability to define useful abstractions.
And then I'll look at something much higher level, which is state machines, which is the way that we will think about the construction of robot controls.
So I'll start with just a few words about how you-- the different ways that you could structure a program.
The reason for doing this is that the basic structure that you use can have an important effect on your ability to abstract.
We'll look at three different methodologies for constructing a program.
I'll refer to the first one as imperative, also procedural.
That's kind of the most basic way you could think about writing a program.
It's kind of a recipe.
It's kind of like cooking with a recipe.
Take this, add this, stir this, bake for 30 minutes, that kind of thing.
So if you define a procedure, if you organize the way you think about it in terms of step by step instructions for, what should I do next?
We refer to that kind of an approach as imperative.
We'll look at functional programming.
There, even though you implement precisely the same algorithm, the perspective is a little bit different.
There, rather than focusing quite so narrowly on the step by step, how do you get from A to B, the idea is going to be, think about the structure of the problem in terms of functions in a mathematical sense.
By which I mean, functions that eat inputs, generate outputs, and don't have side effects.
Side effects are things like setting variables that you can later look at.
Then I'll look at object-oriented programming.
Again, you could be implementing precisely the same algorithm by using an object-oriented approach, but here the focus will be on building collections of data with procedures that are related, and organizing the solution to your problem in terms of a hierarchy of such structures.
So what I'd like to start off today with is to look at an example problem and how you could program it using any of these three approaches.
So the example program is going to be, find a sequence of operations-- by which I mean, an operation is either increment or square-- the idea is that the operations are things that we will do to integers.
Find a sequence of operations, either increment or square, that transforms one integer, which is the initial value, i, into a goal, which I'll call, g. So I want to think about the problem of finding such sequences.
So for example, the sequence increment increment increment square, when applied to 1, would give 16.
So I'm thinking about the first increment increments 1 to give you 2.
The second increment increments 2 to give you 3.
The third increment increments 3 to give you 4.
Then square squares 4 to give you 16.
So I'll refer to this as having found the sequence of operations, increment increment increment square, that transforms 1 into 16.
Everybody with me?
OK, I'll be the judge of that.
So, to prove that you're with me, what's the minimum length sequence of increment and square operations needed to transform 1 into 100.
You've got thirty seconds to discuss it with your neighbor, come to an agreement, and I'm gonna ask you to raise your hands with a number of fingers, (1), (2), (3), (4), or (5), indicating the right answer.
[AUDIENCE DISCUSSION]
OK, everybody raise your hand.
Tell me a number of fingers equal to the right answer, raise your hand, show me a number of fingers.
OK, I'm seeing-- OK.
So keep in mind the number of fingers is the thing before the colon.
That avoids the awkward way of saying less than 4.
OK?
So I want the number before the colon.
So what is the minimum length sequence?
Raise your hand, indicate a number of fingers.
OK, the answers are improving.
Higher so I can see you.
OK, it's about 90% correct, I think.
OK, most people said (3), which is another name for 5.
OK.
So how do you get the answer 5?
What's the answer?
Somebody explain that answer?
Yeah
Increment increment square increment square?
That's exactly right.
So since I have two operators, increments and squares, and since I'm trying to cover a big distance, 1 to 100, square increases faster-- at least for bigger numbers, than increment does.
So what you'd like to do is figure out a way of coercing as many squares as possible.
So a good thing to do is to start at the end, and you can take the square root of the first one evenly, and that gives you 10.
But then you can't take the square root of 10, so you back off and you get 9, and then you can take the square root of 9, et cetera.
So there's any number of ways you could solve this problem, the point is, that there's a simple solution which is, the answer (3), which is a pseudonym for 5.
So 5 operations will get you from 1 to 100.
So what I want to do now-- now that you know the answer to the question, I want to write a program to find that answer.
The most straightforward approach that you could use is what we would call, imperative or procedural.
The idea is to solve the problem by walking your way through it in some premeditated fashion.
A reasonable premeditated fashion would be, think about the sequences and order them by length.
Think about all the sequences of length one, see if they work.
Think about all the sequences of length two, see if they work.
Three, see if they work.
And just keep going until you find one that works.
That's a very reasonable-- that's a procedure.
Start by thinking about short sequences, and proceed by making them longer and longer until you run into one that happens to solve your problem.
So that's what's going on here.
First I gave a name to the operator, increment.
Then I give a name to the operator, square-- that's just for convenience.
Then what I want to do is write a program-- find sequence, that will start at some initial value, say 1, and go to some goal, say 100.
And the way I'll do that is to enumerate over the lengths of sequences that are possible -- 1, 2, 3, et cetera.
I'll represent each one of those sequences of operations by this kind of a representation, I'll make a tuple that has a string that tells me in English, what did I do?
And an integer that tells me what the answer is after I've done that.
So looking ahead, this is the idea-- this is the program that I'm trying to construct.
I would like the output of the program to start by thinking about all sequences of length one, then sequences of length two, then sequences of length three, then sequences of length four.
For each sequence of length one, I'd like to think about, what are all the possible sequences?
Well, I could start with 1 and increment it to get 2.
Or I could start with 1 and I could square it to 1.
That's all the possible sequences of length one.
Then I go on to length two, three, four, by the time I'm down here to length four, I could start with 1, increment square square increment, and that would give me 17.
That's the structure of the program that I'm trying to make.
Everybody with it?
So I'm going to define find sequence, I'm going to loop over all those lengths, I'm going to keep track of all the different sequences I found as a list of tuples.
After last week you're supposed to be very comfortable with those kinds of words.
Each one of the sequences is a tuple, a string and a final value, and the list is all possibilities.
And I'm going to try the ones of length one, then I'm going to append to each one to make sequences of two, and then I gonna append to that to make sequences of three, four, five, and keep on going.
So the point is, that this is a very simple-minded, easy to conceive recipe for how to search through a large number of sequences and find the one with the minimum length.
So when you write that program, it iterates down until it finds number 5, the 5-length sequence here it came up with the answer 100, so that's the answer.
The point is, that it was an easy way to write the program.
We just think about telling somebody with a pencil and paper, what would you do?
And we tell Python, rather than the person with the piece of paper, what to do.
The approach is straightforward.
The only ugliness is that it ended up with three levels of loops.
And the most common kind of error in this kind of programming is you just botch the indices.
Because it's got three nested layers of loops, it's very easy to lose track -- when you're thinking of layer three -- about what's happening in layer two.
So that's the only difficulty in this approach.
The challenge is just to keep all the indices consistent.
But it works.
There's nothing wrong with this approach.
It doesn't necessarily lead to the most modular designs, but you can write functional programs that work this way.
A different way to structure the program, I'll refer it-- here's a different version of the same program, the same algorithm, but I'm going to refer to this one as functional program.
Here the idea is to focus on procedures that implement functions of a mathematical type.
I want to recast the problem, this time thinking about, how would I divide it up into functions that calculate things.
So rather than focusing on, what's the set of nested loops that I have to make, I want to ask, what would be a meaningful calculation that I would want to do to solve this problem?
So the first thing I might do, focusing on this part, I might write a function, apply, where apply -- I'm using the methodology of functional programming -- apply is a pure function.
It's going to eat some inputs, generate an output, and have no side effects.
So what I want it to do, I want to feed it a list of functions and ask it, what's the answer?
So apply is going to have as its first argument a list, but the list is going to be a list of operations.
It's not a list of strings, it's not a list of integers, it's a list of functions.
And what the procedure apply is going to do is step through that list, applying the functions one by one to argument.
So the final goal is written down here.
If I apply nothing to 7, the answer ought to be 7.
If I apply increment to 7, the answer ought to be 8.
If I apply square to 7, the answer ought to be 49.
And if I apply increment square to 7, the answer ought to be 64.
There's a couple things you're supposed to see here.
First off, apply is a pure function, it has no side effects.
It eats its input, it does a calculation, it tells you an answer, and it's done.
There's no extra things that's going on behind your back, it's not setting a variable, or creating a list, or doing anything like that.
It has inputs, from those inputs it generates an output, and that's the end of the story.
Another thing that you should see is that I'm treating procedures as first-class objects.
That's another tenet of functional programming.
Rather than sending the input to apply as a list of text strings, or as a list of integers, I'm giving it a list of function names.
I'm treating functions just as though it were any other kind of data.
I'm making a list of functions just the same as I could make a list of integers, or if I made a list of strings, I'm just making a list of functions.
So another important feature of this program is the idea that the functions are being treated as first-class objects.
The next procedure is addLevel.
addLevel is going to be a second pure function.
The idea there is that I'm going to use addLevel to construct all the different sequences of operations that are possible.
Each sequence is going to be a list, so the entire-- all possible sequences will be a list of lists.
So the idea in addLevel was going to be that you take the list of lists and you generate from that list of lists a new list of lists that has one more element.
So if the first list of lists had four elements in it, how many lists are in the second list of lists?
Five.
Four, five?
The first list of lists represents some number of different sequence of operations.
My second list of lists-- what I want to do is think about all the different ways that could be extended from a length-4 sequence to a length-5 sequence.
How many ways can you extend a length-4 sequence to a length-5 sequence?
Eight.
So four becomes eight.
Why does four become eight?
You've got like two possibilities
Because there's two possibilities for the way that you could extend it.
I could extend the length-4 sequence by adding on an increment operator, or I could extend it by adding on a square operator.
So every time I addLevel, it's going to create a new list of lists with double as many elements in it as the old list of lists.
Does that make sense?
Other than that, the program works very much the same-- I should illustrate that.
So addLevel applied to the list of lists, increment.
I'm only considering one sequence of length-1, so how many ways can I change that if I'm willing to add procedures increment or square?
Well, I could end up with increment increment by taking that one and that one.
Or I could end up with increment square by taking that one and that one.
So if I were to extend the sequence, increment, two possible ways, by either incrementing again or by squaring again, I end up with two possibilities represented by this list of two different lists.
Is that clear?
The important point is that I'm treating functions as any other kind of data element that could be added to a list.
And when you run that program, you get an answer that's very similar to the program that we looked at previously, the imperative program.
Except that the answer now is a list of functions.
So rather than generating a text-string like the previous example did, this program generates a list as the answer-- the program generates a list, which is a list of functions.
The first element in the list is increment, the second element in the list is the function increment, the third element is square increment square.
So it gave you the same answer, it just represented it differently.
Here I'm representing the answer as a list of functions.
That make sense?
So there's a number of advantages to this approach.
One of them is that it kind of automatically coerces you to think about modules.
By specifically focusing my attention on, how could I break up that problem by defining functions?
I end up with a bunch of functions here, apply and addLevel.
And being functions, they're modules.
They are things that are easy to manipulate in other contexts, and in particular, it's very easy to debug them.
Especially in an interpretive environment like Python.
It's very easy to figure out if your program works by figuring out if each module works.
That's one of the important features of modular programming.
When you have a module, it means that you can use that module in multiple different ways.
It's easy to break apart the debugging problem so that rather than trying to debug the one, monolithic structure-- which was the procedural program that we wrote in the first part, here we can debug the individual components, which happen to be functions.
So I can ask, what happens when I apply the empty list to 7?
Well, my answer better be 7.
So that provides a way of checking.
So one of the features of thinking about the algorithm as being broken up into a number of functions is the greater modularity that allows you, for example, easier debugging.
A much more important reason for thinking this way though, is the clarity of thought.
And that can be derived from another feature of the definition of apply.
The particular way I defined apply is what we call recursive.
It's recursive in the sense that the definition of apply calls apply.
OK, that sounds like a bad idea, right?
How do I define something in terms of itself?
The idea is that each application, every time apply calls itself-- the idea is to structure the procedure so that the new incarnation is in some sense, simpler than the previous one.
If you can guarantee that, it will reduce the complexity of the problem from something that you don't know how to solve, to something that you do know how to solve.
And for that reason, it represents a powerful way to think about structuring programs.
So as an example of that-- as an example of structuring programs as recursions, think about raising a number to a non-negative integer power.
So if I'm trying to raise b to the n, if n is 0, and if n is a non-negative integer-- well, if n is 0, b to the n is 1.
And if n is a non-negative integer, then b to the n can be reduced to b times b to the (n minus 1).
Rewriting that functionally, if I say that my function b to the n can be represented by f of n. This statement is precisely equivalent to saying that f of n is 1, if n is 0, or b times f of (n minus 1) if n is bigger than 0.
So the idea then, is that I may not know how to raise 2 to the 10th, but I can use the rule to say 2 to the 10th, oh, that must be 2 times 2 to the 9th.
Great, I don't know how to do 2 to the 9th, either.
But 2 to the 9th is 2 times 2 to the 8th.
And eventually, I boil it down to a case that I do know the answer to, in particular, b to the 0 is 1.
I would express that idea in Python this way, define exponent of b, n. If n is 0, return 1, otherwise return b times exponent of b, (n minus 1).
OK, so what that does then when you invoke it-- invoking exponent of 2, 6, in fact, invokes exponent an additional six times.
If I ask Python to evaluate exponent of 2, 6 it will end up calling exponent of 2, 5.
But to evaluate that, it will call exponent of 2, 4-- 3, 2, 1, 0.
Then finally, it gets to the base case.
When it gets to the call exponent of 2, 0 it falls into this case which returns 1 always.
So now exponent of 2, 0 returns 1, but then that can pick up where call exponent of 2, 1 left off.
When I did 2, 1 it fell into this case, where it was trying to take 2 times exponent of 2, 0 Now it knows the answer to 2, 0 is 1.
So I can multiply by 2 and get the answer is 2, and it backs out then all of the other answers.
So that's an example of how I could use recursion to reduce a complicated case to a base case-- a simple case that I know the answer to.
Here, the recruitment with the power of n causes a linear increase in the number of invocations, so we would call that a linear process.
So the idea then is that recursion is a good way to reduce a complicated problem to a simpler problem.
Now that algorithm wasn't particularly fast.
If I imagine doing a big number like 1024.
Raising a number to the 1024 power, that could take awhile.
How long will it take?
1024 is going to-- it will make 1024 calls to reduce it to the base case.
Here's a way I could speed it up.
Here I'm trying to make something called fast exponentiation, where I'm going to take advantage of another rule for the way exponentiation works.
Not only is it true that I can write b to the n as b times b to the n minus 1, but if n happens to be even, there's another rule that I can use.
If b is even, then I can raise b to the n over 2, and square the answer.
OK, that's more knowledge about the way exponents work.
The point is, that when I think about the problem recursively, when I think about the problem as a function, there's a natural way to take advantage of that additional information.
So what I can do then is implement in Python in a scheme that looks very similar to the one that I used for the simple exponentiation, but it has a new rule embedded in there.
So I can say, if n is 0 -- that was my original base case, if n is 0 the answer is 1.
If the modulus of n when divided by 2 is 1, if it's odd, use the old rule-- which says b times the exponent of b, n minus 1.
However, if it's neither 0 nor odd, then use this new rule.
If I use that procedure compared to the previous program, exponent, how many invocations of fastExponent is generated by a call to fastExponent of 2, 10?
So talk to your neighbor, figure out an answer, raise your hand.
Use the number before the dot, just to keep you alert.
[AUDIENCE DISCUSSION]
So how many invocations of fastExponent is generated when you call it with the arguments 2, 10?
Everybody raise your hand, tell me a number of fingers.
OK, it's about 90%, something like that.
The most common answer is five, how did you get five?
So do you think of reducing fastExponent of 2, 10?
What happens the first call?
It goes to [UNINTELLIGIBLE] divided by 2 is 5
So the first thing that happens, is that it realizes that the 10 is even, so the 10 is not 0, it's not odd, it goes into this case.
So it tries to run it with 5.
Then when it tries to run fastExponent with 5, what happens?
OK, so 5 is not 0, so it's not the base case.
It is odd, so 5 gets reduced to 4.
4 gets reduced to 2.
2 gets reduced to-- et cetera.
So the idea is that we get a faster convergence because, just like in the very first example we did, we can do two different kinds of operations-- either decrement, or in this case half, and half works faster when the numbers are bigger than decrement does, so it's the same idea as in that first program.
So the idea, then, is that this requires 5 where we would have expected in the previous exponent procedure, it would have required 10.
And that difference just gets bigger and bigger as we make n bigger and bigger.
Much more importantly though, than the fact that we can make something fast, is the idea that this is expressive.
The idea is that by structuring the program this way, it was perfectly obvious how to incorporate more knowledge about the way exponents work.
I could have done it by using a procedural approach.
Do this, then do this, then do this.
But then I have very complicated indices.
Check if it's-- check if it's squared, check if it's whatever, and it's inside the loop, right?
So I complicate my looping structure, where here-- because I'm using the functional approach, it's very easy to understand that it just introduces another case.
It's nothing complicated.
So it's much more important-- the reason we think about recursion isn't so much that it's fast, it's because it's expressive.
It's a way to incorporate knowledge about an answer into a solution, and that's what we'd like to do.
We want to have a way of making complicated things simply.
Here's a way to incorporate knowledge about a procedure in a very straightforward fashion.
All we have is a second case.
Rather than making the loops more complicated, all we have is a simple new case in this statement.
And that's especially important when you think about harder problems.
Here is a much harder problem.
So Tower of Hanoi, you probably played with this as a kid.
I want to transfer all the disks from Post A to Post B-- I have a typo on your handout, by the way.
I typed C on the handout, and then I wrote the program the other way.
That should have been a B.
Transfer a stack of disks from Post A to Post B by moving the disks one at a time, never placing a bigger disk on top of a smaller disk.
OK, that's kind of a challenging problem.
It's a very challenging problem if you try to structure it as loops.
If you try to structure it as loops I would recommend that you do this gazillions of times until you get the pattern stuck in your head, because the pattern's not obvious.
But the pattern is obvious if you think about it as a recursion.
I don't know how to move n from Post A to Post B.
How about this algorithm, take the top n minus 1, move those to C-- get them out of the way.
I don't know how you're going to do that, but just trust me, just do it.
So move n minus 1 of them off the top, put them over on C, that exposes the bottom one.
Move the bottom one to B, which is where I'd like it to be.
And then by this mysterious process of moving n minus 1, bring the n minus 1 back from C back on top of B. OK, what did I just do?
I just started with a problem that I don't know the answer to.
How do I move n disks from this post to that post?
And I broke it into a sequence of three operations, two of which I don't know how to do, but one of which I know.
The one will fortuitously be the base case-- or at least similar to the base case.
So the idea is, if I want to transfer the n, do two versions of the n minus 1 problem and glue them together with this middle step.
OK, well I don't know how to do the n minus 1 problem either, just recurse.
How do I do the n minus 1 one?
Well, I'll do the n minus 2 one.
I don't know how to do that either.
Well, then do the n minus 3 one.
Keep going until I get it down to 1.
Because when I get down to 1, I do know how to move 1.
Does that make sense?
This is supposed to illustrate that the power of recursion is in thinking.
Some algorithms are easy to describe recursively.
That set, is the set when the hard problem can be reduced to a problem of the same type that is in some sense simpler.
Here simpler means taking the index n, and turning it into n minus 1.
If you write this simple little Python snippet and run it, you get this procedure.
And if you-- so the procedure means, take the top disk off A, put it on B.
Take the top disk off A, put it on C. Take the top disk off B, put it on C. If you do that procedure, it will magically do exactly the right thing.
If you try to see the pattern here, so that you were to write this in a procedural approach, the pattern is very peculiar, but the algorithm for doing it is very simple.
So the idea-- one of the reasons that we like recursion, is this idea of expressability.
It makes it easy to express what we're trying to do.
OK, so that's a quick view of the first two ways to do the first problem that I talked about.
We started thinking about a sequence of operations, either increment or square, that transforms an integer i into g. The first thing we looked at was a procedural approach where I just gave you a recipe-- do this, do this, do this.
The issue there is that I ended up with nested loops three deep.
The issue there is keeping track of the indices for all of the different loops.
We just talked about a functional approach.
That can have an advantage, especially whenever the functional approach has a very simple implementation.
The last approach I want to talk about very briefly is object-oriented approach.
Here-- I'm doing the same problem again, find the sequence of operations that transforms i into g, 1 into 100.
Here I'm doing the same algorithm again.
I mean, I haven't really changed the algorithm in going from the procedural approach to the functional approach, to here, the object-oriented approach.
I have changed the representation of the program, and that's the point.
Here the representation of the program is in terms of objects.
Here what I'm going to do is think about organizing all the sequences of operations that are possible in a tree.
So the sequences that I could do, I could do no operation followed by increment increment increment.
Or I could do nothing increment increment square.
Or I could do nothing square increment square.
So I'm thinking about all the possible sequences, but this time, I'm not thinking about it is a text-string, I'm not thinking about it as a list of functions that I need to call.
I'm thinking about it as this tree, this tree of objects.
And in the object-oriented approach, what I'm going to do is think about the solution in terms of building up the tree until I find the answer.
So the key in the object-oriented approach is to think of a useful object.
What would I like to remember when I'm constructing the solution as this kind of a tree?
Well, if I call every circle a node, I can define a class that will keep track of all the important things about each node.
And then I can structure my program as creating nodes, connecting them up into that tree, and looking for the answer.
So each one of these nodes, each one of these circles, ought to have a parent, they ought to have an action, and they ought to have an answer.
So, for example, let's say that I started with the initial value i equals 1.
Then this node right here might say, whatever your answer was at this point, the answer here was 1.
So the answer here is going to be 1 after it's been incremented, so answer should be 2.
So associated with each one of these nodes is going to be a parent.
The parent of this guy is this guy, the parent of this guy is this guy, the parent of this guy is that guy.
Associated with each node is going to be an action.
When I'm here, my action is to increment.
When I'm here, my action is to square.
And associated with each node is going to be the answer.
So if I started at 1, by the time I get here, the answer's going to be 2, 4, 16.
So the idea is that I'm structuring the problem as a tree.
Each node has a couple things it wants to remember, it wants to know who it's parent was.
It wants to know, what's the operation associated with this node?
And it wants to know the answer at this node.
Also associated with each node, a method.
And the method will be to print out the answer if this node is the answer.
So the print routine is going to have to say, OK, I ended up here and that happens to be the answer.
Well, how did I get here?
Well, I have a parent, who has a parent, who has a parent, who is none.
So this routine is supposed to figure out the sequence of operations that led me from the initial value up at the top to the final answer that just happens to be goal.
Is that clear?
So then I structure the program in terms of building those node structures.
I start off by making a node, which is the top node.
It has no parent, it has no action, and the answer to the top node is 1.
And then I just think through loops that are very similar to the loops that I was using before.
They're structured similarly to the way they were structured in the case of the imperative program, except now the iterations are generating new nodes.
The iterations are simpler, because each one just generates a node.
It's not like in the imperative case where I treated separately, what was the ASCII string that represented the answer, and what was the cumulative answer?
Here I don't have two pieces of things that I need to do inside my loop, I just make a node.
So the idea is that when I solve the problem using this approach, what I'm trying to do is build a representation for the solution out of pieces that make sense for this problem.
So the important idea that I want to get across was to think about modularity at the next higher level.
We started last time with modularity at the most microscopic level in Python, and we saw how Python had primitives that allowed us to build more complicated structures and simplified our task of understanding complex designs.
We looked at the most primitive level in the first lecture.
What I just went over was how even within the structure of Python, you can structure programs that are more or less modular.
And the important point is that the way you structure the program influences the degree of modularity that you get.
So the structure of the program has a significant effect on how modular your solution is.
What I want to do for the rest of the time today is jump one higher level yet.
So last time, primitives within Python.
First half of this hour, imperative programming, functional programming, object-oriented programming, approaches that affect the modularity of your solution.
Now I want to go up one more level and think about abstraction and modularity at a much higher level.
And there, I'm going to think about programs that control the evolution of processes.
What's a process?
A process is some set of procedures that evolve over time.
An example was a bank account.
OK, with the problems that we've been thinking about so far, there was a clear answer.
What's the minimum-length sequence that blah, blah, blah.
Right?
There was a clear answer to that problem.
Here there isn't a clear answer to the problem, what's the answer to the bank account?
The bank represents a process, it's something that changes with time.
There are transactions.
Every time you make a deposit or a withdrawal, you do a transaction that causes some action to happen in the bank.
So there isn't an answer, there's an evolving answer.
So those are the kinds of problems I want to think about now.
Graphical user interfaces.
Again, there's no answer to Excel.
Right, you can't say calculate the answer to Excel.
Because whatever-- so the right answer for Excel depends on, what's the sequence of buttons that you're pressing, and what's the sequence of pull-down menus that you're doing, and all that sort of stuff.
So graphical user interfaces are processes.
They're things that evolve over time.
So they take a sequence of inputs, and they generate a sequence of outputs, just like a bank does.
Controllers.
How do you make a robotic steerer?
That's a process.
There are inputs-- so for example, where are you in the middle of the lane?
What time is it?
How many pedestrians are you about to hit?
There's all of those kinds of inputs.
And then there is things that you output-- which is like, keep going.
So some of the steering rules are particularly simple.
Other steering rules are you know, avoid the guy this-- well, anyway.
So the idea is that when you were controlling processes, we're going to want to think about other kinds of programming structures.
And even though the programming structures are going to be different, we're still going to want to have this idea of modularity.
We're going to want to be able to think about, what are the primitives that we have, how can we combine them, what are the important abstractions, and how do we capture patterns?
We're still going to want to do PCAP.
But we're going to want to do PCAP at a much higher level.
And the programming module that I want to introduce is the idea of a state machine.
A state machine is a very convenient structure to capture the behavior of a process.
A state machine has an input, a state, and an output.
The idea is that it's a process.
It's operated upon in steps.
Every step there's a new input.
From that input, the state machine can calculate an output and the next state.
By keeping track of the state, this state machine can remember everything it needs to know so that it can continue the process into the future.
So the idea is that this is going to represent a fundamental programming element, that we'll be able to use to construct things that are like processes.
And in particular, we use this to represent the calculations that are done within our robot.
We'll think about the robot as being a process.
It's not got an answer, you can't ask the question, what's the answer, to the robot.
But you can answer the question, if I were trying to steer and the sonar said (x,y,z), how fast should you turn the wheels?
You can think about that as a state machine.
There's some state of the robot.
There are some inputs-- which is what the sonars are currently telling us, and there's some output-- which is how fast we're driving the wheels.
So we want to think about this kind of a representation for processes that evolve over time.
OK, here's a very simple example, a turnstile.
So we can think about the turnstile as a state machine.
The turnstile has two states-- think about getting onto the T-- either it's locked, or it's not locked.
Those are the two states of the turnstile.
There are some inputs.
The inputs are, I put a coin in it, or I walk through it and the turnstile turns, or I don't do anything.
And based on those inputs, and based on the current state, the turnstile has some outputs.
So the turnstile can either allow you to enter, or ask you to pay.
So the output might be an LED sign that says, please enter.
Or it could have an LED sign that says, please pay.
So those are out all of the elements of a state machine.
And so we can think about a graphical representation for the turnstile in terms of states.
Here I'm representing the states as circles.
There are two states in the turnstile, the turnstiles is either locked or unlocked.
You move between states by getting inputs.
So the inputs are represented by the arcs-- it's the top identifier on each arc is the input, the bottom identifier is the output, and the special arrow says I start here.
So the turnstiles start out locked in the morning-- the turnstile is locked.
If you drop a coin in it, you'll move into the unlocked state and it will output enter.
It will tell you that it's OK to enter because you've put a coin in it now.
Then as long as you stay there and do nothing, it'll continue to say enter-- none enter.
Or if you keep dropping coins in it, it'll say hey, keep going, and it'll continue to eat your coins and continue to say enter-- being very nice.
So in those cases it remains unlocked, it continues to tell you that you can enter, and that state persists until you turn the turnstile.
When you turn the turnstile, this particular turnstile forgets how many coins you gave it and it simply locks and it tells you you've got to pay more.
So the idea is that we can capture the desired behavior of the turnstile-- the turnstile is a process, it's a thing that evolves over time.
But we can capture the essence of what we would like the turnstile to do in terms of this diagram.
We'll call this a state diagram.
And then we can think about behaviors in terms of a set of outputs as a function of time.
So imagine that as a function of time-- say I started in the state locked, and the input is nothing.
So you wake up in the morning, you go to the T, overnight the turnstiles have all been locked.
They all say that they're locked, they all say, pay, as their output.
If I don't do anything it just sits there and it says pay.
If I drop a coin, it says enter and it moves into the unlocked state.
If I don't do anything enter persists and it stays unlocked.
If I walk through it, it eats the coin and it says, OK, you gotta pay now, and it locks.
So you can think about the evolution of this process-- the evolution of the turnstile in terms of the diagram with regard to time.
And the idea then is that this is a kind of a representation that allows us to succinctly capture the important information about how processes work.
One of the important features of this representation is that it removes you one level away from the looping structure.
Somewhere somebody's keeping track of the loop.
Something's going from time 0 to time 1, to time 2, to 3 -- we'll talk about that later.
But for the time being, this representation focuses on what do you do at each instance in time?
So it's a way of tearing apart the time part from the state transition part.
So separate out the loop from the essence, it's very much like the functional example that I did.
The functions divorce themselves from the looping structure that we saw in the imperative approach, and you could define the functions independent of the looping.
We're doing the same thing here.
We can define the state machine representation independent of the looping.
And most importantly, this idea of state machines is modular.
Let me show you how that plays out when we think about the robot.
Think about the problem that I showed you last time where we're trying to have the robot go from point A to point B.
Where, for example, it knows where it is -- say it has GPS or something.
It knows-- from a map say, where it wants to be, but it has no idea of the obstacles between where it is and where it wants to be.
So we can think about that kind of a problem-- we looked at this last time.
It takes a step, it uses its sonars-- which are depicted by these lines, to infer that there are some walls-- that's the black dots.
So it gets a reflection off the sonar that tells it that there's some obstacle in the way.
That means it's not a good idea to try to plow straightforward, so if I back up, the original plan, it had no idea what was between it and its destination, so its plan was just buzz straight through.
At the end of its first step, it has got back sonar reports that tell it that there's a wall here, and here, and here, and here, and here, and here.
And based on where the wall is, and the size of the robot, it can't get into any of these red squares.
It's too big.
So it knows its original path, which went from here to here, isn't going to work because it can't fit through.
So it computes a new path, and then repeat.
So the idea is that we're solving a very complicated problem-- the robot 's solving a very complicated problem, and what I'd like to do now is think through a solution to that problem.
As the robot's moving along, on every step it's getting some new input-- it's behaving just like a state machine.
On every step it gets new input from the sonars.
Based on that new input, it knows where there's new obstacles.
Based on that knowledge, it has a better idea of what the map is.
Based on that knowledge, it has a better idea of finding a plan that might work.
And then based on that knowledge, it knows how to tell the wheels to move forward.
So think about where it is right now.
It has done, logically, three different things.
It has figured out where the walls are.
That's the thing represented in black and red.
We'll call that -- it made a map.
Given the map, it made a plan-- that's the blue thing.
Given the plan, it figured out how to turn the wheels.
It's going straight ahead.
So there are logically three things going on, all at the same time.
You can imagine writing an imperative program to capture that behavior, but that's complicated.
Not impossible, you would structure it as a bunch of loops.
You know, it would-- unless you're a crack programmer, and I'm sure you are-- it will end up being an ugly program.
The idea is, that by using the state machine representation we'll be able to think about how to pull apart those three pieces.
So in particular, we can think about the state machine for the robot-- which would take the sensory input from the sonars and turn it into an action, which is turning the wheels-- we can think about that as having three modules.
There's the mapmaker module, which just looks at the sensory input.
Doesn't care what the plan is, doesn't care how it's turning the wheels right now.
All it the cares about is, what was the -- what's my current state?
And what new information can I infer about the map from the sonars?
So given the sonars and my current state, what's the most recent version of the map likely to look like.
Then there's a planner that says, given the sonars and the map, how would I construct this blue line?
Given my current understanding of the map of the world, how would I get from where I am, to where I want to be?
That's a plan.
Then from that plan-- say the robot is here, from the plan you can see that the thing the robot should do is move straight ahead.
So the mover can take the heading-- the heading is just simply the coordinates of the first destination point in the plan.
So the mover can look at the heading, the first destination point, and figure out how to make the wheels turn.
So the idea is that the state machine representation will let us parse a very complicated problem.
A process-- the process by which the robot gets from here to there, a very complicated process.
We're going to be able to break it up into pieces.
The pieces are going to be much easier.
They're going to be modular.
We will be able to write this piece, and debug it and make sure that it all works, independent of these other pieces.
Because we just have it do any arbitrary walk around task or no walk around at all and ask whether the set of sensory inputs generates the correct map.
Similarly, this will be a module.
This will be something that we can debug as well without having the other pieces working.
We'll be able to say, OK, well, what if I fed a map that I hand constructed to the robot-- ignore this, just forget this, assume there is none of these-- I can hand make a map, feed it to this guy, give it sensory input and see if it comes up with the right heading.
It's just like in that functional approach example that I did.
Having made the individual functions, we were able to test them individually because they were modules.
Having made these individual state machines, we'll be able to test them individually because they're modules.
Then at the end we'll be able to paste them all together and get a very complicated behavior-- much more complicated than you would expect from the sum of the three programming activities.
That's what we want.
That's how we want to structure things.
We want to be able to combine simple things and get something that seems more powerful than the linear combination, because it is.
Okay, so that's the goal.
To do that, we're going to invent some new Python structures.
And we're going to think about the representation of the state machine in Python and we will use objects.
In fact, we use objects at three levels.
We'll think about the highest level of abstraction of this problem as a state machine.
So there's going to be state machine class.
The state machine class is going to have all the things in it that all state machines have to know about.
All state machines, as we define them, are going to have to know how to start, how to make one step, and how to combine single steps into a sequence of steps-- and we'll call that operation of transduction.
So if you give me a list of inputs, transduce will make a list of outputs.
So all state machines are going to be able to-- all state machines are going to have to know how to start, step, and transduce.
Then for particular types of state machines-- like the turnstile, we will generate a subclass.
The subclass of turnstiles are going to have to know-- all of those are going to have to do some other things.
All turnstiles are going to have to have a rule for what happens when you're in state A and you get input i-- what should you do?
Well there's going to be a general pattern that all turnstiles will subscribe to.
We'll put those in the subclass, turnstile.
Then individual turnstiles.
The first one in the Kendall stop, the second one in the Kendall stop, the first one in the Central stop, the third one in the Harvard stop-- particular turnstiles will be instances of the turnstile class.
So the way this plays out, we'll have a generic state machine class defined here.
The state machine class will have a method, start.
Start will create the instance variable, state.
Every instance of a turnstile has to remember where it is, that's an instance variable.
That instance variable gets created by the start routine.
So the very first thing you do if you want to talk about a new turnstile, is you have to instantiate the turnstile.
Then you have to make an instance-- and you do that by calling start.
Every state machine has to have a start routine.
Every start routine will do this-- it'll create an instance variable.
Every state machine has to know how to take a step.
We will make a generic method called, getNextValues, which will include all of the rules for how the class turnstile generates a new output, and a new state from the implant.
And every subclass of state machine will have to define what one of those looks like.
The important thing here is that every state machine has to have a step routine, and the way we've decided to implement the step routine is to call the getNextValues, that is particular to the subclass of interest.
So for example, the getNextValues for a turnstile, might be different from the getNextValues for a planner or a for a mapper.
Then we will subclass state machine according to the kind of state machine we're making.
A planner, a mapper, a mover, a turnstile.
So a turnstile will be a class, which is a subclass of state machine, and that's where we'll define how do you do getNextValues.
If you're a turnstile, here's the way you ought to do getNextValues.
When I call getNextValues, I'll tell you your current state and the current input, and from those two you ought to be able to figure out the next state and the output.
So the getNextValues method for the turnstile class ought to take state and input-- the input, and the current state, and it generates a tuple, which is the new state and the output.
And this is the rule by which it happens.
All turnstiles also have the same start state.
So if you look back again at state machine, state machine created the instance variable, state, by looking at start's state.
All turnstiles have the same start state, they all start locked.
And then finally, the way we'll use this, is we will instantiate a turnstile, we will make an instance of turnstile, put it into TS, turnstile, so this could be Central Square 16, for example.
And then that instance, we'll be able to do things like start, step, and transduce.
So if I transduced this the input, none, coin, none, turn, turn, coin, coin, on a particular turnstile, I'll generate some sequence of outputs like so.
So that's a complicated example using turnstiles.
Let me do something simpler just to make sure you get the idea.
Let's think about a simpler class, a subclass of state machines, a subclass called accumulator.
That subclass always has a start state of 0, and has a getNextValues that returns the same value for the next state and for the output.
It always adds the current state to the input-- hence the name accumulator.
It accumulates the input by taking the current state, adding the currently input to generate a new state-- that's just like a bank account.
The bank account would accumulate transactions.
So if the input to the accumulator is 7, then the state gets incremented by 7, and the output is the value of the new state.
So that's what this says, the starting state is always 0, the getNextValues always returns for this new state-- state plus input, and for the current output, the same thing as the current state.
Everybody's clear?
So the question is, what would happen if I did this sequence of operations?
Let's say that I have the state machine class, I have the accumulator subclass.
I make an instance A, I do some stuff to A, I make an instance B. I do some stuff to B, I do some stuff to A, and I type this.
What gets printed?
Take 30 seconds, talk to your neighbor, figure out what the answer is.
[AUDIENCE DISCUSSION]
OK. What will get printed by the print statement?
Answer (1), (2), (3), (4), or (5).
Everybody raise your hand with an answer.
Excellent, excellent, almost 100%.
I don't think I see a single wrong answer.
So everybody seems to be saying (2).
(2), OK, so what's going on?
So A is an accumulator, we start it, we step it.
B is an accumulator, we start it, we step it, we step.
So what is going to be A, how did you get five?
Well, A started at 0 that's what the start method does, right?
Stepping it stepped it to 7.
B made a new one, we can ignore that because it's a separate instance.
The idea is that the state is associated with the instance.
So when we perturb the state of B by doing B.step, we are touching the state of A-- because they're instance variables.
So that means when we decrement it, we're decrementing the 7 to get 5.
OK, seems reasonable.
So the answer is either (1), or (2), or none.
Why is it the same (21, 21) (21, 21), when we called the A.getNextValues, and here we're calling the B.
Why are they the same?
Because it sets the state [INAUDIBLE]
So getNextValues is a pure function.
So it gets passed in the state, the first number is the state, the second number is the input.
So it doesn't pay any attention to the instance variable state, it uses the thing they got passed in.
Furthermore, the getNextValues associated with A, is precisely the same as the getNextValues associated with B, because getNextValues is something that's in the accumulator subclass.
So they're exactly the same because it's the same method.
So what should be in your head, is a picture that looks like this.
When we said create a state machine, that was something we put in our library, and that makes this environment.
When we said subclass of state machine to make accumulator, that made a new class, a new environment.
And then when we made A and B by saying that A equals accumulator, and B equals accumulator, that made instances.
The instances were initially empty, but when I say A.start, that created the state variable in the A instance, and in the B instance, and associated values with them.
So with the picture that you're supposed to have in mind, is that there's an environment associated with state machine.
There's a separate environment associated with accumulator, but just one of those.
But there's two instances.
So there's two sets of instance variables, one associated with A and B.
So the answer was number (2).
Now the robot example was supposed to motivate this idea that what we're going to do with state machines is put them together modularly.
So what we're going to do-- and you start to do that this week, and also continuing into next week, we'll figure out how to compose state machine M1 and M2 in a cascade.
That means the output of the first one becomes the input to the second one.
We'll put things together in parallel, we'll put things together with feedback.
So what we'll do, is we'll figure out operators.
PCAP-- primitives, means of combination, abstraction, patterns.
The primitives are going to be the state machines, the ways we combine them are going to be these kinds of structures.
Cascade them, put them in parallel, that kind of stuff.
And that's going to allow us to make complicated brains out of simple ones.
Here's a very simple example, what if I had a state machine A, which is an accumulator, so A is an instance of accumulator.
B is an instance of accumulator, there's two separate instances.
And C is a new state composed by cascading A and B.
What would happen if I typed out C.transduce so ([7, 3, 4])?
What's the answer to C.transduce ([7,3,4])?
[AUDIENCE DISCUSSION]
So what happens when I transduce C?
Well C is a composition of A and B.
In particular, it's this composition.
C is just the cascade of A into B.
What that means is that if I put some sequence of inputs into the first machine, and if I put some sequence of the inputs into C, it's the same as putting it into A.
The output of the composite machine is the output of B.
And the input to B is the same as the output of A.
So the ([7,3,4]) goes through A, which is an accumulator, and comes out [7, 10, 14].
It accumulates, that's what accumulators do.
That clear?
Then [7, 10, 14] goes into B and comes out [7, 17, 31].
So C, which is the cascade of two accumulators, ends up transforming [7, 3, 4] into [7, 17, 31].
So that's the idea for how we're going to compose complicated behaviors out of simple ones, and that's what the assignment is for this week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, and welcome.
Last week we started to think about programming.
Programming was the first module in this class.
And it was important for two different reasons.
First, we're going to use programming throughout the term in the study of all the different things that we do.
So it's important that you learn to program now, just so you can use that tool.
More importantly perhaps, we didn't just learn how to program, we focused on how to program in a fashion that would let us construct complicated systems out of a simpler systems.
This is the way that we can manage complexity.
This is the only possible way that we can make complicated systems.
So that was the more important intellectual theme from the first part, where we introduced our mantra, PCAP -- primitives, means of combination, abstraction and identifying patterns.
That's the key to this modular approach to hierarchical kind of design.
Today what I want to do is start the second major theme.
First theme was the design of complex systems, we saw that by reference to programming.
Today we're going to start thinking about modeling and controlling physical systems.
The idea here is not so much how you construct systems-- but we will get back to that.
The idea is to characterize the systems that you've constructed and say something about their metrics as being positive or negative.
So what we want to do is, in fact, focus on behavior.
So to illustrate that, I'll start with an example.
This is an example that you did in design lab last week, or for some of you, yesterday.
The idea was to program the robot so that it could sense the distance to a wall.
Represented two ways here, sort of the view that you would get from Soar, and a more schematic representation showing the position of a robot, the position of a wall.
The idea is that you can sense the position to the wall using the sonars.
You know where you would like to be, because some user told you.
You'd like to be say, half a meter away from the wall.
And your job was to write a program that moves the robot from where it is to where you'd like it to be.
So here's the kind of behavior we might have liked-- so I'll do that again.
So we might have liked that if you started here, you have a nice smooth progression up to where you'd like to be.
Very graceful, ballet type, and you just sort of smoothly glide into the position that you'd like.
Some of you probably achieved that behavior, and some of you probably did other things.
So that might be the intended behavior.
One way to achieve the intended behavior is to use what we call a proportional controller.
In a proportional controller, you make the command be, in some way, proportionate to the intended response.
So imagine this code, which might establish a class for finding the wall.
So the important thing-- as we saw, for all state machines, the important thing is to define a start state and a getNextValues routine.
What this getNextValues routine does is it establishes the desired distance to be a half a meter.
It figures out the current distance to the wall by reading the sonars, and then it specifies an action.
So the first question is what would you like fvel to be in order to make a proportional controller?
Which of those expressions makes sense?
Take 30 seconds, talk to your neighbor, figure out some answer between (1) and (5).
I will ask you in 30 seconds to raise your hand with that number of fingers
[INAUDIBLE]
So what's the right kind of expression if we wanted the controller to be proportionate?
Everybody raise your hand, show me some number of fingers.
OK, the vast majority is saying (2).
Everybody likes the idea of current minus desired.
Why is that the right answer?
That is the right answer.
Why is that the right answer?
How do you prove that to somebody?
Well, in current divided desired, once you meet your desired distance, you're still going to have to [UNINTELLIGIBLE] velocity [UNINTELLIGIBLE], which really doesn't make sense at this point
Exactly.
So that method, we might call extreme cases.
Think of the simplifying cases that give you some insight into the problem.
So one simple case is what if desired and current were the same?
You'd better stop.
OK, that's a simple case.
So the simple case says that you better have one of these-- whatever the right answer is, it better have the property that when current equals desired, fvel is 0.
Otherwise it's not going to work.
And in fact, using just that one simple case, you can eliminate all the ones except (2).
There are some other simple cases, what are some other simple cases?
Simple cases.
Yeah
currentDistance greater than desiredDistance?
Current bigger than desired.
So if current were bigger, that would mean that you were starting out way over here someplace.
So if you were way over there someplace, you'd want the velocity to be positive.
The forward velocity should be positive.
That's how you would disambiguate the sign.
Similarly, if the current were shorter than desired, if you were too close to the wall, you'd like the forward velocity to be negative.
OK.
So that's the proportional controller that we'd like.
So we might fill in our wall finder class this way.
And then when we built it, if things went really well, we would get exactly the behavior we wanted.
But if things went more naturally, it wouldn't quite work that way, and you might get a different kind of behavior.
Let's do that again.
So here is the resulting behavior for that simple controller that I just showed.
So in some sense, that's not as good behavior.
And the way we want to think about behaviors-- the way we're going to develop today, is to think about behaviors in terms of signals.
Plots.
So the first question is, what plot best represents the behavior that I saw here?
So which of those plots best represents that behavior?
Take 30 seconds, talk to your neighbor, figure out what's the right answer?
[INAUDIBLE]
OK, so which behavior best represents the illustrated cartoon that I showed previously?
Raise your hand, show some number fingers so that I can see if you're with it.
Not all correct, but more than 90% correct.
The more than 90% answer is number (2).
What's good about number (2) that's not good about numbers (1), (3), and (4)?
The initial value is different than the final value
Initial is different.
So here, I try initial and final being roughly the same.
So what's showed here, is a plot.
Current distance on the y-axis, step number on the x-axis.
And so we can see from here-- and the implication of the axes, by the way, is 0.
So the implication of the vertical axis is it intersects this at 0, unless I label it otherwise.
And similarly, this horizontal line intersects the vertical at 0 unless I label it otherwise.
So that's the point (0, 0).
So the implication here is that the initial and the final values are the same, as they are here.
Here they're not, what makes (2) better than (1)
[INAUDIBLE] the current distance is decreasing [UNINTELLIGIBLE]
The current distance starts out bigger than it ultimately is.
So we start out bigger than the final value, the final value is presumably half a meter.
We start out roughly twice that far, and then we see some approach.
The approach is not monotonic.
So the answer is (2).
Why do you think it under-shot?
So on the way to going from one to one-half, it transiently went through something smaller than one-half.
Why do you think it did that?
Yeah
Because [INAUDIBLE] this way, there's a small-- there's a small interval of time when its moving at the velocity where it couldn't [INAUDIBLE]
So it's a small interval of time between when it does one thing, and it does something else.
When it senses and moves, for example.
So there could be a time delay, in the system and in fact, that's true, there is a time delay.
So it takes some amount of time for the sonars to register the distance.
Then it takes some amount of time for the computer inside the robot to register that the sonar has told it something different.
Than it takes some time from the time the computer commands the wheels until the robot starts moving.
All of those cumulative effects mean that you have the potential to overshoot where you're going, because there's delay in the system, there's inertia in the robot.
All of those reasons can lead to overshoot.
And the point of today is to figure out some way of predicting and correcting for those kinds of unintended behaviors.
So what we will do then, is develop an approach focused on signals, not systems.
So, so far, we've been thinking about how do you build the system?
Now we're going to think about behavior.
We're going to think about analyzing that behavior.
And the focus is going to be on what was the input, what was desired?
What was the output, what was achieved?
So we're going to be looking at those output signals in order to figure out how good the behavior was.
That approach is called the signals and systems approach.
The idea is characterize your system-- whatever that system is, a physical system, a mathematical system, a computational system, whatever it is, think about it by the way it transforms an input signal into an output signal.
That's kind of a bizarre way to think about systems.
So let me just illustrate that by way of a system that you've all seen before.
OK, here's a simple system, right?
You've all seen this, right?
Anything like 8.01 ring any bells?
OK, so that's a simple system, you all know how to solve it.
There's a gazillion ways you could solve this system, right?
Free body diagrams, kinetic energy, potential energy, there's a gazillion ways you could do it.
You all know how to do it, that's not the point.
The point is, that we're going to learn a different way to solve it.
We're going to think about the mass and spring system-- not like potential energy and kinetic energy, not like free body diagrams, we're going to think about it as transforming an input signal into an output signal.
So the input signal-- it's kind of arbitrary what I use besides the output, but clearly the thing I have control over is my hand.
So it seems natural to associate a variable with the position of my hand, that would be x.
Also it seems natural to associate a variable with the position of the mass, that could be the output y.
That's not unique.
Every time we try to solve a problem we ask ourselves, what's the meaningful input, what's the meaningful output?
The meaningful output could have been the force on the spring.
I'm just sort of arbitrarily saying for the purpose of my analysis, I'm going to consider the input to be the position of the hand, the output to be the position of the mass.
And I'm going to think about the mass and spring system as a box that transforms x into y.
So rather than thinking about it in terms of free body diagrams, and kinetic energy, and potential energy, I'm going to think there's some input signal x, and there's some output signal y.
And what I'd like to do is given x, calculate y. OK, bizarre, why would I do that?
One of the reasons I want to do that is that it's a very general way of thinking about behaviors.
It works for the mass and spring system, it works for water tanks.
What happens if I have water flowing into a tank that's leaky?
Well, it leaks into another tank which is leaky, and that leaks more.
Completely different physics.
Probably wasn't covered in 8.01.
Probably is something you could figure out.
The point is that from the signals and systems point of view, I'm going to map this physics, whatever it is, into this structure.
Think about the tank system as the system that transforms some input.
The input is that there's water spurting for some small interval of time.
The output is that there's water coming out.
And the idea is that I'm going to characterize the system-- whatever it is, as the rule that transforms the input signal into the output signal.
Here's a third example.
I could think of a cell phone system.
Here again, there are very complicated ways we could think about the system, but I'm going to take the particularly simple approach called signals and systems.
Then I'm going to characterize the phone system by the way it transforms some input sound into some output sound.
And as you can imagine, there's a way of thinking about performance in terms of that transformation.
Ideally, we would like this to bear some resemblance to that.
So the point then, is that we're going to focus on behaviors.
And to do so, we're going to think about the signals and systems approach.
Represent a system, whatever it is, by the way it transforms inputs into outputs.
One of the reasons we like the approach is that it's so general.
So you can use it for virtually any kind of a system for which you can develop a mathematical underpinning.
You can use it to analyze electrical systems, mechanical systems, optical systems, acoustic systems, biological systems, financial systems if you're on the dark side.
So there's lots of different kinds of systems that are amenable to this kind of approach.
Also, this approach has a nice modularity.
Having represented a cellphone by transformation from sound in to electromagnetic field out, as illustrated by this cartoon depicting sound going in and having a radio wave transmitting to a tower.
Then we represent tower to tower communication some way, maybe via an optical fiber, maybe via a satellite.
Then tower to cell by the same kind of reverse transformation that we used in the first one.
We can piece those all together, we can treat them as modules, because each box takes an input signal and makes an output signal.
The method is oblivious to the underlying physics.
That affords us a certain amount of power.
And in particular, it's very modular.
You can put together modules that represent very different physical substrates.
That allows us to go back to PCAP.
If the underlying representations for the different physical substrates are the same, we will be able to-- and we will over the next three weeks, develop a bunch of techniques for combining multiple modules into one.
That provides the same kind of abstraction and modularity that we saw in programming for the last two weeks.
So that's the idea.
What we want to do is represent a system by the way it transforms an input signal to an output signal.
There are many different kinds of inputs and outputs, but a fundamental distinction that we are going to have to make is continuous time, and discrete time.
This system works in what we will refer to as continuous time.
Because my position of my hand is a continuous function of the continuous variable, time.
The robots, by contrast, work with what we will call, discrete time.
Steps.
It turns out the math for those two different approaches are very different.
And we will focus entirely in this class on discrete time, because our area of application is the robot.
It's not that continuous is deeper, or harder, or anything like that, it's different.
So we'll focus on discrete time.
And the point of today then, is to develop some representations for signals and systems of this free time nature, that will let us analyze and predict behaviors of systems like the robot system.
The first class of methods for representing such systems is difference equations.
Difference equations are a lot like differential equations, except there's no differentials, there's differences.
Difference equations are the discrete time analog of differential equations for continuous time systems.
Simplest possible example here.
Say I have an output y, that for reasons that I don't care about, I know can be represented as the difference between two values of the input -- x at time n, and x at time n minus 1.
That's a way to represent the behavior of a system, a discrete time system, by using a difference equation.
That's in fact, almost a complete description of the system.
So let me just explain the way you would use that.
It's almost-- because I didn't tell you anything about the input.
I'll tell you something about the input for the purpose of example, where I'll use the simplest possible input I can imagine.
Something that we'll call a delta function.
A delta function is a signal.
It's a discrete time signal.
It has the value 1 if the time index is 0, and has the value 0 everywhere else.
It's, in some sense, the simplest possible signal that we could imagine.
So it's natural to start there.
So what I want to do now is think about if this were a characterization of the system, and if this were the input to that system?
What would the output be?
That's after all-- that was the question that I posed at the beginning.
We would like to build a representation for a system so that we can predict the output, given the input.
So how does that work?
So given the difference equation, all we need to do is step through it.
OK, it's trivial.
We call that analyzing by step by step.
So given the difference equation, given the input signal, all we need to do is sequentially go through the different values of n, and think about the implication of the system on that input.
So if I were to use the value of n, being minus 1, this general form of the difference equation tells me that the minus 1 value of n for the output is related by this difference, with the input.
So y of minus 1 is x of minus 1 minus x of minus 2.
Since both of those are 0, it says that the output at time minus 1 is 0.
Trivial, right?
Trivial.
And similarly, we can just iterate through the solution to the whole signal.
So y of 0 is x of 0 minus x of minus 1. x of 0 is that special one, that is 1.
So now we get 1 minus 0, which is 1. y of 1 is x of 1 minus x of 0.
Now the special one is on the other side of the minus sign, so the answer is minus 1. y of 2 is x of 2 minus x of 1 -- they're both 0.
And in fact, all the answers from now on are going to be 0.
So what I just did is a trivial example of -- I use a difference equation to represent a system, and I figured out the output signal from the input signal.
That's the method that we call-- that's the representation for discrete time systems that we refer to as difference equations.
Difference equations are very powerful.
As we will see, of all the representations we look at, difference equations is the most compact representation.
But there are features of other kinds of representations that are also valuable.
So the next representation I want to look at is a block diagram.
What I'm trying to show here is a picture, a diagram, that represents the same system that we just analyzed with difference equations.
Here though, I'm thinking about it as a signal flow path.
I'm thinking about what's the cascade of operations that you need to do on each sample in order to get from the input to the output?
So the difference equation said every value of the output should be equal to the corresponding value of the input less the value before that.
I represent that in a block diagram by saying there's a straight through path, y of n is equal to x of n. The plus just adds the signals on these two paths.
So y of n is equal to x of n. Subtract out, because I'm multiplying by minus 1.
Delay, so I'm getting the one from before.
So this block diagram just is a symbolic representation of that difference equation.
The value of the block diagram-- we'll see several of them, but one of them is focus on signal flow path.
If you want to visualize the transformation from input to output, the block diagram can provide visual insight into what that transformation is like.
So as before, if I gave you this representation rather than the difference equation, you could still step by step and figure out the way it worked.
It's still easy.
There's one new caveat here.
We have to start the system in a state.
The state that we will usually talk about is what we will call, at rest.
At rest just means all the outputs of all the delays are initially 0.
So that specifies the state of the system at the time the signal was turned on.
So the system is at rest, which means that all the delays start out with output equal to 0.
So at rest means this delay has an output of 0.
Well, if I tell you the output is 0 times 0 for this delay, then it's a simple matter of stepping through what is the output for each corresponding input?
So if I-- the special value of the delta function is that it's 1 at the time 0.
So at the time 0, there's 1 coming in.
That 1 makes it through the adder, adds to 0.
Well, this is 0 because it was at rest.
So the 1 adds to 0 and the output becomes 1.
Notice that the 1 also goes down this path, and goes through the gain of minus 1 to give me minus 1.
But I'm in step 0.
So at step 0, the output of the delay is 0, not minus 1.
So the output then, for time equals 0, is y equals 1.
Just like we solved for the difference equation-- after all, I'm hypothesizing that those two systems are the same, they better give me the same answer.
So then at the next instant, as the time index goes from 0 to 1, two things happen.
The input goes from 1 to 0, and the delay box gets updated.
The delay is now reporting to me the value that was at its input.
So the output of the delay, which had been 0 because it was at rest, becomes minus 1.
So then what happens?
The 1 goes to 0, the 0 goes to minus 1, the 0 adds to minus 1, and we get an answer which is minus 1.
Then the input becomes 0.
That 0 comes down here, the minus 1 goes to 0.
We end up with 0 being added to 0, and the next answer is 0, et cetera.
So the idea then, is that you can step through the block diagram representation, just like you would a difference equation, it's just that now we're thinking about these blocks characterizing the system, rather than thinking of math as characterizing the system.
Why on earth would you do that?
What's good and bad?
What's the relative merits of difference equations verses block diagrams?
Take 30 seconds, talk to your neighbor, figure out some good feature of each
[INAUDIBLE]
OK, we'll start with the easy one.
What's a feature, what's a property of the difference equation that makes it very good.
I already said it, so.
What's good about the difference equation?
Yeah
They can be solved mathematically
They can be solved mathematically.
The block diagram could be solved, maybe not mathematically, but kind of.
Could you refine that a little more?
What's special about difference equations that's different from block diagrams?
Oh come on, I said it
Use math?
They use math, yes yes.
They're concise, a little, right?
It's a very simple expression to say that.
It's by contrast, a bit more complicated to draw this picture, right?
It's mathematically concise, right?
It's completely accurate, self-contained, concise, small.
It's a very concise representation of a system.
So why would we want to use block diagrams?
Can anybody think of any good reason for block diagrams other than Freeman's up front saying, do block diagrams?
Electrical engineering?
Electrical engineering.
There should be a deeper reason, I would hope.
I don't disagree with that reason.
Why do electrical engineers like this?
Yes
It's a more physical representation of this
It's more physical?
Is there anything that you can see that you can't see in math?
Yes
In the way that you're actually going to be programming it.
Like, you're gonna make the delay machine, you're gonna make the state machine that multiplies them by negative 1 and you're gonna see how to connect them
That's a really good point.
It's kind of isomorphic to the implementation.
Everybody get that?
It's kind of a picture of the way you would build the system.
Along those lines, there's some bit of information-- there's actually more information in this one, than there is in that one.
There is exactly one bit more information in the block diagram, what's that bit?
Delay?
Delay?
There's kind of delay by the n minus 1.
Yeah
[UNINTELLIGIBLE]
The input and output are explicit.
Yes.
The arrows are the big difference.
You can't tell from the difference equation, what's the input and the output?
You can tell from the block diagram, what is the input and the output by the direction of the arrows.
So there's more information in the block diagram.
There's another way of thinking about it, and this is kind of a summary of several comments that came from the audience.
The difference equation is declarative.
It tells you a true statement about what the system will do.
The block diagram is imperative.
It tells you what to do now.
Take the input, put into an adder.
Take the input, multiply by minus 1.
Put it into a delay.
Take the delay output, put it into an adder.
The representation with a block diagram is imperative.
It tells you what to do.
So there's extra information, but it comes at a cost.
It's a more complicated representation.
It's a whole picture instead of just an equation.
What we'd like to do, and what I'm going to do now, is develop a different mathematical approach where you get a difference equation that has the same properties of concision, the same conciseness, but also contains all of the information that was in the block diagram.
And the way to do that is to change our focus.
And this is the big abstraction of the day.
Change our focus from thinking about samples, to thinking about signals.
Stop thinking about x of n, start thinking about the input signal x.
This is the same kind of lumping that was key to abstraction in Python.
Put all the interesting data together into a list, put all the interesting operations together into a definition.
Here put all of the interesting samples together into one signal.
So what we want to do, is develop a math by which we can operate on signals instead of samples.
So what I'm going to do is replace the representation x of n, little x of n, with cap-X.
Cap-X means, all of the n's, it's the signal X. Cap-Y means the signal Y.
And I'm going to reinterpret all the boxes.
This box means, take this signal, the whole thing, all n values on it, multiply it sample by sample, by minus 1.
Flip the whole signal upside down.
So I'm going to reinterpret all of the operations on the block diagram in terms of signals, rather than samples.
To do that, I need a representation-- a mathematical representation, for the delay box.
And I'm going to call that R, the right shift operator.
If you apply the right shift operator to a single X, it takes the whole signal X and shifts it to the right one.
That's all it does.
So I'm going to say Y equals R applied to X, or more abbreviated, RX.
Simply says, let Y represent the signal that is the same as X, except every sample was shifted to the right.
The entire signal was shifted to the right.
That's going to let me represent this block diagram this way.
Y-- the whole signal Y, is the sum of the whole signal X, subtract out R applied to X.
Or, even more concisely, calculate Y by applying to X the operator 1 minus R. So I'm thinking now of an operator.
An operator is not something that works on a number.
Operations work on numbers, operators operate on signals.
So I'm thinking about operator expression.
I'm going to try to formulate the transformation from the input to the output in terms of operators.
The way I take X, which is the input, and turn it into Y, is to operate on it with the operator 1 minus R. OK, just to see that you're with me, connecting signals and samples, assume the Y is RX, which of the following is also true?
Take 30 seconds, talk to your neighbor.
Figure out some number -- (1) through (5)
[INAUDIBLE]
OK, so which representation works best?
(1), (2), (3), (4), or none of the above?
Everybody raise a hand, tell me some number.
OK, virtually 100% correct.
The answer was (2), why is the answer (2)?
Can somebody explain that concisely?
No no,no.
I asked that wrong, of course everybody can explain it concisely.
Do I have a volunteer to explain it concisely?
Yes
The R operator just shifts all of the values in X of n to the right.
So you just add 1 to each of these basically?
So you think about it-- so a good way to think about this is to think about simple cases.
Right?
That's the same thing I talked about earlier.
What's a simple case?
Well what if X-- let's imagine that X is simple.
So let's say that X looks like that.
So X is delta.
What would happen, what is the signal RX?
It's a right shift operator.
So what does the signal RX look like?
Shifted to the right, right?
That's the whole point.
So the right shift operator gives you that signal, and we've said that that's Y.
So is it true that Y of n is X of n, for all n?
No.
Is it true that Y of (n plus 1) is equal to X of n for all n?
Well, is it true for n equals 0?
So if we did n equals 0 we get Y of 1, which happens to be 1.
And X of 0, which also happens to be 1.
And if we choose any other n, we would get two 0's.
So at least for this simple case, and it seems to be true, and if you think about building upon this simple case, you can convince yourself that number (2) is always true.
And in fact, the general rule is going to be that the left hand side has to have a number that's bigger than the right hand side.
Which is only true for number (2).
So the idea then, is that by changing our focus, by looking not at samples, but looking instead at signals, we can generate an algebra that looks for all the world like difference equations.
Except it knows the direction from the input to the output.
So it's more powerful.
And in fact, this new algebra is going to obey some very simple properties which we can get a hint at here.
If we were to cascade two systems, imagine this system which looks just like the system we've been looking at, but it's cascaded with a clone.
The question is, what would be the behavior of that cascade?
Well, according to our operator representation, this Y1 signal is just the 1 minus R operator applied to X. Analogously, the Y2 signal should be a similar 1 minus R operator applied to the Y1 signal, which gives us a very concise representation for the cascade.
The point of the slide is that the operator representation gives us a representation that is just as compact as difference equations.
It has other features, that it can be manipulated just like difference equations.
So if we continue with this same example and try to think of the transformation on a sample level, we could say that Y2 is Y1, the straight path, minus Y1 delayed.
But then we could substitute for Y1 of n, that Y1 of n is X of n, this path, subtract X of (n minus 1).
Similarly, collapse, and we get a simple expression.
Now if we think about that same sequence of operations in operator notation, we get a much simpler expression.
Throw away the index arithmetic, it's just R. So the Y2 operator is 1 minus R applied to the Y1 signal.
The Y1 signal is 1 minus R applied to the X signal.
The total is 1 minus R the operator squared, which by polynomial math is just 1 minus 2R plus R squared, the same thing we got there.
The point is, the operator notation is just as compact as the difference equation representation.
And it contains all the information that's in block diagrams.
And it's just as easy to manipulate as a polynomial.
So it's got lots of features that make it superior to difference equations.
The most important of which, is you will be able to understand all systems that we represent using the R operator using polynomial arithmetic, something you learned in high school.
There's nothing new here.
That's what we like.
Representations that simplify the task of finding an answer.
We'll be able to find the answer to these operator expressions by treating them as polynomials.
So you can get a feeling for the way that works by looking here, the power of this.
So the power here is that, among other things, you'll be able to use the operator representation to prove equivalences.
The idea is that here's a system that we looked at before, that was the cascade of two simple delay systems.
Here's a somewhat more complicated, somewhat simpler, representation.
The point is, it's different.
And if we compare the operator representations for the two, we see that they are the same.
And what I'm about to prove, is that when the operator representations are the same, the systems represent the same transformations-- provided they all start at rest.
The provided is important.
Obviously since they have different delays in them, if the delays didn't all start out at 0, the differences in the delays could propagate into the output.
So all of the statements that I'm making are premised on the idea of initial rest.
The other important thing to remember about operators is that it's a higher level of abstraction.
We can think about the operator as composing things, but the things that are composed are whole signals, not samples.
And here's an illustration of how to think about that previous example.
How do you think about this transformation, 1 minus 2R plus R squared.
What happens when you apply that operator to an input signal X?
Well let's say that X was our unit sample signal.
In order to apply this operator 1 minus 2R plus R squared, all we need to do is think about each component.
1 times X is X.
Minus 2R applied to X is minus 2RX.
Plus R squared X is just plus R squared X.
So we start out with X, which is a unit sample, minus 2RX means shift it to the right and multiply by minus 2.
Shift it to the right, multiply by minus 2.
Plus R squared X means shift it twice to the right.
So the result, this operator 1 minus 2R plus R squared applied to X is just the sum of these things.
So you can think about the operator expression, it's just like algebra, except that the elements are signals, not samples.
And as I alluded previously, you can make powerful statements about the way these operators work which map isomorphically onto polynomial math.
So for example, it's easy to prove that if you were to cascade the 1 minus R operator with R. So start with X, apply 1 minus R. Start with X, apply 1 minus R. Then apply R. That's going to result in the same signal, assuming initial rest, as if you were to flip those operators.
The way I can see that is by thinking about signal flow.
You'll remember that I said, one of the features, one of the powers of the block diagram representation is that we can look at signal flow paths.
We can use that as a way of proving things.
This system has two signal flow paths, the one that goes straight through that way, and the one that goes down this way and up that way.
Because of the addition, all it does-- the adder just adds the result of those two flow paths.
So the first flow path introduces one delay, the second flow path inverts and puts in two delays.
So there are two ways to get from the input to the output.
Similarly here, there are two ways to get from the input to the output.
One of them goes through a delay and then goes through the adder, the other goes through two delays at a minus 1.
But that's the same two signal paths that were in the top one.
So that's a way of using the block diagram to prove a relationship about the operator.
What I've just showed, is that the operators obey commutativity.
So what I was able to show is that I can commute these two operators-- doing a general proof is slightly more complicated, I proved it in a special case.
But the general case works too.
Just like polynomials, operators commute.
And I indicated why you should think that's true by thinking about signal flow paths.
Multiplication distributes over addition.
I apologize, the diagram in your notes is wrong.
This is right.
I will always post the notes-- I get up in the morning, I make coffee, I read the lecture notes, and I say, oh my goodness, there's a wrong figure.
In this particular case, one of the staff members wrote me an email and said, Hey Freeman, your slides, something or other, is wrong.
He was right.
So this is the right diagram.
So the idea is that if multiplication distributes over addition, we should expect that R applied to 1 minus R, would give R minus R squared.
And we can again get a feeling for why that ought to be true by thinking about the signal flow paths.
The two signals flow paths that represent-- so the signal flow path that represents here, this says take the 1 minus R operator and apply it to X, then apply R to the 1 minus R operator, result.
As opposed to this one says, apply the R operator to X, then apply the R squared operator to the X and subtract them.
If you think about the signal flow paths through those two systems, they're also the same two signal flow paths.
And here's a more complicated example that shows associativity.
You can think through that, same idea.
The reason that the idea-- the big important point is, difference equations are a good representation for discrete time systems.
They're mathematically compact.
Block diagrams are a good representation, but they have more information.
They tell you what is the input, what is the output, and what are all the different flow paths through the system?
Operators kind of combine the best features of both.
It's mathematically concise, it tells you which is the input and which is the output.
And you can visualize all the flow paths by thinking about all the ads in the operator expression.
OK, to make sure that everybody's up with me, how many of the following systems are equivalent?
You have 30 seconds
[INAUDIBLE]
OK, so how many of those are equivalent?
More hands, more hands.
Not necessarily more fingers, but more hands.
OK, about 75% correct, roughly speaking.
OK, how many distinct signal flow paths are going through the first system?
How many distinct signal flow paths can you see?
Well, here's one.
How many more are there?
Three
Three more.
So here's one, here's one, here's one, and here's one.
All you need to do to think about this system, is think about all of the signal flow paths through all of them, make a sum, and see how many of them have the same sum.
So the path with the greatest delay through this path is two times two delay delay.
Two delays with a multiply by 4.
What's the path with the biggest delay through this one?
Also straight through.
Also delay of two, also a coefficient of 4.
How about this one?
So that's this way.
So they all have the same paths with maximum delay.
The delay is 2 and the coefficient is 4.
This one has four possible paths, this one only has three.
So there's one straight through this way.
There's one that has one fewer delay.
And there's one that only has-- so there's a straight through one, there's a delay of 2, and there's a delay of 1.
So let's do the straight through one.
This one has a straight through path.
No delay, coefficient is 1.
This one has a straight through path, coefficient is 1.
This one has a straight through path, coefficient is 1.
So all three systems have the same maximum delay path, they all have the same minimum delay path, we only have one in the middle yet.
This one has two ways to get a delay of 1.
2 delay, 2R, or 2R.
Since they're both 2R, they sum, so that's 4R.
This one only has one way that we can get 1 delay, and that is to come this way and then go that way, that's 4R.
This one, to get 1 delay, I take the center path, and that's 4R.
Each path has the same ways to get through the system with zero delay, 1 delay, and 2 delays.
They're equivalent in the sense that if you started them with initial rest, they would all generate the same output given the same input.
So the answer's (3).
OK?
So far I've only worked with systems that propagate the inputs systematically through to the outputs.
We call such systems feedforward.
Things are a little bit different when you have cycles.
We call such systems feedback.
So what I want to think about now is how do you think about a system that has a feedback loop in it?
The interesting thing that happens when you have a feedback loop is that the operator expression no longer represents a simple sum of input signals.
Let's look at what happens here.
So Y is apparently the sum of two things.
It's the signal RY, which comes around that way.
Does everybody see that?
So if I think about labeling this input as X, labeling this output as Y, then the correct label for this point is-- don't everybody shout at once.
If this can be labeled as the point Y, what is the correct label to a label at this point?
RY.
So the signal Y must be, RY plus X.
What that says, is that if I apply the 1 minus R operator to Y, I should get X. OK, that's a fine operator expression, except that it's not a formulaic operator applied to the input.
The operator is applied to the output.
OK, the difference here is the difference between an imperative system, like we talked about block diagrams when we were thinking about samples.
The block diagram tells you what to do step by step.
Regardless of whether you have feedback, the block diagram always tells you take X of n, add it to Y of n, whatever.
There's an imperative rule, do this.
We took the block diagrams and we turned them into operators, and we ended up with something that is not imperative.
This is much more the kind of statement we got when we did difference equations.
This is a statement of truth, it's declarative.
If you tell me the signal X, it must be true that the resulting signal Y when operated upon by 1 minus R, is X.
So the idea is that it's a declaration, it's not an imperative rule.
Does everybody get that?
So this statement up here told me a rule, start with X, apply the 1 minus R operator and you will get Y.
That's an imperative operation, do this.
This is a declaration.
If you tell me X, Y must be the signal that when operated on by 1 minus R, gives you X.
But it doesn't tell me a way to find it.
It tells me a truth, but it doesn't tell me how to find that truth.
So let's go back, let's back up.
OK, we got a representation, we like the representation.
It's concise, it has many of the features of block diagrams, it doesn't seem to be imperative.
Well, that's a problem.
So let's back up, think about how this same system that ran into a problem with the operator, think about what must the answer be.
Well the answer we can figure out by doing step by step.
Imagine that it starts at rest, so the output starts at 0.
And now I just tick through the samples.
So when the first sample comes in, X is equal to 1, I'm thinking about the unit sample response.
We call the delta function the unit sample.
When the unit sample at time n equals 0 comes in, it has a value of 1.
The 1 adds to the initial condition, which is 0, to give me 1.
Then, this output is 1, so when the clock ticks, the input goes from 1 to 0, but the output of the delay goes from 0 to 1.
So when the clock ticks, I get another 1.
And that persists.
Does everybody see what's going on?
So I initially had a 0 coming out of the delay.
The unit sample made the first output be one, but then that 1 fed back in to make this be 1.
Which combined with the next 0 to give me the same 1, and that state persisted.
What's different, is that the output signal persists long after the input went away.
In fact, there is a prescriptive way to figure out the relationship between the input and the output.
It's just that it takes an infinite number of delays.
Here's an alternative system that would generate the same response to a unit sample signal as was generated by the simple feedback system.
It needs to generate the answer 1, 1, 1, 1, 1 -- when the input is just 1.
Well, the output at 0 happens through this path.
The output at 1 happens through this path.
The output at 2 happens through this path.
3-- et cetera.
There's a separate path for every one of those separate components of the output.
That's how we can think about this construction.
The input had a single non-zero entry, the output has an infinite number.
We can think about that as resulting from an infinite number of paths, or something similar, about the simple feedback system, which can be represented by that operator representation, and the infinite feedforward system.
This is a simple feedback system.
This is an infinite feedforward system.
There's something the same about those two.
In fact, they're equivalent in the sense that if all the delays start out with initial conditions of 0, they will generate the same response to all possible input signals.
Those two signals are equivalent, and that's proved here.
All you do is you say, OK, Y2 depends on X2 this way.
If X2 is the same as X1, I can substitute it.
But X1, according to this rule, looks like 1 minus RY1.
When you multiply out this mess, you get Y1.
What I just showed is that if X1 is equal to X2, then Y1 is equal to Y2.
Those two systems are the same.
Well that's weird.
So there's something the same about that operator and that operator.
We write that this way.
So here's the feedback system, we think about that as representing the operator Y over X, 1 over (1 minus R).
So in order to calculate X, cross multiply by X. Y is the operator, 1 minus R applied to X.
So we want to say Y is the operator 1 minus R applied to X.
What is the operator 1 over (1 minus R)?
Well, if you didn't know anything but polynomial math, you might have expanded this in a series.
And in fact, that gives you exactly the right answer if you were to expand 1 over R in a series.
So, for example, evaluate it using synthetic division, evaluate it with a Taylor series, however you want to do it.
Think about R as though it were a number, just like you would if it were a polynomial.
Expanded just like you would if it were a polynomial.
And what you see, is that there's a representation for this operator 1 over (1 minus R) that is equivalent.
That's exactly the same as if I applied the operator 1 plus R plus R squared plus R cubed to X.
Those two are equivalent in the sense that if both systems start out at rest, and if they are both applied to the same input, they both generate the same output.
So that gives us that way of thinking about operators that have numerators and denominators.
So, to make sure you're up to speed, a system is described by the following operator expression.
Determine the output of the system when the input is a unit sample
[INAUDIBLE]
What's the first thing I should do?
OK, this is one of those systems that has the R polynomial in the bottom.
So it says that X must be the same signal by cross multiplying-- X must be the same signal as the (1 plus 2R) operator on the Y. OK, that's backwards.
That's not the way I want to think about it.
How do I make that into a forward statement that tells me what operator gets applied to X?
The answer is that.
So what do I do?
Multiply by-- actually you could cross multiply.
How do I convert this into an operator that looks like just the numerator times X. Yeah
1 minus 2R plus 4R squared
Exactly.
What I want to do is convert it by synthetic division, Taylor series, whatever method.
I want to think about what would 1 over (1 plus 2R) look like?
What's the reciprocal of 1 plus 2R?
That is 1 minus 2R plus 4R squared, et cetera.
So now I have this, which I apply to X, which is a unit sample.
So now I want to think about applying this operator to the unit sample signal.
But that's easy.
The first term just brings out delta.
Minus 2R applied to delta shifts the delta by 1.
Gives me delta of n minus 1, and multiplies by minus 2.
And that whole mess then just says that my output looks like this.
If the input was X, which was a unit sample signal, my output has an infinite number of terms.
Each one is a delayed version of the predecessor, and the weights go 1, minus 2, plus 4, minus 8, plus 16, and diverge.
So what we just did was pretty complicated.
We just solved a block diagram, but we did it with polynomial math.
We did it with math that you learned in high school.
That's the point.
In fact, the point of today is that any system that's built out of simple parts-- delays, adders, scalers, that sort of thing-- can be represented by a difference equation.
Fine, that's good, difference equations are wonderful.
They can equivalently be represented by an operator equation.
The operator equation has more information in it.
It knows how to get from the input to the output.
It's imperative.
It's easy to manipulate.
You use the same rules that you use for polynomials.
So all in all, this is a more powerful kind of representation.
And any system that can be represented by a difference equation can similarly be represented by an operator equation.
That's why we're focusing on operators.
So final question.
Think about-- do everything backwards now.
Here's a block diagram, find the associated difference equation.
And the idea is to take advantage of operators.
In the interest of time, let me just do it.
If we wanted to-- because I'm running out of time.
So I could start with the block diagram, I could stay in block diagram domain.
Presumably that will work, that's hard.
I want to do the easy way.
So convert it to operators.
How do you convert a block diagram to operators?
Replace the delays by R, label all the signals.
x becomes X, y becomes Y. I don't have a name for this, so I'll call it E, error.
I don't have a name for this, so I'll call it W, who knows.
And then I'll express each of the relationships imposed by the plus sign, this R or this R, by a line of operator reasoning.
The plus says that the E signal is X plus W. The R says that the Y signal is R applied to E. This box says that the W signal is R applied to Y. I get three equations in R. I just solve algebraically.
None of this difference stuff, none of the square brackets with n's in them.
I just use algebra.
So I solve it algebraically and I get this.
And that translates into a corresponding difference equation, showed here.
The point.
The point is three different representations.
Difference equations, block diagrams, operators.
Operators are easiest.
Even when I was asked to solve a problem that has no operators in it, it's easier to cast it into an operator expression, solve it in the operator domain, and then turn it back into a difference equation.
Starting next week, we will figure out much more powerful things that we can do with operators.
This is just the beginning.
So with that, let me just summarize that we looked at three representations, and the point of the labs for the week are going to be to exercise this, to get some experience with representing signals in Python.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Last time, which is now two weeks ago, we started the talk about the signals and systems approach, by which I mean, think about a system by the way it transforms its input signal into an output signal.
That's kind of a bizarre way of thinking about systems.
I demonstrated that last time by thinking about a mass and a spring, something you have a lot of experience with, but you probably didn't use this kind of approach for thinking about it.
Over this lecture and over the next lecture, what I'd like to do is show you the advantages of this kind of approach.
And so for today, what I'd like to do is talk about how to think about feedback within this structure, and I'd like to also think about how you can use this structure to characterize the performance of a system in a quantitative fashion.
So first off, I want to just think about feedback.
Feedback is so pervasive that you don't notice it most of the time.
You use feedback in virtually everything that you do.
Here is a very simple example of driving a car.
If you want to keep the car in the center of your lane, something that many people outside of Boston, at least, think is a good idea, then you are mentally doing feedback.
You're constantly comparing where you are to where you'd like to be and making small adjustments to the system based on that.
When you think of even the most simple of systems, like the thermostat in a house-- I'm not talking about a cheap motel.
The cheap motels don't do this.
But in a real house, there's a thermostat, which regulates the temperature.
That's important, because if the temperature suddenly drops, it would never do that of course, but if the temperature ever dropped, it could compensate for it.
Here's one of my favorite examples.
Feedback is enormously pervasive in biology.
There's no general rules in biology, but to a first cut, everything is regulated.
And in many cases, the regulation is amazing.
Here's an example from 6.021, where it's illustrating the system that your body uses to regulate glucose delivery from food sources to every cell in your body, which is crucial to your being cognizant and mobile.
And the idea is that it does that with amazing precision despite the fact that eating and exercise are enormously episodic.
In order for you to remain healthy and functional, you need to have something between approximately two and ten millimoles per liter of glucose in your blood at all times.
Were it to go higher than that, systematically, you would develop cardiac problems and could lead to even congestive heart failure.
If you were to have lower than two millimoles per liter, you would go comatose.
That's a very narrow range, two to five, especially because what we eat is so episodic, when we exercise is so episodic.
It's amazing, and just to dramatize how amazing it is, how much sugar do you think circulates in your blood right now?
Well, if you convert five millimoles per liter, and if you assume the average person has about three liters of blood, which is true, and if you calculate it, that comes out to 2.7 grams.
That's this much.
This is 2.7 grams of table sugar.
So this is how much sugar is in your blood, now.
This is what's keeping you healthy.
This is what's keeping you from becoming comatose.
Or not.
How much sugar do you think is in this?
The amount in that other cup
Exactly.
We call that the Theory of Lectures.
I don't want to look like an idiot, so my problems make sense.
This is the amount of sugar in a can of soda.
Numerically, this is 39 grams.
That's more than 13 times this much.
So when you down one of these, it's very important that the sugar gets taken out of the blood quickly, and it does.
And that happens by way of a feedback system, and the feedback system is illustrated here.
Basically, the feedback involves the hormone insulin, and that's why insulin deficiency is such a devastating disease.
Finally, everything we do has feedback in it.
Think about how your life would be different if it didn't.
Even a simple task, like removing a light bulb, would be virtually impossible except for the fact that you get feedback from everything.
Your hand's amazing.
You have touch sensors, you have proprioceptive sensors, you have stress sensors on the muscles and ligaments, and they all coordinate to tell you when to stop squeezing on the light bulb so you don't break it.
That's all completely amazing, and what we'd like to do, then, is think about that kind of a system, a feedback system, within the signals and systems construct.
As an example, I want to think through the WallFinder problem that you did last week in Design Lab.
I'm sure you all remember that, in that problem, we were trying to move the robot to a fixed distance away from the wall, and we thought about that as a feedback system comprised of three parts, a controller, a plant, and a sensor.
We wrote difference equations to characterize each of those parts, and then we figured out how to solve those difference equations to make some meaningful prediction about how the robot would work.
So just to make sure you're all on board with me, now, here's a question for you.
These are the equations that describe the WallFinder problem.
How many equations and how many unknowns are there?
Take 20 seconds, talk to your neighbor, figure out an answer between (1) and (5).
Unlike during the exam next week, you are allowed to talk.
[CLASSROOM SIDE CONVERSATIONS]
T and K are no.
[CLASSROOM SIDE CONVERSATIONS]
OK, so what's the answer, number (1), (2), (3), (4), or (5)?
Everybody raise your hands.
Let me see if I got the answer.
OK. Come on, raise your hands.
Come on, everybody vote.
If you're wrong, just blame it on your neighbor.
You had a poor partner.
That's the idea.
Right?
So you can all vote, and you don't need to worry about being wrong.
And you're all wrong, so that worked out well.
OK, so I don't like the-- so the predominant answer is number (2).
I don't like number (2).
Can somebody think of a reason, now that you know the answer by the Theory of Lectures, the answer is not (2)?
Why isn't the answer (2)?
Yeah?
Oh, I was saying (5)
You were saying (5).
Why did you say (5)?
I don't remember for sure, but can you substitute n for different things?
Like Df of n, you could substitute n for, and if you did that you would have two equations, which is [INAUDIBLE]
[UNINTELLIGIBLE] kind of thing, you count before or after doing substitution and simplification.
And I mean to count before you do simplification.
Any other issues?
Yeah?
Is D0 [UNINTELLIGIBLE]
Say again?
D0 [UNINTELLIGIBLE]
Is D0_n different or the same from D0_n-1?
That's the key question.
So I want to think about this as a system of algebraic equations, and if I do that, then there's a lot of them.
So it looks like there's three equations.
The problem with that approach is that there's actually three equations for every value of n. If you think about a system of equations that you could solve with an algebra solver, you would have to treat all the n's separately.
That's what we call the samples approach.
So here's a way you could solve them.
You could think about what if k and t are parameters, so they're known, they're given?
What if my input signal is known, say it's a unit sample signal, for example?
What would I need to solve this system?
Well, I'd need to tell you the initial conditions.
So in some sense, I want to consider those to be known.
So my knowns kind of comprise t and k, the initial conditions for the output of the robot, and the sensor, all of the input signals because I'm telling you the input and asking you to calculate the output.
My unknowns are all of the different velocities for all values of n bigger than or equal to 0 because I didn't tell you those.
All of the values of robot's output at samples n bigger than 0.
All the values of the sensor output for values n bigger than 0.
So I get a lot of unknowns, infinitely many, and I get a lot of equations, also infinitely many.
So the thing I want you think about is, if you're thinking about solving difference equations using algebra, that's a big system of equations.
By contrast, what if you were to try to solve the system using operators?
Now, how many equations and unknowns do you see?
By the Theory of Lectures.
OK, raise your hand.
Or talk to your neighbor so you can blame your neighbor.
Talk to your neighbor.
Get a good alibi.
[CLASSROOM SIDE CONVERSATIONS]
So the idea here is that if you think about operators instead, so that you look at a whole signal at a time, then each equation only specifies one relationship among signals, and there's a small number of signals.
So if I think about the knowns being k, t the parameters and the signal d(i), and if I think about the unknowns being the velocity, the output, and the sensor signal, then I get three equations and three unknowns.
So one of the values of thinking about the operator approach is that it just simply reduces the amount of things you need to think about.
It reduces complexity.
That's what we're trying to do in this course.
We're trying to think of methods that allow you to solve problems by reducing complexity.
We would like, ultimately, to solve very complicated problems, and this operator approach is an approach that lets you do that.
It does a lot more than that, too.
It also generates new kinds of insights.
So it lets you focus on the relations, but the relation is now not quite the same as we would have expected from algebra.
Now, the relation between the input signal and the output signal is an operator.
We're going to represent the operation that transforms the input to the output by this symbol, h. We'll call that the system functional.
It's an operator.
It's a thing that, when operated on x, gives you y.
And this is one of the main purposes of today's lecture, it's also convenient to think about h as a ratio.
We like to think of it that way because, as we'll see, there's a way of thinking about h as a ratio of polynomials in R. So two ways of thinking about it.
We're trying to develop a signals and systems approach for thinking about feedback.
We want to think about the input goes into a box.
The box represents an operation.
We will characterize that by a functional.
We'll call the functional h. The functional, when applied to the input, produces the output, and what we'd like to do is infer what is the nature of that functional, and what are the properties of the system that functional represents.
OK, so I see that you're all with me.
Think about the WallFinder system.
Think about the equations for the various components of that system when expressed an operator form.
And figure out the system functional for that system.
figure out the ratio of polynomials that can be in r that is represented by h. Take 30 seconds, talk to your neighbor, figure out whether the answer is (1), (2), (3), (4), or (5).
So what's the answer -- (1), (2), (3), (4), or (5)?
Come on, more voter participation.
All right?
Blame it on your partner.
OK, virtually 100% correct.
So the idea is algebra.
You solve the operator equations exactly as though they were algebraic.
Here, I've started with the second equation and just done substitutions until I got rid of everything other than d(0) and d(i).
So I express v in terms of ke, then I express e in terms of d(i) minus rd(0).
Then I'm left with one equation that relates d(0) and d(i), which I can solve for the ratio.
And the answer comes out there, which was number (3).
Point is that you can treat the operator just as though it were algebra, so that results in enormous implications.
But what we want to understand is what's the relationship between that functional, that thing that we just calculated, and the behaviors.
These are the kinds of behaviors that you observed with the WallFinder system.
When you built the WallFinder system, depending on what you made k, you could get behaviors that were monotonic and slow, faster and oscillatory, or even faster and even more oscillatory.
And what we'd like to know is, before we build it, how should we have constructed the system so that it has a desirable behavior?
And, incidentally, are these the best you can do?
Or is there some other set of parameters that's lurking behind some door that we just don't know about, and if we could discover it, it would work a lot better?
So the question is, given the structure of our problem, what's the most general kind of answer that we can expect?
And how do we choose the best behavior out of that set of possible behaviors?
So that's what I want to think about for the rest of the hour and a half, and I want to begin by taking a step backwards and look at something simpler.
The idea's going to be the same as the idea that we used when we studied Python.
I want to find simple behaviors, think about that as a primitive, and combine primitives to get a more complicated behavior, so I want to use an approach that's very much PCAP.
Find the most simple behavior, and then see if I can leverage that simple behavior to somehow understand more complicated things.
So let's think about this very simple system that has a feedback loop that has a delay in it and a gain of P0.
What I want to do is think about what would be the response of that very simple system if the input were a unit sample.
So find y, given that the input x is delta.
In order to do that, I have to start the system somehow.
I will start it at rest.
You've all seen already, I'm sure, that rest is the simplest assumption I can make.
I'll say something at the end of the hour about how you deal with things that are not at rest.
For the time being, we'll just assume rest because that's simple.
Assume that the system is at rest, that means that the output of every delay box is 0.
That's what rest means.
If the output of this starts at 0, then the output of the scale by P0 is also 0.
And if that's 0, and if the input is at 0 because I'm at time before 0, then the output is 0, indicated here.
So now if I step, then the input becomes 1 because delta of 0 is 1.
The output of the delay box is still 0, so the first answer is 1, the 1 just propagates straight through [UNINTELLIGIBLE] box.
Then I step, and the 1 that was here goes through the r and becomes 1, the 1 goes through P0 and becomes P0, but at the same time, the 1 that was at the input goes to 0 because the input is 1 only at time equals 0.
So the result, then, is that after one step, the output has become P0.
This propagated to 1, that became P0, add it to 0, and it became P0, so now the answer, which had been 1, is P0.
On the next step, a very similar thing happens.
The P0 that was here becomes the output of the delay, gets multiplied by P0 to give you P0 squared, gets added to 0 to give you P0 squared, et cetera.
The thing that I want you to see is that the output was in some sense simple.
The value simply increased as P0 to the n, geometrically.
There's another way we can think about that.
I just did the sample by sample approach, but the whole theme of this part of the course is the signals approach.
If I think about the whole signal in one fell swoop, then I can develop an operator expression to characterize the system.
The operator expression says the signal y is constructed by adding the signal x to the signal P0RY.
If I solve that for the ratio of the output to input, I get 1 over (1 minus P0R).
Again, going back to the idea that I started with, that we're going to get ratios of polynomials in R now the R is in the bottom, and now I can expand that just as though it were an algebraic expression.
I can expand 1 over P0R in a power series by using synthetic division.
The result is very similar in structure to the result we saw in sample by sample.
It consists of an ascending series in R, which means an ascending number of delays.
Every time you increase the number of delays by 1, you also multiply the amplitude by P0, so this is, in fact, the same kind of result, but viewed from a signal point of view.
Finally, I want to think about it in terms of block diagrams.
Same idea, I've got the same feedback system, but now I want to take advantage of this ascending series expansion that I did and think about each of the terms in that series as a signal flow path through the feedback system.
So one, the first term in the ascending series, represents the path that goes directly from the input to the output, passing through no delays.
The second term in the series, P0R, represents the path that goes to the output, loops around, comes back through the adder, and then comes out.
In traversing that more complicated path, you picked up 1 delay and 1 multiply by P0.
Second term, two loops.
Third term, three loops.
Fourth term, four loops.
The idea is that the block diagram gives us a way to visualize how the answer came about.
It came about by all the possible paths that lead from the input to the output.
Those possible paths all differed by a delay, and that's why the decomposition was so simple, each path corresponding to a different number of delays through the system.
That won't always be true from more complicated systems, but it is true for this one.
This flow diagram also lets you see something that's extremely interesting.
Cyclical flow paths, which are characteristic of feedback-- feedback means the signal comes back.
Cyclical flow paths require that transient inputs generate persistent outputs.
They generate persistent outputs because the output at time n is not triggered by the input at time n. It's triggered by the output at time n minus 1.
It keeps going on itself.
That's fundamental to feedback.
There's no way of getting around that.
That's what feedback is.
And it also shows why you got that funny oscillatory behavior in WallFinder.
There wasn't any way around that.
Feedback meant that you were looping back.
That meant that there was a cycle in the signal flow paths.
That means that even transient signals, signals that go away very quickly like the [INAUDIBLE] sample, generate responses that go on forever.
So that's a fundamental way of thinking about systems.
Systems are either feedforward or feedback.
Feedforward means that there are no cyclic paths in the system.
No path in the system that take you from the input to the output has a cycle in it.
That's what acyclic means.
That's what feedforward means.
Acyclic, feedforward, those all have responses to transient inputs that are transient.
That contrasts with cyclic systems.
A cyclic system has feedback and will have the property that transient signals can generate outputs that go on forever.
OK, how many of these systems are cyclic?
Easy questions.
15 seconds, talk to your neighbor.
OK, so what's the answer?
How many?
OK, virtually 100%.
Correct, the answer's (3).
I've illustrated the cycles in red, so there's a cycle here, there's two cycles in this one, and there's a cycle here.
So the idea is that, when you see a block diagram, one of the first things you want to characterize, because it's such a big difference between systems, is whether or not there's a cycle in it.
If there's a cycle, then you know there's feedback.
If there's feedback, then you know you have the potential to have a persistent response to even a transient signal.
OK, so if you only have one loop of the type that I started with, where we had just one loop with an R and a P0, then the question is, as you go around the loop, do the samples get bigger, or smaller, or do they stay the same?
That's a fundamental characterization of how the simple feedback system works.
So here, if on every cycle the amplitude of the signal diminishes by multiplication by half, that means that the response ultimately decays.
Mathematically, it goes on forever, just like I said previously, but the amplitude is decaying, so practically it stops after a while.
It becomes small enough that you lose track of it.
By contrast, if every time you go around the loop, you pick up amplitude, if the amplitude here were multiplied by 1.2, then it gets bigger.
So the idea, then, is that you can characterize this kind of a feedback by one number.
We call that number the pole.
Very mysterious word.
I won't go into the origins of the word.
For our purposes, it just simply means the base of the geometric sequence that characterizes the response of a system to the unit sample signal.
So here, I've showed an illustration of what can happen if p is 1/2, p is one, p is 1.2, which you can see decay, persistence, divergence.
Can you characterize this system by P0?
And if so, what is P0?
Yes?
No?
[CLASSROOM SIDE CONVERSATIONS]
Yeah, and virtually everybody's getting the right answer.
The right answer's (2).
So we like algebra.
We like negative numbers, so we're allowed to think about poles being negative.
In fact, by the end of the hour, we'll even think about poles having imaginary parts, but for the time being, this is fine.
If the pole were negative, what that means is the consecutive terms in the unit sample response, the response of the system to a unit sample signal, the unit sample response, the unit sample response can alternate in sine.
OK, so this then represents all the possible behaviors that you could get from a feedback system with a single pole.
If a feedback system has a single pole, the only behaviors that you can get are represented by these three cartoons.
So here, this z-axis contains all possible values of P0.
If P0 is bigger than 1, then the magnitude diverges, and the signal grows monotonically.
If the pole is between 0 and 1, the response is also monotonic, but now it converges towards 0.
If you flip the sign, the relations are still the same, except that you now get sign alternation.
So if the P0 is between 0 and minus 1, which is here, the output still converges because the magnitude of the pole is less than 1.
But now the sign flips.
And if the pole is below minus 1, then you get alternation, but you also get divergence.
The important thing is we started with a simple system, and we ended up with an absolutely complete characterization of it.
This is everything that can happen.
That's a powerful statement.
When I can analyze a system, even if it's simple, and find all the possible behaviors, I have something.
If you have a simple system with a single pole, this is all that can happen.
There might be offsets.
There might be delays.
The signal may not start until the fifth sample, but the persistent signal will either grow without bounds, the k to 0, or do one of those two with alternating sign.
That's the only things that can happen, which of course, begs the question, well, what if the system's more complicated.
OK, so here's a more complicated system.
This system cannot be represented by just one pole.
In fact, the system's complicated enough you should think through how you would solve it.
You should all be very comfortable with this sort of thing.
So if you were to think about what if I had a system like so, and I want it to be 1.6 minus 0.63.
What would be the output signal at time 2 if the input were a unit sample signal?
OK, as with all systems we're going to think about, we have to specify initial conditions.
The simplest kind of initial conditions we could think about would be rest.
If I thought about this system at rest, then the initial outputs of the R's would be 0.
That's at rest.
For times less than 0, the input would be 0.
0 times 1.6 plus 0 times -0.63 plus 0 would give me 0.
Now, the clock ticks.
When the clock ticks, it becomes times 0.
At times 0, the input is 1.
This 0 just propagated down to here, but this was 0, so nothing interesting happens at the R's.
But now my output is 1.
Now, the clock ticks.
What happens?
When the clock ticks, this 1 propagates down here.
This 0 propagates down here, but that was 0.
This 1 goes to 0 because the input's only 1 at times 0.
So what's the output?
1.6.
Now, the clock ticks.
What happens?
Well, this 1 comes down here.
This 1.6 comes down here.
This 0 becomes another 0 because the input has an infinite stream of 0's after the initial time.
So what's the output?
Well, it's 1.6 times 1.6 plus 1 times -0.63, so the answer is number (3).
Yeah?
OK.
I forgot to write it up there, so the answer's in red down here.
1.6 squared minus 0.63.
OK?
The point is that it's slightly more complicated to think about than the case with a single pole, and in fact, if you use that logic to simply step through all the responses, you get a response that doesn't look geometric.
The geometric sequences that we looked at previously either monotonically increased, monotonically decreased towards 0, or give one of those two things and alternated.
This does none of those behaviors.
So the point is Freeman's an idiot.
He spent all that time telling us what one pole does, and now two poles does something completely different.
Right?
So the response is not geometric.
The response grows and then decays.
It never changes sign.
It does something completely different from what we would have expected from a single pole system.
As you might expect from the Theory of Lectures, that's not the end of the story.
So the idea is to now capitalize on this notion that we can think about operators as algebra.
If our expressions behaved like I told you they did last lecture, if they behaved as entities upon which-- if they are isomorphic with polynomials, as I said, then there's a very cute thing we can do with this system to make it a lot simpler.
The thing we can do is factor.
If we think about the operator expression to characterize this system, the thing that's different is that there's an R squared.
But if R operators work just like polynomials-- you can factor polynomials.
That's the factor theorem from algebra.
And if I factor it, I get two things that look like first-order systems.
Well, that's good.
The factored form means that I can think about this more complicated system as the cascade of two first-order systems.
Well, that's pretty good.
In fact, it doesn't even matter what order I put them in because, as we've seen previously, if the system started at initial rest, then you can swap things because they obey all the principles of polynomials, which include commutation.
So what we've done, then, is transform this more complicated system into the cascade of two simple systems, and that's very good.
Even better, we can think about the complicated system as the sum of simpler parts, and that uses more intuition from polynomials.
If we have one over a second-order polynomial, we can write it in a factored form here, but we can expand it in what we call partial fractions.
We can expand this thing in this sum, and if you think about putting this over a common denominator and working out the relationship, this difference, 4.5 over 1 minus 0.9R minus 3.5 over 1 minus 0.7R.
That's precisely the same using the normal rules for polynomials.
That's precisely the same as that expression.
But the difference, from the point of view of thinking about systems, is enormous.
We know the answer to that one.
That's the sum of the responses to two first-order systems, so we can write that symbolically this way.
We can think about having a sum system that generates this term.
This term is a simple system of the type of that we looked at previously that, then, gets multiplied by 4.5.
I'm just factoring again.
I'm saying I've got something over something, which means that I can put something in each of two different parts of two things that I multiply together.
And I can think about this as having been generated by this system, and you just add them together.
The amazing thing is that that says that, despite the fact that the response looked complicated, it was in fact the sum of two geometrics.
So it wasn't very different from the answer for a single pole.
What I've just done is amazing.
I've just taken something that, had you studied the difference equations and had you studied the block diagrams, it would have been very hard for you to conclude that something this complicated has a response that can be written as the sum of two geometrics.
By thinking about the system as a polynomial in R, it's completely trivial.
It's a simple application of the rules for polynomials that you all know.
So what we've shown, then, is that this complicated system has a way of thinking about as just two of the simpler systems.
The complicated response that grew and decayed, that's just the difference, really, 4.5 minus 3.5.
It's the weighted difference of a part that goes 0.7 to the n, then a different part that goes 0.9 to the n. So far, we've got to results, the n equals 1 case, the first-order polynomial in our case, the one pole case, that's trivial.
It's just a geometric sequence.
The response is just a geometric sequence.
If it happens to be second-order, this is second-order because when you write the operator expression, the polynomial in the bottom is second-order, second-order polynomial in R. This second-order system has a response that looks like two pieces.
Each piece looks like a piece that was from a first-order system.
And in fact, that idea generalizes.
If we have a system that can be represented by linear difference equation with constant coefficients that will always be true if the system was constructed out of the parts that we talked about, adders, gains, delays.
If the system is constructed out of adders, gains, and delays, then it will be possible to express the system in terms of one difference equation.
General form is showed here.
Y then can be constructed out of parts that are delayed versions of Y and delayed versions of X.
If you do that, then you can always write the operator that expresses the ratio between the output and the input as the ratio of two polynomials.
That will always be true.
So this, now, is the generalization step.
We did the n equals 1 case, we did the n equals 2 case, and now we're generalizing.
We will always get, for any system that can be represented by a linear difference equation with constant coefficients, we can always represent the system functional in this form.
Then just like we did in the second-order case, we can use the factor theorem to break this polynomial in the denominator into factors.
That comes from the factor theorem in algebra.
Then we can re-express that in terms of partial fractions.
And what I've just showed is that, in the general case, regardless of how many delays are in the system, if the system only has adders, gains, and delays, I can always express the answer as a sum of geometrics.
That's interesting.
That means that if I knew the bases for all of those geometric sequences, I know something about the response.
The bases are things we call poles.
If you knew all the poles, you'd know something very powerful about the system.
So every one of the factors corresponds to a pole, and by partial fractions, you'll get one response for each pole.
The response for each pole goes like pole to the n. You know the basic shape.
You don't know the constants, but you know the basic shape of the response just by knowing the poles.
We can go one more step, which makes the computation somewhat simpler.
I used the factor theorem.
Here, I'm using the fundamental theorem of algebra, which says that if I have a polynomial of order n, I have n roots.
The poles are related to the roots of the R polynomial.
The relationship is take the functional, substitute for R -- 1 over Z. Re-express the functional as a ratio of polynomials in Z.
The poles are the roots of the denominator.
So recapping, I started with a first-order system.
I showed you how to get a second-order system.
I showed that, in general, you can use the factor theorem to break down the response of a higher-order system into a sum of responses of first-order systems.
Now, I've shown that you can use the fundamental theorem of algebra to find the poles directly, and then by knowing the poles, you know each of the behaviors, monotonic divergence, monotonic convergence, or alternating signs.
And so here is this same example that I started with, worked out by thinking about what are the poles.
The poles are 0.7 and 0.9, which we see by a simple application of the fundamental theorem of algebra.
OK, we got a long way by just thinking about operators as polynomials.
We haven't done anything that you haven't done in high school.
Polynomials are very familiar and we've made an isomorphism between systems and polynomials.
OK, make sure you're all with me.
Here's a higher order system.
How many of these statements are true -- 0, 1, 2, 3, 4, or 5?
Talk to your neighbor, get an answer.
So how many are true -- 0, 1, 2, 3, 4, or 5?
Oh, come on.
Blame it on your neighbor.
You weren't talking, but I didn't hear you not talking.
How many are true -- 0, 1, 2, 3, 4, or 5?
Raise your hands
They can't all be true
They can't all be true.
Are they mutually contradictory?
Well, yeah.
5 --
N1 of the above, that sounds like-- OK, so you've eliminated 1.
Which one's true?
How many statements are true?
Looks like about 75%.
Correct?
What should I do?
How do I figure it out?
What's my first step?
What do I do?
Operators
Operators, absolutely.
So turn it into operators.
So take the difference equation, turn it into operators.
The important thing to see is that there are three Y terms.
Take them all to the same side, and I get an operator expression like that.
The ones that depend on X, there are two of them, that's represented here.
The thing this is critical for determining poles is figuring out the denominator.
The poles are going to come from this one.
After I get the ratio of two polynomials in R, I substitute 1 over Z for each R. So for this R, I get 1 over Z.
For this R squared, I get 1 over Z squared.
Then I want to turn it back into a ratio of polynomials in Z, so I have to multiply top and bottom by Z squared.
And when I do that, I get this ratio of polynomials in Z.
The poles are the roots of the denominator polynomial in Z.
The poles are minus 1/2 and plus 1/4.
So the unit sample response converges to 0.
What would be the condition that that represents?
Take the polynomial on the bottom
Something about the polynomial on the bottom.
Would all second-order systems have that property that the unit sample response would converge to 0?
[INAUDIBLE]
Louder?
Absolute value of the poles?
Absolute value of the poles has to be?
Less than 1
Less than 1.
If the magnitude of the poles is less than 1, then the response magnitude will decay with time.
So that's true here, and it would be true so long as none of the poles have a magnitude exceeding 1.
There are poles at 1/2 and 1/4.
No, that's not right.
It's -1/2 or 1/4.
There's a pole at 1/2.
No, that's not right.
There's a pole at -1/2.
There are two poles.
Yes, that's true.
None of the above.
No, that's not true.
So the answer was (2).
Everybody's comfortable?
We've done something very astonishing.
We took an arbitrary system, and we've figured out a rule that let's us break it into the sum of geometric sequences.
We can always write the response to a unit sample signal, we can always write, as a weighted sum of geometric sequences, and the number of geometric sequences in the sum is the number of poles, which is the order of the operator that operates on Y. OK, so we've done something very powerful.
There's one more thing that we have to think about, and then we have a complete picture of what's going on.
Think about when you learned polynomials.
One of the big shocks was that roots can be complex.
What would that mean?
What would it mean if we had a system whose poles were complex valued?
So first off, does such a system exist?
Well, here's one.
I just pulled that out of the air.
If I think about the functional, 1 over (1 minus R plus R squared) -- if I convert that into our ratio of polynomials in Z and then find the roots, I find that the roots have a complex part.
The roots are 1/2 plus or minus root 3 over 2j.
There's an imaginary part.
So the question is what would that mean?
Or is perhaps that system just meaningless?
Well, complex numbers work in algebra, and complex numbers work here, too.
So the fact that a pole has a complex value in the context of signals and systems simply means that the pole is complex, that the base of the geometric sequence, that base is complex.
So that means that we can still rewrite the denominator, which was 1 minus R plus R squared, we can rewrite that denominator in terms of a product of two first-order R polynomials.
The coefficients are now complex, but it still works.
The algebra still works right.
That has to work because that's just polynomials.
That's the way polynomials behave.
Now, we can still factor it.
We can still use the factor theorem.
In fact, we can still use the fundamental theorem of algebra to find the poles by the Z trick.
That's fine.
We can still use partial fractions.
All of these numbers are complex, but the math still works.
The funny thing is that it implies that the fundamental modes-- by fundamental mode, I mean the simple geometric for the case of a first-order system, more complex behaviors for higher order systems.
The mode is the time response associated with a pole.
So the modes are, in this case, complexes sequences.
So in general, the modes look like P0 to the n. Here, my modes are simply have a complex value.
So what did I say they were?
The poles were 1/2 plus or minus-- so my modes simply look that.
Same thing.
The strange thing that happened was that those modes, those geometric sequences, are now have complex values.
The first one up here, if I just look at the denominator, these coefficients mean that it's proportionate to the mode associated with this, which is that, which has a real part, which is the blue part, and the imaginary part, which is a red part.
There were two poles, plus and minus.
The other pole just flips the imaginary part.
So if I have imaginary poles, all I get is complex modes, complex geometric sequences.
An easier way of thinking about that is thinking about-- so when we had a simple, real pole, we just had P0 to the n. That's easy to visualize because we just think about each time you go from 0 to 1 to 2 to 3, it goes from 1 to P0 to P0 squared to P0 cubed.
Here, when you're multiplying complex numbers, it's easier to imagine that on the complex plane.
Think about the location of the point 1, think about the location of the point P0, think about the location of the point P0 squared, and in this particular case, where the pole was 1/2 plus or minus the square root of 3 over 2 times j, this would be pole to the 0.
This is pole to the 1.
This is pole squared, pole cubed.
As you can see when you have a complex number, the trajectory in complex space can be complicated.
In this case, it's circular.
The circular trajectory in the complex plane corresponds to the sinusoidal behavior in time.
So there's a correlation between the way you think about the modes evolving on the complex plane and the way you think about the real and imaginary parts evolving in time.
It seems a little weird that the response should be complex.
We're studying this kind of system theory primarily because we're trying to gain insight into real systems.
We want to know how things like robots work.
How does the WallFinder work?
What would it mean if the WallFinder went to position one plus the square root of 3 over 2j?
That doesn't make sense.
So there's a little bit of a strange thing going on here.
How is it that we need complex numbers to model real things?
That doesn't seem to sound right.
But the answer is that, if the difference equation had real coefficients, as they will for a real system-- if you think about a real system, like a bank account, the coefficients in the difference equation are real numbers, not complex numbers.
If you think about the WallFinder system, the coefficients in the WallFinder system, the coefficients of the difference equations-- the coefficients of the different equations describe the WallFinder behavior were all real numbers.
Here, I'm thinking about the denominator polynomial.
If we try to find the roots of a polynomial, and if we find a complex root, if the coefficients were all real, it follows that the complex conjugate of the original root is also a root.
That's pretty simple, if you think about what it means to be a polynomial.
If you think about a polynomial is whatever-- so I've got 1 plus Z plus Z squared plus blah, blah, blah.
2, 3 minus 16, if all of those coefficients real, then the only way the-- If P were a root of this polynomial, then P* would have to be a root, too, because if you complex conjugate each of the Z's, it's the same thing as complex conjugating the whole thing because the coefficients are real valued.
So the idea then is that if I happen to get a complex root for my system that can be described by real value coefficients, it must also be true that it's complex conjugate is a root.
If that happens, the two roots co-conspire so that the modes have canceling imaginary parts.
You can prove that.
I'm not worried about you being able to prove that.
I just want you to understand that if you have two roots that are complex conjugates, they can conspire to have their imaginary parts cancel, and that's exactly what happens.
Here, in this example, the example that I started with, where the system was 1 over (1 minus R plus R squared), the unit sample responses showed here.
You can write as the sum of sinusoidal and cosinusoidal signals, and the sum that falls out has the property that the imaginary parts cancel.
It's still useful to look at the imaginary parts the same as it is when you're trying to solve polynomials.
It's useful because the period of all of these signals is the same.
If I think about the period of the individual modes, if I think about the period of P0 to the n, 1/2 plus or minus root 3 over 2j to the n, the period of that signal -- I can see in the complex plane, this is the n equals 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 -- the period is 6.
If I were to take the minus 1, I would go around the circle the other way.
This would be zero term 1, 2, 3, 4, 5, 6, et cetera.
Both of the modes, the geometric sequence associated with the two complex conjugate poles, both of those modes had the same period, they're both 6, as does the response to the real part.
So you can deduce the period of the real signal by looking at the periods of the two complex signals.
So think about this system.
Here, I've got a system whose responses showed here.
I'm going to tell you that the response was generated by a second-order system, that is to say a system whose polynomial was second-order, whose polynomial in R was second-order.
Which of these statements is true?
I want to think about the pole as being a complex number.
Here, I'm showing you the complex number in polar form.
It's got a magnitude and an angle, and I'd like you to figure how what must the magnitude have been, and what must the angle have been.
So what's the utility?
Why did I tell you the pole in terms of it's magnitude and angle rather than telling it to you in it's Cartesian form as a real and imaginary part?
What's good about thinking about magnitude and angle?
Or if I break the pole into magnitude and angle, and if I think about the mode-- the modes are always of the form P0 to the n. If I think about modes as having a magnitude and an angle, when I raise it to the n, something very special happens.
What happens?
You can separate it.
What is R e to the j omega raised to the n?
That's the same as R to the n, e to the j n omega.
It's the product of a real thing times a very simple complex thing.
What's simple about the complex thing?
The magnitude is everywhere 1 e to the j, the magnitude of that term.
So all of the magnitude is here, none of the magnitude is here.
All of the angle is here, none of the angle is here.
I've separated the magnitude and the angle.
So it's very insightful to think about poles in terms of magnitude and angle because it decouples the parts of the mode.
So I can think, then, of this complicated signal that I gave you as being the product of a magnitude part and a pure angle part.
From the magnitude part, I can infer something about R. R is the ratio of the nth 1 to the n minus 1-th one, so R is a lot bigger than 1/2.
In fact, R in this case is 0.97.
And this is pure angle.
This lets me infer something about the oscillations.
In fact, I can say something about the period.
The period is, here's a peak, here's a peak, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.
The period's 12.
If the period is 12, what is omega?
Omega is a number, such that by the time I got up to 12 times it, I got up to 2 pi.
What's omega?
Pi over 6
2 pi over 12.
So it's about 1/2.
So that's a way that I can infer the form of the answer.
I ask you what pole corresponds to this behavior.
Well, the decay that comes from the real part, that comes from R to the n. The oscillation, that comes from the imaginary part, and I can figure that out by thinking about the period and thinking about that relationship.
So the answer, then, is this one, R is between 1/2 and 1, and omega is about 1/2.
OK, one last-- whoops, wrong button.
One last example.
So what we've seen is a very powerful way of decomposing systems, so that we can always think about them in terms of poles and complex geometrics, which we will call modes, poles and modes.
And that behavior works for any system in an enormous class of systems, so I want to think about one last one, which is Fibonacci sequence.
You all know the Fibonacci sequence.
You've all programmed it.
We started with that when we were doing Python, and we made some illustrations about the recursion and all that sort of thing by thinking about it.
Now, I'm going to use signals and systems do the same problem.
So Fibonacci was interested in population growth.
How many pairs of rabbits can be produced from a single pair in a year if it is suppose that every month each pair begets a new pair, from which the second month it becomes productive?
OK, it's not quite the same English I would have used.
From this statement, you can infer a difference equation.
I've written it in terms of X.
What do you think X is?
X is the input signal, and here, I'm thinking about X as something that's specifies the initial condition.
This is the thing I alluded to earlier.
One trick that we use to make it easy to think about initial conditions is that we embed them in the input.
So in this particular case, I'll think about the initial condition arising from a delta function.
If I think about X as a delta, then the sequence of results Y0, Y1, Y2, Y3 from this difference equation, is the conventional Fibonacci sequence.
It would correspond to what if you had a baby rabbit, one baby rabbit, that's the one, in generation 0, that's the delta.
So the input is a way that I can specify initial conditions, and that's a very powerful way of thinking about initial conditions.
So here's the problem.
I've got one set of baby rabbits at times 0.
They grow up.
They have baby rabbits, which grow up at the same time the parents had more baby rabbits, at which point more babies grow into bigger rabbits.
And big rabbits have more babies, et cetera, et cetera, et cetera, et cetera, et cetera.
So you all know that.
You all know about Fibonacci sequence.
It blows up very quickly.
What are the poles of the Fibonacci sequence?
The difference equation looks just like the difference equations we've looked at throughout this hour and a half.
We can do it just like we did all the other problems.
We write the difference equation in terms of R. We rewrite the system functional in terms of a ratio of two polynomials in R. We substitute R goes to 1 over Z, deduce a ratio of polynomials in Z, factor the denominator, find the roots of the denominator, and find that there are two poles.
The poles for the Fibonacci sequence are plus or minus the root of 5 over 2.
That's curious.
There's no recursion there.
It's a different way of thinking about things.
There are two poles.
The first pole, the plus one -- 1 plus the root of 5 over 1, corresponds to a pole whose magnitude is bigger than 1.
It explodes.
There it is.
The second one is a negative number.
So the first one is the golden ratio, 1.618...
The second one is the negative reciprocal of the golden ratio, which is -0.618... And those two numbers, amazingly, conspire so that their sum, horrendous as they are, is an integer.
And in fact, that's the integer that we computed here.
So we've used Fibonacci before to think about the way you structure programs, iteration, recursion, that sort of thing.
Here, by thinking about signals and systems, we can think about exactly the same problem as poles.
There's no complexity in this problem.
It doesn't take N or N squared or N log N or anything to compute.
It's closed form.
The answer is (pole one) to the N plus (pole two) to the N. That's it.
That's the answer.
So what we've done is we found a whole new way of thinking about the Fibonacci sequence in terms of poles, and more than that, we found that that way of thinking about poles works for any difference equation of this type.
And we found that poles, this way of thinking about systems in terms of polynomials, is a powerful abstraction that's exactly the same kind of PCAP abstraction that we used for Python.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I want to talk a little bit about designing control systems.
This will finish up our discussions on signals and systems.
Let me then just briefly review where we are.
Hopefully this might help you also for perspective with regard to thinking about the exam tonight.
We've looked at a bunch of different kinds of representations for discrete time systems.
The easiest, most concise method we looked at was the representation using a difference equation.
That's mathematically as concise as you can get.
But it doesn't tell you important things like who's the input and who's the output and what are all the different ways that you can get through the system from input to output?
So for that question block diagrams are nice.
Block diagrams are graphical.
It makes it very easy to see when there is, for example, a cyclic path through the network.
But they're graphic.
They're not nearly so concise as difference equations.
So then we went on to operators.
Operators are just as concise as difference equations but they contain additional information because the operators have an implicit argument.
So there's an input, which is the argument to the operator, and there's an output, which is the output of the operator.
So you can tell who is the input and who is the output.
So that's good.
That sort of combines the strengths of difference equations and block diagrams.
You end up with a concise representation that has complete information about the signal flow paths.
Furthermore, you can analyze the operators by using polynomial mathematics and that gives rise to the notion of a system functional.
And that's a very nice closure because that represents an abstraction that lets us think about a whole system as though it were just one part, one thing, one operator.
So we use that structure then, all of those representations, to try to learn about feedback.
And first off, in the block diagram it's very easy to see that any time you have feedback -- feedback so enormously powerful that we want to use it in design -- but you can see immediately from the structure of the block diagram that if you have feedback then you have cycles.
Why is that interesting?
Well, that's interesting because if you have cycles then even transient inputs can generate persistent outputs.
So that's a kind of behavior that we would like to understand.
From the nature of feedback it generates cycles.
From the nature of cycles it generates persistent responses even if there's no input.
And we saw that we could characterize those by thinking about those responses for one part at a time.
And those parts we thought of as poles and the responses to a single pole we called modes.
So we thought through a way of decomposing the response of a complicated system in terms of a number of additive components that are based on poles.
Poles are just the base of a geometric sequence.
So today then what I want to do is use that framework to think about design.
How would you optimize the design of a controller?
Looking back to where we've been, way back in Lab 4, ancient history, we looked at how you could program the robot to approach a wall.
And we saw that depending on how you set up that system you could get very different performances.
And what we'd like to do is have a way that we can design for performance without actually building it.
The kind of thing that we built in the lab, Lab 4, was so easy that building it to determine its behaviors was not a bad problem.
But In general if you were building a 777, there's more than one pole.
And you wouldn't necessarily want to test drive all of the bad configurations.
So we'd like to be able to understand that kind of a problem analytically.
We'd like to be able to analyze it.
So using the different representations you can generate a very concise representation just thinking in terms of difference equations.
You all did this in Lab 4.
And you get a single difference equation that tells you in principle everything there is that you could know, but not in a form that's very easy to analyze.
It's a bit better if you translate the difference equation into a block diagram because now you can see that this system of equations has in fact two feedback loops in it.
Two cycles.
Two things that might potentially generate persistent responses to transient signals, which could then degrade performance.
If the transient signal lasts ten years it might be a bad controller.
If the 777 hits turbulence and never stabilizes that would be bad.
If small disturbances got bigger with time that might be bad, right?
So we can see that this simple controller described by these difference equations has the potential to do that sort of stuff.
And we'd like to understand, when does it?
The easiest way to think about analyzing this is to focus first on the inner loop and ask the question, what's the functional representation for that box which we would call an accumulator?
This box, this thing, accumulates at its output, the sum of all the things that ever came in so we call it an accumulator.
So what's the functional representation of an accumulator?
Well, we just do polynomial math.
Easy so we can recognize from the block diagram that the signal Y could be constructed by applying R to W. But we can also see in the block diagram that W is the sum of X and Y.
And then if we take the left hand side and the right hand side of this double equation we get something that involves just X and Y, which we can solve for the ratio Y over X.
Which then says that the ratio is R over (1 minus R).
That's a functional representation for the effect of the accumulation.
That's also something that comes up so frequently in the design of control systems that we give it a name.
We call this Black's Equation.
And it's especially useful to avoid these little trivial steps in algebra, to just jump to the answer.
So let's see that everybody's following me.
The equation for this box is the thing that we will call Black's Equation.
It's not mysterious.
It's something that you could derive, so derive it.
Figure out the functional form for the system that goes from X to Y and figure out which of these forms is correct.
(1) through (4), or (5) if none of the above applies.
So take 30 seconds, figure out the answer.
I'm going to ask you to raise your hand.
You're free to talk to your neighbors.
OK.
So everybody who raised your hands tell me what the right answer is.
OK, wonderful.
It's about 95%.
No.
It's about 100% correct out of about 95% participation.
So the answer you can form just like we did before, no particular tricks, using simple algebra.
Simple algebra you get F over (1 minus FG).
The thing I want you to recognize is kind of a graphical way of thinking about that.
And you can just memorize F over (1 minus FG) and that's fine.
But there's some interesting things that the designer thinks about.
This functional form is F, that's the forward gain.
That's the gain through the system starting at the input and going directly to the output.
So this form says that the closed loop gain, the system function that results when the loop is closed, is just the forward gain, F, divided by (1 minus the loop gain).
The loop gain is the product around the loop once.
So that's just a convenient way of thinking about it.
Any time you have a feedback system of this type you can think about the closed loop response as being the forward path F divided by (1 minus loop gain FG).
So the answer was 1.
And generally we'll see two different ways of representing the system.
Sometimes we'll represent it with a plus as we did with the accumulator.
Many times we'll represent it with a minus.
That's the way we think about control problems.
We put in the minus because we'd like to think about the controller as trying to make something go to 0.
So when you take the difference, that gives us an error signal.
And then we can think about the controller being the thing that drives that error to 0.
But however you think about it, there's two forms that are very closely related.
They really differ by just a minus sign which you could think of as just multiplying G. So it's sort of like the right hand side is just a minus G plugged into the left hand side.
So then the way you use this idea, you think about the block diagram and you say, OK I can replace this thing with an equivalent system which is R over (1 minus R) and then repeat.
So the R over (1 minus R) means that, if you think about Black's Equation now for this loop, you should think about the forward gain, K minus T, R over (1 minus R), that's this.
Divided by (1 plus the loop gain).
So 1 plus, and the loop gain, well, the wire down here just has a gain of 1.
So the loop gain and the forward gain are the same thing.
So you get this kind of expression which simplifies to this.
There's two things I want you to see about this.
First off, I want you to see that even though the simple-minded way of plugging in said that we should have got a quotient of quotients N over D over N over D, it's simplified to a single ratio.
If you design a system out of just adders, gains and delays, that will always be true.
There's a closure.
It will always be the case that the functional that represents a system of that form will always have the property that it's a polynomial in R divided by another polynomial in R, that's just the way polynomials work.
The other thing is that you can now start to interpret what the behavior of this could represent in a simpler form than thinking about this.
So this kind of a representation that leads to intuition about what the behavior should be is very helpful when you're thinking about design.
And in particular this particular thing says that if we think about a simpler system that could generate that same response we can generate some intuition for how we would like to set the parameter k. So in particular this system functional which we generated for this system could equally apply to that system.
Is that clear?
So there's a numerator, which I've represented here.
There's a numerator, which has an R in it and has a minus kT in it.
I'm going to, for reasons that you'll see in a minute, I'm going to call something P0 because it's the pole.
So the numerator I've represented here and this denominator has this form.
And I wrote it that way because this is the canonical form for the way of thinking about a pole.
So what I showed is that even though this was a more complicated system you can think about it as the cascade of a delay, a gain, and a pole.
The pole can be calculated from the gain, the pole is the multiplier for R, so the pole is (1 plus kT).
So if I were to choose kT to be minus 0.2 then the pole would be at 0.8.
If the pole is at 0.8 then the mode, the natural response to the pole, would have the form P to the n. They always have the form geometric P to the n, so it would look like 0.8 to the n. Because of the pre-multiplier of 1 minus P0 the whole thing gets multiplied by 0.2 and because of the delay the whole thing gets shifted to the right.
The important thing is that by thinking about manipulating this as an operator we can recognize and simplify the form of the behavior.
That gives us an intuitive grasp over how to best choose kT.
It's all clear?
Now, the behaviors that we're interested in are not always unit-sample responses.
We do unit-sample responses because they're the easiest possible thing we could think of, right?
A unit-sample is the simplest non-zero signal.
A unit-sample is the signal that is different from 0 in exactly one place -- the easiest possible place, 0.
And it has its easy-as-possible non-zero value -- 1.
So we focus on the unit-sample signal because it's the easiest possible signal we could think about.
But when we're thinking about behaviors we're often thinking about other things.
Often we'll think about the step response.
Here I have illustrated the way you would measure a step response.
A step response is what would happen if the output were initially 0, if we were at rest, and suddenly we turned on a signal that was constant at 1.
That would happen in the robot case if we started the robot close to the wall, at rest, near 0 -- so that the output signal is close to 0, but the desired input was a meter behind.
Then that would be an input signal that started at time equals 0 equal to 1 and persisted forever at 1.
And the result then would be what we call the step response.
The step response is typically easier to measure in the lab than is the unit-sample response.
So we use the unit-sample response when we're thinking analytically, when we're doing calculations, and we use the step response when we're in the lab trying to measure something.
And the whole theory wouldn't be very useful if there weren't a close relationship between those two things.
This diagram illustrates the relationship.
If we think about a system H for which we would like to find the step response, the step response to that system is what would the system do if you put the unit-step into the system.
I've represented the unit-step here as u[n].
u[n], the signal that is 0 for n less than 0, and 1 for n bigger than or equal to 0 -- is just the accumulation of the delta function, the unit-sample.
So this system, the cascade of an accumulator with H, would measure the step response of H if it were driven with the sample signal.
Because of the properties of polynomials and because block diagrams follow the rules for polynomials, we can flip these whenever the systems both start at rest, and if we flip those we get a different interpretation.
What this says is that if you were to take H and stimulate it with the unit-sample you would get h, little h, which we would call the unit-sample response because it's the response to the system when the input is the unit-sample.
So if you measured h with the unit-sample rather then with the unit-step you would get the unit-sample response from which you could generate the step response by running it through an accumulator.
So what that says is there's a close association, there's a close relationship, between the unit-sample response and the unit-step response.
One is the accumulation of the other.
The unit-step response is the accumulated response to the unit-sample response.
So that means that in that previous example where we saw that setting kT equal to minus 0.2 resulted in this unit-sample response, that would correspond to this unit-step response.
All you do is for every sample you calculate, for this response you calculate the sum of say, n equals 0, you would take the sum of all of the previous answers in H[n].
It's the accumulation.
So it starts at 0 since the sum of all those numbers is 0.
Then at time 1 it becomes the sum from here back so it becomes 0.2.
Then here it's the sum from here back.
And if you add these all up it becomes a number that approaches 1.
Not surprisingly, right?
If you've got a feedback system and if you started the robot up against the wall and the desired position was one meter behind you would monotonically approach 1, OK?
And what you can see is that if you change the value of the pole, here I've changed the kT from minus 0.2 which is what the previous answer was, to minus 0.8, I've changed the value of the pole, the unit-sample response got faster.
And the unit-step response also got faster.
The point is there are different kinds of performance metrics that we might want to use, unit-sample response, unit-step response, but the responses of all of them, you can tell something about the response to all of them from the response of the unit-sample signal.
That's why we focus so much on the unit-sample signal.
It's not because it's the most popular thing to use in the lab.
It's because it's the easiest thing to calculate with and it gives us insight into things that we would like to measure in the lab.
So for this very simple system what you can show is that there's only a few possible behaviors, a few categories of behaviors.
If you were to choose kT to be between 0 and 1, then the pole, which is (1 plus kT) would also be between 0 and 1.
Since the system has a single pole you can say a lot about the response from the numerical value of the pole.
If the pole is between 0 and 1 then the response is going to be monotonic and converging.
That results because the unit-sample response was positive only and decayed towards 0.
Because it's positive only it means monotonic -- goes to 0 and makes it converge.
So you can infer properties about the unit-step's response from properties of the pole just like we could infer properties of the unit-sample response.
If you changed kT to be between minus 2 and 1 you get a P0 that's between minus 1 and 0.
Again that's just that equation.
That says that the response will be alternating.
So the sign of the unit-sample response goes positive then negative.
It still converges in the sense that the unit sample response approaches 0.
And what that means for the unit step-response is that the unit-step response will converge toward 1.
The sign doesn't alternate around 0, the sign alternates around 1.
So again, you can infer the properties of the unit-step response from the properties of the unit-sample response.
And if you have kT that is less than minus 2 then you get a P0 that's less than minus 1 and that's a divergent response.
So the point is you can infer properties about the control system by thinking about the poles of the system where here I've illustrated it for a simple system that only has one pole.
OK.
I told you a bunch of facts.
Now you figure out something.
How would I choose k for this system to get the quote, "best performance?"
So which value of kT would give the fastest convergence for the unit-sample signal?
OK, participation is down but the hit rate is still good.
Virtually everybody who volunteered to answer got the right answer.
The most popular answer was (2).
Why is the answer (2)?
What's the range of possibilities that we could get?
If we choose k, or kT, we could choose kT to be-- what's the range of kT that we could use?
Minus 2 to 0?
[INAUDIBLE] we could use kT, any real number, right?
We couldn't use imaginary numbers because that doesn't sort of make sense for a real system.
But we could choose any real number.
The real numbers map, according to this chart, the real numbers map to a different real number.
If you choose kT you can figure out where is the pole by that mapping.
Where would you put the pole to get the fastest response?
If you have your choice of putting the pole anywhere on this red line, that red line or that red line, where would you put it and why?
Just inside the [INAUDIBLE]
Putting it inside the unit circle would probably be a good idea because?
[UNINTELLIGIBLE]
Yeah.
If you didn't put it inside the unit circle it wouldn't converge.
That's right.
So you like [UNINTELLIGIBLE] the inside.
Given the choice of anywhere here and anywhere here which would you choose?
How would you choose it?
Yeah
Derive
Derive.
And how do you get that?
[UNINTELLIGIBLE]
What would happen if it was close to [UNINTELLIGIBLE]?
It converges quite quickly, right?
Poles always converge geometrically.
The base of the geometric is the pole value.
So you'd like the pole to be as small as possible to get the convergence as fast as possible.
That make sense to everybody?
So in particular for this example if you chose kT to be minus 1 in that limit then this entire factor goes away.
So the entire response degenerates to R and R is not instantaneous but it's pretty fast.
What that says is that you get to the final value in one step.
So if the input consisted of a unit sample, which has non-zero value only at 0, the output would have non-zero value only at 1, right?
Thinking about the way that works in practice, think about the robot and think about we're trying to drive toward the wall.
If we made kT be minus one, and just for the sake of being concrete let me say that T is about 1/10.
That's what the sampling period is for the robots we use in the lab.
If T were 1/10 then the best k would be minus 10.
And what that says is that if we were 1 meter away from where we want to be we would set the velocity to 10 meters per second.
What that says is that if we started 1 meter away from where we want to be, so this is intended to represent position on the same axis that this has showed, so if we started here and we wanted to be here, time is plotted down.
If we use the rule that we just specified then we would set the velocity given this condition which is 1 meter away from where we want to be.
We would set the velocity to be 10.
If we set the velocity to be 10 then after 1 unit of time, after 1/10 of a second, we are 1 meter to the right, which just happens to be exactly where we want to be.
Had we chosen k to be bigger we would have overshot.
Had we chosen k to be smaller we would have undershot.
k equals 10 gave us precisely the right answer so that we get there in one fell swoop.
Then on the very next step we would compute a velocity of 0 because we are at where we want to be so we would stay there.
And that condition would persist forever.
The idea would be this simple system provides a way that we could set the gain so we could get to where we want to be in one step.
It's hard to beat that.
The problem that results and the reason you didn't see that good behavior in the lab was that the sensors in the robot don't work instantaneously.
They introduce delay.
And as an idealization of that delay I want to think through the same problem.
But now let's say that the sensor delays the input to the sensor which is the output of the system.
Let's say that the sensor introduces a delay of 1, so now instead of reporting d sensed, which was d0[n], it reports d0[n minus 1].
So now what would happen?
Now with the delay, if I started here and if by some mysterious process I was here at time n equals 1, then I would calculate my new velocity.
What would be my new velocity here?
I'm right where I want to be.
What would be my new velocity if I assume that the sensor has a delay of [UNINTELLIGIBLE]?
10
10.
Because of the delay the sensor is reporting that I'm a meter away from where I want to be.
So the controller calculates, oh, I need to go forward a meter.
I'll set the velocity to 10.
So having set the velocity to 10 and then one step goes by, now we're completely on the wrong side.
That's what happens when you put delay into the system.
So because we're basing this decision on where we were last time we go to the wrong place.
So now we're here.
What will the controller say next?
[UNINTELLIGIBLE]
Stay here.
You're in a great place.
I'm really 1 meter too close.
In fact I banged into the wall.
But the sensor is telling me I'm exactly where I want to be so stay here.
Say I didn't kill myself, what will the velocity next be?
Minus 10.
So now I tell myself to go back.
That's probably a good move.
But now I still think I'm too close to the wall so I tell myself to continue to back up.
The idea is that I get poor performance.
The delay had a devastating effect on the way that the controller worked.
Even though it's a tiny change to the way the system works it has a devastating effect on behavior.
We'd like to be able to predict that without having to measure it.
Here's the same equations except that I put a delay in the sensor.
Here is the same block diagram but I've represented a delay in the sensor path.
So now the question is, what's the new functional representation for that control system?
So what's the answer?
Can you rate the functional form for this system as one of (1), (2), (3), or (4), or is it none of the above?
About 1/3 participation and about 100% correct.
The answer's four.
You get to use Black's Equation or however you'd like to think about that.
You can think about reducing the inner loop the same as we did before and then think about this as forward over (1 plus loop gain) but now the loop gain has R squared in it instead of R. We get this form.
How does this form differ from when the R wasn't here?
What's the difference between R not there and R is there?
What's the answer?
Anyone?
[UNINTELLIGIBLE]
The squared term.
So this term in the previous form was just an R and in this form is a square and so what's that do?
What's the importance of the fact that there's a square there?
Two poles, right?
We now have a polynomial in the denominator that is quadratic in R. And what that's going to do is it's going to give us two poles instead of one.
The importance of that is that now we're going to have to think through-- we previously categorized what were all the behaviors you could get from one pole.
The behaviors you can get from one pole were monotonic divergence, non-monotonic alternating divergence, monotonic convergence, alternating convergence.
So there were four behaviors that were possible with one pole.
Now we have to think through what are all the possible behaviors that we could get with two poles.
Different problem.
Hopefully they're related.
So here, the way we would find out what the poles are is take this expression, substitute for every R, 1 over z, turn the ratio of polynomials in R into a ratio of polynomials in z.
To do that in this case I had to multiply numerator and denominator by z squared.
Having done that I get a second order polynomial in z in the bottom so there's two poles which are the roots of that polynomial.
And that's just a quadratic equation.
The interesting thing now is to map out what are all the possible behaviors that that system can give us.
It's important to realize that's a simple generalization of what we saw before.
It will be the case that any system that we construct out of adders, gains and delays will have the property that we can write the system functional as a ratio of polynomials in R. By the factor theorem we will always be able to factor the denominator.
And by the notion of partial fractions we'll always be able to write some complicated expression like that in terms of a sum of parts.
Each part being first order.
The intuition we get from this is that what we ought to do is factor the denominator, find the poles, and associate a behavior with each of those poles.
Here's what the problem looks like for the two pole problem.
If we have the general form given here and if we start by thinking about kT having a small magnitude, if kT has a small magnitude then we have 1/2 plus or minus the square root of 1/2 squared.
So that's 1/2 plus or minus 1/2.
That's 0 or 1.
So the poles for this system, if you make k be very small, the poles are at 0, near 0 and near 1.
Is that a good system response or a bad system response?
Bad.
Why?
Well, we're trying to think through the behavior of the second order system by thinking about the separate behaviors of each of the poles.
Is this a good pole or a bad pole?
Why?
The response is always pole [UNINTELLIGIBLE].
The mode associated with the response at a pole near one is something near one to the end.
That never converges.
If you start with some error the error persists forever.
Well, that's not good.
If wind turbulence knocks you into a decline in your airplane and it persists forever, that's not good.
You would like those things to damp out.
So this pole is bad.
How about that pole?
That one has a response that decays quickly.
But the problem is that when you add the two pieces together, that was the reason I showed you this decomposition, you can think about the polynomial being factored and being broken into a number of parts.
The part that's associated with the pole near one has a response that goes for a long time.
So that will asymptotically dominate your response.
So we refer to this as a dominant pole.
This pole dominates the response.
That's a way of inferring the behavior of two poles from the sum of single poles.
In this particular case there's one pole that matters more than the other one.
So we call that pole the dominant pole.
If you were to make kT more negative, So here's the general form.
If you make kT negative, you can make the thing under the radical sign go towards 0.
If you made the thing under the radical sign go to 0 then you would get two poles at 1/2.
So here we would see that if kT were minus 1/4, if kT were minus 1/4 we would have 1/2 squared, which is plus 1/4.
Minus 1/4 would give us 0 under the radical.
So we would get two poles at 1/2.
Is that good or bad?
Well, it's better than the previous example, right?
Because each of those poles is associated with the response where the error gets half what it used to be on every step.
So it converges.
If going from 0 to minus 1/4 is good, then going to minus 1/2 might be better, right?
If you continue that trend, say you make kT be minus 1, if kT is minus 1 then you get 1/2 squared minus one.
So 1/2 squared is 1/4, minus 1 would be minus 3/4.
That gives us a complex pole here.
So we get two poles that are right on the unit circle.
What's that mean?
That means oscillations.
Oscillations is something you can't get with one pole with a real system.
Oscillations result from a poll that has an imaginary component.
If the system is real you could only get such poles in pairs.
So it's this pair that makes sense for a real valued system.
And that gives rise to oscillations and that's exactly what we saw here.
So we can associate the oscillations that we saw in the simulated lab experiment with poles that have imaginary components.
So what would be the period of the oscillation in the system given by 1/2 plus j root 3 over 2?
[INAUDIBLE]
Excuse me?
Excuse me?
Where did the root three come from?
The previous page.
If you substitute minus 1.
So what's the period of the oscillation?
So the period's represented by 5 converged into 6.
So how do you get 6?
The easiest way to think about that is to think about the poles being expressed in polar notation.
The poles we previously said were 1/2 plus or minus j root 3 over 2.
That's the same as e plus or minus j pi over 3.
It's easier to use that form because if you take that form, so if you think about e to the j, what was it, 2 pi over 3?
Pi over 3.
So if you think about that form, that's the pole, we can write that that way.
Then the inside has a magnitude of 1.
So we can think about that just being a magnitude of 1 and an angle of pi over 3.
So when you raise that to the n, the magnitude to the n, one to the n is always 1, and the angle raised to the n, it just increases linearly with n. So the angle goes from pi over 3 to 2pi over 3 to pi to 4pi over 3, et cetera.
So you can think about this going from pi over 3, 2pi over 3, pi, 4, 5, 6.
It takes n equals 6 to get around to where it started so the period is 6.
If you were to further change the game, if you were to make it even more negative, the poles would go outside the unit circle.
And then what would happen?
[UNINTELLIGIBLE]
Right.
So that's completely unacceptable.
The point is that by changing the gain you can get any behavior on this figure which is called the root locus.
So root meaning the root of a polynomial.
Locus meaning the acceptable values of points.
So the root locus shows you all the possible behaviors they could result from this system.
So given that root locus, how would you choose k to make your system response as fast as you could?
So what value of kT would you want?
Everyone raise your hands.
That's very good.
So the most popular answer is number (2).
So why would the answer be number (2)?
What do you look at?
Yeah?
[INAUDIBLE]
Remember what we're trying to do.
We're trying to infer properties of the behavior of this second order system from the pole locations.
We know that there's an expansion that lets us expand the system in terms of the sum of two first order responses.
The slowest of the first order responses will dominate eventually.
So what we need to look at is the slowest of the two responses.
We would like to know of the two poles which one is the slowest?
The slowest is the one that's closest to the unit circle.
So we would get the fastest response when the slowest one is as fast as possible.
As the poles initially go toward each other from 0 to 1 this one is getting faster, this one is getting slower.
So the slowest one is this one.
So the slowest one is fastest when they meet.
And then when they diverge does the slowest one get faster or slower?
It's already slower because it gets closer to the unit circle.
So you get the fastest response whenever you get the two poles both colliding at 1/2 and that was the case that happened when kT was minus 1/4 from two or three slides ago.
So the idea then is to try to infer what would be the behavior of this higher order system by thinking about the behaviors of the individual components, here the poles.
And what we saw was something that's in fact a very important general trend.
What we saw was that we first analyzed the wall finder system assuming there was no delay in the sensor.
And we found that that system was characterized by a single pole and we had the design freedom of putting that pole anywhere we wanted to on the real axis.
And that allowed us to choose the pole to be at 0 which gave terrific performance.
The interesting thing that happened when you add just one more pole by putting a delay in the sensor, you make the system more complicated and now you can't possibly get nearly so good behavior.
The behavior is a lot worse than it was before.
And in fact, if you were to do the same kind of analysis by putting yet another delay in the sensor you would find even worse behavior.
The idea then, the generalization of the way the behaviors is working, generally speaking adding delays inside a feedback loop is a destabilizing thing.
Generally as the number of delays increases you end up having to back off on the maximum gain that you can use because the system becomes less stable.
So the overall moral is that delays are bad generally.
I mean, you could concoct some kind of a weird scheme where that wouldn't be true.
But it's actually hard to concoct such a weird scheme.
In general, and in virtually every physical system that you'll run into, adding delays makes the system harder to stabilize.
And that's the big message.
And the system that we looked at in the lab, the wall finder was actually quite hard because the number of delays was large.
If you try to track where delays can enter the robot system they get in at very many different places.
In the physical sensor, in the microprocessor, in the conversion from analog to digital, there's a number of delays in that system.
And that's why it becomes hard to stabilize.
OK.
So that's the main content for today.
What I want to do is give you one more practice question.
The big problem that I want you to think about from today is how do you characterize performance?
When we had a single pole performance was easy to talk about because performance was diverging monotonically, diverging alternating, converging monotonically, converging alternating.
There were four kinds of behaviors.
When we went to second order we saw some new behaviors.
It could become oscillatory.
What I'd like you to do now is think not just about those properties but many other properties.
So here's some questions.
Think about the system on the top and I'd like you to infer properties about that system.
In particular, does this system have three poles?
Is the unit sample response, is there a way to write that as the sum of three geometric sequences?
What's the unit sample response?
And is one of the poles that z equals 1?
So think about the system, think about five ways of characterizing it and tell me how many of those five characterizations is correct.
So how many of the properties are true?
[UNINTELLIGIBLE] Probably 2/3 correct?
How many poles?
How do you get three?
Where are the poles?
[UNINTELLIGIBLE]
How do I find poles?
What do I do?
Yes?
You use Black's Equation [UNINTELLIGIBLE] the top you can express [UNINTELLIGIBLE] so use Black's Equation to express the system function as R cubed over (1 minus R-cubed) then--
That's right
--the denominator as an order of 3 and you combine the 3s
So a little more formally, we would take this thing and we would rewrite that with R goes to 1 over z.
So we get 1 over z cubed, 1 minus (1 over z cubed), which is then, clearing the z cubes we would get 1 over (z cubed minus 1).
How many poles?
Three.
What are the poles of z cubed minus 1?
Three poles in what?
0 [UNINTELLIGIBLE]
And so let's vote.
Let's take a vote.
There's two poles at z equals 1.
Yes?
No?
[UNINTELLIGIBLE]
Why not?
[UNINTELLIGIBLE]
So there's two poles.
I made a [UNINTELLIGIBLE] plane.
Where's the poles?
Well, you could factor it, right?
If you factored it you'd find that there is a pole at 1, right?
But then there's two more poles like that.
So the poles are the three roots of 1, which can be written like 1 e to the j, 2pi over 3, and e to the j minus 2pi over 3.
Which pole was the dominant pole?
[UNINTELLIGIBLE]
OK, bad question.
What's a better question?
How many dominant poles are there?
How many dominant poles are there?
That's a much better question, yes.
There's sort of three poles that are equally dominant, right?
They all have the same magnitude.
Why do we talk about dominant poles?
What are dominant poles good for?
If I told you that I had a pole at 3 and a pole at minus 1, which one's the dominant pole?
3
Why?
Greater magnitude
Greater magnitude.
Why do we care?
We don't care, right?
What's good about the dominant pole?
Well, we can write this response as something that looks like three to the n plus minus 1 to the n. If you let n get big enough the only one that matters is 3 to the n. So if all you care about is exactly how the plane was flying the instant before it hit the ground then you would only need to worry about long time.
And if you only worry about long time you only need to worry about the pole that's worst behaved.
That's where the concept comes from.
So none of these poles are particularly worse behaved than the others.
What's the unit-sample response associated with that pole?
We have a name for that [INAUDIBLE] right?
It's huge
It's [UNINTELLIGIBLE].
What's the unit-sample response associated with this pole?
Well, it's got a complex value right?
So the unit-sample response associated with that pole is e to the j 2 pi over 3 n. That's a complex number.
That's 1 at time 0 and e to the j 2 pi over 3n at time 1 and e to the j 4 pi over 3 at time two.
So it goes from here at 0 to here at 1 to here at 2,3, 4, 5, 6, 7, 8.
What's the period of this pole?
What's the period of the unit-sample responses associated with that pole?
3.
Because it takes 3 to get around to where you started again.
What's the period of this pole?
3.
You just spin around backward.
What's the period of the response associated with that pole?
Bad question.
All right, what's a better question?
Is there a period associated with it?
You could say that period 1 [UNINTELLIGIBLE] definition of period.
What's the period of this pole?
Dumb question, right?
Period implies repeat.
If the response repeats itself after some time then we would say the response is periodic.
Neither of those poles, well, the minus 1 is.
Is the minus 1 pole periodic?
Yes.
What's the difference between periodic and alternation?
Does a [UNINTELLIGIBLE] alternate?
Yes
Does it oscillate?
No
Bad question.
Alternate is a word that we invented for one pole.
Because the response alternated in sine.
The unit-sample response in one pole, where the pole is a negative number, alternated in sine.
So we gave that a name.
Alternation is not necessarily something that we would like to associate with a higher order system.
Periodic is perfectly reasonable to talk about for a higher order system.
Periodic merely means that if y of n were a periodic signal that I could express y of n plus n as y of n. That would be periodic.
If the thing repeats itself we would say it's periodic.
So the point of going over this stuff is just to give you some exercise in thinking about how to think about properties of systems.
We develop properties initially thinking about one pole.
Those properties were easy.
Converging, diverging, monotonic, alternating.
When we try to think about corresponding properties of higher order systems we can't simply map the simple properties of first order systems into the other.
We have to think about more complicated things.
Then we think about things like dominant.
If one of the poles has a bigger magnitude than the other then for large times we can ignore the smaller one.
What happens for short time?
Does this response monotonically increase with time for all time?
No.
Since the response associated with minus 1 alternates in sine, for short times, for times with n close to 0, that can be just as important as this one.
So the dominant pole idea tells you how things work when you have large times.
It doesn't necessarily tell you how things work when you have small times.
How about, the unit sample response is the sum of three geometrics.
Yes or no?
What are the three geometrics?
And the answer to that's yes.
That's very important.
The three geometrics over here are this pole to the n plus this pole to the n plus something that goes with this other pole to the n. Now it's a weighted sum but the weights are not necessarily 0.
The slide that I showed you for the partial fraction decomposition, you can always write a higher order system as a sum of first order factors.
That's the partial fraction expansion.
That doesn't mean the weights are all unity.
Number (2), can you write the unit-sample response as the sum of three geometric signals?
Yes.
There it is.
And if you're really good at complex math you could find out a, b and c. And that would tell you the unit-sample response and that would tell you the answer to (3) and (4).
Is the unit-sample response 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1?
Or 1, 0, 0, 0, 1, 1 -- whatever.
Is it one of those two or something different?
And how do you figure it out?
How do you figure out the unit?
Is the unit-sample response-- Is number (2) correct?
Is the unit-sample response 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1-- Yes?
How do you know that?
You could solve that equation.
Is there an easier way?
Yeah?
I wrote it as a difference equation
Just write the difference equation.
Exactly.
So even though I bad-mouth difference equations a lot, here it's easy.
In fact, you can see it in the network.
Thinking about the difference equation would be easy.
Thinking about the network would be easy.
If we think about the unit-sample response of this thing started at rest, rest means this is 0, this is 0, and this is 0 initially.
Unit-sample response means this becomes 1 at time 0.
At time 0 this is 1, this is 0, this is 0, this is 0.
So if I think about what's the time response look like and I did plus, this is 0, 1, 0, 0, 0.
The first answer is 0.
Clock ticks.
What happens?
This is 1.
This becomes 1.
This doesn't change.
That doesn't change.
This goes to 0.
0 comes around here.
That goes to 0.
So that's the answer at time 1.
What happens at time 2?
Just keep working it.
The clock ticks, this goes to 1, this goes to 0, these stay 0, this stays 0.
That's the answer at time equals 2.
Now the clock ticks.
Now this comes over to here, that means it comes back here.
This is still 0.
That comes to 1.
That's the answer for time 3.
Now the clock ticks.
And you can see the whole thing would just repeat itself now.
Is the response periodic?
Yes.
The response is periodic.
What's the period?
3.
And I can see [INAUDIBLE] if the [INAUDIBLE] there it's going to have be related to the period over here.
These periods are not same.
This period is 3, this period is 3, this period is 1, if you want to call that a period.
But they are related.
OK.
The point of this exercise is to illustrate two things.
We inferred properties of first order systems by looking at a single pole which for a real system could only behave in one of four different ways.
Second order system introduced new behaviors.
Now we can oscillate, which we couldn't do before.
Having got to oscillation, oscillation came about because of complex numbers.
If you go to higher order systems nothing new happens in algebra.
There's no such thing as meta-complex numbers, right?
Complex is as bad as it gets.
So you can have complex numbers.
The higher order behaviors can still have complex numbers but you have to think when we ask you, what's the property of a higher order system.
You can think about it in terms of the individual parts but it requires some thinking.
OK. Good luck tonight.
See you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today I want to start a new topic, circuits.
No, that's good.
That's good.
Circuits are good
Yay
Thank you, thank you.
Much better.
So just to provide some perspective, I want to remind you where we are, how we got here, and where we're going.
So at the beginning of the course, we promised that there were several intellectual themes that we would talk about.
Probably the most important one there is designing complex systems.
That's what we're really about.
We would like you to be able to make very complicated systems.
How do you think about parts?
How do you think about connecting them?
How do you think about things when you want to make something that's very complicated?
Part of that is modeling.
We just finished a module on modeling.
So in order to make a complex system, we'd like to be able to predict how it will behave before we completely build it.
Sometimes it's impossible to build the entire system before.
Sometimes it's impossible to build prototypes.
Sometimes you're stuck with going with the design at launch, and figuring out how it works.
In those cases in specific, it's very important to be able to model it, to have some confidence that the thing's going to work.
We're going to talk about augmenting physical systems with computation.
That's the module we're about to begin.
And we'll conclude by talking about how to build systems that are robust to change.
So we started with the idea of, how do you make complicated systems?
And we introduced this notion of primitives, combination, abstraction, and pattern in terms of software engineering.
We did that because that's the simplest possible way of getting started.
It provided a very good illustration of PCAP at the low level by thinking about Python, by thinking about the primitive structures that Python gives you, how those can be combined, how you can abstract, how you can recognize patterns.
But then we also built a higher level abstraction, which was the state machine idea.
There the idea was you didn't have to have state machines in order to build the brain for a robot, but it actually turns out to be easy if you do because there's a modularity there.
You can figure out if each part works independent of the other parts.
And then you can be pretty sure when you put it together the whole thing's going to work.
So that was kind of our introduction to this notion of PCAP.
Then we went on to think about signals and systems.
And that was kind of our introduction to modeling.
How do you make a model of something that predicts behavior?
So we transitioned from thinking about, how do you structure a design to how do you think about behavior.
Today we're going to start to think about circuits.
Circuits are really going to a more primitive physical layer.
How do you think about actually making a device?
So the device that we'll think about is a thing to augment the capabilities -- the sensor capabilities of the robot.
We'll think about making a light tracking system.
And the idea is going to be that you'll build a head.
The head has a neck, so you'll have to control the neck.
The head has eyes.
It will mount on top of the robot, and that'll let you drive the robot around looking for light.
And you'll do that by designing a circuit.
So that's kind of the game plan for the next three weeks.
So today, what I want to do is introduce the notion of circuits, introduce the theory for how we think about circuits.
And then in the two labs this week, the idea will be to become familiar with the practice of circuits.
How do you actually build something?
How do you actually make something work?
So today's the theory.
So the theory, the idea in circuits is to think about a physical system as the interconnection of parts and rules that connect them.
In fact, the rules fall into two categories.
We're going to think about the currents that go through parts and the voltage that develops across parts.
And we'll see that there's a way of thinking about the behavior of the entire circuit that integrates all of those three pieces.
How does the part work?
How do the currents that go through the part work?
And how do the voltages that are produced across the part -- how do they work?
So I'll just start with two very simple examples.
The first is the most trivial example you can think about.
You can think about a flashlight as a circuit.
You close the switch, current flows.
Very simple idea.
We will make a model of that that looks like this.
We'll think about the battery being a source.
In this case, a voltage source.
We'll think about the light bulb being a resistor.
We'll have two parts.
We'll have to know the current-voltage relationships for both of the parts.
And we'll have to know the ramifications for those currents and voltages when you put them together.
Very, very simple.
The other simple example that I want to illustrate is this idea of a leaky tank.
Here the idea that I want to get across is that the circuit idea is quite general.
When we talk about circuits, we almost always talk about electronic circuits.
But the theory is by no means limited to electronics.
So for example, if we think about a leaky tank, we think about a pipe spewing water into a reservoir.
Maybe that's the Cambridge Reservoir.
Maybe that's the water coming out of the Woburn Reservoir.
Maybe that's the demand put on by the Cambridge people trying to take showers in the morning -- I don't know.
So we think about flow into a tank, a reservoir, and flow out.
And we can make a model for that in terms of a circuit.
In the circuit there are through variables and across variables.
In an electronic circuit, the through variable is current.
Here the through variable is the flow rate.
So it's the flow of water in and the flow of water out, represented here by these things that look like currents.
And the across variable for an electronic device is voltage.
The across variable for this kind of a fluidic device, the across variable is pressure.
So we think as this thing gets ahead of that thing, as there's more stuff coming in than going out, the height goes up and the pressure builds.
Same idea.
So the point is that we'll develop the theory for circuits in terms of electronics.
But you should keep in the back of your mind that it's a lot more general idea than just electronics.
In fact, there are two completely distinct reasons why we even bother with circuits.
One is that they're very important to physical systems.
If you're designing a power network, of course you have to think about the way the power network, the power grid works as a circuit.
That's obvious.
In electronics, of course, if you're going to build a cell phone, you have to know how the parts interconnect electronically.
That's obvious.
But probably the biggest use for circuits these days is not those applications, although those are very important.
Circuits are also used as models of things.
So many models for complex behaviors are in fact circuit models.
So in terms of electronics, the idea is that we want to get on top of electronics.
We want to understand how circuits work, so we can understand things like that.
If you look at how complex processors have got over my professional life, we start with my professional life down at about 1,000 transistors per processor.
And today, we're up at about a billion.
That's enormous.
Even in the stone ages when we were designing things that had a thousand parts, we still had trouble thinking about those thousands parts all at once.
We still need PCAP.
We still needed ways of combining the activities of many things into a conceptual unit that was bigger.
Here, it's just impossible if you don't have that.
So that's one of the reasons we study circuits.
And the other reason is here.
So here I'm showing a model for the way a nerve cell works.
This model is taken from 6.021.
The idea, this comes from the study of the Hodgkin-Huxley model for neural conduction, arguably the most successful mathematical theory in biophysics.
Which explains the completely non-trivial relationship between how the parts from biology works and the behavior in terms of propagated action potentials works.
So the idea is that we understand how this biological system works because we think about it in terms of a circuit.
That's the only successful way we have to think about that.
So what I want to do then is spend today figuring out circuits At the very most primitive level.
The level that I'm going to talk about in terms of circuits is roughly analogous to the level that we talked about with Python when we were thinking about how Python provides utilities for primitives, combinations, abstractions, and patterns.
So I'm going to start at the very lowest level and think about, what are the basic primitives, the smallest units we'll ever think about in terms of circuits?
And what are the rules by which we combine them?
So I'll start with the very simplest ideas, the very simplest elements.
We will oversimplify things and think about the very simplest kind of electronic elements as resistors that obey Ohm's Law, V equals iR.
Voltage sources, things that maintain a constant voltage regardless of what you do.
And current sources, things that maintain a constant current regardless of what you do.
These things are, as I said, analogous to the primitive things that we looked at in Python.
They're also analogous to the primitive things that we looked at in system functions.
Can somebody think of, when we were doing difference equations, what were the primitives that we started with -- when we started to study difference equations?
What's the most primitive elements that we thought about?
Delay
Delay, yeah.
So we thought about things like-- so we had delay.
Anything else?
Gain
Gain.
Anything else?
Add.
So we had exactly three primitives.
And we got pretty far with those three primitives.
We learned the rules for interconnection.
We didn't really make a big deal out of it.
We didn't formalize it, but the rules for interconnection were something like every node has to have exactly one generator.
You can't connect the output of this to this, that's illegal.
Every node has to have one source.
And every node can source lots of inputs.
That was kind of the rules of the interconnect.
The interconnects here will be a little bit more complicated.
So those are the elements that we'll think about.
And the first step's going to be to think about, how do they interconnect?
The simplest possible interconnections are trivial.
In the case of the battery, you hook up the voltage source to the resistor.
The voltage source makes the voltage across this resistor 1 Volt.
If we say the resistor is 1 Ohm, then there's 1 Amp current period.
Done.
Easy.
Similarly, if we were to hook up the resistor to a current source, we would get something equally easy.
Except now the current source would guarantee that the current through the resistor is an amp.
Therefore, the voltage across the resistor, by Ohm's law, would be a Volt.
So we would end up with the same solution for a completely different reason.
Here the voltage is constrained.
Here the current's constrained.
Just to make sure everybody's with me, figure out, what's the current i that goes through this resistor?
Slightly more complicated system.
Take 20 seconds, talk to your neighbor, figure out a number between (1) and (5).
OK, so what's the answer?
Everybody raise your hand with a number (1) through (5).
Come on, everybody vote.
Come on.
You can blame it on your neighbor, that's the rules.
You talk to your neighbor, then you can blame dumb answers on your neighbor.
OK, about 80% correct I'd say.
So how do you think about this?
What's going to be the current?
How would you calculate the current?
What do I do first?
Shout.
If you shout, and especially if my head's turned away I don't know who you are
Kirchhoff's law
Kirchoff's law, wonderful.
Which one?
There's two of them
[UNINTELLIGIBLE]
[UNINTELLIGIBLE].
What loop do you want to use?
Left
Left side.
So if we use the left side loop, we would conclude that there's a volt across the resistor.
So the current would be?
1 Amp
An amp.
Where's the current come from?
The voltage
The voltage source just like before.
So not quite.
So the voltage source establishes this voltage would be 1.
That makes this current be 1.
That would be consistent with the current coming out of here, except we have to also think about that 1 Amp source.
So the question is, what's does the 1 Amp source do?
Nothing?
It's just there sort of for decoration or for [UNINTELLIGIBLE] so that we can make an interesting question to ask in lecture?
Maybe.
So where's the current?
Where's the 1 Amp that goes through the resistor come from?
[UNINTELLIGIBLE] on the right
It comes from the right.
It comes from the current over here.
So the idea is that if this current, ignore the voltage over here for the moment.
If this current flowed through the resistor, then you'd have 1 Amp going through there, and you'd have 1 Volt generated by that current which just happens to be exactly the right voltage to match the voltage from the voltage source.
So if you think about this, the voltage guarantees that this is 1 Volt, but so does the current.
In order to simultaneously satisfy everything, all you need to do is have all of this current go around and come down through that resistor.
That will generate the volt, so there's no propensity for more current to flow out of the source because the source is 1 Volt and it's facing a circuit that's already 1 Volt.
So the idea was to try to give you something that's relatively simple that you can think through on your own, but not trivial.
So the answer was 1 Amp.
But the 1 Amp was not for the trivial reason.
The 1 Amp is because the current from the right flows through the resistor and makes the voltage be 1.
So the right answer is 1.
But for the reason that you might not have originally thought.
But more importantly, I wanted to use that as a motivation for thinking about, how do we think about bigger circuits?
So when the simple circuit, like two parts, it's no problem figuring out what the answer's going to be.
But when the circuit has even three parts, it may require more thinking.
And you may want to have a more structured way of thinking about the solution.
Yes?
What would have occurred if the current provider on the right side was 2 Amps?
Great question.
Had this been 2 Amps, you can't violate this voltage.
So that would have been 1 Volt.
So that would have been 1 Amp through the resistor.
So then you're left with the problem with this guy's pushing 2 and that guy's only eating 1.
But the rules for the voltage source say eat or source however much current is necessary in order to make the voltage equal to 1.
So the excess amp goes through the voltage source.
So the voltage source is, in fact, being supplied power rather than supplying power itself.
Had this been 2 Amps, some of the power from this source would have gone into the resistor.
And some of the power from this source would have actually gone into the voltage source.
So if the voltage source were, for example, a model for a rechargeable battery, that rechargeable battery would be charging.
Does that make sense?
So if there had been a mismatch in the conditions, you still have to satisfy all the relationships from all of the sources
What if the voltage was larger?
The same thing would have happened.
Except now the flowing current would be in the opposite direction.
Let's say that if the voltage here had been 2 Volts, then the voltage would have required that there is 2 Amps flowing here.
1 Amp would come from here, but another Amp would come from here.
This voltage source will supply whatever current is necessary to make its voltage law real.
Ok.
In fact, what we'll do now is turn toward a discussion of more complicated systems that will let you go back and in retrospect, analyze all those cases that we just did.
And you'll be able to see trivially how that has to be the right answer.
So what I want to do now is generate a formal structure for how you would solve circuits.
Yes?
So did we know of anything that could generate a current without generating a voltage, like in real life?
Can anything generate a current without generating a voltage?
That's a tricky question.
If you think about something as generating a current, then the voltage is not necessarily determined by that part.
So that's kind of illustrated here.
If this guy is generating a current, this guy is not actually the element that is controlling its own voltage.
In general, if you want to speak simultaneously about the current and voltage across the device, you have to know what it was connected to.
Each part-- we'll get to this in a moment in case some of you are worried about launching ahead.
We will cover this.
This is very good motivation for figuring out what's going to happen in the next three slides.
So each part gets to tell you one relationship between voltage and current.
Generally speaking, that's not enough to solve for voltage and current.
Voltage and current is like two unknowns.
Each element relationship is one equation.
So the current source gets to say current equals x, current equals 1 Amp.
It doesn't get to tell you what the voltage is.
So being a little more physical to try to address your question more physically, there are processes that can be extremely well modeled as current generators.
In fact, many electronic semiconductor parts, like transistors, work more like a current source than like anything else.
So there are devices that behave as though they were current sources, but they don't simultaneously get to tell you what is their current and what is their voltage.
They only get to tell you what is their current.
So let's think about now, if you had a more complicated system, how could you systematically go about finding the solution?
As was mentioned earlier, there's something called Kirchhoff's law.
And in fact, there's two of them.
Kirchhoff's voltage law and Kirchhoff's current law.
Kirchhoff's voltage law, in its most elementary form, says that if you trace the path around any closed path in a circuit, regardless of what the path is-- every closed path-- the sum of the voltages going around that closed path is 0.
So for example in this circuit, the red path illustrates one closed path through the circuit.
It goes up through the voltage source, down through this resistor, and then down through that resistor.
Kirchhoff's voltage law says the sum of the voltages around that loop is 0.
That's written mathematically here, minus v1 for here, plus v2 for here, plus v4 for here is 0.
OK, where do the signs come from?
The signs came from the reference directions that we assigned arbitrarily to the elements.
Before I ever do a circuits question, I always assign a reference direction.
Every voltage has a positive terminal and a negative terminal.
And I must be consistent in order to apply these rules.
These rules only work if I declare a reference direction and stick with it.
If midway through a problem I flip it, I'll get the wrong answer.
So the minus sign has to do with the fact that as I trace this path, I enter the minus part of this guy, but the plus part of that guy and that guy.
So the sign of v1 is negated relative to the others.
A different way to think about that is here, we can think that v1 is the sum of v2 and v4.
That's sometimes more intuitive because if you started here, going through this path you would end up with a voltage that is v1 higher than where you started.
Whereas starting here, you would end up with a voltage here that's v4 higher than where you started.
And then by the time you got to here, it would be v2 plus v4 higher than where you started.
You start one place and on one route, you end up v1 higher.
And in the other route, you get v4 plus v2 higher.
So it must be the case that v1 is the same as v2 plus v4.
Those are absolutely equivalent ways of thinking about it.
So those laws are equivalent.
If you think about it a path, you think about some of the paths-- no.
The path coinciding with the negative direction of some of the elements and the positive direction of others.
OK, how many other paths are there?
Take 20 seconds, talk to your neighbor.
Figure out all of the possible paths for which KVL has to apply.
OK, so everybody raise your hand and show a number of fingers equal to the number of KVL equations less two.
Oh, very good.
Virtually 100% correct.
Why do you all say (5)?
Which is to say 7.
Why do you all say 7?
So there's 3 obvious ones.
I was expecting a couple of 3's.
This was supposed to be-- OK, yeah, I do plot against you.
I was expecting some 3's.
So there's 3 obvious paths that are analogous to the first one we looked at.
If I call the first path A, then there's B and C which are the excursions around here.
And you can write the equations just the same.
They each involve three voltages.
And they each go through, some starting at the negative side and some starting at the positive side.
So those are in some sense, the obvious ones.
But there are others too.
So one way to think about it, what I'd like you to do is enumerate all the paths through the circuit.
I should have said all the paths through the circuit that go through each element one or fewer times.
I don't want you to go through the same element twice.
So here's another path that would go through elements at most one time.
So up through here, over through here, which didn't go through any elements.
Down through that element, across that, down through here, et cetera.
And you get an equation for that.
Here's another.
Here's another.
Here's another.
And if you try to think about a general rule, a general rule is something like, how many of those panels can you make and piece together where the loop goes through the perimeter?
You're not allowed to go through an inner place because if you went through an inner node, you'd have to go through it twice.
If you wanted the path to go through an inner element, you'd have to go through that element twice.
So in fact, the answer is 7.
There are 7 different paths according to Kirchhoff's voltage law, all of which the sum of the voltages around those paths has to be 0.
The problem is, of course, that those equations are not all linearly independent.
So if you just had a general purpose equation solver-- and by the way, we'll write one of those in week 8 for solving circuits.
If you just passed those 7 equations into a general purpose equation solver, it would tell you there's something awry with your equations because they're not linearly independent.
So you can, however, think about linearly independent in particularly simple cases.
This network is a particular kind of network that we call a planar network.
A planar network is one that I can draw on a sheet of paper without crossing wires.
So I can draw this network without crossing wires.
I'll call it planar.
And it turns out that Kirchhoff's voltage laws for the innermost loops are always independent of each other.
That's kind of obvious because as you go to a -- so each loop contains at least one element that some other loop didn't have.
So that's kind of the reasoning for why it works.
So if you think about this particular loop, which we included in the 7, you can think about that as being the sum of the loops this way, the A loop and the B loop.
Because if you write KVL for the A loop and KVL for the B loop and add them, you end up deriving KVL for the more complicated path.
And if you think about what's going on, it's not anything terribly magical.
This path is the same as the A path added to that path, where I went through this element down when I did the A path and up when I did the B path.
So those parts canceled out.
That was the rule that I was talking about how I don't really want to go through the same element twice when I'm applying KVL.
So the idea then is that there's a systematic way, an easy way to figure out all the KVL loops.
You just think about all the possible paths through the circuit.
You do have to worry about linearly independent.
In the case of planar networks, that's pretty straightforward.
Planar networks, you can always figure out the linearly independent KVL equations by looking at the smallest possible loops.
The loops with small area.
OK, so that's half of it.
That's KVL.
The other Kirchhoff's law is KCL, Kirchhoff's Current Law.
There we are thinking of the flow of current.
So the flow of current is analogous to the flow of incompressible fluid.
Water, for example.
If you trace the amount of water that flows through a pipe that goes into a Y, then the sum of the flows out has to equal the flow in.
If that weren't true, the water would be building up.
So we think about pipes as transporting the flow of water without allowing it to build up anywhere.
That's precisely how we think about wires in electrical circuits.
The wires allow the transport of electrons but don't allow the buildup of electrons.
OK, do electrons build up?
Sure.
But in our idealized world, we say they don't build up in the wires, they build up in a part.
And we'll have a special part that allows the electrons to build up.
So we're not excluding the possibility that they build up, we're just saying that in this formalism, we don't allow the electrons to build up in the wires.
So for the purpose of the wires, current in is equal to the current out.
The net current in is 0.
So we will think then, about the circuit having nodes.
The nodes are the places where more than one element meets, two or more elements meet.
And we will apply KCL at each node.
So for example, in this simple circuit where I would have three parts connected in what we would call parallel, they share a node at the top and they share a node at the bottom.
So even though it looks like there's multiple interconnects up here, we say that's one node.
And we would say that the sum of the currents into the node is equal to the sum of the currents out.
So if I lab all of the possible currents that come out of that node, I would have i1, i2, i3.
i1 goes through the first one, the second one and the third one.
And so I would conclude from Kirchhoff's current law that the sum of i1, i2, and i3 is 0.
OK. Easy, right?
As I said, we're going to make an abstraction where the electrons don't build up in the wires.
They don't even build up in the parts.
They do get stored in the parts.
That's a little confusing, we'll come back to that.
If they don't build up in the parts, then the current that goes in this leg has to come out that leg.
If that's true, then i1 is i4, i2 is i5, i3 is i6, and we end up with another equation down here, which turns out to be precisely the same as the one at the top.
Everybody's happy with that?
So we're thinking about this just the way we would think about water flow.
If there's water flow into a part, it better be coming out.
If there's water flow in a pipe, the water that goes into the pipe better come out of the pipe someplace.
So here is an arbitrary network made out of four parts.
How many linearly independent KCL equations are there?
So how many linearly independent KCL equations are in that network?
Everyone raise your hand, some number of KCL equations.
OK, I'm seeing a bigger variety.
I see (1)'s, (2)'s, and (3)'s.
I don't see any (4)'s.
That's probably good.
So how do you think about the number of linearly independent KCL equations?
So the first thing to do is to label things.
So you have to have reference directions before you can sort of think about things.
So we have four elements.
We would be expecting to see four element currents.
The same current that goes into an element has to come out of it.
So there's element current 1, 2, 3, and 4.
There are three nodes, so we might be expecting three KCL equations.
Here's one node from which you would conclude that the sum of i1 and i2 better be 0.
Here's a node from which you would conclude that the current in i2 better be i3 plus i4.
And here is a node from which you would conclude that i1 plus i3 plus i4 is 0.
So I can write one KCL equation for every node, that's not surprising.
But if you look at those equations, you'll see that they're not linearly independent.
In fact, if you solve this one for i2-- it's already solved for i2.
Stick that answer up here, you get i3 plus i4 added to i1 is 0, which is just the same as that equation.
So of those three equations, only two of them are linearly independent.
The answer to that problem was (2).
And there's a pattern.
So think about the pattern in terms of figuring out the number of linearly independent KCL equations that are in a slightly more complicated network.
So what's the answer here, how many KCL equations are in this network?
Wow.
Well, I'm not getting any of the answers I would have said.
What does that mean?
Ah, I'm forgetting to add 2.
That's my problem.
OK, now I'm getting some of the answers that I would expect to get.
OK, got it.
I confused myself.
OK, the vast majority say (1).
How do you get that?
Which is 3.
So again, you think of how in this circuit there are four nodes, A, B, C, D. So we can think about writing a KCL equation for each one.
If we go to A, A has three currents coming out of it -- 1, 2, 3.
So the sum of those has to be 0, et cetera.
And if you think about those equations, they're not linearly independent either.
If you work through the math, you see that there's exactly one of those equations that you can eliminate.
So you're left with three linearly independent KCL equations.
And so there's a pattern emerging here.
Somebody see the pattern?
1 minus.
Can somebody prove the pattern?
So there's a pattern here.
The pattern is take the number of nodes and the number of independent KCL equations as one less.
So the challenge is, can you prove it?
And by the theory of lectures--
Yes
Yes.
And by a corollary of the theory of lectures, the way you would prove it is?
On the next slide
On the next slide.
Exactly.
So how do I prove it?
Yeah?
Whenever you take minus 1, you just add all the [UNINTELLIGIBLE] together [UNINTELLIGIBLE]
Yeah.
So there's something special about the last one.
Why should there be something special about the last one
Because the circuit's closed
Because the circuit's closed.
That's right.
So the idea is to sort of generalize the way we think about KCL.
So we start with a circuit.
We think about having four nodes here.
It's certainly the case that KCL holds for each node.
So here's KCL for that node.
But now if you think about KCL for this node, and then add them, that looks like a KCL equation.
But it applies to a super node.
Imagine the node defined by the black box, and think about the net currents into or out of the black node.
This current i2, which leaves the red node, enters the green node, but doesn't go through the surface of the black node at all.
That's exactly the current that's subtracted out when we added the red equation to the green equation.
Does that make sense?
So KCL says, oh, if all the currents at a node have to sum to 0, and if elements have the same current coming out and going in, then if you draw a box around an element, what goes into the element is the same as what comes out of the element.
It doesn't change the net current through the surface.
So the generalization of the KCL equation, KCL says the sum of the currents into a node is 0.
The generalization says take any closed path in a circuit, the sum of the currents going across that closed path is 0.
So if we apply that rule again, think about node 3.
If we add the result of node 3 to the black node, which was the sum of 1 and 2, we get the new green curve.
We get the new green equation.
And what that says is the sum of the currents going across the green super node-- OK, so what's going on?
i1 is coming out of it, i4 is coming out of it, i5 is coming out of it.
So the sum of i1, i4, and i5 has to be 0.
Well, KCL says the sum of the currents coming out of a node must be 0.
The super KCL says the sum of the currents coming out of any closed region is also 0.
But the interesting thing about this closed region is that it encloses all but one of the nodes.
That's always true.
Regardless of the system, regardless of the circuit, you can always draw a line that will isolate one node from all the others.
So what that proves is that you can always write KCL for this node in terms of KCL for those nodes.
Ok.
So there's a generalization then that says that you can always write KCL for every node.
They will always be linearly dependent.
So you can always throw away one.
So in some sense now, we're done.
We've just finished circuit theory.
We talked about how every element has to have a law.
A resistor is Ohm's Law.
A voltage source says that the voltage across the terminals is always a constant.
A current source says that the current through the current source is always a constant.
So every element tells you one law.
We know how to think about KVL.
So we know the rule for how the across variables behave.
What's the aggregate behavior of all the across variables?
Well, KVL has to be satisfied for every possible loop.
The loops don't have to be independent.
You have to worry about whether they're independent.
The only simple rule we came up with-- we'll come up with another one in a moment.
The only simple rule that we came up with was for planar circuits, where the innermost loops were linearly independent of each other.
And you have to write KCL for all the nodes, except one.
One of them never matters.
So in some sense, we're done.
What we would do to solve the circuit, think about every element.
For every element assign a voltage, a reference voltage.
For every element, assign a current.
Make sure they go in the right direction.
We always define currents to go down the potential gradient.
They always go in the directions through the element from the positive to the negative.
So for every element, assign a current and a voltage.
We have 6 elements, that's 12 unknowns.
Now we dig and we find 12 equations.
In this particular circuit, we found those 12 equations.
There were three KCL equations, one for each of the inner loops.
There were three KCL equations, one for each node except one.
There were 5 Ohm's law equations, one for each one of the resistors.
There was one source equation for the voltage source.
12 equations, 12 unknowns, we're done.
The only problem is a lot of equations.
It's not a very complicated circuit.
We've only got 6 elements.
I tried to motivate this in terms of studying networks that had 10 to the 9 elements.
This technique is not particularly great at 10 to the 9.
It would probably work.
But we would probably be interested in finding simpler ways.
So there are simpler ways you might imagine, and we'll discuss two of them just very briefly.
The dumb way that I just talked about is what we call primitive variables, element variables.
If you write all the element variables, v1, v2, v3, v4, v5 v6, all of the element currents, i1, i2, i3, i4, i5, i6, write all the equations, you can solve it.
However, if you're judicious, you can figure out a smaller number of unknowns and a correspondingly smaller number of equations.
One method is called the node method.
When we're thinking about the individual elements, the thing that matters is the voltage across the element.
However, that's not the easy way to write the circuit equations.
A much easier way is not to tell me the voltage across an element, but instead tell me the voltage associated with each of the nodes.
If I tell you the voltage associated with every node, the important thing about that way of defining the variables is that you're guaranteed that from those variables, you can tell me the voltage across every part.
So for example, in this circuit, this voltage source-- so if I call this one ground, we'll always have a magic node called ground.
It is not special in the least.
It's just the reference voltage.
I'll come back to that.
I'll say words in a minute about what the reference is.
We always get to declare one node to be ground.
We get one free node.
It's a node whose voltage we don't care about because it's the reference for all voltages.
It's a node whose current we don't care about because we get to throw away one node when we do current equations.
So we have one special mode called ground, about which we don't care too much.
Except that it's the most important node in the circuit.
Except for that, we don't care about it.
So this guy's ground.
We think about its voltage being 0.
Then this voltage supply makes that node be v0.
I don't know what that is, so I'll call it e1.
And I don't know what that is, so I'll call it e2.
So if I tell you the voltage on all of those nodes, ground voltage is 0, the top voltage is v0, the left voltage is e1, the right voltage is e2.
From those four numbers, 0 and 3 nontrivial numbers, you can find all of the component voltages.
So for example, the voltage v6, the voltage across R6 is e2 minus e1.
The voltage v4, the voltage across the R4 resistor is e1 minus 0.
So if I tell you all the node voltages, you can tell me all of the element voltages.
And in general, there's fewer nodes than there are components.
OK, that's great.
So instead of naming the volts across the elements, we'll name the voltages at the nodes because there's fewer of them.
Then, all we need to do in the node method is write the minimum number of KCL equations.
We know we only have two unknowns, e1 and e2.
And it turns out-- and you can prove this, but I won't prove it today.
It turns out that you need two KCL equations.
Two unknowns, e1, e2, two KCL equations.
And it turns out those two KCL equations are exactly the KCL equations associated with the two nodes.
So the current leaving e1, so KCL at e1 -- well, there's a current that goes that way.
Well, that's the voltage drop in going from e1 to v0, e1 minus v0, divided by R2.
That's Ohm's law.
So this term represents the current going up that leg plus the current that goes through this leg, which is e1 minus e2 over R6.
Plus the current going in that leg, which is e1 minus 0 over R4.
The sum of those three currents better be 0.
Analogously, the sum of the currents at this node must be 0, and the equation looks virtually the same.
Because v0 is known, so it didn't add an unknown.
v0 was set by the voltage, by the voltage source.
So I have two equations, two unknowns, solved.
Done.
So rather than solving 12 equations and 12 unknowns, I can do it with two equations and two unknowns.
That's called the node method.
One of the most interesting theories about circuits is that every simplification that you can think about for voltage has an analogous simplification that you can think about in current.
That's called duality.
We won't do that because it's kind of complicated.
But it's kind of a cute result.
If you can think of a simplification that works in voltage, then there is an analogous one, and you can prove it.
In fact, you can formally derive what it must have been.
This is a rule for how you can simplify things by thinking about voltages in aggregate.
Rather than thinking about the element voltages, think about the node voltages.
The analogous current law is rather than thinking about the currents through the elements, the element currents, think about loop currents.
OK, that's a little bizarre.
So we name the loop, the current that flows in this loop, IA, the current that flows in this loop, IB, and the current hat flows in this loop, IC.
What on earth is he doing?
Well, the element voltages are some linear combination of those loop currents.
And in fact, the coefficients in the linear combination are one and minus one.
So the element current I4, the current that flows through the R4 resistor is the sum of IA coming down minus IC, which is going up.
So there's a way of thinking about each element current as a sum or difference of the loop currents.
Everybody get that?
So instead of thinking about the individual element currents, I think about the loop currents.
And now, I need to write three KVL equations.
So in the node method, I named the nodes and had to write two KCL equations.
Here, I named the loop currents and I have to write three KVL equations, one for each loop.
It's completely analogous.
If you write out a sentence, what did you do?
I assigned a voltage to every node, and I wrote KCL of all the nodes.
Then if you turn the word "current" into the word "voltage," the word "node" into the word "loop," you derive this new method.
So this says that if I write KVL at the A loop, think about spinning around this loop, as I go up through the voltage source, so I go in the negative terminal here.
So that's minus v0.
As I go down through this resistor, I have to use Ohm's law, so that's R2 times the down current.
Well, the down current is IA down minus IB up.
So I went up through here, down through here.
Now I go down through this one.
When I go down through that one, according to Ohm's law, that's R4 times the current through that element.
That current-- well, it's IA down and it's IC up.
So this is the KVL equation for that loop.
I write two more of them, and I end up with three equations and three unknowns.
Both the node method and the loop method resulted in a lot fewer equations than the primitives did.
I had 12 primitive unknowns, 6 voltages and 6 currents.
In the node method, I get the number of independent nodes as the number of equations and unknowns, which is less than the number of primitive variables.
In the loop method, I have the number of independent loops.
Which is again, smaller.
So the idea then is that we have a couple of ways to think about solving circuits.
Fundamentally, all we have are the element relationships and the rules for combination.
Oh, this is starting to sound like PCAP, primitives and combinations.
So the primitives are, how does the element constrain the voltages and currents?
We know three of those, Ohm's law, voltage source, current source.
And what are the rules for combination?
Well, the currents add to the node, and the voltages add around loops.
OK, just to make sure you've absorb all that, figure out the current I for this circuit.
OK, what's a good way to start?
What should I do to start thinking about calculating I?
OK, bad way.
Assign voltages and currents to everything.
4 elements, that's 4 voltages, 4 currents.
That's 8 unknowns.
Find 8 equations, solve.
That'll work.
Bad way.
What's a better way?
OK, [UNINTELLIGIBLE PHRASE]
[UNINTELLIGIBLE PHRASE]
It was on the previous sheet.
[UNINTELLIGIBLE PHRASE]
KVL [UNINTELLIGIBLE]
KVL for where?
Loops
Which loop?
Left loop
So do KVL on the left loop?
Yes
OK, that's good.
But you have to tell me how to assign variables.
Do you want 8 primitive variables?
8 primitive variables are v1, i1, v2, i2, v3, i3, v4, i4.
So that's what I mean by primitive variables.
Or element variables is another word for it.
What's a better way than using element variables?
Yeah
Create 2 loop equations
Create 2 loop equations, that's fantastic
I1 for the first loop, I2 for the second loop
So if you do I1 going around here, then I1 is actually I.
And if you do I2 going around here, what's I2?
[INAUDIBLE]
So if I think about I2 spinning around this loop, so the sum of I1 and I2 goes through that box.
But the only current that goes through this box is?
[INAUDIBLE]
So the suggestion is that I think about-- so if I have I1 here, but I know that's I.
Then I can see immediately that since the only current that goes through here-- so if I have I1 and I2.
That was a very clever idea.
If you have I1 and I2, the only current that goes through here is I.
So I1 must've been I.
The only current that goes over here must've been this guy.
So this must be minus 10.
So I could redo that this way.
I could say I've got 10 going that way.
That make sense?
So now I only have one unknown which is I.
So that's a very clever way of doing it.
So what I could do is showed here.
I have I going around one loop and I have 10 going around that loop.
That completely specifies all the currents.
So now all I need to do is write KVL for these different cases.
So if I write KVL for the left loop, then I get going up through here, that's minus 15, and going down through here, going to the right through this guy is 3I.
Going down through this guy is 2 times I plus 10.
Both of these are going down, so you have to add them.
So I get one equation and one unknown.
And when I solve it, I get minus one.
That make sense?
There's an analogous way you could have done it with one node.
You could have said that the circuit has a single node and figured out KCL for that one node.
KCL would be the sum of the currents here.
There's a current that goes that way, that way, and that way.
And again, you end up with 1 equation and 1 unknown.
Yes?
[UNINTELLIGIBLE PHRASE]
Correct.
If I thought about this current going this way, it would be minus 10.
If I flipped the direction, then it's plus 10.
So the loop current has the property that it's the only current through this element.
So that has to match.
It's one of two currents that go through this element
You said that everything that [UNINTELLIGIBLE PHRASE]
This loop is [UNINTELLIGIBLE].
Yes
So why is the [UNINTELLIGIBLE PHRASE]
Correct.
I want to have this picture now.
So if I'm doing it with loops, I have two loops.
The current through this element is just I.
The current through this element is just I.
The current through this element is just 10.
The current through this element-- well, the sum of these two currents go through that element.
Does that make sense?
[INAUDIBLE]
This loop current is just a fraction of the current in the whole system.
So this loop current goes through this element and contributes to this element.
But so does that one.
OK, if you're still confused, you should try to get it straightened out in one of the software labs or the hardware lab, or talk to me after lecture.
But the idea is to decompose in the case of the node voltages.
Think about the element voltages in terms of differences in the node voltages.
In the case of the loop currents, think about the element currents in terms of a sum of loop currents.
OK, so the answer is minus 1 regardless of how you do it.
Ok.
The remaining thing I want to do today is think about abstraction.
We've talked about the primitives, which are things like resistors, voltage sources, and current sources.
Means of combinations, that's KVL and KCL.
Now we want to think about abstraction.
And the first abstraction that we'll talk about is, how do you think about one element that represents more than one element?
This is the same thing that we did when we thought about linear systems, when we did signals and systems.
We started with R's and K's and pluses, and we made single boxes that had lots of R's and pluses and gains in them.
What was the name of the thing that was inside the box?
If we combined lots of R's, gains, and pluses into a single box, what would we call the thing that's in the box?
[INAUDIBLE]
Shout again
System function
System function.
Right?
So we started with boxes that only had things like R's in them.
But eventually, we got boxes that looked like much more complicated things like that.
We thought about a system function which was a generalized box, that could have lots of R's, or lots of gains, or lots of pluses in it.
And that was a way of abstracting complicated systems so they looked like simple systems.
What we want to do here is the same thing for circuits.
We want to have a single element, a single circuit element, that represents many circuit elements.
And the simplest case of that is for series in parallel combinations of resistors.
It's very simple to think about how if you had two Ohm's law devices connected in series, you could replace those two with a single resistor.
And the voltage-current relationships measured at the outside of the box would be the same.
That's how we think about an abstraction in circuits.
When is it that you can draw a box around a piece of a circuit and think about that as one element?
The very simplest cases, the series combination of two resistors, same sort of thing happens for the parallel combination.
And that simple abstraction makes some things very easy.
What would be the equivalent resistance for a complicated system like that?
Well, that's easy.
All you need to do is think about successively reducing the pieces.
Here I'm thinking about that having four resistors.
I can just successively apply series and parallel in order to reduce that, make it less complicated.
So I can think about combining these two in series to get, instead of two 1 Ohm resistors, one 2 Ohm resistor.
Then I can think about these two 2 Ohm resistors being equivalently one parallel 1 Ohm resistor.
And so this whole thing looks as though it's just 2 Ohms from the outside world.
That's what we mean by an abstraction.
What we're trying to do and what we will do over the next two weeks, is we'll think about ways of combining circuits so that we can reduce the complexity this way.
Another convenient way of thinking about reducing the work that you need to do is to think about common patterns that result.
PCAP, Primitives, Combinations, Abstractions.
So the series of parallel idea was an abstraction.
A pattern, here's a common pattern.
If you've got two resistors in series, if the same current flows through two resistors, then there's a way of very simply calculating the voltage that falls across each.
So you can think about the sum resistor, R1 plus R2 since they're in series.
So that allows you then to compute the current from the voltage.
Then the voltage that falls across this guy is by Ohm's law, just the current times its resistor, which is like that.
And similarly with this one.
So you can see that some fraction of this voltage v occurs across the v1 terminal.
And some different fraction appears across the v2 terminal, such that the sum of the fractions is, of course, v. That's what has to happen for the two.
And there's a proportional drop.
The bigger R1, the bigger is the proportion of the voltage that falls across R1.
So it's a simple way of thinking about how voltage drops across two resistors.
There's a completely analogous way of thinking about how current splits between two resistors.
Here the result looks virtually the same, except it has kind of the unintuitive property that most of the current goes through the resistor that is the smallest.
So you get a bigger current in i1 in proportion to the R2.
So it works very much like the voltage case, except that it has this inversion in it, that the current likes to go through the smaller resistor.
OK, so last problem.
Using those kinds of ideas, think about how you could compute the voltage v0 and determine what's the answer.
So what's the easy way to think about this answer?
What do I do first?
Superposition
So superposition?
That's one thing
Simplify [UNINTELLIGIBLE]
Simplify.
So what's a good simplification?
Collapse?
You can put the two [UNINTELLIGIBLE] in series and treat them as one
So you can treat this as a series combination, and you can replace the series of 1 and 3 with a?
[UNINTELLIGIBLE PHRASE]
4
4.
So this can be replaced by four [UNINTELLIGIBLE] resistor.
Now what?
You can do the same on the parallel one
So you can replace the parallel of the 6 and a 12 with a?
4
Amazing -- with a 4.
So there's a 4 there and there's a 4 there.
And the answer is?
It's half
Half of whatever it was by voltage [UNINTELLIGIBLE] relationship.
So you think about this becoming that.
You think about the parallel becoming that.
You get a simple divide by 2 voltage divider.
So the answer is 7 and 1/2, which was the middle answer.
And so what we did today was basically a whirlwind tour of the theory of circuits.
And the goal for the rest of the week is to go to the lab and do the same sort of thing with practical where you build a circuit, and try to use some of these ideas to understand what it does.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello.
Welcome.
Welcome back from spring break.
Today what I want to do is talk about op-amps.
And in particular what I want to do is talk about modularity in circuit design.
How do you make circuits that are modular?
We'll see that there's special problems when you think about modularity applied to circuits.
And op-amps are one solution for helping us think about those problems.
Before launching straight into something new though, I'd like to recap where we are under the assumption that you may not have been thinking about this exactly currently.
So last time we took our first look at circuits.
Circuits are very different from the kinds of things we've thought about before.
Before, in programming, in linear systems theory, we thought about blocks that had well-defined inputs and well-defined outputs.
Circuits are different.
Circuits are all connected together.
And circuit theory is thinking about how do you organize your thoughts about complicated interactions among things.
The parts interact because they touch each other and they share voltages.
Because they touch each other, they share currents.
And you have to somehow think about that whole complicated system.
We figured out that the way you think about that, you can think about it as three separate enterprises.
How do you think about voltages.
How do you think about currents.
And how do you think about the elements.
How do you think about voltages?
Well, the sum of the voltages around any closed loop is zero.
Done.
KVL -- Kirchoff's Voltage Law.
How do you think about currents?
Draw any surface.
The net current that leaves the surface is zero.
KCL -- Kirchoff's Current Law.
How do you think about the elements?
Well it depends on the element.
If you're thinking about a linear resistor, it's Ohm's Law, V equals IR.
If you're thinking about a source, it could be a voltage source, then V equals v0, some fixed number.
If you're thinking about a current source, then current is fixed at some fixed number.
So it depends.
When you're thinking about the element laws, it depends on what element you are thinking about.
And then you just combine all of those, and you can solve the circuit.
The only problem that arises is that those equations are highly redundant.
There's a lot more KVL equations than you need.
There's a lot more KCL equations than you need.
And so in fact, the trick in analyzing a circuit is to figure out the smallest number of equations that are adequate, the number of equations that are necessary and sufficient to find a solution.
Last time we went through three different ways to do that.
We thought about what I will think of as primitive or element voltages and currents.
The idea in that technique is you think about every element and assign to that element its voltage and its corresponding current.
Then that gives you, as in this example if you've got six elements, that gives you six unknowns, one voltage across every element, one current through every element.
So then you've got to dig up six other relationships.
Sorry, start again.
Six elements, six voltages, six currents, 12 unknowns, you need to come up with 12 equations.
Six of them are element equations.
So six are easy.
Then you have to come up with six more.
And for this particular circuit, it turns out that there is three independent Kirchoff's Voltage Law equations.
And there are three independent KCL, Kirchoff's Current Law equations.
12 equations, 12 unknowns to solve, done.
That's actually more work than you need to do.
So we looked at two other techniques for solving the same kind of circuits.
One's called node voltages and another is called loop currents.
They are duals of each other.
In the node voltage method you figure out exactly the minimum number of voltages that I would have to tell you in order to specify all of the element voltages.
So for example, if I told you this voltage here, and that voltage there, and this voltage there, and that voltage there, four of them, that would let you calculate all of the element voltages.
But now I only have four instead of six.
Four nodes instead of six element voltages.
So it's smaller.
And we talked about last time how you could find the correct equations that go with those node voltages.
In the loop current method, you specify the minimum number of currents that would be sufficient to account for all of the element currents.
And here in this circuit it required three.
Three is smaller than six.
There were six element currents.
So again we've got a reduction in the number of unknowns -- 12 4, 3.
So that's kind of the idea.
And just to make sure that you're all with it, think about this problem.
How many of the following expressions are correct?
Feel free to talk to your neighbor.
At the end of about 45 seconds, I'll ask you to raise your hand with a number of fingers, (1) through (5) -- indicating which is the correct answer.
Dead silence.
You're allowed to talk.
OK, so how many of the relations are true?
Everybody raise your hand.
Indicate a number fingers that tells me how many of the answers are correct.
Come on.
Blame it on your neighbor.
You can say anything, right?
It's always your neighbor's fault.
That's very good.
It's about 95% correct, almost all correct.
This first equation, what is this?
Give that a name.
KVL.
Is it correct?
Sure.
You need to figure out which path it corresponds to.
It goes through-- The only v's are over here.
So therefore it must be something that has to do with this circuit.
So this says that there's 1, 2, 6, and 5.
That's path 1, 2, 6, and 5.
So all you really need to do is check the polarities.
So if we took all of the variables to the same side, then we'd have minus v1 plus v2 plus v6 plus v5.
If we think about going a loop that goes through the minus v1, that would be this way.
Plus v2 plus v6 plus v5.
So that's right.
Easy.
What's the point of this equation?
v6 equals e1 minus e2, what's that say?
Why'd I ask that?
Yeah
It's saying that the--
Exactly, it's intended to make you think through the relationship between the primitive variables, the element voltages, and the node voltages.
If I told you the node voltages e1 and e2, you could trivially compute v6.
You could also compute any of the other v's.
If I told you to find v-- what is that?
My eyes are not very good.
v4, if I told you to find v4, that would be e1 minus ground.
We assign the number 0 to ground, so that would be e1.
So the idea, the reason I ask number (2), is to make you think about how you relate the node voltages to the element voltages.
How about number (3), what's that?
Well it's highly related to number (2).
But it's different from number (2), because?
Ohm's Law?
Ohm's Law.
So equation (3) is the way Ohm's Law looks when you use node equations.
Ohm's Law is a little bit uglier when you use node equations than when you use primitive variables.
When you use primitive variables, the same relationship would've simply said that v6 is R6 times i6.
Because I'm using node voltages instead, the voltage across resistor six shows up as a difference.
How about this one?
What's that?
True or false, this is KCL?
False
False
--
True.
This is KCL, true.
OK.
This is KCL, false.
Well, obviously we're numerous.
And it's not, sort of, 100% participation, but-- This is not KCL.
What is that?
KCL says that the sum of current out of some closed path.
This is mixed thing.
i6 is one of these things.
And IB and IC is one of these things.
So what is the equation for?
Incorrect
Incorrect, yes.
What would be the correct way of saying equation (4)?
i6 equals-- i6 is--
The answer to set questions according to the theory of lectures is go to the next slide.
Here's the correct expression.
Why is this correct, and why is that not correct?
Equation (4) is intended to be how do you relate the loop current to the element currents?
So i6 is an element current.
It's the current that goes through R6.
When we do loop currents, we have two currents going through R6.
So in the loop current view, the total current that goes through R6 is a sum or difference of the two loop currents that go through R6.
So the element current through R6, which is i6, is a sum or difference of loop currents.
There are two loop currents we have to worry about.
Which one goes through in the correct direction?
i6 goes from left to right.
So when we do the loop currents, we need to take positive as the direction from left to right.
Well that's the direction of IC and is the opposite direction of IB.
So if I want to use loop currents to specify i6, it would be IC minus IB.
Everybody clear on that?
So equation (4) is the relation between element currents and loop currents.
So what's equation (5)?
Equation (5) is Ohm's Law for loop currents.
Again, if I were thinking about Ohm's Law for R6, I would have v6 equals i6 R6.
That's the way you say it in element voltages and currents.
Over here, the current through R6 is IC minus IB.
So Ohm's Law looks a little bit more complicated.
So the point is these three methods represent ways of figuring out a linearly independent set of unknowns and equations.
They differ.
The left hand one is probably the easiest to think about, especially when you're thinking about things like Ohm's Law.
It's the natural way to specify the element relationships.
It's the relation between the voltage and current through that part.
That generally gives me a large number of equations and unknowns.
You can reduce the number of unknowns by using something like node voltages or loop currents.
And that gives you fewer equations to solve.
They are completely equivalent.
They look a little different.
And the reason for talking about them is that when we think about writing a program to solve circuits automatically, which by the way will be the exercise in software lab this week.
When we think about writing a program, writing a program is yet a different kind of challenge.
What's the easiest system to automate?
So the system that is easiest for you may or may not be the easiest system to automate.
So that's the point of this week's software lab.
We'll do a method that's closely related to the node voltage method.
It's not quite node voltages.
It's a little simpler than node voltages for a computer.
It's a little simpler to automate.
So we use a method that's called node voltages with component currents.
OK, now I'm going to start new stuff.
That was review.
What I want to think about today is what is it about circuit design that makes it hard.
What are the issues that make it difficult to be modular when we're thinking about the analysis and design of circuits.
And one of the hardest things to deal with is the idea that in a circuit, unlike in a linear time invariant system of the type we talked about in the previous module, in a circuit the presence of every element affects the currents and voltages through, in principle, every other element.
So if you change one thing, you change everything.
So first off, I want to just give an example of that.
Think about what would happen if I were trying to make a circuit to control the brightness of a light bulb.
So I imagine that I've got this circuit and I close the switch.
Closing the switch is equivalent to adding a component.
So before I close the switch, when the switch was open, I have three elements, a voltage source and two resistors.
After I close the switch, I have four elements, the original three plus the bulb.
So the question is how would closing the switch affect V0 and I0.
Take a minute.
Talk to your neighbor.
Figure out how V0 and I0 change.
So what's the answer?
Everybody raise your hand, show a number of fingers equal to the answer.
That's very good.
It's 95% correct at least, maybe 100.
OK, so the answer is (2).
Why is the answer (2)?
How do I figure that out?
So the total resistance of the circuit is going to decrease because you're adding it in parallel.
And then V0 is going to decrease because it's going to have a lower equivalent resistance.
And because that decreases, you have I0 to--
Yes, I think that's exactly correct.
So the idea was that when you add a component-- Let's think about the light bulb being a resistor.
That's kind of pulled out of thin air, but I sort of suggested that you might do that here.
Think about representing the light bulb as a resistor.
Then when you close the switch, these two resistors go in parallel.
When you combine two things in parallel, the result is the same as a resistance that has a smaller value.
And then think about how that smaller value would interact with this resistor and that source.
And you can sort of figure out that the presence of this bulb would reduce this voltage and increase that current.
That's kind of a high level of reasoning given where we are.
If you wanted to think through this a little more step by step, it's easy.
You could think about figuring out what are the voltages and currents before and after you close the switch.
Before you close the switch, you can just ignore this, and you can calculate V0 just from a voltage divider relationship.
Right, that's clear?
So you can see that when the switch is open, the voltage, V0, is going to be 8 Volts.
And when the switch is open, you can figure out I0 by lumping these two resistors together to make a 3 Ohm resistor.
And you see that I0 is 4 Amps.
Then to figure out what happens when you close the switch, just repeat.
And the algebra is a little more tedious.
I won't try to go through it.
By the way, the answers, the slides that I show in lecture are always posted on the web.
So these slides that have the answers, they are there.
On the web there are two handouts per lecture.
There's an electronic version of the thing that we handed out.
There's also an electronic version of my slides.
So everything that I showed is there.
I'm not going to go through the tedious algebra.
It's just tedious algebra.
But if you go through the tedious algebra, you find that if you represent this bulb by resistor R, V0 becomes an expression that looks like this.
If you think about resistors, physical resistors, if you think about light bulbs being represented by a physical resistor, then the physical resistor has to have a resistance between 0 and infinity.
And if you think about that expression, what could the value be if R varied between 0 and infinity?
That expression is always less than or equal to 8 Volts, showing that V0 went down when you closed the switch.
Similar tedious algebra leads to an expression like this in R. And if you think about how this would change as R goes from 0 to infinity, you see that that's always bigger than 4 Amps.
So that means I0 goes up.
So the point is you can think through this in a more sophisticated way.
And we hope by the end of the course you'll all be able to do that.
Or you can think through it in terms of just solving the circuit.
Solve it in two cases when the bulb is there and when the bulb is not.
Either way, the answer is (2).
The V0 went down, and I0 went up.
The point is that when I added the element, currents that were far away from the element still changed.
And that's a general way circuits interact.
They're all connected.
So the idea, the point of doing this is that the addition of a new element changed the voltages and currents through the other element.
That's a drag if what I was trying to do, for example, was design a brightness controller for the flashlight bulb.
Imagine that what I really wanted to do was use a voltage divider to make 8 Volts.
And what was in my head was I'd like that 8 Volts to be across the light bulb.
That's a good idea.
It just won't work.
At least It won't work the way this circuit worked.
If I just build it like so, there's an interaction between the bulb and the voltage divider circuits, so the voltage divider circuit is no longer a voltage divider.
After, in this circuit, when I close the switch, current flows in this wire.
The rule in a voltage divider is the same current has to flow through both resistors.
If the same current flows through two resistors, then you can use the voltage divider relationship to see how voltage partitions between the two resistors.
If current gets siphoned off the node between the two resistors, you can't use the voltage divider relation anymore.
That violates the premise of the voltage divider relation.
So when I close the switch, this is no longer a voltage divider, and it no longer works like a voltage divider.
Is that clear?
So what I'd really like is some magic circuit that I can put here that isolates the effect of the bulb on the effect of the rest of the circuit.
And that's exactly what an op-amp does.
That's what we're going to talk about next.
So this magic circuit is something that we'll call a buffer.
A buffer is a thing that isolates the left from the right.
So the buffer is going to be something that measures the voltage on this side and magically generates that voltage over here without changing this side.
So what we want to do now is develop some thought tools for how you think about op-amps.
Op-amps are different.
And if you just look at the picture, op-amp has to be different because there's too many terminals.
It's not like a resistor that has two legs.
It's not like a V source which has two legs.
It's not like an I source which has two legs.
It has three legs.
In fact, I haven't drawn all the legs.
There's more than three.
There's at least five.
So they're different.
And the way we think about them are different.
The key to thinking about the way an op-amp works is to think about a new class of elements called controlled elements.
So a controlled element is an element whose voltage current relationship depends somehow on a voltage and current measured someplace else in the circuit.
As an example, think about a current controlled current source.
That's depicted here.
I would normally write a current source that was a circle.
That means it's an independent current source.
That means that the current is fixed.
The little diamond thing is my way of representing the idea that the amount of current that comes out of this current source depends on something else.
In this case it depends on IB.
And IB happens to be a current that flows in that circuit.
So the idea is that the current in the current source depends on some other current.
It's a current controlled current source.
It's a current source whose current is controlled by another current.
Got it?
So we'll see.
Figure out, for this current controlled current source circuit, what's the ratio of Vout over Vin?
So what's the answer?
What's the ratio of Vout over Vin?
100%, wonderful.
The answer is (4).
Easy to get.
It's easy to get because I rigged this question to be easy.
I rigged it to be easy because you can sort of figure out everything that's going on on the left.
And then you can figure out everything that's going on on the right.
And so there's no sort of complicated coupling between the two.
So what's going on on the left, well, you can solve for IB.
IB is just Vi over 1,000 Ohms.
Then you can take that value of IB and use that to solve for what's going on over here.
Over here the out is this current, 100 IB times this resistance, 5 Ohms.
But we just found out that IB is VI over 1,000 Ohms.
Substitute it in, you get half Vi.
So the answer is number (4).
The ratio of Vout to Vin is a half.
So the point is these are a different kind of element, controlled sources, dependent sources.
But they're not too hard to work with.
They sort of look different structurally.
So think about what's going on here.
The controlled current source, the current controlled current source, I have to think of that as a box, because the current source has to know the value of IB.
So somehow this box, this thing that's doing this current controlled current source, it knows about IB, and it knows about the current source.
So they're somehow linked.
So that's why I put a box around it.
And then think about the equations that characterize that component.
So now we've got a component that's got four wires coming out of it.
The components we had before had two wires coming out of them, resistors, current sources, voltage sources.
This kind of a component has four wires coming out of them.
We call this kind of a component a two-port because we think of the left port and the right port.
That's compared to resistor, which we would call a one-port.
We think about this being a two-port.
And there's now two equations.
Now I've got, for that two-port, I've got two voltages and two currents.
There's the voltage across the left part.
And there's the voltage across the right part.
And there's current through the left part.
And there's the current through the right part.
So with the elements we've thought about before, I had one voltage across it and one current through it.
Now I've got two voltages across it and two currents.
It's kind of twice as big.
Not surprisingly, it takes twice as many equations.
Ohm's Law was a single equation for one resistor, a one-port.
V equals V0 was one equation for one component, a voltage source.
Here, I've got one component that has four wires, four unknowns.
And I get two equations.
So for this particular dependent source, I know that the voltage across this pair of terminals is 0, because it's essentially a short circuit between the two.
Short circuit just means connected with a wire.
And I know that the current I2 is related to the current over there this way.
The idea then is that this current controlled current source can be represented by a two-port, two voltages, two currents related by two equations.
It's kind of structurally twice as difficult to think about as a one-port.
Functionally, it's different from having two one-ports, because the two one-ports are coupled.
And that's the important part is the coupling between the two that you can't model with a simple resistor and a simple constant source.
So when we think about an op-amp, a good first model for an op-amp is to think about it as a voltage controlled voltage source.
And so that's depicted here.
I want to think about the op-amp, which I'll symbolically write this way.
This means something that has two inputs, a plus input and then a minus input, and a single output, Vout.
I can think about it as a voltage controlled voltage source.
This I mean to be the element representation.
This is how I'll draw it when I make a schematic diagram of a circuit.
This is the functional form.
This is the way I'll think about it when I'm analyzing it.
I'll think about the op-amp as being a voltage controlled voltage source.
This voltage source adopts a voltage that is some number K times the difference voltage Vplus minus Vminus.
And the trick in op-amps is that K is typically a very big number, typically bigger than 10 to the 5th.
We'll see in a minute why that's a frightfully useful component.
Let's just walk through an example to see how you would solve a circuit that has this kind of a voltage controlled voltage source in it.
So think about this circuit where I'm applying a voltage to the plus lead of an op-amp, and I'm wrapping the output back through the minus lead, through two resistors.
And what I want to do is analyze that circuit by thinking about the op-amp as a voltage controlled voltage source.
So I can see just by the way it's wired up that the voltage at the plus lead-- So I'm going to be thinking node voltages.
Node voltages tend to be easy.
I'm thinking node voltages.
So I'm going to define all my voltages referenced to a ground.
So I'll call this node ground.
That's what the funny symbol means.
Then this node voltage, the voltage at the plus terminal, is just the same as the input voltage.
This voltage at the minus terminal looks like a voltage divider relationship.
If you look at my model, it's very clear that I1 is 0, because there's no connection here between the Vplus and the Vminus.
So I1 is 0.
That means the total current that flows into the plus lead of the op-amp is 0.
The total current that flows into the minus lead of the op-amp is 0.
So because there's no current flowing in the plus and minus leads, I can calculate the voltage at this minus terminal as a voltage divider.
And it's just R1 over the sum of R1 and R2 times V0.
Then according to the voltage control voltage source model, Vout should be K times the difference between Vplus and Vminus.
So I just substitute in for Vplus -- this guy, Vi, and for Vminus -- this guy.
Do some algebra.
I get this expression after some algebra.
And then I say, yeah but I know that K's really big.
So if K's really big, KR1 is big compared to R1.
So I can ignore that.
In fact, K is so big that for any reasonable choice of R1 and R2, KR1 is even bigger than R2.
So that means this reduces to this kind of a fraction.
So the response of this circuit, this op-amp circuit-- So what did I do?
I just took the op-amp circuit, and I modelled it as a voltage controlled voltage source, plugged through the equations, and found out that the ratio of the output voltage to the input voltage is R1 over R1 plus R2 divided by R1.
So the idea then is that this simple circuit works like an amplifier.
It's an amplifier in the sense that I can make the output voltage bigger than the input voltage.
That's a very useful thing.
In fact you'll find useful ways to use that when you do the design lab this week.
So this as an amplifier.
So here's a question.
Make sure you follow what I just did.
How could I choose the components R1 and R2 so that I make Vout equal to Vi?
Why is this 0?
There's no wire connecting
Oh
So there's nowhere for current to go.
OK so how would I choose the components R1 and R2 in order to make the output voltage equal to the input voltage?
Wonderful.
So the idea is that all of these manipulations have the same effect.
All you need to do is look at the expression that we developed.
The expression was R1 plus R2 over R1.
If you substitute, R1 goes to infinity, R2 equals 0, or the two at the same time, you get one in all of those cases.
So that's a way that you can turn this amplifier circuit, that in general would make the output bigger than the input, into something that makes the output equal to the input.
Now I want to turn to a simplification.
I just dragged you through the math.
Thinking about the op-amp as a voltage controlled voltage source, there's actually a shortcut.
The shortcut is something we call the ideal op-amp.
So what I want to do in this slide is drag you through the math one more time, but then we're done.
The idea is that if you have an op-amp, a voltage controlled voltage source, if you represent an op-amp as a voltage controlled voltage source, and if K is very big, the effect will be to make the difference between the positive and negative terminals of the op-amp quite small.
You can see that here by way of a simple example.
So let me think about this case with, again, the voltage controlled voltage source model.
So here I've got Vout.
According to the voltage controlled voltage source model, Vout should be K times the difference between the two inputs.
The positive input is clearly Vi.
The negative input is Vo.
One equation, two unknowns, solve for the ratio.
The ratio of Vout to Vi is K over (1 plus K).
That's the answer.
Take that answer and back substitute to figure out how big was the difference between Vplus and Vminus.
That's just algebra.
And what you see is that if this is the answer, then Vplus minus Vminus can be written as a fraction of Vi or a similar fraction of Vo.
K is big.
K is essentially the same as K plus 1.
K is in the denominator of both.
What that says is the difference between the plus and the minus leads, the voltage between the plus and the minus leads, is very small if K is very large.
OK?
So we call that the ideal op-amp relationship.
The utility of that is that it makes solving the op-amp circuits much, much easier than what we've just been doing.
I've been solving the circuits by thinking about the op-amp as a voltage controlled voltage source.
That's fine.
But if I additionally know that K is very big, I can shortcut it.
I can say, look, the effect of the op-amp is going to be to make the positive and negative inputs the same.
If it didn't do that, think of what it would mean.
If K is a big number, and if the output voltage is K times the difference, if the difference is anything other than epsilon, Vout has to be infinity.
I mean if K is very big, right?
If K is very big, the only way the output could be some reasonable number like a Volt would be if the difference between Vplus and Vminus is very small.
OK, well let's work backwards.
Let's start with the assumption that Vplus minus Vminus is very small.
And that lets us solve the circuits very quickly.
So for example, the same circuit that took, previously, a few lines to get the answer to, if I just take as a rule that Vplus has to be Vminus, it's a one step.
Vplus equals Vminus, OK, Vi equals Vo, period, done.
It's a very simple way to think about the answer to an op-amp circuit.
So if the op-amp can be represented by a voltage controlled voltage source and if K is very large, then Vplus is roughly Vminus.
Shortcut, ideal op-amp assumption.
So, use that or ignore it depending on what your mood is.
And figure out the voltage relationship for this slightly more complicated circuit.
So what's the answer?
Yes?
No.
How many are done?
How many are not done?
OK, take a minute.
This is supposed to be easy.
Think ideal op-amp.
OK, what's the output?
Everybody raise your hand, what's the output?
More hands, more hands, more hands.
OK, tiny number of hands, but those who showed hands are about 100% correct.
I don't know how to grade that.
Small participation, 100% among those who did participate.
So how do I think about this?
What's step (1)?
All right, according to the theory of lectures, what is step (1)?
Look at the previous slide
Look at the previous slide, yes.
What was the previous slide?
OK The previous slide had to do with?
The ideal op-amps
Ideal op-amps.
What's ideal op-amps say?
Vplus equals Vminus.
What happens here if Vplus is equal to Vminus?
Well what's Vplus?
0
So what's Vminus?
0
0.
So if Vminus is 0, what do I do now?
That bad, that hard huh?
Well, I got a [UNINTELLIGIBLE] here in which three different currents can flow.
How big is the current that can come into this node?
What's the sum of all the currents that can come into that node?
Well one could come in this way.
One could come in that way.
One could come in that way.
How much current flows in the minus lead of the op-amp?
0
0
None.
No current goes into the minus.
No current enters the op-amp through the minus lead.
So it's the sum of three currents.
How big is this current?
How big is the current that flows in that lead?
V1 -- V1 over 1, right?
Ohm's Law.
So this node is 0.
That node is V1.
The voltage across this resistor is V1 minus 0.
The voltage across is V1.
The current is V1 over R. R is 1.
So the current that flows in this leg is V1.
How much current flows in this leg?
How much current flows in this leg?
V0
V0.
The idea is that the total current that flows into this node is V1 plus V2 plus V0.
Solve for V0.
V0 is minus V1 minus V2.
Right?
Got it?
The idea was that this is really easy to solve if you use the ideal op-amp approximation.
You can see that this is 0.
Therefore, this is 0.
So you have a single KCL equation.
And the result is that this circuit looks like an inverting summer.
It computes the sum of V1 and V2 and then takes the negative of that and presents that at the output.
What I'm trying to motivate is that there's a whole different level of reasoning that you can do when you have this element, which is an op-amp.
Here what we've done is we've made something that performs a numerical operation.
It presents, at the output, the negative sum of the two inputs.
OK another problem.
Determine R so that V0 is twice V1 minus V2.
So how should I choose R?
Not very many hands.
This is the perfect nano quiz practice, right?
This looks like a perfect nano quiz question, right?
Smile.
So what's the answer?
OK we're down to about half correct.
This is harder.
Basically you do exactly the same thing, it's just that it's a little algebraically messier.
So not surprisingly, the first idea is to think about the ideal op-amp.
We'll think about what was the voltage here, what was the voltage there, and then we'll equate them.
What's the voltage at the plus lead?
Well that's easy.
That's just a voltage divider here.
Since no current flows in here, I can compute the voltage relative to this ground as R over (1 plus R).
That's showed here.
Times V1.
This one's a little bit trickier because I've got two sources, V2 and V0, each pumping current into this place.
The way I thought about it was start with V2 and then add to it this component, which can be thought of as a voltage divider here.
So how big is the voltage at Vminus?
Well it's V2 plus a voltage divider here, which is given by this.
That's a little tricky.
If you're not comfortable with that step, you could also solve for that voltage using the node method.
The node method will give you the same answer.
The answer is that the voltage at the Vminus port is two thirds of V2 and one third of V0.
That sort of looks right because the resistors are in a ratio of 2:1.
And then using the ideal op-amp assumption, we equate the two, do some more algebra, and we get some relationship and figure out that R is 2.
The point is that this is a relatively complicated circuit.
You could have done it if I had asked you to do it with the voltage controlled voltage source model.
But the algebra is already hard here with the ideal op-amp assumption, and with the voltage controlled voltage source it's even harder.
So the idea is that the ideal op-amp assumption, the idea that Vplus is equal to Vminus, makes the work of calculating these responses significantly easier.
Everybody's with the bottom line?
We started with the idea that when you add an element to a circuit, in general, adding an element changes voltages and currents throughout the circuit.
We wanted a way to make the design more modular, a way of adding a component without changing everything else.
We thought about this op-amp thing.
The model for the op-amp was voltage controlled voltage source.
We inferred this ideal op-amp model.
The ideal op-amp model was great for calculating the response.
It has this one problem.
It seems to say these circuits are identical.
If I literally believe the ideal op-amp assumption that all the op-amp does is magically make the plus and minus terminals the same, I would conclude that this circuit where the input comes in the plus and the output wraps around to the minus generates precisely the same input-output relationship as this one, where the input comes in the minus and the output wraps around the plus.
The ideal op-amp assumption simply says for both of those, Vi equals Vo.
So I would assume from the ideal op-amp model I get the result that these two circuits work exactly the same way.
Somehow that sounds wrong.
I've got this part, and I can wire it up the right way and it works.
Or I can flip the two wires, and it still works.
Conservation of badness doesn't let that happen, right?
If you do something bad, it should break.
The ideal op-amp assumption seems to lead to a bogus result.
It seems to say these two are the same.
So what I'd like to do is think about that for a moment.
We want to be comfortable with the assumption that we make.
The ideal op-amp assumption makes analysis really easy, but we'd like to understand exactly what we're assuming.
This just doesn't sound right.
OK, so let's back up.
Let's not do the ideal op-amp assumption.
Let's Instead say that we use the voltage controlled voltage source model.
So what I've done here is substitute into the left circuit.
So this circuit is shown here and the other over here.
All I've done is connected the input either to the plus or to the minus port and wrapped the other one around.
But now I'm analyzing it using the voltage controlled voltage source model.
And I've done the tedious algebra.
And I don't think there's any mistakes in the tedious algebra.
In one case I get K over (1 plus K), which for large K is about 1.
In the other case I get minus K over (1 minus K), which for K large is about 1.
The ideal op-amp assumption says that it doesn't matter.
The voltage controlled voltage source model says it doesn't matter.
And Freeman thinks this doesn't make any sense.
So what's going on here?
What's going on is that we've actually made an enormous leap in thinking about the op-amp even as a voltage controlled voltage source.
The two models that we talked about, the voltage controlled voltage source and the ideal op-amp model, are great for calculating answers.
They are not so good at providing a mechanism for how the op-amp is actually working.
Think about just the ideal op-amp assumption.
It's great to think, OK, the op-amp is going to do whatever magic is necessary to make these two leads the same.
Well how does it do that?
The thing about the ideal op-amp assumption is that it doesn't tell you the mechanism by which that happens.
What does the op-amp actually do in order to get Vplus equal to Vminus?
What's actually going on inside the op-amp?
The ideal op-amp assumption is very good for analysis and not very good with mechanism.
What's the mechanism?
What's the op-amp actually doing?
What it's really doing is moving charge around.
8.02 -- charge.
So if you're going to have a change in voltage, there has to be motion of charge.
And that's what's missing from the voltage controlled voltage model and from the ideal op-amp model.
How do you think about moving charge around?
Well it's the same as moving water around.
Somehow, I think it's all those years of playing with water when I was a little kid, I find my intuition about water is better than my intuition about charge.
So let me start by thinking about the intuition for water.
The way a water tank works is if the flow in is different from the flow out, the height changes.
That's continuity.
So if the water is conserved, if there's not significant evaporation over the duration of this experiment, or if molecules of water do not spontaneously disappear or appear, under either of those two assumptions, then the change in the height is proportional to the difference between the rate at which it's coming in and the rate at which it's coming out.
If the rate at which it's going out is equal to the rate at which it's going in, there's no change in height.
If it's coming in faster, it's going up.
If it's coming in slower, it's going down.
All easy, right?
Charge works the same way.
The thing that accumulates charge in an electronic circuit is a capacitor.
And there's a direct analogy between thinking about the way the flux of water generates height and the way flux of charge generates volts.
So we can think about the flux of charge, that's current.
If there's a net flux, if there's a bigger current into a capacitor then there is out, the voltage on that capacitor goes up.
If there's a smaller current in than goes out, the voltage on the capacitor goes down just like the height in a tank of water.
We can make a much more realistic model for the way an op-amp works by explicitly making a representation for how the charge flows.
And that's shown here.
I'm going to take the voltage controlled voltage source model but explicitly make a representation for the output charging up.
What's the op-amp do?
The op-amp senses the voltage at the input and the output and does something to change the voltage at the output.
The thing it does is if the positive voltage is greater than the negative voltage, it pumps charge into the output node.
If the opposite occurs, it sucks charge out.
Now it's important, this is not an accurate depiction of what's inside an op-amp.
This is the model for an op-amp.
I don't want to lead any of you to think that this is literally what's in an op-amp.
This is not literally what's in an op-amp.
Here is much more literally what's in an op-amp.
This is the schematic diagram of a 709 which is a Widlar circuit.
It's a complicated transistor circuit.
It's ingenious.
This is how you build a circuit that has the remarkable property of the ideal op-amp circuit.
It's not perfectly obvious from here.
And even here this doesn't give Widlar enough credit.
Here's what he really did.
He designed masks to shield semiconductor materials so that you could turn them into transistors so that it would turn into an op-amp.
We're not going to worry about this.
This is two levels of abstraction more complex than we're going to worry about.
We're just going to say, I don't know what's in there, but this is the way it behaves.
This is intended to be a behavioral model for how an op-amp works.
And in fact, that's an important idea that we use circuits-- I use circuits on a daily basis not because I design semiconductor devices but because I work on biological issues.
I actually study hearing.
And we make circuit models for the way biological parts work.
And that helps us to understand the way the biological part works.
For example, here is a model taken from 6.021, where we try to understand how a nerve propagates an action potential by making a circuit model.
That's the same thing we're doing here.
We're making a circuit model for how the op-amp works.
it's not intended to be literally what's inside the op-amp.
It's intended to be a model that let's us think about the behavior of the op-amp.
This lets us understand now why it's different when you flip the wires.
Let's start with the original configuration where I put the voltage in the plus lead, and I wrap the output back around to the minus lead.
What's going to happen?
What's the op-amp actually do?
Imagine that things had been stable.
This was 0.
The output was 0.
Everybody was happy.
And now all of a sudden the input voltages steps up.
What happens?
The input voltage steps up.
The voltage source suddenly generates a huge positive voltage.
And that starts to put current into the capacitor that represents the voltage at the output of the op-amp.
So as a result of this voltage being higher than where it was, current is being dumped by the op-amp into this capacitor, and the output voltage starts to increase.
That's shown it red.
As the capacitor voltage gets bigger, as the output voltage increases, the difference between Vplus and Vminus gets smaller, and the rate at which charge is flowing into the capacitor slows.
And in fact, as the voltage at the output gets very close to the voltage at the input, the rate of current into the capacitor goes to 0.
And the output voltage stabilizes at the input voltage.
The same thing happens in reverse if I were to change input voltage and make it go negative.
If the input voltage went negative, then K times Vplus minus Vminus would be a big negative number and it would suck current out.
The voltage controlled voltage source would suck current out of the capacitor and make the voltage fall.
The voltage would fall.
The absolute difference between the plus and minus port would get smaller.
The current flowing would become less.
And it would again stabilize when the output is equal to the input.
Contrast that to what would happen if I put the input into the minus port.
If I put the input into the minus port, and the input goes through a step, then the input going positive makes the voltage controlled voltage source generate a big negative voltage.
It sucks current out.
So the input went positive.
And the output goes negative.
It goes the wrong way.
That's bad.
So here, if I put the input into the minus lead, a positive transition of the input leads to the output going the wrong way.
And as it goes the wrong way, the drive to make it go the wrong way gets bigger.
It's a runaway system.
It's positive feedback.
So the idea is that by supporting the input into the negative lead instead of into the positive lead, leads to a positive feedback situation in which a small change at the input makes the output go the wrong way, convinces the op-amp that things are getting worse, so it makes even more current, which makes it go even more the wrong way.
And the same thing happens with the flip situation.
The idea then is that by thinking about the flow of current-- What's the op-amp actually doing?
The op-amp is actually sensing the difference between the voltage at the positive port and the minus port, and it's sourcing current that changes the output voltage to go up or down.
You need to wire the op-amp so that ultimately the output equals the input.
Otherwise the ideal op-amp condition that Vplus equals Vminus will ever be attained.
So the left one is what we would refer to as a stable feedback situation.
You can think about that as analogous to thinking about a ball in a valley.
So we have a valley, and we have a ball.
The ball's going to roll down here eventually.
It's stable.
Bop it a little bit to the right, it'll roll back into the valley.
Bop it a little to the left, it'll roll back into the valley.
That's as opposed to, when we've wired up the wrong way, it's like we have an unstable equilibrium.
It's as though we're trying to put the ball there.
Bop it a little to right, it runs away to the right.
Bop it a little to the left, it runs away to the left.
That's unstable.
There is a metastability plane.
If you actually balanced it exactly, exactly, exactly at the right place, it would stay there.
That's what the solution is telling you.
That was the minus K over (1 minus K).
The fact that the equations had a solution is correct, it's just an unstable solution.
The tiniest little disturbance will make it go awry.
The idea then is that the ideal op-amp assumption is fine.
It's a good thing to use.
It doesn't tell you about the mechanism.
If you think a little bit more about the way the physics works, you have to wire the circuit up so that it has negative feedback in order to get a stable equilibrium.
So if you understand that, and you just make sure that you've hooked it up so that it has negative feedback, then you can use the ideal op-amp assumption.
Everything's just fine.
And in the interest of time, I'm going to just skip over this because we've already talked about all of this.
It's just another example.
The last thing I want to mention is just that in order to work this magic, the op-amp has to get power from somewhere.
And so that means that it's not a three-terminal device.
It's not the kind of device I've been drawing so far that has got power pins too.
The way the op-amp works is it takes power from those power pins to be able to force the output to a voltage that'll make the input the Vplus equal to Vminus.
That's the mechanism by which it works.
And that'll have important consequences when we build the robot head, because what it means is that we're not going to be able to generate arbitrary voltages at the output of an op-amp.
We're going to be limited to generating voltages that are between the power rails.
So if we were to supply plus and minus 10 Volts to the power pins, we would not be expecting to be able to generate 20 Volts at the output.
So that's an important implication when we do the design lab.
In summary, what we've done is we've showed how circuits can be a pain to make modular, because in principle, adding one component changes voltages and currents everywhere.
But there's a way using op-amps to have this buffering idea that lets us logically separate parts of circuits so the one can control the other.
Have a good day.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today I want to finish up thinking about circuits.
And the major topic for today is just one -- thinking about abstractions that we can use for thinking about circuits.
The attractions are things that capitalize on linearity.
And they include things like [UNINTELLIGIBLE] and superposition.
OK so what I want to do is finish up thinking about circuits.
And just to get you thinking about circuits, let's think about where we are.
Last time we saw that one of the issues in thinking about circuit design, is the fact that every part, in principle, interacts with every other part.
Which makes the design process harder than the design process was for things like linear systems, where we thought about boxes having inputs and outputs.
In linear systems, when we thought about boxes having inputs and outputs, the output didn't necessarily have any affect on the input, unless there was an explicit path feedback.
In circuits, there's feedback always.
There's no way to avoid it.
And in fact, that coupling can make thinking about the circuits very difficult.
And we introduce that idea by just thinking about, even if you wanted to close a switch, you know a circuit, that's logically the same as adding a new part.
And when you do that, that's going to change the currents and voltages everywhere.
So that's kind of a big thing-- that's kind of the thing that's different about circuits from what we've been thinking about before.
Because that complicates design, we'd like some way of dealing with it.
And the way we introduced, and the way that you used in the lab, so far, has been to use a buffer.
We can use an op-amp to make a buffer.
And a buffer has this nice isolation property.
In this particular circuit, it has the property that it copies this voltage to the output, regardless of what's at the output.
That means we could put a switch here, we could change the light bulb, we could do anything we wanted to, and we would still know, because of the properties of the op-amp, we would still know that this voltage is going to be 8 Volts, regardless of what we put there.
So that's very nice, that gives us a modularity, that gives us a way to design things.
It let's us design the left-- make a circuit that generates 8 Volts without knowing precisely what's going to be the ultimate circuit that that drives.
So that's nice, it allows us to do modularity.
And in many instances, that's a very good solution.
In fact in the solution-- in the problems that you are working on in lab, where you're trying to drive a motor-- that's an excellent solution.
It's not always an excellent solution.
In some sense, it's very expensive.
An op-amp is a complicated part.
If you were to look inside an op-amp, there's some two dozen transistors in most op-amps.
So it's not an inexpensive part, especially when you think about this kind of a circuit that only has three parts to the left.
The op-amp is actually a more complex device than anything else up.
So op-amps are wonderful, op-amps allow us to make buffers, buffers are wonderful, but they're not always the best solution for thinking about modularity.
And, in fact, there's other ways.
And so that's what we want to think about today is -- other ways for achieving modularity in circuit design.
And the key to thinking about this, is to think about, well, what's the worst thing that could happen?
If I changed this part arbitrarily, just how bad can things get?
So I'll let you answer that.
Think about that circuit, and assume that this is a 90 Volt power supply, 3 Ohm, 6 Ohm, but this can change-- R0 can change.
I've tabulated some values of R0 and putative, corresponding values for V0 and I0.
Are my putative answers right?
So take a minute, talk to your neighbor, figure out how many of these-- let's see-- ten blue numbers are right or wrong.
It's very quiet, you are allowed to talk to people.
So how many of the numbers are wrong?
So raise your hand, indicate by number fingers how many mistakes are in the table.
More votes, more votes-- come on, come on.
if you had talked more, you could blame it on your neighbor more easily, so talk quickly.
OK.
So does everybody agree with their neighbor?
OK, i don't see a single right answer.
So take 30 more seconds and think about it again.
I don't see any right answers.
So assume your answer's wrong.
[CHUCKLES] Color blind-- OK, so how many of the voltages and currents are incorrect?
OK, how about a re-vote?
So how many of the blue numbers-- the currents and voltages, how many of V0 and I0 -- how many of those are incorrect?
OK, not very many votes-- I'd say about 80% correct, now.
That's definitely an improvement.
What would have to be true?
If I wanted to prove that some of these numbers were wrong, what could I do?
Give me a condition that would have to be true if the numbers were correct
V0 has to equal I0?
V0 has to equal I0.
We know that this resistor better be-- better obey Ohm's Law.
That's what we mean by that symbol.
So it had better be the case that V0 is I0R0.
So you you'd want this number to be the product of that and that.
So 0 is the product of 0 and 30, that looks good.
30 is 2 times 15, 36 is 3 times 12, 48 is 6 times 8.
60 is-- well that's a little marginal.
how about if I rearrange it a little bit, and say what if R is V/I?
If I is 0, that would make the resistor infinity, so there's a way of thinking about that last line as correct.
That's a little funny.
Maybe the forward way of thinking about it is, well what if I made the resistor infinite?
if I made the resistor infinite, what would be I?
0.
And what would be V?
What would be the voltage difference if this were right, then?
The voltage divider--
Oh, yeah
So if there's no current here, then you can use the voltage relationship-- the voltage-divider relationship here.
6 over (3 plus 6) times 90 -- which is 60.
So if you use straightforward reasoning saying, what if the resistor was infinitely large?
What would happen?
Then you would conclude that the bottom line is OK.
If you choose-- if you show that R equals V over I, are you done?
What else has to be true in order for the numbers to be true?
It's not very hard
The voltages drop out of the [UNINTELLIGIBLE].
The resistor has to be the same as the--
So the voltages-- that's an instance-- what you just said-- is an instance of KVL.
Basically, the voltage around all of the loops better be 0.
The sum of the voltages around all of the loops better be 0, and the sum of the currents in all of those closed surfaces better be 0, right?
KVL better be satisfied everywhere, and KCL better be satisfied everywhere.
So, in particular, we can ask about this node.
We could say, do the currents flowing into that node sum to 0?
So if you take a line here-- let's take this line-- if V0 happened to be-- V0 happens to be the voltage here, right?
So if V0 were 36, then that puts some voltage on this leg.
36 would what, 54?
So then you'd have to say-- you'd have to compute, then, the current through here and the current through there, and see if those currents sum to 0.
And, in fact, if you do those calculations, you if you can convince yourself of those are all true.
So the answer was 0.
All of those answers were true.
The point of doing the exercise was to just remind you about how you solve circuits, but also to let us look at patterns.
The interesting thing in this problem is the pattern the results between the V's and the I's.
So if I were to make a plot, in fact, those V's and I's all fall on a straight line.
Well that's pretty interesting.
So if I plot just this V versus this I-- so V equals 0, 30, 36, 0, 30, 36, 48, 60 -- and the corresponding I's, 30,15,12,6,0 -- 30,15,12,6,0.
Those points all fall on a straight line.
That suggests that there's some pattern here.
And if there's a pattern, then there might be a way to exploit it.
So that's what I'm trying to develop-- is a way to exploit the pattern that results when parts of circuits interact.
The interesting thing that we-- so not only is it true that there's a simple pattern, but it turns out that the pattern is completely independent of the thing that I put on the right.
The pattern is a property of the circuit to the left.
One way to convince yourself of that is to substitute-- take that resistor out, and put in a voltage source, and redo the problem.
This time, instead of assuming that there's an Ohm's Law-type resistor here, assume there's a constant voltage, and that constant voltage is adjusted to 0,30,36,48,60.
If you re-solve that problem, you get exactly these same currents.
The answer continues to fall on exactly the same line.
So that's a very interesting pattern.
The idea is that when the circuit on the left interacts with the circuit on the right, regardless of what the circuit on the right is, you get a simple relationship between the voltage and current that comes out of the circuit on the left.
So that motivates the idea that we can think about the left-hand circuit as some kind of a generic part.
We call that generic part a one-port.
Think about this circuit on the left-- the thing that's in the red box-- it's got at its terminals-- the terminals are the things that poke through the red box.
First off it has two terminals, two terminals is just like all of our other parts.
And it's like a resistor, it's like a voltage source, it's like a current source, it's a two terminal device.
And just like a voltage source or a resistor or a current source, there's some voltage across those terminals, and there's some current that flows in those terminals.
So there's some current that goes in the plus.
And that same current comes out the minus.
That'll be true for this circuit, just the same as it's true for a resistor or voltage source or anything else, the only difference is, the thing that's inside the box-- the thing that's inside the one-port-- is more complicated than it was for a simple resistor or voltage source or current source.
So what we can think about, is this whole red box just looks like a super part.
So the interesting thing that happens is, this red box behaves like a one-port, like a super part.
And just like a resistor has a relationship between V and I, V equals IR, Or a voltage source has some kind of a constraint, V equals V0.
Or a current source has some kind of a constraint, I equals I0.
This funny part has a relationship between V and I that's given by that curve.
In some sense, it's not very different.
So we think that-- so what we want to do now is figure out the rules that govern the currents and voltages that flow through one-ports.
And in particular, how special was this straight line thing?
I mean if they were always a straight line, that would be really easy, right?
So the question-- so the next question I'd like to ask is, just how often are we expecting to see straight lines there?
I already said, the primitive elements that we think about-- Ohm's Law, resistors, voltage sources, current sources-- they have straight line constraints between the voltages and currents that they can generate.
So think about what I'm doing.
I'm trying to think about a rule that's going to let me describe the currents and voltages into that red box from the previous slide, much like I would describe the voltages and currents in an Ohm's Law a resistor.
I can tell you the voltage-current relationship, V equals IR for an Ohm's resistor, independent of what it's connected to.
That's the reasoning that I'm using here.
I'm going to try to figure out, independent of what it is connected to, what will be the voltage-current relationship for the red box?
So the question is, when are we expecting straight lines, and when are we not expecting straight lines?
So here's a simple circuit.
Here's a super part made out of one linear resistor, one Ohm's Law resistor, and one voltage source.
What's the current-voltage relationship for that part?
What if I put a red box around the whole thing, and asked you to draw the I-V curve, the current voltage curve?
Which would it look like, A,B,C,D -- which should have been (1), (2), (3), (4) -- so you can raise fingers, or none of the above?
So take 30 seconds.
Figure out what would be the I-V curve for this part.
So if you map A to D into (1) to (4) -- which of those plots describes the current-voltage relationship for that circuit?
Map A to D to (1) to (4).
OK, you're quiet, and the success rate is smaller than usual, about 50% correct.
Take 30 seconds, reconsider your answer, try to get this up to 85% or so.
[CHATTER] So which is the best plot?
Which plot best characterizes the circuit on the top?
OK, that's much better.
That's about 75%.
Certainly not 100%, but better.
OK so how do you think about this?
One way to think about it is special cases.
Can you think about any special cases that are particularly easy to check?
0 -- OK, what equals 0?
V or I --
V or I -- yes.
So what if you were to make -- if you were to make I equal to 0?
First off, if you made I equal to 0, what's special in the plots?
If I equals 0, then you round up x-axis.
Sorry, if you make I 0, what will V be?
If you made I be 0, how big will be V?
5 Volts, right?
If you make I be 0, then there's no voltage across the resistor.
So the total voltage here will be the same as the voltage across the voltage source, V equals 5.
So the intersection on the x-axis should be at the point V equals 5.
Makes sense?
Now if you said V equals 0, how big would I be?
If you set V equal to 0 -- how do you set V equal to 0?
Negative 2.5
Negative 2.5, so what is negative 2.5?
The value of
So do you need-- how do we set V equal to 0?
What's the circuit way of saying, set V equal to 0?
You set V equals 0 in a circuit by--
Grounding
--grounding, by setting it-- by putting in a wire.
So you run a wire from here to here.
Voltage across a wire is always 0, right?
So you set something to 0 by putting a wire across it.
We call that a short circuit.
OK, there's a short path for the voltage travel.
So you put a short circuit here, and then I becomes V over R, and the only trick is that it's up, right?
Current likes to flow down the electrochemical gradient.
So it likes to go down the electrical gradient.
So the current is going to go up, but the reference direction for this I is down, so I is minus 2.5.
So the intersection on this axis is minus 2.5.
So we know that it has to go through a negative on the bottom, and it has to go through a positive V on the x-axis, and the only curve that does that is (A).
And if we wanted to be a little bit more fancy, we could figure out the general rule.
We could just write an expression for Vr.
Well by KVL, Vr is always the difference between V and 5 Volts.
And we could write an expression for the current through the resistor.
That's just V over R. Vr is V minus 5, R is 2.
So I get some expression which is, lo and behold, linear in V. So just like that more complicated circuit that I'd looked at, I ended up with a straight line relationship in the current voltage-- the relationship between the current and voltage falls on a straight line.
So how special is that?
Well it actually happens pretty robustly.
Think about what would happen if I had two parts, the generically-- so I'm thinking about these being generic boxes, inside could be anything.
There could be a current source, a voltage source, a linear resistor, or some other one-port.
So if I had a generic box here, and a generic box here, and I connected them in parallel, under the condition that each of the generic boxes had a straight line current-voltage relationship, it's easy to argue that the resulting relationship between current and voltage for the parallel combination is also a straight line.
All you need to do is realize that if you hook two things in parallel, the parallel voltage is the same as V1 and V2.
So all the voltages are the same.
And the currents add.
So Ip, the current end of the parallel combination is the sum of the two I's.
So if you think about the relationship between V1 and I1, and the relationship between V2 and I2, you could derive the relationship between Ip and Vp by just adding.
V1 equals v2 equals Vp, that's what we saw here.
And the current at the bottom is the sum of the other two currents.
So, in particular, if this was a straight line, and that was a straight line, the sum of two straight lines is a straight line, right?
If you add two straight lines, you get a new straight line.
So what this shows is that if you started with parts that were themselves straight lines, parallel combinations would generate a new part with another straight line.
Same sort of thing happens if the two parts we're in series.
In series we would have Iseries -- it's the same as I1 equals I2-- so that's the equivalent of the y-axes, where previously we had equivalents of the x-axes.
And now if both of these boxes had linear I-V curves, I would now add horizontally rather than adding vertically.
But I have the same result.
If the two, individual one-ports had linear I-V curves, and if I add horizontally, I'll get a new linear I-V curve.
And in fact, if you start-- if you put any combination of parts that have linear I-V curves together to form a new circuit-- a new one-port, the I-V curve for that new one-port will be a linear function.
And the way to see that, is to think about linear equations.
Remember when we solve a circuit, we have to have one law for every element, Like Ohm's Law-- it's V equals IR, or voltage source is V equals V0 or something like that-- we need one law for every element.
And then we have KVL and KCL.
Well all of the-- if we start with the assumption that each component has a straight line relationship between voltage and current, which is true for linear resistors, voltage sources, and current sources.
Those equations that describe those parts are what we call linear equations.
Linear equation is an equation where the function of the unknowns is a quote -- linear function.
It has the form-- some constant times an unknown, plus some other constant times some other unknown, plus, in principle, any number of those.
But it has to be a linear function, so you're not allowed to have things like V-squared in there.
But if you think about things like voltage sources and current sources and Ohm's Law, they don't have squares in there, they're linear equations.
And the idea then, when we're solving for the I-V relationship for this new one-port thing, all we're doing is we're solving a system of linear equation.
Sort of in the abstract, the idea is that we write down all of the component equations inside the box, we write down all of the relevant KVL and KCL, and then we solve.
So they're all going to have-- each one of those equations is going to be linear, so it's going to be something like, say, a0x0 if x is my unknown.
'x' could be a current or a voltage.
And then I might have sum of x1, then I might have sum of x2, and I might have a whole bunch of things, and they all add together to be some constant, like that.
So that might represent part one-- that might represent one of the components in the box.
Then I would have another component, which would be some other linear equation.
Then I'd have a KCL equation, well KCL is easy, because that only has currents in it, and the multipliers are all 1 or minus 1.
Right, so that's clearly linear.
And I have KVL equations.
Those are linear.
And the point is, that when you solve linear equations, you get a new linear equation.
So think about solving this by the method of substitution.
I could figure out what is x0 here.
If I use this equation to figure out x0.
x0 would be a linear function of all the other x's.
So then if I plug that linear function into here, I replace this linear equation with a new linear equation with one fewer unknown.
if I keep doing that, I just keep replacing linear equations with other linear equations.
When I'm all done, I'm left with a new linear equation.
That's why, if you start with parts that linear I-V curves, you'll end up with a straight line I-V curve.
So that idea that the I-V function is a straight line is quite robust.
It will happen anytime you go to circuit out of ideal parts-- whereby ideal parts, I mean Ohm's Law resistors, voltage sources, and current sources.
So that has a very interesting circuit interpretation.
If I know that an arbitrary circuit can be represented by a straight line-- if the I-V curve can be represented by a straight line-- well that generates an equivalent set.
There's obviously more than one circuit that could generate the same straight line.
Here's a circuit that can generate that straight line.
In fact, it will always be true that I can generate a circuit with one voltage source and one Ohm's Law resistor that will mimic the behavior of any arbitrary combination of resistors, voltage sources, and current sources.
All I need to do is think about, you take the complicated thing, figure out its straight line plot, now you read off some critical numbers from this plot.
You say, OK, well what if the current were 0?
Well if the current were 0, I'd be on this axis.
If the current were 0 in the circuit, V would be V0.
So that means you look over here, you figure out the x-intercept, and the x-intercept is the value of the voltage source.
Similarly, if you figure out the rate of growth of I -- so more generally, if you solve for I, I will be the difference between V and V0 divided by R, that's just Ohm's law for the resistor.
And that then lets you figure out the law for the slope.
The slope over here is going to turn out to be one over this resistor-- I should have had-- this should be R0, these two should match.
So the slope of this line is one over that resistor.
So in general, regardless of how complicated the box is, figure out the straight line and I-V curves, read off the x-axis, read off the slope, and that lets you construct a simple circuit that has the same Vi curve.
We call that circuit the Thevenin Equivalent.
What that means is, you can think about a complicated circuit, regardless of how many parts, by a circuit it just has two.
That's an abstraction that lets us think more simply about complicated circuits even if there's no buffers.
This is true always.
OK, it's not true-- it's not it's not the case that if I change the thing that I put here-- if I change the thing I put here, this relationship is still true.
It's the equivalent of Ohm's Law for complicated circuits.
Ohm's Law is what I get if what's in the box is a single resistor.
If what's in the box is complicated, more generally this is what I get, the Thevenin Equivalent.
Of course there's lots of circuits that have this I-V curve.
So a different one is a current source with a resistor.
That's called a Norton Equivalent.
You do the same sort of thing.
What would happen if here, if I set V to be 0?
Well how do I make V be 0?
I make V be 0 -- Did I write that on the slide?
no I didn't write it on the slide.
I make V be 0 by putting a short circuit here.
I connect a wire.
If I put a wire here, then how big is the current I?
Well if I put a wire here, it's easier for this current to go through the wire than it is to go through that resistor.
So all of this current goes through the wire.
All of this current is I0.
So if I put a short circuit across V -- if I make V be 0, how big is i?
Minus I0 right?
All the current I0 goes through, it just goes to backwards.
So that's how I get this point.
So if I wanted to replace some complicated circuit with a Norton Equivalent, I would take the complicated system, to figure out the I-V curve, read off the intercept on this axis, change the sign-- that's the most confusing part, by the way.
This is the error that you all make.
There's a minus sign in the current relationship.
We like to draw-- it just makes us feel good to have the arrow go up.
And if you make the arrow go up, it's in the wrong direction to I.
So when we did the Thevenin equivalent, there was no sign from it, this was V0.
So the voltage on the voltage source is equal to the x-intercept.
But when we do the Norton, the current in the current source is minus the y-intercept.
Same idea, though.
OK?
So the idea is that Thevenin and Norton equivalent circuits are equivalent, in the sense that they generate the same voltage.
that the more complicated circuit did.
So that means for thinking about the circuit we can ignore the complicated stuff, and just know two numbers, V0 and R0, or I0 and R0.
So one more step, what this all means is, that if you can represent the current voltage relationship for an arbitrary circuit in terms of a straight line, that means that when we're trying to characterize an arbitrary circuit, it doesn't matter how complicated it is.
It could have 100 parts in it.
Regardless of how complicated it is, I only need to measure two things in order to fully characterize it.
If a circuit is made out of linear resistors, voltage sources, and current sources, three special linear parts-- if a circuit is composed entirely out of linear parts, doesn't matter how many parts are in it, there could 100, there could be 1,000-- I only need to measure two things to get a complete description.
And that's because two points determine a straight line.
I know, by having proved it by using linear algebra, I know by using linear algebra that the solution is a straight line.
I know from geometry that two points determine a straight line.
I only need to find out two points.
So by convention, the easiest two points is usually the simplest cases.
Set the voltage to 0 and set the current to be 0.
So that motivates the idea that if I have an arbitrary circuit, if I want to figure out this reduced complexity abstraction-- say I want to make a Thevenin Equivalent-- what I would do is first to ask the question, how big is the open circuit voltage?
Open circuit means there's no connection between the terminals.
That means the current is 0.
If the current is 0, I'm on the x-axis.
So over here I'm on the x-axis, so I'm thinking about the red point.
Open circuit over here means there's no connection here, which means the current is 0, and all I need to ask is-- regardless of how complicated it is the circuit-- how big is the voltage that I would measure here?
So in this circuit, if I have a-- yeah, I confused myself for a moment.
if this current is 0, then this voltage drop is 0.
This voltage source prescribes the voltage between these nodes to be 1 Volt, so V0 is 1 Volt.
So I just fell one point.
I set I to be 0, I found the open circuit voltage.
Then I need to find one other point, because I only to find two.
So I'll find the short circuit current.
Imagine that I put a wire between the input and the output, and I'll compute how much current that circuit generates in that wire.
Again the only confusing part is that the reference directions are backwards to the way you might have expected them to be.
If I put a short circuit here, then the voltage between these nodes is still 1 Volt.
That's what the voltage source always says.
So the current that flows through this wire, the short circuit current, is just this V over that R, except it's in the negative direction.
And the way you can think about that is that the slope of this curve has to be positive.
I need-- because of the way Ohm's Law works-- increasing the voltage better tends to increase the current, because that's what Ohm's Law resistors do.
So I need to have this slope be positive if it's going to be Ohm's Law.
So I characterize just those two points, and then the resistance is simply the ratio of the two.
So the resistance is related to the slope.
It's 1 over the slope, you don't need to worry about that.
The resistance is always V over I, so you just take the open circuit voltage and divide by the short circuit current, minus sign, and you get that the resistor must be 2 Ohms.
Is that all clear?
The idea is that we're trying to build an abstraction that lets us simplify the way we think about circuits, without introducing buffers.
So that means then that these two circuits are equivalent to that circuit, in the sense that they all share the same I-V curve.
If you substituted one server for the other, you couldn't tell from outside the red box which was on the inside of the red box.
OK so I'll do an example now.
Think about, what if I want to find the equivalent-- the Thevenin equivalent for this circuit.
I just do the things I just told you to do.
First thing I think about is, what's the open circuit voltage?
So If I think about open circuit, there's no connection here.
That means this current is 0.
That means that the voltage that develops is the voltage divider.
So I get 7 and a 1/2 volts-- 3 over (1 plus 3) times 10, right.
So that gives me one point-- that tells me the voltage source for the Thevenin equivalent.
Then for the second point, I want to think about the short circuit current.
So I consider putting a wire here, and then I compute the amount of current that flows in that wire.
And in this circuit, this wire shorts out that resistor, so all the current goes through this wire, and none of the current goes through that resistor.
So that means that the total current that flows is 10 Volts divided by 1 Ohm, which is 10 Amps.
And then I know that the equivalent resistance is the ratio of the open circuit voltage to the short circuit current, except I have to worry about the minus sign.
And so I end up with the equivalent resistance being the 7.5 Volts, which is the open circuit voltage, divided by 10 Amps, so I get 7.5 Ohms.
And so the answer then, is that here is the circuit I started with, here's the Thevenin equivalent circuit, they're identical in the sense that they have the same I-V curve.
And you can just sort of see why that has to be true.
If you think about, here's the two circuits, and if you think about the simple cases, if you set I to be 0, the voltages better be the same.
Well over here it's 7.5 by the voltage divider, over here it's 7.5 by the fact that there's no current going through that resistor.
And the short circuit current better be the same.
So over here, if you short this out, you're going to get 10 Amps, over here if you short it out, you're going to get 10 Amps.
So you can always go back and forth.
The point is, that you can substitute the simpler circuit for the more complex circuit, because they have the same I-V curve.
OK, just to make sure that you're following me, here's a question that has to do with taking the same circuit, but considering the Thevenin equivalent-- Thevenin or Norton equivalent-- at three different ports.
Here I'm thinking about what would happen if I looked in terminal A, what if I looked in terminal B, or looked in terminal C?
Figure out the Thevenin and Norton parameters, and see if there's an error in the table.
So how many errors are in the table?
About 50% correct again.
So which entry don't you like
2D?
That's exactly right, so 2D is wrong.
How do I figure out 1A?
How do I figure out V0 for the a circuit?
So the definition of V0 is the open circuit voltage.
So I need to figure out, for the a circuit, how big would be the voltage across A if there was no current flowing in the leg of A.
Everybody clear on that?
So the thing that I would do is, I would think about, there's no connection here.
What's the voltage here?
So how would I calculate that?
Yeah
You would use the current-divider relationship to figure out that current flowing through that is going to be 4 Amps
Precisely, so first I need to take-- here are the current sources, so I have two resistor lengths, so that's a perfect set up for the current divider.
So the amount of current that goes in this leg compared to that leg is the ratio of this resistance to the sum of resistances, right?
And if you work that out, you're going to get 4 Amps coming through here.
So then after you know the current through this leg, it's an easy matter to take the current and turn it into a voltage, so the voltage at the a port is 4 Amps times 5 Ohms is 20 Volts.
Is that clear?
And similarly, but with a different answer, the voltage at the B terminal is the same 4 Amps, but now times 10 Ohms.
So that's how we got 40.
And at C, it's the same 4 Amps, but now it's times the sum, 4 times 15 is 60.
Everybody's happy about that?
So the point is that when you generate an equivalent circuit, it depends upon which set of terminals you're using.
You can't just take a circuit and say, give me the Thevenin Equivalent.
You have to say, give me the Thevenin equivalent looking somewhere.
So I could look in the A port, the B port, or the C port, and I get different V0's.
How would I compute the I0?
Short circuit current-- so what I would do is I would short this out.
When I do that, all the current flows in this leg and one of the current flows that leg.
So that means I have, equivalently, 10 Ohms in parallel with 10 Ohms.
Then by the current divider, how much current goes down one leg?
Half of it.
I get a different answer over here, because now I short out that node, which shorts out that resistor, so now I get a different ratio of resistors.
It's not the same as the first, so I know that this answer can't be 5.
And, in fact, if you work it out, the answer is 20 over 3.
And finally, if you short here, then you know that that short circuit shorts out both this series combination and that one.
So all of the 10 Amps goes through that, so you get that 10 Amps.
Then how do you get R0, which is the ratio of V0 over I0.
So if you take V0 over I0, you get 4.
Here if you do right answer, you get 6, and here if you do that you get 6.
The point is, that the Thevenin equivalent you get is different.
The Norton equivalent that you get is different, depending on which ports you're looking at.
OK there's two reasons for thinking about this.
One-- so why am I thinking about all of these equivalent circuits?
So I wanted to have an abstraction that was useful for thinking about how parts interact.
I wanted a way of thinking about what would happen if I changed the load on the circuit without having to recalculate all the voltages and currents throughout the circuit.
And so this Thevenin and Norton idea is a way of doing that.
That's important from a practical sense, because when you buy a part-- when you buy an electronic part, they tell you how it works by telling you the Thevenin equivalent or the Norton equivalent, or whatever is the easy way to think about it.
So there's a practical reason-- when you buy an op-amp, they tell you how good the op-amp is by telling you how big is the equivalent resistance at the output.
So it has a practical value, because it lets you-- it's the way you specify an electronic part.
You can't -- when the manufacturer makes a part, they can't know what you're going to do with it.
So they tell you how it works by telling you something about the equivalent circuit.
There's also a different reason for thinking about this, and that is because it's conceptually simplifying to think about Thevenin and Norton equivalents.
So here's an example that's very much like the first problem that I worked out.
What would be the effect of closing this switch on the current I?
We solved the problem very much like this last time, and one way you can solve it is you figure out I in two cases, when the switch is open and when the switch is closed.
And you figure out whether went up or down, and you know the answer.
The point is, that if you think about this in terms of equivalent circuits, it's completely trivial.
If I think about what would happen-- I'm interested in what happens at I, what I do is split the circuit into two pieces, the stuff to the left of I, and the stuff to the right to of I. I make a Thevenin equivalent for both of them, and then I jam together the two Thevenin equivalents.
What I get, in detail, is showed here.
So if I look left, I see a 20 Volt source with a 4, 4, and a 2.
That has a Thevenin equivalent that is showed here, which is independent of the state of the switch.
So the same Thevenin occurs on the two sides, switch open, or switch closed.
If i make a Thevenin over here, there's no sources.
So it's just going to be a resistance.
So when the switch is open, I have to 2 Ohms, when the switch is closed, I have 1 Ohm.
I don't even need to figure out what are these part values to see that, if I make this resistance smaller, which happens if I close the switch, the current goes up.
I mean it's true that I showed here that the Thevenin voltage is 10 and the Thevenin resistor is 4.
I don't even care.
Regardless of what the Thevenin voltage was, regardless of what the Thevenin current is, it's going to be the same when the switch is open and closed.
And that's a powerful statement.
I know it's the same, so when I close this which, all I've really done is I've made that resistor smaller.
The net series resistance is down, the current is up.
So I have two reasons for thinking about Thevenin and Norton equivalence.
One is practical value, the other is conceptual simplicity.
It lets you simplify way you think about a circuit, and it's a way of gaining intuition without ever even solving the equations.
Most circuit designers don't solve certain equations.
They know what it's going to do just by looking at it, and this is the kind of reasoning that they use.
OK, there's one more topic-- the idea of Thevenins and Nortons really derived from linear algebra.
The basic parts of most interest are linear.
And when you put together a system out of linear parts, you get a linear system.
There's one more consequence of linearity that is terribly useful, and that's the idea of super position.
If you have a system of equations that is linear, and if you have multiple sources, things are very simple.
You can see that over here.
If I had multiple sources, the system of equations that I get wouldn't look quite like this.
These terms, the constant terms, they're the ones that come from the drives, the voltage sources, the current sources.
So if I had two drives, I would get something else here.
I would get plus a at N+1 for example-- source one, source two.
And I would get source one, source two.
So the idea is that, if I have a system that has multiple sources, I know from the structure of the linear equations, there is just more constant terms.
OK, well what's that mean?
That means you can just use linear algebra to see that the answer to this problem is the sum of the answers to that problem plus the answers to that problem.
That's just linear algebra.
What that means in terms of circuits is, I can figure out the response to a circuit by turning on the sources one at a time.
That's called superposition.
And generally speaking, it's a lot easier than solving the circuit out the long way.
So here I've got two sources So say I wanted to compute I, in response to V0 and I0.
What I would do, is I would turn off the I0 source, and calculate the voltage that results just from the voltage source.
That's the same as setting this to 0 and finding the response to this.
So if I want to set I to 0, the way you set I to be 0 is open circuit.
If you open circuit something, there's no current going to flow through it.
So I replace the current source with an open circuit, and compute the i that would be result when the current source isn't there.
OK, well that's easy.
If their current source we're not there, this would be open circuit.
The current, i, would just be the total voltage going through R1 and R2.
So the answer, I1, the first component of the current I, would be V0, the result of the voltage source divided by the sum of R1 and R2.
Now that's not the whole answer.
That would be the whole answer if I0 weren't there.
So now I have to worry about the other case.
What if only I0 were there?
Well now I have to set V to be 0.
Well setting V to 0 is not open circuiting it.
You can't just reach in, grab the voltage source, and throw it away, because that'll make the current 0.
If I want the voltage to be 0, I have to short circuit it.
So what I do then, is I leave the current source alone, and I replace the voltage source with a short circuit, guaranteeing that V is 0.
Then I ask, how big is I2, the component of I that results from the current source?
Well if I've short circuited this, then I just get a current divider.
This current has to do with how readily the current divides between R1 and R2.
The amount that goes through the R1 side is in proportion to R2 make R2 bigger, more of it goes through R1.
Standard current divider, except for the slippery minus sign.
So if I only had the current source, the current I2 would have been current divider operating I0 with the slippery minus sign.
So that means, by superposition, that the result of having both sources on is just the sum of those answers.
That's a very big simplification.
If you've got multiple sources in the circuit, you can think about them all at once.
And if you really good at writing linear equations and solving them with pencils, you'll get the right answer.
But there's an enormous simplification if you just turn off all but one, and do them one at a time.
So here's a problem.
Here's a very simple circuit, compute V by using super position.
V is the voltage across the resistor.
How big would V be if you used the idea of super position?
So what's the answer?
50% correct, roughly speaking.
OK I want to use superposition.
So how big would the voltage v be, if all I had was the voltage source?
One.
How big would the voltage be if i only had the current source?
Ah, got half of you.
That explains the 50% correct.
How big would the voltage be, if I only had the current source?
It's tempting to say (1).
I say tempting, because that's the wrong answer.
[LAUGHTER] What is wrong about the answer (1)?
What does the voltage do to the current source?
The voltage due to the current source is the voltage that the current source would have generated if the voltage source weren't there.
If the voltage source weren't there, then-- half of you got it right, so half of you can shout the answer.
So if the voltage source weren't there, the voltage source would be 0.
If the voltage source were 0, then V would be 0.
So if the voltage due to the voltage source is 1, the voltage due to the current source is 0.
The sum of the two is 1.
The answer is 1.
OK?
Make sense?
So, very closely related problem.
What's the current I?
Solve that by superposition.
So how big is the current I according to superposition?
Wonderful, so how big is the current i generated by the voltage source?
1.
How big is the I generated by the current source?
Negative 1, the sum is 0.
All right, so those two problems were trivial by superposition.
They're not too hard by non-superposition.
But the point is, that they're trivial by super position.
So what we saw today-- the goal for today was to generate some abstractions that let you think about the way parts interact with each other, because that's a central issue when you're thinking about circuit design And we built the idea of Thevenins and Nortons and super positions.
That was the basic idea.
And the sub-theme-- or maybe I should say the major theme is really that.
It's the importance of linear algebra.
So the take home message is probably, take 18.06.
So this was all about the application of 18.06 to the solving of circuits.
See you later.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I'm starting a new topic and that's always the occasion for putting things into perspective.
Keep in mind what we were trying to do in the subject.
We were trying to introduce several intellectual themes.
The first, and absolutely the most important, is how do you design a complex system?
We think that's very important because there's absolutely no way this department could exist the way it does, making things like that, hooking up internets and so forth.
Those are truly complex systems.
And if you didn't have an organized way of thinking about complexity, they're hopeless.
So the kinds of things we're interested to teach you about are just hopeless if you can't get a handle on complexity.
So that's by far the most important thing that we've been thinking about.
We've been interested in modeling, and controlling physical systems.
I hope you remember the way we chased the robot around the lab, and that was the point there.
We've thought about augmenting physical systems by adding computation, I hope you've got a feel for that.
And we're going to start today thinking about how do you build systems that are robust.
So just in review, so far-- you've already seen most of this-- so far we've taught you about abstraction, hierarchy, and controlling complexity starting primarily by thinking about software engineering.
Because that's such a good pedagogical place to start.
We introduced the idea of PCAP, and that has continued throughout the rest of the subject, then we worried about how do you control things.
We developed ways of modeling so that you could predict the outcome before you actually built the system.
That's crucial.
You can't afford to build prototypes for everything, it's just not economical.
And so this was an exercise in making models, figuring out how behaviors relate to the models, and trying to get the design done in the modeling stage rather than in the prototyping stage, and you built circuits.
This had to do with how you augment a system with new capabilities, either hardware or software.
Today what I want to start to think about is, how do you with uncertainty?
And how do you deal with things that are much more complicated to plan?
So the things that we will do in this segment are things like mapping.
What if we gave you a maze-- you, the robot.
What if we gave the robot a maze and didn't tell them the structure of the maze?
How would it discover the structure?
How would it make a map?
How would it localize?
What if you had a maze-- to make it simple, let's say that I tell you what the maze is.
But you wake up-- you're the robot, you wake up, you have no idea where you are.
What do you do?
How do you figure out where you are?
That's a problem we call localization.
And then planning.
What if you have a really complicated objective?
What's the step-by-step things that you could do to get there?
Those are the kinds of things we're going to do, and here's a typical kind of problem.
Let's say that the robot starts someplace, and say that it has something in it that lets it know where it is, like GPS.
And it knows where it wants to go.
Making a plan is not very difficult, right?
I'm here and I want to go there, connect with a straight line.
And that's what I've done here.
The problem is that unbeknownst to the robot, that path doesn't really work.
So on the first step, he thinks he's going to go from here to here in a straight line.
The blue represents the path that the robot would like to take, but then on the first step the sonars report that they hit walls.
And those show up as the black marks over here.
So already it can see that it's not going to be able to do what it wants to do.
So it starts to turn and it finds even more places that don't work.
Try again.
Try again.
Notice that the plan now is, well I don't know what's going on here, but I certainly can't go through there, so I'm going to have to go around it.
Keep trying.
Keep trying.
Notice the plan.
So he's always making a plan that sort of make sense.
He's using for each plan the information about the walls that he's already figured out.
And now he's figured out, well that didn't work.
So now back track, try to get out of here
Is he backtracking right now?
Or is he--
Well, he's going forward.
He's making a forward plan.
He's saying, OK now I know all these walls are here, and I'm way down in this corner, how do I get on the other side of that wall.
Well given the information that I know, I'm going to have to go around the known walls.
So my point of showing you this is several fold.
First off it's uncertain.
You didn't know at the outset just how bad the problem was.
So there's no way to kind of pre-plan for all of this.
Secondly, it's a really hard problem.
If you were to think about structuring a program to solve that problem, in a kind of High School programming sense-- if this happens then do this, if this happens then do this-- you would have a lot of if statements, right?
That's just not the way to do this.
So what we're going to learn to do in this module is think through much more complicated plans.
We're going to be looking at the kind of plans like shown here that just are not practical for, do this until this happens, and then do this until this happens, and then while that's going on, do this.
It's just not going to be practical, that's the idea.
So the very first element, the thing that we have to get on top of first, is how to think about uncertainty.
And there's a theory for that and the theory is actually trivial, the theory is actually simple, except that it's mind-boggling weird that nobody can get their head around it the first time they see it.
It's called Probability Theory.
As you'll see in a minute, the rules are completely trivial.
You'll have no trouble with the basic rules.
What you will have trouble with-- unless you're a lot different from most people-- the first time you see this theory it's very hard to imagine exactly what's going on.
And it's extremely difficult to have an intuition for what's going on.
So the theory is going to give us a framework then for thinking about uncertainty.
In particular, uncertainty sounds uncertain.
What we would like to do is make precise statements about uncertain situations.
Sounds contradictory, but we'll do several examples in lecture and then you'll do a lot more examples in the next week.
So that you learn exactly what that means.
We would like to draw reliable inferences from unreliable observations.
OK, you have a lot of experience with unreliable observations, right?
The seminars don't tell you the same thing each time.
That's what we'd like to deal with.
We would like to be able to take a bunch of different individually, not all that reliable observations, and come up with a conclusion that's a lot more reliable than any particular observation.
And when we're all done with that what we'd like to do is use this theory to help us design robust systems.
Systems that are not fragile.
Systems that are not thrown off track by having a small feature that was not part of the original formulation of the problem.
So that's the goal.
And what I'd like to do is start by motivating it with the kind of practical thing to get you thinking.
So here's the game, Let's Make a Deal.
I'm going to put 4 LEGO bricks in a bag.
OK. LEGO bricks, you've seen those probably.
Bag.
The LEGO bricks are white or red.
There's only going to be 4, and you're not going to know how many of each there is.
Then you get to pull one LEGO brick out, and if you pull a red one out, I'll give you 20$.
The hitch is you have to pay me to play this game.
So the question is, how much are you willing to pay me to play the game?
So, I need a volunteer.
I need somebody to take 4 LEGOs and not let me see, OK, please, please.
I want you to put 4 LEGOs, only four.
They can be white or red.
If you have LEGOs in your pockets that are a different color, don't use them.
You're allowed to know what the answer is but you're not allowed to tell me, or them.
So OK well come over here.
So bag, LEGOs, hide, put some number in.
Oh, no no no.
Wait, wait, wait.
Put them back, put them back.
I'm not supposed to see either.
OK, I'll go way.
OK, 4.
OK, so we'll close the bag, right?
And I'll call you back later, but it'll be nearer the end of the hour.
So here's 4 LEGOs, sort of sounds like 4 LEGOs, it's more than one.
OK, so how much would you be willing to pay me to play the game?
5$
5$ -- Can I get more?
I want to make money.
Can I get a higher bid?
More than 5$
$9.90
How much?
$9.90
$9.90, very interesting.
Can I get more than $9.90?
$9.99 and a half
$9.99 and a half?
Magic number
10$
10$.
Can I hear even a penny more?
A penny more?
I'll offer a penny more.
You just have to go to the bag
I thought we were being very careful and letting them not know
No, no, no.
Aren't you going to put 4 white blocks in all the time?
I didn't do it.
That person did it.
It wasn't me.
I'm innocent.
I'm completely fair.
Yeah?
Are we imagining that you are equally as likely to put any number of blocks in?
So, are we able to say that she's more likely to put it all white?
Because that just changes how you calculate it
OK, that's an interesting question.
We need a model of a person.
That's tricky.
OK, I have another idea.
Two more volunteers.
OK, volunteer, volunteer.
Here's the experiment one person will hold the bag up high, so that the other person can't see it, and the other person-- I didn't look in, notice I'm being very careful, I'm very honest, right?
Except for the X-ray vision, which you don't know about.
Everything is completely fair.
And the little window in the back you don't know about that either.
So, one person holds it up so the other person can't see in, the other person grabs a LEGO and pulls it out and lets everybody see that LEGO.
It was intended to make it hard to see in.
OK, red one.
OK, that's fine, so we're done
We each should get $20, right?
No, no, no.
This was a different part of the bet.
No, no, no, no, no.
Thank you, thank you.
Now how much would you pay me to play the game?
With that one out?
No, we'll put that one back.
OK, so this one came out, it was red.
Now without looking, I'm going to stick it back in.
OK, so we pulled it out.
So what do we know?
We know there's at least 1 red.
OK, now what are you willing to pay to play the game?
5$
5$..
Yes?
5$
$4.99
$4.99?
Wait a minute.
Should you be willing to pay more or less?
I got it up to 10$.
Should you be willing to pay more or less now?
More
More, why?
The same.
More.
The same
You're insured that there's at least 1 red block
I know that there's at least 1, but didn't I know that before?
No.
The person could have been e that first person could have loaded it, because I was giving her a cut.
I didn't talk about this before.
This is not a set-up.
So I want to vote.
How many people would give me less than 10$?
I'm going to give you [UNINTELLIGIBLE] first.
10 to 12.
Let's see, 13 to 15, 16 to 18, more than 18.
So how many people would give me-- you're only allowed to vote once.
Keep in mind that I'm more likely to choose you if you vote high.
Right?
Vote high.
So how many people would give me less than $10 to play the game?
A lot, I would say 20%.
How many people would give me between $10 and $12?
A lot smaller, 5%.
How many people would give me between $13 and $15?
Even smaller, 2%.
How many people would give me between $16 and $18?
Wait, these numbers are not going to add up to 100%.
OK, we'll learn the theory for how to normalize things in a minute.
OK, so we're down to about 1%.
How many people would give me more than $18?
One person.
Thank you, thank you.
So that's 1 in 200 or 0.05%.
OK, so what I'd like to do now is go through the theory that's going to let us make a precise calculation for how much a rational person-- not to say that you're not rational-- but how much a rational person might be willing to pay.
So that was the set up, then we'll do the theory, then we'll come back at the end of the hour and see how many people I would have gypped-- made money, or whatever.
OK, so we're going to think about probability.
And the first idea that we need is set theory.
Because we're going to think about experiments having outcomes, and we're going to talk about the outcomes being an event.
An event is any describable outcome from an experiment.
So for example, what if the experiment were to flip 3 coins in sequence.
An event could be head, head, head.
And you could talk about was the outcome head, head, head.
The event could be head, tail, ahead.
The event could be 1 head and 2 tails.
The event could be the first toss was a head.
So the idea is there's sets that we're rethinking about.
And we're going to think about events as possible outcomes being members of sets.
There's going to be a special kind of event that we're especially interested in, and that is an atomic event.
By which we mean finest grain.
Finest grain is kind of amorphous idea.
What it really means is for the experiment at hand, it doesn't seem to make sense to try to slice the outcome into two smaller units.
You keep slicing them down until slicing them into a smaller unit won't affect the outcome.
So for example, in the coin toss experiment, I might think that there are 8 atomic events.
Head, head, head, head, head tail, head, tail, head, head, tail, tail, blah, blah, blah.
So I've ignored some things like, it took 3 minutes to do the first flip, and it took 2 minutes to do the second one.
Right?
That's the art of figuring out what atomic units are.
So for the class of problems that I'm thinking about, those things can be ignored so I'm not counting them.
But that's an art, that's not really a science.
So you sort of have to use good judgment when you try to figure out what are the atomic events for a particular experiment.
Atomic events always have several properties, they are always mutually exclusive.
If I know the outcome was atomic event 3, then I know for sure that it was not atomic event 4.
And you can see that these events up here don't have those properties, right?
So the first toss-- here's an event head, head, head, which is not mutually exclusive with the first toss was a head.
So atomic that events have to be mutually exclusive.
Furthermore, if you list all of the atomic events, that set has to be collectively exhaustive.
Collectively exhaustive?
What buzz words?
OK, that means that you've exhausted all possibilities when you've accounted for the collective behaviors of all the atomic events.
And we have a very special name for that because it comes up over, and over, and over again.
The set of atomic events, the maximum set of atomic events, is called the sample space.
So the first thing we need to know when we're thinking about probability theory, is how to chunk outcomes into a sample space.
Second thing we need to know are the rules of probability.
These are the things that are so absurdly simple, that everybody who sees these immediately comes to the conclusion that probability theory is trivial, they then don't do anything until the next exam, and then they don't have a clue what we're asking.
Because it's subtle, it's more subtle than you might think.
Here's the rules, probabilities are real numbers that are not negative.
Pretty easy.
Probabilities have the feature that the probability of the sample space is 1.
That's really just scaling.
That's really just telling me how big all the numbers are.
So if I enumerate all the possible atomic events, the probability of having one of those as the outcome of an experiment, that probability is 1.
Doesn't seem like I said much, and I'm already 2/3 of the way through the list.
Yes?
Doesn't that just mean that something happened?
Something happened, yes.
And we are going to say that this certain event has probability 1.
All probabilities are real, all probabilities are bigger than 0, and the probability of the certain event-- written here as the universe, the sample space-- the probability of some element in the sample space is 1.
The only one that's terribly interesting is additivity.
If the intersection between A and B is empty, the probability of the union is the sum of the probabilities of the individual events.
Astonishingly, I'm done.
And this doesn't alter the fact that people are still, to this day, doing fundamental research in probability theory.
There are many subjects in probability theory, including many highly advanced graduate subjects, all of which derive from these three rules.
It's absurd how un-intuitive things can be given such simple beginnings.
Just as an idea.
So you can prove all of the interesting results from probability theory-- you can prove all results from probability theory with these three rules, and here's just one example.
If the intersection of A and B were not empty, you can still compute the probability of the union, it's just more complicated than if they were empty, if the intersection were empty.
Generally speaking, the probability of the union of A and B, is the probability of A plus the probability of B, minus the probability of the intersection.
And you can sort of see why that ought to be true, if you think about a Venn diagram.
If you think about the odds of having A in the universe-- the universe is the sample space-- probability of having sum event A, the probability of having sum event B, the probability of their intersection.
If you were to just add the probability of A and B, you doubly count the intersection.
You don't want to double count it, you want to count it once.
So you have to subtract one off.
So that's sort of what's going on.
OK, as I said the theory is very simple.
But let's make sure that you've got the basics first.
So experiment, I'm going to roll a fair, 6-sided die.
And I'm going to count as the outcome the number of dots on the top surface, not surprisingly.
Find the probability that the roll is odd, and greater than 3 You have 10 seconds.
OK, 10 seconds are up.
What's the answer?
(1), (2), (3), (4) or (5)?
Raise your hands.
Excellent, wonderful.
The answer is (1).
The way I want you to think about that is in terms of the theory that we just generated because it's useful for developing the answers to more complicated questions.
In terms of the theory, what we will always do, the process that always works, is enumerate the sample space.
What's that mean?
That means identify all of the atomic events.
The atomic events here are the faces that show are 1, 2, 3, 4, 5, 6.
Enumerate the sample space.
And then find the event interest.
So here the event was a compound event.
The result is odd and greater than 3.
Odd, well that's 1, 3, 5, shown by the check marks.
Bigger than 3, that's the bottom 3 check marks.
If it's going to be both, then you have to look where there's overlap and that only happens for the outcome 5.
Since there's only 1, and so fair meant that these probabilities were the same.
If you think through the fundamental axioms of probability, if they're equal, they're all non-negative real numbers, and they sum to 1, then they are all 1/6.
So the answer is 1/6, right?
OK, that was easy.
The rule that is most interesting for us, happens not surprisingly to also be the one that people have the most trouble with.
Not excluding the people who originally invented the theory.
The theory goes back to Laplace.
A bunch of people back then who were absolutely brilliant mathematicians, and still it took a while to formulate this rule.
It was formulated a guy named Bayes.
Bayes' theorem gives us a way to think about conditional probability.
What if I tell you, in some sample space, B happened?
How should you relabel the probabilities to take that into account?
Bayes' rule is trivial, it says it if I know B happened, what is the probability that A occurs, given that I know B happens?
And the rule is, you find the probability of the intersection
How do you do that?
We'll do some examples.
So we need to find the probability of the intersection, and then we have to find the probability of B occurring, and then we normalize-- a word I used before, and that's exactly what we need to do to that distribution-- we normalize the intersection by the probability of B.
That's an interesting rule.
It's the kind of thing we're going to want to know about.
We're going to want to know-- OK, I'm a robot.
I'm in a space.
I don't know where I am.
I have some a priori probability idea about where I am, so I think I'm 1/20 likely to be here, I'm 1/20 likely to be there, et cetera, et cetera.
And then I find out the sonars told me that I'm 0.03 meters -- no it can't be that small, 0.72 meters from a wall.
Well, how do I take into account this new information to update my probabilities for where I might be?
That's what this rule is good for.
So here's a picture.
The way to think about the rule is if I condition on B, if I tell you B happened, that's equivalent to shrinking the universe -- the universe U, the square.
That's everything that can happen.
Inside the universe, there's this event A and it does not occupy the entire universe.
There is a fraction of outcomes that belong logically in not A. OK?
That's the part that's in U but not in A.
Similarly with B.
Similarly there's some region, there's some part of the universe where both A and B occur, the intersection of the two occurred.
So what Bayes' theorem says is, if I tell you B occurred, all this part of the universe outside of B is irrelevant.
As far as you're concerned, B's the new universe.
Notice that if B is the new universe, then the intersection-- which is the part where A occurred-- is bigger after the conditioning then it was before the conditioning.
Before the conditioning the universe was this big, now the universe is this big.
The universe is smaller, so this region of overlap occupies a greater part of the new universe.
Is that clear?
So when you condition, you're really making the universe smaller, And the relative likelihood of things that are still in the universe, seem bigger.
So what's the conditional probability of getting a die roll greater than 3, given that it was odd?
Calculate, you have 30 seconds.
This is three times harder.
OK, what's the probability of getting a die roll greater than 3, given that the die role was odd?
Everybody raise your hands.
And it's a landslide, the answer is (2).
You roughly do the same thing we did before, except now the math is incrementally harder because you have to do a divide.
So we think about the same two events, the event that it is odd and the event that it's bigger than 3, and now we ask the question.
If it were odd, what's the likelihood that it's greater than 3?
Before I did the conditioning, what was the likelihood that it was bigger than 3?
1/6
Nope.
1/2.
So bigger than 3 is 4, 5, or 6 -- right?
There are 3 atomic units there.
There are 6 atomic units to start with.
They are equally likely.
So before I did the conditioning, the event of interest had a probability of a 1/2.
After I do the conditioning, I know that half of the possible samples didn't happen.
The universe shrank.
Instead of having a sample space with 6, I now have a sample space with 3.
Similarly the probability law changed.
So now the event of interest is bigger than 3, but bigger than 3 now only happens once.
So what I need to do is rescale my probabilities.
Remember the scaling rule, one of the fundamental properties of probability.
The scaling rule said the sum of the probabilities must be 1.
After I've conditioned, the sum of the probabilities is a 1/2.
That's not good.
I've got to fix it.
So the way to think about Bayes' rule is, if all I know is it the universe got smaller, how should I redo the scaling?
Well if all I've told you is that the answer is odd, then there are three possibilities.
Before I told you that the answer was odd, they were equally likely.
After I tell you that they're odd, has it changed the fact that they're equally likely?
No.
They're still equally likely even under that new condition.
I haven't changed their individual probabilities.
So they started out equally likely, they're still equally likely, they just don't sum to 1 anymore.
Bayes' rule says, make them sum to 1.
OK, so the way I make this sum, sum to one is to divide by 1/2.
If you divide six by 1/2, you get 1/3.
Notice that the probability that it's bigger than 3 went from 1/2 to a 1/3.
It got smaller.
It could have gone either way.
So, think about what happens when the world shrinks, when the universe gets smaller, when I tell you that B happened.
Well when I tell you that B happened, then I ask you whether A happened, here I'm showing a picture that in the original universe A and B sort of covered the same amount of area.
By which I mean, they're about equally likely.
Before I did the conditioning, the probability of A was about the same size as the probability of B.
What happens when I condition?
Well, when I condition now the universe is B.
But notice the way I've drawn them, there's very little overlap.
So now when I condition on B, the odds that I'm in A seems to have got smaller.
Rather than being of equal probability, as I show here, after the conditioning the relative likelihood of being event A is smaller than it used to be.
But that's entirely because of the way I rigged the circles.
I could have rigged the circles to have a large amount of overlap.
Then when I condition, it seems as though it's relatively more likely that I'm in the event A.
That's what we mean by the conditioning.
The conditioning can give you un-intuitive insight.
Because when you condition, probabilities can get bigger or littler.
And that's something that sort of at a gut level, we all have trouble dealing with.
OK, so that's the fundamental ideas, right?
We've talked about events.
Three axioms of probability that are completely trivial.
One, not quite so trivial rule, which is Bayes' rule.
In order to apply it, there's two more things we need to talk about.
The first is, notation.
We could do the entire rest of the course using the notation that I showed so far, drawing circles on the blackboard, it would work.
It would not be very convenient.
So to better take advantage of math, which is a very concise way to write things down, we will define a new notion which is a random variable.
Random variable is just like a variable, except shockingly, it's random.
So where we would normally think about a variable represents a number, a random variable represents a distribution.
So we could, for example in the die rolling case, we could say the sample space has 6 atomic events, and I could think about it as 6 circles.
Circles wouldn't pack all that well.
6 squares inside the universe, right?
Because they are mutually exclusive, and collectively exhaustive, so if I started with a universal that looked like that, I would have this one would be the probability that the number of dots was 1, 2, 3, it has to fill up by the time I've put 6 of them in there.
And they have to not overlap.
A more convenient notation is to say, OK, let's let X represent that outcome.
So I can label the events with math.
I can say, there's the event X equals 1, the event X equals 2, the event X equals 3, and it just makes it much easier to write down the possibilities, then to try to draw pictures with Venn diagrams all the time.
So all we're doing here is introducing a mathematical representation for the same thing we talked about before.
But among the things that you can do, after you've formalized this, so you can have a random variable then it's a very small jump to say you can have a multi-dimensional random variable.
Let's just for example have a 2-space.
X and Y, for example.
So now we can talk very conveniently about situations that factor.
So, for example when I think about flipping 3 coins, I can think about that as a multivariate random variable in three dimensions.
One dimension represents the outcome of the first die-- the first coin toss.
Another dimension is the second, the third dimension is the third.
So there is a very convenient way of talking about it, and we have a more concise notation.
We say, OK let V be the outcome of the first die roll, or whatever.
Let W be the second one, and then we can think about the joint probability distribution, in terms of the multi-dimensional random variable.
So we have the random variable defined by V and W. We will generally to try to make things easy for you to know what we're trying to talk about, we'll try to remember to capitalize things when we're talking about random variables, and then we'll use the small numbers to talk about events.
So this notation would represent the probability that V took on the value little v, and W took on the value little w. We'll see examples of this in a minute.
So the idea is-- you don't need to do this, it's just a convenient notation to write more complicated things concisely.
Now a concept that's very easy to talk about, now we have random variables, is reducing dimensionality.
And in fact, we will constantly reduce dimensionality of complicated problems that are represented by multiple dimensions, to smaller dimensional problems.
And we'll talk about two ways of doing that.
The first is what we will call marginalizing.
Marginalizing means, I don't care what happened in the other dimensions.
So if I have a probability rule that told me, for example, about the outcome of one toss a fair die, and a second toss of a fair die, and if I tell you the joint probability space for that, right?
So I would have 6 outcomes on one dimension, 6 outcomes on another dimension, let's say they're all equally likely.
I have 36 points altogether, if they're all equally likely, then my probability law is a joint distribution.
The joint distribution has 32 non-zero points and each point has height of.
I said the right thing, right.
36 is what I meant to say.
My brain is telling me that I might not have said that.
I meant 36.
So if I have 36 equally likely events, how high is each one?
1/36.
OK, so the joint probability space for two tosses of a fair 6-sided die, is this 6-by-6 space.
And I may be interested in marginalizing.
Marginalizing would mean, I don't care what the second one was.
OK well, how do you infer the rule for the first one from the joint, if I don't care what the second one was, well you sum out the second.
So if I have this 2-space that represented the first and the second.
So, say its X and Y, for example.
So, I've got 6 points that represent 1, 2, 3, 4, 5, 6.
And then 6 this way, that sort of thing, except now I have to draw in tediously all of the others, right?
So you get the idea.
Each one of the X's represents a point with probability 1/36, and imagine direction that they're all in straight lines.
Now if I didn't care what is the second one, how would I find the rule for the first one, well I just sum over the second one.
So, say I'm only interested in what happened in the first one, well I would describe all of the probabilities here to that point.
I would sum out the one that I don't care about.
That's obvious, right?
Because if I marginalized these X's that all represent the number 1/36 have to turn into a single dimension axis, which is just X, and they have to be 6 numbers that are each how high?
1/6, right?
So the way I get 6 numbers that are each 1/6, when I started with 36 numbers that were each 1/36 is use sum.
OK, so that's called marginalization.
The other thing that I can do is condition.
I can tell you something about the sample space and ask you to figure out a conditional probability.
So I might tell you what's the probability rule for Y conditioned on the first one being 3?
OK.
Mathematically that's a different problem, that's a re-scale problem, because that's Bayes' rule.
So generally if I carved out by conditioning some fraction of the sample space, the way you would compute the new probabilities would be to re-scale.
So there's two operations that we will do.
We will marginalize, which means summing out.
And we will condition, which means re-scale.
OK.
So give some practice at that, let's think about a tangible problem.
Example, prevalence and testing for AIDS.
Consider the effectiveness of a test for AIDS.
This is real data.
Data from the United States.
So imagine that we take a population, representative of the population in the United States, and classify every individual as having AIDS or not, and being diagnosed according to some test as positive or negative.
OK, two dimensional.
The two dimensions are what was the value of AIDS, true or false?
And what's the value of the test, positive or negative?
So we've divided the population into four pieces.
And by using the idea of relative frequency, I've written probabilities here.
So what's the probability of choosing by random choice an individual that has AIDS and tested positive.
OK, so that's 0,003648, et cetera.
So I've divided the population into four groups.
Multidimensional, multidimensional random variable.
OK.
The question is, what's the probability that the test is positive given that the subject has AIDS?
I want to know how good the test is.
So the first question I'm going to ask is, given that the person has AIDS what's the probability that the test gives a true answer?
You've got 60 seconds.
This is harder.
Some people don't think it's harder.
So what's the probability that the test is positive, given that the subject has AIDS?
Is it bigger than 90%?
Between 50% and 90%?
Less than 50%?
Or you can't tell from the data?
Everybody vote, and the answer is 100% correct.
Wonderful.
So let me make it harder.
Is it between 90% and 95%?
Or between 95% and a 100%?
95% and a 100%
95%.
Is it between 95% and 97%, or 97% and 100%?
OK, sorry.
This is called marginalization.
I told you something about the population that lets you eliminate some of the numbers.
So if I told you that the person has AIDS, then I know I'm in the first column.
That's marginalization.
I gave you new information.
I'm saying the other cases didn't happen.
I've shrunk the universe, it used to have 4 groups of people, now it has 2 groups of people, I used Bayes' rule.
I need to re-scale the numbers so that they add to 1.
So these 2 numbers, the only 2 possibilities that can occur-- after I've done the conditioning, no longer add to 1.
I've got to make them add to 1.
I do that by dividing by the probability of the event that I'm using to normalize.
So the sum of these two probabilities is something, whatever it is 0.003700.
So I divide each of those probabilities by that sum, that's just Bayes' rule.
And I find out that the answer is the probability that the test is positive-- given that person has AIDS, the probability that the test is positive is 0.986.
Good test?
Good test?
98%.
I won't say that.
98%.
is a good test right?
Not that today is an appropriate day to talk about the outcomes of tests and 98%.
But, I won't mention that.
OK, so good test.
The accuracy of the test is greater than 98%.
Quite good.
New question.
What's the probability that the subject has AIDS given that the test is positive?
Everybody vote.
(1), (2), (3), (4).
Looks 100%.
OK, the answer is less than 50%.
Why is that?
Well that's another marginalization problem, but now we're marginalizing on a different population.
This is how you can go awry thinking about probability.
The 2 numbers seem kind of contradictory.
Here I'm saying that the test came out positive and I'm asking does the subject have AIDS.
It's still marginalization.
I'm still throwing away 2 of the conditions, two fractions of the population, I'm only thinking about 2.
I still have to normalize so that the sums come out 1, but the numbers are different.
Yes?
[INAUDIBLE PHRASE]
Thank you.
Because my brain's not working.
OK, I've been saying marginalization and I meant uniformly, over the last five minutes, to be saying conditioning.
OK, so I skipped breakfast this morning, my blood sugar is low, sorry.
Thank you very much.
I should have been saying conditioning.
Sorry.
OK, so backing up.
OK I conditioned on the fact that the person had AIDS, and then I conditioned on the fact that the test came up positive.
In both cases I was conditioning.
In both cases I was doing Bayes' rule.
Please ignore the person who can't connect his brain to his mouth.
So, here because the conditioning event has a very different set of numbers from these numbers, the relative likelihood that the subject has AIDS is small.
So even though the test is very effective in identifying cases that are known to be true, it is not very effective in taking a random person from the population and saying the test was positive, you have it.
OK, those are very different things and the probability theory gives us a way to say exactly how different those are.
Why are they so different?
The reason they're different is that other word.
Because the marginal probabilities are so different.
And that is because the population is skewed.
So the fact that the test came out positive, is offset at least somewhat by the skew in the population.
So the point here is actually marginalizing.
If I think about how many people in the population have AIDS, that means I'm summing on the columns, rather than conditioning.
And what you see is a very skewed population.
And that's the reason you can't conclude from the test, whether or not this particular subject has the disease or not because the population is so skewed.
So this was intended to be an example of conditioning versus marginalization and how you think about that in a multi-dimensional random variable.
Yes?
Don't you sum [UNINTELLIGIBLE] in order to do Bayes' rule?
In order to condition on has AIDS, you need to sum has AIDS.
And then you use that number.
Yes?
That's right
So how are they different?
One of them has a [UNINTELLIGIBLE] and the other one doesn't.
So when we did Bayes' rule, we did the marginalization here, but then we use that summed number to normalize the individual probabilities by scaling, by dividing.
So that the new sum, over the new smaller sample space is still one.
So your point 's right.
So regardless of whether we're conditioning or marginalizing, we still end up computing the marginals.
it's just that in one case were done, and in the other case we use that marginal to re-scale OK?
So I said, we could just use set theory and we're done.
We'll in fact use random variables because it's simpler.
That's one of the two other things we need to do which are non-essential, it just makes our life easier.
And the other non-essential thing that we will do is represent it in some sort of a Python structure.
So we would like to be able to conveniently represent probabilities in Python.
The way we'll do that, is a little obscure the first time you look at it.
But again, once you've done it a few times it's a very natural way of doing it, otherwise we wouldn't do it this way.
How are we going to represent probability laws in Python?
The way we'll do it, since the labels for random variables can be lots of different things-- so for example, the label in the previous one was in the case of the subject having AIDS or not, the label was true or false.
The label for the test was positive or negative.
So in order to allow you to give symbolic and human meaningful names to events we will use a dictionary as the fundamental way of associating probabilities with events.
So, we'll represent a probability distribution by a class-- what a surprise, by a Python class-- that we will call DDist which means discrete distribution.
DDists want to associate the name of an atomic event which we will let you use any string, or in fact any-- I should generalize that.
You can use any Python data structure to identify an atomic event.
And then we will associate that using a Python dictionary, with the probability.
So what we will do when you instantiate a new discrete distribution, you will-- the instantiation rule, you must call it with a dictionary.
A dictionary is a thing in Python that associates one thing with another thing, I'll give an example in a minute.
And the utility of this is that you'll be able to use as your atomic event a string, like true or false, a string like positive or negative, or something more complicated like a tuple.
And I'll show you an example of where you would want to do that in just a second.
So the idea is going to be you establish a discrete distribution by the unique method called the dictionary.
The dictionary is just a list of keys which tell you which event that you're trying to name the probability of.
Associated with a number, and that number is the probability.
And this shows you that there's one extremely interesting method, which is the Prob method.
The idea is that Prob will tell you what is the probability associated with that key.
If it doesn't find the key in the dictionary, I'll tell you the answer is 0.
We do that for a specific reason too, because a lot of the probability spaces that we will talk about, have lots of 0's in them.
So instead of having to enumerate all of the cases that are 0 we will assume that if you didn't tell us a probability, the answer was 0.
OK so this is the idea.
I could say use the disk module in lib 601 to create the outcome of a coin toss experiment.
And I have a syntax error.
This should have had a squiggle brace.
A dictionary is something that in Python-- So I should have said something like this-- dist.DDist of squiggle.
Sorry about that, that should've said squiggle, I'll fix it and put the answer on the website.
Head should be associated with the probability 0.5 and tail should be associated with the probability 0.5.
End of dictionary, end of call.
Sorry, I missed the squiggle.
Actually what happened was, I put the squiggle in and LaTeX ate it.
Because that's the LaTeX, anyway.
It's sort of my fault.
The dog ate my homework.
LaTeX ate my squiggle, it's sort of the same thing.
So having defined a distribution, then I can ask what's the probability of the event head?
The answer is a half.
The probability of event tail?
The answer is a half.
The probability of event H?
There is no H. The answer 0.
That's what I meant by sparsity.
If I didn't tell you what the probability is, we assume the answer is 0.
Conditional probabilities are a little more obscure.
What's the conditional probability that the test gives me some outcome given that I tell you the status of whether the patient has, or doesn't have AIDS?
OK, well conditionals-- you're going to have to tell me which case I want to condition on.
So in order for me to tell you the right probability law you have to tell me does the person have AIDS or not.
So that becomes an argument.
So we're going to represent conditional probabilities as procedures.
That's a little weird.
So the input to the procedure, specifies the condition.
So if I want to call the procedure and find out what's the distribution for the tests, given that the person has AIDS?
Then I would call, test given AIDS of true.
So if AIDS is true, return this DDist, otherwise return this DDist.
So it's a little bizarre but think about what it has to do.
If I want to specify a conditional probability, I have to tell you an answer.
And that's what the parameter is for.
So the way that would work is illustrated here having defined this as the conditional distribution I could call it by saying what is the distribution on tests given that AIDS was true?
And the answer to that is the DDist.
Or if I had that DDist, which would be this phrase, I could say what's then the probability in that new distribution that the answer is negative?
Then I would look up the dot prob method within the resulting conditional distribution, and look up the condition negative.
And finally the way that I would think about a joint probability distribution, is to use a tuple.
Joint probability distributions are multi-dimensional, tuples are multi-dimensional.
So for example, if I wanted to represent this multi-dimensional data, I might have the joint distribution of AIDS and tests.
OK that's a 2-by-2.
AIDS can take on 2 different values, true or false.
And tests can take on 2 different values, positive or negative.
So there's 4 cases.
The way I would specify a joint distribution would be create a joint distribution starting with the marginal distribution for AIDS and then using Bayes' rule tell me the two different conditional probabilities given AIDS.
And that then will create a new joint distribution that whose DDist is a tuple.
So in this new joint distribution, AIDS and tests, if AIDS is false, and test is negative-- so false negative is this number-- the probability associated with tuple is that number.
Is that clear?
So I'm going to construct joint distributions by thinking about conditional probabilities.
So I have a simple distributions which are defined with dictionaries.
I have conditional probabilities which are defined by procedures.
And I have joint probabilities which are defined by tuples.
OK, so that's the Python magic that we will use and a lot of the exercises for Week 10 have to do with getting that nomenclature straight.
It's a little confusing at first, I assure you that by the time you've practiced with it, it is a reasonable notation.
It just takes a little bit of practice to get onto it, much like other notations.
OK where are we going with this?
What we would like to do is solve that problem that I showed at the beginning of the hour.
So we would like to know things like, where am I?
So the kind of thing that we're going to do is think about where am I based on my current velocity and where I think I am, odometry-- which is uncertain, it's unreliable-- versus for example where I think I am based on noisy sensors.
OK so that's like two independent noisy things.
Right?
The odometry you can't completely rely on it.
You've probably run into that by now.
The sonars are not completely reliable.
So there are two kinds of noisy things.
How do you optimally combine them?
That's where we're heading.
So the idea is going to be here I am, I think I'm a robot, I think I'm heading toward a wall, I'd like to know where am I.
So the kinds of data that we're going to look at are things like, I think I know where I started out.
Now my thinking could be pretty vague.
It could be, I have no clue so I'm going to assume that I'm equally likely anywhere in space.
So I have a small probability of being many places.
That just means that my initial distribution is very broad.
But then I will define where I think I am by taking into account where I think I will be after my next step.
So I think I'm moving at some speed.
If I were here, and if I'm going at some speed I'll be there.
So we will formalize that by thinking about a transition.
I think that if I am here at time T, I will be there at time T plus 1.
And I'll also think about, what do I think the sonars should've told me.
If I think I'm here, what would the sonars have said?
If I think I'm here, what would the sonars have said?
And we'll use those as a way to work backwards in probability, use Bayes' rule.
To say, I have a noisy idea about where I will be if I started there.
I have a noisy idea of what the sonars would have said, if I started there.
But I don't know where I started.
Where did I start?
That's the way we're going to use the probability theory.
So for example, if I thought I was here and if I thought I was going ahead 2 units in space per unit in time, I would think that the next time I'm here.
But since I'm not quite sure where I was maybe I'll be there, and maybe I'll be there, but there's very little chance that I'll be there.
That's what I mean by a transition model.
It's a probabilistic way of describing the difference between where I start and where I finish in one step.
Similarly, we'll think about an observation model.
If I think I'm here, what do I think the sonars would have said.
Well I think I've got some distribution that it's very likely that they'll give me the right answer, but it might be a little short it might be a long.
Maybe it'll make a bigger error.
So I'll think about two things.
Where do I think I will be based on how I'm going?
And where do I think I'll be based on my observations?
And then we'll try to formalize that into a structure that gives me a better idea of where I am.
That's the point of the exercises next week when we won't have a lecture.
So this week we're going to learn how to do some very simple ideas with modelling probabilities.
With thinking about these kinds of distributions.
And the idea next week then is going to be incorporating it into a structure that will let us figure out where the robot is in some sort of an optimal sense.
So thinking about optimal -- let's come back to the original question.
How much would you pay me to play the game?
OK, we had some votes.
They didn't add up to 1.
What should I do to make them add up to 1?
Divide by the sum.
Right?
Look at all of you know already, right?
So you now know all this great probability theory.
So the question is can we use probability theory to come up with a rational way of thinking how much it's worth?
Most of you thought that it's worth less than 10$.
OK, so how do we think about this?
How do we use the theory that we just generated to come up with a rational decision about how much that's worth?
OK, thinking about the bet quantitatively, what we're going to try to do is think about it with probability theory.
There are 5 possibilities inside the bag.
Originally there could have been 4 white, or 3 white and 1 red, or 2 and 2, or 1 and 3, or 0 and 4.
That was the original case.
You didn't know.
I didn't know.
They were thrown into the bag over here.
We didn't know.
How much would that game-- how much should you be willing to pay to play that game?
Someone asked how many white ones and how many red ones did the person put in the bag?
I don't have a clue, right?
We need a model for the person.
Since I don't have a clue, one very common strategy is to say all these things I know nothing about let's just assume they're all equally likely.
So that's called maximum likelihood, when you do that.
There's other possible strategies.
I'll use the maximum likelihood idea just because it's easy.
So I have no idea.
Let's just assume that here's all of the conditions that could have happened.
The number of red that are in the bag could have been 0, 1, 2, 3, or 4.
I have no idea how the person chose the number of LEGO parts.
So I'll assume that each of those cases is 1/5 likely, since there's 5 cases.
OK now I'll think about what's my expected value of the amount of money that I'll make if the random variable S, which is the number of red things that are in the bag was s which is either 0, 1, 2, 3, or 4.
OK, if there are 0, how much money do you expect to make?
None.
If there are 4 reds, how much money would you expect to make?
$20 If there are 2 reds, you would expect to make 10 $.
Everybody see that.
I'm trying to think through a logical sequence of steps for thinking about how much is it worth to play the game.
So this is the amount of money that you would expect given that the number of red in the bag, which you don't know, were 0, 1, 2, 3, or 4.
That's this row.
What's the probability, what's the expected value of the amount of money you would get., and that happens?
Well I have to use Bayes' rule.
What I need to do is I have to take this probability times that amount to get that dollar value.
So over here, in the event that there are 4 reds in the bag, I'm expecting to get $20 but that's only 1/5 likely.
Right?
Because there don't have to be 4 reds in the bag.
So I multiply the 1/5 times the $20, and I get 4$.
So my expected outcome for this trial is 4$.
Here, I'm expecting to make 10$ if I knew that there was 2 reds in the bag.
But I don't know that there's 2 reds in the bag, there's a 1/5 probability there's 2 reds in the bag.
So 1/5 of my expected amount of money which is 10$ is 2$.
So then in order to figure out my expected amount money I just add these all up, marginalizing.
And I get the [UNINTELLIGIBLE] 4 plus 3 is 7 plus 2 is 9 plus 1 is 10.
So this theory says it if I can regard the person who put the LEGOs in the bag as being completely random, I should expect to make 10$ on the experiment.
So that means you should be willing to pay 10$.
Because on average, you'll get back 10$.
If you wanted to make a profit you ought to be willing to pay 9$.
Right?
Because then you would pay 9$ expecting to get 10$.
If you really would like to make a loss, right?
Then you should pay $11.
Yeah?
Why do we assume that these events are equally likely?
Completely arbitrary.
So there's theories, more advanced theories, for how you would make that choice.
So for example if in your head you thought that the person just took a large collection of LEGO parts and reached in, then you would think that the number of red and white might depend on the number that started out in the bin.
But I don't think that's probably true, right.
The person was probably looking at them and saying, oh throw in one red, through in one white.
So you need a theory for doing that, and I'm saying that in the absence of any other information let me assume that those are equally likely and see what the consequence of that would be.
The consequence of assuming that is that I should expect to get 10 $ back.
What happens if you pull out a red?
As we did.
How does that affect things?
Well it increases the bottom line.
I start out again with the assumption that all 5 cases are equally likely.
Now I have to ask the case, how likely is it that the one that we pulled out was red?
Well it's not very likely that the one that I pulled out was red, if they were all white.
The probability of that happening is 0.
What's the probability if there were 2 that the person pulled out a red?
Well 2 of them were red, 2 of them were white, 2 out of 4 cases would have showed this case of pulling out a red.
So this line then tells me how likely is it that the red was pulled.
OK. Then what I want to do is think about what's the probability that I pulled out a red, and there was 0, 1, 2, 3, or 4.
So I multiply 1/5 times 0/40, 0/20, 1/5 times 1/4 to get 1/20, 1/5 times 2/4 you get 2/20.
So those are probabilities of each individual event happening.
But they don't sum to 1.
So then the next step I have to make them sum to 1.
So the sum of these is a 1/2.
So I make them sum to one this way.
So now what's happened is it's relatively more likely 4 out of 10, that this case happened, than that case.
I know for sure, for example, that there's not 4 whites.
The probability of 4 whites is 0-- 0 out of 10 So what I've done is I've skewed the distribution toward more red by learning that there's at least 1, I now know that I know additional information.
These were not equally likely.
In fact, the ones with more red were relatively more likely.
So if I compute this probability times that expected amount, I now get a much bigger answer for the high number of reds.
So I still get 0 just like I did before for this case, because there's no reds in the bag.
But now it's much more likely that they're all red, because I know there was at least 1 red.
And then the answer comes out $15.
So my overall assessment, don't go to Vegas.
You could have made a lot more money by offering $13.
Because on average, you should've expected to make $15.
OK, so what I wanted to do by this example is go through a specific example of how you can speak quantitatively about things that are uncertain.
And that's the theme for the rest of the course.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello.
Welcome.
Today we're going to talk about one last new topic which has to do with search.
So as you remember, we're working on our last topic.
The last topic was probability and planning.
Last lecture we talked about probability.
Mostly we focused on Bayes' theorem, Bayes' rule.
That was a way of updating our belief about some situation based on new information.
And this week in Design Lab 12, you'll get a chance to use that in a robot application.
The idea to Design Lab 12 is going to be that a robot is pedaling along a corridor with some obstacles off to its left.
And the idea will be, you don't know where the robot is, but the robot will be able to estimate where it is by the signals that it receives from its left-facing sonars.
So this is a very realistic type of state-estimation problem.
The idea is going to be that at any given time, t, you have access to your previous belief at time t minus 1.
And you have access to a new observation, which is the sonar to your left.
And based on those two bits of information, you will update your belief.
Which means that when you start out you'll have no idea where you are, but that situation should improve with time.
Ok.
So that's what we're going to do in Design Lab 12.
Today we're going to blast ahead and think about the other important topic, which is search.
So we're going to think about planning.
We're going to be planning ahead.
We're not going to just react to the situation that's given to us.
We're going to try to figure out what's the right thing to do in the future.
And to do that, we're going to think about all the things we could possibly do, search through that space, and figure out the one that's, quote, "best."
And we'll have to define "best" somehow.
Just to get going, I want to show you a very simple kind of a search problem.
This is called the eight puzzle.
The idea is to make a plan to go from this configuration, which we'll call the state, to this configuration, which we'll call the goal state.
And on each move, you get to move one of the tiles into the free space.
So I could move the 8 to the right or the 6 down.
Those are the only two things I could do in the start state.
So I have to make up my mind which of those I would like to do.
And I would like to believe that ultimately, after a series of moves, I'm going to be able to control this state into that state.
And you can imagine guessing.
And we'll estimate in a moment how big is the guess space.
I mean, if the guess space only had like four elements in it, guessing's a fine strategy.
If the guess space has a lot more elements than that, guessing's probably not a good idea.
So I previously ran our search algorithms, the ones that we'll develop during lecture, on this problem.
And here's the solution that our search algorithm came up with.
It's not exactly what you might do the first time you touched it.
OK, I made it.
If you were counting, I made 22 moves.
The question is, how difficult was that problem and how good was that solution?
Was that a good solution or a bad solution?
Is there a better solution?
How much work did I have to do in order to calculate that solution?
To get a handle on that, let's start by asking a simple question.
How many configurations are there?
I got there in 22.
What was the space of things I had to look through?
How many different board configurations exist?
So think about that for 20 seconds.
Talk to your neighbor.
And figure out whether it is 8-squared, 9-squared, 8-factorial, 9-factorial, or none of those.
So how many bar configurations do you see?
Raise your hand, show me a number of fingers, so I can figure out roughly how people were-- that's excellent.
Very good participation, and nearly 100% correct.
The answer is number (4).
You can think about this.
What if you took all the tiles out and threw them on the floor, and then put them in one at a time?
Well, you would have 9 possibilities for where you wanted to put the first one.
Then you would have 8 possibilities for where you wanted to put the second one.
Then you'd have 7 possibilities for where you put the third one, et cetera, et cetera.
Even though the space doesn't have a number on it, it still sort of counts.
And so you end up with 9-factorial.
And the point is, 9-factorial is a big number.
9-factorial is 362880.
So if you thought about simply guessing, that's probably not going to work all that well, right?
Even if you guessed, you have on each-- there's a third of a million different configurations that you have to look at.
And that's if you didn't lose track of things.
If you lost track of-- Oh, my.
It's coming up-- almost, anyway.
It looks like it's chopped off at the top.
So ignore that for now.
Look over here.
So even if you didn't confuse yourself, there's a space of a third of a million things to look at.
And if you confused yourself, there's even more.
So it's not a huge problem by computer science standards.
But it's certainly not a trivial problem.
It's not something that you can just guess and get right.
So what we want to do today is figure out an algorithm for conducting a search like that.
We'd like to figure out the algorithm, analyze how well it works, optimize it, and try to find out a way to find the best solution, where "best" for this particular problem would mean minimum path length.
So figure out the best solution by considering as few cases as possible.
Obviously, if you enumerate all the cases, that should work.
The problem is, it will be interesting to solve problems where that enumeration is quite large.
Even here, the enumeration is quite large.
So let's think about the algorithm.
And I'll think about the algorithm by way of an even simpler, more finite problem.
What if I thought about a grid of possible locations where I could be.
Maybe this is the intersections of streets in Manhattan.
I want to go from point A to point I.
What's the minimum-distance path?
I hope you probably can all figure that out.
What I want to do is write an algorithm that can figure that out.
And then if we write the algorithm well, we'll be able to use it for the tile problem, which is not quite so easy to do.
The way we're going to think about doing that is to organize all of our possible paths through that maze, in a tree.
So if I started at A, I have a decision to make.
I could either go to B or D. Then if I went to, say, B, I could either then go to A or C or E. I've organized them alphabetically for no particularly good reason.
Just-- I needed some order.
Then if I went from A to B to A, say, then I could either go from A to B or D. That's illustrated here.
So the idea then-- ah, it works.
So now it looks like they're all three working.
So the idea is, think about the original problem.
The original problem is find the shortest path through some grid.
I want to go from A to I.
And I'll think about all the possible paths on a tree.
Then the problem is that for the kinds of problems we're going to look at, that tree could be infinite in length.
Oh, that's a bummer.
That means that the strategy of building the tree and then searching it is probably not a good strategy.
So what we'll do instead is, we'll try to write the algorithm in such a way that we construct the tree and search for the best solution all in one pass.
Then hopefully, if we find a solution in some finite number of steps, we'll only have built part of the tree.
But we'll have built the part that has the answer.
The idea, then, is going to be, think about what is the path we want to take, by thinking about the tree of all possible paths.
But what we want to do is write code that will construct the tree on the fly, while it's considering how good were all the different nodes.
Ok.
So how are we going to do that?
We'll be working in Python, not surprisingly.
We'll represent all the possible locations.
We'll call those states.
So the problem will have states A, B, C, D, and we'll just represent those by strings.
That makes it flexible.
That makes it arbitrary.
Then we'll think about transitions, not by enumerating them.
Remember, we don't want to enumerate them, because there could be infinitely many of them.
So how's the other way we could do it?
Well, we'll embody that information in a program.
We'll write a procedure called "successor" that will, given the current state and action, figure out the next state.
So that's a way that we can incrementally build the tree.
So imagine here, if I started in A and I executed action 0 or 1, I would end up in B or D, respectively.
So I tell you the current state and the current action, and the successor program then will return to you the new state.
That's all we need to construct the tree on the fly.
Then, to specify the particular problem with interest, I have to tell you where you start.
So I have to define initial state.
And I have to tell you where to end.
I could just tell you the final state.
But in some of the problems of the type that we will want to do, there could be multiple acceptable answers.
So I don't want to just give you the final state.
I'll give you a test.
I'll give you another procedure, called "goalTest."
And that goal test, when passed an input which is a state, will tell you whether or not you reached the goal.
That way, for example, all the even-numbered squares could satisfy the goal, if that were the problem of interest.
Or all the states on the right could satisfy the goal.
It's just a little bit more flexible.
The idea, then, is that in order to represent that tree, we'll do it by specifying a procedure called successor and specifying the start state and the goal test.
So here's how I might set that up for the simple Manhattan problem that I showed-- that I started with.
So I want ultimately to have something called successor that eats a state and an action.
I've built the structure of Manhattan into a dictionary.
The dictionary lists for every state-- A, B, C, D, E, F, G, H, I-- for every state, it associates that state with a list of possible next states.
So if I'm in A, I could next be in B or D. I could next be in B or D. I've organized these arbitrarily in alphabetical order so I can remember what's going on.
So the next states are all in alphabetical order.
The number of next states depends on the state.
I'm not going to worry about that too much.
I'm just going to specify the action as an integer-- 0,1,2,3 -- however many I need.
So the possible actions are taken from that list.
The possible action might be do action 0, do action 1, do action 2.
So if I did action 2 starting on state E, I would go to-- so action started at 0 -- so 0, 1, 2 -- I would go to state F. OK?
Is that all clear?
The initial state is A, and the goal state is, return S equal to I.
So if S is equal to I, it returns True.
If S is not equal to I, it returns False.
I'm not quite done.
That's enough to completely specify the tree, but now I have to build the tree in Python.
Not surprisingly, from our object-oriented past, we will use an object-oriented representation for that tree.
So we'll specify every node in the tree as an instance of the class SearchNode.
SearchNode is trivial.
SearchNode simply knows, what was the action that got me here?
Who's my parent?
And what's my current state?
So when you make a new node, you have to tell the constructor those three things.
What was the action that got me here?
What's my current state?
And who is my parent?
Knowing the node, you're supposed to know the entire path that got you here.
So we'll also add a method which reports the path.
So if I happen to be in node E, my path ought to be I started in A, I took action 0 and got to B, and then I took action 2 and got to E. This subroutine is intended to do that.
If my parent is "none," which will happen for the initial state, simply report that the path to me is none, A.
However, if I'm anybody other than the initial node, if I'm other than the initial node, then figure out the description of the path to my parent, and add the action that got me here, and my state.
So that's what this is.
OK, so what are we doing?
We specify a problem by telling you the successor function, the start state, and the goal test.
Then we provide a class by which you can build nodes to construct, on the fly, the search tree.
Now we're ready to write the algorithm.
Here's the pseudocode for the algorithm.
What do we do?
We initialize-- so we're going to be doing a s-- oh, this is very confusing.
I'm trying to solve a search problem.
To solve the search problem, I'm going to search through the tree.
So I'm going to think about the state of my search through the tree by way of something we'll call the agenda.
Very jargon-y word.
I completely apologize for it.
I didn't invent it.
It's what everybody calls it.
Sorry.
The agenda is the set of nodes that I'm currently thinking about.
So I'll initialize that to contain the starting node.
Then I'll just systematically keep repeating the same thing over and over again.
Take one of the nodes out of the agenda, think about it, replace that node by its children.
While I'm doing that, two things are supposed to happen.
I'm supposed to construct the search tree.
But I'm also going to be looking over my shoulder to see if I just constructed a child who is the answer.
Because if I just constructed the answer, I'm done.
Ok.
So initialize the agenda to contain just the starting node.
Then repeat the following steps.
Remove one node from the agenda.
Add that node's children to the agenda.
And keep going until one of two things happens.
Either you found it-- goal test returned True-- or the agenda got empty, in which case there must not be a solution.
If I've removed all of my options and still haven't found anything, then there's no solution.
Ok.
So what's the program look like?
It's actually remarkably simple, especially when you think about just how hard the problem is.
Imagine if you wanted to do that tiles problem with a very simple-minded "if this then this, if this then this."
We're talking about a third of a million ifs, right?
That's probably not the right way to do it.
This program is going to end up being about this long.
It'll fit on this page.
And it's going to be able to handle that case, or even harder cases.
Define the search procedure.
The search procedure is something that's going to take the initial state, the goal test, the possible actions, and the successor sub routine, the successor procedure.
That's everything you need to specify the problem.
And it's going to return to me the optimal path.
First, step (1).
Initialize the agenda to contain the start node.
I want to put the start node in.
Well, there's a chance-- I want this procedure to be general purpose-- there's a chance that that start node is the answer.
So take care of that first.
If you're already there, return the answer.
The path to the answer is me.
I'm trying to create the agenda.
I'm trying to put the first node into the agenda.
There's a chance that first node is the answer.
If that first node's the answer, return the path to me.
Which is, take no action.
You're here.
But that's not likely to be the case for the kinds of questions we ask, in which case we will create a list that contains one node, which is the node that represents the start node.
Then, repeat 'remove a node' -- which we'll call the parent-- from the agenda.
And substitute -- replace that node that we pulled out of the agenda -- replace that with the children.
While not empty of agenda-- empty is some kind of a pseudo-routine that I'm going to fill in, in a minute-- while the agenda is not empty, get an element out of the agenda, which we'll call the parent.
Then I want to think about all the children.
Well, there's a list of possible actions.
So for a in actions, do the following things.
Each parent can have multiple children, one for every possible action.
So for a in action, figure out what would be the new state.
The new state is just the successor of the parent state.
Remember, the parent is a node, right?
The parent is a node.
We're constructing nodes in the search tree.
But nodes know their state.
So figure out the new state, which is the successor of my parent, the guy that I pulled out of the agenda.
Make a new node, which corresponds to this child.
Then ask the question, did the new state satisfy the goal test?
If it did, the answer is the path to the new node.
Ok.
So create a new state, which is the successor of the parent under the given action A.
Create a new node.
See if it's the end.
If it is, just return, I'm done.
Return from search.
Otherwise, add it-- again, one of these pseudo-procedures.
We'll fill that in, in a minute-- add the new node into the agenda.
There's several things that could happen when I run this loop.
If the node has no children, I will take out the parent and not put anything back in.
If the node has multiple children, I could take out one node and put in more nodes than I took out, so the agenda could get longer.
So the agenda could either increase in length or decrease in length as a result of spinning around this loop.
Also, we could either identify a goal or fail to identify a goal.
So as it's increasing and decreasing, we either will or won't find an answer.
Ok. Now the trick-- the only thing that makes this complicated-- is that order matters.
So those pseudo-operations, whatever they were-- get element and add-- exactly how I get element and exactly how I add it to the agenda affects the way I conduct the search.
Let's think of something very simple.
Let's always remove the first node from the agenda and replace it by its children.
So pull out the first node.
And put back into the beginning of the agenda the children of the first node.
How?
I would start out in step (0).
I would put the start node into the agenda.
So now there's one element in the agenda, the start node.
Then, on the first pass through the loop, I would pull out that.
That's the first node in the agenda.
There's only one node in the agenda.
That is the first one.
Pull out the first one and replace it by its children.
Its children are AB and AD.
I'm representing the nodes in this notation by the path, because the same state can appear multiple times in the tree.
Notice that I could walk ABAB -- which would correspond to the same state being repeated in the tree.
So I can't, when I'm writing it down here, represent the node by a state.
But I can represent a node by a path.
So on the first pass through the loop, pull out the first item in the agenda, which is A, and push back that A's children.
Well A's children are AB and AD.
Ok.
So now on the second pass, the rule is pull out the first guy and replace it by the children.
Now the first guy is AB.
So I'm here.
So pull that guy out and replace him by his children.
His children are ABA, ABC, and ABE.
AD is left over.
The number of elements in the agenda got bigger.
Next step, pull out the first item in the agenda, replace it by its children.
The first item in the agenda is ABA.
The children of ABA are ABAB and ABAD.
Ok. Notice the structure of what's going on.
Ignore the stuff on the bottom, and just watch the picture on the top.
So I start by putting A in the agenda, then its children, then its children, then its children.
So when I implemented the algorithm-- take out the first and replace it by its children-- I'm searching along the depth first.
I'm going deeper and deeper into the tree without fully exploring all the horizontal spaces.
So I'm tracing a line down that way.
If you imagine this tree-- I've only represented the first three layers of nodes here-- this tree goes on forever.
It's an infinite tree, because you can walk around in that Manhattan grid forever.
There's no limit to how long you can walk around.
So although I'm only listing the first three, the tree, in principle, goes on forever.
And this algorithm will have the feature that it walks along the left edge.
We call that depth-first search because we're exploring depth first, as opposed to breadth.
That results because of our rule.
The rule was, replace the first node by its children.
Let's think about a different rule.
Let's replace the last node by its children.
We start by initializing the agenda to the node that represents the start state.
So that's the path A.
Then pull out the last node in the agenda-- that's A-- and replace it by its children.
Its children are still AB and AD, just like before.
Now the answer differs from the previous answer.
Because when I pull out the last node, I'm pulling out AD now, instead.
And now I replace AD by its children, which are ADA, ADE, ADG.
Repeat, and what I've got is a different, but still depth-first search.
So I've looked at two different orderings-- pull out the first node from the agenda and replace it by children, pull out the last node and replace it by its children.
Both of those algorithms give an exploration of the decision tree searching out depth first.
So it's going to try to exhaustively go through the entire depth before it tries to explore the width.
As an alternative, think about a slightly more complicated rule.
Remove the first element from the agenda and add its children to the end of the agenda.
So initialize it with the start state, A.
Pull out the first element from the agenda-- that's A-- and replace it by its children, which is AB, AD.
Now pull out the first guy-- the first guy is AB-- and put its children at the end.
It's children are ABA, ABC, ABE-- ABA, ABC, ABE-- and they are now put at the end.
So that on the next step, I'll pick up AD-- the guy at the beginning-- and put AD's children at the end.
et cetera, et cetera, et cetera, et cetera, et cetera, et cetera.
The idea being-- and now, pay no attention to the bottom for a moment and just think about the pattern that you see at the top.
In this order, where we remove the first node and put its children at the end of the agenda, has the effect of exploring breadth first.
So we call that a breadth-first.
so the idea is, we got this generic set of tools that let us construct search trees.
But the order by which we manipulate the agenda plays a critical role in how the search is conducted.
And the two that epitomize the two extreme cases are, what would happen if I replace the last node by its children?
Or what would happen if I remove the first node and put its children's at the end?
Those two structures have names because they happen so often.
We'll call the first one a stack and the second one a queue.
The stack-based is going to give us depth first.
The queue-based is going to give us breadth first.
So, stack.
How do you think about a stack?
You think about a stack by saying, OK, I've got a stack.
A stack is like a stack of dishes.
So here's my table, and I'm going to rack my dishes up.
I'm going to put them on a stack.
So I make a stack.
OK, I made the stack-- push a 1, push a 9, push a 3.
Push a 1, push a 9, push a 3.
That's how I do a stack.
Then pop.
When I pop, the 3 comes out.
Then pop, then the 9 comes out.
Then push a minus 2.
Then pop.
Now the minus 2 comes out.
OK?
It's stack-based.
So the last in becomes the first out.
That was the rule that we wanted to have for the depth-first search.
It's very easy to implement this.
We can implement it as a list.
All we need to do is be careful about how we implement the push and pop operators.
So if we set up the push operator to simply append to the end, and then pop ordinarily pops from the end, we'll get the behavior of a stack.
That gives me, then, the rules that I would want to use for those procedures, the get element an add.
I will use these stack-based operators.
The other alternative is a queue.
A queue is different.
A queue is like when you're waiting in line at the Stop & Shop.
The queue is, I've got this queue here and I've got the server over here.
The first person who comes into the queue-- so say I push one.
So now 1 goes into the queue.
Then another person walks up while he's-- the second person lines up behind the first person.
Then I push a 3.
But the way the queue works is that when I pop the next person off the queue, I take the head of the line.
So the 1 comes out.
If I pop again, the 9 comes out.
If I then push a minus 2 and pop, then the next person in the queue comes out.
And it's like that.
It's queue based versus stack based.
And the queue based is the one that we want to do for a breadth-first organization.
And the implementation of a queue is very trivially different from the implementation for a stack.
The only difference is that I'll manipulate the list by popping off from the head of the queue.
So pop takes an optional argument, which when 0, tells you the-- the argument tells you which element to pop.
So when you specify the zero-th one, it takes it from the head of the queue.
That makes it very easy now to replace the pseudo-procedures with real procedures.
If I wanted to implement a depth-first search, I would replace the "create a list that contains the agenda" with "create a stack that will contain the agenda."
So create a new stack, the agenda is a stack.
And then rather than simply sticking the node-- the start node-- into a list, I will push it into the stack.
So agenda is a stack.
Agenda.push, the initial node.
And then every time I want to get a new element out, I'll agenda.pop it.
And every time I want to put something into it, I'll agenda.push it.
Other than that, it looks just the same as the pseudocode that I showed earlier.
So there is an implementation, then, for a depth-first search.
If I wanted instead to do breadth, it's trivial.
Change the word "stack" to the word "queue."
Now create an agenda that is a queue, but queues have the same operations-- push and pop and empty-- that stacks have.
So nothing else in the program changed.
All I needed to do is change the structure of the thing that's holding the agenda.
Everything else just follows.
Ok.
So that's everything we need, right?
Now what I want to do is think through examples and think about the advantages and disadvantages of different kinds of searches.
And I want to go on to the second step that I raised in the first slide.
I want to think about, how do I optimize the search?
As I said, even that simple little tile problem, even the eight puzzle-- eight sounds easy, right?
Even the eight puzzle had a third of a million different states.
I don't necessarily want to look through all of them.
I want to think now about these different search strategies, and how optimal are they relative to each other, and are there ways to improve that?
Now some of you may have noticed that all of these paths don't seem equally good.
So take a minute.
Think about it.
Remember the problem.
The problem was this walk around Manhattan problem.
I wanted to go from A to I.
This was the tree of all possible paths from A to I.
What I'd like you to do is think about whether all of those paths are important.
Could we get rid of some of them?
So the question is, can I throw away some of the terminal nodes?
Notice that I'm using the word "terminal" kind of funny here.
The tree keeps going.
The tree is actually infinite in length.
So by "terminal," I just mean this row three.
So could I throw away some of the nodes in row three?
And in particular, how many of them could I throw away?
0, 2, 4, 6, or 8?
or Raise your hand with the funny coding.
And the answer is-- come on, come on.
Raise your hands, raise your hands.
Blame it on your neighbor.
That's the whole point.
OK, it's 2/3 (5) and 1/3 (4) How'd you get (5) and (4)?
Yes?
When you have that [INAUDIBLE] nodes, you may know that there's [?
a procedure ?]
[INAUDIBLE] before
Good.
If you're walking around Manhattan, and you're trying to go from A to I, and if you spun around in this loop and came back to A, that would probably be a bad path, right?
So revisiting a place you've been before is probably a bad idea, if what your goal was, was to get from A to I in the shortest possible distance.
So that's exactly right.
So I would like to identify instances where I go back to where I started.
So for example, that A.
That A is bad.
That means I went back to the start place.
I'm just starting over.
So if I think about those, I can identify by red all the places where I'm repeating.
So ABA, don't really care what happens after that.
ABCB, well, that's B again.
So that's just brain dead, right?
So I can actually remove a fair amount of the tree by simply getting rid of silliness.
Don't start the path over again, where "over" means if you come to a place you've been before, stop looking there.
That's not the right answer.
And so you can see there that I actually deleted half of the tree.
The number of nodes on the third line was 16.
And 8 of them had the property that they repeated.
Yes?
[INAUDIBLE] [?
after D.
?]
Are you [INAUDIBLE]?
This B and D. So there's no reason to consider this D, even though the D didn't repeat
That would be [?
AD?
?]
ABC,
[?
That would be E?
?]
[INAUDIBLE]
So I didn't, in this path, ever hit D before
When it did lead to the path there, you'll get the same one, AD [INAUDIBLE]
I guess I don't understand
What I'm saying is that without getting the path AD, I still get (2)?
Yes.
So this D seems clearly inferior to that D. Yes, that's absolutely true.
So this is a very simple rule for removing things.
You're thinking of a more advanced rule.
So if you saw-- if there's a shorter path to a particular place, don't look at the longer path.
You're absolutely right.
So in fact, there might be more severe pruning that you could do.
There might have been an answer that was bigger than 8 -- right?
And so you're absolutely right.
Ok. Let me ignore that for the moment and come back to it in about four slides.
You're absolutely right.
So what we want to do now is take that idea of throwing away silly paths and formalize it so that we can put it into the algorithm.
And we'll think about that as pruning rules.
So the first pruning rule is the easy one.
Don't consider any path that visits the same state twice.
That doesn't pick up your case, but it does pick up 8 cases here.
So that's easy to implement.
All we need to do is-- down here where we're thinking about whether this is a good state to add or not-- we just ask, is it in the path?
So if the state that I'm about to put in the path is already in the path, don't put it there.
If you don't shove it back into the agenda, it'll get forgotten.
So before you shove it into the agenda, ask yourself the question, is it already in the path?
And so I do that here.
Keep in mind, I popped out an element called the parent.
I'm looking at the children.
The children's state is called "new state."
So I ask, is new state in the parent's path?
So parent.inpath of new state.
So that means I have to write inpath.
Inpath is easy.
It's especially easy if we use recursion.
So inpath says, if my state is state, then return True.
I'm in the path.
If that's not true, and I don't have a parent, then that means I'm the start state.
That means it wasn't in the path.
And if neither of those is true, ask the same question of my parent.
So that makes it recursive.
So consider two cases that could either make it true or false.
It would be True if I'm currently sitting on a node that happens to be the same state.
It would be False if I recursed the whole way back to the start state and hadn't found it yet.
So there are two termination states-- I landed on a state in the path that was the same as new state, or I ran the whole way back to the start state and didn't find it.
Those two terminate by doing returns -- return True or return False.
The other option is that I don't know the answer, ask my parent.
So just recurse on inpath, and ask my parent to do the same thing.
So that's the way I can figure out-- I can implement pruning rule (1).
Now pruning rule (2) -- if multiple actions lead to the same state, only think about one of them.
That actually doesn't happen on the Manhattan grid problem.
Because you can imagine search cases where there are three different things that you could do.
In fact, you saw some of those when you were coding the robot last week.
There were multiple ways you could end up at the state at the end of the hall.
You could get there by being there and moving left, which you hit the wall.
Or you could get there by being here and moving left.
Both of them left you in the same place.
So if you're planning a search, you don't need to distinguish among those, because they take the same amount of steps.
So since they take the same amount of steps, we don't need to search further.
So we can collapse them.
That's called pruning rule (2).
That's also easy to implement.
What we do is, we keep track of, for every parent, what are all of its children.
If the parent already has a child at that place, throw away the excess children.
That doesn't sound good.
So keep track of how many child states I have.
Make a list.
And if the new state didn't satisfy the goal, ask if it's already in the list of children.
If it's already there, pass.
Don't do anything.
Otherwise, do pruning rule (1).
And then, before you push it into the agenda, also push it into the list of new children.
That's a way of making sure that if there's multiple ways to get the same state, you only keep track of one.
So that's an additional pruning rule.
So now let's think about how we would implement these.
Let's think about the solution to a problem where we want to apply a depth-first search on this Manhattan problem, to get from A to I.
So let's think about-- let's go up-- so I want to think about, how do I apply depth-first search to that problem?
So think about the agenda.
So the agenda, I initialized it with the node that corresponds to the start state, so that's A. I'm doing depth first.
What's the rule for depth first?
Pop the last guy, replace it by his children.
OK, so pop the last guy.
What's the last guy?
The last guy is A.
Replace it by his children.
What's his children of A?
Well, there's two of them, AB and AD.
Ok.
So I'm done with the loop for the first level.
So pop the last guy, that's AD.
Replace it by his children.
What are the children of AD?
Well, what could D do?
D could go to A or E or G. A's brain dead, so I don't want that one.
So I'll think about E and G. So ADE, ADG.
By the way, stop me if I make a mistake.
It's really embarrassing.
OK, pop the end, ADG.
Who's the possible children of ADG?
ADG?
Well, it could go back to D, but that's stupid.
So ADGH seems to be the only good one.
Pop the last one, ADGH.
And who's his children, ADGH?
ADGH has children E, G, and I.
But I don't want G, because that's brain dead.
So ADGH, E or I.
And that one won, right?
Because I got to A.
Everyone see what I did?
I tried to work out the algorithm manually.
So the idea, then, was that-- so how much work I do?
I visited 1, 2, 3, 4, 5, 6, 7.
And then I found it.
So I did 7 visits.
And I got the right answer.
So both of those are good-- 7 is a small number, and getting the right answer.
Both of those are good things.
And in general, if you think about the way a depth-first search works-- here's a transcript of what I just did.
This will be posted on the online version.
So you can see it, even though it's not handed out to you now.
So you can look this up on the web.
So in general, depth-first search won't work for every problem.
It happened to work for this problem.
In fact, it happened to be very good for this problem.
But it won't work for every problem because it could get stuck.
It could run forever in a problem with infinite domain.
This problem has infinite domain.
So if I were to choose my start and end state judiciously, I could get it stuck in an infinite loop.
That's a property of depth-first search.
Even when it finds a path, it doesn't necessarily find the shortest path.
Well, that's a bummer.
But it's very efficient in its use of memory.
So it's not a completely brain-dead search strategy, but it's usually brain dead.
So let's think about breadth-first search as an alternative.
Again, all we need do is switch the idea of thinking about stacks versus queues.
Take off the beginning, add to the end.
That's the way queues work.
So now let's do the same problem with a breadth-first search.
So I start with the agenda.
I put in A. I pop off the head of the queue and stuff the children at the end.
I pop off the beginning and stuff the children AB, AD.
That looks right.
That's the end of path one.
Now I pop off the beginning and stick in the children.
What are the children of AB?
Well, AB could go to ACE.
A is brain dead, so ABC-- ABC or ABE.
ABC or ABE, that looks right.
Now pop off this guy, AD, and put his children at the end.
That's ADE and ADG.
I don't think I made a mistake yet.
Pop off the first guy, ABC.
Stick in his children ABC-- ABC, it could go to B or F. Looks like F is the only one that makes any sense.
ABC, that looks right.
ABE, put his children ABE-- E could go to B-- that's brain dead-- D, F, or H. D, F-- wait

AB, thank you.
I'm supposed to be doing ABE followed by something, ABE followed by something.
I don't want B.
D is fine, F is fine, and H is fine.
D, F, and H. OK so far?
[INAUDIBLE]
Oh no, it's not right?
OK, what did I do wrong?
Just AB.
It's the fourth--
Oh, here.
That's up here.
Is that [INAUDIBLE]
Yeah
Thank you.
That would be embarrassing.
OK, next pop off AD-- This is why we have computers, right?
We don't normally do this by hand.
OK, so ADE-- if I had ADE, I could do B-- that seems OK, D seems bad-- F, or H. So it would look like B, F, H. OK, ADG.
A, D, G. It looks like H is my only option.
ABCF.
A, B, C, F. Looks like I could do E or I.
Finally.
Now the only question is whether I got the right number of states.
Let's assume I did.
So 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 -- which happens to be the right answer.
At least it happens to be the answer I got this morning when I was at breakfast.
So what did I just do?
I just did a breadth-first search.
Here's a transcript.
16 matches, good.
Breadth-first search has a different set of properties.
Notice that it took me longer.
But because it's breadth first, and because each row corresponds to an increasing path length, it's always guaranteed to give you the shortest answer.
That's good.
So it always gives you the shortest answer.
It requires more space.
I mean, you can see that just on the chalkboard.
And also it still didn't take care of your problem.
So this still seems like there's too much work.
I looked at 16 different places.
I did 16 visits.
There's just something completely wrong about that, because there's only 9 states.
How could it take more visits than there are states?
So that just doesn't sound right.
And it's for exactly your point.
And so there's another idea that we can use, which is called "dynamic programming."
The idea in dynamic programming, the principal is, if you think about a path that goes from x is z through y, the best path from x to z through y is the sum of two paths-- the best path from x to y and the best path from y to z.
If you think about that, that has to be the case.
And if we further assume that we're going to do breadth first, then the first time that we see a state is the best way to get there.
So what we can do then, in order to take care of your case, is keep track of the states we've already visited.
If we've already visited a state, it appears earlier in the tree, there's no point in thinking about it further.
That's the idea of dynamic programming.
And that's also easy to implement.
All we need to do is keep track of all those places we've already visited.
So we make a dictionary called "visited."
So I initialize before I start looking at the children.
I initialize right after I set up the initial contents of the agenda.
I create this dictionary called visited.
And every time I visit a new state, I put that state in the visit list.
Then before I add the child to the agenda I ask, is the child already in the visit list?
If the child's already there, well forget it.
I don't need him.
Otherwise, just before you push the new state, remember now that that's an element that's been visited.
So the idea, then, is that by keeping track of who you've already looked at, you can avoid looking-- so if there's a depth-3 way to get to D, and a depth-2, then I don't need to worry about the previous ones, because it's already in the visit list.
Yes?
Why do we still need the new child states up there?
Why do we still have the new child states?
The placement was based [?
on-- ?]
the [INAUDIBLE] state
I think you're right.
I should think about that.
I think you're right.
I think when I was modifying the code for the different places I slipped and could have removed that line.
I think you're right.
I'll have to think about it, but I think you're right.
So if that line magically disappears from the online version, he's right.
OK, so now one last problem.
I want to see if I can figure out what would happen with dynamic programming.
So I want to do breadth first.
And just as a warning, I'm hypoglycemic at this point.
So there may be more errors than usual.
So I need to keep track of two things.
I need to keep track of the visit list.
And I need to keep track of the agenda.
So there's two lists I have to keep track of.
OK. Let's start out by saying that the agenda contains the start element.
That's A.
That means we visited A.
It's breadth first, so I want to take the first guy out of the queue and add his children to the end of the queue.
So take the first guy out of the queue.
Add his children.
A's children are B and D, which means that I've now visited B and D. Now I want to take out the first guy from the queue, AB, and I want to put his children at the end of the queue.
AB's children are A-- that's been visited, C-- not visited, and E-- not visited.
So ABCE.
But that visits C and E. Now I want to take out AD and put its children at the end.
AD is AEG.
A is visited, E is visited, which leaves just G, so ADG .
And that visits G. Then I want to take out ABC and put in its children, A, B, C. ABC, oh dear.
AB-- I'm looking up there.
I said I'm hypoglycemic.
ABC-- ABC -- could be B or F. Well, B's no good.
Which leaves F, but that visits F. So now, ABE.
ABE, so that could be B, D, F, H. B-- visited, D-- visited, F-- visited, H-- OK. That visits H. Now take out ADG.
Children of ADG-- ADG, two children, D and H. D and H, they're both there.
That didn't work.
There are no children of ADG.
ABCF-- ABCF, three children-- C, E, I.
C-- visited, E-- visited, I-- done.
Found the right answer.
1, 2, 3, 4, 5, 6, 7, 8 -- 8 visits.
So I've got the same answer, and it's optimal, and I did fewer than 9 visits.
9 was the number of states.
So this algorithm will always have those properties.
So the dynamic programming-- oh, I forgot a slide.
The dynamic programming with breadth-first search will always find the best.
It will never take longer than the number of states.
So in this problem that had a finite number of states, even though I had an infinite number of paths-- because you can go around in circles-- it'll still never take more than the number of states.
And all that it requires to implement is to maintain two lists instead of one.
So the point, then, is that today we looked at a variety-- we looked at two real different kinds of search algorithms, depth-first search and breadth-first search.
And we looked at a number of different pruning rules.
Pruning rule (1) -- don't go to some place that you've already visited.
Pruning rule (2) -- if you have two children that go to the same place, only think about one of them.
You can consider dynamic programming to be a third pruning rule, because that's the effect of it.
And the final announcement, don't forget about Wednesday.
Have a good week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So hello.
Today I want to talk about, and I want to finish talking about, search algorithms.
So last time we started to think about a framework within which to think about search and the important thing was to figure out a way to be systematic about it.
So we figured out a way to organize the way we think about searching by way of a tree.
Put all the possible places where we could be in the search, consider all the possible actions that we could take, so if we started it at A, there are two different actions we could've taken, we could have gone to B or D Think of the actions as the edges of the graph and then think about where we land.
Then, by way of that graph, think about the shortest path to the goal.
So that was the idea and the big outcome was order matters.
So if we were to construct an agenda, which is the list of nodes that we are currently considering, if we started at A, we would start by putting A on the agenda.
Then we would pop A out of the agenda and replace it with its children, its children are AB and AD.
If the algorithm was replaced, the first node in the agenda by the children, the first node is AB.
So then we would pop AB and replace it with its children, AB has children ACE, etc.
And the result would be something that we call a depth first search, because what's happening is we're following line lines deeply before we look crosswise.
And there's a million varieties of this that you could think of.
You would get the same kind of algorithm if you replaced the last node by its children.
Then you would start with A, replace it by its children, which is still B,D -- but now expand D in terms of its children.
Then you'd take the last trial, expand it and we would get another depth search.
So the idea is whatever order you choose affects the solution that you find.
Generally, we're interested in finding a search that locates the best, shortest path, and so a good method is to remove the first node from the agenda and add the children to the end.
That's a cube-based ordering where the first out is the first in.
Then if you imagine starting at A, replacing it by its children, B,D, take the first guy out-- that's AB-- consider its children and put them at the end of the list.
Then we go back and expand AD before we think about the children of AB and so the result-- if you just follow the red up here-- the result of that search is what we call breadth first.
So we systematically propagate down the search tree looking at short paths first.
So that's the idea, the idea is search is easy, you organize it around one of these graphs and just systematically run through the search by keeping track of what we call the agenda, the nodes under consideration, and the only trick is that order matters.
We found two useful orders last time, first in, first out and last in, first out, one of those giving depth first, which is generally not a good idea.
The other giving breadth first, which is generally a much better idea.
Today what I want to do is generalize that structure to take into account a much more flexible group of problems.
That'll give rise to something we call uniform cost search and the other thing that I want to think about is, again, the order.
By thinking about the order, we can drastically improve the efficiency of a search and that idea gives rise to something that we'll call heuristics.
So the idea then that I want to look at first in terms of uniform cost search is the idea that so far, we've only looked at problems where the cost of each child is the same.
We were only looking at how many children in total, how many generations did we have to go through to get from the starting point to the goal.
That's an important class of problems, but it's by no means the only important class of problems and in fact, the most trivial problems you can think of don't fall into that class.
So for example, imagine that what we were doing, so I motivated the entire problem last time in terms of searching for a path on a Manhattan Grid.
So if I wanted to go from A to I, where all the distances are equal, then minimizing the number of generations, minimizing the number of intersections that I go through, will give rise to the best search.
However, imagine that one particular node is way off the map.
Then the number of generations that I go through is obviously not the right answer because there's a penalty for taking a path that goes through C because the distance between B and C is so much bigger than the distance, for example, from D to G. So the first thing that I want to look at is how do we incorporate that kind of new information into the search algorithm?
So before I do that, think about how the breadth-first search-- the thing we found last time that was the best-- breadth-first search with dynamic programming -- think about how it would approach the problem and think about as we go through step by step, what is it doing wrong?
OK?
So I'm going to go through an algorithm that is known not to work -- with the idea that you're supposed to identify as I'm doing that what step was wrong.
So imagine that I'm doing this and then I'm going to compare it in a moment to C being off the map.
So if I do this problem with dynamic programming, I have to keep track of how many nodes I already visited and I have to keep track of all the nodes under consideration.
The nodes under consideration is what we call the agenda and I will call the list of nodes that we're keeping track of for dynamic programming, I'll call that the visited list because we'll add nodes to the list.
We'll add states to the list as we visit them.
So we start out the algorithm with node A being on the agenda, we're trying to go to node I.
And by the time I've started, I've already visited A.
So the starting point is that the visited list has one element in at A, the agenda has one element in it, A.
So the algorithm is going to be pop the first guy out of the agenda and add the children to the end, paying attention to the visited list.
So pop A out of the agenda, the children of A are B and D. As I visit them, they get added to visited list, B and D. So I pop the first person out, that's AB.
Then I want to think about the children of AB, that's ACE.
A is already visited, so I don't need to add that again, but C and E are not, so I add them and add them to the visited last.
Then I pop AD out, well the children are AEG.
A and E are in the list already, G is not, so I end up adding G to the list.
Next is to pop out C. The children of C are B and F. B was in the list, F was not so I add the trial that has F at the end.
Then the next one is E. E has children BDFH.
B,D,F, the only new one is H, then G. Take G out, the children of G are D and H but they're already in a visited list, so I don't need to worry about them.
Then A,B,C,F, so F is this guy, C,E,I.
CE I is not in the list, and that's my answer.
Yes?
[UNINTELLIGIBLE] once more, then you get another path that's equally short.
So why is that not the correct answer?
A slightly different algorithm might give me that solution.
All that I'm tracing here, I'm trying to be consistent with the algorithm that we discussed last time.
But that's a very good point.
So the breadth-first search with dynamic programming is guaranteed to give you a solution that has minimum length.
It's not guaranteed to give you a particular solution of minimum length.
So when there exists multiple solutions with the same length, this search algorithm might to give you any of them, and that's an important thing to keep in mind.
So with regard to the problem, I'm trying to think ahead.
I'm trying to think ahead to where this C is off the map, it's over here someplace.
So what I want to do is stop thinking about how many hops there were and start thinking about how many miles there are.
The first thing I want you to notice is that this search pattern that we did created a visitation list.
We visited the states in the order of increasing number of hops.
That's obviously a good thing.
If we can always keep in the agenda the smallest number of hops to the next place, and if we faithfully visit states starting at the minimum number and proceeding up, so we started with A, there's no hops in getting to A, so that's 0.
In going from A to B, there's 1 hop.
In going from A to D there's 1 hop.
In going from A to B to C, ABC, there's 2 hops.
So what the algorithm that we described last time-- breadth-first with than dynamic programming does-- is it visits the states in the order of increasing number of hops.
That's obviously a good thing.
OK so what's my next slide?
I want to think about C being off the map.
So what I'd like to do is think about what order would this algorithm visit number of miles?
So If I think about replacing the metric in the bottom with the number of miles, whenever the different actions that can occur incur different costs.
Think about what I'm saying.
So I'm saying that if I were at B and I think about my children ACE, which action I take, go from B to A or go from B to C are go from B to A, which action I take incurs different cost.
So what I want to do now is think about, change the focus from thinking about how many hops is it to thinking about how many miles is it.
So now, if I replace the metric-- so over here, the metric was how many hops, replace how many hops with how many miles-- and what you see is that the algorithm is not picking up.
It's not visiting the states in order of increasing path length.
That's what's wrong.
So what we'd like to do is modify the algorithm somehow so that it proceeds through the paths shortest to longest.
So that's the goal.
And that's pretty easy to do.
The first thing we have to do is put that new information somewhere, and I've already alluded to the fact that the way to think about the new information is that it's associated with actions.
It's not associated with states.
States are where we go to in the diagram, like state E. The extra cost is not summarized in the state, the extra cost is summarized in the exact path that we took.
And the way we'll think about that is incrementally.
So we create the past by doing actions and each action has a different cost.
So the first thing we do is associate this additional cost with the actions.
Then we are looking for a search procedure that will enumerate the paths in the order of path cost.
And the way to do that is to think about the basic ordering schemes that we had before, which were the stack, last in, first out versus the queue, first in, first out.
What we'll do is we'll make a trivial modification to the idea of a queue and we'll call it a priority queue.
So a priority queue is basically like a queue, except the things that are queued have priorities.
So the idea will be that when you push at possible action, say I had actions A,B, or C -- in addition to pushing them onto the queue, which is how we would have done breadth-first search, in addition to pushing them on the queue, I'll also associate with them a cost.
So when I push A on the queue, the priority queue, I'll associate with that a cost, which I've arbitrarily said here, the cost is 3.
When I push B, I'll associate a cost 6.
When I push C, I'll associate a cost 1 so that when I do the first pop, what will come out will be the element that I pushed that has the smallest associated cost.
That'll be a way that I can order then my search through the search tree in terms of the minimum costs.
Is that clear?
So the first time I do a pop, the element that pops out is the one with the least cost, which is this one, so I get C. C is then removed from the list and the second time I do it, when I do a pop, I pick out the one with the cost 3 which is A.
That's an easy modification to the schemes that we used before.
Just like before, we can implement a priority queue with a list.
Here we've put all the complication into popping.
So we just push like we did before, just jam it in.
So just add it to the end of the list.
And we put the complication into pop, pop looks through the list and finds the one that has the biggest negative cost.
That's just because we've got a utility routine that gave us the biggest.
We want the smallest.
We know that the costs are non-negative, so we implement the find the smallest by calling the routine find the biggest with a negative cost.
So notice that all the new complication is in this pop routine, and if you think about it, pop is doing far too much work.
The way this routine works, every time you do a pop, it goes through the whole list looking for something that is small.
Obviously it ought to be able to keep track of progress that it's made in that previously.
There are much better algorithms, but for the purpose of this illustration, we didn't bother with it.
So this is not a very clever implementation.
We're not trying to be clever, we're trying to illustrate the point.
So if you were seriously try to do a big search, if you're writing code for Google, you would never do it this way.
But the idea again is in abstraction.
We're going to bury those details in the way the queue is implemented and then at the next higher level, we don't need to worry about those details.
We'll abstract them away.
Is that all clear?
OK.
So this then is the way we end up doing the search.
So when we do the search, when we create a new node, the idea is going to be that we can generate a cost.
Remember, nodes in the search tree summarize paths.
Nodes are different from states, right?
States are places that we can visit, nodes are paths.
So the thing that we need to keep track of now in addition to what we did before, before we had the idea that nodes had states, they have a place where you are currently in the search.
They have actions, which is which direction do you go next and they have parents and that was enough information to create and maintain a search tree.
Now what we need to do is also keep track of cost.
So we do that by adding instantiation time.
When we create a new node, we have to also pass it what is the action cost.
So in the previous example, the action cost would be the distance from B to C, for example, being 5, which is different from the distance between B to E, which is 1.
So we have to associate at the time we create a node, what is the action cost associated with this new node.
And then the node keeps track of the total path cost, so that's what the red stuff is doing.
So it's a very small change to the code that we used for creating nodes in the previous two searches, and then we have to also change the way we do the basic search algorithm.
And here too, the idea is pretty simple.
It's almost exactly the same thing that we did before, with the idea that we substitute keep track of the agenda with a priority queue rather than with a queue or a stack.
There's one more complication, and that is that in the past, we knew that all children added the same penalty.
Because we only keeping track all of how many generations, how many hops are there to the current node, all children were in some sense created equal because we knew they all incurred one more hop.
Here, because the children can have different associated action costs, we don't know at the time we've picked up the parent, we don't know at that time which child is going to end up being the shortest one.
So previously, the goal test was performed when the children were pushed onto the agenda.
Here we have to wait until we look at all of the children before we will know which child has the shortest length.
And that just means that we take the goal test, which had been inside the 'for a in actions' loop.
We have to factor that out and defer until the next time.
So that's the only change to the algorithm that we need to make.
So is that clear?
So the idea is it's a pretty simple modification of the algorithm that we have so far, but it gives rise to a much more versatile kind of search.
So last time we saw that there was really no good point for not doing the search, the breadth-first search with dynamic programming, we have the same thing here.
So when you're doing the uniform costs search, there's no real good reason for not implementing that with dynamic programming, so we also want to think about the dynamic programming algorithm.
So you remember in breadth-first search and in depth-first search, dynamic programming referred to the principle that the shortest path from X to Z through Y, so if you have a path from X to Z and you know it goes through Y, the shortest way you can get from X to Z through Y is to add the shortest path from X to Y to the shortest path from Y to Z.
Sounds trivial, but it has a tremendous impact on the number of states that can be omitted from the search.
Here, when we do the uniform cost search, we can do the same sort of thing but except that we run into the same sort of problem with not all paths from X to Y are created equally.
We don't want to remember any random path from X to Y, we want to remember the best one, which means that in general, we're going to have to expand all of the children before we'll know which child gives the minimum length path.
So that means that the dynamic programming principle that we'll use is to remember the state when it gets expanded, because it only gets expanded after its already being compared to its siblings.
So wait until a state, wait until a path has been compared to all the siblings of that particular path before you consider it for dynamic programming.
So we'll do that by not keeping track of states as they are visited, but instead keeping track of states as they are expanded.
That's the same idea that we had to do when we had to reorder goal test.
Defer the goal task until after you think about all the children.
Defer putting it in the dynamic programming list until after you've looked at all the children and so that means that we think about expansions instead of visits.
And so that looks like this.
So just like we took the goal test outside the action list, in the previous breadth-first search we did the test goal state in this loop, here we defer it until we've looked at all the children and fetched the parent.
So we defer it into the higher loop, we take it out of this moving into this and similarly, we take the dynamic programming memory.
We now call it expanded to remind ourselves that what we're doing is keeping track of states after they were expanded, not after they were visited.
And updating it again, not when we look at the individual actions, but only after we've decided which trial is the winner.
OK?
That's probably confusing.
That's not important at this point, because I've written an example and hopefully by thinking through the example, you'll be able to see what the code is supposed to be doing.
And then a good exercise is to go through the example, which will be posted on the web with all the gory details, and make sure that you can match up the search-- the partial results of the search-- to the way the algorithm is written.
OK?
So now I want to do the problem of interest.
Think about what if one of these nodes, one of the states, state C, is very distant relative to all the rest.
How will the new search, the uniform cost search, work with this problem?
So we do the same sort of thing we did before.
Now we have an agenda to keep track of the nodes under consideration.
We have a dynamic programming list that is going to be called expanded because we don't add to it until we expand states.
It had previously been called visited.
And we also keep track of the metric, which is the path costs.
So if we start by putting node A on the agenda, its path cost is 0, because it doesn't cost anything to get there because that's where we started.
And we haven't expanded anybody yet.
So notice that the starting state is a little bit different here because we're keeping track of expanded.
The expanded list started out with 0 elements in it.
Previously where we were keeping track of visits, it started out knowing already what was going on with A.
So now we expand A, think about A's children, B and D, put them on the agenda, and add A to the expanded list.
We expanded A so it's time to add it to the dynamic programming list.
Then also keep track of the costs of these paths.
AB costs 1, AD costs 1.
That's the end of the first pass.
Now pop the first guy off the queue.
Expand it, that means we're expanding the B state.
We go from A to B, so we're going to expand B.
B Has children A, C and E. A has already been expanded so we don't need to think about A anymore.
C and E, so it was A, C and E, C and E have not been expanded, so I expand B to get these two and I associate their total path costs.
So the path ABC, ABC has length 6 and the path ABE has length 2.
Then I take the first one off the agenda again, that's AD.
I expand AD, which is expanding D. Put that on the expanded list, on the dynamic programming was list.
D has children A, E, G. A is already on the expanded list, so that means I have to worry about E and G. Those paths have length 2 and 2.
That was the same so far as the previous search.
Now I see something different.
Because I'm using a priority queue, I skip ABC because that's just not the right one to do next.
So ABC, I'm keeping track of with the priority queue.
I know that that path is already length 6.
It could end up being the optimum path, but at this phase of the search, it's not the one I should think of next.
The one I should think of next is the shortest path.
So what I'm trying to do is search the search tree in order of shortest path.
So I skip the ABC because its path is long and the minimum length, so back up one.
So in the priority queue idea, I want to extract from the queue the first item with the minimum length, so that's ABE.
Everyone's with that?
So then I want to expand E. E has children B, D, F, H. So B and D are here, F and H are not, so I add ABEFH to the agenda.
Each of those have length 3.
So what's the next guy out of the agenda?
A and
I need the first one with the smallest path, the smallest cost possible.
The smallest cost possible is 2, so ADE is the next guy.
So I expand ADE, but I've already expanded E. I Don't need to do anything.
OK?
The dynamic expansion list is keeping track of the nodes I've already expanded.
I already did E, there's nothing new to be learned.
So that doesn't do anything.
So the next one is G. Well I didn't do G yet, so add it to the expanded list.
G is shorter than D and H. D was already there, H was not, so add H. The next one is F. I haven't expanded F yet, so let's do F. So F's children are C, E, I.
C is not there, E is there, I is not there, so I add those.
So next is this guy, H. H wasn't expended, so I add H to the list.
H's children are E, G, I. EG are already there, I is not, so I add that.
Try expanding H, but already did expand H so that doesn't count.
Try expanding C, that's fine.
Expand C, I haven't expanded C before, add it to the list.
C's children are B and F. B and F are both there, didn't add any new children
Why'd you expand that C, why not H?
Oh look at that, you're right.
Oh, thank you very much.
My slide is wrong.
Ignore that, you're absolutely correct, thank you.
So let's say I guess this proves that 200 people watching the lecturer have greater insight-- well anyway.
Yes, that's wrong.
Don't bother with that because it's got the wrong priority.
Jump straight to that.
I'll fix the slide on the web.
You're absolutely right.
I should have gone straight to there and when I tried to expand I, I realize that that's my answer, so I'm done.
OK?
Questions?
OK it's a little tedious.
The point is that it really wasn't very much different from doing breadth-first search.
The same ideas still work.
The same idea of organizing the search on a tree, looking for the minimum cost answer on a tree, that's the big picture.
That's what we want you to know about.
I mean, we may ask you a quiz question about breadth-first search or depth-first search or dynamic programming or whatever, but the big picture, the thing we really want you to know is how to think about a search.
The way to think about a search is a sequence of states organized in a tree that you can then systematically search.
That's the idea, and the point of going through the uniform cost search is to see first and foremost, that it fits the same structure, it's the same kind of problem.
Secondly, there are very tiny details and so I've tried go over those details.
The details have to do with keeping track of the action cost and keeping track of which one to do next by way of a priority queue.
But the big picture is the same idea works.
States, nodes, trees, search, cues.
Questions?
Comments?
OK.
The other important thing that I want to talk about today is again, this idea of trying to minimize the length of your search.
The other point that you're supposed to get from these two lectures is depending on exactly how you set up the search, you can do a lot of work or less work.
And we're always interested to do less, especially because if you can do a lot less, you can do a lot harder problem.
So the other thing I want to talk about next is the idea that our searches so far have been starting state centric.
What do I mean by that?
Every search has a starting state and a goal, and the ordering of our searches so far have been go to the starting state, think of all the steps you can make from the starting state and just keep widening your search wider and wider and wider until you stumble onto the goal.
OK, can you all see that that's what we've been doing?
So nowhere in our code was the code aware of the goal other than am I there yet?
So the search algorithm so far has been start at the beginning state, ask if I'm there yet, try the closest place I can go, am I there yet?
Am I at the goal yet?
Try the next closest place I can go from the start, am I at the goal yet?
Try the next closest place I can start from, am I there yet?
Everything has been starting state centric.
Obviously, that's wrong.
Somehow, when you do a search, if you were asked to find the shortest route through the interstate highway system from Kansas to Boston, there's a good chance you wouldn't be looking at Wyoming.
Everybody knows enough geography to do that one, right?
So the idea is that in the searches we've done so far, you would look at Wyoming before you'd look at Massachusetts.
OK?
That's stupid, right?
Everybody sort of see that?
So the searches we've done so far have been starting state centric.
So what I'd like to do is think about a way to undo that.
But again, to set things up, let's think about what I mean by that.
Let's imagine a search very much like what we did before, except let's go from E to I, from Kansas to Boston, and I guess it's Kansas to Tallahassee, but you get the idea.
So we start at E and let's think about how our search proceeds.
So if you start at E, think about if we push E in the agenda, so I'm doing the uniform cost search with dynamic programing.
I'm keeping track of the expanded state as my dynamic programming state.
I'm starting at E. The cost of being at E is 0 because that's where I started.
And now, I think about expanding E. So E goes on the expanded list.
I think about all the children of E, the children are B,D,F,H.
All of those children have distance 1, so when I look among them to figure out the next one that I should do, I do the first guy, EB.
B's children are A and C, so I take off EB from the beginning of the agenda.
Start again, B.
B's children are A, C, E. E is already expanded.
Push A and C. Pop D, D's children are A, E, G push A and G because E's already there.
Expand F. F's children are C, E, I. E is already there, push CI.
Expand H. H's children are G, I, E, G, I. E is already there, push GI.
Expand A.
A's children are B and D. They're already there, don't need to do anything.
Expand C. C's children are BF.
BF are there, I don't need to do anything.
Expand A. BD are already there, I don't need to do anything.
Expand G. G's children are DH, DH are both there, I don't need to do anything.
Expand C. BF, BF nothing.
Expand I, I'm there.
Yes?
Were you supposed adding ACE?
Was I supposed to be adding AC?
You add A when you expand it.
Ah, yes, yes, yes.
Thank you.
I'll fix this one, too.
So the question was when I expanded A, when I did this one for example, when I tried to expand A, I should have added it to the list here so I don't try to do it again.
It wouldn't have affected the outcome, but I should have added it to that list.
Thank you.
Class -- 2 Freeman -- 0.
Yes?
What if you didn't expand it?
I mean, do you consider it [UNINTELLIGIBLE] you should expand it?
I'm sorry, I didn't hear the question
We didn't actually expand it.
We considered expanding it, but then we noticed that
Correct, correct.
So what I really should do is go back and read the code and figure out what I should do.
What I'm trying to do is emulate the code.
So your point is, it's a technical definition about whether I want to think about whether I actually expanded it or not.
If I don't add any children, was it a real expansion?
And that's a question that is determined by where was the if statement in the code, and frankly I don't remember.
It does not expand it, ah I have the right answer
It does expand it
It does expand it.
So I have the wrong answer.
Class -- 2 and a 1/2.
OK so the point of this is that the search was symmetric around E, even though the goal is not.
OK?
And the question is, how could we fix it so the search is not symmetric around E -- the starting point.
How do I fix it so the search is biased toward going toward the answer?
And so the way we think about this is with something we called heuristics.
So if you think about the searches we've been doing, either breadth-first or depth-first from last time where we counted the number of hops or today where we counted the lengths of paths, each of those considered what thing to do next based on the path from the starting point to the point under consideration.
The idea of a heuristic is to add something that informs the search about how distance, how much distance we're expecting to add from the point under consideration to the goal.
So the idea of the heuristic is to put in the second part of the path.
The problem with a heuristic is that finding the second part of the path is just as hard as finding the first part of the path, and it would be a terrible idea to -- for every point in the search tree run a new search to find the best answer from that point to the goal -- because that would increase the length of the search time enormously.
So it's a bad idea.
So the problems are of equal complexity, the problem of getting from the start point to the place of interest and the problem of going from the place of interest to the goal are problems of equal complexity.
We don't want to try to solve the problem of making the search better informed by increasing the complexity of the search drastically.
So that's the issue.
So a heuristic is going to be a way of approximating the amount of work we have to do yet, where what we would like it to be is not all that terribly difficult to compute.
So one way we could think about that would be to consider as an approximation to how much work we have to do, the Manhattan distance for example, to complete the path.
The Manhattan distance is the sum of the x and y distance.
Generally speaking, Manhattan distance is not a good idea for map-like problems because generally you can cut across corners in such searches.
In this particular search, since I've excluded cutting across diagonals, since the search space is already Manhattan, thinking about a heuristic that's based on Manhattan distance is probably OK.
So the idea is to develop a heuristic and he what I'm going to think about is, what if I complete the path by adding the Manhattan distance from the point under consideration to the goal.
OK, so now if I started at E like before and I'm going toward I like before, then I start with the agenda having just E and having expanded nothing.
However, the cost associated with E is no longer 0.
The cost of going from E, the starting point to E, the point under consideration is still 0, but I'm estimating the cost of going from E to I by the Manhattan distance between E and I, which is 2.
The Manhattan distance between E and I is you have to increment x by 1 and you have to document y by 1.
So instead of saying that the cost of state E is 0, I'm saying that it's 2.
Is that clear why I'm doing that?
So now I expand E, I think about the children B,D,F,H -- BDFH, and those children now are not the same cost.
Even though it costs the same amount to go from E to each of its children, going from its children to the goal, going from each child to the goal does not cost the same amount.
If I were to go from E to B, that's a cost of 1.
But the Manhattan distance from B to I is 3.
So I'm estimating, then, that the cost of making the decision go from E to B is 4.
The real cost of going from E to B plus the estimated cost of going from B to I.
Similarly, if I go from E to D, the Manhattan distance is 3, so that's a length 4.
If I go from E to F, Manhattan distance from F to I is just 1, so the total cost is just 2.
So now rather than circling out from the starting point, my next step is biased toward the goal.
So my next step is biased toward so the remaining items, the smallest cost is 2, so the next place that I expand is EF.
F's children are C, E, I. E is already here, so I only think of CI.
C and I also have different costs, So the cost of going EFC, EFC, the direct cost is 2 and the estimated distance, the Manhattan distance between C and I is also 2, so that's a cost 4 whereas the cost of EFI, EFI has a direct cost of 2 and the Manhattan distance from I to I is 0.
So now I look for the minimum distance that's remaining, so that's going to be H. I expand H, the children are E, G, I.
So E is already here.
I think about GI.
Same sort of deal, some of them are short, some of them are long and as I proceed to the search, I very quickly find I without having ever looked in the wrong direction, or at least having looked minimally in the wrong direction.
Is that clear?
So the idea in a heuristic is to add an estimate of how much it will cost to complete the path so that you bias the search toward the goal.
Rather than making circles that spiral out from the starting point, make the spirals biased toward the goal.
And here's the way you do that.
It's very easy.
All you do is every place we would have looked at cost before, add the heuristic function.
So you have to add a heuristic function, that's the part that's hard.
The code, given the heuristic function, is easy.
The really hard part is actually figuring out what a reasonable heuristic is.
The reason that's hard is that you have to be careful not to miss the solution.
So the heuristic function can't be bigger than the actual distance.
OK, why is that?
The agenda is trying to keep track of all the possible places that you could look next.
If your heuristic function makes the next step look too big, it'll be taken out of the agenda and never appear again, and you'll never find that path.
So when you're making a heuristic function, you have to be very careful not to ever overestimate the distance from where you are to the goal.
If you ever overestimate it, then you can exclude forever more, so think about the agenda as a pruning operation.
We did this starting last time.
When we think about pruning, we pop off things from the agenda.
We say -- don't ever need to look there, don't need to look there.
We pruned it.
If you have a heuristic that it is too big you can prune a path that was the answer.
So you have to be careful never to do that.
So it's asymmetrical.
If you were to put in a heuristic that is too small, that causes you to underestimate the penalty of going in the wrong direction.
So if you said, I'm trying to go from Kansas to Boston and you inadvertently said that Wyoming really didn't cost anything, then you would not necessarily exclude the correct answer, but you would include and cause the search to consider a necessary path.
So the idea is that you would like the heuristic to be the same as the real cost or smaller.
You don't want to end up solving another search problem in order to calculate it, because that will increase the cost of doing the search too much.
So you would like some number that's easy to calculate that has the guarantee that it will always be less than or equal to the actual cost, and that's the art of doing a heuristic.
If you satisfy those, then that previous algorithm that we looked at, which is -- in the literature it's call the A-Star Algorithm for historical reasons.
That algorithm, if you obey these rules for finding admission an admissible heuristic, if you find an admissible heuristic, then the A-Star search will be a lot faster and still find a solution with the same length.
OK, so now just to see if you're following what I'm saying, here's a question to ask yourself.
Remember the tiles problem?
The tiles problem is the first problem that I did last time.
The idea was, imagine starting in this configuration 12345678.
Move one tile at a time by moving the tile into the free spot, so I could move 8 to the right or 6 down.
Keep doing that until you perturb this configuration into this configuration.
We saw last time that there are a large number of states-- there's a third of a million states-- so this is a big search problem, even though it's something you've almost certainly also solved as a child.
And in fact, my version, it was the same as yours, I'm sure, was a 4-by-4.
I did the 15 puzzle, not the eight puzzle, but it's the same idea.
So what I want to do is consider heuristics.
Consider three heuristics.
Heuristic A is 0.
It always returns 0.
That's an easy heuristic to calculate, right?
Heuristic B is sum the number of tiles that are in the wrong position.
What is heuristic B for this state?
8?
8 -- there's 8 tiles that are out of position.
So heuristic C is the sum of the Manhattan Distances required to move each title to their respective locations and then calculate two partial sums.
Consider MI to be the number of moves and the best solution if you use heuristic I and EI is the number of states that are expanded while you're doing the search.
If you use heuristic I, which of the following statements are true?
OK, take a minute, talk to your neighbor, figure it out which of these are true.
[CLASS TALKING]
OK so what's the smallest numbered correct answer?
The smallest numbered correct answer.
Oh, come on.
Volunteer.
Explain it with your neighbor.
OK, very good.
The smallest numbered correct answer is (1).
MA equals MB equals MC.
Why is that?
How can I prove that MA equals MB equals MC.
What do I have to do?
Yes?
You'll have to reach the shortest possible solution
So under what conditions will they all reach the same length solution?
They're all doing backflips
They're all doing backflips.
There's another condition.
Yes?
That they're all either exactly [INAUDIBLE]
So the heuristics, all three heuristics have to be admissible, which means that they have to be non-negative numbers.
To be admissible, a heuristic has to be non-negative.
And it has to generate an answer that's smaller than the actual number of moves necessary to complete the path.
So we have to prove that these are all admissible.
So, are they all non-negative?
Yes.
Are they all less than or equal to the number?
So let's do that.
Let's first of all get an answer.
So we'll do that after we do the second part.
What's the second smallest correct statement?
Let me see if I can answer it, yes?
OK, the answer is 4.
So, how do you know that EA is bigger than EB is bigger than EC?
So the number of states expanded has to do with the size of the heuristic.
If the size of the heuristic is 0, that's the same as not using a heuristic.
That will search all possible states in a breadth-first search fashion.
If the heuristic is anything that's bigger, the number of searches will go down.
What's the greater than or equal to thing doing here?
Yes?
Number (3) is technically also true, because if (1) is true, then (3) has to be true
If the number (3) is technically the number M. [LAUGHTER] OK, OK, OK. [APPLAUSE] OK, class, three plus.
Freeman, oh well.
Yes, you're right.
Yes, I agree.
Did you raise your hand for (3)?
So (3) is technically the second, yes that's right, that's right.
OK so moving on.
Number (5) -- the same best solution will result for all the heuristics -- true or false?
Come on, things can only go downhill for me.
So the same best solution will result for all of the heuristics?
How could it possibly be false?
Right?
Didn't we already said MA was equal to MB equals MC.
Yeah?
It's the same amount of moves, but maybe not exactly the same solutions.
So if there are multiple solutions of the same length, the difference heuristics don't have to give you the same solution exactly.
They have to give you solutions with the same length.
So what happens with the heuristics is you perturb the order of search.
If you perturb the order of search, the only thing that is proved is that you get a minimum length solution, not the same minimum length solution.
This particular problem has lots of solutions and so you don't necessarily get the same solution when you use different heuristics, OK?
So the final point is that the addition of the heuristics can be extremely effective.
If you run this problem with our search algorithm, you always get solutions with 22 moves in them.
That's good because all the heuristics were admissible so you always get a right answer -- a shortest answer.
But the number of visited and expanded are drastically different when you add the heuristics.
So if you use here heuristic A, which is equivalent to no heuristic, you end up visiting 170,000 states to find this answer, where if you used the Manhattan distance to the goal, the sum of the Manhattan distances, you do a very small fraction of that.
So the point is that this stuff matters, especially when you do a higher-dimension search, which is all of the searches that we'll will be interested in.
So the idea is that the order really does matter and with that, I'll conclude with a reminder that tomorrow evening is the makeup/retake day for Nano Quizzes.
Please come to the lab if you'd like to make up or retake the Nano Quiz.
Hi.
Let's talk about object oriented programming in Python.
First, object oriented programming is a programming paradigm.
It's in the same category as things like functional programming and imperative programming.
But object oriented programming is going to be the programming paradigm that describes most of the code that you're going to interact with in 6.01.
So it's important to understand how it works and how, in particular, you want to be able to code in an object oriented programming paradigm in Python.
So today, I'm going to go over a quick crash course on object oriented programming, and also indicate all the little tips and tricks you need in order to program in Python.
Let's look at what I have written up.
So the most important thing to remember when you're learning about the object oriented programming paradigm is that everything is an object.
And what I mean when I say that, and what people mean when people say that, is that the ideals behind the paradigm are that you interact with your code in the same way that you would interact with objects in the physical world, right?
There's a particular piece of paper on the desk in front of me.
And it is a kind of piece of paper.
So I know how to interact with it the way you would interact with any other piece of paper.
If you want to codify this in an object oriented programming paradigm, you write up classes.
Classes are your basic unit of code block.
They describe what a thing can do and what a thing has or what attributes a thing has.
And in object oriented programming-- frequently object oriented programming and in object oriented programming in Python-- we refer to those things as methods or functions that a particular object may have, and attributes or particular variables that an object may have.
Once you've codified what any object of a particular class can do, you can then use the code that you've written to instantiate an object.
An object is the functional unit in the object oriented programming paradigm.
It's the thing that you interact with, and tell what to do, and produces results for you.
It's the thing that makes up-- it and classes those are the two things that you need to think about.
But you also have to think about how they're different.
I have a particular sheet of paper in front of me.
It has all the properties of a sheet of paper.
And when I think about all the things I can do to a sheet of paper, that constitutes a class.
But the particular piece of paper that I have is an instance of that class.
It's a particular piece of paper.
That's the gist of object oriented programming and the things that you need to know.
Now that I've covered them, I'm going to go over the most basic class I could come up with in terms of object oriented programming in Python.
This is a class that specifies what a 6.01 staff member has in terms of an attribute or a method in Python.
If I want every instance of a particular class or every staff member of the class, staff member 6.01, to have a particular attribute, I can specify like this.
Every instance of Staff6.01 is going to have an attribute room.
And that attribute room is going to be set to the string describing 34-501, the 6.01 lab.
If I want every 6.01 staff member to be able to do a particular thing, or have a particular method, or call a particular function, then I specify it like this.
This is the beginning of a method in the class Staff6.01.
It's called sayHi.
I'll talk about self in a second.
Don't worry about it.
Act as though-- if this is your introduction to Python programming, then pretty much pretend it's not there.
It's kind of like this, but we'll cover that in a second.
And if any instance of the Staff6.01 class calls sayHi, then "hello" will be printed to standard out.
I have a couple examples up on the board behind me.
And if you type them into Idle and see what their return is like, you'll be able to-- after you've typed in this-- you'll be able to interact a little bit better with what Python considers classes, and objects, and that sort of thing.
If you look at type Staff6.01, it'll tell you about a class, which is an object in itself.
But it's a specification for instances of an object.
If you want to instantiate an object that is of type Staff6.01, you need to use the parenthesess on the end.
This treats Staff6.01 like a call and creates an object.
If you just type Staff6.01, you're just reassigning Staff6.01 class to the name kpugh, and that's not useful.
Every Staff6.01 member should not be considered a Kendra, right?
Once you have instantiated a particular object of type Staff6.01, you can look at the type of that object, right?
Now you've got one object, kpugh, myself.
And that is a class of Staff6.01.
Likewise, now that you have this object, kpugh, you can look at its attributes and methods.
If you look at kpugh.room, then it should print to the screen "34-501."
That's because that's the attribute associated with this instance.
If you call kpugh.sayHi(), it will use the method in the class type of this object.
So when you call .sayHi(), it looks at kpugh, looks at the type, says, that's of type Staff6.01, goes to class Staff6.01, finds the definition for sayHi(), and then executes this code.
Hopefully, that all made sense.
Now, I want to talk about self.
And you might say, Kendra, I don't understand where that comes into play.
And you didn't even use it over here!
If you're familiar with C++ or Java, self is a lot like this.
Self is an implicit argument passed in here.
Even though you specify zero arguments, it's considered the first argument.
And you'll probably see a lot type errors when you're first programming in object oriented programming that say that you've either passed in too many or too few arguments.
It has to do with this definition, with self.
Self says, I am talking about myself.
That's not particularly intuitive.
But I'll try to explain a little bit more.
When kpugh calls .sayHi(), sayHi() always has an implicit reference to whatever called it.
When you look at this code, other instances can call this code, right?
If I had an instance of Adam Hartz or an instance of Ike Chuang they would also have access to the method sayHi().
And when they called sayHi(), sayHi() would point back to the class definition, but also have a reference to whatever instance called it.
So when you substitute in this self, you substitute in whatever instance called the method.
That doesn't seem particularly useful right now because that class definition does not actually make any use of the self, or the ability to use unique instances of an object as sort of unique storage containers.
I'm kpugh.
I'm different from Adam Hartz.
And therefore, I should be able to have different attributes, or different methods, or things that act slightly differently from the way they do for Adam.
I'm going to look at a revised definition of class Staff6.01, and that definition will use self in a way that indicates that you can have different functionality for different instances.
It's right here.
Class Staff6.01-- it looks really similar.
In fact, the first two lines are exactly the same.
We've got a class attribute.
That means that every instance of this class is going to have this attribute because it's a class attribute.
If I want different instantiations of my class to have different properties, then I need to explicitly address the initialization of those properties.
In Python, when we want to do that, we define the method __init__.
__init__ is a very special method.
It's got these underscores.
I think it's a protected keyword.
And it always has the format "self," and then whatever arguments you want to pass in when you're instantiating an object.
__init__ is not exactly a constructor.
But for those of you that are familiar with C++ and Java, it acts like a constructor.
Immediately after the object is constructed, __init__ is called.
And all the set up that is required to set up the object happens.
So any time you instantiate an object, all of these things are going to be executed.
Or all the things under __init__ is going to be executed.
In particular, we're going to set the attribute greeting to whatever argument we passed in when we instantiated the object.
So every instance that we create of this class is going to have the class attribute, room.
They're all going to be in the same room.
But they're also going to have a greeting.
And you have the option of specifying the greeting to be whatever you want.
We're going to make use of this in the method sayHi(), which still only takes the argument self, or the reference to whatever object called the method in the first place.
That reference is going to get substituted in here.
So no matter which object calls the method, you will have access to its particular greeting using this syntax.
Let's walk through an example.
Let's say I make an example of Adam Hartz, and he is a Staff6.01 member.
And his greeting is going to be "hi."
Likewise, let's make another instance of me using the new Staff6.01 definition.
Make sure you type this in because it's not going to work otherwise.
And make my greeting "HELLO," as opposed to just "hi."
If you call the sayHi() method using hartz, then you should get a different result then when you call the sayHi() method using kpugh.
But if you call the room method-- or, excuse me.
If you were after the room attribute of both instances, then you should get the same result because this is the class attribute definition, whereas this attribute is specific to each instance.
That's all I have to say for now about object oriented programming.
In my next video, I'll start to talk about inheritance, which is another really important property in 6.01 and also object oriented programming in Python, and also has some slip ups.
So I'd like to talk to you about those next.
Hi.
I'd like to talk to you today about inheritance as a fundamental concept in object oriented programming, its use in Python, and also tips and tricks for using inheritance in Python and in the object oriented programming paradigm in 6.01.
First thing I'm going to do is give a really quick crash course on inheritance to catch you up to speed, and also so that you're clear on what I mean when I say something like parent class.
And also, I'm going to address the sort of nuances of using inheritance while programming in an object oriented fashion in Python.
A lot of the code in 6.01 uses the object oriented programming paradigm.
And a lot of the code in 6.01 will inherit from parent classes.
So this is part of the motivation for going through a quick review, and then also indicating the most common slip ups, and also the most significant things that you may or may not have seen from other languages.
All right, first, a crash course on inheritance.
Let's look at the board.
Inheritance is the idea that you can arrange objects in a hierarchical fashion such that very, very generalized or basic things that are true of a whole group of objects can be specified at a higher level.
And then, you can work your way down to progressively more specific levels.
The most formal encounter you've probably had with this thing, that approach, is the biological taxonomy, right?
Kingdom, phylum, class, order, family, genus, species.
Every species has all the properties of that particular genus.
All the genuses of a particular family have the properties of that family, and so on, and so forth.
That's a very concrete example.
But I find it a little boring.
So I'm going to talk about dog breeds instead.
You're probably familiar with the fact that golden retrievers are a type of retrievers and that retrievers are a particular kind of dog.
You can make generalizations about goldens based on what you know about dogs, right?
All dogs bark.
All dogs have four legs if they aren't injured or have some sort of congenital defect, that kind of thing.
And goldens also have all the properties of retrievers, right?
They are capable of going and catching game that you've either shot, or possibly chase it down and bring it back to you.
So they're bred to have very particular properties.
Goldens are also bred to have very particular properties.
And those that are very specific to goldens define the difference between a golden versus a retriever in the general sense.
Likewise, when we want to make objects that have very particular properties but also share general properties with other objects, we're going to create a new category of object and put the specifics in that very specific category and then take the things that we can generalize and put them in more general categories so we don't end up rewriting a lot of code.
Or we end up reusing code, but not copying and pasting it everywhere because that's annoying.
The other major advantage of using inheritance is that code is more intuitive.
You can make references to the same piece of code all over the place.
But it's not as intuitively accessible to do that over, and over, and over again, right?
It's really convenient to think of the fact that golden could be a subclass or subtype of retriever, and that retriever could be a subclass or subtype of dog.
When I talk about this relationship in terms of object oriented programming-- when I talk about these categories in terms of object oriented programming and when you're actually looking at code, goldens are a subclass, or child class, of retrievers.
And retrievers are a parent class or super class of goldens.
Likewise, dogs are a parent class of retrievers.
So now, I've defined my terminology and also hopefully given you a very, very, very quick review of inheritance.
Now, I'm going to talk about the specifics in Python.
If I turn over here, I've written up a very short class definition for dog, right?
Every dog has the class attribute, cry.
Every dog has an initialization method that gives every dog a very specific name that is passed in when you initialize the dog.
And every dog has access to the class method, greeting, which returns a string that says, "I'm," whatever the name of the dog is, and also the specific cry, which in this case, is actually the class cry.
If you're unfamiliar with using the plus in terms of strings, it's just a concatenator.
So play around with that in IDLE if you're confused.
I would recommend copying all of this into IDLE, and then playing around with a particular instantiation of dogs, in this case, Lassie.
If you look at Lassie.name, you'll end up going after self.name, which is specified when your initialize the object.
So Lassie's name is Lassie.
Likewise, if you were to type in Lassie.greeting, open paren, close paren, and hit Enter, you should get a string return that says, "I'm Lassie," comma, "bark."
Mostly this is to familiarize you with object oriented in Python in the general sense.
Now, we're going to look at what happens when you want to set up a subclass.
If I set up class Retriever and I want to inherit from the super-class, Dog, I'm going to pass in Dog.
This is in the same syntax that I would use if it were a function and I wanted to pass in a parameter.
If I wanted to inherit from multiple things or multiple classes, I would put multiple classes here.
Right now, we're just going to inherit from Dog.
Note that I have no code here.
This is pretty much meant to explicitly specify the fact that Retriever is not actually going to introduce any new properties to dogs.
Their types are going to be different.
So if I create something that's a Retriever, it will be of object type Retriever, versus if I create something and say, "Dog," open paren, close paren, it's going to be of type Dog.
But what happens when I create a Retriever-- and as an aside, if you know who Benji is, I know he's not a retriever.
But bear with me here.
If I create a Retriever, it's first going to look for an initialization in any other methods or attributes in the Retriever class definition, run any code that's here, and then go to the parent class, and run all the code here.
So even though Retriever did not have any explicit code underneath it, I can still interact with the object, Benji, the same way that I interacted with the object Lassie.
It has all the same methods and all the same attributes.
Phew.
So there's basic inheritance.
And I will make another aside that if you're doing this, you probably don't need to create a subclass in the first place.
If you're designing your own code, and you're trying to think about what the best way to organize things is, if you have to create a subtype or a subclass and there are no new methods or attributes or no different ways of addressing those methods or attributes, then this category is probably actually just this category.
You may want to make a difference so that you can do interesting things with type checking.
I think that's the only thing I can think of that would justify it.
And I might be wrong.
Python gurus out there should correct me.
But a thing to keep in mind.
So we've done the first half of our inheritance.
We're going to inherit one more time and create a class of golden retrievers.
Once again, I've got my class definition and my indication that I'm going to inherit from Retriever.
I don't have any initialization or attribute assignments.
I only have a definition for greeting.
So what happens here?
Well, the first thing we always do is look for an initialization method.
Golden doesn't have one, so it's going to check the Retriever class.
Retriever doesn't have one, so it's going to check the Dog class.
The initialization method is here.
So when it runs the initialization method, it's going to run this code.
The first thing that's going to happen is any code, or any attribute assignments, or method definitions here are going to be considered the canon, or the first thing that any Golden is going to reference.
So greeting is going to be executed before greeting used in any other place.
You notice the only difference between this greeting and the Dog greeting is that "OHAI!"
has been prepended to the phrase.
And the way that we end up doing that is we refer to-- we concatenate, and then refer to the superclass.
And once again, we have to pass in the explicit argument, self, when we're talking about a class definition.
Later, when you actually instantiate an object and use your parens, you're not going to have to put self as an argument.
It'll get confused.
We'll go over that in a second.
So let's say I create a golden retriever, Sidney.
I'm going to pass in one argument, which is the name.
We're going to consider all the definitions here first, which means that goldens are going to have a method for greeting that is specified here.
It's going to use the method for greeting from Retriever.
And we could put in anything here, right?
We could put Dog.greeting.
We could put in some other function that is in the same environment as class Golden.
But here, we can explicitly access the superclass that we defined here.
We're going to head over to Retriever to see if there any additional methods or attributes that are a consequence of being a subclass of Retriever that we need to add to our definition.
Now, we just hit the pass.
On the other hand, Retriever inherits from Dog.
So once again, we have to jump over to a super-class and grab any attributes or methods that are defined there as well.
So all the way back over to Sidney.
When I call Sidney.greeting(), the first thing that happens is that I look in the most specific subclass, or whatever my object type is and see if there's a definition for the method.
Because there is, I'm not going to use Dog.greeting().
I'm going to use Golden.greeting().
Golden.greeting() says return a string that says, "OHAI!"
And also append it to whatever Retriever.greeting() returns.
I go over to Retriever.
It's not here, but I still have a reference to Dog.
I go over to Dog.
It has a method for greeting.
And it says, "I'm Sidney.
Bark."
So the final return type should be, "OHAI.
I'm Sidney.
Bark."
This concludes my basic overview of inheritance of object-oriented programming in Python for 6.01.
Next time, I'll review some interesting features in Python that actually originated in earlier languages and also particularly things in aliasing that people that are new to Python or people that are new to programming find especially confusing.
Hi.
Today I'd like to talk to you about some notable aspects of Python that you'll encounter in the 6.01 software, as well as in the general sense when working with Python.
A lot of these little tidbits of Python have some interesting history associated with them, especially related to the history of computer science.
And also, I want to indicate some things that tend to mess up, especially, first time programmers, but also especially people that are new to Python.
First, I'd like to bring us back to last week.
We were talking about object-oriented programming and inheritance.
Object-oriented programming is a programming paradigm.
It's in the same category as imperative programming or functional programming.
And you might say I have a good sense of the fact that in object-oriented programming, everything is an object.
But I don't really have a good sense of what constitutes imperative programming or functional programming.
I'm going to focus on functional programming right now, because it has more of a historic root in the academic development of computer science.
Functional programming is like object-oriented programming in that everything is a function.
It refers to the idea that you want to write as much of your code as possible in a purely functional manner.
And in particular, you're looking for the ability to avoid things like side-effects or enable many different kinds of evaluation.
Those kinds of things are not the subject of this course, but I think they're worth noting to figure out why we're bothering learning about things like lambdas and list comprehensions in the first place.
Speaking of which, the first thing we need to talk about before we even talk about things like lambdas and list comprehensions, is the concept that in Python, everything is an object.
But because functions are first-class objects, we can also treat Python to a large extent like a functional programming language.
What do I mean when I say functions are first class objects?
I mean that a function can be the return value of another function.
I mean that the function can be passed in as an argument to a function.
And I mean that functions can be assigned variable names and manipulated the same way that we manipulate any other piece of data structure.
This is important, because if you want higher order functions or functions that actually modify or use other functions as a part of you know, whatever it is they do, then you need to be able to interact with the function like it's any other object, in part pass it in or return it out.
Let's look at an example.
So let's say I start off with a very basic function, right?
If you want to square some sort of value, probably numeric in this case, then all you have to do is multiply it by itself.
Pretty simple.
Good place to start.
As a consequence of the idea that functions are first class objects, I can write a function that takes in a function as an argument and then develops a return function that uses the function that I passed in, on itself.
So any arguments that I would pass to someFunction, I would pass into someFunction, take the return value of this function call, and then pass that into someFunction again.
That's what returnFunction does.
And down here, I return returnFunction.
Note that the object type of this return value is a function.
So once I have returnFunction, I can actually pass it arguments, have them run through someFunction, not once but twice, and then get out a value that's of the return type of someFunction.
Note that I made some assumptions while writing this function.
First, I've assumed that someFunction returns out the same number of arguments, and either accept an arbitrary number of arguments or accepts the same number of arguments as it puts out.
The other assumption that I've made is that the data type that someFunction returns is the same as the data types that someFunction accepts.
Or the things that someFunction does to its arguments can be done to multiple kinds of arguments.
But that's a whole different argument.
Right now, we're just focusing on the fact that we pass in some arbitrary number of arguments, call someFunction on it, call someFunction again on the return value of this, and return that as something you can do to whatever it is, someFunction operates on.
OK. Let's review an example, because I promise it'll be more clear.
Let's say f is going to be the composition, a square on itself.
If I do that, I end up with a function that operates on an arbitrary number of arguments-- that's not true.
I end up with a copy of square that takes in the same number of arguments as square called on itself.
So here a square is substituted for someFunction.
Here a square is called on that call of square.
And here is what f is assigned to.
So when I call f of 2, I'm going to substitute 2 in for args.
I'm going to call square on args.
If I call square on args, I get out 4.
And if I call square on args again, or if I call square of args where args is defined as the return value of this, then I'm going to call square on 4, I'll square 4, and I'll get 16.
Take a second to type it into IDLE, and make it make sense to yourself.
All of this code should compile.
Mess around with the parameters, if you're having trouble convincing yourself that it does.
Ok. Now I think we're ready to talk about lambdas.
If you can pass in functions as arguments or return them as values or assign them to variable names and just treat them like any other data type, then you should be able to treat them as some sort of raw value.
And that's where lambdas come in.
Lambda is a key word in Python that tells you you're about to use a function that you have not given any sort of name or defined in any place in your code or environment beforehand.
You're just going to start talking about something that you would like to do to a given number of arguments, and then what you want to return out.
Lambda has roots in lambda calculus which, if you're familiar with the history of computer science, you've probably heard of.
Now might be a good time to look up lambda calculus, if you've never heard it before or possibly Alonzo Church.
But it's still available in code today, which is really cool, and speaks to the continuing power or at least recognition of importance of functional programming.
As I said before, the idea with lambda is that you could write an anonymous function.
Over here, in order to write a function that's squared, I had to write out a define line.
And because Python uses indentation, because in Python, indentation carries meaning, I'd have to enter and return over to a next line-- I'm actually not sure if that's strictly true.
I think you can do def square x, return x, but I'm not positive.
I've only seen it written like this.
And I think it's because in the general sense, people try to respect things like convention and readability in Python.
If you're using lambda, people already know that you want to describe a function really quickly.
Or you want to describe function without assigning it any name.
Therefore, the two most common uses you'll see of lambda is when you want a really fast function and you don't want to spend extra time or lines writing out that function.
It's pretty clear what this is going to do.
Or if, in a particular sense, you need an anonymous function.
You don't want to assign a name or memory space associated with that function.
Note instead of using square, I can just write out this.
So I saved two lines of code defining square.
And then I don't have to refer back to some earlier part of the code in order to understand what this line does.
Pretty cool.
All right.
So there's functions of first class objects.
There's lambda.
Let's talk about how to use lambdas on lists.
In Python, you'll end up doing a lot of list manipulation.
One of the best uses of anonymized functions in any functional programming language is the ability to manipulate lists in a line.
If I've defined just a pretty straightforward list right here, I can use functions like map filter and reduce to-- in one line-- take a list, apply a function to every element in that list, and then return a copy of the result of that list.
I didn't have to write a separate function to do the list handling.
I didn't have to write a separate function to multiply the list by two.
And I've communicated what I'm doing effectively to people that are used to functional programming.
So if you type this line in, you should end up with a return in interactive mode of every element in this list multiplied by two.
Now's a good time to try.
Once you've done that, I also recommend looking at the original copied demolist.
Note that this is unaltered.
Map actually returned a new data structure that represents performing this function on this list.
This becomes important later when I explain aliasing, but I will do that in about two minutes.
If we want to get even more elegant, we can use list comprehensions.
List comprehensions-- if you ask somebody where list comprehensions are from, they'll probably say Haskell, because that is the most popular use of list comprehensions, in terms of pure lazy functional programming.
But it actually goes back further than that, and is in Small Talk, and then something from the '60s.
I can't remember the name of it right now.
They're really nice because they borrow a syntactic approach from mathematicians.
This looks a lot like set notation.
And when you read this, you can probably tell that the list listComprehension is going to describe a set of points from 1 to 4.
But you accomplish that in possibly one line of code.
I've written it on three here because I wanted to keep it in this space.
But it's really concise.
If you type this into IDLE, you should see the kind of list that it returns, and also what's listComprehension is now-- what now constitutes listComprehension.
You can use listComprehensions with functions like map and filter and reduce.
And, along with the anonymized functions, all of these tools provide you with a lot of functionality in a very short amount of space.
It's also good to be able to recognize what's going on when you see something like this or something like this.
Because you'll probably run into it in, in particular, a lot with Lisp code, but also in the artificial intelligence community in particular.
Ok. We've talked about all that.
Let's take a second to talk about the fact that lists are actually mutable, and what that means, and what things you have to be careful about if you're going to be working with mutable objects.
If you're new to programming, or you're new to Python, you've probably already worked with some of these data types, right?
Any numbers, any strings, any tuples, are going to be immutable.
What that means is if you have a variable and you have a second variable that has a different assignment line associated with it-- right?
I haven't assigned g to h here.
I've just signed g to "hello" and h to "hello."
If you look at the space in memory that Python associates with both objects, they point to the same place.
This is the definition of an immutable object.
When g points to "hello" and h points to "hello," they both point to the same place.
If x points to 5 and y points to 5, they're both pointing to the same place.
If you point x at 6, it now points to a different memory address.
This gets confounded when you're talking about mutable objects.
The problem with mutable objects is that you select a memory space to contain the object or memory ID in Python to contain the object.
And then that object is changed in place.
You can use this to your advantage but it can also mess with you.
And here's how.
Let's say I assigned a to a small list.
And I also assigned b to a. b and a now point at the same place.
If I manipulate b, and I'm-- excuse me-- still working with an immutable object, or excuse me, if I'm still working with a mutable object, the object in that memory address has been altered.
And b and a still point to the same place in memory.
So if I look at a, it's going to look like b, which is going to look like 1, 2, 3, 4 -- which is not what I assigned a to originally.
Again, can be powerful, but you have to keep it in mind.
And it might start to mess with, in particular, the 6.01 software when you're dealing with state machines.
In order to get around this, you can create a copy of the list and then modify the copy.
This is actually what map does.
If you want to do it, the easiest way to do it on the first layer is to specify the index into the list with no bounce.
It'll copy the whole thing.
There's also a Python library copy or deep copy, if you want to copy lists of lists of lists of lists of lists.
And just to clarify, you'll note that after you assign c to a copy of a, c and a occupy different memory places.
So if you modify c, a will retain its original value.
I think that's all the notable things I have to say about Python.
and gives us enough power to do some really powerful list manipulation or array manipulation, and covers what we want to say about functional programming before we get into the notion of state, which I'll talk about next time.
Hi.
Today I'd like to talk to you about state machines.
State machines are an incredibly important concept, both in 6.01, and in the general sense when you're going to be doing things like control theory or artificial intelligence.
Any further work you do in terms of computability theory, state machines are really important.
So I'm going to review what we've done so far and talk about why we now need state machines to add additional complexity to the kinds of models that we're going to build, and then talk a little bit about how state machines are represented in different domains, and then also talk about how we're going to use state machines in the 6.01 software.
First of all, let's review what we've learned so far.
We've talked, so far, about functional, imperative and object-oriented programming paradigms.
In functional programming, everything is a function.
And in imperative programming, we're allowed to use functions, but they're also allowed to have side effects.
And in object-oriented programming, everything is an object.
We can actually use the first two to implement the last one, and use the first one plus an idea of maintaining assignments to variables, that sort of thing, to implement this one.
So you can trace the progression of different kinds of computer science language along this paradigm.
But the one thing that none of these languages on their own allows us to do is keep a notion of internal state.
And what I mean by that is that if we have a system that we want to model in terms of the passage of time or keep track of the evolution of that system or keep track of some of the data that has accumulated over time in that system, then we certainly can't do it with functional programming.
Functional programming takes one input, generates one output, and you could generate a list of codes that took in every possible situation and then generated the logical output, but that would be a lot of code.
Same thing for imperative and object-oriented programming.
They alone do not address everything that we want to address, which is the ability to look at everything that has happened as a consequence of the passage of time and all of the data that we've looked at as a consequence of the passage of time, synthesize it in some way, and then generate whatever was supposed to come out of that situation.
That is the notion of internal state.
State machines have been around for a while.
You might see them refer to as discrete finite automata.
There are also such things as continuous state machines, but we're not going to talk about those in this course so much.
So we're only talking about discrete state machines.
And when you see them referred to as state machines or discrete finite automata in literature, especially mathematics, you'll be looking for this five set of things.
One is a set of the possible states you could be in, given a particular state machine.
One is the set of inputs you could possibly encounter while in that state machine.
One is the set of outputs that that state machine could possibly generate.
One is a transition function that looks at pairs of these values and tells you, based on which state you're currently in, what state you will end up in as a consequence of the current input, and then also specify what the output will be as a consequence of that transition.
And finally, it'll tell you where you start out.
That's a lot to absorb.
I'm going to show you a state transition diagram, which many of you have probably seen before, and map these values to this state transition diagram in hopes of making this a little bit more concrete.
Hopefully at this point, all of you who have interacted with an MBTA turnstile.
This is a thing that opens and closes, and you stick your RFID card on it, and then Richard Stallman yells at you and possibly hands you more Charlie tickets or something like that.
It has four states.
One of them is that this turnstile could be closed, and it's waiting for people to interact with it in some way.
It could be open, and I anthropomorphize the turnstile as being happy as a consequence of you putting money in it.
It could be open and quiet as a consequence of usually some sort of other previous interaction with another person.
Or it could be open and angry as a consequence of people interacting with it when they should not.
The vertices in this directed graph are my set of states.
So when you see something like this, and you're asked to map it to the mathematical construct, just grab the names and say this is my set of states.
Let's say I start off in closed.
And actually, it occurs to me that the other thing that should be specified is a start state.
And it's not here.
So let's say the turnstile starts off as closed.
Usually this is an arrow coming out of nowhere that directs into one of the states.
Sometimes it will be explicitly indicated by saying start state, but typically you'll just see an arrow from nowhere.
At this point, I don't have any inputs or outputs.
If I put money in the turnstile, it constitutes input to my system.
It's going to end up in my set of inputs for math.
My transition function looks at the state I'm in and looks at the current input and generates the output and the next state.
So all of these arrows, in addition to whatever information is contained in the receiving vertex, specifies my transition function.
Any transition that's not specified is not considered in the function.
This was not part of our original drawing.
So if I was fed open angry, there would be no way to get to open happy.
Likewise, if I was in open angry and fed money for this simple system, we're going to say nothing would happen.
I think at this point it's become clear how to transform a state transition diagram into the mathematical constructs.
It's good to have the mathematical constructs, because they end up being used in software, which I'll talk about in a second.
But the first thing I'm going to do is just walk around the state transition diagram and indicate what would be inputs, what would be outputs, that sort of thing.
So let's say I walk up to the turnstile and somebody else interacts with the turnstile by exiting.
In this case, exit is the input, none is the output, the turnstile doesn't make any noise, and the turnstile is open.
If at that point I interact with the turnstile by entering, it's going to make noise, which is the output.
And then the new state is going to be that the turnstile is open and angry.
At that point, you and I know that the turnstile is going to close.
So this edge indicates that the only available input at that point is to do nothing.
Or independent of anything else, it's going to squawk again and close.
One more time, here are the states.
Inputs are the first of these two pairs.
Outputs, as a consequence of the transition, is the second of these two pairs.
And the transition function is represented by the directed edge and the new state.
Once you have all those sets figured out, you can start talking about how to implement state machines and software.
We've actually abstracted this away from you.
You don't have to deal with it.
But as a consequence, you should know how to interact with the 6.01 library.
Let's look at an example of the state machine class.
I want to build an accumulator, which at every time step we'll look at the input, add it to every other example of input and output, and output the new value and retain it as the new state.
The first thing I need to do is initialize the accumulator with a value.
This is our start state, which is the same as our start state from the MBTA turnstile, and also the same as our start state from the mathematical construct.
I also want something called getNext Values which is the functional equivalent of the transitions.
Here's our self again from object oriented programming -- but we don't care about that.
We're going to look at the current state and the current input.
Some getNext Values we'll do some internal data munging, possibly multiplication by two or comparing to the previous input and then having some conditionals, that sort of thing.
But there'll be some sort of very simple function under getNext Values at least for the first couple of weeks.
And then we will return the new state and the output into tuple.
In this case, the new state is going to be the linear combination of the current state and the current input.
And the output is going to be the linear combination of the current state and the current input.
If I were to draw this accumulator as a state transition diagram, I would do this.
My start state is the initial value.
If I pass in a new input-- we'll call it input 0 -- both my output and the new state are going to be the linear combination of these two values.
If I made another transition, I would take whatever my next input was and add it to the current state value and return it out as the output, and so on and so forth.
I encourage you try this in Python, or in IDLE and munge around with it and see what you can get it to do.
You might have to type add lib6.01 in order to get the state machine class, or initialize using lib6.01 in order to get the state machine class.
But otherwise, those should be enough to get you started with an introduction to state machines.
If you're having trouble, I highly recommend going through all the examples in the readings.
They're pretty comprehensive, and it also includes the accumulator.
That's all for now.
Next time we'll talk about linear time varying systems.
Hi.
Today I'd like to talk to about a new module called Signals and Systems.
Our previous module on programming, we focused on object-oriented programming and state machines as different methods by which we could model the physical world.
In this module we're going to talk about a new way to model the physical world called the discrete linear time-invariant system.
You might say, Kendra, why do I need to do that, or why would I move away from state machines.
I seem to be able to model everything using state machines.
The short answer is, you want to be able to move away from state machines, because you want to be able to predict the future.
But I'm not really going to be able to get to how you're going to be able to predict the future until two videos from now.
So for now we'll introduce discrete linear time-invariant systems and also talk about different knowledge representations you might encounter them in, so that you recognize them when you see them and can manipulate them as needed to talk to other people about them.
Let's go to the board.
Last time we talked about state machines.
We're able to use them to model pretty much any system.
We can model the evolution of a particular process over time using a record of all previous inputs and outputs and possibly previous states, if we included that in our output.
But the problem with state machines is that they, like most programming, encounter the halting problem.
At some point your program can reach a little complexity at which you cannot determine what it's going to do without actually just running it and seeing what happens.
This is great for researchers.
This is the method by which people go out and have to find something or discover something instead of being able to like, simulate it using a machine.
But if you want to be able to promise things to other people or predict what's going to happen, then it helps if you can model something that has a high level of complexity by abstracting away the complexity and interacting with it as though it has features that indicate that it's going to behave in a particular way.
So if you want to make a control system that does a particular thing, or look at an existing system and say, well, I know that your power plant's going to blow up in five years, because of this, it helps if you have some understanding of particular classes of systems and what kind of long term behaviors those particular classes of systems generate.
And that's why we look at discrete LTI.
It's a particular class of system -- in particular discrete LTI is what we're going to look at in 6.01 because it's fairly simple.
But once you've learned how to deal with the system in that manner, you can use the skills that you've learned there to approach more complex feedback problems or more complex control problems.
First, an incredibly brief review of linear time-invariant systems.
When you're talking about a linear time-invariant system, both the inputs and the outputs are going to be real.
We're not going to deal with the complex plane at all.
If you were to model your LTI system using a state machine, that state machine would be dependent on a fixed amount of previous inputs, outputs or states.
You give some amount of leeway for start state or the fixed number of states before you get to your fixed amount of data structure that represents your current state.
But if you're running a linear time-invariant system, the amount of information that you need to figure out your state in a long term sense is always finite and fixed.
In terms of functional expressions, you've probably seen these associated with linear time-invariant systems.
Know that this also means that you can represent linear time-invariant systems as additions or scalar multiplications of your existing function.
So that's out of the way.
These are all true of LTI.
We're going to focus on discrete LTI.
One, because we can use a discrete state machine to model it.
And two, because it makes it easier to represent using computers.
Once you've got that digital abstraction, you don't have to worry about having access to a truly continuous function.
All right.
Now that you know what we're going to talk about, how do you talk about it to other people?
Here are the different representations that you might see a discrete linear time-invariant system in.
One of them is called a difference equation.
And you've probably encountered this in one of your math classes.
It says that you're going to take a sample from a signal y at some sort of given time step, typically n. And when you're writing out a difference equation that represents an entire system, you usually see y of n on one side on the left, and then everything that represents the functional component on the system on the right, which in this case determines what determines the output at every time step.
In this very particular case, we're talking about an accumulator, which means that at every time step the output is determined by the input plus the previous output.
So at every time step we take the input and whatever it is we were outputting before and output it again.
Pretty straightforward.
If somebody described to you a difference equation in words, you could probably turn it into a equation at this point.
Note that even though there are variables associated with these samples, they are still very particular samples.
And the thing that you're interested in is probably not even the samples at all.
It's the relationship between the samples, when they were sampled, and relative to one another, and how that relates to the output at a given time step.
If you then want to talk about an entire signal, then you can use an operator equation.
And you'll probably see these things when you do any sort of research on control theory or feedback.
Instead of representing the sample of the signal y at a given time n, we're just going to talk about the overall signal capital Y.
Likewise, instead of talking about x at a given time n, we're going to talk about the signal capital X.
Remember that I said that the most important part of this equation is the fact that there's a different relationship between these two signals and when you were sampling from them.
When we want to represent this relative delay, we represent it using an R. And in particular, I want to note the fact that the degree of R represents the amount of delay associated with the sample from a particular signal.
So if I wanted to make this n minus 2, I would reflect that change in my operator equation by changing the degree of R. And because we're working with a linear time-invariant system, if we wanted to scale this by 2, it'd be the same as scaling this by 2, et cetera.
Now I can talk about signals.
What if I want to talk about a physical manifestation of the relationship between these signals, and/or what if I want to use something kind of like circuit diagrams to talk about what kind of signal manipulation I'm going to do?
This is where block diagrams come in.
Block diagrams are incredibly PowerPoint-friendly.
Block diagrams mean that everybody is on the same page, because all you have to do is trace out the arrows.
Block diagrams mean you're going to do a combination of gains, delays and adders to visually represent the relationship between your output signal and any input signals that you have.
Up here I have actually drawn an accumulator.
And it's got a box around it, which will be relevant in two minutes.
But right now you can ignore it.
I've got my input signal, or my input signals, are typically on the left.
And the progression of gains, delays and adders associated with the block diagram represent the relationship between the input signals and the output signal.
And then I'll have my output signal on the right.
Most people indicate flow with arrows, in part to make things easier to read.
But typically, you only need the arrow indicating where you're sampling from and what your output's going to look like.
In this case in order to get Y, and in the general sense, I can backtrace from the signal that I'm interested in through my diagram and figure out what values I'm actually interested in.
So in this particular case, Y is a linear combination of X and whatever this represents, which is RY.
If I did want to put a 2 here, I've been talking about gains, delays and adders, but this diagram doesn't contain a gain yet.
So let me show you how you would include a gain.
Gains are typically represented by the value of the gain inside an arrow.
It looks like an op-amp from schematic diagrams.
We'll get to those later in circuits.
It's not essential that the arrow point in the direction of flow, but people might look at you funny if you don't put it in the direction of flow.
And the other thing that I think I want to note here is that if you see a minus sign here, it means X minus 2RY, which is the same as putting a negative value on your gain.
So right now, we're back to X plus 2RY.
OK. We've got to block diagrams.
You might be saying at this point, Kendra, what are block diagrams good for other than PowerPoint presentations and possibly arguing with my friends over what's going on.
And I say, they're really good for abstraction.
I could draw a box around this and abstract it away into a function and say, if I put something into this box, and I want to get something out where the action that happens in here is identical to this schematic, then I can find an equation that actually represents that operation.
And the way I do that is I'm going to take my operator equation and solve in particular for Y over X.
In this case, with the 2, Y over X is going to be 1 over (1 minus 2R).
Note that if H is equal to the expression Y over X, and I multiply H by X, I get out Y.
This concludes my coverage of motivations for learning about discrete LTI systems and also the different representations that we will want to use when talking to other people about discrete LTI.
At this point we can also start talking about how we get to the point where we can start predicting the future.
And then in the video after that I'll actually start talking about poles.
Hi.
Today I'd like to talk about signals and systems again.
At this point, you're probably familiar with the motivation for why we're talking about discrete linear time and variant systems, and also with a few of the representations that we're going to end up using in this course.
But you're still not sure what it is that we're trying to accomplish.
Or where's the part where we get to predict the future based on the fact that we are capable of manipulating these systems?
Well, we actually have to be capable of manipulating these systems.
And at this point, we can describe this system as we see it, but we can't also manipulate its representation in ways that make sense to us.
So the thing that I'm going to do today is talk about different system equivalences and how to take a system and solve for an expression that represents a complex system and also how if you know that some things in your system are equivalent, how you can convert between them.
At that point, we should be able to talk about poles, which is how we're going to actually predict the future.
So different equivalences that I'd like to talk about.
I'm first going to briefly review the facts that last time we discovered the notion of system function, right?
We can take a representation of a system and abstract it away into some sort of function, where we take the input as it's given to us and then multiply it by this function and then get the output that we're interested in.
How do we deal with something more complex?
I mean, y is all the way over here.
And we've got multiple system functions.
And I don't even know what happens here, but it doesn't have to be that scary.
Let's break it down.
One of the easiest ways to approach something like this is to identify each position where you have a new signal, or if you were to sample here, you would have a new signal, and label those values appropriately.
You can then start with your final output and then back solve for the values that you're interested in as a consequence of that final output.
In this particular example, y is going to be y2 plus y3.
y2 is going to be y1 times H2, where H2 is some system function.
And it probably is abstracting away some combination of gains, delays, and adders like this one here.
y1 is going to be x times H1.
And Y3 is going to be x times H3.
Now I've got all my expressions in terms of either y or something for which I have an equivalent expression for x.
So I can do my substitutions, come up for an expression for y over x, in terms of H1, H2, and H3.
Here I've just made the substitutions of the equations above and factored out the x.
If I wanted the system function, I would then just divide by x.
And then I would have y over x is equal to this expression.
The thing I wanted to indicate is that if I wanted to abstract this away into its own box-- maybe I wanted like a big H or an H0 or something like that-- and it represented what was happening in this top line, cascading two system functions is the functional equivalent of multiplying them together.
So if I have an expression for H1 and I have an expression for H2, and I want the expression that is equal to cascading H1 and H2, I just multiply them together.
Likewise, if I want an expression for the linear combination of two system functions applied to an input individually, like the combination H1 and H2, and H3, it's a summation of those two values which is expressed here.
This is the same as the relationship that we reviewed in a very basic sense when we were originally doing the accumulator.
The only thing I'm attempting to indicate is that, that relationship scales to an arbitrary level of complexity.
So if you need to, you could shift around these values, if you can find some sort of equivalence.
Let's see what happens when H2 is equal to H3.
I'm going to take my operator equation.
What this means is that if I wanted to rewrite this block diagram, I could do so by doing-- This is really similar to bubble pushing.
If you've done 6.004 or 6.002 and have experience with logic gates, I just wanted to indicate that it's also a thing that you can do for block diagrams and system functions.
There's one more type of equivalence that I want to talk about.
I call it feedback equivalence.
Here's our normal accumulator rate.
If I wanted to represent this feedback system as a feed forward system, what would I have to do?
Well, the first time that I sampled from x, it would just be y.
So right now this diagram matches for the first time step.
On the second time step, if I had an input from x from the previous time step, I would also want to account for it by putting in a delay and then summing it with the current value of x in order to get y.
At the second time step, I would want access to the starting value, the value from the previous time step, and the value from the current time step.
And one more time, to exhaust the example, at the third time step, my output would be a linear combination of the starting value, the value from the first time step, the value from the second time step, and the value from the current third time step.
We'd end up doing this ad nauseum to model our feedback system.
So it's difficult to do on paper, but it turns out there's a great relationship between these two equivalences and things that we already know from-- I want to say high school calculus or possibly 18.01, 18.02.
Geometric sequences.
When we solved for the system function, we found an expression for our feedback system.
If I wanted to find an equivalent expression using this feed forward system, I would look at this infinite summation of x terms.
So if I wanted to know something about the long-term behavior of the system, in terms of this system function, I would solve for this expression and then using my knowledge of geometric sequences, in order to express the long-term behavior.
In the general sense, in this course, we're going to be looking at the unit sample response of a system.
What that means is, if the only thing I ever do for input is a single value of 1 at time 0, then what does my output look like?
The reason we're looking at the unit sample response is because it's (a) the simplest way to look at the long-term behavior of a discrete linear time invariant system.
But the other reason (b) is -- once we have this, we can also use it to do things like to make predictions about the long-term step response and other more complicated input signals.
In the case of the accumulator, if I input 1 at time 0, my output is going to be 1 forever more.
That's reflected in the coefficient of my geometric sequence.
If I want to know what my long-term response is going to look like, I can look at the coefficient of R and make a decision about whether or not I'm going to diverge or converge or do neither.
So if I put a coefficient on R, whatever p0 converges to is what my system is going to converge to.
So using my knowledge of p0, I can make long-term predictions about the behavior of the system.
Next time I'm going to go over some general classifications of those behaviors for the system and how to more effectively use our knowledge of p0 and how to deal with things like second order systems.
Today I'd like to talk to about poles.
Last time I ended up talking to you about LTI representations and manipulations.
And, in particular, I want to emphasize the relationship between feedforward and feedback systems, in order to segway us into the poles..
Using what we know about that relationship, we can find the base of a geometric sequence.
And that geometric sequence actually represents the long-term response of our system to a unit sample response or a delta.
That value is also what we refer to the poles.
At this point, I'll go over how to solve for them and very basic properties of them.
So that we can use the information that we have about the poles to try to actually predict the future or look at the long-term behavior of our system.
First a quick review.
Last time we talked about feedforward systems.
And, in particular, I want to emphasize the fact that if you have a transient input to your feedforward system, you're going to end up with a transient response.
There's no method by which a feedforward system can retain information over more than the amount of time steps that you feed information into it.
Feedback systems, on the other hand, represent a persistent response to a transient input.
Because you're working with a feedback system information that you put in can be reflected in more than one time step and possibly multiple time steps, depending upon how many delays you have working in your feedback system.
Last time I also drew out the relationship between feedforward and feedback systems.
You can actually turn a feedback system or talk about a feedback system in terms of a feedforward system that takes infinite samples of the input and feeds them through a summation that takes infinite delays through your system.
You can represent that translation using a geometric sequence.
The basis of that geometric sequence is the object that we're going to use in order to predict the future.
And that's what we're talking about when we talk about poles.
You can have multiple geometric sequences involved in actually determining the long-term behaviors of your system.
If you only have one, then things are pretty simple.
You find your system function.
You find the value associated with p0 in this expression.
It's OK if there's some sort of scalar on the outside of this expression.
We're working with linear time and variance systems, so that scalar is going to affect the initial response to your system.
But in terms of a long term behavior, it doesn't matter as much.
So don't worry about it right now.
Relatedly, if you're solving for these expressions in second or higher order systems, you're going to end up having to solve partial fractions.
You can do this.
And in part, one of the reasons that you would want to do this is so that you can get out those scalars, if you're going to be talking about the very short-term response to something, like a transient input.
We're not going to be too interested in those in this course.
We're mostly going to be talking about long-term response.
So we can get around the fact the we're dealing with a higher order of systems and not solving partial fractions by substituting in for an expression called z, which actually represents the inverse power of R and then solving for the roots of that equation.
If you substitute z in for 1 over R in this denominator and then solve for the root associated with that expression, you'll get the same result.
You'll actually end up out with p0.
All right.
So now we know how to find the pole or multiple poles, if we're interested in multiple poles.
What do we do now?
I still haven't gone over how to figure out the long-term behavior of your system.
The first thing you do is look at the magnitude of all the poles that you've solved for and select the poles with the largest magnitude.
If there are multiple poles with the same magnitude, then you'll end up looking at all of them.
If you have different properties than the ones here, you can end up with some, you know, complex behavior.
I would not worry about that too much.
Or I would ask a professor or TA when that happens.
But in the general sense, if your dominant pole has a magnitude greater than 1, then you're going to see long-term divergence in your system.
This make sense if you think about it.
If at every time step your unit sample response is multiplied by a value that is greater than 1, then it's going to increase.
And, in fact, the extent to which the magnitude of your dominant pole is greater than 1 is going to determine your rate of increase and also determine how fast your envelope explodes.
Similarly, if the magnitude of your dominant pole is less than 1, then in response to a unit sample input or a delta, your system's going to converge.
This also makes sense intuitively.
If you are progressively multiplying the values in your system by a scalar that is less than 1, then eventually you're going to end up converging to 0.
To cover the only category we haven't talked about, if your dominant pole is actually equal in magnitude to 1, then you're not going to see convergence or divergence.
And this is one of the places in which the magnitude of the scalar that you end up multiplying your system by can become relevant.
We're not going to focus on this situation too much.
But it's good to know what actually happens when the magnitude of your dominant pole is equal to 1.
The other feature that we're interested in when we're looking at the dominant pole of a system is if we were to represent the dominant pole in this form, what the angle associated with that pole is, if you were to graph that pole on the complex plane using polar coordinates.
If your pole stays on the real axis, or if your pole does not have a complex component, then you'll see one of two things.
The first thing that it's possible for you to see is that you'll get absolutely non-alternating behavior.
Your system response stays on one side of the x-axis and either converges, diverges, or remains constant as a consequence of input of the unit sample.
And you won't see any sort of alternating or oscillating behavior.
This only happens when your dominant pole is real and positive.
If you're dominant pole is real and negative, this also means that it's still on the real axis, but its value is negative.
So if you're looking at polar coordinates, it's going to have an angle pi associated with it.
This means you get alternating behavior.
And what I mean when I say alternating behavior is that your unit sample response is going to jump across the x-axis at every time step.
This is also equivalent to having a period of 2.
The other situation you can run into is that this angle is neither 0 nor pi.
And at that point you're going to be talking about oscillatory behavior or a sinusoidal response that retains its edges at the envelope of your function.
In order to find the period, or in order to find the amount of time it takes for your unit sample response to complete one period, you're going to take the angle associated with your dominant pole and divide 2pi by it.
This is the general equation for a period.
This covers the basics of what you want to do once you already have your poles.
Next time I'm actually going to solve a pole problem and show you what the long-term response looks like and also talk about some things about poles that I've pretty much skimmed over.
And at that point you should be able to solve and look at poles for yourself.
Last time we introduced poles.
And in particular, we introduced how to move from the manipulation of feed-forward and feedback systems and the geometric sequence that fell out into using the base of that geometric sequence to attempt to predict the long-term behavior of the system.
When we're solving for poles and we're only interested in long term behavior, one of the easiest ways to do so to solve for the roots of z, where z is a substitution for 1 over R in the denominator of the system function.
Once we've done that, we have a list of poles.
From that list of poles, we would like to select, the dominant pole, or the pole with the greatest magnitude, and then based on the magnitude and period of that pole we can determine what the long term behavior of our system looks like.
Today I'd like to mention to you some notable things about poles.
If you are interested in this information or feedback and controls in the general sense, I highly recommend 6.003.
But here's some information you should at least be aware of as a consequence of 6.01.
The other thing I would like to do is just walk through a couple of pole problems to familiarize you, or get you more comfortable with the idea of solving for the poles of a system function, or looking at the unit sample response of a system function and then graphing the poles.
The first thing that I want to mention is pole-zero cancellation.
And what do I mean when I say that?
I mean that if both the numerator and the denominator have a degree of R in them, then you're going to have both a zero and a pole.
If the zero and the pole have the same value associated with them, you may be tempted to cancel them out.
Unless both the zero and the pole are equal to 0 -- don't do it.
The reason why is that when you get to a implementation of a real system, it is highly unlikely that both the zero and the pole will be implemented to a degree of accuracy that you will actually see those two things cancel out.
The only exception to this is when both the pole and the zero are equal to 0, or in this case.
This you should feel free to convert to this.
In almost any other situation, don't factor.
The other thing I want to talk about is repeated roots.
If you have a repeated root, you'll have repeated poles.
This does get tricky when you're talking about how to add the unit response of those poles.
But the long term behavior of your system is going to look the same.
So if both of these poles are the dominant pole, then the characteristics of both, which are the same, are going to determine what your long term behavior looks like.
If they're not, then the dominant pole is going to determine what your long term behavior looks like.
The last thing I want to mention is superposition.
So far we've only talked about the unit sample response of a system function and how we use poles to determine what the long term behavior of our system's going to be.
We can look at the response to more complicated inputs than the unit sample response, or the delta.
In fact, one of the things we' probably end up looking at some point is the step function.
The thing that you need to know to go from talking about unit sample response to any other sort of response, is that we're still working with an LTI system.
What that means is if you take the summation of your inputs, and apply the system function to that summation, it is the same as the output that would result from inputting all those values at once.
The best way I would like to explain it is by referring again to if your function was a system function, the same property applies.
Now let's walk through a pole problem.
Here I have a second order system set up.
We've got two degrees of R. I have feedback, and I can solve for an expression of y in terms of x.
In fact, let's do that right now.
y is the result of the summation or a linear combination of x plus a delayed signal of y scaled by 1.6 in a linear combination with a delayed value of the delayed value of y scaled by negative 0.63.
There's my first degree.
For consistency's sake, there's my second degree.
Let's first solve for the system function.
If you're confused, I recommend doing the algebra from here to this expression.
You should get this fraction out.
Our second step is to solve for the roots of z.
Remember that z is equal to 1 over R in the denominator of the system function.
In this case, we'll be working with-- all I've done there is taken every degree of R, substitute it in for 1/z.
and then multiply it out, so that I'm not working with z in the denominator anymore.
I'm actually just working with everything in the numerator.
If I follow this back out, I get this expression.
And my poles are going to be 0.7 and 0.9.
All right, based on my poles, what are the properties of the unit sample response in the long term?
First thing I'm going to do is look for the dominant pole among the poles that I found.
In this case, I don't even have to worry about finding the length of the distance from the origin for poles in the complex plane.
All I have to worry about is the magnitude of poles on the real axis.
0.9 is my dominant pole, because it's the largest pole.
0.9 is less than 1.
So I'm going to end up with convergence.
Eventually, my system is going to converge, or tend towards 0.
The other interesting property of my system is what is its period, how does that relate to what my function's going to look like.
In this case, we're only working on the positive real axis, so the angle associated with graphing this pole on the complex plane is 0, so there is no period for our system.
This means that our system is going to converge monotonically.
Now let's walk through some unit sample responses and then graph the poles that generated those unit sample responses on the unit circle, where this is the complex plane.
Let's look at this graph first.
The first thing that I notice about this graph is that, like in the previous example, we have monotonic convergence.
We're tending towards 0, and we're not alternating or oscillating about the x-axis.
So I know I'm going to be working somewhere along this line before the edge of the unit circle.
Because at the edge of the unit circle, the distance from the origin is equal to 1.
If you made me guess, then I would look at the distance here and compare it to the distance at the next time step.
I realize this is a blackboard.
It's not entirely to scale.
But for the purposes of this demonstration, I'd like to say that the signal at this time step is 0.5 the signal from the previous times.
Likewise at the next time step, I would like to say that this signal is 0.5 the signal from the previous time step, and so on and so forth.
Therefore, I'm going to graph my pole right here.
Let's take a look at this graph.
I've drawn these squiggles to indicate that the unit sample response exceeds the bounds of the space that I gave for this graph.
So just assume that these values are much larger than I've drawn them.
The first thing that I notice about this unit sample response graph is the fact that not only am I increasing in a way that does not seem to change in any way-- we're going to end up diverging-- is that I'm actually alternating about the x-axis.
And that particularly, that if I were to call this an oscillation, then I would say it's an oscillation with period 2.
This means that I'm working with a negative real pole.
The fact that I'm diverging means I'm working with a negative real pole that has magnitude greater than 1.
If you had to make me guess, I would look at the distance associated with this time step, compare it to the distance associated with this time step.
And if you had to ask me, I would say this is about 1.3 the value at the previous step.
Likewise, if I were to look at the next time step, I would say that this increase is about 30% of the previous value.
I'm not even going to try that one.
But what I'm trying to get at is that you can use comparisons of previous and future time steps in order to attempt to determine the magnitude of the pole if you're working with the first order system.
If you're working with the second order system, then it's possible that you'll see some really interesting initialization effects.
And you should probably ask one of us what's up.
But for this example, we're going to put our pole over here.
Here's the last graph I want to talk about.
The first thing that I notice is that it doesn't seem to be diverging, but it doesn't really seem to be converging either.
If this is the case, then I'm going to put it on the unit circle.
The second thing that I notice is that it's not monotonic and it's not alternating.
This is oscillating.
So in order to determine what angle I'm going to sign to my unit simple response, I'm going to count out the time steps that it takes to cycle through an entire period and then from there figure out what the angle would have to be in order to determine a period of that length.
So I start here.
I'm just going to count 1, 2, 3, 4, 5, 6, 7, 8 -- to complete one full oscillation.
This means that my period is 8.
If I have to divide 2pi by a particular angle in order to get out 8, I want to divide by pi/4.
So at this point, I'm working with a magnitude of about 1, and I want this angle to be about pi/4.
This concludes my tutorial on solving poles.
Next time, we'll end up talking about circuits.
Hi.
Today, I'd like to talk to you about circuits.
Last time, we finished up the LTIs, and signals, and systems, where we learned how to both model existing systems and predict their long-term behavior.
But we haven't forayed into how to actually create systems in the physical world.
We've created some amount of systems in software and made some brains for our robots.
But if we want to make something in the physical world, then we probably have to come up with ways to model physical systems or use physical components.
That starts our new model on circuits.
Circuits are going to be our first foray into designing systems in the physical world, also designing systems using physical components.
It's worth mentioning now that the information that you learn about circuits is good for more things than even circuits.
You can use basic circuit diagrams and properties of circuits to model all sorts of kinds of systems, especially ones in the human body-- circulatory system, neurological system, different kinds of fluid flow, that kind of thing.
In the next few videos, we'll go over how to represent circuits, and also cover some of the basic methods by which people solve circuits.
We'll also introduce an element called an op-amp, and use that element in order to enable us to do things like modularity and abstraction from our circuits.
First, let's talk about representation.
In the general sense, when you come across a circuit diagram, you're going to see-- at the very broad level-- a bunch of elements and a bunch of connections between the elements.
Those things will form loops and nodes.
If you don't actually specify the elements, then your circuit diagram actually looks a whole lot like a block diagram.
And in fact, block diagrams and circuit diagrams are very closely related in part because block diagrams are used to model feedback systems, which frequently are implemented using circuits.
In this course, we're going to be focusing on independent sources and resistors as the two major kinds of elements that we'll use in our circuits.
We'll also use things like potentiometers, which are resistors that you can adjust, and op-amps.
And we'll look at op-amps specifically in a later video.
But I have one drawn up here just so you recognize it when you see it written.
Note that it looks a whole lot like the block diagram symbol for a gain.
And that's intentional, and we'll cover that later.
But in the meantime, the other sources that we're going to be using are independent current, and voltage sources.
We're going to use resistors to adjust the amount of voltage and current that we're actually dealing with and then sample either the current or the voltage at a particular point in our circuit to get the desired values that we're after.
On a circuit diagram, when you're interested in the voltage drop across a particular element, you'll indicate it by putting a plus and minus sign.
This also indicates the directionality of the voltage drop.
Likewise, when you're interested in the current flowing through a particular element, you'll usually see an indication of it by labeling the current i, and then maybe i with some sort of subscript, and an arrow indicating the direction of current flow through that element so that you avoid making sign errors with the person that might be reading or writing your diagram.
A quick note here.
This is the reason that electrical engineers use j to symbolize values in the complex plane.
It's because i is used in particular for values of current.
Let's review Kirchhoff's voltage laws and Kirchhoff's current laws.
You've probably covered this in 8.02, electricity and magnetism, or possibly in an AP physics class.
But we're going to go over it really fast right now.
Kirchhoff's voltage law is that the voltage drop around a loop is equal to 0.
Or if you take the voltage drop across a particular loop in your circuit, the sum of those voltage drop is going to be 0.
Let's demonstrate on this diagram.
Or, I'll demonstrate on this diagram.
Say the voltage drop across this element is equal to V, right?
Doesn't matter what it is.
We're going to stick with that.
The voltage drop across these elements, if I were to move around this loop, is going to sum to 0.
Note that if I'm tracing out my voltage drop across this loop, I'm actually moving through this voltage source in the direction opposite of its indicated potential.
So when I move through this voltage source, I'm going to account for its value as negative V. As I work my way around the rest of the circuit, the voltage drop across these elements is going to sum to V. This is true for all loops in my circuit.
So any loop that includes V, the elements I encounter as a consequence of moving around that loop are going to have voltage drop equal and opposite to the value I get by moving through V in this direction.
This loop counts, too, but it doesn't include V. All this loop tells me is that the voltage drop across this element is equivalent to the voltage drop across this element.
Or, the voltage drop in this direction across that element is equal to the voltage drop in this direction across this element.
That's Kirchhoff's voltage law.
Kirchhoff's current law is that the current flow into a particular node is equal to 0.
Or, if you take all of the current flows in and out of a particular node and sum them, they should sum to 0.
I've actually got the same set up here.
I'm not going to use a current divider.
I'm interested in the current flowing over this element.
It's actually the same as the current flowing over this element because resistance doesn't change current, resistors flowing through a resistor should not change the current.
So this is still the same i.
Here's my node.
The current flowing in this direction and in this direction, if I took the linear combination of these two currents, they would be equal in value to the current flowing into this node.
When I'm looking at the current flowing through a particular node, I pick a direction.
It's usually arbitrary.
I pick a direction.
It's arbitrary which direction I pick.
Typically, you pick currents flowing into the node as being positive.
I sum up all the currents, and I set that equal to 0.
So in this case--.
Or-- pretty simple.
Let's practice on this particular circuit.
One thing to note is that when you're solving circuits in the general sense, both when you want TA help and when you're solving for a mid-term and want partial credit, you want to label all of your nodes, all of your elements, and all of the currents that you're interested in solving.
See, I've got my voltage drop across this resistor, this resistor, and this resistor labeled, as well as these currents, which I'll also be solving for.
The first thing that I would do when approaching this problem is attempt to reduce this circuit to something that is a little bit simpler.
The first thing that I'm going to do is try to figure out how to change these two resistors in parallel into a single resistor and still have an equivalent circuit.
That'll allow me to solve for I1.
There will be 0 nodes in my system.
I'll just have one single loop.
And the current through the system will just be V/R.
So if I'm just looking at these two resistors, I have resistors in parallel.
In the general sense, the way to solve for resistors in parallel is to take the inverse of the sum of their inverses.
When you only have two resistors, you can typically cheat by saying that this is equal to their product over their sum.
I'm going to redraw my current understanding of the circuit.
The other stuff that I've saved myself is that because these resistors are in parallel, they're a current divider.
They take the current in and divide it two ways determined by the ratio between these two values.
The thing I'm actually interested in expressing is that V2 and V3 are the same value.
When you have a current divider, the voltage drop across all elements in the current divider are the same.
So the value of V here is going to be both V2 and V3.
2R plus 6/5 R. I'm going to go with 16/5 R for now.
I've solved for I.
At this point, I have a voltage divider, which means that the current flowing through this part of the system is going to be the same.
But the voltage drop across this element versus this element is going to be proportional to the ratio between these two values.
V1 is going to be the amount of the total resistance in this simple circuit that this resistor contributes over the entire resistance in the system.
Or, 10/5 R over 16/5 R, which is 10/16 R, or 5/8 R. Or, it's going to be 5/8 V. Same thing happens with V2.
Note that these two values should sum to V in order to maintain Kirchoff's voltage law.
We've also found V3.
So the two things that we have to find are I2 and I3.
Here, I've just done Kirchoff's current law for this node.
Because I'm working with a current divider, I can break up the total current flowing into that node into the number of parts equal to the sum of these values and then distribute them.
And then, that's [?
inappropriate ?]
given that less resistance means more current.
What do I mean by that?
Well, I mean that here my current is equal to 5/16 V over R. I2 is going to be equal to this value over the sum of these two values times I1.
Likewise--.
And just to simplify--.
That concludes my tutorial on circuits.
Next time, we'll talk about other ways we can solve this circuit, and then we'll end up talking about op-amps.
Today I'd like to talk to you about a new method for solving circuits.
Last time, we reviewed using KVL, KCL, and Ohm's Law in order to solve circuits in a general sense.
This produced a lot of equations, in particular, a lot of redundant equations or a lot of dependent equations, that we don't necessarily need for solving our circuit.
Today I'm going to review the Node Voltage Component Current method, which is very similar to node analysis.
So if you hear one versus the other, know that you're talking about approximately the same thing.
And I'll review the difference in a second.
Once we know how to use the NVCC method for solving circuits, we can express the relationships between components in our circuit more concisely and more effectively, and possibly solve our equations faster, which is relevant when we're working on a mid-term or in the general sense, just trying to save time.
Let's take a look at NVCC.
As I said before NVCC, stands for Node Voltage Component Current.
And what that means is if you have a very simple circuit-- let's say just a voltage source and a couple of resistors.
Previously, with KVL, we were interested in the voltage drop around the loop being equivalent to 0.
In this case, we're actually going to look at the voltages associated with particular nodes.
To do that we're going to label our nodes, which are anywhere our components connect.
We're also going to go after the component current.
When we're doing KCL, we typically look at the flow in and out of a particular node.
At this point, we're going to look at the flow through a given component.
So we're also going to label all of the currents associated with our circuit.
But when we approach NVCC method, we're going to think about the currents flowing through a particular component as opposed to and out of a particular node.
NVCC is sort of the opposite of KVL and KCL in that sense, even though you're still going to end up using the same relationships.
So at this point, I've labeled the currents going through-- I labeled one individual current for every component in my circuit.
The next step would be to identify what I'm going to call ground, or in particular, one of my nodes -- I'm going to assign to ground or relative voltage 0.
At that point, I'm going to write out the relationships between the voltage drop across particular components, the current flowing through that particular component, and whatever relationship the component requires the voltage and the current to have to one another.
That's a whirlwind review of NVCC method.
Node analysis is very similar.
The main difference between node analysis and NVCC method is when your component is a voltage source.
And there are multiple currents flowing into that voltage source.
You can treat this as a single voltage.
You can treat this as a single voltage node, where this voltage it has value 0.
And actually write your KCL equations as though this point were collapsed.
So current flowing in and current flowing out, or vice versa, have to sum to 0.
That's the major difference.
Let's look at an example.
You can find this example repeated in 6.4.3 of the reading.
And I'm going to walk through these directions.
So the first thing I'm going to do is label my nodes and my currents.
People interchangeably use e or n. It doesn't really matter.
I guess n in particular could refer to the node, while e in particular could refer to the voltage associated with that node.
Now I'm going to specify the voltage drop across a particular component in terms of the node voltages.
I'm also going to assign n0 to 0 as my ground.
As a consequence, I know that n1 is going to be 15 Volts.
My voltage drop is typically specified in the same direction as the current - that I've also decided.
So this convention is arbitrary.
But if you want to be consistent in your work, and make it easier to get partial credit or get help in office hours, et cetera, then assume that the voltage drop occurs in the same direction as the current.
That's it for our relationship associated with voltage drop.
Now I'm going to go over KCL for the relevant nodes.
And in the last step I'm going to combine the two into the equation that you're certainly allowed to use to express your work on midterms.
Or if you can skip to the third step immediately, then that's OK. Just check your signs.
Here's the second step.
Flowing into n1 is i0 and flowing out is i1.
So i0 is going to be equal to i1.
Flowing into n2 is i1 and i3.
And flowing out of n2 is i2.
Flowing into n0 is i2, and flowing out of n0 is i0 and i3.
we're almost certainly not going to end up using that equation.
Because it's the last of our KCL equations and is dependent upon the other equations that we've already written out for KCL.
I3 is equal to 10 Amperes.
So we can go ahead and make that substitution.
I still have to work with i1 and i2 though.
And I can go after expressions for them in terms of my node voltages and components by using the equations for voltage drop I made earlier.
This is the equation that you can jump straight to, if you understand where this expression comes from, as a consequence, of our KCL and also our component voltages.
I'm going to substitute in 15 Volts for n1 here.
And now I have an expression that only contains n2 and known values.
So I can solve for n2.
and I'll do that real quickly right now.
So first I'm just going to copy this over.
I'm going to multiply through by 6 Ohms.
And I solved for the voltage associated with n2.
At this point, I can solve for i2 and i1.
Sorry about that.
What do I have left?
i3 I know is 10 Amperes.
i0 is equal to i1, which is negative 1 Amperes.
There's a voltage drop associated with the 3 Ohm resistor, which is 15 minus 18, negative 3 Volts.
And the voltage drop associated with the 2 Ohm resistor is 18 Volts, since n0 is ground.
This concludes my introduction to node voltage component current method or node analysis without the ability to collapse voltage sources for KCL.
Hi.
Last time we talked about NVCC method and how to reduce the number of equations we had to deal with to solve a particular circuit.
At this point, we're pretty well equipped to solve circuits in the general sense, but we really haven't talked about how to use that information or possibly use circuits in a particular way.
Before we jump into both that and abstraction of circuits, we need to talk about op-amps.
Op-amps is short for Operational Amplifier.
And it's a tool that we can use in order to sample particular voltages from a subsection of the circuit without affecting it.
Another thing we can use op-amps to do is modify our signal.
Or if we're going to sample a voltage from a particular subsection of the circuit, we can then do stuff to that voltage without affecting the circuit, all within the op-amp or within the op-amp's special subset of circuitry.
So first of all what is an operational amplifier?
Well, an operational amplifier is a giant web of transistors.
But what an operational amplifier does is act as a voltage-dependent voltage source.
It can effectively sample voltages from an existing circuit and then use them to power some other object, for instance a light bulb.
If you set up this kind of circuit, you will not actually be powering this light bulb with 5 Volts, because the light bulb itself acts as a resistor.
And so the voltage drop across this part of the circuit is going to be different from just 5 Volts.
If you want to enable a voltage drop of 5 Volts across this light bulb, then you have to stick up an op-amp.
You have to use an op-amp to sample the voltage drop at this component and put it in between the light bulb and the rest of the circuit.
When you see an op-amp on a schematic diagram, it'll frequently look like this.
You'll have a positive input voltage, a negative input voltage, power rails, which are actually the thing that determine the range of expressivity that the op-amp has, and an output voltage.
In reality, the relationship between the output voltage and then input voltages is something like this, where K is a very large number.
The effect that this has is that Vout is going to be whatever Vout needs to be, such that Vplus is equal to Vminus.
That's the basic rule you want to use when you're interacting with op-amps.
So in this case, if we wanted to power this light bulb with 5 Volts, we would do something like this.
Excuse the sloppiness of the second diagram.
We still have our 10 Volt voltage source.
We still have our voltage divider.
This point samples 5 Volts from this sub-circuit -- and isolates this part of the circuit from the light bulb.
Vout has to be whatever value is necessary such that this sample point and this sample point are equal.
Since this value is 5 Volts, this value will also be driven to 5 Volts by the op-amp, which means that this value is 5 Volts.
And we've successfully managed to power a light bulb with 5 Volts.
The other thing you might be asked to do is to take an existing schematic, an existing circuit diagram and figure out what the operational amplifier does to a given signal or possibly what Vout is or possibly what Vout is in terms of the input signal.
So let's practice using this diagram.
Here's what we're after.
I'm going to figure out where Vplus is going to be.
This is another voltage divider.
I'm now interested in Vminus, in terms of Vout, which is another voltage divider.
I can set these two equations equal to one another and solve for Vout.
I found a new expression for Vout in the particular case where V is 10 Volts.
If my input voltage were previously unspecified, or if this voltage source were not specified or just Vin, then I would be after this expression.
Some things I'd like to mention, while we're talking about op-amps, all the operational amplifiers we've been working with so far deal with Vout in terms of Vin, where Vin is driven through the positive terminal, and the negative terminal is typically connected to ground.
You can do the opposite and end up with some interesting effects.
But it comes at a cost.
It is entirely possible that you will end up driving your op-amp to an unstable equilibrium.
What you need to look at is this relationship.
There may be a particular point, in which case your system is stable.
But if you get any sort of minor perturbations, you'll actually end up with divergence.
If this is the case, then you'll probably burn out your op-amp.
You can do this by hooking it up in this way.
This is expensive and could possibly burn you.
The other thing to note is that the power rails on your op-amp limit its range of expressivity.
And I think I've said this before, but it's worth mentioning again.
If your op-amp is only powered by 10 Volts, it cannot amplify your input signal to a final value greater than 10 Volts.
Likewise, if your input value is a negative voltage, and you're working with a non-inverting amplifier, if your ground is truly ground or if your ground is higher relative than your input voltage, you cannot actually express a negative voltage.
The third thing I'd like to quickly mention is that there are some terms associated with op-amps that you might hear used by the staff or online, that sort of thing.
A buffer and a voltage follower are the same thing.
And that's explicitly when you want to sample a signal or you want to sample a particular voltage, and you don't want to multiply it or add it to something or do any kind of LTI operations that we might be able to do using op-amps in this course.
You can work with amplifiers.
And the thing we worked with earlier was an amplifier for a value less than 1.
You can also use op-amps to some signals.
And if you look for a voltage summer amplifier on the internet, you should be able to find some information.
In any case, op-amps are really powerful.
They allow us to both isolate a particular section of a circuit and sample a particular voltage value from that circuit without affecting that circuit, and also allow us to modify that particular voltage value before using it in another part of our overall circuit.
Therefore, we're enabled to design more complicated and powerful things.
Next time, I'll talk about superposition and Thevenin Norton equivalence, which will further enable modularity and abstraction in our circuit design.
Last time we talked about op-amps and the fact that they allowed us to abstract away certain components of a circuit and sample particular voltages from a circuit and then modify those voltages as we would like in an LTI fashion.
Today I'd like to talk to you about some other interesting things that fall out of the fact that we're only dealing with LTI systems.
In particular, Thevenin/Norton equivalence and superposition.
Thevenin/Norton equivalence is the idea that if you have a very complex circuit, but it's still a LTI circuit, you can express it using a linear curve.
And superposition is the idea that because you're also dealing with LTI components, if you're trying to solve a circuit, you can take the individual contributions of independent sources to that circuit and sum them in order to find out either the total current flowing through a particular component or the total voltage drop across a particular component.
At this point, I'll walk you through both.
Thevenin/Norton equivalence is an important concept in that you may have a very complicated circuit.
And you don't really want to talk about the entire complicated circuit.
You just want to sample the voltage drop or current in a very particular location.
Because we're dealing with LTI systems we can actually express that particular sample as its relationship between I and V and possibly whatever resistive component is associated with the voltage drop across that sample.
We can solve for this curve by looking at the location that were sampling and finding the open circuit voltage associated with that position.
Or if we were to leave the two terminals from where we are sampling open, what is the voltage drop across that section of our circuit?
That's the point right here, where the current flowing through the system is 0.
And there's a given voltage associated with it.
Likewise if we want to find the other intercept, we can close those two terminals by running a wire across them and then look at the current that flows across that wire.
That's the close circuit current.
The slope is going to tell us about our resistance, if any.
If we're only dealing with a voltage source or a current source, then we're only going to be dealing with a straight line.
Once we've solved for these values, we tend to express them either as a Thevenin equivalent circuit or a Norton equivalent circuit.
And you can convert between the two.
Let's walk through an example.
Here I've got a simple circuit.
And I would like to find the Thevenin equivalent of that circuit, given that I'm sampling from above this resistor and below this resistor.
The first thing that I'm going to do is find the open circuit voltage.
Or if I were sampling from this point to this point, what is the voltage drop across this section of the circuit?
Well, I've got 36 Volts.
There's 12 Ohms of resistance in the section that I'm sampling.
And there's 6 Ohms of resistance outside the section that I'm sampling.
So 2/3 of my total voltage drop across this voltage divider is going to be accounted for inside my Thevenin equivalent circuit.
So my Thevenin equivalent voltage is 24 Volts.
Now I want to go after the short circuit current.
If I were to draw a wire from this terminal to this terminal, I'm curious what this value would be.
As a consequence of connecting these two terminals, these resistors are going to be completely bypassed.
We don't actually have to drop any voltage or put any current through these resistors because there's actually an infinitely easier way for the current to go.
In this case, we're just going to follow Ohm's law.
36 divided by 6 is 6 Amperes.
But because of the directionality on this short circuit current, current in this system is going to flow in this direction.
So are short circuit current is actually negative 6 Amperes.
That just leaves Rth.
Once again we're going to rely on Ohm's law.
If we divide our voltage by our current-- you can't have a negative resistance.
We're actually going to divide by the negative of this.
We'll get out 4 Ohms.
Let's do it again, this time with a slightly different sample point and find the Norton equivalent of our circuit.
So if I'm only sampling across the 9 Ohm resistor-- I personally still solve for the Thevenin equivalent voltage first because I think it's easiest.
I'll just make note of it over here.
The voltage drop across this resistor is determined by the fact that this is a simple voltage divider.
The other resistors in this circuit-- 7, 2, 9 Ohms.
So the voltage drop across this resistor is going to represent half of the total voltage in our circuit.
That's 18 Volts.
The short circuit current associated with connecting these two terminals is 36 Volts divided by the sum of these two resistors.
Once again we're going to end up bypassing this resistor entirely because, think about it this way, if there was a resistor across here with 0 resistance, all the current would sink through it, or it would represent the location for which infinite current would flow.
36 divided by 9 is 4.
And because we're following in the opposite direction we're going to say this is negative 4 Amperes.
In this orientation, the Norton equivalent amperage is actually the negative of this value.
You sometimes see the arrow pointing downwards.
This preserves our property of short circuit current.
If this was short circuited, Isc would still be negative 4 Amperes.
And our Vth-- if our Vth was the voltage drop across these two points-- ha, I haven't solved for Rth yet.
There are a few ways to do this.
But I'm going to use Vth and In.
18 over 4, we can reduce it to 9 over 2.
And if I were to solve back out for the voltage drop across this resistor just as a consequence of I Norton, I would get 18 Volts, which is also the Thevenin equivalent voltage for this circuit.
That covers Thevenin/Norton equivalence.
And we're going to talk about superposition really quickly.
Superposition is yet another circuit-solving strategy.
It falls out as a consequence of only working with LTI components.
And it means that in order to solve for a given component current or component voltage, you can actually find it by taking the linear combination of every contribution as a consequence of independent sources.
What do I mean when I say that?
Well, if you have a circuit like this-- and you might recognize it from the NVCC lecture earlier-- you can actually express it as a linear combination of just the voltage source and just the current source.
Note what when I remove the current source, I end up with no connection because if you have 0 current flowing through a system, it's the same as if the connection has been removed.
Note that when I remove the voltage source, I no longer have a voltage drop across that connection, but I maintain the connection.
If in this example I was just looking for I1.
I1 would be the linear combination of the contribution of I1 in this particular circuit plus I1 in this particular circuit.
So this is the expression that I'm looking for.
In the first circuit, I'm just going to use V over IR -- or V equals IR -- and I'm going to have 3 Amperes in this direction.
In this circuit I'm actually going to have to use a current divider.
Identify that I have five parts total to distribute among these two branches.
Identify that this side is going to get the inverse of this side's contribution to the total parts, or this side is going to get the proportion of parts equivalent to this side's contribution.
And vice versa.
That this side is the side that I'm interested in.
And so 2/5 of the total current flowing through this current divider is going to end up in this branch.
2/5 of 10 is 4.
It's in the opposite direction of this current flow.
So I'm actually going to contribute negative 4 Amperes from this subcircuit.
This should match the results from the NVCC lecture.
That's my super fast coverage of superposition and also Thevenin/Norton equivalence.
Next time we're going to start a whole new module.
Hi.
Today I'd like to introduce a new module.
In previous modules we've talked about how to model particular systems, how to make predictions about certain kinds and classifications of systems, and also how to design both theoretical and physical systems.
If our systems are operating in a deterministic universe, then we're all set.
We've accounted for all the things we could possibly account for.
But if we're making systems that are going to operate in the real world, then we need to be able to deal with some level of uncertainty.
Today I'm going to talk about probability, which is the method by which we're going to model uncertainty in our world.
And later we're going to talk about different strategies we can take to deal with that uncertainty to hopefully increase the level of autonomy of our systems as they operate in the real world.
The first thing I need to talk about is how to properly model probability, or how to probably talk about probability such that we can use it to talk about uncertainty.
When you're talking about probability, typically you'll end up talking about the sample space or U.
This refers to your entire universe, where your universe is the values that you care about, or the possible assigned values of the variables that you care about.
I'm already talking about variables, but what I really mean to say is events.
There are different states that your universe can take, and those states can be sort of exhaustively enumerated or be atomic, or they can be factored into different variables.
Let me elaborate on this a little.
If I were talking about four coin flips, if those events were specified atomically then I would have to exhaustively specify all possible sequences of four coin flips.
4 heads, 3 heads and 2 tail, 2 heads and 2 tails, that sort of exercise.
Or flipping 4 heads would be one event, flipping 3 heads and then 1 tail would be a separate event, flipping 2 heads 1 tail and another head would be a third, distinct event.
If we're talking about a factored state space, then we'll have a different variable representing each one of these coin flips.
And each one of those variables will say, here is the value associated with that particular sub-event.
And the accumulation of all those values, or the specification for all those values, will end up referring to the same states that were addressed by the atomic events.
Let me talk about variables, because they're the thing that allow us to not exhaustively enumerate every possible event in the universe, and talk about our universe in meaningful ways, or in ways that they can be effectively communicated.
And why am I talking about random variables?
Why aren't they just variables?
Well, random variables is the way that you specify the fact that you're talking about probabilistic variables.
When you're just dealing with regular algebra, variables have some sort of assigned value, and you're not forced to be in the space of 0 to 1.
When you're talking about from 0 to 1, when you're talking about probabilities, you're forced to be in the spaces of 0 to 1 and forced to remain within the reals.
If you want to talk about all the possible assigned values of that random variable, then you're talking about the distribution over that random variable.
This means that A could be anything that A could be.
You're talking about the function that says give me a value of A, and I will tell you a probability associated with that value of A.
If you're talking about the probability of A being assigned to a particular value, then you'll return out the probability.
If A represents the color of the shirt I'm wearing, and I want to know the probability that A is some value, then I look at the color of the shirt I'm wearing and try to determine whether or not it's 1 or 0.
Or possibly look at the colors of shirts that I've worn over the past year and make some sort of ratio of the number of pink shirts I've worn over the past year.
If I'm dealing with a factored state space, I'm going to end up talking about more than one random variable.
There are two main ways to talk about more than one random variable at the same time.
One is joint probabilities, where all the random variables are collectively specified at the same time.
And the other is conditional probabilities, where you've already decided that you specified some value for one or more random variables.
And then within that scope you're going to talk about the probabilities associated with other random variables.
I want to demonstrate this graphically, but there are two more things that I need to mention right now.
One is the difference between frequentist and Bayesian interpretations of probability.
Right now they don't seem particularly relevant, but people will use these words, and it's good to know, approximately, what they're talking about.
The frequentist interpretation of probability is more relevant when you're talking about actions that happen a lot of different times.
For instance, how frequently it rains.
If I say that today is a Wednesday, and I want to know the probability that it rains on Wednesdays, then that probability is open to frequentist interpretation because there are a lot of Wednesdays, and it's rained a lot in the universe of Wednesdays or the space of possible Wednesdays.
So thinking about the fact that there's a 70% chance of rain, or a 30% chance of rain, or whatever probability of rain there is on Wednesdays, makes sense, or is open to frequentist interpretation.
The other interpretation that you'll hear talked about with respect to probabilities is the Bayesian interpretation.
Bayesian interpretation tends to be more relevant when you're talking about spaces that are more atomic as opposed to factored, or represent events that are specified to the point that it does not make sense to talk about them in the frequentist sense.
When we talk about probabilities in the Bayesian interpretation, we frequently use the term likelihood.
For instance, if I'm talking about whether or not it's likely to rain on August 24, 2011 in the afternoon, the specificity of that event is so high that at that point it doesn't make sense to talk about the frequency of Wednesday, August 24, 2011 in the afternoons, at least for now.
At that point we're talking more about likelihood and less about frequency.
That event is more conducive to Bayesian interpretation.
No whirlwind tour of probability would be complete without a mention of the three axioms of probability.
The first axiom is that the likelihood of the universe happening is one, or all random variables are going to be specified at some point.
Relatedly, the likelihood of nothing happening is 0.
What these two really do is establish the boundaries of a graphical representation up here.
The other axiom of probability is that if you're going to be talking about the union of two events, or the probability associated with one or the other event having an assignment, then you're talking about the probability of one of those variables and the probability of the other variable, and then removing the section that you've double-counted.
So if I were to attempt to demonstrate this on this graph, I would be talking about the probability of A, added the probability of B.
And at this point I've double-counted this section the middle where they're both one.
So I would subtract this exactly once.
And then I'm talking about the size of this space.
If I'm talking about the probability of A being equal to 1, then I'm talking about the space in which A is specified to be equal to 1, divided by the area associated with my universe.
When I'm talking about joint distributions, I'm going to find the space in which both of these things are true, scoped to the size of the universe, or the entire sample space.
So in this space both A and B are true, and I'm looking at it relative to the size of the universe.
In contrast, when I'm talking about conditional probability, or the probability of A given B, I'm going to look at where my specifications are true, scoped to where my givens are true.
So if I'm already dealing in the space restricted to B, I'm just looking at this size, or this space.
But because I'm scoped to B, I'm going to end up dividing by the area of B, instead of the area of U.
This is the main difference between the joint and conditional distributions.
And a lot of people get hung up on it, which is why I'm exhaustively walking through it.
A couple of other things before we talk about the first way in which we can use our models for uncertainty to do some amount of addressing of the fact that we're going to have to deal with uncertainty in the future.
If we start off with a joint distribution and we want to reduce the number of variables that we're actually talking about, we can do so by exhaustively walking through all the possible assigned values for the variable that we want to disregard, and then summing up the values appropriately.
An easy example for this is if we had the joint probabilities of all the colors of shirts that I wear and all the colors of pants that I wear, and we only wanted to talk about all the colors of shirts that I wear, then we could exhaustively cover all the different colors of pants that I wear and accumulate all the different values of shirts that I wear simultaneously, and then collect that distribution.
Related is the concept of total probability, which is the same kind of accumulation.
It just operates in the conditional space, as opposed to in the joint space.
So in this case, if I'm already operating in A given B land, I have to scope myself back out to the space of the universe by then accounting for the fact that I've only been operating in the scope of B.
Then exhaustively enumerate all the possible values of B, sum the probabilities associated with those values, and then I've reduced the number of dimensions that I'm talking about.
The final thing we have to talk about before we move on to state estimation is Bayes' evidence.
Or you've probably seen this demonstrated as Bayes' rule.
If I want to talk about B scoped to A, and all I have is A scoped to B, B and A.
When we walked through total probability, we saw that the conditional probability multiplied by the probability associated with the variable that we're conditioning on, is equal to the joint probability.
When we multiply P of A given B by P of B, we're going to end up with the joint probability associated with the two variables.
When we then divide back out by A or scope our joint probability to A, we end up talking about conditional probabilities again, which is where B given A comes from.
Bayes' evidence, or Bayes' rule, is the basis for inference, which is going to be really important for state estimation, which we'll cover next time.
Last time we talked about probability as an introduction on how to model uncertainty.
State estimation is one of the ways that we can deal with uncertainty in our system.
We can take a model that we don't completely understand and attempt to infer information about it based on the things that we can observe about that particular system.
In particular we're going to look at a set of observations and actions that either our system takes or that we observe about the system and that we take on that particular system.
If we continue this process for multiple time steps then we can continue to attempt to learn things about the particular system.
The process of completing that behavior over multiple time steps while making those inferences is what we refer to when we talk about state estimation.
First off, state estimation is a process that's completed as a consequence of wanting to understand a stochastic state machine.
State estimation itself is not a stochastic state machine.
State estimation attempts to take a stochastic state machine, make a model of that stochastic state machine, and then run state estimation on it iteratively to attempt to figure out, or recursively, an attempt to figure out what's going on inside that stochastic state machine.
When you build a stochastic state machine model there are three components that need to be specified.
The first is the starting distribution over states.
For instance, let's say that I believe that I am sick and I'm trying to figure out what it is that I am sick with.
And I could be sick with three things, as far as I'm concerned.
I could be sick with strep or I could be sick with some other more boring virus.
Or I could be sick with mononucleosis.
The starting distribution refers to my starting belief as to the systems.
And if I'm generically sick in the general sense, one of the assumptions that's frequently made with respect to starting distributions is that they're uniform, right?
It could be equally any of these things.
The second thing you need to specify when you're talking about modeling a stochastic state machine is your observation distribution.
Or what is the likelihood associated with making a particular observation given that you're in a current state?
For instance, if I have mononucleosis how likely would it be that I observe a bunch of white ugly patches on the back of my throat?
Or if I had strep?
What is the likelihood associated with that?
That kind of thing.
Typically this observation variable is factored into a couple different phenomena.
In the sick example the best thing to talk about is symptoms.
[INAUDIBLE] Am I lethargic?
Do I have the white spots on the back of my throat?
Do I have a fever?
That sort of thing.
The last thing that you need specify when you're talking about modeling a stochastic state machine is your transition distribution.
You assume that your state machine is going to change over time.
Or it is likely that I will get more or less sick.
And there are things that I can do to induce that kind of change.
Or there are things that I can do that effectively model the passage of time.
Your actions for a stochastic state machine model can either be actions that the model takes and you were exclusively doing observations.
But one of the particular observations that you do also qualifies as an action or something that indicates the passage of time.
Or actions can be something that you do to a particular state.
In the sick example, things I could do to myself to try to make myself feel better or at least figure out better what is going on or what might cause my distribution to sway towards one particular state.
I could take antibiotics.
Or sleep in and drink a lot of orange juice.
Or continue my day as normal.
Given a particular action, any particular state that you're starting from, your transition distribution tells you the likelihood associated with being in a new particular state.
So as a consequence of making those actions, does the distribution of likelihood of a particular illness change?
At this point I'm going to walk through a step of state estimation.
Each step of state estimation is the same.
In fact, if you complete multiple steps based on the information that you gained from the previous step, that's referred to as recursive state estimation.
And I'll keep walking through the sick example.
So when you're doing state estimation you're trying to figure out something about a system that you cannot perfectly model.
For instance, either your own immune system or your own susceptibility to a particular disease.
And you have all the components you have for your stochastic state machine model.
As a consequence of the passage of time or as a consequence of making an observation and either observing an action taken by your stochastic state machine or performing an action upon your stochastic state machine you're going to make a new estimation of what you believe the current state of that unknown system.
Or system that is not completely observable to you.
You're going to make a new estimate of your belief of the state of that system.
In short you're going to solve for the probability distribution over S_(t plus 1).
There are two steps.
The first step is referred to as the Bayesian reasoning step.
And it involves performing Bayes evidence or Bayes rule upon the current state distribution given a particular observation.
So at this point I've made some sort of observation about myself.
If I'm talking about the sick model, right?
I spent all day coughing.
Or I have a fever.
Or my throat is sore.
Or I feel extremely lethargic, right?
Given that observation I can take the P(O given S) from my observation distribution multiply it by my current understanding of the state distribution.
And then divide out by P(O).
The slowest way to complete this action is to build the joint distribution and then condition on a particular column.
It's very proper.
But you can save yourself some cycles by doing this.
Let's say I started off with the uniform distribution, right?
It could be equally likely that I have strep or a normal virus or mono.
As a consequence of making the observation that I don't have white spots on the back of my throat.
I could say, oh the likelihood of me being in that state-- the likelihood of me just having a normal virus is higher and the likelihood of me having either strep throat or mono is lower.
This step takes P(S) and multiplies it by P(O given S).
o Once I have these values I have to scope back out to the universe or I have to normalize these values such that they sum to 1.
That's where I get my P(S_t given O).
At this point I've accounted for the observation that I've made, but I haven't accounted for the action on the system.
That's the next step.
We're going to take our results of Bayesian reasoning which are sometimes referred to as B prime S_t.
And take the action and find the distribution overstates as a consequence of a single time step or a single iteration of state estimation.
The second step is referred to as a transition update.
We've got our updated belief.
We're going to take our transition distribution or our specification for what happens given that we're in a current state and an action has been taken.
At that point we'll have a probability distribution over the new states.
And here are my values from the first step.
As an example let's say that I sleep in and drink a lot of orange juice.
As a consequence of sleeping in and drinking a lot of orange juice there's some amount of likelihood that I will either continue to be sick with strep or it's possible that I actually have just a normal virus.
If I have a normal virus and I sleep in and drink a lot of orange juice, this causality sounds backwards but it's as a consequence of not being able to make perfect observations on the system.
If I have an amount of belief that says that I think I have strep and I sleep in and drink a lot of orange juice, then it's equally likely that I will have either strep or a normal virus after completing that step, right?
It doesn't differentiate between the two.
If I'm sick with a virus and I sleep in and drink a lot of orange juice, then the state that I'm going to encourage myself to be in is I have a virus.
If I have mono and I sleep in and drink a lot of orange juice then there's some likelihood on the next day that I will still be in a state that looks like I have mono.
But there's also some likelihood associated with it that I will be in some state that looks like I have strep.
That's what happens when you run the transition update.
When you run the transition update you end up accumulating all the probabilities associated with being in a particular new state.
As a consequence of being in a particular previous state and entering that new state based on the transition distribution.
Once you accumulate all these values you end up with your new distribution over a new state.
This represents one step of state estimation.
If I wanted to run multiple, I would take the value that I got here for S_(t plus 1), replace it in the value for S_t.
And run the same process of Bayesian reasoning and transition update.
This concludes my review of state estimation.
Next time we'll talk about search.
Hi.
Today, I'd like to talk to you about search.
Previously, we've talked about ways to model uncertainty and also how to make estimations about a particular system when we're observing it from the outside or when we don't have perfect information.
At this point, we've almost got enough components to attempt to make an autonomous system.
But at this point, we also haven't enabled our autonomous systems to make complex decisions for itself.
This is why we want to be able to encode search.
Or we want to be able to enable a system to make an evaluation of a succession of decisions, or a succession of actions, when there are multiple choices and possibly multiple choices at every level.
So as a consequence of being able to do search, our autonomous system should be able to complete a successive grouping of actions.
In 6.01, we're going to be searching state spaces.
And searching state spaces borrows a lot of ideas from state transition diagrams, or what we already know about state machines.
When we're searching a state space, we want to know what states we're searching, what the transitions between them are going to look like, or how the access to the transitions from a given state to all the states that are its neighbors.
We're going to want the start state to be specified so we know where to begin.
We want a goal test, which actually specifies what we're looking for as a consequence of the search.
Or if we get to a state and we want to know whether or not we're done, we use our goal test.
And it could look at the output of the state, or actually just the state name, that sort of thing.
The other thing that we're going to have while we're searching is a legal action list.
The searches that we're going to do today, you and I are going to be able to see the entire state transition diagram at once.
But if we're encoding how to search on our robot, our robot can't see the entire state transition diagram at once because if it could, then it wouldn't have to do search.
It would know how to get there from here.
Therefore, we give our system a legal action list, or all the actions that it should attempt to do at a given state.
It's entirely possible that there aren't legal transitions at every state for every legal action.
But it's good to have an exhaustive list of things to try and see if they succeed or fail and, as a consequence, what state you end up visiting every time you try to do one of these.
Being able to do search is great.
But we also want to be able to keep track of where we searched and how so that once we're done searching, we can actually execute the best thing.
We're going to use a search tree to keep track of where we've been and how we got there.
Search tree is going to be comprised of nodes.
It's otherwise going to look like a directed, acyclic graph.
And it's going to have a lot of similarity to any particular state transition diagram that we end up searching.
But it's going to have nodes instead of states.
Nodes are different.
Nodes represent both the state that you've visited as a consequence of expanding its parent node, the parent node, or the place that you came from as a consequence of getting to that node, and the transition that you made in order to get there, or the action that happened that got you from the parent node to the child.
Keeping track of a list of nodes is known as a path, or it specifies where you've been and how you got there.
And if you're at a given node, you can actually use the reference to the parent node and the action to trace back from whatever node you're at currently to its parent node, to its parent node, to its parent node, and then finally get back to the start state.
At that point, you'll know what path to take.
So the only thing left to do is, how do you figure out which paths to follow first?
That's where the agenda comes in.
The agenda is going to be the collection of all partial paths you've ever created as a consequence of expanding nodes and then putting its child nodes on a partial path meant for future expansion.
The order in which you add and remove things to the agenda is going to determine what your search tree looks like.
That's a lot of information.
At this point, I'm going to go over an example.
We're going to search the state transition diagram.
We're going to start at A.
And our goal test would be whether or not our state was equal to E. Today, we're going to try two different kinds of basic search.
One is referred to as breadth-first search, or BFS.
And one is referred to as depth-first search, or DFS.
Breadth-first search refers to the idea that as you build your search tree, you're going to exhaustively expand all the nodes at a given level before advancing to the next level, or all the given nodes at a given depth before expanding to the next depth.
This means you're being very thorough.
It also means that you're guaranteed to find the shortest path from your start node to the goal if it exists.
Depth-first search is the opposite.
As a consequence of depth-first search, you're going to expand all the nodes in a given branch as far down the tree as you possibly can before advancing to the next branch.
It takes up a lot less space than the breadth-first search, but it's not guaranteed to find the optimal path.
Another way to think about these two types of search is that if you're doing breadth-first search, then your agenda acts as a cue.
First items in, or first partial paths that you discover, are the first items out or the first partial paths that you end up expanding.
Depth-first search is when the agenda is used as a stack, or the first partial paths that you visit are the first partial paths-- or, the most recent partial paths that you visited are going to be the partial paths that you first extend.
First in, last out.
Or, last in, first out.
Let me walk through a couple iterations on this state transition diagram.
And hopefully, it'll be clearer what's going on.
The first thing that happens is that you end up visiting and expanding the start node.
That's pretty straightforward.
So the path A is going to be added to both agendas.
And the node A is going to be visited first on both search trees.
If, in the general sense, I say that I'm going to make a transition to states in alphabetical order, and that's the order in which I'm going to add them to my agenda, that's going to be reflected in what I write up here.
Let's say that I'm going to visit new nodes in alphabetical order.
So nodes I would visit are B and C, and I'm going to add AB and AC to my agenda.
Here's where the difference between breadth-first search and depth-first search comes in.
In breadth-first search, because I'm following the convention first in, first out, if I place the partial path AB in my agenda first, then I'm going to expand B as a consequence of the partial path AB first.
So when I go to B, I'm actually going to expand B now.
I'm going to look at the nodes that I can visit as a consequence of expanding B.
The ones that I can visit are C and D. And I'm going to add the partial paths ABC and ABD to my agenda.
So AC, I'm just going to move to the front of the queue, or the agenda, and I'm going to add ABC and ABD.
And I got there through B.
So I'm going to add C and D here.
Depth-first search grabs from the opposite end of the agenda.
So the first thing I'm going to look at is AC.
I'm going to expand to C, look at the nodes that I can reach as a consequence of expanding C, and visit B and D. AB is still hanging out here.
I popped off AC to use it in order to expand C's children.
And I'm going to add ACB first and ACD second.
Note that our search trees already look different.
And we'll actually end up reaching the goal using one of these search strategies first than the other, or as opposed to the other.
If I go back to breadth-first search, I'm going to pop the partial path AC off the front of my agenda.
I'm going to expand C. Expanding C gets me B and D. I'm going to move over my existing partial paths.
And add ACB and ACD.
I've staggered these in order to indicate that they're a consequence of a third iteration of breadth-first search.
But they're actually considered to be at the same depth, since their parents are considered to be at the same depth, since their parents are parents of the start node.
That's the defining feature of breadth-first research is the fact that we're going to exhaustively search a given depth in our search tree before advancing to the next depth level.
If I run one more iteration of depth-first search, again, I'm popping partial paths off this end of the agenda.
I'm going to expand D. D one transition available to a node.
And in plain old fashioned breadth-first search and depth-first search, I run my goal test when I visit a node.
So at this point, I would test whether or not E was my goal test.
I would discover it's my goal test.
My search would return successfully and return the path found.
So AB and ACB remain on the agenda.
I popped off ACD in order to expand D. And I found this path.
At this point, I've concluded depth-first search.
I'm going to do one more around of breadth-first search to demonstrate an important rule.
If I pop ABC off the agenda and move all these over, if I'm looking at ABC, and I look at the children of C, the two children of C are B and D. So the first partial path that I would end up adding to the agenda as a consequence of expanding C in this case would be ABCD.
And it would look like this.
You'll notice that we're going to create an infinite loop.
And there are two basic rules of basic search that I need to emphasize now to prevent you from doing things like creating an infinite loop.
If you look in the textbook, they're called "How to Not be Completely Stupid."
If at any point you're visiting a node in your partial path that already exists in your partial path, don't add it to that partial path.
You'll prevent yourself from creating a cycle.
Because if you visit the same node more than once, you've actually done more work than you need to.
The second rule-- and it's not demonstrated well on this state transition diagram.
But, for instance, if I had two arrows from B to D, there's no particular reason to consider both of these actions.
And if you have a state transition diagram that allows multiple transitions from one state to another based on different actions, then you need to come up with some sort of rule to decide between the two actions.
That's the second rule of how to not be completely stupid is -- if you have more than one transition from one state to another as a consequence of doing search, pick one and come up with a rule to pick one.
This covers basic search.
Next week, I'm going to talk about dynamic programming, costs, and heuristics.
Today I'd like to talk to you about some techniques that you can add to basic search to enable your systems to make more intelligent decisions and save computation time.
Last time, we introduced basic search and the basic idea of how to encode search so that our system can use search when encountering an unknown territory or state space.
And today, we're going to review some things you can do in terms of using the information that you know and also using estimations of the information that you would like to know to improve your chances of discovering the path that you're interested in the fastest.
The first thing that we can do in order to improve our search is use dynamic programming.
Dynamic programming refers to the idea is that once you've done a computation for a particular kind of problem, you can save that computation and use it later, as opposed to having to engage in that computation a second time.
The way this manifests in search is that once you visited a particular state, you don't have to visit that state again.
Because the way your agenda's set up, you found the fastest way to that particular state, as far as you're concerned.
So the general idea is that once you visit a particular state, you don't have to visit it again.
If you're building a normal search tree, it can be difficult to keep track of where you've been, especially if you're doing something like that depth-first search.
To get around this and to enable dynamic programming, the easiest thing to do is just keep a list of the states that you visited as a consequence of this particular run of search.
I'm going to demonstrate dynamic programming by running depth-first search on our state transition diagram from last time.
The first two steps are the same, except for the fact that we're going to keep track of the fact that we visited both A, B, and C as a consequence of the first two iterations of search.
If I'm running depth-first search, my agenda acts as a stack, which means I'm going to take the partial path that I added most recently to the agenda, pop it off, and expand the last node in the partial path.
When I expand C, I'm going to visit B and D. However, B is already in my list of states that I visited, because it's already in one of the partial paths in my agenda.
So I'm not going to visit B again.
Instead, I'm only going to add one new partial path to my agenda, ACD.
I'm also going to add D to my list of visited states.
If I run another iteration of depth-first search, I find my goal.
For completeness, E is added to list of visited states.
This took even fewer iterations that the original depth-first search.
We didn't waste time expanding different nodes.
We also used less space.
In the general sense, the concept of dynamic programming is great to use whatever you can.
And in search, it can save you a lot of time and energy.
Another way we can intelligently improve our search technique is by making considerations for costs that we know that are associated with particular transitions.
In this state transition diagram, I've indicated some costs associated with the transitions between particular states.
If we know the costs associated with transitions, then we can use a cumulation of the weights accumulated through traversing a particular partial path to prioritize which paths we're going to explore first in an effort to reduce the amount of cost associated with our final actions.
In order to do that, we can keep track of the value associated with that cumulation, and sort the agenda at every iteration based on that cost.
If we're using dynamic programming, while making considerations for both cost and heuristics, and also when we're running the goal test when making considerations for cost and heuristics, we're actually going to make our considerations when we expand the node as opposed to when we visit the node.
This difference is very important because it provides us with the most optimal solution.
If it's possible for us to visit the goal node, but it's 100 cost units away, It may be worth our while to search for alternatives that provide a much shorter path to the goal node.
That's why we switch from considering when we visit to considering when we expand.
Let's look at uniform cost search run with dynamic programming.
In my first step, I expand A and add two partial paths to my queue.
I'm also going to keep track of the cumulative cost associated with that partial path.
When I expand A, I add A to the agenda.
Note that I haven't talked about stacks or queues or depth first or bread first or anything.
We're working with a priority queue.
And what that means is that things with a higher priority float to the top.
Or things with the lowest cost associated with them we're going to consider first.
This means that I'm going to expand B in the partial path AB first.
I'll add it to my list.
When I expand B, B has two child nodes, D and C. ABC has partial path cost 3 associated with it.
And ABD has partial path cost 11 associated with it.
That means when I reprioritize my queue, I'm going to end up sorting everything such that AC comes first, ABC come second, And ABD comes third.
Previously, with our strategy for dynamic programming, C would not have been added to this partial path, because we've already visited it with path AC.
At this step, we're going to expand the path AC, add C to the list, and any other time that we end up visiting C, we will not add it to our paths.
If I expand C, I have a transition to B, which is already in my list, and a transition to D that has cost 7.
ABC is going to float to the top of my priority queue.
ACB is not going to be considered, because B is already part of my list.
And ABD is going to remain with cost 11.
At this point, you might say, but Kendra, why am I considering partial path ABC when C is in my list-- or C should be in my list, excuse me-- as a consequence of expanding the partial path AC?
Even though we've expanded the partial path AC, if we have not made any considerations to weed out our list at every iteration, this is still going to float to the top.
And we're still going to have to deal with it, even though we've already expanded C. Since we've already expanded C, we're going to ignore this partial path and just move ACD 7 and ABD 11 up to the front.
D is not in our expanded list.
When we expand D, we have one child node E. Because we're working with cost and heuristics, we do not actually evaluate the goal test when we visit a node.
We evaluate it when we expand a node.
So I am going to add ACDE to my agenda.
It's going to have cost 8.
And ABD11 is still going to hang out here at the back of the priority queue.
At this point, I get to expand E. I skipped adding D to the expanded list when I expanded it from ACD to ACDE.
At this point, I'm going to expand E. And the first thing I'm going to do is test and see whether or not it passes the goal test.
At that point, I stop search, return that I successfully completed the search, and that my partial path is going to be ACDE.
That covers uniform cost search.
At this point, you might say, Kendra, this is bearing a lot of similarity to things like maps.
And I would really like to be able to use my knowledge of things like the Euclidean in order to make more intelligent, even more intelligent decisions about where it is that I go with myself.
And I would say yes, you should be able to.
In fact, people do.
In fact, people do all the time.
They say well, some thing's this far away as the crow flies.
So I know that if I've gone further than that at any point, then I've wasted some amount of time.
But it represents a good underestimate of the distance that I'm going to cover.
This is the basic concept of heuristics.
If you're attempting to find a goal, and you have an estimate for the remaining cost, but you know it's not exactly right, you can still use that information to attempt to save you an amount of computation or amount of search.
In particular, you probably shouldn't be using a heuristic, if you know it's perfect.
Because if you know the heuristic is perfect, then you should be using the heuristic to solve your problems, instead of doing search in the first place.
Or if the heuristic already tells you how long it's going to take to find something, then it probably also has the path that represents how long or that represents that amount of cost.
If you want to use a heuristic effectively, you have to make sure that your heuristic represents a non-strict underestimate of the amount of cost that is left over.
And what do I mean by that?
I mean that if you have a heuristic, and you're using it as a thing to tell you whether or not you're wasting your time, if your heuristic represents information that is bogus or says this particular path has more cost associated with it than it actually does, then it will lead you astray.
Or you don't want to use a heuristic that will prevent you from using a path that actually costs less than the heuristic advertises.
This is what's known as an admissible heuristic.
An admissible heuristic always underestimates if it makes an error in estimation about your total distance to the goal.
All right.
So at this point, I've covered dynamic programming.
I've covered costs.
I've covered heuristics.
And it turns out, you can use all of these techniques at the same time.
When you use both cost and heuristics, in combination, while evaluating your priority queue, that's known as an A-Star search.
And you'll see a decent amount of literature on A-Star.
This covers all of the intelligent improvements to basic search that I will talk about in this course.
We hope you enjoyed 6.01.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, welcome to the 60002, or if you were in 600, the second half of 600.
I'm John Guttag.
Let me start with a few administrative things.
What's the workload?
There are problem sets.
They'll all be programming problems much in the style of 60001.
And the goal-- really twofold.
60001 problem sets were mostly about you learning to be a programmer.
A lot of that carries over.
No one learns to be a programmer in half a semester.
So a lot of it is to improve your skills, but also there's a lot more, I would say, conceptual, algorithmic material in 60002, and the problem sets are designed to help cement that as well as just to give you programming experience.
Finger exercises, small things.
If they're taking you more than 15 minutes, let us know.
They really shouldn't, and they're generally designed to help you learn a single concept, usually a programming concept.
Reading assignments in the textbooks, I've already posted the first reading assignment, and essentially they should provide you a very different take on the same material we're covering in lectures and recitations.
We've tried to choose different examples for lectures and from the textbooks for the most part, so you get to see things in two slightly different ways.
There'll be a final exam based upon all of the above.
All right, prerequisites-- experience writing object-oriented programs in Python, preferably Python 3.5.
Familiarity with concepts of computational complexity.
You'll see even in today's lecture, we'll be assuming that.
Familiarity with some simple algorithms.
If you took 60001 or you took the 60001 advanced standing exam, you'll be fine.
Odds are you'll be fine anyway, but that's the safest way to do it.
So the programming assignments are going to be a bit easier, at least that's what students have reported in the past, because they'll be more focused on the problem to be solved than on the actual programming.
The lecture content, more abstract.
The lectures will be-- and maybe I'm speaking euphemistically-- a bit faster paced.
So hang on to your seats.
And the course is really less about programming and more about dipping your toe into the exotic world of data science.
We do want you to hone your programming skills.
There'll be a few additional bits of Python.
Today, for example, we'll talk about lambda abstraction.
Inevitably, some comments about software engineering, how to structure your code, more emphasis in using packages.
Hopefully it will go a little bit smoother than in the last problem set in 60001.
And finally, it's the old joke about programming that somebody walks up to a taxi driver in New York City and says, "I'm lost.
How do I get to Carnegie Hall?"
The taxi driver turns to the person and says, "practice, practice, practice."
And that's really the only way to learn to program is practice, practice, practice.
The main topic of the course is what I think of as computational models.
How do we use computation to understand the world in which we live?
What is a model?
To me I think of it as an experimental device that can help us to either understand something that has happened, to sort of build a model that explains phenomena we see every day, or a model that will allow us to predict the future, something that hasn't happened.
So you can think of, for example, a climate change model.
We can build models that sort of explain how the climate has changed over the millennia, and then we can build probably a slightly different model that might predict what it will be like in the future.
So essentially what's happening is science is moving out of the wet lab and into the computer.
Increasingly, I'm sure you all see this-- those of you who are science majors-- an increasing reliance on computation rather than traditional experimentation.
As we'll talk about, traditional experimentation is and will remain important, but now it has to really be supplemented by computation.
We'll talk about three kinds of models-- optimization models, statistical models, and simulation models.
So let's talk first about optimization models.
An optimization model is a very simple thing.
We start with an objective function that's either to be maximized or minimized.
So for, example, if I'm going from New York to Boston, I might want to find a route by car or plane or train that minimizes the total travel time.
So my objective function would be the number of minutes spent in transit getting from a to b.
We then often have to layer on top of that objective function a set of constraints, sometimes empty, that we have to obey.
So maybe the fastest way to get from New York to Boston is to take a plane, but I only have $100 to spend.
So that option is off the table.
So I have the constraints there on the amount of money I can spend.
Or maybe I have to be in Boston before 5:00 PM and while the bus would get me there for $15, it won't get me there before 5:00.
And so maybe what I'm left with is driving, something like that.
So objective function, something you're either minimizing or maximizing, and a set of constraints that eliminate some solutions.
And as we'll see, there's an asymmetry here.
We handle these two things differently.
We use these things all the time.
I commute to work using Waze, which essentially is solving-- not very well, I believe-- an optimization problem to minimize my time from home to here.
When you travel, maybe you log into various advisory programs that try and optimize things for you.
They're all over the place.
Today you really can't avoid using optimization algorithm as you get through life.
Pretty abstract.
Let's talk about a specific optimization problem called the knapsack problem.
The first time I talked about the knapsack problem I neglected to show a picture of a knapsack, and I was 10 minutes into it before I realized most of the class had no idea what a knapsack was.
It's what we old people used to call a backpack, and they used to look more like that than they look today.
So the knapsack problem involves-- usually it's told in terms of a burglar who breaks into a house and wants to steal a bunch of stuff but has a knapsack that will only hold a finite amount of stuff that he or she wishes to steal.
And so the burglar has to solve the optimization problem of stealing the stuff with the most value while obeying the constraint that it all has to fit in the knapsack.
So we have an objective function.
I'll get the most for this when I fence it.
And a constraint, it has to fit in my backpack.
And you can guess which of these might be the most valuable items here.
So here is in words, written words what I just said orally.
There's more stuff than you can carry, and you have to choose which stuff to take and which to leave behind.
I should point out that there are two variants of it.
There's the 0/1 knapsack problem and the continuous.
The 0/1 would be illustrated by something like this.
So the 0/1 knapsack problem means you either take the object or you don't.
I take that whole gold bar or I take none of it.
The continuous or so-called fractional knapsack problem says I can take pieces of it.
So maybe if I take in my gold bar and shaved it into gold dust, I then can say, well, the whole thing won't fit in, but I can fit in a path, part of it.
The continuous knapsack problem is really boring.
It's easy to solve.
How do you think you would solve the continuous problem?
Suppose you had over here a pile of gold and a pile of silver and a pile of raisins, and you wanted to maximize your value.
Well, you'd fill up your knapsack with gold until you either ran out of gold or ran out of space.
If you haven't run out of space, you'll now put silver in until you run out of space.
If you still haven't run out of space, well, then you'll take as many raisins as you can fit in.
But you can solve it with what's called a greedy algorithm, and we'll talk much more about this as we go forward.
Where you take the best thing first as long as you can and then you move on to the next thing.
As we'll see, the 0/1 knapsack problem is much more complicated because once you make a decision, it will affect the future decisions.
Let's look at an example, and I should probably warn you, if you're hungry, this is not going to be a fun lecture.
So here is my least favorite because I always want to eat more than I'm supposed to eat.
So the point is typically knapsack problems are not physical knapsacks but some conceptual idea.
So let's say that I'm allowed 1,500 calories of food, and these are my options.
I have to go about deciding, looking at this food-- and it's interesting, again, there's things showing up on your screen that are not showing up on my screen, but they're harmless, things like how my mouse works.
Anyway, so I'm trying to take some fraction of this food, and it can't add up to more than 1,500 calories.
The problem might be that once I take something that's 1,485 calories, I can't take anything else, or maybe 1,200 calories and everything else is more than 300.
So once I take one thing, it constrains possible solutions.
A greedy algorithm, as we'll see, is not guaranteed to give me the best answer.
Let's look at a formalization of it.
So each item is represented by a pair, the value of the item and the weight of the item.
And let's assume the knapsack can accommodate items with the total weight of no more than w. I apologize for the short variable names, but they're easier to fit on a slide.
Finally, we're going to have a vector l of length n representing the set of available items.
This is assuming we have n items to choose from.
So each element of the vector represents an item.
So those are the items we have.
And then another vector v is going to indicate whether or not an item was taken.
So essentially I'm going to use a binary number to represent the set of items I choose to take.
For item three say, if bit three is zero I'm not taking the item.
If bit three is one, then I am taking the item.
So it just shows I can now very nicely represent what I've done by a single vector of zeros and ones.
Let me pause for a second.
Does anyone have any questions about this setup?
It's important to get this setup because what we're going to see now depends upon that setting in your head.
So I've kind of used mathematics to describe the backpack problem.
And that's typically the way we deal with these optimization problems.
We start with some informal description, and then we translate them into a mathematical representation.
So here it is.
We're going to try and find a vector v that maximizes the sum of V sub i times I sub i.
Now, remember I sub i is the value of the item.
V sub i is either zero or one So if I didn't take the item, I'm multiplying its value by zero.
So it contributes nothing to the sum.
If I did take the item, I'm multiplying its value by one.
So the value of the item gets added to the sum.
So that tells me the value of V. And I want to get the most valuable V I can get subject to the constraint that if I look at the item's dot weight and multiply it by V, the sum of the weights is no greater than w. So I'm playing the same trick with the values of multiplying each one by zero or one, and that's my constraint.
Make sense?
All right, so now we have the problem formalized.
How do we solve it?
Well, the most obvious solution is brute force.
I enumerate all possible combinations of items; that is to say, I generate all subsets of the items that are available-- I don't know why it says subjects here, but we should have said items.
Let me fix that.
This is called the power set.
So the power set of a set includes the empty subset.
It includes the set that includes everything and everything in between.
So subsets of size one, subsets of size two, et cetera.
So now I've generated all possible sets of items.
I can now go through and sum up the weights and remove all those sets that weigh more than I'm allowed.
And then from the remaining combinations, choose any one whose value is the largest.
I say choose any one because there could be ties, in which case I don't care which I choose.
So it's pretty obvious that this is going to give you a correct answer.
You're considering all possibilities and choosing a winner.
Unfortunately, it's usually not very practical.
What we see here is that's what the power set is if you have 100 vec.
Not very practical, right, even for a fast computer generating that many possibilities is going to take a rather long time.
So kind of disappointing.
We look at it and say, well, we got a brute force algorithm.
It will solve the problem, but it'll take too long.
We can't actually do it.
100 is a pretty small number, right.
We often end up solving optimization problems where n is something closer to 1,000, sometimes even a million.
Clearly, brute force isn't going to work.
So that raises the next question, are we just being stupid?
Is there a better algorithm that I should have showed you?
I shouldn't say we.
Am I just being stupid?
Is there a better algorithm that would have given us the answer?
The sad answer to that is no for the knapsack problem.
And indeed many optimization problems are inherently exponential.
What that means is there is no algorithm that provides an exact solution to this problem whose worst case running time is not exponential in the number of items.
It is an exponentially hard problem.
There is no really good solution.
But that should not make you sad because while there's no perfect solution, we're going to look at a couple of really very good solutions that will make this poor woman a happier person.
So let's start with the greedy algorithm.
I already talked to you about greedy algorithms.
So it could hardly be simpler.
We say while the knapsack is not full, put the best available item into the knapsack.
When it's full, we're done.
You do need to ask a question.
What does best mean?
Is the best item the most valuable?
Is it the least expensive in terms of, say, the fewest calories, in my case?
Or is it the highest ratio of value to units?
Now, maybe I think a calorie in a glass of beer is worth more than a calorie in a bar of chocolate, maybe vice versa.
Which gets me to a concrete example.
So you're about to sit down to a meal.
You know how much you value the various different foods.
For example, maybe you like donuts more than you like apples.
You have a calorie budget, and here we're going to have a fairly austere budget-- it's only one meal; it's not the whole day-- of 750 calories, and we're going to have to go through menus and choose what to eat.
That is as we've seen a knapsack problem.
They should probably have a knapsack solver at every McDonald's and Burger King.
So here's a menu I just made up of wine, beer, pizza, burger, fries, Coke, apples, and a donut, and the value I might place on each of these and the number of calories that actually are in each of these.
And we're going to build a program that will find an optimal menu.
And if you don't like this menu, you can run the program and change the values to be whatever you like.
Well, as you saw if you took 60001, we like to start with an abstract data type, like to organize our program around data abstractions.
So I've got this class food.
I can initialize things.
I have a getValue, a getCost, density, which is going to be the value divided by the cost, and then a string representation.
So nothing here that you should not all be very familiar with.
Then I'm going to have a function called buildMenu, which will take in a list of names and a list of values of equal length and a list of calories.
They're all the same length.
And it will build the menu.
So it's going to be a menu of tuples-- a menu of foods, rather.
And I build each food by giving it its name, its value, and its caloric content.
Now I have a menu.
Now comes the fun part.
Here is an implementation of a greedy algorithm.
I called it a flexible greedy primarily because of this key function over here.
So you'll notice in red there's a parameter called keyfunction.
That's going to be-- map the elements of items to numbers.
So it will be used to sort the items.
So I want to sort them from best to worst, and this function will be used to tell me what I mean by best.
So maybe keyfunction will just return the value or maybe it will return the weight or maybe it will return some function of the density.
But the idea here is I want to use one greedy algorithm independently of my definition of best.
So I use keyfunction to define what I mean by best.
So I'm going to come in.
I'm going to sort it from best to worst.
And then for i in range len of items sub copy-- I'm being good.
I've copied it.
That's why you sorted rather than sort.
I don't want to have a side effect in the parameter.
In general, it's not good hygiene to do that.
And so for-- I'll go through it in order from best to worst.
And if the value is less than the maximum cost, if putting it in would keep me under the cost or not over the cost, I put it in, and I just do that until I can't put anything else in.
So I might skip a few because I might get to the point where there's only a few calories left, and the next best item is over that budget but maybe further down I'll find one that is not over it and put it in.
That's why I can't exit as soon as I reach-- as soon as I find an item that won't fit.
And then I'll just return.
Does this make sense?
Does anyone have any doubts about whether this algorithm actually works?
I hope not because I think it does work.
Let's ask the next question.
How efficient do we think it is?
What is the efficiency of this algorithm?
Let's see where the time goes.
That's the algorithm we just looked at.
So I deleted the comment, so we'd have a little more room in the slide.
Who wants to make a guess?
By the way, this is the question.
So please go answer the questions.
We'll see how people do.
But we can think about it as well together.
Well, let's see where the time goes.
The first thing is at the sort.
So I'm going to sort all the items.
And we heard from Professor Grimson how long the sort takes.
See who remembers.
Python uses something called timsort, which is a variant of something called quicksort, which has the same worst-case complexity as merge sort.
And so we know that is n log n where n in this case would be the len of items.
So we know we have that.
Then we have a loop.
How many times do we go through this loop?
Well, we go through the loop n times, once for each item because we do end up looking at every item.
And if we know that, what's the order?
[INAUDIBLE]
N log n plus n-- I guess is order n log n, right?
So it's pretty efficient.
And we can do this for big numbers like a million.
Log of a million times a million is not a very big number.
So it's very efficient.
Here's some code that uses greedy.
Takes in the items, the constraint, in this case will be the weight, and just calls greedy, but with the keyfunction and prints what we have.
So we're going to test greedy.
I actually think I used 750 in the code, but we can use 800.
It doesn't matter.
And here's something we haven't seen before.
So used greedy by value to allocate and calls testGreedy with food, maxUnits and Food.getValue.
Notice it's passing the function.
That's why it's not-- no closed parentheses after it.
Used greedy to allocate.
And then we have something pretty interesting.
What's going on with this lambda?
So here we're going to be using greedy by density to allocate-- actually, sorry, this is greedy by cost.
And you'll notice what we're doing is-- we don't want to pass in the cost, right, because we really want the opposite of the cost.
We want to reverse the sort because we want the cheaper items to get chosen first.
The ones that have fewer calories, not the ones that have more calories.
As it happens, when I define cost, I defined it in the obvious way, the total number of calories.
So I could have gone and written another function to do it, but since it was so simple, I decided to do it in line.
So let's talk about lambda and then come back to it.
Lambda is used to create an anonymous function, anonymous in the sense that it has no name.
So you start with the keyword lambda.
You then give it a sequence of identifiers and then some expression.
What lambda does is it builds a function that evaluates that expression on those parameters and returns the result of evaluating the expression.
So instead of writing def, I have inline defined a function.
So if we go back to it here, you can see that what I've done is lambda x one divided by Food.getCost of x.
Notice food is the class name here.
So I'm taking the function getCost from the class food, and I'm passing it the parameter x, which is going to be what?
What's the type of x going to be?
I can wait you out.
What is the type of x have to be for this lambda expression to make sense?
Well, go back to the class food.
What's the type of the argument of getCost?
What's the name of the argument to getCost?
That's an easier question.
We'll go back and we'll look at it.
What's the type of the argument to getCost?
Food
Food.
Thank you.
So I do have-- speaking of food, we do have a tradition in this class that people who answer questions correctly get rewarded with food.
Oh, Napoli would have caught that.
So it has to be of type food because it's self in the class food.
So if we go back to here, this x has to be of type food, right.
And sure enough, when we use it, it will be.
Let's now use it.
I should point out that lambda can be really handy as it is here, and it's possible to write amazing, beautiful, complicated lambda expressions.
And back in the good old days of 6001 people learned to do that.
And then they learned that they shouldn't.
My view on lambda expressions is if I can't fit it in a single line, I just go right def and write a function definition because it's easier to debug.
But for one-liners, lambda is great.
Let's look at using greedy.
So here's this function testGreedy, takes foods and the maximum number of units.
And it's going to go through and it's going to test all three greedy algorithms.
And we just saw that, and then here is the call of it.
And so I picked up some names and the values.
This is just the menu we saw.
I'm going to build the menus, and then I'm going to call testGreedys.
So let's go look at the code that does this.
So here you have it or maybe you don't, because every time I switch applications Windows decides I don't want to show you the screen anyway.
This really shouldn't be necessary.
Keep changes.
Why it keeps forgetting, I don't know.
Anyway, so here's the code.
It's all the code we just looked at.
Now let's run it.
Well, what we see here is that we use greedy by value to allocate 750 calories, and it chooses a burger, the pizza, and the wine for a total of-- a value of 284 happiness points, if you will.
On the other hand, if we use greedy by cost, I get 318 happiness points and a different menu, the apple, the wine, the cola, the beer, and the donut.
I've lost the pizza and the burger.
I guess this is what I signed up for when I put my preferences on.
And here's another solution with 318, apple, wine-- yeah, all right.
So I actually got the same solution, but it just found them in a different order.
Why did it find them in a different order?
Because the sort order was different because in this case I was sorting by density.
From this, we see an important point about greedy algorithms, right, that we used the algorithm and we got different answers.
Why do we have different answers?
The problem is that a greedy algorithm makes a sequence of local optimizations, chooses the locally optimal answer at every point, and that doesn't necessarily add up to a globally optimal answer.
This is often illustrated by showing an example of, say, hill climbing.
So imagine you're in a terrain that looks something like this, and you want to get to the highest point you can get.
So you might choose as a greedy algorithm if you can go up, go up; if you can't go up, you stop.
So whenever you get a choice, you go up.
And so if I start here, I could right in the middle maybe say, all right, it's not up but it's not down either.
So I'll go either left or right.
And let's say I go right, so I come to here.
Then I'll just make my way up to the top of the hill, making a locally optimal decision head up at each point, and I'll get here and I'll say, well, now any place I go takes me to a lower point.
So I don't want to do it, right, because the greedy algorithm says never go backwards.
So I'm here and I'm happy.
On the other hand, if I had gone here for my first step, then my next step up would take me up, up, up, I'd get to here, and I'd stop and say, OK, no way to go but down.
I don't want to go down.
I'm done.
And what I would find is I'm at a local maximum rather than a global maximum.
And that's the problem with greedy algorithms, that you can get stuck at a local optimal point and not get to the best one.
Now, we could ask the question, can I just say don't worry about a density will always get me the best answer?
Well, I've tried a different experiment.
Let's say I'm feeling expansive and I'm going to allow myself 1,000 calories.
Well, here what we see is the winner will be greedy by value, happens to find a better answer, 424 instead of 413.
So there is no way to know in advance.
Sometimes this definition of best might work.
Sometimes that might work.
Sometimes no definition of best will work, and you can't get to a good solution-- you get to a good solution.
You can't get to an optimal solution with a greedy algorithm.
On Wednesday, we'll talk about how do you actually guarantee finding an optimal solution in a better way than brute force.
See you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
We ended the last lecture looking at greedy algorithms.
Today I want to discuss the pros and cons of greedy.
Oh, I should mention-- in response to popular demand, I have put the PowerPoint up, so if you download the ZIP file, you'll find the questions, including question 1, the first question, plus the code, plus the PowerPoint.
We actually do read Piazza, and sometimes, at least, pay attention.
We should pay attention all the time.
So what are the pros and cons of greedy?
The pro-- and it's a big pro-- is that it's really easy to implement, as you could see.
Also enormously important-- it's really fast.
We looked at the complexity last time-- it was m log n-- quite quick.
The downside-- and this can be either a big problem or not a big problem-- is that it doesn't actually solve the problem, in the sense that we've asked ourselves to optimize something.
And we get a solution that may or may not be optimal.
Worse-- we don't even know, in this case, how close to optimal it is.
Maybe it's almost optimal, but maybe it's really far away.
And that's a big problem with many greedy algorithms.
There are some very sophisticated greedy algorithms we won't be looking at that give you a bound on how good the approximation is, but most of them don't do that.
Last time we looked at an alternative to a greedy algorithm that was guaranteed to find the right solution.
It was a brute force algorithm.
The basic idea is simple-- that you enumerate all possible combinations of items, remove the combination whose total units exceed the allowable weight, and then choose the winner from those that are remaining.
Now let's talk about how to implement it.
And the way I want to implement it is using something called a search tree.
There are lots of different ways to implement it.
In the second half of today's lecture, you'll see why I happen to choose this particular approach.
So what is a search tree?
A tree is, basically, a kind of graph.
And we'll hear much more about graphs next week.
But this is a simple form where you have a root and then children of the root.
In this particular form, research C, you have two children.
So we start with the root.
And then we look at our list of elements to be considered that we might take, and we look at the first element in that list.
And then we draw a left branch, which shows the consequence of choosing to take that element, and a right branch, which shows the consequences of not taking that element.
And then we consider the second element, and so on and so forth, until we get to the bottom of the tree.
So by convention, the left element will mean we took it, the right direction will mean we didn't take it.
And then we apply it recursively to the non-leaf children.
The leaf means we get to the end, we've considered the last element to be considered.
Nothing else to think about.
When we get to the code, we'll see that, in addition to the description being recursive, it's convenient to write the code that way, too.
And then finally, we'll choose the node that has the highest value that meets our constraints.
So let's look at an example.
My example is I have my backpack that can hold a certain number of calories if you will.
And I'm choosing between, to keep it small, a beer, a pizza, and a burger-- three essential food groups.
The first thing I explore on the left is take the beer, and then I have the pizza and the burger to continue to consider.
I then say, all right, let's take the pizza.
Now I have just the burger.
Now I taste the burger.
This traversal of this generation of the tree is called left-most depth-most.
So I go all the way down to the bottom of the tree.
I then back up a level and say, all right, I'm now at the bottom.
Let's go back and see what happens if I make the other choice at the one level up the tree.
So I went up and said, well, now let's see what happens if I make a different decision, as in we didn't take the burger.
And then I work my way-- this is called backtracking-- up another level.
I now say, suppose, I didn't take the piece of pizza.
Now I have the beer only and only the burger to think about, so on and so forth, until I've generated the whole tree.
You'll notice it will always be the case that the leftmost leaf of this tree has got all the possible items in it, and the rightmost leaf none.
And then I just check which of these leaves meets the constraint and what are the values.
And if I compute the value and the calories in each one, and if our constraint was 750 calories, then I get to choose the winner, which is-- I guess, it's the pizza and the burger.
Is that right?
The most value under 750.
That's the way I go through.
It's quite a straightforward algorithm.
And I don't know why we draw our trees with the root at the top and the leaves at the bottom.
My only conjecture is computer scientists don't spend enough time outdoors.
Now let's think of the computational complexity of this process.
The time is going to be based on the total number of nodes we generate.
So if we know the number of nodes that are in the tree, we then know the complexity of the algorithm, the asymptotic complexity.
Well, how many levels do we have in the tree?
Just the number of items, right?
Because at each level of the tree we're deciding to take or not to take an item.
And so we can only do that for the number of items we have.
So if we go back, for example, and we look at the tree-- not that tree, that tree-- and we count the number of levels, it's going to be based upon the total number of items.
We know that because if you look at, say, the leftmost node at the bottom, we've made three separate decisions.
So counting the root, it's n plus 1.
But we don't care about plus 1 when we're doing asymptotic complexity.
So that tells us how many levels we have in the tree.
The next question we need to ask is, how many nodes are there at each level?
And you can look at this and see-- the deeper we go, the more nodes we have at each level.
In fact, if we come here, we can see that the number of nodes at level i-- depth i of the tree-- is 2 to the i.
That makes sense if you remember last time we looked at binary numbers.
We're saying we're representing our choices as either 0 or 1 for what we take.
If we have n items to choose from, then the number of possible choices is 2 to the n, the size of the powerset.
So that will tell us the number of nodes at each level.
So if there are n items, the number of nodes in the tree is going to be the sum from 0 to n of 2 to the i because we have that many levels.
And if you've studied a little math, you know that's exactly 2 to the n plus 1.
Or if you do what I do, you look it up in Wikipedia and you know it's 2 to the n plus 1.
Now, there's an obvious optimization.
We don't need to explore the whole tree.
If we get to a point where the backpack is overstuffed, there's no point in saying, should we take this next item?
Because we know we can't.
I generated a bunch of leaves that were useless because the weight was too high.
So you could always abort early and say, oh, no point in generating the rest of this part of the tree because we know everything in it will be too heavy.
Adding something cannot reduce the weight.
It's a nice optimization.
It's one you'll see we actually do in the code.
But it really doesn't change the complexity.
It's not going to change the worst-cost complexity.
Exponential, as we saw this, I think, in Eric's lecture, is a big number.
You don't usually like 2 to the n. Does this mean that brute force is never useful?
Well, let's give it a try.
We'll look at some code.
Here is the implementation.
So it's maxVal, toConsider, and avail.
And then we say, if toConsider is empty or avail is 0-- avail is an index, we're going to go through the list using that to tell us whether or not we still have an element to consider-- then the result will be the tuple 0 and the empty tuple.
We couldn't take anything.
This is the base of our recursion.
Either there's nothing left to consider or there's no available weight-- the Val, as the amount of weight, is 0 or toConsider is empty.
Well, if either of those are true, then we ask whether to consider * 0, the first element to look at.
Is that cost greater than availability?
If it is, we don't need to explore the left branch.
because it means we can't afford to put that thing in the backpack, the knapsack.
There's just no room for it.
So we'll explore the right branch only.
The result will be whatever the maximum value is of toConsider of the remainder of the list-- the list with the first element sliced off-- and availability unchanged.
So it's a recursive implementation, saying, now we only have to consider the right branch of the tree because we knew we couldn't take this element.
It just weighs too much, or costs too much, or was too fattening, in my case.
Otherwise, we now have to consider both branches.
So we'll set next item to toConsider of 0, the first one, and explore the left branch.
On this branch, there are two possibilities to think about, which I'm calling withVal and withToTake.
So I'm going to call maxVal of toConsider of everything except the current element and pass in an available weight of avail minus whatever-- well, let me widen this so we can see the whole code.
This is not going to let me widen this window any more.
Shame on it.
Let me see if I can get rid of the console.
Well, we'll have to do this instead.
So we're going to call maxVal with everything except the current element and give it avail minus the cost of that next item of toConsider sub 0.
Because we know that the availability, available weight has to have that cost subtracted from it.
And then we'll add to withVal next item dot getValue.
So that's a value if we do take it.
Then we'll explore the right branch-- what happens if we don't take it?
And then we'll choose the better branch.
So it's a pretty simple recursive algorithm.
We just go all the way to the bottom and make the right choice at the bottom, and then percolate back up, like so many recursive algorithms.
We have a simple program to test it.
I better start a console now if I'm going to run it.
And we'll testGreedys on foods.
Well, we'll testGreedys and then we'll testMaxVal.
So I'm building the same thing we did in Monday's lecture, the same menu.
And I'll run the same testGreedys we looked at last time.
And we'll see whether or not we get something better when we run the truly optimal one.
Well, indeed we do.
You remember that last time and, fortunately, this time too, the best we did was a value of 318.
But now we see we can actually get to 353 if we use the truly optimal algorithm.
So we see it ran pretty quickly and actually gave us a better answer than we got from the greedy algorithm.
And it's often the case.
If I have time at the end, I'll show you an optimization program you might want to run that works perfectly fine to use this kind of brute force algorithm on.
Let's go back to the PowerPoint.
So I'm just going through the code again we just ran.
This was the header we saw-- toConsider, as the items that correspond to nodes higher up the tree, and avail, as I said, the amount of space.
And again, here's what the body of the code loooked like, I took out the comments.
One of the things you might think about in your head when you look at this code is putting the comments back in.
I always find that for me a really good way to understand code that I didn't write is to try and comment it.
And that helps me sort of force myself to think about what is it really doing.
So you'll have both versions-- you'll have the PowerPoint version without the comments and the actual code with the comments.
You can think about looking at this and then looking at the real code and making sure that you're understanding jibes.
I should point out that this doesn't actually build the search tree.
We've got this local variable result, starting here, that records the best solution found so far.
So it's not the picture I drew where I generate all the nodes and then I inspect them.
I just keep track-- as I generate a node, I say, how good is this?
Is it better than the best I've found so far?
If so, it becomes the new best.
And I can do that because every node I generate is, in some sense, a legal solution to the problem.
Probably rarely is it the final optimal solution but it's at least a legal solution.
And so if it's better than something we saw before, we can make it the new best.
This is very common.
And this is, in fact, what most people do with it when they use a search tree-- they don't actually build the tree in the pictorial way we've looked at it but play some trick like this of just keeping track of their results.
Any questions about this?
All right.
We did just try it on example from lecture 1.
And we saw that it worked great.
It gave us a better answer.
It finished quickly.
But we should not take too much solace from the fact that it finished quickly because 2 to the eighth is actually a pretty tiny number.
Almost any algorithm is fine when I'm working on something this small.
Let's look now at what happens if we have a bigger menu.
Here is some code to do a bigger menu.
Since, as you will discover if you haven't already, I'm a pretty lazy person, I didn't want to write out a menu with a 100 items or even 50 items.
So I wrote some code to generate the menus.
And I used randomness to do that.
This is a Python library we'll be using a lot for the rest of the semester.
It's used any time you want to generate things at random and do many other things.
We'll come back to it a lot.
Here we're just going to use a very small part of it.
To build a large menu of some numItems-- and we're going to give the maximum value and the maximum cost for each item.
We'll assume the minimum is, in this case, 1.
Items will start empty.
And then for i in range number of items, I'm going to call this function random dot randint that takes a range of integers from 1 to, actually in this case, maxVal minus 1, or 1 to maxVal, actually, in this case.
And it just chooses one of them at random.
So when you run this, you don't know what it's going to get.
Random dot randint might return 1, it might return 23, it might return 54.
The only thing you know is it will be an integer.
And then I'm going to build menus ranging from 5 items to 60 items-- buildLargeMenu, the number of items, with maxVal of 90 and a maxCost of 250, pleasure and calories.
And then I'm going to test maxVal on each of these menus.
So building menus of various sizes at random and then just trying to find the optimal value for each of them.
Let's look at the code.
Let's comment this out, we don't need to run that again.
So we'll build a large menu and then we'll try it for a bunch of items and see what we get.
So it's going along.
Trying the menu up to 30 went pretty quickly.
So even 2 to the 30 didn't take too long.
But you might notice it's kind of bogging down, we got 35.
I guess, I could ask the question now-- it was one of the questions I was going to ask as a poll but maybe I won't bother-- how much patience do we have?
When do you think we'll run out of patience and quit?
If you're out of patience, raise your hand.
Well, some of you are way more patient than I am.
So we're going to quit anyway.
We were trying to do 40.
It might have finished 40, 45.
I've never waited long enough to get to 45.
It just is too long.
That raises the question, is it hopeless?
And in theory, yes.
As I mentioned last time, it is an inherently exponential problem.
The answer is-- in practice, no.
Because there's something called dynamic programming, which was invented by a fellow at the RAND Corporation called Richard Bellman, a rather remarkable mathematician/computer scientist.
He wrote a whole book on it, but I'm not sure why because it's not that complicated.
When we talk about dynamic programming, it's a kind of a funny story, at least to me.
I learned it and I didn't know anything about the history of it.
And I've had all sorts of theories about why it was called dynamic programming.
You know how it is, how people try and fit a theory to data.
And then I read a history book about it, and this was Bellman's own description of why he called it dynamic programming.
And it turned out, as you can see, he basically chose a word because it was the description that didn't mean anything.
Because he was doing mathematics, and at the time he was being funded by a part of the Defense Department that didn't approve of mathematics.
And he wanted to conceal that fact.
And indeed at the time, the head of Defense Appropriations in the US Congress didn't much like mathematics.
And he was afraid that he didn't want to have to go and testify and tell people he was doing math.
So he just invented something that no one would know what it meant.
And years of students spent time later trying to figure out what it actually did mean.
Anyway, what's the basic idea?
To understand it I want to temporarily abandon the knapsack problem and look at a much simpler problem-- Fibonacci numbers.
You've seen this already, with cute little bunnies, I think, when you saw it.
N equals 0, n equals 1-- return 1.
Otherwise, fib of n minus 1 plus fib of n minus 2.
And as I think you saw when you first saw it, it takes a long time to run.
Fib of 120, for example, is a very big number.
It's shocking how quickly Fibonacci grows.
So let's think about implementing it.
If we run Fibonacci-- well, maybe we'll just do that.
So here is fib of n, let's just try running it.
And again, we'll test people's patience.
We'll see how long we're letting it run.
I'm going to try for i in the range of 121.
We'll print fib of i.
Comes clumping along.
It slows down pretty quickly.
And if you look at it, it's kind of surprising it's this slow because these numbers aren't that big.
These are not enormous numbers.
Fib of 35 is not a huge number.
Yet it took a long time to compute.
So you have the numbers growing pretty quickly but the computation, actually, seems to be growing faster than the results.
We're at 37.
It's going to gets slower and slower, even though our numbers are not that big.
The question is, what's going on?
Why is it taking so long for Fibonacci to compute these results?
Well, let's call it and look at the question.
And to do that I want to look at the call tree.
This is for Fibonacci of 6, which is only 13, which, I think, most of us would agree was not a very big number.
And let's look what's going on here.
If you look at this, what in some sense seems really stupid about it?
What is it doing that a rational person would not want to do if they could avoid it?
It's bad enough to do something once.
But to do the same thing over and over again is really wasteful.
And if we look at this, we'll see, for example, that fib 4 is being computed here, and fib 4 is being computed here.
Fib 3 is being considered here, and here, and here.
And do you think we'll get a different answer for fib 3 in one place when we get it in the other place?
You sure hope not.
So you think, well, what should we do about this?
How would we go about avoiding doing the same work over and over again?
And there's kind of an obvious answer, and that answer is at the heart of dynamic programming.
What's the answer?
[INAUDIBLE]
Exactly.
And I'm really happy that someone in the front row answered the question because I can throw it that far.
You store the answer and then look it up when you need it.
Because we know that we can look things up very quickly.
Dictionary, despite what Eric said in his lecture, almost all the time works in constant time if you make it big enough, and it usually is in Python.
We'll see later in the term how to do that trick.
So you store it and then you'd never have to compute it again.
And that's the basic trick behind dynamic programming.
And it's something called memoization, as in you create a memo and you store it in the memo.
So we see this here.
Notice that what we're doing is trading time for space.
It takes some space to store the old results, but negligible related to the time we save.
So here's the trick.
We're going to create a table to record what we've done.
And then before computing fib of x, we'll check if the value has already been computed.
If so, we just look it up and return it.
Otherwise, we'll compute it-- it's the first time-- and store it in the table.
Here is a fast implementation of Fibonacci that does that.
It looks like the old one, except it's got an extra argument-- memo-- which is a dictionary.
The first time we call it, the memo will be empty.
It tries to return the value in the memo.
If it's not there, an exception will get raised, we know that.
And it will branch to here, compute the result, and then store it in the memo and return it.
It's the same old recursive thing we did before but with the memo.
Notice, by the way, that I'm using exceptions not as an error handling mechanism, really, but just as a flow of control.
To me, this is cleaner than writing code that says, if this is in the keys, then do this, otherwise, do that.
It's slightly fewer lines of code, and for me, at least, easier to read to use try-except for this sort of thing.
Let's see what happens if we run this one.
Get rid of the slow fib and we'll run fastFib.
Wow.
We're already done with fib 120.
Pretty amazing, considering last time we got stuck around 40.
It really works, this memoization trick.
An enormous difference.
When can you use it?
It's not that memorization is a magic bullet that will solve all problems.
The problems it can solve, it can help with, really, is the right thing.
And by the way, as we'll see, it finds an optimal solution, not an approximation.
Problems have two things called optimal substructure, overlapping subproblems.
What are these mean?
We have optimal substructure when a globally optimal solution can be found by combining optimal solutions to local subproblems.
So for example, when x is greater than 1 we can solve fib x by solving fib x minus 1 and fib x minus 2 and adding those two things together.
So there is optimal substructure-- you solve these two smaller problems independently of each other and then combine the solutions in a fast way.
You also have to have something called overlapping subproblems.
This is why the memo worked.
Finding an optimal solution has to involve solving the same problem multiple times.
Even if you have optimal substructure, if you don't see the same problem more than once-- creating a memo.
Well, it'll work, you can still create the memo.
You'll just never find anything in it when you look things up because you're solving each problem once.
So you have to be solving the same problem multiple times and you have to be able to solve it by combining solutions to smaller problems.
Now, we've seen things with optimal substructure before.
In some sense, merge sort worked that way-- we were combining separate problems.
Did merge sort have overlapping subproblems?
No, because-- well, I guess, it might have if the list had the same element many, many times.
But we would expect, mostly not.
Because each time we're solving a different problem, because we have different lists that we're now sorting and merging.
So it has half of it but not the other.
Dynamic programming will not help us for sorting, cannot be used to improve merge sort.
Oh, well, nothing is a silver bullet.
What about the knapsack problem?
Does it have these two properties?
We can look at it in terms of these pictures.
And it's pretty clear that it does have optimal substructure because we're taking the left branch and the right branch and choosing the winner.
But what about overlapping subproblems?
Are we ever solving, in this case, the same problem-- add two nodes?
Well, do any of these nodes look identical?
In this case, no.
We could write a dynamic programming solution to the knapsack problem-- and we will-- and run it on this example, and we'd get the right answer.
We would get zero speedup.
Because at each node, if you can see, the problems are different.
We have different things in the knapsack or different things to consider.
Never do we have the same contents and the same things left to decide.
So "maybe" was not a bad answer if that was the answer you gave to this question.
But let's look at a different menu.
This menu happens to have two beers in it.
Now, if we look at what happens, do we see two nodes that are solving the same problem?
The answer is what?
Yes or no?
I haven't drawn the whole tree here.
Well, you'll notice the answer is yes.
This node and this node are solving the same problem.
Why is it?
Well, in this node, we took this beer and still had this one to consider.
But in this node, we took that beer but it doesn't matter which beer we took.
We still have a beer in the knapsack and a burger and a slice to consider.
So we got there different ways, by choosing different beers, but we're in the same place.
So in fact, we actually, in this case, do have the same problem to solve more than once.
Now, here I had two things that were the same.
That's not really necessary.
Here's another very small example.
And the point I want to make here is shown by this.
So here I have again drawn a search tree.
And I'm showing you this because, in fact, it's exactly this tree that will be producing in our dynamic programming solution to the knapsack problem.
Each node in the tree starts with what you've taken-- initially, nothing, the empty set.
What's left, the total value, and the remaining calories.
There's some redundancy here, by the way.
If I know what I've taken, I could already always compute the value and what's left.
But this is just so it's easier to see.
And I've numbered the nodes here in the order in which they're get generated.
Now, the thing that I want you to notice is, when we ask whether we're solving the same problem, we don't actually care what we've taken.
We don't even care about the value.
All we care is, how much room we have left in the knapsack and which items we have left to consider.
Because what I take next or what I take remaining really has nothing to do with how much value I already have because I'm trying to maximize the value that's left, independent of previous things done.
Similarly, I don't care why I have a 100 calories left.
Whether I used it up on beers or a burger, doesn't matter.
All that matters is that I just have 100 left.
So we see in a large complicated problem it could easily be a situation where different choices of what to take and what to not take would leave you in a situation where you have the same number of remaining calories.
And therefore you are solving a problem you've already solved.
At each node, we're just given the remaining weight, maximize the value by choosing among the remaining items.
That's all that matters.
And so indeed, you will have overlapping subproblems.
As we see in this tree, for the example we just saw, the box is around a place where we're actually solving the same problem, even though we've made different decisions about what to take, A versus B.
And in fact, we have different amounts of value in the knapsack-- 6 versus 7.
What matters is we still have C and D to consider and we have two units left.
It's a small and easy step.
I'm not going to walk you through the code because it's kind of boring to do so.
How do you modify the maxVal we looked at before to use a memo?
First, you have to add the third argument, which is initially going to be set to the empty dictionary.
The key of the memo will be a tuple-- the items left to be considered and the available weight.
Because the items left to be considered are in a list, we can represent the items left to be considered by how long the list is.
Because we'll start at the front item and just work our way to the end.
And then the function works, essentially, exactly the same way fastFib worked.
I'm not going to run it for you because we're running out of time.
You might want to run it yourself because it is kind of fun to see how really fast it is.
But more interestingly, we can look at this table.
This column is what we would get with the original recursive implementation where we didn't use a memo.
And it was therefore 2 to the length of items.
And as you can see, it gets really big or, as we say at the end, huge.
But the number of calls grows incredibly slowly for the dynamic programming solution.
In the beginning it's worth Oh, well.
But by the time we get to the last number I wrote, we're looking at 43,000 versus some really big number I don't know how to pronounce-- 18 somethings.
Incredible improvement in performance.
And then at the end, it's a number we couldn't fit on the slide, even in tiny font.
And yet, only 703,000 calls.
How can this be?
We know the problem is inherently exponential.
Have we overturned the laws of the universe?
Is dynamic programming a miracle in the liturgical sense?
No.
But the thing I want you to carry away is that computational complexity can be a very subtle notion.
The running time of fastMaxVal is governed by the number of distinct pairs that we might be able to use as keys in the memo-- toConsider and available.
The number of possible values of toConsider is small.
It's bounded by the length of the items.
If I have a 100 items, it's 0, 1, 2, up to a 100.
The possible values of available weight is harder to characterize.
But it's bounded by the number of distinct sums of weights you can get.
If I start with 750 calories left, what are the possibilities?
Well, in fact, in this case, maybe we can take only 750 because we're using with units.
So it's small.
But it's actually smaller than that because it has to do with the combinations of ways I can add up the units I have.
I know this is complicated.
It's not worth my going through the details in the lectures.
It's covered in considerable detail in the assigned reading.
Quickly summarizing lectures 1 and 2, here's what I want you to take away.
Many problems of practical importance can be formulated as optimization problems.
Greedy algorithms often provide an adequate though often not optimal solution.
Even though finding an optimal solution is, in theory, exponentially hard, dynamic programming really often yields great results.
It always gives you a correct result and it's sometimes, in fact, most of the times gives it to you very quickly.
Finally, in the PowerPoint, you'll find an interesting optimization problem having to do with whether or not you should roll over problem that grades into a quiz.
And it's simply a question of solving this optimization problem.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome back.
Over the last couple of lectures, we've been looking at optimization models.
And the idea was how do I find a way to optimize an objective function-- it could be minimize it or maximize it-- relative to a set of constraints?
And we saw, or Professor Guttag showed you, one of the ways that naturally falls out is by looking at trees, decision trees, where you pass your way through a tree trying to figure out how to optimize that model.
So today, we're going to generalize those trees into another whole broad class of models called graph theoretic or graph models.
And we're going to use those to again look at how do we can do optimization on those kinds of models.
Just to remind you, there is a great piece of information in the text.
There's the reading for today.
And these will, of course, be in the slides that you can download.
So let's take a second just to reset again what are we trying to do?
Generally, we're trying to build computational models.
So what does that mean?
The same way we could do a physical experiment, or a social experiment, or model, if you like, a physical system and a social system, to both try and gather data and analyze it or to do predictions.
We want to do the same thing computationally.
We'd like to be able to build models in code that we can then run to predict effects, which we then might test with an actual physical experiment.
And we've seen, for example, how you could take just the informal problem of choosing what to eat and turning it into an optimization problem-- in this case, it was a version of something we called a knapsack problem-- and how you could then use that to find code to solve it.
And you've already seen two different general methods.
You've seen greedy algorithms that just try and do the best thing at each stage.
And you saw dynamic programming as an elegant solution to finding better ways to optimize this.
We're going to now look at broadening the class of models to talk about graphs.
So, obvious question is, what's a graph?
And a graph has two elements, two components.
It has a set of nodes, sometimes called vertices.
Those nodes probably are going to have some information associated with them.
It could be as simple as it's a name.
It could be more complicated.
A node might represent a student record-- the grades.
And a graph might talk about putting together all of the grades for a class.
Associated with that, we can't just-- well, I should say, we could just have nodes, but that's kind of boring.
We want to know what are the connections between the elements in my system?
And so the second thing we're going to have is what we call edges, sometimes called arcs.
And an edge will connect a pair of nodes.
We're going to see two different ways in which we could build graphs using edges.
The first one, the simple one, is an edge is going to be undirected.
And actually, I should show this to you.
So there is the idea of just nodes.
Those nodes, as I said, might have information in them, just labels or names.
They might have other information in them.
When I want to connect them up, the connections could be undirected.
If you want to think of it this way, it goes both ways.
An edge connects two nodes together, and that allows sharing of information between both of them.
In some cases, we're going to see that we actually want to use what we call a directed graph, sometimes called a digraph, in which case the edge has a direction from a source to a destination, or sometimes from a parent to a child.
And in this case, the information can only flow from the source to the child.
Now in the case I've drawn here, it looks like there's only ever a single directed edge between nodes.
I could, in fact, have them going both directions, from source to destination and a separate directed edge coming from the destination back to the source.
And we'll see some examples of that.
But I'm going to have edges.
Final thing is, those edges could just be connections.
But in some cases, we're going to put information on the edges, for example, weights.
The weight might tell me how much effort is it going to take me to go from a source to a destination.
And one of the things you're going to see as I want to think about how do I pass through this graph, finding a path from one place to another, for example, minimizing the cost associated with passing through the edges?
Or how do I simply find a connection between two nodes in this graph?
So graphs, composed of vertices or nodes, they're composed of edges or arcs.
So why might we want them?
Well, we're going to see-- and you can probably already guess-- there are lots of really useful relationships between entities.
I might want to take a European vacation.
After November 8, I might really want to take a European vacation.
So I'd like to know, what are the possible ways by rail I can get from Paris to London?
Well, I could pull out the schedule and look at it.
But you could imagine, I hope, thinking about this as a graph.
The nodes would be cities.
The links would be rail links between them.
And then, one of the things I might like to know is, first of all, can I get from Paris to London?
And then secondly, what's the fastest way to do it or the cheapest way to do it?
So I'd like to explore that.
Second example, as you can see on the list, drug discovery, modeling of complex molecule in terms of the relationships between the pieces inside of it and then asking questions like, what kind of energy would it take to convert this molecule into a different molecule?
And how might I think about that as a graph problem?
Third and obvious one, ancestral relationships, family trees.
In most families, almost all families, they really are trees not graphs.
Hopefully you don't come from a family that has strange loops in them.
But family trees are-- I know, I'm in trouble here today.
Aren't I?
Family trees-- stay with me-- are a great demonstration of relationships because there its directional edges.
Right?
Parents have children.
Those children have children.
And like I say, it comes in a natural way of thinking about traversing things in that tree.
And in fact, trees are a special case of a graph.
You've already seen decision trees in the last lecture.
But basically, a special kind of directed graph is a tree.
And the property of the tree is, as it says there, any pair of nodes are connected, if they are connected, by only a single path.
There are no loops.
There are no ways to go from one node, find a set of things that brings you back to that node.
You can only have a single path to those points.
And Professor Guttag used this, for example, to talk about solving the knapsack problem.
A decision trees is a really nice way of finding that solution.
Now, I drew it this way.
In computer science, we mostly use Australian trees.
They're upside down.
The roots are at the top.
The leaves are at the bottom, because we want to think about starting at the beginning of the tree, which is typically something we call the root and traversing it.
But however you use it, trees are going to be a useful way of actually thinking about representing particular kinds of graphs.
OK.
So, when I talk in a second about how to build graphs, well let's spend just a second about saying, so why are they useful?
And if you think about it, the world is full of lots of networks that are based on relationships that could be captured by a graph.
We use them all the time.
Some of you are using them right now-- computer networks.
You want to send an email message from your machine to your friend at Stanford.
That's going to get routed through a set of links to get there.
So the network set up by a series of routers that pass it along, sending something requires an algorithm that figures out the best way to actually move that around.
There's a great local company started by an MIT professor called Akamai that thinks about how do you move web content around on the web?
Again, it's a nice computer network problem.
I've already talked about this.
We're going to do some other examples.
Transportation networks-- here, if you think about it, obvious thing is make the nodes cities.
Make the edges roads between them.
And now questions are, can I get to San Jose, if you like old songs?
And what's the best way to get to San Jose, even if you don't like old songs?
A network problem-- how do I analyze it?
Financial networks-- moving money around-- easily modeled by a graph.
Traditional networks-- sewer, water, electrical, anything that distributes content, if you like, and the different kind of content in this way around.
You want to model that in terms of how you think about flows in those networks.
How do I maximize distribution of water in an appropriate way, given I've got certain capacities on different pipes, which would mean those edges in the graph would have different weights?
And you get the idea-- political networks, criminal networks, social networks.
One of the things we're going to see with graphs is that they can capture interesting relationships.
So here's an example.
It's from that little web site you can see there.
You're welcome to go look at it.
And this is a graph analyzing The Wizard of Oz.
And what's been done here is the size of the node reflects the number of scenes in which a character shares dialog.
So you can see, obviously Dorothy is the biggest node there.
The edges represent shared dialog, so you can see who talks to whom in this graph.
And then, this group has done another thing, which I'm going to mention.
We're not going to solve today, which is you can also do analysis on the graphs.
And in fact, the color here has done something called a min-flow or max-cut problem, which is it's tried to identify which clusters in the graph tend to have a lot of interactions within that cluster but not very many with other clusters.
And you can kind of see.
There's some nice things here, right, if you can read it.
This is all the people in Kansas.
This is Glenda and the Munchkins in that part of Oz.
There's another little cluster over here that I can't read and a little cluster over there.
And then the big cluster down here.
But you can analyze the graph to pull out pieces on it.
You can also notice, by the way, the book is probably misnamed.
It's called The Wizard of Oz.
But notice, there's the wizard, who actually doesn't have a lot of interaction with the other people in this story.
It's OK, literary choice.
But the graph is representing interactions.
And I could imagine searching that graph to try and figure out things about what goes on in The Wizard of Oz.
OK.
So why are they useful?
We're going to see that not only do graphs capture relationships in these connected networks, but they're going to support inference.
They're going to be able to reason about them.
And I want to set that up.
And then we'll actually look at how might we build a graph.
And so here are some ways in which I might want to do inference.
Given a graph, I might say, is there a sequence of edges, of links, between two elements?
Is there a way to get from A to B?
What are the sequence of edges I would use to get there?
A more interesting question is, can I find the least expensive path, also known as the shortest path?
If I want to get from Paris to London, I might like to do it in the least amount of time.
What are the set of choices I want to make to get there?
A third graph problem used a lot is called the graph partition problem.
Everything I've shown so far-- actually not quite.
The first example didn't have it.
You might think of all the nodes having some connection to every other node.
But that may not be true.
There may actually be graphs where I've got a set of connected elements and another component with no connections between them.
Can I find those?
That's called the graph partition problem.
How do I separate the graph out into connected sets of elements?
And then the one that we just showed called the min-cut max-flow problem, is is there an efficient way to separate out the highly connected elements, the things that interact a lot, and separate out how many of those kinds of subgraphs, if you like, are there inside of my graph?
All right, let me show you a motivation for graphs.
And then we'll build them.
I use graph theory everyday.
I'm a math nut.
It's OK, but I use graph theory everyday.
You may as well, if you commute.
Because I use it to figure out how to get from my home in Lexington down here to Cambridge.
And I use a nice little system called Waze It's a great way of doing this, which does graph theory inside of it.
So how do I get to my office?
Well, I'm going to model the road system using a directed graph, a digraph.
Directed graph because streets can be one way.
And so I may only have a single direction there.
And the idea is, I'm going to simply let my nodes or my vertices be points where I have intersections.
They're places where I can make a choice or places where I have terminals, things I'm going to end up in.
The edges would just be the connections between points, the roads on which I can drive.
Some Boston drivers have a different kind of digraph in which they don't care whether that road is drivable or not.
They just go on it.
You may have seen some of these.
But I want to keep my graphs as real roads that I can drive on.
And I'm not going to go against the "One Way" sign.
Each edge will have a weight.
Here I actually have some choices.
All right, the obvious one, the one that Waze probably uses, is something like what's the expected time between a source and a destination node?
How long do I expect it to take me to get from this point to that?
And then, as you can see, I'm going to try and find overall what's the best way to get around it.
You could pick just distance.
What's the distance between the two?
And while there there's a relationship here, it's not direct because it will depend on traffic on it.
Or you could take something even funkier like what's the average speed of travel between the source and destination node?
And once I've got the graph, then I'm going to solve an optimization problem.
What's the shortest weight between my house and my office that gets me into work?
You can make a choice here.
As I said, a commercial system like Waze uses this one.
My wife and I actually have arguments about commuting because she's a firm believer in the second one, just shortest distance.
I actually like the third one because I get anxious when I'm driving.
And so as long as I feel like I'm making progress, I like it.
So even though I may be serpentining all the way through the back roads of Cambridge, if I'm driving fast, I feel like I'm getting there.
So I like optimizing this bottom one down there.
And if you see me on the road, you'll know why I say that, and then get out of the way.
Thinking about navigation through systems actually gives us a little bit of history because, in fact, the very first reported use of graph theory was exactly this problem.
Early 1700s, it's called the Bridges of Koenigsberg.
Koenigsberg is a city that has a set of islands and rivers in it.
There are seven bridges that connect up those islands.
And the question that was posed is, is it possible to take a walk that traverses each of the seven bridges exactly once?
So could you take a walk where you go over each bridge exactly once?
I'm showing you this because it lets us think about how to in fact capture things in a model.
This problem was solved by a great Swiss mathematician, Leonhard Euler.
And here's what he said.
Make each island a node.
Each bridge is just an undirected edge.
And notice in doing that, he's abstracted away irrelevant details.
You don't care what the size of the island is.
You don't care how long the bridges are.
You simply want to think about what are the connections here?
And then you can ask a question.
In this graph, is it possible to find a way to walk through it so that you go through each edge exactly once?
And as Euler showed, the answer is no.
And if you're curious, go look it up on Wikipedia.
There's a nice, elegant solution to why that's the case.
But here's what we're going to do.
We're going to use those graphs to think about these kinds of problems.
And in fact, the example I'm going to show you are going to be shortest path problems.
So with that, let's turn to actually building a graph and then thinking about how we're going to use it.
So we're going to start by constructing graphs.
And then what we're going to do is show how we can build search algorithms on top of those graphs.
And I hope that that flicker is going to go away here soon.
Here we go.
So to build a graph-- actually, I shouldn't have put this slide up so fast.
I've got lots of choices here.
If I'm thinking about maps, one way to build a graph would really to just be build something with latitude and longitude on it.
But as we've already seen, we'd like to extract things away from the graphs.
And so a natural choice is to say, let's represent the nodes in the graph just as objects.
I'm going to use classes for these.
So here's my definition of a node.
It's pretty straightforward.
I'm going to assume that the only information for now I store in a node is just a name, which I'm going to assume is a string.
So I've got a class definition for node.
It inherits from the base Python object class.
I need ways to create instances of nodes, so I've got an init function.
And I'm simply going to store inside each instance, in other words, inside of self, under the variable name, whatever I passed in as the name of that node.
Of course, if I've got ways to create things with a name, I need to get them back out.
So I've got a way of selecting it back out.
If I ask an instance of a node, what's your name?
By calling getName it will return that value.
And to print things out, I'm just going to print out the name.
This is pretty straightforward.
And this, of course, lets me now create as many nodes as I would like.
Edges?
Well, an edge connects up two nodes.
So again, I can do a fairly straightforward construction of a class.
Again, it's going to inherit from the base Python object.
To create an instance of an edge, I'm going to make an assumption, an important one which we're going to come back to.
And the assumption is that the arguments passed in, source and destination, are nodes-- not names-- the nodes themselves, the actual instances of the object class.
And what will I do?
Inside of the edge, I'm going to set internal variables.
For each instance of the edge, source and destination are going to point to those nodes, to those objects that I created out of the node class.
Next two things are straightforward.
I can get those things back out.
And then the final piece is, if when I want to print out what an edge looks like, I'm going to ask that it print out the name of the source, and then an arrow, and then the name of the destination.
So notice what I do there.
Given an instance of an edge, I can print it.
And it will get the source or the node associated with source inside this instance, get for that the getName method, and then call it.
Notice the open-close paren there to actually call it.
What does that do?
It says, inside the edge I've got something that points to a node.
I get that node.
I take the method associated with it.
And I call it.
That returns the string.
And then I glue that together with the arrow.
I do the same thing on the destination.
And I just print it out.
Pretty straightforward, hopefully.
OK, now I have to make a decision about the graph.
I'm going to start with digraphs, directed graphs.
And I need to think about how I might represent the graph.
I can create nodes.
I can create edges, but I've got to bring them all together.
So I'll remind you, a digraph is a directed graph.
The edges pass in only one direction.
And here's one way I could do it.
Given all the sources and all the destinations, I could just create a big matrix called an adjacency matrix.
The rows would be all the sources.
The columns would be all the destinations.
And then in a particular spot in the matrix, if there is an edge between a source and a destination, I'd just put a one.
Otherwise I'd put a zero.
Note, by the way, because it's a directed graph, it's not symmetric.
There might be a one between S and D, but not between D and S, unless there are edges both ways.
This would be a perfectly reasonable way to represent a graph, but not the most convenient one.
I'd have to go into the matrix to look things up.
It may also not be a very efficient way of representing things.
For example, if there are very few edges in the graph, I could have a huge matrix with mostly zeros.
And that's not the most effective way to do it.
So I'm going to use an alternative called an adjacency list.
And the idea here is, for every node in the graph.
I'm going to associate with it a list of destinations.
That is, for a node, what are the places I can reach with a single edge?
OK, so let's see what that does if we want to build it.
And yes, there's a lot of code here, but it's pretty easy to look through I hope.
Here's the choice I'm going to make.
Again, what's a graph?
It's a set of nodes.
It's a set of edges.
I'm going to have a way of putting nodes into the graph.
And I'm going to choose to, when I put a node into the graph, to store it as a key in a dictionary.
OK?
When I initialize the graph, I'm just going to set this internal variable, edges, to be an empty dictionary.
And the second part of it is, when I add an edge to the graph between two nodes from a source to a destination, I'm going to take that point in the dictionary associated with the source.
It's a key.
And associated with it, I'm going to just have a list of the nodes I can reach from edges from that source.
So notice what happens here.
If I want to add a node, remember, it's a node not an edge-- I'll first check to make sure that it's not already in the dictionary.
That little loop is basic, or that if is saying, if it's in this set of keys, it will return true.
And I'm going to complain.
I'm trying to copy a node or duplicate a node.
Otherwise, notice what I do.
When I put a node into the dictionary, I go into that dictionary, edges.
I create an entry with the key that is the node.
And the value I put in there is initially an empty list.
I'm going to say one more piece carefully.
It's a node not a name.
And that's OK in Python.
It is literally the key is the node itself.
It's an object, which is what I'd like.
All right, what if I want to add an edge?
Well, an edge is going to go from a source to a destination node.
So, I'm going to get out from the edge the source piece.
I'm going to get out from the edge the destination piece by calling those methods.
Again, notice the open-close paren, which takes the method and actually calls it.
Because remember, an edge was an object itself.
Given those, I'll check to make sure that they are both in the dictionary.
That is, I've already added them to the graph.
I can't make a connection between things that aren't in the graph.
And then notice the nice little thing I do.
Presuming I have both of them in the dictionary, I take the dictionary, I index into it with the source node.
That gives me a key into the dictionary.
I pull out the entry at that point, which is a list, because I created them up here.
And I add the destination node with append into the list, stick it back in.
So this now captures what I said I wanted to do.
The nodes are represented as keys in the dictionary.
And the edges are represented by destinations as values in the list associated with the key.
So you can see, if I want to see is there an edge between a source and a destination, I would look at our source in the dictionary, and then check in the list to see if the destination is there.
OK, the rest of this then follows pretty straightforwardly.
If I want to get all the children of a particular node, I just go into the dictionary, edges, and look up the value associated with that node.
It gives me back the list.
I've got all the things I can reach from that particular node.
If I want to know if a node is in the graph, I just search over the keys of the dictionary.
They'll either return true or false.
If I want to get a node by its name, which is going to be probably more convenient than trying to keep track of all the nodes, well I could pass in a name as a string.
And what will I do?
I'll just search over all the keys in the dictionary, using the getName method associated with it-- there's the call-- then checking to see if it's the thing I'm looking for.
And if it is, I'll return M. I'll return the node itself.
What about this thing here?
It might bother you a little bit.
Wait a minute.
That raise, isn't it always going to throw an error?
No, because I'm going to go through this loop first.
And if I actually find a node, that return is going to pop me out of the call and return the node.
So I'll only ever get to this if in fact I couldn't find anything here.
And so it's an appropriate way to simply raise the error to say, if I get to this point, couldn't find it, raise an error to say the node's not there.
The last piece looks a little funky, Although you may have seen this.
I like to print out information about a graph.
And I made a choice, which is, I'm going to print out all of the links in the graph.
So I'm going to set up a string initially here that's empty.
And then I'm going to loop over every key in the dictionary, every node in the graph.
And for each one, I'm going to look at all the destinations.
So notice, I take the dictionary, I look up the things at that point.
That's a list.
I loop over that.
And I'm just going to add in to result, the name of the source, an arrow, and the name of the destination followed by a carriage return.
I'll show you an example in a second.
But I'm simply walking down the graph, saying for each source, what can it reach?
I'll print them all out.
And then I'll return everything but the last element.
I'm going to throw away the last carriage return because I don't really need it.
So let me show you an example here, trusting that my Python has come up the way I wanted it to.
So I'm going to load that in, ignore that for the moment.
And I'm going to set g to-- I've got something we're going to come back to in a second that actually creates a graph.
And if I print out g, it prints out, in this case, all of the links from source to destination, each one on a new line.
OK.
So I can create the graphs.
That was digraphs.
Suppose I actually want to get a graph.
Well, I'm going to make it as a subclass of digraph.
And in particular, the only thing I'm going to do is I'm going to shadow the addEdge method of digraphs.
So if you think about it, it's so I make a graph.
If I ask it to add edges, it's going to use this version of addEdge.
And what am I going to do?
I know in a graph, I could have both directions work.
So, given an edge that I want to add into this graph, I'll use the method from the digraph class.
And I'll add an edge going from source to destination.
And then I'll just create an edge the other direction.
Destination becomes source.
Source becomes destination.
And I'll add that into the graph.
Nice and easy, straightforward to do.
And this is kind of nice because, in a graph, I don't have any directionality associated with the edge.
I can go in either direction.
I just created something like that.
And you might say, well, wait a minute.
Why did I pick graph to be a subclass of digraph?
Why not the other way around?
Reasonable question, and you actually know the answer.
You've seen this before.
One of the things I'd like to have is the property that if the client code works correctly using an instance of the bigger type, it should also work correctly when it is using an instance of the subtype substituted in, which is another way of saying anything that works for a digraph will also work for a graph, but not vice versa.
And as a consequence, it's easier to make the graph a subclass of digraph.
Notice the other thing that's nice here.
One little piece of code, just change what it means to make an edge.
Everything else still holds.
And also notice-- you've seen this before-- how we nicely inherit the method from the subclass by explicitly calling it.
It says, from the digraph class, get out the addEdge method and apply it.
OK.
So we can build graphs.
We're going to do that in a second.
Let's turn now to thinking about I'd like to search on a graph.
And I'm going to start with the classic graph optimization problem.
I'd like to find the best path home.
So, what's the shortest path from one node to another?
And that shortest path initially will just be the shortest sequence of steps.
I hope I'm not having a little attack here.
You just saw that screen blank out, right?
The shortest path of steps with the property that the source of the first edge is the starting point.
The destination of the last edge is the thing I'm trying to get to.
And for any edge in between, if I go in my first edge from source to say node one, the next edge has that destination as its source.
So there's simply a chain that says can go from here to here to here to here to get all the way through.
And I'd like to find what's the shortest number of steps?
Edges like that that will get me from source to destination.
Ultimately, if those edges have weights on them, the optimization problem I'd like to solve is, what's the shortest weighted path, the shortest amount of work I have to do to get to those places?
And if we can solve one, we'll see that we can solve the other one pretty straightforwardly.
And we've already seen examples of shortest path problems.
Clearly, finding a route navigation is one.
Designing communication networks is another great example of a shortest path problem.
You'd like your message to get to your friend as quickly as possible and not go as many times around the world before it gets there.
So what's the shortest amount of time or the fewest links I have to use to get there?
Lots of nice biological problems that also captured this piece.
So here is an example.
And we're going to use this to look at two different kinds of algorithms to solve this problem.
This is a little navigation problem from a set of cities.
Think of it as flight paths.
If you're from Arizona, my apologies.
But once you get to Phoenix, you can't get out of there unless you grow from the ashes, I guess.
[LAUGHTER] But you know, it's a way of dealing with how to get around in places.
And to think about this, here's the representation that we'd have in the graph.
The adjacency graph here-- or adjacency list here is, from Boston, I can get to Providence.
I can get to New York.
From Providence, I can get to Boston.
I can get to New York.
From New York, I can only get to Chicago.
Chicago, I can go to Denver or Phoenix.
Denver, I can go to Phoenix or New York.
And from L.A., you can only come back to Boston.
And Phoenix has no exits out of it.
So there is that representation.
I just want to let you see that.
Right?
There are the keys in the dictionary.
They're all the nodes.
And there, each one of those lists is a set of edges from the source to the destination.
OK. How would I build this?
Well this is the code I just ran.
I just want to show it to you.
I notice, by the way, in the slides I distributed earlier, the return g is missing there.
If you want to correct it, I'll repost it later on.
I'm going to create a little function that's going to build a city graph.
I'm going to pass in a type of graph, which I will then call to create it.
So I could make this as a digraph.
I could make it as a graph.
I'm going to start off with it as a digraph.
And then notice what I do here.
I just run over a little loop with a set of names, creating a node with that name and then adding it into the graph.
All right, so node is a class instance.
It creates-- or a class definition-- it creates an instance.
And once I've got that, addNode as a method on the graph.
It will simply add it in.
And then this set here, is simply adding in the edges.
And I can do that.
I'm capturing what I had on that previous slide.
And on a given name to getNode, it will get out the actual node.
And I use that coming out of the graph g. I do the same thing with the getNode from graph g for Providence.
And then I make an edge out of that.
And then I use the method from the graph to add the edge.
If this looks like a lot of code, yeah, it's a lot of words.
But it's pretty straightforward.
I'm literally creating nodes with the names, using the appropriate methods, creating an edge, adding it into the graph.
And when I'm done, I'm just going to return the graph g. OK. Now I want to find the shortest path.
I'm going to show you two techniques for doing this.
The first one is called depth first search.
It's similar to something Professor Guttag showed you when you sort of took the left most depth first method in terms of a search tree.
The one trick here is, because I've got graphs not trees, there are the potential for loops.
So I'm simply going to keep track of what's in the path.
And I'm never going to go back to a node that's already in the path.
So I don't just run in circles going from New York to Boston to New York to Boston constantly.
All right.
So, the second thing I'm going to do here is I'm going to take advantage of a problem you've seen before, which is this is literally a version of divide and conquer.
What does that mean?
If I want to find a path from a source node to destination node, if I can find a path to some intermediate node from source intermediate, and then I find a path from intermediate to destination, the combination is obviously a path the entire way.
So recursively, I can just break this down into simpler and simpler versions of that search problem.
So here's the idea behind depth first search.
Start off with that source node, that initial node.
I'm going to look at all the edges that leave that node in some order, however order it was put into the system.
And I'm going to follow the first edge.
I'll check to see if I'm at the right location.
If I am, I'm done.
If I'm not, I'm going to follow the first edge out of that node.
So I'm actually creating a little loop here.
And I'm going to keep doing that until I either find the goal node or I run out of options.
So let me show you an example.
I've got a little search tree here, a very simple one.
Here's my source.
There is my destination.
In depth first, I'm going to start at the source and go down the first path.
See if I'm at the right place.
I'm not.
So I'm going to take the first path out of here, which might be that one.
See if I'm in the right place.
Actually, let me not do it that way.
Let me do it this way.
Am I in the right place?
I'm not.
So I'm going to take the first path out of this one, which gets me there.
I'm still not in the right place, so I'm going to take the first path out of that one.
And you can see why it's called depth first.
I'm going as deep, if you like, in this graph as I can, from here, to there, to there, to there, to there.
At this stage, I'm stuck.
There is no place to go to, so I'm going to go back to this node and say, is there another edge?
In this case there isn't, so I'll go back to here.
There's not another edge.
Go back to here.
There is another edge.
So I'm going to go this direction.
And from here, I'll look down there.
OK, notice I'm now going depth first down the next chain.
There's nothing from here.
I backtrack.
There's nothing from there.
I backtrack over to here.
There's no additional choices there, so go all the way back to here to follow that one.
And then we'll go down this one again, backtrack, backtrack, and eventually I find the thing I'm looking for.
Depth first-- following my way down this path.
So let's write the code for-- yes ma'am?
Pardon me.
Is the choice of depth first node we go down, is that random?
The question is, which node do I, or which edge do I choose?
It's however I stored it in the system.
So since it's a list, I'm going to just make that choice.
I could have other ways of deciding it.
But think of it as, yeah, essentially random, which one I would pick.
OK, let's look at the code.
Don't panic.
It's not as bad as it looks.
It actually just captures that idea.
Ignore for the moment this down here.
It's just going to set it up.
Depth first search, I'm going to give it a graph, a start node, an end node, and a path that got me to that start node, which initially is just going to be an empty list, something that tells me what's the shortest path I've found so far, which would be my best solution?
And then just a little flag here if I want to print out things along the way.
What do I do?
I set up path to add in the start node.
So if path initially is an empty list, the first time around is just, here's the node I'm at.
I print out some stuff and then I say, see if I'm done.
I'm just going to stay at home.
I'm not going to go anywhere.
Unlikely to happen, but you'll see recursively why this is going to be nice.
If I'm not done, then notice the loop.
I'm going to loop over all the children of the start node.
Those are the edges I can reach.
Then those I can reach with a single edge.
I pick the first one.
And in answer to the question, in this case, it would be the order in which I started in the list.
I just pick that one up.
I then say, let's make sure it's not already in the path because I want to avoid loops.
And assuming it isn't, and assuming I don't yet have a solution, or the best solution I have is smaller than what I've done so far, oh, cool, just do the same search.
So notice, there's that nice recursion.
Right?
I'm going to explore.
I just picked the first option out of that first node.
And the first thing I do is try and see if there's a path from that node using the same thing.
So it's literally like I picked this one.
I don't care about those other edges.
I'm going to try and take this search down.
When it comes back with a solution, as long as there is a solution, I'll say that's my best solution so far.
And then I go back around.
Now this last little piece here is just, if in fact the node's already in the path, I'm just going to print something that says don't keep doing it because you don't need to keep going on.
And I'm going to do that loop, taking all the paths down until it comes back.
And only at that stage do I go to the next portion around this loop.
The piece down here just sets this up, calling it with an initial empty list for path and no solution for shortest.
So it's just a nice way of putting a wrap around it that gets things started up.
This may look a little funky.
It may look a little bit twisted.
So let's see if it actually does what we'd expect it to.
And to do that I'm just going to be a little test function.
I'm going to build that city graph I'm just going to call "Shortest Path."
I'm going to print it out.
And I'd like to see, is there a way to get from Boston to Chicago?
So let's go back over to my Python and try that out.
And I've got a call for that.
Oh, and it prints out.
I start off-- oh, so I did it the wrong way.
It's from Chicago to Boston.
Yes, Chicago to Denver to Phoenix, from Denver to New York, it comes back and says, I've already visited.
Basically concludes I can't get from Chicago to Boston.
It's just printing out each stage.
Let's actually look at that a little more carefully to see how it got there.
So there's my example.
There is the adjacency list.
And here's what happens.
I start off in Chicago.
So that's my first node.
From Chicago, the first edge goes to Denver.
Denver is not what I'm looking for.
But since I am in Denver, recursively I'm going to call it again.
So the first edge out of there is to Phoenix.
Again, sorry if you're from Arizona and Phoenix.
There's nowhere to go.
So I'm going to have to backtrack.
And that will take me back up to Denver.
And I look at the next edge.
It takes me to New York.
From New York I'd like to go to Chicago.
But oh, that's nice because, remember, that first check it says, is Chicago already in the path?
It is.
I don't want to loop, because otherwise I'm simply going to go around and around and around here.
And it may be good for frequent flyer miles, but it's not a great way to get to where you're trying to go.
So I break out of it.
And now, what else do I have left?
Chicago to Denver I've explored.
I'll look at Chicago to Phoenix.
From Phoenix there's nowhere to go.
I go back up to Chicago.
There are no more paths.
I'm done.
OK. Now, it turns out you can actually get somewhere in this graph.
So here's just another example.
I'm simply going to show you, if I want to go from Boston to Phoenix, notice the set of stages.
And you can see, notice how at each stage it tends to be growing.
That's that depth first.
I'm exploring the edges.
I find a path.
That's great.
But is it the shortest path?
I don't know.
So having found that path, I try and take the next branch, which finds a loop.
And I keep moving through this, finding paths until I look at all the possible paths and I actually return the shortest path.
You can try running the code on it.
But what I want you to see is, again, this idea that I can explore it.
But in fact, I'm going to have to explore it in a particular order.
But there is depth first search.
It will find a solution for me.
Alternative, it's what's called breadth first search.
Sounds almost the same.
Again, I'm going to start off with initial load.
I'm going to look at all the edges that leave that node, in some order.
I'm going to follow the first edge as before and see if I'm at the right place.
If I'm not, I'm going to follow the next edge and do the same thing.
So whereas this went down through the tree as deeply as it could of the graph, in breadth first, I'm going to start off taking that edge as before.
I'm not done.
I'm going to keep track of that in case I want to explore more of it.
But I'm going to go back over here and follow that edge.
I'm not done.
Again, I'll keep track of that, but I'll come back up here and explore that one.
And oh, cool, I found a solution in three steps.
I've reached the destination.
And notice, because I'm exploring all the paths of length one before I get to paths of length two.
Once I find a solution, I can stop because I know it's the shortest path.
Any other path through here would be longer than that particular solution.
So the loop here is a little different.
I'm looking over all the paths of length one.
There are all the paths of length two.
And the one thing I'm going to have to do is I'm going to have to keep track of the remaining options here in case I have to come down to them.
Because if I didn't find it at the first level, then I come down here and look at things of length two.
OK?
So let's build that code.
Breadth first search, or BFS, again, a graph, a start, and an end node, something that would just print things out as I go along.
My initial path is just the start point.
But now I've got to keep track of what are the paths that I have yet to explore?
And so for that, I'm going to create something called a queue.
And a queue is going to be a list of paths.
Remember, a path is a list of nodes.
A queue is going to be a list of paths.
So the initial queue is just where I've started.
And then, as long as I've got something still to explore and I haven't found a solution, I'm going to pop off the queue the oldest element, the thing at the beginning.
That's my temporary path.
I'll print out some information about it.
And then I'll grab the last element of that path.
That's the last point in that path.
And I'll now explore.
Is it the thing I'm looking for?
In which case I'm done.
I'll return the path.
Otherwise, for each node that you can reach from that point, create a new path by adding that on the end of this path and add it into the queue at the end of the queue.
So I'm going to keep looping around here until I either find a solution here, which I'll return.
And if I get through all of it, I'm going to return none.
And right there, there is that nice thing where once I find a solution, I know it's the shortest thing, I can stop.
OK, let's look at an example of this.
So I'm going to go back over to Python, where I've got a version of this.
I'm going to comment that out.
And down here in breadth first search, I've actually added a little piece of code that I don't have in the handout that's going to print out the queue as well so we can see what happens when we call this.
So let's take a look at it.
My initial call, there's one thing in the queue.
It's just Boston.
I started in Boston.
So the current path is to start in Boston.
I take that element off the queue, and I say what are the things I can reach from Boston?
Oh, nice, I put two things in.
I can get from Boston to Providence.
I can get from Boston to New York.
The top thing is gone off the queue.
I popped it.
I've replaced it with two things.
Or I take this, and say, OK, from Boston to Providence, where can I get from Providence?
Oh, I can get to New York.
So I put that in the queue.
This has gone off.
That one is still there.
And I do that because I haven't yet reached the thing I'm looking for, which was, I think, Phoenix I was trying to get to.
And you could see at each stage, I'm taking the top thing off the queue, and asking for all the things that I can get to, and adding them to it.
And notice, in some cases, it may be more than one.
For example, which one do I want here?
Right here, if I take Boston, New York to Chicago, from Chicago I can get to Denver.
So there's one new path.
I can also get to Phoenix.
There's a second new path.
Also notice how they are only growing slowly as I build them out.
And in fact, if we go back, we can see that nicely by looking at what happens if we were to actually trace this along.
So Boston to Phoenix, I start at Boston.
Then I look at that and then that.
Those are all the paths of length one.
Having exhausted those, oh nice, I'm looking at paths of length two, and then paths of length three, and then paths the length four, until I found the one that I wanted.
And here's one other way of looking at it.
Breadth first says, I'll look at each path of length one.
And then, oh yes, I avoid the loop.
I look at each path of length two, then paths of length three, until I actually find the solution.
Subtle difference, different performance.
Depth first, I'm always following the next available edge until I get stuck and I backtrack.
Breadth first, I'm always exploring the next equal length option.
And I just have to keep track in that queue of the things I have left to do as I walk my way through.
What about weighted shortest path?
Well, as the mathematicians say, we leave this is an easy exercise for the reader.
It's a little unfair.
The idea would be, imagine on my edges, it's not just a step, but I have a weight.
Flying to L.A. Is a little longer than flying from Boston to New York.
What I'd like to do is do the same kind of optimization, but now just minimizing the sum of the weights on the edges, not the number of edges.
As you might guess, depth first search is easily modified to do this.
The cost now would simply be what's the sum of those weights?
And again, I would have to search all possible options till I find a solution.
Unfortunately, breadth first search can't easily be modified because the short weighted path may have many more than the minimum number of loops.
And I'd have to think about how to adjust it to make that happen.
But to pull it together, here's a new model-- graphs.
Great way of representing networks, collections of entities with relationships between them.
There are lots of nice graph optimization problems.
And we've just shown you two examples of that.
But we'll come back to more examples as we go along.
And with that, we'll see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to move on to a fairly different world than the world we've been living in.
And this will be a world we'll be living in for quite a few lectures.
But before I do that, I want to get back to just finish up something that Professor Grimson started.
You may recall he talked about family trees and raised the question, was it actually possible to represent all ancestral relationships as a tree?
Well, as a counterexample, I'm sure some of you are familiar with Oedipus Rex.
For those of you who are not, I'm happy give you a plot summary at the end of the lecture.
It's a rather bizarre plot.
But it was captured in a wonderful song by Tom Lehrer.
The short story is Oedipus ended up marrying his mother and having four children.
And Tom Lehrer, if you've never heard of Tom Lehrer, you're missing one of the world's funniest songwriters.
And he had a wonderful song called "Oedipus Rex," and I recommend this YouTube as a way to go and listen to it.
And you can gather from the quote what the story is about.
I also recommend the play, by the way.
It's really kind of appalling what goes on, but it's beautiful.
Back to the main topic, here's the relevant reading-- a small bit from later in the book and then chapter 14.
You may notice that we're not actually going through the book in order.
And the reason we're not doing that is because we're trying to get you information you need in time to do problem sets.
So the topic of today is really uncertainty and the fact that the world is really annoyingly hard to understand.
This is a signpost related to 6.0002, but we won't go into too much detail about it.
We'd rather things were certain.
But in fact, they usually are not.
And this is a place where 6.0002 diverges from the typical introductory computer science course, which focuses on things that are functional-- given an input, you always get the same output.
It's predictable.
And we like to do that, because that's easier to teach.
But in fact, for reasons we'll be talking about, it's not nearly as useful if you're trying to actually write computations that help you understand the world.
You have to face uncertainty head on.
An analogy is for many years people, believed in Newtonian mechanics-- I guess they still do in 8.01 maybe-- that every effect has a cause.
An apple falls from the tree because of gravity, and you know where it's going to land.
And the world can be understood causally.
And people believed this really for quite a long time, most of history, until the early part of the 20th century, when the so-called Copenhagen doctrine was put forth.
The doctrine there from Bohr and Heisenberg, two very famous physicists, was one of what they called causal nondeterminism.
And their assertion was that the world at its very most fundamental level behaves in a way that you cannot predict.
It's OK to make a statement that x is highly likely to occur, almost certain to occur, but for no case can you make a statement x will occur.
Nothing has a probability of one.
This was hard for us to imagine today, when we all know quantum mechanics.
But at the turn of the century, this was a shocking statement.
And two other very well-known physicists, Albert Einstein and Schrodinger, basically said, no, this is wrong.
Bohr, Heisenberg, you guys are idiots.
It's just not true.
They probably didn't call them idiots.
And this is most exemplified by Einstein's famous quote that "God does not play dice," which is indicative of the fact that this was actually a discussion that permeated not just the world of physics, but society in general people really turned it into literally a religious issue, as did Einstein.
Well, so now we should ask the question, does it really matter?
And to illustrate that, I need two coins.
I forgot to bring any coins with me.
Does anyone got a coin they can lend me?
I have some coins
All right.
Now, this is where I see how much the students trust me.
Do I get a penny?
Do I get a silver dollar?
So what do we got here?
This is someone who's entrusting me with quarters, not so bad.
So we'll take these quarters, and we'll shake them up, and we'll put them down on the table.
And now, we'll ask a question-- do we have two heads, two tails, or one head and one tail?
So who thinks we have two heads?
Who thinks we have two tails?
Who thinks we have one of each?
Well, clearly, everyone except a few people-- for example, the Indians fan, who clearly believe in the counterfactual-- made the most probabilistic decision.
But in fact, there is no nondeterminism here.
I know the answer.
And so in some sense, it doesn't matter whether it's deterministic, because in fact, it's not causally nondeterministic.
The answer is quite clear, but you don't know the answer.
And so whether or not the world is inherently unpredictable, the fact that we never have complete knowledge of the world suggests that we might as well treat it as inherently unpredictable.
And so this is called predictive nondeterminism.
And this really is what's going to underline pretty much everything else we're going to be doing here.
No comments about that?
I wouldn't do that to you.
Thank you.
I know you are wishing to get interest on the money, but you don't get any
Was it heads or tails?
What was that?
So when we think about nondeterminism in computation, we use the word stochastic process.
And that's any process that's ongoing in which the next state depends upon the previous states in some random element.
So typically up till now when we've written code, one line of code did depended only on what the previous lines of code did.
There was no randomness.
Here, we're going to have randomness.
And we can see the difference if we look at these two specifications of rolling a die.
The first one, returns an int between 1 and 6, is what I'll call underdetermined.
By that I mean you can't tell what it's going to return.
Maybe it will return a different number each time you call it, but it's not required to.
Maybe it will return three every time you call it.
The second specification requires randomness.
It says, it returns are randomly chosen int.
So it requires a stochastic implementation.
Let's look at how we implement a random process in Python.
We start by importing the library random.
This is not to say you can import any random library you want.
It's to say you import the library called random.
Let me get my pen out of here.
And we'll use that a lot.
And then we're going to use the function in random called random.choice.
It takes as an argument a sequence, in this case a list, and randomly chooses one member of the list.
And it chooses it uniformly.
It's a uniform distribution.
And what that means is that it's equally probable that it will choose any number in that list each time you call it.
We'll later look at distributions that are not uniform, not equally probable, where things are weighted.
But here, it's quite simple, it's just uniform.
And then we can test it using testRoll-- take some number of n and rolls the die that many times and creates a string telling us what we got.
So let's consider running this on, say, testRoll of five.
And we'll ask the question, if we run it, how probable is it that it's going to return a string of five 1's?
How do we do that?
Now, how many people here are either in 6.041 or would have taken 6.041?
Raise your hand.
Oh, good.
So very few of you know probability.
That helps.
So how do we think about that question?
Well, probability, to me at least, is all about counting, especially discrete probability, which is what we're looking at here.
What you do is you start by counting the number of events that have the property of interest and the number of possible events and divide one by the other.
So if we think about rolling a die five times, we can enumerate all of the possible outcomes of five rolls.
So if we look at that, what are the outcomes?
Well, I could get five 1's.
I could get four 1's and a 2 or four 1's and 3, skip a few.
The next one would be three 1's, a 2 and a 1, then a 2 and 2, and finally, at the end, all 6's.
So remember, we looked before at when we're looking at optimization problems about binary numbers.
And we said we can look at all the possible choices of items in the knapsack by a vector of 0's and 1's.
We said, how many possible choices are there?
Well, it depended on how many binary numbers you could get in that number of digits.
Well, here we're doing the same thing, but instead of base 2, it's base 6.
And so the number of possible outcomes of five rolls is quite high.
How many of those are five 1's?
Only one of them, right?
So in order to get the probability of a five 1's, I divide 1 by 6 to the fifth.
Does that makes sense to everybody?
So in fact, we see it's highly unlikely.
The probability of a five 1's is quite small.
Now, suppose we were to ask about the probability of something else-- instead of five 1's, say 53421.
It kind of looks more likely than that than five 1's in a row, but of course, it isn't, right?
Any specific combination is equally probable.
And there are a lot of them.
So this is all the probability we're going to think about we could think about this way, as simply a matter of counting-- the number of possible events, the number of events that have the property of interest-- in this case being all 1's-- and then simple division.
Given that framework, there were three basic facts about probability we're going to be using a lot of.
So one, probabilities always range from 0 to 1.
How do we know that?
Well, we've got a fraction, right?
And the denominator is all possible events.
The numerator is the subset of that that's of interest.
So it has to range from 0 to the denominator.
And that tells us that the fraction has to range from 0 to 1.
So 1 says it's always going to happen, 0 never.
So if the probability of an event occurring is p, what's the probability of it not occurring?
This follows from the first bullet.
It's simply going to be 1 minus p. This is a trick that we'll find we'll use a lot.
Because it's often the case when you want to compute the probability of something happening, it's easier to compute the probability of it not happening and subtract it from 1.
And we'll see an example of that later today.
Now, here's the biggie.
When events are independent of each other, the probability of all of the events occurring is equal to the product of the probabilities of each of the events occurring.
So if the probability of A is 0.5 and the probability of B is 0.4, the probability of A and B is what?
0.5 times 0.4.
You guys can figure that out.
I think that's 0.2.
So you'd expect that, that it should be much smaller than either of the first two probabilities.
This is the most common rule, it's something we use all the time in probabilities, the so-called multiplicative law.
We have to be careful about it, however, in that it only holds if the events are actually independent.
Two events are independent if the outcome of one has no influence on the outcome of the other.
So when we roll the die, we assume that the first roll, the outcome, was independent of the-- or the second roll was independent of the first roll, independent of the fourth roll.
When we looked at the two coins, we assume that heads and tails of each coin was independent of the other coin.
I didn't, for example, look at one coin and make sure that the other one was different.
The danger here is that people often compute probabilities assuming independence when you don't actually have independence.
So let's look at an example.
For those of you familiar with American football, the New England Patriots and the Denver Broncos are two prominent teams.
And let's look at computing the probability of whether one of them will lose on a given Sunday.
So the Patriots have a winning percentage of 7 of 8-- they've won 7 of their 8 games so far-- and the Broncos 6 of 8.
The probability of both winning next Sunday, assuming that this is indicative of how good they are, we can get with the multiplicative rule.
So it's 7/8 times 6/8, or 42/64.
We could simplify that fraction, I suppose.
Does that makes sense?
So this is probably a pretty good estimate of both of them winning next Sunday.
So the probability of at least one of them losing is 1 minus that.
So here's an example of why we often use the 1 minus rule, because we could compute the probability of both of them winning by simply multiplying.
And we subtract that from 1.
However, what about Sunday, December 18?
What's the probability?
Well, as it happens, that day the Patriots are playing the Broncos.
So now suddenly, the outcomes are not independent.
The probability of one of them losing is influenced by the probability of the other winning.
So you would expect the probability of one of them losing is much closer to 1 than 22/64, which is about 1/3.
So in this case, it's easy.
But as we'll see, as we get through the term, there are lots of cases where you have to work pretty hard to understand whether or not two events really are independent.
And if you get it wrong, you get a totally bogus answer.
1/3 versus 1 is a pretty big difference.
By the way, as it happens, the probability of the Broncos losing is about 1.
Let's go look at some code.
And we'll go back to our dice, because it's much easier to simulate dice games than it is to simulate football games.
So here it is.
And we're going to talk a lot about simulations.
So here, rather than rolling the die, I've written a program to do it.
We've already seen the code for rolling a die.
And so to run this simulation, typically what we're doing here is I'm giving you the goal-- for example, are we going to get five 1's-- the number of trials-- each trial, in this case, will be say of length 5-- so I'm going to roll the same die five times say 1,000 different times, and then just some text as to what I'm going to print.
Almost all the simulations we look at are going to start with lines that look a lot like that.
We're going to initialize some variable.
And then we're going to run some number of trials.
So in this case, we're going to get from the length of the goal-- so if the goal is five 1's, then we're going to roll the dice five times; if it's 10 runs, we'll roll it 10 times.
So this is essentially one trial, one attempt.
And then we'll check the result.
And if it has the property we want-- in this case, it's equal to the goal-- then we're going to increment the total, which we initialized up here by 1.
So we'll keep track with just the counting-- the number of trials that actually meet the goal.
And then when we're done, what we're going to do is divide the number that met the goal by the number of trials-- exactly the counting argument we just looked at.
And then we'll print the result.
Almost every simulation we look at is going to have this structure.
There'll be an outer loop, which is the number of trials.
And then inside-- maybe it'll have a loop, or maybe it won't-- will be a single trial.
We'll sum up the results.
And then we'll divide by the number of trials.
Let's run it.
So a couple of things are going to go on here.
If you look at the code as we've looked at it before, what you're seeing is I'm computing the estimated probability by the simulation.
And I'm comparing it to the actual probability, which we've already seen how to compute.
So if you look at it, there are a couple of things to look at.
The estimated probability is pretty close to the actual probability but not the same.
So let's go back to the PowerPoint.
Here are the results.
And there are at least two questions raised by this result.
First of all, how did I know that this is what would get printed?
Remember, this is random.
How did I know that the estimate-- well, there's nothing random about the actual probability.
But how did I know that the estimated probability would be 0?
And why did it print it twice?
Because I messed up the PowerPoint.
Any rate, so how do I know what would get printed?
Well a confession-- random.choice is not actually random.
In fact, nothing we can do in a computer is actually random.
You can prove that it's impossible to build a computer that actually generates truly random numbers.
What they do instead is generate numbers that called pseudorandom.
How do they do that?
They have an algorithm that given one number generates the next number in a sequence.
And they start that algorithm with a seed.
Now, typically, they get that seed by reading the clock of the computer.
So most computers have a clock that, say, keeps track of the number of microseconds since January 1, 1978.
I don't know if that's still true.
That's what Unix used to do.
So the notion is, you start your program, there's no way of knowing how many microseconds have elapsed.
And so you're getting a random number to start the process.
Since you don't know where it starts, you don't know what the second number is, you don't know what the third number is, you don't know what the fourth number is.
And so it's predictably nondeterministic, because you don't know what the seed is going to be.
Now, you can imagine that this makes programs really hard to debug.
Every time you run it, something different could happen.
Now, we'll see often you want them to be unpredictable.
But for now, we want them to be predictable, makes it easier prepare PowerPoint.
So what you have is a command.
You can call random.seed and give it a value and say, I don't want you to just choose some random seed, I want you to use 0 as the seed.
For the same seed, you always get the same sequence of random values.
And so what I've done is I set the seed to be, I think, 0 in this case, not because there's anything magic about 0, it's just sort of habit.
But it made it predictable.
As you write programs with randomness in and when you're debugging it, you will almost surely want to start by setting random.seed to a value so you get the same answer.
But make sure you debug it with more than one value of this, so you didn't just get lucky with your seed.
So that's how I knew what would get printed.
The next question is, why did the simulation give me the wrong answer?
The actual probability is three 0's and 1286.
But it's estimated a probability of 0.
Why is it wrong?
Well, let's think about this.
I ran 1,000 trials.
What does it mean to say the probability is zero?
It means that I tried it 1,000 times and didn't ever get a sequence of five 1's.
So the numerator of the division at the bottom was 0.
Hence, the answer is 0.
Is this surprising?
Well, no.
Because if that's the actual probability of getting five 1's, it's not very shocking that in 1,000 trials it never happened.
It's not a surprising result.
And so we have to be careful when we run these things to understand the difference between what's in this case an actual probability and what statisticians call a sample probability.
So what we got with the sample was 0.
So what's the obvious thing to do?
If you're doing a simulation of an event and the event is pretty rare, you want to try it on a very large number of trials.
So let's go back to our code.
And we'll change it to instead of 1,000, 1,000,000.
You can see up here, by the way, where I set the seed.
And now, let's run it.
We did a lot better.
If we look at here our estimated probability, it's three 0's 128, still not quite the actual probability but darn close.
And maybe if I had done 10 million, it would have been even closer.
So if you're writing a simulation to compute the probability of an event and the event is moderately rare, then you better run a lot of trials before you believe your estimated probability.
In a week or so, we'll actually look at that more mathematically and say, what is a lot, how do we know what is enough.
What are the morals here?
Moral one, I've just told you-- takes a lot of trials to get a good estimate of the frequency of a rare event.
Moral two, we should always, if we're getting an estimated probability, know that, and probably say that, and not confuse it with the actual probability.
The third moral here is, it was kind of stupid to do a simulation.
Since it was a very simple closed-form answer that we could compute that would really tell us what the actual probability is, why even bother with the simulation?
Well, we're going to see why now, because simulations can be very useful.
Let's look at another problem.
This is the famous birthday problem.
Some of you have seen it.
What's the probability of at least two people in a group having the same birthday?
There's a URL at the bottom.
That's pointing to a Google form.
I'd like please all of you who have a computing device to go to it and fill out your birthday.
It's anonymous, so we won't know how old you are, don't worry.
Actually, it's only the date.
It's not the year.
So suppose there were 367 people in the group, roughly the number of people who took the 6.0001 600 midterm.
If they are 367 people, what's the probability of at least two of them sharing a birthday?
One, by something called the pigeonhole principle.
You got some number of holes.
And if you have more pigeons than holes, two pigeons have to share a whole.
What about smaller numbers?
Well, if we make a simplifying assumption that each birthdate is equally likely, then there's actually a nice closed-form solution for it.
Again, this is a question where it's easier to compute the opposite of what you're trying to do and subtract it from 1.
And so this fraction is giving the probability of two people not sharing a birthday.
The proof that this is right, it's a little bit elaborate.
But you can trust me, it's accurate.
But it's a formula, and it's not that complicated a formula.
So numbers like 366 factorial are big.
So let's approximate a solution.
We'll right a simulation and see if we get the same answer that that formula gave us.
So here's the code for that-- two arguments-- the number of people in the group and the number that we asking do they have the same birthday.
So since I'm assuming for now that every birthday is equally likely, the possible dates range from 1 to 366, because some years have a February 29.
I'll keep track of the number of people born in each date by starting with none.
And then for p in the range of number of people, I'll make a random choice of the possible dates and increment that element of the list by 1.
And then at the end, we can say, look at the maximum number of birthdays and see if it's greater than or equal to the number of same.
So that tells us that.
And then we can actually look at the birthday problem-- number of people, the number of same, and, as usual, the number of trials.
So the number of hits is 0 for t in range number of trials.
If sameDate is true, then we'll increment the number of hits by 1 and then as usual divide by the number of trials.
And we'll try it for 10, 20, 40, and 100 people.
And then just, we'll print the estimated probability and the actual probability computed using that formula I showed you.
I have not shown you, but I've imported a library called math, because it is a factorial implementation.
It's way faster than the recursive one that we've seen before.
Let's run it.
And we'll see what we get.
So for 10, the estimated probability is 0.11 now.
So you can see, the estimates are really pretty good.
Once again, we have this business that for 100, we're estimating 1, when the real answer is point many, many 9's.
But again, this is sample probability.
It just means in the number of trials we did, every 1 for 100 people, there was a shared birthday.
This is a number that usually surprises people, as to why with 100 people the probability is so high.
But we could work out the formula and see it.
And as you can see, the estimates are pretty good from my simulation.
Now, we're going to see why we did a simulation in the first place.
Suppose we want the probability of three people sharing a birthday instead of two.
It's pretty easy to see how we changed the simulation.
I even made a parameter.
I just changed the number 2 to number 3.
The math, on the other hand, is ugly.
Why is the math so much uglier for 3 than for 2?
Because for 2, the complementary problem-- the number we're subtracting from 1-- is simply the question of, are all birthdays different?
So did two people share a birthday is 1 minus or all does everybody have a different birthday.
On the other hand, for 3 people, the complementary problem is a complicated disjunct-- a bunch of ors-- either all birthdays are distinct, or two people share a birthday and the rest are distinct, or there are two groups of two people sharing a birthday and everything is distinct.
So you can see here, there's a lot of possibilities.
And so it's 1 minus now a very complicated formula.
And in fact, if you try and look how to do this, most people will tell you don't bother.
Here's kind of a good approximation.
But the math gets very hairy.
In contrast, changing the simulation is dead easy.
We can do that.
Whoops.
So if we come over here for the code, all I have to do is change this to 2 or 3.
And I'm going to leave in this code, which is the wrong code, computing the actual probability now for 2 people sharing rather than 3, because I want to make it easy for you to see the difference between what happens when we look at 3 shared rather than 2 shared.
And I get invalid syntax.
That's not good.
That's what happens when I type in real time.
Why do I have invalid syntax?
Line 56
Pardon
Line 56
One person, Anna
Line 56, there's a comma
Oh.
That's not a good line.
So now, we see that if we get, say, to n equals 100, for 2, you'll remember, it was 0.99.
But for 3, it's only 0.63.
So we see going from two sharing to three sharing gets us a radically different answer, not surprisingly.
But we also-- and the real thing I wanted you to see-- is how easy it was to answer this question with the simulation.
And that's a primary reason we use simulations to get probabilistic questions rather than sitting down and the pencil and paper and doing fancy probability calculations, because it's often way easier to do a simulation.
We can see that in spades if we look at the next question.
Let's think about this assumption that all birthdays are equally likely.
Well, as you can see, this is a chart of how common birthdates are in the US, a heat map.
And you'll see, for example, that February 29 is quite an uncommon birthday.
So we should probably treat that differently.
Somewhat surprisingly, you'll see that July 4 is a very uncommon birthday as well.
It's easy to understand why February 29.
The only thing I can figure out for July 4 is obstetricians don't like working on holidays.
And so they induce labor sometime around the 2nd or the 3rd, so they don't have to come to work on the 4th or the 5th.
Sounds a horrible thought.
But I can't think of any other explanation for this anomaly.
You'll probably, if you look at it, see Christmas day is not so common either.
So now, the question, which we can answer, since you've all fill out this form, is how exceptional are MIT students?
We like to think that you're different in every respect.
So are your birthdays distributed differently than other dates?
Have we got that data?
So now we'll go look at that.
We should have a heat map for you guys.
This one?
Yep.
I removed all the February 31.
Thank you for those submissions.
[LAUGHTER]
So here it is.
And we can see that, well, they don't seem to be banded quite as much in the summer months, probably says more about your parents than it does about you.
But you can see that, indeed, we do have-- wow, we have a day where there are five birthdays, that look like?
Or no?
February 12
Wow.
You want to raise your hand if you're born on February 12?
[LAUGHTER] So you are exceptional in that you lie about when you're born.
But if you hadn't lied, I think we would have still seen the probabilities would hold.
How many people were there, do we know?
146 with 112 unique birthdays
146 people, 112 unique birthdays.
So indeed, the probability does work.
So we know you're exceptional in a funny way.
Well, you can imagine how hard it would be to adjust the analytic model to account for a weird distribution of birthdates.
But again, adjusting the simulation model is easy.
I could have gone back to that heat map I showed you of birthdays in the US and gotten a separate probability for each day, but I was too lazy.
And instead, what I observed was that we had a few days, like February 29, highly unlikely, and this band in the middle of people who were conceived in the late fall and early winter.
So what I did is I duplicated some dates.
So the 58th day of the year, February 29, occurs only once.
The dates before that, I said, let's pretend they occur four times.
What only matters here is not how often they occur but the relative frequency.
And then the dates after that occur four times except for the dates in that band, which is going to have occur yet more often.
So now-- and don't worry about the exact details here-- but what I'm doing is simply adjusting the simulation to change the probability of each date getting chosen by same date.
And then I can run the simulation model.
And, again, with a very small change to code, I've modeled something that's mathematically enormously complex.
I have no idea how to actually do this probability mathematically.
But the code is, as you can see, quite straightforward.
So let's go to that here.
So what I'm going to do is comment this one out and uncomment this more complicated set of dates and see what we get.
And again, it changes quite dramatically.
You might remember, before it was around I think 0.6-something for 100, and now, it's 0.75.
So getting away from the notion that birthdays are uniformly distributed to saying some birthdays are more common than others, again, dramatically changes the answer.
And we can easily look at that.
So that gets us to the big topic of simulation models.
It's a program that describes a computation that provides information about the possible behaviors of a system.
I say possible behaviors, because I'm particularly interested in stochastic systems.
They're descriptive not prescriptive in the sense that they describe the possible outcomes.
They don't tell you how to achieve possible outcomes.
This is different from what we've looked at earlier in the course, where we looked at optimization models.
So an optimization model is prescriptive.
It tells you how to achieve an effect, how to get the most value out of your knapsack, how to find the shortest path from A to B in a graph.
In contrast, a simulation model says, if I do this, here's what happens.
It doesn't tell you how to make something happened.
So it's very different, and it's why we need both, why we need optimization models and we need simulation models.
We have to remember that a simulation model is only an approximation to reality.
I put in an approximation to the distribution of birthdates, but it wasn't quite right.
And as the very famous statistician George Box said, "all models are wrong, but some are actually very useful."
In the next lecture, we'll look at a useful class of models.
When do we use simulations?
Typically, as we've just shown, to model systems that are mathematically intractable, like the birthday problem we just looked at.
In other situations, to extract intermediate results-- something happens along the way to the answer.
And as I hope you've seen that simulations are used because we can play what if games by successively refining it.
We started with a simple simulation that assumed that we only asked the question of, do two people share a birthday.
We showed how we could change it to ask do three people share a birthday.
We then saw that we could change it to assume a different distribution of birthdates in the group.
And so we can start with something simple.
And we get it ever more complexed to answer questions what if.
We're going to start in the next lecture by producing a simulation of a random walk.
And with that, I'll stop.
And see you guys soon.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're starting a new topic, which is, of course, related to previous topics.
As usual, if you go to either the 60002 or the 600 web site, you'll find both today's PowerPoint and today's Python.
You'll discover if you look at the Python file that there is quite a lot of code in there.
And I'll be talking only about some of it.
But it's probably all worth looking at.
And a fair amount of reading associated with this week.
Why are we looking at random walks?
See a picture here of, think of them as molecules just bouncing around.
This is actually a picture of what's called Brownian motion, though Robert Brown probably did not discover it.
We're looking at random walks because, well, first of all, they're important in many domains.
There are people who will argue, for example, that the movement of prices in the stock market is best modeled as a random walk.
There was a very popular book called A Random Walk Down Wall Street that made this argument.
And a lot of modern portfolio analysis is based upon that.
Those of you who are not interested in making money, and I presume that's most of you, it's also very important in many physical processes.
We use random walks, say, to model diffusion, heat diffusion, or the diffusion of molecules in suspension, et cetera.
So they're very important in a lot of scientific, and indeed, social disciplines.
They're not the only important thing, so why are we looking at those?
Because I think it provides a really good illustration of how we can use simulation to understand the world around us.
And it does give me an excuse to cover some important topics related to programming.
You'll remember that one of the subtexts of the course is while I'm covering a lot of what you might think of as abstract material, we're using it as an excuse to teach more about programming and software engineering.
A little practice with classes and subclassing, and we're going to also look at producing plots.
So the first random walk I want to look at is actually not a diffusion process or the stock market, but an actual walk.
So imagine that you've got a field which has somehow inexplicably been mown to look like a piece of graph paper, and you've got a drunk wandering around the field, taking a step every once in a while in some random direction.
We can then ask the question is there an interesting relationship between the number of steps the drunk takes and how far the drunk is from the origin at the end of those steps?
You could imagine that if the drunk takes more steps, he's ever further from the origin.
Or maybe you could imagine, since it's random, that he just wanders away and he wanders back in all directions and more or less never gets very far.
So just out of curiosity, I'll take a poll.
Who thinks that the drunk doesn't much matter how many steps he takes, he'll be more or less the same distance away?
And who thinks the more steps he takes, the further away he's likely to be?
It seems to be a season where when you take polls, they come out almost tied.
Let's look at a small example.
Suppose he takes one step only.
Well, if he takes one step, and we'll assume for simplicity that he's not so drunk that he moves at random.
He either moves north or south, east or west.
These are all the places he can get to in one step.
What they have in common is that after one step, the drunk is always exactly one unit away from the origin.
Well, how about after two steps?
So without loss of generality, let's assume that the first step-- let me use the pen that you're supposed to use to write on this, rather than this pen, which would make a real mess on my screen.
What did I do with it?
Well, I won't write on it.
So without loss of generality, we'll assume that the drunk is there after one step.
Took one step to the east.
Well, after two steps, those are all the possible places he could be.
So on average, how far is the drunk from the origin?
Well, if we look, he could either be two steps away, if he took another step east, zero steps away, if he took a step west, or what do we see for the top two?
Well, the top and the bottom one, we can go back and use the Pythagorean theorem.
c squared equals a squared plus b squared.
And that will tell us that it'll be the square root of a squared plus b squared.
And that will tell us how far away the upper two are, and then we can just average them and get a distance.
And as we can see, on average, the drunk will be a little bit further away after two steps than after one step.
Well how about after 100,000 steps?
It would be a little bit tedious to go through the case analysis I just did.
There are a lot of cases after 100,000 steps.
So we end up resorting to a simulation.
So we'll structure it exactly the same way we've been structuring our other simulations.
We're going to simulate one walk of k steps, n such walks, and then report the average distance from the origin of the n walks.
Before we do that, in line with the software engineering theme of the course, we'll start by defining some useful abstractions.
There are three of them I want to look at.
Location, the field that the drunk is in, and the drunk him or herself.
So let's first look at location.
This is going to be an immutable type.
So what we see here-- as long as I can't point in the screen, I'll point with a pointer-- is that we'll initiate it.
We'll initialize it with an x and y value.
That makes sense.
We'll be able to have two getters, getX and getY.
And here's how we see it's immutable.
What move is doing is it's not changing the location, it's returning a new location.
Perhaps move is poor choice of name for that.
But that is what it's doing.
It's just returning a new location where it adds the change in x and the change in y to get two new xy values.
Notice, by the way, that I'm not restricting these to be integers, or one, or anything like that.
So this would work even if I did not want to take those nice little east-west, north-south steps.
I've got a underbar underbar string, _str_, and then here's my implementation-- you can see it's very sophisticated-- of the Pythagorean theorem.
So I just do it that way, and that will get me the distance between two things.
It's one of the annoying things about classes in, actually, all languages I know with classes, is you would like to think that self and other-- there's a symmetry here.
The distance from self to other is the same as from other to self.
But syntactically, because of the way the language is structured, we treat them a little bit differently.
How about class Drunk?
Well, this is kind of boring.
Drunk has a name and a string.
And that's all.
The point of this, and I don't think we've looked at this before, is this is not intended to be a useful class on its own.
It's what we call a base class.
The notion here is its only purpose is to be inherited.
It's not supposed to be useful on itself, but it does give me something that will be used for the two subclasses.
And we'll look at two subclasses.
The so-called usual drunk, the one I tried to simulate when I was wandering around, wanders around at random.
And a drunk I like to think of it as a New Englander, or a masochistic drunk, who tries forever to move ever northward, because he or she wants to be frozen.
I do like this picture of entering the state of Maine in the winter.
So here is the usual drunk.
Subclass of drunk, and it can take steps at random, one step, either increasing y, a step north, decreasing y, a step south, increasing x, a step east, or decreasing x a step west.
So those are the choices.
And it's going to return one of those at random.
I think we saw random.choice in the last lecture.
And then our masochistic drunk, it's almost the same, except the choices are slightly different.
If he chooses to head north, he doesn't go one step.
He goes 1.1 steps north.
And if he chooses to go south, he only goes 9/10 of a step.
So what we're seeing here is what's called a biased random walk.
And the bias here is the direction of the walk that he's moving either up or down.
Pretty simple.
How about just for to test things out, we'll ask the question is this an immutable or a mutable type?
Are drunks mutable or immutable?
This is a deep philosophical question.
But if we ignore the philosophical underpinnings of that question, what about the two types here?
Who thinks it's immutable?
Who thinks it's mutable?
Why do you think it's mutable?
What's getting changed?
The answer is nothing.
It gets created, and then it's returning the step, but it's not actually changing the drunk.
So so far we have two things that are immutable, drunks and locations.
Let's look at fields.
Fields are a little bit more complicated.
So field will be a dictionary, and the dictionary is going to map a drunk to his or her location in the field.
So we can add a drunk at some location, and we're going to check.
And if the drunk is already there, we're not going to put the drunk in.
We're going to raise a value error, "Duplicate drunk."
Otherwise we're going to set the value of drunkenness mapping to loc.
Now you see, by the way, why I wanted drunks to be immutable.
Because they have to be hashable so I can use them as a key in a dictionary.
So it was not an idle question whether they were immutable.
It was an important question.
I can get the location of a drunk.
If the drunk is not in there, then I'll raise a different value error, "Drunk not in field."
Otherwise I'll return the location associated with that drunk.
And finally, we're going to have moveDrunk.
Again I'll check whether the drunk is there.
If the drunk is there, I'm going to get the distance on x and the distance in y by calling drunk.takeStep.
So we saw takeStep for a drunk didn't move the drunk anywhere, because the drunks were immutable, but returned new locations.
A new x and new values.
And then I'm going to use that to move the drunk in the field.
So I'll set self.drunk, so drunk to move x distance and y distance.
So it's very simple, but having built this set of classes, we can now actually write the simulation.
Oh.
What about our classes?
Are they mutable or immutable?
Not classes.
What about fields?
Any votes for mutable?
Yeah, exactly.
Because you can see I'm mutating it right here.
I'm changing the value of the dictionary.
And in fact, every time I add a drunk to the field, I'm changing the value of the dictionary, which is to say mutating the field.
So I'll have a bunch of locations, which are immutable objects.
Makes sense that a location is immutable.
A bunch of drunks, and the thing I'm going to change is where the drunks are in the field.
I said we'd start by simulating a single walk.
So here it is, a walk in a field with a drunk, and that drunk will take some number of steps in the field.
And you can see this.
It's very simple.
I just have a loop.
Drunk takes some number of random steps, and I'm going to return the distance from the start to the final location of the drunk.
So how far is the drunk from the origin?
I then need to simulate multiple walks.
Notice here that I've got the number of steps, the number of trials, and dClass stands for class of the drunk.
And that's because I want to use the same function to simulate as many different kinds of drunks as I care about.
We've only seen two here, the masochistic drunk and the usual drunk, but you can imagine many other kinds as well.
So let's do it.
So here I'm going to simulate a walk for one drunk, Homer.
So we'll create a drunk named Homer, or the variable Homer, which is the drunk class.
Then the origin, distances, and for t in range number of trials, we'll just do it, and then we'll return the distances.
So it's initialized to the empty list.
So we're going to return a list for however many trials we do, how far the drunk ended up from the origin.
Then we can average that, and we look at the mean.
Maybe we'll look at the min or the max.
Lots of different questions we could ask about the behavior.
And now we can put it all together here.
So drunkTest will take a set of different walk lengths, a list of different walk lengths, the number of trials, and the class.
And for a number of steps and walk lengths, distances will be simWalks of number of steps, numTrials, dClass.
And then I'm going to just print some statistics.
You may or may not have seen this.
This is something that's built in to Python.
I can ask for the name of a class.
So dClass, remember, is a class, and _name_ will give me the name of the class.
Might be usual, it might be drunk, in this case.
So let's try it.
So the code we've looked at.
So let's go down here, and we'll run it, and we'll try it for walks of 10, 100, 1,000, and 10,000 steps.
And we'll do 100 trials.
Here's what we got.
So my question to you is does this look plausible?
What is it telling us here?
Well, it's telling us here that the length of the walk actually doesn't really affect-- the number of steps doesn't affect how far the drunk gets.
There's some randomness.
8.6, 8.57, 9.2, 8.7.
Not much variance.
So we've done this simulation and we've learned something, maybe.
So does this look plausible?
We can look at it here.
I've just transcribed it.
What do you think?
Well, go ahead
I was going to say, it seems plausible because after the first two steps, there's a 50% chance he's going closer to the origin.
And a 50% chance he's going away from it
So we have at least one vote for plausible, and it's certainly a plausible argument.
Well, one of the things we need to learn to do is whenever we build a simulation, we need to do what I call a sanity check to see whether or not the simulation actually makes sense.
So if we're going to do a sanity check, what might we do in this case?
We should try it on cases where we think we know the answer.
So we say, let's take a really simple case where we're pretty sure we know what the answer is.
Let's run our simulation and make sure it gives us the right answer for this simple case.
So if we think of a sanity check here, maybe we should look at these numbers.
We just did it.
We know how far the drunk should get in zero steps.
How far should the drunk move in zero steps?
Zero.
How far should the drunk move in one steps?
We know that should be one.
Two steps, well, we knew what that should be.
Well, if I run this sanity check, these are the numbers I get.
I should be pretty suspicious.
I should also be suspicious they're kind of the same numbers I got for 10,000 steps.
What should I think about?
I should think that maybe there's a bug in my code.
So if we now go back and look at the code, yes, this fails the pants on fire test that there's clearly something wrong with these numbers.
What we were appending is walk of Homer, numTrials, 1.
Well, numTrials is a constant.
It's always 100.
What I intended to write here was not numTrials but numSteps.
I actually did this the first time I wrote this simulation many years ago.
I made this typo, if you will, and I got these bizarre answers.
So I looked at the code and I said, well, that's actually wrong.
No wonder it's always the same number.
I'm calling it with a constant.
The constant happens to be 100.
So let's go fix the simulation.
So this should have been numSteps.
Now let's run it again.
Well, these results are pretty different.
Now we see that in fact, they're increasing.
Should I just look at this and be happy?
Probably not.
I should run my sanity check again and make sure I get the right results for zero, one, and two.
So let's go back and do that, just to be a little bit safe.
So I'll just change this tuple of values to be-- and I should feel a lot better about this.
The mean, the max, and the min are all zero when he doesn't take any steps.
One is exactly what we should expect, and two is also-- the mean is where we would guess it to be, and the max is two, happened to take two steps in the same direction, and the min is zero, happened to end up where he started.
So I've passed my sanity check.
Doesn't mean my simulation is right, but at least I have reason to be hopeful.
So getting back.
So we saw these results, and now we're getting the indication that in fact, contrary to what we might have thought, it does appear to be that the more steps the drunk takes, the further away the drunk ends up.
And that was the usual drunk.
We can try the masochistic drunk, and then we see something pretty interesting.
I won't make you sit through it, but when we run it, here are the usual drunks, the numbers, and I just looked at it for 1,000 and 10,000 so it would fit on the screen.
You see for the usual drunk, it's 26.8, roughly 90.
Fair dispersion in the min and the max.
And the masochistic drunk seems to be making considerably more progress than the usual drunk.
So we see is this bias actually appears to be changing the distance.
Well, that's interesting.
Now we could ask the question why?
What's going on?
And to do that, I want to go and start visualizing what's the trend?
So rather than just looking at two numbers or three numbers, as we've been doing, I'm going to draw a pretty picture.
Actually, I'm not going to draw.
I'm a terrible artist.
But Python will draw us some pretty pictures.
We're going to simulate walks of multiple lengths for each kind of drunk, and then plot the distance at the end of each length walk for each kind of drunk.
I now digress for a moment to talk about how we do plotting.
So we're going to use something called Pylab.
I've listed here four really important libraries that you will surely end up using extensively if you continue to use Python for research purposes.
NumPy adds vectors, matrices, and many high-level mathematical functions.
Actually, it might be NumPy.
It might be "Num-Pee."
I'm not sure how to pronounce it.
But we'll call it NumPy.
So these are really useful things.
SciPy adds on top of that a bunch of mathematical classes and functions useful to scientists.
Things like-- well, we'll look at some of them as we go on through the term.
MatPlotLib adds an object-oriented programming interface for plotting.
Anybody here used MATLAB?
Great.
Well you'll find that MatPlotLib is the Mat, think MATLAB.
Lets you, in Python, use all the plotting stuff that you've come to either like or hate in MATLAB.
So it's really convenient.
If you know how to do plots in MATLAB, you'll know how to do it in Python.
PyLab combines all of these to give you a MATLAB-like interface to Python.
So once you have PyLab, you can do a lot of things that you would normally want to do in MATLAB, for example, produce weird-looking plots like this one.
I'm going to show you one of the many, many plotting commands.
It is called plot.
It takes two arguments, which must be sequences of the same length.
The first argument is the x-coordinates, the second argument is the y-coordinates corresponding to the x ones.
There are roughly three zillion optional arguments, and you'll see me use only a small subset of them.
It plots the points in order.
First the first xy, then the second xy, then the third xy.
Why is it important that I say it plots them in order?
Because by default, as each point is plotted, it draws a line connecting one point to the next point to the next point.
And so the order in which they're plotted will determine where the lines go.
Now we'll see, as we go on, that we often don't draw the lines, but by default they are drawn.
Here's an example.
You start by importing PyLab.
Then I've given xVals and yVals1, and if I call pylab.plot of xVals, yVals1.
Here is one of the arguments I can give it.
I'm saying I'd like this to be plotted in blue, b for blue, and I'd like it to be plotted as a solid line, a single dash.
And I want to give that line a label, which I've said is first.
YVals2 is a different list.
I'll plot it again.
Here I'm going to say I want a red dotted line, and the label will be second.
And then after plotting it, I'm going to invoke pylab.legend, which puts this nice little box up here in the corner, in this case, saying that first is a solid blue line and second a dashed red line, I should say.
Now again, there are lots of arguments, lots of other arguments I could give in plot.
Also legend, I can tell it where to put the legend, if I so choose.
Here I've just said, put it wherever you happen to want to put it, or PyLab wants to put it.
So a very simple way to produce a plot.
There are lots of details and many more examples about plotting in the assigned reading.
We've posted a video that Professor Grimson produced for an online course, 600.1x.
It's about a 50 minute video broken into multiple segments about how to use plotting in PyLab with a lot more detail than I've given you.
You'll see if you read the code for this lecture.
And as you see this lecture, there'll be lots of other plots showing up of different kinds.
These are my two favorite online sites for finding out what to do.
And of course, you can google all sorts of things.
That's all I'm going to tell you about how to produce plots in class, but we're going to expect you to learn a lot about it, because I think it's a really useful skill.
And at the very least, you should feel comfortable that any plot I show you, you now-- obviously not right now-- but you will eventually know how to produce.
So if you do these things, you'll be more than up to speed.
So I started by saying I wanted to plot the trends in distance, and they're interesting.
So here's the usual drunk and the masochistic drunk.
So you can see, sure enough, the usual drunk, this fuschia line is progressing very slowly and the masochistic drunk, considerably faster.
I looked at these, and after looking at those two, I tried to figure out whether there was some mathematical explanation of what was going on, and decided, well, it looked to me like the usual drunk was moving at about the square root of the number of steps.
Not so odd to think about it, if you go back to old Pythagoras here.
And sure enough, when I plot, and I ran this simulation up to 100,000 steps.
When I plot the square root of the number of steps, it's not identical, but it's pretty darn close.
Seems to be moving just a tad faster than the square root, but not much.
But who knows exactly?
But pretty good.
And then the masochistic drunk seems to be moving at a rate of numSteps times 0.05.
A less intuitive answer than the square root.
Why do you think it might be doing that?
Well, what we notice is that-- and we'll look at this-- maybe there's not much difference between what the masochistic drunk and the usual drunk do on the x-axis, east and west.
In fact, they shouldn't be.
But there should be a difference on the y-axis, because every time, 1/4 of the time, the drunk is taking a step north of 1.1 units, and 1/4 of the time, he's taking a step south of 0.9 units.
And so 1/2 the time, the steps are diverging by a small fraction.
And if we think about it, 0.1 1/2 the time.
We divide it.
We get 0.05.
So at least we need to do some more analysis, but the data is pretty compelling here that it's a very good fit.
Well, let's look at the ending location.
So here you see a rather different kind of plot.
Here I'm showing that you can plot these things without connecting them by lines.
And giving them different shapes.
So what here I've said is that the masochistic drunk we're going to plot using red triangles, and the usual drunk I'm going to plot using black plus signs.
And since I'm going to plot the location at the end of many walks, it doesn't make sense to draw lines connecting everything, because all we're caring about here is the endpoints.
So since I only want the endpoints, I'm plotting them a different way.
So for example, I can write something like plot( xVals, yVals, and then if I do something like let's see.
Just 'b0' in fact.
What that says is plot blue circles.
I could have written-- in fact I did write 'k+' and that says black plus signs.
And I don't actually remember what I did to get the triangles, but it's in the code.
And so that's very flexible what you do.
And as you can see here, we get the insight I'd communicated earlier that if you look at east and west, not much difference between this ball and that ball.
They seem to be moving about the same spread, the same outliers.
But this ball is displaced north.
And not surprisingly, after 10,000 steps, you would not expect any of these points to be below zero, where you'd expect roughly half of these points to be below zero.
And indeed that's about true.
And we see here what's going on that if we look at the mean absolute difference in x and y, we see that not a huge difference between the usual drunk and the masochistic drunk.
There happens to be a distance.
But a huge difference-- sorry, x and x.
Comparing the two y values, there's a big difference, as you see here.
So what's the point of all this?
It's not to learn about different kinds of drunks.
It's to show how, by visualization, we can get insight into our data that if I just printed spreadsheets showing you all of these endpoints, it would be hard to make sense of what was there.
So get accustomed to using plotting to help you understand data.
Now let's play a little bit more with the simulation.
We looked at different kinds of drunks.
Let's look at different kinds of fields.
So I want to look at a field with what we'll call a wormhole in it.
For those of you of a certain generation, you will recognize the Tardis.
So the idea here is that the field is such that as you wander around it, everything is normal.
But every once in a while you hit a place where you're magically transported to a different place.
So it behaves very peculiarly.
So let's call this an OddField.
Not odd numbers, but odd as in strange.
So it's going to be a subclass of field.
We're going to have a parameter that tells us how many worm holes it has.
A default value of 1,000.
And we'll see how we use xRange and yRange shortly.
So what are the first thing we do?
Well, we'll call Field _init to initialize the field in the usual way.
And then we're going to create a dictionary of wormholes.
So for w in the range number of worm holes, I'm going to choose a random x and a random y in xRange minus xRange to plus xRange, minus yRange to yRange.
So this is going to be where the worm holes are located.
And then for each of those, I'm going to get a random location where you're, in some sense, teleported to if you enter the wormhole.
So here we're using random to get random integers.
We've seen that before.
And so the new location will be the location of the new x and the new y, and we're going to update this dictionary of wormholes to say that paired with the location x, y is newLoc.
Now when we move the drunk, and again this is just changing-- we're overriding moveDrunk, so we're overriding one of the methods in Field.
So Field.moveDrunk will take a self and a drunk.
It's going to get the x value, the y value, and if that is in the wormholes, it's going to move the drunk to the location associated with that wormhole.
So we move a drunk, and if the drunk ends up being in the wormhole, he gets transported.
So we're using Field.moveDrunk.
So notice that we're using the moveDrunk of the superclass, even though we're overriding it here.
Because we've overridden it here, I have to say Field.
to indicate I want the one from the superclass, not the subclass.
And then we're doing something peculiar after the move.
So interestingly here, I've taken, I think, a usual drunk and plotted the usual drunk on a walk of 500 steps.
One walk, and shown all the places the drunk visited.
So we've seen three kinds of plots, one showing how far the drunk would get at different length walks, one showing all the places the drunk would end up with many walks of the same length, and here a single walk, all the places the drunk visits.
And as you can see, the wormholes produce a profound effect, in this case, on where the drunks end up.
And again, you have the code.
You can run this yourself and simulate it and see what you go.
And I think I've set random.Seed to zero in each of the simulations in the code, but you should play with it, change it, to just see that you'll actually get different results with different seeds.
Let me summarize here, and say the point of going through these random walks is not the simulations themselves, but how we built them.
That we started by defining the classes.
We then built functions corresponding to one trial, multiple trials, and reported the results.
And then made a set of incremental changes to the simulation so that we could investigate different questions.
So we started with a simple simulation with just the usual drunk and the simple field, and we noticed it didn't work.
How did we know it?
Well, not because when we did the full simulation we had great insight.
I probably could have fooled 1/2 of you and convinced you that that was a reasonable answer.
But as soon as we went and did the sanity check, where we knew the answer, we could know something was wrong.
And then we went and we fixed it.
And then we went and we elaborated it at a step of a time.
I first got a more sophistic-- I shouldn't say sophisticated.
A different kind of drunk.
And then we went to a different kind of field.
Finally, we spent time showing how to use plots to get an insight.
And in the remaining few minutes of the class, I want to go back and show you some of the plotting commands.
To show you how these plots were produced.
So one of the things I did, since I knew I was going to be producing a lot of different plots, I decided I would actually not spend time worrying about what kind of markers-- those are the things like the triangles and the plus sign-- or what colors for each one individually, but instead I'd set up a styleIterator that would just return a bunch of different styles.
So once and for all, I could define n styles, and then when I want to plot a new kind of drunk, I would just call the styleIterator to get the next style.
So this is a fairly common kind of paradigm to say that I just want to do this once and for all.
I don't want to have to go through each time I do this.
So what do the styles look like?
Let me just get this window.
Oh.
So here it is.
I said there were going to be three styles that I'm going to iterate through.
Style one is going to be an m, I guess that's maroon with a line, a blue with a dashed line, and green with a line with a comma and a minus sign.
So these are called the styles.
And you can control the marker, if you have a marker.
You can control the line.
You can control the color.
Also you can control the size.
You can give the sizes of all these things.
What you'll see when you look at the code is I don't like the default styles things, because when they show up on the screen, they're too small.
So there's something called rcParams.
Those of you who are Unix hackers can maybe guess where that name came from.
And I've just said a bunch of things, like that my default line width will be four points.
The size for the titles will be 20.
You can put titles on the graphs.
Various kinds of things.
Again, once and for all trying to set some of these parameters so they get used over and over again.
And then finally down here, you'll see that I did things like you want to put titles on the slides.
So on the graph.
So here's the location at end of walk.
Title is just a string.
You want to label your x and y-axis, so I've labeled them here.
And here I've said where I want the legend to appear in the lower center.
I've also set the y-limits and the x-limits on the axis, because I wanted a little extra room.
Otherwise, by default it will put points right on the axes, which I find hard to read.
Anyway, the point here is not that you understand all of this instantaneously.
The point I want to communicate is that it's very flexible.
And so if you decide you don't like the way a plot looks and you want to change it, and you know what you want it to look like, there's almost surely a way to make it do that.
So don't despair.
You can look at the references I gave earlier and figure that out.
Next lecture we're going to move on.
No more random walks.
We'll look at simulating other things, and in particular, we'll look at the question of how believable is a simulation?
See you Wednesday if the world has not come to an end.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to Lecture 6.
As usual, I want to start by posting some relevant reading.
For those who don't know, this lovely picture is of the Casino at Monte Carlo, and shortly you'll see why we're talking about casinos and gambling today.
Not because I want to encourage you to gamble your life savings away.
A little history about Monte Carlo simulation, which is the topic of today's lecture.
The concept was invented by the Polish American mathematician, Stanislaw Ulam.
Probably more well known for his work on thermonuclear weapons than on mathematics, but he did do a lot of very important mathematics earlier in his life.
The story here starts that he was ill, recovering from some serious illness, and was home and was bored and was playing a lot of games of solitaire, a game I suspect you've all played.
Being a mathematician, he naturally wondered, what's the probability of my winning this stupid game which I keep losing?
And so he actually spent quite a lot of time trying to work out the combinatorics, so that he could actually compute the probability.
And despite being a really amazing mathematician, he failed.
The combinatorics were just too complicated.
So he thought, well suppose I just play lots of hands and count the number I win, divide by the number of hands I played.
Well then he thought about it and said, well, I've already played a lot of hands and I haven't won yet.
So it probably will take me years to play enough hands to actually get a good estimate, and I don't want to do that.
So he said, well, suppose instead of playing the game, I just simulate the game on a computer.
He had no idea how to use a computer, but he had friends in high places.
And actually talked to John von Neumann, who is often viewed as the inventor of the stored program computer.
And said, John, could you do this on your fancy new ENIAC machine?
And on the lower right here, you'll see a picture of the ENIAC.
It was a very large machine.
It filled a room.
And von Neumann said, sure, we could probably do it in only a few hours of computation.
Today we would think of a few microseconds, but those machines were slow.
Hence was born Monte Carlo simulation, and then they actually used it in the design of the hydrogen bomb.
So it turned out to be not just useful for cards.
So what is Monte Carlo simulation?
It's a method of estimating the values of an unknown quantity using what is called inferential statistics.
And we've been using inferential statistics for the last several lectures.
The key concepts-- and I want to be careful about these things will be coming back to them-- are the population.
So think of the population as the universe of possible examples.
So in the case of solitaire, it's a universe of all possible games of solitaire that you could possibly play.
I have no idea how big that is, but it's really big, Then we take that universe, that population, and we sample it by drawing a proper subset.
Proper means not the whole thing.
Usually more than one sample to be useful.
Certainly more than 0.
And then we make an inference about the population based upon some set of statistics we do on the sample.
So the population is typically a very large set of examples, and the sample is a smaller set of examples.
And the key fact that makes them work is that if we choose the sample at random, the sample will tend to exhibit the same properties as the population from which it is drawn.
And that's exactly what we did with the random walk, right?
There were a very large number of different random walks you could take of say, 10,000 steps.
We didn't look at all possible random walks of 10,000 steps.
We drew a small sample of, say 100 such walks, computed the mean of those 100, and said, we think that's probably a good expectation of what the mean would be of all the possible walks of 10,000 steps.
So we were depending upon this principle.
And of course the key fact here is that the sample has to be random.
If you start drawing the sample and it's not random, then there's no reason to expect it to have the same properties as that of the population.
And we'll go on throughout the term, and talk about the various ways you can get fooled and think of a random sample when exactly you don't.
All right, let's look at a very simple example.
People like to use flipping coins because coins are easy.
So let's assume we have some coin.
All right, so I bought two coins slightly larger than the usual coin.
And I can flip it.
Flip it once, and let's consider one flip, and let's assume it came out heads.
I have to say the coin I flipped is not actually a $20 gold piece, in case any of you were thinking of stealing it.
All right, so we've got one flip, and it came up heads.
And now I can ask you the question-- if I were to flip the same coin an infinite number of times, how confident would you be about answering that all infinite flips would be heads?
Or even if I were to flip it once more, how confident would you be that the next flip would be heads?
And the answer is not very.
Well, suppose I flip the coin twice, and both times it came up heads.
And I'll ask you the same question-- do you think that the next flip is likely to be heads?
Well, maybe you would be more inclined to say yes and having only seen one flip, but you wouldn't really jump to say, sure.
On the other hand, if I flipped it 100 times and all 100 flips came up heads, well, you might be suspicious that my coin only has a head on both sides, for example.
Or is weighted in some funny way that it mostly comes up heads.
And so a lot of people, maybe even me, if you said, I flipped it 100 times and it came up heads.
What do you think the next one will be?
My best guess would be probably heads.
How about this one?
So here I've simulated 100 flips, and we have 50 heads here, two heads here, And 48 tails.
And now if I said, do you think that the probability of the next flip coming up heads-- is it 52 out of 100?
Well, if you had to guess, that should be the guess you make.
Based upon the available evidence, that's the best guess you should probably make.
You have no reason to believe it's a fair coin.
It could well be weighted.
We don't see it with coins, but we see weighted dice all the time.
We shouldn't, but they exist.
You can buy them on the internet.
So typically our best guess is what we've seen, but we really shouldn't have very much confidence in that guess.
Because well, could've just been an accident.
Highly unlikely even if the coin is fair that you'd get 50-50, right?
So why when we see 100 samples and they all come up heads do we feel better about guessing heads for the 101st than we did when we saw two samples?
And why don't we feel so good about guessing 52 out of 100 when we've seen a hundred flips that came out 52 and 48?
And the answer is something called variance.
When I had all heads, there was no variability in my answer.
I got the same answer all the time.
And so there was no variability, and that intuitively-- and in fact, mathematically-- should make us feel confident that, OK, maybe that's really the way the world is.
On the other hand, when almost half are heads and almost half are tails, there's a lot of variance.
Right, it's hard to predict what the next one will be.
And so we should have very little confidence that it isn't an accident that it happened to be 52-48 in one direction.
So as the variance grows, we need larger samples to have the same amount of confidence.
All right, let's look at that with a detailed example.
We'll look at roulette in keeping with the theme of Monte Carlo simulation.
This is a roulette wheel that could well be at Monte Carlo.
There's no need to simulate roulette, by the way.
It's a very simple game, but as we've seen with our earlier examples, it's nice when we're learning about simulations to simulate things where we actually can know what the actual answer is so that we can then understand our simulation better.
For those of you who don't know how roulette is played-- is there anyone here who doesn't know how roulette is played?
Good for you.
You grew up virtuous.
All right, so-- well all right.
Maybe I won't go there.
So you have a wheel that spins around, and in the middle are a bunch of pockets.
Each pocket has a number and a color.
You bet in advance on what number you think is going to come up, or what color you think is going to come up.
Then somebody drops a ball in that wheel, gives it a spin.
And through centrifugal force, the ball stays on the outside for a while.
But as the wheel slows down and heads towards the middle, and eventually settles in one of those pockets.
And you win or you lose.
Now you can bet on it, and so let's look at an example of that.
So here is a roulette game.
I've called it fair roulette, because it's set up in such a way that in principle, if you bet, your expected value should be 0.
You'll win some, you'll lose some, but it's fair in the sense that it's not either a negative or positive sum game.
So as always, we have an underbar underbar in it.
Well we're setting up the wheel with 36 pockets on it, so you can bet on the numbers 1 through 36.
That's way range work, you'll recall.
Initially, we don't know where the ball is, so we'll say it's none.
And here's the key thing is, if you make a bet, this tells you what your odds are.
That if you bet on a pocket and you win, you get [?
len ?]
of pockets minus 1.
So This is why it's a fair game, right?
You bet $1.
If you win, you get $36, your dollar plus $35 back.
If you lose, you lose.
All right, self dot spin will be random dot choice among the pockets.
And then there is simply bet, where you just can choose an amount to bet and the pocket you want to bet on.
I've simplified it.
I'm not allowing you to bet here on colors.
All right, so then we can play it.
So here is play roulette.
I've made game the class a parameter, because later we'll look at other kinds of roulette games.
You tell it how many spins.
What pocket you want to bet on.
For simplicity, I'm going to bet on this same pocket all the time.
Pick your favorite lucky number and how much you want to bet, and then we'll have a simulation just like the ones we've already looked at.
So the number you get right starts at 0.
For I and range number of spins, we'll do a spin.
And then tote pocket plus equal game dot that pocket.
And it will come back either 0 if you've lost, or 35 if you've won.
And then we'll just print the results.
So we can do it.
In fact, let's run it.
So here it is.
I guess I'm doing a million games here, so quite a few.
Actually I'm going to do two.
What happens when you spin it 100 times?
What happens when you spin it a million times?
And we'll see what we get.
So what we see here is that we do 100 spins.
The first time I did it my expected return was minus 100%.
I lost everything I bet.
Not so unlikely, given that the odds are pretty long that you could do 100 times without winning.
Next time I did a 100, my return was a positive 44%, and then a positive 28%.
So you can see, for 100 spins it's highly variable what the expected return is.
That's one of the things that makes gambling attractive to people.
If you go to a casino, 100 spins would be a pretty long night at the table.
And maybe you'd won 44%, and you'd feel pretty good about it.
What about a million spins?
Well people aren't interested in that, but the casino is, right?
They don't really care what happens with 100 spins.
They care what happens with a million spins.
What happens when everybody comes every night to play.
And there what we see is-- you'll notice much less variance.
Happens to be minus 0.04 plus 0.6 plus 0.79.
So it's still not 0, but it's certainly, these are all closer to 0 than any of these are.
We know it should be 0, but it doesn't happen to be in these examples.
But not only are they closer to 0, they're closer together.
There is much less variance in the results, right?
So here I show you these three numbers, and ask what do you expect to happen?
You have no clue, right?
So I don't know, maybe I'll win a lot.
Maybe I'll lose everything.
I show you these three numbers, you're going to look at it and say, well you know, I'm going to be somewhere between around 0 and maybe 1%.
But you're never going to guess it's going to be radically different from that.
And if I were to change this number to be even higher, it would go even closer to 0.
But we won't bother.
OK, so these are the numbers we just looked at, because I said the seed to be the same.
So what's going on here is something called the law of large numbers, or sometimes Bernoulli's law.
This is a picture of Bernoulli on the stamp.
It's one of the two most important theorems in all of statistics, and we'll come to the second most important theorem in the next lecture.
Here it says, "in repeated independent tests with the same actual probability, the chance that the fraction of times the outcome differs from p converges to 0 as the number of trials goes to infinity."
So this says if I were to spin this fair roulette wheel an infinite number of times, the expected-- the return would be 0.
The real true probability from the mathematics.
Well, infinite is a lot, but a million is getting closer to infinite.
And what this says is the closer I get to infinite, the closer it will be to the true probability.
So that's why we did better with a million than with a hundred.
And if I did a 100 million, we'd do way better than I did with a million.
I want to take a minute to talk about a way this law is often misunderstood.
This is something called the gambler's fallacy.
And all you have to do is say, let's go watch a sporting event.
And you'll watch a batter strike out for the sixth consecutive time.
The next time they come to the plate, the idiot announcer says, well he struck out six times in a row.
He's due for a hit this time, because he's usually a pretty good hitter.
Well that's nonsense.
It says, people somehow believe that if deviations from expected occur, they'll be evened out in the future.
And we'll see something similar to this that is true, but this is not true.
And there is a great story about it.
This is told in a book by [INAUDIBLE] and [INAUDIBLE].
And this truly happened in Monte Carlo, with Roulette.
And you could either bet on black or red.
Black came up 26 times in a row.
Highly unlikely, right?
2 to the 26th is a giant number.
And what happened is, word got out on the casino floor that black had kept coming up way too often.
And people more or less panicked to rush to the table to bet on red, saying, well it can't keep coming up black.
Surely the next one will be red.
And as it happened when the casino totaled up its winnings, it was a record night for the casino.
Millions of francs got bet, because people were sure it would have to even out.
Well if we think about it, probability of 26 consecutive reds is that.
A pretty small number.
But the probability of 26 consecutive reds when the previous 25 rolls were red is what?
No, that
Oh, I thought you meant [INAUDIBLE]
No, if you had 25 reds and then you spun the wheel once more, the probability of it having 26 reds is now 0.5, because these are independent events.
Unless of course the wheel is rigged, and we're assuming it's not.
People have a hard time accepting this, and I know it seems funny.
But I guarantee there will be some point in the next month or so when you will find yourself thinking this way, that something has to even out.
I did so badly on the midterm, I will have to do better on the final.
That was mean, I'm sorry.
All right, speaking of means-- see?
Professor [?
Grimm's ?]
not the only one who can make bad jokes.
There is something-- it's not the gambler's fallacy-- that's often confused with it, and that's called regression to the mean.
This term was coined in 1885 by Francis Galton in a paper, of which I've shown you a page from it here.
And the basic conclusion here was-- what this table says is if somebody's parents are both taller than average, it's likely that the child will be smaller than the parents.
Conversely, if the parents are shorter than average, it's likely that the child will be taller than average.
Now you can think about this in terms of genetics and stuff.
That's not what he did.
He just looked at a bunch of data, and the data actually supported this.
And this led him to this notion of regression to the mean.
And here's what it is, and here's the way in which it is subtly different from the gambler's fallacy.
What he said here is, following an extreme event-- parents being unusually tall-- the next random event is likely to be less extreme.
He didn't know much about genetics, and he kind of assumed the height of people were random.
But we'll ignore that.
OK, but the idea is here that it will be less extreme.
So let's look at it in roulette.
If I spin a fair roulette wheel 10 times and get 10 reds, that's an extreme event.
Right, here's a probability of basically 1.1024.
Now the gambler's fallacy says, if I were to spin it another 10 times, it would need to even out.
As in I should get more blacks than you would usually get to make up for these excess reds.
What regression to the mean says is different.
It says, it's likely that in the next 10 spins, you will get fewer than 10 reds.
You will get a less extreme event.
Now it doesn't have to be 10.
If I'd gotten 7 reds instead of 5, you'd consider that extreme, and you would bet that the next 10 would have fewer than 7.
But you wouldn't bet that it would have fewer than 5.
Because of this, if you now look at the average of the 20 spins, it will be closer to the mean of 50% reds than you got from the extreme first spins.
So that's why it's called regression to the mean.
The more samples you take, the more likely you'll get to the mean.
Yes?
So, roulette wheel spins are supposed to be independent
Yes
So it seems like the second 10--
Pardon?
It seems like the second 10 times that you spin it.
that shouldn't have to [INAUDIBLE]
Has nothing to do with the first one
But you said it's likely [INAUDIBLE]
Right, because you have an extreme event, which was unlikely.
And now if you have another event, it's likely to be closer to the average than the extreme was to the average.
Precisely because it is independent.
That makes sense to everybody?
Yeah?
Isn't that the same as the gambler's fallacy, then?
By saying that, because this was super unlikely, the next one [INAUDIBLE]
No, the gambler's fallacy here-- and it's a good question, and indeed people often do get these things confused.
The gambler's fallacy would say that the second 10 spins would-- we would expect to have fewer than 5 reds, because you're trying to even out the unusual number of reds in the first Spin Whereas here we're not saying we would have fewer than 5.
We're saying we'd probably have fewer than 10.
That it'll be closer to the mean, not that it would be below the mean.
Whereas the gambler's fallacy would say it should be below that mean to quote, even out, the first 10.
Does that makes sense?
OK, great questions.
Thank you.
All right, now you may not know this, but casinos are not in the business of being fair.
And the way they don't do that is in Europe, they're not all red and black.
They sneak in one green.
And so now if you bet red, well sometimes it isn't always red or black.
And furthermore, there is this 0.
They index from 0 rather than from one, and so you don't get a full payoff.
In American roulette, they manage to sneak in two greens.
They have a 0 in a double 0.
Tilting the odds even more in favor of the casino.
So we can do that in our simulation.
We'll look at European roulette as a subclass of fair roulette.
I've just added this extra pocket, 0.
And notice I have not changed the odds.
So what you get if you get your number is no higher, but you're a little bit less likely to get it because we snuck in that 0.
Than American roulette is a subclass of European roulette in which I add yet another pocket.
All right, we can simulate those.
Again, nice thing about simulations, we can play these games.
So I've simulated 20 trials of 1,000 spins, 10,000 spins, 100,000, and a million.
And what do we see as we look at this?
Well, right away we can see that fair roulette is usually a much better bet than either of the other two.
That even with only 1,000 spins the return is negative.
And as we get more and more as I got to a million, it starts to look much more like closer to 0.
And these, we have reason to believe at least, are much closer to true expectation saying that, while you break even in fair roulette, you'll lose 2.7% in Europe and over 5% in Las Vegas, or soon in Massachusetts.
All right, we're sampling, right?
That's why the results will change, and if I ran a different simulation with a different seed I'd get different numbers.
Whenever you're sampling, you can't be guaranteed to get perfect accuracy.
It's always possible you get a weird sample.
That's not to say that you won't get exactly the right answer.
I might have spun the wheel twice and happened to get the exact right answer of the return.
Actually not twice, because the math doesn't work out, but 35 times and gotten exactly the right answer.
But that's not the point.
We need to be able to differentiate between what happens to be true and what we actually know, in a rigorous sense, is true.
Or maybe don't know it, but have real good reason to believe it's true.
So it's not just a question of faith.
And that gets us to what's in some sense the fundamental question of all computational statistics, is how many samples do we need to look at before we can have real, justifiable confidence in our answer?
As we've just seen-- not just, a few minutes ago-- with the coins, our intuition tells us that it depends upon the variability in the underlying possibilities.
So let's look at that more carefully.
We have to look at the variation in the data.
So let's look at first something called variance.
So this is variance of x.
Think of x as just a list of data examples, data items.
And the variance is we first compute the average of value, that's mu.
So mu is for the mean.
For each little x and big X, we compare the difference of that and the mean.
How far is it from the mean?
And square of the difference, and then we just sum them.
So this takes, how far is everything from the mean?
We just add them all up.
And then we end up dividing by the size of the set, the number of examples.
Why do we have to do this division?
Well, because we don't want to say something has high variance just because it has many members, right?
So this sort of normalizes is by the number of members, and this just sums how different the members are from the mean.
So if everything is the same value, what's the variance going to be?
If I have a set of 1,000 6's, what's the variance?
Yes?
0
0.
You think this is going to be hard, but I came prepared.
I was hoping this would happen.
Look out, I don't know where this is going to go.
[FIRES SLINGSHOT]
[LAUGHTER]
All right, maybe it isn't the best technology.
I'll go home and practice.
And then the thing you're more familiar with is the standard deviation.
And if you look at the standard deviation is, it's simply the square root of the variance.
Now, let's understand this a little bit and first ask, why am I squaring this here, especially because later on I'm just going to take a square root anyway?
Well squaring it has one virtue, which is that it means I don't care whether the difference is positive or negative.
And I shouldn't, right?
I don't care which side of the mean it's on, I just care it's not near the mean.
But if that's all I wanted to do I could take the absolute value.
The other thing we see with squaring is it gives the outliers extra emphasis, because I'm squaring that distance.
Now you can think that's good or bad, but it's worth knowing it's a fact.
The more important thing to think about is standard deviation all by itself is a meaningless number.
You always have to think about it in the context of the mean.
If I tell you the standard deviation is 100, you then say, well-- and I ask you whether it's big or small, you have no idea.
If the mean is 100 and the standard deviation is 100, it's pretty big.
If the mean is a billion and the standard deviation is 100, it's pretty small.
So you should never want to look at just the standard deviation.
All right, here is just some code to compute those, easy enough.
Why am I doing this?
Because we're now getting to the punch line.
We often try and estimate values just by giving the mean.
So we might report on an exam that the mean grade was 80.
It's better instead of trying to describe an unknown value by it-- an unknown parameter by a single value, say the expected return on betting a roulette wheel, to provide a confidence interval.
So what a confidence interval is is a range that's likely to contain the unknown value, and a confidence that the unknown value is within that range.
So I might say on a fair roulette wheel I expect that your return will be between minus 1% and plus 1%, and I expect that to be true 95% of the time you play the game if you play 100 rolls, spins.
If you take 100 spins of the roulette wheel, I expect that 95% of the time your return will be between this and that.
So here, we're saying the return on betting a pocket 10 times, 10,000 times in European roulette is minus 3.3%.
I think that was the number we just saw.
And now I'm going to add to that this margin of error, which is plus or minus 3.5% with a 95% level of confidence.
What does this mean?
If I were to conduct an infinite number of trials of 10,000 bets each, my expected average return would indeed be minus 3.3%, and it would be between these values 95% of the time.
I've just subtracted and added this 3.5, saying nothing about what would happen in the other 5% of the time.
How far away I might be from this, this is totally silent on that subject.
Yes?
I think you want 0.2 not 9.2
Oh, let's see.
Yep, I do.
Thank you.
We'll fix it on the spot.
This is why you have to come to lecture rather than just reading the slides, because I make mistakes.
Thank you, Eric.
All right, so it's telling me that, and that's all it means.
And it's amazing how often people don't quite know what this means.
For example, when they look at a political pole and they see how many votes somebody is expected to get.
And they see this confidence interval and say, what does that really mean?
Most people don't know.
But it does have a very precise meaning, and this is it.
How do we compute confidence intervals?
Most of the time we compute them using something called the empirical rule.
Under some assumptions, which I'll get to a little bit later, the empirical rule says that if I take the data, find the mean, compute the standard deviation as we've just seen, 68% of the data will be within one standard deviation in front of or behind the mean.
Within one standard deviation of the mean.
95% will be within 1.96 standard deviations.
And that's what people usually use.
Usually when people talk about confidence intervals, they're talking about the 95% confidence interval.
And they use this 1.6 number.
And 99.7% of the data will be within three standard deviations.
So you can see if you are outside the third standard deviation, you are a pretty rare bird, for better or worse depending upon which side.
All right, so let's apply the empirical rule to our roulette game.
So I've got my three roulette games as before.
I'm going to run a simple simulation.
And the key thing to notice is really this print statement here.
Right, that I'll print the mean, which I'm rounding.
And then I'm going to give the confidence intervals, plus or minus, and I'll just take the standard deviation times 1.6 times 100, y times 100, because I'm showing you percentages.
All right so again, very straightforward code.
Just simulation, just like the ones we've been looking at.
And well, I'm just going-- I don't think I'll bother running it for you in the interest of time.
You can run it yourself.
But here's what I got when I ran it.
So when I simulated betting a pocket for 20 trials, we see that the-- of 1,000 spins each, for 1,000 spins the expected return for fair roulette happened to be 3.68%.
A bit high.
But you'll notice the confidence interval plus or minus 27 includes the actual answer, which is 0.
And we have very large confidence intervals for the other two games.
If you go way down to the bottom where I've spun, spun the wheel many more times, what we'll see is that my expected return for fair roulette is much closer to 0 than it was here.
But more importantly, my confidence interval is much smaller, 0.8.
So now I really have constrained it pretty well.
Similarly, for the other two games you will see-- maybe it's more accurate, maybe it's less accurate, but importantly the confidence interval is smaller.
So I have good reason to believe that the mean I'm computing is close to the true mean, because my confidence interval has shrunk.
So that's the really important concept here, is that we don't just guess-- compute the value in the simulation.
We use, in this case, the empirical rule to tell us how much faith we should have in that value.
All right, the empirical rule doesn't always work.
There are a couple of assumptions.
One is that the mean estimation error is 0.
What is that saying?
That I'm just as likely to guess high as gas low.
In most experiments of this sort, most simulations, that's a very fair assumption.
There's no reason to guess I'd be systematically off in one direction or another.
It's different when you use this in a laboratory experiment, where in fact, depending upon your laboratory technique, there may be a bias in your results in one direction.
So we have to assume that there's no bias in our errors.
And we have to assume that the distribution of errors is normal.
And we'll come back to this in just a second.
But this is a normal distribution, called the Gaussian.
Under those two assumptions the empirical rule will always hold.
All right, let's talk about distributions, since I just introduced one.
We've been using a probability distribution.
And this captures the notion of the relative frequency with which some random variable takes on different values.
There are two kinds.
, Discrete and these when the values are drawn from a finite set of values.
So when I flip these coins, there are only two possible values, head or tails.
And so if we look at the distribution of heads and tails, it's pretty simple.
We just list the probability of heads.
We list the probability of tails.
We know that those two probabilities must add up to 1, and that fully describes our distribution.
Continuous random variables are a bit trickier.
They're drawn from a set of reals between two numbers.
For the sake of argument, let's say those two numbers are 0 and 1.
Well, we can't just enumerate the probability for each number.
How many real numbers are there between 0 and 1?
An infinite number, right?
And so I can't say, for each of these infinite numbers, what's the probability of it occurring?
Actually the probability is close to 0 for each of them.
Is 0, if they're truly infinite.
So I need to do something else, and what I do that is what's called the probability density function.
This is a different kind of PDF than the one Adobe sells.
So there, we don't give the probability of the random variable taking on a specific value.
We give the probability of it lying somewhere between two values.
And then we define a curve, which shows how it works.
So let's look at an example.
So we'll go back to normal distributions.
This is-- for the continuous normal distribution, it's described by this function.
And for those of you who don't know about the magic number e, this is one of many ways to define it.
But I really don't care whether you remember this.
I don't care whether you know what e is.
I don't care if you know what this is.
What we really want to say is, it looks like this.
In this case, the mean is 0.
It doesn't have to be 0.
I've [INAUDIBLE] a mean of 0 and a standard deviation of 1.
This is called the so-called standard normal distribution.
But it's symmetric around the mean.
And that gets back to, it's equally likely that our errors are in either direction, right?
So it peaks at the mean.
The peak is always at the mean.
That's the most probable value, and it's symmetric about the mean.
So if we look at it, for example, and I say, what's the probability of the number being between 0 and 1?
I can look at it here and say, all right, let's draw a line here, and a line here.
And then I can integrate the curve under here.
And that tells me the probability of this random variable being between 0 and 1.
If I want to know between minus 1 and 1.
I just do this and then I integrate over that area.
All right, so the area under the curve in this case defines the likelihood.
Now I have to divide and normalize to actually get the answer between 0 and 1.
So the question is, what fraction of the area under the curve is between minus 1 and 1?
And that will tell me the probability.
So what does the empirical rule tell us?
What fraction is between minus 1 and 1, roughly?
Yeah?
68%, right?
So that tells me 68% of the area under this curve is between minus 1 and 1, because my standard deviation is 1, roughly 68%.
And maybe your eyes will convince you that's a reasonable guess.
OK, we'll come back and look at this in a bit more detail on Monday of next week.
And also look at the question of, why does this work in so many cases where we don't actually have a normal distribution to start with?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, everybody.
Welcome back to class.
I have a lot to cover today, so I'm just going to get right to it.
So you may remember at the end of last lecture, I talked about the Empirical Rule and said that there were a couple of assumptions underlying it.
That one is the mean estimation error is zero.
And second, that the distribution of errors will be normally distributed.
I didn't probably mention at the time, but we often call this distribution Gaussian after the astronomer Carl Gauss.
And it looks like that.
Normal distributions are very easy to generate in Python.
I have a little example here of generating, not a real normal distribution, but a discrete approximation of one.
And the thing to really notice about it here is this line random.gauss.
So that's a built in function of the random library.
The first argument is the mean.
And the second argument is the standard deviation or a mu and sigma as they're usually called.
Every time I call that, I will get a different-- or usually a different random value drawn from a Gaussian with the mean, in this case of 0 and a standard deviation of 100.
I'm then going to produce a plot of those so you can see what it looks like.
And I'm going to do that using some things we haven't seen before.
So first of all, we've seen histograms.
So pylab.hist produces a histogram.
Bins tells us how many bins we want in the histogram.
I said we want 100.
The default is 10.
Dist is the values it will use for it.
And then this is something we haven't seen before, weights.
Weights is a keyword argument.
So normally when we produce a histogram, we take all of the values, the minimum to the maximum, and in this case, we would divide them into 100 bins, because I said, bins equals 100.
So the first bin might be, well, let's say we only had values ranging from 0 to 100.
The first bin would be all the 0's, all the 1's up to all the 99's.
And it weights each value in the bin by 1.
So if the bin had 10 values falling in it, the y-axis would be a 10.
If the bin had 50 values falling in it, the y-axis would go up to 50.
You can tell it how much you want to weight each bin, the elements in the bins.
And say, no, I don't want them each to count as 1, I want them to count as a half or a quarter, and that will change the y-axis.
So that's what I've done here.
What I've said is I've created a list and I want to say for each of the bins-- in this case I'm going to weigh each of them the same way-- the weight is going to be 1 over the number of samples.
I'm multiplying it by the len of dist, that will be how many items I have.
And that will tell me how much each one is being weighted.
So for example, if I have, say, 1,000 items, I could give 1,000 values and say, I want this item weighted by 1, and I want this item over here weighted by 12 or a half.
We rarely do that.
Usually, what we want is to give each item the same weight.
So why would I want it not to be weighted at just one?
Because I want my y-axis to be more easily interpreted, and essentially give me the fraction of the values that fell in that bin.
And that's what I'm doing here.
The other new thing I'm doing here is the plotting commands, including pylab.hist, many of them return values.
Usually, I just ignore that value when we just say pylab.plot or pylab.hist.
Here I am taking the value.
The value in this case for a histogram is a tuple of length 2.
The first element is a list or an array, giving me how many items are in each bin.
And the second is the patches used to produce the beautiful pictures we're use to seeing.
So here what I'm going to do is take this value so that I can do this at the end.
Now, why would I want to look at the fraction within approximately 200 of the mean?
What is that going to correspond to in this case?
Well, if I divide 200 by 2 I get 100.
Which happens to be the standard deviation.
So in this case, what I'm going to be looking at is what fraction of the values fall within two standard deviations of the mean?
Kind of a check on the empirical rule, right?
All right, when I run the code I get this.
So it is a discrete approximation to the probability density function.
You'll notice, unlike the previous picture I showed you which was nice and smooth, this is jaggedy.
You would expect it to be.
And again, you can see it's very nice that the peak is what we said the mean should be, 0.
And then it falls off.
And indeed, slightly more than 95% fall within two standard deviations of the mean.
I'm not even surprised that it's a little bit more than 95% because, remember the magic number is 1.96, not 2.
But since this is only a finite sample, I only want it to be around 95.
I'm not going to worry too much whether it's bigger or smaller.
All right?
So random.gauss does a nice job of giving us Gaussian values.
We plotted them and now you can see that I've got the relative frequency.
That's why I see fractions in the y-axis rather than counts.
And that would be because of the way I use the weights command.
All right, let's return to PDFs.
So as we said last time, distributions can be defined by Probability Density Functions, and that gives us the probability of some random variable lying between two values.
It defines a curve where the values in the x-axis lie between the minimum and maximum values of the variable.
And it's the area under the curve-- and we'll come back to this-- between those two points that give us the probability of an example falling in that range.
So now let's look at it for a normal distribution.
You may recall from the last lecture this rather exotic looking formula which defines a PDF for a normal distribution.
And here's some code that's as straight forward an implementation as one could imagine of this formula, OK.
So that now is a value that, given a mu and a sigma and an x, gives me the x associated with that mu and sigma, OK?
And you'll notice there's nothing random about this.
All right, it is giving me the value.
Now, let's go down here.
And I'm going to, for a set of x's, get the set of y's corresponding to that and then plot it.
I'm going to set mu to 0 and sigma to 1, the so-called standard normal distribution.
And I'm going to look at the distribution from minus 4 to 4.
Nothing magic about that other than, as you'll see, it's kind of a place where it asymptotes near 0.
So while x is less than 4, I'll get the x-value, I'll get the y-value corresponding to that x by calling Gaussian, increment x by 0.05 and do that until I'm done.
And then simply, I'll plot the x-values against the y-values and throw a title on it using pylab.title.
All right, code make sense?
Well, this is where I got that beautiful picture we've looked at before.
When I plotted here, it looks, actually quite smooth.
It's not, it's really connecting a bunch of tiny little lines but I made the points close enough together that it looks at least here smooth at this resolution.
So we know what the values on the x-axis are.
Those are the values I happen to want to look at, from minus 4 to 4.
What are the values on the y-axis?
We kind of would like to interpret them as probabilities, right?
But we could be pretty suspicious about that and then if we take this one point that's up here, we say the probability of that single point is 0.4.
Well, that doesn't make any sense because, in fact, we know the probability of any particular point is 0 in some sense, right?
So furthermore, if I chose a different value for sigma, I can actually get this to go bigger than 1 on the y-axis.
So if you take sigma to be say, 0.1-- I think the y-axis goes up to something like 40.
So we know we don't have probabilities in the range 40.
So if these aren't probabilities, what are they?
What are the y values?
Well, not too surprising since I claimed this was a probability density function, they're densities.
Well, what's a density?
This makes sense.
I'll say it and then I'll try and explain it.
It's a derivative of the cumulative distribution function.
Now, why are we talking about derivatives in the first place?
Well, remember what we're trying to say.
If we want to ask, what's the probability of a value falling between here and here, we claim that that was going to be the area under this curve, the integral.
Well, as you know from 18.01, there's a very clear relationship between derivatives and integrals.
And so if we interpret each of these points as a derivative, in some sense the slope here, then we can look at this as the area just by integrating under there.
So to interpret a PDF, we always do it mathematically.
Actually, I do it just by looking at it.
But the only really interesting mathematical questions to ask have to do with area.
Once we have the area, we can, then, talk about the probabilities of some value falling within a region of the curve.
So what's interesting here is not the numbers per se on the y-axis but the shape of the curve, because those numbers have to be related to the numbers on the x-axis, dx, dy right?
We're looking at derivatives.
All right, so now, we have to talk about integration.
I promise you'll only hear about it for another few minutes then we'll leave the topic.
So I mentioned before SciPy as a library that contains a lot of useful mathematical functions.
One of them is integrate.quad.
Well, the integrate part is obvious.
It means integration.
Quad is telling you the algorithm it's choosing to do the integration.
All of these integrals are going to be actual approximations to the real integral.
SciPy is not doing some clever mathematics to get an analytical solution.
It's using a numerical technique to approximate the integral.
And the one here happens to be called quadrature, it doesn't matter.
All right, you can pass it up to four arguments.
You must pass it to function to be integrated, that makes sense.
A number representing the lower limit of the integration-- you need to give it that.
A number representing the upper limit-- you need to give it that.
And then the fourth argument is a tuple supplying all values for the arguments, except the first of the function you are integrating.
I'll show you an example of that on the next slide.
And we'll see that it returns a tuple, an approximation to the result, what it thinks the integral is, and an estimate of how much error there might be in that one.
We'll ignore the error for the moment.
All right, let's look at the code.
So we start by importing scipy.integrate.
That gives us all the different integration methods.
I'm not going to show you the code for Gaussian since I showed it to you a couple of minutes ago.
But I wanted you to remember that it takes three arguments, x, mu, and sigma.
Because when we get down here to the integration, we'll pass at the function Gaussian and then the values that we want to integrate over.
So those will be the values that x can take upon.
And that will change as we go from mu minus the number of standard deviations times sigma to mu plus the number of standard deviations times sigma.
And then, this is the optional fourth argument, the tuple, mu, and sigma.
Why do I need to pass that in?
Because Gaussian is a ternary argument, or a function that takes three values.
And I'm going to integrate over values of x so I have to fix mu and sigma to constants, which is what I'm doing down here.
And then I'll take the zeroth value, which is its estimate of the integral.
All right, so that's the new thing.
The rest of the code is all stuff you've seen.
For t and range number of trials, I'm going to choose a random mu between minus 10 and 10 and a random sigma between 1 and 10.
It doesn't matter what those constants are.
And then for the number of standard deviations in 1, 1.96, and 3, I'm going to integrate Gaussian over that range.
And then we're just going to see how many of them fall within that range.
In some sense, what we're doing is we're checking the empirical rule.
We're saying, take the Gaussian.
I don't care what mu and sigma are.
It doesn't matter.
The empirical rule will still hold, I think.
But we're just checking it here, OK?
Well, here are the results.
So from mu equals 9 and sigma equals 6, I happened to choose those, we'll see the fracture within 1, fraction within 1.96 and 3.
And so for these random mus and sigmas, you can see that all of them-- and you can set them to whatever you want when you get your hand them the code.
Essentially, what we have is, whoops, the empirical rule actually works.
One of those beautiful cases where you can test the theory and see that the theory really is sound.
So there we go.
So why am I making such a big deal of normal distributions?
They have lots of nice mathematical properties, some of which we've already talked about.
But all of that would be irrelevant if we didn't see them.
The good news is they're all over the place.
I've just taken a few here.
Up here we'll see SAT scores.
I would never show that to high school students, or GREs to you guys.
But you can see that they are amazingly well-distributed along a normal distribution.
On down here, this is plotting percent change in oil prices.
And again, we see something very close to a normal distribution.
And here is just looking at heights of men and women.
And again, they clearly look very normal.
So it's really quite impressive how often they occur.
But not everything is normal.
So we saw that the empirical rule works for normal distributions.
I won't say I proved it for you.
I illustrated it for you with a bunch of examples.
But are the outcomes of the spins of a roulette wheel normal?
No.
They're totally uniform, right?
Everything is equally probable-- a 4, a 6, an 11, a 13, double-0 if you're in Las Vegas.
They're all equally probable.
So if I plotted those, I'd basically just get a straight line with everything at 1 over however many pockets there are.
So in that case, why does the empirical rule work?
We saw that we were doing some estimates about returns and we used the empirical rule, we checked it and, by George, it was telling us the truth.
And the reason is because we're not reasoning about a single spin of the wheel but about the mean of a set of spins.
So if you think about it, what we were reasoning about was the return of betting.
If we look at one spin-- well, let's say we bet $1.
The return is either minus 1 because we've lost our dollar.
Or if we get lucky and our pocket happens to come up, it was 36, I think, or 35.
I forget which, OK?
But that's all.
So if we plotted a histogram, we would see a huge peak at minus 1 and a little bump here at 36 and nothing in the middle.
Clearly, not a normal distribution.
But what we're reasoning about is not the return of a single spin but the return of many spins.
If we played 1,000 spins, what is our expected return?
As soon as we end up reasoning, not about a single event but about the mean of something, we can imply something called the Central Limit Theorem.
And here it is.
It's actually for something so important, very simple.
It says that given a sufficiently large sample-- and I love terms like sufficiently large but we'll later put a little meat on that-- the following three things are true.
The means of the samples in a set of samples, the so-called sample means will be approximately normally distributed.
So that says if I take a sample-- and remember, a sample will have multiple examples.
So just to remind people.
A population is a set of examples.
A sample is a subset of the population.
So it too is a set of examples, typically.
If this set is sufficiently large-- certainly 1 is not sufficiently large-- then it will be the case that the mean of the means-- so I take the mean of each sample and then I can now plot all of those means and so take the mean of those, right-- and they'll be normally distributed.
Furthermore, this distribution will have a mean that is close to the mean of the population.
The mean of the means will be close to the mean of the population.
And the variance of the sample means will be close to the variance of the population divided by the sample size.
This is really amazing that this is true and dramatically useful.
So to get some insight, let's check it.
To do that, postulate that we have this kind of miraculous die.
So instead of a die that when you roll it you get a number 1, 2, 3, 4, 5, or 6, this particular die is continuous.
It gives you a real number between 0 and 5, or maybe it's between 1 and 6, OK?
So it's a continuous die.
What we're going to do is roll it a lot of times.
We're going to say, how many die?
And then, how many times are we going to roll that number of die?
So the number of die will be the sample size-- number of dice will be the sample size.
And then we'll take a bunch of samples which I'm calling number of rolls.
And then we'll plot it and I'm just choosing some bins and some colors and some style and various other things just to show you how we use the keyword arguments.
Actually, I said the number of rolls is the number of trials.
But it isn't quite that because I'm going to get the number of trials by dividing the number of rolls by the number of dice.
So if I have more dice, I get to have fewer samples, more dice per sample, all right?
Then we'll just do it.
So it will be between 0 and 5 because random.random returns a number between 0 and 1 and I'm multiplying it by 5.
And then we'll look at the means and we'll plot it all.
Again, we're playing games with weights just to make the plot a little easier to read.
And here's what we get.
If we roll one die, the mean is very close to 2.5.
Well, that's certainly what you'd expect, right?
It's some random number between 0 and 5.
2.5 is a pretty good guess as to what it should average.
And it has a standard deviation of 1.44.
And that's a little harder to guess that that's what it would be.
But you could figure it out with a little math or as I did here with the simulation.
But now, if I roll 50 dice, well, again, the mean is close to 2.5.
It's what you'd expect, right?
I roll 50 die, I get the mean value of those 50.
But look how much smaller the standard deviation is.
More importantly, what we see here is that if we look at the value, the probability is flat for all possible values between 0 and 5 for a single die.
But if we look at the distribution for the means, it's not quite Gaussian but it's pretty close.
Why is it not Gaussian?
Well, I didn't do it an infinite number of times.
Did it quite a few, but not an infinite number.
Enough that you didn't want to sit here while it ran.
But you can see the amazing thing here that when I go from looking at 1 to looking at the mean of 50, suddenly I have a normal distribution.
And that means that I can bring to bear on the problem the Central Limit Theorem.
We can try it for roulette.
Again I'm not going to make you sit through a million trials of 200 spins each.
I'll do it only for fair roulette.
And again, this is a very simple simulation, and we'll see what we get.
And what we see is it's not quite normal, again, but it definitely has that shape.
Now, it's going to be a little bit strange because I can't lose more than one, if I'm betting one.
So it will never be quite normal because it's going to be truncated down on the left side, whereas the tail can be arbitrarily long.
So again, mathematically it can't be normal but it's close enough in the main region, where most of the values lie, that we can get away with applying the empirical rule and looking at answers.
And indeed as we saw, it does work.
So what's the moral here?
It doesn't matter what the shape of the distribution of the original values happen to be.
If we're trying to estimate the mean using samples that are sufficiently large, the CLT will allow us to use the empirical rule when computing confidence intervals.
Even if we go back and look at this anomaly over in the left, what do you think would happen if I, instead of had 200, have, say, 1,000?
What's the probability of the average return being minus 1 of 1,000 bets?
Much smaller than for 100 bets.
To lose 1,000 times in a row is pretty unlikely.
So to get all the way to the left is going to be less likely, and, therefore, the thing will start looking more and more normal as the samples get bigger.
All right, and so we can use the CLT to justify using the empirical rule when we compute confidence intervals.
All right, I want to look at one more example in detail.
This is kind of an interesting one.
You might think that randomness is of no use for, say, finding the value of pi because there's nothing random about that.
Similarly, you might think that randomness was of no use in integrating a function, but in fact, the way those numerical algorithms work is they use randomness.
What you're about to see is that randomness, and indeed things related to Monte Carlo simulations, can be enormously useful, even when you're computing something that is inherently not random like the value of pi here.
And we won't ask you to remember that many digits on the quiz and the exam.
All right, so what's pi?
Well, people have known about pi for thousands and thousands of years.
And what people knew was that there was some constant, we'll call it pi.
It wasn't always called that.
Such that it was equal to the circumference of a circle divided by the diameter, and furthermore, the area was going to be pi r squared.
People knew that way back to the Babylonians and the Egyptians.
What they didn't know is what that value was.
The earliest known estimate of pi was by the Egyptians, on something called the Rhind Papyrus pictured here.
And they estimated pi to be 4 times 8 over 9 squared, or 3.16.
Not so bad.
About 1,100 years later an estimate of pi appeared in the Bible, Kings 7:23.
It didn't say pi is but it described-- I think this was a construction of King Solomon's Temple.
"He made a molten sea, ten cubits from one brim to the other."
it was round.
It was a circle.
"His height was five cubits and a line of 30 cubits did compass it round about."
Well, if you do the arithmetic, what does this imply the value of pi to be?
3, and I'm sure that's what Mike Pence thinks it is.
About 300 years later, Archimedes did a better job of it.
He estimated pi by constructing a 96-sided polygon.
There's a picture of one.
It looks a lot like a circle.
On the left is one with fewer sides.
And what he did is he then-- since it was a polygon he knew how to compute its area or it's circumference and he just counted things up and he said, well, pi is somewhere between 223/71 and 22/7.
And that turns out to actually-- if you take the average of that it will be a really good estimate.
2000 years after Archimedes, the French mathematicians Buffon and Laplace proposed finding the value of pi using what we would today call a Monte Carlo simulation.
They did not have a computer to execute this on.
So what they proposed-- it started with this mathematics.
They took a circle and inscribed it in a square, a square of two units of whatever it is per side.
And therefore we know that the area of the square is 2 times 2 or 4.
We know that the area of the circle is pi r squared.
And since we know that r is 1-- because that's the square it's inscribed in-- we know that the area of a circle is exactly pi.
What Buffon then proposed was dropping a bunch of needles at random-- kind of like when Professor Grimson was sorting things-- and seeing where they land.
Some would land in the square but not in the circle.
Some would land in the circle.
You would ignore any that landed outside the square.
And then they said, well, since they're falling at random, the ratio of the needles in the circle to needles in the square should exactly equal the area of the square over the area of the circle, exactly, if you did an infinite number of needles.
Does that make sense?
Now, given that, you can do some algebra and solve for the area of the circle and say it has to be the area of the square times the number of needles in the circle divided by the needles in the square.
And since we know that the area of the circle is pi that tells us that pi is going to equal four times the needles in the circle.
That's 4 is the area of the square divided by the number of needles in the square.
And so the argument was you can just drop a bunch of these needles, see where they land, add them up and from that you would magically now know the actual value of pi.
Well, we tried a simulation one year in class but rather than using needles we had an archer and we blindfolded him so he would shoot arrows at random and we would see where they ended up.
There's a video of this here if you want to see it.
I'm going to play only the very end of the video which describes the results.
Maybe we should just use Python to build a Monte Carlo simulation instead.
So that's what happened.
After it was all over, Anna came up and gave me some sound advice.
Yeah, I was very proud of that particular shot with the apple.
I had hoped the student would put it on his or her head but no one volunteered.
That was not done blindfolded.
Any rate, so here is the simulation.
So first we have throwing the needles.
For needles in range 1 to number of needles plus 1, we're going to choose the random x or random y and we're going to see whether or not it's in the circle.
And there we will use the Pythagorean theorem to tell us that, all right?
And we'll just do this, and then we're going to return exactly the formula we saw in the previous slide.
Now, comes an interesting part.
We need to get an estimate.
So we start with the number of needles and the number of trials.
For t in range number of trials, we'll get a guess by throwing number of needles.
And then we'll append that guess to our list of estimates.
We'll then compute the standard deviation, get the current estimate which will be the sum of the estimates divided by the len of the estimate, just the mean of the estimate.
And then we'll print it, all right?
So given a number of needles and a number of trials, we'll estimate pi and we'll give you the standard deviation of that estimate.
However, to do that within a certain precision, I'm going to have yet another loop.
And I'm showing you this because we often structure simulations this way.
So what I have here is the number of trials, which is not so interesting, but the precision.
I'm saying, I would like to know the value of pi and I would like to have good reason to believe that the value you give me is within 0.005, in this case, of the true value.
Or maybe more precisely, I would like to know with a confidence of 95% that the true value is within a certain range.
So what this is going to do is it's just going to-- and I should probably use 1.96 instead of 2, but oh, well-- it's just going to keep increasing the number of needles, doubling the number of needles until it's confident about the estimate, confident enough, all right?
So this is a very common thing.
We don't know how many needles we should need so let's just start with some small number and we'll keep going until we're good.
All right, what happens when we run it?
Well, we start with some estimates that when we had 1,000 needles it told us that pi was 3.148 and standard deviation was 0.047 et cetera.
So a couple of things to notice.
One, are my estimates of pi getting monotonically better?
You'd like to think, as I add more needles, my estimates are getting more accurate.
So that's question one.
Are they getting more accurate?
Not monotonically, right?
If we look at it, where are they getting worse?
Well, let's see.
All right, 3.148, well, 3.13 is already worse, right?
So I double the number of needles and I get a worse estimate.
Now, the trend is good.
By the time I get here, I've got a pretty good estimate.
So overall as I look at larger samples, a bigger subset of the population, my estimates are trending towards better but not every time.
On the other hand, let's look at the standard deviations.
What we see here is that the standard deviations are, indeed, getting monotonically better.
Now, there's nothing mathematical guaranteeing that, right?
Because there is randomness here.
But I'm increasing the number of needles by so much that it kind of overrides the bad luck of maybe getting a bad random set.
And we see those are getting better.
So the important thing to see here is not that I happened to get a better estimate, but that I know more about the estimate.
I can have more confidence in the estimate because it's closing in on it.
So the moral here is it's not sufficient to produce a good answer.
I've said this before.
We need to have reason to believe that it is close to the right answer.
So in this case, I'm using the standard deviation and say, given that it's gotten quite small-- you know, 1.96 times 0.002 is indeed a small number.
I could make it smaller if I wanted.
I have good reason to believe I'm close to the right value of pi.
Everyone agree with that?
Is that right?
Not quite, actually.
It would be nice if it were true.
But it isn't.
So let's look at some things.
Is it correct to state that 95% of the time we run this simulation you'll get an estimate of pi between these two values?
I don't expect you to do the arithmetic in your head but the answer is yes.
So that is something we believe is true by the math we've been looking at for the last two lectures.
Next statement, with a probability of 0.95, the actual value of pi is between these two things.
Is that true?
In fact, if I were to say, with a probability of 1, the actual value is pi is between those two values, would it be true?
Yes.
So they are both true facts.
However, only the first of these can be inferred from our simulation.
While the second fact is true, we can't infer it from the simulation.
And to show you that, statistically valid is not the same as true, we'll look at this.
I've introduced a bug in my simulation.
I've replaced the 4 that we saw we needed by 2, now, an easy kind of mistake to make.
And now, if we go to the code-- well, what do you think will happen if we go to the code and run it?
We'll try it.
We'll go down here to the code.
We'll make that a 2.
And what you'll see as it runs is that once again we're getting very nice confidence intervals, but totally bogus values of pi.
So the statistics can tell us something about how reproducible our simulation is but not whether the simulation is an actually, accurate model of reality.
So what do you need to do?
You need to do something like a sanity check.
So here you might look at a polygon and say, well, clearly that's a totally wrong number.
Something is wrong with my code.
OK, so just to wrap-up.
What we've shown is a way to find pi.
This is a generally useful technique.
To estimate the area of any region r, you pick an enclosing region, call it e, such that it's easy to estimate the area of e, and r lies within it.
Pick some random sets of points within e, let f be the fraction and fall within r, multiply e by f and you're done.
So this for example, is a very common way to do integration.
I promised you we'd talk about integration.
So here's a sine of x.
If I want to integrate the sine of x over some region, as done here, all I need to do is pick a bunch of random points, red and black in this case, and look at the ratio of one to the other.
So showing how we can use randomness to again, compute something that is not inherently random.
This is a trick people use over and over and over again when confronted with some situation where it's not easy to solve for things mathematically.
You just do a simulation and if you do it right, you get a very good answer.
All right, we will move on to a different topic on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good afternoon, everybody.
Welcome to Lecture 8.
So we're now more than halfway through the lectures.
All right, the topic of today is sampling.
I want to start by reminding you about this whole business of inferential statistics.
We make references about populations by examining one or more random samples drawn from that population.
We used Monte Carlo simulation over the last two lectures.
And the key idea there, as we saw in trying to find the value of pi, was that we can generate lots of random samples, and then use them to compute confidence intervals.
And then we use the empirical rule to say, all right, we really have good reason to believe that 95% of the time we run this simulation, our answer will be between here and here.
Well, that's all well and good when we're doing simulations.
But what happens when you to actually sample something real?
For example, you run an experiment, and you get some data points.
And it's too hard to do it over and over again.
Think about political polls.
Here was an interesting poll.
How were these created?
Not by simulation.
They didn't run 1,000 polls and then compute the confidence interval.
They ran one poll-- of 835 people, in this case.
And yet they claim to have a confidence interval.
That's what that margin of error is.
Obviously they needed that large confidence interval.
So how is this done?
Backing up for a minute, let's talk about how sampling is done when you are not running a simulation.
You want to do what's called probability sampling, in which each member of the population has a non-zero probability of being included in a sample.
There are, roughly speaking, two kinds.
We'll spend, really, all of our time on something called simple random sampling.
And the key idea here is that each member of the population has an equal probability of being chosen in the sample so there's no bias.
Now, that's not always appropriate.
I do want to take a minute to talk about why.
So suppose we wanted to survey MIT students to find out what fraction of them are nerds-- which, by the way, I consider a compliment.
So suppose we wanted to consider a random sample of 100 students.
We could walk around campus and choose 100 people at random.
And if 12% of them were nerds, we would say 12% of the MIT undergraduates are nerds-- if 98%, et cetera.
Well, the problem with that is, let's look at the majors by school.
This is actually the majors at MIT by school.
And you can see that they're not exactly evenly distributed.
And so if you went around and just sampled 100 students at random, there'd be a reasonably high probability that they would all be from engineering and science.
And that might give you a misleading notion of the fraction of MIT students that were nerds, or it might not.
In such situations we do something called stratified sampling, where we partition the population into subgroups, and then take a simple random sample from each subgroup.
And we do that proportional to the size of the subgroups.
So we would certainly want to take more students from engineering than from architecture.
But we probably want to make sure we got somebody from architecture in our sample.
This, by the way, is the way most political polls are done.
They're stratified.
They say, we want to get so many rural people, so many city people, so many minorities-- things like that.
And in fact, that's probably where the election recent polls were all messed up.
They did a very, retrospectively at least, a bad job of stratifying.
So we use stratified sampling when there are small groups, subgroups, that we want to make sure are represented.
And we want to represent them proportional to their size in the population.
This can also be used to reduce the needed size of the sample.
If we wanted to make sure we got some architecture students in our sample, we'd need to get more than 100 people to start with.
But if we stratify, we can take fewer samples.
It works well when you do it properly.
But it can be tricky to do it properly.
And we are going to stick to simple random samples here.
All right, let's look at an example.
So this is a map of temperatures in the United States.
And so our running example today will be sampling to get information about the average temperatures.
And of course, as you can see, they're highly variable.
And we live in one of the cooler areas.
The data we're going to use is real data-- and it's in the zip file that I put up for the class-- from the US Centers for Environmental Information.
And it's got the daily high and low temperatures for 21 different American cities, every day from 1961 through 2015.
So it's an interesting data set-- a total of about 422,000 examples in the dataset.
So a fairly good sized dataset.
It's fun to play with.
All right, so we're sort of in the part of the course where the next series of lectures, including today, is going to be about data science, how to analyze data.
I always like to start by actually looking at the data-- not looking at all 421,000 samples, but giving a plot to sort of give me a sense of what the data looks like.
I'm not going to walk you through the code that does this plot.
I do want to point out that there are two things in it that we may not have seen before.
Simply enough, I'm going to use numpy.std to get standard deviations instead of my own code for it.
And random.sample to take simple random samples from the population.
random.sample takes two arguments.
The first is some sort of a sequence of values.
And the second is an integer telling you how many samples you want.
And it returns a list containing sample size, randomly chosen distinct elements.
Distinct elements is important, because there are two ways that people do sampling.
You can do sampling without replacement, which is what's done here.
You take a sample, and then it's out of the population.
So you won't draw it the next time.
Or you can do sampling with replacement, which allows you to draw the same sample multiple times-- the same example multiple times.
We'll see later in the term that there are good reasons that we sometimes prefer sampling with replacement.
But usually we're doing sampling without replacement.
And that's what we'll do here.
So we won't get Boston on April 3rd multiple times-- or, not the same year, at least.
All right.
So here's the histogram the code produces.
You can run it yourself now, if you want, or you can run it later.
And here's what it looks like.
The daily high temperatures, the mean is 16.3 degrees Celsius.
I sort of vaguely know what that feels like.
And as you can see, it's kind of an interesting distribution.
It's not normal.
But it's not that far, right?
We have a little tail of these cold temperatures on the left.
And it is what it is.
It's not a normal distribution.
And we'll later see that doesn't really matter.
OK, so this gives me a sense.
The next thing I'll get is some statistics.
So we know the mean is 16.3 and the standard deviation is approximately 9.4 degrees.
So if you look at it, you can believe that.
Well, here's a histogram of one random sample of size 100.
Looks pretty different, as you might expect.
Its standard deviation is 10.4, its mean 17.7.
So even though the figures look a little different, in fact, the means and standard deviations are pretty similar.
If we look at the population mean and the sample mean-- and I'll try and be careful to use those terms-- they're not the same.
But they're in the same ballpark.
And the same is true of the two standard deviations.
Well, that raises the question, did we get lucky or is something we should expect?
If we draw 100 random examples, should we expect them to correspond to the population as a whole?
And the answer is sometimes yeah and sometimes no.
And that's one of the issues I want to explore today.
So one way to see whether it's a happy accident is to try it 1,000 times.
We can draw 1,000 samples of size 100 and plot the results.
Again, I'm not going to go over the code.
There's something in that code, as well, that we haven't seen before.
And that's the ax.vline plotting command.
V for vertical.
It just, in this case, will draw a red line-- because I've said the color is r-- at population mean on the x-axis.
So just a vertical line.
So that'll just show us where the mean is.
If we wanted to draw a horizontal line, we'd use ax.hline.
Just showing you a couple of useful functions.
When we try it 1,000 times, here's what it looks like.
So here we see what we had originally, same picture I showed you before.
And here's what we get when we look at the means of 100 samples.
So this plot on the left looks a lot more like it's a normal distribution than the one on the right.
Should that surprise us, or is there a reason we should have expected that to happen?
Well, what's the answer?
Someone tell me why we should have expected it.
It's because of the central limit theorem, right?
That's exactly what the central limit theorem promised us would happen.
And, sure enough, it's pretty close to normal.
So that's a good thing.
And now if we look at it, we can see that the mean of the sample means is 16.3, and the standard deviation of the sample means is 0.94.
So if we go back to what we saw here, we see that, actually, when we run it 1,000 times and look at the means, we get very close to what we had initially.
So, indeed, it's not a happy accident.
It's something we can in general expect.
All right, what's the 95% confidence interval here?
Well, it's going to be 16.28 plus or minus 1.96 times 0.94, the standard deviation of the sample means.
And so it tells us that the confidence interval is, the mean high temperature, is somewhere between 14.5 and 18.1.
Well, that's actually a pretty big range, right?
It's sort of enough to where you wear a sweater or where you don't wear a sweater.
So the good news is it includes the population mean.
That's nice.
But the bad news is it's pretty wide.
Suppose we wanted it tighter bound.
I said, all right, sure enough, the central limit theorem is going to tell me the mean of the means is going to give me a good estimate of the actual population mean.
But I want it tighter bound.
What can I do?
Well, let's think about a couple of things we could try.
Well, one thing we could think about is drawing more samples.
Suppose instead of 1,000 samples, I'd taken 2,000 or 3,000 samples.
We can ask the question, would that have given me a smaller standard deviation?
For those of you who have not looked ahead, what do you think?
Who thinks it will give you a smaller standard deviation?
Who thinks it won't?
And the rest of you have either looked ahead or refused to think.
I prefer to believe you looked ahead.
Well, we can run the experiment.
You can go to the code.
And you'll see that there is a constant of 1,000, which you can easily change to 2,000.
And lo and behold, the standard deviation barely budges.
It got a little bit bigger, as it happens, but that's kind of an accident.
It just, more or less, doesn't change.
And it won't change if I go to 3,000 or 4,000 or 5,000.
It'll wiggle around.
But it won't help much.
What we can see is doing that more often is not going to help.
Suppose we take larger samples?
Is that going to help?
Who thinks that will help?
And who thinks it won't?
OK. Well, we can again run the experiment.
I did run the experiment.
I changed the sample size from 100 to 200.
And, again, you can run this if you want.
And if you run it, you'll get a result-- maybe not exactly this, but something very similar-- that, indeed, as I increase the size of the sample rather than the number of the samples, the standard deviation drops fairly dramatically, in this case from 0.94 0.66.
So that's a good thing.
I now want to digress a little bit before we come back to this and look at how you can visualize this-- Because this is a technique you'll want to use as you write papers and things like that-- is how do we visualize the variability of the data?
And it's usually done with something called an error bar.
You've all seen these things here.
And this is one I took from the literature.
This is plotting pulse rate against how much exercise you do or how frequently you exercise.
And what you can see here is there's definitely a downward trend suggesting that the more you exercise, the lower your average resting pulse.
That's probably worth knowing.
And these error bars give us the 95% confidence intervals for different subpopulations.
And what we can see here is that some of them overlap.
So, yes, once a fortnight-- two weeks for those of you who don't speak British-- it does get a little bit smaller than rarely or never.
But the confidence interval is very big.
And so maybe we really shouldn't feel very comfortable that it would actually help.
The thing we can say is that if the confidence intervals don't overlap, we can conclude that the means are actually statistically significantly different, in this case at the 95% level.
So here we see that the more than weekly does not overlap with the rarely or never.
And from that, we can conclude that this is actually, statistically true-- that if you exercise more than weekly, your pulse is likely to be lower than if you don't.
If confidence intervals do overlap, you cannot conclude that there is no statistically significant difference.
There might be, and you can use other tests to find out whether there are.
When they don't overlap, it's a good thing.
We can conclude something strong.
When they do overlap, we need to investigate further.
All right, let's look at the error bars for our temperatures.
And again, we can plot those using something called pylab.errorbar.
Lab So what it takes is two values, the usual x-axis and y-axis, and then it takes another list of the same length, or sequence of the same length, which is the y errors.
And here I'm just going to say 1.96 times the standard deviations.
Where these variables come from you can tell by looking at the code.
And then I can say the format, I want an o to show the mean, and then a label.
Fmt stands for format.
errorbar has different keyword arguments than plot.
You'll find that you look at different ways like histograms and bar plots, scatterplots-- they all have different available keyword arguments.
So you have to look up each individually.
But other than this, everything in the code should look very familiar to you.
And when I run the code, I get this.
And so what I've plotted here is the mean against the sample size with errorbars.
And 100 trials, in this case.
So what you can see is that, as the sample size gets bigger, the errorbars get smaller.
The estimates of the mean don't necessarily get any better.
In fact, we can look here, and this is actually a worse estimate, relative to the true mean, than the previous two estimates.
But we can have more confidence in it.
The same thing we saw on Monday when we looked at estimating pi, dropping more needles didn't necessarily give us a more accurate estimate.
But it gave us more confidence in our estimate.
And the same thing is happening here.
And we can see that, steadily, we can get more and more confidence.
So larger samples seem to be better.
That's a good thing.
Going from a sample size of 50 to a sample size of 600 reduced the confidence interval, as you can see, from a fairly large confidence interval here, ran from just below 14 to almost 19, as opposed to 15 and a half or so to 17.
I said confidence interval here.
I should not have.
I should have said standard deviations.
That's an error on the slides.
OK, what's the catch?
Well, we're now looking at 100 samples, each of size 600.
So we've looked at a total of 600,000 examples.
What has this bought us?
Absolutely nothing.
The entire population only contained about 422,000 samples.
We might as well have looked at the whole thing, rather than take a few of them.
So it's like, you might as well hold an election rather than ask 800 people a million times who they're going to vote for.
Sure, it's good.
But it gave us nothing.
Suppose we did it only once.
Suppose we took only one sample, as we see in political polls.
What can we can conclude from that?
And the answer is actually kind of surprising, how much we can conclude, in a real mathematical sense, from one sample.
And, again, this is thanks to our old friend, the central limit theorem.
So if you recall the theorem, it had three parts.
Up till now, we've exploited the first two.
We've used the fact that the means will be normally distributed so that we could use the empirical rule to get confidence intervals, and the fact that the mean of the sample means would be close to the mean of the population.
Now I want to use the third piece of it, which is that the variance of the sample means will be close to the variance of the population divided by the sample size.
And we're going to use that to compute something called the standard error-- formerly the standard error of the mean.
People often just call it the standard error.
And I will be, alas, inconsistent.
I sometimes call it one, sometimes the other.
It's an incredibly simple formula.
It says the standard error is going to be equal to sigma, where sigma is the population standard deviation divided by the square root of n, which is going to be the size of the sample.
And then there's just this very small function that implements it.
So we can compute this thing called the standard error of the mean in a very straightforward way.
We can compute it.
But does it work?
What do I mean by work?
I mean, what's the relationship of the standard error to the standard deviation?
Because, remember, that was our goal, was to understand the standard deviation so we could use the empirical rule.
Well, let's test the standard error of the mean.
So here's a slightly longer piece of code.
I'm going to look at a bunch of different sample sizes, from 25 to 600, 50 trials each.
So getHighs is just a function that returns the temperatures.
I'm going to get the standard deviation of the whole population, then the standard error of the means and the sample standard deviations, both.
And then I'm just going to go through and run it.
So for size and sample size, I'm going to append the standard error of the mean.
And remember, that uses the population standard deviation and the size of the sample.
So I'll compute all the SEMs.
And then I'm going to compute all the actual standard deviations, as well.
And then we'll produce a bunch of plots-- or a plot, actually.
All right, so let's see what that plot looks like.
Pretty striking.
So we see the blue solid line is the standard deviation of the 50 means.
And the red dotted line is the standard error of the mean.
So we can see, quite strikingly here, that they really track each other very well.
And this is saying that I can anticipate what the standard deviation would be by computing the standard error.
Which is really useful, because now I have one sample.
I computed standard error.
And I get something very similar to what I get of the standard deviation if I took 50 samples and looked at the standard deviation of those 50 samples.
All right, so not obvious that this would be true, right?
That I could use this simple formula, and the two things would track each other so well.
And it's not a coincidence, by the way, that as I get out here near the end, they're really lying on top of each other.
As the sample size gets much larger, they really will coincide.
So one, does everyone understand the difference between the standard deviation and the standard error?
No.
OK.
So how do we compute a standard deviation?
To do that, we have to look at many samples-- in this case 50-- and we compute how much variation there is in those 50 samples.
For the standard error, we look at one sample, and we compute this thing called the standard error.
And we argue that we get the same number, more or less, that we would have gotten had we taken 50 samples or 100 samples and computed the standard deviation.
So I can avoid taking all 50 samples if my only reason for doing it was to get the standard deviation.
I can take one sample instead and use the standard error of the mean.
So going back to my temperature-- instead of having to look at lots of samples, I only have to look at one.
And I can get a confidence interval.
That make sense?
OK.
There's a catch.
Notice that the formula for the standard error includes the standard deviation of the population-- the standard deviation of the sample.
Well, that's kind of a bummer.
Because how can I get the standard deviation of the population without looking at the whole population?
And if we're going to look at the whole population, then what's the point of sampling in the first place?
So we have a catch, that we've got something that's a really good approximation, but it uses a value we don't know.
So what should we do about that?
Well, what would be, really, the only obvious thing to try?
What's our best guess at the standard deviation of the population if we have only one sample to look at?
What would you use?
Somebody?
I know I forgot to bring the candy today, so no one wants to answer any questions
The standard deviation of the sample?
The standard deviation of the sample.
It's all I got.
So let's ask the question, how good is that?
Shockingly good.
So I looked at our example here for the temperatures.
And I'm plotting the sample standard deviation versus the population standard deviation for different sample sizes, ranging from 0 to 600 by one, I think.
So what you can see here is when the sample size is small, I'm pretty far off.
I'm off by 14% here.
And I think that's 25.
But when the sample sizes is larger, say 600, I'm off by about 2%.
So what we see, at least for this data set of temperatures-- if the sample size is large enough, the sample standard deviation is a pretty good approximation of the population standard deviation.
Well.
Now we should ask the question, what good is this?
Well, as I said, once the sample reaches a reasonable size-- and we see here, reasonable is probably somewhere around 500-- it becomes a good approximation.
But is it true only for this example?
The fact that it happened to work for high temperatures in the US doesn't mean that it will always be true.
So there are at least two things we should consider to asking the question, when will this be true, when won't it be true.
One is, does the distribution of the population matter?
So here we saw, in our very first plot, the distribution of the high temperatures.
And it was kind of symmetric around a point-- not perfectly.
But not everything looks that way, right?
So we should say, well, suppose we have a different distribution.
Would that change this conclusion?
And the other thing we should ask is, well, suppose we had a different sized population.
Suppose instead of 400,000 temperatures I had 20 million temperatures.
Would I need more than 600 samples for the two things to be about the same?
Well, let's explore both of those questions.
First, let's look at the distributions.
And we'll look at three common distributions-- a uniform distribution, a normal distribution, and an exponential distribution.
And we'll look at each of them for, what is this, 100,000 points.
So we know we can generate a uniform distribution by calling random.random.
Gives me a uniform distribution of real numbers between 0 and 1.
We know that we can generate our normal distribution by calling random.gauss.
In this case, I'm looking at it between the mean of 0 and a standard deviation of 1.
But as we saw in the last lecture, the shape will be the same, independent of these values.
And, finally, an exponential distribution, which we get by calling random.expovariate.
Very And this number, 0.5, is something called lambda, which has to do with how quickly the exponential either decays or goes up, depending upon which direction.
And I'm not going to give you the formula for it at the moment.
But we'll look at the pictures.
And we'll plot each of these discrete approximations to these distributions.
So here's what they look like.
Quite different, right?
We've looked at uniform and we've looked at Gaussian before.
And here we see an exponential, which basically decays and will asymptote towards zero, never quite getting there.
But as you can see, it is certainly not very symmetric around the mean.
All right, so let's see what happens.
If we run the experiment on these three distributions, each of 100,000 point examples, and look at different sample sizes, we actually see that the difference between the standard deviation and the sample standard deviation of the population standard deviation is not the same.
We see, down here-- this looks kind of like what we saw before.
But the exponential one is really quite different.
You know, its worst case is up here at 25.
The normal is about 14.
So that's not too surprising, since our temperatures were kind of normally distributed when we looked at it.
And the uniform is, initially, much better an approximation.
And the reason for this has to do with a fundamental difference in these distributions, something called skew.
Skew is a measure of the asymmetry of a probability distribution.
And what we can see here is that skew actually matters.
The more skew you have, the more samples you're going to need to get a good approximation.
So if the population is very skewed, very asymmetric in the distribution, you need a lot of samples to figure out what's going on.
If it's very uniform, as in, for example, the uniform population, you need many fewer samples.
OK, so that's an important thing.
When we go about deciding how many samples we need, we need to have some estimate of the skew in our population.
All right, how about size?
Does size matter?
Shockingly-- at least it was to me the first time I looked at this-- the answer is no.
If we look at this-- and I'm looking just for the uniform distribution, but we'll see the same thing for all three-- it more or less doesn't matter.
Quite amazing, right?
If you have a bigger population, you don't need more samples.
And it's really almost counterintuitive to think that you don't need any more samples to find out what's going to happen if you have a million people or 100 million people.
And that's why, when we look at, say, political polls, they're amazingly small.
They poll 1,000 people and claim they're representative of Massachusetts.
This is good news.
So to estimate the mean of a population, given a single sample, we choose a sample size based upon some estimate of skew in the population.
This is important, because if we get that wrong, we might choose a sample size that is too small.
And in some sense, you always want to choose the smallest sample size you can that will give you an accurate answer, because it's more economical to have small samples than big samples.
And I've been talking about polls, but the same is true in an experiment.
How many pieces of data do you need to collect when you run an experiment in a lab.
And how much will depend, again, on the skew of the data.
And that will help you decide.
When you know the size, you choose a random sample from the population.
Then you compute the mean and the standard deviation of that sample.
And then use the standard deviation of that sample to estimate the standard error.
And I want to emphasize that what you're getting here is an estimate of the standard error, not the standard error itself, which would require you to know the population standard deviation.
But if you've chosen the sample size to be appropriate, this will turn out to be a good estimate.
And then once we've done that, we use the estimated standard error to generate confidence intervals around the sample mean.
And we're done.
Now this works great when we choose independent random samples.
And, as we've seen before, that if you don't choose independent samples, it doesn't work so well.
And, again, this is an issue where if you assume that, in an election, each state is independent of every other state, and you'll get the wrong answer, because they're not.
All right, let's go back to our temperature example and pose a simple question.
Are 200 samples enough?
I don't know why I chose 200.
I did.
So we'll do an experiment here.
This is similar to an experiment we saw on Monday.
So I'm starting with the number of mistakes I make.
For t in a range number of trials, sample will be random.sample of the temperatures in the sample size.
This is a key step.
The first time I did this, I messed it up.
And instead of doing this very simple thing, I did a more complicated thing of just choosing some point in my list of temperatures and taking the next 200 temperatures.
Why did that give me the wrong answer?
Because it's organized by city.
So if I happen to choose the first day of Phoenix, all 200 temperatures were Phoenix-- which is not a very good approximation of the temperature in the country as a whole.
But this will work.
I'm using random.sample.
I'll then get the sample mean.
Then I'll compute my estimate of the standard error by taking that as seen here.
And then if the absolute value of the population minus the sample mean is more than 1.96 standard errors, I'm going to say I messed up.
It's outside.
And then at the end, I'm going to look at the fraction outside the 95% confidence intervals.
And what do I hope it should print?
What would be the perfect answer when I run this?
What fraction should lie outside that?
It's a pretty simple calculation.
Five, right?
Because if they all were inside, then I'm being too conservative in my interval, right?
I want 5% of the tests to fall outside the 95% confidence interval.
If I wanted fewer, then I would look at three standard deviations.
Instead of 1.96, then I would expect less than 1% to fall outside.
So this is something we have to always keep in mind when we do this kind of thing.
If your answer is too good, you've messed up.
Shouldn't be too bad, but it shouldn't be too good, either.
That's what probabilities are all about.
If you called every election correctly, then your math is wrong.
Well, when we run this, we get this lovely answer, that the fraction outside the 95% confidence interval is 0.0511.
That's exactly-- well, close to what you want.
It's almost exactly 5%.
And if I run it multiple times, I get slightly different numbers.
But they're all in that range, showing that, here, in fact, it really does work.
So that's what I want to say, and it's really important, this notion of the standard error.
When I talk to other departments about what we should cover in 60002, about the only thing everybody agrees on was we should talk about standard error.
So now I hope I have made everyone happy.
And we will talk about fitting curves to experimental data starting next week.
All right, thanks a lot.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, welcome back or welcome, depending on whether you've been away or not.
I'm going to start with two simple announcements.
There is a reading assignment for this lecture, actually for the next two lectures, which is chapter 18.
And on a much happier note, there is no lecture Wednesday because we hope that you're going to be busy preparing to get that tryptophan poisoning as you eat way too much turkey and you fall asleep.
More importantly, I hope you have a great break over Thanksgiving, whether you're here or you're back home or wherever you are.
But no lecture Wednesday.
Topic for today, I'm going to start with seems like-- sorry, what's going to seem like a really obvious statement.
We're living in a data intensive world.
Whether you're a scientist, an engineer, social scientist, financial worker, politician, manager of a sports team, you're spending increasingly larger amounts of time dealing with data.
And if you're in one of those positions, that often means that you're either writing code or you're hiring somebody to write code for you to figure out that data.
And this section of the course is focusing on exactly that issue.
We want to help you understand what you can try to do with software that manipulates data, how you can write code that would do that manipulation of data for you, and especially what you should believe about what that software tells you about data, because sometimes it tells you stuff that isn't exactly what you need to know.
And today we're going to start that by looking at particularly the case where we get data from experiments.
So think of this lecture and the next one as sort of being statistics meets experimental science.
So what do I mean by that?
Imagine you're doing a physics lab, biology lab, a chemistry lab, or even something in sociology or anthropology, you conduct an experiment to gather some data.
It could be measurements in a lab.
It could be answers on a questionnaire.
You get a set of data.
Once you've got the data, you want to think about what can I do with it, and that usually will involve using some model, some theory about the underlying process to generate questions about the data.
What does this data and the model associated with it tell me about future expectations, help me predict other results that will come out of this data.
In the social case, it could be how do I think about how people are going to respond to a poll about who are you voting for in the next election, for example.
Given the data, given the model, the third thing we're typically going to want to do is then design a computation to help us answer questions about the data, run a computational experiment to complement the physical experiment or the social experiment we used to gather the data in the first place.
And that computation could be something deep.
It could be something a little more interesting, depending on how you're thinking about it.
But we want to think about how do we use computation to run additional experiments for us.
So I'm going to start by using an example of gathering experimental data, and I want to start with the idea of a spring.
How would I model a spring?
How would I gather data about a spring?
And how would I write software to help me answer questions about a spring?
So what's spring?
Well, there's one kind of spring, a little hard to model, although it could be interesting what's swimming around in there and how do I think about the ecological implications of that spring.
Here's a second kind of spring.
It's about four or five months away, but eventually we'll get through this winter and get to that spring and that would be nice, but I'm not going to model that one either.
And yes, my jokes are really bad, and yes, you can't do a darn thing about them because I am tenured because-- while I'd like to model these two springs, we're going to stick with the one that you see in physics labs, these kinds of springs, so-called linear springs.
And these are springs that have the property that you can stretch or compress them by applying a force to it.
And when you release them, they literally spring back to the position they were originally.
So we're going to deal with these kinds of springs.
And the distinguishing characteristics of these two springs and others in this class is that that force you require to compress it or stretch it a certain amount-- the amount of force you require varies linearly in the distance.
So if it takes some amount of force to compress it some amount of distance, it takes twice as much force to compress it twice as much of a distance.
It's linearly related.
So each one of these springs-- these kinds of springs has that property.
The amount of force needed to stretch or compress it's linear in that distance.
Associated with these springs there is something called a spring constant-- usually represented by the number k-- that determines how much force do you need to stretch or compress the spring.
Now, it turns out that that spring constant can vary a lot.
The slinky actually has a very low spring constant.
It's one newton per meter.
That spring on the suspension of a motorcycle has a much bigger spring constant.
It's a lot stiffer, 35,000 newtons per meter.
And just in case you don't remember, a newton is the amount of force you need to accelerate a one-kilogram mass one meter per second squared.
We'll come back to that in a second.
But the idea is we'd like to think about how do we model these kinds of springs.
Well, turns out, fortunately for us, that that was done about 300-plus years ago by a British physicist named Robert Hooke.
Back in 1676 he formulated Hooke's law of elasticity.
Simple expression that says the force you need to compress or stretch a spring is linearly related to the distance, d, that you've actually done that compression in, or another way of saying it is, if I compress a spring some amount, the force that's stored in it is linearly related to that distance.
And the negative sign here basically says it's pointing in the opposite direction.
So if I compress, the force is going to push it back out.
If I stretch it, the force is going to push back into that resting position.
Now, this law holds for a wide range of springs, which is kind of nice.
It's going to hold both in biological systems as well as in physical systems.
It doesn't hold perfectly.
There's a limit to how much you can stretch, in particular, a spring before the law breaks down, and maybe you did this as a kid, right.
If you take a slinky and pull it too far apart, it stops working because you've exceeded what's called the elastic limit of the spring.
Similarly, if you compress it too far, although I think you have to compress it a long ways, it'll stop working as well.
So it doesn't hold completely, and it also doesn't hold for all springs.
Only those springs that satisfy this linear law, which are a lot of them.
So, for example, it doesn't apply to rubber bands, it doesn't apply to recurved bows.
Those are two examples of springs that do not obey this linear relationship.
But nonetheless, there's Hooke's law.
And one of the things we can do is say, well, let's use it to do a little bit of reasoning about this spring.
So we can ask the question, how much does a rider have to weigh to compress this spring by one centimeter?
And we've got Hooke's law, and I also gave you a little bit of hint here.
So I told you that this spring has a spring constant of 35,000 newtons per meter.
So I could just plug this in, right, one centimeter, it's 1/100 of a meter times-- so that's the-- there's the spring constant.
There's the amount we're going to compress it.
Do a little math, and that says that the force I need is 350 newtons.
So what's a newton?
A small town in Massachusetts, an interesting cookie, and a force that we want to think about.
I keep telling you guys, the jokes are really bad.
So how do I get force?
Well, you know that.
Mass times acceleration, right, F equals ma.
For acceleration here, I'm going to make an assumption, which is that the spring is basically oriented perpendicular to the earth, so that the acceleration is just the acceleration of gravity, which is roughly 9.8 meters per second squared.
It's basically pulling it down.
So I could plug that back in because remember what I want to do is figure out what's the mass I need.
So for the force, I'm substituting that in.
I've got that expression, mass times 9.8 meters divided by seconds squared is 350 newtons, divide through by 9.8 both sides, do a little bit of math.
And it says that the mass I need is 350 kilograms divided by 9.8.
And that k refers to kilograms, not to the spring constant.
Poor choice of example, but there I am.
And if I do the math, it says I need a rider that weighs 35.68 kilos.
And if you're not big on the metric system, it's actually a fairly light rider.
That's about 79 pounds.
So a 79-pound rider would compress that spring one centimeter.
So we can figure out how to use Hooke's law.
We're thinking about what we want to do with springs.
That's kind of nice.
How will we actually get the spring constant?
It's really valuable to know what the spring constant is.
And just to give you a sense of that, it's not just to deal with things like slinkies.
Atomic force microscopes, need to know the spring constants of the components in order to calibrate them properly.
The force you need to deform a strand of DNA is directly related to the spring constants of the biological structures themselves.
So I'd really like to figure out how do I get them.
How many of you have done this experiment in physics and hated it?
Right.
Well, I don't know if you hated it or not, but you've done it, right?
Standard way to do it is I'd take a spring, I suspend it from some point.
Let it come to a resting position.
And then I put a mass on the bottom of the spring.
It kind of bounces around.
And when it settles, I measure the distance from where it was before I put the mass on to the distance of where it is after I've added the mass.
I measure that distance.
And then I just plug in.
I plug into that formula there.
The force is minus k times d. So k the spring constant is the force, forget the minus sign, divided by the distance, and the force here would be 9.8 meters per second squared or-- kilograms per second squared times the mass divided by d. So I could just plug it in.
In an ideal world, I'd plug it in, I'm done, one measurement.
Not so much, right.
Masses aren't always perfectly calibrated.
Maybe the spring has got not perfect materials in it.
So ideally I'd actually do multiple trials.
I would take different weights, put them on the spring, make the measurements, and just record those.
So that's what I'm going to do, and I've actually done that.
I'm not going to make you do it.
But I get out a set of measurements.
What have I done here?
I've used different masses, all increasing by now 0.05 kilograms, and I've measured the distance that the spring has deformed.
And ideally, these would all have that nice linear relationship, so I could just plug them in and I could figure out what the spring constant is.
So let's take this data and let's plot it.
And by the way, all the code you'll be able to see when you download the file, I'm going to walk through some of it quickly.
This is a simple way to deal with it, and I'm going to back up for a second.
There's my data, and I actually have done this in some ways the wrong order.
These are my independent measures, different masses.
I'm going to plot those along the x-axis, the horizontal axis.
These are the dependent things.
These are the things I'm measuring.
I'm going to plot those along the y-axis.
So I really should have put them in the other order.
So just cross your eyes and make this column go over to that column, and we'll be in good shape.
Let's plot this.
So here's a little file.
Having stored those away in a file, I'm just going to read them in, get data.
Just going to do the obvious thing of read in these things and return two tuples or lists, one for the x values-- or if you like, again going back to it, this set of values, and one for the y values.
Now I'm going to play a little trick that you may have seen before that's going to be handy to me.
I'm going to actually call this function out of the PyLab library called array.
I pass in that tuple, and what it does is it converts it into an array, which is a data structure that has a fixed number of slots in it but has a really nice property I want to take advantage of.
I could do all of this with lists.
But by converting that into array and then giving it the same name xVals and similarly for the yVals, I can now do math on the array without having to write loops.
And in particular right here, notice what I'm doing.
I'm taking xVals, which is an array, multiplying it by a number.
And what that does is it takes every entry in the array, multiplies that entry, and puts it into basically a new version of the array, which I then store into xVals.
If you've programmed in Matlab, this is the same kind of feeling, right.
I can take an array, do something to it, and that's really nice.
So I'm going to scale all of my values, and then I'm going to plot them out some appropriate things.
And if I do it, I get that.
I thought we said Hooke's law was a linear relationship.
So in an ideal world, all of these points ought to lay along a line somewhere, where the slope of the line would tell me the spring constant.
Not so good, right.
And in fact, if you look at it, you can kind of see-- in here you can kind of imagine there's a line there, something funky is going on up here.
And we're going to come back to that at the end of the lecture.
But how do we think about actually finding the line?
Well, we know there's noise in the measurement, so our best thing to do is to say, well, could we just fit a line to this data?
And how would we do that?
And that's the first big thing we want to do today.
We want to try and figure out, given that we've got measurement noise, how do we fit a line to it.
So how do we fit a curve to data?
Well, what we're basically going to try and do is find a way to relate an independent variable, which were the masses, the y values, to the dependent-- sorry, wrong way.
The independent values, which are the x-axis, to the dependent value, what is the actual displacement we're going to see?
So another way of saying it is if I go back to here, I want to know for every point along here, how do I fit something that predicts what the y value is?
So I need to figure out how to do that fit.
To decide-- even if I had a curve, a line that I thought was a good fit to that, I need to decide how good it is.
So imagine I was lucky and somebody said, here's a line that I think describes Hooke's law in this case.
Great.
I could draw the line on that data.
I could draw it on this chunk of data here.
I still need to decide how do I know if it's a good fit.
And for that, we need something we call an objective function, and it's going to measure how close is the line to the data to which I'm trying to fit it.
Once we've defined the objective function, then what we say is, OK, now let's find the line that minimizes it, the best possible line, the line that makes that objective function as small as possible, because that's going to be the best fit to the data.
And so that's what I'd like to do.
We're going to see-- we're going to do it for general curves, but we're going to start just with lines, with linear function.
So in this case, we want to say what's the line such that some function of the sum of the distances from the line to the measured points is minimized.
And I'm going to come back in a second to how do we find the line.
But first we've got to think about what does it mean to measure it.
So I've got a point.
Imagine I got a line that I think is a good match for the thing fitting the data.
How do I measure distance?
Well, there's one option.
I could measure just the displacement along the x-axis.
There's a second option.
I could measure the displacement vertically.
Or a third option is I could actually measure the distance to the closest point on the line, which would be that perpendicular distance there.
You're way too quiet, which is always dangerous.
What do you think?
I'm going to look for a show of hands here.
How many people think we should use x as the thing that we measure here?
Hands up.
Please don't use a single finger when you put your hand up.
All right.
Good.
How many people think we should use p, the perpendicular distance?
Reasonable number of hands.
And how about y?
And I see actually about split between p and y.
And that's actually really good.
X doesn't make a lot of sense, right, because I know that my values along the x-axis are independent measurements.
So the displacement in that direction doesn't make a lot of sense.
P makes a lot of sense, but unfortunately isn't what I want.
We're going to see examples later on where, in fact, minimizing things where you minimize that distance is the right thing to do.
When we do machine learning, that is how you find what's called a classifier or a separator.
But actually here we're going to pick y, and the reason is important.
I'm trying to predict the dependent value, which is the y value, given an independent new x value.
And so the displacement, the uncertainty is, in fact, the vertical displacement.
And so I'm going to use y.
That displacement is the thing I'm going to measure as the distance.
How do I find this?
I need an objective function that's going to tell me what is the closeness of the fit.
So here's how I'm going to do it.
I'm going to have some set of observed values.
Think of it as an array.
I've got some index into them, so the indices are giving me the x values.
And the observed values are the things I've actually measured.
If you want to think of it this way, I'm going to go back to this slide really quickly.
The observed values are the displacements or the values along the y-axis.
Sorry about that.
Let's assume that I have some hypothesized line that I think fits this data, y equals ax plus b. I know the a and the b. I've hypothesized it.
Then predicted will basically say given the x value, the line predicts here's what the y value should be.
And so I'm going to take the difference between those two and square them.
So the difference makes sense.
It tells me how far away is the observed value from what the line predicts it should be.
Why am I squaring it?
Well, there are two reasons.
The first one is that squaring is going to get rid of the sign.
It shouldn't matter if my observed value is some amount above the predicted value or some amount below-- the same amount below the predicted value.
The displacement in direction shouldn't matter.
It's how far away is it.
Now, you could say, well, why not just use absolute value?
And the answer is you could, but we're going to see in a couple of slides that by using the square we get a really nice property that helps us find the best fitting line.
So my objective function here basically says, given a bunch of observed values, use the hypothesized line to predict what the value should be, measure the difference in the y direction-- which is what I'm doing because I'm measuring predicted and observed y values-- square them, sum them all up.
It's called least squares.
That's going to give me a measure of how close that line is to a fit.
In a second, I'll get to how you find the best line.
But this hopefully looks familiar.
Anybody recognize this?
You've seen it earlier in this class.
Boy, that's a terrible thing to ask because you don't even remember the last thing you did in this class other than the problem set
[INAUDIBLE]
Sorry?
Variance
Variance.
Thank you.
Absolutely.
Sorry, I didn't bring any candy today.
That's Professor Guttag.
I got a better arm than he does, but I still didn't bring any candy today.
Yeah, it's variance, not quite.
It's almost variance.
That's the variance times the number of observations, or another way of saying it is if I divided this by the number of observations, that would be the variance.
If I took the square root, it would be the standard deviation.
Why is that valuable?
Because that tells you something about how badly things are dispersed, how much variation there is in this measurement.
And so if it says, if I can minimize this expression, that's great because it not only will find what I hope is the best fit, but it's going to minimize the variance between what I predict and what I measure, which makes intuitive sense.
That's exactly the thing I would like to minimize.
This was built on the assumption that I had a line that I thought was a good fit, and this lets me measure how good a fit I have.
But I still have to do a little bit more.
I have to now figure out, OK, how do I find the best-fitting line?
And for that, we need to come up with a minimization technique.
So to minimize this objective function, I want to find the curve for the predicted values-- this thing here-- some way of representing that that leads to the best possible solution.
And I'm going to make a simple assumption.
I'm going to assume that my model for this predicted curve-- I've been using the example of a line, but we're going to say curve-- is a polynomial.
It's a polynomial and one variable.
The one variable is what are the x values of the samples.
And I'm going to assume that the curve is a polynomial.
In the simplest case, it's a line in case order, and two, it's going to be a parabola.
And I'm going to use a technique called linear regression to find the polynomial that best fits the data, that minimizes that objective function.
Quick aside, just to remind you, I'm sure you remember, so polynomial-- polynomials, either the value is zero, which is really boring, or it is a finite sum of non-zero terms that all have the form c times x to the p. C is a constant, a real number.
P is a power, a non-negative integer.
And this is basically-- x is the free variable that's going to capture this.
So easy way to say it is a line would be represented as a degree one polynomial ax plus b.
A parabola is a second-degree polynomial, ax squared plus bx plus c. And we can go up to higher order terms.
We're going to refer to the degree of the polynomial as the largest degree of any term in that polynomial.
So again, degree one, linear degree two, quadratic.
Now how do I use that?
Well, here's the basic idea.
Let's take a simple example.
Let's assume I'm still just trying to fit a line.
So my assumption is I want to find a degree one polynomial, y equals ax plus b, as our model of the day.
That means for every sample, I'm going to plug in x, and if I know a and b, it gives me the predicted value.
I've already seen that's going to give me a good measure of the closeness of the fit.
And the question is, how do I find a and b.
My goal is find a and b such that when we use this polynomial to compute those y values, that sum squared difference is minimized.
So the sum squared difference is my measure of fit.
All I have to do is find a and b.
And that's where linear regression comes in, and I want to just give you a visualization of this.
If a line is described by ax plus b, then I can represent every possible line in a two-dimensional space.
One axis is possible values for a.
The other axis is possible values for b.
So if you think about it, I take any point in that space.
It gives me an a and a B value.
That describes a line.
Why should you care about that?
Because I can put a two-dimensional surface over that space.
In other words, for every a and b, that gives me a line, and I could, therefore, compute this function, given the observed values and the predicted values, and it would give me a value, which is the height of the surface in that space.
If you're with me with the visualization, why is that nice?
Because linear regression gives me a very easy way to find the lowest point on that surface, which is exactly the solution I want, because that's the best fitting line.
And it's called linear regression not because we're solving for a line, but because of how you do that solution.
If you think of this as being-- take a marble on this two-dimensional surface, you want to place the marble on it, you want to let it run down to the lowest point in the surface.
And oh, yeah, I promised you why do we use sum squares, because if we used the sum of the squares, that surface always has only one minimum.
So it's not a really funky, convoluted surface.
It has exactly one minimum.
It's called linear regression because the way to find it is to start at some point and walk downhill.
I linearly regress or walk downhill along the gradient some distance, measure the new gradient, and do that until I get down to the lowest point in the surface.
Could you write code to do it?
Sure.
Are we going to ask you to do it?
No, because fortunately-- I was hoping to get a cheer out of that.
Too bad.
OK, maybe we will ask you to do it on the exam.
What the hell.
You could do it.
In fact, you've seen a version of this.
The typical algorithm for doing it is very similar to Newton's method that we used way back in the beginning of 60001 when we found square roots.
You could write that kind of a solution, but the good news is that the nice people who wrote Python, or particularly PyLab, have given you code to do it.
And we're going to take advantage of it.
So in PyLab there is a built-in function called polyFit.
It takes a collection of x values, takes a collection of equal length of y values-- they need to be the same length.
I'm going to assume they're arrays.
And it takes an integer n, which is the degree of fit, that I want to apply.
And what polyFit will do is it will find the coefficients of a polynomial of that degree that provides the best least squares fit.
So think of it as polyFit walking along that surface to find the best a and b that will come back.
So if I give it a value of n equals one, it'll give me back the a and b that gives me the best line.
If I get a value of n equal two, it gives me back a, b, and c that would fit an ax squared plus bx plus c parabola to best fit the data.
And I could pick n to be any non-negative integer, and it would actually come up with a good fit.
So let's use it.
I'm going to write a little function called fitData.
The first part up here just comes from plotData.
It's exactly the same thing.
I read in the data.
I convert them into arrays.
I convert this because I want to get out the force.
I go ahead and plot it.
And then notice what I do, I use polyFit right here to take the inputted x values and y values and a degree one, and it's going to give me back a tuple, an a and a b that are the best fit line.
Finds that point in the space that best fits it.
Once I've got that, I could go ahead and actually compute now what are the estimated or predicted values.
The line's going to tell me what I should have seen as those values, and I'm going to do the same thing.
I'm going to take x values, convert it into array, multiply it by a, which says every entry in the array is scaled by a.
Add b to every entry.
So I'm just computing ax plus b for all possible x's.
And that then gives me an estimated set of y values, and I can plot those out.
I'm cheating here.
Sorry.
I'm misdirecting you.
I never cheat.
I actually don't need to do the conversion to an array there because I did it up here.
But because I've borrowed this from plot lab, I wanted to show you that I can redundantly do it here to remind you that I want to convert it into array to make sure I can do that kind of algebra on it.
The last thing I could do is say even if I can-- once I show you the fit of this line, I also want to get out the spring constant.
Now, the slope of this line is difference in force over difference in distance.
The spring constant is the opposite of it.
So I could simply take the slope of the line, which is a, invert it, and that gives me the spring constant.
So let's see what happens if we actually run this.
So I'm going to go over to my code, hoping that it works properly.
Here's my Python.
I've loaded this in.
I'm going to run it.
And there you go.
Fits a line, and it prints out the value of a, which is about 0.46, and the value of b.
And if I go back and look at this, there we go, spring constant is about 21 and a half, which is about the reciprocal of 0.046 if you can figure that out.
And you can see, it's not a bad fit to a line through that data.
Again, there's still something funky going on over here that we're going to come back to.
But it's a pretty good fit to the data.
Great.
So now I've got a fit.
I'm going to show you a variation of this that we're going to use in a second.
I could do the same thing, but after I've done polyFit here, I'm going to use another built-in function called polyval.
It's going to take a polynomial, which is captured by that model of the thing that I returned, and I'm going to show you the difference again.
Back sure we returned this as a tuple.
Since it's coming back as a tuple, I can give it a name model.
Polyval will take that tuple plus the x values and do the same thing.
It will give me back an array of predicted values.
But the nice thing here is that this model could be a line.
It could be a parabola.
It could be a quartic.
It could be a quintic.
It could be any order polynomial.
If you like the abstraction here-- which we're going to see in a little bit, that it allows me to use the same code for different orders of model.
And if I ran this, it would do exactly the same thing.
I'm going to come back to thinking about what's going on in that spring in a second.
But I want to show you another example.
So here's another set of data.
In a little bit, I'll show you where that mystery data came from.
But here's another set of data that I've plotted out.
I could run the same thing.
I could run exactly the same code and fit a line to it.
And if I do it, I get that.
What do you think?
Good fit?
Show of hands, how many people like this fit to the data?
Show of hands, how many people don't like this fit to the data?
Show of hands, how many hope that I'll stop asking you questions?
Don't put your hands up.
Yeah, thank you.
I know.
Too bad.
It's a lousy fit.
And you kind of know it, right.
It's clear that this doesn't look like it's coming from a line, or if it is, it's a really noisy line.
So let's think about this.
What if I were to try a higher order degree.
Let's change the one to a two.
So I'm going to come back to it in a second.
I've changed the one to a two.
That says I'm still using the polynomial fit, but now I'm going to ask what's the best fitting parabola, ax squared plus bx plus c. Simple change.
Because I was using polyval, exactly the same code will work.
It's going to do the fit to it.
This is, by the way, still an example of linear regression.
So think of what I'm doing now.
I have a three-dimensional space.
One axis is a values.
Second axis is b values.
Third axis is c values.
Any point in that space describes a parabola, and every point in that space describes every possible parabola.
And now you've got to twist your head a little bit.
Put a four-dimensional surface on that three-dimensional basis, where the point in that surface is the value of that objective function.
Play the same game.
And you can.
It's just a higher-dimensional thing.
So you're, again, going to walk down the gradient to find the solution, and be glad you don't have to write this code because PyLab will do it for you freely.
But it's still an example of regression, which is great.
And if we do that, we get that fit.
Actually just to show you that, I'm going to run it, but it will do exactly the same thing.
If I go over to Python-- wherever I have it here-- I'm going to change that order of the model.
Oops, it went a little too far for me.
Sorry about that.
Let me go back and do this again.
There's the first one, and there's the second one.
So I could fit different models to it.
Quadratic clearly looks like it's a better fit.
I hope you'll agree.
So how do I decide which one's better other than eyeballing it?
And then if I could fit a quadratic to it, what about other orders of polynomials?
Maybe there's an even better fit out there.
So how do I figure out what's the best way to do the fit?
And that leads to the second big thing for this lecture.
How good are these fits?
What's the first big thing?
The idea of linear regression, a way of finding fits of curves to data.
But now I've got to decide how good are these.
And I could ask this question two ways.
One is just relative to each other, how do I measure which one's better other than looking at it by eye?
And then the second part of it is in an absolute sense, how do I know where the best solution is?
Is quadratic the best I could do?
Or should I be doing something else to try and figure out a better solution, a better fit to the data?
The relative fit.
What are we doing here?
We're fitting a curve, which is a function of the independent variable to the dependent variable.
What does it mean by that?
I've got a set of x values.
I'm trying to predict what the y values should be, the displacement should be.
I want to get a good fit to that.
The idea is that given an independent value, it gives me an estimate of what it should be, and I really want to know which fit provides the better estimates.
And since I was simply minimizing mean squared error, average square error, an obvious thing to do is just to use the goodness of fit by looking at that error.
Why not just measure where am I on that surface and see which one does better?
Or actually it would be two surfaces, one for a linear fit, one for a quadratic one.
We'll do what we always do.
Let's write a little bit of code.
I can write something that's going to get the average, mean squared error.
Takes in a set of data points, a set of predicted values, simply measures the difference between them, squares them, adds them all up in a little loop here and returns that divided by the number of samples I have.
So it gives me the average squared error.
And I could do it for that first model I built, which was for a linear fit, and I could do it for the second model I built, which is a quadratic fit.
And if I run it, I get those values.
Looks pretty good.
You knew by eye that the quadratic was a better fit.
And look, this says it's about six times better, that the residual error is six times smaller with the quadratic model than it is the linear model.
But with that, I still have a problem, which is-- OK, so it's useful for comparing two models.
But is 1524 a good number?
Certainly better than 9,000-something or other.
But how do I know that 1524 is a good number?
How do I know there isn't a better fit out there somewhere?
Well, good news is we're going to be able to measure that.
It's hard to know because there's no bound on the values.
And more importantly, this is not scale independent.
What do I mean by that?
If I take all of the values and multiply them by some factor, I would still fit the same models to them.
They would just scale.
But that measure would increase by that amount.
So I could make the error as big or as small as I want by just changing the size of the values.
That doesn't make any sense.
I'd like a way to measure goodness of fit that is scale independent and that tells me for any fit how close it comes to being the perfect fit to the data.
And so for that, we're going to use something called the coefficient of determination written as r squared.
So let me show you what this does, and then we're going to use it.
The y's are measured values.
Those are my samples I got from my experiment.
The p's are the predicted values.
That is, for this curve, here's what I predict those values should be.
So the top here is basically measuring as we saw before the sum squared error in those pieces.
Mu down here is the average, or mean, of the measured values.
It's the average of the y's.
So what I've got here is in the numerator-- this is basically the error in the estimates from my curve fit.
And in the denominator I've got the amount of variation in the data itself.
This is telling me how much does the data change from just being a constant value, and this is telling me how much do my errors vary around it.
That ratio is scale independent because it's a ratio.
So even if I increase all of the values by some amount, that's going to divide out, which is kind of nice.
So I could compute that, and there it is.
R squared is, again, that expression.
I'll take in a set of observed values, a set of predicted values, and I'll measure the error-- again, these are arrays.
So I'm going to take the difference between the arrays.
That's going to give me piecewise or pairwise that difference.
I'll square it.
That's going to give me at every point in the array the square of that distance.
And then because it's an array, I can just use the built-in sum function to add them all up.
So this is going to give me the-- if you like, the values up there.
And then I'm going to play a little trick.
I'm going to compute the mean error, which is that thing divided by the number of observations.
Why would I do that?
Well, because then I can compute this really simply.
I could write a little loop to compute it.
But in fact, I've already said what is that?
If I take that sum and divide it by the number of samples, that's the variance.
So that's really nice.
Right here I can say, get the variance using the non-p version of the observed data.
And because that has associated with it division by the number of samples, the ratio of the mean error to the variance is exactly the same as the ratio of that to that.
Little trick.
It lets me save doing a little bit of computation.
So I can compute r squared values.
So what does r squared actually tell us?
What we're doing is we're trying to compare the estimation errors, the top part, with the variability in the original values, the bottom part.
So r squared, as you're going to see there, it's intended to capture what portion of the variability in the data is accounted for by my model.
My model's a really good fit.
It should account for almost all of that data.
So what we see then is if we do a fit with a linear regression, r squared is always going to be between zero and one.
And I want to just show you some examples.
If r squared is equal to one, this is great.
It says the model explains all of the variability in the data.
And you can see it if we go back here.
How do we make r squared equal to one?
We need this to be zero, which says that the variability in the data is perfectly predicted by my model.
Every point lies exactly along the curve.
That's great.
Second option at the other extreme is if r squared is equal to zero, you basically got bupkis, which is a well-known technical term, meaning there's no relationship between the values predicted by the model and the actual data.
That basically says that all of the variability here is exactly the same as all the variability in the data.
The model doesn't capture anything, and it's making this one, which is making the whole thing zero.
And then in between an r squared of about a half says you're capturing about half the variability.
So what you would like is a system in which your fit is as close to an r squared value of one as possible because it says my model is capturing all the variability in the data really well.
So two functions that will do this for us.
We're going to come back to these in the next lecture.
The first one called generate fits, or genFits, will take a set of x values, a set of y values, and a list or a tuple of degrees, and these will be the different degrees of models I'd like to fit.
I could just give it one.
I could give it two.
I could give a 1, 2, 4, 8, 16, whatever.
And I'll just run through a little loop here where I'm going to build up a set of models for each degree-- or d in degrees.
I'll do the fit exactly as I had before.
It's going to return a model, which is a tuple of coefficients.
And I'm going to store that in models and then return it.
And then I'm going to use that, because in testFits I will take the models that come from genFits, I'll take the set of degrees that I also passed in there as well as the values.
I'll plot them out, and then I'll simply run through each of the models and generate a fit, compute the r squared value, plot it, and then print out some data.
With that in mind, let's see what happens if we run this.
So I'm going to take, again, that example of that data that I started with, assuming I picked the right one here, which I think is this one.
I'm going to do a fit with a degree one and a degree two curve.
So I'm going to fit the best line.
I'm going to fit the best quadratic, the best parabola, and I want to see how well that comes out.
So I do that.
I got some data there.
Looks good.
And what does the data tell me?
Data says, oh, cool-- I know you don't believe it, but it is because notice what it says, it says the r squared value for the line is horrible.
It accounts for less than 0.05% of the data.
You could say, OK, I can see that.
I look at it.
It does a lousy job.
On the other hand, the quadratic is really pretty good.
It's accounting for about 84% of the variability in the data.
This is a nice high value.
It's not one, but it's a nice high value.
So this is now reinforcing what I already knew, but in a nice way.
It's telling me that that r squared value tells me that the quadratic is a much better fit than the linear fit was.
But then you say maybe, wait a minute.
I could have done this by just comparing the fits themselves.
I already saw that.
Part of my goal is how do I know if I've got the best fit possible or not.
So I'm going to do the same thing, but now I'm going to run it with another set of degrees.
I'm going to go over here.
I'm going to take exactly the same code.
But let's try it with a quadratic, with a quartic, an order eight, and an order 16 fit.
So I'm going to take different size polynomials.
As a quick aside, this is why I want to use the PyLab kind of code because now I'm simply optimizing over a 16-dimensional space.
Every point in that 16-dimensional space defines a 16th-degree polynomial.
And I can still use linear regression, meaning walking down the gradient, to find the best solution.
I'm going to run this.
And I get out a set of values.
Looks good.
And let's go look at them.
Here is the r squared value for quadratic, about 84%.
Degree four does a little bit better.
Degree eight does a little bit better.
But wow, look at that, degree 16-- 16th order polynomial does a really good job, accounts for almost 97% of the variability in the data.
That sounds great.
Now, to quote something that your parents probably said to you when you were much younger, just because something looks good doesn't mean we should do it.
And in fact, just because this has a really high r squared value doesn't mean that we want to use the order 16th polynomial.
And I will wonderfully leave you waiting in suspense because we're going to answer that question next Monday.
And with that, I'll let you out a few minutes early.
Have a great Thanksgiving break.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.mit.edu
Welcome back.
I hope you didn't spend time doing 6002 problem sets while eating turkey.
It's not recommended for digestion.
But I hope you're ready to go back into diving into material.
And since it's been a week since we got together, let me remind you of what we were doing.
We were looking at the issue of how to understand experimental data.
Data could come from a physical experiment.
We had the example of measuring the spring constant of a linear spring.
Could come from biological data.
Could come from social data.
And what we looked out was the idea of how do we actually fit models to that data in order to understand them.
So what I want to do is I want to start with that high level reminder of what we were after.
I want to do about a five minute recap of what we were doing last time, because it has been a while.
And then we're going to talk about how do you actually validate models that you're fitting to data to understand are they really good fits or not.
And if you remember.
I know you spend all your time thinking about 6002.
You should remember I left you with a puzzle, where I fit data to-- sorry, fit models to some noisy data.
And there was a question of, did the model really have an order 16 fit?
Right, so what are we trying to do?
Remember, our goal is to try and model experimental data.
And really what we want to do is have a model that both explains the phenomena underlying what we see, gives us a sense of what might be the underlying physical mechanism, the underlying social mechanism, and can let us make predictions about the behavior in new settings.
In the case of my spring, being able to predict what will the displacement be when I actually put a different weight on it than something I measured.
Or if you want to think from a design perspective, working the other direction and saying, I don't want my spring to deflect more than this amount under certain kinds of weights.
So how do I use the model to tell me what the spring constant should be for the spring I want in that case?
So we want to be able to predict behavior in new settings.
The last piece we know is that, if the data was perfect, this is easy.
But it ain't.
There's always going to be noise.
There's always going to be experimental uncertainty.
And so I really want to account for that uncertainty when I fit that model.
And while sometimes I'll have theories that will help-- Hooke says models of springs are linear-- in some cases, I don't.
And in those cases, I want to actually try and figure out what's the best model to fit even when I don't know what the theory tells me.
OK, so quick recap, what do we use to solve this?
So we've got a set of observed values.
My spring case for different displays for different masses I measured the displacements.
Those displacements are my observed values.
And if I had a model that would predict what the displacement should be, I can measure how good the fit is by looking at that expression right there, the sum of the squares of the differences between the observed and the predicted data.
As I said, we could use other measures.
We could use a first order and absolute value.
The square is actually really handy, because it makes the solution space very easy to deal with, which we'll get to in a second.
So given observed data, get a prediction.
I can use the sum of squared differences to measure how good the fit is.
And then the second piece is I now what to find what's the best way to predict the data.
What's the best curve that fits the data.
What's the best model for protecting the values.
And we suggest that last time, we'll focus on mathematical expressions, polynomials.
Professor Guttag is so excited about polynomial expressions, he's throwing laptops on the floor.
Please don't do that to your laptop.
We're going to fit polynomials to these expressions.
And since the polynomials have some coefficients, the game is basically, how do I find the coefficients of the polynomial that minimize that expression.
And that, we said, was an example of linear regression.
So let me just remind you what linear regression says.
Simple example, case of the spring.
I'm going to get a degree 1 polynomial.
So that is something of the form y is ax plus b. a and b are the three variables, the parameters I can change.
And the idea is for every x, in the case of my spring, for every mass, I'm going to use that model to predict what's the displacement, measure the differences, and find the thing that minimizes it.
So I just want to find values of a and b that let me predict values that minimize that expression.
As I suggested, you could solve this.
You could write code to do it.
It's a neat little piece of code to write.
But fortunately, PiLab provides that for you.
And I just want to give you the visualization of what we're doing here.
And then we're going to look at examples.
I'm going to try to find the best line.
It's represented by two values, a and b. I could represent all possible lines in a space that has one access with values and the other access with b values.
Every point in that plane defines a line for me.
Now imagine a surface laid over this two dimensional space, where the value or the height of the surface is the value of that objective function at every point.
Don't worry about computing it all, but just imagine I could do that.
And by the way, one of the nice things about doing sum of squares is that surface always has a concave shape.
And now the idea of linear regression is I'm going to start at some point on that surface.
And I'm just going to walk downhill until I get to the bottom.
There will always be one bottom, one point.
And once I get to that point, that a and b value tell me the best line.
So it's called linear regression because I'm linearly walking downhill on this space.
Now I'm doing this for line with two parameters a,b, because it's easy to visualize.
If you're a good mathematician even if you're not, you can generalize this to think about arbitrary dimensions.
So a fourth order surface in a five dimensional space, for example, would solve a cubic example of this.
That's the idea of linear regression.
That's what we're going to use to actually figure out, to find the best solution.
So here was the example I used.
I gave you a set of data.
In about 3 slides, I'm going to tell you where the data came from.
But I give you a set of data.
We could fit the best line to this using that linear regression idea.
And again, last piece of reminder, I'm going to use polyfit from PiLab.
It just solves that linear regression problem.
And I give it a set of x values.
I give a corresponding set of y values, need to be the same number in each case.
And I give it a dimension.
And in this case, one says, find the best fitting line.
It will produce that and return it as a tuple, which I'll store under the name model 1.
And I could plot it out.
So just remind you, polyfit will find the best fitting n dimensional surface, n being that last parameter there, and return it.
In a second, we're going to use polyval, which will say, given that model and a set of x values, predict what the y value should be.
Apply them.
OK, so I fit the line.
What do you think?
Good fit?
Not so much, right?
Pretty ugly.
I mean, you can see it's probably the best-- or not probably.
It is the best fitting line.
It sort of accounts for the variation on either side of it.
But it's not a very good fit.
So then the question is, well why not try fitting a higher order model?
So I could fit a quadratic.
That is a second order model.
y equals ax squared plus bx plus c. Run the same code.
Block that out.
And I get that.
That's the linear model.
There's the quadratic model.
At least my [?
i ?]
our looks a lot better, right?
It looks like it's following that data reasonably well.
OK, I can fit a linear model.
I can fit a quadratic model.
What about higher order models?
What about a fourth order model, an eighth order model, a 644th order model?
How do I know which one is going to be best?
So for that, I'm going to remind you of the last thing we used.
And then we're going to start talking about how to use it further, which is if we try fitting higher order polynomials, do we get a better fit?
And to do that, we need to measure what it means for the data to fit.
If I don't have any other information.
For example, if I don't have a theory that tells me this should be linear in the case afoot, then the best way to do it is to use what's called, the coefficient of determination, r-squared.
It's a scale independent thing, which is good.
By scale independent, I mean if I take all the data and stretch it out, this will still give me back the same value in terms of the fit.
So it doesn't depend on the size of the data.
And what it does is it basically tells me the a value between 0 and 1, how well does this model fit the data.
So just to remind you, in this case, the y's are the measured values, the p's are the predicted values.
That's what my model is saying, for each one of these cases.
And mu down here is the mean or the average of the measured values.
The way to think about this is this top expression here.
Well, that's exactly what I'm trying to minimize, right?
So it's giving me an estimate or a measure of the error in the estimates between what the model says and what I actually measure.
And the denominator down here basically tells me how much does the data vary away from the mean value.
Now here's the idea.
If in fact, I can get this to 0, I can get a model that completely accounts for all the variation in the estimates, that's great.
It says, the model has fit perfectly.
And that means this is 0 so this r value or r squared value is 1.
On the other hand, if this is equal to that, meaning that all of the variation in the estimates accounts for none of the variation in the data, then this is 1 and this goes to 0.
So the idea is that an r-squared value is close to 1 is great.
It says, the model is a good fit to the data.
r-squared value is getting closer to 0, not so good.
OK, so I ran this, fitting models of order 2, 4, 8, and 16.
Now you can see that model 2, that's the green line here.
That's the one that we saw before.
It's basically a parabolic kind of arc.
It kind of follows the data pretty well.
But if I look at those r-squared values.
Wow, look at that.
Order 16 fit accounts for all but 3% of the variation in the data.
It's a great fit.
And you can see.
You can see how it follows.
It actually goes through most, but not quite all, of the data points.
So it's following them pretty well.
OK, so if that's the case, the order 16 fit is really the best fit.
Should we just use it?
And I left you last time with that quote that says, from your parents, right, your mother telling you, just because you can do something doesn't mean you should do something.
I'll leave it at that.
Same thing applies here.
Why are we building the model?
Remember, I said two reasons.
One is to be able to explain the phenomena.
And the second one is to be able to make predictions.
So I want to be able to explain the phenomena in the case of a spring, with things like it's linear and then that gives me a sense of a linear relationship between compression and force.
In this case, a 16th order model, what kind of physical process has an order 16 variation?
Sounds a little painful.
So maybe not a great insight into the process.
But the second reason is I want to be able to predict future behavior of this system.
In the case of this spring, I put a different weight on than I've done before.
I want to predict what the displacement is going to be.
I've done a set of trials for an FDA approval of a drug.
Now I want to predict the effect of a treatment on a new patient.
How do I use the model to help me with that?
One that maybe not so good, currently, I want to predict the outcome of an election.
Maybe those models need to be fixed from, at least, what happened the last time around.
But I need to be able to make the prediction.
So another way of saying it is, a good model both explains the phenomena and let's me make the predictions.
OK, so let's go back, then, to our example.
And before I do it, let me tell you where that data came from.
I actually built that data by looking at another kind of physical phenomenon.
And it was a lot of them.
Things that follow a parabolic arc.
So for example, comets.
Any particle under the influence of a uniform gravitational field follows a parabolic arc, which i why Halley's comet gets really close for a while, and then goes away off into the solar system, and comes back around.
My favorite example-- I'm biased on this.
And I know you all know which team I root for.
But there is Tom Brady throwing a pass against the Pittsburgh Steelers.
Center of mass of the past follows a nice parabolic arc.
Even in design, you see parabolic arcs in lots of places.
They have nice properties in terms of disbursement of loads and forces, which is why architects like to use them.
So here's how I generated the data.
I wrote a little function.
Actually, I didn't.
Professor Guttag did, but I borrowed it.
It took in three parameters, a, b, and c. ax squared plus bx plus c. I gave it a set of x values.
Those are the independent measurements, the things along the horizontal axis.
And notice what I did.
I generated values given an a, b, and c, for that equation.
And then I added in some noise.
So random.guass takes a mean and a standard deviation, and it generates noise following that bell shaped curve that goes in the distribution.
So the 0 says it's 0 mean, meaning there's no bias.
It's going to be equally likely to be above or below the value, positive or negative.
But 35 is a pretty good standard deviation.
This is putting a lot of noise into the data.
And then I just added that into y values.
The rest of this, you can see, it's simply going to write it into a file, a set of x and y values.
But this will generate, given a value for a, b, and c, data from a parabolic arc with noise added to it.
And in this case, I took it as y equals 3x squared.
And then c and c are 0.
And that's how I generated it.
What I want to do, I want to see how well this model actually predicts behavior.
So one of the ways I could do it, to say, all right, the question I want to ask is, whoa, if I generated the data from a degree 2 polynomial quadratic, why in the world is the 16th order polynomial the, "best fit?"
So let's test it out.
I'm going to give 3-- sorry, 4.
I can't count.
4 different degrees, order 2, order 4, order 8, order 16.
And I've generated two different datasets, using exactly that code.
I just ran it twice.
It's going to have slightly different values, because the noise is going to be different in each case.
But they're both coming from that a, y equals 3x squared equation.
And the code here basically says, I'm going to take those two data sets and basically, get the x and y values out, and then fit models.
So I'll remind you, genFits takes in a collection of x and y values and a list or a tuple of degrees, and for each degree, finds, using Polyfit, the best model.
So models one will be 4 models for order 2, 4, 8, and 16.
And similarly, down here, I'm going to do the same thing, but using the second data set.
And I'm going to fit, again, a set of models.
And then I'll remind you, test fits, which you saw last time.
I know it's a while ago, basically takes a set of models, a corresponding set of degrees, x and y values, and says, for each model in that degree, measure how well that model meets the fit, using that r-squared value.
So testFits is going to get us back a set of r-squared values.
All right, with that in mind, I've got the code here.
Let's run it.
And here we go.
I'm going to run that code.
Ha, I get two fits.
Looks good.
Let's look at the values.
So there's the first data set.
All right, the green line still is doing not a bad job.
The purple line, boy, is fitting it really well.
And again, notice here's the best fit.
That's amazing.
That is accounting for all but 0.4% of the variation in the data.
Great fit.
Order 16.
Came from an order 2 thing.
All right, what about the second data set?
Oh, grumph.
It also says order 16 fit is the best fit.
Not quite as good.
It accounts for all but about 2% of the variation.
Again, the green line, the red line, do OK.
But in this case, again, that purple line is still the best fit.
So I've still got this puzzle.
But I didn't quite test what I wanted, right?
I said I want to see how well it predicts new behavior.
Here what I did was I took two datasets, fit the model, and I got two different fits, one for each dataset.
They both fit well for order 16.
But they're not quite right.
OK, so best fitting model is still order 16 but we know it came from an order 2 polynomial.
So how could I will get a handle on seeing how good this model is?
Well, what we're seeing here is coming from training error.
Or another way of saying it is, what we're measuring is how well does the model perform on the data from which it was learned?
How well do I fit the model to the training data?
I want a small training error.
And if you think about it, go back to the first example, when I fit a line to this data, it did not do well.
It was not a good model.
When I fit a quadratic, it was pretty decent.
And then I got better and better as I went on.
So I certainly need at least a small training error.
But it's, to use the mathematical terms, a necessary, but not sufficient condition to get a great model.
I need a small training error, but I really want to make sure that the model is capturing what I'd like.
And so for that, I want to see how well does it do on other gen data, generated from the same process, whether it's weights on springs, different comets besides Haley's comet, different voters than those surveyed when we tried to figure out what's going to happen in an election.
And I'm set up to do that, by using a really important tool called, validation or cross-validation.
We set the stage, and then we're going to do the example.
I'm going to get a set of data.
I want to fit a model to it, actually, different models, different degrees, different kinds of models.
To see how well they work, I want to see how well they predict behavior under other data than that from which I did the training.
So I could do that right here.
I could generate the models from one data set, but test them on the other.
And so in fact, I had one data set.
I build a set of models for the first data set.
I compared how well it did on that data set.
But I could now apply it to the second dataset.
How well does that account for that data set?
And similarly, take the models I built for the second data set, and see how well they predict the points from the first dataset.
What do I expect?
Certainly, expect that the testing error is likely to be larger than the training error, because I train on one set of data.
And that means this ought to be a better way to think about, how well does this model generalize?
How well does it predict other behavior, besides what I started with.
So here's the code I'm going to use.
It's pretty straightforward.
All I want to draw your attention to here is, remember, models one I built by fitting models of degree 2, 4, 8, and 16 to the first data set.
And I'm going to apply those models to the second dataset, x vals 2 and y vals 2.
Similarly, I'm going to take the models built for the second data set, and test them on the first dataset to see how well they fit.
I know you're eagerly anticipating, as I've been setting this up for a whole week.
All right, let's look at what happens when I do this.
I'm going to run it.
And then we'll look at the examples.
If I go back over to Python and this code was distributed earlier, if you want to play with it yourself.
Should be the right place to do it.
I am going to run that code.
Now I get something a little different.
In fact, if I go look at it, here is model one applied to data set 2.
And we can both eyeball it and look at the numbers.
Eyeballing it, there's that green line, still generally following the form of this pretty well.
What about the purple line?
The order 16 degree.
Remember, that's the purple line from model 1, from training set 1.
Wow, this misses a bunch of points, pretty badly.
And in fact, look at the r-squared values.
Order 2 and order 4, pretty good fit, accounts for all but about 14, 13% of the data.
Look what happened to the degree 16, degrees 16 fit.
Way down at last.
0.7.
Last time around it was 0.997.
What about the other direction?
Taking the model built and the second data set, testing it on the first data set.
Again, notice a nice fit for degree 2 and 4, not so good for degree 16.
And just to give you a sense of this, I'm going to go back.
There is the model one case.
There is the model in the other case.
You can see the model that accounts for variation in one doesn't account for the variation in the other, when I look at order 16 fit.
OK, so what this says is something important.
Now I can see.
In fact, if I look back at this, if I were just looking at the coefficient of determination, this says, in order to predict other behavior, I'm better off with an order 2 or maybe order 4 polynomial.
Those r-squared values are both the same.
I happen to know it's order 2, because that's where I generated from.
But that's a whole lot better than order 16.
And what you're seeing here is an example of something that happens a lot in statistics.
And in fact, I would suggest is often misused in fitting data to statistical samples.
It's called overfitting.
And what it means is I've let there be too many degrees of freedom in my model, too many free parameters.
And what it's fitting isn't just the underlying process.
It's also fitting to the noise.
The message I want you to take out of this part of the lecture is, if we only fit the model to training data, and we look at how well it does, we could get what looks like a great fit, but we may actually have come up with far too complex a model.
Order 16 instead of order 2.
And the only way you are likely to detect that is to train on one test set and test on a different.
And if you do that, it's likely to expose whether, in fact, I have done a good job of fitting or whether I have overfit to the data.
There are lots of horror stories in the literature, especially from early days of machine learning of people overfitting to data and coming up with models that they thought wonderfully predicted an effect, and then when it ran on new data really hit the big one.
All right, so this is something you want to try and stay away from.
And the best way to do it is to do validation.
You can see it here, right?
The upper left is my training data, dataset one.
There's the set of models.
This is now taking that and applying it to a different dataset from the same process.
And notice for the degree to polynomial, the coefficient of determination, 0.86, now 0.87.
The fact that it's slightly higher is just accidental.
But it's really about the same level.
It's doing the same kind of drop on the training data and on the test data.
On the other hand, degree 16, coefficient of determination is a wonderful 0.96 here and a pretty awful 9 down there.
And that's a sign that we're not in good shape, when in fact our coefficient of determination drops significantly when we try and handle new data.
OK, so why do we get a better fit on the training data with a higher order model, but then do less well when we're actually handling new data?
Or another way of saying it is, if I started out with, in the case of that with that data, a linear model it didn't fit well, and then I got to a quadratic model, why didn't that quadratic model still say [INAUDIBLE]?
Why was it the case that, as I added more degrees of freedom, I did better.
Or another way of asking it is, can I actually get a worse fit to training data as I increase the model complexity?
And I see at least one negative head shake.
Thank you.
You're right.
I cannot.
Let's look at why.
If I add in some higher order terms, and they actually don't matter.
If I got perfect data, the coefficient will just be 0.
The fit will basically say, this term doesn't matter.
Ignore it.
And that'll work in perfect data.
But if the data is noisy, what the model is going to do is actually start fitting the noise.
And while it may lead to a better r-squared value, it's not really a better fit.
Right, let me show you an example of that.
I'm going to fit a quadratic to a straight line.
Easy thing to do.
But I want to show you the effect of overfitting or adding in those extra terms.
So let me say it a little bit better.
I'm going to start off with this 3, sorry, 3.
I'm doing it again today.
4 simple values, 0, 1, 2, 3.
The y values are the same as x values.
So this is 0,0, 1, 1, 2, 2, 3 3.
They're all lying on a line.
But I'm going to fit.
I'm going to plot them out.
And then I'm going to fit a quadratic.
y if it equals ax squared plus bx plus c to this.
Now I know it's a line, but I want to see what happens if I fit a quadratic.
So I'm going to use polyfit to fit my quadratic.
I'm going to print out some data about it.
And then I'm going to use Polyval to estimate what those values should be.
Plot them out.
And then compute r squared value, and see what happens.
All right, OK, and let me set this up better.
What am I doing?
I want to just fit it to a line.
I know it's a line, but I'm going to fit a quadratic to it.
And what I'd expect is, even though there's an extra term there, it shouldn't matter.
So if I go to Python, and I run this, I run exactly that example, look at that.
a equals 0, b is 1, c equals 0.
Look at the r-squared value.
I'll pull that together for you.
It says, in this perfect case, there's what I get.
The blue line is drawn through the actual values.
The dotted red line is drawn through the predicted values.
They exactly line up.
And in fact, the solution implied says, the higher order term coefficient 0, it doesn't matter.
So what it found was y equals x. I know you're totally impressed I could find a straight line.
But notice what happened there.
I dropped or that system said, you don't need the higher order term.
Wonderful r-squared value.
OK, let's see how well it predicts.
Let's add in one more point, out at 20.
So this is 0, 1, 2, 3.
That's 0, 1, 2, 3.
I'm going to add 20 in there, so it's 0, 0 , 1, 2, 2, 3, 3, 20, 20.
Again, I can estimate using the same model.
So I'm not recomputing the model, the model I predicted from using those first set of four points.
I can get the estimated y values, plot those out, and you again, compute the r-squared value here.
And even adding that point in, there's the line.
And guess what.
Perfectly predicts it.
No big surprise.
So it says, in the case of perfect data, adding the higher order terms isn't going to cause a problem.
The system will say coefficients are 0.
That's all I need.
All right, now, let's go back and add in just a tiny bit of noise right there.
0, 0, 1, 1, 2, 2, and 3, 3.1.
So I've got a slight deviation in the y value there.
Again, I can plot them.
I'm going to fit a quadratic to them.
I'm going to print out some information about it and then get the estimated values using that new model to see what it should look like.
I'm not going to run it.
I'm going to show you the result.
I get a really good r-squared value.
And there's the equation it comes up with.
Not so bad, right?
It's almost y equal to x.
But because of that little bit of noise there, there's a small second order term here and a little constant term down there.
The y squared value is really pretty good.
And if you really squint and look carefully at this, you'll actually see there's a little bit of a deviation between the red and the blue line.
It undershoots-- sorry, overshoots there, undershoots here, but it's really pretty close.
All right, so am I just whistling in the dark here?
What's the difference?
Well, now let's add in that extra point.
And what happens?
So again, I'm now taking the same set of points 0, 0, 1, 1, 2, 2, 3, and 3.1.
I'm going to do 20, 20.
Using the model I captured from fitting to that first set, I want to see what happens here.
Crap.
I'm sorry.
Shouldn't say that.
Darn.
Pick up some other word.
Shouldn't surprise you, right?
A small variation here is now causing a really large variation up there.
And this is why the ideal case overfitting is not a problem, because the coefficients get zeroed out.
But even a little bit of noise can cause a problem.
Now I'll grant you, we set this up deliberately to show a big effect here.
But a 3% error in one data point is causing a huge problem when I get further out on this curve.
And by the way, there is the r-squared values.
It's 0.7.
It doesn't do a particularly good job OK, so how would I fix this?
Well, what if I had simply done a first degree fit, same situation.
Let's say fit a line to this rather than fitting a quadratic.
Remember, my question was, what's the harm of fitting a higher order model if the coefficients would be zeroed out?
We've seen they won't be zeroed out.
But if I were just to have fit a line to this, exactly the same experiment, 0, 0, 1, 1, 2, 2, 3, and 3.1, 20 and 20.
Now you can see it still does a really good job of fitting.
The r-squared value is 0.9988.
So again, fitting the right level of model, the noise doesn't cause nearly as much of a problem.
And so just to pull that together, basically it says, the predictive ability of the first order model is much better than the second order model.
And that's why, in this case, I would want to use that first order model.
So take home message.
And then we're going to amplify this.
If I pick an overly complex model, I have the danger of overfitting to the training data, overfitting meaning that I'm not only fitting the underlying process, I'm fitting the noise.
I get an order 16 model is the best fit when it's in fact, in order 2 model that was generating it.
That increases the risk that it's not going to do well with the data, not what I'd like.
I want to be able to predict what's going to go on well here.
On the other hand.
So that would say, boy, just stick with the simplest possible model.
But there's a trade off here.
And we already saw that when I tried to fit a line to a data that was basically quadratic.
I didn't get a good fit.
So I'd want to find the balance.
An insufficiently complex model won't explain the data well.
An overly complex model will overfit the training data.
So I'd like to find the place where the model is as simple as possible, but still explains the data.
And I can't resist the quote from Einstein that captures it pretty well, "everything should be made as simple as possible, but not simpler."
In the case of where I started, it should be fit to a quadratic, because it's the right fit.
But don't fit more than that, because it's getting overly complex Now how might we go about finding the right model?
We're not going to dwell on this but here is a standard way in which you might do it.
Start with a low order model.
Again, take that data.
Fit a linear model to it.
Look at not only the r-squared value, but see how well it accounts for new data.
Increase the order of the model.
Repeat the process.
And keep doing that until you find a point at which a model does a good job both on the training data and on predicting new data.
An after it starts to fall off, that gives you a point where you might say there's a good sized model.
In the case of this data, whether I would have stopped at a quadratic or I might have used a cubic or a quartic depends on the values.
But I certainly wouldn't have gone much beyond that.
And this is one way, if you don't have a theory to drive you, to think about, how do I actually fit the model the way I would like.
Let's go back to where we started.
We still have one more big topic to do, and we still have a few minutes left.
But let's go back to where we started Hooke's law.
There was the data from measuring displacements of a spring, as I added different weights to the bottom of the spring.
And there's the linear fit.
It's not bad.
There's the quadratic fit.
And it's certainly got a better r-squared value, though.
That could be just fitting to the noise.
But you actually can see, I think, that that green curve probably does a better job of fitting the data.
Well, wait a minute.
Even though the quadratic fit is tighter here, Hooke says, this is linear.
So what's going on?
Well, this is another place where you want to think about your model.
And I'll remind you, in case you don't remember your physics, unless we believe that Hooke was wrong, this should tell us something.
And in particular, Hooke's law says, the model holds until you reach the elastic limit of the spring.
You stretch a slinky too far, it never springs back.
You go beyond that elastic limit.
And that's probably what's happening right up there.
Through here, it's following that linear relationship.
Up at this point, I've essentially broken the spring.
The elastic limit doesn't hold anymore.
And so really, in this case, I should probably fit different models to different segments.
And there's a much better fit.
Linear through the first part and another later line once I hit that elastic limit.
How might I find this?
Well, you could imagine a little search process in which you try and find where's the best place along here to break the data into two sets, fit linear segments to both, and get really good fits for both examples.
And I raise it because that's the kind of thing you've also seen before.
You could imagine writing code to do that search to find that good fit.
OK, that gives you a sense, then, of why you want to be careful about overfitting, why you want to not just look at the coefficient of determination, but see how well does this predict behavior on new data sets.
Now suppose I don't have a theory, like Hooke, to guide me.
Can I still figure out what's a good model to fit to the data?
And the answer is, you bet.
We're going to use cross-validation to guide the choice of the model complexity.
And I want to show you two examples.
If the data set's small, we can use what's called leave one out cross-validation.
I'll give you a definition of that in a second.
If the data sets bigger than that, we can use k-fold cross-validation.
I'll give you a definition that a second.
Or just what's called, repeated random sampling.
But we can use this same idea of validating new data to try and figure out whether the model is a good model or not.
Leave one out cross-validation.
This is as written in pseudocode, but the idea is pretty simple.
I'm given a dataset.
It's not too large.
The idea is to walk through a number of trials, number trials equal to the size of the data set.
And for each one, take the data set or a copy of it, and drop out one of the samples.
So leave one out.
Start off by leaving out the first one, then leaving out the second one, and then leaving out the third one.
For each one of those training sets, build the model.
For example, by using linear regression.
And then test that model on that data point that you left out.
So leave out the first one, build a model on all of the other ones, and then see how well that model predicts the first one.
Leave out the second one, build a model using all of them but the second one, see how well it predicts the second one.
And just average the result.
Works when you don't have a really large data set, because it won't take too long.
But it's a nice way of actually testing validation.
If the data set's a lot bigger, you can still use the same idea.
You can use what's called, k-fold.
Divide the data set up into k equal sized chunks.
Leave one of them out.
Use the rest to build the model.
And then use that model to predict that first chunk you left out.
Leave out the second chunk, and keep doing it.
Same idea, but now with groups of things rather than just leaving those single data points.
All right, the other way you can deal with it, which has a nice effect to it, is to use what's called, repeated random sampling.
OK, start out with some data set.
And what I'm going to do here is I'm going to run through some number of trials.
I'm going to call that, k. But I'm also going to pick some number of random samples from the data set.
Usually, I think, and as I recall, it is somewhere between reserving 20% to 50% of the samples.
But the idea is again, walk over all of those k trials.
And in each one, pick out at random n elements for the test set.
Use the remainder is the training set.
Build the model on the training set.
And then apply that model to the test set.
So rather than doing k-fold, where I select k, in turn, and keep them.
This is just randomly selecting which ones to pull out.
So I'm going to show you one last example.
Let's look at that idea of, I don't have a model here.
I want to use this idea of cross-validation to try and figure out what's the best possible model.
And for this, I'm going to use a different data set.
The data set here is I want to model or the task here is I want to try model how the mean daily high temperature in the US has varied over about a 55 year period, from '61 to 2015.
Got a set of data.
It's mean-- sorry, the daily high for every day of the year through that entire period.
And what I'm going to do is I'm going to compute the means for each year and plot them out.
And then I'm going to try and fit models to them.
And in particular, I'm going to take a set of different dimensionalities, linear, quadratic, cubic, quartic And in each case, I'm going to run through a trial where I train on one half of the data, test on the other.
There again, is that idea of seeing how well it predicts other data.
Record the coefficient of determination.
And do that and get out an average, and report what I get as the mean for each of those values across each dimensionality.
OK, here we go.
Set a code that's pretty easy to see.
Hopefully, you can just look at it and grok it.
We start off with a boring class, which Professor guttag suggests refers to this lecture.
But it doesn't.
This may be a boring lecture, but it's not a boring class.
This is a great class.
And boy, those jokes are really awful, aren't they?
But here we go.
Simple class that builds temperature data.
This reads in some information, splits it up, and basically, records the high for the day and the year in which I got that.
So for each day, I've got a high temperature for that day.
I'm going to give you back the high temperature and the year in which it was recorded, because I don't care whether it was in January or June.
A little function that opens up a file.
We've actually given you a file, if you want to go look at it.
And simply walk through the file reading it in and returning a big list of all those data objects.
OK, then what I want to do is I want to get the mean high temperature for each year.
Given that data, I'm going to set up a dictionary called, years.
I'm just going to run through a loop through all the data points, where in the dictionary, under that year.
So there a data point.
I use the method get year to get out the year.
At that point, I add in the high temperature corresponding to that data point.
And I'm using that nice little try except loop.
I'll do that, unless I haven't had anything yet for this year, in which case this'll fail.
And I'll simply store the first one in as a list.
So after I've run through this loop in the dictionary, under the year, I have a list of the high temperatures for each day associated with it.
Excuse me.
And then I can just compute the average, that is for each year in the years.
I get that value.
I add them up.
I get the length.
I divide them out.
And I store that in as the average high temperature for the year.
Now I can plot it.
Get the data, get out the information by computing those yearly means, run through a little loop that basically, in the x values, puts in the year, in the y values, puts in the high temperature.
And I can do a plot.
And if I do that, I get that.
I'll let you run this yourself.
Now this is a little bit deceptive, because of the scale I've used here.
But nonetheless, it shows, in the US, over a 55 year period, the mean high day-- I'm sorry.
The mean daily high has gone from about 15.5 degrees Celsius up to about 17 and 1/2.
So what's changed?
Now the question is, how could I model this?
Could I actually get a model that would give me a sense of how this is changing?
And that's why I'm going to use cross-validation.
I'm going to run through a number of trials, 10 trials.
I'm going to try and fit four different models, linear, quadratic, cubic, quartic.
And for each of these dimensions, I'm going to get out a set of r-squared values.
So I'm just going to initialize that dictionary.
an empty list.
Now here is how I'm going to do this.
Got a list of x-values.
Those are years.
Got a list of y values.
Those are average highs, daily highs.
I'm going to create a list of random samples.
So if you haven't seen this before, random.sample says, given this iterator, which you can think of as the collection from 0 up to n minus 1, it's going to select this many or half of them, in this case, of those numbers at random.
So if I give it 0 up to 9, and I say, pick five of them, it will, at random, give me back 5 of those 10 numbers, with no duplicates.
Ah, that's nice.
Because now notice what I can do.
I'm going to set up a training-- sorry, an x and y values for a training set, x and y values for the test set.
And I'm just going to run through a loop here, where if this index is in that list, I'll stick it in the training set.
Otherwise, I'll stick it in the test set.
And then I just return them.
So this is a really nice way of, at random, just splitting the data set into a test set and a training set.
And then finally, I can run over the number of trials I want to deal with.
In each case, get a different training and test set, at random.
And then, for each dimension, do the fit.
There's polyfit on the training x and training y values in that dimension.
Gives you back a model.
I could just check to see how well the training set gets, but I really want to look at, given that model, how well does polyval predict the test set, right?
The model will say, here's what I expect is the values.
I'm going to compare that to the actual values that I saw from the training set, computing that r squared value and adding it in.
And then the last of this just says, I'll run this through a set of examples.
OK, here's what happens if I do that.
I'm not going to run it, although the code will run it.
Let me, again, remind you what I'm doing.
I got a big set of data I'm going to pick out at random, subsets of it, build the model on one part, test it on the other part.
And if I run it, I get a linear fit, quadratic fit, cubic fit, and a quartic fit.
And here's the standard deviation of those samples.
Remember, I've got multiple trials.
I've got 10 trials, in this case.
So this gives me the average over those trials.
And this tells me how much they vary.
What can I conclude from this?
Well, I would argue that the linear fit's probably the winner here.
Goes back to Einstein.
I want the simplest possible model that accounts for it.
And you can see it's got the highest r-squared value, which is already a good sign.
It's got the smallest deviation across the trials, which says it's probably a pretty good fit.
And it's the simplest model.
So linear sounds like a pretty good fit.
Now, why should we run multiple data sets to test this?
I ran 10 trials of each one of these dimensions.
Why bother with it?
Well, notice that those deviations-- I'll go back to it here-- they're pretty good.
They're about an order of magnitude less than the actual mean, which says they're pretty tight, but they're still reasonable size.
And that suggests that, while there's good agreement, the deviations are large enough that you could see a range of variation across the trials.
So in fact, if I had just run one trial, I could have been screwed.
Sorry, oh-- sorry, pick your favorite [INAUDIBLE] here.
[?
Hose ?]
is a Canadian expression, in case you haven't seen it.
Here are the r-squared values for each trial of the linear fit.
And you can see the mean comes up pretty well.
But notice, if I'd only run one trial and I happened to get that one, oh, darn.
That's a really low r-squared value.
And we might have decided, in this case, a different conclusion, that the linear fit was not a good fit.
So this is a way of saying, even in a random sampling, run multiple trials, because it lets you get statistics on those trials, as well as statistics within each trial.
So with any trial, I'm doing a whole bunch of different random samples on measuring those values.
And then, across those trials, I'm seeing what the deviation is.
I'm going to hope my machine comes back, because what I want to do is then pull this together.
What have we done?
Something you're going to use.
We've seen how you can use linear regression to fit a curve to data, 2D, 3D, 6D, however big the data set is.
It gives us a mapping from the independent values to the dependent values.
And that can then be used to predict values associated with the independent values that we haven't seen yet.
That leads, naturally, to both a way to measure, which is r squared, but especially to see that we want to look at how well does that model actually predict new data, because that lets us select the simplest model we can that accounts for the data, but predicts new data in an effective way.
And that complexity can either be based on theory, in the case of Hooke, or in more likely cases, by doing cross-validation to try and figure out which one is the simplest model that still does a good job of predicting out of data behavior.
And with that, I'll see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Welcome back.
You know, it's that time a term when we're all kind of doing this.
So let me see if I can get a few smiles by simply noting to you that two weeks from today is the last class.
Should be worth at least a little bit of a smile, right?
Professor Guttag is smiling.
He likes that idea.
You're almost there.
What are we doing for the last couple of lectures?
We're talking about linear regression.
And I just want to remind you, this was the idea of I have some experimental data.
Case of a spring where I put different weights on measure displacements.
And regression was giving us a way of deducing a model to fit that data.
And In some cases it was easy.
We knew, for example, it was going to be a linear model.
We found the best line that would fit that data.
In some cases, we said we could use validation to actually let us explore to find the best model that would fit it, whether a linear, a quadratic, a cubic, some higher order thing.
So we'll be using that to deduce something about a model.
That's a nice segue into the topic for the next three lectures, the last big topic of the class, which is machine learning.
And I'm going to argue, you can debate whether that's actually an example of learning.
But it has many of the elements that we want to talk about when we talk about machine learning.
So as always, there's a reading assignment.
Chapter 22 of the book gives you a good start on this, and it will follow up with other pieces.
And I want to start by basically outlining what we're going to do.
And I'm going to begin by saying, as I'm sure you're aware, this is a huge topic.
I've listed just five subjects in course six that all focus on machine learning.
And that doesn't include other subjects where learning is a central part.
So natural language processing, computational biology, computer vision robotics all rely today, heavily on machine learning.
And you'll see those in those subjects as well.
So we're not going to compress five subjects into three lectures.
But what we are going to do is give you the introduction.
We're going to start by talking about the basic concepts of machine learning.
The idea of having examples, and how do you talk about features representing those examples, how do you measure distances between them, and use the notion of distance to try and group similar things together as a way of doing machine learning.
And we're going to look, as a consequence, of two different standard ways of doing learning.
One, we call classification methods.
Example we're going to see, there is something called "k nearest neighbor" and the second class, called clustering methods.
Classification works well when I have what we would call labeled data.
I know labels on my examples, and I'm going to use that to try and define classes that I can learn, and clustering working well, when I don't have labeled data.
And we'll see what that means in a couple of minutes.
But we're going to give you an early view of this.
Unless Professor Guttag changes his mind, we're probably not going to show you the current really sophisticated machine learning methods like convolutional neural nets or deep learning, things you'll read about in the news.
But you're going to get a sense of what's behind those, by looking at what we do when we talk about learning algorithms.
Before I do it, I want to point out to you just how prevalent this is.
And I'm going to admit with my gray hair, I started working in AI in 1975 when machine learning was a pretty simple thing to do.
And it's been fascinating to watch over 40 years, the change.
And if you think about it, just think about where you see it.
AlphaGo, machine learning based system from Google that beat a world-class level Go player.
Chess has already been conquered by computers for a while.
Go now belongs to computers.
Best Go players in the world are computers.
I'm sure many of you use Netflix.
Any recommendation system, Netflix, Amazon, pick your favorite, uses a machine learning algorithm to suggest things for you.
And in fact, you've probably seen it on Google, right?
The ads that pop up on Google are coming from a machine learning algorithm that's looking at your preferences.
Scary thought.
Drug discovery, character recognition-- the post office does character recognition of handwritten characters using a machine learning algorithm and a computer vision system behind it.
You probably don't know this company.
It's actually an MIT spin-off called Two Sigma, it's a hedge fund in New York.
They heavily use AI and machine learning techniques.
And two years ago, their fund returned a 56% return.
I wish I'd invested in the fund.
I don't have the kinds of millions you need, but that's an impressive return.
56% return on your money in one year.
Last year they didn't do quite as well, but they do extremely well using machine learning techniques.
Siri.
Another great MIT company called Mobileye that does computer vision systems with a heavy machine learning component that is used in assistive driving and will be used in completely autonomous driving.
It will do things like kick in your brakes if you're closing too fast on the car in front of you, which is going to be really bad for me because I drive like a Bostonian.
And it would be kicking in constantly.
Face recognition.
Facebook uses this, many other systems do to both detect and recognize faces.
IBM Watson-- cancer diagnosis.
These are all just examples of machine learning being used everywhere.
And it really is.
I've only picked nine.
So what is it?
I'm going to make an obnoxious statement.
You're now used to that.
I'm going to claim that you could argue that almost every computer program learns something.
But the level of learning really varies a lot.
So if you think back to the first lecture in 60001, we showed you Newton's method for computing square roots.
And you could argue, you'd have to stretch it, but you could argue that that method learns something about how to compute square roots.
In fact, you could generalize it to roots of any order power.
But it really didn't learn.
I really had to program it.
All right.
Think about last week when we talked about linear regression.
Now it starts to feel a little bit more like a learning algorithm.
Because what did we do?
We gave you a set of data points, mass displacement data points.
And then we showed you how the computer could essentially fit a curve to that data point.
And it was, in some sense, learning a model for that data that it could then use to predict behavior.
In other situations.
And that's getting closer to what we would like when we think about a machine learning algorithm.
We'd like to have program that can learn from experience, something that it can then use to deduce new facts.
Now it's been a problem in AI for a very long time.
And I love this quote.
It's from a gentleman named Art Samuel.
1959 is the quote in which he says, his definition of machine learning is the field of study that gives computers the ability to learn without being explicitly programmed.
And I think many people would argue, he wrote the first such program.
It learned from experience.
In his case, it played checkers.
Kind of shows you how the field has progressed.
But we started with checkers, we got to chess, we now do Go.
But it played checkers.
It beat national level players, most importantly, it learned to improve its methods by watching how it did in games and then inferring something to change what it thought about as it did that.
Samuel did a bunch of other things.
I just highlighted one.
You may see in a follow on course, he invented what's called Alpha-Beta Pruning, which is a really useful technique for doing search.
But the idea is, how can we have the computer learn without being explicitly programmed?
And one way to think about this is to think about the difference between how we would normally program and what we would like from a machine learning algorithm.
Normal programming, I know you're not convinced there's such a thing as normal programming, but if you think of traditional programming, what's the process?
I write a program that I input to the computer so that it can then take data and produce some appropriate output.
And the square root finder really sits there, right?
I wrote code for using Newton method to find a square root, and then it gave me the process of given any number, I'll give you the square root.
But if you think about what we did last time, it was a little different.
And in fact, in a machine learning approach, the idea is that I'm going to give the computer output.
I'm going to give it examples of what I want the program to do, labels on data, characterizations of different classes of things.
And what I want the computer to do is, given that characterization of output and data, I wanted that machine learning algorithm to actually produce for me a program, a program that I can then use to infer new information about things.
And that creates, if you like, a really nice loop where I can have the machine learning algorithm learn the program which I can then use to solve some other problem.
That would be really great if we could do it.
And as I suggested, that curve-fitting algorithm is a simple version of that.
It learned a model for the data, which I could then use to label any other instances of the data or predict what I would see in terms of spring displacement as I changed the masses.
So that's the kind of idea we're going to explore.
If we want to learn things, we could also ask, so how do you learn?
And how should a computer learn?
Well, for you as a human, there are a couple of possibilities.
This is the boring one.
This is the old style way of doing it, right?
Memorize facts.
Memorize as many facts as you can and hope that we ask you on the final exam instances of those facts, as opposed to some other facts you haven't memorized.
This is, if you think way back to the first lecture, an example of declarative knowledge, statements of truth.
Memorize as many as you can.
Have Wikipedia in your back pocket.
Better way to learn is to be able to infer, to deduce new information from old.
And if you think about this, this gets closer to what we called imperative knowledge-- ways to deduce new things.
Now, in the first cases, we built that in when we wrote that program to do square roots.
But what we'd like in a learning algorithm is to have much more like that generalization idea.
We're interested in extending our capabilities to write programs that can infer useful information from implicit patterns in the data.
So not something explicitly built like that comparison of weights and displacements, but actually implicit patterns in the data, and have the algorithm figure out what those patterns are, and use those to generate a program you can use to infer new data about objects, about string displacements, whatever it is you're trying to do.
OK.
So the idea then, the basic paradigm that we're going to see, is we're going to give the system some training data, some observations.
We did that last time with just the spring displacements.
We're going to then try and have a way to figure out, how do we write code, how do we write a program, a system that will infer something about the process that generated the data?
And then from that, we want to be able to use that to make predictions about things we haven't seen before.
So again, I want to drive home this point.
If you think about it, the spring example fit that model.
I gave you a set of data, spatial deviations relative to mass displacements.
For different masses, how far did the spring move?
I then inferred something about the underlying process.
In the first case, I said I know it's linear, but let me figure out what the actual linear equation is.
What's the spring constant associated with it?
And based on that result, I got a piece of code I could use to predict new displacements.
So it's got all of those elements, training data, an inference engine, and then the ability to use that to make new predictions.
But that's a very simple kind of learning setting.
So the more common one is one I'm going to use as an example, which is, when I give you a set of examples, those examples have some data associated with them, some features and some labels.
For each example, I might say this is a particular kind of thing.
This other one is another kind of thing.
And what I want to do is figure out how to do inference on labeling new things.
So it's not just, what's the displacement of the mass, it's actually a label.
And I'm going to use one of my favorite examples.
I'm a big New England Patriots fan, if you're not, my apologies.
But I'm going to use football players.
So I'm going to show you in a second, I'm going to give you a set of examples of football players.
The label is the position they play.
And the data, well, it could be lots of things.
We're going to use height and weight.
But what we want to do is then see how would we come up with a way of characterizing the implicit pattern of how does weight and height predict the kind of position this player could play.
And then come up with an algorithm that will predict the position of new players.
We'll do the draft for next year.
Where do we want them to play?
That's the paradigm.
Set of observations, potentially labeled, potentially not.
Think about how do we do inference to find a model.
And then how do we use that model to make predictions.
What we're going to see, and we're going to see multiple examples today, is that that learning can be done in one of two very broad ways.
The first one is called supervised learning.
And in that case, for every new example I give you as part of the training data, I have a label on it.
I know the kind of thing it is.
And what I'm going to do is look for how do I find a rule that would predict the label associated with unseen input based on those examples.
It's supervised because I know what the labeling is.
Second kind, if this is supervised, the obvious other one is called unsupervised.
In that case, I'm just going to give you a bunch of examples.
But I don't know the labels associated with them.
I'm going to just try and find what are the natural ways to group those examples together into different models.
And in some cases, I may know how many models are there.
In some cases, I may want to just say what's the best grouping I can find.
OK. What I'm going to do today is not a lot of code.
I was expecting cheers for that, John, but I didn't get them.
Not a lot of code.
What I'm going to do is show you basically, the intuitions behind doing this learning.
And I"m going to start with my New England Patriots example.
So here are some data points about current Patriots players.
And I've got two kinds of positions.
I've got receivers, and I have linemen.
And each one is just labeled by the name, the height in inches, and the weight in pounds.
OK?
Five of each.
If I plot those on a two dimensional plot, this is what I get.
OK?
No big deal.
What am I trying to do?
I'm trying to learn, are their characteristics that distinguish the two classes from one another?
And in the unlabeled case, all I have are just a set of examples.
So what I want to do is decide what makes two players similar with the goal of seeing, can I separate this distribution into two or more natural groups.
Similar is a distance measure.
It says how do I take two examples with values or features associated, and we're going to decide how far apart are they?
And in the unlabeled case, the simple way to do it is to say, if I know that there are at least k groups there-- in this case, I'm going to tell you there are two different groups there-- how could I decide how best to cluster things together so that all the examples in one group are close to each other, all the examples in the other group are close to each other, and they're reasonably far apart.
There are many ways to do it.
I'm going to show you one.
It's a very standard way, and it works, basically, as follows.
If all I know is that there are two groups there, I'm going to start by just picking two examples as my exemplars.
Pick them at random.
Actually at random is not great.
I don't want to pick too closely to each other.
I'm going to try and pick them far apart.
But I pick two examples as my exemplars.
And for all the other examples in the training data, I say which one is it closest to.
What I'm going to try and do is create clusters with the property that the distances between all of the examples of that cluster are small.
The average distance is small.
And see if I can find clusters that gets the average distance for both clusters as small as possible.
This algorithm works by picking two examples, clustering all the other examples by simply saying put it in the group to which it's closest to that example.
Once I've got those clusters, I'm going to find the median element of that group.
Not mean, but median, what's the one closest to the center?
And treat those as exemplars and repeat the process.
And I'll just do it either some number of times or until I don't get any change in the process.
So it's clustering based on distance.
And we'll come back to distance in a second.
So here's what would have my football players.
If I just did this based on weight, there's the natural dividing line.
And it kind of makes sense.
All right?
These three are obviously clustered, and again, it's just on this axis.
They're all down here.
These seven are at a different place.
There's a natural dividing line there.
If I were to do it based on height, not as clean.
This is what my algorithm came up with as the best dividing line here, meaning that these four, again, just based on this axis are close together.
These six are close together.
But it's not nearly as clean.
And that's part of the issue we'll look at is how do I find the best clusters.
If I use both height and weight, I get that, which was actually kind of nice, right?
Those three cluster together.
they're near each other, in terms of just distance in the plane.
Those seven are near each other.
There's a nice, natural dividing line through here.
And in fact, that gives me a classifier.
This line is the equidistant line between the centers of those two clusters.
Meaning, any point along this line is the same distance to the center of that group as it is to that group.
And so any new example, if it's above the line, I would say gets that label, if it's below the line, gets that label.
In a second, we'll come back to look at how do we measure the distances, but the idea here is pretty simple.
I want to find groupings near each other and far apart from the other group.
Now suppose I actually knew the labels on these players.
These are the receivers.
Those are the linemen.
And for those of you who are football fans, you can figure it out, right?
Those are the two tight ends.
They are much bigger.
I think that's Bennett and that's Gronk if you're really a big Patriots fan.
But those are tight ends, those are wide receivers, and it's going to come back in a second, but there are the labels.
Now what I want to do is say, if I could take advantage of knowing the labels, how would I divide these groups up?
And that's kind of easy to see.
Basic idea, in this case, is if I've got labeled groups in that feature space, what I want to do is find a subsurface that naturally divides that space.
Now subsurface is a fancy word.
It says, in the two-dimensional case, I want to know what's the best line, if I can find a single line, that separates all the examples with one label from all the examples of the second label.
We'll see that, if the examples are well separated, this is easy to do, and it's great.
But in some cases, it's going to be more complicated because some of the examples may be very close to one another.
And that's going to raise a problem that you saw last lecture.
I want to avoid overfitting.
I don't want to create a really complicated surface to separate things.
And so we may have to tolerate a few incorrectly labeled things, if we can't pull it out.
And as you already figured out, in this case, with the labeled data, there's the best fitting line right there.
Anybody over 280 pounds is going to be a great lineman.
Anybody under 280 pounds is more likely to be a receiver.
OK.
So I've got two different ways of trying to think about doing this labeling.
I'm going to come back to both of them in a second.
Now suppose I add in some new data.
I want to label new instances.
Now these are actually players of a different position.
These are running backs.
But I say, all I know about is receivers and linemen.
I get these two new data points.
I'd like to know, are they more likely to be a receiver or a linemen?
And there's the data for these two gentlemen.
So if I go back to now plotting them, oh you notice one of the issues.
So there are my linemen, the red ones are my receivers, the two black dots are the two running backs.
And notice right here.
It's going to be really hard to separate those two examples from one another.
They are so close to each other.
And that's going to be one of the things we have to trade off.
But if I think about using what I learned as a classifier with unlabeled data, there were my two clusters.
Now you see, oh, I've got an interesting example.
This new example I would say is clearly more like a receiver than a lineman.
But that one there, unclear.
Almost exactly lies along that dividing line between those two clusters.
And I would either say, I want to rethink the clustering or I want to say, you know what?
As I know, maybe there aren't two clusters here.
Maybe there are three.
And I want to classify them a little differently.
So I'll come back to that.
On the other hand, if I had used the labeled data, there was my dividing line.
This is really easy.
Both of those new examples are clearly below the dividing line.
They are clearly examples that I would categorize as being more like receivers than they are like linemen.
And I know it's a football example.
If you don't like football, pick another example.
But you get the sense of why I can use the data in a labeled case and the unlabeled case to come up with different ways of building the clusters.
So what we're going to do over the next 2 and 1/2 lectures is look at how can we write code to learn that way of separating things out?
We're going to learn models based on unlabeled data.
That's the case where I don't know what the labels are, by simply trying to find ways to cluster things together nearby, and then use the clusters to assign labels to new data.
And we're going to learn models by looking at labeled data and seeing how do we best come up with a way of separating with a line or a plane or a collection of lines, examples from one group, from examples of the other group.
With the acknowledgment that we want to avoid overfitting, we don't want to create a really complicated system.
And as a consequence, we're going to have to make some trade-offs between what we call false positives and false negatives.
But the resulting classifier can then label any new data by just deciding where you are with respect to that separating line.
So here's what you're going to see over the next 2 and 1/2 lectures.
Every machine learning method has five essential components.
We need to decide what's the training data, and how are we going to evaluate the success of that system.
We've already seen some examples of that.
We need to decide how are we going to represent each instance that we're giving it.
I happened to choose height and weight for football players.
But I might have been better off to pick average speed or, I don't know, arm length, something else.
How do I figure out what are the right features.
And associated with that, how do I measure distances between those features?
How do I decide what's close and what's not close?
Maybe it should be different, in terms of weight versus height, for example.
I need to make that decision.
And those are the two things we're going to show you examples of today, how to go through that.
Starting next week, Professor Guttag is going to show you how you take those and actually start building more detailed versions of measuring clustering, measuring similarities to find an objective function that you want to minimize to decide what is the best cluster to use.
And then what is the best optimization method you want to use to learn that model.
So let's start talking about features.
I've got a set of examples, labeled or not.
I need to decide what is it about those examples that's useful to use when I want to decide what's close to another thing or not.
And one of the problems is, if it was really easy, it would be really easy.
Features don't always capture what you want.
I'm going to belabor that football analogy, but why did I pick height and weight.
Because it was easy to find.
You know, if you work for the New England Patriots, what is the thing that you really look for when you're asking, what's the right feature?
It's probably some other combination of things.
So you, as a designer, have to say what are the features I want to use.
That quote, by the way, is from one of the great statisticians of the 20th century, which I think captures it well.
So feature engineering, as you, as a programmer, comes down to deciding both what are the features I want to measure in that vector that I'm going to put together, and how do I decide relative ways to weight it?
So John, and Ana, and I could have made our job this term really easy if we had sat down at the beginning of the term and said, you know, we've taught this course many times.
We've got data from, I don't know, John, thousands of students, probably over this time.
Let's just build a little learning algorithm that takes a set of data and predicts your final grade.
You don't have to come to class, don't have to go through all the problems, because we'll just predict your final grade.
Wouldn't that be nice?
Make our job a little easier, and you may or may not like that idea.
But I could think about predicting that grade?
Now why am I telling this example.
I was trying to see if I could get a few smiles.
I saw a couple of them there.
But think about the features.
What I measure?
Actually, I'll put this on John because it's his idea.
What would he measure?
Well, GPA is probably not a bad predictor of performance.
You do well in other classes, you're likely to do well in this class.
I'm going to use this one very carefully.
Prior programming experience is at least a predictor, but it is not a perfect predictor.
Those of you who haven't programmed before, in this class, you can still do really well in this class.
But it's an indication that you've seen other programming languages.
On the other hand, I don't believe in astrology.
So I don't think the month in which you're born, the astrological sign under which you were born has probably anything to do with how well you'd program.
I doubt that eye color has anything to do with how well you'd program.
You get the idea.
Some features matter, others don't.
Now I could just throw all the features in and hope that the machine learning algorithm sorts out those it wants to keep from those it doesn't.
But I remind you of that idea of overfitting.
If I do that, there is the danger that it will find some correlation between birth month, eye color, and GPA.
And that's going to lead to a conclusion that we really don't like.
By the way, in case you're worried, I can assure you that Stu Schmill in the dean of admissions department does not use machine learning to pick you.
He actually looks at a whole bunch of things because it's not easy to replace him with a machine-- yet.
All right.
So what this says is we need to think about how do we pick the features.
And mostly, what we're trying to do is to maximize something called the signal to noise ratio.
Maximize those features that carry the most information, and remove the ones that don't.
So I want to show you an example of how you might think about this.
I want to label reptiles.
I want to come up with a way of labeling animals as, are they a reptile or not.
And I give you a single example.
With a single example, you can't really do much.
But from this example, I know that a cobra, it lays eggs, it has scales, it's poisonous, it's cold blooded, it has no legs, and it's a reptile.
So I could say my model of a reptile is well, I'm not certain.
I don't have enough data yet.
But if I give you a second example, and it also happens to be egg-laying, have scales, poisonous, cold blooded, no legs.
There is my model, right?
Perfectly reasonable model, whether I design it or a machine learning algorithm would do it says, if all of these are true, label it as a reptile.
OK?
And now I give you a boa constrictor.
Ah.
It's a reptile.
But it doesn't fit the model.
And in particular, it's not egg-laying, and it's not poisonous.
So I've got to refine the model.
Or the algorithm has got to refine the model.
And this, I want to remind you, is looking at the features.
So I started out with five features.
This doesn't fit.
So probably what I should do is reduce it.
I'm going to look at scales.
I'm going to look at cold blooded.
I'm going to look at legs.
That captures all three examples.
Again, if you think about this in terms of clustering, all three of them would fit with that.
OK. Now I give you another example-- chicken.
I don't think it's a reptile.
In fact, I'm pretty sure it's not a reptile.
And it nicely still fits this model, right?
Because, while it has scales, which you may or not realize, it's not cold blooded, and it has legs.
So it is a negative example that reinforces the model.
Sounds good.
And now I'll give you an alligator.
It's a reptile.
And oh fudge, right?
It doesn't satisfy the model.
Because while it does have scales and it is cold blooded, it has legs.
I'm almost done with the example.
But you see the point.
Again, I've got to think about how do I refine this.
And I could by saying, all right.
Let's make it a little more complicated-- has scales, cold blooded, 0 or four legs-- I'm going to say it's a reptile.
I'll give you the dart frog.
Not a reptile, it's an amphibian.
And that's nice because it still satisfies this.
So it's an example outside of the cluster that says no scales, not cold blooded, but happens to have four legs.
It's not a reptile.
That's good.
And then I give you-- I have to give you a python, right?
I mean, there has to be a python in here.
Oh come on.
At least grown at me when I say that.
There has to be a python here.
And I give you that and a salmon.
And now I am in trouble.
Because look at scales, look at cold blooded, look at legs.
I can't separate them.
On those features, there's no way to come up with a way that will correctly say that the python is a reptile and the salmon is not.
And so there's no easy way to add in that rule.
And probably my best thing is to simply go back to just two features, scales and cold blooded.
And basically say, if something has scales and it's cold blooded, I'm going to call it a reptile.
If it doesn't have both of those, I'm going to say it's not a reptile.
It won't be perfect.
It's going to incorrectly label the salmon.
But I've made a design choice here that's important.
And the design choice is that I will have no false negatives.
What that means is there's not going to be any instance of something that's not a reptile that I'm going to call a reptile.
I may have some false positives.
So I did that the wrong way.
A false negative says, everything that's not a reptile I'm going to categorize that direction.
I may have some false positives, in that, I may have a few things that I will incorrectly label as a reptile.
And in particular, salmon is going to be an instance of that.
This trade off of false positives and false negatives is something that we worry about, as we think about it.
Because there's no perfect way, in many cases, to separate out the data.
And if you think back to my example of the New England Patriots, that running back and that wide receiver were so close together in height and weight, there was no way I'm going to be able to separate them apart.
And I just have to be willing to decide how many false positives or false negatives do I want to tolerate.
Once I've figured out what features to use, which is good, then I have to decide about distance.
How do I compare two feature vectors?
I'm going to say vector because there could be multiple dimensions to it.
How do I decide how to compare them?
Because I want to use the distances to figure out either how to group things together or how to find a dividing line that separates things apart.
So one of the things I have to decide is which features.
I also have to decide the distance.
And finally, I may want to decide how to weigh relative importance of different dimensions in the feature vector.
Some may be more valuable than others in making that decision.
And I want to show you an example of that.
So let's go back to my animals.
I started off with a feature vector that actually had five dimensions to it.
It was egg-laying, cold blooded, has scales, I forget what the other one was, and number of legs.
So one of the ways I could think about this is saying I've got four binary features and one integer feature associated with each animal.
And one way to learn to separate out reptiles from non reptiles is to measure the distance between pairs of examples and use that distance to decide what's near each other and what's not.
And as we've said before, it will either be used to cluster things or to find a classifier surface that separates them.
So here's a simple way to do it.
For each of these examples, I'm going to just let true be 1, false be 0.
So the first four are either 0s or 1s.
And the last one is the number of legs.
And now I could say, all right.
How do I measure distances between animals or anything else, but these kinds of feature vectors?
Here, we're going to use something called the Minkowski Metric or the Minkowski difference.
Given two vectors and a power, p, we basically take the absolute value of the difference between each of the components of the vector, raise it to the p-th power, take the sum, and take the p-th route of that.
So let's do the two obvious examples.
If p is equal to 1, I just measure the absolute distance between each component, add them up, and that's my distance.
It's called the Manhattan metric.
The one you've seen more, the one we saw last time, if p is equal to 2, this is Euclidean distance, right?
It's the sum of the squares of the differences of the components.
Take the square root.
Take the square root because it makes it have certain properties of a distance.
That's the Euclidean distance.
So now if I want to measure difference between these two, here's the question.
Is this circle closer to the star or closer to the cross?
Unfortunately, I put the answer up here.
But it differs, depending on the metric I use.
Right?
Euclidean distance, well, that's square root of 2 times 2, so it's about 2.8.
And that's three.
So in terms of just standard distance in the plane, we would say that these two are closer than those two are.
Manhattan distance, why is it called that?
Because you can only walk along the avenues and the streets.
Manhattan distance would basically say this is one, two, three, four units away.
This is one, two, three units away.
And under Manhattan distance, this is closer, this pairing is closer than that pairing is.
Now you're used to thinking Euclidean.
We're going to use that.
But this is going to be important when we think about how are we comparing distances between these different pieces.
So typically, we'll use Euclidean.
We're going to see Manhattan actually has some value.
So if I go back to my three examples-- boy, that's a gross slide, isn't it?
But there we go-- rattlesnake, boa constrictor, and dart frog.
There is the representation.
I can ask, what's the distance between them?
In the handout for today, we've given you a little piece of code that would do that.
And if I actually run through it, I get, actually, a nice little result.
Here are the distances between those vectors using Euclidean metric.
I'm going to come back to them.
But you can see the two snakes, nicely, are reasonably close to each other.
Whereas, the dart frog is a fair distance away from that.
Nice, right?
That's a nice separation that says there's a difference between these two.
OK. Now I throw in the alligator.
Sounds like a Dungeons & Dragons game.
I throw in the alligator, and I want to do the same comparison.
And I don't get nearly as nice a result.
Because now it says, as before, the two snakes are close to each other.
But it says that the dart frog and the alligator are much closer, under this measurement, than either of them is to the other.
And to remind you, right, the alligator and the two snakes I would like to be close to one another and a distance away from the frog.
Because I'm trying to classify reptiles versus not.
So what happened here?
Well, this is a place where the feature engineering is going to be important.
Because in fact, the alligator differs from the frog in three features.
And only in two features from, say, the boa constrictor.
But one of those features is the number of legs.
And there, while on the binary axes, the difference is between a 0 and 1, here it can be between 0 and 4.
So that is weighing the distance a lot more than we would like.
The legs dimension is too large, if you like.
How would I fix this?
This is actually, I would argue, a natural place to use Manhattan distance.
Why should I think that the difference in the number of legs or the number of legs difference is more important than whether it has scales or not?
Why should I think that measuring that distance Euclidean-wise makes sense?
They are really completely different measurements.
And in fact, I'm not going to do it, but if I ran Manhattan metric on this, it would get the alligator much closer to the snakes, exactly because it differs only in two features, not three.
The other way I could fix it would be to say I'm letting too much weight be associated with the difference in the number of legs.
So let's just make it a binary feature.
Either it doesn't have legs or it does have legs.
Run the same classification.
And now you see the snakes and the alligator are all close to each other.
Whereas the dart frog, not as far away as it was before, but there's a pretty natural separation, especially using that number between them.
What's my point?
Choice of features matters.
Throwing too many features in may, in fact, give us some overfitting.
And in particular, deciding the weights that I want on those features has a real impact.
And you, as a designer or a programmer, have a lot of influence in how you think about using those.
So feature engineering really matters.
How you pick the features, what you use is going to be important.
OK.
The last piece of this then is we're going to look at some examples where we give you data, got features associated with them.
We're going to, in some cases have them labeled, in other cases not.
And we know how now to think about how do we measure distances between them.
John
You probably didn't intend to say weights of features.
You intended to say how they're scaled
Sorry.
The scales and not the-- thank you, John.
No, I did.
I take that back.
I did not mean to say weights of features.
I meant to say the scale of the dimension is going to be important here.
Thank you, for the amplification and correction.
You're absolutely right
Weights, we use in a different way, as we'll see next time
And we're going to see next time why we're going to use weights in different ways.
So rephrase it.
Block that out of your mind.
We're going to talk about scales and the scale on the axes as being important here.
And we already said we're going to look at two different kinds of learning, labeled and unlabeled, clustering and classifying.
And I want to just finish up by showing you two examples of that.
How we would think about them algorithmically, and we'll look at them in more detail next time.
As we look at it, I want to remind you the things that are going to be important to you.
How do I measure distance between examples?
What's the right way to design that?
What is the right set of features to use in that vector?
And then, what constraints do I want to put on the model?
In the case of unlabelled data, how do I decide how many clusters I want to have?
Because I can give you a really easy way to do clustering.
If I give you 100 examples, I say build 100 clusters.
Every example is its own cluster.
Distance is really good.
It's really close to itself, but it does a lousy job of labeling things on it.
So I have to think about, how do I decide how many clusters, what's the complexity of that separating service?
How do I basically avoid the overfitting problem, which I don't want to have?
So just to remind you, we've already seen a little version of this, the clustering method.
This is a standard way to do it, simply repeating what we had on an earlier slide.
If I want to cluster it into groups, I start by saying how many clusters am I looking for?
Pick an example I take as my early representation.
For every other example in the training data, put it to the closest cluster.
Once I've got those, find the median, repeat the process.
And that led to that separation.
Now once I've got it, I like to validate it.
And in fact, I should have said this better.
Those two clusters came without looking at the two black dots.
Once I put the black dots in, I'd like to validate, how well does this really work?
And that example there is really not very encouraging.
It's too close.
So that's a natural place to say, OK, what if I did this with three clusters?
That's what I get.
I like the that.
All right?
That has a really nice cluster up here.
The fact that the algorithm didn't know the labeling is irrelevant.
There's a nice grouping of five.
There's a nice grouping of four.
And there's a nice grouping of three in between.
And in fact, if I looked at the average distance between examples in each of these clusters, it is much tighter than in that example.
And so that leads to, then, the question of should I look for four clusters?
Question, please
Is that overlap between the two clusters not an issue?
Yes.
The question is, is the overlap between the two clusters a problem?
No.
I just drew it here so I could let you see where those pieces are.
But in fact, if you like, the center is there.
Those three points are all closer to that center than they are to that center.
So the fact that they overlap is a good question.
It's just the way I happened to draw them.
I should really draw these, not as circles, but as some little bit more convoluted surface.
OK?
Having done three, I could say should I look for four?
Well, those points down there, as I've already said, are an example where it's going to be hard to separate them out.
And I don't want to overfit.
Because the only way to separate those out is going to be to come up with a really convoluted cluster, which I don't like.
All right?
Let me finish with showing you one other example from the other direction.
Which is, suppose I give you labeled examples.
So again, the goal is I've got features associated with each example.
They're going to have multiple dimensions on it.
But I also know the label associated with them.
And I want to learn what is the best way to come up with a rule that will let me take new examples and assign them to the right group.
A number of ways to do this.
You can simply say I'm looking for the simplest surface that will separate those examples.
In my football case that were in the plane, what's the best line that separates them, which turns out to be easy.
I might look for a more complicated surface.
And we're going to see an example in a second where maybe it's a sequence of line segments that separates them out.
Because there's not just one line that does the separation.
As before, I want to be careful.
If I make it too complicated, I may get a really good separator, but I overfit to the data.
And you're going to see next time.
I'm going to just highlight it here.
There's a third way, which will lead to almost the same kind of result called k nearest neighbors.
And the idea here is I've got a set of labeled data.
And what I'm going to do is, for every new example, say find the k, say the five closest labeled examples.
And take a vote.
If 3 out of 5 or 4 out of 5 or 5 out of 5 of those labels are the same, I'm going to say it's part of that group.
And if I have less than that, I'm going to leave it as unclassified.
And that's a nice way of actually thinking about how to learn them.
And let me just finish by showing you an example.
Now I won't use football players on this one.
I'll use a different example.
I'm going to give you some voting data.
I think this is actually simulated data.
But these are a set of voters in the United States with their preference.
They tend to vote Republican.
They tend to vote Democrat.
And the two categories are their age and how far away they live from Boston.
Whether those are relevant or not, I don't know, but they are just two things I'm going to use to classify them.
And I'd like to say, how would I fit a curve to separate those two classes?
I'm going to keep half the data to test.
I'm going to use half the data to train.
So if this is my training data, I can say what's the best line that separates these?
I don't know about best, but here are two examples.
This solid line has the property that all the Democrats are on one side.
Everything on the other side is a Republican, but there are some Republicans on this side of the line.
I can't find a line that completely separates these, as I did with the football players.
But there is a decent line to separate them.
Here's another candidate.
That dash line has the property that on the right side you've got-- boy, I don't think this is deliberate, John, right-- but on the right side, you've got almost all Republicans.
It seems perfectly appropriate.
One Democrat, but there's a pretty good separation there.
And on the left side, you've got a mix of things.
But most of the Democrats are on the left side of that line.
All right?
The fact that left and right correlates with distance from Boston is completely irrelevant here.
But it has a nice punch to it
Relevant, but not accidental
But not accidental.
Thank you.
All right.
So now the question is, how would I evaluate these?
How do I decide which one is better?
And I'm simply going to show you, very quickly, some examples.
First one is to look at what's called the confusion matrix.
What does that mean?
It says for this, one of these classifiers for example, the solid line.
Here are the predictions, based on the solid line of whether they would be more likely to be Democrat or Republican.
And here is the actual label.
Same thing for the dashed line.
And that diagonal is important because those are the correctly labeled results.
Right?
It correctly, in the solid line case, gets all of the correct labelings of the Democrats.
It gets half of the Republicans right.
But it has some where it's actually Republican, but it labels it as a Democrat.
That, we'd like to be really large.
And in fact, it leads to a natural measure called the accuracy.
Which is, just to go back to that, we say that these are true positives.
Meaning, I labeled it as being an instance, and it really is.
These are true negatives.
I label it as not being an instance, and it really isn't.
And then these are the false positives.
I labeled it as being an instance and it's not, and these are the false negatives.
I labeled it as not being an instance, and it is.
And an easy way to measure it is to look at the correct labels over all of the labels.
The true positives and the true negatives, the ones I got right.
And in that case, both models come up with a value of 0.7.
So which one is better?
Well, I should validate that.
And I'm going to do that in a second by looking at other data.
We could also ask, could we find something with less training error?
This is only getting 70% right.
Not great.
Well, here is a more complicated model.
And this is where you start getting worried about overfitting.
Now what I've done, is I've come up with a sequence of lines that separate them.
So everything above this line, I'm going to say is a Republican.
Everything below this line, I'm going to say is a Democrat.
So I'm avoiding that one.
I'm avoiding that one.
I'm still capturing many of the same things.
And in this case, I get 12 true positives, 13 true negatives, and only 5 false positives.
And that's kind of nice.
You can see the 5.
It's those five red ones down there.
It's accuracy is 0.833.
And now, if I apply that to the test data, I get an OK result.
It has an accuracy of about 0.6.
I could use this idea to try and generalize to say could I come up with a better model.
And you're going to see that next time.
There could be other ways in which I measure this.
And I want to use this as the last example.
Another good measure we use is called PPV, Positive Predictive Value which is how many true positives do I come up with out of all the things I labeled positively.
And in this solid model, in the dashed line, I can get values about 0.57.
The complex model on the training data is better.
And then the testing data is even stronger.
And finally, two other examples are called sensitivity and specificity.
Sensitivity basically tells you what percentage did I correctly find.
And specificity said what percentage did I correctly reject.
And I show you this because this is where the trade-off comes in.
If sensitivity is how many did I correctly label out of those that I both correctly labeled and incorrectly labeled as being negative, how many them did I correctly label as being the kind that I want?
I can make sensitivity 1.
Label everything is the thing I'm looking for.
Great.
Everything is correct.
But the specificity will be 0.
Because I'll have a bunch of things incorrectly labeled.
I could make the specificity 1, reject everything.
Say nothing as an instance.
True negatives goes to 1, and I'm in a great place there, but my sensitivity goes to 0.
I've got a trade-off.
As I think about the machine learning algorithm I'm using and my choice of that classifier, I'm going to see a trade off where I can increase specificity at the cost of sensitivity or vice versa.
And you'll see a nice technique called ROC or Receiver Operator Curve that gives you a sense of how you want to deal with that.
And with that, we'll see you next time.
We'll take your question off line if you don't mind, because I've run over time.
But we'll see you next time where Professor Guttag will show you examples of this.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'm a little reluctant to say good afternoon, given the weather, but I'll say it anyway.
I guess now we all do know that we live in Boston.
And I should say, I hope none of you were affected too much by the fire yesterday in Cambridge, but that seems to have been a pretty disastrous event for some.
Anyway, here's the reading.
This is a chapter in the book on clustering, a topic that Professor Grimson introduced last week.
And I'm going to try and finish up with respect to this course today, though not with respect to everything there is to know about clustering.
Quickly just reviewing where we were.
We're in the unit of a course on machine learning, and we always follow the same paradigm.
We observe some set of examples, which we call the training data.
We try and infer something about the process that created those examples.
And then we use inference techniques, different kinds of techniques, to make predictions about previously unseen data.
We call that the test data.
As Professor Grimson said, you can think of two broad classes.
Supervised, where we have a set of examples and some label associated with the example-- Democrat, Republican, smart, dumb, whatever you want to associate with them-- and then we try and infer the labels.
Or unsupervised, where we're given a set of feature vectors without labels, and then we attempt to group them into natural clusters.
That's going to be today's topic, clustering.
So clustering is an optimization problem.
As we'll see later, supervised machine learning is also an optimization problem.
Clustering's a rather simple one.
We're going to start first with the notion of variability.
So this little c is a single cluster, and we're going to talk about the variability in that cluster of the sum of the distance between the mean of the cluster and each example in the cluster.
And then we square it.
OK?
Pretty straightforward.
For the moment, we can just assume that we're using Euclidean distance as our distance metric.
Minkowski with p equals two.
So variability should look pretty similar to something we've seen before, right?
It's not quite variance, right, but it's very close.
In a minute, we'll look at why it's different.
And then we can look at the dissimilarity of a set of clusters, a group of clusters, which I'm writing as capital C, and that's just the sum of all the variabilities.
Now, if I had divided variability by the size of the cluster, what would I have?
Something we've seen before.
What would that be?
Somebody?
Isn't that just the variance?
So the question is, why am I not doing that?
If up til now, we always wanted to talk about variance, why suddenly am I not doing it?
Why do I define this notion of variability instead of good old variance?
Any thoughts?
What am I accomplishing by not dividing by the size of the cluster?
Or what would happen if I did divide by the size of the cluster?
Yes
You normalize it?
Absolutely.
I'd normalize it.
That's exactly what it would be doing.
And what might be good or bad about normalizing it?
What does it essentially mean to normalize?
It means that the penalty for a big cluster with a lot of variance in it is no higher than the penalty of a tiny little cluster with a lot of variance in it.
By not normalizing, what I'm saying is I want to penalize big, highly-diverse clusters more than small, highly-diverse clusters.
OK?
And if you think about it, that probably makes sense.
Big and bad is worse than small and bad.
All right, so now we define the objective function.
And can we say that the optimization problem we want to solve by clustering is simply finding a capital C that minimizes dissimilarity?
Is that a reasonable definition?
Well, hint-- no.
What foolish thing could we do that would optimize that objective function?
Yeah
You could have the same number of clusters as points?
Yeah.
I can have the same number of clusters as points, assign each point to its own cluster, whoops.
Ooh, almost a relay.
The dissimilarity of each cluster would be 0.
The variability would be 0, so the dissimilarity would be 0, and I just solved the problem.
Well, that's clearly not a very useful thing to do.
So, well, what do you think we do to get around that?
Yeah
We apply a constraint?
We apply a constraint.
Exactly.
And so we have to pick some constraint.
What would be a suitable constraint, for example?
Well, maybe we'd say, OK, the clusters have to have some minimum distance between them.
Or-- and this is the constraint we'll be using today-- we could constrain the number of clusters.
Say, all right, I only want to have at most five clusters.
Do the best you can to minimize dissimilarity, but you're not allowed to use more than five clusters.
That's the most common constraint that gets placed in the problem.
All right, we're going to look at two algorithms.
Maybe I should say two methods, because there are multiple implementations of these methods.
The first is called hierarchical clustering, and the second is called k-means.
There should be an S on the word mean there.
Sorry about that.
All right, let's look at hierarchical clustering first.
It's a strange algorithm.
We start by assigning each item, each example, to its own cluster.
So this is the trivial solution we talked about before.
So if you have N items, you now have N clusters, each containing just one item.
In the next step, we find the two most similar clusters we have and merge them into a single cluster, so that now instead of N clusters, we have N minus 1 clusters.
And we continue this process until all items are clustered into a single cluster of size N. Now of course, that's kind of silly, because if all I wanted to put them all it in is in a single cluster, I don't need to iterate.
I just go wham, right?
But what's interesting about hierarchical clustering is you stop it, typically, somewhere along the way.
You produce something called a [?
dendogram.
?]
Let me write that down.
At each step here, it shows you what you've merged thus far.
We'll see an example of that shortly.
And then you can have some stopping criteria.
We'll talk about that.
This is called agglomerative hierarchical clustering because we start with a bunch of things and we agglomerate them.
That is to say, we put them together.
All right?
Make sense?
Well, there's a catch.
What do we mean by distance?
And there are multiple plausible definitions of distance, and you would get a different answer depending upon which measure you used.
These are called linkage metrics.
The most common one used is probably single-linkage, and that says the distance between a pair of clusters is equal to the shortest distance from any member of one cluster to any member of the other cluster.
So if I have two clusters, here and here, and they have bunches of points in them, single-linkage distance would say, well, let's use these two points which are the closest, and the distance between these two is the distance between the clusters.
You can also use complete-linkage, and that says the distance between any two clusters is equal to the greatest distance from any member to any other member.
OK?
So if we had the same picture we had before-- probably not the same picture, but it's a picture.
Whoops.
Then we would say, well, I guess complete-linkage is probably the distance, maybe, between those two.
And finally, not surprisingly, you can take the average distance.
These are all plausible metrics.
They're all used and practiced for different kinds of results depending upon the application of the clustering.
All right, let's look at an example.
So what I have here is the air distance between six different cities, Boston, New York, Chicago, Denver, San Francisco, and Seattle.
And now let's say we're-- want to cluster these airports just based upon their distance.
So we start.
The first piece of our [?
dendogram ?]
says, well, all right, I have six cities, I have six clusters, each containing one city.
All right, what happens next?
What's the next level going to look like?
Yeah?
You're going from Boston [INAUDIBLE]
I'm going to join Boston and New York, as improbable as that sounds.
All right, so that's the next level.
And if for some reason I only wanted to have five clusters, well, I could stop here.
Next, what happens?
Well, I look at it, I say well, I'll join up Chicago with Boston and New York.
All right.
What do I get at the next level?
Somebody?
Yeah
Seattle [INAUDIBLE]
Doesn't look like it to me.
If you look at San Francisco and Seattle, they are 808 miles, and Denver and San Francisco is 1,235.
So I'd end up, in fact, joining San Francisco and Seattle
That's what I said
Well, that explains why I need my hearing fixed.
[LAUGHTER] All right.
So I combine San Francisco and Seattle, and now it gets interesting.
I have two choices with Denver.
Obviously, there are only two choices, and which I choose depends upon which linkage criterion I use.
If I'm using single-linkage, well, then Denver gets joined with Boston, New York, and Chicago, because it's closer to Chicago than it is to either San Francisco or Seattle.
But if I use complete-linkage, it gets joined up with San Francisco and Seattle, because it is further from Boston than it is from, I guess it's San Francisco or Seattle.
Whichever it is, right?
So this is a place where you see what answer I get depends upon the linkage criteria.
And then if I want, I can consider to the next step and just join them all.
All right?
That's hierarchical clustering.
So it's good because you get this whole history of the [?
dendograms, ?]
and you get to look at it, say, well, all right, that looks pretty good.
I'll stick with this clustering.
It's deterministic.
Given a linkage criterion, you always get the same answer.
There's nothing random here.
Notice, by the way, the answer might not be optimal with regards to that linkage criteria.
Why not?
What kind of algorithm is this?
Greedy
It's a greedy algorithm, exactly.
And so I'm making locally optimal decisions at each point which may or may not be globally optimal.
It's flexible.
Choosing different linkage criteria, I get different results.
But it's also potentially really, really slow.
This is not something you want to do on a million examples.
The naive algorithm, the one I just sort of showed you, is N cubed.
N cubed is typically impractical.
For some linkage criteria, for example, single-linkage, there exists very clever N squared algorithms.
For others, you can't beat N cubed.
But even N squared is really not very good.
Which gets me to a much faster greedy algorithm called k-means.
Now, the k in k-means is the number of clusters you want.
So the catch with k-means is if you don't have any idea how many clusters you want, it's problematical, whereas hierarchical, you get to inspect it and see what you're getting.
If you know how many you want, it's a good choice because it's much faster.
All right, the algorithm, again, is very simple.
This is the one that Professor Grimson briefly discussed.
You randomly choose k examples as your initial centroids.
Doesn't matter which of the examples you choose.
Then you create k clusters by assigning each example to the closest centroid, compute k new centroids by averaging the examples in each cluster.
So in the first iteration, the centroids are all examples that you started with.
But after that, they're probably not examples, because you're now taking the average of two examples, which may not correspond to any example you have.
Actually the average of N examples.
And then you just keep doing this until the centroids don't move.
Right?
Once you go through one iteration where they don't move, there's no point in recomputing them again and again and again, so it is converged.
So let's look at the complexity.
Well, at the moment, we can't tell you how many iterations you're going to have, but what's the complexity of one iteration?
Well, let's think about what you're doing here.
You've got k centroids.
Now I have to take each example and compare it to each-- in a naively, at least-- to each centroid to see which it's closest to.
Right?
So that's k comparisons per example.
So that's k times n times d, where how much time each of these comparison takes, which is likely to depend upon the dimensionality of the features, right?
Just the Euclidean distance, for example.
But this is a way small number than N squared, typically.
So each iteration is pretty quick, and in practice, as we'll see, this typically converges quite quickly, so you usually need a very small number of iterations.
So it is quite efficient, and then there are various ways you can optimize it to make it even more efficient.
This is the most commonly-used clustering algorithm because it works really fast.
Let's look at an example.
So I've got a bunch of blue points here, and I actually wrote the code to do this.
I'm not going to show you the code.
And I chose four centroids at random, colored stars.
A green one, a fuchsia-colored one, a red one, and a blue one.
So maybe they're not the ones you would have chosen, but there they are.
And I then, having chosen them, assign each point to one of those centroids, whichever one it's closest to.
All right?
Step one.
And then I recompute the centroid.
So let's go back.
So we're here, and these are the initial centroids.
Now, when I find the new centroids, if we look at where the red one is, the red one is this point, this point, and this point.
Clearly, the new centroid is going to move, right?
It's going to move somewhere along in here or something like that, right?
So we'll get those new centroids.
There it is.
And now we'll re-assign points.
And what we'll see is this point is now closer to the red star than it is to the fuchsia star, because we've moved the red star.
Whoops.
That one.
Said the wrong thing.
They were red to start with.
This one is now suddenly closer to the purple, so-- and to the red.
It will get recolored.
We compute the new centroids.
We're going to move something again.
We continue.
Points will move around.
This time we move two points.
Here we go again.
Notice, again, the centroids don't correspond to actual examples.
This one is close, but it's not really one of them.
Move two more.
Recompute centroids, and we're done.
So here we've converged, and I think it was five iterations, and nothing will move again.
All right?
Does that make sense to everybody?
So it's pretty simple.
What are the downsides?
Well, choosing k foolishly can lead to strange results.
So if I chose k equal to 3, looking at this particular arrangement of points, it's not obvious what "the right answer" is, right?
Maybe it's making all of this one cluster.
I don't know.
But there are weird k's and if you choose a k that is nonsensical with respect to your data, then your clustering will be nonsensical.
So that's one problem we have think about.
How do we choose k?
Another problem, and this is one somebody raised last time, is that the results can depend upon the initial centroids.
Unlike hierarchical clustering, k-means is non-deterministic.
Depending upon what random examples we choose, we can get a different number of iterations.
If we choose them poorly, it could take longer to converge.
More worrisome, you get a different answer.
You're running this greedy algorithm, and you might actually get to a different place, depending upon which centroids you chose.
So these are the two issues we have to think about dealing with.
So let's first think about choosing k. What often happens is people choose k using a priori knowledge about the application.
If I'm in medicine, I actually know that there are only five different kinds of bacteria in the world.
That's true.
I mean, there are subspecies, but five large categories.
And if I had a bunch of bacterium I wanted to cluster, may just set k equal to 5.
Maybe I believe there are only two kinds of people in the world, those who are at MIT and those who are not.
And so I'll choose k equal to 2.
Often, we know enough about the application, we can choose k. As we'll see later, often we can think we do, and we don't.
A better approach is to search for a good k. So you can try different values of k and evaluate the quality of the result.
Assume you have some metric, as to say yeah, I like this clustering, I don't like this clustering.
And we'll talk about do that in detail.
Or you can run hierarchical clustering on a subset of data.
I've got a million points.
All right, what I'm going to do is take a subset of 1,000 of them or 10,000.
Run hierarchical clustering.
From that, get a sense of the structure underlying the data.
Decide k should be 6, and then run k-means with k equals 6.
People often do this.
They run hierarchical clustering on a small subset of the data and then choose k. And we'll look-- but one we're going to look at is that one.
What about unlucky centroids?
So here I got the same points we started with.
Different initial centroids.
I've got a fuchsia one, a black one, and then I've got red and blue down here, which I happened to accidentally choose close to one another.
Well, if I start with these centroids, certainly you would expect things to take longer to converge.
But in fact, what happens is this-- I get this assignment of blue, this assignment of red, and I'm done.
It converges on this, which probably is not what we wanted out of this.
Maybe it is, but the fact that I converged on some very different place shows that it's a real weakness of the algorithm, that it's sensitive to the randomly-chosen initial conditions.
Well, couple of things you can do about that.
You could be clever and try and select good initial centroids.
So people often will do that, and what they'll do is try and just make sure that they're distributed over the space.
So they would look at some picture like this and say, well, let's just put my centroids at the corners or something like that so that they're far apart.
Another approach is to try multiple sets of randomly-chosen centroids, and then just select the best results.
And that's what this little algorithm on the screen does.
So I'll say best is equal to k-means of the points themselves, or something, then for t in range number of trials, I'll say C equals k-means of points, and I'll just keep track and choose the one with the least dissimilarity.
The thing I'm trying to minimize.
OK?
The first one is got all the points in one cluster.
So it's very dissimilar.
And then I'll just keep generating for different k's and I'll choose the k that seems to be the best, that does the best job of minimizing my objective function.
And this is a very common solution, by the way, for any randomized greedy algorithm.
And there are a lot of randomized greedy algorithms that you just choose multiple initial conditions, try them all out and pick the best.
All right, now I want to show you a slightly more real example.
So this is a file we've got with medical patients, and we're going to try and cluster them and see whether the clusters tell us anything about the probability of them dying of a heart attack in, say, the next year or some period of time.
So to simplify things, and this is something I have done with research, but we're looking at only four features here-- the heart rate in beats per minute, the number of previous heart attacks, the age, and something called ST elevation, a binary attribute.
So the first three are obvious.
If you take an ECG of somebody's heart, it looks like this.
This is a normal one.
They have the S, the T, and then there's this region between the S wave and the T wave.
And if it's higher, hence elevated, that's a bad thing.
And so this is about the first thing that they measure if someone is having cardiac problems.
Do they have ST elevation?
And then with each patient, we're going to have an outcome, whether they died, and it's related to the features, but it's probabilistic not deterministic.
So for example, an older person with multiple heart attacks is at higher risk than a young person who's never had a heart attack.
That doesn't mean, though, that the older person will die first.
It's just more probable.
We're going to take this data, we're going to cluster it, and then we're going to look at what's called the purity of the clusters relative to the outcomes.
So is the cluster, say, enriched by people who died?
If you have one cluster and everyone in it died, then the clustering is clearly finding some structure related to the outcome.
So the file is in the zip file I uploaded.
It looks more or less like this.
Right?
So it's very straightforward.
The outcomes are binary.
1 is a positive outcome.
Strangely enough in the medical jargon, a death is a positive outcome.
I guess maybe if you're responsible for the medical bills, it's positive.
If you're the patient, it's hard to think of it as a good thing.
Nevertheless, that's the way that they talk.
And the others are all there, right?
Heart rate, other things.
All right, let's look at some code.
So I've extracted some code.
I'm not going to show you all of it.
There's quite a lot of it, as you'll see.
So we'll start-- one of the files you've got is called cluster dot pi.
I decided there was enough code, I didn't want to put it all in one file.
I was getting confused.
So I said, let me create a file that has some of the code and a different file that will then import it and use it.
Cluster has things that are pretty much unrelated to this example, but just useful for clustering.
So an example here has name, features, and label.
And really, the only interesting thing in it-- and it's not that interesting-- is distance.
And the fact that I'm using Minkowski with 2 says we're using Euclidean distance.
Class cluster.
It's a lot more code to that one.
So we start with a non-empty list of examples.
That's what init does.
You can imagine what the code looks like, or you can look at it.
Update is interesting in that it takes the cluster and examples and puts them in-- if you think of k-means in the cluster closest to the previous centroids and then returns the amount the centroid has changed.
So if the centroid has changed by 0, then you don't have anything, right?
Creates the new cluster.
And the most interesting thing is computeCentroid.
And if you look at this code, you can see that I'm a slightly unreconstructed Python 2 programmers.
I just noticed this.
I really shouldn't have written 0.0.
I should have just written 0, but in Python 2, you had to write that 0.0.
Sorry about that.
Thought I'd fixed these.
Anyway, so how do we compute the centroid?
We start by creating an array of all 0s.
The dimensionality is the number of features in the example.
It's one of the methods from-- I didn't put up on the PowerPoint.
And then for e in examples, I'm going to add to vals e.getFeatures, and then I'm just going to divide vals by the length of self.examples, the number of examples.
So now you see why I made it a pylab array, or a numpy array rather than a list, so I could do nice things like divide the whole thing in one expression.
As you do math, any kind of math things, you'll find these arrays are incredibly convenient.
Rather than having to write recursive functions or do bunches of iterations, the fact that you can do it in one keystroke is incredibly nice.
And then I'm going to return the centroid.
Variability is exactly what we saw in the formula.
And then just for fun, so you could see this, I used an iterator here.
I don't know that any of you have used the yield statement in Python.
I recommend it.
It's very convenient.
One of the nice things about Python is almost anything that's built in, you can make your own version of it.
And so once I've done this, if c is a cluster, I can now write something like for c in big C, and this will make it work just like iterating over a list.
Right, so this makes it possible to iterate over it.
If you haven't read about yield, you probably should read the probably about two paragraphs in the textbook explaining how it works, but it's very convenient.
Dissimilarity we've already seen.
All right, now we get to patients.
This is in the file lec 12, lecture 12 dot py.
In addition to importing the usual suspects of pylab and numpy, and probably it should import random too, it imports cluster, the one we just looked at.
And so patient is a sub-type of cluster.Example.
Then I'm going to define this interesting thing called scale attributes.
So you might remember, in the last lecture when Professor Grimson was looking at these reptiles, he ran into this problem about alligators looking like chickens because they each have a large number of legs.
And he said, well, what can we do to get around this?
Well, we can represent the feature as a binary number.
Has legs, doesn't have legs.
0 or 1.
And the problem he was dealing with is that when you have a feature vector and the dynamic range of some features is much greater than the others, they tend to dominate because the distances just look bigger when you get Euclidean distance.
So for example, if we wanted to cluster the people in this room, and I had one feature that was, say, 1 for male and 0 for female, and another feature that was 1 for wears glasses, 0 for doesn't wear glasses, and then a third feature which was weight, and I clustered them, well, weight would always completely dominate the Euclidean distance, right?
Because the dynamic range of the weights in this room is much higher than the dynamic range of 0 to 1.
And so for the reptiles, he said, well, OK, we'll just make it a binary variable.
But maybe we don't want to make weight a binary variable, because maybe it is something we want to take into account.
So what we do is we scale it.
So this is a method called z-scaling.
More general than just making things 0 or 1.
It's a simple code.
It takes in all of the values of a specific feature and then performs some simple calculations, and when it's done, the resulting array it returns has a known mean and a known standard deviation.
So what's the mean going to be?
It's always going to be the same thing, independent of the initial values.
Take a look at the code.
Try and see if you can figure it out.
Anybody want to take a guess at it?
0.
Right?
So the mean will always be 0.
And the standard deviation, a little harder to figure, but it will always be 1.
OK?
So it's done this scaling.
This is a very common kind of scaling called z-scaling.
The other way people scale is interpolate.
They take the smallest value and call it 0, the biggest value, they call it 1, and then they do a linear interpolation of all the values between 0 and 1.
So the range is 0 to 1.
That's also very common.
So this is a general way to get all of the features sort of in the same ballpark so that we can compare them.
And we'll look at what happens when we scale and when we don't scale.
And that's why my getData function has this parameter to scale.
It either creates a set of examples with the attributes as initially or scaled.
And then there's k-means.
It's exactly the algorithm I showed you with one little wrinkle, which is this part.
You don't want to end up with empty clusters.
If I tell you I want four clusters, I don't mean I want three with examples and one that's empty, right?
Because then I really don't have four clusters.
And so this is one of multiple ways to avoid having empty clusters.
Basically what I did here is say, well, I'm going to try a lot of different initial conditions.
If one of them is so unlucky to give me an empty cluster, I'm just going to skip it and go on to the next one by raising a value error, empty cluster.
And if you look at the code, you'll see how this value error is used.
And then try k-means.
We'll call k-means numTrial times, each one getting a different set of initial centroids, and return the result with the lowest dissimilarity.
Then I have various ways to examine the results.
Nothing very interesting, and here's the key place where we're going to run the whole thing.
We'll get the data, initially not scaling it, because remember, it defaults to true.
Then initially, I'm only going to try one k. k equals 2.
And we'll call testClustering with the patients.
The number of clusters, k. I put in seed as a parameter here because I wanted to be able to play with it and make sure I got different things for 0 and 1 and 2 just as a testing thing.
And five trials it's defaulting to.
And then we'll look at testClustering is returning the fraction of positive examples for each cluster.
OK?
So let's see what happens when we run it.
All right.
So we got two clusters.
Cluster of size 118 with .3305, and a cluster of size 132 with a positive fraction of point quadruple 3.
Should we be happy?
Does our clustering tell us anything, somehow correspond to the expected outcome for patients here?
Probably not, right?
Those numbers are pretty much indistinguishable statistically.
And you'd have to guess that the fraction of positives in the whole population is around .33, right?
That about a third of these people died of their heart attack.
And I might as well have signed them randomly to the two clusters, right?
There's not much difference between this and what you would get with the random result.
Well, why do we think that's true?
Because I didn't scale, right?
And so one of the issues we had to deal with is, well, age had a big dynamic range, and, say, ST elevation, which I told you was highly diagnostic, was either 0 or 1.
And so probably everything is getting swamped by age or something else, right?
All right, so we have an easy way to fix that.
We'll just scale the data.
Now let's see what we get.
All right.
That's interesting.
With casting rule?
Good grief.
That caught me by surprise.
Good thing I have the answers in PowerPoint to show you, because the code doesn't seem to be working.
Try it once more.
No.
All right, well, in the interest of getting through this lecture on schedule, we'll go look at the results that we get-- I got last time I ran it.
All right.
When I scaled, what we see here is that now there is a pretty dramatic difference, right?
One of the clusters has a much higher fraction of positive patients than others, but it's still a bit problematic.
So this has pretty good specificity, or positive predictive value, but its sensitivity is lousy.
Remember, a third of our initial population more or less, was positive.
26 is way less than a third, so in fact I've got a class, a cluster, that is strongly enriched, but I'm still lumping most of the positive patients into the other cluster.
And in fact, there are 83 positives.
Wrote some code to do that.
And so we see that of the 83 positives, only this class, which is 70% positive, only has 26 in it to start with it.
So I'm clearly missing most of the positives.
So why?
Well, my hypothesis was that different subgroups of positive patients have different characteristics.
And so we could test this by trying other values of k to see with-- we would get more clusters.
So here, I said, let's try k equals 2, 4, and 6.
And here's what I got when I ran that.
So what you'll notice here, as we get to, say, 4, that I have two clusters, this one and this one, which are heavily enriched with positive patients.
26 as before in the first one, but 76 patients in the third one.
So I'm now getting a much higher fraction of patients in one of the "risky" clusters.
And I can continue to do that, but if I look at k equals 6, we now look at the positive clusters.
There were three of them significantly positive.
But I'm not really getting a lot more patients total, so maybe 4 is the right answer.
So what you see here is that we have at least two parameters to play with, scaling and k. Even though I was only wanted a structure that would separate the risk-- high-risk patients from the lower-risk, which is why I started with 2, I later discovered that, in fact, there are multiple reasons for being high-risk.
And so maybe one of these clusters is heavily enriched by old people.
Maybe another one is heavily enriched by people who have had three heart attacks in the past, or ST elevation or some combination.
And when I had only two clusters, I couldn't get that fine gradation.
So this is what data scientists spend their time doing when they're doing clustering, is they actually have multiple parameters.
They try different things out.
They look at the results, and that's why you actually have to think to manipulate data rather than just push a button and wait for the answer.
All right.
More of this general topic on Wednesday when we're going to talk about classification.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, everybody.
Before we start the material, a couple of announcements.
As usual, there's some reading assignments, and you might be surprised to see something from Chapter 5 suddenly popping up.
But this is my relentless attempt to introduce more Python.
We'll see one new concept later today, list comprehension.
Today we're going to look at classification.
And you remember last, on Monday, we looked at unsupervised learning.
Today we're looking at supervised learning.
It can usually be divided into two categories.
Regression, where you try and predict some real number associated with the feature vector, and this is something we've already done really, back when we looked at curve fitting, linear regression in particular.
It was exactly building a model that, given some features, would predict a point.
In this case, it was pretty simple.
It was given x predict y.
You can imagine generalizing that to multi dimensions.
Today I'm going to talk about classification, which is very common, in many ways more common than regression for-- in the machine learning world.
And here the goal is to predict a discrete value, often called a label, associated with some feature vector.
So this is the sort of thing where you try and, for example, predict whether a person will have an adverse reaction to a drug.
You're not looking for a real number, you're looking for will they get sick, will they not get sick.
Maybe you're trying to predict the grade in a course A, B, C, D, and other grades we won't mention.
Again, those are labels, so it doesn't have to be a binary label but it's a finite number of labels.
So here's an example to start with.
We won't linger on it too long.
This is basically something you saw in an earlier lecture, where we had a bunch of animals and a bunch of properties, and a label identifying whether or not they were a reptile.
So we start by building a distance matrix.
How far apart they are, an in fact, in this case, I'm not using the representation you just saw.
I'm going to use the binary representation, As Professor Grimson showed you, and for the reasons he showed you.
If you're interested, I didn't produce this table by hand, I wrote some Python code to produce it, not only to compute the distances, but more delicately to produce the actual table.
And you'll probably find it instructive at some point to at least remember that that code is there, in case you need to ever produce a table for some paper.
In general, you probably noticed I spent relatively little time going over the actual vast amounts of codes we've been posting.
That doesn't mean you shouldn't look at it.
In part, a lot of it's there because I'm hoping at some point in the future it will be handy for you to have a model on how to do something.
All right.
So we have all these distances.
And we can tell how far apart one animal is from another.
Now how do we use those to classify animals?
And the simplest approach to classification, and it's actually one that's used a fair amount in practice is called nearest neighbor.
So the learning part is trivial.
We don't actually learn anything other than we just remember.
So we remember the training data.
And when we want to predict the label of a new example, we find the nearest example in the training data, and just choose the label associated with that example.
So here I've just drawing a cloud of red dots and black dots.
I have a fuschia colored X.
And if I want to classify X as black or red, I'd say well its nearest neighbor is red.
So we'll call X red.
Doesn't get much simpler than that.
All right.
Let's try and do it now for our animals.
I've blocked out this lower right hand corner, because I want to classify these three animals that are in gray.
So my training data, very small, are these animals.
And these are my test set here.
So let's first try and classify the zebra.
We look at the zebra's nearest neighbor.
Well it's either a guppy or a dart frog.
Well, let's just choose one.
Let's choose the guppy.
And if we look at the guppy, it's not a reptile, so we say the zebra is not a reptile.
So got one right.
Look at the python, choose its nearest neighbor, say it's a cobra.
The label associated with cobra is reptile, so we win again on the python.
Alligator, it's nearest neighbor is clearly a chicken.
And so we classify the alligator as not a reptile.
Oh, dear.
Clearly the wrong answer.
All right.
What might have gone wrong?
Well, the problem with K nearest neighbors, we can illustrate it by looking at this example.
So one of the things people do with classifiers these days is handwriting recognition.
So I just copied from a website a bunch of numbers, then I wrote the number 40 in my own inimitable handwriting.
So if we go and we look for, say, the nearest neighbor of four-- or sorry, of whatever that digit is.
It is, I believe, this one.
And sure enough that's the row of fours.
We're OK on this.
Now if we want to classify my zero, the actual nearest neighbor, in terms of the bitmaps if you will, turns out to be this guy.
A very poorly written nine.
I didn't make up this nine, it was it was already there.
And the problem we see here when we use nearest neighbor is if something is noisy, if you have one noisy piece of data, in this case, it's rather ugly looking version of nine, you can get the wrong answer because you match it.
And indeed, in this case, you would get the wrong answer.
What is usually done to avoid that is something called K nearest neighbors.
And the basic idea here is that we don't just take the nearest neighbors, we take some number of nearest neighbors, usually an odd number, and we just let them vote.
So now if we want to classify this fuchsia X, and we said K equal to three, we say well these are it's three nearest neighbors.
One is red, two are black, so we're going to call X black is our better guess.
And maybe that actually is a better guess, because it looks like this red point here is really an outlier, and we don't want to let the outliers dominate our classification.
And this is why people almost always use K nearest neighbors rather than just nearest neighbor.
Now if we look at this, and we use K nearest neighbors, those are the three nearest to the first numeral, and they are all fours.
And if we look at the K nearest neighbors for the second numeral, we still have this nine but now we have two zeros.
And so we vote and we decide it's a zero.
Is it infallible?
No.
But it's typically much more reliable than just nearest neighbors, hence used much more often.
And that was our problem, by the way, with the alligator.
The nearest neighbor was the chicken, but if we went back and looked at it-- maybe we should go do that.
And we take the alligator's three nearest neighbors, it would be the chicken, a cobra, and the rattlesnake-- or the boa, we don't care, and we would end up correctly classifying it now as a reptile.
Yes?
Is there like a limit to how many [INAUDIBLE]?
The question is is there a limit to how many nearest neighbors you'd want?
Absolutely.
Most obviously, there's no point in setting K equal to-- whoops.
Ooh, on the rebound-- to the size of the training set.
So one of the problems with K nearest neighbors is efficiency.
If you're trying to define K nearest neighbors and K is bigger, it takes longer.
So we worry about how big K should be.
And if we make it too big-- and this is a crucial thing-- we end up getting dominated by the size of the class.
So let's look at this picture we had before.
It happens to be more red dots than black dots.
If I make K 10 or 15, I'm going to classify a lot of things as red, just because red is so much more prevalent than black.
And so when you have an imbalance, which you usually do, you have to be very careful about K. Does that make sense?
[INAUDIBLE] choose K?
So how do you choose K?
Remember back on Monday when we talked about choosing K for K means clustering?
We typically do a very similar kind of thing.
We take our training data and we split it into two parts.
So we have training and testing, but now we just take the training, and we split that into training and testing multiple times.
And we experiment with different K's, and we see which K's gives us the best result on the training data.
And then that becomes our K. And that's a very common method.
It's called cross-validation, and it's-- for almost all of machine learning, the algorithms have parameters in this case, it's just one parameter, K. And the way we typically choose the parameter values is by searching through the space using this cross-validation in the training data.
Does that makes sense to everybody?
Great question.
And there was someone else had a question, but maybe it was the same.
Do you still have a question?
Well, just that you were using like K nearest and you get, like if my K is three and I get three different clusters for the K [INAUDIBLE]
Three different clusters?
[INAUDIBLE]
Well, right.
So if K is 3, and I had red, black, and purple and I get one of each, then what do I do?
And then I'm kind of stuck.
So you need to typically choose K in such a way that when you vote you get a winner.
Nice.
So if there's two, any odd number will do.
If it's three, well then you need another number so that there's some-- so there's always a majority.
Right?
You want to make sure that there is a winner.
Also a good question.
Let's see if I get this to you directly.
I'm much better at throwing overhand, I guess.
Wow.
Finally got applause for something.
All right, advantages and disadvantages KNN?
The learning is really fast, right?
I just remember everything.
No math is required.
Didn't have to show you any theory.
Was obviously an idea.
It's easy to explain the method to somebody, and the results.
Why did I label it black?
Because that's who it was closest to.
The disadvantages is it's memory intensive.
If I've got a million examples, I have to store them all.
And the predictions can take a long time.
If I have an example and I want to find its K nearest neighbors, I'm doing a lot of comparisons.
Right?
If I have a million tank training points I have to compare my example to all a million.
So I have no real pre-processing overhead.
But each time I need to do a classification, it takes a long time.
Now there are better algorithms and brute force that give you approximate K nearest neighbors.
But on the whole, it's still not fast.
And we're not getting any information about what process might have generated the data.
We don't have a model of the data in the way we say when we did our linear regression for curve fitting, we had a model for the data that sort of described the pattern.
We don't get that out of k nearest neighbors.
I'm going to show you a different approach where we do get that.
And I'm going to do it on a more interesting example than reptiles.
I apologize to those of you who are reptologists.
So you probably all heard of the Titanic.
There was a movie about it, I'm told.
It was one of the great sea disasters of all time, a so-called unsinkable ship-- they had advertised it as unsinkable-- hit an iceberg and went down.
Of the 1,300 passengers, 812 died.
The crew did way worse.
So at least it looks as if the curve was actually pretty heroic.
They had a higher death rate.
So we're going to use machine learning to see if we can predict which passengers survived.
There's an online database I'm using.
It doesn't have all 1,200 passengers, but it has information about 1,046 of them.
Some of them they couldn't get the information.
Says what cabin class they were in first, second, or third, how old they were, and their gender.
Also has their name and their home address and things, which I'm not using.
We want to use these features to see if we can predict which passengers were going to survive the disaster.
Well, the first question is something that Professor Grimson alluded to is, is it OK, just to look at accuracy?
How are we going to evaluate our machine learning?
And it's not.
If we just predict died for everybody, well then we'll be 62% accurate for the passengers and 76% accurate for the crew members.
Usually machine learning, if you're 76% you say that's not bad.
Well, here I can get that just by predicting died.
So whenever you have a class imbalance that much more of one than the other, accuracy isn't a particularly meaningful measure.
I discovered this early on in my work and medical area.
There are a lot of diseases that rarely occur, they occur in say 0.1% of the population.
And I can build a great model for predicting it by just saying, no, you don't have it, which will be 0.999% accurate, but totally useless.
Unfortunately, you do see people doing that sort of thing in the literature.
You saw these in an earlier lecture, just to remind you, we're going to be looking at other metrics.
Sensitivity, think of that as how good is it at identifying the positive cases.
In this case, positive is going to be dead.
How specific is it, and the positive predictive value.
If we say somebody died, what's the probability is that they really did?
And then there's the negative predictive value.
If we say they didn't die, what's the probability they didn't die?
So these are four very common metrics.
There is something called an F score that combines them, but I'm not going to be showing you that today.
I will mention that in the literature, people often use the word recall to mean sensitivity or sensitivity I mean recall, and specificity and precision are used pretty much interchangeably.
So you might see various combinations of these words.
Typically, people talk about recall n precision or sensitivity and specificity.
Does that makes sense, why we want to look at the measures other than accuracy?
We will look at accuracy, too, and how they all tell us kind of different things, and how you might choose a different balance.
For example, if I'm running a screening test, say for breast cancer, a mammogram, and trying to find the people who should get on for a more extensive examination, what do I want to emphasize here?
Which of these is likely to be the most important?
Or what would you care about most?
Well, maybe I want sensitivity.
Since I'm going to send this person on for future tests, I really don't want to miss somebody who has cancer, and so I might think sensitivity is more important than specificity in that particular case.
On the other hand, if I'm deciding who is so sick I should do open heart surgery on them, maybe I want to be pretty specific.
Because the risk of the surgery itself are very high.
I don't want to do it on people who don't need it.
So we end up having to choose a balance between these things, depending upon our application.
The other thing I want to talk about before actually building a classifier is how we test our classifier, because this is very important.
I'm going to talk about two different methods, leave one out class of testing and repeated random subsampling.
For leave one out, it's typically used when you have a small number of examples, so you want as much training data as possible as you build your model.
So you take all of your n examples, remove one of them, train on n minus 1, test on the 1.
Then you put that 1 back and remove another 1.
Train on n minus 1, test on 1.
And you do this for each element of the data, and then you average your results.
Repeated random subsampling is done when you have a larger set of data, and there you might say split your data 80/20.
Take 80% of the data to train on, test it on 20.
So this is very similar to what I talked about earlier, and answered the question about how to choose K. I haven't seen the future examples, but in order to believe in my model and say my parameter settings, I do this repeated random subsampling or leave one out, either one.
There's the code for leave one out.
Absolutely nothing interesting about it, so I'm not going to waste your time looking at it.
Repeated random subsampling is a little more interesting.
What I've done here is I first sample-- this one is just to splitted 80/20.
It's not doing anything repeated, and I start by sampling 20% of the indices, not the samples.
And I want to do that at random.
I don't want to say get consecutive ones.
So we do that, and then once I've got the indices, I just go through and assign each example, to either test or training, and then return the two sets.
But if I just sort of sampled one, then I'd have to do a more complicated thing to subtract it from the other.
This is just efficiency.
And then here's the-- sorry about the yellow there-- the random splits.
Obviously, I was searching for results when I did my screen capture.
I'm just going to for range and number of splits, I'm going to split it 80/20.
It takes a parameter method, and that's interesting, and we'll see the ramifications of that later.
That's going to be the machine learning method.
We're going to compare KNN to another method called logistic regression.
I didn't want to have to do this code twice, so I made the method itself a parameter.
We'll see that introduces a slight complication, but we'll get to it when we get to it.
So I split it, I apply whatever that method is the training the test set, I get the results, true positive false positive, true negative false negatives.
And then I call this thing get stats, but I'm dividing it by the number of splits, so that will give me the average number of true positives, the average number of false positives, etc.
And then I'm just going to return the average.
Get stats actually just prints a bunch of statistics for us.
Any questions about the two methods, leave one out versus repeated random sampling?
Let's try it for KNN on the Titanic.
So I'm not going to show you the code for K nearest classify.
It's in the code we uploaded.
It takes four arguments the training set, the test set, the label that we're trying to classify.
Are we looking for the people who died?
Or the people who didn't die?
Are we looking for reptiles or not reptiles?
Or if case there were six labels, which one are we trying to detect?
And K as in how many nearest neighbors?
And then it returns the true positives, the false positives, the true negatives, and the false negatives.
Then you'll recall we'd already looked at lambda in a different context.
The issue here is K nearest classify takes four arguments, yet if we go back here, for example, to random splits, what we're seeing is I'm calling the method with only two arguments.
Because after all, if I'm not doing K nearest neighbors, maybe I don't need to pass in K. I'm sure I don't.
Different methods will take different numbers of parameters, and yet I want to use the same function here method.
So the trick I use to get around that-- and this is a very common programming trick-- in math.
It's called currying, after the mathematician Curry, not the Indian dish.
I'm creating a function a new function called KNN.
This will be a function of two arguments, the training set and the test set, and it will be K nearest classifier with training set and test set as variables, and two constants, survived-- so I'm going to predict who survived-- and 3, the K. I've been able to turn a function of four arguments, K nearest classify, into a function of two arguments KNN by using lambda abstraction.
This is something that people do fairly frequently, because it lets you build much more general programs when you don't have to worry about the number of arguments.
So it's a good trick to keeping your bag of tricks.
Again, it's a trick we've used before.
Then I've just chosen 10 for the number of splits, and we'll try it, and we'll try it for both methods of testing.
Any questions before I run this code?
So here it is.
We'll run it.
Well, I should learn how to spell finished, shouldn't I?
But that's OK.
Here we have the results, and they're-- well, what can we say about them?
They're not much different to start with, so it doesn't appear that our testing methodology had much of a difference on how well the KNN worked, and that's actually kind of comforting.
The accurate-- none of the evaluation criteria are radically different, so that's kind of good.
We hoped that was true.
The other thing to notice is that we're actually doing considerably better than just always predicting, say, didn't survive.
We're doing better than a random prediction.
Let's go back now to the Power Point.
Here are the results.
We don't need to study them anymore.
Better than 62% accuracy, but not much difference between the experiments.
So that's one method.
Now let's look at a different method, and this is probably the most common method used in machine learning.
It's called logistic regression.
It's, in some ways, if you look at it, similar to a linear regression, but different in some important ways.
Linear regression, you will I'm sure recall, is designed to predict a real number.
Now what we want here is a probability, so the probability of some event.
We know that the dependent variable can only take on a finite set of values, so we want to predict survived or didn't survive.
It's no good to say we predict this person half survived, you know survived, but is brain dead or something.
I don't know.
That's not what we're trying to do.
The problem with just using regular linear regression is a lot of time you get nonsense predictions.
Now you can claim, OK 0.5 is there, and it means has a half probability of dying, not that half died.
But in fact, if you look at what goes on, you could get more than one or less than 0 out of linear regression, and that's nonsense when we're talking about probabilities.
So we need a different method, and that's logistic regression.
What logistic regression does is it finds what are called the weights for each feature.
You may recall I complained when Professor [?
Grimson ?]
used the word weights to mean something somewhat different.
We take each feature, for example the gender, the cabin class, the age, and compute for that weight that we're going to use in making predictions.
So think of the weights as corresponding to the coefficients we get when we do a linear regression.
So we have now a coefficient associated with each variable.
We're going to take those coefficients, add them up, multiply them by something, and make a prediction.
A positive weight implies-- and I'll come back to this later-- it almost implies that the variable is positively correlated with the outcome.
So we would, for example, say the have scales is positively correlated with being a reptile.
A negative weight implies that the variable is negatively correlated with the outcome, so number of legs might have a negative weight.
The more legs an animal has, the less likely it is to be a reptile.
It's not absolute, it's just a correlation.
The absolute magnitude is related to the strength of the correlation, so if it's being positive it means it's a really strong indicator.
If it's big negative, it's a really strong negative indicator.
And then we use an optimization process to compute these weights from the training data.
It's a little bit complex.
It's key is the way it uses the log function, hence the name logistic, but I'm not going to make you look at it.
But I will show you how to use it.
You start by importing something called sklearn.linear_model.
Sklearn is a Python library, and in that is a class called logistic regression.
It's the name of a class, and here are three methods of that class.
Fit, which takes a sequence of feature vectors and a sequence of labels and returns an object of type logistic regression.
So this is the place where the optimization is done.
Now all the examples I'm going to show you, these two sequences will be-- well all right.
So think of this as the sequence of feature vectors, one per passenger, and the labels associated with those.
So this and this have to be the same length.
That produces an object of this type, and then I can ask for the coefficients, which will return the weight of each variable, each feature.
And then I can make a prediction, given a feature vector returned the probabilities of different labels.
Let's look at it as an example.
So first let's build the model.
To build the model, we'll take the examples, the training data, and I just said whether we're going to print something.
You'll notice from this slide I've elighted the printed stuff.
We'll come back in a later slide and look at what's in there.
But for now I want to focus on actually building the model.
I need to create two vectors, two lists in this case, the feature vectors and the labels.
For e in examples, featurevectors.a ppend(e.getfeatures e.getfeatures e.getlabel.
Couldn't be much simpler than that.
Then, just because it wouldn't fit on a line on my slide, I've created this identifier called logistic regression, which is sklearn.linearmo del.logisticregression.
So this is the thing I imported, and this is a class, and now I'll get a model by first creating an instance of the class, logistic regression.
Here I'm getting an instance, and then I'll call dot fit with that instance, passing it feature vecs and labels.
I now have built a logistic regression model, which is simply a set of weights for each of the variables.
This makes sense?
Now we're going to apply the model, and I think this is the last piece of Python I'm going to introduce this semester, in case you're tired of learning about Python.
And this is at least list comprehension.
This is how I'm going to build my set of test feature vectors.
So before we go and look at the code, let's look at how list comprehension works.
In its simplest form, says some expression for some identifier in some list, L. It creates a new list by evaluating this expression Len (L) times with the ID in the expression replaced by each element of the list L. So let's look at a simple example.
Here I'm saying L equals x times x for x in range 10.
What's that going to do?
It's going to, essentially, create a list.
Think of it as a list, or at least a sequence of values, a range type actually in Python 3-- of values 0 to 9.
It will then create a list of length 10, where the first element is going to be 0 times 0.
The second element 1 times 1, etc.
OK?
So it's a simple way for me to create a list that looks like that.
I can be fancier and say for x times L equals x times x for x in range 10, and I add and if.
If x mod 2 is equal to 0.
Now instead of returning all-- building a list using each value in range 10, it will use only those values that satisfy that test.
We can go look at what happens when we run that code.
You can see the first list is 1 times 1, 2 times 2, et cetera, and the second list is much shorter, because I'm only squaring even numbers.
Well, you can see that list comprehension gives us a convenient compact way to do certain kinds of things.
Like lambda expressions, they're easy to misuse.
I hate reading code where I have list comprehensions that go over multiple lines on my screen, for example.
So I use it quite a lot for small things like this.
If it's very large, I find another way to do it.
Now we can move forward.
In applying the model, I first build my testing feature of x, my e.getfeatures for e in test set, so that will give me the features associated with each element in the test set.
I could obviously have written a for loop to do the same thing, but this was just a little cooler.
Then we get model.predict for each of these.
Model.predict_proba is nice in that I don't have to predict it for one example at a time.
I can pass it as set of examples, and what I get back is a list of predictions, so that's just convenient.
And then setting these to 0, and for I in range len of probs, here a probability of 0.5.
What's that's saying is what I get out of logistic regression is a probability of something having a label.
I then have to build a classifier, give a threshold.
And here what I've said, if the probability of it being true is over a 0.5, call it true.
So if the probability of survival is over 0.5, call it survived.
If it's below, call it not survived.
We'll later see that, again, setting that probability is itself an interesting thing, but the default in most systems is half, for obvious reasons.
I get my probabilities for each feature vector, and then for I in ranged lens of probabilities, I'm just testing whether the predicted label is the same as the actual label, and updating true positives, false positives, true negatives, and false negatives accordingly.
So far, so good?
All right, let's put it all together.
I'm defining something called LR, for logistic regression.
It takes the training data, the test data, the probability, it builds a model, and then it gets the results by calling apply model with the label survived and whatever this prob was.
Again, we'll do it for both leave one out and random splits, and again for 10 random splits.
You'll notice it actually runs-- maybe you won't notice, but it does run faster than KNN.
One of the nice things about logistic regression is building the model takes a while, but once you've got the model, applying it to a large number of variables-- feature vectors is fast.
It's independent of the number of training examples, because we've got our weights.
So solving the optimization problem, getting the weights, depends upon the number of training examples.
Once we've got the weights, it's just evaluating a polynomial.
It's very fast, so that's a nice advantage.
If we look at those-- and we should probably compare them to our earlier KNN results, so KNN on the left, logistic regression on the right.
And I guess if I look at it, it looks like logistic regression did a little bit better.
That's not guaranteed, but it often does outperform because it's more subtle in what it does, in being able to assign different weights to different variables.
It's a little bit better.
That's probably a good thing, but there's another reason that's really important that people prefer logistic regression, is it provides insights about the variables.
We can look at the feature weights.
This code does that, so remember we looked at build model and I left out the printing?
Well here I'm leaving out everything except the printing.
Same function, but leaving out everything except the printing.
We can do model underbar classes, so model.classes underbar gives you the classes.
In this case, the classes are survived, didn't survive.
I forget what I called it.
We'll see.
So I can see what the classes it's using are, and then for I in range len model dot cof underbar, these are giving the weights of each variable.
The coefficients, I can print what they are.
So let's run that and see what we get.
We get a syntax error because I turned a comment into a line of code.
Our model classes are died and survived, and for label survived-- what I've done, by the way, in the representation is I represented the cabin class as a binary variable.
It's either 0 or 1, because it doesn't make sense to treat them as if they were really numbers because we don't know, for example, the difference between first and second is the same as the difference between second and third.
If we treated the class, we just said cabin class and used an integer, implicitly the learning algorithm is going to assume that the difference between 1 and 2 is the same as between 2 and 3.
If you, for example, look at the prices of these cabins, you'll see that that's not true.
The difference in an airplane between economy plus and economy is way smaller than between economy plus him first.
Same thing on the Titanic.
But what we see here is that for the label survived, pretty good sized positive weight for being in first class cabin.
Moderate for being in the second, and if you're in the third class well, tough luck.
So what we see here is that rich people did better than the poor people.
Shocking.
If We look at age, we'll see it's negatively correlated.
What does this mean?
It's not a huge weight, but it basically says that if you're older, the bigger your age, the less likely you are to have survived the disaster.
And finally, it says it's really bad to be a male, that the men-- being a male was very negatively correlated with surviving.
We see a nice thing here is we get these labels, which we can make sense of.
One more slide and then I'm done.
These values are slightly different, because different randomization, different example, but the main point I want to say is you have to be a little bit wary of reading too much into these weights.
Because not in this example, but other examples-- well, also in these features are often correlated, and if they're correlated, you run-- actually it's 3:56.
I'm going to explain the problem with this on Monday when I have time to do it properly.
So I'll see you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, everybody.
Some announcements.
The last reading assignment of the semester, at least from us.
Course evaluations are still available through this Friday.
But only till noon.
Again, I urge you all to do it.
And then finally, for the final exam, we're going to be giving you some code to study in advance of the exam.
And then we will ask questions about that code on the exam itself.
This was described in the announcement for the exam.
And we will be making this code available later today.
Now, I would suggest that you try and get your heads around it.
If you are confused, that's a good thing to talk about in office hours, to get some help with it, as opposed to waiting till 20 minutes before the exam and realizing you're confused.
All right.
I want to pick up where we left off on Monday.
So you may recall that we were comparing results of KNN and logistic regression on our Titanic data.
And we have this up using 10 80/20 splits for KNN equals 3 and logistic regression with p equals 0.5.
And what I observed is that logistic regression happened to perform slightly better, but certainly nothing that you would choose to write home about.
It's a little bit better.
That isn't to say it will always be better.
It happens to be here.
But the point I closed with is one of the things we care about when we use machine learning is not only our ability to make predictions with the model.
But what can we learn by studying the model itself?
Remember, the idea is that the model is somehow capturing the system or the process that generated the data.
And by studying the model we can learn something useful.
So to do that for logistic regression, we begin by looking at the weights of the different variables.
And we had this up in the last slide.
The model classes are "Died" and "Survived."
For the label Survived, we said that if you were in a first-class cabin, that had a positive impact on your survival, a pretty strong positive impact.
You can't interpret these weights in and of themselves.
If I said it's 1.6, that really doesn't mean anything.
So what you have to look at is the relative weights, not the absolute weights.
And we see that it's a pretty strong relative weight.
A second-class cabin also has a positive weight, in this case, of 0.46.
So it was indicating you better had a better-than-average chance of surviving, but much less strong than a first class.
And if you are one of those poor people in a third-class cabin, well, that had a negative weight on survival.
You were less likely to survive.
Age had a very small effect here, slightly negative.
What that meant is the older you were, the less likely you were to have survived.
But it's a very small negative value.
The male gender had a relatively large negative gender, suggesting that if you were a male you were more likely to die than if you were a female.
This might be true in the general population, but it was especially true on the Titanic.
Finally, I warned you that while what I just went through is something you will read in lots of papers that use machine learning, you will hear in lots of talks about people who have used machine learning.
But you should be very wary when people speak that way.
It's not nonsense, but some cautionary notes.
In particular, there's a big issue because the features are often correlated with one another.
And so you can't interpret the weights one feature at a time.
To get a little bit technical, there are two major ways people use logistic regression.
They're called L1 and L2.
We used an L2.
I'll come back to that in a minute.
Because that's the default in Python, or in [INAUDIBLE].
You can set that parameter at L2 and do that to L1 if you want.
I experimented with it.
It didn't change the results that much.
But what an L1 regression is designed to do is to find some weights and drive them to 0.
This is particularly useful when you have a very high-dimensional problem relative to the number of examples.
And this gets back to that question we've talked about many times, of overfitting.
If you've got 1,000 variables and 1,000 examples, you're very likely to overfit.
L1 is designed to avoid overfitting by taking many of those 1,000 variables and just giving them 0 weight.
And it does typically generalize better.
But if you have two variables that are correlated, L1 will drive 1 to 0, and it will look like it's unimportant.
But in fact, it might be important.
It's just correlated with another, which has gotten all the credit.
L2, which is what we did, does the opposite.
Is spreads the weight across the variables.
So have a bunch of correlated variables, it might look like none of them are very important.
Because each of them gets a small amount of the weight.
Again, not so important when you have four or five variables, is what I'm showing you.
But it matters when you have 100 or 1,000 variables.
Let's look at an example.
So the cabin classes, the way we set it up, c1 plus c2 plus c3-- whoops-- is not equal to 0.
What is it equal to?
I'll fix this right now.
What should that have said?
What's the invariant here?
Well, a person is in exactly one class.
I guess if you're really rich, maybe you rented two cabins, one in first and one in second.
But probably not.
Or if you did, you put your servants in second or third.
But what does this got to add up to?
Yeah?
1
Has to add up to 1.
Thank you.
So it adds up to 1.
Whoa.
Got his attention, at least.
So what this tells us is the values are not independent.
Because if c1 is 1, then c2 and c3 must be 0.
Right?
And so now we could go back to the previous slide and ask the question well, is it that being in first class is protective?
Or is it that being in second or third class is risky?
And there's no simple answer to that.
So let's do an experiment.
We have these correlated variables.
Suppose we eliminate c1 altogether.
So I did that by changing the init method of class passenger.
Takes the same arguments, but we'll look at the code.
Because it's a little bit clearer there.
So there was the original one.
And I'm going to replace that by this.
combine that with the original one.
So what you see is that instead of having five features, I now have four.
I've eliminated the c1 binary feature.
And then the code is straightforward, that I've just come through here, and I've just enumerated the possibilities.
So if you're in first class, then second and third are both 0.
Otherwise, one of them is a 1.
So my invariant is gone now, right?
It's not the case that we know that these two things have to add up to 1, because maybe I'm in the third case.
OK, let's go run that code and see what happens.
Well, if you remember, we see that our accuracy has not really declined much.
Pretty much the same results we got before.
But our weights are really quite different.
Now, suddenly, c2 and c3 have large negative weights.
We can look at them side by side here.
So you see, not much difference.
It actually performs maybe-- well, really no real difference in performance.
But you'll notice that the weights are really quite different.
That now, what had been a strong positive weight and relatively weak negative weights is now replaced by two strong negative weights.
And age and gender change just a little bit.
So the whole point here is that we have to be very careful, when you have correlated features, about over-interpreting the weights.
It is generally pretty safe to rely on the sign, whether it's negative or positive.
All right, changing the topic but sticking with logistic regression, there is this parameter you may recall, p, which is the probability.
And that was the cut-off.
And we set it to 0.5, saying if it estimates the probability of survival to be 0.5 or higher, then we're going to guess survived, predict survived.
Otherwise, deceased.
You can change that.
And so I'm going to try two extreme values, setting p to 0.1 and p to 0.9.
Now, what do we think that's likely to change?
Remember, we looked at a bunch of different attributes.
In particular, what attributes do we think are most likely to change?
Anyone who has not answered a question want to volunteer?
I have nothing against you, it's just I'm trying to spread the wealth.
And I don't want to give you diabetes, with all the candy.
All right, you get to go again
Sensitivity
Pardon?
The sensitivity and specificity
Sensitivity and specificity, positive predictive value.
Because we're shifting.
And we're saying, well, by changing the probability, we're making a decision that it's more important to not miss survivors than it is to, say, ask gets too high.
So let's look at what happens when we run that.
I won't run it for you.
But these are the results we got.
So as it happens, 0.9 gave me higher accuracy.
But the key thing is, notice the big difference here.
So what is that telling me?
Well, it's telling me that if I predict you're going to survive you probably did.
But look what it did to the sensitivity.
It means that most of the survivors, I'm predicting they died.
Why is the accuracy still OK?
Well, because most people died on the boat, on the ship, right?
So we would have done pretty well, you recall, if we just guessed died for everybody.
So it's important to understand these things.
I once did some work using machine learning for an insurance company who was trying to set rates.
And I asked them what they wanted to do.
And they said they didn't want to lose money.
They didn't want to insure people who were going to get in accidents.
So I was able to change this p parameter so that it did a great job.
The problem was they got to write almost no policies.
Because I could pretty much guarantee the people I said wouldn't get in an accident wouldn't.
But there were a whole bunch of people who didn't, who they wouldn't write policies for.
So they ended up not making any money.
It was a bad decision.
So we can change the cutoff.
That leads to a really important concept of something called the Receiver Operating Characteristic.
And it's a funny name, having to do with it originally going back to radio receivers.
But we can ignore that.
The goal here is to say, suppose I don't want to make a decision about where the cutoff is, but I want to look at, in some sense, all possible cutoffs and look at the shape of it.
And that's what this code is designed to do.
So the way it works is I'll take a training set and a test set, usual thing.
I'll build one model.
And that's an important thing, that there's only one model getting built.
And then I'm going to vary p. And I'm going to call apply model with the same model and the same test set, but different p's and keep track of all of those results.
I'm then going to plot a two-dimensional plot.
The y-axis will have sensitivity.
And the x-axis will have one minus specificity.
So I am accumulating a bunch of results.
And then I'm going to produce this curve calling sklearn.metrics.auc, that's not the curve.
AUC stands for Area Under the Curve.
And we'll see why we want to get that area under the curve.
When I run that, it produces this.
So here's the curve, the blue line.
And there's some things to note about it.
Way down at this end I can have 0, right?
I can set it so that I don't make any predictions.
And this is interesting.
So at this end it is saying what?
Remember that my x-axis is not specificity, but 1 minus specificity.
So what we see is this corner is highly sensitive and very unspecific.
So I'll get a lot of false positives.
This corner is very specific, because 1 minus specificity is 0, and very insensitive.
So way down at the bottom, I'm declaring nobody to be positive.
And way up here, everybody.
Clearly, I don't want to be at either of these places on the curve, right?
Typically I want to be somewhere in the middle.
And here, we can see, there's a nice knee in the curve here.
We can choose a place.
What does this green line represent, do you think?
The green line represents a random classifier.
I flip a coin and I just classify something positive or negative, depending on the heads or tails, in this case.
So now we can look at an interesting region, which is this region, the area between the curve and a random classifier.
And that sort of tells me how much better I am than random.
I can look at the whole area, the area under the curve.
And that's this, the area under the Receiver Operating Curve.
In the best of all worlds, the curve would be 1.
That would be a perfect classifier.
In the worst of all worlds, it would be 0.
But it's never 0 because we don't do worse than 0.5.
We hope not to do worse than random.
If so, we just reverse our predictions.
And then we're better than random.
So random is as bad as you can do, really.
And so this is a very important concept.
And it lets us evaluate how good a classifier is independently of what we choose to be the cutoff.
So when you read the literature and people say, I have this wonderful method of making predictions, you'll almost always see them cite the AUROC.
Any questions about this or about machine learning in general?
If so, this would be a good time to ask them, since I'm about to totally change the topic.
Yes?
At what level does AUROC start to be statistically significant?
And how many data points do you need to also prove that [INAUDIBLE]?
Right.
So the question is, at what point does the AUROC become statistically significant?
And that is, essentially, an unanswerable question.
Whoops, relay it back.
Needed to put more air under the throw.
I look like the quarterback for the Rams, if you saw them play lately.
So if you ask this question about significance, it will depend upon a number of things.
So you're always asking, is it significantly better than x?
And so the question is, is it significantly better than random?
And you can't just say, for example, that 0.6 isn't and 0.7 is.
Because it depends how many points you have.
If you have a lot of points, it could be only a tiny bit better than 0.5 and still be statistically significant.
It may be uninterestingly better.
It may not be significant in the English sense, but you still get statistical significance.
So that's a problem when studies have lots of points.
In general, it depends upon the application.
For a lot of applications, you'll see things in the 0.7's being considered pretty useful.
And the real question shouldn't be whether it's significant, but whether it's useful.
Can you make useful decisions based upon it?
And the other thing is, typically, when you're talking about that, you're selecting some point and really talking about a region relative to that point.
We usually don't really care what it does out here.
Because we hardly ever operate out there anyway.
We're usually somewhere in the middle.
But good question.
Yeah?
Why are we doing 1 minus specificity?
Why are we doing 1 minus specificity instead of specificity?
Is that the question?
And the answer is, essentially, so we can do this trick of computing the area.
It gives us this nice curve.
This nice, if you will, concave curve which lets us compute this area under here nicely if you were to take specificity and just draw it, it would look different.
Obviously, mathematically, they're, in some sense, the same right.
If you have 1 minus x and x, you can get either from the other.
So it really just has to do with the way people want to draw this picture
[INAUDIBLE]?
Pardon?
Does that not change [INAUDIBLE]?
Does it not--
Doesn't it change the meaning of what you're [INAUDIBLE]?
Well, you'd have to use a different statistic.
You couldn't cite the AUROC if you did specificity directly.
Which is why they do 1 minus.
The goal is you want to have this point at 0 and this 0.00 and 1.1.
And playing 1 minus gives you this trick, of anchoring those two points.
And so then you get a curve connecting them, which you can then easily compare to the random curve.
It's just one of these little tricks that statisticians like to play to make things easy to visualize and easy to compute statistics about.
It's not a fundamentally important issue.
Anything else?
All right, so I told you I was going to change topics-- finally got one completed-- and I am.
And this is a topic I approach with some reluctance.
So you have probably all heard this expression, that there are three kinds of lies, lies, damn lies, and statistics.
And we've been talking a lot about statistics.
And now I want to spend the rest of today's lecture and the start of Wednesday's lecture talking about how to lie with statistics.
So at this point, I usually put on my "Numbers Never Lie" hat.
But do say that numbers never lie, but liars use numbers.
And I hope none of you will ever go work for a politician and put this knowledge to bad use.
This quote is well known.
It's variously attributed, often, to Mark Twain, the fellow on the left.
He claimed not to have invented it, but said it was invented by Benjamin Disraeli.
And I prefer to believe that, since it does seem like something a Prime Minister would invent.
So let's think about this.
The issue here is the way the human mind works and statistics.
Darrell Huff, a well-known statistician who did write a book called How to Lie with Statistics, says, "if you can't prove what you want to prove, demonstrate something else and pretend they are the same thing.
In the daze that follows the collision of statistics with the human mind, hardly anyone will notice the difference."
And indeed, empirically, he seems to be right.
So let's look at some examples.
Here's one I like.
This is from another famous statistician called Anscombe.
And he invented this thing called Anscombe's Quartet.
I take my hat off now.
It's too hot in here.
A bunch of numbers, 11 x, y pairs.
I know you don't want to look at the numbers, so here are some statistics about them.
Each of those pairs has the same mean value for x, the same mean for y, the same variance for x, the same variance for y.
And then I went and I fit a linear regression model to it.
And lo and behold, I got the same equation for everyone, y equals 0.5x plus 3.
So that raises the question, if we go back, is there really much difference between these pairs of x and y?
Are they really similar?
And the answer is, that's what they look like if you plot them.
So even though statistically they appear to be kind of the same, they could hardly be more different, right?
Those are not the same distributions.
So there's an important moral here, which is that statistics about data is not the same thing as the data itself.
And this seems obvious, but it's amazing how easy it is to forget it.
The number of papers I've read where I see a bunch of statistics about the data but don't see the data is enormous.
And it's easy to lose track of the fact that the statistics don't tell the whole story.
So the answer is the old Chinese proverb, a picture is worth a thousand words, I urge you, the first thing you should do when you get a data set, is plot it.
If it's got too many points to plot all the points, subsample it and plot of subsample.
Use some visualization tool to look at the data itself.
Now, that said, pictures are wonderful.
But you can lie with pictures.
So here's an interesting chart.
These are grades in 6.0001 by gender.
So the males are blue and the females are pink.
Sorry for being such a traditionalist.
And as you can see, the women did way better than the men.
Now, I know for some of you this is confirmation bias.
You say, of course.
Others say, impossible, But in fact, if you look carefully, you'll see that's not what this chart says at all.
Because if you look at the axis here, you'll see that actually there's not much difference.
Here's what I get if I plot it from 0 to 5.
Yeah, the women did a little bit better.
But that's not a statistically-significant difference.
And by the way, when I plotted it last year for 6.0002, the blue was about that much higher than the pink.
Don't read much into either of them.
But the trick was here, I took the y-axis and ran it from 3.9 to 4.05.
I cleverly chose my baseline in such a way to make the difference look much bigger than it is.
Here I did the honest thing of put the baseline at 0 and run it to 5.
Because that's the range of grades at MIT.
And so when you look at a chart, it's important to keep in mind that you need to look at the axis labels and the scales.
Let's look at another chart, just in case you think I'm the only one who likes to play with graphics.
This is a chart from Fox News.
And they're arguing here.
It's the shocking statistics that there are 108.6 million people on welfare, and 101.7 with a full-time job.
And you can imagine the rhetoric that accompanies this chart.
This is actually correct.
It is true from the Census Bureau data.
Sort of.
But notice that I said you should read the labels on the axes.
There is no label here.
But you can bet that the y-intercept is not 0 on this.
Because you can see how small 101.7 looks like.
So it makes the difference look bigger than it is.
Now, that's not the only funny thing about it.
I said you should look at the labels on the x-axis.
Well, they've labeled them.
But what do these things mean?
Well, I looked it up, and I'll tell you what they actually mean.
People on welfare counts the number of people in a household in which at least one person is on welfare.
So if there is say, two parents, one is working and one is collecting welfare and there are four kids, that counts as six people on welfare.
People with a full-time job, is actually does not count households.
So in the same family, you would have six on the bar on the left, and one on the bar on the right.
Clearly giving a very different impression.
And so again, pictures can be good.
But if you don't dive deep into them, they really can fool you.
Now, before I should leave this slide, I should say that it's not the case that you can't believe anything you read on Fox News.
Because in fact, the Red Sox did beat the St. Louis Cardinals 4 to 2 that day.
So the moral here is to ask whether the things being compared are actually comparable.
Or you're really comparing apples and oranges, as they say.
OK, this is probably the most common statistical sin.
It's called GIGO.
And perhaps this picture can make you guess what the G's stand for GIGO is Garbage In, Garbage Out.
So here's a great, again, quote about it.
So Charles Babbage designed the first digital computer, the first actual computation engine.
He was unable to build it.
But hundreds of years after he died one was built according to his design, and it actually worked.
No electronics, really.
So he was a famous person.
And he was asked by Parliament about his machine, which he was asking them to fund.
Well, if you put wrong numbers into the machine, will the machine have right numbers come out the other end?
And of course, he was a very smart guy.
And he was totally baffled.
This question seems so stupid, he couldn't believe anyone would even ask it.
That it was just computation.
And the answers you get are based on the data you put in.
If you put in garbage, you get out garbage.
So here is an example from the 1840s.
They did a census in the 1840s.
And for those of you who are not familiar with American history, it was a very contentious time in the US.
The country was divided between states that had slavery and states that didn't.
And that was the dominant political issue of the day.
John Calhoun, who was Secretary of State and a leader in the Senate, was from South Carolina and probably the strongest proponent of slavery.
And he used the census data to say that slavery was actually good for the slaves.
Kind of an amazing thought.
Basically saying that this data claimed that freed slaves were more likely to be insane than enslaved slaves.
He was rebutted in the House by John Quincy Adams, who had formerly been President of the United States.
After he stopped being President, he ran for Congress.
From Braintree, Massachusetts.
Actually now called Quincy, the part he's from, after his family.
And he claimed that atrocious misrepresentations had been made on a subject of deep importance.
He was an abolitionist.
So you don't even have to look at that statistics to know who to believe.
Just look at these pictures.
Are you going to believe this nice gentleman from Braintree or this scary guy from South Carolina?
But setting looks aside, Calhoun eventually admitted that the census was indeed full of errors.
But he said that was fine.
Because there were so many of them that they would balance each other out and lead to the same conclusion, as if they were all correct.
So he didn't believe in garbage in, garbage out.
He said yeah, it is garbage.
But it'll all come out in the end OK. Well, now we know enough to ask the question.
This isn't totally brain dead, in that we've already looked at experiments and said we get experimental error.
And under some circumstances, you can manage the error.
The data isn't garbage.
It just has errors.
But it's true if the measurement errors are unbiased and independent of each other.
And almost identically distributed on either side of the mean, right?
That's why we spend so much time looking at the normal distribution, and why it's called Gaussian.
Because Gauss said, yes, I know I have errors in my astronomical measurements.
But I believe my errors are distributed in what we now call a Gaussian curve.
And therefore, I can still work with them and get an accurate estimate of the values.
Now, of course, that wasn't true here.
The errors were not random.
They were, in fact, quite systematic, designed to produce a certain thing.
And the last word was from another abolitionist who claimed it was the census that was insane.
All right, that's Garbage In, Garbage Out.
The moral here is that analysis of bad data is worse than no analysis at all, really.
Time and again we see people doing, actually often, correct statistical analysis of incorrect data and reaching conclusions.
And that's really risky.
So before one goes off and starts using statistical techniques of the sort we've been discussing, the first question you have to ask is, is the data itself worth analyzing?
And it often isn't.
Now, you could argue that this is a thing of the past, and no modern politician would make these kinds of mistakes.
I'm not going to insert a photo here.
But I leave it to you to think which politician's photo you might paste in this frame.
All right, onto another statistical sin.
This is a picture of a World War II fighter plane.
I don't know enough about planes to know what kind of plane it is.
Anyone here?
There must be an Aero student who will be able to tell me what plane this is.
Don't they teach you guys anything in Aero these days?
Shame on them.
All right.
Anyway, it's a plane.
That much I know.
And it has a propeller.
And that's all I can tell you about the airplane.
So this was a photo taken at a airfield in Britain.
And the Allies would send planes over Germany for bombing runs and fighters to protect the bombers.
And when they came back, the planes were often damaged.
And they would inspect the damage and say look, there's a lot of flak there.
The Germans shot flak at the planes.
And that would be a part of the plane that maybe we should reinforce in the future.
So when it gets hit by flak it survives it.
It does less damage.
So you can analyze where the Germans were hitting the planes, and you would add a little extra armor to that part of the plane.
What's the flaw in that?
Yeah?
They didn't look at the planes that actually got shot down
Yeah.
This is what's called, in the jargon, survivor bias.
S-U-R-V-I-V-O-R.
The planes they really should have been analyzing were the ones that got shot down.
But those were hard to analyze.
So they analyzed the ones they had and drew conclusions, and perhaps totally the wrong conclusion.
Maybe the conclusion they should have drawn is well, it's OK if you get hit here.
Let's reinforce the other places.
I don't know enough to know what the right answer was.
I do know that this was statistically the wrong thing to be thinking about doing.
And this is an issue we have whenever we do sampling.
All statistical techniques are based upon the assumption that by sampling a subset of the population we can infer things about the population as a whole.
Everything we've done this term has been based on that.
When we were fitting curves we were doing that.
When we were talking about the empirical rule and Monte Carlo Simulation, we were doing that, when we were building models, with machine learning, we were doing that.
And if random sampling is used, you can make meaningful mathematical statements about the relation of the sample to the entire population.
And that's why so much of what we did works.
And when we're doing simulations, that's really easy.
When we were choosing random values of the needles for trying to find pi, or random value if the roulette wheel spins.
We could be pretty sure our samples were, indeed, random.
In the field, it's not so easy.
Right?
Because some samples are much more convenient to acquire than others.
It's much easier to acquire a plane on the field in Britain than a plane on the ground in France.
Convenient sampling, as it's often called, is not usually random.
So you have survivor bias.
So I asked you to do course evaluations.
Well, there's survivor bias there.
The people who really hated this course have already dropped it.
And so we won't sample them.
That's good for me, at least.
But we see that.
We see that with grades.
The people who are really struggling, who were most likely to fail, have probably dropped the course too.
That's one of the reasons I don't think it's fair to say, we're going to have a curve.
And we're going to always fail this fraction, and give A's to this fraction.
Because by the end of the term, we have a lot of survivor bias.
The students who are left are, on average, better than the students who started the semester.
So you need to take that into account.
Another kind of non-representative sampling or convenience sampling is opinion polls, in that you have something there called non-response bias.
So I don't know about you, but I get phone calls asking my opinion about things.
Surveys about products, whatever.
I never answer.
I just hang up the phone.
I get a zillion emails.
Every time I stay in a hotel, I get an email asking me to rate the hotel.
When I fly I get e-mails from the airline.
I don't answer any of those surveys.
But some people do, presumably, or they wouldn't send them out.
But why should they think that the people who answer the survey are representative of all the people who stay in the hotel or all the people who fly on the plane?
They're not.
They're the kind of people who maybe have time to answer surveys.
And so you get a non-response bias.
And that tends to distort your results.
When samples are not random and independent, we can still run statistics on them.
We can compute means and standard deviations.
And that's fine.
But we can't draw conclusions using things like the Empirical Rule or the Central Limit Theorem, Standard Error.
Because the basic assumption underlying all of that is that the samples are random and independent.
This is one of the reasons why political polls are so unreliable.
They compute statistics using Standard Error, assuming that the samples are random and independent.
But they, for example, get them mostly by calling landlines.
And so they get a bias towards people who actually answer the phone on a landline.
How many of you have a land line where you live?
Not many, right?
Mostly you rely on your cell phones.
And so any survey that depends on landlines is going to leave a lot of the population out.
They'll get a lot of people of my vintage, not of your vintage.
And that gets you in trouble.
So the moral here is always understand how the data was collected, what the assumptions in the analysis were, and whether they're satisfied.
If these things are not true, you need to be very wary of the results.
All right, I think I'll stop here.
We'll finish up our panoply of statistical sins on Wednesday, in the first half.
Then we'll do a course wrap-up.
Then I'll wish you all godspeed and a good final.
See you Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, everybody.
Well, here we are at the last lecture.
We're going to finish talking about statistical sins and then do a little bit of a wrap-up.
Let's look at a hot topic-- global fiction-- or global warming, fact or fiction.
You've done a problem set related to temperatures in the US.
Here is a plot generally accepted of the change in temperatures on the planet between 1880 and 2014.
Now, if we look at this plot, we could see this commits one of the statistical sins I complained about on Monday, that look where it's starting the y-axis, way down here at 55.
And you remember, I told you to beware of charts for the y-axis doesn't start at 0.
So maybe the people who are trying to claim about global warming are just deceiving us with this trick of the axis.
So here's what happens when you put it at 0.
And as you can see-- or barely see-- this axis runs from 0 up to 110 as the average temperature.
And as you can see quite clearly, it's hardly changed at all.
So what's the deal here?
Well, which is a more accurate presentation of the facts?
Which conveys the accurate impression?
Let's look at another example, maybe a little less controversial than climate change-- fever and flu.
It's generally accepted that when you get the flu you might run a fever.
So here is someone who had the flu.
And this is plotting their fever from the beginning to its peak.
And it does appear, if we were to fit a curve to this, it would look pretty much like that.
On the other hand, if we assume that somebody's temperature could range between 0 and 200, we can see that, in fact, your temperature doesn't move at all when you get the flu.
So the moral is pretty clear, I think.
Even though on Monday I talked about being suspicious when people start the y-axis too far from 0, you should truncate it to eliminate totally preposterous values.
No living person has a temperature of 0 degrees Fahrenheit.
So again, don't truncate it just to make something look like it isn't, but don't expand it to deceive either.
Let's return to global warming.
This is a chart that was actually shown on the floor of the US Senate by a senator from Texas, who I shall not name.
And obviously, the argument here was that, well, sure global warming bounces up and down.
But if we go back, we can see here, the date is 19-- can we see it?
I can see it.
Maybe 1986, I think.
You can see that the argument here is, in fact, if you fit a trend line to this, as he's done, it hasn't changed at all.
And so even though we've had a lot of carbon emissions during this period, maybe global warming is not actually happening.
This is in contradiction to the trend I showed before.
Well, what's going on here?
This is a very common way that people use statistics poorly.
They confuse fluctuations with trends.
What we see in any theories of data-- time series, or other series-- you always have fluctuations.
And that's not to be confused with the trend.
And in particular, what you need to think about when you're looking at a phenomenon is choose an interval consistent with the thing that's being considered.
So we believe that climate change is something that happens over very long periods of time.
And it's a little bit silly to look at it on a short period of time.
Some of you may remember two years ago, we had a very cold winter here.
And there were people who were saying, well, that shows we don't have global warming.
Well, you can't really conclude anything about climate change looking at a year, or probably not even looking at 10 years or 20 years.
It's a very slow phenomenon.
On the other hand, if you're looking at the change in somebody's heart rate, seeing if they have a heart condition, you probably don't want to look at it over a 10-year period.
So you have to decide what you're doing and find an interval that lets you look at the trends rather than the fluctuations.
Any rate, maybe even if we're having global warming, at least the Arctic ice isn't melting, though apparently, I read in the paper this morning they found a huge crack in it.
So this was reported in the Financial Post on April 15, 2013.
You can read it yourself.
But the basic import of it is they took the period from April 14, 1989 to April 15, 2013 and said, look, it's not changing.
In fact, the amount of arctic ice is unchanged.
Well, what's the financial-- not the financial-- what's the statistical sin being committed here?
If we look at this data, this is an anomaly chart.
I think you saw one of these in one of the problems sets, where you fix something at 0 and then you show fluctuations relative to that.
So here, it's the Arctic ice relative to a point.
And what we see here is that if you go and choose the right date-- say this one in 1989-- and you come over here and you choose the right date in 2013-- say this one-- you can then draw a line and say, oh, look, it hasn't changed.
This is something people frequently do, is they take a whole set of data, and they find two points that are consistent with something they believe.
And they draw a line between those two points, fit a curve to those two points, and draw some conclusion.
This is what we call cherry picking, I guess from the notion that when you go to pick cherries you only want to pick the right ones, leave the others to ripen for a bit on the tree.
It's really bad.
And it's something that, unfortunately, the scientific literature is replete with, where people look at a lot of data, and they pick the points that match what they want to prove.
And so as you can see, while the trend is quite clear, you could prove almost anything you wanted by selecting two points very carefully.
I could also show that it's crashing much faster than people think it is by picking these two points.
If I wanted to argue that it's catastrophic, I'd pick those two points and say, look at that, it's disappearing at an incredible rate.
So you can lie in either direction with this data by careful cherry picking.
As a service to you, I know the holidays are coming and many of you have not bought presents for your parents, so here's a modest gift suggestion, that the family that shoots together something or other.
Well, all right, so we can ask, is this a good gift?
Well, probably.
We can look at this statistic.
It's not dangerous at least.
We see that 99.8% of the firearms in the US will not be used to commit a violent crime.
So guns apparently are not actually dangerous, or at least not in the hands of criminals.
Well, let's look at this.
How many privately owned firearms are there in the US?
And anyone want to guess who hasn't looked ahead?
Yeah
400 million
400 million.
340 million people and 400 million guns is the guess, more than one per person.
You certainly are the right order of magnitude.
I think it's about 300 million, but it's hard to count them.
Maybe this doesn't count water pistols.
So if you assume there are 300 million firearms and 0.2% of them are used to commit a violent crime in every year, we see that how many crimes is that?
600,000.
So in fact, it's not necessarily very meaningful to say that most of them are not used to commit a crime.
Well, let's look at another place where we look at a statistic.
Probably most of you don't even remember the scary swine flu epidemic.
This was a big headline.
And people got so scared of the swine flu they were doing things like closing schools to try limit the spread of the flu.
New York City closed some schools because of it, for example.
So is this a scary statistic?
Well, maybe, but here's an interesting statistic.
How many deaths per year are from the seasonal flu in the US-- the ones we try and prevent with a flu shot?
36,000.
So what we see is that, it doesn't make a lot of sense to panic over 159 in the light of this number.
So the point here for both this and the issue about the firearms is that context matters.
Yeah, I love this cartoon.
A number without context is just a number.
And numbers by themselves don't mean anything.
So to say that there were 159 deaths from the swine flu is not very meaningful without some context.
To say that only 0.2% of firearms are used to commit a violent crime is not very meaningful without context.
Whenever you're presenting a statistic, reading about a statistic, and you just see a number that seems comforting or terrifying, try and put some context around it.
So a related thing is relative to what?
Suppose I told you that skipping lectures increases your probability of failing this course by 50%.
Well, you would all feel great, because you're here.
And you would be laughing at your friends who are not here, because figuring that will leave much better grades for you.
What does this mean, though?
Well, if I told you that it changed the probability of failing from a half to 0.75, you would be very tempted to come to lectures.
On the other hand, if I told you that it changed the probability from 0.005 to 0.0075, you might say, the heck with it, I'd rather go to the gym.
Again this, is an issue.
And this is something that we see all the time when people talk about percentage change.
This is particularly prominent in the pharmaceutical field.
You will read a headline saying that drug x for arthritis increases the probability of a heart attack by 1% or 5%.
Well, what does that mean?
If the probability was already very low, increasing it by 5%, it's still very low.
And maybe it's worth it not to be in pain from arthritis.
So talking in percentages is, again, one of these issues of it doesn't make sense without the context.
In order to know what this means, I need to know what regime I'm in here in order to make a intelligent decisions about whether to attend lecture or not.
It goes without saying, you have all made the right decision.
So beware of percentage change when you don't know the denominator.
You get a percentage by dividing by something.
And if you don't know what you're dividing by, then the percentage is itself a meaningless number.
While we're sort of talking about medical things, let's look at cancer clusters to illustrate another statistical question.
So this is a definition of a cancer cluster by the CDC-- "a greater-than-expected number of cancer cases that occurs in a group of people in a geographic area over a period of time."
And the key part of this definition is greater-than-expected.
About 1,000 cancer clusters per year are reported in the US, mostly to the Centers for Disease Control, but in general to other health agencies.
Upon analysis, almost none of them pass this test.
So the vast majority, some years all of them, are deemed actually not to be cancer clusters.
So I don't know if-- has anyone here seen the movie Erin Brockovich?
Subsequent analysis showed that was actually not a cancer cluster.
It's a good movie, but turns out statistically wrong.
This, by the way, is not a cancer cluster.
This is a constellation.
So let's look at a hypothetical example.
By the way, the other movie about cancer clusters was the one set in Massachusetts.
What was the name?
A Civil Action.
Anyone see that?
No.
That was a cancer cluster.
Massachusetts is about 10,000 square miles.
And there are about 36,000 cancer cases per year reported in Massachusetts.
Those two numbers are accurate.
And the rest of this is pure fiction.
So let's assume that we had some ambitious attorney who partitioned the state into 1,000 regions of 10 square miles each and looked at the distribution of cancer cases in these regions trying to find cancer clusters that he or she could file a lawsuit about.
Well, you can do some arithmetic.
And if there are 36,000 new cancer cases a year and we have 1,000 regions, that should say that we should get about 36 cancer cases per year and per region.
Well, when the attorney look at the data, this mythical attorney, he discovered that region number 111 had 143 new cancer cases over a three-year period.
He compared that to 3 times 36 and said, wow, that's 32% more than expected.
I've got a lawsuit.
So he went to tell all these people-- they lived in a cancer cluster.
And the question is, should they be worried?
Well, another way to look at the question is, how likely is it that it was just bad luck?
That's the question we've always ask when we do statistical analysis-- is this result meaningful, or is it just random variation that you would expect to see?
So I wrote some code to simulate it to see what happens-- so number of cases, 36,000, number of years, 3.
So all of this is just the numbers I had on the slide.
We'll do a simulation.
We'll take 100 trials.
And then what I'm going to do is for t in the range number of trials, the locations, the regions, if you will, I'll initialize each to 0, 1,000 of them, in this case.
And then for i in the range number of years times number of cases per year, so this will be 3 times 36,000.
At random, I will assign the case to one of these regions.
This is the random.
Nothing to do with cancer clusters, just at random, this case gets assigned to one of the 1,000 regions.
And then I'm going to check if region number 111 had greater than or equal to 143, the number of cases we assumed it had.
If so, we'll increment the variable num greater by 1, saying, in this trial of 100, indeed, it had that many.
And then we'll see how often that happens.
That will tell us how improbable it is that region 111 actually had that many cases.
And then we'll print it.
Does that makes sense to everyone, that here I am doing my simulation to see whether or not how probable is it that 111 would have had this many cases?
Any questions?
Let's run it.
So here's the code we just looked at.
Takes just a second.
That's why I did only 100 trials instead of 1,000.
I know the suspense is killing you.
It's killing me.
I don't know why it's taking so long.
We'll finish.
I wish I had the Jeopardy music or something to play while we waited for this.
Anna, can you home some music or something to keep people amused?
She will not.
Wow.
So here it is.
The estimated probability of region 111 having at least 1 case-- at least 143 cases-- easier to read if I spread this out is 0.01.
So it seems, in fact, that it's pretty surprising-- unlikely to have happened at random.
Do you buy it?
Or is there a flaw here?
Getting back to this whole question.
Yes
I think it's flawed because first off you have to look at the population.
That is more important
You have to look at what?
Population as opposed to like the number of areas, because when you get past the Boston area, you'd expect a--
Let's assume that, in fact, instead of by square miles-- let's assume the populations were balanced
Then I also think it's flawed because I don't think the importance of block 111 having 143 is important.
I think the importance is just one area having a higher--
Exactly right.
Exactly right.
I'm sorry, I forgot my candy bag today.
Just means there'll be more candy for the final.
What we have here is a variant of cherry picking.
What I have done in this simulation is I've looked at 1,000 different regions.
What the attorney did is, not in a simulation, is he looked at 1,000 different regions, found the one with the most cancer cases, and said, aha, there are too many here.
And that's not what I did in my simulation.
My simulation didn't ask the question, how likely is it that there is at least one region with that many cases.
But it asked the question, how likely is it that this specific region has that many cases.
Now, if the attorney had reason in advance to be suspicious of region 111, then maybe it would have been OK to just go check that.
But having looked at 1,000 and then cherry pick the best is not right.
So this is a simulation that does the right thing.
I've left out the initialization.
But what you can see I'm doing here is I'm looking at the probability of there being any region that has at least 143 cases.
What this is called in the technical literature, what the attorney did is multiple hypothesis checking.
So rather than having a single hypothesis, that region 111 is bad, he checked 1,000 different hypotheses, and then chose the one that met what he wanted.
Now, there are good statistical techniques that exist for dealing with multiple hypotheses, things like the Bonferroni correction.
I love to say that name.
But you have to worry about it.
And in fact, if we go back to the code and comment out this one and run this one, we'll see we get a very different answer.
The answer we get is-- let's see.
Oh, I see.
All right, let me just comment this out.
Yeah, this should work, right?
Well, maybe you don't want to wait for it.
But the answer you'll get is that it's actually very probable.
My recollection is it's a 0.6 probability that at least one region has that many cases.
And that's really what's going on with this whole business of people reporting cancer clusters.
It's just by accident, by pure randomness, some region has more than its share.
This particular form of cherry picking also goes by the name of the Texas sharpshooter fallacy.
I don't know why people pick on Texas for this.
But they seem to.
But the notion is, you're driving down a road in Texas and you see a barn with a bunch of bullet holes in the wall right in the middle of a target.
But what actually happened was you had a barn.
The farmer just shot some things at random at the barn, then got out his paint brush and painted a target right around where they happened to land.
And that's what happens when you cherry pick hypotheses.
What's the bottom line of all these statistical fallacies?
When drawing inferences from data, skepticism is merited.
There are, unfortunately, more ways to go wrong than to go right.
And you'll read the literature that tells you that in the scientific literature more than half of the papers were later shown to be wrong.
You do need to remember that skepticism and denial are different.
It's good to be skeptical.
And I love Ambrose Bierce's description of the difference here.
If you had never read Ambrose Bierce, he's well worth reading.
He wrote something called The Devil's Dictionary, among other things, in which he has his own definition of a lot of words.
And he went by the nickname Bitter Bierce.
And if you read The Devil's Dictionary, you'll see why.
But this, I think, has a lot of wisdom in it.
Let's, in the remaining few minutes, wrap up the course.
So what did we cover in 6.0002?
A lot of things.
If you look at the technical, things were three major units-- optimization problems, stochastic thinking, and modeling aspects of the world.
But there was a big subtext amongst all of it, which was this.
There was a reason our problem sets were not pencil and paper probability problems, but all coding.
And that's because we really want, as an important part of the course, is to make you a better programmer.
We introduced a few extra features of Python.
But more importantly, we emphasized the use of libraries, because in the real world when you're trying to build things, you rarely start from scratch.
And if you do start from scratch, you're probably making a mistake.
And so we wanted to get you used to the idea of finding and using libraries.
So we looked at plotting libraries and machine learning libraries and numeric libraries.
And hopefully, you got a lot of practice in that you're a way better programmer than you were six weeks ago.
A little more detailed-- the optimization problems, the probably most important takeaway is that many important problems can be formulated in terms of an objective function that you either maximize or minimize and some set of constraints.
Once you've done that, there are lots of toolboxes, lots of libraries that you can use to solve the problem.
You wrote some optimization code yourself.
But most of the time, we don't solve them ourselves.
We just call a built-in function that does it.
So the hard part is not writing the code, but doing the formulation.
We talked about different algorithms-- greedy algorithms, very often useful, but often don't find the optimal solution.
So for example, we looked at k-means clustering.
It was a very efficient way to find clusters.
But it did not necessarily find the optimal set of clusters.
We then observed that many optimization problems are inherently exponential.
But even so, dynamic programming often works and gives us a really fast solution.
And the notion here is this is not an approximate solution.
It's not like using a greedy algorithm.
It gives you an exact solution and in many circumstances gives it to you quickly.
And the other thing I want you to take away is, outside the context of dynamic programming, memoization is a generally useful technique.
What we've done there is we've traded time for space.
We compute something, we save it, and when we need it, we look it up.
And that's a very common programming technique.
And we looked at a lot of different examples of optimization-- knapsack problems, several graph problems, curve fitting, clustering, logistic regression.
Those are all optimization problems, can all be formulated as optimization problems.
So it's very powerful and fits lots of needs.
The next unit-- and, of course, I'm speaking as if these things were discrete in time, but they're not.
We talked about optimization at the beginning.
And I talk to an optimization last week.
So these things were sort of spread out over the term.
We talked about stochastic thinking.
And the basic notion here is the world is nondeterministic, or at least predictably nondeterministic.
And therefore, we need to think about things in terms of probabilities most of the time, or frequently.
And randomness is a powerful tool for building computations that model the world.
If you think the world is stochastic, then you need to have ways to write programs that are stochastic, if you're trying to model the world itself.
The other point we made is that random computations-- randomness is a computational technique-- is useful even for problems that don't appear to involve any randomness.
So we used it to find the value of pi.
We showed you can use it to do integration.
There's nothing random about the value of the integral of a function.
Yet, the easiest way to solve it in a program is to use randomness.
So randomness is a very powerful tool.
And there's this whole area of random algorithms-- research area and practical area that's used to solve non-probabilistic problems.
Modeling the world-- well, we just talked about part of it.
Models are always inaccurate.
They're providing some abstraction of reality.
We looked at deterministic models-- the graph theory models.
There was nothing nondeterministic about the graphs we looked at.
And then we spent more time on statistical models.
We looked at simulation models.
In particular, spent quite a bit of time on the Monte Carlo simulation.
We looked at models based on sampling.
And there-- and also when we talked about simulation-- I really hope I emphasized enough the notion that we need to be able to characterize how believable the results are.
It's not good enough to just run a program and say, oh, it has an answer.
You need to know whether to believe the answer.
And the point we made is it's not a binary question.
It's not yes, it's right, no, it's wrong.
Typically, what we do is we have some statement about confidence intervals and confidence levels.
We used two variables to describe how believable the answer is.
And that's an important thing.
And then we looked at tools we use for doing that.
We looked at the central limit theorem.
We looked at the empirical rule.
We talked about different distributions.
And especially, we spent a fair amount of time on the normal or Gaussian distribution.
And then finally, we looked at statistical models based upon machine learning.
We looked at unsupervised learning, basically just clustering, looked at two algorithms-- hierarchical and k-means.
And we looked at supervised learning.
And there, we essentially focused mostly on classification.
And we looked at two ways of doing that-- k-nearest neighbors and logistic regression.
Finally, we talked about presentation of data-- how to build plots, utility of plots, and recently, over the last two lectures, good and bad practices in presenting results about data.
So my summary is, I hope that you think you've come a long way, particularly those of you-- how many of you were here in September when we started 6.0001?
All right, most of you.
Yeah, this, by the way, was a very popular ad for a long time, saying that, finally women are allowed to smoke, isn't this great.
And Virginia Slims sponsored tennis-- the women's tennis tour to show how good it was that women were now able to smoke.
But anyway, I know not everyone in this class is a woman.
So just for the men in the room, you too could have come a long way.
I hope you think that, if you look back at how you struggled in those early problems sets, I hope you really feel that you've learned a lot about how to build programs.
And if you spend enough time in front of a terminal, this is what you get to look like.
What might be next?
I should start by saying, this is a hard course.
We know that many of you worked hard.
And the staff and I really do appreciate it.
You know your return on investment.
I'd like you to remember that you can now write programs to do useful things.
So if you're doing a UROP, you're sitting in a lab, and you get a bunch of data from some experiments, don't just stare at it.
Sit down and write some code to plot it to do something useful with it.
Don't be afraid to write programs to help you out.
There are some courses that I think you're now well-prepared to take.
I've listed the ones I know best-- the courses in course 6.
6.009 is a sort of introduction to computer science.
I think many of you will find that too easy after taking this course.
But maybe, that's not a downside.
6.005 is a software engineering course, where they'll switch programming languages on you.
You get to program in Java.
6.006 is a algorithms course in Python and I think actually quite interesting.
And students seem to like it a lot, and they learn about algorithms and implementing them.
And 6.034 is an introduction to artificial intelligence also in Python.
And I should have listed 6.036, another introduction to machine learning in Python.
You should go look for an interesting UROP.
A lot of students come out of this course and go do UROPs, where they use what they've learned in this course.
And many of them really have a very positive experience.
So if you were worried that you're not ready for a UROP, you probably are-- a UROP using what's been done here.
You can minor in computer science.
This is now available for the first time this year.
But really, if you have time, you should major in computer science, because it is really the best major on campus-- not even close, as somebody I know would say.
Finally, sometimes people ask me where I think computing is headed.
And I'll quote one of my favorite baseball players.
"It's tough to make predictions, especially about the future."
And instead of my predictions, let me show you the predictions of some famous people.
So Thomas Watson, who was the chairman of IBM-- a company you've probably heard of-- and he said, "I think there is a world market for maybe five computers."
This was in response to, should they become a computer company, which they were not at the time.
He was off by a little bit.
A few years later, there was an article in Popular Mechanics, which was saying, computers are amazing.
They're going to change enormously.
Someday, they may be no more than 1 and 1/2 tons.
You might get a computer that's no more than 3,000 pounds-- someday.
So we're still waiting for that, I guess.
I like this one.
This is, having written a book recently, the editor in charge of books for Prentice Hall.
"I traveled the length and breadth of this country and talked with the best people.
And I can assure you that data processing is a fad that won't last out the year."
MIT had that attitude for a while.
For about 35 years, computer science was in a building off campus, because they weren't sure we were here to stay.
Maybe that's not why, but that's why I interpret it.
Ken Olsen, an MIT graduate-- I should say, a course 6 graduate-- was the founder and president and chair of Digital Equipment Corporation, which in 1977 was the second largest computer manufacturer in the world based in Maynard, Massachusetts.
None of you have ever heard of it.
They disappeared.
And this is in part why, because Ken said, "there's no reason anyone would want a computer in their home," and totally missed that part of computation.
Finally, since this is the end of some famous last words, Douglas Fairbanks, Sr., a famous actor-- this is true-- the last thing he said before he died was, "never felt better."
Amazing.
This was from the movie The Mark of Zorro.
Scientists are better.
Luther Burbank, his last words were, I don't feel so good.
And well, I guess not.
[LAUGHTER] And this is the last one.
John Sedgwick was a Union general in the Civil War.
This is a true story.
He was riding behind the lines and trying to rally his men to not hide behind the stone walls but to stand up and shoot at the enemy.
And he said, "they couldn't hit an elephant at this distance."
Moments later, he was shot in the face and died.
[LAUGHTER] And I thought this was an apocryphal story.
But in fact, there's a plaque at the battlefield where this happened, documenting this story.
And apparently, it's quite true.
So with that, I'll say my last words for the chorus, which is I appreciate all your coming.
And I guess you were the survivors.
So thank you for being here.
[APPLAUSE]
So, one question to ask ourselves is, what is engineering?
How do we define, what is engineering?
Well, the definition I like to use is one put forth by Steve Senturia, one of our professors who is now retired.
He defined engineering to be the purposeful use of science.
All right, so what is 6.002 about?
So, 6.002 is a first course in engineering.
And I like to view 6.002 as the gainful employment of Maxwell's equations.
Many of you have seen Maxwell's equations before.
Most of you should have.
And they are hard stuff.
6.002 is all about teaching you how to simplify our lives, make things simple.
So, if you can gainfully employ Maxwell's equations, gainfully employ the facts of nature to build very interesting systems.
So let me show you how the transition is made.
So, there's a world around us, nature, so we made some observations in nature.
We make measurements, and we can write down large tables of measurements.
So, for example, we can take objects and measure the voltage across them, and look at the resulting current through the elements.
So, we may end up getting a bunch of values such as [CHALKBOARD].
So, we start out life with making measurements on what exists.
And we build a bunch of tables.
Now, we could directly take these tables, and based on observations of these tables, we could go ahead and build very interesting engineering systems that help us out in day-to-day lives.
But that's incredibly hard.
Imagine having to resort to a set of tables to do any kind of useful work.
So what we do as engineers, we first layer a level of abstraction.
We look at all the data, and somehow layer abstraction such that we can simplify or much more succinctly put in a simple equation or a simple statement what these numbers are telling us.
OK, so for example, our physics laws, so laws of physics for example are simply abstractions, the laws of abstractions.
So, these sets of numbers can be codified by Ohm's law, for example, V is equal to RI, the voltage current, relates to the resistance of the object.
So, V is equal to RI is a law that succinctly describes a set of experiments, and replaces a large number of tables with a very simple statement.
You could call this the law, or you could call it an abstraction.
OK so you see laws of physics, call them abstractions of physics if you like.
Similarly, there are Maxwell's equations and so on and so forth.
So, this is what is.
This is what's out there.
OK, and a law as an abstraction describe the properties of nature, as we see it, in some succinct form.
Now, if you want to go and build useful things, we could take these abstractions, take Maxwell's equations, and go and build things.
But it's hard.
It's really, really hard.
And what you learn in, at MIT is this place is all about simplifying things.
Take complicated things, build layers of abstraction, and simplify things so that we can build useful systems.
Even in 6.002 we start life by making a huge leap from Maxwell's equations to a couple of very, very simple laws.
OK, I'm going to show you that leap that we will make today.
So, the first abstraction that we layer is called the lump circuit abstraction.
OK, in the lump circuit abstraction, what we do is we make a set of simplifications that allows us to view a set of objects as discrete or lumped elements.
So, we may, I will define voltage sources.
We'll define resistors.
We'll define capacitors, and so on.
OK, and I'm going to make the jump, and show you how we make the jump in a few minutes.
So, on that sort of abstraction, we then layer yet another abstract layer.
And let me call that the amplifier abstraction.
OK, remember, here we are absolutely down and dirty.
We are setting the probes, measuring objects, and building huge tables.
We abstracted things into simple laws, and life got a little better.
OK, I'm going to show you can abstract things further out and build discrete objects, and, you could build even more interesting components called amplifiers and begin playing around with amplifiers.
OK, so when you are using amplifiers, you don't really have to worry about the details of Maxwell's equations.
OK, I'll give you some very simple abstract rules of behavior for an amplifier, and you can go build very interesting systems without really, really knowing how Maxwell's equations applies to that because you will be working at this abstract layer.
However, since you're engineers, and you are good at building such systems, it's very important for you to understand how we make this leap from the laws of physics into some of our very primitive engineering abstractions.
So, once we make the amplified abstraction in 6.002, by the way, 6.002 starts here.
We start from the laws of physics and then proceed all the way out.
So, once we talk about amplifiers we will take two pads.
On the amplifier, you will build the next abstraction called the digital abstraction.
OK, and with the digital abstraction, we will build new elements such as inverters and combinational gates, OK?
So, notice we are building bigger, and bigger things, which have more and more complicated behavior inside them, but which are very simple to describe, right?
So, following the digital abstraction, we will superimpose the combinational logic abstraction on top of that, and define functional blocks that look like this: some inputs, some function, some outputs.
The next abstraction on top of that will be the clock digital abstraction, where we will have some notion of time introduced into the system.
There will be a clock, and this will be some function.
And there will be a clock that introduces time into the sort of logic values that functions operate upon.
Following that, the next level of abstraction that we build is called instruction set abstraction.
OK, now you begin to see things that consumers get to look at.
Can someone give me an example of, or name an instruction set, or instruction set abstraction?
Bingo.
So, x86 is one set of abstractions.
And in fact, in many universities, education could well start just by saying, OK, here's an abstraction.
These are the x86 instructions, OK?
Some MIT gurus have designed this awesome little microprocessor, OK?
So you just worry about, you take this abstraction layer here, the assembly instructions, and you go and build systems on top of that.
OK, so this is an abstraction layer called the x86 layer.
There are other abstraction layers.
In 6.004, you will learn about, I believe, the alpha or the beta, OK, and various other abstractions at this point.
So, 6.002 kind of goes until here.
6.002 takes me from the world of physics all the way to the world of interesting analog and digital systems.
OK, 004, the course on computation structures, will show you how to build computers all the way from simple digital objects all the way to big systems.
Following that, you learn about language abstractions, Java, C, and other languages, and that's in 6.002.
And there are several other courses that will cover that.
Following this, you learn about software system abstractions, and software systems, you will learn about operating systems.
Any example of an operating system abstraction that people know out there?
What's that?
Linux.
What else?
I'm just wondering how long I'll have to go before I hear what I want to hear.
[LAUGHTER] OK, so we have a bunch of software systems.
So, if we have a bunch of software systems, these are nothing but abstractions.
Linux simply implies a set of system calls that the programs must adhere to.
Windows is another set of system calls.
That's it.
And see how much money they made out of it?
OK, it's all about abstraction layers, that all start from nature.
All right?
Build abstraction upon abstraction upon abstraction upon abstraction, and someone out here are lots of dollars.
OK, so based on these abstractions, we can then build useful things for human beings.
We can build very useful things, video games, so we can send space shuttles up, and a whole bunch of other systems.
But it's based on these abstraction layers.
What's unique about education at MIT?
What's unique about 6.002 and EECS?
Is to my knowledge, there are not many other places in the world where you will get an education in everything going all the way from nature to how to build very complicated analog and digital systems.
OK, we will show you layer upon layer upon layer upon layer, peel away the onion until you are down to raw nature, OK, through Maxwell's equations.
So, 6.002, 004, this is 033, OK, 6.170, and so on.
OK, the whole EECS is about building abstraction layers, one on top of the other.
So that's one path.
There's the analog path.
The analog path would take an amplifier, and build an abstraction layer called the op-amp.
See how similar they all look?
You know the amplifier, the inverter of the digital world, and the operational amplifier in the analog world, just different ways of looking at the same devices.
So, to build an analog system, to build an operational amplifier, and then, here we go end up building a whole bunch of different interesting analog system components.
OK, and these components might look like oscillators.
They might look like filters.
OK, they look like power supplies, a whole bunch of very interesting abstract components, which pulled together can then give you the next set of systems.
And these systems might be toasters, or say for example other analog systems like the various control systems for various power plants and so on and so forth, and ultimately, fun and dollars.
OK, so 6.002 is about going from physics all the way to this point.
We will build interesting analog systems, and take you up to interesting digital system components, from which 004 will take you all the way to building computer architectures.
So that, in a nutshell, kind of gives you a feel for the space of EECS.
OK, this chart here is almost a vignette of what EECS at MIT is all about.
And this is the world according to Agarwal, because he's teaching 002.
OK, so this is 6.002, and the rest of EECS is somewhere out there.
OK, so I'm going to do now is throughout this course; I want you to think about which part in this vignette we are in.
So, right now, I'm going to start here and take you here.
OK, and as you get closer and closer, things get simpler, and simpler, and simpler.
Still, the final abstractions are pedal, brake, steering wheel.
I mean, that's the abstraction to play a game, right, four or five very simple interfaces, and that's all you need to know.
And everybody in the world can play stuff.
So remember, this stuff is complicated.
This stuff is very, very simple.
OK, and the more we build abstractions and come to this side, things get simpler and simpler.
So, a large part of what I'll cover today is make the biggest simplification.
The biggest simplification we will make his go from Maxwell's equation to some very, very simple algebraic rules.
OK, I did Maxwell's equations myself.
And I tell you, they were very interesting stuff but complicated.
I can't imagine building efficient systems using Maxwell's equations.
So, let's take an example, OK?
So, let's say I have a battery.
Just switch to page three of your course notes.
And let's say I connect that to a bulb.
OK, and this is a wire.
And, the battery supplies some voltage, V, and I ask you a simple question.
What is the current through the bulb?
OK, so here is something that I can build using objects.
I can pick a round from stores and so on.
And I can collect them up in this way, and ask the question, what is the current, I?
Now, if all you've done is learn about Maxwell's equations, you can roll up your sleeves and say, ah-ha!
The first step is to write down all of Maxwell's equations, and you can say, del cross E is minus del and go on, and on, and on, OK, and write out all of Maxwell's equations and say, now how do I get from there to here?
OK, it's very good.
You can do it.
OK, you can do it, but it's very complicated.
OK, so instead, what you're going to do is take the easy way.
So, what I want to remind you is that this course is actually very easy.
OK remember, we're going to be building abstraction upon abstraction to make your lives easier.
If you think your lives are getting more complicated, then you are not using intuition enough.
OK, just remember the big I word.
It's all about making things simple.
OK, so let me give you an analogy.
So, suppose you have an object.
OK, and I apply a force to the object.
It's an analogy, OK to get some insight into how to do this.
So, I say here's an object.
I apply a force, and I ask you the question.
What is the acceleration of the object when I apply a force, F?
So, how would you do it?
OK, and eighth, or ninth, or tenth grader can do this.
OK, they would ask me, what's the mass of the object?
OK, I ask you what is the acceleration?
You would turn around and ask me, what is the mass of the object?
I tell you, the mass of the object is M. And then you say, oh sure, A is F divided by M, done.
It's as simple as that.
OK, I could have gone into all kinds of differential equations and so on to figure that out, but you asked me for the mass.
And you gave me the answer, A is F divided by M. So, you ignored a bunch of things.
You ignored the shape of the object.
You ignored its color.
You ignored its temperature.
OK, and you ignored the soft or hard or whatever.
OK, you ignored a whole bunch of things.
You were focused on one thing.
OK, you're focused on its mass.
And, it turns out that the process really was developed from a set of simplifications.
That is called, does anybody remember this?
Point mass simplification.
OK, so, in physics, you've done this before.
OK, you've simplified your lives by viewing objects as having a mass at a point, and force is acting at that point.
OK, M is that property of the object that is of interest to you.
This process is called, in physics, point mass discretization.
OK, now using an analogy, and I'm going to show you a similar simple process to do the problem with the light bulb.
OK, so take my light bulb again, And I focus on the filament of the light bulb.
OK, all I care about is the current flowing through the light bulb.
OK, I don't care about whether the filament is twisted, whether it's hot.
I don't care about its shape.
I don't care about its color.
All I care about is the current.
OK, so to do that, what we can do here at a very high level is since we just need the current and don't care about a bunch of other properties, we will simply replace the bulb with a discrete object called a resistor.
So the discrete object is a resistor, much like the point mass simplification that we did earlier that replaced the bulb filament with a object called a resistor, a discrete object called a resistor.
Or a lump object called resister, and put a value next to it just like the mass for the object, a resistance value, R. OK, now what I can do is in the same manner, replace the battery with an object called a battery object, and connect that here, the voltage, V, applied to it.
V falls across the resistor, and I get my I simply from Ohm's law as we divide by R. So, notice here, to replace this complicated bulb, this really twisty, weird old thing with this discreet thing called a resistor, and its only property of interest was its resistance value, R, direct analogy to what we did there.
So, since R represents the only property of interest, we can simply ignore all the other things.
So, notice here, we've done things the simple way.
And remember, in EE, in the electrical engineering, we do things the simple way.
OK, we could go the hard route and do Maxwell's equations, and get PhD's in physics, and so on.
But out here, we are looking to do useful, interesting systems in the simplest way that we can.
OK, we do things a simple way.
All right, so we just did this, and boom, I found out what the current was.
Now, I cheated a little bit.
I've cheated a little bit.
R is a lumped abstraction for the bulb.
So, you look at this resistor here.
That is simply a placeholder.
It's a stand-in for this complicated thing called a bulb.
It's a discreet object.
It's a lumped object, and represents the bulb.
Now, so most of 6.002 will take off from here, OK, and that's it.
To very simple stuff, like V is equal to IR, it's a simple high school algebra to take off in that direction.
But before we go there, it's important to understand, why was it that we were able to make the simplification?
OK, we did something else.
Something's going on under the covers here.
On the one hand, I say let's use Maxwell's, and then I jump out and say, hey, we can just use this simple thing.
I did something that allowed me to go from here to here.
And you need to understand why I did that and how I did that.
Understand it once, and then you won't have to need that information again.
You just need to understand it.
So, let's take a closer look at the bulb filament, and look at what we really did.
So, here's my filament, A, and let's say that the surface area here, I label that SA, and the one down here SB, my voltage, V, applied there, and this is what I call my black box that I've replaced with a resistor.
Notice that, in order for this to work, V and I need to be defined.
So I needs to be defined, and V needs to be defined.
OK, if I give you a random object, and I don't tell you anything else about the object, it's not clear I can do that.
OK, if it's a much more general situation, I have to write down Maxwell's equations, and this is what I would write down.
Write down J dot dS as a function of the coordinate here integrated over the area minus, OK, I would have to start from there from one of Maxwell's equations.
All right, notice that this becomes IA, and this becomes IB in our simplification.
But, if I don't tell you anything else, you have to start from here.
You will have some varying current here by point.
You might have some other current coming out here because I may have some charge buildup happening inside.
If charge is building up inside the filament; then I would have to put del q by del t out here, right, the current in minus the current out must equal charge buildup.
Whoa, where is this and where is that?
So this is reality.
This is really, really what I have to do.
But how did I get there?
How did I get there?
The key answer is, as engineers, when in doubt we simplify.
Remember, we are engineers.
Our goal in life is to build interesting systems.
OK and some are motivated by money.
OK, so our goal is to build interesting systems and do good to humanity.
So, as long as we can build a good light bulb, we are happy.
So what we can do is we can say, look, all I care about is building interesting systems.
So I can say, hey, this stuff is too hard.
Let's make the assumption that all the systems that we will consider will have this thing be zero.
OK, in other words, if I take a complete object, if I take an element like a resistor or a capacitor, the box around the entire element, OK, and I want to just deal with those systems in which this thing is zero.
You can come and beat me up and say, but why?
Why not?
Why am I doing this?
And I am saying the world is arbitrary.
I'm an engineer; I want to build good systems.
By making this simplification, I eliminate this squiggle thing, and so on.
I don't want to deal with it.
I want to make my life simple.
So this is gone to zero because, why?
Because I have said that in the future I will only deal with those elements for which this is true.
I'm going to discipline myself.
I'm going to discipline myself to only deal with those systems.
OK, Maxwell is turning around and, you know, mad at me and all that stuff, but tough.
So this, what I've said about making a simplification here, and this is one of the simplifications I'm making.
And I give a name to the simplification.
And that's called the lumped matter discipline.
OK, so I'm saying I will only deal with elements for which if I put a black box around it, this is going to be true.
And if this is going to be true, then notice, there is no charge buildup.
Current in must equal current out.
Ah-ha!
So this becomes IA.
This becomes IB.
Yes.
OK, I can now deal with IA's and IB's.
And IB and IA are equal because this is zero.
Notice that there is a whole bunch of depth here in the jump from here to here.
As MIT graduates, you really, really need to understand why it is that we made that jump, and then go and use that, and do cool things.
All right, this allows us to define I.
We have a unique I associated with an element for the current through the element.
We still have to worry about B, and I won't go through that in detail.
The course notes have some discussion of that and so does the textbook.
So V, AB is defined when del phi B, the rate of change of magnetic flux is zero.
So, if I take the element and I take any region outside the element, this must be true.
And you say, why should that be true?
That's not true in general.
Absolutely.
It's not true in general.
But I, because I choose to, I going to deal with only those elements.
I will discipline myself.
But these are only those elements for which this is true, and this is true.
I'm going to limit my world.
I'm going to create a play field for myself.
You want to play; follow my rules.
OK, and that's called the lumped matter discipline.
So once you say that I'm going to adhere to the lump matter discipline, and this is true inside your elements.
This is true outside the elements.
You can define VA and VB, and good things happen to you.
OK, let me show you a few examples of lumped elements.
But remember, a large part of what we're doing is based on these two assumptions.
And to just go through the background on that, I would encourage you to go to chapter 1 of your course notes and read through just as how this came about, that comes about.
So, by doing that by adhering to a lumped matter discipline, we can now lump objects.
We could lump a bulb into a resistor.
OK, so to be clear, a certain number of lumped objects, and now, the universe is going to be comprised into lumped objects.
OK, so before this, when he went home, we talked about eggs, and omelets, and light bulbs, and switches, but once you come to MIT, and after you've taken 6.002, you begin talking about lumped elements, you know, resistors, voltage sources, capacitors, little inky-dinky objects that follow the lumped matter discipline.
OK, they stick to very simple rules, and the math that you have to do to analyze them is incredibly simple.
What could be simpler than V is equal to IR?
So, let me give you an example of interesting lumped elements, and then show you a couple of really nasty lumped elements.
OK. OK, so what you see out here, so we characterize lumped elements by the VI characteristics.
OK, you apply voltage, measure the current.
OK, so what I can do is I can plot I here, and V here, and see what it looks like.
OK, I can characterize elements by their VI relationship.
And there are a bunch of elements that I can create based on the VI relationship.
So let me show you a few examples.
So for the resistor, since V is directly proportional to I, and R is a constant, I get a straight line.
That's the I axis, the V axis, and this is the resistor.
What I actually have is a variable resistor, so I'm going to change the resistance value, R, and the curve will also change slope.
OK, I changed the value of R because it's a variable resistor, and the changes slope because my R is different.
OK, next, let me go to a fixed resistor, and this guy here on the screen to your left is a fixed resistor.
And you see that its IV characteristic is a line of a given slope, 1 by R, and that's it.
I can't change it.
Number three, I have another lumped element called a Zener diode that you will see in the fourth week of this class, and the characteristics for the Zener diode look like this: IV.
If my voltage goes across the Zener diode goes up slightly, the current shoots up.
But if the voltage becomes negative I don't have any current flowing into it until the voltage passes on the threshold, at which point my current begins to build up.
OK, so I can increase the voltage a little bit, and it can show that the current starts building up again.
So that's another interesting lumped element called a Zener diode.
Let's switch to the next one called a diode.
So a diode looks like this: IV.
As the voltage across the diode becomes positive, around .6 volts, or thereabout, the current begins to shoot up.
But when the voltage is below that threshold of .6, then my current is almost zero.
It's another lumped element called a diode.
And you will begin using these elements in your 002 lives to build interesting systems.
The next example is a thermistor.
A thermistor is a resistor whose resistance varies with temperature.
OK, so this is a very expensive little hairdryer, and what I'm going to do is blow some hot air at my resistor, and you're going to see that its value is going to change depending on how much I heat it.
So as it cools down, let me cool it down, so you can see it's coming down.
I can zap it again.
I could do this all day.
This is so much fun.
OK, so that's another interesting lumped element.
As the temperature rises, its resistance changes.
The next thing is called a photo resistor.
It's a resistor.
It used to be a resistor; Lorenzo?
Oh OK, that's fine.
So this is a photo resistor.
And notice that it almost behaves like an open circuit.
But what I'm going to do is shine some light on it.
When I shine light on it, it begins to conduct and becomes a resistor of some value.
There you go.
OK, so that's a photo resistor.
So now I'm going to show you a battery.
Notice we did talk about batteries before.
I'll show you a battery.
So before you show a battery, just thinking your own minds, what should the IV characteristic of a battery look like?
IV.
A battery supplies a constant voltage.
You know your little cell, the AA battery, 1.5 volts?
So, think of what the IV characteristic of a battery should look like for three seconds before it shows you.
This is the one I showed, Lorenzo?.
It's a straight line.
This is a good battery.
It's a straight, vertical line, but says that the voltage is 1.5 volts, or thereabouts.
No matter what current it supplies as an ideal voltage source, it has a fixed voltage, V, and no matter what the current going through is.
Now, I'll show you a dud, a bad battery, and this is what the bad battery looks like.
So, many of you have had your car batteries die on you.
When you go to the store, they check your batteries.
They use exactly this principle, that dead batteries have resistance.
By the way, you see slopes here.
You're thinking of resistance.
OK, they can use this property to figure out that your battery is dead.
So that's a dead battery.
And finally, let me show you a bulb.
We started with a bulb, and so I need to end, OK, we started with a bulb, so I need to end with a bulb.
And what you will see is that a bulb simply behaves like a resistor.
Its IV curve is going to look like this.
OK, notice this is my bulb.
And guess what, it behaves like a resistor.
It's a very interesting kind of resistor, so I won't go into details for now.
But notice its IV characteristic behaves like a resistor.
OK, so those are some pretty standard lumped elements.
You deal with a lot more sets of lumped elements, switches, MOSFETs, capacitors, inductors, a bunch of other fun stuff.
But before we do that, what I wanted to tell you, don't go berserk on this abstraction binge.
Too much of anything is bad for you.
So what I'm going to show you is, abstractions or models are only valid provided you work within a set of constraints.
Notice, we have already had this tacit handshake which said that we follow the discipline.
Even after we follow the discipline, there are ranges to how well physical elements can behave like ideal lumped elements.
OK, for example, what we will do is show you the resistor.
And it's going to look like a resistor.
And I'm going to keep increasing the voltage around it.
OK, what's going to happen at some point?
I just keep doing that.
If it's an ideal element, if you're a theorist, you say, oh yeah, the curve will keep extending until I reach infinity.
But this is a practical resistor, so people out here can cover your eyes or something.
OK, so you're abstraction can't predict that.
All it says is the current is an amp.
It can't predict the heat, light, or the smell.
In the laboratory, even, you get the smell.
You know what somebody has just done.
So that's one example of the lumped abstraction breaking down.
So, if I really believe that my own BS, anything is a lumped element.
So here's a pickle.
A pickle is a lumped element.
I can choose it as a lumped resistor.
But this is a very interesting lumped resistor.
Don't try this at home.
This is a standard pickle into which you are pumping 110 V AC.
I promise you, this is a standard pickle.
So, it has a fixed resistance, but your lumped abstraction cannot predict the nice light and sound effect.
OK, so the last two or three minutes what I want to do, so remember, don't get carried away by abstractions.
There are limits.
OK, you can't predict everything.
OK, that's the smell of a pickle.
OK, so let me give you a preview of some upcoming attractions, and show you one more quick simplification in the last few minutes.
So what we can do, once we build these lumped elements, we can connect them in circuits.
OK, so I can build a circuit, of the sort.
So here's a voltage source with a bunch of resistors.
I can connect them with wires and build a circuit of the sort.
One interesting question we can ask ourselves is, under the lumped matter discipline, what can we say about the voltages?
OK, if I go around the loop, provided my world adheres to the lumped matter discipline, what can I say about the voltages around this loop?
Ah-ha, Maxwell again, right?
So, I can write Maxwell's appropriate equation to solve that.
OK, voltages have something to do with E and your integral of E dot dl and all of that stuff, right?
So this is the appropriate Maxwell's equations to use.
And I want to find out what happens here.
Now remember, under LMD, I made the assumption.
OK, my world, my playground, has del phi B by del t being zero.
The rate of change of flux is zero.
So, under these circumstances, I can write this.
I can break up this line integral into three parts across the voltage source and across the two resistors and write that down.
OK, and then when I can do, is now that the right-hand side is zero, I can simply take this.
And I know that E dot dl across this element is simply VCA.
This is VAB, and this is VBC equals zero.
OK, so when I make the assumption that del phi B by del t is zero, and I go around this loop, apply Maxwell's equations, what do I find?
I find that the sum of the voltages, VCA plus VAB plus VBC, is zero.
That's fantastic.
So now, I could say hasta la vista to this baby here.
And I can focus on this guy and say, Maxwell's equations, this thing with squiggles and dels and all that stuff, can be simplified to the sum of the voltages across a set of elements in a loop in a circuit is zero.
OK, and this is called Kirchhoff's first first law, KVL.
OK, similarly, in recitation section, you'll see the application of Kirchhoff's current law, which comes from this be equal to zero, and all the currents coming into a node being zero.
So, KVL and KCl directly come out of the lumped matter discipline.
And you can use those to solve circuits like this.
Good morning, OK. Let's get started.
We have one handout today.
That's your lecture notes.
There's some copies still outside for those who haven't picked one up.
In general, what I do is, in the lecture notes, I leave out large amounts of material.
So, this will enable you to keep your hands busy while I'm lecturing and take down some notes and so on.
So, don't assume that everything that I talk about is on here.
Please follow along.
OK, so as is my usual practice, let me start with a quick review of what we covered so far.
So what we did primarily was looked at this discipline that we call the lump matter discipline, which was very similar, very reminiscent of the point mass simplification in physics.
And this discipline, this set of constraints we imposed on ourselves, allowed us to move from Maxwell's equations to a very, very simple form of algebraic equations.
And specifically, the discipline took two forms.
One is, we said that we will deal with elements for whom the rate of change of magnetic flux is zero outside of the elements, and for whom the rate of change of charge I want to charge inside the element was zero.
So, if I took any element, any element that I called a lump circuit element, like a resistor or a voltage source, and I put a black box around it, then what I'm saying is that the net charge inside that is going to be zero.
And this is not true in general.
We will see examples where, if you choose some piece of an element for example, there might be charge buildup, but net inside the, if I put a box around the entire element, I am going to assume that the rate of change of charge is going to be zero.
So, what this did was it enabled us to create the lump circuit abstraction, where I could take elements, some element of the sort, this could be a resistor, a voltage source, or whatever, and I could now ascribe a voltage, some voltage across an element, and also some current, "i," that was going into the element.
And as I go forward, when I label the voltages and currents across and through elements, I'm going to be following a convention.
OK, the convention is that I'm going to label, if I label V in the following manner, then I'm going to label "i" for that element as a current flowing into the positive terminal.
It's just a convention.
By doing this, it turns out that the power consumed by the element is "vi" is positive.
OK, so by choosing I going in this way into the positive terminal, the power consumed by the element is going to be positive.
OK, so in general of even simply following this convention, when I label voltages and currents, I'll be labeling the current into an element entering in through the plus terminal.
Remember, of course, if the current is going this way, let's have one amp of current flowing this way, then when I compute the current, "i" will come out to be negative.
OK, so by making these assumptions, the assumptions of the lumped matter discipline, I said I was able to simplify my life tremendously.
And, in particular what it did was it allowed me to take Maxwell's equations, OK, and simplify them into a very simple algebraic form, which has both a voltage law and a current law that I call Kirchhoff's voltage law, and Kirchhoff's current law.
KVL simply states that if I have some circuit, and if I measured the voltages in any loop in the circuit, so if I look at the voltages in any loop, then the voltages in the loop would sum to zero.
OK, so I measure voltages in the loop, and they will sum to zero.
Similarly, for the current, if I take a node of a circuit, if I build the circuit, a node is a point in the circuit where multiple edges connect.
If I take a node, then the current coming into that node, the net current coming into a node is going to be zero.
OK, so if I take any node of the circuit and sum up all the currents going into that node, they will all net sum to zero.
So, notice what I've done is by this discipline, by this constraint I imposed on myself, I was able to make this incredible leap from Maxwell's equations to these really, really simple algebraic equations, KVL and KCL.
And I promise you, going forward to the rest of 6.002, if this is all you know, you can pretty much solve any circuit using these two very simple relations.
It's actually really, really simple.
It's all very simple algebra, OK?
So, just to show you an example, let me do a little demonstration.
Let me build let me build a small circuit and measure some voltages for you, and show you that the voltages, indeed, add up to zero.
So, here's my little circuit.
So, I'm going to show you a simple circuit that looks like this, and let's go ahead and measure some voltages and currents.
In terms of terminology to remember, this is called a loop.
So if I start from the point C and I travel through the voltage source, come to the node A down through R1 and all the way down through R2 back to C, that's a loop.
Similarly, this point A is a node where resistor R1 the voltage source V0, and R4 are connected.
OK, just make sure your terminology is correct.
So, what I'll do is I'll make some quick measurements for you, and show you that these KVL and KCL are indeed true.
So, the circuits up there, could I have a volunteer?
Any volunteer?
All you have to do is write things on the board.
Come on over.
OK, so let me take some measurements, and why don't you write down what I measure on the board?
What I'll do is, let me borrow another piece of chalk here.
What I'll do is focus on this loop here, and focus on this node and make some measurements.
All right, so you see the circuit up there.
OK, so I get 3 volts for the voltage from C to A. so why don't you write down 3 volts?
OK, so the next one is -1.6.
And so that will be, I'm doing AB, V_AB.
OK, and then let me do the last one.
It is -1.37.
The measurements, I guess, have been this way.
So, what's written is V_AC.
But it's OK for now.
Don't worry about it.
So, well, thank you.
I appreciate your help here.
OK, so within the bonds of experimental error, noticed that if I add up these three voltages, they nicely sum up to zero.
OK, next let me focus on this node here.
And at this node, let me go ahead and measure some currents.
What I'll do now is change to an AC voltage so that I can go ahead and measure the current without breaking my circuit.
OK, this time around, you'll get to see the measurements that I'm taking as well.
So, what I have here, I guess you can see it this way.
What I have here is three wires that I have pulled out from D. And this is the node D, OK?
So, I have three wires coming into the node D just to make it a little bit easier for me to measure stuff.
OK, so everybody keep your fingers crossed so I don't look like a fool here.
I hope this works out.
So, you roughly get, what's that, 10 mV.
OK, so it's about 10 mV peak to peak out there, and let's say that if the waveform raises on the left-hand side, it's positive.
So, it's positive 10 mV.
And another positive 10 mV, so that's 20 mV.
And this time, it's a negative, roughly 20, I guess, -20.
So, I'm getting, in terms of currents, I have a -10, -10, I'm sorry, positive 10, positive 10, and a -20 that adds up to zero.
But more interestingly, I can show you the same thing by holding this current measuring probe directly across the node.
And, notice that the net current that is entering into this node here is zero.
OK, so that should just show you that KCL does indeed hold in practice, and it is not just a figment of our imaginations.
So, before I go on, I wanted to point one other thing out.
Notice that I've written down two assumptions of the lumped matter discipline, OK?
There is a total assumption of the lump matter discipline, and that assumption is, in spirit, at least, shared by the point mass simplification in physics as well.
Can someone tell me what that assumption is?
A total assumption, which I did not mention, which you can read in your notes in section 8.2 in the appendix, what's a total assumption that is shared in spirit with the point mass simplification?
Anybody?
A total assumption to be made here is that in all the signals that we will study in this course, we've made the assumption that the signal speeds of interest, transition speeds, and so on, are much slower than the speed of light.
OK, that my signal transition speeds of interest are much slower than the speed of light.
Remember, the laws of motion, the discrete laws of motion break down if your objects begin moving at the speed of light.
OK, the same token here, our lump circuit abstraction breaks down if we approach the speed of light.
And there are follow on courses that talk about waveguides and other distributed analysis techniques that deal with signals that travel close to speeds of light.
OK, so with that, let me go on to talking about method one of circuit analysis.
This is called the basic KVL KCL method.
So just based on those two simple algebraic relations, I can analyze very interesting and complicated circuits.
The method goes as follows.
So, let's say our goal is, given a circuit like this, our goal is to solve it.
OK, in this course, we will do two kinds of things: analysis and synthesis.
Analysis says, given a circuit, OK, what can you tell me about the circuit?
OK, so we'll solve existing circuits for all the voltages and currents, voltages across elements, and currents through those elements.
Synthesis says, given a function, I may ask you to go and build circuits.
OK, so for analysis here, we can apply this method that I want to show you.
And the idea here is that, given a circuit like this, let us figure out all the voltages and currents that are a function of the way these elements are connected.
So, the basic KVL and KCL method has the following steps.
The first step is to write down the element VI relationships.
OK, right down the element VI relationships for all the elements.
The second step is write KCL for all the nodes, and the third step is to write KVL for all the loops in the circuit.
That's it.
Just go ahead and write down element rules, KVL, and KCL, and then go ahead and solve the circuit.
So, what we'll do, we'll do an example, of course.
But, just as a refresher, we've looked at a bunch of elements so far, and for the resistor, the element relation says that V is pi R, where R is the resistance of the element here.
For a voltage source, V is equal to V nought.
That's the element relationship.
And for a current source, the element is the relation is, "i" is simply the current flowing through the element.
OK, so these are some of the simple element rules for the devices that the current source, voltage source, and the resistor.
So let's go ahead and solve this simple circuit.
And what I'll do is go ahead and solve the circuit for you.
OK, if you turn to page five of your notes, I'm going to go ahead and edit the circuit here.
You can scribble the values on your notes on page five.
OK, so as a first step of my KVL KCL method, I need to write down all my element VI relationships.
So, before I do that, let me go ahead and label all the voltages and currents that are unknowns in the circuit.
So, let me label the voltages and currents associated with the voltage source as here.
Notice, I continue to follow this convention where whenever I label voltages and currents for an element, I will show the current going into the positive terminal of the element variable, OK, after element variable voltage.
So here, I have V nought and I nought.
Let me pause here for five seconds and show you a point of confusion that happens sometimes.
Often times, people confuse between what is called the variable that is associated with the element versus the element value.
OK, notice that here, capital V nought is the voltage that this voltage source provides, while this name here, v nought, is simply a variable that we've used to label the voltage across that element.
So, similarly, I can label v1 as the voltage across the resistor, and i1 is the current flowing through the resistor.
So this method of labeling, where I follow the convention, that the current flows into the positive terminal is called the associated variables discipline.
I was trying to use the word discipline in situations where you have a choice, OK, and of a variety of possible choices, you pick one as the convention.
OK, so here, as a convention, we use the associated variables discipline, and use that method to consistently label the unknown voltages and currents in our circuits.
OK, so let me continue the labeling here, v4, i4, i3, v3 here, and v2 and i2, v5, and i5.
I think that's it.
So, I've gone ahead and labeled all my unknowns.
So each of these voltages and currents are the voltages and currents associated with each of the elements.
And my goal is to solve for these.
OK, so in terms of our solution here, let's follow the method that I outlined for you.
So, as the first step I am simply going to go ahead and write down all the element VI relationships.
OK, so as a first step, I'm going to go ahead and write down all the VI relationships.
So, can someone yell out for me the VI relationship for the voltage source?
OK, good.
So, v0 is capital V nought, that is that the variable V nought is simply equal to the voltage, v0.
Similarly, I can write the others.
v1 is i1, R1.
v2 is i2, R2, and so on.
OK, and I have one, two, three, four, five, six elements.
So, I will get six such equations.
Step two, I'm going to go ahead and write KCL for the nodes in my system.
So, let me start with node A.
So, for node A, let me take as positive the currents going out of the node.
So, I get i nought flowing out, plus i1 flowing out, plus i4 flowing out, and they must sum to zero for node A.
Then, I can go ahead and do the other nodes, let's say, for example, I do node B.
For node B, I have i2 going out.
That's positive, i3, and i1 is coming in, so I get -i1 equals zero.
OK, so I have one, two, three, four, I have four nodes.
OK, so I would get four equations.
It turns out that the fourth equation is not independent.
You can derive it from the others.
So, I get three independent equations out of this.
I can then write KVL.
And for KVL, I just go down my loops here.
And let me go through this first loop here in this manner.
OK, and a simple trick that I use, you have to be incredibly careful when you go through this in keeping your minuses and pluses correct.
Otherwise you can get hopelessly muddled.
Once you label it, you need to be sure that you get all your minuses and pluses correct.
So, for KVL, what I'd like to do is, let's say I start at C, and from C I'm going to go to A.
For A I go to B, and from B I'm going to come back to C. OK, that's how I traverse my loop.
And, the trick that I'm going to follow is, as my finger walks through that loop, I'm going to label the voltage as the first sign that I see for that voltage.
OK, so I'm going to start with C, and I go up.
I start by punching into the voltage source element, and then punch into it, I hit the minus sign for the V nought.
OK, so I'm just going to write down minus V nought, plus then I go through and as I come up to A and go down to B, I punch to the plus sign of the V1.
So, that's plus V1.
And then I punch into the plus sign of the V2, and so I get plus V2, and that is zero.
OK, good.
So, that matches what you have in your notes as well.
So, this is the first equation.
Similarly, I can go through my other loops and write down equations for each of the loops.
OK, and the convention that I like to follow is as I go through the loop, I write down as a sign for the voltage the first sign that I counter for that element.
OK, you can do the exact opposite, if you want, just to be different.
But, as long as you stay consistent, you'll be OK. All right, so in the same manner here, there are four loops that I can have, so four equations.
Again, one of them turns out to be dependent on the others.
So I end up getting three independent equations.
So, I get a total of 12 equations.
I get 12 equations.
There are six elements, OK, voltage source, and five resistors.
So, there are six unknown voltages, and six unknown currents.
So, I have 12 equations, and 12 unknowns.
OK, I can take all of the equations and put them through a big crank, and sit there and grind.
And if I was really cruel, I'd give this as a homework problem, and have you grind, and grind, and grind until you get your six voltages and six currents.
OK, it works.
OK, so you get 12 equations, and this method just works.
However, notice that this is quite a grubby method.
It's quite grungy.
I get 12 equations, and it's quite a pain even for a simple circuit like this.
However, suffice it to say that this fundamental method is one step away from Maxwell's equations, simply works.
OK?
So what you'll do is the rest of this lecture, I'll introduce you to a couple more methods.
One is an intuitive method, and another one called the node method is a little bit more formal, but is much more, I guess, terse Than the KVL KCL method.
Method 2.
So the relevant section to read in the course notes is section One of the things that I will be stressing this semester is intuition.
What you'll find is that as you become EECS majors, and so on, and go on, or if you talk to your TAs or your professors and so on, you will find that very rarely do they actually go ahead and apply the formal methods of analysis.
OK, by and large, engineers are able to look at a circuit and simply by observation write down an answer.
And usually in the past, what we have tried to do is kind of ignore that process and told our students, look, we teach you all the formal methods, and you will develop your own intuition and be able to do it.
What we'll try to do this term is try to stress the intuitive methods, and try to show you how the intuitive process goes, so you can very quickly solve many of these circuits simply by inspection.
OK, so this method that I'm going to show you here is one such an intuitive method.
And I'll call it element combination tools.
OK, for many simple circuits, you can solve them very quickly by applying this method.
The components of this method are these.
I learned about how to compose a bunch of elements.
So, let's say, for example, I have a set of resistors, R1 through RN, in series.
OK, you can use KVL and KCL to show that this is equivalent to a single resistor whose value is given by the sum of the resistances.
OK, so if I have resistors in series, then effectively it's the same as if there was a single resistor whose value is the sum of all the resistances.
OK, you can look at the course notes for a proof for derivation of this fact.
Similarly, if I have resistances in parallel, so let me call them conductances.
A conductance is the reciprocal of a resistance.
If resistance is measured in ohms, conductance is measured in mhos, M-H-O-S. OK, so that's the conductance is G1, G2, and G3.
And effectively, this is the same as having a single conductance whose effective value is given by the sum of the conductances.
OK, the conductances in parallel add, and resistances in series add.
Similarly, for voltage sources, if I have voltage sources in series, then they are tantamount to the sum of the voltages.
And similarly, for currents, if I have currents in parallel, then they can be viewed as a single current source, whose currents are the sum of the individual parallel currents.
So, let's do a quick example.
So let's do this example.
So, let's say I have a circuit that looks like this, and three resistances.
And let's say all I care about is the current, I, that flows through this wire.
All I care about is that current.
Of course, you can go ahead and write KVL and KCL.
You will get four equations, and there are four unknowns.
And you can solve it.
But, I can apply my element combination rules, and very quickly figure out what the current I is, using the following technique.
So, what I can do is, I can, first of all, take this circuit.
And, I can compose these two resistances and show that the circuit is equivalent as far as this current, I, is concerned to the following circuit, R1.
And I take the sum of the two conductances, OK, and that comes out to be R1, R2, R3, R2 plus R3.
And then, I can further simplify it, and I get a single resistance, whose value is given by R1 plus R2, R3, R3.
OK, I'm just simplifying the circuit.
Now, from this circuit, I can get the answer that I need.
I is simply the voltage, V, divided by R1 plus.
OK, so in situations like this where I'm looking for a single current, I can very quickly get to the answer by applying some of these element combination rules.
And, I can get rid of having to go through formal steps.
So, in general, whenever you encounter a circuit, by and large attempt to use intuitive methods to solve it.
And go to a formal method only if some intuitive method fails.
Even in your homework, by and large, the homeworks are not meant to be grungy.
OK, if you find a lot of grunge in your homework, just remember you're probably not using some intuitive method.
OK, so just be cautious.
All right, so let me go on to the third method of circuit analysis, and the third method is called the node method.
So, the node method is simply a specific application of the KVL KCL method and results in a much, much more compact form of the final equations.
If there's one method that you have to remember for life, then I would say just remember this method.
OK, the node method is a workhorse of the easiest industry.
OK, if there's one method that you want to consistently apply, then this is the one to remember.
So, let me quickly outline for you to method, and then work out an example for you.
The first step of the node method will be to select a reference or a ground node.
This is the symbol for a ground node.
The ground node simply says that I'm going to denote voltages at that point to be zero, and measure all my other voltages with reference to that point.
So, I'm going to select a ground node in my circuit.
Second, I want to label the remaining voltages with respect to the ground node.
So, label voltages for all the other nodes with respect to the ground node.
Next, write KCL for each of the nodes write KCL.
OK, but don't write KCL for the ground node.
Remember, if you have N nodes, the node equations will give you N-1 independent equations.
So, write KCL for the nodes, but don't do so for the ground node.
Then, solve for the node voltages.
So, let's say when we label voltages.
I want to be labeling them as E something or the other.
So, solve for the unknown node voltages.
And then, once I know all the voltages associated with the nodes, I can then back solve for all the branch voltages and currents.
OK, once I know all the node voltages, I can then go ahead and figure out all the branch voltages and the branch currents.
So, let's go ahead and apply this method, and work out an example.
Again, remember, if there's one method that you should remember, it's the node method.
OK, and when in doubt, consistently apply the node method and it will work whether your circuit is linear or nonlinear, if the resistors are built in the US or the USSR it doesn't matter.
OK, the node method will simply work, linear or nonlinear, OK?
So, what I'm going to do is I'm going to build a circuit that's my old faithful.
It's our old faithful, plus I'll make it a little bit more complicated by adding in the current source.
So, let's go have some fun.
Let's do this.
So here's my voltage source, as before.
OK, what I'll do is for fun, add a current source out there.
And, you can convince yourselves that if you go ahead and apply the KVL KCL method, it'll really be a mess of equations.
OK, so R1, R3, R4, R2, R5.
OK, so let's follow our method and just plug and chug here.
So let's apply the first step.
I select a ground node.
It's a reference node from which I'll measure all my other voltages.
OK, now without knowing anything about the node method, try to use intuition as to which node you should choose as a ground node.
Remember, you want to label the ground node with the voltage zero, and measure all the other voltages with respect to that node.
OK, a usual trick is to pick a node which has the largest number of elements connected to it as the ground node.
OK, and in particular, you will find out later it's useful to pick a node in which all your voltage sources, the maximum number of your voltage sources are also connected.
OK, so in this instance, I'm going to choose this as my ground node.
OK, that's my first step.
I chose that as my ground node.
And I'm going to label that as having a voltage zero.
Second step, I'll label voltages of the other branches with respect to the ground node.
OK, so what I'll do is add this node here.
So I'm going to label that voltage E1.
These are my unknowns.
Remember, node method, because my node voltages are my unknowns.
So, I label this as E1.
I label this one as my unknown voltage, E2.
What about this one here?
Is that voltage unknown?
No.
I know what the voltage is because I know that this node is at a voltage, V0, higher than the ground node.
OK, notice that to go from here to here, I directly go through a voltage source.
And so, this node has voltage V0.
And I'll simply write down V0.
OK, try to simplify the number of steps that you have to go through, so directly go ahead and write down the voltage, V0, for that node.
What I will also do, is for convenience, I'm going to write down, I'm going to use conductances.
So I'm going to use GI in the place of one by RI, and write down a bunch of node equations.
OK, so step one, I've chosen my ground node.
Step two, I've labeled my node voltages, E, OK?
I've done that with two of my steps.
Now, let me go ahead and -- OK, so let me go ahead and apply step three.
And, step three says go ahead and apply KCL for each of the nodes at which you have an unknown node voltage.
And then that will give you your equations.
So let me start by applying KCL at E1.
So, let me write KCL at E1.
I do one more thing.
Notice, I don't have any currents there.
OK, so how do I write KCL?
KCL simply says the sum of currents into a node is zero again, remember, by the lump matter discipline.
So, if I don't have currents in there, so the trick that I adopt is that to write KCL, I use the node voltages, and implicitly substitute for the node voltages, divide by the elemental the resistance, for instance, so I take the node voltages, and divide by the resistance, get the current.
OK, so I implicitly apply element relationships to get the node currents.
So, the example that make it clear, so I take node E1 and, again, for currents going out I'm going to assume to have, to be positive.
So, the current going up is E1 minus V nought, divide by R1, so I multiplied by the G1.
That's the current going up.
Plus, the current going down is E1 minus zero where the ground node potential is zero, G2, OK, plus the current that is going to resistor R3, which is simply E1 minus E2, divide by R3.
So, E1 minus E2, divide by R3, or multiplied by G3 is equal to zero.
OK, see how I got this?
This is simply KCL, but to get my currents, I simply take the differences of voltages across elements, and divide by the element of resistance, and I get the currents.
OK, so I can similarly write KCL at E2.
So, at KCL at E2, again, let me go outwards.
So, the current going up is E2 minus V nought multiplied by G4.
The current going left is E2 minus E1 divided by R3 or multiplied by G3.
The current going down is E2 minus zero multiplied by G5.
And, the current going down is -I1.
OK, you've got to be careful with your polarities here.
So all the currents going out sum to zero.
And here are the currents that are going out at this point.
So what I do next is I can move the constant terms to the left-hand side and collect my unknowns.
So, let me write them out here.
So, let's say I get E1 here, OK, and from this equation, I have a V nought, G1, which comes out here.
So, minus V nought G1 comes over to the other side.
And, let me collect all the values that multiply E1.
So I get, G1 is one example.
I have G2, and I have G3.
And then, for E2, I have minus G3.
OK, so I'll simply express this as the element voltages multiplied by some terms in parentheses, and I put my external sources on the right hand side.
Similarly, I go ahead and do the same thing here.
In this instance, let me move my sources to the right.
So, I get I1 coming out there, and I get V nought G4 coming out there.
By the way, I just want to mention to you that if you're looking to fall asleep, this is a good time to do so because as soon as I write down these two equations, OK, from now on it's nap time.
There's nothing new that you're going to learn from here on.
It's just Anant Agarwal having fun at the blackboard, pushing symbols around.
So, once you write down these two node equations, the rest of it is just grubby math.
So, let me just have some fun.
So let me just go ahead and do that.
So, I moved my voltages and currents to the other side.
And let me collect all the coefficients for E1 here.
So, E1 minus G3, and that's it, I guess.
OK, and then I'll do the same for E2.
So, I get G4, and I get G3, and I get G5.
OK, so notice here that I have two equations, and two unknowns.
OK, the two equations are on the right hand side, I have some voltages and currents which are my dry voltages and dry currents.
OK, so actually this is getting quite boring.
I'm going to pause here, and talk about something else.
So, you can take this and you can put it in matrix form, so I've done that for you on page ten.
It's all matrix form.
Yeah, I know that.
You can use any technique to solve it, use algebraic techniques, use linear algebraic methods to solve it, use a computer, whatever you want.
And, computers, when computers analyze circuits, they write down these equations, and deal with solving matrices.
So, when you take the linear algebra across, how many people here have taken a linear algebra class?
How many people here have heard of Gaussian elimination?
How can more people have heard of Gaussian elimination than took a linear algebra class?
Well anyway, so now you know why you took those linear algebra classes.
And so, if I just collected these into matrix form -- OK, so I just simply expressed those two equations in linear algebraic form, and here's my column vector of unknowns, and you can apply any of the techniques you've learned in linear algebra to solve for this.
Gaussian elimination works.
OK, and in computer, people doing research in computer techniques, or solving such equations simply deals with huge equations like this, building computer programs that, given equations like this, can go ahead and solve them.
OK, so let me stop here and reemphasize that what you've done is made a huge leap from Maxwell's equations to using the lump matter discipline to KVL and KCL, which ended up giving a simple algebraic equation to solve, and not having to worry about partial differential equations that were the form of Maxwell's equations.
Let's get started.
Can you hear me back there?
Loud and clear.
OK. Let's get started.
Before I begin, just a couple of announcements.
Brad Buren is one of our students here and he needs a note-taker.
It's a paid position.
So if you are interested you can stop by after class and see him.
He's sitting right here out there, OK?
Second, just a reminder that 6.002 does have prerequisites.
And the prerequisites are 8.02 and 18.03.
So with that let me start off with the usual.
Do a quick review of what we've done so far.
So we started out life looking at the laws of physics and Maxwell's equations and so on.
And those were way too hard so we said let's make life easy for ourselves.
So we chose to play in this playground in which we said we shall adhere to the lumped matter discipline.
OK?
The LMD.
So we are in that playground.
So this entire course, and for that matter large parts of EECS are within that playground, within which the lumped matter discipline applies.
So as soon as we jumped into the playground, the LMD playground, we could take Maxwell's equations and abstract them out into two very, very simple rules.
And the very simple rules were KVL and KCL.
KVL simply said that I can sum the voltages in any loop in a circuit and the result then would be zero.
Similarly, I can sum the currents that enter or exit any node and the sum will also be zero.
So what you can now do is, if you feel like, you can go around and brag.
Oh, yeah, we use Maxwell's equations in everyday life and, yeah, it's good stuff.
And the key is that this is really an encapsulation of Maxwell's equations within this playground that we are in.
So I talked about the first method of circuit analysis in the last lecture.
And that method simply took the, wrote KVL for all the loops, wrote KCL for all the nodes and wrote element vi relationships.
And together gave you a big bunch of equations.
And you sat down and grunged through the equations and you solved for branch voltages and currents.
So we reviewed a second method of circuit analysis.
And I'll simply call it circuit composition.
The basic idea behind this method was to learn some simple rules of how resistors add and conductances add and so on and so forth and look at a circuit and simplify the circuit by making series simplifications when the resistors are in series and so on and so forth, and compose it and play around with it till we end up with the current, the voltages that we are looking for.
This is the intuitive method.
And so a section in Chapter 2, I believe, of the course notes discusses several examples using this method and attempts to make a little bit formal the intuitive approach that is applied in this method.
So we then looked at the node method.
And the node method was simply a particular way of applying KVL and KCL.
Node method, remember?
We took a ground node.
Then we labeled the nodes of the remaining voltages with respect to that ground.
Then we wrote KCL for each of the nodes.
And when we wrote KCL for each of the nodes, remember, KVL was implicit in this expression that we used for each of the currents that were exiting each node.
So if Ej was a node voltage, then Ej minus Ei multiplied by the conductance Gi was the current that was going through one of those, I should call it Gij.
This is a conductance that connects nodes i and j.
That gave us the KVL that fed into the same system.
So these are three methods.
The node method, by the way, is sort of the workhorse of the 6.002 industry.
And for that matter for all of the circuits industry.
When in doubt, apply the mode method, you'll be OK. That applies to linear circuits, nonlinear circuits, what have you.
What I'm going to do today is go through two more methods.
So notice that the first few lectures of this course, the first three lectures simply comprise transitioning you from the world of physics to the world of EECS.
And then two lectures on giving you a bag of tricks.
So we start you off with the sort of tools, your mallets and chisels and so on and so forth.
And these five methods are your tools.
We'll look at two methods today.
One method is called the method of superposition and the second method is called the Thevenin method.
And these methods apply only to linear circuits.
So we look at the subset of circuits that are linear, and these two methods apply to only those circuits.
These are methods that combined with intuition really enables you to solve very interesting circuits very, very quickly.
So let me do an example using a usual node method.
And then jump into introducing the superposition methods and Thevenin methods using that same example.
So let me draw you an example circuit here.
So, again, I'm using this example, I will use this example to introduce the method of superposition and the Thevenin method.
So what I'm going to do is start off the usual way and analyze the circuit using a method that you know now, the node method.
And what I'll do is write down the node equations for this by applying the node method.
So if you recall the node method.
I choose a ground node.
I'm going to choose this node.
It's got both the voltage source connected to it, and it's also got many other edges impinging on it.
So I'm going to choose that as my ground node and I'm going to label the other nodes with their voltages.
So this is an unknown.
I'll label it as e. I guess we just have one unknown e. And I know the voltage of this node, and that is simply V. Since it's V, there's a voltage source between the ground node and that node.
So what I can do next is that I can write down the node equation for this node and then go from there.
So let me go ahead and do that.
So let me sum up the currents going outside, going outwards.
So I have e minus v divide by R1, I have e minus zero divide by R2, and I have minus i equals zero.
This is a node equation.
The first thing I want you to observe is that this equation is linear in V and i.
What I mean by linear is that you don't see terms like Vi or V-squared and things like that.
It's some constant times V plus some constant times i equals some other constant.
So that's quite nice.
So I'm going to rearrange the terms in the following manner.
I'll move the known sources to the right-hand side and collect the coefficients of e on this side, so I get one by R1 plus one by R2 over here.
So stare at this for a moment and notice again here I have e, my unknown node voltage, there is some constant multiplier, and that equals some function of V summed up with some function of i.
And, again, notice that this is a linear combination of V and i.
No multiplication terms and so on and so forth.
This is a pretty standard form in which we will represent equations quite often.
And just to label it, this is often labeled G as the conductance matrix.
Of course this is e, our unknown node voltages, and this is a linear sum of sources.
So this is a very standard way that we will represent equations.
We did that last week as well, or rather on Tuesday where I took a conductance matrix, multiplied that by a column vector of unknown node voltages and equated that to some linear combination of my source voltages.
The reason the circuit is linear is that I have only linear elements in the circuit.
I don't have any nonlinear elements.
And because of that I can rewrite this in the following manner.
I'm just going to express e as a function of V and i and bring it over to this side.
So it's some function of i.
So I get R1 R2 divide by R1 plus R2.
And I bring R1 R2 to this side.
That's what I get.
So stare at this for a few seconds, very common form.
My unknown node voltage is equal to this stuff on the right-hand side.
The stuff on the right-hand side has a term multiplying the source voltage V and some other term multiplying the current I.
And if I were to put this in sort of symbol-like form my unknown node voltage is some constant times V1 plus some constant times, is of the form constant times the source current, constant times the source voltage and so on.
The units of As and Vs are different because in this case A has no units because V is a voltage.
And so is e. In this case V has units of resistance.
So that V times i gives me a voltage.
So stare at this equation for a few seconds and this should help us build up some insight that will allow us to write down the answer almost by inspection.
I'm going to show you a method now, in a few minutes, which will allow you to write down the answer e just by starring at the circuit without having to go through node equations and so on.
The more and more methods I teach you, the more you will be able to do a lot of this completely by yourselves.
In this particular example it's a relatively simple circuit but these methods would be particularly useful when you have more complicated situations.
But before I go on let me spend a few minutes pontificating on linearity.
So that's a linear circuit.
And this equation gives me the unknown node voltage e as a linear sum of source voltages and source currents.
Linearity implies two properties, the property of homogeneity and also gives vice to the property of superposition.
Let's do homogeneity first.
What this says is if I have a circuit, some circuit and I feed it some sort of inputs, A, then let's say my output is S. If you're feeling hungry think of these as apples and the circuit converts them into applesauce.
So what homogeneity says is that what I can do is if I take each of my apples and instead of feeding it an entire apple what if I give it three-quarters of an apple?
Say I multiple all my inputs by some constant alpha, three-quarters.
What that says is that at the output instead of getting one full bottle of applesauce I'm going to get three-quarters of a bottle of apple sauce.
So if I proportionately reduce all the inputs and if this is a linear circuit then so shall my output be reduced in the same proportion.
So that's homogeneity.
Next, let's look at superposition.
The property of superposition says the following.
The same kind of circuit.
If I feed it apples then I get applesauce.
I take the same circuit, and this time around if I feed the circuit a different set of inputs, say blueberries.
And let's say my output, oops, let me do it this way.
So as my output I get blueberry sauce, if such exists.
So apples applesauce, blueberries give me blueberry sauce.
Then what I'm going to get if I mix up the two, so let's say I take my circuit, the same circuit with a set of inputs and in this example one output.
Let's say I mix up my inputs and some of my inputs in the following way, here I feed an A1 plus B1 and here A2 plus B2 and so on then at the output I am going to get a mush of apple sauce and blueberry sauce.
All this says is that if I apply just apples I get applesauce.
If I apply just blueberries I get blueberry sauce.
Then if I were to figure out how this blender would have worked had I fed in the combinations of apples and blueberries, then for the purposes of understanding that blender all I could have done was taken by two outputs and just mixed them up together myself and that's exactly what I'd get.
So if I sum up the inputs my outputs would also be the sum of the outputs with the inputs applied by themselves.
So let me take this here and munge around with hit for a few seconds and get something interesting out of it.
So notice two inputs, two inputs, outputs.
In your notes I've given you another template for the next set of scribbles I'm going to make here.
So use the next set of templates on page three.
What I'm going to do here is something very simple, set one output to zero and feed a voltage V1.
So that's feed a voltage V1 and set the other output to zero.
And let's say I get Y1 as an output.
And in this case I set the first voltage to zero and feed a different voltage V2 on the second input.
And let's say my output is Y2.
This is just a particular application of the superposition principle I just outlined.
Apply V1 set one output to zero.
Apply V2 set the original output to zero.
Then what I'm going to find is that the answer will simply look like this, just replace for As and Bs what I just did and we get V1 and zero here and we get zero and V2 here.
And as my output I'm going to get exactly the sum Y1 plus Y2.
This is simply a particular application of superposition where what I'm saying is the following.
If you look at this circuit here effectively what have I done?
Effectively what I've done is apply the voltage V1 on one input and a voltage V2 on the other input.
V1 here.
V2 here.
And the output is Y1 plus Y2.
What I'm saying is look backwards now.
What I'm saying is that the whole components of the output Y1 plus Y2 could individually be derived in the following manner.
I could get the component Y1 by simply applying one of the voltages and setting the other to zero.
I can get the other component Y2 by setting yet another input to zero and applying the voltage V2 to get Y2.
And sum then up and that's my answer.
This will become a lot clearer with an example.
Again, remember if I have a bunch of inputs applied to a circuit, V1, V2 and so on, and I get some output then what this is saying is that I can alternatively find out the answer by applying just one voltage, setting all the others to zero, measuring the output, apply a second voltage, set all inputs to zero, measure the output and sum of applesauce and blueberry sauce and there you get the answer.
Let's do an example.
And before we go into that I talked about setting voltage sources and current sources to zero.
First of all, what does it mean to set a voltage source to zero?
This is the same as this.
Setting a voltage source to zero is simply replacing the voltage source with a short, and setting a current source to zero simply implies an open circuit.
So when I say zero that source, if it's a voltage source short it, if it's a current source open it.
I can take any two nodes in the world and measure the potential difference across them.
So there may be some potential difference across these set by the circuit that I haven't shown you on this side.
There might be some other circuit that is controlling the voltage of these two nodes.
The same with the short.
What's V going to be?
But there is a V. It's zero.
So that's method four, method of superposition.
And this method says that the output of a circuit -- Again, remember I'm focusing on linear circuits.
Remember, I have this playground where LMD applies.
And within that playground I'm playing in the south goal area.
In the south goal area, in that subset of the playground circuits are linear.
So in that part of the playground superposition applies because there circuits are linear.
So the output of a circuit is determined by summing up the responses to each source acting alone.
Now, in this statement here this source stands for independent source.
I haven't talked about independent versus dependent sources.
We'll talk about dependent sources a few weeks from today.
And just so you don't get confused, for dependent sources you will be looking at Section 3.3.3 of your course notes to see how superposition works with dependent sources.
But remember we haven't covered dependent sources yet.
We will be covering them about two weeks from now.
So let's go back to our example and apply the method of superposition to an example.
So the method says sum up the outputs of each of the sub-circuits where I'm applying one source acting alone.
So let me just do this here.
Let me start with the circuit.
And let me start with shutting I off.
So I have voltage V -- I have R2.
And I'm shutting I off.
So I have replaced this with an open circuit.
So I is zero.
Let me call the node voltage eV to reflect that component of the node voltage that arises due to V acting alone.
And you should look at this pattern here and very quickly be able to write the answer for patterns like this voltage, the two resistors.
That's called a resistive divider.
It will appear again and again and again.
And eV is simply V times R2 divided by R1 plus R2.
That's still my ground node.
So the voltage here is simply this voltage divided by the two resistors to give you the current multiplied by R2 to give you the voltage across this R. Remember this pattern.
You apply voltage divider patterns probably more times than any other pattern that you might imagine.
So that's with the V acting alone.
Now, let me do I acting alone.
So for I acting alone -- And what I do this time around is replace this with a short, replace the voltage source to the short.
And let me call this voltage eI for the component of the voltage due to the current I.
And eI, in this case, is simply given by yet another pattern here, the current across a pair or resistors is simply the effective resistance multiplied by the current so it's i and the effective resistance is R1, R2 or R1 plus R2.
That's eI.
That's a component that node due to the current I.
Now, so the method says that.
Then take these components, sum them up and there you have the answer.
So E is simply ev plus ei.
The components of V and I acting alone, just simply V times R2 divided by R1 plus R2 plus R1, R2.
There we go.
Fortunately, the fates have been kind to us and the answer is the same as the answer we obtained with the node method.
No surprise here.
So this is actually an incredibly simple method.
So you can take a very complex circuit.
What have you really done here?
You can take a very complex circuit and you can solve a very complex circuit by breaking it down into many simple individual sub problems.
You will do this in EECS time and time and time again.
Whether it's in software systems or hardware systems or what have you, you're often times building complicated systems.
Remember doom on this side?
And the way and when you put these things together, let's say a large software system, is you don't write the whole piece of software starting main and grunge down.
You build a lot of little components and tie the components together.
In the same manner here you take a big circuit and you find its behavior for each source acting alone.
Lots of little inky dinky simple little circuits.
And you will see examples in your homework where you're given a big circuit or because it set all the Is to zero and the other Vs to zero the whole circuit almost vanishes and all that you're left with is a little resistor or two.
So this is the very, very powerful method.
I'd like to do a little demonstration for you.
And what I'm going to show you is the demo is a vat of water.
Actually, I'll tell you what it is in a second.
But assume it is salt water for now.
I'll apply two voltages.
In this case I'm going to apply a sinusoid.
That's not very good.
A sinusoid and a triangular wave.
And what I'm going to do is measure the response at this site.
Now, this is a vat of salt water.
And I'm going to tell you it behaves like a linear system.
If you view each little particle, or each little cubic-centimeter or whatever of water, it'll behave like little resistor.
So this vat of salt water behaves like big distributed resistor in the following manner.
And so on.
This of this big mesh of little resistors, but it's all resistors.
It's a linear circuit.
So I'm going to apply two voltages, a triangular and a sinusoid, and we're going to observe the output.
And what do you expect to see there?
You will see the superposition of the two, which is you'll see a sinusoid.
And then you'll see the jagged triangular thing articulating the sinusoid pattern.
What I'm going to do right now, don't put any water yet.
This is the vat of nothing right now.
It's all empty.
Can we show the screen on this side?
The oscilloscope screen?
OK. Oh, there you go.
So this is the screen of the oscilloscope now.
Notice that I have a sinusoid and I have a triangular wave and the output is zero.
And the reason is there is nothing in this vat.
It's empty.
So previously when I taught this course I would get saltwater and pour saltwater.
Then we discovered a much better source of water that conducted electricity like one real mean fluid.
Cambridge water.
It just works very pleasantly.
It just conducts electricity like nothing at all.
And I've been thinking of using Charles River water next time and see what happens, although there we'd probably get some biological organisms doing strange things at you.
But go ahead.
Our friendly demonstration expert, Lorenzo, will pour some water into the vat.
And you should begin seeing the output being a superposition of the two.
So as he pours, there you go, do you see that?
So you do see the sinusoidal articulation and the jagged wave form.
And just to have some more fun, what I can do is increase one of the voltages.
And you'll see -- Now you know what would have happened if I had used Charles River water.
So my output keeps increasing as I increase the corresponding wave form.
I could do this, this is fun.
So let me pause there and go onto the next topic.
So that little demonstration showed you that even something as simple as this physical entity vat of water behaves like a linear system, and we can model that linear system as a set of resistors.
Unbeknownst to you, right now, in the past ten seconds I introduced a new concept.
It's called subliminal advertising.
So one of the things we do in EE a lot is model real systems.
So often times if I wanted to look at the behavior of salt, behavior of a vat of water, I can model it as a set of resistors for certain kinds of activities.
Just hold that thought for some time later in your careers.
All right.
That's method four, the superposition method.
Remember, it is methods like this that will make your life really, really, really easy.
If you find that you are having to do a lot of grunging homework or something, just step back and think superposition, think Thevenin or think composition rule.
There must be a simpler way usually.
Let's do the next method.
This is called the Thevenin method.
To derive this method let me start by applying superposition to some circuit.
So let's say I have some arbitrary network N. Assume it's a linear network and the network has a whole bunch of goodies in it.
It has a bunch of resistors, it has a bunch of voltage sources, and it has a bunch of current sources.
Many current sources.
Many voltage sources.
Many resistors.
Some jumbled voltage sources, current sources and resistors.
And I look at two nodes in this network.
Here are two nodes in the network, two points in the network were elements connect.
I'm looking at those two nodes and all I want to do is the following.
I want to figure out if I take a rinky-dinky little current source and apply it there, all I want to figure out is what is V and what is I.
There is this mongo box out here, a black box of resistors, voltage source and current sources, too many to count.
I pick two nodes, apply a current source, and all I care about is what is the voltage that I will measure by applying it here.
Notice the current here will be I because the current here is I.
And I apply it here.
I want to measure what the voltage is.
Now, with the insight you've obtained from superposition, you should be able to jump up and state the form of the answer.
So by superposition we know the following.
We know that the effect of the circuit will be the same as the sum of components being added up.
Sum of component, sum of component, a bunch of components added up.
Each component will be the response of one source acting alone.
So if I can figure out the effect of one source acting alone and put that down here, and do the same thing for all the sources, that's what I will get.
So for the source Vm it's a linear circuit.
So I know that my answer is going to be, in the final answer is going to be a Vm term and it's going to be multiplied by some alpha M term.
I know that.
It's a linear circuit so I know that the answer shall have a term Vm multiplied by some constant.
Simple, I know that.
Similarly, the same is true for, oh, this is the term Vm.
And what I can do is I can measure just this effect by setting all the other sources to zero.
So I can set all the other current sources to zero and all voltage sources, except for this one, and I can get that answer.
So, similarly, for every voltage source I am going to get a term.
So for every single voltage source, M1, M2, M3 and so on I'm going to get such a term and they're all going to sum up.
Similarly, I'm going to get a term for In.
And I know there will be an In term, and I know it's going to be some constant beta multiplying In.
In this example of ours here, in this example, remember alpha was this and beta was this constant here.
There's some constant beta, some constant alpha.
And because I have a whole bunch of current sources there's going to be such a term for each one of them.
And each one of these terms, Vm, In will be the voltage I would see here if I set all the other Vms to zero and I set all the other current sources, except for that one to zero.
What am I missing?
Is that it?
The response here, V here.
Am I missing anything here?
Is that it?
Now, don't all yell at once.
What am I missing?
Current source i, exactly.
So if I have a current source i then there's an effect of this current as well.
And so I write down i there, too.
It's going to be some constant multiplying I.
And that constant is going to look like a resistor, right, because this circuit contains current sources, voltage sources and resistors.
If I've shorted all my voltage sources and opened all my current sources, what's left in here?
Just a whole caboodle full of Rs.
It's just going to look like some resistance R. And that's what I get here.
So this is what V is going to look like and that's a form.
So let's take a look at these components.
Let's focus on the easy part first.
What does this look like?
This component looks like an I, it looks like a current and has some resistance.
What is that resistance given by?
Supposing I gave you this network and this currency source and I asked you tell me R. How would you measure R?
What you would do is open all the current sources, short all the voltage sources, put a ohmmeter in there and measure the resistance R. That's R. OK, so we understand this term.
What about this term here?
Can someone tell me the units of this term here, this big thing here?
Voltage.
This is a voltage.
This is a voltage.
iR is a voltage.
So this does behave like a voltage.
And it behaves like some voltage V. So notice that as far as this current I is concerned the rest of the universe looks like a resistor and a voltage source behaving in some manner.
And let me just call it Vth for now, and you'll know why in a second.
The voltage has a form, some voltage plus Ri.
So, in other words, as far as this I is concerned this whole network here N full of all the nice stuff is indistinguishable to this I here.
So my I is sitting out there injecting a current into two nodes.
If I am i, I'm looking at this, this network looks no different than a voltage source in series with the resistor R. Notice that the equation for this simple circuit is this, so I is given by V minus Vth divided by R. Just remember.
It's a circuit.
In other words, Agarwal sitting here cannot tell the difference if I'm measuring the voltage here between a circuit that looks like a Vth in series to the resistor or this huge mess of voltage sources and current sources and so on.
Now, we will talk about Vth and R. R is called the resistance of the network as seen from the port with all the sources shut off.
And similarly Vth, what is Vth?
Vth is the open circuit voltage.
In other words, if I apply the voltage here this is the response of all the current sources and all the voltage sources acting together.
So it's as if I took this out and simply measured my V here as if I didn't exist, correct?
Because this is the component of i.
So if I opened i and measured V, I would get that big term on the left-hand side.
That's my Vth.
So that inspires the next method called the Thevenin method.
In this method what I'm going to do is take some circuit, I'm on Page 9, with a mess of stuff.
It's a big mess of stuff.
And if I care to look at its impact on something else that I add from the outside then as far as the outside world is concerned this is indistinguishable from a circuit that looks like this.
So what I can do is if I want to figure out what's happening here then, for the purpose of my analysis, this simple network here with R and Vth becomes a surrogate for this entire mess.
So for the purpose of finding out the behavior at this point, I can take this huge mess and replace it with its Thevenin surrogate or Thevenin equivalent.
This is called the Thevenin equivalent of this big network.
Let me do an example that will make the method completely clear.
Again, remember in EECS, most of our lives are about how can we make things so simple as being able to be analyzed by inspection?
And so this is a method that takes you further down that path.
So let me use the same circuit that I've been using before, my voltage V, R1, R2.
This is an R. I'm 55 minutes fast so we have another three or four minutes.
So this is my circuit.
And let's say all I care about is finding out i1.
That's all I care about.
And what I'm going to do is I'm going to box this up and see if I can replace that with its Thevenin equivalent.
So I'm going to box that up.
What I'm saying is that I'm going to box it up and replace it with this Thevenin equivalent.
I don't know what Vth and R are at this point.
I'm just calling it Rth for fun.
I don't know what these two values are, but if I knew what these two values were I can determine I really trivially as follows.
I can get i1 as simply V minus Vth divided by R1 plus Rth.
So if I knew Vth and Rth, I can write down i1 by inspection in that manner.
So next, finally, how do I get Vth and Rth?
You get Rth by looking at this network and shutting off all the voltage sources and measuring the resistance there.
So I short my voltage source, that's R1.
Oops, wrong way.
I need to look this way.
So looking this way, that's what I get.
So what's Rth?
Rth is simply R2.
So I have opened my current source.
Similarly, for Vth, remember all I want to do is look at the two nodes, step back, put a voltmeter there, measure the voltage, that's my open circuit voltage.
So the way I do it is I take the circuit and simply measure the voltage there.
That's R2.
That's my current capital I.
And I simply want to measure the open circuit voltage here, which is what?
Just simply if I stand back and I kind of gingerly measure the voltage here without disturbing anything, I simply get IR2.
So Vth is IR2 and Rth is R2 and here is the formula for the current in this branch when I apply a voltage source and a resistor R1 to this little circuit here.
OK, let's pause and let me summarize this in about ten seconds.
I had this circuit here.
I wanted to find out i1.
So what I said I'd do is take this complicated mess, well, it's not a complicated mess but assume it is, and replace with it a resistance Rth got by turning off all the sources.
And the voltage in series, Vth, which I get simply by pulling this thing out, taking my input, this part out and simply measuring the open circuit voltage out there, Vth.
And then I replaced the whole network with this new network that they call the Thevenin network, and voila, I get the answer in a second.
So today we are going to talk about another process of lumping or another process of discretization what will lead to the digital abstraction.
So today's lecture is titled "Go Digital".
So let me begin with a usual review.
And so in the first lecture we started out by looking at elements and lumping them.
For example, we took an element and said for the purpose of analyzing electrical properties let's lump this element into a single lumped value called a resistor, R. And this led to the lumped circuit abstraction.
The lumped circuit abstraction says let's take these elements, connect them with wires and analyze the properties of these using a sort of analysis technique.
So a set of a methods.
We've looked at the KVL, KCL method.
Another example of a method we looked at was the node method.
And of this category there is one method you should remember, which you can apply to every single circuit and it will simply work, is the node method.
For linear circuits other methods also apply, and these include superposition, Thevenin method, and in recitation or in your course notes you would have looked at the Norton method.
So that's what we did so far.
So this is a toolkit.
So now you have a utility belt with a bunch of tools in it, and you can draw from those tools.
And, just like any good carpenter, you know, the carpenter has to cut a piece of wood.
He could use a chisel.
He could use a saw.
He could use an electric saw.
And the reason you pay carpenters $80 an hour in the Boston region is because they know which tool to use for what job.
So what we'll learn today is, so this was one process of discretization.
We discretized matter.
This gave us the discipline here that we decided to follow, lumped matter discipline, that moved us from Maxwell's equations into this new playground called EECS.
Where all elements looked like these rinky-dinky little values like resistors and voltage sources and so on.
What we'll do today, if that wasn't simple enough, let's simplify our lives even further.
What we're going to do is lump some more.
So what else can we lump?
We've lumped matter, so all matter is taken care of.
So what can we lump to make life even easier?
When in doubt, if things are complicated, discretize it or lump it, right?
So what do you think?
What we will do today is lump signal values.
So we'll just deal with lumped values.
And this will lead to the digital abstraction.
And the related reading is Chapter 5 of the course notes.
So before we do this kind of lumping, let me motivate why we do this.
One reason is to simplify our lives, but there is no need to just go around simplifying things just because we can.
Let's try to see if there are other reasons motivating the digital abstraction.
So what I would like to start with is a simple example of a analog processing circuit that you should now be able to analyze.
So I'm going to be motivating digital.
So let's start with an analog circuit that looks like this, two resistors, R1 and R2.
And what I'm going to do is apply a voltage source here, V1, apply another one here, V2, and make this connection.
And let me call this voltage V nought and call this my output.
This voltage with respect to ground node, rather than drawing this wire here, I often times draw a ground here and simply throw ground wherever I want.
This symbol simply refers to the fact that the other terminal is taken at the ground node.
So here is my V nought.
Now, let's go and analyze this and see what it gives us.
In this example, V1 and V2 may be outputs of two sensors, maybe heat sensors or something like that.
This is a heat sensor on that side of the room and this is a heat sensor on this side of the room.
And I pass their signals through two resistors and I look at the voltage there.
So by now you should be able to write the answer V nought, or the value V nought almost by inspection.
Just to show you, let me use superposition.
When you see multiple sources, the first thing you should think about is can I use superposition to simplify my life?
And let me do that.
V nought here is the sum of two voltages, one due to V1 acting alone and one due to V2 acting alone.
So what's the voltage here due to V1 acting alone?
To find out that I short this voltage, I zero out this voltage and look at the effect of V1.
So the effect of V1, if this were shorted out, is simply V1 x R2 / R1 + R2.
This is now a voltage divider, right?
A voltage V applied across two resistors and the output taken across one resistor.
So that's this value.
Then I could do the second part.
To look at the effect of V2, what I will do is short this voltage and look at the effect of this.
Now, this voltage is across this resistor divider.
And so I get R1 / (R1 + R2) here.
So you'll notice that for something like this, if I had applied KVL and KCL of the node method I would have gotten a bunch of equations, but here I wrote it just by inspection.
You should be able to look at circuit patterns like this and write the answers down very quickly.
Let's say if I chose R1 to be equal to R2 then V nought would simply be (V1 + V2) / 2.
So if these two values were equal, I simply get the output, the average of the two voltages.
So this guy is an adder circuit.
It adds up these two voltages.
But more precisely it's an averaging circuit.
It takes two voltages and gives me the average value.
Now, if you have two sensors in the room, you might think of why you want to take that average value to control the temperature of the room.
But suffice it to say that V nought is the average of the two values.
So let me show you a quick demo of this example and then look at what the problems are with this example.
So let's say, as one example, I applied a square wave at V1, which is the top curve, the green curve, and I applied a triangular wave at V2, that's the second one.
As you expect, the output is going to be the sum of the two voltages scaled appropriately.
So notice that I have a square wave with a superimposed triangular wave on top.
And I can play around.
What I could do is change the amplitude of my wave form here.
And, as you notice, the amplitude of the output component also changes accordingly.
So this is one simple example of an adder circuit, and the two wave forms get summed up and I get the output.
So I'll switch to Page 3.
Let me just draw a little sketch for you here.
Here, what I showed you was I had a triangular wave coming on one of these inputs and I had a square wave on the other one, and the output looks something like this.
OK?
No surprise here.
This is a simple analog signal processing circuit which gives me the average of two wave forms.
Now, let me do the following.
Often times I may need to look at this value some distance away.
So let's say this person here wants to look at the value.
So I bring this wire here.
And I also bring the ground connection and I look at it.
I look at this value here.
And when I have a long wire I can get noise added onto the circuit.
So let's say a bunch of noise gets added into the signal there.
And what I end up seeing here is not something that looks like this but something that looks like that.
That's not unusual.
And the problem with this is now when I look at this, if I'm looking to distinguish between, say, a 3.9 and a 3.8, it's really hard to do that because my noise is overwhelming my signal.
I have a real problem, a real problem here.
Noise is a fact of life.
So what do we do?
This is so fundamental.
Large bodies of courses in electrical engineering are devoted to how do I carefully analyze signals in the presence of noise?
You'll take courses in speech processing that look at clever techniques to recognize speech in the presence of noise and so on and so forth.
One technique we adopt that we'll talk about here, which is fundamental to EECS, is using the digital abstraction.
Let me show you how it can really help with the noise problem.
So the idea is value lumping or value discretization.
Much like we lumped matter, we've discretized matter into discrete chunks, let's discretize value into two chunks.
Let's simply say that now I'm going to deal with two values and I can, say, call them high, low.
I have a bunch of choices here.
I may call it 5 volts and 0 volts.
I may call it true and false.
What I'm doing is I'm just restricting my universe to deal with just two values, zero and one.
This is like dealing with a number system with only two digits.
And these are zero and one.
So what I've now done is I'm saying that rather than dealing with all possible continuous values, 0.1, 3.9999 recurring and so on and so forth, what I'm going to do is simply deal with a high and a low.
Dealing with this whole continuum of numbers is really complicated.
Let me simplify my life and just postulate that I am going to be looking at high and low.
Whenever I see something I'll look at it and say high or low, is it black or white, period.
There's no choice here, just two individual values.
So that sounds simple, and nice and so on, but what's the point?
What do we get by doing that?
Let's take our example.
Let's take what might be a digital system.
Let's take a digital system and let's say I have a sender.
Much like I sent a signal value a long distance, let me have a sender, and I have a ground as well and here is a receiver.
This symbol simply says that both of them share a ground wire.
So the sender and a receiver.
And what I'm interested in doing, the sender is interested in sending a signal to the receiver.
And in the digital system, the way I would send a digital signal is all I can use is ones and zeros, OK?
So let's say the sender sends something like this.
The sender wants to send a value.
This is my time axis and this is 2.5 volts, this is 0 volts and this is 5 volts.
My sender has some agreement with the receiver and says I'm just going to be sending to you low values and high values.
And this signal here would correspond to "0" "1" "0".
It's a symbol.
That's why I have input zero in quotes there.
We'll go into this in much more detail later, but for now suffice it to say that I'm sending a set of signals here "0" "1" "0".
This simplistic scheme will not work in many situations but go along with this for a few seconds.
So I send the signal sequence "0" "1" "0" out here.
And notice that there is a high and a low.
And the agreement the sender and the receiver have is that, look, if you see a value that's higher than 2.5 volts that's a high.
If you see a value below 2.5 volts in the wire that's a low.
And I'm going to send a 0 volt and a 5 volt from here.
So now at the sending site let's say I don't have any noise in this system.
Let's say this is my Vn, some noise being added.
And let's say Vn is 0.
Then in that case I will receive exactly what is sent "0" "0" 5, 2.5, 0 volts.
And this is time.
Nothing fancy here, right?
My receiver receives a "0" "1" "0".
Now, the beauty of this is that now suppose I were to impose noise much like I had noise out there and Vn was not 0.
Rather Vn was some noise voltage, let's say 0.2 volts peak to peak.
Let's say that simply got superposed on the signal.
In which case what do I get?
What I end up here with is a signal that looks like this.
So the receiver gets that signal because a noise is added into my signal and that's what I get.
But guess what?
No problem.
The receiver says oh, yeah, this is a 0 because the values are less than 2.5, this is a 1 and this is a 0.
"0" "1" "0".
So here my receiver was able to receive the signal and correctly interpret it without any problems.
So because I used this value discretization and because I had this agreement with the receiver, I had better noise immunity.
Consequently, I had what is called a noise margin.
Noise margin says how much noise can I tolerate?
And in this situation, because the sender sends 5 volts and 0 volts, the 5 volts can creep all the way down to 2.5, I'll still be OK.
Similarly, 0 could go all the way up to 2.5, I'd still be OK.
So in this case I have a noise margin of 2.5 volts for a 1 and similarly 2.5 volts for a 0, because there are 2.5 volts between a 0 volt and 2.5.
So notice that I have a nice little noise margin here, which simply is the English meaning of the term there is a margin for noise.
And even though I can change the signal value by up to 2.5 volts, the receiver will still correctly interpret the signal.
So I've decided to discretize values into highs and lows.
And because of that, if all I wanted to do in life is send highs and lows I can send them very effectively.
There are many complications, but if all I care about is sending highs and lows I can send it with a lot of tolerance to noise.
So many of you are saying but what about this, but what about that?
There are lots of buts here.
And let's take a look at some of them.
If you look up there.
What I ended up doing was creating a design space that looked like this.
This is on Page 6.
What I did was I said with a range of values from 0 to 5, what I'm going to do is at 2.5 I drew a line and I said as a sender if you wanted to send a 0 then you would send a value here.
And if you wanted to send a 1 you would send a value here.
Similarly, for a receiver.
And if the sender sent a value all the way up in 5 volts that was the best thing, but technically the sender could send any value between 2.5 and 5.
And if there was no noise then the receiver could correctly interpret a 1 if it was above this and 0 if it was below this.
The problem with this approach really is that if I allow the sender to send any value above 2.5 all the way to 5 then there really is no noise margin in this situation.
OK?
Because if I allowed the sender to send any value between 2.5 and 5 then what if I have a value 2.5 for a 1?
Then I may end up getting very little noise margin on the other side.
Worse yet, what if I get a value 2.5?
That's a much worse situation.
What if the receiver receives a value of 2.5?
Now what?
What does the receiver do?
The receiver cannot tell whether it's a 1 or a 0.
The receiver gets hopelessly confused.
So to deal with that, I'm going to fix this, what I'm going to do is the following.
Switch to Page 7.
What I'll do here is to prevent the receiver from getting confused, if the receiver saw 2.5, what I'm going to do is define what is called "no man's land".
I'm going to define the region of my voltage space called the forbidden region.
And what I'm going to do is, say, let's say I defined it as 2 volts, 3 volts and 5 volts, 0, 2, 3 and 5.
With my forbidden region, if I have a sender then I tell the sender you can send any value between 3 and 5 for a 1.
And you can send any value between 2 and 0 for a 0.
To send the symbol 0, I can send any voltage between 0 and 2, and similarly for 1.
At the receiving side, if I see any value between 3 and 5, I read that as a 0, and any value between 0 and 2 I read that as 2 volts.
So I may label this value VH and label this threshold VL, so there's a high threshold and a low threshold.
So this solves one problem.
Now the receiver can never see a value in the forbidden region.
Now, I can stand her and pontificate and say, oops, that's a forbidden region, thou shalt not go there.
But what if I get some noise and a value goes in there?
In real systems values may enter there.
But what I'm saying, so this is the beauty of using a discipline.
Let me use my playground analogy.
This is my playground.
We got into this playground using the discrete matter of discipline, the playground of EECS, but in that playground some region of that playground deals with just high and low values.
I further restrict the playground and I say I'm only going to focus on that playground in which all signal values have a forbidden region.
All senders and receivers adhere to a forbidden region.
And if there is any signal in this space, in the forbidden space then my behavior is undefined.
I don't care.
You want to go there?
Sure.
I don't know what's going to happen to you.
Now, we're engineers, right?
So we've disciplined ourselves to play in this playground.
It's like I tell my 9-year-old, don't go there, right?
And of course he wants to go there.
He says what will happen if I go there?
And the answer here will be undefined, OK?
Something really bad could happen to you.
I don't know what it is but something really bad, you know, a lightening bolt or who knows what, but something really bad.
And you as a designer of a circuit can, let's say you were Intel.
Intel designs its chips.
And let's say Intel decides to play in this playground and there is a forbidden region.
So Intel says oh, it's really easy for me if in the forbidden region the chip simply burns up and catches fire, we'll sell more chips.
That's fine.
Whatever you want.
The key here is that all I'm saying is that I am going to discipline myself into playing in this playground and that's where I will define my rules, and you stay within the boundaries and all the rules will apply.
It's called a "discipline."
You're disciplining yourselves to stay within it.
There's no logic to it.
It's just a discipline.
Just do it and you'll be OK.
When we look at practical circuits and so on, we have to address the issue of what happens when things go in there.
But let's postpone that discussion.
For now I've solved one of my problems, which is, the previous problem was what does a receiver do if it saw a 2.5?
Now it can't see a 2.5.
But then the receiver asks, Agarwal, but what if I see a 2.5?
I can tell the receiver you can do whatever you want to do.
You can stomp it.
You can squish it.
You can burn it.
You can chuck it.
Whatever you want.
It's up to you.
Do whatever you want.
You won't see a value.
If you do, do whatever you want.
It's undefined.
That works.
So you, as the receiver designer can do whatever you want when you see a 2.5.
You can say yeah, I'll just put out a 1 if I see a 2.5 or a 2.6.
I'll just do something.
No one cares.
So this is pretty good.
This is pretty good.
We still have a problem, though.
Do people see the problem here?
This still doesn't quite work.
If Intel did this, instead of your laptops failing and blue-screening every hour they'd be doing it every millisecond.
So the problem is this discipline have allowed the sender to send any value between 3 and 5 as a 1.
And any value between 3 and 5 at the receiver is treated as a Do you see where the problem is?
Yes?
The sender sends a 1.99 and the noise pumps it into forbidden region.
Exactly.
So the sender says it's legitimate, I'm Intel.
They've told me stick to 0 and And Intel parts will be sending to values between 0 and 2.
And Motorola parts, which are receivers, you know they have to receive 0 and 2.
So Intel can send the value, They can because it's 1.9 out of 2.
It's legal.
This way I can make really cheap parts.
But now the problem is that even the smallest amount of noise will bump it into the forbidden region, and so therefore this one has a problem.
And the problem is that this one offers zero noise margin.
There is no noise margin.
There is no margin for noise in the discipline.
All right, back to the drawing board, folks.
Switch to Page 8.
Let's get rid of all this stuff and go back to the drawing board.
OK, so what do we do now?
How about the following?
How, about as before I say, as a receiver, if you see a value between 3 and 5 you treat that as a 1 and a value between 0 and 2 you treat that as a 0.
No difference.
So as a receiver same as before.
But now what I do is I hold the sender to tougher standards.
I hold the feet of the sender to the fire and say you have to adhere to tougher standards.
So what I'm going to do is hold the sender to tougher standards, maybe four walls.
That is tell the sender that if you want to send to 0 or a 1, for a 1 you have to send a value between 4 and 5, and for a 0 a value between 0 and 1.
Sender is now held to tougher standards.
This is what my chart looks like.
So now I do have some noise margin.
Can someone tell me what is the noise margin here for a 1?
1 volt.
And the reason is that the lowest voltage a sender can send is 4 volts, OK?
If the 4 leaks down to 2.99 that's in the forbidden region, I'm in trouble.
This is my forbidden region here.
And 2.99 is in the forbidden region.
I'm in trouble.
So notice that the lowest value that the receiver can receive is 3 volts.
So if I sent the 4 and sent this over a long cable to you, the value can be beaten up by noise to such an extent that you may begin receiving 3s but nothing lower than a 3.
So this is a noise margin, 1 volt.
Similarly, for a 0 the noise margin is also 1 volt.
So let me label these.
There are four important thresholds here.
This threshold is called VOL.
V output low.
These have special meanings.
This threshold here is called VOH, V output high.
This threshold here is called V input high and this threshold here is called V input low.
So VOH simply says that senders must send voltages higher than VOH.
Receivers must receive values higher than VIH as a 1.
So these four thresholds together give you your threshold.
For the sender gets 2.5, what does sender do?
It could do that.
So, in that case, you can do that.
If all you want to do is have one value here then what you have is an infinitesimal value here for the forbidden region.
That's fine.
It's up to you to design it that way.
You can.
But it turns out that when you design circuits, when we see some examples in the next lecture it turns out to be fairly practical and easy to do it this way.
But, again, these are design choices.
If I'm Intel, Intel wants all its parts to work together.
So parts that follow a common discipline can work together, right?
Because senders will send values, receivers will receive these values here, so it will simply work.
So the noise margin for a 1 here is simply VOH minus VIH and the noise margin for a 0 is VIL minus VOL.
VIL minus VOL is the noise margin for a 0.
So what do we have here?
What we have here is a discipline that we've agreed to follow where senders are held to a tough standard and receivers are held to a different standard so that I allow myself some margin for error.
And it's up to you as a designer to choose ranges for the forbidden region.
Now, you may say that I want to make my forbidden region as small as possible.
But you will see in practical circuits it's very hard to achieve that.
Practical devices that you get, they have a natural region that gets very, very hard to break apart, and that tends to establish what that region looks like.
So to continue with an example here, I may have the following voltage wave form for a sender.
So I have some sender, I have a sender here.
I have VOL, VIL, VIH, VOH and some other high voltage.
And then, as a sender, if I want to send a "0" "1" "0" then I send a 0.
I have to be within this band.
And then for a 1 I have to be within this band.
So this is an example of, say, "0" "1" "0" "1".
And at the receiver -- Let's have VOL, VIL, VIH, VOH.
So at the receiver, I interpret any signal below VIL as a 0.
So I may get some signal that looks like this.
And I'll still interpret that as a "0" "1" "0" "1".
So to summarize here, this discipline that forms the foundations of digital systems is called "a static discipline".
The static discipline says if inputs meet input thresholds -- So if an input to a digital system meets the input thresholds then outputs will meet, or the digital system should ensure that the outputs -- Output thresholds.
So this means that if I have a system like this then if I give it good inputs.
And by giving it good inputs I mean for 1s I have signal values that are greater than VIH and for 0s signal values which are less than VIL.
These are valid inputs.
So if my inputs are valid, that is below VIL for a 0 and above VIH for a 1 then this digital system D will produce corresponding outputs that follow output thresholds.
For a 1 it will produce outputs that are greater than VOH and if it needs to produce a 0 it will produce outputs that are less than VOL.
So notice that there is this tough requirement in digital systems that for the inputs, I should recognize as a 1 anything higher than a VIH.
But if I want to produce a 1, I have to produce a tough 1 like a 4-volt 1.
So there is a discipline that all my digital systems must follow, and that discipline is called a static discipline.
So static discipline encodes the thresholds, encodes four thresholds that all digital systems must follow so that they can talk to each other.
So if Intel and Motorola want to make parts that are compatible with, say, Pentium 4 devices then they will all talk over the phone or something and agree on a static discipline.
We will say that, all right, all my peripherals will follow a static discipline with the following volted thresholds.
And this way parts made by different manufacturers can interoperate and still provide immunity to noise.
Yes.
Question?
Absolutely.
There are many constraints on how you as a designer choose the noise margin.
As a designer you want to make your noise margin as large as possible.
The larger the noise margin the better you can tolerate noise which is why, how many people have heard of some devices called rad hard devices, radiation hard devices?
Some of you have.
There are a bunch of devices.
Different manufacturers make different kinds of devices for different markets.
For consumer markets they use parts which may have relatively poor noise margins because consumers can tolerate more faults.
But if you're building devices for, say, the medical industry or for spaceships and so on, you need to be held to a much, much tougher standard.
So for those devices you may end up having much, much tighter bands in which you have to operate so you have a tougher noise margin.
So that leads us to, given these sort of voltage thresholds, we now move into the digital world.
And in the digital world we can build a bunch of digital devices.
The first device we will look at is called a combinational gate.
A combinational gate is a device that adheres to the static discipline, Page 11, and this is a device whose outputs are a function of inputs alone.
So I can build little boxes which take some inputs, produces an output where the outputs are a function of the existing inputs.
And this kind of a device is called a combinational gate.
And I can analyze such devices for the kinds of things that I would like to do.
Before I go into the kinds of devices I'd like to build, let's spend a few minutes talking about how to process signals.
How to process digital signals, Page 10.
So notice that you have two values, 0 and a 1.
So devices like my combinational gate, for example, can only deal with 0s and 1s.
So I have to come up with some kind of a mathematics or some kind of a set of processing that can work with 0,1 values.
So 0,1 map completely natural to the logic true and false.
So I can borrow from logic and use true and false to do my processing of signals.
So if all I care about is processing logic values, 0s and 1s, trues and falses then that's all I need.
I can also use numbers.
How do I represent a number?
3.9 which is 0s and 1s.
It turns out that this is a whole field in itself.
You'll hear more about this in recitation.
Let me also point you to the last section of the course notes, Chapter 5.6 I believe, that talks about how to represent numbers.
The basic insight is much like you can represent arbitrary long numbers with the digits 0 through 9 in the same way, but concatenating digits you can represent arbitrary long numbers with 0-1-1-1-0-0 and so on.
So you can have a whole sequence of digits and you can build a binary number system.
So you can read A&L Section 5.6, I believe.
It's the last section for numbers.
And you will also discuss this in your recitation tomorrow.
Let me spend some more time talking about Boolean logic, two-valued logic, and how to process these systems.
So one way of processing it is using logic statements of the following form.
If X is true and Y is true then Z is true, else is Z false.
So this is a logic statement.
It says if X is true and Y is true then Z is true, else Z is false.
So I can process this with 0s and 1s, trues and falses.
And I do this all the time so I have a succinct notation for this.
I express this as Z is X anded with Y. X and Y is Z.
So Z is true if X is true and Y is true.
A shorthand notation for this is just a dot.
And a circuit notation for this is called an "AND gate".
That's a little circuit.
I haven't told you what's inside it.
It's an abstract little device called an AND gate which takes two inputs, produces one output Z where the output is related to the inputs in the following manner.
That's a little device called an AND gate.
I could also represent logic in truth tables.
And truth tables simply enumerate all the values and the corresponding outputs.
Inputs can be 0-0-0-1-1-0 or 1-1.
For an AND system output is 1, only if both are ones, it's a 0 otherwise.
So that's a truth table for AND gate.
So from 0s and 1s we deal with logic and we create devices like the AND gate to process digital signals.
And what we will do is look at a whole bunch of little symbols like this, like the AND gate to process our input signals.
And these devices might look like other functions like OR gates and so on.
Let me show you a quick demo.
What I'm going to show you is a signal feeding an AND gate.
And one signal is going to look like this, and my signal Y is going to look like this.
So you expect a processed output.
So 1-0-1-0-1-0-1.
And the output is simply going to be -- This is my time axis going this way.
It is going to be an AND-ing of these two signal values like so.
What I'm also going to show you is I'm going to superimpose noise on this wire.
I'm going to superimpose noise on the wire, and what I want you to observe is the output of this digital gate.
The output will stay exactly like this, even though I impose noise.
The ultimate test.
So stay right there.
Let's do this demo.
Give me a couple of seconds.
If you look at the signal up there, look at the middle wave form, and I'm imposing let's have a digital system in a noisy environment like a lumberyard, for example, or chopping a bunch of trees in my backyard and building digital systems on the side.
And if I have my buddies revving up chainsaws superimposing noise on my second input, but look at the output.
And just to show that I'm not bluffing here, what I'll do is I'll pass the noise through and make the noise larger.
And you'll notice that when the noise begins to surpass the noise margins the output begins to go berserk.
Watch.
Can you increase it gradually?
Notice that as I put in a lot more noise then the output begins to go berserk, but as long as my input is within the noise margin my output stays perfectly stable.
So that's the "Intro to Digital Systems".
You'll see numbers in recitation.
And we'll see you at lecture on Tuesday.
All right.
Good morning.
Let's get started.
So the last lecture we showed you how to go digital.
The fact that going digital had some key benefits for us.
And what we'll do today is go inside the digital gate.
Let's do a quick review.
We began life by observing nature.
We said those Maxwell's equations are tough.
Let's simplify our lives by discretizing or lumping matter.
So we got the lumped circuit abstraction.
Then we had this noise problem here.
In order to be able to handle that let's do some more discretization, some more lumping.
So we said let's discretize values and deal with two levels, a high and a low.
That's where the binary voltage levels come up, a high level and a low level.
And then we said that in discretizing it we have to make some assumptions.
We have to impose some constraints on ourselves.
Just as with the lumped matter discipline, we imposed a couple of constraints in going from the continuous matter world to a lumped matter world.
Similarly, we have to impose some discipline on ourselves, some constraints on ourselves in going from the continuous value regime to the digital value regime.
And that discipline is called the static discipline.
And what the static discipline says is that if you have senders and receivers in a digital system then they all need to adhere to some standard.
If I was a sender I had to adhere to some tough output standards.
I had to be sure to shift values that exceeded some high voltage threshold.
And if I was sending a low value I had to make sure my values were lower than some output low voltage threshold.
Similarly, if I was the receiver then I had to guarantee to recognize as a one all voltages that where above some input high voltage threshold.
And similarly I had to guarantee to recognize as a zero voltages that were below some input low voltage threshold.
So provided senders and receivers in a system adhere to these voltage levels, to this discipline then they would all very comfortably work correctly in a digital system.
Then we also said that once you deal with such values, one you deal with digital values we can now postulate a bunch of digital elements that process these values in a manner very reminiscent of our analog circuits where we get analog signals.
And you've already learned how to process analog signals.
You've learned about resistor dividers and so on and so forth.
You feed in an analog signal and you get an output analog signal as well.
Now, here the resistor in the analog domain, elements like resistors and voltage sources were the symbols that you dealt with.
Here, in the digital domain, the primitive elements that we will be using are called gates.
As one example, this is called the NAND gate.
So we looked at the AND gate in the previous lecture.
This is an example of another gate called the NAND gate.
The NAND gate has the following truth table.
Our two inputs A and B and this output C. And the NAND gate works as follows.
The output -- In English I can describe its properties as the output is a high at all times when at least one of these inputs is a low value.
So it's high whenever at least one input is a low.
So it's high here.
It's high here.
Oops, it's high here, high here.
And when, oops.
And when both inputs are a high the output is a low.
This is a NAND gate.
Notice that these are exactly complimentary to the AND gate.
The AND gate outputs were 0-0-0-1.
And the AND gate symbol looked like this.
In general, notice that this little bubble here, it's called a bubble.
That bubble implies a negation, an inversion.
So we take the AND gate, invert the output and negate the output and you get the NAND gate.
So these elements are combinational gates.
And in combinational gates they adhere to two properties.
One is that they must satisfy the static discipline.
All the systems, all the elements in our repertoire in the digital domain need to satisfy the static discipline.
And the properties of a combinational gate are that its outputs are a function of inputs alone.
In other words, it doesn't store any state or doesn't store any history inside it.
You can figure out its output just by looking at the inputs at that instant.
Think of it as a completely transparent entity where its output reflects some function of the inputs at every instant of time.
So I'll show you an example of a digital circuit.
So much as I could interconnect resistors and voltage sources and current sources to build analog circuits, I can now build digital circuits using primitive elements such as these.
So, for example, I could build a simple circuit that looked like this, two inputs A and B here, I get an output.
And I feed that to another NAND gate with another input C. This device is called an inverter.
The inverter simply flips the sense of the input.
So if C is a 1 the output is a 0, if C is a 0 the output becomes a 1.
It's an inverter.
It simply inverts its input.
Yet another primitive device.
And this is my output D. So there are three gates in this design.
And I can quickly write down what the output looks like using some very simple Boolean algebra or dealing with Boolean values here.
So for AND gate the output is A and B.
Remember dot is a short form for and.
But there's a negation, inversion, so represent inversions with a bar.
So my output is A dot B bar.
There is a C here.
So this is my output C bar.
And this is a NAND gate.
So it takes one input A dot B.
It takes the second input C bar and ANDs those and inverts them.
So that's the output.
So there are three gates in this example.
So you can think of building very complicated circuits containing large numbers of gates.
In fact, the microprocessors that you use in your laptop contain a large number of gates.
Can someone guess how many gates are in the Pentium IV, roughly?
Approximate, how many?
How many gates in a Pentium IV?
40 million.
100 million.
In the Pentium IV you have on the order of 20 million gates.
20 million gates in the Pentium IV.
And life begins in 002.
Here you learn about onsies and twosies, and in the real world you will be dealing with tens of millions of gates.
But this is for the Pentium IV.
My research group at Laboratory for Computer Science built a chip called the Raw chip.
And this chip has 3 million gates.
And so there are several undergraduate students involved in this project in their third year, and they're beginning to deal with millions of gates.
So the key thing to remember is that 002 provides the foundations where you make the switch from the analog signal to the digital signal or from continuous matter to lumped matter.
And learn about the foundations of these primitive elements.
And by the end of this course you will begin dealing with small systems, analog systems that contain on the order of 10 to 20 primitive elements.
You will also begin dealing with small digital systems that contain tens of gates.
In your final project you will build a mixed signal circuit involving an audio playback system.
You will have digital data stored in a memory chip and you will build a circuit to extract that data, filter it and then convert it to the analog domain and then play it on a set of speakers.
And that has on the order of about 50 to 100 primitive elements.
So by the end of 002 you will have learned to deal with hundreds of elements.
And then you will take other courses like 004 and so on where you will then make the leap to learn further abstractions that will take you from subsystems to systems with millions of gates.
So the key is to manage the complexity of dealing with millions of gates it's all about abstractions.
You have to build abstractions and double abstractions so you can deal with complexity.
So the rest of EECS will take you from three gates to 20 million gates and software systems that operate on 20 million gates or whatever.
So there is still a ways to go.
Lorenzo, our friend has gone to bring a demonstration that we forgot to bring today.
That will show you that little digital circuit in a mock up form.
So what's today's lecture about?
Today's lecture is going to be about what's inside a gate?
How to build a gate.
Once you build a gate you can then put millions of them into computer systems or analog systems or other sorts of systems.
And what we'll do here is understand what's inside this abstraction.
This is an abstract element that looks like a little circle and a line with some stuff inside it, with some properties.
But someone's got to build that.
It doesn't come from nature.
You don't go and harvest gates from trees, you got to go build that, and someone has got to do that.
So what to learn here is how do we go about building a gate?
And here you will see practically how do you deal with voltage thresholds that satisfy a given static discipline?
So before I jump into building a gate, let me try to build up some intuition.
As is my usual practice, I'd love to get you to build some intuition as to how to build a gate.
And then we'll go through the mechanics of doing it.
So to build intuition, let me show you an analogous situation in fluids.
So let's say I have a cauldron of water.
This is like a power supply.
And I need to feed this fluid down at some output source.
And what I do in the middle is put in a couple of taps, faucets, all right?
And so what do these guys do?
Under what condition do you have fluid flow out of the tube at the other end?
You will have fluid flow if -- So let me call this A and B.
If A is on and B is on then C has water.
Otherwise, if both A and B are not on then C has no water.
So this is already beginning to sound like a AND gate, correct, where you get water only if A and B are both turned on.
So we're going to use this insight, a stream of some flow and I put things to obstruct the flow.
And when both the obstructions are lifted I get the output.
I want to use that intuition to build an AND gate.
Similarly, I could build a system that allows me to build the following structure -- So in this scenario let me call this -- -- the signal of A and B here.
And in this situation under what conditions, provided the power supply has water, under what conditions do I get water out?
In this situation, it is I get water if A or B are turned on.
So I don't need to turn both A and B on.
If either one of them is on, I'm going to get fluid flow here.
So this will help us build the inside to build the OR gate.
So that's an analogy involving items we see in everyday life.
Let me now move into the electrical domain.
In the electrical domain my analogy would be something like this.
Let's say I have a power supply and I have two switches A and B.
And I build a little circuit that connects this voltage source across the bulb using a couple of switches.
In this case, the bulb is on if both switches A and B are on.
My bulb turns on.
If I switch either one of them off my bulb turns off.
So notice that I can begin implementing things like this if I had this element.
I had sources already.
I know how to deal with bulbs.
I model them as resistors.
So I need to do something about this new element called a "switch".
So let me build an abstract device.
I'll tell you how to do that in real life in a second.
So if I had the switch I could build things like this.
I could put switches in series in a circuit and get myself something that looks like a AND function.
So let me go ahead and build an equivalent circuit for a switch.
So the switch has a couple of terminals here and I have a control.
Switches have a control and they have a pair of terminals.
And the equivalent circuit for this looks like this.
This is for my switch.
So when control is a 0.
Then my switch is open to give me an open circuit in the circuit that I've shown you here.
And, by the same token, if my control is a 1 then -- -- I have a connection between in and out.
And this is a short circuit.
So, in other words, if my switch has 0 at its control, I'll talk about how to get that, I have an open circuit, and if it's a 1 then I have a short circuit.
This is a switch going on and off.
Now, in traditional switches mechanical pressure is my control signal.
If I apply mechanical pressure my switch could turn on.
And if I take away the mechanical pressure then I could get an off situation.
So let's for now imagine that we have a switch.
I still haven't told you how I am going to get a switch in real life.
Let's imagine you have a switch.
It's a three terminal device.
There's a control thingamajig coming in.
Input and an output.
So let's build the following little circuit containing a switch.
So what I'm going to do, I will take a resistance RL and plug it in here.
And connect my power supply like so.
So the little circuit that I build has a resistor.
And I connect the switch in this pattern and I get a VS. Lorenzo, you can set that up there if you'd like.
No problem.
So I get a VS here.
Now, a couple of lectures ago I told you that 6.002, and for that matter, 004 and many of our other courses deal with combinations of elements.
And we often deal with the same kinds of combinations again and again and again.
We see the same sorts of patterns happening, and we need to begin to learn to identify these patterns.
This is an incredibly common pattern.
You'll see this pattern more times in 6.002 than any other pattern, I promise you.
A power supply connected to a resistor and connected to a couple of terminals of some interesting device.
I promise there will be at least one such pattern on the quiz, for example.
These patterns are incredibly common.
So let's take a look at the interesting properties of this pattern.
Since this pattern occurs so commonly, I am going to create a short form.
I have already created a short form which is this ground node here.
By putting ground 0 all I'm really saying is that there is a wire connecting these two and that's my ground.
So I already have a short form here.
My second short form is when I connect a power supply to a node.
Then what I'm going to do is come up with yet another short form that looks like this, an up arrow with the voltage written there.
This symbol simply says that this node is connected to a power supply with voltage, or a voltage source voltage VS.
So I just have come up with a slightly simpler representation for the little pattern that I have.
Now let's take a look at the properties of this little system.
Let's first look at what happens when C is 0.
When C is 0, let me draw the equivalent circuit for this using the open circuit out there.
That's what I get, OK?
So when C is 0, if VS is a high voltage, let's say 5 volts, what do you expect at the output if C is a 0?
This voltage VS appears at V out because this is an open circuit here.
Remember, RL and this little device form a voltage divider.
But since it's an open circuit its resistance is infinity.
And so therefore in this resistor divider all the voltage falls across this open circuit.
So, in this case, v out is a 1 or a high voltage.
But let's take a look at what happens when C is a 1.
In this situation, I have my RL, that's what I have.
It's a short circuit at the switch and C is a 1.
So what's the voltage v out in this case?
Not surprisingly, since I've shorted this node to ground the voltage at this point is 0.
So if I have low voltage that's corresponding to logical 0s that corresponds to a 0.
So I can build a simple truth table for C and use logical symbols here.
So when C is a 0 I get a high at the output and when C is 1 I get a low at the output.
Have you seen a device that behaves like this so far?
That's a little inverter.
That's the exact behavior of an inverter.
So this thing I've written here is a truth table for an inverter.
So notice with just a simple little switch and a resistor, I have managed to build an inverter.
Before I go on, I guess we have some things to show you.
And let me pause for a couple of seconds and do that.
First of all, what I want to show you is the following idea.
So as I was preparing for this lecture last night I said, now here I am telling the 6.002 gang that you need to learn about analog circuits and resistors and all of that stuff, and you also need to learn about digital systems and all of that stuff.
And I said, because these two are very commonplace and often times they occur together.
So I said well, if I really believe in my own BS then there should be something around me where I can find both of them instantaneously.
So I said let me do the following experiment.
Let me close my eyes and reach out and see what I touch.
So I closed my eyes, reached out, and guess what?
I touched the lonely mouse.
The mouse.
So I said let me see what is in side the mouse.
And if I believe in my BS we should find analog, little components and digital components in there, right?
So let's see what is inside the mouse.
All right.
There we go.
Don't try this at home, as with many other things we do in lecture.
Come on.
Show me what I want to see.
OK, here we go.
Not bad.
Let me show you what we have here in this poor shattered mouse.
That's my finger, silly.
You should recognize this little resistor here.
That thing with the little bands, oh, here we go.
We'll use this.
That's a resistor.
And you'll see capacitors in about four weeks.
That's a capacitor.
And there is a digital IC here.
That's a digital IC.
That contains a bunch of gates inside it.
So this mouse has not made a liar out of me.
So what I just showed you was a little device that we use in everyday life that has both analog components and digital components.
A large number of devices that we use in daily life are this way.
You can do the same thing to your laptop.
You could go try it out.
And you will find a bunch of analog components and a bunch of digital components.
And you really, really need to understand the whole caboodle here.
Let me show you a fun little demo involving gates.
Now, I want you to be very careful here.
Lots of caveats here.
If your grandmother asks you how big is a gate don't say this big.
This is how big gates used to be, I would say, when they were first invented.
When they built gates out of discrete vacuum tubes and so on, this is how big a gate used to be.
This is roughly that big.
Today in a chip, in a small VLSI, very large scaled integrated circuit in a chip, which is about 1 cm on the side, how many gates do you think I can fit in a thumbnail sized chip?
Any guesses?
With today's technology, how many gates can I fit on a chip?
It has to be more than a million because I just told you that Pentium IV was 20 million and that was a year ago.
How many?
40 million is a good guess.
So on the order of 40 to 80 million gates in a 1 square centimeter.
Intel just announced that they will be shipping a chip containing 1 billion switches.
Remember, this whole thing is a gate, right?
Inverter, a resistor and a switch.
This thing is a switch.
So Intel is going to be shipping something containing a billion of those little elements.
Just keep those large numbers in mind.
So here is a little circuit that I showed you here, A, B, the NAND gate, the NAND gate at the output and the inverter.
So this output A is going to be 1 whenever either A or B is off.
So the output is a 1 in this case when both A and B are off.
I turn A to 1, output is still a 1.
So the moment I turn both of these inputs into a 1, these are 1s, the output goes to 0.
That's behavior for NAND gate.
If I switch any one of the inputs to a 0 the output should go to a 1.
Similarly, for the inverter here, when the input is a 0 the output is a 1.
And when I switch it so should the output.
Now imagine a circuit, a little chip containing billions of these devices.
And just imagine all of these 1s and 0s flying around.
So one simple switch in the input, like a click of a keystroke could actually cause a billion signals in your circuit to be flipping around.
And that causes some fun stuff to happen, which we will learn about a few months from now.
But for now that's a quick show of a little circuit that looks like that.
Let me go back to talking about building other types of gates.
So that was an inverter.
So now you know.
You're almost halfway to being able to build a Pentium IV.
You've come all the way from nature to gates.
And Pentium IV contains 20 million of them so you now know how gates are built.
So that's an inverter.
Let's look at how we can build other forms of gates.
To build another gate let me do this.
How about this pattern?
If I build a pattern like this with A and B coming in here and I put two switches with their inputs in and out, so two switches in series.
Let's write down the truth table for what this looks like.
Let's see.
When A and B are both 0, what should the output be?
These are both off so the output is directly VS which is a high.
When either of these switches is off 0-1 or 1-0.
If either switch is off then this node is cut off from ground.
There is no current flowing here.
So this entire voltage drops across this infinite resistance here, and so I get 1s at the output as well.
If both switches are on what happens?
If both A and B are on then I get a short circuit to ground and my output is a 0.
So can someone tell me what gate this is?
Awesome.
We just build a NAND gate.
This is unbelievable.
Five lectures and you've already come all the way from nature to the primitive building blocks of microprocessors.
It's pretty amazing.
So what about this one here?
What's this?
I haven't told you this before but if an AND gate becomes a NAND gate, this is kind of an OR arrangement, what should an OR become?
NOR.
It's all completely logical.
So you can go home and practice a truth table for this.
A, B and C. I'll just fill in one of the rows.
So in this particular situation, if both A and B are 0, if A is 0 and B is 0, both the switches are off, so it's as if this little sucker here is cut off from ground and VS falls across from C to ground here and the output is a 1, so on and so forth.
So I can build other interesting forms of gates.
So let's say I build something that looks like this.
I build something like this.
You can write the truth table for this or you can look at this and write down the function that this one supports.
Notice that this output here is going to be a high only when both of these are not connected to ground.
And if you stare at it some more the function this one presents, this is my AND function.
Suppose this one didn't exist, that would be my AND function.
But because this one exists that's in an OR configuration and so I get a C. And so because of that I get something that looks like this.
So this is my A dot B, this is my plus because of a parallel here, and ultimately this caused an inversion in this gate.
So the primitive pattern has a generic inversion built into the output.
That is why they commonly end up building NAND gates and NOR gates and so on as the simplest gates.
We don't build AND gates and OR gates.
How can I convert this one to an AND gate?
Anybody?
Put an inverter on the output.
So what I can do is take this little sucker here, put an inverter here and I get an AND gate.
So the real primitives in circuits tend to be NANDs and NORs.
OK.
So the real practical among you should be saying at this point all right, all right, I buy this, if there existed a switch.
I know exactly how to go from nature to building Pentium IVs if there exists a switch.
So that the obvious next step for me is to show you a switch, a physical switch device.
And to introduce a switch device, let me show you a three terminal element.
Remember, the switch has three terminals, an input, output and something called the control, C. So I'm going to introduce a new primitive element called "The MOSFET Device".
MOSFET stands for metal-oxide semiconductor field-effect transistor.
This is shortened to FET or transistor.
Now I'm going to show you that this works like a switch.
And before I do that, in fact, let me do that first.
Then I'll show you something else.
So this device has the following symbol.
It has a terminal called a gate, the drain and the source.
Gate, drain and source.
Three terminals.
This is the primitive element that forms virtually every electronic component built today.
This is the foundation of the universe.
So this little MOSFET device, we can look at how it behaves.
I'll show you this thing on the screen in a second, but this guy behaves very much like this device I was postulating earlier.
Let's take a look at this device on the scope.
To do so let me label some voltages and currents.
So let me label this voltage as vDS.
Let me label this voltage as vGS between the gate and the source.
And let me label the current coming into this node iG.
In this device, the physical device that I'm going to show you, the current going into the gate is always 0.
So iG is always going to be 0 for 6.002.
In real life there is some leakage and so on.
But in 6.002 for now we deal with a very simple abstract model, iG is 0.
And let me label the current here as iDS.
To be correct with the nomenclation, the current into node D should be labeled iD, but because iG is 0 iD flows out through the source as well, so I would simply call it iDS just so that I can show that vDS and iDS are the two voltages and currents that I am going to deal with.
So that's my little device here.
And notice that the source terminal is common.
I use the source both for the control GS and I use the source for the drain as well.
So you can view this as input, view this as out, and you can view this, if you like, as the control abstractly.
So let me show you a plot of how this behaves.
To understand how it behaves, I can draw an equivalent circuit for it.
So in this particular situation, if its behavior is characterized by the voltage applied to vGS.
Much like the control on the switch, vGS is my control.
So if vGS is 0, oh, I'm sorry.
If vGS is greater than or equal to some threshold voltage VT -- So vGS, the voltage applied here is greater than some voltage, VT, a threshold voltage, or the pressure of the switch is greater than some threshold pressure then this guy behaves like a short circuit.
This is iDS, this is my drain and this is my source.
So if the voltage applied between the gate and the source is higher than some threshold then this behaves like a short circuit.
Similarly, if the voltage vGS is less than some threshold VT then in that situation -- -- I get an open circuit.
And when I have an open circuit between D and S then the current iDS is going to be 0.
So this is the idealized model.
And this idealized model is called "the switch model of the MOSFET".
The switch model or the S model of the MOSFET.
Well, if you want to see the internals of the MOSFET, I won't cover that in lecture or recitation.
You can look at the section, I believe Section 6.7 of the course notes.
That has the internal structure of the MOSFET and how you physically construct such a device.
So what I can do here is step back and stare at the device for a second or two.
And what it says is that if I apply a lot of pressure, if vGS is greater than a threshold VT then I get a short circuit here just like my switch.
When in doubt think faucet.
If you put pressure on the faucet, think of this as closing, and when I open it, when vGS goes less than VD, less than a threshold, I take off the pressure and then it becomes an open circuit.
So I can plot the following.
Much like I plotted the iV characteristics of two terminal elements, I can plot the iV characteristics of this three terminal element in the following way.
I can focus on two terminals and look at vDS and iDS for that terminal pair and draw the curves for how it will behave as I change vGS that I applied.
So what I'm going to show you is that if vGS is less than a threshold then this behaves like a open circuit.
So no matter what the voltage is the current is 0.
Similarly, if vGS greater than equal to some threshold voltage then I get the behavior iV curve of a short circuit where the current can be anything and controlled by external forces like in any short circuit.
So let me show you on the screen.
Lorenzo has kindly put the graph up already.
So I'm showing the iV curve of a switch.
Notice that when vGS is greater than VT, greater than a threshold I get the vertical line corresponding to a short circuit.
Is it this one?
This one.
There we go.
So what I'm going to do here is I'm going to reduce vGS to below VT. What should you see happening?
The curve, from being a short circuit, should hammer down to becoming an open circuit.
That's the curve for an open circuit as I drew out there for you.
VGS pressure ain't enough.
Lots of pressure, boom, it's a short circuit.
I really like to think of this pressure analogy if I get confused whenever I look at a MOS transistor and I need to look at vGS and so on I always think vGS is greater than VT. Lots of pressure on the switch it turns on.
Just remember that, and then you won't forget this vGS thing here.
So that's the behavior of a switch.
And so viola, there's our switch.
So I've given you a three terminal element that is a switch that is controlled like a mechanical switch.
So I can build a, if I replace -- This was my switch earlier.
And what I can do is replace this with my MOSFET and that's what I get.
And I won't bother showing you this is your inverter.
All of that has replaced the abstract switch with a physical switch which behaves as shown in the graph up there.
And so I apply an input here and I take the output here.
So as 6.002 you could look at this and say ah-ha, that is an inverter.
When you go to 004 what you will do is build this triangle and a circle around it and you will ignore what's inside and just look at that.
So in 002 we showed you that the internals look like a pattern with a MOSFET and a resistor, but it's really the abstract inverter looking in from the outside.
I'm just going to close the loop inside the digital gate, and this was inside your little inverter with a resistor and a switch.
Let me continue with this for a little longer here -- -- and do something that we like to do a lot, which is plot what are called input / output curves.
So let's say the voltage applied here is v in and let's call this v out.
For fun let's plot a v in versus v out for this inverter.
So when input is a 0, let's say VT is 1 volt for the inverter.
The threshold voltage is 1 volt.
The threshold pressure is 1 volt.
So when input is a 0, and let's say VS is 5 volts.
So when the input is a 0, this guy is turned off.
So what's the output?
What's the output voltage?
If this is turned off, what's the output voltage?
It's the supply.
The supply directly shows up here.
And so as long as the input is 0 the output is at 5 volts.
And this is true until the input reaches 1 volt.
As long as the input is less than 1 volt my output stays high.
And then when my input exceeds or hits 1 volt then at that point the switch turns on and the MOSFET turns on and shorts the output to ground in which case boom, this is what I get.
And then, no matter how much I increase the input, my switch stays on and the output follows a zero volts at the output.
So this is my v in versus v out curve for the inverter.
One of the interesting things that we do a lot is see whether this satisfies some voltage threshold.
So let's say I have a VOL of 0.5 volts, VOH of 4.5, VIL of 0.9 and VIH of 4.1 volts.
So VOL says in its low value is the output less than 0.5?
Yup, output less than 0.5.
In its high is it more than 4.5?
Yup, it's more than 4.5.
Does it recognize all values below VIL as a low input?
Yup.
So anything below 0.9 or 1 for that matter is viewed as a low.
That's good.
So these pass.
And high, anything above 4.1, is that treated as a high?
Yes.
So anything above 4.1 is treated as a high and the output goes low.
So therefore this inverter that I've designed for you here satisfies the static discipline and this inverter can be used in circuits or other devices that conform to this value here.
In your recitation, you will look at a slightly more detailed model of the switch where the switch behaves like a resistor.
OK, good morning all.
So before we begin, I just thought I'd show you a little news item that I happened to read that was very relevant to what we covered recently in So you recall when we did the digital section a few days ago last Thursday, we talked about a switch.
We talked about the MOSFET switch, which when turned on and off, by input signals could help build gates which would then be combined in tens of millions of quantities and go into chips like the Pentium 4 and AMD Athlon 64, and so on it so forth.
So I just saw this news item that I came across, and this says they are rethinking the basic construction of the products.
It talks about the semiconductor manufacturers like AMD, Intel, and others that build digital chips.
They are rethinking the basic construction of the products down to the architecture of the transistor.
That's a MOS transistor, and the on/off switch inside the chip.
OK, now this might imply that there is a single switch inside the chip, but no, there's tens of millions of transistors, or tens of millions of switches inside a chip.
And pretty much any advancement that can be made to the basic transistor can have a 10 million to 20 million times effect because there are that many of them on a single chip.
So I thought that was very appropriate.
OK. Let's dive into a quick review.
So this week, we had begun nonlinear analysis, and I just thought I'd blast through a few animations that I've created, trying to give you more insight into the behavior of some of the things that we have done.
Now first of all, as I did the last time, let me try to put it in perspective most of what you've learned thus far, and what we will be learning today.
So the past week, we have been focusing on nonlinear analysis.
And as I pointed out, here is how this fits into the big picture.
So, we had our 6.002 world, at what we said is that we are engineers.
We are going to devise our own playground in which to play with our own rules.
And that's our playground.
That's what we're going to learn about in 002, and for that matter, the rest of EECS at MIT.
It's all within this playground here.
And this is the playground with lumped circuit abstraction, and good old KVL, KCl, node method, your basic composition rules apply within this playground that directly come from Maxwell's equations because you have made the lumped matter discipline assumptions.
OK, so then we said a large part of the playground is linear, and some other much more intuitive techniques apply within the linear portion of that playground, techniques like the superposition, Thevenin and Norton.
In most exercises, and quizzes, and experiments, and so on that you do in real life, you can pretty much apply these simple techniques.
Very rarely do you have to go into the node method for circuits that are more complicated than single source and a couple of elements.
And then, there's the nonlinear part.
Remember, the reason I showed this is that this is the same playground.
OK, linear and nonlinear are part of the same playground.
OK, even nonlinear elements are lumped circuit elements, and they follow KVL, KCl, the node equation, and so on.
And then, last week we spent some time talking about the digital abstraction.
So we focused on a smaller region of the playground.
And the assumptions we made in there were even tighter.
We said that it is part of the playground we shall only deal with binary values.
We'll digitize or lump values into highs and lows, and that's where our circuits are going to be.
And these circuits, when looked at as a whole, were nonlinear.
So, this is a simple NAND gate circuit.
And this is the input/output characteristic.
So, for example, if I hold B at zero, and I apply a zero to one transition at A, then this is the output that I will see at C. So notice, this is decidedly nonlinear.
Then I said that, look, suppose we had to fix the input values at a given set.
OK, so let's say, for example, I fix A at one, and B at one.
OK, and then look at the circuit in this situation.
What do I find?
What I find is that the entire digital set of circuits that we were looking at move over into the linear space for a given set of switch settings, OK?
So, when I set A 1 and B 1, A equal to one and B equal to one, my NAND gate becomes like this.
OK, it's a simple resistive network with a voltage source, VS.
So, for a fixed set of inputs, for a given set of inputs, if I don't change my inputs, then my circuit looks like a linear circuit, and my good old linear analysis techniques apply.
So that was last week.
And this week, we are looking at the nonlinear space.
And we looked at a couple of techniques in the nonlinear space, analytical techniques and graphical techniques.
And then, I showed you an example.
OK, I showed you an example circuit that was something that I would like to build involving the light emitting expo dweeb, my little garage door opener thingamajig, and I wanted to transmit music over that light beam.
I also showed you that it was highly distorted because it was in the nonlinear space.
So, today what I'm going to do is introduce a new part of the playground.
There's a new part of the playground, and I'll show you a technique whereby by focusing on this part of the playground and disciplining ourselves in the kind of inputs we apply to circuits, I'm going to show you that certain kinds of nonlinear circuits also move over, when used in a particular way, also move into the linear analysis domain.
OK, so let me leave that for now and go back into quickly reviewing the motivating example of music that I had taken last time.
OK, so here was a little example.
So I have a music source, VI, and I apply that.
This device that I call the, lightheartedly, the Light Emitting Expo Dweeb has a current, VD, across it, or a voltage, VD, across it, and a current ID through it.
And the light intensity, I said, was proportional to the current.
And because of that, I was able to get the light to impinge on a receiving device, which produced a current that was proportional to the intensity of light falling on it.
And that signal would then be amplified somehow.
We haven't talked about all of this stuff yet.
This will happen next week.
But let's say we somehow amplify the signal and then played out through a set of speakers.
All right, so if I had some sort of a music signal here, then I could then transmit the music signal over to the side on top of this light beam.
But the problem, as I said the last time, was that our device, the Light Emitting Expo Dweeb had an exponential characteristic, so that I had some trouble in getting undistorted music.
So, the characteristic of the VI characteristics of my device looked like so.
The ID versus VD curve looked as follows.
OK, it was decidedly nonlinear.
And because of that, I was getting a lot of distortions in my signal, and I showed you a little trick to plot, given an input waveform at a transfer function such as here to plot the output function.
OK, let me show you another little animation that I have created here for you that should give you even more intuition in terms of how it happens.
So, this is a characteristic I showed you up here.
It's on both sides, but I guess it points to only one unless I shuttle back and forth really fast.
So on average, I'll be in both places.
But anyway, so here's my ID versus VD characteristic.
And as I said, there's an exponential ID versus VD curve.
And I want to see what the output looks like, for example, a sinusoidal input.
So I said, let's place the input along a little graph, rotate it so, and take a sinusoid, and apply a sinusoid to the input, VI, which would also appear across the Light Emitting Expo Dweeb.
And then, what I wanted to see was how the output looked.
OK, so let me tell you that the output is going to look like this.
OK, the output is going to look like so.
And, a little artifice to discover curves like this is to think about a point here corresponding to the point on the transfer curve here, because this is VD, looking at the Y intercept.
That's a value of ID, and that's a value of ID here.
And, time moves along here, and time moves along here.
So, I did this little animation.
You'd better be impressed.
It took me six hours to do it.
So, here it goes.
So, let's say I start by focusing on this little point that corresponds to this point on the transfer function, which then, in turn, points to a time, zero, this point on my ID curve.
OK, I hope this works.
So, as my point moves down [LAUGHTER], this was fun to do, I promise you.
So notice that as this point has the following excursion, this had the following excursions here.
OK, all right.
So let me pause that little animation there.
At the end of the lecture, I'll put that up again if you like, and you all can come and play with it.
So, you can actually do this in PowerPoint.
It took me quite a bit of time to figure out how to do it, though, but it's fun.
OK, so let me show you a little demo, and show you a sinusoid, and show you what the output looks like if I apply a sinusoid for VI.
So, I'll show you ID as a function of VI when VI is a sinusoid.
There you go.
So, I applied my sinusoid VI, and this is the current that I get.
And notice, this is the transfer function that I talked about, the ID versus VD curve of my Light Emitting Expo Dweeb.
And I get this highly nonlinear transformation of the input as I get to the output.
OK, so that is a problem.
And then, I also played some music for you.
Let's do that, too.
I played some music for you.
I applied the music as an input to the circuit, and that's the output.
OK, that's the output that I'm observing at the amplifier.
It's highly distorted.
OK, we can stop that.
There you go.
OK, so that was my problem.
OK, so we had covered, we had gone this far last Tuesday.
I set the problem up for you, motivated what we had to do, and showed you that I was able to transmit music over my garage door opener, but I did not think I could listen to that music for very long.
So, I challenged all of us to think about how a trick that I could use to be able to transmit music and have a linear response.
So, did you people get time to think about it?
So how many people here think they know the answer?
It's OK, don't be modest.
Go ahead.
Could you speak louder?
Yeah, you find another something, kind of element, that's got the opposite graph so that when you add them together.
Oh, this guy wants to cheat.
No.
He wants a new element.
So, no, no new elements.
Pardon?
Build an MP3 encoder.
Ah-ha, so that will happen much later.
Yes?
Digitize the signal before you send it to the LED?
Digitize the signal before you send it to the LED.
But in some sense, each of these solutions is a huge sledgehammer approach to look at solving it.
There's a much simpler technique I can apply here.
Yeah?
Add a voltage offset.
Ah, ah-ha, that might work.
What else?
So let's say, here's my signal, right?
If I add a voltage offset, that will just bump the signal up here.
Then the curve is still nonlinear.
But you're getting there.
Well, I'll tell you what.
Let's pause here.
Let me quit while I'm ahead.
OK, so the answer here, folks, is Zen.
OK, what I want you to do is, so, in Zen, what you have to do is you have to sit down in a courtyard, and look at a rock, like a small rock on the ground.
And you got a focus on it till the rest of Earth kind of vanishes.
Just focus on the rock.
OK, now make like you're in a courtyard, and you're looking at this little area here.
Just look at this.
OK, and I'll give you ten seconds.
Sit down quietly, and no sounds.
Just stare at the spot here.
OK, make believe this is your little rock, and just stand there and think about it.
OK, I'll give you five seconds to do that.
Just stare at it.
And very soon, the answer should pop into your heads.
OK, what do you see?
This guy, if I focus on this really small region of the graph, this small little piece looks more or less linear.
OK, hmm, so that should give me some insight.
This whole thing, the macrograph is nonlinear.
But I focus on a little rinky dinky piece of that graph like so, that appears more or less linear.
If it's small enough, that appears linear.
So, I'm staring at this, and that appears linear.
The question is, how do I exploit this little small, little, linear region to get a linear response from my device.
OK, so here's the trick that I'm going to use.
The little trick that I'm going to use is the following.
Notice that, let me call this voltage at the center of this region capital VD.
What I can do, if I take my input signal, and I just pointed out earlier, I bump it up.
I boost it.
OK, so I apply a DC offset to my input signal, like so.
So I apply some input signal, VI, which is also equal to the VD if I look at a variable across the nonlinear element.
If I apply a DC offset, VI, and I superimpose the music on top of that, let me call my music, just to distinguish between the two, capital VI, and the small vi.
OK, that's my music.
So here's my capital VD, my DC offset.
And I want to superimpose my music on top of that.
OK, so I've gotten halfway there.
By superimposing my music here instead of having excursions out here, I now have excursions out here.
OK, and so I'm using some portion of the graph here.
But that's still way beyond the small little element there.
So a second think that I do in addition to boosting up the signal is shrink it.
Think of boost and shrink, BS.
So what I want to do is boost up the signal using a DC offset, and shrink the sucker.
OK, so I'm going to go with a small signal and bump it up.
OK, so now what happens is that small signal in its excursions, only uses that little portion of the graph.
OK, again, remember: bump and shrink, bump and shrink, two things, boost and shrink.
So what do you think of that trick?
So, by doing that, what happens is that signal that has excursions here will produce a corresponding response in this region, OK?
And I argue that since this is more or less like a straight line, I invoke Zen here, and argue that this little signal now gets transformed, and I get a linear response
boost and shrink.
So in terms of my circuit, let me draw it out for you.
My Light Emitting Expo Dweeb, and this whole signal was what I used to call V capital I, and that's made up of two components now, a bump offset, and a shrunk voltage VI.
It shrunk, so therefore I've used the small v and small i, like, really, really small.
In the same manner, I get a VD ID across the LED, and the corresponding values here will also have a DC offset and a small response.
Let me call that ID plus I small d. I'll do all this mathematically in a second as well, but first let me do it completely intuitively so you get some insight into what's going on.
And, VD is simply capital VD plus small vd.
OK, and this is the same as VI, I, and VI.
OK, so what have I done?
I've done two things.
I have said, as an engineer, OK, I care about getting music across my garage door opener.
And I'll do what it takes to do that.
OK, so as an engineer, I'll do two things.
I'm going to bump my signal up and shrink it.
And the bumping and shrinking, and I do it like this.
I shrink my signal, the music signal here, and add a DC offset.
OK, and I claim that the music I listened on the other side now, provided I have enough amplification there, is going to be undistorted.
OK, so far I've showing this to you completely intuitively using little sketches, no math.
I promise you, I'll give you a bunch of math in a few seconds, but just get the basic idea, and get the intuition behind it.
So let's go back to our demo and take a look.
So remember, BS, right, bump and shrink.
So what I'm going to do is first of all, let me bump up the signal.
So, what I'll do is I want to add an offset to my input, and let me bump it up.
Let me shrink it first.
It'll make the point a little clearer.
So, the big input, green, is a big input.
Let me shrink it.
OK, so I've made my input small, and in the middle of that picture out there, you see the region of the transfer curve that's being articulated.
OK, this region of the curve is being articulated by the small signal.
It's a much smaller signal.
And the output is still distorted because I have to do two things: bump and shrink.
I've only shrunk.
OK, let me bump it up now.
What's the yellow curve?
It's going to get linear.
It's going to get proportional to the input.
Then I'm bumping it up now.
I can make it smaller, make it even smaller, there you go.
Isn't that fantastic?
So, I'm making nature do my bidding here, OK?
So, this is one of those, when I learned electronics and so on many, many years ago, this was one of those really big ah-ha moments for me, saying, wow, that stuff is cool.
It's something that I couldn't think about myself, and it's not obvious, and by being disciplined and creative in how I use circuits, I can do really, really cool things.
OK, remember this as a big ah-ha moment for you.
So, here's my little signal that I've shrunk and bumped up, and my output is a sinusoid, and not this funny, distorted waveform.
And notice that this is the region of the curve that is being articulated.
So, I can make the signal even smaller if I like.
OK, and what I'd like to do next is play music for you, and if you don't believe your eyes, you can at least believe your ears.
Let me go to the distorted signal again, switch to music, and raise it up.
OK, now what we'll do is shrink the music signal and then bump it up.
Can I turn the volume down a little bit?
That's good.
OK, so if I shrunk the volume a little bit, and let me bump it up, now.
[MUSIC PLAYS] Just remember this as a big ah-ha moment.
OK, the signal is really, really small.
I like that.
I like the enthusiasm.
OK, so the signal's very small, and I get a more or less linear response.
OK. All right, so that's intuition, and the approach that I've taken is called, it's variously called small signal analysis, incremental analysis, small signal method, small signal discipline, whatever you want.
OK, this simply says that by boosting and shrinking my signal, I get a response that's more or less linear even when I have a nonlinear device.
And this technique is called the small signal approach.
So, just to focus on that a little bit longer, switch to page five of your notes and let me draw something out for you.
OK, so what I have here, this is my offset VD, and from the VD offset I have my little signal V small d, and the total signal is called V capital D. Offset, small signal, and that's my total signal.
OK, notice the offset is all capital.
The total signal is small v capital D, and the music or the small signal is small v small d. Similarly, the output is going to look like this, and here I get an offset in the output ID.
I get a corresponding signal, I small d, and I get a total signal, I capital D, OK?
The cool thing to notice is that the signal here, the output signal here corresponding to the input signal, the music signal, VD, is small I small D, and that is more or less linear.
OK, and I can even plot the signal like so.
This is my input, v capital D. That's T. This is VD, V small d. That is my total input.
And similarly, I have an output.
And this is my output ID.
And, that looks like this, I capital D, small i small d, total signal I capital D. OK, so that's the small signal method.
So, let me summarize that for you.
There are three steps to the method.
So, first of all, operate at some DC offset.
This is also called DC bias, and in that example it's VDID.
OK, so I choose an operating point that bumps up the operation in some region of interest.
The second step is to superimpose small signal on top of VD, capital V capital D, to superimpose a small signal, and the third step is observe the response -- -- and the response, small i small d, that's the music part of the response, ID, is approximately linear.
OK, three steps to the method here, and just remember this notation.
And, my notation in the small signal model is as follows.
My total signal ID is the sum of two signals, I capital D plus small i small d. This is called the total signal.
That's called the DC offset.
And this is the superimposed small signal.
OK, total signal, DC offset, plus the small signal.
And sometimes, especially when doing math, and so on, we may oftentimes represent ID as a delta, I capital D, OK, to show that ID is incremental change in the value of I capital D. And because of that, this method is also often called the incremental method, incremental analysis.
OK, so far what I've done is given you some intuition.
I've developed a small, simple method, given you some insight into why we use this method, and also shown you some demonstrations that show that when I bump and shrink, and observe the response, I do get a more or less linear response.
So let me now do this mathematically and show you that mathematically, you can also derive your response to be a linear response.
This is page seven.
So, I know that ID is some function of the diode voltage.
F was my nonlinear function.
OK, so my function F was a nonlinear function.
So therefore, ID was nonlinearly related to VD.
So, let's do the math.
So as a first step, what we did was replace VD by a DC offset, the small signal method, a DC offset, plus a small incremental change.
OK, by doing the math, let me simply use the delta VD notation to show you that I'm dealing with small increments, and also because in the mathematics community, when you learn about some of these techniques, they will use the incremental change notation, which is the delta VD notation.
In electrical engineering, we use a small v, small d notation.
So, this is a large DC offset, and this is a small change about that offset.
So, you folks have taken math courses before, and been looking at finding out the value of a function, which is a small change for an input value, which is a small change about a big input value or a big DC point is Taylor's expansion.
OK, so let's use Taylor's series expansion, OK, and substitute VD plus delta VD into this, and see what ID looks like.
Again, let me tell you where I'm going with this.
ID equals F of VD.
This is a nonlinear function, OK?
I claim that by replacing VD, the input, with the DC offset plus a small value, the resulting response to the small value will be linear, OK?
So what I'm going to do next is replace VD with this sum here, and then do the math, and show you that the response corresponding, or the change in ID corresponding to the change in VD is going to be linear.
All right, so let's expand this function using Taylor's series near the DC offset point, capital V capital D. OK, so ID is simply, by Taylor's series, I want to find out a value of the function close to V capital D. OK, so I take the value of the function at that point, and then I add a few terms in my Taylor's series expansion.
The first term is simply the good old Taylor's series stuff.
OK, the first term is the first derivative of the function times the change.
And then, the second one is second derivative.
OK, and then I get higher order terms.
So this is nothing new here.
This is good old Taylor series expansion, and again, let me tell you where I'm going.
I want to look at the response for an input that looks like this, and I want to show you at the end of the day that the response in ID, the effect on ID of using an input like this is as if that effect, the incremental change is linearly related to the small input, delta VD.
So here's my Taylor's series expansion for delta V. Now remember, I told you that delta VD is much, much smaller than V capital D. OK, it's a very, very small quantity.
But that quantity is really very small.
Then what I'm going to get is that my output is, I can begin to ignore my second order terms.
OK, delta VD is very, very, very small.
Then, what I'm going to do is that ignore higher order terms.
So I'll go and ignore higher order terms.
They'll all go to zero.
Remember, I can do this because by design I've chosen delta VD to be very, very, very small.
OK, remember, we are engineers.
I've chosen it in a way that this is very small.
OK, so I'm telling you that's the case, and under those conditions, I can ignore second higher order terms, in which case I am left with this expression here.
So let me rewrite this.
Let me rewrite this down here.
OK, I've just copied this turnout, I've ignored all these terms here, and so I have a more or less equal to sign that remains.
So what I'm going to do is when I apply a small input of this form to a large DC offset, my output is also going to look like some output offset with a change in the output offset.
And let me call the output offset I capital D, and some small change in the output delta ID.
OK, we'll make sure we can convince ourselves that this is indeed the case.
Notice that this guy here, F of capital V capital D is a constant.
That's a constant with respect to the incremental change, delta VD.
Similarly, this part here is a constant.
Notice that this term here is the first derivative of the function evaluated at the DC bias point, capital V capital D. OK, so this term is also a constant with respect to delta VD.
So notice, then, I have a constant term plus a constant term multiplying a small change, delta VD.
So what I can do next is, in this case, given that I have a constant term on both sides, and on this side it's a time varying term, what I can do is equate the two constant terms.
I can go ahead and equate these two terms.
Remember, I have a constant plus a time varying term, OK, if I'm assuming here that delta VD, my little music signal is a time varying term.
So, this constant will equal this, so ID must equal F of VD.
And I know that's the case because the function evaluated at the DC offset gives me the DC current ID.
And similarly, ID is equal to that component.
Delta ID is equal to D, F of -- OK, so my incremental change in the output is the first derivative multiplied by the small change in the current.
OK, so I'm pretty much done.
So, therefore, notice that delta ID is proportional to delta VD.
OK, and that's what I had set out to show.
Remember, I had set out to show that provided my input is a small excursion around a large DC offset, then my output could also be a large DC offset with a small excursion on top of it where the two excursions, the input excursion and the output excursion would be linearly related like so.
OK, and the method is very simple.
I simply expanded the function about that point, that DC point, neglected higher order terms, and notice that my incremental term was simply the derivative plus the incremental change, a derivative times the incremental change in the input.
Move onto page nine, and I'd like to give you a quick graphical interpretation of this.
So I gave an intuitive explanation earlier.
This is a mathematical explanation that shows you that the input could be linearly related to the output, provided, the outputs would be linearly related to the input, provided the input has a DC offset, and small excursions about that DC offset.
So, let me give you some intuition in what you've really done here, using a little graph here.
So, I'm going to plot ID versus VD, and notice that I have some point here, V capital D, I capital D. That's my DC bias.
So, I have some DC bias point here.
OK, what is this?
That is simply the slope of the curve at that point.
OK, it's the slope of this curve evaluated at this point.
So this guy here is simply the slope of this curve evaluated at ID VD.
OK, now, what I care about is this point here, and this point here.
So let's say that this is delta VD, all right, and that corresponds to this point here.
So what I've done is taken the slope and multiplied that by delta VD.
So I've taken the slope, and multiplied it by delta VD, OK, and that gives me this component here.
OK, and so, this is the point that I'm going to get.
So in other words, what I've done is approximated point A using the Taylor trick by the point B. OK, so this is a point, A, which is what I really want, and I've approximated that by taking the slope of the function at V capital D, and multiplying that by the change in the input to get the corresponding Y offset, and that's the point that I get.
And notice that if I make this delta VD small enough, then the error between these two points becomes smaller and smaller.
So back to our example, so ID was a e to the BVD.
This was the relation for our Expo Dweeb, and let me just plug in the values.
So, ID plus small id.
Notice, I'm just shuttling back and forth between the notation delta VD, and small v small d. OK, and so that is given by a e to the BVD, oops, plus, I'm just writing that equation up there.
Let me call this equation X.
And so, I get the second term is the derivative, ab times e to the BVD times delta VD, small VD, and equating this term that the DC offset.
Notice that this is the DC offset in the output, and the small signal, ID is, further notice that in this particular example, what's that?
a e to the BVD.
That's simply ID again.
It just happens to be that way in this example.
So, I get ID times BVD.
So, for my input, small id, my incremental change in the output is some ID times B times VD.
And notice that this is a constant.
And because that is a constant, my small signal behavior ID is going to be linearly related to the signal, VD, the input signal VD.
OK, in the last three minutes, I'd like to give you one additional insight.
So what we've shown so far is if I have an offset and a small change above it, then my output ID will be linearly related to my input.
Now let's stare at this thing again.
Let me rewrite it.
It's some constant IDB times VD.
So, where have we seen such an expression before?
OK, where ID was some constant times VD.
OK, remember, I equals V divided by
Ohm's law.
What I want to show you now is how we constantly keep simplifying our lives.
The moment we hit some complication and things get too painful to analyze, as engineers, we come up with some clever tricks to make an analysis and use of circuits simple again.
And so, notice that this is similar to some, one by RD VD, where RD is simply one over IDB.
I'm just defining this to be RD.
And what that means is that I can take a nonlinear circuit that looks like this.
OK, and what I can do is replace this by its incremental equivalent, and build what is called a small signal circuit.
And I'll just introduce it here.
And we will revisit the circuit in much more gory detail a couple of weeks from now.
So, what I can do is build a small signal circuit where I have all the small signal variables, and replace a nonlinear device by a simple little resistor whose value is given by IDB.
OK, so therefore, what I can do is take my nonlinear circuit, and for small, incremental changes, replace that circuit with this equivalent small signal circuit, and go back to doing simple stuff again.
Thank you.
All right, good morning.
So today, we are going to talk about what is both a basic device in itself, the amplifier, and it also serves as a real key example of both nonlinear analysis and small signal analysis.
So, today, dependent sources and amplifiers.
So, let me first spend a few seconds just pointing out to you some of the key points from our previous lectures.
I also want to point out that each chapter in the course notes has a summary at the end of it.
And if you take a quick scan of the summary at the end of each chapter, it highlights the major takeaway points from each chapter.
It stresses what's important, and if you have to remember a few things, what are those things to remember?
So, to quickly review, we talked about a few primitive elements: resistors, voltage sources, and so on.
And by now, you should have the facility to play around with these device elements.
And then we talked about the Node method, and this is kind of the workhorse of 6.002.
When in doubt, use the Node method.
OK, and this will work both for linear circuits and nonlinear circuits.
OK, so if you see a problem, or if you see a situation in real life that requires analysis, then as a first step, you should try to think of whether you could apply some of the key intuitive shortcut methods, superposition.
One of my favorites, the Thevenin method, the Norton method, or the method that involves composition, that is very quickly analyzing circuits that have resistors in series and parallel.
OK, so if you can apply one of these quick, intuitive, shortcut methods, go do so.
If you can't, then usually you can resort to the Node method irrespective of whether the circuit is linear or nonlinear.
So the last week was focused on the nonlinear method or nonlinear circuits, and we spent the first lecture talking about a straightforward application of the Node method, which gave us a bunch of nonlinear equations that we had to solve.
In the last lecture, we talked about the small signal trick.
What we said is if you look at the whole space of nonlinear circuits, then within that space, if we focus on small variations, small perturbations about an operating point, then even the behavior of nonlinear circuits in that small regime would be linear.
So small signal method.
And as an example, I showed you how I could take a highly nonlinear device like the garage door opener LED, and using that, build a pretty nice transmitter that would transmit music.
And as long as we kept the signal small, and operated the device in a region where its transfer curve was relatively smooth, and I biased, or set the operating point appropriately, I would get a linear, small signal response.
OK.
So today, we're going to do a couple things.
We're going to look at dependent sources.
And the reading for this is section 2.6 of your course notes.
And, the dependent source will be a new element in your tool chest.
We will also do amplifiers, and amplifiers are in section 7.1 and section 7.2 of your course notes.
So, before I begin with dependent sources, I'm just a huge believer in motivating things with real world examples.
OK, so let me start by motivating: why we need an amplifier?
Why do we need to do things like this?
Or why do we even bother?
And, spend a few minutes really getting you to appreciate that amplification is fundamental.
OK, it's as foundational to life as high fat potato chips and stuff like that.
So, let's do some basic examples here.
So first, let me talk about, why do we need to amplify signals.
Why amplify?
Why do we care about building an amplifier?
So, an amplifier, think of a little box, and apply some sort of small input.
And I get a larger output.
In this example, this may be a voltage with a swing of 10 mV, and in this case, the output might be another voltage with a swing of, say, 100 mV.
And commonly, the amplifier, in addition to an input and an output, input port and output port, may also contain the power port, OK, so that I can apply a power supply to the amplifier because commonly as an amplifier signal, I'm looking for a power gain as well, an increase in the power provided by the output.
So, that's an abstract definition of an amplifier, and let's take a look at an example of why we may need this.
So let's say I have a small, useful signal, and let's say the signal has 1 mV peak to peak.
And, I'm looking to transmit the signal over a wire to some other point.
But let's say that in this environment, I get a bunch of noise that is in a noisy environment.
And in this environment, let's assume that some noise may get superimposed.
And if I have a 1 mV signal, and 10 mV of noise, then what I end up with at the output is something that looks like this.
And it's really hard to distinguish my 1 mV signal from that large amount of noise.
On the other hand, if I do the following, if I took the signal and passed the signal to an amplifier, and I amplified the signal to be a much larger version of the same signal, let's say in this particular situation 100 mV peak to peak signal.
OK, so I magnified the signal by a factor of 100.
OK, let's say it's a linear amplifier, I linearly amplified signal to be 100 mV, then in that case, if I had a noise on top of this, it's going to be less discernible.
The signal will look like this.
OK, my 10 mV noise would add on to it.
But, this is still pretty decent.
I can still recognize the input.
And so, this is one application of amplification.
If I need to send something from point A to point B as an analog signal, then an amplified signal is less prone to noise attacks than a small signal.
Not surprisingly, a large number of devices that are used in everyday life have amplifiers built into them.
So, get a little cell phone, and virtually every single cell phone contains an amplifier.
By the way, this is an all digital cell phone.
It's a Kyocera, I forget the number now.
It's completely digital.
OK, although they say it's completely digital, it turns out that a significant fraction of the circuitry is analog, in particular, so digital is sort of a marketing term to say that there's something special about this.
But remember, there's a bunch of analog stuff.
So, here's my little antenna from the cell phone.
OK, and typically the first thing that happens to a signal as it comes out of the antenna in your cell phone is, look at cell phone circuits, or cell phone systems would be something that looks like this, OK, this, and may have a label LNA.
If someone were to take a guess at what LNA might stand for?
What's that?
Linear amplifier.
That's pretty good.
So that's LNA.
Close enough.
A is correct.
It's amplifier.
What does L and N stand for?
Low noise.
OK, so this stands for low noise amplifier.
So, I get a really rinky dinky small signal here, and then the low noise amplifier amplifies a signal.
And in real cell phones, and for that matter, in your 802.11b, or 802.11a, or 802.11g wireless cards, same thing.
Antenna, low noise amplifier, and then you may have a bunch of processing.
And commonly, you have a bunch of analog processing.
And then, you convert the analog to a digital signal.
OK, I recall last week I asked somebody in class here, how would we transmit the signal from point A to point B without it being impacted way too much by noise, and he said, oh, go digital.
Good point.
OK, so if I go digital, I can transfer the signal without noise being a real factor.
But the analog to digital converters need the signal strengths to be a given value before it can chop it up into digital levels.
OK, so an amplifier is very fundamental.
OK, and so in this case, what may be a signal of a few tens of microvolts to be amplified to some large enough value that it can be further processed.
So, that's application of amplification in the analog domain.
Let me talk about amplification in the digital domain.
So, that's in the analog domain.
This amplification is in the domain that I have both analog and digital.
OK, and now let me talk about amplification in the digital domain, OK?
I'm going to argue that amplification is absolutely foundational to the digital domain.
OK, the digital abstraction would not occur if I did not have basic amplification.
OK, and the next minute and 37 seconds I will prove that to you, OK?
So, let's do so.
So, let's suppose I have a very simple digital system, and the system simply contains a pair of inverters.
So, if I send a one here, it's a zero here and a one here, which is a very simple, trivial, digital system.
And here's the input.
Here's the output.
And we said that for digital systems of this sort to work, they have to follow a static discipline.
OK, our signals and our circuits must follow a discipline for them all to work together.
And, the discipline we described comprised of signals adhering to certain voltage thresholds so that all the components in the system could agree on what comprised a zero, and what comprised a one, OK?
So the way we did that was we said that you would have a threshold called VIH, V input high, and another threshold called VIL, V input low.
OK, and we said that this circuit must recognize signals that are higher than VIH, 3 V for example as a one, and simultaneously, any signal that has a voltage level less than VIL, say, two volts, should be recognized as a zero.
That was the input constraint.
On the output, it had a similar set of constraints, where we had tougher constraints on devices, where we said that the output had to satisfy a output low constraint, output high constraint.
What this said is that for this circuit to be called a good digital circuit that satisfies the static discipline, signals that were ones here should be recognized as such.
And if I am producing a one as an output, then the signal level should be higher than VOH.
Similarly, if the signal's a zero, then it should be less than VOL.
So as an example, this may be 2 V, this may be 3 V, and this may be 4 V, and this may be 1 V. OK, so input, I should recognize 2 V and less as a zero, but at the output I have to produce a very, very low value, 1 V. So, I have some noise margin.
So as an example, say if I made a plot of the input/output, so I get my VIL here and VIH here.
This is time.
This would comprise a valid digital signal: zero, one, zero, one, and so on.
OK, now, I had a tougher set of constraints at the output.
I would have VOL, VOH.
So, at the output, OK, I'm required to stretch the ones and zeros to be further apart from each other so that I get noise margin, and the corresponding signal for our little circuit there would look like so.
Right, if this is a valid input, then this would be the corresponding, valid output.
OK, and need I say more?
OK, you can see that, intuitively, look, there's amplification happening here, and the reason is that VOL is chosen to be less than VIL, and VOH is higher than VIH.
So therefore, the signal has to be stretched.
The signal has to be amplified.
OK, and what's the minimum amplification needed for the system to work?
The minimum amplification is if I had a signal that looked like this.
OK, that barely skimmed the VIL, VIH level.
OK, so if signal were this high peak to peak, VIH minus VIL, and what's the absolute minimum signal at the output?
It would look something like this.
OK, barely skimming VOL and VOH, OK, so the corresponding output level would be VOH minus VOL.
OK, so this is the absolute minimum amplification that my digital circuit has to provide.
OK, and notice, VOH is larger than VIH.
VOL is smaller than VIL.
Therefore, this quantity needs to be greater than one.
OK, so I've shown you both a simple, graphical, intuitive explanation, and this is a slightly more formal proof that even the digital circuit really requires to have amplification built into it, if it is to satisfy valid static disciplines.
Yes?
Yes.
The question is, is that the same as gain?
Good question.
Yes, the term amplification has many, many variants.
You could say gain.
You could say amplification.
You could say increase in signal strength, and so on and so forth.
And in fact, when talking about low noise amplifiers, people sometimes talk about having the low noise, high gain amplifier at the input stage.
OK, so let me pause there in terms of motivation.
So, I believe I've motivated every which way: pure analog, analog/digital, and digital.
OK, so I've covered every single base here.
And so, we need amplification.
OK, so let's look at how to build a fundamental, primitive device called the amplifier.
Before we do that, however, let me take a quick detour.
It will be convenient for me, as I show you how to build an amplifier, to introduce a new device, a new element, called the dependent source.
OK, let me introduce a new device for your arsenal of devices, along with resistors, You learned about a MOSFET, a switch, voltage source, current source, and now a dependent source.
So, a dependent source looks like this, OK, has an output port, and has a control port.
So, a dependent source in its simplest form has two ports: an input port and an output port.
Remember, a port is a convenient pairing of terminals, and I apply signals to such terminal pairs.
But this is a abstract diagram for a dependent source, and to get a little bit more specific, let me show you an example of a dependent source.
So, let's say, here's my input, and I label the terminal variables for the input.
VC is the voltage applied to the input, and IC is the current into this terminal here.
And, here is the symbol for the dependent source.
Much like a current source or a voltage source has a circle around it, the corresponding symbol for a dependent source is like so.
So this example, for instance, is a dependent, current source.
I can apply the corresponding output variables, I0, OK, and I can say that the current, I, is some function.
In this example, I've designed the example that the current through the current source, I, is some function of the input voltage or the control voltage, VC.
OK, so notice that the current through a current source, the current through this current source, I, is some function of another variable.
OK, in this example, it's the voltage across its control port.
Not surprisingly, this device is called a voltage controlled current source -- -- or a VCCS.
So, in like manner I can also devise other forms of sources.
You can think of this is a device where a voltage controls an output current.
You can think of all other combinations, current controlling current, voltage controlling voltage, current controlling voltage, and so on.
So, another example, I give you another dependent source, and in this situation, my output current is controlled by an input current, VC.
IC rather.
And I claim that I for this one is some function of a current, IC.
OK, it's another dependent source where the output current for its output port is related to the current, IC.
And, this is a current controlled current source.
OK, it's a current controlled current source.
And, if I had lots of time on my hands, and I was wanting to kill time, I'd sit around drawing for you, other types of dependent sources.
I would draw for you a current controlled voltage sourced, and I could also draw for you a voltage controlled voltage source.
OK, so that's an abstract diagram for such a source.
And so, let's do a few examples involving elements like this.
To begin, just so you can build up your intuition, let me start by doing a very simple circuit, involving an independent current source, OK, just so we can relate back to what we've been doing so far.
So, let's say I have some resistor, and I have a standard current source with current I nought.
This is an independent current source.
Remember the circle?
And, some resistor, R, and let's say I care about the voltage across the resistor.
OK, so I have a current I nought flowing through it.
So, I can very quickly write down VR as, simply, I0 R. OK, it's the drop across the resistor when a current I nought flows through it.
OK, so this is what you've been used to doing.
Correspondingly, I can do an example with a dependent current source.
And, as an example, I'll use a voltage controlled current source.
OK, a voltage controlled current source is a dependent current source whose output current depends on the voltage applied at the control port of the current source.
So let me build a little circuit.
OK, so here's my current.
And let's say it's VC IC for the control port, and similarly, let's say my current I here is some function of the control port voltage.
And let's say, to be specific, there is some K over VC, some function.
OK, there are a variety of dependent sources that can be built, and here's a hypothetical device where the output current is mathematically related to the input in the following manner.
So, let me build a circuit of the following form.
So, let's add the resistor, R, and here's my circuit, OK?
And, as before, let me look to figuring out what VR is.
So, notice that I have to supply some voltage at the input so that the output can depend on the input because right now I don't know what the input here.
So what I'll do is let me apply VR over here.
OK, so let me make this connection.
OK, let me make the connection from here to here.
What I've done is I've applied VR at the control port of the dependent current source.
OK, and I often draw a circuit like this.
This looks pretty messy.
I will often draw the circuit like so: R, VR.
OK, short form circuit drawing would look like this.
This is a complete drawing that I show you the explicit connections of the control port, but oftentimes, when the control port does not have any other impact in the circuit, you can eliminate, don't explicitly show the control port.
Rather, you can simply show the dependence of the output current on whatever circuit variable you have in mind.
So, you can draw the diamond like this, and see its current is some function of VR.
VR in this is case is K divided by VR, OK?
OK, so let's go ahead and analyze this little circuit here, and look at what this might give us.
Our goal, as before, is to find out the value, VR.
So, in this case, let's apply the Node method to this node, and sum the currents into that node to be zero.
OK, so sum the currents going into that node to be zero.
The current going down is simply VR divided by R. OK, and that is equal to the current that is going out of the node.
And so that is equal to F of VR.
And I know that F of VR is given by K divided by VR.
OK, a simple application of the Node method.
So then, I collect VR's on the left hand side, and I get VR squared is K times R, OK, and VR is simply the square root of KR.
There you go: I'm done.
OK, I've gone ahead an applied the Node method to this, and when have to figure out the current here, I simply reflect the fact that it depends on VR like so, and I just go ahead and solve the circuit.
Remember, the workhorse of the circuit industry, the Node method, when in doubt, apply it.
It simply works.
And notice, this is a nonlinear circuit.
OK, the dependence is nonlinear, and I get the response like so.
So, to plug in some numbers, supposing K was 10 to the minus 3 amperes per volt, and R was one kilo ohm, then I can plug the numbers in and the kilo here cancels with the 10 to the minus 3, and I get VR equals 1 V. OK, this simply says, if I build a circuit like this, then this voltage here will be 1 V. So, again, as long as you remember that the dependent source is simply another little circuit element, OK, and you usually draw just the output port for dependent sources, and reflect the way that the control affects the current, that'll suffice, and you get, through the application of the Node method, the variable you're interested in.
Let's do another example, OK, of another fun current source, a voltage controlled current source, and look at it this way.
So, let's say I have a resistor, and I have a current source, a resistor, RL, and this goes to some, I apply a VS here.
Remember this short form notation; that's simply applying a supply VS between that node and the ground.
OK, and let us say the current IV through the device is some function of the current at its control port.
OK, so I'm not going to show you that.
But remember that the device already looks like this, that there is a control port here.
I'm not showing that to you.
And let us say that I apply some voltage, VI, to the input port.
The reason we often don't show the input port is for many practical dependent sources, the input has no other effect on the circuit.
So, for example, in this case, the input has infinite resistance looking in.
So therefore, if I apply a VI here, it doesn't draw any current from VI.
I simply apply the voltage, VI.
It doesn't affect the circuit in any other way except in terms of how it controls the current ID.
So let's say the current ID is some function of VI because VI is applied at the control port.
OK, and as I pointed out before, I oftentimes, just for clarity, just to show this dependent source explicitly.
OK, so let's work the example.
So as I said, I'm going to choose ID to be F of VI, and let's pick some specific parameters here.
Let's say it's K by two VI minus one, both squared.
OK, and let's say this is true for VI less than equal to one volt.
And let us also say that ID equals zero for VI less than one volt.
OK, it's a dependent source, and it can have various forms of dependences on the input.
And, I just picked an example of some hypothetical, or as yet, hypothetical dependent source, the current through which is related to the input using a square law relation, VI minus one all squared as long as VI is greater than one.
And if VI is less than one, then the current is simply zero, it shuts off.
So, I can go ahead and apply.
So, let's say I want to find out V0 versus VI.
So, I care about finding out V0.
V0 is the voltage of this node with respect to ground.
OK, so it's a slightly more complicated circuit than you saw up here, than you saw up there.
So, let's go ahead and do this example.
Start by applying the workhorse of the circuits business, the Node method, and let's start with doing this for VI.
Let's first do it for VI greater than one, notice the behavior of this is different for different ranges of VI.
So let's first do it for VI greater than or equal to one and apply the Node method.
Node method says sum the currents going into this node; we know the voltage at this node.
It's VI.
We know the voltage at this node.
It's VS. OK, the only unknown is V nought.
And so, let's go ahead and write the node equations for that node.
So, the current going up, let me simply equate the current going up to the current that has been supplied by this particular node here.
And, that should equate that the two of them should sum to zero, the current going up plus the current going down should sum to zero.
So, I get V0 minus VS divided by R. That's the current going up.
Plus, the current going down must sum to zero, plus ID must sum to zero.
And ID is going to be K divided by two VI minus one all squared.
That must equal zero.
Straightforward application of Node method, current going up plus the current going down at this node should equal zero because the total current leaving the node must be zero, OK?
So I can go ahead and simplify this, multiply it throughout by, I call this RL here.
So, multiply it throughout by RL, and move all of this to the other side, so I get VS divided by RL, multiply it throughout by RL.
I get VS at this side.
I take this term to the other side.
This becomes a minus.
RL multiplies here, so I get KRL.
That's the expression I get.
V nought is VS minus KRL divided by two times VI minus one all squared.
Let me put a box around this because I will be referring to this more times in 6.002 for a variety of reasons than probably any other equation on Earth.
OK, this is the first time you saw it.
You saw it here.
OK, mark it down.
You'll smile every other time you look at it in quizzes, and you will find out why this comes up very often in 6.002.
So, I'll just give you a few seconds to savor this big moment in your 6.002 life.
All right, OK, so it's pretty simple actually.
I mean, there's really not much.
A lot of this stuff is just a plain old, simple application of the Node method, and things just fall out.
It's just so simple.
So, the V nought, I apply the Node method, I get V nought for this nonlinear circuit.
I can also it for VI less than one.
For VI less than one, when VI is less than one, what happens?
ID is zero.
OK, since ID is zero, think of this as an open circuit.
OK, so there's no voltage drop across RL.
And, this voltage V nought is equal to VS.
So, I like to see things in pictures.
I'm not an equations kind of person.
I'm much more of a graphical person.
So, let me draw a little graph to show how V nought, to see the form of V nought, and then let's study that little system a little bit more carefully.
So, this is page seven, and we plot V nought versus VI for you.
And let's take a look at how this really simple circuit looks.
This has got nothing.
It's got an RL resistor connected to a supply, and a dependent current source, and I apply some voltage VI at the input.
It's a very, very simple circuit.
So, let's see.
So as long as VI is less than one, the output stays at VS. OK, that makes intuitive sense, right?
As long as the current here is zero, this is like an open circuit here.
If this is an open circuit, then effectively, V nought is simply the voltage VS. V nought simply appears here.
If you want to grunge through KVL and KCL, go ahead.
VS minus RL times the current is V nought, and the current is zero so it's, yes.
So, this is simply VS.
When VI goes above one volt, fun stuff begins to happen.
OK, when V nought goes above one volt, then this equation applies because VI is greater than one.
This equation applies.
And, when VI is a one, one minus one is zero.
This term cancels out, so this is VS. OK, phew!
So, I start off here.
As VI increases, what happens now?
As VI increases, this term here becomes increasingly negative, OK, subtracting from VS. OK, so I get some behavior like this.
V nought begins to drop.
And it makes intuitive sense, right?
As ID begins to increase, the voltage here will begin to drop because I'm drawing more and more current through RL.
I'm dropping more and more across RL.
So more and more drops across RL, so V nought begins to drop too.
So, it looks something like this.
I'll show you a little demo, but my claim is that you have just seen an amplifier.
Whoa.
You just saw an amplifier.
So, I snuck an amplifier by you, OK?
So, I just snuck an amplifier past you.
I'll show you why in a second.
So, let's take a look at this waveform here.
Let's not worry about what happens way down here.
We'll talk about that a little later.
But, look at this curve here.
I claim there is amplification in the following sense.
Focus on some change in the input voltage, delta VI, OK, and for that change in input voltage, I get some change in the output voltage.
OK, for some change in the input voltage, delta VI, I get some change in the output voltage.
And guess what?
In this, at least the way I have drawn it, delta V nought divided by delta VI, if I can find regions of the curve where this is greater than one, then I have amplification.
OK, so what's that saying?
What that's saying is that if I apply some voltage here, OK, and I change that voltage by a small amount from, let's say, 2 V to 2.1.
OK, I am going to find the output voltage.
Let's say I go from 2 V to 2.1 here.
OK, abstractly out there, I might have an output that goes from three to, let's say, two V perhaps.
OK, so for a 0.1 change here, I'm going to get a bigger drop here, so from 3 V to 2 V, giving me an amplification in this little circuit.
OK, so we'll see this again and again, and you'll really understand it.
So, I have a small change in the input, and I have a corresponding larger change in the output.
So, I've shown you an amplifier.
I haven't shown you a linear amplifier.
There's an extra charge for that.
OK, that'll happen later.
OK, all I've shown you so far is an amplifier, and this happens to be a crummy amplifier.
It's a nonlinear amplifier because, notice, this is not linear.
It's a nice little curve, and so it's not linear.
But, I promised you an amplifier, and I'm cheap, and that's all you get for now.
OK, we'll see linear stuff later, but for now, I have a little amplifier.
So, let's do some real numbers, and plot some numbers down, and also look at a demo.
So, let's do an example.
Let's say VS is 10 V, that the K is two milliamps per V squared, and let's say RL is five kilo-ohms, OK?
So, let me substitute these values into that equation, and I get V nought is, VS is ten.
So, it's ten minus, KRL divided by two.
So, K is two milliamps.
Two milliamps times five kilo-ohms is ten divided by two gives me five, and VI minus one squared.
That's what I have.
I just plug in a bunch of numbers, and that's what I get.
So, what I'll do is let me just do a little table for you, and plot using real numbers, simply plot those values for you.
All right.
Let's get started.
I guess this watch is a couple minutes fast.
First a quick announcement.
In case you have forgotten, your lab notebooks are due tomorrow with the post-lab exercises for the first lab.
OK, so I am going to continue with amplifiers today.
And to just give you a sense of where we headed, we have this five lecture sequence covering different aspects of amplifiers with dependent sources and showed how we could build an amplifier with it on Tuesday.
Today I am going to show you a real device that implements a dependent source.
And then next Tuesday we will talk about analysis of an amplifier.
Wednesday is our quiz.
Thursday and the Tuesday after that we then talk about small signal analysis and small signal use of the amplifier.
Today we will talk about the MOSFET amplifier.
So let's start with a quick review.
And in the last lecture, I showed you that I could build a amplifier using a dependent source.
And a dependent source worked as follows.
Let's say I had a circuit and I connected a dependent source into the circuit.
Let's say in this example I have a current source.
So this is some circuit.
And the current i is a function of some parameter in the circuit.
That's why this is a dependent source.
This is a dependent current source.
So it could be that I have some element inside.
And I measure, I sample the voltage across the element or between any two points in the circuit.
And, in this little example here, this current could be dependent on that voltage.
So notice that although I showed you the two terminals of the dependent source that carried a current, there is another implicit port, another implicit terminal there.
And that terminal there is called the "control port" of the dependent source at which I apply a voltage or current that will control the value of the current source.
As a quick aside.
There is a small glitch with the tools in your tool chest.
We talked about the superposition technique where you were taught to turn on one source at a time, for a linear circuit one source at a time, and then sum up the responses to all the sources acting one at a time.
Well, what do you do about dependent sources?
A dependent source is a source.
And we have to modify the superposition statement just a little bit.
And for details you can look at Section 3.5.1 of your course notes on the details and some examples on how to do this.
So the approach is very simple, actually.
The approach is, for the purpose of superposition, to not treat your dependent source as sources that you turn on and turn off.
So what you do is when you do superposition with dependent sources simply leave all your dependent sources in the circuit.
Just leave them in there and turn on and off only your independent sources.
So look at the response of the circuit by turning on your independent sources one at a time and summing up the responses.
And your dependent sources stay within the circuit and simply analyze them as you do anything else.
So essentially what it says is that just be a little cautious when you have dependent sources, but the basic method applies almost without any change.
The readings for today's lecture are Section 7.3 to 7.6.
So since we are going to build up on the dependent source amplifier, let me start with a quick review of that amplifier.
We built our amplifier as follows.
We connected our dependent source in the following manner.
And the current through the dependent source in the example we took was related to an input voltage vI.
So some voltage vI.
And so these two were the control port of the dependent source and a vI was applied there.
And I showed you a simple amplifier built with a dependent source that behaved in this manner.
And again I will keep reminding you, just remember that the dependent source is actually this box here, the control port and the output port.
And commonly we don't explicitly show the control port for those dependent sources for which the control port does not have any other affect on the circuit, like it doesn't draw any current or things like that.
So in this particular example we said that this behaved in the following manner for vI greater than or equal to 1 volt and iD was zero otherwise.
So we can analyze the circuit to figure out what vO is going to look like.
And a simple application of KVL at this loop here, again, you know, when I say this loop here, I am pointing at something here.
That is the VS source that is implicitly across these two nodes.
Again, this is a shorthand notation where this little up arrow here implies that I have a voltage source connected between these two terminals here.
And so there is a loop here that involves VS.
So Vo is simply VS minus the drop across this resistor.
So it's VS minus the drop across this resistor gives me vO.
And the drop across the resistor is simply iD RL.
iD is the current here and that's the drop across the resistor.
And I could get the explicit relationship of vO versus vI by substituting for iD as vI minus one all squared.
So vO relates to vI in the following manner.
Nothing new so far.
I have pretty much reviewed what we did the last time.
Here is where we take our next step forward with some new material.
Up to now I have talked as a theoretician would where I said just imagine that you had spherical cow or something like that.
Here I just asked you to imagine this ideal dependent source, control port and an output port, and it behaved in this manner.
So as a next step what I would like to do is show you a practical dependent source which turns out to be a little bit more complicated than this idealized dependent source that I showed you in many dimensions.
Real life tends to impose a bunch of practical constraints on you, and we will look at those in a second.
If I could find a dependent source that looked like this -- We had a control port A prime and output port B prime.
And I looked at some examples where the current through the dependent current source was some function of the input voltage.
This is a "voltage controlled current source".
What I am going to do is talk about a device that can give me this behavior or some close approximation to it.
It turns out that under certain conditions the MOSFET that you have already looked at behaves in this manner.
The MOSFET that you've seen sort of behaves like this.
And let me show you under what conditions the MOSFET behaves in that manner.
Let me create some room for myself.
Notice that I need a control port, needed an output port.
And I am going to view my MOSFET in a slightly different manner than you have seen before.
I draw these two terminals here.
And this was a three terminal MOSFET.
This was my drain, my gate and my source terminal.
It was a three terminal device, but what I do is I view the MOSFET slightly differently.
I will just use this terminal to be common across both the gate and the drain.
And so this voltage here is vGS.
I am just using the source port, the source terminal along with the gate as a terminal pair.
I am using the same source along with the drain as another terminal pair.
So I have a vDS out there and I have some current iDS that flows out here.
Notice that when I view the MOSFET in this manner I have accomplished my first step, which is I seem to have a box which has a port here and a port here.
And I also explained to you that a MOSFET behaves in a particular manner.
For one, the output port behaved as an open circuit under certain conditions when -- This was vGS, G, drain and source.
When vGS was less than a threshold voltage VT this MOSFET had an equivalent circuit that looked like this.
So when vGS was less than some threshold voltage VT then there was an open circuit between the drain and the source.
And you saw this before.
So far nothing new here.
However, when vGS is greater than or equal to VT -- vGS was greater than VT.
The MOSFET behavior we looked at earlier showed that this behaved either like a short circuit in the simplest form or in a slightly more detailed form it behaved like a resistor.
We call that the SR model of the MOSFET.
So when vGS was greater than VT we said that a simple way to approximate MOSFET behavior was to view this as a resistor connected between the drain and the source.
That was our SR model use of the MOSFET.
It turns out that we kind of lied.
We were sort of looking at the MOSFET in a really funny way.
And I shone the light on the MOSFET in a really, really clever way.
Well, I shouldn't say clever.
A really, really tricky way.
And tricked you into believing that it was just a resistor.
And we constrained how you use the MOSFET.
So that behavior was indeed a resistive behavior.
But it turns out that in real life the behavior of the MOSFET between the drain and the source terminals is much more complicated than the limited form in which you saw it.
So today what I am going to do is take the wraps off the complete MOSFET and show you its full behavior in all its gory glory.
And I will spend a bit of time on that to clearly emphasize under what conditions the MOSFET behaves like a resistor, as you saw when you did digital circuits, or behaves differently in other domains of use.
Let me pause for a second and leave this space blank here.
And let's do some investigations.
Let me leave this here.
I won't draw in anything yet.
You will figure out what it looks like yourselves under certain conditions.
What I will do next is apply some voltages on a MOSFET and observe the current versus vDS behavior and plot that on a scope and take a look at it.
What I am going to do -- -- is figure out what iDS looks like for -- Remember iG into the gate for 6.002 is always going to be zero.
In much more detailed analyses of the MOSFET, in future courses you may see slightly more complex behavior.
But as far as we are concerned it is an open circuit looking into the gate.
So I am going to apply a vGS across the MOSFET, apply a vDS across the MOSFET and plot iDS versus vDS.
First let me show you what you already know.
What you already know -- This is vDS.
I will just keep doing as much as I can of what you already know.
And then when I do some new stuff I will tell you explicitly.
You've seen this before.
The MOSFET behaves like an open circuit when vGS less than VT. That is when vG is less than a threshold voltage VT, I have zero current flowing through the MOSFET.
And when vGS was greater than VT then the S model of the MOSFET the switch model simply said that look, we can model the D2S as a short circuit.
You saw this in your labs and you saw that it was a very, very small resistance between the drain and the source and it kind of looked like a short circuit.
But then we said well, that's not quite it.
There is some resistance.
And so we said a slightly more accurate model would have this line droop a little bit to imply that there was some resistance R_on between the drain and the source, so vDS iDS.
So this was when vGS less than VT and vGS greater than or equal to VT.
I have some resistance.
And that showed me a straight line kind of like behavior.
And I showed you that behavior.
So far absolutely nothing new.
Now what I have plotted there for you is that behavior.
Up here notice that this is the vDS axis, this is the iDS axis.
I am plotting iDS versus vDS.
And when vGS -- The gate voltage is more than a threshold, notice that I see what looks like something more or less like a straight line.
And this is a straight line with some slope, more or less a straight line implying resistive behavior.
And we also had some fun and games here.
We said hey, what if I turn vGS off?
Boom.
That would be my iDS of zero implying that the MOSFET behaved like an open circuit between the drain and the source.
I applied a positive vGS more than VT and it began to look like a resistor.
Open circuit, resistor, open circuit, resistor, OK?
Up until now nothing new.
So you shouldn't have learned anything at all that is new until now in today's lecture.
Now watch.
What I am going to do is, as I said, I kind of lied all this time and I just showed you this behavior.
And what I have been doing all along is very carefully using a very small value of vDS.
Notice it's a small values of vDS.
I haven't told you what it looks like as vDS increases.
Well, let's go try it out.
We have a scope here.
We have the MOSFET here.
Now, I am not sure what is going to happen now.
You may see smoke or have an explosion, who knows what?
But look up there for a second.
I am just going to increase vDS and you can figure out what happens for yourselves.
I increase vDS.
Whoa, what a liar.
Agarwal is a liar.
I have been kind of tricking you.
I have been putting -- Covering up all this part here and showing you just this region of the curve for small values of vDS.
But as I increase vDS this is nothing that looks even close to that of resistive behavior.
So what's happening here?
What's happening is that as I increase my vDS the iDS curve tails off and saturates at some value of current.
Notice it saturates at some value of current.
And so I am going to look at this region of behavior.
Notice that what we have looked at so far was the behavior for small vDS.
It kind of looks resistive.
But when I pump up the vDS, really whack this node really hard with a much larger vDS the guy says, oh, I give up.
And the current saturates out and flattens out and holds the value steady at some value.
So what's that behavior look like?
What is my horizontal line above the X axis in terms of V I elements?
What is that behavior like?
Current source, exactly.
So this is current source like behavior.
And so let me start by drawing you a little model and explaining it in more detail.
What happens is that under certain conditions, and the conditions are the following, when vDS, that is my drain to source voltage is greater than or equal to vGS minus VT.
When my drain voltage goes above vGS minus VT, so if vGS is 3 volts and if VT is 1 volt, then if vDS goes above 2 volts, if I am hammering the drain of the MOSFET with a higher voltage then this guy says I give up, can't show you nice restive behavior, and the current saturates out and it doesn't allow you draw any more current than a maximum value.
And that's the current source behavior.
This one behaves like a current source.
And the current iDS is given by the following expression.
The current is given by iDS is equal to a constant K divide by two times (vGS-VT) all squared.
Kind of reminiscent of the carefully chosen dependent source example, just that this one here is VT.
This model, which applies when vGS is greater than VT, the MOSFET has to be on and the drain to source voltage in the MOSFET must be larger than some value, and that value is vGS minus VT then this guy begins to behave like a current source.
This model of the MOSFET is called the "switch current source model".
So in the region of the MOSFET characteristics where vGS is greater than VT and the drain to source voltage is larger than vGS minus VT, the MOSFET behaved like a current source between its drain and source terminals.
And in that part we model the MOSFET as a current source.
And so not surprisingly that part of the model is called the SCS model in contrast with the SR model where we had a resistor.
Again, remember, this is not meant to be conflicting.
It is not like gee, how can the MOSFET look like a resistor, and then suddenly what happens it becomes a current source.
Well, the two regions are different.
It is not that it is behaving as a current source for the same parameters, no.
When vDS is less than this right-hand side it does behave resistive.
The SR model applies.
But increase vDS beyond a point, the current saturates and the SCS applies like so.
So let's draw.
The SCS behavior can be drawn here vDS and iDS.
As I mentioned to you, for small values of vDS, let's say I pick some value of vGS, let's say vGS3, some value vGS, it is going to look like a resistor until vDS becomes equal to vGS3 minus VT. And after that it saturates out and begins to look like a current source.
And this point is where vDS becomes equal to vGS minus VT. And this way is when this equal sign becomes a greater than sign, vDS becomes larger then I move into this part of the curve.
Similarly, for various other values of vGS it will look like this -- -- and so on.
And it behaved like an open circuit as before when vGS less than VT.
When vGS less than VT it is still behaving like an open circuit.
And so as I increase my vGS, provided I keep my vDS greater than vGS minus VT, I get current source like behavior.
And notice that this is increasing vGS.
I have purposely drawn these curves at greater distances from each other to imply that it is a nonlinear relationship in that if I increase vGS by some amount, the increase in vDS is related to the square of vGS.
It is vGS minus VT all squared.
So I get a family of curves of that look like this.
And this is in the region of operation where vDS equals vGS minus VT. And this applies in this regime where vDS less than vGS minus VT.
This region of operation is called, as you might expect, the "saturation region".
We say the MOSFET has been hammered, the MOSFET has been walloped, the MOSFET is in saturation.
So the MOSFET is in saturation.
This region, corresponding to this, is called the triode region.
This is really very simple.
All we are doing is saying that when vDS is increased beyond a certain limit, given my vGS minus VT, the MOSFET begins to behave like a current source.
It cannot draw any more current.
It limits the current to a given value like a current source.
But on the left-hand side of this it behaves in a resistive manner.
So what I would like to do is -- What I will do is, we've plotted for you, for the MOSFET, all its characteristics in its full glory for a whole bunch of values of vGS and a whole bunch of values of vDS.
And let me stare at those curves with you for a few seconds and walk you through them.
So what do I have here?
One of these curves corresponds to a given value of vGS.
This may be vGS equals 2 volts.
This is vDS, the drain to source voltage, and this is the current.
So focus on this curve for now.
In the beginning I hid the right-hand side behavior from you and showed you just the resistive behavior out here.
When I increase vDS to be much larger the curve saturated and I got the saturation region operation of the MOSFET.
And notice as I increase my value of vGS the saturation current also increases according to a square law behavior.
So these are the entire curves of the MOSFET.
Finally the truth comes out.
And notice that when vDS is less than vGS minus VT, I have more or less resistive behavior.
But when vDS is greater than vGS minus VT I get current source like behavior.
So one question you may ask is when do I use one model or the other?
When do I use the SR model and when do I use the SCS model?
If you want to do a real detailed analysis then you can use the SR model when vDS is less than vGS minus VT. And you would use this model when vDS is greater than or equal to vGS minus VT. That is simple enough.
In 6.002, to eliminate confusion we constrain how we look at things a little bit more stringently.
And what we do is that for our entire digital analysis, for the entire digital world we focus on the SR model.
And I will tell you why in a second.
So for all digital circuits, invertors, look at power of invertors, look at delay, a bunch of other things, we will be using the SR model in 6.002.
And I will tell you why in a second.
And for analog -- That is for amplifier designs and situations like that, we will be operating the MOSFET in a saturation region.
And I will talk about that in a second.
What I am saying here is that in 6.002, when we do analog designs, we are going to discipline ourselves to using the MOSFET only in this region.
We are going to constrain ourselves to play in only this region of the playground where vDS is quite large.
Why?
Because I am asking you to.
I am saying let's play in that part of the playground and keep your vDS high.
And so the MOSFET is going to be operating somewhere in here.
So we can apply just the SCS model, just the current source behavior in that region.
There is another important reason, which I will get to in a second.
And for digital designs we will simply use the SR model.
And it turns out that this is realistic because in the digital designs that you have you seen and will be seeing in this course, the pull down MOSFET is on, or when these pull down MOSFETs are on, the output voltage is pulled down close to ground.
So vDS is very, very small.
So it does make sense that this model apply.
And when we talk about amplifiers, I am asking you to follow this discipline.
I will tell you why in a second.
I am saying analog designs follow this discipline that I call the saturation discipline.
It says simply operate the MOSFET operating in saturation as a current source.
We will look at an amplifier in a second, and I will tell you why.
Now let's do a MOSFET amplifier.
Remember my amplifier had an input port and an output port.
And in general in our use we are going to have a common ground.
And we have a VS and a ground here as well.
That is the power port of the amplifier.
The input port and the output port.
And let me redraw the circuit putting a MOSFET in place of the current source, RL, VS, vO, drain, gate, source, vI.
So my input is vI.
Again, the MOSFET output is vO.
And I have a resistor RL.
Hey, we've seen that before.
It turns out this is not surprising.
You've seen this before.
This was our primitive inverter circuit.
So what's different here?
We showed you the circuit as an inverter.
What's different here is that when we look at MOSFET behavior as a current source, this behaves like an amplifier.
In other words, when vDS is greater than some value then this behaves like a current source.
When vDS is small, in other words, in the digital design when vDS was small here, because when the MOSFET was on it pulled the voltage down to ground, we could view this behavior as a resistor.
And exactly the same thing, it is an amplifier.
And with digital designs, I was driving it with 5 volts and 0 volts and that was it, rail to rail.
As an amplifier, what I am doing now is looking at a small region of its behavior when vDS is greater than vGS minus VT. What I am saying is that for amplification let's follow the saturation discipline.
And the reason is that when this behaves like a current source, what I have shown you is that if this behaves like a current source I have shown you that this expression up here gives you amplification.
In last lecture we plotted a bunch of values for vO versus vI, and we saw that we were getting amplification.
For a small change in vI, I was getting a larger change in vO, and that was when I had the equation for a current source in there.
And so we know for a fact that if I can operate this as a current source, with a reasonable choice of values here, I am going to be able to get amplification.
What I haven't told you is if this is operated in the linear region, in fact, you do not get amplification.
I won't cover that, but you can check that out in your course notes as a discussion or you can try it out for yourself.
Replace this with the SR model for small vDS and you can show yourselves that you don't get any amplification.
In order to get the amplification we are telling ourselves let's focus on this part of the playground where vDS is greater than or equal to vGS minus VT. And for vGS greater than or equal to VT.
So when vGS is greater than VT the MOSFET is on.
Further, when vDS is large, larger than vGS minus VT this behaves like a current source.
So we have now created a small playground for ourselves where we can build lots of fun little amplifiers and other circuits.
And provided our circuits follow the saturation discipline where for the MOSFET or MOSFETs in the circuit these expressions are true then the MOSFETs are going to be in saturation, the current source model applies, and I will be indeed getting saturation.
In future courses you may actually see the MOSFET used in other regimes of operation for a variety of reasons.
But in 6.002 when we talk about amplifiers and so on we will be adopting the saturation discipline.
And your homework problems and so on will state that.
Assume that the MOSFETs are in saturation.
What that means is that you can begin to model them as a current source and simply analyze their behavior accordingly.
One minor nit.
Note that vDS for the MOSFET is the same as vO.
And vGS for the MOSFET is the same as vI.
So if you see me jumping back and forth using vOs and vIs or vDSs and vGSs they are the same thing in this circuit.
If you are dealing with circuits with many MOSFETs then you will have vDS1s and vGS1s and so on and so forth.
But for this simple circuit, vO and vDS are the same, vI and vGS are the same.
So we could go ahead and analyze that circuit.
What I do to analyze the circuit, I am telling you this.
I am telling you that the MOSFET is behaving in saturation.
I am telling you this.
We have disciplined ourselves to say that in that circuit the MOSFET is in saturation.
As soon as we tell you that we can then go ahead and analyze that circuit.
And to analyze that circuit what you will do is simply replace the MOSFET with its equivalent model, and that looks like this.
Since you have been told that it is in saturation, we can replace the MOSFET with its current source model.
And the current iDS for the MOSFET is given by K/2(vI-VT)^2.
And it is always good to write the constraints under which you are implicitly working close by.
So the constraints are one, vGS is greater than or equal to VT, vDS is greater than or equal to vGS minus VT.
These constraints immediately follow from a statement of the type we are operating under the saturation discipline or the MOSFET is in saturation.
Let me just mark this equation as A, and we will refer to it again.
So with this new little circuit with the MOSFET working as a current source, let's go ahead and analyze our amplifier.
Notice that to analyze the circuit I have a current source.
It's a dependent current source where the current depends on the square of the input.
So I want to go and analyze it.
This is a nonlinear circuit.
So I can apply any one of the methods that we talked about last week for nonlinear circuits.
To analyze it I will go ahead and use the analytical method.
And my goal will be to obtain vO versus vI.
Again, remember where are we here?
The MOSFET circuit operating in saturation so I can replace this with a current source.
It is nonlinear.
And so I can apply one of the two methods, the analytical method or the graphical method.
Let's do both and start with the analytical method.
The analytical method simply says go forth, apply the node method and solve.
Simple stuff.
Let's go ahead and do that.
Node method.
I have a single node here that is of interest.
I know the voltage vI at this node.
I know the voltage VS at this node.
So the only unknown is here at vO.
So I will go ahead and do that.
Let me go ahead and equate the currents into the node to be zero.
So the currents out of the node here are iDS.
And that was equal the current into that same node.
So iDS must equal VS minus vO divided by RL.
iDS=VS-vO/RL.
For later reference, let me call that B.
Simplifying, what I can do is, we know that iDS is given by K/2(vI-VT)^2.
So I replace iDS with this expression and I multiply that by RL.
So I get K/2(vI-VT)RL.
So iDS gets multiplied by RL and I get vO on this side and VS remains out here.
All I have done is multiplied both sides by RL.
So it is RL iDS, taken RL iDS to this side, that is here, I get the minus sign, and VS stays here, vO comes here.
So that is my final expression.
Remember this is true under certain conditions.
I will keep hammering that home because some of the most common errors made by people is in forgetting the constraints under which this was obtained.
And the constraint under which this was obtained is the saturation discipline.
And that was true when vGS for a MOSFET was greater than or equal to VT and vDS for a MOSFET was greater than or equal to vGS minus VT.
I also know that for vGS less than VT, vO=VS.
So when vGS is less than VT then this one turns off.
That's why it is the SCS model, switch current source model.
When vGS is less than zero it turns off and VS directly appears at vO.
I would like to stare at this constraint with you for a second, vDS greater than or equal to vGS minus VT here.
And vDS is simply vO.
I want to rewrite this constraint in terms of iDS.
It will come in handy.
So iDS is K/2(vI-VT)^2.
This is vI-VT.
So vI-VT is simply square root of 2iDS/K.
In other words, I can write iDS less than or equal to K/2vO^2.
So this constraint expressed in terms of iDS is simply iDS less than or equal to K/2vO^2.
So all I've done here is analyzed this nonlinear circuit.
I can also analyze it using the graphical method.
And in order to do that, for my nonlinear circuit, in order to do that, all I have to do is plot.
Let's have iDS here and vDS here.
And as we did with a nonlinear expo dweeb, what I do is I plot the device characteristics iDS versus vDS.
The device characteristics under saturation look like this, so vGS increasing.
iDS versus vDS has a bunch of curves that look like current sources of increasing values.
That simply reflects equation A.
And then I superimpose on top of that the expression that comes up due to equation B which is iDS equals, let me write that down here, iDS equals VS/RL - vO/RL.
That's B.
And let me plot that.
That is a straight line relationship between iDS and vO.
And so when vO is zero iDS is VS/RL.
And when iDS is zero vO equals VS.
Remember, vO and vDS are the same.
So this is what I get.
This is the straight line corresponding to equation B here.
And, as before, we just find the point where the two intersect.
Let's say I am given some value of vGS.
And let's say I am given some known value of vDS.
So for that I can go ahead and find out the corresponding value of iDS from this graph.
Just as I told you when we did the expo dweeb stuff, this line here is called a load line.
You will be seeing that again and again and again where we have the equation corresponding to the one shown here, the equation written for the output loop superimposed on the device characteristics.
That's called a load line.
So I can get this point corresponding to the operating point of the MOSFET for this iDS, vDS and vGS by using the graphical method.
In the next lecture we are going to look at, given a device of this sort, how do we figure out the boundaries of valid operation so that the MOSFET stays in saturation?
Good morning, all.
Let's get going.
So I guess you had your quiz review yesterday.
I hope you guys didn't beat up on (name of TA) and who else was it?
(Name of TA) too much.
As you know, the quiz is tomorrow.
And unfortunately MIT couldn't give us one big room so we are broken up into three rooms.
And you will go to your room based on the first letter of your last name.
OK, so today we shall cover a topic called "Large Signal Analysis".
So in the last couple lectures we looked at one dependent sources abstractly, and then we looked at an amplifier built using a practical dependent source called the MOSFET.
Now, the MOSFET had to be operated in a given region of its operation in order to behave like a current source.
And while it behaved like a current source you would get amplification or a FET amplifier.
So that was in the past two lectures.
What you are going to do today is called large signal analysis, and this is a loaded term.
So large signal analysis means something very specific in our business, and I will describe to what that is.
This analysis involves looking at a circuit containing, for example, a MOSFET and figuring out how to get that device to operate in a way that the MOSFET was always in saturation.
So you had to figure out, based on parameters that you could control, how to establish those parameters so that the circuit operated in a way that the MOSFET was always in saturation.
So large signal analysis involves that.
And although the examples we use, use the MOSFET, the same kind of analysis can apply to any other device.
Remember, your MOSFET is a primitive element that we use as an example in this course.
There are other primitive elements that you can use.
The course notes, for example, discusses a couple other devices.
One is the "bipolar junction transistor", the BJT, and works through a complete example from start to finish involving a circuit containing a bipolar junction transistor.
And you can do a large signal analysis of that device as well.
It turns out that you need to operate that device in an interesting region of its operating space, and so you can conduct a large signal analysis of a circuit containing that device and figure out how best to operate that circuit.
So that is large signal analysis, and we will do an example and explain how this is done using an example today.
So to quickly review where we have been so far, we looked at this little structure here, our MOSFET amplifier.
Notice that when I write a voltage at a node, that's a short form for saying I am looking at the voltage between the ground node and the node at which the voltage is written down.
So VO here and VI applied here.
This is a very, very common circuit that we use.
To emphasize one more point, in general, in the kind of circuits we look at both in this course and in real life, there are a few patterns that we use very commonly that keep repeating all the time.
Very often you don't have to look at every possible permutation and combination of how things could be connected.
This sort of connecting thing is very, very, very common.
And you will see a lot of this pattern.
And we do the equivalent circuit for this.
In the equivalent circuit we replace the MOSFET with a dependent source provided this operated in the saturation region.
So I will just say while operating under saturation the equivalent circuit would look like this, VO, VI.
And IDS for the dependent source was given by K/2 (VI-VT)^2.
So this was an amplifier.
Here was the equivalent circuit while this device was in saturation.
And to operate in saturation, I said that certain properties need to be true for the MOSFET.
And there are two properties that need to be true for this to be operating in saturation.
One is that its gate to source voltage needs to be greater than VT, so VGS for the MOSFET should be greater than VT. And the second one was that the output voltage needed to be greater than the input voltage minus one threshold drop.
And this was the same as VDS for the MOSFET, this was the same as VGS for the MOSFET.
So what are we really saying here?
What we are saying is that look, we built this circuit using a MOSFET, and it is up to us as engineers to choose its operating points in a way that these two properties hold.
For example, to make the first condition true, I can discipline myself to operate such that VI is always greater than VT.
Similarly, I can choose VS, RL and VI in a way that this condition is true, which says that the drain to source voltage across my MOSFET drain and source should be greater than VI minus VT. As an example, if VI was 2 volts and VT was, say, 1 volt, then what I am saying is that VO should be greater than or equal to 2 minus 1 or 1 volt.
So I need to keep this high, 2, 3, 4, 5, whatever, a high voltage so that this guy stays in saturation.
The relevant readings for the material that we are going to cover in the course notes are in 7.5.1 and 7.6.
So that is pretty much a review of where we were.
We said we could build an amplifier.
Its equivalent circuit was shown on the right.
And, provided that, I discipline myself to operate in the saturation region or to have the MOSFET operating in the saturation region, then this would work like an amplifier and all would be good with the world.
So today -- -- we look at large signal analysis of a circuit.
And an example would be this circuit up here containing a MOSFET.
And, again, as I mentioned earlier, a large signal analysis is a loaded term in 6.002, or for that matter in circuits.
And large signal analysis involves two steps.
The first step involves writing down the transfer function of your little circuit.
In our case, VO is the output, VI is the input, so involves writing down VO versus VI.
Simply write down the transfer function.
In other words, the relationship between the output and the input for that circuit.
And, in our case, again, we've disciplined ourselves to adhere to the "saturation discipline".
And the second part of large signal analysis is to find out the valid input operating range.
Find out for the given circuit parameters, let's say I apply a VS and I use some value of RL and I use a given MOSFET, which has a given value of VT.
The question then is that what is a valid set of input voltages that would operate the circuit in a way that I would be in saturation.
And so find out the valid input range, and this would give me a corresponding output range -- -- for saturation operation of the MOSFET.
That is what we will dwell on in the lecture today.
So what we are saying here is that if I am careful with how I apply VI for a given value of RL and VS and for a given choice of my MOS transistor then I can stay within saturation provided I select my input voltages carefully.
And the analysis that we will go through today will figure out what that range of input voltages is.
And, again, I will use this as a motivating example, the MOSFET amplifier.
But in general large signal analysis would apply to any other circuit as well.
For example, in recitation you may learn about other circuits containing a MOSFET.
And you can do a large signal analysis of other circuits containing a MOSFET or you might learn about some other devices like the bipolar junction transistor, and you could do the same kind of analysis for that device.
So remember that the MOSFET amplifier here is an example.
I will be using that as a driving example to explain large signal analysis.
So the first step, as I mentioned earlier, is to get the VO versus VI.
And in general for some circuit that you build the output will not even be a voltage.
There are certain circuits where the output might be some kind of a current.
Let's say I am building some kind of a circuit where I would like the output current or the current through some edge of the circuit to depend on some input.
In that case the transfer function would be the output current versus VI.
And if I had an input current here it would be output current versus input current, you know, whatever the given problem tells you.
So this is under the saturation discipline.
And I will not rederive this for you.
You can apply a good old technique like the analytical method.
Or you can use the graphical method to get the appropriate answer here.
I wanted to point out in a quick aside that why do we care about graphical analysis?
Once you have the analytical method, why do you care about the graphical method?
And a student asked me a question after lecture last Thursday, and it occurred to me that it's not obvious why you need the graphical method.
So it turns out that often times you do not have an equation describing the device.
So let's say, for example, I am a manufacturer.
Let's say I am AMD.
As AMD I sit down and my semiconductor division builds a MOSFET.
And when you build a MOSFET your experiments and your fabrication division often times doesn't give you an equation with the MOSFET.
They build something and then you look at it and you experiment with it.
You apply various input voltages and you measure currents and output voltages and so on.
And so what you end up getting is a graph that describes the behavior of the MOSFET.
And you have seen this in your lab as well, your 2N7000 or was it 2000?
So your 2N7000, the MOSFET you use in the lab also gives you a data sheet.
And in that data sheet you see a bunch of curves.
So very often devices come with data sheets.
And when you have a data sheet but no equation then you can apply the graphical method and solve your circuits.
In this example, assuming I can apply the analytical method, here was the expression that I had derived for you in the last lecture.
So VO was related to VI using the square law relationship.
And we can plot and do other fun stuff with this equation.
So here is the input voltage VI.
That is my VT.
So notice that VO is VS.
This is true when VI greater than or equal to VT and VO greater than or equal to VI minus VT.
So these are the constraints of the saturation discipline.
And in our particular situation when VI was less than VT output would simply be VS.
If VI is less than VT the MOSFET would turn off, switch off, and I would have no current flowing through RL, and VS would appear at the output.
So until VT, I have VS, and then following that I get the square law behavior articulated by this equation.
And this was simply VS-K/2 (VI-VT)(RL^2).
So that's the first part.
You have seen this before.
The transfer function shows that I have a square law dependence between VI and VO.
So now I can embark on the second step of my large signal analysis, and my goal is to find the valid input operating range.
So what does that mean?
What I am looking to do here is, for this little circuit, is drain, source, gate, VI, VO, RL and VS. What I am looking to do is that given the value of the supply VS, RL and a MOSFET, in our case given a MOSFET implies that it is a given value of K and a given value of VT for that MOSFET.
So what I am going to do is find out, let's assume VI is my free variable here.
So my goal will be to find out the range of VI for which this device stays in saturation.
And I will use a couple of methods to do that, and I will use both a combination of a graphical method to give you intuition and then apply analytical analysis to get down to specific answers.
So let's start with the intuitive part.
So here is VI, VO.
I will use the transfer curve VO versus VI to help build intuition here.
So that is what it looks like.
So the first step, looking at this graph, we know that this point here, that VI needs to be greater than VT to satisfy the first equation.
Let me just write down both equations here.
So VI greater than or equal to VT is one of them, and VO is greater than VI minus VT is a second equation.
And just remember that this is the same as VDS and this is the same as VGS.
So VI must be greater than VT for the MOSFET to turn on.
And so therefore the valid operating range starts at this point and is somewhere up here.
So the first part is pretty easy.
Somewhere here -- Somewhere at that point, my output voltage VO.
I'm not quite sure what that point is.
My output voltage VO, as this keeps falling down, my output voltage VO goes lower than one threshold below VI.
And at that point it goes into its triode region, and I need to find out what that point is.
So somewhere here I go into my triode region and begin to show a different behavior than the amplifying square law relationship there and go into my triode behavior.
So I need to find out what this point is.
Once I find out what that point is then this will be my valid operating range.
So let's figure out what that point is.
At that point the following is true.
Certainly VI is greater than VT. And at that point the output goes below one threshold, the input minus one threshold.
So at this point the following is true, VO is equal to VI minus VT. At that point the output voltage is equal to the input minus VT. And if the output goes lower then it will violate this equation here.
It is no longer greater than that number.
So how do we find out what this point is?
The principle intuition.
Let's draw some lines here.
Let's assume that VI and VT use the same scale, say, volts.
So if I draw a straight line at 45 degrees then that is a curve representing VI equals VO.
We all know that.
No big shakes.
So the line at 45 degrees here is the line at which VI equal VO.
And if I take that line now, the VI equals VO line, and I begin translating it to the right.
So let's take a line here.
Let's take a line there.
That line will be simply equal to VO equals VI minus VT.
I have translated that to the right.
And so this line is simply VO equals VI minus VT.
So this line is a locus of points at which VO is equal to this value.
This minus VT shows up as a translation to the right.
So I take my VO equals VI line, translate that to the right and it becomes VO equals VI minus VT.
Elementary geometry 101 or whatever.
So what do we have here?
Above this we have the condition VO greater than or equal to VI minus VT, and below that we have VO less than VI minus VT.
If we look at this graph here, this is the valid input operating range.
Starting at this point greater than VT, and at this point my output equals VO equals VI minus VT.
This must be the valid operating range for the input here to here.
And correspondingly the outputs are from here to this point to here like so.
So this is my valid input operating range and this is my valid output operating range or the corresponding valid output operating range.
So what does this say?
What this is saying is that if I, as the designer of the circuit, am disciplined enough to apply inputs that are in this range, VT to some value here, graphically shown here.
Then my MOSFET will remain in saturation.
And correspondingly my outputs will go between VS and some value here.
So hopefully that gives you some of the intuition behind how we get it.
And let's continue.
Let me label this point X.
So continuing with two to get the valid operating range.
I have shown you intuitively where that point is, but what I will do next is actually compute that for you.
It is a pretty simple computation.
Note that point X is the intersection of two curves VO equals VI minus VT. And the second curve is VO equals VS minus K divide by 2, VI minus V2 all squared RL.
So the point X iss at the intersection of these curves, and I can very easily get that as follows.
What I will do is I will simply substitute for VI minus VT from this equation here and then solve for it, so I get VO equals VS minus K divide by 2 VO squared RL.
And so this gives me a quadratic in VO.
And I can solve for it pretty easily.
And I get for a quadratic AX squared plus BX plus C equals 0.
The solution is given by VO is minus B plus/minus square root of B squared minus 4AC divided by 2A.
And so I am just going to get those numbers here.
So the coefficient of VO, that is B, is minus 1.
Take the positive root because we are up in the positive voltages here.
And square root of B squared, that is 1, minus 4AC.
So I get a plus 4 times K divide by 2 RL.
And 2A is simply 2 times K divided by 2 times RL.
So that is what I get.
That gives me VO.
So it tells me that VO, at the point where the output just equals one threshold drop below VI is given by the other circuit perimeter such as VS, RL and so on.
Oh, I am missing a VS here.
I just forgot the VS up here.
That is my VO.
So what is VI equal to?
Remember that at this point VO equals VI minus VT, so VI is simply VT plus -- I have not taught you anything earth shattering here.
I have just done some grubby math here to solve these two equations.
So this is a straight line at 45 degrees from VT and this is the transfer function.
And I need to find the intersection.
And the intersection is given by this point.
So that point, VI being VT plus something, is simply the second dot on the X axis.
So therefore I am pretty much done.
My valid input range for VI goes from VT.
So it starts at VT. That is where the transistor just turns on.
And then goes all the way to this point, VT plus minus 1 plus square root of 1 plus 2 K RL VS K RL.
So this is my valid operating range.
And again remember I won't dwell on this equation because, in some sense, you will get a different set of limits for other devices, for other circuits containing a MOSFET.
Or, for that matter, for other outputs that one may be focusing on.
So what is more important here is not so much the results that you see but the process that I have gone through.
So what is more important here is how did I get here?
And the way I got here was looked at the graph and said look, the MOSFET is in saturation in that regime.
And I am finding the bounding points of the regime of saturation operation.
So now, as an engineer, I can say that hey, look, if you build a MOSFET circuit like so, with a given value of RL, a given MOSFET and a given VS, then if you limit yourself you are operating with input voltages in this range thou shalt be happy.
If you go beyond that range then you will be violating the saturation discipline.
So the corresponding output range -- I can write the corresponding output range, and that goes from VS, when the input is at VT the output is at VS and goes from VS down to the input minus VT.
Which is simply minus 1 plus -- Let me go back and quickly show you a little MOSFET circuit, amplified circuit so you can stare at a real transfer curve yourselves.
And indeed convince yourselves that roughly at the point where proportionately shown in the curve up there the MOSFET indeed goes into its triode region and begins heading out of its saturation region.
Notice that here that is the same curve, the transfer function.
And the amplified output is at VS until input reaches a threshold voltage VT. And once input goes beyond VT the output begins to drop precipitously.
And at some point here this begins to go into its triode region.
And what I am going to do is simply increase the input voltage VI, and you will see that the output them begins to go into its triode region.
It keeps dropping.
And, as you can see, the output begins to go into a space where the gain is no longer more than 1.
And this is a triode region of MOSFET operation.
So the MOSFET is in saturation, things are going great.
As I increase my VI notice at some point I begin to go out of my saturation region of the MOSFET.
And somewhere here I go from the saturation region and transition into the triode region.
And this value shown here gives you the corresponding input voltage and the output voltage.
Other practical devices like bipolar junction transistors or MOSFETs and other circuits and so on can be subjected to a similar analysis.
And you can find out the valid operating regions for that device as well, or for that circuit.
So as a next step what I would like to do -- Out here I began by looking at the transfer function, the VO versus VI curve, and used that to drive the intuition behind how we calculated the bounding regions.
You can do the same kind of analysis intuitively looking at yet another curve, another set of graphs that you are familiar with, and that is a load line characteristic.
And it is interesting to get two interpretations.
And you can use whichever one you feel comfortable with.
So I will do two alternatively and show you another set of curves that you can use to get that.
Here I am going to plot IDS versus VDS, which is the same as VO.
This was the load line graph that we had seen earlier.
And, just for our reference, remember that VI must be greater than VT for saturation operation.
Similarly VO should be greater than or equal to VI minus VT for saturation operation.
Those are my limits.
The way we got the load line graph was we superimposed the load line equation over the device characteristics.
And so let me plot the device characteristics in the saturation region.
Remember that this constraint could be related to the current as I derived for you in the last lecture as follows.
IDS being less than or equal to K divided by 2 VO squared.
So in terms of my IDS versus VDS relation, this lateral constraint is equivalent to IDS being less than K by 2 VO squared.
So this is that equation.
So this line is IDS equals K by 2 VO squared.
And in this region I have the valid operating region where IDS is less than that quality.
So here are all my other curves for various values VGS.
So here are my devices curves, IDS versus VDS.
Remember that these curves come down like this, for the MOSFET, right?
Just that we focus on the right-hand side because that is where the MOSFET is in saturation.
And on this side the MOSFET is in its triode region, and we discipline ourselves not to operate the MOSFET such that it is in its triode region.
So those were the device characteristics.
And then I could plot my load line equation.
My load line equation, if you recall, was IDS = VS/RL - VO/RL.
This was simply obtained by writing KVL at the loop containing the output node and the supply VS. Notice there that VO is equal to VS minus IDS times RL.
And that is simply by dividing by RL on both sides and moving IDS to the left-hand side we get this equation.
And this equation gives rise to a curve that looks like this.
And what is this point here?
This point is where VO is 0.
So when VO is 0 my IDS is simply VS divided by RL.
And this point is obtained when IDS is 0.
And under those conditions VS and VO are equal so this is VS.
This is my saturation region and this is the triode region.
This was another interesting graph.
We often times fondly call it the load line graph.
So here is a load line superimposed on the MOSFET device IDS versus VDS curves for a variety of values of VGS.
So by looking at this curve, we can also intuitively determine the valid operating range.
So what are the two points here?
I will let you stare at it for a couple of seconds yourselves to figure out what two points here bound the valid operating range of the MOSFET, the valid operating range of the circuit.
I will start.
One is this point, because at this point the output is VS and VGS has just begun to equal VT.
So think about where the second point is for valid operation.
It is here, and, somewhere along that load line.
Remember the load line is a constraint that must be met by the output VO.
It is the constraint imposed by KVL on the output.
So the output is constrained to operate in this regime for various values of VGS.
So as the output keeps going from here all the way here, at some point I exit my saturation region.
And that other point is given by this one.
So notice that this is the curve that bounds.
On the left-hand side of this the MOSFET is no longer in saturation.
It is on the right-hand side, and so therefore this is the valid operating region.
Here to here.
This is good.
This is VS. That is good to know.
And for this point I know that VI, which is VGS, equals VT.
I know VO is equal to VS. And IDS, at this point, is 0.
So VO and IDS being VS and 0 correspondingly are the output operating perimeters when VI equals VD.
So that is one point.
And let's find out what this point is.
At that point I get my output just entering the range of the MOSFET triode region operation.
Notice that this point is the intersection of two curves, this line and this curve.
So this curve here is given by IDS equals K divided by 2 VO squared.
And this is my load line equation.
So that is VS divided by RL minus VO divided by RL.
That's it.
So I won't go ahead and solve that for you.
You can go and check it out and convince yourselves that if you solve these two equations and find out the VO for this, it should be the same VO that you obtained using the other graph.
What I have done here, obtaining the valid regions of operation is no different from what I did here.
The two are alternative approaches to getting to the same place.
Just that over the years what I have discovered is that there are one class of people that are output transfer function people, this graph, and another set of people that are load line people that like to think that way.
I have always been a transfer function person myself, but some of you may be load line people and so you can use that to drive your intuition.
It is pretty amazing.
As we get into this business and keep going down the path, it is amazing how some people really kind of get the load line thing and others feel much more comfortable with the transfer function.
So pick whatever you want.
So what we have so far is we have conducted a large signal analysis of a MOSFET amplifier.
It is an analysis of a circuit, and we found two things.
One is the transfer function under saturation operation, and we found the valid input operating ranges and the corresponding valid output operating ranges for the circuit.
In the last five or six minutes let me talk about a couple of other issues.
And the first issue is what we have done so far is intuitively and mathematically shown you what the valid regions are.
Now you are thinking that's fine, but how do I get there?
This region is good, VT through that other point, that's good, but how do I get there?
How do I make my amplifier operate in that region?
The answer is pretty simple, and let me drive the intuition again using a graph.
So this is a graph.
And I showed you that -- That was my valid region here.
Take a 45 degree line, find out where it intersects the transfer function, then this is the valid region here, VT through that coordinating function that we developed out there.
If I have an input that looks like so, some input whose gyrations fall within this range, will constantly keep the MOSFET in saturation.
And the corresponding output will look like this.
If my input is in this range, my output will be within this range.
And how do I get my input to be here?
Let's say I have a sinusoid that is 1 volt peak to peak or whatever.
How do I get my sinusoid up there?
Well, you have learned the trick on how to boost things.
Remember boost?
All you have to do is boost up your signal by some value capital VI.
And the way you do that is as follows.
VS, RL, VO.
What you do is you apply a DC offset to your input.
You take your sinusoid and boost it up so that all the gyrations of the input are in the valid range.
This is my input, some VA. Then I apply some DC offset capital VI given by this value here.
And boost up the interesting input?
My interesting input is the VA. And I boost it up by capital VI so that this guy is always in saturation.
I would like to show you a little demo now.
I am going to show you an input that is a triangular wave.
And what we will do is I'll play with a wide variety of offset voltages.
This guy is a triangular wave.
And what I am going to do is apply a triangular wave and we'll look at the output and convince ourselves that I get amplification when VI is big enough that the input goes into a valid operating range.
And we will look at a variety of ranges here.
You can put it a little larger.
OK.
So the triangular wave is my input.
And this is my output.
This looks nothing like a triangular wave.
And the reason is that I do not have the right offset.
So what I will do is gradually increase the offset on the MOSFET.
So at this point the offset is very low, a very small near zero offset.
And so therefore my output is a disaster.
My MOSFET is not in saturation all the time.
So what I will do here is apply some sort of offset.
Is this the one?
We want to switch.
This is the input.
You can see I am applying an offset by bumping and boosting up the input.
I don't have clipping happening at both ends, but I get something.
And I get amplification.
Now let me apply way too much of an offset.
With this offset I am kind of operating here.
What I will do now is apply an even higher offset so that this triangular wave begins to move here.
If I apply a very high offset what I am doing is overdriving the amplifier, boosting it so high that the MOSFET is going to go into its triode region and you are going to see that I won't have any gain.
My output is going to shrink noticeably if I overdrive the input.
You will notice the input going higher and higher.
Pull the trigger point down.
There you go.
Notice that as I boost up my input even higher notice that the output is a really small image of what the right input should be.
The right answer here, of course, is that I apply some right amount of offset to boost up the input into the right regime so that the output is seen to be some amplified version of this input.
So I showed you three things.
One is very little offset.
That was like so, as the thing comes down.
A very high offset, it gets killed again.
And the right amount of offset.
But notice that we still have a problem, even with the right offset.
The output is not linearly related to the input.
It is nonlinear.
And the answer to get a linear response is good old small signal stuff.
And we will be looking at the small signal part in the next lecture.
Good morning, all.
Good morning.
I hope you guys did not spend all of last night celebrating the Red Sox victory, but there is one more tonight.
OK. Let's see.
I trust the quiz went OK. What I will do today is take off from where we left off on Tuesday.
And continue our discussion of the large signal and small signal analysis of our amplifier.
Today the focus will be on "Small Signal Analysis".
So let me start by reviewing some of the material.
And, as you know, our MOSFET amplifier looks like this.
One of the things you will notice in circuits, as I have been mentioning all along in this course, is that certain kinds of patterns keep repeating time and time again.
And this is one such pattern.
A three terminal device like the MOSFET with an input and the drain to source port connected to RL and VS in series in the following manner, this is a very common pattern.
There are several other common patterns.
The voltage divider is a common pattern.
We keep running into that again and again and again.
The Thevenin form, a voltage source in series with the resistor is another very common form.
The Norton equivalent form, which is a current source in parallel with a resistor is also very common.
And it behooves all of us to be very familiar with the analyses of these things.
Voltage dividers in particular are just so common that you need to be able to look at it and boom, be able to write down the expression for voltage dividers.
I would also encourage you to go and look at current dividers.
When you have two resistors in parallel and you have some current flowing into the resistors to find out the current in one branch versus the other very quickly.
The expression is very analogous to the voltage divider expression.
And some of these very common patterns are highlighted in the summary pages in the course notes, so it is good to keep track of those and be extremely familiar with those patterns to the point where if you see it you should be able to jump up and shout out the answer just by looking at it without having to do any math.
So here was an amplifier.
And then we noticed that when the MOSFET was in saturation it behaved like a current source.
And this circuit would give us amplification while the MOSFET was in saturation.
So we agreed to adhere to the saturation discipline which simply said that I was going to use my circuit in a way that the MOSFET would always remain in saturation in building things like amplifiers and so on.
And by doing that throughout the analysis I could make the assumption that the MOSFET was in saturation.
I didn't have to go through -- Analysis became easier.
I didn't have to figure out now, what region is the MOSFET in?
Well, because of my discipline it is always going to be in saturation.
But in turn what we had to do was conduct a large signal analysis.
Again, in follow on courses you will be given circuits like this.
In fact, this very circuit with a very high likelihood.
And you will be looking at more complicated models of the MOSFET.
Or you will be given the MOSFET like this and, let's say in that course the designers do not adhere to the saturation discipline, in which case you have to first figure out is my MOSFET in its triode region or in the saturation region?
And depending on the region it is in you have to apply different equations.
So it is one step more complicated than in 002.
In 002 we simplified our lives by following a discipline.
And let me tell you that following a discipline is quite OK.
When it simplifies our lives and we can do good things with it, it is quite OK to do that.
We are not wimps or anything like that.
It is quite OK to have a discipline and agree that we are going to play in this region of the playground and build circuits in that manner.
By doing so, we could assume the MOSFET was in saturation all the time.
And analysis simply used a current source model.
By the same token, what becomes important is to figure out what are the boundaries of valid operation of the MOSFET in saturation?
To do that we conducted a large signal analysis.
And it had two components to it.
One of course was to figure out the output versus input response.
And what this usually does is that it does a nonlinear analysis of this circuit.
If it is a linear circuit it is a linear analysis.
And figures out what the values of the various voltages and currents are in the circuit as a function of the applied inputs and chosen parameters.
And the second step we said was to figure out valid operating ranges -- -- for input and corresponding ranges for the other dependent parameters such as VO.
You could also find out the corresponding operating range for the current IDS and so on.
So by doing this you could first analyze the circuit, find out the "bias" parameters, find out the values of VI and VO and so on.
And then you could say all right, provided, as long as VI stays within these bounds my assumption that this is in saturation will hold and everything will be fine.
The reading for this is Chapter And today we will take the next step and revisit small signal analysis.
In the demo that I showed you at the end of last lecture, I showed you an input triangular wave.
And the input triangular wave gave rise to an output.
And we noticed that we did have amplification, I had a small input and a much bigger output.
I did have amplification when the MOSFET was in saturation but it was highly nonlinear.
The input was a triangular wave and the output was some funny, it kind of looked like a sinusoid whose extremities had been whacked down and kind of flattened.
And its upward going peak had been shrunk.
So it was a kind of weird nonlinear behavior.
I will show that to you again later on.
And so it amplified but it was nonlinear.
And remember our goal of two weeks ago?
We set out to build a linear amplifier.
So today we will walk down that path and talk about building a linear amplifier.
So to very quickly revisit the input versus output characteristic, VI versus VO, this is VT and this is VS, this is what things looked like.
Also to quickly review the valid ranges, until some point here the amplifier was in saturation, the MOSFET was in saturation and somewhere here I had VO being equal to VI minus a threshold drop.
At that point the MOSFET went into its triode region and I no longer was following the saturation discipline.
So therefore this is my valid region of operation.
We also know that the output was given by VS minus K (VI-VT) all squared RL over 2.
Again assuming the MOSFET is in saturation.
It is very important to keep stating this because this is true only when the MOSFET is in saturation, when I am following the discipline.
Notice that this is a nonlinear relationship.
So VO depends on some funny square law dependence on VI.
The key here is how do we go about building our amplifier?
Take a look at this point here.
At this point here let's say I have a VI input.
Corresponding output is VO.
Focus is this point.
And left to itself this was a nonlinear curve.
Remember the trick that we used in our nonlinear Expo Dweeb example?
We used the Zen Method.
Remember the Zen Method?
We said look, this is nonlinear, but if you can focus your mind on this little piece of the curve here this looks more or less linear.
If I look at a small itty-bitty portion of the curve and I do the Zen thing, and kind of zoom in on here.
This looked more or less linear.
This means that if I could work with very small signals and apply the signal in a way that I also had a DC offset of some sort.
Then I would be in a region of the curve, I would be delineating a small region of the curve which would be more or less linear.
This was a small signal trick.
And what we will do here is simply revisit the small signal model.
Most of what I am going to do from here on will be more or less a repeat of what you saw for the light emitting expo dweeb.
Just that here I have a three terminal device, with a little bit more complication.
The equation is different.
I don't have to resort to a Taylor series expansion.
I will just do a complete expansion of this expression and develop the small signal values for you.
Recall the small signal model.
It had the following steps.
The first step will operate at some bias point, VI, VO, and of course some corresponding point IDS.
This is Page 3.
And then superimpose a small signal VI on top of the big fat bias.
Remember the "boost"?
So VI is the boost.
Boom.
And above VI, I have small signal VI that I apply.
And our claim is that response of the amplifier to VI is approximately linear.
The key trick with this is that for my small signal model here, this is Page 3 here, and Page 2.
The key trick here is that with the small signal model, I operate my amplifier at some operating point, VO, VI.
I superimpose a small signal VI on top of small VI on top of big VI.
And then I claim that the response to VI is approximately linear.
And let me just embellish that curve a little bit more.
Notice that in this situation this was my VI, which is my bias voltage, this is VO, which is the output bias, and of course not shown on this graph is the output operating current which is IDS.
One nice way of thinking about this is to redraw this and think that your coordinate axes have kind of shifted in the following manner.
This is VI.
This is also on your Page 3.
This is VT.
Remember this was the operating point, VO and VI.
And notice that we were operating in this small regime of our transfer curve here.
And in effect what we are saying is that I am going to apply small variations about VI and call those variations delta VI or small VI.
And the resulting variations are going to look like delta VO.
Also referred to as small V, small O.
So I will have small variations here.
And they give rise to corresponding small variations there.
One way to view this is as if we are working with a new coordinate system.
Another way to view this is that so the capital VI and capital VO correspond to my VI and VO as the total voltages in my circuit, but at this bias point I can think of another coordinate system here with small VI and VO out there.
And for small changes to VI, I can figure out the corresponding small changes to VO.
Just that all the analysis I perform here is going to be linear.
And I will prove it to you in a couple of different ways in the next few seconds.
When I am doing small signal analysis I am operating here in this regime at some bias point.
You have also seen this before.
How do I get a bias?
This is my amplifier RL and VS.
This is Page 4.
VO.
The way I get a bias is I apply some DC voltage VI and superimpose on top of that my small signal small VI.
This is my DC bias that has boosted up the signal to an interesting value.
And because of that what I can get is by varying VI as a small signal with a very small amplitude, I am going to get a linear response here.
And I can draw that for you as well.
This is my bias point here.
And if I vary my signal like so then my output should look like this.
This is point VI, this is point VO, and this is my small signal VI and this is my small signal VO and this is capital VO.
So this small thing here is VI.
I would like to show you a little demo.
I will start with the same demo I showed you the last time.
I showed you the amplifier.
In the demo I am going to apply a triangular wave.
And initially I start with a large signal.
And you will see that the output looks really corny, is going to look something like this.
That's large signal response.
And then I will begin playing with the input making it smaller, and you can see how it looks yourselves.
There you go.
So this is where I stopped the last time.
The last lecture I applied this input, time is going to the right, and the purple curve in the background is the output.
It looks much more like a sinusoid with some flattening of its tips.
Nothing like an interesting triangular wave.
What I will do next is that let me make sure I have enough of a boost here, enough of a DC voltage so that I am operating at some point here.
I believe I already have that.
Notice that I can shift up the triangular wave input, or I can shift it down.
So let me bias it here.
I have chosen a VI that's about, I forget how many volts per division it is, but I have chosen some VI here.
And I biased it such that this is the input.
You get a nonlinear response.
It is amplified.
It is much bigger.
What I will do next is make VI that I apply smaller and smaller.
I have already done the boosting.
Boom, that's a boost.
So I have boosted up your VI already.
Next is I am going to shrink it, and hopefully you will see that if all that I am saying is truthful here you will see a triangular response.
Let's go try it out.
Watch the yellow.
I am going to shrink the yellow and make it smaller and smaller.
There you go.
It is great when nature works like you expect it to.
I have never seen a triangular wave looks so pretty in my life.
It is awesome.
Look at this.
Here is a tiny triangular wave.
And the output is also a triangular wave but it is much more linear.
Yes.
Question?
What's that?
The question is that the output here is only as big as the input used to be before.
That's a good question.
What I have done here is I am showing you a laboratory experiment.
And let's assume that this input is the input I am getting from some sensor in the field.
Assume that this is my input, not what I had before.
Assume that this is my input to begin with and this is the amplified output.
What I can also do is I can also change the bias.
And we will see this at the end of the lecture, in the last ten minutes of lecture.
How do you select a bias point?
By changing your bias point you can change the properties of an amplifier to give you a preview of upcoming attractions.
Let me ask you, what do you think should happen if I change the bias point?
I have not shown you the math yet, so intuitively what do you think should happen?
If I increase the bias what do you think is going to happen?
Yes.
Good insight.
Higher bias will be more amplification.
Let's see if our friend is correct.
Let me set a higher bias.
Not necessarily, I guess.
You're actually right, by the way.
I am playing a trick on everybody here.
As I change my input bias.
Notice that under certain conditions my output becomes smaller and gets more distorted.
Under other conditions what is going to happen to my output is that it is becoming smaller and is going to get distorted again.
So there are a bunch of funny effects happening that reflect on the bias point, but for an appropriate choice of bias point as I increase the bias the amplification should increase.
And I will show you that in a few minutes.
But it is a complicated relationship.
Yes.
This is finally getting fun.
Here is the question.
Professor Agarwal, we love your song and dance, but if you really want to get a high signal at the output and you want to amplify your big input signal how do you do it?
So the question is let's say I have an input that is this big here, if it is this big, I have shown you how I can get things that are this big, but what if my input was this big?
How do I get an output that is this big?
Well, I will use one of those learned by questioning methods and have you tell me the answer.
Someone tell me the answer.
How do I do that?
Yes.
Use another amplifier.
So the answer is I will use one amplifier to go from here to here.
And the suggestion is use another amplifier to go from here to here.
And, in fact, I believe that you may have a problem in your problem set where you will do that.
And so you have only yourselves to blame.
So how do you make this work?
What you have to do is this VI has to be much smaller than the bias point VI on this one.
I have to build a different amplifier, choose a different set of parameters such that VI prime, which is the VI for this guy, is much less than V capital I prime for this guy.
It's a design question.
You need to design it in a way that the signals of interest need to be much smaller than the bias voltage of this amplifier.
So you may have to use much higher supply voltages.
My amplifier, I believe, has a 4 volt supply or 5 volt supply.
You might have to use an amplifier with a much bigger supply, different values of RL and so on.
And I know that the course notes also have some exercises and problem sets that discuss that in more detail.
Yes.
This is even more fun.
The question is, good question.
The question is why do you need this guy here?
Just use this guy, right?
Why do you need this guy?
Big guys rule, right?
Who needs the little guys?
Well, let me use the Socratic method again.
Why don't you give me the answer?
You guys are smart.
Why do you need little guys?
Why do you need the small guy here?
Anybody with the answer?
Yeah.
The big guy may not be as sensitive.
I like that.
You know what?
He is almost correct.
I will show you why in a second.
Anything else?
Any other reason?
Yes.
Bingo.
That is another good answer.
So let me address both the answers.
The answer given was that look, this amplifier is amplifying the signal by a certain amount, by a factor of 7.
And I have designed this such that this amplifies a signal by a factor of maybe 10.
So in all I am getting an amplification of 70.
This would be a great design question for lab next year.
I give you a bunch of components and ask you to design an amplifier given the constraints with the highest amount of amplification.
It turns out that when you design your amplifier, in order to meet the saturation discipline and so on, you have to choose values of RL and VS and stuff like that and be within power constraints so the amplifier doesn't blow up and stuff.
And by the end of it all you are going to get a measly 7X gain out of it.
The same way here, to be able to deal with a very small signal here and get some amplification, another set of values and you get 10X.
So they multiply.
It is much harder to build one amplifier with a much larger gain.
You know what?
I just realized that we will be looking at this in the last five or seven minutes of lecture.
I am going to show you what the amplification depends upon.
It depends upon K. It depends upon RL.
It depends upon VI.
Now the question is I have had all this time to think about how to stitch in sensitive into this, and I believe I can.
It turns out that when you have large voltages and so on and you have practical devices, it turns out that the more current you pump through devices they tend to produce noise of various kinds.
So very powerful amplifiers are not very good at dealing with really tiny signals because they have some inherent noise capabilities.
And so I guess that is sensitive.
It is sensitive to noise.
Another question?
Yes.
Ask me the question again.
I didn't follow.
Let me just explain it.
It turns out that I will not be able to pass this through the big amplifier to begin with because it is just going to give me a gain of just a factor of 7.
However, if I have a signal that is this big to begin with then I may just need this amplifier.
I don't need the smaller guy.
If my signal was this big to begin with, if I had a strong sensor that produced a strong signal to begin with, yeah, I can deal with just a single stage.
I don't need to two stages.
It is all a matter of design.
And it is actually a fun design exercise.
Given a budget, dollars, right?
You go to your supply room and look at the parts that you have and you go to build what you have to build with the parts that you have.
And so sometimes you need to build two amplifiers to get the gain or build a signal amplifier.
It's all a design thing.
All right.
Moving on to Page 7.
That brings us to the small signal model.
Page 5.
What I showed you up on the little demo was that provided the signal input in this example VI was much smaller than capital VI out there as I shrank my input, I was able to get a more or less linear response at the output.
And so to repeat my notation at the input, the total input is a sum of the operating point input plus a small signal input.
This is called the total variable.
This is called the DC bias.
It is also called the operating point voltage.
And this is called my small signal input.
It is also variously called incremental input.
This is more a mathematical term relating to incremental analysis or perturbation analysis.
So VI, call it small signal, call it small perturbation, call it increment, whatever you want.
Similarly, at the output I have my total variable at the output a sum of the output operating voltage and the small signal voltage.
I do not like using Os in symbols because big O and small O is simply a function of how big you write them.
It is not super clear.
And in terms of a graph, let me plot the input and output for you.
Let's say this is the total input and that is the total output.
I may have some bias VI.
And corresponding to that I may have some bias VO.
Hold that thought for a second while I give you a preview of something that we will be covering in about three or four weeks.
Notice that as I couple amplifiers together, the output operating point voltage of this amplifier in this connection becomes the input operating point voltage of this amplifier, right?
So when they connect this output to this input, the output operating point voltage becomes coupled to the input here so it becomes the input operating point voltage here.
Now I have a nightmare on my hands.
As I adjust the bias of this guy, the bias of this guy changes, too.
The two are dependent.
It is a pain in the neck.
And we being engineers find ways to simplify our lives.
And you will learn another trick in about three or four weeks.
And that trick will let you decouple these two stages in a way that you can design this stage in isolation, go have a cup of coffee and then come back to this stage and design this stage in isolation.
For those of you who want to run ahead and think about how to do it, think about it.
What trick can you use to get them in isolation?
Moving on.
What I would like to do next is address this from a mathematical point of view.
And much as I did for the light emitting expo dweeb analyze this mathematically and show you that if VI is much smaller than capital VI, I indeed get a linear response.
This time around I won't use Taylor series because it turns out that this expression can be expanded fully.
So you don't have to buy into Taylor series and so on.
I am going to list everything down for you.
We know, to begin with, that VO for the amplifier is VS-RLK/2 (VI-VT)^2.
What I am going to do for this, much as I did for the LED, what I'm going to do is derive for you the output as a function of the input when the input VI is very small.
In other words, when I substitute for VI, V capital I squared plus small VI.
Much as I did for the expo dweeb, I want to substitute for VI a big DC VI.
So VI is much smaller than VI.
And show you for yourselves that the output response, V small O is going to be linearly connected to VI.
Notice that, let me write another equation here.
This is a total variable.
This simply says that if the input is VI then the output is going to be VO, which means that the operating point input voltage should satisfy this equation, correct?
In other words, the operating point output voltage V capital O should equal VS-RLK/2 (VI-VT)^2.
This is at VI equals capital VI.
This is very simple but may seem confusing.
All this is saying is that look, this equation gives me the relationship between VI and VO.
Therefore, if I apply capital VI as the input, I'm given that my corresponding output is capital VO, so they must satisfy this equation, right?
Those are bias point values and that must satisfy this equation.
Simple.
I know that.
So hold that thought.
Stash it away in the back of your minds.
Now let me go through a bunch of grubby math and substitute for VI in this expression here.
Let me go ahead and do that.
VS-RLK/2((VI+vi)-VT)^2.
When I do something that is other than math I will wake you up.
I will just keep doing a bunch of steps that are pure math.
No cheating.
No nothing.
Watch my fingers.
When I do anything that is not obvious math I will wake you up.
Next I am going to simply move VT over and rewrite this as follows, RLK/2((VI-VT)+vi)^2.
Again, I haven't done anything interesting so far.
I have just substituted this.
I am just juggling things around just to pass away some time, I guess.
All right.
Next what I am going to do is simply expand this out and write it this way RLK/2, expand that out and treat this as one unit VS - RLK/2((VI-VT)^2+ 2(VI-VT)vi+vi^2).
Nothing fancy here.
This is like the honest board.
Nothing fancy here.
Standard stuff.
Only math.
I will move to this blackboard here where I do some fun EE stuff.
Yes.
Good.
At least one person isn't asleep here.
Thank you.
So just math here.
Nothing fancy.
Plain old simple math.
I have not done any trickery.
I still have all my ten fingers.
Now what I am going to do, now watch me.
I am not using Taylor series here because this expression lends itself to this analysis.
Notice VI squared here.
I made the assumption that VI is much smaller than capital VI, so what I can do is assuming that VT is small enough that VI minus VT is still a big number compared to small VI, what I can do is ignore this in comparison to the capital VI terms.
So I have a capital VI term here.
I am going to ignore VI squared.
So, for example, if capital VI was 5 volts and small VI was 100 millivolts 0.1, so 0.1 squared is 0.01.
So it is comparing 0.01 to 5.
So I am off by a factor of 500.
So now watch me.
Now I begin playing some fun and games here.
I eliminate this, and because I eliminate that it now becomes approximately equal.
What I do in addition is let me write down the output.
The total variable is the sum of the DC bias and some variation of the output.
And let me simply expand that term and write it down again.
VS-RLK/2(VI-VT)^2-RLK/2.
I get a two here.
And I get VI-VT.
I won't forget the VI this time.
Again, from here to there nothing fancy.
This is the one step where I have used a trick.
I have said small VI is much smaller than capital VI, and so I have simply expanded this out and written it here.
So do you see the obvious next trick here?
From star look at this guy.
I can cancel this out from star because I know that at the operating point these two expressions are equal, and so therefore I can cancel out the operating point voltage and this.
What I am left with is small VO is simply minus RLK(VI-VT) times vi.
Only one place where I did something funny.
Other than that it is purely math.
So this is what I get.
Notice that this whole thing is a constant, minus RLK(VI-VT).
This whole thing is a constant.
And so VO is equal to some constant times VI.
Let me just define some terms for you that you will use again and again.
For reasons that will be obvious next lecture, I am going to call this term here GM.
I am going to call this term a constant, K(VI - VT).
It is a constant for a given bias point voltage.
So I am going to call that GM.
And then I am going to call this whole thing A.
And of course this is VI.
There you go.
I have my linear amplifier.
A is the gain times small VI.
And the gain has these terms in it.
I just call this GM.
You will see why later.
But notice that the gain relates to RL.
The size of the load resistor RL, how big it is, 1K, 10K, whatever.
K, this is a MOSFET parameter, and VI minus VT. That is a constant for a given bias point voltage and small VI.
So VO equals small VI.
I won't give you a graphical interpretation, but I encourage you to go and look at Figure 8.9 in the course notes.
And it gives you a graphical interpretation of that expression.
Move to Page 7.
Another way of looking at this, another way of mathematically analyzing it, here I went through a full blown expansion and pretty much deriving the small signal response.
What I can also do is take a shortcut here.
So let me just give you the shortcut.
You might find this handy.
VO=VS-KRL/2(VI-VT)^2.
And my shortcut is as follows.
My small signal response is simply this relationship.
I find the slope at the point capital VI and multiply by the increment.
Slope times the increment gives me the incremental change in VO as follows.
d/dI (VS-KRL/2(VI-VT)^2) evaluated at vI=VI times vi.
This is math again.
I want to find out the change in VO for a small change in VI, and I do that by taking the first derivative of this with respect to VI substituting V capital I and multiplying by the small change delta VI or small VI.
So this is simply the slope of the VO versus VI curve at VI.
And so therefore taking the derivative here of this.
This is a constant so it vanishes.
But twice 2 to cancel out, so I get KRL(VI-VT) times small vi evaluated at capital VI.
So I get twice KRL, VI evaluated at capital VI, so it is VI minus VT times small VI.
Same thing.
Oh, and I have a minus sign here.
I get the same expression that I derived for you up there, and this is just taking the slope and going with it.
And this, as I mentioned before, this is A.
The last few minutes let me kind of pull everything together and also hit upon something that many of your questions are touched upon.
And that all relates to how to choose the bias point.
So here I have taken an analysis approach.
When teaching we often teach you are given something, you analyze it, but as you begin to master it you can begin to design things where you can ask a lot of questions and so on.
And here what we have is an analysis given a value of RLK, VI and so on.
How to choose the bias point becomes more of a design issue.
If you are designing an amplifier, you asked me the question, how do I choose two small amplifiers versus one big amplifier, that sort of stuff?
It boils down to how do you choose the bias point?
How do you choose VI?
How do you choose RL and so on?
What I would like to do is touch upon some of these things.
First of all, gain or the amplification.
One of the most important design perimeters for an amplifier is what is the gain?
Let's say you get a job at Maxim Integrated Technologies, and they say we would like you to build a linear power amplifier for cell phones.
You can say I know how to do that.
And then they say the next stage needs a 100 millivolt input.
While this thing coming from the antenna is only a few tens or a few hundreds of a microvolt.
So you sit down and say oh, my gosh, I need an amplification of so much, and you go design an amplifier.
So gain tends to be a key parameter.
And notice that gain is proportional to RL.
It relates to VI minus VT, so proportional to VI.
It is also related to RL.
The second point is the gain point determines where I bias something.
If I choose my bias too high I get distortion, or if I choose my bias too low I get distortion.
So depending on how I choose my bias point, as a signal goes up it may begin clipping or begin distorting.
And I will show you a demo the next time on that particular example.
So bias point will determine how big of a signal you can send without getting too much distortion.
And the other thing is that, relates to how big of an input, what is a valid input range?
So let's say you have a signal.
And you want that signal to have both positive and negative excursions of the same value.
Then, depending on where you choose a bias point, your input range may become smaller or larger.
And we will go through these in the context of and amplifier and look at some design issues in the next lecture.
We have put some of the quiz stats here.
The mean was about 75%.
And I must tell you that that is very impressive.
I guess MIT undergrads never cease to amaze me.
And this was not an easy quiz.
This was a relatively hard quiz.
And that average implies that you guys did well on a relatively hard quiz.
Good.
Let's get back to our final lecture on amplifiers and small signal circuits.
And as always let me start with a review.
Very quickly -- -- we came up with a notation to represent small signals.
And our notation looked like this.
Our total variable was small and capital, and this was a DC bias and this was a small signal.
This is also called the operating point.
And the small signal is also called the incremental signal.
In general, if you have some function, some variable of interest in the circuit, say a total variable V out, let's say it relates to some input variable as F of VI.
So mathematically we can find out V out by simply finding the slope of this function at the operating point and then multiplying it by the incremental change in the input.
Gold standard math.
So we do the slope of this function and evaluate it at the operating point.
So this would give us the slope of the function.
And multiply that by small VI, which is incremental change.
This is standard math.
What this will tell you is given a small change in VI this function gives you, this expression gives you the small change in V out.
And in lecture we have pretty much used this method so far, used the math to get to where we wanted it to be.
And then the way we provided biasing and so on was for our amplifier in particular we had a bias voltage, some small signal value, VS. And this was output which was also given to be some output operating point plus a small change, which was a change in the output voltage.
So what we have done here is mathematically computed small V out.
And what I am showing you here is to get the same effect in a circuit is you build your circuit and replace what used to be a total variable with a DC bias plus a small change.
And then you will get your output here.
And this output will relate to this input using this expression.
So this is more review.
To continue on with the math review, for our amplifier VO was given to be VS-K/2(vI-VT)^2 RL.
So this was the output versus input relationship for the amplifier.
And mathematically I could get the small change in the output VO by simply differentiating this function with respect to VI, evaluating that function, at capital VI and multiplying by the small change in the input.
And the resulting expression that we got for small VO -- -- was simply minus K, this was our DC value, and RL times small VI.
So we derived all of this the last time.
So nothing new so far.
So my small signal output was some function given by K(VI-VT)RL times small vi.
And notice that this is how VI relates to VO.
And this is a constant with respect to VI.
V capital I is a DC bias, so this is a constant.
So therefore this is the linear relationship that we had set out to get.
This term here, for reasons we will see today, this term here K(VI-VT) is called gm.
Transconductance.
We will look at it in more detail a little later.
Even more review.
So I can draw the transfer function and plot VO versus VI.
Another way to graphically view what is going on is by plotting the load line curve for this circuit, so this is VI.
And I said we draw that by first plotting the -- These were our MOSFET curves.
And we know that at some point the MOSFET gets into saturation, so this curve was iDS=K/2 VO^2.
And to the right side of the curve the MOSFET is in saturation.
And we said we will adhere to the saturation discipline and operate in this regime.
When the MOSFET gets into this region it is in its triode region.
And then we could draw the load line here.
The load line codified the following relationship, iDS=VS/RL-VO/RL.
This was a load line.
So I have superimposed a load line on the device characteristics, and I am going to show you a little demonstration based on that at this point.
So these curves were drawn for increasing values of VI.
And if I choose some operating point here then this point would correspond to some bias, this bias point would correspond to some input voltage VI, a corresponding output bias VO and a corresponding current iDS.
So iDS capitals, VO capitals, VI capitals represent the operating point values for our little circuit.
So far there is nothing new.
One thing we stopped the last time by pointing out that the gain of our amplifier, this is the gain, -K(VI-VT)RL.
That is the gain A of the amplifier.
That gain related to VI.
A gain was proportional to VI-VT.
So therefore if I increased VI, I would get more gain.
So the question is how do we choose a bias point?
And in our particular example, let's say we are free to play around with VI.
So we play around with VI and I can choose various bias points.
So where do you set the bias point?
What are the various characteristics of the circuit that relate to my bias point?
Well, first, of course, is gain.
The gain depends on how I choose VI.
I will show you that in a moment.
The second important thing, in other words, if I choose a bias point that is a small VI then my gain is going to be smaller.
If I choose a bias point that's at a much higher value of VI, I get a bigger gain.
The second important consideration is operating range.
Notice that if I choose a bias point here then as the input changes -- Notice VI in this graph goes up or down, and I would be traversing and following different lines here in my MOSFET characteristic.
And as VI increases the operating point would come up here and so on.
So if about this operating point I varied my input voltage VI then, so let's say about this operating point, if my input VI, my small signal VI varied about a small range then correspondingly the output value would vary about this part of my load line.
So notice now that the operating range, how far can VI vary before the MOSFET goes out of its saturation discipline?
Well, on the low side my VI can come down to here.
And we looked at the operating ranges for an amplifier.
And I can come all the way down to VT. At that point the output will come here.
Similarly at the high end VI could get up to a high value.
And we computed that value in the last lecture.
And the corresponding value of the input would be here.
So in some sense I can traverse all the way from here to here and have the MOSFET remain in saturation.
Remember we are not talking about linearity right now, just about the valid operating range based on my definition which is that the MOSFET should stay in saturation.
So if I chose my operating point here then I get this range here.
And, on the other hand, if I chose my operating point to be here, for negative excursions of the input signal I have a very small amount before I hit cutoff.
So if I chose my operating point here then for negative traversals of VI about the operating point I very quickly hit cutoff.
So if I want symmetric swings then this is the best that I can do in terms of the valid input operating range if I want symmetric swings given that this is my bias point.
On the other hand, if I chose my bias point somewhere here, or very carefully chose my bias point then my input can vary on a much wider region and still get symmetric swings.
And so therefore the choice of bias point also influences the maximum swing range of my input signal.
I shouldn't call this operating range.
I should call it input swing range.
We defined the valid input operating range as the range for which the amplifier satisfied the saturation discipline.
So the two key issues, gain and the input swing.
Let me show you a quick demo and try to point out on a graph some of the characteristics that relate to the matter we have been talking about so far.
So what I show here are these curves for the MOSFET.
This is VO and this iDS.
This is the zero point.
Ignore this line down here.
This line up here corresponds to the output voltage VO.
What I am going to do now is, through some careful circuit hacking, I'm going to show show you a load line and show you the bias point, and show you how the bias point can be moved up and down by changing the input voltage which changes the corresponding output voltage.
It is hardly visible out there.
Is it there?
OK.
It is not really clear, but notice that as I increase my input, I am increasing my input.
My output keeps coming down.
And I hope your eyesight is better than mine because I don't see a dot up there.
I am amazed.
This is the first time this has happened to me.
That's OK. All right.
As you can see, as I change the input value the output operating point changes, and the dot out there traverses, articulates a load line.
I guess I have to believe that there is a dot out there.
Next what I will do is show you some more fun stuff.
What I will do is instead of having just a dot by having a DC voltage, let me apply an input sinusoid.
So if I apply an input sinusoid at some bias then I should see an articulation of the corresponding region of the load line corresponding to the input.
So, as you can see here, now the bottom line, here is my input and this is my output.
And notice that this the region of the load line articulated when the input is of this magnitude.
Now let's have some fun.
As I increase my input, you can see that a larger portion of the load line is articulated, right?
There you go.
And as I decrease my input a smaller region of the load line is articulated.
Let's leave it here for a moment.
And what I will do next, this is the region here that we are looking at, let me increase the bias.
If I increase the bias, if I increase VI, what do you think should happen to this line here?
Well, if I increase the bias, the line should go up, right?
Because remember the dot?
The dot is in the middle of this thing here.
If I increase the bias this should move up here.
So that line moves up.
Do you expect anything else to happen to that line?
Pardon?
It increases, exactly.
If I increase the bias point to here then this must also increase because my gain has increased.
Let me do that.
So let me increase the input bias.
Indeed notice that the region of the load line articulated is larger now.
Let me decrease the bias.
And notice that because the gain is smaller the little segment shown is also smaller.
I have shown you two things so far.
One is that I as I increase my bias the line indeed rises up corresponding to a higher value for the input operating point.
And the second is that I get a larger swing in the output as I increase the bias.
Just to show that for those like me who were visually challenged in terms of viewing that little dot up there, let me get some audio so you can actually hear the sinusoidal tone.
It is a big annoying.
As I reduce the bias the gain is decreased.
As I increase the bias you can see that the gain is increased and the tone is louder.
Let's have some more fun and let's play some music now.
And what I am going to show you with the music -- The reason I play the music is not just for fun.
Well, it's 85% fun and 15% learning.
Can we turn it on for a second?
What I would like to do is, as we play the music, the reason I am playing the music for that 15% is so you can listen to distortion.
I want you to listen to the distortion.
That is when the articulation is here you are not going to get much distortion.
But as I get into cutoff you should be getting a bunch of distortion.
Similarly, as you get into the triode region you should also be getting distortion because the amplification from being somewhat nonlinear here becomes highly nonlinear at those two points.
So let's just play the signal.
So volume increases, or rather the amplitude increases by increasing the bias.
Now you should hear the volume go down and distortion.
So notice now that the bias point is way down here.
So the gain is very low, and plus there is a distortion because of cutoff.
Now what I will do is blast it up here, and you will see that the volume has gone up but then you see distortion again.
Let's see if you can stand the volume here.
Even the CD doesn't like that.
Notice that as I went up here the volume kept increasing because the gain kept increasing, but as I got into the triode region I began to lose my gain because, remember, the amplifier doesn't have gain in the triode region, MOSFET in its triode region, and we also get a bunch of distortion out there.
Finally, it turns out that as people are building amplifiers -- I think this was in the mid to late `50s and `60s and so on.
They said man, electrical engineers are not going to get their thing right.
So they invented a new kind of music which was much more tolerant to distortion.
And I will play that music for you.
It is called hard rock.
I challenge you to tell me it is distorting.
Sounds good to me.
OK. All right.
That'll do it.
Thank you.
I hope there are no hard rock musicians in here who will come and beat me up after lecture or something.
All right.
Believe it or not most of that was review.
There is nothing new today besides some fun and games and so on.
I will give you a breather for five seconds before jumping into something even more fun.
I want you to look at the middle board here.
And, as I told you in the beginning of 6.002, engineering is about building useful systems.
Engineering is not about showing off at math or saying man, I am really cool in math and stuff.
Engineering is about building useful systems, and you want to find the simplest, easiest, cheapest way to get there.
Unlike deep areas of math and theory and so on, the beauty is in the simplicity.
So the aesthetics are in how simply can we make things and still get to where we want to be?
All through the course what you will be seeing happening again and again and again is when things begin to get too grovelly in terms of math, we will step back and say oops, we are engineers, remember?
Let's find a much simpler way to do it and use intuition.
So time and time and time again, I am going to take you on a simpler path where you can solve things by inspection by pure intuition.
Most circuit designers do that.
So take a look at this.
I don't like this nasty differentiation here.
That's getting into late high school calculus and so on.
Let's avoid the math and let's see if you find some way of doing it that is even much more simpler.
And that is what I will do next and show you what is called the small signal circuit view.
A purely circuit way of developing the small signal model.
So let me just start by drawing the large signal equivalent circuit for you.
I will draw it here for reasons that will be obvious at the end of the class.
All right.
This is the large signal equivalent circuit model for our MOSFET amplifier.
VS and here is my current source.
iDS relates to the square of VI minus VT.
So stare at that for a second.
And that is a nonlinear circuit.
iDS relates to the square of VI minus VT. Let me start by making the following claim.
Let me shoot from the hip here and make the following grand claim, and then I will show you how I can prove that claim.
The grand claim I am about to make says the following.
A bunch of little devices here.
It is a nonlinear circuit.
Just suppose for a moment we do a Gedanken experiment.
Suppose I replace each of my circuit elements here with its linearized element equivalent.
In other words, here is a VS source, here is a dependent current source, let me replace them with their linear equivalent circuit models.
In other words, with their corresponding small signal element models.
And I will show you what those are in a second.
The resistor has a corresponding small signal element.
The dependent current source has a corresponding small signal behavioral element model.
And what I am going to do is keep the same circuit connections and simply pull out the large signal model for the element and replace it with a small signal element model.
And by the nature of the small signal model they are all going to be linear.
So what I am going to be left with is a linear circuit with simple linear circuit elements in there.
And then once I have a linear circuit, I should be able to analyze that linear circuit using methods 1, 2 and 3, superposition, Thevenin, node method and so on.
And certainly the intuitive methods like superposition and Thevenin, which make life a lot easier for me with linear models, and thereby get the function that I am looking for very quickly.
Again, my claim is that I can replace each of these large signal models by just small signal equivalents and then just analyze the resultant circuit.
And I claim that I should be able to get the same answer.
That's a claim.
All right?
So what I will do is give you an informal proof for why I can do that.
And I also ask you to refer to Section 8.2.1 of the course notes to go through the foundations of the small circuit model in more detail.
The intuition is that, remember KVL and KCL?
I can write down KVL and KCL for every loop in that circuit and every node in that circuit.
If I do KVL and KCL, I will end up with something like this.
For the input loop I get VI something or the other applying KVL.
For the output loop I get V out something or the other.
And then applying KCL I get some other equation in iDS.
So here are my KVL and KCL equations for that circuit.
Now, KVL and KCL are simply a different representation of the circuit because within those KVL and KCL is encoded the topology of the circuit.
Remember each KVL equation represents a loop and each KCL equation represents how nodes are connected together.
So KVL and KCL equations encode within them the topology of my circuit.
What I do next is, say, I replace each of these with the bias plus the small signal, so I get the bias plus the small signal and keep the equations the same.
All I have done in my big set of KVL, KCL equations, I have simply replaced the total variable with the large signal variable and the small signal quantity.
Then comes a key trick.
The key trick is that because the bias point variables, they are a valid solution to the circuit.
The circuit is in this quiescent state, and those are valid solutions to circuit.
So therefore I can cancel them out.
So the VI, the large signal values can be cancelled out leaving just small signal variables in there.
So from the KVL, KCL equations I can cancel out the large signal values, the DC bias points because they satisfy the KVL and KCL themselves.
In other words, I could have written VI plus V out and so on.
Since they are satisfied I just strike out the large signal variable from both sides of each of these equations, so what is left is the same KVL, KCL equations but with small variables in place of the big variables.
What that should tell you, this informal proof should tell you is that the small signal variables should then satisfy the same form of the KVL, KCL equations that the total variables satisfy.
And because the KVL, KCL equations are a reflection of the topology of the circuit, what that says is that the small signal variables must also satisfy KVL and KCL.
And since these arrive from the small signal elements that says that I can replace the big elements with the small elements and KVL and KCL will hold for the resulting circuit.
This is a very quick breeze through, an informal proof to show that I can replace the big elements with the corresponding little element models and then simply apply linear techniques.
Refer to Section 8.2.1 for more foundations and more discussion about the foundations for why we can do this.
That brings up the small signal circuit method.
The circuit method for small signal analysis has three steps.
The first step is find operating point by using LS.
First you analyze your large signal circuit and find the operating point.
You have to do this, because remember, the small signal models depend on the operating point values.
Remember the gain of our amplifier depended on the bias point.
Second step is develop small signal models of elements.
Second step is take each of the elements in your circuit and find their equivalent small circuit model for each of the elements.
Third step is replace original elements with their small signal model elements.
Third step is simply take the large elements and replace them with their small signal equivalent models.
Then analyze resulting circuit, and that circuit will be a linear circuit.
So let's do an example.
I will just use the amplifier as an example of this method.
And convince you that you are going to get the same expression at the end, but just so, so simply without even the smallest amount of grubby math.
Three steps.
The first step is to find the operating point using the large signal model.
And let me just do that here.
I get my V out = VS-K/2(VI-VT)^2 RL.
Let me just write down that out here.
Don't worry about copying that down.
It is on the last page of your notes.
The first step of the method simply applies the large signal model and finds out the behavior of that circuit to find out what the bias point values are.
The second step is to develop the small signal model of my elements.
How do I go about developing the small signal models of elements?
Let's start with the MOSFET.
The large signal model for the MOSFET looks like this.
Here is my Vgs.
This is my gate.
This is my drain.
This is my source.
And I know my iDS to be K/2(Vgs-VT)^2.
So this is the large signal model for the MOSFET, again in saturation.
I am talking about all of these models are under the saturation discipline.
So Vgs relates to iDS in the following way for the MOSFET.
That is iDS, is K/2 and that is my square law relationship.
So what is a corresponding small signal model?
I go ahead and start with this.
The corresponding small signal model simply says that iDS relates to Vgs in the following way.
All I have to do is find a small signal equivalent where I need to find out, given a small change in the input Vgs, what is the small change in the iDS?
So I can apply my standard trick to a much simpler expression here, which is iDS simply, I differentiate this function with respect to Vgs.
So I don't completely eliminate the math here, but it is a much simpler problem here.
At Vgs equals the bias point times small vgs.
I can find the small change in iDS corresponding to a small change in the input using this expression.
That gives me iDS as simply K(Vgs-VT) vgs.
I call this gm, and I will tell you why in a second.
So what does this expression say?
This expression says that if I have a small change in Vgs then this will be my small change in iDS.
Notice that the resulting small signal model is also a dependent current source.
It is a voltage controlled dependent current source.
So the output is the current, and it is a dependent current source and it depends on the input voltage.
The good news is that notice that this one, this expression here gm is a constant related to the bias point values.
Therefore, notice that the small signal model for the MOSFET in saturation, not surprisingly, is a linear voltage controlled current source according to the following expression.
So iDS=gm Vgs.
Gm is a representation for K(Vgs-VT) and is called a transconductance.
It is called a transconductance because it, in some sense, deflects the conductance properties of this based on the input.
So it is a transconductance.
So this value is called Vgs.
Therefore, I can build the small signal model as follows.
Vgs is a voltage controlled current source and iDS is simply gm Vgs.
So this is my gate, drain, source.
So that is the small signal model for my MOSFET.
As a next step what are the other elements in my circuit?
Let's see.
I have a voltage source and I have a resistor, so let me find out the corresponding small signal model for a DC supply VS.
This is Page 7.
I will do it mathematically for you, but often times it is always good to do a sanity check using intuition.
Let me ask you, the large signal for a DC supply looks like this.
The element law for a voltage source is VS equals some capital VS.
It is a constant voltage.
So what do you expect to be the small signal model for a voltage source?
In other words, for a small change, suppose I have a small change in the current, by how much should the output VS change?
It shouldn't change.
It is a voltage source.
So what does intuition tell you is a small signal model for the voltage source?
A short.
So the key here is that a voltage source behaves like a short circuit for small perturbations.
In other words, if I change the current flowing through it by a small amount somehow, the output is still going to held at VS.
In other words, small signals are simply going to scoot through this voltage source without having any impact whatsoever on the voltage.
Or mathematically I could also do small vs is del by del IS of VS evaluated at IS equals some capital IS times small IS.
And therefore VS equals zero.
What that means is that the small signal model for my voltage source is simply a short circuit.
So in a small circuit voltage sources appear like a short circuit.
Finally, I have a resistor, my resistor R. Let me find out its corresponding small signal model.
The large signal model looks like this R, VR, IR.
And I know that VR is simply RIR.
And to find the small signal equivalent I do del of IRR divided by del IR for IR calculated at some constant value times small IR.
What I am looking to do is to find out what is the change in the voltage across R for a small perturbation in the current?
Again, let me exhort you to rely on intuition to at least sanity check your answers.
So what do you think this should look like?
It's a resistor and I have a small change in the current, by what do you expect the voltage to change?
Think about, for the next five seconds, what the small signal model for this should look like and then I will go ahead and write down the answer.
So differentiating I simply get RIR.
In other words, for a resistor the small signal model is the resistor itself.
So what I have done so far, let me just take you through where we are right now, give you the big picture there.
I began by suggesting that looking to find an even simpler way to do small signal analysis.
I gave you an informal proof to show that if I had small signal element models for all of my elements, I could simply replace them in the circuit and then do a corresponding linear circuit analysis phase to get the result I am looking for.
There are three steps to the method.
As a first step we began by finding small signal models for each of our elements.
For the nonlinear MOSFET the small signal model was a linear dependent current source.
For a voltage source the corresponding small signal model was a short circuit.
Again, that makes sense intuitively if I change the current through a voltage source by a small amount.
By how much does the voltage change?
It is a voltage source, silly.
The voltage doesn't change.
So the small signal V, the small change in the voltage is zero, and that is the same thing as a short circuit.
For a resistor by how much does the voltage change if I change the current by a small amount?
Well, it will change by R times the current change, and that is the property of a resistor, R. As a final step what I would like to do, on Page 8, I'd like to very quickly draw for you the small signal circuit and then analyze it.
This is the large signal circuit.
That is a large signal circuit.
And let me draw the small signal circuit.
And the method says simply pluck out, gouge out each of these elements.
And simply replace each of these nasty nonlinear elements with the corresponding small signal linear equivalents.
So let's do that.
Remember, for the input you replace input with its small signal voltage because I am telling you that it's sourcing a small change in VI.
So that is VI.
And then I replace a short for VS.
I replace an R for RL because it is an RL itself for the small signal model.
And then for the dependent source, we discovered that the dependent source was a linear dependent source given where ids=gmvi.
Remember, this was my small VO.
Here you go.
I have a small signal circuit here where I have simply created that by replacing each of the big elements by little rinky-dink elements.
Now these are all linear elements so I can do a really simple linear analysis.
What method shall we use?
Well, this is so simple.
I will just go ahead and use the node method.
So applying the node method at the node with voltage VO, what I will do is the current going up, VO divided by RL equals the current going down iDS.
And so the current going up is VO divided by RL and the current going down is -- Oops, I should have done this.
The total current going out is zero, so the sum of these two is zero.
That is my good old node method here.
And I know that iDS is simply gmvi equals zero.
So right there I have the relationship between VO and VI.
So VO is simply minus gmviRL.
And remember gm was simply K VI minus VT. We are done, OK?
What have we here?
I created a linear circuit which simply comprised small signal models for each of my big elements.
And then I simply did a straightforward linear analysis using any one of the linear techniques I knew about.
This is simple enough so I apply the node method.
And I've got the equation at this node, simplified it and I directly got the answer.
In one or two steps I directly gave you the output as a function of the input.
It can't get any simpler.
Thank you.
Good morning.
Today we move in the direction that takes a big turn from the direction we have been going in so far.
All the devices we have had up until now, resistors and voltage sources, and even your digital devices like the AND gate or the inverter and so on had a very specific property.
We didn't dwell on that property, but that property was that these were not what are called memory devices.
In other words, the outputs at any given time are a function of the inputs alone.
In other words, if you took your inverter or your NAND gate for that matter and you build a circuit comprising 50 NAND gates connected in structures that we have talked about, you apply an input and boom you get an output.
And your output is a function of the inputs alone, right?
The same thing with your resistors and voltage sources.
At any given point in time your output VO of T was some function of the input VI of T. What we are going to do today is discuss a new element which will introduce a whole new class of fun stuff for all of us to deal with.
And that is called storage.
In other words, the output of a circuit is now going to depend not just on the inputs but it is going to depend on the background or it is going to depend on where the circuit has been in the past.
So past is going to matter.
It is a very fundamental difference.
And what I would like to do is start by giving you folks a little bit of a surprise.
I am going to do a little demo taking two of your inverter circuits.
I am going to start by taking a couple of inverters.
Remember, I am using this structure here as an inverter.
And I am going to couple this to another inverter and take an output C, some VS, some load resistance RL, my B terminal and my A terminal.
So I'm going to apply some input between ground and my A terminal.
And for fun I want to apply a square wave at the input.
A square wave between zero and 5 volts.
And this is how my time goes.
Let's assume that VS is 5 volts.
So what I am going to do is plot for you the behavior of this inverter.
I am going to plot for you A, which would look like this.
I am going to plot for you B, which would be the inverted wave form.
And then plot C, which would be a wave form that looks like this again.
Let me do a plot here.
So this is A.
-- and so on.
Time goes this way.
And let's say this is between zero and 5 volts.
And B should be an inverted wave form that should look like this.
If all that we believe of the world so far is true then this is how things should behave, so C should look like this.
This is what the world should look like and if everything that you learned about is true and correct and all of the good stuff.
Let me show you a little demo and see if I can try to pull the rug out from under all that you have learned so far and show you some surprising stuff.
Here are the three wave forms that I showed you up here.
This is my A.
This is my A wave form.
This is the B wave form.
Notice that B, as you expect, is an inverted form of A.
And this is C. We all expect this, correct?
But what I am going to do is let me expand the time scale on this so that I can look at these transitions a little bit more carefully.
I am just going to expand the time scale.
There you go.
All I have done is expanded the time scale and spread that out a little bit.
And what you see there is quite different from what you expect.
A is a square wave as expected, but B is stunningly different.
It is a zero as expected because this is a one.
But here I get some really strange behavior, behavior that is like nothing on earth.
Like nothing you have seen before.
And then, of course, it becomes a one eventually, but there's some really, really shady stuff going on here.
And so far you are not prepared to deal with this.
We have not given you the facility to deal with his issue.
What is the problem with this?
We could say who cares?
What is the problem with this?
Let's look at the result.
I am looking at this, I am focusing on this piece here.
And notice that instead of being a sharp rise it looks like this.
It is going up a little bit more slowly.
What kind of problem would that create?
The problem that it creates is the following.
Let me play around with this graph a little bit more.
What I am going to do is just take this output here, the C output and line it up against the A output.
And so I am going to line up the C wave form on top of the A wave form.
So you can see for yourself if something really, really strange and nasty is happening, I am just going to move up the C wave form and line it up.
What is happening out there?
If you look carefully, what you observe is that the C wave form transitions just ever so slightly later than the A wave form.
Look here.
And I claim that it is because of this.
Because of this, the C wave form falls just a little bit later, and that little thing we see out there is a delay.
So nothing you have learned so far prepares you for this.
Suddenly, instead of the output exactly following the input, my output is following the input but a little bit later.
And it is this fact of life that things happen a little bit later, is really the reason why each of you and all of us needs to buy new computers every couple of years.
This simple basic fact.
If this fact of life didn't exist, you would buy one computer and be done with it for life.
Intel would make gobs of money one year, and so would Dell and Gateway and so on, and then no more.
That's it.
This is it.
But because of this a little itty-bitty difference here the entire semiconductor technology is charging along trying to do something about that.
You buy newer and newer computers each year.
It turns out this little itty-bitty thing here, that is called the delay, the inverter delay.
And it happens because of a specific element that has been introduced here that we have not shown you so far.
And a large part of the semiconductor industry and follow-on courses and design and so on focuses on how could I make my delay smaller, how can I get to be faster and faster and faster?
This relates to how fast we can clock your Pentium IV.
Remember it came all the way to 1.3 gigahertz?
What's the fasted Pentium money can buy today?
What is the fastest P4?
Oh, 3.2 have come out?
I don't know.
Ken claims 3.2.
But, yeah, there you go, 3.2 gigahertz.
It all has to do with this little itty-bitty thing.
You saw it for the first time here.
When some of you become CTOs at Intel and so on, just remember that it all began on October 16th with this little rinky-dink thing here.
What you are going to learn now is some really cool stuff that has huge implications for life.
So why does that happen?
Why did this transition happen just a little bit later?
The reason is that remember when this wave form reaches VT, the threshold voltage of this MOSFET, this guy is going to switch, right?
So because of the slower rise of the voltage, the VT is going to be reached a small amount of time later.
So I am going to hit VT slightly later.
And because of that this guy is going to transition just a bit later because this intermediate wave form B is slower.
It hits VT just a little bit later than if it would have made an instantaneous transition.
And therefore my output falls just a little bit later and this gives rise to my delay in the inverter.
We can call that d if you would like, some delay.
In your course notes, this material is covered in Chapters 9 and 10.
That was to kind of motivate why we are going to be doing all that you we will be doing.
Don't anybody come within a foot of this even by mistake.
I mean it.
It is pretty deadly stuff.
Today we will talk about the capacitor.
And in the next couple of lectures I am going to tie it all together and show you how this relates to that.
I will show you exactly how the delay happens.
You can compute it based on some simple principles that you will learn about in the next couple of lectures.
What I am going to do is first of all show you, I claim that that delay happens because of the presence of a capacitor somewhere in there.
What I will do now is take you into a closer look, take a closer look at the MOSFET and show you were the capacitor is.
This is the MOSFET that you have seen so far, drain, gate and source.
This is called an n-channel MOSFET.
And what I am going to do is dissect this and show you what is actually happening, what this looks like on silicon.
So here is my slab of silicon.
It is very thin.
And let's say this is, I won't go into details here.
You will learn a lot more about this in future device classes like 301 and so on, but suffice it to say I will just introduce it here to give you a sense of where the capacitor is.
This is p-type silicon.
And the way you build a MOSFET is you create a couple of tubs in which you dope to be n-type.
The basic silicon is dope p-type.
And this guy here is n-type.
And what you do is a thin oxide layer is placed on top of that and then on top of that a thin metal layer.
This is a metal layer.
This is a thin piece of oxide, silicon dioxide.
And this is my P substrate.
Now this is a little metal layer that is really a wire on top of the silicone.
This metal layer could be some sort of a wire that meanders around on the surface of silicone.
And this is a wire that connects to the gate.
This is the gate of my MOSFET.
And this guy here is the drain.
And this guy here is the source.
And this is my gate.
So there is a little piece of metal here.
This is this piece of metal here.
And there is a piece of oxide and then my silicone substrate.
Notice that this is my oxide.
When I apply a positive voltage to the gate here with respect to the substrate, what happens is that I draw up negative charges.
I draw up electrons here into this channel region and I have corresponding plus type out here so that I get a view here that looks like a couple of plates.
And I end up with an oxide in the middle.
There is no connection.
Two plates separated by a small distance with plus q and minus q on the plates.
And, because of that, what ends up happening here is that this piece behaves like a capacitor.
So a capacitor has two plates with a thin insulating material in the middle with some permittivity epsilon.
And so I get a little piece of a capacitor here.
That is the capacitor that is forming.
I did not set out to build that capacitor, but there is a capacitor nonetheless.
So when I apply a positive voltage at the gate, negative electrons are pulled up here which forms a channel, and then a current can then flow.
And that is how the MOSFET turns on.
So n-type electrons back to n-type, and I get electron flow here and that gives me my channel.
This is just kind of devices in four minutes or less.
You will do an entire course on this, if you like, if you take 301.
What we do is to be able to capture the behavior that we just saw, the funny delayed behavior, we have to augment our model.
We have to introduce a new element.
So what we do is here is a MOSFET, gate, drain and source.
And notice here we model this by putting a little capacitor, CGS between our gate and the source.
So this becomes a simple model for our MOSFET device which is the good old gate drain source device from the past with a little capacitor CGS having some value for CGS in maybe ten to the minus 14 or thereabouts farads.
So that is a little capacitor that has come about in this device that we fabricated here.
It is that capacitor that is at between node B and ground because it is between the gate and the source of the second inverter.
And it is that capacitor that is playing the games that we saw out there.
So let's look at some of the behavior of an ideal linear capacitor.
A capacitor, as I said, has a couple of plates.
There are a couple of plates.
Between the plates is some dieletric, permittivity epsilon.
Let's say the area of the plates is A, and let's say the plates are separated by a distance D. I get some charge here, let's say q.
So q and minus q on the capacitor.
And the capacitance C is given by epsilon A divided by D. Epsilon, as I said, is the productivity of the dielectric.
So if it is free space then it would be epsilon zero which is the permittivity of free space.
That is the capacitance in farads.
And the symbol looks like this.
Capacitor C. Voltage v. Current i.
So this, much like the resistor, voltage source and so on, this now becomes a primitive element in your tool chest of elements like the voltage source and so onn.
Capacitance with the voltage v across it and a current i.
And I have assigned the associated variables here according to the associated variable discipline.
A question to ask ourselves is remember we said we are all now in a playground from all of nature, in this playground where the lumped matter discipline holds?
And also remember that we said that for the lumped matter discipline to hold we have to make a couple of assumptions.
One of those assumptions was that dq/dt, for all their elements should be zero for all time.
So right now what about the capacitor?
It has got some charge q.
So charge must have built up somehow.
Does that mean that I lied all along, that we are no longer in this playground, that we have been ejected from the playground because of the capacitor, or are we still in the circuits playground in which the lumped matter discipline holds and all good things happen and so on?
It seems like a contradiction, doesn't it?
I took you from Maxwell's playgrounds to the EECS playground where I said the lumped matter discipline holds.
And one of the foundations of the LMD was that dq/dt should be zero for all time inside the elements that we are going to deal with.
And right now boom, it's not four weeks into the course and Agarwal introduces an element and it has q in it.
It turns out that the capacitor also adheres to the lumped matter discipline.
Remember the discipline says that dq/dt is zero for all time within elements.
So I am going to be clever.
What I am going to do is I want to choose element boundaries in a very cleaver way.
Notice that if I have q here on this plate then I get minus q on the other plate.
So if I take the whole element, the element as a whole, if I am careful in terms of how I package my boundaries, if I put both my plates inside my element boundary then I still do get the net charge being zero.
So dq/dt is indeed zero for all time provided I make sure that my element has both the plates.
Therefore, if you come across somebody else that gives you an element that says I have an idea.
Let's create a new branch of electrical engineering in which we model the capacitor not as one element for two plates, but let's build a capacitor by combining two new elements, two garbage elements called G1 and G2.
G1 is like the top plate.
G2 is the bottom plate.
I put them together and I get a capacitor.
But notice if I just pick one plate then the element G1 will not adhere to the LMD.
It adheres to the LMD because I choose my element boundaries in a way that both plates come within it.
So it is very fundamental and key.
And you can read a lot more about it in the course notes.
I purposely dwelt on that simple point because I think it is foundational and important.
And you really need to understand that the capacitor does satisfy LMD.
We are still in the good old playground.
A few simple facts here.
These are in the notes.
And you have also seen this before, I am sure.
I can relate the charge to the capacitance and the voltage as q is equal to Cv.
And q is in coulombs, this is in farads and this is in volts.
So there is some charge q stored on the capacitor and it is in coulombs and q is equal to Cv.
So I can differentiate this with respect to time to get the current, and that becomes i=dq/dt.
So the current at any given time is dq/dt.
And so I substitute for q in terms of Cv here.
That is what I get.
So the current i=d(Cv)/dt.
A 6.002 assumption, capacitance in general can be time-varying.
I can get time-varying capacitors.
In fact, there are some sensors which are capacitive.
And, as I talk, my sound waves can change the pressure on the top plate of the capacitor.
And move the top plate of the capacitor, thereby changing the capacitance by moving the plate.
Remember d here, as the plate moves closer I get a higher capacitance.
So we won't be dealing, unless explicitly said so, with time-varying capacitances.
So what we can do is 6.002 allows us to write Cdv/dt.
So my current source capacitor is Cdv/dt.
I can also write down the energy, capacitors store energy.
E=1/2Cv^2.
I am sure you have seen all this before in physics and so on.
That is the amount of energy stored in the capacitor if it is holding a charge q.
Let me do a little demonstration for you.
They don't make glasses like they used to.
Our friend Lorenzo has charged up this capacitor.
It is a huge capacitor.
It is a 250 volt capacitor so it is nasty.
He has charged it up and has kept it there.
And to show you that it does contain stored charges it has been sitting there holding charge.
Maybe the first row should go backwards, just step back for a second.
I think you guys would be safe but I just don't want to take any chances.
This is holding a bunch of charge.
It is kind of sitting there.
If I short the terminals it should try to say oh, I've got a path, let me get my charge out.
All right.
Let's do it.
This is always a scary moment for me.
And I say a little prayer before I do this.
Good?
OK. Gee, you guys would love to see me getting fried, huh?
All right.
Let's see.
So it did contain charge.
So there is a reason why Lorenzo puts one hand inside his pocket when he shorts it, because there is a natural tendency to hold the wire with both hands, and la, la, la, la, la and put it across the capacitor.
By doing this you are guaranteed that you will just be touching it with one hand.
Hopefully you folks will remember for life that a capacitor can sit around and hold its charge for a while.
All right.
That is enough of fun and games.
Let's get on with our business of building circuits.
What I am going to do is, as I promised you, I am going to close the loop on that example by halfway through the next lecture.
I'm going take you on a bit of a journey involving capacitors and resistors and involving some analysis, and then we will close it all up for you at about the middle of next lecture.
What I would like to do next is here is a new element.
And let's do some fun stuff with elements.
Well, you know about voltage sources, you know about resistors, let's put them together and see how they behave.
Let's have a capacitor here, C, vc(t) and some current i.
What I am going to do, in general, whenever I have something new or something strange, let's say like a capacitor or some other device.
It is interesting to model the rest of the circuit behind it if it contains only resistors and voltages and linear elements as a Thevenin equivalent.
So let me do that.
This is R and this is vi.
This stuff in the back is my standard pattern, voltage source in series with a resistor, and I connect that across my capacitor.
But remember, although you saw those funny wave forms and so on, the capacitor is a linear device.
Because you can see from here that the current relates to dv/dt.
That is a linear operation.
You don't see V squareds and Vis and things like that in there.
It's is a linear device.
Let's go back to our trusty old method, the node method.
If you just blindly apply the node method and simply grunge through a bunch of math, you should be able to get to the answer, that is for some voltage v or some form of voltage vi, I should be able to figure out what vc looks like.
So let's do that.
This is the node that is of interest here with the unknown node voltage vc.
So let me apply the node method.
(vc-vi)/R is the current going this way.
That plus the current through the capacitor should equal zero.
And what is the current through the capacitor?
The node method tells me that, get the current in terms of the element values.
We know that the current is given by CdvC/dt.=O.
Just shuffling things around a little bit, I can write RC dvc/dt+vc=vi.
We are writing the node equation and then getting the equation that characterizes this little circuit.
Notice here that this has units of volts.
And since I have time here, this also must have units of time.
Let's go about solving this little circuit and understanding how it behaves.
The specific example that we will look at looks like this.
Let's say the capacitor voltage at time T=0 is V0.
This is given.
So at time T=0, I am telling you that the capacitor contains a charge.
And because of that there is a voltage V0 across it.
That capacitor had a voltage of 250 volts across it and most of the devices we deal with in laptops and so on today, like the Pentium IV, voltages are on the order of 1.5 volts, very small voltages.
So that is the value in the capacitor, the voltage.
That is called a state.
This is called the state, capacitor state.
It is the state of the capacitor.
And I also give you that vi(t)=VI.
So my voltage is VI.
And somehow, I am not telling you how, but some how it arranged to have the capacitor voltage be V0 at time T=0.
Now I want to look to the solution to this for t greater than or equal to zero.
And in that time my voltage vi is at some capital VI, some DC voltage VI.
So I am going to solve the differential equation RC dvc/dt+vc=vi given these two values.
Input is DC voltage VI and VC0 is V0, the initial charge in the capacitor.
So from now until almost to the end of the lecture, it is just going to be math by solving this very simple first order differential equation.
And the key here will be that throughout 6.002 we will be following one method to solve these.
There are many methods to solving differential equations, and we will follow one method.
That method is called the method of homogenous and particular solutions.
In 1802, I believe, you would have learned maybe this, and certainly other methods.
You can use any method to solve it.
We will just stick to one method.
And this is also used in the course notes.
In this method what we do is take the solution VC by finding two other components.
One is called the homogenous solution.
And summing that up with the particular solution.
And that is the total solution.
So total solution is the sum of the homogenous and the particular solutions.
And the method has three steps.
As I said before, we will be using this method again and again with every differential equation that we encounter in this course.
And you won't encounter a while lot.
The first step we find the particular solution.
The second step, find the homogenous solution.
The total solution is the sum of the two.
And then find --- There will be some unknown constants depending on the equation that you have.
And in the end we simply find the unknown constants by applying the initial conditions that we have.
Boom, boom, boom.
Particular.
Homogenous.
Find constants.
Three things.
So let's go about solving this equation and apply those three conditions.
Again, remember, what I am doing now for the next 10 minutes or 15 minutes is using math that you know about to simply solve this first order of differential equations.
There is nothing really new that I am going to talk about here.
One is to find the particular solution vCP, which will then be added into the vCH to get me the solution.
So the way you find the vCP is you find any solution that satisfies this equation.
This is the equation.
You find any solution that satisfies it.
And find the simplest possible solution that money can buy.
Find it.
That's the particular solution.
Any solution is fine.
In this case, a really simple one would be vCP equals VI.
Let's see if a constant works.
One thing you will realize in differential equations is that they are actually much simpler than they seem.
And the reason is that almost every time you have to assume you know the answer, and then you are checking to see what you assumed was correct.
Assume the answer is this like you are really smart, and then check it out and say oh, yeah, that must have been the answer.
So here we assume that I think VI is going to work so let's try it out.
Substituting in here.
RC dvc/dt is 0. vi is a constant.
So I get vi equals vi, so therefore this is a particular solution.
Done.
I substitute vi here.
So dvi/dt=0.
This vanishes and vi=VI.
Bingo.
Therefore, VI is a solution to this equation.
So I am done with my vCP.
And in general what you have to do is use trial and error.
By trial and error try out a bunch of solutions until you get lucky.
In general, again, in all of 6.002 for many of the excitations a simple constant usually suffices.
Our second step is to find the homogenous solution.
And we can also do that very quickly.
And to do that we have to find a general solution to the homogenous equation.
The homogenous equation is the same differential equation but with the drive set to zero.
We want to follow a set pattern to solve the differential equations here, and the set pattern is find vCP, vCH, find constants.
And to find vCH we are also going to follow a set pattern to find the homogenous solution.
So we set the drive to zero, so vi is set to be zero.
And I need to find a general solution to this.
As I promised earlier, diff equations are really, really simple because the way we are going to solve them is we are going to assume we know the answer and then go check it.
So let's try Ae^st.
Let's try and see if this can solve this particular equation for some values of A and S. I am telling you that the solution is going to be of this form.
Assume it.
And then simply go ahead and find me A and S, and do that by substituting it back into the equation and find out the corresponding As and Ss.
So let's go ahead and do that.
I get RC.
I substitute this back up so I get dAe^(st)/dt+Ae^st=0.
And let me plug that in and see what comes.
I get RCAse^st+Ae^st=0.
I want to discard the trivial solution of A being 0.
That is a trivial solution so I will discard that.
And what I will do is cancel out the As from here, assuming A is not zero, and cancel e^st here.
And what is left is RCs+1=0.
What this is saying is that if I can find an S such that this is true then Aest is a general solution to my homogenous equation.
This is easy enough.
And so S=-1/RC.
If I choose my S to be -1/RC then the simple math that I have gone through shows me that this must be the solution to the homogenous equation.
Or in other words vCH=Ae^(-t/RC).
All this is saying is that Ae^(-t/RC) is a solution to my homogenous equation.
A is an unknown constant.
A is some constant.
I don't know what that is yet.
Notice RC has popped up again.
And the cool thing about RC is that, this is time, this also has units of time.
We commonly represent RC as some time constant tau, as units of time.
Associated with that circuit is the time constant tau, which is simply RC.
I commonly write this as Ae^(-t/tau).
I am very the end here.
I have the particular solution here.
I have got the homogenous solution there.
I need to tell you about something else.
The way I found the homogenous solution was in four steps.
I assumed a solution of the form Ae^st.
I created this equation here in S. This is called the characteristic equation for that circuit.
We will see this time and time again for RC and other forms of circuits.
Assume a solution of this form.
Construct the characteristic equation.
Find the roots of the characteristic equation.
In this case it is an equation in S. So this is the root.
And then form the solution based on that root.
Four steps.
Ae^st, characteristic equation, root and then write down the general homogenous solution.
Four steps there.
And finally I want to write down the total solution.
And the total solution is simply vCP+vCH.
And vCP was VI and vCH was Ae^(-t/tau).
tau was simply RC.
That is my solution.
Now, remember the last step.
The last step was form the total solution and find out the remaining constants.
Find out the remaining constants by using my initial conditions.
At t=0, I know that vC=V0.
I know that.
And so therefore I can substitute t=0 to find the constant.
So I know that VO=VI+A.
t=0, this thing becomes 1, and so I get this equation from which I get A=V0-Vi.
In other words, my solution vC is simply VI+(VO-VI) e^(-t/tau).
So the last 15 minutes have just been math.
No electrical engineering here, but electrical engineering stopped at the point where you wrote this differential equation down, went through a bunch of math and came up with a solution.
Purely mathematically.
So here I simply used math to get you the solution.
And, as I have been promising you throughout this course, in the next lecture I will give you an intuitive EE method of doing it.
Real electrical engineers, real EECS folks don't do it this way.
Real EECS folks do it intuitively.
And I will show you how to do it in four easy seconds in the next lecture.
But you need to understand the foundations of how this comes about, and so this is the answer.
You can also get the current iC is simply Cdvc/dt.
I won't do that for you, but you can simply differentiate it and get the current.
So I can plot for you vC, time t, vC.
The intuitive way of looking at this is I have VI which is the final value of the voltage.
When t is infinity this part goes to zero so the vC is simply VI.
And then there is a component V0-VI which decays according to this starting out at an initial value of V0.
Notice when t is zero vC is V0, you can see that in the equation, and so it starts out at V0 and ends up at VI.
I start here, I end up here.
And this portion V0-VI decays out over time like this.
And this decay is governed by the RC time constant or tau.
I am going to show you very quickly a couple of examples of wave forms, one that goes like this and one that looks like this.
This is when I start with some value V0 and I don't apply any input, it should decay down to zero, t, t, vC, vC.
If I apply zero for VI then this should simply decay down to nothing over time.
And if I apply some VI but there is no state in the capacitor then that same equation is going to look like this.
You can go and confirm for yourselves that when I apply some input but the capacitor has zero state, I start at zero, I finish up at VI and my wave form looks like this.
There you go.
That's the first one.
The second one where I have 5 volts on the capacitor and no input.
Assume that at time equals zero I take away an input, short the input voltage to ground for example, apply zero volts.
You will see the decay from 5 volts to 0 volts.
And in the first case I start with zero volts in my capacitor, I apply input of 5 volts, and notice that at t=0 the capacitor rises up to that level.
So notice that these circuits with capacitor and resistors are typified by wave forms that are exponential rises and exponential decays.
We will see more of that next time.
OK. Good morning.
In the last lecture I did a little demonstration for you where I showed you a pair of inverters.
And showed you that the output of the first inverter looked weird, certainly not like anything we have seen thus far.
It looked like a slow rising transition like this.
And using that motivation we have begun our study of RC circuits.
And in particular for today the lecture is titled "Digital Circuit Speed".
We are going to look at the fundamentals of digital circuit speed.
And it all boils down to an RC delay.
By the end of the lecture, I am going to show you two numbers that you can look at a circuit and obtain by observation, multiply them out and you will get a good idea of the speed at which a circuit will run.
It is pretty amazing.
So as a quick review -- The relevant section for this is Chapter 10.4.
As a review, we said to understand things like this we need to develop the foundations for RC circuits.
And the example I covered was that of a very simple circuit that looked like this -- An RC circuit of this form.
And I also showed you that for an input of the form, input that steps from zero to VI at time T equal to zero.
And assuming that the capacitor state at time T equals zero was zero.
What this means is that the capacitor starts from rest, so at time T=0, oops, this is VI, I'm sorry.
So we assume that the capacitor starts from rest.
At time T=0 I apply a VI step, capital VI.
And then I want to look at how the voltage across the capacitor behaves.
And we did a bunch of analysis.
And at the end of the day, in the final demo in the lecture last time I showed you that the capacitor would behave like this.
It would start off at, oops.
I am sorry.
This should be, let's assume that started off at VO.
We get a different equation for zero.
So let's say the capacitor started off at VO, in which case VC at time T=0 is VO as expected.
And we showed that the output would look something like this.
After a long period of time this would come up to VI and this rise had a time constant of tau=RC.
So we wrote the equation for this waveform.
And this is the case when VI is greater than VO.
I would like you to stare at the circuit and this result here to get more intuition on what is going on.
At time T=0, VC starts off at VO as expected because I am telling you that is the case, that is initial condition.
It starts off at VO.
Then this one steps to VI.
There is no infinite transition anywhere here, and so the capacitor holds its voltage at VO, at time T=0.
And then the VI here, which is greater than VO, begins to charge the capacitor up, charge it through this resistor.
And so therefore the capacitor charges up.
After a long period of time, from the basic foundations of capacitors, we know that the capacitor appears like a long-term open circuit to DC.
This is a DC voltage VI.
So it appears like an open circuit.
So after a long period of time VI appears at the end.
And from here to here I have an exponential rise that is typified by an equation of the form -t/RC.
This kind of waveform rising from a smaller value to a higher value is typified by this expression.
We saw the expression when we developed the equations last time.
On the other hand, if the input was such that VI was smaller than VO, so let's say VI was smaller than VO then what will happen is that the capacitor voltage would start off at VO, because I am telling you that is the initial condition, and would then decay in this manner to the final value of VI which is the input.
Instead of going up this way it decays down to the final value applied to the circuit.
Again, the time constant is RC.
But this is typified by a form, this is exponential rise and this guy e^-t/RC is an exponential decay.
The key thing to remember is that when you have RC circuits of this form, the waveforms that you get are either each of the e^-t/RC or 1-e^-t/RC.
So you can now begin to see how waveforms such as that come about.
We will do an example and sit down and compute the inverter delay.
And notice that this waveform here is very typical or corresponds to this waveform that we see here.
Here I am starting at VO.
And assuming this axis starts off at zero, this one starts very close to zero and then rises up to some final value.
So far I have reviewed some material for you that I covered the last time.
As a second step, I would like to give you a much more intuitive approach -- -- that doesn't involve solving any differential equations.
And the reason I do this is that most experienced circuit designers do not sit down and write differential equations each time they see an RC circuit.
When you are starting out and you see an RC circuit, you say node method and you write the differential equation, but experienced people don't do that.
They look at it and they can sketch the waveform out by inspection.
And I will show you how to do that.
It is indeed incredibly simple once I give you some intuition.
Throughout the rest of this course, I will be showing you many such examples where initially I develop the foundations of stuff and then show you an intuitive approach that very quickly lets you either get the final answer or at least sanity check the answer that you have gotten.
And this is how experienced circuit designers deal with stuff.
How many people here have seen this movie Bend it Like Beckham?
So you know this Beckham character doesn't think about how he is going to curve the ball.
He just does it and it happens.
He doesn't sit down writing differential equations to find out the projectile trajectory and all of that stuff.
You just kind of do it.
These series of intuitions I am going to give you is going to be in line with the Bend it Like Beckham kind of intuition.
And this one in particular I would like to do in honor of one of your recitation instructions Professor David Perreault.
And so this piece of intuition is going to be termed "Practice it Like Perreault".
Watch what I do with the other names.
Professor David Perreault is really a world expert in designing really incredible power supplies for very, very small chips and so on.
He doesn't start writing differential equations to do this stuff.
He looks at it and sketches it out.
Let me show you how he would do this.
Suppose I have my circuit like before, VI, R and C, and I am telling you that VC(0)=VO.
And my input VI is a step that looks like this.
VI is a step.
How would Professor Perreault do this?
Let's do it completely by intuition.
No math here.
All right.
We know that I have told you that this guy starts off at VO.
I am telling you that.
You know it is going to start at VO.
And there is no impulse or huge infinite transition, and so the capacitor starts off at VO.
We also know from basic capacitor properties that after a long period of time, in the steady state, this is but a DC voltage.
If you apply a DC and here is my capacitor.
After a long period of time this guy is going to look like an open circuit.
It is going to charge up to some value and then is going to look like an open circuit.
Because if it didn't, you would keep charging it and its voltage would keep increasing.
That doesn't happen, it looks like an open circuit.
So it looks like an open circuit in the long run.
The voltage across it must be capital VI.
If I don't have current flowing in the circuit then the only way that can happen is -- This open circuit.
Capital VI appears across the capacitor.
Well, after a long period of time I know that the output must look like this.
In this case, I have assumed VI is greater than VO.
So you have two points of your curve, VO and VI after a long period of time.
And, as I told you earlier, with capacitors you get two kinds of curves.
Two things.
What you do is go zoop.
There you go.
You're done.
And this has an exponential rise.
This is with the form 1-e^-t/RC.
So we can write an equation for that as follows.
VC we know has something to do with minus t/RC.
This is of that form, so there has to be that term in there somewhere.
And I start off with VO.
At time T=0 this is one and this is one, so this term becomes a zero.
At time T=0 that becomes a zero so I get VO here.
I am going to make sure this stuff stays zero at time T=0, so I start off with VO.
Now, as time wears on what happens here?
This voltage here, VI-VO, if you look at this difference.
That is exponentially decaying over time.
And so therefore all I have to do here is write VI-VO.
There is the answer.
I know the form of the curve.
I am just fitting an expression that meets this form.
This starts off at VO.
When time T=0 this second expression is zero and so it is VO.
And this difference here decays down to zero.
And this difference here, VI-VO is multiplied by this term here and that is what I get.
And you can confirm this.
At time T=0 this is zero.
At time T infinity this goes to zero, this goes to zero leaving a one, and VO and minus VO cancel and I get a VI.
Virtually any such simple voltage source, current source, resistor, capacitor, circuit for most inputs like steps and so on can be analyzed in this manner.
Initial value, final value, it's simple.
And just to show you that this is simple, I am going to label this expression this way.
It is of the form 1-e^-t/RC.
Just remember that.
Now, by the same token, what if VI had been smaller than VO?
Then that is simple, too.
I would have had my VI being here.
VI would have been here.
And that is of the form.
In this particular situation, here is my VI, my starting value and I do this.
And just to label that, let me label that this way.
I just told you that for RC circuits you go this way or you go this way.
So it is down here.
I get some kind of an exponential decay.
And, like before, think of this one.
This one has VI as a base value here.
And the difference between the two is VO minus VI.
And that difference decays.
So I have a VI out here, and this difference decays so I get VO-VI and that decays in this form.
So I get an exponential decay of this difference here.
Just stare at it for a while longer.
You should be able to just go and knock it off like this, just like Professor Perreault would.
No differential equations.
Just write it down by looking at the curve.
Let's keep these two in mind, OK, these forms?
One is the 1-e^-t/RC form and the e^-t/RC.
Both have a time constant RC.
Let me just make this a dashed line just to be on the safe side here.
That is our first piece of intuition.
And, as I pointed out before, in problems you face in life or in ones that we give you, feel free to use the intuitive method.
Or what you can do is apply the mathematical method and then check your answer by using your intuition.
What I would like to do next is apply what you have learned so far to figure out what we set out to figure out, which is the delay of my inverter.
I had promised you that by the end of this lecture I was going to close the loop on that little demo.
I was going to close the loop for you on this little circuit that we had looked at, one inverter driving another inverter.
This was A, this was inverter X, and this was my node B.
The green curve you see out there, the middle one has a transition shown up there.
And what I am going to do next is use the results we have gotten so far to compute a number.
We are going to compute a delay number both for a rising transition.
We will call that delay DR for rising transition.
And we will compute a delay for the falling transition DF.
Remember, that this is the input that falls down sharply.
The intermediate node B rises much more slowly.
And because this rises much more slowly this guy here falls a little after this transition here, and so there is a delay.
And I am going to apply what we have learned so far and do an example for you and figure out what that delay is.
This is an absolute foundational calculation done in building digital circuits all the time.
It is remarkable that something so simple is used in designing even the most complex of circuits to obtain very quick ideas of what my delay will look like when I have some subcircuit driving some other piece of subcircuit.
Let me just draw a few equivalent circuits for you.
The internal circuit looks like this.
This is my inverter X, A, my node B.
And notice that I have this capacitor CGS.
Since I am interested in this node, let me show you that, this capacitor explicitly, it is because of this capacitor here that arises because of this MOSFET here between the gate and the source.
And that capacitor gives rise to this RC thing that we are seeing.
This is RL, this is RL, VS, VS. And let's say, just as up there, at time T=0 I get a transition like so, a falling transition from say 5 volts to 0 volts at the node A.
This is VA here.
That is shown up there.
And VB -- We had expected that VB would look like this.
We expected VB to be instantaneous and looking like that, but instead because of the capacitor VB looks like this.
And remember, again, this is of the form 1-e^-t/RC.
And we will write down the answers by inspection.
From this let me draw the connection to circuit delay by showing you another little graph here t, VB, zero.
And what I am going to show you, this is 5 volts.
And so the output goes like this from close to zero to 5 volts.
It is close to zero.
Because, at least with the inverters we have been seeing in lab and so on, the RON for the inverter is very, very small compared RL.
So it is virtually zero down here.
And so what is the delay?
I mentioned there are two delays of interest.
One is the rising delay.
That is the logical value at the end, if I wait a long enough period of time, is a logical one.
Delay is simply defined as starting from here how long does this output take to get to a valid one?
At what voltage here can I say that this transition corresponds to a logical one?
At what voltage here can I say that that represents a valid one?
Any ideas?
Yes.
It depends on the discipline, bingo.
So it depends on the discipline.
Now let's get more specific.
Since it depends on the discipline, at what value based on something in the discipline can I say this thing is a logical one?
This is an output remember.
VOH, bingo.
There is some VOH somewhere.
And it takes some amount of time to get to a valid logical one output, ergo there is your delay.
This is tR.
And I call this the rising delay of the inverter X.
It is interesting that the rising delay of inverter X, based on our model, depends on the parameters of this inverter and the parameters of whatever it is driving.
So remember that the delay is not necessarily just the property of the inverter itself, but it depends on the context.
If I stick my inverter before another inverter like this, it is the capacitance on that inverter by our model that tells me what the delay is going to look like, of course in addition to RL.
And we will do the math in a few seconds.
By the same token, if I had this wire connecting not to one inverter but going to ten other inverters, I expect to have a capacitance equal to ten times CGS.
And so therefore this thing should rise even more slowly, correct?
The more capacitance on here the slower it rises up.
Simple.
If I put more and more load on this line by putting more and more MOSFETs on that line, more and more inverters this will rise slower.
In our example I just have one, so let's go ahead and compute the delay.
This is called the rising delay of X.
That says that for this node here to go from its output value to a valid one, which is VOH how long does it take?
Notice that if this capacitor was zero then you would have seen an instantaneous transition.
If you have an instantaneous transition then notice that the rising delay was zero.
That was the model we had looked at up until learning about capacitors.
So let's go ahead and compute the number.
I can draw an equivalent circuit for computing a rising delay.
The equivalent circuit for the rising delay looks like the following.
The VS voltage source, with a resistor RL and a capacitor CGS, because when I turn this guy off, this guy has gone off, and so as far as the rise time of this node is concerned I can look at this circuit, ground through CGS through RL through VS back to ground.
And just for simplicity, let me draw this in a form that we understand.
CGS.
Let me use this as my ground node.
And this is the voltage VB.
And this is RL.
And V is simply VS once that transition happens.
My other equations here, VI=VS.
And what is VB(0)?
VB(0) is at what value does this node start out?
Notice that for simplicity here if this RON is much, much smaller than RL, then this node would be very close to ground.
So I will just go ahead and say that VB at T=0 is approximately zero.
And then what I want to find out is what does the value look like for time starting from zero and then going forward?
Well, we have become experts at this now.
Let's do the intuition here.
Start off with zero.
That's good.
Because my initial value is zero, I start off here.
What is the final value?
After a long time, since this is a DC voltage, what would be the value at VB after a long time?
Pardon?
VS.
If I wait long enough then it is going to be at VS.
This is greater than the initial value, so we're done.
That is my 1-e^-t/RC form.
It took me three seconds there.
It's pretty cool.
We could add the expression for this.
And the expression was I take my starting value, which is zero, and I add to that this difference VS and I multiply that by this form.
There we go.
And remember I get this from that rising form up here.
V0=0, this is zero, so it is simply VI times that, and VI=VS.
I really would like you to get this intuition.
If I had two choices, one is that you understand the intuition and are able to sketch that versus in your sleep be able to solve the differential equation and get to the answer.
I would much rather you get the intuition, if it is one or the other.
It is very simple.
Start off at zero, I go chuck, and boom, I get to VS and this is my 1-e^-t/RC form.
I need to compute tR.
And tR is the time that this takes to get to VOH.
For what value of time, for what T, does VB reach VOH?
I want to find tR.
What's tR?
From that equation, that simply tells me the trajectory of VB as a function of time.
And so I need to find out what is T for which VB is VOH?
I write VOH=VS (1-e^-t/RC).
So after a rise time my output is going to be VOH.
And so let me go ahead and find tR.
Let's see.
I bring this to this left-hand side and divide VOH by VS, and then I move things around and what I end up getting is -tR/RC and on the other side I get ln(1-VOH/VS).
Divide VOH by VS, that is this, move this to the other side, and move e^-t/RC to this side.
And take logarithms on both sides.
This is what I get.
tR is therefore -RLCGS ln(1-VOH/VS).
That is my rise time.
You can just do this by inspection.
It is just so awfully simple.
Just to give to some intuition with numbers and so on.
Let's say that RL=1K, VS=5 volts, VOH=4 volts, CGS=0.1 pF.
This happens so often that we often time call it "puff".
0.1 puff.
It is pF, it's called puff.
If it is nF, I don't know why they didn't call it "nuff".
They just call it nanofarads.
TR for these numbers gets to be one times ten to the three times point one times ten to the minus twelve for pico-farads ln(1-4/5).
And if you do the math you get this down to 0.16 nanoseconds.
This means that if I had an inverter like that droving another inverter then my output transition would be delayed by 0.16 nanoseconds.
Trust me, when Intel builds microprocessors or when Broadcom builds its cable modem chips, they have to do this one way or the other using a computer tool or by hand for virtually every little subcircuit in their chip.
That is how you get the delays or some approximation thereof.
What I want you also to do is, for no particular reason, I will just compute for you the following quantity RLCGS.
The time constant of that circuit for no reason at all.
I am just going to compute it and stick it here.
And RLCGS 1 K times 1 pF is simply 0.1 nanoseconds.
I am just writing it and sticking it there for no particular reason.
The next step let's do the falling delay, DF.
That is the rising delay.
And, although I didn't show this to you in the demo, there is a corresponding delay of the fall time.
It doesn't fall instantly, but rather it falls rather slowly.
Let's draw the equivalent circuit for when the node X falls.
Notice that in my inverters here, this node starts off being at VS.
This is high.
And this is going to fall because when I turn this transistor on it is going to pull this node to ground or it is going to fall down.
And what is the equivalent circuit?
The equivalent circuit is that ground through capacitor to this node.
At this node I have RON connecting to ground and I have RL connecting to ground through VS. Let me draw that little circuit for you.
Remember life begins and ends on storage elements, so I will draw them first.
My storage element CGS.
That is VB.
And, as I said, this is node X, it goes from RON to ground, and it also goes through RL through VS to ground.
And in this particular situation VB of zero for the following delay, VB starts off at VS so VB of zero is VS. And the final output I am not sure yet.
What is the final value of the voltage at this node?
I don't know that yet.
I need to compute that.
So what I will do is whenever you see something like this, a capacitor connecting to linear stuff, or a nonlinear element connecting to linear stuff.
For no apparent reason you should at least think about what?
Think Thevenin, exactly.
And then see if you can use the Thevenin method to simplify your life.
Capacitor, a bunch of stuff here, I need to find out the initial value.
Oh, I know that.
That is VS.
Done.
I need to find the final value using my intuitive method.
For the final value, I could do it just by looking at this, but I wanted to throw in Thevenin.
Hey, let me try to the Thevenin equivalent and see if that makes my life any easier.
VTH.
The Thevenin method says that you can replace this circuit here with a Thevenin equivalent of the sort for the purpose of determining what happens at this node given that that is linear.
So I need to find out that for the purpose of determining what happens at the node X. I have to replace this with its Thevenin equivalent.
And I now need to find out RTH and VTH.
So I get RTH by looking in here, shorting this guy and looking at the resistance.
So I look in like this, then I short this guy here and I get RL in parallel with RON because this one shorts to ground.
So RTH is simply RL in parallel with RON.
This is a convenient notation for RL being in parallel with RON.
And you all know the value of that.
It is another one of our very simple patterns like voltage divider and so on.
Resistances in parallel can be computed as RL RON divided by RL plus RON.
What is VTH?
VTH is the open circuit voltage here.
If I take out this capacitor, I want to find out what the voltage here is.
Ah-ha, voltage divider.
VS, the voltage divider here, RL and RON.
I could write this down as VS times RON/(RL+RON).
Remember you will see again and again and again and again in 6.002 or any circuit stuff that you do, you will see them all over Thevenin.
Voltage dividers, current dividers, resistances in series, resistances in parallel, RC thing-a-ma-jigs like this.
So if you just remember those 10 to 15 intuitive patterns then you are pretty much set for life.
It just comes on again and again and again.
Parallel resistors.
Voltage dividers.
You should be able to write down a voltage divider in your sleep.
So this is what I have.
Let me now write down intuitively what I expect the node X to do just by inspection.
Let's see.
What is the initial value of the voltage across the capacitor, intuitive method?
This is how Professor Perreault would do it, remember?
He would start off by saying ah-ha, initial value is VS because I am told it is VS.
I start off with VS. And so I start off here.
What is the value after a long, long time based on this circuit here?
V Thevenin.
After a long time this is a DC voltage because that is a DC voltage.
The capacitor looks like an open circuit after a long time.
And VTH appears there so it is simply V Thevenin.
And then when you see those two, boy, I love doing this, you go like this.
That is the coolest part.
And then I am done.
It is so simple.
Three seconds or less, I am able to tell you what the delay of an inverter is purely by intuition, completely intuitively.
I mean I haven't done any solving.
It is just by observation.
Took this circuit, made my life easy, Thevenin, looked at RTH, VTH and then sketched it by inspection.
Again, if you find that things are really, really, really simple don't be surprised.
Once you get some conceptual understanding things are indeed very simple.
You can eliminate a lot of math just by staring at things attempting to build up the intuition.
As a next step what I can do is write down the expression for VB.
And I write down the expression from a falling transition.
How do I do it?
What was it?
What is the method?
I take the lowest value of interest here.
That is VTH.
And then I add to that this difference decaying exponentially.
And that difference is simply VS-VTH.
And that decays exponentially.
This form is the e^-t/RC form.
And, boom, I am done.
Many of you are wondering, Professor Agarwal, if life was so simple, why on earth did you have us mess around with those differential equations to get here?
You show us differential equations and then you don't use them anymore.
Well, that is a good question.
The answer to that is that you need to understand the foundations.
Once you understand the foundations you can find simplifying techniques to get to where you need to be, but you need to understand the foundations.
You need to at least see why things are the way they are at least once.
Understand the foundations and then find intuitive ways of getting your answers.
So now my falling delay here is, I start off with VOS and I need to get all the way down to what value to compute.
At some point here, this is a valid one, at some point VB becomes a valid zero for the output.
And that is when I stop my tF block.
What is the value here for this to be a valid zero?
Don't all yell at once.
VOL.
I simply had to figure out what is the value of time, this is Page 7, for which this expression decays down to VOL.
So it is VTH+(VS-VTH) e^-tF/RC.
Then I simplify this.
How do I do that?
VOL-VTH.
Then I divide that by VS-VTH.
So VOL-VTH.
Divide that by VS-VTH.
Take logarithms on both sides and then multiply by RC.
So I get tF is -RC log of that.
This is R Thevenin and this is CGS.
How did I get this?
VOL-VTH divided by VS-VTH.
Take logs on both sides.
And then multiply throughout by -1/-RC and I get my tF.
Done.
Let's do it for the same set numbers, just that we add an RON of 10 ohms.
I will do this for RON of 10 ohms and compute the value for you.
tF=-RTH.
RTH is RON parallel RL.
This is 10 ohms.
That is 1K.
So 10 ohms in parallel with 1K is approximately 10 ohms.
So let me just use approximately 10 ohms.
1 pF, that is RC times ln of VOL.
Oh, I need to give you a VOL.
Let's say my discipline has VOL being 1 volt.
And so therefore I end up getting a VOL-VTH divided by VS-VTH.
Since RON is much, much, much smaller than RL, since RON is 10 ohms and this is 1K, most of VS will drop across RL.
This is a hundred times smaller.
Compared to VOL, which is 1 volt, VTH is very, very small.
VTH will be on the order of 0.05, and so therefore I simply write down VOL here and say VTH is approximately zero, and I get VS-VTH.
This is approximately 5.
So let me just say this is approximately.
And if you do it you will get 1.6 pico-seconds.
Again, just for fun, let me write the corresponding RC time constant for the circuit, which is RTHCGS.
So RTH is approximately 10 ohms and CGS is 1 pF, so this is 1 picosecond.
Now you will understand why I have been writing this time constant down.
It turns out that the time constant is a very, very important number.
So you see an RC circuit, and you compute its time constant for an RLC connection like this, it is the series resistance times the capacitor.
The time constant is a very important number.
And usually the circuit delays are in the neighborhood of the time constant value.
In this case this is 1 pS.
That is 1.6 pS.
And in this case we had 0.1 nS and 0.16 nS.
So the time constant itself is a good indicator of what your delays are going to be like.
If you have no time, you are sloshing your cereal down in the morning and you need to know how long the delay of the inverter very quickly, you have three seconds.
Just do the RC and that is a good first approximation.
What I would like to do next in the last three or four minutes is set up a little demo for you for your recitation, and then your recitation will cover it.
This is a true story.
This really, really happened.
In this West Coast school, which shall remain nameless, they had a chip, they built a chip.
And the chip had a bunch of pins, as you might imagine.
And the pin, as you have a trace on a board, a wire on a board there are some capacitance attached to wires, between the wire and ground.
And that is a capacitor.
And they just called it a load capacitance.
It could have been 0.1 pF or 0.01 pF or something like that.
What they found when they built this chip -- What they found was that the voltage here they expected to look like this, this computer science abstraction and so on, zero to one transition, boom, it should look like this.
But for the reasons we saw today the observed transition was much slower and looked like this.
So the students said ah-ha, let's speed up this chip.
We can speed up the chip by looking at the RL and RON of my driving inverters.
And if I make RL small -- Notice if I make RL small my delay is small.
If I make RON small my falling delay is small.
So let's make really small RLs and RONs and let's all have fun.
Unfortunately, what they observed was that by making RL and RON both small, the RC time constant small they expected to see a much sharper rise time.
And this was the original.
But what really happened was -- They expected this to get faster and kind of look like this, but what happened was disaster struck.
What they observed was something like that.
This is a real-life story.
And so instead of getting something like this they go something like this.
And why is that a problem?
That is a problem because notice when I expect to be at a zero, I got some spikes that went higher than VIL into the forbidden region and did bad things to me.
So let me show you a little demo and show you that that's exactly how the circuit is behaving.
Notice that this is what I expect but this is what I see.
Look at the purple curve here.
Notice these spikes that are showing up there.
This is true.
They saw it happen.
And why is this happening?
It turns out that what was happening was that the two pins were next to each other.
And I will show you a little demonstration here.
Let's see if you can figure out why this was happening.
Think of these as two pins and the pins are close together.
I am just modeling the two pins with a role of wire.
And what I am going to do is -- I am going to separate the wires and keep them far apart.
It is like keeping my pins far apart.
Hey, guess what happened?
Those nasty spikes went away.
But then I cannot keep my pins 1 meter apart on a chip.
Your laptops are going to look 20 yards long.
You want the pins to be very close to each other so that you can have many pins on chips and therefore have very small systems.
But then look, I get the spikes.
Any idea why that is happening?
Why is that when the pins are close together I get those spikes?
Any ideas?
Somewhat?
We just learned about capacitors, so this must have to do with capacitors.
There is this parasitic capacitor between the pins, exactly.
Here is what is happening.
Here is what I expect.
I expect a nice square wave at the output.
But instead I have a pin next to me.
And I have a faster wave form driving it.
And so therefore there is a parasitic capacitor here.
And because of that I get something called "crosstalk".
And the model for crosstalk is some resultant resistance with the parasitic capacitor and I get those spikes.
And the 6.002 experts saw the solution.
They said how do we fix this problem?
6.002 experts said the way we fix this problem if it is slow it may be better.
Instead of having sharp transitions let me drive it with slower transitions.
Let's switch to the demo again.
You will see this in recitation, but I will show you the demo very quickly.
I have a sharp transition of the input, which is that yellow thing out there.
I am going to make the transition slower.
Switch to a triangular wave.
And you will notice the spikes go away.
Oh, no.
That is the wrong one.
The other one.
There you go.
The moment I switch to a slower transition boom, the spikes go away.
You want to switch back to square?
There you go.
The 6.002 experts saw the solution.
Slower transitions.
And you will do this example in detail in Section tomorrow.
Thank you.
I will be replacing Professor Agarwal today because he is away.
I am one of the recitation instructors for those of you who have not seen me.
We will talk today about a neat application of RC networks and expand those to application in MOS memory systems.
To connect with everything, we will get back to the basic circuit that we have been discussing so far.
And you recall the circuit that we have been studying, the canonical RC with an input voltage function of t. And we had specified that we solved this problem for the case of a step input or a condition in which a t=0.
At t greater or equal to zero vI is equal to some capital VI value that for now on is constant.
And the other condition that we discussed was the value of the voltage on the capacitor that would exist at time t=0.
Let's call that vc(0).
And in general there is some finite value here.
It can be zero or it can be different from zero.
Given that, we learned how to write down directly, without messing around with differential equations, the answer for the voltage on the capacitor vc(t), let me define also my vc right here, is equal to VI, the final value, plus vc(0), the initial value on the capacitor, minus the final value, e^-t/RC.
This is our standard equation to which we plug in, and it's either a rising exponential if VI is larger than VC or a decaying exponential if VI is a smaller value than VC.
This should all be familiar.
And, again, as pointed out in the notes, the reading for today is 10.3 and for the new material you should look at Chapter 11 where we discuss memory.
This is where we stood as of last time.
Now, I would like to discuss a little bit more about the storage of charge in capacitors.
And how we can take advantage of that for storing logic state.
One of the things that I am sure you must be aware of is that one of the perhaps most massively produced chips is actually the so-called DRAM which you find in every PC and every computer that exists anywhere.
This DRAM is dynamic random access memory in which we can store a state and come back and look at it at any time later, provided we don't power off our machine.
The logic state in the basic memory elements, of which instead there are close to 1 giga elements per chip, are stored on capacitors.
And so we will play a little bit with that concept today.
And, although we're not going to discuss the specific example of the DRAM, the basic elements of the DRAM you will see actually in a demo shortly.
So that's the general response of this network that I have here to an input VI that happens at t=0.
Now, the one thing that you recognize immediately is that it really doesn't matter what the value of VI was for t less than zero.
What really counts is the value of VI at t=0.
And that's the value that we're interested in.
Now, there is an implicit statement in that.
And that statement is that somehow that network appears like this at t=0.
So, there has to be some switch there, and you will see that, that basically starts my condition to that at t=0.
And so the history of VI really doesn't matter.
The response following that equation that we have there will depend on the initial value which is vc(0) here.
Now that is the voltage on the capacitor at that time.
And then assuming that VI is a value that is larger than vc(0) will have a rising exponential that will come to this value.
And this is the time constant RC and this is time.
So, the capacitor starts with some voltage here and goes to a new voltage that is imposed by the input for time greater than zero.
We can define at any one time, say this time, this time, this time, this time the state of the capacitor.
The state.
What is the state of the capacitor?
The state is the summary of all inputs that are relevant to predicting the future.
If I know the state of the capacitor this time, I can predict what it is going to go given a response VI here in the future.
So, predicts the future.
Now, what is the state variable on the capacitor?
What is actually stored on the capacitor?
You can say, well, what is stored is voltage.
The real physical quantity that is stored is the charge q which is for linear capacitors related to the voltage, let me actually write it correctly, vc like this.
So, the real state variable is this.
But for a linear capacitor, since there is one-to-one relationship between the two, v is also a state variable.
OK, so let's then go back to our original circuit.
What we have is -- -- vc(t), so that's the future value of the voltage on the capacitor, is a function of vc(0), the initial value and the variable input now in the future time.
And for the case of vI(t) being constant VI for t greater or equal than zero we have the equation that we just described.
Nothing new.
All the past inputs to the capacitor for time t less or equal to zero is summarized in this value.
And vi being constant the future is predicted from that.
So, that's the concept of the state.
There is an initial state on the capacitor.
And then there is a final state that will be reached when equilibrium actually is achieved.
There is a fair amount of discussion in the text, and we don't go in great detail here, but it is both convenient for analysis and also it's interesting in many cases to look at the response of a linear network for two different conditions.
So, we're interested in two cases.
One is the so-called zero state response.
Now, what is the zero state response?
It's the response to a condition in which we impose an input and impose also the condition that the initial value, initial state of the capacitor is zero.
So then we ask how does it respond to vi(t)?
So, starting with a capacitor at zero state what is the response?
It allows us to decouple the initial conditions from the response to the input.
Now, you will see that this is actually very useful.
The second condition to which we're also very interested is the so-called zero input response.
What is that?
That is vi(t)=0.
Now, it's the condition under which there is no input.
vi(t)=0.
The question here is how does it relax?
We're starting with an initial state.
So, how this state relaxes out in the circuit.
Now, the zero state response, this one here is Z so called SR for our case, which I will write like this, vC, ZSR is simply a rising exponential.
We start from zero and we go to VI.
So, it's VI-VI e^-t/RC.
So, that's the ZSR.
The ZIR, the zero input response is like this.
It's the decay of the initial voltage on the capacitor to zero or to equilibrium.
Starting from vC(0) we're decaying like this.
Now, do you see something that's rather obvious from what's on the board in terms of ZIR and ZSR and the final complete answer which is there?
They are specific cases, but how do they relate to the full answer?
It's the sum.
It's the superposition of the two.
What basically we see here -- And that's actually a general statement, is that vC = vC,ZSR + vC,ZIR.
Now, you may say this is trivial because we started from that, ended back in that from some very simple observations.
However, we are not always solving networks for responses that are steps.
The input voltage may be a ramp.
We did that in recitation.
Or, it could be an impulse.
Or, it can be a more complicated function.
Having this observation in place actually allows us to solve the problem rather neatly.
If I have time at the end, I might come back to this.
So, this is the same equation as I started with, arrived at from a principle of superposition of two different solutions.
One application of state which can be, since we have energy storage element here, the capacitor, which can be stored on the capacitor is in memory.
And you may ask, so why do we need a memory node to perform logic?
Well, there are cases in which a result depends on previous results.
So, a computation proceeds in time.
In order to do that, we need to store intermediate results and proceed forward.
One good example is if you're doing a continuous summation, say, on your calculator, you keep putting things in the memory.
The M+ button, right?
And you keep adding a series of numbers.
Every time we store the sum of the previous operation we add another number and so on.
Clearly we need some way of storing state.
For a complete computing system, we need combinational logic and we need memory.
In fact, these are the two basic elements that are essential for any kind of computing system.
We need to remember intermediate results.
We need to remember transient inputs.
And that's the role that all these enormous amount of memory that comes to play in computers is doing.
The basic memory abstraction is as follows.
Imagine a block which needs to be populated by transistor, resistor, capacitor, whatever elements.
And it has a control input, which we will call the store.
It has a state input that we will call dIN and has an output dOUT.
When we're telling this element, OK, now it's time to store, it looks at the input dIN and stores it for, in principle, an infinite amount of time.
If we were to make a drawing of this, of what this looks like, let's suppose, let me do all this in one axis.
So, time moves this way.
Let's suppose that we have an input dIN that looks like this, and the store command comes in the form of a logic.
Let's actually suggest here this is logic one, this is logic zero.
And, although this is not absolutely necessary, let's also define that the store command comes in the form of a logic one at this store input.
Store, let's say, looks like this.
What does the output look like then in this particular case?
Assuming that the output was dOUT, the stored element was zero prior to the store, then the output would look like this.
This is dOUT.
As you can see, it would remember the one that it saw at this point.
In fact, it would do that irrespective of what was stored in this memory cell.
For example, suppose it was storing one and the output didn't change, it's still one.
If it was storing a zero, it would flip to a one.
If we had another store, let's say here, then what happens?
Then it would go back down to zero because now we sampled an input that is zero and we flipped the state.
That's what a memory -- -- element or cell would do for us.
It would remember the output state.
And, not only that, but in principle it should be undisturbable.
In other words, I may do something to this dOUT but it should not flip the state.
And that comes about quite a bit.
Because in actual integrated circuit memory there is lots and lots and lots of nearest neighbors to this cell which, when they're flipped, have a cross-coupling to the cell.
The cell must be designed robust enough that it doesn't flip, that no coupling actually occurs.
All right.
Now we're going to try to apply what we've learned so far to invent a basic memory element.
And, believe it or not, this is the key to the DRAM.
Let's implement this in a circuit.
Suppose I have a switch here like this.
And I will put a capacitor.
I take my dOUT here.
This is dIN.
And the switch is operated by a command here that we will call store.
When store is one it goes up.
When store is zero it is down here.
That's capacitor C. This is the storage node.
What are we actually storing in this case?
Let's suppose that this voltage here is 5 volts.
I flip the switch up to one and I flip it back down to zero.
What's the voltage in this capacitor here?
5 volts.
Now the capacitor is at 5 volts, I put dIN to ground, flip the switch back up and then back down to its known storing condition.
What's the voltage in the capacitor?
It's zero, exactly.
So, it does store the value of the voltage that it saw, five or zero, high and low.
It stores it because it stores charge.
That's actually the physical quantity that's stored.
It's manifested as a voltage, which we see.
All right.
Now, is this, oh, before I move from here.
What is the basic cell in a DRAM, one that you go out and buy by the billions of cells?
It's actually this.
The only difference is that this switch here is replaced with a MOSFET.
And that's all it is.
So, a MOSFET plays the role of the switch.
When the gate is high this is a resistor and connects the input to the capacitor.
And when the gate voltage is below the threshold voltage this is an open, as we've seen, and it isolates the transistor from the output.
So, that's the basic memory element.
And, as I said, it's the key to a DRAM.
OK. Now let's consider a little bit the conditions of operation of this thing.
Let me draw the circuit in two conditions.
One in which it is storing, one in which it is sampling and one in which it is storing.
Not to redraw this thing.
Assuming that I have a MOSFET there, I would have the on resistance in place here when store=1.
Now, in principle, the output is connected to -- -- some load resistance.
We'll talk a little bit more about this load resistance in a minute.
This is the situation when we are at store=1 situation.
For example, let's suppose that dIN is 5 volts.
Now, what is the situation for store=0?
It's very simple.
We have the capacitor C and dOUT and here we have a resistance.
The switch is open.
This is store=0 condition.
What we have in this case is we have a problem similar to what I was discussing earlier.
It is a ZIR, if you like, situation.
And this you can think of as a ZSR if we're starting with zero charge on the capacitor, but I'm interested in this part.
In this case, I am starting with a vC(0)=5 volts.
And I'm asking myself how long will this cell hold the value?
And, in fact, that is actually what happens in a dynamic RAM.
The value on the capacitor is not stored forever.
In fact, that's why we call it dynamic because we have to come back and restore it every once in a while.
For how long are we going to store the charge?
What's the response of vc for t greater than zero after the switch flicked?
It's very simple.
It's vc is equal to 5 volts e to the minus t over RC, right?
That's the response.
We have a decay.
And applying to the things we know.
We start from 5 volts, let's say here, I have a decay going down towards zero, at some point we are going to cross the threshold for high.
The only period in which I have a valid output, if the capacitor was storing a one, is this period here.
This is the only period in which I have valid stored one because, once I go beyond capital T here, I have crossed the legal limit, threshold for discriminating a high output.
And from then on the output is no longer valid.
So, this memory is good provided time is less than capital T. It's not a case in which the capacitor can hold charge forever.
In fact, we can calculate, that is we can solve for T in this particular case.
It's in your notes.
Nothing really profound.
T is equal to minus RC log VOH over 5 volts.
So, this is basically what the response is going to be.
Now, there is an implicit assumption here, which is that the store pulse width is much, much larger than RON C. In other words, when we want to store a one here starting from zero, we better charge it all the way up to 5 volts in the time that our switch is connected here.
And what is the relevant time constant?
It's going to be the RON C. In fact, it's actually the RON parallel RL with C. But typically RON is much, much less than RL so we don't have to worry about that.
Dominant time constant is RON C. So, provided these things are happening, we have a memory.
Now, we can try to improve things a little bit.
We see here that we will have a decay to an invalid state in time T. How can we improve things?
One way to improve things are the buffer.
Here is our memory element again.
Here is the capacitor.
This is the storing node.
Now I am going to put the buffering effect.
I am going to put two buffers here.
Two invertors, I should say, because if I am storing a one here I want to be able to see a one here as well.
And, in this case, what I am looking at is the RIN of the buffer.
And, in principle, I have out here the RL.
Now, this is better because if RIN is much larger than RL then the time T, in this case, is much larger than the case without buffer.
So, we buffer the effect of VL.
This could be one of these neat circuits we saw in recitation like a source faller, for example, or it can be just an inverter in which case you just see the input of a transistor.
So, now this condition can be satisfied.
Let me give you some cases which are some numbers that are typical for a dynamic RAM.
Typical times we're talking about is RIN on order of 1 gigaohm and storage node capacitor on order of 1 femtofarad to one picofarad.
Now, if you can do the math in your head, which is just multiplication, you will see that the time constant, the RC is between 1 millisecond to 1 microsecond.
And for DRAMs, actually, we try to be in the order of milliseconds.
These are the times we're talking about.
If I have this kind of circuit, somehow there has got to be additional circuitry that comes back, samples the voltage here and restores it.
And that is actually what is happening in a DRAM.
And my laptop is working there and its DRAM keeps getting refreshed every, say, millisecond or whatever the condition is.
But, in our case, we are going to do a slightly different case in which we will create a static memory.
Let's actually look at, first of all, the case of the discharge.
Pay attention to, let me actually break the loop here.
This is my capacitor.
This is a resistor that is in series with a capacitor like you see here.
Actually, I am going to keep that resistor in series with the capacitor, even in this case, because I have it for my second part of my example.
I charge the capacitor to 5 volts.
And you can see here this lights up, I hope everybody can see it, proportional to the voltage that I have here.
From here on it's all logic levels.
So, the intensity of light here will always be the same.
It's either lit or it's not lit.
Right now I am charging the capacitor.
In fact, let's see.
Maybe I can discharge the capacitor first.
Here the capacitor is discharged.
As you can see, the input is zero, the output is a one, and then the output of this inverter here is a one.
I have two inverters in series.
And I am going to charge the capacitor.
I charged it to 5 volts and this lit up, this is off of course, that's an inverter, this is a valid zero, produce a valid one.
And now I am going to take the input out.
As you can see it's stored.
In fact, we have to wait for a very long time.
We don't have enough time to wait for this to discharge, so instead what I am going to do now is I am going to add also the resistor.
Now I am going to flip the resistor in parallel with the capacitor to imitate what happens when we have an input resistance.
You saw that there was a discharge of the capacitor.
This input level went down.
Voltage here flipped over to a one.
Let me do it again now with a resistor in place.
Storing charge on the capacitor.
That's the store command.
Now, don't store.
I have less, about a second.
The element here is 20,000 microfarads and 100 ohms which gives me a time constant of two seconds.
Assuming a VOC of the order of, let's say, I don't know what it is for this case, 2.5, the log would be about 0.5, so it cuts basically the time to about one.
So, it lasts about one second, if my math is all correct.
It's actually a little longer than a second, excuse me, but the point is that the charge is gone.
Now, notice, however, that there is something I can do here, which is that suppose I take the switch or a switch and bring it back and provide a path from the output to the input here.
And this switch is open when this is closed and closed when this is open.
So, this basically is the compliment of store.
What I am doing now is I put a charge here, it produces a valid one at this point, and then I am feeding this valid one back to the input.
As you can see, this will now allow me, even though I have a high resistance, to store the value for a long time.
In this case, what I am going to do is I am going to connect the output, as you can see here.
And I have my resistor in.
And I am storing zero here, storing 5 volts.
Now I am going to flip the switch.
Basically, I mean the don't store, don't look case.
You notice this dims a little bit.
Sorry.
No, I want the resistor in.
There.
Yes.
OK, so the output remain value.
This dimmed a little bit but the output has remained OK. All right.
So, we've provided a feedback.
Now we've created a static memory.
This will hold charge for as long as the circuit is powered up.
Now, there is still one little problem that I have with this kind of configuration.
And that is if I disturb this output the charge may, the state may change.
So, for example, let's say that I have -- I disturbed it by coming close to it, so let's charge it again.
OK.
I flipped the switch.
I flipped the state from the output.
That is an invalid condition.
I shouldn't be able to do that.
How do I avoid that?
How can I avoid this problem that you just saw?
Well, I need yet another buffer.
The answer is in your notes.
If I don't take the output here but rather take the output here, or if I don't want an inverted output, if I don't want an inverted output, I could put yet another element there.
Then the situation would be fine.
In this case, let me do it again.
Charge.
Why isn't this lit?
A bad one?
Now, of course we disturbed the input.
Now, of course I can do anything I want here.
Nothing happens, but you may say this is a trivial case because this is already zero.
So, I am going to change the state.
Here's is the changed state.
See.
I can show this.
Nothing happens up there.
So, this is an interesting situation in which I am buffering the output so that the output does not feed back to the input.
And, by and large, in designing circuits this is something that we do.
Now, in the remaining three minutes there is an example that we have.
Can we put the laptop here?
OK, so here is an example of how memory can be put together now to create something a little bit more complicated.
And you can see the memory cells that we were discussing here.
There's four of them, so this is a four bit memory.
There is a decoder at the beginning here which decodes the address of each cell, so the input here will tell me which cell I need to address.
Let's look at the truth table.
This is the truth table for the decoder.
As you can see, depending on the address that I have here, this is zero, one, two and three in a binary system, only A, B, C or D is up, is high.
Which means that this end operation here only allows the input that is presented to all of the cells, what is going through the AND gate here to appear at the output.
If, for example, we have a one, zero, the only end input that is going to be high is going to be this one.
And that means the only cell that will look at the input when the store comes up is going to be this one here.
At that point it will store whatever is on the input cell because that's an AND operation.
That is a simple example of a memory.
And following that simple arrangement you can build incredibly large memory systems.
So, that's all I had for today.
And I will see you on Tuesday.
All right.
Good morning, all.
So we take another big step forward today and get onto a new plane of understanding, if you will.
In the last week and a half, our focus was on the storage element or storage elements called inductors and capacitors.
And capacitors stored change and inductors essentially stored energy in the field, the magnetic flux.
And the state variable for an inductor was the current while that for a capacitor was the capacitor voltage.
We also looked at circuits containing a single storage element, we looked at RC circuits and we also looked at circuits containing a single inductor.
And this was a single inductor with a resistor and a current source or a voltage source and so on.
What we are going to do today is do what are called "second-order systems".
So they are on the next plane now.
And with this second-order of systems, they are characterized by circuits containing two independent storage elements.
They could be an inductor and a capacitor or two independent capacitors.
And you will see towards the end what I mean by two independent capacitors.
If I have two capacitors in parallel, they can be represented as a single equivalent capacitor so that doesn't count.
It has to be two independent energy storage elements and resistors and voltage sources and so on.
And what we end up getting is what is called "second-order dynamics".
And much as first order circuits were represented using first order differential equations, this kind you end up getting second-order differential equations.
Before we go into this, I would like to start motivating this and give you one example of why this is important to study.
There are many, many examples but I will give you one.
What I would like to do is draw your attention to our good old inverter driving a second inverter.
The same circuit that we used to motivate RC studies, one inverter driving another.
So let me draw the circuit.
Here is one inverter.
This is, let's say, 5 volts and this is, let's say, 2 kilo ohms.
And I connect the output of this inverter to a second inverter.
And what we saw in the last few lectures was that in this specific example there was a parasitic capacitor or a capacitor associated with the gate of this MOSFET.
And that could be modeled by sticking a capacitor CGS between the gate of the MOSFET and ground.
And we saw that the waveforms here, if I had some kind of step here.
Let's say, for example, a step that went from high to low.
Then out here I would have a transition that instead of going up rapidly like this would transition a little bit more slowly.
And this transition was characterized by an RC time constant.
And this is what gave rise to a delay in the eventual output.
So that is what we saw previously, single energy storage element.
Today what we are going to do is we are going to look at the same circuit, the exact same circuit, and have some fun with it.
What we are going to say is look, this thing is pretty slow, so what I would like to do is -- why don't we go ahead and put that up.
What we are going to see is that the yellow waveform is the waveform at the input here.
And the green waveform here is the waveform at this intermediate node.
And notice that this waveform here is characterized by the slowly rising characteristics that are typical of an RC circuit.
There are some other weirdnesses and so on going on here like a little bump and stuff like that.
You can ignore all of that for now.
It happens because of certain other very subtle circuit effects that you won't be dealing with, called Miller effects and so on that you won't be dealing with in 6.002.
So focus then on this part here.
It is pretty slow.
And because of that slow rising, I get a very slow transition and I get some delay in my inverter.
So you say ah-ha, we learned about this in 6.002, I can make it go faster.
How can you make the circuit go faster?
What could you do?
This is rising very slowly.
How can you make it go faster?
Anybody?
You have multiple choices, actually.
What are your choices here?
Pardon.
Decrease the time constant.
And how would you decrease the time constant?
The capacitance is connected to this MOSFET gate here.
I didn't want it in the first place but it is there, I cannot help it, so I can decrease the resistance.
Good.
Let me go ahead and do that.
What I will do is I am going to knock this sucker out and stick in a new resistance that is say 50 ohms, a much smaller resistance.
That should speed things up, right?
That should make things go much faster because this is a smaller time constant because R is smaller, correct?
OK, let's go do it.
And let's see if we get what we expect.
I have a little switch here.
And using that switch, I am going to switch in this little resistance.
Whoa, what on earth is happening out there?
This is so much fun.
What I did is I switched in a small resister here to decrease the time constant, but it looks like I got a whole bunch of crapola that I did not bargain for.
This is certainly very fast, it goes up really fast, but I am not sure where it is going, though.
Let's stare at that a little while longer.
Let me expand the time scale for you.
Look at this.
Instead of a nice little smooth thing going up.
I get something that looks like this.
It looks something like a sinusoid.
It looks sinusoidal, but then it is a sinusoid that kind of gives up and kind of gets tired and kind of goes away.
Right?
It kind of dies out.
So nothing that you have learned so far has prepared you for this.
And, trust me, when I first did some circuit designs myself a long, long time ago I got nailed by that.
I looked at my circuit, and what ended up happening was I was noticing these sharp lines at all my transitions.
When I looked at my scope, I expected to see nice little square waves but I saw these little nasty spikes sitting out there.
And then when I stared at it more carefully, those spikes were really sinusoids that seemed to kind of get tired and kind of go away.
So those are nasty, those are real and they happen all the time.
And what we will do today is try to get into that and understand why that is the case.
We will understand how to design that away.
And that is a real problem, by the way.
And the reason that is a real problem is the following.
Look at this.
Look down here.
Because this intermediate voltage is meandering all over the countryside here, at this particular point the intermediate voltage dips quite low.
And because it dips quite low look at the output.
The output has a bump here.
And it is quite possible for this output bump to now go into the forbidden region.
Or worse.
If this swing here was higher, this could have actually gone onto a one, so I would have gotten a false one pulse here.
Instead of having a nice one to zero transition, I would have gotten a one to zero, oh, back to one, oh, back to zero and then back down to zero.
So this is nasty stuff, really, really nasty stuff.
What we will do is understand why that is the case today and see if we can explain it.
What is going on here?
What is really going on here is take a look at this circuit here.
I will take a look at this path here.
So this is your VS voltage source.
Path kind of goes like this and around.
It turns out that this circuit is a loop here.
And when there is current flow, going down to basic physics you remember that I also enclose some amount.
So there is a current flowing in a loop.
And because of that there is an effective inductance here.
And, in fact, any current flowing through a wire above a ground plane, for that matter, can be characterized by the inductance.
So I can model that by sticking a little inductor here.
So my real circuit is not exactly a resistor and a capacitor, but my real circuit is an inductor as well that comes into play because of this wire.
Every wire, when there is a current flow, has an inductance associated with it.
And because of that the real circuit is resistor, inductor and capacitor.
So I end up with two storage elements now, and the dynamics of that are very different from that with a single storage element.
That is just a bit of motivation for why our study of inductors is important.
And I can draw a quick circuit here.
If you look at the circuit, start from ground, the voltage VS and there is a resistor here.
And then I have an inductor and then I have a capacitor.
So it is a voltage source, resistor, inductor and capacitor.
For this whole week we will be looking at circuits like this.
Today what I would like to do is start very simple, start with the simplest possible form of this so that you can begin building up your insight and then go into more complicated cases.
Today what I will do is simply begin with a case where I don't have a resistor here and simply study a voltage source, an inductor and a capacitor and understand what the voltage looks like out here.
So we look at the dynamics of a little system like this.
Before we go on, I want to caution you about something.
It is just happenstance that I have introduced for you capacitors based on the parasitic capacitance here and inductance based on parasitic inductance.
I would hate to leave you with the impression that inductors and capacitors are "bad".
Because when you think of a parasitic, you know, parasites.
These are parasitic.
You didn't expect them there, didn't expect this here and we got the weird behavior.
So parasitics have a bad connotation to them.
I do not want to leave you with a bad taste in your mouth about capacitors and inductors that these are just bad things.
We just have to deal with them and deal with second-order differential equations and all that stuff because they're just bad stuff and we just have to deal with them.
I don't want you to end up going through life hating capacitors and inductors.
Just because of my choice of examples, it just happened to be introducing them as capacitors.
I want to point out that these are fundamental lumped elements in their own right.
They are very, incredibly important and useful circuits where we designed capacitors and inductors because we want to have them in there.
There are many circuits that we will look at where we really want the inductor in there.
We will design an inductor by wrapping wire around in a coil and get bigger inductances and so.
Just remember that this can be parasitic in some cases, but in many cases it's good, inductors are good, so just stick with that thought.
These are mostly good so don't go around hating them.
All right.
Let's go on and analyze a basic circuit like this.
And what I would like to cover in the next hour are the foundations of something like that.
I will take you through the foundations so you understand how it works.
And, as always, what I am going to end up with is build up the foundations, help you understand why we got where we were and then help you build intuition.
And then show you a really, really simple intuitive way of doing things in terms of how experts do it.
And the real cool thing about EECS is that the way experts do things, things are really, really very simple in the end.
But you need to build up some intuition to get there.
So our circuit looks like this in terms of my two storage elements.
I have a voltage vI, inductor L, capacitor C and I am going to look at the voltage across the capacitor and my current through the capacitor.
So v(t) is the voltage across the capacitor and my current is the current through this loop here, which is the same as the current through the capacitor or the current through the inductor.
And we are going to proceed in exactly the same manner as we did for first order differential equations, write the equations down and just boom, boom, boom, boom, go down the same sets of steps but just get to some place different.
We are going to start by writing a node equation for this node here.
That's the only node for which I have an unknown voltage.
The node here is vI, so I need to find this, there's just one unknown node voltage.
And I am going to need some element laws.
For the capacitor I know the iV relation is given by the i for the capacitor is Cdv/dt.
And just to show the capacitor I am just calling it dvc/dt.
Similarly, for an inductor, L, the voltage across the inductor is given by Ldi/dt.
So this is the vI relation for the capacitor, the vI relation for an inductor.
It also suits us to write this in an integral form.
So if I integrate both sides of this equation and I bring L down to this side, I end up getting something like this, 1/L minus infinity to t, VLdt, and that is simply iL.
I am just simply replacing this with an integral form.
So this is a VI relationship for the inductor and this is for the capacitor.
So let me now go ahead and apply the node method for my circuit here.
Here, for the node method, I have to equate the currents coming into the node or sum the currents coming into the node and equate that to zero.
And while I do that I simply replace the currents by the corresponding voltages using the element laws.
So what do I get?
I get the current going in here to the inductor is equal to the current going through the capacitor.
What is the current going the capacitor?
In terms of its v relationship it is Cdv/dt.
And the current going to the inductor is given by this relation here, which is simply 1/L minus infinity to t. The voltage across the capacitor is simply (vI-v)dt.
I have just written down the node quotation for this node here.
Now I will just apply a bit of math and simplify it and get the resulting equation.
What I can do is simply differentiate with respect to t here.
And get this to be Cd^2v/dt^2, the second derivative of v. And here what I end up getting is 1/L(vI-v).
So I just differentiated the whole thing by d/dt here.
And then I just move L up here.
I bring d^2v/dt^2 out here.
And then I get a minus v here, and that will be equal to, oh, I'm sorry.
Let me leave this here.
Bring the minus v to this side so it becomes a plus and leave vI on this side.
So I end up getting LCd^2v/dt^2.
I bring L up here.
And then I take v to the other side.
Plus v and leave vI here so I get vI.
That is second order differential equation that governs the characteristics of the voltage, v. So much as the voltage across the capacitor was a state variable in our RC circuits or the current through the inductor was a state variable in our RL circuits, out here both the current through the inductor and the voltage across the capacitor are my two state variables.
And so here I have a second-order equation in my voltage, v. Again, going through the foundations here, I am now going to go through a bunch of math.
Up to here it was circuit analysis, and now I am just going to do math.
For the next three or four blackboards just math.
You can solve this second-order differential equation any which way you want.
But just to keep things as simple as possible, in 6.002 I solve all the differential equations, it turns out we are fortunate enough we can do that, using the exact same method again and again and again, the same thing can be applied.
And the method that we use to solve it is the method of homogenous and particular solutions.
So the first step we are going to find the particular solution, vP.
Second step we find the homogenous solution, vH.
And the third step we are going to find the total solution as the sum of, v is simply the particular plus the homogenous solution and then solve for constants based on the initial conditions and the applied voltage.
So let's write down initial conditions.
Let's assume, for simplicity, that my initial conditions are simply the voltage across the capacitor is zero to begin and the current through my inductor is also zero as I begin life.
Now, this is what is called "zero state".
v and i are both zero, and so the response of my circuit for some input is going to be called ZSR.
You've probably heard this term in one of your recitations.
So zero state response simply says I start with my circuit at rest and looks at how it behaves for some given input.
That is a little term you may end up using.
My input next.
I am going to use the following input.
vI of t is going to be a step, is going to look like this.
My input is at t=0 v is going from zero to some voltage VI and then stay at that voltage.
It is going to be a step.
Kaboom.
And you can see why I am going with this set of variables, because I want make this situation as close as possible to the funny behavior we observed there.
Remember we had a step, and because of the step we had some behavior at that node?
So I will try to bring you as close to that.
In tomorrow's lecture, I am going to close the loop around that and derive for you exactly the behavior we saw on the scope.
And to get there I am going to be try to be as close as possible to the constants and other parameters in the demo.
So VI is a step and zero state.
Just in terms of notation, this kind of a step input occurs pretty frequently.
And we just have a special notation for it.
We simply call it VI is the final value here.
And we call it u(t).
So VIu(t), u(t) simply represents a step at time t=0, steps from zero volts to VI.
That is just a little more notation that will come in handy at some point.
More math now.
Three steps, particular solution, homogenous solution, total solution/constants.
This is almost like a mantra here, like a chorus.
Homogenous solution we compute using a four-step method.
And four-step method for homogenous solutions, it turns out that it happens to be that way for all the equations we will see in our course.
The first step would be assume a solution of the form Ae^st.
Exactly as with RCs.
If you close your eyes and do exactly what you did for RCs you will get to where you want to be.
You assume a solution of the form Ae^st.
Substitute that into your homogenous equation.
Obtain the characteristic equation.
Solve for the roots.
And then write down your homogenous solution.
Same sort of steps again and again and again until you get bored to tears.
Particular solution.
For the particular solution, I simply need to find a solution, any solution, if not the most general one but any solution that satisfies the particular equation which satisfies that equation.
LCd^2vP/dt^2+vP=VI.
My input is a step and I am going to look for the solution for time t greater than zero.
Notice that for time t less than or equal to zero, v is going to be zero.
So I am looking for a solution greater than t=0.
Here, if I substitute vP=VI, that is a particular solution.
Because if I substitute VI here this goes to zero and then I get VI=VI, so this works.
I promised you this was going to be simple.
You cannot get any simpler than that.
I have done my first step.
I found the particular solution.
And VI is a good enough particular solution so I will use it, I will take it.
As my second step I am going to find vH or the solution to the homogenous equation.
And the homogenous equation is simply that equation with drive set to zero.
What I get here is LCd^2vH/dt^2+vH=0.
That is my homogenous equation.
I simply set the drive to be zero.
And to find the solution here, I go through my four-step method.
Again, in 6.002 following the kind of Occam's principle, we just show you the absolute minimum necessary to get to where you want.
The absolute minimum necessary is it turns out that we can solve all our differential equations that we use here by using the methods of homogenous and particular solutions.
And every homogenous solution can be solved by a four-step method.
That is about as minimal as it can get.
So no extraneous stuff there.
The four-step method, four steps.
The first step is assume a solution of the form vH=Ae^st.
What I have noticed is that students starting out are usually scared of differential equations.
I know I was when I was a student.
And the trick with differential equations is that it is all a matter of psych.
Just because you see some squigglies and squagglies and a bunch of math and so on you say oh, that must be hard.
But differential equations are actually the simplest thing there is because in a large majority of cases the way you solve them is you assume you know the answer, someone tells you the answer.
And then all you are left to do is shove the answer into the equation and find out the constants that makes it the answer.
Just a matter of psych.
Psych yourselves that this stuff is easy, because I am telling you what the solution is.
All you have to do is substitute and verify.
If you think about differential equations that way or a large majority of them, it really is very simple if you can just get past the squigglies here.
Just get past the squigglies and then just simply stick in some simple stuff and it works.
I mean it just cannot get any easier.
I cannot think of any other field where the way you find a solution is assume you know the solution and stick it in.
It has never made any sense to me but that is how it is.
So we assume the solution to the form Ae^st, you stick it in there, and you have to find out the A and s that make it so.
It cannot get any simpler than that.
Let's stick the sucker in here and see what we can get.
Substitute Ae^st here I get LCA, and second derivative, so it's s^2 e^st.
And Ae^st on this one here.
And that equals zero.
And then let me just solve for whatever I can find.
Assuming I don't take the trivial case A=0, I cancel these guys out.
And what I am left with is simply LCs^2+1=0.
In other words, what I end up getting is B, s^2=-1/LC.
My first step was, I am giving you solutions, stick them in there, assume a solution of this form.
Second step is get the characteristic equation.
And the way you get the characteristic equation is that you simply stick this guy in there.
And what you end up getting is some equation in s^2.
Do you remember what you got for first order circuits?
What s was?
What is s?
For first order circuits, what did you get as a characteristic equation?
s+1/RC=0.
The same thing.
Just remember to blindly apply the steps.
It will lead you to the answer.
This is called the "characteristic equation".
This is incredibly important.
You will see in about a couple weeks from now that once you write the characteristic equation down for a circuit, it tells you all there is to know about the circuit.
And often times you can stop solving right here.
To experienced circuit designers this tells me everything there is to know.
This is really key.
That's why it's called a characteristic equation.
I believe in problem number three of the homework that will be coming out this week, that is exactly what you are going to do.
I am going to give you a circuit, ask you to get to the characteristic equation quickly and then from there intuit the solution.
Write the characteristic equation and then just intuit solution, it's that simple.
So, step A, assume a solution of the form, step B, write the characteristic equation down.
And let me just simplify that a little bit.
I go ahead and find my roots.
And my roots here, remember that j is the square root of minus one.
And so what I end up getting is, my two roots here are, plus j square root of 1/LC and minus j square root of 1/LC.
Two roots.
And just as a shorthand notation, much like I had a shorthand notation for RC, what was my shorthand notation for RC?
Tau.
Just as tau was big in first order, we have a corresponding thing that is big in second order and that is omega nought.
Omega nought is simply square root 1/LC.
Just as tau was RC, omega nought is a shorthand here.
And so s is simply plus or minus j omega nought.
Notice that in this equation here, if you take the square root of LC there that has units of time, so one divided by that has units of frequency.
Notice that this guy is a frequency in radians.
I end up getting my roots of the homogenous equation, and that is my third step.
And as my fourth step, I simply write down the homogenous solution as substituting s with its roots and writing the most general possible form of the solution, and that would be A1e^(j omega nought t)+A2e^(-j omega nought t).
Done.
Some constant times this solution plus some other constant times, the other solution.
Plus zero omega nought.
Remember it comes from here, Ae^st.
I assume the solution of this form, so my solution in this most general case would be s being j omega nought in one case, minus j omega nought in the other case, and I sum the two to get the most general solution.
So blasting ahead.
I now have my homogenous solution.
And as my third step of solution to differential equations I write down the total solution, v=vP+vH, particular plus the homogenous solutions.
And v=VI, was my particular solution, +A1e^(j omega nought t)+A2e^(-j omega nought t) is my complete solution.
The final step, write down the total solution and find the constants from the initial conditions.
To find the constants from the initial conditions, let's start with, the voltage is zero to begin with.
This equation governs the characteristics of v, so I need to find the initial conditions.
First of all, I know that know that v(0)=0.
From there I substitute t=0.
And so this goes to one, this goes to one, and I end up getting 0=VI+A1+A2.
That is my first expression.
And then I am also given that i(0)=0.
And so I can get that as well.
How do I get i?
This is v. I know that i=Cdv/dt, so I can get i by simply multiplying by C and differentiating this with respect to t. I get C, this guy vanishes so I get d/dt of this.
So it is CA1(j omega nought) e^(j omega nought t)+CA2(-j omega nought)e^(-j omega nought t).
From here I am given that that is zero, and so therefore this guy becomes a one, this guy becomes a one, j omega nought, j omega nought cancel out.
What I end up getting is A1=A2.
From the second initial condition I get A1=A2.
From these two, if I substitute here for A2, I get VI + 2A1 = 0, or A1=-VI/2.
That is also equal to A2.
Therefore, my total solution now can be written in terms of the actual values of the constants I have obtained.
I get VI-VI/2.
So A1 and A2 are equal.
I just pull them outside.
I pull VI-2 outside and I stick these two guys in parenthesis in.
Again, I promised you no more circuits from here on until the very last board or something like that.
It is all math, so not much else happening there.
More math.
If you would like, I could skip all the way to the end and show you the answer.
But I just love to write equations on the board so let me just go through that.
I am going to simplify this a little further here.
And we should remember this form by the Euler relation, ejx=cos x+j sin x.
And by the same token, (e^jx + e^-jx)/2=cos x.
You all should know this from the Euler relation.
So were are using this guy here, ej^x + e^-jx=2cos x.
And so this one is 2 cosine of omega nought t, 2 and 2 cancel out, and what I am left with is v(t)=VI-VI cos( omega nought t).
And the current is Cdv/dt, which is simply CVI sin( omega nought t).
Just remember that omega nought is the square root of 1/LC.
We are done.
In fact, I did not give that answer the importance that was due so let me just draw.
There.
That is better.
Enough math.
In a nutshell, what did we do.
We wrote the node method, it's a very simple circuit, to write down the equation governing that circuit.
And then we grunged through a bunch of math.
Not a whole lot here.
It is pretty simple.
And ended up with a relation that says the voltage across the capacitor for a step input, assuming zero state, is a constant VI-VI cos omega t. Notice that even though I have a step input, the circuit dynamics are such that I get a cosine in there.
You can begin to see where these cosines are coming from now.
They come in here.
And if you recall the example I showed you earlier of the inverter circuit, remember there was a cosine that decayed, that was sort of losing energy and kind of dying out?
So you can see where the cosines are coming from.
And just to draw you a little sketch here.
Let me draw v and i for you and let me plot omega t, pi/2, pi and so on.
Let me plot VI.
When time t=0, VI=0, cosine omega t is one, and so VI-VI=0.
That is simply a cosine that starts out at zero here, and at pi I get cosine omega t is minus one, so I get plus VI on the other side.
So I end up at +2VI.
At this point the voltage is here.
And notice that this guy looks like this.
It is a cosine that is translated up so that its mean value is not zero but VI.
It is just a translation up of a cosine.
Similarly, in this case for the current it is a sinusoidal characteristic.
And it looks something like this where the peak is given by CVI, oh, I messed up.
When I differentiated this is missed the omega nought out there.
What I would like to do now -- This is the form of the output for a step input.
What I would like to do next is show you a demo.
But before I show you a demo, I always found it strange that I have a step input and then I have two little elements, how can I get a sine coming out of the output?
I would like to get some intuition as to why things behave the way they are.
I could go and pray to find out, but let me just give you some very basic insight as to why this behaves the way it does.
Let me draw the circuit for you here.
And this is my inductor L and capacitance C. Remember this is v. Let me just walk you through what is happening there and get you to understand this.
Now, you have seen sines occur before.
If you go and write down the equation of motion of a pendulum, you know, you have a pendulum, you move it to one side, let go.
It is also governed by sinusoidal characteristics.
And you will find that the equation governing its motion is very much of the same form, and you get the sinusoid where you have energy that is sloshing back and forth between maximum potential energy to maximum kinetic energy and zero potential energy back to maximum potential energy, zero kinetic.
So it is energy sloshing back and forth.
The same way here.
Capacitors and inductors store energy.
Let's walk through and see what happens.
I start off with both of them having the stage zero, zero current, zero voltage.
I apply a step here.
Boom, the step comes instanteously to VI.
I notice that the capacitor voltage cannot change instantly unless there is an infinite pulse of a sort, so this guy cannot change instantly.
And so its voltage starts off being zero.
So the entire voltage here, KVL must be true no matter what.
They are absolutely fundamental principles from Maxwell's equations.
KVL must hold, which means that the entire voltage VI must appear across the inductor.
I put a big voltage across the inductor and its current begins to build up.
There you go.
A voltage across the inductor, its current begins to build up.
As its current begins to build up that current must flow through the capacitor, too.
And as current flows through a capacitor it is depositing charge into the capacitor.
As the capacitor begins to get charge deposited on it, its voltage begins to rise.
Let's see what happens here.
Its voltage keeps rising.
At some point, the voltage across the capacitor is equal to VI.
But then VI equals this VI here.
So when the two become VI, the inductor has zero volts across it.
So there is no longer a potential difference that is increasing the current in that direction.
At that point, at pi divided by 2, I have some current going into the inductor so there is no longer a pressure that is forcing more current through the inductor because this voltage reaches VI.
But remember capacitors like to sit around holding voltages.
Just remember that demo.
That rinky-dink capacitor sat there stubbornly holding its voltage.
And it had a huge spark towards the end.
It just sat there holding its voltage.
In the same manner, inductors love to sit around holding a current.
They will do whatever they can to keep the current going through them.
It has got the current going through.
And few forces on earth can change that.
And so therefore, even though the capacitor voltage is VI and the voltage drop across the inductor is zero, it still keeps supplying a current.
It has got the current.
It's got inertia.
It keeps going.
It is like a runaway train.
You may not be pushing the train from the back, but once it is running it has got kinetic energy and is going to run no matter what for a least some more time, even if you take away the force on the train.
So I have taken away the force on the punching more current through, but it has kinetic energy.
It has current flowing through it so it continues to supply a current.
Because it continues to supply the current the capacitor voltage keeps increasing.
This is a subtle insight which is absolutely spectacular that with zero volts across it, it still keeps pumping that current.
Capacitor voltage has gone up.
And guess what?
The voltage on this side is higher now but this guy is still pumping a current.
Man, I have been born to do this, you know, I shall pump a current.
However, because the voltage has now gone up here gradually the current begins to diminish.
So the capacitor is concerned.
You pump a current into me, my voltage goes up.
At some point, like a runaway train, it comes to a halt.
The current through the capacitor drains and now goes to zero and the capacitor voltage reaches 2VI.
So this is at 2VI now and this is at VI.
Now the situation is not in equilibrium.
At this point there is zero current through it, but guess what?
I have a VI pumping in this direction now.
I have the same VI punching in this direction.
So guess what?
Its current must now build up in this direction and its current begins to build up in that direction.
That begins to discharge the capacitor and the capacitor then goes on to a negative, or the current goes down to a maximum negative current, and this process continues.
What you are seeing here is energy.
It is sloshing back and forth between the two, and that is kind of a key.
I will just quickly put up a demo that you can watch as you are walking out.
With a step input, notice the green is the voltage across the capacitor and the orange is the current through the capacitor.
Before I begin today, I thought I would take the first five minutes and show you some fun stuff I have been hacking on for the past three years.
This has to do with 6.002 and circuits and all that stuff, but this is completely optional, this is for fun, this is to go build your intuition, this is to check your answers, whatever you want.
This is not a required part of the course.
Just for fun.
There is this URL out here that I put down here.
I have been hacking on this system for the past three years, and for the first time this year and very tentatively and gingerly introducing it to students.
The idea here is that it is a, that is kind of defocused.
Any chance of focusing that a little bit better?
The idea of this is that it is a Web-based interactive simulation package that I have pulled together.
And what you can do is you can pull up a bunch of circuits.
Notice that the URL is up here.
It is euryale.lcs.mit.edu/websim.
And there is the pointer to it.
So you have a bunch of fun things you can play with.
And we have gone through all of these things in lecture.
Let's pick the MOSFET amplifier.
You come to this page.
This is something you have seen in class.
And let's play with this little circuit.
And you see the mouse?
Good.
You can set up a bunch of parameters.
You can set up the MOSFET parameters VT and K. You can set up the value of R for your resistor, you can establish a bias voltage, and you can have an input voltage vIN.
So you can apply a bunch of input voltages.
You can apply a zero input, unit in pulse, unit step, sine wave, square waves.
Or this was the part that took me the longest to get right.
You can also input a bunch of music.
And so far I just have two clips, so you are going to get bored listening to them.
Good.
So you can also input music.
And what you can do is you can watch the waveforms, you can listen to the output and do a bunch of fun stuff.
One experiment I would love for you guys to try out.
Again, remember, this is completely optional.
Just for fun.
You can apply some input.
Step input, for example, to an RLC circuit and spend 30 seconds thinking about what should the output look like.
I divine that the output should look like this and then do this and see if what you thought was correct.
And it's fun to kind of play around with it.
Let me start with, just as an example, let's say I input classical music.
And let us say I would like to listen to the output here that is the voltage at the drain terminal of the MOSFET.
For listening it sets up a default timeframe to listen to, so you go ahead and do it.
This shows you the time domain waveform of a clip of the music and then you can listen to it.
Lot's of distortion, right?
As you can see, there is a bunch of distortion.
And that is as you expect because the peak-to-peak voltage is 1 volt, the bias is 2.5, and so this is clipping at the lower end, plus the MOSFET is nonlinear.
You can play around with a bunch of things and you can have a lot of fun.
And the reason I created this is that MIT is putting a bunch of its courses on the Web.
And one of the hottest things about courses like this is the lab component.
If you are beaming a course to, say, a Third World country or something, how do you get people to set up the massive lab infrastructure?
I know you hate your oscilloscopes, I know you hate your wires, I know you hate the clips, but the fact is you have them.
I know a lot of places those are way too expensive to pull together, which is why I have been creating this Web-based kind of interactive laboratory so that people can learn this stuff over the Web.
Let's go do another example very quickly.
Let's say you learned about, well, let's do RC circuits.
Here is the parallel RC circuit.
And you can set up capacitor values, resistor values, you can set up input.
Here, let me look at the time domain waveform for the voltage across the capacitor.
And this time around let me play a unit step.
And let's see what the output is going to look like.
You can think in your minds what should the output look like, and then you can go and plot it.
There you go.
That's what the output looks like.
So you can play around with it and have fun.
That's all the good news.
The bad news is that so far I just have one Pentium III machine behind us.
It is a Linux box, so don't all of you try it at once.
However, what I have also done, and that took me another six months of hacking in the small amount of time professors have to hack on stuff, I've hacked an incredibly elaborate cashing system so that once anyone in class tries out some combination of parameters it goes and squirrels away all the outputs.
If anybody else types in the same sets of parameters it will just get all the output and play it back to you.
So if enough of you play with over time, we may end up cashing all the important waveforms and music clips and all of that stuff.
I have allocated a few gigabytes of storage, so I am hoping that it may work.
Go forth.
Play with it.
And this is completely my fault, so if there are any bugs or anything simply email them to me.
This is the first time this is coming alive so bear with it.
Now let me switch back to the scheduled presentation for today.
All right, hope and pray that this works.
Yes.
Good.
I am going to do today's lecture using view graphs.
And the reason I am going to do that and not do my usual blackboard presentation which I way, way, way prefer to a view graph presentation.
The only reason I am going to do this for today, and maybe one more lecture, is that there is just a huge amount of math grunge in this lecture.
What I want to do is kind of blast through that, but you will have it all in the notes that you have, so that you don't waste time in class as you watch me stumbling over twiddles and tildes and all that stuff.
The key thing here is that the insight is actually very simple.
The beginning and the end are connected very tightly and very simple.
There is a bunch of math grunge in the middle that we are going to work through and, again, follows a complete old established pattern.
So, in that sense, there is really nothing dramatically new in there.
Let me spend the next five minutes reviewing for you how we got here, what have we covered so far and set up the presentation.
The first ten view graphs I am going to blast through and just tell you where we are in terms of LC and RLC circuits.
I began by showing you this little demo, two inverters, one driving.
I can model the inductance here with a little inductor, the capacitor of the gate here.
And recall that when I wanted to speed this up by introducing a 50 ohm smaller resistance, I got some really strange behavior.
Just to remind you, for Tuesday's lecture it would help if you quickly reviewed the appendix on complex algebra in the course notes.
Remember all the real and imaginary j and omega stuff?
It would be good to very quickly skim through that.
It is a couple of pages.
Remember this demo?
And the relevant circuit that is of interest to us is this one here.
It is the resistor, there is the inductor and there is a capacitor.
This is Page 3.
I am just going to blast through the first ten view graphs.
It is all old stuff.
Then we observed the following output.
We applied this input at VA and we got this output, a very slowly rising waveform because of the RC transient.
And because of that you saw a delay.
Notice that this delay was because of the slowly rising transient.
This waveform took some time to hit the threshold of the neighboring transistor.
So we say ah-ha, let's try to speed this sucker up by reducing the resistance in the collector of the first inverter.
And so I had this input.
Now, to my surprise, instead of seeing a nice little much higher and much faster transitioning circuit, well, I did see a much faster transitioning circuit but I got all this strange behavior on the output that I was interested in.
And because of that, if these excursions were low enough, I could actually trigger the output and get a whole bunch of false ones here because of these negative excursions which should not really be there.
That was kind of strange.
In the last lecture we said let's take this one step at a time.
Let's not jump into an RLC circuit.
Let's go step by step.
Let's start with an LC, understand the behavior.
We started off with an LC circuit of this sort, and using the node equation we showed that this was the equation that governed the behavior of the circuit.
And then we said that for a step input and for zero initial conditions, that is the zero state response, let's find out what the output, the voltage across the capacitor looks like.
And so we obtained the total solution to be this.
And there was a sinusoidal term in there.
And the omega nought which was one by square root of LC.
And this was the circuit.
And so for this step input notice that the output looked like this.
So far an input step I had an output that went like this.
Notice that it is indeed possible for the output voltage to actually go above the input value VI.
This is kind of non-intuitive but this can happen.
So this waveform jumps up and down.
But the steady state value, on average if you will, is VI.
On the other hand, it does have sinusoidal excursions and this kind of goes on because there is nothing to dissipate the energy inside that circuit.
By the way, the fact that the capacitor voltage shoots above the input voltage is actually a very important property.
We won't dwell on it in 6.002, but just squirrel that away in your brain somewhere.
I promise you that some time in your life you will have to create a little design somewhere that will need a higher voltage than your DC input.
And you can use this primitive fact to actually produce a DC voltage higher than you are given, and then use that somehow.
In fact, there is a whole research area of what are called DC to DC converters, voltage converters.
Let's say you have 1.5 volt battery, a AA battery, but let's say a circuit needs 1.8 volts.
The Pentium IIIs, for example, needed 1.8 volts.
And the strong arm is another chip that required 1.8 volts a few years ago, but the AA cell was 1.5 volts.
How do get 1.8 from 1.5?
Well, you have to step it up somehow.
And this basic principle where the voltage can jump up above the input is actually used, of course with additional circuitry, to kind of get higher voltages.
It is a really key point that you can squirrel away.
This was pretty much where we got to in the last lecture.
This starts off the material for today.
What we are going to do is take that same circuit, but instead we are going to put in this little resistor here.
This is what we set out to analyze.
And for details you can read the course notes Section 13.6.
The green curve here was the behavior of the LC circuit.
And what we are going to show today is that the moment we introduce R this sinusoid here gets damp.
It kind of loses energy.
And I am going to show you that the behavior is going to look like this.
By introducing R this guy doesn't keep oscillating forever.
Rather it begins to oscillate and then kind of loses energy and kind of gets tired and settles down at VI.
And remember the demo.
This is exactly what you saw in the demo.
You had a step input and you had this funny behavior.
And for the RLC that is exactly what it was.
So today's lecture will close the loop on what you saw in the demo and the weird behavior, and I am going to show you the mathematics foundations for that today.
Let's go ahead and analyze the RLC circuit.
I purposely created the entire presentation to follow as closely as possible both the discussion of the RC networks and the LC networks so that the math is all the same.
Exactly the same steps in the mathematics are in the exposition of the analysis.
What's different are the results because the circuit is different.
So don't get bogged down or whatever in the mathematics.
Just remember it is the same set of steps that you are going to be applying.
We start by writing down the element rules for our elements.
Nothing new here.
For the inductor V is Ldi/dt.
The integral form which is simply 1/L integral vLdt=i.
We saw this the last time.
And for the capacitor, the current through the capacitor is simply Cdv/dt.
Those are the two element rules for the capacitor and inductor.
The element rule for the resistor, of course, is V=iR.
You know that.
And for the voltage source we know that, too, the voltage is a constant.
Just follow the same established pattern.
By the way, just so you are aware, I have booby trapped the presentation a little bit to prevent you from falling asleep.
You see the dash lines here?
Whenever you see a dash line, that stuff needs to be copied down.
Don't trip over that.
Don't say I didn't warn you.
We start by using the usual node method.
And I have two nodes in this case.
Unlike the LC circuits, I have two unknown nodes.
One is this node here with the node voltage vA and the second node is the node with voltage vT. Let me start with vA and write the node equation for that.
It is simply 1/L, the node equation for this is the current going in this direction with is vI-vA integral and that equals the current going this way which is vA-v/R, node equation.
I then write the node equation for the node v, for this node here, and that is simply (vA-v)/R=Cdvdt.
And that is what I have here, two node equations.
Let me summarize the results for you and then show you a view graph where I grind through the math as to how I got the result.
Here is the result I am going to get.
If I take these two node equations and I massage some of the mathematics, I am going to get this result.
And I will show you that in a second.
By grinding through some math and solving these two equations and expressing this in terms of v, I get a second order differential equation, d^2v blah, blah, blah.
Notice that this is different from the LC in this term.
Every step of the way you can check to see if I am lying or I am correct.
I will indulge you, indulge myself rather with a little story here.
Richard Fineman was a known smart guy.
And one of the reasons that he was that was in the middle of talks he was known to get up and ask some of the darndest, hardest questions and say ah-ha, you have a bug in this proof here or a bug in this equation that is not right.
And usually he would be correct.
So his trick in doing this and which is one reason how he became a known smart guy.
What he would do is, as the speaker went on talking he would kind of follow along and think of a simple initial primitive case.
In this case, I have an RLC circuit.
So think of a simpler case of this.
A simpler case of this is R=0.
Whenever you set R to be zero, you should get exactly what we got in the last lecture, correct?
That is what Fineman would do.
He would boil this down to a simpler case, make some assumptions and just follow along.
And whenever he found a discrepancy between the math here and his simple case he would say oh, there is a bug there.
If you want you can catch me that way.
Here, what Fineman would do is replace R being zero, and notice then this equation here is exactly what we got the last time with R being zero.
Just remember that Fineman trick.
This is the equation we get, the second-order differential equation with an R term in there.
And let me just grind through the math and show you how I got this from this.
So the two node equations again.
And what I do is I start by taking these two equations and differentiating this with respect to t and this is what I get.
And, at the same time, I have replaced (vA-v)/R here by this term.
I replace this with this and differentiate.
Then I simply divide the whole thing by C. Then I take this expression here and write down vA is equal to this stuff here.
Next I am going to substitute this back for vA and eliminate vA.
So I take this vA, stick the sucker in here, and thereby eliminate vA and get this.
And then I simplify it and here is what I get.
That is what I get.
I just grind through the two equations and get that result.
So like a stuck record I will repeat our mantra here, which is here is how we solve the equations that we run across in this course, the same three steps.
Find the particular solution.
Find the homogenous solution.
Find the total solution and then find the constants using the initial conditions.
Same steps.
You could recite this in your sleep.
And the homogenous solution is obtained using a further four steps.
Let's just go through and apply this method to our equation and get the results.
vP is a particular solution and vH is the homogenous solution.
With a particular solution, oh.
Before I go on to do that, let me set up my inputs and my state variables.
My input is going to be a step.
Remember, I am trying to take you to the point where the demo left off.
The demo had a step input, so I am going to use a step input rising to vI.
And I am going to with the initial conditions being all zeros.
So the capacitor voltage is zero, inductor current, another state variable is also zero, and therefore this is also fondly called the ZSR or the zero state response because there is only an input but zero state.
Again, remember the dashed lines here.
Don't say I didn't warn you.
Let's start with a particular solution.
This is as simple as it gets.
I simply write down the particular equation and stick my specific input.
And remember the solution to the particular equation is any old solution, it doesn't have to be a general solution, any old solution that satisfies it.
And I am going to find a simple solution here.
And V particular is a constant VI.
It works.
Because remember this has been working all along.
And I am going to keep pushing this and see if this works until the end of the course.
Guess what?
It will.
So this is a solution.
I'm done.
That is my particular solution.
Simple.
Second, I go and do my homogenous solution.
And the homogenous equation, remember, is the same old differential equation with the drive set to zero.
Remember that sometimes this equation with the drive set to zero is the entire equation you have to deal with in situations where you have zero input, for example.
Or in other situations in which you have an impulse at the input.
And the impulse simply sets up the initial conditions like a charge in the capacitor or something like that.
So we are going to blast through this four-step method.
The method simply says that four steps, I am going to assume a solution of the form Ae^st.
And if you think you've seen that before, yes, you have seen it many times before.
And you will see it again, again and again.
And we need to find A and s. We want to form the characteristic equation, find the roots of the equation and then write down the general solution to the homogenous equation as this.
Same old same old.
Let me just walk through the steps here.
Step A, assume a solution to the form Ae^st.
And so I substitute Ae^st as my tentative solution to the equation.
Again, let me remind you that the differential equations that we solve here are really easy because the way you solve them is you begin by assuming you know the solution and stick it in and find out what makes it work.
I am going to stick Ae^st into this differential equation, and A comes out here.
Differentiate this d squared, I get s squared down here, A s here and this simply gets stuck down here with the 1/LC coefficient.
The next step I begin eliminating what I can, so I eliminate the A's, then eliminate the e^st's, and I end up with this equation here.
I end up with this equation.
This is my characteristic equation.
It is an equation in s. Do people remember the characteristic equation we got for the LC circuit?
Remember the Fineman trick?
That's right, LC.
S^2+1/LC=0.
This thing wasn't there.
All you do is simply follow the R. Just follow the R. Just imagine this is a dollar sign and kind of follow it.
And you will see what the differences are between the LC and the RLC.
So this is the characteristic equation.
What I am going to do, iss much as I wrote the characteristic equation for the LC circuit, by substituting omega nought squared for 1/LC.
Let me do the same thing here but introduce something for R and L as well.
What I will do is let me give you this canonic form.
The very first second-order equation I learned about when I was a kid was this one, S^2+2AS+B^2 or something like that.
Let me write it in that form where I get 2 alpha s plus omega nought squared.
Again, remember the alpha comes about because of R. So omega nought squared is 1/LC and alpha is RL/2.
Omega nought squared is 1/LC and R/L is equal to two alpha.
I am just writing this in a simpler form so that from now on going forward I am just going to deal with alphas and omega noughts.
Once I get to this characteristic equation, after that I can give you one generic way of solving it.
And depending on the kind of circuit you have, a series RLC, which is what we have, or a parallel RLC we will simply get different coefficients for the alpha term.
This is going to stay the same but this term will look different, alpha is going to look different.
There is a real pattern here.
And what I am doing is simply focusing on what is important, what the differences are between the pattern.
You learned the LC situation and the RLC situation.
Given this I can now write down, I am just simply replacing this as my characteristic equation in dealing with alphas and omegas.
I will give you a physical significance of alpha in a little bit.
Do you remember the physical significance of omega nought?
That was the oscillation frequency.
In other words, given an inductor and capacitor, you put some charge on the capacitor and you watch it, it will oscillate.
And its oscillation frequency will be one by a square root of LC.
The magnitude of the initial conditions will determine how high are the oscillations or what the phase is in terms of when it starts, but the frequency is going to be the same.
Step three, to solve the homogenous equation, is find the roots of the equation, s1 and s2, and here are my roots.
Good old roots for a second-order, little s squared equation here.
Finally, given that I have the roots, I can write down the general homogenous solution.
So general solution is simply A1e^s1t, A2e^s2t.
That's it.
That's the solution.
This looks big and corny, but we are going to make some simplifications as we go along and show that it ends up boiling down to something cos omega t. The math is kind of involved but we get down to something very simple, a cosine.
Hold this general solution.
From that, as a step three of the differential equation solution, I write the total solution down.
And my total solution is the sum of the particular and the homogenous, so therefore I get this.
VI was my particular and this term here is my homogenous solution.
Now, if I wasn't doing circuits and simply trying to solve this mathematically here is what I would do.
I would find the unknown from the initial conditions, so I know that v(0)=0.
And so therefore if I substitute zero for V(0) I get this.
If I substitute zero here, t is 0, t is 0, so I simply get V1+A1+A2.
And let me just blast through because I am going to redo this differently.
i=Cdv/dt.
And so that's what I get.
I substitute zero and this is what I would get.
I hurried through this.
Don't worry.
I'm going to do it again.
If you just do it mathematically, you can solve this equation here and these two simultaneous equations in a1 and a2 and get the coefficients and you are done.
But it doesn't give us a whole lot of insight into the behavior of these terms here.
What I am going to do for now is kind of ignore that.
Ignore I did that and instead try to go down a path that is a little bit more intuitive.
Let's stare at this expression we got for the total solution.
That is the expression we got.
All I did is, I had alpha in there, I simply pulled out the alpha outside.
So this is my total solution, V1-A1e^(-alpha t) something else and something else.
Three cases to consider depending on the relative values of alpha and omega nought.
If alpha is greater than omega nought then I get a real quantity here.
The square root of a positive number, I get a real number, and that number will add up to the minus alpha and I am going to get a solution that will look like, oh, I'm sorry.
Let me just do it a little differently.
There are three situations here.
One is alpha greater than omega nought.
Alpha equal to omega nought.
Alpha less than omega nought.
Alpha is greater, alpha is less, alpha is equal to this term inside the square root sign.
For reasons you will understand shortly, we call this "overdamped" case, the "underdamped" case and the "critically damped" case.
When alpha is greater than omega nought this term gives me a real number, and I get something as simple as this.
Remember, for the series RLC circuit, alpha was R/2L.
So if R is big, in other words, if in my RLC circuit R is huge then I am going to get this situation.
My output voltage on the capacitor is going to look like this, the sum of two exponentials.
And if I were to plot it very quickly for you, for a VI step, V would look like this.
So v would simply look like this because it is the sum of a couple of exponentials.
All right.
Now, alpha is positive here.
Remember alpha1 and alpha2 are both positive.
These two added up, because of this constant VI, give rise to something that increases in the following manner.
Let's look at the situation where alpha is less than omega nought, where the term inside the square root sign is negative.
What I can do is pull the negative sign out and express it this way.
What I am going to do is since alpha is less than omega nought, I am going to reverse these two and pull out square root of minus one to the outside.
This is what I get.
I am just playing around with this so that whatever is under the square root sign ends up giving me a positive real number.
So I pull the j outside and this is what I get.
Now, let me blast through a bunch of math and end up with something very, very simple for this underdamped case.
Let me define a few other terms.
I am going to call omega nought minus alpha squared the square root of that.
I am going to call it omega d. And here is what I get.
So I have defined three things for you now, alpha, omega nought and omega d. And I get this equation in terms of alpha and omega d. And then, remember from your good-old Euler relationship?
e to the j omega d is simply cosine plus a j sine.
I am just going to blast through a bunch of math rather quickly.
Once I replace this in terms of a cosine and sine, cosine and a j sine and then collect all the coefficients together, I get an equation of the form VI plus some constant e to the minus alpha t, cosine, the sum of the constant e to the minus alpha t, sine.
Remember the sines and cosines are coming out, but because of my R I am getting this funny alpha here, e to the minus alpha here.
So I am getting sums of sine and cosine.
And K1 and K2 are some constants which I will need to determine for my initial conditions.
I am going to continue on with this and keep on simplifying it because, as I promised you, I want to get to something that is just a cosine.
I want to go down this path.
I am not going to cover this case, the critically damped case.
And I will touch upon it later but not dwell on it.
Let me continue down the path of the underdamped case, and this is what we have.
Continuing with the math, let's start with the initial conditions, v nought equals zero, and that gives me K1 is simply -VI.
So at v(0)=0 t is zero, so this terms goes away, the cosine becomes a 1, e^(alpha t) goes away, and K1=-VI.
Then I know that i(0) and i is simply Cdv/dt.
And I get this nasty expression.
I substitute t=0 and I get something that looks like this.
I know what K1 is, and so therefore K2 is simply -V1alpha divided by omega nought.
I have taken this expression where the unknowns K1 and K2 are to be found.
I set the initial conditions down at t=0 and I get K1 and K2 as follows, which gives me the following solution.
This is the solution I get where I do not have any unknowns anymore.
Remember that omega d and alpha are directly related to circuit parameters.
Alpha was R/2L and omega d was square root of alpha squared minus omega nought squared.
** omega d = sqrt(alpha^2 - omega_0^2) ** And omega nought squared was 1 by square root of LC.
So I know it all now.
I still have sines and cosines here, so I am going to simplify this a little further.
Oh, before I go on to do that, let's do the Fineman trick again and notice if I am still true to the LC circuit I did the last time.
Remember when R goes to zero alpha goes to zero.
Because alpha is R divided by 2L.
If alpha was zero what happens?
If alpha was zero, this guy goes to one, this whole term goes to zero and omega dt now ends up becoming omega nought, and I get this term here.
I get VI-VIcosine(omega t), which is exactly what I expected in my equation.
This is the same as the LC case that I got.
Let's go back to this situation and simply if further.
If you look at Appendix B.7 in your course notes, Appendix B.7 is a quick tutorial on trig.
And in that trig tutorial you will see that, and you have probably seen this before, too, multiple times, the scaled sum of sines are also sines.
This is an incredibly cool fact of sinusoids.
If you take two sinusoids of the same frequency and you scale them up in any which way and add them up you also end up with a sinusoid.
It is hard to believe but it is true.
It is an incredible property of sinusoids.
Take any two sinusoids, scale them in any way you like, same frequency, add them up, you will get a sinusoid.
What that is saying is that, look, here is a sinusoid, here is a sinusoidal function, and I am scaling them up in some manner.
So I should be able to add them up and be able to express that as single sine.
And to be sure you can, look at the Appendix, and there is an expression for a1 sinX plus a2 cosX is equal to a cosine of blah, blah, blah.
This is what you get.
No magic here.
Just math.
From here I directly get this.
And look at what I have.
It is absolutely unbelievable.
v(t) is simply VI, there is a constant here, this an e to the minus alpha term and there is a cosine.
Again, to pull the Fineman trick, if this alpha were to go to zero here then you would end up with the expression you had for the LC situation.
Let's stare at this a little while longer.
There is a constant plus a minus, a cosine term, so there is a sinusoid at the output, and there is an e to the minus alpha which ends up giving you the decay you have seen before.
In other words, to a step input, the LC circuit would give you a sinusoid.
That is what the LC circuit would do if alpha was zero.
But because of this alpha term here, e to the minus alpha t, that gives rise to a damping effect, so this causes this thing to become smaller and smaller as time goes by until this term goes to zero at t equals infinity.
This guy damps down and so therefore you end up getting the curve that you saw like this.
Twenty minutes of juggling math solving a second-order differential equation, but what ends up is the same sinusoid but it is damped in the following manner such that the frequency, rather the amplitude keeps decaying until it starts off at zero and then settles down at vI.
This is exactly what you saw in the demo that we showed you earlier.
The critically damped case, I am not going to do it here.
I am going to point you to the following insight.
The underdamped case looked like this.
It was a sinusoid that kind of decayed out.
That is the underdamped case.
And then I showed you the overdamped case.
The overdamped case looked like this.
And, as you might expect, the critically damped case is kind of in the middle and looks like this.
So the overdamped case would look like this, underdamped like this, and the critically damped case kind of goes up and kind of settles down almost immediately.
This is when alpha equals omega nought.
I won't do that case here, but I will simply point you to Section 13.2.3.
Just to tie things together, recall this demo here that we showed you in class yesterday.
This is exactly the kind of form of the sinusoid you saw because of that input step.
If you want to see a complete analysis of inverter pairs and look at the delays and so on because of that, you can look at Page 170 and example 898.
In the next five or six minutes, what I would like to do is stare at the RLC circuit.
And much like I showed you some intuitive methods to get the RC response, what we are going to do is do the same thing for the RLC.
In the RLC situation, much like the RC situation, experts don't go around writing 15 pages of differential equations and solving them each time they see an RLC circuit.
They stare at it and boom, the response pops out, the sketch pops out.
This one is going to be another one like our Bend it Like Beckham series here.
And this one is in honor of Leslie Kolodziejski.
And I call it "Konquer it like Kolodziejski".
Again, as I said, experts don't go around solving long differential equations and spending ten pages of notes trying to get a sinusoid.
They look at a circuit and sketch response.
I am going to show you how to do that, too.
And what you can do is, to practice, go to Websim and try out various combinations of inputs and initial conditions and sketch it, time yourself, give yourself 30 seconds or a minute if you like, and sketch it and check it against the Websim response.
If it doesn't match either you are wrong or there is a bug in Websim.
What I am going to do is, the response to the critically damped and underdamped case was very easy to sketch out.
You started with an initial condition, you settled at VI and just kind of drew it like that.
The interesting case is the underdamped case, and that is what I am going to dwell on.
Before we go on and I show you the intuitive method, as a first step I would like to build some intuition.
Let's stare at this response here and try to understand what is going on.
This is the response that we saw.
And this fact that you see an oscillation happening is also called "ringing".
You say that your circuit is ringing.
All right.
You see some interesting facts.
You see that frequency of the ringing is given by omega d. This cosine omega d, so that is the frequency omega d. So the time is 2 pi divided by omega d. The oscillation frequency is omega d, but omega d is simply omega nought squared minus alpha squared.
Once you have a big value of R alpha becomes very small and omega d is very commonly equal to, very close to omega nought.
So omega d and omega nought very commonly are very close together.
And when that happens this frequency is directly omega nought.
Alpha governs how quickly your sinusoid decays.
e to the alpha t here is the envelope that governs how quickly my sinusoid decays.
And notice that each of these terms, alpha and omega nought, comes directly from my characteristic equation.
Which means that once you get your characteristic equation you really don't have to do much else.
And up until now you still have to write the differential equation to get the characteristic equation, so you still have to do some differential equation stuff, but in two lectures I am going to show you a way that you can even write down the characteristic equation by inspection.
Look at your circuit and boom, in 15 seconds or less write down the characteristic equation.
It is absolutely unbelievable.
What are the other factors that are interesting here?
Of course I need to find out initial values.
I start off at zero.
This is my capacitor voltage.
If I don't have an infinite spike or an impulse my capacitor voltage tries to stay where it is and starts off at zero.
And the final value is given by VI, the capacitor is a long-term open so therefore VI appears across the capacitor.
In the long-term my final value is going to be VI.
There is one other interesting parameter, which I will simply define today but dwell on about a week from today, and that is called the Q.
Some of you may have heard the term oh, that's a high Q circuit.
Q is an indication of how ringy the circuit is.
And Q is defined as omega nought by 2 alpha.
It is called the "quality factor".
And it turns out that Q is approximately the number of cycles of ringing.
So if you have a high Q you ring for a long time and if you have a low Q you ring for a very short time.
That is called the quality factor defined by omega nought by 2 alpha.
Notice that Q, omega nought, alpha, omega d, all of these come from the terms in the characteristic equation.
We will spend more time on Q later.
With this insight here is how I can go about very quickly sketching out the form of the response.
Here is my circuit.
I want to sketch the form of the response for a step input at vI.
Zero to vI step input here, I want to find out what happens at this point.
This is described to you in a lot more detail in Section 13.8 in your course notes.
Let's go through the steps.
Let's do the really simple situation first.
Let's also assume for fun that you are given that v(0) starts out being some positive value.
Some v(0) which is a positive number.
And, to make it harder on ourselves, let's say i(0) starts out being some negative number.
So i(0) is some negative current.
The first thing I know is v(0), the capacitor voltage starts out here, which can change suddenly.
And I also know that in the long-term this is an open circuit.
So that this voltage vI will appear directly across the capacitor in the long-term.
So I get starting out at v(0), ending at vI, I am also half the way there.
I know the initial and ending point of the curve.
And then I know that somewhere in here there must be some funny gyrations here, because remember I am dealing with the underdamped case.
And you can determine that from alpha and omega nought.
If alpha is less than omega nought, you know that you are in the underdamped case and this is what you get.
Let's compute and write the characteristic equation down.
A week from today you will write it by inspection, but for now you will do it by writing down a differential equation.
And from the characteristic equation you will get omega d, you will get alpha, omega nought and Q.
So omega d gives you the frequency of oscillations.
My frequency of oscillation is now known.
From Q I know how long it rings, because I know it rings for about Q cycles.
I know that ringing stops approximately here.
And then I know that between that the start and end point my curve kind of looks like this, something like this.
Right there we are 95% of the way there.
The only question is I do not know if it goes like this or it goes like this.
I am not quite sure yet if it starts off going high or starts off going low.
Not quite clear.
I also do not know what the maximum amplitude is.
It turns out this is rather complicated to determine so we won't deal with that.
Just simply so you can draw a rough sketch.
The questions is which way does it start?
I could leave it for you to think about.
Yeah, let me do that.
It is given on this page so don't look at it.
Think about it, and think about how you can determine whether it goes up or down.
It turns out that in this case it is going to down and then ring.
See if you can figure it out for yourselves and then we will talk about it next week.
OK. Good morning.
Let's get going.
As always, I will start with a review.
And today we embark on another major milestone in our analysis of lumped circuits.
And it is called the "Sinusoidal Steady-state".
Again, I believe this will be the second and the last lecture for which I will be using view graphs.
And the idea here is that, just like in the last lecture, there is a bunch of mathematical grunge that needs to be gone through.
And I want to show you a sequence in a little chart today that talks about the effort level in doing some problems based on time domain differential equations, as in the last lecture, something slightly different today and something much better on Thursday.
And so, in some sense, Thursday's lecture and today's lecture involve talking about the foundations of the behavior of certain types of circuits.
And it is good for you to have this foundation as background, but when you actually go to solve circuits you don't quite use these methods.
You use much easier techniques, which I will talk about next Thursday.
Let's start with a quick review, and then we will go into sinusoidal steady state.
The last lecture we talked about this circuit and we did the same two lectures ago on Tuesday.
We had one inverter driving another inverter.
And we said that the wire over ground had some inductance.
CGS is the capacitor of the gate and R is the resistance at the drain of the first inverter.
And if you look at this circuit, that circuit formed an RLC pattern.
And what we did was we said let's drive this with a one to zero transition at the input.
And the one to zero transition at the input would cause this transistor to switch off, and this node would then go from a very low value to a high value.
So it as if a 5 volt step was applied at this input.
We also saw that using time domain differential equations that by applying a step input here the output looked like this.
The output would show some oscillatory behavior when we have a capacitor and inductor.
I also gave you some insight as to why it oscillates like this.
And you also heard in recitation that the reason for this oscillation was because of these two storage elements.
Each of these storage elements tries to hold onto its state variable.
For example, the capacitor tries to maintain its voltage while the inductor tries to maintain its current.
And, much like a pendulum which oscillates back and forth, it swaps potential energy versus kinetic energy down here and swings back and forth.
In the same way, in an LC circuit like this, energy swaps back and forth between a potential energy and a kinetic energy equivalent, which swaps back and forth between energy stored in the inductor and energy stored in the capacitor and sloshes back and forth.
And because of this resistor the energy eventually dissipates and you end up getting a final value which corresponds to the 5 volts appearing here.
And why is that?
That is because remember the capacitor is a long-term open for DC.
It is a DC voltage.
After a long time this capacitor looks like an open circuit and the inductor looks like a complete short circuit, an ideal inductor as a complete short circuit for DC.
And so therefore in the long-term it is as if this guy is a short, this guy is an open, so 5 volts simply appears here.
And this is the transient behavior.
Then we just switch the first transistor off.
In the last lecture, I left off with intuitive analysis.
Let me quickly cover that and redo the intuitive analysis for you.
I left it the last time by having you think about whether the transient response would begin by going down or begin by going up.
And I will cover that today.
This was the circuit that we analyzed.
A VI input with a step and an RLC out here.
And we were plotting the capacitor voltage.
And intuitively we can plot this in the following way.
I have also marked for you the section number in the course notes which has a discussion of this intuitive analysis, (Section 13.8).
Let's do the easy stuff first.
Notice that the capacitor wants to hold its voltage.
And so given that we don't have any infinite impulse here, I am going to start out with the capacitor voltage being where it is.
And the initial conditions are given to you.
You are given that the capacitor voltage starts out being positive at v zero and the current starts out being negative at time zero.
So I am telling you that there is a voltage v across the capacitor at time t=0 and there is a current that is flowing.
Since i is negative there is a current initially that is flowing in the opposite direction to this arrow here.
The i zero is negative.
In light of that, I can start plotting my curve here by intuition.
I start by saying at time t=0 I am told that the initial voltage of the capacitor is at zero.
This is about as simple as it gets.
Completely intuitive.
I also know that after a long time, can someone tell me after a long time what the voltage will be at the end of the capacitor?
You should be able to get his by observation?
VI.
And why is it VI?
It is vI because this is a DC value VI.
And after a long time this guy behaves like an open circuit to DC.
This guy behaves like a short circuit to DC.
So since this is an open circuit, VI will appear here after a long time.
And so therefore, after a long time, the capacitor voltage is going to be at VI.
And I just showed you that.
There you go.
You already have the two endpoints of your curve completely by observation, intuition.
No DEs, no nothing.
Just by staring at it and understanding the fundamentals of how simple primitive circuit elements work.
Absolutely simple stuff.
So you've nailed the two ends now and you cannot go wrong with the stuff in the middle.
Let's see.
As a next step what I do is I need to understand what the dynamics of the circuit looks like here.
What I do is I develop the characteristic equation.
And initially you will write the differential equation and then substitute e^st and get this characteristic equation.
But you could also remember it as a pattern.
For a series RLC circuit you always get an equation of this form, always.
If this were R, L and C. And whether you are looking at L up here or C up here, as long as you're looking at the capacitor voltage, the capacitor voltage is going to behave the same.
And for this circuit the characteristic equation remains the same as well for a series RLC.
It is exactly this.
And I write the standard canonic form as s squared plus two alpha s + omega nought squared.
And omega nought is simply one by square root of LC and alpha is simply R divided by L and I have two in the denominator as well.
And then I get omega d which is my damped frequency given by omega nought squared minus alpha squared.
And Q is simply called the quality factor.
And we will learn about Q in a lot more detail in about a couple of lectures from today.
That is given where omega nought divided by two alpha.
These parameters, alpha, omega nought, Q and omega d pretty much characterize everything else that I need to know about the circuit.
First of all, omega d is the frequency of oscillation.
And so since omega d is a frequency of oscillation then I know that the time period of oscillation is two pi by omega d. Omega is in radians.
Notice that for typical values of circuits like this when R is pretty small, alpha squared is going to be very small.
It's a common case for underdamped circuit that omega d will more or less be equal to omega nought.
Commonly that is going to be the case.
This frequency is governed by LC.
And if R is large it is governed by this omega d here.
So I have the frequency of oscillation.
I also know that Q is related to the frequency of oscillation divided by alpha.
It is a ratio of the frequency divided by how badly I am being damped.
So it is a fight between the frequency of oscillation and now heavily I damp.
And the ratio of that is an indication of how many cycles I ring.
So Q tells me that the ringing stops approximately after Q cycles.
These four values, omega d, Q, alpha and omega nought are telling me more and more now.
So I have got these two factors.
So I know now, based on omega d and Q, that it is going to look something like this.
Some ringing here and then I stop at this point.
The last thing that is left to do here for me for now is to figure out whether I start out going down or I start out going up.
I start out going down or I start out going up?
I don't know that yet.
And I stopped at this point in the last lecture and let you think about how you can stare at the circuit and intuitively figure out whether this goes down or that goes up.
So here is the insight.
Let's stare at this for a minute purely through intuition.
The capacitor has a voltage v across it, right?
And that is because I am telling you that it has an initial voltage v. Now, I want to find out at prime t equals zero plus, in which direction does a capacitor voltage go?
Increase or decrease?
What do I do?
Let me take a look at the inductor current.
I am told that the inductor current is negative which means I am told that the inductor current is going in this direction initially.
The inductor current is pushing in this direction.
Now, remember, just as the capacitor is one stubborn nut trying to hold its voltage, the inductor is as stubborn.
It is trying to hold its current.
It is trying to maintain its current.
And its initial current i(0) is in this direction.
Capacitor has a voltage here, that is v(0), and the inductor is yanking the current in that direction.
So what should happen to the capacitor voltage initially?
If I am at v(0) and someone is yanking current out, at least initially in that direction, what should the initial dynamics of the capacitor voltage look like?
Pardon?
It should drop, which means that if the initial current is being pulled in that direction the capacitor voltage must droop to begin with.
Completely through intuition.
No math here.
This means that i(0) is negative.
It is as if i(0) is being pulled out in this manner, so the capacitor voltage must drop to begin life.
Therefore, the dynamics look somewhat like this.
Notice that this is very reminiscent of the ringing that we saw at the gate node of the second inverter.
Let's stop here in terms of time domain analysis of RLC, and today let's take another big step forward.
Today marks another transition in life here.
This is actually a huge transition so I want to just pause and take like ten seconds of a breather just to clearly demarcate the fact that we have a huge transition coming up.
The key to this transition is that I want to look at today the steady-state response of networks to a sinusoidal drive.
We are going to do two things differently starting today on this new journey of ours.
In the past, we looked at time domain behavior of circuits.
For RLC, for example, we looked at the transient response.
And what happened the instant I turn something on?
The transient response.
Today we are going to look at a steady-state response.
Steady-state response says that if I wait long enough, for whatever the circuit wants to do in the beginning of life to die out.
If I wait long enough, how is the circuit going to behave after a long time?
I will tell you why that is important in a second.
I look at the steady-state behavior.
Second, I am going to look today at sinusoidal drive.
Two things that are different from, say for example, what I covered in the past ten minutes.
In the past ten minutes I covered two things which were different.
One is that I looked at the transient response and then steady-state.
And remember for a DC input, for a DC voltage the steady-state was a DC voltage across the capacitor, correct?
So the steady-state was pretty boring, it was steady-state DC.
But what we are going to do today is instead of having DC inputs or step inputs which settle to a DC value after some time, we are going to drive a circuit through the sinusoidal input.
So you may ask me, Agarwal, are you nuts?
Why do you want to drive something with a sinusoidal input?
Why not just steps in DC and so on?
That was painful enough.
Why sinusoidal?
Why not do triangular or why not do some other exponentially decaying stuff or something really cool like a whacko music signal?
What is special about sinusoidal stuff?
The key thing to realize is that, well, let me ask you a question first.
How many people here know about Fourier series?
Good.
It looks like some of you have taken the prerequisites.
Good.
Need I say more as to why this is important?
Just that question should give you the answer, right?
You've learned about Fourier series.
And when you learned about Fourier series you were wondering why on earth are we learning about Fourier series?
Who cares that you can represent the periodic signals as a summation of a series of sine waves?
Why is that interesting?
Why are you telling me that I can take a square wave and represent that as a summation of periodic square waves and represent that as a summation of sines?
Who cares that I can take a sequence of pulses with a fixed period and represent that as a sum of sines?
Who cares that I can take a triangular wave and represent that as a sum of sines?
I am not sure what answer your math professors gave you when they taught you Fourier series.
But in math they are purists.
They don't care about applications.
The answer could well have been because it is aesthetically pleasing.
I mean isn't it cool that you can represent a sequence of pulses as a sum of sines?
That is good enough for mathematicians.
But I am an engineer.
If it I cannot see how it helps humanity in the short-term then I probably don't care too much about it.
Let me give you some practical significance of this.
So it turns out that, since we know that we can represent periodic signals with sums of sines.
What this means is that if I can figure out the behavior of networks to a sinusoidal input, if I can understand how to analyze a network for a sinusoidal input that means that if the circuit is linear I can then compute the response of the circuit to any periodic waveform.
Here is the argument.
I can represent any periodic waveform as a sum of sines.
The Fourier series, remember?
If I just figure out the response of my network for a sine wave, then if this is a linear network, I can compute the response to any sequence of scaled sum of sines.
A some sine, B sine two, omega whatever, C sine something or the other.
I can simply take the response of the one sine.
And from there I can go ahead, and knowing the response of a sine wave I can compute the response to a sum of sines.
That is pretty cool.
Therefore, doing it for sinusoidal drives is really important.
Why steady-state now?
Hopefully, I have convinced you that looking at the response of circuits to sinusoidal drive is important and interesting because we can long ways from there.
What about steady-state?
Well, it turns out that, and I am going to show you, when you listen to music, you have an amplifier and listen to music, what you are observing by and large is the steady-state behavior of the amplifier.
You are listening to something over many seconds or many hours.
And the transients used for most of our common circuits, the transients die out pretty quickly.
And so the transients are quite complicated and they die out quickly.
We say we are engineers.
Let's focus on what is practically important.
And we focus on the steady-state behavior as a large part of our analysis and just completely ignore the transient response when we care about the response to sinusoidal input.
The transient response will die away, and I will show that mathematically to you in a few seconds.
And let's focus on the steady-state because that what I am going to hear most of the time.
I am going to listen to an average over long periods of time.
That's the motivation behind this.
And let me put this in perspective for you.
By now this should bring memories to your mind.
This is the playground that we are in.
This is the lumped circuit playground here.
Remember we came from the playground of nature to the playground of EECS where we made the big leap from Maxwell's equations to lumped circuits, that's lumped circuit abstraction.
And within there we spent a large part of the last couple of months doing linear circuits.
We also analyzed nonlinear circuits.
Remember the amplifier circuit of the MOSFET large signal analysis was nonlinear?
Well, there is linear and nonlinear.
Within linear we also showed that if you take a digital circuit, at least as we understood them, and draw the subcircuit for a given set of switch settings, if I set the switches in a given way what I was left with was another linear circuit for a given value of all the switch settings.
My small signal analysis was also linear.
If I focused on small signals I also had linear analysis.
Today what we are going to do is this.
We are going to articulate a different part of the playground.
This was a big linear playground.
We've done this.
We've done this.
We are going to explore this territory.
This is that territory of the playground in which we have sinusoidal inputs to circuits.
Furthermore, we are going to look at a subcircuit of that region which is steady-state.
We will look at sinusoidal input and in the steady-state.
So that is going to be our focus for the next two or three lectures just to give you a perspective of where we are.
To motivate this, what I would like to do is consider your amplifier.
This is our friend the amplifier circuit, this part here.
And remember, even though this is an amplifier, I am using a MOSFET here.
And a MOSFET, as you know, has this gate capacitance CGS.
I am explicitly drawing it out for you here.
And I am going to drive this with a bias voltage plus some small signal vI, the usual template for amplifiers.
And there is some Thevenin resistance attached to that source.
I am going to model my source as a bias voltage, a small signal plus some source resistance.
And I want to apply a sine wave here and I am going to look at what this looks like.
You may think, look, this is a linear amplifier.
And so if I apply a sine wave here I should see some response here, and that should be the same amplitude if I feed the same amplitude here over any frequency.
But let's see what happens.
Keep a look at, switch over to that view graph while I show you a little demonstration here.
What you see here are three sine waves, a yellow sine wave which is the input here, you see a green sine wave which is the input vC at the gate of the MOSFET, and then you see the blue which is the output here.
For now simply focus on the yellow and the blue.
The yellow is the input and the blue is the output.
So I apply some input and I get an output that looks more or less some linear function of this input here.
It is a small signal.
What I am going to do is I am going to change the frequency of the input.
Remember, I want to look at the response of the circuit to a sinusoid.
I am feeding a sinusoid here.
I look at the response.
I am going to change the frequency.
That is a big shift that we are making in that the curve that we drew in the last lecture had to do with varying time.
Now I am going to focus on sinusoids and vary their frequency.
I am going to change the frequency.
Stare at the blue curve as I increase the frequency and just think of what you might expect.
Based on the knowledge you have so far you would expect that look, as I change the frequency, the frequency should change but I should see the same amplitude.
OK but take a look.
Let me increase the frequency of the input.
What do you see at the output?
I am increasing the frequency.
What do you see happening there?
If you don't see anything changing there you will need to see an optometrist.
What do we see here?
As I changed the frequency, as I increased the frequency what happened to the blue?
The blue kept decreasing in amplitude.
And you are saying whoa, what is happening here?
We don't have the tools to deal with this.
I expected that when I changed my frequency, my frequency here should change of course, but why is the amplitude changing?
What is happening here?
That is weird.
I noticed that this amplitude became smaller because the amplitude of the green became smaller.
And remember the green was the voltage across the capacitor.
So this is your RC.
And here is my input.
My input has the amplitude, which I am holding more or less constant.
And notice that vC decreased in value as I increased my frequency.
Just hold that thought.
As I increased the frequency of my input the amplitude of the output kept diminishing.
In other words, the gain of the system seemed to have decreased as I increased by frequency.
And today we will look at why that is so and how we can analyze that.
The other thing that is not so obvious here is there is a phase shift.
What I am going to do is try to see if we can observe the phase shift here.
Notice here.
What we have been used to is for the amplifier we get a complete inversion at the output.
Inversion means a phase difference of 180 degrees for a sine wave, right?
This peak should have been here, but notice that there is a shifting of the peak.
In other words, if the yellow was my input my output should have had its minimum when the input had its maximum.
But notice there is a shifting of the signal such that the output is a maximum, not quite at the point where the input is a minimum but a little bit after that.
Also weird.
Not only has this little circuit here lost its gain somehow, but also it seems to have shifted the signal in phase.
That is weird.
And today we will look at why that is so and try to understand the frequency behavior of this little subcomponent here.
Notice that vC is exactly 180 out of phase with vO.
So vO is faithfully an inverted amplified form of the input vC.
However, vC itself should have been the same as vI but it looks quite different.
So let's understand why that is so.
The subcircuit to model is the subcircuit comprising the source, resistor and the capacitor.
And I am just showing that to you here.
I have a vI, a resistor and capacitor.
And I am going to understand how this behaves.
And it is a first order circuit, single capacitor.
My input is a vI cosine of omega t. vI is a real number for t greater than zero.
And I am telling you that the capacitor voltage starts out being zero.
And my vI is a sinusoid.
It's not a step this time.
It's a sinusoid.
So vI is a sinusoid and I want to find out what vC looks like.
The behavior here tells me, I will give you the answer, that when I feed a sinusoidal input as the frequency increases, vC should be decreasing somehow and also be shifting in phase.
Let's do the derivation for that and see what happens.
To give you some insight as to how to go about analyzing this let me draw a little graph as to the effort level of doing this.
To determine vC of t on the y-axis here is our effort.
How hard do we have to work to solve this circuit for a sinusoidal input?
And on this graph, down here is easy and up here is pure agony.
Sheer agony up here.
So it's the scale of effort level ranging from easy to complete agony.
And this is the time axis.
And the time axis starts out at 11 o'clock, the early part of today's lecture, and ends at roughly 12, that is today's lecture and this is next lecture.
What I am going to show you today is a method that uses a usual differential equation approach, and this is going to be pure agony.
If you thought last Thursday was agony watch today.
This is going to be sheer, sheer, sheer hell.
So I am going to grunge through that, and I think I will sort of give up halfway because it's just too painful even for me here.
Then what I am going to do is at the end of this lecture I am going to show you an approach that I give a cutesy name.
I call it the "sneaky approach".
We are going to sneak something in there it is going to be a lot easier.
And then in the next lecture I am going to show you an even sneakier approach that is just going to be absolute bliss.
So let's start here.
Indulge me as I go through the agony part.
I am going to blast through it because that is not of how we are going to do things, but you just need to know that that is agony.
Let's do a usual differential equation approach.
Steps one, two, three and four.
Set up differential equation, find the particular solution, find the homogenous solution, add up the two and solve for the unknowns.
It's a mantra.
The four-step manta.
Let's do it.
Step one, write the DE.
That's easy.
We have done this before the RC circuit.
It's RC dvc/dt+vc=vI.
This is no different from what you got from what you got from your RC circuit with a step input, just that my input is VI cosine of omega t in this case.
It is not just a DC voltage VI.
Stare at that.
Enjoy it while the going is easy.
It's like traversing rapids, and before you come to a class five, you have calm and raging waters there, you kind of sit there saying oh, the scenery around here looks really good and so on.
All you are doing is stalling before you have dive down to class five.
We will get to class five rapids in a few seconds here, so just enjoy this.
RC dvc/dt+vC=vI.
You've seen this before.
Nothing fancy.
Good old stuff.
VI cosine of omega t. You want to hold onto your seatbelts?
OK. Let's find the particular solution to the cosine input.
Let's use our standard method.
What I will do is just so, there is going to be so much crapola up there, so that I draw your attention to vP, which is what we are trying to get, I am just going to put a box around vP in red.
If you see like all sorts of garbage appear, just look for the red square.
That is what we are trying to get at.
That's the equation.
Let's try.
First try, A worked before.
A constant value A worked before for DC inputs.
Let's try that again.
If it worked then it may work now.
If I use A, a constant value, and I substitute it here, I get dA/dt goes to zero, vP is A, but on the right-hand side I have VI cosine omega t. So clearly A doesn't work.
Sorry.
I struck out.
Well, cosine omega t here, let's try A cosine omega T as my particular solution.
Things are getting a little harder now, a little more painful.
So substitute A cosine omega t here.
So I do get A cosine omega T for vP, but out here I get RCA sine omega t times omega times minus one.
So I have a sine and a cosine, and I have a cosine on the right-hand side.
Sorry, it doesn't work.
Nope, doesn't work either.
Well, let's try A cosine omega t plus phi.
We are now embarking into the rapids here.
You can begin feeling the pressure.
Just to refresh your memories of sines and cosines.
A is the amplitude of the cosine.
Omega is the frequency.
Phi is the phase.
Remember the signal I showed you earlier?
If something happens to the amplitude of the sine, something happens to the phase.
A cosine omega t plus phi.
Let me plug it in here and go by standard practice.
Here is what I get.
I plug in A cosine omega t to this equation, and this is what I get.
I differentiate it.
I get omega out minus sine, sine of negative d plus phi, A cosine omega t plus phi equals VI cosine omega t. That might work.
Now we get into the class six part of the class five.
All class fives have a little bit of class six rapids.
Remember, the rapids go up on an exponential scale so it like earthquakes.
What I do now is expand out sine omega t plus phi, blah, blah, blah, it goes on and on.
I could go on and on, but this is even tiring me.
This can be made to work, but I am not sure I want to put all of us through this trig nightmare here.
If I am really, really nasty I could give this to you as a homework assignment or something, but I am not that nasty so you won't see that.
But if I go down this path it will get me to the answer, but I would have to soon negotiate class six, class seven rapids to get to where I want.
So let me punt on it, let me start from scratch.
I am at step two, let me start from scratch.
Instead what I would like to do is let's get sneaky here.
Rather than negotiating the class five rapids, what we can say is ah-ha, we can take our canoes and jump onto shore and run down and then get back onto the river.
Let's do that.
That is called the sneaky approach.
So that all our friends who are behind us think we are negotiating the rapids, but what we are going to do is get sneaky and take the shore path.
Let's get sneaky, just walk down the shore and see what is there.
I want to do something completely weird here.
I want to look at solving this equation through the shore method.
S stands for shore or S stands for sneaky, whatever you want.
What I am going to do is rather than trying to solve for VI cosine omega t. I am going to say let's try a different input all together.
And you will understand why in a second.
It's like I am the captain of my canoe and I tell my teammates, hey, let's not negotiate the rapids, let's go and explore the shore.
Maybe down the shore we can find a path that gets us across to the other side more easily.
So here is me and my colleagues carrying our canoe and getting onto shore and taking a sneaky path.
This is not what I set out to solve.
I don't know where this will lead me.
But let's see where the shore path leads us.
I want to try solving this equation Vie^st.
S stands for shore or sneaky or whatever you want.
I don't know where I am going, but let's see where this leads us.
Let's explore.
Make believe you are Columbus or something.
I don't know.
Let's use the usual techniques and see how this works out.
Let's try a particular solution, Vpe^st.
My input is Vie^st.
I am trying to solve the circuit for a different input.
And let me try solution Vpe^st and see if that works out nicely.
I substitute Vpe^st into my equation here, so RCVpe^st blah blah blah.
What I get here is Vie^st, Vpe^st stays the same, but here vP comes out, s comes out and e^st stays the same.
That is nice property of exponentials.
This is what I get.
A really cool property of exponentials is that when I differentiate it I get the exponential back.
Unlike a cosine I got a sine, and for a sine I got a cosine.
Exponentials are very plain and simple, are straightforward.
What you see is what you get.
You differentiate it.
You get the same thing, you get scaling vP, S and so on, and some scaling here.
You get S scaling here, but the basic form stays the same.
This is quite nice.
I have e^st in all three places, so I can cancel those out and I get this expression.
And I get this.
Wow.
So if I go down the shore I get some place fast.
I don't know where I am yet, but whatever I did, it was easy.
I am just exploring this path, down the shore path.
I am making progress.
I don't know where I have gotten yet.
We will see where we got to in a second, but I got some place quickly, fast.
I was able to solve for this input Vie^st and get this solution very quickly.
So what happened here?
I assumed the solution of the form Vpe^st, substituted it back, and found that if vP were equal to Vi/(1+sRC) then Vpe^st is a solution.
What I have done here is that Vi/(1+sRC) is a particular solution to this equation for the input Vie^st.
I put a box around it because this is important.
This was easy.
We went down the shore, and said let's try this other input.
We made rapid progress on shore and I got some place.
I don't know where I am yet.
I got this.
Let me pause here and let me give you the final answer.
I am going to show you over the next five minutes that this is our answer.
You are staring at the answer already.
I am a party, I have taken a shore path and we have gotten some place.
We see the river there, so it turns out we are exactly where we want to be, just after the rapids.
All I have to do now is get my colleagues into the river with myself in the canoe and we are done.
You don't know that yet.
My colleagues and I are sitting on the shore looking at the river.
We've gotten some place.
There are no rapids there.
We have gotten some place.
We don't quite know is this just after the rapids or not.
We don't know yet, but I got there very quickly.
And I will tell you right now, that is the place we wanted to go to.
The next five view graphs I am going to blast through.
There is going to be more pain and agony to show you why that is the case.
It's me thinking I am Columbus and proving to my colleagues that this is where we want to be.
And pulling out my sextant, and the compasses and so on that cartographers and people use to prove to my colleagues that this is where I want to be.
This is the answer.
The next five view graphs will be demonstrating that this is indeed the answer, or close enough to the answer that we will be satisfied.
Isn't this spectacular?
I am going to show you in about five minutes that this gives us all the information we need to know to compute the sinusoidal steady-state response to this differential equation.
Let me write that down here just so you know.
Just so you remember.
I am going to put a marker on the shore that says this is where we are right now.
Now let me prove to you.
As I just said, vPS is Vi, it's this stuff here multiplied by e^st is the solution to Vie^st.
This guy here is a solution for Vie^st and vP is simply the coefficient that multiplies e^st.
Similarly, if I substitute S equals j omega.
I told you five view graphs of more hell, but I am just going to prove to you that this is it.
I am substituting S equals j omega.
This is Columbus giving a big speech at the end convincing his colleagues that we are where we want to be.
I substitute j omega for S and this is what I get.
This is a solution for e to the st, and so this is a solution for e to the j omega t. And let me mark this for you as something to remember.
Vi/(1+j omega RC).
In terms of that, I am substituting j omega for S. And this is a complex number.
It is a complex amplitude.
It is a complex number because of j here, and it multiplies e to the j omega t. Just keep this in mind.
So that was easy.
The steps were easy.
I am still proving to you that this is where we want to be.
Now let me draw the connection back to vP.
And the first fact was that finding the response to Vie^(j omega t) was easy.
We know that.
Second was the following observation that Vi cosine omega t is simply the real part of this number here.
So Vi cosine omega t is simply the real part of Vie^(j omega t) from the Euler relation.
So cosine omega t is simply the real part of this guy.
Light bulbs beginning to go off?
The first fast was that finding the response to Vie^(j omega t) was easy.
And the response was this, right?
Times e to the j omega t. That was easy.
All right.
And the second part is that the real part of this is Vi cosine omega t was our input.
Draw the connection between two steps.
Finding the response to Vie^(j omega t) was easy.
The real part of that was the input we cared about.
Are light bulbs going off?
Let me draw you a little picture here to show you what is happening.
Response to vI is vP.
It's the particular response we are looking for.
Remember the red square?
But we threw in a sneaky input vIS and we formed the response vPS to that.
This step was easy.
This step was hard.
vI to vP was hard, trig nightmare, remember?
But vIS to vPS was easy.
It was a simple Vpe^st thing.
We also know that the real part of vIS is vI.
The real part of this is simply vI.
If you have a real circuit, if you have a real linear circuit, for a linear circuit, if the real part of this gives me vI then the real part of the solution should give me vP.
So computing vPS was easy.
If I take the real part of this, I take the corresponding real part of this.
This is sort of an inverse superposition argument.
Superposition, I said, take the response for A, take the response for B, add them up and you get the response for A plus B.
Here what we are saying is that get the response to A plus B, or to A plus jB and the real part of the input will produce the response given by the real part of the output.
So this is an inverse superposition argument.
If it is a linear circuit, then if vI is the real part of this sneaky input then I find the response to the sneaky input and take its real part I should get vP.
Here I am, Columbus, staring down at the entrance to this part of the river.
I just proved to my colleagues that all we have to do is take the real part of what we have.
We can just jump right back into the river and get back to vP.
And what I am going to do next is just grind through the math and just show you that.
I will just blast through it.
It is not important, but you have it in your notes.
I am telling you that vP is simply the real part of the sneaky output.
And I take the real part of vP e to the j omega t. And I take the real part.
And just a bunch of math here.
I am just taking the real part and doing a bunch of complex math.
Remember vP was given by this quantity here.
And I take the real part and I end up with vP is simply this quantity multiplied by cosine omega t plus phi, where phi is given by is given by tan inverse of omega RC, and this is the coefficient multiplying the cosine.
So by taking the sneaky path and then taking the real part of that output answer, I was able to very quickly get to where I wanted to be.
So from here to here it is only math.
Recall, that vP, the thing in the red was what we set out to find out, which was the particular response to VI cosine of omega t. And remember that two grunge is all of this stuff.
I am going to blast through two or three more view graphs that just give you more insight and more math, nothing particular.
And remember to solve the equation we have to find a homogenous solution, too.
But recall that the homogenous solution for an RC circuit is of the form Ae^-t/RC.
This means that as time becomes very large this part goes to zero.
As time becomes large in the steady state, remember I care about the steady state?
This goes to zero.
I don't care about the homogenous solution.
Isn't that fantastic?
Most the circuits we will deal with, except for purely oscillatory ones, the homogenous part dies away.
You have something like e to the -t whatever.
It just dies away.
It's gone.
So the total solution has vH going away.
And what I end up with is just vP.
My total solution in the steady state is simply vP.
And A is given by this that we just calculated.
I just have a bunch more insight that I talk about that you can look through in your notes.
And I just want to show you a very quick summary.
In summary, what we have is we computed vP.
It was a complex coefficient.
And all these steps, 2 grunge, 3 and 4 were a waste of time.
And what I showed you was that for the input VI the coefficient vP was complex.
And I can take the ratio and represent it in this manner as well.
And from vP, I can then compute the multiplier for the cosine as follows.
I divide by vP here.
Remember the cosine was multiplied by, in the mathematical step that I did, VI divided one plus, this stuff here, so I could get the magnitude and phase of the transfer function of this circuit in the following manner.
And to wrap up very quickly, I am going to cover this again the next time and show you a magnitude plot.
Notice here that if I plot Vp/Vi.
Remember this was Vp here.
That's the answer.
The magnitude looks like this.
On a log scale Vp/Vi for small frequencies omega is at one, but as omega increases Vp/Vi keeps decreasing.
That is the output.
Remember Vp was the amplitude of the output?
That keeps decreasing.
And this is the reason why.
As I increase the frequency, the amplitude of my output cosine kept decreasing.
I could also plot the phase for you.
And the phase, in the same manner as omega increased, my phase also kept shifting from zero initially to pi/2 finally.
Let me stop here and start with this the next time and revisit this.
Unfortunately, I won't have time for the demo.
I will show it to you next time.
All right.
Let's get moving.
Good morning.
Today, if everything works out, we have some fun for you guys.
I hope it works out.
We'll see.
What I am going to do today is a very major application of the frequency response and the frequency domain analysis of circuits.
And this application area is called filters.
The area of filters often times demands a full course or a couple of full courses all by itself.
And filters are incredibly useful.
They're used in virtually every electronic device in some form or another.
They're used in radio tuners.
We will show you a demo of that today.
They're also used in your cell phones.
Every single cell phone has a set of filters.
So, for example, how do you pick a conversation?
You pick a conversation by picking a certain frequency and grabbing data from there.
They are also in wide area network wireless transmitters.
Do we have an access point here?
I don't see one, but you've seen wireless access points.
Again, there they have filters in them.
So, virtually every single electronic device contains a filter at some point or another.
And so, today we will look at this major, major application of frequency domain analysis.
Before we get into that, I'd like to do a bit of review.
The readings for today correspond to Chapter 14.4.2, 14.5 and 15.2 in the course notes.
All right.
Let's start with the review.
We looked at this circuit last Friday -- -- where I said that for our analysis, we are going to focus on this small, small region of the playground.
And what's special about this region of our playground is that I am going to focus on sinusoidal inputs.
And, second, I am going to focus on the steady state response.
How does the response look like if I wait a long, long time?
And then we said that the full blown time domain analysis was hard.
This was, remember, the agonizing approach?
And then I taught you the impedance approach in the last lecture, which was blindingly simple.
And, in that impedance approach, what we said we would do is -- I will apply the approach right now and in seconds derive the result for you.
But the basic idea was we said what we are going to do is assume that we are going to apply inputs of the form Vi e to the j omega t. Wherever you see a capital and a small, there is an implicate e to the j omega t next to it.
I'm not showing you that.
And what I showed last time, and the class before that was once you find out the amplitude -- Once you find out the multiplier that multiplies e to the j omega t, it's a complex number, you have all the information you need.
And once you have this, you can find out the time domain response by simply taking the modulus of that, or the amplitude and the phase of that to get the angle.
And that gives you the time domain response.
So, our focus has been on these quantities.
The impedance method says what I am going to do is replace each of these by impedances.
And then the corresponding impedance model looks like this.
Instead of R, I replace that with ZR.
And instead of the capacitor, I am going to replace that with ZC.
And this is my Vc.
ZR is simply R and ZC was going to be one divided by sC where s was simply a shorthand notation for j omega.
Based on this, once I converted all my elements into impedances, I can go ahead and apply all the good-old linear analysis techniques.
I will discuss a bunch of them today.
As an example, I could analyze this using my simple voltage divider relationship.
Vc is simply ZC divided by ZC plus ZR times Vi.
And that, in turn, is, well, let's say I divide this by Vi so I can get the response relation, is ZC divided by ZC plus ZR.
And ZC I know to be one by j omega C, plus R. And multiplying throughout by j omega C, I get one divided by one plus j omega CR.
It's incredibly simple.
This is simply called the frequency response.
And it's a transfer function representing the relationship between the output complex amplitude with the input.
We can also plot this.
Notice that in our entire analysis we have not only assumed sinusoidal input, but we're also saying that let us look at this only in the steady state.
So, we will wait for time to be really, really large, and then look at the response.
And so, therefore, we will plot the response not as a function of time, but rather we are going to plot the response as a function of omega.
What we are going to say is I am going to input a sinusoid and my output is going to be some other sinusoid.
And since I'm waiting for a long time to look at the output, time doesn't make sense anymore.
Rather, my free variable is going to be my frequency, so I am going to change the frequency of the input that I apply.
And so, I am going to plot this as a function of omega.
This represents a completely complimentary view of circuits, the time domain view and then there is a frequency domain view.
The frequency domain view says how did this circuit behave as I apply sinusoids of differing frequencies?
I can plot that relationship in a graph like this, and this relationship is simply given by a parameter edge the transfer function, it's a function of omega.
And I can also plot the absolute value of that.
And let's take a look at what it looks like.
So, I can look at functions like this and very quickly plot the response.
I am going to do a whole bunch of plots just by staring at circuits and staring at expressions like this.
And you will see a number of them today.
First of all, the way you plot these is look for the values where omega is very small and when omega is very large.
When omega is very, very small this term goes away.
And so, for very small values of omega the output is simply one.
Vc by Vi is simply one.
This part goes away.
What happens when omega is very, very large?
When omega is really large, this part dominates, is much greater than one.
If I ignore one in relation to this guy and take the absolute value of that then I simply get one divided by omega CR when omega is very large.
So, when omega is very large, I get a decay of the form one over omega CR.
I know the value for small omega, and it looks like this for very large omega.
And, if you plot it out, this is how it's going to look like.
Let's stare at this form for a little while longer.
And let's plot some properties off it.
First of all, you notice something else.
When omega CR equals one then, in other words, when omega equals one by RC, notice that the output is given by one plus j.
And the absolute value of that is simply one divided the square root of two.
So, in other words, when omega is one by RC -- When omega is one by CR then the output is one by square root two times its value when omega is very, very small.
So, that is one little piece of information.
If you look at the form of this, I would like you to stare at it for a few minutes and try to understand what this represents.
This says that for very low frequencies the response is virtually the same as the input in amplitude.
In other words, if I apply some very low frequency sinusoid of some amplitude then the output amplitude is going to be same as that amplitude.
And that's a one.
Now, it also says when I apply a very high frequency, at very high frequencies it decays.
So, this graph which says I am going to pass low frequencies without any attenuation, without hammering it, but I am going to clobber high frequencies and give you a very low amplitude signal at the output but pass through, almost without attenuation, the input at low frequencies.
And so this is an example of what is called a low pass filter or LPF.
What this is saying is that this little circuit here acts like a low pass filter.
It's a low pass filter because it passes low frequencies without attenuation but kills high frequencies.
If I take some music, and you will do experiments with this in lab.
When is lab three?
People are doing lab three right now, right?
Lab three is going on right now and early next week as well.
And, in lab three, you will play with looking at the response to music of different types of filters.
If apply some music here, you will see that the output will pass low frequencies but really attenuate high frequencies.
You will hear a lot of the low sounding base and so on but attenuate a lot of the high frequencies.
All right.
The other thing that I encourage you to do is Websim has built in pages for a large number of such circuits.
You can go in there and play with the values of RC, or L for that matter, for a variety of circuits.
And, if you click on frequency response, you actually get both the amplitude response and the phase as well.
You can play with various values of RLC and see how the frequency response looks like for each of the circuits.
As a next step, what I would like to do is just give you a sense of how impedances combine.
This won't be very surprising given that they behave just like resistors, but it's good to go through it nonetheless.
Suppose, just to build some insight, suppose I had two resistors in series.
All right.
R1 and R2.
And this was my A and B terminals respectively.
And let's say the complex amplitude of the voltage was Vab across this.
Then I could relate, let's say Iab was the current, I can relate these resistances.
Or, I could relate Vab and Iab as follows.
Simply Vab divided by Iab equals R1 plus R2.
I know that.
And the same thing applies to R viewed as an impedance.
It's still impedance R, and so this one still goes ahead and applies.
The second thing I can try is the circuit of this form.
A, B, and I have an R1 and an L in this case.
And what I can do is, in the impedance model, I can view this as an impedance of value j omega L. And I can also combine them to get the impedance between A and B.
Much as I got a resistance between A and B, I can get an impedance between A and B as Vab divided by Iab.
And that will be given by ZR1 plus ZL, and that is simply R1 plus j omega L. Similarly, I can do an even more complicated circuit.
So, resistance.
And here I have a capacitor in series with the resistance, and then I apply inductor to it.
This is A, B, Iab and plus, minus Vab.
And let me call this R1 and let me call this R2 and this is C and L. I can go about combining these in much the same manner that I combine my resistances in the series parallel simplifications.
I can define an impedance Zab between the A and B terminals as ZR1 plus Z of this combination, impedance of this combination, which is simply impedance of C and that of R2 in parallel with each other.
I get Zc in parallel with ZR2.
Notice that this notation simply says that look at the impedance of the capacitor in parallel with a resistor.
And then, finally, I add to that the series impedance of the inductor ZL.
Exactly as you would have done for resistances, if all of these resistances you would have said R of this piece plus the R of the parallel combination plus the R of whatever was here.
This time around we have impedances.
And replacing this with the values, this is R1.
I know for ZL it's j omega L. And so, for ZL, parallel ZR2 it is given by ZCZR2 divided by ZC plus ZR2, which is simply R1 here and j omega L. And let me just substitute the values here.
I know that ZR2 is simply R2, ZC is one by j omega C, and then one by j omega C plus R2.
And I can go ahead and simplify that further and get my impedance Zab.
Notice how simple analysis has become.
Using this technique, using the impedance method we've managed to convert our analysis from solving differential equations to going back to algebra.
A large part of what we do in circuits is see how we can get back to really simple algebra and try to be clever about how we do things.
So, this is as far as analysis is concerned.
In the next five minutes, I want to give you some insight into how you can build different kinds of impedances.
And I won't go into too much detail but give some insight into how you can get a sense for the kind of filters you want to design.
Or, at the very least, given a filter, how can you very quickly get some insight into what kind of filter it is, how it performs, what its frequency response is and so on.
And, this time around, this piece of intuition will be in honor of Umans.
And back to our Bend it Like Beckham series, I call this "Unleash it like Umans".
What experts in the field do is they don't go about sitting around writing differential equations, but rather use a lot of insight into how to solve these things.
And so in honor of Umans, I will label this unleash it like Umans.
Let's get some insight into how the response of various elements look like.
Let's take, for example, I have some impedance Z.
Let's say this could be a resistor, it could be an inductor or it could be a capacitor.
Let's take a look at what the frequency response of just these elements look like.
In other words, what are the frequency dependents of Z itself?
Let me just plot the impedance of each of these elements as a function of frequency.
Let me just take the absolute value of their impedance.
Notice that it's a complex number.
For the inductor it's j omega L. And let me take the absolute value omega L in that case and plot it for you.
And use that to develop some insight.
Let's do a simple case first.
If Z is a resistance of value R then no matter what the frequency my value is going to be R. If I have an inductor of value L then the impedance is going to look like j omega L, and so I am going to omega L for that.
And the dependence of that simply says that for low omega the impedance is very small.
For omega zero the impedance is zero and it increases linearly with omega.
So, it's omega L for the inductor.
Impedance increases linerally as I increase the frequency.
What about for the capacitor?
For the capacitor, the impedance is one divided by j omega C. And so, therefore, I get the dependence being related to omega C. Which says that for very high frequencies impedance is very low, but for very low frequencies the impedance is very high and I get a behavior pattern that looks something like this.
It goes as one by omega C. As omega is very large, my impedance is very small.
If omega is very small, my impedance goes towards that of an open circuit.
This is not surprising.
You've known this before, right?
That a capacitor behaves like an open circuit for DC.
An inductor behaves like a short circuit for DC.
Notice that zero frequency here corresponds to DC.
The capacitor looks like an open circuit for DC, very high impedance.
The inductor looks like a short circuit for DC, very low impedance.
And the opposite is true at very high frequencies.
While R is a constant throughout.
Let's use this to build some insight into how our circuits might look.
Let me do this example.
Let's say I have a Vi and I measure the response across the resistor.
So, I measure Vr divided by Vi and take the absolute value and take a look at how it's going to look like.
I want you to stare at this for me and help me with what the response is going to look like.
Let's take incredibly high frequencies.
At very high frequencies, this has a very high frequency, what do the capacitor look like to very high frequencies?
Is it an open or is it a short?
A short circuit.
At very high frequencies the capacitor looks like a short circuit.
Then Vi simply appears across the resistor, which means that at very high frequencies the output is very close to the input.
At very low frequencies what happens?
At very low frequencies the capacitor looks like an open circuit.
If this looks like an open circuit then very little voltage will drop across this resistor here because most of it is going to drop across the capacitor.
What is going to happen is, for very low values, I am going to be looking at something out here.
And, because of that, my response looks like this.
And this is of a different form than the one you saw earlier.
In this case, I pass high frequencies but attenuate low frequencies.
Not surprisingly, this is called a high pass filter.
You need to begin to be able to think about capacitors and inductors in terms of their high and low frequency properties.
And, if you develop that intuition, once you develop the intuition about capacitors and inductors and their frequency relationship, that will be a big step forward in 002.
If you get that insight, you will go a long way in terms of knowing how to tackle problems and being able to quickly sketch responses.
Yes.
In the case of, if we get something like j omega L, what you can do is take the limit as omega goes to zero.
If it is omega L then notice that it is going to start linear.
And, on the other hand, if when you get very high frequencies, for example, if you get one by something omega C then this is a hyperbolic relationship, so it is going to go ahead looking like this.
So, you can take a look at a lot of these functions at their very low values and see how they look like at that point.
All right.
The next one I would like to draw for you is something that looks like this.
Let's say, for example, I have an inductor L and a resistor R and I want to see what that looks like.
In this particular example, I have H, take the absolute value.
So, what is this going to look like?
I am going to look at the value across the resistor here.
Here what I am going to find is that at very low frequencies this guy is a short circuit.
Since this guy is a short circuit, all the voltage drops across the resistor so it's going to look like this.
And, at very high frequencies, what I am going to find is that the inductor is going to appear like an open circuit.
And so, therefore, all the voltage is going to pretty much drop across the inductor.
It will be R divided by something plus omega L. So, at high frequencies this guy is going to taper off to zero and is going to look like this.
And this is back to my low pass filter.
Just to go back to a question asked earlier, how do you know what this looks like?
I can very quickly write down the expression for H of j omega.
This is simply going to be R divided by R plus if this is VR.
VR is simply R divided by one by j omega C. I multiply it out by j omega C in the numerator and the denominator.
I'm going to find j omega C here and I am going to get one by j omega C here.
And what is going to happen with something like this is that as omega becomes very small then I am going to ignore this.
When omega becomes very small, I can ignore this with respect to one, and I get R j omega C. Given that, is what I've drawn here correct or wrong?
This goes away with respect to one.
I am left with R j omega C, right?
For very low frequencies.
Given what I have drawn here, is that correct or is that wrong?
Well, it's hard to say.
For very, very low frequencies it starts out being linear because it's an omega relationship, and then it goes up like this and then goes out there.
Let me go onto another example.
Let me do another example here which is something like -- I need to make sure I don't make a mistake here.
If I get R j omega C by R j omega C, you know what, this ends up being a first order system, and so is going to look like this.
I blew it there.
Back to this system here.
If I have an L and an R and I look at this equation to look at what happens across L, you can plot that again.
And for very low frequencies it is going to be zero amplitude here and for very high frequencies this is going to be an open circuit, and so the response is going to look something like this.
That's going to end up being your high pass filter.
As another example, I would like to do a series RLC circuit -- -- and try to get you some sense of what that output looks like.
Let's use our intuition and first write down what this looks like and then go and do some math and see if the math corresponds to what our intuition tells us.
I want to plot Vr with respect to Vi.
I want to plot it there.
For something like this, what happens at very low frequencies?
We are just looking to get very, very crudely what this graph is going to look like.
Very, very crudely what this graph is going to look like.
Given that I am taking the voltage across VR, what happens at very low frequencies?
At incredibly low frequencies, the inductor looks like a short circuit, but the capacitor looks like open circuit.
An open circuit in series with a short circuit that ends up looking like an open circuit.
And so, therefore, all my voltage falls across VR.
Now, what happens at very high frequencies?
At very high frequencies the capacitor looks like a short.
But the inductor looks like an open circuit now for very high frequencies, correct?
Just remember, capacitor is short for high frequencies inductor open for high frequencies.
So, this ends up having a very high impedance.
At very high frequencies this guy has a very high impedance.
And, because of that, for a high value of frequency, I end up going in that manner.
This behavior has the effect of the capacitor here.
And for very high frequencies I get the effect of the inductor.
And so this means that I have very low values for low frequencies, very low values for high frequencies.
And, as the frequency increases, I do something like this.
I keep building up, then the inductor begins to play a role, and then I taper off again.
This kind of a filter where I kill low and high frequencies and pass intermediate frequencies is called a band pass filter, BPF.
This means that it passes frequencies in some band.
Let's get some more insight on this by writing down the equations.
So, Vr divided by Vi is simply R. Using the impedance relation it is R divided by j omega L plus one divided by j omega C plus R. I am going to use this equation later, so let me stash it away on my stack and put a little notation there.
I am going to multiply throughout by j omega C. And what I end up getting is j omega RC divided by one plus R j omega RC, and then here, I get j times j is minus one, so I get minus omega squared.
Let me rewrite it this way.
I get minus omega squared.
So, j j is minus one, omega times omega is omega squared, and then I get an LC.
That's what I end up getting.
And if I take the absolute value here, I end up getting, back to your complex algebra, the square root of this real value squared plus imaginary value squared.
So, one minus omega squared LC plus omega RC squared.
This is from, you can look it up in your complex algebra appendix in the course notes.
It's simply omega RC here, then square of the real value plus the square of the imaginary value, and take the square root of that.
By staring at this, you can notice that you realize a really important property.
When omega equals LC.
I'm sorry.
When omega equals one divided by LC, what happens?
Sorry, square root of LC.
When omega is one divided by square root of LC then omega squared times LC becomes one.
When this is true then this becomes one, and one and one cancel out.
And, not only that, when these cancel out, these two cancel out at that point, so I end up getting a one, which means that when omega equals omega nought equals one by square root of LC and I end up getting a value that is one.
It's pretty amazing.
Which means that if I drive this at omega nought, if my sinusoid has a frequency omega nought where omega nought is one by square root of LC, if I'm sitting here and this is a black box on the right-hand side, and I drive this at a frequency omega nought equals one divided by square root of LC, what does this entire circuit look like to me?
I'm sitting there, the black box here.
I'm driving it at omega nought equals one by square root of LC at that frequency.
What does that circuit look like?
Yes.
It looks like a resistor.
It's pretty amazing.
It means that even though I have an L and a C here, if I happen to drive this at omega nought then the circuit looks purely resistive and it seems to give me the same input appearing at the output.
In other words, the effect of these two cancels out.
And that aspect is called driving the circuit at its resonance point.
Resonance is when you're driving the circuit at omega nought equals one by a square root of LC.
I will very quickly sketch for you a couple of other ways of looking at circuits.
Supposing I looked at this value here, Vlc, I looked at the value across the inductor and the capacitor, what will the frequency response look like?
I am looking at the voltage across the inductor and the capacitor in series.
Let's see.
Let's go back to our usual mantra.
Think about Steve Umans when you do this.
What would he do?
He would say ah-ha, at very low frequencies the capacitor is going to look like an open circuit.
In my voltage divider, I am measuring the voltage across an open circuit, so the entire Vi must drop across the inductor and capacitor.
Similarly, at very high frequencies the inductor looks like an open circuit now, so it looks like this.
At very high frequencies inductor is an open circuit.
And, again, I'm looking at the voltage divider across the near infinite resistance, impedance, so I get a high value here as well.
Well, in the middle the value dips and I get something like this.
So, this thing is called a band stop filter.
Here I can nail any specific frequency, as long as the frequency falls in roughly that regime.
Yet another example.
The reason I'm working on so many examples is that to experts, a large part of what they do is look at a circuit and boom, give a rough form of how it looks like.
That can get you half the way there in most of what you're going to do.
How did this look like?
If I take the voltage Vo versus Vi, let's take a look.
At very low frequencies, the inductor looks like a short circuit, correct?
I am talking the voltage across a short circuit, so it looks like this.
At very high frequencies, I am taking a voltage across a parallel combination, but the capacitor is now a short circuit.
So, that looks like a capacitor.
This looks like an inductor out here and this is a capacitor holding sway here.
And so, somewhere in the middle it goes up and comes down like that.
So, it's a band pass filter.
What is amazing is that you can take fairly complicated circuits, and just by doing a quick analysis of what happens at very low frequencies, what happens at very high frequencies, you can roughly sketch the response.
And then what you should do, in addition to that, is if it's a second order circuit, just assume that it's going to do something interesting at its resonance frequency, at omega nought equals one by square root of LC.
Something interesting is going to happen.
Check it out.
And for circuits that are first order, RC or RL, the important number is the time constant RC.
Usually, when you're driving it at one by RC, omega equals one by RC then what happens is that you often times end up getting a value that is one by square root two times the input value in the circuits we looked at here.
Next, what I am going to do is talk about a major, major application of filters.
And that is an AM receiver.
Let me do Radios 101 for 30 seconds.
These guys have an antenna.
You take a ground here.
You pick up a signal at your antenna.
There is an implied ground as well.
And what you do, as a first step, is you begin processing the signal now.
What we place right there is a little filter that looks like this.
It is a inductor and a capacitor in parallel.
And this capacitor is really your tuner that you can tune to radio frequencies.
And then what you have here is a bunch of other processing and end up with your speaker.
And the processing that happens here is you have a demodulator, you have an amplifier and a bunch of other things that let's not worry about them for now.
What we do here is the antenna picks up a signal.
So, in some sense, this part of the circuit here is your source.
I could replace it with its Thevenin equivalent as follows.
So, the front end of your radio looks like a Vi, R, L and a C. Where have you seen this before?
Right there.
That's the front end of radios.
Let me tell you why I need a band pass filter in a radio out here.
The way life works is as follows.
I have my frequency.
Let me do this not in radians but in kilohertz for now, and let me plot your radio signal strength.
In the Boston area, the signals go between 540 kilohertz and they go all the way to 1600 kilohertz.
In some areas we have begun to use the 1700 extra band as well for some new stations.
This is the frequency range of interest.
If you look at your radio tuner, you will see 540 kilohertz all the way up to 1600 and you can tune your AM radio.
The way it works is that each station is given 10 kilohertz of spectrum here.
And so, this is at 1000 kilohertz, 1010 kilohertz and so on.
And each station transmits its signal in plus or minus 5 kilohertz around that point.
And this station transmits it here and this station transmits it here and so on.
This is 1030.
This guy is WBZ News Radio 1030, for those of you who listen to it.
What happens is that at 10 kilohertz, each station gets 10 kilohertz, and so WBZ transmits in the 10 kilohertz around 1030.
Notice that each of these signals transmitted by radio stations happen within small bands.
Now, you will learn a lot more about modulation and how do you get a signal to go in a small band and all that stuff.
You will learn about that in For now, don't worry about how I did all of this.
How do you listen to that station?
The way you listen to that station is you put a low pass filter here.
You put a low pass filter that does the following.
Let's say I want to hear WBZ If I pass this entire signal through that filter.
And if I arrange to have the omega nought of my filter at If I can arrange to have the omega nought at 1030 then this is the response of my filter.
And I am going to pick out this guy and cut out everything else.
I am just going to get this.
Let's listen to the station for some time.
So, you can see I can tune to the station WBUL.
All right.
Good morning all.
Today we embark on another new chapter in what we do.
And the topic is going to be -- We will talk about this thing called an Operational Amplifier.
Before I get into the lecture, I want to point out a couple of things.
One is that you are going to hear about two big words in today's lecture.
Two big and incredibly important words.
And I want to mention those words to you right now so that when I come to them in lecture you can say OK, I better pay really close attention, these are important words.
All right.
One of them is abstraction.
The second one is feedback.
Two incredibly important concepts.
Abstraction, you have been seeing a couple times during this course, once in the beginning where we abstracted out Maxwell's equations by focusing on a smaller playground and simply using KVL, KCL in place of those equations.
A big abstraction.
It turns out that almost all of EECS is based upon abstractions at various levels.
In the first lecture, I also showed you the layer upon layer of abstraction that we built to be able to build interesting systems.
The second big thing is feedback.
And I am going to relate this to anti-lock breaks in cars.
And so, you can wait and see how we do that.
It's an incredibly important concept.
Before we dive into the amplifier abstraction, let me first talk about something that you know.
Start with something that you know and then lead up into the operational amplifier and its circuits.
You know about the MOSFET amplifier.
The MOSFET amplifier that you know about looked like this.
It was based on a MOSFET.
There was a VS supply.
There was a vI input, a vO output and, as I said, a VS supply.
So, this was a MOSFET circuit that you've seen before.
One way of viewing this is that this circuit has three major ports.
This here is the input port with voltage vI.
This here, between the drain terminal and the ground, is the output port.
I take the output between the drain terminal and ground.
And, finally, we have a third port, which is this one.
It is called the power port.
I apply VS between this terminal here and the ground terminal.
And that gives us the power port.
This device here was a three port device.
Input port or control port, output port and a power port.
And so we looked at the circuit and did a whole bunch of analyses of it.
Then what I can do at this point, now that you've seen this, it's often times interesting to think about abstracting this out into some kind of a building block.
Much like in software, you write a procedure and you abstract out the internal details of the procedure in the procedure declaration and in the call that you make.
In the same way, we can take this little device here and abstract that out into the following abstraction.
We could abstract that out as a device that looks like this.
I have my input port, I have my output port and I have my power port.
So, I can apply VS here.
Notice that I've taken these six terminals here, one, two, three, four, five and six, and put a box around it.
And just exposed the terminals to you.
And I need to tell you a little bit more about the internal properties, but suffice it to say that you can begin working with this little block.
An even simpler version of this for many applications might just look like this, vI and vO where there is a ground that is shared among them that is implicit in this picture.
And vI and vO can simply be the node voltages at these nodes.
This is a progressively more abstract representation of this amplifier.
What we can do is, provided we know, we can abstract out the relevant properties of this block and expose them outside.
And the relevant properties might well be that, let's say here the properties may be that I in is always zero.
I can also express to you the gain of this amplifier.
I may also be able to tell you the Thevenin equivalent for the output.
There are some properties that I can give you that will let you use this building block abstractly.
Today, what we will do is introduce a powerful abstraction of a type of amplifier.
This is called the operational amplifier or "op amp" for short.
What I am going to do is give you a slightly more involved building block than the one I have shown you there.
But suffice it to say that the idea is going to be the same.
This building block looks like this.
This building block has an input port.
This building block also has a port in which to connect power or the power port.
And the way I am going to connect power, I am going to connect a plus VS supply here.
That is going to be my ground node.
And I am going to connect a minus VS supply to this node here.
So, these voltages are both VS.
I want to apply a plus VS here and a negative VS out here.
And I am going to take the output between the ground node and the output node of the operational amplifier and call that a vO.
This is the output port.
So, input port and output port and a power port.
Think of this as a pattern where I have an input port across which I connect the input.
I have a power port across which I connect a plus VS, minus VS supply, and then I take the output terminal and take a ground terminal, which is defined by external components of my circuitry, and use this as my reference node.
Remember ground is just a reference node.
I am going to use this as a reference node.
These two are equal in magnitude.
And take this as my output.
And when I do something like this, I can build an even simpler, so this is an abstract differential input amplifier.
In other words, this amplifier is going to amplify whatever I apply at the input.
A slightly more abstract representation of this looks like this.
vOUT and plus/minus vIN.
This is a slightly more abstract representation where, remember, we are going to draw this again and again, maybe at least 38 or 39 times in this course.
And, remember, each time you draw it, remember that there is an implicit power port, a plus/minus supply that is connected which we don't show.
And I remember when I first learned about it a long time ago there was a confusion in me initially.
How does this work?
Where is the power coming from?
Just remember that power comes from a plus/minus supply, and we just don't show that in this abstraction.
Now, the details, a lot of details are in Chapter 16 of your course notes.
That's the reading for that.
The other thing is that there are some other key properties of this amplifier.
And let me discuss those very quickly.
First of all, I can draw a circuit model for the amplifier.
Make some room for myself here.
And this is a circuit model for what we call the ideal operational amplifier.
And the circuit model is going to look like this.
This is an abstract device.
And, in terms of analyzing how this behaves in a circuit, I am going to show you this abstract circuit that looks as follows.
Some input v is applied at these two terminals here.
And this terminal is called my v plus terminal and this is called my v minus terminal, so this corresponds to these two terminals.
I am telling you that the current going in is going to be zero, so i plus is going to be zero and i minus is going to be zero.
i plus is the current in here and i minus is the current into the v minus terminal, and both these currents are going to be zero in this device here.
The output is going to look like this.
Let me just call it vOUT to be consistent with this here.
And taken with ground as my reference.
The output is simply Av.
In other words, what I am doing is I am going to model this as a device that has a dependent source at its output.
And the dependent source here is a voltage controlled voltage source.
It is a dependent source, it is a voltage controlled voltage source such that the output voltage is A times the voltage v across its input.
This is actually very simple.
Think of these three terminals I have shown you here.
I applied input across these.
And the output is going to be A times whatever I applied.
And A is going to tend towards infinity.
A is going to be huge.
And specific values for A might be a hundred thousand or a million or things of that sort.
Huge A in this abstract amplifier.
In addition to that, the other properties are that it is going to have infinite input resistance.
That means looking in this looks like an open circuit.
The fact that this is open here implies the infinite input resistance across this port.
What about the output here?
Remember, this is a voltage source.
And we have a zero output resistance, which means that no matter how the load affects this, as I apply a load this is going to behave like an ideal voltage source and keep holding the voltage constant based on whatever the function I establish here.
And A is virtually infinite.
Let me pause there for a few seconds and just dwell on this so you just understand what the basic device is.
Following this basic definition, I am just going to build a whole bunch of fun little circuits.
The analysis will be pretty straightforward, but this is a big conceptual leap here where there is some circuitry inside.
Containing resistors, MOSFETs, a whole bunch of stuff in there.
I am not telling you what is inside it.
Much like I could build an abstract amplifier, I could put an abstract box around the amplifier you saw earlier, I want to put a box around some circuitry.
I am not telling you what the circuitry is.
And, if you are curious, you should look at page 581 of your course notes.
There is an example solved.
The example is for a differential amplifier.
This is the small signal analysis chapter.
That differential amplifier that's solved in that example is usually the first stage in an operational amplifier circuit.
That differential amplifier is the first stage at the input.
And that differential amplifier, as the name implies, amplifies not a single voltage but amplifies a differential voltage.
Note that this guy amplifies the voltage difference between these two terminals.
That's v here.
And v is simply the same as v plus minus v minus.
It's the node voltage here minus the node voltage here.
That is what's amplified.
It amplifies a difference.
Therefore, it is called a difference amplifier or a differential amplifier.
And so that input stage is what is inside the op amp.
It's got a bunch of other circuitry like level shifters and so on.
And at the output it has got a buffer.
At at the output it has something that is reminiscent of the source follower circuit that you learned about in recitations, solved an example in the course notes and in your homework as well.
And you solved a variant of the source follower on your quiz as well in problem two.
So, a circuit that looks like that appears at the output.
Remember, for the source follower, the resistance looking in from the output was very, very small.
You have seen some of the pieces that go inside the amplifier, but we will deal with this as a building block and simply represent it using this abstract little circuit.
To dwell on this a little longer, this little device here is the workhorse of the analog industry.
Much like your primitive gate abstraction, your inverter and NAND gate and so on, much as your primitive inverter or NAND gate was from the foundations of the digital industry.
Remember we learned how to build this little abstract device called a NAND gate or an inverter?
We noticed that those form the foundations of very complicated microprocessors.
Those were the building blocks of the digital industry.
In the same way, this little beast here is the building block of the analog industry.
Just to give you an analogy from software, think of this abstract little device as a library routine from a library of functions when you program in C++ or whatever.
Can someone give me an example of an incredibly popular routine that we use all the time that may be called the workhorse of the software industry?
Pardon?
An abstraction, an abstract procedure.
One example might be something like a printf.
Printf is an abstract name for a procedure that goes and does something for you.
It is amazing how we take the lowly printf for granted.
I stick my printf into my program, it includes the standard IO library and it goes and prints a value.
You won't believe how complicated the printf is.
As you go into learning more advanced software subjects, implementing the printf is a nightmare.
It is horrendously complicated.
Just imagine.
You give it a string and it has to go and print that on your terminal or on your Windows system or whatever.
Think of the complicated steps it has to go through.
But, as far as you're concerned, it's simple.
Just print out something and you're done.
The same way.
Think of this as the printf of the analog business.
It is really simple, and the analysis is going to be incredibly simple, it will be mind-bogglingly simple, but inside it, heavens forbid if you look inside it.
Tell you what, go into to S-T-D-I-O dot in one of the library routines and just pore through printf.
The world's worst horrendous macros are in there.
I mean it is just nasty.
The same way inside the op amp, it is nasty.
You don't want to go there.
Much like in your C programming in your classes, you were able to use printf without fully knowing how it was implemented.
Probably some MIT god or some key graduate implemented it, but once it was implemented you just used it based on simple abstract rules as to how it behaved.
You didn't have to know what was inside it to use it.
The same way with the operational amplifier.
So, just think of printf when you see this and just imagine how simple it is going to be to use it.
You may think that I spend way too much time, ten minutes dwelling on this abstract concept, but I like to dwell on things that I think are incredibly important.
The concept of abstraction is very important.
And it's not just in software.
The concept of abstraction pervades all of EECS.
And if I were to give you a project to say go and ask every professor what is the one word that you think best describes all of EECS?
Just pick one word.
Go ask every single professor you know.
What is a single word?
If you were to characterize all of EECS with just one word, what might that word be?
In my mind, it is the A word, abstraction.
It is all over.
If you do a grep on all the words used by all your professors in your four years here, I promise you the first one will be know.
And the second one will be abstraction.
Check it out.
See if what I am saying is true or not.
It is all over the place.
In 6.001, how many times do you think the word abstraction was used in 6.001?
It's all over the map.
It's the A word all over.
Imagine your shock when you see it being used in 002 because the same concept applies.
We build more complicated systems by abstracting out the details of lesser objects, and then using those to build the more complicated systems.
Abstraction is a very powerful mechanism of dealing with complexity.
Next step is how do I go about using the op amp?
Let me show you how it looks on a scope.
What I am going to do is apply input to the op amp, I am going to look at the output, place the resistor RL to ground and look at the output.
And here I am going to apply a plus VS and out here a minus VS. Again, remember that a plus VS simply looks like this and a minus VS simply looks like this.
It's just an inverted VS applied here so I get a minus VS at this input.
First of all, what I would like to do is as I change vIN, I am going to plot for you how vOUT looks.
vIN and this is vO.
I am going to plot vIN in terms of microvolts and vO in volts.
vIN is going to have a very very small, the scale is going to be in microvolts because remember the gain of this is huge.
It's on the order of ten to the sixth.
It's huge.
Small changes in vIN are going to cause massive changes in vO.
I have a very fine scale on the X axis.
What is going to happen if I somehow magically make vIN exactly zero?
If I short these two terminals, if this was a completely ideal op amp, which it never is, if it's a completely ideal op amp, then my output should be zero.
As I increase my vIN the output should be A times vIN.
For some small value of vIN, small v, let's say one microvolt, the output should be one volt.
A is a constant so this would look like a straight line.
And let's say my supply voltages are 12 volts minus 12 volts, if this were an ideal amplifier and I didn't have to worry about the supply, this would just go on extending forever.
But I have a plus 12 volt supply and a minus 12 volt supply.
My output cannot go past those limits.
And so, therefore, my output kind of flattens out at these two points.
And it is called hitting the rails.
Output goes up and you hear a thunk sound and you hit the rails.
When you play with op amps in your next lab, if you listen really, really carefully you may hear it.
So, this saturates out.
Not surprisingly, this region where the output saturates at the supply is called the saturation region.
Remember, don't confuse it with-- It's not the same as your saturation in the MOSFET.
It is a totally different thing.
It is just happenstance that we call this saturation.
And if you would like to think about it, you can think of it as the thunk region.
That's probably more appropriate to distinguish it from the saturation region in the MOSFET.
And, not surprisingly, this one is called the active region.
And it is in this region that we use the op amp.
Here it has hit the rails and is kind of dangling out there.
It's not much use to us.
It's in this active region that we use it because this is where the gain is seen.
Now, it turns out that this is a very high gain device.
It is very skittish.
This gain is kind of a really funny thing.
It's dependent on a bunch of factors.
This could be temperature dependent.
This gain here and this curve is just completely skittish.
It could depend on temperature.
It could depend on time of day.
It could depend on what medication this amplifier is on.
It could depend on its mood swings.
Who knows what?
This is kind of unstable.
And A in particular is highly unstable.
It is going to be big, that's for sure, but it could be ten to the six, on a rainy day it might be two times ten to the six.
If it feeling sleepy it may be point five times ten to the sixth.
It is big but I cannot rely on it.
Let me show you an example.
I want to show you this curve for this MOSFET, apply an input and plotting the output.
What I will do is take a look at this curve.
Then what I am going to do is use a heat gun to heat the op amp and you are going to see this vary all over the map.
If you still remember last week, some of you may remember that from some place in a similar situation where the gm for the MOSFETs you were given was also dependent on temperature and stuff like that.
It is a very common occurrence.
And that is certainly the case for the MOSFET.
Let's apply input.
Let's do this.
This is vIN versus vOUT for the amplifier.
Notice that this is plus 12 volts, this is minus 12 volts.
It is about two volts per division.
This axis here is in microvolts, I believe.
For a very small change, for a few tens of microvolts, I have an incredibly high gain.
Notice that this has an incredibly high gain here.
The gain is the slope of this line, almost a vertical line.
What I am going to do next, is to have some fun, is I am going to heat the op amp.
To show you that A is kind of really skittish and also the fact that it doesn't quite hit zero, it does all kinds of weird things, I am going to heat the op amp.
And then let's take a look at how that curve fluctuates.
What you saw there was that the op amp began to behave really weirdly as I heated it.
Instead of doing this it sometimes did this really weirdly, like getting an offset from the center and so on.
And it does a bunch of other weird things, but we won't go into those details.
It's not relevant for this course.
But the point is that the gain and the offset at the input are dependent on temperature.
And we look for ways to make it less dependent on temperature.
As the next step, what I would like to do is build a circuit.
This is model equivalent of your Hello World program.
We are going to use the printf and build a small program on the printf.
You don't have to worry about how printf is implemented, just that we can build very highly interesting circuits with this horrendously complicated function based on a simple abstraction of the device.
The circuit that we will build is called a noninverting amplifier.
From now on, I am not going to show you the plus/minus VS.
I am not going to show the power port, but it is in there.
It's hidden under the abstraction layer.
This is my op amp.
And I am going to build the following circuit.
This is my v plus and this is my v minus.
What I am going to do is for the v plus I shall apply a vIN.
Let me talk a little bit about ground as well.
Ground is commonly taken as the point at which I connect my VS and minus VS supply.
It is kind of at the midpoint.
And if VS and minus VS are very carefully tuned then the output is also going to be at that same ground reference when the input is zero.
So, the ground is defined as the point at which I connect my plus/minus VS supplies.
I apply my vIN out here.
Then what I am going to do, here is my output vO.
I am going to have a resistive divider to ground here and label these R1 and R2.
And what I am going to do here is feed this back to the input, to the v minus input.
I am going to sample the voltage here and feed that into here.
So, this is my abstract model and this is my Hello World program.
What we are going to do is simply analyze how this little program behaves.
So, my equivalent circuit model.
The way to analyze these is after one or two of these examples, you will be able to directly analyze this just by looking at it, by inspection.
But, much as we did for the other pieces, let me grunge through drawing the equivalent circuit and grinding through the analysis, and then show you the much simpler way of doing it.
And even here, even with this grinding analysis, it is going to be pretty simple in any case.
So, I will replace the op amp with its equivalent circuit model.
Its equivalent circuit was v plus, v minus.
So, that was the equivalent circuit model of the operational amplifier, just this piece.
I draw that for you.
Then what I am going to do is I connect my v in here.
And, remember, I have an R1, R2 resistive divider here.
And this one gets connected to this terminal there.
I also know that i plus is zero.
I also know that i minus is zero.
All I've done is simply replaced the amplifier with its equivalent circuit.
Let's go ahead and analyze that circuit now.
Let's go ahead and analyze that circuit.
And it's going to be pretty simple, actually.
What I am going to show you is the hard way of doing it.
I will show you a much easier way, but the hard way itself is pathetically easy.
What I want to do is find vO in terms of vIN.
And there will be a bunch of other factors thrown in, including things like R1 and R2, A and stuff like that.
Let's go and analyze it.
vO, let's look at that circuit.
By the way, let me take 30 seconds and make a little speech at this point.
When you see circuits like this, and I saw this happen in quiz two as well, for some reason, when you see a new kind of circuit, don't completely go berserk or freeze or whatever.
There is just no reason to.
You know the node method.
The node method is the workhorse of our business.
When in doubt apply the node method.
It will simply work.
Don't freeze.
Don't think oh, man, I need to apply a pattern that I know already.
I must have seen this somewhere.
When in doubt boom, apply the node method.
This circuit here, all I have here is one unknown node voltage.
I know the voltage of v plus, I need to compute the voltage vO.
There are two unknowns, vO is an unknown and the voltage here at v minus is another unknown.
This is a very simple circuit involving a dependent voltage controlled voltage source, and you need to find out vO and v minus using the node method.
Just apply it.
It's simple.
Don't freeze.
Just look at it and say I can do it and apply the node method.
It will simply work.
So, let's do that.
What I can do here is vO is A times v plus minus v minus.
This is actually really simple.
And then, if I take v plus here, I know v plus is simply vIN so I will just make that substitution right away.
So, v plus is simply vIN.
What is v minus?
v minus here is vO -- What is v plus?
I'm sorry, v minus.
v minus is simply the voltage that is between R1 and R2.
Notice that no current flows in to the v minus node.
There is no current flowing in.
Voltage at v minus is simply the voltage given by the resistive divider, which is vO times R2 divided by R1 plus R2.
Stare at that for another second.
The voltage at this node here is simply given by the resistive divider.
Because no current is flowing in this direction.
And no current flows in because I am telling you there is no current there based on my abstraction.
I am telling you i minus is zero.
That voltage is simply the voltage at this resistive divider.
And so I can simplify it further and write this as vO.
So I get, there is a one here.
And I move this thing over to this side so I get one plus A times R2 divided by R1 plus R2.
And that is equal to AvIN.
And simplifying it some more, I get vO is AvIN divided by one plus AR2 divided by R1 plus R2.
Notice how simple this is, and this is the hard method.
All I have done is analyze the circuit using the basic circuit analysis principle that you learned the first week of the course, and I have the output for you.
I just noted very carefully what the relationships were between the various elements in the abstraction.
Notice here that I am told that A is extremely large.
A is on the order of ten to the six and so on.
And suppose it is the case that, let me write that down again.
vO is AvIN, one plus AR2, R2.
Suppose R1 and R2 are more or less comparable and A is ten to the six, it's a huge number, so this whole number is much, much greater than one.
If it is much huger than one, what I can do is I can then write this as follows.
I can say that this is more or less equal to AvIN divided by AR2 divided by R1 plus R2.
I am ignoring the one here.
As soon as I do that, notice I can cancel out A and I get vO to be approximately equal to vIN times R1 plus R2 divided by R2.
Notice now that when the gain is very large the output is a function of the input multiplied by some number.
The beauty of this thing here is that when A is very large, or this expression is very large, A cancels out and there is no A in this relationship.
This means that even though the basic amplifier was very skittish, the output here relates to the input based on components that I have control over.
These are soldiers in my army.
I control them.
So, to give you a sense of some numbers here, suppose A was ten to the six.
And I choose R1 to be 9R.
And R to be some R. Then vO is ten to the sixth vIN divided by one plus ten to the six R divided by 9R plus R. So, that is ten to the six vIN divided by one plus ten to the six divided by ten.
All right.
If I ignore the one here, the ten to the six and ten to the six cancel out, this ends up giving me 10vIN.
So, I get a really nice amplifier whose output is simply ten times the input and determined solely by some resistor values.
Let me show you another quick demo this time and show you the amplifier again, but with resistors connected like that.
And then I show you that I want to heat the amplifier to the wazoo, the op amp to the wazoo, but vO is going to be absolutely rock solid.
Let's try that out.
This time around, this is the transfer function, the vO versus vIN.
And notice that this time around I have similar scales on the X and Y axes, and this has a slope of 10.
This is the point where the amplifier saturates at plus 12 volts, and this is minus 12 volts, and this point here is a zero.
So, this is vIN, vOUT, plus 12, minus 12 and this slope is 10.
What I am going to do now is heat the op amp to the wazoo and this ain't going to change because it's my external resistors that control it independent of the value of A, provided A continues to be very large.
I am just articulating the vOUT, vIN curve.
And let me start heating the op amp.
Notice that it's pretty stable.
It doesn't change because it is independent of the amplifier values.
What I have done now is by connecting these resistors in this way, I have a nice amplifier with a gain of ten.
The question you may ask yourselves is why?
There is this little sucker in there that wants to shoot things up by ten to the sixth.
Wants to knock things off the one rail or the negative rail.
Why is it that it's behaving like a docile lamb here and giving us a nice little factor of ten gain no matter what I do to it?
Why is it doing that?
What is the intuition behind it?
I will draw something on the board, but for the next ten seconds I want you think about it.
See if you can come up with some insight as to why is it doing that.
Why is it exactly ten?
Why isn't the ten to the sixth kind of killing me somehow?
Why am I getting exactly ten no matter what happens?
See if you can come up with some intuition and then I will show you how it works.
I will redraw the circuit in the meantime.
Let me see if I can give you some intuition.
This is my circuit, and let's say this is R and this is R. As an example, let's assume that the input is 5 volts, vIN is 5 volts.
If R and R are equal, what should the output be?
It's R and R, so it's R1 plus R2 divided by R2, right?
It's 2R divided by R, so it has a gain of two.
My amplifier has a gain of two because R1 plus R2 divided by R2, which is my gain, is R plus R divided by R equals two.
So, this will be 10 volts.
If that is 10 volts this is going to be 5 volts, correct?
This R and R, voltage divider, this is five, so I get 5 volts here.
This is v plus.
This is v minus.
I get R and R, 5 volts here, that's how the circuit looks.
Now let's understand what is going on.
And listen very carefully.
This is going to be a key insight that I hope you will carry with you for the rest of your lives.
This is really, really key.
What you are going to see is, I think, the third big ah-ha moment in 6.002.
Like small signal analysis, like the frequency domain stuff we saw, I think this is the third big one in the next 30 or 40 seconds, things that are completely either not necessarily intuitive but are just spectacular in terms of what they can do for you.
Let's see.
Let's suppose that because I am heating it, let's suppose that A suddenly tends to increase.
It wants to increase because I have heated it.
A is saying I want to get out this mold here and starts to break through its shackles here.
Let's say, as a Gedanken experiment, that it tries to shoot up this to 12 volts.
It tries to push it up higher.
This is just a Gedanken experiment.
The up arrow says that the increase in A is trying to push up vO momentarily.
Let's see what happens.
It is trying to push up vO momentarily, so let's say this goes to 12 hypothetically.
If that goes to 12, what should this volt node go to?
Six, exactly.
This goes to 6 volts.
If that goes to six, what does v minus go to?
6 volts again.
So, v minus goes to 6 volts.
Now at the input I have 5 volts at v plus and 6 volts at v minus, so where should the output go?
The output should go down because the voltage of the negative terminal is higher.
And so the output is A times v plus minus v minus.
And because this has gone down, this has gone up here it is going to try to pull the output down.
That is going to pull the output down let's say to 9 volts or something.
Cachunk, there is a big battle going on here.
A has gone up, it has boosted it up to 12, but the moment that goes to 12, this goes to 6, this goes to 6, and the op amp output has to go down to 9 volts now because this input is higher here.
If this goes to 9, this goes to 4.5.
If that goes to 4.5, this goes to 4.5.
What happens now?
If this goes to 4.5, what happens?
It wants to go back up.
Can't it make up its mind?
This guy wants to go back up now because v plus is higher than v minus.
What am I seeing here?
This whole circuit here behaves like my little son, my 9-year-old.
If say do this, he wants to do the exact opposite.
So, there is a trick in how you make them do things for you.
Look at this.
Because of this arrangement of the circuit when A tries to push the output up, the rest of the circuit tries to pull it back down to where it used to be.
If the circuit tries not to follow the true path, the rest of the circuit tries to whack it into shape so it follows a true path.
And what's happening is because, in this arrangement, I have fed back a portion of the output to the negative input.
I have fed back some of the output to the negative input.
And by providing this feedback of a portion of the output to the negative input, I have arranged it in a way that I have something called negative feedback.
What negative feedback does is that if this wanted to go wild and crazy, the circuit provides it with some negative feedback like you just saw.
Feedback, a big word.
If you take a poll of all the EECS faculty, I suspect that feedback would rank at least as the ninth or tenth most important word in the EECS.
If abstract is number one, I think this would rank like a nine or a ten or something.
So, that's the reason why it worked.
In the last couple of minutes, let me give you some insight, based on something that you know, on how feedback works.
This is a road here.
Let's look at anti lock breaks.
This is my tire.
And let's say I have a set of disk brakes here.
As the car is moving forward, if I apply the brakes the tire stops rolling, but if I apply the breaks too hard it can lock up the tire and the whole car can skid.
The way anti lock breaks work is as follows.
There is a controller that sits here.
And there is a little person looking at the wheel and seeing is it turning.
So, this is a feedback.
And it is saying is it turning?
Yes.
Or, is it not turning?
No.
All this person watching the tire is doing is saying is it turning or is it not turning.
That is it.
That is a negative feedback.
And so, if it is no and if it is yes.
If it is yes then what this does is it applies the brakes even more strongly.
It is turning so I can apply more brakes.
But if it says oops, it stopped turning, what it does is it simply releases, the controller releases the brakes.
And when the controller releases the brakes this one tends to loosen up a little bit and the tire starts turning again.
So, this way you are constantly keeping the tire in its region of critical friction so that it is constantly moving.
And static friction applies to how hard you can brake and it doesn't start skidding.
In fact, if you take your car out, and I don't say you do this.
Let's say go onto the Charles River in the dead of winter and you drive on the lake and you slam your anti lock brakes on, on an icy patch, you will notice that there is a constant sound that looks like something is vibrating in there.
That is exactly what is happening.
Oops, the tire is locked.
Release the brakes.
The wheel is turning.
Jam the brakes on.
That is exactly what is happening.
The same way as out there, you notice that oops, the output is going up, pull it down, oops, it's going down, pull it up.
So, there is constant negative feedback that is keeping the output stable.
A very important concept.
And I will ask your recitation instructors to cover the very simple method that is on page 9.
All right.
Good morning.
Let's get going.
In today's lecture we continue with the operational amplifier, "op amp" for short.
And what we are going to do is just build up a bunch of fun building blocks using the op amp.
As a quick review -- To quickly review what we've seen about the op amp -- We represented the op amp as a device that looked like this where the amplifier had an incredibly high gain.
So, if I had a small voltage difference here -- I call this v plus and this v minus with respect to ground.
And if I had a small voltage difference then this gain here would multiply the difference by a large number and thereby giving me an output that was on the order of a million times greater than this difference.
And because of that when I use the op amp in a mode like this without any negative feedback the output would usually crank up to the positive rail or the negative rail.
We also saw that it had infinite input resistance so that the current flowing in here or here was zero and also had zero output resistance.
This is my ideal op amp where irrespective of what load I connect here the op amp would supply pretty much any current.
Now, in practical op amps that's not the case.
But suffice it to say that when used as an ideal op amp the output impedance, the output resistance is going to be zero.
The op amp is a huge workhorse of the analog industry.
You will see based both on what you've done on Tuesday and Wednesday but also today that it's very, very simple to build circuits using the op amp.
When you use the amplifier, you don't have to worry about things like nonlinear analysis.
You don't have to worry about am I really meeting the criteria for saturation limits and so on?
To some extent you have to think about that with the op amp, too, because if the output hits the positive rail or negative rail it isn't going to behave like you expect it to.
But fundamentally with this primitive model, this idea model it becomes really simple to build circuits with the op amp.
Therefore it has become a key building block for circuits.
When circuit designers build analog circuits very often their primitive building blocks are really an amplifier of this sort, an op amp, resistors, capacitors and some of our other primitive building elements.
If you look at the course notes the readings are -- There are a bunch of examples solved in Chapter 16.
And you will see that using the op amp it is indeed possible to build current sources that look like more or less ideal current sources.
It is also possible to build voltage sources and so on.
It is an incredibly neat building block using which you can do all kinds of cool stuff.
In this course you will see a whole bunch of example circuits using the op amp.
In today's lecture you will see things like a subtractor.
You will also see integrators and a differentiator.
And then in your lab, lab four, you will build a really fun mixed signal circuit involving both digital and analog components.
And you will build what is called a digital to an analog converter using the op amp.
And of course I can build all our good-old amplifiers and circuits of that sort.
In a later lecture you will also see how we can build filters using an op amp.
This is going to be using the knowledge you learn in terms of connecting resistors, capacitors and inductors together and doing a frequency domain analysis, well we can throw the op amp in there and build filters, too.
This is just to give you a preview of upcoming attractions.
For today I am going to focus on these circuits.
I won't be covering any new theory or any new set of foundations but pretty much take the simple properties that I have explained to you about the op amp.
And using those simple properties very quickly build up a bunch of circuits that you can use to analyze signals in a variety of ways.
Let's start with the following circuit.
With op amps I start with this little guy.
And what I am going to do is use two voltage sources, v1, and this is a resistor, not an inductor.
And value R1, value R2.
So, I have a voltage connected by a divider, voltage divider to the plus input.
And I am going to provide some negative feedback in the following way.
This is going to be R2, the same as this one here, a resistor R1.
And then a voltage source v2 that I connect out here.
So notice that- Oh, and I take the output vOUT out here.
And that vOUT of course is with respect to ground, and R2, v1 and v2 are also connected to ground.
What I am going to do is analyze the circuit it two different ways, and as I analyze it describe some other interesting properties to you.
In the last lecture the technique I used to analyze op amps was one in which I replaced the op amp with its ideal model involving a dependent source and so on with a large gain A and showed that.
I wrote the expression and then I let A increase to infinity to the limits and got an expression that was independent of A.
And then in recitation yesterday you would have covered another technique which makes it much simpler to analyze op amps.
Let me very quickly review that method.
We fondly call that technique, there is no formal name for it, but we fondly call that v plus more or less equal to v minus method.
This is also variously called the virtual ground method and so on, but we shall call it the v plus more or less equal to v minus method.
The insight here is that whenever I use the op amp in a way in which I am giving it negative feedback, so I am feeding some portion of the output to its negative input.
I am giving it negative feedback.
That's one property.
Second property is that my inputs, v1 and v2, and my resistance values are chosen such that the output is not in saturation.
So, the op amp is not at the plus VS rail or minus VS rail.
Rather it's somewhere in the middle in its active region.
When that happens we claim that the v minus and v plus for the op amp are more or less equal.
And to give you some intuition as to why that is so, let's say the output is 6 volts and my supply is plus/minus 12.
This is 6 volts and the amplifier is a gain of a million, ten to the six.
To sustain 6 volts at the output all I need is a difference of 6 microvolts here.
Six divided by ten to the six is the difference between v plus and v minus.
It's very, very, very small.
It's so small as to make v plus more or less equal to v minus.
All it takes is a very small differential voltage here to give you 6 volts at the output.
The key thing to observe is under negative feedback, when the op amp is not in saturation the property that v plus equals v minus holds.
And the way it works is that it's not that it's a magical property.
It is simply that when I apply negative feedback the negative feedback is such that it will force this v minus node here to be at more or less the same voltage as v plus.
Remember the when in doubt simply go back and think about the anti lock brakes example we did last time.
For example if v plus increases the output will increase and so will the voltage here and tend to make these two equal.
What we can do, being rather tricky here, what we'll do is say look, if we know for a fact that under negative feedback the op amp is going to engineer these two node voltages to be more or less equal then why don't I just use that fact to begin with and analyze my circuit assuming that it's true.
This is just a bit of inverted logic here that says look, the circuit is going to make that happen.
If the circuit is going to make that happen to analyze the circuit in its steady state, why don't I just go ahead and assume that to begin with?
This again goes back to us wanting to be engineers here and do whatever is simply and find the simplest possible way of getting some place.
I want to use that method, the v plus equals v minus method.
Let me just first write down some values that I know about.
I know that v plus is simply a voltage divider relation here.
That's v1 times R2 divided by R1 plus R2.
And by the v plus equals v minus method I know that this is going to be equal to v minus.
And this is going to be true because I am giving you negative feedback here.
And we are going to engineer the values of R1, R2, v1 and v2 such that the op amp is not in saturation.
So, we know that.
The next thing that we know, let's say this is a current i.
This current i flows here.
Know that there is no current going in here.
Op amp has an infinite input resistance so there is nothing going in there.
There is no current going in there.
If there is no current going in here, what must happen to i?
Remember, from the foundations of the universe Maxwell's equations and therefore KVL and KCL hold.
KVL and KCL simply come straight from nature.
You and I cannot mess with that.
Bad things happen to you if you do.
So, nature, Maxwell's equations, KVL, KCL.
It's simply nature.
So, KCL applies here.
Current comes in here.
Nothing goes there.
Don't argue.
The current has to go here, period.
No if, ands or buts.
There is i coming in here, nothing goes there, so that current must flow here.
It has no choice.
It's from basic nature.
I can write down what my current i is going to look like.
What is i going to look like?
Well, I know v2, I know v minus.
v minus is the same as v plus.
And v plus is the i expression given here.
So, I can write i as v2 minus v minus divided by R1.
Let me keep track of those two and then go ahead and compute vOUT.
So, my goal in life is compute vOUT as a function of the two input voltages v1 and v2.
And just for kicks I have gone ahead and computed some of the intermediate node voltages and currents.
How do I write vOUT?
What is vOUT?
vOUT is simply v minus from KVL.
vOUT is simply v minus minus the drop across this resistor.
So, the drop across that resistor is simply iR2.
From good-old KVL from the first lecture, a voltage minus the drop across the resistor is equal to vOUT.
Therefore it's simply v minus minus iR2.
One thing to be very cautious about, I will tell you right now, is that the output here relates to the inversion of the voltage across this resistor R2.
Be very, very careful in that if I have a voltage across this resistor here that impacts vOUT with a minus sign attached to it.
Notice that iR2 is the voltage across R2 and vOUT relates to the negative of that.
Be very cautious.
That's one of the commonest silly mistakes I have seen people make in solving problems like this.
Let's go ahead.
I know v minus and I don't know i.
Let me substitute for i for now, and that is v2 minus v minus divided by R1 times R2.
Let me go ahead and collect all the v minuses.
v minus, I get a one here, minus minus becomes a plus, and so I get R2 divided by R1 out there.
And then I minus v2 R2 divided by R1.
That is vOUT.
Now let me go ahead and substitute for v minus.
And that is simply v1 R2 divided by R1 plus R2.
That is v minus.
And this character here is simplified to be R1, R1 plus R2 minus v2 R2 divided by R1.
What do we get?
I cancel these two suckers out and what I end up with is v1 R2 divided by R1 minus v2 R2 divided by R1, which is simply R2/R1(v1-v2).
What is interesting here is that what I have ended up building is a very primitive subtractor.
So, my output relates to v1 minus v2 multiplied by the constant factor given by R2 divided by R1.
Again, as I pointed out to you at the beginning of this lecture, no knew foundations today, no new theories, no new disciplines, no new laws.
We are just going to take what you have learned -- Three simple things, infinite gain, infinite input resistance, zero output resistance, plus this new thing v plus equals v minus.
And just being armed with those four principles we are just going to charge ahead and analyze a bunch of circuits.
It is purely intellectual and pure applications today.
This is one way of doing it.
There is another way of solving it.
We can solve the circuit.
Remember, whenever you see a linear circuit and you see two sources or three sources, just think superposition, right?
You see a linear circuit and two or three sources, think superposition.
We should be able to apply superposition to this.
The op amp is simply another building block.
It's a linear circuit.
So, let's see if we get the same answer.
Let's try to solve the circuit using superposition and see if we get the same answer.
To do superposition what I am going to do is build two subcircuits.
One subcircuit in which v1 is zero, and that subcircuit looks like this.
If I set v1 to be zero then I get R1 parallel R2 going to ground.
So, if v1 is set to zero then R1 goes to ground.
And I get R1 parallel R2 here.
And of course I have v2 as before.
And this was R1, this was R2, and let me call that vOUT1.
Oh, I'm sorry.
Let me call it vOUT2 corresponding to that component of the output that relates to v2 acting alone.
Remember superposition?
Build two subcircuits, one that depends on v2 and another one that depends on v1.
Let's do the second one, too.
Second one is v2 going to zero.
Here is my little op amp.
And what I will do is simply flip the op amp just to see if you can identify some interesting patterns.
Just flip the op amp around.
And this is v1 as before.
And recall that v1 was going to the plus node through a resistor R1.
And then I had a R2 to ground.
And then let me short v2 to ground.
And when I short v2 to ground what happens?
When I short v2 to ground what happens is that the tail of R1 here goes to ground.
And so it is as if the output is connected to the node v minus through a resistor, so it as if the output v R2 is connected to the minus input through a resistor.
We will draw it like this.
And the minus input goes through a resistor R1, to ground.
If you thought that patterns were important in the earlier part of the course doing voltage divider patterns and current divider patterns and amplifier pattern, the source follower pattern, op amps is all about patterns.
You should remember two or three simple patterns and be able to write down the expression for those just by observation.
So, this is one common pattern that you have seen before in the very first lecture.
And I just wrote it down in that manner.
Let me go ahead and solve this circuit.
It turns out that this is also a pattern.
I will analyze it today but in the future v2 going to this node through R1 and then R2 to the output.
You have probably also seen this in your recitation.
This one is called an inverting connection and this one here is called a non-inverting connection.
Let's go ahead and do vOUT2.
vOUT2 is simply given by, notice that since this is ground, no current flowing here, this voltage is zero.
If this voltage is zero, this voltage is zero by the v plus equals v minus method.
If this is zero, the current that goes through here is v2 divided by R1.
And that same current must flow through the resistance R2 as well.
If the current v2 divided by R1 flows through this resistor, the drop across this resistor is simply given by, let me hide this for a second, is simply given by v2.
So, v2 divided by R1 is the current here.
This is zero.
So, the drop across this resistor is v2 R1 multiplied by R2.
That's a drop across this resistor.
This voltage is simply zero minus a drop across the resistor.
So, it's zero minus the drop across the resistor and that gives me v2.
Again, remember this minus sign comes in when I want to convert this to get the output voltage from that.
This is a very common pattern.
It's called an inverting connection where the output is some factor of the input voltage and the factor is given by R2 divided by R1.
Let's go ahead and analyze this guy now.
What is vOUT1 equal to?
I should have called this vOUT1 because it relates to v1.
vOUT1.
There is a v plus here.
From our first lecture I know that vOUT1 relates to v plus in the following way.
I know that it is v plus times the sum of the resistances divided by R1.
Based on the first lecture this is true.
vOUT1 is simply an amplified version of v plus where the amplification factor is given by R1 plus R2 divided by R1.
And I know v plus is simply a voltage divider action here.
And I can take a simple voltage divider action here because the current going in is zero.
Looking in here this is as if it's an infinite resistance, so it is as if the element simply does not exist.
The voltage here is simply v1 divided by R1 plus R2 multiplied by R2, our voltage divider pattern.
So, I get v1 times R2 divided by R1 plus R2 times R1 plus R2 divided by R1.
These two cancel out which gives me vOUT1 is simply v1 R2 divided by R1.
To get vOUT I add up the two.
vOUT is vOUT1 plus vOUT2, which is my goal.
And that is simply v1 R2 by R1 minus v2 R2 by R1.
Thankfully what we have here is the same as here.
Again, there is really nothing new that I am going to cover today.
Simply apply, apply, apply, four simple principles.
Here I have used superposition and I am showing you a circuit.
So, it turns out with op amps you should really remember that pattern.
You will see it again and again and again.
And each time you see it, it will save you six minutes of having to solve the circuit without knowing the pattern.
So, remember this pattern.
You can pick up another three or four minutes by remembering this pattern here.
This pattern is simply v2 R2 divided by R1.
Imprint those two patterns into your brains.
OK, so those are a couple of simple circuits using the op amp.
We built a subtractor.
The next step, let's go ahead and try to build an integrator.
Using this little building block we can go ahead and try to build a bunch of circuits.
We can build filters, A to D converters and so on.
Let's build an integrator.
Abstractly I need to build this box.
Which when fed a vI, I want that box to integrate and give me a vO which is vI integrated over time.
That is what I want to build.
How do I go about building it?
What I would like to do next is give you some flavor for design.
How do you go about designing things with an op amp?
Knowing that you do not know the pattern for this yet, how do you go about designing things?
Well, let's start with the following intuition.
The intuition that I begin with is that if I have a current i, and remember that capacitors and inductors related to, you saw differentiation and integration happening when we dealt with capacitors and inductors.
So, I think we have to invoke a capacitor here or an inductor.
In this example I invoke a capacitor.
Notice that if I stick a capacitor in here this current is i, capacitance C, then my voltage vO is given by what?
Voltage is simply the integral of the current flowing through it or vice versa i is C dv/dt.
If i is C dv/dt then v is simply one by C integral.
If I can pass the current through a capacitor then the voltage across the capacitor must be a current.
Notice then that vO is related to i dt.
I have some multiplying constants and so on, but fundamentally what I have found is if I can stick a current through a capacitor then the voltage across the capacitor relates to the integral of the current.
OK, that's interesting.
So, I have an integral in there.
But I have a current.
Notice my goal was to integrate a voltage.
What I figured out how to do was if I can turn that voltage into a current -- If I can turn that voltage into a proportional current and then pump that current through a capacitor I will get the integration that I want.
How do I convert my vI to i?
How do I do that?
Well, let's take a stab at it.
Here is my vI.
Let's take the resistor R. And remember I need to stick the capacitor here.
I have some current I here.
I don't know what the current is yet.
And I stick a voltage here.
And what I am trying to do is trying to see if I stick a voltage and a resistance in series then there is some relationship between the current and this voltage.
Recall that I am trying to make this current be directly proportional to the voltage vI.
But it turns out that i here is not equal to vI divided by R. If i was vI divided by R somehow, I am done.
If i was vI divided by R, by some magic, then I have converted my voltage to a current, I feed that current through my capacitor and vO is my integral that I am looking for.
But unfortunately i is not equal to vI divided by R. You know that.
i relates to vI minus the capacitor voltage divided by R. So, i is not simply vI divided by R for all time but i is really vI minus the capacitor voltage divided by R. And, in fact, when we did RC circuits you wrote this equation to represent the dynamics of the circuit, RC dvO by dt plus vO equals vI.
We wrote down this circuit for a first order RC, wrote this equation for a first order RC circuit.
Now, it does turn out, to wrap up on this wild goose chase that we went on, it does turn out that if this term here is much bigger than that term.
If this term is much bigger than that term then I can ignore that term and write down RC dvO by dt more or less equal to vI.
If that were true, this would be true, and then vO would be more or less equal to one by RC integral of vI dt.
Again, if this were true.
If this were true for all time then vO would be integral of vI dt.
Again, remember this is all a wild goose chase.
Just write down WGC there just so you don't get confused.
I am on this wild goose hunt here trying to find a way to get a current from a voltage which I can then feed into a capacitor.
This was one thing I knew, but this was not what I want.
But it does turn out to be what I want when vO is very, very small.
So, I see some glimmer of hope but not quite.
It turns that in R and C, if I make R and C very, very big, if I have a huge time constant, with a huge time constant the voltage vO looks like an integral of vI, but only when I have a very huge time constant.
So, I give up on that track.
Instead I try something else.
Another try.
I would like you to notice if you take your op amp, here is your op amp, if you take this op amp and you stick the positive terminal to ground, under reasonable feedback, under reasonable negative feedback what do you notice about the current?
If I had a current i flowing here what did you notice?
Look at this picture.
I had a current i flowing in here, v2 divided by R1.
And because this resistance was infinite all the current went through the upper terminal.
So, this is zero volts.
And by the v plus equals v minus method this is also more or less equal to zero.
And I have a current i flowing in here, nothing goes here, so then the i must flow up there.
So, all I am doing here is causing a reflection of the current from this grounded node.
My current is being reflected into, or deflected if you feel like it, the upper edge here after coming in through this edge.
That is interesting.
We are just one step away from the key insight.
I have an i coming in here, an i going out there.
Notice that, as I said before, this is zero volts.
How do I get my voltage vI to look like a current, to become proportional to a current?
It is simple now.
All I do is put a voltage vI and put a resistor R out there.
If I do that, and since this is zero, the current i is given by vI divided by R. I have gotten to where I want to be.
So, by using an op amp and using the fact that the minus node here, v minus is at the same potential as v plus when there is negative feedback then I can stick a resistor here.
And because this is zero the current here is simply vI divided by R. I have gotten to the first place.
Now all I need to do is simply pump this current through a capacitor and I get the integral of the, the voltage becomes an integral of the current.
That is easy.
I stick my capacitor here and I get my answer out there as vO.
Notice that when I do this, let's say this is plus/minus VC.
This is zero.
So, vO is minus VC.
Again, I will keep emphasizing it maybe 17 times throughout this course that if this is zero then the output here is related to the negative of this voltage, common, common, common mistake.
I will be very upset after doing all this if I see this mistake happen in any of the future homeworks or finals or whatever.
This should not happen.
So, vO is a minus sign here VC.
And I know that if I have a current i through a capacitor what is VC?
If I have current i through a capacitor than this is simply t i dt.
And i by design is -- So, I have my integrator.
It is a two-step process.
I stuck a resistor here, so the current became equal to vI divided by R. Then I took that current and pumped it through a capacitor through this terminal here, and the voltage across the capacitor for a current i is given by this expression.
This is Capacitors 101.
OK Capacitors 101 says that the voltage across the capacitor is simply one by C integral i dt.
Another way of looking at it is the voltage across the capacitor is C, I'm sorry, the current through a capacitor is C dv/dt.
This is simply the integral form of that equation.
And I am done with my integrator.
So, this is another very common building block.
Remember this.
Most of the circuits we will be seeing with op amps simply involve something here and some there.
And the output in this inverting connection is the output times, if it is a resistance it is simply R2 divided by R1, if it's a capacitor I get the integral form looking like this.
Yes.
Can someone tell me where the negative sign went?
The blackboard ate it up.
Good catch.
After all that lecture about watching the negative sign.
After this little bit of faux pas here, now I will be doubly mad if you guys make that mistake.
All right.
Now that we have built the integrator, I could give this out as a homework problem.
And you should be able to design a differentiator based on what you've learned here.
You now have the tools to go and do some design like this, but we don't have any more homeworks left so I guess I will go ahead and solve this for you right here and do the design for you.
The building block that we need looks like this, d/dt here.
Let me take a vI and stick a vI in there.
That's what I want to build.
And what I built here is that different integrator box.
And what I would like to do now is build a differentiator box.
How do I go about doing it?
I will go really slow here so you will have some time to think about it for yourselves and see if you folks are crack op amp circuit designers already, if you have the right instincts here.
Again, when you see differentiation integration think capacitors or inductors, it doesn't matter.
In fact, as a homework exercise, you may want to go back and see how you can get a similar effect using inductors.
Can you play with inductors and get a similar effect?
So, inductors are devices that are a dual of the capacitor.
Whatever we will do with capacitors, there must be a corresponding way with inductors.
You can try it out in your spare time.
Let's go back to this one here.
I will stick with the capacitor way of looking at things.
I need a differentiation now.
Remember this.
If I have a vI and I stick this across a capacitor, I have a current C and some voltage vc across the capacitor, what does i relate to?
i is simply C dv/dt and vc in this case is simply C dvI/dt.
If I can stick a voltage across a capacitor, if my input voltage is stuck across a capacitor then the resulting current relates to dvI/dt.
Here we have the opposite problem.
By doing this simple trick, I can obtain a current that has the right form.
Now what I need to do is somehow convert that current into a voltage because the abstraction that I need is a voltage to voltage.
The next step, what I need to do is somehow convert a current to a voltage.
How do I go about doing that?
Again, remember for the op amp, if I have a current i flowing here then by the reflection property i gets pushed up into this edge, provided that the whole circuit is working with descent negative feedback.
Given this trick what I can do is say look, suppose I did this.
Remember, my goal here is how do I convert a current to a voltage?
I have a current i coming in here, and I can turn that into a voltage because I know the current must come out here, I know this current must come out there.
All I have to do is stick a resistor in there.
If I stick a resistor in there what is vO equal to?
vO is simply iR, right?
That's right.
vO, I get i here, so i pumps through here.
Remember, what comes in here must get reflected up because the current going in here is zero.
All the i must come out here.
So, that i must pump through this resistor.
The drop across this resistor is iR.
That's the voltage drop across that resistor.
And since this at a virtual ground the output here is simply zero minus this drop which is minus iR.
So, I have gotten to where I want to be.
I have my current i being converted to a voltage.
I have taken my current, and I have been able to convert that into a voltage by sticking a resistor in here.
As a final step, I simply need to produce the current.
And that is pretty easy to do.
Abstractly what I need to do, again, this is design here so we will talk about abstract stuff.
If I had a voltage vI, I need to produce a current which relates to C dvI/dt.
And I know I can do that by simply doing this.
By doing this I know my i is C dvI, correct?
If I can get this effect, I put this in quotes because that's my pattern.
I am looking for a pattern, where a voltage vI is directly applied across a capacitor.
And when that happens the current relates to C dv/dt.
Let's go back to our op amp pattern here, op amp circuit.
So far I have achieved -- I just repeated this out there.
And so somehow I need to take this pattern here and learn from that pattern and apply the pattern here.
So, what I can do is, this is a ground node, correct?
Now, the poor little capacitor, what does it care, whether it's a ground node or a virtual ground node?
As long as it's a zero volt node down here what does it care?
What I am going to do is stick this point, not here but into a virtual ground node.
I am going to grab that point, take it here and stick it here.
The poor little capacitor doesn't know the difference.
I have really suckered the little beast.
This is vI.
Remember this.
My i through the capacitor is proportional to C dv/dt.
Instead what I have done is taken this guy and stuck it here to get something like this.
Just remember these four or five little tricks.
And you apply them in op amp circuits again and again and again and again.
So, this is vI, this is my virtual ground.
As far as this poor little capacitor is concerned, it is chugging along merrily thinking that it is connected to ground.
Little does it know it is only a virtual ground, all right?
But the current i here is simply C dvI/dt.
And that current, the C dvI/dt, that current flows through here and gives me vO as iR.
So, vO is simply minus R. Let me substitute for i there, C dvI/dt.
OK, so notice then that my vO is now proportional to dvI/dt.
So, vO is some RC time constant times dvI/dt.
Therefore, I have my differentiator circuit.
Remember this as a closing thought.
Remember this v plus more or less equal to v minus trick.
And to the extent possible simply use that trick to analyze op amp circuits under feedback and not in saturation.
Just remember these two.
Very quickly for the demo, I have a square wave input here to the op amp, that's my vI to the integrator.
And this is the output vO.
The integral of a square wave is a triangular wave, as you can see.
And we will do the same thing for a differentiator.
And for the differentiator, I input the square wave to this differentiator circuit.
And I get this, wherever there is a sharp rise, I get this huge negative spike and a positive spike because of the minus sign.
So, this is the differentiator circuit.
Then I feed this into the op amp.
OK.
Thank you.
All right.
Good morning, all.
You have two handouts, lecture notes and an article on mixed signal chips.
A mixed signal stands for circuits that have both analog and digital components to them.
The reason I am giving you the handout is that Lab 4 and also your last homework involve designing and building a mixed signal circuit.
It's a real fun exercise.
And I just wanted to tell you that from past experience people who have taken 6.002 often view the last lab as the single most fun thing they did in all of So, as you go into Lab 4, you should be telling yourself I should be having fun, I should be having, I should be having fun.
You have to positively psych yourself.
Otherwise, it's going to go by.
And then you're going to say boy, that was fun, I wish I had savored the moment as I was doing it.
All right.
Let's see.
What do we do today?
Today's lecture is actually going to be a fair amount of fun.
We are going to blast through a bunch of fun things.
And some things that you will be quite unprepared for.
Until now, in the last two lectures with op amps we talked about negative feedback.
That is applying some portion of the output voltage to the negative input so that I could control this high strung device, my op amp.
Today, what we are going to do is try to get a handle on what happens if we use positive feedback.
It's the usual curious child.
You tell them to do this, and of course they're going to try to do this as well.
And we are going to try to do that and see what happens and look to see if we can build some useful circuits.
Today -- As motivation, let me do a quick review of a circuit that should now become affixed in your brains in a standard pattern.
This is a circuit that gives you negative feedback.
R1 and R2.
And I apply a vIN.
By now you should be able to look at this pattern.
And this is your inverting amplifier pattern.
So, you should be able to write down by inspection this is simply vIN or the minus vIN times R2 divided by R1.
This is an amplifier whose gain is controlled by the ratio of R2 and R1.
This is a negative feedback circuit because it is always fun to do the intuition thing and say that look, if this voltage tends to go more positive than I care then this negative input goes more positive than I care.
If that goes more positive then the negative input v minus becomes more positive in the plus input which yanks the output down.
So, there is a nice counteracting force that keeps the output stable.
Let's look at this circuit.
Being curious engineers, let's look at the opposite here where I give myself some positive feedback in this op amp.
And it is going to be interesting to analyze this because what we find out on the face of it is not quite actually how it behaves.
We are going to spend most of the lecture today on understanding the dynamics of circuits that look like this and to see if we can build some fun and interesting circuits and systems based on this kind of positive feedback.
It is positive feedback because I am feeding back a portion of the output to the positive input.
And you should be able to stare at this and already begin to intuit what should happen to this.
Let's think about it.
This is zero.
Remember, with positive feedback, the famous v plus is equal to v minus method doesn't apply anymore.
Let's apply very simple analyses.
If this is zero, let's say for example that this output tends to go a little bit more positive.
This output, due to some noise or perturbation, tends to go up a little bit.
If that goes up a little bit then because of feedback this node tends to go up a little bit.
If this node tends to go up a little bit this exacerbates the positive input here and this one goes cachunk, whacks into the positive rail.
Let's take the other point of view and look at it intuitively.
What if this one tries to droop a little bit?
If it droops a little bit then the input at the plus terminal droops a little bit.
If that tends to go down a little bit, that makes the output droop further and it goes and hits into the negative rail.
I can see that this circuit wants to hammer into the positive rail or hammer into the negative rail because of the positive feedback.
It is like if you give incredibly positive feedback all the time, and by positive feedback I mean feedback encouraging the child to do whatever the child is doing.
It could be if he does bad stuff you give a lot of positive feedback or good stuff you give a lot of positive feedback then you are guaranteed to have a very good child or a very bad child.
You are not going to have anybody in the middle.
Same way here.
By giving positive feedback you're going to drive this into the positive rail or drive this into the negative rail.
Now, I am going to analyze this in two steps.
First I am going to analyze this using a method you've seen before which is replace the op amp with its equivalent circuit and analyze it statically.
And by analyzing it statically we are going to show that the simple static analysis will yield the following expression.
I put this in quotes, well, for a reason you will see shortly.
When I apply a plain and simple static analysis here is what I find.
Let's go ahead with the analysis and see what is basically different about these two.
And, first of all, I will confirm for you that our naive analysis we have seen so far will give rise to that expression.
So, let's go ahead and analyze that circuit.
And to analyze that circuit what I will do is replace the op amp with its equivalent circuit.
If you remember the op amp is characterized by the following circuit, A times v+ minus v-, vOUT.
This is the equivalent circuit of my op amp.
And let me just impose that external circuit on this op amp.
I have grounded my v- terminal.
My v+ terminal goes through a resistor and a supply, the v into ground, it's the resistance R1.
This terminal goes to the output through a resistor R2.
So, this is the equivalent circuit.
And I can apply the same good-old techniques I have learned about all through this course to this circuit and see what vOUT looks like.
Very simply, vOUT is this expression here A times v+ minus v-.
And because of my ground connection v- is zero.
Then let me go ahead and replace v+ with the voltage that relates vOUT and vIN.
What is v+?
v+ is simply the current through this part of the circuit, the current flowing here times the resistance R1.
That gives me the drop across R1.
And to that I add vIN and that will give me V+.
And then of course I multiply this by the gain here.
So, let me write down that expression.
The current through this is simply vOUT minus vIN.
That is the voltage drop between these two points.
I divide that by the resistance R1 plus R2.
That gives me the current flowing through here.
That times R1 is the drop across resistor R1.
And to that I add vIN and that gives me the voltage v+.
So, this is v+.
That is simply vIN plus the drop across the resistance R1.
Let me shuffle things around and put all the vOUT terms on this side here.
I get a 1+ for that vOUT and let me move AR1 divided by R1 plus R2 to the left-hand side.
And I pick up a minus sign.
So, I get AR1 divide by R1 plus R2.
I pick up that.
And on the left-hand sign I end up with vIN, and my vIN here is a function of the vIN that I have here.
I have an A multiplying both the vINs.
And then I get a one for this vIN here and there is a minus sign, so I get a minus R1 divided by R1+R2.
That is the expression that I have.
Let me go ahead and simplify that a little further and move this whole thing down here.
That gives me my expression as a function of vIN.
What I will do is, let me continue here.
vOUT=vIN A(1-R1/(R1+R2)).
By the way, you may be wondering why I am going through so laboriously what is seemingly a very simple exercise.
The reason I want to do is it I want to very carefully show you that the result produced by this exercise is exactly that.
No magic here.
No cheating.
We are going to get exactly that.
And then stare at it and say huh, how did that happen?
And then we are going to try to figure out how it actually behaves following that.
I divide this by 1-AR1/(R1+R2).
And by now you should be familiar with the technique of ignoring small numbers when I have a big number next to it.
So, AR1/(R1+R2) can be very much larger than one because A is very large.
So, I can ignore my one there.
And then what I am going to do is multiply the numerator and denominator by R1+R2.
Oh, this A and this A is going to cancel out.
This A and this A will then cancel out.
And then I multiply the numerator and denominator by R1+R2, so this R1+R2 vanishes.
I get R1+R2 here.
R1+R2 minus R1 is simply R2.
And then down here I get a R1 and then I have a minus sign out there.
Notice that vOUT we have found to be equal to vIN R2 divided by R1.
That is not wrong.
That is correct.
Technically that is correct.
But you will see in a few seconds that in practice that that's rarely what you are going to see happen.
And we will try to understand why that is so.
What we have done so far, if you stare at these two panels here, first of all, we know that the inverting amplifier has the expression for vOUT up there.
And through this laborious exercise we have also shown that even with positive feedback, if I take a static view of the circuit -- If I take a snapshot of the circuit and simply analyze it as a static circuit, I get the same expression vOUT.
But what we are going to do is when I explain to you that look, a small perturbation in vOUT is going to drive the op amp to the positive and negative rail, that is where the insight begins to show.
That if everything were magical and I could somehow exactly keep things just so that will be true.
I will be able to build that positive feedback circuit where the output is equal to R2/R1 vIN.
But remember even the slightly amount of perturbation is going to send the op amp scurrying off to the positive rail or the negative rail.
How do we analyze that?
How do we analyze the behavior of a circuit that based on a small perturbation begins to move one place or another?
We want to analyze the dynamics of the op amp.
And to analyze the dynamics what I need to do is give you a slightly more detailed view of the operational amplifier.
If the operational amplifier is not moving instantaneously between the plus and minus rail, I need to give you a more detailed model that encapsulates the behavior of the op amp.
And so let me do that.
If you want to study the dynamics of an op amp -- By dynamics I mean how an op amp moves as I perturb the input or the output and so on.
To capture the dynamics of the op amp we build a slightly more involved circuit, so v+ and v-.
This is what we've seen before, two terminals and dependent source that amplifies the difference input here by a large amount.
Instead what we are going to do here is something slightly different and interpose the following circuit in the middle here.
This is a model of the dynamics of an op amp.
We are going to impose a small RC circuit in here.
This is R. This is C. And I am going to call the voltage across the capacitor v*.
Notice what I have done is rather than say this is Av+ minus v- I am breaking it apart in two dependent sources, the first dependent source, which is simply v+ minus v-, and there is a RC time constant surrounding it and then here I simply add on my gain Av*.
Notice that if it turned out that the resistance here, for example, was zero then v+ minus v- would appear across v* and this would be A(v+ - v-), what you have seen before.
It is always good to take a look at circuits and look at what happens when some component goes to an extreme value.
This would give you your basic op amp circuit.
What I would like to do next is analyze the following circuit to understand how positive and negative feedback work together.
And by understanding that then be able to explain how a positive feedback circuit works or a negative feedback circuit works.
Here is what I will do.
This part simply corresponds to my positive feedback circuit, R2, R1.
So, that is my positive feedback circuit.
And I will do the same thing on this side.
All I am doing is applying both a positive feedback through R2 and R1 and negative feedback through R4 and R3 and representing the dynamics of the op amp and then standing back and ee, all right, let's see what happens to you.
So, I am sticking positive feedback, negative feedback, the dynamics of the op amp here and let's see what happens.
What I would like to do is impose this circuit on top of this op amp model.
To save myself some effort, let me just go ahead and modify this circuit directly.
I get an R2 here, an R1 here, and then up here I get an R4, R3 here.
The math is going to be just a little bit grubby but the result is actually pretty spectacular.
So, all I have done is replace the op amp with its internal circuit out here.
And now we are going to take a look at what happens to op amp dynamics when there is a small perturbation.
Let's develop an equation of this circuit containing a capacitor using techniques that we already know.
Just to give you some insight into what you're going to see, notice that if I make a small perturbation in the voltage across the capacitor, let's say I make a small perturbation to the capacitor voltage let's say by applying some initial condition kind of thing onto the capacitor.
Then let's say that the output changes to some value K. So, the change on the capacitor must have been K divided by A.
And what you are going to see is what happens to the op amp when the initial condition on the capacitor is such that this output gets perturbed to the value K. Let's write an equation for this little circuit and see what happens.
Recall our goal was to understand what happens when I perturbed the output a little bit.
Here I perturbed the output such that its value goes to K. And I can perturb the output by changing what happens at the capacitor.
Let me write the equation for this circuit now and then to understand what happens to this capacitor circuit if I let go after giving it a small perturbation.
What I am going to do is let me start by writing the good old equation for this little circuit here.
And that equation is simply the voltage here v+ minus v- equals the voltage across the RC.
So, v+ minus v- will be equal to the voltage drop across the resistor plus that across the capacitor.
The voltage across the capacitor is v*.
The voltage across the resistor is the current through the capacitor C dv*/dt times R. So, v* plus RC dv/dt is equal to v+ minus v-.
RC dv*/dt plus v* is v+ minus v-.
You have done this millions of times before, but yet again.
This voltage here is equal to the drop across these two, and the drop across these two is v*, the drop across C, plus the current through the capacitor C dv/dt times the resistance R. Or you can apply the node method as well and get the same expression.
Now, we also know here that vO divided by A is v*.
I can go ahead and replace this guy here, v* by vO divided by A. RC/A dvO/dt.
Recall, I want the dynamics of vO so let me just get an expression in vO.
So, I get vO divided by A plus v+ minus v- equals.
Now, I want an expression in vO, an equation in vO, so I need to express v+ and v- in terms of vO.
What are these expressions?
The expression for v- is vO and this voltage divider, so it's vOR3/(R3+R4).
And just for simplicity, let me call this some constant gamma minus.
This is some fraction R3/(R3+R4).
And let me call that fraction gamma minus.
Similarly, v+ is vO R1/(R1+R2).
And let me call that gamma plus.
All I am doing is replacing v+ and v- in terms of vO.
So, effectively, what I have here is v+ is some fraction of vO.
That's the best intuitive way of thinking about it, some fraction of vO.
And v- is some fraction of vO as well.
And I just stick these.
I now have an expression in vO.
Don't get psyched by gamma plus and gamma minus.
Simply read this as if it is an F1 and F2 if you would like.
So, vO times some fraction minus vO times some other fraction.
I am feeding back some fraction of the output to the positive and to the negative terminals.
Then, just moving things around a little bit, dividing throughout by A divided by RC.
So, I divided by A divided by RC.
Plus vO divided by RC.
And what I am going to do here in a second, vO gamma plus minus gamma minus.
And I have multiplied by A divided by RC throughout.
Finally, collecting all the vO terms I get vO times one divided by RC plus A divided by RC.
I got a plus sign here so I will just reverse these two guys in there, gamma minus minus gamma plus equals zero.
All I have done here is simply grunged through some math to express this equation in terms of vO.
And just to make it even simpler, I will just replace this thing by one divided by T, much as we did for first order equations.
What I end up with is dvO/dt+vO/T=0.
Despite all the grubbiness, I end up with something that is very, very familiar to all of us.
I went through a bunch of gyrations to substitute for v+, v- and v*, but at the end of the day I got the simple expression which was dvO/dt+vO/T=0.
Where capital T is the time constant of the circuit, and the time constant of the circuit relates to the expression in there 1/RC+A/RC(gamma minus - gamma plus).
The gamma minus and gamma plus are the respective portions of the output fed back to the negative input and the positive input.
Now, as we all know, based on very simple intuition that we can completely predict the behavior of a first order of an RC circuit once we know what the initial condition of the capacitor is and once you know the time constant.
That's it.
We know, we are masters at the fact that the capacitor is going to behave like this.
It is going to be exponential.
And I do know that the time constant capital T. What's here?
It is simply the initial condition.
There is no drive input.
I am not driving this with any input here.
There is no input drive anywhere here.
This is simply the natural dynamics of the system.
And, recall, I start off with bumping the capacitor voltage such that the output starts off being K. That is it.
You should be able to write down this expression and the form of the response simply based on this.
So, this is what I bumped up the output to be by perturbing the capacitor voltage.
My output response based on this equation is going to look like that.
Let's try to understand what that means.
It is actually quite a lot of fun.
How do we plot that response?
You all learned that the way to plot the response is plot the initial value, plot the final value, and go cachoock, right?
It's pretty simple.
I am going to start at K. I know that.
I am going to start at K and I am going to go and find out what the steady state value is.
Here is where the interesting stuff comes in.
The final value on the capacitor depends a lot on whether T is positive or negative.
In my RC circuits that I looked at what was T?
In the very simple RC circuit we looked at what was capital T?
What was the time constant?
RC.
This was RC.
This was a positive quantity.
When capital T is positive my output is going to look like this.
When T is positive.
And T is positive when this expression is positive.
And if A is so large that I can ignore the 1/RC term, if A is very, very large and I can ignore the left-hand term here then T is positive when gamma minus is greater than gamma plus.
So, when gamma minus is greater than gamma plus, I have a stable circuit, this is the good-old stuff we have seen before.
Now things begin to make sense.
Intuitively, what am I saying here?
All the gammas and other pieces of crapola aside, what am I really saying here in English?
What I am saying here is that if the portion of the output fed to the negative input is greater than that fed to the positive input then I have net negative feedback.
I have net negative feedback.
I am feeding the output back to both the positive and negative inputs.
And if my negative input has a stronger effect then I am going to see the op amp output decay down to a value that I expect which is going to be zero.
Notice that since I am not applying any input here, I expect the stable point for this to be output going to zero.
I don't have any input there.
Let's take a look at another situation.
What happens when the opposite is true?
What happens when gamma minus is less than gamma plus?
When I feedback more, what happens when I do this, when gamma plus is greater than gamma minus?
The opposite is true.
This means that I am feeding back more to the positive input.
A bigger proportion goes to the positive than the negative.
What happens then?
Then what happens is capital T becomes negative.
We cannot see this happening on the RC circuit because capital T is equal to RC, but here we have a more complicated circuit and capital T can go negative.
If capital T goes negative then this whole thing in the exponent there goes positive.
If that goes positive what should the output look like?
It should take off into never-never land.
There we go.
I start off at zero and a make a small perturbation, and the output should go as t divided by capital T. The dynamics of this it goes berserk, so it is net positive feedback.
This is called a stable situation.
This is unstable.
What happens when capital T goes to infinity?
When capital T goes to infinity, spend five seconds thinking about what it means physically.
What does it mean for the time constant of an RC circuit to go to infinity?
That means that your R and C are very, very, very large.
That means that circuit is going to be very, very sluggish.
Think elephant.
A big time constant.
I want to move a leg.
It takes a while to do that.
Think big.
Big time constant.
So, everything is going to happen really slowly.
It's like moving in molasses.
Big time constant.
Everything is going to happen really, really slowly.
If gamma minus is greater than gamma plus with a huge time constant it is going to look like this.
And the output is going to look like this.
I make T even larger.
All right.
It is going to like this.
I make these so large that T tends to zero, T tends to infinity in which case I get this situation.
The output goes dah.
OK?
Very slow.
Very lethargic.
Big time constant.
T tends to infinity.
And so if this is stable, this is unstable, this is called corresponding neutral.
And there is a mechanical analog to all of this.
You can show that this situation is akin to let's say I had a physical well of the sort and I had a ball in there.
I let the ball go.
Then the ball will come down here and settle down in a stable state.
Any small perturbation of the ball will get it to come down and settle down here.
The unstable situation is this situation where I have a ball sitting up here where any small perturbation will get it to zip down to a positive rail or to a negative rail.
So, this is an unstable equilibrium situation.
And exactly the reason we got this analysis in the static situation is that this can happen.
If I do this circuit here and don't perturb it then I could get the output sitting at zero, but the slightest perturbation, boom, it is going to fall down or go up.
What about the neutral equilibrium state?
That can be modeled like a table top and the ball is here.
It doesn't matter where you go.
There you are.
How many people saw the Buckaroo Bonzi thing?
Possibly well before your time.
OK.
I have this table here.
No matter what I do to it, it just goes and settles down where it is, and that is neutral equilibrium.
But what this gives you is a fun view of the dynamics of the operational amplifier as I make small perturbations to it.
And the even more interesting thing here is you have the tools based on your first order RC analysis to analyze the dynamics of a simple op amp circuit.
OK, so much for theory.
Now let's get to some action here.
All right.
Fine.
That is really pretty, good and so on, but what can you do for me?
What good does this property do for me?
What can I build?
What we will do is look at the op amp circuit and focus on the situation where I have net positive feedback.
In particular just look at this circuit with R1 and R2 and send both to infinity.
So, I have no negative feedback and I ground this terminal here and take a look at what happens to a circuit with positive feedback and see if I can build some interesting circuits.
What you are going to do is build on a circuit called the basic comparator.
What is that?
If I have an op amp that looks like this, and remember a VS rail and minus VS supply there, this is v+, this is v-, I can build a very basic comparator by doing the following.
All the circuits I am going to show you are going to build on this basic little circuit.
What I am going to do is consider applying an input to the v- terminal, applying some sort of an input and taking a look at how the output behaves.
So, I apply some input vIN.
And if I just do that, if this is v+ minus v- here then I am going to get something that goes like this.
That is when this is positive here then this guy is going to go to the VS rail and this guy is going to go to the minus VS rail.
In terms of the, if I plot the same thing, in terms of vIN, and this is vOUT, if I plot the thing in terms of vIN then notice that as vIN increases this guy should go to a negative rail.
So, in terms of vIN it looks like this.
What this says is that as the input becomes more and more positive applied to v- then the output goes to minus VS, and if the input becomes more and more negative then the output goes to VS.
This is what is called a very basic comparator circuit.
It compares the two inputs and goes up if the input is in one direction and goes to the other rail if the input is in the opposite direction.
So supposing I feed this- I can plot this is a function of time.
Let's say I plot vIN.
Let's say I feed some vIN here.
Let me just call this.
I feed some vIN to this circuit here, then what do you expect the output to look like, the output wave form?
For all positive vINs the output is negative.
So, my output vO is going to be negative as long as vIN is positive.
And when vIN becomes negative this one shoots up and behaves like this.
This is minus VS. That is plus VS.
This is my input vIN.
Then this guy is going to be my output.
As vIN is positive output slams to the negative rail.
When vIN becomes negative the output slams to the positive rail.
So, that is quite nice.
And so such a circuit is pretty useful to me.
Let's say, for example, I want to build a little digital circuit that is fed ones and zeros.
I can use a comparator to turn my vIN voltage into a sequence of ones and zeros.
When vIN is positive I produce a zero and when vIN is negative I produce a one.
I can get this one, zero, one, zero sequence coming out corresponding to the values of vIN being greater or less than zero.
Now, one problem with something like this is that this circuit can be quite messy in the following situation.
Suppose I superimpose a small amount of noise in vIN.
In particular, let's say that I have some amount of noise on vIN.
I get a bunch of noise sitting around here.
What happens is that at this point where the value goes negative, I do bump up.
But when for a second I have my input going above zero again -- -- this output comes down again and out here it goes up again.
I get this nasty behavior at the point where the input is around zero.
When the input is around zero, the input is meandering around zero because of noise, I get a huge amount of up and down glitches on the output.
That's not very nice.
And we will do a little circuit that attempts to fix that little problem.
What we are going to do is use positive feedback.
And I am going to build you a circuit that shows that we can eliminate this for small noise on the input.
So, let's build the following circuit.
So I still feed vi to the negative input, but this time around I give it some positive feedback.
So, I give it some positive feedback.
And what I am going to do is feedback a portion of vO to the positive input.
This is positive feedback.
And, in particular, let's assume that VS equals 12 volts.
And to the negative one I connect -VS.
This guy is going to go between 12 and -12.
And correspondingly because these two are equal this one is going to go between 6 and -6.
This is going to be a 12 or -12.
Remember, the top rail and the bottom rail.
And this one is going to be a +6 or -6.
And let's understand how this circuit works when I apply an input vIN.
Let's start by saying that assume my input is zero for a moment.
And let's say my output starts off being 12 volts.
The output is 12 volts then the input here is going to be 6 volts.
In this case v+ is going to be 6 volts.
The output is 12, v+ is going to be 6 volts.
And my circuit is sitting out there doing nothing.
Now, this started off being zero.
Let's say vIN increases.
As vIN begins to increase what happens?
Well, nothing until vIN reaches 6 volts.
Since this is 6, vIN has to go up to 6 volts, has to equal this voltage before I can flip the circuit.
What happens when vIN is greater than 6 volts, if vIN goes above 6 then I have more voltage on a negative terminal than the positive so the op amp flips its state.
And vO gets to -12 volts.
When vi goes above 6, vO gets to 12 volts.
And what does v+ go to?
In this state v+ goes to half of -12 which is -6 volts.
Now, this guy is sitting at -6 and this guy is sitting at -12.
If this one keeps rising nothing happens, so output can stay at -12.
So I am pretty safe.
Then let's say v begins to come down.
As v begins to come down, does anything happen when v gets to 6 again?
If v is equal to 6 what happens?
Nothing because this is at -6 now.
So, there is still a huge net negative voltage here from v+ to v-.
And so therefore I sit at -12.
Oh, well, I keep coming down until I reach -6.
When I reach -6 here these two become equal.
And what happens when this becomes less than -6?
v- becomes less than -6.
If this one goes below this voltage, this is -6 and this is -7.
There is a net positive voltage between v+ and v-, so this output swings to the positive rail like so.
We will spend a lot more time on this in the next few minutes to really hammer the point home.
What is interesting about this is that even though the moment vi became more than 6, I swung to the positive rail, and then I had to go all the way back down to -6 before I could change state.
I had to go way down before it could flip again.
How can we make use of that?
Well, let me draw you a little vi versus vO diagram and then talk about how that can be useful to us.
This is vi, this is vO, this is zero.
Let's say this is 12, -12, -6, +6.
Let's plot that on the screen and see what it looks like.
As I told you, the output was at 12 volts to begin with and my input was at zero.
So, my input kept increasing.
When the input hit +6 what happened to my output?
My output swung down to -12.
As the input kept increasing nothing happened.
This was step one, this was step two, step three.
My input kept increasing and output stayed at -12 volts.
Then what I said was well, let's bring the input down.
So, my input began to go down, step four, became more and more negative.
Nothing happened until I reached -6.
When I reached -6 I swung positive, step five.
Again, one, two, three, four, five.
I am going up here.
It came up here.
And nothing happens until I reach -6, but at -6 boom, I switch to the positive rail.
And as I get more and more negative I stay there.
Then again, as I start increasing again, nothing happens until I reach +6.
Think of that as your seventh step.
What is spectacular about this is that I seem to have a circuit that now has some knowledge of where it came.
If it is coming from here it switches at +6, but if it is coming from here it switches at -6.
So, there seems to be sort of a lag in the behavior of the circuit or some memory property in the circuit.
This kind of behavior is called hysteresis.
The word comes from magnetic circuits where, or rather elements that you're trying to magnetize.
Where if you take a magnet and move it over a piece of metal it may leave some residual magnetism in it.
And, in the same way, that is called hysteresis.
Same way here.
As the voltage increases it seems to leave some residual in the circuit so that it effects when it shifts.
The good news with this is that now, if I take the same kind of noisy wave form that I had before and do this -- If this is vi then what is going to happen is for vO I am going to be negative at this point.
Nothing happens here because I have to get to -6 or +6 before something happens.
Out here I get to -6 and I switch state and go up to +12.
And then this one comes up above -6 very slightly out there.
Nothing happens because the next change will happen only when the input goes to +6.
So, if eventually the input gets to +6 and then I am going to change state again.
It is actually a really cool property and something that is completely non-obvious.
In the last 30 seconds let me show you a quick demo.
And, based on this property of hysteresis, I have actually built a little circuit.
Let me do that first.
Notice here that I am showing you the input on the X axis vi and vO on the Y axis.
Notice how the output switches at +6 volts and switches at a -6 volts to +12 or -12.
That's the hysteresis property.
And we can actually use this property to build a clock circuit, which is on page 9, build an oscillator that sits there and oscillates by itself.
And you will see details of that in recitation tomorrow.
All right, let's get moving.
Good morning.
Let me take a quick poll.
So, how many of you have completed Lab 4.
Completed Lab 4?
Wow, that's great.
So, how many people have begun Lab 4?
OK, well that's good.
I won't ask the last question.
OK so, well I hope you're having fun with this lab.
Lab 4 was designed to be almost like a mini-project.
And, it sort of ties together a lot of the content of the entire course.
And, it's not unlike the kind of systems that people design in industry, in systems that go into a variety of devices like, say, for example, digital CD players and stuff like that.
A lot of mixed signal stuff goes in.
OK, so today, I'm going to continue with our discussion of energy and CMOS.
CMOS will be a new topic that I will introduce.
So, the last lecture, we spent a fair bit of time talking about energy, and how to compute the energy of our inverter.
So, let me start from where I left off, and I've given you a couple of extra pages of notes today just to sort of tie it to the previous lecture.
Right now, I'm going to start off on page three.
So, what we saw last time was an inverter of this sort, Vs, VIN, and we said, let's study the situation where this inverter was driving a load capacitor, C. Where did this load capacitor come from?
Well, this inverter could be driving one, or two, or three, or four other larger gates, OK?
So, this C is lumped value of the gate capacitances of all of those inverters.
This may also include some component due to wiring capacitance and stuff like that.
So, for an inverter like this, we showed in the last lecture that the formula for the average power was, so this was a static power independent of frequency, and this was called dynamic power, and it had some bearing, it's related to the frequency at which you clocked your circuit.
So, this was related to standby power, and this to dynamic.
So, what I also said is that I gave you a bunch of numbers so you could compute the power consumption of a chip that included 10^8 gates, 100 million gates, and at a frequency of 1 GHz, and a bunch of other numbers.
C was given to be 0.1 femtofarads.
Femto is 10^-15.
So, F was 10^9.
VS was 5V, and for these numbers, if you plonk them down in something like this, for 10^8 gates on a chip, the average power would be 10^8 times these two.
So, this would be five squared, which is 25, divided by twice.
RL was given to be 10 kilo-ohms, so, twice, 10^4.
And here we had CVS^2.
So, C was 10^-16, 0.1 femtofarads.
Vs^2 was 25, and F was 10^9.
So, if you commence through the numbers here, what you end up getting is something that looks like this, 10^8 times this guy here.
This is 1.25mW plus this guy ends up being 2.5 microwatts.
So, this should come as a bit of a shocker.
If I take 1.25mW, and multiply that out by 10^8, this says that each gate suffers a standby power loss of 1.25mW.
So times 10^8, I get 125kW, and this guy yields 250W.
OK, the 250W is manageable.
It's still high, and just so you don't think that this is unreasonable, when the Pentium 4 first came out, it was consuming 170W of power.
OK, you should see the heat sinks on there.
There's actually a huge heat sink with a fan built into the top of the heat sink.
OK, today it's down to more reasonable numbers like 100W and so on, but when it came out it was in this range.
So it's high but not unreasonable.
But this, of course, is totally wacko.
OK, imagine carrying a laptop around, and the sucker is blowing 125kW.
That'll be fun.
So, clearly there's something wrong here.
What this is saying is that this gate here consumes 125kW, there are 10^8 of these on a single chip.
OK, so we clearly have to do something about this, otherwise the semiconductor industry would fail.
So, anybody have any ideas?
What do you think you might do here?
What do you think you might do to this inverter to make this look better, to bring it down?
What can I do?
Anybody?
Any ideas?
What do you think?
Well, the problem is that if I look at this 125kW, well, there's a VS term here and an RL term here.
So, I can increase RL.
OK, I can make RL four times or eight times as large.
That'll bring the power down somewhat.
Can anybody think of any problem with increasing RL?
If I make RL really, really large, will I run into other problems?
Yes?
Exactly, the slowdown of the inverter.
Remember, the rise time of the inverter depends on how quickly I can charge this capacitor through RL.
So, if I make my RL really large, I will consume less standby power from hundreds of kilowatts to merely tens of kilowatts.
But my gates will run as slow as molasses.
So, clearly that's not a tradeoff I would like to make.
So, I can reduce my voltage to maybe a volt.
But that just reduces it by a factor of 25, VS squared.
So clearly, this is not going to work.
I have to somehow do something else, and that will be the topic of today's lecture.
Also, I will dwell for a moment on this term.
So, if you look at the spec sheet for the IBM's ASIC processor that we handed out, if you recall, we talked about power dissipation of 0.006 microwatts per MHz per gate.
OK, now you see where this is coming from.
Per MHz, that's because it's a multiple of f, the power.
Second is that it's per gate, so this is the power per gate.
So, as I have more gates, I just have that much more power dissipation.
It also says power supply voltage in the range of 0.7 to 1.3 right next to the power expression.
So, you can see why they tell you all of that, because both voltage, and the frequency, and the number of gates come into the power of equation.
OK, this really simple expression here, it's amazing how close this is to what people use for the dynamic power in chips.
OK, so as the next step, what I'd like to do is, this guy, what do we do about that?
OK, so we've taught you to build gates in a particular matter, but it's a non-starter.
So, how do we get rid of static power?
How do we get rid of static power?
OK, to do so, let's build up a little bit of intuition.
OK, so the intuition goes as follows.
So let's say I take my inverter.
Let me draw the circuit both on the on state and in the off state.
So, when VIN is high, when VIN is high, I get the MOSFET turning on and has a resistance, RON, and Vo is the output voltage.
Similarly, when VIN is low, so when VIN was high, Vo was low because RON is much less than RL.
So, this voltage was low, while here, when VIN is low, the MOSFET is off, and so I have an open circuit out here.
And because of that open circuit, the voltage here was going to be high because VS would simply appear there.
So let's tailor this and see if we can build up some intuition as to what to do.
So, when VIN is low, I don't have any static power being dissipated because I don't have a connection from VS to ground.
OK, the current, i, is zero.
And, VS simply appears at the output.
The reason this is so is have a switch here.
So when this is low, the switch opens up and cuts the path from power to ground.
This is a nice situation.
Here, when VIN was high, there was no switch that turns off.
Rather, I get a connection from VS to ground.
OK, so think about this situation here.
The insight here is, just imagine if I could do the following.
Imagine if I could somehow magically elevate RL to be a very, very, very large number, if I could make this so high as to make the power really low only in the situation when the input was high, OK?
So, imagine if I could do something like this.
Imagine I could open circuit this guy, RON, so when VIN was high, if I could, instead of having an RL here, what if somehow I could make this RL become infinity?
OK, so in this case, output VO would be low.
OK, I get many benefits by doing this.
One benefit is that, look, I have opened this switch here so I don't have any standby current.
OK, the standby current is zero.
The second benefit is that my output gets dragged down to ground, OK?
Out here, my output was VS multiplied by RON divided by the sum of these two.
Out here, I have a direct connection to ground, and nothing to the power supply, VS, and so therefore I have a nice, solid low.
So the question is that, can I get this situation?
OK, that is a key insight.
So, imagine that somehow, when this was high, I could get this to open up, much like when this was low, I got this to open up.
OK, so think about it.
So, the intuition is that what I need instead of a resistor here, what if I have something like the MOSFET that I have here?
So, I have a MOSFET here that turned off when VIN was low.
OK, what if I did the complementary thing?
What if I put in some kind of MOSFET here that would turn off when VIN was high?
OK, so, much like the MOSFET turned off when VIN was low down here, imagine if I could find a device that could turn off when VIN was high?
OK, this would be on, but this would be off.
So the behavior of this device would have to be complementary to this device.
So, we need some sort of a switch to introduce this new, little MOSFET device with slightly different properties, let me quickly review for you the properties of the MOSFET that we know about, so our N channel MOSFET, also called the NFET, this is what we've been seeing all this while, is drawn like this.
I have a gate; I have a drain; I have a source.
And this guy is on when VGS is greater than or equal to VT, OK, and off when VGS is less than VT. You saw this before, OK, nothing new here.
So, what I need is a device that behaves in a complementary manner.
OK, so the device is a P channel MOSFET.
By the way, I must point out, till about 1983-84 until the early '80s, that's exactly pretty much how chips were designed, OK, using an NFET for the switch looking down here, and a variety of different kinds of devices to be used as resistors.
OK, that's when technology began moving towards this new kind of technology I'm going to talk about, and that dramatically reducing the power consumed.
And, the P channel MOSFET was created, and this guy's called the PFET.
It's a complementary device that looks as follows.
OK, the difference here is that, to show this is complementary, I'll put a little circle here.
It has a gate.
Just to make things a little clearer, flip the drain and source terminals, and this guy is on at a distinguished threshold voltage of this with the NFET device, let me put an N here to say that this is the VT for the N channel device.
And for this guy, this guy came on when VGS was greater than some voltage.
So, VTN could be, for example, one volt.
So, VGS was more than one.
This turned on.
In this case, I wanted this to turn on when VGS is some value which is lower than, or much lower than, the source voltage.
OK, so this guy turns on when the gate voltage is higher.
This guy should turn on when the gate voltage is significantly lower than the source voltage, just the complementary behavior.
OK, so when VGS is less than or equal to VTP.
And in this case, the threshold voltage for the PMOS device, say, just as an example, maybe -1V.
So this means that if the source is at, say, 5V, OK, then this device would turn on if the gate, for example, using that example was less than 4V.
So, this is five.
If the gate fell below 4V, this guy would turn on.
In this situation, remember, if this was at zero, the gate would have to be greater than 1V to turn on.
In this situation, the gate has to be less than 4V if the source was at five to turn on.
And, it's off.
OK, so this is a complementary device that I postulate that behaves in a complementary manner.
So, the gate voltage rises, this guy turns on, and in this situation, when the gate voltage drops below the source voltage, this guy turns on.
OK, so when there's a rising guy that turns on in this particular situation when it falls, the gate turns on and shows some resistance.
In this case, the resistance would be RON.
And to show that it's N channel, let me say N. And in this case, the resistance, when it turns on, would be RONp to represent P channel.
OK, so now consider the following circuit for the inverter.
So, instead of my resistor, I put a complementary device, OK, and that's it.
So all I've done here is replace my resistor with a MOSFET that behaves complementary to the N channel MOSFET.
So this is my gate, my drain.
This is my source, my gate, my source, and my drain.
OK, and this guy is called a pull up, and this guy is called a pull down.
OK, and the reason is that this guy pulls the output to ground when it's turned on, while this guy, when switched on, will pull this node up to VS.
So, I pull it down or pull it up based on when the VIN is high or low.
So, let's look at the two situations.
So, let's say, as an example, my VS is 5V, and let's say VIN in one situation being 5V, and another situation being equal to 0V.
Let's draw the equivalent circuit in both these cases.
So, when VIN is high, I have my usual circuit.
When VIN is high, this MOSFET, as before, when VIN is 5V, the N channel MOSFET below is turned on, and so I have an RON resistance here.
But remember, VIN is 5, and VS is 5V, then the voltage across the source and the gate of this P channel FET is now equal, five and five.
OK, so this one would turn off.
And that's the circuit that I get.
The output is suitably low.
In this situation, if VIN is zero, what happens in this situation?
Here's my output.
If VIN is 0V, the lower device turns off.
This is zero.
This is zero.
This guy turns off, and that's the situation for the N channel MOSFET.
How about this guy here?
What happens here?
This is at 5.
So let me just, this is at 5V.
OK, and VIN is at 0V.
OK, so therefore, the GS of this is -5V.
If this is zero and this is five, G, source, and drain, GS is -5V, and -5V is significantly less than the threshold -1V in our example.
So, this one will switch on.
And if this one switches on, what I end up getting is RONp out there.
So, when this one kicks in, it pulls the output high and VO goes high.
So, all I've done is replaced my resistor with a complementary device, which switches off when the input is high, and switches on when the input is low.
And the beauty of this is that at no point, assuming all the devices are ideal here, at no point do I have a short circuit between the output, do I have a current path from the output to the ground from the supply to ground, OK, I have this turned off or this turned off.
So, this type of logic involving a PMOS transistor here, and the N channel transistor here is called CMOS logic for, OK, it's called complementary MOS logic.
That's what CMOS comes from.
OK, so I'm sure you've read in a number of places that most digital chips today use CMOS technology.
It comes from complementary MOS, and complementary comes from the use of complementary transistors: N channel, P channel, turns on when high, turns off when high, turns off when low, turns on when low.
OK, that's exactly complementary to each other.
OK, so what you've seen here has been the workhorse of the digital industry for the past two decades, 20 years, CMOS logic.
OK, and even the most advanced chip from Intel has an inverter that looks exactly like that.
OK, if you count all the inverters in the universe today, I would say a significant fraction of those look exactly like that, no difference, just so simple.
So, the key with something like that is there is no path from the power supply to the ground, and so by that model, I did not consume any standby power.
OK, my standby power in that idealized model is zero.
So, let's compute P. So, what is P dynamic?
Let's use the method that we adopted in the last lecture, and draw the equivalent circuit, and compute the power.
OK, so I'm going to model the following situation, and assume that I drive a capacitive load, C. OK, and as an input, as I did the last time, I'm going to assume I have some input voltage, VIN, that looks like this.
The cycle time, T, and the frequency is 1/t, and let me assume that this is T1, and this is T2.
OK, and I'm assuming that T1 and T2 are both much larger than the respective time constants.
OK, the time constants when, for discharging here, is C RONn, and here the relevant resistance is RONp.
The charging time constant is RONp times C. OK, so T1 and T2 are assumed to be much greater than these two.
So when you look at this, there's one other benefit besides the power benefit, OK, of using CMOS logic compared to using NMOS.
OK, it not only cuts out my standby power, but there is another significant advantage which is almost equal to the power advantage of this kind of CMOS technology.
Anybody have any ideas?
What's the advantage?
What does intuition tell you?
Is CMOS going to be faster or slower than NMOS?
Why?
That's right.
The key here is that the NMOS design I showed you earlier was relatively slow because it took me a while to charge up the load capacitor from RL.
In this situation, RL will become really, really small; it's RONp.
It's roughly the same magnitude as RONm.
OK, if so both of these on resistances are more or less equal and small, then the rise time will be of the same order of magnitude as the fall time, which makes this much faster than the NMOS.
In NMOS, my time constant was RLC, and RL was pretty large.
In this case it's RONp C, and RONp can be made to be very small because when it's switched off, the resistance here is infinity.
So, in this situation, if I assume T1 and T2 are much larger than the respective time constants, I can go ahead and draw my equivalent circuit.
So, here's VS.
So, for charging up, let's say this one is going to a one, or to a high.
So, I have VS going through a resistor, RONp, to a capacitor, and this thing is a switch.
So I have RONp, an ideal switch, going to a capacitor, C, this is my V out node, OK, so it's VS going through a resistance, RONp, an ideal switch, to a capacitor, C. That's a charging circuit.
For discharging, I have C, discharging through an ideal switch with RONn.
So, this situation, I have an ideal switch, RONn.
OK, so that's the equivalent circuit for something like this.
So, in this circuit, during T1, this guy's off, and this guy's on, on during T1, and off otherwise.
This guy is on during T2, and off otherwise.
OK, so just imagine, this guy switches on, this guy switches off, this guy switches on, this guy switches off, OK?
And remember, this is exactly the circuit I had analyzed last time in the last lecture, and the result given by v double asterisk.
And that result was simply average power being CVS^2f.
That's the exact circuit we used to compute the dynamic power, CVS^2f.
OK, so we're done.
And how did this come about?
This came about because the intuition here is that I'm charging up the capacitor fully, and then I'm discharging the capacitor through this other side, OK, and I'm consuming power, dissipating power, in these two resistances during charge up and during the discharge.
Half the power gets consumed during charge up, and half during the discharge.
So, I'd like to go back to doing a few numbers here, and taking a look at how, even with this expression, life can get pretty thorny as we go ahead into the next decade.
OK, so for our previous example, we assumed that 10^8 gates, F=1 GHz, C=0.1 femtofarads, VS was 5V, and I don't need RL anymore.
OK, why is it that I don't have any resistance component here?
I don't have it here because the power consumed by this circuit is independent of those resistances, provided T1 and T2 are long enough, are much longer than the two time constants, RONp C, and RONn C. OK, so I don't have RL in my equation anymore.
I don't have any standby power.
So, based on this calculation, the calculation I did up there showed that I had 2.5 microwatts per gate, and for 10^8 gates I had 250W for a chip with 10^8 gates.
So, I'd like to dwell on this, if you can move over to page eight in your notes, here.
Let me dwell on this for some time, and pontificate on a few things.
First of all, this number, as I said before, is high, but not a disaster.
OK, so you can't use this in laptops, but it's quite OK for a desktop or a server, and so on.
If you just go and put your ear to a pedestal computer, you'll always hear it making a sound, and that sound is because of a big fan that's inside it.
And, if you have a big enough fan, 250W is not such a big deal.
But, this is certainly a real problem for mobile devices.
For a laptop, this is unthinkable.
OK, so we have to deal with this.
The second issue is the following, that it's 250W for 1GHz.
Now, the fastest Pentium 4s that money can by today are, what, how many GHz?
What's the fastest Pentium 4 you can buy today?
What's that?
Does anybody have a 4GHz Pentium 4 here?
Oh, darn, you beat me.
Anybody have a 3?
3GHz?
A couple.
So, I have a couple of 3GHz machines, and our lab has a whole ton of them.
So, if Intel comes out with 4GHz machines today, they've been going up by about 1GHz roughly every year for the past couple of years.
And, within three or four years, you're going to see chips, microprocessors that are in the 5-10GHz range, OK, assuming that all other things stay equal, which of course they're not, but just to give you some insight here, if I clock these guys and build circuits that are ten times faster, I very soon go up to 2.5kW, again as I said, all things being equal which they're not.
But just to give you a sense, as I increase my frequency, so does the power consumed by the chip, OK?
So, I really have to do something here.
So, if I stare at this equation, CVS^2f, I want to increase f because people will buy computers if I have higher frequencies.
And, Intel has managed to use its marketing campaigns to pretty much convince consumers that high frequencies are a good thing.
OK, and whether they really mean anything or not, that's a different issue.
So, we've got this huge power for assuming 5V, OK, so it turns out that microprocessors, as they come out, newer and newer versions run at lower and lower voltages.
OK, they invent technologies that use lower and lower voltages, and go from VS 5V to, today, VS on the order of 1.5 to 1V, somewhere in that range.
So the moment you do that, you get a 25x reduction in power.
OK, so in going from 2.5kW, you would now come down to something on the order of 100W, which is, again, much more reasonable, again, all other things being equal.
It turns out that the capacitance of devices also changes as you go to smaller and smaller devices.
And, 100W is also pretty high, and still not good enough for mobile computers.
So, there are many, many other tricks that people use to get even lower powers.
One trick is to play games with the clock.
OK, what you do is, let's say for example in some computation you are not going to be using your floating point unit.
Or let's say I'm going to be using your integer adder unit.
OK, so what you can do is you can turn off the clock to those devices so that those devices do not even switch when they're not working.
OK, if I turn off the clock to a device, the device isn't even going to switch, it's just going to sit there in limbo without consuming any power.
It's equivalent to turning off both transistors.
If you turn off both the PMOS and NMOS somehow, OK, it's not consuming any power.
And by doing that, you can further cut down the power.
So, if you can idle some of your function units, it's called idling a function unit, idle a function unit for, let's say, half the time.
OK, you would cut down power by another factor of two.
We can idle, then, 75% of the time, come down to 25W.
So, those are the classes of tricks that people play.
I'm going to stop here and allow the underground guide folks to do the survey.
But, suffice it to say that the power discussion that I've gone through with you is a very high level discussion as to the real thing.
In real life, what actually happens is that there is a fair amount of standby power even for CMOS logic.
It turns out that although I don't have a path from VS to ground for my two transistors, it turns out that there are many leakage currents.
OK, currents leak through all kinds of places through the drain of the inverter, and so on and so forth.
And so, there is some standby power.
So, let me show you a quick demo while, I guess, the review handouts are going around.
And this shows the temperature of my CMOS inverter, and as I increase the frequency, you can just watch the temperature go up, and hopefully we'll blow this transistor.
So, I'm increasing the frequency as you can see on the side here, and higher frequency implies more power consumption, more temperature, OK, and hopefully you will see some smoke coming out of, OK, I think I blew the inverter.
So, the output is gone.
So, it's at 110 degrees there, and that blew it.
Sometimes we see smoke come out, but I guess today is not one of our lucky days.
OK, so let me stop here and have the underground guide folks go through the reviews.
All right.
Good morning.
Good morning.
So, we have some fun stuff for today's lecture, and as far as the final is concerned and so on, I'd like you to forget about anything we do today, absolutely.
So, get your mind to become a blank, and forget anything you hear in today's lecture.
So, what I'm going to show you today will hopefully completely blow your minds.
And I'm not talking about controlled substances or anything.
So what I'm going to do is show you a few things that behave completely and spectacularly differently than how you expect them to.
And, today's lecture is appropriately called -- OK.
So, we're going to violate the abstraction barrier here, and do some fun things.
And, the important thing to realize is that in all of 6.002, we have, after all, based on some assumptions we made at the beginning of the course like lumped matter discipline and so on, we have landed ourselves in this playground called the playground of 6.002.
And, within that playground, certain ground rules apply.
OK, and our entire course depended on those assumptions being true.
So, for example, the first assumption we made that brought us from Maxwell's equations to the lumped matter discipline was, or rather the circuit abstraction, was a lumped matter discipline.
And there were three tenets of the lumped matter discipline.
One is that the rate of change of flux was going to be zero within our circuits, not inside elements, but in the circuit itself, and second, the dq by dt was going to be zero outside the elements, and third, something we did not dwell upon in the course, but it's certainly present in the course notes is that the speeds of signals that we are going to consider are going to be much slower than the speed of light.
OK, so we're going to be working in a realm where we are going to be well slower than the speed of light.
OK, so starting with that, let me walk you through some examples and some fun stuff.
So, the first case is called the Double Take.
So, let me sketch out a small little circuit for you, and take a look at the expected behavior, and then show you what really happens in real life.
So, the first case, I have a voltage source, and what I'm going to do is make a transition from a zero to a one.
Think of it as a step input, and through a Thevenin like resistance, I want to feed it to a circuit.
The circuit will go to an inverter.
This node goes to an inverter, and goes through some other circuits within our own design here.
So, again, remember, a step input here, and this input goes through a Thevenin like resistance, or is applied to some other circuit elements.
So, if I apply a step here, what do you expect?
You expect that, so let me call that VI, and let me call that Vo.
So, if I plot VI as a function of time, and let's say this step input happens at t=0.
So let's say this is t=0 here, and let's say this is a 5V step.
So, I expect that this input here is going to go to, VI here, is going to go to 5V at t=0.
What do I expect at Vo?
At Vo, based on our circuit abstraction, I get a step input here.
I should get a step of some magnitude here, depending on what's connected in this direction.
And let's simply say that what's connected here is an inverter, and maybe other inverters at the other side.
So essentially, as far as this node is concerned, it's got some wires connected to it.
And at the end of the wires, it has an open circuit, an open circuit, for example, like the gate input of this inverter.
So what do you expect at V nought?
A step input here, and at V nought I see an open circuit.
OK, so I expect the same step at V nought: 5V.
So, that's what we've prepared you for, OK?
But, the fun thing that we're going to see, so this is what you expect, and I'll show you a little demo that is going to show you something very different.
What you're going to see is not this.
OK, you're not going to be seeing that.
Rather, I'm going to show you something that looks like this.
So, at t=0, I do see Vo looking like a step, and approximately halfway through, decides, ah, well never mind, and flattens out, OK, then says, oh, OK, and zoom, it goes back up to 5V.
So, it sort of does a bit of a double take up there saying, hey, what's going on here?
And zoom, jumps up to 5V, and then it's five as you expect.
OK, so this is some finite amount of time that looks like that.
OK, so try to understand what's going on.
So let me show you a quick little demo.
So that's the input VI.
OK, so that's the input VI that you expect, and I won't do anything to my circuit at this point.
And, go ahead.
So, let's see what happens now.
There you go.
So now, I'm showing you the output here at Vo.
So at VI, there's a nice little step, and at Vo, notice that I get something that behaves like this.
OK, and I promise you, nothing we've taught you in 6.002 prepares you for this.
OK, and as I mentioned at the beginning of this lecture, it would behoove you to forget about everything you learn in today's lecture for the next two weeks at least.
So what's going on here?
Any ideas?
Anybody?
Any thoughts?
So what's up with my circuit here?
It says, oh, OK, a step.
It starts off and says, oh, never mind, and then meanders along at 2.5V and then says oh, step, yes, I remember, and then boom, it jumps up to 5V.
So, any theories?
Any guesses?
Any wild guesses?
OK, so let me draw you a little bit more of a detailed circuit, and see if you can explain what's going on here.
So, the circuit that I've drawn there is not quite the circuit I have at least in terms of my wires.
So, what I have is something that looks like this, VI, and this is going to step to 5V.
I do have a resistance, R. This is Vo, this does go to an inverter.
But what is also happening is that I have a long wire.
OK, you see this guy here?
We had one of our union folks stretch out along the floor here.
We have a really long wire that connects to the Vo node, and there's also a long corresponding ground.
So, this wire is a coaxial cable that is used for Ethernet and such like.
It's got a core that carries a signal, and around the core is shielding that is the ground.
OK, so that goes a long way, and at the end, it is open.
OK, it's an open circuit at the end.
I haven't connected anything out there: open circuit.
So, you know, something's happening here that's making the circuit behave like this.
So, this is VI.
At Vo -- So at Vo I'm getting this funny behavior.
OK, so does anybody want to take the next piece of clues here, does anybody want to take a stab at guessing what might be going on here?
Yes?
Ah, we have a shill in the audience here.
So, the theory is that the step here, think of it as an electromagnetic pulse that goes from zero to five, and things in real life don't travel instantaneously.
So, there's something with a wave that flies down, and the wave goes to the end, flips, and then comes back, and then establishes the full voltage here.
So that is indeed at the root of what's going on.
And let me put it in layman's terms and then describe the details of what's going on here.
OK, so the way to view what's going on is that I have this long wire.
OK, in the very first lecture, I started off by saying wires are ideal.
OK, ideal wires are such that I can transmit signals on them.
Wires are small so that the propagation time of signals is inconsequential compared to the rise times and fall times of the signals of interest.
By having this really long cable here, I have clearly violated that assumption, which is the wires are really, really long here.
OK, and so I somehow need to model what the wire is doing to my circuit when I don't have a small wire.
So what actually happens, the way to view it is the following.
So, although this is a wire, to understand the mechanics of what's going on, I really have to model it much more accurately, OK?
And, the way to model a wire like this is that notice that every small element of a wire has associated with it some inductance.
OK, so let's take a small segment of the coax cable here.
The coax cable is a small core surrounded by a metallic shield.
OK, that's a ground.
And so, when I have a wire surrounded by a metallic shield, that also has the capacitance, OK, inductance and capacitance.
So this small segment can be modeled as a really small inductance, and a really tiny capacitance.
Similarly, the next segment can be modeled as a tiny inductance and a capacitance.
There is also a resistance here, but let's assume that the resistance is zero for our model, and also the parallel resistance is also infinity.
OK, so it's an inductor, capacitor, and really the situation that I have is not a pair of ideal wires, but really a really, really small inductance, and a small capacitance in parallel.
So, it's more of a set of distributed elements that I have here.
Notice that in my lump circuit abstraction, when we talked about the RLC model for the wire between two inverters, we lumped it.
We lumped this thing into a model that looked like this.
OK, we lumped the resistance into a source resistance.
We lumped all the inductors into a lumped inductor.
We lumped all the capacitances into a lumped capacitance.
OK, but in this situation, I can do this when the signal speeds of interest are much, much, much slower than the speed of light than the propagation speeds of electromagnetic signals.
In this case, that is not quite true.
And so, therefore, we have to model it much more exactly.
We need to see what's going on.
So, what's happening here is that at t=0, I get this step.
So, think of that as a pulse of energy, and the instant it comes here, and instantaneously this guy looks like a voltage divider, OK?
I've chosen my resistance, R, here to match the instantaneous impedance looking in, which is also R. I've arranged it to be that way.
So, instantaneously, the point at which the pulse appears at this point, looking down here looks like another resistor to this pulse.
OK, therefore, when I start out, I start out going up and pausing at 2.5 because instantaneously, this looks like a resistance, R. So instantaneously, it's a voltage divider, R, and so it's 2.5 here, instantaneously.
OK, then what happens?
Then those little pulse propagates down.
What does it mean for a pulse of energy to propagate down?
Well, it begins sending a current through the inductor, begins charging up the capacitor, current here, so that's what I mean by saying that the pulse of energy goes down.
OK, it's a step that sends current to the inductor and charges of the capacitors, and that wave front moves out here and comes all the way here.
What happens there?
Well, think about it.
Supposing you stand here, and you hold a long string in your hand somehow, and just do this Gedanken experiment.
It's not easy to do.
And so, let's say you somehow have the long string that you're holding onto, and the string on the other side is not connected to anything.
OK, just imagine this experiment.
OK, and what you do is you suddenly raise the string up at your end by about a foot.
What are you going to see happen?
So instantaneously, the string is up here, but the rest of the string is down a foot below.
And then you see this wave propagate down the string, right?
So here's a string.
I lift this thing, and you see this wave propagate all the way down, the one foot wave propagate all the way down until you come here.
What happens here?
So, out here, the string is down here, the wave propagates out here and pulls it up to one.
And then what?
There's nothing connected there, so the string is zipped up, but it's got the energy.
OK, where does energy go?
Well, it continues going up, and sends a wave back.
OK, so just think of a string that you pull up like this and propagates down, boom, hits the other end, reverses, and comes back at me.
OK, you can look at a complementary situation, not the same as this, but complementary by taking a string, tying it to a door, and lifting it up.
It's not the same situation.
It's a complementary situation where it's tied down.
Tying down a string is tantamount to shorting the ends here.
OK, in that case what you'll see happen: as the wave goes down, at the end the string can't move, so the wave goes and flips around and comes back.
Try it out at home.
Take a long piece of string, tie it up there, do this, OK?
And you'll see the wave go out, flip, and then come back at you.
So, if your friends see you tying a long piece of string doing this, hopefully they won't think you're nuts or something.
OK, so the same way here: this thing flies down, OK, there's no way to dissipate the energy here, so this thing continues up.
And then, what I'm going to see happen is the wave move back.
OK, the wave begins to move back, and that's another 2.5V, resulting in a net 5V at this terminal.
That wave begins to blast back, OK, and then when it comes back here, after some amount of time, it raises this to 5V, and that's what you see happen here.
So, this is a wave going down, and then after a time, 2t, it goes back up to 5V.
That's a return wave.
It's 2t because to get down here is t seconds, and then t seconds to come back, which is why we have 2t.
OK, that is why you see that pulse at 2.5.
OK, so I'd like to show you a few more things here.
Clearly we don't want that in our circuits.
Could someone tell me what problem would happen if my signals looked like this in my digital circuits?
Instead of being nice little steps, if there was a little thing in the middle and then a step, what's the problem with signals like this?
In digital circuits, what did it violate?
Yeah?
Exactly.
This little sucker here is meandering out in the forbidden region for all of 2T.
Can't do that.
OK, can't have that.
Well, so we need to fix the problem because this is real life.
OK, but what if you and your buddy were signaling each other but using digital signals from one dorm room to another maybe a few hundred feet down?
Your circuit isn't going to work because the signal's going to meander around in the forbidden region for some time.
So, any ideas what might you do?
Yeah?
Put a resistor on the end.
OK, trick the circuit.
So, what you can do, and I'm going to show you a little demo here, what you can do is the reason I got this wave propagating back, was that there was nothing to absorb the energy.
So instead, what if I put another resistor here, R?
So, as far as a burst of energy is concerned, it says, oh, yeah, it just looks the same.
It's R, and goes and dissipates in this resistor, R, and guess what?
I don't have any wave going back, and I'm done.
So, what I'm going to find, then, is that out here, this goes up to 5V, but out here, I will have a signal that starts out and goes up to 2.5, and that's it.
OK, I lift it up, it goes down, it goes to 2.5 because in the lumped model that you've been dealing with, it's a resistor R, a resistor R to ground, and you're taking the connection here or here.
So, it's your standard lumped model, your voltage resistive divider, and it just simply works.
Yeah, that's it.
So, this is the end of the cable.
OK, if somehow you could watch this and that at the same time, so what I'm going to do, and this is a resistor, R, I'm just going to plug it in.
OK, if the fates are smiling at me, what should you see there?
What should happen is that the second jump from 2.5 to 5 should simply go away.
It should just go to 2.5.
Let's try that.
There you go.
I take it out, it jumps back up.
OK, so all I've done here is put in a resistor at the end, and I'm still measuring the voltage here.
So, that's one solution.
One solution is to put a resistor here.
So, I absorb the energy, and the resistance has to be equal to the instantaneous impedance looking in.
And the instantaneous impedance, for many of these cables is 50 ohms.
It's called a characteristic impedance.
OK, you'll learn a lot more about it if you take 6.014.
That course starts out with assuming that things are distributed in that matter.
OK, so if you want to design multi-gigahertz chips, it turns out that if you have signals that are traveling around at edge speeds in the 0.1-1 nanosecond range, remember, light travels roughly one nanosecond a foot.
And if the signals are roughly of interest are 0.1 nanoseconds, then if the chips are one inch in size, right there, the propagation speed of a signal across a chip is 0.1 nanoseconds.
OK, so today, we have to deal with these issues and try to figure out what to do about them.
OK, so that's one solution that somebody pointed out.
There is a second solution.
Anybody else have a second solution for me?
And then there's a third solution, too.
So it's OK. You can give me either the second or the third solution.
It doesn't matter.
Anybody?
You have two to choose from, come on.
Yeah?
You can do that, yeah.
So we could define the problem away by saying this transition is such that my high is below So, once it goes above 2.5, who cares what it does?
That's a good point.
That's solution number four, and that works.
OK, so I still need two and three.
Put a diode in there?
Yeah, I guess you could.
If the diode had the same kind of impedance looking in, it kind of may work.
That's solution 4.2.
I'm still waiting for solution two and three.
Pardon?
Cut off the cable?
Exactly.
So, the solution says, work on a different problem.
And that is solution number two.
OK, so the idea is, the root of all evil, this long wire, which is why I had this thing here.
So instead, if I had short wires, then what will happen is if it's a very small wire, it'll look like this.
And the wire's small enough.
I will see an itty-bitty thingamajig out there, but not a whole lot.
By the way, the fun thing is that you can actually calculate the speed of light, the experiment I just showed you.
Can we put that up again?
No, the big one.
So, in the experiment that I showed you, this distance was about 500 nanoseconds, OK?
This distance was 500 nanoseconds this time interval.
The length of this cable is about 500 feet, somewhere around 500 feet.
So you can figure out the speed of light.
What's the speed of light?
So, this is about 500 nanoseconds, and this cable is roughly 500 feet.
What's the speed of light?
Roughly a foot per nanosecond.
So, would you believe that in 6.002 we've figured out the speed of light from a simple experiment?
All right, so let's do the next experiment now.
Let's take out the long cable, and connect a short cable instead.
So, what I'm going to do is disconnect the long cable, and instead, connect a small cable.
It's still relatively long, but much shorter than the 500 foot cable.
So what you should see happen now is that the little step should not be this big, but much, much smaller.
So, take a look up there.
There you go.
OK, so with this thingamajig, the little blip there is very small.
And of course, if I make it even smaller, then that can virtually vanish.
OK, so that is solution number two.
So, we've done one, two, four, 4.2.
So, what's solution number three?
One more solution.
Pardon?
So, another solution we mentioned is we change this resistance.
and that will work, if I make this very, very low, then I'll get much closer to 5V here.
Yeah, that's a possibility.
That's solution six I guess.
So what was solution number three?
And you all should be able to solve this.
You guys know the answer.
OK, you folks should be able to solve this.
Yes?
Ah, clock.
So, what I can do is just as was pointed out, that I leveraged my abstraction by changing my VOH and VIH thresholds.
So that'll work.
The alternative thing is to use a clock.
A clock is a distinguished signal that I send around in my digital circuit, OK?
So all I do is if I arrange it such that my clock doesn't happen in this vicinity, but rather, my clock happens late enough, then I'm going to sample and look at my signals only on the rising and falling edges of the clock, in which case I won't be looking at the signal, but the signal is doing weird things.
OK, so a decent clock would also solve the problem.
OK, any last minute questions before we go onto the next one?
OK, the next problem that we're going to look at is titled the Double Dip.
OK, so what I'm going to do here is our Vs power supply, and what I'm going to do is feed the power supply to an inverter.
OK, so we've been doing this all along; Vs, I feed the supply to an inverter.
And what I'm also going to do is, so this is ground, and I'm going to feed it to, so feed the power supply connection to a couple of inverters.
OK, and what I'm going to do is apply some sort of a signal to this inverter, and I'm going to observe, and I'm going to look at this signal here.
So, the abstraction should tell you that here's a power supply.
This is 5V, or whatever the supply voltage is to these two inverters.
That should be fine, and feed some sort of input to this inverter, OK, and the output here should be simply determined by this input.
This signal can have absolutely no bearing on this output.
OK, and let's look at that and actually confirm it.
So, I build a circuit like this, and we look at this output, and initially there should not be any, it should simply work fine.
OK, so it should work now, right?
OK.
So what you have here, this input here is the input that I'm feeding to this inverter.
That is a straight line.
Is that the power supply?
It doesn't matter?
OK, so I believe this is the, we'll check in a few minutes, but I suspect this is the power supply, and this guy here is the output looking here.
So, the green one is the look here part.
So, there must have been a one-to-zero transition here, and that's all fine.
So, so far, so good.
OK, no problem so far.
Now what I'm going to do is I'm going to do something to the circuit that as far as abstraction is concerned, it doesn't show up on the circuit.
OK, it's below the abstraction layer.
OK, I'm going to do something, and suddenly, some things are going to happen.
Look up there.
The circuit hasn't changed.
It's the same circuit.
I've done nothing to the circuit.
OK, look at the green output.
I've done nothing to the circuit that is visible here.
OK, it's below the radar screen here.
It's below the abstraction barrier.
But, look at the disaster here.
OK, in particular, the spikes going up are not so much of a problem.
Because of the static discipline, if I am at five or six or seven, it doesn't matter as long as I am higher than VOH.
So as long as I'm higher than VOH I don't have a problem.
But the problems are these repeated dips.
OK, the dips are a problem here, which is why I labeled this experiment the Double Dip.
OK, the dips are bad because if they are large enough, they can then group the output down into the forbidden region, or worse yet, make it look like a zero.
OK, so you're not prepared for this.
So what I'm going to do is tell you what I did to the circuit, and then ask you to help me figure it out.
So all I did was applied a load resistance to this, I think of 50 ohms or some RL.
I just applied a load resistor.
And this inverter here, I believe, is a CMOS inverter that looks, OK?
So I have this input applied to this inverter, and all I did is I applied an RL load here.
And notice that the load here should not really change what's happening if this is an ideal inverter, OK, the load here should simply draw some current but really should not change any other property.
OK, so just remember, what's the signal doing?
The signal is high.
This guy turns on, and current flows like this.
So, let's say I had some sort of a capacitor here.
This charges like this, and when it's slow, the PFET is on, and current flows through here down here.
And then when this goes high, this guy goes off, and this guy turns on.
OK, so the current flows out this way and this charges through this guy.
When I turn it off, the P fret turns on and draws current from the top.
OK, so do we have any theories as to why I'm getting that messy stuff, the dips and the spikes, on the output of this inverter?
So why does this inverter care what the load of this inverter is?
I mean, who cares?
So, put your thinking caps on.
Any theories?
You guys did pretty well with the previous one.
And this is much easier, actually.
Need a better power supply; OK, so what I'm going to do is I'm going to replace the power supply, and instead, use a much bigger power supply at 5V.
A big, mongo power supply that can supply 100 amps, and guess what, I've made the changes, but guess what, I still see the spikes.
Good try, but it didn't work out.
Good try, good try.
What next?
Any other solutions?
Yes?
So dips are because of the resistance, and the spikes are because of the inductance?
You're half correct.
So, which one is it?
So, dips are because of resistances, and spikes are because of inductances.
You're half correct.
It turns out that both the dips and the spikes are because of inductances.
OK, but be that as it may, let me give you the next clue here, and then see if you can come closer to the answer.
So, what I've done here is I've made this wire really, really long.
OK, it's a really long wire, OK, but it's a thick wire, so it's a long, long, thick wire.
So it's not the resistance.
It's really, really thick and mongo, and it's a long wire, so a signal wire above a ground plane behaves like an inductor.
And so here, it has the capacitance to, but in this case it's inductance.
It's inductance here.
So, I'll give you another ten seconds to think about it and then tell you the answer.
But despite the inductance here, it turns out if I take out this resistor, the problem goes away.
Look, I take out the resistor, the problem goes away.
Yes, there is an inductor here.
OK, I take out this resistor, problem goes away.
I put the resistor back in, boom.
Yes?
OK, pretty good.
That's 86 points.
So here's what's going on.
There's an inductor here, and when I put a 50 ohm resistor here, I put this resistor.
When the PFET turns on, it draws a current.
OK, it's going to draw a current.
It draws a current; remember that across an inductor, I have a drop.
And the drop relates to the di/dt.
Remember, for a capacitor, the current is Cdv/dt.
For the inductor, the voltage across the inductor is Ldi/dt.
So, if di/dt, from switching a large current through the inductor every cycle, OK, big di/dt, di/dt is large.
I've made it large by having a very small RL, so, you know, pulling a big current through every few, whatever, every cycle, and then stopping it.
And so therefore, I'm getting these big drops across this inductor that relate to Ldi/dt.
In other words, the power supply here is fine.
While you guys were watching, I switched to the huge, mongo power supply, and so this voltage is fine.
But then this voltage after the wire is the problem.
So, this voltage here doesn't look like this anymore.
Rather, it has spikes that go down, for example, and when I switch the other way, they go up.
OK, so therefore, what I end up having here is big spikes on this power supply.
And when this guy's power supply goes wacko, then I see the spikes on its output as well.
OK, so what are the solutions for that?
Any solutions here?
What can I do to fix the problem?
Pardon?
Stop using the, exactly.
When in doubt, do something else.
Build a different design.
So what I could do is this is pretty dumb, using a long wire.
And so, no, but trust me, oftentimes you go to the store room and they give you a big roll of wire, and you're too lazy to cut a piece out.
Use the whole roll, and use the two ends, and connect it in, OK?
So, if I had a much shorter piece of wire, then that can solve my problem.
But again, remember, what's small to you may not be small to the circuit.
OK, so let's say, for example, I'm Intel, and I'm building a 10 GHz Pentium 6 processor.
OK, it's 0.1 nanosecond is my cycle time.
There, even a small, itty bitty wire can be a real problem.
OK, and so therefore, distributing power throughout a one inch chip that's clocking at 10 GHz is a really, really hard problem.
And our own David Perreault, who is doing one of our sections, is one of the world's experts in this field.
Distributing power, something as simple as, how do I get 1V in a stable manner to every single device on my chip?
It's a hard problem.
OK, so now, you have to begin feeding your power supply connections much like RC circuits, OK, and you have to solve some hard problems to be able to simply distribute power decently throughout your circuit.
So, what else can I do?
Yeah?
Say it again?
Ah, I can do that.
I could use different wires to connect each of the inverters.
That's a good point.
So here, the coupling happens because I connect the two inverters way out here.
So instead, I use a different cable.
I hadn't thought of that.
That's a creative solution.
OK, so in fact, if you build a chip, so we built this chip called RAW in our group, and it has on the order of 10 million gates.
And this chip we built with IBM's technology, and it turns out that you don't send power supply in through a pin and then connect that 1.5V supply to all your gates.
What you do is from that pin, you then build special power supply buffering trees.
And each tree, each leaf of the tree drives a subcircuit.
In other words, if this is a chip, you have lots and lots of gates throughout your chip.
What you do not do is bring in a power supply like this, and then connect.
You don't do that.
That's the worst possible thing you can do.
It's an absolute disaster for the reason just brought up.
OK, so instead what you do is divide up the chip into, say, four quadrants.
OK, in our case, we have 16 quadrants.
And then what you do is from this point, you take one wire that goes to this quadrant, one wire that comes here, one here, and one here, so that you're getting the power supply very close to the source, and you have different connections going to each quadrant so that switching in this quadrant will not affect this guy because of the inductance of this lead here.
OK, and if you hadn't taken 6.002, you'd have been arguing with IBM, I don't want 16 wires.
I want just one wire.
OK, so there are other solutions, of course.
There's a couple more solution.
One is that what you can do is part of the problem here is that all my transitions are really, really sharp.
OK, so di/dt is very, very large.
So, there's a whole new technique in design of digital and analog circuits, which talks about, maybe I should call it waveform engineering, OK, or edge engineering.
OK, it's also called edge smoothing.
The idea is that rather than have very sharp edges in your circuit, you try to have smoother edges.
And when you have smoother edges, OK, then your di/dt is now going to be less.
It's not going to be very, very high.
Rather, your delta I is spread out over a longer period of time.
Of course, that means the circuits may have to run a little slower, but that can also solve the problem.
And in fact, that same smoothing of the waveforms was also the solution you saw in the capacitive coupling we saw a month and a half ago.
And let me show you the demo, and then close up.
Not working?
OK, that's OK.
It doesn't matter.
So if you remember the demo from the lecture about a month and a half ago in capacitors, I talked about a chip with two pins, and there was this capacitive coupling between the pins.
And because of this, if this waveform is switching, then because of this coupling, you will end up getting, if this is the signal here, you will end up getting spikes on this pin because of the signaling of the other pin.
And that's good old capacitive coupling.
OK, and to eliminate this, what you can do is much like with the inductance system, if you, rather than having sharp transitions on this pin, if you have smooth transitions that look like this, then what you can do is you'll now spread delta V from here to here over a longer delta T. OK, delta T has become longer, and because of that, you end up getting much better behavior, and you don't end up getting these spikes.
So therefore, if you want to build really, really fast circuits, you have to be really careful.
You can build fast circuits, but watch out for them fast edges.
OK, fast edges are nasty.
They kill you.
That's something to remember as you build the next generation of circuits.
Well, thank you all.
I had a blast, and I hope you guys had fun too.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so welcome to 6.041/6.431, the class on probability models and the like.
I'm John Tsitsiklis.
I will be teaching this class, and I'm looking forward to this being an enjoyable and also useful experience.
We have a fair amount of staff involved in this course, your recitation instructors and also a bunch of TAs, but I want to single out our head TA, Uzoma, who is the key person in this class.
Everything has to go through him.
If he doesn't know in which recitation section you are, then simply you do not exist, so keep that in mind.
All right.
So we want to jump right into the subject, but I'm going to take just a few minutes to talk about a few administrative details and how the course is run.
So we're going to have lectures twice a week and I'm going to use old fashioned transparencies.
Now, you get copies of these slides with plenty of space for you to keep notes on them.
A useful way of making good use of the slides is to use them as a sort of mnemonic summary of what happens in lecture.
Not everything that I'm going to say is, of course, on the slides, but by looking them you get the sense of what's happening right now.
And it may be a good idea to review them before you go to recitation.
So what happens in recitation?
In recitation, your recitation instructor is going to maybe review some of the theory and then solve some problems for you.
And then you have tutorials where you meet in very small groups together with your TA.
And what happens in tutorials is that you actually do the problem solving with the help of your TA and the help of your classmates in your tutorial section.
Now probability is a tricky subject.
You may be reading the text, listening to lectures, everything makes perfect sense, and so on, but until you actually sit down and try to solve problems, you don't quite appreciate the subtleties and the difficulties that are involved.
So problem solving is a key part of this class.
And tutorials are extremely useful just for this reason because that's where you actually get the practice of solving problems on your own, as opposed to seeing someone else who's solving them for you.
OK but, mechanics, a key part of what's going to happen today is that you will turn in your schedule forms that are at the end of the handout that you have in your hands.
Then, the TAs will be working frantically through the night, and they're going to be producing a list of who goes into what section.
And when that happens, any person in this class, with probability 90%, is going to be happy with their assignment and, with probability 10%, they're going to be unhappy.
Now, unhappy people have an option, though.
You can resubmit your form together with your full schedule and constraints, give it back to the head TA, who will then do some further juggling and reassign people, and after that happens, 90% of those unhappy people will become happy.
And 10% of them will be less unhappy.
OK.
So what's the probability that a random person is going to be unhappy at the end of this process?
It's 1%.
Excellent.
Good.
Maybe you don't need this class.
OK, so 1%.
We have about 100 people in this class, so there's going to be about one unhappy person.
I mean, anywhere you look in life, in any group you look at, there's always one unhappy person, right?
So, what can we do about it?
All right.
Another important part about mechanics is to read carefully the statement that we have about collaboration, academic honesty, and all that.
You're encouraged, it's a very good idea to work with other students.
You can consult sources that are out there, but when you sit down and write your solutions you have to do that by setting things aside and just write them on your own.
You cannot copy something that somebody else has given to you.
One reason is that we're not going to like it when it happens, and then another reason is that you're not going to do yourself any favor.
Really the only way to do well in this class is to get a lot of practice by solving problems yourselves.
So if you don't do that on your own, then when quiz and exam time comes, things are going to be difficult.
So, as I mentioned here, we're going to have recitation sections, that some of them are for 6.041 students, some are for 6.431 students, the graduate section of the class.
Now undergraduates can sit in the graduate recitation sections.
What's going to happen there is that things may be just a little faster and you may be covering a problem that's a little more advanced and is not covered in the undergrad sections.
But if you sit in the graduate section, and you're an undergraduate, you're still just responsible for the undergraduate material.
That is, you can just do the undergraduate work in the class, but maybe be exposed at the different section.
OK. A few words about the style of this class.
We want to focus on basic ideas and concepts.
There's going to be lots of formulas, but what we try to do in this class is to actually have you understand what those formulas mean.
And, in a year from now when almost all of the formulas have been wiped out from your memory, you still have the basic concepts.
You can understand them, so when you look things up again, they will still make sense.
It's not the plug and chug kind of class where you're given a list of formulas, you're given numbers, and you plug in and you get answers.
The really hard part is usually to choose which formulas you're going to use.
You need judgment, you need intuition.
Lots of probability problems, at least the interesting ones, often have lots of different solutions.
Some are extremely long, some are extremely short.
The extremely short ones usually involve some kind of deeper understanding of what's going on so that you can pick a shortcut and use it.
And hopefully you are going to develop this skill during this class.
Now, I could spend a lot of time in this lecture talking about why the subject is important.
I'll keep it short because I think it's almost obvious.
Anything that happens in life is uncertain.
There's uncertainty anywhere, so whatever you try to do, you need to have some way of dealing or thinking about this uncertainty.
And the way to do that in a systematic way is by using the models that are given to us by probability theory.
So if you're an engineer and you're dealing with a communication system or signal processing, basically you're facing a fight against noise.
Noise is random, is uncertain.
How do you model it?
How do you deal with it?
If you're a manager, I guess you're dealing with customer demand, which is, of course, random.
Or you're dealing with the stock market, which is definitely random.
Or you play the casino, which is, again, random, and so on.
And the same goes for pretty much any other field that you can think of.
But, independent of which field you're coming from, the basic concepts and tools are really all the same.
So you may see in bookstores that there are books, probability for scientists, probability for engineers, probability for social scientists, probability for astrologists.
Well, what all those books have inside them is exactly the same models, the same equations, the same problems.
They just make them somewhat different word problems.
The basic concepts are just one and the same, and we'll take this as an excuse for not going too much into specific domain applications.
We will have problems and examples that are motivated, in some loose sense, from real world situations.
But we're not really trying in this class to develop the skills for domain-specific problems.
Rather, we're going to try to stick to general understanding of the subject.
OK.
So the next slide, of which you do have in your handout, gives you a few more details about the class.
Maybe one thing to comment here is that you do need to read the text.
And with calculus books, perhaps you can live with a just a two page summary of all of the interesting formulas in calculus, and you can get by just with those formulas.
But here, because we want to develop concepts and intuition, actually reading words, as opposed to just browsing through equations, does make a difference.
In the beginning, the class is kind of easy.
When we deal with discrete probability, that's the material until our first quiz, and some of you may get by without being too systematic about following the material.
But it does get substantially harder afterwards.
And I would keep restating that you do have to read the text to really understand the material.
OK.
So now we can start with the real part of the lecture.
Let us set the goals for today.
So probability, or probability theory, is a framework for dealing with uncertainty, for dealing with situations in which we have some kind of randomness.
So what we want to do is, by the end of today's lecture, to give you anything that you need to know how to set up what does it take to set up a probabilistic model.
And what are the basic rules of the game for dealing with probabilistic models?
So, by the end of this lecture, you will have essentially recovered half of this semester's tuition, right?
So we're going to talk about probabilistic models in more detail-- the sample space, which is basically a description of all the things that may happen during a random experiment, and the probability law, which describes our beliefs about which outcomes are more likely to occur compared to other outcomes.
Probability laws have to obey certain properties that we call the axioms of probability.
So the main part of today's lecture is to describe those axioms, which are the rules of the game, and consider a few really trivial examples.
OK, so let's start with our agenda.
The first piece in a probabilistic model is a description of the sample space of an experiment.
So we do an experiment, and by experiment we just mean that just something happens out there.
And that something that happens, it could be flipping a coin, or it could be rolling a dice, or it could be doing something in a card game.
So we fix a particular experiment.
And we come up with a list of all the possible things that may happen during this experiment.
So we write down a list of all the possible outcomes.
So here's a list of all the possible outcomes of the experiment.
I use the word "list," but, if you want to be a little more formal, it's better to think of that list as a set.
So we have a set.
That set is our sample space.
And it's a set whose elements are the possible outcomes of the experiment.
So, for example, if you're dealing with flipping a coin, your sample space would be heads, this is one outcome, tails is one outcome.
And this set, which has two elements, is the sample space of the experiment.
OK. What do we need to think about when we're setting up the sample space?
First, the list should be mutually exclusive, collectively exhaustive.
What does that mean?
Collectively exhaustive means that, no matter what happens in the experiment, you're going to get one of the outcomes inside here.
So you have not forgotten any of the possibilities of what may happen in the experiment.
Mutually exclusive means that if this happens, then that cannot happen.
So at the end of the experiment, you should be able to point out to me just one, exactly one, of these outcomes and say, this is the outcome that happened.
OK.
So these are sort of basic requirements.
There's another requirement which is a little more loose.
When you set up your sample space, sometimes you do have some freedom about the details of how you're going to describe it.
And the question is, how much detail are you going to include?
So let's take this coin flipping experiment and think of the following sample space.
One possible outcome is heads, a second possible outcome is tails and it's raining, and the third possible outcome is tails and it's not raining.
So this is another possible sample space for the experiment where I flip a coin just once.
It's a legitimate one.
These three possibilities are mutually exclusive and collectively exhaustive.
Which one is the right sample space?
Is it this one or that one?
Well, if you think that my coin flipping inside this room is completely unrelated to the weather outside, then you're going to stick with this sample space.
If, on the other hand, you have some superstitious belief that maybe rain has an effect on my coins, you might work with the sample space of this kind.
So you probably wouldn't do that, but it's a legitimate option, strictly speaking.
Now this example is a little bit on the frivolous side, but the issue that comes up here is a basic one that shows up anywhere in science and engineering.
Whenever you're dealing with a model or with a situation, there are zillions of details in that situation.
And when you come up with a model, you choose some of those details that you keep in your model, and some that you say, well, these are irrelevant.
Or maybe there are small effects, I can neglect them, and you keep them outside your model.
So when you go to the real world, there's definitely an element of art and some judgment that you need to do in order to set up an appropriate sample space.
So, an easy example now.
So of course, the elementary examples are coins, cards, and dice.
So let's deal with dice.
But to keep the diagram small, instead of a six-sided die, we're going to think about the die that only has four faces.
So you can do that with a tetrahedron, doesn't really matter.
Basically, it's a die that when you roll it, you get a result which is one, two, three or four.
However, the experiment that I'm going to think about will consist of two rolls of a dice.
A crucial point here-- I'm rolling the die twice, but I'm thinking of this as just one experiment, not two different experiments, not a repetition twice of the same experiment.
So it's one big experiment.
During that big experiment various things could happen, such as I'm rolling the die once, and then I'm rolling the die twice.
OK.
So what's the sample space for that experiment?
Well, the sample space consists of the possible outcomes.
One possible outcome is that your first roll resulted in two and the second roll resulted in three.
In which case, the outcome that you get is this one, a two followed by three.
This is one possible outcome.
The way I'm describing things, this outcome is to be distinguished from this outcome here, where a three is followed by two.
If you're playing backgammon, it doesn't matter which one of the two happened.
But if you're dealing with a probabilistic model that you want to keep track of everything that happens in this composite experiment, there are good reasons for distinguishing between these two outcomes.
I mean, when this happens, it's definitely something different from that happening.
A two followed by a three is different from a three followed by a two.
So this is the correct sample space for this experiment where we roll the die twice.
It has a total of 16 elements and it's, of course, a finite set.
Sometimes, instead of describing sample spaces in terms of lists, or sets, or diagrams of this kind, it's useful to describe the experiment in some sequential way.
Whenever you have an experiment that consists of multiple stages, it might be useful, at least visually, to give a diagram that shows you how those stages evolve.
And that's what we do by using a sequential description or a tree-based description by drawing a tree of the possible evolutions during our experiment.
So in this tree, I'm thinking of a first stage in which I roll the first die, and there are four possible results, one, two, three and four.and 4.
And, given what happened, let's say in the first roll, suppose I got a one.
Then I'm rolling the second dice, and there are four possibilities for what may happen to the second die.
And the possible results are one, tow, three and four again.
So what's the relation between the two diagrams?
Well, for example, the outcome two followed by three corresponds to this path on the tree.
So this path corresponds to two followed by a three.
Any path is associated to a particular outcome, any outcome is associated to a particular path.
And, instead of paths, you may want to think in terms of the leaves of this diagram.
Same thing, think of each one of the leaves as being one possible outcome.
And of course we have 16 outcomes here, we have 16 outcomes here.
Maybe you noticed the subtlety that I used in my language.
I said I rolled the first dice and the result that I get is a two.
I didn't use the word "outcome."
I want to reserve the word "outcome" to mean the overall outcome at the end of the overall experiment.
So "2, 3" is the outcome of the experiment.
The experiment consisted of stages.
Two was the result in the first stage, three was the result in the second stage.
You put all those results together, and you get your outcome.
OK, perhaps we are splitting hairs here, but it's useful to keep the concepts right.
What's special about this example is that, besides being trivial, it has a sample space which is finite.
There's 16 possible total outcomes.
Not every experiment has a finite sample space.
Here's an experiment in which the sample space is infinite.
So you are playing darts and the target is this square.
And you're perfect at that game, so you're sure that your darts will always fall inside the square.
So, but where exactly your dart would fall inside that square, that itself is random.
We don't know what it's going to be.
It's uncertain.
So all the possible points inside the square are possible outcomes of the experiment.
So a typical outcome of the experiment is going to a pair of numbers, x,y, where x and y are real numbers between zero and one.
Now there's infinitely many real numbers, there's infinitely many points in the square, so this is an example in which our sample space is an infinite set.
OK, so we're going to revisit this example a little later.
So these are two examples of what the sample space might be in simple experiments.
Now, the more important order of business is now to look at those possible outcomes and to make some statements about their relative likelihoods.
Which outcome is more likely to occur compared to the others?
And the way we do this is by assigning probabilities to the outcomes.
Well, not exactly.
Suppose that all you were to do was to assign probabilities to individual outcomes.
If you go back to this example, and you consider one particular outcome-- let's say this point-- what would be the probability that you hit exactly this point to infinite precision?
Intuitively, that probability would be zero.
So any individual point in this diagram in any reasonable model should have zero probability.
So if you just tell me that any individual outcome has zero probability, you're not really telling me much to work with.
For that reason, what instead we're going to do is to assign probabilities to subsets of the sample space, as opposed to assigning probabilities to individual outcomes.
So here's the picture.
We have our sample space, which is omega, and we consider some subset of the sample space.
Call it A.
And I want to assign a number, a numerical probability, to this particular subset which represents my belief about how likely this set is to occur.
OK. What do we mean "to occur?"
And I'm introducing here a language that's being used in probability theory.
When we talk about subsets of the sample space, we usually call them events, as opposed to subsets.
And the reason is because it works nicely with the language that describes what's going on.
So the outcome is a point.
The outcome is random.
The outcome may be inside this set, in which case we say that event A occurred, if we get an outcome inside here.
Or the outcome may fall outside the set, in which case we say that event A did not occur.
So we're going to assign probabilities to events.
And now, how should we do this assignment?
Well, probabilities are meant to describe your beliefs about which sets are more likely to occur versus other sets.
So there's many ways that you can assign those probabilities.
But there are some ground rules for this game.
First, we want probabilities to be numbers between zero and one because that's the usual convention.
So a probability of zero means we're certain that something is not going to happen.
Probability of one means that we're essentially certain that something's going to happen.
So we want numbers between zero and one.
We also want a few other things.
And those few other things are going to be encapsulated in a set of axioms.
What "axioms" means in this context, it's the ground rules that any legitimate probabilistic model should obey.
You have a choice of what kind of probabilities you use.
But, no matter what you use, they should obey certain consistency properties because if they obey those properties, then you can go ahead and do useful calculations and do some useful reasoning.
So what are these properties?
First, probabilities should be non-negative.
OK?
That's our convention.
We want probabilities to be numbers between zero and one.
So they should certainly be non-negative.
The probability that event A occurs should be a non-negative number.
What's the second axiom?
The probability of the entire sample space is equal to one.
Why does this make sense?
Well, the outcome is certain to be an element of the sample space because we set up a sample space, which is collectively exhaustive.
No matter what the outcome is, it's going to be an element of the sample space.
We're certain that event omega is going to occur.
Therefore, we represent this certainty by saying that the probability of omega is equal to one.
Pretty straightforward so far.
The more interesting axiom is the third rule.
Before getting into it, just a quick reminder.
If you have two sets, A and B, the intersection of A and B consists of those elements that belong both to A and B.
And we denote it this way.
When you think probabilistically, the way to think of intersection is by using the word "and."
This event, this intersection, is the event that A occurred and B occurred.
If I get an outcome inside here, A has occurred and B has occurred at the same time.
So you may find the word "and" to be a little more convenient than the word "intersection."
And similarly, we have some notation for the union of two events, which we write this way.
The union of two sets, or two events, is the collection of all the elements that belong either to the first set, or to the second, or to both.
When you talk about events, you can use the word "or."
So this is the event that A occurred or B occurred.
And this "or" means that it could also be that both of them occurred.
OK.
So now that we have this notation, what does the third axiom say?
The third axiom says that if we have two events, A and B, that have no common elements-- so here's A, here's B, and perhaps this is our big sample space.
The two events have no common elements.
So the intersection of the two events is the empty set.
There's nothing in their intersection.
Then, the total probability of A together with B has to be equal to the sum of the individual probabilities.
So the probability that A occurs or B occurs is equal to the probability that A occurs plus the probability that B occurs.
So think of probability as being cream cheese.
You have one pound of cream cheese, the total probability assigned to the entire sample space.
And that cream cheese is spread out over this set.
The probability of A is how much cream cheese sits on top of A. Probability of B is how much sits on top of B.
The probability of A union B is the total amount of cream cheese sitting on top of this and that, which is obviously the sum of how much is sitting here and how much is sitting there.
So probabilities behave like cream cheese, or they behave like mass.
For example, if you think of some material object, the mass of this set consisting of two pieces is obviously the sum of the two masses.
So this property is a very intuitive one.
It's a pretty natural one to have.
OK. Are these axioms enough for what we want to do?
I mentioned a while ago that we want probabilities to be numbers between zero and one.
Here's an axiom that tells you that probabilities are non-negative.
Should we have another axiom that tells us that probabilities are less than or equal to one?
It's a desirable property.
We would like to have it in our hands.
OK, why is it not in that list?
Well, the people who are in the axiom making business are mathematicians and mathematicians tend to be pretty laconic.
You don't say something if you don't have to say it.
And this is the case here.
We don't need that extra axiom because we can derive it from the existing axioms.
Here's how it goes.
One is the probability over the entire sample space.
Here we're using the second axiom.
Now the sample space consists of A together with the complement of A. OK?
When I write the complement of A, I mean the complement of A inside of the set omega.
So we have omega, here's A, here's the complement of A, and the overall set is omega.
OK. Now, what's the next step?
What should I do next?
Which axiom should I use?
We use axiom three because a set and the complement of that set are disjoint.
They don't have any common elements.
So axiom three applies and tells me that this is the probability of A plus the probability of A complement.
In particular, the probability of A is equal to one minus the probability of A complement, and this is less than or equal to one.
Why?
Because probabilities are non-negative, by the first axiom.
OK.
So we got the conclusion that we wanted.
Probabilities are always less than or equal to one, and this is a simple consequence of the three axioms that we have.
This is a really nice argument because it actually uses each one of those axioms.
The argument is simple, but you have to use all of these three properties to get the conclusion that you want.
OK.
So we can get interesting things out of our axioms.
Can we get some more interesting ones?
How about the union of three sets?
What kind of probability should it have?
So here's an event consisting of three pieces.
And I want to say something about the probability of A union B union C. What I would like to say is that this probability is equal to the sum of the three individual probabilities.
How can I do it?
I have an axiom that tells me that I can do it for two events.
I don't have an axiom for three events.
Well, maybe I can manage things and still be able to use that axiom.
And here's the trick.
The union of three sets, you can think of it as forming the union of the first two sets and then taking the union with the third set.
OK?
So taking unions, you can take the unions in any order that you want.
So here we have the union of two sets.
Now, ABC are disjoint, by assumption or that's how I drew it.
So if A, B, and C are disjoint, then A union B is disjoint from C. So here we have the union of two disjoint sets.
So by the additivity axiom, the probability of that the union is going to be the probability of the first set plus the probability of the second set.
And now I can use the additivity axiom once more to write that this is probability of A plus probability of B plus probability of C. So by using this axiom which was stated for two sets, we can actually derive a similar property for the union of three disjoint sets.
And then you can repeat this argument as many times as you want.
It's valid for the union of ten disjoint sets, for the union of a hundred disjoint sets, for the union of any finite number of sets.
So if A1 up to An are disjoint, then the probability of A1 union An is equal to the sum of the probabilities of the individual sets.
OK. Special case of this is when we're dealing with finite sets.
Suppose I have just a finite set of outcomes.
I put them together in a set and I'm interested in the probability of that set.
So here's our sample space.
There's lots of outcomes, but I'm taking a few of these and I form a set out of them.
This is a set consisting of, in this picture, three elements.
In general, it consists of k elements.
Now, a finite set, I can write it as a union of single element sets.
So this set here is the union of this one element set, together with this one element set together with that one element set.
So the total probability of this set is going to be the sum of the probabilities of the one element sets.
Now, probability of a one element set, you need to use the brackets here because probabilities are assigned to sets.
But this gets kind of tedious, so here one abuses notation a little bit and we get rid of those brackets and just write probability of this single, individual outcome.
In any case, conclusion from this exercise is that the total probability of a finite collection of possible outcomes, the total probability is equal to the sum of the probabilities of individual elements.
So these are basically the axioms of probability theory.
Or, well, they're almost the axioms.
There are some subtleties that are involved here.
One subtlety is that this axiom here doesn't quite do the job for everything we would like to do.
And we're going to come back to this at the end of the lecture.
A second subtlety has to do with weird sets.
We said that an event is a subset of the sample space and we assign probabilities to events.
Does this mean that we are going to assign probability to every possible subset of the sample space?
Ideally, we would wish to do that.
Unfortunately, this is not always possible.
If you take a sample space, such as the square, the square has nice subsets, those that you can describe by cutting it with lines and so on.
But it does have some very ugly subsets, as well, that are impossible to visualize, impossible to imagine, but they do exist.
And those very weird sets are such that there's no way to assign probabilities to them in a way that's consistent with the axioms of probability.
OK.
So this is a very, very fine point that you can immediately forget for the rest of this class.
You will only encounter these sets if you end up doing doctoral work on the theoretical aspects of probability theory.
So it's just a mathematical subtlety that some very weird sets do not have probabilities assigned to them.
But we're not going to encounter these sets and they do not show up in any applications.
OK.
So now let's revisit our examples.
Let's go back to the die example.
We have our sample space.
Now we need to assign a probability law.
There's lots of possible probability laws that you can assign.
I'm picking one here, arbitrarily, in which I say that every possible outcome has the same probability of 1/16.
OK. Why do I make this model?
Well, empirically, if you have well-manufactured dice, they tend to behave that way.
We will be coming back to this kind of story later in this class.
But I'm not saying that this is the only probability law that there can be.
You might have weird dice in which certain outcomes are more likely than others.
But to keep things simple, let's take every outcome to have the same probability of 1/16.
OK. Now that we have in our hands a sample space and the probability law, we can actually solve any problem there is.
We can answer any question that could be posed to us.
For example, what's the probability that the outcome, which is this pair, is either 1,1 or 1,2.
We're talking here about this particular event, 1,1 or 1,2.
So it's an event consisting of these two items.
According to what we were just discussing, the probability of a finite collection of outcomes is the sum of their individual probabilities.
Each one of them has probability of 1/16, so the probability of this is 2/16.
How about the probability of the event that x is equal to one.
x is the first roll, so that's the probability that the first roll is equal to one.
Notice the syntax that's being used here.
Probabilities are assigned to subsets, to sets, so we think of this as meaning the set of all outcomes such that x is equal to one.
How do you answer this question?
You go back to the picture and you try to visualize or identify this event of interest.
x is equal to one corresponds to this event here.
These are all the outcomes at which x is equal to one.
There's four outcomes.
Each one has probability 1/16, so the answer is 4/16.
OK. How about the probability that x plus y is odd?
OK. That will take a little bit more work.
But you go to the sample space and you identify all the outcomes at which the sum is an odd number.
So that's a place where the sum is odd, these are other places, and I guess that exhausts all the possible outcomes at which we have an odd sum.
We count them.
How many are there?
There's a total of eight of them.
Each one has probability 1/16, total probability is 8/16.
And harder question.
What is the probability that the minimum of the two rolls is equal to 2?
This is something that you probably couldn't do in your head without the help of a diagram.
But once you have a diagram, things are simple.
You ask the question.
OK, this is an event, that the minimum of the two rolls is equal to two.
This can happen in several ways.
What are the several ways that it can happen?
Go to the diagram and try to identify them.
So the minimum is equal to two if both of them are two's.
Or it could be that x is two and y is bigger, or y is two and x is bigger.
OK.
I guess we rediscover that yellow and blue make green, so we see here that there's a total of five possible outcomes.
The probability of this event is 5/16.
Simple example, but the procedure that we followed in this example actually applies to any probability model you might ever encounter.
You set up your sample space, you make a statement that describes the probability law over that sample space, then somebody asks you questions about various events.
You go to your pictures, identify those events, pin them down, and then start kind of counting and calculating the total probability for those outcomes that you're considering.
This example is a special case of what is called the discrete uniform law.
The model obeys the discrete uniform law if all outcomes are equally likely.
It doesn't have to be that way.
That's just one example of a probability law.
But when things are that way, if all outcomes are equally likely and we have N of them, and you have a set A that has little n elements, then each one of those elements has probability one over capital N since all outcomes are equally likely.
And for our probabilities to add up to one, each one must have this much probability, and there's little n elements.
That gives you the probability of the event of interest.
So problems like the one in the previous slide and more generally of the type described here under discrete uniform law, these problems reduce to just counting.
How many elements are there in my sample space?
How many elements are there inside the event of interest?
Counting is generally simple, but for some problems it gets pretty complicated.
And in a couple of weeks, we're going to have to spend the whole lecture just on the subject of how to count systematically.
Now the procedure we followed in the previous example is the same as the procedure you would follow in continuous probability problems.
So, going back to our dart problem, we get the random point inside the square.
That's our sample space.
We need to assign a probability law.
For lack of imagination, I'm taking the probability law to be the area of a subset.
So if we have two subsets of the sample space that have equal areas, then I'm postulating that they are equally likely to occur.
The probably that they fall here is the same as the probability that they fall there.
The model doesn't have to be that way.
But if I have sort of complete ignorance of which points are more likely than others, that might be the reasonable model to use.
So equal areas mean equal probabilities.
If the area is twice as large, the probability is going to be twice as big.
So this is our model.
We can now answer questions.
Let's answer the easy one.
What's the probability that the outcome is exactly this point?
That of course is zero because a single point has zero area.
And since this probability is equal to area, that's zero probability.
How about the probability that the sum of the coordinates of the point that we got is less than or equal to 1/2?
How do you deal with it?
Well, you look at the picture again, at your sample space, and try to describe the event that you're talking about.
The sum being less than 1/2 corresponds to getting an outcome that's below this line, where this line is the line where x plus y equals to 1/2.
So the intercepts of that line with the axis are 1/2 and 1/2.
So you describe the event visually and then you use your probability law.
The probability law that we have is that the probability of a set is equal to the area of that set.
So all we need to find is the area of this triangle, which is 1/2 times 1/2 times 1/2, half, equals to 1/8.
OK.
Moral from these two examples is that it's always useful to have a picture and work with a picture to visualize the events that you're talking about.
And once you have a probability law in your hands, then it's a matter of calculation to find the probabilities of an event of interest.
The calculations we did in these two examples, of course, were very simple.
Sometimes calculations may be a lot harder, but it's a different business.
It's a business of calculus, for example, or being good in algebra and so on.
As far as probability is concerned, it's clear what you will be doing, and then maybe you're faced with a harder algebraic part to actually carry out the calculations.
The area of a triangle is easy to compute.
If I had put down a very complicated shape, then you might need to solve a hard integration problem to find the area of that shape, but that's stuff that belongs to another class that you have presumably mastered by now.
Good, OK.
So now let me spend just a couple of minutes to return to a point that I raised before.
I was saying that the axiom that we had about additivity might not quite be enough.
Let's illustrate what I mean by the following example.
Think of the experiment where you keep flipping a coin and you wait until you obtain heads for the first time.
What's the sample space of this experiment?
It might happen the first flip, it might happen in the tenth flip.
Heads for the first time might occur in the millionth flip.
So the outcome of this experiment is going to be an integer and there's no bound to that integer.
You might have to wait very much until that happens.
So the natural sample space is the set of all possible integers.
Somebody tells you some information about the probability law.
The probability that you have to wait for n flips is equal to two to the minus n. Where did this come from?
That's a separate story.
Where did it come from?
Somebody tells this to us, and those probabilities are plotted here as a function of n. And you're asked to find the probability that the outcome is an even number.
How do you go about calculating that probability?
So the probability of being an even number is the probability of the subset that consists of just the even numbers.
So it would be a subset of this kind, that includes two, four, and so on.
So any reasonable person would say, well the probability of obtaining an outcome that's either two or four or six and so on is equal to the probability of obtaining a two, plus the probability of obtaining a four, plus the probability of obtaining a six, and so on.
These probabilities are given to us.
So here I have to do my algebra.
I add this geometric series and I get an answer of 1/3.
That's what any reasonable person would do.
But the person who only knows the axioms that they posted just a little earlier may get stuck.
They would get stuck at this point.
How do we justify this?
We had this property for the union of disjoint sets and the corresponding property that tells us that the total probability of finitely many things, outcomes, is the sum of their individual probabilities.
But here we're using it on an infinite collection.
The probability of infinitely many points is equal to the sum of the probabilities of each one of these.
To justify this step we need to introduce one additional rule, an additional axiom, that tells us that this step is actually legitimate.
And this is the countable additivity axiom, which is a little stronger, or quite a bit stronger, than the additivity axiom we had before.
It tells us that if we have a sequence of sets that are disjoint and we want to find their total probability, then we are allowed to add their individual probabilities.
So the picture might be such as follows.
We have a sequence of sets, A1, A2, A3, and so on.
I guess in order to fit them inside the sample space, the sets need to get smaller and smaller perhaps.
They are disjoint.
We have a sequence of such sets.
The total probability of falling anywhere inside one of those sets is the sum of their individual probabilities.
A key subtlety that's involved here is that we're talking about a sequence of events.
By "sequence" we mean that these events can be arranged in order.
I can tell you the first event, the second event, the third event, and so on.
So if you have such a collection of events that can be ordered as first, second, third, and so on, then you can add their probabilities to find the probability of their union.
So this point is actually a little more subtle than you might appreciate at this point, and I'm going to return to it at the beginning of the next lecture.
For now, enjoy the first week of classes and have a good weekend.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So here's the agenda for today.
We're going to do a very quick review.
And then we're going to introduce some very important concepts.
The idea is that all information is-- Information is always partial.
And the question is what do we do to probabilities if we have some partial information about the random experiments.
We're going to introduce the important concept of conditional probability.
And then we will see three very useful ways in which it is used.
And these ways basically correspond to divide and conquer methods for breaking up problems into simpler pieces.
And also one more fundamental tool which allows us to use conditional probabilities to do inference, that is, if we get a little bit of information about some phenomenon, what can we infer about the things that we have not seen?
So our quick review.
In setting up a model of a random experiment, the first thing to do is to come up with a list of all the possible outcomes of the experiment.
So that list is what we call the sample space.
It's a set.
And the elements of the sample space are all the possible outcomes.
Those possible outcomes must be distinguishable from each other.
They're mutually exclusive.
Either one happens or the other happens, but not both.
And they are collectively exhaustive, that is no matter what the outcome of the experiment is going to be an element of the sample space.
And then we discussed last time that there's also an element of art in how to choose your sample space, depending on how much detail you want to capture.
This is usually the easy part.
Then the more interesting part is to assign probabilities to our model, that is to make some statements about what we believe to be likely and what we believe to be unlikely.
The way we do that is by assigning probabilities to subsets of the sample space.
So as we have our sample space here, we may have a subset A.
And we assign a number to that subset P(A), which is the probability that this event happens.
Or this is the probability that when we do the experiment and we get an outcome it's the probability that the outcome happens to fall inside that event.
We have certain rules that probabilities should satisfy.
They're non-negative.
The probability of the overall sample space is equal to one, which expresses the fact that we're are certain, no matter what, the outcome is going to be an element of the sample space.
Well, if we set the top right so that it exhausts all possibilities, this should be the case.
And then there's another interesting property of probabilities that says that, if we have two events or two subsets that are disjoint, and we're interested in the probability, that one or the other happens, that is the outcome belongs to A or belongs to B.
For disjoint events the total probability of these two, taken together, is just the sum of their individual probabilities.
So probabilities behave like masses.
The mass of the object consisting of A and B is the sum of the masses of these two objects.
Or you can think of probabilities as areas.
They have, again, the same property.
The area of A together with B is the area of A plus the area B.
But as we discussed at the end of last lecture, it's useful to have in our hands a more general version of this additivity property, which says the following, if we take a sequence of sets-- A1, A2, A3, A4, and so on.
And we put all of those sets together.
It's an infinite sequence.
And we ask for the probability that the outcome falls somewhere in this infinite union, that is we are asking for the probability that the outcome belongs to one of these sets, and assuming that the sets are disjoint, we can again find the probability for the overall set by adding up the probabilities of the individual sets.
So this is a nice and simple property.
But it's a little more subtle than you might think.
And let's see what's going on by considering the following example.
We had an example last time where we take our sample space to be the unit square.
And we said let's consider a probability law that says that the probability of a subset is just the area of that subset.
So let's consider this probability law.
OK. Now the unit square is the set --let me just draw it this way-- the unit square is the union of one element set consisting all of the points.
So the unit square is made up by the union of the various points inside the square.
So union over all x's and y's.
OK?
So the square is made up out of all the points that this contains.
And now let's do a calculation.
One is the probability of our overall sample space, which is the unit square.
Now the unit square is the union of these things, which, according to our additivity axiom, is the sum of the probabilities of all of these one element sets.
Now what is the probability of a one element set?
What is the probability of this one element set?
What's the probability that our outcome is exactly that particular point?
Well, it's the area of that set, which is zero.
So it's just the sum of zeros.
And by any reasonable definition the sum of zeros is zero.
So we just proved that one is equal to zero.
OK.
Either probability theory is dead or there is some mistake in the derivation that I did.
OK, the mistake is quite subtle and it comes at this step.
We're sort of applied the additivity axiom by saying that the unit square is the union of all those sets.
Can we really apply our additivity axiom.
Here's the catch.
The additivity axiom applies to the case where we have a sequence of disjoint events and we take their union.
Is this a sequence of sets?
Can you make up the whole unit square by taking a sequence of elements inside it and cover the whole unit square?
Well if you try, if you start looking at the sequence of one element points, that sequence will never be able to exhaust the whole unit square.
So there's a deeper reason behind that.
And the reason is that infinite sets are not all of the same size.
The integers are an infinite set.
And you can arrange the integers in a sequence.
But the continuous set like the units square is a bigger set.
It's so-called uncountable.
It has more elements than any sequence could have.
So this union here is not of this kind, where we would have a sequence of events.
It's a different kind of union.
It's a Union that involves a union of many, many more sets.
So the countable additivity axiom does not apply in this case.
Because, we're not dealing with a sequence of sets.
And so this is the incorrect step.
So at some level you might think that this is puzzling and awfully confusing.
On the other hand, if you think about areas of the way you're used to them from calculus, there's nothing mysterious about it.
Every point on the unit square has zero area.
When you put all the points together, they make up something that has finite area.
So there shouldn't be any mystery behind it.
Now, one interesting thing that this discussion tells us, especially the fact that the single elements set has zero area, is the following-- Individual points have zero probability.
After you do the experiment and you observe the outcome, it's going to be an individual point.
So what happened in that experiment is something that initially you thought had zero probability of occurring.
So if you happen to get some particular numbers and you say, "Well, in the beginning, what did I think about those specific numbers?
I thought they had zero probability.
But yet those particular numbers did occur."
So one moral from this is that zero probability does not mean impossible.
It just means extremely, extremely unlikely by itself.
So zero probability things do happen.
In such continuous models, actually zero probability outcomes are everything that happens.
And the bumper sticker version of this is to always expect the unexpected.
Yes?
[INAUDIBLE]
Well, probability is supposed to be a real number.
So it's either zero or it's a positive number.
So you can think of the probability of things just close to that point and those probabilities are tiny and close to zero.
So that's how we're going to interpret probabilities in continuous models.
But this is two chapters ahead.
Yeah?
How do we interpret probability of zero?
If we can use models that way, then how about probability of one?
That it it's extremely likely but not necessarily for certain?
That's also the case.
For example, if you ask in this continuous model, if you ask me for the probability that x, y, is different than the zero, zero this is the whole square, except for one point.
So the area of this is going to be one.
But this event is not entirely certain because the zero, zero outcome is also possible.
So again, probability of one means essential certainty.
But it still allows the possibility that the outcome might be outside that set.
So these are some of the weird things that are happening when you have continuous models.
And that's why we start to this class with discrete models, on which would be spending the next couple of weeks.
OK.
So now once we have set up our probability model and we have a legitimate probability law that has these properties, then the rest is usually simple.
Somebody asks you a question of calculating the probability of some event.
While you were told something about the probability law, such as for example the probabilities are equal to areas, and then you just need to calculate.
In these type of examples somebody would give you a set and you would have to calculate the area of that set.
So the rest is just calculation and simple.
Alright, so now it's time to start with our main business for today.
And the starting point is the following-- You know something about the world.
And based on what you know when you set up a probability model and you write down probabilities for the different outcomes.
Then something happens, and somebody tells you a little more about the world, gives you some new information.
This new information, in general, should change your beliefs about what happened or what may happen.
So whenever we're given new information, some partial information about the outcome of the experiment, we should revise our beliefs.
And conditional probabilities are just the probabilities that apply after the revision of our beliefs, when we're given some information.
So lets make this into a numerical example.
So inside the sample space, this part of the sample space, let's say has probability 3/6, this part has 2/6, and that part has 1/6.
I guess that means that out here we have zero probability.
So these were our initial beliefs about the outcome of the experiment.
Suppose now that someone comes and tells you that event B occurred.
So they don't tell you the full outcome with the experiment.
But they just tell you that the outcome is known to lie inside this set B.
Well then, you should certainly change your beliefs in some way.
And your new beliefs about what is likely to occur and what is not is going to be denoted by this notation.
This is the conditional probability that the event A is going to occur, the probability that the outcome is going to fall inside the set A given that we are told and we're sure that the event lies inside the event B Now once you're told that the outcome lies inside the event B, then our old sample space in some ways is irrelevant.
We have then you sample space, which is just the set B.
We are certain that the outcome is going to be inside B.
For example, what is this conditional probability?
It should be one.
Given that I told you that B occurred, you're certain that B occurred, so this has unit probability.
So here we see an instance of revision of our beliefs.
Initially, event B had the probability of (2+1)/6 -- that's 1/2.
Initially, we thought B had probability 1/2.
Once we're told that B occurred, the new probability of B is equal to one.
OK. How do we revise the probability that A occurs?
So we are going to have the outcome of the experiment.
We know that it's inside B.
So we will either get something here, and A does not occur.
Or something inside here, and A does occur.
What's the likelihood that, given that we're inside B, the outcome is inside here?
Here's how we're going to think about.
This part of this set B, in which A also occurs, in our initial model was twice as likely as that part of B.
So outcomes inside here collectively were twice as likely as outcomes out there.
So we're going to keep the same proportions and say, that given that we are inside the set B, we still want outcomes inside here to be twice as likely outcomes there.
So the proportion of the probabilities should be two versus one.
And these probabilities should add up to one because together they make the conditional probability of B.
So the conditional probabilities should be 2/3 probability of being here and 1/3 probability of being there.
That's how we revise our probabilities.
That's a reasonable, intuitively reasonable, way of doing this revision.
Let's translate what we did into a definition.
The definition says the following, that the conditional probability of A given that B occurred is calculated as follows.
We look at the total probability of B.
And out of that probability that was inside here, what fraction of that probability is assigned to points for which the event A also occurs?
Does it give us the same numbers as we got with this heuristic argument?
Well in this example, probability of A intersection B is 2/6, divided by total probability of B, which is 3/6, and so it's 2/3, which agrees with this answer that's we got before.
So the former indeed matches what we were trying to do.
One little technical detail.
If the event B has zero probability, and then here we have a ratio that doesn't make sense.
So in this case, we say that conditional probabilities are not defined.
Now you can take this definition and unravel it and write it in this form.
The probability of A intersection B is the probability of B times the conditional probability.
So this is just consequence of the definition but it has a nice interpretation.
Think of probabilities as frequencies.
If I do the experiment over and over, what fraction of the time is it going to be the case that both A and B occur?
Well, there's going to be a certain fraction of the time at which B occurs.
And out of those times when B occurs, there's going to be a further fraction of the experiments in which A also occurs.
So interpret the conditional probability as follows.
You only look at those experiments at which B happens to occur.
And look at what fraction of those experiments where B already occurred, event A also occurs.
And there's a symmetrical version of this equality.
There's symmetry between the events B and A.
So you also have this relation that goes the other way.
OK, so what do we use these conditional probabilities for?
First, one comment.
Conditional probabilities are just like ordinary probabilities.
They're the new probabilities that apply in a new universe where event B is known to have occurred.
So we had an original probability model.
We are told that B occurs.
We revise our model.
Our new model should still be legitimate probability model.
So it should satisfy all sorts of properties that ordinary probabilities do satisfy.
So for example, if A and B are disjoint events, then we know that the probability of A union B is equal to the probability of A plus probability of B.
And now if I tell you that a certain event C occurred, we're placed in a new universe where event C occurred.
We have new probabilities for that universe.
These are the conditional probabilities.
And conditional probabilities also satisfy this kind of property.
So this is just our usual additivity axiom but the applied in a new model, in which we were told that event C occurred.
So conditional probabilities do not taste or smell any different than ordinary probabilities do.
Conditional probabilities, given a specific event B, just form a probability law on our sample space.
It's a different probability law but it's still a probability law that has all of the desired properties.
OK, so where do conditional probabilities come up?
They do come up in quizzes and they do come up in silly problems.
So let's start with this.
We have this example from last time.
Two rolls of a die, all possible pairs of roles are equally likely, so every element in this square has probability of 1/16.
So all elements are equally likely.
That's our original model.
Then somebody comes and tells us that the minimum of the two rolls is equal to zero.
What's that event?
The minimum equal to zero can happen in many ways, if we get two zeros or if we get a zero and-- sorry, if we get two two's, or get a two and something larger.
And so the is our new event B.
The red event is the event B.
And now we want to calculate probabilities inside this new universe.
For example, you may be interested in the question, questions about the maximum of the two rolls.
In the new universe, what's the probability that the maximum is equal to one?
The maximum being equal to one is this black event.
And given that we're told that B occurred, this black events cannot happen.
So this probability is equal to zero.
How about the maximum being equal to two, given that event B?
OK, we can use the definition here.
It's going to be the probability that the maximum is equal to two and B occurs divided by the probability of B.
The probability that the maximum is equal to two.
OK, what's the event that the maximum is equal to two?
Let's draw it.
This is going to be the blue event.
The maximum is equal to two if we get any of those blue points.
So the intersection of the two events is the intersection of the red event and the blue event.
There's only one point in their intersection.
So the probability of that intersection happening is 1/16.
That's the numerator.
How about the denominator?
The event B consists of five elements, each one of which had probability of 1/16.
So that's 5/16.
And so the answer is 1/5.
Could we have gotten this answer in a faster way?
Yes.
Here's how it goes.
We're trying to find the conditional probability that we get this point, given that B occurred.
B consist of five elements.
All of those five elements were equally likely when we started, so they remain equally likely afterwards.
Because when we define conditional probabilities, we keep the same proportions inside the set.
So the five red elements were equally likely.
They remain equally likely in the conditional world.
So conditional event B having happened, each one of these five elements has the same probability.
So the probability that we actually get this point is going to be 1/5.
And so that's the shortcut.
More generally, whenever you have a uniform distribution on your initial sample space, when you condition on an event, your new distribution is still going to be uniform, but on the smaller events of that we considered.
So we started with a uniform distribution on the big square and we ended up with a uniform distribution just on the red point.
Now besides silly problems, however, conditional probabilities show up in real and interesting situations.
And this example is going to give you some idea of how that happens.
OK. Actually, in this example, instead of starting with a probability model in terms of regular probabilities, I'm actually going to define the model in terms of conditional probabilities.
And we'll see how this is done.
So here's the story.
There may be an airplane flying up in the sky, in a particular sector of the sky that you're watching.
Sometimes there is one sometimes there isn't.
And from experience you know that when you look up, there's five percent probability that the plane is flying above there and 95% probability that there's no plane up there.
So event A is the event that the plane is flying out there.
Now you bought this wonderful radar that's looks up.
And you're told in the manufacturer's specs that, if there is a plane out there, your radar is going to register something, a blip on the screen with probability 99%.
And it will not register anything with probability one percent.
So this particular part of the picture is a self-contained probability model of what your radar does in a world where a plane is out there.
So I'm telling you that the plane is out there.
So we're now dealing with conditional probabilities because I gave you some particular information.
Given this information that the plane is out there, that's how your radar is going to behave with probability 99% is going to detect it, with probability one percent is going to miss it.
So this piece of the picture is a self-contained probability model.
The probabilities add up to one.
But it's a piece of a larger model.
Similarly, there's the other possibility.
Maybe a plane is not up there and the manufacturer specs tell you something about false alarms.
A false alarm is the situation where the plane is not there, but for some reason your radar picked up some noise or whatever and shows a blip on the screen.
And suppose that this happens with probability ten percent.
Whereas with probability 90% your radar gives the correct answer.
So this is sort of a model of what's going to happen with respect to both the plane -- we're given probabilities about this -- and we're given probabilities about how the radar behaves.
So here I have indirectly specified the probability law in our model by starting with conditional probabilities as opposed to starting with ordinary probabilities.
Can we derive ordinary probabilities starting from the conditional number ones?
Yeah, we certainly can.
Let's look at this event, A intersection B, which is the event up here, that there is a plane and our radar picks it up.
How can we calculate this probability?
Well we use the definition of conditional probabilities and this is the probability of A times the conditional probability of B given A.
So it's 0.05 times 0.99.
And the answer, in case you care-- It's 0.0495.
OK.
So we can calculate the probabilities of final outcomes, which are the leaves of the tree, by using the probabilities that we have along the branches of the tree.
So essentially, what we ended up doing was to multiply the probability of this branch times the probability of that branch.
Now, how about the answer to this question.
What is the probability that our radar is going to register something?
OK, this is an event that can happen in multiple ways.
It's the event that consists of this outcome.
There is a plane and the radar registers something together with this outcome, there is no plane but the radar still registers something.
So to find the probability of this event, we need the individual probabilities of the two outcomes.
For the first outcome, we already calculated it.
For the second outcome, the probability that this happens is going to be this probability 95% times 0.10, which is the conditional probability for taking this branch, given that there was no plane out there.
So we just add the numbers.
0.05 times 0.99 plus 0.95 times 0.1 and the final answer is 0.1445.
OK. And now here's the interesting question.
Given that your radar recorded something, how likely is it that there is an airplane up there?
Your radar registering something -- that can be caused by two things.
Either there's a plane there, and your radar did its job.
Or there was nothing, but your radar fired a false alarm.
What's the probability that this is the case as opposed to that being the case?
OK.
The intuitive shortcut would be that it should be the probability-- you look at their relative odds of these two elements and you use them to find out how much more likely it is to be there as opposed to being there.
But instead of doing this, let's just write down the definition and just use it.
It's the probability of A and B happening, divided by the probability of B.
This is just our definition of conditional probabilities.
Now we have already found the numerator.
We have already calculated the denominator.
So we take the ratio of these two numbers and we find the final answer -- which is 0.34.
OK.
There's this slightly curious thing that's happened in this example.
Doesn't this number feel a little too low?
My radar -- So this is a conditional probability, given that my radar said there is something out there, that there is indeed something there.
So it's sort of the probability that our radar gave the correct answer.
Now, the specs of our radar we're pretty good.
In this situation, it gives you the correct answer 99% of the time.
In this situation, it gives you the correct answer 90% of the time.
So you would think that your radar there is really reliable.
But yet here the radar recorded something, but the chance that the answer that you get out of this is the right one, given that it recorded something, the chance that there is an airplane out there is only 30%.
So you cannot really rely on the measurements from your radar, even though the specs of the radar were really good.
What's the reason for this?
Well, the reason is that false alarms are pretty common.
Most of the time there's nothing.
And there's a ten percent probability of false alarms.
So there's roughly a ten percent probability that in any given experiment, you have a false alarm.
And there is about the five percent probability that something out there and your radar gets it.
So when your radar records something, it's actually more likely to be a false alarm rather than being an actual airplane.
This has probability ten percent roughly.
This has probability roughly five percent So conditional probabilities are sometimes counter-intuitive in terms of the answers that they get.
And you can make similar stories about doctors interpreting the results of tests.
So you tested positive for a certain disease.
Does it mean that you have the disease necessarily?
Well if that disease has been eradicated from the face of the earth, testing positive doesn't mean that you have the disease, even if the test was designed to be a pretty good one.
So unfortunately, doctors do get it wrong also sometimes.
And the reasoning that comes in such situations is pretty subtle.
Now for the rest of the lecture, what we're going to do is to take this example where we did three things and abstract them.
These three trivial calculations that's we just did are three very important, very basic tools that you use to solve more general probability problems.
So what's the first one?
We find the probability of a composite event, two things happening, by multiplying probabilities and conditional probabilities.
More general version of this, look at any situation, maybe involving lots and lots of events.
So here's a story that event A may happen or may not happen.
Given that A occurred, it's possible that B happens or that B does not happen.
Given that B also happens, it's possible that the event C also happens or that event C does not happen.
And somebody specifies for you a model by giving you all these conditional probabilities along the way.
Notice what we move along the branches as the tree progresses.
Any point in the tree corresponds to certain events having happened.
And then, given that this has happened, we specify conditional probabilities.
Given that this has happened, how likely is it for that C also occurs?
Given a model of this kind, how do we find the probability or for this event?
The answer is extremely simple.
All that you do is move along with the tree and multiply conditional probabilities along the way.
So in terms of frequencies, how often do all three things happen, A, B, and C?
You first see how often does A occur.
Out of the times that A occurs, how often does B occur?
And out of the times where both A and B have occurred, how often does C occur?
And you can just multiply those three frequencies with each other.
What is the formal proof of this?
Well, the only thing we have in our hands is the definition of conditional probabilities.
So let's just use this.
And-- OK. Now, the definition of conditional probabilities tells us that the probability of two things is the probability of one of them times a conditional probability.
Unfortunately, here we have the probability of three things.
What can I do?
I can put a parenthesis in here and think of this as the probability of this and that and apply our definition of conditional probabilities here.
The probability of two things happening is the probability that the first happens times the conditional probability that the second happens, given A and B, given that the first one happened.
So this is just the definition of the conditional probability of an event, given another event.
That other event is a composite one, but that's not an issue.
It's just an event.
And then we use the definition of conditional probabilities once more to break this apart and make it P(A), P(B given A) and then finally, the last term.
OK.
So this proves the formula that I have up there on the slides.
And if you wish to calculate any other probability in this diagram.
For example, if you want to calculate this probability, you would still multiply the conditional probabilities along the different branches of the tree.
In particular, here in this branch, you would have the conditional probability of C complement, given A intersection B complement, and so on.
So you write down probabilities along all those tree branches and just multiply them as you go.
So this was the first skill that we are covering.
What was the second one?
What we did was to calculate the total probability of a certain event B that consisted of-- was made up from different possibilities, which corresponded to different scenarios.
So we wanted to calculate the probability of this event B that consisted of those two elements.
Let's generalize.
So we have our big model.
And this sample space is partitioned in a number of sets.
In our radar example, we had a partition in two sets.
Either a plane is there, or a plane is not there.
Since we're trying to generalize, now I'm going to give you a picture for the case of three possibilities or three possible scenarios.
So whatever happens in the world, there are three possible scenarios, A1, A2, A3.
So think of these as there's nothing in the air, there's an airplane in the air, or there's a flock of geese flying in the air.
So there's three possible scenarios.
And then there's a certain event B of interest, such as a radar records something or doesn't record something.
We specify this model by giving probabilities for the Ai's-- That's the probability of the different scenarios.
And somebody also gives us the probabilities that this event B is going to occur, given that the Ai-th scenario has occurred.
Think of the Ai's as scenarios.
And we want to calculate the overall probability of the event B.
What's happening in this example?
Perhaps, instead of this picture, it's easier to visualize if I go back to the picture I was using before.
We have three possible scenarios, A1, A2, A3.
And under each scenario, B may happen or B may not happen.
And so on.
So here we have A2 intersection B.
And here we have A3 intersection B.
In the previous slide, we found how to calculate the probability of any event of this kind, which is done by multiplying probabilities here and conditional probabilities there.
Now we are asked to calculate the total probability of the event B.
The event B can happen in three possible ways.
It can happen here.
It can happen there.
And it can happen here.
So this is our event B.
It consists of three elements.
To calculate the total probability of our event B, all we need to do is to add these three probabilities.
So B is an event that consists of these three elements.
There are three ways that B can happen.
Either B happens together with A1, or B happens together with A2, or B happens together with A3.
So we need to add the probabilities of these three contingencies.
For each one of those contingencies, we can calculate its probability by using the multiplication rule.
So the probability of A1 and B happening is this-- It's the probability of A1 and then B happening given that A1 happens.
The probability of this contingency is found by taking the probability that A2 happens times the conditional probability of A2, given that B happened.
And similarly for the third one.
So this is the general rule that we have here.
The rule is written for the case of three scenarios.
But obviously, it has a generalization for the case of four or five or more scenarios.
It gives you a way of breaking up the calculation of an event that can happen in multiple ways by considering individual probabilities for the different ways that the event can happen.
OK.
So-- Yes?
Does this have to change for infinite sample space?
No.
This is true whether your sample space is infinite or finite.
What I'm using in this argument that we have a partition into just three scenarios, three events.
So it's a partition to a finite number of events.
It's also true if it's a partition into an infinite sequence of events.
But that's, I think, one of the theoretical problems at the end of the chapter.
You probably may not need it for now.
OK, going back to the story here.
There are three possible scenarios about what could happen in the world that are captured here.
Event, under each scenario, event B may or may not happen.
And so these probabilities tell us the likelihoods of the different scenarios.
These conditional probabilities tell us how likely is it for B to happen under one scenario, or the other scenario, or the other scenario.
The overall probability of B is found by taking some combination of the probabilities of B in the different possible worlds, in the different possible scenarios.
Under some scenario, B may be very likely.
Under another scenario, it may be very unlikely.
We take all of these into account and weigh them according to the likelihood of the scenarios.
Now notice that since A1, A2, and three form a partition, these three probabilities have what property?
Add to what?
They add to one.
So it's the probability of this branch, plus this branch, plus this branch.
So what we have here is a weighted average of the probabilities of the B's into the different worlds, or in the different scenarios.
Special case.
Suppose the three scenarios are equally likely.
So P of A1 equals 1/3, equals to P of A2, P of A3.
what are we saying here?
In that case of equally likely scenarios, the probability of B is the average of the probabilities of B in the three different words, or in the three different scenarios.
OK.
So to finally, the last step.
If we go back again two slides, the last thing that we did was to calculate a conditional probability of this kind, probability of A given B, which is a probability associated essentially with an inference problem.
Given that our radar recorded something, how likely is it that the plane was up there?
So we're trying to infer whether a plane was up there or not, based on the information that we've got.
So let's generalize once more.
And we're just going to rewrite what we did in that example, but in terms of general symbols instead of the specific numbers.
So once more, the model that we have involves probabilities of the different scenarios.
These we call them prior probabilities.
They're are our initial beliefs about how likely each scenario is to occur.
We also have a model of our measuring device that tells us under that scenario how likely is it that our radar will register something or not.
So we're given again these conditional probabilities.
We're given the conditional probabilities for these branches.
Then we are told that event B occurred.
And on the basis of this new information, we want to form some new beliefs about the relative likelihood of the different scenarios.
Going back again to our radar example, an airplane was present with probability 5%.
Given that the radar recorded something, we're going to change our beliefs.
Now, a plane is present with probability 34%.
The radar, since we saw something, we are going to revise our beliefs as to whether the plane is out there or is not there.
And so what we need to do is to calculate the conditional probabilities of the different scenarios, given the information that we got.
So initially, we have these probabilities for the different scenarios.
Once we get the information, we update them and we calculate our revised probabilities or conditional probabilities given the observation that we made.
OK.
So what do we do?
We just use the definition of conditional probabilities twice.
By definition the conditional probability is the probability of two things happening divided by the probability of the conditioning event.
Now, I'm using the definition of conditional probabilities once more, or rather I use the multiplication rule.
The probability of two things happening is the probability of the first and the second.
So these are things that are given to us.
They're the probabilities of the different scenarios.
And it's the model of our measuring device, which we assume to be available.
And how about the denominator?
This is total probability of the event B.
But we just found that's it's easy to calculate using the formula in the previous slide.
To find the overall probability of event B occurring, we look at the probabilities of B occurring under the different scenario and weigh them according to the probabilities of all the scenarios.
So in the end, we have a formula for the conditional probability, A's given B, based on the data of the problem, which were probabilities of the different scenarios and conditional probabilities of B, given the A's.
So what this calculation does is, basically, it reverses the order of conditioning.
We are given conditional probabilities of these kind, where it's B given A and we produce new conditional probabilities, where things go the other way.
So schematically, what's happening here is that we have model of cause and effect and-- So a scenario occurs and that may cause B to happen or may not cause it to happen.
So this is a cause/effect model.
And it's modeled using probabilities, such as probability of B given Ai.
And what we want to do is inference where we are told that B occurs, and we want to infer whether Ai also occurred or not.
And the appropriate probabilities for that are the conditional probabilities that A occurred, given that B occurred.
So we're starting with a causal model of our situation.
It models from a given cause how likely is a certain effect to be observed.
And then we do inference, which answers the question, given that the effect was observed, how likely is it that the world was in this particular situation or state or scenario.
So the name of the Bayes rule comes from Thomas Bayes, a British theologian back in the 1700s.
It actually-- This calculation addresses a basic problem, a basic philosophical problem, how one can learn from experience or from experimental data and some systematic way.
So the British at that time were preoccupied with this type of question.
Is there a basic theory that about how we can incorporate new knowledge to previous knowledge.
And this calculation made an argument that, yes, it is possible to do that in a systematic way.
So the philosophical underpinnings of this have a very long history and a lot of discussion around them.
But for our purposes, it's just an extremely useful tool.
And it's the foundation of almost everything that gets done when you try to do inference based on partial observations.
Very well.
Till next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let us start.
So as always, we're to have a quick review of what we discussed last time.
And then today we're going to introduce just one new concept, the notion of independence of two events.
And we will play with that concept.
So what did we talk about last time?
The idea is that we have an experiment, and the experiment has a sample space omega.
And then somebody comes and tells us you know the outcome of the experiments happens to lie inside this particular event B.
Given this information, it kind of changes what we know about the situation.
It tells us that the outcome is going to be somewhere inside here.
So this is essentially our new sample space.
And now we need to we reassign probabilities to the various possible outcomes, because, for example, these outcomes, even if they had positive probability beforehand, now that we're told that B occurred, those outcomes out there are going to have zero probability.
So we need to revise our probabilities.
The new probabilities are called conditional probabilities, and they're defined this way.
The conditional probability that A occurs given that we're told that B occurred is calculated by this formula, which tells us the following-- out of the total probability that was initially assigned to the event B, what fraction of that probability is assigned to outcomes that also make A to happen?
So out of the total probability assigned to B, we see what fraction of that total probability is assigned to those elements here that will also make A happen.
Conditional probabilities are left undefined if the denominator here is zero.
An easy consequence of the definition is if we bring that term to the other side, then we can find the probability of two things happening by taking the probability that the first thing happens, and then, given that the first thing happened, the conditional probability that the second one happens.
Then we saw last time that we can divide and conquer in calculating probabilities of mildly complicated events by breaking it down into different scenarios.
So event B can happen in two ways.
It can happen either together with A, which is this probability, or it can happen together with A complement, which is this probability.
So basically what we're saying that the total probability of B is the probability of this, which is A intersection B, plus the probability of that, which is A complement intersection B.
So these two facts here, multiplication rule and the total probability theorem, are basic tools that one uses to break down probability calculations into a simpler parts.
So we find probabilities of two things happening by looking at each one at a time.
And this is what we do to break up a situation with two different possible scenarios.
Then we also have the Bayes rule, which does the following.
Given a model that has conditional probabilities of this kind, the Bayes rule allows us to calculate conditional probabilities in which the events appear in different order.
You can think of these probabilities as describing a causal model of a certain situation, whereas these are the probabilities that you get after you do some inference based on the information that you have available.
Now the Bayes rule, we derived it, and it's a trivial half-line calculation.
But it underlies lots and lots of useful things in the real world.
We had the radar example last time.
You can think of more complicated situations in which there's a bunch or lots of different hypotheses about the environment.
Given any particular setting in the environment, you have a measuring device that can produce many different outcomes.
And you observe the final outcome out of your measuring device, and you're trying to guess which particular branch occurred.
That is, you're trying to guess the state of the world based on a particular measurement.
That's what inference is all about.
So real world problems only differ from the simple example that we saw last time in that this kind of tree is a little more complicated.
You might have infinitely many possible outcomes here and so on.
So setting up the model may be more elaborate, but the basic calculation that's done based on the Bayes rule is essentially the same as the one that we saw.
Now something that we discuss is that sometimes we use conditional probabilities to describe models, and let's do this by looking at a model where we toss a coin three times.
And how do we use conditional probabilities to describe the situation?
So we have one experiment.
But that one experiment consists of three consecutive coin tosses.
So the possible outcomes, our sample space, consists of strings of length 3 that tell us whether we had heads, tails, and in what sequence.
So three heads in a row is one particular outcome.
So what is the meaning of those labels in front of the branches?
So this P here, of course, stands for the probability that the first toss resulted in heads.
And let me use this notation to denote that the first was heads.
I put an H in toss one.
How about the meaning of this probability here?
Well the meaning of this probability is a conditional one.
It's the conditional probability that the second toss resulted in heads, given that the first one resulted in heads.
And similarly this label here corresponds to the probability that the third toss resulted in heads, given that the first one and the second one resulted in heads.
So in this particular model that I wrote down here, those probabilities, P, of obtaining heads remain the same no matter what happened in the previous toss.
For example, even if the first toss was tails, we still have the same probability, P, that the second one is heads, given that the first one was tails.
So we're assuming that no matter what happened in the first toss, the second toss will still have a conditional probability equal to P. So that conditional probability does not depend on what happened in the first toss.
And we will see that this is a very special situation, and that's really the concept of independence that we are going to introduce shortly.
But before we get to independence, let's practice once more the three skills that we covered last time in this example.
So first skill was multiplication rule.
How do you find the probability of several things happening?
That is the probability that we have tails followed by heads followed by tails.
So here we're talking about this particular outcome here, tails followed by heads followed by tails.
And the way we calculate such a probability is by multiplying conditional probabilities along the path that takes us to this outcome.
And so these conditional probabilities are recorded here.
So it's going to be (1 minus P) times P times (1 minus P).
So this is the multiplication rule.
Second question is how do we find the probability of a mildly complicated event?
So the event of interest here that I wrote down is the probability that in the three tosses, we had a total of one head.
Exactly one head.
This is an event that can happen in multiple ways.
It happens here.
It happens here.
And it also happens here.
So we want to find the total probability of the event consisting of these three outcomes.
What do we do?
We just add the probabilities of each individual outcome.
How do we find the probability of an individual outcome?
Well, that's what we just did.
Now notice that this outcome has probability P times (1 minus P) squared.
That one should not be there.
So where is it?
Ah.
It's this one.
OK, so the probability of this outcome is (1 minus P times P) times (1 minus P), the same probability.
And finally, this one is again (1 minus P) squared times P. So this event of one head can happen in three ways.
And each one of those three ways has the same probability of occurring.
And this is the answer.
And finally, the last thing that we learned how to do is to use the Bayes rule to calculate and make an inference.
So somebody tells you that there was exactly one head in your three tosses.
What is the probability that the first toss resulted in heads?
OK, I guess you can guess the answer here if I tell you that there were three tosses.
One of them was heads.
Where was that head in the first, the second, or the third?
Well, by symmetry, they should all be equally likely.
So there should be probably just 1/3 that that head occurred in the first toss.
Let's check our intuition using the definitions.
So the definition of conditional probability tells us the conditional probability is the probability of both things happening.
First toss is heads, and we have exactly one head divided by the probability of one head.
What is the probability that the first toss is heads, and we have exactly one head?
This is the same as the event heads, tails, tails.
If I tell you that the first is heads, and there's only one head, it means that the others are tails.
So this is the probability of heads, tails, tails divided by the probability of one head.
And we know all of these quantities probability of heads, tails, tails is P times (1 minus P) squared.
Probability of one head is 3 times P times (1 minus P) squared.
So the final answer is 1/3, which is what you should have a guessed on intuitive grounds.
Very good.
So we got our practice on the material that we did cover last time.
Again, think.
There's basically three basic skills that we are practicing and exercising here.
In the problems, quizzes, and in the real life, you may have to apply those three skills in somewhat more complicated settings, but in the end that's what it boils down to usually.
Now let's focus on this special feature of this particular model that I discussed a little earlier.
Think of the event heads in the second toss.
Initially, the probability of heads in the second toss, you know, that it's P, the probability of success of your coin.
If I tell you that the first toss resulted in heads, what's the probability that the second toss is heads?
It's again P. If I tell you that the first toss was tails, what's the probability that the second toss is heads?
It's again P. So whether I tell you the result of the first toss, or I don't tell you, it doesn't make any difference to you.
You would always say the probability of heads in the second toss is going to P, no matter what happened in the first toss.
This is a special situation to which we're going to give a name, and we're going to call that property independence.
Basically independence between two things stands for the fact that the first thing, whether it occurred or not, doesn't give you any information, does not cause you to change your beliefs about the second event.
This is the intuition.
Let's try to translate this into mathematics.
We have two events, and we're going to say that they're independent if your initial beliefs about B are not going to change if I tell you that A occurred.
So you believe something how likely B is.
Then somebody comes and tells you, you know, A has happened.
Are you going to change your beliefs?
No, I'm not going to change them.
Whenever you are in such a situation, then you say that the two events are independent.
Intuitively, the fact that A occurred does not convey any information to you about the likelihood of event B.
The information that A provides is not so useful, is not relevant.
A has to do with something else.
It's not useful for your guessing whether B is going to occur or not.
So we can take this as a first attempt into a definition of independence.
Now remember that we have this property, the probability of two things happening is the probability of the first times the conditional probability of the second.
If we have independence, this conditional probability is the same as the unconditional probability.
So if we have independence according to that definition, we get this property that you can find the probability of two things happening by just multiplying their individual probabilities.
Probability of heads in the first toss is 1/2.
Probability of heads in the second toss is 1/2.
Probability of heads heads is 1/4.
That's what happens if your two tosses are independent of each other.
So this property here is a consequence of this definition, but it's actually nicer, better, simpler, cleaner, more beautiful to take this as our definition instead of that one.
Are the two definitions equivalent?
Well, they're are almost the same, except for one thing.
Conditional probabilities are only defined if you condition on an event that has positive probability.
So this definition would be limited to cases where event A has positive probability, whereas this definition is something that you can write down always.
We will say that two events are independent if and only if their probability of happening simultaneously is equal to the product of their two individual probabilities.
And in particular, we can have events of zero probability.
There's nothing wrong with that.
If A has 0 probability, then A intersection B will also have zero probability, because it's an even smaller event.
And so we're going to get zero is equal to zero.
A corollary of what I just said, if an event A has zero probability, it's actually independent of any other event in our model, because we're going to get zero is equal to zero.
And the definition is going to be satisfied.
This is a little bit harder to reconcile with the intuition we have about independence, but then again, it's part of the mathematical definition.
So what I want you to retain is this notion that the independence is something that you can check formally using this definition, but also you can check intuitively by if, in some cases, you can reason that whatever happens and determines whether A is going to occur or not, has nothing absolutely to do with whatever happens and determines whether B is going to occur or not.
So if I'm doing a science experiment in this room, and it gets hit by some noise that's causes randomness.
And then five years later, somebody somewhere else does the same science experiment somewhere else, it gets hit by other noise, you would usually say that these experiments are independent.
So what events happen in one experiment are not going to change your beliefs about what might be happening in the other, because the sources of noise in these two experiments are completely unrelated.
They have nothing to do with each other.
So if I flip a coin here today, and I flip a coin in my office tomorrow, one shouldn't affect the other.
So the events that I get from these should be independent.
So that's usually how independence arises.
By having distinct physical phenomena that do not interact.
Sometimes you also get independence even though there is a physical interaction, but you just happen to have a numerical accident.
A and B might be physically related very tightly, but a numerical accident happens and you get equality here, that's another case where we do get independence.
Now suppose that we have two events that are laid out like this.
Are these two events independent or not?
The picture kind of tells you that one is separate from the other.
But separate has nothing to do with independent.
In fact, these two events are as dependent as Siamese twins.
Why is that?
If I tell you that A occurred, then you are certain that B did not occur.
So information about the occurrence of A definitely affects your beliefs about the possible occurrence or non-occurrence of B.
When the picture is like that, knowing that A occurred will change drastically my beliefs about B, because now I suddenly become certain that B did not occur.
So a picture like this is a case actually of extreme dependence.
So don't confuse independence with disjointness.
They're very different types of properties
Question
Yes?
So I understand the explanation, but the probability of A intersect B [INAUDIBLE] to zero, because they're disjoint
Yes
But then the product of probability A and probability B, one of them is going to be 1.
[INAUDIBLE]
No, suppose that the probabilities are 1/3, 1/4, and the rest is out there.
You check the definition of independence.
Probability of A intersection B is zero.
Probability of A times the probability of B is 1/12.
The two are not equal.
Therefore we do not have independence
Right.
So what's wrong with the intuition of the probability of A being 1, and the other one being 0?
[INAUDIBLE]
No.
The probability of A given B is equal to 0.
Probability of A is equal to 1/3.
So again, these two are different.
So we had some initial beliefs about A, but as soon as we are told that B occurred, our beliefs about A changed.
And so since our beliefs changed, that means that B conveys information about
So can you not draw independent [INAUDIBLE] on a Venn diagram?
I can't hear you
Can you draw independence on a Venn diagram?
No, the Venn diagram is never enough to decide independence.
So the typical picture in which you're going to have independence would be one event this way, and another event this way.
You need to take the probability of this times the probability of that, and check that, numerically, it's equal to the probability of this intersection.
So it's more than a Venn diagram.
Numbers need to come out right.
Now we did say some time ago that conditional probabilities are just like ordinary probabilities, and whatever we do in probability theory can also be done in conditional universes.
Talking about conditional probabilities.
So since we have a notion of independence, then there should be also a notion of conditional independence.
So independence was defined by the probability that A intersection B is equal to the probability of A times the probability of B.
What would be a reasonable definition of conditional independence?
Conditional independence would mean that this same property could be true, but in a conditional universe where we are told that the certain event happens.
So if we're told that the event C has happened, then were transported in a conditional universe where the only thing that matters are conditional probabilities.
And this is just the same plain, previous definition of independence, but applied in a conditional universe.
So this is the definition of conditional independence.
So it's independence, but with reference to the conditional probabilities.
And intuitively it has, again, the same meaning, that in the conditional world, if I tell you that A occurred, then that doesn't change your beliefs about B.
So suppose you had a picture like this.
And somebody told you that events A and B are independent unconditionally.
Then somebody comes and tells you that event C actually has occurred, so we now live in this new universe.
In this new universe, is the independence of A and B going to be preserved or not?
Are A and B independent in this new universe?
The answer is no, because in the new universe, whatever is left of event A is this piece.
Whatever is left of event B is this piece.
And these two pieces are disjoint.
So we are back in a situation of this kind.
So in the conditional universe, A and B are disjoint.
And therefore, generically, they're not going to be independent.
What's the moral of this example?
Having independence in the original model does not imply independence in a conditional model.
The opposite is also possible.
And let's illustrate by another example.
So I have two coins, and both of them are badly biased.
One coin is much biased in favor of heads.
The other coin is much biased in favor of tails.
So the probabilities being 90%.
Let's consider independent flips of coin A.
This is the relevant model.
This is a model of two independent flips of the first coin.
There's going to be two flips, and each one has probability 0.9 of being heads.
So that's a model that describes coin A.
You can think of this as a conditional model which is a model of the coin flips conditioned on the fact that they have chosen coin A. Alternatively we could be dealing with coin B In a conditional world where we chose coin B and flip it twice, this is the relevant model.
The probability of two heads, for example, is the probability of heads the first time, heads the second time, and each one is 0.1.
Now I'm building this into a bigger experiment in which I first start by choosing one of the two coins at random.
So I have these two coins.
I blindly pick one of them.
And then I start flipping them.
So the question now is, are the coin flips, or the coin tosses, are they independent of each other?
If we just stay inside this sub-model here, are the coin flips independent?
They are independent, because the probability of heads in the second toss is the same, 0.9, no matter what happened in the first toss.
So the conditional probabilities of what happens in the second toss are not affected by the outcome of the first toss.
So the second toss and the first toss are independent.
So here we're just dealing with plain, independent coin flips.
Similarity the coin flips within this sub-model are also independent.
Now the question is, if we look at the big model as just one probability model, instead of looking at the conditional sub-models, are the coin flips independent of each other?
Does the outcome of a few coin flips give you information about subsequent coin flips?
Well if I observe ten heads in a row-- So instead of two coin flips, now let's think of doing more of them so that the tree gets expanded.
So let's start with this.
I don't know which coin it is.
What's the probability that the 11th coin toss is going to be heads?
There's complete symmetry here, so the answer could not be anything other than 1/2.
So let's justify it, why is it 1/2?
Well, the probability that the 11th toss is heads, how can that outcome happen?
It can happen in two ways.
You can choose coin A, which happens with probability 1/2.
And having chosen coin A, there's probability 0.9 that it results in that you get heads in the 11th toss.
Or you can choose coin B.
And if it's coin B when you flip it, there's probably 0.1 that you have heads.
So the final answer is 1/2.
So each one of the coins is biased, but they're biased in different ways.
If I don't know which coin it is, their two biases kind of cancel out, and the probability of obtaining heads is just in the middle, then it's 1/2.
Now if someone tells you that the first ten tosses were heads, is that going to change your beliefs about the 11th toss?
Here's how a reasonable person would think about it.
If it's coin B the probability of obtaining 10 heads in a row is negligible.
It's going to be 0.1 to the 10th.
If it's coin A.
The probability of 10 heads in a row is a more reasonable number.
It's 0.9 to the 10th.
So this event is a lot more likely to occur with coin A, rather than coin B.
The plausible explanation of having seen ten heads in a row is that I actually chose coin A.
When you see ten heads in a row, you are pretty certain that it's coin A that we're dealing with.
And once you're pretty certain that it's coin A that we're dealing with, what's the probability that the next toss is heads?
It's going to be 0.9.
So essentially here I'm doing an inference calculation.
Given this information, I'm making an inference about which coin I'm dealing with.
I become pretty certain that it's coin A, and given that it's coin A, this probability is going to be 0.9.
And I'm putting an approximate sign here, because the inference that I did is approximate.
I'm pretty certain it's coin A. I'm not 100% certain that it's coin A.
But in any case what happens here is that the unconditional probability is different from the conditional probability.
This information here makes me change my beliefs about the 11th toss.
And this means that the 11th toss is dependent on the previous tosses.
So the coin tosses have now become dependent.
What is the physical link that causes this dependence?
Well, the physical link is the choice of the coin.
By choosing a particular coin, I'm introducing a pattern in the future coin tosses.
And that pattern is what causes dependence.
OK, so I've been playing a little bit too loose with the language here, because we defined the concept of independence of two events.
But here I have been referring to independent coin tosses, where I'm thinking about many coin tosses, like 10 or 11 of them.
So to be proper, I should have defined for you also the notion of independence of multiple events, not just two.
We don't want to just say coin toss one is independent from coin toss two.
We want to be able to say something like, these 10 then coin tosses are all independent of each other.
Intuitively what that means should be the same thing-- that information about some of the coin tosses doesn't change your beliefs about the remaining coin tosses.
How do we translate that into a mathematical definition?
Well, an ugly attempt would be to impose requirements such as this.
Think of A1 being the event that the first flip was heads.
A2 is the event of that the second flip was heads.
A3, the third flip, was heads, and so on.
Here is an event whose occurrence is not determined by the first three coin flips.
And here's an event whose occurrence or not is determined by the fifth and sixth coin flip.
If we think physically that all those coin flips have nothing to do with each other, information about the fifth and sixth coin flip are not going to change what we expect from the first three.
So the probability of this event, the conditional probability, should be the same as the unconditional probability.
And we would like a relation of this kind to be true, no matter what kind of formula you write down, as long as the events that show up here are different from the events that show up there.
OK. That's sort of an ugly definition.
The mathematical definition that actually does the job, and leads to all the formulas of this kind, is the following.
We're going to say that the collection of events are independent if we can find the probability of their joint occurrence by just multiplying probabilities.
And that will be true even if you look at sub-collections of these events.
Let's make that more precise.
If we have three events, the definition tells us that the three events are independent if the following are true.
Probability A1 and A2 and A3, you can calculate this probability by multiplying individual probabilities.
But the same is true even if you take fewer events.
Just a few indices out of the indices that we have available.
So we also require P(A1 intersection A2) is P(A1) times P(A2).
And similarly for the other possibilities of choosing the indices.
OK, so independence, mathematical definition, requires that calculating probabilities of any intersection of the events we have in our hands, that calculation can be done by just multiplying individual probabilities.
And this has to apply to the case where we consider all of the events in our hands or just sub-collections of those events.
Now these relations just by themselves are called pairwise independence.
So this relation, for example, tells us that A1 is independent from A2.
This tells us that A2 is independent from A3.
This will tell us that A1 is independent from A3.
But independence of all the events together actually requires a little more.
One more equality that has to do with all three events being considered at the same time.
And this extra equality is not redundant.
It actually does make a difference.
Independence and pairwise independence are different things.
So let's illustrate the situation with an example.
Suppose we have two coin flips.
The coin tosses are independent, so the bias is 1/2, so all possible outcomes have a probability of 1/2 times 1/2, which is 1/4.
And let's consider now a bunch of different events.
One event is that the first toss is heads.
This is this blue set here.
Another event is the second toss is heads.
And this is this black event here.
OK. Are these two events independent?
If you check it mathematically, yes.
Probability of A is probability of B is 1/2.
Probability of A times probability of B is 1/4, which is the same as the probability of A intersection B, which is this set.
So we have just checked mathematically that A and B are independent.
Now lets consider a third event which is that the first and second toss give the same result.
I'll use a different color.
First and second toss to give the same result.
This is the event that we obtain heads, heads or tails, tails.
So this is the probability of C. What's the probability of C?
Well, C is made up of two outcomes, each one of which has probability 1/4, so the probability of C is 1/2.
What is the probability of C intersection A?
C intersection A is just this one outcome, and has probability 1/4.
What's the probability of A intersection B intersection C?
The three events intersect just this outcome, so this probability is also 1/4.
OK. What's the probability of C given A and B?
If A has occurred, and B has occurred, you are certain that this outcome here happened.
If the first toss is H and the second toss is H, then you're certain of the first and second toss gave the same result.
So the conditional probability of C given A and B is equal to 1.
So do we have independence in this example?
We don't.
C, that we obtain the same result in the first and the second toss, has probability 1/2.
Half of the possible outcomes give us two coin flips with the same result-- heads, heads or tails, tails.
So the probability of C is 1/2.
But if I tell you that the events A and B both occurred, then you're certain that C occurred.
If I tell you that we had heads and heads, then you're certain the outcomes were the same.
So the conditional probability is different from the unconditional probability.
So by combining these two relations together, we get that the three events are not independent.
But are they pairwise independent?
Is A independent from B?
Yes, because probability of A times probability of B is 1/4, which is probability of A intersection B.
Is C independent from A?
Well, the probability of C and A is 1/4.
The probability of C is 1/2.
The probability of A is 1/2.
So it checks.
1/4 is equal to 1/2 and 1/2, so event C and event A are independent.
Knowing that the first toss was heads does not change your beliefs about whether the two tosses are going to have the same outcome or not.
Knowing that the first was heads, well, the second is equally likely to be heads or tails.
So event C has just the same probability, again, 1/2, to occur.
To put it the opposite way, if I tell you that the two results were the same-- so it's either heads, heads or tails, tails-- what does that tell you about the first toss?
Is it heads, or is it tails?
Well, it doesn't tell you anything.
It could be either over the two, so the probability of heads in the first toss is equal to 1/2, and telling you C occurred does not change anything.
So this is an example that illustrates the case where we have three events in which we check that pairwise independence holds for any combination of two of these events.
We have the probability of their intersection is equal to the product of their probabilities.
On the other hand, the three events taken all together are not independent.
A doesn't tell me anything useful, whether C is going to occur or not.
B doesn't tell me anything useful.
But if I tell you that both A and B occurred, the two of them together tell me something useful about C. Namely, they tell me that C certainly has occurred.
Very good.
So independence is this somewhat subtle concept.
Once you grasp the intuition of what it really means, then things perhaps fall in place.
But it's a concept where it's easy to get some misunderstanding.
So just take some time to digest.
So to lighten things up, I'm going to spend the remaining four minutes talking about the very nice, simple problem that involves conditional probabilities and the like.
So here's the problem, formulated exactly as it shows up in various textbooks.
And the formulation says the following.
Well, consider one of those anachronistic places where they still have kings or queens, and where actually boys take precedence over girls.
So if there is a boy-- if the royal family has a boy, then he will become the king even if he has an older sister who might be the queen.
So we have one of those royal families.
That royal family had two children, and we know that there is a king.
There is a king, which means that at least one of the two children was a boy.
Otherwise we wouldn't have a king.
What is the probability that the king's sibling is female?
OK.
I guess we need to make some assumptions about genetics.
Let's assume that every child is a boy or a girl with probability 1/2, and that different children, what they are is independent from what the other children were.
So every childbirth is basically a coin flip.
OK, so if you take that, you say, well, the king is a child.
His sibling is another child.
Children are independent of each other.
So the probability that the sibling is a girl is 1/2.
That's the naive answer.
Now let's try to do it formally.
Let's set up a model of the experiment.
The royal family had two children, as we we're told, so there's four outcomes-- boy boy, boy girl, girl boy, and girl girl.
Now, we are told that there is a king, which means what?
This outcome here did not happen.
It is not possible.
There are three outcomes that remain possible.
So this is our conditional sample space given that there is king.
What are the probabilities for the original model?
Well with the model that we assume that every child is a boy or a girl independently with probability 1/2, then the four outcomes would be equally likely, and they're like this.
These are the original probabilities.
But once we are told that this outcome did not happen, because we have a king, then we are transported to the smaller sample space.
In this sample space, what's the probability that the sibling is a girl?
Well the sibling is a girl in two out of the three outcomes.
So the probability that the sibling is a girl is actually 2/3.
So that's supposed to be the right answer.
Maybe a little counter-intuitive.
So you can play smart and say, oh I understand such problems better than you, here is a trick problem and here's why the answer is 2/3.
But actually I'm not fully justified in saying that the answer is 2/3.
I made lots of hidden assumptions when I put this model down, which I didn't yet state.
So to reverse engineer this answer, let's actually think what's the probability model for which this would have been the right answer.
And here's the probability model.
The royal family-- the royal parents decided to have exactly two children.
They went and had them.
It turned out that at least one was a boy and became a king.
Under this scenario-- that they decide to have exactly two children-- then this is the big sample space.
It turned out that one was a boy.
That eliminates this outcome.
And then this picture is correct and this is the right answer.
But there's hidden assumptions being there.
How about if the royal family had followed the following strategy?
We're going to have children until we get a boy, so that we get a king, and then we'll stop.
OK, given they have two children, what's the probability that the sibling is a girl?
It's 1.
The reason that they had two children was because the first was a girl, so they had to have a second.
So assumptions about reproductive practices actually need to come in, and they're going to affect the decisions.
Or, if it's one of those ancient kingdoms where a king would always make sure too strangle any of his brothers, then the probability that the sibling is a girl is actually 1 again, and so on.
So it means that one needs to be careful when you start with loosely worded problems to make sure exactly what it means and what assumptions you're making.
All right, see you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So today's lecture will be on the subject of counting.
So counting, I guess, is a pretty simple affair conceptually, but it's a topic that can also get to be pretty tricky.
The reason we're going to talk about counting is that there's a lot of probability problems whose solution actually reduces to successfully counting the cardinalities of various sets.
So we're going to see the basic, simplest methods that one can use to count systematically in various situations.
So in contrast to previous lectures, we're not going to introduce any significant new concepts of a probabilistic nature.
We're just going to use the probability tools that we already know.
And we're going to apply them in situations where there's also some counting involved.
Now, today we're going to just touch the surface of this subject.
There's a whole field of mathematics called combinatorics who are people who actually spend their whole lives counting more and more complicated sets.
We were not going to get anywhere close to the full complexity of the field, but we'll get just enough tools that allow us to address problems of the type that one encounters in most common situations.
So the basic idea, the basic principle is something that we've already discussed.
So counting methods apply in situations where we have probabilistic experiments with a finite number of outcomes and where every outcome-- every possible outcome-- has the same probability of occurring.
So we have our sample space, omega, and it's got a bunch of discrete points in there.
And the cardinality of the set omega is some capital N. So, in particular, we assume that the sample points are equally likely, which means that every element of the sample space has the same probability equal to 1 over N. And then we are interested in a subset of the sample space, call it A.
And that subset consists of a number of elements.
Let the cardinality of that subset be equal to little n. And then to find the probability of that set, all we need to do is to add the probabilities of the individual elements.
There's little n elements, and each one has probability one over capital N. And that's the answer.
So this means that to solve problems in this context, all that we need to be able to do is to figure out the number capital N and to figure out the number little n. Now, if somebody gives you a set by just giving you a list and gives you another set, again, giving you a list, it's easy to count there element.
You just count how much there is on the list.
But sometimes the sets are described in some more implicit way, and we may have to do a little bit more work.
There's various tricks that are involved in counting properly.
And the most common one is to-- when you consider a set of possible outcomes, to describe the construction of those possible outcomes through a sequential process.
So think of a probabilistic experiment that involves a number of stages, and in each one of the stages there's a number of possible choices that there may be.
The overall experiment consists of carrying out all the stages to the end.
And the number of points in the sample space is how many final outcomes there can be in this multi-stage experiment.
So in this picture we have an experiment in which of the first stage we have four choices.
In the second stage, no matter what happened in the first stage, the way this is drawn we have three choices.
No matter whether we ended up here, there, or there, we have three choices in the second stage.
And then there's a third stage and at least in this picture, no matter what happened in the first two stages, in the third stage we're going to have two possible choices.
So how many leaves are there at the end of this tree?
That's simple.
It's just the product of these three numbers.
The number of possible leaves that we have out there is 4 times 3 times 2.
Number of choices at each stage gets multiplied, and that gives us the number of overall choices.
So this is the general rule, the general trick that we are going to use over and over.
So let's apply it to some very simple problems as a warm up.
How many license plates can you make if you're allowed to use three letters and then followed by four digits?
At least if you're dealing with the English alphabet, you have 26 choices for the first letter.
Then you have 26 choices for the second letter.
And then 26 choices for the third letter.
And then we start the digits.
We have 10 choices for the first digit, 10 choices for the second digit, 10 choices for the third, 10 choices for the last one.
Let's make it a little more complicated, suppose that we're interested in license plates where no letter can be repeated and no digit can be repeated.
So you have to use different letters, different digits.
How many license plates can you make?
OK, let's choose the first letter, and we have 26 choices.
Now, I'm ready to choose my second letter, how many choices do I have?
I have 25, because I already used one letter.
I have the 25 remaining letters to choose from.
For the next letter, how many choices?
Well, I used up two of my letters, so I only have 24 available.
And then we start with the digits, 10 choices for the first digit, 9 choices for the second, 8 for the third, 7 for the last one.
All right.
So, now, let's bring some symbols in a related problem.
You are given a set that consists of n elements and you're supposed to take those n elements and put them in a sequence.
That is to order them.
Any possible ordering of those elements is called a permutation.
So for example, if we have the set 1, 2, 3, 4, a possible permutation is the list 2, 3, 4, 1.
That's one possible permutation.
And there's lots of possible permutations, of course, the question is how many are there.
OK, let's think about building this permutation by choosing one at a time.
Which of these elements goes into each one of these slots?
How many choices for the number that goes into the first slot or the elements?
Well, we can choose any one of the available elements, so we have n choices.
Let's say this element goes here, having used up that element, we're left with n minus 1 elements and we can pick any one of these and bring it into the second slot.
So here we have n choices, here we're going to have n minus 1 choices, then how many we put there will have n minus 2 choices.
And you go down until the end.
What happens at this point when you are to pick the last element?
Well, you've used n minus of them, there's only one left in your bag.
You're forced to use that one.
So the last stage, you're going to have only one choice.
So, basically, the number of possible permutations is the product of all integers from n down to one, or from one up to n. And there's a symbol that we use for this number, it's called n factorial.
So n factorial is the number of permutations of n objects.
The number of ways that you can order n objects that are given to you.
Now, a different equation.
We have n elements.
Let's say the elements are 1, 1,2, up to n. And it's a set.
And we want to create a subset.
How many possible subsets are there?
So speaking of subsets means looking at each one of the elements and deciding whether you're going to put it in to subsets or not.
For example, I could choose to put 1 in, but 2 I'm not putting it in, 3 I'm not putting it in, 4 I'm putting it, and so on.
So that's how you create a subset.
You look at each one of the elements and you say, OK, I'm going to put it in the subset, or I'm not going to put it.
So think of these as consisting of stages.
At each stage you look at one element, and you make a binary decision.
Do I put it in the subset, or not?
So therefore, how many subsets are there?
Well, I have two choices for the first element.
Am I going to put in the subset, or not?
I have two choices for the next element, and so on.
For each one of the elements, we have two choices.
So the overall number of choices is 2 to the power n. So, conclusion-- the number of subsets, often n element set, is 2 to the n. So in particular, if we take n equal to 1, let's check that our answer makes sense.
If we have n equal to one, how many subsets does it have?
So we're dealing with a set of just one.
What are the subsets?
One subset is this one.
Do we have other subsets of the one element set?
Yes, we have the empty set.
That's the second one.
These are the two possible subsets of this particular set.
So 2 subsets when n is equal to 1, that checks the answer.
All right.
OK, so having gone so far, we can do our first example now.
So we are given a die and we're going to roll it 6 times.
OK, let's make some assumptions about the rolls.
Let's assume that the rolls are independent, and that the die is also fair.
So this means that the probability of any particular outcome of the die rolls-- for example, so we have 6 rolls, one particular outcome could be 3,3,1,6,5.
So that's one possible outcome.
What's the probability of this outcome?
There's probability 1/6 that this happens, 1/6 that this happens, 1/6 that this happens, and so on.
So the probability that the outcome is this is 1/6 to the sixth.
What did I use to come up with this answer?
I used independence, so I multiplied the probability of the first roll gives me a 2, times the probability that the second roll gives me a 3, and so on.
And then I used the assumption that the die is fair, so that the probability of 2 is 1/6, the probably of 3 is 1/6, and so on.
So if I were to spell it out, it's the probability that we get the 2 in the first roll, times the probability of 3 in the second roll, times the probability of the 5 in the last roll.
So by independence, I can multiply probabilities.
And because the die is fair, each one of these numbers is 1/6 to the sixth.
And so the same calculation would apply no matter what numbers I would put in here.
So all possible outcomes are equally likely.
Let's start with this.
So since all possible outcomes are equally likely to find an answer to a probability question, if we're dealing with some particular event, so the event is that all rolls give different numbers.
That's our event A.
And our sample space is some set capital omega.
We know that the answer is going to be the cardinality of the set A, divided by the cardinality of the set omega.
So let's deal with the easy one first.
How many elements are there in the sample space?
How many possible outcomes are there when you roll a dice 6 times?
You have 6 choices for the first roll.
You have 6 choices for the second roll and so on.
So the overall number of outcomes is going to be 6 to the sixth.
So number of elements in the sample space is 6 to the sixth power.
And I guess this checks with this.
We have 6 to the sixth outcomes, each one has this much probability, so the overall probability is equal to one.
Right?
So the probability of an individual outcome is one over how many possible outcomes we have, which is this.
All right.
So how about the numerator?
We are interested in outcomes in which the numbers that we get are all different.
So what is an outcome in which the numbers are all different?
So the die has 6 faces.
We roll it 6 times.
We're going to get 6 different numbers.
This means that we're going to exhaust all the possible numbers, but they can appear in any possible sequence.
So an outcome that makes this event happen is a list of the numbers from 1 to 6, but arranged in some arbitrary order.
So the possible outcomes that make event A happen are just the permutations of the numbers from 1 to 6.
One possible outcome that makes our events to happen-- it would be this.
Here we have 6 possible numbers, but any other list of this kind in which none of the numbers is repeated would also do.
So number of outcomes that make the event happen is the number of permutations of 6 elements.
So it's 6 factorial.
And so the final answer is going to be 6 factorial divided by 6 to the sixth.
All right, so that's a typical way that's one solves problems of this kind.
We know how to count certain things.
For example, here we knew how to count permutations, and we used our knowledge to count the elements of the set that we need to deal with.
So now let's get to a slightly more difficult problem.
We're given once more a set with n elements.
We already know how many subsets that set has, but now we would be interested in subsets that have exactly k elements in them.
So we start with our big set that has n elements, and we want to construct a subset that has k elements.
Out of those n I'm going to choose k and put them in there.
In how many ways can I do this?
More concrete way of thinking about this problem-- you have n people in some group and you want to form a committee by picking people from that group, and you want to form a committee with k people.
Where k is a given number.
For example, a 5 person committee.
How many 5 person committees are possible if you're starting with 100 people?
So that's what we want to count.
How many k element subsets are there?
We don't yet know the answer, but let's give a name to it.
And the name is going to be this particular symbol, which we read as n choose k. Out of n elements, we want to choose k of them.
OK. That may be a little tricky.
So what we're going to do is to instead figure out a somewhat easier problem, which is going to be-- in how many ways can I pick k out of these people and puts them in a particular order?
So how many possible ordered lists can I make that consist of k people?
By ordered, I mean that we take those k people and we say this is the first person in the community.
That's the second person in the committee.
That's the third person in the committee and so on.
So in how many ways can we do this?
Out of these n, we want to choose just k of them and put them in slots.
One after the other.
So this is pretty much like the license plate problem we solved just a little earlier.
So we have n choices for who we put as the top person in the community.
We can pick anyone and have them be the first person.
Then I'm going to choose the second person in the committee.
I've used up 1 person.
So I'm going to have n minus 1 choices here.
And now, at this stage I've used up 2 people, so I have n minus 2 choices here.
And this keeps going on.
Well, what is going to be the last number?
Is it's n minus k?
Well, not really.
I'm starting subtracting numbers after the second one, so by the end I will have subtracted k minus 1.
So that's how many choices I will have for the last person.
So this is the number of ways-- the product of these numbers there gives me the number of ways that I can create ordered lists consisting of k people out of the n that we started with.
Now, you can do a little bit of algebra and check that this expression here is the same as that expression.
Why is this?
This factorial has all the products from 1 up to n. This factorial has all the products from 1 up to n minus k. So you get cancellations.
And what's left is all the products starting from the next number after here, which is this particular number.
So the number of possible ways of creating such ordered lists is n factorial divided by n minus k factorial.
Now, a different way that I could make an ordered list-- instead of picking the people one at a time, I could first choose my k people who are going to be in the committee, and then put them in order.
And tell them out of these k, you are the first, you are the second, you are the third.
Starting with this k people, in how many ways can I order them?
That's the number of permutations.
Starting with a set with k objects, in how many ways can I put them in a specific order?
How many specific orders are there?
That's basically the question.
In how many ways can I permute these k people and arrange them.
So the number of ways that you can do this step is k factorial.
So in how many ways can I start with a set with n elements, go through this process, and end up with a sorted list with k elements?
By the rule that-- when we have stages, the total number of stages is how many choices we had in the first stage, times how many choices we had in the second stage.
The number of ways that this process can happen is this times that.
This is a different way that that process could happen.
And the number of possible of ways is this number.
No matter which way we carry out that process, in the end we have the possible ways of arranging k people out of the n that we started with.
So the final answer that we get when we count should be either this, or this times that.
Both are equally valid ways of counting, so both should give us the same answer.
So we get this equality here.
So these two expressions corresponds to two different ways of constructing ordered lists of k people starting with n people initially.
And now that we have this relation, we can send the k factorial to the denominator.
And that tells us what that number, n choose k, is going to be.
So this formula-- it's written here in red, because you're going to see it a zillion times until the end of the semester-- they are called the binomial coefficients.
And they tell us the number of possible ways that we can create a k element subset, starting with a set that has n elements.
It's always good to do a sanity check to formulas by considering extreme cases.
So let's take the case where k is equal to n. What's the right answer in this case?
How many n elements subsets are there out of an element set?
Well, your subset needs to include every one.
You don't have any choices.
There's only one choice.
It's the set itself.
So the answer should be equal to 1.
That's the number of n element subsets, starting with a set with n elements.
Let's see if the formula gives us the right answer.
We have n factorial divided by k, which is n in our case-- n factorial.
And then n minus k is 0 factorial.
So if our formula is correct, we should have this equality.
And what's the way to make that correct?
Well, it depends what kind of meaning do we give to this symbol?
How do we define zero factorial?
I guess in some ways it's arbitrary.
We're going to define it in a way that makes this formula right.
So the definition that we will be using is that whenever you have 0 factorial, it's going to stand for the number 1.
So let's check that this is also correct, at the other extreme case.
If we let k equal to 0, what does the formula give us?
It gives us, again, n factorial divided by 0 factorial times n factorial.
According to our convention, this again is equal to 1.
So there is one subset of our set that we started with that has zero elements.
Which subset is it?
It's the empty set.
So the empty set is the single subset of the set that we started with that happens to have exactly zero elements.
So the formula checks in this extreme case as well.
So we're comfortable using it.
Now these factorials and these coefficients are really messy algebraic objects.
There's lots of beautiful identities that they satisfy, which you can prove algebraically sometimes by using induction and having cancellations happen all over the place.
But it's really messy.
Sometimes you can bypass those calculations by being clever and using your understanding of what these coefficients stand for.
So here's a typical example.
What is the sum of those binomial coefficients?
I fix n, and sum over all possible cases.
So if you're an algebra genius, you're going to take this expression here, plug it in here, and then start doing algebra furiously.
And half an hour later, you may get the right answer.
But now let's try to be clever.
What does this really do?
What does that formula count?
We're considering k element subsets.
That's this number.
And we're considering the number of k element subsets for different choices of k. The first term in this sum counts how many 0-element subsets we have.
The next term in this sum counts how many 1-element subsets we have.
The next term counts how many 2-element subsets we have.
So in the end, what have we counted?
We've counted the total number of subsets.
We've considered all possible cardinalities.
We've counted the number of subsets of size k. We've considered all possible sizes k. The overall count is going to be the total number of subsets.
And we know what this is.
A couple of slides ago, we discussed that this number is equal to 2 to the n. So, nice, clean and simple answer, which is easy to guess once you give an interpretation to the algebraic expression that you have in front of you.
All right.
So let's move again to sort of an example in which those binomial coefficients are going to show up.
So here's the setting-- n independent coin tosses, and each coin toss has a probability, P, of resulting in heads.
So this is our probabilistic experiment.
Suppose we do 6 tosses.
What's the probability that we get this particular sequence of outcomes?
Because of independence, we can multiply probability.
So it's going to be the probability that the first toss results in heads, times the probability that the second toss results in tails, times the probability that the third one results in tails, times probability of heads, times probability of heads, times probability of heads, which is just P to the fourth times (1 minus P) squared.
So that's the probability of this particular sequence.
How about a different sequence?
If I had 4 tails and 2 heads, but in a different order-- let's say if we considered this particular outcome-- would the answer be different?
We would still have P, times P, times P, times P, times (1 minus P), times (1 minus P).
We would get again, the same answer.
So what you observe from just this example is that, more generally, the probability of obtaining a particular sequence of heads and tails is P to a power, equal to the number of heads.
So here we had 4 heads.
So there's P to the fourth showing up.
And then (1 minus P) to the power number of tails.
So every k head sequence-- every outcome in which we have exactly k heads, has the same probability, which is going to be P to the k, (1 minus p), to the (n minus k).
This is the probability of any particular sequence that has exactly k heads.
So that's the probability of a particular sequence with k heads.
So now let's ask the question, what is the probability that my experiment results in exactly k heads, but in some arbitrary order?
So the heads could show up anywhere.
So there's a number of different ways that this can happen.
What's the overall probability that this event takes place?
So the probability of an event taking place is the sum of the probabilities of all the individual ways that the event can occur.
So it's the sum of the probabilities of all the outcomes that make the event happen.
The different ways that we can obtain k heads are the number of different sequences that contain exactly k heads.
We just figured out that any sequence with exactly k heads has this probability.
So to do this summation, we just need to take the common probability of each individual k head sequence, times how many terms we have in this sum.
So what we're left to do now is to figure out how many k head sequences are there.
How many outcomes are there in which we have exactly k heads.
OK.
So what are the ways that I can describe to you a sequence with k heads?
I can take my n slots that corresponds to the different tosses.
I'm interested in particular sequences that have exactly k heads.
So what I need to do is to choose k slots and assign heads to them.
So to specify a sequence that has exactly k heads is the same thing as drawing this picture and telling you which are the k slots that happened to have heads.
So I need to choose out of those n slots, k of them, and assign them heads.
In how many ways can I choose this k slots?
Well, it's the question of starting with a set of n slots and choosing k slots out of the n available.
So the number of k head sequences is the same as the number of k element subsets of the set of slots that we started with, which are the n slots 1 up to n. We know what that number is.
We counted, before, the number of k element subsets, starting with a set with n elements.
And we gave a symbol to that number, which is that thing, n choose k. So this is the final answer that we obtain.
So these are the so-called binomial probabilities.
And they gave us the probabilities for different numbers of heads starting with a fair coin that's being tossed a number of times.
This formula is correct, of course, for reasonable values of k, meaning its correct for k equals 0, 1, up to n. If k is bigger than n, what's the probability of k heads?
If k is bigger than n, there's no way to obtain k heads, so that probability is, of course, zero.
So these probabilities only makes sense for the numbers k that are possible, given that we have n tosses.
And now a question similar to the one we had in the previous slide.
If I write down this summation-- even worse algebra than the one in the previous slide-- what do you think this number will turn out to be?
It should be 1 because this is the probability of obtaining k heads.
When we do the summation, what we're doing is we're considering the probability of 0 heads, plus the probability of 1 head, plus the probability of 2 heads, plus the probability of n heads.
We've exhausted all the possibilities in our experiment.
So the overall probability, when you exhaust all possibilities, must be equal to 1.
So that's yet another beautiful formula that evaluates into something really simple.
And if you tried to prove this identity algebraically, of course, you would have to suffer quite a bit.
So now armed with the binomial probabilities, we can do the harder problems.
So let's take the same experiment again.
We flip a coin independently 10 times.
So these 10 tosses are independent.
We flip it 10 times.
We don't see the result, but somebody comes and tells us, you know, there were exactly 3 heads in the 10 tosses that you had.
OK?
So a certain event happened.
And now you're asked to find the probability of another event, which is that the first 2 tosses were heads.
Let's call that event A. OK.
So are we in the setting of discrete uniform probability laws?
When we toss a coin multiple times, is it the case that all outcomes are equally likely?
All sequences are equally likely?
That's the case if you have a fair coin-- that all sequences are equally likely.
But if your coin is not fair, of course, heads/heads is going to have a different probability than tails/tails.
If your coin is biased towards heads, then heads/heads is going to be more likely.
So we're not quite in the uniform setting.
Our overall sample space, omega, does not have equally likely elements.
Do we care about that?
Not necessarily.
All the action now happens inside the event B that we are told has occurred.
So we have our big sample space, omega.
Elements of that sample space are not equally likely.
We are told that a certain event B occurred.
And inside that event B, we're asked to find the conditional probability that A has also occurred.
Now here's the lucky thing, inside the event B, all outcomes are equally likely.
The outcomes inside B are the sequences of 10 tosses that have exactly 3 heads.
Every 3-head sequence has this probability.
So the elements of B are equally likely with each other.
Once we condition on the event B having occurred, what happens to the probabilities of the different outcomes inside here?
Well, conditional probability laws keep the same proportions as the unconditional ones.
The elements of B were equally likely when we started, so they're equally likely once we are told that B has occurred.
So to do with this problem, we need to just transport us to this smaller universe and think about what's happening in that little universe.
In that little universe, all elements of B are equally likely.
So to find the probability of some subset of that set, we only need to count the cardinality of B, and count the cardinality of A.
So let's do that.
Number of outcomes in B-- in how many ways can we get 3 heads out of 10 tosses?
That's the number we considered before, and it's 10 choose 3.
This is the number of 3-head sequences when you have 10 tosses.
Now let's look at the event A.
The event A is that the first 2 tosses where heads, but we're living now inside this universe B.
Given that B occurred, how many elements does A have in there?
In how many ways can A happen inside the B universe.
If you're told that the first 2 were heads-- sorry.
So out of the outcomes in B that have 3 heads, how many start with heads/heads?
Well, if it starts with heads/heads, then the only uncertainty is the location of the third head.
So we started with heads/heads, we're going to have three heads, the question is, where is that third head going to be.
It has eight possibilities.
So slot 1 is heads, slot 2 is heads, the third heads can be anywhere else.
So there's 8 possibilities for where the third head is going to be.
OK.
So what we have counted here is really the cardinality of A intersection B, which is out of the elements in B, how many of them make A happen, divided by the cardinality of B.
And that gives us the answer, which is going to be 10 choose 3, divided by 8.
And I should probably redraw a little bit of the picture that they have here.
The set A is not necessarily contained in B.
It could also have stuff outside B.
So the event that the first 2 tosses are heads can happen with a total of 3 heads, but it can also happen with a different total number of heads.
But once we are transported inside the set B, what we need to count is just this part of A.
It's A intersection B and compare it with the total number of elements in the set B.
Did I write it the opposite way?
Yes.
So this is 8 over 10 choose 3.
OK.
So we're going to close with a more difficult problem now.
OK.
This business of n choose k has to do with starting with a set and picking a subset of k elements.
Another way of thinking of that is that we start with a set with n elements and you choose a subset that has k, which means that there's n minus k that are left.
Picking a subset is the same as partitioning our set into two pieces.
Now let's generalize this question and start counting partitions in general.
Somebody gives you a set that has n elements.
Somebody gives you also certain numbers-- n1, n2, n3, let's say, n4, where these numbers add up to n. And you're asked to partition this set into four subsets where each one of the subsets has this particular cardinality.
So you're asking to cut it into four pieces, each one having the prescribed cardinality.
In how many ways can we do this partitioning?
n choose k was the answer when we partitioned in two pieces, what's the answer more generally?
For a concrete example of a partition, you have your 52 card deck and you deal, as in bridge, by giving 13 cards to each one of the players.
Assuming that the dealing is done fairly and with a well shuffled deck of cards, every particular partition of the 52 cards into four hands, that is four subsets of 13 each, should be equally likely.
So we take the 52 cards and we partition them into subsets of 13, 13, 13, and 13.
And we assume that all possible partitions, all possible ways of dealing the cards are equally likely.
So we are again in a setting where we can use counting, because all the possible outcomes are equally likely.
So an outcome of the experiment is the hands that each player ends up getting.
And when you get the cards in your hands, it doesn't matter in which order that you got them.
It only matters what cards you have on you.
So it only matters which subset of the cards you got.
All right.
So what's the cardinality of the sample space in this experiment?
So let's do it for the concrete numbers that we have for the problem of partitioning 52 cards.
So think of dealing as follows-- you shuffle the deck perfectly, and then you take the top 13 cards and give them to one person.
In how many possible hands are there for that person?
Out of the 52 cards, I choose 13 at random and give them to the first person.
Having done that, what happens next?
I'm left with 39 cards.
And out of those 39 cards, I pick 13 of them and give them to the second person.
Now I'm left with 26 cards.
Out of those 26, I choose 13, give them to the third person.
And for the last person there isn't really any choice.
Out of the 13, I have to give that person all 13.
And that number is just equal to 1.
So we don't care about it.
All right.
So next thing you do is to write down the formulas for these numbers.
So, for example, here you would have 52 factorial, divided by 13 factorial, times 39 factorial, and you continue.
And then there are nice cancellations that happen.
This 39 factorial is going to cancel the 39 factorial that comes from there, and so on.
After you do the cancellations and all the algebra, you're left with this particular answer, which is the number of possible partitions of 52 cards into four players where each player gets exactly 13 hands.
If you were to generalize this formula to the setting that we have here, the more general formula is-- you have n factorial, where n is the number of objects that you are distributing, divided by the product of the factorials of the-- OK, here I'm doing it for the case where we split it into four sets.
So that would be the answer when we partition a set into four subsets of prescribed cardinalities.
And you can guess how that formula would generalize if you want to split it into five sets or six sets.
OK.
So far we just figured out the size of the sample space.
Now we need to look at our event, which is the event that each player gets an ace, let's call that event A.
In how many ways can that event happens?
How many possible hands are there in which every player has exactly one ace?
So I need to think about the sequential process by which I distribute the cards so that everybody gets exactly one ace, and then try to think in how many ways can that sequential process happen.
So one way of making sure that everybody gets exactly one ace is the following-- I take the four aces and I distribute them randomly to the four players, but making sure that each one gets exactly one ace.
In how many ways can that happen?
I take the ace of spades and I send it to a random person out of the four.
So there's 4 choices for this.
Then I'm left with 3 aces to distribute.
That person already gotten an ace.
I take the next ace, and I give it to one of the 3 people remaining.
So there's 3 choices for how to do that.
And then for the next ace, there's 2 people who have not yet gotten an ace, and they give it randomly to one of them.
So these are the possible ways of distributing for the 4 aces, so that each person gets exactly one.
It's actually the same as this problem.
Starting with a set of four things, in how many ways can I partition them into four subsets where the first set has one element, the second has one element, the third one has another element, and so on.
So it agrees with that formula by giving us 4 factorial.
OK.
So there are different ways of distributing the aces.
And then there's different ways of distributing the remaining 48 cards.
How many ways are there?
Well, I have 48 cards that I'm going to distribute to four players by giving 12 cards to each one.
It's exactly the same question as the one we had here, except that now it's 48 cards, 12 to each person.
And that gives us this particular count.
So putting all that together gives us the different ways that we can distribute the cards to the four players so that each one gets exactly one ace.
The number of possible ways is going to be this four factorial, coming from here, times this number-- this gives us the number of ways that the event of interest can happen-- and then the denominator is the cardinality of our sample space, which is this number.
So this looks like a horrible mess.
It turns out that this expression does simplify to something really, really simple.
And if you look at the textbook for this problem, you will see an alternative derivation that gives you a short cut to the same numerical answer.
All right.
So that basically concludes chapter one.
From next time we're going to consider introducing random variables and make the subject even more interesting.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu, OK.
So let us start.
All right.
So today we're starting a new unit in this class.
We have covered, so far, the basics of probability theory-- the main concepts and tools, as far as just probabilities are concerned.
But if that was all that there is in this subject, the subject would not be rich enough.
What makes probability theory a lot more interesting and richer is that we can also talk about random variables, which are ways of assigning numerical results to the outcomes of an experiment.
So we're going to define what random variables are, and then we're going to describe them using so-called probability mass functions.
Basically some numerical values are more likely to occur than other numerical values, and we capture this by assigning probabilities to them the usual way.
And we represent these in a compact way using the so-called probability mass functions.
We're going to see a couple of examples of random variables, some of which we have already seen but with different terminology.
And so far, it's going to be just a couple of definitions and calculations of the type that you already know how to do.
But then we're going to introduce the one new, big concept of the day.
So up to here it's going to be mostly an exercise in notation and definitions.
But then we got our big concept which is the concept of the expected value of a random variable, which is some kind of average value of the random variable.
And then we're going to also talk, very briefly, about close distance of the expectation, which is the concept of the variance of a random variable.
OK.
So what is a random variable?
It's an assignment of a numerical value to every possible outcome of the experiment.
So here's the picture.
The sample space is this class, and we've got lots of students in here.
This is our sample space, omega.
I'm interested in the height of a random student.
So I'm going to use a real line where I record height, and let's say this is height in inches.
And the experiment happens, I pick a random student.
And I go and measure the height of that random student, and that gives me a specific number.
So what's a good number in inches?
Let's say 60.
OK. Or I pick another student, and that student has a height of 71 inches, and so on.
So this is the experiment.
These are the outcomes.
These are the numerical values of the random variable that we call height.
OK.
So mathematically, what are we dealing with here?
We're basically dealing with a function from the sample space into the real numbers.
That function takes as argument, an outcome of the experiment, that is a typical student, and produces the value of that function, which is the height of that particular student.
So we think of an abstract object that we denote by a capital H, which is the random variable called height.
And that random variable is essentially this particular function that we talked about here.
OK.
So there's a distinction that we're making here-- H is height in the abstract.
It's the function.
These numbers here are particular numerical values that this function takes when you choose one particular outcome of the experiment.
Now, when you have a single probability experiment, you can have multiple random variables.
So perhaps, instead of just height, I'm also interested in the weight of a typical student.
And so when the experiment happens, I pick that random student-- this is the height of the student.
But that student would also have a weight, and I could record it here.
And similarly, every student is going to have their own particular weight.
So the weight function is a different function from the sample space to the real numbers, and it's a different random variable.
So the point I'm making here is that a single probabilistic experiment may involve several interesting random variables.
I may be interested in the height of a random student or the weight of the random student.
These are different random variables that could be of interest.
I can also do other things.
Suppose I define an object such as H bar, which is 2.58.
What does that correspond to?
Well, this is the height in centimeters.
Now, H bar is a function of H itself, but if you were to draw the picture, the picture would go this way.
60 gets mapped to 150, 71 gets mapped to, oh, that's too hard for me.
OK, gets mapped to something, and so on.
So H bar is also a random variable.
Why?
Once I pick a particular student, that particular outcome determines completely the numerical value of H bar, which is the height of that student but measured in centimeters.
What we have here is actually a random variable, which is defined as a function of another random variable.
And the point that this example is trying to make is that functions of random variables are also random variables.
The experiment happens, the experiment determines a numerical value for this object.
And once you have the numerical value for this object, that determines also the numerical value for that object.
So given an outcome, the numerical value of this particular object is determined.
So H bar is itself a function from the sample space, from outcomes to numerical values.
And that makes it a random variable according to the formal definition that we have here.
So the formal definition is that the random variable is not random, it's not a variable, it's just a function from the sample space to the real numbers.
That's the abstract, right way of thinking about them.
Now, random variables can be of different types.
They can be discrete or continuous.
Suppose that I measure the heights in inches, but I round to the nearest inch.
Then the numerical values I'm going to get here would be just integers.
So that would make it an integer valued random variable.
And this is a discrete random variable.
Or maybe I have a scale for measuring height which is infinitely precise and records your height to an infinite number of digits of precision.
In that case, your height would be just a general real number.
So we would have a random variable that takes values in the entire set of real numbers.
Well, I guess not really negative numbers, but the set of non-negative numbers.
And that would be a continuous random variable.
It takes values in a continuous set.
So we will be talking about both discrete and continuous random variables.
The first thing we will do will be to devote a few lectures on discrete random variables, because discrete is always easier.
And then we're going to repeat everything in the continuous setting.
So discrete is easier, and it's the right place to understand all the concepts, even those who may appear to be elementary.
And then you will be set to understand what's going on when we go to the continuous case.
So in the continuous case, you get all the complications of calculus and some extra math that comes in there.
So it's important to have been down all the concepts very well in the easy, discrete case so that you don't have conceptual hurdles when you move on to the continuous case.
Now, one important remark that may seem trivial but it's actually very important so that you don't get tangled up between different types of concepts-- there's a fundamental distinction between the random variable itself, and the numerical values that it takes.
Abstractly speaking, or mathematically speaking, a random variable, x, or H in this example, is a function.
OK. Maybe if you like programming the words "procedure" or "sub-routine" might be better.
So what's the sub-routine height?
Given a student, I take that student, force them on the scale and measure them.
That's the sub-routine that measures heights.
It's really a function that takes students as input and produces numbers as output.
The sub-routine we denoted by capital H. That's the random variable.
But once you plug in a particular student into that sub-routine, you end up getting a particular number.
This is the numerical output of that sub-routine or the numerical value of that function.
And that numerical value is an element of the real numbers.
So the numerical value is a real number, where this capital X is a function from omega to the real numbers.
So they are very different types of objects.
And the way that we keep track of what we're talking about at any given time is by using capital letters for random variables and lower case letters for numbers.
OK.
So now once we have a random variable at hand, that random variable takes on different numerical values.
And we want to describe to say something about the relative likelihoods of the different numerical values that the random variable can take.
So here's our sample space, and here's the real line.
And there's a bunch of outcomes that gave rise to one particular numerical value.
There's another numerical value that arises if we have this outcome.
There's another numerical value that arises if we have this outcome.
So our sample space is here.
The real numbers are here.
And what we want to do is to ask the question, how likely is that particular numerical value to occur?
So what we're essentially asking is, how likely is it that we obtain an outcome that leads to that particular numerical value?
We calculate that overall probability of that numerical value and we represent that probability using a bar so that we end up generating a bar graph.
So that could be a possible bar graph associated with this picture.
The size of this bar is the total probability that our random variable took on this numerical value, which is just the sum of the probabilities of the different outcomes that led to that numerical value.
So the thing that we're plotting here, the bar graph-- we give a name to it.
It's a function, which we denote by lowercase b, capital X.
The capital X indicates which random variable we're talking about.
And it's a function of little x, which is the range of values that our random variable is taking.
So in mathematical notation, the value of the PMF at some particular number, little x, is the probability that our random variable takes on the numerical value, little x.
And if you want to be precise about what this means, it's the overall probability of all outcomes for which the random variable ends up taking that value, little x.
So this is the overall probability of all omegas that lead to that particular numerical value, x, of interest.
So what do we know about PMFs?
Since there are probabilities, all these entries in the bar graph have to be non-negative.
Also, if you exhaust all the possible values of little x's, you will have exhausted all the possible outcomes here.
Because every outcome leads to some particular x.
So the sum of these probabilities should be equal to one.
This is the second relation here.
So this relation tell us that some little x is going to happen.
They happen with different probabilities, but when you consider all the possible little x's together, one of those little x's is going to be realized.
Probabilities need to add to one.
OK.
So let's get our first example of a non-trivial bar graph.
Consider the experiment where I start with a coin and I start flipping it over and over.
And I do this until I obtain heads for the first time.
So what are possible outcomes of this experiment?
One possible outcome is that I obtain heads at the first toss, and then I stop.
In this case, my random variable takes the value 1.
Or it's possible that I obtain tails and then heads.
How many tosses did it take until heads appeared?
This would be x equals to 2.
Or more generally, I might obtain tails for k minus 1 times, and then obtain heads at the k-th time, in which case, our random variable takes the value, little k. So that's the experiment.
So capital X is a well defined random variable.
It's the number of tosses it takes until I see heads for the first time.
These are the possible outcomes.
These are elements of our sample space.
And these are the values of X depending on the outcome.
Clearly X is a function of the outcome.
You tell me the outcome, I'm going to tell you what X is.
So what we want to do now is to calculate the PMF of X.
So Px of k is, by definition, the probability that our random variable takes the value k. For the random variable to take the value of k, the first head appears at toss number k. The only way that this event can happen is if we obtain this sequence of events.
T's the first k minus 1 times, tails, and heads at the k-th flip.
So this event, that the random variable is equal to k, is the same as this event, k minus 1 tails followed by 1 head.
What's the probability of that event?
We're assuming that the coin tosses are independent.
So to find the probability of this event, we need to multiply the probability of tails, times the probability of tails, times the probability of tails.
We multiply k minus one times, times the probability of heads, which puts an extra p at the end.
And this is the formula for the so-called geometric PMF.
And why do we call it geometric?
Because if you go and plot the bar graph of this random variable, X, we start at 1 with a certain number, which is p. And then at 2 we get p(1-p).
At 3 we're going to get something smaller, it's p times (1-p)-squared.
And the bars keep going down at the rate of geometric progression.
Each bar is smaller than the previous bar, because each time we get an extra factor of 1-p involved.
So the shape of this PMF is the graph of a geometric sequence.
For that reason, we say that it's the geometric PMF, and we call X also a geometric random variable.
So the number of coin tosses until the first head is a geometric random variable.
So this was an example of how to compute the PMF of a random variable.
This was an easy example, because this event could be realized in one and only one way.
So to find the probability of this, we just needed to find the probability of this particular outcome.
More generally, there's going to be many outcomes that can lead to the same numerical value.
And we need to keep track of all of them.
For example, in this picture, if I want to find this value of the PMF, I need to add up the probabilities of all the outcomes that leads to that value.
So the general procedure is exactly what this picture suggests.
To find this probability, you go and identify which outcomes lead to this numerical value, and add their probabilities.
So let's do a simple example.
I take a tetrahedral die.
I toss it twice.
And there's lots of random variables that you can associate with the same experiment.
So the outcome of the first throw, we can call it F. That's a random variable because it's determined once you tell me what happens in the experiment.
The outcome of the second throw is another random variable.
The minimum of the two throws is also a random variable.
Once I do the experiment, this random variable takes on a specific numerical value.
So suppose I do the experiment and I get a 2 and a 3.
So this random variable is going to take the numerical value of 2.
This is going to take the numerical value of 3.
This is going to take the numerical value of 2.
And now suppose that I want to calculate the PMF of this random variable.
What I will need to do is to calculate Px(0), Px(1), Px(2), Px(3), and so on.
Let's not do the entire calculation then, let's just calculate one of the entries of the PMF.
So Px(2)-- that's the probability that the minimum of the two throws gives us a 2.
And this can happen in many ways.
There are five ways that it can happen.
Those are all of the outcomes for which the smallest of the two is equal to 2.
That's five outcomes assuming that the tetrahedral die is fair and the tosses are independent.
Each one of these outcomes has probability of 1/16.
There's five of them, so we get an answer, 5/16.
Conceptually, this is just the procedure that you use to calculate PMFs the way that you construct this particular bar graph.
You consider all the possible values of your random variable, and for each one of those random variables you find the probability that the random variable takes on that value by adding the probabilities of all the possible outcomes that leads to that particular numerical value.
So let's do another, more interesting one.
So let's revisit the coin tossing problem from last time.
Let us fix a number n, and we decide to flip a coin n consecutive times.
Each time the coin tosses are independent.
And each one of the tosses will have a probability, p, of obtaining heads.
Let's consider the random variable, which is the total number of heads that have been obtained.
Well, that's something that we dealt with last time.
We know the probabilities for different numbers of heads, but we're just going to do the same now using today's notation.
So let's, for concreteness, n equal to 4.
Px is the PMF of that random variable, X. Px(2) is meant to be, by definition, it's the probability that a random variable takes the value of 2.
So this is the probability that we have, exactly two heads in our four tosses.
The event of exactly two heads can happen in multiple ways.
And here I've written down the different ways that it can happen.
It turns out that there's exactly six ways that it can happen.
And each one of these ways, luckily enough, has the same probability-- p-squared times (1-p)-squared.
So that gives us the value for the PMF evaluated at 2.
So here we just counted explicitly that we have six possible ways that this can happen, and this gave rise to this factor of 6.
But this factor of 6 turns out to be the same as this 4 choose 2.
If you remember definition from last time, 4 choose 2 is 4 factorial divided by 2 factorial, divided by 2 factorial, which is indeed equal to 6.
And this is the more general formula that you would be using.
In general, if you have n tosses and you're interested in the probability of obtaining k heads, the probability of that event is given by this formula.
So that's the formula that we derived last time.
Except that last time we didn't use this notation.
We just said the probability of k heads is equal to this.
Today we introduce the extra notation.
And also having that notation, we may be tempted to also plot a bar graph for the Px.
In this case, for the coin tossing problem.
And if you plot that bar graph as a function of k when n is a fairly large number, what you will end up obtaining is a bar graph that has a shape of something like this.
So certain values of k are more likely than others, and the more likely values are somewhere in the middle of the range.
And extreme values-- too few heads or too many heads, are unlikely.
Now, the miraculous thing is that it turns out that this curve gets a pretty definite shape, like a so-called bell curve, when n is big.
This is a very deep and central fact from probability theory that we will get to in a couple of months.
For now, it just could be a curious observation.
If you go into MATLAB and put this formula in and ask MATLAB to plot it for you, you're going to get an interesting shape of this form.
And later on we will have to sort of understand where this is coming from and whether there's a nice, simple formula for the asymptotic form that we get.
All right.
So, so far I've said essentially nothing new, just a little bit of notation and this little conceptual thing that you have to think of random variables as functions in the sample space.
So now it's time to introduce something new.
This is the big concept of the day.
In some sense it's an easy concept.
But it's the most central, most important concept that we have to deal with random variables.
It's the concept of the expected value of a random variable.
So the expected value is meant to be, let's speak loosely, something like an average, where you interpret probabilities as something like frequencies.
So you play a certain game and your rewards are going to be-- use my standard numbers-- your rewards are going to be one dollar with probability 1/6.
It's going to be 2 dollars with probability 1/2, and four dollars with probability 1/3.
So this is a plot of the PMF of some random variable.
If you play that game and you get so many dollars with this probability, and so on, how much do you expect to get on the average if you play the game a zillion times?
Well, you can think as follows-- one sixth of the time I'm going to get one dollar.
One half of the time that outcome is going to happen and I'm going to get two dollars.
And one third of the time the other outcome happens, and I'm going to get four dollars.
And you evaluate that number and it turns out to be 2.5.
OK.
So that's a reasonable way of calculating the average payoff if you think of these probabilities as the frequencies with which you obtain the different payoffs.
And loosely speaking, it doesn't hurt to think of probabilities as frequencies when you try to make sense of various things.
So what did we do here?
We took the probabilities of the different outcomes, of the different numerical values, and multiplied them with the corresponding numerical value.
Similarly here, we have a probability and the corresponding numerical value and we added up over all x's.
So that's what we did.
It looks like an interesting quantity to deal with.
So we're going to give a name to it, and we're going to call it the expected value of a random variable.
So this formula just captures the calculation that we did.
How do we interpret the expected value?
So the one interpretation is the one that I used in this example.
You can think of it as the average that you get over a large number of repetitions of an experiment where you interpret the probabilities as the frequencies with which the different numerical values can happen.
There's another interpretation that's a little more visual and that's kind of insightful, if you remember your freshman physics, this kind of formula gives you the center of gravity of an object of this kind.
If you take that picture literally and think of this as a mass of one sixth sitting here, and the mass of one half sitting here, and one third sitting there, and you ask me what's the center of gravity of that structure.
This is the formula that gives you the center of gravity.
Now what's the center of gravity?
It's the place where if you put your pen right underneath, that diagram will stay in place and will not fall on one side and will not fall on the other side.
So in this thing, by picture, since the 4 is a little more to the right and a little heavier, the center of gravity should be somewhere around here.
And that's what for the math gave us.
It turns out to be two and a half.
Once you have this interpretation about centers of gravity, sometimes you can calculate expectations pretty fast.
So here's our new random variable.
It's the uniform random variable in which each one of the numerical values is equally likely.
Here there's a total of n plus 1 possible numerical values.
So each one of them has probability 1 over (n + 1).
Let's calculate the expected value of this random variable.
We can take the formula literally and consider all possible outcomes, or all possible numerical values, and weigh them by their corresponding probability, and do this calculation and obtain an answer.
But I gave you the intuition of centers of gravity.
Can you use that intuition to guess the answer?
What's the center of gravity infrastructure of this kind?
We have symmetry.
So it should be in the middle.
And what's the middle?
It's the average of the two end points.
So without having to do the algebra, you know that's the answer is going to be n over 2.
So this is a moral that you should keep whenever you have PMF, which is symmetric around a certain point.
That certain point is going to be the expected value associated with this particular PMF.
OK.
So having defined the expected value, what is there that's left for us to do?
Well, we want to investigate how it behaves, what kind of properties does it have, and also how do you calculate expected values of complicated random variables.
So the first complication that we're going to start with is the case where we deal with a function of a random variable.
OK.
So let me redraw this same picture as before.
We have omega.
This is our sample space.
This is the real line.
And we have a random variable that gives rise to various values for X.
So the random variable is capital X, and every outcome leads to a particular numerical value x for our random variable X.
So capital X is really the function that maps these points into the real line.
And then I consider a function of this random variable, call it capital Y, and it's a function of my previous random variable.
And this new random variable Y takes numerical values that are completely determined once I know the numerical value of capital X.
And perhaps you get a diagram of this kind.
So X is a random variable.
Once you have an outcome, this determines the value of x. Y is also a random variable.
Once you have the outcome, that determines the value of y. Y is completely determined once you know X.
We have a formula for how to calculate the expected value of X.
Suppose that you're interested in calculating the expected value of Y.
How would you go about it?
OK.
The only thing you have in your hands is the definition, so you could start by just using the definition.
And what does this entail?
It entails for every particular value of y, collect all the outcomes that leads to that value of y.
Find their probability.
Do the same here.
For that value, collect those outcomes.
Find their probability and weight by y.
So this formula does the addition over this line.
We consider the different outcomes and add things up.
There's an alternative way of doing the same accounting where instead of doing the addition over those numbers, we do the addition up here.
We consider the different possible values of x, and we think as follows-- for each possible value of x, that value is going to occur with this probability.
And if that value has occurred, this is how much I'm getting, the g of x.
So I'm considering the probability of this outcome.
And in that case, y takes this value.
Then I'm considering the probabilities of this outcome.
And in that case, g of x takes again that value.
Then I consider this particular x, it happens with this much probability, and in that case, g of x takes that value, and similarly here.
We end up doing exactly the same arithmetic, it's only a question whether we bundle things together.
That is, if we calculate the probability of this, then we're bundling these two cases together.
Whereas if we do the addition up here, we do a separate calculation-- this probability times this number, and then this probability times that number.
So it's just a simple rearrangement of the way that we do the calculations, but it does make a big difference in practice if you actually want to calculate expectations.
So the second procedure that I mentioned, where you do the addition by running over the x-axis corresponds to this formula.
Consider all possibilities for x and when that x happens, how much money are you getting?
That gives you the average money that you are getting.
All right.
So I kind of hand waved and argued that it's just a different way of accounting, of course one needs to prove this formula.
And fortunately it can be proved.
You're going to see that in recitation.
Most people, once they're a little comfortable with the concepts of probability, actually believe that this is true by definition.
In fact it's not true by definition.
It's called the law of the unconscious statistician.
It's something that you always do, but it's something that does require justification.
All right.
So this gives us basically a shortcut for calculating expected values of functions of a random variable without having to find the PMF of that function.
We can work with the PMF of the original function.
All right.
So we're going to use this property over and over.
Before we start using it, one general word of caution-- the average of a function of a random variable, in general, is not the same as the function of the average.
So these two operations of taking averages and taking functions do not commute.
What this inequality tells you is that, in general, you can not reason on the average.
So we're going to see instances where this property is not true.
You're going to see lots of them.
Let me just throw it here that it's something that's not true in general, but we will be interested in the exceptions where a relation like this is true.
But these will be the exceptions.
So in general, expectations are average, something like averages.
But the function of an average is not the same as the average of the function.
OK.
So now let's go to properties of expectations.
Suppose that alpha is a real number, and I ask you, what's the expected value of that real number?
So for example, if I write down this expression-- expected value of 2.
What is this?
Well, we defined random variables and we defined expectations of random variables.
So for this to make syntactic sense, this thing inside here should be a random variable.
Is 2 -- the number 2 --- is it a random variable?
In some sense, yes.
It's the random variable that takes, always, the value of 2.
So suppose that you have some experiment and that experiment always outputs 2 whenever it happens.
Then you can say, yes, it's a random experiment but it always gives me 2.
The value of the random variable is always 2 no matter what.
It's kind of a degenerate random variable that doesn't have any real randomness in it, but it's still useful to think of it as a special case.
So it corresponds to a function from the sample space to the real line that takes only one value.
No matter what the outcome is, it always gives me a 2.
OK.
If you have a random variable that always gives you a 2, what is the expected value going to be?
The only entry that shows up in this summation is that number 2.
The probability of a 2 is equal to 1, and the value of that random variable is equal to 2.
So it's the number itself.
So the average value in an experiment that always gives you 2's is 2.
All right.
So that's nice and simple.
Now let's go to our experiment where age was your height in inches.
And I know your height in inches, but I'm interested in your height measured in centimeters.
How is that going to be related to your height in inches?
Well, if you take your height in inches and convert it to centimeters, I have another random variable, which is always, no matter what, two and a half times bigger than the random variable I started with.
If you take some quantity and always multiplied by two and a half what happens to the average of that quantity?
It also gets multiplied by two and a half.
So you get a relation like this, which says that the average height of a student measured in centimeters is two and a half times the average height of a student measured in inches.
So that makes perfect intuitive sense.
If you generalize it, it gives us this relation, that if you have a number, you can pull it outside the expectation and you get the right result.
So this is a case where you can reason on the average.
If you take a number, such as height, and multiply it by a certain number, you can reason on the average.
I multiply the numbers by two, the averages will go up by two.
So this is an exception to this cautionary statement that I had up there.
How do we prove that this fact is true?
Well, we can use the expected value rule here, which tells us that the expected value of alpha X, this is our g of X, essentially, is going to be the sum over all x's of my function, g of X, times the probability of the x's.
In our particular case, g of X is alpha times X.
And we have those probabilities.
And the alpha goes outside the summation.
So we get alpha, sum over x's, x Px of x, which is alpha times the expected value of X.
So that's how you prove this relation formally using this rule up here.
And the next formula that I have here also gets proved the same way.
What does this formula tell you?
If I take everybody's height in centimeters-- we already multiplied by alpha-- and the gods give everyone a bonus of ten extra centimeters.
What's going to happen to the average height of the class?
Well, it will just go up by an extra ten centimeters.
So this expectation is going to be giving you the bonus of beta just adds a beta to the average height in centimeters, which we also know to be alpha times the expected value of X, plus beta.
So this is a linearity property of expectations.
If you take a linear function of a single random variable, the expected value of that linear function is the linear function of the expected value.
So this is our big exception to this cautionary note, that we have equal if g is linear.
OK. All right.
So let's get to the last concept of the day.
What kind of functions of random variables may be of interest?
One possibility might be the average value of X-squared.
Why is it interesting?
Well, why not.
It's the simplest function that you can think of.
So if you want to calculate the expected value of X-squared, you would use this general rule for how you can calculate expected values of functions of random variables.
You consider all the possible x's.
For each x, you see what's the probability that it occurs.
And if that x occurs, you consider and see how big x-squared is.
Now, the more interesting quantity, a more interesting expectation that you can calculate has to do not with x-squared, but with the distance of x from the mean and then squared.
So let's try to parse what we've got up here.
Let's look just at the quantity inside here.
What kind of quantity is it?
It's a random variable.
Why?
X is random, the random variable, expected value of X is a number.
Subtract a number from a random variable, you get another random variable.
Take a random variable and square it, you get another random variable.
So the thing inside here is a legitimate random variable.
What kind of random variable is it?
So suppose that we have our experiment and we have different x's that can happen.
And the mean of X in this picture might be somewhere around here.
I do the experiment.
I obtain some numerical value of x.
Let's say I obtain this numerical value.
I look at the distance from the mean, which is this length, and I take the square of that.
Each time that I do the experiment, I go and record my distance from the mean and square it.
So I give more emphasis to big distances.
And then I take the average over all possible outcomes, all possible numerical values.
So I'm trying to compute the average squared distance from the mean.
This corresponds to this formula here.
So the picture that I drew corresponds to that.
For every possible numerical value of x, that numerical value corresponds to a certain distance from the mean squared, and I weight according to how likely is that particular value of x to arise.
So this measures the average squared distance from the mean.
Now, because of that expected value rule, of course, this thing is the same as that expectation.
It's the average value of the random variable, which is the squared distance from the mean.
With this probability, the random variable takes on this numerical value, and the squared distance from the mean ends up taking that particular numerical value.
OK.
So why is the variance interesting?
It tells us how far away from the mean we expect to be on the average.
Well, actually we're not counting distances from the mean, it's distances squared.
So it gives more emphasis to the kind of outliers in here.
But it's a measure of how spread out the distribution is.
A big variance means that those bars go far to the left and to the right, typically.
Where as a small variance would mean that all those bars are tightly concentrated around the mean value.
It's the average squared deviation.
Small variance means that we generally have small deviations.
Large variances mean that we generally have large deviations.
Now as a practical matter, when you want to calculate the variance, there's a handy formula which I'm not proving but you will see it in recitation.
It's just two lines of algebra.
And it allows us to calculate it in a somewhat simpler way.
We need to calculate the expected value of the random variable and the expected value of the squares of the random variable, and these two are going to give us the variance.
So to summarize what we did up here, the variance, by definition, is given by this formula.
It's the expected value of the squared deviation.
But we have the equivalent formula, which comes from application of the expected value rule, to the function g of X, equals to x minus the (expected value of X)-squared.
OK.
So this is the definition.
This comes from the expected value rule.
What are some properties of the variance?
Of course variances are always non-negative.
Why is it always non-negative?
Well, you look at the definition and your just adding up non-negative things.
We're adding squared deviations.
So when you add non-negative things, you get something non-negative.
The next question is, how do things scale if you take a linear function of a random variable?
Let's think about the effects of beta.
If I take a random variable and add the constant to it, how does this affect the amount of spread that we have?
It doesn't affect-- whatever the spread of this thing is, if I add the constant beta, it just moves this diagram here, but the spread doesn't grow or get reduced.
The thing is that when I'm adding a constant to a random variable, all the x's that are going to appear are further to the right, but the expected value also moves to the right.
And since we're only interested in distances from the mean, these distances do not get affected.
x gets increased by something.
The mean gets increased by that same something.
The difference stays the same.
So adding a constant to a random variable doesn't do anything to it's variance.
But if I multiply a random variable by a constant alpha, what is that going to do to its variance?
Because we have a square here, when I multiply my random variable by a constant, this x gets multiplied by a constant, the mean gets multiplied by a constant, the square gets multiplied by the square of that constant.
And because of that reason, we get this square of alpha showing up here.
So that's how variances transform under linear transformations.
You multiply your random variable by constant, the variance goes up by the square of that same constant.
OK. That's it for today.
See you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK so let's start.
So today, we're going to continue the subject from last time.
And the subject is random variables.
As we discussed, random variables basically associate numerical values with the outcomes of an experiment.
And we want to learn how to manipulate them.
Now to a large extent, what's going to happen, what's happening during this chapter, is that we are revisiting the same concepts we have seen in chapter one.
But we're going to introduce a lot of new notation, but really dealing with the same kind of stuff.
The only difference where we go beyond the new notation, the new concept in this chapter is the concept of the expectation or expected values.
And we're going to learn how to manipulate expectations.
So let us start with a quick review of what we discussed last time.
We talked about random variables.
Loosely speaking, random variables are random quantities that result from an experiment.
More precisely speaking, mathematically speaking, a random variable is a function from the sample space to the real numbers.
That is, you give me an outcome, and based on that outcome, I can tell you the value of the random variable.
So the value of the random variable is a function of the outcome that we have.
Now given a random variable, some of the numerical outcomes are more likely than others.
And we want to say which ones are more likely and how likely they are.
And the way we do that is by writing down the probabilities of the different possible numerical outcomes.
Notice here, the notation.
We use uppercase to denote the random variable.
We use lowercase to denote real numbers.
So the way you read this, this is the probability that the random variable, capital X, happens to take the numerical value, little x.
This is a concept that's familiar from chapter one.
And this is just the new notation we will be using for that concept.
It's the Probability Mass Function of the random variable, capital X.
So the subscript just indicates which random variable we're talking about.
And it's the probability assigned to a particular outcome.
And we want to assign such probabilities for all possibly numerical values.
So you can think of this as being a function of little x.
And it tells you how likely every little x is going to be.
Now the new concept we introduced last time is the concept of the expected value for random variable, which is defined this way.
You look at all the possible outcomes.
And you form some kind of average of all the possible numerical values over the random variable capital X.
You consider all the possible numerical values, and you form an average.
In fact, it's a weighted average where, to every little x, you assign a weight equal to the probability that that particular little x is going to be realized.
Now, as we discussed last time, if you have a random variable, you can take a function of a random variable.
And that's going to be a new random variable.
So if capital X is a random variable and g is a function, g of X is a new random variable.
You do the experiment.
You get an outcome.
This determines the value of X.
And that determines the value of g of X.
So the numerical value of g of X is determined by whatever happens in the experiment.
It's random.
And that makes it a random variable.
Since it's a random variable, it has an expectation of its own.
So how would we calculate the expectation of g of X?
You could proceed by just using the definition, which would require you to find the PMF of the random variable g of X.
So find the PMF of g of X, and then apply the formula for the expected value of a random variable with known PMF.
But there is also a shortcut, which is just a different way of doing the counting and the calculations, in which we do not need to find the PMF of g of X.
We just work with the PMF of the original random variable.
And what this is saying is that the average value of g of X is obtained as follows.
You look at all the possible results, the X's, how likely they are.
And when that particular X happens, this is how much you get.
And so this way, you add these things up.
And you get the average amount that you're going to get, the average value of g of X, where you average over the likelihoods of the different X's.
Now expected values have some properties that are always true and some properties that sometimes are not true.
So the property that is not always true is that this would be the same as g of the expected value of X.
So in general, this is not true.
You cannot interchange function and expectation, which means you cannot reason on the average, in general.
But there are some exceptions.
When g is a linear function, then the expected value for a linear function is the same as that same linear function of the expectation.
So for linear functions, so for random variable, the expectation behaves nicely.
So this is basically telling you that, if X is degrees in Celsius, alpha X plus b is degrees in Fahrenheit, you can first do the conversion to Fahrenheit and take the average.
Or you can find the average temperature in Celsius, and then do the conversion to Fahrenheit.
Either is valid.
So the expected value tells us something about where is the center of the distribution, more specifically, the center of mass or the center of gravity of the PMF, when you plot it as a bar graph.
Besides the average value, you may be interested in knowing how far will you be from the average, typically.
So let's look at this quantity, X minus expected value of X.
This is the distance from the average value.
So for a random outcome of the experiment, this quantity in here measures how far away from the mean you happen to be.
This quantity inside the brackets is a random variable.
Why?
Because capital X is random.
And what we have here is capital X, which is random, minus a number.
Remember, expected values are numbers.
Now a random variable minus a number is a new random variable.
It has an expectation of its own.
We can use the linearity rule, expected value of something minus something else is just the difference of their expected value.
So it's going to be expected value of X minus the expected value over this thing.
Now this thing is a number.
And the expected value of a number is just the number itself.
So we get from here that this is expected value minus expected value.
And we get zero.
What is this telling us?
That, on the average, the assigned difference from the mean is equal to zero.
That is, the mean is here.
Sometimes X will fall to the right.
Sometimes X will fall to the left.
On the average, the average distance from the mean is going to be zero, because sometimes the realized distance will be positive, sometimes it will be negative.
Positives and negatives cancel out.
So if we want to capture the idea of how far are we from the mean, just looking at the assigned distance from the mean is not going to give us any useful information.
So if we want to say something about how far we are, typically, we should do something different.
One possibility might be to take the absolute values of the differences.
And that's a quantity that sometimes people are interested in.
But it turns out that a more useful quantity happens to be the variance of a random variable, which actually measures the average squared distance from the mean.
So you have a random outcome, random results, random numerical value of the random variable.
It is a certain distance away from the mean.
That certain distance is random.
We take the square of that.
This is the squared distance from the mean, which is again random.
Since it's random, it has an expected value of its own.
And that expected value, we call it the variance of X.
And so we have this particular definition.
Using the rule that we have up here for how to calculate expectations of functions of a random variable, why does that apply?
Well, what we have inside the brackets here is a function of the random variable, capital X.
So we can apply this rule where g is this particular function.
And we can use that to calculate the variance, starting with the PMF of the random variable X.
And then we have a useful formula that's a nice shortcut, sometimes, if you want to do the calculation.
Now one thing that's slightly wrong with the variance is that the units are not right, if you want to talk about the spread a of a distribution.
Suppose that X is a random variable measured in meters.
The variance will have the units of meters squared.
So it's a kind of a different thing.
If you want to talk about the spread of the distribution using the same units as you have for X, it's convenient to take the square root of the variance.
And that's something that we define.
And we call it to the standard deviation of X, or the standard deviation of the distribution of X.
So it tells you the amount of spread in your distribution.
And it is in the same units as the random variable itself that you are dealing with.
And we can just illustrate those quantities with an example that's about as simple as it can be.
So consider the following experiment.
You're going to go from here to New York, let's say, 200 miles.
And you have two alternatives.
Either you'll get your private plane and go at a speed of 200 miles per hour, constant speed during your trip, or otherwise, you'll decide to walk really, really slowly, at the leisurely pace of one mile per hour.
So you pick the speed at random by doing this experiment, by flipping a coin.
And with probability one-half, you do one thing.
With probably one-half, you do the other thing.
So your V is a random variable.
In case you're interested in how much time it's going to take you to get there, well, time is equal to distance divided by speed.
So that's the formula.
The time itself is a random variable, because it's a function of V, which is random.
How much time it's going to take you depends on the coin flip that you do in the beginning to decide what speed you are going to have.
OK, just as a warm up, the trivial calculations.
To find the expected value of V, you argue as follows.
With probability one-half, V is going to be one.
And with probability one-half, V is going to be 200.
And so the expected value of your speed is 100.5.
If you wish to calculate the variance of V, then you argue as follows.
With probability one-half, I'm going to travel at the speed of one, whereas, the mean is 100.5.
So this is the distance from the mean, if I decide to travel at the speed of one.
We take that distance from the mean squared.
That's one contribution to the variance.
And with probability one-half, you're going to travel at the speed of 200, which is this much away from the mean.
You take the square of that.
OK, so approximately how big is this number?
Well, this is roughly 100 squared.
That's also 100 squared.
So approximately, the variance of this random variable is 100 squared.
Now if I tell you that the variance of this distribution is 10,000, it doesn't really help you to relate it to this diagram.
Whereas, the standard deviation, where you take the square root, is more interesting.
It's the square root of 100 squared, which is a 100.
And the standard deviation, indeed, gives us a sense of how spread out this distribution is from the mean.
So the standard deviation basically gives us some indication about this spacing that we have here.
It tells us the amount of spread in our distribution.
OK, now let's look at what happens to time.
V is a random variable.
T is a random variable.
So now let's look at the expected values and all of that for the time.
OK, so the time is a function of a random variable.
We can find the expected time by looking at all possible outcomes of the experiment, the V's, weigh them according to their probabilities, and for each particular V, keep track of how much time it took us.
So if V is one, which happens with probability one-half, the time it takes is going to be 200.
If we travel at speed of one, it takes us 200 time units.
And otherwise, if our speed is equal to 200, the time is one.
So the expected value of T is once more the same as before.
It's 100.5.
So the expected speed is 100.5.
The expected time is also 100.5.
So the product of these expectations is something like 10,000.
How about the expected value of the product of T and V?
Well, T times V is 200.
No matter what outcome you have in the experiment, in that particular outcome, T times V is total distance traveled, which is exactly 200.
And so what do we get in this simple example is that the expected value of the product of these two random variables is different than the product of their expected values.
This is one more instance of where we cannot reason on the average.
So on the average, over a large number of trips, your average time would be 100.
On the average, over a large number of trips, your average speed would be 100.
But your average distance traveled is not 100 times 100.
It's something else.
So you cannot reason on the average, whenever you're dealing with non-linear things.
And the non-linear thing here is that you have a function which is a product of stuff, as opposed to just linear sums of stuff.
Another way to look at what's happening here is the expected value of the time.
Time, by definition, is 200 over the speed.
Expected value of the time, we found it to be about a 100.
And so expected value of 200 over V is about a 100.
But it's different from this quantity here, which is roughly equal to 2, and so 200.
Expected value of V is about 100.
So this quantity is about equal to two.
Whereas, this quantity up here is about 100.
So what do we have here?
We have a non-linear function of V. And we find that the expected value of this function is not the same thing as the function of the expected value.
So again, that's an instance where you cannot interchange expected values and functions.
And that's because things are non-linear.
OK, so now let us introduce a new concept.
Or maybe it's not quite a new concept.
So we discussed, in chapter one, that we have probabilities.
We also have conditional probabilities.
What's the difference between them?
Essentially, none.
Probabilities are just an assignment of probability values to give different outcomes, given a particular model.
Somebody comes and gives you new information.
So you come up with a new model.
And you have a new probabilities.
We call these conditional probabilities, but they taste and behave exactly the same as ordinary probabilities.
So since we can have conditional probabilities, why not have conditional PMFs as well, since PMFs deal with probabilities anyway.
So we have a random variable, capital X.
It has a PMF of its own.
For example, it could be the PMF in this picture, which is a uniform PMF that takes for possible different values.
And we also have an event.
And somebody comes and tells us that this event has occurred.
The PMF tells you the probability that capital X equals to some little x.
Somebody tells you that a certain event has occurred that's going to make you change the probabilities that you assign to the different values.
You are going to use conditional probabilities.
So this part, it's clear what it means from chapter one.
And this part is just the new notation we're using in this chapter to talk about conditional probabilities.
So this is just a definition.
So the conditional PMF is an ordinary PMF.
But it's the PMF that applies to a new model in which we have been given some information about the outcome of the experiment.
So to make it concrete, consider this event here.
Take the event that capital X is bigger than or equal to two.
In the picture, what is the event A?
The event A consists of these three outcomes.
OK, what is the conditional PMF, given that we are told that event A has occurred?
Given that the event A has occurred, it basically tells us that this outcome has not occurred.
There's only three possible outcomes now.
In the new universe, in the new model where we condition on A, there's only three possible outcomes.
Those three possible outcomes were equally likely when we started.
So in the conditional universe, they will remain equally likely.
Remember, whenever you condition, the relative likelihoods remain the same.
They keep the same proportions.
They just need to be re-scaled, so that they add up to one.
So each one of these will have the same probability.
Now in the new world, probabilities need to add up to 1.
So each one of them is going to get a probability of 1/3 in the conditional universe.
So this is our conditional model.
So our PMF is equal to 1/3 for X equals to 2, 3 and 4.
All right.
Now whenever you have a probabilistic model involving a random variable and you have a PMF for that random variable, you can talk about the expected value of that random variable.
We defined expected values just a few minutes ago.
Here, we're dealing with a conditional model and conditional probabilities.
And so we can also talk about the expected value of the random variable X in this new universe, in this new conditional model that we're dealing with.
And this leads us to the definition of the notion of a conditional expectation.
The conditional expectation is nothing but an ordinary expectation, except that you don't use the original PMF.
You use the conditional PMF.
You use the conditional probabilities.
It's just an ordinary expectation, but applied to the new model that we have to the conditional universe where we are told that the certain event has occurred.
So we can now calculate the condition expectation, which, in this particular example, would be 1/3.
That's the probability of a 2, plus 1/3 which is the probability of a 3 plus 1/3, the probability of a 4.
And then you can use your calculator to find the answer, or you can just argue by symmetry.
The expected value has to be the center of gravity of the PMF we're working with, which is equal to 3.
So conditional expectations are no different from ordinary expectations.
They're just ordinary expectations applied to a new type of situation or a new type of model.
Anything we might know about expectations will remain valid about conditional expectations.
So for example, the conditional expectation of a linear function of a random variable is going to be the linear function of the conditional expectations.
Or you can take any formula that you might know, such as the formula that expected value of X is equal to the-- sorry-- expected value of g of X is the sum over all X's of g of X times the PMF of X.
So this is the formula that we already know about how to calculate expectations of a function of a random variable.
If we move to the conditional universe, what changes?
In the conditional universe, we're talking about the conditional expectation, given that event A has occurred.
And we use the conditional probabilities, given that A has occurred.
So any formula has a conditional counterpart.
In the conditional counterparts, expectations get replaced by conditional expectations.
And probabilities get replaced by conditional probabilities.
So once you know the first formula and you know the general idea, there's absolutely no reason for you to memorize a formula like this one.
You shouldn't even have to write it on your cheat sheet for the exam, OK?
OK, all right, so now let's look at an example of a random variable that we've seen before, the geometric random variable, and this time do something a little more interesting with it.
Do you remember from last time what the geometric random variable is?
We do coin flips.
Each time there's a probability of P of obtaining heads.
And we're interested in the number of tosses we're going to need until we observe heads for the first time.
The probability that the random variable takes the value K, this is the probability that the first K appeared at the K-th toss.
So this is the probability of K minus 1 consecutive tails followed by a head.
So this is the probability of having to weight K tosses.
And when we plot this PMF, it has this kind of shape, which is the shape of a geometric progression.
It starts at 1, and it goes all the way to infinity.
So this is a discrete random variable that takes values over an infinite set, the set of the positive integers.
So it's a random variable, therefore, it has an expectation.
And the expected value is, by definition, we'll consider all possible values of the random variable.
And we weigh them according to their probabilities, which leads us to this expression.
You may have evaluated that expression some time in your previous life.
And there are tricks for how to evaluate this and get a closed-form answer.
But it's sort of an algebraic trick.
You might not remember it.
How do we go about doing this summation?
Well, we're going to use a probabilistic trick and manage to evaluate the expectation of X, essentially, without doing any algebra.
And in the process of doing so, we're going to get some intuition about what happens in coin tosses and with geometric random variables.
So we have two people who are going to do the same experiment, flip a coin until they obtain heads for the first time.
One of these people is going to use the letter Y to count how many heads it took.
So that person starts flipping right now.
This is the current time.
And they are going to obtain tails, tails, tails, until eventually they obtain heads.
And this random variable Y is, of course, geometric, so it has a PMF of this form.
OK, now there is a second person who is doing that same experiment.
That second person is going to take, again, a random number, X, until they obtain heads for the first time.
And of course, X is going to have the same PMF as Y.
But that person was impatient.
And they actually started flipping earlier, before the Y person started flipping.
They flipped the coin twice.
And they were unlucky, and they obtained tails both times.
And so they have to continue.
Looking at the situation at this time, how do these two people compare?
Who do you think is going to obtain heads first?
Is one more likely than the other?
So if you play at the casino a lot, you'll say, oh, there were two tails in a row, so a head should be coming up sometime soon.
But this is a wrong argument, because coin flips, at least in our model, are independent.
The fact that these two happened to be tails doesn't change anything about our beliefs about what's going to be happening here.
So what's going to be happening to that person is they will be flipping independent coin flips.
That person will also be flipping independent coin flips.
And both of them wait until the first head occurs.
They're facing an identical situation, starting from this time.
OK, now what's the probabilistic model of what this person is facing?
The time until that person obtains heads for the first time is X.
So this number of flips until they obtain heads for the first time is going to be X minus 2.
So X is the total number until the first head.
X minus 2 is the number or flips, starting from here.
Now what information do we have about that person?
We have the information that their first two flips were tails.
So we're given the information that X was bigger than 2.
So the probabilistic model that describes this piece of the experiment is that it's going to take a random number of flips until the first head.
That number of flips, starting from here until the next head, is that number X minus 2.
But we're given the information that this person has already wasted 2 coin flips.
Now we argued that probabilistically, this person, this part of the experiment here is identical with that part of the experiment.
So the PMF of this random variable, which is X minus 2, conditioned on this information, should be the same as that PMF that we have down there.
So the formal statement that I'm making is that this PMF here of X minus 2, given that X is bigger than 2, is the same as the PMF of X itself.
What is this saying?
Given that I tell you that you already did a few flips and they were failures, the remaining number of flips until the first head has the same geometric distribution as if you were starting from scratch.
Whatever happened in the past, it happened, but has no bearing what's going to happen in the future.
Remaining coin flips until a head has the same distribution, whether you're starting right now, or whether you had done some other stuff in the past.
So this is a property that we call the memorylessness property of the geometric distribution.
Essentially, it says that whatever happens in the future is independent from whatever happened in the past.
And that's true almost by definition, because we're assuming independent coin flips.
Really, independence means that information about one part of the experiment has no bearing about what's going to happen in the other parts of the experiment.
The argument that I tried to give using the intuition of coin flips, you can make it formal by just manipulating PMFs formally.
So this is the original PMF of X.
Suppose that you condition on the event that X is bigger than 3.
This conditioning information, what it does is it tells you that this piece did not happen.
You're conditioning just on this event.
When you condition on that event, what's left is the conditional PMF, which has the same shape as this one, except that it needs to be re-normalized up, so that the probabilities add up to one.
So you take that picture, but you need to change the height of it, so that these terms add up to 1.
And this is the conditional PMF of X, given that X is bigger than 2.
But we're talking here not about X.
We're talking about the remaining number of heads.
Remaining number of heads is X minus 2.
If we have the PMF of X, can we find the PMF of X minus 2?
Well, if X is equal to 3, that corresponds to X minus 2 being equal to 1.
So this probability here should be equal to that probability.
The probability that X is equal to 4 should be the same as the probability that X minus 2 is equal to 2.
So basically, the PMF of X minus 2 is the same as the PMF of X, except that it gets shifted by these 2 units.
So this way, we have formally derived the conditional PMF of the remaining number of coin tosses, given that the first two flips were tails.
And we see that it's exactly the same as the PMF that we started with.
And so this is the formal proof of this statement here.
So it's useful here to digest both these formal statements and understand it and understand the notation that is involved here, but also to really appreciate the intuitive argument what this is really saying.
OK, all right, so now we want to use this observation, this memorylessness, to eventually calculate the expected value for a geometric random variable.
And the way we're going to do it is by using a divide and conquer tool, which is an analog of what we have already seen sometime before.
Remember our story that there's a number of possible scenarios about the world?
And there's a certain event, B, that can happen under any of these possible scenarios.
And we have the total probability theory.
And that tells us that, to find the probability of this event, B, you consider the probabilities of B under each scenario.
And you weigh those probabilities according to the probabilities of the different scenarios that we have.
So that's a formula that we already know and have worked with.
What's the next step?
Is it something deep?
No, it's just translation in different notation.
This is the exactly same formula, but with PMFs.
The event that capital X is equal to little x can happen in many different ways.
It can happen under either scenario.
And within each scenario, you need to use the conditional probabilities of that event, given that this scenario has occurred.
So this formula is identical to that one, except that we're using conditional PMFs, instead of conditional probabilities.
But conditional PMFs, of course, are nothing but conditional probabilities anyway.
So nothing new so far.
Then what I do is to take this formula here and multiply both sides by X and take the sum over all X's.
What do we get on this side?
We get the expected value of X.
What do we get on that side?
Probability of A1.
And then here, sum over all X's of X times P. That's, again, the same calculation we have when we deal with expectations, except that, since here, we're dealing with conditional probabilities, we're going to get the conditional expectation.
And this is the total expectation theorem.
It's a very useful way for calculating expectations using a divide and conquer method.
We figure out the average value of X under each one of the possible scenarios.
The overall average value of X is a weighted linear combination of the expected values of X in the different scenarios where the weights are chosen according to the different probabilities.
OK, and now we're going to apply this to the case of a geometric random variable.
And we're going to divide and conquer by considering separately the two cases where the first toss was heads, and the other case where the first toss was tails.
So the expected value of X is the probability that the first toss was heads, so that X is equal to 1, and the expected value if that happened.
What is the expected value of X, given that X is equal to 1?
If X is known to be equal to 1, then X becomes just a number.
And the expected value of a number is the number itself.
So this first line here is the probability of heads in the first toss times the number 1.
So the probability that X is bigger than 1 is 1 minus P. And then we need to do something about this conditional expectation.
What is it?
I can write it in, perhaps, a more suggested form, as expected the value of X minus 1, given that X minus 1 is bigger than 1.
Ah.
OK, X bigger than 1 is the same as X minus 1 being positive, this way.
X minus 1 is positive plus 1.
What did I do here?
I added and subtracted 1.
Now what is this?
This is the expected value of the remaining coin flips, until I obtain heads, given that the first one was tails.
It's the same story that we were going through down there.
Given that the first coin flip was tails doesn't tell me anything about the future, about the remaining coin flips.
So this expectation should be the same as the expectation faced by a person who was starting just now.
So this should be equal to the expected value of X itself.
And then we have the plus 1 that's come from there, OK?
Remaining coin flips until a head, given that I had a tail yesterday, is the same as expected number of flips until heads for a person just starting now and wasn't doing anything yesterday.
So the fact that they I had a coin flip yesterday doesn't change my beliefs about how long it's going to take me until the first head.
So once we believe that relation, than we plug this here.
And this red term becomes expected value of X plus 1.
So now we didn't exactly get the answer we wanted, but we got an equation that involves the expected value of X.
And it's the only unknown in that equation.
Expected value of X equals to P plus (1 minus P) times this expression.
You solve this equation for expected value of X, and you get the value of 1/P.
The final answer does make intuitive sense.
If P is small, heads are difficult to obtain.
So you expect that it's going to take you a long time until you see heads for the first time.
So it is definitely a reasonable answer.
Now the trick that we used here, the divide and conquer trick, is a really nice one.
It gives us a very good shortcut in this problem.
But you must definitely spend some time making sure you understand why this expression here is the same as that expression there.
Essentially, what it's saying is that, if I tell you that X is bigger than 1, that the first coin flip was tails, all I'm telling you is that that person has wasted a coin flip, and they are starting all over again.
So they've wasted 1 coin flip.
And they're starting all over again.
If I tell you that the first flip was tails, that's the only information that I'm basically giving you, a wasted flip, and then starts all over again.
All right, so in the few remaining minutes now, we're going to quickly introduce a few new concepts that we will be playing with in the next ten days or so.
And you will get plenty of opportunities to manipulate them.
So here's the idea.
A typical experiment may have several random variables associated with that experiment.
So a typical student has height and weight.
If I give you the PMF of height, that tells me something about distribution of heights in the class.
I give you the PMF of weight, it tells me something about the different weights in this class.
But if I want to ask a question, is there an association between height and weight, then I need to know a little more how height and weight relate to each other.
And the PMF of height individuality and PMF of weight just by itself do not tell me anything about those relations.
To be able to say something about those relations, I need to know something about joint probabilities, how likely is it that certain X's go together with certain Y's.
So these probabilities, essentially, capture associations between these two random variables.
And it's the information I would need to have to do any kind of statistical study that tries to relate the two random variables with each other.
These are ordinary probabilities.
This is an event.
It's the event that this thing happens and that thing happens.
This is just the notation that we will be using.
It's called the joint PMF.
It's the joint Probability Mass Function of the two random variables X and Y looked at together, jointly.
And it gives me the probability that any particular numerical outcome pair does happen.
So in the finite case, you can represent joint PMFs, for example, by a table.
This particular table here would give you information such as, let's see, the joint PMF evaluated at 2, 3.
This is the probability that X is equal to 3 and, simultaneously, Y is equal to 3.
So it would be that number here.
It's 4/20.
OK, what is a basic property of PMFs?
First, these are probabilities, so all of the entries have to be non-negative.
If you adopt the probabilities over all possible numerical pairs that you could get, of course, the total probability must be equal to 1.
So that's another thing that we want.
Now suppose somebody gives me this model, but I don't care about Y's.
All I care is the distribution of the X's.
So I'm going to find the probability that X takes on a particular value.
Can I find it from the table?
Of course, I can.
If you ask me what's the probability that X is equal to 3, what I'm going to do is to add up those three probabilities together.
And those probabilities, taken all together, give me the probability that X is equal to 3.
These are all the possible ways that the event X equals to 3 can happen.
So we add these, and we get the 6/20.
What I just did, can we translate it to a formula?
What did I do?
I fixed the particular value of X.
And I added up the values of the joint PMF over all the possible values of Y.
So that's how you do it.
You take the joint.
You take one slice of the joint, keeping X fixed, and adding up over the different values of Y.
The moral of this example is that, if you know the joint PMFs, then you can find the individual PMFs of every individual random variable.
And we have a name for these.
We call them the marginal PMFs.
We have the joint that talks about both together, and the marginal that talks about them one at the time.
And finally, since we love conditional probabilities, we will certainly want to define an object called the conditional PMF.
So this quantity here is a familiar one.
It's just a conditional probability.
It's the probability that X takes on a particular value, given that Y takes a certain value.
For our example, let's take little y to be equal to 2, which means that we're conditioning to live inside this universe.
This red universe here is the y equal to 2 universe.
And these are the conditional probabilities of the different X's inside that universe.
OK, once more, just an exercise in notation.
This is the chapter two version of the notation of what we were denoting this way in chapter one.
The way to read this is that it's a conditional PMF having to do with two random variables, the PMF of X conditioned on information about Y.
We are fixing a particular value of capital Y, that's the value on which we are conditioning.
And we're looking at the probabilities of the different X's.
So it's really a function of two arguments, little x and little y.
But the best way to think about it is to fix little y and think of it as a function of X.
So I'm fixing little y here, let's say, to y equal to 2.
So I'm considering only this.
And now, this quantity becomes a function of little x.
For the different little x's, we're going to have different conditional probabilities.
What are those conditional probabilities?
OK, conditional probabilities are proportional to original probabilities.
So it's going to be those numbers, but scaled up.
And they need to be scaled so that they add up to 1.
So we have 1, 3 and 1.
That's a total of 5.
So the conditional PMF would have the shape zero, 1/5, 3/5, and 1/5.
This is the conditional PMF, given a particular value of Y.
It has the same shape as those numbers, where by shape, I mean try to visualize a bar graph.
The bar graph associated with those numbers has exactly the same shape as the bar graph associated with those numbers.
The only thing that has changed is the scaling.
Big moral, let me say in different words, the conditional PMF, given a particular value of Y, is just a slice of the joint PMF where you maintain the same shape, but you rescale the numbers so that they add up to 1.
Now mathematically, of course, what all of this is doing is it's taking the original joint PDF and it rescales it by a certain factor.
This does not involve X, so the shape, is a function of X, has not changed.
We're keeping the same shape as a function of X, but we divide by a certain number.
And that's the number that we need, so that the conditional probabilities add up to 1.
Now where does this formula come from?
Well, this is just the definition of conditional probabilities.
Probability of something conditioned on something else is the probability of both things happening, the intersection of the two divided by the probability of the conditioning event.
And last remark is that, as I just said, conditional probabilities are nothing different than ordinary probabilities.
So a conditional PMF must sum to 1, no matter what you are conditioning on.
All right, so this was sort of quick introduction into our new notation.
But you get a lot of practice in the next days to come.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, good morning.
So today, we're going to have a fairly packed lecture.
We are going to conclude with chapter two, discrete random variables.
And we will be talking mostly about multiple random variables.
And this is also the last lecture as far as quiz one is concerned.
So it's going to cover the material until today, and of course the next recitation and tutorial as well.
OK, so we're going to review quickly what we introduced at the end of last lecture, where we talked about the joint PMF of two random variables.
We're going to talk about the case of more than two random variables as well.
We're going to talk about the familiar concepts of conditioning and independence, but applied to random variables instead of events.
We're going to look at the expectations once more, talk about a few properties that they have, and then solve a couple of problems and calculate a few things in somewhat clever ways.
So the first point I want to make is that, to a large extent, whatever is happening in our chapter on discrete random variables is just an exercise in notation.
There is stuff and concepts that you are already familiar with-- probabilities, probabilities of two things happening, conditional probabilities.
And all that we're doing, to some extent, is rewriting those familiar concepts in new notation.
So for example, this is the joint PMF of two random variable.
It gives us, for any pair or possible values of those random variables, the probability that that pair occurs simultaneously.
So it's the probability that simultaneously x takes that value, and y takes that other value.
And similarly, we have the notion of the conditional PMF, which is just a list of the -- condition of -- the various conditional probabilities of interest, conditional probability that one random variable takes this value given that the other random variable takes that value.
Now, a remark about conditional probabilities.
Conditional probabilities generally are like ordinary probabilities.
You condition on something particular.
So here we condition on a particular y.
So think of little y as a fixed quantity.
And then look at this as a function of x.
So given that y, which we condition on, given our new universe, we're considering the various possibilities for x and the probabilities that they have.
Now, the probabilities over all x's, of course, needs to add to 1.
So we should have a relation of this kind.
So they're just like ordinary probabilities over the different x's in a universe where we are told the value of the random variable y.
Now, how are these related?
So we call these the marginal, these the joint, these the conditional.
And there are some relations between these.
For example, to find the marginal from the joint, it's pretty straightforward.
The probability that x takes a particular value is the sum of the probabilities of all of the different ways that this particular value may occur.
What are the different ways?
Well, it may occur together with a certain y, or together with some other y, or together with some other y.
So you look at all the possible y's that can go together with this x, and add the probabilities of all of those pairs for which we get this particular value of x.
And then there's a relation between that connects these two probabilities with the conditional probability.
And it's this relation.
It's nothing new.
It's just new notation for writing what we already know, that the probability of two things happening is the probability that the first thing happens, and then given that the first thing happens, the probability that the second one happened.
So how do we go from one to the other?
Think of A as being the event that X takes the value, little x, and B being the event that Y takes the value, little y.
So the joint probability is the probability that these two things happen simultaneously.
It's the probability that X takes this value times the conditional probability that Y takes this value, given that X took that first value.
So it's the familiar multiplication rule, but just transcribed in our new notation.
So nothing new so far.
OK, why did we go through this exercise and this notation?
It's because in the experiments where we're interested in the real world, typically there's going to be lots of uncertain quantities.
There's going to be multiple random variables.
And we want to be able to talk about them simultaneously.
Okay.
Why two and not more than two?
How about three random variables?
Well, if you understand what's going on in this slide, you should be able to kind of automatically generalize this to the case of multiple random variables.
So for example, if we have three random variables, X, Y, and Z, and you see an expression like this, it should be clear what it means.
It's the probability that X takes this value and simultaneously Y takes that value and simultaneously Z takes that value.
I guess that's an uppercase Z here, that's a lowercase z.
And if I ask you to find the marginal of X, if I tell you the joint PMF of the three random variables and I ask you for this value, how would you find it?
Well, you will try to generalize this relation here.
The probability that x occurs is the sum of the probabilities of all events that make X to take that particular value.
So what are all the events?
Well, this particular x can happen together with some y and some z.
We don't care which y and z.
Any y and z will do.
So when we consider all possibilities, we need to add here over all possible values of y's and z's.
So consider all triples, x, y, z.
Fix x and consider all the possibilities for the remaining variables, y and z, add these up, and that gives you the marginal PMF of X.
And then there's other things that you can do.
This is the multiplication rule for two events.
We saw back in chapter one that there's a multiplication rule when you talk about more than two events.
And you can write a chain of conditional probabilities.
We can certainly do the same in our new notation.
So let's look at this rule up here.
Multiplication rule for three random variables, what does it say?
The probability of three things happening simultaneously, X, Y, Z taking specific values, little x, little y, little z, that probability is the probability that the first thing happens, that X takes that value.
Given that X takes that value, we multiply it with the probability that Y takes also a certain value.
And now, given that X and Y have taken those particular values, we multiply with a conditional probability that the third thing happens, given that the first two things happen.
So this is just the multiplication rule for three events, which would be probability of A intersection B intersection C equals-- you know the rest of the formula.
You just rewrite this formula in PMF notation.
Probability of A intersection B intersection C is the probability of A, which corresponds to this term, times the probability of B given A, times the probability of C given A and B.
So what else is there that's left from chapter one that we can or should generalize to random variables?
Well, there's the notion of independence.
So let's define what independence means.
Instead of talking about just two random variables, let's go directly to the case of multiple random variables.
When we talked about events, things were a little complicated.
We had a simple definition for independence of two events.
Two events are independent if the probability of both is equal to the product of the probabilities.
But for three events, it was kind of messy.
We needed to write down lots of conditions.
For random variables, things in some sense are a little simpler.
We only need to write down one formula and take this as the definition of independence.
Three random variables are independent if and only if, by definition, their joint probability mass function factors out into individual probability mass functions.
So the probability that all three things happen is the product of the individual probabilities that each one of these three things is happening.
So independence means mathematically that you can just multiply probabilities to get to the probability of several things happening simultaneously.
So with three events, we have to write a huge number of equations, of equalities that have to hold.
How can it be that with random variables we can only manage with one equality?
Well, the catch is that this is not really just one equality.
We require this to be true for every little x, y, and z.
So in some sense, this is a bunch of conditions that are being put on the joint PMF, a bunch of conditions that we need to check.
So this is the mathematical definition.
What is the intuitive content of this definition?
The intuitive content is the same as for events.
Random variables are independent if knowing something about the realized values of some of these random variables does not change our beliefs about the likelihood of various values for the remaining random variables.
So independence would translate, for example, to a condition such as the conditional PMF of X , given y, should be equal to the marginal PMF of X.
What is this saying?
That you have some original beliefs about how likely it is for X to take this value.
Now, someone comes and tells you that Y took on a certain value.
This causes you, in principle, to revise your beliefs.
And your new beliefs will be captured by the conditional PMF, or the conditional probabilities.
Independence means that your revised beliefs actually will be the same as your original beliefs.
Telling you information about the value of Y doesn't change what you expect for the random variable X.
Why didn't we use this definition for independence?
Well, because this definition only makes sense when this conditional is well-defined.
And this conditional is only well-defined if the events that Y takes on that particular value has positive probability.
We cannot condition on events that have zero probability, so conditional probabilities are only defined for y's that are likely to occur, that have a positive probability.
Now, similarly, with multiple random variables, if they're independent, you would have relations such as the conditional of X, given y and z, should be the same as the marginal of X.
What is this saying?
Again, that if I tell you the values, the realized values of random variables Y and Z, this is not going to change your beliefs about how likely x is to occur.
Whatever you believed in the beginning, you're going to believe the same thing afterwards.
So it's important to keep that intuition in mind, because sometimes this way you can tell whether random variables are independent without having to do calculations and to check this formula.
OK, so let's check our concepts with a simple example.
Let's look at two random variables that are discrete, take values between one and for each.
And this is a table that gives us the joint PMF.
So it tells us the probability that X equals to 2 and Y equals to 1 happening simultaneously.
It's an event that has probability 1/20.
Are these two random variables independent?
You can try to check a condition like this.
But can we tell directly from the table?
If I tell you a value of Y, could that give you useful information about X?
Certainly.
If I tell you that Y is equal to 1, this tells you that X must be equal to 2.
But if I tell you that Y was equal to 3, this tells you that, still, X could be anything.
So telling you the value of Y kind of changes what you expect or what you consider possible for the values of the other random variable.
So by just inspecting here, we can tell that the random variables are not independent.
Okay.
What's the other concept we introduced in chapter one?
We introduced the concept of conditional independence.
And conditional independence is like ordinary independence but applied to a conditional universe where we're given some information.
So suppose someone tells you that the outcome of the experiment is such that X is less than or equal to 2 and Y is larger than or equal to 3.
So we are given the information that we now live inside this universe.
So what happens inside this universe?
Inside this universe, our random variables are going to have a new joint PMF which is conditioned on the event that we were told that it has occurred.
So let A correspond to this sort of event here.
And now we're dealing with conditional probabilities.
What are those conditional probabilities?
We can put them in a table.
So it's a two by two table, since we only have two possible values.
What are they going to be?
Well, these probabilities show up in the ratios 1, 2, 2, and 4.
Those ratios have to stay the same.
The probabilities need to add up to one.
So what should the denominators be since these numbers add up to nine?
These are the conditional probabilities.
So this is the conditional PMF in this example.
Now, in this conditional universe, is x independent from y?
If I tell you that y takes this value, so we live in this universe, what do you know about x?
What you know about x is at this value is twice as likely as that value.
If I condition on y taking this value, so we're living here, what do you know about x?
What you know about x is that this value is twice as likely as that value.
So it's the same.
Whether we live here or we live there, this x is twice as likely as that x.
So the conditional PMF in this new universe, the conditional PMF of X given y, in the new universe is the same as the marginal PMF of X, but of course in the new universe.
So no matter what y is, the conditional PMF of X is the same.
And that conditional PMF is 1/3 and 2/3.
This is the conditional PMF of X in the new universe no matter what y occurs.
So Y does not give us any information about X, doesn't cause us to change our beliefs inside this little universe.
And therefore the two random variables are independent.
Now, the other way that you can verify that we have independence is to find the marginal PMFs of the two random variables.
The marginal PMF of X, you find it by adding those two terms.
You get 1/3.
Adding those two terms, you get 2/3.
Marginal PMF of Y, you find it, you add these two terms, and you get 1/3.
And the marginal PMF of Y here is going to be 2/3.
And then you ask the question, is the joint the product of the marginals?
And indeed it is.
This times this gives you 1/9.
This times this gives you 2/9.
So the values in the table with the joint PMFs is the product of the marginal PMFs of X and Y in this universe, so the two random variables are independent inside this universe.
So we say that they're conditionally independent.
All right.
Now let's move to the new topic, to the new concept that we introduce in this chapter, which is the concept of expectations.
So what are the things to know here?
One is the general idea.
The way to think about expectations is that it's something like the average value for random variable if you do an experiment over and over, and if you interpret probabilities as frequencies.
So you get x's over and over with a certain frequency -- P(x) -- a particular value, little x, gets realized.
And each time that this happens, you get x dollars.
How many dollars do you get on the average?
Well, this formula gives you that particular average.
So first thing we do is to write down a definition for this sort of concept.
But then the other things you need to know is how to calculate expectations using shortcuts sometimes, and what properties they have.
The most important shortcut there is is that, if you want to calculate the expected value, the average value for a random variable, you do not need to find the PMF of that random variable.
But you can work directly with the x's and the y's.
So you do the experiment over and over.
The outcome of the experiment is a pair (x,y).
And each time that a certain (x,y) happens, you get so many dollars.
So this fraction of the time, a certain (x,y) happens.
And that fraction of the time, you get so many dollars, so this is the average number of dollars that you get.
So what you end up, since it is the average, then that means that it corresponds to the expected value.
Now, this is something that, of course, needs a little bit of mathematical proof.
But this is just a different way of accounting.
And it turns out we give you the right answer.
And it's a very useful shortcut.
Now, when we're talking about functions of random variables, in general, we cannot speak just about averages.
That is, the expected value of a function of a random variable is not the same as the function of the expected values.
A function of averages is not the same as the average of a function.
So in general, this is not true.
But what it's important to know is to know the exceptions to this rule.
And the important exceptions are mainly two.
One is the case of linear functions of a random variable.
We discussed this last time.
So the expected value of temperature in Celsius is, you first find the expected value of temperature in Fahrenheit, and then you do the conversion to Celsius.
So whether you first average and then do the conversion to the new units or not, it shouldn't matter when you get the result.
The other property that turns out to be true when you talk about multiple random variables is that expectation still behaves linearly.
So let X, Y, and Z be the score of a random student at each one of the three sections of the SAT.
So the overall SAT score is X plus Y plus Z.
This is the average score, the average total SAT score.
Another way to calculate that average is to look at the first section of the SAT and see what was the average.
Look at the second section, look at what was the average, and so the third, and add the averages.
So you can do the averages for each section separately, add the averages, or you can find total scores for each student and average them.
So I guess you probably believe that this is correct if you talk just about averaging scores.
Since expectations are just the variation of averages, it turns out that this is also true in general.
And the derivation of this is very simple, based on the expected value rule.
And you can look at it in the notes.
So this is one exception, which is linearity.
The second important exception is the case of independent random variables, that the product of two random variables has an expectation which is the product of the expectations.
In general, this is not true.
But for the case where we have independence, the expectation works out as follows.
Using the expected value rule, this is how you calculate the expected value of a function of a random variable.
So think of this as being your g(X, Y) and this being your g(little x, y).
So this is something that's generally true.
Now, if we have independence, then the PMFs factor out, and then you can separate this sum by bringing together the x terms, bring them outside the y summation.
And you find that this is the same as expected value of X times the expected value of Y.
So independence is used in this step here.
OK, now what if X and Y are independent, but instead of taking the expectation of X times Y, we take the expectation of the product of two functions of X and Y?
I claim that the expected value of the product is still going to be the product of the expected values.
How do we show that?
We could show it by just redoing this derivation here.
Instead of X and Y, we would have g(X) and h(Y), so the algebra goes through.
But there's a better way to think about it which is more conceptual.
And here's the idea.
If X and Y are independent, what does it mean?
X does not convey any information about Y.
If X conveys no information about Y, does X convey information about h(Y)?
No.
If X tells me nothing about Y, nothing new, it shouldn't tell me anything about h(Y).
Now, if X tells me nothing about h of h(Y), could g(X) tell me something about h(Y)?
No.
So the idea is that, if X is unrelated to Y, doesn't have any useful information, then g(X) could not have any useful information for h(Y).
So if X and Y are independent, then g(X) and h(Y) are also independent.
So this is something that one can try to prove mathematically, but it's more important to understand conceptually why this is so.
It's in terms of conveying information.
So if X tells me nothing about Y, X cannot tell me anything about Y cubed, or X cannot tell me anything by Y squared, and so on.
That's the idea.
And once we are convinced that g(X) and h(Y) are independent, then we can apply our previous rule, that for independent random variables, expectations multiply the right way.
Apply the previous rule, but apply it now to these two independent random variables.
And we get the conclusion that we wanted.
Now, besides expectations, we also introduced the concept of the variance.
And if you remember the definition of the variance, let me write down the formula for the variance of aX.
It's the expected value of the random variable that we're looking at minus the expected value of the random variable that we're looking at.
So this is the difference of the random variable from its mean.
And we take that difference and square it, so it's the squared distance from the mean, and then take expectations of the whole thing.
So when you look at that expression, you realize that a can be pulled out of those expressions.
And because there is a squared, when you pull out the a, it's going to come out as an a-squared.
So that gives us the rule for finding the variance of a scale or product of a random variable.
The variance captures the idea of how wide, how spread out a certain distribution is.
Bigger variance means it's more spread out.
Now, if you take a random variable and the constants to it, what does it do to its distribution?
It just shifts it, but it doesn't change its width.
So intuitively it means that the variance should not change.
You can check that mathematically, but it should also make sense intuitively.
So the variance, when you add the constant, does not change.
Now, can you add variances is the way we added expectations?
Does variance behave linearly?
It turns out that not always.
Here, we need a condition.
It's only in special cases-- for example, when the two random variables are independent-- that you can add variances.
The variance of the sum is the sum of the variances if X and Y are independent.
The derivation of this is, again, very short and simple.
We'll skip it, but it's an important fact to remember.
Now, to appreciate why this equality is not true always, we can think of some extreme examples.
Suppose that X is the same as Y.
What's going to be the variance of X plus Y?
Well, X plus Y, in this case, is the same as 2X, so we're going to get 4 times the variance of X, which is different than the variance of X plus the variance of X.
So that expression would give us twice the variance of X.
But actually now it's 4 times the variance of X.
The other extreme would be if X is equal to -Y.
Then the variance is the variance of the random variable, which is always equal to 0.
Now, a random variable which is always equal to 0 has no uncertainty.
It is always equal to its mean value, so the variance, in this case, turns out to be 0.
So in both of these cases, of course we have random variables that are extremely dependent.
Why are they dependent?
Because if I tell you something about Y, it tells you an awful lot about the value of X.
There's a lot of information about X if I tell you Y, in this case or in that case.
And finally, a short drill.
If I tell you that the random variables are independent and you want to calculate the variance of a linear combination of this kind, then how do you argue?
You argue that, since X and Y are independent, this means that X and 3Y are also independent.
X has no information about Y, so X has no information about -Y. X has no information about -Y, so X should not have any information about -3Y.
So X and -3Y are independent.
So the variance of Z should be the variance of X plus the variance of -3Y, which is the variance of X plus 9 times the variance of Y.
The important thing to note here is that no matter what happens, you end up getting a plus here, not a minus.
So that's the sort of important thing to remember in this type of calculation.
So this has been all concepts, reviews, new concepts and all that.
It's the usual fire hose.
Now let's use them to do something useful finally.
So let's revisit our old example, the binomial distribution, which counts the number of successes in independent trials of a coin.
It's a biased coin that has a probability of heads, or probability of success, equal to p at each trial.
Finally, we can go through the exercise of calculating the expected value of this random variable.
And there's the way of calculating that expectation that would be the favorite of those people who enjoy algebra, which is to write down the definition of the expected value.
We add over all possible values of the random variable, over all the possible k's, and weigh them according to the probabilities that this particular k occurs.
The probability that X takes on a particular value k is, of course, the binomial PMF, which is this familiar formula.
Clearly, that would be a messy and challenging calculation.
Can we find a shortcut?
There's a very clever trick.
There's lots of problems in probability that you can approach really nicely by breaking up the random variable of interest into a sum of simpler and more manageable random variables.
And if you can make it to be a sum of random variables that are just 0's or 1's, so much the better.
Life is easier.
Random variables that take values 0 or 1, we call them indicator variables.
They indicate whether an event has occurred or not.
In this case, we look at each coin flip one at a time.
For the i-th flip, if it resulted in heads or a success, we record it 1.
If not, we record it 0.
And then we look at the random variable.
If we take the sum of the Xi's, what is it going to be?
We add one each time that we get a success, so the sum is going to be the total number of successes.
So we break up the random variable of interest as a sum of really nice and simple random variables.
And now we can use the linearity of expectations.
We're going to find the expectation of X by finding the expectation of the Xi's and then adding the expectations.
What's the expected value of Xi?
Well, Xi takes the value 1 with probability p, and takes the value 0 with probability 1-p.
So the expected value of Xi is just p. So the expected value of X is going to be just n times p. Because X is the sum of n terms, each one of which has expectation p, the expected value of the sum is the sum of the expected values.
So I guess that's a pretty good shortcut for doing this horrendous calculation up there.
So in case you didn't realize it, that's what we just established without doing any algebra.
Good.
How about the variance of X, of Xi?
Two ways to calculate it.
One is by using directly the formula for the variance, which would be -- let's see what it would be.
With probability p, you get a 1.
And in this case, you are so far from the mean.
That's your squared distance from the mean.
With probability 1-p, you get a 0, which is so far away from the mean.
And then you can simplify that formula and get an answer.
How about a slightly easier way of doing it.
Instead of doing the algebra here, let me indicate the slightly easier way.
We have a formula for the variance that tells us that we can find the variance by proceeding this way.
That's a formula that's generally true for variances.
Why is this easier?
What's the expected value of Xi squared?
Backtrack.
What is Xi squared, after all?
It's the same thing as Xi.
Since Xi takes value 0 and 1, Xi squared also takes the same values, 0 and 1.
So the expected value of Xi squared is the same as the expected value of Xi, which is equal to p. And the expected value of Xi squared is p squared, so we get the final answer, p times (1-p).
If you were to work through and do the cancellations in this messy expression here, after one line you would also get to the same formula.
But this sort of illustrates that working with this formula for the variance, sometimes things work out a little faster.
Finally, are we in business?
Can we calculate the variance of the random variable X as well?
Well, we have the rule that for independent random variables, the variance of the sum is the sum of the variances.
So to find the variance of X, we just need to add the variances of the Xi's.
We have n Xi's, and each one of them has variance p_n times (1-p).
And we are done.
So this way, we have calculated both the mean and the variance of the binomial random variable.
It's interesting to look at this particular formula and see what it tells us.
If you are to plot the variance of X as a function of p, it has this shape.
And the maximum is here at 1/2.
p times (1-p) is 0 when p is equal to 0.
And when p equals to 1, it's a quadratic, so it must have this particular shape.
So what does it tell us?
If you think about variance as a measure of uncertainty, it tells you that coin flips are most uncertain when your coin is fair.
When p is equal to 1/2, that's when you have the most randomness.
And this is kind of intuitive.
if on the other hand I tell you that the coin is extremely biased, p very close to 1, which means it almost always gives you heads, then that would be a case of low variance.
There's low variability in the results.
There's little uncertainty about what's going to happen.
It's going to be mostly heads with some occasional tails.
So p equals 1/2.
Fair coin, that's the coin which is the most uncertain of all coins, in some sense.
And it corresponds to the biggest variance.
It corresponds to an X that has the widest distribution.
Now that we're on a roll and we can calculate such hugely complicated sums in simple ways, let us try to push our luck and do a problem with this flavor, but a little harder than that.
So you go to one of those old-fashioned cocktail parties.
All males at least will have those standard big hats which look identical.
They check them in when they walk in.
And when they walk out, since they look pretty identical, they just pick a random hat and go home.
So n people, they pick their hats completely at random, quote, unquote, and then leave.
And the question is, to say something about the number of people who end up, by accident or by luck, to get back their own hat, the exact same hat that they checked in.
OK, first what do we mean completely at random?
Completely at random, we basically mean that any permutation of the hats is equally likely.
Any way of distributing those n hats to the n people, any particular way is as likely as any other way.
So there's complete symmetry between hats and people.
So what we want to do is to calculate the expected value and the variance of this random variable X.
Let's start with the expected value.
Let's reuse the trick from the binomial case.
So total number of hats picked, we're going to think of total number of hats picked as a sum of (0, 1) random variables.
X1 tells us whether person 1 got their own hat back.
If they did, we record a 1.
X2, the same thing.
By adding all X's is how many 1's did we get, which counts how many people selected their own hats.
So we broke down the random variable of interest, the number of people who get their own hats back, as a sum of random variables.
And these random variables, again, are easy to handle, because they're binary.
The only take two values.
What's the probability that Xi is equal to 1, the i-th person has a probability that they get their own hat?
There's n hats by symmetry.
The chance is that they end up getting their own hat, as opposed to any one of the other n - 1 hats, is going to be 1/n.
So what's the expected value of Xi?
It's one times 1/n.
With probability 1/n, you get your own hat, or you get a value of 0 with probability 1-1/n, which is 1/n.
All right, so we got the expected value of the Xi's.
And remember, we want to do is to calculate the expected value of X by using this decomposition?
Are the random variables Xi independent of each other?
You can try to answer that question by writing down a joint PMF for the X's, but I'm sure that you will not succeed.
But can you think intuitively?
If I tell you information about some of the Xi's, does it give you information about the remaining ones?
Yeah.
If I tell you that out of 10 people, 9 of them got their own hat back, does that tell you something about the 10th person?
Yes.
If 9 got their own hat, then the 10th must also have gotten their own hat back.
So the first 9 random variables tell you something about the 10th one.
And conveying information of this sort, that's the case of dependence.
All right, so the random variables are not independent.
Are we stuck?
Can we still calculate the expected value of X?
Yes, we can.
And the reason we can is that expectations are linear.
Expectation of a sum of random variables is the sum of the expectations.
And that's always true.
There's no independence assumption that's being used to apply that rule.
So we have that the expected value of X is the sum of the expected value of the Xi's.
And this is a property that's always true.
You don't need independence.
You don't care.
So we're adding n terms, each one of which has expected value 1/n.
And the final answer is 1.
So out of the 100 people who selected hats at random, on the average, you expect only one of them to end up getting their own hat back.
Very good.
So since we are succeeding so far, let's try to see if we can succeed in calculating the variance as well.
And of course, we will.
But it's going to be a little more complicated.
The reason it's going to be a little more complicated is because the Xi's are not independent, so the variance of the sum is not the same as the sum of the variances.
So it's not enough to find the variances of the Xi's.
We'll have to do more work.
And here's what's involved.
Let's start with the general formula for the variance, which, as I mentioned before, it's usually the simpler way to go about calculating variances.
So we need to calculate the expected value for X-squared, and subtract from it the expectation squared.
Well, we already found the expected value of X.
It's equal to 1.
So 1-squared gives us just 1.
So we're left with the task of calculating the expected value of X-squared, the random variable X-squared.
Let's try to follow the same idea.
Write this messy random variable, X-squared, as a sum of hopefully simpler random variables.
So X is the sum of the Xi's, so you square both sides of this.
And then you expand the right-hand side.
When you expand the right-hand side, you get the squares of the terms that appear here.
And then you get all the cross-terms.
For every pair of (i,j) that are different, i different than j, you're going to have a cross-term in the sum.
So now, in order to calculate the expected value of X-squared, what does our task reduce to?
It reduces to calculating the expected value of this term and calculating the expected value of that term.
So let's do them one at a time.
Expected value of Xi squared, what is it going to be?
Same trick as before.
Xi takes value 0 or 1, so Xi squared takes just the same values, 0 or 1.
So that's the easy one.
That's the same as expected value of Xi, which we already know to be 1/n.
So this gives us a first contribution down here.
The expected value of this term is going to be what?
We have n terms in the summation.
And each one of these terms has an expectation of 1/n.
So we did a piece of the puzzle.
So now let's deal with the second piece of the puzzle.
Let's find the expected value of Xi times Xj.
Now by symmetry, the expected value of Xi times Xj is going to be the same no matter what i and j you see.
So let's just think about X1 and X2 and try to find the expected value of X1 and X2.
X1 times X2 is a random variable.
What values does it take?
Only 0 or 1?
Since X1 and X2 are 0 or 1, their product can only take the values of 0 or 1.
So to find the probability distribution of this random variable, it's just sufficient to find the probability that it takes the value of 1.
Now, what does X1 times X2 equal to 1 mean?
It means that X1 was 1 and X2 was 1.
The only way that you can get a product of 1 is if both of them turned out to be 1's.
So that's the same as saying, persons 1 and 2 both picked their own hats.
The probability that person 1 and person 2 both pick their own hats is the probability of two things happening, which is the product of the first thing happening times the conditional probability of the second, given that the first happened.
And in words, this is the probability that the first person picked their own hat times the probability that the second person picks their own hat, given that the first person already picked their own.
So what's the probability that the first person picks their own hat?
We know that it's 1/n.
Now, how about the second person?
If I tell you that one person has their own hat, and that person takes their hat and goes away, from the point of view of the second person, there's n - 1 people left looking at n - 1 hats.
And they're getting just hats at random.
What's the chance that I will get my own?
It's 1/n - 1.
So think of them as person 1 goes, picks a hat at random, it happens to be their own, and it leaves.
You're left with n - 1 people, and there are n - 1 hats out there.
Person 2 goes and picks a hat at random, with probability 1/n - 1, is going to pick his own hat.
So the expected value now of this random variable is, again, that same number, because this is a 0, 1 random variable.
So this is the same as expected value of Xi times Xj when i different than j.
So here, all that's left to do is to add the expectations of these terms.
Each one of these terms has an expected value that's 1/n times (1/n - 1).
And how many terms do we have?
How many of these are we adding up?
It's n-squared - n. When you expand the quadratic, there's a total of n-squared terms.
Some are self-terms, n of them.
And the remaining number of terms is n-squared - n. So here we got n-squared - n terms.
And so we need to multiply here with n-squared - n. And after you realize that this number here is 1, and you realize that this is the same as the denominator, you get the answer that the expected value of X squared equals 2.
And then, finally going up to the top formula, we get the expected value of X squared, which is 2 - 1, and the variance is just equal to 1.
So the variance of this random variable, number of people who get their own hats back, is also equal to 1, equal to the mean.
Looks like magic.
Why is this the case?
Well, there's a deeper explanation why these two numbers should come out to be the same.
But this is something that would probably have to wait a couple of chapters before we could actually explain it.
And so I'll stop here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. We can start.
Good morning.
So we're going to start now a new unit.
For the next couple of lectures, we will be talking about continuous random variables.
So this is new material which is not going to be in the quiz.
You are going to have a long break next week without any lecture, just a quiz and recitation and tutorial.
So what's going to happen in this new unit?
Basically, we want to do everything that we did for discrete random variables, reintroduce the same sort of concepts but see how they apply and how they need to be modified in order to talk about random variables that take continuous values.
At some level, it's all the same.
At some level, it's quite a bit harder because when things are continuous, calculus comes in.
So the calculations that you have to do on the side sometimes need a little bit more thinking.
In terms of new concepts, there's not going to be a whole lot today, some analogs of things we have done.
We're going to introduce the concept of cumulative distribution functions, which allows us to deal with discrete and continuous random variables, all of them in one shot.
And finally, introduce a famous kind of continuous random variable, the normal random variable.
OK, so what's the story?
Continuous random variables are random variables that take values over the continuum.
So the numerical value of the random variable can be any real number.
They don't take values just in a discrete set.
So we have our sample space.
The experiment happens.
We get some omega, a sample point in the sample space.
And once that point is determined, it determines the numerical value of the random variable.
Remember, random variables are functions on the sample space.
You pick a sample point.
This determines the numerical value of the random variable.
So that numerical value is going to be some real number on that line.
Now we want to say something about the distribution of the random variable.
We want to say which values are more likely than others to occur in a certain sense.
For example, you may be interested in a particular event, the event that the random variable takes values in the interval from a to b.
And we want to say something about the probability of that event.
In principle, how is this done?
You go back to the sample space, and you find all those outcomes for which the value of the random variable happens to be in that interval.
The probability that the random variable falls here is the same as the probability of all outcomes that make the random variable to fall in there.
So in principle, you can work on the original sample space, find the probability of this event, and you would be done.
But similar to what happened in chapter 2, we want to kind of push the sample space in the background and just work directly on the real axis and talk about probabilities up here.
So we want now a way to specify probabilities, how they are bunched together, or arranged, along the real line.
So what did we do for discrete random variables?
We introduced PMFs, probability mass functions.
And the way that we described the random variable was by saying this point has so much mass on top of it, that point has so much mass on top of it, and so on.
And so we assigned a total amount of 1 unit of probability.
We assigned it to different masses, which we put at different points on the real axis.
So that's what you do if somebody gives you a pound of discrete stuff, a pound of mass in little chunks.
And you place those chunks at a few points.
Now, in the continuous case, this total unit of probability mass does not sit just on discrete points but is spread all over the real axis.
So now we're going to have a unit of mass that spreads on top of the real axis.
How do we describe masses that are continuously spread?
The way we describe them is by specifying densities.
That is, how thick is the mass that's sitting here?
How dense is the mass that's sitting there?
So that's exactly what we're going to do.
We're going to introduce the concept of a probability density function that tells us how probabilities accumulate at different parts of the real axis.
So here's an example or a picture of a possible probability density function.
What does that density function kind of convey intuitively?
Well, that these x's are relatively less likely to occur.
Those x's are somewhat more likely to occur because the density is higher.
Now, for a more formal definition, we're going to say that a random variable X is said to be continuous if it can be described by a density function in the following sense.
We have a density function.
And we calculate probabilities of falling inside an interval by finding the area under the curve that sits on top of that interval.
So that's sort of the defining relation for continuous random variables.
It's an implicit definition.
And it tells us a random variable is continuous if we can calculate probabilities this way.
So the probability of falling in this interval is the area under this curve.
Mathematically, it's the integral of the density over this particular interval.
If the density happens to be constant over that interval, the area under the curve would be the length of the interval times the height of the density, which sort of makes sense.
Now, because the density is not constant but it kind of moves around, what you need is to write down an integral.
Now, this formula is very much analogous to what you would do for discrete random variables.
For a discrete random variable, how do you calculate this probability?
You look at all x's in this interval.
And you add the probability mass function over that range.
So just for comparison, this would be the formula for the discrete case-- the sum over all x's in the interval from a to b over the probability mass function.
And there is a syntactic analogy that's happening here and which will be a persistent theme when we deal with continuous random variables.
Sums get replaced by integrals.
In the discrete case, you add.
In the continuous case, you integrate.
Mass functions get replaced by density functions.
So you can take pretty much any formula from the discrete case and translate it to a continuous analog of that formula, as we're going to see.
OK.
So let's take this now as our model.
What is the probability that the random variable takes a specific value if we have a continuous random variable?
Well, this would be the case.
It's a case of a trivial interval, where the two end points coincide.
So it would be the integral from a to itself.
So you're integrating just over a single point.
Now, when you integrate over a single point, the integral is just 0.
The area under the curve, if you're only looking at a single point, it's 0.
So big property of continuous random variables is that any individual point has 0 probability.
In particular, when you look at the value of the density, the density does not tell you the probability of that point.
The point itself has 0 probability.
So the density tells you something a little different.
We are going to see shortly what that is.
Before we get there, can the density be an arbitrary function?
Almost, but not quite.
There are two things that we want.
First, since densities are used to calculate probabilities, and since probabilities must be non-negative, the density should also be non-negative.
Otherwise you would be getting negative probabilities, which is not a good thing.
So that's a basic property that any density function should obey.
The second property that we need is that the overall probability of the entire real line should be equal to 1.
So if you ask me, what is the probability that x falls between minus infinity and plus infinity, well, we are sure that x is going to fall in that range.
So the probability of that event should be 1.
So the probability of being between minus infinity and plus infinity should be 1, which means that the integral from minus infinity to plus infinity should be 1.
So that just tells us that there's 1 unit of total probability that's being spread over our space.
Now, what's the best way to think intuitively about what the density function does?
The interpretation that I find most natural and easy to convey the meaning of a density is to look at probabilities of small intervals.
So let us take an x somewhere here and then x plus delta just next to it.
So delta is a small number.
And let's look at the probability of the event that we get a value in that range.
For continuous random variables, the way we find the probability of falling in that range is by integrating the density over that range.
So we're drawing this picture.
And we want to take the area under this curve.
Now, what happens if delta is a fairly small number?
If delta is pretty small, our density is not going to change much over that range.
So you can pretend that the density is approximately constant.
And so to find the area under the curve, you just take the base times the height.
And it doesn't matter where exactly you take the height in that interval, because the density doesn't change very much over that interval.
And so the integral becomes just base times the height.
So for small intervals, the probability of a small interval is approximately the density times delta.
So densities essentially give us probabilities of small intervals.
And if you want to think about it a little differently, you can take that delta from here and send it to the denominator there.
And what this tells you is that the density is probability per unit length for intervals of small length.
So the units of density are probability per unit length.
Densities are not probabilities.
They are rates at which probabilities accumulate, probabilities per unit length.
And since densities are not probabilities, they don't have to be less than 1.
Ordinary probabilities always must be less than 1.
But density is a different kind of thing.
It can get pretty big in some places.
It can even sort of blow up in some places.
As long as the total area under the curve is 1, other than that, the curve can do anything that it wants.
Now, the density prescribes for us the probability of intervals.
Sometimes we may want to find the probability of more general sets.
How would we do that?
Well, for nice sets, you will just integrate the density over that nice set.
I'm not quite defining what "nice" means.
That's a pretty technical topic in the theory of probability.
But for our purposes, usually we will take b to be something like a union of intervals.
So how do you find the probability of falling in the union of two intervals?
Well, you find the probability of falling in that interval plus the probability of falling in that interval.
So it's the integral over this interval plus the integral over that interval.
And you think of this as just integrating over the union of the two intervals.
So once you can calculate probabilities of intervals, then usually you are in business, and you can calculate anything else you might want.
So the probability density function is a complete description of any statistical information we might be interested in for a continuous random variable.
OK.
So now we can start walking through the concepts and the definitions that we have for discrete random variables and translate them to the continuous case.
The first big concept is the concept of the expectation.
One can start with a mathematical definition.
And here we put down a definition by just translating notation.
Wherever we have a sum in the discrete case, we now write an integral.
And wherever we had the probability mass function, we now throw in the probability density function.
This formula-- you may have seen it in freshman physics-- basically, it again gives you the center of gravity of the picture that you have when you have the density.
It's the center of gravity of the object sitting underneath the probability density function.
So that the interpretation still applies.
It's also true that our conceptual interpretation of what an expectation means is also valid in this case.
That is, if you repeat an experiment a zillion times, each time drawing an independent sample of your random variable x, in the long run, the average that you are going to get should be the expectation.
One can reason in a hand-waving way, sort of intuitively, the way we did it for the case of discrete random variables.
But this is also a theorem of some sort.
It's a limit theorem that we're going to visit later on in this class.
Having defined the expectation and having claimed that the interpretation of the expectation is that same as before, then we can start taking just any formula you've seen before and just translate it.
So for example, to find the expected value of a function of a continuous random variable, you do not have to find the PDF or PMF of g(X).
You can just work directly with the original distribution of the random variable capital X.
And this formula is the same as for the discrete case.
Sums get replaced by integrals.
And PMFs get replaced by PDFs.
And in particular, the variance of a random variable is defined again the same way.
The variance is the expected value, the average of the distance of X from the mean and then squared.
So it's the expected value for a random variable that takes these numerical values.
And same formula as before, integral and F instead of summation, and the P. And the formulas that we have derived or formulas that you have seen for the discrete case, they all go through the continuous case.
So for example, the useful relation for variances, which is this one, remains true.
All right.
So time for an example.
The most simple example of a continuous random variable that there is, is the so-called uniform random variable.
So the uniform random variable is described by a density which is 0 except over an interval.
And over that interval, it is constant.
What is it meant to convey?
It's trying to convey the idea that all x's in this range are equally likely.
Well, that doesn't say very much.
Any individual x has 0 probability.
So it's conveying a little more than that.
What it is saying is that if I take an interval of a given length delta, and I take another interval of the same length, delta, under the uniform distribution, these two intervals are going to have the same probability.
So being uniform means that intervals of same length have the same probability.
So no interval is more likely than any other to occur.
And in that sense, it conveys the idea of sort of complete randomness.
Any little interval in our range is equally likely as any other little interval.
All right.
So what's the formula for this density?
I only told you the range.
What's the height?
Well, the area under the density must be equal to 1.
Total probability is equal to 1.
And so the height, inescapably, is going to be 1 over (b minus a).
That's the height that makes the density integrate to 1.
So that's the formula.
And if you don't want to lose one point in your exam, you have to say that it's also 0, otherwise.
OK. All right?
That's sort of the complete answer.
How about the expected value of this random variable?
OK. You can find the expected value in two different ways.
One is to start with the definition.
And so you integrate over the range of interest times the density.
And you figure out what that integral is going to be.
Or you can be a little more clever.
Since the center-of-gravity interpretation is still true, it must be the center of gravity of this picture.
And the center of gravity is, of course, the midpoint.
Whenever you have symmetry, the mean is always the midpoint of the diagram that gives you the PDF.
OK.
So that's the expected value of X.
Finally, regarding the variance, well, there you will have to do a little bit of calculus.
We can write down the definition.
So it's an integral instead of a sum.
A typical value of the random variable minus the expected value, squared, times the density.
And we integrate.
You do this integral, and you find it's (b minus a) squared over that number, which happens to be 12.
Maybe more interesting is the standard deviation itself.
And you see that the standard deviation is proportional to the width of that interval.
This agrees with our intuition, that the standard deviation is meant to capture a sense of how spread out our distribution is.
And the standard deviation has the same units as the random variable itself.
So it's sort of good to-- you can interpret it in a reasonable way based on that picture.
OK, yes.
Now, let's go up one level and think about the following.
So we have formulas for the discrete case, formulas for the continuous case.
So you can write them side by side.
One has sums, the other has integrals.
Suppose you want to make an argument and say that something is true for every random variable.
You would essentially need to do two separate proofs, for discrete and for continuous.
Is there some way of dealing with random variables just one at a time, in one shot, using a sort of uniform notation?
Is there a unifying concept?
Luckily, there is one.
It's the notion of the cumulative distribution function of a random variable.
And it's a concept that applies equally well to discrete and continuous random variables.
So it's an object that we can use to describe distributions in both cases, using just one piece of notation.
So what's the definition?
It's the probability that the random variable takes values less than a certain number little x.
So you go to the diagram, and you see what's the probability that I'm falling to the left of this.
And you specify those probabilities for all x's.
In the continuous case, you calculate those probabilities using the integral formula.
So you integrate from here up to x.
In the discrete case, to find the probability to the left of some point, you go here, and you add probabilities again from the left.
So the way that the cumulative distribution function is calculated is a little different in the continuous and discrete case.
In one case you integrate.
In the other, you sum.
But leaving aside how it's being calculated, what the concept is, it's the same concept in both cases.
So let's see what the shape of the cumulative distribution function would be in the two cases.
So here what we want is to record for every little x the probability of falling to the left of x.
So let's start here.
Probability of falling to the left of here is 0-- 0, 0, 0.
Once we get here and we start moving to the right, the probability of falling to the left of here is the area of this little rectangle.
And the area of that little rectangle increases linearly as I keep moving.
So accordingly, the CDF increases linearly until I get to that point.
At that point, what's the value of my CDF?
1.
I have accumulated all the probability there is.
I have integrated it.
This total area has to be equal to 1.
So it reaches 1, and then there's no more probability to be accumulated.
It just stays at 1.
So the value here is equal to 1.
OK. How would you find the density if somebody gave you the CDF?
The CDF is the integral of the density.
Therefore, the density is the derivative of the CDF.
So you look at this picture and take the derivative.
Derivative is 0 here, 0 here.
And it's a constant up there, which corresponds to that constant.
So more generally, and an important thing to know, is that the derivative of the CDF is equal to the density-- almost, with a little bit of an exception.
What's the exception?
At those places where the CDF does not have a derivative-- here where it has a corner-- the derivative is undefined.
And in some sense, the density is also ambiguous at that point.
Is my density at the endpoint, is it 0 or is it 1?
It doesn't really matter.
If you change the density at just a single point, it's not going to affect the value of any integral you ever calculate.
So the value of the density at the endpoint, you can leave it as being ambiguous, or you can specify it.
It doesn't matter.
So at all places where the CDF has a derivative, this will be true.
At those places where you have corners, which do show up sometimes, well, you don't really care.
How about the discrete case?
In the discrete case, the CDF has a more peculiar shape.
So let's do the calculation.
We want to find the probability of b to the left of here.
That probability is 0, 0, 0.
Once we cross that point, the probability of being to the left of here is 1/6.
So as soon as we cross the point 1, we get the probability of 1/6, which means that the size of the jump that we have here is 1/6.
Now, question.
At this point 1, which is the correct value of the CDF?
Is it 0, or is it 1/6?
It's 1/6 because-- you need to look carefully at the definitions, the probability of x being less than or equal to little x.
If I take little x to be 1, it's the probability that capital X is less than or equal to 1.
So it includes the event that x is equal to 1.
So it includes this probability here.
So at jump points, the correct value of the CDF is going to be this one.
And now as I trace, x is going to the right.
As soon as I cross this point, I have added another 3/6 probability.
So that 3/6 causes a jump to the CDF.
And that determines the new value.
And finally, once I cross the last point, I get another jump of 2/6.
A general moral from these two examples and these pictures.
CDFs are well defined in both cases.
For the case of continuous random variables, the CDF will be a continuous function.
It starts from 0.
It eventually goes to 1 and goes smoothly-- well, continuously from smaller to higher values.
It can only go up.
It cannot go down since we're accumulating more and more probability as we are going to the right.
In the discrete case, again it starts from 0, and it goes to 1.
But it does it in a staircase manner.
And you get a jump at each place where the PMF assigns a positive mass.
So jumps in the CDF are associated with point masses in our distribution.
In the continuous case, we don't have any point masses, so we do not have any jumps either.
Now, besides saving us notation-- we don't have to deal with discrete and continuous twice-- CDFs give us actually a little more flexibility.
Not all random variables are continuous or discrete.
You can cook up random variables that are kind of neither or a mixture of the two.
An example would be, let's say you play a game.
And with a certain probability, you get a certain number of dollars in your hands.
So you flip a coin.
And with probability 1/2, you get a reward of 1/2 dollars.
And with probability 1/2, you are led to a dark room where you spin a wheel of fortune.
And that wheel of fortune gives you a random reward between 0 and 1.
So any of these outcomes is possible.
And the amount that you're going to get, let's say, is uniform.
So you flip a coin.
And depending on the outcome of the coin, either you get a certain value or you get a value that ranges over a continuous interval.
So what kind of random variable is it?
Is it continuous?
Well, continuous random variables assign 0 probability to individual points.
Is it the case here?
No, because you have positive probability of obtaining 1/2 dollar.
So our random variable is not continuous.
Is it discrete?
It's not discrete, because our random variable can take values also over a continuous range.
So we call such a random variable a mixed random variable.
If you were to draw its distribution very loosely, probably you would want to draw a picture like this one, which kind of conveys the idea of what's going on.
So just think of this as a drawing of masses that are sitting over a table.
We place an object that weighs half a pound, but it's an object that takes zero space.
So half a pound is just sitting on top of that point.
And we take another half-pound of probability and spread it uniformly over that interval.
So this is like a piece that comes from mass functions.
And that's a piece that looks more like a density function.
And we just throw them together in the picture.
I'm not trying to associate any formal meaning with this picture.
It's just a schematic of how probabilities are distributed, help us visualize what's going on.
Now, if you have taken classes on systems and all of that, you may have seen the concept of an impulse function.
And you my start saying that, oh, I should treat this mathematically as a so-called impulse function.
But we do not need this for our purposes in this class.
Just think of this as a nice picture that conveys what's going on in this particular case.
So now, what would the CDF look like in this case?
The CDF is always well defined, no matter what kind of random variable you have.
So the fact that it's not continuous, it's not discrete shouldn't be a problem as long as we can calculate probabilities of this kind.
So the probability of falling to the left here is 0.
Once I start crossing there, the probability of falling to the left of a point increases linearly with how far I have gone.
So we get this linear increase.
But as soon as I cross that point, I accumulate another 1/2 unit of probability instantly.
And once I accumulate that 1/2 unit, it means that my CDF is going to have a jump of 1/2.
And then afterwards, I still keep accumulating probability at a fixed rate, the rate being the density.
And I keep accumulating, again, at a linear rate until I settle to 1.
So this is a CDF that has certain pieces where it increases continuously.
And that corresponds to the continuous part of our randomize variable.
And it also has some places where it has discrete jumps.
And those district jumps correspond to places in which we have placed a positive mass.
And by the-- OK, yeah.
So this little 0 shouldn't be there.
So let's cross it out.
All right.
So finally, we're going to take the remaining time and introduce our new friend.
It's going to be the Gaussian or normal distribution.
So it's the most important distribution there is in all of probability theory.
It's plays a very central role.
It shows up all over the place.
We'll see later in the class in more detail why it shows up.
But the quick preview is the following.
If you have a phenomenon in which you measure a certain quantity, but that quantity is made up of lots and lots of random contributions-- so your random variable is actually the sum of lots and lots of independent little random variables-- then invariability, no matter what kind of distribution the little random variables have, their sum will turn out to have approximately a normal distribution.
So this makes the normal distribution to arise very naturally in lots and lots of contexts.
Whenever you have noise that's comprised of lots of different independent pieces of noise, then the end result will be a random variable that's normal.
So we are going to come back to that topic later.
But that's the preview comment, basically to argue that it's an important one.
OK. And there's a special case.
If you are dealing with a binomial distribution, which is the sum of lots of Bernoulli random variables, again you would expect that the binomial would start looking like a normal if you have many, many-- a large number of point fields.
All right.
So what's the math involved here?
Let's parse the formula for the density of the normal.
What we start with is the function X squared over 2.
And if you are to plot X squared over 2, it's a parabola, and it has this shape -- X squared over 2.
Then what do we do?
We take the negative exponential of this.
So when X squared over 2 is 0, then negative exponential is 1.
When X squared over 2 increases, the negative exponential of that falls off, and it falls off pretty fast.
So as this goes up, the formula for the density goes down.
And because exponentials are pretty strong in how quickly they fall off, this means that the tails of this distribution actually do go down pretty fast.
OK.
So that explains the shape of the normal PDF.
How about this factor 1 over square root 2 pi?
Where does this come from?
Well, the integral has to be equal to 1.
So you have to go and do your calculus exercise and find the integral of this the minus X squared over 2 function and then figure out, what constant do I need to put in front so that the integral is equal to 1?
How do you evaluate that integral?
Either you go to Mathematica or Wolfram's Alpha or whatever, and it tells you what it is.
Or it's a very beautiful calculus exercise that you may have seen at some point.
You throw in another exponential of this kind, you bring in polar coordinates, and somehow the answer comes beautifully out there.
But in any case, this is the constant that you need to make it integrate to 1 and to be a legitimate density.
We call this the standard normal.
And for the standard normal, what is the expected value?
Well, the symmetry, so it's equal to 0.
What is the variance?
Well, here there's no shortcut.
You have to do another calculus exercise.
And you find that the variance is equal to 1.
OK.
So this is a normal that's centered around 0.
How about other types of normals that are centered at different places?
So we can do the same kind of thing.
Instead of centering it at 0, we can take some place where we want to center it, write down a quadratic such as (X minus mu) squared, and then take the negative exponential of that.
And that gives us a normal density that's centered at mu.
Now, I may wish to control the width of my density.
To control the width of my density, equivalently I can control the width of my parabola.
If my parabola is narrower, if my parabola looks like this, what's going to happen to the density?
It's going to fall off much faster.
OK. How do I make my parabola narrower or wider?
I do it by putting in a constant down here.
So by putting a sigma here, this stretches or widens my parabola by a factor of sigma.
Let's see.
Which way does it go?
If sigma is very small, this is a big number.
My parabola goes up quickly, which means my normal falls off very fast.
So small sigma corresponds to a narrower density.
And so it, therefore, should be intuitive that the standard deviation is proportional to sigma.
Because that's the amount by which you are scaling the picture.
And indeed, the standard deviation is sigma.
And so the variance is sigma squared.
So all that we have done here to create a general normal with a given mean and variance is to take this picture, shift it in space so that the mean sits at mu instead of 0, and then scale it by a factor of sigma.
This gives us a normal with a given mean and a given variance.
And the formula for it is this one.
All right.
Now, normal random variables have some wonderful properties.
And one of them is that they behave nicely when you take linear functions of them.
So let's fix some constants a and b, suppose that X is normal, and look at this linear function Y.
What is the expected value of Y?
Here we don't need anything special.
We know that the expected value of a linear function is the linear function of the expectation.
So the expected value is this.
How about the variance?
We know that the variance of a linear function doesn't care about the constant term.
But the variance gets multiplied by a squared.
So we get these variance, where sigma squared is the variance of the original normal.
So have we used so far the property that X is normal?
No, we haven't.
This calculation here is true in general when you take a linear function of a random variable.
But if X is normal, we get the other additional fact that Y is also going to be normal.
So that's the nontrivial part of the fact that I'm claiming here.
So linear functions of normal random variables are themselves normal.
How do we convince ourselves about it?
OK.
It's something that we will do formerly in about two or three lectures from today.
So we're going to prove it.
But if you think about it intuitively, normal means this particular bell-shaped curve.
And that bell-shaped curve could be sitting anywhere and could be scaled in any way.
So you start with a bell-shaped curve.
If you take X, which is bell shaped, and you multiply it by a constant, what does that do?
Multiplying by a constant is just like scaling the axis or changing the units with which you're measuring it.
So it will take a bell shape and spread it or narrow it.
But it will still be a bell shape.
And then when you add the constant, you just take that bell and move it elsewhere.
So under linear transformations, bell shapes will remain bell shapes, just sitting at a different place and with a different width.
And that sort of the intuition of why normals remain normals under this kind of transformation.
So why is this useful?
Well, OK. We have a formula for the density.
But usually we want to calculate probabilities.
How will you calculate probabilities?
If I ask you, what's the probability that the normal is less than 3, how do you find it?
You need to integrate the density from minus infinity up to 3.
Unfortunately, the integral of the expression that shows up that you would have to calculate, an integral of this kind from, let's say, minus infinity to some number, is something that's not known in closed form.
So if you're looking for a closed-form formula for this-- X bar-- if you're looking for a closed-form formula that gives you the value of this integral as a function of X bar, you're not going to find it.
So what can we do?
Well, since it's a useful integral, we can just tabulate it.
Calculate it once and for all, for all values of X bar up to some precision, and have that table, and use it.
That's what one does.
OK, but now there is a catch.
Are we going to write down a table for every conceivable type of normal distribution-- that is, for every possible mean and every variance?
I guess that would be a pretty long table.
You don't want to do that.
Fortunately, it's enough to have a table with the numerical values only for the standard normal.
And once you have those, you can use them in a clever way to calculate probabilities for the more general case.
So let's see how this is done.
So our starting point is that someone has graciously calculated for us the values of the CDF, the cumulative distribution function, that is the probability of falling below a certain point for the standard normal and at various places.
How do we read this table?
The probability that X is less than, let's say, 0.63 is this number.
This number, 0.7357, is the probability that the standard normal is below 0.63.
So the table refers to the standard normal.
But someone, let's say, gives us some other numbers and tells us we're dealing with a normal with a certain mean and a certain variance.
And we want to calculate the probability that the value of that random variable is less than or equal to 3.
How are we going to do it?
Well, there's a standard trick, which is so-called standardizing a random variable.
Standardizing a random variable stands for the following.
You look at the random variable, and you subtract the mean.
This makes it a random variable with 0 mean.
And then if I divide by the standard deviation, what happens to the variance of this random variable?
Dividing by a number divides the variance by sigma squared.
The original variance of X was sigma squared.
So when I divide by sigma, I end up with unit variance.
So after I do this transformation, I get a random variable that has 0 mean and unit variance.
It is also normal.
Why is its normal?
Because this expression is a linear function of the X that I started with.
It's a linear function of a normal random variable.
Therefore, it is normal.
And it is a standard normal.
So by taking a general normal random variable and doing this standardization, you end up with a standard normal to which you can then apply the table.
Sometimes one calls this the normalized score.
If you're thinking about test results, how would you interpret this number?
It tells you how many standard deviations are you away from the mean.
This is how much you are away from the mean.
And you count it in terms of how many standard deviations it is.
So this number being equal to 3 tells you that X happens to be 3 standard deviations above the mean.
And I guess if you're looking at your quiz scores, very often that's the kind of number that you think about.
So it's a useful quantity.
But it's also useful for doing the calculation we're now going to do.
So suppose that X has a mean of 2 and a variance of 16, so a standard deviation of 4.
And we're going to calculate the probability of this event.
This event is described in terms of this X that has ugly means and variances.
But we can take this event and rewrite it as an equivalent event.
X less than 3 is this same as X minus 2 being less than 3 minus 2, which is the same as this ratio being less than that ratio.
So I'm subtracting from both sides of the inequality the mean and then dividing by the standard deviation.
This event is the same as that event.
Why do we like this better than that?
We like it because this is the standardized, or normalized, version of X.
We know that this is standard normal.
And so we're asking the question, what's the probability that the standard normal is less than this number, which is 1/4?
So that's the key property, that this is normal (0, 1).
And so we can look up now with the table and ask for the probability that the standard normal random variable is less than 0.25.
Where is that going to be?
0.2, 0.25, it's here.
So the answer is 0.987.
So I guess this is just a drill that you could learn in high school.
You didn't have to come here to learn about it.
But it's a drill that's very useful when we will be calculating normal probabilities all the time.
So make sure you know how to use the table and how to massage a general normal random variable into a standard normal random variable.
OK.
So just one more minute to look at the big picture and take stock of what we have done so far and where we're going.
Chapter 2 was this part of the picture, where we dealt with discrete random variables.
And this time, today, we started talking about continuous random variables.
And we introduced the density function, which is the analog of the probability mass function.
We have the concepts of expectation and variance and CDF.
And this kind of notation applies to both discrete and continuous cases.
They are calculated the same way in both cases except that in the continuous case, you use sums.
In the discrete case, you use integrals.
So on that side, you have integrals.
In this case, you have sums.
In this case, you always have Fs in your formulas.
In this case, you always have Ps in your formulas.
So what's there that's left for us to do is to look at these two concepts, joint probability mass functions and conditional mass functions, and figure out what would be the equivalent concepts on the continuous side.
So we will need some notion of a joint density when we're dealing with multiple random variables.
And we will also need the concept of conditional density, again for the case of continuous random variables.
The intuition and the meaning of these objects is going to be exactly the same as here, only a little subtler because densities are not probabilities.
They're rates at which probabilities accumulate.
So that adds a little bit of potential confusion here, which, hopefully, we will fully resolve in the next couple of sections.
All right.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK let's start.
So we've had the quiz.
And I guess there's both good and bad news in it.
Yesterday, as you know, the bad news.
The average was a little lower than what we would have wanted.
On the other hand, the good news is that the distribution was nicely spread.
And that's the main purpose of this quiz is basically for you to calibrate and see roughly where you are standing.
The other piece of the good news is that, as you know, this quiz doesn't count for very much in your final grade.
So it's really a matter of calibration and to get your mind set appropriately to prepare for the second quiz, which counts a lot more.
And it's more substantial.
And we'll make sure that the second quiz will have a higher average.
All right.
So let's go to our material.
We're talking now these days about continuous random variables.
And I'll remind you what we discussed last time.
I'll remind you of the concept of the probability density function of a single random variable.
And then we're going to rush through all the concepts that we covered for the case of discrete random variables and discuss their analogs for the continuous case.
And talk about notions such as conditioning independence and so on.
So the big picture is here.
We have all those concepts that we developed for the case of discrete random variables.
And now we will just talk about their analogs in the continuous case.
We already discussed this analog last week, the density of a single random variable.
Then there are certain concepts that show up both in the discrete and the continuous case.
So we have the cumulative distribution function, which is a description of the probability distribution of a random variable and which applies whether you have a discrete or continuous random variable.
Then there's the notion of the expected value.
And in the two cases, the expected value is calculated in a slightly different way, but not very different.
We have sums in one case, integrals in the other.
And this is the general pattern that we're going to have.
Formulas for the discrete case translate to corresponding formulas or expressions in the continuous case.
We generically replace sums by integrals, and we replace must functions with density functions.
Then the new pieces for today are going to be mostly the notion of a joint density function, which is how we describe the probability distribution of two random variables that are somehow related, in general, and then the notion of a conditional density function that tells us the distribution of one random variable X when you're told the value of another random variable Y.
There's another concept, which is the conditional PDF given that the certain event has happened.
This is a concept that's in some ways simpler.
You've already seen a little bit of that in last week's recitation and tutorial.
The idea is that we have a single random variable.
It's described by a density.
Then you're told that the certain event has occurred.
Your model changes the universe that you are dealing with.
In the new universe, you are dealing with a new density function, the one that applies given the knowledge that we have that the certain event has occurred.
All right.
So what exactly did we say about continuous random variables?
The first thing is the definition, that a random variable is said to be continuous if we are given a certain object that we call the probability density function and we can calculate interval probabilities given this density function.
So the definition is that the random variable is continuous if you can calculate probabilities associated with that random variable given that formula.
So this formula tells you that the probability that your random variable falls inside this interval is the area under the density curve.
OK.
There's a few properties that a density function must satisfy.
Since we're talking about probabilities, and probabilities are non-negative, we have that the density function is always a non-negative function.
The total probability over the entire real line must be equal to 1.
So the integral when you integrate over the entire real line has to be equal to 1.
That's the second property.
Another property that you get is that if you let a equal to b, this integral becomes 0.
And that tells you that the probability of a single point in the continuous case is always equal to 0.
So these are formal properties.
When you want to think intuitively, the best way to think about what the density function is to think in terms of little intervals, the probability that my random variable falls inside the little interval.
Well, inside that little interval, the density function here is roughly constant.
So that integral becomes the value of the density times the length of the interval over which you are integrating, which is delta.
And so the density function basically gives us probabilities of little events, of small events.
And the density is to be interpreted as probability per unit length at a certain place in the diagram.
So in that place in the diagram, the probability per unit length around this neighborhood would be the height of the density function at that point.
What else?
We have a formula for calculating expected values of functions of random variables.
In the discrete case, we had the formula where here we had the sum, and instead of the density, we had the PMF.
The same formula is also valid in the continuous case.
And it's not too hard to derive, but we will not do it.
But let's think of the intuition of what this formula says.
You're trying to figure out on the average how much g(X) is going to be.
And then you reason, and you say, well, X may turn out to take a particular value or a small interval of values.
This is the probability that X falls inside the small interval.
And when that happens, g(X) takes that value.
So this fraction of the time, you fall in the little neighborhood of x, and you get so much.
Then you average over all the possible x's that can happen.
And that gives you the average value of the function g(X).
OK.
So this is the easy stuff.
Now let's get to the new material.
We want to talk about multiple random variables simultaneously.
So we want to talk now about two random variables that are continuous, and in some sense that they are jointly continuous.
And let's see what this means.
The definition is similar to the definition we had for a single random variable, where I take this formula here as the definition of continuous random variables.
Two random variables are said to be jointly continuous if we can calculate probabilities by integrating a certain function that we call the joint density function over the set of interest.
So we have our two-dimensional plane.
This is the x-y plane.
There's a certain event S that we're interested in.
We want to calculate the probability.
How do we do that?
We are given this function f_(X,Y), the joint density.
It's a function of the two arguments x and y.
So think of that function as being some kind of surface that sits on top of the two-dimensional plane.
The probability of falling inside the set S, we calculate it by looking at the volume under the surface, that volume that sits on top of S. So the surface underneath it has a certain total volume.
What should that total volume be?
Well, we think of these volumes as probabilities.
So the total probability should be equal to 1.
The total volume under this surface, should be equal to 1.
So that's one property that we want our density function to have.
So when you integrate over the entire space, this is of the volume under your surface.
That should be equal to 1.
Of course, since we're talking about probabilities, the joint density should be a non-negative function.
So think of the situation as having one pound of probability that's spread all over your space.
And the height of this joint density function basically tells you how much probability tends to be accumulated in certain regions of space as opposed to other parts of the space.
So wherever the density is big, that means that this is an area of the two-dimensional plane that's more likely to occur.
Where the density is small, that means that those x-y's are less likely to occur.
You have already seen one example of continuous densities.
That was the example we had in the very beginning of the class with a uniform distribution on the unit square.
That was a special case of a density function that was constant.
So all places in the unit square were roughly equally likely as any other places.
But in other models, some parts of the space may be more likely than others.
And we describe those relative likelihoods using this density function.
So if somebody gives us the density function, this determines for us probabilities of all the subsets of the two-dimensional plane.
Now for an intuitive interpretation, it's good to think about small events.
So let's take a particular x here and then x plus delta.
So this is a small interval.
Take another small interval here that goes from y to y plus delta.
And let's look at the event that x falls here and y falls right there.
What is this event?
Well, this is the event that will fall inside this little rectangle.
Using this rule for calculating probabilities, what is the probability of that rectangle going to be?
Well, it should be the integral of the density over this rectangle.
Or it's the volume under the surface that sits on top of that rectangle.
Now, if the rectangle is very small, the joint density is not going to change very much in that neighborhood.
So we can treat it as a constant.
So the volume is going to be the height times the area of the base.
The height at that point is whatever the function happens to be around that point.
And the area of the base is delta squared.
So this is the intuitive way to understand what a joint density function really tells you.
It specifies for you probabilities of little squares, of little rectangles.
And it allows you to think of the joint density function as probability per unit area.
So these are the units of the density, its probability per unit area in the neighborhood of a certain point.
So what do we do with this density function once we have it in our hands?
Well, we can use it to calculate expected values.
Suppose that you have a function of two random variables described by a joint density.
You can find, perhaps, the distribution of this random variable and then use the basic definition of the expectation.
Or you can calculate expectations directly, using the distribution of the original random variables.
This is a formula that's again identical to the formula that we had for the discrete case.
In the discrete case, we had a double sum here, and we had PMFs.
So the intuition behind this formula is the same that one had for the discrete case.
It's just that the mechanics are different.
Then something that we did in the discrete case was to find a way to go from the joint density of the two random variables taken together to the density of just one of the random variables.
So we had a formula for the discrete case.
Let's see how things are going to work out in the continuous case.
So in the continuous case, we have here our two random variables.
And we have a density for them.
And let's say that we want to calculate the probability that x falls inside this interval.
So we're looking at the probability that our random variable X falls in the interval from little x to x plus delta.
Now, by the properties that we already have for interpreting the density function of a single random variable, the probability of a little interval is approximately the density of that single random variable times delta.
And now we want to find a formula for this marginal density in terms of the joint density.
OK.
So this is the probability that x falls inside this interval.
In terms of the two-dimensional plane, this is the probability that (x,y) falls inside this strip.
So to find that probability, we need to calculate the probability that (x,y) falls in here, which is going to be the double integral over the interval over this strip, of the joint density.
And what are we integrating over?
y goes from minus infinity to plus infinity.
And the dummy variable x goes from little x to x plus delta.
So to integrate over this strip, what we do is for any given y, we integrate in this dimension.
This is the x integral.
And then we integrate over the y dimension.
Now what is this inner integral?
Because x only varies very little, this is approximately constant in that range.
So the integral with respect to x just becomes delta times f(x,y).
And then we've got our dy.
So this is what the inner integral will evaluate to.
We are integrating over the little interval.
So we're keeping y fixed.
Integrating over here, we take the value of the density times how much we're integrating over.
And we get this formula.
OK. Now, this expression must be equal to that expression.
So if we cancel the deltas, we see that the marginal density must be equal to the integral of the joint density, where we have integrated out the value of y.
So this formula should come as no surprise at this point.
It's exactly the same as the formula that we had for discrete random variables.
But now we are replacing the sum with an integral.
And instead of using the joint PMF, we are using the joint PDF.
Then, continuing going down the list of things we did for discrete random variables, we can now introduce a definition of the notion of independence of two random variables.
And by analogy with the discrete case, we define independence to be the following condition.
Two random variables are independent if and only if their joint density function factors out as a product of their marginal densities.
And this property needs to be true for all x and y.
So this is the formal definition.
Operationally and intuitively, what does it mean?
Well, intuitively it means the same thing as in the discrete case.
Knowing anything about X shouldn't tell you anything about Y.
That is, information about X is not going to change your beliefs about Y.
We are going to come back to this statement in a second.
The other thing that it allows you to do-- I'm not going to derive this-- is it allows you to calculate probabilities by multiplying individual probabilities.
So if you ask for the probability that x falls in a certain set A and y falls in a certain set B, then you can calculate that probability by multiplying individual probabilities.
This takes just two lines of derivation, which I'm not going to do.
But it comes back to the usual notion of independence of events.
Basically, operationally independence means that you can multiply probabilities.
So now let's look at an example.
There's a sort of pretty famous and classical one.
It goes back a lot more than a 100 years.
And it's the famous Needle of Buffon.
Buffon was a French naturalist who, for some reason, also decided to play with probability.
And look at the following problem.
So you have the two-dimensional plane.
And on the plane we draw a bunch of parallel lines.
And those parallel lines are separated by a length.
And the lines are apart at distance d. And we throw a needle at random, completely at random.
And we'll have to give a meaning to what "completely at random" means.
And when we throw a needle, there's two possibilities.
Either the needle is going to fall in a way that does not intersect any of the lines, or it's going to fall in a way that it intersects one of the lines.
We're taking the needle to be shorter than this distance, so the needle cannot intersect two lines simultaneously.
It either intersects 0, or it intersects one of the lines.
The question is to find the probability that the needle is going to intersect a line.
What's the probability of this?
OK. We are going to approach this problem by using our standard four-step procedure.
Set up your sample space, describe a probability law on that sample space, identify the event of interest, and then calculate.
These four steps basically correspond to these three bullets and then the last equation down here.
So first thing is to set up a sample space.
We need some variables to describe what happened in the experiment.
So what happens in the experiment is that the needle lands somewhere.
And where it lands, we can describe this by specifying the location of the center of the needle.
And what do we mean by the location of the center?
Well, we can take as our variable to be the distance from the center of the needle to the nearest line.
So it tells us the vertical distance of the center of the needle from the nearest line.
The other thing that matters is the orientation of the needle.
So we need one more variable, which we take to be the angle that the needle is forming with the lines.
We can put the angle here, or you can put in there.
Yes, it's still the same angle.
So we have these two variables that described what happened in the experiment.
And we can take our sample space to be the set of all possible x's and theta's.
What are the possible x's?
The lines are d apart, so the nearest line is going to be anywhere between 0 and d/2 away.
So that tells us what the possible x's will be.
As for theta, it really depends how you define your angle.
We are going to define our theta to be the acute angle that's formed between the needle and a line, if you were to extend it.
So theta is going to be something between 0 and pi/2.
So I guess these red pieces really correspond to the part of setting up the sample space.
OK.
So that's part one.
Second part is we need a model.
OK. Let's take our model to be that we basically know nothing about how the needle falls.
It can fall in any possible way, and all possible ways are equally likely.
Now, if you have those parallel lines, and you close your eyes completely and throw a needle completely at random, any x should be equally likely.
So we describe that situation by saying that X should have a uniform distribution.
That is, it should have a constant density over the range of interest.
Similarly, if you kind of spin your needle completely at random, any angle should be as likely as any other angle.
And we decide to model this situation by saying that theta also has a uniform distribution over the range of interest.
And finally, where we put it should have nothing to do with how much we rotate it.
And we capture this mathematically by saying that X is going to be independent of theta.
Now, this is going to be our model.
I'm not deriving the model from anything.
I'm only saying that this sounds like a model that does not assume any knowledge or preference for certain values of x rather than other values of theta.
In the absence of any other particular information you might have in your hands, that's the most reasonable model to come up with.
So you model the problem that way.
So what's the formula for the joint density?
It's going to be the product of the densities of X and Theta.
Why is it the product?
This is because we assumed independence.
And the density of X, since it's uniform, and since it needs to integrate to 1, that density needs to be 2/d.
That's the density of X.
And the density of Theta needs to be 2/pi.
That's the value for the density of Theta so that the overall probability over this interval ends up being 1.
So now we do have our joint density in our hands.
The next thing to do is to identify the event of interest.
And this is best done in a picture.
And there's two possible situations that one could have.
Either the needle falls this way, or it falls this way.
So how can we tell if one or the other is going to happen?
It has to do with whether this interval here is smaller than that or bigger than that.
So we are comparing the height of this interval to that interval.
This interval here is capital X.
This interval here, what is it?
This is half of the length of the needle, which is l/2.
To find this height, we take l/2 and multiply it with the sine of the angle that we have.
So the length of this interval up here is l/2 times sine theta.
If this is smaller than x, the needle does not intersect the line.
If this is bigger than x, then the needle intersects the line.
So the event of interest, that the needle intersects the line, is described this way in terms of x and theta.
And now that we have the event of interest described mathematically, all that we need to do is to find the probability of this event, we integrate the joint density over the part of (x, theta) space in which this inequality is true.
So it's a double integral over the set of all x's and theta's where this is true.
The way to do this integral is we fix theta, and we integrate for x's that go from 0 up to that number.
And theta can be anything between 0 and pi/2.
So the integral over this set is basically this double integral here.
We already have a formula for the joint density.
It's 4 over pi d, so we put it here.
And now, fortunately, this is a pretty easy integral to evaluate.
The integral with respect to x -- there's nothing in here.
So the integral is just the length of the interval over which we're integrating.
It's l/2 sine theta.
And then we need to integrate this with respect to theta.
We know that the integral of a sine is a negative cosine.
You plug in the values for the negative cosine at the two end points.
I'm sure you can do this integral .
And we finally obtain the answer, which is amazingly simple for such a pretty complicated-looking problem.
It's 2l over pi d. So some people a long, long time ago, after they looked at this answer, they said that maybe that gives us an interesting way where one could estimate the value by pi, for example, experimentally.
How do you do that?
Fix l and d, the dimensions of the problem.
Throw a million needles on your piece of paper.
See how often your needless do intersect the line.
That gives you a number for this quantity.
You know l and d, so you can use that to infer pi.
And there's an apocryphal story about a wounded soldier in a hospital after the American Civil War who actually had heard about this and was spending his time in the hospital throwing needles on pieces of paper.
I don't know if it's true or not.
But let's do something similar here.
So let's look at this diagram.
We fix the dimensions.
This is supposed to be our little d. That's supposed to be our little l. We have the formula from the previous slide that p is 2l over pi d. In this instance, we choose d to be twice l. So this number is 1/pi.
So the probability that the needle hits the line is 1/pi.
So I need needles that are 3.1 centimeters long.
I couldn't find such needles.
But I could find paper clips that are 3.1 centimeters long.
So let's start throwing paper clips at random and see how many of them will end up intersecting the lines.
Good.
OK.
So out of eight paper clips, we have exactly four that intersected the line.
So our estimate for the probability of intersecting the line is 1/2, which gives us an estimate for the value of pi, which is two.
Well, I mean, within an engineering approximation, we're in the right ballpark, right?
So this might look like a silly way of trying to estimate pi.
And it probably is.
On the other hand, this kind of methodology is being used especially by physicists and also by statisticians.
It's used a lot.
When is it used?
If you have an integral to calculate, such as this integral, but you're not lucky, and your functions are not so simple where you can do your calculations by hand, and maybe the dimensions are larger-- instead of two random variables you have 100 random variables, so it's a 100-fold integral-- then there's no way to do that in the computer.
But the way that you can actually do it is by generating random samples of your random variables, doing that simulation over and over many times.
That is, by interpreting an integral as a probability, you can use simulation to estimate that probability.
And that gives you a way of calculating integrals.
And physicists do actually use that a lot, as well as statisticians, computer scientists, and so on.
It's a so-called Monte Carlo method for evaluating integrals.
And it's a basic piece of the toolbox in science these days.
Finally, the harder concept of the day is the idea of conditioning.
And here things become a little subtle when you deal with continuous random variables.
OK. First, remember again our basic interpretation of what a density is.
A density gives us probabilities of little intervals.
So how should we define conditional densities?
Conditional densities should again give us probabilities of little intervals, but inside a conditional world where we have been told something about the other random variable.
So what we would like to be true is the following.
We would like to define a concept of a conditional density of a random variable X given the value of another random variable Y.
And it should behave the following way, that the conditional density gives us the probability of little intervals-- same as here-- given that we are told the value of y.
And here's where the subtleties come.
The main thing to notice is that here I didn't write "equal," I wrote "approximately equal."
Why do we need that?
Well, the thing is that conditional probabilities are not defined when you condition on an event that has 0 probability.
So we need the conditioning event here to have posed this probability.
So instead of saying that Y is exactly equal to little y, we want to instead say we're in a new universe where capital Y is very close to little y.
And then this notion of "very close" kind of takes the limit and takes it to be infinitesimally close.
So this is the way to interpret conditional probabilities.
That's what they should mean.
Now, in practice, when you actually use probability, you forget about that subtlety.
And you say, well, I've been told that Y is equal to 1.3.
Give me the conditional distribution of X.
But formally or rigorously, you should say I'm being told that Y is infinitesimally close to 1.3.
Tell me the distribution of X.
Now, if this is what we want, what should this quantity be?
It's a conditional probability, so it should be the probability of two things happening-- X being close to little x, Y being close to little y.
And that's basically given to us by the joint density divided by the probability of the conditioning event, which has something to do with the density of Y itself.
And if you do things carefully, you see that the only way to satisfy this relation is to define the conditional density by this particular formula.
OK. Big discussion to come down in the end to what you should have probably guessed by now.
We just take any formulas and expressions from the discrete case and replace PMFs by PDFs.
So the conditional PDF is defined by this formula where here we have joint PDF and marginal PDF, as opposed to the discrete case where we had the joint PMF and the marginal PMF.
So in some sense, it's just a syntactic change.
In another sense, it's a little subtler on how you actually interpret it.
Speaking about interpretation, what are some ways of thinking about the joint density?
Well, the best way to think about it is that somebody has fixed little y for you.
So little y is being fixed here.
And we look at this density as a function of X. I've told you what Y is.
Tell me what you know about X.
And you tell me that X has a certain distribution.
What does that distribution look like?
It has exactly the same shape as the joint density.
Remember, we fixed Y.
So this is a constant.
So the only thing that varies is X.
So we get the function that behaves like the joint density when you fix y, which is really you take the joint density, and you take a slice of it.
You fix a y, and you see how it varies with x.
So in that sense, the conditional PDF is just a slice of the joint PDF.
But we need to divide by a certain number, which just scales it and changes its shape.
We're coming back to a picture in a second.
But before going to the picture, lets go back to the interpretation of independence.
If the two random the variables are independent, according to our definition in the previous slide, the joint density is going to factor as the product of the marginal densities.
The density of Y in the numerator cancels the density in the denominator.
And we're just left with the density of X.
So in the case of independence, what we get is that the conditional is the same as the marginal.
And that solidifies our intuition that in the case of independence, being told something about the value of Y does not change our beliefs about how X is distributed.
So whatever we expected about X is going to remain true even after we are told something about Y.
So let's look at some pictures.
Here is what the joint PDF might look like.
Here we've got our x and y-axis.
And if you want to calculate the probability of a certain event, what you do is you look at that event and you see how much of that mass is sitting on top of that event.
Now let's start slicing.
Let's fix a value of x and look along that slice where we obtain this function.
Now what does that slice do?
That slice tells us for that particular x what the possible values of y are going to be and how likely they are.
If we integrate over all y's, what do we get?
Integrating over all y's just gives us the marginal density of X.
It's the calculation that we did here.
By integrating over all y's, we find the marginal density of X.
So the total area under that slice gives us the marginal density of X.
And by looking at the different slices, we find how likely the different values of x are going to be.
How about the conditional?
If we're interested in the conditional of Y given X, how would you think about it?
This refers to a universe where we are told that capital X takes on a specific value.
So we put ourselves in the universe where this line has happened.
There's still possible values of y that can happen.
And this shape kind of tells us the relative likelihoods of the different y's.
And this is indeed going to be the shape of the conditional distribution of Y given that X has occurred.
On the other hand, the conditional distribution must add up to 1.
So the total probability over all of the different y's in this universe, that total probability should be equal to 1.
Here it's not equal to 1.
The total area is the marginal density.
To make it equal to 1, we need to divide by the marginal density, which is basically to renormalize this shape so that the total area under that slice, under that shape, is equal to 1.
So we start with the joint.
We take the slices.
And then we adjust the slices so that every slice has an area underneath equal to 1.
And this gives us the conditional.
So for example, down here-- you can not even see it in this diagram-- but after you renormalize it so that its total area is equal to 1, you get this sort of narrow spike that goes up.
And so this is a plot of the conditional distributions that you get for the different values of x.
Given a particular value of x, you're going to get this certain conditional distribution.
So this picture is worth about as much as anything else in this particular chapter.
Make sure you kind of understand exactly all these pieces of the picture.
And finally, let's go, in the remaining time, through an example where we're going to throw in the bucket all the concepts and notations that we have introduced so far.
So the example is as follows.
We start with a stick that has a certain length.
And we break it a completely random location.
And-- yes, this 1 should be l. OK.
So it has length l. And we're going to break it at the random place.
And we call that random place where we break it, we call it X. X can be anywhere, uniform distribution.
So this means that X has a density that goes from 0 to l. I guess this capital L is supposed to be the same as the lower-case l. So that's the density of X.
And since the density needs to integrate to 1, the height of that density has to be 1/l.
Now, having broken the stick and given that we are left with this piece of the stick, I'm now going to break it again at a completely random place, meaning I'm going to choose a point where I break it uniformly over the length of the stick.
What does this mean?
And let's call Y the location where I break it.
So Y is going to range between 0 and x. x is the stick that I'm left with.
So I'm going to break it somewhere in between.
So I pick a y between 0 and x.
And of course, x is less than l. And I'm going to break it there.
So y is uniform between 0 and x.
What does that mean, that the density of y, given that you have already told me x, ranges from 0 to little x?
If I told you that the first break happened at a particular x, then y can only range over this interval.
And I'm assuming a uniform distribution over that interval.
So we have this kind of shape.
And that fixes for us the height of the conditional density.
So what's the joint density of those two random variables?
By the definition of conditional densities, the conditional was defined as the ratio of this divided by that.
So we can find the joint density by taking the marginal and then multiplying by the conditional.
This is the same formula as in the discrete case.
This is our very familiar multiplication rule, but adjusted to the case of continuous random variables.
So Ps become Fs.
OK.
So we do have a formula for this.
What is it?
It's 1/l-- that's the density of X -- times 1/x, which is the conditional density of Y.
This is the formula for the joint density.
But we must be careful.
This is a formula that's not valid anywhere.
It's only valid for the x's and y's that are possible.
And the x's and y's that are possible are given by these inequalities.
So x can range from 0 to l, and y can only be smaller than x.
So this is the formula for the density on this part of our space.
The density is 0 anywhere else.
So what does it look like?
It's basically a 1/x function.
So it's sort of constant along that dimension.
But as x goes to 0, your density goes up and can even blow up.
It sort of looks like a sail that's raised and somewhat curved and has a point up there going to infinity.
So this is the joint density.
Now once you have in your hands a joint density, then you can answer in principle any problem.
It's just a matter of plugging in and doing computations.
How about calculating something like a conditional expectation of Y given a value of x?
OK. That's a concept we have not defined so far.
But how should we define it?
Means the reasonable thing.
We'll define it the same way as ordinary expectations except that since we're given some conditioning information, we should use the probability distribution that applies to that particular situation.
So in a situation where we are told the value of x, the distribution that applies is the conditional distribution of Y.
So it's going to be the conditional density of Y given the value of x.
Now, we know what this is.
It's given by 1/x.
So we need to integrate y times 1/x dy.
And what should we integrate over?
Well, given the value of x, y can only range from 0 to x.
So this is what we get.
And you do your integral, and you get that this is x/2.
Is it a surprise?
It shouldn't be.
This is just the expected value of Y in a universe where X has been realized and Y is given by this distribution.
Y is uniform between 0 and x.
The expected value of Y should be the midpoint of this interval, which is x/2.
Now let's do fancier stuff.
Since we have the joint distribution, we should be able to calculate the marginal.
What is the distribution of Y?
After breaking the stick twice, how big is the little piece that I'm left with?
How do we find this?
To find the marginal, we just take the joint and integrate out the variable that we don't want.
A particular y can happen in many ways.
It can happen together with any x.
So we consider all the possible x's that can go together with this y and average over all those x's.
So we plug in the formula for the joint density from the previous slide.
We know that it's 1/lx.
And what's the range of the x's?
So to find the density of Y for a particular y up here, I'm going to integrate over x's.
The density is 0 here and there.
The density is nonzero only in this part.
So I need to integrate over x's going from here to there.
So what's the "here"?
This line goes up at the slope of 1.
So this is the line x equals y.
So if I fix y, it means that my integral starts from a value of x that is also equal to y.
So where the integral starts from is at x equals y.
And it goes all the way until the end of the length of our stick, which is l. So we need to integrate from little y up to l. So that's something that almost always comes up.
It's not enough to have just this formula for integrating the joint density.
You need to keep track of different regions.
And if the joint density is 0 in some regions, then you exclude those regions from the range of integration.
So the range of integration is only over those values where the particular formula is valid, the places where the joint density is nonzero.
All right.
The integral of 1/x dx, that gives you a logarithm.
So we evaluate this integral, and we get an expression of this kind.
So the density of Y has a somewhat unexpected shape.
So it's a logarithmic function.
And it goes this way.
It's for y going all the way to l. When y is equal to l, the logarithm of 1 is equal to 0.
But when y approaches 0, logarithm of something big blows up, and we get a shape of this form.
OK.
Finally, we can calculate the expected value of Y.
And we can do this by using the definition of the expectation.
So integral of y times the density of y.
We already found what that density is, so we can plug it in here.
And we're integrating over the range of possible y's, from 0 to l. Now this involves the integral for y log y, which I'm sure you have encountered in your calculus classes but maybe do not remember how to do it.
In any case, you look it up in some integral tables or do it by parts.
And you get the final answer of l/4.
And at this point, you say, that's a really simple answer.
Shouldn't I have expected it to be l/4?
I guess, yes.
I mean, when you break it once, the expected value of what you are left with is going to be 1/2 of what you started with.
When you break it the next time, the expected length of what you're left with should be 1/2 of the piece that you are now breaking.
So each time that you break it at random, you expected it to become smaller by a factor of 1/2.
So if you break it twice, you are left something that's expected to be 1/4.
This is reasoning on the average, which happens to give you the right answer in this case.
But again, there's the warning that reasoning on the average doesn't always give you the right answer.
So be careful about doing arguments of this type.
Very good.
See you on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today's agenda is to say a few more things about continuous random variables.
Mainly we're going to talk a little bit about inference.
This is a topic that we're going to revisit at the end of the semester.
But there's a few things that we can already say at this point.
And then the new topic for today is the subject of derived distributions.
Basically if you know the distribution of one random variable, and you have a function of that random variable, how to find a distribution for that function.
And it's a fairly mechanical skill, but that's an important one, so we're going to go through it.
So let's see where we stand.
Here is the big picture.
That's all we have done so far.
We have talked about discrete random variables, which we described by probability mass function.
So if we have multiple random variables, we describe them with the a joint mass function.
And then we define conditional probabilities, or conditional PMFs, and the three are related according to this formula, which is, you can think of it either as the definition of conditional probability.
Or as the multiplication rule, the probability of two things happening is the product of the probabilities of the first thing happening, and then the second happening, given that the first has happened.
There's another relation between this, which is the probability of x occurring, is the sum of the different probabilities of the different ways that x may occur, which is in conjunction with different values of y.
And there's an analog of all that in the continuous world, where all you do is to replace p's by f's, and replace sums by integrals.
So the formulas all look the same.
The interpretations are a little more subtle, so the f's are not probabilities, they're probability densities.
So they're probabilities per unit length, or in the case of joint PDf's, these are probabilities per unit area.
So they're densities of some sort.
Probably the more subtle concept to understand what it really is the conditional density.
In some sense, it's simple.
It's just the density of X in a world where you have been told the value of the random variable Y.
It's a function that has two arguments, but the best way to think about it is to say that we fixed y.
We're told the value of the random variable Y, and we look at it as a function of x.
So as a function of x, the denominator is a constant, and it just looks like the joint density.
when we keep y fixed.
So it's really a function of one argument, just the argument x.
And it has the same shape as the joint's density when you take that slice of it.
So conditional PDFs are just slices of joint PDFs.
There's a bunch of concepts, expectations, variances, cumulative distribution functions that apply equally well for to both universes of discrete or continuous random variables.
So why is probability useful?
Probability is useful because, among other things, we use it to make sense of the world around us.
We use it to make inferences about things that we do not see directly.
And this is done in a very simple manner using the base rule.
We've already seen some of that, and now we're going to revisit it with a bunch of different variations.
And the variations come because sometimes our random variable are discrete, sometimes they're continuous, or we can have a combination of the two.
So the big picture is that there's some unknown random variable out of there, and we know the distribution that's random variable.
And in the discrete case, it's going to be given by PMF.
In the continuous case, it's given a PDF.
Then we have some phenomenon, some noisy phenomenon or some measuring device, and that measuring device produces observable random variables Y.
We don't know what x is, but we have some beliefs about how X is distributed.
We observe the random variable Y.
We need a model of this box.
And the model of that box is going to be either a PMF, for the random variable Y.
And that model tells us, if the true state of the world is X, how do we expect to Y to be distributed?
That's for the case where Y is this discrete.
If Y is a continuous, you might instead have a density for Y, or something of that form.
So in either case, this should be a function that's known to us.
This is our model of the measuring device.
And now having observed y, we want to make inferences about x.
What does it mean to make inferences?
Well the most complete answer in the inference problem is to tell me the probability distribution of the unknown quantity.
But when I say the probability distribution, I don't mean this one.
I mean the probability distribution that takes into account the measurements that you got.
So the output of an inference problem is to come up with the distribution of X, the unknown quantity, given what we have already observed.
And in the discrete case, it would be an object like that.
If X is continuous, it would be an object of this kind.
OK, so we're given conditional probabilities of this type, and we want to get conditional distributions of the opposite type where the order of the conditioning is being reversed.
So the starting point is always a formula such as this one.
The probability of x happening, and then y happening given that x happens.
This is the probability that a particular x and y happen simultaneously.
But this is also equal to the probability that y happens, and then that x happens, given that y has happened.
And you take this expression and send one term to the denominator of the other side, and this gives us the base rule for the discrete case.
Which is this one that you have already seen, and you have played with it.
So this is what the formula looks like in the discrete case.
And the typical example where both random variables are discrete is the one we discussed some time ago.
X is, let's say, a binary variable, or whether an airplane is present up there or not.
Y is a discrete measurement, for example, whether our radar beeped or it didn't beep.
And we make inferences and calculate the probability that the plane is there, or the probability that the plane is not there, given the measurement that we have made.
And of course X and Y do not need to be just binary.
They could be more general discrete random variables.
So how does the story change in the continuous case?
First, what's a possible application of the continuous case?
Well, think of X as being some signal that takes values over a continuous range.
Let's say X is the current through a resistor.
And then you have some measuring device that measures currents, but that device is noisy, it gets hit, let's say for example, by Gaussian noise.
And the Y that you observe is a noisy version of X.
But your instruments are analog, so you measure things on a continuous scale.
What are you going to do in that case?
Well the inference problem, the output of the inference problem, is going to be the conditional distribution of X.
What do you think your current is based on a particular value of Y that you have observed?
So the output of our inference problem is, given the specific value of Y, to calculate this entire function as a function of x, and then go and plot it.
How do we calculate it?
You go through the same calculation as in the discrete case, except that all of the x's gets replaced by p's.
In the continuous case, it's equally true that the joint's density is the product of the marginal density with the conditional density.
So the formula is still valid with just a little change of notation.
So we end up with the same formula here, except that we replace x's with p's.
So all of these functions are known to us.
We have formulas for them.
We fix a specific value of y, we plug it in, so we're left with a function of x.
And that gives us the posterior distribution.
Actually there's also a denominator term that's not necessarily given to us, but we can always calculate it if we have the marginal of X, and we have the model for measuring device.
Then we can always find the marginal distribution of Y.
So this quantity, that number, is in general a known one, as well, and doesn't give us any problems.
So to complicate things a little bit, we can also look into situations where our two random variables are of different kinds.
For example, one random variable could be discrete, and the other it might be continuous.
And there's two versions.
Here one version is when X is discrete, but Y is continuous.
What's an example of this?
Well suppose that I send a single bit of information so my X is 0 or 1.
And what I measure is Y, which is X plus, let's say, Gaussian noise.
This is the standard example that shows up in any textbook on communication, or signal processing.
You send a single bit, but what you observe is a noisy version of that bit.
You start with a model of your x's.
These would be your prior probabilities.
For example, you might be believe that either 0 or 1 are equally likely, in which case your PMF gives equal weight to two possible values.
And then we need a model of our measuring device.
This is one specific model.
The general model would have a shape such as follows.
Y has a distribution, its density.
And that density, however, depends on the value of X.
So when x is 0, we might get a density of this kind.
And when x is 1, we might get the density of a different kind.
So these are the conditional densities of y in a universe that's specified by a particular value of x.
And then we go ahead and do our inference.
OK, what's the right formula for doing this inference?
We need a formula that's sort of an analog of this one, but applies to the case where we have two random variables of different kinds.
So let me just redo this calculation here.
Except that I'm not going to have a probability of taking specific values.
It will have to be something a little different.
So here's how it goes.
Let's look at the probability that X takes a specific value that makes sense in the discrete case, but for the continuous random variable, let's look at the probability that it takes values in some little interval.
And now this probability of two things happening, I'm going to write it as a product.
And I'm going to write this as a product in two different ways.
So one way is to say that this is the probability that X takes that value and then given that X takes that value, the probability that Y falls inside that interval.
So this is our usual multiplication rule for multiplying probabilities, but I can use the multiplication rule also in a different way.
It's the probability that Y falls in the range of interest.
And then the probability that X takes the value of interest given that Y satisfies the first condition.
So this is something that's definitely true.
We're just using the multiplication rule.
And now let's translate it into PMF is PDF notation.
So the entry up there is the PMF of X evaluated at x.
The second entry, what is it?
Well probabilities of little intervals are given to us by densities.
But we are in the conditional universe where X takes on a particular value.
So it's going to be the density of Y given the value of X times delta.
So probabilities of little intervals are given by the density times the length of the little interval, but because we're working in the conditional universe, it has to be the conditional density.
Now let's try the second expression.
This is the probability that the Y falls into the little interval.
So that's the density of Y times delta.
And then here we have an object which is the conditional probability X in a universe where the value of Y is given to us.
Now this relation is sort of approximate.
This is true for very small delta in the limit.
But we can cancel the deltas from both sides, and we're left with a formula that links together PMFs and PDFs.
Now this may look terribly confusing because there's both p's and f's involved.
But the logic should be clear.
If a random variable is discrete, it's described by PMF.
So here we're talking about the PMF of X in some particular universe.
X is discrete, so it has a PMF.
Similarly here.
Y is continuous so it's described by a PDF.
And even in the conditional universe where I tell you the value of X, Y is still a continuous random variable, so it's been described by a PDF.
So this is the basic relation that links together PMF and PDFs.
In this mixed the world.
And now in this inequality, you can take this term and send it to the new denominator to the other side.
And what you end up with is the formula that we have up here.
And this is a formula that we can use to make inferences about the discrete random variable X when we're told the value of the continuous random variable Y.
The probability that X takes on a particular value has something to do with the prior.
And other than that, it's proportional to this quantity, the conditional of Y given X.
So these are the quantities that we plotted here.
Suppose that the x's are equally likely in your prior, so we don't really care about that term.
It tells us that the posterior of X is proportional to that particular density under the given x's.
So in this picture, if I were to get a particular y here, I would say that x equals 1 has a probability that's proportional to this quantity.
x equals 0 has a probability that's proportional to this quantity.
So the ratio of these two quantities gives us the relative odds of the different x's given the y that we have observed.
So we're going to come back to this topic and redo plenty of examples of these kinds towards the end of the class, when we spend some time dedicated to inference problems.
But already at this stage, we sort of have the basic skills to deal with a lot of that.
And it's useful at this point to pull all the formulas together.
So finally let's look at the last case that's remaining.
Here we have a continuous phenomenon that we're trying to measure, but our measurements are discrete.
What's an example where this might happen?
So you have some device that emits light, and you drive it with a current that has a certain intensity.
You don't know what that current is, and it's a continuous random variable.
But the device emits light by sending out individual photons.
And your measurement is some other device that counts how many photons did you get in a single second.
So if we have devices that emit a very low intensity you can actually start counting individual photons as they're being observed.
So we have a discrete measurement, which is the number of problems, and we have a continuous hidden random variable that we're trying to estimate.
What do we do in this case?
Well we start again with a formula of this kind, and send the p term to the denominator.
And that's the formula that we use there, except that the roles of x's and y's are interchanged.
So since here we have Y being discrete, we should change all the subscripts.
It would be p_Y f_X given y f_X, and P(Y given X).
So just change all those subscripts.
Because now what we're used to be continuous became discrete, and vice versa.
Take that formula, send the other terms to the denominator, and we have a formula for the density, or X, given the particular measurements for Y that we have obtained.
In some sense that's all there is in Bayesian inference.
It's using these very simple one line formulas.
But why are there people then who make their living solving inference problems?
Well, the devil is in the details.
As we're going to discuss, there are some real world issues of how exactly do you design your f's, how do you model your system, then how do you do your calculations.
This might not be always easy.
For example, there's certain integrals or sums that have to be evaluated, which may be hard to do and so on.
So this object is a lot of richer than just these formulas.
On the other hand, at the conceptual level, that's the basis for Bayesian inference, that these are the basic concepts.
All right, so now let's change gear and move to the new subject, which is the topic of finding the distribution of a functional for a random variable.
We call those distributions derived distributions, because we're given the distribution of X.
We're interested in a function of X.
We want to derive the distribution of that function based on the distribution that we already know.
So it could be a function of just one random variable.
It could be a function of several random variables.
So one example that we are going to solve at some point, let's say you have to run the variables X and Y.
Somebody tells you their distribution, for example, is a uniform of the square.
For some reason, you're interested in the ratio of these two random variables, and you want to find the distribution of that ratio.
You can think of lots of cases where your random variable of interest is created by taking some other unknown variables and taking a function of them.
And so it's legitimate to care about the distribution of that random variable.
A caveat, however.
There's an important case where you don't need to find the distribution of that random variable.
And this is when you want to calculate the expectations.
If all you care about is the expected value of this function of the random variables, you can work directly with the distribution of the original random variables without ever having to find the PDF of g. So you don't do unnecessary work if it's not needed, but if it's needed, or if you're asked to do it, then you just do it.
So how do we find the distribution of the function?
As a warm-up, let's look at the discrete case.
Suppose that X is a discrete random variable and takes certain values.
We have a function g that maps x's into y's.
And we want to find the probability mass function for Y.
So for example, if I'm interested in finding the probability that Y takes on this particular value, how would they find it?
Well I ask, what are the different ways that these particular y value can happen?
And the different ways that it can happen is either if x takes this value, or if X takes that value.
So we identify this event in the y space with that event in the x space.
These two events are identical.
X falls in this set if and only if Y falls in that set.
Therefore, the probability of Y falling in that set is the probability of X falling in that set.
The probability of X falling in that set is just the sum of the individual probabilities of the x's in this set.
So we just add the probabilities of the different x's where the summation is taken over all x's that leads to that particular value of y.
Very good.
So that's all there is in the discrete case.
It's a very nice and simple.
So let's transfer these methods to the continuous case.
Suppose we are in the continuous case.
Suppose that X and Y now can take values anywhere.
And I try to use same methods and I ask, what is the probability that Y is going to take this value?
At least if the diagram is this way, you would say this is the same as the probability that X takes this value.
So I can find the probability of Y being this in terms of the probability of X being that.
Is this useful?
In the continuous case, it's not.
Because in the continuous case, any single value has 0 probability.
So what you're going to get out of this argument is that the probability Y takes this value is 0, is equal to the probability that X takes that value which also 0.
That doesn't help us.
We want to do something more.
We want to actually find, perhaps, the density of Y, as opposed to the probabilities of individual y's.
So to find the density of Y, you might argue as follows.
I'm looking at an interval for y, and I ask what's the probability of falling in this interval.
And you go back and find the corresponding set of x's that leads to those y's, and equate those two probabilities.
The probability of all of those y's collectively should be equal to the probability of all of the x's that map into that interval collectively.
And this way you can relate the two.
As far as the mechanics go, in many cases it's easier to not to work with little intervals, but instead to work with cumulative distribution functions that used to work with sort of big intervals.
So you can instead do a different picture.
Look at this set of y's.
This is the set of y's that are smaller than a certain value.
The probability of this set is given by the cumulative distribution of the random variable Y.
Now this set of y's gets produced by some corresponding set of x's.
Maybe these are the x's that map into y's in that set.
And then we argue as follows.
The probability that the Y falls in this interval is the same as the probability that X falls in that interval.
So the event of Y falling here and the event of X falling there are the same, so their probabilities must be equal.
And then I do the calculations here.
And I end up getting the cumulative distribution function of Y.
Once I have the cumulative, I can get the density by just differentiating.
So this is the general cookbook procedure that we will be using to calculate it derived distributions.
We're interested in a random variable Y, which is a function of the x's.
We will aim at obtaining the cumulative distribution of Y.
Somehow, manage to calculate the probability of this event.
Once we get it, and what I mean by get it, I don't mean getting it for a single value of little y.
You need to get this for all little y's.
So you need to get the function itself, the cumulative distribution.
Once you get it in that form, then you can calculate the derivative at any particular point.
And this is going to give you the density of Y.
So a simple two-step procedure.
The devil is in the details of how you carry the mechanics.
So let's do one first example.
Suppose that X is a uniform random variable, takes values between 0 and 2.
We're interested in the random variable Y, which is the cube of X.
What kind of distribution is it going to have?
Now first notice that Y takes values between 0 and 8.
So X is uniform, so all the x's are equally likely.
You might then say, well, in that case, all the y's should be equally likely.
So Y might also have a uniform distribution.
Is this true?
We'll find out.
So let's start applying the cookbook procedure.
We want to find first the cumulative distribution of the random variable Y, which by definition is the probability that the random variable is less than or equal to a certain number.
That's what we want to find.
What we have in our hands is the distribution of X.
That's what we need to work with.
So the first step that you need to do is to look at this events and translate it, and write it in terms of the random variable about which you know you have information.
So Y is X cubed, so this event is the same as that event.
So now we can forget about the y's.
It's just an exercise involving a single random variable with a known distribution and we want to calculate the probability of some event.
So we're looking at this event.
X cubed being less than or equal to Y.
We massage that expression so that's it involves X directly, so let's take cubic roots of both sides of this inequality.
This event is the same as the event that X is less than or equal to Y to the 1/3.
Now with a uniform distribution on [0,2], what is that probability going to be?
It's the probability of being in the interval from 0 to y to the 1/3, so it's going to be in the area under the uniform going up to that point.
And what's the area under that uniform?
So here's x.
Here is the distribution of X.
It goes up to 2.
The distribution of X is this one.
We want to go up to y to the 1/3.
So the probability for this event happening is this area.
And the area is equal to the base, which is y to the 1/3 times the height.
What is the height?
Well since the density must integrate to 1, the total area under the curve has to be 1.
So the height here is 1/2, and that explains why we get the 1/2 factor down there.
So that's the formula for the cumulative distribution.
And then the rest is easy.
You just take derivatives.
You differentiate this expression with respect to y 1/2 times 1/3, and y drops by one power.
So you get y to 2/3 in the denominator.
So if you wish to plot this, it's 1/y to the 2/3.
So when y goes to 0, it sort of blows up and it goes on this way.
Is this picture correct the way I've drawn it?
What's wrong with it?
[?
Something.
?]
Yes.
y only takes values from 0 to 8.
This formula that I wrote here is only correct when the preview picture applies.
I took my y to the 1/3 to be between 0 and 2.
So this formula here is only correct for y between 0 and 8.
And for that reason, the formula for the derivative is also true only for a y between 0 and 8.
And any other values of why are impossible, so they get zero density.
So to complete the picture here, the PDF of y has a cut-off of 8, and it's also 0 everywhere else.
And one thing that we see is that the distribution of Y is not uniform.
Certain y's are more likely than others, even though we started with a uniform random variable X.
All right.
So we will keep doing examples of this kind, a sequence of progressively more interesting or more complicated.
So that's going to continue in the next lecture.
You're going to see plenty of examples in your recitations and tutorials and so on.
So let's do one that's pretty similar to the one that we did, but it's going to add to just a small twist in how we do the mechanics.
OK so you set your cruise control when you start driving.
And you keep driving at the constants based at the constant speed.
Where you set your cruise control is somewhere between 30 and 60.
You're going to drive a distance of 200.
And so the time it's going to take for your trip is 200 over the setting of your cruise control.
So it's 200/V.
Somebody gives you the distribution of V, and they tell you not only it's between 30 and 60, it's roughly equally likely to be anything between 30 and 60, so we have a uniform distribution over that range.
So we have a distribution of V. We want to find the distribution of the random variable T, which is the time it takes till your trip ends.
So how are we going to proceed?
We'll use the exact same cookbook procedure.
We're going to start by finding the cumulative distribution of T. What is this?
By definition, the cumulative distribution is the probability that T is less than a certain number.
OK. Now we don't know the distribution of T, so we cannot to work with these event directly.
But we take that event and translate it into T-space.
So we replace the t's by what we know T to be in terms of V or the v's All right.
So we have the distribution of V. So now let's calculate this quantity.
OK. Let's massage this event and rewrite it as the probability that V is larger or equal to 200/T.
So what is this going to be?
So let's say that 200/T is some number that falls inside the range.
So that's going to be true if 200/T is bigger than 30, and less than 60.
Which means that t is less than 30/200.
No, 200/30.
And bigger than 200/60.
So for t's inside that range, this number 200/t falls inside that range.
This is the range of t's that are possible, given the description of the problem the we have set up.
So for t's in that range, what is the probability that V is bigger than this number?
So V being bigger than that number is the probability of this event, so it's going to be the area under this curve.
So the area under that curve is the height of the curve, which is 1/3 over 30 times the base.
How big is the base?
Well it's from that point to 60, so the base has a length of 60 minus 200/t.
And this is a formula which is valid for those t's for which this picture is correct.
And this picture is correct if 200/T happens to fall in this interval, which is the same as T falling in that interval, which are the t's that are possible.
So finally let's find the density of T, which is what we're looking for.
We find this by taking the derivative in this expression with respect to t. We only get one term from here.
And this is going to be 200/30, 1 over t squared.
And this is the formula for the density for t's in the allowed to range.
OK, so that's the end of the solution to this particular problem as well.
I said that there was a little twist compared to the previous one.
What was the twist?
Well the twist was that in the previous problem we dealt with the X cubed function, which was monotonically increasing.
Here we dealt with the function that was monotonically decreasing.
So when we had to find the probability that T is less than something, that translated into an event that V was bigger than something.
Your time is less than something if and only if your velocity is bigger than something.
So for when you're dealing with the monotonically decreasing function, at some point some inequalities will have to get reversed.
Finally let's look at a very useful one.
Which is the case where we take a linear function of a random variable.
So X is a random variable with given distribution, and we can see there is a linear function.
So in this particular instance, we take a to be equal to 2 and b equal to 5.
And let us first argue just by picture.
So X is a random variable that has a given distribution.
Let's say it's this weird shape here.
And x ranges from -1 to +2.
Let's do things one step at the time.
Let's first find the distribution of 2X.
Why do you think you know about 2X?
Well if x ranges from -1 to 2, then the random variable X is going to range from -2 to +4.
So that's what the range is going to be.
Now dealing with the random variable 2X, as opposed to the random variable X, in some sense it's just changing the units in which we measure that random variable.
It's just changing the scale on which we draw and plot things.
So if it's just a scale change, then intuition should tell you that the random variable X should have a PDF of the same shape, except that it's scaled out by a factor of 2, because our random variable of 2X now has a range that's twice as large.
So we take the same PDF and scale it up by stretching the x-axis by a factor of 2.
So what does scaling correspond to in terms of a formula?
So the distribution of 2X as a function, let's say, a generic argument z, is going to be the distribution of X, but scaled by a factor of 2.
So taking a function and replacing its arguments by the argument over 2, what it does is it stretches it by a factor of 2.
You have probably been tortured ever since middle school to figure out when need to stretch a function, whether you need to put 2z or z/2.
And the one that actually does the stretching is to put the z/2 in that place.
So that's what the stretching does.
Could that to be the full answer?
Well there's a catch.
If you stretch this function by a factor of 2, what happens to the area under the function?
It's going to get doubled.
But the total probability must add up to 1, so we need to do something else to make sure that the area under the curve stays to 1.
So we need to take that function and scale it down by this factor of 2.
So when you're dealing with a multiple of a random variable, what happens to the PDF is you stretch it according to the multiple, and then scale it down by the same number so that you preserve the area under that curve.
So now we found the distribution of 2X.
How about the distribution of 2X + 5?
Well what does adding 5 to random variable do?
You're going to get essentially the same values with the same probability, except that those values all get shifted by 5.
So all that you need to do is to take this PDF here, and shift it by 5 units.
So the range used to be from -2 to 4.
The new range is going to be from 3 to 9.
And that's the final answer.
This is the distribution of 2X + 5, starting with this particular distribution of X.
Now shifting to the right by b, what does it do to a function?
Shifting to the right to by a certain amount, mathematically, it corresponds to putting -b in the argument of the function.
So I'm taking the formula that I had here, which is the scaling by a factor of a.
The scaling down to keep the total area equal to 1.
And then I need to introduce this extra term to do the shifting.
So this is a plausible argument.
The proof by picture that this should be the right answer.
But just in order to keep our skills tuned and refined, let us do this derivation in a more formal way using our two-step cookbook procedure.
And I'm going to do it under the assumption that a is positive, as in the example that's we just did.
So what's the two-step procedure?
We want to find the cumulative of Y, and after that we're going to differentiate.
By definition the cumulative is the probability that the random variable takes values less than a certain number.
And now we need to take this event and translate it, and express it in terms of the original random variables.
So Y is, by definition, aX + b, so we're looking at this event.
And now we want to express this event in a clean form where X shows up in a straight way.
Let's say I'm going to massage this event and write it in this form.
For this inequality to be true, x should be less than or equal to (y minus b) divided by a. OK, now what is this?
This is the cumulative distribution of X evaluated at the particular point.
So we got a formula for the cumulative Y based on the cumulative of X.
What's the next step?
Next step is to take derivatives of both sides.
So the density of Y is going to be the derivative of this expression with respect to y. OK, so now here we need to use the chain rule.
It's going to be the derivative of the F function with respect to its argument.
And then we need to take the derivative of the argument with respect to y.
What is the derivative of the cumulative?
The derivative of the cumulative is the density itself.
And we evaluate it at the point of interest.
And then the chain rule tells us that we need to take the derivative of this with respect to y, and the derivative of this with respect to y is 1/a.
And this gives us the formula which is consistent with what I had written down here, for the case where a is a positive number.
What if a was a negative number?
Could this formula be true?
Of course not.
Densities cannot be negative, right?
So that formula cannot be true.
Something needs to change.
What should change?
Where does this argument break down when a is negative?
So when I write this inequality in this form, I divide by a.
But when you divide by a negative number, the direction of an inequality is going to change.
So when a is negative, this inequality becomes larger than or equal to.
And in that case, the expression that I have up there would change when this is larger than here.
Instead of getting the cumulative, I would get 1 minus the cumulative of (y minus b) divided by a.
So this is the probability that X is bigger than this particular number.
And now when you take the derivatives, there's going to be a minus sign that shows up.
And that minus sign will end up being here.
And so we're taking the negative of a negative number, and that basically is equivalent to taking the absolute value of that number.
So all that happens when we have a negative a is that we have to take the absolute value of the scaling factor instead of the factor itself.
All right, so this general formula is quite useful for dealing with linear functions of random variables.
And one nice application of it is to take the formula for a normal random variable, consider a linear function of a normal random variable, plug into this formula, and what you will find is that Y also has a normal distribution.
So using this formula, now we can prove a statement that I had made a couple of lectures ago, that a linear function of a normal random variable is also linear.
That's how you would prove it.
I think this is it for today so.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
So today we're going to continue the subject from last time.
So we're going to talk about derived distributions a little more, how to derive the distribution of a function of a random variable.
So last time we discussed a couple of examples in which we had a function of a single variable.
And we found the distribution of Y, if we're told the distribution of X.
So today we're going to do an example where we deal with the function of two random variables.
And then we're going to consider the most interesting example of this kind, in which we have a random variable of the form W, which is the sum of two independent, random variables.
That's a case that shows up quite often.
And so we want to see what exactly happens in this particular case.
Just one comment that I should make.
The material that we're covering now, chapter four, is sort of conceptually a little more difficult than one we have been doing before.
So I would definitely encourage you to read the text before you jump and try to do the problems in your problem sets.
OK, so let's start with our example, in which we're given two random variables.
They're jointly continuous.
And their distribution is pretty simple.
They're uniform on the unit square.
In particular, each one of the random variables is uniform on the unit interval.
And the two random variables are independent.
What we're going to find is the distribution of the ratio of the two random variables.
How do we go about it?
, Well, the same cookbook procedure that we used last time for the case of a single random variable.
The cookbook procedure that we used for this case also applies to the case where you have a function of multiple random variables.
So what was the cookbook procedure?
The first step is to find the cumulative distribution function of the random variable of interest and then take the derivative in order to find the density.
So let's find the cumulative.
So, by definition, the cumulative is the probability that the random variable is less than or equal to the argument of the cumulative.
So if we write this event in terms of the random variable of interest, this is the probability that our random variable is less than or equal to z.
So what is that?
OK, so the ratio is going to be less than or equal to z, if and only if the pair, (x,y), happens to fall below the line that has a slope z. OK, so we draw a line that has a slope z.
The ratio is less than this number, if and only if we get the pair of x and y that falls inside this triangle.
So we're talking about the probability of this particular event.
Since this line has a slope of z, the height at this point is equal to z.
And so we can find the probability of this event.
It's just the area of this triangle.
And so the area is 1 times z times 1/2.
And we get the answer, z/2.
Now, is this answer always correct?
Now, this answer is going to be correct only if the slope happens to be such that we get a picture of this kind.
So when do we get a picture of this kind?
When the slope is less than 1.
If I consider a different slope, a number, little z -- that happens to be a slope of that kind -- then the picture changes.
And in that case, we get a picture of this kind, let's say.
So this is a line here of slope z, again.
And this is the second case in which our number, little z, is bigger than 1.
So how do we proceed?
Once more, the cumulative is the probability that the ratio is less than or equal to that number.
So it's the probability that we fall below the red line.
So we're talking about the event, about this event.
So to find the probability of this event, we need to find the area of this red shape.
And one way of finding this area is to consider the whole area and subtract the area of this triangle.
So let's do it this way.
It's going to be 1 minus the area of the triangle.
Now, what's the area of the triangle?
It's 1/2 times this side, which is 1 times this side.
How big is that side?
Well, if y and the slope is z, now z is the ratio y over x.
So if y over x-- at this point we have y/x = z and y =1.
This means that z is 1/x.
So the coordinate of this point is 1/x.
And this means that we're going to-- 1/z So here we get the factor of 1/z.
And we're basically done.
I guess if you want to have a complete answer, you should also give the formula for z less than 0.
What is the cumulative when z is less than 0, the probability that you get the ratio that's negative?
Well, since our random variables are positive, there's no way that you can get a negative ratio.
So the cumulative down there is equal to 0.
So we can plot the cumulative.
And we can take its derivative in order to find the density.
So the cumulative that we got starts at 0, when z's are negative.
Then it starts going up in proportion to z, at the slope of 1/2.
So this takes us up to 1.
And then it starts increasing towards 1, according to this function.
When you let z go to infinity, the cumulative is going to go to 1.
And it has a shape of, more or less, this kind.
So now to get the density, we just take the derivative.
And the density is, of course, 0 down here.
Up here the derivative is just 1/2.
And beyond that point we need to take the derivative of this expression.
And the derivative is going to be 1/2 times 1 over z-squared.
So it's going to be a shape of this kind.
And we're done.
So you see that problems involving functions of multiple random variables are no harder than problems that deal with the functional of a single random variable.
The general procedure is, again, exactly the same.
You first find the cumulative, and then you differentiate.
The only extra difficulty will be that when you calculate the cumulative, you need to find the probability of an event that involves multiple random variables.
And sometimes this could be a little harder to do.
By the way, since we dealt with this example, just a couple of questions.
What do you think is going to be the expected value of the random variable Z?
Let's see, the expected value of the random variable Z is going to be the integral of z times the density.
And the density is equal to 1/2 for z going from 0 to 1.
And then there's another contribution from 1 to infinity.
There the density is 1/(2z-squared).
And we get the z, since we're dealing with expectations, dz.
So what is this integral?
Well, if you look here, you're integrating 1/z, all the way to infinity.
1/z has an integral, which is the logarithm of z.
And since the logarithm goes to infinity, this means that this integral is also infinite.
So the expectation of the random variable Z is actually infinite in this example.
There's nothing wrong with this.
Lots of random variables have infinite expectations.
If the tail of the density falls kind of slowly, as the argument goes to infinity, then it may well turn out that you get an infinite integral.
So that's just how things often are.
Nothing strange about it.
And now, since we are still in this example, let me ask another question.
Would we reason, on the average, would it be true that the expected value of Z -- remember that Z is the ratio Y/X -- could it be that the expected value of Z is this number?
Or could it be that it's equal to this number?
Or could it be that it's none of the above?
OK, so how many people think this is correct?
Small number.
How many people think this is correct?
Slightly bigger, but still a small number.
And how many people think this is correct?
OK, that's-- this one wins the vote.
OK, let's see.
This one is not correct, just because there's no reason it should be correct.
So, in general, you cannot reason on the average.
The expected value of a function is not the same as the same function of the expected values.
This is only true if you're dealing with linear functions of random variables.
So this is not-- this turns out to not be correct.
How about this one?
Well, X and Y are independent, by assumption.
So 1/X and Y are also independent.
Why is this?
Independence means that one random variable does not convey any information about the other.
So Y doesn't give you any information about X.
So Y doesn't give you any information about 1/X.
Or to put it differently, if two random variables are independent, functions of each one of those random variables are also independent.
If X is independent from Y, then g(X) is independent of h(Y).
So this applies to this case.
These two random variables are independent.
And since they are independent, this means that the expected value of their product is equal to the product of the expected values.
So this relation actually is true.
And therefore, this is not true.
OK. Now, let's move on.
We have this general procedure of finding the derived distribution by going through the cumulative.
Are there some cases where we can have a shortcut?
Turns out that there is a special case or a special structure in which we can get directly from densities to densities using directly just a formula.
And in that case, we don't have to go through the cumulative.
And this case is also interesting, because it gives us some insight about how one density changes to a different density and what affects the shape of those densities.
So the case where things easy is when the transformation from one random variable to the other is a strictly monotonic one.
So there's a one-to-one relation between x's and y's.
Here we can reason directly in terms of densities by thinking in terms of probabilities of small intervals.
So let's look at the small interval on the x-axis, like this one, when X ranges from-- where capital X ranges from a small x to a small x plus delta.
So this is a small interval of length delta.
Whenever X happens to fall in this interval, the random variable Y is going to fall in a corresponding interval up there.
So up there we have a corresponding interval.
And these two intervals, the red and the blue interval-- this is the blue interval.
And that's the red interval.
These two intervals should have the same probability.
They're exactly the same event.
When X falls here, g(X) happens to fall in there.
So we can sort of say that the probability of this little interval is the same as the probability of that little interval.
And we know that probabilities of little intervals have something to do with densities.
So what is the probability of this little interval?
It's the density of the random variable X, at this point, times the length of the interval.
How about the probability of that interval?
It's going to be the density of the random variable Y times the length of that little interval.
Now, this interval has length delta.
Does that mean that this interval also has length delta?
Well, not necessarily.
The length of this interval has something to do with the slope of your function g. So slope is dy by dx.
Is how much-- the slope tells you how big is the y interval when you take an interval x of a certain length.
So the slope is what multiplies the length of this interval to give you the length of that interval.
So the length of this interval is delta times the slope of your function.
So the length of the interval is delta times the slope of the function, approximately.
So the probability of this interval is going to be the density of Y times the length of the interval that we are considering.
So this gives us a relation between the density of X, evaluated at this point, to the density of Y, evaluated at that point.
The two densities are closely related.
If these x's are very likely to occur, then this is big, which means that that density will also be big.
If these x's are very likely to occur, then those y's are also very likely to occur.
But there's also another factor that comes in.
And that's the slope of the function at this particular point.
So we have this relation between the two densities.
Now, in interpreting this equation, you need to make sure what's the relation between the two variables.
I have both little x's and little y's.
Well, this formula is true for an (x,y) pair, that they're related according to this particular function.
So if I fix an x and consider the corresponding y, then the densities at those x's and corresponding y's will be related by that formula.
Now, in the end, you want to come up with a formula that just gives you the density of Y as a function of y.
And that means that you need to eliminate x from the picture.
So let's see how that would go in an example.
So suppose that we're dealing with the function y equal to x cubed, in which case our function, g(x), is the function x cubed.
And if x cubed is equal to a little y, If we have a pair of x's and y's that are related this way, then this means that x is going to be the cubic root of y.
So this is the formula that takes us back from y's to x's.
This is the direct function from x, how to construct y.
This is essentially the inverse function that tells us, from a given y what is the corresponding x.
Now, if we write this formula, it tells us that the density at the particular x is going to be the density at the corresponding y times the slope of the function at the particular x that we are considering.
The slope of the function is 3x squared.
Now, we want to end up with a formula for the density of Y.
So I'm going to take this factor, send it to the other side.
But since I want it to be a function of y, I want to eliminate the x's.
And I'm going to eliminate the x's using this correspondence here.
So I'm going to get the density of X evaluated at y to the 1/3.
And then this factor in the denominator, it's 1/(3y to the power 2/3).
So we end up finally with the formula for the density of the random variable Y.
And this is the same answer that you would get if you go through this exercise using the cumulative distribution function method.
You end up getting the same answer.
But here we sort of get it directly.
Just to get a little more insight as to why the slope comes in-- suppose that we have a function like this one.
So the function is sort of flat, then moves quickly, and then becomes flat again.
What should be -- and suppose that X has some kind of reasonable density, some kind of flat density.
Suppose that X is a pretty uniform random variable.
What's going to happen to the random variable Y?
What kind of distribution should it have?
What are the typical values of the random variable Y?
Either x falls here, and y is a very small number, or-- let's take that number here to be -- let's say 2 -- or x falls in this range, and y takes a value close to 2.
And there's a small chance that x's will be somewhere in the middle, in which case y takes intermediate values.
So what kind of shape do you expect for the distribution of Y?
There's going to be a fair amount of probability that Y takes values close to 0.
There's a small probability that Y takes intermediate values.
That corresponds to the case where x falls in here.
That's not a lot of probability.
So the probability that Y takes values between 0 and 2, that's kind of small.
But then there's a lot of x's that produces y's that are close to 2.
So there's a significant probability that Y would take values that are close to 2.
So you-- the density of Y would have a shape of this kind.
By looking at this picture, you can tell that it's most likely that either x will fall here or x will fall there.
So the g(x) is most likely to be close to 0 or to be close to 2.
So since y is most likely to be close to 0 or close to most of the probability of y is here.
And there's a small probability of being in between.
Notice that the y's that get a lot of probability are those y's associated with flats regions off your g function.
When the g function is flat, that gives you big densities for Y.
So the density of Y is inversely proportional to the slope of the function.
And that's what you get from here.
The density of Y is-- send that term to the other side-- is inversely proportional to the slope of the function that you're dealing with.
OK, so this formula works nicely for the case where the function is one-to-one.
So we can have a unique association between x's and y's and through an inverse function, from y's to x's.
It works for the monotonically increasing case.
It also works for the monotonically decreasing case.
In the monotonically decreasing case, the only change that you need to do is to take the absolute value of the slope, instead of the slope itself.
OK, now, here's another example or a special case.
Let's talk about the most interesting case that involves a function of two random variables.
And this is the case where we have two independent, random variables, and we want to find the distribution of the sum of the two.
We're really interested in the continuous case.
But as a warm-up, it's useful to look at the discrete case first of discrete random variables.
Let's say we want to find the probability that the sum of X and Y is equal to a particular number.
And to illustrate this, let's take that number to be equal to 3.
What's the probability that the sum of the two random variables is equal to 3?
To find the probability that the sum is equal to 3, you consider all possible ways that you can get the sum of 3.
And the different ways are the points in this picture.
And they correspond to a line that goes this way.
So the probability that the sum is equal to a certain number is the probability that -- is the sum of the probabilities of all of those points.
What is a typical point in this picture?
In a typical point, the random variable X takes a certain value.
And Y takes the value that's needed so that the sum is equal to W. Any combination of an x with a w minus x, any such combination gives you a sum of w. So the probability that the sum is w is the sum over all possible x's.
That's over all these points of the probability that we get a certain x.
Let's say x equals 2 times the corresponding probability that random variable Y takes the value 1.
And why am I multiplying probabilities here?
That's where we use the assumption that the two random variables are independent.
So the probability that X takes a certain value and Y takes the complementary value, that probability is the product of two probabilities because of independence.
And when we write that into our usual PMF notation, it's a formula of this kind.
So this formula is called the convolution formula.
It's an operation that takes one PMF and another PMF-- p we're given the PMF's of X and Y -- and produces a new PMF.
So think of this formula as giving you a transformation.
You take two PMF's, you do something with them, and you obtain a new PMF.
This procedure, what this formula does is -- nicely illustrated sort of by mechanically.
So let me show you a picture here and illustrate how the mechanics go, in general.
So you don't have these slides, but let's just reason through it.
So suppose that you are given the PMF of X, and it has this shape.
You're given the PMF of Y.
It has this shape.
And somehow we are going to do this calculation.
Now, we need to do this calculation for every value of W, in order to get the PMF of W. Let's start by doing the calculation just for one case.
Suppose the W is equal to 0, in which case we need to find the sum of Px(x) and Py(-x).
How do you do this calculation graphically?
It involves the PMF of X.
But it involves the PMF of Y, with the argument reversed.
So how do we plot this?
Well, in order to reverse the argument, what you need is to take this PMF and flip it.
So that's where it's handy to have a pair of scissors with you.
So you cut this down.
And so now you take the PMF of the random variable Y and just flip it.
So what you see here is this function where the argument is being reversed.
And then what do we do?
We cross-multiply the two plots.
Any entry here gets multiplied with the corresponding entry there.
And we consider all those products and add them up.
In this particular case, the flipped PMF doesn't have any overlap with the PMF of X.
So we're going to get an answer that's equal to 0.
So for w's equal to 0, the Pw is going to be equal to 0, in this particular plot.
Now if we have a different value of w -- oops.
If we have a different value of the argument w, then we have here the PMF of Y that's flipped and shifted by an amount of w. So the correct picture of what you do is to take this and displace it by a certain amount of w. So here, how much did I shift it?
I shifted it until one falls just below 4.
So I have shifted by a total amount of 5.
So 0 falls under 5, whereas 0 initially was under 0.
So I'm shifting it by 5 units.
And I'm now going to cross-multiply and add.
Does this give us the correct-- does it do the correct thing?
Yes, because a typical term will be the probability that this random variable is 3 times the probability that this random variable is 2.
That's a particular way that you can get a sum of 5.
If you see here, the way that things are aligned, it gives you all the different ways that you can get the sum of 5.
You can get the sum of 5 by having 1 + 4, or 2 + 3, or 3 + 2, or 4 + 1.
You need to add the probabilities of all those combinations.
So you take this times that.
That's one product term.
Then this times 0, this times that.
And so 1-- you cross-- you find all the products of the corresponding terms, and you add them together.
So it's a kind of handy mechanical procedure for doing this calculation, especially when the PMF's are given to you in terms of a picture.
So the summary of these mechanics are just what we did, is that you put the PMF's on top of each other.
You take the PMF of Y.
You flip it.
And for any particular w that you're interested in, you take this flipped PMF and shift it by an amount of w. Given this particular shift for a particular value of w, you cross-multiply terms and then accumulate them or add them together.
What would you expect to happen in the continuous case?
Well, the story is familiar.
In the continuous case, pretty much, almost always things work out the same way, except that we replace PMF's by PDF's.
And we replace sums by integrals.
So there shouldn't be any surprise here that you get a formula of this kind.
The density of W can be obtained from the density of X and the density of Y by calculating this integral.
Essentially, what this integral does is it fits a particular w of interest.
We're interested in the probability that the random variable, capital W, takes a value equal to little w or values close to it.
So this corresponds to the event, which is this particular line on the two-dimensional space.
So we need to find the sort of odd probabilities along that line.
But since the setting is continuous, we will not add probabilities.
We're going to integrate.
And for any typical point in this picture, the probability of obtaining an outcome in this neighborhood is the-- has something to do with the density of that particular x and the density of the particular y that would compliment x, in order to form a sum of w. So this integral that we have here is really an integral over this particular line.
OK, so I'm going to skip the formal derivation of this result.
There's a couple of derivations in the text.
And the one which is outlined here is yet a third derivation.
But the easiest way to make sense of this formula is to consider what happens in the discrete case.
So for the rest of the lecture we're going to consider a few extra, more miscellaneous topics, a few remarks, and a few more definitions.
So let's change-- flip a page and consider the next mini topic.
There's not going to be anything deep here, but just something that's worth being familiar with.
If you have two independent, normal random variables with certain parameters, the question is, what does the joined PDF look like?
So if they're independent, by definition the joint PDF is the product of the individual PDF's.
And the PDF's each one of them involves an exponential of something.
The product of two exponentials is the exponential of the sum.
So you just add the exponents.
So this is the formula for the joint PDF.
Now, you look at that formula and you ask, what does it look like?
OK, you can understand it, a function of two variables by thinking about the contours of this function.
Look at the points at which the function takes a constant value.
Where is it?
When is it constant?
What's the shape of the set of points where this is a constant?
So consider all x's and y's for which this expression here is a constant, that this expression here is a constant.
What kind of shape is this?
This is an ellipse.
And it's an ellipse that's centered at-- it's centered at mu x, mu y.
These are the means of the two random variables.
If those sigmas were equal, that ellipse would be actually a circle.
And you would get contours of this kind.
But if, on the other hand, the sigmas are different, you're going to get an ellipse that has contours of this kind.
So if my contours are of this kind, that corresponds to what?
Sigma x being bigger than sigma y or vice versa.
OK, contours of this kind basically tell you that X is more likely to be spread out than Y.
So the range of possible x's is bigger.
And X out here is as likely as a Y up there.
So big X's have roughly the same probability as certain smaller y's.
So in a picture of this kind, the variance of X is going to be bigger than the variance of Y.
So depending on how these variances compare with each other, that's going to determine the shape of the ellipse.
If the variance of Y we're bigger, then your ellipse would be the other way.
It would be elongated in the other dimension.
Just visualize it a little more.
Let me throw at you a particular picture.
This is one-- this is a picture of one special case.
Here, I think, the variances are equal.
That's the kind of shape that you get.
It looks like a two-dimensional bell.
So remember, for a normal random variables, for a single random variable you get a PDF that's bell shaped.
That's just a bell-shaped curve.
In the two-dimensional case, we get the joint PDF, which is bell shaped again.
And now it looks more like a real bell, the way it would be laid out in ordinary space.
And if you look at the contours of this function, the places where the function is equal, the typcial contour would have this shape here.
And it would be an ellipse.
And in this case, actually, it will be more like a circle.
So these would be the different contours for different-- so the contours are places where the joint PDF is a constant.
When you change the value of that constant, you get the different contours.
And the PDF is, of course, centered around the mean of the two random variables.
So in this particular case, since the bell is centered around the (0, 0) vector, this is a plot of a bivariate normal with 0 means.
OK, there's-- bivariate normals are also interesting when your bell is oriented differently in space.
We talked about ellipses that are this way, ellipses that are this way.
You could imagine also bells that you take them, you squash them somehow, so that they become narrow in one dimension and then maybe rotate them.
So if you had-- we're not going to go into this subject, but if you had a joint pdf whose contours were like this, what would that correspond to?
Would your x's and y's be independent?
No.
This would indicate that there's a relation between the x's and the y's.
That is, when you have bigger x's, you would expect to also get bigger y's.
So it would be a case of dependent normals.
And we're coming back to this point in a second.
Before we get to that point in a second that has to do with the dependencies between the random variables, let's just do another digression.
If we have our two normals that are independent, as we discussed here, we can go and apply the formula, the convolution formula that we were just discussing.
Suppose you want to find the distribution of the sum of these two independent normals.
How do you do this?
There is a closed-form formula for the density of the sum, which is this one.
We do have formulas for the density of X and the density of Y, because both of them are normal, random variables.
So you need to calculate this particular integral here.
It's an integral with respect to x.
And you have to calculate this integral for any given value of w. So this is an exercise in integration, which is not very difficult.
And it turns out that after you do everything, you end up with an answer that has this form.
And you look at that, and you suddenly recognize, oh, this is normal.
And conclusion from this exercise, once it's done, is that the sum of two independent normal random variables is also normal.
Now, the mean of W is, of course, going to be equal to the sum of the means of X and Y.
In this case, in this formula I took the means to be 0.
So the mean of W is also going to be 0.
In the more general case, the mean of W is going to be just the sum of the two means.
The variance of W is always the sum of the variances of X and Y, since we have independent random variables.
So there's no surprise here.
The main surprise in this calculation is this fact here, that the sum of independent normal random variables is normal.
I had mentioned this fact in a previous lecture.
Here what we're doing is to basically outline the argument that justifies this particular fact.
It's an exercise in integration, where you realize that when you convolve two normal curves, you also get back a normal one once more.
So now, let's return to the comment I was making here, that if you have a contour plot that has, let's say, a shape of this kind, this indicates some kind of dependence between your two random variables.
So instead of a contour plot, let me throw in here a scattered diagram.
What does this scattered diagram correspond to?
Suppose you have a discrete distribution, and each one of the points in this diagram has positive probability.
When you look at this diagram, what would you say?
I would say that when y is big then x also tends to be larger.
So bigger x's are sort of associated with bigger y's in some average, statistical sense.
Whereas, if you have a picture of this kind, it tells you in association that the positive y's tend to be associated with negative x's most of the time.
Negative y's tend to be associated mostly with positive x's.
So here there's a relation that when one variable is large, the other one is also expected to be large.
Here there's a relation of the opposite kind.
How can we capture this relation between two random variables?
The way we capture it is by defining this concept called the covariance, that looks at the relation of was X bigger than usual?
That's the question, whether this is positive.
And how does this relate to the answer-- to the question, was Y bigger than usual?
We're asking-- by calculating this quantity, we're sort of asking the question, is there a systematic relation between having a big X with having a big Y?
OK , to understand more precisely what this does, let's suppose that the random variable has 0 means, So that we get rid of this-- get rid of some clutter.
So the covariance is defined just as this product.
What does this do?
If positive x's tends to go together with positive y's, and negative x's tend to go together with negative y's, this product will always be positive.
And the covariance will end up being positive.
In particular, if you sit down with a scattered diagram and you do the calculations, you'll find that the covariance of X and Y in this diagram would be positive, because here, most of the time, X times Y is positive.
There's going to be a few negative terms, but there are fewer than the positive ones.
So this is a case of a positive covariance.
It indicates a positive relation between the two random variables.
When one is big, the other also tends to be big.
This is the opposite situation.
Here, when one variable-- here, most of the action happens in this quadrant and that quadrant, which means that X times Y, most of the time, is negative.
You get a few positive contributions, but there are few.
When you add things up, the negative terms dominate.
And in this case we have covariance of X and Y being negative.
So a positive covariance indicates a sort of systematic relation, that there's a positive association between the two random variables.
When one is large, the other also tends to be large.
Negative covariance is sort of the opposite.
When one tends to be large, the other variable tends to be small.
OK, so what else is there to say about the covariance?
One observation to make is the following.
What's the covariance of X with X itself?
If you plug in X here, you see that what we have is expected value of X minus expected of X squared.
And that's just the definition of the variance of a random variable.
So that's one fact to keep in mind.
We had a shortcut formula for calculating variances.
There's a similar shortcut formula for calculating covariances.
In particular, we can calculate covariances in this particular way.
That's just the convenient way of doing it whenever you need to calculate it.
And finally, covariances are very useful when you want to calculate the variance of a sum of random variables.
We know that if two random variables are independent, the variance of the sum is the sum of the variances.
When the random variables are dependent, this is no longer true, and we need to supplement the formula a little bit.
And there's a typo on the slides that you have in your hands.
That term of 2 shouldn't be there.
And let's see where that formula comes from.
Let's suppose that our random variables are independent of -- not independent -- our random variables have 0 means.
And we want to calculate the variance.
So the variance is going to be expected value of (X1 plus Xn) squared.
What you do is you expand the square.
And you get the expected value of the sum of the Xi squared.
And then you get all the cross terms.
OK. And so now, here, let's assume for simplicity that we have 0 means.
The expected value of this is the sum of the expected values of the X squared terms.
And that gives us the variance.
And then we have all the possible cross terms.
And each one of the possible cross terms is the expected value of Xi times Xj.
This is just the covariance.
So if you can calculate all the variances and the covariances, then you're able to calculate also the variance of a sum of random variables.
Now, if two random variables are independent, then you look at this expression.
Because of independence, expected value of the product is going to be the product of the expected values.
And the expected value of just this term is always equal to 0.
You're expected deviation from the mean is just 0.
So the covariance will turn out to be 0.
So independent random variables lead to 0 covariances, although the opposite fact is not necessarily true.
So covariances give you some indication of the relation between two random variables.
Something that's not so convenient conceptually about covariances is that it has the wrong units.
That's the same comment that we had made regarding variances.
And with variances we got out of that issue by considering the standard deviation, which has the correct units.
So with the same reasoning, we want to have a concept that captures the relation between two random variables and, in some sense, that doesn't have to do with the units that we're dealing.
We want to have a dimensionless quantity.
That tells us how strongly two random variables are related to each other.
So instead of considering the covariance of just X with Y, we take our random variables and standardize them by dividing them by their individual standard deviations and take the expectation of this.
So what we end up doing is the covariance of X and Y, which has units that are the units of X times the units of Y.
But divide with a standard deviation, so that we get a quantity that doesn't have units.
This quantity, we call it the correlation coefficient.
And it's a very useful quantity, a very useful measure of the strength of association between two random variables.
It's very informative, because it falls always between -1 and +1.
This is an algebraic exercise that you're going to see in recitation.
And the way that you interpret it is as follows.
If the two random variables are independent, the covariance is going to be 0.
The correlation coefficient is going to be 0.
So 0 correlation coefficient basically indicates a lack of a systematic relation between the two random variables.
On the other hand, when rho is large, either close to 1 or close to -1, this is an indication of a strong association between the two random variables.
And the extreme case is when rho takes an extreme value.
When rho has a magnitude equal to 1, it's as big as it can be.
In that case, the two random variables are very strongly related.
How strongly?
Well, if you know one random variable, if you know the value of y, you can recover the value of x and conversely.
So the case of a complete correlation is the case where one random variable is a linear function of the other random variable.
In terms of a scatter plot, this would mean that there's a certain line and that the only possible (x,y) pairs that can happen would lie on that line.
So if all the possible (x,y) pairs lie on this line, then you have this relation, and the correlation coefficient is equal to 1.
A case where the correlation coefficient is close to 1 would be a scatter plot like this, where the x's and y's are quite strongly aligned with each other, maybe not exactly, but fairly strongly.
All right, so you're going to hear a little more about correlation coefficients and covariances in recitation tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to finish with the core material of this class.
That is the material that has to do with probability theory in general.
And then for the rest of the semester we're going to look at some special types of models, talk about inference.
Well, there's also going to be a small module of core material coming later.
But today we're basically finishing chapter four.
And what we're going to do is we're going to look at a somewhat familiar concept, the concept of the conditional expectation.
But we're going to look at it from a slightly different angle, from a slightly more sophisticated angle.
And together with the conditional expectation we will also talk about conditional variances.
It's something that we're going to denote this way.
And we're going to see what they are, and there are some subtle concepts that are involved here.
And we're going to apply some of the tools we're going to develop to deal with a special type of situation in which we're adding random variables.
But we're adding a random number of random variables.
OK, so let's start talking about conditional expectations.
I guess you know what they are.
Suppose we are in the discrete the world.
xy, or discrete random variables.
We defined the conditional expectation of x given that I told you the value of the random variable y.
And the way we define it is the same way as an ordinary expectation, except that we're using the conditional PMF.
So we're using the probabilities that apply to the new universe where we are told the value of the random variable y.
So this is still a familiar concept so far.
If we're dealing with the continuous random variable x the formula is the same, except that here we have an integral, and we have to use the conditional density function of x.
Now what I'm going to do, I want to introduce it gently through the example that we talked about last time.
So last time we talked about having a stick that has a certain length.
And we take that stick, and we break it at some point that we choose uniformly at random.
And let's denote why the place where we chose to break it.
Having chosen y, then we're left with a piece of the stick.
And I'm going to choose a place to break it once more uniformly at random between 0 and y.
So this is the second place at which we are going to break it, and we call that place x. OK, so what's the conditional expectation of x if I tell you the value of y?
I tell you that capital Y happens to take a specific numerical value.
So this capital Y is now a specific numerical value, x is chosen uniformly over this range.
So the expected value of x is going to be half of this range between 0 and y.
So the conditional expectation is little y over 2.
The important thing to realize here is that this quantity is a number.
I told you that the random variable took a certain numerical value, let's say 3.5.
And then you tell me given that the random variable took the numerical value 3.5 the expected value of x is 1.75.
So this is an equality between numbers.
On the other hand, before you do the experiment you don't know what y is going to turn out to be.
So this little y is the numerical value that has been observed when you start doing the experiments and you observe the value of capital Y.
So in some sense this quantity is not known ahead of time, it is random itself.
So maybe we can start thinking of it as a random variable.
So to put it differently, before we do the experiment I ask you what's the expected value of x given y?
You're going to answer me well I don't know, it depends on what y is going to turn out to be.
So the expected value of x given y itself can be viewed as a random variable, because it depends on the random variable capital Y.
So hidden here there's some kind of statement about random variables instead of numbers.
And that statement about random variables, we write it this way.
By thinking of the expected value, the conditional expectation, as a random variable instead of a number.
It's a random variable when we do not specify a specific number, but we think of it as an abstract object.
The expected value of x given the random variable y is the random variable y over 2 no matter what capital Y turns out to be.
So we turn and take a statement that deals with equality of two numbers, and we make it a statement that's an equality between two random variables.
OK so this is clearly a random variable because capital Y is random.
What exactly is this object?
I didn't yet define it for you formally.
So let's now give the formal definition of this object that's going to be denoted this way.
The conditional expectation of x given the random variable y is a random variable.
Which random variable is it?
It's the random variable that takes this specific numerical value whenever capital Y happens to take the specific numerical value little y.
In particular, this is a random variable, which is a function of the random variable capital Y.
In this instance, it's given by a simple formula in terms of capital Y.
In other situations it might be a more complicated formula.
So again, to summarize, it's a random.
The conditional expectation can be thought of as a random variable instead of something that's just a number.
So in any specific context when you're given the value of capital Y the conditional expectation becomes a number.
This is the realized value of this random variable.
But before the experiment starts, before you know what capital Y is going to be, all that you can say is that the conditional expectation is going to be 1/2 of whatever capital Y turns out to be.
This is a pretty subtle concept, it's an abstraction, but it's a useful abstraction.
And we're going to see today how to use it.
All right, I have made the point that the conditional expectation, the random variable that takes these numerical values is a random variable.
If it is a random variable this means that it has an expectation of its own.
So let's start thinking what the expectation of the conditional expectation is going to turn out to be.
OK, so the conditional expectation is a random variable, and in general it's some function of the random variable y that we are observing.
In terms of numerical values if capital Y happens to take a specific numerical value then the conditional expectation also takes a specific numerical value, and we use the same function to evaluate it.
The difference here is that this is an equality of random variables, this is an equality between numbers.
Now if we want to calculate the expected value of the conditional expectation we're basically talking about the expected value of a function of a random variable.
And we know how to calculate expected values of a function.
If we are in the discrete case, for example, this would be a sum over all y's of the function who's expected value we're taking times the probability that y takes on a specific numerical value.
OK, but let's remember what g is.
So g is the numerical value of the conditional expectation of x with y.
And now when you see this expression you recognize it.
This is the expression that we get in the total expectation theorem.
Did I miss something?
Yes, in the total expectation theorem to find the expected value of x, we divide the world into different scenarios depending on what y happens.
We calculate the expectation in each one of the possible worlds, and we take the weighted average.
So this is a formula that you have seen before, and you recognize that this is the expected value of x.
So this is a longer, more detailed derivation of what I had written up here, but the important thing to keep in mind is the moral of the story, the punchline.
The expected value of the conditional expectation is the expectation itself.
So this is just our total expectation theorem, but written in more abstract notation.
And it comes handy to have this more abstract notation, as as we're going to see in a while.
OK, we can apply this to our stick example.
If we want to find the expected value of x how much of the stick is left at the end?
We can calculate it using this law of iterated expectations.
It's the expected value of the conditional expectation.
We know that the conditional expectation is y over 2.
So expected value of y is l over 2, because y is uniform so we get l over 4.
So this gives us the same answer that we derived last time in a rather long way.
All right, now that we have mastered conditional expectations, let's raise the bar a little more and talk about conditional variances.
So the conditional expectation is the mean value, or the expected value, in a conditional universe where you're told the value of y.
In that same conditional universe you can talk about the conditional distribution of x, which has a mean-- the conditional expectation-- but the conditional distribution of x also has a variance.
So we can talk about the variance of x in that conditional universe.
The conditional variance as a number is the natural thing.
It's the variance of x, except that all the calculations are done in the conditional universe.
In the conditional universe the expected value of x is the conditional expectation.
This is the distance from the mean in the conditional universe squared.
And we take the average value of the squared distance, but calculate it again using the probabilities that apply in the conditional universe.
This is an equality between numbers.
I tell you the value of y, once you know that value for y you can go ahead and plot the conditional distribution of x.
And for that conditional distribution you can calculate the number which is the variance of x in that conditional universe.
So now let's repeat the mental gymnastics from the previous slide, and abstract things, and define a random variable-- the conditional variance.
And it's going to be a random variable because we leave the numerical value of capital Y unspecified.
So ahead of time we don't know what capital Y is going to be, and because of that we don't know ahead of time what the conditional variance is going to be.
So before the experiment starts if I ask you what's the conditional variance of x?
You're going to tell me well I don't know, It depends on what y is going to turn out to be.
It's going to be something that depends on y.
So it's a random variable, which is a function of y.
So more precisely, the conditional variance when written in this notation just with capital letters, is a random variable.
It's a random variable whose value is completely determined once you learned the value of capital Y.
And it takes a specific numerical value.
If capital Y happens to get a realization that's a specific number, then the variance also becomes a specific number.
And it's just a conditional variance of y over x in that universe.
All right, OK, so let's continue what we did in the previous slide.
We had the law of iterated expectations.
That told us that expected value of a conditional expectation is the unconditional expectation.
Is there a similar rule that might apply in this context?
So you might guess that the variance of x could be found by taking the expected value of the conditional variance.
It turns out that this is not true.
There is a formula for the variance in terms of conditional quantities.
But the formula is a little more complicated.
If involves two terms instead of one.
So we're going to go quickly through the derivation of this formula.
And then, through examples we'll try to get some interpretation of what the different terms here correspond to.
All right, so let's try to prove this formula.
And the proof is sort of a useful exercise to make sure you understand all the symbols that are involved in here.
So the proof is not difficult, it's 4 and 1/2 lines of algebra, of just writing down formulas.
But the challenge is to make sure that at each point you understand what each one of the objects is.
So we go into formula for the variance affects.
We know in general that the variance of x has this nice expression that we often use to calculate it.
The expected value of the squared of the random variable minus the mean squared.
This formula, for the variances, of course it should apply to conditional universes.
I mean it's a general formula about variances.
If we put ourselves in a conditional universe where the random variable y is given to us the same math should work.
So we should have a similar formula for the conditional variances.
It's just the same formula, but applied to the conditional universe.
The variance of x in the conditional universe is the expected value of x squared-- in the conditional universe-- minus the mean of x-- in the conditional universe-- squared.
So this formula looks fine.
Now let's take expected values of both sides.
Remember the conditional variance is a random variable, because its value depends on whatever realization we get for capital Y.
So we can take expectations here.
We get the expected value of the variance.
Then we have the expected value of a conditional expectation.
Here we use the fact that we discussed before.
The expected value of a conditional expectation is the same as the unconditional expectation.
So this term becomes this.
And finally, here we just have some weird looking random variable, and we take the expected value of it.
All right, now we need to do something about this term.
Let's use the same rule up here to write down this variance.
So variance of an expectation, that's kind of strange, but you remember that the conditional expectation is random, because y is random.
So this thing is a random variable, so this thing has a variance.
What is the variance of this thing?
It's the expected value of the thing squared minus the square of the expected value of the thing.
Now what's the expected value of that thing?
By the law of iterated expectations, once more, the expected value of this thing is the unconditional expectation.
And that's why here I put the unconditional expectation.
So I'm using again this general rule about how to calculate variances, and I'm applying it to calculate the variance of the conditional expectation.
And now you notice that if you add these two expressions c and d we get this plus that, which is this.
It's equal to-- these two terms cancel, we're left with this minus that, which is the variance of x.
And that's the end of the proof.
This one of those proofs that do not convey any intuition.
This, as I said, it's a useful proof to go through just to make sure you understand the symbols.
It starts to get pretty confusing, and a little bit on the abstract side.
So it's good to understand what's going on.
Now there is intuition behind this formula, some of which is better left for later in the class when we talk about inference.
The idea is that the conditional expectation you can interpret it as an estimate of the random variable that you are trying to-- an estimate of x based on measurements of y, you can think of these variances as having something to do with an estimation error.
And once you start thinking in those terms an interpretation will come about.
But again as I said this is better left for when we start talking about inference.
Nevertheless, we're going to get some intuition about all these formulas by considering a baby example where we're going to apply the law of iterated expectations, and the law of total variance.
So the baby example is that we do this beautiful experiment of giving a quiz to a class consisting of many sections.
And we're interested in two random variables.
So we have a number of students, and they're all allocated to sections.
The experiment is that I pick a student at random, and I look at two random variables.
One is the quiz score of the randomly selected student, and the other random variable is the section number of the student that I have selected.
We're given some statistics about the two sections.
Section one has 10 students, section two has 20 students.
The quiz average in section one was 90.
Quiz average in section two was 60.
What's the expected value of x?
What's the expected quiz score if I pick a student at random?
Well, each student has the same probability of being selected.
I'm making that assumption out of the 30 students.
I need to add the quiz scores of all of the students.
So I need to add the quiz scores in section one, which is 90 times 10.
I need to add the quiz scores in that section, which is 60 times 20.
And we find that the overall average was 70.
So this is the usual unconditional expectation.
Let's look at the conditional expectation, and let's look at the elementary version where we're talking about numerical values.
If I tell you that the randomly selected student was in section one what's the expected value of the quiz score of that student?
Well, given this information, we're picking a random student uniformly from that section in which the average was 90.
The expected value of the score of that student is going to be 90.
So given the specific value of y, the specific section, the conditional expectation or the expected value of the quiz score is a specific number, the number 90.
Similarly for the second section the expected value is 60, that's the average score in the second section.
This is the elementary version.
What about the abstract version?
In the abstract version the conditional expectation is a random variable because it depends.
In which section is the student that I picked?
And with probability 1/3, I'm going to pick a student in the first section, in which case the conditional expectation will be 90, and with probability 2/3 I'm going to pick a student in the second section.
And in that case the conditional expectation will take the value of 60.
So this illustrates the idea that the conditional expectation is a random variable.
Depending on what y is going to be, the conditional expectation is going to be one or the other value with certain probabilities.
Now that we have the distribution of the conditional expectation we can calculate the expected value of it.
And the expected value of such a random variable is 1/3 times 90, plus 2/3 times 60, and it comes out to equal 70.
Which miraculously is the same number that we got up there.
So this tells you that you can calculate the overall average in a large class by taking the averages in each one of the sections and weighing each one of the sections according to the number of students that it has.
So this section had 90 students but only 1/3 of the students, so it gets a weight of 1/3.
So the law of iterated expectations, once more, is nothing too complicated.
It's just that you can calculate overall class average by looking at the section averages and combine them.
Now since the conditional expectation is a random variable, of course it has a variance of it's own.
So let's calculate the variance.
How do we calculate variances?
We look at all the possible numerical values of this random variable, which are 90 and 60.
We look at the difference of those possible numerical values from the mean of this random variable, and the mean of that random variable, we found that's it's 70.
And then we weight the different possible numerical values according to their probabilities.
So with probability 1/3 the conditional expectation is 90, which is 20 away from the mean.
And we get this squared distance.
With probability 2/3 the conditional expectation is 60, which is 10 away from the mean, has this squared distance and gets weighed by 2/3, which is the probability of 60.
So you do the numbers, and you get the value for the variance equal to 200.
All right, so now we want to move towards using that more complicated formula involving the conditional variances.
OK, suppose someone goes and calculates the variance of the quiz scores inside each one of the sections.
So someone gives us these two pieces of information.
In section one we take the differences from the mean in that section, and let's say that the various turns out to be a number equal to 10 similarly in the second section.
So these are the variances of the quiz scores inside individual sections.
The variance in one conditional universe, the variance in the other conditional universe.
So if I pick a student in section one and I don't tell you anything more about the student, what's the variance of the random score of that student?
The variance is 10.
I know why, but I don't know the student.
So the score is still a random variable in that universe.
It has a variance, and that's the variance.
Similarly, in the other universe, the variance of the quiz scores is this number, 20.
Once more, this is an equality between numbers.
I have fixed the specific value of y.
So I put myself in a specific universe, I can calculate the variance in that specific universe.
If I don't specify a numerical value for capital Y, and say I don't know what Y is going to be, it's going to be random.
Then what kind of section variance I'm going to get itself will be random.
With probability 1/3, I pick a student in the first section in which case the conditional variance given what I have picked is going to be 10.
Or with probability 2/3 I pick y equal to 2, and I place myself in that universe.
And in that universe the conditional variance is 20.
So you see again from here that the conditional variance is a random variable that takes different values with certain probabilities.
And which value it takes depends on the realization of the random variable capital Y.
So this happens if capital Y is one, this happens if capital Y is equal to 2.
Once you have something of this form-- a random variable that takes values with certain probabilities-- then you can certainly calculate the expected value of that random variable.
Don't get intimidated by the fact that this random variable, it's something that's described by a string of eight symbols, or seven, instead of just a single letter.
Think of this whole string of symbols there as just being a random variable.
You could call it z for example, use one letter.
So z is a random variable that takes these two values with these corresponding probabilities.
So we can talk about the expected value of Z, which is going to be 1/3 times 10, 2/3 times 20, and we get a certain number from here.
And now we have all the pieces to calculate the overall variance of x.
The formula from the previous slide tells us this.
Do we have all the pieces?
The expected value of the variance, we just calculated it.
The variance of the expected value, this was the last calculation in the previous slide.
We did get a number for it, it was 200.
You add the two, you find the total variance.
Now the useful piece of this exercise is to try to interpret these two numbers, and see what they mean.
The variance of x given y for a specific y is the variance inside section one.
This is the variance inside section two.
The expected value is some kind of average of the variances inside individual sections.
So this term tells us something about the variability of this course, how widely spread they are within individual sections.
So we have three sections, and this course happens to be-- OK, let's say the sections are really different.
So here you have undergraduates and here you have post-doctoral students.
And these are the quiz scores, that's section one, section two, section three.
Here's the mean of the first section.
And the variance has something to do with the spread.
The variance in the second section has something to do with the spread, similarly with the third spread.
And the expected value of the conditional variances is some weighted average of the three variances that we get from individual sections.
So variability within sections definitely contributes something to the overall variability of this course.
But if you ask me about the variability over the entire class there's a second effect.
That has to do with the fact that different sections are very different from each other.
That these courses here are far away from those scores.
And this term is the one that does the job.
This one looks at the expected values inside each section, and these expected values are this, this, and that.
And asks a question how widely spread are they?
It asks how different from each other are the means inside individual sections?
And in this picture it would be a large number because the difference section means are quite different.
So the story that this formula is telling us is that the overall variability of the quiz scores consists of two factors that can be quantified and added.
One factor is how much variability is there inside individual sections?
And the other factor is how different are the sections from each other?
Both effects contribute to the overall variability of this course.
Let's continue with just one more numerical example.
Just to get the hang of doing these kinds of calculations, and apply this formula to do a divide and conquer calculation of the variance of a random variable.
Just for variety now we're going to take a continuous random variable.
Somebody gives you a PDF if this form, and they ask you for the variance.
And you say oh that's too complicated, I don't want to do integrals.
Can I divide and conquer?
And you say OK, let me do the following trick.
Let me define a random variable, y.
Which takes the value 1 if x falls in here, and takes the value 2 if x falls in the second interval.
And let me try to work in the conditional world where things might be easier, and then add things up to get the overall variance.
So I have defined y this particular way.
In this example y becomes a function of x. y is completely determined by x.
And I'm going to calculate the overall variance by trying to calculate all of the terms that are involved here.
So let's start calculating.
First observation is that this event has probability 1/3, and this event has probability 2/3.
The expected value of x given that we are in this universe is 1/2, because we have a uniform distribution from 0 to 1.
Here we have a uniform distribution from 1 to 2, so the conditional expectation of x in that universe is 3/2.
How about conditional variances?
In the world who are y is equal to 1 x has a uniform distribution on a unit interval.
What's the variance of x?
By now you've probably seen that formula, it's 1 over 12.
1 over 12 is the variance of a uniform distribution over a unit interval.
When y is equal to 2 the variance is again 1 over 12.
Because in this instance again x has a uniform distribution over an interval of unit length.
What's the overall expected value of x?
The way you find the overall expected value is to consider the different numerical values of the conditional expectation.
And weigh them according to their probabilities.
So with probability 1/3 the conditional expectation is 1/2.
And with probability 2/3 the conditional expectation is 3 over 2.
And this turns out to be 7 over 6.
So this is the advance work we need to do, now let's calculate a few things here.
What's the variance of the expected value of x given y?
Expected value of x given y is a random variable that takes these two values with these probabilities.
So to find the variance we consider the probability that the expected value takes the numerical value of 1/2 minus the mean of the conditional expectation.
What's the mean of the conditional expectation?
It's the unconditional expectation.
So it's 7 over 6.
We just did that calculation.
So I'm putting here that number, 7 over 6 squared.
And then there's a second term with probability 2/3, the conditional expectation takes this value of 3 over 2, which is so much away from the mean, and we get this contribution.
So this way we have calculated the variance of the conditional expectation, this is this term.
What is this?
Any guesses what this number is?
It's 1 over 12, why?
The conditional variance just happened in this example to be 1 over 12 no matter what.
So the conditional variance is a deterministic random variable that takes a constant value.
So the expected value of this random variable is just 1 over 12.
So we got the two pieces that we need, and so we do have the overall variance of the random variable x.
So this was just an academic example in order to get the hang of how to manipulate various quantities.
Now let's use what we have learned and the tools that we have to do something a little more interesting.
OK, so by now you're all in love with probabilities.
So over the weekend you're going to bookstores to buy probability books.
So you're going to visit a random number bookstores, and at each one of the bookstores you're going to spend a random amount of money.
So let n be the number of stores that you are visiting.
So n is an integer-- non-negative random variable-- and perhaps you know the distribution of that random variable.
Each time that you walk into a store your mind is clear from whatever you did before, and you just buy a random number of books that has nothing to do with how many books you bought earlier on the day.
It has nothing to do with how many stores you are visiting, and so on.
So each time you enter as a brand new person, and buy a random number of books, and spend a random amount of money.
So what I'm saying, more precisely, is that I'm making the following assumptions.
That for each store i-- if you end up visiting the i-th store-- the amount of money that you spend is a random variable that has a certain distribution.
That distribution is the same for each store, and the xi's from store to store are independent from each other.
And furthermore, the xi's are all independent of n. So how much I'm spending at the store-- once I get in-- has nothing to do with how many stores I'm visiting.
So this is the setting that we're going to look at.
y is the total amount of money that you did spend.
It's the sum of how much you spent in the stores, but the index goes up to capital N. And what's the twist here?
It's that we're dealing with the sum of independent random variables except that how many random variables we have is not given to us ahead of time, but it is chosen at random.
So it's a sum of a random number of random variables.
We would like to calculate some quantities that have to do with y, in particular the expected value of y, or the variance of y.
How do we go about it?
OK, we know something about the linearity of expectations.
That expectation of a sum is the sum of the expectations.
But we have used that rule only in the case where it's the sum of a fixed number of random variables.
So expected value of x plus y plus z is expectation of x, plus expectation of y, plus expectation of z.
We know this for a fixed number of random variables.
We don't know it, or how it would work for the case of a random number.
Well, if we know something about the case for fixed random variables let's transport ourselves to a conditional universe where the number of random variables we're summing is fixed.
So let's try to break the problem divide and conquer by conditioning on the different possible values of the number of bookstores that we're visiting.
So let's work in the conditional universe, find the conditional expectation in this universe, and then use our law of iterated expectations to see what happens more generally.
If I told you that I visited exactly little n stores, where little n now is a number, let's say 10.
Then the amount of money you're spending is x1 plus x2 all the way up to x10 given that we visited 10 stores.
So what I have done here is that I've replaced the capital N with little n, and I can do this because I'm now in the conditional universe where I know that capital N is little n. Now little n is fixed.
We have assumed that n is independent from the xi's.
So in this universe of a fixed n this information here doesn't tell me anything new about the values of the x's.
If you're conditioning random variables that are independent from the random variables you are interested in, the conditioning has no effect, and so it can be dropped.
So in this conditional universe where you visit exactly 10 stores the expected amount of money you're spending is the expectation of the amount of money spent in 10 stores, which is the sum of the expected amount of money in each store.
Each one of these is the same number, because the random variables have identical distributions.
So it's n times the expected value of money you spent in a typical store.
This is almost obvious without doing it formally.
If I'm telling you that you're visiting 10 stores, what you expect to spend is 10 times the amount you expect to spend in each store individually.
Now let's take this equality here and rewrite it in our abstract notation, in terms of random variables.
This is an equality between numbers.
Expected value of y given that you visit 10 stores is 10 times this particular number.
Let's translate it into random variables.
In random variable notation, the expected value of money you're spending given the number of stores-- but without telling you a specific number-- is whatever that number of stores turns out to be times the expected value of x.
So this is a random variable that takes this as a numerical value whenever capital N happens to be equal to little n. This is a random variable, which by definition takes this numerical value whenever capital N is equal to little n. So no matter what capital N happens to be what specific value, little n, it takes this is equal to that.
Therefore the value of this random variable is going to be equal to that random variable.
So as random variables, these two random variables are equal to each other.
And now we use the law of iterated expectations.
The law of iterated expectations tells us that the overall expected value of y is the expected value of the conditional expectation.
We have a formula for the conditional expectation.
It's n times expected value of x.
Now the expected value of x is a number.
Expected value of something random times a number is expected value of the random variable times the number itself.
We can take a number outside the expectation.
So expected value of x gets pulled out.
And that's the conclusion, that overall the expected amount of money you're going to spend is equal to how many stores you expect to visit on the average, and how much money you expect to spend on each one on the average.
You might have guessed that this is the answer.
If you expect to visit 10 stores, and you expect to spend $100 on each store, then yes, you expect to spend $1,000 today.
You're not going to impress your Harvard friends if you tell them that story.
It's one of the cases where reasoning, on the average, does give you the plausible answer.
But you will be able to impress your Harvard friends if you tell them that I can actually calculate the variance of how much I can spend.
And we're going to work by applying this formula that we have, and the difficulty is basically sorting out all those terms here, and what they mean.
So let's start with this term.
So the expected value of y given that you're visiting n stores is n times the expected value of x.
That's what we did in the previous slide.
So this thing is a random variable, it has a variance.
What is the variance?
Is the variance of n times the expected value of x.
Remember expected value of x is a number.
So we're dealing with the variance of n times a number.
What happens when you multiply a random variable by a constant?
The variance becomes the previous variance times the constant squared.
So the variance of this is the variance of n times the square of that constant that we had here.
So this tells us the variance of the expected value of y given n. This is the part of the variability of how much money you're spending, which is attributed to the randomness, or the variability, in the number of stores that you are visiting.
So the interpretation of the two terms is there's randomness in how much you're going to spend, and this is attributed to the randomness in the number of stores together with the randomness inside individual stores.
Well, after I tell you how many stores you're visiting.
So now let's deal with this term-- the variance inside individual stores.
Let's take it slow.
If I tell you that you're visiting exactly little n stores, then y is how much money you spent in those little n stores.
You're dealing with the sum of little n random variables.
What is the variance of the sum of little n random variables?
It's the sum of their variances.
So each store contributes a variance of x, and you're adding over little n stores.
That's the variance of money spent if I tell you the number of stores.
Now let's translate this into random variable notation.
This is a random variable that takes this numerical value whenever capital N is equal to little n. This is a random variable that takes this numerical value whenever capital N is equal to little n. This is equal to that.
Therefore, these two are always equal, no matter what happens to y.
So we have an equality here between random variables.
Now we take expectations of both.
Expected value of the variance is expected value of this.
OK it may look confusing to think of the expected value of the variance here, but the variance of x is a number, not a random variable.
You think of it as a constant.
So its expected value of n times a constant gives us the expected value of n times the constant itself.
So now we got the second term as well, and now we put everything together, this plus that to get an expression for the overall variance of y.
Which again, as I said before, the overall variability in y has to do with the variability of how much you spent inside the typical store.
And the variability in the number of stores that you are visiting.
OK, so this is it for today.
We'll change subjects quite radically from next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So by now you have seen pretty much every possible trick there is in basic probability theory, about how to calculate distributions, and so on.
You have the basic tools to do pretty much anything.
So what's coming after this?
Well, probability is useful for developing the science of inference, and this is a subject to which we're going to come back at the end of the semester.
Another chapter, which is what we will be doing over the next few weeks, is to deal with phenomena that evolve in time.
So so-called random processes or stochastic processes.
So what is this about?
So in the real world, you don't just throw two random variables and go home.
Rather the world goes on.
So you generate the random variable, then you get more random variables, and things evolve in time.
And random processes are supposed to be models that capture the evolution of random phenomena over time.
So that's what we will be doing.
Now when we have evolution in time, mathematically speaking, you can use discrete time or continuous time.
Of course, discrete time is easier.
And that's where we're going to start from.
And we're going to start from the easiest, simplest random process, which is the so-called Bernoulli process, which is nothing but just a sequence of coin flips.
You keep flipping a coin and keep going forever.
That's what the Bernoulli process is.
So in some sense it's something that you have already seen.
But we're going to introduce a few additional ideas here that will be useful and relevant as we go along and we move on to continuous time processes.
So we're going to define the Bernoulli process, talk about some basic properties that the process has, and derive a few formulas, and exploit the special structure that it has to do a few quite interesting things.
By the way, where does the word Bernoulli come from?
Well the Bernoulli's were a family of mathematicians, Swiss mathematicians and scientists around the 1700s.
There were so many of them that actually-- and some of them had the same first name-- historians even have difficulty of figuring out who exactly did what.
But in any case, you can imagine that at the dinner table they were probably flipping coins and doing Bernoulli trials.
So maybe that was their pass-time.
OK.
So what is the Bernoulli process?
The Bernoulli process is nothing but a sequence of independent Bernoulli trials that you can think of as coin flips.
So you can think the result of each trial being heads or tails.
It's a little more convenient maybe to talk about successes and failures instead of heads or tails.
Or if you wish numerical values, to use a 1 for a success and 0 for a failure.
So the model is that each one of these trials has the same probability of success, p. And the other assumption is that these trials are statistically independent of each other.
So what could be some examples of Bernoulli trials?
You buy a lottery ticket every week and you win or lose.
Presumably, these are independent of each other.
And if it's the same kind of lottery, the probability of winning should be the same during every week.
Maybe you want to model the financial markets.
And a crude model could be that on any given day the Dow Jones is going to go up or down with a certain probability.
Well that probability must be somewhere around 0.5, or so.
This is a crude model of financial markets.
You say, probably there is more into them.
Life is not that simple.
But actually it's a pretty reasonable model.
It takes quite a bit of work to come up with more sophisticated models that can do better predictions than just pure heads and tails.
Now more interesting, perhaps to the examples we will be dealing with in this class-- a Bernoulli process is a good model for streams of arrivals of any kind to a facility.
So it could be a bank, and you are sitting at the door of the bank.
And at every second, you check whether a customer came in during that second or not.
Or you can think about arrivals of jobs to a server.
Or any other kind of requests to a service system.
So requests, or jobs, arrive at random times.
You split the time into time slots.
And during each time slot something comes or something does not come.
And for many applications, it's a reasonable assumption to make that arrivals on any given slot are independent of arrivals in any other time slot.
So each time slot can be viewed as a trial, where either something comes or doesn't come.
And different trials are independent of each other.
Now there's two assumptions that we're making here.
One is the independence assumption.
The other is that this number, p, probability of success, is constant.
Now if you think about the bank example, if you stand outside the bank at 9:30 in the morning, you'll see arrivals happening at a certain rate.
If you stand outside the bank at 12:00 noon, probably arrivals are more frequent.
Which means that the given time slot has a higher probability of seeing an arrival around noon time.
This means that the assumption of a constant p is probably not correct in that setting, if you're talking about the whole day.
So the probability of successes or arrivals in the morning is going to be smaller than what it would be at noon.
But if you're talking about a time period, let's say 10:00 to 10:15, probably all slots have the same probability of seeing an arrival and it's a good approximation.
So we're going to stick with the assumption that p is constant, doesn't change with time.
Now that we have our model what do we do with it?
Well, we start talking about the statistical properties that it has.
And here there's two slightly different perspectives of thinking about what a random process is.
The simplest version is to think about the random process as being just a sequence of random variables.
We know what random variables are.
We know what multiple random variables are.
So it's just an experiment that has associated with it a bunch of random variables.
So once you have random variables, what do you do instinctively?
You talk about the distribution of these random variables.
We already specified for the Bernoulli process that each Xi is a Bernoulli random variable, with probability of success equal to p. That specifies the distribution of the random variable X, or Xt, for general time t. Then you can calculate expected values and variances, and so on.
So the expected value is, with probability p, you get a 1.
And with probability 1 - p, you get a 0.
So the expected value is equal to p. And then we have seen before a formula for the variance of the Bernoulli random variable, which is p times 1-p.
So this way we basically now have all the statistical properties of the random variable Xt, and we have those properties for every t. Is this enough of a probabilistic description of a random process?
Well, no.
You need to know how the different random variables relate to each other.
If you're talking about a general random process, you would like to know things.
For example, the joint distribution of X2, with X5, and X7.
For example, that might be something that you're interested in.
And the way you specify it is by giving the joint PMF of these random variables.
And you have to do that for every collection, or any subset, of the random variables you are interested in.
So to have a complete description of a random processes, you need to specify for me all the possible joint distributions.
And once you have all the possible joint distributions, then you can answer, in principle, any questions you might be interested in.
How did we get around this issue for the Bernoulli process?
I didn't give you the joint distributions explicitly.
But I gave them to you implicitly.
And this is because I told you that the different random variables are independent of each other.
So at least for the Bernoulli process, where we make the independence assumption, we know that this is going to be the product of the PMFs.
And since I have told you what the individual PMFs are, this means that you automatically know all the joint PMFs.
And we can go to business based on that.
All right.
So this is one view of what a random process is, just a collection of random variables.
There's another view that's a little more abstract, which is the following.
The entire process is to be thought of as one long experiment.
So we go back to the chapter one view of probabilistic models.
So there must be a sample space involved.
What is the sample space?
If I do my infinite, long experiment of flipping an infinite number of coins, a typical outcome of the experiment would be a sequence of 0's and 1's.
So this could be one possible outcome of the experiment, just an infinite sequence of 0's and 1's.
My sample space is the set of all possible outcomes of this kind.
Here's another possible outcome, and so on.
And essentially we're dealing with a sample space, which is the space of all sequences of 0's and 1's.
And we're making some sort of probabilistic assumption about what may happen in that experiment.
So one particular sequence that we may be interested in is the sequence of obtaining all 1's.
So this is the sequence that gives you 1's forever.
Once you take the point of view that this is our sample space-- its the space of all infinite sequences-- you can start asking questions that have to do with infinite sequences.
Such as the question, what's the probability of obtaining the infinite sequence that consists of all 1's?
So what is this probability?
Let's see how we could calculate it.
So the probability of obtaining all 1's is certainly less than or equal to the probability of obtaining 1's, just in the first 10 tosses.
OK.
This is asking for more things to happen than this.
If this event is true, then this is also true.
Therefore the probability of this is smaller than the probability of that.
This event is contained in that event.
This implies this.
So we have this inequality.
Now what's the probability of obtaining 1's in 10 trials?
This is just p to the 10th because the trials are independent.
Now of course there's no reason why I chose 10 here.
The same argument goes through if I use an arbitrary number, k. And this has to be true for all k. So this probability is less than p to the k, no matter what k I choose.
Therefore, this must be less than or equal to the limit of this, as k goes to infinity.
This is smaller than that for all k's.
Let k go to infinity, take k arbitrarily large, this number is going to become arbitrarily small.
It goes to 0.
And that proves that the probability of an infinite sequence of 1's is equal to 0.
So take limits of both sides.
It's going to be less than or equal to the limit-- I shouldn't take a limit here.
The probability is less than or equal to the limit of p to the k, as k goes to infinity, which is 0.
So this proves in a formal way that the sequence of all 1's has 0 probability.
If you have an infinite number of coin flips, what's the probability that all of the coin flips result in heads?
The probability of this happening is equal to zero.
So this particular sequence has 0 probability.
Of course, I'm assuming here that p is less than 1, strictly less than 1.
Now the interesting thing is that if you look at any other infinite sequence, and you try to calculate the probability of that infinite sequence, you would get a product of (1-p) times 1, 1-p times 1, 1-p, times p times p, times 1-p and so on.
You keep multiplying numbers that are less than 1.
Again, I'm making the assumption that p is between 0 and 1.
So 1-p is less than 1, p is less than 1.
You keep multiplying numbers less than 1.
If you multiply infinitely many such numbers, the infinite product becomes 0.
So any individual sequence in this sample space actually has 0 probability.
And that is a little bit counter-intuitive perhaps.
But the situation is more like the situation where we deal with continuous random variables.
So if you could draw a continuous random variable, every possible outcome has 0 probability.
And that's fine.
But all of the outcomes collectively still have positive probability.
So the situation here is very much similar.
So the space of infinite sequences of 0's and 1's, that sample space is very much like a continuous space.
If you want to push that analogy further, you could think of this as the expansion of a real number.
Or the representation of a real number in binary.
Take a real number, write it down in binary, you are going to get an infinite sequence of 0's and 1's.
So you can think of each possible outcome here essentially as a real number.
So the experiment of doing an infinite number of coin flips is sort of similar to the experiment of picking a real number at random.
When you pick real numbers at random, any particular real number has 0 probability.
So similarly here, any particular infinite sequence has 0 probability.
So if we were to push that analogy further, there would be a few interesting things we could do.
But we will not push it further.
This is just to give you an indication that things can get pretty subtle and interesting once you start talking about random processes that involve forever, over the infinite time horizon.
So things get interesting even in this context of the simple Bernoulli process.
Just to give you a preview of what's coming further, today we're going to talk just about the Bernoulli process.
And you should make sure before the next lecture-- I guess between the exam and the next lecture-- to understand everything we do today.
Because next time we're going to do everything once more, but in continuous time.
And in continuous time, things become more subtle and a little more difficult.
But we are going to build on what we understand for the discrete time case.
Now both the Bernoulli process and its continuous time analog has a property that we call memorylessness, whatever happened in the past does not affect the future.
Later on in this class we're going to talk about more general random processes, so-called Markov chains, in which there are certain dependences across time.
That is, what has happened in the past will have some bearing on what may happen in the future.
So it's like having coin flips where the outcome of the next coin flip has some dependence on the previous coin flip.
And that gives us a richer class of models.
And once we get there, essentially we will have covered all possible models.
So for random processes that are practically useful and which you can manipulate, Markov chains are a pretty general class of models.
And almost any real world phenomenon that evolves in time can be approximately modeled using Markov chains.
So even though this is a first class in probability, we will get pretty far in that direction.
All right.
So now let's start doing a few calculations and answer some questions about the Bernoulli process.
So again, the best way to think in terms of models that correspond to the Bernoulli process is in terms of arrivals of jobs to a facility.
And there's two types of questions that you can ask.
In a given amount of time, how many jobs arrived?
Or conversely, for a given number of jobs, how much time did it take for them to arrive?
So we're going to deal with these two questions, starting with the first.
For a given amount of time-- that is, for a given number of time periods-- how many arrivals have we had?
How many of those Xi's happen to be 1's?
We fix the number of time slots-- let's say n time slots-- and you measure the number of successes.
Well this is a very familiar random variable.
The number of successes in n independent coin flips-- or in n independent trials-- is a binomial random variable.
So we know its distribution is given by the binomial PMF, and it's just this, for k going from 0 up to n. And we know everything by now about this random variable.
We know its expected value is n times p. And we know the variance, which is n times p, times 1-p.
So there's nothing new here.
That's the easy part.
So now let's look at the opposite kind of question.
Instead of fixing the time and asking how many arrivals, now let us fix the number of arrivals and ask how much time did it take.
And let's start with the time until the first arrival.
So the process starts.
We got our slots.
And we see, perhaps, a sequence of 0's and then at some point we get a 1.
The number of trials it took until we get a 1, we're going to call it T1.
And it's the time of the first arrival.
OK. What is the probability distribution of T1?
What kind of random variable is it?
We've gone through this before.
The event that the first arrival happens at time little t is the event that the first t-1 trials were failures, and the trial number t happens to be a success.
So for the first success to happen at time slot number 5, it means that the first 4 slots had failures and the 5th slot had a success.
So the probability of this happening is the probability of having failures in the first t -1 trials, and having a success at trial number 1.
And this is the formula for t equal 1,2, and so on.
So we know what this distribution is.
It's the so-called geometric distribution.
Let me jump this through this for a minute.
In the past, we did calculate the expected value of the geometric distribution, and it's 1/p.
Which means that if p is small, you expect to take a long time until the first success.
And then there's a formula also for the variance of T1, which we never formally derived in class, but it was in your textbook and it just happens to be this.
All right.
So nothing new until this point.
Now, let's talk about this property, the memorylessness property.
We kind of touched on this property when we discussed-- when we did the derivation in class of the expected value of T1.
Now what is the memoryless property?
It's essentially a consequence of independence.
If I tell you the results of my coin flips up to a certain time, this, because of independence, doesn't give you any information about the coin flips after that time.
So knowing that we had lots of 0's here does not change what I believe about the future coin flips, because the future coin flips are going to be just independent coin flips with a given probability, p, for obtaining tails.
So this is a statement that I made about a specific time.
That is, you do coin flips until 12 o'clock.
And then at 12 o'clock, you start watching.
No matter what happens before 12 o'clock, after 12:00, what you're going to see is just a sequence of independent Bernoulli trials with the same probability, p. Whatever happened in the past is irrelevant.
Now instead of talking about the fixed time at which you start watching, let's think about a situation where your sister sits in the next room, flips the coins until she observes the first success, and then calls you inside.
And you start watching after this time.
What are you're going to see?
Well, you're going to see a coin flip with probability p of success.
You're going to see another trial that has probability p as a success, and these are all independent of each other.
So what you're going to see starting at that time is going to be just a sequence of independent Bernoulli trials, as if the process was starting at this time.
How long it took for the first success to occur doesn't have any bearing on what is going to happen afterwards.
What happens afterwards is still a sequence of independent coin flips.
And this story is actually even more general.
So your sister watches the coin flips and at some point tells you, oh, something really interesting is happening here.
I got this string of a hundred 1's in a row.
Come and watch.
Now when you go in there and you start watching, do you expect to see something unusual?
There were unusual things that happened before you were called in.
Does this means that you're going to see unusual things afterwards?
No.
Afterwards, what you're going to see is, again, just a sequence of independent coin flips.
The fact that some strange things happened before doesn't have any bearing as to what is going to happen in the future.
So if the roulettes in the casino are properly made, the fact that there were 3 reds in a row doesn't affect the odds of whether in the next roll it's going to be a red or a black.
So whatever happens in the past-- no matter how unusual it is-- at the time when you're called in, what's going to happen in the future is going to be just independent Bernoulli trials, with the same probability, p. The only case where this story changes is if your sister has a little bit of foresight.
So your sister can look ahead into the future and knows that the next 10 coin flips will be heads, and calls you before those 10 flips will happen.
If she calls you in, then what are you going to see?
You're not going to see independent Bernoulli trials, since she has psychic powers and she knows that the next ones would be 1's.
She called you in and you will see a sequence of 1's.
So it's no more independent Bernoulli trials.
So what's the subtle difference here?
The future is independent from the past, provided that the time that you are called and asked to start watching is determined by someone who doesn't have any foresight, who cannot see the future.
If you are called in, just on the basis of what has happened so far, then you don't have any information about the future.
And one special case is the picture here.
You have your coin flips.
Once you see a one that happens, once you see a success, you are called in.
You are called in on the basis of what happened in the past, but without any foresight.
OK. And this subtle distinction is what's going to make our next example interesting and subtle.
So here's the question.
You buy a lottery ticket every day, so we have a Bernoulli process that's running in time.
And you're interested in the length of the first string of losing days.
What does that mean?
So suppose that a typical sequence of events could be this one.
So what are we discussing here?
We're looking at the first string of losing days, where losing days means 0's.
So the string of losing days is this string here.
Let's call the length of that string, L. We're interested in the random variable, which is the length of this interval.
What kind of random variable is it?
OK.
Here's one possible way you might think about the problem.
OK.
Starting from this time, and looking until this time here, what are we looking at?
We're looking at the time, starting from here, until the first success.
So the past doesn't matter.
Starting from here we have coin flips until the first success.
The time until the first success in a Bernoulli process-- we just discussed that it's a geometric random variable.
So your first conjecture would be that this random variable here, which is 1 longer than the one we are interested in, that perhaps is a geometric random variable.
And if this were so, then you could say that the random variable, L, is a geometric, minus 1.
Can that be the correct answer?
A geometric random variable, what values does it take?
It takes values 1, 2, 3, and so on.
1 minus a geometric would take values from 0, 1, 2, and so on.
Can the random variable L be 0?
No.
The random variable L is the length of a string of losing days.
So the shortest that L could be, would be just 1.
If you get just one losing day and then you start winning, L would be equal to 1.
So L cannot be 0 by definition, which means that L + 1 cannot be 1, by definition.
But if L +1 were geometric, it could be equal to 1.
Therefore this random variable, L + 1, is not a geometric.
OK. Why is it not geometric?
I started watching at this time.
From this time until the first success, that should be a geometric random variable.
Where's the catch?
If I'm asked to start watching at this time, it's because my sister knows that the next one was a failure.
This is the time where the string of failures starts.
In order to know that they should start watching here, it's the same as if I'm told that the next one is a failure.
So to be asked to start watching at this time requires that someone looked in the future.
And in that case, it's no longer true that these will be independent Bernoulli trials.
In fact, they're not.
If you start watching here, you're certain that the next one is a failure.
The next one is not an independent Bernoulli trial.
That's why the argument that would claim that this L + 1 is geometric would be incorrect.
So if this is not the correct answer, which is the correct answer?
The correct answer goes as follows.
Your sister is watching.
Your sister sees the first failure, and then tells you, OK, the failures-- or losing days-- have started.
Come in and watch.
So you start to watching at this time.
And you start watching until the first success comes.
This will be a geometric random variable.
So from here to here, this will be geometric.
So things happen.
You are asked to start watching.
After you start watching, the future is just a sequence of independent Bernoulli trials.
And the time until the first failure occurs, this is going to be a geometric random variable with parameter p. And then you notice that the interval of interest is exactly the same as the length of this interval.
This starts one time step later, and ends one time step later.
So conclusion is that L is actually geometric, with parameter p. OK, it looks like I'm missing one slide.
Can I cheat a little from here?
OK.
So now that we dealt with the time until the first arrival, we can start talking about the time until the second arrival, and so on.
How do we define these?
After the first arrival happens, we're going to have a sequence of time slots with no arrivals, and then the next arrival is going to happen.
So we call this time that elapses-- or number of time slots after the first arrival until the next one-- we call it T2.
This is the second inter-arrival time, that is, time between arrivals.
Once this arrival has happened, then we wait and see how many more it takes until the third arrival.
And we call this time here, T3.
We're interested in the time of the k-th arrival, which is going to be just the sum of the first k inter-arrival times.
So for example, let's say Y3 is the time that the third arrival comes.
Y3 is just the sum of T1, plus T2, plus T3.
So we're interested in this random variable, Y3, and it's the sum of inter-arrival times.
To understand what kind of random variable it is, I guess we should understand what kind of random variables these are going to be.
So what kind of random variable is T2?
Your sister is doing her coin flips until a success is observed for the first time.
Based on that information about what has happened so far, you are called into the room.
And you start watching until a success is observed again.
So after you start watching, what you have is just a sequence of independent Bernoulli trials.
So each one of these has probability p of being a success.
The time it's going to take until the first success, this number, T2, is going to be again just another geometric random variable.
It's as if the process just started.
After you are called into the room, you have no foresight, you don't have any information about the future, other than the fact that these are going to be independent Bernoulli trials.
So T2 itself is going to be geometric with the same parameter p. And then you can continue the arguments and argue that T3 is also geometric with the same parameter p. Furthermore, whatever happened, how long it took until you were called in, it doesn't change the statistics about what's going to happen in the future.
So whatever happens in the future is independent from the past.
So T1, T2, and T3 are independent random variables.
So conclusion is that the time until the third arrival is the sum of 3 independent geometric random variables, with the same parameter.
And this is true more generally.
The time until the k-th arrival is going to be the sum of k independent random variables.
So in general, Yk is going to be T1 plus Tk, where the Ti's are geometric, with the same parameter p, and independent.
So now what's more natural than trying to find the distribution of the random variable Yk?
How can we find it?
So I fixed k for you.
Let's say k is 100.
I'm interested in how long it takes until 100 customers arrive.
How can we find the distribution of Yk?
Well one way of doing it is to use this lovely convolution formula.
Take a geometric, convolve it with another geometric, you get something.
Take that something that you got, convolve it with a geometric once more, do this 99 times, and this gives you the distribution of Yk.
So that's definitely doable, and it's extremely tedious.
Let's try to find the distribution of Yk using a shortcut.
So the probability that Yk is equal to t. So we're trying to find the PMF of Yk.
k has been fixed for us.
And we want to calculate this probability for the various values of t, because this is going to give us the PMF of Yk.
OK. What is this event?
What does it take for the k-th arrival to be at time t?
For that to happen, we need two things.
In the first t -1 slots, how many arrivals should we have gotten?
k - 1.
And then in the last slot, we get one more arrival, and that's the k-th one.
So this is the probability that we have k - 1 arrivals in the time interval from 1 up to t. And then, an arrival at time t. That's the only way that it can happen, that the k-th arrival happens at time t. We need to have an arrival at time t. And before that time, we need to have exactly k - 1 arrivals.
Now this is an event that refers-- t-1.
In the previous time slots we had exactly k -1 arrivals.
And then at the last time slot we get one more arrival.
Now the interesting thing is that this event here has to do with what happened from time 1 up to time t -1.
This event has to do with what happened at time t. Different time slots are independent of each other.
So this event and that event are independent.
So this means that we can multiply their probabilities.
So take the probability of this.
What is that?
Well probability of having a certain number of arrivals in a certain number of time slots, these are just the binomial probabilities.
So this is, out of t - 1 slots, to get exactly k - 1 arrivals, p to the k-1, (1-p) to the t-1 - (k-1), this gives us t-k. And then we multiply with this probability, the probability of an arrival, at time t is equal to p. And so this is the formula for the PMF of the number-- of the time it takes until the k-th arrival happens.
Does it agree with the formula in your handout?
Or its not there?
It's not there.
OK. Yeah.
OK.
So that's the formula and it is true for what values of t?
[INAUDIBLE].
It takes at least k time slots in order to get k arrivals, so this formula should be true for k larger than or equal to t. For t larger than or equal to k. All right.
So this gives us the PMF of the random variable Yk.
Of course, we may also be interested in the mean and variance of Yk.
But this is a lot easier.
Since Yk is the sum of independent random variables, the expected value of Yk is going to be just k times the expected value of your typical t. So the expected value of Yk is going to be just k times 1/p, which is the mean of the geometric.
And similarly for the variance, it's going to be k times the variance of a geometric.
So we have everything there is to know about the distribution of how long it takes until the first arrival comes.
OK.
Finally, let's do a few more things about the Bernoulli process.
It's interesting to talk about several processes at the time.
So in the situation here of splitting a Bernoulli process is where you have arrivals that come to a server.
And that's a picture of which slots get arrivals.
But actually maybe you have two servers.
And whenever an arrival comes to the system, you flip a coin and with some probability, q, you send it to one server.
And with probability 1-q, you send it to another server.
So there is a single arrival stream, but two possible servers.
And whenever there's an arrival, you either send it here or you send it there.
And each time you decide where you send it by flipping an independent coin that has its own bias q.
The coin flips that decide where do you send it are assumed to be independent from the arrival process itself.
So there's two coin flips that are happening.
At each time slot, there's a coin flip that decides whether you have an arrival in this process here, and that coin flip is with parameter p. And if you have something that arrives, you flip another coin with probabilities q, and 1-q, that decides whether you send it up there or you send it down there.
So what kind of arrival process does this server see?
At any given time slot, there's probability p that there's an arrival here.
And there's a further probability q that this arrival gets sent up there.
So the probability that this server sees an arrival at any given time is p times q.
So this process here is going to be a Bernoulli process, but with a different parameter, p times q.
And this one down here, with the same argument, is going to be Bernoulli with parameter p times (1-q).
So by taking a Bernoulli stream of arrivals and splitting it into two, you get two separate Bernoulli processes.
This is going to be a Bernoulli process, that's going to be a Bernoulli process.
Well actually, I'm running a little too fast.
What does it take to verify that it's a Bernoulli process?
At each time slot, it's a 0 or 1.
And it's going to be a 1, you're going to see an arrival with probability p times q.
What else do we need to verify, to be able to tell-- to say that it's a Bernoulli process?
We need to make sure that whatever happens in this process, in different time slots, are statistically independent from each other.
Is that property true?
For example, what happens in this time slot whether you got an arrival or not, is it independent from what happened at that time slot?
The answer is yes for the following reason.
What happens in this time slot has to do with the coin flip associated with the original process at this time, and the coin flip that decides where to send things.
What happens at that time slot has to do with the coin flip here, and the additional coin flip that decides where to send it if something came.
Now all these coin flips are independent of each other.
The coin flips that determine whether we have an arrival here is independent from the coin flips that determined whether we had an arrival there.
And you can generalize this argument and conclude that, indeed, every time slot here is independent from any other time slot.
And this does make it a Bernoulli process.
And the reason is that, in the original process, every time slot is independent from every other time slot.
And the additional assumption that the coin flips that we're using to decide where to send things, these are also independent of each other.
So we're using here the basic property that functions of independent things remain independent.
There's a converse picture of this.
Instead of taking one stream and splitting it into two streams, you can do the opposite.
You could start from two streams of arrivals.
Let's say you have arrivals of men and you have arrivals of women, but you don't care about gender.
And the only thing you record is whether, in a given time slot, you had an arrival or not.
Notice that here we may have an arrival of a man and the arrival of a woman.
We just record it with a 1, by saying there was an arrival.
So in the merged process, we're not keeping track of how many arrivals we had total.
We just record whether there was an arrival or not an arrival.
So an arrival gets recorded here if, and only if, one or both of these streams had an arrival.
So that we call a merging of two Bernoull-- of two processes, of two arrival processes.
So let's make the assumption that this arrival process is independent from that arrival process.
So what happens at the typical slot here?
I'm going to see an arrival, unless none of these had an arrival.
So the probability of an arrival in a typical time slot is going to be 1 minus the probability of no arrival.
And the event of no arrival corresponds to the first process having no arrival, and the second process having no arrival.
So there's no arrival in the merged process if, and only if, there's no arrival in the first process and no arrival in the second process.
We're assuming that the two processes are independent and that's why we can multiply probabilities here.
And then you can take this formula and it simplifies to p + q, minus p times q.
So each time slot of the merged process has a certain probability of seeing an arrival.
Is the merged process a Bernoulli process?
Yes, it is after you verify the additional property that different slots are independent of each other.
Why are they independent?
What happens in this slot has to do with that slot, and that slot down here.
These two slots-- so what happens here, has to do with what happens here and there.
What happens in this slot has to do with whatever happened here and there.
Now, whatever happens here and there is independent from whatever happens here and there.
Therefore, what happens here is independent from what happens there.
So the independence property is preserved.
The different slots of this merged process are independent of each other.
So the merged process is itself a Bernoulli process.
So please digest these two pictures of merging and splitting, because we're going to revisit them in continuous time where things are little subtler than that.
OK. Good luck on the exam and see you in a week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So last time we started talking about random processes.
A random process is a random experiment that evolves over time.
And conceptually, it's important to realize that it's a single probabilistic experiment that has many stages.
Actually, it has an infinite number of stages.
And we discussed the simplest random process there is, the Bernoulli process, which is nothing but the sequence of Bernoulli trials-- an infinite sequence of Bernoulli trials.
For example, flipping a coin over and over.
Once we understand what's going on with that process, then what we want is to move into a continuous time version of the Bernoulli process.
And this is what we will call the Poisson process.
And for the Poisson process, we're going to do exactly the same things that we did for the Bernoulli process.
That is, talk about the number of arrivals during a given time period, and talk also about the time between consecutive arrivals, and for the distribution of inter-arrival times.
So let's start with a quick review of what we discussed last time.
First, a note about language.
If you think of coin tosses, we then talk about heads and tails.
If you think of these as a sequence of trials, you can talk about successes and failures.
The language that we will be using will be more the language of arrivals.
That is, if in a given slot you have a success, you say that something arrived.
If you have a failure, nothing arrived.
And that language is a little more convenient and more natural, especially when we talk about continuous time-- to talk about arrivals instead of successes.
But in any case, for the Bernoulli process let's keep, for a little bit, the language of successes.
Whereas working in discrete time, we have time slots.
During each time slot, we have an independent Bernoulli trial.
There is probability p of having a success.
Different slots are independent of each other.
And this probability p is the same for any given time slot.
So for this process we will discuss the one random variable of interest, which is the following.
If we have n time slots, or n trials, how many arrivals will there be?
Or how many successes will there be?
Well, this is just given by the binomial PMF.
Number of successes in n trials is a random variable that has a binomial PMF, and we know what this is.
Then we talked about inter-arrival times.
The time until the first arrival happens has a geometric distribution.
And we have seen that from some time ago.
Now if you start thinking about the time until k arrivals happen, and we denote that by Yk, this is the time until the first arrival happens.
And then after the first arrival happens, you have to wait some time until the second arrival happens, and so on.
And then the time from the (k -1)th arrival, until arrival number k. The important thing to realize here is that because the process has a memorylessness property, once the first arrival comes, it's as if we're starting from scratch and we will be flipping our coins until the next arrival comes.
So the time it will take until the next arrival comes will also be a geometric random variable.
And because different slots are independent, whatever happens after the first arrival is independent from whatever happened before.
So T1 and T2 will be independent random variables.
And similarly, all the way up to Tk.
So the time until the k-th arrival is a sum of independent geometric random variables, with the same parameter p. And we saw last time that we can find the probability distribution of Yk.
The probability that Yk takes a value of t is equal to-- there's this combinatorial factor here, and then you get p to the k, (1-p) to the (t-k), and this formula is true for t equal to k, k+1, and so on.
And this distribution has a name.
It's called the Pascal PMF.
So this is all there is to know about the Bernoulli process.
One important comment is to realize what exactly this memorylessness property is saying.
So I discussed it a little bit last time.
Let me reiterate it.
So we have a Bernoulli process, which is a sequence of Bernoulli trials.
And these are (0,1) random variables that keep going on forever.
So someone is watching this movie of Bernoulli trials B_t.
And at some point, they say they think, or something interesting has happened, why don't you come in and start watching?
So at some time t, they tell you to come in and start watching.
So what you will see once you come in will be this future trials.
So actually what you will see is a random process, whose first random variable is going to be the first one that you see, B_(t +1).
The second one is going to be this, and so on.
So this is the process that's seen by the person who's asked to come in and start watching at that time.
And the claim is that this process is itself a Bernoulli process, provided that the person who calls you into the room does not look into the future.
The person who calls you into the room decides to call you in only on the basis of what they have seen so far.
So for example, who calls you into the room might have a rule that says, as soon as I see a sequence of 3 heads, I ask the other person to come in.
So if they use that particular rule, it means that when you're called in, the previous 3 were heads.
But this doesn't give you any information about the future.
And so the future ones will be just independent Bernoulli trials.
If on the other hand, the person who calls you in has seen the movie before and they use a rule, such as, for example, I call you in just before 3 heads show up for the first time.
So the person calls you in based on knowledge that these two would be three heads.
If they have such foresight-- if they can look into the future-- then X1, X2, X3, they're certain to be three heads, so they do not correspond to random independent Bernoulli trials.
So to rephrase this, the process is memoryless.
It does not matter what has happened in the past.
And that's true even if you are called into the room and start watching at a random time, as long as that random time is determined in a causal way on the basis of what has happened so far.
So you are called into the room in a causal manner, just based on what's happened so far.
What you're going to see starting from that time will still be a sequence of independent Bernoulli trials.
And this is the argument that we used here, essentially, to argue that this T2 is an independent random variable from T1.
So a person is watching the movie, sees the first success.
And on the basis of what they have seen-- they have just seen the first success-- they ask you to come in.
You come in.
What you're going to see is a sequence of Bernoulli trials.
And you wait this long until the next success comes in.
What you see is a Bernoulli process, as if the process was just starting right now.
And that convinces us that this should be a geometric random variable of the same kind as this one, as independent from what happened before.
All right.
So this is pretty much all there is to know about the Bernoulli process.
Plus the two things that we did at the end of the last lecture where we merge two independent Bernoulli processes, we get a Bernoulli process.
If we have a Bernoulli process and we split it by flipping a coin and sending things one way or the other, then we get two separate Bernoulli processes.
And we see that all of these carry over to the continuous time.
And our task for today is basically to work these continuous time variations.
So the Poisson process is a continuous time version of the Bernoulli process.
Here's the motivation for considering it a Bernoulli process.
So you have that person whose job is to sit outside the door of a bank.
And they have this long sheet, and for every one second slot, they mark an X if a person came in, or they mark something else if no one came in during that slot.
Now the bank manager is a really scientifically trained person and wants very accurate results.
So they tell you, don't use one second slots, use milliseconds slots.
So you have all those slots and you keep filling if someone arrived or not during that slot.
Well then you come up with an idea.
Why use millisecond slots and keep putting crosses or zero's into each slot?
It's much simpler if I just record the exact times when people came in.
So time is continuous.
I don't keep doing something at every time slot.
But instead of the time axis, I mark the times at which customers arrive.
So there's no real need for slots.
The only information that you want is when did we have arrivals of people.
And we want to now model a process of this kind happening in continuous time, that has the same flavor, however, as the Bernoulli process.
So that's the model we want to develop.
OK.
So what are the properties that we're going to have?
First, we're going to assume that intervals over the same length behave probabilistically in an identical fashion.
So what does that mean?
Think of an interval of some given length.
During the interval of that length, there's going to be a random number of arrivals.
And that random number of arrivals is going to have a probability distribution.
So that probability distribution-- let's denote it by this notation.
We fix t, we fix the duration.
So this is fixed.
And we look at the different k's.
The probability of having 0 arrivals, the probability of 1 arrival, the probability of 2 arrivals, and so on.
So this thing is essentially a PMF.
So it should have the property that the sum over all k's of this P_(k, tau) should be equal to 1.
Now, hidden inside this notation is an assumption of time homogeneity.
That is, this probability distribution for the number of arrivals only depends on the length of the interval, but not the exact location of the interval on the time axis.
That is, if I take an interval of length tau, and I ask about the number of arrivals in this interval.
And I take another interval of length tau, and I ask about the number of arrivals during that interval.
Number of arrivals here, and number of arrivals there have the same probability distribution, which is denoted this way.
So the statistical behavior of arrivals here is the same as the statistical behavioral of arrivals there.
What's the relation with the Bernoulli process?
It's very much like the assumption-- the Bernoulli process-- that in different slots, we have the same probability of success.
Every slot looks probabilistically as any other slot.
So similarly here, any interval of length tau looks probabilistically as any other interval of length tau.
And the number of arrivals during that interval is a random variable described by these probabilities.
Number of arrivals here is a random variable described by these same probabilities.
So that's our first assumption.
Then what else?
In the Bernoulli process we had the assumption that different time slots were independent of each other.
Here we do not have time slots, but we can still think in a similar way and impose the following assumption, that these joint time intervals are statistically independent.
What does that mean?
Does a random number of arrivals during this interval, and the random number of arrivals during this interval, and the random number of arrivals during this interval-- so these are three different random variables-- these three random variables are independent of each other.
How many arrivals we got here is independent from how many arrivals we got there.
So this is similar to saying that different time slots were independent.
That's what we did in discrete time.
The continuous time analog is this independence assumption.
So for example, in particular, number of arrivals here is independent from the number of arrivals there.
So these are two basic assumptions about the process.
Now in order to write down a formula, eventually, about this probability distribution-- which is our next objective, we would like to say something specific about this distribution of number of arrivals-- we need to add a little more structure into the problem.
And we're going to make the following assumption.
If we look at the time interval of length delta-- and delta now is supposed to be a small number, so a picture like this-- during a very small time interval, there is a probability that we get exactly one arrival, which is lambda times delta.
Delta is the length of the interval and lambda is a proportionality factor, which is sort of the intensity of the arrival process.
Bigger lambda means that a little interval is more likely to get an arrival.
So there's a probability lambda times delta of 1 arrival.
The remaining probability goes to 0 arrivals.
And when delta is small, the probability of 2 arrivals can be approximated by 0.
So this is a description of what happens during a small, tiny slot.
Now this is something that's supposed to be true in some limiting sense, when delta is very small.
So the exact version of this statement would be that this is an equality, plus order of delta squared terms.
So this is an approximate equality.
And what approximation means is that in the limit of small deltas, the dominant terms-- the constant and the first order term are given by this.
Now when delta is very small, second order terms in delta do not matter.
They are small compared to first order terms.
So we ignore this.
So you can either think in terms of an exact relation, which is the probabilities are given by this, plus delta squared terms.
Or if you want to be a little more loose, you just write here, as an approximate equality.
And the understanding is that this equality holds-- approximately becomes more and more correct as delta goes to 0.
So another version of that statement would be that if you take the limit as delta goes to 0, of p, the probability of having 1 arrival in an interval of length delta, divided by delta, this is equal to lambda.
So that would be one version of an exact statement of what we are assuming here.
So this lambda, we call it the arrival rate, or the intensity of the process.
And clearly, if you double lambda, then a little interval is likely -- you expect to get -- the probability of obtaining an arrival during that interval has doubled.
So in some sense we have twice as intense arrival process.
If you look at the number of arrivals during delta interval, what is the expected value of that random variable?
Well with probability lambda delta we get 1 arrival.
And with the remaining probability, we get 0 arrivals.
So it's just lambda times delta.
So expected number of arrivals during a little interval is lambda times delta.
So expected number of arrivals is proportional to lambda, and that's again why we call lambda the arrival rate.
If you send delta to the denominator in this equality, it tells you that lambda is the expected number of arrivals per unit time.
So the arrival rate is expected number of arrivals per unit time.
And again, that justifies why we call lambda the intensity of this process.
All right.
So where are we now?
For the Bernoulli process, the number of arrivals during a given interval of length n had the PMF that we knew it was the binomial PMF.
What is the formula for the corresponding PMF for the continuous time process?
Somehow we would like to use our assumptions and come up with the formula for this quantity.
So this tells us about the distribution of number of arrivals during an interval of some general length.
We have made assumptions about the number of arrivals during an interval of small length.
An interval of big length is composed of many intervals of small length, so maybe this is the way to go.
Take a big interval, and split it into many intervals of small length.
So we have here our time axis.
And we have an interval of length tau.
And I'm going to split it into lots of little intervals of length delta.
So how many intervals are we going to have?
The number of intervals is going to be the total time, divided by delta.
Now what happens during each one of these little intervals?
As long as the intervals are small, what you have is that during an interval, you're going to have either 0 or 1 arrival.
The probability of more than 1 arrival during a little interval is negligible.
So with this picture, you have essentially a Bernoulli process that consists of so many trials.
And during each one of those trials, we have a probability of success, which is lambda times delta.
Different little intervals here are independent of each other.
That's one of our assumptions, that these joint time intervals are independent.
So approximately, what we have is a Bernoulli process.
We have independence.
We have the number of slots of interest.
And during each one of the slots we have a certain probability of success.
So if we think of this as another good approximation of the Poisson process-- with the approximation becoming more and more accurate as delta goes to 0 -- what we should do would be to take the formula for the PMF of number of arrivals in a Bernoulli process, and then take the limit as delta goes to 0.
So in the Bernoulli process, the probability of k arrivals is n choose k, and then you have p to the k. Now in our case, we have here lambda times delta, delta is tau over n. Delta is tau over n, so p is lambda times tau divided by n. So here's our p -- Lambda tau over n -- to the power k, and then times one minus this-- this is our one minus p-- to the power n-k.
So this is the exact formula for the Bernoulli process.
For the Poisson process, what we do is we take that formula and we let delta go to 0.
As delta goes to 0, n goes to infinity.
So that's the limit that we're taking.
On the other hand, this expression lambda times tau-- lambda times tau, what is it going to be?
Lambda times tau is equal to n times p. n times p, is that what I want?
No, let's see.
Lambda tau is np.
Yeah.
So lambda tau is np.
All right.
So we have this relation, lambda tau equals np.
These two numbers being equal kind of makes sense.
np is the expected number of successes you're going to get in the Bernoulli process.
Lambda tau-- since lambda is the arrival rate and you have a total time of tau, lambda tau you can think of it as the number of expected arrivals in the Bernoulli process.
We're doing a Bernoulli approximation to the Poisson process.
We take the formula for the Bernoulli, and now take the limit as n goes to infinity.
Now lambda tau over n is equal to p, so it's clear what this term is going to give us.
This is just p to the power k. It will actually take a little more work than that.
Now I'm not going to do the algebra, but I'm just telling you that one can take the limit in this formula here, as n goes to infinity.
And that will give you another formula, the final formula for the Poisson PMF.
One thing to notice is that here you have something like 1 minus a constant over n, to the power n. And you may recall from calculus a formula of this kind, that this converges to e to the minus c. If you remember that formula from calculus, then you will expect that here, in the limit, you are going to get something like an e to the minus lambda tau.
So indeed, we will get such a term.
There is some work that needs to be done to find the limit of this expression, times that expression.
The algebra is not hard, it's in the text.
Let's not spend more time doing this.
But let me just give you the formula of what comes at the end.
And the formula that comes at the end is of this form.
So what matters here is not so much the specific algebra that you will do to go from this formula to that one.
It's kind of straightforward.
What's important is the idea that the Poisson process, by definition, can be approximated by a Bernoulli process in which we have a very large number of slots-- n goes to infinity.
Whereas we have a very small probability of success during each time slot.
So a large number of slots, but tiny probability of success during each slot.
And we take the limit as the slots become smaller and smaller.
So with this approximation we end up with this particular formula.
And this is the so-called Poisson PMF.
Now this function P here -- has two arguments.
The important thing to realize is that when you think of this as a PMF, you fix t to tau.
And for a fixed tau, now this is a PMF.
As I said before, the sum over k has to be equal to 1.
So for a given tau, these probabilities add up to 1.
The formula is moderately messy, but not too messy.
One can work with it without too much pain.
And what's the mean and variance of this PMF?
Well what's the expected number of arrivals?
If you think of this Bernoulli analogy, we know that the expected number of arrivals in the Bernoulli process is n times p. In the approximation that we're using in these procedure, n times p is the same as lambda tau.
And that's why we get lambda tau to be the expected number of arrivals.
Here I'm using t instead of tau.
The expected number of arrivals is lambda t. So if you double the time, you expect to get twice as many arrivals.
If you double the arrival rate, you expect to get twice as many arrivals.
How about the formula for the variance?
The variance of the Bernoulli process is np, times one minus p. What does this go to in the limit?
In the limit that we're taking, as delta goes to zero, then p also goes to zero.
The probability of success in any given slot goes to zero.
So this term becomes insignificant.
So this becomes n times p, which is again lambda t, or lambda tau.
So the variance, instead of having this more complicated formula of the variance is the Bernoulli process, here it gets simplified and it's lambda t. So interestingly, the variance in the Poisson process is exactly the same as the expected value.
So you can look at this as just some interesting coincidence.
So now we're going to take this formula and see how to use it.
First we're going to do a completely trivial, straightforward example.
So 15 years ago when that example was made, email was coming at a rate of five messages per hour.
I wish that was the case today.
And now emails that are coming in, let's say during the day-- the arrival rates of emails are probably different in different times of the day.
But if you fix a time slot, let's say 1:00 to 2:00 in the afternoon, there's probably a constant rate.
And email arrivals are reasonably well modeled by a Poisson process.
Speaking of modeling, it's not just email arrivals.
Whenever arrivals happen in a completely random way, without any additional structure, the Poisson process is a good model of these arrivals.
So the times at which car accidents will happen, that's a Poisson processes.
If you have a very, very weak light source that's shooting out photons, just one at a time, the times at which these photons will go out is well modeled again by a Poisson process.
So it's completely random.
Or if you have a radioactive material where one atom at a time changes at random times.
So it's a very slow radioactive decay.
The time at which these alpha particles, or whatever we get emitted, again is going to be described by a Poisson process.
So if you have arrivals, or emissions, that happen at completely random times, and once in a while you get an arrival or an event, then the Poisson process is a very good model for these events.
So back to emails.
Get them at a rate of five messages per day, per hour.
In 30 minutes this is half an hour.
So what we have is that lambda t, total number of arrivals is-- the expected number of arrivals is-- lambda is five, t is one-half, if we talk about hours.
So lambda t is two to the 0.5.
The probability of no new messages is the probability of zero, in time interval of length t, which, in our case, is one-half.
And then we look back into the formula from the previous slide, and the probability of zero arrivals is lambda t to the power zero, divided by zero factorial, and then an e to the lambda t. And you plug in the numbers that we have.
Lambda t to the zero power is one.
Zero factorial is one.
So we're left with e to the minus 2.5.
And that number is 0.08.
Similarly, you can ask for the probability that you get exactly one message in half an hour.
And that would be-- the probability of one message in one-half an hour-- is going to be lambda t to the first power, divided by 1 factorial, e to the minus lambda t, which-- as we now get the extra lambda t factor-- is going to be 2.5, e to the minus 2.5.
And the numerical answer is 0.20.
So this is how you use the PMF formula for the Poisson distribution that we had in the previous slide.
All right.
So this was all about the distribution of the number of arrivals.
What else did we do last time?
Last time we also talked about the time it takes until the k-th arrival.
OK.
So let's try to figure out something about this particular distribution.
We can derive the distribution of the time of the k-th arrival by using the exact same argument as we did last time.
So now the time of the k-th arrival is a continuous random variable.
So it has a PDF.
Since we are in continuous time, arrivals can happen at any time.
So Yk is a continuous random variable.
But now let's think of a time interval of length little delta.
And use our usual interpretation of PDFs.
The PDF of a random variable evaluated at a certain time times delta, this is the probability that the Yk falls in this little interval.
So as I've said before, this is the best way of thinking about PDFs.
PDFs give you probabilities of little intervals.
So now let's try to calculate this probability.
For the k-th arrival to happen inside this little interval, we need two things.
We need an arrival to happen in this interval, and we need k minus one arrivals to happen during that interval.
OK. You'll tell me, but it's possible that we might have the k minus one arrival happen here, and the k-th arrival to happen here.
In principle, that's possible.
But in the limit, when we take delta very small, the probability of having two arrivals in the same little slot is negligible.
So assuming that no two arrivals can happen in the same mini slot, then for the k-th one to happen here, we must have k minus one during this interval.
Now because we have assumed that these joint intervals are independent of each other, this breaks down into the probability that we have exactly k minus one arrivals, during the interval from zero to t, times the probability of exactly one arrival during that little interval, which is lambda delta.
We do have a formula for this from the previous slide, which is lambda t, to the k minus 1, over k minus one factorial, times e to minus lambda t. And then lambda times delta.
Did I miss something?
Yeah, OK. All right.
And now you cancel this delta with that delta.
And that gives us a formula for the PDF of the time until the k-th arrival.
This PDF, of course, depends on the number k. The first arrival is going to happen somewhere in this range of time.
So this is the PDF that it has.
The second arrival, of course, is going to happen later.
And the PDF is this.
So it's more likely to happen around these times.
The third arrival has this PDF, so it's more likely to happen around those times.
And if you were to take k equal to 100, you might get a PDF-- it's extremely unlikely that the k-th arrival happens in the beginning, and it might happen somewhere down there, far into the future.
So depending on which particular arrival we're talking about, it has a different probability distribution.
The time of the 100th arrival, of course, is expected to be a lot larger than the time of the first arrival.
Incidentally, the time of the first arrival has a PDF whose form is quite simple.
If you let k equal to one here, this term disappears.
That term becomes a one.
You're left with just lambda, e to the minus lambda.
And you recognize it, it's the exponential distribution.
So the time until the first arrival in a Poisson process is an exponential distribution.
What was the time of the first arrival in the Bernoulli process?
It was a geometric distribution.
Well, not coincidentally, these two look quite a bit like the other.
A geometric distribution has this kind of shape.
The exponential distribution has that kind of shape.
The geometric is just a discrete version of the exponential.
In the Bernoulli case, we are in discrete time.
We have a PMF for the time of the first arrival, which is geometric.
In the Poisson case, what we get is the limit of the geometric as you let those lines become closer and closer, which gives you the exponential distribution.
Now the Poisson process shares all the memorylessness properties of the Bernoulli process.
And the way one can argue is just in terms of this picture.
Since the Poisson process is the limit of Bernoulli processes, whatever qualitative processes you have in the Bernoulli process remain valid for the Poisson process.
In particular we have this memorylessness property.
You let the Poisson process run for some time, and then you start watching it.
What ever happened in the past has no bearing about the future.
Starting from right now, what's going to happen in the future is described again by a Poisson process, in the sense that during every little slot of length delta, there's going to be a probability of lambda delta of having an arrival.
And that probably lambda delta is the same-- is always lambda delta-- no matter what happened in the past of the process.
And in particular, we could use this argument to say that the time until the k-th arrival is the time that it takes for the first arrival to happen.
OK, let me do it for k equal to two.
And then after the first arrival happens, you wait a certain amount of time until the second arrival happens.
Now once the first arrival happened, that's in the past.
You start watching.
From now on you have mini slots of length delta, each one having a probability of success lambda delta.
It's as if we started the Poisson process from scratch.
So starting from that time, the time until the next arrival is going to be again an exponential distribution, which doesn't care about what happened in the past, how long it took you for the first arrival.
So these two random variables are going to be independent and exponential, with the same parameter lambda.
So among other things, what we have done here is we have essentially derived the PDF of the sum of k independent exponentials.
The time of the k-th arrival is the sum of k inter-arrival times.
The inter-arrival times are all independent of each other because of memorylessness.
And they all have the same exponential distribution.
And by the way, this gives you a way to simulate the Poisson process.
If you wanted to simulate it on your computer, you would have one option to break time into tiny, tiny slots.
And for every tiny slot, use your random number generator to decide whether there was an arrival or not.
To get it very accurate, you would have to use tiny, tiny slots.
So that would be a lot of computation.
The more clever way of simulating the Poisson process is you use your random number generator to generate a sample from an exponential distribution and call that your first arrival time.
Then go back to the random number generator, generate another independent sample, again from the same exponential distribution.
That's the time between the first and the second arrival, and you keep going that way.
So as a sort of a quick summary, this is the big picture.
This table doesn't tell you anything new.
But it's good to have it as a reference, and to look at it, and to make sure you understand what all the different boxes are.
Basically the Bernoulli process runs in discrete time.
The Poisson process runs in continuous time.
There's an analogy of arrival rates, p per trial, or intensity per unit time.
We did derive, or sketched the derivation for the PMF of the number of arrivals.
And the Poisson distribution, which is the distribution that we get, this Pk of t. Pk and t is the limit of the binomial when we take the limit in this particular way, as delta goes to zero, and n goes to infinity.
The geometric becomes an exponential in the limit.
And the distribution of the time of the k-th arrival-- we had a closed form formula last time for the Bernoulli process.
We got the closed form formula this time for the Poisson process.
And we actually used exactly the same argument to get these two closed form formulas.
All right.
So now let's talk about adding or merging Poisson processes.
And there's two statements that we can make here.
One has to do with adding Poisson random variables, just random variables.
There's another statement about adding Poisson processes.
And the second is a bigger statement than the first.
But this is a warm up.
Let's work with the first statement.
So the claim is that the sum of independent Poisson random variables is Poisson.
OK.
So suppose that we have a Poisson process with rate-- just for simplicity-- lambda one.
And I take the interval from zero to two.
And that take then the interval from two until five.
The number of arrivals during this interval-- let's call it n from zero to two-- is going to be a Poisson random variable, with parameter, or with mean, two.
The number of arrivals during this interval is n from time two until five.
This is again a Poisson random variable with mean equal to three, because the arrival rate is 1 and the duration of the interval is three.
These two random variables are independent.
They obey the Poisson distribution that we derived before.
If you add them, what you get is the number of arrivals during the interval from zero to five.
Now what kind of distribution does this random variable have?
Well this is the number of arrivals over an interval of a certain length in a Poisson process.
Therefore, this is also Poisson with mean five.
Because for the Poisson process we know that this number of arrivals is Poisson, this is Poisson, but also the number of overall arrivals is also Poisson.
This establishes that the sum of a Poisson plus a Poisson random variable gives us another Poisson random variable.
So adding Poisson random variables gives us a Poisson random variable.
But now I'm going to make a more general statement that it's not just number of arrivals during a fixed time interval-- it's not just numbers of arrivals for given time intervals-- but rather if you take two different Poisson processes and add them up, the process itself is Poisson in the sense that this process is going to satisfy all the assumptions of a Poisson process.
So the story is that you have a red bulb that flashes at random times at the rate of lambda one.
It's a Poisson process.
You have an independent process where a green bulb flashes at random times.
And you happen to be color blind, so you just see when something is flashing.
So these two are assumed to be independent Poisson processes.
What can we say about the process that you observe?
So in the processes that you observe, if you take a typical time interval of length little delta, what can happen during that little time interval?
The red process may have something flashing.
So red flashes.
Or the red does not.
And for the other bulb, the green bulb, there's two possibilities.
The green one flashes.
And the other possibility is that the green does not.
OK.
So there's four possibilities about what can happen during a little slot.
The probability that the red one flashes and the green one flashes, what is this probability?
It's lambda one delta that the first one flashes, and lambda two delta that the second one does.
I'm multiplying probabilities here because I'm making the assumption that the two processes are independent.
OK. Now the probability that the red one flashes is lambda one delta.
But the green one doesn't is one, minus lambda two delta.
Here the probability would be that the red one does not, times the probability that the green one does.
And then here we have the probability that none of them flash, which is whatever is left.
But it's one minus lambda one delta, times one minus lambda two delta.
Now we're thinking about delta as small.
So think of the case where delta goes to zero, but in a way that we keep the first order terms.
We keep the delta terms, but we throw away the delta squared terms.
Delta squared terms are much smaller than the delta terms when delta becomes small.
If we do that-- if we only keep the order of delta terms-- this term effectively disappears.
This is delta squared.
So we make it zero.
So the probability of having simultaneously a red and a green flash during a little interval is negligible.
What do we get here?
Lambda delta times one survives, but this times that doesn't.
So we can throw that away.
So the approximation that we get is lambda one delta.
Similarly here, this goes away.
We're left with a lambda two delta.
And this is whatever remains, whatever is left.
So what do we have?
That there is a probability of seeing a flash, either a red or a green, which is lambda one delta, plus lambda two delta.
So if we take a little interval of length delta here, it's going to see an arrival with probability approximately lambda one, plus lambda two, delta.
So every slot in this merged process has an arrival probability with a rate which is the sum of the rates of these two processes.
So this is one part of the definition of the Poisson process.
There's a few more things that one would need to verify.
Namely, that intervals of the same length have the same probability distribution and that different slots are independent of each other.
This can be argued by starting from here because different intervals in this process are independent from each other.
Different intervals here are independent from each other.
It's not hard to argue that different intervals in the merged process will also be independent of each other.
So the conclusion that comes at the end is that this process is a Poisson process, with a total rate which is equal to the sum of the rate of the two processes.
And now if I tell you that an arrival happened in the merged process at a certain time, how likely is it that it came from here?
How likely is it?
We go to this picture.
Given that an arrival occurred-- which is the event that this or that happened-- what is the probability that it came from the first process, the red one?
Well it's the probability of this divided by the probability of this, times that.
Given that this event occurred, you want to find the conditional probability of that sub event.
So we're asking the question, out of the total probability of these two, what fraction of that probability is assigned here?
And this is lambda one delta, after we ignore the other terms.
This is lambda two delta.
So that fraction is going to be lambda one, over lambda one plus lambda two.
What does this tell you?
If lambda one and lambda two are equal, given that I saw an arrival here, it's equally likely to be red or green.
But if the reds have a much higher arrival rate, when I see an arrival here, it's more likely this number will be large.
So it's more likely to have come from the red process.
OK so we'll continue with this story and do some applications next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to finish our discussion of the Poisson process.
We're going to see a few of its properties, do a few interesting problems, some more interesting than others.
So go through a few examples and then we're going to talk about some quite strange things that happen with the Poisson process.
So the first thing is to remember what the Poisson processes is.
It's a model, let's say, of arrivals of customers that are, in some sense, quote unquote, completely random, that is a customer can arrive at any point in time.
All points in time are equally likely.
And different points in time are sort of independent of other points in time.
So the fact that I got an arrival now doesn't tell me anything about whether there's going to be an arrival at some other time.
In some sense, it's a continuous time version of the Bernoulli process.
So the best way to think about the Poisson process is that we divide time into extremely tiny slots.
And in each time slot, there's an independent possibility of having an arrival.
Different time slots are independent of each other.
On the other hand, when the slot is tiny, the probability for obtaining an arrival during that tiny slot is itself going to be tiny.
So we capture these properties into a formal definition what the Poisson process is.
We have a probability mass function for the number of arrivals, k, during an interval of a given length.
So this is the sort of basic description of the distribution of the number of arrivals.
So tau is fixed.
And k is the parameter.
So when we add over all k's, the sum of these probabilities has to be equal to 1.
There's a time homogeneity assumption, which is hidden in this, namely, the only thing that matters is the duration of the time interval, not where the time interval sits on the real axis.
Then we have an independence assumption.
Intervals that are disjoint are statistically independent from each other.
So any information you give me about arrivals during this time interval doesn't change my beliefs about what's going to happen during another time interval.
So this is a generalization of the idea that we had in Bernoulli processes that different time slots are independent of each other.
And then to specify this function, the distribution of the number of arrivals, we sort of go in stages.
We first specify this function for the case where the time interval is very small.
And I'm telling you what those probabilities will be.
And based on these then, we do some calculations and to find the formula for the distribution of the number of arrivals for intervals of a general duration.
So for a small duration, delta, the probability of obtaining 1 arrival is lambda delta.
The remaining probability is assigned to the event that we get to no arrivals during that interval.
The probability of obtaining more than 1 arrival in a tiny interval is essentially 0.
And when we say essentially, it's means modular, terms that of order delta squared.
And when delta is very small, anything which is delta squared can be ignored.
So up to delta squared terms, that's what happened during a little interval.
Now if we know the probability distribution for the number of arrivals in a little interval.
We can use this to get the distribution for the number of arrivals over several intervals.
How do we do that?
The big interval is composed of many little intervals.
Each little interval is independent from any other little interval, so is it is as if we have a sequence of Bernoulli trials.
Each Bernoulli trial is associated with a little interval and has a small probability of obtaining a success or an arrival during that mini-slot.
On the other hand, when delta is small, and you take a big interval and chop it up, you get a large number of little intervals.
So what we essentially have here is a Bernoulli process, in which is the number of trials is huge but the probability of success during any given trial is tiny.
The average number of trials ends up being proportional to the length of the interval.
If you have twice as large an interval, it's as if you're having twice as many over these mini-trials, so the expected number of arrivals will increase proportionately.
There's also this parameter lambda, which we interpret as expected number of arrivals per unit time.
And it comes in those probabilities here.
When you double lambda, this means that a little interval is twice as likely to get an arrival.
So you would expect to get twice as many arrivals as well.
That's why the expected number of arrivals during an interval of length tau also scales proportional to this parameter lambda.
Somewhat unexpectedly, it turns out that the variance of the number of arrivals is also the same as the mean.
This is a peculiarity that happens in the Poisson process.
So this is one way of thinking about Poisson process, in terms of little intervals, each one of which has a tiny probability of success.
And we think of the distribution associated with that process as being described by this particular PMF.
So this is the PMF for the number of arrivals during an interval of a fixed duration, tau.
It's a PMF that extends all over the entire range of non-negative integers.
So the number of arrivals you can get during an interval for certain length can be anything.
You can get as many arrivals as you want.
Of course the probability of getting a zillion arrivals is going to be tiny.
But in principle, this is possible.
And that's because an interval, even if it's a fixed length, consists of an infinite number of mini-slots in some sense.
You can divide, chop it up, into as many mini-slots as you want.
So in principle, it's possible that every mini-slot gets an arrival.
In principle, it's possible to get an arbitrarily large number of arrivals.
So this particular formula here is not very intuitive when you look at it.
But it's a legitimate PMF.
And it's called the Poisson PMF.
It's the PMF that describes the number of arrivals.
So that's one way of thinking about the Poisson process, where the basic object of interest would be this PMF and you try to work with it.
There's another way of thinking about what happens in the Poisson process.
And this has to do with letting things evolve in time.
You start at time 0.
There's going to be a time at which the first arrival occurs, and call that time T1.
This time turns out to have an exponential distribution with parameter lambda.
Once you get an arrival, it's as if the process starts fresh.
The best way to understand why this is the case is by thinking in terms of the analogy with the Bernoulli process.
If you believe that statement for the Bernoulli process, since this is a limiting case, it should also be true.
So starting from this time, we're going to wait a random amount of time until we get the second arrival This random amount of time, let's call it T2.
This time, T2 is also going to have an exponential distribution with the same parameter, lambda.
And these two are going to be independent of each other.
OK?
So the Poisson process has all the same memorylessness properties that the Bernoulli process has.
What's another way of thinking of this property?
So think of a process where you have a light bulb.
The time at the light bulb burns out, you can model it by an exponential random variable.
And suppose that they tell you that so far, we're are sitting at some time, T. And I tell you that the light bulb has not yet burned out.
What does this tell you about the future of the light bulb?
Is the fact that they didn't burn out, so far, is it good news or is it bad news?
Would you rather keep this light bulb that has worked for t times steps and is still OK?
Or would you rather use a new light bulb that starts new at that point in time?
Because of the memorylessness property, the past of that light bulb doesn't matter.
So the future of this light bulb is statistically the same as the future of a new light bulb.
For both of them, the time until they burn out is going to be described an exponential distribution.
So one way that people described the situation is to say that used is exactly as good as a new.
So a used on is no worse than a new one.
A used one is no better than a new one.
So a used light bulb that hasn't yet burnt out is exactly as good as a new light bulb.
So that's another way of thinking about the memorylessness that we have in the Poisson process.
Back to this picture.
The time until the second arrival is the sum of two independent exponential random variables.
So, in principle, you can use the convolution formula to find the distribution of T1 plus T2, and that would be what we call Y2, the time until the second arrival.
But there's also a direct way of obtaining to the distribution of Y2, and this is the calculation that we did last time on the blackboard.
And actually, we did it more generally.
We found the time until the case arrival occurs.
It has a closed form formula, which is called the Erlang distribution with k degrees of freedom.
So let's see what's going on here.
It's a distribution Of what kind?
It's a continuous distribution.
It's a probability density function.
This is because the time is a continuous random variable.
Time is continuous.
Arrivals can happen at any time.
So we're talking about the PDF.
This k is just the parameter of the distribution.
We're talking about the k-th arrival, so k is a fixed number.
Lambda is another parameter of the distribution, which is the arrival rate So it's a PDF over the Y's, whereas lambda and k are parameters of the distribution.
OK.
So this was what we knew from last time.
Just to get some practice, let us do a problem that's not too difficult, but just to see how we use the various formulas that we have.
So Poisson was a mathematician, but Poisson also means fish in French.
So Poisson goes fishing.
And let's assume that fish are caught according to a Poisson process.
That's not too bad an assumption.
At any given point in time, you have a little probability that a fish would be caught.
And whether you catch one now is sort of independent about whether at some later time a fish will be caught or not.
So let's just make this assumption.
And suppose that the rules of the game are that you-- Fish are being called it the certain rate of 0.6 per hour.
You fish for 2 hours, no matter what.
And then there are two possibilities.
If I have caught a fish, I stop and go home.
So if some fish have been caught, so there's at least 1 arrival during this interval, I go home.
Or if nothing has being caught, I continue fishing until I catch something.
And then I go home.
So that's the description of what is going to happen.
And now let's starts asking questions of all sorts.
What is the probability that I'm going to be fishing for more than 2 hours?
I will be fishing for more than 2 hours, if and only if no fish were caught during those 2 hours, in which case, I will have to continue.
Therefore, this is just this quantity.
The probability of catching 2 fish in-- of catching 0 fish in the next 2 hours, and according to the formula that we have, this is going to be e to the minus lambda times how much time we have.
There's another way of thinking about this.
The probability that I fish for more than 2 hours is the probability that the first catch happens after time 2, which would be the integral from 2 to infinity of the density of the first arrival time.
And that density is an exponential.
So you do the integral of an exponential, and, of course, you would get the same answer.
OK. That's easy.
So what's the probability of fishing for more than 2 but less than 5 hours?
What does it take for this to happen?
For this to happen, we need to catch 0 fish from time 0 to 2 and catch the first fish sometime between 2 and 5.
So if you-- one way of thinking about what's happening here might be to say that there's a Poisson process that keeps going on forever.
But as soon as I catch the first fish, instead of continuing fishing and obtaining those other fish I just go home right now.
Now the fact that I go home before time 5 means that, if I were to stay until time 5, I would have caught at least 1 fish.
I might have caught more than 1.
So the event of interest here is that the first catch happens between times 2 and 5.
So one way of calculating this quantity would be-- Its the probability that the first catch happens between times 2 and 5.
Another way to deal with it is to say, this is the probability that I caught 0 fish in the first 2 hours and then the probability that I catch at least 1 fish during the next 3 hours.
This.
What is this?
The probability of 0 fish in the next 3 hours is the probability of 0 fish during this time.
1 minus this is the probability of catching at least 1 fish, of having at least 1 arrival, between times 2 and 5.
If there's at least 1 arrival between times 2 and 5, then I would have gone home by time 5.
So both of these, if you plug-in numbers and all that, of course, are going to give you the same answer.
Now next, what's the probability that I catch at least 2 fish?
In which scenario are we?
Under this scenario, I go home when I catch my first fish.
So in order to catch at least 2 fish, it must be in this case.
So this is the same as the event that I catch at least 2 fish during the first 2 time steps.
So it's going to be the probability from 2 to infinity, the probability that I catch 2 fish, or that I catch 3 fish, or I catch more than that.
So it's this quantity.
k is the number of fish that I catch.
At least 2, so k goes from 2 to infinity.
These are the probabilities of catching a number k of fish during this interval.
And if you want a simpler form without an infinite sum, this would be 1 minus the probability of catching 0 fish, minus the probability of catching 1 fish, during a time interval of length 2.
Another way to think of it.
I'm going to catch 2 fish, at least 2 fish, if and only if the second fish caught in this process happens before time 2.
So that's another way of thinking about the same event.
So it's going to be the probability that the random variable Y2, the arrival time over the second fish, is less than or equal to 2.
OK.
The next one is a little trickier.
Here we need to do a little bit of divide and conquer.
Overall, in this expedition, what the expected number of fish to be caught?
One way to think about it is to try to use the total expectations theorem.
And think of expected number of fish, given this scenario, or expected number of fish, given this scenario.
That's a little more complicated than the way I'm going to do it.
The way I'm going to do is to think as follows-- Expected number of fish is the expected number of fish caught between times 0 and 2 plus expected number of fish caught after time 2.
So what's the expected number caught between time 0 and 2?
This is lambda t. So lambda is 0.6 times 2.
This is the expected number of fish that are caught between times 0 and 2.
Now let's think about the expected number of fish caught afterwards.
How many fish are being caught afterwards?
Well it depends on the scenario.
If we're in this scenario, we've gone home and we catch 0.
If we're in this scenario, then we continue fishing until we catch one.
So the expected number of fish to be caught after time 2 is going to be the probability of this scenario times 1.
And the probability of that scenario is the probability that they call it's 0 fish during the first 2 time steps times 1, which is the number of fish I'm going to catch if I continue.
The expected total fishing time we can calculate exactly the same way.
I'm jumping to the last one.
My total fishing time has a period of 2 time steps.
I'm going to fish for 2 time steps no matter what.
And then if I caught 0 fish, which happens with this probability, my expected time is going to be the expected time from here onwards, which is the expected value of this geometric random variable with parameter lambda.
So the expected time is 1 over lambda.
And in our case this, is 1/0.6.
Finally, if I tell you that I have been fishing for 4 hours and nothing has been caught so far, how much do you expect this quantity to be?
Here is the story that, again, that for the Poisson process used is as good as new.
The process does not have any memory.
Given what happens in the past doesn't matter for the future.
It's as if the process starts new at this point in time.
So this one is going to be, again, the same exponentially distributed random variable with the same parameter lambda.
So expected time until an arrival comes is an exponential distribut -- has an exponential distribution with parameter lambda, no matter what has happened in the past.
Starting from now and looking into the future, it's as if the process has just started.
So it's going to be 1 over lambda, which is 1/0.6.
OK. Now our next example is going to be a little more complicated or subtle.
But before we get to the example, let's refresh our memory about what we discussed last time about merging Poisson independent Poisson processes.
Instead of drawing the picture that way, another way we could draw it could be this.
We have a Poisson process with rate lambda1, and a Poisson process with rate lambda2.
They have, each one of these, have their arrivals.
And then we form the merged process.
And the merged process records an arrival whenever there's an arrival in either of the two processes.
This process in that process are assumed to be independent of each other.
Now different times in this process and that process are independent of each other.
So what happens in these two time intervals is independent from what happens in these two time intervals.
These two time intervals to determine what happens here.
These two time intervals determine what happens there.
So because these are independent from these, this means that this is also independent from that.
So the independence assumption is satisfied for the merged process.
And the merged process turns out to be a Poisson process.
And if you want to find the arrival rate for that process, you argue as follows.
During a little interval of length delta, we have probability lambda1 delta of having an arrival in this process.
We have probability lambda2 delta of an arrival in this process, plus second order terms in delta, which we're ignoring.
And then you do the calculation and you find that in this process, you're going to have an arrival probability, which is lambda1 plus lambda2, again ignoring second order in delta-- terms that are second order in delta.
So the merged process is a Poisson process whose arrival rate is the sum of the arrival rates of the individual processes.
And the calculation we did at the end of the last lecture-- If I tell you that the new arrival happened here, where did that arrival come from?
Did it come from here or from there?
If the lambda1 is equal to lambda2, then by symmetry you would say that it's equally likely to have come from here or to come from there.
But if this lambda is much bigger than that lambda, the fact that they saw an arrival is more likely to have come from there.
And the formula that captures this is the following.
This is the probability that my arrival has come from this particular stream rather than that particular stream.
So when an arrival comes and you ask, what is the origin of that arrival?
It's as if I'm flipping a coin with these odds.
And depending on outcome of that coin, I'm going to tell you came from there or it came from there.
So the origin of an arrival is either this stream or that stream.
And this is the probability that the origin of the arrival is that one.
Now if we look at 2 different arrivals, and we ask about their origins-- So let's think about the origin of this arrival and compare it with the origin that arrival.
The origin of this arrival is random.
It could be right be either this or that.
And this is the relevant probability.
The origin of that arrival is random.
It could be either here or is there, and again, with the same relevant probability.
Question.
The origin of this arrival, is it dependent or independent from the origin that arrival?
And here's how the argument goes.
Separate times are independent.
Whatever has happened in the process during this set of times is independent from whatever happened in the process during that set of times.
Because different times have nothing to do with each other, the origin of this, of an arrival here, has nothing to do with the origin of an arrival there.
So the origins of different arrivals are also independent random variables.
So if I tell you that-- yeah.
OK.
So it as if that each time that you have an arrival in the merge process, it's as if you're flipping a coin to determine where did that arrival came from and these coins are independent of each other.
OK. OK. Now we're going to use this-- what we know about merged processes to solve the problem that would be harder to do, if you were not using ideas from Poisson processes.
So the formulation of the problem has nothing to do with the Poisson process.
The formulation is the following.
We have 3 light-bulbs.
And each light bulb is independent and is going to die out at the time that's exponentially distributed.
So 3 light bulbs.
They start their lives and then at some point they die or burn out.
So let's think of this as X, this as Y, and this as Z.
And we're interested in the time until the last light-bulb burns out.
So we're interested in the maximum of the 3 random variables, X, Y, and Z.
And in particular, we want to find the expected value of this maximum.
OK.
So you can do derived distribution, use the expected value rule, anything you want.
You can get this answer using the tools that you already have in your hands.
But now let us see how we can connect to this picture with a Poisson picture and come up with the answer in a very simple way.
What is an exponential random variable?
An exponential random variable is the first act in the long play that involves a whole Poisson process.
So an exponential random variable is the first act of a Poisson movie.
Same thing here.
You can think of this random variable as being part of some Poisson process that has been running.
So it's part of this bigger picture.
We're still interested in the maximum of the 3.
The other arrivals are not going to affect our answers.
It's just, conceptually speaking, we can think of the exponential random variable as being embedded in a bigger Poisson picture.
So we have 3 Poisson process that are running in parallel.
Let us split the expected time until the last burnout into pieces, which is time until the first burnout, time from the first until the second, and time from the second until the third.
And find the expected values of each one of these pieces.
What can we say about the expected value of this?
This is the first arrival out of all of these 3 Poisson processes.
It's the first event that happens when you look at all of these processes simultaneously.
So 3 Poisson processes running in parallel.
We're interested in the time until one of them, any one of them, gets in arrival.
Rephrase.
We merged the 3 Poisson processes, and we ask for the time until we observe an arrival in the merged process.
When 1 of the 3 gets an arrival for the first time, the merged process gets its first arrival.
So what's the expected value of this time until the first burnout?
It's going to be the expected value of a Poisson random variable.
So the first burnout is going to have an expected value, which is-- OK.
It's a Poisson process.
The merged process of the 3 has a collective arrival rate, which is 3 times lambda.
So this is the parameter over the exponential distribution that describes the time until the first arrival in the merged process.
And the expected value of this random variable is 1 over that.
When you have an exponential random variable with parameter lambda, the expected value of that random variable is 1 over lambda.
Here we're talking about the first arrival time in a process with rate 3 lambda.
The expected time until the first arrival is 1 over (3 lambda).
Alright.
So at this time, this bulb, this arrival happened, this bulb has been burned.
So we don't care about that bulb anymore.
We start at this time, and we look forward.
This bulb has been burned.
So let's just look forward from now on.
What have we got?
We have two bulbs that are burning.
We have a Poisson process that's the bigger picture of what could happen to that light bulb, if we were to keep replacing it.
Another Poisson process.
These two processes are, again, independent.
From this time until that time, how long does it take?
It's the time until either this process records an arrival or that process records and arrival.
That's the same as the time that the merged process of these two records an arrival.
So we're talking about the expected time until the first arrival in a merged process.
The merged process is Poisson.
It's Poisson with rate 2 lambda.
So that extra time is going to take-- the expected value is going to be 1 over the (rate of that Poisson process).
So 1 over (2 lambda) is the expected value of this random variable.
So at this point, this bulb now is also burned.
So we start looking from this time on.
That part of the picture disappears.
Starting from this time, what's the expected value until that remaining light-bulb burns out?
Well, as we said before, in a Poisson process or with exponential random variables, we have memorylessness.
A used bulb is as good as a new one.
So it's as if we're starting from scratch here.
So this is going to be an exponential random variable with parameter lambda.
And the expected value of it is going to be 1 over lambda.
So the beauty of approaching this problem in this particular way is, of course, that we manage to do everything without any calculus at all, without striking an integral, without trying to calculate expectations in any form.
Most of the non-trivial problems that you encounter in the Poisson world basically involve tricks of these kind.
You have a question and you try to rephrase it, trying to think in terms of what might happen in the Poisson setting, use memorylessness, use merging, et cetera, et cetera.
Now we talked about merging.
It turns out that the splitting of Poisson processes also works in a nice way.
The story here is exactly the same as for the Bernoulli process.
So I'm having a Poisson process.
And each time, with some rate lambda, and each time that an arrival comes, I'm going to send it to that stream and the record an arrival here with some probability P. And I'm going to send it to the other stream with some probability 1 minus P. So either of this will happen or that will happen, depending on the outcome of the coin flip that I do.
Each time that then arrival occurs, I flip a coin and I decide whether to record it here or there.
This is called splitting a Poisson process into two pieces.
What kind of process do we get here?
If you look at the little interval for length delta, what's the probability that this little interval gets an arrival?
It's the probability that this one gets an arrival, which is lambda delta times the probability that after I get an arrival my coin flip came out to be that way, so that it sends me there.
So this means that this little interval is going to have probability lambda delta P. Or maybe more suggestively, I should write it as lambda P times delta.
So every little interval has a probability of an arrival proportional to delta.
The proportionality factor is lambda P. So lambda P is the rate of that process.
And then you go through the mental exercise that you went through for the Bernoulli process to argue that a different intervals here are independent and so on.
And that completes checking that this process is going to be a Poisson process.
So when you split a Poisson process by doing independent coin flips each time that something happens, the processes that you get is again a Poisson process, but of course with a reduced rate.
So instead of the word splitting, sometimes people also use the words thinning-out.
That is, out of the arrivals that came, you keep a few but throw away a few.
OK.
So now the last topic over this lecture is a quite curious phenomenon that goes under the name of random incidents.
So here's the story.
Buses have been running on Mass Ave. from time immemorial.
And the bus company that runs the buses claims that they come as a Poisson process with some rate, let's say, of 4 buses per hour.
So that the expected time between bus arrivals is going to be 15 minutes.
OK. Alright.
So people have been complaining that they have been showing up there.
They think the buses are taking too long.
So you are asked to investigate.
Is the company-- Does it operate according to its promises or not.
So you send an undercover agent to go and check the interarrival times of the buses.
Are they 15 minutes?
Or are they longer?
So you put your dark glasses and you show up at the bus stop at some random time.
And you go and ask the guy in the falafel truck, how long has it been since the last arrival?
So of course that guy works for the FBI, right?
So they tell you, well, it's been, let's say, 12 minutes since the last bus arrival.
And then you say, "Oh, 12 minutes.
Average time is 15.
So a bus should be coming any time now."
Is that correct?
No, you wouldn't think that way.
It's a Poisson process.
It doesn't matter how long it has been since the last bus arrival.
So you don't go through that fallacy.
Instead of predicting how long it's going to be, you just sit down there and wait and measure the time.
And you find that this is, let's say, 11 minutes.
And you go to your boss and report, "Well, it took-- I went there and the time from the previous bus to the next one was 23 minutes.
It's more than the 15 that they said."
So go and do that again.
You go day after day.
You keep these statistics of the length of this interval.
And you tell your boss it's a lot more than 15.
It tends to be more like 30 or so.
So the bus company is cheating us.
Does the bus company really run Poisson buses at the rate that they have promised?
Well let's analyze the situation here and figure out what the length of this interval should be, on the average.
The naive argument is that this interval is an interarrival time.
And interarrival times, on the average, are 15 minutes, if the company runs indeed Poisson processes with these interarrival times.
But actually the situation is a little more subtle because this is not a typical interarrival interval.
This interarrival interval consists of two pieces.
Let's call them T1 and T1 prime.
What can you tell me about those two random variables?
What kind of random variable is T1?
Starting from this time, with the Poisson process, the past doesn't matter.
It's the time until an arrival happens.
So T1 is going to be an exponential random variable with parameter lambda.
So in particular, the expected value of T1 is going to be 15 by itself.
How about the random variable T1 prime.
What kind of random variable is it?
This is like the first arrival in a Poisson process that runs backwards in time.
What kind of process is a Poisson process running backwards in time?
Let's think of coin flips.
Suppose you have a movie of coin flips.
And for some accident, that fascinating movie, you happen to watch it backwards.
Will it look any different statistically?
No.
It's going to be just the sequence of random coin flips.
So a Bernoulli process that's runs in reverse time is statistically identical to a Bernoulli process in forward time.
The Poisson process is a limit of the Bernoulli.
So, same story with the Poisson process.
If you run it backwards in time it looks the same.
So looking backwards in time, this is a Poisson process.
And T1 prime is the time until the first arrival in this backward process.
So T1 prime is also going to be an exponential random variable with the same parameter, lambda.
And the expected value of T1 prime is 15.
Conclusion is that the expected length of this interval is going to be 30 minutes.
And the fact that this agent found the average to be something like 30 does not contradict the claims of the bus company that they're running Poisson buses with a rate of lambda equal to 4.
OK.
So maybe the company can this way-- they can defend themselves in court.
But there's something puzzling here.
How long is the interarrival time?
Is it 15?
Or is it 30?
On the average.
The issue is what do we mean by a typical interarrival time.
When we say typical, we mean some kind of average.
But average over what?
And here's two different ways of thinking about averages.
You number the buses.
And you have bus number 100.
You have bus number 101, bus number 102, bus number 110, and so on.
One way of thinking about averages is that you pick a bus number at random.
I pick, let's say, that bus, all buses being sort of equally likely to be picked.
And I measure this interarrival time.
So for a typical bus.
Then, starting from here until there, the expected time has to be 1 over lambda, for the Poisson process.
But what we did in this experiment was something different.
We didn't pick a bus at random.
We picked a time at random.
And if the picture is, let's say, this way, I'm much more likely to pick this interval and therefore this interarrival time, rather than that interval.
Because, this interval corresponds to very few times.
So if I'm picking a time at random and, in some sense, let's say, uniform, so that all times are equally likely, I'm much more likely to fall inside a big interval rather than a small interval.
So a person who shows up at the bus stop at a random time.
They're selecting an interval in a biased way, with the bias favor of longer intervals.
And that's why what they observe is a random variable that has a larger expected value then the ordinary expected value.
So the subtlety here is to realize that we're talking between two different kinds of experiments.
Picking a bus number at random verses picking an interval at random with a bias in favor of longer intervals.
Lots of paradoxes that one can cook up using Poisson processes and random processes in general often have to do with the story of this kind.
The phenomenon that we had in this particular example also shows up in general, whenever you have other kinds of arrival processes.
So the Poisson process is the simplest arrival process there is, where the interarrival times are exponential random variables.
There's a larger class of models.
They're called renewal processes, in which, again, we have a sequence of successive arrivals, interarrival times are identically distributed and independent, but they may come from a general distribution.
So to make the same point of the previous example but in a much simpler setting, suppose that bus interarrival times are either 5 or 10 minutes apart.
So you get some intervals that are of length 5.
You get some that are of length 10.
And suppose that these are equally likely.
So we have -- not exactly -- In the long run, we have as many 5 minute intervals as we have 10 minute intervals.
So the average interarrival time is 7 and 1/2.
But if a person shows up at a random time, what are they going to see?
Do we have as many 5s as 10s?
But every 10 covers twice as much space.
So if I show up at a random time, I have probability 2/3 falling inside an interval of duration 10.
And I have one 1/3 probability of falling inside an interval of duration 5.
That's because, out of the whole real line, 2/3 of it is covered by intervals of length 10, just because they're longer.
1/3 is covered by the smaller intervals.
Now if I fall inside an interval of length 10 and I measure the length of the interval that I fell into, that's going to be 10.
But if I fall inside an interval of length 5 and I measure how long it is, I'm going to get a 5.
And that, of course, is going to be different than 7.5.
OK. And which number should be bigger?
It's the second number that's bigger because this one is biased in favor of the longer intervals.
So that's, again, another illustration of the different results that you get when you have this random incidence phenomenon.
So the bottom line, again, is that if you talk about a typical interarrival time, one must be very precise in specifying what we mean typical.
So typical means sort of random.
But to use the word random, you must specify very precisely what is the random experiment that you are using.
And if you're not careful, you can get into apparent puzzles, such as the following.
Suppose somebody tells you the average family size is 4, but the average person lives in a family of size 6.
Is that compatible?
Family size is 4 on the average, but typical people live, on the average, in families of size 6.
Well yes.
There's no contradiction here.
We're talking about two different experiments.
In one experiment, I pick a family at random, and I tell you the average family is 4.
In another experiment, I pick a person at random and I tell you that this person, on the average, will be in their family of size 6.
And what is the catch here?
That if I pick a person at random, large families are more likely to be picked.
So there's a bias in favor of large families.
Or if you want to survey, let's say, are trains crowded in your city?
Or are buses crowded?
One choice is to pick a bus at random and inspect how crowded it is.
Another choice is to pick a typical person and ask them, "Did you ride the bus today?
Was it's crowded?"
Well suppose that in this city there's one bus that's extremely crowded and all the other buses are completely empty.
If you ask a person.
"Was your bus crowded?"
They will tell you, "Yes, my bus was crowded."
There's no witness from the empty buses to testify in their favor.
So by sampling people instead of sampling buses, you're going to get different result.
And in the process industry, if your job is to inspect and check cookies, you will be faced with a big dilemma.
Do you want to find out how many chocolate chips there are on a typical cookie?
Are you going to interview cookies or are you going to interview chocolate chips and ask them how many other chips where there on your cookie?
And you're going to get different answers in these cases.
So moral is, one has to be very precise on how you formulate the sampling procedure that you have.
And you'll get different answers.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to start now with a new chapter.
We're going to talk about Markov processes.
The good news is that this is a subject that is a lot more intuitive and simple in many ways than, let's say, the Poisson processes.
So hopefully this will be enjoyable.
So Markov processes is, a general class of random processes.
In some sense, it's more elaborate than the Bernoulli and Poisson processes, because now we're going to have dependencies between difference times, instead of having memoryless processes.
So the basic idea is the following.
In physics, for example, you write down equations for how a system evolves that has the general form.
The new state of a system one second later is some function of old state.
So Newton's equations and all that in physics allow you to write equations of this kind.
And so if that a particle is moving at a certain velocity and it's at some location, you can predict when it's going to be a little later.
Markov processes have the same flavor, except that there's also some randomness thrown inside the equation.
So that's what Markov process essentially is.
It describes the evolution of the system, or some variables, but in the presence of some noise so that the motion itself is a bit random.
So this is a pretty general framework.
So pretty much any useful or interesting random process that you can think about, you can always described it as a Markov process if you define properly the notion of the state.
So what we're going to do is we're going to introduce the class of Markov processes by, example, by talking about the checkout counter in a supermarket.
Then we're going to abstract from our example so that we get a more general definition.
And then we're going to do a few things, such as how to predict what's going to happen n time steps later, if we start at the particular state.
And then talk a little bit about some structural properties of Markov processes or Markov chains.
So here's our example.
You go to the checkout counter at the supermarket, and you stand there and watch the customers who come.
So customers come, they get in queue, and customers get served one at a time.
So the discussion is going to be in terms of supermarket checkout counters, but the same story applies to any service system.
You may have a server, jobs arrive to that server, they get put into the queue, and the server processes those jobs one at a time.
Now to make a probabilistic model, we need to make some assumption about the customer arrivals and the customer departures.
And we want to keep things as simple as possible to get started.
So let's assume that customers arrive according to a Bernoulli process with some parameter b.
So essentially, that's the same as the assumption that the time between consecutive customer arrivals is a geometric random variable with parameter b.
Another way of thinking about the arrival process-- that's not how it happens, but it's helpful, mathematically, is to think of someone who's flipping a coin with bias equal to b.
And whenever the coin lands heads, then a customer arrives.
So it's as if there's a coin flip being done by nature that decides the arrivals of the customers.
So we know that coin flipping to determine the customer arrivals is the same as having geometric inter-arrival times.
We know that from our study of the Bernoulli process.
OK. And now how about the customer service times.
We're going to assume that-- OK.
If there is no customer in queue, no one being served, then of course, no one is going to depart from the queue.
But if there a customer in queue, then that customer starts being served, and is going to be served for a random amount of time.
And we make the assumption that the time it takes for the clerk to serve the customer has a geometric distribution with some known parameter q.
So the time it takes to serve a customer is random, because it's random how many items they got in their cart, and how many coupons they have to unload and so on.
So it's random.
In the real world, it has some probability distribution.
Let's not care exactly about what it would be in the real world, but as a modeling approximation or just to get started, let's pretend that customer service time are well described by a geometric distribution, with a parameter q.
An equivalent way of thinking about the customer service, mathematically, would be, again, in terms of coin flipping.
That is, the clerk has a coin with a bias, and at each time slot the clerk flips the coin.
With probability q, service is over.
With probability 1-q, you continue the service process.
An assumption that we're going to make is that the coin flips that happen here to determine the arrivals, they're all independent of each other.
The coin flips that determine the end of service are also independent from each other.
But also the coin flips involved here are independent from the coin flips that happened there.
So how arrivals happen is independent with what happens at the service process.
OK.
So suppose now you want to answer a question such as the following.
The time is 7:00 PM.
What's the probability that the customer will be departing at this particular time?
Well, you say, it depends.
If the queue is empty at that time, then you're certain that you're not going to have a customer departure.
But if the queue is not empty, then there is probability q that a departure will happen at that time.
So the answer to a question like this has something to do with the state of the system at that time.
It depends what the queue is.
And if I ask you, will the queue be empty at 7:10?
Well, the answer to that question depends on whether at 7 o'clock whether the queue was huge or not.
So knowing something about the state of the queue right now gives me relevant information about what may happen in the future.
So what is the state of the system?
Therefore we're brought to start using this term.
So the state basically corresponds to anything that's relevant.
Anything that's happening right now that's kind of relevant to what may happen in the future.
Knowing the size of the queue right now, is useful information for me to make predictions about what may happen 2 minutes later from now.
So in this particular example, a reasonable choice for the state is to just count how many customers we have in the queue.
And let's assume that our supermarket building is not too big, so it can only hold 10 people.
So we're going to limit the states.
Instead of going from 0 to infinity, we're going to truncate our model at ten.
So we have 11 possible states, corresponding to 0 customers in queue, 1 customer in queue, 2 customers, and so on, all the way up to 10.
So these are the different possible states of the system, assuming that the store cannot handle more than 10 customers.
So this is the first step, to write down the set of possible states for our system.
Then the next thing to do is to start describing the possible transitions between the states.
At any given time step, what are the things that can happen?
We can have a customer arrival, which moves the state 1 higher.
We can have a customer departure, which moves the state 1 lower.
There's a possibility that nothing happens, in which case the state stays the same.
And there's also the possibility of having simultaneously an arrival and a departure, in which case the state again stays the same.
So let's write some representative probabilities.
If we have 2 customers, the probability that during this step we go down, this is the probability that we have a service completion, but to no customer arrival.
So this is the probability associated with this transition.
The other possibility is that there's a customer arrival, which happens with probability p, and we do not have a customer departure, and so the probability of that particular transition is this number.
And then finally, the probability that we stay in the same state, this can happen in 2 possible ways.
One way is that we have an arrival and a departure simultaneously.
And the other possibility is that we have no arrival and no departure, so that the state stays the same.
So these transition probabilities would be the same starting from any other states, state 3, or state 9, and so on.
Transition probabilities become a little different at the borders, at the boundaries of this diagram, because if you're in a state 0, then you cannot have any customer departures.
There's no one to be served, but there is a probability p that the customer arrives, in which case the number of customers in the system goes to 1.
Then probability 1-p, nothing happens.
Similarly with departures, if the system is full, there's no room for another arrival.
But we may have a departure that happens with probability q, and nothing happens with probability 1-q.
So this is the full transition diagram annotated with transition probabilities.
And this is a complete description of a discrete time, finite state Markov chain.
So this is a complete probabilistic model.
Once you have all of these pieces of information, you can start calculating things, and trying to predict what's going to happen in the future.
Now let us abstract from this example and come up with a more general definition.
So we have this concept of the state which describes the current situation in the system that we're looking at.
The current state is random, so we're going to think of it as a random variable Xn is the state, and transitions after the system started operating.
So the system starts operating at some initial state X0, and after n transitions, it moves to state Xn.
Now we have a set of possible states.
State 1 state 2, state 3, and in general, state i and state j.
To keep things simple, we assume that the set of possible states is a finite set.
As you can imagine, we can have systems in which the state space is going to be infinite.
It could be discrete, or continuous.
But all that is more difficult and more complicated.
It makes sense to start from the simplest possible setting where we just deal with the finite state space.
And time is discrete, so we can think of this state in the beginning, after 1 transition, 2 transitions, and so on.
So we're in discrete time and we have finite in many states.
So the system starts somewhere, and at every time step, the state is, let's say, here.
A whistle blows, and the state jumps to a random next state.
So it may move here, or it may move there, or it may move here, or it might stay in the place.
So one possible transition is the transition before you jump, and just land in the same place where you started from.
Now we want to describe the statistics of these transitions.
If I am at that state, how likely is it to that, next time, I'm going to find myself at that state?
Well, we describe the statistics of this transition by writing down a transition probability, the transition probability of going from state 3 to state 1.
So this transition probability is to be thought of as a conditional probability.
Given that right now I am at state i what is the probability that next time I find myself at state j?
So given that right now I am at state 3, P31 is the probability that the next time I'm going to find myself at state 1.
Similarly here, we would have a probability P3i, which is the probability that given that right now I'm at state 3, next time I'm going to find myself at state i.
Now one can write such conditional probabilities down in principle, but we need to make-- so you might think of this as a definition here, but we need to make one additional big assumption, and this is the assumption that to make a process to be a Markov process.
This is the so-called Markov property, and here's what it says.
Let me describe it first in words here.
Every time that I find myself at state 3, the probability that next time I'm going to find myself at state 1 is this particular number, no matter how I got there.
That is, this transition probability is not affected by the past of the process.
It doesn't care about what path I used to find myself at state 3.
Mathematically, it means the following.
You have this transition probability that from state i jump to state j.
Suppose that I gave you some additional information, that I told you everything else that happened in the past of the process, everything that happened, how did you get to state i?
The assumption we're making is that this information about the past has no bearing in making predictions about the future, as long as you know where you are right now.
So if I tell you, right now, you are at state i, and by the way, you got there by following a particular path, you can ignore the extra information of the particular path that you followed.
You only take into account where you are right now.
So every time you find yourself at that state, no matter how you got there, you will find yourself next time at state 1 with probability P31.
So the past has no bearing into the future, as long as you know where you are sitting right now.
For this property to happen, you need to choose your state carefully in the right way.
In that sense, the states needs to include any information that's relevant about the future of the system.
Anything that's not in the state is not going to play a role, but the state needs to have all the information that's relevant in determining what kind of transitions are going to happen next.
So to take an example, before you go to Markov process, just from the deterministic world, if you have a ball that's flying up in the air, and you want to make predictions about the future.
If I tell you that the state of the ball is the position of the ball at the particular time, is that enough for you to make predictions where the ball is going to go next?
No.
You need to know both the position and the velocity.
If you know position and velocity, you can make predictions about the future.
So the state of a ball that's flying is position together with velocity.
If you were to just take position, that would not be enough information, because if I tell you current position, and then I tell you past position, you could use the information from the past position to complete the trajectory and to make the prediction.
So information from the past is useful if you don't know the velocity.
But if both position and velocity, you don't care how you got there, or what time you started.
From position and velocity, you can make predictions about the future.
So there's a certain art, or a certain element of thinking, a non-mechanical aspect into problems of this kind, to figure out which is the right state variable.
When you define the state of your system, you need to define it in such a way that includes all information that has been accumulated that has some relevance for the future.
So the general process for coming up with a Markov model is to first make this big decision of what your state variable is going to be.
Then you write down if it may be a picture of the different states.
Then you identify the possible transitions.
So sometimes the diagram that you're going to have will not include all the possible arcs.
You would only show those arcs that correspond to transitions that are possible.
For example, in the supermarket example, we did not have a transition from state 2 to state 5, because that cannot happen.
You can only have 1 arrival at any time.
So in the diagram, we only showed the possible transitions.
And for each of the possible transitions, then you work with the description of the model to figure out the correct transition probability.
So you got the diagram by writing down transition probabilities.
OK, so suppose you got your Markov model.
What will you do with it?
Well, what do we need models for?
We need models in order to make predictions, to make probabilistic predictions.
So for example, I tell you that the process started in that state.
You let it run for some time.
Where do you think it's going to be 10 time steps from now?
That's a question that you might want to answer.
Since the process is random, there's no way for you to tell me exactly where it's going to be.
But maybe you can give me probabilities.
You can tell me, with so much probability, the state would be there.
With so much probability, the state would be there, and so on.
So our first exercise is to calculate those probabilities about what may happen to the process a number of steps in the future.
It's handy to have some notation in here.
So somebody tells us that this process starts at the particular state i.
We let the process run for n transitions.
It may land at some state j, but that state j at which it's going to land is going to be random.
So we want to give probabilities.
Tell me, with what probability the state, n times steps later, is going to be that particular state j?
The shorthand notation is to use this symbol here for the n-step transition probabilities that you find yourself at state j given that you started at state i.
So the way these two indices are ordered, the way to think about them is that from i, you go to j.
So the probability that from i you go to j if you have n steps in front of you.
Some of these transition probabilities are, of course easy to write.
For example, in 0 transitions, you're going to be exactly where you started.
So this probability is going to be equal to 1 if i is equal to j, And 0 if i is different than j.
That's an easy one to write down.
If you have only 1 transition, what's the probability that 1 step later you find yourself in state j given that you started at state i?
What is this?
These are just the ordinary 1-step transition probabilities that we are given in the description of the problem.
So by definition, the 1-step transition probabilities are of this form.
This equality is correct just because of the way that we defined those two quantities.
Now we want to say something about the n-step transition probabilities when n is a bigger number.
OK.
So here, we're going to use the total probability theorem.
So we're going to condition in two different scenarios, and break up the calculation of this quantity, by considering the different ways that this event can happen.
So what is the event of interest?
The event of interest is the following.
At time 0 we start i.
We are interested in landing at time n at the particular state j.
Now this event can happen in several different ways, in lots of different ways.
But let us group them into subgroups.
One group, or one sort of scenario, is the following.
During the first n-1 time steps, things happen, and somehow you end up at state 1.
And then from state 1, in the next time step you make a transition to state j.
This particular arc here actually corresponds to lots and lots of different possible scenarios, or different spots, or different transitions.
In n-1 time steps, there's lots of possible ways by which you could end up at state 1.
Different paths through the state space.
But all of them together collectively have a probability, which is the (n-1)-step transition probability, that from state i, you end up at state 1 And then there's other possible scenarios.
Perhaps in the first n-1 time steps, you follow the trajectory that took you at state m. And then from state m, you did this transition, and you ended up at state j.
So this diagram breaks up the set of all possible trajectories from i to j into different collections, where each collection has to do with which one happens to be the state just before the last time step, just before time n. And we're going to condition on the state at time n-1.
So the total probability of ending up at state j is the sum of the probabilities of the different scenarios -- the different ways that you can get to state j.
If we look at that type of scenario, what's the probability of that scenario happening?
With probability Ri1(n-1), I find myself at state 1 at time n-1.
This is just by the definition of these multi-step transition probabilities.
This is the number of transitions.
The probability that from state i, I end up at state 1.
And then given that I found myself at state 1, with probability P1j, that's the transition probability, next time I'm going to find myself at state j.
So the product of these two is the total probability of my getting from state i to state j through state 1 at the time before.
Now where exactly did we use the Markov assumption here?
No matter which particular path we used to get from i to state 1, the probability that next I'm going to make this transition is that same number, P1j.
So that number does not depend on the particular path that I followed in order to get there.
If we didn't have the Markov assumption, we should have considered all possible individual trajectories here, and then we would need to use the transition probability that corresponds to that particular trajectory.
But because of the Markov assumption, the only thing that matters is that right now we are at state 1.
It does not matter how we got there.
So now once you see this scenario, then this scenario, and that scenario, and you add the probabilities of these different scenarios, you end up with this formula here, which is a recursion.
It tells us that once you have computed the (n-1)-step transition probabilities, then you can compute also the n-step transition probabilities.
This is a recursion that you execute or you run for all i's and j's simultaneously.
That is fixed.
And for a particular n, you calculate this quantity for all possible i's, j's, k's.
You have all of those quantities, and then you use this equation to find those numbers again for all the possible i's and j's.
Now this is formula which is always true, and there's a big idea behind the formula.
And now there's variations of this formula, depending on whether you're interested in something that's slightly different.
So for example, if you were to have a random initial state, somebody gives you the probability distribution of the initial state, so you're told that with probability such and such, you're going to start at state 1.
With that probability, you're going to start at state 2, and so on.
And you want to find the probability at the time n you find yourself at state j.
Well again, total probability theorem, you condition on the initial state.
With this probability you find yourself at that particular initial state, and given that this is your initial state, this is the probability that n time steps later you find yourself at state j.
Now building again on the same idea, you can run every recursion of this kind by conditioning at different times.
So here's a variation.
You start at state i.
After 1 time step, you find yourself at state 1, with probability pi1, and you find yourself at state m with probability Pim.
And once that happens, then you're going to follow some trajectories.
And there is a possibility that you're going to end up at state j after n-1 time steps.
This scenario can happen in many possible ways.
There's lots of possible paths from state 1 to state j.
There's many paths from state 1 to state j.
What is the collective probability of all these transitions?
This is the event that, starting from state 1, I end up at state j in n-1 time steps.
So this one has here probability R1j of n-1.
And similarly down here.
And then by using the same way of thinking as before, we get the formula that Rij(n) is the sum over all k's of Pik, and then the Rkj(n-1).
So this formula looks almost the same as this one, but it's actually different.
The indices and the way things work out are a bit different, but the basic idea is the same.
Here we use the total probability theory by conditioning on the state just 1 step before the end of our time horizon.
Here we use total probability theorem by conditioning on the state right after the first transition.
So this generally idea has different variations.
They're all valid, and depending on the context that you're dealing with, you might want to work with one of these or another.
So let's illustrate these calculations in terms of an example.
So in this example, we just have 2 states, and somebody gives us transition probabilities to be those particular numbers.
Let's write down the equations.
So the probability that starting from state 1, I find myself at state 1 n time steps later.
This can happen in 2 ways.
At time n-1, I might find myself at state 2.
And then from state 2, I make a transition back to state 1, which happens with probability-- why'd I put 2 there -- anyway, 0.2.
And another way is that from state 1, I go to state 1 in n-1 steps, and then from state 1 I stay where I am, which happens with probability 0.5.
So this is for R11(n).
Now R12(n), we can write a similar recursion for this one.
On the other hand, seems these are probabilities.
The state at time n is going to be either state 1 or state 2.
So these 2 numbers need to add to 1, so we can just write this as 1 - R11(n).
And this is an enough of a recursion to propagate R11 and R12 as time goes on.
So after n-1 transitions, either I find myself in state 2, and then there's a point to transition that I go to 1, or I find myself in state 1, which with that probability, and from there, I have probability 0.5 of staying where I am.
Now let's start calculating.
As we discussed before, if I start at state 1, after 0 transitions I'm certain to be at state , and I'm certain not to be at state 1.
If I start from state 1, I'm certain to not to be at state at that time, and I'm certain that I am right now, it's state 1.
After I make transition, starting from state 1, there's probability 0.5 that I stay at state 1.
And there's probability 0.5 that I stay at state 2.
If I were to start from state 2, the probability that I go to 1 in 1 time step is this transition that has probability 0.2, and the other 0.8.
OK.
So the calculation now becomes more interesting, if we want to calculate the next term.
How likely is that at time 2, I find myself at state 1?
In order to be here at state 1, this can happen in 2 ways.
Either the first transition left me there, and the second transition is the same.
So these correspond to this 0.5, that the first transition took me there, and the next transition was also of the same kind.
That's one possibility.
But there's another scenario.
In order to be at state 1 at time 2 -- this can also happen this way.
So that's the event that, after 1 transition, I got there.
And the next transition happened to be this one.
So this corresponds to 0.5 times 0.2.
It corresponds to taking the 1-step transition probability of getting there, times the probability that from state 2 I move to state 1, which in this case, is 0.2.
So basically we take this number, multiplied with 0.2, and then add those 2 numbers.
And after you add them, you get 0.35.
And similarly here, you're going to get 0.65.
And now to continue with the recursion, we keep doing the same thing.
We take this number times 0.5 plus this number times 0.2.
Add them up, you get the next entry.
Keep doing that, keep doing that, and eventually you will notice that the numbers start settling into a limiting value at 2/7.
And let's verify this.
If this number is 2/7, what is the next number going to be?
The next number is going to be 2/7 -- (not 2.7) -- it's going to be 2/7.
That's the probability that I find myself at that state, times 0.5-- that's the next transition that takes me to state 1 -- plus 5/7-- that would be the remaining probability that I find myself in state 2 -- times 1/5.
And so that gives me, again, 2/7.
So this calculation basically illustrates, if this number has become 2/7, then the next number is also going to be 2/7.
And of course this number here is going to have to be 5/7.
And this one would have to be again, the same, 5/7.
So the probability that I find myself at state 1, after a long time has elapsed, settles into some steady state value.
So that's an interesting phenomenon.
We just make this observation.
Now we can also do the calculation about the probability, starting from state 2.
And here, you do the calculations -- I'm not going to do them.
But after you do them, you find this probability also settles to 2/7 and this one also settles to 5/7.
So these numbers here are the same as those numbers.
What's the difference between these?
This is the probability that I find myself at state 1 given that I started at 1.
This is the probability that I find myself at state 1 given that I started at state 2.
These probabilities are the same, no matter where I started from.
So this numerical example sort of illustrates the idea that after the chain has run for a long time, what the state of the chain is, does not care about the initial state of the chain.
So if you start here, you know that you're going to stay here for some time, a few transitions, because this probability is kind of small.
So the initial state does that's tell you something.
But in the very long run, transitions of this kind are going to happen.
Transitions of that kind are going to happen.
There's a lot of randomness that comes in, and that randomness washes out any information that could come from the initial state of the system.
We describe this situation by saying that the Markov chain eventually enters a steady state.
Where a steady state, what does it mean it?
Does it mean the state itself becomes steady and stops at one place?
No, the state of the chain keeps jumping forever.
The state of the chain will keep making transitions, will keep going back and forth between 1 and 2.
So the state itself, the Xn, does not become steady in any sense.
What becomes steady are the probabilities that describe Xn.
That is, after a long time elapses, the probability that you find yourself at state 1 becomes a constant 2/7, and the probability that you find yourself in state 2 becomes a constant.
So jumps will keep happening, but at any given time, if you ask what's the probability that right now I am at state 1, the answer is going to be 2/7.
Incidentally, do the numbers sort of makes sense?
Why is this number bigger than that number?
Well, this state is a little more sticky than that state.
Once you enter here, it's kind of harder to get out.
So when you enter here, you spend a lot of time here.
This one is easier to get out, because the probability is 0.5, so when you enter there, you tend to get out faster.
So you keep moving from one to the other, but you tend to spend more time on that state, and this is reflected in this probability being bigger than that one.
So no matter where you start, there's 5/7 probability of being here, 2/7 probability being there.
So there were some really nice things that happened in this example.
The question is, whether things are always as nice for general Markov chains.
The two nice things that happened where the following-- as we keep doing this calculation, this number settles to something.
The limit exists.
The other thing that happens is that this number is the same as that number, which means that the initial state does not matter.
Is this always the case?
Is it always the case that as n goes to infinity, the transition probabilities converge to something?
And if they do converge to something, is it the case that the limit is not affected by the initial state i at which the chain started?
So mathematically speaking, the question we are raising is whether Rij(n) converges to something.
And whether that something to which it converges to has only to do with j.
It's the probability that you find yourself at state j, and that probability doesn't care about the initial state.
So it's the question of whether the initial state gets forgotten in the long run.
So the answer is that usually, or for nice chains, both of these things will be true.
You get the limit which does not depend on the initial state.
But if your chain has some peculiar or unique structure, this might not happen.
So let's think first about the issue of convergence.
So convergence, as n goes to infinity at a steady value, really means the following.
If I tell you a lot of time has passed, then you tell me, OK, the state of the probabilities are equal to that value without having to consult your clock.
If you don't have convergence, it means that Rij can keep going up and down, without settling to something.
So in order for you to tell me the value of Rij, you need to consult your clock to check if, right now, it's up or is it down.
So there's some kind of periodic behavior that you might get when you do not get convergence, and this example here illustrates it.
So what's happened in this example?
Starting from state 2, next time you go here, or there, with probability half.
And then next time, no matter where you are, you move back to state 2.
So this chain has some randomness, but the randomness is kind of limited type.
You go out, you come in.
You go out, you come in.
So there's a periodic pattern that gets repeated.
It means that if you start at state 2 after an even number of steps, you are certain to be back at state 2.
So this probability here is 1.
On the other hand, if the number of transitions is odd, there's no way that you can be at your initial state.
If you start here, at even times you would be here, at odd times you would be there or there.
So this probability is 0.
As n goes to infinity, these probabilities, the n-step transition probability does not converge to anything.
It keeps alternating between 0 and 1.
So convergence fails.
This is the main mechanism by which convergence can fail if your chain has a periodic structure.
And we're going to discuss next time that, if periodicity absent, then we don't have an issue with convergence.
The second question if we have convergence, whether the initial state matters or not.
In the previous chain, where you could keep going back and forth between states 1 and 2 numerically, one finds that the initial state does not matter.
But you can think of situations where the initial state does matter.
Look at this chain here.
If you start at state 1, you stay at state 1 forever.
There's no way to escape.
So this means that R11(n) is 1 for all n. If you start at state 3, you will be moving between stage 3 and 4, but there's no way to go in that direction, so there's no way that you go to state 1.
And for that reason, R31 is 0 for all n. OK So this is a case where the initial state matters.
R11 goes to a limit, as n goes to infinity, because it's constant.
It's always 1 so the limit is 1.
R31 also has a limit.
It's 0 for all times.
So these are the long term probabilities of finding yourself at state 1.
But those long-term probabilities are affected by where you started.
If you start here, you're sure that's, in the long term, you'll be here.
If you start here, you're sure that, in the long term, you will not be there.
So the initial state does matter here.
And this is a situation where certain states are not accessible from certain other states, so it has something to do with the graph structure of our Markov chain.
Finally let's answer this question here, at least for large n's.
What do you think is going to happen in the long term if you start at state 2?
If you start at state 2, you may stay at state 2 for a random amount of time, but eventually this transition will happen, or that transition would happen.
Because of the symmetry, you are as likely to escape from state 2 in this direction, or in that direction, so there's probability 1/2 that, when the transition happens, the transition happens in that direction.
So for large N, you're certain that the transition does happen.
And given that the transition has happened, it has probability 1/2 that it has gone that particular way.
So clearly here, you see that the probability of finding yourself in a particular state is very much affected by where you started from.
So what we want to do next is to abstract from these two examples and describe the general structural properties that have to do with periodicity, and that have to do with what happened here with certain states, not being accessible from the others.
We're going to leave periodicity for next time.
But let's talk about the second kind of phenomenon that we have.
So here, what we're going to do is to classify the states in a transition diagram into two types, recurrent and transient.
So a state is said to be recurrent if the following is true.
If you start from the state i, you can go to some places, but no matter where you go, there is a way of coming back.
So what's an example for the recurrent state?
This one.
Starting from here, you can go elsewhere.
You can go to state 7.
You can go to state 6.
That's all where you can go to.
But no matter where you go, there is a path that can take you back there.
So no matter where you go, there is a chance, and there is a way for returning where you started.
Those states we call recurrent.
And by this, 8 is recurrent.
All of these are recurrent.
So this is recurrent, this is recurrent.
And this state 5 is also recurrent.
You cannot go anywhere from 5 except to 5 itself.
Wherever you can go, you can go back to where you start.
So this is recurrent.
If it is not the recurrent, we say that it is transient.
So what does transient mean?
You need to take this definition, and reverse it.
Transient means that, starting from i, there is a place to which you could go, and from which you cannot return.
If it's recurrent, anywhere you go, you can always come back.
Transient means there are places where you can go from which you cannot come back.
So state 1 is recurrent - because starting from here, there's a possibility that you get there, and then there's no way back.
State 4 is recurrent, starting from 4, there's somewhere you can go and-- sorry, transient, correct.
State 4 is transient starting from here, there are places where you could go, and from which you cannot come back.
And in this particular diagram, all these 4 states are transients.
Now if the state is transient, it means that there is a way to go somewhere where you're going to get stuck and not to be able to come.
As long as your state keeps circulating around here, eventually one of these transitions is going to happen, and once that happens, then there's no way that you can come back.
So that transient state will be visited only a finite number of times.
You will not be able to return to it.
And in the long run, you're certain that you're going to get out of the transient states, and get to some class of recurrent states, and get stuck forever.
So, let's see, in this diagram, if I start here, could I stay in this lump of states forever?
Well as long as I'm staying in this type of states, I would keep visiting states 1 and 2 Each time that I visit state 2, there's going to be positive probability that I escape.
So in the long run, if I were to stay here, I would visit state 2 an infinite number of times, and I would get infinite chances to escape.
But if you have infinite chances to escape, eventually you will escape.
So you are certain that with probability 1, starting from here, you're going to move either to those states, or to those states.
So starting from transient states, you only stay at the transient states for random but finite amount of time.
And after that happens, you end up in a class of recurrent states.
And when I say class, what they mean is that, in this picture, I divide the recurrent states into 2 classes, or categories.
What's special about them?
These states are recurrent.
These states are recurrent.
But there's no communication between the 2.
If you start here, you're stuck here.
If you start here, you are stuck there.
And this is a case where the initial state does matter, because if you start here, you get stuck here.
You start here, you get stuck there.
So depending on the initial state, that's going to affect the long term behavior of your chain.
So the guess you can make at this point is that, for the initial state to not matter, we should not have multiple recurrent classes.
We should have only 1.
But we're going to get back to this point next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So today, we're going to start by taking stock of what we discussed last time, review the definition of Markov chains.
And then most of the lecture, we're going to concentrate on their steady-state behavior.
Meaning, we're going to look at what does a Markov chain do if it has run for a long time.
What can we say about the probabilities of the different states?
So what I would like to repeat is a statement I made last time that Markov chains is a very, very useful class of models.
Pretty much anything in the real world can be approximately modeled by a Markov chain provided that you set your states in the proper way.
So we're going to see some examples.
You're going to see more examples in the problems you're going to do in homework and recitation.
On the other hand, we're not going to go too deep into examples.
Rather, we're going to develop the general methodology.
OK. All right.
Markov models can be pretty general.
They can run in continuous or discrete time.
They can have continuous or discrete state spaces.
In this class, we're going to stick just to the case where the state space is discrete and time is discrete because this is the simplest case.
And also, it's the one where you build your intuition before going to more general cases perhaps in other classes.
So the state is discrete and finite.
There's a finite number of states.
At any point in time, the process is sitting on one of those states.
Time is discrete, so at each unit of time, somebody whistles and then the state jumps.
And when it jumps, it can either land in the same place, or it can land somewhere else.
And the evolution of the process is described by transition probabilities.
Pij is the probability that the next state is j given that the current state is i.
And the most important property that the Markov chain has, the definition of a Markov chain or Markov process, is that this probability, Pij, is the same every time that you land at state i -- no matter how you got there and also no matter what time it is.
So the model we have is time homogeneous, which basically means that those transition probabilities are the same at every time.
So the model is time invariant in that sense.
So we're interested in what the chain or the process is going to do in the longer run.
So we're interested, let's say, in the probability that starting at a certain state, n times steps later, we find ourselves at some particular state j. Fortunately, we can calculate those probabilities recursively.
Of course, at the first time 1, the probability of being 1 time later at state j given that we are right now at state i, by definition, this is just the transition probabilities.
So by knowing these, we can start a recursion that tells us the transition probabilities for more than n steps.
This recursion, it's a formula.
It's always true.
You can copy it or memorize it.
But there is a big idea behind that formula that you should keep in mind.
And basically, the divide and conquer idea.
It's an application of the total probability law.
So let's fix i.
The probability that you find yourself at state j, you break it up into the probabilities of the different ways that you can get to state j.
What are those different ways?
The different ways are the different states k at which you might find yourself the previous time.
So with some probability, with this probability, you find yourself at state k the previous time.
And then with probability Pkj, you make a transition to state j.
So this is a possible scenario that takes you to state j after n transitions.
And by summing over all the k's, then we have considered all the possible scenarios.
Now, before we move to the more serious stuff, let's do a little bit of warm up to get a handle on how we use transition probabilities to calculate more general probabilities, then talk about some structural properties of Markov chains, and then eventually get to the main business of today, which is a steady-state behavior.
So somebody gives you this chain, and our convention is that those arcs that are not shown here corresponds to 0 probabilities.
And each one of the arcs that's shown has a non-zero probability, and somebody gives it to us.
Suppose that the chain starts at state 1.
We want to calculate the probability that it follows this particular path.
That is, it goes to 2, then to 6, then to 7.
How do we calculate the probability of a particular trajectory?
Well, this is the probability-- so it's the probability of the trajectory from 1 that you go to 2, then to 6, then to 7.
So the probability of this trajectory is we use the multiplication rule.
The probability of several things happening is the probability that the first thing happens, which is a transition from 1 to 2.
And then given that we are at state 2, we multiply with a conditional probability that the next event happens.
That is, that X2 is equal to 6 given that right now, we are at state 1.
And that conditional probability is just P26.
And notice that this conditional probability applies no matter how we got to state 2.
This is the Markov assumption.
So we don't care about the fact that we came in in a particular way.
Given that we came in here, this probability P26, that the next transition takes us to 6.
And then given that all that stuff happened, so given that right now, we are at state 6, we need to multiply with a conditional probability that the next transition takes us to state 7.
And this is just the P67.
So to find the probability of following a specific trajectory, you just multiply the transition probabilities along the particular trajectory.
Now, if you want to calculate something else, such as for example, the probability that 4 time steps later, I find myself at state 7 given that they started, let's say, at this state.
How do you calculate this probability?
One way is to use the recursion for the Rijs that we know that it is always valid.
But for short and simple examples, and with a small time horizon, perhaps you can do this in a brute force way.
What would be the brute force way?
This is the event that 4 time steps later, I find myself at state 7.
This event can happen in various ways.
So we can take stock of all the different ways, and write down their probabilities.
So starting from 2.
One possibility is to follow this trajectory, 1 transition, 2 transitions, 3 transitions, 4 transitions.
And that takes me to state 7.
What's the probability of this trajectory?
It's P26 times P67 times P76 and then times P67.
So this is a probability of a particular trajectory that takes you to state 7 after 4 time steps.
But there's other trajectories as well.
What could be it?
I might start from state 2, go to state 6, stay at state 6, stay at state 6 once more.
And then from state 6, go to state 7.
And so there must be one more.
What's the other one?
I guess I could go 1, 2, 6, 7.
OK. That's the other trajectory.
Plus P21 times P12 times P26 and times P67.
So the transition probability, the overall probability of finding ourselves at state 7, is broken down as the sum of the probabilities of all the different ways that I can get to state 7 in exactly 4 steps.
So we could always do that without knowing much about Markov chains or the general formula for the Rij's that we had.
What's the trouble with this procedure?
The trouble with this procedure is that the number of possible trajectories becomes quite large if this index is a little bigger.
If this 4 was 100, and you ask how many different trajectories of length 100 are there to take me from here to there, that number of trajectories would be huge.
It grows exponentially with the time horizon.
And this kind of calculation would be impossible.
The basic equation, the recursion that have for the Rij's is basically a clever way of organizing this computation so that the amount of computation that you do is not exponential in the time horizon.
Rather, it's sort of linear with the time horizon.
For each time step you need in the time horizon, you just keep repeating the same iteration over and over.
OK. Now, the other thing that we discussed last time, briefly, was a classification of the different states of the Markov chain in two different types.
A Markov chain, in general, has states that are recurrent, which means that from a recurrent state, I can go somewhere else.
But from that somewhere else, there's always some way of coming back.
So if you have a chain of this form, no matter where you go, no matter where you start, you can always come back where you started.
States of this kind are called recurrent.
On the other hand, if you have a few states all this kind, a transition of this type, then these states are transient in the sense that from those states, it's possible to go somewhere else from which place there's no way to come back where you started.
The general structure of a Markov chain is basically a collection of transient states.
You're certain that you are going to leave the transient states eventually.
And after you leave the transient states, you enter into a class of states in which you are trapped.
You are trapped if you get inside here.
You are trapped if you get inside there.
This is a recurrent class of states.
From any state, you can get to any other state within this class.
That's another recurrent class.
From any state inside here, you can get anywhere else inside that class.
But these 2 classes, you do not communicate.
If you start here, there's no way to get there.
If you have 2 recurrent classes, then it's clear that the initial conditions of your Markov chain matter in the long run.
If you start here, you will be stuck inside here for the long run and similarly about here.
So the initial conditions do make a difference.
On the other hand, if this class was not here and you only had that class, what would happen to the chain?
Let's say you start here.
You move around.
At some point, you make that transition.
You get stuck in here.
And inside here, you keep circulating, because of the randomness, you keep visiting all states over and over.
And hopefully or possibly, in the long run, it doesn't matter exactly what time it is or where you started, but the probability of being at that particular state is the same no matter what the initial condition was.
So with a single recurrent class, we hope that the initial conditions do not matter.
With 2 or more recurrent classes, initial conditions will definitely matter.
So how many recurrent classes we have is something that has to do with the long-term behavior of the chain and the extent to which initial conditions matter.
Another way that initial conditions may matter is if a chain has a periodic structure.
There are many ways of defining periodicity.
The one that I find sort of the most intuitive and with the least amount of mathematical symbols is the following.
The state space of a chain is said to be periodic if you can lump the states into a number of clusters called d clusters or groups.
And the transition diagram has the property that from a cluster, you always make a transition into the next cluster.
So here d is equal to 2.
We have two subsets of the state space.
Whenever we're here, next time we'll be there.
Whenever we're here, next time we will be there.
So this chain has a periodic structure.
There may be still some randomness.
When I jump from here to here, the state to which I jump may be random, but I'm sure that I'm going to be inside here.
And then next time, I will be sure that I'm inside here.
This would be a structure of a diagram in which we have a period of 3.
If you start in this lump, you know that the next time, you would be in a state inside here.
Next time, you'll be in a state inside here, and so on.
So these chains certainly have a periodic structure.
And that periodicity gets maintained.
If I start, let's say, at this lump, at even times, I'm sure I'm here.
At odd times, I'm sure I am here.
So the exact time does matter in determining the probabilities of the different states.
And in particular, the probability of being at the particular state cannot convert to a state value.
The probability of being at the state inside here is going to be 0 for all times.
In general, it's going to be some positive number for even times.
So it goes 0 positive, zero, positive, 0 positive.
Doesn't settle to anything.
So when we have periodicity, we do not expect the states probabilities to converge to something, but rather, we expect them to oscillate.
Now, how can we tell whether a Markov chain is periodic or not?
There are systematic ways of doing it, but usually with the types of examples we see in this class, we just eyeball the chain, and we tell whether it's periodic or not.
So is this chain down here, is it the periodic one or not?
How many people think it's periodic?
No one.
One.
How many people think it's not periodic?
OK. Not periodic?
Let's see.
Let me do some drawing here.
OK. Is it periodic?
It is.
From a red state, you can only get to a white state.
And from a white state, you can only get to a red state.
So this chain, even though it's not apparent from the picture, actually has this structure.
We can group the states into red states and white states.
And from reds, we always go to a white, and from a white, we always go to a red.
So this tells you that sometimes eyeballing is not as easy.
If you have lots and lots of states, you might have some trouble doing this exercise.
On the other hand, something very useful to know.
Sometimes it's extremely easy to tell that the chain is not periodic.
What's that case?
Suppose that your chain has a self-transition somewhere.
Then automatically, you know that your chain is not periodic.
So remember, the definition of periodicity requires that if you are in a certain group of states, next time, you will be in a different group.
But if you have self-transitions, that property is not true.
If you have a possible self-transition, it's possible that you stay inside your own group for the next time step.
So whenever you have a self-transition, this implies that the chain is not periodic.
And usually that's the simplest and easy way that we can tell most of the time that the chain is not periodic.
So now, we come to the big topic of today, the central topic, which is the question about what does the chain do in the long run.
The question we are asking and which we motivated last time by looking at an example.
It's something that did happen in our example of last time.
So we're asking whether this happens for every Markov chain.
We're asking the question whether the probability of being at state j at some time n settles to a steady-state value.
Let's call it pi sub j.
That these were asking whether this quantity has a limit as n goes to infinity, so that we can talk about the steady-state probability of state j.
And furthermore, we asked whether the steady-state probability of that state does not depend on the initial state.
In other words, after the chain runs for a long, long time, it doesn't matter exactly what time it is, and it doesn't matter where the chain started from.
You can tell me the probability that the state is a particular j is approximately the steady-state probability pi sub j.
It doesn't matter exactly what time it is as long as you tell me that a lot of time has elapsed so that n is a big number.
So this is the question.
We have seen examples, and we understand that this is not going to be the case always.
For example, as I just discussed, if we have 2 recurrent classes, where we start does matter.
The probability pi(j) of being in that state j is going to be 0 if we start here, but it would be something positive if we were to start in that lump.
So the initial state does matter if we have multiple recurrent classes.
But if we have only a single class of recurrent states from each one of which you can get to any other one, then we don't have that problem.
Then we expect initial conditions to be forgotten.
So that's one condition that we need.
And then the other condition that we need is that the chain is not periodic.
If the chain is periodic, then these Rij's do not converge.
They keep oscillating.
If we do not have periodicity, then there is hope that we will get the convergence that we need.
It turns out this is the big theory of Markov chains-- the steady-state convergence theorem.
It turns out that yes, the rijs do converge to a steady-state limit, which we call a steady-state probability as long as these two conditions are satisfied.
We're not going to prove this theorem.
If you're really interested, the end of chapter exercises basically walk you through a proof of this result, but it's probably a little too much for doing it in this class.
What is the intuitive idea behind this theorem?
Let's see.
Let's think intuitively as to why the initial state doesn't matter.
Think of two copies of the chain that starts at different initial states, and the state moves randomly.
As the state moves randomly starting from the two initial states a random trajectory.
as long as you have a single recurrent class at some point, and you don't have periodicity at some point, those states, those two trajectories, are going to collide.
Just because there's enough randomness there.
Even though we started from different places, the state is going to be the same.
After the state becomes the same, then the future of these trajectories, probabilistically, is the same because they both started at the same state.
So this means that the initial conditions stopped having any influence.
That's sort of the high-level idea of why the initial state gets forgotten.
Even if you started at different initial states, at some time, you may find yourself to be in the same state as the other trajectory.
And once that happens, your initial conditions cannot have any effect into the future.
All right.
So let's see how we might calculate those steady-state probabilities.
The way we calculate the steady-state probabilities is by taking this recursion, which is always true for the end-step transition probabilities, and take the limit of both sides.
The limit of this side is the steady-state probability of state j, which is pi sub j.
The limit of this side, we put the limit inside the summation.
Now, as n goes to infinity, n - also goes to infinity.
So this Rik is going to be the steady-state probability of state k starting from state i.
Now where we started doesn't matter.
So this is just the steady-state probability of state k. So this term converges to that one, and this gives us an equation that's satisfied by the steady-state probabilities.
Actually, it's not one equation.
We get one equation for each one of the j's.
So if we have 10 possible states, we're going to get the system of 10 linear equations.
In the unknowns, pi(1) up to pi(10).
OK. 10 unknowns, 10 equations.
You might think that we are in business.
But actually, this system of equations is singular.
0 is a possible solution of this system.
If you plug pi equal to zero everywhere, the equations are satisfied.
It does not have a unique solution, so maybe we need one more condition to get the uniquely solvable system of linear equations.
It turns out that this system of equations has a unique solution.
If you impose an additional condition, which is pretty natural, the pi(j)'s are the probabilities of the different states, so they should add to 1.
So you want this one equation to the mix.
And once you do that, then this system of equations is going to have a unique solution.
And so we can find the steady-state probabilities of the Markov chain by just solving these linear equations, which is numerically straightforward.
Now, these equations are quite important.
I mean, they're the central point in the Markov chain.
They have a name.
They're called the balance equations.
And it's worth interpreting them in a somewhat different way.
So intuitively, one can sometimes think of probabilities as frequencies.
For example, if I toss an unbiased coin, probability 1/2 of heads, you could also say that if I keep flipping that coin, in the long run, 1/2 of the time, I'm going to see heads.
Similarly, let's try an interpretation of this pi(j), the steady-state probability, the long-term probability of finding myself at state j.
Let's try to interpret it as the frequency with which I find myself at state j if I run a very, very long trajectory over that Markov chain.
So the trajectory moves around, visits states.
It visits the different states with different frequencies.
And let's think of the probability that you are at a certain state as being sort of the same as the frequency of visiting that state.
This turns out to be a correct statement.
If you were more rigorous, you would have to prove it.
But it's an interpretation which is valid and which gives us a lot of intuition about what these equation is saying.
So let's think as follows.
Let's focus on a particular state j, and think of transitions into the state j versus transitions out of the state j, or transitions into j versus transitions starting from j.
So transition starting from that includes a self-transition.
Ok.
So how often do we get a transition, if we interpret the pi(j)'s as frequencies, how often do we get a transition into j?
Here's how we think about it.
A fraction pi(1) of the time, we're going to be at state 1.
Whenever we are at state 1, there's going to be a probability, P1j, that we make a transition of this kind.
So out of the times that we're at state 1, there's a frequency, P1j with which the next transition is into j.
So out of the overall number of transitions that happen at the trajectory, what fraction of those transitions is exactly of that kind?
That fraction of transitions is the fraction of time that you find yourself at 1 times the fraction with which out of one you happen to visit next state j.
So we interpreted this number as the frequency of transitions of this kind.
At any given time, our chain can do transitions of different kinds, transitions of the general form from some k, I go to some l. So we try to do some accounting.
How often does a transition of each particular kind happen?
And this is the frequency with which transitions of that particular kind happens.
Now, what's the total frequency of transitions into state j?
Transitions into state j can happen by having a transition from 1 to j, from 2 to j, or from state m to j.
So to find the total frequency with which we would observe transitions into j is going to be this particular sum.
Now, you are at state j if and only if the last transition was into state j.
So the frequency with which you are at j is the frequency with which transitions into j happen.
So this equation expresses exactly that statement.
The probability of being at state j is the sum of the probabilities that the last transition was into state j.
Or in terms of frequencies, the frequency with which you find yourself at state j is the sum of the frequencies of all the possible transition types that take you inside state j.
So that's a useful intuition to have, and we're going to see an example a little later that it gives us short cuts into analyzing Markov chains.
But before we move, let's revisit the example from last time.
And let us write down the balance equations for this example.
So the steady-state probability that I find myself at state 1 is the probability that the previous time I was at state 1 and I made a self-transition-- So the probability that I was here last time and I made a transition of this kind, plus the probability that the last time I was here and I made a transition of that kind.
So plus pi(2) times 0.2.
And similarly, for the other states, the steady-state probably that I find myself at state 2 is the probability that last time I was at state 1 and I made a transition into state 2, plus the probability that the last time I was at state 2 and I made the transition into state 1.
Now, these are two equations and two unknowns, pi(1) and pi(2).
But you notice that both of these equations tell you the same thing.
They tell you that 0.5pi(1) equals 0.2pi(2).
Either of these equations tell you exactly this if you move terms around.
So these two equations are not really two equations.
It's just one equation.
They are linearly dependent equations, and in order to solve the problem, we need the additional condition that pi(1) + pi(2) is equal to 1.
Now, we have our system of two equations, which you can solve.
And once you solve it, you find that pi(1) is 2/7 and pi(2) is 5/7.
So these are the steady state probabilities of the two different states.
If we start this chain, at some state, let's say state 1, and we let it run for a long, long time, the chain settles into steady state.
What does that mean?
It does not mean that the state itself enters steady state.
The state will keep jumping around forever and ever.
It will keep visiting both states once in a while.
So the jumping never ceases.
The thing that gets into steady state is the probability of finding yourself at state 1.
So the probability that you find yourself at state 1 at time one trillion is approximately 2/7.
The probability you find yourself at state 1 at time two trillions is again, approximately 2/7.
So the probability of being in that state settles into a steady value.
That's what the steady-state convergence means.
It's convergence of probabilities, not convergence of the process itself.
And again, the two main things that are happening in this example, and more generally, when we have a single class and no periodicity, is that the initial state does not matter.
There's enough randomness here so that no matter where you start, the randomness kind of washes out any memory of where you started.
And also in this example, clearly, we do not have periodicity because we have self arcs.
And this, in particular, implies that the exact time does not matter.
So now, we're going to spend the rest of our time by looking into a special class of chains that's a little easier to deal with, but still, it's an important class.
So what's the moral from here?
This was a simple example with two states, and we could find the steady-state probabilities by solving a simple system of two-by-two equations.
If you have a chain with 100 states, it's no problem for a computer to solve a system of 100-by-100 equations.
But you can certainly not do it by hand, and usually, you cannot get any closed-form formulas, so you do not necessarily get a lot of insight.
So one looks for special structures or models that maybe give you a little more insight or maybe lead you to closed-form formulas.
And an interesting subclass of Markov chains in which all of these nice things do happen, is the class of birth/death processes.
So what's a birth/death process?
It's a Markov chain who's diagram looks basically like this.
So the states of the Markov chain start from 0 and go up to some finite integer m. What's special about this chain is that if you are at a certain state, next time you can either go up by 1, you can go down by 1, or you can stay in place.
So it's like keeping track of some population at any given time.
One person gets born, or one person dies, or nothing happens.
Again, we're not accounting for twins here.
So we're given this structure, and we are given the transition probabilities, the probabilities associated with transitions of the different types.
So we use P's for the upward transitions, Q's for the downward transitions.
An example of a chain of this kind was the supermarket counter model that we discussed last time.
That is, a customer arrives, so this increments the state by 1.
Or a customer finishes service, in which case, the state gets decremented by 1, or nothing happens in which you stay in place, and so on.
In the supermarket model, these P's inside here were all taken to be equal because we assume that the arrival rate was sort of constant at each time slot.
But you can generalize a little bit by assuming that these transition probabilities P1 here, P2 there, and so on may be different from state to state.
So in general, from state i, there's going to be a transition probability Pi that the next transition is upwards.
And there's going to be a probability Qi that the next transition is downwards.
And so from that state, the probability that the next transition is downwards is going to be Q_(i+1).
So this is the structure of our chain.
As I said, it's a crude model of what happens at the supermarket counter but it's also a good model for lots of types of service systems.
Again, you have a server somewhere that has a buffer.
Jobs come into the buffer.
So the buffer builds up.
The server processes jobs, so the buffer keeps going down.
And the state of the chain would be the number of jobs that you have inside your buffer.
Or you could be thinking about active phone calls out of a certain city.
Each time that the phone call is placed, the number of active phone calls goes up by 1.
Each time that the phone call stops happening, is terminated, then the count goes down by 1.
So it's for processes of this kind that a model with this structure is going to show up.
And they do show up in many, many models.
Or you can think about the number of people in a certain population that have a disease.
So 1 more person gets the flu, the count goes up.
1 more person gets healed, the count goes down.
And these probabilities in such an epidemic model would certainly depend on the current state.
If lots of people already have the flu, the probability that another person catches it would be pretty high.
Whereas, if no one has the flu, then the probability that you get a transition where someone catches the flu, that probability would be pretty small.
So the transition rates, the incidence of new people who have the disease definitely depends on how many people already have the disease.
And that motivates cases where those P's, the upward transition probabilities, depend on the state of the chain.
So how do we study this chain?
You can sit down and write the system of n linear equations in the pi's.
And this way, find the steady-state probabilities of this chain.
But this is a little harder.
It's more work than one actually needs to do.
There's a very clever shortcut that applies to birth/death processes.
And it's based on the frequency interpretation that we discussed a little while ago.
Let's put a line somewhere in the middle of this chain, and focus on the relation between this part and that part in more detail.
So think of the chain continuing in this direction, that direction.
But let's just focus on 2 adjacent states, and look at this particular cut.
What is the chain going to do?
Let's say it starts here.
It's going to move around.
At some point, it makes a transition to the other side.
And that's a transition from i to i+1.
It stays on the other side for some time.
It gets here, and eventually, it's going to make a transition to this side.
Then it keeps moving and so on.
Now, there's a certain balance that must be obeyed here.
The number of upward transitions through this line cannot be very different from the number of downward transitions.
Because we cross this way, then next time, we'll cross that way.
Then next time, we'll cross this way.
We'll cross that way.
So the frequency with which transitions of this kind occur has to be the same as the long-term frequency that transitions of that kind occur.
You cannot go up 100 times and go down only 50 times.
If you have gone up 100 times, it means that you have gone down 99, or 100, or 101, but nothing much more different than that.
So the frequency with which transitions of this kind get observed.
That is, out of a large number of transitions, what fraction of transitions are of these kind?
That fraction has to be the same as the fraction of transitions that happened to be of that kind.
What are these fractions?
We discussed that before.
The fraction of times at which transitions of this kind are observed is the fraction of time that we happen to be at that state.
And out of the times that we are in that state, the fraction of transitions that happen to be upward transitions.
So this is the frequency with which transitions of this kind are observed.
And with the same argument, this is the frequency with which transitions of that kind are observed.
Since these two frequencies are the same, these two numbers must be the same, and we get an equation that relates the Pi to P_(i+1).
This has a nice form because it gives us a recursion.
If we knew pi(i), we could then immediately calculate pi(i+1).
So it's a system of equations that's very easy to solve almost.
But how do we get started?
If I knew pi(0), I could find by pi(1) and then use this recursion to find pi(2), pi(3), and so on.
But we don't know pi(0).
It's one more unknown.
It's an unknown, and we need to actually use the extra normalization condition that the sum of the pi's is 1.
And after we use that normalization condition, then we can find all of the pi's.
So you basically fix pi(0) as a symbol, solve this equation symbolically, and everything gets expressed in terms of pi(0).
And then use that normalization condition to find pi(0), and you're done.
Let's illustrate the details of this procedure on a particular special case.
So in our special case, we're going to simplify things now by assuming that all those upward P's are the same, and all of those downward Q's are the same.
So at each point in time, if you're sitting somewhere in the middle, you have probability P of moving up and probability Q of moving down.
This rho, the ratio of P/Q is frequency of going up versus frequency of going down.
If it's a service system, you can think of it as a measure of how loaded the system is.
If P is equal to Q, it's means that if you're at this state, you're equally likely to move left or right, so the system is kind of balanced.
The state doesn't have a tendency to move in this direction or in that direction.
If rho is bigger than 1 so that P is bigger than Q, it means that whenever I'm at some state in the middle, I'm more likely to move right rather than move left, which means that my state, of course it's random, but it has a tendency to move in that direction.
And if you think of this as a number of customers in queue, it means your system has the tendency to become loaded and to build up a queue.
So rho being bigger than 1 corresponds to a heavy load, where queues build up.
Rho less than 1 corresponds to the system where queues have the tendency to drain down.
Now, let's write down the equations.
We have this recursion P_(i+1) is Pi times Pi over Qi.
In our case here, the P's and the Q's do not depend on the particular index, so we get this relation.
And this P over Q is just the load factor rho.
Once you look at this equation, clearly you realize that by pi(1) is rho times pi(0).
pi(2) is going to be -- So we'll do it in detail.
So pi(1) is pi(0) times rho.
pi(2) is pi(1) times rho, which is pi(0) times rho-squared.
And then you continue doing this calculation.
And you find that you can express every pi(i) in terms of pi(0) and you get this factor of rho^i.
And then you use the last equation that we have -- that the sum of the probabilities has to be equal to 1.
And that equation is going to tell us that the sum over all i's from 0 to m of pi(0) rho to the i is equal to 1.
And therefore, pi(0) is 1 over (the sum over the rho to the i for i going from 0 to m).
So now we found pi(0), and by plugging in this expression, we have the steady-state probabilities of all of the different states.
Let's look at some special cases of this.
Suppose that rho is equal to 1.
If rho is equal to 1, then pi(i) is equal to pi(0).
It means that all the steady-state probabilities are equal.
It's means that every state is equally likely in the long run.
So this is an example.
It's called a symmetric random walk.
It's a very popular model for modeling people who are drunk.
So you start at a state at any point in time.
Either you stay in place, or you have an equal probability of going left or going right.
There's no bias in either direction.
You might think that in such a process, you will tend to kind of get stuck near one end or the other end.
Well, it's not really clear what to expect.
It turns out that in such a model, in the long run, the drunk person is equally likely to be at any one of those states.
The steady-state probability is the same for all i's if rho is equal to 1.
And so if you show up at a random time, and you ask where is my state, you will be told it's equally likely to be at any one of those places.
So let's make that note.
If rho equal to 1, implies that all the pi(i)'s are 1/(M+1) -- M+1 because that's how many states we have in our model.
Now, let's look at a different case.
Suppose that M is a huge number.
So essentially, our supermarket has a very large space, a lot of space to store their customers.
But suppose that the system is on the stable side.
P is less than Q, which means that there's a tendency for customers to be served faster than they arrive.
The drift in this chain, it tends to be in that direction.
So when rho is less than 1, which is this case, and when M is going to infinity, this infinite sum is the sum of a geometric series.
And you recognize it (hopefully) -- this series is going to 1/(1-rho).
And because it's in the denominator, pi(0) ends up being 1-rho.
So by taking the limit as M goes to infinity, in this case, and when rho is less than 1 so that this series is convergent, we get this formula.
So we get the closed-form formula for the pi(i)'s.
In particular, pi(i) is (1- rho)(rho to the i).
to So these pi(i)'s are essentially a probability distribution.
They tell us if we show up at time 1 billion and we ask, where is my state?
You will be told that the state is 0.
Your system is empty with probability 1-rho, minus or there's one customer in the system, and that happens with probability (rho - 1) times rho.
And it keeps going down this way.
And it's pretty much a geometric distribution except that it has shifted so that it starts at 0 whereas the usual geometric distribution starts at 1.
So this is a mini introduction into queuing theory.
This is the first and simplest model that one encounters when you start studying queuing theory.
This is clearly a model of a queueing phenomenon such as the supermarket counter with the P's corresponding to arrivals, the Q's corresponding to departures.
And this particular queuing system when M is very, very large and rho is less than 1, has a very simple and nice solution in closed form.
And that's why it's very much liked.
And let me just take two seconds to draw one last picture.
So this is the probability of the different i's.
It gives you a PMF.
This PMF has an expected value.
And the expectation, the expected number of customers in the system, is given by this formula.
And this formula, which is interesting to anyone who tries to analyze a system of this kind, tells you the following.
That as long as a rho is less than 1, then the expected number of customers in the system is finite.
But if rho becomes very close to 1 -- So if your load factor is something like .99, you expect to have a large number of customers in the system at any given time.
OK. All right.
Have a good weekend.
We'll continue next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So what we're going to do is to review what we have discussed last time.
Then we're going to talk about the classic application of Markov chains to analyze how do you dimension a phone system.
And finally, there will be two new things today.
We will see how we can calculate certain interesting quantities that have to do with Markov chains.
So let us start.
We've got our Markov chain and let's make the assumption that our chain is kind of nice.
And by nice we mean that we've got maybe some transient states.
And then we've got a single recurrent class of recurrent states.
So this is a single recurrent class in the sense that from any state in that class you can get to any other state.
So once you're in here you're going to circulate and keep visiting all of those states.
Those states appear transient.
The trajectory may move around here, but eventually one of these transitions will happen and you're going to end up in this lump.
Let's make the assumption that the single recurrent class is not periodic.
These are the nicest kind of Markov chains.
And they're nicest because they have the following property, the probability that you find yourself at some particular state j at the time n when that time is very large.
That probability settles to a steady state value that we denote by pi sub j.
And there are two parts in the statement.
One part is that this limit exists.
So the probability of state j settles to something, and furthermore that probability is not affected by i.
It doesn't matter where you started, no matter where you started, the probability of state j is going to be the same in the long run.
Maybe a clearer notation could be of this form.
The probability of being at state j given the initial state being i is equal to pi(j) in the limit.
Now, if I don't tell you where you started and you look at the unconditional probability of being at state i, you can average over the initial states, use the total expectation theorem and you're going to get the same answer pi(j) in the limit.
So this tells you that to the conditional probability given the initial state in the limit is the same as the unconditional probability.
And that's a situation that we recognize as being one where we have independence.
So what this result tells us is that Xn and Xi are approximately independent.
They become independent in the limit as n goes to infinity.
So that's what the steady state theorem tells us.
The initial conditions don't matter, so your state at some large time n has nothing to do, is not affected by what your initial state was.
Knowing the initial state doesn't tell you anything about your state at time n, therefore the states at the times-- sorry that should be a 1, or it should be a 0 -- so the state is not affected by where the process started.
So if the Markov chain is to operate for a long time and we're interested in the question where is the state, then your answer would be, I don't know, it's random.
But it's going to be a particular j with this particular probability.
So the steady state probabilities are interesting to us and that raises the question of how do we compute them.
The way we compute them is by solving a linear system of equations, which are called the balance equations, together with an extra equation, the normalization equation that has to be satisfied by probability, because probabilities must always add up to 1.
We talked about the interpretation of this equation last time.
It's basically a conservation of probability flow in some sense.
What comes in must get out.
The probability of finding yourself at state j at a particular time is the total probability of the last transition taking me into state j.
The last transition takes me into state j in various ways.
It could be that the previous time I was at the particular state, j and i made a transition from k into j.
So this number here, we interpret as the frequency with which transitions of these particular type k to j, occur.
And then by adding over all k's we consider transitions of all types that lead us inside state j.
So the probability of being at the j is the sum total of the probabilities of getting into j.
What if we had multiple recurrent classes?
So if we take this picture and change it to this.
So here we got a secondary recurrent class.
If you're here, you cannot get there.
If you are here, you cannot get there.
What happens in the long run?
Well, in the long run, if you start from here you're going to make a transition eventually, either of this type and you would end up here, or you will make a transitional of that type and you will end up there.
If you end up here, the long term statistics of your chain, that is, the probabilities of the different states, will be the steady state probabilities of this chain regarded in isolation.
So you go ahead and you solve this system of equations just for this chain, and these will be your steady state probabilities.
If you happened to get in here.
If, on the other hand, it happens that you went there, given that event, then what happens in the long run has to do with just this chain running by itself.
So you find the steady state probabilities inside that sub chain.
So you solve the linear system, the steady state equations, for this chain separately and for that chain separately.
If you happen to start inside here then the steady state probabilities for this sub chain are going to apply.
Now of course this raises the question, if I start here, how do I know whether I'm going to get here or there?
Well, you don't know, it's random.
It may turn out that you get to here, it may turn out that you get there.
So we will be interested in calculating the probabilities that eventually you end up here versus the probability that eventually you end up there.
This is something that we're going to do towards the end of today's lecture.
So, as a warm up, just to see how we interpret those steady state probabilities, let us look at our familiar example.
This is a 2-state Markov chain.
Last time we did write down the balance equations for this chain and we found the steady state probabilities to be 2/7 and 5/7 respectively.
So let us try to calculate some quantities.
Suppose that you start at state 1, and you want to calculate to this particular probability.
So since we're assuming that we're starting at state 1, essentially here we are conditioning on the initial state being equal to 1.
Now the conditional probability of two things happening is the probability that the first thing happens.
But we're living in the world where we said that the initial state was 1.
And then given that this thing happened, the probability that the second thing happens.
But again, we're talking about conditional probabilities given that the initial state was 1.
So what is this quantity?
This one is the transition probability from state 1 to state 1, so it's P11.
How about the second probability?
So given that you started at 1 and the next time you were at 1, what's the probability that at the time 100 you are at 1?
Now because of the Markov property, if I tell you that at this time you are at 1, it doesn't matter how you get there.
So this part of the conditioning doesn't matter.
And what we have is the 99 step transition probability from state 1 to state 1.
So the probability that you get to 1 and then 99 steps later you find yourself again at one is the probability that the first transition takes you to 1 times the probability that over the next 99 transitions starting from 1, after 99 steps you end up again at state 1.
Now, 99 is possibly a big number, and so we approximate this quantity.
We're using the steady state probability of state 1.
And that gives us an approximation for this particular expression.
We can do the same thing to calculate something of the same kind.
So you start at state 1.
What's the probability that 100 steps later you are again at state 1?
So that's going to be P11-- not P -- R11.
The 100 step transition probability that starting from 1 you get to 1, and then after you get to 1 at time 100 what's the probability that the next time you find yourself at state 2?
This is going to be the probability P12.
And approximately, since 100 is a large number, this is approximately pi(1) times P12.
OK.
So that's how we can use steady state probabilities to make approximations.
Or you could, for example, if you continue doing examples of this kind, you could ask for what's the probability that X at time 100 is 1, and also X at time 200 is equal to 1.
Then this is going to be the transition probability from 1 to 1 in 100 steps, and then over the next 100 steps from 1 you get again to 1.
And this is going to be approximately pi(1) times pi(1).
So we approximate multi-step transition probabilities by the steady state probabilities when the number n that's involved in here is big.
Now I said that's 99 or 100 is big.
How do we know that it's big enough so that the limit has taken effect, and that our approximation is good?
This has something to do with the time scale of our Markov chain, and by time scale, I mean how long does it take for the initial states to be forgotten.
How long does it take for there to be enough randomness so that things sort of mix and it doesn't matter where you started?
So if you look at this chain, it takes on the average, let's say 5 tries to make a transition of this kind.
It takes on the average 2 tries for a transition of that kind to take place.
So every 10 time steps or so there's a little bit of randomness.
Over 100 times steps there's a lot of randomness, so you expect that the initial state will have been forgotten.
It doesn't matter.
There's enough mixing and randomness that happens over 100 time steps.
And so this approximation is good.
On the other hand, if the numbers were different, the story would have been different.
Suppose that this number is 0.999 and that number is something like 0.998, so that this number becomes 0.002, and that number becomes 0.001.
Suppose that the numbers were of this kind.
How long does it take to forget the initial state?
If I start here, there's a probability of 1 in 1,000 that next time I'm going to be there.
So on the average it's going to take me about a thousand tries just to leave that state.
So, over roughly a thousand time steps my initial state really does matter.
If I tell you that you started here, you're pretty certain that, let's say over the next 100 time steps, you will still be here.
So the initial state has a big effect.
In this case we say that this Markov chain has a much slower time scale.
It takes a much longer time to mix, it takes a much longer time for the initial state to be forgotten, and this means that we cannot do this kind of approximation if the number of steps is just 99.
Here we might need n to be as large as, let's say, 10,000 or so before we can start using the approximation.
So when one uses that approximation, one needs to have some sense of how quickly does the state move around and take that into account.
So there's a whole sub-field that deals with estimating or figuring out how quickly different Markov chains mix, and that's the question of when can you apply those steady state approximations.
So now let's get a little closer to the real world.
We're going to talk about a famous problem that was posed, started, and solved by a Danish engineer by the name of Erlang.
This is the same person whose name is given to the Erlang distribution that we saw in the context of the Poisson processes.
So this was more than 100 years ago, when phones had just started existing.
And he was trying to figure out what it would take to set up a phone system that how many lines should you set up for a community to be able to communicate to the outside world.
So here's the story.
You've got a village, and that village has a certain population, and you want to set up phone lines.
So you want to set up a number of phone lines, let's say that number is B, to the outside world.
And how do you want to do that?
Well, you want B to be kind of small.
You don't want to set up too many wires because that's expensive.
On the other hand, you want to have enough wires so that if a reasonable number of people place phone calls simultaneously, they will all get a line and they will be able to talk.
So if B is 10 and 12 people want to talk at the same time, then 2 of these people would get a busy signal, and that's not something that we like.
We would like B to be large enough so that there's a substantial probability, that there's almost certainty that, under reasonable conditions, no one is going to get a busy signal.
So how do we go about modeling a situation like this?
Well, to set up a model you need two pieces, one is to describe how do phone calls get initiated, and once a phone call gets started, how long does it take until the phone call is terminated?
So we're going to make the simplest assumptions possible.
Let's assume that phone calls originate as a Poisson process.
That is, out of that population people do not really coordinate.
At completely random times, different people with decide to pick up the phone.
There's no dependencies between different people, there's nothing special about different times, different times are independent.
So a Poisson model is a reasonable way of modeling this situation.
And it's going to be a Poisson process with some rate lambda.
Now, the rate lambda would be easy to estimate in practice.
You observe what happens in that village just over a couple of days, and you figure out what's the rate at which people attempt to place phone calls.
Now, about phone calls themselves, we're going to make the assumption that the duration of a phone call is a random variable that has an exponential distribution with a certain parameter mu.
So 1/mu is the mean duration of a phone call.
So the mean duration, , again, is easy to estimate.
You just observe what's happening, see on the average how long these phone calls are.
Is the exponential assumption a good assumption?
Well, it's means that most phone calls will be kind of short, but there's going to be a fraction of phone calls that are going to be larger, and then a very small fraction that are going to be even larger.
So it sounds plausible.
It's not exactly realistic, that is, phone calls that last short of 15 seconds are not that common.
So either nothing happens or you have to say a few sentences and so on.
Also, back into the days when people used to connect to the internet using dial up modems, that assumption was completely destroyed, because people would dial up and then keep their phone line busy for a few hours, if the phone call was a free one.
So at those times the exponential assumption for the phone call duration was completely destroyed.
But leaving that detail aside, it's sort of a reasonable assumption to just get started with this problem.
All right, so now that we have those assumptions, let's try to come up with the model.
And we're going to set up a Markov process model.
Now the Poisson process runs in continuous time, and call durations being exponential random variables also are continuous random variables, so it seems that we are in a continuous time universe.
But we have only started Markov chains for the discrete time case.
What are we going to do?
We can either develop the theory of continuous time Markov chains, which is possible.
But we are not going to do that in this class.
Or we can discretize time and work with a discrete time model.
So we're going to discretize time in the familiar way, the way we did it when we started the Poisson process.
We're going to take the time axis and split it into little discrete mini slots, where every mini slot has a duration delta.
So this delta is supposed to be a very small number.
So what is the state of the system?
So, you look at the situation in the system at some particular time and I ask you what is going on right now, what's the information you would tell me?
Well, you would tell me that right now out of these capital B lines, 10 of them are busy, or 12 of them are busy.
That describes the state of the system, that tells me what's happening at this point.
So we set up our states base by being the numbers from 0 to B.
0 corresponds to a state in which all the phone lines are free, no one is talking.
Capital B corresponds to a case where all the phone lines are busy.
And then you've got states in between.
And now let's look at the transition probabilities.
Suppose that right so now we have i-1 lines that are busy.
Or maybe, let me look here.
Suppose that there's i lines that are busy.
What can happen the next time?
What can happen is that the new phone call gets placed, in which case my state moves up by 1, or an existing call terminates, in which case my state goes down by 1, or none of the two happens, in which case I stay at the same state.
Well, it's also possible that the phone call gets terminated and a new phone call gets placed sort of simultaneously.
But when you take your time slots to be very, very small, this is going to have a negligible probability order of delta squared, so we ignore this.
So what's the probability of an upwards transition?
That's the probability that the Poisson process records an arrival during a mini slot of duration delta.
By the definition of the Poisson process, the probability of this happening is just lambda delta.
So each one of these upwards transitions has the same probability of lambda delta.
So you've got lambda deltas everywhere in this diagram.
How about, now, phone call terminations?
If you had the single call that was active, so if you were here, what's the probability that the phone call terminates?
So the phone call has an exponential duration with parameter mu.
And we discussed before that an exponential random variable can be thought of as the first arrival time in a Poisson process.
So the probability that you get this event to happen over a delta time interval is just mu times delta.
So if you have a single phone call that's happening right now, with probability mu times delta, that call is going to terminate.
But suppose that we have i phone calls that are currently active.
Each one of them has a probability of mu delta, of terminating, but collectively the probability that one of them terminates becomes i times mu delta.
So that's because you get the mu delta contribution -- the probability of termination from each one of the different phone calls.
OK, now this is an approximate calculation, because it ignores the possibility that two phone calls terminate at the same time.
Again, the way to think of why this is the correct rate, when you have i phone calls that are simultaneously running and waiting for one of them to terminate, this is like having i separate Poisson processes that are running in parallel, and you ask for the probability that one of those processes records an event.
Now when you put all those process together, it's like having a Poisson process with total rate i times mu, and so i times mu delta is the overall probability that something happens in terms of phone call terminations at those times.
So in any case, this is the transition probability for downwards transitions.
Now that we've got this, we can analyze this chain.
This chain has the birth death form that we discussed towards the end of last lecture.
And for birth death chains, it's easy to write it out to find the steady state probabilities.
Instead of writing down the balance equations in the general form, we think in terms of a conservation of probabilities or of transitions by looking at what happens across a particular cut in this diagram.
Number of transitions in the chain that cross from here to here has to be approximately equal to the number of transitions from here to there because whatever comes up must come down and then come up and so on.
So the frequency with which transitions of this kind are observed has to be the same as the frequency of transitions of this kind.
What's the frequency of how often the transitions of this kind happen?
And by frequency I mean quite percentage of the mini slots involve a transition of this kind?
Well, for a transition of that kind to happen we need to be at states i-1, which happens this much of the time.
And then the probability lambda delta that the transition is of this kind.
So the frequency of transitions of with which this kind of transition is observed is lambda delta times pi(i-1).
This is the fraction of time steps at which a transition from specifically this state to specifically that state are observed.
This has to be the same as the frequency with which transitions of that kind are observed, and that frequency is going to be i mu delta times pi(i), and then we cancel the deltas, and we are left with this equation here.
So this equation expresses pi(i) in terms of pi(i-1).
So if we knew pi(0) we can use that equation to determine pi(1).
Once we know pi(1), we can use that equation to determine pi(2), and so on, you keep going.
And the general formula that comes out of this, I will not do the algebra, it's a straightforward substitution, you find that pi(i), the steady state probability of state i is given by this expression, which involves the pi(0) from which we started.
Now what is pi(0)?
Well, we don't know yet, but we can find it by using the normalization equation.
The sum of pi(i) has to be equal to 1.
So the sum of all of those numbers has to be equal to 1.
And the only way that this can happen is by setting pi(0) to be equal to that particular number.
So if I tell you the value of capital B, you can set up this Markov chain, you can calculate pi(0), and then you can calculate pi(i), and so you know what fraction, you know the steady state probabilities of this chain, so you can answer the question.
If I drop in at a random time, how likely is it that I'm going to find the states to be here, or the states to be there?
So the steady state probabilities are probabilities, but we also interpret them as frequencies.
So once I find pi(i), it also tells me what fraction of the time is the state equal to i.
And you can answer that question for every possible i.
Now, why did we do this exercise?
We're interested in the probability of the system is busy.
So if a person, a new phone call gets placed, it just drops out of the sky.
According to that Poisson process, that new phone call is going to find the system at a random state.
That random state is described in steady state by the probabilities pi(i)'s.
And the probability that you find the system to be busy is the probability that when you drop in the state happens to be that particular number B.
So i sub b is the probability of being busy.
And this is the probability that you would like to be small in a well engineered system.
So you ask the question, how should, given my lambda and mu, my design question is to determine capital B the number of phone lines so that this number is small.
Could we have done, could we figure out a good value for B by doing a back of the envelope calculation?
Let's suppose that lambda is 30 and that mu is 1/3.
So I guess that's, let us these rates to be calls per minute.
And this mu, again, is a rate per minute.
Again, the units of mu are going to be calls per minute.
So since our time unit is minutes, the mean duration of calls is 1/mu minutes.
So a typical call, or on the average a call lasts for 3 minutes.
So you get 30 calls per minute.
Each call lasts for 3 minutes on the average.
So on the average, if B was infinite, every call goes through.
How many calls would be active on the average?
So you get 30 per minute.
If a call lasted exactly 1 minute, then at any time you would have 30 calls being active.
Now a call lasts on the average for 3 minutes.
So during each minute you generate 90 minutes of talking time.
So by thinking in terms of averages you would expect that at any time there would be about 90 calls that are active.
And if 90 calls are active on the average, you could say OK, I'm going to set up my capital B to be 90.
But that's not very good, because if the average number of phone calls that want to happen is if the average number is 90, sometimes you're going to have 85, sometimes you will have 95.
And to be sure that the phone calls will go through you probably want to choose your capital B to be a number a little larger than 90.
How much larger than 90?
Well, this is a question that you can answer numerically.
So you go through the following procedure.
I tried different values of capital B.
For any given value of capital B, I do this numerical calculation, I find the probability that the system is busy, and then I ask what's the value of B that makes my probability of being busy to be, let's say, roughly 1 %.
And if you do that calculation with the parameters that they gave you, you find that B would be something like 106.
So with the parameters they gave where you have, on the average, 90 phone calls being active, you actually need some margin to protect against the [?]
fluctuation, if suddenly by chance more people want to talk, and if you want to have a good guarantee that an incoming person will have a very small probability of finding a busy system, then you will need about 106 phone lines.
So that's the calculation and the argument that the Erlang went through a long time ago.
It's actually interesting that Erlang did this calculation before Markov chains were invented.
So Markov's work, and the beginning of work on Markov chains, happens about 10-15 years after Erlang.
So obviously he didn't call that a Markov chain.
But it was something that he could study from first principles.
So this is a pretty useful thing.
These probabilities that come out of that model, at least in the old days, they would all be very well tabulated in handbooks that every decent phone company engineer would sort of have with them.
So this is about as practical as it gets.
It's one of the sort of standard real world applications of Markov chains.
So now to close our subjects, we're going to consider a couple of new skills and see how we can calculate the few additional interesting quantities that have to do with the Markov chain.
So the problem we're going to deal with here is the one I hinted that when I was talking about this picture.
You start at a transient state, you're going to eventually end up here or there.
We want to find the probabilities of one option of the two happening or the other happening.
So in this picture we have a class of states that's are transient.
These are transient because you're going to move around those states, but there's a transition that you can make, and you go to a state from which you cannot escape afterwards.
Are you going to end up here or are you going to end up there?
You don't know.
It's random.
Let's try to calculate the probability that you end up at state 4.
Now, the probability that you end up at state 4 will depend on where you start.
Because if you start here, you probably have more chances of getting to 4 because you get that chance immediately, whereas if you start here there's more chances that you're going to escape that way because it kind of takes you time to get there.
It's more likely that you exit right away.
So the probability of exiting and ending up at state 4 will depend on the initial state.
That's why when we talk about these absorption probability we include an index i that tells us what the initial state is.
And we want to find this absorption probability, the probability that we end up here for the different initial states.
Now for some initial states this is very easy to answer.
If you start at state 4, what's the probability that eventually you end up in this part of the chain?
It's 1.
You're certain to be there, that's where you started.
If you start at state 5, what's the probability that you end up eventually at state 4?
It's probability 0, there's no way to get there.
Now, how about if you start at a state like state 2?
If you start at state 2 then there's a few different things that can happen.
Either you end up at state 4 right away and this happens with probability 0.2, or you end up at state 1, and this happens with probability 0.6.
So if you end up at state 4, you are done.
We are there.
If you end up at state 1, then what?
Starting from state 1 there's two possibilities.
Either eventually you're going to end up at state 4, or eventually you're going to end up at state 5.
What's the probability of this happening?
We don't know what it is, but it's what we defined to be a1.
This is the probability -- a1 is the probability -- that eventually you settle in state 4 given that the initial state was 1.
So this probability is a1.
So our event of interest can happen in two ways.
Either I go there directly, or I go here with probability 0.6.
And given that I go there, eventually I end up at state 4, which happens with probability a1.
So the total probability of ending up at state 4 is going to be the sum of the probabilities of the different ways that this event can happen.
So our equation, in this case, is going to be, that's a2, is going to be 0.2 (that's the probability of going there directly) plus with probability 0.8 I end up at state 1, and then from state 1 I will end up at state 4 with probability a1.
So this is one particular equation that we've got for what happens if we start from this state.
We can do a similar argument starting from any other state.
Starting from state i the probability that eventually I end up at state 4 is, we consider the different possible scenarios of where do I go next, which is my state j, with probability Pij.
Next time I go to j, and given that I started at j, this is the probability that I end up at state 4.
So this equation that we have here is just an abstract version in symbols of what we wrote down for the particular case where the initial state was 2.
So you write down an equation of this type for every state inside here.
You'll have a separate equation for a1, a2, and a3.
And that's going to be a system of 3 equations with 3 unknowns, the a's inside the transient states.
So you can solve that 3 by 3 system of equations.
Fortunately, it turns out to have a unique solution, and so once you solve it you have found the probabilities of absorption and the probability that eventually you get absorbed at state 4.
Now, in the picture that we had here, this was a single state, and that one was a single state.
How do things change if our recurrent, or trapping sets consist of multiple states?
Well, it doesn't really matter that we have multiple states.
All that matters is that this is one lump and once we get there we are stuck in there.
So if the picture was, let's say, like this, 0.1 and 0.2, that basically means that whenever you are in that state there's a total probability of 0.3 of ending in that lump and getting stuck inside that lump.
So you would take that picture and change it and make it instead a total probability of 0.3, of ending somewhere inside that lump.
And similarly, you take this lump and you view it as just one entity, and from any state you record the total probability that given that I'm here I end up in that entity.
So basically, if the only thing you care is the probability that you're going to end up in this lump, you can replace that lump with a single state, view it as a single state, and calculate probabilities using this formula.
All right, so now we know where the chain is going to get to.
At least we know probabilistically.
We know with what probability it is going to go here, and that also tells us the probability that eventually it's going to get there.
Other question, how long is it going to take until we get to either this state or that state?
We can call that event absorption, meaning that the state got somewhere into a recurrent class from which it could not get out.
Okay.
Let's deal with that question for the case where we have only 1 absorbing state.
So here our Markov chain is a little simpler than the one in the previous slide.
We've got our transient states, we've got our recurrent state, and once you get into the recurrent state you just stay there.
So here we're certain that no matter where we start we're going to end up here.
How long is it going to take?
Well, we don't know.
It's a random variable.
The expected value of that random variable, let's call it mu.
But how long it takes to get there certainly depends on where we start.
So let's put in our notation again this index i that indicates where we started from.
And now the argument is going to be of the same type as the one we used before.
We can think in terms of a tree once more, that considers all the possible options.
So suppose that you start at state 1.
Starting from state 1, the expected time until you end up dropping states is mu1.
Now, starting from state 1, what are the possibilities?
You make your first transition, and that first transition is going to take you either to state 2 or to state 3.
It takes you to state 2 with probability 0.6, it takes you to state 3 with probability 0.4.
Starting from state 2, eventually you're going to get to state 4.
How long does it take?
We don't know, it's a random variable.
But the expected time until this happens is mu2.
Starting from state 2, how long does it take you to get to state 4.
And similarly starting from state 3, it's going to take you on the average mu3 time steps until you get to state 4.
So what's the expected value of the time until I end at state 4?
So with probability 0.6, I'm going to end up at state 2 and from there on it's going to be expected time mu2, and with probability 0.4 I'm going to end up at state 3, and from there it's going to take me so much time.
So this is the expected time it's going to take me after the first transition.
But we also spent 1 time step for the first transition.
The total time to get there is the time of the first transition, which is 1, plus the expected time starting from the next state.
This expression here is the expected time starting from the next state, but we also need to account for the first transition, so we add 1.
And this is going to be our mu1.
So once more we have a linear equation that ties together the different mu's.
And the equation starting from state 4 in this case, of course is going to be simple, starting from that state the expected number of steps it takes you to get there for the first time is of course, 0 because you're already there.
So for that state this is fine, and for all the other states you get an equation of this form.
Now we're going to have an equation for every state.
It's a system of linear equations, once more we can solve them, and this gives us the expected times until our chain gets absorbed in this absorbing state.
And it's nice to know that this system of equations always has a unique solution.
OK so this was the expected time to absorption.
For this case where we had this scene absorbing state.
Suppose that we have our transient states and that we have multiple recurrent classes, or multiple absorbing states.
Suppose you've got the picture like this.
And we want to calculate the expected time until we get here or there.
Expected time until we get to an absorbing state.
What's the trick?
Well, we can lump both of these states together and think of them as just one bad state, one place for which we're interested in how long it takes us to get there.
So lump them as one state, and accordingly kind of merge all of those probabilities.
So starting from here, my probability that the next I end up in this lump and they get absorbed is going to be this probability plus that probability.
So we would change that picture.
Think of this as being just one big state.
And sort of add those two probabilities together to come up with a single probability, which is the probability that starting from here next time I find myself at some absorbing state.
So once you know how to deal with a situation like this, you can also find expected times to absorption for the case where you've got multiple absorbing states.
You just lump all of those multiple absorbing states into a single one.
Finally, there's a kind of related quantity that's of interest.
The question is almost the same as in the previous slide, except that here we do not have any absorbing states.
Rather, we have a single recurrent class of states.
You start at some state i.
You have a special state, that is state s. And you ask the question, how long is it going to take me until I get to s for the first time?
It's a single recurrent class of states.
So you know that the state keeps circulating here and it keeps visiting all of the possible states.
So eventually this state will be visited.
How long does it take for this to happen?
Ok.
So we're interested in how long it takes for this to happen, how long it takes until we get to s for the first time.
And we don't care about what happens afterwards.
So we might as well change this picture and remove the transitions out of s and to make them self transitions.
Is the answer going to change?
No.
The only thing that we changed was what happens after you get to s. But what happens after you get to s doesn't matter.
The question we're dealing with is how long does it take us to get to s. So essentially, it's after we do this transformation -- it's the same question as before, what's the time it takes until eventually we hit this state.
And it's now in this new picture, this state is an absorbing state.
Or you can just think from first principles.
Starting from the state itself, s, it takes you 0 time steps until you get to s. Starting from anywhere else, you need one transition and then after the first transition you find yourself at state j with probability Pij and from then on you are going to take expected time Tj until you get to that terminal state s. So once more these equations have a unique solution, you can solve them and find the answer.
And finally, there's a related question, which is the mean recurrence time of s. In that question you start at s, the chain will move randomly, and you ask how long is it going to take until I come back to s for the next time.
So notice the difference.
Here we're talking the first time after time 0, whereas here it's just the first time anywhere.
So here if you start from s, Ts* is not 0.
You want to do at least one transition and that's how long it's going to take me until it gets back to s. Well, how long does it take me until I get back to s?
I do my first transition, and then after my first transition I calculate the expected time from the next state how long it's going to take me until I come back to s. So all of these equations that I wrote down, they all kind of look the same.
But they are different.
So you can either memorize all of these equations, or instead what's better is to just to get the basic idea.
That is, to calculate probabilities or expected values you use the total probability or total expectation theorem and conditional the first transition and take it from there.
So you're going to get a little bit of practice with these skills in recitation tomorrow, and of course it's in your problem set as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're going to start today a new unit.
so we will be talking about limit theorems.
So just to introduce the topic, let's think of the following situation.
There's a population of penguins down at the South Pole.
And if you were to pick a penguin at random and measure their height, the expected value of their height would be the average of the heights of the different penguins in the population.
So suppose when you pick one, every penguin is equally likely.
Then the expected value is just the average of all the penguins out there.
So your boss asks you to find out what that the expected value is.
One way would be to go and measure each and every penguin.
That might be a little time consuming.
So alternatively, what you can do is to go and pick penguins at random, pick a few of them, let's say a number n of them.
So you measure the height of each one.
And then you calculate the average of the heights of those penguins that you have collected.
So this is your estimate of the expected value.
Now, we called this the sample mean, which is the mean value, but within the sample that you have collected.
This is something that's sort of feels the same as the expected value, which is again, the mean.
But the expected value's a different kind of mean.
The expected value is the mean over the entire population, whereas the sample mean is the average over the smaller sample that you have measured.
The expected value is a number.
The sample mean is a random variable.
It's a random variable because the sample you have collected is random.
Now, we think that this is a reasonable way of estimating the expectation.
So in the limit as n goes to infinity, it's plausible that the sample mean, the estimate that we are constructing, should somehow get close to the expected value.
What does this mean?
What does it mean to get close?
In what sense?
And is this statement true?
This is the kind of statement that we deal with when dealing with limit theorems.
That's the subject of limit theorems, when what happens if you're dealing with lots and lots of random variables, and perhaps take averages and so on.
So why do we bother about this?
Well, if you're in the sampling business, it would be reassuring to know that this particular way of estimating the expected value actually gets you close to the true answer.
There's also a higher level reason, which is a little more abstract and mathematical.
So probability problems are easy to deal with if you're having in your hands one or two random variables.
You can write down their mass functions, joints density functions, and so on.
You can calculate on paper or on a computer, you can get the answers.
Probability problems become computationally intractable if you're dealing, let's say, with 100 random variables and you're trying to get the exact answers for anything.
So in principle, the same formulas that we have, they still apply.
But they involve summations over large ranges of combinations of indices.
And that makes life extremely difficult.
But when you push the envelope and you go to a situation where you're dealing with a very, very large number of variables, then you can start taking limits.
And when you take limits, wonderful things happen.
Many formulas start simplifying, and you can actually get useful answers by considering those limits.
And that's sort of the big reason why looking at limit theorems is a useful thing to do.
So what we're going to do today, first we're going to start with a useful, simple tool that allows us to relates probabilities with expected values.
The Markov inequality is the first inequality we're going to write down.
And then using that, we're going to get the Chebyshev's inequality, a related inequality.
Then we need to define what do we mean by convergence when we talk about random variables.
It's a notion that's a generalization of the notion of the usual convergence of limits of a sequence of numbers.
And once we have our notion of convergence, we're going to see that, indeed, the sample mean converges to the true mean, converges to the expected value of the X's.
And this statement is called the weak law of large numbers.
The reason it's called the weak law is because there's also a strong law, which is a statement with the same flavor, but with a somewhat different mathematical content.
But it's a little more abstract, and we will not be getting into this.
So the weak law is all that you're going to get.
All right.
So now we start our digression.
And our first tool will be the so-called Markov inequality.
So let's take a random variable that's always non-negative.
No matter what, it gets no negative values.
To keep things simple, let's assume it's a discrete random variable.
So the expected value is the sum over all possible values that a random variable can take.
The values of the random variables that can take weighted according to their corresponding probabilities.
Now, this is a sum over all x's.
But x takes non-negative values.
And the PMF is also non-negative.
So if I take a sum over fewer things, I'm going to get a smaller value.
So the sum when I add over everything is less than or equal to the sum that I will get if I only add those terms that are bigger than a certain constant.
Now, if I'm adding over x's that are bigger than a, the x that shows up up there will always be larger than or equal to a.
So we get this inequality.
And now, a is a constant.
I can pull it outside the summation.
And then I'm left with the probabilities of all the x's that are bigger than a.
And that's just the probability of being bigger than a. OK, so that's the Markov inequality.
Basically tells us that the expected value is larger than or equal to this number.
It relates expected values to probabilities.
It tells us that if the expected value is small, then the probability that x is big is also going to be small.
So it's translates a statement about smallness of expected values to a statement about smallness of probabilities.
OK. What we actually need is a somewhat different version of this same statement.
And what we're going to do is to apply this inequality to a non-negative random variable of a special type.
And you can think of applying this same calculation to a random variable of this form, (X minus mu)-squared, where mu is the expected value of X.
Now, this is a non-negative random variable.
So, the expected value of this random variable, which is the variance, by following the same thinking as we had in that derivation up to there, is bigger than the probability that this random variable is bigger than some-- let me use a-squared instead of an a times the value a-squared.
So now of course, this probability is the same as the probability that the absolute value of X minus mu is bigger than a times a-squared.
And this side is equal to the variance of X.
So this relates the variance of X to the probability that our random variable is far away from its mean.
If the variance is small, then it means that the probability of being far away from the mean is also small.
So I derived this by applying the Markov inequality to this particular non-negative random variable.
Or just to reinforce, perhaps, the message, and increase your confidence in this inequality, let's just look at the derivation once more, where I'm going, here, to start from first principles, but use the same idea as the one that was used in the proof out here.
Ok.
So just for variety, now let's think of X as being a continuous random variable.
The derivation is the same whether it's discrete or continuous.
So by definition, the variance is the integral, is this particular integral.
Now, the integral is going to become smaller if I integrate, instead of integrating over the full range, I only integrate over x's that are far away from the mean.
So mu is the mean.
Think of c as some big number.
These are x's that are far away from the mean to the left, from minus infinity to mu minus c. And these are the x's that are far away from the mean on the positive side.
So by integrating over fewer stuff, I'm getting a smaller integral.
Now, for any x in this range, this distance, x minus mu, is at least c. So that squared is at least c squared.
So this term over this range of integration is at least c squared.
So I can take it outside the integral.
And I'm left just with the integral of the density.
Same thing on the other side.
And so what factors out is this term c squared.
And inside, we're left with the probability of being to the left of mu minus c, and then the probability of being to the right of mu plus c, which is the same as the probability that the absolute value of the distance from the mean is larger than or equal to c. So that's the same inequality that we proved there, except that here I'm using c. There I used a, but it's exactly the same one.
This inequality was maybe better to understand if you take that term and send it to the other side and write it this form.
What does it tell us?
It tells us that if c is a big number, it tells us that the probability of being more than c away from the mean is going to be a small number.
When c is big, this is small.
Now, this is intuitive.
The variance is a measure of the spread of the distribution, how wide it is.
It tells us that if the variance is small, the distribution is not very wide.
And mathematically, this translates to this statement that when the variance is small, the probability of being far away is going to be small.
And the further away you're looking, that is, if c is a bigger number, that probability also becomes small.
Maybe an even more intuitive way to think about the content of this inequality is to, instead of c, use the number k, where k is positive and sigma is the standard deviation.
So let's just plug k sigma in the place of c. So this becomes k sigma squared.
These sigma squared's cancel.
We're left with 1 over k-square.
Now, what is this?
This is the event that you are k standard deviations away from the mean.
So for example, this statement here tells you that if you look at the test scores from a quiz, what fraction of the class are 3 standard deviations away from the mean?
It's possible, but it's not going to be a lot of people.
It's going to be at most, 1/9 of the class that can be 3 standard deviations or more away from the mean.
So the Chebyshev inequality is a really useful one.
It comes in handy whenever you want to relate probabilities and expected values.
So if you know that your expected values or, in particular, that your variance is small, this tells you something about tailed probabilities.
So this is the end of our first digression.
We have this inequality in our hands.
Our second digression is talk about limits.
We want to eventually talk about limits of random variables, but as a warm up, we're going to start with limits of sequences.
So you're given a sequence of numbers, a1, a2, a3, and so on.
And we want to define the notion that a sequence converges to a number.
You sort of know what this means, but let's just go through it some more.
So here's a.
We have our sequence of values as n increases.
What do we mean by the sequence converging to a is that when you look at those values, they get closer and closer to a.
So this value here is your typical a sub n. They get closer and closer to a, and they stay closer.
So let's try to make that more precise.
What it means is let's fix a sense of what it means to be close.
Let me look at an interval that goes from a - epsilon to a + epsilon.
Then if my sequence converges to a, this means that as n increases, eventually the values of the sequence that I get stay inside this band.
Since they converge to a, this means that eventually they will be smaller than a + epsilon and bigger than a - epsilon.
So convergence means that given a band of positive length around the number a, the values of the sequence that you get eventually get inside and stay inside that band.
So that's sort of the picture definition of what convergence means.
So now let's translate this into a mathematical statement.
Given a band of positive length, no matter how wide that band is or how narrow it is, so for every epsilon positive, eventually the sequence gets inside the band.
What does eventually mean?
There exists a time, so that after that time something happens.
And the something that happens is that after that time, we are inside that band.
So this is a formal mathematical definition, which actually translates what I was telling in the wordy way before, and showing in terms of the picture.
Given a certain band, even if it's narrow, eventually, after a certain time n0, the values of the sequence are going to stay inside this band.
Now, if I were to take epsilon to be very small, this thing would still be true that eventually I'm going to get inside of the band, except that I may have to wait longer for the values to get inside here.
All right, that's what it means for a deterministic sequence to converge to something.
Now, how about random variables.
What does it mean for a sequence of random variables to converge to a number?
We're just going to twist a little bit of the word definition.
For numbers, we said that eventually the numbers get inside that band.
But if instead of numbers we have random variables with a certain distribution, so here instead of a_n we're dealing with a random variable that has a distribution, let's say, of this kind, what we want is that this distribution gets inside this band, so it gets concentrated inside here.
What does it means that the distribution gets inside this band?
I mean a random variable has a distribution.
It may have some tails, so maybe not the entire distribution gets concentrated inside of the band.
But we want that more and more of this distribution is concentrated in this band.
So that -- in a sense that -- the probability of falling outside the band converges to 0 -- becomes smaller and smaller.
So in words, we're going to say that the sequence random variables or a sequence of probability distributions, that would be the same, converges to a particular number a if the following is true.
If I consider a small band around a, then the probability that my random variable falls outside this band, which is the area under this curve, this probability becomes smaller and smaller as n goes to infinity.
The probability of being outside this band converges to 0.
So that's the intuitive idea.
So in the beginning, maybe our distribution is sitting everywhere.
As n increases, the distribution starts to get concentrating inside the band.
When a is even bigger, our distribution is even more inside that band, so that these outside probabilities become smaller and smaller.
So the corresponding mathematical statement is the following.
I fix a band around a, a +/- epsilon.
Given that band, the probability of falling outside this band, this probability converges to 0.
Or another way to say it is that the limit of this probability is equal to 0.
If you were to translate this into a complete mathematical statement, you would have to write down the following messy thing.
For every epsilon positive -- that's this statement -- the limit is 0.
What does it mean that the limit of something is 0?
We flip back to the previous slide.
Why?
Because a probability is a number.
So here we're talking about a sequence of numbers convergent to 0.
What does it mean for a sequence of numbers to converge to 0?
It means that for any epsilon prime positive, there exists some n0 such that for every n bigger than n0 the following is true -- that this probability is less than or equal to epsilon prime.
So the mathematical statement is a little hard to parse.
For every size of that band, and then you take the definition of what it means for the limit of a sequence of numbers to converge to 0.
But it's a lot easier to describe this in words and, basically, think in terms of this picture.
That as n increases, the probability of falling outside those bands just become smaller and smaller.
So the statement is that our distribution gets concentrated in arbitrarily narrow little bands around that particular number a. OK.
So let's look at an example.
Suppose a random variable Yn has a discrete distribution of this particular type.
Does it converge to something?
Well, the probability distribution of this random variable gets concentrated at 0 -- there's more and more probability of being at 0.
If I fix a band around 0 -- so if I take the band from minus epsilon to epsilon and look at that band-- the probability of falling outside this band is 1/n.
As n goes to infinity, that probability goes to 0.
So in this case, we do have convergence.
And Yn converges in probability to the number 0.
So this just captures the facts obvious from this picture, that more and more of our probability distribution gets concentrated around 0, as n goes to infinity.
Now, an interesting thing to notice is the following, that even though Yn converges to 0, if you were to write down the expected value for Yn, what would it be?
It's going to be n times the probability of this value, which is 1/n.
So the expected value turns out to be 1.
And if you were to look at the expected value of Yn-squared, this would be 0. times this probability, and then n-squared times this probability, which is equal to n. And this actually goes to infinity.
So we have this, perhaps, strange situation where a random variable goes to 0, but the expected value of this random variable does not go to 0.
And the second moment of that random variable actually goes to infinity.
So this tells us that convergence in probability tells you something, but it doesn't tell you the whole story.
Convergence to 0 of a random variable doesn't imply anything about convergence of expected values or of variances and so on.
So the reason is that convergence in probability tells you that this tail probability here is very small.
But it doesn't tell you how far does this tail go.
As in this example, the tail probability is small, but that tail acts far away, so it gives a disproportionate contribution to the expected value or the expected value squared.
OK.
So now we've got everything that we need to go back to the sample mean and study its properties.
So the sad thing is that we have a sequence of random variables.
They're independent.
They have the same distribution.
And we assume that they have a finite mean and a finite variance.
We're looking at the sample mean.
Now in principle, you can calculate the probability distribution of the sample mean, because we know how to find the distributions of sums of independent random variables.
You use the convolution formula over and over.
But this is pretty complicated, so let's not look at that.
Let's just look at expected values, variances, and the probabilities that the sample mean is far away from the true mean.
So what is the expected value of this random variable?
The expected value of a sum of random variables is the sum of the expected values.
And then we have this factor of n in the denominator.
Each one of these expected values is mu, so we get mu.
So the sample mean, the average value of this Mn in expectation is the same as the true mean inside our population.
Now here, this is a fine conceptual point, there's two kinds of averages involved when you write down this expression.
We understand that expectations are some kind of average.
The sample mean is also an average over the values that we have observed.
But it's two different kinds of averages.
The sample mean is the average of the heights of the penguins that we collected over a single expedition.
The expected value is to be thought of as follows, my probabilistic experiment is one expedition to the South Pole.
Expected value here means thinking on the average over a huge number of expeditions.
So my expedition is a random experiment, I collect random samples, and they record Mn.
The average result of an expedition is what we would get if we were to carry out a zillion expeditions and average the averages that we get at each particular expedition.
So this Mn is the average during a single expedition.
This expectation is the average over an imagined infinite sequence of expeditions.
And of course, the other thing to always keep in mind is that expectations give you numbers, whereas the sample mean is actually a random variable.
All right.
So this random variable, how random is it?
How big is its variance?
So the variance of a sum of random variables is the sum of the variances.
But since we're dividing by n, when you calculate variances this brings in a factor of n-squared.
So the variance is sigma-squared over n. And in particular, the variance of the sample mean becomes smaller and smaller.
It means that when you estimate that average height of penguins, if you take a large sample, then your estimate is not going to be too random.
The randomness in your estimates become small if you have a large sample size.
Having a large sample size kind of removes the randomness from your experiment.
Now let's apply the Chebyshev inequality to say something about tail probabilities for the sample mean.
The probability that you are more than epsilon away from the true mean is less than or equal to the variance of this quantity divided by this number squared.
So that's just the translation of the Chebyshev inequality to the particular context we've got here.
We found the variance.
It's sigma-squared over n. So we end up with this expression.
So what does this expression do?
For any given epsilon, if I fix epsilon, then this probability, which is less than sigma-squared over n epsilon-squared, converges to 0 as n goes to infinity.
And this is just the definition of convergence in probability.
If this happens, that the probability of being more than epsilon away from the mean, that probability goes to 0, and this is true no matter how I choose my epsilon, then by definition we have convergence in probability.
So we have proved that the sample mean converges in probability to the true mean.
And this is what the weak law of large numbers tells us.
So in some vague sense, it tells us that the sample means, when you take the average of many, many measurements in your sample, then the sample mean is a good estimate of the true mean in the sense that it approaches the true mean as your sample size increases.
It approaches the true mean, but of course in a very specific sense, in probability, according to this notion of convergence that we have used.
So since we're talking about sampling, let's go over an example, which is the typical situation faced by someone who's constructing a poll.
So you're interested in some property of the population.
So what fraction of the population prefers Coke to Pepsi?
So there's a number f, which is that fraction of the population.
And so this is an exact number.
So out of a population of 100 million, 20 million prefer Coke, then f would be 0.2.
We want to find out what that fraction is.
We cannot ask everyone.
What we're going to do is to take a random sample of people and ask them for their preferences.
So the ith person either says yes for Coke or no.
And we record that by putting a 1 each time that we get a yes answer.
And then we form the average of these x's.
What is this average?
It's the number of 1's that we got divided by n. So this is a fraction, but calculated only on the basis of the sample that we have.
So you can think of this as being an estimate, f_hat, based on the sample that we have.
Now, even though we used the lower case letter here, this f_hat is, of course, a random variable.
f is a number.
This is the true fraction in the overall population.
f_hat is the estimate that we get by using our particular sample.
Ok.
So your boss told you, I need to know what f is, but go and do some sampling.
What are you going to respond?
Unless I ask everyone in the whole population, there's no way for me to know f exactly.
Right?
There's no way.
OK, so the boss tells you, well OK, then that'll me f within an accuracy.
I want an answer from you, that's your answer, which is close to the correct answer within 1 % point.
So if the true f is 0.4, your answer should be somewhere between 0.39 and 0.41.
I want a really accurate answer.
What are you going to say?
Well, there's no guarantee that my answer will be within 1 %.
Maybe I'm unlucky and I just happen to sample the wrong set of people and my answer comes out to be wrong.
So I cannot give you a hard guarantee that this inequality will be satisfied.
But perhaps, I can give you a guarantee that this inequality will be satisfied, this accuracy requirement will be satisfied, with high confidence.
That is, there's going to be a smaller probability that things go wrong, that I'm unlikely and I use a bad sample.
But leaving aside that smaller probability of being unlucky, my answer will be accurate within the accuracy requirement that you have.
So these two numbers are the usual specs that one has when designing polls.
So this number is the accuracy that we want.
It's the desired accuracy.
And this number has to do with the confidence that we want.
So 1 minus that number, we could call it the confidence that we want out of our sample.
So this is really 1 minus confidence.
So now your job is to figure out how large an n, how large a sample should you be using, in order to satisfy the specs that your boss gave you.
All you know at this stage is the Chebyshev inequality.
So you just try to use it.
The probability of getting an answer that's more than 0.01 away from the true answer is, by Chebyshev's inequality, the variance of this random variable divided by this number squared.
The variance, as we argued a little earlier, is the variance of the x's divided by n. So we get this expression.
So we would like this number to be less than or equal to 0.05.
OK, here we hit a little bit off a difficulty.
The variance, (sigma_x)-squared, what is it?
(Sigma_x)-squared is, if you remember the variance of a Bernoulli random variable, is this quantity.
But we don't know it.
f is what we're trying to estimate in the first place.
So the variance is not known, so I cannot plug in a number inside here.
What I can do is to be conservative and use an upper bound of the variance.
How large can this number get?
Well, you can plot f times (1-f).
It's a parabola.
It has a root at 0 and at 1.
So the maximum value is going to be, by symmetry, at 1/2 and when f is 1/2, then this variance becomes 1/4.
So I don't know (sigma_x)-squared, but I'm going to use the worst case value for (sigma_x)-squared, which is 4.
And this is now an inequality that I know to be always true.
I've got my specs, and my specs tell me that I want this number to be less than 0.05.
And given what I know, the best thing I can do is to say, OK, I'm going to take this number and make it less than 0.05.
If I choose my n so that this is less than 0.05, then I'm certain that this probability is also less than 0.05.
What does it take for this inequality to be true?
You can solve for n here, and you find that to satisfy this inequality, n should be larger than or equal to 50,000.
So you can just let n be equal to 50,000.
So the Chebyshev inequality tells us that if you take n equal to 50,000, then by the Chebyshev inequality, we're guaranteed to satisfy the specs that we were given.
Ok. Now, 50,000 is a bit of a large sample size.
Right?
If you read anything in the newspapers where they say so much of the voters think this and that, this was determined on the basis of a sample of 1,200 likely voters or so.
So the numbers that you will typically see in these news items about polling, they usually involve sample sizes about the 1,000 or so.
You will never see a sample size of 50,000.
That's too much.
So where can we cut some corners?
Well, we can cut corners basically in three places.
This requirement is a little too tight.
Newspaper stories will usually tell you, we have an accuracy of +/- 3 % points, instead of 1 % point.
And because this number comes up as a square, by making it 3 % points instead of 1, saves you a factor of 10.
Then, the five percent confidence, I guess that's usually OK.
If we use that factor of 10, then we make our sample that we gain from here, then we get a sample size of 10,000.
And that's, again, a little too big.
So where can we fix things?
Well, it turns out that this inequality that we're using here, Chebyshev's inequality, is just an inequality.
It's not that tight.
It's not very accurate.
Maybe there's a better way of calculating or estimating this quantity, which is smaller than this.
And using a more accurate inequality or a more accurate bound, then we can convince ourselves that we can settle with a smaller sample size.
This more accurate kind of inequality comes out of a difference limit theorem, which is the next limit theorem we're going to consider.
We're going to start the discussion today, but we're going to continue with it next week.
Before I tell you exactly what that other limit theorem says, let me give you the big picture of what's involved here.
We're dealing with sums of i.i.d random variables.
Each X has a distribution of its own.
So suppose that X has a distribution which is something like this.
This is the density of X.
If I add lots of X's together, what kind of distribution do I expect?
The mean is going to be n times the mean of an individual X.
So if this is mu, I'm going to get a mean of n times mu.
But my variance will also increase.
When I add the random variables, I'm adding the variances.
So since the variance increases, we're going to get a distribution that's pretty wide.
So this is the density of X1 plus all the way to Xn.
So as n increases, my distribution shifts, because the mean is positive.
So I keep adding things.
And also, my distribution becomes wider and wider.
The variance increases.
Well, we started a different scaling.
We started a scaled version of this quantity when we looked at the weak law of large numbers.
In the weak law of large numbers, we take this random variable and divide it by n. And what the weak law tells us is that we're going to get a distribution that's very highly concentrated around the true mean, which is mu.
So this here would be the density of X1 plus Xn divided by n. Because I've divided by n, the mean has become the original mean, which is mu.
But the weak law of large numbers tells us that the distribution of this random variable is very concentrated around the mean.
So we get a distribution that's very narrow in this kind.
In the limit, this distribution becomes one that's just concentrated on top of mu.
So it's sort of a degenerate distribution.
So these are two extremes, no scaling for the sum, a scaling where we divide by n. In this extreme, we get the trivial case of a distribution that flattens out completely.
In this scaling, we get a distribution that gets concentrated around a single point.
Again, we look at some intermediate scaling that makes things more interesting.
Things do become interesting if we scale by dividing the sum by square root of n instead of dividing by n. What effect does this have?
When we scale by dividing by square root of n, the variance of Sn over square root of n is going to be the variance of Sn over sum divided by n. That's how variances behave.
The variance of Sn is n sigma-squared, divide by n, which is sigma squared, which means that when we scale in this particular way, as n changes, the variance doesn't change.
So the width of our distribution will be sort of constant.
The distribution changes shape, but it doesn't become narrower as was the case here.
It doesn't become wider, kind of keeps the same width.
So perhaps in the limit, this distribution is going to take an interesting shape.
And that's indeed the case.
So let's do what we did before.
So we're looking at the sum, and we want to divide the sum by something that goes like square root of n. So the variance of Sn is n sigma squared.
The variance of the sigma Sn is the square root of that.
It's this number.
So effectively, we're scaling by order of square root n. Now, I'm doing another thing here.
If my random variable has a positive mean, then this quantity is going to have a mean that's positive and growing.
It's going to be shifting to the right.
Why is that?
Sn has a mean that's proportional to n. When I divide by square root n, then it means that the mean scales like square root of n. So my distribution would still keep shifting after I do this division.
I want to keep my distribution in place, so I subtract out the mean of Sn.
So what we're doing here is a standard technique or transformation where you take a random variable and you so-called standardize it.
I remove the mean of that random variable and I divide by the standard deviation.
This results in a random variable that has 0 mean and unit variance.
What Zn measures is the following, Zn tells me how many standard deviations am I away from the mean.
Sn minus (n times expected value of X) tells me how much is Sn away from the mean value of Sn.
And by dividing by the standard deviation of Sn -- this tells me how many standard deviations away from the mean am I.
So we're going to look at this random variable, which is just a transformation Zn.
It's a linear transformation of Sn.
S And we're going to compare this random variable to a standard normal random variable.
So a standard normal is the random variable that you are familiar with, given by the usual formula, and for which we have tables for it.
This Zn has 0 mean and unit variance.
So in that respect, it has the same statistics as the standard normal.
The distribution of Zn could be anything -- can be pretty messy.
But there is this amazing theorem called the central limit theorem that tells us that the distribution of Zn approaches the distribution of the standard normal in the following sense, that probability is that you can calculate -- of this type -- that you can calculate for Zn -- is the limit becomes the same as the probabilities that you would get from the standard normal tables for Z.
It's a statement about the cumulative distribution functions.
This quantity, as a function of c, is the cumulative distribution function of the random variable Zn.
This is the cumulative distribution function of the standard normal.
The central limit theorem tells us that the cumulative distribution function of the sum of a number of random variables, after they're appropriately standardized, approaches the cumulative distribution function over the standard normal distribution.
In particular, this tells us that we can calculate probabilities for Zn when n is large by calculating instead probabilities for Z.
And that's going to be a good approximation.
Probabilities for Z are easy to calculate because they're well tabulated.
So we get a very nice shortcut for calculating probabilities for Zn.
Now, it's not Zn that you're interested in.
What you're interested in is Sn.
And Sn -- inverting this relation here -- Sn is square root n sigma Zn plus n expected value of X.
All right.
Now, if you can calculate probabilities for Zn, even approximately, then you can certainly calculate probabilities for Sn, because one is a linear function of the other.
And we're going to do a little bit of that next time.
You're going to get, also, some practice in recitation.
At a more vague level, you could describe the central limit theorem as saying the following, when n is large, you can pretend that Zn is a standard normal random variable and do the calculations as if Zn was standard normal.
Now, pretending that Zn is normal is the same as pretending that Sn is normal, because Sn is a linear function of Zn.
And we know that linear functions of normal random variables are normal.
So the central limit theorem essentially tells us that we can pretend that Sn is a normal random variable and do the calculations just as if it were a normal random variable.
Mathematically speaking though, the central limit theorem does not talk about the distribution of Sn, because the distribution of Sn becomes degenerate in the limit, just a very flat and long thing.
So strictly speaking mathematically, it's a statement about cumulative distributions of Zn's.
Practically, the way you use it is by just pretending that Sn is normal.
Very good.
Enjoy the Thanksgiving Holiday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're going to finish today our discussion of limit theorems.
I'm going to remind you what the central limit theorem is, which we introduced briefly last time.
We're going to discuss what exactly it says and its implications.
And then we're going to apply to a couple of examples, mostly on the binomial distribution.
OK, so the situation is that we are dealing with a large number of independent, identically distributed random variables.
And we want to look at the sum of them and say something about the distribution of the sum.
We might want to say that the sum is distributed approximately as a normal random variable, although, formally, this is not quite right.
As n goes to infinity, the distribution of the sum becomes very spread out, and it doesn't converge to a limiting distribution.
In order to get an interesting limit, we need first to take the sum and standardize it.
By standardizing it, what we mean is to subtract the mean and then divide by the standard deviation.
Now, the mean is, of course, n times the expected value of each one of the X's.
And the standard deviation is the square root of the variance.
The variance is n times sigma squared, where sigma is the variance of the X's -- so that's the standard deviation.
And after we do this, we obtain a random variable that has 0 mean -- its centered -- and the variance is equal to 1.
And so the variance stays the same, no matter how large n is going to be.
So the distribution of Zn keeps changing with n, but it cannot change too much.
It stays in place.
The mean is 0, and the width remains also roughly the same because the variance is 1.
The surprising thing is that, as n grows, that distribution of Zn kind of settles in a certain asymptotic shape.
And that's the shape of a standard normal random variable.
So standard normal means that it has 0 mean and unit variance.
More precisely, what the central limit theorem tells us is a relation between the cumulative distribution function of Zn and its relation to the cumulative distribution function of the standard normal.
So for any given number, c, the probability that Zn is less than or equal to c, in the limit, becomes the same as the probability that the standard normal becomes less than or equal to c. And of course, this is useful because these probabilities are available from the normal tables, whereas the distribution of Zn might be a very complicated expression if you were to calculate it exactly.
So some comments about the central limit theorem.
First thing is that it's quite amazing that it's universal.
It doesn't matter what the distribution of the X's is.
It can be any distribution whatsoever, as long as it has finite mean and finite variance.
And when you go and do your approximations using the central limit theorem, the only thing that you need to know about the distribution of the X's are the mean and the variance.
You need those in order to standardize Sn.
I mean -- to subtract the mean and divide by the standard deviation -- you need to know the mean and the variance.
But these are the only things that you need to know in order to apply it.
In addition, it's a very accurate computational shortcut.
So the distribution of this Zn's, in principle, you can calculate it by convolution of the distribution of the X's with itself many, many times.
But this is tedious, and if you try to do it analytically, it might be a very complicated expression.
Whereas by just appealing to the standard normal table for the standard normal random variable, things are done in a very quick way.
So it's a nice computational shortcut if you don't want to get an exact answer to a probability problem.
Now, at a more philosophical level, it justifies why we are really interested in normal random variables.
Whenever you have a phenomenon which is noisy, and the noise that you observe is created by adding the lots of little pieces of randomness that are independent of each other, the overall effect that you're going to observe can be described by a normal random variable.
So in a classic example that goes 100 years back or so, suppose that you have a fluid, and inside that fluid, there's a little particle of dust or whatever that's suspended in there.
That little particle gets hit by molecules completely at random -- and so what you're going to see is that particle kind of moving randomly inside that liquid.
Now that random motion, if you ask, after one second, how much is my particle displaced, let's say, in the x-axis along the x direction.
That displacement is very, very well modeled by a normal random variable.
And the reason is that the position of that particle is decided by the cumulative effect of lots of random hits by molecules that hit that particle.
So that's a sort of celebrated physical model that goes under the name of Brownian motion.
And it's the same model that some people use to describe the movement in the financial markets.
The argument might go that the movement of prices has to do with lots of little decisions and lots of little events by many, many different actors that are involved in the market.
So the distribution of stock prices might be well described by normal random variables.
At least that's what people wanted to believe until somewhat recently.
Now, the evidence is that, actually, these distributions are a little more heavy-tailed in the sense that extreme events are a little more likely to occur that what normal random variables would seem to indicate.
But as a first model, again, it could be a plausible argument to have, at least as a starting model, one that involves normal random variables.
So this is the philosophical side of things.
On the more accurate, mathematical side, it's important to appreciate exactly quite kind of statement the central limit theorem is.
It's a statement about the convergence of the CDF of these standardized random variables to the CDF of a normal.
So it's a statement about convergence of CDFs.
It's not a statement about convergence of PMFs, or convergence of PDFs.
Now, if one makes additional mathematical assumptions, there are variations of the central limit theorem that talk about PDFs and PMFs.
But in general, that's not necessarily the case.
And I'm going to illustrate this with-- I have a plot here which is not in your slides.
But just to make the point, consider two different discrete distributions.
This discrete distribution takes values 1, 4, 7.
This discrete distribution can take values 1, 2, 4, 6, and 7.
So this one has sort of a periodicity of 3, this one, the range of values is a little more interesting.
The numbers in these two distributions are cooked up so that they have the same mean and the same variance.
Now, what I'm going to do is to take eight independent copies of the random variable and plot the PMF of the sum of eight random variables.
Now, if I plot the PMF of the sum of 8 of these, I get the plot, which corresponds to these bullets in this diagram.
If I take 8 random variables, according to this distribution, and add them up and compute their PMF, the PMF I get is the one denoted here by the X's.
The two PMFs look really different, at least, when you eyeball them.
On the other hand, if you were to plot the CDFs of them, then the CDFs, if you compare them with the normal CDF, which is this continuous curve, the CDF, of course, it goes up in steps because we're looking at discrete random variables.
But it's very close to the normal CDF.
And if we, instead of n equal to 8, we were to take 16, then the coincidence would be even better.
So in terms of CDFs, when we add 8 or 16 of these, we get very close to the normal CDF.
We would get essentially the same picture if I were to take 8 or 16 of these.
So the CDFs sit, essentially, on top of each other, although the two PMFs look quite different.
So this is to appreciate that, formally speaking, we only have a statement about CDFs, not about PMFs.
Now in practice, how do you use the central limit theorem?
Well, it tells us that we can calculate probabilities by treating Zn as if it were a standard normal random variable.
Now Zn is a linear function of Sn.
Conversely, Sn is a linear function of Zn.
Linear functions of normals are normal.
So if I pretend that Zn is normal, it's essentially the same as if we pretend that Sn is normal.
And so we can calculate probabilities that have to do with Sn as if Sn were normal.
Now, the central limit theorem does not tell us that Sn is approximately normal.
The formal statement is about Zn, but, practically speaking, when you use the result, you can just pretend that Sn is normal.
Finally, it's a limit theorem, so it tells us about what happens when n goes to infinity.
If we are to use it in practice, of course, n is not going to be infinity.
Maybe n is equal to 15.
Can we use a limit theorem when n is a small number, as small as 15?
Well, it turns out that it's a very good approximation.
Even for quite small values of n, it gives us very accurate answers.
So n over the order of 15, or 20, or so give us very good results in practice.
There are no good theorems that will give us hard guarantees because the quality of the approximation does depend on the details of the distribution of the X's.
If the X's have a distribution that, from the outset, looks a little bit like the normal, then for small values of n, you are going to see, essentially, a normal distribution for the sum.
If the distribution of the X's is very different from the normal, it's going to take a larger value of n for the central limit theorem to take effect.
So let's illustrates this with a few representative plots.
So here, we're starting with a discrete uniform distribution that goes from 1 to 8.
Let's add 2 of these random variables, 2 random variables with this PMF, and find the PMF of the sum.
This is a convolution of 2 discrete uniforms, and I believe you have seen this exercise before.
When you convolve this with itself, you get a triangle.
So this is the PMF for the sum of two discrete uniforms.
Now let's continue.
Let's convolve this with itself.
These was going to give us the PMF of a sum of 4 discrete uniforms.
And we get this, which starts looking like a normal.
If we go to n equal to 32, then it looks, essentially, exactly like a normal.
And it's an excellent approximation.
So this is the PMF of the sum of 32 discrete random variables with this uniform distribution.
If we start with a PMF which is not symmetric-- this one is symmetric around the mean.
But if we start with a PMF which is non-symmetric, so this is, here, is a truncated geometric PMF, then things do not work out as nicely when I add 8 of these.
That is, if I convolve this with itself 8 times, I get this PMF, which maybe resembles a little bit to the normal one.
But you can really tell that it's different from the normal if you focus at the details here and there.
Here it sort of rises sharply.
Here it tails off a bit slower.
So there's an asymmetry here that's present, and which is a consequence of the asymmetry of the distribution we started with.
If we go to 16, it looks a little better, but still you can see the asymmetry between this tail and that tail.
If you get to 32 there's still a little bit of asymmetry, but at least now it starts looking like a normal distribution.
So the moral from these plots is that it might vary, a little bit, what kind of values of n you need before you get the really good approximation.
But for values of n in the range 20 to 30 or so, usually you expect to get a pretty good approximation.
At least that's what the visual inspection of these graphs tells us.
So now that we know that we have a good approximation in our hands, let's use it.
Let's use it by revisiting an example from last time.
This is the polling problem.
We're interested in the fraction of population that has a certain habit been.
And we try to find what f is.
And the way we do it is by polling people at random and recording the answers that they give, whether they have the habit or not.
So for each person, we get the Bernoulli random variable.
With probability f, a person is going to respond 1, or yes, so this is with probability f. And with the remaining probability 1-f, the person responds no.
We record this number, which is how many people answered yes, divided by the total number of people.
That's the fraction of the population that we asked.
This is the fraction inside our sample that answered yes.
And as we discussed last time, you might start with some specs for the poll.
And the specs have two parameters-- the accuracy that you want and the confidence that you want to have that you did really obtain the desired accuracy.
So the specs here is that we want, probability 95% that our estimate is within 1 % point from the true answer.
So the event of interest is this.
That's the result of the poll minus distance from the true answer is less or bigger than 1 % point.
And we're interested in calculating or approximating this particular probability.
So we want to do it using the central limit theorem.
And one way of arranging the mechanics of this calculation is to take the event of interest and massage it by subtracting and dividing things from both sides of this inequality so that you bring him to the picture the standardized random variable, the Zn, and then apply the central limit theorem.
So the event of interest, let me write it in full, Mn is this quantity, so I'm putting it here, minus f, which is the same as nf divided by n. So this is the same as that event.
We're going to calculate the probability of this.
This is not exactly in the form in which we apply the central limit theorem.
To apply the central limit theorem, we need, down here, to have sigma square root n. So how can I put sigma square root n here?
I can divide both sides of this inequality by sigma.
And then I can take a factor of square root n from here and send it to the other side.
So this event is the same as that event.
This will happen if and only if that will happen.
So calculating the probability of this event here is the same as calculating the probability that this events happens.
And now we are in business because the random variable that we got in here is Zn, or the absolute value of Zn, and we're talking about the probability that Zn, absolute value of Zn, is bigger than a certain number.
Since Zn is to be approximated by a standard normal random variable, our approximation is going to be, instead of asking for Zn being bigger than this number, we will ask for Z, absolute value of Z, being bigger than this number.
So this is the probability that we want to calculate.
And now Z is a standard normal random variable.
There's a small difficulty, the one that we also encountered last time.
And the difficulty is that the standard deviation, sigma, of the Xi's is not known.
Sigma is equal to f times-- sigma, in this example, is f times (1-f), and the only thing that we know about sigma is that it's going to be a number less than 1/2.
OK, so we're going to have to use an inequality here.
We're going to use a conservative value of sigma, the value of sigma equal to 1/2 and use that instead of the exact value of sigma.
And this gives us an inequality going this way.
Let's just make sure why the inequality goes this way.
We got, on our axis, two numbers.
One number is 0.01 square root n divided by sigma.
And the other number is 0.02 square root of n. And my claim is that the numbers are related to each other in this particular way.
Why is this?
Sigma is less than 2.
So 1/sigma is bigger than 2.
So since 1/sigma is bigger than 2 this means that this numbers sits to the right of that number.
So here we have the probability that Z is bigger than this number.
The probability of falling out there is less than the probability of falling in this interval.
So that's what that last inequality is saying-- this probability is smaller than that probability.
This is the probability that we're interested in, but since we don't know sigma, we take the conservative value, and we use an upper bound in terms of the probability of this interval here.
And now we are in business.
We can start using our normal tables to calculate probabilities of interest.
So for example, let's say that's we take n to be 10,000.
How is the calculation going to go?
We want to calculate the probability that the absolute value of Z is bigger than 0.2 times 1000, which is the probability that the absolute value of Z is larger than or equal to 2.
And here let's do some mechanics, just to stay in shape.
The probability that you're larger than or equal to 2 in absolute value, since the normal is symmetric around the mean, this is going to be twice the probability that Z is larger than or equal to 2.
Can we use the cumulative distribution function of Z to calculate this?
Well, almost the cumulative gives us probabilities of being less than something, not bigger than something.
So we need one more step and write this as 1 minus the probability that Z is less than or equal to 2.
And this probability, now, you can read off from the normal tables.
And the normal tables will tell you that this probability is 0.9772.
And you do get an answer.
And the answer is 0.0456.
OK, so we tried 10,000.
And we find that our probably of error is 4.5%, so we're doing better than the spec that we had.
So this tells us that maybe we have some leeway.
Maybe we can use a smaller sample size and still stay without our specs.
Let's try to find how much we can push the envelope.
How much smaller can we take n?
To answer that question, we need to do this kind of calculation, essentially, going backwards.
We're going to fix this number to be 0.05 and work backwards here to find-- did I do a mistake here?
10,000.
So I'm missing a 0 here.
Ah, but I'm taking the square root, so it's 100.
Where did the 0.02 come in from?
Ah, from here.
OK, all right.
0.02 times 100, that gives us 2.
OK, all right.
Very good, OK.
So we'll have to do this calculation now backwards, figure out if this is 0.05, what kind of number we're going to need here and then here, and from this we will be able to tell what value of n do we need.
OK, so we want to find n such that the probability that Z is bigger than 0.02 square root n is 0.05.
OK, so Z is a standard normal random variable.
And we want the probability that we are outside this range.
We want the probability of those two tails together.
Those two tails together should have probability of 0.05.
This means that this tail, by itself, should have probability 0.025.
And this means that this probability should be 0.975.
Now, if this probability is to be 0.975, what should that number be?
You go to the normal tables, and you find which is the entry that corresponds to that number.
I actually brought a normal table with me.
And 0.975 is down here.
And it tells you that to the number that corresponds to it is 1.96.
So this tells us that this number should be equal to 1.96.
And now, from here, you do the calculations.
And you find that n is 9604.
So with a sample of 10,000, we got probability of error 4.5%.
With a slightly smaller sample size of 9,600, we can get the probability of a mistake to be 0.05, which was exactly our spec.
So these are essentially the two ways that you're going to be using the central limit theorem.
Either you're given n and you try to calculate probabilities.
Or you're given the probabilities, and you want to work backwards to find n itself.
So in this example, the random variable that we dealt with was, of course, a binomial random variable.
The Xi's were Bernoulli, so the sum of the Xi's were binomial.
So the central limit theorem certainly applies to the binomial distribution.
To be more precise, of course, it applies to the standardized version of the binomial random variable.
So here's what we did, essentially, in the previous example.
We fixed the number p, which is the probability of success in our experiments.
p corresponds to f in the previous example.
Let every Xi a Bernoulli random variable and are standing assumption is that these random variables are independent.
When we add them, we get a random variable that has a binomial distribution.
We know the mean and the variance of the binomial, so we take Sn, we subtract the mean, which is this, divide by the standard deviation.
The central limit theorem tells us that the cumulative distribution function of this random variable is a standard normal random variable in the limit.
So let's do one more example of a calculation.
Let's take n to be-- let's choose some specific numbers to work with.
So in this example, first thing to do is to find the expected value of Sn, which is n times p. It's 18.
Then we need to write down the standard deviation.
The variance of Sn is the sum of the variances.
It's np times (1-p).
And in this particular example, p times (1-p) is 1/4, n is 36, so this is 9.
And that tells us that the standard deviation of this n is equal to 3.
So what we're going to do is to take the event of interest, which is Sn less than 21, and rewrite it in a way that involves the standardized random variable.
So to do that, we need to subtract the mean.
So we write this as Sn-3 should be less than or equal to 21-3.
This is the same event.
And then divide by the standard deviation, which is 3, and we end up with this.
So the event itself of--
[INAUDIBLE].
Should subtract, 18, yes, which gives me a much nicer number out here, which is 1.
So the event of interest, that Sn is less than 21, is the same as the event that a standard normal random variable is less than or equal to 1.
And once more, you can look this up at the normal tables.
And you find that the answer that you get is 0.43.
Now it's interesting to compare this answer that we got through the central limit theorem with the exact answer.
The exact answer involves the exact binomial distribution.
What we have here is the binomial probability that, Sn is equal to k. Sn being equal to k is given by this formula.
And we add, over all values for k going from 0 up to 21, we write a two lines code to calculate this sum, and we get the exact answer, which is 0.8785.
So there's a pretty good agreements between the two, although you wouldn't call that's necessarily excellent agreement.
Can we do a little better than that?
OK.
It turns out that we can.
And here's the idea.
So our random variable Sn has a mean of 18.
It has a binomial distribution.
It's described by a PMF that has a shape roughly like this and which keeps going on.
Using the central limit theorem is basically pretending that Sn is normal with the right mean and variance.
So pretending that Zn has 0 mean unit variance, we approximate it with Z, that has 0 mean unit variance.
If you were to pretend that Sn is normal, you would approximate it with a normal that has the correct mean and correct variance.
So it would still be centered at 18.
And it would have the same variance as the binomial PMF.
So using the central limit theorem essentially means that we keep the mean and the variance what they are but we pretend that our distribution is normal.
We want to calculate the probability that Sn is less than or equal to 21.
I pretend that my random variable is normal, so I draw a line here and I calculate the area under the normal curve going up to 21.
That's essentially what we did.
Now, a smart person comes around and says, Sn is a discrete random variable.
So the event that Sn is less than or equal to 21 is the same as Sn being strictly less than 22 because nothing in between can happen.
So I'm going to use the central limit theorem approximation by pretending again that Sn is normal and finding the probability of this event while pretending that Sn is normal.
So what this person would do would be to draw a line here, at 22, and calculate the area under the normal curve all the way to 22. Who is right?
Which one is better?
Well neither, but we can do better than both if we sort of split the difference.
So another way of writing the same event for Sn is to write it as Sn being less than 21.5.
In terms of the discrete random variable Sn, all three of these are exactly the same event.
But when you do the continuous approximation, they give you different probabilities.
It's a matter of whether you integrate the area under the normal curve up to here, up to the midway point, or up to 22.
It turns out that integrating up to the midpoint is what gives us the better numerical results.
So we take here 21 and 1/2, and we integrate the area under the normal curve up to here.
So let's do this calculation and see what we get.
What would we change here?
Instead of 21, we would now write 21 and 1/2.
This 18 becomes, no, that 18 stays what it is.
But this 21 becomes 21 and 1/2.
And so this one becomes 1 + 0.5 by 3.
This is 117.
So we now look up into the normal tables and ask for the probability that Z is less than 1.17.
So this here gets approximated by the probability that the standard normal is less than 1.17.
And the normal tables will tell us this is 0.879.
Going back to the previous slide, what we got this time with this improved approximation is 0.879.
This is a really good approximation of the correct number.
This is what we got using the 21.
This is what we get using the 21 and 1/2.
And it's an approximation that's sort of right on-- a very good one.
The moral from this numerical example is that doing this 1 and 1/2 correction does give us better approximations.
In fact, we can use this 1/2 idea to even calculate individual probabilities.
So suppose you want to approximate the probability that Sn equal to 19.
If you were to pretend that Sn is normal and calculate this probability, the probability that the normal random variable is equal to 19 is 0.
So you don't get an interesting answer.
You get a more interesting answer by writing this event, 19 as being the same as the event of falling between 18 and 1/2 and 19 and 1/2 and using the normal approximation to calculate this probability.
In terms of our previous picture, this corresponds to the following.
We are interested in the probability that Sn is equal to 19.
So we're interested in the height of this bar.
We're going to consider the area under the normal curve going from here to here, and use this area as an approximation for the height of that particular bar.
So what we're basically doing is, we take the probability under the normal curve that's assigned over a continuum of values and attributed it to different discrete values.
Whatever is above the midpoint gets attributed to 19.
Whatever is below that midpoint gets attributed to 18.
So this is green area is our approximation of the value of the PMF at 19.
So similarly, if you wanted to approximate the value of the PMF at this point, you would take this interval and integrate the area under the normal curve over that interval.
It turns out that this gives a very good approximation of the PMF of the binomial.
And actually, this was the context in which the central limit theorem was proved in the first place, when this business started.
So this business goes back a few hundred years.
And the central limit theorem was first approved by considering the PMF of a binomial random variable when p is equal to 1/2.
People did the algebra, and they found out that the exact expression for the PMF is quite well approximated by that expression hat you would get from a normal distribution.
Then the proof was extended to binomials for more general values of p. So here we talk about this as a refinement of the general central limit theorem, but, historically, that refinement was where the whole business got started in the first place.
All right, so let's go through the mechanics of approximating the probability that Sn is equal to 19-- exactly 19.
As we said, we're going to write this event as an event that covers an interval of unit length from 18 and 1/2 to 19 and 1/2.
This is the event of interest.
First step is to massage the event of interest so that it involves our Zn random variable.
So subtract 18 from all sides.
Divide by the standard deviation of 3 from all sides.
That's the equivalent representation of the event.
This is our standardized random variable Zn.
These are just these numbers.
And to do an approximation, we want to find the probability of this event, but Zn is approximately normal, so we plug in here the Z, which is the standard normal.
So we want to find the probability that the standard normal falls inside this interval.
You find these using CDFs because this is the probability that you're less than this but not less than that.
So it's a difference between two cumulative probabilities.
Then, you look up your normal tables.
You find two numbers for these quantities, and, finally, you get a numerical answer for an individual entry of the PMF of the binomial.
This is a pretty good approximation, it turns out.
If you were to do the calculations using the exact formula, you would get something which is pretty close-- an error in the third digit-- this is pretty good.
So I guess what we did here with our discussion of the binomial slightly contradicts what I said before-- that the central limit theorem is a statement about cumulative distribution functions.
In general, it doesn't tell you what to do to approximate PMFs themselves.
And that's indeed the case in general.
One the other hand, for the special case of a binomial distribution, the central limit theorem approximation, with this 1/2 correction, is a very good approximation even for the individual PMF.
All right, so we spent quite a bit of time on mechanics.
So let's spend the last few minutes today thinking a bit and look at a small puzzle.
So the puzzle is the following.
Consider Poisson process that runs over a unit interval.
And where the arrival rate is equal to 1.
So this is the unit interval.
And let X be the number of arrivals.
And this is Poisson, with mean 1.
Now, let me take this interval and divide it into n little pieces.
So each piece has length 1/n.
And let Xi be the number of arrivals during the Ith little interval.
OK, what do we know about the random variables Xi?
Is they are themselves Poisson.
It's a number of arrivals during a small interval.
We also know that when n is big, so the length of the interval is small, these Xi's are approximately Bernoulli, with mean 1/n.
Guess it doesn't matter whether we model them as Bernoulli or not.
What matters is that the Xi's are independent.
Why are they independent?
Because, in a Poisson process, these joint intervals are independent of each other.
So the Xi's are independent.
And they also have the same distribution.
And we have that X, the total number of arrivals, is the sum over the Xn's.
So the central limit theorem tells us that, approximately, the sum of independent, identically distributed random variables, when we have lots of these random variables, behaves like a normal random variable.
So by using this decomposition of X into a sum of i.i.d random variables, and by using values of n that are bigger and bigger, by taking the limit, it should follow that X has a normal distribution.
On the other hand, we know that X has a Poisson distribution.
So something must be wrong in this argument here.
Can we really use the central limit theorem in this situation?
So what do we need for the central limit theorem?
We need to have independent, identically distributed random variables.
We have it here.
We want them to have a finite mean and finite variance.
We also have it here, means variances are finite.
What is another assumption that was never made explicit, but essentially was there?
Or in other words, what is the flaw in this argument that uses the central limit theorem here?
Any thoughts?
So in the central limit theorem, we said, consider-- fix a probability distribution, and let the Xi's be distributed according to that probability distribution, and add a larger and larger number or Xi's.
But the underlying, unstated assumption is that we fix the distribution of the Xi's.
As we let n increase, the statistics of each Xi do not change.
Whereas here, I'm playing a trick on you.
As I'm taking more and more random variables, I'm actually changing what those random variables are.
When I take a larger n, the Xi's are random variables with a different mean and different variance.
So I'm adding more of these, but at the same time, in this example, I'm changing their distributions.
That's something that doesn't fit the setting of the central limit theorem.
In the central limit theorem, you first fix the distribution of the X's.
You keep it fixed, and then you consider adding more and more according to that particular fixed distribution.
So that's the catch.
That's why the central limit theorem does not apply to this situation.
And we're lucky that it doesn't apply because, otherwise, we would have a huge contradiction destroying probability theory.
OK, but now that's still leaves us with a little bit of a dilemma.
Suppose that, here, essentially we're adding independent Bernoulli random variables.
So the issue is that the central limit theorem has to do with asymptotics as n goes to infinity.
And if we consider a binomial, and somebody gives us specific numbers about the parameters of that binomial, it might not necessarily be obvious what kind of approximation do we use.
In particular, we do have two different approximations for the binomial.
If we fix p, then the binomial is the sum of Bernoulli's that come from a fixed distribution, we consider more and more of these.
When we add them, the central limit theorem tells us that we get the normal distribution.
There's another sort of limit, which has the flavor of this example, in which we still deal with a binomial, sum of n Bernoulli's.
We let that sum, the number of the Bernoulli's go to infinity.
But each Bernoulli has a probability of success that goes to 0, and we do this in a way so that np, the expected number of successes, stays finite.
This is the situation that we dealt with when we first defined our Poisson process.
We have a very, very large number so lots, of time slots, but during each time slot, there's a tiny probability of obtaining an arrival.
Under that setting, in discrete time, we have a binomial distribution, or Bernoulli process, but when we take the limit, we obtain the Poisson process and the Poisson approximation.
So these are two equally valid approximations of the binomial.
But they're valid in different asymptotic regimes.
In one regime, we fixed p, let n go to infinity.
In the other regime, we let both n and p change simultaneously.
Now, in real life, you're never dealing with the limiting situations.
You're dealing with actual numbers.
So if somebody tells you that the numbers are like this, then you should probably say that this is the situation that fits the Poisson description-- large number of slots with each slot having a tiny probability of success.
On the other hand, if p is something like this, and n is 500, then you expect to get the distribution for the number of successes.
It's going to have a mean of 50 and to have a fair amount of spread around there.
It turns out that the normal approximation would be better in this context.
As a rule of thumb, if n times p is bigger than 10 or 20, you can start using the normal approximation.
If n times p is a small number, then you prefer to use the Poisson approximation.
But there's no hard theorems or rules about how to go about this.
OK, so from next time we're going to switch base again.
And we're going to put together everything we learned in this class to start solving inference problems.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
It involves real phenomena out there.
So we have real stuff that happens.
So it might be an arrival process to a bank that we're trying to model.
This is a reality, but this is what we have been doing so far.
We have been playing with models of probabilistic phenomena.
And somehow we need to tie the two together.
The way these are tied is that we observe the real world and this gives us data.
And then based on these data, we try to come up with a model of what exactly is going on.
For example, for an arrival process, you might ask the model in question, is my arrival process Poisson or is it something different?
If it is Poisson, what is the rate of the arrival process?
Once you come up with your model and you come up with the parameters of the model, then you can use it to make predictions about reality or to figure out certain hidden things, certain hidden aspects of reality, that you do not observe directly, but you try to infer what they are.
So that's where the usefulness of the model comes in.
Now this field is of course tremendously useful.
And it shows up pretty much everywhere.
So we talked about the polling examples in the last couple of lectures.
This is, of course, a real application.
You sample and on the basis of the sample that you have, you try to make some inferences about, let's say, the preferences in a given population.
Let's say in the medical field, you want to try whether a certain drug makes a difference or not.
So people would do medical trials, get some results, and then from the data somehow you need to make sense of them and make a decision.
Is the new drug useful or is it not?
How do we go systematically about the question of this type?
A sexier, more recent topic, there's this famous Netflix competition where Netflix gives you a huge table of movies and people.
And people have rated the movies, but not everyone has watched all of the movies in there.
You have some of the ratings.
For example, this person gave a 4 to that particular movie.
So you get the table that's partially filled.
And the Netflix asks you to make recommendations to people.
So this means trying to guess.
This person here, how much would they like this particular movie?
And you can start thinking, well, maybe this person has given somewhat similar ratings with another person.
And if that other person has also seen that movie, maybe the rating of that other person is relevant.
But of course it's a lot more complicated than that.
And this has been a serious competition where people have been using every heavy, wet machinery that there is in statistics, trying to come up with good recommendation systems.
Then the other people, of course, are trying to analyze financial data.
Somebody gives you the sequence of the values, let's say of the SMP index.
You look at something like this and you can ask questions.
How do I model these data using any of the models that we have in our bag of tools?
How can I make predictions about what's going to happen afterwards, and so on?
On the engineering side, anywhere where you have noise inference comes in.
Signal processing, in some sense, is just an inference problem.
You observe signals that are noisy and you try to figure out exactly what's happening out there or what kind of signal has been sent.
Maybe the beginning of the field could be traced a few hundred years ago where people would observe, make astronomical observations of the position of the planets in the sky.
They would have some beliefs that perhaps the orbits of planets is an ellipse.
Or if it's a comet, maybe it's a parabola, hyperbola, don't know what it is.
But they would have a model of that.
But, of course, astronomical measurements would not be perfectly exact.
And they would try to find the curve that fits these data.
How do you go about choosing this particular curve on the base of noisy data and try to do it in a somewhat principled way?
OK, so questions of this type-- clearly the applications are all over the place.
But how is this related conceptually with what we have been doing so far?
What's the relation between the field of inference and the field of probability as we have been practicing until now?
Well, mathematically speaking, what's going to happen in the next few lectures could be just exercises or homework problems in the class in based on what we have done so far.
That means you're not going to get any new facts about probability theory.
Everything we're going to do will be simple applications of things that you already do know.
So in some sense, statistics and inference is just an applied exercise in probability.
But actually, things are not that simple in the following sense.
If you get a probability problem, there's a correct answer.
There's a correct solution.
And that correct solution is unique.
There's no ambiguity.
The theory of probability has clearly defined rules.
These are the axioms.
You're given some information about probability distributions.
You're asked to calculate certain other things.
There's no ambiguity.
Answers are always unique.
In statistical questions, it's no longer the case that the question has a unique answer.
If I give you data and I ask you what's the best way of estimating the motion of that planet, reasonable people can come up with different methods.
And reasonable people will try to argue that's my method has these desirable properties but somebody else may say, here's another method that has certain desirable properties.
And it's not clear what the best method is.
So it's good to have some understanding of what the issues are and to know at least what is the general class of methods that one tries to consider, how does one go about such problems.
So we're going to see lots and lots of different inference methods.
We're not going to tell you that one is better than the other.
But it's important to understand what are the concepts between those different methods.
And finally, statistics can be misused really badly.
That is, one can come up with methods that you think are sound, but in fact they're not quite that.
I will bring some examples next time and talk a little more about this.
So, they want to say, you have some data, you want to make some inference from them, what many people will do is to go to Wikipedia, find a statistical test that they think it applies to that situation, plug in numbers, and present results.
Are the conclusions that they get really justified or are they misusing statistical methods?
Well, too many people actually do misuse statistics and conclusions that people get are often false.
So it's important to, besides just being able to copy statistical tests and use them, to understand what are the assumptions between the different methods and what kind of guarantees they have, if any.
All right, so we'll try to do a quick tour through the field of inference in this lecture and the next few lectures that we have left this semester and try to highlight at the very high level the main concept skills, and techniques that come in.
Let's start with some generalities and some general statements.
One first statement is that statistics or inference problems come up in very different guises.
And they may look as if they are of very different forms.
Although, at some fundamental level, the basic issues turn out to be always pretty much the same.
So let's look at this example.
There's an unknown signal that's being sent.
It's sent through some medium, and that medium just takes the signal and amplifies it by a certain number.
So you can think of somebody shouting.
There's the air out there.
What you shouted will be attenuated through the air until it gets to a receiver.
And that receiver then observes this, but together with some random noise.
Here I meant S. S is the signal that's being sent.
And what you observe is an X.
You observe X, so what kind of inference problems could we have here?
In some cases, you want to build a model of the physical phenomenon that you're dealing with.
So for example, you don't know the attenuation of your signal and you try to find out what this number is based on the observations that you have.
So the way this is done in engineering systems is that you design a certain signal, you know what it is, you shout a particular word, and then the receiver listens.
And based on the intensity of the signal that they get, they try to make a guess about A.
So you don't know A, but you know S. And by observing X, you get some information about what A is.
So in this case, you're trying to build a model of the medium through which your signal is propagating.
So sometimes one would call problems of this kind, let's say, system identification.
In a different version of an inference problem that comes with this picture, you've done your modeling.
You know your A.
You know the medium through which the signal is going, but it's a communication system.
This person is trying to communicate something to that person.
So you send the signal S, but that person receives a noisy version of S. So that person tries to reconstruct S based on X.
So in both cases, we have a linear relation between X and the unknown quantity.
In one version, A is the unknown and we know S. In the other version, A is known, and so we try to infer S. Mathematically, you can see that this is essentially the same kind of problem in both cases.
Although, the kind of practical problem that you're trying to solve is a little different.
So we will not be making any distinctions between problems of the model building type as opposed to models where you try to estimate some unknown signal and so on.
Because conceptually, the tools that one uses for both types of problems are essentially the same.
OK, next a very useful classification of inference problems-- the unknown quantity that you're trying to estimate could be either a discrete one that takes a small number of values.
So this could be discrete problems, such as the airplane radar problem we encountered back a long time ago in this class.
So there's two possibilities-- an airplane is out there or an airplane is not out there.
And you're trying to make a decision between these two options.
Or you can have other problems would you have, let's say, four possible options.
You don't know which one is true, but you get data and you try to figure out which one is true.
In problems of these kind, usually you want to make a decision based on your data.
And you're interested in the probability of making a correct decision.
You would like that probability to be as high as possible.
Estimation problems are a little different.
Here you have some continuous quantity that's not known.
And you try to make a good guess of that quantity.
And you would like your guess to be as close as possible to the true quantity.
So the polling problem was of this type.
There was an unknown fraction f of the population that had some property.
And you try to estimate f as accurately as you can.
So the distinction here is that usually here the unknown quantity takes on discrete set of values.
Here the unknown quantity takes a continuous set of values.
Here we're interested in the probability of error.
Here we're interested in the size of the error.
Broadly speaking, most inference problems fall either in this category or in that category.
Although, if you want to complicate life, you can also think or construct problems where both of these aspects are simultaneously present.
OK, finally since we're in classification mode, there is a very big, important dichotomy into how one goes about inference problems.
And here there's two fundamentally different philosophical points of view, which is how do we model the quantity that is unknown?
In one approach, you say there's a certain quantity that has a definite value.
It just happens that they don't know it.
But it's a number.
There's nothing random about it.
So think of trying to estimate some physical quantity.
You're making measurements, you try to estimate the mass of an electron, which is a sort of universal physical constant.
There's nothing random about it.
It's a fixed number.
You get data, because you have some measuring apparatus.
And that measuring apparatus, depending on what that results that you get are affected by the true mass of the electron, but there's also some noise.
You take the data out of your measuring apparatus and you try to come up with some estimate of that quantity theta.
So this is definitely a legitimate picture, but the important thing in this picture is that this theta is written as lowercase.
And that's to make the point that it's a real number, not a random variable.
There's a different philosophical approach which says, well, anything that I don't know I should model it as a random variable.
Yes, I know.
The mass of the electron is not really random.
It's a constant.
But I don't know what it is.
I have some vague sense, perhaps, what it is perhaps because of the experiments that some other people carried out.
So perhaps I have a prior distribution on the possible values of Theta.
And that prior distribution doesn't mean that the nature is random, but it's more of a subjective description of my subjective beliefs of where do I think this constant number happens to be.
So even though it's not truly random, I model my initial beliefs before the experiment starts.
In terms of a prior distribution, I view it as a random variable.
Then I observe another related random variable through some measuring apparatus.
And then I use this again to create an estimate.
So these two pictures philosophically are very different from each other.
Here we treat the unknown quantities as unknown numbers.
Here we treat them as random variables.
When we treat them as a random variables, then we know pretty much already what we should be doing.
We should just use the Bayes rule.
Based on X, find the conditional distribution of Theta.
And that's what we will be doing mostly over this lecture and the next lecture.
Now in both cases, what you end up getting at the end is an estimate.
But actually, that estimate is what kind of object is it?
It's a random variable in both cases.
Why?
Even in this case where theta was a constant, my data are random.
I do my data processing.
So I calculate a function of the data, the data are random variables.
So out here we output something which is a function of a random variable.
So this quantity here will be also random.
It's affected by the noise and the experiment that I have been doing.
That's why these estimators will be denoted by uppercase Thetas.
And we will be using hats.
Hat, usually in estimation, means an estimate of something.
All right, so this is the big picture.
We're going to start with the Bayesian version.
And then the last few lectures we're going to talk about the non-Bayesian version or the classical one.
By the way, I should say that statisticians have been debating fiercely for 100 years whether the right way to approach statistics is to go the classical way or the Bayesian way.
And there have been tides going back and forth between the two sides.
These days, Bayesian methods tend to become a little more popular for various reasons.
We're going to come back to this later.
All right, so in Bayesian estimation, what we got in our hands is Bayes rule.
And if you have Bayes rule, there's not a lot that's left to do.
We have different forms of the Bayes rule, depending on whether we're dealing with discrete data, And discrete quantities to estimate, or continuous data, and so on.
In the hypothesis testing problem, the unknown quantity Theta is discrete.
So in both cases here, we have a P of Theta.
We obtain data, the X's.
And on the basis of the X that we observe, we can calculate the posterior distribution of Theta, given the data.
So to use Bayesian inference, what do we start with?
We start with some priors.
These are our initial beliefs about what Theta that might be.
That's before we do the experiment.
We have a model of the experimental aparatus.
And the model of the experimental apparatus tells us if this Theta is true, I'm going to see X's of that kind.
If that other Theta is true, I'm going to see X's that they are somewhere else.
That models my apparatus.
And based on that knowledge, once I observe I have these two functions in my hands, we have already seen that if you know those two functions, you can also calculate the denominator here.
So all of these functions are available, so you can compute, you can find a formula for this function as well.
And as soon as you observe the data, that X's, you plug in here the numerical value of those X's.
And you get a function of Theta.
And this is the posterior distribution of Theta, given the data that you have seen.
So you've already done a fair number of exercises of these kind.
So we not say more about this.
And there's a similar formula as you know for the case where we have continuous data.
If the X's are continuous random variable, then the formula is the same, except that X's are described by densities instead of being described by a probability mass functions.
OK, now if Theta is continuous, then we're dealing with estimation problems.
But the story is once more the same.
You're going to use the Bayes rule to come up with the posterior density of Theta, given the data that you have observed.
Now just for the sake of the example, let's come back to this picture here.
Suppose that something is flying in the air, and maybe this is just an object in the air close to the Earth.
So because of gravity, the trajectory that it's going to follow it's going to be a parabola.
So this is the general equation of a parabola.
Zt is the position of my objects at time t. But I don't know exactly which parabola it is.
So the parameters of the parabola are unknown quantities.
What I can do is to go and measure the position of my objects at different times.
But unfortunately, my measurements are noisy.
What I want to do is to model the motion of my object.
So I guess in the picture, the axis would be t going this way and Z going this way.
And on the basis of the data that they get, these are my X's.
I want to figure out the Thetas.
That is, I want to figure out the exact equation of this parabola.
Now if somebody gives you probability distributions for Theta, these would be your priors.
So this is given.
We need the conditional distribution of the X's given the Thetas.
Well, we have the conditional distribution of Z, given the Thetas from this equation.
And then by playing with this equation, you can also find how is X distributed if Theta takes a particular value.
So you do have all of the densities that you might need.
And you can apply the Bayes rule.
And at the end, your end result would be a formula for the distribution of Theta, given to the X that you have observed-- except for one sort of computation, or to make things more interesting.
Instead of these X's and Theta's being single random variables that we have here, typically those X's and Theta's will be multi-dimensional random variables or will correspond to multiple ones.
So this little Theta here actually stands for a triplet of Theta0, Theta1, and Theta2.
And that X here stands here for the entire sequence of X's that we have observed.
So in reality, the object that you're going to get at to the end after inference is done is a function that you plug in the values of the data and you get the function of the Theta's that tells you the relative likelihoods of different Theta triplets.
So what I'm saying is that this is no harder than the problems that you have dealt with so far, except perhaps for the complication that's usually in interesting inference problems.
Your Theta's and X's are often the vectors of random variables instead of individual random variables.
Now if you are to do estimation in a case where you have discrete data, again the situation is no different.
We still have a Bayes rule of the same kind, except that densities gets replaced by PMF's.
If X is discrete, you put a P here instead of putting an f. So an example of an estimation problem with discrete data is similar to the polling problem.
You have a coin.
It has an unknown parameter Theta.
This is the probability of obtaining heads.
You flip the coin many times.
What can you tell me about the true value of Theta?
A classical statistician, at this point, would say, OK, I'm going to use an estimator, the most reasonable one, which is this.
How many heads did they obtain in n trials?
Divide by the total number of trials.
This is my estimate of the bias of my coin.
And then the classical statistician would continue from here and try to prove some properties and argue that this estimate is a good one.
For example, we have the weak law of large numbers that tells us that this particular estimate converges in probability to the true parameter.
This is a kind of guarantee that's useful to have.
And the classical statistician would pretty much close the subject in this way.
What would the Bayesian person do differently?
The Bayesian person would start by assuming a prior distribution of Theta.
Instead of treating Theta as an unknown constant, they would say that Theta would speak randomly or pretend that it would speak randomly and assume a distribution on Theta.
So for example, if you don't know they need anything more, you might assume that any value for the bias of the coin is as likely as any other value of the bias of the coin.
And this way so the probability distribution that's uniform.
Or if you have a little more faith in the manufacturing processes that's created that coin, you might choose your prior to be a distribution that's centered around 1/2 and sits fairly narrowly centered around 1/2.
That would be a prior distribution in which you say, well, I believe that the manufacturer tried to make my coin to be fair.
But they often makes some mistakes, so it's going to be, I believe, it's approximately 1/2 but not quite.
So depending on your beliefs, you would choose an appropriate prior for the distribution of Theta.
And then you would use the Bayes rule to find the probabilities of different values of Theta, based on the data that you have observed.
So no matter which version of the Bayes rule that you use, the end product of the Bayes rule is going to be either a plot of this kind or a plot of that kind.
So what am I plotting here?
This axis is the Theta axis.
These are the possible values of the unknown quantity that we're trying to estimate.
In the continuous case, theta is a continuous random variable.
I obtain my data.
And I plot for the posterior probability distribution after observing my data.
And I'm plotting here the probability density for Theta.
So this is a plot of that density.
In the discrete case, theta can take finitely many values or a discrete set of values.
And for each one of those values, I'm telling you how likely is that the value to be the correct one, given the data that I have observed.
And in general, what you would go back to your boss and report after you've done all your inference work would be either a plot of this kinds or of that kind.
So you go to your boss who asks you, what is the value of Theta?
And you say, well, I only have limited data.
That I don't know what it is.
It could be this, with so much probability.
There's probability.
OK, let's throw in some numbers here.
There's probability 0.3 that Theta is this value.
There's probability 0.2 that Theta is this value, 0.1 that it's this one, 0.1 that it's this one, 0.2 that it's that one, and so on.
OK, now bosses often want simple answers.
They say, OK, you're talking too much.
What do you think Theta is?
And now you're forced to make a decision.
If that was the situation and you have to make a decision, how would you make it?
Well, I'm going to make a decision that's most likely to be correct.
If I make this decision, what's going to happen?
Theta is this value with probability 0.2, which means there's probably 0.8 that they make an error if I make that guess.
If I make that decision, this decision has probably 0.3 of being the correct one.
So I have probably of error 0.7.
So if you want to just maximize the probability of giving the correct decision, or if you want to minimize the probability of making an incorrect decision, what you're going to choose to report is that value of Theta for which the probability is highest.
So in this case, I would choose to report this particular value, the most likely value of Theta, given what I have observed.
And that value is called them maximum a posteriori probability estimate.
It's going to be this one in our case.
So picking the point in the posterior PMF that has the highest probability.
That's the reasonable thing to do.
This is the optimal thing to do if you want to minimize the probability of an incorrect inference.
And that's what people do usually if they need to report a single answer, if they need to report a single decision.
How about in the estimation context?
If that's what you know about Theta, Theta could be around here, but there's also some sharp probability that it is around here.
What's the single answer that you would give to your boss?
One option is to use the same philosophy and say, OK, I'm going to find the Theta at which this posterior density is highest.
So I would pick this point here and report this particular Theta.
So this would be my Theta, again, Theta MAP, the Theta that has the highest a posteriori probability, just because it corresponds to the peak of the density.
But in this context, the maximum a posteriori probability theta was the one that was most likely to be true.
In the continuous case, you cannot really say that this is the most likely value of Theta.
In a continuous setting, any value of Theta has zero probability, so when we talk about densities.
So it's not the most likely.
It's the one for which the density, so the probabilities of that neighborhoods, are highest.
So the rationale for picking this particular estimate in the continuous case is much less compelling than the rationale that we had in here.
So in this case, reasonable people might choose different quantities to report.
And the very popular one would be to report instead the conditional expectation.
So I don't know quite what Theta is.
Given the data that I have, Theta has this distribution.
Let me just report the average over that distribution.
Let me report to the center of gravity of this figure.
And in this figure, the center of gravity would probably be somewhere around here.
And that would be a different estimate that you might choose to report.
So center of gravity is something around here.
And this is a conditional expectation of Theta, given the data that you have.
So these are two, in some sense, fairly reasonable ways of choosing what to report to your boss.
Some people might choose to report this.
Some people might choose to report that.
And a priori, if there's no compelling reason why one would be preferable than other one, unless you set some rules for the game and you describe a little more precisely what your objectives are.
But no matter which one you report, a single answer, a point estimate, doesn't really tell you the whole story.
There's a lot more information conveyed by this posterior distribution plot than any single number that you might report.
So in general, you may wish to convince your boss that's it's worth their time to look at the entire plot, because that plot sort of covers all the possibilities.
It tells your boss most likely we're in that range, but there's also a distinct change that our Theta happens to lie in that range.
All right, now let us try to perhaps differentiate between these two and see under what circumstances this one might be the better estimate to perform.
Better with respect to what?
We need some rules.
So we're going to throw in some rules.
As a warm up, we're going to deal with the problem of making an estimation if you had no information at all, except for a prior distribution.
So this is a warm up for what's coming next, which would be estimation that takes into account some information.
So we have a Theta.
And because of your subjective beliefs or models by others, you believe that Theta is uniformly distributed between, let's say, 4 and 10.
You want to come up with a point estimate.
Let's try to look for an estimate.
Call it c, in this case.
I want to pick a number with which to estimate the value of Theta.
I will be interested in the size of the error that I make.
And I really dislike large errors, so I'm going to focus on the square of the error that they make.
So I pick c. Theta that has a random value that I don't know.
But whatever it is, once it becomes known, it results into a squared error between what it is and what I guessed that it was.
And I'm interested in making a small air on the average, where the average is taken with respect to all the possible and unknown values of Theta.
So the problem, this is a least squares formulation of the problem, where we try to minimize the least squares errors.
How do you find the optimal c?
Well, we take that expression and expand it.
And it is, using linearity of expectations-- square minus 2c expected Theta plus c squared-- that's the quantity that we want to minimize, with respect to c. To do the minimization, take the derivative with respect to c and set it to 0.
So that differentiation gives us from here minus 2 expected value of Theta plus 2c is equal to 0.
And the answer that you get by solving this equation is that c is the expected value of Theta.
So when you do this optimization, you find that the optimal estimate, the things you should be reporting, is the expected value of Theta.
So in this particular example, you would choose your estimate c to be just the middle of these values, which would be 7.
OK, and in case your boss asks you, how good is your estimate?
How big is your error going to be?
What you could report is the average size of the estimation error that you are making.
We picked our estimates to be the expected value of Theta.
So for this particular way that I'm choosing to do my estimation, this is the mean squared error that I get.
And this is a familiar quantity.
It's just the variance of the distribution.
So the expectation is that best way to estimate a quantity, if you're interested in the mean squared error.
And the resulting mean squared error is the variance itself.
How will this story change if we now have data as well?
Now having data means that we can compute posterior distributions or conditional distributions.
So we get transported into a new universe where instead the working with the original distribution of Theta, the prior distribution, now we work with the condition of distribution of Theta, given the data that we have observed.
Now remember our old slogan that conditional models and conditional probabilities are no different than ordinary probabilities, except that we live now in a new universe where the new information has been taken into account.
So if you use that philosophy and you're asked to minimize the squared error but now that you live in a new universe where X has been fixed to something, what would the optimal solution be?
It would again be the expectation of theta, but which expectation?
It's the expectation which applies in the new conditional universe in which we live right now.
So because of what we did before, by the same calculation, we would find that the optimal estimates is the expected value of X of Theta, but the optimal estimate that takes into account the information that we have.
So the conclusion, once you get your data, if you want to minimize the mean squared error, you should just report the conditional estimation of this unknown quantity based on the data that you have.
So the picture here is that Theta is unknown.
You have your apparatus that creates measurements.
So this creates an X.
You take an X, and here you have a box that does calculations.
It does calculations and it spits out the conditional expectation of Theta, given the particular data that you have observed.
And what we have done in this class so far is, to some extent, developing the computational tools and skills to do with this particular calculation-- how to calculate the posterior density for Theta and how to calculate expectations, conditional expectations.
So in principle, we know how to do this.
In principle, we can program a computer to take the data and to spit out condition expectations.
Somebody who doesn't think like us might instead design a calculating machine that does something differently and produces some other estimate.
So we went through this argument and we decided to program our computer to calculate conditional expectations.
Somebody else came up with some other crazy idea for how to estimate the random variable.
They came up with some function g and the programmed it, and they designed a machine that estimates Theta's by outputting a certain g of X.
That could be an alternative estimator.
Which one is better?
Well, we convinced ourselves that this is the optimal one in a universe where we have fixed the particular value of the data.
So what we have proved so far is a relation of this kind.
In this conditional universe, the mean squared error that I get-- I'm the one who's using this estimator-- is less than or equal than the mean squared error that this person will get, the person who uses that estimator.
For any particular value of the data, I'm going to do better than the other person.
Now the data themselves are random.
If I average over all possible values of the data, I should still be better off.
If I'm better off for any possible value X, then I should be better off on the average over all possible values of X.
So let us average both sides of this quantity with respect to the probability distribution of X.
If you want to do it formally, you can write this inequality between numbers as an inequality between random variables.
And it tells that no matter what that random variable turns out to be, this quantity is better than that quantity.
Take expectations of both sides, and you get this inequality between expectations overall.
And this last inequality tells me that the person who's using this estimator who produces estimates according to this machine will have a mean squared estimation error that's less than or equal to the estimation error that's produced by the other person.
In a few words, the conditional expectation estimator is the optimal estimator.
It's the ultimate estimating machine.
That's how you should solve estimation problems and report a single value.
If you're forced to report a single value and if you're interested in estimation errors.
OK, while we could have told you that story, of course, a month or two ago, this is really about interpretation -- about realizing that conditional expectations have a very nice property.
But other than that, any probabilistic skills that come into this business are just the probabilistic skills of being able to calculate conditional expectations, which you already know how to do.
So conclusion, all of optimal Bayesian estimation just means calculating and reporting conditional expectations.
Well, if the world were that simple, then statisticians wouldn't be able to find jobs if life is that simple.
So real life is not that simple.
There are complications.
And that perhaps makes their life a little more interesting.
OK, one complication is that we would deal with the vectors instead of just single random variables.
I use the notation here as if X was a single random variable.
In real life, you get several data.
Does our story change?
Not really, same argument-- given all the data that you have observed, you should still report the conditional expectation of Theta.
But what kind of work does it take in order to report this conditional expectation?
One issue is that you need to cook up a plausible prior distribution for Theta.
How do you do that?
In a given application , this is a bit of a judgment call, what prior would you be working with.
And there's a certain skill there of not making silly choices.
A more pragmatic, practical issue is that this is a formula that's extremely nice and compact and simple that you can write with minimal ink.
But the behind it there could be hidden a huge amount of calculation.
So doing any sort of calculations that involve multiple random variables really involves calculating multi-dimensional integrals.
And the multi-dimensional integrals are hard to compute.
So implementing actually this calculating machine here may not be easy, might be complicated computationally.
It's also complicated in terms of not being able to derive intuition about it.
So perhaps you might want to have a simpler version, a simpler alternative to this formula that's easier to work with and easier to calculate.
We will be talking about one such simpler alternative next time.
So again, to conclude, at the high level, Bayesian estimation is very, very simple, given that you have mastered everything that has happened in this course so far.
There are certain practical issues and it's also good to be familiar with the concepts and the issues that in general, you would prefer to report that complete posterior distribution.
But if you're forced to report a point estimate, then there's a number of reasonable ways to do it.
And perhaps the most reasonable one is to just the report the conditional expectation itself.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to finish today our discussion of Bayesian Inference, which we started last time.
As you probably saw there's not a huge lot of concepts that we're introducing at this point in terms of specific skills of calculating probabilities.
But, rather, it's more of an interpretation and setting up the framework.
So the framework in Bayesian estimation is that there is some parameter which is not known, but we have a prior distribution on it.
These are beliefs about what this variable might be, and then we'll obtain some measurements.
And the measurements are affected by the value of that parameter that we don't know.
And this effect, the fact that X is affected by Theta, is captured by introducing a conditional probability distribution-- the distribution of X depends on Theta.
It's a conditional probability distribution.
So we have formulas for these two densities, the prior density and the conditional density.
And given that we have these, if we multiply them we can also get the joint density of X and Theta.
So we have everything that's there is to know in this second.
And now we observe the random variable X.
Given this random variable what can we say about Theta?
Well, what we can do is we can always calculate the conditional distribution of theta given X.
And now that we have the specific value of X we can plot this as a function of Theta.
OK. And this is the complete answer to a Bayesian Inference problem.
This posterior distribution captures everything there is to say about Theta, that's what we know about Theta.
Given the X that we have observed Theta is still random, it's still unknown.
And it might be here, there, or there with several probabilities.
On the other hand, if you want to report a single value for Theta then you do some extra work.
You continue from here, and you do some data processing on X.
Doing data processing means that you apply a certain function on the data, and this function is something that you design.
It's the so-called estimator.
And once that function is applied it outputs an estimate of Theta, which we call Theta hat.
So this is sort of the big picture of what's happening.
Now one thing to keep in mind is that even though I'm writing single letters here, in general Theta or X could be vector random variables.
So think of this-- it could be a collection Theta1, Theta2, Theta3.
And maybe we obtained several measurements, so this X is really a vector X1, X2, up to Xn.
All right, so now how do we choose a Theta to report?
There are various ways of doing it.
One is to look at the posterior distribution and report the value of Theta, at which the density or the PMF is highest.
This is called the maximum a posteriori estimate.
So we pick a value of theta for which the posteriori is maximum, and we report it.
An alternative way is to try to be optimal with respects to a mean squared error.
So what is this?
If we have a specific estimator, g, this is the estimate it's going to produce.
This is the true value of Theta, so this is our estimation error.
We look at the square of the estimation error, and look at the average value.
We would like this squared estimation error to be as small as possible.
How can we design our estimator g to make that error as small as possible?
It turns out that the answer is to produce, as an estimate, the conditional expectation of Theta given X.
So the conditional expectation is the best estimate that you could produce if your objective is to keep the mean squared error as small as possible.
So this statement here is a statement of what happens on the average over all Theta's and all X's that may happen in our experiment.
The conditional expectation as an estimator has an even stronger property.
Not only it's optimal on the average, but it's also optimal given that you have made a specific observation, no matter what you observe.
Let's say you observe the specific value for the random variable X.
After that point if you're asked to produce a best estimate Theta hat that minimizes this mean squared error, your best estimate would be the conditional expectation given the specific value that you have observed.
These two statements say almost the same thing, but this one is a bit stronger.
This one tells you no matter what specific X happens the conditional expectation is the best estimate.
This one tells you on the average, over all X's may happen, the conditional expectation is the best estimator.
Now this is really a consequence of this.
If the conditional expectation is best for any specific X, then it's the best one even when X is left random and you are averaging your error over all possible X's.
OK so now that we know what is the optimal way of producing an estimate let's do a simple example to see how things work out.
So we have started with an unknown random variable, Theta, which is uniformly distributed between 4 and 10.
And then we have an observation model that tells us that given the value of Theta, X is going to be a random variable that ranges between Theta - 1, and Theta + 1.
So think of X as a noisy measurement of Theta, plus some noise, which is between -1, and +1.
So really the model that we are using here is that X is equal to Theta plus U -- where U is uniform on -1, and +1.
one, and plus one.
So we have the true value of Theta, but X could be Theta - 1, or it could be all the way up to Theta + 1.
And the X is uniformly distributed on that interval.
That's the same as saying that U is uniformly distributed over this interval.
So now we have all the information that we need, we can construct the joint density.
And the joint density is, of course, the prior density times the conditional density.
We go both of these.
Both of these are constants, so the joint density is also going to be a constant.
1/6 times 1/2, this is one over 12.
But it is a constant, not everywhere.
Only on the range of possible x's and thetas.
So theta can take any value between four and ten, so these are the values of theta.
And for any given value of theta x can take values from theta minus one, up to theta plus one.
So here, if you can imagine, a line that goes with slope one, and then x can take that value of theta plus or minus one.
So this object here, this is the set of possible x and theta pairs.
So the density is equal to one over 12 over this set, and it's zero everywhere else.
So outside here the density is zero, the density only applies at that point.
All right, so now we're asked to estimate theta in terms of x.
So we want to build an estimator which is going to be a function from the x's to the thetas.
That's why I chose the axis this way-- x to be on this axis, theta on that axis-- Because the estimator we're building is a function of x.
Based on the observation that we obtained, we want to estimate theta.
So we know that the optimal estimator is the conditional expectation, given the value of x.
So what is the conditional expectation?
If you fix a particular value of x, let's say in this range.
So this is our x, then what do we know about theta?
We know that theta lies in this range.
Theta can only be sampled between those two values.
And what kind of distribution does theta have?
What is the conditional distribution of theta given x?
Well, remember how we built conditional distributions from joint distributions?
The conditional distribution is just a section of the joint distribution applied to the place where we're conditioning.
So the joint is constant.
So the conditional is also going to be a constant density over this interval.
So the posterior distribution of theta is uniform over this interval.
So if the posterior of theta is uniform over that interval, the expected value of theta is going to be the meet point of that interval.
So the estimate which you report-- if you observe that theta-- is going to be this particular point here, it's the midpoint.
The same argument goes through even if you obtain an x somewhere here.
Given this x, theta can take a value between these two values.
Theta is going to have a uniform distribution over this interval, and the conditional expectation of theta given x is going to be the midpoint of that interval.
So now if we plot our estimator by tracing midpoints in this diagram what you're going to obtain is a curve that starts like this, then it changes slope.
So that it keeps track of the midpoint, and then it goes like that again.
So this blue curve here is our g of x, which is the conditional expectation of theta given that x is equal to little x.
So it's a curve, in our example it consists of three straight segments.
But overall it's non-linear.
It's not a single line through this diagram.
And that's how things are in general.
g of x, our optimal estimate has no reason to be a linear function of x.
In general it's going to be some complicated curve.
So how good is our estimate?
I mean you reported your x, your estimate of theta based on x, and your boss asks you what kind of error do you expect to get?
Having observed the particular value of x, what you can report to your boss is what you think is the mean squared error is going to be.
We observe the particular value of x.
So we're conditioning, and we're living in this universe.
Given that we have made this observation, this is the true value of theta, this is the estimate that we have produced, this is the expected squared error, given that we have made the particular observation.
Now in this conditional universe this is the expected value of theta given x.
So this is the expected value of this random variable inside the conditional universe.
So when you take the mean squared of a random variable minus the expected value, this is the same thing as the variance of that random variable.
Except that it's the variance inside the conditional universe.
Having observed x, theta is still a random variable.
It's distributed according to the posterior distribution.
Since it's a random variable, it has a variance.
And that variance is our mean squared error.
So this is the variance of the posterior distribution of Theta given the observation that we have made.
OK, so what is the variance in our example?
If X happens to be here, then Theta is uniform over this interval, and this interval has length 2.
Theta is uniformly distributed over an interval of length 2.
This is the posterior distribution of Theta.
What is the variance?
Then you remember the formula for the variance of a uniform random variable, it is the length of the interval squared divided by 12, so this is 1/3.
So the variance of Theta -- the mean squared error-- is going to be 1/3 whenever this kind of picture applies.
This picture applies when X is between 5 and 9.
If X is less than 5, then the picture is a little different, and Theta is going to be uniform over a smaller interval.
And so the variance of theta is going to be smaller as well.
So let's start plotting our mean squared error.
Between 5 and 9 the variance of Theta -- the posterior variance-- is 1/3.
Now when the X falls in here Theta is uniformly distributed over a smaller interval.
The size of this interval changes linearly over that range.
And so when we take the square size of that interval we get a quadratic function of how much we have moved from that corner.
So at that corner what is the variance of Theta?
Well if I observe an X that's equal to 3 then I know with certainty that Theta is equal to 4.
Then I'm in very good shape, I know exactly what Theta is going to be.
So the variance, in this case, is going to be 0.
If I observe an X that's a little larger than Theta is now random, takes values on a little interval, and the variance of Theta is going to be proportional to the square of the length of that little interval.
So we get a curve that starts rising quadratically from here.
It goes up forward 1/3.
At the other end of the picture the same is true.
If you observe an X which is 11 then Theta can only be equal to 10.
And so the error in Theta is equal to 0, there's 0 error variance.
But as we obtain X's that are slightly less than 11 then the mean squared error again rises quadratically.
So we end up with a plot like this.
What this plot tells us is that certain measurements are better than others.
If you're lucky, and you see X equal to 3 then you're lucky, because you know Theta exactly what it is.
If you see an X which is equal to 6 then you're sort of unlikely, because it doesn't tell you Theta with great precision.
Theta could be anywhere on that interval.
And so the variance of Theta -- even after you have observed X -- is a certain number, 1/3 in our case.
So the moral to keep out of that story is that the error variance-- or the mean squared error-- depends on what particular observation you happen to obtain.
Some observations may be very informative, and once you see a specific number than you know exactly what Theta is.
Some observations might be less informative.
You observe your X, but it could still leave a lot of uncertainty about Theta.
So conditional expectations are really the cornerstone of Bayesian estimation.
They're particularly popular, especially in engineering contexts.
There used a lot in signal processing, communications, control theory, so on.
So that makes it worth playing a little bit with their theoretical properties, and get some appreciation of a few subtleties involved here.
No new math in reality, in what we're going to do here.
But it's going to be a good opportunity to practice manipulation of conditional expectations.
So let's look at the expected value of the estimation error that we obtained.
So Theta hat is our estimator, is the conditional expectation.
Theta hat minus Theta is what kind of error do we have?
If Theta hat, is bigger than Theta then we have made the positive error.
If not, if it's on the other side, we have made the negative error.
Then it turns out that on the average the errors cancel each other out, on the average.
So let's do this calculation.
Let's calculate the expected value of the error given X.
Now by definition the error is expected value of Theta hat minus Theta given X.
We use linearity of expectations to break it up as expected value of Theta hat given X minus expected value of Theta given X.
And now what?
Our estimate is made on the basis of the data of the X's.
If I tell you X then you know what Theta hat is.
Remember that the conditional expectation is a random variable which is a function of the random variable, on which you're conditioning on.
If you know X then you know the conditional expectation given X, you know what Theta hat is going to be.
So Theta hat is a function of X.
If it's a function of X then once I tell you X you know what Theta hat is going to be.
So this conditional expectation is going to be Theta hat itself.
Here this is-- just by definition-- Theta hat, and so we get equality to 0.
So what we have proved is that no matter what I have observed, and given that I have observed something on the average my error is going to be 0.
This is a statement involving equality of random variables.
Remember that conditional expectations are random variables because they depend on the thing you're conditioning on.
0 is sort of a trivial random variable.
This tells you that this random variable is identically equal to the 0 random variable.
More specifically it tells you that no matter what value for X you observe, the conditional expectation of the error is going to be 0.
And this takes us to this statement here, which is inequality between numbers.
No matter what specific value for capital X you have observed, your error, on the average, is going to be equal to 0.
So this is a less abstract version of these statements.
This is inequality between two numbers.
It's true for every value of X, so it's true in terms of these random variables being equal to that random variable.
Because remember according to our definition this random variable is the random variable that takes this specific value when capital X happens to be equal to little x.
Now this doesn't mean that your error is 0, it only means that your error is as likely, in some sense, to fall on the positive side, as to fall on the negative side.
So sometimes your error will be positive, sometimes negative.
And on the average these things cancel out and give you a 0 --.
on the average.
So this is a property that's sometimes giving the name we say that Theta hat is unbiased.
So Theta hat, our estimate, does not have a tendency to be on the high side.
It does not have a tendency to be on the low side.
On the average it's just right.
So let's do a little more playing here.
Let's see how our error is related to an arbitrary function of the data.
Let's do this in a conditional universe and look at this quantity.
In a conditional universe where X is known then h of X is known.
And so you can pull it outside the expectation.
In the conditional universe where the value of X is given this quantity becomes just a constant.
There's nothing random about it.
So you can pull it out, the expectation, and write things this way.
And we have just calculated that this quantity is 0.
So this number turns out to be 0 as well.
Now having done this, we can take expectations of both sides.
And now let's use the law of iterated expectations.
Expectation of a conditional expectation gives us the unconditional expectation, and this is also going to be 0.
So here we use the law of iterated expectations.
OK. OK, why are we doing this?
We're doing this because I would like to calculate the covariance between Theta tilde and Theta hat.
Theta hat is, ask the question -- is there a systematic relation between the error and the estimate?
So to calculate the covariance we use the property that we can calculate the covariances by calculating the expected value of the product minus the product of the expected values.
And what do we get?
This is 0, because of what we just proved.
And this is 0, because of what we proved earlier.
That the expected value of the error is equal to 0.
So the covariance between the error and any function of X is equal to 0.
Let's use that to the case where the function of X we're considering is Theta hat itself.
Theta hat is our estimate, it's a function of X.
So this 0 result would still apply, and we get that this covariance is equal to 0.
OK, so that's what we proved.
Let's see, what are the morals to take out of all this?
First is you should be very comfortable with this type of calculation involving conditional expectations.
The main two things that we're using are that when you condition on a random variable any function of that random variable becomes a constant, and can be pulled out the conditional expectation.
The other thing that we are using is the law of iterated expectations, so these are the skills involved.
Now on the substance, why is this result interesting?
This tells us that the error is uncorrelated with the estimate.
What's a hypothetical situation where these would not happen?
Whenever Theta hat is positive my error tends to be negative.
Suppose that whenever Theta hat is big then you say oh my estimate is too big, maybe the true Theta is on the lower side, so I expect my error to be negative.
That would be a situation that would violate this condition.
This condition tells you that no matter what Theta hat is, you don't expect your error to be on the positive side or on the negative side.
Your error will still be 0 on the average.
So if you obtain a very high estimate this is no reason for you to suspect that the true Theta is lower than your estimate.
If you suspected that the true Theta was lower than your estimate you should have changed your Theta hat.
If you make an estimate and after obtaining that estimate you say I think my estimate is too big, and so the error is negative.
If you thought that way then that means that your estimate is not the optimal one, that your estimate should have been corrected to be smaller.
And that would mean that there's a better estimate than the one you used, but the estimate that we are using here is the optimal one in terms of mean squared error, there's no way of improving it.
And this is really captured in that statement.
That is knowing Theta hat doesn't give you a lot of information about the error, and gives you, therefore, no reason to adjust your estimate from what it was.
Finally, a consequence of all this.
This is the definition of the error.
Send Theta to this side, send Theta tilde to that side, you get this relation.
The true parameter is composed of two quantities.
The estimate, and the error that they got with a minus sign.
These two quantities are uncorrelated with each other.
Their covariance is 0, and therefore, the variance of this is the sum of the variances of these two quantities.
So what's an interpretation of this equality?
There is some inherent randomness in the random variable theta that we're trying to estimate.
Theta hat tries to estimate it, tries to get close to it.
And if Theta hat always stays close to Theta, since Theta is random Theta hat must also be quite random, so it has uncertainty in it.
And the more uncertain Theta hat is the more it moves together with Theta.
So the more uncertainty it removes from Theta.
And this is the remaining uncertainty in Theta.
The uncertainty that's left after we've done our estimation.
So ideally, to have a small error we want this quantity to be small.
Which is the same as saying that this quantity should be big.
In the ideal case Theta hat is the same as Theta.
That's the best we could hope for.
That corresponds to 0 error, and all the uncertainly in Theta is absorbed by the uncertainty in Theta hat.
Interestingly, this relation here is just another variation of the law of total variance that we have seen at some point in the past.
I will skip that derivation, but it's an interesting fact, and it can give you an alternative interpretation of the law of total variance.
OK, so now let's return to our example.
In our example we obtained the optimal estimator, and we saw that it was a nonlinear curve, something like this.
I'm exaggerating the corner of a little bit to show that it's nonlinear.
This is the optimal estimator.
It's a nonlinear function of X -- nonlinear generally means complicated.
Sometimes the conditional expectation is really hard to compute, because whenever you have to compute expectations you need to do some integrals.
And if you have many random variables involved it might correspond to a multi-dimensional integration.
We don't like this.
Can we come up, maybe, with a simpler way of estimating Theta?
Of coming up with a point estimate which still has some nice properties, it has some good motivation, but is simpler.
What does simpler mean?
Perhaps linear.
Let's put ourselves in a straitjacket and restrict ourselves to estimators that's are of these forms.
My estimate is constrained to be a linear function of the X's.
So my estimator is going to be a curve, a linear curve.
It could be this, it could be that, maybe it would want to be something like this.
I want to choose the best possible linear function.
What does that mean?
It means that I write my Theta hat in this form.
If I fix a certain a and b I have fixed the functional form of my estimator, and this is the corresponding mean squared error.
That's the error between the true parameter and the estimate of that parameter, we take the square of this.
And now the optimal linear estimator is defined as one for which these mean squared error is smallest possible over all choices of a and b.
So we want to minimize this expression over all a's and b's.
How do we do this minimization?
Well this is a square, you can expand it.
Write down all the terms in the expansion of the square.
So you're going to get the term expected value of Theta squared.
You're going to get another term-- a squared expected value of X squared, another term which is b squared, and then you're going to get to various cross terms.
What you have here is really a quadratic function of a and b.
So think of this quantity that we're minimizing as some function h of a and b, and it happens to be quadratic.
How do we minimize a quadratic function?
We set the derivative of this function with respect to a and b to 0, and then do the algebra.
After you do the algebra you find that the best choice for a is this 1, so this is the coefficient next to X.
This is the optimal a.
And the optimal b corresponds of the constant terms.
So this term and this times that together are the optimal choices of b.
So the algebra itself is not very interesting.
What is really interesting is the nature of the result that we get here.
If we were to plot the result on this particular example you would get the curve that's something like this.
It goes through the middle of this diagram and is a little slanted.
In this example, X and Theta are positively correlated.
Bigger values of X generally correspond to bigger values of Theta.
So in this example the covariance between X and Theta is positive, and so our estimate can be interpreted in the following way: The expected value of Theta is the estimate that you would come up with if you didn't have any information about Theta.
If you don't make any observations this is the best way of estimating Theta.
But I have made an observation, X, and I need to take it into account.
I look at this difference, which is the piece of news contained in X?
That's what X should be on the average.
If I observe an X which is bigger than what I expected it to be, and since X and Theta are positively correlated, this tells me that Theta should also be bigger than its average value.
Whenever I see an X that's larger than its average value this gives me an indication that theta should also probably be larger than its average value.
And so I'm taking that difference and multiplying it by a positive coefficient.
And that's what gives me a curve here that has a positive slope.
So this increment-- the new information contained in X as compared to the average value we expected apriori, that increment allows us to make a correction to our prior estimate of Theta, and the amount of that correction is guided by the covariance of X with Theta.
If the covariance of X with Theta were 0, that would mean there's no systematic relation between the two, and in that case obtaining some information from X doesn't give us a guide as to how to change the estimates of Theta.
If that were 0, we would just stay with this particular estimate.
We're not able to make a correction.
But when there's a non zero covariance between X and Theta that covariance works as a guide for us to obtain a better estimate of Theta.
How about the resulting mean squared error?
In this context turns out that there's a very nice formula for the mean squared error obtained from the best linear estimate.
What's the story here?
The mean squared error that we have has something to do with the variance of the original random variable.
The more uncertain our original random variable is, the more error we're going to make.
On the other hand, when the two variables are correlated we explored that correlation to improve our estimate.
This row here is the correlation coefficient between the two random variables.
When this correlation coefficient is larger this factor here becomes smaller.
And our mean squared error become smaller.
So think of the two extreme cases.
One extreme case is when rho equal to 1 -- so X and Theta are perfectly correlated.
When they're perfectly correlated once I know X then I also know Theta.
And the two random variables are linearly related.
In that case, my estimate is right on the target, and the mean squared error is going to be 0.
The other extreme case is if rho is equal to 0.
The two random variables are uncorrelated.
In that case the measurement does not help me estimate Theta, and the uncertainty that's left-- the mean squared error-- is just the original variance of Theta.
So the uncertainty in Theta does not get reduced.
So moral-- the estimation error is a reduced version of the original amount of uncertainty in the random variable Theta, and the larger the correlation between those two random variables, the better we can remove uncertainty from the original random variable.
I didn't derive this formula, but it's just a matter of algebraic manipulations.
We have a formula for Theta hat, subtract Theta from that formula.
Take square, take expectations, and do a few lines of algebra that you can read in the text, and you end up with this really neat and clean formula.
Now I mentioned in the beginning of the lecture that we can do inference with Theta's and X's not just being single numbers, but they could be vector random variables.
So for example we might have multiple data that gives us information about X.
There are no vectors here, so this discussion was for the case where Theta and X were just scalar, one-dimensional quantities.
What do we do if we have multiple data?
Suppose that Theta is still a scalar, it's one dimensional, but we make several observations.
And on the basis of these observations we want to estimate Theta.
The optimal least mean squares estimator would be again the conditional expectation of Theta given X.
That's the optimal one.
And in this case X is a vector, so the general estimator we would use would be this one.
But if we want to keep things simple and we want our estimator to have a simple functional form we might restrict to estimator that are linear functions of the data.
And then the story is exactly the same as we discussed before.
I constrained myself to estimating Theta using a linear function of the data, so my signal processing box just applies a linear function.
And I'm looking for the best coefficients, the coefficients that are going to result in the least possible squared error.
This is my squared error, this is (my estimate minus the thing I'm trying to estimate) squared, and then taking the average.
How do we do this?
Same story as before.
The X's and the Theta's get averaged out because we have an expectation.
Whatever is left is just a function of the coefficients of the a's and of b's.
As before it turns out to be a quadratic function.
Then we set the derivatives of this function of a's and b's with respect to the coefficients, we set it to 0.
And this gives us a system of linear equations.
It's a system of linear equations that's satisfied by those coefficients.
It's a linear system because this is a quadratic function of those coefficients.
So to get closed-form formulas in this particular case one would need to introduce vectors, and matrices, and metrics inverses and so on.
The particular formulas are not so much what interests us here, rather, the interesting thing is that this is simply done just using straightforward solvers of linear equations.
The only thing you need to do is to write down the correct coefficients of those non-linear equations.
And the typical coefficient that you would get would be what?
Let say a typical quick equations would be -- let's take a typical term of this quadratic one you expanded.
You're going to get the terms such as a1x1 times a2x2.
When you take expectations you're left with a1a2 times expected value of x1x2.
So this would involve terms such as a1 squared expected value of x1 squared.
You would get terms such as a1a2, expected value of x1x2, and a lot of other terms here should have a too.
So you get something that's quadratic in your coefficients.
And the constants that show up in your system of equations are things that have to do with the expected values of squares of your random variables, or products of your random variables.
To write down the numerical values for these the only thing you need to know are the means and variances of your random variables.
If you know the mean and variance then you know what this thing is.
And if you know the covariances as well then you know what this thing is.
So in order to find the optimal linear estimator in the case of multiple data you do not need to know the entire probability distribution of the random variables that are involved.
You only need to know your means and covariances.
These are the only quantities that affect the construction of your optimal estimator.
We could see this already in this formula.
The form of my optimal estimator is completely determined once I know the means, variance, and covariance of the random variables in my model.
I do not need to know how the details distribution of the random variables that are involved here.
So as I said in general, you find the form of the optimal estimator by using a linear equation solver.
There are special examples in which you can get closed-form solutions.
The nicest simplest estimation problem one can think of is the following-- you have some uncertain parameter, and you make multiple measurements of that parameter in the presence of noise.
So the Wi's are noises.
I corresponds to your i-th experiment.
So this is the most common situation that you encounter in the lab.
If you are dealing with some process, you're trying to measure something you measure it over and over.
Each time your measurement has some random error.
And then you need to take all your measurements together and come up with a single estimate.
So the noises are assumed to be independent of each other, and also to be independent from the value of the true parameter.
Without loss of generality we can assume that the noises have 0 mean and they have some variances that we assume to be known.
Theta itself has a prior distribution with a certain mean and the certain variance.
So the form of the optimal linear estimator is really nice.
Well maybe you cannot see it right away because this looks messy, but what is it really?
It's a linear combination of the X's and the prior mean.
And it's actually a weighted average of the X's and the prior mean.
Here we collect all of the coefficients that we have at the top.
So the whole thing is basically a weighted average.
1/(sigma_i-squared) is the weight that we give to Xi, and in the denominator we have the sum of all of the weights.
So in the end we're dealing with a weighted average.
If mu was equal to 1, and all the Xi's were equal to 1 then our estimate would also be equal to 1.
Now the form of the weights that we have is interesting.
Any given data point is weighted inversely proportional to the variance.
What does that say?
If my i-th data point has a lot of variance, if Wi is very noisy then Xi is not very useful, is not very reliable.
So I'm giving it a small weight.
Large variance, a lot of error in my Xi means that I should give it a smaller weight.
If two data points have the same variance, they're of comparable quality, then I'm going to give them equal weight.
The other interesting thing is that the prior mean is treated the same way as the X's.
So it's treated as an additional observation.
So we're taking a weighted average of the prior mean and of the measurements that we are making.
The formula looks as if the prior mean was just another data point.
So that's the way of thinking about Bayesian estimation.
You have your real data points, the X's that you observe, you also had some prior information.
This plays a role similar to a data point.
Interesting note that if all random variables are normal in this model these optimal linear estimator happens to be also the conditional expectation.
That's the nice thing about normal random variables that conditional expectations turn out to be linear.
So the optimal estimate and the optimal linear estimate turn out to be the same.
And that gives us another interpretation of linear estimation.
Linear estimation is essentially the same as pretending that all random variables are normal.
So that's a side point.
Now I'd like to close with a comment.
You do your measurements and you estimate Theta on the basis of X.
Suppose that instead you have a measuring device that's measures X-cubed instead of measuring X, and you want to estimate Theta.
Are you going to get to different a estimate?
Well X and X-cubed contained the same information.
Telling you X is the same as telling you the value of X-cubed.
So the posterior distribution of Theta given X is the same as the posterior distribution of Theta given X-cubed.
And so the means of these posterior distributions are going to be the same.
So doing transformations through your data does not matter if you're doing optimal least squares estimation.
On the other hand, if you restrict yourself to doing linear estimation then using a linear function of X is not the same as using a linear function of X-cubed.
So this is a linear estimator, but where the data are the X-cube's, and we have a linear function of the data.
So this means that when you're using linear estimation you have some choices to make linear on what?
Sometimes you want to plot your data on a not ordinary scale and try to plot a line through them.
Sometimes you plot your data on a logarithmic scale, and try to plot a line through them.
Which scale is the appropriate one?
Here it would be a cubic scale.
And you have to think about your particular model to decide which version would be a more appropriate one.
Finally when we have multiple data sometimes these multiple data might contain the same information.
So X is one data point, X-squared is another data point, X-cubed is another data point.
The three of them contain the same information, but you can try to form a linear function of them.
And then you obtain a linear estimator that has a more general form as a function of X.
So if you want to estimate your Theta as a cubic function of X, for example, you can set up a linear estimation model of this particular form and find the optimal coefficients, the a's and the b's.
All right, so the last slide just gives you the big picture of what's happening in Bayesian Inference, it's for you to ponder.
Basically we talked about three possible estimation methods.
Maximum posteriori, mean squared error estimation, and linear mean squared error estimation, or least squares estimation.
And there's a number of standard examples that you will be seeing over and over in the recitations, tutorial, homework, and so on, perhaps on exams even.
Where we take some nice priors on some unknown parameter, we take some nice models for the noise or the observations, and then you need to work out posterior distributions in the various estimates and compare them.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So for the last three lectures we're going to talk about classical statistics, the way statistics can be done if you don't want to assume a prior distribution on the unknown parameters.
Today we're going to focus, mostly, on the estimation side and leave hypothesis testing for the next two lectures.
So where there is one generic method that one can use to carry out parameter estimation, that's the maximum likelihood method.
We're going to define what it is.
Then we will look at the most common estimation problem there is, which is to estimate the mean of a given distribution.
And we're going to talk about confidence intervals, which refers to providing an interval around your estimates, which has some properties of the kind that the parameter is highly likely to be inside that interval, but we will be careful about how to interpret that particular statement.
Ok.
So the big framework first.
The picture is almost the same as the one that we had in the case of Bayesian statistics.
We have some unknown parameter.
And we have a measuring device.
There is some noise, some randomness.
And we get an observation, X, whose distribution depends on the value of the parameter.
However, the big change from the Bayesian setting is that here, this parameter is just a number.
It's not modeled as a random variable.
It does not have a probability distribution.
There's nothing random about it.
It's a constant.
It just happens that we don't know what that constant is.
And in particular, this probability distribution here, the distribution of X, depends on Theta.
But this is not a conditional distribution in the usual sense of the word.
Conditional distributions were defined when we had two random variables and we condition one random variable on the other.
And we used the bar to separate the X from the Theta.
To make the point that this is not a conditioned distribution, we use a different notation.
We put a semicolon here.
And what this is meant to say is that X has a distribution.
That distribution has a certain parameter.
And we don't know what that parameter is.
So for example, this might be a normal distribution, with variance 1 but a mean Theta.
We don't know what Theta is.
And we want to estimate it.
Now once we have this setting, then your job is to design this box, the estimator.
The estimator is some data processing box that takes the measurements and produces an estimate of the unknown parameter.
Now the notation that's used here is as if X and Theta were one-dimensional quantities.
But actually, everything we say remains valid if you interpret X and Theta as vectors of parameters.
So for example, you may obtain several measurements, X1 up to 2Xn.
And there may be several unknown parameters in the background.
Once more, we do not have, and we do not want to assume, a prior distribution on Theta.
It's a constant.
And if you want to think mathematically about this situation, it's as if you have many different probabilistic models.
So a normal with this mean or a normal with that mean or a normal with that mean, these are alternative candidate probabilistic models.
And we want to try to make a decision about which one is the correct model.
In some cases, we have to choose just between a small number of models.
For example, you have a coin with an unknown bias.
The bias is either 1/2 or 3/4.
You're going to flip the coin a few times.
And you try to decide whether the true bias is this one or is that one.
So in this case, we have two specific, alternative probabilistic models from which we want to distinguish.
But sometimes things are a little more complicated.
For example, you have a coin.
And you have one hypothesis that my coin is unbiased.
And the other hypothesis is that my coin is biased.
And you do your experiments.
And you want to come up with a decision that decides whether this is true or this one is true.
In this case, we're not dealing with just two alternative probabilistic models.
This one is a specific model for the coin.
But this one actually corresponds to lots of possible, alternative coin models.
So this includes the model where Theta is 0.6, the model where Theta is 0.7, Theta is 0.8, and so on.
So we're trying to discriminate between one model and lots of alternative models.
How does one go about this?
Well, there's some systematic ways that one can approach problems of this kind.
And we will start talking about these next time.
So today, we're going to focus on estimation problems.
In estimation problems, theta is a quantity, which is a real number, a continuous parameter.
We're to design this box, so what we get out of this box is an estimate.
Now notice that this estimate here is a random variable.
Even though theta is deterministic, this is random, because it's a function of the data that we observe.
The data are random.
We're applying a function to the data to construct our estimate.
So, since it's a function of random variables, it's a random variable itself.
The distribution of Theta hat depends on the distribution of X.
The distribution of X is affected by Theta.
So in the end, the distribution of your estimate Theta hat will also be affected by whatever Theta happens to be.
Our general objective, when designing estimators, is that we want to get, in the end, an error, an estimation error, which is not too large.
But we'll have to make that specific.
Again, what exactly do we mean by that?
So how do we go about this problem?
One general approach is to pick a Theta, under which the data that we observe, that this is the X's, our most likely to have occurred.
So I observe X.
For any given Theta, I can calculate this quantity, which tells me, under this particular Theta, the X that you observed had this probability of occurring.
Under that Theta, the X that you observe had that probability of occurring.
You just choose that Theta, which makes the data that you observed most likely.
It's interesting to compare this maximum likelihood estimate with the estimates that you would have, if you were in a Bayesian setting, and you were using maximum approach theory probability estimation.
In the Bayesian setting, what we do is, given the data, we use the prior distribution on Theta.
And we calculate the posterior distribution of Theta given X.
Notice that this is sort of the opposite from what we have here.
This is the probability of X for a particular value of Theta, whereas this is the probability of Theta for a particular X.
So it's the opposite type of conditioning.
In the Bayesian setting, Theta is a random variable.
So we can talk about the probability distribution of Theta.
So how do these two compare, except for this syntactic difference that the order X's and Theta's are reversed?
Let's write down, in full detail, what this posterior distribution of Theta is.
By the Bayes rule, this conditional distribution is obtained from the prior, and the model of the measurement process that we have.
And we get to this expression.
So in Bayesian estimation, we want to find the most likely value of Theta.
And we need to maximize this quantity over all possible Theta's.
First thing to notice is that the denominator is a constant.
It does not involve Theta.
So when you maximize this quantity, you don't care about the denominator.
You just want to maximize the numerator.
Now, here, things start to look a little more similar.
And they would be exactly of the same kind, if that term here was absent, it the prior was absent.
The two are going to become the same if that prior was just a constant.
So if that prior is a constant, then maximum likelihood estimation takes exactly the same form as Bayesian maximum posterior probability estimation.
So you can give this particular interpretation of maximum likelihood estimation.
Maximum likelihood estimation is essentially what you have done, if you were in a Bayesian world, and you had assumed a prior on the Theta's that's uniform, all the Theta's being equally likely.
Okay.
So let's look at a simple example.
Suppose that the Xi's are independent, identically distributed random variables, with a certain parameter Theta.
So the distribution of each one of the Xi's is this particular term.
So Theta is one-dimensional.
It's a one-dimensional parameter.
But we have several data.
We write down the formula for the probability of a particular X vector, given a particular value of Theta.
But again, when I use the word, given, here it's not in the conditioning sense.
It's the value of the density for a particular choice of Theta.
Here, I wrote down, I defined maximum likelihood estimation in terms of PMFs.
That's what you would do if the X's were discrete random variables.
Here, the X's are continuous random variables, so instead of I'm using the PDF instead of the PMF.
So this a definition, here, generalizes to the case of continuous random variables.
And you use F's instead of X's, our usual recipe.
So the maximum likelihood estimate is defined.
Now, since the Xi's are independent, the joint density of all the X's together is the product of the individual densities.
So you look at this quantity.
This is the density or sort of probability of observing a particular sequence of X's.
And we ask the question, what's the value of Theta that makes the X's that we observe most likely?
So we want to carry out this maximization.
Now this maximization is just a calculational problem.
We're going to do this maximization by taking the logarithm of this expression.
Maximizing an expression is the same as maximizing the logarithm.
So the logarithm of this expression, the logarithm of a product is the sum of the logarithms.
You get contributions from this Theta term.
There's n of these, so we get an n log Theta.
And then we have the sum of the logarithms of these terms.
It gives us minus Theta.
And then the sum of the X's.
So we need to maximize this expression with respect to Theta.
The way to do this maximization is you take the derivative, with respect to Theta.
And you get n over Theta equals to the sum of the X's.
And then you solve for Theta.
And you find that the maximum likelihood estimate is this quantity.
Which sort of makes sense, because this is the reciprocal of the sample mean of X's.
Theta, in an exponential distribution, we know that it's 1 over (the mean of the exponential distribution).
So it looks like a reasonable estimate.
So in any case, this is the estimates that the maximum likelihood estimation procedure tells us that we should report.
This formula here, of course, tells you what to do if you have already observed specific numbers.
If you have observed specific numbers, then you observe this particular number as your estimate of Theta.
If you want to describe your estimation procedure more abstractly, what you have constructed is an estimator, which is a box that's takes in the random variables, capital X1 up to Capital Xn, and produces out your estimate, which is also a random variable.
Because it's a function of these random variables and is denoted by an upper case Theta, to indicate that this is now a random variable.
So this is an equality about numbers.
This is a description of the general procedure, which is an equality between two random variables.
And this gives you the more abstract view of what we're doing here.
All right.
So what can we tell about our estimate?
Is it good or is it bad?
So we should look at this particular random variable and talk about the statistical properties that it has.
What we would like is this random variable to be close to the true value of Theta, with high probability, no matter what Theta is, since we don't know what Theta is.
Let's make a little more specific the properties that we want.
So we cook up the estimator somehow.
So this estimator corresponds, again, to a box that takes data in, the capital X's, and produces an estimate Theta hat.
This estimate is random.
Sometimes it will be above the true value of Theta.
Sometimes it will be below.
Ideally, we would like it to not have a systematic error, on the positive side or the negative side.
So a reasonable wish to have, for a good estimator, is that, on the average, it gives you the correct value.
Now here, let's be a little more specific about what that expectation is.
This is an expectation, with respect to the probability distribution of Theta hat.
The probability distribution of Theta hat is affected by the probability distribution of the X's.
Because Theta hat is a function of the X's.
And the probability distribution of the X's is affected by the true value of Theta.
So depending on which one is the true value of Theta, this is going to be a different expectation.
So if you were to write this expectation out in more detail, it would look something like this.
You need to write down the probability distribution of Theta hat.
And this is going to be some function.
But this function depends on the true Theta, is affected by the true Theta.
And then you integrate this with respect to Theta hat.
What's the point here?
Again, Theta hat is a function of the X's.
So the density of Theta hat is affected by the density of the X's.
The density of the X's is affected by the true value of Theta.
So the distribution of Theta hat is affected by the value of Theta.
Another way to put it is, as I've mentioned a few minutes ago, in this business, it's as if we are considering different possible probabilistic models, one probabilistic model for each choice of Theta.
And we're trying to guess which one of these probabilistic models is the true one.
One way of emphasizing the fact that this expression depends on the true Theta is to put a little subscript here, expectation, under the particular value of the parameter Theta.
So depending on what value the true parameter Theta takes, this expectation will have a different value.
And what we would like is that no matter what the true value is, that our estimate will not have a bias on the positive or the negative sides.
So this is a property that's desirable.
Is it always going to be true?
Not necessarily, it depends on what estimator we construct.
Is it true for our exponential example?
Unfortunately not, the estimate that we have in the exponential example turns out to be biased.
And one extreme way of seeing this is to consider the case where our sample size is 1.
We're trying to estimate Theta.
And the estimator from the previous slide, in that case, is just 1/X1.
Now X1 has a fair amount of density in the vicinity of 0, which means that 1/X1 has significant probability of being very large.
And if you do the calculation, this ultimately makes the expected value of 1/X1 to be infinite.
Now infinity is definitely not the correct value.
So our estimate is biased upwards.
And it's actually biased a lot upwards.
So that's how things are.
Maximum likelihood estimates, in general, will be biased.
But under some conditions, they will turn out to be asymptotically unbiased.
That is, as you get more and more data, as your X vector is longer and longer, with independent data, the estimate that you're going to have, the expected value of your estimator is going to get closer and closer to the true value.
So you do have some nice asymptotic properties, but we're not going to prove anything like this.
Speaking of asymptotic properties, in general, what we would like to have is that, as you collect more and more data, you get the correct answer, in some sense.
And the sense that we're going to use here is the limiting sense of convergence in probability, since this is the only notion of convergence of random variables that we have in our hands.
This is similar to what we had in the pollster problem, for example.
If we had a bigger and bigger sample size, we could be more and more confident that the estimate that we obtained is close to the unknown true parameter of the distribution that we have.
So this is a desirable property.
If you have an infinitely large amount of data, you should be able to estimate an unknown parameter more or less exactly.
So this is it desirable property of estimators.
It turns out that maximum likelihood estimation, given independent data, does have this property, under mild conditions.
So maximum likelihood estimation, in this respect, is a good approach.
So let's see, do we have this consistency property in our exponential example?
In our exponential example, we used this quantity to estimate the unknown parameter Theta.
What properties does this quantity have as n goes to infinity?
Well this quantity is the reciprocal of that quantity up here, which is the sample mean.
We know from the weak law of large numbers, that the sample mean converges to the expectation.
So this property here comes from the weak law of large numbers.
In probability, this quantity converges to the expected value, which, for exponential distributions, is 1/Theta.
Now, if something converges to something, then the reciprocal of that should converge to the reciprocal of that.
That's a property that's certainly correct for numbers.
But you're not talking about convergence of numbers.
We're talking about convergence in probability, which is a more complicated notion.
Fortunately, it turns out that the same thing is true, when we deal with convergence in probability.
One can show, although we will not bother doing this, that indeed, the reciprocal of this, which is our estimate, converges in probability to the reciprocal of that.
And that reciprocal is the true parameter Theta.
So for this particular exponential example, we do have the desirable property, that as the number of data becomes larger and larger, the estimate that we have constructed will get closer and closer to the true parameter value.
And this is true no matter what Theta is.
No matter what the true parameter Theta is, we're going to get close to it as we collect more data.
Okay.
So these are two rough qualitative properties that would be nice to have.
If you want to get a little more quantitative, you can start looking at the mean squared error that your estimator gives.
Now, once more, the comment I was making up there applies.
Namely, that this expectation here is an expectation with respect to the probability distribution of Theta hat that corresponds to a particular value of little theta.
So fix a little theta.
Write down this expression.
Look at the probability distribution of Theta hat, under that little theta.
And do this calculation.
You're going to get some quantity that depends on the little theta.
And so all quantities in this equality here should be interpreted as quantities under that particular value of little theta.
So if you wanted to make this more explicit, you could start throwing little subscripts everywhere in those expressions.
And let's see what those expressions tell us.
The expected value squared of a random variable, we know that it's always equal to the variance of this random variable, plus the expectation of the random variable squared.
So the expectation value of that random variable, squared.
This equality here is just our familiar formula, that the expected value of X squared is the variance of X plus the expected value of X squared.
So we apply this formula to X equal to Theta hat minus Theta.
Now, remember that, in this classical setting, theta is just a constant.
We have fixed Theta.
We want to calculate the variance of this quantity, under that particular Theta.
When you add or subtract a constant to a random variable, the variance doesn't change.
This is the same as the variance of our estimator.
And what we've got here is the bias of our estimate.
It tells us, on the average, whether we fall above or below.
And we're taking the bias to be b squared.
If we have an unbiased estimator, the bias term will be 0.
So ideally we want Theta hat to be very close to Theta.
And since Theta is a constant, if that happens, the variance of Theta hat would be very small.
So Theta is a constant.
If Theta hat has a distribution that's concentrated just around own little theta, then Theta hat would have a small variance.
So this is one desire that have.
We're going to have a small variance.
But we also want to have a small bias at the same time.
So the general form of the mean squared error has two contributions.
One is the variance of our estimator.
The other is the bias.
And one usually wants to design an estimator that simultaneously keeps both of these terms small.
So here's an estimation method that would do very well with respect to this term, but badly with respect to that term.
So suppose that my distribution is, let's say, normal with an unknown mean Theta and variance 1.
And I use as my estimator something very dumb.
I always produce an estimate that says my estimate is 100.
So I'm just ignoring the data and report 100.
What does this do?
The variance of my estimator is 0.
There's no randomness in the estimate that I report.
But the bias is going to be pretty bad.
The bias is going to be Theta hat, which is 100 minus the true value of Theta.
And for some Theta's, my bias is going to be horrible.
If my true Theta happens to be 0, my bias squared is a huge term.
And I get a large error.
So what's the moral of this example?
There are ways of making that variance very small, but, in those cases, you pay a price in the bias.
So you want to do something a little more delicate, where you try to keep both terms small at the same time.
So these types of considerations become important when you start to try to design sophisticated estimators for more complicated problems.
But we will not do this in this class.
This belongs to further classes on statistics and inference.
For this class, for parameter estimation, we will basically stick to two very simple methods.
One is the maximum likelihood method we've just discussed.
And the other method is what you would do if you were still in high school and didn't know any probability.
You get data.
And these data come from some distribution with an unknown mean.
And you want to estimate that the unknown mean.
What would you do?
You would just take those data and average them out.
So let's make this a little more specific.
We have X's that come from a given distribution.
We know the general form of the distribution, perhaps.
We do know, perhaps, the variance of that distribution, or, perhaps, we don't know it.
But we do not know the mean.
And we want to estimate the mean of that distribution.
Now, we can write this situation.
We can represent it in a different form.
The Xi's are equal to Theta.
This is the mean.
Plus a 0 mean random variable, that you can think of as noise.
So this corresponds to the usual situation you would have in a lab, where you go and try to measure an unknown quantity.
You get lots of measurements.
But each time that you measure them, your measurements have some extra noise in there.
And you want to kind of get rid of that noise.
The way to try to get rid of the measurement noise is to collect lots of data and average them out.
This is the sample mean.
And this is a very, very reasonable way of trying to estimate the unknown mean of the X's.
So this is the sample mean.
It's a reasonable, plausible, in general, pretty good estimator of the unknown mean of a certain distribution.
We can apply this estimator without really knowing a lot about the distribution of the X's.
Actually, we don't need to know anything about the distribution.
We can still apply it, because the variance, for example, does not show up here.
We don't need to know the variance to calculate that quantity.
Does this estimator have good properties?
Yes, it does.
What's the expected value of the sample mean?
If the expectation of this, it's the expectation of this sum divided by n. The expected value for each one of the X's is Theta.
So the expected value of the sample mean is just Theta itself.
So our estimator is unbiased.
No matter what Theta is, our estimator does not have a systematic error in either direction.
Furthermore, the weak law of large numbers tells us that this quantity converges to the true parameter in probability.
So it's a consistent estimator.
This is good.
And if you want to calculate the mean squared error corresponding to this estimator.
Remember how we defined the mean squared error?
It's this quantity.
Then it's a calculation that we have done a fair number of times by now.
The mean squared error is the variance of the distribution of the X's divided by n. So as we get more and more data, the mean squared error goes down to 0.
In some examples, it turns out that the sample mean is also the same as the maximum likelihood estimate.
For example, if the X's are coming from a normal distribution, you can write down the likelihood, do the maximization with respect to Theta, you'll find that the maximum likelihood estimate is the same as the sample mean.
In other cases, the sample mean will be different from the maximum likelihood.
And then you have a choice about which one of the two you would use.
Probably, in most reasonable situations, you would just use the sample mean, because it's simple, easy to compute, and has nice properties.
All right.
So you go to your boss.
And you report and say, OK, I did all my experiments in the lab.
And the average value that I got is a certain number, 2.37.
So is that the informative to your boss?
Well your boss would like to know how much they can trust this number, 2.37.
Well, I know that the true value is not going to be exactly that.
But how close should it be?
So give me a range of what you think are possible values of Theta.
So the situation is like this.
So suppose that we observe X's that are coming from a certain distribution.
And we're trying to estimate the mean.
We get our data.
Maybe our data looks something like this.
You calculate the mean.
You find the sample mean.
So let's suppose that the sample mean is a number, for some reason take to be 2.37.
But you want to convey something to your boss about how spread out these data were.
So the boss asks you to give him or her some kind of interval on which Theta, the true parameter, might lie.
So the boss asked you for an interval.
So what you do is you end up reporting an interval.
And you somehow use the data that you have seen to construct this interval.
And you report to your boss also the endpoints of this interval.
Let's give names to these endpoints, Theta_n- and Theta_n+.
The ends here just play the role of keeping track of how many data we're using.
So what you report to your boss is this interval as well.
Are these Theta's here, the endpoints of the interval, lowercase or uppercase?
What should they be?
Well you construct these intervals after you see your data.
You take the data into account to construct your interval.
So these definitely should depend on the data.
And therefore they are random variables.
Same thing with your estimator, in general, it's going to be a random variable.
Although, when you go and report numbers to your boss, you give the specific realizations of the random variables, given the data that you got.
So instead of having just a single box that produces estimates.
So our previous picture was that you have your estimator that takes X's and produces Theta hats.
Now our box will also be producing Theta hats minus and Theta hats plus.
It's going to produce an interval as well.
The X's are random, therefore these quantities are random.
Once you go and do the experiment and obtain your data, then your data will be some lowercase x, specific numbers.
And then your estimates and estimator become also lower case.
What would we like this interval to do?
We would like it to be highly likely to contain the true value of the parameter.
So we might impose some specs of the following kind.
I pick a number, alpha.
Usually that alpha, think of it as a probability of a large error.
Typical value of alpha might be 0.05, in which case this number here is point 0.95.
And you're given specs that say something like this.
I would like, with probability at least 0.95, this to happen, which says that the true parameter lies inside the confidence interval.
Now let's try to interpret this statement.
Suppose that you did the experiment, and that you ended up reporting to your boss a confidence interval from 1.97 to 2.56.
That's what you report to your boss.
And suppose that the confidence interval has this property.
Can you go to your boss and say, with probability 95%, the true value of Theta is between these two numbers?
Is that a meaningful statement?
So the statement is, the tentative statement is, with probability 95%, the true value of Theta is between 1.97 and 2.56.
Well, what is random in that statement?
There's nothing random.
The true value of theta is a constant.
1.97 is a number.
2.56 is a number.
So it doesn't make any sense to talk about the probability that theta is in this interval.
Either theta happens to be in that interval, or it happens to not be.
But there are no probabilities associated with this.
Because theta is not random.
Syntactically, you can see this.
Because theta here is a lower case.
So what kind of probabilities are we talking about here?
Where's the randomness?
Well the random thing is the interval.
It's not theta.
So the statement that is being made here is that the interval, that's being constructed by our procedure, should have the property that, with probability 95%, it's going to fall on top of the true value of theta.
So the right way of interpreting what the 95% confidence interval is, is something like the following.
We have the true value of theta that we don't know.
I get data.
Based on the data, I construct a confidence interval.
I get my confidence interval.
I got lucky.
And the true value of theta is in here.
Next day, I do the same experiment, take my data, construct a confidence interval.
And I get this confidence interval, lucky once more.
Next day I get data.
I use my data to come up with an estimate of theta and the confidence interval.
That day, I was unlucky.
And I got a confidence interval out there.
What the requirement here is, is that 95% of the days, where we use this certain procedure for constructing confidence intervals, 95% of those days, we will be lucky.
And we will capture the correct value of theta by your confidence interval.
So it's a statement about the distribution of these random confidence intervals, how likely are they to fall on top of the true theta, as opposed to how likely they are to fall outside.
So it's a statement about probabilities associated with a confidence interval.
They're not probabilities about theta, because theta, itself, is not random.
So this is what the confidence interval is, in general, and how we interpret it.
How do we construct a 95% confidence interval?
Let's go through this exercise, in a particular example.
The calculations are exactly the same as the ones that you did when we talked about laws of large numbers and the central limit theorem.
So there's nothing new calculationally but it's, perhaps, new in terms of the language that we use and the interpretation.
So we got our sample mean from some distribution.
And we would like to calculate a 95% confidence interval.
We know from the normal tables, that the standard normal has 2.5% on the tail, that's after 1.96.
Yes, by this time, the number 1.96 should be pretty familiar.
So if this probability here is 2.5%, this number here is 1.96.
Now look at this random variable here.
This is the sample mean.
Difference, from the true mean, normalized by the usual normalizing factor.
By the central limit theorem, this is approximately normal.
So it has probability 0.95 of being less than 1.96.
Now take this event here and rewrite it.
This the event, well, that Theta hat minus theta is bigger than this number and smaller than that number.
This event here is equivalent to that event here.
And so this suggests a way of constructing our 95% percent confidence interval.
I'm going to report the interval, which gives this as the lower end of the confidence interval, and gives this as the upper end of the confidence interval In other words, at the end of the experiment, we report the sample mean, which is our estimate.
And we report also, an interval around the sample mean.
And this is our 95% confidence interval.
The confidence interval becomes smaller, when n is larger.
In some sense, we're more certain that we're doing a good estimation job, so we can have a small interval and still be quite confident that our interval captures the true value of the parameter.
Also, if our data have very little noise, when you have more accurate measurements, you're more confident that your estimate is pretty good.
And that results in a smaller confidence interval, smaller length of the confidence interval.
And still you have 95% probability of capturing the true value of theta.
So we did this exercise by taking 95% confidence intervals and the corresponding value from the normal tables, which is 1.96.
Of course, you can do it more generally, if you set your alpha to be some other number.
Again, you look at the normal tables.
And you find the value here, so that the tail has probability alpha over 2.
And instead of using these 1.96, you use whatever number you get from the normal tables.
And this tells you how to construct a confidence interval.
Well, to be exact, this is not necessarily a 95% confidence interval.
It's approximately a 95% confidence interval.
Why is this?
Because we've done an approximation.
We have used the central limit theorem.
So it might turn out to be a 95.5% confidence interval instead of 95%, because our calculations are not entirely accurate.
But for reasonable values of n, using the central limit theorem is a good approximation.
And that's what people almost always do.
So just take the value from the normal tables.
Okay, except for one catch.
I used the data.
I obtained my estimate.
And I want to go to my boss and report this theta minus and theta hat, which is the confidence interval.
What's the difficulty?
I know what n is.
But I don't know what sigma is, in general.
So if I don't know sigma, what am I going to do?
Here, there's a few options for what you can do.
And the first option is familiar from what we did when we talked about the pollster problem.
We don't know what sigma is, but maybe we have an upper bound on sigma.
For example, if the Xi's Bernoulli random variables, we have seen that the standard deviation is at most 1/2.
So use the most conservative value for sigma.
Using the most conservative value means that you take bigger confidence intervals than necessary.
So that's one option.
Another option is to try to estimate sigma from the data.
How do you do this estimation?
In special cases, for special types of distributions, you can think of heuristic ways of doing this estimation.
For example, in the case of Bernoulli random variables, we know that the true value of sigma, the standard deviation of a Bernoulli random variable, is the square root of theta1 minus theta, where theta is the mean of the Bernoulli.
Try to use this formula.
But theta is the thing we're trying to estimate in the first place.
We don't know it.
What do we do?
Well, we have an estimate for theta, the estimate, produced by our estimation procedure, the sample mean.
So I obtain my data.
I get my data.
I produce the estimate theta hat.
It's an estimate of the mean.
Use that estimate in this formula to come up with an estimate of my standard deviation.
And then use that standard deviation, in the construction of the confidence interval, pretending that this is correct.
Well the number of your data is large, then we know, from the law of large numbers, that theta hat is a pretty good estimate of theta.
So sigma hat is going to be a pretty good estimate of sigma.
So we're not making large errors by using this approach.
So in this scenario here, things were simple, because we had an analytical formula.
Sigma was determined by theta.
So we could come up with a quick and dirty estimate of sigma.
In general, if you do not have any nice formulas of this kind, what could you do?
Well, you still need to come up with an estimate of sigma somehow.
What is a generic method for estimating a standard deviation?
Equivalently, what could be a generic method for estimating a variance?
Well the variance is an expected value of some random variable.
The variance is the mean of the random variable inside of those brackets.
How does one estimate the mean of some random variable?
You obtain lots of measurements of that random variable and average them out.
So this would be a reasonable way of estimating the variance of a distribution.
And again, the weak law of large numbers tells us that this average converges to the expected value of this, which is just the variance of the distribution.
So we got a nice and consistent way of estimating variances.
But now, we seem to be getting in a vicious circle here, because to estimate the variance, we need to know the mean.
And the mean is something we're trying to estimate in the first place.
Okay.
But we do have an estimate from the mean.
So a reasonable approximation, once more, is to plug-in, here, since we don't know the mean, the estimate of the mean.
And so you get that expression, but with a theta hat instead of theta itself.
And this is another reasonable way of estimating the variance.
It does have the same consistency properties.
Why?
When n is large, this is going to behave the same as that, because theta hat converges to theta.
And when n is large, this is approximately the same as sigma squared.
So for a large n, this quantity also converges to sigma squared.
And we have a consistent estimate of the variance as well.
And we can take that consistent estimate and use it back in the construction of confidence interval.
One little detail, here, we're dividing by n. Here, we're dividing by n-1.
Why do we do this?
Well, it turns out that's what you need to do for these estimates to be an unbiased estimate of the variance.
One has to do a little bit of a calculation, and one finds that that's the factor that you need to have here in order to be unbiased.
Of course, if you get 100 data points, whether you divide by 100 or divided by 99, it's going to make only a tiny difference in your estimate of your variance.
So it's going to make only a tiny difference in your estimate of the standard deviation.
It's not a big deal.
And it doesn't really matter.
But if you want to show off about your deeper knowledge of statistics, you throw in the 1 over n-1 factor in there.
So now one basically needs to put together this story here, how you estimate the variance.
You first estimate the sample mean.
And then you do some extra work to come up with a reasonable estimate of the variance and the standard deviation.
And then you use your estimate, of the standard deviation, to come up with a confidence interval, which has these two endpoints.
In doing this procedure, there's basically a number of approximations that are involved.
There are two types of approximations.
One approximation is that we're pretending that the sample mean has a normal distribution.
That's something we're justified to do, by the central limit theorem.
But it's not exact.
It's an approximation.
And the second approximation that comes in is that, instead of using the correct standard deviation, in general, you will have to use some approximation of the standard deviation.
Okay so you will be getting a little bit of practice with these concepts in recitation and tutorial.
And we will move on to new topics next week.
But the material that's going to be covered in the final exam is only up to this point.
So next week is just general education.
Hopefully useful, but it's not in the exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And we're going to continue today with our discussion of classical statistics.
We'll start with a quick review of what we discussed last time, and then talk about two topics that cover a lot of statistics that are happening in the real world.
So two basic methods.
One is the method of linear regression, and the other one is the basic methods and tools for how to do hypothesis testing.
OK, so these two are topics that any scientifically literate person should know something about.
So we're going to introduce the basic ideas and concepts involved.
So in classical statistics we basically have essentially a family of possible models about the world.
So the world is the random variable that we observe, and we have a model for it, but actually not just one model, several candidate models.
And each candidate model corresponds to a different value of a parameter theta that we do not know.
So in contrast to Bayesian statistics, this theta is assumed to be a constant that we do not know.
It is not modeled as a random variable, there's no probabilities associated with theta.
We only have probabilities about the X's.
So in this context what is a reasonable way of choosing a value for the parameter?
One general approach is the maximum likelihood approach, which chooses the theta for which this quantity is largest.
So what does that mean intuitively?
I'm trying to find the value of theta under which the data that I observe are most likely to have occurred.
So is the thinking is essentially as follows.
Let's say I have to choose between two choices of theta.
Under this theta the X that I observed would be very unlikely.
Under that theta the X that I observed would have a decent probability of occurring.
So I chose the latter as my estimate of theta.
It's interesting to do the comparison with the Bayesian approach which we did discuss last time, in the Bayesian approach we also maximize over theta, but we maximize a quantity in which the relation between X's and thetas run the opposite way.
Here in the Bayesian world, Theta is a random variable.
So it has a distribution.
Once we observe the data, it has a posterior distribution, and we find the value of Theta, which is most likely under the posterior distribution.
As we discussed last time when you do this maximization now the posterior distribution is given by this expression.
The denominator doesn't matter, and if you were to take a prior, which is flat-- that is a constant independent of Theta, then that term would go away.
And syntactically, at least, the two approaches look the same.
So syntactically, or formally, maximum likelihood estimation is the same as Bayesian estimation in which you assume a prior which is flat, so that all possible values of Theta are equally likely.
Philosophically, however, they're very different things.
Here I'm picking the most likely value of Theta.
Here I'm picking the value of Theta under which the observed data would have been more likely to occur.
So maximum likelihood estimation is a general purpose method, so it's applied all over the place in many, many different types of estimation problems.
There is a special kind of estimation problem in which you may forget about maximum likelihood estimation, and come up with an estimate in a straightforward way.
And this is the case where you're trying to estimate the mean of the distribution of X, where X is a random variable.
You observe several independent identically distributed random variables X1 up to Xn.
All of them have the same distribution as this X.
So they have a common mean.
We do not know the mean we want to estimate it.
What is more natural than just taking the average of the values that we have observed?
So you generate lots of X's, take the average of them, and you expect that this is going to be a reasonable estimate of the true mean of that random variable.
And indeed we know from the weak law of large numbers that this estimate converges in probability to the true mean of the random variable.
The other thing that we talked about last time is that besides giving a point estimate we may want to also give an interval that tells us something about where we might believe theta to lie.
And 1-alpha confidence interval is in interval generated based on the data.
So it's an interval from this value to that value.
These values are written with capital letters because they're random, because they depend on the data that we have seen.
And this gives us an interval, and we would like this interval to have the property that theta is inside that interval with high probability.
So typically we would take 1-alpha to be a quantity such as 95% for example.
In which case we have a 95% confidence interval.
As we discussed last time it's important to have the right interpretation of what's 95% means.
What it does not mean is the following-- the unknown value has 95% percent probability of being in the interval that we have generated.
That's because the unknown value is not a random variable, it's a constant.
Once we generate the interval either it's inside or it's outside, but there's no probabilities involved.
Rather the probabilities are to be interpreted over the random interval itself.
What a statement like this says is that if I have a procedure for generating 95% confidence intervals, then whenever I use that procedure I'm going to get a random interval, and it's going to have 95% probability of capturing the true value of theta.
So most of the time when I use this particular procedure for generating confidence intervals the true theta will happen to lie inside that confidence interval with probability 95%.
So the randomness in this statement is with respect to my confidence interval, it's not with respect to theta, because theta is not random.
How does one construct confidence intervals?
There's various ways of going about it, but in the case where we're dealing with the estimation of the mean of a random variable doing this is straightforward using the central limit theorem.
Basically we take our estimated mean, that's the sample mean, and we take a symmetric interval to the left and to the right of the sample mean.
And we choose the width of that interval by looking at the normal tables.
So if this quantity, 1-alpha is 95% percent, we're going to look at the 97.5 percentile of the normal distribution.
Find the constant number that corresponds to that value from the normal tables, and construct the confidence intervals according to this formula.
So that gives you a pretty mechanical way of going about constructing confidence intervals when you're estimating the sample mean.
So constructing confidence intervals in this way involves an approximation.
The approximation is the central limit theorem.
We are pretending that the sample mean is a normal random variable.
Which is, more or less, right when n is large.
That's what the central limit theorem tells us.
And sometimes we may need to do some extra approximation work, because quite often we do not know the true value of sigma.
So we need to do some work either to estimate sigma from the data.
So sigma is, of course, the standard deviation of the X's.
We may want to estimate it from the data, or we may have an upper bound on sigma, and we just use that upper bound.
So now let's move on to a new topic.
A lot of statistics in the real world are of the following flavor.
So suppose that X is the SAT score of a student in high school, and Y is the MIT GPA of that same student.
So you expect that there is a relation between these two.
So you go and collect data for different students, and you record for a typical student this would be their SAT score, that could be their MIT GPA.
And you plot all this data on an (X,Y) diagram.
Now it's reasonable to believe that there is some systematic relation between the two.
So people who had higher SAT scores in high school may have higher GPA in college.
Well that may or may not be true.
You want to construct a model of this kind, and see to what extent a relation of this type is true.
So you might hypothesize that the real world is described by a model of this kind.
That there is a linear relation between the SAT score, and the college GPA.
So it's a linear relation with some parameters, theta0 and theta1 that we do not know.
So we assume a linear relation for the data, and depending on the choices of theta0 and theta1 it could be a different line through those data.
Now we would like to find the best model of this kind to explain the data.
Of course there's going to be some randomness.
So in general it's going to be impossible to find a line that goes through all of the data points.
So let's try to find the best line that comes closest to explaining those data.
And here's how we go about it.
Suppose we try some particular values of theta0 and theta1.
These give us a certain line.
Given that line, we can make predictions.
For a student who had this x, the model that we have would predict that y would be this value.
The actual y is something else, and so this quantity is the error that our model would make in predicting the y of that particular student.
We would like to choose a line for which the predictions are as good as possible.
And what do we mean by as good as possible?
As our criteria we're going to take the following.
We are going to look at the prediction error that our model makes for each particular student.
Take the square of that, and then add them up over all of our data points.
So what we're looking at is the sum of this quantity squared, that quantity squared, that quantity squared, and so on.
We add all of these squares, and we would like to find the line for which the sum of these squared prediction errors are as small as possible.
So that's the procedure.
We have our data, the X's and the Y's.
And we're going to find theta's the best model of this type, the best possible model, by minimizing this sum of squared errors.
So that's a method that one could pull out of the hat and say OK, that's how I'm going to build my model.
And it sounds pretty reasonable.
And it sounds pretty reasonable even if you don't know anything about probability.
But does it have some probabilistic justification?
It turns out that yes, you can motivate this method with probabilistic considerations under certain assumptions.
So let's make a probabilistic model that's going to lead us to these particular way of estimating the parameters.
So here's a probabilistic model.
I pick a student who had a specific SAT score.
And that could be done at random, but also could be done in a systematic way.
That is, I pick a student who had an SAT of 600, a student of 610 all the way to 1,400 or 1,600, whatever the right number is.
I pick all those students.
And I assume that for a student of this kind there's a true model that tells me that their GPA is going to be a random variable, which is something predicted by their SAT score plus some randomness, some random noise.
And I model that random noise by independent normal random variables with 0 mean and a certain variance.
So this is a specific probabilistic model, and now I can think about doing maximum likelihood estimation for this particular model.
So to do maximum likelihood estimation here I need to write down the likelihood of the y's that I have observed.
What's the likelihood of the y's that I have observed?
Well, a particular w has a likelihood of the form e to the minus w squared over (2 sigma-squared).
That's the likelihood of a particular w. The probability, or the likelihood of observing a particular value of y, that's the same as the likelihood that w takes a value of y minus this, minus that.
So the likelihood of the y's is of this form.
Think of this as just being the w_i-squared.
So this is the density -- and if we have multiple data you multiply the likelihoods of the different y's.
So you have to write something like this.
Since the w's are independent that means that the y's are also independent.
The likelihood of a y vector is the product of the likelihoods of the individual y's.
The likelihood of every individual y is of this form.
Where w is y_i minus these two quantities.
So this is the form that the likelihood function is going to take under this particular model.
And under the maximum likelihood methodology we want to maximize this quantity with respect to theta0 and theta1.
Now to do this maximization you might as well consider the logarithm and maximize the logarithm, which is just the exponent up here.
Maximizing this exponent because we have a minus sign is the same as minimizing the exponent without the minus sign.
Sigma squared is a constant.
So what you end up doing is minimizing this quantity here, which is the same as what we had in our linear regression methods.
So in conclusion you might choose to do linear regression in this particular way, just because it looks reasonable or plausible.
Or you might interpret what you're doing as maximum likelihood estimation, in which you assume a model of this kind where the noise terms are normal random variables with the same distribution -- independent identically distributed.
So linear regression implicitly makes an assumption of this kind.
It's doing maximum likelihood estimation as if the world was really described by a model of this form, and with the W's being random variables.
So this gives us at least some justification that this particular approach to fitting lines to data is not so arbitrary, but it has a sound footing.
OK so then once you accept this formulation as being a reasonable one what's the next step?
The next step is to see how to carry out this minimization.
This is not a very difficult minimization to do.
The way it's done is by setting the derivatives of this expression to 0.
Now because this is a quadratic function of theta0 and theta1-- when you take the derivatives with respect to theta0 and theta1-- you get linear functions of theta0 and theta1.
And you end up solving a system of linear equations in theta0 and theta1.
And it turns out that there's very nice and simple formulas for the optimal estimates of the parameters in terms of the data.
And the formulas are these ones.
I said that these are nice and simple formulas.
Let's see why.
How can we interpret them?
So suppose that the world is described by a model of this kind, where the X's and Y's are random variables.
And where W is a noise term that's independent of X.
So we're assuming that a linear model is indeed true, but not exactly true.
There's always some noise associated with any particular data point that we obtain.
So if a model of this kind is true, and the W's have 0 mean then we have that the expected value of Y would be theta0 plus theta1 expected value of X.
And because W has 0 mean there's no extra term.
So in particular, theta0 would be equal to expected value of Y minus theta1 expected value of X.
So let's use this equation to try to come up with a reasonable estimate of theta0.
I do not know the expected value of Y, but I can estimate it.
How do I estimate it?
I look at the average of all the y's that I have obtained.
so I replace this, I estimate it with the average of the data I have seen.
Here, similarly with the X's.
I might not know the expected value of X's, but I have data points for the x's.
I look at the average of all my data points, I come up with an estimate of this expectation.
Now I don't know what theta1 is, but my procedure is going to generate an estimate of theta1 called theta1 hat.
And once I have this estimate, then a reasonable person would estimate theta0 in this particular way.
So that's how my estimate of theta0 is going to be constructed.
It's this formula here.
We have not yet addressed the harder question, which is how to estimate theta1 in the first place.
So to estimate theta0 I assumed that I already had an estimate for a theta1.
OK, the right formula for the estimate of theta1 happens to be this one.
It looks messy, but let's try to interpret it.
What I'm going to do is I'm going to take this model for simplicity let's assume that they're the random variables have 0 means.
And see how we might estimate how we might try to estimate theta1.
Let's multiply both sides of this equation by X.
So we get Y times X equals theta0 plus theta0 times X plus theta1 times X-squared, plus X times W. And now take expectations of both sides.
If I have 0 mean random variables the expected value of Y times X is just the covariance of X with Y. I have assumed that my random variables have 0 means, so the expectation of this is 0.
This one is going to be the variance of X, so I have theta1 times variance of X.
And since I'm assuming that my random variables have 0 mean, and I'm also assuming that W is independent of X this last term also has 0 mean.
So under such a probabilistic model this equation is true.
If we knew the variance and the covariance then we would know the value of theta1.
But we only have data, we do not necessarily know the variance and the covariance, but we can estimate it.
What's a reasonable estimate of the variance?
The reasonable estimate of the variance is this quantity here divided by n, and the reasonable estimate of the covariance is that numerator divided by n. So this is my estimate of the mean.
I'm looking at the squared distances from the mean, and I average them over lots and lots of data.
This is the most reasonable way of estimating the variance of our distribution.
And similarly the expected value of this quantity is the covariance of X with Y, and then we have lots and lots of data points.
This quantity here is going to be a very good estimate of the covariance.
So basically what this formula does is-- one way of thinking about it-- is that it starts from this relation which is true exactly, but estimates the covariance and the variance on the basis of the data, and then using these estimates to come up with an estimate of theta1.
So this gives us a probabilistic interpretation of the formulas that we have for the way that the estimates are constructed.
If you're willing to assume that this is the true model of the world, the structure of the true model of the world, except that you do not know means and covariances, and variances.
Then this is a natural way of estimating those unknown parameters.
All right, so we have a closed-form formula, we can apply it whenever we have data.
Now linear regression is a subject on which there are whole courses, and whole books that are given.
And the reason for that is that there's a lot more that you can bring into the topic, and many ways that you can elaborate on the simple solution that we got for the case of two parameters and only two random variables.
So let me give you a little bit of flavor of what are the topics that come up when you start looking into linear regression in more depth.
So in our discussions so far we made the linear model in which we're trying to explain the values of one variable in terms of the values of another variable.
We're trying to explain GPAs in terms of SAT scores, or we're trying to predict GPAs in terms of SAT scores.
But maybe your GPA is affected by several factors.
For example maybe your GPA is affected by your SAT score, also the income of your family, the years of education of your grandmother, and many other factors like that.
So you might write down a model in which I believe that GPA has a relation, which is a linear function of all these other variables that I mentioned.
So perhaps you have a theory of what determines performance at college, and you want to build a model of that type.
How do we go about in this case?
Well, again we collect the data points.
We look at the i-th student, who has a college GPA.
We record their SAT score, their family income, and grandmother's years of education.
So this is one data point that is for one particular student.
We postulate the model of this form.
For the i-th student this would be the mistake that our model makes if we have chosen specific values for those parameters.
And then we go and choose the parameters that are going to give us, again, the smallest possible sum of squared errors.
So philosophically it's exactly the same as what we were discussing before, except that now we're including multiple explanatory variables in our model instead of a single explanatory variable.
So that's the formulation.
What do you do next?
Well, to do this minimization you're going to take derivatives once you have your data, you have a function of these three parameters.
You take the derivative with respect to the parameter, set the derivative equal to 0, you get the system of linear equations.
You throw that system of linear equations to the computer, and you get numerical values for the optimal parameters.
There are no nice closed-form formulas of the type that we had in the previous slide when you're dealing with multiple variables.
Unless you're willing to go into matrix notation.
In that case you can again write down closed-form formulas, but they will be a little less intuitive than what we had before.
But the moral of the story is that numerically this is a procedure that's very easy.
It's a problem, an optimization problem that the computer can solve for you.
And it can solve it for you very quickly.
Because all that it involves is solving a system of linear equations.
Now when you choose your explanatory variables you may have some choices.
One person may think that your GPA a has something to do with your SAT score.
Some other person may think that your GPA has something to do with the square of your SAT score.
And that other person may want to try to build a model of this kind.
Now when would you want to do this?
?
Suppose that the data that you have looks like this.
If the data looks like this then you might be tempted to say well a linear model does not look right, but maybe a quadratic model will give me a better fit for the data.
So if you want to fit a quadratic model to the data then what you do is you take X-squared as your explanatory variable instead of X, and you build a model of this kind.
There's nothing really different in models of this kind compared to models of that kind.
They are still linear models because we have theta's showing up in a linear fashion.
What you take as your explanatory variables, whether it's X, whether it's X-squared, or whether it's some other function that you chose.
Some general function h of X, doesn't make a difference.
So think of you h of X as being your new X.
So you can formulate the problem exactly the same way, except that instead of using X's you choose h of X's.
So it's basically a question do I want to build a model that explains Y's based on the values of X, or do I want to build a model that explains Y's on the basis of the values of h of X.
Which is the right value to use?
And with this picture here, we see that it can make a difference.
A linear model in X might be a poor fit, but a quadratic model might give us a better fit.
So this brings to the topic of how to choose your functions h of X if you're dealing with a real world problem.
So in a real world problem you're just given X's and Y's.
And you have the freedom of building models of any kind you want.
You have the freedom of choosing a function h of X of any type that you want.
So this turns out to be a quite difficult and tricky topic.
Because you may be tempted to overdo it.
For example, I got my 10 data points, and I could say OK, I'm going to choose an h of X. I'm going to choose h of X and actually multiple h's of X to do a multiple linear regression in which I'm going to build a model that's uses a 10th degree polynomial.
If I choose to fit my data with a 10th degree polynomial I'm going to fit my data perfectly, but I may obtain a model is does something like this, and goes through all my data points.
So I can make my prediction errors extremely small if I use lots of parameters, and if I choose my h functions appropriately.
But clearly this would be garbage.
If you get those data points, and you say here's my model that explains them.
That has a polynomial going up and down, then you're probably doing something wrong.
So choosing how complicated those functions, the h's, should be.
And how many explanatory variables to use is a very delicate and deep topic on which there's deep theory that tells you what you should do, and what you shouldn't do.
But the main thing that one should avoid doing is having too many parameters in your model when you have too few data.
So if you only have 10 data points, you shouldn't have 10 free parameters.
With 10 free parameters you will be able to fit your data perfectly, but you wouldn't be able to really rely on the results that you are seeing.
OK, now in practice, when people run linear regressions they do not just give point estimates for the parameters theta.
But similar to what we did for the case of estimating the mean of a random variable you might want to give confidence intervals that sort of tell you how much randomness there is when you estimate each one of the particular parameters.
There are formulas for building confidence intervals for the estimates of the theta's.
We're not going to look at them, it would take too much time.
Also you might want to estimate the variance in the noise that you have in your model.
That is if you are pretending that your true model is of the kind we were discussing before, namely Y equals theta1 times X plus W, and W has a variance sigma squared.
You might want to estimate this, because it tells you something about the model, and this is called standard error.
It puts a limit on how good predictions your model can make.
Even if you have the correct theta0 and theta1, and somebody tells you X you can make a prediction about Y, but that prediction will not be accurate.
Because there's this additional randomness.
And if that additional randomness is big, then your predictions will also have a substantial error in them.
There's another quantity that gets reported usually.
This is part of the computer output that you get when you use a statistical package which is called R-square.
And its a measure of the explanatory power of the model that you have built linear regression.
Using linear regression.
Instead of defining R-square exactly, let me give you a sort of analogous quantity that's involved.
After you do your linear regression you can look at the following quantity.
You look at the variance of Y, which is something that you can estimate from data.
This is how much randomness there is in Y.
And compare it with the randomness that you have in Y, but conditioned on X.
So this quantity tells me if I knew X how much randomness would there still be in my Y?
So if I know X, I have more information, so Y is more constrained.
There's less randomness in Y.
This is the randomness in Y if I don't know anything about X.
So naturally this quantity would be less than 1, and if this quantity is small it would mean that whenever I know X then Y is very well known.
Which essentially tells me that knowing x allows me to make very good predictions about Y.
Knowing X means that I'm explaining away most of the randomness in Y.
So if you read a statistical study that uses linear regression you might encounter statements of the form 60% of a student's GPA is explained by the family income.
If you read the statements of this kind it's really refers to quantities of this kind.
Out of the total variance in Y, how much variance is left after we build our model?
So if only 40% of the variance of Y is left after we build our model, that means that X explains 60% of the variations in Y's.
So the idea is that randomness in Y is caused by multiple sources.
Our explanatory variable and random noise.
And we ask the question what percentage of the total randomness in Y is explained by variations in the X parameter?
And how much of the total randomness in Y is attributed just to random effects?
So if you have a model that explains most of the variation in Y then you can think that you have a good model that tells you something useful about the real world.
Now there's lots of things that can go wrong when you use linear regression, and there's many pitfalls.
One pitfall happens when you have this situation that's called heteroskedacisity.
So suppose your data are of this kind.
So what's happening here?
You seem to have a linear model, but when X is small you have a very good model.
So this means that W has a small variance when X is here.
On the other hand, when X is there you have a lot of randomness.
This would be a situation in which the W's are not identically distributed, but the variance of the W's, of the noise, has something to do with the X's.
So with different regions of our x-space we have different amounts of noise.
What will go wrong in this situation?
Since we're trying to minimize sum of squared errors, we're really paying attention to the biggest errors.
Which will mean that we are going to pay attention to these data points, because that's where the big errors are going to be.
So the linear regression formulas will end up building a model based on these data, which are the most noisy ones.
Instead of those data that are nicely stacked in order.
Clearly that's not to the right thing to do.
So you need to change something, and use the fact that the variance of W changes with the X's, and there are ways of dealing with it.
It's something that one needs to be careful about.
Another possibility of getting into trouble is if you're using multiple explanatory variables that are very closely related to each other.
So for example, suppose that I tried to predict your GPA by looking at your SAT the first time that you took it plus your SAT the second time that you took your SATs.
I'm assuming that almost everyone takes the SAT more than once.
So suppose that you had a model of this kind.
Well, SAT on your first try and SAT on your second try are very likely to be fairly close.
And you could think of coming up with estimates in which this is ignored.
And you build a model based on this, or an alternative model in which this term is ignored, and you make predictions based on the second SAT.
And both models are likely to be essentially as good as the other one, because these two quantities are essentially the same.
So in that case, your theta's that you estimate are going to be very sensitive to little details of the data.
You change your data, you have your data, and your data tell you that this coefficient is big and that coefficient is small.
You change your data just a tiny bit, and your theta's would drastically change.
So this is a case in which you have multiple explanatory variables, but they're redundant in the sense that they're very closely related to each other, and perhaps with a linear relation.
So one must be careful about the situation, and do special tests to make sure that this doesn't happen.
Finally the biggest and most common blunder is that you run your linear regression, you get your linear model, and then you say oh, OK. Y is caused by X according to this particular formula.
Well, all that we did was to identify a linear relation between X and Y.
This doesn't tell us anything.
Whether it's Y that causes X, or whether it's X that causes Y, or maybe both X and Y are caused by some other variable that we didn't think about.
So building a good linear model that has small errors does not tell us anything about causal relations between the two variables.
It only tells us that there's a close association between the two variables.
If you know one you can make predictions about the other.
But it doesn't tell you anything about the underlying physics, that there's some physical mechanism that introduces the relation between those variables.
OK, that's it about linear regression.
Let us start the next topic, which is hypothesis testing.
And we're going to continue with it next time.
So here, instead of trying to estimate continuous parameters, we have two alternative hypotheses about the distribution of the X random variable.
So for example our random variable could be either distributed according to this distribution, under H0, or it might be distributed according to this distribution under H1.
And we want to make a decision which distribution is the correct one?
So we're given those two distributions, and some common terminologies that one of them is the null hypothesis-- sort of the default hypothesis, and we have some alternative hypotheses-- and we want to check whether this one is true, or that one is true.
So you obtain a data point, and you want to make a decision.
In this picture what would a reasonable person do to make a decision?
They would probably choose a certain threshold, Xi, and decide that H1 is true if your data falls in this interval.
And decide that H0 is true if you fall on the side.
So that would be a reasonable way of approaching the problem.
More generally you take the set of all possible X's, and you divide the set of possible X's into two regions.
One is the rejection region, in which you decide H1, or you reject H0.
And the complement of that region is where you decide H0.
So this is the x-space of your data.
In this example here, x was one-dimensional.
But in general X is going to be a vector, where all the possible data vectors that you can get, they're divided into two types.
If it falls in this set you'd make one decision.
If it falls in that set, you make the other decision.
OK, so how would you characterize the performance of the particular way of making a decision?
Suppose I chose my threshold.
I may make mistakes of two possible types.
Perhaps H0 is true, but my data happens to fall here.
In which case I make a mistake, and this would be a false rejection of H0.
If my data falls here I reject H0.
I decide H1.
Whereas H0 was true.
The probability of this happening?
Let's call it alpha.
But there's another kind of error that can be made.
Suppose that H1 was true, but by accident my data happens to falls on that side.
Then I'm going to make an error again.
I'm going to decide H0 even though H1 was true.
How likely is this to occur?
This would be the area under this curve here.
And that's the other type of error than can be made, and beta is the probability of this particular type of error.
Both of these are errors.
Alpha is the probability of error of one kind.
Beta is the probability of an error of the other kind.
You would like the probabilities of error to be small.
So you would like to make both alpha and beta as small as possible.
Unfortunately that's not possible, there's a trade-off.
If I go to my threshold it this way, then alpha become smaller, but beta becomes bigger.
So there's a trade-off.
If I make my rejection region smaller one kind of error is less likely, but the other kind of error becomes more likely.
So we got this trade-off.
So what do we do about it?
How do we move systematically?
How do we come up with rejection regions?
Well, what the theory basically tells you is it tells you how you should create those regions.
But it doesn't tell you exactly how.
It tells you the general shape of those regions.
For example here, the theory who tells us that the right thing to do would be to put the threshold and make decisions one way to the right, one way to the left.
But it might not necessarily tell us where to put the threshold.
Still, it's useful enough to know that the way to make a good decision would be in terms of a particular threshold.
Let me make this more specific.
We can take our inspiration from the solution of the hypothesis testing problem that we had in the Bayesian case.
In the Bayesian case we just pick the hypothesis which is more likely given the data.
The produced posterior probabilities using Bayesian rule, they're written this way.
And this term is the same as that term.
They cancel out, then let me collect terms here and there.
I get an expression here.
I think the version you have in your handout is the correct one.
The one on the slide was not the correct one, so I'm fixing it here.
OK, so this is the form of how you make decisions in the Bayesian case.
What you do in the Bayesian case, you calculate this ratio.
Let's call it the likelihood ratio.
And compare that ratio to a threshold.
And the threshold that you should be using in the Bayesian case has something to do with the prior probabilities of the two hypotheses.
In the non-Bayesian case we do not have prior probabilities, so we do not know how to set this threshold.
But we're going to do is we're going to keep this particular structure anyway, and maybe use some other considerations to pick the threshold.
So we're going to use a likelihood ratio test, that's how it's called in which we calculate a quantity of this kind that we call the likelihood, and compare it with a threshold.
So what's the interpretation of this likelihood?
We ask-- the X's that I have observed, how likely were they to occur if H1 was true?
And how likely were they to occur if H0 was true?
This ratio could be big if my data are plausible they might occur under H1.
But they're very implausible, extremely unlikely to occur under H0.
Then my thinking would be well the data that I saw are extremely unlikely to have occurred under H0.
So H0 is probably not true.
I'm going to go for H1 and choose H1.
So when this ratio is big it tells us that the data that we're seeing are better explained if we assume H1 to be true rather than H0 to be true.
So I calculate this quantity, compare it with a threshold, and that's how I make my decision.
So in this particular picture, for example the way it would go would be the likelihood ratio in this picture goes monotonically with my X.
So comparing the likelihood ratio to the threshold would be the same as comparing my x to the threshold, and we've got the question of how to choose the threshold.
The way that the threshold is chosen is usually done by fixing one of the two probabilities of error.
That is, I say, that I want my error of one particular type to be a given number, so I fix this alpha.
And then I try to find where my threshold should be.
So that this probability theta, probability out there, is just equal to alpha.
And then the other probability of error, beta, will be whatever it turns out to be.
So somebody picks alpha ahead of time.
Based on the probability of a false rejection based on alpha, I find where my threshold is going to be.
I choose my threshold, and that determines subsequently the value of beta.
So we're going to continue with this story next time, and we'll stop here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, if you have not yet done it, please take a moment to go through the course evaluation website and enter your comments for the class.
So what we're going to do today to wrap things up is we're going to go through a tour of the world of hypothesis testing.
See a few examples of hypothesis tests, starting from simple ones such as the one the setting that we discussed last time in which you just have two hypotheses, you're trying to choose between them.
But also look at more complicated situations in which you have one basic hypothesis.
Let's say that you have a fair coin and you want to test it against the hypotheses that your coin is not fair, but that alternative hypothesis is really lots of different hypothesis.
So is my coin fair?
Is my die fair?
Do I have the correct distribution for random variable, and so on.
And I'm going to end up with a few general comments about this whole business.
So the sad thing in simple hypothesis testing problems is the following-- we have two possible models, and this is the classical world so we do not have any prior probabilities on the two hypotheses.
Usually we want to think of these hypotheses as not being completely symmetrical, but rather one is the default hypothesis, and usually it's referred to as the null hypothesis.
And you want to check whether the null hypothesis is true, whether things are normal as you would have expected them to be, or whether it turns out to be false, in which case an alternative hypothesis would be correct.
So how does one go about it?
No matter what approach you use, in the end you're going to end up doing the following.
You have the space of all simple observations that you may obtain.
So when you do the experiment you're going to get an X vector, a vector of data that's somewhere.
And for some vectors you're going to decide that you accept H. Note for some vectors that you reject H0 and you accept H1.
So what you will end up doing is that you're going to have some division of the space of all X's into two parts, and one part is the rejection region, and one part is the acceptance region.
So if you fall in here you accept H0, if you fall here you'd reject H0.
So to design a hypothesis test basically you need to come up with a division of your X space into two pieces.
So the figuring out how to do this involves two elements.
One element is to decide what kind of shape so I want for my dividing curve?
And having chosen the shape of the dividing curve, where exactly do I put it?
So if you were to cut this space using, let's say, a straight cut you might put it here, or you might put it there, or you might put it there.
Where exactly are you going to put it?
So let's look at those two steps.
The first issue is to decide the general shape of your rejection region, which is the structure of your test.
And the way this is done for the case of two hypothesis is by writing down the likelihood ratio between the two hypothesis.
So let's call that quantity l of X.
It's something that you can compute given the data that you have.
A high value of l of X basically means that this probability here tends to be bigger than this probability.
It means that the data that you have seen are quite likely to have occurred under H1, but less likely to have occurred under H0.
So if you see data that they are more plausible, can be better explained, under H1, then this ratio is big, and you're going to choose in favor of H1 or reject H0.
That's what you do if you have discrete data.
You use the PMFs.
If you have densities, in the case of continues data, again you consider the ratio of the two densities.
So a big l of X is evidence that your data are more compatible with H1 rather than H0.
Once you accept this kind of structure then your decision is really made in terms of that single number.
That is, you had your data that was some kind of vector, and you condense your data into a single number-- a statistic as it's called-- in this case the likelihood ratio, and you put the dividing point somewhere here call it Xi.
And in this region you accept H1, in this region you accept H0.
So by committing ourselves to using the likelihood ratio in order to carry out the test we have gone from this complicated picture of finding a dividing line in x-space, to a simpler problem of just finding a dividing point on the real line.
OK, how are we going?
So what's left to do is to choose this threshold, Xi.
Or as it's called, the critical value, for making our decision.
And you can place it anywhere, but one way of deciding where to place it is the following-- look at the distribution of this random variable, l of X.
It's has a certain distribution under H0, and it has some other distribution under H1.
If I put my threshold here, here's what's going to happen.
When H0 is true, there is this much probability that I'm going to end up making an incorrect decision.
If H0 is true there's still a probability that my likelihood ratio will be bigger than Xi, and that's the probability of making an incorrect decision of this particular type.
That is of making a false rejection of H0.
Usually one sets this probability to a certain number, alpha.
For example alpha being 5 %.
And once you decide that you want this to be 5 %, that determines where this number Psi(Xi) is going to be.
So the idea here is that I'm going to reject H0 if the data that I have seen are quite incompatible with H0.
if they're quite unlikely to have occurred under H0.
And I take this level, 5%.
So I see my data and then I say well if H0 was true, the probability that I would have seen data of this kind would be less than 5 %.
Given that I saw those data, that suggests that H0 is not true, and I end up rejecting H0.
Now of course there's the other type of error probability.
If I put my threshold here, if H1 is true but my likelihood ratio falls here I'm going to make a mistake of the opposite kind.
H1 is true, but my likelihood ratio turned out to be small, and I decided in favor of H0.
This is an error of the other kind, this probability of error we call beta.
And you can see that there's a trade-off between alpha and beta.
If you move your threshold this way alpha become smaller, but beta becomes larger.
And the general picture is, in your trade-off, depending on where you put your threshold is as follows-- you can make this beta to be 0 if you put your threshold out here, but in that case you are certain that you're going to make a mistake of the opposite kind.
So beta equals 0, alpha equals 1 is one possibility.
Beta equals 1 alpha equals 0 is the other possibility if you send your thresholds complete to the other side.
And in general you're going to get a trade-off curve of some sort.
And if you want to use a specific value of alpha, for example alpha being 0.05, then that's going to determine for you the probability for beta.
Now there's a general, and quite important theorem in statistics, which were are not proving.
And which tells us that when we use likelihood ratio tests we get the best possible trade-off curve.
You could think of other ways of making your decisions.
Other ways of cutting off your x-space into a rejection and acceptance region.
But any other way that you do it is going to end up with some probabilities of error that are going to be above this particular curve.
So the likelihood ratio test turns out to give you the best possible way of dealing with this trade-off between alpha and beta.
We cannot minimize alpha and beta simultaneously, there's a trade-off between them.
But at least we would like to have a test that deals with this trade-off in the best possible way.
For a given value of alpha we want to have the smallest possible value of beta.
And as the theorem is that the likelihood ratio tests do have this optimality property.
For a given value of alpha they minimize the probability of error of a different kind.
So let's make all these concrete and look at the simple example.
We have two normal distributions with different means.
So under H0 you have a mean of 0.
Under H1 you have a mean of 1.
You get your data, you actually get several data drawn from one of the two distributions.
And you want to make a decision, which one of the two is true?
So what you do is you write down the likelihood ratio.
The density for a vector of data, if that vector was generated according to H0 -- which is this one, and the density if it was generated according to H1.
Since we have multiple data the density of a vector is the product of the densities of the individual elements.
Since we're dealing with normals we have those exponential factors.
A product of exponentials gives us an exponential of the sum.
I'll spare you the details, but this is the form of the likelihood ratio.
The likelihood ratio test tells us that we should calculate this quantity after we get your data, and compare with a threshold.
Now you can do some algebra here, and simplify.
And by tracing down the inequalities you're taking logarithms of both sides, and so on.
One comes to the conclusion that using a test that has a threshold on this ratio is equivalent to calculating this quantity, and comparing it with a threshold.
Basically this quantity here is monotonic in that quantity.
This being larger than the threshold is equivalent to this being larger than the threshold.
So this tells us the general structure of the likelihood ratio test in this particular case.
And it's nice because it tells us that we can make our decisions by looking at this simple summary of the data.
This quantity, this summary of the data on the basis of which we make our decision is called a statistic.
So you take your data, which is a multi-dimensional vector, and you condense it to a single number, and then you make a decision on the basis of that number.
So this is the structure of the test.
If I get a large sum of Xi's this is evidence in favor of H1 because here the mean is larger.
And so I'm going to decide in favor of H1 or reject H0 if the sum is bigger than the threshold.
How do I choose my threshold?
Well I would like to choose my threshold so that the probability of an incorrect decision when H0 is true the probability of a false rejection equals to a certain number.
Alpha, such as for example 5 %.
So you're given here that this is 5 %.
You know the distribution of this random variable, it's normal.
And you want to find the threshold value that makes this to be true.
So this is a type of problem that you have seen several times.
You go to the normal tables, and you figure it out.
So the sum of the Xi's has some distribution, it's normal.
So that's the distribution of the sum of the Xi's.
And you want this probability here to be alpha.
For this to happen what is the threshold value that makes this to be true?
So you know how to solve problems of this kind using the normal tables.
A slightly different example is one in which you have two normal distributions that have the same mean -- let's take it to be 0 -- but they have a different variance.
So it's sort of natural that here, if your X's that you see are kind of big on either side you would choose H1.
If your X's are near 0 then that's evidence for the smaller variance you would choose H0.
So to proceed formally you again write down to the form of the likelihood ratio.
So again the density of an X vector under H0 is this one.
It's the product of the densities of each one of the Xi's.
Product of normal densities gives you a product of exponentials, which is exponential of the sum, and that's the expression that you get.
Under the other hypothesis the only thing that changes is the variance.
And the variance, in the normal distribution, shows up here in the denominator of the exponent.
So you put it there.
So this is the general structure of the likelihood ratio test.
And now you do some algebra.
These terms are constants comparing this ratio to a constant is the same as just comparing the ratio of the exponentials to a constant.
Then you take logarithms, you want to compare the logarithm of this thing to a constant.
You do a little bit of algebra, and in the end you find that the structure of the test is to reject H0 if the sum of the squares of the Xi's is bigger than the threshold.
So by committing to a likelihood ratio test you are told that you should be making it your decision according to a rule of this type.
So this fixes the shape or the structure of the decision region, of the rejection region.
And the only thing that's left, once more, is to pick this threshold in order to have the property that the probability of a false rejection is equal to say 5 %.
So that's the probability that H0 is true, but the sum of the squares accidentally happens to be bigger than my threshold.
In which case I end up deciding H1.
How do I find the value of Xi prime?
Well what I need to do is to look at the picture, more or less of this kind, but now I need to look at the distribution of the sum of the Xi's squared.
Actually the sum of the Xi's squared is a non-negative random variable.
So it's going to have a distribution that's something like this.
I look at that distribution, and once more I want this tail probability to be alpha, and that determines where my threshold is going to be.
So that's again a simple exercise provided that you know the distribution of this quantity.
Do you know it?
Well we don't really know it, we have not dealt with this particular distribution in this class.
But in principle you should be able to find what it is.
It's a derived distribution problem.
You know the distribution of Xi, it's normal.
Therefore, by solving a derived distribution problem you can find the distribution of Xi squared.
And the Xi squared's are independent of each other, because the Xi's are independent.
So you want to find the distribution of the sum of random variables with known distributions.
And since they're independent, in principle, you can do this using the convolution formula.
So in principle, and if you're patient enough, you will be able to find the distribution of this random variable.
And then you plot it or tabulate it, and find where exactly is the 95th percentile of that distribution, and that determines your threshold.
So this distribution actually turns out to have a nice and simple closed-form formula.
Because this is a pretty common test, people have tabulated that distribution.
It's called the chi-square distribution.
There's tables available for it.
And you look up in the tables, you find the 95th percentile of the distribution, and this way you determine your threshold.
So what's the moral of the story?
The structure of the likelihood ratio test tells you what kind of decision region you're going to have.
It tells you that for this particular test you should be using the sum of the Xi squared's as your statistic, as the basis for making your decision.
And then you need to solve a derived distribution problem to find the probability distribution of your statistic.
Find the distribution of this quantity under H0, and finally, based on that distribution, after you have derived it, then determine your threshold.
So now let's move on to a somewhat more complicated situation.
You have a coin, and you are told that I tried to make a fair coin.
Is it fair?
So you have the hypothesis, which is the default-- the null hypothesis-- that the coin is fair.
But maybe it isn't.
So you have the alternative hypothesis that your coin is not fair.
Now what's different in this context is that your alternative hypothesis is not just one specific hypothesis.
Your alternative hypothesis consists of many alternatives.
It includes the hypothesis that p is 0.6.
It includes the hypothesis that p is 0.51.
It includes the hypothesis that p is 0.48, and so on.
So you're testing this hypothesis versus all this family of alternative hypothesis.
What you will end up doing is essentially the following-- you get some data.
That is, you flip the coin a number of times.
Let's say you flip it 1,000 times.
You observe some outcome.
Let's say you saw 472 heads.
And you ask the question if this hypothesis is true is this value really possible under that hypothesis?
Or would it be very much of an outlier?
If it looks like an extreme outlier under this hypothesis then I reject it, and I accept the alternative.
If this number turns out to be something within the range that you would have expected then you keep, or accept your null hypothesis.
OK so what does it mean to be an outlier or not?
First you take your data, and you condense them to a single number.
So your detailed data actually would have been a sequence of heads/tails, heads/tails and all that.
Any reasonable person would tell you that you shouldn't really care about the exact sequence of heads and tails.
Let's just base our decision on the number of heads that we have observed.
So using some kind of reasoning which could be mathematical, or intuitive, or involving artistry-- you pick a one-dimensional, or scalar summary of the data that you have seen.
In this case, the summary of the data is just the number of heads that's a quite reasonable one.
And so you commit yourself to make a decision on the basis of this quantity.
And you ask the quantity that I'm seeing does it look like an outlier?
Or does it look more or less OK?
OK, what does it mean to be an outlier?
You want to choose the shape of this rejection region, but on the basis of that single number s. And again, the reasonable thing to do in this context would be to argue as follows-- if my coin is fair I expect to see n over 2 heads.
That's the expected value.
If the number of heads I see is far from the expected number of heads then I consider this to be an outlier.
So if this number is bigger than some threshold Xi.
I consider it to be an outlier, and then I'm going to reject my hypothesis.
So we picked our statistic.
We picked the general form of how we're going to make our decision, and then we pick a certain significance, or confidence level that we want.
Again, this famous 5% number.
And we're going to declare something to be an outlier if it lies in the region that has 5% or less probability of occurring.
That is I'm picking my rejection region so that if H0 is true under the default, or null hypothesis, there's only 5% chance that by accident I fall there, and the thing makes me think that H1 is going to be true.
So now what's left to do is to pick the value of this threshold.
This is a calculation of the usual kind.
I want to pick my threshold, my Xi number so that the probability that s is further from the mean by an amount of Xi is less than 5%.
Or that the probability of being inside the acceptance region-- so that the distance from the default is less than my threshold.
I want that to be 95%.
So this is an equality that you can get using the central limit theorem and the normal tables.
There's 95% probability that the number of heads is going to be within 31 from the correct mean.
So the way the exercise is done of course, is that we start with this number, 5%.
Which translates to this number 95%.
And once we have fixed that number then you ask the question what number should we have here to make this equality to be true?
It's again a problem of this kind.
You have a quantity whose distribution you know.
Why do you know it?
The number of heads by the central limit theorem is approximately normal.
So this here talks about the normal distribution.
You set your alpha to be 5%, and you ask where should I put my threshold so that this probability of being out there is only 5%?
Now in our particular example the threshold turned out to be 31.
This number turned out was just 28 away from the correct mean.
So these distance was less than the threshold.
So we end up not rejecting H0.
So we have our rejection region.
The way we designed it is that when H0 is true there's only a small chance, 5%, that we get to data out of there.
Data that we would call an outlier.
If we see such an outlier we reject H0.
If what we see is not an outlier as in this case, where that distance turned out to be kind of small, then we do not reject H0.
An interesting little piece of language here, people generally prefer to use this terminology-- to say that H0 is not rejected by the data.
Instead of saying that H0 is accepted.
In some sense they're both saying the same thing, but the difference is sort of subtle.
When I say not rejected what I mean is that I got some data that are compatible with my hypothesis.
That is the data that I got do not falsify the hypothesis that I had, my null hypothesis.
So my null hypothesis is still alive, and may be true.
But from data you can never really prove that the hypothesis is correct.
Perhaps my coin is not fair in some other complicated way.
Perhaps I was just lucky, and even though my coin is not fair I ended up with an outcome that suggests that it's fair.
Perhaps my coin flips are not independent as I assumed in my model.
So there's many ways that my null hypothesis could be wrong, and still I got data that tells me that my hypothesis is OK.
So this is the general way that things work in science.
One comes up with a model or a theory.
This is the default theory, and we work with that theory trying to find whether there are examples that violate the theory.
If you find data and examples that violate the theory your theory is falsified, and you need to look for a new one.
But when you have your theory, really no amount of data can prove that your theory is correct.
So we have the default theory that the speed of light is constant as long as we do not find any data that runs counter to it.
We stay with that theory, but there's no way of really proving this, no matter how many experiments we do.
But there could be experiments that falsify that theory, in which case we need to do look for a new one.
So there's a bit of an asymmetry here in how we treat the alternative hypothesis.
H0 is the default which we'll accept until we see some evidence to the contrary.
And if we see some evidence to the contrary we reject it.
As long as we do not see evidence to the contrary then we keep working with it, but always take it with a grain of salt.
You can never really prove that a coin has a bias exactly equal to 1/2.
Maybe the bias is equal to 0.50001, so the bias is not 1/2.
But with an experiment with 1,000 coin tosses you wouldn't be able to see this effect.
OK, so that's how you go about testing about whether your coin is fair.
You can also think about testing whether a die is fair.
So for a die the null hypothesis would be that every possible result when you roll the die has equal probability and equal to 1/6.
And you also make the hypothesis that your die rolls are statistically independent from each other.
So I take my die, I roll it a number of times, little n, and I count how many 1's I got, how many 2's I got, how many 3's I got, and these are my data.
I count how many times I observed a specific result in my die roll that was equal to sum i.
And now I ask the question-- the Ni's that I observed, are they compatible with my hypothesis or not?
What does compatible to my hypothesis mean?
Under the null hypothesis Ni should be approximately equal, or is equal in expectation to N times little Pi.
And in our example this little Pi is of course 1/6.
So if my die is fair the number of ones I expect to see is equal to the number of rolls times 1/6.
The number of 2's I expect to see is again that same number.
Of course there's randomness, so I do not expect to get exactly that number.
But I can ask how far away from the expected values was i?
If my capital Ni's turn to be very different from N/6 this is evidence that my die is not fair.
If those numbers turn out to be close to N times 1/6 then I'm going to say there's no evidence that would lead me to reject this hypothesis.
So this hypothesis remains alive.
So someone has come up with this thought that maybe the right statistic to use, or the right way of quantifying how far away are the Ni's from their mean is to look at this quantity.
So I'm looking at the expected value of Ni under the null hypothesis.
See what I got, take the square of this, and add it over all i's.
But also throw in these terms in the denominator.
And why that term is there, that's a longer story.
One can write down certain likelihood ratios, do certain Taylor Series approximations, and there's a Heuristic argument that justifies why this would be a good form for the test to use.
So there's a certain art that's involved in this step that some people somehow decided that it's a reasonable thing to do is to calcelate.
Once you get your results to calculate this one-dimensional summary of your result, this is going to be your statistic, and compare that statistic to a threshold.
And that's how you make your decision.
So by this point we have fixed the type of the rejection region that we're going to have.
So we've chosen the qualitative structure of our test, and the only thing that's now left is to choose the particular threshold we're going to use.
And the recipe, once more, is the same.
We want to set our threshold so that the probability of a false rejection is 5%.
We want the probability that our data fall in here is only 5% when the null hypothesis is true.
So that's the same as setting our threshold Xi so that the probability that our test statistic is bigger than that threshold.
We want that probability to be only 0.05.
So to solve a problem of this kind what is it that you need to do?
You need to find the probability distribution of capital T. So once more it's the same picture.
You need to do some calculations of some sort, and come up with the distribution of the random variable T, where T is defined this way.
You want to find this distribution under hypothesis H0.
Once you find what that distribution is then you can solve this usual problem.
I want this probability here to be 5%.
What should my threshold be?
So what does this boil down to?
Finding the distribution of capital T is in some sense a messy, difficult, derived distribution problem.
From this model we know the distribution of the capital Ni's.
And actually we can even write down the joint distribution of the capital Ni's.
In fact we can make an approximation here.
Capital Ni is a binomial random variable.
Let's say the number of 1's that I got in little N rolls off my die.
So that's a binomial random variable.
When little n is big this is going to be approximately normal.
So we have normal random variables, or approximately normal minus a constant.
They're still approximately normal.
We take the squares of these, scale them so you can solve a derived distribution problem to find the distribution of this quantity.
You can do more work, more derived distribution work, and find the distribution of capital T. So this is a tedious matter, but because this test is used quite often, again people have done those calculations.
They have found the distribution of capital T, and it's available in tables.
And you go to those tables, and you find the appropriate threshold for making a decision of this type.
Now to give you a sense of how complicated hypothesis one might have to deal with let's make things one level more complicated.
So here you can think this X is a discrete random variable.
This is the outcome of my roll.
And I had a model in which the possible values of my discrete random variables they have probabilities all equal to 1/6.
So my null hypothesis here was a particular PMF for the random variable capital X.
So another way of phrasing what happened in this problem was the question is my PMF correct?
So this is the PMF of the result of one die roll.
You're asking the question is my PMF correct?
Make it more complicated.
How about the question of the type is my PDF correct when I have continuous data?
So I have hypothesized that's the probability distribution that I have is let's say a particular normal.
I get lots of results from that random variable.
Can I tell whether my results look like normal or not?
What are some ways of going about it?
Well, we saw in the previous slide that there is a methodology for deciding if your PMF is correct.
So you could take your normal results, the data that you got from your experiment, and discretize them, and so now you're dealing with discrete data.
And sort of used in previous methodology to solve a discrete problem of the type is my PDF correct?
So in practice the way this is done is that you get all your data, let's say data points of this kind.
You split your space into bins, and you count how many you have in each bin.
So you get this, and that, and that, and nothing.
So that's a histogram that you get from the data that you have.
Like the very familiar histograms that you see after each one of our quizzes.
So if you look at these histogram, and you ask does it look like normal?
OK, we need a systematic way of going about it.
If it were normal you can calculate the probability of falling in this interval.
The probability of falling in that interval, probability of falling into that interval.
So you would have expected values of how many results, or data points, you would have in this interval.
And compare these expected values for each interval with the actual ones that you observed.
And then take the sum of squares, and so on, exactly as in the previous slide.
And this gives you a way of going about it.
This is a little messy.
It gets hard to do because you have the difficult decision of how do you choose the bin size?
If you take your bins to be very narrow you would get lots of bins with 0's, and a few bins that only have one outcome in them.
It probably wouldn't feel right.
If you choose your bins to be very wide then you're losing a lot of information.
Is there some way of making a test without creating bins?
This is just to illustrate the clever ideas of what statisticians have thought about.
And here's a really cute way of going about a test, whether my distribution is correct or not.
Here we're essentially plotting a PMF, or an approximation of a PDF.
And we ask does it look like the PDF we assumed?
Instead of working with PDFs let's work with cumulative distribution functions.
So how does this go?
The true normal distribution that I have hypothesized, the density that I'm hypothesizing-- my null hypothesis-- has a certain CDF that I can plot.
So supposed that my hypothesis H0 is that the X's are normal with our standard normals, and I plot the CDF of the standard normal, which is the sort of continuous looking curve here.
Now I get my data, and I plot the empirical CDF.
What's the empirical CDF?
In the empirical CDF you ask the question what fraction of the data fell below 0?
You get a number.
What fraction of my data fell below 1?
I get a number.
What fraction of my data fell below 2, and so on.
So you're talking about fractions of the data that fell below each particular number.
And by plotting those fractions as a function of this number you get something that looks like a CDF.
And it's the CDF suggested by the data.
Now the fraction of the data that fall below 0 in my experiment is-- if my hypothesis were true-- expected to be 1/2.
1/2 is the value of the true CDF.
I look at the fraction that I got, it's expected to be that number.
But there's randomness, so it's might be a little different than that.
For any particular value, the fraction that I got below a certain number-- the fraction of data that we're below, 2, its expectation is the probability of falling below 2, which is the correct CDF.
So if my hypothesis is true the empirical CDF that I get based on data should, when n is large, be very close to the true CDF.
So a way of judging whether my model is correct or not is to look at the assumed CDF, the CDF under hypothesis H0.
Look at the CDF that I constructed based on the data, and see whether they're close enough or not.
And by close enough, I mean I'm going to look at all the possible X's, and look at the maximum distance between those two curves.
And I'm going to have a test that decides in favor of H0 if this distance is small, and in favor of H1 if this distance is large.
That still leaves me the problem of coming up with a threshold.
Where exactly do I put my threshold?
Because this test is important enough, and is used frequently people have made the effort to try to understand the probability distribution of this quite difficult random variable.
One needs to do lots of approximations and clever calculations, but these have led to values and tabulated values for the probability distribution of this random variable.
And, for example, those tabulated values tell us that if we want 5% false rejection probability, then our threshold should be 1.36 divided by the square root of n. So we know where to put our threshold for this particular value.
If we want this particular error or error probability to occur.
So that's about as hard and sophisticated classical statistics get.
You want to have tests for hypotheses that are not so easy to handle.
People somehow think of clever ways of doing tests of this kind.
How to compare the theoretical predictions with the observed predictions with the observed data.
Come up with some measure of the difference between theory and data, and if that difference is big, than you reject your hypothesis.
OK, of course that's not the end of the field of statistics, there's a lot more.
In some ways, as we kept moving through today's lecture, the way that we constructed those rejection regions was more and more ad hoc.
I pulled out of a hat a particular measure of fit between data and the model.
And I said let's just use a test based on this.
There are attempts at more or less systematic ways of coming up with the general shape of rejection regions that have at least some desirable or favorable theoretical properties.
Some more specific problems that people study-- instead of having a test, is this the correct PDF?
Yes or no.
I just give you data, and I ask you tell me, give me a model or a PDF for those data.
OK, my thoughts of this kind are of many types.
One general method is you form a histogram, and then you take your histogram and plot a smooth line, that kind of fits the histogram.
This still leaves the question of how do you choose the bins?
The bin size in your histograms.
How narrow do you take them?
And that depends on how many data you have, and there's a lot of theory that tells you about the best way of choosing the bin sizes, and the best ways of smoothing the data that you have.
A completely different topic is in signal processing -- you want to do your inference.
Not only you want it to be good, but you also want it to be fast in a computational way.
You get data in real time, lots of data.
You want to keep processing and revising your estimates and your decisions as they come and go.
Another topic that was briefly touched upon the last couple of lectures is that when you set up a model, like a linear regression model, you choose some explanatory variables, and you try to predict y from your X, these variables.
You have a choice of what to take as your explanatory variables.
Are there systematic ways of picking the right X variables to try to estimate a Y.
For example should I try to estimate Y on the basis of X?
Or on the basis of X-squared?
How do I decide between the two?
Finally, the rage these days has to do with anything big, high-demensional.
Complicated models of complicated things, and tons and tons of data.
So these days data are generated everywhere.
The amounts of data are humongous.
Also, the problems that people are interested in tend to be very complicated with lots of parameters.
So I need specially tailored methods that can give you good results, or decent results even in the face of these huge amounts of data, and possibly with computational constraints.
So with huge amounts of data you want methods that are simple, but still can deliver for you meaningful answers.
Now as I mentioned some time ago, this whole field of statistics is very different from the field of probability.
In some sense all that we're doing in statistics is probabilistic calculations.
That's what the theory kind of does.
But there's a big element of art.
You saw that we chose the shape of some decision regions or rejection regions in a somewhat ad hoc way.
There's even more basic things.
How do you organize your data?
How do you think about which hypotheses you would like to test, and so on.
There's a lot of art that's involved here, and there's a lot that can go wrong.
So I'm going to close with a note that you can take either as pessimistic or optimistic.
There is a famous paper that came out a few years ago and has been cited about a 1,000 times or so.
And the title of the paper is Why Most Published Research Findings Are False.
And it's actually a very good argument why, in fields like psychology or the medical science and all that a lot of what you see published-- that yes, this drug has an effect on that particular disease-- is actually false, because people do not do their statistics correctly.
There's lots of biases in what people do.
I mean an obvious bias is that you only published a result when you see something.
So the null hypothesis is that the drug doesn't work.
You do your tests, the drug didn't work, OK, you just go home and cry.
But if by accident that 5% happens, and even though the drug doesn't work, you got some outlier data, and it seemed to be working.
Then you're excited, you publish it.
So that's clearly a bias.
That gets results to be published, even though they do not have a solid foundation behind them.
Then there's another thing, OK?
I'm picking my 5%.
So H0 is true there's a small probability that the data will look like an outlier, and in that case I published my result.
OK it's only 5% -- it's not going to happen too often.
But suppose that I go and do a 1,000 different tests?
Test H0 against this hypothesis, test H0 against that hypothesis , test H0 against that hypothesis.
Some of these tests, just by accident might turn out to be in favor of H1, and again these are selected to be published.
So if you do lots and lots of tests and in each one you have a 5% probability of error, when you consider the collection of all those tests, actually the probability of making incorrect inferences is a lot more than 5%.
One basic principle in being systematic about such studies is that you should first pick your hypothesis that you're going to test, then get your data, and do your hypothesis testing.
What would be wrong is to get your data, look at them, and say OK I'm going now to test for these 100 different hypotheses, and I'm going to choose my hypothesis to be for features that look abnormal in my data.
Well, given enough data, you can always find some abnormalities just by chance.
And if you choose to make a statistical test-- is this abnormality present?
Yes, it will be present.
Because you first found the abnormality, and then you tested for it.
So that's another way that things can go wrong.
So the moral of this story is that while the world of probability is really beautiful and solid, you have your axioms.
Every question has a unique answer that by now you can, all of you, find in a very reliable way.
Statistics is a dirty and difficult business.
And that's why the subject is not over.
And if you're interested in it, it's worth taking follow-on courses in that direction.
OK so have good luck in the final, do well, and have a nice vacation afterwards.
Hi.
Welcome back.
Today, we're going to do a fun problem called the chess tournament problem.
Now, it's a very long problem, so I just want to jump straight in.
Essentially, the problem statement describes a very special chess tournament, which involves players named Al, Bo, and Chi.
Now Al is the current reigning championship, and Bo and Chi are this year's contenders, and, of course, they're vying with each other to beat out Al and become the new champion.
And so essentially, the tournament is divided into two rounds-- a first round, during which Bo and Chi play against each other, and then a second round, during which the surviving contender from the first round plays against Al.
And the problem statement also gives you a bunch of information like what's the probability that Bo beats Chi in a particular game, et cetera.
So without further ado, let's get started on part a.
In part a, the first thing we're asked to compute is the probability that a second round is required.
Now, to save myself some writing, I used the notation R2 to denote that event.
So we are interested in probability of R2.
Now, I claim that this problem is very sequential in nature so I would like to draw a tree to describe what's happening.
So in the first part of the tournament, when Bo and Chi play their first game, exactly one of two things can happen-- either Bo can win or Chi can win.
And we're told by the problem statement that Bo wins with the probability of 0.6 and, therefore, Chi must win with the probability of 0.4, right?
Because these two possibilities must sum to 1, because either this must happen or this happen.
Now, let's imagine that the first game has been played and that Bo won.
Well, during the second game, there's still two options for the outcome-- Bo could win the second game or Chi could win the second game.
And because the problem statement says that in every scenario Bo always wins against Chi with the probability of 0.6, we can go ahead and put a 0.6 along this branch as well.
Similarly, 0.4 here.
And similar logic, you've got a tree that looks like this.
And for those of you who haven't seen trees before, it's just a structure that looks something like this.
And it helps us do better accounting.
It helps us keep straight in our head what are the various outcomes, so that we don't get confused.
And so very quickly here, you can see that there's four possible outcomes.
So each node in this tree corresponds to an outcome.
And the leaves are those nodes at the furthest stage.
And it's convention to draw the probability of a particular-- so for instance, the probability that Bo wins the first game-- it's just convention to draw that probability over the corresponding branch.
And the reason why such diagrams are so useful is because to compute the probability of a particular outcome, if you've designed your tree correctly, all you have to do is multiply the probabilities along the branches that get into that outcome.
So let's see that in action.
When is a second round required?
Well, a second round is required here, right?
Because in this case, Bo would be the surviving challenger and he'd play the next round against Al.
It's also required here.
But of course, it's not required here or here, because no second round is played.
And so these two outcomes comprise the event R2.
And now, to get the probability of this outcome, you multiply along the branches.
So 0.6 times 0.6 give you 0.36.
And 0.4 times 0.4 gives you 0.16.
And we're almost done.
We know that these two events are disjoint, because if Bo won the first two games, then, certainly, Chi could've won the first two games.
And so you can just sum the probabilities to get the probability of the reunion.
So the probability of R2 is equal to the probability that Bo won the first two games or Chi won the first two games.
And that's equal to 0.36 plus 0.16, which is equal to a 0.52.
OK, now the second part of part a asks for the probability that Bo wins the first round.
And this is first round.
This is a very straightforward one.
So Bo wins the first round, that correspondence only to this particular outcome.
And we already know the probability associated with that outcome is equal to the 0.36.
So we're done with that one.
And now the last part is sort of an interesting one.
It asks for the probability that Al retains his championship this year.
So I'm going to just call that A for short.
A is the event that Al retains his championship this year.
And for that we're going to need a larger tree, because Al has a lot of activity in the second round, and so far our tree only describes what happens in the first round.
Now, to save time, I've actually drawn the rest of the tree over there up in the corner.
So let's get rid of this one and let's look at the full tree.
So let's see when does Al retain his championship?
Well, Al certainly retains his championship here, right?
Because no second round is required.
Similarly, here.
Al retains his championship here, because the second round was required, but Al beat Bo.
And similarly, here Bo didn't win both games in the second round against Al, so Al wins.
Here, Bo is the new championship.
So we don't want to include about one.
And sort of by symmetry, we also get this one and this one.
So by my argument before, we know that the outcomes that comprise our event of interest are this one, this one, this one, this one, this one, and this one.
So we could multiply the probabilities along each branch and sum them, because they're disjoint, to get the total probability.
But we're not going to do that because that's a lot of algebra.
Instead, we're going to look at the complement of the event.
So we're going to notice, there's only two branches on which Al does not retain his current championship.
So P of A is, of course, equal to 1 minus P of A.
And we're going to get P of A by inspection.
I'm sorry, so P of A compliment.
I'm just testing you, guys.
So P of A compliment corresponds to here and to here, because those are the outcomes where Al didn't win.
And so again, you multiply along the branches to get the probabilities.
So you get 0.6 squared times 0.5 squared plus 0.4 squared times 0.3 squared.
And if you do all the algebra, you should get around 0.8956.
So we're cruising through this problem.
Let's go to part b.
Part b is a little bit less straightforward than part a, because it starts asking you for conditional probabilities, as opposed to a priori probabilities.
So in the first part-- and again, I'm going to continue my notation with R2-- we want the probability that Bo is the surviving challenger-- so I'm just going to use B to denote that-- given R2.
Now, by definition, you should remember from lecture that this is equal to probability of B and R2 divided by the probability of R2.
And of course, we've already computed this value right up here with part a.
We know it's 2.5.
So we don't have to do any more work there.
We only have to look at the numerator.
So we need to go and figure out what nodes in that tree correspond to the event B intersect R2.
So let's use a new color.
Let's see, Bo is the surviving challenger here only, right?
And R2 is automatically satisfied, right?
Because a second round is required there and there, not on those two.
But here Chi is the surviving challenger, not Bo, so we're really only interested in that node.
And you multiply along the branches to get the probabilities.
So we have 0.36 over 0.52, which is approximately equal to 0.6923.
OK, now, the next part wants the conditional probability that AL retains his championship, conditioned, again, on R2.
So we already have A being the event Al retains his championship.
So we want the probability of A, given R2.
And let's just apply the direct definition of conditional probability again.
You get P of A and R2 divided by a probability of R2.
Of course, we have the probability of R2 already, so we just need to find the node in the tree that corresponds to A and R2.
So where is R2?
R2 is going to correspond to every node to the right that is not one of these two.
So a second round is required here, here, here, here, here, and here.
Now, where does Al retain his championship?
So Al retains his championship here.
He retains his championship here.
He retains his championship here and here, but no second round is required, so these guys don't belong in the intersection.
But this does, and this does.
So we can again multiply the probabilities along the branches and then some them.
So let's see, we get-- this marker's not working very well, so I'm going to switch back to the pink-- so you get 0.6 squared times 0.5.
That gets rid of this one.
And then we want 0.6 squared times 0.5 squared.
That gets rid of that one.
And then plus-- let's see-- 0.4 squared times 0.7, which takes care of this one.
And then lastly, 0.4 squared times 0.3 times 0.7.
And that is a long expression.
But it happens to be about 0.7992.
OK, so we are done with part b and we can move along to part c. And I am, since we're running out of room, I'm actually just going to erase this.
And hopefully you guys have had a chance to copy it down by now.
If not, you can always pause the video and go back.
So let's see, part c asks us given that the second round is required and that it comprised of one game only.
So let's denote I.
So let's I be the event that the second round was one game only.
So essentially, in math conditioned on R2 and I, what is the probability that it was Bo who won the first round?
So let's let B be the event that Bo won the first round.
OK, so again translating the English to math, we just want the probability of B given R2 and I.
Now, I am once again going to use the definition of conditional probability.
You might be concerned that we haven't defined explicitly yet the definition of conditional probability, when what lies behind the conditioning bar is not a single event, but it's rather an intersection of an event.
And so my claim to you is that it doesn't matter and that the same exact definition applies.
But we'll go through it slowly.
So R2 is an event, I is an event, and we know that the intersection of two events is itself an event.
So I'm going to make up a new letter, and I'm going to call this event W. So just using the new notation, this is equal to probability of B, given W. Now, this is the normal definition that we know.
We know that this is probability of B intersect W over probability of W. And then we just resubstitute what the definition of W was.
And so if you do that over here, you get probability of B and R2 and I divided by probability of R2 and I.
So hopefully, jumping from here ahead to here, you see that the definitions act exactly the same way.
But these are two very short intermediate steps that should help you convince yourself that the same definition still works.
So let's start with the denominator, because the denominator looks a little bit easier.
Where is R2 and I in our tree?
Well, let's see.
Here, a second round was required, but it comprised two games.
Same with this one.
Here, a second round was required and it was comprised only of one game.
So this is good.
This is one of the outcomes that we're looking for.
Here, no second round was required.
So this doesn't qualify.
Same with this one.
Here, a second round was required, and there was only one game, so that's good.
And then these don't qualify for the same reasons as we set up there.
So we just have to multiply the probabilities along those branches.
And we see that it's 0.4 squared times 0.7 plus 0.6 squared times 0.5.
OK, we're almost done.
We just need to look at the intersection of R2 and I.
So R2 and I are the ones we've already circled.
But now, we want to add one more constraint, which is that Bo had to have won the first round.
And so we see here that Chi won the first round, if we're looking at this outcome.
And so he's no good.
Let's use a different color.
Let's see, maybe this one.
But here Bo did win the first round.
So we're going to get 0.6 squared times 0.5.
And I got that, of course, just by multiplying the probabilities along the right branches.
And this, if you're curious, comes out to be about 0.6164.
OK, so I know that was a lengthy problem, but you should feel really comfortable now doing sort of basic probability manipulations.
One thing that this problem emphasized a lot was your ability to compute conditional probabilities.
So you saw me apply the definition of conditional probability twice in part b.
And then you saw me apply the definition again in part c in a sort of slightly modified way.
So that's one thing that you should have gotten out of this problem.
And then another thing is that hopefully, you noticed that by using a tree diagram, we made the problem much easier.
We almost didn't even have to think about computing probabilities anymore.
We reduced the problem to just saying, OK, what are the outcomes that comprise our event of interest?
And then once you select those, to compute their probability you multiply the probabilities along the branches.
You have the right to just add those together, because if you draw your tree correctly, all of these guys should be disjoint from one another.
So you have to be careful, of course, to set up your tree appropriately.
But once you do set up your tree appropriately, your life is much simpler.
So that's it for today.
And we'll see you next time.
Hi.
In this problem, we're going to be dealing with a variation of the usual coin-flipping problem.
But in this case, the bias itself of the coin is going to be random.
So you could think of it as, you don't even know what the probability of heads for the coin is.
So as usual, we're still taking one coin and we're flipping it n times.
But the difference here is that the bias is because it was random variable Q.
And we're told that the expectation of this bias is some mu and that the variance of the bias is some sigma squared, which we're told is positive.
And what we're going to be asked is find a bunch of different expectations, covariances, and variances.
And we'll see that this problem gives us some good exercise in a few concepts, a lot of iterated expectations, which, again, tells you that when you take the expectation of a conditional expectation, it's just the expectation of the inner random variable.
The covariance of two random variables is just the expectation of the product minus the product of the expectations.
Law of total variance is the expectation of a variance, of a conditional variance plus the variance of a conditional expectation.
And the last thing, of course, we're dealing with a bunch of Bernoulli random variables, coin flips.
So as a reminder, for a Bernoulli random variable, if you know what the bias is, it's some known quantity p, then the expectation of the Bernoulii is just p, and the variance of the Bernoulli is p times 1 minus p. So let's get started.
The problem tells us that we're going to define some random variables.
So xi is going to be a Bernoulli random variable for the i coin flip.
So xi is going to be 1 if the i coin flip was heads and 0 if it was tails.
And one very important thing that the problem states is that conditional on Q, the random bias, so if we know what the random bias is, then all the coin flips are independent.
And that's going to be important for us when we calculate all these values.
OK, so the first thing that we need to calculate is the expectation of each of these individual Bernoulli random variables, xi.
So how do we go about calculating what this is?
Well, the problem gives us a int.
It tells us to try using the law of iterated expectations.
But in order to use it, you need to figure out what you need the condition on.
What this y?
What takes place in y?
And in this case, a good candidate for what you condition on would be the bias, the Q that we're unsure about.
So let's try doing that and see what we get.
So we write out the law of iterated expectations with Q.
So now hopefully, we can simplify it with this inter-conditional expectation is.
Well, what is it really?
It's saying, given what Q is, what is the expectation of this Bernoulli random interval xi?
Well, we know that if we knew what the bias was, then the expectation is just the bias itself.
But in this case, the bias is random.
But remember a conditional expectation is still a random variable.
And so in this case, this actually just simplifies into Q.
So whatever the bias is, the expectation is just equal to the bias.
And so that's what it tells us.
And this part is easy because we're given that the expectation of q is mu.
And then the problem also defines the random variable x. X is the total number of heads within the n tosses.
Or you can think of it as a sum of all these individual xi Bernoulli random variables.
And now, what can we do with this?
Well we can remember that linearity of expectations allows us to split up this sum.
Expectation of a sum, we could split up into a sum of expectations.
So this is actually just expectation of x1 plus dot dot dot plus all the way to expectation of xn.
All right.
And now, remember that we're flipping the same coin.
We don't know what the bias is, but for all the n flips, it's the same coin.
And so each of these expectations of xi should be the same, no matter what xi is.
And each one of them is mu.
We already calculated that earlier.
And there's 10 of them, so the answer would be n times mu.
So let's move on to part B.
Part B now asks us to find what the covariance is between xi and xj.
And we have to be a little bit careful here because there are two different scenarios, one where i and j are different indices, different tosses, and another where i and j are the same.
So we have to consider both of these cases separately.
Let's first do the case where x and i are different.
So i does not equal j.
In this case, we can just apply the formula that we talked about in the beginning.
So this covariance is just equal to the expectation of xi times xj minus the expectation of xi times expectation of xj.
All right, so we actually know what these two are, right?
Expectation of xi is mu.
Expectation of xj is also mu.
So this part is just mu squared.
But we need to figure out what this expectation of xi times xj is.
Well, the expectation of xi times xj, we can again use the law of iterated expectations.
So let's try conditioning on cue again.
And remember we said that this second part is just mu squared.
All right, well, how can we simplify this inner-conditional expectation?
Well, we can use the fact that the problem tells us that, conditioned on Q, the tosses are independent.
So that means that, conditioned on Q, xi and xj are independent.
And remember, when random variables are independent, the expectation of product, you could simplify that to be the product of the expectations.
And because we're in the condition world on Q, you have to remember that it's going to be a product of two conditional expectations.
So this will be expectation of xi given Q times expectation of xj given Q minus mu squared still.
All right, now what is this?
Well the expectation of xi given Q, we already argued earlier here that it should just be Q.
And then the same thing for xj.
That should also be Q.
So this is just expectation of Q squared minus mu squared.
All right, now if we look at this, what is the expectation of Q squared minus mu squared?
Well, remember mu is just, we're told that mu is the expectation of Q.
So what we have is the expectation of Q squared minus the quantity expectation of Q squared.
And what is that, exactly?
That is just the formula or the definition of what the variance of Q should be.
So this is, in fact, exactly equal to the variance of Q, which we're told is sigma squared.
All right, so what we found is that for i not equal to j, the coherence of xi and xj is exactly equal to sigma squared.
And remember, we're told that sigma squared is positive.
So what does that tell us?
That tells us that xi and xj, or i not equal to j, these two random variables are correlated.
And so, because they're correlated, they can't be independent.
Remember, if two intervals are independent, that means they're uncorrelated.
But the converse isn't true.
But if we do know that two random variables are correlated, that means that they can't be independent.
And now let's finish this by considering the second case.
The second case is when i actually does equal j.
And in that case, well, the covariance of xi and xi is just another way of writing the variance of xi.
So covariance, xi, xi, it's just the variance of xi.
And what is that?
That is just the expectation of xi squared minus expectation of xi quantity squared.
And again, we know what the second term is.
The second term is expectation of xi quantity squared.
Expectation of xi we know from part A is just mu, right?
So that's just second term is just mu squared.
But what is the expectation of xi squared?
Well, we can think about this a little bit more.
And you can realize that xi squared is actually exactly the same thing as just xi.
And this is just a special case because xi is a Bernoulli random variable.
Because Bernoulli is either 0 or 1.
And if it's 0 and you square it, it's still 0.
And if it's 1 and you square it, it's still 1.
So squaring it doesn't really doesn't actually change anything.
It's exactly the same thing as the original random variable.
And so, because this is a Bernoulli random variable, this is exactly just the expectation of xi.
And we said this part is just mu squared.
So this is just expectation of xi, which we said was mu.
So the answer is just mu minus mu squared.
OK, so this completes part B.
And the answer that we wanted was that in fact, xi and xj are in fact not independent.
Right.
So let's write down some facts that we'll want to remember.
One of them is that expectation of xi is mu.
And we also want to remember what this covariance is.
The covariance of xi and xj is equal to sigma squared when i does not equal j.
So we'll be using these facts again later.
And the variance of xi is equal to mu minus mu squared.
So now let's move on to the last part, part C, which asks us to calculate the variance of x in two different ways.
So the first way we'll do it is using the law of total variance.
So the law of total variance will tell us that we can write the variance of x as a sum of two different parts.
So the first is variance of x expectation of the variance of x conditioned on something plus the variance of the initial expectation of x conditioned on something.
And as you might have guessed, what we're going to condition on is Q.
Let's calculate what these two things are.
So let's do the two terms separately.
What is the expectation of the conditional variance of x given Q?
Well, what is-- this, we can write out x.
Because x, remember, is just the sum of a bunch of these Bernoulli random variables.
And now what we'll do was, well, again, use the important fact that the x's, we're told, are conditionally independent, conditional on Q.
And because they're independent, remember the variance of a sum is not the sum of the variance.
It's only the sum of the variance if the terms in the sum are independent.
In this case, they are conditionally independent given Q.
So we can in fact split this up and write it as the variance of x1 given Q plus all the way to the variance of xn given Q.
And in fact, all these are the same, right?
So we just have n copies of the variance of, say, x1 given Q.
Now, what is the variance of x1 given Q?
Well, x1 is just a Bernoulli random variable.
But the difference is that for x, we don't know what the bias or what the Q is.
Because it's some random bias Q But just like we said earlier in part A, when we talked about the expectation of x1 given Q, this is actually just Q times 1 minus Q.
Because if you knew what the bias were, it would be p times 1 minus p. So the bias times 1 minus the bias.
But you don't know what it is.
But if you did, it would just be q.
So what we do is we just plug in Q, and you get Q times 1 minus 2.
All right, and now this is expectation of n. I can pull out the n. So it's n times the expectation of Q minus Q squared, which is just n times expectation Q, we can use linearity of expectations again, expectation of Q is mu.
And the expectation of Q 2 squared is, well, we can do that on the side.
Expectation of Q squared is the variance of Q plus expectation of Q quantity squared.
So that's just sigma squared plus mu squared.
And so this is just going to be then minus sigma squared minus mu squared.
All right, so that's the first term.
Now let's do the second term.
The variance the conditional expectation of x given Q.
And again, what we can do is we can write x as the sum of all these xi's.
And now we can apply linearity of expectations.
So we would get n times one of these expectations.
And remember, we said earlier the expectation of x1 given Q is just Q.
So it's the variance of n times Q.
And remember now, n is just-- it's not random.
It's just some number.
So when you pull it out of a variance, you square it.
So this is n squared times the variance of Q.
And the variance of Q we're given is sigma squared.
So this is n squared times sigma squared.
So the final answer is just a combination of these two terms.
This one and this one.
So let's write it out.
The variance of x, then, is equal to-- we can combine terms a little bit.
So the first one, let's take the mus and we'll put them together.
So it's n mu minus mu squared.
And then we have n squared times sigma squared from this term and minus n times sigma squared from this term.
So it would be n squared minus n times sigma squared, or n times n minus 1 times sigma squared.
So that is the final answer that we get for the variance of x.
And now, let's try doing it another way.
So that's one way of doing it.
That's using the law of total expectations and conditioning on Q.
Another way of finding the variance of x is to use the formula involving covariances, right?
And we can use that because x is actually a sum of multiple random variables x1 through xn.
And the formula for this is, you have n variance terms plus all these other ones.
Where i is not equal to j, you have the covariance terms.
And really, it's just, you can think of it as a double sum of all pairs of xi and xj where if i and j happen just to be the same, that it simplifies to be just the variance.
Now, so we pulled theses n terms out because they are different than these because they have a different value.
And now fortunately, we've already calculated what these values are in part B.
So we can just plug them them.
All the variances are the same.
And there's n of them, so we get n times the variance of each one.
The variance of each one we calculated already was mu minus mu squared.
And then, we have all the terms were i is not equal to j.
Well, there are actually n squared minus n of them.
So because you can take any one of the n's to be the first to be i, any one of the n to be j.
So that gives you n squared pairs.
But then you have to subtract out all the ones where i and j are the same.
And there are n of them.
So that leaves you with n squared minus n of these pairs where i is not equal to j.
And the coherence for this case where i is not equal to j, we also calculated in part B.
That's just sigma squared.
All right, and now if we compare these two, we'll see that they are proportionally exactly the same.
So we've use two different methods to calculate the variance, one using this summation and one using the law of total variance.
So what do we learn from this problem?
Well, we saw that first of all, in order to find some expectations, it's very useful to use law of iterated expectations.
But the trick is to figure out what you should condition on.
And that's kind of an art that you learn through more practice.
But one good rule of thumb is, when you have kind of a hierarchy or layers of randomness where one layer of randomness depends on the randomness of the layer above-- so in this case, whether or not you get heads or tails depends on, that's random, but that depends on the randomness on the level above, which was the random bias of the coin itself.
So the rule of thumb is, when you want to calculate the expectations for the layer where you're talking about heads or tails, it's useful to condition on the layer above where that is, in this case, the random bias.
Because once you condition on the layer above, that makes the next level much simpler.
Because you kind of assume that you know what all the previous levels of randomness are, and that helps you calculate what the expectation for this current level.
And the rest of the problem was just kind of going through exercises of actually applying the--
Hi.
In this session, we're going to cover a nice review problem that will look at how to infer one random variable based on another.
And in this problem, we're given two random variables-- X and Y-- and we're also given their joint pdf, which we're told is a constant 2/3 within the region bounded by these orange lines.
And outside of the region, the joint pdf is 0.
So the first thing we're going to look at, or the first thing that we're asked to do, is find the LMS estimator of Y based on X.
Now, remember that LMS estimator is really just a conditional expectation.
So the LMS estimator of Y based on X is the conditional expectation of Y, given X.
Now, when we have a plot of the joint pdf and we're dealing with these two random variables, and especially when the joint pdf is constant like this, it's often easy to calculate this conditional expectation visually.
So what we really need to do is just say, given any particular value of X, what is the conditional expectation of Y?
So what we can do is we can just pick some values of X and see visually what that initial expectation is.
So for example, if X is 1/2, given that X is 1/2, and since this whole joint pdf is uniform, then the conditional slice of Y will be from here to here.
And that slice, the conditional distribution of Y, given that X is 1/2, will also be uniform.
So it'll be uniform from here to here.
And if it's uniform, we know that the conditional expectation will just be the midpoint here.
And so that would be what the conditional expectation of Y would be, given that X is 1/2.
And we could do the same thing for X equals 1.
And we'll see that again, because everything is uniform, this slice is also going to be uniform.
And so the conditional expectation will again be the midpoint, which is there.
And then if we just look at it within this region, it's always going to be the midpoint.
And so we get that the initial expectation of Y, given X, will just look like that line, which you can think of it as just bisecting this angle formed by these two parts of the region.
But things are a little bit different, though, when we move to the region where X is between 1 and 2.
Between 1 and 2, say at 1 and 1/2, this line doesn't continue.
Because now, the slice of Y goes from here to here, and again, it's still uniform.
So the midpoint would be there.
And similarly for X equals 2, it would be here.
And so for X between 1 and 2, the conditional expectation actually looks like this.
So you see that there's actually two linear parts of it, but there's a kink at X equals 1.
And so by looking at this visually and taking advantage of the fact that everything is uniform, we can pretty easily figure out what this conditional expectation is.
So now, let's actually just write it out algebraically.
So for X between 0 and 1, we said that it's this line, which if we look at it, that's just 1/2 of X.
Now, this is for X between 0 and 1.
And if X is between 1 and 2, it's going to be this line, which is a slope of 1.
And if we extend this down, it hits the y-axis at negative 1/2.
So it's X minus 1/2, if X is between 1 and 2.
And otherwise, it's undefined.
So we'll focus on these two cases here.
Now, the second part of the question, now that we know what the LMS estimator is, we're asked to find what is the traditional mean squared error of this estimator?
So we want to know how good is it.
And one way of capturing that is to look at the mean squared error.
And so recall that the conditional mean squared error is given by this expression.
So what we're saying is this is what we estimate Y to be.
This is what y really is, so this difference is how wrong, or the error in our estimate.
We square it, because otherwise, positive and negative errors might cancel each other out, so we square it.
And then this just looking at each individual value of x for now.
So this is why it's the conditional mean squared error.
So how do we calculate this?
Well, remember that this g of X, we said the LMS estimator is just a conditional expectation.
So it's just expectation of Y, given X.
Well, then if you look at this, what this reminds you of, it reminds you of the definition of what a conditional variance is.
A variance is just, you take the random variable, subtract its mean, square it, and take the expectation of that.
This is no different, except that everything is now the conditional world of X.
So this is actually the conditional variance of Y, given X is little x.
What is the conditional variance of Y, given that X is little x?
Now, we can again go back to this plot to try to help us out.
We can split this up into regions again.
So just take some little x as an example and see what the variance is.
So if little x is 1/2, then we know that the conditional distribution of Y would be uniform, we said, from 0 up to here.
Well, that point is this from 0 to 1/2.
And remember, the variance of a uniform distribution is just the width of the uniform distribution squared, divided by 12.
And so in this case, the width would be 1/2 squared over 12.
And in general, for the region of X between 0 and 1, the width of the conditional distribution of Y will always be X, because the width will go from 0 to wherever X is.
So because of that, the conditional variance will just be X squared, the width squared, over 12, when X is between 0 and 1.
Now, let's think about the other case, where X is between 1 and 2.
Well, if X is between 1 and 2, we're over here.
And now, if we take the conditional distribution of Y, it's again uniform.
But the width now, instead of varying with Y, it's always going to be the same width.
Each of these slices have the same width, and the width goes from here-- this is X minus 1, and that's X.
So if the width is always going to be a constant of 1.
And so this variance is going to be 1/12.
And from that, we get our answer for the conditional mean squared error.
Now, part c asks us to find the mean squared error, which is given by this expression.
And we'll see that it looks very similar to this, which was the conditional mean squared error.
And now, given what we know from part b, this is easy to calculate.
We can just apply total expectation, because this is just equal to the integral of the conditional mean squared error.
And then we need to also multiply this by the pf of x, and then integrate over X.
And that integral will should be from X equals 0 to 2, because that's the only range that applies for x, given this joint pdf.
Now, in order to do this first, though, we need to figure out what the pdf of X is.
In order to do that, we can go back to our original joint pdf of X and Y and marginalize it.
So marginalizing, you could think of it as taking this joint pdf and collapsing it onto the x-axis so that you take everything and integrate out Y.
Now to do that, let's do that up here.
We can split it up into two sections.
So the section of X between 0 and 1, we integrate the joint pdf from Y equals 0 to Y equals X, which is this portion of this line.
So we integrate Y.
The joint pdf is 2/3, and we integrate Y out from Y equals 0 to X.
And then for the portion of X from 1 to 2, we again integrate Y out.
Now we integrate Y from X minus 1 up to X.
So this is X between 0 and 1, and this is X between 1 and 2.
So we just do some little bit of calculus, and we get that this is going to be 2/3 X when X is between 0 and 1.
And it's going to be 2/3 when X is between 1 and 2.
So now that we have what the marginal pdf of X is, we can plug that into this, and plug in what we had for b, and then calculate what this actually is.
So remember, we need to take care of these two cases, these two regions-- X between 0 and 1, and X between 1 and 2.
So the conditional mean squared error for X between 0 and 1 is X squared over 2.
So between 0 and 1, this first part is X squared over 12.
The pdf of X in that same region is 2/3 x.
And we integrate that in the region from x equals 0 to 1.
And then, we also have the second region which is X from 1 to 2.
In that region, the traditional mean squared error from part b is 1/12.
The marginal pdf of X is 2/3, and we do this integral.
And if you just carry out some calculus here, you'll get that the final answer is equal to 5/72.
Now, the last part of this question asks us, is this mean squared error the same thing-- does it equal the expectation of the conditional variance?
And it turns out that yes, it does.
And to see that, we can just take this, and use the law of iterated expectations, because iterated expectations tells us this is in fact equal to the expectation of Y minus g of X squared, given X.
That's just applying law of iterated expectations.
And then, if we look at this, this part that's inside is exactly equal to the conditional variance of Y, given X.
And so these two are, in fact, the same.
In part c, we calculated what the marginal pdf of X is, and it'll actually be used later on in this problem.
So for future reference, let's just write it down here in this corner.
Now, so far in this problem, we've looked at the LMS estimator.
And of course, there are other estimators that you can use as well.
And now in part d, let's look at the linear LMS estimator.
Now remember, the linear LMS estimator is special, because it forces the estimator to have a linear relationship.
So the estimator is going to be a linear function of X.
Now, compare that to what the LMS estimator was in this case.
It was two linear pieces, but there was a kink.
And so the entire estimator wasn't actually linear in X.
Now, the LLMS estimator, or the linear LMS estimator, will give us the linear estimator.
It's going to be a linear function of X.
And we know that we have a formula for this.
Is the expectation of Y plus the covariance of X and Y over the variance of X times X minus expectation of X.
All right, so in order to calculate what this is, we just need to calculate what four things are.
Now, let's start with this last one, the expected value of X.
To calculate the expected value of X, we just use a formula.
And from before, we know what the pdf of X is.
And so we know that this is just going to be X times fx of x dx.
And in particular, this will give us that from 0 to 1, it's going to be X times the pdf of X is 2/3 X, so it's 2/3 X squared.
And from 1 to 2, it's going to be X times the pdf of X, which is just 2/3, so it's 2/3 X dx.
And when you calculate this out, you'll get that is equal to 11/9.
Now, let's calculate the variance of X next.
In order to calculate that, let's use the formula that variance is equal to the expectation of X squared minus the expectation of X quantity squared.
We had the expectation of X, so let's calculate what the expectation of X squared is.
Now, it's the same idea.
Instead, we have X squared times f of X dx.
And we'll get the same sort of formula.
We'll split it up again into two different parts from X equals 0 to X equals 1.
It's going to be X squared times pdf, so it's 2/3 X cubed dx.
And then from X equals 1 to 2, it's going to be X squared times 2/3.
So it's 2/3 X squared dx.
And when we calculate this out, we'll get that it's equal to 31/18.
From that, we know that the variance is going to be equal to expectation of X squared minus expectation of X quantity squared.
Now, expectation of X squared is 31/18.
Expectation of X is 11/9.
And when we calculate this, we get that the variance is equal to 37/162.
So now we have this, and we have that.
Let's calculate what expectation of Y is.
Expectation of Y, let's calculate it using the law of iterated expectations.
The law of iterated expectations tells us that this is equal to the expectation of Y conditioned on X.
Now, we already know what expectation of Y conditioned on X is.
That was the LMS estimator that we calculated earlier.
It's this.
So now we just need to calculate this out, and we can do that.
So we know that in the range from X between 0 and 1, it's equal to 1/2 X.
So in the range from 0 to 1, it's equal to 1/2 X.
But then, we have to use total expectation, so we have to multiply by the pdf of X in that region which is 2/3 X dx.
And then in the range from X equals 1 to 2, this conditional expectation is X minus 1/2.
And the pdf of X in that region is 2/3.
Now, when we calculate out this value, we'll get that it's equal to 7/9.
And now, the last piece is the covariance of X and Y.
Remember, the covariance, we can calculate that as the expectation of X times Y minus the expectation of X times the expectation of Y.
We already know the expectation of X and the expectation of Y, so we just need to calculate the expectation of X times Y, the product of the two.
And for that, we'll use the definition, and we'll use the joint pdf that we have.
So this is going to be a double integral of X times Y times the joint pdf.
And the tricky part here is just figuring out what these limits are.
So we'll integrate in this order-- X and Y.
Now, let's split this up.
So let's focus on splitting X up.
So for X between 0 and 1, we just need to figure out what's the rate right range of Y to integrate over such that this is actually non-zero.
Because remember, the joint pdf is easy.
It's just a constant 2/3.
But it's only a constant 2/3 within this region.
So the difficult part is just figuring out what the limits are in order to specify that region.
So for X between 0 and 1, Y has to be between 0 and X, because this line is Y equals X.
So we need to integrate from 0 to X-- X times Y times the joint pdf, which is 2/3.
And now, let's do the other part, which is X from 1 to 2.
Well, if X is from 1 to 2, in order to fall into this region, Y has to be between X minus 1 and X.
So we integrate Y from X minus 1 to X.
Against, it's X times Y times the joint pdf, which is 2/3.
And now, once we have this set up, the rest of it we can just do some calculus.
And what we find is that the final answer is equal to 41/36.
Now, what that tells us is that the covariance of X and Y, which is just expectation of X times Y, the product, minus expectation of X times expectation of Y.
We know expectation of X times Y now.
It's 41/36.
Expectation of X is 11/9.
Expectation of Y is 7/9.
So when we substitute all of that in, we get that this covariance is 61/324.
All right, so now we have everything we need.
Expectation of Y is here.
Covariance is here.
Variance of X is here.
And expectation of X is here.
So let's substitute that in, and we can figure out what the actual LLMS estimator is.
So expectation of Y we know is 7/9.
Expectation of X is 11/9.
And when you divide the covariance, which is 61/324, by the variance, which is 37/162, we'll get 61/74.
And so that is the LLMS estimator that we calculated.
And notice that it is, in fact, linear in X.
So let's plot that and see what it looks like.
So it's going to be a line, and it's going to look like this.
So at X equals 2, it's actually a little bit below 1 and 1/2, which is what the LMS estimator would be.
At X equals 1, it's actually a little bit above 1/2, which is what the LMS estimator would be.
And then it crosses 0 around roughly 1/4, and it drops actually below 0.
So if we connect the dots, it's going to look something like this.
So notice that it's actually not too far away from the LMS estimator here.
But it doesn't have the kink because it is a line.
And note also that it actually drops below.
So when X is very small, you actually estimate negative values of Y, which is actually impossible, given the joint pdf distribution that we're given.
And that is sometimes a feature or artifact of the linear LMS estimator, that you'll get values that don't necessarily seem to make sense.
So now that we've calculated the linear LMS estimator in part d, which is this, and the LMS estimator in part a, which is this, we've also compared them visually.
The linear LMS estimator is the one in pink, the straight line.
And the LMS estimator is the one in black with the kink.
It's an interesting question to now ask, which one of these is better?
And in order to judge that, we need to come up with some sort of criterion to compare the two with.
And the one that we're going to look at in part e is the mean squared error.
Which one gives the lower mean squared error.
And so specifically, we're going to ask ourselves which of these two estimators gives us the smaller mean squared error?
Is it the linear LMS estimator given by l of X?
Or is it the LMS estimator, given by g of X?
Now, we know that the LMS estimator is the one that actually minimizes this.
The LMS estimator is designed to minimize the mean squared error.
And so we know that given any estimator of X, this one will have the smallest mean squared error.
And so the linear LMS estimator, its mean squared error has to be at least as large as the LMS estimators.
And the last part of the question now asks us to look at a third type of estimator, which is the MEP estimator.
Now, we want to ask, why is it that we haven't been using the MEP estimator in this problem?
Well, remember what the MEP estimator does.
In this case, what we would do is it would take the conditional distribution ratio of Y given any value of X.
And then it would pick the value of Y that gives the highest value in the conditional distribution.
And that would be the MEP estimate of Y.
But the problem in this case is that if you take any slice here, so a condition on any value of X, any of these slices, if you just take this out and look at it, it's going to be uniform.
This is what the conditional distribution of Y given X is.
It's going to be uniform between 0 and X.
Now, what the MEP rule tells us is we're going to pick the value of Y that gives us the highest point in this conditional distribution.
You can think of it as a posterior distribution.
Now, what's the problem here?
Well, every single point gives us exactly the same value for this conditional distribution.
And so there's no unique MEP rule.
Every single value of Y has just the same conditional distribution.
So there's no sensible way of choosing a value based on the MEP rule in this case.
But compare that with the LMS estimator, which is just get conditional expectation.
In that case, we can always find a conditional expectation.
In this case, the conditional expectation is the midpoint, which is X/2, just as had found in part a. OK, so in this problem, we reviewed a bunch of different ideas in terms of inference, and we took a joint pdf of X and Y, and we used that to calculate the LMS estimator, the linear LMS estimator.
We compared the two, and then we also looked at why in this case, the MEP estimator doesn't really make sense.
All right, so I hope that was helpful, and we'll see you next time.
Hi everyone.
Today I'm going to talk about Bernoulli process practice number one.
In this problem, you are visiting a rain forest.
But unfortunately you have run out of insect repellent.
As a result, the probability of you getting mosquito bites is really high.
At each second, the probability that a mosquito will land on your neck is 0.5.
If a mosquito lands on your neck, the probability that it will bite you is 0.2.
And the probability that it will never bother you is 0.8.
All of this happens independently among all mosquitoes.
For part A of the problem, we're interested in finding the expected value of the time between successive mosquito bites and the variance of the time between successive mosquito bites.
From the problem statement we know that the probability distributions of getting mosquito bites at different times are identically distributed and independent.
Therefore, the mosquito bites occur as a Bernoulli process with parameter p, where p represents the probability of getting a mosquito bite at each second.
And p can be calculated as the probability that a mosquito lands on your neck at each second multiplied by the probability that a mosquito will bite you, given that it has landed on your neck.
And this is equal to 0.5 times 0.2, which is equal to 0.1.
Next let us define x as the time between successive mosquito bites.
Because of the memory-less property of the Bernoulli process, which means the probability of getting mosquito bites at different times are independent, x is equivalent to the time until the next mosquito bite.
And x is a geometrical random variable whose PMF is like the following.
For all x, let's say equal to 0, the probabilities are equal to 0.
For x equal to 1, the probability that it takes 1 second to the next mosquito bite is simply equal to p. And for x equal to 2, the probability that it takes 2 seconds until the next mosquito bite is equal to 1 minus p times p. And for x equal to 3, the probability that it takes 3 seconds until the next mosquito bite is equal to 1 minus p to the power of 2 times p. Similarly, for x equal to k, the probability that it takes k seconds until the next mosquito bite is equal to 1 minus p to the power of k minus 1 times p. Therefore the expected value of x is equal to 1 over p, which is equal to 1 over 0.1, which is equal to 10.
And the variance of x is equal to 1 minus p over p squared, which is equal to 1 minus 0.1 over 0.1 squared, which is equal to 90.
For part B of the problem, we're considering another type of bug.
Similar to the case as the mosquitoes, here at each second the probability that a tick will land on your neck is equal to 0.1.
And if a tick lands on your neck, the probability that it will bite you is equal to 0.7.
And the probability that it will never bother you is equal to 0.3.
And all this happens independently among all ticks and all mosquitoes.
So similar to the case as part A, where mosquito bites occurs as a Bernoulli process with parameter p equal to 0.1, here the tick bites also across a Bernoulli process with parameter q equal to 0.1 times 0.7, which is equal to 0.07.
And q is the probability of getting a tick bite at each second.
Therefore, the bug bites occurs as a merged process from the mosquito bites and the tick bites.
And let r represent the parameter for the bug bites.
So here r is equal to the probability of getting either a mosquito bite or a tick bite.
And this is equivalent to 1 minus the probability of getting no mosquito bite and no tick bite.
Because the mosquito bites and the tick bites happens independently, therefore this can be written as 1 minus the probability of no mosquito bites times the probability of no tick bites at each second.
And this is equal to 1 minus p times 1 minus q, which is p plus q minus pq, which is equal to 0.1 plus 0.7 minus 0.1 times 0.7.
Which is equal to 0.163.
Next, let us define y as the time between successive bug bites.
So similar as x in part a, here y is a geometric distribution with parameter r. And therefore the expected value of y is equal to 1 over r, which is equal to 1 over 0.163.
That is approximately 6.135.
And the variance of y is equal to 1 minus r over r squared, which is equal to 1 minus 0.163 over 0.163 squared.
And this is approximately 31.503.
So this gives us the expected value of the time between successive bug bites and the variance of the time between successive bug bites.
And this concludes our two days practice on Bernoulli process.
Hi.
In this problem, we'll get some practice working with PDFs and also using PDFs to calculate CDFs.
So the PDF that we're given in this problem is here.
So we have a random variable, z, which is a continuous random variable.
And we're told that the PDF of this random variable, z, is given by gamma times 1 plus z squared in the range of z between negative 2 and 1.
And outside of this range, it's 0.
All right, so first thing we need to do and the first part of this problem is we need to figure out what gamma is because it's not really a fully specified PDF yet.
We need to figure out exactly what the value gamma is.
And how do we do that?
Well, we've done analogous things before for the discrete case.
So the tool that we use is that the PDF must integrate to 1.
So in the discrete case, the analogy was that the PMF had to sum to 1.
So what do we know?
We know that when you integrate this PDF from negative infinity to infinity, fz of z, it has to equal 1.
All right, so what do we do now?
Well, we know what the PDF is-- partially, except for gamma-- so let's plug that in.
And the first thing that we'll do is we'll simplify this because we know that the PDF is actually only non-zero in the range negative 2 to 1.
So instead of integrating from negative infinity to infinity, we'll just integrate from negative 2 to 1.
And now let's plug in this gamma times 1 plus z squared dc.
And now the rest of the problem is just applying calculus and integrating this.
So let's just go through that process.
So we get z plus 1/3 z cubed from minus 2 to 1.
And now we'll plug in the limits.
And we get gamma, and that's 1 plus 1/3 minus minus 2 plus 1/3 times minus 2 cubed.
And then if we add this all up, you get 4/3 plus 2 plus 8/3, which will give you 6.
So what we end up with in the end is that 1 is equal to 6 gamma.
So what does that tell us?
That tells us that, in this case, gamma is 1/6.
OK, so we've actually figured out what this PDF really is.
And let's just substitute that in.
So we know what gamma is.
So it's 1/6.
So from this PDF, we can calculate anything that we want to.
This PDF, basically, fully specifies everything that we need to know about this random variable, z.
And one of the things that we can calculate from the PDF is the CDF.
So the next part of the problem asks us to calculate the CDF.
So remember the CDF, we use capital F. And the definition is that you integrate from negative infinity to this z.
And what do you integrate?
You integrate the PDF.
And all use some dummy variable, y, here in the integration.
So what is it really doing?
It's basically just taking the PDF and taking everything to the left of it.
So another way to think about this-- this is the probability that the random variable is less than or equal to some little z.
It's just accumulating probability as you go from left to right.
So the hardest part about calculating the CDFs, really, is actually just keeping track of the ranges, because unless the PDF is really simple, you'll have cases where the PDF cold be 0 in some ranges and non-zero in other ranges.
And then what you really have to keep track of is where those ranges are and where you actually have non-zero probability.
So in this case, we actually break things down into three different ranges because this PDF actually looks something like this.
So it's non-zero between negative 2 and 1, and it's 0 everywhere else.
So then what that means is that our job is a little simpler because everything to the left of negative 2, the CDF will be 0 because there's no probability density to the left.
And then everything to the right of 1, well we've accumulated all the probability in the PDF because we know that when you integrate from negative 2 to 1, you capture everything.
So anything to the right of 1, the CDF will be 1.
So the only hard part is calculating what the CDF is in this intermediate range, between negative 2 and 1.
So let's do that case first-- so the case of z is between negative 2 and 1.
So what is the CDF in that case?
Well, the definition is to integrate from negative infinity to z.
But we know that everything to the left of negative 2, there's no probably density.
So we don't need to include that.
So we can actually change this lower limit to negative 2.
And the upper limit is wherever this z is.
So that becomes our integral.
And the inside is still the PDF.
So let's just plug that in.
We know that it's 1/6 1 plus-- we'll make this y squared-- by.
And now it's just calculus again.
And in fact, it's more or less the same integral, so what we get is y plus 1/3 y cubed from negative 2 to z.
Notice the only thing that's different here is that we're integrating from negative 2 to z instead of negative 2 to 1.
And when we calculate this out, what we get is z plus 1/3 z cubed minus minus 2 plus 1/3 minus 2 cubed, which gives us 1/6 z plus 1/3 z cubed plus plus 2 plus 8/3 gives us 14/3.
So that actually is our CDF between the range of negative 2 to 1.
So for full completeness, let's actually write out the entire CDF, because there's two other parts in the CDF.
So the first part is that it's 0 if z is less than negative 2.
And it's 1 if z is greater than 1.
And in between, it's this thing that we've just calculated.
So it's 1/6 z plus 1/3 z cubed plus 14/3 if z is between minus 2 and 1.
So that is our final answer.
So the main point of this problem was to drill a little bit more the concepts of PDFs and CDFs.
So for the PDF, the important thing to remember is that in order to be a valid PDF, the PDF has to integrate to 1.
And you can use that fact to help you calculate any unknown constants in the PDF.
And then to calculate the CDF, it's just integrating the PDF from negative infinity to whatever point that you want to cut off at.
And the tricky part, as I said earlier, was really just keeping track of the ranges.
In this case, we've broke it down into three ranges.
If we had a slightly more complicated PDF, then you would have to keep track of even more ranged.
All right, so I hope that was helpful, and we'll see you next time.
Hi.
In this problem, we'll be going over practice with the calculation of conditional probabilities.
We'll start with a game where our friend Alice will be tossing a coin with certain bias of having a head, and tosses this coin twice.
And we're interested in knowing, what's the probability that both coin tosses will end up being a head?
The first step we're going to do is to convert the problem into a mathematical form by defining two events as the following.
Event A is where the first coin toss is a head.
And similarly, event B will be having the second coin toss also being a head.
Having these two events will allow us to say, well, the event that A intersection B will be the event that both coin tosses are a head.
And we'd like to know the probability of such an event.
In particular, the probability of A and B will be calculated under two types of information.
In the first case, we'll be conditioning on that we know the first coin toss is a head.
I'd like to know what the probability of A and B is.
In the second case, we know that at least one of the two coin tosses is a head expressed in the form A union B.
And under this conditioning, what is the probability of A and B, A intersection B?
So Alice, in this problem, says-- well, her guess will be that the first quantity is no smaller than the second quantity.
Namely, knowing that the first coin toss is a head somehow more strongly implies that both coin tosses will be a head, compared to the case that we only know at least one of the two coin tosses is a head.
And we'd like to verify if this inequality is indeed true.
To do so, let's just use the basic calculation of conditional probability.
Now, from the lectures, you've already learned that to calculate this quantity, we'll write out a fraction where the numerator is the probability of the intersection of these two events.
So we have A intersect B intersection A divided by the probability of the event that we're conditioning on, which is A.
Now, the top quantity, since we know that A and B is a subset of event A, then taking the intersection of these two quantities will just give us the first event.
So we have A and B.
And the bottom is still probability of A.
Let's do the same thing for the second quantity here.
We have the top probability of A and B intersection the event A union B, and on the bottom, probability of the event A and B.
Again, we see the event A and B is a subset of the event A union B.
So the top will be A and B.
And the bottom-- A union B. OK, now let's stop for a little bit.
We've computed the probability for each expression in the following fractional form.
And we observed that for both fractions, the numerator is the same.
So the numerator is a probability of A and B.
And the denominator in the first case is probably of A, and the second case, probably of A union B.
Since we know that A is a subset of the event A union B, and by the monotonicity of probabilities, we know that the probability of A is hence no greater than a probability of A union B.
Substituting this back into these expressions, we know that because they lie in the denominators, the first expression is indeed no smaller than the second expression.
So our friend Alice was correct.
So throughout this problem, we never used the fact that the probability of a particular coin toss results, let's say, in a head is a certain number.
Actually, this bias for the coin is irrelevant.
Whether the coin is fair or unfair, this fact is always true.
So indeed, it does not depend on the probability of the coin.
But if you're really curious what happens when the coin is fair, we can plug in the numbers.
And here, we're assuming the coin is fair, which means probability of having a head is 1/2.
Then, we'll see after going through the calculations that the first probability is 1/2, whereas the second probability is 1/3, which means, in this case, the [?
dominance ?]
actually is strict.
So the first one is strictly greater than the second one, OK?
So this completes the first part of the problem.
How do we generalize this into more general settings?
There are multiple ways, but we'll go over one particular form.
And to do so, we'll be defining three events somewhat more abstractly.
Let's say we have three events-- C, D, and E. Imagine any event, but all three events have to satisfy the following condition.
First, event D will be a subset of E. And second, the intersection of C and D is equal to the intersection of C and E, OK?
So this will be our choice events.
And let's see a particular example.
Let's say you have a sample space here and some event E. Now, by the first condition, D will have to lie somewhere in E. For the second condition, we'll pick some event C such that this is true.
And one way to do so is simply picking C that lies within both D and E. And you can see C intersection D will be C. And C intersection E will still be C. Hence, the second equality is true.
So if both equalities are true, we have the following relationship, that the probability of C conditional on D will be no smaller than the probability of C conditional on event E. And this will be the more general form of the inequality that we saw before.
So first of all, the way to prove this is in fact the same.
We simply write out the value of this using the fractional form.
And based on these two facts, we can arrive at this equation, which I shall now go over.
But just to see why this form is more general, in fact, if we-- say we let C be the event A intersection B, D be the event A, and E be the event A and B where A and B are the events that we defined earlier.
We can verify that, indeed, these conditions are true, namely D is a subset of E. Because A is a subset of A union B, and C is a subset of both D and E. And hence, condition two is also true.
And if that's the case, we will actually recover the result we got earlier for events A and B.
And hence, this equation here is a more general form.
So that's the end of the problem.
See you next time.
In this exercise, we'll be looking at a problem, also know as the coupons collector's problem.
We have a set of K coupons, or grades in our case.
And each time slot we're revealed with one random grade.
And we'd like to know how long it would take for us to collect all K grades.
In our case, K is equal to 6.
Now the key to solving the problem is essentially twofolds.
First, we'll have to find a way to intelligently define sequence random variables that captured, essentially, stopping time of this process.
And then we'll employ the idea of linearity of expectations in breaking down this value in simpler terms.
So let's get started.
We'll define Yi as the number of papers till we see the i-th new grade.
What does that mean?
Well, let's take a look at an example.
Suppose, here we have a timeline from no paper yet, first paper, second paper, third paper, so on, and so forth.
Now, if we got grade A on the first slot, grade A minus on second slot, A again on the third slot, let's say there's a fourth's slot, we got B.
According to this process, we see that Y1 is always 1, because whatever we got on the first slot will be a new grade.
Now, Y2 is 2, because the second paper is, again, a new grade.
On the third paper we got a grade, which is the same as the first grade.
So that would not count as any Yi.
And the third time we saw new grade would now be paper four.
According to this notation, we're interested in knowing what is the expected value of E of Y6, which is the time it takes to receive all six grades.
So so far this notation isn't really helping us in solving the problem, but kind of just staying a different way.
It turns out, it's much easier to look at the following variable derived from the Yis.
We'll define Xi as the difference between Yi plus 1 minus Yi.
And in [?
words, ?]
it says, Xi is a number of papers you need until you see the i plus 1-th new grade, after you have received i new grade so far.
So in this case, X1 will be if we call 0, Y0, will be the difference between Y1 and Y0, which is always 1-- that's X1.
And the difference between these two will be X2.
And the difference between Y3 and Y2-- Sorry.
It should be Y X0, 1, 2, and so on.
OK?
Through this notation we see that Y6 now can be written as the summation of i equal to 0, 2, 5, X, i.
So all I did was to break down i6 into a sequence of summation of the differences, like Y.
6 Minus Y5, Y5 minus Y4, and so on.
It turns out, this expression will be very useful.
OK.
So now that we have the two variables Y and X, let's see if it will be easier to look at the distribution of X in studying this process.
Let's say, we have seen a new grade so far-- one.
How many trials would it take for us to see the second new grade?
It turns out it's not that hard.
In this case, we know there is a total of six grades, and we have seen one of them.
So that leaves us five more grades that we'll potentially see.
And therefore, on any random trial after that, there is a probability of 5 over 6 that we'll see a new grade.
And hence, we know that X1 has a distribution geometric with a success probability, or a parameter, 5/6.
Now, more generally, if we extend this idea further, we see that Xi will have a geometric distribution of parameter 6 minus i over 6.
And this is due to the fact that so far we have already seen i new grades.
And that will be the success probability of seeing a further new grade.
So from this expression, we know that the expected value of Xi will simply be the inverse of the parameter of the geometric distribution, which is 6 over 6 minus i or 6 times 1 over 6 minus i.
And now we're ready to compute a final answer.
So from this expression we know expected value of Y6 is equal to the expected value of sum of i equal to 0 to 5 Xi.
And by the linearity of expectation, we can pull out the sum and write it as 2, 5 expected value of Xi.
Now, since we know that expective of Xi is the following expression.
We see that this term is equal to 6 times expected value of i equals 0, 5, 1 over 6 minus i.
Or written in the other way this is equal to 6 times i equal to 0, 2, 5.
In fact, 1, 2, 5, 1 over i.
And all I did here was to, essentially, change the variable, so that these two summations contained exactly the same terms.
And this will give us the answer, which is 14.7.
Now, more generally, we can see that there's nothing special about number 6 here.
We could have substituted 6 with a number, let's say, K. And then we'll get E of YK, let's say, there's more than six labels.
And this will give us K times summation i equal to 1, so K minus 1, 1 over i. Interestingly, it turns out this quantity has an [?
asymptotic ?]
expression that, essentially, is roughly equal to K times the natural logarithm of K. And this is known as the scaling [?
la ?]
for the coupon collector's problem that says, essentially, takes about K times [?
la ?]
K many trials until we collect all K coupons.
And that'll be the end of the problem.
See you next time.
In this problem, we'll be working with a object called random walk, where we have a person on the line-- or a tight rope, according to the problem.
Let's start from the origin, and each time step, it would randomly either go forward or backward with certain probability.
In our case, with probability P, the person would go forward, and 1 minus P going backwards.
Now, the walk is random in the following sense-- that the choice going forward or backward in each step is random, and it's completely independent from all past history.
So let's look at the problem.
It has three parts.
In the first part, we'd like to know what's the probability that after two steps the person returns to the starting point, which in this case is 0?
Now, throughout this problem, I'm going to be using the following notation.
F indicates the action of going forward and B indicates the action of going backwards.
A sequence says F and B implies the sample that the person first goes forward, and then backwards.
If I add another F, it will mean, forward, backward, forward again.
OK?
So in order for the person to go somewhere after two steps and return to the origin, the following must happen.
Either the person went forward followed by backward, or backward followed by forward.
And indeed, this event-- namely, the union of these two possibilities-- defines the event of interest in our case.
And we'd like to know what's the probability of A, which we'll break down into the probability of forward, backward, backward, forward.
Now, forward, backward and backward, forward-- they are two completely different outcomes.
And we know that because they're disjoint, this would just be the sum of the two probabilities-- plus probability of backward/forward.
Here's where the independence will come in.
When we try to compute the probability of going forward and backward, because the action-- each step is completely independent from the past, we know this is the same as saying, in the first step, we have probability P of going forward, in the next step, probability 1 minus P of going backwards.
We can do so-- namely, writing the probability of forward, backward as a product of going forward times the probability of going backwards, because these actions are independent.
And similarly, for the second one, we have going backwards first, times going forward the second time.
Adding these two up, we have 2 times P times 1 minus P. And that will be the answer to the first part of the problem.
In the second part of the problem, we're interested in the probability that after three steps, the person ends up in position 1, or one step forward compared to where he started.
Now, the only possibilities here are that among the three steps, exactly two steps are forward, and one step is backwards, because otherwise there's no way the person will end up in position 1.
To do so, there, again, are three possibilities in which we go forward, forward, backward, or forward, backward, forward, or backward, forward, forward.
And that exhausts all the possibilities that the person can end up in position 1 after three steps.
And we'll define the collection of all these outcomes as event C. The probability of event C-- same as before-- is simply the sum of the probability of each individual outcome.
Now, based on the independence assumption that we used before, each outcome here has the same probability, which is equal to P squared times 1 minus P. The P squared comes from the fact that two forward steps are taken, and 1 minus P, the probability of that one backwards step.
And since there are three of them, we multiply 3 in front, and that will give us the probability.
In the last part of the problem, we're asked to compute that, conditional on event C already took place, what is the probability that the first step he took was a forward step?
Without going into the details, let's take a look at the C, in which we have three elements, and only the first two elements correspond to a forward step in the first step.
So we can define event D as simply the first two outcomes-- forward, forward, backward, and forward, backward, forward.
Now, the probability we're interested in is simply probability of D conditional on C. We'd write it out using the law of conditional probability-- D intersection C conditional on C. Now, because D is a subset of C, we have probability of D divided by the probability of C. Again, because all samples here have the same probability, all we need to do is to count the number of samples here, which is 2, and divide by the number of samples here, which is 3.
So we end up with 2 over 3.
And that concludes the problem.
See you next time.
Hi.
In this problem, we're going to use the set of probability axioms to derive the probability of the difference of two events.
Now, before we get started, there's one thing you might notice that, the equation we're trying to prove is actually quite complicated.
And I don't like it either, so the first thing we're going to do will be to find a simpler notation for the events that we're interested in.
So we start with two events, A and B, and there might be some intersection between the two events.
We'll label the set of points or samples in A that are not in B, as a set C. So C will be A intersection B complement.
Similarly, for all points that are in B but not in A, this area, we'll call it D. And D will be the set A complement intersection B.
And finally, for points that are in the intersection of A and B, we'll call it E. So E is A intersection B.
And for the rest of our problem, we're going to be using the notation C, D, and E instead of whatever's down below.
If we use this notation, we can rewrite our objective as the following.
We want to show that the probability of C union D is equal to the probability of the event A plus the probability of B minus twice the probability of E. And that will be our goal for the problem.
Now, let's take a minute to review what the axioms are, what the probability axioms are.
The first one says non-negativity.
We take any event A, then the probability of A must be at least 0.
The second normalization says the probability of the entire space, the entire sample space omega, must be equal to 1.
And finally, the additivity axiom, which will be the axiom that we're going to use for this problem says, if there are two events, A and B that are disjoint-- which means they don't have anything in common, therefore.
the intersection is the empty set.
Then the probability of their union will be equal to the probably A plus the probability of B.
For the rest of the problem, I will refer to this axiom as add.
So whenever we invoke this axiom, I'll write "add" on the board.
Let's get started.
First, we'll invoke the additivity axioms to argue that the probability of C union D is simply the sum of probability of C plus probability of D. Why is this true?
We can apply this axiom, because the set C here and the set D here, they're completely disjoint from each other.
And in a similar way, we'll also notice the following.
We see that A is equal to the union of the set C and E. And also, C and E, they're disjoint with each other, because C and E by definition don't share any points.
And therefore, we have probably A is equal to probability of C plus the probability of E. Now, in a similar way, the probability of event B can also be written as a probability of D plus the probability of E, because event B is the union of D and E. And D and E are disjoint from each other.
So we again invoke the additivity axiom.
Now, this should be enough to prove our final claim.
We have the probability of C union D. By the very first line, we see this is simply probability of C plus the probability of D. Now, I'm going to insert two terms here to make the connection with a second part of the equation more obvious.
That is, I will write probability C plus probability E plus probability D plus probability of E. Now, I've just added two terms here-- probability E. So to make the equality valid or subtract it out two times, the probability of E. Hence this equality is valid.
So if we look at this equation, we see that there are two parts here that we've already seen before right here.
The very first parenthesis is equal to the probability of A.
And the value of the second parenthesis is equal to the probability of B.
We just derived these here.
And finally, we have the minus 2 probability of E. This line plus this line gives us the final equation.
And that will be the answer for the problem.
Hi, in this problem, we're going to look at competing exponential.
So we have three exponential random variables, X with parameter lambda, Y with parameters mu, and Z with parameter nu.
And we want to calculate some probability.
And the probability that we want to calculate is the probability that X is less than Y is less than Z.
Now we can reinterpret this as 3 plus Poisson processes.
Because the link between exponentials and Poisson processes is that the inter-arrival time of Poisson processes are exponentially distributed.
So you can think of X being the time until the first arrival in a Poisson process with parameter lambda.
And same thing for Y is the first arrival time of a Poisson process with parameter mu.
The same thing for Z and nu.
And so in that interpretation, X less than Y less than Z, you could interpret as a race, meaning that X finishes first followed by Y and then Z comes in last.
So with that interpretation, let's see if we can calculate what this probability is.
We can rewrite this probability as a combination of two things occurring.
One is that X is less than the minimum of Y and Z.
And then the other is that Y is less than Z.
So what is this first event mean?
This first event means that X comes in first.
And it doesn't matter whether Y comes in second or Z comes in second.
So we first say that X has to come in first which means it has to beat the better of Y and Z.
And then that combined with the fact that Y does better than Z is the same thing as saying that X is first, Y is second, and Z is third.
And now, let's try to argue using Poisson processes, that these two events are actually independent.
So this event occurring means that X is smaller than Y and Z.
So let's take these Poisson processes, and because these random variables are assumed to be independent, these are independent Poisson processes.
So we can merge them.
So let's merge these two.
And we'll get a Poisson process that has rate mu plus nu.
And we can also merge this first one and that one.
And we'll get another Poisson process with predator lambda plus mu plus nu.
So in that context, what does it mean that X is less than the minimum Y and Z?
It just means that in this merged process, the first arrival came from the X process.
In that case, if that's true, then X is less than minimum Y and Z.
Well, let's say that event does occur that the first arrival is from the X process.
Now we're interested in what the order of the second two arrivals are.
Is it Y first and then Z?
Or Z first and then Y?
Well, it doesn't matter because of the fresh start property.
Because after this arrival comes, and say it is from the X process, the Poisson processes start anew, and they're still independent.
And so what happens after that is independent of what happened here, when X arrived.
And so whether Y came first followed by Z, or Z came first followed by Y is independent of what happened here.
And so because of that, these two events are independent, and so when we have the probability of the intersection of two independent events, we can write that as the product of those two probabilities.
Now, what is the probability of this first event?
The probability that X is less than the minimum Y and Z?
Well, we just said that that corresponds to the first arrival of this merge process being from the X process.
Well, that probability is lambda over lambda plus mu plus nu.
So it's equal to this ratio where the process that you're interested in, its rate comes in the numerator.
And then the merged rate is on the denominator, And what about the second one?
What's the probability that Y is less than Z?
Well, let's go now to this merge process where we merged just the Y and Z processes and see which one comes first.
Well, in that case what we want to know is in this merge process, what is the probability that the first arrival came from the Y process?
Well, analogously, that probability is going to mu over mu plus nu.
And that gives us our answer.
And so we see that what looked like a pretty complex calculation when we reinterpreted it in terms of Poisson processes, it becomes relatively easy to solve.
But this still seems like a complicated expression.
So let's try to check to see whether it actually makes sense.
So one way to do that is to look at a specific example of the choice of lambda, mu, and nu, and see if it actually makes sense.
So one example is suppose that all three of these parameters are the same.
Well, if they're all the same then this probability, the first part becomes 1 over 3, 1/3.
And the second one is 1 over 2.
And so if all three parameters are the same, probability becomes 1/6.
And let's see if that makes sense.
If all three parameters are the same, that means that these rates, these arrival rates are all the same.
And what that means is that any three ordering of these three arrivals is as likely as any other ordering.
And what we're interested in is the probability of one's particular ordering happening which is X first then Y then Z.
But if everything is symmetric then any of the orderings is as likely as any other one.
And how many orderings are there?
Well, there's three choices for who comes in first.
Two for who comes in second.
And one for who comes in last.
So there's a total of six possible orders in which these three contestants, if you think of it that way, could finish this race.
And out of none of those, we want the probability that one of those outcomes happens.
And so the probability should be 1/6.
And that's what our formula tells us.
So as I said, in this problem, we saw how to reinterpret exponentials in the context of Poisson processes that helped us solve a--
Hi.
Today we're going to do another fun problem that involves rolling two dice.
So if you guys happen to frequent casinos, this problem might be really useful for you.
I'm just kidding.
But in all seriousness, this problem is a good problem, because it's going to remind us how and when to use the discrete uniform law.
Don't worry, I'll review what that says.
And it's also going to exercise your understanding of conditional probability.
So quick recap.
The discrete uniform law says that when your sample space is discrete, and when the outcomes in your sample space are equally likely, then to compute the probability of any event A, you can simply count the number of outcomes in A and divide it by the total number of possible outcomes.
OK, so coming back to our problem.
The problem statement tells us that we roll two fair six-sided die.
And it also tells us that each one of the 36 possible outcomes is assumed to be equally likely.
So you know alarm bell should be going off in your head.
Our sample space is clearly discrete.
And it says explicitly that all outcomes are equally likely.
So clearly, we can use the discrete uniform law.
And again, this is helpful because it reduces a problem of computing probabilities to a problem of counting.
OK, and before we go any further, I just want to review what this graph is plotting.
You've seen it a few times, but just to clarify, on one axis, we're plotting the outcome of the first die roll, and on the second axis, we're plotting the outcome of the second die roll.
So if you got a 4 on your first die, and you get a 1 on your second die, that corresponds to this point over 4 and up 1.
OK, so part a asks us to find the probability that doubles our rolls.
So let's use some shorthand.
We're going to let D be the event that doubles are rolled.
And we want to compute the probability of D. I argue before we can use the discrete uniform law.
So if we apply that, we just get the number of outcomes that comprise the event "doubles rolled" divided by 36, because there are 36 possible outcomes, which you can see just by counting the dots in this graph.
Six possible outcomes for the first die, six possible outcomes for the second die.
That's how you-- 6 times 6 is 36.
So I've been assuming this entire time that you know what doubles are.
For those of you who don't know, doubles is essentially when that number on the first die matches the number on the second die.
So this outcome here 1-1 is part of the event "doubles rolled."
Similarly, 2-2, 3-3, 4-4, 5-5, and 6-6-- these six points comprise the event "doubles rolled."
So we can go ahead and put 6 over 36, which is equal to 1/6.
So we're done with part a.
We haven't seen any conditioning yet.
The conditioning comes in part b.
So in part b we're still interested in the event D, in the event that "doubles are rolled."
But now we want to compute this probability conditioned on the event that the sum of the results is less than or equal to 4.
So I'm going to use this shorthand sum less than or equal to 4 to denote the event that the role results in the sum of 4 or smaller.
So there's two ways we're going to go about solving part b.
Let's just jump right into the first way.
The first way is applying the definition of conditional probability.
So hopefully you remember that this is just probability of D intersect sum less than or equal to 4, divided by probability of sum less than or equal to 4.
Now, sum less than or equal to 4 and D intersect sum less than or equal to 4 are just two events.
And so we can apply the discrete uniform law to calculate both the numerator and the denominator.
So let's start with the denominator first because it seems a little bit easier.
So sum less than or equal to 4, let's figure this out.
Well, 1-1 gives us a sum of 2, that's less than or equal to 4.
2-1 gives us 3.
3-1 gives us 4.
4-1 gives us 5, so we don't want to include this or this, or this point.
And you can sort of convince yourself that the next point we want to include is this one.
That corresponds to 2-2, which is 4, so it makes sense that these guys should form the boundary, because all dots sort of up and to the right will have a bigger sum.
3-1 gives us 4.
And 1-2 gives us 3.
So these six points-- 1, 2, 3, 4, 5, 6-- are the outcomes that comprise the event sum less than or equal to 4.
So we can go ahead and write in the denominator, 6 over 36, because we just counted the outcomes in sum less than or equal to, 4 and divided it by the number of outcomes in omega.
Now, let's compute the numerator.
D intersect sum less than or equal to 4.
So we already found the blue check marks.
Those correspond to sum less than or equal to 4.
Out of the points that have blue check marks, which one correspond to doubles?
Well, they're actually already circled.
It's just these two points.
So we don't even need to circle those, so we get 2 over 36, using the discrete uniform law.
And you see that these two 36s cancel each other.
So you just get 2/6 or 1/3.
So that is one way of solving part b, but I want to take you, guys, through a different way, which I think is important, and that make sure you really understand what conditioning means.
So another way that you can solve part b is to say, OK, we are now in the universe, we are in the conditional universe, where we know the sum of our results is 4 or smaller.
And so that means our new sample space is really just this set of six points.
And one thing that it's worth noting is that conditioning never changes the relative frequencies or relative likelihoods of the different outcomes.
So because all outcomes were equally likely in our original sample space omega, in the conditional worlds the outcomes are also equally likely.
So using that argument, we could say that in our sort of blue conditional universe all of the outcomes are equally likely.
And therefore, we can apply a conditional version of the discrete uniform law.
So namely, to compute the probability of some event in that conditional world.
So the conditional probability that "doubles are rolled", we need only count the number of outcomes in that event and divide it by the total number of outcomes.
So in the conditional world, there's only two outcomes that comprise the event "doubles rolled."
These are the only two circles in the blue region, right?
So applying the conditional version number law, we have two.
And then we need to divide by the size of omega.
So our conditional universe, we've already said, has six possible dots.
So we just divide by 6, and you see that we get the same answer of 1/3.
And so again, we used two different strategies.
I happen to prefer the second one, because it's slightly faster and it makes you think about what does conditioning really mean.
Conditioning means you're now restricting your attention to a conditional universe.
And given that you're in this conditional universe where the sum was less than or equal to 4, what is then the probability that doubles also happened?
OK, hopefully you, guys, are following.
Let's move on to part c. So part c asks for the probability that at least one die roll is a 6.
So I'm going to use the letter S to denote this, the probability that at least one die roll is a 6.
So let's go back to our picture and we'll use a green marker.
So hopefully you agree that anything in this column corresponds to at least one 6.
So this point, this point, this point, this point, this point, and this point your first die landed on a 6, so at least one 6 is satisfied.
Similarly, if your second die has a 6, then we're also OK.
So I claim we want to look at these 11 points.
Let me just check that, yeah, 6 plus 5-- 11.
So using the discrete uniform law again, we get 11 divided by 36.
OK, last problem, we're almost done.
So again, we're interested in the event S again, so the event that at least one die roll is a 6.
But now we want to compute the probability of that event in the conditional world where the two dice land on different numbers.
So I'm going to call this probability of S. Let's see, I'm running out of letters.
Let's for lack of a better letter, my name is Katie, so we'll just use a K. We want to compute the probability of S given K. And instead of using the definition of conditional probability, like we did back in part b, we're going to use the faster route.
So essentially, we're going to find the number of outcomes in the conditional world.
And then we're also going to compute the number of outcomes that comprise S in the conditional world.
So let's take a look at this.
We are conditioning on the event that the two dice land on different numbers.
So hopefully you agree with me that every single dot that is not on the diagonal, so every single dot that doesn't correspond to doubles, is a dot that we care about.
So our conditional universe of that the two dice land on "different numbers", that corresponds to these dots.
And it corresponds to these dots.
I don't want to get this one.
OK, that's good.
So let's see, how many outcomes do we have in our conditional world?
And I'm sorry I don't know why I didn't include this.
This is absolutely included.
I'm just testing to see if you, guys, are paying attention.
So we counted before that there are six dots on the diagonal, and we know that there are 36 dots total.
So the number of dots, or outcomes to use the proper word, in our conditional world is 36 minus 6, or 30.
So we get a 30 on the denominator.
And now we're sort of using a conditional version of our discrete uniform law, again.
And the reason why we can do this is, as I argued before, that conditioning doesn't change the relative frequency of the outcomes.
So in this conditional world, all of the outcomes are still equally likely, hence we can apply this law again.
So now we need to count the number of outcomes that are in the orange conditional world, but that also satisfy at least one die roll is a 6.
So you can see-- 1-- we just need to count the green circles that are also in the orange.
So that's 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.
So we get a 10, so our answer is 10 over 30, or 1/3.
So now we're done with this problem.
As you see, hopefully, it wasn't too painful.
And what are the important takeaways here for this problem?
Well, one is that whenever you have a discrete sample space, in which all of outcomes are equally likely, you should think about using the discrete uniform law, because this law lets you reduce the problem from computing probabilities to just counting outcomes within events.
And the second takeaway is the way we thought about conditioning.
So we talked about one thing, which is that in your conditional world, when you condition, the relative likelihoods of the various outcomes don't change.
So in our original universe, all of the outcomes were equally likely.
So in our conditional universe, all of the outcomes are equally likely.
And we saw it was much faster to apply a conditional version of the discrete uniform law.
So that's it for today.
And we'll do more problems next time.
Hi.
In this problem we'll work through an example of calculating a distribution for a minute variable using the method of derived distributions.
So in general, the process goes as follows.
We know the distribution for some random variable X and what we want is the distribution for another random variable of Y, which is somehow related to X through some function g. So Y is a g of X.
And the steps that we follow-- we can actually just kind of summarize them using this four steps.
The first step is to write out the CDF of Y.
So Y is thing that we want.
And what we'll do is we'll write out the CDF first.
So remember the CDF is just capital F of y, y is the probability that random variable Y is less than or equal to some value, little y.
The next thing we'll do is, we'll use this relationship that we know, between Y and X.
And we'll substitute in, instead of writing the random variable Y In here, we'll write it in terms of X.
So we'll plug in for-- instead of Y, we'll plug-in X.
And we'll use this function g in order to do that.
So what we have now is that up to here, we would have that the CDF of Y is now the probability that the random variable X is less than or equal to some value, little y.
Next what we'll do is we'll actually rewrite this probability as a CDF of X.
So the CDF of X, remember, would be-- F of x is that the probability of X is less than or equal to some little x.
And then once we have that, if we differentiate this-- when we differentiate the CDF of X, we get the PDF of X.
And what we presume is that we know this PDF already.
And from that, what we get is, when we differentiate this thing, we get the PDF of Y.
So through this whole process what we get is, we'll get the relationship between the PDF of Y and the PDF of X.
So that is the process for calculating the PDF of Y using X.
So let's go into our specific example.
In this case, what we're told is that X, the one that we know, is a standard normal random variable.
Meaning that it's mean 0 and variance 1.
And so we know the form of the PDF.
The PDF of x is this, 1 over square root of 2 pi e to the minus x squared over 2.
And then the next thing that we're told is this relationship between X and Y.
So what we're told is, if X is negative, then Y is minus X.
If X is positive, then Y is the square root of X.
So this is a graphical its representation of the relationship between X and Y.
All right, so we have everything that we need.
And now let's just go through this process and calculate what the PDF of Y is.
So the first thing we do is we write out the PDF of Y.
So the PDF of Y is what we've written.
It's the probability that the random variable Y is less than or equal to some little y.
Now the next step that we do is we have to substitute in, instead of in terms of Y, we want to substitute it in terms of X.
Because we actually know stuff about X, but we don't know anything about Y.
So what is the probability that Y, the random variable Y, is less than or equal to some little y?
Well, let's go back to this relationship and see if we can figure that out.
So let's pretend that here is our little y.
Well, if the random variable Y is less than or equal to little y, it has to be underneath this horizontal line.
And in order for it to be underneath this horizontal line, that means that X has to be between this range.
And what is this range?
This range goes from minus Y to Y squared.
So why is that?
It's because in this portion X and Y are related as, Y is negative X and here it's Y is square root of X.
So if X is Y squared, then Y would be Y.
If X is negative Y, then Y would be Y.
All right, so this is the range that we're looking for.
So if Y, the random variable Y is less than or equal to little y, then this is the same as if the random variable X is between negative Y and Y squared.
So let's plug that in.
This is the same as the probability that X is between negative Y and Y squared.
So those are the first two steps.
Now the third step is, we have to rewrite this as the CDF of x.
So right now we have it in terms of a probability of some event related to X.
Let's actually transform that to be explicitly in terms of the CDF of X.
So how do we do that?
Well, this is just the probability that X is within some range.
So we can turn that into the CDF by writing it as a difference of two CDFs.
So this is the same as the probability that X is less than or equal to Y squared minus the probability that X is less than or equal to negative Y.
So in order to find the probability that X is between this range, we take the probability that it's less than Y squared, which is everything here.
And then we subtract that probability that it's less than Y, negative Y.
So what we're left with is just within this range.
So these actually are now exactly CDFs of X.
So this is F of X evaluated at Y squared and this is F of X evaluated at negative Y.
So now we've completed step three.
And the last step that we need to do is differentiate.
So if we differentiate both sides of this equation with respect to Y, we'll get that the left side would get what we want, which is the PDF of Y.
Now we differentiate the right side-- we'll have to invoke the chain rule.
So the first thing that we do is, well, this is a CDF of X.
So when we differentiate we'll get the PDF of X.
But then we also have invoke the chain rule for this argument inside.
So the derivative of Y squared would give us an extra term, 2Y.
And then similarly this would give us the PDF of X evaluated at negative Y plus the chain will give us an extra term of negative 1.
So let's just clean this up a little bit.
So it's 2y F X squared plus F X minus Y.
All right, so now we're almost done.
We've differentiated.
We have the PDF of Y, which is what we're looking for.
And we've written it in terms of the PDF of X.
And fortunately we know what that is, so once we plug that in, then we're essentially done.
So what is the PDF?
Well, the PDF of X evaluated at Y squared is going to give us 1 over square root of 2 pi e to the minus-- so in this case, X is Y squared-- so we get Y to the fourth over 2.
And then we get another 1 over square root of 2 pi e to the minus Y squared over 2.
OK, and now we're almost done.
The last thing that we need to take care of is, what is the range?
Now remember, it's important when you calculate out PDFs to always think about the ranges where things are valid.
So when we think about this, what is the range where this actually is valid?
Well, Y, remember is related to X in this relationship.
So as we look at this, we see that Y can never be negative.
Because no matter what X is, Y gets transformed into some non-negative version.
So what we know is that this is now actually valid only for Y greater than 0 and for Y less than 0, the PDF is 0.
So this gives us the final PDF of Y.
All right, so it seems like at first when you start doing these derived restriction problems that it's pretty difficult.
But if we just remember that there are these pretty straightforward steps that we follow, and as long as you go through these steps and do them methodically, then you can actually come up with the solution for any of these problems.
And one last thing to remember is to always think about what are the ranges where these things are valid?
Because the relationship between these two random variables could be pretty complicated and you need to always be aware of when things are non-zero and when they are 0.
In this problem, we're given a random variable X which has a uniform distribution in the interval negative 1 to 1.
In other words, if we were to draw out the PDF of X, we see that in the interval negative 1 to 1, it has value 1/2.
Now we're given a sequence random variables X1, X2, and so on, where each Xi has the same distribution as X and different Xi's are independent.
For part a, we would like to know if the sequence Xi converges to some number-- let's call it c-- in probability as i goes to infinity-- whether this is true.
Let's first recall the definition of convergence in probability.
If this does happen, then by definition, we'll have that for every epsilon greater than 0, the probability Xi minus c greater equal to epsilon, this quantity will go to 0 in the limit of i going to infinity.
In other words, with very high probability, we will find Xi to be very concentrated around the number c if this were to be the PDF of Xi.
Now, can this be true?
Well, we know that each Xi is simply a uniform distribution over negative 1 to 1.
It doesn't really change as we increase i.
So intuitively, the concentration around any number c is not going to happen.
So we should not expect a convergence in probability in this sense.
For part b, we would like to know whether the sequence Yi, defined as Xi divided by i, converges to anything in probability.
Well, by just looking at the shape of Yi, we know that since the absolute value of Xi is less than 1, then we expect the absolute value of Yi is less than 1/i.
So eventually, Yi gets very close to 0 as i goes to infinity.
So it's safe to bet that maybe Yi will converge to 0 in probability.
Let's see if this is indeed the case.
The probability of Yi minus 0 greater equal to epsilon is equal to the probability of Yi absolute value greater equal to epsilon.
Now, previously we know that the absolute value of Yi is at most 1/i by the definition of Yi.
And hence the probability right here is upper bounded by the probability of 1i greater equal to epsilon.
Notice in this expression, there is nothing random.
i is simply a number.
Hence this is either 1 if i is less equal to 1/epsilon, or 0 if i is greater than 1/epsilon.
Now, this tells us, as long as i is great enough-- it's big enough compared to epsilon-- we know that this quantity here is [INAUDIBLE] 0.
And that tells us in the limit of i goes to infinity probability of Yi deviating from 0 by more than epsilon goes to 0.
And that shows that indeed, Yi converges to 0 in probability because the expression right here, this limit, holds for all epsilon.
Now, in the last part of the problem, we are looking at a sequence Zi defined by Xi raised to the i-th power.
Again, since we know Xi is some number between negative 1 and 1, this number raised to the i-th power is likely to be very small.
And likely to be small in the sense that it will have absolute value close to 0.
So a safe guess will be the sequence Zi converges to 0 as well as i goes to infinity.
How do we prove this formally?
We'll start again with a probability that Zi stays away from 0 by more than epsilon and see how that evolves.
And this is equal to the probability that Xi raised to the i-th power greater equal to epsilon.
Or again, we can write this by taking out the absolute value that Xi is less equal to negative epsilon raised to the 1 over i-th power or Xi greater equal to epsilon 1 over i-th power.
So here, we'll divide into two cases, depending on the value of epsilon.
In the first case, epsilon is greater than 1.
Well, if that's the case, then we know epsilon raised to some positive power is still greater than 1.
But again, Xi cannot have any positive density be on the interval negative 1 or 1.
And hence we know the probability above, which is Xi less than some number smaller than negative 1 or greater than some number bigger than 1 is 0.
So that case is handled.
Now let's look at a case where epsilon is less than 1, greater than 0.
So in this case, epsilon to the 1/i will be less than 1.
And it's not that difficult to check that since Xi has uniform density between negative 1 and 1 of magnitude 1/2, then the probability here was simply 2 times 1/2 times the distance between epsilon to the 1 over i-th power and 1.
So in order to prove this quantity converge to 0, we simply have to justify why does epsilon to the 1/i converge to 1 as i goes to infinity.
For that, we'll recall the properties of exponential functions.
In particular, if a is a positive number and x is its exponent, if we were to take the limit as x goes to 0 and look at the value of a to the power of x, we see that this goes to 1.
So in this case, we'll let a be equal to epsilon and x be equal to 1/i.
As we can see that as i goes to infinity, the value of x, which is 1/i, does go to 0.
And therefore, in the limit i going to infinity, the value of epsilon to the 1 over i-th power goes to 1.
And that shows if we plug this limit into the expression right here that indeed, the term right here goes to 0 as i goes to infinity.
And all in all, this implies the probability of Zi minus 0 absolute value greater equal to epsilon in the limit of i going to infinity converges to 0 for all positive epsilon.
And that completes our proof that indeed, Zi converges to 0 in probability.
In this problem, we're given an urn with n balls in it, out of which m balls are red balls.
To visualize it, we can draw a box that represents the set of all n balls.
Somewhere in the middle or somewhere else we have a cut, such that to the left we have all the red balls (there are m), and non-red balls.
Let's for now call it black balls.
That is n minus m. Now, from this box, we are to draw k balls, and we'd like to know the probability that i out of those k balls are red balls.
For the rest of the problem, we'll refer to this probability as p-r, where r stands for the red balls.
So from this picture, we know that we're going to draw a subset of the balls, such that i of them are red, and the remaining k minus i are black.
And we'll like to know what is the probability that this event would occur.
To start, we define our sample space, omega, as the set of all ways to draw k balls out of n balls.
We found a simple counting argument -- we know that size of our sample space has n-choose-k, which is the total number of ways to draw k balls out of n balls.
Next, we'd like to know how many of those samples correspond to the event that we're interested in.
In particular, we would like to know c, which is equal to the number of ways to get i red balls after we draw the k balls.
To do so, we'll break c into a product of two numbers -- let's call it a times b -- where a is the total number of ways to select i red balls out of m red balls.
So the number of ways to get i out of m red balls.
Going back to the picture, this corresponds to the total number of ways to get these balls.
And similarly, we define b as the total number of ways to get the remaining k minus i balls out of the set n minus m black balls.
This corresponds to the total number of ways to select the subset right here in the right side of the box.
Now as you can see, once we have a and b, we multiply them together, and this yields the total number of ways to get i red balls.
To compute what these numbers are, we see that a is equal to m-choose-i number of ways to get i red balls, and b is n minus m, the total number of black balls, choose k minus i, the balls that are not red within those k balls.
Now putting everything back, we have p-r, the probability we set out to compute, is equal to c, the size of the event, divided by the size of the entire sample space.
From the previous calculations, we know that c is equal to a times b, which is then equal to m-choose-i times (n minus m)-choose-(k minus i).
And on the denominator, we have the entire sample space is a size n-choose-k. And that completes our derivation.
Now let's look at a numerical example of this problem.
Here, let's say we have a deck of 52 cards.
And we draw a box with n equals 52, out of which we know that there are 4 aces.
So we'll call these the left side of the box, which is we have m equals 4 aces.
Now if we were to draw seven cards-- call it k equal to 7-- and we'd like to know what is the probability that out of the 7 cards, we have 3 aces.
Using the notation we did earlier, if we were to draw a circle representing the seven cards, we want to know what is the probability that we have 3 aces in the left side of the box and 4 non-aces for the remainder of the deck.
In particular, we'll call i equal to 3.
So by this point, we've cast the problem of drawing cards from the deck in the same way as we did earlier of drawing balls from an urn.
And from the expression right here, which we computed earlier, we can readily compute the probability of having 3 aces.
In particular, we just have to substitute into the expression right here the value of m equal to 4, n equal to 52, k equal to 7, finally, i equal to 3.
So we have 4-choose-3 times n minus m, in this case would be 48, choose k minus i, will be 4, and on the denominator, we have 52 total number of cards, choosing 7 cards.
That gives us [the] numerical answer [for] the probability of getting 3 aces when we draw 7 cards.
In this problem, we're given a collection of 10 variables, x1 through x10, where each i, xi, is a uniform random variable between 0 and 1.
So each i is uniform between 0 and 1, and all 10 variables are independent.
And we'd like to develop a bound on the probability that some of the 10 variables, 1 to 10, being greater than 7 using different methods.
So in part A we'll be using the Markov's inequality written here.
That is, if we have a random variable, positive random variable x, the probability x is greater than a, where a is again some positive number, is bounded above by the expected value of x divided by a.
And let's see how that works out in our situation.
In our situation, we will call x the summation of i equal to 1 to 10xi, and therefore, E of x is simply 10 times E of x1, the individual ones, and this gives us 5.
Here we use used the linearity of expectation such that the expectation of the sum of the random variable is simply the sum of the expectations.
Now, we can invoke Markov's Inequality.
It says x greater or equal to 7.
This is less than E of x over 7, and this gives us 5 over 7.
For part B, let's see if we can improve the bound we got in part A using the Chebyshev inequality, which takes into account the variance of random variable x.
Again, to refresh you on this, the Chebyshev Inequality says the probability that x deviates from its mean E of x, by more than a is bound above by the variance of x divided by a squared.
So we have to actually do some work to transform the probability we're interested in, which is x greater or equal to 7, into the form that's convenient to use using the Chebyshev Inequality.
To do so, we'll rewrite this probability as the probability of x minus 5 greater or equal to 2 simply by moving 5 from the right to the left.
The reason we chose 5 is because 5 is equal to the expected value of x from part A as we know before.
And in fact, this quantity is also equal to the probability that x minus 5 less or equal to negative 2.
To see why this is true, recall that x is simply the summation of the xi's, the 10 xi's, and each xi is a uniform random variable between 0 and 1.
And therefore, each xi, the distribution of which is symmetric around its mean 1/2.
So we can see that after we add up all the xi's, the resulting distribution x is also symmetric around its mean 5.
And as a result, the probability of x minus 5 greater than 2 is now equal to the probability that x minus 5 less than negative 2.
And knowing these two, we can then say they're both equal to 1/2 the probability x minus 5 absolute value greater or equal to 2, because this term right here is simply the sum of both terms here and here.
At this point, we have transformed the probability of x greater or equal to 7 into the form right here, such that we can apply the Chebyshev's Inequality basically directly.
And we'll write the probably here being less than or equal to 1/2 times, applying the Chebyshev Inequality, variance of x divided by 2 squared.
Now, 2 is the same as a right here, and this gives us 1/8 times-- now, the variance of x, we know is 10 times the variance of a uniform random variable between 0 and 1, which is 1/12, and that gives us 5/48.
Now, let's compare this with the number we got earlier using the Markov Inequality, which was 5/7.
We see that 5/48 is much smaller, and this tells us that, at least for this example, using the Chebyshev Inequality combined with the information of the variance of x, we're able to get a stronger upper bound on the probability of the event that we're interested in.
Now, in part B, we saw that by using the additional information of the variance combined with the Chebyshev Inequality, we can improve upon bound given by Markov's Inequality.
Now, in part C, we'll use a somewhat more powerful approach in addition to the Chebyshev Inequality, the so-called central limit theorem.
Let's see if we can even get a better bound.
To remind you what a central limit theorem is, let's say we have a summation of i equal to 1 to some number n of independent and identically distributed random variables xi.
Now, the central limit theorem says the following.
We take the sum right here, and subtract out its means, which is E of the same summation, and further, we'll divide out, what we call normalize, by the standard deviation of the summation.
In other words, the square root of the variance of the sum of xi.
So if we perform this procedure right here, then as the number of terms in the sums going to infinity, here as in goes to infinity, we will actually see that this random variable will converge in distribution in some way that will eventually look like a standard normal random variable with means 0 and 1.
And since we know how the distribution of a standard normal looks like, we can go to table and look up certain properties of the resulting distribution.
So that is a plan to do.
So right now, we have about 10 variables.
It's not that many compared to a huge numbering, but again, if we believe it's a good approximation, we can get some information out of it by using the central limit theorem.
So we are interesting knowing that probability summation of i equal to 1 to 10 x1 greater or equal to 7.
We'll rewrite this as 1 minus the probability the summation i equal to 1 to 10, and xi less equal to 7.
Now, we're going to apply the scaling to the summation right here.
So this is equal to 1 minus the probability summation i equal to 1 to 10xi minus 5.
Because we know from previous parts that 5 is the expected value of the sum right here, and divided by square root of 10/12.
Again, earlier we know that 10/12 is the variance of the sum of xi's.
And we'll do the same on the other side, writing it 7 minus 5 divided by square root of 10/12.
Now, if we compute out the quantity right here, we know that this quantity is roughly 2.19, and by the central limit theorem, if we believe 10 is a large enough number, then this will be roughly equal to 1 minus the CDF of a standard normal evaluated at 2.19.
And we could look up the number in the table, and this gives us number roughly, 0.014.
Now let's do a quick summary of what this problem is about.
We're asked to compute the probability of x greater or equal to 7, where x is the sum of 10 uniform random variables between 0 and 1, so we'll call it xi.
We know that because each random variable has expectation 1/2, adding 10 of them up, gives us expectation of 5.
So this is essentially asking, what is the chance that x is more than two away from its expectation?
So if this is a real line, and 5 is here, maybe x has some distribution around 5, so the center what the expected value is at 5, we wonder how likely is it for us to see something greater than 7?
Now, let's see where do we land on the probably spectrum from 0 to 1.
Well, without using any information, we know the probability cannot be greater than 1, so a trivial upper bound for the probability right here will be 1.
Well, for the first part we use Markov's Inequality and that gives us some number, which is roughly equal to 0.7.
In fact, we got number 5/7, and this is from Markov's Inequality.
Oh, it's better than 1, already telling us it cannot be between 0.7 and 1, but can we do better?
Well, the part B, we see that all the way, using the additional information variance, we can get this number down to 5/48, which is roughly 0.1.
Already, that's much better than 0.7.
Can we even do better?
And this is the Chebyshev, and it turns out we can indeed do better.
Using the central limit theorem, we can squeeze this number all the way down to 0.014, almost a 10 times improvement over the previous number.
This is from central limit theorem.
As we can see, by using different bounding techniques, we can progressively improve the bound on the probability of x exceeding 7, and from this problem we learned that even with 10 variables, the truth is more like this, which says that the distribution of x concentrates very heavily around 5, and hence, the probability of x being greater or equal to 7 could be much smaller than one might expect.
Hi.
Today, we're going to do a really fun problem called geniuses and chocolates.
And what this problem is exercising is your knowledge of properties of probability laws.
So let me just clarify what I mean by that.
Hopefully, by this point, you have already learned what the axioms of probability are.
And properties of probability laws are essentially any rules that you can derive from those axioms.
So take for example the fact that the probability of A union B is equal to the probability of A plus the probability of B minus the probability of the intersection.
That's an example of a property of a probability law.
So enough with the preamble.
Let's see what the problem is asking us.
In this problem, we have a class of students.
And we're told that 60% of the students are geniuses.
70% of the students love chocolate.
So I would be in that category.
And 40% fall into both categories.
And our job is to determine the probability that a randomly selected student is neither a genius nor a chocolate lover.
So first I just want to write down the information that we're given in the problem statement.
So if you let G denote the event that a randomly selected student is a genius then the problem statement tells us that the probability of G is equal to 0.6.
Similarly, if we let C denote the event that a randomly selected student is a chocolate lover, then we have that the probability of C is equal to 0.7.
Lastly, we are told that the probability a randomly selected student falls into both categories is 0.4.
And the way we can express that using the notation already on the board is probability of G intersect C is equal to 0.4.
OK, now one way of approaching this problem is to essentially use this information and sort of massage it using properties of probability laws to get to our answer.
Instead, I'm going to take a different approach, which I think will be helpful.
So namely, we're going to use something called a Venn diagram.
Now a Venn diagram is just a tool that's really useful for telling you how different sets relate to each other and how their corresponding probabilities relate to each other.
So the way you usually draw this is you draw a rectangle, which denotes your sample space, which of course, we call omega.
And then you draw two intersecting circles.
So one to represent our geniuses and one to represent our chocolate lovers.
And the reason why I drew them intersecting is because we know that there are 40% of the students in our class are both geniuses and chocolate lovers.
OK, and the way you sort of interpret this diagram is the space outside these two circles correspond to students who are neither geniuses nor chocolate lovers.
And so just keep in mind that the probability corresponding to these students on the outside, that's actually what we're looking for.
Similarly, students in this little shape, this tear drop in the middle, those would correspond to geniuses and chocolate lovers.
You probably get the idea.
So this is our Venn diagram.
Now I'm going to give you guys a second trick if you will.
And that is to work with partitions.
So I believe you've seen partitions in lecture by now.
And a partition is essentially a way of cutting up the sample space into pieces.
But you need two properties to be true.
So the pieces that you cut up your sample space into, they need to be disjoint, so they can't overlap.
So for instance, G and C are not disjoint because they overlap in this tear drop region.
Now the second thing that a partition has to satisfy is that if you put all the pieces together, they have to comprise the entire sample space.
So I'm just going to put these labels down on my graph.
X, Y, Z, and W. So X is everything outside the two circles but inside the rectangle.
And just note, again, that what we're actually trying to solve in this problem is the probability of X, the probability that you're neither genius, because you're not in this circle, and you're not a chocolate lover, because you're not in this circle.
So Y I'm using to refer to this sort of crescent moon shape.
Z, I'm using to refer to this tear drop.
And W, I'm using to refer to this shape.
So, hopefully, you agree that X, Y, Z, and W form a partition because they don't overlap.
So they are disjoint.
And together they form omega.
So now we're ready to do some computation.
The first step is to sort of get the information we have written down here in terms of these new labels.
So hopefully, you guys buy that G is just the union of Y and Z.
And because Y and Z are disjoint, we get that the probability of the union is the sum of the probabilities.
And, of course, we have from before that this is 0.6.
Similarly, we have that the probability of C is equal to the probability of Z union W. And, again, using the fact that these two guys are disjoint, you get this expression.
And that is equal to 0.7.
OK, and the last piece of information, G intersects C corresponds to Z, or our tear drop, and so we have that the probability of Z is equal to 0.4.
And now, if you notice, probability of Z shows up in these two equations.
So we can just plug it in.
So plug in 0.4 into this equation.
We get P of Y plus 0.4 is 0.6.
So that implies that P of Y is 0.2.
That's just algebra.
And similarly we have point.
0.4 plus P of W is equal to 0.7.
So that implies that P of W is 0.3.
Again, that's just algebra.
So now we're doing really well because we have a lot of information.
We know the probability of Y, the probability of Z, the probability of W. But remember we're going for, we're trying to find the probability of X.
So the way we finally put all this information together to solve for X is we use the axiom that tells us that 1 is equal to the probability of the sample space.
And then, again, we're going to use sort of this really helpful fact that X, Y, Z, and W form a partition of omega to go ahead and write this as probability of X plus probability of Y plus probability, oops, I made a mistake.
Hopefully, you guys caught that.
It's really, oh, no.
I'm right.
Never mind.
Probability of X plus probability of Y plus probability of Z plus probability of W. And now we can go ahead and plug-in the values that we solved for previously.
So we get probability of X plus 0.2 plus 0.4 plus 0.3.
These guys sum to 0.9.
So, again, just simple arithmetic, we get that the probability of X is equal to 0.1.
So we're done because we've successfully found that the probability that a randomly selected student is neither a genius nor a chocolate lover is 0.1.
So this was a fairly straightforward problem.
But there are some important takeaways.
The first one is that Venn diagrams are a really nice tool.
Whenever the problem is asking you how different sets relate to each other or how different probabilities relate to each other, you should probably draw Venn diagram because it will help you.
And the second takeaway is that it's frequently useful to divide your sample space into a partition mainly because sort of the pieces that compose a partition are disjoint.
So we will be back soon to solve more problems.
In this problem, we're going to look at how to infer a continuous random variable from a discrete measurement.
And the continuous random variable that we're interested in in this problem is q, which is given by this PDF.
It's 6q times 1 minus q for a q between 0 and 1 and 0 otherwise.
And here is a graph of what it looks like.
It just kind of has this curve shape.
And it's symmetric.
And it's peak is at 1/2.
And the way to interpret q is q is the unknown bias of a coin.
So the bias of a coin, the probability of heads of that coin is somewhere between 0 and 1.
We're not sure exactly what it is.
And here is our, say, prior belief about how this random bias is distributed.
And we're going try to infer what this bias is by flipping the coin and observing whether or not we got heads or tails.
And because of that, the measurement, or the observation that we get, is discreet.
Either we get heads, or we get tails.
And we model that using a discrete random variable x, which is, in this case, it turns out it's just a Bernoulli random variable, either 0 or 1.
And the model that we have is that, if we knew what this bias q was, if we knew that it was a little q, then this coin, I mean, it behaves as if it was a coin with that bias.
So the probability of getting heads, or the probability that x equals 1, is just equal to q, which is the bias.
And the probability that it's equal to 0 is 1 minus q.
So that's just our model of how the coin works.
We can also write this another way, as just more like a conditional PMF.
So the conditional PMF of x, given q of little x, is q, if x is 1, 1 minus q, if x equals 0, and 0 otherwise.
Just a more compact way of writing this.
All right, so what we want to do in this problem is find this conditional PDF.
What is the conditional PDF of q given x?
So we observe what x is, either a 0 or 1.
And we want to know now, given that information, given that measurement, what is the new distribution of q the bias of the coin?
And to do that, well, we apply Bayes' rule.
And remember, Bayes' rule, it consists of several terms.
The first one is the numerator, which is our prior initial belief, so which is just the regular PDF of q, times the conditional PMF of x given q.
All right, so because we have a continuous variable that we want to infer from a discreet measurement, we use this variation of Bayes' rule, where we have a PDF here and a conditional PMF here.
And the denominator is-- well, it's the PMF of x.
And of course, we can take this PMF of x, this denominator, and expand it, using the law of total probability where the PMF of x, you can think of it as you can get x with a combination of lots of different possible values of the bias q.
And so we just calculate all those possibilities and integrate.
And what we want to integrate here is q, so we want to integrate-- remember to keep in mind the limits of integration.
And this is just referenced by the limits of what the PDF of q is.
OK. All right, so now we're asked to find what this value is for x equals to 0 or 1 and for all values of q.
And the values of q we care about are the ones between 0 and 1.
So let's focus on the two different possibilities for x.
So the first one is, let's look at the case where x equals 1 first.
And then now let's just plug-in what all these different terms should be.
Well, the PDF of q we're given.
And of course, we're looking here at q between 0 and 1, so within that range.
The PDF of q is just 6q times 1 minus q.
And the conditional PMF of x where we know that from our model, because we're looking at the case where x equals 1.
That conditional PMF is just q.
And the denominator is really the same as the numerator, except you integrate it.
So it's the integral from 0 to 1 of 6q times 1 minus q times q dq.
OK. And now we can simplify this.
So under the numerator, we have integral-- sorry, 6q squared times 1 minus q.
And then the bottom we have the integral of 6q squared minus q cubed, d cubed from 0 to 1.
And now this is just some calculus now.
So we still have the numerator 6q squared times 1 minus q.
The denominator, we have 2q cubed-- that would give us the 6q squared term-- minus 6/4 q to the fourth.
And we integrate that from 0 to 1.
OK. And what does that give us?
We get 6q squared 1 minus q still on the top.
And the bottom, we get-- well the 0-- the case where it's 0, it's just 0.
The case where it's 1, it's 2 minus 3/2, so it's 1/2.
So really, it just becomes 12 q squared 1 minus q.
And of course, this only true when q is between 0 and 1.
All right, so the case where it's equal to 1, we have our answer.
And it turns out that, if you plot this, what does it look like?
It looks something like this where the peak is now at 2/3.
So how do you interpret this?
The interpretation is that what you have is you observe that you've got, actually, heads on this toss.
So that is evidence that the bias of the coin is relatively high.
So it's relatively more likely to get heads with this coin.
So q, in that case, you would believe that it's more likely to be higher than 1/2.
And it turns out that, when you go through this reasoning and the Bayes' rule, what you get is that it looks like this.
And the peak is now at 2/3.
And you can repeat this exercise now with the case where x is 0.
So when x is 0, you still get the same term here, 6q 1 minus q, but the conditional PMF is now the-- you want the conditional PMF when x equals 0, which is now 1 minus q.
So you get 1 minus q here.
And now this term becomes 6q times 1 minus q squared.
And so really, the bottom is also 6q times 1 minus q squared dq.
And if you go through the same sort of calculus, what you get is that, in this case, the answer is 12q 1 minus q squared.
So let me rewrite what the first case was.
The first case, when x equals 1 was equal to 12q squared 1 minus q.
So they look quite similar, except that this one has q squared, this one has 1 minus q squared.
And if you take this one, the case where you observe a 0, and you plot that, it turns out it looks something like this.
And this actually doesn't look like it, but it should be the peak is at 1/3.
And so you notice, first of all, that these are symmetric.
There's some symmetry going on.
And this interpretation in this case is that, because you observed 0, which corresponds to observing tails, that gives you evidence that the bias of the coin is relatively low, or the probability of getting heads with this coin is relatively low, which pushes your belief about q towards the smaller values.
OK, so it turns out that this distribution, this distribution, and the original distribution of q, all fall under family of distributions called the beta distribution.
And they're parameterized by a couple of parameters.
And it's used frequently to model things like the bias of a coin, or anything that's a random variable that's bounded between 0 and 1.
OK, so this problem allowed us to get some more exercise with Bayes' rule, this continuous discrete version of Bayes' rule.
And if you go through all the steps, you'll find that it's relatively straightforward to go through the formula and plug in the different parts of the terms and go through a little bit of calculus and find the answer.
And it's always good to go back, once you find the answer, to look at it a little bit and make sure that actually makes sense.
So you can convince yourself that it's not something that looks--
Hi.
In this problem, we're going to look at how to infer a discrete random variable from a continuous measurement.
And really, what it's going to give us is some practice working with a variation of Bayes' rule.
So the problem tells us that we have a discrete random variable x with this PMF.
It is 1 with probability P, minus 1 with probability 1 minus P, and 0 otherwise.
So here is just a diagram of this PMF.
And then we also have another random variable, y, which is continuous.
And its PDF is given by this.
It's 1/2 lambda e to the minus lambda times the absolute value of y.
And so this may look familiar.
It looks kind of like an exponential.
And in fact, it's just a two-sided exponential.
That's flattened by a factor of 1/2.
And this is what it looks like, kind of like a tent that goes on both ways.
And then we have a random variable z, which is equal to the sum of x and y.
And the problem is going to be figuring out what x is, based on an observed value of what z is.
So because x is discrete and y is random-- sorry, x is discrete, and y is continuous, z is also going to be continuous.
So our measurement is z, which is continuous.
And we want to infer x, which is discrete.
So the problem asked us to find is this what is the probability that x equals 1, given that z is a little z.
And you can write this another way, just as a conditional PMF as well.
It's the conditional PMF of x, given z, evaluated to 1 conditioned on little z.
All right, so now let's apply the correct variation of Bayes' rule.
So remember, it's going to be this, the probability that x equals 1, or the PMF of x evaluated to 1, times the-- you flip this conditioning.
So now it's going to be a conditional PDF of z, since z is continuous.
It's going to be a conditional PDF of z, given x, evaluated at some little z condition on x being 1.
And the bottom is the conditional PDF-- or sorry, just the regular PDF of z.
And of course, we can rewrite this denominator.
Remember, the denominator is always just-- you can use the law of total probability and rewrite it.
And one of the terms is going to be exactly the same as the numerator.
So one of the ways that z can be some little z is it's in combination with x being equal to 1.
And the probability of that is exactly the same thing as the numerator.
And the other way is if x is equal to negative 1.
And that gives us this second term.
All right.
And now let's just fill in what all these different terms are.
So with the PMF of x evaluated at 1, that is just P. What is the conditional PDF of z, given that x is equal to 1?
Well, that takes a little bit more work.
Given that x is 1, then z is just going to be-- so if x equals 1, then z is just y plus 1, which means that you can just imagine taking y-- this is what y is, the distribution of y-- and just adding 1 to it, which, in this diagram, would amount to shifting it over by one.
So now, it would look like this, the distribution.
And algebraically, all you would do is just change this thing in the absolute value to y minus 1.
That amounts to shifting it over to the right by one.
All right.
So what is that?
That's just 1/2 lambda, 1/2 lambda, e to the minus lambda y-- sorry, not y, z-- z minus 1.
And the denominator, well, the first term is going to be exactly the same.
It's just also P 1/2 lambda e to the minus lambda z minus 1.
What about the second term?
The second term, first we need to figure out what is the PMF of x evaluated at a negative 1.
Or in other words, what's the probability that x is negative 1?
That is given to us by the PMF.
It's 1 minus P. And then the second part is, what is the conditional PDF of z, given that x is negative 1?
Well, we can just do the same sort of trick here.
If x is negative 1, then z is just y minus 1.
In which case, the PDF of z would just look like this.
You're shifted to the left by one now.
And now what you have to do is change this into a plus 1.
So this conditional PDF would be 1/2 lambda e to the minus lambda z plus 1, absolute value of z plus 1.
All right, so this looks pretty messy.
And we can try to simplify things a little bit.
So we can get rid of these 1/2 lambdas.
And then we can multiply the numerator and the denominator by the same term.
Let's multiply it by e to the lambda absolute value of z minus 1.
So what we're going to do is try to cancel out some of these exponential terms.
So that will cancel out this top term.
So all we have in the numerator now is just P. It will also cancel out this exponential in the denominator.
And then we'll have to change this here, because it'll have an extra e to the lambda absolute value of z minus 1.
All right, now let's rewrite this.
And what we get is plus 1 minus P e to the minus lambda absolute value of z plus 1 minus absolute value of z minus 1.
OK, so that is pretty much as far as you can go in terms of simplifying it.
And now the question is, are we comfortable with this answer?
And it helps always to try to interpret it a little bit, to make sure that it makes intuitive sense.
And one way to do that is to try to-- some of the limiting cases of what some of the parameters can be.
So in this case, the parameters are P and lambda.
So P is the parameter related to x.
And lambda is the parameter related to y.
So let's try to see if it makes sense under some limiting cases.
The first one we want to think about is when P goes to 0.
So if P goes to 0, what happens to our answer?
Well, the numerator is 0, this is 0, this is 1.
But it doesn't matter, because the numerator is 0.
So in this case, this would go to 0.
Now does that make sense?
Well, what does that mean when P goes to 0?
When P goes to 0, that means that the probability that x is equal to 1 is 0.
So even without thinking about y or z, there is already a 0 probability that x is equal to 1.
Now this whole calculation, what we found is, well, if I had some more information, like what z is, does that help me find out what the probability of x being 1 is?
Well, no matter what z tells me, I know for a fact that x can't be 1, because P is 0.
So this posterior, or this conditional probability, should also be 0, because there's just no way that x can be 1.
So in this case, this formula does check out.
Now let's think about another case where P goes to 1.
If P goes to 1, that means that X is for sure going to be 1.
And it can't be anything else.
In which case, what does our formula tell us?
Well, this numerator is 1.
This term is 1.
1 minus 1 is 0.
So the second term gets zeroed out, and the answer is just 1/1 is 1.
So what does this tell us?
This tells us that if I know beforehand that x is for sure equal to 1, then, if I now give myself more information and condition on what I observe for z, that shouldn't change anything else.
I should still know for sure that x is equal to 1.
So the probability of this conditional probability should still be equal to 1.
And it does, so our formula also works in this case.
Now let's think about lambda.
What about when lambda goes to 0?
Well, when lambda goes to 0, that's a little harder to visualize.
But really, what would happen is that you can imagine this distribution getting shallower, shallower and shallower, lower and lower, so that it's like it is kind of flat and goes on forever.
And so what this tells you is that, basically, y-- this is the distribution y-- so when lambda goes to 0, that tells you that y has a really flat and kind of short distribution.
And so what does our formula tell us in this case?
Well, when lambda goes to 0, this exponent is equal to 0.
And so e to the 0 is 1.
So we get P over P plus 1 minus P, which is just 1.
So the answer here, our formula will give us an answer of P. So what does that tell us?
That tells us that, in this case, if lambda goes to 0, then our posterior probability, the probability that x equals 1 conditioned on z being some value, conditioned on our continuous measurement, is still P. So the prior, or the original probability for x being equal to 1 is P. And with this additional continuous measurement, our guess of the probability that x equal to 1 is still P. So it hasn't changed.
So basically, it's telling us that this additional information was not informative.
It didn't actually help us change our beliefs.
And so why is that?
Well, one way to think about it is that, because the distribution of y looks like this, is very flat and it could be anything, then, if you observe some value of z, then it could be that that was due to the fact that it was x equal to 1 plus some value of y that made z equal to that value.
Or it could have just as equally been likely that x equal to negative 1 y equals to some other value that made it equal to z.
And so, essentially, it's z-- because y has a shape, it can be likely to take on any value that complements either x being equal to 1 or x equal being to negative 1, to make z equal to whatever the value it is that you observe.
And so because of that, in this case, y is not very informative.
And so this probability is still just equal to P. Now the last case is when lambda goes to infinity.
And now we have to break it down into the two other cases now.
The first case is when-- lets write this over here-- when lambda goes to infinity.
The first case, it depends on what this value is, the sine of this value.
If this value, the absolute value of z plus 1 minus the absolute value of z minus 1, if that's positive, then, because lambda goes to infinity and you have a negative sign, then this entire exponential term will go to 0.
In which case, the second term goes to 0.
And the answer is P/P, or is 1.
And so if absolute value of z plus 1 minus absolute value of z minus 1 is greater than 0, then the answer is 1.
But in the other case, if this term in the exponent, if it's actually negative, if it's negative, then this negative sign turns to a positive, and lambda goes to infinity.
And so this term blows up, and it dominates everything else.
And so the denominator goes to infinity.
The numerator is fixed at P, so this entire expression would go to 0.
OK, so now let's try to interpret this case.
Let's start with the first one.
When is it that absolute value of z plus 1 minus absolute value of z minus 1 is greater than 0?
Or you can also rewrite this as absolute value of z plus 1 is greater than absolute value of z minus 1.
Well, when is that case?
Well, it turns out, if you think about it, this is only true if z is positive.
If z is positive, then adding 1-- let me draw a line here, and if this is 0-- if z is positive, something here, adding 1 to it and taking the absolute value-- the absolute value doesn't do anything-- but you will get something bigger.
Where subtracting 1 will take you closer to 0, and so because of that, the absolute value, the magnitude, or the distance from 0 will be less.
Now if you're on the other side, adding 1 will take you-- if you're on the other side, adding 1 will take you closer to 0.
And so this magnitude would be smaller.
Whereas, subtracting will take you farther away, so the absolute value actually increased the magnitude.
And so this is the same as z being positive.
And so this is the same as z being negative.
So what this tells you is that, if z is positive, then this probability is equal to 1.
And if z is negative, this probability is equal to 0.
Now why does that make sense?
Well, it's because when lambda goes to infinity, you have the other case.
Essentially, you pull this all the way up, really, really far, and it drops off really quickly.
And so when you take the limit, as lambda goes to infinity, effectively, it just becomes a spike at 0.
And so, more or less, you're sure that y is going to be equal to 0.
And so, effectively, z is actually going to be equal to x, effectively.
And because of that, because x can only be 1 or negative 1, then, depending on if you get a z that's positive, then you know for sure that it must have been that x was equal to 1.
And if you get a z that's negative, you know for sure that it must have been that x was equal to negative 1.
And so because of that, you get this interpretation.
And so we've looked at four different cases of the parameters.
And in all four cases, our answer seems to make sense.
And so we feel more confident in the answer.
And so to summarize, this whole problem involved using Bayes' rule.
You start out with some distributions, and you apply Bayes' rule.
And you go through the steps, and you plug-in the right terms.
And then, in the end, it's always helpful to try to check your answers to make sure that it makes sense in some of the limiting.
Hi.
In this problem, Romeo and Juliet are back and they're still looking to meet up for a date.
Remember, the last time we met up with them, it was back in the beginning of the course and they were trying to meet up for a date but they weren't always punctual.
So we modeled their delay as uniformly distributed between 0 and 1 hour.
So now in this problem, we're actually going to look at variation.
And we're going to ask the question, how do we actually know that the distribution is uniformly distributed between 0 and 1 hour?
Or it could also be the case that it is uniformly distributed between 0 and half an hour, or zero and two hours.
How do we actually know what this parameter of the uniform distribution is?
OK, so let's put ourselves in the shoes of Romeo who's tired of being stood up by Juliet on all these dates.
And fortunately, he's learned some probability since the beginning of course, and so have we.
And in particular we've learned Bayesian inference.
And so in this problem, we're actually going to use basically all the concepts and tools of Bayesian inference that we learned chapter eight and apply them.
So it's a nice review problem, and so let's get started.
The set of the problem is similar to the first Romeo and Juliet problem that we dealt with.
They are meeting up for a date, and they're not always punctual and they have a delay.
But instead of the delay being uniformly distributed between 0 and 1 hour, now we have an extra layer of uncertainty.
So if we know sum theta, then we know that the delay, which we'll call x is uniformly distributed between 0 and the theta.
So here's one possible theta, theta 1.
But we don't actually know what this theta is.
So in the original problem we knew that theta was exactly one hour.
But in this problem we don't know what theta is.
So theta could also be like this, some other theta 2.
And we don't know what this theta is.
And we choose to model it as being uniformly distributed between 0 and 1.
So like I said, we have two layers now.
We have uncertainty about theta, which is the parameters of the uniform distribution.
And then we have uncertainty in regards to the actual delay, x. OK, so let's actually write out what these distributions are.
So theta, the unknown parameter, we're told in the problem that we're going to assume that is uniformly distributed between 0 and 1.
And so the PDF is just 1, when theta is between 0 and 1, and 0 otherwise.
And we're told that, given what theta is, given what this parameter is, the delay is uniformly distributed between 0 and this theta.
So what that means is that we know this conditional PDF, the conditional PDF of x given theta is going to be 1 over theta if x is between 0 and theta, and 0 otherwise.
All right, because we know that given a theta, x is uniformly distributed between 0 and theta.
So in order to make this uniform distribution, it's the normalization or the heights, you can think of it, has to be exactly 1 over theta.
So just imagine for a concrete case, if theta were 1, 1 hour in the original problem, then this would just be a PDF of 1 or a standard uniform distribution between 0 and 1.
OK, so now this is, we have the necessary fundamentals for this problem.
And what do we do in inference?
Well the objective is to try to infer some unknown parameter.
And what we have is we have a prior which is our initial belief for what this parameter might be.
And then we have some data.
So in this case, the data that we collect is the actual observed delayed for Juliet, x.
And this model tells us how this data is essentially generated.
And now what we do is, we want to use the data and our prior belief, combined them somehow, and use it to update our belief into what we call our posterior.
In order to do that, we use Bayes' rule, which is why this is called Bayesian inference.
So when we use Bayes' rule, remember the Bayes' rule is just, we want to now find the posterior which is the conditional PDF of theta, the unknown parameter, given x.
So essentially just flip this condition.
And remember Bayes' rule is given as the following.
It's just the prior times this conditional PDF of x given theta divided by the PDF of x.
All right, and we know what most of these things are.
The prior or just the PDF of theta is 1.
The condition PDF of x given theta is 1 over theta.
And then of course we have this PDF of x.
But we always have to be careful because these two values are only valid for certain ranges of theta and x.
So in order for this to be valid we need theta to be between 0 and 1 because otherwise it would be 0.
So we need theta to be between 0 and 1.
And we need x to be between 0 and theta.
And otherwise this would be 0.
So now we're almost done.
One last thing we need to do is just calculate what this denominator is, f x of x.
Well the denominator, remember, is just a normalization.
And it's actually relatively less important because what we'll find out is that this has no dependence on theta.
It will only depend on x.
So the importance, the dependence on theta, will be captured just by the numerator.
But for completeness let's calculate out what this is.
So it's just a normalization.
So it's actually just the integral of the numerator.
You can think of it as an application of kind of total probability.
So we have this that we integrate over and what do we integrate this over?
Well we know that we're integrating over theta.
And we know that theta has to be between x-- has to be greater than x and it has to be less than 1.
So we integrate from theta equals x to 1.
And this is just the integral from x to 1 of the numerator, right?
This is just 1 and this is 1 over theta.
So it's the integral of 1 over theta, d theta from x to 1.
Which when you do it out, this is the integral, this is log of theta.
So it's log of 1 minus log of x. Log of 1 is 0.
X, remember x is between 0 and theta.
Theta is less than 1.
So x has to be between 0 and 1.
The log of something between 0 and 1 is negative.
So this is a negative number.
This is 0.
And then we have a negative sign.
So really what we can write this as is the absolute value of log of x.
This is just so that it would actually be negative log of x.
But because log of x is negative we can just-- we know that this is actually going to be a positive number.
So this is just to make it look more intuitive.
OK so now to complete this we can just plug that back in and the final answer is-- this is going to be the absolute value log of x or you could also rewrite this as 1 over theta times absolute value log of x.
And of course, remember that the actual limits for where this is valid are very important.
OK, so what does this actually mean?
Let's try to interpret what this answer is.
So what we have is this is the posterior distribution.
And now what have we done?
Well we started out with the prior, which was that theta is uniform between 0 and between 0 and 1.
This is our prior belief.
Now we observed some data.
And this allows us to update our belief.
And this is the update that we get.
So let's just assume that we observe that Juliet is late by half an hour.
Well if she's late by half an hour, what does that tell us about what theta can be?
Well what we know from that at least is that theta cannot be anything less than half an hour because if theta were less than half an hour there's no way that her delay-- remember her delay we know has to be distributed between 0 and theta.
There's no way that her delay could be half an hour if theta were less than half an hour.
So automatically we know that now theta has to be somewhere between x and one which is where this limit comes in.
So we know that theta have to be between x and 1 now instead of just 0 and 1.
So by observing an x that cuts down and eliminates part of the range of theta, the range that theta can take on.
Now what else do we know?
Well this, we can actually plot this.
This is a function of theta.
The log x, we can just think of it as some sort of scaling factor.
So it's something like 1 over theta scaled.
And so that's going to look something like this.
And so what we've done is we've transformed the prior, which looks like flat and uniform into something that looks like this, the posterior.
So we've eliminated small values of x because we know that those can't be possible.
And now what's left is everything between x and 1.
So now why is it also that it becomes not uniform between x and 1?
Well it's because, if you think about it, when theta is close to x, so say x is half an hour.
If theta is half an hour, that means that there's higher probability that you will actually observe something, a delay of half an hour because there's only a range between 0 and half an hour that x can be drawn from.
Now if theta was actually 1 then x could be drawn anywhere from 0 to 1 which is a wider range.
And so it's less likely that you'll get a value of x equal to half an hour.
And so because of that values of theta closer to x are more likely.
That's why you get this decreasing function.
OK, so now let's continue and now what we have is this is the case for if you observe one data point.
So you arrange a date with Juliet, you observe how late she is, and you get one value of x.
And now suppose you want to get collect more data so you arrange say 10 dates with Juliet.
And for each one you observe how late she was.
So now we can collect multiple samples, say n samples of delays.
So x1 is her delay on the first date.
Xn is her delay on the nth date.
And x we can now just call a variable that's a collection of all of these.
And now the question is, how do you incorporate in all this information into updating your belief about theta?
And it's actually pretty analogous to what we've done here.
The important assumption that we make in this problem is that conditional on theta, all of these delays are in fact conditionally independent.
And that's going to help us solve this problem.
So the set up is essentially the same.
What we still need is a-- we still need the prior.
And the prior hasn't changed.
The prior is still uniform between 0 and 1.
The way the actual delays are generated is we still assume to be the same given conditional on theta, each one of these is conditionally independent, and each one is uniformly distributed between 0 and theta.
And so what we get is that this is going to be equal to-- you can also imagine this as a big joint PDF, joint conditional PDF of all the x's.
And because we said that they are conditionally independent given theta, then we can actually split this joint PDF into the product of a lot of individual conditional PDFs.
So this we can actually rewrite as PDF of x1 given theta times all the way through the condition PDF of xn given theta.
And because we assume that each one of these is-- for each one of these it's uniformly distributed between 0 and theta, they're all the same.
So in fact what we get is 1 over theta for each one of these.
And there's n of them.
So it's 1 over theta to the n. But what values of x is this valid for?
What values of x and theta?
Well what we need is that for each one of these, we need that theta has to be at least equal to whatever x you get.
Whatever x you observe, theta has to at least that.
So we know that theta has to at least equal to x1 and all the way through xn.
And so theta has to be at least greater than or equal to all these x's and otherwise this would be 0.
So let's define something that's going to help us.
Let's define x bar to be the maximum of all the observed x's.
And so what we can do is rewrite this condition as theta has to be at least equal to the maximum, equal to x bar.
All right, and now we can again apply Bayes' rule.
Bayes' rule will tell us what this posterior distribution is.
So again the numerator will be the prior times this conditional PDF over PDF of x. OK, so the numerator again, the prior is just one.
This distribution we calculated over here.
It's 1 over theta to the n. And then we have this denominator.
And again, we need to be careful to write down when this is actually valid.
So it's actually valid when x bar is greater than theta-- I'm sorry, x bar is less than or equal to theta, and otherwise it's zero.
So this is actually more or less complete.
Again we need to calculate out what exactly this denominator is but just like before it's actually just a scaling factor which is independent of what theta is.
So if we wanted to, we could actually calculate this out.
It would be just like before.
It would be the integral of the numerator, which is 1 over theta to the n d theta.
And we integrate theta from before, it was from x to 1.
But now we need to integrate from x bar to 1.
And if we wanted to, we can actually do others.
It's fairly simple calculus to calculate what this normalization factor would be.
But the main point is that the shape of it will be dictated by this 1 over theta to the n term.
And so now we know that with n pieces of data, it's actually going to be 1-- the shape will be 1 over theta to the n, where theta has to be at least greater than or equal to x bar.
Before it was actually just 1 over theta and has to be between x and 1.
So you can kind of see how the problem generalizes when you collect more data.
So now imagine that this is the new-- when you collect n pieces of data, the maximum of all the x's is here.
Well, it turns out that it's the posterior now is going to look something like this.
So it becomes steeper because it's 1 over theta to the n as opposed to 1 over theta.
And it's limited to be between x bar and 1.
And so with more data you're more sure of the range that theta can take on because each data points eliminates parts of theta, the range of theta that theta can't be.
And so you're left with just x bar to 1.
And you're also more certain.
So you have this kind of distribution.
OK, so this is kind of the posterior distribution which tells you the entire distribution of what the unknown parameter-- the entire distribution of the unknown parameter given all the data that you have plus the prior distribution that you have.
But if someone were to come to ask you, your manager asks you, well what is your best guess of what theta is?
It's less informative or less clear when you tell them, here's the distribution.
Because you still have a big range of what theta could be, it could be anything between x and 1 or x bar and 1.
So if you wanted to actually come up with a point estimate which is just one single value, there's different ways you can do it.
The first way that we'll talk about is the map rule.
What the map rule does is it takes the posterior distribution and just finds the value of the parameter that gives the maximum posterior distribution, the maximum point in the posterior distribution.
So if you look at this posture distribution, the map will just take the highest value.
And in this case, because the posterior looks like this, the highest value is in fact x.
And so theta hat map is actually just x.
And if you think about it, this kind of an optimistic estimate because you always assume that it's whatever, if Juliet were 30 minutes late then you assume that her delay is uniformly distributed between 0 and 30 minutes.
Well in fact, even though she arrived 30 minutes late, that could have been because she's actually distributed between 0 and 1 hour and you just happened to get 30 minutes.
But what you do is you always take kind of the optimistic, and just give her the benefit of the doubt, and say that was actually kind of the worst case scenario given her distribution.
So another way to take this entire posterior distribution and come up with just a single number, a point estimate, is to take the conditional expectation.
So you have an entire distribution.
So there's two obvious ways of getting a number out of this.
One is to take the maximum and the other is to take the expectation.
So take everything in the distribution, combine it and come up with a estimate.
So if you think about it, it will probably be something like here, would be the conditional distribution.
So this is called the LMS estimator.
And the way to calculate it is just like we said, you take the conditional expectation.
So how do we take the conditional expectation?
Remember it is just the value and you weight it by the correct distribution, in this case it's the conditional PDF of theta given x which is the posterior distribution.
And what do we integrate theta from?
Well we integrate it from x to 1.
Now if we plug this in, we integrate from x to 1, theta times the posterior.
The posterior we calculated earlier, it was 1 over theta times the absolute value of log x.
So the thetas just cancel out, and you just have 1 over absolute value of log x.
Well that doesn't depend on theta.
So what you get is just 1 minus x over absolute value of log x.
All right, so we can actually plot this, so we have two estimates now.
One is that the estimate is just theta-- the estimate is just x.
The other one is that it's 1 minus x over absolute value of log x.
So we can plot this and compare the two.
So here's x, and here is theta hat, theta hat of x for the two different estimates.
So here's you the estimate from the map rule which is whatever x is, we estimate that theta is equal to x.
So it just looks like this.
Now if we plot this, turns out that it looks something like this.
And so whatever x is, this will tell you what the estimate, the LMS estimate of theta would be.
And it turns out that it's always higher than the map estimate.
So it's less optimistic.
And it kind of factors in the entire distribution.
So because there are several parts to this problem, we're going to take a pause for a quick break and we'll come back and finish the problem in a little bit.
Welcome back.
So now we're going to finish the rest of this problem.
For part e, we've calculated what the map and LMS estimators are.
And now we're going to calculate what the conditional mean squared error is.
So it's a way to measure how good these estimators are.
So let's start out generically.
For any estimator theta hat, the conditional MSE is-- conditional mean squared error-- is equal to this.
It's the estimator minus the actual value squared conditioned on X being equal to some little x.
So the mean squared error.
So you take the error, which is the difference between your estimator and the true value, square it, and then take the mean.
And it's conditioned on the actual value of what x is.
Or, conditioned on the data that you get is.
So to calculate this, we use our standard definition of what conditional expectation would be.
So it's theta hat minus theta squared.
And we weight that by the appropriate conditional PDF, which in this case would be the posterior.
And we integrate this from x-- from theta equals x to theta equals 1.
Now, we can go through some algebra and this will tell us that this is theta hat squared minus 2 theta hat theta minus plus theta squared.
And this posterior we know from before is 1 over theta times absolute value of log x d theta.
And when we do out this integral, it's going to be-- we can split up in to three different terms.
So there's theta hat squared times this and you integrate it.
But in fact, this is just a conditional density.
When you integrate it from x to 1, this will just integrate up to 1 because it is a valid density.
So the first term is just theta hat squared.
Now, the second term is you can pull out of 2 theta hat and integrate theta times 1 over theta times absolute value of log of x d theta from x to 1.
And then the last one is integral of theta squared 1 over theta absolute value of log x d theta from x to 1.
OK, so we can do some more-- with some more calculus, we get a final answer is this.
So this will integrate to 1 minus x over absolute value of log x.
And this will integrate to 1 minus x squared over 2 times absolute value of log x.
So this tells us for any generic estimate theta hat, this would be what the conditional mean squared error would be.
Now, let's calculate what it actually is for the specific estimates that we actually came up with.
So for the MAP rule, the estimate of theta hat is just equal to x.
So when we plug that into this, we get that the conditional MSE is just equal to x squared minus 2x 1 minus x absolute value of log x plus 1 minus x squared over 2 times absolute value of log of x.
And for the LMS estimate, remember this was equal to-- theta hat was 1 minus x over absolute value of log x.
And so when you plug this particular theta hat into this formula, what you get is that the conditional mean squared error is equal to 1 minus x squared over 2 times absolute value of log of x minus 1 minus x over log of x quantity squared.
So these two expressions tells us what the mean squared error is for the MAP rule and the LMS rule.
And it's kind of hard to actually interpret exactly which one is better based on just these expressions.
So it's helpful to plot out what the conditional mean squared error is.
So we're plotting for x.
For each possible actual data that we observe-- data point that we observe, what is the mean squared error?
So let's do the MAP rule first.
The MAP rule would look something like this.
And it turns out that the LMS rule is better, and it will look like this dotted line here on the bottom.
And so it turns out that if your metric for how good your estimate is is the conditional mean squared error, then LMS is better than MAP.
And this is true because LMS is actually designed to actually minimize what this mean squared error is.
And so in this case, the LMS estimator should have a better mean squared error than the map estimator.
OK, now the last part of the problem, we calculate one more type of estimator, which is the linear LMS estimator.
So notice that the LMS estimator was this one.
It was 1 minus x over absolute value of log of x.
And this is not linear in x, which means sometimes it's difficult to calculate.
And so what we do is we tried to come up with a linear form of this, something that is like ax plus b, where a and b are some constant numbers.
But that also does well in terms of having a small mean squared error.
And so we know from the class that in order to calculate the linear LMS, the linear LMS we know we just need to calculate a few different parts.
So it's equal to the expectation of the parameter plus the covariance of theta and x over the variance of x times x minus expectation of x.
Now, in order to do this, we just need to calculate four things.
We need the expectation of theta, the covariance, the variance, and the expectation of x. OK, so let's calculate what these things are.
Expectation of theta.
We know that theta is uniformly distributed between 0 and 1.
And so the expectation of theta is the easiest one to calculate.
It's just 1/2.
What about the expectation of x?
Well, expectation of x is a little bit more complicated.
But remember, like in previous problems, it's helpful when you have a hierarchy of randomness to try to use the law of iterated expectations.
So the delay, which is x, is random.
But it's randomness depends on the actual distribution, which is theta.
Which itself is random.
And so let's try to condition on theta and see if that helps us.
OK, so if we knew what theta was, then what is the expectation of x?
Well, we know that given theta, x is uniformly distributed between 0 and theta.
And so the mean would be just theta over 2.
And so this would just be expectation of theta over 2.
And we know this is just 1/2 times the expectation of theta, which is 1/2.
So this is just 1/4.
Now, let's calculate the variance of x.
The variance of x takes some more work because we need to use the law of total variance, which is this.
That the variance of theta is equal to the expectation of the conditional variance plus the variance of the conditional expectation.
Let's see if we can figure out what these different parts are.
What is the conditional variance of x given theta?
Well, given theta, x we know is uniformly distributed between 0 and theta.
And remember for uniform distribution of width c, the variance of that uniform distribution is just c squared over 12.
And so in this case, what is the width of this uniform distribution?
Well, it's uniformly distributed between 0 and theta, so the width is theta.
So this variance should be theta squared over 12.
OK, what about the expectation of x given theta?
Well, we already argued earlier that the expectation of x given theta is just theta over 2.
So now let's fill in the rest.
What's the expectation of theta squared over 12?
Well, that takes a little bit more work because this is just-- you can think of it as 1/12.
You could pull the 1/12 out times the expectation of theta squared.
Well, the expectation of theta squared we can calculate from the variance of theta plus the expectation of theta quantity squared.
Because that is just the definition of variance.
Variance is equal to expectation of theta squared minus expectation of theta quantity squared.
So we've just reversed the formula.
Now, the second half is the variance of theta over 2.
Well, remember when you pull out a constant from a variance, you have to square it.
So this is just equal to 1/4 times the variance of theta.
Well, what is the variance of theta?
The variance of theta is the variance of uniform between 0 and 1.
So the width is 1.
So you get 1 squared over 12.
And the variance is 1/12.
What is the mean of theta?
It's 1/2 when you square that, you get 1/4.
Finally for here, the variance of theta like we said, is 1/12.
So you get 1/12.
And now, when you combine all these, you get that the variance ends up being 7/144.
Now we have almost everything.
The last thing we need to calculate is this covariance term.
What is the covariance of theta and x?
Well, the covariance we know is just the expectation of the product of theta and x minus the product of the expectations.
So the expectation of x times the expectation of theta.
All right, so we already know what expectation of theta is.
That's 1/2.
And expectation of x was 1/4.
So the only thing that we don't know is expectation of the product of the two.
So once again, let's try to use iterated expectations.
So let's calculate this as the expectation of this conditional expectation.
So we, again, condition on theta.
And minus the expectation of theta is 1/2.
Times 1/4, which is the expectation of x.
Now, what is this conditional expectation?
Well, the expectation of theta-- if you know what theta is, then the expectation of theta is just theta.
You already know what it is, so you know for sure that the expectation is just equal to theta.
And what is the expectation of x given theta?
Well, the expectation of x given theta we already said was theta over 2.
So what you get is this entire expression is just going to be equal to theta times theta over 2, or expectation of theta squared over 2 minus 1/8.
Now, what is the expectation of theta squared over 2?
Well, we know that-- we already calculated out what expectation of theta squared is.
So we know that expectation of theta squared is 1/12 plus 1/4.
So what we get is we need a 1/2 times 1/12 plus 1/4, which is 1/3 minus 1/8.
So the answer is 1/6 minus 1/8, which is 1/24.
Now, let's actually plug this in and figure out what this value is.
So when you get everything-- when you combine everything, you get that the LMS estimator is-- the linear LMS estimator is going to be-- expectation of theta is 1/2.
The covariance is 1/24.
The variance is 7/144.
And when you divide that, it's equal to 6/7 times x minus 1/4 because expectation of x is 1/4.
And you can simplify this a little bit and get that this is equal to 6/7 times x plus 2/7.
So now we have three different types of estimators.
The map estimator, which is this.
Notice that it's kind of complicated.
You have x squared terms.
You have more x squared terms.
And you have absolute value of log of x.
And then you have the LMS, which is, again, nonlinear.
And now you have something that looks very simple-- much simpler.
It's just 6/7 x plus 2/7.
And that is the linear LMS estimator.
And it turns out that you can, again, plot these to see what this one looks like.
So here is our original plot of x and theta hat.
So the map estimator-- sorry, the map estimator was just theta hat equals x.
This was the mean squared error of the map estimator.
So the map estimator is just this diagonal straight line.
The LMS estimator looked like this.
And it turns out that the linear LMS estimator will look something like this.
So it is fairly close to the LMS estimator, but not quite the same.
And note, especially that depending on what x is, if x is fairly close to the 1, you might actually get an estimate of theta that's greater than 1.
So for example, if you observe that Julian is actually an hour late, then x is 1 and your estimate of theta from the linear LMS estimator would be 8/7, which is greater than 1.
That doesn't quite make sense because we know that theta is bounded to be only between 0 and 1.
So you shouldn't get an estimate of theta that's greater than 1.
And that's one of the side effects of having the linear LMS estimator.
So that sometimes you will have an estimator that doesn't quite make sense.
But what you get instead when sacrificing that is you get a simple form of the estimator that's linear.
And now, let's actually consider what the performance is.
And it turns out that the performance in terms of the conditional mean squared error is actually fairly close to the LMS estimator.
So it looks like this.
Pretty close, pretty close, until you get close to 1.
In which case, it does worse.
And it does worse precisely because it will come up with estimates of theta which are greater than 1, which are too large.
But otherwise, it does pretty well with a estimator that is much simpler in form than the LMS estimator.
So in this problem, which had several parts, we actually went through, basically, all the different concepts and tools within Chapter Eight for Bayesian inference.
We talked about the prior, the posterior, calculating the posterior using the Bayes' rule.
We calculated the MAP estimator.
We calculated the LMS estimator.
From those, we calculated what the mean squared error for each one of those and compared the two.
And then, we looked at the linear LMS estimator as another example and calculated what that estimator is, along with the mean squared error for that and compared all three of these.
So I hope that was a good review problem for Chapter Eight, and we'll see you next time.
Welcome back guys.
Today we're going to work on a problem that tests your knowledge of joint PMFs.
And we're also going to get some practice computing conditional expectations and conditional variances.
So in this problem, we are given a set of points in the xy plane.
And we're told that these points are equally likely.
So there's eight of them.
And each point has a probability of 1/8 of occurring.
And we're also given this list of questions.
And we're going to work through them together.
So in part a, we are asked to find the values of x that maximize the conditional expectation of y given x.
So jumping right in, this is the quantity we're interested in.
And so this quantity is a function of x.
You plug-in various values of x.
And then this will spit out a scalar value.
And that value will correspond to the conditional expectation of y conditioned on the value of x that you put in.
So let's see, when x is equal to 0, for instance, let's figure out what this value is.
Well, when x is equal to 0 we're living in a world, essentially, on this line.
So that means that only these two points could have occurred.
And in particular, y can only take on the values of 1 and 3.
Now, since all these points in the unconditional universe were equally likely, in the conditional universe they will still be equally likely.
So this happens with probability 1/2.
And this happens with probability 1/2.
And therefore, the expectation would just be 3/2 plus 1/2 which is 4/2, or 2.
But a much faster way of seeing this-- and it's the strategy that I'm going to use for the rest of the problem-- is to remember that expectation acts like center of mass.
So the center of mass, when these two points are equally likely, is just the midpoint, which of course is 2.
So we're going to use that intuition on the other ones.
So I'm skipping to x is equal to 2 because 1 and 3 are not possible.
So when x is equal to 2, y can only take on the values of 1 or 2.
Again, they're equally likely.
So the center of mass is in the middle which happens at 1.5 or 3/2.
Similarly, x is equal to 4.
We're living in this conditional universe, where y can take on of these four points with probability 1/4 each.
And so again, we expect the center of mass to be at 1.5 or 3/2.
And this quantity is undefined otherwise.
OK, so we're almost done.
Now we just need to find which value of x maximizes this.
Well, let's see, 2 is the biggest quantity out of all of these numbers.
So the maximum is 2.
And it occurs when x is equal to 0.
So we come over here.
And we found our answer.
x is equal to 0 is the value, which maximizes the conditional expectation of y given x.
So part b is very similar to part a.
But there is slightly more computation involved.
Because now we're dealing with the variance and not an expectation.
And variance is usually a little bit tougher to compute.
So we're going to start in the same manner.
But I want you guys to see if you can figure out intuitively what the right value is.
I'm going to do the entire computation now.
And then you can compare whether your intuition matches with the real results.
So variance of x conditioned on a particular value of y, this is now a function of y.
For each value of y you plug in you're going to get out a scalar number.
And that number represents the conditional variance of x when you condition on the value of y that you plugged in.
So let's see, when y is equal to 0 we have a nice case.
If y is equal to 0 we have no freedom about what x is.
This is the only point that could have occurred.
Therefore, x definitely takes on a value of 4.
And there's no uncertainty left.
So in other words, the variance is 0.
Now, if y is equal to 1, x can take on a value of 0, a value of 2 or a value of 4.
And these all have the same probability of occurring, of 1/3, And again, the reasoning behind that is that all eight points were equally likely in the unconditional universe.
If you condition on y being equal to 1 these outcomes still have the same relative frequency.
Namely, they're still equally likely.
And since there are three of them they now have a probability of 1/3 each.
So we're going to go ahead and use a formula that hopefully, you guys remember.
So in particular, variance is the expectation of x squared minus the expectation of x all squared, the whole thing squared.
So let's start by computing this number first.
So conditioned on y is equal to 1-- so we're in this line-- the expectation of x is just 2, right?
The same center-of-mass to argument.
So this, we have a minus 2 squared over here.
Now, x squared is only slightly more difficult.
With probability 1/3, x squared will take on a value of 0.
With probability 1/3, x squared will take on a value of 4.
I'm just doing 2 squared.
And with probability 1/3, x squared takes on a value of 4 squared or 16.
So writing down when I just said, we have 0 times 1/3 which is 0.
We have 2 squared, which is 4 times 1/3.
And then we have 4 squared, which is 16 times 1/3.
And then we have our minus 4 from before.
So doing this math out, we get, let's see, 20/3 minus 12/3, which is equal to 8/3, or 8/3.
So we'll come back up here and put 8/3.
So I realize I'm going through this pretty quickly.
Hopefully this step didn't confuse you.
Essentially, when I was doing is, if you think of x squared as a new random variable, x squared, the possible values that it can take on are 0, 4, and 16 when you're conditioning on y is equal to 1.
And so I was simply saying that that random variable takes on those values with equal probability.
So let's move on to the next one.
So if we condition on y is equal to 2 we're going to do a very similar computation.
Oops, I shouldn't have erased that.
OK, so we're going to use the same formula that we just used, which is the expectation of x given y is equal to 2.
Sorry, x squared minus the expectation of x conditioned on y is equal to 2, all squared.
So conditioned on y is equal to 2, the expectation of x is 3.
Same center of mass argument.
So 3 squared is 9.
And then x squared can take on a value of 4.
Or it can take on a value of 16.
And it does so with equal probability.
So we get 4/2, 4 plus 16 over 2.
So this is 2 plus 8, which is 10, minus 9.
That'll give us 1.
So we get a 1 when y is equal to 2.
And last computation and then we're done.
I'm still recycling the same formula.
But now we're conditioning on y is equal to 3.
And then we'll be done with this problem, I promise.
OK, so when y is equal to 3 x can take on the value of 0.
Or it can take on the value of 4.
Those two points happen with probability 1/2, 1/2.
So the expectation is right in the middle which is 2.
So we get a minus 4.
And similarly, x squared can take on the value of 0.
When x takes on the value of 0-- and that happens with probability 1/2-- similarly, x squared can take on the value of 16 when x takes on the value of 4.
And that happens with probability 1/2.
So we just have 0/2 plus 16/2 minus 4.
And this gives us 8 minus 4, which is simply 4.
So finally, after all that computation, we are done.
We have the conditional variance of x given y.
Again, we're interested in when this value is largest.
And we see that 4 is the biggest value in this column.
And this value occurs when y takes on a value of 3.
So our answer, over here, is y is equal to 3.
All right, so now we're going to switch gears in part c and d a little bit.
And we're going to be more concerned with PMFs, et cetera.
So in part c, we're given a random variable called r which is defined as the minimum of x and y.
So for instance, this is the 0.01.
The minimum of 0 and 1 is 0.
So r would have a value of 0 here.
Now, we can be a little bit smarter about this.
If we plot the line, y is equal to x.
So that looks something like this.
We see that all of the points below this line satisfy y being less or equal to x.
And all the points above this line have y greater than or equal to x.
So if y is less than or equal to x, you hopefully agree that here the min, or r, is equal to y.
But over here, the min, r, is actually equal to x, since x is always smaller.
So now we can go ahead quickly.
And I'm going to write the value of r next each point using this rule.
So here, r is the value of y, which is 1.
Here, r is equal to 0.
Here r is 1.
Here r is 2.
Here r is 3.
Over here, r is the value of x.
So r is equal to 0.
And r is equal to 0 here.
And so the only point we didn't handle is the one that lies on the line.
But in that case it's easy.
Because x is equal to 2.
And y is equal to 2.
So the min is simply 2.
So with this information I claim we're now done.
We can just write down what the PMF of r is.
So in particular, r takes on a value of 0.
When this point happens, this point happens, or this point happens.
And those collectively have a probability of 3/8 of occurring.
r can take on a value of 1 when either of these two points happen.
So that happens with probability 2/8.
r is equal to 2.
This can happen in two ways.
So we get 2/8.
And r equal to 3 can happen in only one way.
So we get 1/8.
Quick sanity check, 3 plus 2 is 5, plus 2 is 7, plus 1 is 8.
So our PMF sums to 1.
And to be complete, we should sketch it.
Because the problem asks us to sketch it.
So we're plotting PR of r, 0, 1, 2, 3.
So here we get, let's see, 1, 2, 3.
For 0 we have 3/8.
For 1 we have 2/8.
For 2 we have 2/8.
And for 3 we have 1/8.
So this is our fully labeled sketch of Pr of r. And forgive me for erasing so quickly, but you guys can pause the video, presumably, if you need more time.
Let's move on to part d. So in part d we're given an event named a, which is the event that x squared is greater than or equal to y.
And then we're asked to find the expectation of xy in the unconditional universe.
And then the expectation of x times y conditioned on a.
So let's not worry about the conditioning for now.
Let's just focus on the unconditional expectation of x times y.
So I'm just going to erase all these r's so I don't get confused.
But we're going to follow a very similar strategy, which is at each point I'm going to label what the value of w is.
And we'll find the expectation of w that way.
So let's see, here, we have 4 times 0.
So w is equal to 0.
Here we have 4 times 1. w is equal to 4.
4 times 2, w is equal to 8.
4 times 3, w is equal to 12. w is equal to 2. w is equal to 4. w is equal to 0. w is equal to 0.
OK, so that was just algebra.
And now, I claim again, we can just write down what the expectation of x times y is.
And I'm sorry, I didn't announce my notation.
I should mention that now.
I was defining w to be the random variable x times y.
And that's why I labeled the product of x times y as w over here.
My apologies about not defining that random variable.
So the expectation of w, well, w takes on a value of 0.
When this happens, this happens or that happens.
And we know that those three points occur with probability 3/8.
So we have 0 times 3/8.
I'm just using the normal formula for expectation.
w takes on a value of 2 with probability 1/8.
Because this is the lead point in which it happens, 2 times 1/8.
Plus it can take on the value of 4 with probability 2/8, 4 times 2/8.
And 8, with 1/8 probability.
And similarly, 12 with 1/8 probability.
So this is just algebra.
The numerator sums up to 30.
Yes, that's correct.
So we have 30/8, which is equal to 15/4.
So this is our first answer for part d. And now we have to do this slightly trickier one, which is the conditional expectation of x times y, or w conditioned on a.
So similar to what I did in part c, I'm going to draw the line y equals x squared.
So y equals x squared is 0 here, 1 here.
And at 2, it should take on a value of 4.
So the curve should look something like this.
This is the line y is equal to x squared.
So we know all the points below this line satisfy y less than or equal to x squared.
And all the points above this line have y greater than or equal to x squared.
And a is y less than or equal to x squared.
So we are in the conditional universe where only points below this line can happen.
So that one, that one, that one, that one, that one and that one.
So there are six of them.
And again, in the unconditional world, all of the points were equally likely.
So in the conditional world these six points are still equally likely.
So they each happen with probability 1/6.
So in this case, the expectation of w is simply 2 times 1/6 plus 0 times 1/6.
But that's 0.
So I'm not going to write it.
4 times 2/6 plus 4 times 2/6 plus 8 times 1/6, plus 12 times 1 over 6.
And again, the numerator summed to 30.
But this time our denominator is 6.
So this is simply 5.
So we have, actually, finished the problem.
Because we've computed this value and this value.
And so the important takeaways of this problem are, essentially, honestly, just to get you comfortable with computing things involving joint PMFs.
We talked a lot about finding expectations quickly by thinking about center of mass and the geometry of the problem.
We've got practice computing conditional variances.
And we did some derived distributions.
And we'll do a lot more of those later.
Hey, guys.
Welcome back.
Today, we're going to do another fun problem, which is a drill problem on joint PMFs.
And the goal is that you will feel more comfortable by the end of this problem, manipulating joint PMFs.
And we'll also review some ideas about independents in the process.
So just to go over what I've drawn here, we are given an xy plane.
And we're told what the PMF is.
And it's plotted for you here.
What these stars indicate is simply that there is a value there.
But we don't know what it is.
It could be anything between 0 and 1.
And so we're given this list of questions.
And we're just going to work through them linearly together.
So we start off pretty simply.
We want to compute, in part a, the probability that x takes on a value of 1.
So for those of you who like formulas, I'm going to use the formula, which is usually referred to as marginalization.
So the marginal over x is given by summing over the joint.
So here we are interested in the probability that x is 1.
So I'm just going to freeze the value of 1 here.
And we sum over y.
And in particular, 1, 2, and 3.
So carrying this out, this is the Pxy of 1, 1, plus Pxy of 1, 2, plus Pxy 1, 3.
And this, of course, reading from the graph, is 1/12 plus 2/12 plus 1/12, which is equal to 4/12, or 1/3.
So now you guys know the formula.
Hopefully you'll remember the term marginalization.
But I want to point out that intuitively you can come up with the answer much faster.
So the probability that x is equal to 1 is the probability that this dot happens or this dot happens or this dot happens.
Now, these dots, or outcomes, they're disjoint.
So you can just sum the probability to get the probability of one of these things happening.
So it's the same computation.
And you'll probably get there a little bit faster.
So we're done with a already, which is great.
So for part b, conditioning on x is equal to 1, we want to sketch the PMF of y.
So if x is equal to 1 we are suddenly living in this universe.
y can take values of 1, 2, or 3 with these relative frequencies.
So let's draw this here.
So this is y. I said, already, y can take on a value of 1. y can take on a value of 2.
Or it can take on a value of 3.
And we're plotting here, Py given x, y, conditioned on x is equal to 1.
OK, so what I mean by preserving the relative frequencies is that in unconditional world this is dot is twice as likely to happen as either this dot or this dot.
And that relative likelihood remains the same after conditioning.
And the reason why we have to change these values is because they have to sum to 1.
So in other words, we have to scale them up.
So you can use a formula.
But again, I'm here to show you faster ways of thinking about it.
So my little algorithm for figuring out conditional PMFs is to take the numerators-- so 1, 2, and 1-- and sum them.
So here that gives us 4.
And then to preserve the relative frequency, you actually keep the same numerators but divide it by the sum, which you just computed.
So I'm going fast.
I'll review in a second.
But this is what you will end up getting.
So to recap, I did 1 plus 2 plus 1, which is 4, to get these denominators.
And so I skipped a step here.
This is really 2/4, which is 1/2, obviously.
So you add these guys to get 4.
And then you keep the numerators and just divide them by 4.
So 1/4, 2/4, which is 1/2 and 1/4.
And that's what we mean by preserving the relative frequency.
Except so this thing now sums to 1, which is what we want.
OK, so we're done with part b.
Part c actually follows almost immediately from part b.
In part c we're interested in computing the conditional expectation of y given that x is equal to 1.
So we've already done most of the legwork because we have the conditional PMF that we need.
And so expectation, you guys have calculated a bunch of these by now.
So I'm just going to appeal to your intuition and to symmetry.
Expectation acts like center of mass.
This is a symmetrical distribution of mass.
And so the center is right here at 2.
So this is simply 2.
And if that went too fast, just convince yourselves.
Use the normal formula for expectations.
And your answer will agree with ours.
OK, so d is a really cool question.
Because you can do a lot of math.
Or you can think and ask yourself, at the most fundamental level, what is independents?
And if you think that way you'll come to the answer very easily.
So essentially, I rephrased this to truncate it from the problem statement that you guys are reading.
But the idea is that these stars are unknown probability masses.
And this question is asking can you figure out a way of assigning numbers between 0 and 1 to these values such that you end up with a valid probability mass function, so everything sums to 1 and such that x and y are independent?
So it seems hard a priori.
But let's think about it a bit.
And in the meantime I'm going to erase this so I have more room.
What does it mean for x and y to be independent?
Well, it means that they don't, essentially, have information about each other.
So if I tell you something about x and if x and y are independent, your belief about y shouldn't change.
In other words, if you're a rational person, x shouldn't change your belief about y.
So let's look more closely at this diagram.
Now, the number 0 should be popping out to you.
Because this essentially means that the 0.31 can't happen.
Or it happens with 0 probability.
So let's say fix x equal to 3.
If you condition on x is equal to 3, as I just said, this outcome can't happen.
So y could only take on values of 2 or 3.
However, if you condition on x is equal to 1, y could take on a value of 1 with probability 1/4 as we computed in the other problem.
It could take on a value of 2 with probability of 1/2.
Or it could take on a value of 3 with probability 1/4.
So these are actually very different cases, right?
Because if you observe x is equal to 3 y can only be 2 or 3.
But if you observe x is equal to 1, y can be 1, 2, or 3.
So actually, x, no matter what values these stars have on, x always tells you something about y.
Therefore, the answer to this, part d, is no.
So let's put a no with an exclamation point.
So I like that problem a lot.
And hopefully it clarifies independents for you guys.
So parts e and f, we're going to be thinking about independents again.
To go over what the problem statement gives you, we defined this event, b, which is the event that x is less than or equal to 2 and y is less than or equal to 2.
So let's get some colors.
So do bright pink.
So that means we're essentially living in this world.
There's only those four dots.
And we're also told a very important piece of information that conditions on B. x and y are conditionally independent.
OK, so part e, now that we have this.
And by the way, these two assumptions apply to both parts e and part f. So in part e, we want to find out Pxy of 2, 2.
Or in English, what is the probability that x takes on a value of 2 and y takes on a value of 2?
So determine the value of this star.
And the whole trick here is that the possible values that this star could take on are constrained by the fact that we need to make sure that x and y are conditionally independent given B.
So my claim is that if two random variables are independent and you condition on one of them, say we condition on x.
If you condition on different values of x, the relative frequencies of y should be the same.
So here, the relative frequency, condition on x is equal to 1.
The relative frequencies of y are 2 to 1.
This outcome is twice as likely to happen as this one.
If we condition on 2 this outcome needs to be twice as likely to happen as this outcome.
If they weren't, x would tell you information about y.
Because you would know that the distribution over 2 and 1 would be different.
OK?
So because the relative frequencies have to be the same and 2/12 is 2 times 1/12 this guy must also be 2 times 2/12.
So that gives us our answer for part e. Let me write up here.
Part e, we need Pxy 2, 2 to be equal to 4/12.
And again, the way we got this is simply we need x and y to be conditionally independent given B.
And if this were anything other than 4 the relative frequency of y is equal to 2 to 1 would be different from over here.
So here condition on x is equal to 1.
The outcome, y is equal to 2 is twice as likely as x is equal to 1.
Here, if we put a value of 4/12 and you condition on x is equal to 2, the outcome y is equal to 2 is still twice as likely as the outcome y is equal to 1.
And if you put any other number there the relative frequencies would be different.
So x would be telling you something about y.
So there would not be independent condition on B. OK, that was a mouthful.
But hopefully you guys have it now.
And lastly, we have part f, which follows pretty directly from part e. So we were still in the unconditional universe.
In part e, we were figuring out what's the value of star in the whole unconditional universe?
Now, in part f, we want the value of star in the conditional universe where B occurred.
So let's come over here and plot a new graph so we don't confuse ourselves.
So we have xy.
x can be 1 or 2.
Y could be 1 or 2.
So we have a plot that looks something like this.
And so again, same argument as before.
Let me just fill this in.
From part e, we have that this is 4/12.
And we're going to use my algorithm again.
So in the conditional world, the relative frequencies of these four dots should be the same.
But you need to scale them up so that if you sum over all of them the probability sums to 1.
So you have a valid PMF.
So my algorithm from before was to add up all the numerators.
So 1 plus 2 plus 4 plus 2 gives you 9.
And then to preserve the relative frequency you keep the same numerator.
So here we had a numerator of 1.
That becomes 1/9.
Here we had a numerator of 2.
This becomes 2/9.
Here we had a numerator of 4.
That becomes 4/9.
Here we had a numerator of 2, so 2/9.
And indeed, the relative frequencies are preserved.
And they all sum to 1.
So our answer for part f-- let's box it here-- is that Pxy 2, 2 conditioned on B is equal to 4/9, is just that guy.
So we're done.
Hopefully that wasn't too painful.
And this is a good drill problem, because we got more comfortable working with PMFs, joint PMFs.
We went over marginalization.
We went over conditioning.
We went over into independents.
And I also gave you this quick algorithm for figuring out what conditional PMFs are if you don't want to use the formulas.
Namely, you sum all of the numerators to get a new denominator and then divide all the old numerators by the new denominator you computed.
So I hope that was helpful.
I'll see you next time.
Hey guys.
Welcome back.
Today we're going to do a fun problem that will test your knowledge of the law of total variance.
And in the process, we'll also get more practice dealing with joint PDFs and computing conditional expectations and conditional variances.
So in this problem, we are given a joint PDF for x and y.
So we're told that x and y can take on the following values in the shape of this parallelogram, which I've drawn.
And moreover, that x and y are uniformly distributed.
So the joint PDF is just flat over this parallelogram.
And because the parallelogram has an area of 1, the height of the PDF must also be 1 so that the PDF integrates to 1.
OK. And then we are asked to compute the variance of x plus y.
So you can think of x plus y as a new random variable whose variance we want to compute.
And moreover, we're told we should compute this variance by using something called the law of total variance.
So from lecture, you should remember or you should recall that the law of total variance can be written in these two ways.
And the reason why there's two different forms for this case is because the formula always has you conditioning on something.
Here we condition on x, here we condition on y.
And for this problem, the logical choice you have for what to condition on is x or y.
So again, we have this option.
And my claim is that we should condition on x.
And the reason has to do with the geometry of this diagram.
So notice that if you freeze an x and then you sort of vary x, the width of this parallelogram stays constant.
However, if you condition on y and look at the width this way, you see that the width of the slices you get by conditioning vary with y.
So to make our lives easier, we're going to condition on x.
And I'm going to erase this bottom one, because we're not using it.
So this really can seem quite intimidating, because we have nested variances and expectations going on, but we'll just take it slowly step by step.
So first, I want to focus on this term-- the conditional expectation of x plus y conditioned on x.
So coming back over to this picture, if you fix an arbitrary x in the interval, 0 to 1, we're restricting ourselves to this universe.
So y can only vary between this point and this point.
Now, I've already written down here that the formula for this line is given by y is equal to x.
And the formula for this line is given by y is equal to x plus 1.
So in particular, when we condition on x, we know that y varies between x and x plus 1.
But we actually know more than that.
We know that in the unconditional universe, x and y were uniformly distributed.
So it follows that in the conditional universe, y should also be uniformly distributed, because conditioning doesn't change the relative frequency of outcomes.
So that reasoning means that we can draw the conditional PDF of y conditioned on x as this.
We said it varies between x and x plus 1.
And we also said that it's uniform, which means that it must have a height of 1.
So this is py given x, y given x.
Now, you might be concerned, because, well, we're trying to compute the expectation of x plus y and this is the conditional PDF of y, not of the random variable, x plus y.
But I claim that we're OK, this is still useful, because if we're conditioning on x, this x just acts as a constant.
It's not really going to change anything except shift the expectation of y by an amount of x.
So what I'm saying in math terms is that this is actually just x plus the expectation of y given x.
And now our conditional PDF comes into play.
Conditioned on x, this is the PDF of y.
And because it's uniformly distributed and because expectation acts like center of mass, we know that the expectation should be the midpoint, right?
And so to compute this point, we simply take the average of the endpoints, x plus 1 plus x over 2, which gives us 2x plus 1 over 2.
So plugging this back up here, we get 2x/2 plus 2x plus 1 over 2, which is 4x plus 1 over 2, or 2x plus 1/2.
OK.
So now I want to look at the next term, the next inner term, which is this guy.
So this computation is going to be very similar in nature, actually.
So we already discussed that the joint-- sorry, not the joint, the conditional PDF of y given x is this guy.
So the variance of x plus y conditioned on x, we sort of have a similar phenomenon occurring.
x now in this conditional world just acts like a constant that shifts the PDF but doesn't change the width of the distribution at all.
So this is actually just equal to the variance of y given x, because constants don't affect the variance.
And now we can look at this conditional PDF to figure out what this is.
So we're going to take a quick tangent over here, and I'm just going to remind you guys that we have a formula for computing the variance of a random variable when it's uniformly distributed between two endpoints.
So say we have a random variable whose PDF looks something like this.
Let's call it, let's say, w. This is pww.
We have a formula that says variance of w is equal to b minus a squared over 12.
So we can apply that formula over here.
b is x plus 1, a is x.
So b minus a squared over 12 is just 1/12.
So we get 1/12.
So we're making good progress, because we have this inner quantity and this inner quantity.
So now all we need to do is take the outer variance and the outer expectation.
So writing this all down, we get variance of x plus y is equal to variance of this guy, 2x plus 1/2 plus the expectation of 1/12.
So this term is quite simple.
We know that the expectation of a constant or of a scalar is simply that scalar.
So this evaluates to 1/12.
And this one is not bad either.
So similar to our discussion up here, we know constants do not affect variance.
You know they shift your distribution, they don't change the variance.
So we can ignore the 1/2.
This scaling factor of 2, however, will change the variance.
But we know how to handle this already from previous lectures.
We know that you can just take out this scalar scaling factor as long as we square it.
So this becomes 2 squared, or 4 times the variance of x plus 1/12.
And now to compute the variance of x, we're going to use that formula again, and we're going to use this picture.
So here we have the joint PDF of x and y, but really we want now the PDF of x, so we can figure out what the variance is.
So hopefully you remember a trick we taught you called marginalization.
To get the PDF of x given a joint PDF, you simply marginalize over the values of y.
So if you freeze x is equal to 0, you get the probability density line over x by integrating over this interval, over y.
So if you integrate over this strip, you get 1.
If you move x over a little bit and you integrate over this strip, you get 1.
This is the argument I was making earlier that the width of this interval stays the same, and hence, the variance stays the same.
So based on that argument, which was slightly hand wavy, let's come over here and draw it.
We're claiming that the PDF of x, px of x, looks like this.
It's just uniformly distributed between 0 and 1.
And if you buy that, then we're done, we're home free, because we can apply this formula, b minus a squared over 12, gives us the variance.
So b is 1, a is 0, which gives variance of x is equal to 1/12.
So coming back over here, we get 4 times 1/12 plus 1/12, which is 5/12.
And that is our answer.
So this problem was straightforward in the sense that our task was very clear.
We had to compute this, and we had to do so by using the law of total variance.
But we sort of reviewed a lot of concepts along the way.
We saw how, given a joint PDF, you marginalize to get the PDF of x.
We saw how constants don't change variance.
We got a lot of practice finding conditional distributions and computing conditional expectations and variances.
And we also saw this trick.
And it might seem like cheating to memorize formulas, but there's a few important ones you should know.
And it will help you sort of become faster at doing computations.
And that's important, especially if you guys take the exams.
So that's it.
See you next time.
Hi, everyone.
Today, I'm going to talk about Markov Chain Practice number one.
Before we start, let's first take a look at this Markov chain.
This Markov chain has six states.
In this problem, we always assume the process starts from state S0.
On the first trial, the process can either make a transition from S0 to S1 with probability 1/3 or from S0 to S3 with probability 1/3 third or from S0 to S5 with probability 1/3.
If on the first trial, the process makes the transition from S0 to S1 or from S0 to S5, it will always be stuck in either S1 or S5 forever, because both of the states S1 and S5 have a self-transition probability of one.
On the other hand, if on the first trial, the process makes the transition from S0 to S3, it can then either transition to the left or transition to the right or make self-transition back to the state S3.
If the process ever enters the left of the chain, it will never be able to come to the right.
On the other hand, if the process ever enters the right of the chain, it would never be able to go to the left.
For part A of the problem, we have to calculate the probability that the process enter S2 for the first time at the case trial.
First, notice that it would take at least two trials for the process to make a transition from S0 to S2.
Therefore, for k equal to 1, the probability of ak is simply equal to 0.
For k equal to 1, probability of a1 is equal to 0.
Then for k equal to 2, 3 and on, the probability that the process enters S2 for the first time at a case trial is equivalent to the probability that the process first makes a transition from S0 to S3 and then stays in S3 for the next two k minus 2 trials and finally makes a transition from S3 to S2 on the kth trial.
So let's write this out.
For k equal to 2, 3, and on, the probability of ak is equal to the probability that the process first makes transition from S0 to S3 on the first trial, which is probability 03, times the probability that the process makes self-transition for the next k minus 2 trials, which is probability 33 to the power of k minus 2, and finally makes a transition from S3 to S2 on the kth trial, which is p32.
And this gives us 1/3 times 1/4 to the power of k minus 2 times 1/4, which is equal to 1/3 times 1/4 to the power of k minus-- For part B of the problem, we have to calculate the probability that the process never enters as four.
This event can happen in three ways.
The first way is that the process makes a transition from S0 to S1 on the first trial and be stuck in S1 forever.
The second way that the process makes a transition from S0 to S5 on the first trial and be stuck at S5 forever.
The third way is that the process makes a transition from S0 to S3 on the first trial and then it makes a transition from S3 to S2 on the next state change so that it would never be able to go to S4.
Therefore, the probability of B is equal to the sum of probabilities of this three events.
So the probability of B is equal to the probability that the process makes a transition from S0 to S1 on the first trial, which is 1/3, plus the probability that the process makes a transition from S0 to S5 on the first trial, which is also 1/3, plus the probability that the process makes a transition from S0 to S3 on the first trial times the probability that the process then makes a transition from S3 to S2 on the next state change.
So transition to S2, given that the processes are already in state S3 and there's a state change.
Let's take a look at this conditional probability.
The condition that the processes are already in state S3 and there's a state change imply two possible events, which are the transition from S3 to S2 and the transition from S3 to S4.
Therefore, we can write this conditional probability as the conditional probability of transition from as S3 to S2, given that another event, S3 to S2 or S3 to S4 has happened.
And this is simply equal to the proportion of p32 and p32 plus p34, which is equal to 1/4 over 1/4 plus 1/2, which is equal to 1/3.
Therefore, the probability of B is equal to 1/3 plus 1/3 plus 1/3 times the 1/3 here, which is equal to 7/9.
For part C of the problem, we have to calculate the probability that the process enter S2 and leaves S2 on the next trial.
This probability can be written as the product of two probabilities-- the probability that the process enters S2 and the probability that it leaves S2 on the next trial, given it's already in S2.
Let's first look at the probability that the process enters S2.
Using a similar approach as part B, we know that the probability the process ever enters S2 is equal to the probability of the event that the process first makes a transition from S0 to S3 on the first trial and then makes a transition from S3 to S2 on the next state change.
So the probability that the process enters S2 is equal to the probability that it first makes a transition from S0 to S3 on the first trial, which is P03, times the probability that it makes a transition to S2, given that it's already in S3 and there is a state change.
We have already calculated this conditional probability in part B.
Let's then look at the second probability term, the probability that the process leaves S2 on the next trial, given that it's already in S2.
So given that the process is already in S2, it can take two transitions.
In can either transition from S2 to S1 or make a self-transition from S2 back to S2.
Therefore, this conditional probability that it leaves S2 on the next trial, given that it was already in S2 is simply equal to the transition probability from S2 to S1, which is P21.
Therefore, this is equal to P03, which is 1/3, times 1/3 from the result from part B times P21, which is 1/2, and gives us 1/18.
For part D of the problem, we have to calculate the probability that the process enters S1 for the first time on the third trial.
So if you take a look at this Markov chain, you'll notice that the only way for this event to happen is when a process first makes a transition from S0 to S3 on the first trial and from S3 to S2 on the second trial and from S2 to S1 on the third trial.
Therefore, the probability of D is equal to the probability of the event that the process makes a transition from S0 to S3 on the first trial and from S3 to S2 on the second trial and finally from S2 to S1 on the third trial.
So this is equal to P03 times P32 times P21, which is equal to 1/3 times 1/4 times 1/2, which is equal to 1/24.
For part E of the problem, we have to calculate the probability that the process is in S3 immediately after the nth trial.
If you take a look at this Markov chain, you'll notice that if on the first trial, the process makes a transition from S0 to S1 or from S0 to S5, it will never be able to go to S3.
On the other hand, if on the first trial, the process makes a transition from S0 to S3 and if it leaves S3 at some point, it will never be able to come back to S3.
Therefore, in order for the process to be S3 immediately after the nth trial, we will need the process to first make transition from S0 to S3 on the first trial and then stay in S3 for the next n minus 1 trials.
Therefore, the probability of the event e is simply equal to the probability of this event, which is equal to P03 times P33 to the power n minus 1, which is equal to 1/3 times 1/4 to the power of n minus 1.
And this concludes our practice on Markov chain today.
Hi.
In this video, we're going to compute some useful quantities for the exponential random variable.
So we're given that x is exponential with rate lambda.
PDF looks like this, and the formula is here.
First question, part a, what's the CDF?
So let's go right in.
The CDF of x is the probability that X is less than or equal to little x.
Let's look at some cases here.
What if little x is less than 0?
Well, x random variable only takes on these non-negative values.
And so the probability that X is less than or equal to some negative number is going to be 0.
On the other hand, if x is greater than or equal to 0, we do actually have to integrate here.
So to do that, we take the integral from minus infinity to x of fx of t-- the dummy variable here used is t. Notice that again, fx of t is going to be 0 for negative values, so we take the integral here from 0.
And now we plug in for fx of t. That's minus lambda t dt.
And recall that the integral of u to the a t is 1 over a times e to the a t. So here in this case, we'll get lambda, which is just a constant.
And then a here is going to be negative lambda.
So we get this, 0 to x. Lambdas cancel and we actually get 1 minus e to the minus lambda x.
So do this.
And we are done with the CDF.
Now for the expectation.
We use the standard formula, which is minus infinity to infinity t times fx of t dt.
So again, fx of t is going to be 0 for a negative value.
So we do the integral from 0.
We get 0 to infinity t lambda e to the minus lambda t dt.
Now, you can try all you want to get rid of this t. It's not going to go even if you try all kinds of u substitution.
But at the end the day, you're going to have to pull out your calculus textbook and find the integration by parts formula, which is-- v du.
So the hope is that this integral is going to be easier than the one on the left.
Notice that this is the integral of one of the terms here.
And this is the derivative of one of the terms.
So that may help you decide on how you select u and v. In our case actually, I'm going to use u as-- t for u.
Because when you take the derivative, it's going to become 1.
And the derivative is what's going to go in that integral.
So this is going to be dt for du.
And then, dv I'm going to select as whatever's left over.
It's lambda e to minus lambda t dt.
So v is going to be-- we already did the integral-- minus e to the minus lambda t. And so if we do this, it's going to be negative t times e to the minus lambda t. So that's uv.
Minus v, which is negative e to the minus lambda t times dt.
That goes from 0 to infinity.
This is evaluated from 0 to infinity.
Well, what does it mean for this to be evaluated from 0 to infinity?
A better and easier way to look at this is to say, well, it's going to go from 0 to x.
But then you take the limit as x goes to infinity.
So that's going to help us here.
And this negative-- these negatives cancel.
And we're left with-- let's plug in the bounds.
We're left with negative x minus lambda x plus the integral of this is going to be 1 over negative lambda e to the minus lambda t evaluated from 0 to infinity.
All right, so now the limit.
So for the limit, notice that x increases as x goes to infinity.
And this exponential decays.
So they're kind of competing for each other.
But the exponential is going to win because it decays way faster than x.
And so this first term is going to go off-- the limit is going to go to 0.
All right.
For this, if you evaluate the balance, the infinity makes this 0.
And 0, you're going to get 1 over lambda.
So that's 1 over lambda.
All right.
And so the expectation is 1 over lambda.
OK, so now what's the variance?
That's part c, right?
So we use the standard formula for variance, which is this.
We already figured out the expectation.
We just need to figure out the expectation of x squared.
Well, we're just going to follow the same set of steps from before.
For x squared, it's just going to be t squared, t squared, t squared, x squared.
The only thing that's going to change is what we choose for u here, for the u substitution.
So it's going to be t squared.
So the derivative is going to change to 2t dt.
v is going to be exactly the same.
And so here in this term, we get negative 2t e to the minus lambda t. But there's a negative sign out here, so the negatives cancel and we're left with a positive sign here.
This is going to change.
All right.
OK.
So in order to do this integral, we can use a trick.
We can move this-- so there's a 2t here.
We move this 2 in here, leave the t inside.
And you have to leave the t inside.
But multiply by lambda and divide by lambda.
Now, look at that integral.
0 to infinity t times lambda e to the minus lambda t dt.
Exactly the expectation that we computed.
We already did that.
That is just 1 over lambda, so it's 2 over lambda times 1 over lambda.
Again, the limit as x goes to infinity-- the exponential will beat x squared.
No matter what polynomial we put in there, the exponential's going to win.
So this is going to be 0 still.
This one's going to be 2 over lambda squared.
So we're left with 2 over lambda squared for expectation of x squared.
And so we have 1 over lambda squared for the variance.
OK, so we're done with the variance.
Part d. We're given that x1, x2, and x3 are independent and identically distributed.
They're exponentials with rate lambda.
We're asked for the PDF of z, which is the max of x1, x2, and x2.
How do we generally find a PDF?
We take the CDF and then take the derivative, right?
We first find the CDF, and then take the derivative.
So let's do that.
So first, let's see.
Part d, find the CDF of z, which is going to be the probability that Z is less than or equal to little z, which is going to be equal to the probability that the max of x1, x2, x3 is less than or equal to z.
And this is going to have the same sort of structure as before.
If z is less than 0, x1, x2, x3 are positive-- non-negative.
And so this is the probability that if you get little z less than 0, you're not going to have any probability there.
And so if z is greater than or equal to 0 is where it gets interesting.
We need to do something special.
So the special thing here is to recognize that the probability of the max being less than or equal to z is actually also the probability of each of these random variables individually being less than or equal to z.
Why is that true?
One way to check whether the events-- these two events are the same is to check the two directions.
One direction say, if the max of x1, x2, x3 is less than or equal to z, does that mean x1 is less than or equal to z, x2 is less than or equal to z, and x3 is less than or equal to z?
Yes.
OK. And then, if x1, x2, and x3 are individually less than or equal to z, then the max is also less than or equal to z.
So these two events are equivalent and this is true.
By independence we can break this up.
And we get-- these are all CDFs of the exponential and they all have this form.
So it's just going to be 1 minus e to the minus lambda z cubed.
Plug this in here.
And then, try to take the derivative to get the PDF.
Let's see.
So it's going to be the same, like this for z less than 0.
For z greater than or equal to 0, it's going to be the derivative of this thing.
Derivative of this thing is by chain rule, 3 times 1 minus e to the minus lambda z squared.
Then the derivative of negative e to the minus lambda z, that's just lambda e to the minus lambda z.
There we go.
This is the PDF we were looking for.
So last problem.
We're looking for the PDF of w, which is the min of x1 and x2.
So let's try this as a similar approach.
Try the same thing, actually.
See if it works.
So w, w, w, w, min, less than or equal to w. OK.
So let's see if this works.
Is it true that the min-- if the min of x1 and x2 is less than or equal to w, that each of them is less than or equal to w?
No, right?
X1 could be less than or equal to w and x2 could be bigger than w. And the min could still be less than or equal to w. So that's definitely not true.
So what do we do here?
The trick is to flip it and say we want to compute the min of x1 and x2 being greater than w. In that case, let's check if we can do this trick.
If the min of x1 and x2 is greater than w, then clearly x1 is bigger than w and x2 is bigger than w. And if x1 and x2 are individually bigger than w, then clearly the min's also bigger than w. So this works.
And now we can use independence as before.
And for this, this is just 1 minus the CDF here.
So it's just going to be e to the minus lambda w for each of them.
But that's the same as e to the minus lambda 2w.
Or e to the 2 lambda w. So it's going to be-- Notice the similarity between this and this.
The only difference is this has a 2 lambda in there.
That means that w is an exponential random variable with rate 2 lambda.
So then the PDF is going to be an exponential, whatever it is for an exponential.
Except with rate 2 lambda.
You can also take the derivative of this and find that you get this.
OK, so we're done with the problems.
We computed some interesting quantities for the exponential random variable in this--
In this video, we'll look at an example in which we compute the expectation and cumulative density function of a mixed random variable.
The problem is as follows.
Al arrives at some bus stand or taxi stand at a given time-- let's say time t equals 0.
He finds a taxi waiting for him with probability 2/3 in which he takes it.
Otherwise, he takes the next arriving taxi or bus.
The time that the next taxi arrives between 0 and 10 minutes, and it's uniformly distributed.
The next bus leaves exactly in 5 minutes.
So the question is, if X is Al's waiting time, what is the CDF and expectation of X?
So one way to view this problem that's convenient is the tree structure.
So I've drawn it for you here in which the events of interest are B1, B2, and B3, B1 being Al catches the waiting taxi, B2 being Al catches the next taxi, which arrives between 0 and 5 minutes, and B3 being Al catches the bus at the time t plus 5.
Notice that these three events are disjoint.
So Al catching the waiting taxi means he can't catch the bus or the next arriving taxi.
And it also covers the entire set of outcomes.
So, in fact, B1, B2, and B3 are a partition.
So let's look at the relevant probabilities.
Whether or not B1 happens depends on whether or not the taxi's waiting for Al.
So if the taxi is waiting for him, which happens with 2/3 probability, B1 happens.
Otherwise, with 1/3 probability, we see whether or not a taxi is going to arrive between 0 and 5 minutes.
If it arrives, which is going to happen with what probability?
Well, we know that the next taxi is going to arrive between 0 and 10 minutes uniform.
It's a uniform distribution.
And so half the mass is going to be between 0 and 5.
And the other half is going to be between 5 and 10.
And so this is going to be 1/2 and 1/2.
And let's look at what X looks like.
If B1 happens, Al isn't waiting at all, so x is going to be equal to 0.
If B3 happens, which is the other easy case, Al's going to be waiting for 5 minutes exactly.
And if B2 happens, well, it's going to be some value between 0 and 5.
We can actually draw the density, so let's see if we can do that here.
The original next taxi was uniformly distributed between 0 and 10.
But now, we're told two pieces of information.
We're told that B2 happens, which means that there's no taxi waiting, and the next taxi arrives between 0 and 5 minutes.
Well, the fact that there was no taxi waiting has no bearing on that density.
But the fact that the next taxi arrives between 0 and 5 does make a difference, because the density then is going to be definitely 0 in any region outside 0 and 5.
Now, the question is, how is it going to look between 0 and 5?
Well, it's not going to look crazy.
It's not going to look like something different.
It's simply going to be a scale version of the original density between 0 and 5.
You can verify this by looking at the actual formula for when you condition events on a random variable.
Here, it's going to be 1/5 in order for this to integrate out to 1.
And now we can jump right into figuring out the expectation.
Now, notice that X is actually a mixed random variable?
What does that mean?
Well, X either takes on values according to either a discrete probability law or a continuous one.
So if B1 happens, for example, X is going to be exactly equal to 0 with probability 1, which is a discrete probability problem.
On the other hand, if B2 happens, then the value of X depends on the density, which is going to be continuous.
So X is going to be a continuous random variable here.
So how do you define an expectation in this case?
Well, you can do it so that it satisfies the total expectation theorem, which means that the expectation of X is the probability of B1 times the expectation given B1 plus the probability of B2 times the expectation given B2 plus the probability of B3 times the expectation given B3.
So this will satisfy the total expectation theorem.
So the probability of B1 is going to be exactly 2/3.
It's simply the probability of a taxi waiting for Al.
The expected value of X-- well, when B1 happens, X is going to be exactly equal to 0.
So the expected value is also going to be 0.
The probability of B2 happening is the probability of a taxi not being there times the probability of a taxi arriving between 0 and 5.
It's going to be 1/3 times 1/2.
And the expected value of X given B2 is going to be the expected value of this density.
The expected value of this density is the midpoint between 0 and 5.
And so it's going to be 5/2.
And the probability of B3 is going to be 1/3 times 1/2.
Finally, the expected value of X given B3.
Well, when B3 happens, X is going to be exactly equal to 5.
So the expected value is also going to be 5.
Now we're left with 5/12 plus 5/6, which is going to be 15/12.
And we can actually fill that in here so that we can clear up the board to do the other part.
Now we want to compute the CDF of X.
Well, what is the CDF?
Well, the CDF of X is going to be equal to the probability that the random variable X is less than or equal to some little x.
It's a constant [INAUDIBLE].
Before we jump right in, let's try to understand what's the form of the CDF.
And let's consider some interesting cases.
You know that the random variable X, the waiting time, is going to be somewhere between 0 and 5, right?
So let's consider what happens if little x is going to be less than 0.
That's basically saying, what's the probability of the random variable X being less than some number that's less than 0?
Waiting time can't be negative, so the probablility of this is going to be 0.
Now, what if X is between equaling 0 and strictly less than 5?
In that case, either X can fall between 0 and 5 according to this case, in the case of B2, or X can be exactly equal to 0.
It's not clear.
So let's do that later.
Let's fill that in later.
What about if x is greater than or equal to 5?
Little x, right?
That's the probability that the random variable X is less than some number that's bigger than or equal to 5.
The waiting time X, the random variable, is definitely going to be less than or equal to 5, so the probability of this is going to be 1.
So now this case.
How do we do it?
Well, let's try to use a similar kind of approach that we did for the expected value and use the total probability theorem in this case.
So let's try to review this.
First of all, let's assume that this is true, that little x is between 0 and 5, including 0.
And let's use the total probability theorem, and use the partitions B1, B2, and B3.
So what's the probability of B1?
It's the probability that Al catches waiting taxi, which happens with probability 2/3.
What's the probability that the random variable X, which is less than or equal to little x under this condition, when B1 happens?
Well, if B1 happens, then random variable X is going to be exactly equal to 0, right?
So in that case, it's definitely going to be less than or equal to any value of x, including 0.
So the probability will be 1.
What's the probability that B2 happens now?
The probability that B2 happens is 1/3 times 1/2, as we did before.
And the probability that the random variable X is less than or equal to little x when B2 happens.
Well, if B2 happens, this is your density.
And this is our condition.
And so x is going to be somewhere in between these spots.
And we'd like to compute what's the probably that random variable X is less than or equal to little x.
So we want this area.
And that area is going to have height of 1/5 and width of x.
And so the area's going to be 1/5 times x.
And finally, the probability that B3 happens is going to be 1/3 times 1/2 again times the probability that the random variable X is less than or equal to little x given B3.
Well, when B3 happens, X is going to be exactly 5 as a random variable.
But little x, you know-- we're assuming in this condition-- is going to be between 0 and 5, but strictly less than 5.
So there's no way that if the random variable X is 5 and this is strictly less than 5, this is going to be true.
And so that probability will be 0.
So we're now left with 2/3 plus 1/30.
And now we can fill this in.
2/3 plus 1/30 x.
And this is our CDF.
So now we've finished the problem, computed the expected value here and then the CDF here, and this was a great illustration of how you would do so for a mixed random variable.
Hi.
In the session, we'll be solving the Monty Hall problem.
And this problem is based on an old game show that was called "Let's Make a Deal."
And the host of this game show, his name was Monty Hall, which is why this problem is now known as the Monty Hall problem.
And this problem is actually pretty well-known, because there was some disagreement at the time over what the right answer to this problem should be.
Even some really smart people didn't agree on what the right answer should be.
And part of what might explain that disagreement is that they probably were considering slightly different variations of the problem, because as in all probability problems, the assumptions that you're working with are very important, because otherwise you may be solving an actually different problem.
And so what we'll do first is really layout concretely what all the assumptions are, what the rules of the game are.
And then we'll go through the methodology to solve for the actual answer.
So the game is actually relatively simple.
So you're on a game show and you're presented with three doors.
These doors are closed.
And behind one of these doors is a prize, let's say, a car.
And behind the other two doors, there's nothing.
You don't know which one it is.
And the rules of the game are that, first, you get to choose any one of these three.
So you pick one of the doors that you want.
They don't show you what's behind that door, but your friend, who actually knows which door has the prize behind it, will look at the remaining doors.
So let's, just for example, let's say you chose door one.
Your friend will look at the other two doors and open one of them.
And you will make sure that the one that he opens is empty.
That is the prize not behind that one.
And at this point, one of the doors is open and its empty, you have your original door plus another unopened door.
And you're given an option-- you could either stay with your initial choice or you can switch to the other unopened door.
And whichever one is your final choice, they will open that door.
And if there's a price behind it, you win, and if there not, then you don't win.
So the question that we're trying to answer is what is the better strategy here?
Is the better strategy to stay with your initial choice or is it better to switch to the other unopened door?
OK, so it turns out that the specific rules here actually are very important.
Specifically, the rule about how your friend chooses to open doors.
And the fact that he will always open one of the two other door that you haven't picked and he will make sure that that door doesn't have a prize behind it.
And let's see how that actually plays out in this problem.
So the simplest way, I think, of thinking about this problem is just to think about under what circumstances does staying with your initial choice win?
So if you think about it, the only way that you can win by staying with your initial choice is if your initial choice happened to be the door that has a prize behind it.
And because you're sticking with the initial choice, you can actually kind of forget about the rest of the game, about opening of the other door and about switching.
It's as if you're playing a simpler game, which is just you have three doors, one of them has a prize behind it, and you choose one of them.
And if you guessed right, then you win.
If you didn't, then you don't win.
And because the another important assumption is that the prize has an equal probability of being behind any one of three doors so one third, one third, one third.
Because of that, then if you stay with your first choice, you win only if your first choice happened to the right one.
And that is the case with probably one third.
So with that simple argument you can convince yourself that the probability of winning, given the strategy of staying with your first choice, is one third.
Now, let's think about the other strategy, which is to switch.
So under what circumstances does switching win for you?
Well, if your first choice happened to be the right door, then switching away from that door will always lose.
But let's say, that happens with probably one third.
But the rest of the time with probably 2/3, your first choice would be wrong.
So let's give an example here.
Let's say, the prize, which I'll denote by happy face, is behind door two.
And your first choice was door one.
So your first choice was wrong.
Now, your friend can open door two, because door two has the prize behind it.
He also doesn't open the door that you initially picked.
So he has to open door three.
So door three is open, and now you have an option of sticking with your first choice-- door one-- or switching to door two.
So in this case, it's obvious to see that switching wins for you.
And now, if instead, you picked door one first, and the prize was behind door three, again, you are wrong.
And again, your friend is forced to open door two.
And switching, again, wins for you.
And so if you think about it, switching will win for you, as long as your initial pick was wrong.
If your initial pick was wrong, then the prize is behind one of the doors.
Your friend has to open one of the doors, but he can't open the door that has the prize behind it.
So he has to open the other bad door, leaving the good door with the prize behind it, as the one that you can switch to.
And so by switching you will win in this scenario.
And what is the probability of that happening?
Well, that happens if your initial pick was wrong, which happens with probably 2/3.
So the final answer then, it's pretty simple, the probability of winning if you stay is one third, and the probability of winning if you switch is 2/3.
And so maybe counterintuitively the result is that it's actually better for you, twice as good for you, to switch rather than stay.
And so that was the argument, the kind of simple argument.
We can also be more methodical about this and actually list out all of the possible outcomes.
Because it's relatively small problem-- there's only three doors-- we can actually just list out all the possible outcomes.
So for example, if you chose door one first, and the prize was behind door one, your friend has a choice.
He can open door two or door three, because they're both empty.
And then in that case, if you stay, you win, you picked the door correctly.
And if you switch to two or three, then you lose.
But if you chose door one, the prize is behind door two, then your friend has to open door three, he is forced to do that, then staying with lose but switching would win.
And so on for the other cases.
And so again, this is just an exhaustive list of all the possible outcomes, from which you can see that, in fact, staying wins, only if your first choice was correct.
And switching wins in all the other cases.
And so one third of the time, staying would win, 2/3 of the time switching would win.
OK, so now, we have the answer.
Let's try to figure out and convince ourselves that it is actually right, because you might think before going through this process that maybe it doesn't matter whether you stay or you switch, they both have the same probably of winning, or maybe even staying is better.
So why is staying worse and switching better?
Well, the first argument really is something that we've already talked about.
By staying, you're essentially banking on your first choice being correct, which is a relatively poor bet, because you have only one in three chance of being right.
But by switching, you're actually banking on your first choice being wrong, which is a relatively better bet, because you're more likely to be wrong than right in your first choice, because you're just picking blindly.
OK, so that is one intuitive explanation for why switching is better.
Another slightly different way to think about it is that instead of picking single doors, you're actually picking groups of doors.
So let's say that your first pick was door one.
Then you're actually really deciding between door one or doors two and three combined.
So why is that?
It's because by staying with door one, you're staying with door one.
But by switching, you're actually getting two doors for the price of one, because you know that your friend will reveal one of these to be empty, and the other one will stay closed.
But switching really kind of buys you both of these.
And so because it buys you two opportunities to win, you get 2/3 chance of winning, versus a one third chance.
Another way of thinking about this is to increase the scale of the problem, and maybe that will help visualize the counterintuitive answer.
So instead of having three doors, imagine that you have 1,000 doors that are closed.
And again, one prize is behind one of the doors.
And the rules are similar-- you pick one door first, and then your friend will open 998 other doors.
And these doors are guaranteed to be empty.
And now you're left with your initial door plus one other door that is unopened.
So now the question is should you stay with your first choice or switch to your other choice?
And it should be more intuitively obvious now that the better decision would be to switch, because you're overwhelmingly more likely to have picked incorrectly for your first pick.
You have only 1 in 1,000 chance of getting it right.
So that is kind of just taking this to a bigger extreme and really driving home the intuition.
OK, so what we've really discovered is that the fact that the rules of the game are that your friend has to open one of the other two doors and cannot reveal the prize plays a big role in this problem.
And that is an important assumption.
OK, so now let's think about a slightly different variation now.
So a different strategy.
Instead of just always staying or always switching, we have a specific other strategy, which is that you will choose door one first and then, depending on what your friend does, you will act accordingly.
So if your friend opens door two, you will not switch.
And if your friend opens door three, you will switch.
So let's draw out exactly what happens here.
So you have door one that you've chosen.
And the prize can be behind doors one, two, or three.
And again, it's equally likely.
So the probabilities of these branches are one third, one third, and one third.
And now given that, your friend in this scenario has a choice between opening doors two or three.
And so because of doors, you chose one, the prize actually is behind one, and so two and three are both empty, so he can choose whichever one he wants to open.
And the problem actually hasn't specified how your friend actually decides between this.
So we'll leave it in general.
So we'll say that the probability p, your friend will open two, door two, in this case.
And with the remaining probability 1 minus p, he will open door three.
What about in this case?
Well, you chose door one.
The prize is actually behind door two.
So following the rules of the game, your friend is forced to open door three.
So this happens with probability 1.
And similarly, if the prize is behind door three, your friend is forced to open door two, which, again, happens with probably 1.
So now let's see how this strategy works.
When do you win?
You win when, according to the strategy, your final choice is the right door.
So according to the strategy, in this case, your friend opened door two.
And according to your strategy, if door two is open, you don't switch.
So you stay with your first choice of one.
And that happens to the right one, so you win in this case.
But what about here?
Your friend opened door three, and by your strategy, you do switch, which is the wrong choice here, so you lose.
Here, you switch, because you open door three, and you switch to the right door, so that wins.
And this one, you don't switch, and you lose.
All right, so what is the final probability of winning?
And the final probably of winning is the probability of getting to these two outcomes, which happens with probability one third times p plus one third times 1.
So one third.
So the final answer is one third p plus one third.
And notice now that the answer isn't just a number.
Like in this case, the answer was one third and 2/3.
And it didn't actually matter how your friend chose between these two doors when he had a choice.
But in this case, it actually doesn't matter, because p stays in the answer.
But one thing that we can do is we can compare this with these strategies.
So what we see is that, well p is a probability, so it has to be between 0 and 1.
So this probability winning for this strategy is somewhere between one third times 0 plus one third, which is one third.
And one third times 1 plus one third, which is 2/3.
So the strategy is somewhere between 2/3 and one third.
So what we see is that no matter what, this strategy is at least as good as staying all the time, because that was only one third.
And no matter what it can't be any better than switching, which was 2/3.
So you can also come up with lots of other different strategies and see what the probabilities of winning are in that case.
OK, so what have we learned in this problem?
What are the key takeaways?
One important takeaway is that it's important to really understand a problem and arrive at a concrete and precise set of assumptions.
So really have a precise problem that you're solving.
And another important takeaway is to think about your final answer, make sure that that actually makes sense to you, make sure that you can justify it somehow intuitively.
In that case, you can actually convince yourself that your answer is actually correct, because sometimes go through a lot of formulas, and sometimes your formula may have an error in there somewhere.
But you could take the final answer and ask yourself does this actually makes sense intuitively?
That's often a very good check and sometimes you can catch errors in your calculations that way.
OK so we'll see next time.
In this exercise, we'll be working with the notion of convergence in probability, as well as some other notion of converge of random variables that we'll introduce later.
First type of random variable is xn, where xn has probability 1 minus 1 minus over n to be as 0 and probability of 1 over n to be a 1.
And graphically, we see that we have a pretty big mess.
1 minus 1 over n at location 0, and a tiny bit somewhere here, only 1 over n. So this will be the PMF for x.
On the other hand, we have the sequence of random variables, yn.
Fairly similar to xn with a slight tweak.
The similar part says it also has a very high probability of being at 0, mass 1 over 1 minus n. But on the off chance that yn is not at 0, it has a pretty big value n. So it has probability 1 over n of somewhere out there.
So to contrast the two graphs, we see at 0, they have the same amount of mass, 1 over 1 minus n, but for y, it's all the way out there that has a small mass 1 over n. So this will be our Pyn of y.
And for the remainder of the problem, we'll be looking at the regime where the number n tends to infinity, and study what will happen to these two sequences of random variables.
In part A, we're to compute the expected value n variance for both xn and yn.
Let's get started.
The expected value of xn is given by the probability-- it's at one, which is 1 over n times 1 plus the probability being at 0, 1 over n times value 0.
And that gives us 1 over n. To calculate the variance of xn, see that variance is simply the expected value of xn minus expected value of xn, which in this case is 1 over n from the previous calculation we have here.
We take the square of this value and compute the whole expectation, and this gives us 1 over n, 1 minus 1 over n plus the remainder probability 1 over 1 minus n of x being at 0, so 0 minus 1 over n squared.
And if we carry out the calculations here, we'll get n minus 1 over n squared.
Now, let's turn to yn.
The expected value of yn is equal to probability of being at 0, 0 plus the probability of being at n and times the value n. And this gives us 1.
The variance of yn.
We do the same thing as before, we have 1 minus 1 over n probability of being at 0, multiplied 0 minus 1 squared, where 1 is the expected value of y.
And with probability 1 over n, out there, equal to n, and this is multiplied by n minus 1 squared.
And this gives us n minus 1 Already, we can see that while the expected value for x was 1 over n, the expected value for y is sitting right at 1.
It does not decrease as it increases.
And also, while the variance for x is n minus 1 over n squared, the variance for y is much bigger.
It is actually increasing to infinity as n goes infinity.
So these intuitions will be helpful for the remainder of the problem.
In part B, we're asked to use Chebyshev's Inequality and see whether xn or yn converges to any number in probability.
Let's first recall what the inequality is about.
It says that if we have random variable x, in our case, xn, then the probability of xn minus the expected value of xn, in our case, 1 over n, that this deviation, the absolute value of this difference is greater than epsilon is bounded above by the variance of xn divided by the value of epsilon squared.
Well, in our case, we know the variance is n minus 1 over n squared, hence this whole term is this term divided by epsilon squared.
Now, we know that as n gets really big, the probability of xn being at 0 is very big.
It's 1 minus 1 over n. So a safe bet to guess is that if xn work to converge anywhere on the real line, it might just converge to the point 0.
And let's see if that is true.
Now, to show that xn converges to 0 in probability, formally we need to show that for every fixed epsilon greater than 0, the probability that xn minus 0 greater than epsilon has to be 0, and the limit has n going to infinity.
And hopefully, the inequalities above will help us achieve this goal.
And let's see how that is done.
I would like to have an estimate, in fact, an upper bound of the probability xn absolute value greater or equal to epsilon.
And now, we're going to do some massaging to this equation so that it looks like what we know before, which is right here.
Now, we see that this equation is in fact, less than probability xn minus 1 over n greater or equal to epsilon plus 1 over n. Now, I will justify this inequality in one second.
But suppose that you believe me for this inequality, we can simply plug-in the value right here, namely substituting epsilon plus 1 over n, in the place of epsilon right here and use the Chebyshev Inequality we did earlier to arrive at the following inequality, which is n minus 1 over n squared times, instead of epsilon, now we have epsilon plus 1 over n squared.
Now, if we take n to infinity in this equation, see what happens.
Well, this term here converges to 0 because n squared is much bigger than n minus 1.
And this term here converges to number 1 over epsilon squared.
So it becomes 0 times 1 over epsilon squared, hence the whole term converges to 0.
And this proves that indeed, the limit of the term here as n going to infinity is equal to 0, and that implies xn converges to 0 in probability.
Now, there is the one thing I did not justify in the process, which is why is probability of absolute value xn greater than epsilon less then the term right here?
So let's take a look.
Well, the easiest way to see this is to see what ranges of xn are we talking about in each case.
Well, in the first case, we're looking at interval around 0 plus minus epsilon and xn can lie anywhere here.
While in the second case, right here, we can see that the set of range values for xn is precisely this interval here, which was the same as before, but now, we actually have less on this side, where the starting point and the interval on the right is epsilon plus 2 over n. And therefore, the right hand style captures strictly less values of xn than the left hand side, hence the inequality is true.
Now, we wonder if we can use the same trick, Chebyshev Inequality, to derive the result for yn as well.
Let's take a look.
The probability of yn minus it's mean, 1, greater or equal to epsilon.
From the Chebyshev Inequality, we have variance of yn divided by epsilon squared.
Now, there is a problem.
The variance of yn is very big.
In fact, it is equal to n minus 1.
And we calculated in part A, divided by epsilon squared.
And this quantity here diverges as n going to infinity to infinity itself.
So in this case, the Chebyshev Inequality does not tell us much information of whether yn converges or not.
Now, going to part C, the question is although we don't know anything about yn from just the Chebyshev Inequality, does yn converge to anything at all?
Well, it turns out it does.
In fact, we don't have to go through anything more complicated than distribution yn itself.
So from the distribution yn, we know that absolute value of yn greater or equal to epsilon is equal to 1 over n whenever epsilon is less than n. And this is true because we know yn has a lot of mass at 0 and a tiny bit a mass at value 1 over n at location n. So if we draw the cutoff here at epsilon, then the probability of yn landing to the right of epsilon is simply equal to 1 over n. And this tells us, if we take the limit of n going to infinity and measure the probability that yn-- just to write it clearly-- deviates from 0 by more than epsilon, this is equal to the limit as n going to infinity of 1 over n. And that is equal to 0.
From this calculation, we know that yn does converge to 0 in probability as n going to infinity.
For part D, we'd like to know whether the convergence in probability implies the convergence in expectation.
That is, if we have a sequence of random variables, let's call it zn, that converges to number c in probability as n going to infinity, does it also imply that the limit as n going to infinity of the expected value of zn also converges to c. Is that true?
Well, intuitively it is true, because in the limit, zn almost looks like it concentrates on c solely, hence we might expect that the expected value is also going to c itself.
Well, unfortunately, that is not quite true.
In fact, we have the proof right here by looking at yn.
For yn, we know that the expected value of yn is equal to 1 for all n. It does not matter how big n gets.
But we also know front part C that yn does converge to 0 in probability.
And this means somehow, yn can get very close to 0, yet it's expected value still stays one away.
And the reason again, we go back to the way yn was constructed.
Now, as n goes to infinity, the probability of yn being at 0, 1 minus 1 over n, approaches 1.
So it's very likely that yn is having a value 0, but whenever on the off chance that yn takes a value other than 0, it's a huge number.
It is n, even though it has a small probability of 1 over n. Adding these two factors together, it tells us the expected value of yn always stays at 1.
And however, in probability, it's very likely that y is around 0.
So this example tells us converges in probability is not that strong.
That tells us something about the random variables but it does not tell us whether the mean value of the random variables converge to the same number.
For part E and F of the problem, we'll be introducing a new notion of convergence, so-called the convergence E mean squared sense.
We say that xn converges to a number c in mean squared, if as we take and go to infinity, the expected value of xn minus c squared goes to 0.
To get a sense of what this looks like, let's say we let c equal to the expected value of xn, and let's say the expected value of xn is always the same.
So the sequence of random variables has the same mean.
Well, if that is true, then mean square convergence simply says the limit of the variance of xn is 0.
So as you can imagine, somehow as xn becomes big, the variance of xn is very small, so xn is basically highly concentrated around c. And by this I mean, the density function for xn.
So that's the notion of convergence we'll be working with.
Our first task here is to show that the mean square convergence is in some sense stronger than the convergence in probability that we have been working with from part A to part D. That is, if I know that xn converged to some number c in mean squared, then this must imply that xn converges to c in probability.
And now, we'll go show that for part E. Well, let's start with a definition of convergence in probability.
We want to show that for a fixed constant epsilon the probability that xn minus c, greater than epsilon, essentially goes to 0 as n goes to infinity.
To do so, we look at the value of this term.
Well, the probability of absolute value xn minus c greater than epsilon is equal to the case if we were to square both sides of the inequality.
So that is equal to the probability that xn minus c squared greater than epsilon squared.
We can do this because both sides are positive, hence this goes through.
Now, to bound this equality, we'll invoke the Markov's Inequality, which it says this probability of xn, some random variable greater than epsilon squared, is no more than is less equal to the expected value of the random variable.
In this case, the expected value of x minus c squared divided by the threshold that we're trying to cross.
So that is Markov's Inequality.
Now, since we know xn converges to c in mean squared, and by definition, mean square we know this precise expectation right here goes to 0.
And therefore, the whole expression goes to 0 as n goes to infinity.
Because the denominator here is a constant and the top, the numerator here, goes to 0.
So now we have it.
We know that the probability of xn minus c absolute value greater than epsilon goes to 0 as n goes to infinity, for all fixed value of epsilons and this is the definition of convergence in probability.
Now that we know if xn converges to c mean squared, it implies that xn converges to c in probability.
One might wonder whether the reverse is true.
Namely, if we know something converges in probability to a constant, does the same sequence of random variables converge to the same constant in mean squared?
It turns out that is not quite the case.
The notion of probability converges in probability is not as strong as a notion of convergence in mean squared.
Again, to look for a counter example, we do not have to go further than the yn's we have been working with.
So here we know that yn converges to 0 in probability.
But it turns out it does not converge to 0 in the mean squared.
And to see why this is the case, we can take the expected value of yn minus 0 squared, and see how that goes.
Well, the value of this can be computed easily, which is simply 0, if yn is equal to 0, with probability 1 minus n plus n squared when yn takes a value of n, and this happens with probability 1 over n. The whole expression evaluates to n, which blows up to infinity as n going to infinity.
As a result, the limit n going to infinity of E of yn minus 0 squared is infinity and is not equal to 0.
And there we have it, even though yn converges to 0 in probability, because the variance of yn, in some sense, is too big, it does not converge in a mean squared sense.
Previously, we learned the concept of independent experiments.
In this exercise, we'll see how the seemingly simple idea of independence can help us understand the behavior of quite complex systems.
In particular, we'll combined the concept of independence with the idea of divide and conquer, where we break a larger system into smaller components, and then using independent properties to glue them back together.
Now, let's take a look at the problem.
We are given a network of connected components, and each component can be good with probability P or bad otherwise.
All components are independent from each other.
We say the system is operational if there exists a path connecting point A here to point B that go through only the good components.
And we'd like to understand, what is the probability that system is operational?
Which we'll denote by P of A to B.
Although the problem might seem a little complicated at the beginning, it turns out only two structures really matter.
So let's look at each of them.
In the first structure, which we call the serial structure, we have a collection of k components, each one having probability P being good, connected one next to each other in a serial line.
Now, in this structure, in order for there to be a good path from A to B, every single one of the components must be working.
So the probability of having a good path from A to B is simply P times P, so on and so, repeated k times, which is P raised to the k power.
Know that the reason we can write the probability this way, in terms of this product, is because of the independence property.
Now, the second useful structure is parallel structure.
Here again, we have k components one, two, through k, but this time they're connected in parallel to each other, namely they start from one point here and ends at another point here, and this holds for every single component.
Now, for the parallel structure to work, namely for there to exist a good path from A to B, it's easy to see that as long as one of these components works the whole thing will work.
So the probability of A to B is the probability that at least one of these components works.
Or in the other word, the probability of the complement of the event where all components fail.
Now, if each component has probability P to be good, then the probability that all key components fail is 1 minus P raised to the kth power.
Again, having this expression means that we have used the property of independence, and that is probability of having a good parallel structure.
Now, there's one more observation that will be useful for us.
Just like how we define two components to be independent, we can also find two collections of components to be independent from each other.
For example, in this diagram, if we call the components between points C and E as collection two, and the components between E and B as collection three.
Now, if we assume that each component in both collections-- they're completely independent from each other, then it's not hard to see that collection two and three behave independently.
And this will be very helpful in getting us the breakdown from complex networks to simpler elements.
Now, let's go back to the original problem of calculating the probability of having a good path from point big A to point big B in this diagram.
Based on that argument of independent collections, we can first divide the whole network into three collections, as you see here, from A to C, C to E and E to B.
Now, because they're independent and in a serial structure, as seen by the definition of a serial structure here, we see that the probability of A to B can be written as a probability of A to C multiplied by C to E, and finally, E to B.
Now, the probability of A to C is simply P because the collection contains only one element.
And similarly, the probability of E to B is not that hard knowing the parallel structure here.
We see that collection three has two components in parallel, so this probability will be given by 1 minus 1 minus P squared.
And it remains to calculate just the probability of having a good path from point C to point E. To get a value for P C to E, we notice again, that this area can be treated as two components, C1 to E and C2 to E connected in parallel.
And using the parallel law we get this probability is 1 minus 1 minus P C1 to E multiplied by the 1 minus P C2 to E. Know that I'm using two different characters, C1 and C2, to denote the same node, which is C. This is simply for making it easier to analyze two branches where they actually do note the same node.
Now P C1 to E is another serial connection of these three elements here with another component.
So the first three elements are connected in parallel, and we know the probability of that being successful is 1 minus P3, and the last one is P. And finally, P C2 to E. It's just a single element component with probability of successful being P. At this point, there is no longer any unknown variables, and we have indeed obtained exact values for all the quantities that we're interested in.
So starting from this equation, we can plug in the values for P C2 to E, P C1 to E back here, and then further plug in P C to E back here.
That will give us the final solution, which is given by the following somewhat complicated formula.
So in summary, in this problem, we learned how to use the independence property among different components to break down the entire fairly complex network into simple modular components, and use the law of serial and parallel connections to put the probabilities back together in common with the overall success probability of finding a path from A to B.
In this problem, we're looking at a two stage process in which the first stage, we roll a fair die which has four faces to obtain a number N, where N belongs to the set 0, 1, 2, and 3 with equal probability.
Now, given the result of the die roll, N will toss a fair coin N times in getting K heads from the coin tosses.
For instance, if from the first die roll, we get N equal to 3, then we'll toss a coin 3 times.
Let's say the outcome is heads, heads, and tails.
And that will give us K equal to 2.
For part A, we're asked to compute the PMF for N, which is a result of the first die roll.
Now, since we had assumed the die roll was uniformly distributed in the set in the set 0, 1, 2, and 3, we have that the chance of N being equal to any little n is equal to 1/4 if n is in the set 0, 1, 2, 3, and 0 otherwise.
If we were to plot this in a figure, we'll have the following plot.
For part B, things are getting a little more complicated.
This time, we want to compute the joint PMF between N and K for N equal to little n and K equal to little k. What we'll do first is to use the law of conditional probability to break the joint probability into the product of probability of K is equal to little k conditional on N is equal to little n, multiply by the probability that N is equal to little n. Now, the second term right here is simply the PMF of N, which will be computed earlier.
So this gives us 1/4 times probability K equal to little k, N equal to little n, for all N in the set 0, 1, 2, and 3.
Now, clearly if N is not one of those four values, this whole event couldn't have happened in the first place, and hence will have P and K equal to 0.
We'll now go over all the cases for little n in this expression right here.
The first case is the simplest.
If we assume that little n is equal to 0, that means the die roll was 0, and hence we're not tossing any coins afterwards.
And this implies that we must have K is equal to 0, which, mathematically speaking, is equivalent to saying probability of K equal to 0 conditional on N equal to 0 is 1.
And K being any other value conditional N equal to 0 is 0.
So we're done with the case that little n is equal to 0.
Now, let's say little n is in the set 1, 2, and 3.
In this case, we want to notice that after having observed the value of N, all the coin tosses for N times are conditionally independent from each other.
What this means is now the total number of heads in the subsequent coin toss is equal in distribution to a binomial random variable with parameter n and 1/2.
And here says the number of trials is n, and 1/2 is because the coin is fair.
And the reason it is a binomial random variable, again, is because the coin tosses are independent conditional on the outcome of the die roll.
And now we're done, since we know what the binomial distribution looks like given parameter n and 1/2.
And we'll simply substitute based on the case of n the conditional distribution of K back into the product we had earlier, which in turn will give us the joint PMF.
This table summarizes the PMF we were computing earlier.
P of N, K, little n, and little k. Now, as we saw before, if n equal to 0, the only possibility for k is equal to 0.
And this is scaled by the probability of n equal to 0, which is 1/4.
For any other values of n, we see that the distribution of k, conditional n, is the same as a binomial random variable with n trials.
And again, every entry here is scaled by 1/4.
And this completes part B.
In part C, we're asked for the conditional PMF of K conditioning on the value of N being equal to 2.
Now, as we discussed in part B, when N is equal to 2, we're essentially flipping a fair coin twice, and this should give us the same distribution as a binomial random variable with parameter 2 and 1/2.
Now, 2 is the number of flips, and 1/2 is the chance of seeing a head in each flip.
And that gives us the following distribution.
But there's another way to see this.
It's to write P K given N, little k, and you go to 2 by using the law of conditional probability as P K, N, the joint PMF, k n2, divided by the probability that N is equal to 2.
Now, we know that probability n equal to 2 is simply 1/4, so this gives us 4 times the joint density K, N, k, 2.
In other words, in order to arrive at the distribution right here, [INAUDIBLE] to go back to the table we had earlier and look at the role where n is equal to 2 and multiply each number by 4.
Finally, in part D, we're asked for the conditional distribution of N, write as P N, given K of N equal to little n conditional on K is equal to 2.
Again, we'll apply the formula for conditional probability.
This is equal to the joint PMF evaluated at n and 2 divided by the probability of K being equal to 2.
Since we have computed the entire table of the joint PMF, this shouldn't be too difficult.
In particular, for the denominator, the probability that k is ever equal to 2, we just look at the column right here.
So the entries in this column shows all the cases where k can be equal to 2.
And in fact, we can see that k can be equal to 2 only if n is equal to 2 or 3.
Clearly, if you toss the coin fewer than 2 times, there's no chance that we'll get 2 heads.
So to get this probability right here, we'll add up the number in these two cells.
So we get P N, K, little n, and 2 divided by 1/16 plus 3/32.
Now, the numerator, again, can be read off from the table right here.
In particular, this tells us that there are only two possibilities.
Either n is equal to 2 or n equal to 3.
When n is equal to 2, we know this quantity gives us 1/16 reading off this cell divided by 1/16 plus 3/32 for n equal to 2.
And the remaining probability goes to the case where n is equal to 3.
So this is 3 divided by 32, 1/16 plus 3/32, which simplifies to 2/5 and 3/5.
And this distribution gives us the following plot.
And this completes our problem.
In this problem, we'll be looking at an ambulance that is traveling back and forth in interval of size l. Say from 0 to l. At some point in time, there's an accident occurring, let's say at location x.
And we'll assume the accident occurs in a random location so that x is uniformly distributed between 0 and l. Now, at this point in time, let's say the ambulance turns out to be at location y.
Again, we'll assume that y is a uniform random variable between 0 and l, and also that x and y are independently distributed.
The question we're interested in answering is how long it would take an ambulance to respond to travel from point y to point x.
Let's call this time T. And in particular, we want to know everything about distribution of T. For example, what is the CDF of T given by the probability of big T, less than or equal to little t, or the PDF, which is done by differentiating the CDF once we have it.
Now, to start, we'll express T you as a function of X and Y.
Since we know that the ambulance travels at a speed V-- V meters or V units of distance per second-- then we can write that big T is simply equal to Y minus X, absolute value the distance between X and Y, divided by the speed at which the ambulance is traveling at, V. So now if we look at the probability of T less than or equal to little t, this is then equal to the probability that Y minus X divided by V less than or equal to little t. We now take off the absolute value by writing the expression as negative vt less equal to Y minus X less equal to positive vt.
Here we multiply v on the other side of t, and then took out the absolute value sign.
As a final step, we'll also move X to the other side of inequalities by writing this as X minus vt less equal to y less equal to x plus vt. To compute this quantity, we'll define a set A as a set of all points that satisfies this condition right here.
In particular, it's a pair of all X and Y such that X minus vt less equal to little y less equal to X plus vt, and also that X is within 0 and l, and so is Y.
So the set A will be the set of values we'll be integrating over.
Now that we have A, we can express the above probability as the integral of all X and Y, this pair within the set A, integrating the PDF of f of X, Y, little x, little y.
Let's now evaluate this expression right here in a graphical way.
On the right, we're plotting out what we just illustrated here, where the shaded region is precisely the set A.
As we can see, this is a set of values of X and Y where Y is sandwiched between two lines, the upper one being X plus vt right here, and the lower line being X minus vt, right here.
So these are the values that correspond to the set A.
Now that we have A, let's look f of x, y.
We know that both x and y are uniform random variables between 0 and l, and therefore, since they're independent, the probability density of x and y being at any point between 0 and l is precisely 1 over l squared, where l squared is the size of this square box right here.
So given this picture, all we need to do is to multiply by 1 over l squared the area of the region A.
And depending on the value of T, we'll get different answers as right here.
If T is less than 0, obviously, the area of A diminishes to nothing, so we get 0.
If T is greater than l over V, the area of A fills up the entire square, and we get 1.
Now, if T is somewhere in between 0 and l over v, we will have 1 over l squared, multiply by the area looking like something like that right here-- the shaded region.
Now, if you wonder how we arrive at exactly this expression right here, here is a simple way to calculate it.
What we want is 1 over l squared times the area A.
Now, area A can be viewed as the entire square, l squared, minus whatever's not in area A, which is these two triangles right here.
Now, each triangle has area 1/2, l minus vt squared.
This multiply 2, and this, after some algebra, will give the answer right here.
At this point, we have obtained the probability of big T less equal to little t. Namely, we have gotten the CDF for T. And as a final step, we can also compute the probability density function for T. We'll call it little f of t. And we do so by simply differentiating the CDF in different regions of T. To begin, we'll look at t between 0 and l over v right here at differentiating the expression right here with respect to t. And doing so will give us 2v over l minus 2v squared t over L squared.
And this applies to t greater or equal to 0, less than l/v.
Now, any other region, either t less than 0 or t greater than l/v, we have a constant for the CDF, and hence its derivative will be 0.
So this is for any other t. We call it otherwise.
Now, this completely characterized the PDF of big T, and hence, we've also finished a problem.
In this problem, we are looking at a student whose performance from day to day sort of oscillates according to a Markov chain.
In particular, the student can either be in state 1, which is a state of being up to date, or in state 2, which is a state of being kind of fallen behind.
Now, the transition probabilities between these two states are given by the numbers here, which is 0.2 from state 1 to 2, 0.6 from 2 to 1, 0.4 from 2 back to 2, and 0.8 from 1 back to state 1.
The quantity we're interesting calculating is this notion of first passage time.
Let me define what that means.
Suppose we are looking at a time horizon of time 0, 1, 2, 3.
And let's call the state of the Markov chain x of t. Suppose we start from the chain being in state 2 here.
Now, if we look at a particular sample path, let's say 2 and 2 again on day 1, and 2 again on day 2, and on day 3, the student enters state 1.
So in this sample path, we start from time 0 and time 3 is the first time we enter state 1.
And we'll say that the first passage time, namely, the first time we enter state 1 in this case, is equal to 3.
More formally, we'll define tj as the first pass the time to state 1 conditional on that we start from state j at time 0.
Now, this quantity, of course, is random.
Depending on the realization, we have different numbers.
And we are interested in calculating the expected value of t2.
That is, on average, if we start from state 2 here, how long would it take for us to enter state 1?
Now to calculate this quantity, in the following recursion will be very important.
The idea is we don't know exactly what t2 is.
But t2 has to satisfy a certain recurrent equation, namely, t2 must be equal to 1 plus summation j equal to 1 to 2 P2jtj.
Now let me explain what this equation means.
Let's say we are at state 2.
Well, we don't actually know how long it's going to take for us to enter state 1.
But we do know after one step, I will be go into some other state.
Let's call it state j.
And from state j, it's going to take some time to enter state 1 finally.
So this equation essentially says the time for us to first enter state 1 from 2 is 1-- which is the next step-- plus the expected time from that point on to enter 1.
So that constitutes our [?
recurrent ?]
relationship.
Now, by this definition, we can see that this is simply 1 plus P21 times t1 plus P22 times t2.
Now, the definition of tj says t1 must be 0 because, by definition, if we start from state 1, we are already in state 1.
So the time to reach state 1 is simply 0.
So this term disappears.
And we end up with 1 plus P22 t2.
If we plug in a number of P22-- which is 0.4 right here-- we get 1 plus 0.4 t2.
Now we started from t2 and we ended up with another expression involving numbers and only one unknown, which is t2.
Combining this together and solving for t2, we get t2 equals 1 divided by 1 minus 0.4, which is 5/3.
And that is the answer for the first part of the problem.
In the second part of the problem, we are asked to do something similar as before but with a slight twist.
Here, I copied over the definition for tj, which is the first time to visit state 1 starting from state j at time t equals 0.
And the little tj is this expectation.
And here we're going to define a similar quantity, which is t1, let's say, star, defined as the first time to visit state 1 again.
So that's the recurrence part starting from state 1, 1 at t equals 0.
So this is the recurrence time from state 1 back to state 1 again.
As an example, again, we look at t equals 0, 1, 2, 3, 4.
And here, if we start from state 1 on time 0, we went to state 2, 2, 1, 1 again.
Now here, again, time 3 will be the first time to visit state 1 after time 0.
And we don't count the very first 0.
And that will be our t1 star.
So t1 star in this particular case is equal to 3.
OK.
Same as before, we like to calculate the expected time to revisit state 1.
Define little t1 star expected value of t1 star.
And we'll be using the same recurrence trick through the following equation.
We say that t1 star is equal to 1 plus j from 1 to 2.
Now, since we started from state 1, this goes from 1 to state 1j and tj.
Again, the interpretation is we started at state 1 at time t equals 0, we went to some other state-- we call it j-- and front of state j, it goes around, and after time expected value tj, we came back to state 1.
Here, and as before, this equation works because we are working with a Markov chain whereby the time to reach some other state only depends on the current state.
And that's why we're able to break down the recursion as follows.
If we write out the recursion, we get 1 plus P11 t1 plus P12 t2.
As before, t1 now is just the expected first passage time from state 1.
And by definition, it is 0.
Because if we start from state 1, it's already in state 1 and takes 0 time to get there.
So again, like before, this term goes out.
And we have 1 plus 0.2 times 5/3.
And this number came from the previous calculation of t2.
And this gives us 4/3.
So this completes the problem.
And just to remind ourselves, the kind of crux of the problem is this type of recursion which expresses a certain quantity in the one incremental step followed by the expected time to reach a certain destination after that one step.
And we can do so because the dynamics is modeled by a Markov chain.
And hence, the time to reach a certain destination after this first step only depends on where you start again, in this case, state j.
Hi.
In this problem, we'll be talking about communication across a noisy channel.
But before we dive into the problem itself, I wanted to first motivate the context a little bit and talk more about what exactly a communication channel is and what "noise" means.
So in our everyday life, we deal with a lot of communication channels, for example, the internet, where we download data and we watch videos online, or even just talking to a friend.
And the air could be your communication channel for our voice.
But as you probably have experienced, sometimes these channels have noise, which just means that what the sender was trying to send isn't necessarily exactly what the receiver receives.
And so in probability, we try to model these communication channels and noise and try to understand the probability behind it.
And so now, let's go into the problem itself.
In this problem, we're dealing with a pretty simple communication channel.
It's just a binary channel, which means that what we're sending is just one bit at a time.
And here, a bit just means either 0 or 1-- so essentially, the simplest case of information that you could send.
But sometimes when you send a 0, the receiver actually receives a 1 instead, or vice versa.
And that is where noise comes in.
So here in this problem, we actually have a probabilistic model of this channel and the noise that hits the channel.
What we're trying to send is either 0 or a 1.
And what we know is that on the receiving end, a 0 can either be received when a 0 is sent, or a 1 can be received when a 0 is sent.
And when a 1 is sent, we could also have noise that corrupts it.
And you get a 0 instead.
Or you can have a 1 being successfully transmitted.
And the problem actually tells us what the probabilities here are.
So we know that if a 0 is sent, then with probability 1 minus epsilon naught, a 0 is received.
And with the remaining probability, it actually gets corrupted and turned into a 1.
And similarly, if a 1 is sent, then with probability 1 minus epsilon 1, the 1 is correctly transmitted.
And with the remaining probability epsilon 1, it's turned into a 0 instead.
And the last bit of information is that we know that with the probability p, any random bit is actually is 0 that is being sent.
And with probability 1 minus p, we're actually trying to send a 1.
So that is the basic setup for the problem.
And the first part that the problem asks us to find, what is the probability of a successful transmission when you have just any arbitrary bit that's being sent.
So what we can do here is, use this tree that we've already drawn and identify what are the cases, the outcomes where a bit is actually successfully transmitted.
So if a 0 is sent and a 0 is received, then that corresponds to a successful transmission.
Similarly, if a 1 is sent and a 1 is received, that also corresponds to a successful transmission.
And then we can calculate what these probabilities are, because we just calculate the probabilities along the branches.
And so here implicitly, what we're doing is invoking the multiplication rule.
So we can calculate the probabilities of these two individual outcomes and their disjoint outcomes.
So we can actually just sum the two probabilities to find the answer.
So the probability here is p times 1 minus epsilon naught-- that's the probability of a 0 being successfully transmitted-- plus 1 minus p times 1 minus epsilon, 1, which is the probability that a 1 is successfully transmitted.
And so what we've done here is actually just looked at this kind of diagram, this tree to find the answer.
What we also could have done is been a little bit more methodical maybe and actually apply the law of total probability, which is really what we're trying to do here.
So you can see that this actually corresponds to-- the p corresponds to the probability of 0 being sent.
And 1 minus epsilon naught is the probability of success, given that a 0 is sent.
And this second term is analogous.
It's the probability that a 1 was sent times the probability that you have a success, given that a 1 was sent.
And this is just an example of applying the law of total probability, where we partitioned into the two cases of a 0 being sent and a 1 being sent and calculated the probabilities for each of those two cases and added those up.
So that's kind of a review of the multiplication rule and law of total probability.
So now, let's move on to part B.
Part B is asking, what is the probability that a particular sequence of bits, not just a single one, but a sequence of four bits in a row is successfully transmitted?
And the sequence that we're looking for is, 1, 0, 1, 1.
So this is how I'll denote this event.
1, 0, 1, 1 gets successfully transmitted into 1, 0, 1, 1.
Now, instead of dealing with single bits in isolation, we have a sequence of four bits.
But we can really just break this out into the four individual bits and look at those one by one.
So in order to transmit successfully 1, 0, 1, 1, that whole sequence, we first need to transmit a 1 successfully, then a 0 successfully, then another 1 successfully, and then finally, the last 1 successfully.
So really, this is the same as the intersection of four different smaller events, a 1 being successfully transmitted and a 0 being successfully transmitted and two more 1's being successfully transmitted.
So why are we able to do this, first of all?
We are using an important assumption that we make in the problem that each transmission of an individual bit has the same probabilistic structure so that no matter which bit you're talking about, they all have the same [?
error ?]
probability, the same probabilities of being either successfully transmitted or having noise corrupt it.
So because of that, it doesn't really matter which particular 1 or 0 we're talking about.
And now, we'll make one more step, and we'll invoke independence, which is the third topic here.
And the other important assumption here we're making is that every single bit is independent from any other bit.
So the fact that this one was successfully transmitted has no impact on the probability of the 0 being successfully transmitted or not.
And so because of that, we can now break this down into a product of four probabilities.
So this becomes the probability of 1 transmitted into a 1 times the probability of 0 transmitted into a 0, 1 to a 1, and 1 to 1.
And that simplifies things, because we know what each one of these are.
The probability of 1 being successful transmitted into a 1, we know that's just 1 minus epsilon 1.
And similarly, probability of 0 being transmitted into a 0 is 1 minus epsilon naught.
So our final answer then is just-- well, we have three of these and one of these.
So the answer is going to be 1 minus epsilon naught times 1 minus epsilon 1 to the third power.
Now, let's move on go on to part C, which adds another wrinkle to the problem.
So now, maybe we're not satisfied with the success rate of our current channel.
And we want to improve it somehow.
And one way of doing this is to add some redundancy.
So instead of just sending a single 0 and hoping that it gets successfully transmitted, instead what we can do is, send three 0's in a row to represent a single 0 and hope that because we've added some redundancy, we can somehow improve our error rates.
So in particular what we're going to do is, for a 0, when we want to send a 0, which I'll put in quotes here, what we're actually going to send is a sequence of three 0s.
And what's going to happen is, this sequence of three 0s, each one of these bits is going to go through the same channel.
So the 0, 0, 0 can stay and get transmitted successfully as a 0, 0, 0.
Or maybe the last 0 gets turned into a 1, or the second 0 gets turned into a 1, or we can have any one of these eight possible outcomes on the receiving end.
And then similarly, for a 1, when we want to send a 1, what we'll actually send is a sequence of three 1's, three bits.
And just like above, this 1, 1, 1, due to the noise in the channel, it can get turned into any one of these eight sequences on the receiving end.
So what we're going to do now is, instead of sending just a single 0, we'll send three 0s, and instead of sending a 1, we'll send three 1s.
But now, the question is, this is what you'll get on the receiving end.
How do you interpret-- 0, 0, 0, maybe intuitively you'll say that's obviously a 0.
But what if you get something like 0, 1, 0 or 1, 0, 1, when there's both 0s and 1s in the received message?
What are you going to do?
So one obvious thing to do is to take a majority rule.
So because there's three of them, if there's two or more 0s, we'll say that what was meant to be sent was actually a 0.
And if there's two or more 1s, then we'll interpret that as a 1 being sent.
So in this case, let's look at the case of 0.
The majority rule here would say that, well, if 0, 0, 0 was sent, then the majority is 0s.
And similarly, in these two cases, 0, 0, 1 or 0, 1, 0, the majority is also 0s.
And then finally, in this last case, 1, 0, 0, you get a majority of 0s.
So in these four received messages, we'll interpret that as a 0 have being set.
So part C is asking, given this majority rule and this redundancy, what is the probability that a 0 is correctly transmitted?
Well, to answer that, we've already identified these are the four outcomes, where a 0 would be correctly transmitted.
So to find the answer to this question, all we have to do is find the probability that a sequence of 0, 0, 0 gets turned into one of these four sequences.
So let's do that.
What is the probability that a 0, 0, 0 gets turned into a 0, 0, 0?
Well, that means that all three of these 0s had no errors.
So we would have the answer being 1 minus epsilon 0 cubed, because all three of these bits had to have been successfully transmitted.
Now, let's consider the other ones.
For example, what's the probability that a 0, 0, 0 gets turned into a 0, 0, 1?
Well, in this case, we need two successful transmissions of 0s, plus one transmission of 0 that had an error.
So that is going to be 1 minus epsilon naught squared for the two successful transmissions of 0, times epsilon naught for the single one that was wrong.
And if you think about it, that was only for this case-- 0, 0, 1.
But the case where 0, 1, 0 and 1, 0, 0 are the same, because for all three of these, you have two successful transmissions of 0, plus one that was corrupted with noise.
And so it turns out that all three of those probabilities are going to be the same.
So this is our final answer for this part.
Now, let's move on to part D. Part D is asking now a type of inference problem.
And we'll talk more about inference later on in this course.
The purpose of this problem-- what it's asking is, suppose you received a 1, 0, 1.
That's the sequence of three messages, three bits that you received.
Given that you received a 1, 0, 1, what's the probability that 0 was actually the thing that was being sent.
So if you look at this, you'll look at it and say, this looks like something where we can apply Bayes' rule.
So that's the fourth thing that we're covering in this problem.
And if you apply Bayes' rule, what you'll get is, this is equal to the probability of 0 times the probability of 1, 0, 1 being received, given that 0 was what was sent, divided by the probability that 1, 0, 1 is received.
So we have this basic structure.
And we also know that we can use the law of total probability again on this denominator.
So we know that the probability that 1, 0, 1 is received is equal to the probability of 0 being sent times probability of 1, 0, 1 being received, given that 0 was sent, plus the probability that 1 was sent times the probability that 1, 0, 1 is received, given that 1 is sent.
And as you'll notice in applications of Bayes' rule, usually what you'll have is a numerator is then repeated as one of the terms in the denominator, because it's just an application of total probability.
So if you put these pieces together, really, what we need is just these four terms.
Once we have those four terms, we can just plug them into this equation, and we'll have our answer.
So let's figure out what those four terms are.
The probability of 0 being sent-- well, we said that earlier.
Probability of 0 being sent is just p. And the probability of 1 being sent is 1 minus p. That's just from the model that we're given in the problem.
Now, let's figure out this part.
What is the probability of a 1, 0, 1 being received, given that 0 was sent?
So if 0 was sent, then we know that what really was sent was 0, 0, 0, that sequence of three bits.
And now, what's the probability that 0, 0, 0 got turned into 1, 0, 1?
Wall, in this case, what we have is one successful transmission of a 0, plus two failed transmission of a 0.
So that one successful transmission of a 0, that probability is 1 minus epsilon naught.
And now, we have two failed transmissions of a 0.
So we have to multiply that by epsilon naught squared.
And now, for the last piece, what's the probability of receiving the 1, 0, 1, given that 1 was actually sent?
Well, in that case, if a 1 was sent, what was really sent was a sequence of three 1s.
And now, we want the probability that a 1, 1, 1 got turned into a 1, 0, 1.
In this case, we have two successful transmissions of the 1 with one failed transmission.
So the two successful transmissions will have 1 minus epsilon 1 squared.
And then the one failed transmission will give us an extra term of epsilon 1.
So just for completeness, let's actually write out what this final answer would be.
So probability of 0 is p. Probability of 1, 0, 1, given 0 is, we calculated that as 1 minus epsilon naught times epsilon naught squared.
The same term appears again in the denominator.
Plus the other term is, probability of 1 times the probability of 1, 0, 1, given 1.
So that is 1 minus epsilon squared times epsilon 1.
So that is our final answer.
And it's really just a application of Bayes' rule.
So this was a nice problem, because it represents a real world phenomenon that happens.
And we can see that you can apply a pretty simple probabilistic model to it and still be able to answer some interesting questions.
And there are other extensions that you can ask also.
For example, we've talked about adding redundancy by tripling the number of bits, but tripling the number of bits also reduces the throughputs, because instead of sending one, you have to send three bits just to send one.
So if there's a cost of that, at what point does the benefit of having lower ever outweigh the cost of having to send more things?
And so that's a question that you can answer with some more tools in probability.
So we hope you enjoyed this problem.
And we'll see you again next time.
Hi.
In this video, we're going to do standard probability calculations for normal random variables.
We're given that x is standard normal with mean 0 and variance 1.
And y is normal with mean one and variance 4.
And we're asked for a couple of probabilities.
For the normal CDF, we don't have a closed form expression.
And so people generally tabulate values and for the standard normal case.
So if we want little x equal to 3.49, we just look for 3.4 along the rows and 0.09 along the columns, and then pick the value appropriately.
So for part A, we're asked what's the probability that x is less than equal to 1.5?
That's exactly phi of 1.5 and we can look that up.
1.5 directly and that's 0.9332.
Then were asked, what's the probability that x is less than equal to negative 1?
Notice that negative values are not on this table.
And the reason that is is because the standard normal is symmetric around zero.
And we don't really need that.
We just recognize that the area in this region is exactly the area in this region.
And so that's equal to the probability that x is greater than equal to 1.
This is equal to 1 minus the probability that x is less than 1.
And we can put the equal sign in here because x is continuous, it doesn't matter.
And so we're going to get, this is equal to 1 minus phi of one.
And we can look up phi of 1, which is 1.00, and that's 0.8413.
OK. For part B, we're asked for this distribution of y minus 1 over 2.
So any linear function of a normal random variable is also normal.
And you can see that by using the derived distribution for linear functions of random variables.
So in this case, we only need to figure out what's the mean and the variance of this normal random variable.
So the mean in this case, I'm going to write that as y over 2 minus 1/2.
The expectation operator is linear and so that's going to be-- and the expectation in this case is 1, so that's going to be 0.
Now the variance.
For the shift, it doesn't affect the spread.
And so the variance is exactly going to be the same without the minus 1/2.
And for the constant, you can just pull that out and square it.
And the variance of y we know is 4.
And so that's 1/4 times 4, that's 1.
OK.
So now we know that y minus 1 over 2 is actually standard normal.
Actually for any normal random variable, you can follow the same procedure.
You just subtract its mean, which is 1 in this case.
And divide by its standard deviation and you will get a standard normal distribution.
All right, so for part C we want the probability that y is between negative 1 and 1.
So let's try to massage it so that we can use the standard normal table.
And we already know that this is standard normal, so let's subtract both sides by negative 1.
And that's equal to-- I'm going to call this standard normal z, so that's easier to write.
And that's equal to negative 1 less than equal to z, less than equal to zero.
So we're looking for this region, 0, 1, negative 1.
So that's just the probability that it's less than zero minus probability that it's less than negative 1.
Well for a standard normal, half the mass is below zero and a half the mass is above.
And so that's just going to be 0.5 directly.
And for this, we've already computed this for a standard normal, which was x in our case.
And that was 1 minus 0.8413.
Done.
So we basically calculated a few standard probabilities for normal distributions.
And we did that by looking them up from the standard normal table.
Hey guys.
Welcome back.
Today, we're going to be working on a problem that asks you to find the PMF of a function of a random variable.
So let's just jump right in.
The problem statement gives you the PMF for a random variable called x.
So we're told that there's this random variable x that takes on values minus 3, minus 2, minus 1, 1, 2, and 3.
And for each of those values, the probability mass lying over that value is given by this formula, x squared over a.
Now I didn't write it here to save room, but we're also told that a is a real number that is greater than 0.
And we're told that the probability of x taking on any value outside of the set is 0.
Now we're asked to do two things in the problem.
First is to find the value of the parameter a.
And that's sort of a natural question to ask, because if you think about it, the PMF isn't fully specified.
And in fact, if you plug in the wrong number for a, you actually won't get a valid PMF.
So we'll explore that idea in the first part.
And then the second part, you're given a new random variable called z.
And z happens to be a function of x.
In fact, it's equal to x squared.
And then you're asked to compute that PMF.
So this problem is a good practice problem.
I think, at this point, you guys are sort of newly acquainted with the idea of a PMF, or probability mass function.
So this problem will hopefully help you get more familiar with that concept and how to manipulate PMFs.
And by the way, just to make sure we're all on the same page, what does a PMF really tell you?
So p sub X, where this is a capital X, because the convention in this class is to use capital letters for random variables.
So p X of k, this is defined to be the probability that your random variable X takes on a value of k. So essentially, this says-- and this is just some number.
So in our particular case, this would be equal to k squared over a.
And how you can interpret this is this px guy is sort of like a machine.
He takes in some value that your random variable could take on, and then he spits out the amount of probability mass lying over that value.
OK.
So now that we've done that quick recap, let's get back to the first part of the problem.
So we have this formula for px of x, and we need to solve for a.
So in order to do that, we're going to use one of our axioms of probability to set up an equation.
And then we can solve precisely for a.
So namely, we know that every PMF must sum to 1.
And so essentially, if you sum this guy over all possible values of x, you should get a 1, and that equation will let us solve for a.
So let's do that.
Summation over x of px of x.
So here, essentially you're only summing over these six values.
So this is equal to px of minus 3, plus px of minus 2, plus px of minus 1, et cetera.
Oops.
px of 2 plus px of 3.
OK. And again, like the interpretation as we said, this number here should be interpreted as the amount of probability mass lying over minus 3.
And to help you visualize this, actually, before we go further with the computation, let's actually plot this PMF.
So the amount of probability mass lying over minus 3, the way we figure that out is we take minus 3 and we plug it into this formula up here.
So you get 9/a.
Now you can do this for minus 2.
You've got 4/a, looking at the formula.
For 1, you get 1/a.
And of course, this graph, it's the mirror image over 0, because of the symmetry.
So hopefully this little visualization helps you understand what I'm talking about.
And now we can just read these values off of the plot we just made.
So we know px minus 3 is equal to px of 3.
So we can go ahead and just take 2 times 9/a.
Similarly, we get 2 times 4/a, and then plus 2 times 1/a.
So now it's just a question of algebra.
So simplifying this, you're going to get 18 plus 8 plus 2, divided by a.
And this gives you 28/a.
And as I argued before, you know that if you sum a PMF over all possible values, you must get 1.
So this is equal to 1, which of course implies that a is equal to 28.
So what we've shown here is that you actually don't have a choice for what value a can take on.
It must take on 28.
And in fact, if you plug in any other value than 28 in here, you actually are not going to have a valid PMF, because it's not going to sum to 1.
OK.
So I'm going to write my answer here, and then erase to give myself more room for part
In this problem, we're going to look at the probability that when you take a stick and break it into three pieces randomly that these three pieces can actually be used to form a triangle.
All right, so we start out with a stick of unit length, so-- length 1.
And we'll choose a point along the stick to break.
And we'll choose that point uniformly at random.
So let's say that we chose it here, that was the point where we'll break it.
And then independently of this first choice we'll again choose a second point to break it.
Again, uniformly at random along the entire stick.
So let's say the second point we chose was here.
So what we have now is, we'll break it here, here, and so we'll have three pieces-- the first one, the left one and the right one.
And we want to know, what's the probability that when you take these three pieces you could form a triangle?
So the first thing we should ask ourselves is, what do you need-- what conditions must be satisfied in order to actually be able to form a triangle with three pieces?
So you could think about, what would stop you from being able to do that?
Well, one possibility is that you have pieces that look like this.
So in that case you would try to form something that looks like this.
But you can't get a triangle because these two pieces are too short and they can't touch each other.
So actually the condition that must be satisfied is that when you take any two of the three pieces, their combined length has to be greater than the length of the remaining third piece.
And that has to be true for any two pieces.
And really that's just so that any two pieces, they can touch and still form a triangle.
So let's try to add some probability to this.
So we have a unit length stick.
So let's actually give a coordinate system.
The stick goes from 0 to 1.
And let's say that we break it at these two points.
So the first point where we choose, we'll call that x.
So that's the first point that we choose to break it.
And then the second point we choose, we'll call that y.
Now note that I've drawn it so that x is to the left of y.
But it could actually be the case that the first point I chose is here and the second point that I chose is to the left.
But for now, let's first assume that this scenario holds.
That the first point is to the left of the second point.
So under this assumption, we can see that-- from the definition of these random variables-- we can actually see that the lengths are given by these three lengths.
So the lengths are x, the left most piece has length x.
The second, middle piece has length y minus x.
And the last piece has length 1 minus y.
And now let's recall our three conditions.
So the conditions were that any two of these, the sum of any two lengths, has to be at least-- has to be greater than the length of the third piece.
So let's do these together.
So x plus y minus x has to be greater than 1 minus y.
So with these two pieces you can cover this third piece.
We also need that with the first and third pieces, we can cover the middle piece.
And we need with the second and third pieces, we can cover the first piece.
Now this looks kind of messy, but in fact we can actually simplify this.
So this actually simplifies.
x minus x, that disappears.
And so this actually simplifies to 2y has to be at least 1.
Or even more simply, y has to be greater than 1/2.
What about this one?
This one, we can rearrange things again.
x we can move over.
y we can move over here.
And we get that 2x plus 1 has to be greater than 2y.
Or put in other words, y is less than x plus 1/2.
And for the last one, again we can simplify.
The y's cancel each other out.
And we're left with 2x is less than 1.
Or x is less than 1/2.
So these are our three conditions that need to be satisfied.
So now we just have to figure out what's the probability that this is actually satisfied?
Now let's go back to original definition and see what are the actual distributions for these random variables, x and y.
Remember, we defined them to be x is the location of the first break and y is the location of the second break.
And as we said in the problem, these are chosen uniformly at random and they're independent.
And so we can actually draw out their joint PDF.
So x and y, you can cover any point in the square.
And moreover, it's actually uniform within the square.
Because each one is chosen uniformly at random and they're independent.
So it's anywhere in here.
And so what do we need to do?
We just need to identify, what is the probability that these three conditions hold?
Rewrite this, line these up.
So these are our three conditions that we need.
And now remember, we're still working under the assumption that the first point that we chose is actually to the left of the second point.
So what does that mean?
That means that we are actually in this top triangle, top half-- x is less than y.
All right, so what do we need?
We need y to be at least 1/2, so here's 1/2.
So we need y to be above this line.
We need x to be less than 1/2.
So we need x to be to the left of here.
So now so far we're stuck in this upper square.
And the last thing we need is y to be less than x plus 1/2.
What is y?
The line y equals x and a 1/2, x plus 1/2, is this one.
So y has to be less than that, so it would have to be in this triangle here.
So these three conditions tell us that in order for us to have a triangle we need to for x and y to fall jointly in this small triangle here.
Now because the joint distribution is uniform, we know that the density is just 1, right?
Because the area here is just 1.
So the height is just 1 as well.
And so the density, or the probability of falling within this small triangle, is just going to be also the area of this triangle.
And what is the area of this triangle?
Well, you can fit 8 of these triangles in here, or you could think of it as 1/2 times 1/2 times 1/2.
So the area is just 1/8.
So assuming that x is less than y, then the probability of forming a triangle is 1/8.
Now, that's only half this story, though.
Because it's possible that when you chose these two break points that we actually had the opposite result.
That x, the point that you chose first, falls to the right of the point that you chose second.
In which case everything kind of flips.
Now we assume that y is less than x, which means that now we're in this lower triangle in the square.
Now we can go through this whole exercise again.
But really, what we can see is that all we've really done is just swap the names.
Instead of having x and y we now call x-- we call x y and we call y x.
And so if we just swap names, we can see that-- let's just fast forward through all these steps and see that we could just swap names here, too, as well, in the three conditions.
So instead of needing y to be greater than 1/2, we just need x to be greater than 1/2.
Instead of having x less than 1/2, we need y less than 1/2.
We also swap this.
So we need x to be less than y plus 1/2 or y is greater than x minus 1/2.
All right, now let's figure out what this corresponds to.
We need x to be greater than 1/2, so it needs to be to the right of here.
We need y to be less than 1/2, so we need it to be below this line.
And we need y to be greater than x minus 1/2.
What is the line y equals x minus 1/2?
That is this line here.
And we need y to be greater than that, so it needs to be above this line.
And so we get that this is the triangle, the small triangle that we need in this case.
And notice that it's exactly the same area as this one, right?
And so we get another contribution of 1/8 here.
So the final answer is 1/8 plus 1/8 is 1/4.
So the probability of forming a triangle using this three pieces is exactly 1/4.
And so notice that we've done is, you've set things up very methodically in the beginning by assigning these random variables.
And you consider different cases.
Because you don't actually know the order in which x and y might fall, let's just assume that one particular order and work from there.
And then do the other case, as well.
And it just so happened that because of the symmetry of the problem the second case was actually very simple.
We could just see that it is actually symmetric and so we get the same answer.
So this is kind of an interesting problem because it's actually a practical application of something that you might actually do.
And you can see that just by applying these probability concepts you can actually--
Hi.
In this problem, we're going to look at random incidence under Erlang arrivals.
First, let's parse what that means.
In a Poisson process, remember, the time between arrivals, or the inter-arrival time, is distributed as an exponential random variable.
And random incidence for a Poisson process refers to the somewhat surprising result that when you consider a specific time, say, T-star, then the length of the inter-arrival interval that contains that time T-star is not distributed as an exponential random variable.
It's actually distributed as an Erlang random variable of order 2 or it's distributed as a sum of two exponential random variables.
And the reason for that is that it comprises of two parts.
One is the time since the last arrival until T-star, which is exponentially distributed, and the time from T-star until the next arrival, which is also exponentially distributed.
So that brings us to a review of what Erlang random variables are.
An Erlang random variable of order k is just the sum of k independent and identically distributed exponential random variables.
So to be more specific, if Ti is an exponential random variable with parameter lambda, then if you take kiid copies of Ti and add them up, and call that Yk, then Yk is an Erlang random variable of order k. And one other fact is that the mean of Yk, the mean of an Erlang random variable of order k, is just k, the order, over lambda, which is the rate of the underlying exponential random variables.
So as a consequence, if you have an Erlang random variable of order two and that random variable also has a mean of two over lambda, we can interpret that random variable as just being the sum of two exponential random variables.
2 iid exponential random variables, T1 and T2, where each one takes exponential with the rate in lambda.
OK, so in this problem now, we're dealing with the random incidence not under Poisson processes, but under something else, which we call here an Erlang process with Erlang arrival times.
So to be more specific, what we're saying is that, instead of inter-arrival time being exponentially distributed, in this process, and inter-arrival time is actually distributed as an Erlang random variable of order 2 with mean 2 over lambda.
So to be explicit, this is no longer a Poisson process.
It's some other process because the inter-arrival times are not exponential.
So let's make use of this fact that we talked about earlier because now we know that the inter-arrival times of this Erlang process are Erlang order 2 with mean 2 over lambda.
But we know that that can just be re-interpreted as a sum of two simple exponentials, each with parameter lambda.
So let's just draw another picture and imagine that for each of these arrivals, so say we have three sample arrivals in this Erlang process, we can fill in, kind of, the gaps between these with additional arrivals.
And then think of each one of these times as all being exponential with parameter lambda.
So this is a valid interpretation because when we connect these, these inter-arrival times correspond to the combination of two inter-arrival times, which we know we can split that into just two exponentials.
So each one of these is an exponential random variable.
And when you combine them, you get an Erlang order of 2.
But the nice thing about this is that if we look at this diagram, it actually is just exactly a Poisson process with a rate lambda because now, what we're dealing with are exactly-- the inter-arrival times are now exactly exponential random variables.
And so this is in fact, now, just a simple Poisson process.
And we can also just think of it as we take the Poisson process, and take every other arrival, say, all the even-numbered arrivals, and make those corresponds to be arrivals in the Erlang process.
OK, so now let's think about some specific time T-star.
We want to know what is the distribution of the length of this to be specific inter-arrival interval that T-star is in.
Well, what we can do is take it down to the level of this Poisson process and look at it from there.
Well, we do that because, for a Poisson process, we know about random incidence for Poisson processes.
And we know how to deal with Poisson processes.
So let's think about this now.
Well, T-star is here.
And what we know from random incidence for a Poisson processes is that the length of this inter-arrival interval for the Poisson process, we know that this is an exponential plus an exponential.
So combined, this is Erlang order 2.
But that only covers from here to here.
And what we want is actually from here to there.
Well now, we tack on an extra exponential because we know that the inter-arrival times-- the time between this arrival and that arrival in the Poisson process is just another exponential.
And now all of these are in [INAUDIBLE] time intervals.
And they're all independent.
And so the time of this inter-arrival interval in the Erlang process is just going to be the sum of three independent exponentials within the underlying Poisson process.
And so to answer here is actually, it's going to be an Erlang of order 3.
Now this is one possible scenario for how this might occur.
Another scenario is actually that T-star is somewhere else.
So let's draw this again.
And suppose now, in this case, T-star landed between an even numbered arrival in the Poisson process and an odd numbered arrival.
Now it could also arrive between an odd numbered and an even numbered arrival.
So it could be that T-star is actually here.
Well, but in this case, it's actually more or less the same thing because now what we want is the length of this entire inter-arrival interval, which, in the Poisson world, we can break it down into random incidence within this interval, this inter-arrival interval, which is two exponentials, or an Erlang of 2, plus this interval, which is just a standard inter-arrival time, which is another exponential.
So in this case as well, we have the sum of three independent exponential random variables.
And so, in either case, we have that the inter-arrival time in the Erlang process is an Erlang of order 3.
And so the final answer is, in fact, that the inter-arrival for random incidence under these Erlang-type arrivals is an Erlang of order 3.
OK, so in this problem we looked at the random incidence under a different type of an arrival process, not Poisson, but with Erlang random variables.
But we used the insight that Erlang really can be re-interpreted as the sum of independent and identically distributed exponential random variables.
And exponential random variables can be viewed as one way of interpreting and viewing a Poisson process.
And so by going through those steps, we were able to use what we know about random incidence under Poisson processes to help us solve this problem of random incidence its Erlang arrivals.
So I hope that was helpful.
And I'll see you next time.
Hey, everyone.
Welcome back.
Today, we're going to do another fun problem that has to do with a random number of coin flips.
So the experiment we're going to run is as follows.
We're given a fair six-sided die, and we roll it.
And then we take a fair coin, and we flip it the number of times indicated by the die.
That is to say, if I roll a four on my die, then I flip the coin four times.
And then we're interested in some statistics regarding the number of heads that show up in our sequence.
In particular, we want to compute the expectation and the variance of the number of heads that we see.
So the first step of this problem is to translate the English to the math.
So we have to define some notation.
I went ahead and did that for us.
I defined n to be the outcome of the die role.
Now, since we flip the coin the number of times shown by the die roll, n is equivalently the number of flips that we perform.
And n, of course, is a random variable, and I've written its PMF up here.
So Pn of n is just a discrete uniform random variable between 1 and 6, because we're told that the die has six sides and that it's fair.
Now, I also defined h to be the number of heads that we see.
So that's the quantity of interest.
And it turns out that Bernoulli random variables will be very helpful to us in this problem.
So I defined x sub i as 1 if the ith flip is heads, and 0 otherwise.
And what we're going to do now is, we're going to use these x sub i's to come up with an expression for h. So if you want to count the number of heads, one possible thing you could do is start with 0 and then look at the first coin flip.
If it's heads, you add 1 to 0, which I'm going to call your running sum.
If the first flip is tails, you add 0.
And similarly, after that, after every trial, if you see heads, you add 1 to your running sum.
If you see a tails, you add 0.
And in that way, we can precisely compute h. So the mathematical statement of what I just said is that h is equal to x1 plus x2 plus x3, all the way through x sub n. So now, we are interested in computing e of h, the expectation of h. So your knee jerk reaction might be to say, oh, well, by linearity of expectation, we know that this is an expectation of x1, et cetera through the expectation of xn.
But in this case, you would actually be wrong.
Don't do that.
And the reason that this is not going to work for us is because we're dealing with a random number of random variables.
So each xi is a random variable.
And we have capital n of them.
But capital n is a random variable.
It denotes the outcome of our die roll.
So we actually cannot just take the sum of these expectations.
Instead, we're going to have to condition on n and use iterated expectation.
So this is the mathematical statement of what I just said.
And the reason why this works is because conditioning on n will take us to the case that we already know how to deal with, where we have a known number of random variables.
And of course, iterated expectations holds, as you saw in lecture.
I will briefly mention here that the formula we're going to derive is derived in the book.
And it was probably derived in lecture.
So if you want, you can just go to that formula immediately.
But I think the derivation of the formula that we need is quick and is helpful.
So I'm going to go through it quickly.
Let's do it over here.
Plugging in our running sum for h, we get this expression-- x1 plus x2 et cetera plus xn, conditioned on n. And this, of course, is n times the expectation of x sub i.
So again, I'm going through this quickly, because it's in the book.
But this step holds, because each of these xi's have the same statistics.
They're all Bernoulli with parameter of 1/2, because our coin is fair.
And so I used x sub i to say it doesn't really matter which integer you pick for i, because the expectation of xi is the same for all i.
So this now, the expectation of x sub i, this is just a number, it's just some constant, so you can pull it out of the expectation.
So you get the expectation of x sub i times the expectation of n. So I gave away the answer to this a second ago.
But x sub i is just a Bernoulli random variable with parameter of success of 1/2.
And we know already that the expectation of such a random variable is just p, or 1/2.
So this is 1/2 times expectation of n. And now n we know is a discrete uniform random variable.
And there's a formula that I'm going to use, which hopefully some of you may remember.
If you have a discrete uniform random variable that takes on values between a and b-- let's use w-- if you call this random variable w, then we have that the variance of w is equal to b minus a times b minus a plus 2 divided by 12.
So that's the variance.
We don't actually need the variance, but we will need this later.
And the expectation of w-- actually, let's just do it up here right ahead for this problem.
Because we have a discrete uniform random variable, the expectation is just the middle.
So you agree hopefully that the middle is right at 3.5, which is also 7/2.
So this is times 7/2, which is equal to 7/4.
So we are done with part of part a. I'm going to write this answer over here, so I can erase.
And we're going to do something very similar to compute the variance.
To compute the variance, we are going to also condition on n. So we get rid of this source of randomness.
And then we're going to use law of total variance, which you've also seen in lecture.
And again, the formula for this variance is derived in the book.
So I'm going to go through it quickly.
But make sure you understand this derivation, because it exercises a lot of stuff we taught you.
So this, just using law of total variance, is the variance of expectation of h given n, plus the expectation of the variance of h given n. And now, plugging in this running sum for h, you get this.
It's a mouthful to write.
Bear with me.
x1 through xn given n-- so I didn't do anything fancy.
I just plugged this into here.
So this term is similar to what we saw in a previous problem.
By linearity of expectation and due to the fact that all of the x i's are distributed in the same way, they have the same expectation, this becomes n times the expectation of x sub i.
And let's do this term over here.
This term-- well, conditioned on n, this n is known.
So we essentially have a finite known sum of independent random variables.
We know that the variance of a sum of independent random variables is the sum of the variances.
So this is the variance of x1 plus the variance of x2 et cetera, plus the variance of xn.
And furthermore, again, because all of these xi's have the same distribution, the variance is the same.
So we can actually write this as n times the variance of x sub i, where x sub i just corresponds to one of the trials.
It doesn't matter which one, because they all have the same variance and expectation.
So now, we're almost home free.
This is just some scaler.
So we can take it out of the variance, but we have to square it.
So this becomes expectation of xi squared times the variance of n. And then this variance is also just a scalar, so we can take it outside.
So then we get variance of x sub i times expectation of n. Now, we know that the expectation of x sub i is just the probability of success, which is 1/2.
So we have 1/2 squared, or 1/4, times the variance of n. So that's where this formula comes in handy.
b is equal to 6, a is equal to 1.
So we get that the variance of n is equal to 5 times-- and then 5 plus 2 is 7-- divided by 12.
So this is just a formula from the book that you guys hopefully remember.
So we get 35/12.
And then the variance of xi, we know the variance of a Bernoulli random variable is just p times 1 minus p. So in our case, that's 1/2 times 1/2, which is 1/4.
So we get 1/4.
And then the expectation of n, we remember from our previous computation, is just 7/2.
So I will let you guys do this arithmetic on your own time.
But the answer comes out to be 77/48.
So I will go ahead and put our answer over here-- 77/48-- so that I can erase.
So I want you guys to start thinking about part b while I erase.
Essentially, you do the same experiment that we did in part a, except now we use two dice instead of one.
So in part b, just to repeat, you now have two dice.
You roll them.
You look at the outcome.
If you have an outcome of four on one die and six on another die, then you flip the coin 10 times.
So it's the same exact experiment.
We're interested in the number of heads we want the expectation and the variance.
But this step is now a little bit different.
Again, let's approach this by defining some notation first.
Now, I want to let n1 be the outcome of the first die.
And then you can let n2 be the outcome of the second die.
And we'll start with just that.
So one way you could approach this problem is say, OK, if n1 is the outcome of my first die and n2 is the outcome of my second die, then the number of coin flips that I'm going to make is n1 plus n2.
This is the total coin flips.
So you could just repeat the same exact math that we did in part a, except everywhere that you see an n, you replace that n with n1 plus n2.
So that will get you to your answer, but it will require slightly more work.
We're going to think about this problem slightly differently.
So the way we are thinking about it just now, we roll two dice at the same time.
We add the results of the die rolls.
And then we flip the coin that number of times.
But another way you can think about this is, you roll one die, and then you flip the coin the number of times shown by that die and count the number of heads.
And then you take the second die and you roll it.
And then you flip the coin that many more times and count the number of heads after that.
So you could define h1 to be number of heads in the first n1 coin flips.
And you could just let h2 be the number of heads in the last n2 coin flips.
So hopefully that terminology is not confusing you.
Essentially, what I'm saying is, n1 plus n2 means you'll have n1 flips, followed by n2 flips, for a total of n1 plus n2 flips.
And then within the first n1 flips, you can get some number of heads, which we're calling h1.
And in the last n2 flips, you can get some number of heads, which is h2.
So the total number of heads that we get at the end-- I'm going to call it h star-- is equal to h1 plus h2.
And what part b is really asking us for is the expectation of h star and the variance of h star.
But here's where something really beautiful happens.
h1 and h2 are independent, and they are statistically the same.
So the reason why they're independent is because-- well, first of all, all of our coin flips are independent.
And they're statistically the same, because the experiment is exactly the same.
And everything's independent.
So instead of imagining one person rolling two die and then summing the outcomes and flipping a coin that many times and counting heads, you can imagine one person takes one die and goes into one room.
A second person takes a second die and goes into another room.
They run their experiments.
Then they report back to a third person the number of heads.
And that person adds them together to get h star.
And in that scenario, everything is very clearly independent.
So the expectation of h star-- you actually don't need independence for this part, because linearly of expectation always holds.
But you get the expectation of h1 plus the expectation of h2.
And because these guys are statistically equivalent, this is just two times the expectation of h. And the expectation of h we calculated in part a.
So this is 2 times 7 over 4.
Now, for the variance, here's where the independence comes in.
I'm actually going to write this somewhere where I don't have to bend over.
So the variance of h star is equal to the variance of h1 plus the variance of h2 by independence.
And that's equal to 2 times the variance of h, because they are statistically the same.
And the variance of h we computed already.
So this is just 2 times 77 over 48.
So the succient answer to part b is that both the mean and the variance double from part A.
So hopefully you guys enjoyed this problem.
We covered a bunch of things.
So we saw how to deal with having a random number of random variables.
Usually we have a fixed number of random variables.
In this problem, the number of random variables we were adding together was itself random.
So to handle that, we conditioned on n. And to compute expectation, we use iterated expectation.
To compute variance, we used law of total variance.
And then in part b, we were just a little bit clever.
We thought about how can we reinterpret this experiment to reduce computation.
And we realized that part b is essentially two independent trials of part a.
So both the mean and the variance should double.
Today, we're going to do a fun problem called rooks on a chessboard.
And rooks on a chessboard is a problem that's going to test your ability on counting.
So hopefully by now in class, you've learned a few tricks to approach counting problems.
You've learned about permutations, you've learned about k-permutations, you've learned about combinations, and you've learned about partitions.
And historically for students that we've taught in the past and many people, counting can be a tricky topic.
So this is just one drill problem to help you get those skills under your belt.
So what does the rooks on a chessboard problem ask you?
Well, you're given an 8-by-8 chessboard, which I've tried to draw here.
It's not very symmetrical.
Sorry about that.
And you're told that you have eight rooks.
I'm sure most of you guys are familiar with chess.
But if any of you aren't, chess is a sophisticated board game.
And one of the types of pieces you have in this game is called a rook.
And in this particular problem, there are eight rooks.
And your job is to place all eight rooks onto this 8-by-8 chessboard.
Now, you're told in the problem statement that all placements of rooks are equally likely.
And you are tasked with finding the probability that you get a safe arrangement.
So that is to say, you place your eight rooks on the board.
What is the probability that the way you placed them is safe?
So what do I mean by "safe"?
Well, if you're familiar with the way chess works, so if you place a rook here, it can move vertically or it can move horizontally.
Those are the only two legal positions.
So if you place a rook here and you have another piece here, then this is not a safe arrangement, because the rook can move this way and kill you.
Similarly, if you have a rook here and another piece here, the rook can move horizontally and kill you that way.
So two rooks on this board are only safe from each other if they are neither in the same column nor in the same row.
And that's going to be key for us to solve this problem.
So let's see-- where did my marker go?
I've been talking a lot, and I haven't really been writing anything.
So our job is again, to find the probability that you get a safe arrangement.
So I'm just going to do "arrange" for short.
Now, I talked about this previously, and you guys have heard it in lecture.
Hopefully you remember something called the discrete uniform law.
So the discrete uniform law is applicable when your sample space is discrete and all outcomes are equally likely.
So let's do a quick check here.
What is our sample space for this problem?
Well, a logical choice would be that the set of all possible outcomes is the set of all possible spatial arrangements of rooks.
And hopefully it's clear to you that that is discrete.
And the problem statement furthermore gives us that they're equally likely.
So the discrete uniform law is in fact applicable in our setting.
So I'm going to go ahead and write what this means.
So when your sample space is discrete and all outcomes are equally likely, then you can compute the probability of any event, A, simply by counting the number of outcomes in A and then dividing it by the total number of outcomes in your sample space.
So here we just have to find the number of total safe arrangements and then divide it by the total number of arrangements.
So again, as you've seen in other problems, the discrete uniform law is really nice, because you reduce the problem of computing probabilities to the problem of counting.
And so here's where we're going to exercise those counting skills, as I promised earlier.
Now, I would like to start with computing the denominator, or the total number of arrangements, because I think it's a slightly easier computation.
So we don't care about the arrangements being safe.
We just care about how many possible arrangements are there.
Now, again, we have eight rooks, and we need to place all of them.
And we have this 8-by-8 board.
So pretty quickly, you guys could probably tell me that the total number of square is 64, because this is just 8 times 8.
Now, I like to approach problems sequentially.
That sort of really helps me think clearly about them.
So I want you to imagine a sequential process during which we place each rook one at a time.
So pick a rook.
The chessboard is currently empty.
So how many squares can you place that rook in?
Well, nobody's on the board.
You can place it in 64 spots.
So for the first rook that you pick, there are 64 spots.
Now, once you place this rook, you need to place the second rook, because again, we're not done until all eight are placed.
So how many possible spots are left.
Well, I claim that there are 63, because one rule of chess is that if you put a piece in a particular square, you can no longer put anything else on that square.
You can't put two or more things.
So the first rook is occupying one spot, so there's only 63 spots left.
So the second rook has 63 spots that it could go in.
Similarly, the third rook has 62 spots.
Hopefully you see the pattern.
You can continue this down.
And remember, we have to place all eight rooks.
So you could do it out yourself or just do the simple math.
You'll figure out that the eighth rook only has 57 spots that it could be in.
So this is a good start.
We've sort of figured out if we sequentially place each rook, how many options do we have.
But we haven't combined these numbers in any useful way yet.
We haven't counted the number of total arrangements.
And this may already be obvious to some, but it wasn't obvious to me when I was first learning this material, so I want to go through this slowly.
You have probably heard in lecture by now about the counting principle.
And what the counting principle tells you is that whenever you have a process that is done in stages and in each stage, you have a particular number of choices, to get the total number of choices available at the end of the process, you simply multiply the number of choices at each stage.
This might be clear to you, again, simply from the statement, for some of you.
But for others, it might still not be clear.
So let's just take a simple example.
Forget about the rook problem for a second.
Let's say you're at a deli, and you want to make a sandwich.
And to make a sandwich, you need a choice of bread and you need a choice of meat.
So we have a sandwich-building process, and there's two stages.
First, you have to pick the bread, and then you have to pick the meat.
So let's say for the choice of bread, you can choose wheat or rye.
So again, you can always use a little decision tree-- wheat or rye.
And then let's say that for the meats, you have three options.
You have ham, turkey, and salami.
So you can have ham, turkey, or salami-- ham, turkey, or salami.
How many total possible sandwiches can you make?
Well, six.
And I got to that by 2 times 3.
And hopefully this makes sense for you, because there's two options in the first stage.
Freeze an option.
Given this choice, there's three options at the second stage.
But you have to also realize that for every other option you have at the first stage, you have to add an additional three options for the second stage.
And this is the definition of multiplication.
If you add three two times, you know that's 3 times 2.
So if you extrapolate this example to a larger, more general picture, you will have derived for yourself the counting principle.
And we're going to use the counting principle here to determine what the total number of arrangements are.
So we have a sequential process, because we're placing the first rook and then the second rook, et cetera.
So at the first stage, we have 64 choices.
At the second stage, we have 63 choices.
At the third stage, we have 62 choices, et cetera.
And so I'm just multiplying these numbers together, because the counting principle says I can do this.
So my claim is that this product is equal to the total number of arrangements.
And we could stop here, but I'm going to actually write this in a more useful way.
You guys should have been introduced to the factorial function.
So you can express this equivalently as 64 factorial divided by 56 factorial.
And this is not necessary for your problem solution, but sometimes it's helpful to express these types of products in factorials, because you can see cancellations more easily.
So if it's OK with everybody, I'm going to erase this work to give myself more room.
So we'll just put our answer for the denominator up here, and then we're going to get started on the numerator.
So for the numerator, thanks to the discrete uniform law, we only need to count the number of safe arrangements.
But this is a little bit more tricky, because now, we have to apply our definition of what "safe" means.
But we're going to use the same higher-level strategy, which is realizing that we can place rooks sequentially.
So we can think of it as a sequential process.
And then if we figure out how many choices you have in each stage that sort of maintain the "safeness" of the setup, then you can use the counting principle to multiply all those numbers together and get your answer.
So we have to place eight rooks.
Starting the same way we did last time, how many spots are there for the first rook that are safe?
Nobody is on the board yet, so nobody can harm the first rook we put down.
So I claim that it's just our total of 64.
Now, let's see what happens.
Let's pick a random square in here.
Let's say we put our first rook here.
Now, I claim a bunch of spots get invalidated because of the rules of chess.
So before, I told you a rook can kill anything in the same column or in the same row.
So you can't put a rook here, because they'll kill each other, and you can't put a rook here.
So by extension, you can see that everything in the column and the row that I'm highlighting in blue, it's no longer an option.
You can't place a rook in there.
Otherwise, we will have violated our "safety" principle.
So where can our second rook go?
Well, our second rook can go in any of the blank spots, any of the spots that are not highlighted by blue.
And let's stare at this a little bit.
Imagine that you were to take scissors to your chessboard and cut along this line and this line and this line and this line.
So you essentially sawed off this cross that we created.
Then you would have four free-floating chessboard pieces-- this one, this one, this one, and this one.
So this is a 3-by-4 piece, this is 3-by-3, this is 4-by-3, and this is 4-by-4.
Well, because you cut this part out, you can now slide those pieces back together.
And hopefully you can convince yourself that that would leave you with a 7-by-7 chessboard.
And you can see that the dimensions match up here.
So essentially, the second rook can be placed anywhere in the remaining 7-by-7 chessboard.
And of course, there are 49 spots in a 7-by-7 chessboard.
So you get 49.
So let's do this experiment again.
Let me rewrite the reduced 7-by-7 chessboard.
You're going to have to forgive me if the lines are not perfect-- one, two, three, four, five, six, seven; one, two, three, four, five, six, seven.
Yep, I did that right.
And then we have one, two, three, four, five, six, seven.
That's not too bad for my first attempt.
So again, how did I get this chessboard from this one?
Well, I took scissors and I cut off of the blue strips, and then I just merged the remaining four pieces.
So now, I'm placing my second rook.
So I know that I can place my second rook in any of these squares, and it'll be safe from this rook.
Of course, in reality, you wouldn't really cut up your chessboard.
I'm just using this as a visual aid to help you guys see why there are 49 spots.
Another way you could see 49 spots is literally just by counting all the white squares, but I think it takes time to count 49 squares.
And this is a faster way of seeing it.
So you can put your second rook anywhere here.
Let's actually put in the corner, because the corner is a nice case.
If you put your rook in the corner, immediately, all the spots in here and all the spots in here become invalid for the third rook, because otherwise, the rooks can hurt each other.
So again, you'll see that if you take scissors and cut off the blue part, you will have reduced the dimension of the chessboard again.
And you can see pretty quickly that what you're left with is a 6-by-6 chessboard.
So for the third rook, you get a 6-by-6 chessboard, which has 36 free spots.
And I'm not going to insult your intelligence.
You guys can see the pattern-- 64, 49, 36.
These are just perfect squares decreasing.
So you know that the fourth rook will have 25 spots.
I'm going to come over here because I'm out of room.
The fifth rook will have 16 spots.
The sixth rook will have nine spots.
The seventh rook will have four spots.
And the eighth rook will just have one spot.
And now, here we're going to invoke the counting principle again.
Remember the thing that I just defined to you by talking about sandwiches.
And we'll see that to get the total number of safe arrangements, we can just multiply these numbers together.
So I'm going to go ahead and put that up here.
You get 64 times 49 times 36 times 25 times 16 times 9 times 4.
And in fact, this is our answer.
So we're all done.
So I really like this problem, because we don't normally ask you to think about different spatial arrangements.
So it's a nice exercise, because it lets you practice your counting skills in a new and creative way.
And in particular, the thing that we've been using for a while now is the discrete uniform law.
But now, I also introduced the counting principle.
And we used the counting principle twice-- once to compute the numerator and once to compute the denominator.
Counting can take a long time for you to absorb it.
So if you still don't totally buy the counting principle, that's OK.
I just recommend you do some more examples and try to convince yourself that it's really counting the right number of things.
So counting principle is the second takeaway.
And then the other thing that is just worth mentioning is, you guys should get really comfortable with these factorials, because they will just show up again and again.
So that's the end of the problem, and I'll see you next time.
Hi.
In this problem, we're dealing with buses of students going to a job convention.
And in the problem, we'll be exercising our knowledge of PMFs-- probability mass functions.
So we'll get a couple of opportunities to write out some PMFs, and also calculating expectations or expected values.
And also, importantly, we'll actually be exercising our intuition to help us not just rely on numbers, but also to just have a sense of what the answers to some probability questions should be.
So the problem specifically deals with four buses of students.
So we have buses, and in each one carries a different number of students.
So the first one carries 40 students, the second one 33, the third one has 25, and the last one has 50 students for a total of 148 students.
And because these students are smart, and they like probability, they are interested in a couple questions.
So suppose that one of these 148 students is chosen randomly, and so we'll assume that what that means is that each one has the same probability of being chosen.
So they're chosen uniformly at random.
And let's assign a couple of random variables.
So we'll say x corresponds to the number of students in the bus of the selected student.
OK, so one of these 148 students is selected uniformly at random, and we'll let x correspond to the number of students in that student's bus.
So if a student from this bus was chosen, then x would be 25, for example.
OK, and then let's come up with another random variable, y, which is almost the same thing.
Except instead of now selecting a random student, we'll select a random bus.
Or equivalently, we'll select a random bus driver.
So each bus has one driver, and instead of selecting one of the 148 students at random, we'll select one of the four bus drivers also uniformly at random.
And we'll say the number of students in that driver's bus will be y.
So for example, if this bus driver was selected, then y would be 33.
OK, so the main problem that we're trying to answer is what do you expect the expectation-- which one of these random variables do you expect to have the higher expectation or the higher expected value?
So, would you expect x to be higher on average, or y to be higher?
And what would be the intuition for this?
So obviously, we can actually write out the PMFs for x and y.
These are just discrete random variables.
And we can actually calculate out what the expectation is.
But it's also useful to exercise your intuition, and your sense of what the answer should be.
So it might not be immediately clear which one would be higher, or you might even say that maybe it doesn't make a difference.
They're actually the same.
But a useful way to approach some of these questions is to try to take things to the extreme and see how that plays out.
So let's take the simpler example and take it to the extreme and say, suppose a set of four buses carrying these number of students.
We have only two buses-- one bus that has only 1 student, and we have another bus that has 1,000 students.
OK. And suppose we ask the same question.
Well, now if you look at it, there's a total of 1,001 students now.
If you select one of the students at random, it's overwhelmingly more likely that that student will be one of the 1,000 students on this huge bus.
It's very unlikely that you'll get lucky and select the one student who is by himself.
And so because of that, you have a very high chance of selecting the bus with the high number of students.
And so you would expect x, the number of students, to be high-- to be almost 1,000 in the expectation.
But on the other hand, if you selected the driver at random, then you have a 50/50 chance of selecting this one or that one.
And so you would expect the expectation there to be roughly 500 or so.
And so you can see that if you take this to the extreme, then it becomes more clear what the answer would be.
And the argument is that the expectation of x should be higher than the expectation of y, and the reason here is that because you select the student at random, you're more likely to select a student who is in a large bus, because that bus just has more students to select from.
And because of that, you're more biased in favor of selecting large buses, and therefore, that makes x higher in expectation.
OK, so that's the intuition behind this problem.
And now, as I actually go through some of the more mechanics and write out what the PMFs and the calculation for the expectation would be to verify that our intuition is actually correct.
OK, so we have two random variables that are defined.
Now let's just write out what their PMFs are.
So the PMF-- we write it as little P of capital X and little x.
So the random variable-- what we do is we say the probability that it will take on a certain value, right?
So what is the probability that x will be 40?
Well, x will be 40 if a student from this bus was selected.
And what's the probability that a student from this bus is selected?
That probability is 40/148, because there's 148 students, 40 of whom are sitting in this bus.
And similarly, x will be 33 with probability 33/148, and x will be 25 with probability 25/148.
And x will be 50 with probability 50/148.
And it will be 0 otherwise.
OK, so there is our PMF for x, and we can do the same thing for y.
The PMF of y-- again, we say what is the probability that y will take on certain values?
Well, y can take on the same values as x can, because we're still dealing with the number of students in each bus.
So y can be 40.
But the probability that y is 40, because we're selecting the driver at random now, is 1/4, right?
Because there's a 1/4 chance that we'll pick this driver.
And the probability that y will be 33 will also be 1/4, and the same thing for 25 and 50.
And it's 0 otherwise.
OK, so those are the PMFs for our two random variables, x and y.
And we can also draw out what the PMFs look like.
So if this is 25, 30, 35, 40, 45, and 50, then the probability that it's 25 is 25/148.
So we can draw a mass right there.
For 33, it's a little higher, because it's 33/148 instead of 25.
For 40, it's even higher still.
It's 40/148.
And for 50, it is still higher, because it is 50/148.
And so you can see that the PMF is more heavily favored towards the larger values.
We can do the same thing for y, and we'll notice that there's a difference in how these distributions look.
So if we do the same thing, the difference now is that all four of these masses will have the same height.
Each one will have height 1/4, whereas this one for x, it's more heavily biased in favor of the larger ones.
And so because of that, we can actually now calculate what the expectations are and figure out whether or not our intuition was correct.
OK, so now let's actually calculate out what these expectations are.
So as you recall, the expectation is calculated out as a weighted sum.
So for each possible value of x, you take that value and you weight it by the probability of the random variable taking on that value.
So in this case, it would be 40 times 40/148, 33 times 33/148, and so on.
48 plus 25 times 25/148 plus 50 times 50/148.
And if you do out this calculation, what you'll get is that it is around 39.
Roughly 39.
And now we can do the same thing for y.
But for y, it's different, because now instead of weighting it by these probabilities, we'll weight it by these probabilities.
So each one has the same weight of 1/4.
So now we get 40 times 1/4 plus 33 times 1/4.
That's 25 times 1/4 plus 50 times 1/4.
And if you do out this arithmetic, what you get is that this expectation is 37.
And so what we get is that, in fact, after we do out the calculations, the expected value of x is indeed greater than the expected value of y, which confirms our intuition.
OK, so this problem, to summarize-- we've reviewed how to write out a PMF and also how to calculate expectations.
But also, we've got a chance to figure out some intuition behind some of these problems.
And so sometimes it's helpful to take simpler things and take things to the extreme and figure out intuitively whether or not the answer makes sense.
It's useful just to verify whether the numerical answer that you get in the end is correct.
Does this actually make sense?
It's a useful guide for when you're solving these problems.
OK, so we'll see you next time.
Hi.
In this problem, we're going to practice setting up a Markov chain by going fishing in this lake, which has n fish in it, some of which are green.
And the rest of the fish are blue.
So, what we do is, every day we go to this lake, and we catch exactly 1 fish.
And all the fish are equally likely to be the 1 that's caught.
Now, if we catch a green fish, we paint it blue, and we throw back into the lake.
And if we catch a blue fish, we just keep it blue, and we also throw it back.
Now, what we're interested in modeling is, how does this lake evolve over time?
And specifically what we're interested in is the number of green fish that are left in the lake.
So, let's let Gi be the event that there are i green fish left in the lake.
And we want to know, how does Gi evolve over time?
Now, one thing that we've learned that we can use to model this is a Markov chain.
But before we can use it, we need to make sure that this actually satisfies the Markov property.
Now, recall that the Markov property essentially says that, given the current state of the system, that's all you need in order to predict the future states.
So, any past history of the previous states that it was in, that's all irrelevant.
All you need is the current state.
Now, in the context of this particular problem, what that means is that if I tell you that there are 10 green fish left, that's all the information you need in order to predict how many fish there will be tomorrow.
So, why is that?
Well, it's because what influences the number of green fish that are left?
What influences it is which fish you catch because, depending on which fish you catch, you may paint the green fish blue, in which case the number of green fish decrease.
But what affects which fish you catch?
Well, that probability is dictated solely based on just the number of green fish in the lake right now, today.
So, it doesn't matter that there were 20 fish yesterday.
All that matters is how many green fish there are in the lake today.
And so, because of that argument, the number of green fish-- this does satisfy the Markov property, so we can use this and model it as a Markov chain.
So, like we alluded to just now, the key dynamic that we need to look at is, how does the number of green fish change?
And if we look at it, we notice that after each day, the number of green fish can only have two possible transitions.
One possible transition is that it goes down by exactly 1, which happens if you happen to catch a green fish and paint it blue.
So, that green fish is no longer green, so the number of green fish goes down by 1.
The other possible transition is that Gi doesn't change because you caught a blue fish that day.
So, all the green fish are still green.
So, now given that, let's see if we can come up with a Markov chain.
So, the first thing we've done is we've written down all the different states, right?
So, this represents the number of green fish left in the lake.
So, there could be 0 green fish left, 1 green fish, all the way through n, which means that every single fish in the lake is green.
Now, we have the states.
What we need to do now is to fill in the transition probabilities, which are the Pij's.
And remember, the Pij is the probability of transitioning from state i to state j in the next transition.
So, what that means in this context is, what's the probability that there will be j green fish tomorrow given that there are i green fish today?
Now, if we go back to our earlier argument, we see that for any given i, you can only transition to two possible j's.
One of them is you stay at i because the number of green fish doesn't change because you caught a blue fish.
And the other is that you'd go from i to i minus 1.
The number of green fish decreases by 1.
Now, what we need to do now is fill in what those probabilities are.
So, if j equals i, meaning that the number of green fish doesn't change, well, what's the probability that you have the same number of green fish tomorrow as you do today?
Well, if you have i green fish today, that happens if you catch 1 of the n minus i blue fish.
So, what's the probability of catching one of the n minus i blue fish?
Well, it's n minus i over n. Now, the other possible transition is you go from a i to j equals i minus 1, so i goes down by 1.
And that happens when you catch a green fish.
So, given that there are i green fish, what's the probability that you catch 1 of those?
Well, it's going to be i/n.
And finally, every other transition has 0 probability.
All right.
So, now we can add those transitions on to our Markov chain.
So, for example, we have these.
So, let's look at this general case i.
So, if you're state i, you have i green fish left.
You will transition to i minus 1 green fish left if that day you caught a green fish.
And we said that that probability is i/n.
And the self transition probability is you caught a blue fish that day, so you still stay a i green fish.
And that probability, we said, was n minus i over n. All right.
Now, it's helpful to verify that this formula works by looking at some cases where it's intuitive to calculate what these probabilities should be.
So, let's look at state n. That is the state where every single fish in the lake is green.
So, if ever single fish in the lake is green, then no matter what fish you catch, it's going to be green.
And you're going to paint it blue and return it, so you're guaranteed to go down to n minus 1 green fish.
And so, this transition probability down to n minus 1 is guaranteed to be 1.
And so, the self transition probability has to be 0.
Now, let's go back to our formula and verify that actually gives us the right value.
So, if i is n, then there's only these transition probabilities.
So, if i is n, then the transition probability to j, for j is also n, is n minus n over n, which is 0.
And that's exactly what we said.
We argued that the self transition probability should be 0.
And also, if i is in, the probability of transitioning to n minus 1 should be n over n, which is 1.
And that's exactly what we argued here.
So, it seems like these transition probabilities do make sense.
And if we wanted to, we could fill in the rest of these.
So, for example, this would be 2/n, 1/n, n minus 1 over n, n minus 2 over n. And now, let's also consider the case of state 0, which means that every single fish is blue.
There are 0 green fish left.
Well, if that's the case, then what's the probability of staying at 0?
Well, that's n minus 0 over n is 1, all right?
So, the self transition probability is 1.
And that makes sense because if you have 0 green fish, there's no way to generate more green fish because you don't paint blue fish green.
And so, you're going to stay at 0 green fish forever.
All right.
So, we've characterized the entire Markov chain now.
And so, now let's just answer some simple questions about this.
So, the problem asks us to identify, what are the recurrent and transient states?
So, remember that recurrent state means that if you start out at that state, no matter where you go, what other states you end up at, there is some positive probability path that will take you back to your original state.
And if you're not recurrent, then you're transient, which means that if you're transient, if you start out at the transient state, there is some other state that you can go to, from which there's no way to come back to the original transient state.
All right.
So, now let's look at this and see which states are recurrent and which are transient.
And we can fill this in more.
And if we look at it, let's look at state n. Well, we're guaranteed to go from state n to state n minus 1.
And once we're in state n minus 1, there's no way for us to go back to state n because we can't generate more green fish.
And so, n is transient.
And similarly, we can use the same argument to show that everything from 1 through n, all of these states, are transient for the same reason because there's no way to generate more green fish.
And so, the chain can only stay at a given state or go down 1.
And so, it always goes down.
It can only go left, and it can never go right.
So, once you leave a certain state, there's no way to come back.
And so, states 1 through n are all transient.
And 0 the only recurrent state because, well, the only place you go from 0 is itself.
So, you always stay at 0.
And in fact, 0 is not only recurrent, it's absorbing because every single other state, no matter where you start out at, you will always end up at 0.
So, this was just an example of how to set up a Markov chain.
You just think about the actual dynamics of what's going on and make sure that it satisfies the Markov property.
Then, figure out what all the states are and calculate all the transition probabilities.
And once you have that, you've specified your Markov chain.
Hi.
In this problem, we have an absent-minded professor who will inadvertently give us some practice with exponential random variables.
So the professor has made two appointments with two students and inadvertently made them at the same time.
And what we do is we model the duration of these appointments with an exponential random variable.
So remember, an exponential random variable is a continuous random variable that takes on non-negative values, and it's parametrized by a rate parameter, lambda.
And the exponential random variable is often used to model durations of time-- so time until something happens, so for example, in this case, time until the student leaves or the appointment is over.
Or sometimes you will also use it to be as a model of time until something fails.
And one thing that will be useful is the CDF of this exponential random variable.
So the probability that it's less than or equal to some value, little t, is equal to 1 minus e to the minus lambda t. So this is, of course, valid only when t is non-negative.
The other useful property is that the expected value of an exponential random variable is just 1 over the parameter lambda.
And the last thing that we'll use specifically in this problem is the memory list property of exponential random variables.
And so recall that that just means that if you pop in the middle of an exponential random variable, the distribution for that random variable going forward from the point where you popped in is exactly the same as if it had just started over.
So that's why we call it the memory list property.
Basically, the past doesn't really matter, and going forward from whatever point that you observe it at, it looks as if it had just started over afresh.
And last thing we'll use, which is a review of a concept from earlier, is total expectation.
So let's actually model this problem with two random variables.
Let's let T1 be the time that the first student takes in the appointment and T2 be the time that the second student takes.
And what we're told in the problem is that they're both exponential with mean 30 minutes.
So remember the mean being 30 minutes means that the lambda is 1 over the mean.
And so the lambda in this case would be 1/30.
And importantly, we're also told that they are independent.
So how long the first person takes is independent of how long the second person takes.
So the first student arrives on time and take some random amount of time, T1.
The second student arrives exactly five minutes late.
And whatever the second person meets with the professor, that student will then take some random amount of time, T2.
What we're asked to do is find the expected time between when the first student arrives-- so we can just call that time 0-- and when the second student leaves.
Now you may say, well we're dealing with expectations, so it's easier.
And in this case, it probably is just the expectation of how long the first student takes plus the expectation of how long the second student takes.
So it should be about 60 minutes or exactly 60 minutes.
Now, why is that not exactly right?
It's because there is a small wrinkle, that the students may not go exactly back to back.
So let's actually draw out a time frame of what might actually happen.
So here's time 0, when the first student arrives.
And the first will go for some amount of time and leave.
And now let's consider two scenarios.
One scenario is that the first student takes more than five minutes to complete.
Well then the second student will have arrived at 5 minutes and then will be already waiting whenever this first student leaves.
So then the second student will immediately pick up and continue.
And in that case, we do have two exponentials back to back.
But there could be another situation.
Suppose that the first student didn't take very long at all and finished within five minutes, in which case the second student hasn't arrived yet.
So this professor is idle in between here.
And so we actually don't necessarily have two of them going back to back.
So there's an empty period in between that we have to account for.
So with that in mind, we see that we have two scenarios.
And so what does that beg to use?
Well, we can split them up into the two scenarios and then calculate expectations with each one and then use total expectation to find the overall expected length of time.
OK, so let's begin with the first scenario.
The first scenario is that, let's say, the first student finished within five minutes.
So what does that mean in terms of the definitions that we've used?
That means T1 is less than or equal to 5.
So if the first student took less than five minutes, then what happens?
Then we know that the amount of time that you'd need to take-- let's call that something else.
Let's call that X.
So X is the random variable that we're interested in, the time between when the first student comes and the second student leaves.
This is the value that we want to find.
Well we know that we're guaranteed that there will be a five-minute interval.
So first student will come, and then the second person will take over.
So we're guaranteed that the first five minutes will be the difference between when time starts and when the second student arrives.
And then, after that, it's just however long the second student takes, which is just the expected value of T2.
And T2 is a exponential random variable with mean 30.
So in this case, it's just 35.
So the first student doesn't take very long.
Then we just get the five minutes, that little buffer, plus however long the second student takes, which, on average, is 30 minutes.
Now what is the probability of this happening?
The probability of this happening is the probability that the first student takes less than five minutes.
And here is where we use the CDF that we wrote out earlier.
It's going to be 1 minus e to the minus lambda t. So in this case, t is five and lambda is 1/30.
So it's minus 5/30 is the probability.
All right, now let's consider the second case.
The second case is that the first student actually takes longer than five minutes.
OK, so what happens in that case?
Here's five minutes.
The first student came to five minutes.
The second student arrived, and the first student is still going.
So he goes for some amount of time.
And then whenever he finishes, the second student continues.
So now the question is, what is the total amount of time in this case?
Well, you can think of it as using the memory list property.
This is where it comes in.
So the first five minutes, we know that it was already taken because we're considering the second scenario, which we're given that T1 is greater than 5.
And so the question now is, if we know that, how much longer does it take?
How much longer past the five-minute mark does the first student take?
And by the member list property, we know that it's as if the first student started over.
So there was no memory of the first five minutes, and it's as if the first student just arrived also at the five-minute mark and met with the professor.
So past the five-minute mark, it's as if you have a new exponential random variable, still with mean 30.
And so what we get is that, in this case, you get the guaranteed five minutes, and then you get the memory list continuation of the first student's appointment.
So you get another 30 minutes on average because of the memory list property.
And then whenever the first student finally does finish up, the second student will immediately take over because he has already arrived.
It's past the five-minute mark.
And then that second student will take, again, on average, 30 more minutes.
So what you get is, in this case, the appointment lasts 65 minutes on average.
Now what is the probability of this case?
The probability of this case is the probability that T1 is greater than 5.
And now we know that that is just the complement of this, 1 minus that.
So it's just e to the minus 5/30.
So now we have both scenarios.
We have the probabilities of each scenario, and we have the expectation under each scenario.
Now all that remains now is to combine them using total expectation.
So I really should have written expectation of X given T1 is less than or equal to 5 here.
And this is expectation of X given that T1 is greater than 5.
So expectation of X overall is the probability that T1 is less than or equal to 5 times the expectation of X given that T1 is less than or equal to 5 plus the probability that T1 is greater than 5 times the expectation of X given that T1 is greater than 5.
And we have all four of these pieces here.
so it's 35 times 1 minus e to the minus 5/30 plus 65 times e to the minus 5 5/30.
And it turns out that this is approximately equal to 60.394 minutes.
All right, so what have we found?
We found that the original guess that we had, if we just had two meetings back to back, was on average it would take 60 minutes.
It turns out that, because of the way that things are set up, because of the five minute thing, it actually takes a little longer than 60 minutes on average.
And why is that?
It's because the five sometimes adds an extra buffer, adds a little bit of extra amount, because it would have been shorter in this scenario because, if the both students had arrived on time, then the second student would have been able to pick up right here immediately.
And so both appointments would have ended sooner.
But because the second student didn't arrive until five minutes, there was some empty space that was wasted.
And that's where you get you the little bit of extra time.
So this is a nice problem just to get some more exercise with exponential random variables and also nicely illustrates the memory list property, which was a key points in order to solve this.
And it also is nice because we get to review a useful tool that we've been using all course long, which is to split things into different scenarios and then solve the simpler problems and then combine them up, for example using total expectation.
So I hope that was helpful, and see you next time.
In this problem, Romeo and Juliet are to meet up for a date, where Romeo arrives at time x and Juliet at time y, where x and y are independent exponential random variables, with parameters lambda.
And we're interested in knowing the difference between the two times of arrivals, we'll call it z, written as x minus y.
And we'll like to know what the distribution of z is, expressed by the probability density function, f of z.
Now, we'll do so by using the so-called convolution formula that we learn in the lecture.
Recall that if we have a random variable w that is the sum of two independent random variables, x plus y, now, if that's the case, we can write the probability [INAUDIBLE] function, fw, [INAUDIBLE] as the following integration-- negative infinity to infinity fx little x times f of y w minus x, integrated over x.
And to use this expression to calculate f of z, we need to do a bit more work.
Notice w is expressed as a sum of two random variables, whereas z is expressed as the subtraction of y from x.
But that's fairly easy to fix.
Now, we can write z.
Instead of a subtraction, write it as addition of x plus negative y.
So in the expression of the convolution formula, we'll simply replace y by negative y, as it will show on the next slide.
Using the convolution formula, we can write f of z little z as the integration of f of x little x and f of negative y z minus x dx.
Now, we will use the fact that f of negative y, evaluated z minus x, is simply equal to f of y evaluated at x minus z.
To see why this is true, let's consider, let's say, a discreet random variable, y.
And now, the probability that negative y is equal to negative 1 is simply the same as probability that y is equal to 1.
And the same is true for probability density functions.
With this fact in mind, we can further write equality as the integration of x times f of y x minus z dx.
We're now ready to compute.
We'll first look at the case where z is less than 0.
On the right, I'm writing out the distribution of an exponential random variable with a parameter lambda.
In this case, using the integration above, we could write it as 0 to infinity, lambda e to the negative lambda x times lambda e to the negative lambda x minus z dx.
Now, the reason we chose a region to integrate from 0 to positive infinity is because anywhere else, as we can verify from the expression of fx right here, that the product of fx times fy here is 0.
Follow this through.
We'll pull out the constant.
Lambda e to the lambda z, the integral from 0 to infinity, lambda e to the negative 2 lambda x dx.
This will give us lambda e to the lambda z minus 1/2 e to the negative 2 lambda x infinity minus this expression value at 0.
And this will give us lamdba over 2 e to the lambda z.
So now, we have an expression for f of z evaluated at little z when little z is less than 0.
Now that have the distribution of f of z when z is less than 0, we'd like to know what happens when z is greater or equal to 0.
In principle, we can go through the same procedure of integration and calculate that value.
But it turns out, there's something much simpler.
z is the difference between x and y, at negative z, simply the difference between y and x.
Now, x and y are independent and identically distributed.
And therefore, x minus y has the same distribution as y minus x.
So that tells us z and negative z have the same distribution.
What that means is, is the distribution of z now must be symmetric around 0.
In other words, if we know that the shape of f of z below 0 is something like that, then the shape of it above 0 must be symmetric.
So here's the origin.
For example, if we were to evaluate f of z at 1, well, this will be equal to the value of f of z at negative 1.
So this will equal to f of z at negative 1.
Well, with this information in mind, we know that in general, f of z little z is equal to f of z negative little z.
So what this allows us to do is to get all the information for z less than 0 and generalize it to the case where z is greater or equal to 0.
In particular, by the symmetry here, we can write, for the case z greater or equal to 0, as lambda over 2 e to the negative lambda z.
So the negative sign comes from the fact that the distribution of f of z is symmetric around 0.
And simply, we can go back to the expression here to get the value.
And all in all, this implies that f of z little z is equal to lambda over 2 e to the negative lambda absolute value of z.
This completes our problem.
Hi, In this problem, we'll be looking at the PDF the absolute value of x.
So if we know a random variable, x, and we know it's PDF, how can we use that information to help us find the PDF of another random variable-- the absolute value of x?
And so throughout this problem, we'll define a new random variable called y.
And we'll define that y to be equal to the absolute value of x, just to make things simpler.
So we'll do a couple of concrete examples, and then we'll try to generalize at the end.
The first example that we'll deal with in part A is this PDF for x.
So we're told that the PDF of x is 1/3 between negative 2 and 1, and 0 otherwise.
And here's a picture of what it looks like.
It's just a rectangle from negative 2 to 1.
So now we want to find out what is the PDF of the absolute value of x, which we've called y?
And at this point, it may be helpful to step back and think about this problem from the discrete point of view again.
So if x were a discrete random variable, the problem would be, what is the probability that the absolute value of x is equal to, say, 1/2?
Well, the probability that the absolute value of x is equal to 1/2-- that can occur in two different ways.
One is that x itself is 1/2.
Or x could be negative 1/2, in which case, the absolute value of x would still be 1/2.
So those two events are mutually exclusive.
And so the probability of either one of them happening is you can just add them up.
And so the probability of the absolute value of x being 1/2 would have two contributions, one from x being 1/2, and one from x being negative 1/2.
The analogous idea carries over to the continuous case, when you have a PDF.
So now let's say that we're interested in the case where we want to know the PDF of y at 1/2.
Well, that again, is going to have two contributions, one from where x is 1/2, and one from where x is minus 1/2.
And so you can just imagine that each one of these values for y-- and remember, y has to be non-negative, because it's an absolute value-- has two contributions, one from the right side of 0, and one from the left, or negative, side of 0.
So you can come up and write an algebraic expression for this, and we'll do that in Part C. But you can also look at this from a visual point of view.
And you can take the PDF diagram itself and imagine transforming it to find out what the PDF of the absolute value of x would look like.
So the way to do it would be you take what's on the negative side.
You flip it over and take the mirror image, and then you stack it on top of what you have on the right-hand side, or the positive side.
So take this, flip it over, and stack it on top.
You can imagine just taking this block, flipping it over.
And just think of it as like a Tetris block that's falling down from above.
And it stacks on top of wherever it lands.
So it'll turn it into something that looks like this.
So there's already a block of height 1/3 from 0 to 1.
That's from the original 1.
And now we take this, and flip it over, and drop it on top.
Well, this part is going to fall on top of the segment from 0 to 1.
And then this part gets flipped over and dropped over here.
And it falls down here.
And so the final PDF actually looks like this kind of staircase, where this is 2/3 now, because this has two contributions of 1/3 each, and this is 1/3.
So that is the graphical way of approaching this.
And the PDF for completeness, the PDF of y would be 2/3 for y between 0 and 1, 1/3 for y from 1 to 2, and 0 otherwise.
All right, so let's move on to part B, and get some more practice.
Part B, we're given that this PDF of x now is 2 times e to the negative 2x for x positive, and 0 otherwise.
Now you may just recognize this as an exponential random variable with a parameter of 2.
And again, we can graph this and see what it looks like.
And it turns out that it's going to start out at 2 and fall off exponentially.
So in this case, this is actually quite simple.
Because if you look at it, x is already positive.
It doesn't have any negative parts.
So in fact, the absolute value of x is the same as x itself, because x is never negative.
And so y is just the same thing as x.
And so in this case, actually, the PDF of y is exactly the same as the PDF of x.
It's just 2e to the minus 2y, for y positive and zero otherwise.
Now you can also see this graphically also, because to the left of 0, the negative part, there is no PDF.
The PDF is 0.
And so if you were to take this, flip it over, and drop it on top, you wouldn't get anything, because there's nothing there.
And so the entire PDF, even after you take the absolute value, is just the original one.
So to generalize, what I said at the beginning was that, remember, the probability in the discrete case, if you wanted the probability that the absolute value of a random variable equals something, that would just be the probability that the random variable equals that value of little x, or the random variable equals negative little x.
In either of those two cases, the absolute value would equal x.
So you get those two contributions.
And so to generalize in the continuous case with PDFs, you get something that looks very similar.
So in this case, the PDF or y is just the PDF of x at y.
So this is the case where x is just equal to y, plus the PDF of x evaluated negative y.
So you, again, have both of these two contributions.
And we can rewrite this top one to make it look more similar.
So the PMF of some discrete [?
number ?]
y, where this is a discrete random variable that's equal to the absolute value of x, would be the PMF of x evaluated at y, plus the PMF of x evaluated at negative y.
So in both the discrete and continuous cases, you have the same thing.
So the overall summary of this problem is that, when you take a transformation-- in this case, an absolute value-- you can reason about it and figure out how to decompose that into arguments about the original random variable, just plain old x.
And for the specific case of the absolute value, it just becomes taking a mirror image and popping it on top of what you originally had.
So remember, you always have these two contributions.
And so if you ever have a random variable that you need to take an absolute value of, you don't have to be scared.
All you have to do is consider both of these contributions and add them up, and you have the PDF that you want.
So I'll see you next time.
Hi.
In this problem, we'll get a chance to see the usefulness of conditioning in helping us to calculate quantities that would otherwise be difficult to calculate.
Specifically, we'll be using the law of iterated expectations and the law of total variance.
Before we get started, let's just take a quick moment to interpret what these two laws are saying.
Really, what it's saying is, in order to calculate the expectation or the variance of some random variable x, if that's difficult to do, we'll instead attack this problem in stages.
So the first stage is, we'll condition on some related random variable, y.
And the hope is that by conditioning on this and reducing it to this conditional universe, the expectation of x will be easier to calculate.
Now, recall that this conditional expectation is really a random variable, which is a function of the random variable y.
So what we've done is we first average out x given some y.
What remains is some new random variable, which is a function of y.
And now, what we have is randomness in y, which will then average out again to get the final expectation of x. OK, so in this problem, we'll actually see an example of how this plays out.
One more thing before we get started that's useful to recall is if y is a uniform random variable, distributed between a and b, then the variance of y is b minus a squared over 12, and the expectation of y is just a midpoint, a plus b over 2.
All right, so let's get started on the problem.
So what we have is we have a stick of some fixed length, l, and what we do is we break it uniformly at random.
So what we do is we choose a point uniformly at random along this stick.
And we break it there, and then we keep the left portion of that stick.
So let's call the length of this left portion after the first break random variable y.
So it's random because the point where we break it is random.
And then what we do is we repeat this process.
We'll take this left side of the stick that's left.
And we'll pick another point, uniformly at random, along this left remaining side.
And we'll break it again, and keep the left side of that break.
And we'll call that the length of the final remaining piece, x, which again is random.
The problem is really asking us to calculate the expectation of variance of x.
So at first, it seems difficult to do, because the expectation and variance of x depends on where you break it the second time and also where you break it the first time.
So let's see if conditioning can help us here.
So the first thing that we'll notice is that, if we just consider y, the length of the stick after the first break, it's actually pretty easy to calculate the expectation and variance of y.
Because y, when you think about it, is actually just the simple uniform in a variable, uniformly distributed between 0 and l, the length of the stick.
And this is because we're told that we choose the point of the break uniformly at random between 0 and l. And so wherever we choose it, that's going to be the length of the left side of the stick.
And so because of this, we know that the expectation of y is just l/2, and the variance of y is l squared over 12.
But unfortunately, calculating the expectation variance of x is not quite as simple, because x isn't just uniformly distributed between 0 and some fixed number.
Because it's actually uniformly distributed between 0 and y, wherever the first break was.
But where the first break is is random too.
And so we can't just say that x is a uniformly distributed random variable.
So what do we do instead?
Well, we'll make the nice observation that let's pretend that we actually know what y is.
If we knew what y was, then calculating the expectation of x would be simple, right?
So if we were given that y is just some little y, then x would in fact just be uniformly distributed between 0 and little y.
And then if that's the case, then our calculation is simple, because the expectation of x would just be y/2, and the variance would just be y squared over 12.
All right, so let's make that a little bit more formal.
What we're saying is that the expectation of x, If we knew what y was, would just be y/2.
And the variance of x If we knew what y was would just be y squared over 12.
All right, so notice what we've done.
We've taken the second stage and we've said, let's pretend we know what happens in the first stage where we break it.
And we know what y, the first break, was.
Then the second stage becomes simple, because the average of x is just going to be the midpoint.
Now what we do to calculate the actual expectation of x, well, we'll invoke the law of iterated expectations.
So expectation of x is expectation of the conditional expectation of x given 1, which in this case is just expectation of y/2.
And we know what the expectation of y is.
It's l/2.
And so this is just l/4.
l/4.
All right, and so notice what we've done.
We've taken this calculation and done it in stages.
So we assume we know where the first break is.
Given that, the average location of the second break becomes simple.
It's just in the midpoint.
And then, we move up to the higher stage.
And that now we average out over where the first break could have been.
And that gives us our final answer.
And notice that this actually makes sense, if we just think about it intuitively, because on average, the first break will be somewhere in the middle.
And then that will leave us with half the stick left, and we break it again.
On average, that will leave us with another half.
So on average, you get a quarter of the original stick left, which makes sense.
All right, so that's the first part, where we use the law of iterated expectations.
Now, let's go to part B, where we're actually asked to find the variance.
The variance is given by the law of total variance.
So let's do it in stages.
We'll first calculate the first term, the expectation of the conditional variance.
Well, what is the expectation of the conditional variance?
We've already calculated out what this conditional variance is.
The conditional variance is y squared over 12.
So let's just plug that in.
It's expectation of y squared over 12.
All right, now this looks like it could be a little difficult to calculate.
But let's just first pull out the 1/12.
And then remember, one way to calculate the expectation of the square of a random variable is to use the variance.
So recall that the variance of any random variable is just expectation of the square minus the square of the expectation.
So if we want to calculate the expectation of the square, we can just take the variance and add the square of the expectation.
So this actually we can get pretty easily.
It's actually just the variance of y plus the square of the expectation of y.
And we know what these two terms are.
The variance of y is l squared over 12.
And the expectation of y is l/2.
So when you square that, you get l squared over 4.
So l squared over 12 plus l squared over 4 gives you l squared over 3.
And you get that the first term is l squared over 36.
All right, now let's calculate the second term.
Second term is the variance of the conditional expectation.
So the variance of expectation of x given y.
Well, what is the expectation of x given y?
We've already calculated that.
That's y/2.
So what we really want is the variance of y/2.
And remember, when you have a constant inside the variance, you pull it out but you square it.
So what you get is 1/4 the variance of y, which we know that the variance of y is l squared over 12.
So we get that this is l squared over 48.
OK, so we've calculated both terms of this conditional variance.
So all we need to do to find the final answer is just to add them.
So it's l squared over 36 plus l squared over 48.
And so, the final answer is 7 l squared over 144.
OK, and so this is the first, the expectation of x, maybe you could have guessed intuitively.
But the variance of x is not something that looks like something that you could have calculated off the top of your head.
And so I guess the lesson from this example is that it is often very helpful if you condition on some things, because it allows you to calculate things in stages and build up from the bottom.
But it's important to note that the choice of what you condition on-- so the choice of y-- is actually very important, because you could choose lots of other y's that wouldn't actually help you at all.
And so how to actually choose this y is something that you can learn based on just having practiced with these kinds of problems.
So again, the overall lesson is, conditioning can often help when you calculate these problems.
And so you should look to see if that could be a possible solution.
So I hope that was helpful, and see you next time.
In this problem, we will be helping Romeo and Juliet meet up for a date.
And in the process, also we'll review some concepts in basic probability theory, including sample spaces and probability laws.
This problem, the basic setup is that Romeo and Juliet are trying to meet up for a date.
And let's say they're trying to meet up for lunch tomorrow at noon.
But they're not necessarily punctual.
So they may arrive on time with a delay of 0, or they may actually be up to 1 hour late and arrive at 1:00 PM.
So the other thing that we assume in this problem is that all pairs of arrival times-- so the time that Romeo arrives paired with the time they Juliet arrives-- all of these pairs are equally likely.
And I've put this in quotes, because we haven't really specify exactly what this means.
And we'll come back to that in a little bit.
The last important thing is that each person will wait for 15 minutes for the other person to arrive.
If within that 15-minute window the other person doesn't arrive, then they'll give up and they'll end up not meeting up for lunch.
So to solve this problem, let's first try to set up a sample space and come up with a probability law to describe this scenario.
And let's actually start with a simpler version of this problem.
And instead of assuming that they can arrive at any delay between 0 and 1 hour, let's pretend instead that Romeo and Juliet can only arrive in 15-minute increments.
So Romeo can arrive on time with a delay 0, or be 15 minutes late, 30 minutes late, 45 minutes late, or one hour late.
But none of the other times are possible.
And the same thing for Juliet.
Let's start out with just the simple case first, because it helps us get the intuition for the problem, and it's an easier case to analyze.
So it's actually easy to visualize this.
It's a nice visual tool to group this sample space into a grid.
So the horizontal axis here represents the arrival time of Romeo, and the vertical axis represents the arrival time of Juliet.
And so, for example, this point here would represent Romeo arriving 15 minutes late and Juliet arriving 30 minutes late.
So this is our sample space now.
This is our omega.
And now let's try to assign a probability law.
And we'll continue to assume that all pairs of arrival times are equally likely.
And now we can actually specifically specify what this term means.
And in particular, we'll be invoking the discrete uniform law, which basically says that all of these points, which are just outcomes in our probabilistic experiment-- all of these outcomes are equally likely.
And so since there are 25 of them, each one of these outcomes has a probability of 1 over 25.
So now we've specified our sample space and our probability law.
So now let's try to answer the question, what is the probability that Romeo and Juliet will meet up for their date?
So all that amounts to now is just identifying which of these 25 outcomes results in Romeo and Juliet arriving within 15 minutes of each other.
So let's start with this one that I've picked out.
If Romeo arrives 15 minutes late and Juliet arrives 30 minutes late, then they will arrive within 15 minutes of each other.
So this outcome does result in the two of them meeting.
And so we can actually highlight all of these.
And it turns out that these outcomes that I'm highlighting result in the two them arriving within 15 minutes of each other.
So because each one has a probability of 1 over 25, all we really need to do now is just count how many outcomes there are.
So there's 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13.
So the probability in the end is for the discrete case.
The discrete case-- I'm referring to the case where we simplified it and considered only arrival times with increments of 15 minutes.
In this case, the probability is 13 over 25.
So now we have an idea of how to solve this problem.
It amounts to basically coming up with a sample space, a probability law, and then identifying the events of interest in calculating the probability of that event.
So now let's actually solve the problem that we really are interested in, which is that instead of confining Romeo and Juliet to arrive in only 15-minute minute increments, really, time is continuous, and Romeo and Juliet can arrive at any time.
So they don't necessarily have to arrive 15 minutes late.
Romeo could arrive 15 minutes and 37 seconds late if he wanted to.
So now our new sample space is actually just, instead of only these 25 points in the grid, it's this entire square.
So any point within the square could be a possible pair of meeting times between Romeo and Juliet.
So that is our new sample space, our new omega.
And now let's assign a new probability law.
And now, instead of being in the discrete world, we're in the continuous world.
And the analogy here is to consider probabilities as areas.
So the area of this entire square is one.
And that also corresponds to the probability of omega, the sample space.
And imagine just spreading probability evenly across this square so that the probability of any event-- which in this case would just be any shape within this square-- is exactly equal to the area of that shape.
So now that is our new sample space and our new probability law.
So what we have to do now is just to identify the event of interest, which is still the event that Romeo and Juliet arrive within 15 minutes of each other.
So let's do that.
If Romeo and Juliet arrive both on time, then obviously they'll meet.
And if Romeo's on time and Juliet is 15 minutes late, then they will still meet.
And in fact, any pairs of meeting times between these would still work, because now Romeo can be on time, and Juliet can arrive at any time between 0 and 15 minutes late.
But you notice that if Juliet is even a tiny bit later than 15 minutes, then they won't end up meeting.
So this segment here is part of the event of interest.
And similarly, this segment here is also part of the event.
And if you take this exercise and extend it, you can actually verify that the event of interest is this strip shape in the middle of the square.
Which, if you think about it, makes sense, because you want the arrival times between Romeo and Juliet to be close to each other, so you would be expect it to be somewhere close to a diagonal in this square.
So now we have our event of interest.
We have our sample space and our probability law.
So all we have to do now is just calculate what this probability is.
And we've already said that the probability in this probability law is just areas.
So now it actually just boils down to not a probability problem, but a problem in geometry.
So to calculate this area, you can do it in lots of ways.
One way is to calculate the area of the square, which is 1, and subtract the areas of these two triangles.
So let's do that.
So in the continuous case, the probability of meeting is going to be 1 minus the area of this triangle.
The base here is 3/4 and 3/4, so it's 1/2 times 3/4 times 3/4.
That's the area of one of these triangles.
There's two of them, so we'll multiply by two.
And we end up with 1 minus 9/16, or 7/16 as our final answer.
So in this problem, we've reviewed some basic concepts of probability, and that's also helped us solve this problem of helping Romeo and Juliet meet up for a date.
And if you wanted to, you could even extend this problem even further and turn it on its head.
And instead of calculating given that they arrive within 15 minutes of each other, what is the probability that they'll meet, let's say that Romeo really wants to meet up with Juliet, and he wants to assure himself a least, say, a 90% chance of meeting Juliet.
Then you can ask, if he wants to have at least a 90% chance of meeting her, how long should he be willing to wait?
And so that's the flip side of the problem.
And you can see that with just some basic concepts of probability, you can answer some already pretty interesting problems.
So I hope this problem was interesting, and we'll see you next time.
Hi.
In this problem, we're going to get a bunch of practice working with multiple random variables together.
And so we'll look at joint PDFs, marginal PDFs, conditional PDFs, and also get some practice calculating expectations as well.
So the problem gives us a pair of random variables-- x and y.
And we're told that the joint distribution is uniformly distributed on this triangle here, with the vertices being 0, 0 1, 0, and 0, 1.
So it's uniform in this triangle.
And the first part of the problem is just to figure out what exactly is disjoint PDF of the two random variables.
So in this case, it's pretty easy to calculate, because we have a uniform distribution.
And remember, when you have a uniform distribution, you can just imagine it being a sort of plateau coming out of the board.
And it's flat.
And so the height of the plateau, in order to calculate it, you just need to figure out what the area of this thing is, of this triangle is.
So remember, when you had single random variables, what we had to do was calculate, for uniform distribution, we had to integrate to 1.
So you took the length, and you took 1 over the length was the correct scaling factor.
Here, you take the area.
And the height has to make it so that the entire volume here integrates to 1.
So the joint PDF is just going to be 1 over whatever this area is.
And the area is pretty simple to calculate.
It's 1/2 base times height.
So it's 1/2.
And so what we have is that the area is 1/2.
And so the joint PDF of x and y is going to equal 2.
But remember, you always have to be careful when writing these things to remember the ranges when these things are valid.
So it's only 2 within this triangle.
And outside of the triangle, it's 0.
So what exactly does inside the triangle mean?
Well, we can write it more mathematically.
So this diagonal line, it's given by x plus y equals 1.
So everything in the triangle is really x plus y is less than or equal to 1.
It means everything under this triangle.
And so we need x plus y to be less then or equal to 1 and also x to be non-negative and y to be non-negative.
So with these inequalities, that captures everything within this triangle.
And otherwise, the joint PDF is going to be 0.
The next part asks us to find, using this joint PDF, the marginal of y.
And remember, when you have a joint PDF of two random variables, you essentially have everything that you need, because from this joint PDF, you can calculate marginals, you can calculate from the margins, you can calculate conditionals.
The joint PDF captures everything that there is to know about this pair of random variables.
Now, to calculate a marginal PDF of y, remember a marginal really just means collapsing the other random variable down.
And so you can just imagine taking this thing and collapsing it down onto the y-axis.
And mathematically, that is just saying that we integrate out the other random variable.
So the other random variable in this case will be x.
We take x and we get rid of it by integrating out from negative infinity to infinity.
Of course, this joint PDF is 0 in a lot of places.
And so a lot of these will be 0.
And only for a certain range of x's will this integral actually be non-zero.
And so again, the other time when we have to be careful is when we have these limits of integration, we need to make sure that we have the right limits.
And so we know that the joint PDF is 2.
It's nonzero only within this triangle.
And so it's only 2 within this triangle, which means what for x?
Well, depending on what x and y are, this will be either 2 or 0.
So let's just fix some value of y. Pretend that we've picked some value y, let's say here.
We want this value of y.
Well, what are the values of x such that the joint PDF for that value y is actually nonzero, it's actually 2?
Well, it's everything from x equals 0 to whatever x value this is.
But this x value, actually, if you think about it, is just 1 minus y, because this line is x plus y equals 1.
So whatever y is, x is going to be 1 minus that.
And so the correct limits would actually be from 0 to 1 minus y.
And then the rest of that is pretty simple.
You integrate this.
This is a pretty simple integral.
And you get that it's actually two times 1 minus y.
That's a y.
But of course, again, we need to make sure that we have the right regions.
So this is not always true for y, of course.
This is only true for y between 0 and 1.
And otherwise, it's actually 0, because when you take a y down here, well, there's no values of x that will give you a nonzero joint PDF.
And if you take a value of y higher than this, the same thing happens.
So we can actually draw this out and see what it looks like.
So let's actually draw a small picture here.
Here's y.
Here's the marginal PDF of y.
And here's 2.
And it actually looks like this.
It's a triangle and a 0 outside this range.
So does that make sense?
Well, first of all, you see that actually does in fact integrates to 1, which is good.
And the other thing we notice is that there is a higher density for smaller values of y.
So why is that?
Why are smaller values of y more likely than larger values of y?
Well, because when you have smaller values of y, you're down here.
And it's more likely because there are more values of x that go along with it that make that value of y more likely to appear.
Say you have a large value of y.
Then you're up here at the tip.
Well, there aren't very many combinations of x and y that give you that large a value of y.
And so that large value of y becomes less likely.
Another way to think about it is, when you collapse this down, there's a lot more stuff to collapse down its base.
There's a lot of x's to collapse down.
But up here, there's only a very little bit of x to collapse down.
And the PDF of y becomes more skewed towards smaller values of y.
So now, the next thing that we want to do is calculate the conditional PDF of x, given y.
Well, let's just recall what that means.
This is what we're looking for-- the conditional PDF of x, given y.
And remember, this is calculated by taking the joint and dividing by the marginal of y.
So we actually have the top and the bottom.
We have to joint PDF from part A.
And from part B, we calculated the marginal PDF of y.
So we have both pieces.
So let's actually plug them in.
Again, the thing that you have to be careful here is about the ranges of x and y where these things are valid, because this is only non-zero when x and y fall within this triangle.
And this is only non-zero when y is between 0 and 1.
So we need to be careful.
So the top, when it's non-zero, it's 2.
And the bottom, when it's non-zero, it's 2 times 1 minus y.
So we can simplify that to be 1 over 1 minus y.
And when is this true?
Well, it's true when x and y are in the triangle and y is between 0 and 1.
So put another way, that means that this is valid when y is between 0 and 1 and x is between 0 and 1 minus y, because whatever x has to be, it has to be such that they actually still fall within this triangle.
And outside of this, it's 0.
So let's see what this actually looks like.
So this is x, and this is the conditional PDF of x, given y.
Let's say this is 1 right here.
Then what it's saying is, let's say we're given that y is some little y.
Let's say it's somewhere here.
Then it's saying that the conditional PDF of x given y is this thing.
But notice that this value, 1 over 1 minus y, does not depend on x.
So in fact, it actually is uniform.
So it's uniform between 0 and 1 minus y.
And the height is something like 1 over 1 minus y.
And this is so that the scaling makes it so that actually is a valid PDF, because the integral is to 1.
So why is the case?
Why is that when you condition on y being some value, you get that the PDF of x is actually uniform?
Well, when you look over here, let's again just pretend that you're taking this value of y.
Well, when you're conditioning on y being this value, you're basically taking a slice of this joint PDF at this point.
But remember, the original joint PDF was uniform.
So when you take a slice of a uniform distribution, joint uniform distribution, you still get something that is uniform.
Just imagine that you have a cake that is flat.
Now, you take a slice at this level.
Then whatever slice you have is also going to be imagine being a flat rectangle.
So it's still going to be uniform.
And that's why the conditional PDF of x given y is also uniform.
Part D now asks us to find a conditional expectation of x.
So we want to find the expectation of x, given that y is some little y.
And for this, we can use the definition.
Remember, expectations are really just weighted sums.
Or in the [?
continuous ?]
case, it's an integral.
So you take the value.
And then you weight it by the density.
And in this case, because we're taking conditional a expectation, what we weight it by is the conditional density.
So it's the conditional density of x given that y is little y.
We integrate with respect to x.
And fortunately, we know what this conditional PDF is, because we calculated it earlier in part C. And we know that it's this-- 1 over 1 minus y.
But again, we have to be careful, because this formula, 1 over 1 minus y, is only valid certain cases.
So let's think about this first.
Let's think about some extreme cases.
What if y, little y, is negative?
If little y is negative, we're conditioning on something over here.
And so there is no density for y being negative or for y, say, in other cases when y is greater than 1.
And so in those cases, this expectation is just undefined, because conditioning on that doesn't really make sense, because there's no density for those values of y.
Now, let's consider the case that actually makes, sense where y is between 0 and 1.
Now, we're in business, because that is the range where this formula is valid.
So this formula is valid, and we can plug it in.
So it's 1 over 1 minus y dx.
And then the final thing that we again need to check is what the limits of this integration is.
So we're integrating with respect to x.
So we need to write down what values of x, what ranges of x is this conditional PDF valid.
Well, luckily, we specified that here.
x has to be between 0 and 1 minus y.
So let's actually calculate this integral.
This 1 over 1 minus y is a constant with respect to x.
You can just pull that out.
And then now, you're really just integrating x from 0 to 1 minus y.
So the integral of x is [?
1 ?
], 1/2x squared.
So you get a 1/2x squared, and you integrate that from 0 to 1 minus y.
And so when you plug in the limits, you'll get a 1 minus y squared.
That will cancel out the 1 over 1 minus y.
And what you're left with is just 1 minus y over 2.
And again, we have to specify that this is only true for y between 0 and 1.
Now, we can again actually verify that this makes sense.
What we're really looking for is the conditional expectation of x given some value of y.
And we already said that condition on y being some value of x is uniformly distributed between 0 and 1 minus y.
And so remember for our uniform distribution, the expectation is simple.
It's just the midpoint.
So the midpoint of 0 and 1 minus y is exactly 1 minus y/2.
So that's a nice way of verifying that this answer is actually correct.
Now, the second part of part D asks us to do a little bit more.
We have to use the total expectation theorem in order to somehow write the expectation of x in terms of the expectation of y.
So the first thing we'll do is use the total expectation theorem.
So the total expectation theorem is just saying, well, we can take these conditional expectations.
And now, we can integrate this by the marginal density of y, then we'll get the actual expectation of x.
You can think of it as just kind of applying the law of iterated expectations as well.
So this integral is going to look like this.
You take the conditional expectation.
So this is the expectation of x if y were equal to little y.
And now, what is that probability?
Well, now we just multiply that by the density of y at that actual value of little y.
And we integrate with respect to y.
Now, we've already calculated what this conditional expectation is.
It's 1 minus y/2.
So let's plug that in.
1 minus y/2 times the marginal of y.
There's a couple ways of attacking this problem now.
One way is, we can actually just plug in that marginal of y.
We've already calculated that out in part B.
And then we can do this integral and calculate out the expectation.
But maybe we don't really want to do so much calculus.
So let's do what the problem says and try a different approach.
So what the problem suggests is to write this in terms of the expectation of y.
And what is the expectation of y?
Well, the expectation of y is going to look something like the integral of y times the marginal of y.
So let's see if we can identify something like that and pull it out.
Well, yeah, we actually do have that.
We have y times the marginal of y, integrated.
So let's isolate that.
So besides that, we also have this.
We have the integral of the first term, is 1/2 times the marginal of y.
And then the second term is minus 1/2 times the integral of y of dy.
This is just me splitting this integral up into two separate integrals.
Now, we know what this is.
The 1/2 we can pull out.
And then the rest of it is just the integral of a marginal of a density from minus infinity to infinity.
And by definition, that has to be equal to 1.
So this just gives us a 1/2.
And now, what is this?
We get a minus 1/2.
And now this, we already said that is the expectation of y.
So what we have is the expectation of y.
So in the second part of this part D, we've expressed the expectation of x in terms of the expectation of y.
Now, maybe that seems like that's not too helpful, because we don't know what either of those two are.
But if we think about this problem, and as part E suggests, we can see that there's symmetry in this problem, because x and y are essentially symmetric.
So imagine this is x equals y.
There's symmetry in this problem, because if you were to swap the roles of x and y, you would have exactly the same joint PDF.
So what that suggests is that by symmetry then, it must be that the expectation of x and the expectation of y are exactly the same.
And that is using the symmetry argument.
And that helps us now, because we can plug that in and solve for expectation of x.
So expectation of x is 1/2 minus 1/2 expectation of x.
So we have 3/2 expectation of x equals 1/2.
So expectation of x equals 1/3.
And of course, expectation of y is also 1/3.
And so it turns out that the expectation is around there.
So this problem had several parts.
And it allowed us to start out from just a raw joint distribution, calculate marginals, calculate conditionals, and then from there, calculate all kinds of conditional expectations and expectations.
And a couple of important points to remember are, when you do these joint distributions, it's very important to consider where values are valid.
So you have to keep in mind when you write out these conditional PDFs and joint PDFs and marginal PDFs, what ranges the formulas you calculated are valid for.
And that also translates to when you're calculating expectations and such.
When you have integrals, you need to be very careful about the limits of your integration, to make sure that they line up with the range where the values are actually valid.
And the last thing, which is kind of unrelated, but it is actually a common tool that's used in a lot of problems is, when you see symmetry in these problems, that can help a lot, because it will simplify things and allow you to use facts like these to help you calculate what the final answer is.
Of course, this is also comes along with practice.
You may not immediately see that there could be a symmetry argument that will help with this problem.
But with practice, when you do more of these problems, you'll eventually build up that kind of--
Hi.
In this video, we're going to do some approximate calculations using the central limit theorem.
We're given that Xn is the number of gadgets produced on day n by a factory.
And it has a normal distribution with mean 5 and variance 9.
And they're all independent and identically distributed.
We're looking for the probability that the total number of gadgets in 100 days is less than 440.
To start, we can first write this as the probability of the sum of the gadgets produced on each of 100 days being less than 440.
Notice that this is a sum of a large number of independent random variables.
So we can use the central limit theorem and approximate the sum as a normal random variable.
And then, basically, in order to compute this probability, we'd basically need to standardize this and then use the standard normal table.
So let's first compute the expectation and variance of the sum.
So I'm going to actually sum up from 1 to n instead of 100, to do it more generally.
So the linearity is preserved for the expectation operator.
So this is the sum of the expected value.
And since they're all identically distributed, they all have the same expectation, and there are n of them.
And so we have this being n times 5.
For the variance of the sum is also the sum of the variances because the independents.
And so they're identically distributed to the -- so we have n times the variance of Xi, and this is n times 9.
So now, we can standardize it, or make it 0 mean and variance 1.
So to do that we would take these Xi's, subtract by their mean.
So it's going to be 5 times 100 of them, so it's 500 over the square root of the variance, which is going to be 9 times 100 of them, so it's going to be 900.
So that's going to be less than 440 minus 500 over square root of 900.
So notice what we're trying to do here is-- notice that the sum of Xi's is a discrete quantity.
So it's a discrete random variable, so it may have a PMF like this.
And we're trying to approximate it with a normal density.
So this is not drawn to scale, but let's say that this is 440 and this is 439.
Basically, we're trying to say what's the probability of this being less than 440, so it's the probability that it's 439, or 438, or 437.
But in the continuous case, a good approximation to this would be to take the middle, say, 439.5, and compute the area below that.
So in this case, when we do the normal approximation, it works out better if we use this half correction.
And so, this, in this case, probability, let's call Z the standard normal.
And so this is approximately equal to a standard normal with the probability of standard normal being less than whatever that is.
And if you plug that into your calculator, you get negative 2.02.
So now, if we try to figure out what this-- from the table, we'll find that negative values are not tabulated.
But we know that the normal, the center of normal is symmetric, and so if we want to compute the area in this region, it's the same as the area in this region, above 2.02.
So this is the same as the probability that Z is bigger than 2.02.
That's just 1 minus the probability that Z is less than or equal to 2.02, and so that's, by definition, phi of 2.02.
And if we look it up on the table, 2.02 has probability here of 0.9783.
And we can just write that in.
That's the answer for Part A.
So now for Part B.
We're asked what's the largest n, approximately, so that it satisfies this.
So again, we can use the central limit theorem.
Use similar steps here so that we have, in this case, n greater than or equal to 200 plus 5n.
And standardized.
So we have n and the mean here-- this is where this comes handy.
It's going to be 5n and the variance is 9n.
It's greater than or equal to.
5n's will cancel and you subtract.
And then you get 200 over the square root of 9n.
And we can, again, use the half approximation here, half correction here.
But I'm not going to do it, to keep the problem simple.
And so in this case, this is approximately equal to the standard normal being greater than probability of the center of normal being greater than or equal to 200 over square root of 9n.
And so same sort of thing here.
This is just 1 minus this.
The equal sign doesn't matter because Z is a continuous random variable.
And so we have this here.
And we want this to be less than or equal 0.05.
So that means that phi of 200 over square root of 9 has to be greater than or equal to 0.95.
So we're basically looking for something here that ensures that this region's at least 0.95.
So if you look at the table, 0.95 lies somewhere in between 1.64 and 1.65.
And I'm going to use 1.65 to be conservative, because we want this region to be at least 0.95.
So 1.65 works better here.
And so we want this thing, this here, which is going to be 200 over square root of n-- square root of 9n, to be bigger than or equal to 1.65.
So n here is going to be less than or equal to 200 over 1.65 squared, 1 over 9.
If you plug this into your calculator, you might have a decimal in there.
Then we just pick n, the largest integer that satisfies this.
So we can plug that into your calculator, you'll find that it's going to be 1,632.
That's part B.
Last part.
Let n be the first day when the total number of gadgets is greater than 1,000.
What's the probability that n is greater than or equal to 220?
Again, we want to use the central limit theorem, but the trick here is to recognize that this is actually equal to the probability that the sum from i equals 1 to 219 of Xi, is less than or equal to 1,000.
So let's look at both directions to check this.
If n is greater than or equal to 220, then this has to be true.
Because if it weren't true, and if this were greater than 1,000, then n would have been less than or equal to 219.
So this direction works.
The other direction.
If this were the case, it has to be the case that n is greater than or equal to 220, because up till 219 it hasn't exceeded 1,000.
And so, at some point beyond that, it's going to exceed 1,000 and n is going to be greater than or equal to 220.
So this is the key trick here.
And once you see this, you realize that this is very easy because we do the same steps as we did before.
So you're looking for this, this is equal to, again, you do your standardization.
So this is from 219, and you get 5 times 219 for the mean, and 9 times 219 for the variance, less than or equal to 1,000 minus 5 times 219 over square root of 9 times 219.
Again, you can do the half correction here, make it 1,000.5, but I'm not going to do that in this case, for simplicity.
So this is approximately equal to Z being less than whatever this is.
And if you plug it in, you'll find that this is negative 2.14.
So in this case, we have this is the probability that Z-- again, we do the same thing-- is greater than or equal to 2.14.
And this is 1 minus the probability that Z is less than or equal to 2.14.
And that's just phi of 2.14-- 1 minus Z of 2.14.
And that's-- if you look it up on the table, it's 2.14.
It's 0.9838.
So here's your answer.
So we're done with Part C as well.
So in this exercise, we did a lot of approximate calculations using the central--
Hi.
In this problem, we'll get more practice using conditioning to help us calculate expectations of variances.
We'll see that in this problem, which deals with widgets and crates, it's actually similar in flavor to an earlier problem that we did, involving breaking a stick twice.
And you'll see that in this problem, we'll again use the law of iterated expectations and the law of total variance to help us calculate expectations of variances.
And again, we'll be taking the approach of attacking the problem by splitting into the stages and building up from the bottom up.
So in this problem, what we have is a crate, which contains some number of boxes.
And we don't know how many boxes are.
It's random.
And it's given by some discrete random variable, n. And in each box, there are some number of widgets.
And again, this is also random.
And in each box, say for Box I, there are xi number of widgets in each one.
What we're really interested in in this problem is, how many widgets are there total in this crate?
So in the crate, there are boxes, and in the boxes, there are widgets.
How many widgets are there total within the crate?
And we'll call that a random variable, t. And the problem gives us some information.
It tells us that the expectation of the number of widgets in each box for all the boxes is the same.
It's 10.
And also, the expectation of the number of boxes is also 10.
And furthermore, the variance of x of the number of widgets and the number of boxes is all 16.
And lastly, an important fact is that all the xi's, so all the widgets for each box, and the total number of boxes, these random variables are all independent.
So to calculate t, t is just a sum of x1 through xn.
So x1 is the number of widgets in Box 1, z2 is the number of widgets in Box 2, and all the way through Box n. So what makes this difficult is that the n is actually random.
We don't actually know how many boxes there are.
So we don't even know how many terms there are in the sum.
Well, let's take a slightly simpler problem.
Let's pretend that we actually know there are exactly 12 boxes.
And in that case, the only thing that's random now is how many widgets there are in each box.
And so let's call [?
sum ?]
a new random variable, s, the sum of x1 through x12.
So this would tell us, this is the number of widgets in 12 boxes.
All right.
And because each of these xi's are independent, and they have the same expectation, just by linearity of expectations, we know that the expectation of s is just 12 copies of the same expectation of xi.
And similarly, because we also assume that all the xi's are independent, the variance of s, we can just add the variances of each of these terms.
So again, there are 12 copies of the variance of xi.
So we've done a simpler version of this problem, where we've assumed we know what n is, that n is 12.
And we've seen that in this simpler case, it's pretty simple to calculate what the expectation of the sum is.
So let's try to use that knowledge to help us calculate the actual problem, where n is actually random.
So what we'll do is use the law of iterated expectations.
And so this is written in terms of x and y, but we can very easily just substitute in for the random variables that we care about.
Where in this case, what we see is that in order to build things up, it would be helpful if we condition on something that is useful.
And in this case, it's fairly clear that it would be helpful if we condition on n, the number of boxes.
So if we knew how many boxes there were, then we can drop down to the level of widgets within each box.
And then once we have that, we can build up and average over the total number of boxes.
So what we should do is condition on n, the number of boxes.
So what have we discovered through this simpler exercise earlier?
Well, we've discovered that if we knew the number of boxes, then the expectation of the total number of widgets is just the number of boxes times the number of widgets in each one, or the expectation of the number of widgets in each one.
So we can use that information to help us here.
Because now, this is basically the same scenario, except that the number of boxes is now random.
Instead of being 12, it could be anything.
But if we just condition on the number of boxes being equal to n, then we know that there are exactly n copies of this.
But notice that n here is still random.
And so what we get is that the expectation is n times the expectation of the number of widgets in each box, which we know is 10.
So it's expectation of 10 times n or 10 times the expectation of n, which gives us 100.
Because there are, on expectation, 10 boxes.
So this, again, makes intuitive sense.
Because we know that on average, there are 10 boxes.
And on average, each box has 10 widgets inside.
And so on average, we expect that there will be 100 widgets.
And the key thing here is that we actually relied on this independence.
So if the number of widgets in each box vary depending on-- or if the distribution of the number of widgets in each box vary depending on how many boxes there were, then we wouldn't be able to do it this simply.
OK, so that gives us the answer to the first part, the expectation of the total number of widgets.
Now let's do the second part, which is the variance.
The variance, we'll again use this idea of conditioning and splitting things up, and use the law of total variance.
So the variance of t is going to be equal to the expectation of the conditional variance plus the variance of the conditional expectation.
So what we have to do now is just to calculate what all of these pieces are.
So let's start with this thing here, the conditional variance.
So what is the conditional variance?
Well, again, let's go back to our simpler case.
We know that if we knew what n is, then the variance would just be n times the variance of each xi.
So what does that tell us?
That tells us that, well, if we knew what n was, so condition on n, the variance would just be n times the variance of each xi.
So we've just taken this analogy and generalized it to the case where we don't actually know what n is.
We just condition on n, and we still have a random variable.
So then from that, we know that the expectation now, to get this first term, take the expectation of this conditional variance, it's just the expectation of n and the variance of xi, we're given that.
That's equal to 16.
So it's n times 16, which we know is 160, because the expectation of n, we also know, is 10.
All right, let's do this second term now.
We need the variance of the conditional expectation of t given n. Well, what is the conditional expectation of t given n?
We've already kind of used that here.
And again, it's using the fact that if we knew what n was, the expectation would just be n times the expectation of the number of widgets in each box.
So it would be n times the expectation of each xi.
Now, to get the second term, we just take the variance of this.
So the variance is the variance of n times the expectation of each xi.
And the expectation of each xi is 10.
So it's n times 10.
And now remember, when you calculate variances, [?
if you ?]
have a constant term inside, when you pull it out, you have to square it.
So you get 100 times the variance of n. And we know that the variance of n is also 16.
So this gives us 1600.
All right.
So now we've calculated both terms here.
The first term is equal to 160.
The second term is equal to 1600.
So to get the final answer, all we have to do is add this up.
So we get that the final answer is equal to 1760.
And this is not as obvious as the expectation, where you could have just kind of guessed that it was equal to 100.
So again, this was just another example of using conditioning and the laws of total variance and iterated expectations in order to help you solve a problem.
And in this case, you could kind of see that there is a hierarchy, where you start with widgets.
Widgets are contained in boxes, and then crates contain some number of boxes.
And so it's easy to just condition and do it level by level.
So you condition on the number of boxes.
If you know what the number of boxes are, then you can easily calculate how many widgets there are, on average.
And then you average over the number of boxes to get the final answer.
So I hope that was helpful.
And we'll see you next time.
And finally we look at the 6 And at that point we are done.
We're going to get started.
Handouts are the by the door if anybody didn't pick one up.
My name is Charles Leiserson.
I will be lecturing this course this term, Introduction to Algorithms, with Erik Demaine.
In addition, this is an SMA course, a Singapore MIT Alliance course which will be run in Singapore by David Hsu.
And so all the lectures will be videotaped and made available on the Web for the Singapore students, as well as for MIT students who choose to watch them on the Web.
If you have an issue of not wanting to be on the videotape, you should sit in the back row.
OK?
Otherwise, you will be on it.
There is a video recording policy, but it seems like they ran out.
If anybody wants to see it, people, if they could just sort of pass them around maybe a little bit, once you're done reading it, or you can come up.
I did secure one copy.
Before we get into the content of the course, let's briefly go over the course information because there are some administrative things that we sort of have to do.
As you can see, this term we have a big staff.
Take a look at the handout here.
Including this term six TAs, which is two more TAs than we normally get for this course.
That means recitations will be particularly small.
There is a World Wide Web page, and you should bookmark that and go there regularly because that is where everything will be distributed.
Email.
You should not be emailing directly to, even though we give you our email addresses, to the individual members of the staff.
You should email us generally.
And the reason is you will get much faster response.
And also, for any communications, generally we like to monitor what the communications are so it's helpful to have emails coming to everybody on the course staff.
As I mentioned, we will be doing distance learning this term.
And so you can watch lectures online if you choose to do that.
I would recommend, for people who have the opportunity to watch, to come live.
It's better live.
You get to interact.
There's an intangible that comes with having it live.
In fact, in addition to the videos, I meet weekly with the Singapore students so that they have a live session as well.
Prerequisites.
The prerequisites for this course are 6.042, which is Math for Computer Science, and 6.001.
You basically need discrete mathematics and probability, as well as programming experience to take this course successfully.
People do not have that background should not be in the class.
We will be checking prerequisites.
If you have any questions, please come to talk to us after class.
Let's see.
Lectures are here.
For SMA students, they have the videotapes and they will also have a weekly meeting.
Students must attend a one hour recitation session each week.
There will be new material presented in the recitation.
Unlike the lectures, they will not be online.
Unlike the lectures, there will not be lecture notes distributed for the recitations in general.
And, yet, there will be material there that is directly on the exams.
And so every term we say oh, when did you cover that?
That was in recitation.
You missed that one.
So, recitations are mandatory.
And, in particular, also let me just mention your recitation instructor is the one who assigns your final grade.
So we have a grade meeting and keep everybody normal, but your recitation has the final say on your grade.
Handouts.
Handouts are available on the course Web page.
We will not generally, except for this one, first handout, be bringing handouts to class.
Textbook is this book, Introduction to Algorithms.
MIT students can get it any of the local bookstores, including the MIT Coop.
There is also a new online service that provides textbooks.
You can also get a discount if you buy it at the MIT Press Bookstore.
There is a coupon in the MIT Student Telephone Directory for a discount on MIT Press books.
And you can use that to purchase this book at a discount.
Course website.
This is the course website.
It links to the Stellar website, which is where, actually, everything will be kept.
And SMA students have their own website.
Some students find this course particularly challenges so we will have extra help.
We will post weekly office hours on the course website for the TAs.
And then as an experiment this term, we are going to offer homework labs for this class.
What a homework lab is, is it's a place and a time you can go where other people in the course will go to do homework.
And there will be typically two TAs who staff the lab.
And so, as you're working on your homework, you can get help from the TAs if you need it.
And it's generally a place, we're going to schedule those, and they will be on the course calendar for where it is and when it is that they will be held, but usually Sundays 2:00 to 4:00 pm, or else it will be some evening.
I think the first one is an evening, right?
Near to when the homework is due.
Your best bet is try to do the homework in advance of the homework lab.
But then, if you want extra help, if you want to talk over your solutions with people because as we will talk about problem sets you can solve in collaboration with other people in the class.
In addition, there are several peer assistance programs.
Also the office of Minority Education has an assistance program, and those usually get booked up pretty quickly.
If you're interested in those, good idea to make an appointment to get there and get help soon.
The homework labs, I hope a lot of people will try that out.
We've never done this.
I don't know of any other course.
Do other people know of courses at MIT that have done this?
6.011 did it, OK. Good.
And was it successful in that class?
It never went, OK. Good.
[LAUGHTER] We will see.
If it's not paying off then we will just return to ordinary office hours for those TAs, but I think for some students that is a good opportunity.
If you wish to be registered in this course, you must sign up on the course Web page.
So, that is requirement one.
It must be done today.
You will find it difficult to pass the course if you are not in the class.
And you should notify your TA if you decide to drop so that we can get you off and stop the mailings, stop the spam.
And you should register today before 7:00 PM.
And then we're going to email your recitation assignment to you before Noon tomorrow.
And if you don't receive this information by Thursday Noon, please send us an email to the course staff generally, not to me individually, saying that you didn't receive your recitation assignment.
And so if you haven't received it by Thursday Noon you want to.
I think generally they are going to send them out tonight or at least by tomorrow morning.
Yeah.
OK. SMA students don't have to worry about this.
Problem sets.
We have nine problem sets that we project will be assigned during the semester.
A couple things about problem sets.
Homeworks won't generally be accepted, if you have extenuating circumstances you should make prior arrangements with your recitation instructor.
In fact, almost all of the administrative stuff, you shouldn't come to me to ask and say can I hand in something late?
You should be talking to your recitation instructor.
You can read the other things about the form, but let me just mention that there are exercises that should be solved but not handed in as well to give you drill on the material.
I highly recommend you doing the exercises.
They both test your understanding of the material, and exercises have this way of finding themselves on quizzes.
You're often asked to describe algorithms.
And here is a little outline of what you can use to describe an algorithm.
The grading policy is something that somehow I cover.
And always every term there are at least a couple of students who pretend like I never showed them this.
If you skip problems it has a nonlinear effect on your grade.
Nonlinear, OK?
If you don't skip any problems, no effect on your grade.
If you skip one problem, a hundredth of a letter grade, we can handle that.
But two problems it's a tenth.
And, as you see, by the time you have skipped like five letter grades, it is already five problems.
This is not problem sets, by the way.
This is problems, OK?
You're down a third of a letter grade.
And if you don't do nine or more, so that's typically about three to four problem sets, you don't pass the class.
I always have some students coming at the end of the year saying oh, I didn't do any of my problems.
Can you just pass me because I did OK on the exams?
Answer no, a very simple answer because we've said it upfront.
So, the problem sets are an integral part of the course.
Collaboration policy.
This is extremely important so everybody pay attention.
If you are asleep now wake up.
Like that's going to wake anybody up, right?
[LAUGHTER] The goal of homework.
Professor Demaine and my philosophy is that the goal of homework is to help you learn the material.
And one way of helping to learn is not to just be stuck and unable to solve something because then you're in no better shape when the exam roles around, which is where we're actually evaluating you.
So, you're encouraged to collaborate.
But there are some commonsense things about collaboration.
If you go and you collaborate to the extent that all you're doing is getting the information from somebody else, you're not learning the material and you're not going to do well on the exams.
In our experience, students who collaborate generally do better than students who work alone.
But you owe it to yourself, if you're going to work in a study group, to be prepared for your study group meeting.
And specifically you should spend a half an hour to 45 minutes on each problem before you go to group so you're up to speed and you've tried out your ideas.
And you may have solutions to some, you may be stuck on some other ones, but at least you applied yourself to it.
After 30 to 45 minutes, if you cannot get the problem, just sitting there and banging your head against it makes no sense.
It's not a productive use of your time.
And I know most of you have issues with having time on your hands, right?
Like it's not there.
So, don't go banging your head against problems that are too hard or where you don't understand what's going on or whatever.
That's when the study group can help out.
And, as I mentioned, we'll have homework labs which will give you an opportunity to go and do that and coordinate with other students rather than necessarily having to form your own group.
And the TAs will be there.
If your group is unable to solve the problem then talk to other groups or ask your recitation instruction.
And, that's how you go about solving them.
Writing up the problem sets, however, is your individual responsibility and should be done alone.
You don't write up your problem solutions with other students, you write them up on your own.
And you should on your problem sets, because this is an academic place, we understand that the source of academic information is very important, if you collaborated on solutions you should write a list of the collaborators.
Say I worked with so and so on this solution.
It does not affect your grade.
It's just a question of being scholarly.
It is a violation of this policy to submit a problem solution that you cannot orally explain to a member of the course staff.
You say oh, well, my write up is similar to that other person's.
I didn't copy them.
We may ask you to orally explain your solution.
If you are unable, according to this policy, the presumption is that you cheated.
So, do not write up stuff that you don't understand.
You should be able to write up the stuff that you understand.
Understand why you're putting down what you're putting down.
If it isn't obvious, no collaboration whatsoever is permitted on exams.
Exams is when we evaluate you.
And now we're not interested in evaluating other people, we're interested in evaluating you.
So, no collaboration on exams.
We will have a take home exam for the second quiz.
You should look at the schedule.
If there are problems with the schedule of that, we want to know early.
And we will give you more details about the collaboration in the lecture on Monday, November 28th.
Now, generally, the lectures here, they're mandatory and you have to know them, but I know that some people say gee, 9:30 is kind of early, especially on a Monday or whatever.
It can be kind of early to get up.
However, on Monday, November 28th, you fail the exam if you do not show up to lecture on time.
That one day you must show up.
Any questions about that?
That one day you must show up here, even if you've been watching them on the Web.
And generally, if you think you have transgressed, the best is to come to us to talk about it.
We can usually work something out.
It's when we find somebody has transgressed from a third party or from obvious analyses that we do with homeworks, that's when things get messy.
So, if you think, for some reason or other, oh, I may have done something wrong, please come and talk to us.
We actually were students once, too, albeit many years ago.
Any questions?
So, this class has great material.
Fabulous material.
And it's really fun, but you do have to work hard.
Let's talk content.
This is the topic of the first part of the course.
The first part of the course is focused on analysis.
The second part of the course is focused on design.
Before you can do design, you have to master a bunch of techniques for analyzing algorithms.
And then you'll be in a position to design algorithms that you can analyze and that which are efficient.
The analysis of algorithm is the theoretical study -- -- of computer program performance -- -- and resource usage.
And a particular focus on performance.
We're studying how to make things fast.
In particular, computer programs.
We also will discover and talk about other resources such as communication, such as memory, whether RAM memory or disk memory.
There are other resources that we may care about, but predominantly we focus on performance.
Because this is a course about performance, I like to put things in perspective a little bit by starting out and asking, in programming, what is more important than performance?
If you're in an engineering situation and writing code, writing software, what's more important than performance?
Correctness.
Good.
OK. What else?
Simplicity can be.
Very good.
Yeah.
Maintainability often much more important than performance.
Cost.
And what type of cost are you thinking?
No, I mean cost of what?
We're talking software here, right?
What type of cost do you have in mind?
There are some costs that are involved when programming like programmer time.
So, programmer time is another thing also that might be.
Stability.
Robustness of the software.
Does it break all the time?
What else?
Come on.
We've got a bunch of engineers here.
A lot of things.
How about features?
Features can be more important.
Having a wider collection of features than your competitors.
Functionality.
Modularity.
Is it designed in a way where you can make changes in a local part of the code and you don't have to make changes across the code in order to affect a simple change in the functionality?
There is one big one which definitely, especially in the `90s, was like the big thing in computers.
The big thing.
Well, security actually.
Good.
I don't even have that one down.
Security is excellent.
That's actually been more in the 2000.
Security has been far more important often than performance.
Scalability has been important, although scalability, in some sense, is performance related.
But, yes, scalability is good.
What was the big breakthrough and why do people use Macintosh rather than Windows, those people who are of that religion?
User-friendliness.
Wow.
If you look at the number of cycles of computers that went into user friendliness in the `90s, it grew from almost nothing to where it's now the vast part of the computation goes into user friendly.
So, all those things are more important than performance.
This is a course on performance.
Then you can say OK, well, why do we bother and why study algorithms and performance if it's at the bottom of the heap?
Almost always people would rather have these other things than performance.
You go off and you say to somebody, would I rather have performance or more user friendliness?
It's almost always more important than performance.
Why do we care then?
Yeah?
That wasn't user friendly.
Sometimes performance is correlated with user friendliness, absolutely.
Nothing is more frustrating than sitting there waiting, right?
So, that's a good reason.
What are some other reasons why?
Sometimes they have real time constraints so they don't actually work unless they perform adequately.
Yeah?
Hard to get, well, we don't usually quantify user friendliness so I'm not sure, but I understand what you're saying.
He said we don't get exponential performance improvements in user friendliness.
We often don't get that in performance either, by the way.
[LAUGHTER] Sometimes we do, but that's good.
There are several reasons that I think are important.
Once is that often performance measures the line between the feasible and the infeasible.
We have heard some of these things.
For example, when there are real time requirements, if it's not fast enough it's simply not functional.
Or, if it uses too much memory it's simply not going to work for you.
And, as a consequence, what you find is algorithms are on the cutting edge of entrepreneurship.
If you're talking about just re implementing stuff that people did ten years ago, performance isn't that important at some level.
But, if you're talking about doing stuff that nobody has done before, one of the reasons often that they haven't done it is because it's too time consuming.
Things don't scale and so forth.
So, that's one reason, is the feasible versus infeasible.
Another thing is that algorithms give you a language for talking about program behavior, and that turns out to be a language that has been pervasive through computer science, is that the theoretical language is what gets adopted by all the practitioners because it's a clean way of thinking about things.
A good way I think about performance, and the reason it's on the bottom of the heap, is sort of like performance is like money, it's like currency.
You say what good does a stack of hundred dollar bills do for you?
Would you rather have food or water or shelter or whatever?
And you're willing to pay those hundred dollar bills, if you have hundred dollar bills, for that commodity.
Even though water is far more important to your living.
Well, similarly, performance is what you use to pay for user friendliness.
It's what you pay for security.
And you hear people say, for example, that I want greater functionality, so people will program in Java, even though it's much slower than C, because they'll say it costs me maybe a factor of three or something in performance to program in Java.
But Java is worth it because it's got all these object oriented features and so forth, exception mechanisms and so on.
And so people are willing to pay a factor of three in performance.
So, that's why you want performance because you can use it to pay for these other things that you want.
And that's why, in some sense, it's on the bottom of the heap, because it's the universal thing that you quantify.
Do you want to spend a factor of two on this or spend a factor of three on security, et cetera?
And, in addition, the lessons generalize to other resource measures like communication, like memory and so forth.
And the last reason we study algorithm performance is it's tons of fun.
Speed is always fun, right?
Why do people drive fast cars, race horses, whatever?
Rockets, et cetera, why do we do that?
Because speed is fun.
Ski.
Who likes to ski?
I love to ski.
I like going fast on those skis.
It's fun.
Hockey, fast sports, right?
We all like the fast sports.
Not all of us, I mean.
Some people say he's not talking to me.
OK, let's move on.
That's sort of a little bit of a notion as to why we study this, is that it does, in some sense, form a common basis for all these other things we care about.
And so we want to understand how can we generate money for ourselves in computation?
We're going to start out with a very simple problem.
It's one of the oldest problems that has been studied in algorithms, is the problem of sorting.
We're going to actually study this for several lectures because sorting contains many algorithmic techniques.
The sorting problem is the following.
We have a sequence a 1, a 2 up to a n of numbers as input.
And our output is a permutation of those numbers.
A permutation is a rearrangement of the numbers.
Every number appears exactly once in the rearrangement such that, I sometimes use a dollar sign to mean "such that," a 1 is less than or equal to a 2 prime.
Such that they are monotonically increasing in size.
Take a bunch of numbers, put them in order.
Here's an algorithm to do it.
It's called insertion sort.
And we will write this algorithm in what we call pseudocode.
It's sort of a programming language, except it's got English in there often.
And it's just a shorthand for writing for being precise.
So this sorts A from 1 to n. And here is the code for it.
This is what we call pseudocode.
And if you don't understand the pseudocode then you should ask questions about any of the notations.
You will start to get used to it as we go on.
One thing is that in the pseudocode we use indentation, where in most languages they have some kind of begin end delimiters like curly braces or something in Java or C, for example.
We just use indentation.
The whole idea of the pseudocode is to try to get the algorithms as short as possible while still understanding what the individual steps are.
In practice, there actually have been languages that use indentation as a means of showing the nesting of things.
It's generally a bad idea, because if things go over one page to another, for example, you cannot tell what level of nesting it is.
Whereas, with explicit braces it's much easier to tell.
So, there are reasons why this is a bad notation if you were doing software engineering.
But it's a good one for us because it just keeps things short and makes fewer things to write down.
So, this is insertion sort.
Let's try to figure out a little bit what this does.
It basically takes an array A and at any point the thing to understand is, we're setting basically, we're running the outer loop from j is 2 to n, and the inner loop that starts at j minus 1 and then goes down until it's zero.
Basically, if we look at any point in the algorithm, we essentially are looking at some element here j.
A of j, the jth element.
And what we do essentially is we pull a value out here that we call the key.
And at this point the important thing to understand, and we'll talk more about this in recitation on Friday, is that there is an invariant that is being maintained by this loop each time through.
And the invariant is that this part of the array is sorted.
And the goal each time through the loop is to increase, is to add one to the length of the things that are sorted.
And the way we do that is we pull out the key and we just copy values up like this.
And keep copying up until we find the place where this key goes, and then we insert it in that place.
And that's why it's called insertion sort.
We just sort of move the things, copy the things up until we find where it goes, and then we put it into place.
And now we have it from A from one to j is sorted, and now we can work on j plus one.
Let's give an example of that.
Imagine we are doing 8, 2, 4, 9, 3, 6.
We start out with j equals 2.
And we figure out that we want to insert it there.
Now we have 2, 8, 4, 9, 3, 6.
Then we look at the four and say oh, well, that goes over here.
We get 2, 4, 8, 9, 3, 6 after the second iteration of the outer loop.
Then we look at 9 and discover immediately it just goes right there.
Very little work to do on that step.
So, we have exactly the same output after that iteration.
Then we look at the 3 and that's going to be inserted over there.
2, 3, 4, 8, 9, and that goes in there.
2, 3, 4, 6, 8, Question?
The array initially starts at one, yes.
A[1...n], OK?
So, this is the insertion sort algorithm.
And it's the first algorithm that we're going to analyze.
And we're going to pull out some tools that we have from our math background to help to analyze it.
First of all, let's take a look at the issue of running time.
The running time depends, of this algorithm depends on a lot of things.
One thing it depends on is the input itself.
For example, if the input is already sorted -- -- then insertion sort has very little work to do.
Because every time through it's going to be like this case.
It doesn't have to shuffle too many guys over because they're already in place.
Whereas, in some sense, what's the worst case for insertion sort?
If it is reverse sorted then it's going to have to do a lot of work because it's going to have to shuffle everything over on each step of the outer loop.
In addition to the actual input it depends, of course, on the input size.
Here, for example, we did six elements.
It's going to take longer if we, for example, do six times ten to the ninth elements.
If we were sorting a lot more stuff, it's going to take us a lot longer.
Typically, the way we handle that is we are going to parameterize things in the input size.
We are going to talk about time as a function of the size of things that we are sorting so we can look at what is the behavior of that.
And the last thing I want to say about running time is generally we want upper bonds on the running time.
We want to know that the time is no more than a certain amount.
And the reason is because that represents a guarantee to the user.
If I say it's not going to run, for example, if I tell you here's a program and it won't run more than three seconds, that gives you real information about how you could use it, for example, in a real time setting.
Whereas, if I said here's a program and it goes at least three seconds, you don't know if it's going to go for three years.
It doesn't give you that much guarantee if you are a user of it.
Generally we want upper bonds because it represents a guarantee to the user.
There are different kinds of analyses that people do.
The one we're mostly going to focus on is what's called worst case analysis.
And this is what we do usually where we define T of n to be the maximum time on any input of size n. So, it's the maximum input, the maximum it could possibly cost us on an input of size n. What that does is, if you look at the fact that sometimes the inputs are better and sometimes they're worse, we're looking at the worst case of those because that's the way we're going to be able to make a guarantee.
It always does something rather than just sometimes does something.
So, we're looking at the maximum.
Notice that if I didn't have maximum then T(n) in some sense is a relation, not a function, because the time on an input of size n depends on which input of size n. I could have many different times, but by putting the maximum at it, it turns that relation into a function because there's only one maximum time that it will take.
Sometimes we will talk about average case.
Sometimes we will do this.
Here T of n is then the expected time over all inputs of size n. It's the expected time.
Now, if I talk about expected time, what else do I need to say here?
What does that mean, expected time?
I'm sorry.
Raise your hand.
Expected inputs.
What does that mean, expected inputs?
I need more math.
What do I need by expected time here, math?
You have to take the time of every input and then average them, OK. That's kind of what we mean by expected time.
Good.
Not quite.
I mean, what you say is completely correct, except is not quite enough.
Yeah?
It's the time of every input times the probability that it will be that input.
It's a way of taking a weighted average, exactly right.
How do I know what the probability of every input is?
How do I know what the probability a particular input occurs is in a given situation?
I don't.
I have to make an assumption.
What's that assumption called?
What kind of assumption do I have to meet?
I need an assumption -- -- of the statistical distribution of inputs.
Otherwise, expected time doesn't mean anything because I don't know what the probability of something is.
In order to do probability, you need some assumptions and you've got to state those assumptions clearly.
One of the most common assumptions is that all inputs are equally likely.
That's called the uniform distribution.
Every input of size n is equally likely, that kind of thing.
But there are other ways that you could make that assumption, and they may not all be true.
This is much more complicated, as you can see.
Fortunately, all of you have a strong probability background.
And so we will not have any trouble addressing these probabilistic issues of dealing with expectations and such.
If you don't, time to go and say gee, maybe I should take that Probability class that is a prerequisite for this class.
The last one I am going to mention is best case analysis.
And this I claim is bogus.
Bogus.
No good.
Why is best-case analysis bogus?
Yeah?
The best case probably doesn't ever happen.
Actually, it's interesting because for the sorting problem, the most common things that get sorted are things that are already sorted interestingly, or at least almost sorted.
For example, one of the most common things that are sorted is check numbers by banks.
They tend to come in, in the same order that they are written.
They're sorting things that are almost always sorted.
I mean, it's good.
When upper bond, not lower bound?
Yeah, you want to make a guarantee.
And so why is this not a guarantee?
You're onto something there, but we need a little more precision here.
How can I cheat?
Yeah?
Yeah, you can cheat.
You cheat.
You take any slow algorithm that you want and just check for some particular input, and if it's that input, then you say immediately yeah, OK, here is the answer.
And then it's got a good best case.
But I didn't tell you anything about the vast majority of what is going on.
So, you can cheat with a slow algorithm that works fast on some input.
It doesn't really do much for you so we normally don't worry about that.
Let's see.
What is insertion sorts worst case time?
Now we get into some sort of funny issues.
First of all, it sort of depends on the computer you're running on.
Whose computer, right?
Is it a big supercomputer or is it your wristwatch?
They have different computational abilities.
And when we compare algorithms, we compare them typically for relative speed.
This is if you compared two algorithms on the same machine.
You could argue, well, it doesn't really matter what the machine is because I will just look at their relative speed.
But, of course, I may also be interested in absolute speed.
Is one algorithm actually better no matter what machine it's run on?
And so this kind of gets sort of confusing as to how I can talk about the worst case time of an algorithm of a piece of software when I am not talking about the hardware because, clearly, if I had run on a faster machine, my algorithms are going to go faster.
So, this is where you get the big idea of algorithms.
Which is why algorithm is such a huge field, why it spawns companies like Google, like Akamai, like Amazon.
Why algorithmic analysis, throughout the history of computing, has been such a huge success, is our ability to master and to be able to take what is apparently a really messy, complicated situation and reduce it to being able to do some mathematics.
And that idea is called asymptotic analysis.
And the basic idea of asymptotic analysis is to ignore machine-dependent constants -- -- and, instead of the actual running time, look at the growth of the running time.
So, we don't look at the actual running time.
We look at the growth.
Let's see what we mean by that.
This is a huge idea.
It's not a hard idea, otherwise I wouldn't be able to teach it in the first lecture, but it's a huge idea.
We are going to spend a couple of lectures understanding the implications of that and will basically be doing it throughout the term.
And if you go on to be practicing engineers, you will be doing it all the time.
In order to do that, we adopt some notations that are going to help us.
In particular, we will adopt asymptotic notation.
Most of you have seen some kind of asymptotic notation.
Maybe a few of you haven't, but mostly you should have seen a little bit.
The one we're going to be using in this class predominantly is theta notation.
And theta notation is pretty easy notation to master because all you do is, from a formula, just drop low order terms and ignore leading constants.
For example, if I have a formula like 3n^3 = 90n^2 - 5n + 6046, I say, well, what low order terms do I drop?
Well, n^3 is a bigger term n^2 than.
I am going to drop all these terms and ignore the leading constant, so I say that's Theta(n^3).
That's pretty easy.
So, that's theta notation.
Now, this is an engineering way of manipulating theta notation.
There is actually a mathematical definition for this, which we are going to talk about next time, which is a definition in terms of sets of functions.
And, you are going to be responsible, this is both a math and a computer science engineering class.
Throughout the course you are going to be responsible both for mathematical rigor as if it were a math course and engineering commonsense because it's an engineering course.
We are going to be doing both.
This is the engineering way of understanding what you do, so you're responsible for being able to do these manipulations.
You're also going to be responsible for understanding the mathematical definition of theta notion and of its related O notation and omega notation.
If I take a look as n approached infinity, a Theta(n^2) algorithm always beats, eventually, a Theta(n^3) algorithm.
As n gets bigger, it doesn't matter what these other terms were if I were describing the absolute precise behavior in terms of a formula.
If I had a Theta(n^2) algorithm, it would always be faster for sufficiently large n than a Theta(n^3) algorithm.
It wouldn't matter what those low order terms were.
It wouldn't matter what the leading constant was.
This one will always be faster.
Even if you ran the Theta(n^2) algorithm on a slow computer and the Theta(n^3) algorithm on a fast computer.
The great thing about asymptotic notation is it satisfies our issue of being able to compare both relative and absolute speed, because we are able to do this no matter what the computer platform.
On different platforms we may get different constants here, machine dependent constants for the actual running time, but if I look at the growth as the size of the input gets larger, the asymptotics generally won't change.
For example, I will just draw that as a picture.
If I have n on this axis and T(n) on this axis.
This may be, for example, a Theta(n^3) algorithm and this may be a Theta(n^2) algorithm.
There is always going to be some point n o where for everything larger the Theta(n^2) algorithm is going to be cheaper than the Theta(n^3) algorithm not matter how much advantage you give it at the beginning in terms of the speed of the computer you are running on.
Now, from an engineering point of view, there are some issues we have to deal with because sometimes it could be that that n o is so large that the computers aren't big enough to run the problem.
That's why we, nevertheless, are interested in some of the slower algorithms, because some of the slower algorithms, even though they may not asymptotically be slower, I mean asymptotically they will be slower.
They may still be faster on reasonable sizes of things.
And so we have to both balance our mathematical understanding with our engineering commonsense in order to do good programming.
So, just having done analysis of algorithms doesn't automatically make you a good programmer.
You also need to learn how to program and use these tools in practice to understand when they are relevant and when they are not relevant.
There is a saying.
If you want to be a good program, you just program ever day for two years, you will be an excellent programmer.
If you want to be a world class programmer, you can program every day for ten years, or you can program every day for two years and take an algorithms class.
Let's get back to what we were doing, which is analyzing insertion sort.
We are going to look at the worse case.
Which, as we mentioned before, is when the input is reverse sorted.
The biggest element comes first and the smallest last because now every time you do the insertion you've got to shuffle everything over.
You can write down the running time by looking at the nesting of loops.
What we do is we sum up.
What we assume is that every operation, every elemental operation is going to take some constant amount of time.
But we don't have to worry about what that constant is because we're going to be doing asymptotic analysis.
As I say, the beautify of the method is that it causes all these things that are real distinctions to sort of vanish.
We sort of look at them from 30,000 feet rather than from three millimeters or something.
Each of these operations is going to sort of be a basic operation.
One way to think about this, in terms of counting operations, is counting memory references.
How many times do you actually access some variable?
That's another way of sort of thinking about this model.
When we do that, well, we're going to go through this loop, j is going from 2 to n, and then we're going to add up the work that we do within the loop.
We can sort of write that in math as summation of j equals 2 to n. And then what is the work that is going on in this loop?
Well, the work that is going on in this loop varies, but in the worst case how many operations are going on here for each value of j?
For a given value of j, how much work goes on in this loop?
Can somebody tell me?
Asymptotically.
It's j times some constant, so it's theta j.
So, there is theta j work going on here because this loop starts out with i being j minus 1, and then it's doing just a constant amount of stuff for each step of the value of i, and i is running from j minus one down to zero.
So, we can say that is theta j work that is going on.
Do people follow that?
OK. And now we have a formula we can evaluate.
What is the evaluation?
If I want to simplify this formula, what is that equal to?
Sorry.
In the back there.
Yeah.
OK. That's just Theta(n^2), good.
Because when you're saying is the sum of consecutive numbers, you mean what?
What's the mathematic term we have for that so we can communicate?
You've got to know these things so you can communicate.
It's called what type of sequence?
It's actually a series, but that's OK. What type of series is this called?
Arithmetic series, good.
Wow, we've got some sharp people who can communicate.
This is an arithmetic series.
You're basically summing 1 + 2 + 3 + 4, some constants in there, but basically it's 1 + 2 + 3 + 4 + 5 + 6 up to n. That's Theta(n^2).
If you don't know this math, there is a chapter in the book, or you could have taken the prerequisite.
Arithmetic series.
People have this vague recollection.
Oh, yeah.
Good.
Now, you have to learn these manipulations.
We will talk about a bit next time, but you have to learn your theta manipulations for what works with theta.
And you have to be very careful because theta is a weak notation.
A strong notation is something like Leibniz notation from calculus where the chain rule is just canceling two things.
It's just fabulous that you can cancel in the chain rule.
And Leibniz notation just expresses that so directly you can manipulate.
Theta notation is not like that.
If you think it is like that you are in trouble.
You really have to think of what is going on under the theta notation.
And it is more of a descriptive notation than it is a manipulative notation.
There are manipulations you can do with it, but unless you understand what is really going on under the theta notation you will find yourself in trouble.
And next time we will talk a little bit more about theta notation.
Is insertion sort fast?
Well, it turns out for small n it is moderately fast.
But it is not at all for large n. So, I am going to give you an algorithm that is faster.
It's called merge sort.
I wonder if I should leave insertion sort up.
Why not.
I am going to write on this later, so if you are taking notes, leave some space on the left.
Here is merge sort of an array A from 1 up to n. And it is basically three steps.
If n equals 1 we are done.
Sorting one element, it is already sorted.
All right.
Recursive algorithm.
Otherwise, what we do is we recursively sort A from 1 up to the ceiling of n over 2.
And the array A of the ceiling of n over 2 plus one up to n. So, we sort two halves of the input.
And then, three, we take those two lists that we have done and we merge them.
And, to do that, we use a merge subroutine which I will show you.
The key subroutine here is merge, and it works like this.
I have two lists.
Let's say one of them is 20.
I am doing this in reverse order.
I have sorted this like this.
And then I sort another one.
I don't know why I do it this order, but anyway.
Here is my other list.
I have my two lists that I have sorted.
So, this is AA[1] to AA[|n/2|] and AA[|n/2|+1] to AA[n] for the way it will be called in this program.
And now to merge these two, what I want to do is produce a sorted list out of both of them.
What I do is first observe where is the smallest element of any two lists that are already sorted?
It's in one of two places, the head of the first list or the head of the second list.
I look at those two elements and say which one is smaller?
This one is smaller.
Then what I do is output into my output array the smaller of the two.
And I cross it off.
And now where is the next smallest element?
And the answer is it's going to be the head of one of these two lists.
Then I cross out this guy and put him here and circle this one.
Now I look at these two guys.
This one is smaller so I output that and circle that one.
Now I look at these two guys, output 9.
So, every step here is some fixed number of operations that is independent of the size of the arrays at each step.
Each individual step is just me looking at two elements and picking out the smallest and advancing some pointers into the array so that I know where the current head of that list is.
And so, therefore, the time is order n on n total elements.
The time to actually go through this and merge two lists is order n. We sometimes call this linear time because it's not quadratic or whatever.
It is proportional to n, proportional to the input size.
It's linear time.
I go through and just do this simple operation, just working up these lists, and in the end I have done essentially n operations, order n operations each of which cost constant time.
That's a total of order n time.
Everybody with me?
OK.
So, this is a recursive program.
We can actually now write what is called a recurrence for this program.
The way we do that is say let's let the time to sort n elements to be T(n).
Then how long does it take to do step one?
That's just constant time.
We just check to see if n is 1, and if it is we return.
That's independent of the size of anything that we are doing.
It just takes a certain number of machine instructions on whatever machine and we say it is constant time.
We call that theta one.
This is actually a little bit of an abuse if you get into it.
And the reason is because typically in order to say it you need to say what it is growing with.
Nevertheless, we use this as an abuse of the notation just to mean it is a constant.
So, that's an abuse just so people know.
But it simplifies things if I can just write theta one.
And it basically means the same thing.
Now we recursively sort these two things.
How can I describe that?
The time to do this, I can describe recursively as T of ceiling of n over 2 plus T of n minus ceiling of n over 2.
That is actually kind of messy, so what we will do is just be sloppy and write 2T(n/2).
So, this is just us being sloppy.
And we will see on Friday in recitation that it is OK to be sloppy.
That's the great thing about algorithms.
As long as you are rigorous and precise, you can be as sloppy as you want.
[LAUGHTER] This is sloppy because I didn't worry about what was going on, because it turns out it doesn't make any difference.
And we are going to actually see that that is the case.
And, finally, I have to merge the two sorted lists which have a total of n elements.
And we just analyze that using the merge subroutine.
And that takes us to theta n time.
That allows us now to write a recurrence for the performance of merge sort.
Which is to say that T of n is equal to theta 1 if n equals 1 and 2T of n over 2 plus theta of n if n is bigger than 1.
Because either I am doing step one or I am doing all steps one, two and three.
Here I am doing step one and I return and I am done.
Or else I am doing step one, I don't return, and then I also do steps two and three.
So, I add those together.
I could say theta n plus theta 1, but theta n plus theta 1 is just theta n because theta 1 is a lower order term than theta n and I can throw it away.
It is either theta 1 or it is 2T of n over 2 plus theta n. Now, typically we won't be writing this.
Usually we omit this.
If it makes no difference to the solution of the recurrence, we will usually omit constant base cases.
In algorithms, it's not true generally in mathematics, but in algorithms if you are running something on a constant size input it takes constant time always.
So, we don't worry about what this value is.
And it turns out it has no real impact on the asymptotic solution of the recurrence.
How do we solve a recurrence like this?
I now have T of n expressed in terms of T of n over 2.
That's in the book and it is also in Lecture 2.
We are going to do Lecture 2 to solve that, but in the meantime what I am going to do is give you a visual way of understanding what this costs, which is one of the techniques we will elaborate on next time.
It is called a recursion tree technique.
And I will use it for the actual recurrence that is almost the same 2T(n/2), but I am going to actually explicitly, because I want you to see where it occurs, plus some constant times n where c is a constant greater than zero.
So, we are going to look at this recurrence with a base case of order one.
I am just making the constant in here, the upper bound on the constant be explicit rather than implicit.
And the way you do a recursion tree is the following.
You start out by writing down the left hand side of the recurrence.
And then what you do is you say well, that is equal to, and now let's write it as a tree.
I do c of n work plus now I am going to have to do work on each of my two children.
T of n over 2 and T of n over If I sum up what is in here, I get this because that is what the recurrence says, T(n)=2T(n/2)+cn.
I have 2T(n/2)+cn.
Then I do it again.
I have cn here.
I now have here cn/2.
And here is cn/2.
And each of these now has a T(n/4).
And these each have a T(n/4).
And this has a T(n/4).
And I keep doing that, the dangerous dot, dot, dots.
And, if I keep doing that, I end up with it looking like this.
And I keep going down until I get to a leaf.
And a leaf, I have essentially a T(1).
That is T(1).
And so the first question I ask here is, what is the height of this tree?
Yeah.
It's log n. It's actually very close to exactly log n because I am starting out at the top with n and then I go to n/2 and n/4 and all the way down until I get to The number of halvings of n until I get to 1 is log n so the height here is log n. It's OK if it is constant times log n. It doesn't matter.
How many leaves are in this tree, by the way?
How many leaves does this tree have?
Yeah.
The number of leaves, once again, is actually pretty close.
It's actually n. If you took it all the way down.
Let's make some simplifying assumption.
n is a perfect power of 2, so it is an integer power of 2.
Then this is exactly log n to get down to T(1).
And then there are exactly n leaves, because the number of leaves here, the number of nodes at this level is 1, 2, 4, 8.
And if I go down height h, I have 2 to the h leaves, 2 to the log n, that is just n. We are doing math here, right?
Now let's figure out how much work, if I look at adding up everything in this tree I am going to get T(n), so let's add that up.
Well, let's add it up level by level.
How much do we have in the first level?
Just cn.
If I add up the second level, how much do I have?
cn.
How about if I add up the third level?
cn.
How about if I add up all the leaves?
Theta n. It is not necessarily cn because the boundary case may have a different constant.
It is actually theta n, but cn all the way here.
If I add up the total amount, that is equal to cn times log n, because that's the height, that is how many cn's I have here, plus theta n. And this is a higher order term than this, so this goes away, get rid of the constants, that is equal to theta(n lg n).
And theta(n lg n) is asymptotically faster than theta(n^2).
So, merge sort, on a large enough input size, is going to beat insertion sort.
Merge sort is going to be a faster algorithm.
Sorry, you guys, I didn't realize you couldn't see over there.
You should speak up if you cannot see.
So, this is a faster algorithm because theta(n lg n) grows more slowly than theta(n^2).
And merge sort asymptotically beats insertion sort.
Even if you ran insertion sort on a supercomputer, somebody running on a PC with merge sort for sufficient large input will clobber them because actually n^2 is way bigger than n log n once you get the n's to be large.
And, in practice, merge sort tends to win here for n bigger than, say, 30 or so.
If you have a very small input like 30 elements, insertion sort is a perfectly decent sort to use.
But merge sort is going to be a lot faster even for something that is only a few dozen elements.
It is going to actually be a faster algorithm.
That's sort of the lessons, OK?
Remember that to get your recitation assignments and attend recitation on Friday.
Because we are going to be going through a bunch of the things that I have left on the table here.
And see you next Monday.
OK. Today we are going to talk about a very interesting algorithm called Quicksort -- -- which was invented by Tony Hoare in 1962.
And it has ended up being a really interesting algorithm from many points of view.
And because of that, it turns out today's lecture is going to be both hard and fast.
If you see the person next to you sleeping, you will want to say let's get going.
It's a divide-and-conquer algorithm.
And it sorts, as they say, in place, meaning that it just rearranged the elements where they are.
That is like insertion sort rearranges elements where they are.
Mergesort does not.
Mergesort requires extra storage in order to do the merge operation.
To merge in linear time and place, it doesn't merge in place in linear time.
It doesn't do it just by rearranging.
It is nice because it is in place, so that means that it is fairly efficient in its use of storage.
And it also happens to be very practical if you tune it a bit.
The basic algorithm turns out, if you just implement that, it's not necessarily that efficient.
But if what you do was then do the standard kinds of things you do to goose up the runtime of something, and we'll talk a little about what those things are, then it can be very, very practical.
So, it uses divide-and-conquer paradigm.
First step is divide.
And to do this basically it does it by partitioning.
So, it partitions the input array into two subarrays around an element we call the pivot -- -- such that elements in the lower subarray are less than or equal to x, are less than or equal to elements in the upper subarray.
If we draw a picture of the input array, this partition step basically takes some element x and everything over here is less than or equal to x after the partition step and everything over here is greater than or equal to x.
And so now the conquer step is pretty easy.
You just recursively sort the two subarrays.
So, I recursively sort the elements less than or equal to x, I recursively sort the elements greater than or equal to x.
And then combine is then just trivial.
Because once I have sorted the things less than or equal to x, then sorted the things greater than or equal to x, the whole thing is sorted.
So, there is nothing to do really for the combine.
The key step in quicksort is this partition step.
That is the thing that does all of the work.
And so you can view quicksort of just as recursive partitioning.
That's all it is.
Just as mergesort was recursive merging, quicksort sort of goes the other way around and does recursive partitioning.
The key is the linear time, by which I mean theta n, partitioning subroutine.
And here are some pseudocode for it.
This is actually slightly different from the book.
The book has one.
In fact, there is a nice problem in the book that has even a different one, but they are all basically the same idea.
Partition (A, p, q).
And what we are looking at, at this step of the recursion, is the subarray A from p to q.
And basically we pick a pivot, which is we are going to just pick as the first element of the array A of p. And the book, just for your information, uses A of q. I use A of p. It doesn't really matter.
And then we set an index to p and then we have a loop.
This is the code.
Basically the structure of it is a for loop with an "if" statement in the middle.
And so the structure of the algorithm of this partitioning step looks as follows.
We set the pivot to be the first element.
Here is p and here is q.
This is going to be our invariant for the loop.
And, at any time during the execution of a loop, I essentially have some values up to i which are already less than or equal to x and then some values that end at j minus 1 that are greater than or equal to x.
And then I don't know about the rest.
And so we start out with i equal to p and j equal to p plus It starts out at p plus 1 so that everything is unknown except for x here.
And then the idea is that it is going to preserve this invariant.
And the way it does it is, as we go through the loop, it looks at a of j and says is it greater than or equal to x?
Sorry, is it less than or equal to x?
If it is greater than or equal to x it does nothing, because what can happen?
If this is greater than or equal to x, essentially it just goes to the next iterational loop which moves this boundary and the invariant is satisfied.
Does everybody see that?
Yeah, OK.
But if it is less than or equal to x, I have got a problem if I want to maintain the invariant if this next element is less than or equal to x.
And so what it does then is it says oh, let me just move this boundary and swap this element here, which is greater than or equal to x, with this one here that is less than or equal to x, thereby increasing the size of this subarray and then the invariant is satisfied again.
It is a fairly simple algorithm.
And it is actually a very tight and easy algorithm.
That is one reason that this is such a great piece of code because it is very efficient.
Now, in principle, the running time for this on n elements is order n. Because I am basically just going through the n elements and just doing a constant amount of work and then just a constant amount of work outside.
This is a clever piece of code.
In fact, in principle partition is easy, right?
If I weren't worrying about doing it in place, it is really a pretty easy thing to do.
I take an element and just compare every other element with it.
I throw one into one bin and one into the other.
That is clearly linear time.
But often what you find is that just because you can do it that way theoretically doesn't mean that that is going to end up giving you good code.
And this is a nice piece of code that allows you to do it in place.
And that is one reason why this is a particularly good algorithm, because the constants are good.
So, yes, when we do asymptotic analysis we tend to ignore the constants, but when you're actually building code you care about the constants.
But first you care much more than just about the constants, is whether overall it is going to be a fast algorithm.
Let's go through an example of this, I guess I will do it over here, just so we get the gist.
Here is a sample array that I have created out of hallcloth.
And here we are going to set x, the pivot, to be 6.
Let's look to see how this algorithm works.
So, i starts out here and j starts out here if we initialize.
And what we do is start scanning right, essentially that code is scanning right until it gets something which is less than or equal to the pivot.
It keeps going here until it finds, j keeps incrementing until it finds something that is less than or equal to the pivot.
And, in that case, it is the number 5.
Then it says we will swap these two things.
And it does that and we get 6, 5, 13, 10, 8, 3, 2, 11.
And meanwhile now i gets incremented and j continues where it left off.
And so now we keep scanning right until we get to something that is less than or equal to the pivot.
In this case it is 3.
We swap 3 and 5 and get 6, 3, etc.
And now, at this step we increment i, we start j out here.
And in this case, right off the bat, we have something which is less than or equal to x, so we swap these two.
I blew it, didn't I?
Oops.
What did I do?
I swapped the wrong thing, didn't I, here?
Ah-ha.
That is why I am not a computer.
Good.
We should have swapped this guy, right?
Swapped i plus 1, right?
This was i.
We swap i plus 1, good.
So, that's all wrong.
Let's swap the right things.
Now we have 6, 5, 3, 10, 8, 13, 2, 11.
That even corresponds to my notes for some strange reason.
This is i and now this is j.
And now when I look, I immediately have something that is less than or equal to the pivot.
We swap this and i plus 1, so now we have 6, 5, 3, 2, 8, 13, 10, 11.
And we, at that point, increment i to here.
And we have j now going here and j runs to the end.
And the loop terminates.
When the loop terminates there is one less swap that we do, which is to put our pivot element in the middle between the two subarrays.
Here we swap this one and this one, and so that gives us then 2, 5, 3, 6, 8, 13, 10, 11.
And this is the pivot.
And everything over here is less than or equal to the pivot.
And everything over here is greater than or equal to the pivot.
OK, so the quicksort routine.
Once we have this partition routine, quicksort is a pretty easy piece of code to write.
I should have said return here i, right?
You have got to return with the pivot.
Here I have got to return i because we want to know where the pivot element is.
Sorry.
I will plug in my code.
r gets partition of (A, p, q) and then we quicksort (A, p, r-1) and quicksort of (A, r+1, q).
And that is it.
That's the code.
The initial call is quicksort of (A, 1, n).
Because once we partitioned, we just have to quicksort the two portions, the left and right portions.
Just the boundary case is probably worth mentioning for a second.
If there are zero or one elements, that is basically what can possibly happen here, is that I get zero or one elements here.
Then the point is there is nothing to do because the array is sorted, either because it is an empty array or because it only has one element.
One of the tricks to making quicksort go fast, as one tunes this, is to, in fact, look at having a special purpose sorting routine for small numbers of elements.
For example, if you get down to five elements having some straight line piece of code that knows how to sort five elements sufficiently as opposed to continuing to go through recursion in order to accomplish that.
And there are a variety of other things.
This is a tail recursive code, and so you can use certain tail recursion optimizations.
And there are a variety of other kinds of optimizations that you can use to make this code go fast.
So, yeah, you can tune it up a bit beyond what is there, but the core of it is this efficient partitioning routine.
That is the algorithm.
It turns out that looking at how fast it runs is actually a little bit challenging.
In the analysis, we are going to assume that all elements are distinct.
It turns out that this particular code does not work very well when you have repeated elements, but Hoare's original partitioning routine is actually more efficient in that case if there are duplicates in what you are sorting.
And I encourage you to look at that.
It has a much more complicated invariant for partitioning routine, but it does a similar kind of thing.
It's just a bit more complicated.
If they weren't all distinct, there are things you can do to make them distinct or you can just use this code.
The easiest thing to do is just use Hoare's original code because that works pretty well when they are nondistinct.
But this is a little bit easier to understand.
Let's let T(n) be the worst-case running time on n elements.
And so what is the worse-case?
What is the worse-case going to be for quicksort?
That's right.
If you always pick the pivot and everything is greater than or everything is less than, you are not going to partition the array very well.
And when does that happen?
What does the original input look like that makes that happen?
If it is already sorted or reverse sorted.
So, if the input is sorted or reverse sorted.
That is actually kind of important to understand, because it turns out the most common thing to sort is something that is already sorted, surprisingly, or things that are nearly sorted.
But often it is just sorted and somebody wants to make sure it is sorted.
Well, let's just sort it again rather than checking to see if it is sorted.
And, in those cases, one side of the partition of each partition has no elements.
Then we can write out what the recursion is for that.
We have T(n).
If one side has no elements, we are going to have T(0) on that side.
And on the other side we are going to have T(n-1).
We are just writing out the recursion for this.
One side has no elements.
The other side has n-1 elements.
And then partitioning and all the bookkeeping and so forth is order n. What is T(0)?
What is T(0)?
What is that asymptotically?
It's a constant, order 1.
That is just order 1 + T(n-1) + order n. Well, the order 1 can be absorbed into the order n, so this is really just saying it is T(n-1) + order n. And what is that equal to?
That is order n^2.
Why is that order n^2?
It is an arithmetic series.
Actually, just like we got for insertion sort.
Just like for insertion sort it is an arithmetic series.
Going through all that work and we have an algorithm called quicksort, and it is no faster than insertion sort.
Nevertheless, I said it was a good algorithm.
The reason it is a good algorithm is because its average case time, as we are going to see, is very good.
But let's try to understand this a little bit more just so that we understand the difference between what is going to happen in the average case and what is going to happen in the worse-case.
Let's draw a recursion tree for this for T(n) = T(0) + T(n-1) + and I will make the constant explicit for cn.
So, we get an intuition of what is going on.
Some constant times n. And then we have T(n) is equal to, and we write it with the constant part here, cn, and then T(0) here, and then T(n-1) here.
Now, I know that all you folks are really fast and want to jump immediately to the full-blown tree.
But, let me tell you, my advice is that you spend just a couple of minutes writing it out.
Since the tree grows exponentially, it only costs you a constant overhead to write out the small cases and make sure that you have got the pattern that you are developing.
So, I am going to go one more step.
Here we have T(0) and now this becomes c(n-1) and now we have another T(0) over here and T(n-2).
And we continue that, dot, dot, dot.
That is all equal to cn with a T(0) here, c(n-1) with a T(0), c(n-2), T(0) here, and that goes all the way down until we end up with T(1) down here.
What is the height of this tree?
What is the height of the tree here?
Yeah, n. Good.
Because every step we are just decrementing the argument by 1.
So, the height is n. To analyze this, let's first add up everything that is here.
Just so we understand where these things are coming from, this is just theta of the summation of k equals 1 to n of k, actually of ck.
That is what is in there.
And that is equal to order n^2.
That is where our algorithmatic series is coming from.
So, that is Theta(n^2).
And then all of these things here are all Theta(1).
And how many of them are there?
There are n Theta(1)'s.
So, the total amount is T(n) = Theta(n) + Theta(n^2) = Theta(n^2).
Just to see what the structure is in terms of the recursion tree, it is a highly unbalanced recursion tree.
Now I am going to do something that I told you should never do, which is we are going to be do a best-case analysis.
This is for intuition only.
And, in general, we don't do best-case analyses.
It doesn't mean anything, unless we get some intuition for it maybe.
But basically it means nothing mathematically because it's providing no guarantee.
And so this is intuition only.
If we are really lucky what happens for partition?
What is going to be the lucky case?
Yeah, it splits right in the middle.
Which is essentially -- -- n/2 : n/2.
It is really (n-1)/2 : (n-1)/2, but we're not going to worry about the details because we're only doing intuition for the best-case because best-case is not what we want.
If that happened, what is the recurrence I get?
Imagine it split it exactly in the middle every time, then what happens?
You get T(n) = 2T(n/2) + order n for partitioning and bookkeeping.
And what is the solution of that recurrence?
That is n log n. That is the same as the merge sort recurrence.
It is which case of the master theorem?
Case 2, right?
Because n to the log base 2 of 2 is n to the 1, it is the same, so we tack on the extra log n. Case 2 of the master theorem.
That is pretty good.
That says that in the best-case quicksort is going to do well.
How about let's suppose the split is always let's say 1/10 : 9/10, 1/10n : 9/10n.
In that case, are we lucky or are we unlucky?
I mean, if the split is really skewed, we clearly are going to be unlucky, right, because then it's, say, 1 to n. If it is really in the middle it is n log n. What do you suppose it is if it is 1/10 : 9/10?
Is that lucky or unlucky?
We will have a little democracy here.
Who thinks that that is a lucky case?
It is going to be fast running time.
And who thinks it is an unlucky case?
OK, so we have some brave souls.
And who didn't vote?
Oh, come on.
Come on.
It is always better, by the way, to say yes or no and be right or wrong, because then you have some emotional commitment to it and we will remember better, rather than just sitting and being quiet.
You don't manipulate your own emotions well enough to remember things well.
Those people who voted win over the people who don't vote, whether they are right or wrong.
Well, let's take a look.
Here is the recurrence.
T(n) = T(1/10n) + T(9/10n) + Theta(n).
And we will assume that this part here is less than or equal to some cn in order to analyze it.
We will just do a recursion tree for this and see.
Here is a recursion tree.
We have T(n) = cn, T(1/10n), T(9/10n).
Now we have again cn at the top.
This gets complicated, right?
This is 1/10cn.
Now, over here we have 1/10.
And then we are plugging it into the recursion again, so we now get T(1/100n) and over here we get T(9/100n).
And over here we have now 9/10cn.
And that gives us T(9/100n) again.
And here we get T(81/100n).
And we keep going on.
That is equal to cn, 1/10cn here.
Down this way we have 1/100cn.
And that keeps going down until we get to order 1 down here.
And over here we have 9/10cn.
And here, let's see, this is 9/100cn and this is now 9/100cn and this is 81/100cn.
And these things keep going down until they get down to order 1.
But the leaves are not all at uniform depth here, right?
This side is way further up than this side, right?
Because here we are only going down by 9/10 each time.
So, in fact, what is the length of this path here?
What is the length of this path down to this, if I take the left most spine?
Somebody raise there hand.
Yeah?
Log base 10 of n. Because I am basically cutting down by a factor of 10 each time.
And how long does it take me to reduce it to 1?
That is the definition, if you will, of what a log is, log base 10.
What is this one?
What is this path going that way?
Log of n. Log base 10/9 of n. Because we're going down by 9/10 each time.
Once again, essentially the definition of n. And everything in between there is somewhere between log base 10 of n and log base 10/9 of n. So, everything is in between there.
Now what I can do is do the trick that we did for mergesort in looking at what the evaluation of this is by adding up what is the cost of the total level.
That is just cn.
What is the cost of the next level?
cn.
And what is the cost of the next level?
cn.
Every level we are still doing the same amount of work.
And we take that all the way down.
And the last levels -- Eventually we hit some point where it is not equal to cn where we start getting things that are less than or equal to cn because some of the leaves start dropping out starting at this level.
Basically this part is going to be log base 10 of n, and then we start getting things that are less than or equal to cn, and so forth, until finally we get to add it all up.
T(n) is going to be less than or equal to cn times, well, what is the longest that this could possibly be?
Log base 10/9 of n. Plus we have all of the leaves that we have to add in, but all the leaves together add up to just order n. All the leaves add up to order n, so we have + Theta(n).
And so this is how much?
If I add all of this together, what is this asymptotically?
That is n log n. So, T(n) is actually bounded by n log n. We are lucky.
Those people who guessed lucky were right.
A 1/10 : 9/10 split is asymptotically as good as a 50 : 50 split.
And, in fact, we can lower bound this by just looking at these things here and discover that, in fact, T(n) is lower bounded by cn log base 10 of n + order n. And so T(n) is lower bounded by also asymptotically n log n. So, T(n) is actually Theta(n lg n).
Now, this is not really proof.
I generally recommend that you don't do this kind of thing to do a proof.
This is a good intuition of a recursion tree.
The way you prove this is what?
Substitution method.
Good.
What you do is use this to get your guess and then use substitution method to prove that your guess is right.
It is too easy to make mistakes with this method.
It is very easy to make mistakes.
With the substitution method it is harder to make mistakes because there is just algebra there that you are cranking through.
It is easier to verify rather than dot, dot, dots and trees that you drew improperly and wrote in wrong amounts and so forth.
OK?
So, this is n log n. That's pretty good.
It is order n log n. And we are lucky.
Now let's try another one.
This is all for intuition because, I will tell you, by the time we get to the end of this class you folks are going to bolting for the door because we are going to do some good math today, actually.
It is actually fun math, I think, but it is challenging.
If you are not awake, you can still sleep now, but I will tell you when to wake up.
One more bit of intuition.
Suppose that we alternate steps.
Suppose we do the partitioning thing.
And it happens that we start out lucky and then we have a partitioning step that is unlucky and then we have a step that is lucky and a step that is unlucky and we do that all the way down the tree.
Suppose we alternate.
Are we lucky or unlucky if we do that?
This time I want everybody voting.
It doesn't matter what your answer is.
Everybody has to have a stake in the game.
It is sort of like horseracing.
If ever you have watched horseracing, it is really boring, but if you put a little bit of money down, a little skin in the game suddenly it is interesting.
The same thing here.
I want everybody to put some skin in the game.
Who thinks that this is going to be lucky?
Who thinks it is going to be unlucky?
OK. Who didn't vote?
[LAUGHTER] You guys.
No skin in the game, ha?
Let's analyze this so we can once again write a recurrence.
On the lucky step, we will have L(n) be the running time on a lucky step of size n. And that is going to be twice.
While the next step is going to be unlucky.
It is two unluckies over 2 plus order n. That is our lucky step.
And then for the unlucky step it is essentially going to be L of n minus 1, it is going to be lucky on the next step, plus order n. That is unlucky.
See how I have described this behavior with a system now of recurrences that are dependent where the boundary cases, once again which are unstated, is that the recurrences have a constant solution with constant input.
Now we just do a little bit of algebra using substitution.
L(n) is then equal to, well, I can just plug in, for U(n/2) plug in the value of U(n/2).
And that gives me 2[L(n/2-1) + Theta(n) + Theta(n)].
See what I did here?
I simply plugged in, for U(n/2), this recurrence.
In fact, technically I guess I should have said Theta(n/2) just to make this substitution more straightforward.
It is the same thing, but just to not skip a step.
That we can now crank through.
And that is 2L(n/2 - 1) +, and now I have two T(n/2) plus another one, so all of that is just order n. And what is the solution to that recurrence?
n log n. Theta(n lg n).
Does everybody see that?
OK?
Theta(n lg n).
This is basically just, once again, master theorem with a little bit of jiggering here.
That minus one is only going to help us, actually, in the solution of the master theorem.
So, it is order n lg n. We are lucky.
If we alternate lucky and unlucky, we are lucky.
How can we insure that we are usually lucky?
If I have the input already sorted, I am going to be unlucky.
Excuse me?
You could randomly arrange the elements, that is one way.
What is another way?
That is a perfectly good way, actually.
In fact, it is a common thing to do.
Randomly choose the pivot, OK.
It turns out those are effectively equivalent, but we are going to do the randomly choose the pivot because it is a little bit easier to analyze.
But they are effectively equivalent.
That gives us the algorithm called randomized quicksort.
And the nice thing about randomized quicksort is that the running time is independent of the input ordering.
Very much for the same reason that if I just scramble the input, it would be independent of the input ordering.
If I randomly scramble the input then it doesn't matter what the order of the input was.
Whereas, original quicksort has some slow cases, input sorted or reverse sorted, and some fast cases.
In particular, it turns out that if it is random it is going to be pretty fast.
If I actually randomly scramble the input or pivot on a random element, it doesn't matter what the input was.
One way of thinking about this is with an adversary.
Imagine your adversary, you are saying I have a good sorting algorithm and he says I have a good sorting algorithm and you're trying to sell to a single customer.
And the customer says OK, you guys come up with benchmarks for each of your algorithms.
And you get to look at his algorithm.
Well, you look and you say oh, he is using quicksort.
I will just give him something that is already sorted.
That is what you could do to him.
If you had quicksort, he would do the same thing to you.
So, how can you defeat him?
Well, one way is with randomization.
Big idea in computer science, use random numbers.
The idea here is if I permute the ordering at random, as one suggestion, or I pivot at random places, then the input ordering didn't matter.
And so there is no bad ordering that he can provide that is going to make my code run slowly.
Now, I might get unlucky.
But that is just unlucky in my use of my random-number generator.
It is not unlucky with respect to what the input was.
What the input was doesn't matter.
Everybody follow that?
OK.
The nice thing about randomized quicksort is that it makes no assumptions about the input distribution.
You don't have to assume that all inputs are equally likely because either you can make it that way or you pivot in a way that makes that effectively whole.
And, in particular, there is no specific input that can elicit the worst-case behavior.
The worst-case is determined only by a random-number generator.
And, therefore, since it is only determined by a random-number generator, we can essentially bound the unluckiness mathematically.
We can say what are the odds?
So, we are going to analyze this.
And this is where you know if you belong in this course or not.
If you have skipped 6.042 or whatever, this is a good place to do the comparison.
Since it is going to be a little bit, why don't people just stand up for a moment and take a stretch break.
Since this is going to be a nice piece of mathematics we are going to do, you are going to want to feel fresh for it.
Stretch break is over.
Analysis.
Good.
I think we are going to make this.
I am sort of racing.
There is a lot of stuff to cover today.
Good.
Let's let T(n) now be the random variable for the running time assuming -- Wow.
I didn't even write here what we did here.
So, we are going to pivot on a random element.
That is the basic scheme we are going to do.
And the way I do that, by the way, is just in the code for partition, rather than partitioning on the first element, before I do the partition, I just swap the first element with some other element in the array chosen at random, perhaps itself.
So, they are all equally likely to be pivoted on.
And then just run the ordinary partition.
This is a random variable for running in time assuming, we have to make an assumption for doing probability, the random numbers are independent.
So that when I pivot in one place, it is independent of how I pivoted in some other place as I am running this algorithm.
Then, to analyze this, what I am going to do is I want to know where we pivoted.
For k = 0, 1, ..., n-1, let's let, for a particular partition, the random variable X_k = 1 if partition generates a k : n-k-1 split, and 0 otherwise.
In the partition routine, I am picking a random element to pivot on.
And X_k is going to be my random variable that is 1 if it generates a split that has k elements on the left side and n-k-1 elements on the right side of the pivot.
Some of those, too, of course are n-1 because I also have the pivot.
And 0 otherwise.
So, I now have n random variables that I have defined associated with a single partition where all of them are going to be zero except one of them, whichever one happens to occur is going to have the value This is called, by the way.
What is the name of this type of random variable?
Bernoulli.
Well, Bernoulli has other assumptions.
It is an indicator random variable.
It turns out it is Bernoulli, but that's OK.
It is an indicator random variable.
It just takes on the value of 0, 1.
And Bernoulli random variables are a particular type of indicator random variable.
Which it turns out these are.
That is an indicator random variable.
Indicator random variables are a good way when you are trying to understand what the sum of a bunch of things is.
It is a good way to break apart your big random variables into smaller ones that can be analyzed.
Let's just take a look at this indicator random variable.
What is the expectation of X_k equal to?
In other words, what is the probability that I generate a k : n-k-1 split?
X_k is, let's just write out what that means, just to refresh people's memory.
That is 0 times the probability that X_k equals 0 plus 1 times the probability that X_k equals 1, which is equal, well, that is all zero.
That is just equal to the probability that X_k equals 1.
And that is a general property of indicator random variables, is that their expectation is the probability that they are 1.
The nice thing about indicator random variables is it directly connects the probability to the expectation without any other terms going on.
What is the probability of X_k equals 1?
1/n.
So, all splits are equally likely.
And I have n elements, so each ones has a 1/n chance of being picked as the pivot.
And, once you pick the pivot, that determines what is on the left and the right and so forth.
So, it is 1/n.
Everybody with me so far?
More or less?
OK. As I say, this is going to test whether you're in the class.
If you go home and you study this and you cannot get it, and you have a deficiency in your math background in trying to take the course, this is a good indication that probably you have taken something a little over your head.
Let's write out what T(n) is equal to here.
T(n) is going to be equal to T(0) + T(n-1) + Theta(n) if we get a 0 : n-1 split and is equal to T(1) + T(n-2) + order n if we have a 1 : n-2 split.
And now down here it is going to be T(n-1) + T(0) + Theta(n) if we end up with an n-1 : 0 split.
So, this is our recurrence for T(n).
And, unfortunately, the recurrence is kind of hairy because it has got n cases.
And this is, once again, where the brilliance of being able to use indicator random variables comes in.
Because we will be able to take this case analysis and reduce it to mathematics so we don't have cases using indicator random variables.
And the way we do that is using the following trick of converting the cases into a summation.
Let's just take a look at why these two things are the same.
The indicator random variable is zero, except if you get the particular split.
Therefore, this summation is going to be zero, except for that k which actually appeared in which case it is the value that we say it is.
See the trick using multiplication by 0, 1 variable to handle all the cases?
I think that is damn clever.
I think that is damn clever.
And this is like the classic thing that you do with indicator random variables.
It's one of the reasons they are a very powerful method.
Because now we actually have a mathematical expression, hairy although it may be, for our recurrence.
Now, what we are going to analyze is the expected value of T(n).
That is what we want to do.
What is the expected value of T(n)?
To do that, I just write the expected value of T(n) is equal to the expected value of this big summation.
And now we can go ahead and start to evaluate the expected value of that summation.
Everybody with me?
Yes?
Any questions at this point?
I see a thumbs up.
That's nice to see.
But I generally believe that what I want to see is no thumbs down.
It is good to see the thumbs up, but that means one person understands, or thinks he understands.
[LAUGHTER] So, this is, I claim, equal to the following.
Actually, I am going to need a little space here so I am going to move the equal sign over a little bit.
I claim that summation is equal to that.
This expectation is equal to that summation of expectations.
Why is that?
What are the magic words that justify this step?
Linearity of expectation.
The expectation of a sum is the sum of the expectations.
So, that is linearity of expectation.
I don't need independence for that.
That is just always true for expectation of any random variables.
The sum of the expectations is the expectation of the sum and vice versa.
Here we did the vice versa.
That is equal to now the sum of k=0 to n-1 of expectation of X_k [T(k) + T(n-k-1) + Theta(n)].
Why is that true?
What I have done is I've said the expectation of the product is the product of the expectations.
That is because of independence.
What is independent of what?
The X_k here, random variable, are independent of any of the other partitionings in, if you will, the X_k that would exist for any of the other recursive calls.
So, whatever happens in here is independent of what happened there.
We are actually hiding.
Since we have a recurrence, we are not partitioning the same wage time.
We have a different one.
We actually have something going on underneath the mathematics you have to pay attention to that the mathematics alone isn't really showing, which is that in T(k) there is actually a set of random choices that are being made, if you will.
And so you have to understand that those are independent of those, in which case we can multiple the probabilities of their expectations.
Is everybody with me?
That is a big one, independence of X_k from other random choices.
That is equal to now, well, first of all, this is nice.
What is the expectation of X_k?
1/n.
That actually doesn't even belong in the summation.
We will just pop it outside.
I get 1/n times the sum of k=0 to n-1 of expectation of T(k) + 1/n summation k=0 to n-1 of expectation of T(n-k-1) + 1/n summation k=0 to n-1 up to Theta(n).
That is, again, using linearity of expectation there this time to split up these pieces and just factoring out the expectation of k as being 1/n.
Everybody with me still?
All of this is elementary.
It is just one of these things that is hard just because there are so many steps.
And it takes that you have seen some of this before.
Now the next observation is that these two summations are, in fact, identical.
They are the same summation, just in a different order.
This is going T(0), T(1), T(2), T(3) up to T(n-1).
This one is going T(n-1), T(n-2), T(n-3) down to T(0).
These are, in fact, equal.
So, therefore, I have two of them.
And then what is this term equal to?
What is that one equal to?
Theta(n).
Let's just see why.
The summation of 0 : n of Theta(n) is Theta(n^2) divided by n. Or, if I want to bring the Theta(n) out, I have 1 times the summation of k equals1 to n of Theta(1) or of So, once again, you get n, either way of doing it.
This is, in some sense, because the summations have identical terms, and this is just algebra.
Now what we are going to do is do something for technical convenience.
Because we are going to absorb the k=0, 1 terms into the Theta(n) for technical convenience.
We have a recurrence here where I have an order n. And, if I look at the cases where k=0 or k=1, I know what the expectation is.
For 0, 1, the expected cost is the worst case cost, which is constant.
Because I am only solving the problem for a constant size.
And we know that for any of the boundary cases that our solution of recurrence, our assumption is that it is constant time.
So, I basically can just take those two terms out.
And all that does it accumulate some more constant here in the Theta(n).
It is going to make the solution of the recurrence a little bit easier.
And, if I do that, I get expectation of T(n) = 2/n summation k=2 to n-1 of expectation of T(k) + Theta(n).
So, all of that work was to derive the recurrence.
And now we have to solve it.
Just to review what we did, we started out with a recurrence which was for the random variable which involved a case statement.
We converted that into some mathematics without the case statement, just with a product, and then we derived a recurrence for the expectation.
And now we are in the process of trying to solve that recurrence.
We have done some simplification of the recurrence so that we understand what it is that we are going to solve here.
By the way, I don't give things like this on quizzes.
I do expect you to understand it.
The elements of this you will find on a quiz.
This is a lot of work to figure out.
This took smart people to do.
Even though it is all elementary, but working out something like this at the elementary level is still a bit of work even for somebody who is knowledgeable in this area.
Now we are going to solve that last recurrence over there and we are going to prove that the expectation of T(n) is less than or equal to (an lg n) for some constant a greater than 0.
That is going to be what we are going to do.
And so what technique do you think we should use to prove this?
Does this look like a master method?
It is nothing like the master method.
So, when in doubt do substitution.
It is the grand-daddy of all methods.
What we will do is solve the base case by simply choosing a big enough so that (an lg n) is bigger than the expectation of T(n) for sufficiently large small n. So, I just pick a to be big enough.
And this is, by the way, why I wanted to exclude 0 and 1 from the recurrence.
Because, for example, when n=0, log of 0 is, it's like dividing by 0, right, you cannot do it.
Log of 1 is 0.
So here, even if I restricted it to 1, here I would have a 0, and I can't ever pick a big enough to dominate those cases.
What I do is I just say look, I just absorb whatever the cost is into the T(n) for technical convenience.
And that lets me address it as (an lg n) to be big enough to handle the base case.
So, that is why we made that technical assumption.
We are going to use a fact which is that the summation of k=2 to n-1 of k lg k is less than or equal to 1/2n^2 lg n - 1/8n^2.
I am going to leave that as an exercise for you to workout.
I think it is an exercise in the book, too.
I want you to go and evaluate this.
There are two ways to evaluate it.
One is by using purely summations and facts about summations by splitting the summation into two pieces and reconstituting it to prove this bound.
The other way is to use the integral method for solving summations.
Either way you can prove.
The integral method actually gets you a tighter bound than this.
This is a basic fact, and you should go off and know how to do that.
Now let's do substitution.
The expectation of T(n) is less than or equal to 2/n times the summation k=2 to n-1, now we do the substitution of ak lg k, the smaller values plus Theta(n).
I might mentioned, by the way, that the hard part of doing this, it is easy to get the bound without this term, it is easy to get this bound, 1/2n^2 lg n, it is harder to get the second order term.
It turns out you need the second order term in order to do what we are going to do.
You have to be able to subtract a quadratic amount of the n^2 lg n in order to make this proof work.
And that is the trickier part in evaluating that summation.
So, we get this.
That is less than or equal to?
Well, I happen to know how much this is by using that formula.
I use my fact and get 2a/n (1/2n^2 lg n - 1/8n^2) + Theta(n).
Did I do something wrong?
There we go.
Very good.
That is equal to - If I multiply this first part through that is an lg n. And now, so I don't make a mistake, I want to express this as my desired, this is what I want it to be, minus a residual.
I am going to write the residual as this part.
And so, the way to write that is, that is going to be minus.
And then it is going to be this term here, which is going to be an/4 - Theta(n).
And that is going to be less than or equal to an lg n if this part is positive.
And I can make that part positive by picking a big enough such that an/4 dominates the constant in the Theta(n) here.
Whatever the constant is here, I can find an a that is big enough so that this term makes this part positive.
If a is big enough so that an/4 dominates Theta(n).
And so the running time of randomized quicksort is order n lg n. That is what we just proved, the expected running time is order n lg n. Now, in practice, quicksort is a great algorithm.
It is typically three or more times faster than mergesort.
It doesn't give you the strong guarantee necessarily of mergesort and being worst-case n lg n. But in practice, if you use randomized quicksort, it is generally as much as three times faster.
It does require code tuning in order to get it up to be that fast.
You do have to go and coarsen the base cases and do some other tricks there, but most good sorting algorithms that you will find are based on quicksort.
Also one of the other reasons it works well is because it tends to work well with caches in virtual memory.
We are not really talking much about caching models and so forth, big topic these days in algorithms, but it does work very well with caches in virtual memory.
It is another reason that this is a good algorithm to use.
Good recitation, by the way, on Friday.
We are going to see another n log n time algorithm, a very important one in recitation on Friday.
Today we're going to talk about sorting, which may not come as such a big surprise.
We talked about sorting for a while, but we're going to talk about it at a somewhat higher level and question some of the assumptions that we've been making so far.
And we're going to ask the question how fast can we sort?
A pretty natural question.
You may think you know the answer.
Perhaps you do.
Any suggestions on what the answer to this question might be?
There are several possible answers.
Many of them are partially correct.
Let's hear any kinds of answers you'd like and start waking up this fresh morning.
Sorry?
Theta n log n. That's a good answer.
That's often correct.
Any other suggestions?
N squared.
That's correct if all you're allowed to do is swap adjacent elements.
Good.
That was close.
I will see if I can make every answer correct.
Usually n squared is not the right answer, but in some models it is.
Yeah?
Theta n is also sometimes the right answer.
The real answer is "it depends".
That's the point of today's lecture.
It depends on what we call the computational model, what you're allowed to do.
And, in particular here, with sorting, what we care about is the order of the elements, how are you allowed to manipulate the elements, what are you allowed to do with them and find out their order.
The model is what you can do with the elements.
Now, we've seen several sorting algorithms.
Do you want to shout some out?
I think we've seen four, but maybe you know even more algorithms.
Quicksort.
Keep going.
Heapsort.
Merge sort.
You can remember all the way back to Lecture 1.
Any others?
Insertion sort.
All right.
You're on top of it today.
I don't know exactly why, but these two are single words and these two are two words.
That's the style.
What is the running time of quicksort?
This is a bit tricky.
N log n in the average case.
Or, if we randomize quicksort, randomized quicksort runs in n log n expected for any input sequence.
Let's say n lg n randomized.
That's theta.
And the worst-case with plain old quicksort where you just pick the first element as the partition element.
That's n^2.
Heapsort, what's the running time there?
n lg n always.
Merge sort, I hope you can remember that as well, n lg n. And insertion sort?
n^2.
All of these algorithms run no faster than n lg n, so we might ask, can we do better than n lg n?
And that is a question, in some sense, we will answer both yes and no to today.
But all of these algorithms have something in common in terms of the model of what you're allowed to do with the elements.
Any guesses on what that model might be?
Yeah?
You compare pairs of elements, exactly.
That is indeed the model used by all four of these algorithms.
And in that model n lg n is the best you can do.
We have so far just looked at what are called comparison sorting algorithms or "comparison sorts".
And this is a model for the sorting problem of what you're allowed to do.
Here all you can do is use comparisons meaning less than, greater than, less than or equal to, greater than or equal to, equals to determine the relative order of elements.
This is a restriction on algorithms.
It is, in some sense, stating what kinds of elements we're dealing with.
They are elements that we can somehow compare.
They have a total order, some are less, some are bigger.
But is also restricts the algorithm.
You could say, well, I'm sorting integers, but still I'm only allowed to do comparisons with them.
I'm not allowed to multiply the integers or do other weird things.
That's the comparison sorting model.
And this lecture, in some sense, follows the standard mathematical progression where you have a theorem, then you have a proof, then you have a counter example.
It's always a good way to have a math lecture.
We're going to prove the theorem that no comparison sorting algorithm runs better than n lg n. Comparisons.
State the theorem, prove that, and then we'll give a counter example in the sense that if you go outside the comparison sorting model you can do better, you can get linear time in some cases, better than n lg n. So, that is what we're doing today.
But first we're going to stick to this comparison model and try to understand why we need n lg n comparisons if that's all we're allowed to do.
And for that we're going to look at something called decision trees, which in some sense is another model of what you're allowed to do in an algorithm, but it's more general than the comparison model.
And let's try and example to get some intuition.
Suppose we want to sort three elements.
This is not very challenging, but we'll get to draw the decision tree that corresponds to sorting three elements.
Here is one solution I claim.
This is, in a certain sense, an algorithm, but it's drawn as a tree instead of pseudocode.
What this tree means is that each node you're making a comparison.
This says compare a_1 versus a_2.
If a_1 is smaller than a_2 you go this way, if it is bigger than a_2 you go this way, and then you proceed.
When you get down to a leaf, this is the answer.
Remember, the sorting problem is you're trying to find a permutation of the inputs that puts it in sorted order.
Let's try it with some sequence of numbers, say 9, 4 and 6.
We want to sort 9, 4 and 6, so first we compare the first element with the second element.
9 is bigger than 4 so we go down this way.
Then we compare the first element with the third element, that's 9 versus 6.
9 is bigger than 6, so we go this way.
And then we compare the second element with the third element, 4 is less than 6 and, so we go this way.
And the claim is that this is the correct permutation of the elements.
You take a_2, which is 4, then you take a_3, which is 6, and then you take a_1, which is 9, so indeed that works out.
And if I wrote this down right, this is a sorting algorithm in the decision tree model.
In general, let me just say the rules of this game.
In general, we have n elements we want to sort.
And I only drew the n = 3 case because these trees get very big very quickly.
Each internal node, so every non-leaf node, has a label of the form i : j where i and j are between 1 and n. And this means that we compare a_i with a_j.
And we have two subtrees from every such node.
We have the left subtree which tells you what the algorithm does, what subsequent comparisons it makes if it comes out less than.
And we have to be a little bit careful because it could also come out equal.
What we will do is the left subtree corresponds to less than or equal to and the right subtree corresponds to strictly greater than.
That is a little bit more precise than what we were doing here.
Here all the elements were distinct so no problem.
But, in general, we care about the equality case too to be general.
So, that was the internal nodes.
And then each leaf node gives you a permutation.
So, in order to be the answer to that sorting problem, that permutation better have the property that it orders the elements.
This is from the first lecture when we defined the sorting problem.
Some permutation on n things such that a_pi(1) is less than or equal to a_pi(2) and so on.
So, that is the definition of a decision tree.
Any binary tree with these kinds of labels satisfies all these properties.
That is, in some sense, a sorting algorithm.
It's a sorting algorithm in the decision tree model.
Now, as you might expect, this is really not too different than the comparison model.
If I give you a comparison sorting algorithm, we have these four, quicksort, heapsort, merge sort and insertion sort.
All of them can be translated into the decision tree model.
It's sort of a graphical representation of what the algorithm does.
It's not a terribly useful one for writing down an algorithm.
Any guesses why?
Why do we not draw these pictures as a definition of quicksort or a definition of merge sort?
It depends on the size of the input, that's a good point.
This tree is specific to the value of n, so it is, in some sense, not as generic.
Now, we could try to write down a construction for an arbitrary value of n of one of these decision trees and that would give us sort of a real algorithm that works for any input size.
But even then this is not a terribly convenient representation for writing down an algorithm.
Well, let's write down a transformation that converts a comparison sorting algorithm to a decision tree and then maybe you will see why.
This is not a useless model, obviously, I wouldn't be telling you otherwise.
It will be very powerful for proving that we cannot do better than n lg n, but as writing down an algorithm, if you were going to implement something, this tree is not so useful.
Even if you had a decision tree computer, whatever that is.
But let's prove this theorem that decision trees, in some sense, model comparison sorting algorithms, which we call just comparison sorts.
This is a transformation.
And we're going to build one tree for each value of n. The decision trees depend on n. The algorithm hopefully, well, it depends on n, but it works for all values of n. And we're just going to think of the algorithm as splitting into two forks, the left subtree and the right subtree whenever it makes a comparison.
If we take a comparison sort like merge sort.
And it does lots of stuff.
It does index arithmetic, it does recursion, whatever.
But at some point it makes a comparison and then we say, OK, there are two halves of the algorithm.
There is what the algorithm would do if the comparison came out less than or equal to and what the algorithm would do if the comparison came out greater than.
So, you can build a tree in this way.
In some sense, what this tree is doing is listing all possible executions of this algorithm considering what would happen for all possible values of those comparisons.
We will call these all possible instruction traces.
If you write down all the instructions that are executed by this algorithm, for all possible input arrays, a_1 to a_n, see what all the comparisons, how they could come and what the algorithm does, in the end you will get a tree.
Now, how big will that tree be roughly?
As a function of n. Yeah?
Right.
If it's got to be able to sort every possible list of length n, at the leaves I have to have all the permutations of those elements.
That is a lot.
There are a lot of permeations on n elements.
There's n factorial of them.
N factorial is exponential, it's really big.
So, this tree is huge.
It's going to be exponential on the input size n. That is why we don't write algorithms down normally as a decision tree, even though in some cases maybe we could.
It's not a very compact representation.
These algorithms, you write them down in pseudocode, they have constant length.
It's a very succinct representation of this algorithm.
Here the length depends on n and it depends exponentially on n, which is not useful if you wanted to implement the algorithm because writing down the algorithm would take a long time.
But, nonetheless, we can use this as a tool to analyze these comparison sorting algorithms.
We have all of these.
Any algorithm can be transformed in this way into a decision tree.
And now we have this observation that the number of leaves in this decision tree has to be really big.
Let me talk about leaves in a second.
Before we get to leaves, let's talk about the depth of the tree.
This decision tree represents all possible executions of the algorithm.
If I look at a particular execution, which corresponds to some root to leaf path in the tree, the running time or the number of comparisons made by that execution is just the length of the path.
And, therefore, the worst-case running time, over all possible inputs of length n, is going to be -- n - 1?
Could be.
Depends on the decision tree.
But, as a function of the decision tree?
The longest path, right, which is called the height of the tree.
So, this is what we want to measure.
We want to claim that the height of the tree has to be at least n lg n with an omega in front.
That is what we'll prove.
And the only thing we're going to use is that the number of leaves in that tree has to be big, has to be n factorial.
This is a lower bound on decision tree sorting.
And the lower bound says that if you have any decision tree that sorts n elements then its height has to be at least n lg n up to constant factors.
So, that is the theorem.
Now we're going to prove the theorem.
And we're going to use that the number of leaves in that tree must be at least n factorial.
Because there are n factorial permutations of the inputs.
All of them could happen.
And so, for this algorithm to be correct, it has detect every one of those permutations in some way.
Now, it may do it very quickly.
We better only need n lg n comparisons because we know that's possible.
The depth of the tree may not be too big, but it has to have a huge number of leaves down there.
It has to branch enough to get n factorial leaves because it has to give the right answer in possible inputs.
This is, in some sense, counting the number of possible inputs that we have to distinguish.
This is the number of leaves.
What we care about is the height of the tree.
Let's call the height of the tree h. Now, if I have a tree of height h, how many leaves could it have?
What's the maximum number of leaves it could have?
2^h, exactly.
Because this is binary tree, comparison trees always have a branching factor of 2, the number of leaves has to be at most 2^h, if I have a height h tree.
Now, this gives me a relation.
The number of leaves has to be greater than or equal to n factorial and the number of leaves has to be less than or equal to 2^h.
Therefore, n factorial is less than or equal to 2^h, if I got that right.
Now, again, we care about h in terms of n factorial, so we solve this by taking logs.
And I am also going to flip sides.
Now h is at least log base 2, because there is a 2 over here, of n factorial.
There is a property that I'm using here in order to derive this inequality from this inequality.
This is a technical aside, but it's important that you realize there is a technical issue here.
The general principle I'm applying is I have some inequality, I do the same thing to both sides, and hopefully that inequality should still be true.
But, in order for that to be the case, I need a property about that operation that I'm performing.
It has to be a monotonic transformation.
Here what I'm using is that log is a monotonically increasing function.
That is important.
If I multiply both sides by -1, which is a decreasing function, the inequality would have to get flipped.
The fact that the inequality is not flipping here, I need to know that log is monotonically increasing.
If you see log that's true.
We need to be careful here.
Now we need some approximation of n factorial in order to figure out what its log is.
Does anyone know a good approximation for n factorial?
Not necessarily the equation but the name.
Stirling's formula.
Good.
You all remember Stirling.
And I just need the highest order term, which I believe is that.
N factorial is at least (n/e)^n.
So, that's all we need here.
Now I can use properties of logs to bring the n outside.
This is n lg (n/e).
And then lg (n/e) I can simplify.
That is just lg n - lg e. So, this is n(lg n - lg e).
Lg e is a constant, so it's really tiny compared to this lg n which is growing within.
This is Omega(n lg n).
All we care about is the leading term.
It is actually Theta(n lg n), but because we have it greater than or equal to all we care about is the omega.
A theta here wouldn't give us anything stronger.
Of course, not all algorithms have n lg n running time or make n lg n comparisons.
Some of them do, some of them are worse, but this proves that all of them require a height of at least n lg n. There you see proof, once you observe the fact about the number of leaves, and if you remember Stirling's formula.
So, you should know this proof.
You can show that all sorts of problems require n lg n time with this kind of technique, provided you're in some kind of a decision tree model.
That's important.
We really need that our algorithm can be phrased as a decision tree.
And, in particular, we know from this transformation that all comparison sorts can be represented as the decision tree.
But there are some sorting algorithms which cannot be represented as a decision tree.
And we will turn to that momentarily.
But before we get there I phrased this theorem as a lower bound on decision tree sorting.
But, of course, we also get a lower bound on comparison sorting.
And, in particular, it tells us that merge sort and heapsort are asymptotically optimal.
Their dependence on n, in terms of asymptotic notation, so ignoring constant factors, these algorithms are optimal in terms of growth of n, but this is only in the comparison model.
So, among comparison sorting algorithms, which these are, they are asymptotically optimal.
They use the minimum number of comparisons up to constant factors.
In fact, their whole running time is dominated by the number of comparisons.
It's all Theta(n lg n).
So, this is good news.
And I should probably mention a little bit about what happens with randomized algorithms.
What I've described here really only applies, in some sense, to deterministic algorithms.
Does anyone see what would change with randomized algorithms or where I've assumed that I've had a deterministic comparison sort?
This is a bit subtle.
And I only noticed it reading the notes this morning, oh, wait.
I will give you a hint.
It's over here, the right-hand side of the world.
If I have a deterministic algorithm, what the algorithm does is completely determinate at each step.
As long as I know all the comparisons that it made up to some point, it's determinate what that algorithm will do.
But, if I have a randomized algorithm, it also depends on the outcomes of some coin flips.
Any suggestions of what breaks over here?
There is more than one tree, exactly.
So, we had this assumption that we only have one tree for each n. In fact, what we get is a probability distribution over trees.
For each value of n, if you take all the possible executions of that algorithm, all the instruction traces, well, now, in addition to branching on comparisons, we also branch on whether a coin flip came out heads or tails, or however we're generating random numbers it came out with some value between 1 and n. So, we get a probability distribution over trees.
This lower bound still applies, though.
Because, no matter what tree we get, I don't really care.
I get at least one tree for each n. And this proof applies to every tree.
So, no matter what tree you get, if it is a correct tree it has to have height Omega(n lg n).
This lower bound applies even for randomized algorithms.
You cannot get better than n lg n, because no matter what tree it comes up with, no matter how those coin flips come out, this argument still applies.
Every tree that comes out has to be correct, so this is really at least one tree.
And that will now work.
We also get the fact that randomized quicksort is asymptotically optimal in expectation.
But, in order to say that randomized quicksort is asymptotically optimal, we need to know that all randomized algorithms require Omega(n lg n) comparisons.
Now we know that so all is well.
That is the comparison model.
Any questions before we go on?
Good.
The next topic is to burst outside of the comparison model and try to sort in linear time.
It is pretty clear that, as long as you don't have some kind of a parallel algorithm or something really fancy, you cannot sort any better than linear time because you've at least got to look at the data.
No matter what you're doing with the data, you've got to look at it, otherwise you're not sorting it correctly.
So, linear time is the best we could hope for.
N lg n is pretty close.
How could we sort in linear time?
Well, we're going to need some more powerful assumption.
And this is the counter example.
We're going to have to move outside the comparison model and do something else with our elements.
And what we're going to do is assume that they're integers in a particular range, and we will use that to sort in linear time.
We're going to see two algorithms for sorting faster than n lg n. The first one is pretty simple, and we will use it in the second algorithm.
It's called counting sort.
The input to counting sort is an array, as usual, but we're going to assume what those array elements look like.
Each A[i] is an integer from the range of 1 to k. This is a pretty strong assumption.
And the running time is actually going to depend on k. If k is small it is going to be a good algorithm.
If k is big it's going to be a really bad algorithm, worse than n lg n. Our goal is to output some sorted version of this array.
Let's call this sorting of A.
It's going to be easier to write down the output directly instead of writing down permutation for this algorithm.
And then we have some auxiliary storage.
I'm about to write down the pseudocode, which is why I'm declaring all my variables here.
And the auxiliary storage will have length k, which is the range on my input values.
Let's see the algorithm.
This is counting sort.
And it takes a little while to write down but it's pretty straightforward.
First we do some initialization.
Then we do some counting.
Then we do some summing.
And then we actually write the output.
Is that algorithm perfectly clear to everyone?
No one.
Good.
This should illustrate how obscure pseudocode can be.
And when you're solving your problem sets, you should keep in mind that it's really hard to understand an algorithm just given pseudocode like this.
You need some kind of English description of what's going on because, while you could work through and figure out what this means, it could take half an hour to an hour.
And that's not a good way of expressing yourself.
And so what I will give you now is the English description, but we will refer back to this to understand.
This is sort of our bible of what the algorithm is supposed to do.
Let me go over it briefly.
The first step is just some initialization.
The C[i]'s are going to count some things, count occurrences of values.
And so first we set them to zero.
Then, for every value we see A[j], we're going to increment the counter for that value A[j].
Then the C[i]s will give me the number of elements equal to a particular value i.
Then I'm going to take prefix sums, which will make it so that C[i] gives me the number of keys, the number of elements less than or equal to [i] instead of equals.
And then, finally, it turns out that's enough to put all the elements in the right place.
This I will call distribution.
This is the distribution step.
And it's probably the least obvious of all the steps.
And let's do an example to make it more obvious what's going on.
Let's take an array A = [4, 1, 3, 4, 3].
And then I want some array C. And let me add some indices here so we can see what the algorithm is really doing.
Here it turns out that all of my numbers are in the range 1 to 4, so k = 4.
My array C has four values.
Initially, I set them all to zero.
That's easy.
And now I want to count through everything.
And let me not cheat here.
I'm in the second step, so to speak.
And I look for each element in order.
I look at the C[i] value.
The first element is 4, so I look at C4.
That is 0.
I increment it to 1.
Then I look at element 1.
That's 0.
I increment it to 1.
Then I look at 3 and that's here.
It is also 0.
I increment it to 1.
Not so exciting so far.
Now I see 4, which I've seen before, how exciting.
I had value 1 in here, I increment it to 2.
Then I see value 3, which also had a value of 1.
I increment that to 2.
The result is [1, 0, 2, 2].
That's what my array C looks like at this point in the algorithm.
Now I do a relatively simple transformation of taking prefix sums.
I want to know, instead of these individual values, the sum of this prefix, the sum of this prefix, the sum of this prefix and the sum of this prefix.
I will call that C prime just so we don't get too lost in all these different versions of C. This is just 1.
And 1 plus 0 is 1.
1 plus 2 is 3.
3 plus 2 is 5.
So, these are sort of the running totals.
There are five elements total, there are three elements less than or equal to 3, there is one element less than or equal to 2, and so on.
Now, the fun part, the distribution.
And this is where we get our array B.
B better have the same size, every element better appear here somewhere and they should come out in sorted order.
Let's just run the algorithm.
j is going to start at the end of the array and work its way down to 1, the beginning of the array.
And what we do is we pick up the last element of A, A[n].
We look at the counter.
We look at the C vector for that value.
Here the value is 3, and this is the third column, so that has number 3.
And the claim is that's where it belongs in B.
You take this number 3, you put it in index 3 of the array B.
And then you decrement the counter.
I'm going to replace 3 here with 2.
And the idea is these numbers tell you where those values should go.
Anything of value 1 should go at position 1.
Anything with value 3 should go at position 3 or less.
This is going to be the last place that a 3 should go.
And then anything with value 4 should go at position 5 or less, definitely should go at the end of the array because 4 is the largest value.
And this counter will work out perfectly because these counts have left enough space in each section of the array.
Effectively, this part is reserved for ones, there are no twos, this part is reserved for threes, and this part is reserved for fours.
You can check if that's really what this array means.
Let's finish running the algorithm.
That was the last element.
I won't cross it off, but we've sort of done that.
Now I look at the next to last element.
That's a 4.
Fours go in position 5.
So, I put my 4 here in position 5 and I decrement that counter.
Next I look at another 3.
Threes now go in position 2, so that goes there.
And then I decrement that counter.
I won't actually use that counter anymore, but let's decrement it because that's what the algorithm says.
I look at the previous element.
That's a 1.
Ones go in position 1, so I put it here and decrement that counter.
And finally I have another 4.
And fours go in position 4 now, position 4 is here, and I decrement that counter.
So, that's counting sort.
And you'll notice that all the elements appear and they appear in order, so that's the algorithm.
Now, what's the running time of counting sort?
kn is an upper bound.
It's a little bit better than that.
Actually, quite a bit better.
This requires some summing.
Let's go back to the top of the algorithm.
How much time does this step take?
k. How much time does this step take?
n. How much time does this step take?
k. Each of these operations in the for loops is taking constant time, so it is how many iterations of that for loop are there?
And, finally, this step takes n. So, the total running time of counting sort is k + n. And this is a great algorithm if k is relatively small, like at most n. If k is big like n^2 or 2^n or whatever, this is not such a good algorithm, but if k = O(n) this is great.
And we get our linear time sorting algorithm.
Not only do we need the assumption that our numbers are integers, but we need that the range of the integers is pretty small for this algorithm to work.
If all the numbers are between 1 and order n then we get a linear time algorithm.
But as soon as they're up to n lg n we're toast.
We're back to n lg n sorting.
It's not so great.
So, you could write a combination algorithm that says, well, if k is bigger than n lg n, then I will just use merge sort.
And if it's less than n lg n I'll use counting sort.
And that would work, but we can do better than that.
How's the time?
It is worth noting that we've beaten our bound, but only assuming that we're outside the comparison model.
We haven't really contradicted the original theorem, we're just changing the model.
And it's always good to question what you're allowed to do in any problem scenario.
In, say, some practical scenarios, this would be great if the numbers you're dealing with are, say, a byte long.
Then k is only 2^8, which is 256.
You need this auxiliary array of size 256, and this is really fast.
256 + n, no matter how big n is it's linear in n. If you know your numbers are small, it's great.
But if you're numbers are bigger, say you still know they're integers but they fit in like 32 bit words, then life is not so easy.
Because k is then 2^32, which is 4.2 billion or so, which is pretty big.
And you would need this auxiliary array of 4.2 billion words, which is probably like 16 gigabytes.
So, you just need to initialize that array before you can even get started.
Unless n is like much, much more than 4 billion and you have 16 gigabytes of storage just to throw away, which I don't even have any machines with 16 gigabytes of RAM, this is not such a great algorithm.
Just to get a feel, it's good, the numbers are really small.
What we're going to do next is come up with a fancier algorithm that uses this as a subroutine on small numbers and combines this algorithm to handle larger numbers.
That algorithm is called radix sort.
But we need one important property of counting sort before we can go there.
And that important property is stability.
A stable sorting algorithm preserves the order of equal elements, let's say the relative order.
This is a bit subtle because usually we think of elements just as numbers.
And, yeah, we had a couple threes and we had a couple fours.
It turns out, if you look at the order of those threes and the order of those fours, we kept them in order.
Because we took the last three and we put it here.
Then we took the next to the last three and we put it to the left of that where O is decrementing our counter and moving from the end of the array to the beginning of the array.
No matter how we do that, the orders of those threes are preserved, the orders of the fours are preserved.
This may seem like a relatively simple thing, but if you look at the other four sorting algorithms we've seen, not all of them are stable.
So, this is an exercise.
Exercise is figure out which other sorting algorithms that we've seen are stable and which are not.
I encourage you to work that out because this is the sort of thing that we ask on quizzes.
But for now all we need is that counting sort is stable.
And I won't prove this, but it should be pretty obvious from the algorithm.
Now we get to talk about radix sort.
Radix sort is going to work for a much larger range of numbers in linear time.
Still it has to have an assumption about how big those numbers are, but it will be a much more lax assumption.
Now, to increase suspense even further, I am going to tell you some history about radix sort.
This is one of the oldest sorting algorithms.
It's probably the oldest implemented sorting algorithm.
It was implemented around 1890.
This is Herman Hollerith.
Let's say around 1890.
Has anyone heard of Hollerith before?
A couple people.
Not too many.
He is sort of an important guy.
He was a lecturer at MIT at some point.
He developed an early version of punch cards.
Punch card technology.
This is before my time so I even have to look at my notes to remember.
Oh, yeah, they're called punch cards.
You may have seen them.
If not they're in the PowerPoint lecture notes.
There's this big grid.
These days, if you've used a modern punch card recently, they are 80 characters wide and, I don't know, I think it's something like 16, I don't remember exactly.
And then you punch little holes here.
You have this magic machine.
It's like a typewriter.
You press a letter and that corresponds to some character.
Maybe it will punch out a hole here, punch out a hole here.
You can see the website if you want to know exactly how this works for historical reasons.
You don't see these too often anymore, but this is in particular the reason why most terminals are 80 characters wide because that was how things were.
Hollerith actually didn't develop these punch cards exactly, although eventually he did.
In the beginning, in 1890, the big deal was the US Census.
If you watched the news, I guess like a year or two ago, the US Census was a big deal because it's really expensive to collect all this data from everyone.
And the Constitution says you've got to collect data about everyone every ten years.
And it was getting hard.
In particular, in 1880, they did the census.
And it took them almost ten years to complete the census.
The population kept going up, and ten years to do a ten-year census, that's going to start getting expensive when they overlap with each other.
So, for 1890 they wanted to do something fancier.
And Hollerith said, OK, I'm going to build a machine that you take in the data.
It was a modified punch card where you would mark out particular squares depending on your status, whether you were single or married or whatever.
All the things they wanted to know on the census they would encode in binary onto this card.
And then he built a machine that would sort these cards so you could do counting.
And, in some sense, these are numbers.
And the numbers aren't too big, but they're big enough that counting sort wouldn't work.
I mean if there were a hundred numbers here, 2^100 is pretty overwhelming, so we cannot use counting sort.
The first idea was the wrong idea.
I'm going to think of these as numbers.
Let's say each of these columns is one number.
And so there's sort of the most significant number out here and there is the least significant number out here.
The first idea was you sort by the most significant digit first.
That's not such a great algorithm, because if you sort by the most significant digit you get a bunch of buckets each with a pile of cards.
And this was a physical device.
It wasn't exactly an electronically controlled computer.
It was a human that would push down some kind of reader.
It would see which holes in the first column are punched.
And then it would open a physical bin in which the person would sort of swipe it and it would just fall into the right bin.
It was a semi-automated.
I mean the computer was the human plus the machine, but never mind.
This was the procedure.
You sorted it into bins.
Then you had to go through and sort each bin by the second digit.
And pretty soon the number of bins gets pretty big.
And if you don't have too many digits this is OK, but it's not the right thing to do.
The right idea, which is what Hollerith came up with after that, was to sort by the least significant digit first.
And you should also do that using a stable sorting algorithm.
Now, Hollerith probably didn't call it a stable sorting algorithm at the time, but we will.
And this won Hollerith lots of money and good things.
He founded this tabulating machine company in 1911, and that merged with several other companies to form something you may have heard of called IBM in 1924.
That may be the context in which you've heard of Hollerith, or if you've done punch cards before.
The whole idea is that we're doing a digit by digit sort.
I should have mentioned that at the beginning.
And we're going to do it from least significant to most significant.
It turns out that works.
And to see that let's do an example.
I think I'm going to need a whole two boards ideally.
First we'll see an example.
Then we'll prove the theorem.
The proof is actually pretty darn easy.
But, nonetheless, it's rather counterintuitive this works if you haven't seen it before.
Certainly, the first time I saw it, it was quite a surprise.
The nice thing also about this algorithm is there are no bins.
It's all one big bin at all times.
Let's take some numbers.
This is a three digit number.
I'm spacing out the digits so we can see them a little bit better.
657, 839, 436, 720 and 355.
I'm assuming here we're using decimal numbers.
Why not?
Hopefully this are not yet sorted.
We'd like to sort them.
The first thing we do is take the least significant digit, sort by the least significant digit.
And whenever we have equal elements like these two nines, we preserve their relative order.
So, 329 is going to remain above 839.
It doesn't matter here because we're doing the first sort, but in general we're always using a stable sorting algorithm.
When we sort by this column, first we get the zero, so that's 720, then we get 5, Then we get 6, Stop me if I make a mistake.
Then we get the 7s, and we preserve the order.
Here it happens to be the right order, but it may not be at this point.
We haven't even looked at the other digits.
Then we get 9s, there are two 9s, 329 and 839.
All right so far?
Good.
Now we sort by the middle digit, the next least significant.
And we start out with what looks like the 2s.
There is a 2 up here and a 2 down here.
Of course, we write the first 2 first, 720, then 329.
Then we have the 3s, so we have 436 and 839.
Then we have a bunch of 5s it looks like.
Have I missed anyone so far?
No.
Good.
We have three 5s, 355, 457 and 657.
I like to check that I haven't lost any elements.
We have seven here, seven here and seven elements here.
Good.
Finally, we sort by the last digit.
One thing to notice, by the way, is before we sorted by the last digit -- Currently these numbers don't resemble sorted order at all.
But if you look at everything beyond the digit we haven't yet sorted, so these two digits, that's nice and sorted, 20, 29, 36, 39, 55, 57, 57.
Pretty cool.
Let's finish it off.
We stably sort by the first digit.
And the smallest number we get is a 3, so we get 329 and then Then we get some 4s, 436 and 457, then we get a 6, 657, then a 7, and then we have an 8.
And check.
I still have seven elements.
Good.
I haven't lost anyone.
And, indeed, they're now in sorted order.
And you can start to see why this is working.
When I have equal elements here, I have already sorted the suffix.
Let's write down a proof of that.
What is nice about this algorithm is we're not partitioning into bins.
We always keep the huge batch of elements in one big pile, but we're just going through it multiple times.
In general, we sort of need to go through it multiple times.
Hopefully not too many times.
But let's first argue correctness.
To analyze the running time is a little bit tricky here because it depends how you partition into digits.
Correctness is easy.
We just induct on the digit position that we're currently sorting, so let's call that t. And we can assume by induction that it's sorted beyond digit t. This is our induction hypothesis.
We assume that we're sorted on the low-order t - 1 digits.
And then the next thing we do is sort on the t-th digit.
We just need to check that things work.
And we restore the induction hypothesis for t instead of t - When we sort on the t-th digit there are two cases.
If we look at any two elements, we want to know whether they're put in the right order.
If two elements are the same, let's say they have the same t-th digit -- This is the tricky case.
If they have the same t-th digit then their order should not be changed.
So, by stability, we know that they remain in the same order because stability is supposed to preserve things that have the same key that we're sorting on.
And then, by the induction hypothesis, we know that that keeps them in sorted order because induction hypothesis says that they used to be sorted.
Adding on this value in the front that's the same in both doesn't change anything so they remain sorted.
And if they have differing t-th digits -- -- then this sorting step will put them in the right order.
Because that's what sorting does.
This is the most significant digit, so you've got to order them by the t-th digit if they differ.
The rest are irrelevant.
So, proof here of correctness is very simple once you know the algorithm.
Any questions before we go on?
Good.
We're going to use counting sort.
We could use any sorting algorithm we want for individual digits, but the only algorithm that we know that runs in less than n lg n time is counting sort.
So, we better use that one to sort of bootstrap and get an even faster and more general algorithm.
I just erased the running time.
Counting sort runs in order k + n time.
We need to remember that.
And the range of the numbers is 1 to k or 0 to k - 1.
When we sort by a particular digit, we shouldn't use n lg n algorithm because then this thing will take n lg n for one round and it's going to have multiple rounds.
That's going to be worse than n lg n. We're going to use counting sort for each round.
We use counting sort for each digit.
And we know the running time of counting sort here is order k + n .
But I don't want to assume that my integers are split into digits for me.
That's sort of giving away too much flexibility.
Because if I have some number written in whatever form it is, probably written in binary, I can cluster together some of those bits and call that a digit.
Let's think of our numbers as binary.
Suppose we have n integers.
And they're in some range.
And we want to know how big a range they can be.
Let's say, a sort of practical way of thinking, you know, we're in a binary world, each integer is b bits long.
So, in other words, the range is from 0 to 2b - 1.
I will assume that my numbers are non-negative.
It doesn't make much difference if they're negative, too.
I want to know how big a b I can handle, but I don't want to split into bits as my digits because then I would have b digits and I would have to do b rounds of this algorithm.
The number of rounds of this algorithm is the number of digits that I have.
And each one costs me, let's hope, for linear time.
And, indeed, if I use a single bit, k = 2.
And so this is order n. But then the running time would be order n per round.
And there are b digits, if I consider them to be bits, order n times b time.
And even if b is something small like log n, if I have log n bits, then these are numbers between 0 and n - 1.
I already know how to sort numbers between 0 and n - 1 in linear time.
Here I'm spending n lg n time, so that's no good.
Instead, what we're going to do is take a bunch of bits and call that a digit, the most bits we can handle with counting sort.
The notation will be I split each integer into b/r digits.
Each r bits long.
In other words, I think of my number as being in base 2^r.
And I happen to be writing it down in binary, but I cluster together r bits and I get a bunch of digits in base 2^r.
And then there are b/ r digits.
This b/r is the number of rounds.
And this base -- This is the maximum value I have in one of these digits.
It's between 0 and 2^r.
This is, in some sense, k for a run of counting sort.
What is the running time?
Well, I have b/r rounds.
It's b/r times the running time for a round.
Which I have n numbers and my value of k is 2^r.
This is the running time of counting sort, n + k, this is the number of rounds, so this is b/r (n+2^r).
And I am free to choose r however I want.
What I would like to do is minimize this run time over my choices of r. Any suggestions on how I might find the minimum running time over all choices of r?
Techniques, not necessarily solutions.
We're not used to this because it's asymptomatic, but forget the big O here.
How do I minimize a function with respect to one variable?
Take the derivative, yeah.
I can take the derivative of this function by r, differentiate by r, set the derivative equal to 0, and that should be a critical point in this function.
It turns out this function is unimodal in r and you will find the minimum.
We could do that.
I'm not going to do it because it takes a little bit more work.
You should try it at home.
It will give you the exact minimum, which is good if you know what this constant is.
Differentiate with respect to r and set to 0.
I am going to do it a little bit more intuitively, in other words less precisely, but I will still get the right answer.
And definitely I will get an upper bound because I can choose r to be whatever I want.
It turns out this will be the right answer.
Let's just think about growth in terms of r. There are essentially two terms here.
I have b/r(n) and I have b/r(2^r).
Now, b/r(n) would like r to be as big as possible.
The bigger r is the number of rounds goes down.
This number in front of n, this coefficient in front of n goes down, so I would like r to be big.
So, b/r(n) wants r big.
However, r cannot be too big.
This is saying I want digits that have a lot of bits in them.
It cannot be too big because there's 2^r term out here.
If this happens to be bigger than n then this will dominate in terms of growth of r. This is going to be b times 2 to the r over r. 2 the r is much, much bigger than r, so it's going to grow much faster is what I mean.
And so I really don't want r to be too big for this other term.
So, that is b/4(2^r) wants r small.
Provided that this term is bigger or equal to this term then I can set r pretty big for that term.
What I want is the n to dominate the 2^r.
Provided I have that then I can set r as large as I want.
Let's say I want to choose r to be maximum subject to this condition that n is greater than or equal to 2^r.
This is an upper bound to 2^r, and upper bound on r. In other words, I want r = lg n. This turns out to be the right answer up to constant factors.
There we go.
And definitely choosing r to be lg n will give me an upper bound on the best running time I could get because I can choose it to be whatever I want.
If you differentiate you will indeed get the same answer.
This was not quite a formal argument but close, because the big O is all about what grows fastest.
If we plug in r = lg n we get bn/lg n. The n and the 2^r are equal, that's a factor of 2, 2 times n, not a big deal.
It comes out into the O.
We have bn/lg n which is r. We have to think about what this means and translate it in terms of range.
b was the number of bits in our number, which corresponds to the range of the number.
I've got 20 minutes under so far in lecture so I can go 20 minutes over, right?
No, I'm kidding.
Almost done.
Let's say that our numbers, are integers are in the range, we have 0 to 2^b, I'm going to say that it's range 0 to nd.
This should be a -1 here.
If I have numbers that are between 0 and n^d - 1 where d is a constant or d is some parameter, so this is a polynomial in n, then you work out this running time.
It is order dn.
This is the way to think about it because now we can compare to counting sort.
Counting sort could handle 0 up to some constant times d in linear time.
Now I can handle 0 up to n to some constant power in linear time.
This is if d = order 1 then we get a linear time sorting algorithm.
And that is cool as long as d is at most lg n. As long as your numbers are at most n lg n then we have something that beats our n lg n sorting algorithms.
And this is pretty nice.
Whenever you know that your numbers are order log end bits long we are happy, and you get some smooth tradeoff there.
For example, if we have our 32 bit numbers and we split into let's say eight bit chunks then we'll only have to do four rounds each linear time and we have just 256 working space.
We were doing four rounds for 32 bit numbers.
If you use n lg n algorithm, you're going to be doing lg n rounds through your numbers.
n is like 2000, and that's at least 11 rounds for example.
You would think this algorithm is going to be much faster for small numbers.
Unfortunately, counting sort is not very good on a cache.
In practice, rating sort is not that fast an algorithm unless your numbers are really small.
Something like quicksort can do better.
It's sort of shame, but theoretically this is very beautiful.
And there are contexts where this is really the right way to sort things.
I will mention finally that if you have arbitrary integers that are one word length long.
Here we're assuming that there are b bits in a word and we have some depends indirectly on b here.
But, in general, if you have a bunch of integers and they're one word length long, and you can manipulate a word in constant time, then the best algorithm we know for sorting runs in n times square root of lg lg n time expected.
It is a randomized algorithm.
We're not going to cover that algorithm in this class.
It's rather complicated.
I didn't even cover it in Advanced Algorithms when I taught it.
If you want something easier, you can get n times square root of lg lg n time worst-case.
And that paper is almost readable.
I have taught that in Advanced Algorithms.
If you're interested in this kind of stuff, take Advanced Algorithms next fall.
It's one of the follow-ons to this class.
These are much more complicated algorithms, but it gives you some sense.
You can even break out of the dependence on b, as long as you know that b is at most a word.
And I will stop there unless there are any questions.
Then see you Wednesday.
Today starts a two-lecture sequence on the topic of hashing, which is a really great technique that shows up in a lot of places.
So we're going to introduce it through a problem that comes up often in compilers called the symbol table problem.
And the idea is that we have a table S holding n records where each record, just to be a little more explicit here.
So each record typically has a bunch of, this is record x. x is usually a pointer to the actual data.
So when we talk about the record x, what it usually means some pointer to the data.
And in the data, in the record, so this is a record, there is a key called a key of x.
In some languages it's key, it's x dot key or x arrow key, OK, are other ways that that will be denoted in some languages.
And there's usually some additional data called satellite data, which is carried around with the key.
This is also true in sorting, but usually you're sorting records.
You're not sorting individual keys.
And so the idea is that we have a bunch of operations that we would like to do on this data on this table.
So we want to be able to insert an item x into the table, which just essentially means that we update the table by adding the element x.
We want to be able to delete an item from the table -- -- so removing the item x from the set and we want to be able to search for a given key.
So this returns the value x such that key of x is equal to k, where it returns nil if there's no such x.
So be able to insert items in, delete them and also look to see if there's an item that has a particular key.
So notice that delete doesn't take a key.
Delete takes a record.
OK, so if you want to delete something of a particular key and you don't happen to have a pointer to it, you have to say let me search for it and then delete it.
So these, whenever you have a set operations, where operations that change the set like in certain delete, we call it a dynamic set.
So these two operations make the set dynamic.
It changes over time.
Sometimes you want to build a fixed data structure.
It's going to be a static set.
All you're going to do is do things like look it up and so forth.
But most often, it turns out that in programming and so forth, we want to have the set be dynamic.
Want to be able to add elements to it, delete elements to it and so forth.
And there may be other operations that modify the set, modify membership in the set.
So the simplest implementation for this is actually often overlooked.
I'm actually surprised how often people use more complicated data structures when this simple data structure will work.
It's called a direct access table.
Doesn't always work.
I'll give the conditions where it does.
So it works when the keys are drawn from our small distribution.
So suppose the keys are drawn from a set U of m elements.
OK, zero to m minus one.
And we're going to assume the keys are distinct.
So the way a direct access table works is that you set up an array T -- -- from zero to m minus one to represent the dynamic set S -- -- such that T of k is going to be equal to x if x is in the set and its key is k and nil otherwise.
So you just simply have an array and if you have a record whose key is some value k, the key is 15 say, then slot 15 if the element is there has the element.
And if it's not in the set, it's nil.
Very simple data structure.
OK, insertion.
Just go to that location and set the value to the inserted value.
For deletion, just remove it from there.
And to look it up, you just index it and see what's in that slot.
OK, very simple data structure.
All these operations, therefore, take constant time in the worst case.
But as a practical matter, the places you can use this strategy are pretty limited.
What's the issue of limitation here?
Yes.
OK, so that's a limitation surely.
But there's actually a more severe limitation.
Yeah.
What does that mean, it's hard to draw?
No.
Yeah.
m minus one could be a huge number.
Like for example, suppose that I want to have my set drawn over 64 bit values.
OK, the things that I'm storing in my table is a set of 64-bit numbers.
And so, maybe a small set.
Maybe we only have a few thousand of these elements.
But they're drawn from a 64-bit value.
Then this strategy requires me to have an array that goes from zero to 2 to the 64th minus one.
How big is 2^64 minus one?
Approximately?
It's like big.
It's like 18 quintillion or something.
I mean, it's zillions literally because it's like it's beyond the illions we normally use.
Not a billion or a trillion.
It's 18 quintillion.
OK, so that's a really big number.
So, or even worse, suppose the keys were drawn from character strings, so people's names or something.
This would be an awful way to have to represent it.
Because most of the table would be empty for any reasonable set of values you would want to keep.
So the idea is we want to try to keep something that's going to keep the table small, while still preserving some of the properties.
And that's where hashing comes in.
So hashing is we use a hash function H which maps the keys randomly.
And I'm putting that in quotes because it's not quite at random.
Into slots table T. So we call each of the array indexes here a slot.
So you can just sort of think of it as a big table and you've got slots in the table where you're storing your values.
And so, we may have a big universe of keys.
Let's call that U.
And we have our table over here that we've set up that has -- -- m slots.
And so we actually have then a set that we're actually going to try to represent S, which is presumably a very small piece of the universe.
And what we'll do is we'll take an element from here and map it to let's say to there and take another one and we apply the hash function to the element.
And what the hash function is going to give us is it's going to give us a particular slot.
Here's one that might go up here.
Might have another one over here that goes down to there.
And so, we get it to distribute the elements over the table.
So what's the problem that's going to occur as we do this?
So far, I've been a little bit lucky.
What's the problem potentially going to be?
Yeah, when two things are in S, more specifically, get assigned to the same value.
So I may have a guy here and he gets mapped to the same slot that somebody else has already been mapped to.
And when this happens, we call that a collision.
So we're trying to map these things down into a small set but we could get unlucky in our mapping, particularly if we map enough of these guys.
They're not going to fit.
So when a record -- -- to be inserted maps to an already occupied slot -- -- a collision occurs.
OK.
So looks like this method's no good.
But no, there's a pretty simple thing we can do.
What should we do when two things map to the same slot?
If we want to represent the whole set, but you can't lose any data, can't treat it like a cache.
In a cache what you do is it uses a hashing scheme, but in a cache, you just kick it out because you don't care about representing a set precisely.
But in a hash table you're programming, you often want to make sure that the values you have are exactly the values in the sets so you can tell whether something belongs to the set or not.
So what's a good strategy here?
Yeah.
Create a list for each slot and just put all the elements that hash to the same slot into the list.
And that's called resolving collisions by chaining.
And the idea is to link records in the same slot -- -- into a list.
So for example, imagine this is my hash table and this for example is slot i. I may have several things that are, so I'm going to put the key value -- -- have several things that may have been inserted into this table that are elements of S. And what I'll do is just link them together.
OK, so nil pointer here.
And this is the satellite data and these are the keys.
So if they're all linked together in slot i, then the hash function applied to 49 has got to be equal to the hash function of 86 is equal to the hash function of 52, which equals what?
There's only one thing I haven't.
i.
Good.
Even if you don't understand it, your quizmanship should tell you.
He didn't mention i.
That's equal to i.
So the point is when I hash 49, the hash of 49 produces me some index in the table, say i, and everything that hashes to that same location is linked together into a list OK. Every record.
Any questions about what the mechanics of this.
I hope that most of you have seen this, seen hashing, basic hashing in 6.001, right?
They teach it in?
They used to teach it 6.001.
Yeah.
OK.
Some people are saying maybe.
They used to teach it.
Good.
So let's analyze this strategy.
The analysis.
We'll first do worst case.
So what happens in the worst case?
With hashing?
Yeah, raise your hand so that I could call on you.
Yeah.
Yeah, all hash keys, well all, all the keys in S. I happen to pick a set S where my hash function happens to map them all to the same value.
That would be bad.
So every key hashes to the same slot.
And so, therefore if that happens, then what I've essentially built is a fancy linked list for keeping this data structure.
All this stuff with the tables, the hashing, etc., irrelevant.
All that matters is that I have a long linked list.
And then how long does an access take?
How long does it take me to insert something or well, more importantly, to search for something.
Find out whether something's in there.
In the worst case.
Yeah, it takes order n time.
Because they're all just a link, we just have a linked list.
So access takes data n time if as we assume the size of S is equal to n. So from a worst case point of view, this doesn't look so attractive.
And we will see data structures that in worst case do very well for this problem.
But they don't do as good as the average case of hashing.
So let's analyze the average case.
In order to analyze the average case, I have to, whenever you have averages, whenever you have probability, you have to state your assumptions.
You have to say what is the assumption about the behavior of the system.
And it's very hard to do that because you don't know necessarily what the hash function is.
Well, let's imagine an ideal hash function.
What should an ideal hash function do?
Yeah, map the keys essentially at random to a slot.
Should really distribute them randomly.
So we call this the assumption -- -- of simple uniform hashing.
And what it means is that each key k in S is equally likely -- -- to be hashed to any slot in T and we're actually have to make an independence assumption.
Independent of where other records, other keys are hashed.
So we're going to make this assumption and includes n an independence assumption.
That if I have two keys the odds that they're hashed to the same place is therefore what?
What are the odds that two keys under this assumption are hashed to the same slot, if I have, say, m slots?
One over m. What are the odds that one key is hashed to slot 15?
One over m. Because they're being distributed, but the odds in particular two keys are hashed to the same slot, one over m. So let's define.
Is there a question?
No.
OK.
The load factor of a hash table with n keys at m slots to be alpha which is equal to n over m, which is also if you think about it, just the average number of keys per slot.
So alpha is the average number of keys per, we call it the load factor of the table.
OK. How many on average keys do I have?
So the expected, we'll look first at unsuccessful search time.
So by unsuccessful search, I mean I'm looking for something that's actually not in the table.
It's going to return nil.
I look for a key that's not in the table.
It's going to be what?
It's going to be order.
Well, I have to do a certain amount of work just to calculate the hash function and so forth.
It's going to be order at least one plus, then I have to search the list and on average how much of the list do I have to search?
What's the cost of searching that list?
On average.
If I'm searching at random.
If I'm searching for a key that's not in the table.
Whichever one it is, I got to search to the end of the list, right?
So what's the average cost over all the slots in the table?
Alpha.
Right?
Alpha.
That's the average length of a list.
So this is essentially the cost of doing the hash and then accessing the slot and that is just the cost of searching the list.
So the expected unsuccessful search time is proportional essentially to alpha and if alpha's bigger than one, it's order alpha.
If alpha's less than one, it's constant.
So when is the expected search time -- -- equal to order one?
So when is this order one?
Simple questions, by the way.
I only ask simple questions.
Some guys ask hard questions.
Yeah.
Or in terms first we'll get there in two steps, OK.
In terms of alpha, it's when?
When alpha is constant.
If alpha in particular is.
Alpha doesn't have to be constant.
It could be less than constant.
It's O of one, right.
OK, or equivalently, which is what you said, if n is O of m. OK, which is to say if the number of elements in the table is order, is upper bounded by a constant times n. Then the search cost is constant.
So a lot of people will tell you oh, a hash table runs in constant search time.
OK, that's actually wrong.
It depends upon the load factor of the hash table.
And people have made programming errors based on that misunderstanding of hash tables.
Because they have a hash table that's too small for the number of elements they're putting in there.
Doesn't help.
The number may in fact will grow with the, since this is one plus n over m, it actually grows with n. So unless you make sure that m keeps up with n, this doesn't stay constant.
Now it turns out for a successful search, it's also one plus alpha.
And for that you need to do a little bit more mathematics because you now have to condition on searching for the items in the table.
But it turns out it's also one plus alpha and that you can read about in the book.
And also, there's a more rigorous proof of this.
I sort of have glossed over the expectation stuff here, doing sort of a more intuitive proof.
So both of those things you should look for in the book.
So this is one reason why hashing is such a popular method, is it basically lets you represent a dynamic set with order one cost per operation, constant cost per operation, inserting, deleting and so forth, as long as the table that you're keeping is not much smaller than the number of items that you're putting in there.
And then all the operations end up being constant time.
But it depends upon, strongly upon this assumption of simple uniform hashing.
And so no matter what hash function you pick, I can always find a set of elements that are going to hash, that that hash function is going to hash badly.
I just could generate a whole bunch of them and look to see where the hash function takes them and in the end pick a whole bunch that hash to the same place.
We're actually going to see a way of countering that, but in practice people understand that most programs that need to use things aren't really reverse engineering the hash function.
And so, there's some very simple hash functions that seem to work fairly well in practice.
So in choosing a hash function -- -- we would like it to distribute -- keys uniformly into slots and we also would like that regularity in the key distributions -- -- should not affect uniformity.
For example, a regularity that you often see is that all the keys that are being inserted are even numbers.
Somebody happens to have that property of his data, that they're only inserting even numbers.
In fact, on many machines, since they use byte pointers, if they're sorting things that are for example, indexes to arrays or something like that, in fact, they're numbers that are typically divisible by four.
Or by eight.
So you don't want regularity in the key distribution to affect the fact that you're distributing slots.
So probably the most popular method that's used just for a quick hash function is what's called the division method.
And the idea here is that you simply let h of k for a key equal k modulo m, where m is the number of slots in your table.
And this works reasonably well in practice, but you want to be careful about your choice of modulus.
In other words, it turns out it doesn't work well for every possible size of table you might want to pick.
Fortunately when you're building hash tables, you don't usually care about the specific size of the table.
If you pick it around some size, that's probably fine because it's not going to affect their performance.
So there's no need to pick a specific value.
In particular, you don't want to pick -- -- m with a small divisor -- -- and let me illustrate why that's a bad idea for this particular hash function.
I should have said small divisor d. So for example -- -- if D is two, in other words m is an even number, and it turns out that we have the situation I just mentioned, all keys are even, what happens to my usage of the hash table?
So I have an even slot, even number of slots, and all the keys that the user of the hash table chooses to pick happen to be even numbers, what's going to happen in terms of my use of the hash table?
Well, in the worst case, they are always all going to point in the same slot no matter what hash function I pick.
But here, let's say that, in fact, my hash function does do a pretty good job of distributing, but I have this property.
What's a property that's going to have no matter what set of keys I pick that satisfies this property?
What's going to happen to the hash table?
So, I have even number, mod an even number.
What does that say about the hash function?
It's even, right?
I have an even number mod.
It's even.
So, what's going to happen to my use of the table?
Yeah, you're never going to hash anything to an odd-numbered slot.
You wasted half your slots.
It doesn't matter what the key distribution is.
OK, as long as they're all even, OK, that means the odds slots are never used.
OK, an extreme example, here's another example, imagine that m is equal to two to the r. In other words, all its factors are small divisors, OK?
In that case, if I think about taking k mod n, OK, the hash doesn't even depend on all the bits of k, OK?
So, for example, suppose I had one..., and r equals six, OK, so m is two to the sixth.
So, I take this binary number, mod two to the sixth, what's the hash value?
If I take something mod a power of two, what does it do?
So, I hash this function.
This is k, OK, in binary.
And I take it mod two to the sixth.
Well, if I took it mod two, what's the answer?
What's this number mod two?
Zero, right.
OK, what's this number mod four?
One zero.
What is it mod two to the sixth?
Yeah, it's just these last six bits.
This is H of k. OK, when you take something mod a power of two, all you're doing is taking its low order bits.
OK, mod two to the r, you are taking its r low order bits.
So, the hash function doesn't even depend on what's up here.
So, that's a pretty bad situation because generally you would like a very common regularity that you'll see in data is that all the low order bits are the same, and all the high order bits differ, or vice versa.
So, this particular is not a very good one.
So, good heuristics for this is to pick m to be a prime, not too close to a power of two or ten because those are the two common bases that you see regularity in the world.
A prime is sometimes inconvenient, however.
But generally, it's fairly easy to find primes.
And there's a lot of nice theorems about primes.
So, generally what you do, if you're just coding up something and you know what it is, you can pick a prime out of a textbook or look it up on the web or write a little program, or whatever, and pick a prime.
Not too close to a power of two or ten, and it will probably work pretty well.
It will probably work pretty well.
So, this is a very popular method, the division method.
OK, but the next method we are going to see is actually usually superior.
The reason people do this is because they can write in-line in their code.
OK, but it's not usually the best method.
And the reason is because division, one of the reasons is division tends to take a lot of cycles to compute on most computers compared with multiplication or addition.
OK, in fact, it's usually done with taking several multiplications.
So, the next method is actually generally better, but none of the hash function methods that we are talking about today are, in some sense, provably good hash functions.
OK, so for the multiplication method, the nice thing about it is just essentially requires multiplication to do.
And, for that is, also, we are going to assume that the number of slots is a power of two which is also often very convenient.
OK, and for this, we're going to assume that the computer has w bit words.
So, it would be convenient on a computer with 32 bits, or 64 bits, for example.
OK, this would be very convenient.
So, the hash function is the following.
h of k is equal to A times k mod, two to the w, right shifted by w minus r. OK, so the key part of this is A, which has chosen to be an odd integer in the range between two to the w minus one and two to the w. OK, so it's an odd integer that the full width of the computer word.
OK, and what you do is multiply it by whatever your key is, by this funny integer.
And, then take it mod two to the w. And then, you take the result and right shift it by this fixed amount, w minus r. So, this is a bit wise right shift.
OK, so let's look at what this does.
But first, let me just give you a couple of tips on how you pick, or what you don't pick for A.
So, you don't pick A too close to a power of two.
And, it's generally a pretty fast method because multiplication mod two to the w is faster than division.
And the other thing is that a right shift is fast, especially because this is a known shift.
OK, you know it before you are computing the hash function.
Both w and r are known in advance.
So, the compiler can often do tricks there to make it go even faster.
So, let's do an example to understand how this hash function works.
So, we will have, in this case, a number of slots will be eight, which is two to the three.
And, we'll have a bizarre word size of seven bits.
Anybody know any seven bit computers out there?
OK, well, here's one.
So, A is our fixed value that's used for hashing all our keys.
And, in this case, let's say it's 1011001.
So, that's A.
And, I take in some value for k that I'm going to multiply.
So, k is going to be 1101011.
So, that's my k. And, I multiply them.
What I multiply two, each of these is the full word width.
You can view it as the full word width of the machine, in this case, seven bits.
So, in general, this would be like a 32 bit number, and my key, I'd be multiplying two 32 bit numbers, for example.
OK, and so, when I multiply that out, I get a 2w bit answer.
So, when you multiply two w bit numbers, you get a 2w bit answer.
In this case, it happens to be that number, OK?
So, that's the product part, OK?
And then we take it mod two to the w. Well, what mod two to the w says is that I'm just taking, ignoring the high order bits of this product.
So, all of these are ignored, because, remember that if I take something, mod, a power of two, that's just the low order bits.
So, I just get these low order bits as being the mod.
And then, the right shift operation, and that's good also, by the way, because a lot of machines, when I multiply two 32 bit numbers, they'll have an instruction that gives you just the 32 lower bits.
And, it's usually an instruction that's faster than the instruction that gives you the full 64 bit answer.
OK, so, that's very convenient.
And, the second thing is, then, that I want just the, in this case, three bits that are the high order bits of this word.
So, this ends up being my H of k. And these end up getting removed by right shifting this word over.
So, you just right shift that in, zeros come in, in a high order bit, and you end up getting that value of H of k. OK, so to understand what's going on here, why this is a pretty good method, or what's happening with it, you can imagine that one way to think about it is to think of A as being a binary fraction.
So, imagine that the decimal point is here, sorry, the binary point, OK, the radix point is here.
Then when I multiply things, I'm just taking, the binary point ends up being there.
OK, so if you just imagine that conceptually, we don't have to actually put this into the hardware because we just do what the hardware does.
But, I can imagine that it's there, and that it's here.
And so, what I'm really taking is the fractional part of this product if I treat A as a fraction of a number.
So, we can certainly look at that as sort of a modular wheel.
So, here I have a wheel where this is going to be, that I'm going to divide into eight parts, OK, where this point is zero.
And then, I go around, and this point is then one.
And, I go around, and this point is two, and so forth, so that all the integers, if I wrap it around this unit wheel, all the integers lined up at the zero point here, OK?
And then, we can divide this into the fractional pieces.
So, that's essentially the zero point.
This is the one eighth, because we are dividing into eight, two, three, four, five, six, seven.
So, if I have one times A, in this case, I'm basically saying, well, one times A, if I multiply, is basically going around to about there, five and a half I think, right, because one times A is about five and a half, OK, or five halves of 5.5 eighths, essentially.
So, it takes me about to there.
That's A.
And, if I do 2^A, that continues around, and takes me up to about, where?
About, a little past three, about to there.
So, that's 2^A.
OK, and 3^A takes me, then, around to somewhere like about there.
So, each time I add another A, it's taking me another A's distance around.
And, the idea is that if A is, for example, odd, and it's not too close to a power of two, then what's happening is sort of throwing it into another slot on a different thing.
So, if I now go around, if I have k being very big, then k times A is going around k times.
Where does it end up?
It's like spinning a wheel of fortune or something.
OK, it ends somewhere.
OK, and so that's basically the notion.
That's basically the notion, that it's going to end up in some place.
So, you're basically looking at, where does ka end up?
Well, it sort of whirls around, and ends up at some point.
OK, and so that's why that tends to be a fairly good one.
But, these are only heuristic methods for hashing, because for any hash function, you can always find a set of keys that's going to make it operate badly.
So, the question is, well, what do you use in practice?
OK, the second topic that I want to tie it, so, we talked about resolving collisions by chaining.
OK, there's another way of resolving collisions, which is often useful, which is resolving collisions by what's called open addressing.
OK, and the idea is, in this method, is we have no storage for links.
So, when I result by chaining, I'd need an extra linked field in each record in order to be able to do that.
Now, that's not necessarily a big overhead, but for some applications, I don't want to have to touch those records at all.
OK, and for those, open addressing is a useful way to resolve collisions.
So, the idea is, with open addressing, is if I hash to a given slot, and the slot is full, OK, what I do is I just hash again with a different hash function, with my second hash function.
I check that slot.
OK, if that slot is full, OK, then I hash again.
And, I keep this probe sequence, which hopefully is a permutation so that I'm not going back and checking things that I've already checked until I find a place to put it.
And, if I got a good probe sequence that I will hopefully, then, find a place to put it fairly quickly.
OK, and then to search, I just follow the same probe sequence.
So, the idea, here, is we probe the table systematically until an empty slot is found, OK?
And so, we can extend that by looking as if the sequence of hash functions were, in fact, a hash function that took two arguments: a key and a probe step.
In other words, is it the zero of one our first one?
It's the second one, etc.
So, it takes two arguments.
So, H is then going to map our universe of keys cross, our probe number into a slot.
So, this is the universe of keys.
This is the probe number.
And, this is going to be the slot.
Now, as I mentioned, the probe sequence should be permutation.
In other words, it should just be the numbers from zero to n minus one in some fairly random order.
OK, it should just be rearranged.
And the other thing about open addressing is that you don't have to worry about n chaining is that the table may actually fill up.
So, you have to have that the number of elements in the table is less than or equal to the table size, the number of slots because the table may fill up.
And, if it's full, you're going to probe everywhere.
You are never going to get a place to put it.
And, the final thing is that in this type of scheme, deletion is difficult.
It's not impossible.
There are schemes for doing deletion.
But, it's basically hard because the danger is that you remove a key out of the table, and now, somebody who's doing a probe sequence who would have hit that key and gone to find his element now finds that it's an empty slot.
And he says, oh, the key I am looking for probably isn't there.
OK, so you have that issue to deal with.
So, you can delete things but keep them marked, and there's all kinds of schemes that people have for doing deletion.
But it's difficult.
It's messy compared to chaining, where you can just remove the element out of the chain.
So, let's do an example -- -- just so that we make sure we're on the same page.
So, we'll insert a key.
k is 496.
OK, so here's my table.
And, I've got some values in it, 586, 133, 204, 481, etc.
So, the table looks like that; the other places are empty.
So, on my zero step, I probe H of 496, zero.
OK, and let's say that takes me to the slot where there's 204.
And so, I say, oh, there's something there.
I have to probe again.
So then, I probe H of 496, one.
Maybe that maps me there, and I discover, oh, there's something there.
So, now, I probe H of 496, two.
Maybe that takes me to there.
It's empty.
So, if I'm doing a search, I report nil.
If I'm doing in the insert, I put it there.
And then, if I'm looking for that value, if I put it there, then when I'm looking, I go through exactly the same sequence.
I'll find these things are busy, and then, eventually, I'll come up and discover the value.
OK, and there are various heuristics that people use, as well, like keeping track of the longest probe sequence because there's no point in probing beyond the largest number of probes that need to be done globally to do an insertion.
OK, so if it took me 5, 5 is the maximum number of probes I ever did for an insertion.
A search never has to look more than five, OK, and so sometimes hash tables will keep that auxiliary value so that it can quit rather than continuing to probe until it doesn't find something.
OK, so, search is the same probe sequence.
And, if it's successful, it finds the record.
And, if it's unsuccessful, you find a nil.
OK, so it's pretty straightforward.
So, once again, as with just hash functions to begin with, there are a lot of ideas about how you should form a probe sequence, ways of doing this effectively.
OK, so the simplest one is called linear probing, and what you do there is you have H of k comma i.
You just make that be some H prime of k, zero plus i mod m. Sorry, no prime there.
OK, so what happens is, so, the idea here is that all you are doing on the I'th probe is, on the zero'th probe, you look at H of k zero.
On probe one, you just look at the slot after that.
Probe two, you look at the slot after that.
So, you're just simply, rather than sort of jumping around like this, you probe there and then just find the next one that will fit in.
OK, so you just scan down mod m. So, if you hit the bottom, you go to the top.
OK, so the I'th one, so that's fairly easy to do because you don't have to recomputed a full hash function each time.
All you have to do is add one each time you go because the difference between this and the previous one is just one.
OK, so you just go down.
Now, the problem with that is that you get a phenomenon of clustering.
If you get a few things in a given area, then suddenly everything, everybody has to keep searching to the end of those things.
OK, so that turns out not to be one of the better schemes, although it's not bad if you just need to do something quick and dirty.
So, it suffers from primary clustering, where regions of the hash table get very full.
And then, anything that hashes into that region has to look through all the stuff that's there.
OK, so: long runs of filled slots.
OK, there's also things like quadratic clustering, where you basically make this be, instead of adding one each time, you add i each time.
OK, but probably the most effective popular scheme is what's called double hashing.
And, you can do statistical studies.
People have done statistical studies to show that this is a good scheme, OK, where you let H of k, i, let me do it below here because I have for them.
So, H of k, i is equal to an H_1 of k plus i times H_2 of k. So, you have two hash functions on m. You have two hash functions, H_1 of k and H_2 of k. OK, so you compute the two hash functions, and what you do is you start by just using H_1 of k for the zero probe, because here, i, then, will be zero.
OK. Then, for the probe number one, OK, you just add H_2 of k. For probe number two, you just add that hash function amount again.
You just keep adding H_2 of k for each successive probe you make.
So, it's fairly easy; you compute two hash functions up front, OK, or you can delay the second one, in case.
But basically, you compute two up front, and then you just keep adding the second one in.
You start at the location of the first one, and keep adding the second one, mod m, to determine your probe sequences.
So, this is an excellent method.
OK, it does a fine job, and you usually pick m to be a power of two here, OK, so that you're using, usually people use this with the multiplication method, for example, so that m is a power of two, and H_2 of k you force to be odd.
OK, so we don't use and even value there, because otherwise for any particular key, you'd be skipping over.
Once again, you would have the problem that everything could be even, or everything could be odd as you're going through.
But, if you make H_2 of k odd, and m is a power of two, you are guaranteed to hit every slot.
OK, so let's analyze this scheme.
This turns out to be a pretty interesting scheme to analyze.
It's got some nice math in it.
So, once again, in the worst case, hashing is lousy.
So, we're going to analyze average case.
OK, and for this, we need a little bit stronger assumption than for chaining.
And, we call it the assumption of uniform hashing, which says that each key is equally likely, OK, to have any one of the m factorial permutations as its probe sequence, independent of other keys.
And, the theorem we're going to prove is that the expected number of probes is, at most, one over one minus alpha if alpha is less than one, OK, that is, if the number of keys in the table is less than number of slots.
OK, so we're going to show that the number of probes is one over one minus alpha.
So, alpha is the load factor, and of course, for open addressing, we want the load factor to be less than one because if we have more keys than slots, open addressing simply doesn't work, OK, because you've got to find a place for every key in the table.
So, the proof, we'll look at an unsuccessful search, OK?
So, the first thing is that one probe is always necessary.
OK, so if I have n over m, sorry, if I have n items stored in m slots, what's the probability that when I do that probe I get a collision with something that's already in the table?
What's the probability that I get a collision?
Yeah?
Yeah, n over m, right?
So, with probability, n over m, we have a collision because my table has got n things in there.
I'm hashing, at random, to one of them.
OK, so, what are the odds I hit something, n over m?
And then, a second probe is necessary.
OK, so then, I do a second probe.
And, with what probability on the second probe do I get a collision?
So, we're going to make the assumption of uniform hashing.
Each key is equally likely to have any one of the m factorial permutations as its probe sequence.
So, what is the probability that on the second probe, OK, I get a collision?
Yeah?
If it's a permutation, you're not, right?
Something like that.
What is it exactly?
So, that's the question.
OK, so you are not going to hit the same slot because it's going to be a permutation.
Yeah?
That's exactly right.
n minus one over m minus one because I'm now, I've essentially eliminated that slot that I hit the first time.
And so, I have, now, and there was a key there.
So, now I'm essentially looking, at random, into the remaining n minus one slots where there are aggregately n minus one keys in those slots.
OK, everybody got that?
OK, so with that probability, I get a collision.
That means that I need a third probe necessary, OK?
And, we keep going on.
OK, so what is it going to be the next time?
Yeah, it's going to be n minus two over m minus two.
So, let's note, OK, that n minus i over m minus i is less than n over m, which equals alpha, OK?
So, n minus i over m minus i is less than n over m. And, the way you can sort of reason that is that if n is less than m, I'm subtracting a larger fraction of n when I subtract i than I am subtracting a fraction of m. OK, so therefore, n minus i over m minus i is going to be less than n over m. OK, so, or you can do the algebra.
I think it's always helpful when you do algebra to sort of think about it sort of quantitatively as well, you know, qualitatively what's going on.
So, the expected number of probes is, then, going to be equal to, it's going to be equal to because we're going to need some space, well, we have one which is forced because we've got to do one probe, plus with probability n over m, I have to do another probe plus with probability of n over m minus one I have to do another probe up until I do one plus one over m minus n. OK, so each one is cascading what's happened.
In the book, there is a more rigorous proof of this using indicator random variables.
I'm going to give you the short version.
OK, so basically, this is my first probe.
With probability n over m, I had to do a second one.
And, the result of that is that with probability n minus one over m minus one, I have to do another.
And, with probability n over two minus m over two, I have to do another, and so forth.
So, that's how many probes I'm going to end up doing.
So, this is less than or equal to one plus alpha.
There's one plus alpha times one plus alpha times one plus alpha, OK, just using the fact that I had here.
OK, and that is less than or equal to one plus I just multiply through here.
Alpha plus alpha squared plus alpha cubed plus k. I can just take that out to infinity.
It's going to bound this.
OK, does everybody see the math there?
OK, and that is just the sum, I, equals zero to infinity, alpha to the I, which is equal to one over one minus alpha using your familiar geometric series bound.
OK, and there's also, in the textbook, an analysis of the successful search, which, once again, is a little bit more technical because you have to worry about what the distribution is that you happen to have in the table when you are searching for something that's already in the table.
But, it turns out it's also bounded by one over one minus alpha.
So, let's just look to see what that means.
So, if alpha is less than one is a constant, it implies that it takes order one probes.
OK, so if alpha is a constant, it takes order one probes.
OK, but it's helpful to understand what's happening with the constant.
So, for example, if the table is 50% full, so alpha is a half, what's the expected number of probes by this analysis?
Two, because one over one minus a half is two.
If I let the table fill up to 90%, how many probes do I need on average?
Ten.
So, you can see that as you fill up the table, the cost is going dramatically, OK?
And so, typically, you don't let the table get too full.
OK, you don't want to be pushing 99.9% utilization.
Oh, I got this great hash table that's got full utilization.
It's like, yeah, and it's slow.
It's really, really slow, OK, because as alpha approaches one, the time is approaching and essentially m, or n. Good.
So, next time, we are going to address head-on in what was one of the most, I think, interesting ideas in algorithms.
We are going to talk about how you solve this problem that no matter what hash function you pick, there's a bad set of keys.
OK, so next time we're going to show that there are ways of confronting that problem, very clever ways.
And we use a lot of math for it so will be a really fun lecture.
Hashing.
Today we're going to do some amazing stuff with hashing.
And, really, this is such neat stuff, it's amazing.
We're going to start by addressing a fundamental weakness of hashing.
And that is that for any choice of hash function There exists a bad set of keys that all hash to the same slot.
OK.
So you pick a hash function.
We looked at some that seem to work well in practice, that are easy to put into your code.
But whichever one you pick, there's always some bad set of keys.
So you can imagine, just to drive this point home a little bit.
Imagine that you're building a compiler for a customer and you have a symbol table in your compiler and one of the things that the customer is demanding is that compilations go fast.
They don't want to sit around waiting for compilations.
And you have a competitor who's also building a compiler and they're going to test the compiler, both of your compilers and sort of have a run-off.
And one of the things in the test that they're going to allow you to do is not only will the customer run his own benchmarks, but he'll let you make up benchmarks for the other program, for your competitor.
And your competitor gets to make up benchmarks for you.
So and not only that, but you're actually sharing code.
So you get to look at what the competitor is actually doing and what hash function they're actually using.
So it's pretty clear that in this circumstance, you have an adversary who is going to look at whatever hash function you have and figure out OK, what's a set of variable names and so forth that are going to all hash to the same slot so that essentially you're just chasing through a linked list whenever it comes to looking something up.
Slowing down your program enormously compared to if in fact they got distributed nicely across the hash table which is, what after all, you have a hash table in there to do in the first place.
And so the question is, how do you defeat this adversary?
And the answer is one word.
One word.
How do you achieve?
How do you defeat any adversary in this class?
Randomness.
OK. Randomness.
OK. You make it so that he can't guess.
And the idea is that you choose a hash function at random.
Independent.
So he can look at the code, but when it actually runs, it's going to use a random hash function that he has no way of predicting what the hash function is that will actually be used.
OK.
So that's the game and that way he can provide an input, but he can't provide an input that's guaranteed to force you to run slowly.
You might get unlucky in your choice of hash function, but it's not going to be because of the adversary.
So the idea is to choose a hash function -- -- at random, independently from the keys that you're, that are going to be fed to it.
So even if your adversary can see your code, he can't tell which hash function is going to be actually used at run time.
Doesn't get to see the output of the random numbers.
And so it turns out you can make this scheme work and the name of the scheme is universal hashing, OK, is one way of making this scheme work.
So let's do some math.
So let U be a universe of keys.
And let H be a finite collection -- -- of hash functions -- -- mapping U to what are going to be the slots in our hash table.
OK.
So we just have H as some finite collection.
We say that H is universal -- -- if for all pairs of the keys, distinct keys -- -- so the keys are distinct, the following is true.
So if the set of keys, if for any pair of keys I pick, the number of hash functions that hash those two keys to the same place is a one over m fraction of the total set of keys.
So let m just, so to view that, another way of viewing that is if H is chosen randomly -- -- from the set of keys H, the probability of collision between x and y is what?
What's the probability if the fraction of hash functions, OK, if the number of hash functions is H over m, what's the probability of a collision between x and y?
If I pick a hash function at random.
So I pick a hash function at random, what's the odds they collide?
One over m. Now let's draw a picture for that, help people see that that's in fact the case.
So imagine this is our set of all hash functions.
OK. And then if I pick a particular x and y, let's say that this is the set of hash functions such that H of x is equal to H of y.
And so what we're saying is that the cardinality of that set is one over m times the cardinality of H. So if I throw a dart and pick one hash function at random, the odds are one in m that the hash function falls into this particular set.
And of course, this has to be true of every x and y that I can pick.
Of course, it will be a different set, a different x and y will somehow map the hash functions differently, but the odds that for any x and y that I pick, the odds that if I have a random hash function, it hashes it to the same place, is one over m. Now this is a little bit hard sometimes for people to get their head around because we're used to thinking of perhaps picking keys at random or something.
OK, that's not what's going on here.
We're picking hash functions at random.
So our probability space is defined over the hash functions, not over the keys.
And this has to be true now for any particular two keys that I pick that are distinct.
That the places that they hash, this set of hash functions, I mean this is like a marvelous property if you think about it.
OK, that you can actually find ones where no matter what two elements I pick, the odds are exactly one in m that a random hash function from this set is going to hash them to the same place.
So very neat.
Very, very neat property and we'll see the mathematics associated with this is very cool.
So our theorem is that if we choose h randomly from the set of hash functions H, and then we suppose we're hashing n keys into m slots in Table T -- -- then for given key x -- -- the expected number of collisions with x -- -- is less than n over m. And who remembers what we call n over m?
Alpha, which is the, what's the term that we use there?
Load factor.
The load factor of the table.
OK, load factor alpha.
So the average number of keys per slot is the load factor of the table.
So we're saying, so what is this theorem saying?
It's saying that in fact, if we have one of these universal sets of hash functions, then things perform exactly the way we want them to.
Things get distributed evenly.
The number of things that are going to collide with any particular key that I pick is going to be n over m. So that's a really good property to have.
Now I haven't shown you, the construction of U is going, sorry, of the set of hash functions H, that that construction will take us a little bit of effort.
But first I want to show you why this is such a great property.
Basically it's this theorem.
So let's prove this theorem.
So any questions about what the statement of the theorem is?
So we're going to go actually kind of fast today.
We've got a lot of good stuff today.
So I want to make sure people are onboard as we go through.
So if there are any questions, make sure, you know, statement of theorem of whatever, best to get them out early so that you're not confused later on when the going gets a little more exciting.
OK?
OK, good.
So to prove this, let's let C sub x be the random variable denoting the total number of collisions -- -- of keys in T with x.
So this is a total number and one of the techniques that you use a lot in probabilistic analysis of randomized algorithms is recognizing that C of x is in fact a sum of indicator random variables.
If you can decompose things into indicator random variables, the analysis goes much more easily than if you're left with aggregate variables.
So here we're going to let our indicator random variable be little c of x., which is going to be one if h of x equals h of y and 0 otherwise.
And so we can note two things.
First, what is the expectation of C of x.. OK, if I have a process which is picking a hash function at random, what's the expectation of C of x.?
One over m. Because that's basically this definition here.
Now in other words I pick a hash function at random, what's the odds that the hash is the same?
It's one over m. And then the other thing is, and the reason we pick this thing is that I can express capital C sub x, the random variable denoting the total number of collisions as being just the sum over all the keys in the table except x of C of x..
So for each one that would cause me a collision, with x, I add one and if it wouldn't cause me a collision, I add 0.
And that adds up all of the collisions that I would have in the table with x.
Is there any questions so far?
Because this is the set-up.
The set-up in most of these things, the set-up is where most students make mistakes and most practicing researchers make mistakes as well, let me tell you.
And then once you get the set-up right, then working out the math is fine, but it's often that set-up of how do you actually translate the situation into the math.
That's the hard part.
Once you get that right, well, then, algebra, we can all do algebra.
Of course, we can also all make mistakes doing algebra, but at least those mistakes are much more easy to check than the one that does the translation.
So I want to make sure people are sort of understanding of how that's set up.
So now we just have to use our math skills.
So the expectation then of the number of collisions is the expectation of C sub x and that's just the expectation of just plugging the sum of y and T minus the element x of c_xy.
So that's just definition.
And that's equal to the sum of y and T minus x of expectation of c_xy.
So why is that?
Yeah, that's linearity.
Linearity of expectation, doesn't require independence.
It's true of all random variables.
And that's equal to, and now the math gets easier.
So what is that?
One over m. That makes the summation easy to evaluate.
That's just n minus one over m. So fairly simple analysis and shows you why we would love to have one of these sets of universal hash functions because if you have them, then they behave exactly the way you would want it to behave.
And you defeat your adversary by just picking up the hash function at random.
There's nothing he can do.
Or she.
OK, any questions about that proof?
OK, now we get into the fun math.
Constructing one of these babies.
OK.
This is not the only construction.
This is a construction of a classic universal hash function.
And there are other constructions in the literature and I think there's one on the practice quiz.
So let's see.
So this one works when m is prime.
So it works when the set of slots is a prime number.
Number of slots is a prime number.
So the idea here is we're going to decompose any key k in our universe into r plus 1 digits.
So k, we're going to look at as being a k 0, k one, k_r where 0 is less than or equal to k sub I, is less than or equal to m minus one.
So the idea is in some sense we're looking at what the representation would be of k base m. So if it were base two, it would be just one bit at a time.
These would just be the bits.
I'm not going to do base two.
We're going to do base min general and so each of these represents one of the digits.
And the way I've done it is I've done low order digit first.
It actually doesn't matter.
We're not actually going to care really about what the order is, but basically we're just looking at busting it into a twofold represented by each of those digits.
So one algorithm for computing this out of k is take the remainder mod m. That's the low order one.
OK, take what's left.
Take the remainder of that mod m. Take whatever's left, etc.
So you're familiar with the conversion to a base representation.
That's exactly how we're getting this representation.
So we treat, this is just a question of taking the data that we've got and treating it as an r plus one base m number.
And now we invoke our randomized strategy.
The randomized strategy is going to be able to have a class of hash functions that's dependent essentially on random numbers.
And the random numbers we're going to pick is we're going to pick an a at random -- -- which we're also going to look at as a base mnumber.
For each a_i is chosen randomly -- -- from -- -- 0 to m minus one.
So one of our, it's a random if you will, it's a random base mdigit.
Random base m digit.
So each one of these is picked at random.
And for each one we, possible value of A, we're going to get a different hash function.
So we're going to index our hash functions by this random number.
So this is where the randomness is going to come in.
Everybody with me?
And here's the hash function.
So what we do is we dot product this vector with this vector and take the result, mod m. So each digit of k of our key gets multiplied by a random other digit.
We add all those up and we take that mod m. So that's a dot product operator.
And this is what we're going to show is universal, that this set of h sub a, where I look over that whole set.
So one of the things we need to know is how big is the set of hash functions here.
So how big is this set of hash functions?
How many different hash functions do I have in this set?
It's basic 6.042 material.
It's basically how many vectors of length r plus one where each element of the vector is a number of 0 to m minus one, has m different values.
So how many?
m minus one to the r. No.
Close.
It's up there.
It's a big number.
m to the r plus one.
Good.
It's m, so the size of H is equal to m to the r plus one.
So we're going to want to remember that.
OK, so let's just understand why that is.
I have m choices for the first value of A. m for the second, etc.
m for the r th.
And since there are plus one things here, for each choice here, I have this many same number of choices here, so it's a product.
OK, so this is the product rule in counting.
So if you haven't reviewed your 6.042 notes for counting, this is going to be a good idea to go back and review that because we're doing stuff of that nature.
This is just the product rule.
Good.
So then the theorem we want to prove is that H is universal.
And this is going to involve a little bit of number theory, so it gets kind of interesting.
And it's a non-trivial proof, so this is where if there's any questions as I'm going along, please ask because the argument is not as simple as other arguments we've seen so far.
OK, not the ones we've seen so far have been simple, but this is definitely a more involved mathematical argument.
So here's a proof.
So let's let, so we have two keys.
What are we trying to show if it's universal, that if I pick any two keys, the number of hash functions for which they hash to the same thing is the size of set of hash functions divided by m. OK, so I'm going to look at two keys.
So let's pick two keys arbitrarily.
So x, and we'll decompose it into our base r representation and y, y_0, y_1 -- So these are two distinct keys.
So if these are two distinct keys, so they're different, then this base representation has the property that they've got to differ somewhere.
Right?
OK, they differ in at least one digit.
OK, and this is where most people get lost because I'm going to make a simplification.
They could differ in any one of these digits.
I'm going to say they differ in position 0 because it doesn't matter which one I do, the math is the same, but it'll make it so that if I pick some said they differ in some position i, I would have to be taking summations as you'll see over the elements that are not i, and that's complicated.
If I do it in position 0, then I can just sum for the rest of them.
So the math is going to be identical if I were to do it for any position because it's symmetric.
All the digits are symmetric.
So let's say they differ in position 0, but the same argument is going to be true if they differed in some other position.
So let's say, so we're saying without loss of generality.
So that's without loss of generality.
Position 0.
Because all the positions are symmetric here.
And so, now we need to ask the question for how many -- -- hash functions in our universal, purportedly universal set do x and y collide?
OK, we've got to count them up.
So how often do they collide?
This is where we're going to pull out some heavy duty number theory.
So we must have, if they collide -- -- that h sub a of x is equal to h sub a of y.
That's what it means for them to collide.
So that implies that the sum of i equal 0 to r of a sub i x sub i is equal to the sum of i equals 0 to r of a sub i y sub i mod m. Actually this is congruent mod m. So congruence for those people who haven't seen much number theory, is basically the way of essentially, rather than having to say mod everywhere in here and mod everywhere in here, we just at the end say OK, do a mod at the end.
Everything is being done mod, module m. And then typically we use a congruence sign.
OK, there's a more mathematical definition but this will work for us engineers.
OK, so everybody with me so far?
This is just applying the definition.
So that implies that the sum of i equals 0 to r of a i x i minus y i is congruent to zeros mod m. OK, just threw it on the other side and applied the distributive law.
Now what I'm going to do is pull out the 0-th position because that's the one that I care about.
And this is where it saves me on the math, compared to if I didn't say that it was 0.
I'd have to pull out x_i.
It wouldn't matter, but it just would make the math a little bit cruftier OK, so now we've just pulled out one term.
That implies that a_0 x_0 minus y_0 is congruent to minus -- -- mod m. Now remember that when I have a minus number mod m, I just map it into whatever, into that range from 0 to m minus one.
So for example, minus five mod seven is two.
So if any of these things are negative, we simply translate them into by adding multiples of mbecause adding multiples of m doesn't affect the congruence.
OK. And now for the next step, we need to use a number theory fact.
So let's pull out our number theory -- -- textbook and take a little digression So this comes from the theory of finite fields.
So for people who are knowledgeable, that's where you're plugging your knowledge in.
If you're not knowledgeable, this is a great area of math to learn about.
So here's the fact.
So let m be prime.
Then for any z, little z element of z sub m, and z sub m is the integers mod m. So this is essentially numbers from 0 to m minus one with all the operations, times, minus, plus, etc., defined on that such that if you end up outside of the range of 0 to m minus one, you re-normalize by subtracting or adding multiples of m to get back within the range from 0 to m minus one.
So it's the standard thing of just doing things module m. So for any z such that z is not congruent to 0, there exists a unique z inverse in z sub m, such that if I multiply z times the inverse, it produces something congruent to one mod m. So for any number it says, I can find another number that when multiplied by it gives me one.
So let's just do an example for m equals seven.
So here we have, we'll make a little table.
So z is not equal to 0, so I just write down the other numbers.
And let's figure out what z inverse is.
So what's the inverse of one?
What number when multiplied by one gives me one?
One.
Good.
How about two?
What number when I multiply it by two gives me one?
Four.
Because two times four is eight and eight is congruent to one mod seven.
So I've re-normalized it.
What about three?
Five.
Good.
Five.
Three times five is 15.
That's congruent to one mod seven because 15 divided by seven is two remainder of one.
So that's the key thing.
What about four?
Two.
Five?
Three.
And six.
Yeah.
Six.
Yeah, six it turns out.
OK, six times six is 36.
OK, mod seven.
Basically subtract off the 35, gives m one.
So people have observed some interesting facts that if one number's an inverse of another, then that other is an inverse of the one.
So that's actually one of these things that you prove when you do group theory and field theory and so forth.
There are all sorts of other great properties of this kind of math.
But the main thing is, and this turns out not to be true if m is not a prime.
So can somebody think of, imagine we're doing something mod 10.
Can somebody think of a number that doesn't have an inverse mod 10?
Yeah.
Two.
Another one is five.
OK, it turns out the divisors in fact actually, more generally, something that is not relatively prime, meaning that it has no common factors, the GCD is not one between that number and the modulus.
OK, those numbers do not have an inverse mod m. OK, but if it's prime, every number is relatively prime to the modulus.
And that's the property that we're taking advantage of.
So this is our fact and so, in this case what I'm after is I want to divide by x_0 minus y_0.
That's what I want to do at this point.
But I can't do that if x_0, first of all, if m isn't prime, I can't necessarily do that.
I might be able to, but I can't necessarily.
But if m is prime, I can definitely divide by x_0 minus y_0.
I can find that inverse.
And the other thing I have to do is make sure x_0 minus y_0 is not 0.
OK, it would be 0 if these two were equal, but our supposition was they weren't equal.
And once again, just bringing it back to the without loss of generality, if it were some other position that we were off, I would be doing exactly the same thing with that position.
So now we're going to be able to divide.
So we continue with our -- -- continue with our proof.
So since x_0 is not equal to y_0, there exists an inverse for x_0 minus y_0.
And that implies, just continue on from over there, that a_0 is congruent therefore to minus the sum of i equal one to r of a_i, x_i minus y_i times x_0 minus y_0 inverse.
So let's just go back to the beginning of our proof and see what we've derived.
If we're saying we have two distinct keys, and we've picked all of these a_i randomly, and we're saying that these two distinct keys hash to the same place.
If they hash to the same place, it says that a_0 essentially had to have a particular value as a function of the other a_i.
Because in other words, once I've picked each of these a_i from one to r, if I did them in that order, for example, then I don't have a choice for how I pick a_0 to make it collide.
Exactly one value allows it to collide, namely the value of a_0 given by this.
If I picked a different value of a_0, they wouldn't collide.
So let m write that down.
Thus, while you think about it So for any choice of these a_i, there's exactly one of the impossible choices of a_0 that cause a collision.
And for all the other choices I might make of a_0, there's n collision.
So essentially I don't have, if they're going to collide, I've reduced essentially the number of degrees of freedom of my randomness by a factor of m. So if I count up the number of h_a's that cause x and y to collide, that's equal to, well, there's m choices, just using the product rule again.
There's m choices for a_1 times m choices for a_2, up to m choices for a_r and then only one choice for a_0.
So this is choices for a_1, a_2, a_r and only one choice for a_0 if they're going to collide.
If they're not going to collide, I've got more choices for a_0.
But if I want them to collide, there's only one value I can pick, namely this value.
That's the only value for which I will pick.
And that's equal to m to the r, which is just the size of H divided by m. And that completes the proof.
So there are other universal constructions, but this is a particularly elegant one.
So the point is that I have m plus one, sorry, r plus one degrees of freedom where each degree of freedom I have m choices.
But if I want them to collide, once I've picked any of the, once I've picked r of those possible choices, the last one is forced if I want it to collide.
So therefore, the set of functions for which it collides is only one in m. A very slick construction.
Very slick.
OK. Everybody with me here?
Didn't lose too many people?
Yeah, question.
Well, part of it is, actually this is a quite common type of thing to be doing actually.
If you take a class, so we have follow on classes in cryptography and so forth, and this kind of thing of taking dot products, modulo m and also Galois fields which are particularly simple finite fields and things like that, people play with these all the time.
So Galois fields are like using exor's as your, same sort of thing as this except base two.
And so there's a lot of study of this sort of thing.
So people understand these kind of properties.
But yeah, it's like what's the algorithm for having a brilliant insight into algorithms?
It's like OK.
Wish I knew.
Then I'd just turn the crank.
[LAUGHTER] But if it were that easy, I wouldn't be standing up here today.
[LAUGHTER] Good.
OK, so now I want to take on another topic which is also I find, I think this is astounding.
It's just beautiful, beautiful mathematics and a big impact on your ability to build good hash functions.
Now I want to talk about another one topic, which is related, which is the topic of perfect hashing.
So everything we've done so far does expected time performance.
Hashing is good in the expected sense.
A perfect hashing addresses the following questions.
Suppose that I gave you a set of keys, and I said just build me a static table so I can look up whether the key is in the table with worst case time.
Good worst case time.
So I have a fixed set of keys.
They might be something like for example, the hundred most common or thousand most common words in English.
And when I get a word I want to check quickly in this table, is the word that I've got one of the most common words in English.
I would like to do that not with expected performance, but guaranteed worst case performance.
Is there a way of building it so that I can find this quickly?
So the problem is given n keys -- -- construct a static hash table.
In other words, no insertion and deletion.
We're just going to put the elements in there.
A size -- -- m equal Order n. So I don't want it to be a huge table.
I want it to be a table that is the size of my keys.
Table of size m equals Order n, such that search takes O(1) time in the worst case.
So there's no place in the table where I'm going to have, I know in the average case, that's not hard to do.
But in the worst case, I want to make sure that there's no particular spot where the number of keys piles up to be a large number.
OK, in no spot should that happen.
Every single search I do should take Order one time.
There shouldn't be any statistical variation in terms of how long it takes me to get something.
Does everybody understand what the puzzle is?
So this is a great, because this actually ends up having a lot of uses.
You know, you want to build a table for something and you know what the values are that you're going look up in it.
But you don't want to spend a lot of space on it and so forth.
So the idea here is actually going to be to use a two-level scheme.
So the idea is we're going to use a two-level scheme with universal hashing at both levels.
So the idea is we're going to hash, we're going to have a hash table, we're going to hash into slots, but rather than using chaining, we're going to have another hash table there.
We're going to do a second hash into the second hash table.
And the idea is that we're going to do it in such a way that we have no collisions at level two.
So we may have collisions at level one.
We'll take anything that collides at level one and put them into a hash table and then our second level hash table, but that hash table, no collisions.
Boom.
We're just going to hash right in there.
And it'll just go boom to its thing.
So let's draw a picture of this to illustrate the scheme.
OK, so we have -- -- 0 one, let's say six, m minus one.
So here's our hash table.
And what we're going to do is we're going to use universal hashing at the first level, OK.
So we find a universal hash function.
We pick a hash function at random.
And what we'll do is we'll hash into that level.
And then what we'll do is we'll keep track of two things.
One is what the size of the hash table is at the next level.
So in this case, the size of the hash table will only use the number of slots.
There's going to be four.
And we're also going to keep a separate hash key for the second level.
So each slot will have its own hash function for the second level.
So for example, this one might have a key of 31 that is a random number.
The a's here.
a's up there.
There we go, a's up there.
So that's going to be the basis of my hash function, the key with which I'm going to hash.
This one say has 86.
And let's say that this, and then we have a pointer to the hash table.
This is say S_1.
And it's got four slots and we stored up 14 and 27.
And these two slots are empty.
And this one for example, had what?
Two nines.
So the idea here is that in this case if we look over all our top level hash function, which I'll just call H, has that H of 14 is equal to H of 27 is equal to one.
Because we're in slot one.
OK, so these two both hash to the same slot in the level one hash table.
This is level one.
And this is level two over here.
So level one hashing, 14 and 27 collided.
They went into the same slot here.
But at level two, they got hashed to different places and the hash function I use is going to be indexed by whatever the random numbers are that I chose and found for those and I'll show you how we find those.
We have then h of 31 of 14 is equal to one h of 31 of 27 is equal to two.
For level two.
So I go, hash in here, find the, use this as the basis of my hash function to hash into whatever table I've got here.
And so, if there are no, if I can guarantee that there are no collisions at level two, this is going to cost me Order one time in the worst case to look something up.
How do I look it up?
Take the value.
I apply h to it.
That takes me to some slot.
Then I look to see what the key is for this hash function.
I apply that hash function and that takes me to another slot.
Then I go there.
And that took me basically two applications of hash functions plus some look-up, plus who knows what minor amount of bookkeeping.
So the reason we're going to have no collisions at this level is the following.
If they're n sub i items that hash to a level one slot i, then we're going to use m sub i, which is equal to n sub i squared slots in the level two hash table.
OK, so I should have mentioned here this is going to be m sub i, the size of the hash table and this is going to be my a sub i essentially.
So I'm going to use, so basically I'm going to hash n sub i things into n sub i squared locations here.
So this is going to be incredibly sparse.
OK, it's going to be quadratic in size.
And so what I'm going to show is that under those circumstances, it's easy for me to find hash functions such that there are n collisions.
That's the name of the game.
Figure out how can I make these hash functions so that there are no collisions.
So that's why I draw this with so few elements here.
So here for example, I have two elements and I have a hash table size four here.
I have three elements.
I need a hash table size nine.
OK, if there are a hundred elements, I need a hash table size 10,000.
I'm not going to pick something so there's likely that there's anything of that size.
And then the fact that this actually works and gives us all the properties that we want, that's part of the analysis.
So does everybody see that this takes Order one worst case time and what the basic structure of it is?
These things, by the way, are not in this case prime.
I could always pick primes that were close to this.
I didn't do that in this case.
Or I could use a universal hash function that in fact would work for things other than primes.
But I didn't do that for this example.
We all ready for analysis?
OK, let's do some analysis then.
And this is really pretty analysis.
Partly as you'll see because we've already done some of this analysis.
So the trick is analyzing level two.
That's the main thing that I want to analyze, to show that I can find hash functions here that are going to, when I map them into, very sparsely, into these arrays here, that in fact, such hash functions exist and I can compute them in advance.
So that I have a good way of storing those.
So here's the theorem we're going to use.
My hash and keys into m equals n squared slots using a random hash function in a universal set H. Then the expected number of collisions is less than one half.
OK.
The expected number of collisions I don't expect there to be even one collision.
I expect there to be less than half a collision on average.
And so, let's prove this, so that the probability that two given keys collide under h is what?
What's the probability that two given keys collide under h when h is chosen randomly from the universal set?
One over m. Right?
That's the definition, right, of, which is in this case equal to one over n squared.
So now how many keys, how many pairs of keys do I have in this table?
How many keys could possibly collide with each other?
OK.
So that's basically just looking at how many different pairs of keys do I have to evaluate this for.
So that's n choose two pairs of keys.
n choose two pairs of keys.
So therefore, the expected number of collisions is while for each of these n, not n over two.
n choose two pairs of keys.
The probability that it collides is one in n squared.
So that's equal to n times n minus one over two, if you remember your formula, times one in n squared.
And that's less than a half.
So for every pair of keys, so those of you who remember from 6.042 the birthday paradox, this is related to the birthday paradox a little bit.
But here I basically have a large set, and I'm looking at all pairs, but my set is sufficiently big that the odds that I get a collision is relatively small.
If I start increasing it beyond the square root of m, OK, the number of elements, it starts getting bigger in the square root of m then the odds of a collision go up dramatically as you know from the birthday paradox.
But if I'm less than, if I'm really sparse in there, I don't get collisions.
Or at least I get a relatively small number expected.
Now I want to remind you of something which actually in the past I have just assumed, but I want to actually go through it briefly.
It's Markov's inequality.
So who remembers Markov's inequality?
Don't everybody raise their hand at once.
So Markov's inequality says the following.
This is one of these great probability facts.
For random variable x which is bounded below by 0, says the probability that x is bigger than, greater than or equal to any given value T is less than or equal to the expectation of x divided by T. It's a great fact.
Doesn't happen if x isn't bound below by 0.
But it's a great fact.
It allows me to relate the probability of an event to its expectation.
And the idea is in general that if the expectation is going to be small, then I can't have a high probability that the value of the random variable is large.
It doesn't make sense.
How could you have a high probability that it's a million when my expectation is one or in this case we're going to apply it when the expectation is a half?
Couldn't happen.
And the proof follows just directly on the definition of expectation, and so I'mdoing this for a discrete random variable.
So the expectation by definition is just the sum from little x goes to 0 to infinity of x times the probability that my random variable takes on the value x.
That's the definition.
And now it's just a question of doing like the coarsest approximation you can imagine.
First of all, let me just simply throw away all small terms that can be greater to or equal to x equals T to infinity of x times the probability that x is equal to little x.
So just throw away all the low order terms.
Now what I'm going to do is replace every one of these terms is lower bounded by the value x equals T. So that's just the summation of x equals T to infinity of T times the probability that x equals x. OK. Over x going from T larger.
Because these are only bigger values.
And that's just equal then to T, because I can pull that out, and the summation of x equals T to infinity of the probability that x equals x is just the probability that x is greater than or equal to T. And that's done because I just divide by T. So that's Markov's inequality.
Really dumb.
Really simple.
There are much stronger things like Chebyshev bounds and Chernoff bounds and things of that nature.
But Markov's is like unbelievably simple and useful.
So we're going to just apply that as a corollary.
So the probability now of no collisions, when I hash n keys into n squared slots using a universal hash function, I claim is the probability of no collisions is greater than or equal to a half.
So I pick a hash function at random.
What are the odds that I got no collisions when I hashed those n keys into n squared slots?
Answer.
Probability is I have no collisions is at least a half.
Half the time I'm guaranteed that there won't be a collision.
And the proof, pretty simple.
The probability of no collisions is the same as the probability as, sorry, is one minus the probability that I have at most one collision.
So the odds that I have at least one collision, the odds that I have at least one collision, probability greater than or equal to one collision is less than or equal to, now I just apply Markov's inequality with this.
So it's just the expected number of collisions -- -- divided by one.
And that is by Markov's inequality less than, by definition, excuse me, of expected number of collisions, which we've already shown, is less than a half.
So the probability of at least one collision is less than a half.
The probability of 0 collisions is at least a half.
So we're done here.
So to find a good level to hash function is easy.
I just test a few at random.
Most of them out there, OK, half of them, at least half of them are going to work.
So this is in some sense, if you think about it, a randomized construction, because I can't tell you which one it's going to be.
It's non-constructive in that sense, but it's a randomized construction.
But they have to exist because most of them out there have this good property.
So I'mgoing to be able to find for each one of these, I just test a few at random, and I find one.
Test a few at random, find one, etc.
Fill in my table there.
Because all that is pre-computation.
And I'mgoing to find them because the odds are good that one exists.
So -- -- we just test a few at random.
And we'll find one quickly -- -- since at least half will work.
I just want to show that there exists good ones.
All I have to prove is that at least one works for each of these cases.
In fact, I've shown that there's a huge number that will work.
Half of them will work.
But to show it exists, I would just have to show that the probability was greater than So to finish up, we need to still analyze the storage because I promised in my theorem that the table would be of size order n. And yet now I've said there's all of these quadratic-sized slots here.
So I'mgoing to show that that's order n. So for level one, that's easy.
We'll just choose the number of slots to be equal to the number of keys.
And that way the storage at level one is just order n. And now let's let n sub i be the random variable for the number of keys -- -- that hash to slot i in T. OK, so n sub i is just what we've called it.
Number of elements that slot there.
And we're going to use m sub i equals n sub i squared slots in each level two table S sub i.
So the expected total storage -- -- is just n for level one, order n if you want, but basically n slots for level one plus the expected value, whatever I expect the sum of i equals 0 to m minus one of theta of n sub i squared to be.
Because I basically have to add up the square for every element that applies here, the square of what's in there.
Who recognizes this summation?
Where have we seen that before?
Who attends recitation?
Where have we seen this before?
What's the -- We're summing the expected value of a bunch of -- Yeah, what was that algorithm?
We did the sorting algorithm, right?
What was the sorting algorithm for which this was an important thing to evaluate?
Don't everybody shout it out at once.
What was that sorting algorithm called?
Bucket sort.
Good.
Bucket sort.
Yeah.
We showed that the sum of the squares of random variables when they're falling randomly into n bins is order n. Right?
And you can also out of this get a, as we did before, get a probability bound.
What's the probability that it's more than a certain amount times n using Markov's inequality.
But this is the key thing is we've seen this analysis.
OK, we used it there in time, so there's a little bit, but that's one of the reasons we study sorting at the beginning of the term is because the techniques of sorting, they just propagate into all these other areas of analysis.
You see a lot of the same kinds of things.
And so now that you know bucket sort clearly so well, now you know that this without having to do any extra work.
So you might want to go back and review your bucket sort analysis, because it's applied now.
Same analysis.
Two places.
OK. Good recitation this Friday, which will be a quiz review and we have a quiz next, there's no class on Monday, but we have a quiz on next Wednesday.
OK, so good luck everybody on the quiz.
Make sure you get plenty of sleep.
Good morning.
Today we're going to talk about augmenting data structures.
And this is a -- Normally, rather than designing data structures from scratch, you tend to take existing data structures and build your functionality into them.
And that is a process we call data-structure augmentation.
And this also today marks sort of the start of the design phase of the class.
We spent a lot of time doing analysis up to this point.
And now we're still going to learn some new analytical techniques.
But we're going to start turning our focus more toward how is it that you design efficient data structures, efficient algorithms for various problems?
So this is a good example of the design phase.
It is a really good idea, at this point, if you have not done so, to review the textbook Appendix B.
You should take that as additional reading to make sure that you are familiar, because over the next few weeks we're going to hit almost every topic in Appendix B.
It is going to be brought to bear on the subjects that we're talking about.
If you're going to go scramble to learn that while you're also trying to learn the material, it will be more onerous than if you just simply review the material now.
We're going to start with an illustration of the problem of dynamic order statistics.
We are familiar with finding things like the median or the kth order statistic or whatever.
Now we want to do the same thing but we want to do it with a dynamic set.
Rather than being given all the data upfront, we're going to have a set.
And then at some point somebody is going to be doing typically insert and delete.
And at some point somebody is going to say OK, select for me the ith largest guy or the ith smallest guy -- -- in the dynamic set.
Or, something like OS-Rank of x.
The rank of x in the sorted order of the set.
So either I want to just say, for example, if I gave n over 2, if I had n elements in the set and I said n over 2, I am asking for the median.
I could be asking for the mean.
I could be asking for quartile.
Here I take an element and say, OK, so where does that element fall among all of the other elements in the set?
And, in addition, these are dynamic sets so I want to be able to do insert and delete, I want to be able to add and remove elements.
The solution we are going to look at for this one, the basic idea is to keep the sizes of subtrees in the nodes of a red-black tree.
Let me draw a picture as an example.
In this tree -- I didn't draw the NILs for this.
I am going to keep two values.
I am going to keep the key.
And so for the keys, what I will do is just use letters of the alphabet.
And this is a red-black tree.
Just for practice, how can I label this tree so it's a red-black tree?
I haven't shown the NILs.
Remember the NILs are all black.
How can I label this, red and black?
Make sure it is a red-black tree.
Not every tree can be labeled as a red-black tree, right?
This is good practice because this sort of thing shows up on quizzes.
Make F red, good, and everything else black, that is certainly a solution.
Because then that basically brings the level of this guy up to here.
Actually, I had a more complicated one because it seemed like more fun.
What I did was I made this guy black and then these two guys red and black and red, black and red, black and black.
But your solution is perfectly good as well.
So we don't have any two reds in a row on any path.
And all the black height from any particular point going down we get the same number of blacks whichever way we go.
Good.
The idea here now is that, we're going to keep the subtree sizes, these are the keys that are stored in our dynamic set, we're going to keep the subtree sizes in the red-black tree.
For example, this guy has size one.
These guys have size one because they're leaves.
And then we can just work up.
So this has size three, this guy has size five, this guy has size three, and this guy has five plus three plus one is nine.
In general, we will have size of x is equal to size of left of x plus the size of the right child of x plus one.
That is how I compute it recursively.
A very simple formula for what the size is.
It turns out that for the code that we're going to want to write to implement these operations, it is going to be convenient to be talking about the size of NIL.
So what is the size of NIL?
Zero.
Size of NIL, there are no elements there.
However, in most program languages, if I take size of NIL, what will happen?
You get an error.
That is kind of inconvenient.
What I have to do in my code is that everywhere that I might want to take size of NIL, or take the size of anything, I have to say, well, if it's NIL then return zero, otherwise return the size field, etc.
There is an implementation trick that we're going to use to simplify that.
It's called using a sentinel.
A sentinel is nothing more than a dummy record.
Instead of using a NIL, we will actually use a NIL sentinel.
We will use a dummy record for NIL such that size of NIL is equal to zero.
Instead of any place I would have used NIL in the tree, instead I will have a special record that I will call NIL.
But it will be a whole record.
And that way I can set its size field to be zero, and then I don't have to check that as a special case.
That is a very common type of programming trick to use, is to use sentinels to simplify code so you don't have all these boundary cases or you don't have to write an extra function when all I want to do is just index the size of something.
Everybody with me on that?
So let's write the code for OS-Select given this representation.
And this is going to basically give us the ith smallest in the subtree rooted at x.
It's actually going to be a little bit more general.
If I want to implement the OS-Select i of up there, I basically give it the root n_i.
But we're going to build this recursively so it's going to be helpful to have the node in which we're trying to find the subtree.
Here is the code.
This is the code.
And let's just see how it works and then we will argue why it works.
As an example, let's do OS-Select of the root and 5.
We're going to find the fifth largest in the set.
We have OS-Select of the root and 5.
This is inconvenient.
We start out at the top, well, let's just switch the boards.
Here we go.
We start at the top, and i is the root.
Excuse me, i is 5, sorry, and the root.
i=5.
We want to five the fifth largest.
We first compute this value k. k is the size of left of x plus What is that value?
What is k anyway?
What is it?
Well, in this case it is 6.
Good.
But what is the meaning of k?
The order.
The rank.
Good, the rank of the current node.
This is the rank of the current node.
k is always the size of the left subtree plus 1.
That is just the rank of the current node.
We look here and we say, well, the rank is k. Now, if it is equal then we found the element we want.
But, otherwise, if i is less, we know it's going to be in the left subtree.
All we're doing then is recursing in the left subtree.
And here we will recurse.
We will want the fifth largest one.
And now this time k is going to be equal to what?
Two.
Now here we say, OK, this is bigger, so therefore the element we want is going to be in the right subtree.
But we don't want the ith largest guy in the right subtree, because we already know there are going to be two guys over here.
We want the third largest guy in this subtree.
We have i equals 3 as we recurse into this subtree.
And now we compute k for here.
This plus 1 is 2.
And that says we recursed right here.
And then we have i=1, k=1, and we return in this code a pointer to this node.
So this returns a pointer to the node containing H whose key is H. Just to make a comment here, we discovered k is equal to the rank of x.
Any questions about what is going on in this code?
OK.
It's basically just finding its way down.
The subtree sizes help it make the decision as to which way it should go to find which is the ith largest.
We can do a quick analysis.
On our red-black tree, how long does OS-Select take to run?
Yeah?
Yeah, order log n if there are n elements in the tree.
Because the red-black tree is a balance tree.
Its height is order log n. In fact, this code will work on any tree that has order log n the height of the tree.
And so if you have a guaranteed height, the way that red-black trees do, you're in good shape.
OS-Rank, we won't do but it is in the book, also gets order log n. Here is a question I want to pose.
Why not just keep the ranks themselves?
Yeah?
It's the node itself.
Otherwise, you cannot take left of it.
I mean, if we were doing this in a decent language, strongly typed language there would be no confusion.
But we're writing in this pseudocode that is good because it's compact, which lets you focus on the algorithm.
But, of course, it doesn't have a lot of the things you would really want if you were programming things of scale like type safety and so forth.
Yeah?
It is basically hard to maintain when you modify it.
For example, if we actually kept the ranks in the nodes, certainly it would be easy to find the element of a given rank.
But all I have to do is insert the smallest element, an element that is smaller than all of the other elements.
And what happens?
All the ranks have to be changed.
Order n changes have to be made if that's what I was maintaining, whereas with subtree sizes that's a lot easier.
Because it's hard to maintain -- -- when the red-black tree is modified.
And that is the other sort of tricky thing when you're augmenting a data structure.
You want to put in the things that your operations go fast, but you cannot forget that there are already underlying operations on the data structure that have to be maintained in some way.
Can we close this door, please?
Thank you.
We have to look at what are the modifying operations and how do we maintain them.
The modifying operations for red-black trees are insert and delete.
If I were augmenting a binary heap, what operations would I have to worry about?
If I were augmenting a heap, what are the modifying operations?
Binary min heap, for example, classic priority queue?
Who remembers heaps?
What are the operations on a heap?
There's a good final question.
Take-home exam, don't worry about it.
Final, worry about it.
What are the operations on a heap?
Just look it up on Books24 or whatever it is, right?
AnswerMan?
What does AnswerMan say?
OK. And?
If it's a min heap.
It's min, extract min, typical operations and insert.
And of those which are modifying?
Insert and extract min, OK?
So, min is not.
You don't have to worry about min because all that is is a query.
You want to distinguish operations on a dynamic data structure those that modify and those that don't, because the ones that don't modify the data structure are all perfectly fine as long as you haven't destroyed information.
The queries, those are easy.
But the operations that modify the data structure, those we're very concerned about in making sure we can maintain.
Our strategy for dealing with insert and delete in this case is to update the subtree sizes -- -- when inserting or deleting.
For example, let's look at what happens when I insert k. Element key k. I am going to want to insert it in here, right?
What is going to happen to this subtree size if I am inserting k in here?
This is going to increase to And then I go left.
This one is going to increase to 6.
Here it is going to increase to Here 2.
And then I will put my k down there with a 1.
So I just updated on the way down.
Pretty easy.
Yeah?
But now it's not a red-black tree anymore.
You have to rebalance, so you must also handle rebalancing.
Because, remember, and this is something that people tend to forget so it's always, I think, helpful when I see patterns going on to tell everybody what the pattern is so that you can be sure of it in your work that you're not falling into that pattern.
What people tend to forget when they're doing red-black trees is they tend to remember the tree insert part of it, but red-black insert, that RB insert procedure actually has two parts to it.
First you call tree insert and then you have to rebalance.
And so you've got to make sure you do the whole of the red-black insert.
Not just the tree insert part.
We just did the tree insert part.
That was easy.
We also have to handle rebalancing.
So there are two types of things we have to worry about.
One is red-black color changes.
Well, unfortunately those have no effect on subtree sizes.
If I change the colors of things, no effect, no problem.
But also the interesting one is rotations.
Rotations, it turns out, are fairly easy to fix up.
Because when I do a rotation, I can update the nodes based on the children.
I will show you that.
You basically look at children and fix up, in this case, in order one time per rotation.
For example, imagine that I had a piece of my tree that looked like this.
And let's say it was 7, 3, 4, the subtree sizes.
I'm not going to put the values in here.
And I did a right rotation on that edge to put them the other way.
And so these guys get hooked up this way.
Always the three children stay as three children.
We just swing this guy over to there and make this guy be the parent of the other one.
And so now the point is that I can just simply update this guy to be, well, he's got 8, 3 plus 4 plus 1 using our formula for what the size is.
And now, for this one, it's going to be 8 plus 7 plus 1 is 16, or, if I think about it, it's going to be whatever that was before because I haven't changed this subtree size with a rotation.
Everything beneath this edge is still beneath this edge.
And so I fixed it up in order one time.
There are certain other types of operations sometimes that occur where this isn't the value.
If I wasn't doing subtree sizes but was doing some other property of the subtree, it could be that this was no longer 16 in which case the effect might propagate up towards the root.
There is a nice little lemma in the book that shows the conditions under which you can make sure that the re-balancing doesn't cost you too much.
So that was pretty good.
Now, insert and delete, that is all we have to do for rotations, are therefore still order log n time, because a red-black tree only has to do order one rotations.
Do they normally take constant time?
Well, they still take constant time.
They just take a little bit bigger constant.
And so now we've been able to build this great data structure that supports dynamic order statistic queries and it works in order log n time for insert, delete and the various queries.
OS-Select.
I can also just search for an element.
I have taken the basic data structure and have added some new operations on it.
Any questions about what we did here?
Do people understand this reasonably well?
OK. Then let's generalize, always a dangerous thing.
Augmenting data structures.
What I would like to do is give you a little methodology for how you go about doing this safely so you don't forget things.
The most common thing, by the way, if there is an augmentation problem on the take-home or if there is one on the final, I guarantee that probably a quarter of the class will forget the rotations if they augmented red-black tree.
I guarantee it.
Anyway, here is a little methodology to check yourself.
As I mentioned, the reason why this is so important is because this is, in practice, the thing that you do most of the time.
You don't just use a data structure as given.
You take a data structure.
You say I have my own operations I want to layer onto this.
We're going to give a methodology.
And what I will do, as I go along, is will use the example of order statistics trees to illustrate the methodology.
It is four steps.
The first is choose an underlying data structure.
Which in the case of order statistics tree was what?
Red-black tree.
And the second thing we do is we figure out what additional information we wish to maintain in that data structure.
Which in this case is the subtree sizes.
Subtree sizes is what we keep for this one.
And when we did this we could make mistakes, right?
We could have said, oh, let's keep the rank.
And we start playing with it and discover we can do that.
It just goes really slowly.
It takes some creativity to figure out what is the information that you're going to be able to keep, but also to maintain the other properties that you want.
The third step is verify that the information can be maintained -- -- for the modifying operations on the data structure.
And so in this case, for OS trees, the modifying operations were insert and delete.
And, of course, we had to make sure we dealt with rotations.
And because rotations are part of that we could break it down into the tree insert, the tree delete and rotations.
And once we've did that everything was fine.
We didn't, for this particular problem, have to worry about color changes.
But that's another thing that under some things you might have to worry about.
For some reason the color made a difference.
Usually that doesn't make a difference.
And then the fourth step is to develop new operations.
Presumably that use the info that you have now stored.
And this was OS-Select and OS-Rank, which we didn't give but which is there.
And also it's a nice little puzzle to figure out yourself, how you would build OS-Rank.
Not a hard piece of code.
This methodology is not actually the way you do this.
This is one of these things that's more like a checklist, because you see whether or not you've got -- When you're actually doing this maybe you developed the new operations first.
You've got to keep in mind the new operations while you're verifying that the information you're storing can be here.
Maybe you will then go back and change this and sort of sort through it.
This is more a checklist that when you're done this is how you write it up.
This is how you document that what you've done is, in fact, a good thing.
You have a checklist.
Here is my underlying data structure.
Here is the addition information I need.
See, I can still support the modifying operations that the data structure used to have and now here are my new operations and see what those are.
It's really a checklist.
Not a prescription for the order in which you do things.
You must do all these steps, not necessarily in this order.
This is a guide for your documentation.
When we ask for you to augment a data structure, generally we're asking you to tell us what the four steps are.
It will help you organize your things.
It will also help make sure you don't forget some step along the way.
I've seen people who have added the information and developed new operations but completely forgot to verify that the information could be maintained.
So you want to make sure that you've done all those.
Usually you have to play -- -- with interactions -- -- between steps.
It's not just a do this, do this, do this.
We're going to do now a more complicated data structure.
It's not that much more complicated, but its correctness is actually kind of challenging.
And it is actually a very practical and useful data structure.
I am amazed at how many people aren't aware that there are data structures of this nature that are useful for them when I see people writing really slow code.
And so the example we're going to do is interval trees.
And the idea of this is that we want to maintain a set of intervals.
For example, time intervals.
I have a whole database of time intervals that I'm trying to maintain.
Let's just do an example here.
This is going from 7 to 10, 5 to 11 and 4 to 8, from 15 to 18, 17 to 19 and 21 to 23.
This is a set of intervals.
And if we have an interval i, let's say this is interval i, which is 7,10.
We're going to call this endpoint the low endpoint of i and this we're going to call the high endpoint of i.
The reason I use low and high rather than left or right is because we're going to have a tree, and we're going to want the left subtree and the right subtree.
So if I start saying left and right for intervals and left and right for tree we're going to get really confused.
This is also a tip.
Let me say when you're coding, you really have to think hard sometimes about the words that you're using for things, especially things like left and right because they get so overused throughout programming.
It's a good idea to come up with a whole wealth of synonyms for different situations so that it is clear in any piece of code when you're talking, for example, about the intervals versus the tree, because we're going to have both going on here.
And what we're going to do is we want to support insertion and deletion of intervals here.
And we're going to have a query, which is going to be the new operation we're going to develop, which is going to be to find an interval, any interval in the set that overlaps a given query interval.
So I give you a query interval like say 6, 14 and you can return this guy or this guy, this guy, couldn't return any of these because these are all less than 14.
So I can return any one of those.
I only have to return one.
I just have to find one guy that overlaps.
Any question about what we're going to be setting up here?
OK. Our methodology is we're going to pick, first of all, step one.
And here is our methodology.
Step one is we're going chose underlying data structure.
Does anybody have a suggestion as to what data structure we ought to use here to support interval trees?
What data structure should we try to start here to support interval trees?
Anybody have any idea?
A red-black tree.
A binary search tree.
Red-black tree.
We're going to use a red-black tree.
Oh, I've got to say what it is keyed on.
What is going to be the key for my red-black tree?
For each interval, what should I use for a key?
This is where there are a bunch of options, right?
Throw out some ideas.
It's always better to branch than it is to prune.
You can always prune later, but if you don't branch you will never get the chance to prune.
So generation of ideas.
You'll need that when you're doing the design phase and doing the take-home exam.
Yeah?
We're calling that the low endpoint.
OK, you could do low endpoint.
What other ideas are there?
High end point.
Now you can look at low endpoint, high endpoint.
Well, between low and high which is better?
That one is not going to matter, right?
So doing high versus low, we don't have to consider that, but there is another natural point you want to think about using like the median, the middle point.
At least that is symmetric.
What do you think?
What else might I use?
The length?
I think the length doesn't feel to me productive.
This is just purely a matter of intuition.
It doesn't feel productive, because if I know the length I don't know where it is so it's going to be hard to maintain information about where it is for queries.
It turns out we're going to use the low left endpoint, but I think to me that was sort of a surprise that you'd want to use that and not the middle one.
Because you're favoring one endpoint over the other.
It turns out that's the right thing to do, surprisingly.
There is another strategy.
Actually, there's another type of tree called a segment tree.
Actually, what you do is you store both the left and right endpoints separately in the tree.
And then you maintain a data structure where the line segments go up through the tree on to the other.
There are lots of things you can do, but we're just going to keep it keyed on the low endpoint.
That's why this is a more clever data structure in some ways.
Now, this is harder.
That is why this is a clever data structure.
What are we going to store in the -- I think any of those ideas are good ideas to throw out and look at.
You don't know which one is going to work until you play with it.
This one, though, is, I think, much harder to guess.
You're going to store in a node the largest value, I will call it m, in the subtree rooted at that node.
We'll draw it like this, a node like this.
We will put the interval here and we will put the m value here.
Let's draw a picture.
Once again, I am not drawing the NILs.
I hope that that is a search tree that is keyed on the low left endpoint.
4, 5, 7, 15, 17, 21.
It is keyed on the low left endpoint.
If this a red-black tree, let's just do another practice.
How can I color this so that it is a legal red-black tree?
Not too relevant to what we're doing right now But a little drill doesn't hurt sometimes.
Remember, the NILs are not there and they are all black.
And the root is black.
I will give that one to you.
Good.
This will work.
You sort of go through a little puzzle.
A logic puzzle.
Because this is really short so it better not have any reds in it.
This has got to be black.
Now, if I'm going to balance the height, I have got to have a layer of black here.
It couldn't be that one.
It's got to be these two.
Good.
Now let's compute the m value for each of these.
It's the largest value in the subtree rooted at that node.
What's the largest value in the subtree rooted at this node?
And in this one?
In this one?
That one is 23 and that is 23.
In general, m is going to be the maximum of three possible values.
Either the high point of the interval at x or m of the left of x or m of the right of x.
Does everybody see that?
It is going to be m of x for any node.
I just have to look, what is the maximum here, what is the maximum here and what is the high point of the interval.
Whichever one of those is largest, that's the largest for that subtree.
The modifying operations.
Let's first do insert.
How can I do insert?
There are two parts.
The first part is to do the tree insert, just a normal insert into a binary search tree.
What do I do?
Insert a new interval?
Insert a new interval here?
How can I fix up the m's?
That's right.
You just go down the tree and look at my current interval.
And if it's got a bigger max, this is something that is going into that subtree.
If its high endpoint is bigger than the current max, update the current max.
I just do that as I'm going through the insertion, wherever it happens to land up in every subtree that it hits, every node that it hits on the way down.
I just update it with the maximum wherever it happens to fall.
Good.
You just fix them on the way down.
But we also have to do the other section.
Also need to handle rotations.
So let's just see how we might do rotations as an example.
Let's say this is 11, 15, 30.
Let's say I'm doing a right rotation.
This is coming off from somewhere.
That is coming off.
This is still going to be the child that has 30, the one that 14 and the one that has 19.
And so now we've rotated this way, so this is the 11, 15 and this is the 6, For this one, I just use my formula here.
I just look here and say which is the biggest, 14, 15 or 19?
And I look here.
Which is the biggest?
30, 19 or 20?
Or, once again, it turns out, not too hard to show, that it's always whatever was there, because we're talking about the biggest thing in the subtree.
And the membership of the subtree hasn't changed when we do the rotation.
That just took me order one time to fix up.
Fixing up the m's during rotation takes O(1) time.
So the total insert time is O(lg n).
Once I figured out that this is the right information, of course we don't know what we're using this information for yet.
But once I know that that is the information, showing you that it works in certain delete continuing work in order log n time is easy.
Now, delete is actually a little bit trickier but I will just say it is similar.
Because in delete you go through and you find something, you may have to go through the whole business of swapping it.
If it's an internal node you've got to swap it with its successor or predecessor.
And so there are a bunch of things that have to be dealt with, but it is all stuff where you can update the information using this thing.
And it's all essentially local changes when you're updating this information because you can do it essentially only on a path up from the root and most of the tree is never dealt with.
I will leave that for you folks to work out.
It's also in the book if you want to cheat, but it is a good exercise.
Any questions about the first three steps?
Fourth step is new operations.
Interval search of i is going to find an interval that overlaps the interval i.
So i here is an interval.
It's got two coordinates.
And this, rather than writing recursively, we're going to write as, it's sort of going to be recursive, but we're going to write it with a while loop.
You could write it recursively.
The other one that we wrote, we could have written as a while loop as well and not had the recursive call.
Here we're going to basically just start x gets the root.
And then while -- That is the code.
Let's just see how it works.
Let's search for the interval 14, 16 -- -- in this tree.
Let's see.
x starts out at the root.
And while it is not NIL, and it's not NIL because it's the root, what is this doing?
Somebody tell me what that code does.
Well, what is this doing?
This is testing something between i and int of x. Int of x is the interval stored at x.
What is this testing for?
I hope I got it right.
What is this testing for?
Yeah?
Above or below?
I need just simple words.
Test for overlaps.
In particular test whether they do or don't?
Do?
Don't?
If I get to this point, what do I know about i and int of x?
Don't overlap.
They don't overlap because the high of one is smaller than the low of the other.
The high of one is smaller than the low of the other.
They don't overlap that way.
Could they overlap the other way?
No because we're testing also whether the low of the one is bigger than the high of the other.
They're saying it's either like this or like this.
This is testing not overlap.
That makes it simpler.
When I'm searching for 14, 16, I check here.
And I say do they overlap?
And the answer is, now we can understand it without having to go through all the arithmetic calculations, no they don't overlap.
If they did overlap, I found what I want.
And what's going to happen?
I am going to drop out of the while loop and just return x, because I will return something that overlaps.
That is my goal.
Here it says they don't overlap.
So then I say, well, if left of x is not NIL, in other words, I've got a left child and low of i is less than or equal to m of left of x, then we go left.
What happens in this case if I'm searching for 14, 16?
Is the low of i less than or equal to m of left of x?
Low of i is 14.
And I am searching.
And is it less than 18?
Yes.
Therefore, what do I do?
I go left and make x point to this guy.
Now I check.
Does it overlap?
No.
I take a look at the left guy.
It is 8.
I compare 8 with 14, right?
And is it lower?
No, so I go right.
And now I discover that I have an overlap here and it overlaps.
It returns then the 15, 18 as an overlapping one.
If I were searching for 12, I would go up to the top.
And I look, 12, 14, it doesn't overlap here.
I look at the 18 and it is greater so I go left.
I then look here.
Does it overlap?
No.
So then what happens?
I look at the left.
It says I go right.
I look here.
Then I go and I look at the left.
It says, no, go right.
I go here, which is NIL, and now it is NIL.
I return NIL.
And does 12, 14 overlap anything in the set?
No.
So, therefore, it always works.
OK?
OK. We're going to do correctness in a minute, but let's just do our analysis first so we don't have to do it because the correctness is going to be a little bit tricky.
Time = O(lg n) because all I am doing is going down the tree.
It takes time proportional to the height of the tree.
That's pretty easy.
If I need to list all overlaps, suppose I want to list all the overlaps, how quickly can I do that?
Can somebody suggest how I could use this as a subroutine to list all overlaps?
Suppose I have k overlaps, k intervals that overlap my query interval and I want to find every single one of them, how fast can I do that?
How do I do it?
How do I do it?
If I search a second time, I might get the same value.
Yeah, there you go.
Do what?
When you find it delete it.
Put it over to the side.
Find the next one, delete it until there are none left.
And then, if I don't want to modify the data structure, insert them all back in.
It costs me k lg n if they are k overlaps.
That's actually called an output sensitive algorithm.
Because the running time of it depends upon how much it outputs, so this is output sensitive.
The best to date for this problem, by the way, of listing all is O(k+lg n) with a different data structure.
And, actually, that was open for a while as an open problem.
OK. Correctness.
Why does this algorithm always work correctly?
The key issue of the correctness is that I am picking one way to go, left or right.
And that's great, as long as it is in that subtree.
But how do I know that when I pick I decide I'm going to go left that it might not be in the right subtree and I went the wrong way?
Or, if I went right, that I accidentally left one out on the left side?
We're always going just one direction.
And that's sort of the cleverness of the code.
The theorem is let's let L be the set of intervals i prime in the left of a node x.
And R be the set of i primes in the right of x.
And now there are two parts I am going to show.
If the search goes right then the set of i prime in L, such that i prime overlaps i is the empty set.
That's the first thing I do.
If it goes right then there is nothing in the left subtree that overlaps.
It's always, whenever the code goes right, no problem, because there was nothing in the left subtree to be found.
Does everybody understand what that says?
We are going to prove this, but I want to make sure people understand.
Because the second one is going to be harder to understand so you've got to make sure you understand this one first.
Any questions about this?
OK.
If the search goes left -- -- then the set of i prime in L such that i prime overlaps i empty set implies that i prime -- OK. What is this saying?
If the search goes left, if the left was empty, in other words, if you went left and you discovered that there was nothing in there to find, no overlapping interval to find then it is OK because it wouldn't have helped me to go right anyway because there is nothing in the right to be found.
So it is not guaranteeing that there is nothing to be found in the left, but if there happens to be nothing to find in the left it is OK because there was nothing to be found in the right either.
That is what the second one says.
In either case, you're OK to go the way.
So let's do this proof.
Does everybody understand what the proof says?
Understanding the proof is tricky.
It's logic.
Logic is tricky.
Suppose the search goes right.
We'll do the first one.
If left of x is NIL then we are done since we proved what we wanted to prove.
If we go right there are two possibilities, either we have left of x be NIL or left of x is not NIL.
So if it is NIL we are OK because we said if it goes right I want to prove this, that the things in the left subtree that overlap is empty.
If there is nothing there, there is clearly nothing there that overlaps.
Otherwise, the low of i is greater than m of the left of x.
If I look at x here, either x was NIL in the while statement here or this is true.
We just said it is not NIL so let's take a look at, excuse me.
I'm on the wrong line.
I am in this loop.
Left of x was not NIL and the low of i was this.
Which way am I going here?
I am going right.
Therefore, this was not true.
So either left of x was not NIL, which was the first one, or low of i is greater than m of left of x if I am going right.
If I'm going right one of those two had to be true.
The first one was easy.
Otherwise, we have this, low of i is greater than m of left of x.
Now this has got to be that value.
m of left of x is the right endpoint, is the high endpoint of some interval in that subtree.
This is equal to the high of j for some j in L. So m of left of x must be equal to the high of some endpoint because that's how we're picking the m's.
For some j in the left subtree.
And no other interval in L has a larger high endpoint -- -- than high of j.
If I draw a picture here, I have over here i and this is the low of i.
And I have j where we say its high endpoint is less than the low of i.
This is j, and I don't know how far over it goes.
And this has high of j -- -- which is the highest one in the left subtree.
There is nobody else who has got a higher right endpoint.
There is nobody else in this subtree who could possibly overlap I, because all of them end somewhere before this point.
This point is the highest one in a subtree.
Therefore, i prime in L such that i prime overlaps i is the empty set.
And now the hard case.
Everybody stretch.
Hard case.
Does everybody follow this?
The point is that because this is the highest guy everybody else has to be left, so if you didn't overlap the highest guy you're not going to overlap anybody.
Suppose the search goes left -- -- and that there is nothing to overlap in the left subtree.
I went left here but I am not going to find anything.
Now I want to prove that it wouldn't have helped me to go right.
That's essentially what the theorem here says.
That if I assume this it wouldn't have helped to go right.
I want to show that there is nothing in the right subtree.
So going left was OK because I wasn't going to find anything anyway.
Similarly, we go through a similar analysis.
Low of i is less than or equal to m of the left of x, which once again is equal to the high of j for some j in L. We are just saying if I go left these things must be true.
I went left.
Since j is in L it doesn't overlap i, because the set of things that overlap i in L is empty set.
Since j doesn't overlap i that implies that the high of i must be less than the low of j.
Since j is in L and it doesn't overlap i, what are the possibilities?
We essentially have here, if I draw a picture, I have j and L and I have i here.
The point is that it doesn't overlap it, therefore, it must be to the left because its low endpoint is less than this.
But it doesn't overlap it, therefore its high endpoint must be left of the low of this one.
Now we will use the binary search tree property.
That implies that for all i prime in R, everything in the right subtree, we have a low of j is less than or equal to low of i prime, so we're sorted on the low endpoints.
Everything in the right subtree must have a low endpoint that starts to the right of the low endpoint of j because j in the left subtree.
And everything in the whole tree is sorted by low endpoints, so anything in the right subtree is going to start over here.
Those are other things.
These are the i primes in R. We don't know how many there are, but they all start to the right of this point.
So they cannot overlap i either, therefore, there is nothing.
All the i primes in R is also nobody.
Just to go back again, the basic idea is that since this guy doesn't overlap the guy who is in the left and everybody to the right is going to be further to the right, if I go left and don't find anything that's OK because I am not going to find anything over here anyway.
They are not going to overlap.
Data-structure augmentation, great stuff.
It will give you a lot of rich, rich data structures built on any ones you know, hash tables, heaps, binary search trees and so forth.
Good morning.
Today we're going to talk about it a balanced search structure, so a data structure that maintains a dynamic set subject to insertion, deletion, and search called skip lists.
So, I'll call this a dynamic search structure because it's a data structure.
It supports search, and it's dynamic, meaning insert and delete.
So, what other dynamic search structures do we know, just for sake of comparison, and to wake everyone up?
Shut them out, efficient, I should say, also good, logarithmic time per operation.
So, this is a really easy question to get us off the ground.
You've seen them all in the last week, so it shouldn't be so hard.
Treap, good.
On the problems that we saw treaps.
That's, in some sense, the simplest dynamic search structure you can get from first principles because all we needed was a bound on a randomly constructed binary search tree.
And then treaps did well.
So, that was sort of the first one you saw depending on when you did your problem set.
What else?
Charles?
Red black trees, good answer.
So, that was exactly one week ago.
I hope you still remember it.
They have guaranteed log n performance.
So, this was an expected bound.
This was a worst-case order log n per operation, insert, delete, and search.
And, there was one more for those who want to recitation on Friday: B trees, good.
And, by B trees, I also include two-three trees, two-three-four trees, and all those guys.
So, if B is a constant, or if you want your B trees knows a little bit cleverly, that these have guaranteed order log n performance, so, worst case, order log n. So, you should know this.
These are all balanced search structures.
They are dynamic.
They support insertions and deletions.
They support searches, finding a given key.
And if you don't find the key, you find its predecessor and successor pretty easily in all of these structures.
If you want to augment some data structure, you should think about which one of these is easiest to augment, as in Monday's lecture.
So, the question I want to pose to you is supposed I gave you all a laptop right now, which would be great.
Then I asked you, in order to keep this laptop you have to implement one of these data structures, let's say, within this class hour.
Do you think you could do it?
How many people think you could do it?
A couple people, a few people, OK, all front row people, good.
I could probably do it.
My preference would be B trees.
They're sort of the simplest in my mind.
This is without using the textbook.
This would be a closed book exam.
I don't have enough laptops to do it, unfortunately.
So, B trees are pretty reasonable.
Deletion, you have to remember stealing from a sibling and whatnot.
So, deletions are a bit tricky.
Red black trees, I can never remember it.
I'd have to look it up, or re-derive the three cases.
treaps are a bit fancy.
So, that would take a little while to remember exactly how those work.
You'd have to solve your problem set again, if you don't have it memorized.
Skip lists, on the other hand, are a data structure you will never forget, and something you can implement within an hour, no problem.
I've made this claim a couple times before, and I always felt bad because I had never actually done it.
So, this morning, I implemented skip lists, and it took me ten minutes to implement a linked list, and 30 minutes to implement skip lists.
And another 30 minutes debugging them.
There you go.
It can be done.
Skip lists are really simple.
And, at no point writing the code did I have to think, whereas every other structure I would have to think.
There was one moment when I thought, ah, how do I flip a coin?
That was the entire amount of thinking.
So, skip lists are a randomized structure.
Let's add in another adjective here, and let's also add in simple.
So, we have a simple, efficient, dynamic, randomized search structure: all those things together.
So, it's sort of like treaps and that the bound is only a randomized bound.
But today, we're going to see a much stronger bound than an expectation bound.
So, in particular, skip lists will run in order log n expected time.
So, the running time for each operation will be order log n in expectation.
But, we're going to prove a much stronger result that their order log n, with high probability.
So, this is a very strong claim.
And it means that the running time of each operation, the running time of every operation is order log n almost always in a certain sense.
Why don't I foreshadow that?
So, it's something like, the probability that it's order log n is at least one minus one over some polynomial, and n. And, you get to set the polynomial however large you like.
So, what this basically means is that almost all the time, you take your skip lists, you do a polynomial number of operations on it, because presumably you are running a polynomial time algorithm that using this data structure.
Do polynomial numbers of inserts, delete searches, every single one of them will take order log n time, almost guaranteed.
So this is a really strong bound on the tail of the distribution.
The mean is order log n. That's not so exciting.
But, in fact, almost all of the weight of this probability distribution is right around the log n, just tiny little epsilons, very tiny probabilities you could be bigger than log n. So that's where we are going.
This is a data structure by Pugh] in 1989.
This is the most recent.
Actually, no, sorry, treaps are more recent.
They were like '93 or so, but a fairly recent data structure for just insert, delete, search.
And, it's very simple.
You can derive it if you don't know anything about data structures, well, almost nothing.
Now, analyzing that the performance is log n, that, of course, takes our sophistication.
But the data structure itself is very simple.
We're going to start from scratch.
Suppose you don't know what a red black tree is.
You don't know what a B tree is.
Suppose you don't even know what a tree is.
What is the simplest data structure for storing a bunch of items for storing a dynamic set?
A list, good, a linked list.
Now, suppose that it's a sorted linked list.
So, I'm going to be a little bit fancier there.
So, if you have a linked list of items, here it is, maybe we'll make it doubly linked just for kicks, how long does it take to search in a sorted linked list?
Log n is one answer.
n is the other answer.
Which one is right?
n is the right answer.
So, even though it's sorted, we can't do binary search because we don't have random-access into a linked list.
So, suppose I'm only given a pointer to the head.
Otherwise, I'm assuming it's an array.
So, in a sorted array you can search in log n. Sorted linked list: you've still got to scan through the darn thing.
So, theta n, worst case search.
Not so good, but if we just try to improve it a little bit, we will discover skip lists automatically.
So, this is our starting point: sorted linked lists, data n time.
And, I'm not going to think too much about insertions and deletions for the moment.
Let's just get search better, and then we'll worry about dates.
Updates are where randomization will come in.
Search: pretty easy idea.
So, how can we make a linked list better?
Suppose all we know about our linked lists.
What can I do to make it faster?
This is where you need a little bit of innovation, some creativity.
More links: that's a good idea.
So, I do try to maybe add pointers to go a couple steps ahead.
If I had log n pointers, I could do all powers of two ahead.
That's a pretty good search structure.
Some people use that; like, some peer-to-peer networks use that idea.
But that's a little too fancy for me.
Ah, good.
You could try to build a tree on this linear structure.
That's essentially where we're going.
So, you could try to put pointers to, like, the middle of the list from the roots.
So, you search between either here.
You point to the median, so you can compare against the median, and know whether you should go in the first half or the second half that's definitely on the right track, also a bit too sophisticated.
Another list: yes.
Yes, good.
So, we are going to use two lists.
That's sort of the next simplest thing you could do.
OK, and as you suggested, we could maybe have pointers between them.
So, maybe we have some elements down here, some of the elements up here.
We want to have pointers between the lists.
OK, it gets a little bit crazy in how exactly you might do that.
But somehow, this feels good.
So this is one linked list: L_1.
This is another linked list: L_2.
And, to give you some inspiration, I want to give you, so let's play a game.
The game is, what is this sequence?
So, the sequence is 14.
If you know the answer, shout it out.
Anyone yet?
OK, it's tricky.
It's a bit of a small class, so I hope someone knows the answer.
How many TA's know the answer?
Just a couple, OK, if you're looking at the slides, probably you know the answer.
That's cheating.
OK, I'll give you a hint.
It is not a mathematical sequence.
This is a real-life sequence.
Yeah?
Yeah, and what city?
New York, yeah, this is the 7th Ave line.
This is my favorite subway line in New York.
But, what's a cool feature of the New York City subway?
OK, it's a skip list.
Good answer.
[LAUGHTER] Indeed it is.
Skip lists are so practical.
They've been implemented in the subway system.
How cool is that?
OK, Boston subway is pretty cool because it's the oldest subway definitely in the United States, maybe in the world.
New York is close, and it has other nice features like it's open 24 hours.
That's a definite plus, but it also has this feature of express lines.
So, it's a bit of an abstraction, but the 7th Ave line has essentially two kinds of cars.
These are street numbers by the way.
This is, Penn Station, Times Square, and so on.
So, there are essentially two lines.
There's the express line which goes 14, to 34, to 42, to 72, to 96.
And then, there's the local line which stops at every stop.
And, they accomplish this with four sets of tracks.
So, I mean, the express lines have their own dedicated track.
If you want to go to stop 59 from, let's say, Penn Station, well, let's say from lower west side, you get on the express line.
You jump to 42 pretty quickly, and then you switch over to the local line, and go on to 59 or wherever I said I was going.
OK, so this is express and local lines, and we can represent that with a couple of lists.
We have one list, sure, we have one list on the bottom, so leave some space up here.
This is the local line, L_2, 34, 42, 50, 59, 66, 72, 79, and so on.
And then we had the express line on top, which only stops at 14, 34, 42, 72, and so on.
I'm not going to redraw the whole list.
You get the idea.
And so, what we're going to do is put links between in the local and express lines, wherever they happen to meet.
And, that's our two linked list structure.
So, that's what I actually meant what I was trying to draw some picture.
Now, this has a property that in one list, the bottom list, every element occurs.
And the top list just copies some of those elements.
And we're going to preserve that property.
So, L_2 stores all the elements, and L_1 stores some subset.
And, it's still open which ones we should store.
That's the one thing we need to think about.
But, our inspiration is from the New York subway system.
OK, there, that the idea.
Of course, we're also going to use more than two lists.
OK, we also have links.
Let's say it links between equal keys in L_1 and L_2.
Good.
So, just for the sake of completeness, and because we will need this later, let's talk about searches before we worry about how these lists are actually constructed.
Of course, if I wanted that board.
So, if you want to search for an element, x, what do you do?
Well, this is the taking the subway algorithm.
And, suppose you always start in the upper left corner of the subway system, if you're always in the lower west side, 14th St, and I don't know exactly where that is, but more or less, somewhere down at the bottom of Manhattan.
And, you want to go to a particular station like 59.
Well, you'd stay on the express line as long as you can because it happens that we started on the express line.
And then, you go down.
And then you take the local line the rest of the way.
That's clearly the right thing to do if you always start in the top left corner.
So, I'm going to write that down in some kind of an algorithm because we will be generalizing it.
It's pretty obvious at this point.
It will remain obvious.
So, I want to walk right in the top list until that would go too far.
So, you imagine giving someone directions on the subway system they've never been on.
So, you say, OK, you start at 14th.
Take the express line, and when you get to 72nd, you've gone too far.
Go back one, and then go down to the local line.
It's really annoying directions.
But this is what an algorithm has to do because it's never taken the subway before.
So, it's going to check, so let's do it here.
So, suppose I'm aiming for 59.
So, I started 14, say the first thing I do is go to 34.
Then from there, I go to 42.
Still good because 59 is bigger than 42.
I go right again.
I say, oops, 72 is too big.
That was too far.
So, I go back to where it just was.
Then I go down and then I keep going right until I find the element that I want, or discover that it's not in the bottom list because bottom list has everyone.
So, that's the algorithm.
Stop when going right would go too far, and you discover that with a comparison.
Then you walk down to L_2.
And then you walk right in L_2 until you find x, or you find something greater than x, in which case x is definitely not on your list.
And you found the predecessor and successor, which may be your goal.
If you didn't find where x was, you should find where it would go if it were there, because then maybe you could insert there.
We're going to use this algorithm in insertion.
OK, but that search: pretty easy at this point.
Now, what we haven't discussed is how fast the search algorithm is, and it depends, of course, which elements we're going to store in L_1, which subset of elements should go in L_1.
Now, in the subway system, you probably put all the popular stations in L_1.
But here, we want worst-case performance.
So, we don't have some probability distribution on the nodes.
We just like every node to be accessed sort of as quickly as possible, uniformly.
So, we want to minimize the maximum time over all queries.
So, any ideas what we should do with L_1?
Should I put all the nodes of L_1 in the beginning?
OK, it's a strict subset.
Suppose I told you what the size of L_1 was.
I can tell you, I could afford to build this many express stops.
How should you distribute them among the elements of L_2?
Uniformly, good.
So, what nodes, sorry, what keys, let's say, go in L_1?
Well, definitely the best thing to do is to spread them out uniformly, OK, which is definitely not what the 7th Ave line looks like.
But, let's imagine that we could reengineer everything.
So, we're going to try to space these things out a little bit more.
So, 34 and 42nd are way too close.
We'll take a few more stops.
And, now we can start to analyze things.
OK, as a function of the length of L_1.
So, the cost of a search is now roughly, so, I want a function of the length of L_1, and the length of L_2, which is all the elements, n. What is the cost of the search if I spread out all the elements in L_1 uniformly?
Yeah?
Right, the total number of elements in the top lists, plus the division between the bottom and the top.
So, I'll write the length of L_1 plus the length of L_2 divided by the length of L_1.
OK, this is roughly, I mean, there's maybe a plus one or so here because in the worst case, I have to search through all of L_1 because the station I could be looking for could be the max.
OK, and maybe I'm not lucky, and the max is not on the express line.
So then, I have to go down to the local line.
And how many stops will I have to go on the local line?
Well, L_1 just evenly partitions L_2.
So this is the number of consecutive stations between two express stops.
So, I take the express, possibly this long, but I take the local possibly this long.
And, this is an L_2.
And there is, plus, a constant, for example, go walking down.
But that's basically the number of nodes that I visit.
So, I'd like to minimize this function.
Now, L_2, I'm going to call that n because that's the total number of elements.
L_1, I can choose to be whatever I want.
So, let's go over here.
So, I want to minimize L_1 plus n over L_1.
And I get to choose L_1.
Now, I could differentiate this, set it to zero, and go crazy.
Or, I could realize that, I mean, that's not hard.
But, that's a little bit too fancy for me.
So, I could say, well, this is clearly best when L_1 is small.
And this is clearly best when L_1 is large.
So, there's a trade-off there.
And, the trade-off will be roughly minimized up to constant factors when these two terms are equal.
That's when I have pretty good balance between the two ends of the trade-off.
So, this is up to constant factors.
I can let L_1 equal n over L_1, OK, because at most I'm losing a factor of two there when they happen to be equal.
So now, I just solve this.
This is really easy.
This is (L_1)^2 equals n. So, L_1 is the square root of n. OK, so the cost that I'm getting over here, L_1 plus L_2 over L_1 is the square root of n plus n over root n, which is, again, root n. So, I get two root n. So, search cost, and I'm caring about the constant here, because it will matter in a moment.
Two square root of n: I'm not caring about the additive constant, but the multiplicative constant I care about.
OK, that seems good.
We started with a linked list that searched in n time, theta n time per operation.
Now we have two linked lists, search and theta root n time.
It seems pretty good.
This is what the structure looks like.
We have root n guys here.
This is in the local line.
And, we have one express stop which represents that.
But we have another root n values in the local line.
And we have one express stop that represents that.
And these two are linked, and so on.
Well, I should put some dot, dot, dots in there.
OK, so each of these chunks has length root n, and the number of representatives up here is square root of n. The number of express stops is square root of n. So clearly, things are balanced now.
I search for, at most, square root of n up here.
Then I search in one of these lists for, at most, square root of n. So, every search takes, at most, two root n. Cool, what should we do next?
So, again, ignore insertions and deletions.
I want to make searches faster because square root of n is not so hot as we know.
Sorry?
More lines.
Let's add a super express line, or another linked list.
OK, this was two.
Why not do three?
So, we started with a sorted linked list.
Then we went to two.
This gave us two square root of n. Now, I want three sorted linked lists.
I didn't pluralize here.
Any guesses what the running time might be?
This is just guesswork.
Don't think.
From two square root of n, you would go to, sorry?
Two square root of two, fourth root of n?
That's on the right track.
Both the constant and the root change, but not quite so fancily.
Three times the cubed root: good.
Intuition is very helpful here.
It doesn't matter what the right answer is.
Use your intuition.
You can prove that.
It's not so hard.
You now have three lists, and what you want to balance are at the length of the top list, the ratio between the top two lists, and the ratio between the bottom two lists.
So, you want these three to multiply out to n, because the top times the ratio times the ratio: that has to equal n. And, so that's where you get the cubed root of n. Each of these should be equal.
So, you set them because the cost is the sum of those three things.
So, you set each of them to cubed root of n, and there are three of them.
OK, check it at home if you want to be more sure.
Obviously, we want a few more.
So, let's think about k sorted lists.
k sorted lists will be k times the k'th root of n. You probably guessed that by now.
So, what should we set k to?
I don't want the exact minimum.
What's a good value for k?
Should I set it to n?
n's kind of nice, because the n'th root of n is just one.
Now that's n. So, this is why I cared about the lead constant because it's going to grow as I add more lists.
What's the biggest reasonable value of k that I could use?
Log n, because I have a k out there.
I certainly don't want to use more than log n. So, log n times the log n'th root, and this is a little hard to draw of n. Now, what is the log n'th root of n?
That's what you're all thinking about.
What is the log n'th root of n minus two?
It's one of these good questions whose answer is?
Oh man.
Remember the definition of root?
OK, the root is n to the one over log n. OK, good, remember the definition of having a power, A to the B?
It was like two to the power, B log A?
Does that sound familiar?
So, this is two to the log n over log n, which is, I hope you can get it at this point, two.
Wow, so the log n'th root of n minus two is zero: my favorite answer.
OK, this is to.
So this whole thing is two log n: pretty nifty.
So, you could be a little fancier and tweak this a little bit, but two log n is plenty good for me.
We clearly don't want to use any more lists, but log n lists sounds pretty good.
I get, now, logarithmic search time.
Let's check.
I mean, we sort of did this all intuitively.
Let's draw what the list looks like.
But, it will work.
So, I'm going to redraw this example because you have to, also.
So, let's redesign that New York City subway system.
And, I want you to leave three blank lines up here.
So, you should have this memorized by now.
But I don't.
So, we are not allowed to change the local line, though it would be nice, add a few more stops there.
OK, we can stop at 79th Street.
That's enough.
So now, we have log n lists.
And here, log n is about four.
So, I want to make a bunch of lists here.
In particular, 14 will appear on all of them.
So, why don't I draw those in?
And, the question is, which elements go in here?
So, I have log n lists.
And, my goal is to balance the number of items up here, and the ratio between these two lists, and the ratio between these two lists, and the ratio between these two lists.
I want all these things to be balanced.
There are log n of them.
So, the product of all those ratios better be n, the number of elements down here.
So, the product of all these ratios is n. And there's log n of them; how big is each ratio?
So, I'll call the ratio r. The ratio's r. I should have r to the power of log n equals n. What's r?
What's r minus two?
Zero.
OK, this should be two to the power of log n. So, if the ratio between the number of elements here and here is to all the way down, then I will have an elements at the bottom, which is what I want.
So, in other words, I want half the elements here, a quarter of the elements here, an eighth of the elements here, and so on.
So, I'm going to take half of the elements evenly spaced out: 34th, 50th, 66th, 79th, and so on.
So, this is our new semi-express line: not terribly fast, but you save a factor of two for going up there.
And, when you're done, you go down, and you walk, at most, one step.
And you find what you're looking for.
OK, and then we do the same thing over and over and over until we run out of elements.
I can't read my own writing.
It's 79th.
OK, if I had a bigger example, I would be more levels, but this is just barely enough.
Let's say two elements is where I stop.
So, this looks good.
Does this look like a structure you've seen before, at all, vaguely?
Yes?
A tree: yes.
It looks a lot like a binary tree.
I'll just leave it at that.
In your problem set, you'll understand why skip lists are really like trees.
But it's more or less a tree.
Let's say at this level, it looks sort of like binary search.
You look at 14; you look at 15, and therefore, you decide whether you are in the left half for the right half.
And that's sort of like a tree.
It's not quite a tree because we have this element repeated all over.
But more or less, this is a binary tree.
At depth I, we have two to the I nodes, just like a tree, just like a balanced tree.
I'm going to call this structure an ideal skip list.
And, if all we are doing our searches, ideal skip lists are pretty good.
Maybe at practice: not quite as good as a binary search tree, but up to constant factors: just as good.
So, for example, I mean, we can generalize search, just check that it's log n. So, the search procedure is you start at the top left.
So, let's say we are looking for 72.
You start at the top left.
14 is smaller than 72, so I try to go right.
79 is too big.
So, I follow this arrow, but I say, oops, that's too much.
So, instead, I go down 14 still.
I go to the right: oh, 50, that's still smaller than 72: OK.
I tried to go right again.
Oh: 79, that's too big.
That's no good.
So, I go down.
So, I get 50.
I do the same thing over and over.
I try to go to the right: oh, 66, that's OK.
Try to go to the right: oh, 79, that's too big.
So I go down.
Now I go to the right and, oh, 72: done.
Otherwise, I'd go too far and try to go down and say, oops, element must not be there.
It's a very simple search algorithm: same as here except just remove the L_1 and L_2.
Go right until that would go too far.
Then go down.
Then go right until we'd go too far, and then go down.
You might have to do this log n times.
In each level, you're clearly only walking a couple of steps because the ratio between these two sizes is only two.
So, this will cost two log n for search.
Good, I mean, so that was to check because we were using intuition over here; a little bit shaky.
So, this is an ideal skip list, we have to support insertions and deletions.
As soon as we do an insert and delete, there's no way we're going to maintain the structure.
It's a bit too special.
There is only one of these where everything is perfectly spaced out, and everything is beautiful.
So, we can't do that.
We're going to maintain roughly this structure as best we can.
And, if anyone of you knows someone in New York City subway planning, you can tell them this.
OK, so: skip lists.
So, I mean, this is basically our data structure.
You could use this as a starting point, but then you start using skip lists.
And, we need to somehow implement insertions and deletions, and maintain roughly this structure well enough that the search still costs order log n time.
So, let's focus on insertions.
If we do insertions right, it turns out deletions are really trivial.
And again, this is all from first principles.
We're not allowed to use anything fancy.
But, it would be nice if we used some good chalk.
This one looks better.
So, suppose you want to insert an element, x.
We said how to search for an element.
So, how do we insert it?
Well, the first thing we should do is figure out where it goes.
So, we search for x.
We call search of x to find where x fits in the bottom list, not just any list.
Pretty easy to find out where it fits in the top list.
That takes, like, constant time.
What we want to know: because the top list has constant length, we want to know where x goes in the bottom list.
So, let's say we want to insert a search for 80.
Well, it is a bit too big.
Let search for 75.
So, we'll find the 75 fits right here between 72 and 79 using the same path.
OK, if it's there already, we complain because I'm going to assume all keys are distinct for now just so the picture stays simple.
But this works fine even if you are inserting the same key over and over.
So, that seems good.
One thing we should clearly do is insert x into the bottom list.
We now know where it fits.
It should go there.
Because we want to maintain this invariant, that the bottom list contains all the elements.
So, there we go.
We've maintained the invariant.
The bottom list contains all the elements.
So, we search for 75.
We say, oh, 75 goes here, and we just sort of link in 75.
You know how to do a linked list, I hope.
Let me just erase that pointer.
All the work in implementing skip lists is the linked list manipulation.
Is that enough?
No, it would be fine for now because now there's only a chain of length three here that you'd have to walk over if you're looking for something in this range.
But if I just keep inserting 75, and 76, than 76 plus epsilon, 76 plus two epsilon, and so on, just pack a whole bunch of elements in here, this chain will get really long.
Now, suddenly, things are not so balanced.
If I do a search, I'll pay an arbitrarily long amount time here to search for someone.
If I insert k things, it'll take k time.
I want it to stay log n. If I only insert log n items, it's OK for now.
What I want to do is decide which of these lists contain 75.
So, clearly it goes on the bottom.
Every element goes in the bottom.
Should it go up a level?
Maybe.
It depends.
It's not clear yet.
If I insert a few items here, definitely some of them should go on the next level.
Should I go to levels up?
Maybe, but even less likely.
So, what should I do?
Yeah?
Right, so you maintain the ideal partition size, which may be like the length of this chain.
And you see, well, if that gets too long, then I should split it in the middle, promote that guy up to the next level, and do the same thing up here.
If this chain gets too long between two consecutive next level express stops, then I'll promote the middle guy.
And that's what you'll do in your problem set.
That's too fancy for me.
I don't need no stinking counters.
What else could I do?
I could try to maintain the ideal skip list structure.
That will be too expensive.
Like I say, 75 is the guy that gets promoted, and this guy gets demoted all the way down.
But that will propagate everything to the right.
And that could cost linear time for update.
Other idea?
If I only want half of them to go up, I could flip a coin.
Good idea.
All right, for that, I will give you a quarter.
It's a good one.
It's the old line state, Maryland.
There you go.
However, you have to perform some services for that quarter, namely, flip the coin.
Can you flip a coin?
Good.
What did you get?
Tails, OK, that's the first random bit.
But we are going to do is build a skip list.
Maybe I should tell you how first.
OK, but the idea is flip a coin.
If it's heads, so, sorry, if it's heads, we will promote it to the next level, and flip again.
So, this is an answer to the question, which other lists should store x?
How many other lists should we add x to?
Well, the algorithm is, flip a coin, and if it comes out heads, then promote x. to the next level up, and flip again.
OK, that's key because we might want this element to go arbitrarily high.
But for starters, we flip a coin.
It doesn't go to the next level.
Well, we'd like it to go to the next level with probability one half because we want the ratio between these two sizes to be a half, or sorry, two, depending which way you take the ratio.
So, I want roughly half the elements up here.
So, I flip a coin.
If it comes up heads, I go up here.
This is a fair coin.
So I want it 50-50.
OK, then how many should that element go up to the next level up?
Well, with 50% probability again.
So, I flip another point.
If it comes up heads, I'll go up another level.
And that will maintain the approximate ratio between these two guys as being two.
The expected ratio will definitely be two, and so on, all the way up.
If I go up to the top and flip a coin, it comes up heads, I'll make another level.
This is the insertion algorithm: dead simple.
The fancier one you will see on your problem set.
So, let's do it.
OK, I also need someone to generate random numbers.
Who can generate random numbers?
Pseudo-random?
I'll give you a quarter.
I have one here.
Here you go.
That's a boring quarter.
Who would like to generate random numbers?
Someone volunteering someone else: that's a good way to do it.
Here you go.
You get a quarter, but you're not allowed to flip it.
No randomness for you; well, OK, you can generate bits, and then compute a number.
So, give me a number.
44, can answer.
OK, we already flipped a coin and I got tails.
Done.
That's the insertion algorithm.
I'm going to make some more space actually, put it way down here.
OK, so 44 does not get promoted because we got a tails.
So, give me another number.
Nine, OK, I search for nine in this list.
I should mention one other thing, sorry.
I need a small change.
This is just to make sure searches still work.
So, the worry is suppose I insert something bigger and then I promote it.
This would look very bad for a skip list data structure because I always want to start at the top left, and now there's no top left.
So, just minor change: just let me remember that.
The minor change is that I'm going to store a special value minus infinity in every list.
So, minus infinity always gets promoted all the way to the top, whatever the top happens to be now.
So, initially, that way I'll always have a top left.
Sorry, I forgot to mention that.
So, initially I'll just have minus infinity.
Then I insert 44.
I say, OK, 44 goes there, no promotion, done.
Now, we're going to insert nine.
Nine goes here.
So, minus infinity to nine, flip your coin, heads.
Did he actually flip it?
OK, good.
He flipped it before, yeah, sure.
I'm just giving you a hard time.
So, we have nine up here.
We need to maintain this minus infinity just to make sure it gets promoted along with everything else.
So, that looks like a nice skip list.
Flip it again.
Tails, good.
OK, so this looks like an ideal skip list.
Isn't that great?
It works every time.
OK, give me another number.
26, OK, so I search for 26.
26 goes here.
It clearly goes on the bottom list.
Here we go, 26, and then I you raised 44.
Flip.
Tails, OK, another number.
50, oh, a big one.
It costs me a little while to search, and I get over here.
Flip.
Heads, good.
So 50 gets promoted.
Flip it again.
Tails, OK, still a reasonable number.
Another number?
12, it takes a little while to get exciting here.
OK, 12 goes here between nine and 26.
You're giving me a hard time here.
OK, flip.
Heads, OK, 12 gets promoted.
I know you have to work a little bit, but we just came here to search for 12.
So, we know that nine was the last point we went down.
So, we promote 12.
It gets inserted up here.
We are just inserting into this particular linked list: nothing fancy.
We link the two twelves together.
It still looks kind of like a linked list.
Flip again.
OK, tails, another number.
Jeez.
It's a good test of memory.
37, what was it, 44 and 50?
And 50 was at the next level up.
I think I should just keep appending elements and have you flip coins.
OK, we just inserted 37.
Tails.
OK, that's getting to be a long chain.
That looks a bit worse.
OK, give me another number larger than 50.
51, good answer.
Thank you.
OK, flip again.
And again.
Tails.
Another number.
Wait, someone else should pick a number.
It's not working.
What did you say?
52, good answer.
Flip.
Tails, not surprising.
We've gotten a lot of heads there.
OK, another number.
53, thank you.
Flip.
Heads, heads, OK.
Heads, heads, you didn't flip.
All right, 53, you get the idea.
If you get two consecutive heads, then the guy goes up two levels.
OK, now flip for real.
Heads.
Finally.
Heads we've been waiting for.
If you flipped three heads in a row, you go three levels.
And each time, we keep promoting minus infinity.
Look again.
Heads, oh my God.
Where were they before?
Flip again.
It better be tails this time.
Tails, good.
OK, you get the idea.
Eventually you run out of board space.
Now, it's pretty rare that you go too high.
What's the probability that you go higher than log n?
Another easy log computation.
Each time, I have a 50% probability of going up.
One in n probability of going up log n levels because half to the power of log n is one out of n. So, it depends on n, but I'm not going to go too high.
And, intuitively, this is not so bad.
So, these are skip lists.
You have the ratios right in expectation, which is a pretty weak statement.
This doesn't say anything about the lengths of these change.
But intuitively, it's pretty good.
Let's say pretty good on average.
So, I had two semi-random processes going on here.
One is picking the numbers, and that, I don't want to assume anything about.
The numbers could be adversarial.
It could be sequential.
It could be reverse sorted.
It could be random.
I don't know.
So, it didn't matter what he said.
At least, it shouldn't matter.
I mean, it matters here.
Don't worry.
You're still loved.
You still get your $0.25.
But what the algorithm cares about is the outcomes of these coins.
And the probability, the statement that this data structure is fast with high probability is only about the random coins.
Right, it doesn't matter what the adversary chooses for numbers as long as those coins are random, and the adversary doesn't know the coins.
It doesn't know the outcomes of the coins.
So, in that case, on average, overall of the coin flips, you should be OK.
But the claim is not just that it's pretty good on average.
But, it's really, really good almost always.
OK, with really high probability it's log n. So, for example, with probability, one minus one over n, it's order of log n, with probability one minus one over n^2 it's log n, probability one minus one over n^100, it's order log n. All those statements are true for any value of 100.
So, that's where we're going.
OK, I should mention, how do you delete in a skip list?
Find the element.
You delete it all the way.
There's nothing fancy with delete.
Because we have all these independent, random choices, all of these elements are sort of independent from each other.
We don't really care.
So, delete an element, just throw it away.
The tricky part is insertion.
When I insert an element, I'm just going to randomly see how high it should go.
With probability one over two to the i, it will go to height i.
Good, that's my time.
I've been having too much fun here.
I've got to go a little bit faster, OK.
So here's the theorem.
Let's see exactly what we are proving first.
With high probability, this is a formal notion which I will define a second.
Every search in n elements skip lists costs order of log n. So, that's the theorem.
Now I need to define with high probability.
So, with high probability.
And, it's a bit of a long phrase.
So, often we will, and you can abbreviate it WHP.
So, if I have a random event, and the random event here is that every search in an n element skip list costs order log n, I want to know what it means for that event E to occur with high probability.
So this is the definition.
So, the statement is that for any alpha greater than or equal to one, there is a suitable choice of constants -- -- for which the event, E, occurs with this probability I keep mentioning.
So, the probability at least one minus one over n to the alpha.
So, this is a bit imprecise, but it will suffice for our purposes.
If you want to really formal definition, you can read the lecture notes.
There are special lecture notes for this lecture on the stellar site.
And, there's the PowerPoint notes on the SMA site.
But, right, there's a bit of a subtlety in the choice of constants here.
There is a choice of this constant.
And there's a choice of this constant.
And, these are related.
And, there's alpha, which we get to whatever we want.
But the bottom line is, we get to choose what probability we want this to be true.
If I want it to be true, with probability one minus one over n^100, I can do that.
I just sat alpha to a hundred, and up to this little constant that's going to grow much slower than n to the alpha.
I get the error probability.
So this thing is called the error probability.
The probability that I fail is polynomially small, for any polynomial I want.
Now, with the same data structure, right, I fixed the data structure.
It doesn't depend on alpha.
Anything you want, any alpha value you want, this data structure will take order of log n time.
Now, this constant will depend on alpha.
So, you know, you want error probability one over n^100 is probably going to be, like, 100 log n. It's still log n. OK, this is a very strong claim about the tale of the distribution of the running time of search, very strong.
Let me give you an idea of how strong it is.
How many people know what Boole's inequality is?
How many people know what the union bound is in probability?
You should.
It's in appendix c. Maybe you'll know it by the theorem.
It's good to know it by name.
It's sort of like linearity of expectations.
It's a lot easier to communicate to someone.
Linearity of expectations: instead of saying, you know that thing where you sum up all the expectations of things, and that's the expectation of the sum?
It's a lot easier to say linearity of expectation.
So, let me quiz you in a different way.
So, if I take a bunch of events, and I take their union, either this happens or this happens, or so on.
So, this is the inclusive OR of k events.
And, instead, I look at the sum of the probabilities of those events.
OK, easy question: are these equal?
No, unless they are independent.
But can I say anything about them, any relation?
Smaller, yeah.
This is less than or equal to that.
OK, this should be intuitive to you from a probability point of view.
Look at the textbook
very basic result, trivial result almost.
What does this tell us?
Well, suppose that E_i is some kind of error event.
We don't want it to happen.
OK, and suppose, mix some letters here.
Suppose I have a bunch of events which occur with high probability.
OK, call those E_i complement.
So, suppose, so this is the end of that statement, E_i complement occurs with high probability.
OK, so then the probability of E_i is very small, polynomially small.
One over n to the alpha for any alpha I want.
Now, suppose I take a whole bunch of these events, and let's say that k is polynomial in n. So, I take a bunch of events, which I'd like to happen.
They all occur with high probability.
There is only polynomially many of them.
So let's say, let me give this constant a name.
Let's call it c. Let's say I take n to the c such events.
Well, what's the probability that all those events occur together?
Because they should rest of the time occurred together because each one occurs most of the time, occurs with high probability.
So, I want to look at E_1 bar intersect, E_2 bar, and so on.
So, each of these occurs as high probability.
What's the chance that they all occur?
It's also with high probability.
I'm changing the alpha.
So, the union bound tells me the probability of any one of these failing, the probability of this failing, or this failing, or this failing, which is this thing, is, at most, the sum of the probabilities of each failure.
These are the error probabilities.
I know that each of them is, at most, one over n to the alpha, with a constant in front.
If I add them all up, there's only n to the c of them.
So, I take this error probability, and I multiply by n to the c. So, I get like n to the c over n to the alpha, which is one over n to the alpha minus c. I can set alpha as big as I want.
So, I said it much, much bigger than c, and this event occurs with high probability.
I sort of made a mess here, but this event occurs with high probability because of this.
Whatever the constant is here, however many events I'm taking, I just set alpha to be bigger than that.
And, this event will occur with high probability, too.
So, when I say here that every search of cost order log n with high probability, not only do I mean that if you look at one search, it costs order log n with high probability.
You look at another search, and it costs log n with high probability.
I mean, if you take every search, all of them take order log n time with high probability.
So, this event that every single search you do takes order log n, is true with high probability estimate the number of searches you are doing is polynomial in n. So, I'm assuming that I'm not using this data structure forever, just for a polynomial amount of time.
But, who's got more than a polynomial amount of time anyway?
This is MIT.
So, hopefully that's clear.
We'll see it a few more times.
Yeah?
The algorithm doesn't depend on Alpha.
The question is how do you choose alpha in the algorithm.
So, we don't need to.
This is just sort of for an analysis tool.
This is saying that the farther out you get, so you say, well, what's the probability that more than ten log n. Well, it's like one over n^10.
Let's say it's linear.
Well, what's the chance that you're more than 20 log n?
Well that's one over n^20.
So, the point is the tail of this distribution is getting a really small, really fast.
And, such using alpha is more like sort of for your own feeling good.
OK, you can set it to 100, and then n is at least two.
So, that's like one over 2^100 chance that you fail.
That's damn small.
If you've got a real random number generator, the chance that you're going to hit one over 2^200 is pretty tiny, right?
So, let's say you set alpha to 256, which is always a good number.
2^256 is much bigger than the number of particles in the known universe, so, the light matter.
So, actually I think this even accounts for some notion of dark matter.
So, this is really, really, really big.
So, the chance that you pick a random particle in the universe that happens to be your favorite particle, this one right here, that's over one over 2^256, or even smaller.
So, set alpha to 256, the chance to your algorithm takes more than order log n time is a lot smaller than the chance that a meteor strikes your computer at the same time that it has a flooding point error, at the same time that the earth explodes because they're putting a transport through this part of the solar system at the same time, I mean, I could go on, right?
It's really, really unlikely that you are more than log n. And how unlikely: you get to choose.
But it's just in the analysis the algorithm doesn't depend on it.
It's the same algorithm, very cool.
Sometimes, with high probability, bounds depends on alpha.
I mean, the algorithm depends on alpha.
But here, it will not.
OK, away we go.
So now you all understand the claim.
So let's do a warm up.
We will also need this fact.
But it's pretty easy.
The lemma is that with high probability, the number of levels in the skip list is order log n. I think it's order log n, certainly.
So, how do we prove that something happens with high probably?
Compute the probability that it happened; show that it's high.
Even if you don't know what high probability means, in fact, I used to ask that earlier on.
So, let's compute the chance that it doesn't happen, the error probability, because that's just a one minus the cleaner.
So, I'd like to say, let's say, that it's, at most, c log n levels.
So, what's the error probability for that event?
This is sort of an event.
I'll put it in squiggles just for, all set.
This is the probability that they are strictly greater than c log n levels.
So, I want to say that that probability is particularly small, polynomially small.
Well, how do I make levels?
When I insert an element, the probability half, it goes up.
And, the number of levels in the skip list is the max over all the elements of how high it goes up.
But, max, oh, that's a mess.
All right, you can compute the expectation of the max if you have a bunch of unknown variables; there is expectation there is a constant, and you take the max.
It's like log in and expectation, but we want a much stronger statement.
And, we have this Boole's inequality that says I have a bunch of things, polynomially many things.
Let's say we have n items.
Each one independently, I don't even care if it's a dependent.
If it goes up more than c log n, yeah, the number of levels is more than c log n. So, this is, at most, and then I want to know, do any of those events happen for any of the n elements?
So, I just multiplied by n. It's certainly, at most, n times the probability that x gets promoted, this much here, greater than or equal to log n times.
OK, if I pick, for any element, x, because it's the same for each element.
They are done independently.
So, I'm just summing over x here, and that's just a factor of n. Clear?
This is Boole's inequality.
Now, what's the probability that x gets promoted c log n times?
We did this before for log n. It was one over n. For c log n, it's one over n to the c. OK, this is n times two.
Let's be nicer: one half to the power of c log n. One half to the power of c log n is one over two to the c log n. The log n comes out here, becomes an n. We get n to the c. So, this is n divided by n to the c, which is n to the c minus one.
And, I get to choose c to be whatever I want.
So, I choose c minus one to be alpha.
I think exactly that.
Oh, sorry, one over n to the c minus one.
Thank you.
It better be small.
This is an upper bound.
So, probability is polynomially small.
I get to choose, and this is a bit of the trik.
I'm choosing this constant to be large, large enough for alpha.
The point is, as c grows, alpha grows.
Therefore, I can set alpha to be whatever I want, set c accordingly.
So, there's a little bit more words that have to go here.
But, they're in the notes.
I can set alpha to be as large as I want.
So, I can make this probability as small as I want in the polynomial sets.
So, that's it.
Number of levels, order log n: wasn't that easy?
Rules and equality, the point is that when you're dealing with high probability, use Boole's inequality.
And, anything that's true for one element is true for all of them, just like that.
Just lose a factor of n, but that's just one in the alpha, and alpha is big: big constant, but it's big.
OK, so let's prove the theorem.
High probability searches cost order log n. We now know the height is order log n, but it depends how balanced this thing is.
It depends how long the chains are to really know that a search costs log n. Just knowing a bound on the height is not enough, unlike a binary tree.
So, we have one cool idea for this analysis.
And it's called backwards analysis.
So, normally you think of a search as starting in the top left corner going left and down until you get to the item that you're looking for.
I'm going to look at the reverse process.
You start at the item you're looking for, and you go left and up until you get to the top left corner.
The number of steps in those two walks is the same.
And, I'm not implementing an algorithm here, I'm just doing analysis.
So, those are the same processes, just in reverse.
So, here's what it looks like.
You have a search, and it starts, which really means that it ends at a node in the bottom list.
Then, each time you visit a node in this search, you either go left or up.
And, when do you go left or up?
Well, it depends with the coin flip was.
So, if the node wasn't promoted at this level.
So, if it wasn't promoted higher, and that happened exactly when we got a tails.
Then, we go left, which really means we came from the left.
Or, if we got a heads, so if this node was promoted to the next level, which happened whenever we got a heads at that particular moment.
This is in the past some time when we did the insertion.
Then we go, or came from, up.
And, we stop at the root.
This is really where we start; same thing.
So, either at the root or I'm also going to think of this as stopping at minus infinity.
OK, that was a bit messy, but let me review.
So, normally we start up here.
Well, just looking at everything backwards, and in brackets is what's really happening.
So, this search ends at the node you were looking for.
It's always in the bottom list.
Then it says, well, was this node promoted higher?
If it was, I came from above.
If not, I came to the left.
It must have been in the bottom chain somewhere.
OK, and that's true at every node you visit.
It depends whether that quite slipped heads or tails at the time that you inserted that node into that level.
But, these are just a bunch of events.
I'm just going to check, what is the probability that its heads, and what is the probability that a tails?
It's always a half.
Every time I look at a coin flip, when it was flipped, there was a probability of half going out of their way.
That's the magic.
And, I'm not using that these events are independent anyway.
For every element that I search for, for every value, x, that's another search.
Those events may not be independent.
I can still use Boole's inequality and conclude that all of them are order log n with high probability.
As long as I can prove that any one event happens with high probability.
So, I don't need independence between, I knew that these coin flips in a single search are independent, but everything else, for different searches I don't care.
So, how long can this process go on?
We want to know how many times can I make this walk?
Well, when I hit the root node, I'm done.
Well, how quickly would I hit the root node?
Well, with probability, a half, I go up each step.
The number of times I go up is, at most, the number of levels minus one.
And that's order log n with high probability.
So, this is the only other idea.
So, we are now improving this theorem.
So, the number of up moves in a search, which are really down moves, but same thing, is less than the number of levels.
Certainly, you can't go up more than there are levels in the search.
And in insert, you can go arbitrarily high.
But a search: as high as you can go.
And this is, at most, c log n with high probability.
This is what we proved in the lemma.
So, we have a bound on the number of up moves.
Half of the moves, roughly, are going to be up moves.
So, this pretty much down to the number of moves.
Not quite.
So, what this means is that with high probability, so this is the same probability, but I could choose that as high as I want by setting c large enough.
The number of moves, in other words, the cost of the search is at most the number of coin flips until we get c long n heads, right, because in every step of the search, I make a move, and then I flip another coin, conceptually.
There is another independent coin lying there.
And it's either heads or tails.
Each of those is independent.
So, how many independent coin flips does it take until I get c log n heads?
The claim is that that's order log n with high probability.
But we need to prove that.
So, this is a claim.
So, if you just sit there with a coin, and you want to know how many times does it take until I get c log n heads, the claim is that that number is order log n with high probability.
As long as I prove that, I know that the total number of steps I make, which is the number of heads and tails is order log n because I definitely know the number of heads is, at most, c log n. The claim is that the number of tails can't be too much bigger.
Notice, I can't just say c here.
OK, it's really important that I have log n. Why?
Because with high probability, it depends on n. This notion depends on n. Log n: it's true.
Anything bigger that log n: it's true, like n. If I put n here, this is also true.
But, if I put a constant or a log log n, this is not true.
It's really important that I have log n here because my notion of high probability depends on what's written here.
OK, it's clear so far.
We're almost done, which is good because I just ran out of time.
Sorry, we're going to go a couple minutes over.
So, I want to compute the error probability here.
So, I want to compute the probability that there is less than c log n heads.
Let me skip this step.
So, I will be approximate and say, what's the probability that there is, at most, c log n heads?
So, I need to say how many coins we are flipping here for what this event is.
So, I need to specify this constant.
Let's say we flip ten c log n coins.
Now I want to look at the error probability under that event.
The probability that there is at most c log n heads among those ten c log n flips.
So, the claim is this should be pretty small.
It's going to depend on ten.
Then I'll choose ten to be arbitrarily large, and I'll be done, OK, make my life a little bit easier.
Well, I would ask you normally, but this is 6.042 material.
So, what's the probability that we have, at most, this many heads?
Well, that means that nine c log n of the coins, because there are ten c log n flips, c log n heads at most, nine c log n at least better be tails.
So this is the probability that all those other guys become tails, which is already pretty small.
And then, there is this permutation thing.
So, if I had exactly c log n heads, this would be the number of ways to rearrange c log n heads among ten c log n coin flips.
OK, that's just the number of permutations.
So, this is a bit big, which is kind of annoying.
This is really, really small.
The claim is this is much smaller than that is big.
So, this is just some math.
I'm going to whiz through it.
So, you don't have to stay too long.
But you should go over it.
You should know that y choose x is, at most, ey over x to the x, good fact.
Therefore, this is, at most, ten c log n over c log n, also known as ten.
These cancel.
There's an e out here.
And then I raise that to the c log n power.
OK, then I divide by two to the power, nine c log n. OK, so what's this?
This is e times ten to the c log n divided by two to the nine c log n. OK, claim this is very big.
This is not so big, because I have a nine here.
So, let's work it out.
This e times ten, that's a good number, we can put upstairs.
So, we get log of e times ten, ten times, e, and then c log n. And then, we have over two to the nine c log n. So, we have this two to the c log n in both cases.
So, this is two to the log, ten e minus nine, c, log n: some basic algebra.
So, I'm going to set, not quite.
This is one over two to the nine minus log: so, just inverting everything here, negating the sign in here.
And, this is my alpha because the rest is n. So, this is one over n to the alpha when alpha is this particular value: nine minus log of ten times e times c. It's a bit of a strange thing.
But, the point is, as ten goes to infinity, nine here is the number one smaller than ten, right?
We subtracted one somewhere along the way.
So, as ten goes to infinity, this is basically, this is ten minus one.
This is log of ten times e. e doesn't really matter.
The point is, this is logarithmic in ten.
This is linear in ten.
The thing that's linear in ten is much bigger than the thing that's logarithmic in ten.
This is called abusive notation.
OK, as ten goes to infinity, this goes to infinity, gets bigger.
And, there is a c out here.
But, for any value of c that you want, whatever value of c you wanted in that claim, I can make alpha arbitrarily large by changing the constant in the big O, which here was ten.
OK, so that claim is true with high probability.
Whatever probability you want, which tells you alpha, you set a constant effort of the log N to be this number, which grows, and you're done.
You get the claim that is order log N heads, order log N flips with the high probability, therefore.
[None of the steps?]
in the search is order log N with high probability.
Really cool stuff; read the notes.
Sorry I went so fast at the end.
OK, good morning.
So today we are going to, as I mentioned last week, we've started the part of the course where we are doing more things having to do with design than purely analysis.
Today, we're actually going to do analysis, but it's the type of analysis that leads to really interesting design issues.
And we're going to follow it up on Wednesday with an application of the methods we're going to learn today with a really interesting and practical problem.
So we're talking today about amortized analysis.
And I want to motivate this topic by asking the question, how large should a hash table be?
So, how large should a hash table be?
Any suggestions?
You have got to make a hash table, how big should I make it?
Let's say it's a simple hash table, resolving collisions with chaining.
How big should it be?
Twice as big as you need: OK, how big would that be?
So, twice the number of elements, for example, OK. As I increase the size of a hash table, what happens to the search time?
What happens to search time as I increase the size of the hash table?
Yeah, but what does it, in general, do you?
It decreases, right?
OK, the bigger I make it, in fact, if I make it sufficiently large, then I essentially get a direct access table, and everything is worst-case, order one time.
So in some sense, we'll get back to your answer in a minute.
We should make it as large as possible, OK, so that the searching is cheap.
The flipside of that is what?
It takes a lot of space so I should make it as small as possible so as not to waste space.
So I want it big, and the happy medium, as we've discussed in our analysis, is to make it order n size for n items, OK, because making it larger than order n, the payoff in search time is not worth the extra amount of space that you are paying.
OK, or at least you can view it that way.
OK, however, this begs the question, which is, how do I make it, if I start out with a hash table, and I don't know how many elements are going to be hashed into it?
OK, how big should I make it?
OK, so what if we don't know -- -- in advance?
OK, what if we don't know n?
OK, so the solution to this problem turns out it's fairly elegant?
It's a strategy called dynamic tables.
OK, and the idea is that whenever the table gets too many elements in it, gets too full, OK, so the idea is -- -- OK, and we say that says the table overflows, OK, we grow it and make a bigger table.
So, for hashing, although there's going to be no point at which you could say that it overflows in the sense that it wouldn't be functional at least if it was done with chaining.
There would be, by the way, if you were doing it with open addressing.
But let's say with chaining, when it gets too big, say, as many elements as the size of the table, what we do is we grow the table.
So, the way we do that as we allocate using, in a language like C, it's called Malloc, or in a language like Java called New, a larger table.
So, we create a larger table.
We move the items from the old table to the new.
And then, we free the old table.
So, let's do an example.
So, let's say I have, over here, a table of size one, and it's empty to begin with.
And I do an insert.
So what I do is stick it in the table.
It fits.
OK, so here, I'm not going to do it with hashing.
I'm just going to do it as if I just had a table that I was filling up with elements to abstract the problem.
But it would work with hashing.
It would work with any kind of fixed size data structure.
I insert again, oops, doesn't fit.
I get an overflow.
OK, so what I do is I create a new, actually, I'm going to need a little more space than that.
I create a new table of size two, doubling the size.
And, I copy the old value into the new.
I freed this one, and now I can insert item two.
So, I do it again.
I get another overflow.
So now, I make a table of size four.
I copied these guys in, and then I insert my number three.
I do insert here.
I do five.
I guess I should be using ditto marks.
That would be a lot smarter.
Whoops, what am I doing?
I overflow.
And now, I make one of size eight, OK, copy these over, and now I can insert five.
And I can do six, seven, etc., OK?
Does everybody understand the basic idea?
So, whenever I overflow, I'm going to create a table of twice the size.
OK, so let's do a quick analysis of this.
So, we have a sequence of n insertion operations.
OK, what is the worst-case cost of one insert operation?
What's the worst case for any one of these?
Yeah, it's order n, whatever the overhead is of copying; if we counted it as one, it would be basically n or n plus one because we've got to copy all those.
OK, so it's order n. So therefore, if I have n of those, so the worst-case cost of n inserts is equal to n times order n, which is order n^2.
Any questions?
Does that make sense?
Raise hands.
Yeah, not all of them can be worst-case, good.
And in fact, this is totally wrong analysis.
Just because one can be worst-case order n doesn't mean n are necessarily order n. OK, so this is totally wrong analysis.
n inserts, in fact, take order n time in the worst case.
OK, it doesn't take order n^2.
So, the analysis is correct up to the point where we set the worst-case of one insert was order n. Therefore, that's the wrong step.
OK, whenever you see bugs in proofs, you want to know, which step is the one that failed so you can make sure that you don't have a confusion there?
So, let's do the proper analysis, OK?
So let's let c_i be the cost of the i'th insert.
OK, so that's equal to i, if i minus one is an exact power of two.
And it's one otherwise.
OK, so as I was going through here, it was only when I inserted something where the previous one had been the exact power of two, because that was my table size.
That's when I got the overflow and had to do all that copying.
And otherwise, the cost, for example, for inserting six, was just one.
I just inserted it.
Does everybody see that?
So, let's actually make a little table here so we can see this a little bit more clearly.
OK, so here's i, and in the size of the table at step i, and the cost at step i. OK, so let's see, the size of i, let's see, at step one it was one.
At step two, it was two.
And at step three, that's when, to get three in the table, we had to double the size here.
So, this is four, and four, it fit.
And then five, it had to bump up to eight.
And then, six, it was eight.
Seven, it was eight.
Eight, it was eight.
And nine, it bumps up to 16, 16, etc.
So that's the size.
And let's take a look at what the cost was.
So, the cost here was one, OK, to insert one.
The cost here was, I had to copy one, and then insert one.
So, the cost was two.
Here, I had to copy two and insert one.
So, the cost was three.
Here, I had to just insert one.
So, the cost was one.
Here, I had to copy four and insert one.
So, the cost was five.
Excuse me?
I think it is.
Yeah, see, it's i cost.
The cost for five is i, OK, is five if this is a power of two.
OK, one, one, and now we paid nine, and then one again.
So that's the cost we are paying.
It's a little bit easier to see what the costs are if I break them down.
OK, so let's just redraw this as two values because there is always the cost for inserting the one thing that I want to insert.
And now, the residual amount that I have to pay is I have to pay one here.
I've got to pay two additional, four additional, eight additional.
That makes the pattern a little bit easier to see.
OK, this is the cost of copying versus the cost of just doing the actual insert, OK?
Now, if you're taking notes, leave some space here because I'm going to come back to this table later.
OK, so leave a little bit of space because I'm going to come back and add some more things at there at a later time.
OK, so, I can then just add up the cost of n inserts.
That's just the sum, I equals one to n of c_i, which is equal to, well, by this analysis it is essentially n because that's what this thing adds up to plus I just have to add the powers of two up to but not exceeding whatever my n was.
So, if I do my algebra properly there, that's up to the floor of log n minus one, OK, of two to the J.
So, I'm just adding up all the powers of two up to that aren't going to exceed my n. And, this is what type of series?
That's geometric.
That's geometric, so it is bounded by its largest term.
Its largest term is two to the ceiling; it's dominated by its largest term, two to the ceiling of log n minus one, which is, at most, n. OK, and then all the other terms at up to, at most, n. So this is actually less than or equal to 3n, which is order n as we want it to show.
OK, that's algebra.
OK, so, thus, the average cost per insert is theta of n over n, which is theta one.
So, the average cost of an insert is order one, which is what we would like it to be especially if we're building hash tables.
Even though sometimes you have to pay a big price with amortized over the previous insertions that we've done, so that the overall cost of n operations is order n. And that's the notion of amortized analysis, OK, that if I look at a sequence of operations, I can spread the cost out over a whole bunch of operations, so that the average cost is order n. So, if we sort of summarize that, OK, OK, with basically an amortized analysis, we analyze a sequence of operations to show that the average cost per operation is small, even though one operation, or several, even, may be expensive.
OK, there's no probability.
Even though we're doing it with averages, there's no probability going on.
OK, what you do probability, and you are looking at means, there's averages.
OK, here's average, but there's no probability going on.
It's average performance in the worst case because n operations take me a constant amount of time per operation in the worst case.
n operations take me order n time.
OK, each operation takes order one time, OK, but it's amortized over the n operations, OK?
Yeah, question?
Yes.
Yes, yes, you can mix, but you don't have to.
Yeah, but the point is that the basic amortized analysis is actually saying something very strong.
It's giving you worst-case bounds, but over a sequence as opposed to looking at each individual element of the sequence.
Now, there are three types of amortized arguments that appear in the literature.
Maybe there are more.
At one point, there were two.
And then, a third one was developed.
So, maybe there's a fourth.
The first one is an aggregate, what's called an aggregate analysis.
And this is what we just saw, OK, where basically you just analyze, what do the n operations take?
OK, and then we're going to see two more today.
One is called an accounting argument, and the other is a potential argument.
These two are more precise because they allocate specific amortized costs to each operation.
So, one of the things about the aggregate analysis is that you can't really say what the amortized cost of a single operation is easily.
You can in this case.
You can say it's order one, OK?
But, in the accounting and potential arguments, it gives you a much more precise way of characterizing what an amortized cost of a particular operation is.
So, let's pitch in and look at the accounting method as our first method.
So, these we're going to go through the exact same example.
In some sense, this example, the easiest argument to make is the aggregate analysis.
OK, so we're going get into arguments that, in some sense, see more complicated.
But it turns out that these methods are more powerful in many circumstances.
OK, and so I want to do it in a simple situation where you have some sort of appreciation of the fact that you can look at any particular problem and approach it from different ways.
OK, so the accounting method is putting yourself in the position of a financial accountant.
So what you do is we are going to charge the i'th operation a fictitious amortized cost.
We'll call it c hat sub i, where we are going to use the abstraction that $1 pays for one unit of work.
There's Time manipulating the data structure or whatever.
OK, so the idea is you charge the cost.
You say, this operation will cost you $5 or whatever.
OK, and that phi is consumed to perform the operation, but there may be some unused part.
So, if there's any unused amount, it's going to be stored in the bank for use by later operations.
So the idea is that if the phi that is being paid, the c_i hat phi, isn't sufficient to pay for performing the operation, then you take money out of the bank to pay for it.
OK, and so you don't get arrested, what's the property that you've got to have?
You've got to have the bank balance.
What about the bank balance?
What mathematical fact has to hold the bank balance?
Yeah, it better be greater than or equal to zero, right?
Most people are familiar with that.
So, the bank balance must not go negative.
In other words, the amortized costs minus the costs of the operations up to that point have to always be enough to pay for all the operations that you're doing.
Otherwise, you're borrowing on the future.
In amortized analysis, we don't borrow on the future, at least not on the simple ones that we are doing here.
OK, so that means we must have that the sum, I equals one to n of c_i, the true costs, therefore, if the balance is not going to ever go negative, must be bounded above by the amortized costs for all n. OK, for the bank balance not to go negative, if I add up the true costs, it's got to be the case that I can always pay for them.
This is what I'm charging.
This is what it actually costs me.
So, it better be the case that whatever I've actually had to pay to operate on that data structure, that's what this is, better be covered by the amount that I've been charging people for the use of that data structure up to that point.
And that's got to be true for all n. But notice that this now gives me a way of charging a particular operation a certain amount.
So, the total amortized costs provide an upper bound on the total true costs.
Total amortized costs are an upper bound on the true costs.
Any question about this?
That we'll do the example of the dynamic table using this methodology.
OK, so, back to the dynamic table.
OK, so what we're going to do in this case is we're going to charge an amortized cost of $3 for the i'th insert for all i.
And the idea is that $1 is going to pay for an immediate insert, and $2 is going to be stored for doubling the table.
And, it needs to be expanded.
When the table doubles, of the stored dollars, we'll use one dollar to move a recent item, I'll call it, and one dollar we'll move an old item.
So, let's do the example.
So, imagine that I'm in this situation where I have a table of size eight, and I just doubled my table.
So I have four items of the table.
What I'm going to do is have no dollars in my table.
So, along comes an insertion of item number five.
I charge $3 for it.
$1 lets me put the item in the table, and I have $2 left over.
So let me store those $2 in the slot corresponding to where that item is.
Now, item six comes in.
Once again, $1, charge $3, $1 is paid for the insert, $2 left over, I'm going to play $2.
Let me put it down there, and so forth.
The next one comes in, $2, $2 leftover, and now the ninth item comes in.
So, I double the size of my table.
OK, and now I copy all of these guys and to all of these here.
And what happens?
Look at that: I've got $8, and I've got eight items that have to be copied.
Perfect.
OK, so one of these dollars pays for one of the ones that was inserted in the last round, and one of them pays for an old one.
OK, and so, I copy them in, and now, none of those guys have any money.
And the ninth guy comes in: he has $2 left over.
And then, we keep going on.
OK, so you see that by that argument, if I charge everybody $3, OK, I can always handle all of the table doubling, the charges for the table doubling because the inductive invariant that I've maintained is that after it doubles, there's nothing in the bank account.
And now, I put in $2.
Well then, I can pay, and I'm now left in the same situation.
OK, and it's the case that the bank balance never goes negative.
So that's a really important invariant to verify.
And so, therefore, the sum of the true costs, or the amortized costs, upper bound the sum of the true costs.
And, since the sum of the amortized cost, here, is, if I go i equals one to n, OK, this is 3n.
So, the point is, now I bounded the sum of the true costs by 3n.
OK, so let's go back to this table here, and look to see what happens, OK, if I put in c_i hat, and the bank balance.
OK, so in fact, so the first thing I do is insert; I charge $3, right, and I do an insert.
How much do I have left?
I'm going to have $2.
It turns out I'm actually going to charge $2 and have only $1 left.
OK, so I'm actually going to under charge the first guy.
I'm going to show you that it works if I charge everybody $3.
Except the first guy: I charge $2.
I can actually save a little bit on number one.
OK, that for this guy I'm going to charge $3.
OK, what's the size of my bank balance when I'm done?
Well, I have to copy one guy.
He's all paid for, so I have $2 left.
OK, people with me?
OK, the next guy: I charge $3.
Actually, I'm going to charge all these guys $3.
OK, so here now I basically get to, I've got a table of size four.
So, I basically have, I have to copy, oh, when I insert the third guy, I've got to copy two guys.
That'll use up that, so I'll have only $2 left in the table after I've inserted him.
OK, now I insert the fourth guy, OK, and that's a good one because now I've built up a balance here of $4 because I didn't have to copy anybody.
OK, now I insert the fifth guy.
I've got to copy four items.
So that expends that balance.
I have, then, two left.
OK, and then here basically I add two to it.
And then at this point, I use it all up and go back to two, etc.
OK, so you see one of the things I want you to notice is I could have charged three here.
And then I would've had an extra dollar lying around throughout here.
It wouldn't have mattered.
It still would be upper bounded by 3n.
OK, so the idea is that different schemes for charging amortized costs can work.
They don't all have to be the same.
It's not like when you do in amortized analysis that there is one scheme that will work.
I could have charged $4 to everybody.
And it would have worked.
But it turns out, I couldn't have charged two dollars for everybody.
If I charged $2 for everybody, my balance would go negative, OK?
My balance would go negative, but I can charge three dollars, and that will work.
OK, four, five, six, I could charge that.
The bound that I would get would be simply a looser bound.
Instead of it being less than or equal to 3n, it would be less than or equal to 4n or 5n, or what have you.
But if I tried to do 2n, it wouldn't have worked because I wouldn't have enough money left to copy everything.
What would happen is I would have only $1 in this, and then when it came time to table double, I would need to copy eight guys.
And I'd only have built up a bank account of $4, sorry, if I charged $2 and had $1 left over.
OK, so to actually make these things work out, you have to play a little bit, OK, see what works, see what doesn't work.
OK, no algorithmic formulas for algorithm design.
OK, good.
In the book, you can read about table contraction.
What happens when you start deleting elements?
Now you want to make the table be smaller.
Now, you have to be very careful because unless you put, who remembers from physics, hysteresis?
Vaguely?
A couple people?
OK, you have to worry about hysteresis.
OK, if you're not careful, if whenever it gets to be less than a power of two, you go in the half, you can find that you're thrashing.
So, you need to make it so that there is some memory in the system so that you only collapse after you've done a sufficient number of deletions, OK, and so forth.
And the book has analysis of the more general case.
OK, so any questions about the accounting method?
Accounting method is really very cute, OK, very cute.
And, it's the one most students prefer to do, OK?
They usually hate the next one until they learn it.
Once they learn it, they say, ooh, that's cool.
OK, but to start out with, it takes a little bit more intestinal fortitude, OK?
But it's amazing.
Good potential arguments are really sweet.
And we are going to see one next time, so you'll definitely want to review and make sure you understand it before Wednesday's lecture because Wednesday's lecture, we're going to assume we understand potential method.
OK, so let's do, enough advertisement.
I think the potential method is one of the beautiful results in algorithmic analysis, OK, just beautiful result, beautiful set of techniques.
OK, and it's also, just in terms of, I mean, what do you aspire: to be a bookkeeper or to be a physicist?
OK, so, the idea is we don't want to be bankers.
We want to be physicists.
And so, this bank account, we are going to say about the potential energy of the dynamics that that we are analyzing.
OK, because, why?
It delivers up work just like a spring does, for example, OK, when you study potential energy, or putting something up high and having gravity pull it down.
We convert dynamic to potential, and that's exactly what we're going to be doing here, and it's similar mathematics except that in our case it turns out to be discrete mathematics rather than continuous math for most of it.
So here's the framework for the potential method.
So, we start with some data structure, D_0, and operation i transforms D_(i - 1) into D_i.
So, we view the operation on the data structure as a mapping, mapping one data structure to another data structure, the one from before to the one after.
OK, already it's nicely mathematical.
OK, and of course, the costs of operation i remains at c_i.
And, now what we are going to do is define the potential function, phi, which maps the set of data structures into the reals.
So, associated with every data structure, now, is a potential, OK, a real-valued potential, often integer potential such that phi of D_0 is equal to zero.
So, the initial potential is zero, and phi of D_i is greater than or equal to zero for all i.
So, potential can never be nonnegative, just like the bank account because the potential is essentially representing the bank account, if you will, in the accounting method.
OK, so we always want the potential to be nonnegative.
Now, actually, there are times where you use potential functions where you violate both of these properties.
There's some really interesting potential arguments which don't violate like these, but for the simple ones we're going to be doing in this class, we'll just assume that these tend to be true.
OK, but we will actually see some times where phi of D_0 isn't zero, it doesn't matter.
OK, but generally this is what we are going to assume in the type of potential function argument that we are going to be doing.
OK, so I just want to let you know that there are bigger, there is a bigger space of potential function arguments than the one that I'm showing you here.
OK, so then, under this circumstance, we define the amortized cost c_i hat with respect to phi as, and this is one of these formulas that if you can't remember it, definitely put it down on your crib sheet for the final.
OK, so c_i hat is equal to c_i plus phi of D_i minus phi of D_i minus one.
OK, so this is the change in potential difference.
And, let's call it delta phi i for shorthand.
OK, and let's see what it means to have -- -- in the different circumstances.
So, if delta of phi i is greater than zero, OK, so if this is greater than zero, then what's the relationship between c_i hat and c_i?
This is greater than zero.
Yeah, c_i hat is then greater than c_i, OK?
Then, c_i hat is greater than c_i.
And what does that mean?
That means when I do operation I, I charged more than it cost me to do the operation.
So, the extra amount that I charged beyond what I actually used is being put into the bank, OK, is being stored as potential energy.
So, op I stores work in the data structure for later.
Similarly, if delta phi i is less than zero, then c_i hat is less than c_i.
And so, the data structure delivers up work -- -- to help pay for op I, OK, for operation I.
So, if it's less than zero, that means that my change in potential, that means my bank account went down as a result, and so therefore, what happens was the data structure provided work to be done in order, because the true cost was bigger than the amortized cost.
So, if you think about it, the difference between looking at it from the potential function point of view versus the accounting point of view, the accounting point of view, you sort of say, here is what my amortized cost will be.
Now let me analyze my bank account, make sure it never went negative.
In some sense, in the potential function argument, you're saying, here's what my bank account is all the time.
Now let me analyze what the amortized costs are.
So, that's sort of the difference in approaches.
One is you are sort of specifying the bank account.
The other, you're specifying the amortized costs.
So, we look at the, why is it that this is a reasonable way to proceed?
Well, let's look at the total amortized cost of n operations.
OK, that's just the sum, i equals one to n of c_i hat.
That's the total amortized cost.
And that's equal to, but substitution, just substitute c_i hat for this formula.
OK, so that's c_i plus phi of D_i minus phi of D_i minus one.
OK, and that's equal to c_i.
And now, what happens when I sum up these terms?
What happens when some of these terms?
What's the mathematical term we use?
It telescopes.
OK, every term on the left is added in once when it's I, and subtract it out when it's I minus one, except for the first and last terms.
The term for n is only added in, and the term for zero is only subtracted out.
OK, so that's because it telescopes.
OK, so this term is what?
What property do we know of this?
It's greater than or equal to zero.
And this one equals zero.
So, therefore, this is greater than or equal to c_i.
And thus, the amortized costs are an upper bound on the true costs, which is what we want.
Some of the amortized costs is an upper bound up on the sum of the true costs.
OK, but here, the way that we define the amortized costs was by first defining the potential function.
OK, so the potential function is sort of, as I said, the difference between the accounting and the potential method is, do you specify the bank account or do you specify the cost?
OK, do you specify the potential energy at any point, or do you specify the cost at any point?
OK, but in any case, you get, this bound, also this math is nicer math.
I like telescopes.
OK, so the amortized costs upper bound the true costs.
OK, let's do table doubling over here.
So, to analyze this, we have to define our potential.
OK, if anybody can guess this off the top of their head, they're better than I am.
I struggled with this for probably easily a couple hours to get it right, OK, because I'm not too smart.
OK, that's a potential function I'm going to use, OK, 2i minus two to the ceiling of log i.
And, we're going to assume that two to the ceiling of log of zero is equal to zero, because that's what it is in the limit.
For log of zero, this becomes minus infinity, so, two to the minus infinity is zero.
So, that's just going to be a mathematical convenience.
Assume that.
OK, so where did I get this from?
I played around.
I looked at that sequence that I have erased and I said, OK, because reversing, there are some problems for which defining a potential function is fairly easy.
But, defining the amortized costs is hard, OK, to define the accounting.
So, for this one, the accounting method is, I would say, an easier method to use.
However, I'm going to show you that you still can do it with potential method if you come up with the right potential.
So, intuitively, this is basically what's left in the bank account at the I'th operation because I've put in 2i things into the bank, and I've subtracted out this many, essentially, from table doublings, OK, up to that point, OK?
So, first let's observe, what is phi of D_0?
Zero.
So, that's good.
And, the phi of D_i is greater than or equal to zero.
Why is that?
Why is that?
So, what's the biggest that ceiling of log i could be?
Ceiling of log i is either log i or log i quantity plus one, OK?
So, the biggest it is, is log i plus one.
If it's log i plus one, two to the log i plus one, it's just 2i.
That's the biggest it could be, right?
So, two, the log of i, plus one, is just, let's do it the other way, is i times two, OK, i for that part, two for that part.
OK, so that the biggest it could be.
OK, or it's just log of i.
So, either this is going to be 2i minus i or 2i minus 2i.
In either case, it's bigger than zero, OK?
So, those are the two properties I need for this to be a well-defined potential function.
Now, that doesn't say that the amortized costs are going to be satisfied the property that things are going to be cheap, OK, that I'm going to be able to do my analysis and get the kind of bounds I want.
But, it sets up to say, yes, I've satisfied the syntax of having a proper potential function.
So, let's just do a quick example here just to see what this means.
So, imagine that I am in the situation where I have eight, did I do that right, OK, yeah, eight slots, and say six of them are full.
So then, phi by this is going to be 2i.
That's two times six minus two to the 2i.
What's that?
Sorry, minus two to the ceiling of log i.
So, i is six, right, so log of i is log of six.
The ceiling of it is three.
So, that's minus 2^3, which is eight.
So, that's 12 minus eight.
That's four.
And if you think about this in the accounting method, these would be zeros, and these would be twos, right, for the accounting method if we do the same thing, because this is halfway through, right, all zeros, and that we add two for each one that we're going in.
So, I function is, in fact, telling me what the actual cost is.
OK, everybody with me?
OK, so that's what we mean by this particular potential function.
OK, so now let's add up the amortized cost of the i'th insert.
OK, so that's the amortized cost of the i'th insert, just by definition.
OK, and now that's equal to, well, what is c_i?
Do we still have that written down somewhere, or have we erased that at this point?
I think we erased it.
Well, we can write it down again.
It is i if i minus one is an exact power of two.
And, it's one otherwise.
That's c_i.
That's this term, plus, and now phi of D_i: so, what is that?
phi of D_i is this business, 2i minus two ceiling of log i minus 2i minus one minus two ceiling of log of I minus one.
OK, so that's the amortized cost.
That's a nice, pretty formula, right?
OK, let's hope it simplifies a little.
OK, so that's equal to, well, we have the i and the one here, if, etc., that business, plus, OK, well, we have some things we can cancel here, right?
So here, we have 2i minus 2i.
That cancels.
And then we have what's left over here is a minus two.
So, that's a plus two.
And now, I have minus this term plus this term.
That's a lot prettier.
OK, it's still a mess.
We've got to case analysis.
Why is it suggestive of a case analysis?
We have a case.
OK, so let's do a case analysis.
OK, so case one, I minus one is an exact power of two.
So then, c_i hat is equal to, well, c_i is now just i.
That's that case.
And then we have the rest there, plus two, minus two, ceiling of log i minus two ceiling of log of i minus one.
OK, and that's equal to i plus two.
Well, let's see.
If I minus one is an exact power of two, what is this term?
i minus one is an exact power of two.
Plus, thank you, sorry about that.
It's good to have students, let me say, because, boy, my math is so bad.
This is actually why I end up being a pretty good theoretician is because I don't ever trust what I've written down.
And so, I write it down in a way that I can verify it because otherwise I just am not smart enough to carry through five equations in a row and expect that everyone is going to be transformed appropriately.
OK, so you write it down.
So I always write it down so I can verify it.
And that fortunately has the side benefit that other people can understand what I've done as well.
OK, so what is this one?
This is two to the log of i minus one because the ceiling, if this is an exact power of two, right, then ceiling of log of i minus one is just log of i minus one.
So, this is two to the log of i minus one, which is i minus one, right?
Yeah?
OK. OK, if it's an exact power of two, then the log is an integer, right?
So, taking the ceiling, it doesn't matter, get rid of the ceiling.
OK, this one, however, is not an exact power of two.
But what is it?
It's just one more than this guy.
We know that i minus one is not an exact power of two, so it's going to be the next bigger one.
OK, so that means this is what?
So, it's going to be, how do these two compare?
How much bigger is this one than this one?
It's twice the size.
We know what this one is.
OK, or it can reason it from first principles.
This is going to be the log of i minus one plus one.
OK, and so then you can reduce it to this.
So, you've got to think about those floors and ceilings, right?
It's like, what's happening in the round off there?
OK, so now we can simplify this.
OK, so what do we have here?
We have, OK, so if I multiply this through, I have an i plus two minus 2i plus two plus i minus one.
I know a lot of you, probably 90% of you will do this step.
You will go directly from this step to the last step.
And that's where 30% of you, or some number, will get it wrong.
OK, so let me encourage you to do that step.
OK, it's easier to find your bugs if you take it slow.
Actually, taking it slow is faster in the long run.
It's hard to teach young stallions, or phillies, or whatever, OK?
Just take it easy.
Just patience, just do it slow, get it right.
It's actually faster.
OK, everybody knows the tortoise and hare story.
Yeah, yeah, yeah, OK, but nobody believes it.
OK, so now here we have 2i here, an i here, and an i here.
And then, that leaves us with two plus two minus one, equals three.
Awesome, awesome.
OK, amortized cost is three when i minus one is an exact power of two.
OK, case two.
OK, i minus one is not an exact power of two.
So then we have c_i hat is equal to, now instead of i it's one plus, and then the two minus two to the ceiling of log i plus two to the ceiling of log of i minus one.
OK, now what can somebody tell me about these two terms in the case where i minus one is not an exact power of two?
What are they?
Equal.
Why is that?
Yeah, the ceiling is going to do the same thing to both, going to take it up to the same integer.
So these two things are equal, which means this is equal to three.
OK, so therefore, n inserts, OK, so now I say, oh, the amortized cost is three for every operation for every insert.
So therefore, n inserts costs, well, the amortized cost of each is three.
So n of them, the amortized cost is 3n.
That's an upper bound on the worst-case true costs.
So, n inserts costs order n in the worst case.
OK, there is a bug in this analysis.
It's a minor bug.
It's the one I pointed out before the first insert has amortized cost of two, and not three.
I didn't actually deal with that one carefully enough.
OK, so that's an exercise to just go and look to see where it is that that happens and how you show that, in fact, the amortized cost of the first one is two, OK, where that shows up.
OK, so to summarize, actually let me summarize over here, conclusions about amortized analysis.
So amortized costs provide a clean abstraction for data structure performance.
So, what I can tell somebody, so suppose I built a dynamic table, for example, OK.
It's easier to say, in terms of your own performance modeling, it costs a constant amount of time for each insert.
As long as you don't care about real-time behavior, but only the aggregate behavior, that's a great abstraction for the performance Rather than saying it's got that complicated thing which sometimes cost you a lot, how do they reason about that?
But you could say every operation costs me order one, that's really simple.
But they have to understand, it's order one in an amortized sense, OK.
So, if they do have a real-time constraint to make, amortized doesn't cut it.
OK, but for many problems, it's perfectly good.
It lets me explain it rather simply.
OK, we will see some other data structures that have amortized costs where different operations have different amortized costs.
And the nice thing about that is I just add up, what's the cost of all my different operations, OK, where there is a different cost for each operation?
Some will be log n. Some will be order one, or whatever.
Add them up: that's an upper bound on the true costs
tremendous simplification in abstracting and reasoning about those complicated data structures.
OK, now, so this is probably, this is huge.
OK, abstraction, you know, computer science, what teach you through four years of undergraduate, and another year if you go on to M.Eng., and then if you get a Ph.D., it's another 15 years or whatever it takes to get a Ph.D. OK, all you teach about is abstraction: abstraction, abstraction, abstraction.
So, this is a powerful abstraction: quite good.
Now, we learned three methods.
In general, any method can be used.
You can convert one to the other.
But each has situations where it is arguably simplest or most precise.
So, any of the methods can be used.
However, you must learn all of them, OK, because there are going to be some situations where you need one, where it's better to do one, better to do another.
If you're reading in the literature, you want to know these different ways of doing it.
And that means that even though you may get really comfortable with accounting, OK, in an exam, or whatever, I may say solve this with a potential function argument.
So, you want to be comfortable with all the methods, OK. Last point is that, in fact, different potential functions or accounting costs may yield different bounds.
OK, so when you do an amortized analysis, there's nothing to say that one set of costs is better than another.
So, just as an example, OK, in any data structure generally that supports delete, I can amortize all the deletes against the inserts.
So, in general, what I could do is say deletes are free, OK, and charge twice as much for each insert.
OK, charging enough when I do the insert to amortize it against the delete.
That you could do with an accounting method.
You can also do it with a potential method, the potential, then, being the number of items that I actually have in my data structure times the cost of the delete for each of those items.
OK, so the point is that I can allocate costs in different ways.
Or I could have amortized costs which are equal to the true costs.
So, there are different ways that I could assign amortized costs.
There is no one way, OK, and choosing different ones may yield different bounds.
It may not, but it may yield different bounds.
OK, generally it does yield different ones.
OK, next time: an amazing use of potential functions.
OK, the stuff is cool, but let me tell you, Wednesday's lecture: amazing.
Amazing the type of analysis we are going to be able to do.
And this is going to use some of the techniques we learned last time with respect to amortized analysis.
And, what's neat about what we're going to talk about today is it's a way of comparing algorithms that are so-called online algorithms.
And we're going to introduce this notion with a problem which is called self organizing lists.
OK, and so the set up for this problem is that we have a list, L, of n elements.
And, we have an operation.
Woops, I've got to spell things right, access of x, which accesses item x in the list.
It could be by searching, or it could be however you want to do it.
But basically, it goes and touches that element.
And we were to say the cost of that operation is whatever the rank of x is in the list, which is just the distance of x from the head of the list.
And the other thing that we can do that the algorithm can do, so this is what the user would do.
He just simply runs a whole bunch of accesses on the list, OK, accessing one element after another in any order that he or she cares to.
And then, L can be reordered, however, by transposing adjacent elements.
And the cost for that is one.
So, for example, suppose the list is the following.
OK, I missed something here.
It doesn't matter.
Well, I'll just make it be what I have so that it matches the online video.
OK, so here we have a list.
And so, if I do something like access element 14 here, the element of key 14, OK, that this costs me one, two, three, four.
So here the cost is four to access.
And so, we're going to have some sequence of accesses that the user is going to do.
And obviously, if something is accessed more frequently, we'd like to move it up to the front of the list so that you don't have to search as far.
OK, and to do that, if I want to transpose something, so, for example, if I transpose three and 50, that just costs me one.
OK, so then I would make this be 50, and make this be three.
OK, sorry, normally you just do it by swapping pointers.
OK, so those are the two operations.
And, we are going to do this in what's called an online fashion.
So, let's just define online.
So, a sequence, S, of operations is provided one at a time for each operation.
An online algorithm must execute the operation immediately without getting a chance to look at what else is coming in the sequence.
So, when you make your decision for the first element, you don't get to see ahead as to what the second, or third, or whatever is.
And the second one you get, you get and you have to make your decision as to what to do and so forth.
So, that's an online algorithm.
Similarly, an off-line algorithm, OK, may see all of S in advance.
OK, so you can see an off-line algorithm gets to see the whole sequence, and then decide what it wants to do about element one, element two, or whatever.
OK, so an off-line algorithm can look at the whole sequence and say, OK, I can see that item number 17 is being accessed a lot, or early on move him up closer to the front of the list, and then the accesses cost less for the off-line algorithm.
The online algorithm doesn't get to see any of that.
OK, so this is sort of like, if you're familiar with the game Tetris.
OK, and Tetris, you get one shape after another that starts coming down, and you have to twiddle it, and move it to the side, and drop it into place.
And there, sometimes you get a one step look-ahead on some of them so you can see what the next shape is, but often it's purely online.
You don't get to see that next shape or whatever, and you have to make a decision for each one.
And you make a decision, and you realize that the next shape, ah, if you had made a different decision it would have been better.
OK, so that's the kind of problem.
Off-line Tetris would be, I get to see the whole sequence of shapes.
And now let me decide what I'm going to do with this one.
OK, and so, in this, the goal is for any of the algorithms, either online or off-line is to minimize the total cost, which we'll denote by, I forgot to name this.
This is algorithm A here.
The total cost, C_A of S, OK, so the cost of algorithm A on the sequence, S. That's just the notation we'll use for what the total cost is.
So, any questions about the setup to this problem?
So, we have an online problem.
We're going to get these things one at a time, OK, and we have to decide what to do.
So, let's do a worst-case analysis for this.
OK, so if we're doing a worst-case analysis, we can view that we have an adversary that we are playing against who's going to provide the sequence.
The user is going to be able to see what we do.
And so, what's the adversary strategy?
Thwart our plots, yes, that's his idea.
And how is he going to thwart them, or she?
Which is what for this problem?
What's he going to do?
Yeah.
No matter how we reorder elements using the transposes, he's going to look at every step and say, what's the last element?
That's the one I'm going to access, right?
So, the adversary always, always accesses the tail element of L. No matter what it is, no matter how we reorder things, OK, for each one, adversary just accesses the tail.
So the cost of this, of any algorithm, then, is going to be omega size of S times n, OK, because you're always going to pay for every sequence.
You're going to have to go in to pay a cost of n, OK, for every element in the sequence.
OK, so not terribly in the worst-case.
Not terribly good.
So, people in studying this problem: question?
That analysis is for the online algorithm, right.
The off-line algorithm, right, if you named those things, that's right.
OK, so we're looking at trying to solve this in an off-line sense, sorry, in an online sense.
OK, and so the point is that for the online algorithm, the adversary can be incredibly mean, OK, and just always access the thing at the end, OK?
So, what sort of the history of this problem is, that people said, well, if I can't do well in the worst-case, maybe I should be looking at average case, OK, and look at, say, the different elements having some probability distribution.
OK, so the average case analysis, OK, let's suppose that element x is accessed with probability, P of x. OK, so suppose that we have some a priori distribution on the elements.
OK, then the expected cost of the algorithm on a sequence, OK, so if I put all the elements into some order, OK, and don't try to reorder, but just simply look at, is there a static ordering that would work well for a distribution?
It's just going to be, by definition of expectation, the probability of x times, in this case, the cost, which is the rank of x in whatever that ordering is that I decide I'm going to use.
OK, and this is minimized when?
So, this is just the definition of expectations: the probability that I access x times the cost summed over all the elements.
And the cost is just going to be its position in the list.
So, when is this value, this summation, going to be minimized?
When the element is most likely as the lowest rank, and then what, what about the other element?
OK, so what does that mean?
Yeah, sort them, yeah, sort them on the basis of decreasing probability, OK?
So, it's minimized when L is sorted, OK, in decreasing order with respect to P. OK, so just sort them with the most likely one at the front, and then just decreasing probability.
That way, whenever I access something with some probability, OK, I'm going to access, it's more likely that I'm going to access.
And that's not too difficult to actually prove.
You just look at, suppose there were two that were out of order, and show that if you swap them, you would improve this optimization function.
OK, so if you didn't know it, this suggests the following heuristic, OK, which is simply keep account of the number of times each element is accessed, and maintain the list in order of decreasing count.
OK, so whenever something is accessed, increment its count, OK, and that will move it, at most, one position, which only costs me one transposed to move it, perhaps, forward.
OK, actually, I guess it could be more if you have a whole bunch of ties, right?
Yeah.
So, it could cost more.
But the idea is, over time, the law of large numbers says that this is going to approach the probability distribution.
The frequency with which you access this, divided by the total number of accesses, will be the probability.
And so, therefore you will get things in decreasing probability, OK, assuming that there is some distribution that all of these elements are chosen according to, or accessed according to.
So, it doesn't seem like there's that much more you could really do here.
And that's why I think this notion of competitive analysis is so persuasive, because it's really amazingly strong, OK?
And it came about because of what people were doing in practice.
So practice, what people implement it was a so-called move to front heuristic.
OK, and the basic idea was, after you access an element, just move it up to the front.
OK, that only doubles the cost of accessing the element because I go and I access it, chasing it down paying the rank, and then I have to do rank number of transposes to bring it back to the front.
So, it only cost me a factor of two, and now, if it happens to be a frequently accessed elements, over time you would hope that the most likely elements were near the front of that list.
So, after accessing x, move x, the head of the list using transposes, and the cost is just equal to twice the rank in L of x, OK, where the two here has two parts.
One is the access, and the other is the transposes.
OK, so that's sort of what they did.
And one of the nice properties of this is that if it turns out that there is locality in the access pattern, if it's not just a static distribution, but rather once I've accessed something, if it's more likely I'm going to access it again, which tends to be the case for many input types of patterns, this responds well to locality because it's going to be up near the front if I access it very soon after I've accessed.
So, there is what's called temporal locality, meaning that in time, I tend to access, so it may be that I access some thing's very hot for awhile; then it gets very cold.
This type of algorithm responds very well to the hotness of the accessing.
OK, so it responds well to locality in S. So, this is sort of what was known up to the point that a very famous paper was written by Danny Sleator and Bob Tarjan, where they took a totally different approach to looking at this kind of problem.
OK, and it's an approach that matter you see everywhere from analysis of caching and high-performance processors to analyses of disk paging to just a huge number of applications of this basic technique.
And, that's the technique of competitive analysis.
OK, so here's the definition.
So, online algorithm A is alpha competitive.
If there exists a constant, k, such that for any sequence, S, of operations, the cost of S using algorithm A is bounded by alpha times the cost of opt, where opt is the optimal offline algorithm.
OK, so the optimal off-line, the one that knows the whole sequence and does the absolute best it could do on that sequence, OK, that's this cost here.
This is sometimes called God's algorithm, OK, not to bring religion into the classroom, or to offend anybody, but that is what people sometimes call it.
OK, so the fully omniscient knows absolutely the best thing that could be done, sees into the future, the whole works, OK?
It gets to apply that.
That's what opts algorithm is.
And, what we're saying is that the cost is basically whatever this alpha factor is.
It could be a function of things, or it could be a constant, OK, times whatever the best algorithm is.
Plus, there's a potential for a constant out here.
OK, so for example, if alpha is two, and we say it's two competitive, that means you're going to do, at worst, twice the algorithm that has all the information.
But you're doing it online, for example.
OK, it's a really pretty powerful notion.
And what's interesting about this, it's not even clear these things should exist to my mind.
OK, what's pretty remarkable about this, I think, is that there is no assumption of distribution, of probability distribution or anything.
It's whatever the sequence is that you give it.
You are within a factor of alpha, essentially, of the best algorithm, OK, which is pretty remarkable.
OK, and so, we're going to prove the following theorem, which is the one that Sleator and Tarjan proved.
And that is that MTF is four competitive for self organizing lists.
OK, so the idea here is that suppose the adversary says, oh, I'm always going to access the thing at the end of the list like we said in the beginning.
So, the adversary says, I'm always going to access the thing there.
I'm going to make MTF work really bad, because you're going to go and move that thing all the way up to the front.
And I'm just going to access the thing way at the end again.
OK, well it turns out, yeah, that's a bad sequence for move to front, OK, and it will take a long time.
But it turns out God couldn't have done better, OK, by more than a factor of four no matter how long the list is.
OK, that's pretty amazing.
OK, so that's a bad sequence.
But, if there's a way that the sequence exhibits any kind of locality or anything that can be taken advantage of, if you could see the whole thing, MTF takes advantage of it too, OK, within a factor of four.
OK, it's a pretty remarkable theorem, and it's the basis of many types of analysis of online algorithms.
Almost all online algorithms today are analyzed using some kind of competitive analysis.
OK, not always.
Sometimes you do probabilistic analysis, or whatever, but the dominant thing is too competitive analysis because then you don't have to make any statistical assumptions.
OK, just prove that it works well no matter what.
This is remarkable, I think.
Isn't it remarkable?
So, let's prove this theorem, we're just going to spend the rest of the lecture on this proof.
OK, and the proof in some ways is not hard, but it's also not necessarily completely intuitive.
So, you will have to pay attention.
OK, so let's get some notation down.
Let's let L_i be MTF's list after the i'th access.
And, let's let L be opt's list after the i'th access.
OK, so generally what I'll do is I'll put a star if we are talking about opt, and have nothing if we're talking about MTF.
OK, so that's going to be the list.
So, we can say, what's the list?
So, we're going to set it up where we have one in operation that transforms list i minus one into list i. OK, that's what the i'th operation does.
OK, and move to front does it by moving whatever the thing that was accessed to the front.
And opt does whatever opt thinks is the best thing to do.
We don't know.
So, we're going to let c_i be MTF's cost for the I'th operation.
And that's just twice the rank in L_i minus one of x if the operation accesses x, OK, two times the rank in L_i minus one because we're going to be accessing it in L_i minus one and transforming it into L_i.
And similarly, we'll let c_i star be opt's cost for the i'th operation.
And that's just equal to, well, to access it, it's going to be the rank in L_i minus one star, whatever its list is of x at that step, because it's got to access it.
And then, some number of transposes, t_i if opt forms t_i transposes.
OK, so we have the setup where we have two different lists that are being managed, and we have different costs in the list.
And, we're interested in is comparing in some way MTF's list with opt's list at any point in time.
And, how do you think we're going to do that?
What technique do you think we should use to compare these two lists, general technique from last lecture?
Well, it is going to be amortized, but what?
How are we going to compare them?
What technique did we learn last time?
Potential function, good.
OK, we're going to define a potential function, OK, that measures how far apart these two lists are.
OK, and the idea is, if, let's define that and then we'll take a look at it.
So, we're going to define the potential function phi mapping the set of MTF's lists into the real numbers by the following.
phi of L_i is going to be twice the cardinality of this set.
OK, so this is the precedes-operation and list, i.
So, we can define a relationship between any two elements that says that x precedes y in L_i if, as I'm accessing it from the head, I hit x first.
OK, so what I'm interested in, here, are in some sense the disagreements between the two lists.
This is where x precedes y in MTF's list, but y precedes x in opt's list.
They disagree, OK?
And, what we're interested in is the cardinality of the set.
And we're going to multiply it by two.
OK, so that's equal to two times; so there is a name for this type of thing.
We saw that when we were doing sorting.
Anybody remember the name?
It was very briefly.
I don't expect anybody to remember, but somebody might.
Inversions: good, OK, twice the number of inversions.
So, let's just do an example.
So, let's say L_i is the list with five elements.
OK, I'll use characters for the order just to keep things simple.
So, in this case phi of L_i is going to be twice the cardinality of the set.
So what we want to do is see which things are out of order.
So here, I look at E and C are in this order, but C and E in that order.
So, those are out of order.
So, that counts as one of my elements, EC, and then, E and A, A and E. OK, so those are out of order, and then ED, DE, out of order, and then EB, BE, those are out of order.
And now, I go C, A, C, A.
Those are in order, so it doesn't count.
CD, CD, CB, CB, so, nothing with C. Then, A, D, A, D, those are in order, A, B, A, B, those are in order.
So then, DB, BD, so BD.
And that's the last one.
So, that's my potential function, which is equal to, therefore, ten, because the cardinality of the set is five.
I have five inversions, OK, between the two lists.
OK, so let's just check some properties of this potential function.
The first one is notice that phi of L_i is greater than or equal to zero for all i.
The number of inversions might be zero, but is never less than zero.
OK, it's always at least zero.
So, that's one of the properties that we normally have we are dealing with potential functions.
And, the other thing is, well, what about phi of L0?
Is that equal to zero?
Well, it depends upon what list they start with.
OK, so what's the initial ordering?
So, it's zero if they start with the same list.
Then there are no inversions.
But, they might start with different lists.
We'll talk about different lists later on, but let's say for now that it's zero because they start with the same list.
That seems like a fair comparison.
OK, so we have this potential function now that's counting up, how different are these two lists?
Intuitively, we're going to do is the more differences there are in the list, the more we are going to be able to have more stored up work than we can pay for it.
That's the basic idea.
So, the more that opt changes the list, so it's not the same as ours, in some sense the more we are going to be in a position as MTF to take advantage of that difference in delivering up work for us to do.
And we'll see how that plays out.
So, let's first also make another observation.
So, how much does phi change from one transpose?
How much does phi change from one transpose?
So, basically that's asking, if you do a transpose, what happens to the number of inversions?
So, what happens when a transposing is done?
What's going to happen to phi?
What's going to happen to the number of inversions?
So, if I change, it is less than n minus one, yes, if n is sufficiently large, yes.
OK, if I change, so you can think about it here.
Suppose I switch two of these elements here.
How much are things going to change?
Yeah, it's basically one or minus one, OK, because a transpose creates or destroys one inversion.
So, if you think about it, what if I change, for example, C and A, the relationship of C and A to everything else in the list is going to stay the same.
The only thing, possibly, that happens is that if they are in the same order when I transpose them, I've created an inversion.
Or, if they were in the wrong order when I transpose them, now they're in the right order.
So therefore, the change to the potential function is going to be plus or minus two because we're counting twice the number of inversions.
OK, any questions about that?
So, transposes don't change the potential very much, just by one.
It either goes up by two or down by two, just by one inversion.
So now, let's take a look at how these two algorithms operate.
OK, so what happens when op i accesses x in the two lists?
What's going to be going on?
To do that, let's define the following sets.
Why do I keep doing that?
OK, so we're going to look at the, when we access x, we are going to look at the two lists, and see what the relationship is, so, based on things that come before and after.
So, I think a picture is very helpful to understand what's going on here.
OK, so let's let, so here's L_i minus one, and we have our list, which I'll draw like this.
And somewhere in there, we have x. OK, and then we have L_i minus one star, which is opt's list, OK, and he's got x somewhere else, or she.
OK, and so, what is this set?
This is the set of Y that come before x.
So, that basically sets A and B. OK, those things that come before x in both.
And, some of them, the A's come before it in x, but come after it in, come before it in A, but come after it in B. OK, and similarly down here, what's this set?
That's A union C, good.
And this one?
Duh.
Yeah, it better be C union D because I've got A union B over there, and I've got x.
So that better be everything else.
OK, and here is B union D. OK, so those are the four sets that we're going to care about.
We're actually mostly going to care about these two sets.
OK, and we also know something about the r here.
The position of x is going to be the rank in L_i minus one of x.
And here, this is our star.
It's just to the rank in L_i minus one star of x.
So, we know what these ranks are.
And what we're going to be interested in is, in fact, characterizing the rank in terms of the sets.
OK, so what's the position of this?
Well, the rank, we have that r is equal to the size of A.
What's the size of B plus one?
OK, and r star is equal to the size of A plus the size of C plus one.
So, let's take a look at what happens when these two algorithms do their thing.
So, when the access to x occurs, we move x to the front of the list.
OK, it goes right up to the front.
So, how many inversions are created and destroyed?
So, how many are created by this?
That's probably a, how many inversions are created?
How many inversions are created?
So, we move x to the front.
So what we are concerned about is that anything that was in one of these sets that came, where it's going to change in order versus down here.
So, if I look in B, well, let's take a look at A. OK, so A, those are the things that are in the same order in both.
So, everything that's in A, when I move x to the front, each thing in A is going to count for one more inversion.
Does everybody see that?
So, I create a cardinality of A inversions.
And, we are going to destroy, well, everything in B came before x in this list, and after x in this.
But after we move x, they're in the right order.
So, I'm going to destroy B inversions, cardinality of B inversions.
OK, so that's what happens we operate with move to front.
We destroy.
We create A inversions and destroy B inversions, OK, by doing this movement.
OK, now, let's take a look at what opt does.
So, each transpose, we don't know what opt does.
He might move x this way or that way.
We don't know.
But each transpose, I opt, well, we're going to be interested in how many inversions it creates, and we already argued that it's going to create, at most, one inversion per transpose.
So, he can go and create more inversions, OK?
So, let me write it over here.
Thus -- -- the change in potential is going to be, at most, twice, A minus B plus t_i.
OK, so t_i, remember, was the number of transposes that opt does on the i'th step for the i'th operation.
OK, so we're going to create the change in potential is, at most, twice this function.
So, we are now going to look to see how we use this fact, and these two facts, this fact and this fact, OK, to show that opt can't be much better than MTF.
OK, good.
The way we are going to do that is look at the amortized cost of the I'th operation.
OK, what's MTF's amortized cost?
OK, and then we'll make the argument, which is the one you always make that the amortized cost upper bound the true costs, OK?
But the amortized cost is going to be easier to calculate.
OK, so amortized cost is just C hat, actually, let me make sure I have lots of room here on the right, c_i hat, which is equal to the true cost plus the change in potential.
OK, that's just the definition of amortized cost when given potential functions, OK?
So, what is the cost of operation i, OK, in this context here?
OK, we accessed x there.
What's the cost of operation i?
Two times the rank of x, which is 2r.
OK, so 2r, that part of it.
OK, well, we have an upper bound on the change in potential.
That's this.
OK, so that's two times the cardinality of A minus cardinality of B plus t_i.
OK, everybody with me?
Yeah?
OK, I see lots of nods.
That's good.
OK, that's equal to 2r plus two of size of A minus, OK, I want to plug in for B, and it turns out very nicely.
I have an equation involving A, B, and r. So, I get rid of the variable size of B by just plugging that in.
OK, and so what do I plug in here?
What's B equal to?
Yeah, r minus size of A minus one.
I wrote it the other way.
OK, and then plus t_i.
OK, and this is since r is A plus B plus one.
OK, everybody with me still?
I'm just doing algebra.
We've got to make sure we do the algebra right.
OK, so that's equal to, let's just multiply all this out now and get 2r plus, I have 2A here minus A.
So, that's 4A.
And then, two times minus r is minus 2r.
Two times minus one is minus two.
Oh, but it's minus-minus two, so it's plus two.
OK, and then I have 2t_i.
So, that's just algebra.
OK, so that's not bad.
We've just got rid of another variable.
What variable did we get rid of?
r. It didn't matter what the rank was as long as I knew what the number of inversions was here.
OK, so that's now equal to 4A plus two plus 2t_i.
And, that's less than or equal to, I claim, four times r star plus t_i using our other fact.
Since r star is equal to the size of A plus the size of C, plus one, then that's greater than or equal to the size of A plus one.
OK, if I look at this, I'm basically looking at A.
The fact that A, what did I do here?
If r star is greater than or equal to A plus one, right, so therefore, A plus one, good.
Yeah, so this is basically less than or equal to 4A plus four, which is four times A plus one.
I probably should have put in another algebra step here, OK, because if I can't verify it like this, then I get nervous.
This is basically, at most, 4A plus four.
That's four times A plus one, and A plus one is less than or equal to r star.
And then, 2t_i is, at most, 4TI.
So, I've got this.
Does everybody see where that came from?
But what is r star plus t_i?
What is r star plus t_i?
What is it?
It's c_i star.
That's just c_i star.
So, the amortized cost of i'th operation is, at most, four times opt's cost.
OK, that's pretty remarkable.
OK, so amortized cost of the i'th operation is just four times opt's cost.
Now, of course, we have to now go through and analyze the total cost.
But this is now the routine way that we analyze things with a potential function.
So, the costs of MTF of S is just the summation of the individual costs, OK, by definition.
And that is just the sum, i equals one, to S of the amortized cost plus, minus the change in potential.
OK, did I do this right?
No, I put the parentheses in the wrong place.
Now I've got it right.
Good.
I just missed a parenthesis.
OK, so this is, so in the past what I did was I expressed the amortized cost as being equal to c_i plus the change in potential.
I'm just throwing these two terms over to the other side and saying, what's the true cost in terms of the amortized cost?
OK, so I get c hat of i plus phi sub L_i minus one minus phi of L_i, OK, by making that substitution.
OK, that's less than or equal to since this is linear.
Well, I know what the sum of the amortized cost is.
It's, at most, 4c_i star.
So, the sum of them is, at most, to that sum, I equals one to S of 4c_i star.
And then, as happens in all these things, you get a telescope with these terms.
Every term is added in once and subtracted out once, except for the ones at the limit.
So, I get plus phi of L_0 minus phi of L sub cardinality of S. And now, this term is zero.
And this term is greater than or equal to zero.
OK, so therefore this whole thing is less than or equal to, well, what's that?
That's just four times opt's cost.
And so, we're four competitive.
OK, this is amazing, I think.
It's not that hard, OK, but it's quite amazing that just by doing a simple heuristic, you're nearly as good as any omniscient algorithm could possibly be.
OK, you're nearly as good.
And, in fact, in practice, this is a great heuristic.
So, if ever you have things like a hash table that you're actually seeing by chaining, OK, often it's the case that if when you access the elements, you're just bringing them up to the front of the list if it's an unsorted list that you've put them into, just bring them up to the front.
You can easily save 30 to 40% in run time for the accessing to the hash table because you will be much more likely to find the elements inside.
Of course, it depends on the distribution and so forth, for empirical matters, but the point is that you are not going to be too far off from the ordering that an optimal algorithm would do, optimal off-line algorithm: I mean, amazing
optimal off-line.
Now, it turns out that in the reading that we assigned, so, we assigned you Sleator and Tarjan's original paper.
In that reading, they actually have a slightly different model where they count transposes that move in excess to element x towards the front of the list as free.
OK, so, and this basically models, so here's the idea is if I actually have a linked list, and when I chase down, once I find x, I can actually move x up to the front with just a constant number of pointer operations to splice it out and put it up to the front.
I don't actually have to transpose all way back down.
OK, so that's kind of the model that they use, which is a more realistic model.
OK, I presented this argument because it's a little bit simpler.
OK, and the model is a little bit simpler.
But in our model, they have, when you access something, you want to bring it up to the front, or anything that you happen to go across during that time, you could bring up to the front essentially for free.
This model is the splicing in, splicing x in and out of L in constant time.
Then, MTF is, it turns out, too competitive.
It's within a factor of two of optimal, OK, if you use that.
And that's actually a good exercise to work through.
You could also go read about it in the reading to understand this better, to look to see where you would use those things.
You have to have another term representing the number of, quote, "free" transposes.
But it turns out that all the math works out pretty much the same.
OK, let's see, another thing I promised you is, what if, to look at the case, what if they don't start with the same lists?
OK, what if the two lists are different when they start?
Then, the potential function at the beginning might be as big as what?
How big are the potential function start out as if the lists are different?
So, suppose we're starting out, you have a list, and opt says, OK, I'm going to start out by ordering my list according to the sequence that I want to use, OK, and MTF orders it according to the sequence it must use.
What list is opt going to start out with as an adversary?
Yeah, it's going to pick the reverse of what ever MTF starts out with, right, because then, if he picks the reverse, what's the number of inversions?
It's how many inversions in a reverse ordered list?
Yeah, n choose two, OK. Is it n choose two, or n minus one choose two?
n minus one choose two, OK, inversions that you get because it's basically a triangular number when you add them up.
But in any case, it's order n^2, worst case.
So, what does that do to our analysis here?
It says that the cost of MTF of S is going to be, well, this is no longer zero.
This is now n^2.
OK, so we get that costs of MTF of S is, at most, four times opt's thing plus order n^2, OK?
And, if we look at the definition, did we erase it already?
OK, this is still for competitive, OK, since n^2 is constant as the size of S goes to infinity.
This is, once again, sort of your notion of, what does it mean to be a constant?
OK, so as the size of the list gets bigger, all we're doing is accessing whatever that number, n, is of elements.
That number doesn't grow with the problem size, OK, even if it starts out as some variable number, n. OK, it doesn't grow with the problem size.
We still end up being competitive.
This is just the k that was in that definition of competitiveness.
OK, any questions?
Yeah?
Well, so you could change the cost model a little bit.
Yeah.
And that's a good one to work out.
But if you say the cost of transposing, so, the cost of transposing is probably moving two pointers, approximately.
No, one, three pointers.
So, suppose that the cost of, wow, that's a good exercise, OK?
Suppose the cost was three times to do a transpose, was three times the cost of doing an access, of following a pointer.
OK, how would that change the number here?
OK, good exercise, great exercise.
OK, hmm, good final question.
OK, yes, it will affect the constant here just as when we do the free transpose, when we move things towards the front, that we consider those as free, OK. Those operations end up reducing the constant as well.
OK, but the point is that this constant is independent of the constant having to do with the number of elements in the list.
So that's a different constant.
So, this is a constant.
OK, and so as with a lot of these things, there's two things.
One is, there's the theory.
So, theory here backs up practice.
OK, those practitioners knew what they were doing, OK, without knowing what they were doing.
OK, so that's really good.
OK, and we have a deeper understanding that's led to, as I say, many algorithms for things like, the important ones are like paging.
So, what's the comment page replacement policy that people study, people have at most operating systems?
Who's done 6.033 or something?
Yeah, it's Least Recently Used, LRU.
People have heard of that, OK.
So, you can analyze LRU competitive, and show that LRU is actually competitive with optimal page replacement under certain assumptions.
OK, and there are also other things.
Like, people do random replacement algorithms, and there are a whole bunch of other kinds of things that can be analyzed with the competitive analysis framework.
OK, so it's very cool stuff.
And, we are going to see more in recitation on Friday, see a couple of other really good problems that are maybe a little bit easier than this one, OK, definitely easier than this one.
OK, they give you hopefully some more intuition about competitive analysis.
I also want to warn you about next week's problem set.
So, next week's problem set has a programming assignment on it.
OK, and the programming assignment is mandatory, meaning, well, all the problem sets are mandatory as you know, but if you decide not to do a problem there's a little bit of a penalty and then the penalties scale dramatically as you stop doing problem sets.
But this one is mandatory-mandatory.
OK, you don't pass the class.
You'll get an incomplete if you do not do this programming assignment.
Now, I know that some people are less practiced with programming.
And so, what I encourage you to do over the weekend is spent a few minutes and work on your programming skills if you're not up to snuff in programming.
It's not going to be a long assignment, but if you don't know how to read a file and write out a file, and be able to write a dozen lines of code, OK, if you are weak on that, this weekend would be a good idea to practice reading in a text file.
It's going to be a text file.
Read it in a text file, decent manipulations, write out a text file, OK?
So, I don't want people to get caught with this being mandatory and that not have time to finish it because they are busy trying to learn how to program in short order.
I know some people take this course without quite getting all the programming prerequisites.
Here's where you need it.
Question?
No language limitations.
Pick your language.
The answer will be written in, I think, Java, and Eric has graciously volunteered to use Python for his solution to this problem.
We'll see whether he lives up to that promise.
You did already?
OK, and George wrote the Java solution.
And so, C is fine.
Matlab is fine.
OK, what else is fine?
Anything is fine.
Scheme is fine.
Scheme is fine.
Scheme is great.
OK, so any such things will be just fine.
So, we don't care what language you program in, but you will have to do programming to solve this problem.
OK, so thanks very much.
See you next week.
-- valuable experience.
OK, today we're going to start talking about a particular class of algorithms called greedy algorithms.
But we're going to do it in the context of graphs.
So, I want to review a little bit about graphs, which mostly you can find in the textbook in appendix B.
And so, if you haven't reviewed in appendix B recently, please sit down and review appendix B.
It will pay off especially during our take-home quiz.
So, just reminder, a digraph, what's a digraph?
What's that short for?
Directed graph, OK?
Directed graph, G equals (V,E), OK, has a set, V, of vertices.
And, I always get people telling me that I have one vertice.
The singular is not vertice; it is vertex, OK?
The plural is vertices.
The singular is vertex.
It's one of those weird English words.
It's probably originally like French or something, right?
I don't know.
OK, anyway, and we have a set, E, which is a subset of V cross V of edges.
So that's a digraph.
And undirected graph, E contains unordered pairs.
OK, and, sorry?
It's Latin, OK, so it's probably pretty old, then, in English.
I guess the vertex would be a little bit of a giveaway that maybe it wasn't French.
It started to be used in 1570, OK. OK, good, OK, so the number of edges is, whether it's directed or undirected, is O of what?
V^2, good.
OK, and one of the conventions that will have when we're dealing, once we get into graphs, we deal a lot with sets.
We generally drop the vertical bar notation within O's just because it's applied.
It just makes it messier.
So, once again, another abuse of notation.
It really should be order the size of V^2, but it just messes up, I mean, it's just more stuff to write down.
And, you're multiplying these things, and all those vertical bars.
Since they don't even have a sense to the vertical bar, it gets messy.
So, we just drop the vertical bars there when it's in asymptotic notation.
So, E is order V^2 when it's a set of pairs, because if it's a set of pairs, it's at most n choose two, which is where it's at most n^2 over 2, here it could be, at most, sorry, V^2 over 2, here it's at most V^2.
And then, another property that sometimes comes up is if the G is connected, we have another bound, implies that the size of E is at least the size of V minus one.
OK, so if it's connected, meaning, what does it mean to have a graph that's connected?
Yeah, there's a path from any vertex to any other vertex in the graph.
That's what it means to be connected.
So if that's the case, that a number of edges is at least the number of vertices minus one, OK?
And so, what that says, so one of the things we'll get into, a fact that I just wanted to remind you, is that in that case, if I look at log E, OK, log of the number of edges, that is O of log V. And by this, is omega of log V. So, it's equal to theta of log V. OK, so basically the number of, in the case of a connected graph, the number of edges, and the number of vertices are polynomially related.
So, their logs are comparable.
OK, so that's helpful just to know because sometimes I just get questions later on where people will say, oh, you showed it was log E but you didn't show it was log V. And I could point out that it's the same thing.
OK, so there's various ways of representing graphs in computers, and I'm just going to cover a couple of the important ones.
There's actually more.
We'll see some more.
So, the simplest one is what's called an adjacency matrix.
An adjacency matrix of the graph, G, equals (V,E), where, for simplicity, I'll let V be the set of integers from one up to n, OK, is the n by n matrix A given by the ij-th at the entry is simply one if the edge, ij, is in the edge set and zero if ij is not in the edge set.
OK, so it's simply the matrix where you say, the ij entry is one if it's in the matrix.
So, this is, in some sense, giving you the predicate for, is there an edge from i to j?
OK, remember, predicate is Boolean formula that is either zero or one, and in this case, you're saying it's one if there is an edge from i to j and zero otherwise.
OK, sometimes you have edge weighted graphs, and then sometimes what people will do is replace this by edge weights.
OK, it will be the weight of the edge from i to j.
So, let's just do an example of that just to make sure that our intuition corresponds to our mathematical definitions.
So, here's an example graph.
Let's say that's our graph.
So let's just draw the adjacency the matrix.
OK, so what this says: is there's an edge from one to one?
And the answer is no.
Is there an edge from one to two?
Yes.
Is there an edge from one to three here?
Yep.
Is there an edge for one to four?
No.
Is there an edge from two until one?
No.
Two to two?
No.
Two to three?
Yes.
Two to four?
No.
No edges going out of three.
Edge from four to three, and that's it.
That's the adjacency matrix for this particular graph, OK?
And so, I can represent a graph as this adjacency matrix.
OK, when I represent it in this way, how much storage do I need?
OK, n^2 or V^2 because the size is the same thing for V^2 storage, OK, and that's what we call a dense representation.
OK, it works well when the graph is dense.
So, the graph is dense if the number of edges is close to all of the edges possible.
OK, then this is a good representation.
But for many types of graphs, the number of edges is much less than the possible number of edges, in which case we say the graph is sparse.
Can somebody give me an example of a sparse graph?
A class of graphs: so, I want a class of graphs that as n grows, the number of edges in the graph doesn't grow as the square, but grows rather as something much smaller.
A linked list, so, a chain, OK, if you look at it from a graph theoretically, is a perfectly good example: only n edges in the chain for a chain of length n. So therefore, the number of edges would be order V. And in particular, you'd only have one edge per row here.
What other graphs are sparse?
Yeah?
Good, a planar graph, a graph that can be drawn in a plane turns out that if it has V vertices has, and V is at least three, then it has, at most, three V minus six edges.
So, it turns out that's order V edges again.
What's another example of a common graph?
Yeah, binary tree, or even actually any tree, you know, what's called a free tree if you read the appendix, OK, a tree that just is a connected graph that has no cycles, OK, is another example.
What's an example of a graph that's dense?
A complete graph,
it's all ones, OK, or if you have edge weights, it would be a completely filled in matrix.
OK, good.
So, this is good for dense representation.
But sometimes you want to have a sparse representation so we don't have to spend V^2 space to deal with all of the, where most of it's going to be zeroes.
OK, it's sort of like, if we know why it's zero, why bother representing it as zero?
So, one such representation is an adjacency list representation.
Actually, adjacency list of a given vertex is the list, which we denote by Adj of V, of vertices adjacent to V. OK, just in terms by their terminology, vertices are adjacent, but edges are incident on vertices.
OK, so the incidence is a relation between a vertex and an edge.
An adjacency is a relation between two vertices.
OK, that's just the language.
Why they use to different terms, I don't know, but that's what they do.
So, in the graph, for example, the adjacency list for vertex one is just the list or the set of two three because one has going out of one are edges to two and three.
The adjacency list for two is just three, four, three.
It's the empty set, and for four, it is three.
OK, so that's the representation.
Now, if we want to figure out how much storage is required for this representation, OK, we need to understand how long the adjacency list is.
So, what is the length of an adjacency list of a vertex, V?
What name do we give to that?
It's the degree.
So, in an undirected graph, we call it the degree of the vertex.
This is undirected.
OK, about here, OK.
So that's an undirected case.
In the directed case, OK, actually I guess the way we should do this is say this.
If the degree, we call it the out degree for a digraph.
OK, so in a digraph, we have an out degree and an in degree for each vertex.
So here, the in degree is three.
Here, the out degree is two, OK?
So, one of the important lemma that comes up is what's called the handshaking lemma.
OK, it's one of these mathematical lemmas.
And so, it comes from a story.
Go to a dinner party, and everybody at the dinner party shakes other people's hands.
Some people may not shake anybody's hand.
Some people may shake several people's hands.
Nobody shakes hands with themselves.
And at some point during the dinner party, the host goes around and counts up how many, the sum, of the number of hands that each person has shaken.
OK, so he says, how many did you shake?
How many did you shake?
How many did you shake?
He adds them up, OK, and that number is guaranteed to be even.
OK, that's the handshaking lemma.
Or, stated a little bit more precisely, if I take for any graph the degree of the vertex, and sum them all up, that's how many hands everybody shook, OK, that's actually equal to always twice the number of edges.
So, why is that going to be true?
Why is that going to be twice the number of edges?
Yeah?
Yeah.
Every time you put in an edge, you add one to the degree of each person on each end.
So, it's just two different ways of counting up the same number of edges.
OK, I can go around, and if you imagine that, that every time I count the degree of the node, I put a mark on every edge.
Then, when I'm done, every edge has two marks on it, one for each end
a pretty simple theorem.
So, what that says is that for undirected graphs, that implies that the adjacency list representation, uses how much storage?
OK, at most, 2E, so order E because that's not all.
Yeah, so you have to have the number of vertices plus order the number of edges, OK, whether it's directed or undirected because I may have a graph, say it has a whole bunch of vertices and no edges, that's still going to cost me order V, OK?
So, it uses theta of V plus E storage.
And, it's basically the same thing asymptotically.
In fact, it's easier to see in some sense for digraphs because for digraphs, what I do is I just add up the out degrees, and that equal to E, OK, if I add up the out degrees as equally.
In fact, this is kind of like it amortized analysis, if you will, a book keeping analysis, that if I'm adding up the total number of edges, one way of doing it is accounting for a vertex by vertex.
OK, so for each vertex, I basically can take each degree, and basically each vertex, look at the degree, and that allocating of account per edge, and then ending up with twice the number of edges, that's exactly accounting type of analysis that we might do for amortized analysis.
OK, so we'll see that.
So, this is a sparse representation, and it's often better than an adjacency matrix.
For example, you can imagine if the World Wide Web were done with an adjacency matrix as opposed to, essentially, with an adjacency list type of representation.
Every link on the World Wide Web, I had to say, here are the ones that I'm connected to, and here are all the ones I'm not connected to.
OK, that list of things you're not connected to for a given page would be pretty dramatically, show you that there is an advantage to sparse representation.
On the other hand, one of the nice things about an adjacency matrix representation is that each edge can be represented with a single bit, whereas typical when I'm representing things with an adjacency list representation, how many bits am I going to need to represent each adjacency?
You'll need order log of V to be able to name each different vertex.
OK, the log of the number is the number of bits that I need.
So, there are places where this is actually a far more efficient representation.
In particular, if you have a very dense graph, OK, this may be a better way of representing it.
OK, the other thing I want you to get, and we're going to see more of this in particular next week, is that a matrix and a graph, there are two ways of looking at the same thing.
OK, and in fact, there's a lot of graph theory that when you do things like multiply the adjacency matrix, OK, and so forth.
So, there's a lot of commonality between graphs and matrices, a lot of mathematics that if it applies for one, it applies to the other.
Do you have a question, or just holding your finger in the air?
OK, good.
OK, so that's all just review.
Now I want to get onto today's lecture.
OK, so any questions about graphs?
So, this is a good time to review appendix B. there are a lot of great properties in there, and in particular, there is a theorem that we're going to cover today that we're going to talk about today, which is properties of trees.
Trees are very special kinds of graphs, so I really want you to go and look to see what the properties are.
There is, I think, something like six different definitions of trees that are all equivalent, OK, and so, I think a very good idea to go through and read through that theorem.
We're not going to prove it in class, but really, provides a very good basis for the thinking that we're going to be doing today.
And we'll see more of that in the future.
OK, so today, we're going to talk about minimum spanning trees.
OK, this is one of the world's most important algorithms.
OK, it is important in distributed systems.
It's one of the first things that almost any distributed system tries to find is a minimum spanning tree of the nodes that happened to be alive at any point, OK?
And one of the people who developed an algorithm for this, we'll talk about this a little bit later, OK, it was the basis of the billing system for AT&T for many years while it was a monopoly.
OK, so very important kind of thing.
It's got a huge number of applications.
So the problem is the following.
You have a connected undirected graph, G equals (V,E), with an edge weight function, w, which maps the edges into weights that are real numbers.
And for today's lecture, we're going to make an important assumption, OK, for simplicity.
The book does not make this assumption.
And so, I encourage you to look at the alternative presentation or, because what they do in the book is much more general, but for simplicity and intuition, I'm going to make this a little bit easier.
We're going to assume that all edge weights are distinct.
OK, all edge weights are distinct.
So what does that mean?
What does that mean that this function, w, what property does the function, w, have if all edge weights are distinct?
Who remembers their discreet math?
It's injective.
OK, it's one to one.
OK, it's not one to one and onto necessarily.
In fact, it would be kind of hard to do that because that's a pretty big set.
OK, but it's one to one.
It's injective.
OK, so that's what we're going to assume for simplicity.
OK, and the book, they don't assume that.
It just means that the way you have to state things is just a little more precise.
It has to be more technically precise.
So, that's the input.
The output is-- The output is a spanning tree, T, and by spanning tree, we mean it connects all the vertices.
OK, and it's got to have minimum weight.
OK, so we can write the weight of the tree is going to be, by that, we meet the sum over all edges that are in the tree of the weight of the individual edges.
OK, so here I'(V,E) done a little bit of abusive notation, which is that what I should be writing is w of the edge (u,v) because this is a mapping from edges, which would give me a double parentheses.
And, you know, as you know, I love to abuse notation.
So, I'm going to drop that extra parentheses, because we understand that it's really the weight of the edge, OK, not the weight of the ordered pair.
So, that's just a little notational convenience.
OK, so one of the things, when we do the take-home exam, notational convenience can make the difference between having a horrible time writing up a problem, and an easy time.
So, it's worth thinking about what kinds of notation you'll use in writing up solutions to problems, and so forth.
OK, and just in general, a technical communication, you adopt good notation people understand you.
You adopt a poor notation: nobody pays attention to what you're doing because they don't understand what you're saying.
OK, so let's do an example.
OK, so here's a graph.
I think for this, somebody asked once if I was inspired by biochemistry or something, OK, but I wasn't.
I was just writing these things down, OK?
So, here's a graph.
And let's give us some edge weights.
OK, so there are some edge weights.
And now, what we want is we want to find a tree.
So a connected acyclic graph such that every vertex is part of the tree.
But it's got to have the minimum weight possible.
OK, so can somebody suggest to me some edges that have to be in this minimum spanning tree?
Yeah, so nine, good.
Nine has to be in there because, why?
It's the only one connecting it to this vertex, OK?
And likewise, 15 has to be in there.
So those both have to be in.
What other edges have to be in?
Which one?
14 has to be it.
Why does 14 have to be in?
Well, one of 14 and three has to be in there.
I want the minimum weight.
The one that has the overall smallest weight.
So, can somebody argue to me that three has to be in there?
Yeah?
That's the minimum of two, which means that if I had a, if you add something you said was a minimum spanning tree that didn't include three, right, and so therefore it had to include 14, then I could just delete this edge, 14, and put in edge three.
And, I have something of lower weight, right?
So, three has to be in there.
What other edges have to be in there?
Do a little puzzle logic.
Six and five have to be in there.
Why do they have to be in there?
Yeah, well, I mean, it could be connected through this or something.
It doesn't necessarily have to go this way.
Six definitely has to be in there for the same reason that three had to be, right?
Because we got two choices to connect up this guy.
And so, if everything were connected but it weren't, 12, I mean, and 12 was in there.
I could always, then, say, well, let's connect them up this way instead.
OK, so definitely that's in there.
I still don't have everything connected up.
What else has to be in there for minimum spanning tree?
Seven, five, and eight, why seven, five, and eight?
OK, so can we argue those one at a time?
Why does five have to be in there?
Yeah?
OK, so we have four connected components because we have this one, this one, we actually have, yeah, this one here, and this one, good.
We need at least three edges to connect them because each edge is going to reduce the connected components by one.
OK, so we need three edges, and those are the three cheapest ones.
And they work.
That works, right?
Any other edges are going to be bigger, so that works.
Good.
OK, and so, now do we have a spanning tree?
Everything is, we have one big connected graph here, right?
Is that what I got?
Hey, that's the same as what I got.
Life is predictable.
OK, so, so everybody had the idea of what a minimum spanning tree is, then, out of this, OK, what's going on there?
So, let's first of all make some observations about this puzzle.
And what I want to do is remind you about the optimal substructure property because it turns out minimum spanning tree has a great optimal substructure property.
OK, so the setup is going to be, we're going to have some minimum spanning tree.
Let's call it T. And, I'm going to show that with the other edges in the graph, are not going to be shown.
OK, so here's a graph.
OK, so here's a graph.
It looks like the one I have my piece of paper here.
OK, so the idea is, this is some minimum spanning tree.
Now, we want to look at a property of optimal substructure.
And the way I'm going to get that, is, I'm going to remove some edge, (u,v), move an arbitrary edge, (u,v), in the minimum spanning tree.
So, let's call this u and this V. And so, we're removing this edge.
OK, so when I remove an edge in a tree, what happens to the tree?
What's left?
I have two trees left, OK?
I have two trees left.
Now, proving that, that's basically one of the properties in that appendix, and the properties of trees that I want you to read, OK, because you can actually prove that kind of thing rather than it just being obvious, which is, OK?
OK, so we remove that.
Then, T is partitioned into two subtrees.
And, we'll call them T_1 and T_2.
So, here's one subtree, and here's another subtree.
We'(V,E) partitioned it.
No matter what edge I picked, there would be two subtrees that it's partitioned into.
Even if the sub tree is a trivial subtree, for example, it just has a single node in it and no edges.
So, the theorem that we'll prove demonstrates a property of optimal substructure.
T_1 is a minimum spanning tree for the graph, G_1, E_1, a subgraph of G induced by the vertices in T_1.
OK, that is, V_1 is just the vertices in T_1 is what it means to be induced.
OK, so V_1 is the vertices in T_1.
So, in this picture, I didn't label it.
This is T_1.
This is T_2.
In this picture, these are the vertices of T_1.
So, that's V_1, OK?
And, E_1 is the set of pairs of vertices, x and y, that are the edges that are in E_1 such that both x and y belong to V_1.
OK, so I haven't shown the edges of G here.
But basically, if an edge went from here to here, that would be in the E_1.
If it went from here to here, it would not.
And if it went from here to here, it would not.
OK, so the vertices, the subgraph induced by the vertices of T_1 are just those that connect up things in T_1, and similarly for T_2.
So, the theorem says that if I look at just the edges within the graph here, G_1, those that are induced by these vertices, T_1 is, in fact, a minimum spanning tree for that subgraph.
That's what the theorem says.
OK, if I look over here conversely, or correspondingly, if I look at the set of edges that are induced by this set of vertices, the vertices in T_2, in fact, T_2 is a minimum spanning tree on that subgraph.
OK, OK, we can even do it over here.
If I took a look, for example, at these, let's see, let's say we cut out five, and if I cut out edge five, that T_1 would be these four vertices here.
And, the point is that if I look at the subgraph induced on that, that these edges here.
In fact, the six, eight, and three are all edges in a minimum spanning tree for that subgraph.
OK, so that's what the theorem says.
So let's prove it.
OK, and so what technique are we going to use to prove it?
OK, we learned this technique last time: hint, hint.
It's something you do it in your text editor all the time: cut and paste, good, cut and paste.
OK, so the weight of T I can express as the weight of the edge I removed, plus the weight of T_1, plus the weight of T_2.
OK, so that's the total weight.
So, the argument is pretty simple.
Suppose that there were some T_1 prime that was better than T_1 for G_1.
Suppose I had some better way of forming a spanning tree.
OK, then I would make up a T prime, which just contained the edges, (u,v), and T_1 prime, union T_2.
So, I would take, if I had a better spanning tree, a spanning tree of lower weight for T_1.
And I call that T_1 prime.
I just substitute that and make up a spanning tree that consisted of my edge, (u,v), whatever works well for T_1 prime and whatever works well for T. And, that would be a spanning tree.
And it would be better than T itself was for G, OK, because the weight of these is just as the weight for this, I now just get to use the weight of T_1 prime, and that's less.
And so, therefore, the assumption that T was a minimum spanning tree would be violated if I could find a better one for the subpiece.
So, we have this nice property of optimal substructure.
OK, I have subproblems that exhibit optimal, if I have a globally optimal solution to the whole problem within it, I can find optimal solutions to subproblems.
So, now the question is, that's one hallmark.
That's one hallmark of dynamic programming.
What about overlapping subproblems?
Do I have that property?
Do I have overlapping subproblems over here for this type of problem?
So, imagine, for example, that I'm removing different edges.
I look at the space of taking a given edge, and removing it.
It partitions it into two pieces, and now I have another piece.
And I remove it, etc.
Am I going to end up getting a bunch of subproblems that are similar in there?
Yeah, I am.
OK, if I take out this one, then I take out, say, this one here, and then I'll have another tree here and here.
OK, that would be the same as if I had originally taken this out, and then taken that one out.
If I look at simple ordering of taking out the edges, I'm going to end up with a whole bunch of overlapping subproblems.
Yeah, OK.
So then, what does that suggest we use as an approach?
Dynamic programming, good.
What a surprise!
Yes, OK, you could use dynamic programming.
But it turns out that minimum spanning tree exhibits an even more powerful property.
OK, so we'(V,E) got all the clues for dynamic programming, but it turns out that there's an even bigger clue that's going to help us to use an even more powerful technique.
And that, we call, the hallmark for greedy algorithms.
And that is, we have a thing called the greedy choice property, which says that a locally optimal choice is globally optimal.
And, of course, as all these hallmarks is the kind of thing you want to box, OK, because these are the clues that you're going to be able to do that.
So, we have this property that we call the greedy choice property.
I'm going to show you how it works in this case.
And when you have a greedy choice property, it turns out you can do even better that dynamic programming.
OK, so when you see the two dynamic programming properties, there is a clue that says dynamic programming, yes, but also it says, let me see whether it also has this greedy property because if it does, you're going to come up with something that's even better than dynamic programming, OK?
So, if you just have the two, you can usually do dynamic programming, but if you have this third one, it's like, whoa!
Jackpot!
OK, so here's the theorem we'll prove to illustrate this idea.
Once again, these are not, all these hallmarks are not things.
They are heuristics.
I can't give you an algorithm to say, here's where dynamic programming works, or here's where greedy algorithms work.
But I can sort of indicate when they work, the kind of structure they have.
OK, so here's the theorem.
So let's let T be the MST of our graph.
And, let's let A be any subset of V, so, some subset of vertices.
And now, let's suppose that edge, (u,v), is the least weight edge connecting our set A to A complement, that is, V minus A.
Then the theorem says that (u,v) is in the minimum spanning tree.
So let's just take a look at our graph over here and see if that's, in fact, the case.
OK, so let's take, so one thing I could do for A is just take a singleton node.
So, I take a singleton node, let's say this guy here, that can be my A, and everything else is V minus A.
And I look at the least weight edge connecting this to everything else.
Well, there are only two edges that connect it to everything else.
And the theorem says that the lighter one is in the minimum spanning tree.
Hey, I win.
OK, if you take a look, every vertex that I pick, the latest edge coming out of that vertex is in the minimum spanning tree.
OK, the lightest weight edge coming out, but that's not all the edges that are in here.
OK, or let's just imagine, let's take a look at these three vertices connected to this set of vertices.
I have three edges is going across.
The least weight one is five.
That's the minimum spanning tree.
Or, I can cut it this way.
OK, the ones above one, the edges going down are seven, eight, and 14.
Seven is the least weight.
It's in the minimum spanning tree.
So, no matter how I choose, I could make this one in, this one out, this one in, this one out, this one in, this one out, this one in, this one out, take a look at what all the edges are.
Which ever one to the least weight: it's in the minimum spanning tree.
So, in some sense, that's a local property because I don't have to look at what the rest of the tree is.
I'm just looking at some small set of vertices if I wish, and I say, well, if I wanted to connect that set of vertices to the rest of the world, what would I pick?
I'd pick the cheapest one.
That's the greedy approach.
It turns out, that wins, OK, that picking that thing that's locally good for that subset, A, OK, is also globally good.
OK, it optimizes the overall function.
That's what the theorem says, OK?
So, let's prove this theorem.
Any questions about this?
OK, let's prove this theorem.
So, we have (u,v) is the least weight edge connecting A to D minus A.
So, let's suppose that this edge, (u,v), is not in the minimum spanning tree.
OK, let's suppose that somehow there is a minimum spanning tree that doesn't include this least weight edge.
OK, so what technique you think will use to prove to get a contradiction here?
Cut and paste, good.
Yeah, we're going to cut paste.
OK, we're going to cut and paste.
So here, I did an example.
OK, so -- OK, and so I'm going to use the notation.
I'm going to color some of these in.
OK, and so my notation here is this is an element of A, and color it in.
It's an element of V minus A. OK, so if it's not colored it, that's an A.
This is my minimum spanning tree.
Once again, I'm not showing the overall edges of all the graphs, but they're there, OK?
So, my edge, (u,v), which is not my minimum spanning tree I say, let's say is this edge here.
It's an edge from u, u as in A, v as in V minus A. OK, so everybody see the setup?
So, I want to prove that this edge should have been in the minimum spanning tree, OK, that the contention that this is a minimum spanning tree, and does include (u,v) is wrong.
So, what I want to do, that, is I have a tree here, T, and I have two vertices, u and v, and in a tree, between any two vertices there is a unique, simple path: simple path meaning it doesn't go back and forth and repeat edges or vertices.
OK, there's a unique, simple path from u to v. So, let's consider that path.
OK, and the way that I know that that path exists is because I'(V,E) read appendix B of the textbook, section B.5.1, OK, which has this nice theorem about properties of trees.
OK, so that's how I know that there exists a unique, simple path.
OK, so now we're going to do is take a look at that path.
So in this case, it goes from here, to here, to here, to here.
And along that path, there must be a point where I connect from a vertex in A to a vertex in V minus A.
Why?
Well, because this is in A.
This is in V minus A.
So, along the path somewhere, there must be a transition.
OK, they are not all in A, OK, because in particular, V isn't.
OK, so we're going to do is swap (u,v) with the first edge on this path that connects a vertex in A to a vertex in V minus A.
So in this case, it's this edge here.
I go from A to V minus A.
In general, I might be alternating many times, OK, and I just picked the first one that I encounter.
OK, that this guy here.
And what I do is I put this edge in.
OK, so then, what happens?
Well, the edge, (u,v), is the lightest thing connecting something in A to something in V minus A.
So that means, in particular, it's lighter than this edge, has lower weight.
So, by swapping this, I'(V,E) created a tree with lower overall weight, contradicting the assumption that this other thing was a minimum spanning tree
so, a lower weight spanning tree than T results, and that's a contradiction -- -- than T results.
And that's a contradiction, OK?
How are we doing?
Everybody with me?
OK, now we get to do some algorithms.
Yea!
So, we are going to do an algorithm called Prim's algorithm.
Prim eventually became a very high-up at AT&T because he invented this algorithm for minimum spanning trees, and it was used in all of the billing code for AT&T for many years.
He was very high up at Bell Labs back in the heyday of Bell Laboratories.
OK, so it just shows, all you have to do is invent an algorithm.
You too can be a president of a corporate monopoly.
Of course, the government can do things to monopolies, but anyway, if that's your mission in life, invent an algorithm.
OK, so here's the idea.
What we're going to do is we're going to maintain V minus A as a priority queue.
We'll call it Q.
And each vertex, we're going to key each vertex in Q with the weight of the least weight edge, connecting it to a vertex in A.
So here's the code.
So, we're going to start out with Q being all vertices.
So, we start out with A being, if you will, the empty set.
OK, and what we're going to do it is the least weight edge, therefore, for everything in the priority queue is basically going to be infinity because none of them have any edges.
The least weight edge to the empty set is going to be empty.
And then, we're going to start out with one guy.
We'll call him S, which will set to zero for some arbitrary S in V. And then, the main part of the algorithm kicks in.
So that's our initialization.
OK, when we do the analysis, I'm going to write some stuff on the left hand side of the board.
So if you're taking notes, you may want to also leave a little bit of space on the left hand side of your notes.
So, while Q is not empty, we get the smallest element out of it.
And then we do some stuff.
That's it.
And the only thing I should mention here is, OK, so let's just see what's going on here.
And then we'll run it on the example.
OK, so what we do is we take out the smallest element out of the queue at each step.
And then for each step in the adjacency list, in other words, everything for which I have an edge going from v to u, we take a look, and if v is still in our set V minus A, so things we'(V,E) taken out are going to be part of A. OK, every time we take something out, that's going to be a new A that we construct.
At every step, we want to find, what's the cheapest edge connecting that A to everything else?
We basically are going to take whatever that cheapest thing is, OK, add that edge in, and now bring that into A and find the next cheapest one.
And we just keep repeating the process.
OK, we'll do it on the example.
And what we do, is every time we bring it in, I keep track of, what was the vertex responsible for bringing me in.
And what I claim is that at the end, if I look at the set of these pairs that I'(V,E) made here, V and pi of V, that forms the minimum spanning tree.
So let's just do this.
And, what's that?
We're all set up.
So let's get rid of these guys here because we are going to recompute them from scratch.
OK, so you may want to copy the graph over again in your notes.
I was going to do it, but it turned out, this is exactly the board is going to erase this.
OK, well let me just modify it.
OK, so we start out.
We make everything be infinity.
OK, so that's where I'm going to keep the key value.
OK, and then what I'm going to do is find one vertex.
And I'm going to call him S. And I'm going to do this vertex here.
We'll call that S. So basically, I now make him be zero.
And now, what I do, is I execute extract min.
So basically, what I'll do is I'll just shade him like this, indicating that he has now joined the set A.
So, this is going to be A.
And this is element of V minus A. OK, so then what we do is we take a look.
We extract him, and then for each edge in the adjacency list, OK, so for each vertex in the adjacency lists, that these guys here, OK, we're going to look to see if it's still in Q, that is, in V minus A.
And if so, and its key value is less than what the value is at the edge, there, we're going to replace it by the edge value.
So, in this case, we're going to replace this by seven.
We're going to replace this by 15, and we're going to replace this by ten, OK, because what we're interested in is, what is the cheapest?
Now, notice that everything in V minus A, that is, what's in the priority queue, everything in there, OK, now has its cheapest way of connecting it to the things that I'(V,E) already removed, the things that are in A. OK, and so now I just, OK, when I actually do that update, there's actually something implicit going on in this priority queue.
And that is that I have to do a decreased key.
So, there's an implicit decrease of the key.
So, decreased key is a priority queue operation that lowers the value of the key in the priority queue.
And so, that's implicitly going on when I look at what data structure I'm going to use to implement that priority queue.
OK, so common data structures for implementing a priority queue are a min heap.
OK, so I have to make sure that I'm actually doing this operation.
I can't just change it and not affect my heap.
So, there is an implicit operation going on there.
OK, now I repeat.
I find the cheapest thing, oh, and I also have to set, now, a pointer from each of these guys back to u.
So here, this guy sets a pointer going this way.
This guy sets a pointer going this way, and this guy sets a pointer going this way.
That's my pi thing that's going to keep track of who caused me to set my value to what it is.
So now, we go in and we find the cheapest thing, again.
And we're going to do it fast, too.
OK, this is a fast algorithm.
OK, so now we're going to go do this again.
So now, what's the cheapest thing to extract?
This guy here, right?
So, we'll take him out, OK, and now we update all of his neighbors.
So this guy gets five.
This guy gets 12.
This guy gets nine.
This guy we don't update.
We don't update him because he's no longer in the priority queue.
And all of these guys now, we make point to where they're supposed to point to.
And, we're done with that step.
Now we find the cheapest one.
What's the cheapest one now?
The five over here.
Good.
So, we take him out.
OK, we update the neighbors.
Here, yep, that goes to six now.
And, we have that pointer.
And, this guy we don't do, because he's not in there.
This guy becomes 14, and this guy here becomes eight.
So, we update that guy, make him be eight.
Did I do this the right way?
Yeah, because pi is a function of this guy.
So basically, this thing, then, disappears.
Yeah, did I have another one that I missed?
12, yes, good, it's removed, OK, because pi is just a function.
And now I'm OK. OK, so now what do I do?
OK, so now my set, A, consists of these three things, and now I want the cheapest edge.
I know it's in the minimum spanning tree.
So let me just greedily pick it.
OK, so what's the cheapest thing now?
This guy appear?
Yeah, six.
So we take it.
We go to update these things, and nothing matters here.
OK, nothing changes because these guys are already in A. OK, so now the cheapest one is eight here.
Good.
So, we take eight out.
OK, we update this.
Nothing to be done.
This: nothing to be done.
This: oh, no, this one, instead of 14 we can make this be three.
So, we get rid of that pointer and make it point that way.
Now three is the cheapest thing.
So, we take it out, and of course there's nothing to be done over there.
And now, last, I take nine.
And it's done.
And 15: it's done.
And the algorithm terminates.
OK, and as I look at, now, all the edges that I picked, those are exactly all the edges that we had at the beginning.
OK, let's do an analysis here.
OK, so let's see, this part here costs me order V, right?
OK, and this part, let's see what we are doing here.
Well, we're going to go through this loop how many times?
V times.
It's V elements we put into the queue.
We are not inserting anything.
We're just taking them out.
This goes V times, OK, and we do a certain number of extract Mins.
So, we're going to do order V extract Mins.
And then we go to the adjacency list, and we have some constant things.
But we have these implicit decreased keys for this stuff here.
That's this thing here.
OK, and so how many implicit decreased keys do we have?
That's going to be the expensive thing.
OK, we have, in this case, the degree of u of those.
OK, so overall, how many implicit decreased keys do we have?
Well, we have V times through.
How big could the degree of u be?
OK, it could be as big as V, order V. So, that's V^2 decreased use.
But we can do a better bound than that.
How many do we really have?
Yeah, at most order E, OK, because what am I doing?
I'm summing up the degrees of all the vertices.
That's how many times I actually execute that.
So, I have order E, implicit decreased keys.
So the time overall is order V times time for whatever the extract Min is plus E times the time for decreased key.
So now, let's look at data structures, and we can evaluate for different data structures what this formula gives us.
So, we have different ways of implementing a data structure.
We have the cost of extract Min, and of decreased key, and total.
So, the simplest way of implementing a data structure is an unsorted array.
If I have an unsorted array, how much time does it take me to extract the minimum element?
If I have an unsorted array?
Right, order V in this case because it's an array of size V. And, to do a decreased key, OK, I can do it in order one.
So, the total is V^2, good, order V^2 algorithm.
Or, as people suggested, how about a binary heap?
OK, to do an extract Min in a binary heap will cost me what?
O of log V. Decreased key will cost me, yeah, it turns out you can do that in order log V because basically you just have to shuffle the value, actually shuffle it up towards the root, OK?
Or at log V. And, the total cost therefore is?
E log V, good.
Which of these is better?
It depends, good.
When is one better, and when is the other better?
Yeah, if it's a dense graph, E is close to V^2, the array is better.
But if it's a sparse graph, and E is much smaller than V^2, then the binary heap is better.
So that motivated the invention of a data structure, OK, called a Fibonacci Heap.
So, Fibonacci Heap is covered in Chapter 20 of CLRS.
We're not going to hold you responsible for the content, but it's an interesting data structure because it's an amortized data structure.
And it turns out that it is data structure, you can do extract Min in order log V amortized time.
And remarkably, you can do decreased key in order one amortized.
So, when I plug those in, what do I get over here?
What's that going to be?
Plug that it here.
It's going to be V times log V plus
E plus V log V. These are amortized, so what's this?
Trick question.
It's worst-case.
It's not amortized over here.
These are amortized, but that's the beauty of amortization.
I can say it's going to be worst case: E plus V log V over here, because when I add up the amortized cost of my operations, it's an upper bound on the true costs.
OK, so that's why I say, one of the beauties of this amortized analysis, and in particular, being able to assign different costs to different operations is I can just add them up and I get my worst-case costs.
So this is already V log V. There are a couple other algorithms just before I let you go.
Kruskal's Algorithm in the book uses another amortized data structure called a disjoint set data structure, which also runs in E log V, that is, this time: runs in this time, the same as using a binary heap.
So, I'll refer you to the book.
The best algorithm to date with this problem is done by our own David Karger on the faculty here with one of our former graduates, Phil Kline, who is now a professor at Brown, and Robert Tarjan, who is sort of like the master of all data structures who was a professor at Princeton in 1993.
OK, it's a randomized algorithm, and it gives you order V plus E expected time.
OK, so that's the best to date.
It's still open as to whether there is a deterministic, there is worst-case bound, whether there is a worst-case bound that is linear time.
OK, but there is a randomized to linear time, and otherwise, this is essentially the best bound without additional assumptions.
OK, very cool stuff.
Next, we're going to see a lot of these ideas of greedy and dynamic programming in practice.
We're going to talk about shortest paths, and we're going to talk about shortest paths for three lectures.
So, this is a trilogy.
Today will be Shortest Paths One.
I've been watching far too many versions of Star Wars this weekend.
I saw the musical yesterday, matinee.
That was an MIT musical.
That was fun, of all three movies in about four hours.
That was a bit long and then I saw the one-man show on Friday.
One-man Star Wars: the original three movies in one hour.
That was the opposite of too long.
Both were fun.
So I get my trilogy fix.
All episodes, first we're going to start with The New Hope, and we're going to talk about the shortest paths problem and solve one particular problem of it, a very interesting version.
And then we're going to look at increasingly more general versions as we go on.
Shortest paths are sort of an application of dynamic programming, which we saw last week, and greedy algorithms, which we also saw last week.
So, were going to build that and get some pretty interesting algorithms for an important problem, which is how to get from Alderon to, I don't know, Cambridge as quickly as possible, OK, when you live in a graph.
So, there's geometric shortest paths which is a little bit harder.
Here, we're just going to look at shortest paths in graphs.
Now, hopefully you all know what a path in a graph is.
But, so, very quick review in particular because we're going to be looking at weighted graphs.
So, the usual setup: suppose we have directed graph, G, have some vertices, some edges.
We have edge weights, make it a little more interesting.
So, this is just a real number on each edge.
So, edge weights are usually given by function, w. For every edge, you get a real number.
And then, if we look at the paths in the graph, so we're going to use some simple notation for paths called a path, p, starts at some vertex, and it goes to some other vertex, and so on.
Say the last vertex is v_k, and each of these should be a directed edge in the digraph.
So, this is a directed path.
It has to respect edges in here.
And, we'll say that the weight of such a path is just the sum of the weights of the edges along the path.
And, we'll call that w(p).
This is sum, i equals one to k minus one of w(v_i, v_(i+1)) plus one.
OK, so just to rub it in, and in particular, how general this can be, we have some path, it starts at some vertex, there's some edge weights along the way.
This is some arbitrary path in the graph, in some hypothetical graph.
OK, this is mainly to point out that some of the edge weights could be negative.
Some of them could be zero.
This sum here is minus two.
So, the weight of this path is minus two.
And, presumably, the graph is much bigger than this.
This is just one path in the graph.
We're usually thinking about simple paths that can't repeat a vertex.
But, sometimes we allow that.
And then, what we care about is the shortest path, or a shortest path.
Again, this may not be unique, but we'll still usually call it the shortest path.
So, we want the shortest path from some A to some B.
Or, we'll call the vertices u and v. And we want this to be some path of minimum possible weight, subject to starting at u, and going to v. OK, so that's what we're looking for.
In general, give you a vertex, u, give you a vertex, v, find a shortest path as quickly as possible.
What's a good algorithm for that?
That's the topic for the next three lectures.
We'll usually think about a slightly simpler problem, which is just computing the weight of that path, which is essentially computing the distance from A to B.
So, we'll call this the shortest path weight from u to v. And, we'll denote it by delta of (u,v), delta .
So, I mean, it's the weight of the shortest path, or a weight of every shortest path.
Or, in other words, it's the Min over the weight of each path from u to v. So, p here is a path.
OK, so you just consider, there could be a lot of different paths.
There could, in principle, be infinitely many, if you're allowed to repeat vertices.
You look at all those paths hypothetically.
You take the minimum weight.
Question?
Good.
My next question was going to be, when do shortest paths not exist?
And you've hit upon one version, which is when you have negative edge weights.
So, in principle, when you have negative edge weights, some shortest paths may not exist in the sense that there is no shortest paths.
There are no shortest paths.
There is no shortest path from u to v. OK, in particular, if I have two vertices, u and v, and I want the shortest path between them, and I have negative edge weights, well, this is fine.
I mean, I can still compute the weight of a path that has negative weights.
But when specifically won't I have a single shortest path from u to v?
So, go ahead.
Good.
So, if I can find the cycle somewhere along here whose total weight, say, the sum of all the weights of these images is negative, then I get there, I go around as many times as I want.
I keep decreasing the weight because the weight is negative.
I decrease it by some fixed amount, and then I can go to v. So, as long as there is a negative weights cycle reachable from u that can also reach v, then there's no shortest path because if I take any particular path, I can make it shorter by going around a couple more times.
So, in some sense, this is not really a minimum.
It's more like an infimum for those who like to get fancy about such things.
But we'll just say that delta of (u,v) is minus infinity in this case.
There's a negative weights cycle from u to v. So, that's one case we have to worry about in some sense.
But, as long as there are no negative weight cycles, delta of (u,v) will be something bigger than minus infinity, bounded below by some finite value even if you could have negative weights, but still no negative weights cycle for example, there might not be any cycles in your graph.
So that's still interesting.
And, I guess it's useful to note that you can get from A to B in negative infinite time.
It's time travel, if the weights happen that correspond to time.
But when else might shortest paths not exist?
So, this is one case, but there's another, simpler case.
It's not connected.
There might not be any path from u to v. This path might be empty.
There may be no path from u to v. Here we have to define what happens, and here, we'll say it's infinity if there's no path from u to v. So, there are these exceptional cases plus infinity and minus infinity, which are pretty intuitive because it takes a really long time to get from u to v if there's no path there.
You can't get there from here.
OK, but that's the definition.
Most of the time, this is the case we care about, of course.
Usually this is a finite set.
OK, good, so that's the definition.
We're going to get a few basic structural properties about shortest paths that will allow us to obtain good algorithms finding these paths when they exist.
And, in particular, we want to use ideas from dynamic programming.
So, if I want to use dynamic programming to solve shortest paths, what do I need to establish?
What's the first thing I should check?
You've all implemented dynamic programming by now, so should make complete sense hopefully, at least more sense than it did a couple of weeks ago, last week, when we learned it.
Dynamic programming is something that grows on you.
Every year I think I understand it better than the previous year.
But, in particular, when you learned dynamic programming in this class, there is this nice key property that you should check.
Yeah?
Optimal substructure: good.
This is the phrase you should keep in mind.
It's not really enough for dynamic programming to be useful in an efficient way, but it at least tells you that you should be able to try to apply it.
That's a pretty weak statement, but it's something that you should check.
It's definitely pretty much a necessary condition for dynamic programming to make sense.
And so, optimal some structure here means that if I take some shortest path, and I look at a subpath of that shortest path, I claimed that it too is a shortest path, OK, with its respective endpoints; obviously not between the same endpoints.
But if I have some shortest path between two endpoints, I take any subpath and that's also the shortest path.
This is one version of optimal substructure.
This one turns out to be true for this setup.
And, how should I prove an optimal substructure property?
Cut and paste.
Yep, that works here too.
I mean, this isn't always true.
But it's a good technique here.
So, we're going to think about, and I'll do essentially a proof by picture here.
So, suppose you have some subpath of some shortest path.
So, let's say the subpath is x to y.
And, the path goes from u to v. So, we assume that (u,v) is a shortest path.
We want to prove that (x,y) is a shortest path.
Well, suppose (x,y) isn't a shortest path.
Then there is some shorter path that goes from x to y.
But, if you have some shorter path from x to y than this one.
Then I should just erase this part of the shortest path from u to v, and replace it with this shorter one.
So, this is some hypothetical shorter path.
So, suppose this existed.
If that existed, then I should just cut the old path from x to y, and paste in this new one from x to y.
It's strictly shorter.
Therefore, I get a strictly shorter path from u to v. But I assumed u to v was a shortest path: contradiction.
OK, so there is no shorter path.
And that proves the lemma that we have this: subpaths of shortest paths are shortest paths.
OK, this should now be a pretty familiar proof technique.
But, there is yet another instance of cut and paste.
OK, so that's a good sign for computing shortest paths.
I mean, in terms of dynamic programming, we won't look directly at dynamic programming here because we are going to aim for greedy, which is even stronger.
But, next Monday we'll see some dynamic programming approaches.
Intuitively, there are some pretty natural sub-problems here.
I mean, going from u to v, if I want to find what is the shortest path from u to v, well, that's a particular problem.
Maybe it involves computing shortest paths from u to some intermediate point, x, and then from x to u, something like that.
That feels good.
That's like, quadratically, many subproblems.
And, V^2 subproblems, it sounds like that would lead to a dynamic program.
You can make it work out; it's just a little bit trickier than that.
We'll see that next Monday.
But thinking about this intermediate point we get something called the triangle inequality.
So, you've probably heard some form of the triangle inequality before.
It holds in all sorts of geometric spaces, but it also holds for shortest paths, which is slightly less obvious, or more obvious, I guess, depending on your inclination.
So, if you have any triple of vertices, the shortest path from u to v is, at most, the shortest path from u to x plus the shortest path from x to v. Of course, here I need a shortest path weight from u to x, and shortest path weight from x to v. So, this should be pretty natural just from the statement, even more natural if you draw the picture.
So, we have some vertex, u. I'm using wiggly lines to denote potentially long paths as opposed to edges.
We have some intermediate point, x, and we have some target, v, and we are considering these three shortest paths.
This is the shortest path from u to v, or this is its weights.
This is the shortest path from u to x.
And here's its weight, and the shortest path from x to v. And here's its weight.
And, the point is, this should be the shortest path or a shortest path from u to v. And, in particular, one such path is you go from u to x, and then you go from x to v. So, I mean, this sum is just measuring the length of this particular path.
Take the shortest path here; take the shortest path here.
And, this is supposed to be the Min over all paths.
So, certainly this is, at most, this particular path, the sum of these two values, OK, another proof by picture.
Clear?
OK, this stuff is easy.
I assume we'll get into some more set exciting algorithms in particular, which is always more exciting.
Today, we're going to look at a particular version of shortest paths called, or the shortest paths problem called the single source shortest path problem.
OK, it's a little bit more general than go from A to B.
The problem is, you're given a source vertex, and you want to know how to get from that source vertex to everywhere else.
So, we'll call this source vertex s. And from that source, we want to find, let's say, the shortest path weights from s to everyone.
In particular, we'd also like to know the shortest paths, but that isn't too much harder.
So, that's delta of s, v for all vertices, v. OK, so this is actually a little bit harder than the problem we started with a getting from Alderon to Cambridge.
Now, we want to get from Alderon to the entire universe.
OK, it turns out, this is one of the weird things about shortest paths, according to the state-of-the-art we know today, it seems like the following statement will remain true for all time.
But we don't know.
The best algorithms for solving the A to B problem, given s, given t, go from s to t, is no easier than this problem.
It's the best ways we know how to solve going from A to B is to solve how to go from A to everywhere else.
So, we sort of can't help ourselves, but to solve this problem it turns out.
Today, we're going to look at a further restriction on this problem because this is a bit tricky.
Will solve it next class.
But, today we're going to get rid of the negative weight cycle issue by forbidding negative weights.
So, we're going to assume that all of the edge weights are nonnegative, so, for all vertices, u and v. So, in particular, shortest paths exist, provided paths exist.
And, we don't have to worry about these minus infinities.
Delta of (u,v) is always bigger than minus infinity.
It still might be plus infinity if there is no path, but this will make life a lot easier.
And the algorithm we'll cover today really requires this property.
You can't get away without it.
Next class, we'll get away without it with a fancier and slower algorithm.
So, as I hinted at, the main idea we're going to use for the algorithm today is greedy, which should be faster than dynamic programming generally.
And, the tricky part will be proving that the greedy algorithm actually works.
So, I think there's pretty much only one natural way to go about, well, there's one way that works to go about greedy, let's say.
This may be not the obvious one.
So, let me give you a little bit of setup.
The invariant we are going to maintain is that at all times, we have estimates on the distances from the source to every vertex.
When I say distance, I mean shortest path weight.
I'm going to use weight and distance interchangeably here for more intuition.
And, in particular, I want to maintain the set of vertices where those estimates are actually the right answer.
OK, this is little s. This is big S. So, the big S will be the set of all vertices where I know the answer.
What is the shortest path distance from little S to that vertex in big S?
So, for starters, which distance do I know?
Sorry?
s. I know the shortest path distance from s to s because if I assume that all of my weights are nonnegative, I really can't get from s to s any faster than not doing anything.
OK, if I had a negative weight cycle, maybe the distance from s to s is minus infinity.
OK, but I can't have negative weights so there's no way I can get from s to s any faster than zero time.
There might be a longer path that still has zero cost, but it can't be any better than zero.
So, in particular, I know that.
So, initially, S is certainly an s. OK, and the idea is we're going to accumulate more and more vertices that we know.
So, at some point we know the distances from some of the vertices.
So, we have some cloud here.
This is S, and this is everything else.
This is the graph, G. This is the subset of the vertices.
And, there's some edges that go out from there.
And, so we have estimates on how to get to these vertices.
Some of them, we may not have even seen yet.
They may not be connected to this portion of S. I mean: not directly.
They might be connected by some longer path.
They might be in a completely different connected component.
We don't know yet.
Some of them, we have estimates for because we've sort of seen how to get there from S. And the idea is, among all of these nodes where we have estimates, and on to get from little S, which is some vertex in here, to these vertices, we're going to take the one for which the estimate is smallest.
That's the greedy choice.
And, we're just going to add that vertex to S. So, S grows one vertex per step.
Each step, we're going to add to S, the vertex.
Of course, again, this is not a unique, it's a vertex, v, in V minus S. So, it's something we haven't yet computed yet whose estimated distance from S is minimum.
So, we look at all the vertices we haven't yet added to S. Just take the one where we have the estimated smallest distance.
The intuition is that that should be a good choice.
So, if I pick the one that's closest to little s among all the ones that I've seen, among all the paths that I've seen, I sort of have to buy into that those are good paths.
But, I mean, maybe there's some path I didn't see.
Maybe you go out to here and then you take some other path to some vertex, which we've already seen.
OK, the worry is, well, I'd better not say that that's the shortest path because there may have been some other way to get there.
Right, as soon as I add something to S, I declare I've solved the problem for that vertex.
I can't change my answer later.
OK, the estimates can change until they get added to S. So, I don't want to add this vertex to S because I haven't considered this path.
Well, if all my weights are nonnegative, and I take the vertex here that has the shortest estimate from S, so let's suppose this one is the shortest one, then this can't be a shorter path because the distance estimate, at least, from S to that vertex is larger from S to that vertex.
So, no way can I make the path longer and decrease the distance.
That's the intuition.
OK, it's a little bit fuzzy here because I don't have any induction hypotheses set up, and it's going to be a lot more work to prove that.
But that's the intuition why this is the right thing to do.
OK, you have to prove something about the distance estimates for that to be a proof.
But, intuitively, it feels good.
It was a good starting point.
OK, and then presumably we have to maintain these distance estimates.
So, the heart of the algorithm is updating distance estimates, I mean, choosing the best vertex to add to S, that's one step.
Then, updating the distance estimates is sort of where the work is.
And, it turns out we'll only need to update distance estimates of some of the vertices, the ones that are adjacent to v. v was the vertex we just added to S. So, once we add somebody to S, so we grow S by a little bit, then we look at all the new edges that go out of S from that vertex.
We update something.
That's the idea.
So, that's the idea for how we're going to use greedy.
Now I'll give you the algorithm.
So, this is called Dijkstra's algorithm.
Dijkstra is a famous, recently late, if that makes sense, computer scientist from the Netherlands.
And, this is probably the algorithm he is most famous for.
So, the beginning of the algorithm is just some initialization, not too exciting.
OK, but let me tell you what some of the variables mean.
OK, so d is some array indexed by vertices, and the idea is that d of x is the distance estimate for x, so, from S to x. so in particular, it's going to equal the real shortest path weight from S to x when we've added x to our set capital, S. OK, so this is, in particular, going to be the output to the algorithm.
Did you have a question?
Or were you just stretching?
Good.
So, in d of x, when we are done, d of x is the output.
For every vertex, it's going to give us the shortest path weight from S to that vertex.
Along the way, it's going to be some estimated distance from S to that vertex.
And, we're going to improve it over time.
This is an infinity.
So initially, we know that the distance, we know the distance from S to S is zero.
So, we're going to set that to be our estimate.
It's going to be accurate.
Everything else we're going to just set to infinity because we may not be connected.
From the beginning, we don't know much.
S, initially, is going to be infinity.
Immediately, we're going to add little s to big S. And then, the interesting part here is Q, which is going to consist of, initially all the vertices in the graph.
And, it's going to not just be a queue as the letter suggests.
It's going to be a priority queue.
So, it's going to maintain, in particular, the vertex that has the smallest distance estimate.
So, this is a priority queue.
This is really an abuse of notation for a data structure.
OK, so this could be a heap or whatever.
The vertices are keyed on d, our distance estimate.
So, in particular, S will have the, this is going to be a Min heap.
S will be the guy who has the minimum.
Everyone else has the same key initially.
And, we're going to repeatedly extract the minimum element from this queue and do other things.
OK, so this is initialization.
OK, I'm going to call that initialization.
It's a pretty simple thing.
It just takes linear time, nothing fancy going on.
The heart of the algorithm is all in six lines.
And, so this is not really a step.
The first step here that we need to do is we take the vertex whose distance estimate is minimum.
So that, among all the vertices, not yet, and that's currently S is empty.
Q has everyone.
In general, Q will have everyone except S. So, we'll take the vertex, u, that has the minimum key in that priority queue.
So, extract the Min from Q. OK. We're going to add a little u to S, claim that that is now, I mean, that's exactly what we're saying here.
We add to S that vertex that has minimum distance estimate.
And now, we need to update the distances.
So, we're going to look at each adjacent vertex for each v in the adjacency list for u.
We look at a few distances.
So that's the algorithm or more or less.
This is the key.
I should define it a little bit what's going on here.
We talked mainly about undirected graph last time.
Here, we're thinking about undirected graphs.
And, the adjacency list for u here is just going to mean, give me all the vertices for which there is an edge from u to v. So, this is the outgoing adjacency list, not the incoming adjacency list.
Undirected graphs: you list everything.
Directed graphs: here, we're only going to care about those ones.
So, for every edge, (u,v), is what this is saying, we are going to compare the current estimate for v, and this candidate estimate, which intuitively means you go from s to u.
That's d of u because we now know that that's the right answer.
This, in fact, equals, we hope, assuming the algorithm is correct, this should be the shortest path weight from s to u because we just added u to S. And whenever we add something to S, it should have the right value.
So, we could say, well, you take the shortest path from S to u, and then you follow this edge from u to v. That has weight, w, of (u,v).
That's one possible path from S to v. And, if that's a shorter path than the one we currently have in our estimate, if this is smaller than that, then we should update the estimate to be that sum because that's a better path, so, add it to our database of paths, so to speak: OK, very intuitive operation; clearly should not do anything bad.
I mean, these should be paths that makes sense.
We'll prove that in a moment.
That's the first part of correctness, that this never screws up.
And then, the tricky part is to show that it finds all the paths that we care about.
This step is called a relaxation step.
Relaxation is always a difficult technique to teach to MIT students.
It doesn't come very naturally.
But it's very simple operation.
It comes from optimization terminology, programming terminology, so to speak.
And, does this inequality look familiar at all especially when you start writing it this way?
You say, the shortest path from S to v and the shortest path from S to u in some edge from u to v, does that look like anything we've seen?
In fact, it was on this board but I just erased it.
Triangle inequality, yeah.
So, this is trying to make the triangle inequality true.
Certainly, the shortest path from S to v should be less than or equal to, not greater than.
The shortest path from S to u, plus whatever path from u to v, the shortest path should be, at most, that.
So, this is sort of a somewhat more general triangle inequality.
And, we want to, certainly it should be true.
So, if it's not true, we fix it.
If it's greater than, we make it equal.
But we don't want to make it less than because that's not always true.
OK, but certainly, it should be less than or equal to.
So, this is fixing the triangle inequality.
It's trying to make that constraint more true.
In optimization, that's called relaxing the constraint.
OK, so we're sort of relaxing the triangle inequality here.
In the end, we should have all the shortest paths.
That's a claim.
So: a very simple algorithm.
Let's try it out on a graph, and that should make it more intuitive why it's working, and that the rest of the lecture will be proving that it works.
Yeah, this is enough room.
So, oh, I should mention one other thing here.
Sorry.
Whenever we change d of v, this is changing the key of v in the priority queue.
So, implicitly what's happening here in this assignment, this is getting a bit messy, is a decreased key operation, OK, which we talked briefly about last class in the context of minimum spanning trees where we were also decreasing the key.
The point is we were changing the key of one element industry like station step in the priority queue so that if it now becomes the minimum, we should extract here.
And, we are only ever decreasing keys because we are always replacing larger values with smaller values.
So, we'll come back to that later when we analyze the running time.
But, there is some data structure work going on here.
Again, we are abusing notation a bit.
OK, so here is a graph with edge weights.
OK, and I want my priority queue over here.
And, I'm also going to draw my estimates.
OK, now I don't want to cheat.
So, we're going to run the algorithm on this graph.
s will be A, and I want to know the shortest path from A to everyone else.
So, you can check, OK, paths exist.
So, hopefully everything should end up a finite value by the end.
All the weights are nonnegative, so this algorithm should work.
The algorithm doesn't even need connectivity, but it does mean that all the weights are nonnegative.
So, we run the algorithm.
For the initialization, we set the distance estimate for our source to be zero because, in fact, there's only one path from A to A, and that to do nothing, the empty path.
So, I'm going to put the key of zero over here.
And, for everyone else, we're just going to put infinity because we don't know any better at this point.
So, I'll put keys of infinity for everyone else.
OK, so now you can see what the algorithm does is extract the minimum from the queue.
And, given our setup, we'll definitely choose s, or in this case, A.
So, it has a weight of zero.
Everyone else has quite a bit larger weight.
OK, so we look at s, or I'll use A here.
So, we look at A.
We add A to our set, S. So, it's now removed from the queue.
It will never go back in because we never add anything to the queue, start with all the vertices, and extract, and decrease keys.
But we never insert.
So, A is gone.
OK, and now I want to update the keys of all of the other vertices.
And the claim is I only need to look at the vertices that have edges from A.
So, there's an edge from A to B, and that has weight ten.
And so, I compare: well, is it a good idea to go from A to A, which costs nothing, and then to go along this edge, AB, which costs ten?
Well, it seems like a pretty good idea because that has a total weight of zero plus ten, which is ten, which is much smaller than infinity.
So, I'm going to erase this infinity; write ten, and over in the queue as well.
That's the decreased key operation.
So now, I know a path from A to B.
Good.
A to C is the only other edge.
Zero plus three is less than infinity, so, cool.
I'll put three here for C, and C is there.
OK, the other vertices I don't touch.
I'm going to rewrite them here, but the algorithm doesn't have to copy them.
Those keys were already there.
It's just touching these two.
OK, that was pretty boring.
Now we look at our queue, and we extract the minimum element.
So, A is no longer in there, so the minimum key here is three.
So, the claim is that this is a shortest path; from A to C, here is the shortest path from A to C. There's no other shorter way.
You could check that, and we'll prove it in a moment.
Cool, so we'll remove C from the list.
It's gone.
Then we look at all of the outgoing edges from C. So, there's one that goes up to B, which has weight four, four plus three, which is the shortest path weight from A to C. So, going from A to C, and C to B should cost three plus four, which is seven, which is less than ten.
So, we found an even better path to get to B.
It's better to go like this than it is to go like that.
So, we write seven for B, and there's an outgoing edge from C to d which costs eight.
Three plus eight is 11.
11 is less than infinity last time I checked.
So, we write 11 for d. Then we look at E. We have three plus two is five, which is less than infinity.
So, we write five for the new key for E. At this point, we have finite shortest paths to everywhere, but they may not be the best ones.
So, we have to keep looking.
OK, next round of the algorithm, we extract the minimum key among all these.
OK, it's not B, which we've seen though probably know the answer to.
But it's E. E has the smallest key.
So, we now declare this to be a shortest path.
The way we got to E was along this path: A to C, C to E, declare that to be shortest.
We claim we're done with E. But we still have to update.
What about all the outgoing edges from E?
There's only one here.
It costs five plus nine, which is 14, which is bigger than 11.
So, no go.
That's not an interesting path.
Our previous path, which went like this at a cost of the 11, is better than the one we are considering now.
I'm drawing the whole path, but the algorithm is only adding these two numbers.
OK, good.
So, I don't change anything.
Seven, 11, and five is removed, or E is removed.
Our new keys are seven and 11.
So, we take the key, seven, here, which is for element B, vertex B.
We declare the path we currently have in our hands from A to B, which happens to be this one.
Algorithm can't actually tell this, by the way, but we're drawing it anyway.
This path, A, C, B, is the candidate shortest path.
The claim is it is indeed shortest.
Now, we look at all the outgoing edges.
There's one that goes back to C at a cost of seven plus one, which is eight, which is bigger than three, which is good.
We already declared C to be done.
But the algorithm checks this path and says, oh, that's no better.
And then we look at this other edge from B to d. That costs seven plus two, which is nine, which is better than 11.
So, we, in fact, found an even shorter path.
So, the shortest path weight, now, for d, is nine because there is this path that goes A, C, B, d for a total cost of three plus four plus two is nine.
Cool, now there's only one element in the queue.
We remove it.
d: we look at the outgoing edges.
There's one going here which costs nine plus seven, which is 16, which is way bigger than five.
So, we're done.
Don't do anything.
At this point, the queue is empty.
And the claim is that all these numbers that are written here, the final values are the shortest path weights.
This looks an awful lot like a five, but it's an s. It has a weight of zero.
I've also drawn in here all the shortest paths.
And, this is not hard to do.
We're not going to talk about it too much in this class, but it's mentioned in a little bit more detail at the end of the textbook.
And it's something called the shortest path tree.
It's just something good to know about if you actually want to compute shortest paths.
In this class, we mainly worry about the weights because it's pretty much the same problem.
The shortest path tree is the union of all shortest paths.
And in particular, if you look at each vertex in your graph, if you consider the last edge into that vertex that was relaxed among all vertices, u, you look at the edges, (u,v), say, was that last one to relax?
So, just look at the last edges we relaxed here.
You put them all together: that's called a shortest path tree.
And, it has the property that from S to everywhere else, there is a unique path down the tree.
And it's the shortest path.
It's the shortest path that we found.
OK, so you actually get shortest paths out of this algorithm even though it's not explicitly described.
All we are mainly talking about are the shortest path weights.
Algorithm clear at this point?
Feels like it's doing the right thing?
You can check all those numbers are the best paths.
And now we're going to prove that.
So: correctness.
So the first thing I want to prove is that relaxation never makes a mistake.
If it ever sets d of v to be something, I want to prove that d of v is always an upper bound on delta.
So, we have this variant.
It's greater than or equal to delta of s, v for all v. And, this invariant holds at all times.
So, after initialization, it doesn't hold before initialization because d isn't defined then.
But if you do this initialization where you set S to zero, and everyone else to infinity, and you take any sequence of relaxation steps, then this variant will hold after each relaxation step you apply.
This is actually a very general lemma.
It's also pretty easy to prove.
It holds not only for Dijkstra's algorithm, but for a lot of other algorithms we'll see.
Pretty much every algorithm we see will involve relaxation.
And, this is saying no matter what relaxations you do, you always have a reasonable estimate in the sense that it's greater than or equal to the true shortest path weight.
So, it should be converging from above.
So, that's the lemma.
Let's prove it.
Any suggestions on how we should prove this lemma?
What technique might we use?
What's that?
Cut and paste?
It would be good for optimal substructure.
Cut and paste: maybe sort of what's going on here but not exactly.
Something a little more general.
It's just intuition here; it doesn't have to be the right answer.
In fact, many answers are correct, have plausible proofs.
Induction, yeah.
So, I'm not going to write induction here, but effectively we are using induction.
That's the answer I was expecting.
So, there is sort of an induction already in time going on here.
We say after initialization it should be true.
That's our base case.
And then, every relaxation we do, it should still be true.
So, we're going to assume by induction that all the previous relaxations worked, and then we're going to prove that the last relaxation, whatever it is, works.
So, first let's do the base case.
So, this is after an initialization, let's say, initially.
So, initially we have d of s equal to zero.
And we have d of v equal to infinity for all other vertices, for all vertices, v, not equal to little s. OK, now we have to check that this inequality holds.
Well, we have delta of s, s. We've already argued that that's zero.
You can't get negative when there are only nonnegative edge weights.
So, that's the best.
So, certainly zero is greater than or equal to zero.
And, we have everything else, well, I mean, delta of S, v is certainly less than or equal to infinity.
So this holds.
Everything is less than or equal to infinity.
So: base case is done.
So, now we do an induction.
And, I'm going to write it as a proof by contradiction.
So, let's say, suppose that this fails to hold at some point.
So, suppose for contradiction that the invariant is violated.
So, we'd like to sue the violator and find a contradiction.
So, it's going to be violated.
So, let's look at the first violation, the first time it's violated.
So, this is, essentially, again, a proof by induction.
So, let's say we have some violation, d of v is less than delta of s, v. That would be bad if we somehow got an estimate smaller than the shortest path.
Well, then I think about looking at the first violation is we know sort of by induction that all other values are correct.
OK, d of v is the first one where we've screwed up.
So, the invariant holds everywhere else.
Well, what caused this to fail, this invariant to be violated, is some relaxation, OK, on d of v. So, we had some d of v, and we replaced it with some other d of u plus the weight of the edge from u to v. And somehow, this made it invalid.
So, d of v is somehow less than that.
We just set d of v to this.
So, this must be less than delta of s, v. The claim is that that's not possible because, let me rewrite a little bit.
We have d of u plus w of (u,v).
And, we have our induction hypothesis, which holds on u, u of some other vertex.
We know that d of u is at least delta of s, u.
So, this has to be at least delta of s, u plus w of u, v. Now, what about this w of u, v?
Well, that's some path from u to v. So, it's got to be bigger than the shortest path or equal.
So certainly, this is greater than or equal to delta of u, v. OK, it could be larger if there's some multi-edged path that has a smaller total weight, but it's certainly no smaller than delta of u, v. And, this looks like a good summation, delta of S to u, and u to v is a triangle inequality, yeah.
So, that is, it's upside down here.
But, the triangle S, u, u to v, so this is only longer than S to v. OK, so we have this thing, which is simultaneously greater than or equal to the shortest path weight from S to v, and also strictly less than the shortest path weight from S to v. So, that's a contradiction.
Maybe contradiction is the most intuitive way isn't the most intuitive way to proceed.
The intuition, here, is whatever you assign d of v, you have a path in mind.
You inductively had a path from s to u.
Then you added this edge.
So, that was a real path.
We always know that every path has weight greater than or equal to the shortest path.
So, it should be true, and here's the inductive proof.
All right, moving right along, so this was an easy warm-up.
We have greater than or equal to.
Now we have to prove less than or equal to at the end of the algorithm.
This is true all the time; less than or equal to will only be true at the end.
So, we are not going to prove less than or equal to quite yet.
We're going to prove another lemma, which again, so both of these lemmas are useful for other algorithms, too.
So, we're sort of building some shortest path theory that we can apply later.
This one will give you some intuition about why relaxation, not only is it not bad, it's actually good.
Not only does it not screw up anything, but it also makes progress in the following sense.
So, suppose you knew the shortest path from s to some vertex.
OK, so you go from s to some other vertices.
Then you go to u.
Then you go to v. Suppose that is a shortest path from s to v. OK, and also suppose that we already know in d of u the shortest path weight from s to u.
So, suppose we have this equality.
We now know that we always have a greater than or equal to.
Suppose they are equal for u, OK, the vertex just before v in the shortest path.
OK, and suppose we relax that edge, (u,v), OK, which is exactly this step.
This is relaxing the edge, (u,v).
But we'll just call it relaxation here.
After that relaxation, d of v equals delta of (s,v).
So, if we had the correct answer for u, and we relax (u,v), then we get the correct answer for v. OK, this is good news.
It means, if inductively we can somehow get the right answer for u, now we know how to get the right answer for v. In the algorithm, we don't actually know what the vertex just before v in the shortest path is, but in the analysis we can pretty much know that.
So, we have to prove this lemma.
This is actually even easier than the previous one: don't even need induction because you just work through what's going on in relaxation, and it's true.
So, here we go.
So, we're interested in this value, delta of Ss v. And we know what the shortest path is.
So, the shortest path weight is the weight of this path.
OK, so we can write down some equality here.
Well, I'm going to split out the first part of the path and the last part of the path.
So, we have, I'll say, the weight from s, so, this part of the path from s to u, plus the weight of this edge, u, v. Remember, we could write w of a path, and that was the total weight of all those edges.
So, what is this, the weight of this path from S to u?
Or, what property should I use to figure out what that value is?
Yeah?
s to v is the shortest path, right?
So, by optimal substructure, from s to u is also a shortest path.
So, this is delta of s, u.
Cool.
We'll hold on for now.
That's all we're going to say.
On the other hand, we know from this lemma that matter what we do, d of v is greater than or equal to delta of s, v. So, let's write that down.
So, there's a few cases, and this will eliminate some of the cases.
By that lemma correctness one, we know that d of v is greater than or equal to delta of s, v. So, it's either equal or greater than at all times.
So, I'm thinking about the time before we do the relaxation, this (u,v).
So, at that point, this is certainly true.
So, either they're equal before relaxation or it's greater.
OK, if they are equal before relaxation, we're happy because relaxation only decreases values by correctness one.
It can't get any smaller than this, so after relaxation it will also be equal.
OK, so in this case we're done.
So, that's a trivial case.
So let's now suppose that d of v is greater than delta of s, v before relaxation.
That's perfectly valid.
Hopefully now we fix it.
OK, well the point is, we know this delta s, v. It is this sum.
OK, we also know this.
So, delta of s, u we know is d of u.
And, we have this w u, v. So, delta of s, v is d of u plus w of (u,v) because we are assuming we have this shortest path structure where you go from s to u, and then you follow the edge, (u,v).
So, we know this.
So, we know d of v is greater than d of u plus w of (u,v).
By golly, that's this condition in relaxation.
So, we're just checking, relaxation actually does something here.
OK, if you had the wrong distance estimate, this if condition is satisfied.
Therefore, we do this.
So, in this case, we relax.
So, I'm just relaxing.
Then, we set d of v to d of u plus WUV, which is what we want.
OK, so we set d of v to d of u plus w of (u,v).
And, this equals, as we said here, delta of S, v, which is what we wanted to prove.
Done.
OK, I'm getting more and more excited as we get into the meat of this proof.
Any questions so far?
Good.
Now comes the hard part.
These are both very easy lemmas, right?
I'll use these two boards.
We don't need these proofs anymore.
We just need these statements: correctness one, correctness lemma; great names.
So, now finally we get to correctness two.
So, we had one and one and a half.
So, I guess correctness is, itself, a mini-trilogy, the mini-series.
OK, so correctness two says when the algorithm is done, we have the right answer.
This is really correctness.
But, it's going to build on correctness one and correctness lemma.
So, we want d of v to equal delta of s, v for all vertices, v at the end of the algorithm.
That is clearly our goal.
Now, this theorem is assuming that all of the weights are nonnegative, just to repeat.
It doesn't assume anything else.
So, it's going to get the infinities right.
But, if there are minus infinities, all bets are off.
OK, even if there's any negative weight edge anywhere, it's not going to do the right thing necessarily.
But, assuming all the weights are nonnegative, which is reasonable if they're measuring time.
Usually it costs money to travel along edges.
They don't pay you to do it.
But who knows?
So, I need just to say a few things.
One of the things we've mentioned somewhere along the way is when you add a vertex to S, you never change its weight.
OK, that actually requires proof.
I'm just going to state it here.
It's not hard to see.
d of v doesn't change.
OK, this is essentially an induction once v is added to S. OK, this will actually followed by something we'll say in a moment.
OK, so all I really care about is when a vertex is added to S, we better have the right estimate because after that, we're not going to change it, let's say.
OK, we could define the algorithm that way.
We are not, but we could.
I'll say more about this in a second.
So, all we care about is whether d of v equals delta of s, v. That's what we want to prove.
So, it's clearly that.
It should be true at the end.
But, it suffices to prove that it holds when v is added to S, to capital S. OK, this actually implies the first statement.
It has sort of a funny implication.
But, if we can prove this, that d of v equals delta of s, v, when you add to S, we know relaxation only decreases value.
So, it can't get any smaller.
It would be from correctness one.
Correctness one says we can't get any smaller than delta.
So, if we get a quality at that point, we'll have a quality from then on.
So, that actually implies d of v never changes after that point.
OK, so we're going to prove this.
Good.
Well, suppose it isn't true.
So this would be a proof by a contradiction.
Suppose for contradiction that this fails to hold.
And, let's look at the first failure.
Suppose u is the first vertex -- -- that's about to be added to S. I want to consider the time right before it's added to S, for which we don't have what we want.
These are not equal.
d of u does not equal delta of s, u.
Well, if they're not equal, we know from correctness one that d of E is strictly greater than delta of s, u, so, d of u.
So, we have d of u is strictly greater than delta of s, u. OK, that's the beginning of the proof, nothing too exciting yet, just some warm-up.
OK, but this, used already correctness one.
I think that's the only time that we use it in this proof.
OK, so I sort of just want to draw picture of what's going on.
But I need a little bit of description.
So, let's look at the shortest path.
Somehow, d of u is greater than the shortest path.
So, consider the shortest path or a shortest path.
Let p be a shortest path, not just any shortest path, but the shortest path from s to u. OK, so that means that the weight of this path is the shortest path weight.
So, we have some equations for what's going on here.
So, we care about delta of s, u.
Here's a path with that weight.
It's got to be one because shortest paths exist here; slight exceptional cases if it's a plus infinity, but I'm not going to worry about that.
So, let me draw a picture somewhere.
So, we have s. We have u.
Here is the shortest path from s to u.
That's p. No idea what it looks like so far.
Now, what we also have is the notion of capital S. So, I'm going to draw capital S. So, this is big S. We know that little s is in big S. We know that u is not yet in big S. So, I haven't screwed up anything yet, right?
This path starts in S and leaves it at some point because until we are about to add u to S, so it hasn't happened yet, so u is not in S. Fine.
What I want to do is look at the first place here where the path, p, exits S. So, there is some vertex here.
Let's call it x.
There's some vertex here.
We'll call it y. OK, possibly x equals S. Possibly y equals u.
But it's got to exit somewhere, because it starts inside and ends up outside.
And it's a finite path.
OK, so consider the first time it happens; not the second time, the first.
OK, so consider the first edge, (x,y), where p exits capital S. The shortest path from s to u exits capital S. It's got to happen somewhere.
Cool, now, what do we know?
Little x is in S. So, it has the right answer because u, we were about to add u to S, and that was the first violation of something in S that has the wrong d of x estimate.
So, d of x equals delta of s, x.
Because we are looking at the first violation, x is something that got added before.
So, by induction on time, or because we had the first violation, d of x equals the shortest path weight from S to x.
So, that's good news.
Now we are trying to apply this lemma.
It's the only thing left to do.
We haven't used this lemma for anything.
So, we have the setup.
If we already know that one of the d values is the right answer, and we relaxed the edge that goes out from it, then we get another right answer.
So that's what I want to argue over here.
We know that the d of x equals this weight because, again, subpaths of shortest paths are shortest paths.
We have optimal substructure, so this is a shortest path, from S to x.
It might not be the only one, but it is one.
So we know that matches.
Now, I want to think about relaxing this edge, (x,y).
Well, x is in capital S. And, the algorithm says, whenever you add a vertex, u, to the big set, S, you relax all the edges that go out from there.
OK, so when we added x to S, and we now look far in the future, we're about to add some other vertex.
Right after we added x to S, we relax this edge, (x,y), because we relaxed every edge that goes out from x, OK, whatever they were.
Some of them went into S. Some of them went out.
Here's one of them.
So, when we added x to S, we got XS.
When we added x to S, we relaxed the edge, (x,y).
OK, so now we're going to use the lemma.
So, by the correctness lemma -- What do you get?
Well, we add this correct shortest path weight to x now.
We relax the edge, (x,y).
So, now we should have the correct shortest path weight for y. d of y equals delta of s, y. OK, this is sometime in the past.
In particular, now, it should still be true because once you get down to the right answer you never change it.
OK, we should be done.
OK, why are we done?
Well, what else do we know here?
We assumed something for contradiction, so we better contradict that.
We assume somehow, d of u is strictly greater than delta of s, u.
So, d of u here is strictly greater than the length of this whole path.
Well, we don't really know whether u equals y.
It could, could not.
And, but what do we know about this shortest path from S to y?
Well, it could only be shorter than from S to u because it's a subpath.
And it's the shortest path because it's the subpath of the shortest path.
The shortest path from S to y has to be less than or equal to the shortest path from S to u. OK, S to y: less than or equal to s, u, OK, just because the subpath.
I'm closer.
I've got delta of s, u now.
Somehow, I want to involve d of u.
So, I want to relate d of y to d of u.
What do I know about d of u?
Yeah?
d of u is smaller because we have a Min heap, yeah.
We always chose, let's erase, it's way down here.
We chose u.
This is the middle of the algorithm.
It's the reason I kept this to be the minimum key.
This is keyed on d. So, we know that at this moment, when we're trying to add u to S, right, y is not in S, and u is not in S. They might actually be the same vertex.
But both of these vertices, same or not, are outside S. We chose u because d of u has the smallest d estimate.
So, d of y has to be greater than or equal to d of u.
It might be equal if they're the same vertex, but it's got to be greater than or equal to.
So, d of y here is greater than or equal to d of u.
So, here we're using the fact that we actually made a greedy choice.
It's the one place we're using the greedy choice.
Better use it somewhere because you can't just take an arbitrary vertex and declare it to be done.
You've got to take the greedy one.
OK, now we have d of u is less than or equal to delta of s, u, which contradicts this.
OK, sort of magical that that all just worked out.
But sort of like the previous proofs, you just see what happens and it works.
OK, that's the approximation.
The only real idea here is to look at this edge.
In fact, you could look at this edge too.
But let's look at some edge that comes from S and goes out of S, and argue that while x has to be correct, and what we made x correct, y had to be correct, and now, why the hell are we looking at u?
y is the thing you should have looked at.
And, there you get a contradiction because y had the right answer.
If u equals y, that's fine, or if u and y were sort of equally good, that's also fine if all these weights were zero.
So, the picture might actually look like this.
But, in that case, d of u is the correct answer.
It was delta SU.
We assumed that it wasn't.
That's where we're getting a contradiction.
Pretty clear?
Go over this proof.
It's a bit complicated, naturally.
OK, we have a little bit more to cover, some easier stuff.
OK, the first thing is what's the running time of this algorithm?
I'll do this very quick because we're actually seen this many times before last class.
There was some initialization.
The initialization, which is no longer here, is linear time.
No big deal.
OK, extract Min.
Well, that's some data structure.
So, we have something like size of V. Every vertex we extract the Min once, and that's it.
So, size of V, extract mins.
OK, so that's pretty simple.
OK, then we had this main loop.
This is a completely conceptual operation.
S is not actually used in the algorithm.
It's just for thinking.
OK, so this takes zero time.
Got to love it.
OK, and now the heart is here.
So, how many times does this loop iterate?
That's the degree of u.
So, what is the total number of times that we execute a relaxation step?
It doesn't necessarily mean we do this, but we at least execute this body.
Over the whole algorithm, how many times do we do this?
Every vertex, we look at all the outgoing edges from there.
So, the total would be?
Number of edges, yeah.
So, this number of edges iterations.
OK, this is essentially the handshaking lemma we saw last time, but for directed graphs.
And we are only looking at the outgoing edges.
So, it's not a factor of two here because you're only outgoing from one side.
So, we have number of reiterations.
In the worst case, we do a decreased key for everyone.
So, at most: E decreased keys.
OK, so the time is, well, we have v extract Mins, so the time to do an extract Min, whatever that is.
And we have E decreased keys, whatever that is, and this is exactly the running time we had for Prim's algorithm for a minimum spanning tree last time.
And, it depends what data structure you use, what running time you get.
So, I'm going to skip the whole table here.
But, if you use an array, the final running time will be V^2 because you have order of v extract Min, and you have constant time decreased key.
If you use a binary heap, which we know and love, then we have order log v for each operation.
And so, this is V plus E log V. And, so that's what we know how to do.
And, if you use this fancy data structure called a Fibonacci heap, you get constant time decreased key amortized.
And, you get an E plus v log v worst case bound on the running time.
So, this is the best we know how to solve shortest paths without any extra assumptions, single source shortest paths with non-negative edge weights in general.
OK, this is almost as good and this is sometimes better than that.
But these are essentially irrelevant except that you know how to do these.
You don't know how to do a Fibonacci heap unless you read that in the chapter of the book.
That's why we mention the top two running times.
OK, I want to talk briefly about a simpler case, which you may have seen before.
And so it's sort of fun to connect this up to breadth first search in a graph.
So, I mean that ends Dijkstra, so to speak.
But now I want to think about a special case where the graph is unweighted, meaning w of (u,v) equals one for all vertices, u and v. OK, suppose we had that property.
Can we do any better than Dijkstra?
Can we do better than this running time?
Well, we probably have to look at all the edges and all the vertices.
So, the only thing I'm questioning is this log v. Can I avoid that?
I gave away the answer a little bit.
The answer is called breadth first search, or BFS, which you have probably seen before.
Next to depth first search, it's one of the standard ways to look at the graph.
But we can say a little bit more than you may have seen before.
Breadth for search is actually Dijkstra's algorithm: kind of nifty.
There are two changes.
First change is that breadth for search does not use a priority queue.
I'll just tell you what it uses instead.
You can use a queue first in first out honest-to-goodness queue instead of a priority queue.
OK, it turns out that works.
Instead of doing extract Min, you just take the first thing off the queue.
Instead of doing decreased key, OK, here's a subtlety.
But, this if statement changes a little bit.
So, here is the relaxation step.
So, in order to relax, you say this much simpler thing.
If we haven't visited v yet, then we declare it to have the shortest path weight, say, d of v is d of u plus one, which is the weight of the edge, (u,v).
And we add v to the end of the queue.
So, now, we start with the queue empty.
Actually, it will just contain the vertex, S, because that's the only thing we know the shortest path for.
So, the queue is just for, I know the shortest path of this thing.
Just deal with it when you can't look at all the outgoing edges when you can.
So, initially that's just S. You say, well, for all the outgoing edges, S has zero.
All the outgoing edges from there have weight one.
The shortest path weight from the source is one.
You certainly can't do any better than that if all the weights are one.
OK, so we add all those vertices to the end of the queue.
Then, we process things in order, and we just keep incrementing, if their value is d of u, add one to it.
That's d of v. And then we are going to add v to S what we get to it in the queue.
OK, that is breadth for search, very simple.
And, you can look at the text for the algorithm and for an example because I don't have time to cover that.
But the key thing is that the time is faster.
The time is order V plus E because as before, we only look at each edge once we look at all the outgoing edges from all the vertices.
As soon as we set d of v to something, it will remain that.
We never touch it.
We are going to add it to S. That only happens once.
So, this if statement, and so on, in the in-queuing, is done order E times, or actually E times, exactly.
An in-queuing to a queue, and de-queuing from a queue, that's what we use instead of extract Min, take constant time, so the total running time, number of vertices plus the number of edges.
OK, not so obvious that this works, but you can prove that it works using the Dijkstra analysis.
All you have to do is prove that the FIFO priority queue.
Once you know that, by the correctness of Dijkstra you get the correctness of breadth for search.
So, not only is breadth for search finding all the vertices, which is maybe what you normally use it for, but it finds the shortest path weights from S to every other vertex when the weights are all one.
So, there we go: introduction to shortest paths.
Next time we'll deal with negative weights.
Good morning, everyone.
Glad you are all here bright and early.
I'm counting the days till the TA's outnumber the students.
They'll show up.
We return to a familiar story.
This is part two, the Empire Strikes Back.
So last time, our adversary, the graph, came to us with a problem.
We have a source, and we had a directed graph, and we had weights on the edges, and they were all nonnegative.
And there was happiness.
And we triumphed over the Empire by designing Dijkstra's algorithm, and very efficiently finding single source shortest paths, shortest path weight from s to every other vertex.
Today, however, the Death Star has a new trick up its sleeve, and we have negative weights, potentially.
And we're going to have to somehow deal with, in particular, negative weight cycles.
And we saw that when we have a negative weight cycle, we can just keep going around, and around, and around, and go back in time farther, and farther, and farther.
And we can get to be arbitrarily far back in the past.
And so there's no shortest path, because whatever path you take you can get a shorter one.
So we want to address that issue today, and we're going to come up with a new algorithm actually simpler than Dijkstra, but not as fast, called the Bellman-Ford algorithm.
And, it's going to allow negative weights, and in some sense allow negative weight cycles, although maybe not as much as you might hope.
We have to leave room for a sequel, of course.
OK, so the Bellman-Ford algorithm, invented by two guys, as you might expect, it computes the shortest path weights.
So, it makes no assumption about the weights.
Weights are arbitrary, and it's going to compute the shortest path weights.
So, remember this notation: delta of s, v is the weight of the shortest path from s to v. s was called a source vertex.
And, we want to compute these weights for all vertices, little v. The claim is that computing from s to everywhere is no harder than computing s to a particular location.
So, we're going to do for all them.
It's still going to be the case here.
And, it allows negative weights.
And this is the good case, but there's an alternative, which is that Bellman-Ford may just say, oops, there's a negative weight cycle.
And in that case it will just say so.
So, they say a negative weight cycle exists.
Therefore, some of these deltas are minus infinity.
And that seems weird.
So, Bellman-Ford as we'll present it today is intended for the case, but there are no negative weights cycles, which is more intuitive.
It sort of allows them, but it will just report them.
In that case, it will not give you delta values.
You can change the algorithm to give you delta values in that case, but we are not going to see it here.
So, in exercise, after you see the algorithm, exercise is: compute these deltas in all cases.
So, it's not hard to do.
But we don't have time for it here.
So, here's the algorithm.
It's pretty straightforward.
As I said, it's easier than Dijkstra.
It's a relaxation algorithm.
So the main thing that it does is relax edges just like Dijkstra.
So, we'll be able to use a lot of dilemmas from Dijkstra.
And proof of correctness will be three times shorter because the first two thirds we already have from Dijkstra.
But I'm jumping ahead a bit.
So, the first part is initialization.
Again, d of v will represent the estimated distance from s to v. And we're going to be updating those estimates as the algorithm goes along.
And initially, d of s is zero, which now may not be the right answer conceivably.
Everyone else is infinity, which is certainly an upper bound.
OK, these are both upper bounds on the true distance.
So that's fine.
That's initialization just like before.
And now we have a main loop which happens v minus one times.
We're not actually going to use the index i.
It's just a counter.
And we're just going to look at every edge and relax it.
It's a very simple idea.
If you learn about relaxation, this is the first thing you might try.
The question is when do you stop.
It's sort of like I have this friend to what he was like six years old he would claim, oh, I know how to spell banana.
I just don't know when to stop.
OK, same thing with relaxation.
This is our relaxation step just as before.
We look at the edge; we see whether it violates the triangle inequality according to our current estimates we know the distance from s to v should be at most distance from s to plus the weight of that edge from u to v. If it isn't, we set it equal.
We've proved that this is always an OK thing to do.
We never violate, I mean, these d of v's never get too small if we do a bunch of relaxations.
So, the idea is you take every edge.
You relax it.
I don't care which order.
Just relax every edge, one each.
And that do that V minus one times.
The claim is that that should be enough if you have no negative weights cycles.
So, if there's a negative weight cycle, we need to figure it out.
And, we'll do that in a fairly straightforward way, which is we're going to do exactly the same thing.
So this is outside before loop here.
We'll have the same four loops for each edge in our graph.
We'll try to relax it.
And if you can relax it, the claim is that there has to be a negative weight cycle.
So this is the main thing that needs proof.
OK, and that's the algorithm.
So the claim is that at the ends we should have d of v, let's see, L's so to speak.
d of v equals delta of s comma v for every vertex, v. If we don't find a negative weight cycle according to this rule, that we should have all the shortest path weights.
That's the claim.
Now, the first question is, in here, the running time is very easy to analyze.
So let's start with the running time.
We can compare it to Dijkstra, which is over here.
What is the running time of this algorithm?
V times E, exactly.
OK, I'm going to assume, because it's pretty reasonable, that V and E are both positive.
Then it's V times E. So, this is a little bit slower, or a fair amount slower, than Dijkstra's algorithm.
There it is: E plus V log V is essentially, ignoring the logs is pretty much linear time.
Here we have something that's at least quadratic in V, assuming your graph is connected.
So, it's slower, but it's going to handle these negative weights.
Dijkstra can't handle negative weights at all.
So, let's do an example, make it clear why you might hope this algorithm works.
And then we'll prove that it works, of course.
But the proof will be pretty easy.
So, I'm going to draw a graph that has negative weights, but no negative weight cycles so that I get an interesting answer.
Good.
The other thing I need in order to make the output of this algorithm well defined, it depends in which order you visit the edges.
So I'm going to assign an arbitrary order to these edges.
I could just ask you for an order, but to be consistent with the notes, I'll put an ordering on it.
Let's say I put number four, say that's the fourth edge I'll visit.
It doesn't matter.
But it will affect what happens during the algorithm for a particular graph.
Do they get them all?
One, two, three, four, five, six, seven, eight, OK. And my source is going to be A.
And, that's it.
So, I want to run this algorithm.
I'm just going to initialize everything.
So, I set the estimates for s to be zero, and everyone else to be infinity.
And to give me some notion of time, over here I'm going to draw or write down what all of these d values are as the algorithm proceeds because I'm going to start crossing them out and rewriting them that the figure will get a little bit messier.
But we can keep track of it over here.
It's initially zero and infinities.
Yeah?
It doesn't matter.
So, for the algorithm you can go to the edges in a different order every time if you want.
We'll prove that, but here I'm going to go through the same order every time.
Good question.
It turns out it doesn't matter here.
OK, so here's the starting point.
Now I'm going to relax every edge.
So, there's going to be a lot of edges here that don't do anything.
I try to relax n minus one.
I'd say, well, I know how to get from s to B with weight infinity.
Infinity plus two I can get to from s to E. Well, infinity plus two is not much better than infinity.
OK, so I don't do anything, don't update this to infinity.
I mean, infinity plus two sounds even worse.
But infinity plus two is infinity.
OK, that's the edge number one.
So, no relaxation edge number two, same deal as number three, same deal, edge number four we start to get something interesting because I have a finite value here that says I can get from A to B using a total weight of minus one.
So that seems good.
I'll write down minus one here, and update B to minus one.
The rest stay the same.
So, I'm just going to keep doing this over and over.
That was edge number four.
Number five, we also get a relaxation.
Four is better than infinity.
So, c gets a number of four.
Then we get to edge number six.
That's infinity plus five is worse than four.
OK, so no relaxation there.
Edge number seven is interesting because I have a finite value here minus one plus the weight of this edge, which is three.
That's a total of two, which is actually better than four.
So, this route, A, B, c is actually better than the route I just found a second ago.
So, this is now a two.
This is all happening in one iteration of the main loop.
We actually found two good paths to c. We found one better than the other.
OK, and that was edge number seven, and edge number eight is over here.
It doesn't matter.
OK, so that was round one of this outer loop, so, the first value of i. i equals one.
OK, now we continue.
Just keep going.
So, we start with edge number one.
Now, minus one plus two is one.
That's better than infinity.
It'll start speeding up.
It's repetitive.
It's actually not too much longer until we're done.
Number two, this is an infinity so we don't do anything.
Number three: minus one plus two is one; better than infinity.
This is vertex d, and it's number three.
Number four we've already done.
Nothing changed.
Number five: this is where we see the path four again, but that's worse than two.
So, we don't update anything.
Number six: one plus five is six, which is bigger than two, so no good.
Go around this way.
Number seven: same deal.
Number eight is interesting.
So, we have a weight of one here, a weight of minus three here.
So, the total is minus two, which is better than one.
So, that was d. And, I believe that's it.
So that was definitely the end of that round.
So, it's I plus two because we just looked at the eighth edge.
And, I'll cheat and check.
Indeed, that is the last thing that happens.
We can check the couple of outgoing edges from d because that's the only one whose value just changed.
And, there are no more relaxations possible.
So, that was in two rounds.
The claim is we got all the shortest path weights.
The algorithm would actually loop four times to guarantee correctness because we have five vertices here and one less than that.
So, in fact, in the execution here there are two more blank rounds at the bottom.
Nothing happens.
But, what the hell?
OK, so that is Bellman-Ford.
I mean, it's certainly not doing anything wrong.
The question is, why is it guaranteed to converge in V minus one steps unless there is a negative weight cycle?
Question?
Right, so that's an optimization.
If you discover a whole round, and nothing happens, so you can keep track of that in the algorithm thing, you can stop.
In the worst case, it won't make a difference.
But in practice, you probably want to do that.
Yeah?
Good question.
All right, so some simple observations, I mean, we're only doing relaxation.
So, we can use a lot of our analysis from before.
In particular, the d values are only decreasing monotonically.
As we cross out values here, we are always making it smaller, which is good.
Another nifty thing about this algorithm is that you can run it even in a distributed system.
If this is some actual network, some computer network, and these are machines, and they're communicating by these links, I mean, it's a purely local thing.
Relaxation is a local thing.
You don't need any global strategy, and you're asking about, can we do a different order in each step?
Well, yeah, you could just keep relaxing edges, and keep relaxing edges, and just keep going for the entire lifetime of the network.
And eventually, you will find shortest paths.
So, this algorithm is guaranteed to finish in V rounds in a distributed system.
It might be more asynchronous.
And, it's a little harder to analyze.
But it will still work eventually.
It's guaranteed to converge.
And so, Bellman-Ford is used in the Internet for finding shortest paths.
OK, so let's finally prove that it works.
This should only take a couple of boards.
So let's suppose we have a graph and some edge weights that have no negative weight cycles.
Then the claim is that we terminate with the correct answer.
So, Bellman-Ford terminates with all of these d of v values set to the delta values for every vertex.
OK, the proof is going to be pretty immediate using the lemmas that we had from before if you remember them.
So, we're just going to look at every vertex separately.
So, I'll call the vertex v. The claim is that this holds by the end of the algorithm.
So, remember what we need to prove is that at some point, d of v equals delta of s comma v because we know it decreases monotonically, and we know that it never gets any smaller than the correct value because relaxations are always safe.
So, we just need to show at some point this holds, and that it will hold at the end.
So, by monotonicity of the d values, and by correctness part one, which was that the d of v's are always greater than or equal to the deltas, we only need to show that at some point we have equality.
So that's our goal.
So what we're going to do is just look at v, and the shortest path to v, and see what happens to the algorithm relative to that path.
So, I'm going to name the path.
Let's call it p. It starts at vertex v_0 and goes to v_1, v_2, whatever, and ends at v_k.
And, this is not just any shortest path, but it's one that starts at s. So, v_0's s, and it ends at v. So, I'm going to give a couple of names to s and v so I can talk about the path more uniformly.
So, this is a shortest path from s to v. Now, I also want it to be not just any shortest path from s to v, but among all shortest paths from s to v I want it to be one with the fewest possible edges.
OK, so shortest here means in terms of the total weight of the path.
Subject to being shortest in weight, I wanted to also be shortest in the number of edges.
And, the reason I want that is to be able to conclude that p is a simple path, meaning that it doesn't repeat any vertices.
Now, can anyone tell me why I need to assume that the number of edges is the smallest possible in order to guarantee that p is simple?
The claim is that not all shortest paths are necessarily simple.
Yeah?
Right, I can have a zero weight cycle, exactly.
So, we are hoping, I mean, in fact in the theorem here, we're assuming that there are no negative weight cycles.
But there might be zero weight cycles still.
As a zero weight cycle, you can put that in the middle of any shortest path to make it arbitrarily long, repeat vertices over and over.
That's going to be annoying.
What I want is that p is simple.
And, I can guarantee that essentially by shortcutting.
If ever I take a zero weight cycle, I throw it away.
And this is one mathematical way of doing that.
OK, now what else do we know about this shortest path?
Well, we know that subpaths are shortest paths are shortest paths.
That's optimal substructure.
So, we know what the shortest path from s to v_i is sort of inductively.
It's the shortest path, I mean, it's the weight of that path, which is, in particular, the shortest path from s to v minus one plus the weight of the last edge, v minus one to v_i.
So, this is by optimal substructure as we proved last time.
OK, and I think that's pretty much the warm-up.
So, I want to sort of do this inductively in I, start out with v zero, and go up to v_k.
So, the first question is, what is d of v_0, which is s?
What is d of the source?
Well, certainly at the beginning of the algorithm, it's zero.
So, let's say equals zero initially because that's what we set it to.
And it only goes down from there.
So, it certainly, at most, zero.
The real question is, what is delta of s comma v_0.
What is the shortest path weight from s to s?
It has to be zero, otherwise you have a negative weight cycle, exactly.
My favorite answer, zero.
So, if we had another path from s to s, I mean, that is a cycle.
So, it's got to be zero.
So, these are actually equal at the beginning of the algorithm, which is great.
That means they will be for all time because we just argued up here, only goes down, never can get too small.
So, we have d of v_0 set to the right thing.
Great: good for the base case of the induction.
Of course, what we really care about is v_k, which is v. So, let's talk about the v_i inductively, and then we will get v_k as a result.
So, yeah, let's do it by induction.
That's more fun.
Let's say that d of v_i is equal to delta of s v_i after I rounds of the algorithm.
So, this is actually referring to the I that is in the algorithm here.
These are rounds.
So, one round is an entire execution of all the edges, relaxation of all the edges.
So, this is certainly true for I equals zero.
We just proved that.
After zero rounds, at the beginning of the algorithm, d of v_0 equals delta of s, v_0.
OK, so now, that's not really what I wanted, but OK, fine.
Now we'll prove it for d of v_i plus one.
Generally, I recommend you assume something.
In fact, why don't I follow my own advice and change it?
It's usually nicer to think of induction as recursion.
So, you assume that this is true, let's say, for j less than the i that you care about, and then you prove it for d of v_i.
It's usually a lot easier to think about it that way.
In particular, you can use strong induction for all less than i.
Here, we're only going to need it for one less.
We have some relation between I and I minus one here in terms of the deltas.
And so, we want to argue something about the d values.
OK, well, let's think about what's going on here.
We know that, let's say, after I minus one rounds, we have this inductive hypothesis, d of v_i minus one equals delta of s v_i minus one.
And, we want to conclude that after i rounds, so we have one more round to do this.
We want to conclude that d of v_i has the right answer, delta of s comma v_i.
Does that look familiar at all?
So we want to relax every edge in this round.
In particular, at some point, we have to relax the edge from v_i minus one to v_i.
We know that this path consists of edges.
That's the definition of a path.
So, during the i'th round, we relax every edge.
So, we better relax v_i minus one v_i.
And, what happens then?
It's a test of memory.
Quick, the Death Star is approaching.
So, if we have the correct value for v_i minus one, that we relax an outgoing edge from there, and that edge is an edge of the shortest path from s to v_i.
What do we know?
d of v_i becomes the correct value, delta of s comma v_i.
This was called correctness lemma last time.
One of the things we proved about Dijkstra's algorithm, but it was really just a fact about relaxation.
And it was a pretty simple proof.
And it comes from this fact.
We know the shortest path weight is this.
So, certainly d of v_i was at least this big, and let's suppose it's greater, or otherwise we were done.
We know d of v_i minus one is set to this.
And so, this is exactly the condition that's being checked in the relaxation step.
And, the d of v_i value will be greater than this, let's suppose.
And then, we'll set it equal to this.
And that's exactly d of s v_i.
So, when we relax that edge, we've got to set it to the right value.
So, this is the end of the proof, right?
It's very simple.
The point is, you look at your shortest path.
Here it is.
And if we assume there's no negative weight cycles, this has the correct value initially.
d of s is going to be zero.
After the first round, you've got to relax this edge.
And then you get the right value for that vertex.
After the second round, you've got to relax this edge, which gets you the right d value for this vertex and so on.
And so, no matter which shortest path you take, you can apply this analysis.
And you know that by, if the length of this path, here we assumed it was k edges, then after k rounds you've got to be done.
OK, so this was not actually the end of the proof.
Sorry.
So this means after k rounds, we have the right answer for v_k, which is v. So, the only question is how big could k be?
And, it better be the right answer, at most, v minus one is the claim by the algorithm that you only need to do v minus one steps.
And indeed, the number of edges in a simple path in a graph is, at most, the number of vertices minus one.
k is, at most, v minus one because p is simple.
So, that's why we had to assume that it wasn't just any shortest path.
It had to be a simple one so it didn't repeat any vertices.
So there are, at most, V vertices in the path, so at most, V minus one edges in the path.
OK, and that's all there is to Bellman-Ford.
So: pretty simple in correctness.
Of course, we're using a lot of the lemmas that we proved last time, which makes it easier.
OK, a consequence of this theorem, or of this proof is that if Bellman-Ford fails to converge, and that's what the algorithm is checking is whether this relaxation still requires work after these d minus one steps.
Right, the end of this algorithm is run another round, a V'th round, see whether anything changes.
So, we'll say that the algorithm fails to converge after V minus one steps or rounds.
Then, there has to be a negative weight cycle.
OK, this is just a contrapositive of what we proved.
We proved that if you assume there's no negative weight cycle, then we know that d of s is zero, and then all this argument says is you've got to converge after v minus one rounds.
There can't be anything left to do once you've reached the shortest path weights because you're going monotonically; you can never hit the bottom.
You can never go to the floor.
So, if you fail to converge somehow after V minus one rounds, you've got to have violated the assumption.
The only assumption we made was there's no negative weight cycle.
So, this tells us that Bellman-Ford is actually correct.
When it says that there is a negative weight cycle, it indeed means it.
It's true.
OK, and you can modify Bellman-Ford in that case to sort of run a little longer, and find where all the minus infinities are.
And that is, in some sense, one of the things you have to do in your problem set, I believe.
So, I won't cover it here.
But, it's a good exercise in any case to figure out how you would find where the minus infinities are.
What are all the vertices reachable from negative weight cycle?
Those are the ones that have minus infinities.
OK, so you might say, well, that was awfully fast.
Actually, it's not over yet.
The episode is not yet ended.
We're going to use Bellman-Ford to solve the even bigger and greater shortest path problems.
And in the remainder of today's lecture, we will see it applied to a more general problem, in some sense, called linear programming.
And the next lecture, we'll really use it to do some amazing stuff with all pairs shortest paths.
Let's go over here.
So, our goal, although it won't be obvious today, is to be able to compute the shortest paths between every pair of vertices, which we could certainly do at this point just by running Bellman-Ford v times.
OK, but we want to do better than that, of course.
And, that will be the climax of the trilogy.
OK, today we just discovered who Luke's father is.
So, it turns out the father of shortest paths is linear programming.
Actually, simultaneously the father and the mother because programs do not have gender.
OK, my father likes to say, we both took improv comedy lessons so we have degrees in improvisation.
And he said, you know, we went to improv classes in order to learn how to make our humor better.
And, the problem is, it didn't actually make our humor better.
It just made us less afraid to use it.
[LAUGHTER] So, you are subjected to all this improv humor.
I didn't see the connection of Luke's father, but there you go.
OK, so, linear programming is a very general problem, a very big tool.
Has anyone seen linear programming before?
OK, one person.
And, I'm sure you will, at some time in your life, do anything vaguely computing optimization related, linear programming comes up at some point.
It's a very useful tool.
You're given a matrix and two vectors: not too exciting yet.
What you want to do is find a vector.
This is a very dry description.
We'll see what makes it so interesting in a moment.
So, you want to maximize some objective, and you have some constraints.
And they're all linear.
So, the objective is a linear function in the variables x, and your constraints are a bunch of linear constraints, inequality constraints, that's one makes an interesting.
It's not just solving a linear system as you've seen in linear algebra, or whatever.
Or, of course, it could be that there is no such x
vaguely familiar you might think to the theorem about Bellman-Ford.
And, we'll show that there's some kind of connection here that either you want to find something, or show that it doesn't exist.
Well, that's still a pretty vague connection, but I also want to maximize something, or are sort of minimize the shortest paths, OK, somewhat similar.
We have these constraints.
So, yeah.
This may be intuitive to you, I don't know.
I prefer a more geometric picture, and I will try to draw such a geometric picture, and I've never tried to do this on a blackboard, so it should be interesting.
I think I'm going to fail miserably.
It sort of looks like a dodecahedron, right?
Sort of, kind of, not really.
A bit rough on the bottom, OK.
So, if you have a bunch of linear constraints, this is supposed to be in 3-D. Now I labeled it.
It's now in 3-D. Good.
So, you have these linear constraints.
That turns out to define hyperplanes in n dimensions.
OK, so you have this base here that's three-dimensional space.
So, n equals three.
And, these hyperplanes, if you're looking at one side of the hyperplane, that's the less than or equal to, if you take the intersection, you get some convex polytope or polyhedron.
In 3-D, you might get a dodecahedron or whatever.
And, your goal, you have some objective vector c, let's say, up.
Suppose that's the c vector.
Your goal is to find the highest point in this polytope.
So here, it's maybe this one.
OK, this is the target.
This is the optimal, x.
That is the geometric view.
If you prefer the algebraic view, you want to maximize the c transpose times x.
So, this is m. This is n. Check out the dimensions work out.
So that's saying you want to maximize the dot product.
You want to maximize the extent to which x is in the direction c. And, you want to maximize that subject to some constraints, which looks something like this, maybe.
So, this is A, and it's m by n. You want to multiply it by, it should be something of height n. That's x.
Let me put x down here, n by one.
And, it should be less than or equal to something of this height, which is B, the right hand side.
OK, that's the algebraic view, which is to check out all the dimensions are working out.
But, you can read these off in each row here, when multiplied by this column, gives you one value here.
And as just a linear constraints on all the x sides.
So, you want to maximize this linear function of x_1 up to x_n subject to these constraints, OK?
Pretty simple, but pretty powerful in general.
So, it turns out that with, you can formulate a huge number of problems such as shortest paths as a linear program.
So, it's a general tool.
And in this class, we will not cover any algorithms for solving linear programming.
It's a bit tricky.
I'll just mention that they are out there.
So, there's many efficient algorithms, and lots of code that does this.
It's a very practical setup.
So, lots of algorithms to solve LP's, linear programs.
Linear programming is usually called LP.
And, I'll mention a few of them.
There's the simplex algorithm.
This is one of the first.
I think it is the first, the ellipsoid algorithm.
There's interior point methods, and there's random sampling.
I'll just say a little bit about each of these because we're not going to talk about any of them in depth.
The simplex algorithm, this is, I mean, one of the first algorithms in the world in some sense, certainly one of the most popular.
It's still used today.
Almost all linear programming code uses the simplex algorithm.
It happens to run an exponential time in the worst-case, so it's actually pretty bad theoretically.
But in practice, it works really well.
And there is some recent work that tries to understand this.
It's still exponential in the worst case.
But, it's practical.
There's actually an open problem whether there exists a variation of simplex that runs in polynomial time.
But, I won't go into that.
That's a major open problem in this area of linear programming.
The ellipsoid algorithm was the first algorithm to solve linear programming in polynomial time.
So, for a long time, people didn't know.
Around this time, people started realizing polynomial time is a good thing.
That happened around the late 60s.
Polynomial time is good.
And, the ellipsoid algorithm is the first one to do it.
It's a very general algorithm, and very powerful, theoretically: completely impractical.
But, it's cool.
It lets you do things like you can solve a linear program that has exponentially many constraints in polynomial time.
You've got all sorts of crazy things.
So, I'll just say it's polynomial time.
I can't say something nice about it; don't say it at all.
It's impractical.
Interior point methods are sort of the mixture.
They run in polynomial time.
You can guarantee that.
And, they are also pretty practical, and there's sort of this competition these days about whether simplex or interior point is better.
And, I don't know what it is today but a few years ago they were neck and neck.
And, random sampling is a brand new approach.
This is just from a couple years ago by two MIT professors, Dimitris Bertsimas and Santosh Vempala, I guess the other is in applied math.
So, just to show you, there's active work in this area.
People are still finding new ways to solve linear programs.
This is completely randomized, and very simple, and very general.
It hasn't been implemented, so we don't know how practical it is yet.
But, it has potential
pretty neat.
OK, we're going to look at a somewhat simpler version of linear programming.
The first restriction we are going to make is actually not much of a restriction.
But, nonetheless we will consider it, it's a little bit easier to think about.
So here, we had some polytope we wanted to maximize some objective.
In a feasibility problem, I just want to know, is the polytope empty?
Can you find any point in that polytope?
Can you find any set of values, x, that satisfy these constraints?
OK, so there's no objective.
c, just find x such that AX is less than or equal to B. OK, it turns out you can prove a very general theorem that if you can solve linear feasibility, you can also solve linear programming.
We won't prove that here, but this is actually no easier than the original problem even though it feels easier, and it's easier to think about.
I was just saying actually no easier than LP.
OK, the next restriction we're going to make is a real restriction.
And it simplifies the problem quite a bit.
And that's to look at different constraints.
And, if all this seemed a bit abstract so far, we will now ground ourselves little bit.
A system of different constraints is a linear feasibility problem.
So, it's an LP where there's no objective.
And, it's with a restriction, so, where each row of the matrix, so, the matrix, A, has one one, and it has one minus one, and everything else in the row is zero.
OK, in other words, each constraint has its very simple form.
It involves two variables and some number.
So, we have something like x_j minus x_i is less than or equal to w_ij.
So, this is just a number.
These are two variables.
There's a minus sign, no values up here, no coefficients, no other of the X_k's appear, just two of them.
And, you have a bunch of constraints of this form, one per row of the matrix.
Geometrically, I haven't thought about what this means.
I think it means the hyperplanes are pretty simple.
Sorry I can't do better than that.
It's a little hard to see this in high dimensions.
But, it will start to correspond to something we've seen, namely the board that its next to, very shortly.
OK, so let's do a very quick example mainly to have something to point at.
Here's a very simple system of difference constraints -- -- OK, and a solution.
Why not?
It's not totally trivial to solve this, but here's a solution.
And the only thing to check is that each of these constraints is satisfied.
x_1 minus x_2 is three, which is less than or equal to three, and so on.
There could be negative values.
There could be positive values.
It doesn't matter.
I'd like to transform this system of difference constraints into a graph because we know a lot about graphs.
So, we're going to call this the constraint graph.
And, it's going to represent these constraints.
How'd I do it?
Well, I take every constraint, which in general looks like this, and I convert it into an edge.
OK, so if I write it as x_j minus x_i is less than or equal to some w_ij, w seems suggestive of weights.
That's exactly why I called it w. I'm going to make that an edge from v_i to v_j.
So, the order flips a little bit.
And, the weight of that edge is w_ij.
So, just do that.
Make n vertices.
So, you have the number of vertices equals n. The number of edges equals the number of constraints, which is m, the height of the matrix, and just transform.
So, for example, here we have three variables.
So, we have three vertices, v_1, v_2, v_3.
We have x_1 minus x_2.
So, we have an edge from v_2 to v_1 of weight three.
We have x_2 minus x_3.
So, we have an edge from v_3 to v_2 of weight minus two.
And, we have x_1 minus x_3.
So, we have an edge from v_3 to v_1 of weight two.
I hope I got the directions right.
Yep.
So, there it is, a graph: currently no obvious connection to shortest paths, right?
But in fact, this constraint is closely related to shortest paths.
So let me just rewrite it.
You could say, well, an x_j is less than or equal to x_i plus w_ij.
Or, you could think of it as d[j] less than or equal to d[i] plus w_ij.
This is a conceptual balloon.
Look awfully familiar?
A lot like the triangle inequality, a lot like relaxation.
So, there's a very close connection between these two problems as we will now prove.
So, we're going to have two theorems.
And, they're going to look similar to the correctness of Bellman-Ford in that they talk about negative weight cycles.
Here we go.
It turns out, I mean, we have this constraint graph.
It can have negative weights.
It can have positive weights.
It turns out what matters is if you have a negative weight cycle.
So, the first thing to prove is that if you have a negative weight cycle that something bad happens.
OK, what could happen bad?
Well, we're just trying to satisfy this system of constraints.
So, the bad thing is that there might not be any solution.
These constraints may be infeasible.
And that's the claim.
The claim is that this is actually an if and only if.
But first we'll proved the if.
If you have a negative weight cycle, you're doomed.
The difference constraints are unsatisfiable.
That's a more intuitive way to say it.
In the LP world, they call it infeasible.
But unsatisfiable makes a lot more sense.
There's no way to assign the x_i's in order to satisfy all the constraints simultaneously.
So, let's just take a look.
Consider a negative weight cycle.
It starts at some vertex, goes through some vertices, and at some point comes back.
I don't care whether it repeats vertices, just as long as this cycle, from v_1 to v_1 is a negative weight cycle strictly negative weight.
OK, and what I'm going to do is just write down all the constraints.
Each of these edges corresponds to a constraint, which must be in the set of constraints because we had that graph.
So, these are all edges.
Let's look at what they give us.
So, we have an edge from v_1 to v_2.
That corresponds to x_2 minus x_1 is, at most, something, w_12.
Then we have x_3 minus x_2.
That's the weight w_23, and so on.
And eventually we get up to something like x_k minus x_(k-1).
That's this edge: w_(k-1),k , and lastly we have this edge, which wraps around.
So, it's x_1 minus x_k, w_k1 if I've got the signs right.
Good, so here's a bunch of constraints.
What do you suggest I do with them?
Anything interesting about these constraints, say, the left hand sides?
Sorry?
It sounded like the right word.
What was it?
Telescopes, yes, good.
Everything cancels.
If I added these up, there's an x_2 and a minus x_2.
There's a minus x_1 and an x_1.
There's a minus XK and an XK.
Everything here cancels if I add up the left hand sides.
So, what happens if I add up the right hand sides?
Over here I get zero, my favorite answer.
And over here, we get all the weights of all the edges in the negative weight cycle, which is the weight of the cycle, which is negative.
So, zero is strictly less than zero: contradiction.
Contradiction: wait a minute, we didn't assume anything that was false.
So, it's not really a contradiction in the mathematical sense.
We didn't contradict the world.
We just said that these constraints are contradictory.
In other words, if you pick any values of the x_i's, there is no way that these can all be true because that you would get a contradiction.
So, it's impossible for these things to be satisfied by some real x_i's.
So, these must be unsatisfiable.
Let's say there's no satisfying assignment, a little more precise, x_1 up to x_m, no weights.
Can we satisfy those constraints?
Because they add up to zero on the left-hand side, and negative on the right-hand side.
OK, so that's an easy proof.
The reverse direction will be only slightly harder.
OK, so, cool.
We have this connection.
So motivation is, suppose you'd want to solve these difference constraints.
And we'll see one such application.
I Googled around for difference constraints.
There is a fair number of papers that care about difference constraints.
And, they all use shortest paths to solve them.
So, if we can prove a connection between shortest paths, which we know how to compute, and difference constraints, then we'll have something cool.
And, next class will see even more applications of difference constraints.
It turns out they're really useful for all pairs shortest paths.
OK, but for now let's just prove this equivalence and finish it off.
So, the reverse direction is if there's no negative weight cycle in this constraint graph, then the system better be satisfiable.
The claim is that these negative weight cycles are the only barriers for finding a solution to these difference constraints.
I have this feeling somewhere here.
I had to talk about the constraint graph.
Good.
Satisfied, good.
So, here we're going to see a technique that is very useful when thinking about shortest paths.
And, it's a bit hard to guess, especially if you haven't seen it before.
This is useful in problem sets, and in quizzes, and finals, and everything.
So, keep this in mind.
I mean, I'm using it to prove this rather simple theorem, but the idea of changing the graph, so I'm going to call this constraint graph G. Changing the graph is a very powerful idea.
So, we're going to add a new vertex, s, or source, use the source, Luke, and we're going to add a bunch of edges from s because being a source, it better be connected to some things.
So, we are going to add a zero weight edge, or weight zero edge from s to everywhere, so, to every other vertex in the constraint graph.
Those vertices are called v_i, v_1 up to v_n.
So, I have my constraint graph.
But I'll copy this one so I can change it.
It's always good to backup your work before you make changes, right?
So now, I want to add a new vertex, s, over here, my new source.
I just take my constraint graph, whatever it looks like, add in weight zero edges to all the other vertices.
Simple enough.
Now, what did I do?
What did you do?
Well, I have a candidate source now which can reach all the vertices.
So, shortest path from s, hopefully, well, paths from s exist.
I can get from s to everywhere in weight at most zero.
OK, maybe less.
Could it be less?
Well, you know, like v_2, I can get to it by zero minus two.
So, that's less than zero.
So I've got to be a little careful.
What if there's a negative weight cycle?
Oh no?
Then there wouldn't be any shortest paths.
Fortunately, we assume that there's no negative weight cycle in the original graph.
And if you think about it, if there's no negative weight cycle in the original graph, we add an edge from s to everywhere else.
We're not making any new negative weight cycles because you can start at s and go somewhere at a cost of zero, which doesn't affect any weights.
And then, you are forced to stay in the old graph.
So, there can't be any new negative weight cycles.
So, the modified graph has no negative weight cycles.
That's good because it also has paths from s, and therefore it also has shortest paths from s. The modified graph has no negative weight because it didn't before.
And, it has paths from s. There's a path from s to every vertex.
There may not have been before.
Before, I couldn't get from v_2 to v_3, for example.
Well, that's still true.
But from s I can get to everywhere.
So, that means that this graph, this modified graph, has shortest paths.
Shortest paths exist from s. In other words, if I took all the shortest path weights, like I ran Bellman-Ford from s, then, I would get a bunch of finite numbers, d of v, for every value, for every vertex.
That seems like a good idea.
Let's do it.
So, shortest paths exist.
Let's just assign x_i to be the shortest path weight from s to v_i.
Why not?
That's a good choice for a number, the shortest path weight from s to v_i.
This is finite because it's less than infinity, and it's greater than minus infinity, so, some finite number.
That's what we need to do in order to satisfy these constraints.
The claim is that this is a satisfying assignment.
Why?
Triangle inequality.
Somewhere here we wrote triangle inequality.
This looks a lot like the triangle inequality.
In fact, I think that's the end of the proof.
Let's see here.
What we want to be true with this assignment is that x_j minus x_i is less than or equal to w_ij whenever ij is an edge.
Or, let's say v_i, v_j, for every such constraint, so, for v_i, v_j in the edge set.
OK, so what is this true?
Well, let's just expand it out.
So, x_i is this delta, and x_j is some other delta.
So, we have delta of s, vj minus delta of s_vi.
And, on the right-hand side, well, w_ij, that was the weight of the edge from I to J.
So, this is the weight of v_i to v_j.
OK, I will rewrite this slightly.
Delta s, vj is less than or equal to delta s, vi plus w of v_i, v_j.
And that's the triangle inequality more or less.
The shortest path from s to v_j is, at most, shortest path from s to v_i plus a particular path from v_i to v_j, namely the single edge v_i to v_j.
This could only be longer than the shortest path.
And so, that makes the right-hand side bigger, which makes this inequality more true, meaning it was true before.
And now it's still true.
And, that proves it.
This is true.
And, these were all equivalent statements.
This we know to be true by triangle inequality.
Therefore, these constraints are all satisfied.
Magic.
I'm so excited here.
So, we've proved that having a negative weight cycle is exactly when these system of difference constraints are unsatisfiable.
So, if we want to satisfy them, if we want to find the right answer to x, we run Bellman-Ford.
Either it says, oh, no negative weight cycle.
Then you are hosed.
Then, there is no solution.
But that's the best you could hope to know.
Otherwise, it says, oh, there was no negative weight cycle, and here are your shortest path weights.
You just plug them in, and bam, you have your x_i's that satisfy the constraints.
Awesome.
Now, it wasn't just any graph.
I mean, we started with constraints, algebra, we converted it into a graph by this transform.
Then we added a source vertex, s. So, I mean, we had to build a graph to solve our problem, very powerful idea.
Cool.
This is the idea of reduction.
You can reduce the problem you want to solve into some problem you know how to solve.
You know how to solve shortest paths when there are no negative weight cycles, or find out that there is a negative weight cycle by Bellman-Ford.
So, now we know how to solve difference constraints.
It turns out you can do even more.
Bellman-Ford does a little bit more than just solve these constraints.
But first let me write down what I've been jumping up and down about.
The corollary is you can use Bellman-Ford.
I mean, you make this graph.
Then you apply Bellman-Ford, and it will solve your system of difference constraints.
So, let me put in some numbers here.
You have m difference constraints.
And, you have n variables.
And, it will solve them in order m times n time.
Actually, these numbers go up slightly because we are adding n edges, and we're adding one vertex, but assuming all of these numbers are nontrivial, m is at least n. It's order MN time.
OK, trying to avoid cases where some of them are close to zero.
Good.
So, some other facts, that's what I just said.
And we'll leave these as exercises because they're not too essential.
The main thing we need is this.
But, some other cool facts is that Bellman-Ford actually optimizes some objective functions.
So, we are saying it's just a feasibility problem.
We just want to know whether these constraints are satisfiable.
In fact, you can add a particular objective function.
So, you can't give it an arbitrary objective function, but here's one of interest.
x_1 plus x_2 plus x_n, OK, but not just that.
We have some constraints.
OK, this is a linear program.
I want to maximize the sum of the x_i's subject to all the x_i's being nonpositive and the difference constraints.
So, this we had before.
This is fine.
We noticed at some point you could get from s to everywhere with cost, at most, zero.
So, we know that in this assignment all of the x_i's are negative.
That's not necessary, but it's true when you run Bellman-Ford.
So if you solve your system using Bellman-Ford, which is no less general than anything else, you happen to get nonpositive x_i's.
And so, subject to that constraint, it actually makes them is close to zero as possible in the L1 norm.
In the sum of these values, it tries to make the sum as close to zero, it tries to make the values as small as possible in absolute value in this sense.
OK, it does more than that.
It cooks, it cleans, it finds shortest paths.
It also minimizes the spread, the maximum over all i of x_i minus the minimum over all i of x_i.
So, I mean, if you have your real line, and here are the x_i's wherever they are.
It minimizes this distance.
And zero is somewhere over here.
So, it tries to make the x_i's as compact as possible.
This is actually the L infinity norm, if you know stuff about norms from your linear algebra class.
OK, this is the L1 norm.
I think it minimizes every LP norm.
Good, so let's use this for something.
Yeah, let's solve a real problem, and then we'll be done for today.
Next class we'll see the really cool stuff, the really cool application of all of this.
For now, and we'll see a cool but relatively simple application, which is VLSI layout.
We talked a little bit about VLSI way back and divide and conquer.
You have a bunch of chips, or you want to arrange them, and minimize some objectives.
So, here's a particular, tons of problems that come out of VLSI layout.
Here's one of them.
You have a bunch of features of an integrated circuit.
You want to somehow arrange them on your circuit without putting any two of them too close to each other.
You have some minimum separation like at least they should not get top of each other.
Probably, you also need some separation to put wires in between, and so on, so, without putting any two features too close together.
OK, so just to give you an idea, so I have some objects and I'm going to be a little bit vague about how this works.
You have some features.
This is stuff, some chips, whatever.
We don't really care what their shapes look like.
I just want to be able to move them around so that the gap at any point, so let me just think about this gap.
This gap should be at least some delta.
Or, I don't want to use delta.
Let's say epsilon, good, small number.
So, I just need some separation between all of my parts.
And for this problem, I'm going to be pretty simple, just say that the parts are only allowed to slide horizontally.
So, it's a one-dimensional problem.
These objects are in 2-d, or whatever, but I can only slide them an x coordinate.
So, to model that, I'm going to look at the left edge of every part and say, well, these two left edges should be at least some separation.
So, I think of it as whatever the distance is plus some epsilon.
But, you know, if you have some funky 2-d shapes you have to compute, well, this is a little bit too close because these come into alignment.
But, there's some constraint, well, for any two pieces, I could figure out how close they can get.
They should get no closer.
So, I'm going to call this x_1.
I'll call this x_2.
So, we have some constraint like x_2 minus x_1 is at least d plus epsilon, or whatever you compute that weight to be.
OK, so for every pair of pieces, I can do this, compute some constraint on how far apart they have to be.
And, now I'd like to assign these x coordinates.
Right now, I'm assuming they're just variables.
I want to slide these pieces around horizontally in order to compactify them as much as possible so they fit in the smallest chip that I can make because it costs money, and time, and everything, and power, everything.
You always want your chip small.
So, Bellman-Ford does that.
All right, so Bellman-Ford solves these constraints because it's just a bunch of difference constraints.
And we know that they are solvable because you could spread all the pieces out arbitrarily far.
And, it minimizes the spread, minimizes the size of the chip I need, a max of x_i minus the min of x_i.
So, this is it maximizes compactness, or minimizes size of the chip.
OK, this is a one-dimensional problem, so it may seem a little artificial, but the two dimensional problem is really hard to solve.
And this is, in fact, the best you can do with a nice polynomial time algorithm.
There are other applications if you're scheduling events in, like, a multimedia environment, and you want to guarantee that this audio plays at least two seconds after this video, but then there are things that are playing at the same time, and they have to be within some gap of each other, so, lots of papers about using Bellman-Ford, solve difference constraints to enable multimedia environments.
OK, so there you go.
And next class we'll see more applications of Bellman-Ford to all pairs shortest paths.
Questions?
Great.
-- shortest paths.
This is the finale.
Hopefully it was worth waiting for.
Remind you there's a quiz coming up soon, you should be studying for it.
There's no problem set due at the same time as the quiz because you should be studying now.
It's a take-home exam.
It's required that you come to class on Monday.
Of course, you'll all come, but everyone watching at home should also come next Monday to get the quiz.
It's the required lecture.
So, we need a bit of a recap in the trilogy so far.
So, the last two lectures, the last two episodes, or about single source shortest paths.
So, we wanted to find the shortest path from a source vertex to every other vertex.
And, we saw a few algorithms for this.
Here's some recap.
We saw in the unweighted case, that was sort of the easiest where all the edge weights were one.
Then we could use breadth first search.
And this costs what we call linear time in the graph world, the number of vertices plus the number of edges.
The next simplest case, perhaps, is nonnegative edge weights.
And in that case, what algorithm do we use?
Dijkstra, all right, everyone's awake.
Several answers at once, great.
So this takes almost linear time if you use a good heap structure, so, V log V plus E. And, in the general case, general weights, we would use Bellman-Ford which you saw.
And that costs VE, good, OK, which is quite a bit worse.
This is ignoring log factors.
Dijkstra is basically linear time, Bellman-Ford you're quadratic if you have a connected graph.
So, in the sparse case, when E is order V, this is about linear.
This is about quadratic.
In the dense case, when E is about V^2, this is quadratic, and this is cubic.
So, Dijkstra and Bellman-Ford are separated by about an order of V factor, which is pretty bad.
OK, but that's the best we know how to do for single source shortest paths, negative edge weights, Bellman-Ford is the best.
We also saw in recitation the case of a DAG.
And there, what do you do?
Topological sort, yeah.
So, you can do a topological sort to get an ordering on the vertices.
That you run Bellman-Ford, one round.
This is one way to think of what's going on.
You run Bellman-Ford in the order given by the topological sort, which is once, and you get a linear time algorithm.
So, DAG is another case where we know how to do well even with weights.
Unweighted, we can also do linear time.
But most of the time, though, will be, so you should keep these in mind in the quiz.
When you get a shortest path problem, or what you end up determining is the shortest path problem, think about what's the best algorithm you can use in that case?
OK, so that's single source shortest paths.
And so, in our evolution of the Death Star, initially it was just nonnegative edge weights.
Then we got negative edge weights.
Today, the Death Star challenges us with all pair shortest paths, where we want to know the shortest path weight between every pair of vertices.
OK, so let's get some quick results.
What could we do with this case?
So, for example, suppose I have an unweighted graph.
Any suggestions of how I should compute all pair shortest paths?
Between every pair of vertices, I want to know the shortest path weight.
BFS, a couple more words?
Yeah?
Right, BFS V times.
OK, I'll say V times BFS, OK?
So, the running time would be V^2 plus V times E, yeah, which is assuming your graph is connected, V times E. OK, good.
That's probably about the best algorithm we know for unweighted graphs.
So, a lot of these are going to sort of be the obvious answer.
You take your single source algorithm, you run it V times.
That's the best you can do, OK, or the best we know how to do.
This is not so bad.
This is like one iteration of Bellman-Ford, for comparison.
We definitely need at least, like, V^2 time, because the size of the output is V^2, shortest path weight we have to compute.
So, this is not perfect, but pretty good.
And we are not going to improve on that.
So, nonnegative edge weights: the natural thing to do is to run Dijkstra V times, OK, no big surprise.
And the running time of that is, well, V times E again, plus V^2, log V, which is also not too bad.
I mean, it's basically the same as running BFS.
And then, there's the log factor.
If you ignore the log factor, this is the dominant term.
And, I mean, this had an [added?]
V^2 as well.
So, these are both pretty good.
I mean, this is kind of neat.
Essentially, the time it takes to run one Bellman-Ford plus a log factor, you can compute all pair shortest paths if you have nonnegative edge weights.
So, I mean, comparing all pairs to signal source, this seems a lot better, except we can only handle nonnegative edge weights.
OK, so now let's think about the general case.
Well, this is the focus of today, and here's where we can actually make an improvement.
So the obvious thing is V times Bellman-Ford, which would cost V^2 times E. And that's pretty pitiful, and we're going to try to improve that to something closer to that nonnegative edge weight bound.
So it turns out, here, we can actually make an improvement whereas in these special cases, we really can't do much better.
OK, I don't have a good intuition why, but it's the case.
So, we'll cover something like three algorithms today for this problem.
The last one will be the best, but along the way we'll see some nice connections between shortest paths and dynamic programming, which we haven't really seen yet.
We've seen shortest path, and applying greedy algorithms to it, but today will actually do dynamic programming.
The intuition is that with all pair shortest paths, there's more potential subproblem reuse.
We've got to compute the shortest path from x to y for all x and y.
Maybe we can reuse those shortest paths in computing other shortest paths.
OK, there's a bit more reusability, let's say.
OK, let me quickly define all pair shortest paths formally, because we're going to change our notation slightly.
It's because we care about all pairs.
So, as usual, the input is directed graph, so, vertices and edges.
We're going to say that the vertices are labeled one to n for convenience because with all pairs, we're going to think of things more as an n by n matrix instead of edges in some sense because it doesn't help to think any more in terms of adjacency lists.
And, you have edge weights as usual.
This is what makes it interesting.
Some of them are going to be negative.
So, w maps to every real number, and the target output is a shortest path matrix.
So, this is now an n by n matrix.
So, n is just the number of vertices of shortest path weights.
So, delta of i, j is the shortest path weight from i to j for all pairs of vertices.
So this, you could represent as an n by n matrix in particular.
OK, so now let's start doing algorithms.
So, we have this very simple algorithm, V times Bellman-Ford, V^2 times E, and just for comparison's sake, I'm going to say, let me rewrite that, V times Bellman-Ford gives us this running time of V^2 E, and I'm going to think about the case where, let's just say the graph is dense, meeting that the number of edges is quadratic, and the number of vertices.
So in that case, this will take V^4 time, which is pretty slow.
We'd like to do better.
So, first goal would just be to beat V^4, V hypercubed, I guess.
OK, and we are going to use dynamic programming to do that.
Or at least that's what the motivation will come from.
It will take us a while before we can even beat V^4, which is maybe a bit pathetic, but it takes some clever insights, let's say.
OK, so I'm going to introduce a bit more notation for this graph.
So, I'm going to think about the weighted adjacency matrix.
So, I don't think we've really seen this in lecture before, although I think it's in the appendix.
What that means, so normally adjacency matrix is like one if there's an edge, and zero if there isn't.
And this is in a digraph, so you have to be a little bit careful.
Here, these values, the entries in the matrix, are going to be the weights of the edges.
OK, this is this if ij is an edge.
So, if ij is an edge in the graph, and it's going to be infinity if there is no edge.
OK, in terms of shortest paths, this is a more useful way to represent the graph.
All right, and so this includes everything that we need from here.
And now we just have to think about it as a matrix.
Matrices will be a useful tool in a little while.
OK, so now I'm going to define some sub problems.
And, there's different ways that you could define what's going on in the shortest paths problem.
OK, the natural thing is I want to go from vertex i to vertex j.
What's the shortest path?
OK, we need to refine the sub problems a little but more than that.
Not surprising.
And if you think about my analogy to Bellman-Ford, what Bellman-Ford does is it tries to build longer and longer shortest paths.
But here, length is in terms of the number of edges.
So, first, it builds shortest paths of length one.
We've proven the first round it does that.
The second round, it provides all shortest paths of length two, of count two, and so on.
We'd like to do that sort of analogously, and try to reuse things a little bit more.
So, I'm going to say d_ij^(m) is the weight of the shortest path from i to j with some restriction involving m. So: shortest path from i to j using at most m edges.
OK, for example, if m is zero, then we don't have to really think very hard to find all shortest paths of length zero.
OK, they use zero edges, I should say.
So, Bellman-Ford sort of tells us how to go from m to m plus one.
So, let's just figure that out.
So one thing we know from the Bellman-Ford analysis is if we look at d_ij^(m-1), we know that in some sense the longest shortest path of relevance, unless you have negative weight cycle, the longest shortest path of relevance is when m equals n minus one because that's the longest simple path you can have.
So, this should be a shortest path weight from i to j, and it would be no matter what larger value you put in the superscript.
This should be delta of i comma j if there's no negative weight cycles.
OK, so this feels good for dynamic programming.
This will give us the answer if we can compute this for all m. Then we'll have the shortest path weights in particular.
We need a way to detect negative weight cycles, but let's not worry about that too much for now.
There are negative weights, but let's just assume for now there's no negative weight cycles.
OK, and we get a recursion recurrence.
And the base case is when m equals zero.
This is pretty easy.
They have the same vertices, the weight of zero, and otherwise it's infinity.
OK, and then the actual recursion is for m. OK, if I got this right, this is a pretty easy, intuitive recursion for d_ij^(m) is a min of smaller things in terms of n minus one.
I'll just show the picture, and then the proof of that claim should be obvious.
So, this is proof by picture.
So, we have on the one hand, I over here, and j over here.
We want to know the shortest path from i to j.
And, we want to use, at most, m edges.
So, the idea is, well, you could use m minus one edges to get somewhere.
So this is, at most, m minus one edges, some other place, and we'll call it k. So this is a candidate for k. And then you could take the edge directly from k to j.
So, this costs A_k^j, and this costs DIK m minus one.
OK, and that's a candidate path of length that uses, at most, m edges from I to j.
And this is essentially just considering all of them.
OK, so there's sort of many paths we are considering.
All of these are candidate values of k. We are taking them in over all k as intermediate nodes, whatever.
So there they are.
We take the best such path.
That should encompass all shortest paths.
And this is essentially sort of what Bellman-Ford is doing, although not exactly.
We also sort of want to think about, well, what if I just go directly with, say, m minus one edges?
What if there is no edge here that I want to use, in some sense?
Well, we always think about there being, and the way the A's are defined, there's always this zero weight edge to yourself.
So, you could just take a path that's shorter, go from d i to j, and j is a particular value of k that we might consider, and then take a zero weight edge at the end from A and jj.
OK, so this really encompasses everything.
So that's a pretty trivial claim.
OK, now once we have such a recursion, we get a dynamic program.
I mean, there, this is it in some sense.
It's written recursively.
You can write a bottom up.
And I would like to write it bottom up it little bit because while it doesn't look like it, this is a relaxation.
This is yet another relaxation algorithm.
So, I'll give you, so, this is sort of the algorithm.
This is not a very interesting algorithm.
So, you don't have to write it all down if you don't feel like it.
It's probably not even in the book.
This is just an intermediate step.
So, we loop over all m. That's sort of the outermost thing to do.
I want to build longer and longer paths, and this vaguely corresponds to Bellman-Ford, although it's actually worse than Bellman-Ford.
But hey, what the heck?
It's a stepping stone.
OK, then for all i and j, and then we want to compute this min.
So, we'll just loop over all k, and relax.
And, here's where we're actually computing the min.
And, it's a relaxation, is the point.
This is our good friend, the relaxation step, relaxing edge.
Well, it's not, yeah.
I guess we're relaxing edge kj, or something, except we don't have the same clear notion.
I mean, it's a particular thing that we're relaxing.
It's not just a single edge because we don't have a single source anymore.
It's now relative to source I, we are relaxing the edge kj, something like that.
But this is clearly a relaxation.
We are just making the triangle inequality true if it wasn't before.
The tribal inequality has got to hold between all pairs.
And that's just implementing this min, right?
You're taking d ij.
You take the min of what it was before in some sense.
That was one of the possibilities we considered when we looked at the zero weight edge.
We say, well, or you could go from i to some k in some way that we knew how to before, and then add on the edge, and check whether that's better if it's better, set our current estimate to that.
And, you do this for all k. In particular, you might actually compute something smaller than this min because I didn't put superscripts up here.
But that's just making paths even better.
OK, so you have to argue that relaxation is always a good thing to do.
So, by not putting superscripts, maybe I do some more relaxation, but more relaxation never hurts us.
You can still argue correctness using this claim.
So, it's not quite the direct implementation, but there you go, dynamic programming algorithm.
The main reason I'll write it down: so you see that it's a relaxation, and you see the running time is n^4, OK, which is certainly no better than Bellman-Ford.
Bellman-Ford was n^4 even in the dense case, and it's a little better in the sparse case.
So: not doing so great.
But it's a start.
OK, it gets our dynamic programming minds thinking.
And, we'll get a better dynamic program in a moment.
But first, there's actually something useful we can do with this formulation, and I guess I'll ask, but I'll be really impressed if anyone can see.
Does this formula look like anything else that you've seen in any context, mathematical or algorithmic?
Have you seen that recurrence anywhere else?
OK, not exactly as stated, but similar.
I'm sure if you thought about it for awhile, you could come up with it.
Any answers?
I didn't think you would be very intuitive, but the answer is matrix multiplication.
And it may now be obvious to you, or it may not.
You have to think with the right quirky mind.
Then it's obvious that it's matrix multiplication.
Remember, matrix multiplication, we have A, B, and C. They're all n by n matrices.
And, we want to compute C equals A times B.
And what that meant was, well, c_ij was a sum over all k of a_ik times b_kj.
All right, that was our definition of matrix multiplication.
And that formula looks kind of like this one.
I mean, notice the subscripts: ik and kj.
Now, the operators are a little different.
Here, we're multiplying the inside things and adding them all together.
There, we're adding the inside things and taking them in.
But other than that, it's the same.
OK, weird, but here we go.
So, the connection to shortest paths is you replace these operators.
So, let's take matrix multiplication and replace, what should I do first, plus this thing with min.
So, why not just change the operators, replace dot with plus?
This is just a different algebra to work in, where plus actually means min, and dot actually means plus.
So, you have to check that things sort of work out in that context, but if we do that, then we get that c_ij is the min overall k of a_ik plus, a bit messy here, b_kj.
And that looks like what we actually want to compute, here, for one value of m, you have to sort of do this m times.
But this conceptually is d_ij^(m), and this is d_ik^(m-1).
So, this is looking like a matrix product, which is kind of cool.
So, if we sort of plug in this claim, then, and think about things as matrices, the recurrence gives us, and I'll just write this now at matrix form, that d^(m) is d^(m) minus one, funny product, A.
All right, so these are the weights.
These were the weighted adjacency matrix.
This was the previous d value.
This is the new d value.
So, I'll just rewrite that in matrix form with capital letters.
OK, I have the circle up things that are using this funny algebra, so, in particular, circled product.
OK, so that's kind of nifty.
We know something about computing matrix multiplications.
We can do it in n^3 time.
If we were a bit fancier, maybe we could do it in sub-cubic time.
So, we could try to sort of use this connection.
And, well, think about what we are computing here.
We are saying, well, d to the m is the previous one times A.
So, what is d^(m)?
Is that some other algebraic notion that we know?
Yeah, it's the exponent.
We're taking A, and we want to raise it to the power, m, with this funny notion of product.
So, in other words, d to the m is really just A to the m in a funny way.
So, I'll circle it, OK?
So, that sounds good.
We also know how to compute powers of things relatively quickly, if you remember how.
OK, for this notion, this power notion, to make sense, I should say what A to the zero means.
And so, I need some kind of identity matrix.
And for here, the identity matrix is this one, if I get it right.
So, it has zeros along the diagonal, and infinities everywhere else.
OK, that sort of just to match this definition.
d_ij zero should be zeros on the diagonals and infinity everywhere else.
But you can check this is actually an identity.
If you multiply it with this funny multiplication against any other matrix, you get the matrix back.
Nothing changes.
This really is a valid identity matrix.
And, I should mention that for A to the m to make sense, you really knew that your product operation is associative.
So, actually A to the m circled makes sense because circled multiplication is associative, and you can check that; not hard because, I mean, min is associative, and addition is associative, and all sorts of good stuff.
And, you have some kind of distributivity property.
And, this is, in turn, because the real numbers with, and get the right order here, with min as your addition operation, and plus as your multiplication operation is a closed semi-ring.
So, if ever you want to know when powers make sense, this is a good rule.
If you have a closed semi-ring, then matrix products on that semi-ring will give you an associative operator, and then, good, you can take products.
OK, that's just some formalism.
So now, we have some intuition.
The question is, what's the right.
Algorithm?
There are many possible answers, some of which are right, some of which are not.
So, we have this connection to matrix products, and we have a connection to matrix powers.
And, we have algorithms for both.
The question is, what should we do?
So, all we need to do now is to compute A to the funny power, n minus one.
n minus one is when we get shortest paths, assuming we have no negative weight cycles.
In fact, we could compute a larger power than n minus one.
Once you get beyond n minus one, multipling by A doesn't change you anymore.
So, how should we do it?
OK, you're not giving any smart answers.
I'll give the stupid answer.
You could say, well, I take A. I multiply it by A.
Then I multiply it by A, and I multiply it by A, and I use normal, boring matrix to multiplication.
So, I do, like, n minus two, standard matrix multiplies.
So, standard multiply costs, like, n^3.
And I'm doing n of them.
So, this gives me an n^4 algorithm, and compute all the shortest pathways in n^4.
Woohoo!
OK, no improvement.
So, how can I do better?
Right, natural thing to try which sadly does not work, is to use the sub cubic matrix multiply algorithm.
We will, in some sense, get there in a moment with a somewhat simpler problem.
But, it's actually not known how to compute shortest paths using fast matrix multiplication like Strassen's system algorithm.
But, good suggestion.
OK, you have to think about why it doesn't work, and I'll tell you.
It's not obvious, so it's a perfectly reasonable suggestion.
But in this context it doesn't quite work.
It will come up in a few moments.
The problem is, Strassen requires the notion of subtraction.
And here, addition is min.
And, there's no inverse to min.
Once you take the arguments, you can't sort of undo a min.
OK, so there's no notion of subtraction, so it's not known how to pull that off, sadly.
So, what other tricks do we have up our sleeve?
Yeah?
Divide and conquer, log n powering, yeah, repeated squaring.
That works.
Good, we had a fancy way.
If you had a number n, you sort of looked at the binary number representation of n, and you either squared the number or squared it and added another factor of A.
Here, we don't even have to be smart about it.
OK, we can just compute, we really only have to think about powers of two.
What we want to know, and I'm going to need a bigger font here because there's multiple levels of subscripts, A to the circled power, two to the ceiling of log n. Actually, n minus one would be enough.
But there you go.
You can write n if you didn't leave yourself enough space like me, n the ceiling, n the circle.
This just means the next power of two after n minus one, two to the ceiling log.
So, we don't have to go directly to n minus one.
We can go further because anything farther than n minus one is still just the shortest pathways.
If you look at the definition, and you know that your paths are simple, which is true if you have no negative weight cycles, then fine, just go farther.
Why not?
And so, to compute this, we just do ceiling of log n minus one products, just take A squared, and then take the result and square it; take the result and square it.
So, this is order log n squares.
And, we don't know how to use Strassen, but we can use the boring, standard multiply of n^3, and that gives us n^3 log n running time, OK, which finally is something that beats Bellman-Ford in the dense case.
OK, in the dense case, Bellman-Ford was n^4.
Here we get n^3 log n, finally something better.
In the sparse case, it's about the same, maybe a little worse.
E is order V. Then we're going to get, like, V3 for Bellman-Ford.
Here, we get n^3 log n. OK, after log factors, this is an improvement some of the time.
OK, it's about the same the other times.
Another nifty thing that you get for free out of this, is you can detect negative weight cycles.
So, here's a bit of a puzzle.
How would I detect, after I compute this product, A to the power to ceiling log n minus one, how would I know if I found a negative weight cycle?
What would that mean it this matrix of all their shortest paths of, at most, a certain length?
If I found a cycle, what would have to be in that matrix?
Yeah?
Right, so I could, for example, take this thing, multiply it by A, see if the matrix changed at all.
Right, that works fine.
That's what we do in Bellman-Ford.
It's an even simpler thing.
It's already there.
You don't have to multiply.
But that's the same running time.
That's a good answer.
The diagonal would have a negative value, yeah.
So, this is just a cute thing.
Both approaches would work, can detect a negative weight cycle just by looking at the diagonal of the matrix.
You just look for a negative value in the diagonal.
OK.
So, that's algorithm one, let's say.
I mean, we've seen several that are all bad, but I'll call this number one.
OK, we'll see two more.
This is the only one that will, well, I shouldn't say that.
Fine, there we go.
So, this is one dynamic program that wasn't so helpful, except it showed us a connection to matrix multiplication, which is interesting.
We'll see why it's useful a little bit more.
But, it bled to this nasty four nested loops.
And, using this trick, we got down to n^3 log n. Let's try, just for n^3.
OK, just get rid of that log.
It's annoying.
It makes you a little bit worse than Bellman-Ford, and the sparse case.
So, let's just erase one of these nested loops.
OK, I want to do that.
OK, obviously that algorithm doesn't work because it's for first decay, and it's not defined, but, you know, I've got enough variables.
Why don't I just define k to the m?
OK, it turns out that works.
I'll do it from scratch, but why not?
I don't know if that's how Floyd and Warshall came up with their algorithm, but here you go.
Here's Floyd-Warshall.
The idea is to define the subproblems a little bit more cleverly so that to compute one of these values, you don't have to take the min of n things.
I just want to take the min of two things.
If I could do that, and I still only have n^3 subproblems, then I would have n^3 time.
So, all right, the running time of dynamic program is number of subproblems times the time to compute the recurrence for one subproblem.
So, here's linear times n^3, and we want n^3 times constant.
That would be good.
So that's Floyd-Warshall.
So, here's the way we're going to redefine c_ij.
Or I guess, there it was called d_ij.
Good, so we're going to define something new.
So, c_ij superscript k is now going to be the weight of the shortest path from I to j as before.
Notice I used the superscript k instead of m because I want k and m to be the same thing.
Deep.
OK, now, here's the new constraint.
I want all intermediate vertices along the path, meeting all vertices except for I and j at the beginning and the end to have a small label.
So, they should be in the set from one up to k. And this is where we are really using that our vertices are labeled one up to m. So, I'm going to say, well, first think about the shortest paths that don't use any other vertices.
That's when k is zero.
Then think about all the shortest paths that maybe they use vertex one.
And then think about the shortest paths that maybe use vertex one or vertex two.
Why not?
You could define it in this way.
It turns out, then when you increase k, you only have to think about one new vertex.
Here, we had to take min over all k. Now we know which k to look at.
OK, maybe that made sense.
Maybe it's not quite obvious yet.
But I'm going to redo this claim, redo a recurrence.
So, maybe first I should say some obvious things.
So, if I want delta of ij of the shortest pathway, well, just take all the vertices.
So, take c_ij superscript n. That's everything.
And this even works, this is true even if you have a negative weight cycle.
Although, again, we're going to sort of ignore negative weight cycles as long as we can detect them.
And, another simple case is if you have, well, c_ij to zero.
Let me put that in the claim to be a little bit more consistent here.
So, here's the new claim.
If we want to compute c_ij superscript zero, what is it?
Superscript zero means I really shouldn't use any intermediate vertices.
So, this has a very simple answer, a three letter answer.
So, it's not zero.
It's four letters.
What's that?
Nil.
No, not working yet.
It has some subscripts, too.
So, the definition would be, what's the shortest path weight from I to j when you're not allowed to use any intermediate vertices?
Sorry?
So, yeah, it has a very simple name.
That's the tricky part.
All right, so if i equals j, [LAUGHTER] you're clever, right, open bracket i equals j means one, well, OK.
It sort of works, but it's not quite right.
In fact, I want infinity if i does not equal j.
And I want to zero if i equals j, a_ij, good.
I think it's a_ij.
It should be, right?
Maybe I'm wrong.
Right, a_ij.
So it's essentially not what I said.
That's the point.
If i does not equal j, you still have to think about a single edge connecting i to j, right?
OK, so that's a bit of a subtlety.
This is only intermediate vertices, so you could still go from i to j via a single edge.
That will cost a_ij.
If there is an edge: infinity.
If there isn't one: that is a_ij.
So, OK, that gets us started.
And then, we want a recurrence.
And, the recurrence is, well, maybe you get away with all the vertices that you had before.
So, if you want to know paths that you had before, so if you want to know paths that use one up to k, maybe I just use one up to k minus one.
You could try that.
Or, you could try using k. So, either you use k or you don't.
If you don't, it's got to be this.
If you do, then you've got to go to k. So why not go to k at the end?
So, you go from I to k using the previous vertices.
Obviously, you don't want to repeat k in there.
And then, you go from k to j somehow using vertices that are not k. This should be pretty intuitive.
Again, I can draw a picture.
So, either you never go to k, and that's this wiggly line.
You go from i to j using things only one up to k minus one.
In other words, here we have to use one up to k. So, this just means don't use k. So, that's this thing.
Or, you use k somewhere in the middle there.
OK, it's got to be one of the two.
And in this case, you go from i to k using only smaller vertices, because you don't want to repeat k. And here, you go from k to j using only smaller labeled vertices.
So, every path is one of the two.
So, we take the shortest of these two subproblems.
That's the answer.
So, now we have a min of two things.
It takes constant time to compute.
So, we get a cubic algorithm.
So, let me write it down.
So, this is the Floyd-Warshall algorithm.
I'll write the name again.
You give it a matrix A.
That's all it really needs to know.
It codes everything.
You copy C to A.
That's the warm up.
Right at time zero, C equals A.
And then you just have these three loops for every value of k, for every value of i, and for every value of j.
You compute that min.
And if you think about it a little bit, that min is a relaxation.
Surprise, surprise.
So, that is the Floyd-Warshall algorithm.
And, the running time is clearly n^3, three nested loops, constant time inside.
So, we're finally getting something that is never worse than Bellman-Ford.
In the sparse case, it's the same.
And anything denser, the number of edges is super linear.
This is strictly better than Bellman-Ford.
And, it's better than everything we've seen so far for all pair, shortest paths.
And, this handles negative weights; very simple algorithm, even simpler than the one before.
It's just relaxation within three loops.
What more could you ask for?
And we need to check that this is indeed what min we're computing here, except that the superscripts are omitted.
That's, again, a bit of hand waving a bit.
It's OK to omit subscripts because that can only mean that you're doing more relaxation techniques should be.
Doing more relaxations can never hurt you.
In particular, we do all the ones that we have to.
Therefore, we find the shortest path weights.
And, again, here, we're assuming that there is no negative weight cycles.
It shouldn't be hard to find them, but you have to think about that a little bit.
OK, you could run another round of Bellman-Ford, see if it relaxes in a new edges again.
For example, I think there's no nifty trick for that version.
And, we're going to cover, that's our second algorithm for all pairs shortest paths.
Before we go up to the third algorithm, which is going to be the cleverest of them all, the one Ring to rule them all, to switch trilogies, we're going to take a little bit of a diversion, side story, whatever, and talk about transitive closure briefly.
This is just a good thing to know about.
And, it relates to the algorithms we've seen so far.
So, here's a transitive closure problem.
I give you a directed graph, and for all pair vertices, i and j, I want to compute this number.
It's one if there's a path from i to j.
From i to j, OK, and then zero otherwise.
OK, this is sort of like a boring adjacency matrix with no weights, except it's about paths instead of being about edges.
OK, so how can I compute this?
That's very simple.
How should I compute this?
This gives me a graph in some sense.
This is adjacency matrix of a new graph called the transitive closure of my input graph.
So, breadth first search, yeah, good.
So, all I need to do is find shortest paths, and if the weights come out infinity, then there's no path.
If it's less than infinity, that there's a path.
And so here, so you are saying maybe I don't care about the weights, so I can run breadth first search n times, and that will work indeed.
So, if we do B times B of S, so it's maybe weird that I'm covering here in the middle, but it's just an interlude.
So, we have, then, something like V times E. OK, you can run any of these algorithms.
You could take Floyd-Warshall for example.
Why not?
OK, then it would just be V^ I mean, you could run in any of these algorithms with weights of one or zero, and just check whether the values are infinity or not.
So, I mean, t_ij equals zero, if and only if the shortest path weight from i to j is infinity.
So, just solve this.
This is an easier problem than shortest paths.
It is, in fact, strictly easier in a certain sense, because what's going on with transitive closure, and I just want to mention this out of interest because transitive closure is a useful thing to know about.
Essentially, what we are doing, let me get this right, is using a different set of operators.
We're using or and and, a logical or and and instead of min and plus, OK, because we want to know, if you think about a relaxation, in some sense, maybe I should think about it in terms of this min.
So, if I want to know, is there a pathway from I to j that uses vertices labeled one through k in the middle?
Well, either there is a path that doesn't use the vertex k, or there is a path that uses k, and then it would have to look like that.
OK, so there would have to be a path here, and there would have to be a path there.
So, the min and plus get replaced with or and and.
And if you remember, this used to be plus, and this used to be product in the matrix world.
So, plus is now like or.
And, multiply is now like and, which sounds very good, right?
Plus does feel like or, and multiply does feel like and if you live in a zero-one world.
So, in fact, this is not quite the field Z mod two, but this is a good, nice, field to work in.
This is the Boolean world.
So, I'll just write Boole.
Good old Boole knows all about this.
It's like his master's thesis, I think, talking about Boolean algebra.
And, this actually means that you can use fast matrix multiply.
You can use Strassen's algorithm, and the fancier algorithms, and you can compute the transitive closure in subcubic time.
So, this is sub cubic if the edges are sparse.
But, it's cubic in the worst case if there are lots of edges.
This is cubic.
You can actually do better using Strassen.
So, I'll just say you can do it.
No details here.
I think it should be, so in fact, there is a theorem.
This is probably not in the textbook, but there's a theorem that says transitive closure is just as hard as matrix multiply.
OK, they are equivalent.
Their running times are the same.
We don't know how long it takes to do a matrix multiply over a field.
It's somewhere between n^2 and n^2.3.
But, whatever the answer is: same for transitive closure.
OK, there's the interlude.
And that's where we actually get to use Strassen and friends.
Remember, Strassen was n to the log base two of seven algorithm.
Remember that, especially on the final.
Those are things you should have at the tip of your tongue.
OK, the last algorithm we're going to cover is really going to build on what we saw last time: Johnson's algorithm.
And, I've lost some of the running times here.
But, when we had unweighted graphs, we could do all pairs really fast, just as fast as a single source Bellman-Ford.
That's kind of nifty.
We don't know how to improve Bellman-Ford in the single source case.
So, we can't really help to get anything better than V times E. And, if you remember running V times Dijkstra, V times Dijkstra was about the same.
So, just put this in the recall bubble here: V times Dijkstra would give us V times E plus V^2 log V. And, if you ignore that log factor, this is just VE.
OK, so this was really good.
Dijkstra was great.
And this was for nonnegative edge weights.
So, with negative edge weights, somehow we'd like to get the same running time.
Now, how might I get the same running time?
Well, it would be really nice if I could use Dijkstra.
Of course, Dijkstra doesn't work with negative weights.
So what could I do?
What would I hope to do?
What could I hope to?
Suppose I want, in the middle of the algorithm, it says run Dijkstra n times.
Then, what should I do to prepare for that?
Make all the weights positive, or nonnegative.
Why not, right?
We're being wishful thinking.
That's what we'll do.
So, this is called graph re-weighting.
And, what's cool is we actually already know how to do it.
We just don't know that we know how to do it.
But I know that we know that we know how to do it.
You don't yet know that we know that I know that we know how to do it.
So, it turns out you can re-weight the vertices.
So, at the end of the last class someone asked me, can you just, like, add the same weight to all the edges?
That doesn't work.
Not quite, because different paths have different numbers of edges.
What we are going to do is add a particular weight to each vertex.
What does that mean?
Well, because we really only have weights on the edges, here's what well do.
We'll re-weight each edge, so, (u,v), let's say, going to go back into graph speak instead of matrix speak, (u,v) instead of I and j, and we'll call this modified weight w_h.
h is our function.
It gives us a number for every vertex.
And, it's just going to be the old weight of that edge plus the weight of the start vertex minus the weight of the terminating vertex.
I'm sure these have good names.
One of these is the head, and the other is the tail, but I can never remember which.
OK, so we've directed edge (u,v).
Just add one of them; subtract the other.
And, it's a directed edge, so that's a consistent definition.
OK, so that's called re-weighting.
Now, this is actually a theorem.
If you do this, then, let's say, for any vertices, u and v in the graph, for any two vertices, all paths from u to v have the same weight as they did before, well, not quite.
They have the same re-weighting.
So, if you look at all the different paths and you say, well, what's the difference between vh, well, sorry, let's say delta, which is the old shortest paths, and deltas of h, which is the shortest path weights according to this new weight function, then that difference is the same.
So, we'll say that all these paths are re-weighted by the same amounts.
OK, this is actually a statement about all paths, not just shortest paths.
There we go.
OK, to how many people is this obvious already?
A few, yeah, it is.
And what's the one word?
OK, it's maybe not that obvious.
All right, shout out the word when you figure it out.
Meanwhile, I'll write out this rather verbose proof.
There's a one word proof, still waiting.
So, let's just take one of these paths that starts at u and ends at v. Take any path.
We're just going to see what its new weight is relative to its old weight.
And so, let's just write out w_h of the path, which we define in the usual way as the sum over all edges of the new weight of the edge from v_i to v_i plus one.
Do you have the word?
No?
Tough puzzle then, OK.
So that's the definition of the weight of a path.
And, then we know this thing is just w of v_i, v_i plus one.
I'll get it right, plus the weight of the first vertex, plus, sorry, the re-weighting of v_i minus the re-weighting of v_i plus one.
This is all in parentheses that's summed over I.
Now I need the magic word.
Telescopes, good.
Now this is obvious: each of these telescopes with an extra previous, except the very beginning and the very end.
So, this is the sum of these weights of edges, but then outside the sum, we have plus h of v_1, and minus h of v_k.
OK, those guys don't quite cancel.
We're not looking at a cycle, just a path.
And, this thing is just w of the path, as this is the normal weight of the path.
And so the change, the difference between w_h of P and w of P is this thing, which is just h of u minus h of v. And, the point is that's the same as long as you fix the endpoints, u and v, of the shortest path, you're changing this path weight by the same thing for all paths.
This is for any path from u to v, and that proves the theorem.
So, the one word here was telescopes.
These change in weights telescope over any path.
Therefore, if we want to find shortest paths, you just find the shortest paths in this re-weighted version, and then you just change it by this one amount.
You subtract off this amount instead of adding it.
That will give you the shortest path weight in the original weights.
OK, so this is a tool.
We now know how to change weights in the graph.
But what we really want is to change weights in the graph so that the weights all come out nonnegative.
OK, how do we do that?
Why in the world would there be a function, h, that makes all the edge weights nonnegative?
It doesn't make sense.
It turns out we already know.
So, I should write down this consequence.
Let me get this in the right order.
So in particular, the shortest path changes by this amount.
And if you want to know this value, you just move the stuff to the other side.
So, we compute deltas of h, then we can compute delta.
That's the consequence here.
How many people here pronounce this word corollary?
OK, and how many people pronounce it corollary?
Yeah, we are alone.
Usually get at least one other student, and they're usually Canadian or British or something.
I think that the accent.
So, I always avoid pronouncing his word unless I really think, it's corollary, and get it right.
I at least say Z not Zed.
OK, here we go.
So, what we want to do is find one of these functions.
I mean, let's just write down what we could hope to have.
We want to find a re-weighted function, h, the signs of weight to each vertex such that w_h of (u,v) is nonnegative.
That would be great for all edges, all (u,v) in E. OK, then we could run Dijkstra.
We could run Dijkstra, get the delta h's, and then just undo the re-weighting, and get what we want.
And, that is Johnson's algorithm.
The claim is that this is always possible.
OK, why should it always be possible?
Well, let's look at this constraint.
w_h of (u,v) is that.
So, it's w of (u,v) plus h of u minus h of V should be nonnegative.
Let me rewrite this a little bit.
I'm going to put these guys over here.
That would be the right thing, h of v minus h of u is less than or equal to w of (u,v).
Does that look familiar?
Did I get it right?
It should be right.
Anyone seen that inequality before?
Yeah, yes, correct answer.
OK, where?
In a previous lecture?
In the previous lecture.
What is this called if I replace h with x?
Charles knows.
Good, anyone else remember all the way back to episode two?
I know there was a weekend.
What's this operator called?
Not subtraction but, I think I heard it, oh man.
All right, I'll tell you.
It's a difference constraint, all right?
This is the difference operator.
OK, it's our good friend difference constraints.
So, this is what we want to satisfy.
We have a system of difference constraints.
h of V minus h of u should be, we want to find these.
These are our unknowns.
Subject to these constraints, we are given the w's.
Now, we know in these difference constraints are satisfiable.
Can someone tell me when these constraints are satisfiable?
We know exactly when for any set of difference constraints.
You've got to remember the math.
Terminology, I can understand.
It's hard to remember words unless you're a linguist, perhaps.
So, when is the system of different constraints satisfiable?
All right, you should definitely, very good.
[LAUGHTER] Yes, very good.
Someone brought their lecture notes: when the constraint graph has no negative weight cycles.
Good, thank you.
Now, what is the constraint graph?
OK, this has a one letter answer more or less.
I'll accept the one letter answer.
What?
A?
close.
G. Yeah, I mean, same thing.
Yeah, so the constraint graph is essentially G. Actually, it is G. The constraint graph is G, good.
And, we prove this by adding a new source for text, and connecting that to everyone.
But that's sort of beside the point.
That was in order to actually satisfy them.
But this is our characterization.
So, if we assume that there are no negative weight cycles in our graph, which we've been doing all the time, then we know that this thing is satisfiable.
Therefore, there is an assignment of this h's.
There is a re-weighting that makes all the weights nonnegative.
Then we can run Dijkstra.
OK, we're done.
Isn't that cool?
And how do we satisfy these constraints?
We know how to do that with one run of Bellman-Ford, which costs order VE, which is less than V times Dijkstra.
So, that's it, write down the details somewhere.
So, this is Johnson's algorithm.
This is the fanciest of them all.
It will be our fastest, all pairs shortest path algorithm.
So, the claim is, we can find a function, h, from V to R such that the modified weight of every edge is nonnegative for every edge, (u,v), in our graph.
And, we do that using Bellman-Ford to solve the difference constraints.
These are exactly the different constraints that we were born to solve that we learned to solve last time.
The graphs here are corresponding exactly if you look back at the definition.
Or, Bellman-Ford will tell us that there is a negative weight cycle.
OK, great, so it's not that we really have to assume that there is no negative weight cycle.
We'll get to know.
And if your fancy, you can actually figure out the minus infinities from this.
But, at this point, I just want to think about the case where there is no negative weight cycle.
But if there is, we can find out that it exists, and that just tell the user.
OK, then we'd stop.
Otherwise, there is no negative weight cycle.
Therefore, there is an assignment that gives is nonnegative edge weights.
So, we just use it.
We use it to run Dijkstra.
So, step two is, oh, I should say the running time of all this is V times E. So, we're just running Bellman-Ford on exactly the input graph.
Plus, we add a source, if you recall, to solve a set of difference constraints.
You add a source vertex, S, connected to everyone at weight zero, run Bellman-Ford from there because we don't have a source here.
We just have a graph.
We want to know all pairs.
So, this, you can use to find whether there is a negative weight cycle anywhere.
Or, we get this magic assignment.
So now, w_h is nonnegative, so we can run Dijkstra on w_h.
We'll say, using w_h, so you compute w_h.
That takes linear time.
And, we run Dijkstra for each possible source.
I'll write this out explicitly.
We've had this in our minds several times.
But, when we said n times Dijkstra over n times BFS, here it is.
We want to compute delta sub h now, of (u,v) for all V, and we do this separately for all u.
And so, the running time here is VE plus V^2 log V. This is just V times the running time of Dijkstra, which is E plus V log V. OK, it happens that this term is the same as this one, which is nice, because that means step one costs us nothing asymptotically.
OK, and then, last step is, well, now we know delta h. We just need to compute delta.
So, for each pair of vertices, we'll call it (u,v), we just compute what the original weights would be, so what delta (u,v) is.
And we can do that using this corollary.
It's just delta sub h of (u,v) minus h of u plus h of v. I got the signs right.
Yeah, so this takes V^2 time, also dwarfed by the running time of Dijkstra.
So, the overall running time of Johnson's algorithm is just the running time of step two, running Dijkstra n times -- -- which is pretty cool.
When it comes to single source shortest paths, Bellman-Ford is the best thing for general weights.
Dijkstra is the best thing for nonnegative weights.
But for all pair shortest paths, we can skirt the whole negative weight issue by using this magic we saw from Bellman-Ford.
But now, running Dijkstra n times, which is still the best thing we know how to do, pretty much, for the all pairs nonnegative weights, now we can do it for general weights too, which is a pretty nice combination of all the techniques we've seen.
In the trilogy, and along the way, we saw lots of dynamic programming, which is always good practice.
Any questions?
This is the last new content lecture before the quiz.
On Wednesday it will be quiz review, if I recall correctly.
And then it's Thanksgiving, so there's no recitation.
And then the quiz starts on Monday.
So, study up.
See you then.
We only have four more lectures left, and what Professor Demaine and I have decided to do is give two series of lectures on sort of advanced topics.
So, today at Wednesday we're going to talk about parallel algorithms, algorithms where you have more than one processor whacking away on your problem.
And this is a very hot topic right now because all of the chip manufacturers are now producing so-called multicore processors where you have more than one processor per chip.
So, knowing something about that is good.
The second topic we're going to cover is going to be caching, and how you design algorithms for systems with cache.
Right now, we've sort of program to everything as if it were just a single level of memory, and for some problems that's not an entirely realistic model.
You'd like to have some model for how the caching hierarchy works, and how you can take advantage of that.
And there's been a lot of research in that area as well.
So, both of those actually turn out to be my area of research.
So, this is actually fun for me.
Actually, most of it's fun anyway.
So, today we'll talk about parallel algorithms.
And the particular topic, it turns out that there are lots of models for parallel algorithms, and for parallelism.
And it's one of the reasons that, whereas for serial algorithms, most people sort of have this basic model that we've been using.
It's sometimes called a random access machine model, which is what we've been using to analyze things, whereas in the parallel space, there's just a huge number of models, and there is no general agreement on what is the best model because there are different machines that are made with different configurations, etc.
and people haven't, sort of, agreed on, even how parallel machines should be organized.
So, we're going to deal with a particular model, which goes under the rubric of dynamic multithreading, which is appropriate for the multicore machines that are now being built for shared memory programming.
It's not appropriate for what's called distributed memory programs particularly because the processors are able to access things.
And for those, you need more involved models.
And so, let me start just by giving an example of how one would write something.
I'm going to give you a program for calculating the nth Fibonacci number in this model.
This is actually a really bad algorithm that I'm going to give you because it's going to be the exponential time algorithm, whereas we know from week one or two that you can calculate the nth Fibonacci number and how much time?
Log n time.
So, this is too exponentials off what you should be able to get, OK, two exponentials off.
OK, so here's the code.
OK, so this is essentially the pseudocode we would write.
And let me just explain a little bit about, we have a couple of key words here we haven't seen before: in particular, spawn and sync.
OK, so spawn, this basically says that the subroutine that you're calling, you use it as a keyword before a subroutine, that it can execute at the same time as its parent.
So, here, what we say x equals spawn of n minus one, we immediately go onto the next statement.
And now, while we're executing fib of n minus one, we can also be executing, now, this statement which itself will spawn something off.
OK, and we continue, and then we hit the sync statement.
And, what sync says is, wait until all children are done.
OK, so it says once you get to this point, you've got to wait until everything here has completed before you execute the x plus y because otherwise you're going to try to execute the calculation of x plus y without having computed it yet.
OK, so that's the basic structure.
What this describes, notice in here we never said how many processors or anything we are running on.
OK, so this actually is just describing logical parallelism -- -- not the actual parallelism when we execute it.
And so, what we need is a scheduler, OK, to determine how to map this dynamically, unfolding execution onto whatever processors you have available.
OK, and so, today actually we're going to talk mostly about scheduling.
OK, and then, next time we're going to talk about specific application algorithms, and how you analyze them.
OK, so you can view the actual multithreaded computation.
If you take a look at the parallel instruction stream, it's just a directed acyclic graph, OK?
So, let me show you how that works.
So, normally when we have an instruction stream, I look at each instruction being executed.
If I'm in a loop, I'm not looking at it as a loop.
I'm just looking at the sequence of instructions that actually executed.
I can do that just as a chain.
Before I execute one instruction, I have to execute the one before it.
Before I execute that, I've got to execute the one before it.
At least, that's the abstraction.
If you've studied processors, you know that there are a lot of tricks there in figuring out instruction level parallelism, and how you can actually make that serial instruction stream actually execute in parallel.
But what we are going to be mostly talking about is the logical parallelism here, and what we can do in that context.
So, in this DAG, the vertices are threads, which are maximal sequences of instructions not containing -- -- parallel control.
And by parallel control, I just mean spawn, sync, and return from a spawned procedure.
So, let's just mark the, so the vertices are threads.
So, let's just mark what the vertices are here, OK, what the threads are here.
So, when we enter the function here, we basically execute up to the point where, basically, here, let's call that thread A where we are just doing a sequential execution up to either returning or starting to do the spawn, fib of n minus one.
So actually, thread A would include the calculation of n minus one right up to the point where you actually make the subroutine jump.
That's thread A. Thread B would be the stuff that you would do, executing from fib of, sorry, B would be from the, right.
We'd go up to the spawn.
So, we've done the spawn.
I'm really looking at this.
So, B would be up to the spawn of y. OK, spawn of fib of n minus two to compute y, and then we'd have essentially an empty thread.
So, I'll ignore that for now, but really then we have after the sync up to the point that we get to the return of x plus y.
So basically, we're just looking at maximal sequences of instructions that are all serial.
And every time I do a parallel instruction, OK, spawn or a sync, or return from it, that terminates the current thread.
OK, so we can look at that as a bunch of small threads.
So those of you who are familiar with threads from Java threads, or POSIX threads, OK, so-called P threads, those are sort of heavyweight static threads.
This is a much lighter weight notion of thread, OK, that we are using in this model.
OK, so these are the vertices.
And now, let me map out a little bit how this works, so we can where the edges come from.
So, let's imagine we're executing fib of four.
So, I'm going to draw a horizontal oval.
That's going to correspond to the procedure execution.
And, in this procedure, there are essentially three threads.
We start out with A, so this is our initial thread is this guy here.
And then, when he executes a spawn, OK, he's going to execute a spawn, we are going to create a new procedure, and he's going to execute a new A recursively within that procedure.
But at the same time, we're also going to be, now, aloud to go on and execute B in the parent, we have parallelism here when I do a spawn.
OK, and so there's an edge here.
This edge we are going to call a spawn edge, and this is called a continuation edge because it's just simply continuing the procedure execution.
OK, now at this point, this guy, we now have two things that can execute at the same time.
Once I've executed A, I now have two things that can execute.
OK, so this one, for example, may spawn another thread here.
Oh, so this is fib of three, right?
And this is now fib of two.
OK, so he spawns another guy here, and simultaneously, he can go on and execute B here, OK, with a continued edge.
And B, in fact, can also spawn at this point.
OK, and this is now fib of two also.
And now, at this point, we can't execute C yet here even though I've spawned things off.
And the reason is because C won't execute until we've executed the sync statement, which can't occur until A and B have both been executed, OK?
So, he just sort of sits there waiting, OK, and a scheduler can't try to schedule him.
Or if he does, then nothing's going to happen here, OK?
So, we can go on.
Let's see, here we could call fib of one.
The fib of one is only going to execute an A statement here.
OK, of course it can't continue here because A is the only thing, when I execute fib of one, if we look at the code, it never executes B or C. OK, and similarly here, this guy here to do fib of one.
OK, and this guy, I guess, could execute A here of fib of one.
OK, and maybe now this guy calls his another fib of one, and this guy does another one.
This is going to be fib of zero, right?
I keep drawing that arrow to the wrong place, OK?
And now, once these guys return, well, let's say these guys return here, I can now execute C. But I can't execute with them until both of these guys are done, and that guy is done.
So, you see that we get a synchronization point here before executing C. And then, similarly here, now that we've executed this and this, we can now execute this guy here.
And so, those returns go to there.
Likewise here, this guy can now execute his C, and now once both of those are done, we can execute this guy here.
And then we are done.
This is our final thread.
So, I should have labeled also that when I get one of these guys here, that's a return edge.
So, the three types of edges are spawn, return, and continuation.
OK, and by describing it in this way, I essentially get a DAG that unfolds.
So, rather than having just a serial execution trace, I get something where I have still some serial dependencies.
There are still some things that have to be done before other things, but there are also things that can be done at the same time.
So how are we doing?
Yeah, question?
Is every spawn were covered by a sync, effectively, yeah, yeah, effectively.
There's actually a null thread that gets executed in there, which I hadn't bothered to show.
But yes, basically you would then not have any parallelism, OK, because you would spawn it off, but then you're not doing anything in the parent.
So it's pretty much the same, yeah, as if it had executed serially.
Yep, OK, so you can see that basically what we had here in some sense is a DAG embedded in a tree.
OK, so you have a tree that's sort of the procedure structure, but in their you have a DAG, and that DAG can actually get to be pretty complicated.
OK, now what I want to do is now that we understand that we've got an underlying DAG, I want to switch to trying to study the performance attributes of a particular DAG execution, so looking at performance measures.
So, the notation that we'll use is we'll let T_P be the running time of whatever our computation is on P processors.
OK, so, T_P is, how long does it take to execute this on P processors?
Now, in general, this is not going to be just a particular number, OK, because I can have different scheduling disciplines would lead me to get numbers for T_P, OK?
But when we talk about the running time, we'll still sort of use this notation, and I'll try to be careful as we go through to make sure that there's no confusion about what that means in context.
There are a couple of them, though, which are fairly well defined.
One is based on this.
One is T_1.
So, T_1 is the running time on one processor.
OK, so if I were to execute this on one processor, you can imagine it's just as if I had just gotten rid of the spawn, and syncs, and everything, and just executed it.
That will give me a particular running time.
We call that running time on one processor the work.
It's essentially the serial time.
OK, so when we talk about the work of a computation, we just been essentially a serial running time.
OK, the other measure that ends up being interesting is what we call T infinity.
OK, and this is the critical pathlength, OK, which is essentially the longest path in the DAG.
So, for example, if we look at the fib of four in this example, it has T of one equal to, so let's assume we have unit time threads.
I know they're not unit time, but let's just imagine, for the purposes of understanding this, that every thread costs me one unit of time to execute.
What would be the work of this particular computation?
17, right, OK, because all we do is just add up three, six, nine, 12, 13, 14, 15, 16, 17.
So, the work is 17 in this case if it were unit time threads.
In general, you would add up how many instructions or whatever were in there.
OK, and then T infinity is the longest path.
So, this is the longest sequence.
It's like, if you had an infinite number of processors, you still can't just do everything at once because some things have to come before other things.
But if you had an infinite number of processors, as many processors as you want, what's the fastest you could possibly execute this?
A little trickier.
Seven?
So, what's your seven?
So, one, two, three, four, five, six, seven, eight, yeah, eight is the longest path.
So, the work and the critical path length, as we'll see, are key attributes of any computation.
And abstractly, and this is just for [the notes?
], if they're unit time threads.
OK, so we can use these two measures to derive lower bounds on T_P for P that fall between one and infinity, OK?
OK, so the first lower bound we can derive is that T_P has got to be at least T_1 over P. OK, so why is that a lower bound?
Yeah?
But if I have P processors, and, OK, and why would I have this lower bound?
OK, yeah, you've got the right idea.
So, but can we be a little bit more articulate about it?
So, that's right, so you want to use all of processors.
If you could use all of processors, why couldn't I use all the processors, though, and have T_P be less than this?
Why does it have to be at least as big as T_1 over P?
I'm just asking for a little more precision in the answer.
You've got exactly the right idea, but I need a little more precision if we're going to persuade the rest of the class that this is the lower bound.
Yeah?
Yeah, that's another way of looking at it.
If you were to serialize the computation, OK, so whatever things you execute on each step, you do P of them, and so if you serialized it, somehow then it would take you P steps to execute one step of a P way, a machine with P processors.
So then, OK, yeah?
OK, maybe a little more precise.
David?
Yeah, good, so let me just state this a little bit.
So, P processors, so what are we relying on?
P processors can do, at most, P work in one step, right?
So, in one step they do, at most P work.
They can't do more than P work.
And so, if they can do, at most P work in one step, then if the number of steps was, in fact, less than T_1 over P, they would be able to do more than T_1 work in P steps.
And, there's only T_1 work to be done.
OK, I just stated that almost as badly as all the responses I got.
[LAUGHTER] OK, P processors can do, at most, P work in one step, right?
So, if there's T_1 work to be done, the number of steps is going to be at least T_1 over P, OK?
There we go.
OK, it wasn't that hard.
It's just like, I've got a certain amount of, I've got T_1 work to do.
I can knock off, at most, P on every step.
How many steps?
Just divide.
OK, so it's going to have to be at least that amount.
OK, good.
The other lower bound is T_P is greater than or equal to T infinity.
Somebody explain to me why that might be true.
Yeah?
Yeah, if you have an infinite number of processors, you have P. so if you could do it in a certain amount of time with P, you can certainly do it in that time with an infinite number of processors.
OK, this is in this model where, you know, there is lots of stuff that this model doesn't model like communication costs and interference, and all sorts of things.
But it is simple model, which actually in practice works out pretty well, OK, you're not going to be able to do more work with P processors than you are with an infinite number of processors.
OK, so those are helpful bounds to understand when we are trying to make something go faster, it's nice to know what you could possibly hope to achieve, OK, as opposed to beating your head against a wall, how come I can't get it to go much faster?
Maybe it's because one of these lower bounds is operating.
OK, well, we're interested in how fast we can go.
That's the main reason for using multiple processors is you hope you're going to go faster than you could with one processor.
So, we define T_1 over T_P to be the speedup on P processors.
OK, so we say, how much faster is it on P processors than on one processor?
OK, that's the speed up.
If T_1 over T_P is order P, we say that it's linear speedup.
OK, in other words, why?
Because that says that it means that if I've thrown P processors at the job I'm going to get a speedup that's proportional to P. OK, so when I throw P processors at the job and I get T_P, if that's order P, that means that in some sense my processors each contributed within a constant factor its full measure of support.
If this, in fact, were equal to P, we'd call that perfect linear speedup.
OK, so but here we're looking at giving ourselves, for theoretical purposes, a little bit of a constant buffer here, perhaps.
If T_1 over T_P is greater than P, we call that super linear speedup.
OK, so can somebody tell me, when can I get super linear speedup?
When can I get super linear speed up?
Never.
OK, why never?
Yeah, if we buy these lower bounds, the first lower bound there, it is T_P is greater than or equal to T_1 over P. And, if I just take T_1 over T_P, that says it's less than or equal to P. so, this is never, OK, not possible in this model.
OK, there are other models where it is possible to get super linear speed up due to caching effects, and things of that nature.
But in this simple model that we are dealing with, it's not possible to get super linear speedup.
OK, not possible.
Now, the maximum possible speedup, given some amount of work and critical path length is what?
What's the maximum possible speed up I could get over any number of processors?
What's the maximum I could possibly get?
No, I'm saying, no matter how many processors, what's the most speedup that I could get?
T_1 over T infinity, because this is the, so T_1 over T infinity is the maximum I could possibly get.
OK, if I threw an infinite number of processors at the problem, that's going to give me my biggest speedup.
OK, and we call that the parallelism.
OK, so that's defined to be the parallelism.
So the parallelism of the particular algorithm is essentially the work divided by the critical path length.
Another way of viewing it is that this is the average amount of work that can be done in parallel along each step of the critical path.
And, we denote it often by P bar.
So, do not get confused.
P bar does not have anything to do with P at some level.
OK, P is going to be a certain number of processors you're running.
P bar is defined just in terms of the computation you're executing, not in terms of the machine you're running it on.
OK, it's just the average amount of work that can be done in parallel along each step of the critical path.
OK, questions so far?
So mostly we're just doing definitions so far.
OK, now we get into, OK, so it's helpful to know what the parallelism is, because the parallelism is going to, there's no real point in trying to get speed up bigger than the parallelism.
OK, so if you are given a particular computation, you'll be able to say, oh, it doesn't go any faster.
You're throwing more processors at it.
Why is it that going any faster?
And the answer could be, no more parallelism.
OK, let's see what I want to, yeah, I think we can raise the example here.
We'll talk more about this model.
Mostly, now, we're going to just talk about DAG's.
So, we'll talk about the programming model next time.
So, let's talk about scheduling.
The goal of scheduler is to map the computation to P processors.
And this is typically done by a runtime system, which, if you will, is an algorithm that is running underneath the language layer that I showed you.
OK, so the programmer designs an algorithm using spawns, and syncs, and so forth.
Then, underneath that, there's an algorithm that has to actually map that executing program onto the processors of the machine as it executes.
And that's the scheduler.
OK, so it's done by the language runtime system, typically.
OK, so it turns out that online schedulers, let me just say they're complex.
OK, they're not necessarily easy things to build.
OK, they're not too bad actually.
But, we are not going to go there because we only have two lectures to do this.
Instead, we're going to do is we'll illustrate the ideas using off-line scheduling.
OK, so you'll get an idea out of this for what a scheduler does, and it turns out that doing these things online is another level of complexity beyond that.
And typically, the online schedulers that are good, these days, are randomized schedulers.
And they have very strong proofs of their ability to perform.
But we're not going to go there.
We'll keep it simple.
And in particular, we're going to look at a particular type of scheduler called a greedy scheduler.
So, if you have a DAG to execute, so the basic rules of the scheduler is you can't execute a node until all of the nodes that precede it in the DAG have executed.
OK, so you've got to wait until everything is executed.
So, a greedy scheduler, what it says is let's just try to do as much as possible on every step, OK?
In other words, it says I'm never going to try to guess that it's worthwhile delaying doing something.
If I could do something now, I'm going to do it.
And so, each step is going to correspond to be one of two types.
The first type is what we'll call a complete step.
And this is a step in which there are at least P threads ready to run.
And, I'm executing on P processors.
There are at least P threads ready to run.
So, what's a greedy strategy here?
I've got P processors.
I've got at least P threads.
Run any P. Yeah, first P would be if you had a notion of ordering.
That would be perfectly reasonable.
Here, we are just going to execute any P. We might make a mistake there, because there may be a particular one that if we execute now, that'll enable more parallelism later on.
We might not execute that one.
We don't know.
OK, but basically, what we're going to do is just execute any P willy-nilly.
So, there's some, if you will, non-determinism in this step here because which one you execute may or may not be a good choice.
OK, the second type of step we're going to have is an incomplete step.
And this is a situation where we have fewer than P threads ready to run.
So, what's our strategy there?
Execute all of them.
OK, if it's greedy, no point in not executing.
OK, so if I've got more than P threads ready to run, I execute any P. If I have fewer than P threads ready to run, we execute all of them.
So, it turns out this is a good strategy.
It's not a perfect strategy.
In fact, the strategy of trying to schedule optimally a DAG on P processors is NP complete, meaning it's very difficult.
So, those of you going to take 6.045 or 6.840, I highly recommend these courses, and we'll talk more about that in the last lecture as we talked a little bit about what's coming up in the theory engineering concentration.
You can learn about NP completeness and about how you show that certain problems, there are no good algorithms for them, OK, that we are aware of, OK, and what exactly that means.
So, it turns out that this type of scheduling problem turns out to be a very difficult problem to get it optimal.
But, there's nice theorem, due independently to Graham and Brent.
It says, essentially, a greedy scheduler executes any computation, G, with work, T_1, and critical path length, T infinity in time, T_P, less than or equal to T_1 over P plus T infinity -- -- on a computer with P processors.
OK, so, it says that I can achieve T_1 over P plus T infinity.
So, what does that say?
If we take a look and compare this with our lower bounds on runtime, how efficient is this?
How does this compare with the optimal execution?
Yeah, it's two competitive.
It's within a factor of two of optimal because this is a lower bound and this is a lower bound.
And so, if I take twice the max of these two, twice the maximum of these two, that's going to be bigger than the sum.
So, I'm within a factor of two of which ever is the stronger, lower bound for any situation.
So, this says you get within a factor of two of efficiency of scheduling in terms of the runtime on P processors.
OK, does everybody see that?
So, let's prove this theorem.
It's quite an elegant theorem.
It's not a hard theorem.
One of the nice things, by the way, about this week, is that nothing is very hard.
It just requires you to think differently.
OK, so the proof has to do with counting up how many complete steps we have, and how many incomplete steps we have.
OK, so we'll start with the number of complete steps.
So, can somebody tell me what's the largest number of complete steps I could possibly have?
Yeah, I heard somebody mumble it back there.
T_1 over P. Why is that?
Yeah, so the number of complete steps is, at most, T_1 over P because why?
Yeah, once you've had this many, you've done T_1 work, OK?
So, every complete step I'm getting P work done.
So, if I did more than T_1 over P steps, there would be no more work to be done.
So, the number of complete steps can't be bigger than T_1 over P. OK, so that's this piece.
OK, now we're going to count up the incomplete steps, and show its bounded by T infinity.
OK, so let's consider an incomplete step.
And, let's see what happens.
And, let's let G prime be the subgraph of G that remains to be executed.
OK, so we'll draw a picture here.
So, imagine we have, let's draw it on a new board.
So here, we're going to have a graph, our graph, G. We're going to do actually P equals three as our example here.
So, imagine that this is the graph, G. And, I'm not showing the procedures here because this actually is a theorem that works for any DAG.
And, the procedure outlines are not necessary.
All we care about is the threads.
I missed one.
OK, so imagine that's my DAG, G, and imagine that I have executed up to this point.
Which ones have I executed?
Yeah, I've executed these guys.
So, the things that are in G prime are just the things that have yet to be executed.
And these guys are the ones that are already executed.
And, we'll imagine that all of them are unit time threads without loss of generality.
The theorem would go through, even if each of these had a particular time associated with it.
The same scheduling algorithm will work just fine.
So, how can I characterize the threads that are ready to be executed?
Which are the threads that are ready to be executed here?
Let's just see.
So, that one?
No, that's not ready to be executed.
Why?
Because it's got a predecessor here, this guy.
OK, so this guy is ready to be executed, and this guy is ready to be executed.
OK, so those two threads are ready to be, how can I characterize this?
What's their property?
What's a graph theoretic property in G prime that tells me whether or not something is ready to be executed?
It has no predecessor, but what's another way of saying that?
It's got no predecessor in G prime.
What does it mean for a node not to have a predecessor in a graph?
Its in degree is zero, right?
Same thing.
OK, the threads with in degree, zero and G prime are the ones that are ready to be executed.
OK, and if it's incomplete step, what do I do?
I'm going to execute says, if it's an incomplete step, I execute all of them.
OK, so I execute all of these.
OK, now I execute all of the in degree zero threads, what happens to the critical path length of the graph that remains to be executed?
It decreases by one.
OK, so the critical path length of what remains to be executed, G prime, is reduced by one.
So, what's left to be executed on every incomplete step, what's left to be executed always reduces by one.
Notice the next step here is going to be a complete step, because I've got four things that are ready to go.
And, I can execute them in such a way that the critical path length doesn't get reduced on that step.
OK, but if I had to execute all of them, then it does reduce the critical path length.
Now, of course, both could happen, OK, at the same time, OK, but any time that I have an incomplete step, I'm guaranteed to reduce the critical path length by one.
OK, so that implies that the number of incomplete steps is, at most, T infinity.
And so, therefore, T of P is, at most, the number of complete steps plus the number of incomplete steps.
And we get our bound.
This is sort of an amortized argument if you want to think of it that way, OK, that at every step I'm either amortizing the step against the work, or I'm amortizing it against the critical path length, or possibly both.
But I'm at least doing one of those for every step, OK, and so, in the end, I just have to add up the two contributions.
Any questions about that?
So this, by the way, is the fundamental theorem of all scheduling.
If ever you study anything having to do with scheduling, this basic result is sort of the foundation of a huge number of things.
And then what people do is they gussy it up, like, let's do this online, OK, with a scheduler, etc., that everybody's trying to match these bounds, OK, of what an omniscient greedy scheduler would achieve, OK, and there are all kinds of other things.
But this is sort of the basic theorem that just pervades the whole area of scheduling.
OK, let's do a quick corollary.
I'm not going to erase those.
Those are just too important.
I want to erase those.
Let's not erase those.
I want to erase that either.
We're going to go back to the top.
Actually, we'll put the corollary here because that's just one line.
OK.
The corollary says you get linear speed up if the number of processors that you allocate, that you run your job on is order, the parallelism.
OK, so greedy scheduler gives you linear speed up if you're running on essentially parallelism or fewer processors.
OK, so let's see why that is.
And I hope I'll fit this, OK?
So, P bar is T_1 over T infinity.
And that implies that if P equals order T_1 over T infinity, then that says just bringing those around, T infinity is order T_1 over P. So, everybody with me?
It's just algebra.
So, it says this is the definition of parallelism, T_1 over T infinity, and so, if P is order parallelism, then it's order T_1 over T infinity.
And now, just bring it around.
It says T infinity is order T_1 over P. So, that says T infinity is order T_1 over P. OK, and so, therefore, continue the proof here, thus T_P is at most T_1 over P plus T infinity.
Well, if this is order T_1 over P, the whole thing is order T_1 over P. OK, and so, now I have T_P is order T_1 over P, and what we need is to compute T_1 over T_P, and that's going to be order T_P.
OK?
Does everybody see that?
So what that says is that if I have a certain amount of parallelism, if I run essentially on fewer processors than that parallelism, I get linear speed up if I use greedy scheduling.
OK, if I run on more processors than the parallelism, in some sense I'm being wasteful because I can't possibly get enough speed up to justify those extra processors.
So, understanding parallelism of a job says that's sort of a limit on the number of processors I want to have.
And, in fact, I can achieve that.
Question?
Yeah, really, in some sense, this is saying it should be omega P. Yeah, so that's fine.
It's a question of, so ask again.
No, no, it's only if it's bounded above by a constant.
T_1 and T infinity aren't constants.
They're variables in this.
So, we are doing multivariable asymptotic analysis.
So, any of these things can be a function of anything else, and can be growing as much as we want.
So, the fact that we say we are given it for a particular thing, we're really not given that number.
We're given a whole class of DAG's or whatever of various sizes is really what we're talking about.
So, I can look at the growth.
Here, where it's talking about the growth of the parallelism, sorry, the growth of the runtime T_P as a function of T_1 and T infinity.
So, I am talking about things that are growing here, OK?
OK, so let's put this to work, OK?
And, in fact, so now I'm going to go back to here.
Now I'm going to tell you about a little bit of my own research, and how we use this in some of the work that we did.
OK, so we've developed a dynamic multithreaded language called Cilk, spelled with a C because it's based on the language, C. And, it's not an acronym because silk is like nice threads, OK, although at one point my students had a competition for what the acronym silk could mean.
The winner, turns out, was Charles' Idiotic Linguistic Kluge.
So anyway, if you want to take a look at it, you can find some stuff on it here.
OK, OK, and what it uses is actually one of these more complicated schedulers.
It's a randomized online scheduler, OK, and if you look at its expected runtime on P processors, it gets effectively T_1 over P plus O of T infinity provably.
OK, and empirically, if you actually look at what kind of runtimes you get to find out what's hidden in the big O there, it turns out, in fact, it's T_1 over P plus T infinity with the constants here being very close to one empirically.
So, no guarantees, but this turns out to be a pretty good bound.
Sometimes, you see a coefficient on T infinity that's up maybe close to four or something.
But generally, you don't see something that's much bigger than that.
And mostly, it tends to be around, if you do a linear regression curve fit, you get that the constant here is close to one.
And so, with this, you get near perfect if you use this formula as a model for your runtime.
You get near perfect linear speed up if the number of processors you're running on is much less than your average parallelism, which, of course, is the same thing as if T infinity is much less than T_1 over P. So, what happens here is that when P is much less than P infinity, that is, T infinity is much less than T_1 over P, this term ceases to matter very much, and you get very good speedup, OK, in fact, almost perfect speedup.
So, each processor gives you another processor's work as long as you are the range where the number of processors is much less than the number of parallelism.
Now, with this language many years ago, which seems now like many years ago, OK, it turned out we competed.
We built a bunch of chess programs.
And, among our programs were Starsocrates, and Cilkchess, and we also had several others.
And these were, I would call them, world-class.
In particular, we tied for first in the 1995 World Computer Chess Championship in Hong Kong, and then we had a playoff and we lost.
It was really a shame.
We almost won, running on a big parallel machine.
That was, incidentally, some of you may know about the Deep Blue chess playing program.
That was the last time before they faced then world champion Kasparov that they competed against programs.
They tied for third in that tournament.
OK, so we actually out-placed them.
However, in the head-to-head competition, we lost to them.
So we had one loss in the tournament up to the point of the finals.
They had a loss and a draw.
Most people aren't aware that Deep Blue, in fact, was not the reigning World Computer Chess Championship when they faced Kasparov.
The reason that they faced Kasparov was because IBM was willing to put up the money.
OK, so we developed these chess programs, and the way we developed them, let me in particular talk about Starsocrates.
We had this interesting anomaly come up.
We were running on a 32 processor computer at MIT for development.
And, we had access to a 512 processor computer for the tournament at NCSA at the University of Illinois.
So, we had this big machine.
Of course, they didn't want to give it to us very much, but we have the same machine, just a small one, at MIT.
So, we would develop on this, and occasionally we'd be able to run on this, and this was what we were developing for on our processor.
So, let me show you sort of the anomaly that came up, OK?
So, we had a version of a program that I'll call the original program, OK, and we had an optimized program that included some new features that were supposed to make the program go faster.
And so, we timed it on our 32 processor machine.
And, it took us 65 seconds to run it.
OK, and then we timed this new program.
So, I'll call that T prime of sub 32 on our 32 processor machine, and it ran and 40 seconds to do this particular benchmark.
Now, let me just say, I've lied about the actual numbers here to make the calculations easy.
But, the same idea happened.
Just the numbers were messier.
OK, so this looks like a significant improvement in runtime, but we rejected the optimization.
OK, and the reason we rejected it is because we understood about the issues of work and critical path.
So, let me show you the analysis that we did, OK?
So the analysis, it turns out, if we looked at our instrumentation, the work in this case was 2,048.
And, the critical path was one second, which, over here with the optimized program, the work was, in fact, 1,024.
But the critical path was eight.
So, if we plug into our simple model here, the one I have up there with the approximation there, I have T_32 is equal to T_1 over 32 plus T infinity, and that's equal to, well, the work is 2,048 divided by 32.
What's that?
64, good, plus the critical path, one, that's 65.
So, that checks out with what we saw.
OK, in fact, we did that, and it checked out.
OK, it was very close.
OK, over here, T prime of 32 is T prime, one over 32 plus T infinity prime, and that's equal to 1,024 divided by 32 is 32 plus eight, the critical path here.
That's 40.
So, that checked out too.
So, now what we did is we said is we said, OK, let's extrapolate to our big machine.
How fast are these things going to run on our big machine?
Well, for that, we want T of 512.
And, that's equal to T_1 over 512 plus T infinity.
And so, what's 2,048 divided by 512?
It's four, plus T infinity is one.
That's equal to five.
So, go quite a bit faster on this.
But here, T prime of 512 is equal to T one prime over 512 plus T infinity prime is equal to, well, 1,024 plus divided by 512 is two plus critical path of eight, that's ten.
OK, and so, you see that on the big machine, we would have been running twice as slow had we adopted that, quote, "optimization", OK, because we had run out of parallelism, and this was making the path longer.
We needed to have a way of doing it where we could reduce the work.
Yeah, it's good to reduce the work but not as the critical path ends up getting rid of the parallels that we hope to be able to use during the runtime.
So, it's twice as slow, OK, twice as slow.
So the moral is that the work and critical path length predict the performance better than the execution time alone, OK, when you look at scalability.
And a big issue on a lot of these machines is scalability; not always, sometimes you're not worried about scalability.
Sometimes you just care.
Had we been running in the competition on a 32 processor machine, we would have accepted this optimization.
It would have been a good trade-off.
OK, but because we knew that we were running on a machine with a lot more processors, and that we were close to running out of the parallelism, it didn't make sense to be increasing the critical path at that point, because that was just reducing the parallelism of our calculation.
OK, next time, any questions about that first?
No?
OK. Next time, now that we understand the model for execution, we're going to start looking at the performance of particular algorithms what we code them up in a dynamic, multithreaded style, OK?
-- week of 6.046.
Woohoo!
The topic of this final week, among our advanced topics, is cache oblivious algorithms.
This is a particularly fun area, one dear to my heart because I've done a lot of research in this area.
This is an area co-founded by Professor Leiserson.
So, in fact, the first context in which I met Professor Leiserson was him giving a talk about cache oblivious algorithms at WADS '99 in Vancouver I think.
Yeah, that has to be an odd year.
So, I learned about cache oblivious algorithms then, started working in the area, and it's been a fun place to play.
But this topic in some sense was also developed in the context of this class.
I think there was one semester, probably also '98-'99 where all of the problem sets were about cache oblivious algorithms.
And they were, in particular, working out the research ideas at the same time.
So, it must have been fun semester.
We consider doing that this semester, but we kept it to the simple.
We know a lot more about cache oblivious algorithms by now as you might expect.
Right, I think that's all the setting.
I mean, it was kind of developed also with a bunch of MIT students in particular, M.Eng.
student, Harold Prokop.
It was his M.Eng.
thesis.
There is all the citations I will give for now.
I haven't posted yet, but there are some lecture notes that are already on my webpage.
But I will link to them from the course website that gives all the references for all the results I'll be talking about.
They've all been done in the last five years or so, in particular, starting in '99 when the first paper was published.
But I won't give the specific citations in lecture.
And, this topic is related to the topic of last week, multithreaded algorithms, although at a somewhat high level.
And then it's also dealing with parallelism in modern machines.
And we've had throughout all of these last two lectures, we've had this very simple model of a computer where we have random access.
You can access memory at a cost of one.
You can read and write a word of memory.
There is some details on how big a word can be and whatnot.
It's pretty basic, simple, flat model.
And, at the multithreaded algorithm is the idea that, well, maybe you have multiple threads of computation running at once, but you still have this very flat memory.
Everyone can access anything in memory at a constant cost.
We're going to change that model now.
And we are going to realize that a real machine, the memory of a real machine is some hierarchy.
You have some CPU, you have some cache, probably on the same chip, level 1 cache, you have some level 2 cache, if you're lucky, maybe you have some level 3 cache, before you get to main memory.
And then, you probably have a really big disk and probably there's even some cache out here, but I won't even think about that.
So, the point is, you have lots of different levels of memory and what's changing here is that things that are very close to the CPU are very fast to access.
Usually level 1 cache you can access in one clock cycle or a few.
And then, things get slower and slower.
Memory still costs like 70 ns or so to access a chunk out of.
And that's a long time.
70 ns is, of course, a very long time.
So, as we go out here, we get slower.
But we also get bigger.
I mean, if we could put everything at level 1 cache, the problem would be solved.
But what would be a flat memory.
Accessing everything in here, we assumed takes the same amount of time.
But usually, we can't afford, it's not even possible to put everything in level 1 cache.
I mean, there's a reason why there is a memory hierarchy.
Does anyone have a suggestion on what that reason might be?
It's like one of these limits in life.
Yeah?
Fast memory is expensive.
That's the practical limitations indeed, that you could try to build more and more at level 1 cache and maybe you could try to, well, yeah.
Expenses is a good reason, and practically, that's why they may be the sizes are what they are.
But suppose really fast memory were really cheap.
There is a physical limitation of what's going on, yeah?
The speed of light.
Yeah, that's a bit of a problem, right?
No matter how much, let's suppose you can only fit so many bits in an atom.
You can only fit so many bits in a particular amount of space.
If you want more bits, and you need more space, and the more space you have, the longer it's going to take for a round-trip.
So, if you assume your CPU is like this point in space, so it's relatively small and it has to get the data in, the bigger the data, the farther it has to be away.
But, you can have these cores around the CPU that are, we don't usually live in 3-D, and chips were usually in 2-D, but never mind.
You can have the sphere that's closer to the CPU that's a lot faster to access.
And as you get further away it costs more.
And that's essentially what this model is representing, although it's a bit approximated from the intrinsic physics and geometry and whatnot.
But that's the idea.
The latency, the round-trip time to get some of this memory has to be big.
In general, the costs to access memory is made up of two things.
There's the latency, the round-trip time, which in particular is limited by the speed of light.
And, plus the round-trip time, you also have to get the data out.
And depending on how much data you want, it could take longer.
OK, so there's something.
There could be, get this right, let's say, the amount of data divided by the bandwidth.
OK, the bandwidth is at what rate can you get the data out?
And if you look at the bandwidth of these various levels of memory, it's all pretty much the same.
If you have a well-designed computer the bandwidths should all be the same.
OK, as you can still get data off disc really, really fast, usually at about the speed of your bus, and that the bus gets the CPU hopefully as fast as everything else.
So, even though they're slower, they're really only slower in terms of latency.
And so, this part is maybe reasonable.
The bandwidth looks pretty much the same universally.
It's the latency that's going up.
So, if the latency is going up but we still get to divide by the same amount of bandwidth, what should we do to make the access cost at all these levels about the same?
This is fixed.
Let's say this is increasing, but this is still staying big.
What could we do to balance this formula?
Change the amounts.
As the latency goes up, if we increase the amount, then the amortized cost to access one element will go down.
So, this is amortization in a very simple sense.
So, this was to access a whole block, let's say, and this amount was the size of the block.
So, the amortized cost, then, to access one element is going to be the latency divided by the size of the block, the amount plus one over the bandwidth.
OK, so this is what you should implicitly be thinking in your head.
So, I'm just dividing here by the amounts because the amount is how many elements you get in one access, let's suppose.
OK, so we get this formula for the amortized cost.
The one over bandwidth is going to be good no matter what level we are on, I claim.
There's no real fundamental limitation there except it might be expensive.
And the latency week at the amortized out by the amounts, so whatever the latency is, at the latency gets bigger out here, we just get more and more stuff and then we make these two terms equal, let's say.
That would be a good way to balance things.
So what particular, disc has a really high latency.
Not only is there speed of light issues here, but there's actually the speed of the head moving on the tracks of the disk.
That takes a long time.
There's a physical motion.
Everything else here doesn't usually have physical motion.
It's just electric.
So, this is really, really slow and latency, so when you read something out of disk, you might as well read a lot of data from disc, like a megabyte or so.
It's probably even old these days.
Maybe you read multiple megabytes when you read anything from disk if you want these to be matched.
OK, there's a bit of a problem with doing that.
Any suggestions what the problem would be?
So, you have this algorithm.
And, whenever it reads something off of desk, it reads an entire megabyte of stuff around the element it asked for.
So the amortized cost to access is going to be reasonable, but that's actually sort of assuming something.
Yeah?
Right.
I'm assuming I'm ever going to use the rest of that data.
If I'm going to read 10 MB around the one element that asked for, I access A bracket I, and I get 10 million items from A around I, it would be kind of good if the algorithm actually used that data for something.
It seems reasonable.
So, this would be spatial locality.
So, we want, I mean the goal of this world in cache oblivious algorithms and cache efficient algorithms in general is you want algorithms that perform well when this is happening.
So, this is the idea of blocking.
And we want the algorithm to use all or at least most of the elements in a block, a consecutive chunk of memory.
So, this is spatial locality.
Ideally, we'd use all of them right then.
But I mean, depending on your algorithm, that's a little bit tricky.
There is another issue, though.
So, you read in your thing into, read your 10 MB into main memory, let's say, and your memory, let's say, is at least, these days you should have a 4 GB memory or something.
So, you could read and actually a lot of different blocks into main memory.
What you'd like is that you can use those blocks for as long as possible.
Maybe you don't even use them.
If you have a linear time algorithm, you're probably only going to visit each element a constant number of times.
So, this is enough.
But if your algorithm is more than linear time, you're going to be accessing elements more than once.
So, it would be a good idea not only to use all the elements of the blocks, but use them as many times as you can before you have to throw the block out.
That's temporal locality.
So ideally, you even reuse blocks as much as possible.
So, I mean, we have all these caches.
So, I didn't write this word.
Just in case I don't know how to spell it, it's not the money.
We should use those caches for something.
I mean, the fact that they store more than one block, each cache can store several blocks.
How many?
Well, we'll get to that in a second.
OK, so this is the general motivation, but at this point the model is still pretty damn ugly.
If you wanted to design an algorithm that runs well on this kind of machine directly, it's possible but pretty difficult, and essentially never done, let's say, even though this is what real machines look like.
At least in theory, and pretty much in practice, the main thing to think about is two levels at a time.
So, this is a simplification where we can say a lot more about algorithms, a simplification over this model.
So, in this model, each of these levels has different block sizes, and a different total sizes, it's a mess to deal with and design algorithms for.
If you just think about two levels, it's relatively easy.
So, we have our CPU which we assume has a constant number of registers only.
So, you know, once it has a couple of data items, you can add them and whatnot.
Then we have this really fast pipe.
So, I draw it thick to some cache.
So this is cache.
And, we have a relatively narrow pipe to some really big other storage, which I will call main memory.
So, I mean, that's the general picture.
Now, this could represent any two of these levels.
It could be between L3 cache and make memory.
That's maybe, what?
The naming corresponds to best.
Or cache could in fact be main memory, what we consider the RAM of the machine, and what's called a memory over there to be the disk.
It's whatever you care about.
And usually, if you have a program, that's what usually we assume everything fits in main memory.
Then you care about the caching behavior.
So you probably look between these two levels.
That's probably what matters the most inner program because the cost differential here is really big relative to the cost differential here.
If your data doesn't even fit it main memory, and you have to go to disk, then you really care about this level because the cost differential here is huge.
It's like six orders of magnitude, let's say.
So, in practice you may think of just two memory levels that are the most relevant.
OK, now I'm going to define some parameters.
I'm going to call them cache and make memory just for clarity because I like to think of main memory just the way it used to be.
And now all we have to worry about is this extra thing called cache.
It has some bounded size, and there's a block size.
The block size is B. and a number of blocks is M over B.
So, the total size of the cache is M. OK, main memory is also blocked into blocks of size B.
And we assume that it has essentially infinite size.
We don't care about its size in this picture.
It's whatever is big enough to hold the size of your algorithm, or data structure, or whatever.
OK, so that's the general model.
And for strange, historical reasons, which I don't want to get into, these things are called capital M and capital B.
Even though M sounds a lot like memory, it's really for cache, and don't ask.
OK, this is to preserve history.
OK, now what do we do with this model?
It seems nice, but now what do we measure about it?
What I'm going to assume is that the cache is really fast.
So the CPU can access cache essentially instantaneously.
I still have to pay for the computation that the CPU is doing, but I'm assuming cache is close enough that I don't care.
And that may memory is so big that it has to be far away, and therefore, this pipe is a problem.
I mean, what I should really draw is that pipe is still thick, but is really long.
So, the latency is high.
The bandwidth is still high.
OK, and all transfers here happened as blocks.
So, when you don't have something, so the idea is CPU asks for A of I, as for something in memory, if it's in the cache, it gets it.
That's free.
Otherwise, it has to grab the entire block containing that element from main memory, brings it into cache, maybe kicks somebody out if the cache was full, and then the CPU can use that data and keep going.
Until it accesses something else that's not in cache, then it has to grab it from main memory.
When you kick something out, you're actually writing back to memory.
That's the model.
So, we suppose the accesses to cache are free.
But we can still think about the running time of the algorithm.
I'm not going to change the definition of running time.
This would be the computation time, or the work if you want to use multithreaded lingo, computation time.
OK, so we still have time, and T of N will still mean what it did before.
This is just an extra level of refinement of understanding of what's going on.
Essentially, measuring the parallelism that we can exploit out of the memory system, that when you access something you actually get B items.
So, this is the old stuff.
Now, what I want to do is count memory transfers.
These are transfers of blocks, so I should say block memory transfers between the two levels, so, between the cache and main memory.
So, memory transfers are either reading reads or writes.
Maybe I should say that.
These are number of block reads and writes from and to the main memory.
OK, so I'm going to introduce some notation.
This is new notation, so we'll see how it works out.
MT of N I want to represent the number of memory transfers instead of just normal time of the problem of size N. Really, this is a function that depends not only on N but also on these parameters, B and M, in our model.
So, this is what it should be, MT_B,M(N), but that's obviously pretty messy, so I'm going to stick to MT of N. But this will always, because mainly I care about the growth in terms of N. well, I care about the growth in terms of all things, but the only thing I could change is N. So, most of the time I only think about, like when we are writing recurrences, only N is changing.
I can't recurse on the block size.
I can't recurse on the size of cache.
Those are given to me.
They're fixed.
OK, so we'll be changing N mainly.
But B and M always matter here.
They're not constants.
They're parameters of the model.
OK, easy enough.
This is something called the disk access model, if you like DAM models, or the external memory model, or the cache aware model.
Maybe I should mention that; this is the cache aware.
In general, you have some algorithm that runs on this kind of model, machine model.
That's a cache aware algorithm.
OK, we're not too interested in cache aware algorithms.
We've seen one, B trees.
B trees are cache aware data structure.
You assume that there is some block size, B, underlying.
Maybe you didn't see exactly this model.
In particular, it didn't really matter how big the cache was because you just wanted to know.
When I read B items, I can use all of them as much as possible and figure out where I fit among those B items, and that gives me log base B of N memory transfers instead of log N, which would be, if you just threw your favorite balanced binary search tree.
So, log base B of N is definitely better than log base 2 of N. B trees are a cache aware algorithm.
OK, what we would like to do today and next lecture is get cache oblivious algorithms.
So, there's essentially only one difference between cache aware algorithms and cache oblivious algorithms.
In cache oblivious algorithms, the algorithm doesn't know what B and M are.
So this is a bit of a subtle point, but very cool idea.
You assume that this is the model of the machine, and you care about the number of memory transfers between this cache of size M with blocking B, and main memory with blocking B.
But you don't actually know what the model is.
You don't know the other parameters of the model.
It looks like this, but you don't know the width.
You don't know the height.
Why not?
So, the analysis knows what B and M are.
We are going to write some algorithms which look just like boring old algorithms that we've seen throughout the lecture.
That's one of the nice things about this model.
Every algorithm we have seen is a cache oblivious algorithm, all right, because we didn't even know the word cache in this class until today.
So, we already have lots of algorithms to choose from.
The thing is, some of them will perform well in this model, and some of them won't.
So, we would like to design algorithms that just like our old algorithms that happened to perform well in this context, no matter what B and M are.
So, another way this is the same algorithm should work well for all values of B and M if you have a good cache oblivious algorithm.
OK, there are a few consequences to this assumption.
In a cache aware algorithm, you can explicitly say, OK, I'm blocking my memory into chunks of size B.
Here they are.
I was going to store these B elements here, these B elements here, because you know B, you can do that.
You can say, well, OK, now I want to read these B items into my cache, and then write out these ones over here.
You can explicitly maintain your cache.
With cache oblivious algorithms, you can't because you don't know what it is.
So, it's got to be all implicit.
And this is pretty much how caches work anyway except for disk.
So, it's a pretty reasonable model.
In particular, when you access an element that's not in cache, you automatically fetch the block containing that element.
And you pay one memory transfer for that if it wasn't already there.
Another bit of a catch here is, what if your cache is full?
Then you've got to kick some block out of your cache.
And then, so we need some model of which block gets kicked out because we can't control that.
We have no knowledge of what the blocks are in our algorithm.
So what we're going to assume in this model is the ideal thing, that when you fetch a new block, if your cache is full, you evict a block that will be used farthest in the future.
Sorry, the furthest.
Farthest is distance.
Furthest is time.
Furthest in the future.
OK, this would be the best possible thing to do.
It's a little bit hard to do in practice because you don't know the future generally, unless you're omniscient.
So, this is a bit of an idealized model.
But it's pretty reasonable in the sense that if you've read the reading handout number 20, this paper by Sleator and Tarjan, they introduce the idea of competitive algorithms.
So, we only talked about a small portion of that paper that moved to front heuristic for storing a list.
But, it also proves that there are strategies, and maybe you heard this in recitation.
Some people covered it; some didn't, that these are called paging strategies.
So, you want to maintain some cache of pages or blocks, and you pay whenever you have to access a block that's not in your cache.
The best thing to do is to always kick out the block that will be used farthest in the future because that way you'll use all the blocks that are in there.
This turns out to be the offline optimal strategy if you knew the future.
But, there are algorithms that are essentially constant competitive against this strategy.
I don't want to get into details because they're not exactly constant competitive.
But they are sufficiently constant competitive for the purposes of this lecture that we can assume this, not have to worry about it.
Most of the time, we don't even really use this assumption.
But there it is.
That's the cache oblivious model.
It makes things cleaner to think about just anything that should be done, will be done.
And you can simulate that with least recently used or whatever good heuristic that you want to that's competitive against the optimal.
OK, that's pretty much the cache oblivious algorithm.
Once you have the two level model, you just assume you don't know B and M. You have this automatic request in writing, and whatnot.
A little bit more to say, I guess, it may be obvious at this point, but I've been drawing everything as tables.
So, it's not really clear what the linear order is.
Linear order is just the reading order.
So, although we don't explicitly say it most of the time, a typical model is that memory is a linear array.
Everything that you ever store in your program is written in this linear array.
If you've ever programmed in Assembly or whatever, that's the model.
You have the address space, and any number between here and here, that's where you can actually, this is physical memory.
This is all you can write to.
So, it starts at zero and goes out to, let's call it infinity over here.
And, if you allocate some array, maybe it occupies some space in the middle.
Who knows?
OK, we usually don't think about that much.
What I care about now is that memory itself is blocked in this view.
So, however your stuff is stored in memory, it's blocked into clusters of length B.
So, if this is, let me call it one and be a little bit nicer.
This is B.
This is position B plus one.
This is 2B, and 2B plus one, and so on.
These are the indexes into memory.
This is how the blocking happens.
If you access something here, you get that chunk from U, round it down to the previous multiple of B, round it up to the next multiple of B.
That's what you always get.
OK, so if you think about some array that's maybe allocated here, OK, you have to keep in mind that that array may not be perfectly aligned with the blocks.
But more or less it will be so we don't care too much.
But that's a bit of a subtlety there.
OK, so that's pretty much the model.
So every algorithm we've seen, except B trees, is a cache oblivious algorithm.
And our question is, now, we know how everything runs in terms of running time.
Now we want to measure the number of memory transfers, MT of N. I want to mention one other fact or theorem.
I'll put it in brackets because I don't want to state it precisely.
But if you have an algorithm that is efficient on two levels, so in other words, what we're looking at, if we just think about the two level world and your algorithm is cache oblivious, then it is efficient on any number of levels in your memory hierarchy, say, L levels.
So, I don't want to define what efficient means.
But the intuition is, if your machine really looks like this and you have a cache oblivious algorithm, you can apply the cache oblivious analysis for all B and M. So you can analyze the number of memory transfers here, here, here, here, and here.
And if you have a good cache oblivious algorithm, the performances at all those levels has to be good.
And therefore, the whole performance is good.
Good here means asymptotically optimal up to constant factors, something like that.
OK, so I don't want to prove that, and you can read the cache oblivious papers.
That's a nice fact about cache oblivious algorithms.
If you have a cache aware algorithm that tunes to a particular value of B, and a particular value of M, you're not going to have that problem.
So, this is one nice feature of cache obliviousness.
Another nice feature is when you are coding the algorithm, you don't have to put in B and M. So, that simplifies things a bit.
So, let's do some algorithms.
Enough about models.
OK, we're going to start out with some really simple things just to get warmed up on the analysis side.
The most basic thing you can do that's good in a cache oblivious world is scanning.
So, scanning is just visiting the items in an array in order.
So, visit A_1 up to A_N in order.
For some notion of visit, this is presumably some constant time operation.
For example, suppose you want to compute the aggregate of the array.
You want to sum all the elements in the array.
So, you have one extra variable using, but you can store that in a register or whatever, so that's one simple example.
Sum the array.
OK, so here's the picture.
We have our memory.
Each of these cells represents one item, one element, log N bits, one word, whatever.
Our array is somewhere in here.
Maybe it's there.
And we go from here to here to here to here.
OK, and so on.
So, what does this cost?
What is the number of memory transfers?
We know that this is a linear time algorithm.
It takes order N time.
What does it cost in terms of memory transfers?
N over B, pretty much.
We like to say it's order N over B plus two or one in the big O.
This is a bit of worry.
I mean, N could be smaller than B.
We really want to think about all the cases, especially because usually you're not doing this on something of size N. You're doing it on something of size k, where we don't really know much about k. But in general, it's N over B plus one because we always need at least one memory transfer to look at something, unless N is zero.
And in particular, it's plus two if you care about the constants.
If I don't write the big O, then it would be plus two at most because you could essentially waste the first block and that everything is fine for awhile.
And then, if you're unlucky, you essentially waste the last blocked.
There is just one element in that block, and you're not getting much out of it.
Everything in the middle, though, every block between the first and last block has to be full.
So, you're using all of those elements.
So out of the N elements, you only have N over B blocks because the block has B elements.
OK, that's pretty trivial.
Let me do something slightly more interesting, which is two scans at once.
OK, here we are not assuming anything about M. we're not assuming anything about the size of the cache, just that I can hold a single block.
The last block that we visited has to be there.
OK, you can also do a constant number of parallel scans.
This is not really parallel in the sense of multithreaded, bit simulated parallelism.
I mean, if you have a constant number, do one, do the other, do the other, come back, come back, come back, all right, visit them in turn round robin, whatever.
For example, here's a cute piece of code.
If you want to reverse an array, OK, then you can do it.
This is a good puzzle.
You can do it by essentially two scans where you repeatedly swapped the first and last element.
So I was swapping A_i with N minus i plus one, and just restart at one.
So, here's your array.
Suppose this is actually my array.
I swap these two guys, and I saw these two guys, and so on.
That will reverse my array, and this should work hopefully the middle as well if it's odd.
It should not do anything.
And you can view this as two scans.
There is one scan that's coming in this way.
There's also a reverse scan, ooh, some more sophisticated, coming back this way.
Of course, reverse scan has the same analysis.
And as long as your cache is big enough to store at least two blocks, which is a pretty reasonable assumption, so let's write it.
Assuming the number of blocks in the cache, which is M over B, is at least two in this algorithm, the number of memory transfers is still order N over B plus one.
OK, the constant goes up maybe, but in this case it probably doesn't.
But who cares.
OK, as long as you're doing a constant number of scans, and some constant number of arrays, it happens to be one of them's reversed, whatever, it will take, we call this linear time.
It's linear in the number of blocks in your input.
OK, great.
So now you can reverse an array: exciting.
Let's try another simple algorithm on another board.
Let's try binary search.
So just like last week, we're going back to our basics here.
Scanning we didn't even talk about in this class.
Binary search is something we talked about a little bit.
It was a simple divide and conquer algorithm.
I hope you all remember it.
And if we look at an array, and I'm not going to draw the cells here because I want to imagine a really big array, binary search, but suppose it always goes to left.
It starts by visiting this element in the middle.
Then ago so the quarter marked.
Then it goes to the one eighth mark.
OK, this is one hypothetical execution of a binary search.
OK, and eventually it finds the element it's looking for.
It finds where it fits at least.
So x is over here.
So, we know that it takes log N time.
How many memory transfers of the take?
Now, I blocked this array into chunks of size B, blocks of size B.
How many blocks do I touch?
This one's a little bit more subtle.
It depends on the relative sizes of N and B, yeah.
Log base B of N would be a good guess.
We would like it to be, let's say, hope, is that it's log base B of N because we know that B trees can search in what's essentially a sorted list of N items in log base B of N time.
That turns out to be optimal in the cache oblivious model or in the two level model you've got to pay log base B of N. I won't prove that here.
The same reason you need log N comparisons to do a binary search in the normal model.
Alas, it is possible to get log base B of N even without knowing B.
But, binary search does not do it.
Log of N over B, yes.
So the number of memory transfers on N items is log of N over B also known as, let's say, plus one, also known as log N minus log B. OK, whereas log base B of N is log N divided by log B, OK, clearly this is much better than subtracting.
So, this would be good, but this is bad.
Most of the time, this is log N, which is no better, I mean, you're not using blocks at all essentially.
The idea is, out here, I mean, there's some little, tiny block that contains this thing.
I mean, tiny depends on how big B is.
But, each of these items will be in a different block until you get essentially within one block worth of x.
When you get within one block worth of x, there's only like a constant number of blocks that matter, and so all of these accesses are indeed within the same block.
But, how many are there?
Well, just log B because you're only spending log B within a, if you're within an interval of size k, you're only going to spend log k steps in it.
So, you're saving log B in here, but overall you're paying log N, so you only get log N minus log B plus some constant.
OK, so this is bad news for binary search.
So, not all of the algorithms we've seen are going to work well in this model.
We need a lot more thinking before we can solve what is essentially the binary search problem, finding an element in a sorted list, in log base B of N without knowing B. OK, we know we could use B trees.
If you knew B, great, that works, and that's optimal.
But without knowing B, it's a little bit harder.
And this gets us into the world of divide and conquer.
Also like last week, and like the first few weeks of this class, divide and conquer is your friend.
And, it turns out divide and conquer is not the only tool, but it's a really useful tool in designing cache oblivious algorithms.
And, let me say why.
So, we'll see a bunch of divide and conquer based algorithms, cache oblivious.
And, the intuition is that we can take all the favorite algorithms we have, obviously it doesn't always work.
Binary search was a divide and conquer algorithm.
It's not so great.
But, in general, the idea is that your algorithm can just do the normal divide and conquer thing, right?
You divide your problem into subproblems of smaller size repeatedly, all the way down to problems of constant size, OK, just like before.
But, if you're recursively dividing your problem into smaller things, at some point you can think about it and say, well, wait, I mean, the algorithm divides all the way, but we can think about the point at which the problem fits in a block or fits in cache.
OK, and that's the analysis.
OK, we'll think about the time when your problem is small enough that we can analyze it in some other way.
So, usually, we analyze it recursively.
We get a recurrence.
What we're changing, essentially, is the base case.
So, in the base case, we don't want to go down to a constant size.
That's too far.
I'll show you some examples.
We want to consider the point in recursion at which either the problem fits in cache, so it has size less than or equal to M, or it fits in order one blocks.
That's another natural time to do it.
Order one blocks would be even better than fitting in cache.
So, this means a size order B. OK, this will change the base case of the recurrence, and it will turn out to give us good answers instead of bad ones.
So, let's do a simple example.
Our good friend order statistics, in particular, for finding medians.
So, I hope you all know this by heart.
Remember the worst case linear time, median finding algorithm by Bloom et al.
I'll write this fast.
We partition our array.
It turns out, this is a good algorithm as it is.
We partition our array conceptually into N over five, five tuples into little groups of five.
This may not have been exactly how I wrote it last time.
I didn't check.
But, it's the same algorithm.
You compute the median of each five tuple.
Then you recursively compute the median of the medians of these medians.
Then, you partition around x.
So, that gave us some element that was roughly in the middle.
It was within the middle half, I think.
Partition around x, and then we show that you could always recurse on just one of the sides.
OK, this was our good old friend for computing, order statistics, or medians, or whatnot.
OK, so how much time does this, well, we know how much time this takes.
It should be linear time.
But how many memory transfers does this take?
Well, conceptually partitioning that, I can do, in zero.
Maybe I have to compute N over five, no big deal here.
We're not thinking about computation.
I have to find the median of each tuple.
So, here it matters how my array is laid out.
But, what I'm going to do is take my array, take the first five elements, and then the next five elements and so on.
Those will be my five tuples.
So, I can implement this just by scanning, and then computing the median on those five elements, which I stored in the five registers on my CPU.
I'll assume that there are enough registers for that.
And, I compute the median, write it out to some array out here.
So, it's going to be one element.
So, the median of here goes into there.
The median of these guys goes into there, and so on.
So, I'm scanning in here, and in parallel, I'm scanning an output in here.
So, it's two parallel scans.
So, that takes linear time.
So, this takes order N over B plus one memory transfers.
OK, then we have recursively compute the median of the medians.
This step used to be T of N over five.
Now it's MT of N over five, OK, with the same values of B and M. Then we partition around x. Partitioning is also like three parallel scans if you work it out.
So, this is also going to take linear memory transfers, N over B plus one.
And then, we recurse on one of the sides, and this is the fun part of the analysis which I won't repeat here.
But, we get MT of, like, three quarters N. I think originally it was seven tenths, so we simplified to three quarters, which is hopefully bigger than seven tenths.
Yeah, it is.
OK, so this is the new analysis.
Now we get a recurrence.
So, let's do that.
So, the analysis is we get this MT of N is MT of N over five plus MT of three quarters N plus, this is just as before.
Before we had linear work here.
And now, we have what we call linear number of memory transfers, linear number of blocks.
OK, I'll sort of ignore this plus one.
It's not too critical.
So, this is our recurrence.
Now, it depends what our base case is.
And, usually we would use a base case of constant size.
So, let's see what happens if we use a base case of constant size just so that it's clear why this base case is so important.
OK, this describes a recurrence as one of these hairy recurrences.
And, I don't want to use substitution.
I just want the intuition of why this is going to solve to something rather big.
OK, and for me, the best intuition always comes from recursion trees.
If you don't know the solution to recurrence and you need a good guess, use recursion trees.
And today, I will only give you good guesses.
I don't want to prove anything with substitution because I want to get to the bigger ideas.
So, this is even messy from a recursion tree point of view because you have these unbalanced sizes where you start at the root with some of size N over B.
Then you split it into something size one fifth N over B, and something of size three quarters N over B, which is annoying because now this subtree will be a lot bigger than this one, or this one will terminate faster.
So, it's pretty unbalanced.
But, summing per level doesn't really tell you a lot at this point.
But let's just look at the bottom level.
Look at all the leaves in this recursion tree.
So, that's the base cases.
How many base cases are there?
This is an interesting question.
We've never thought about it in the context of this recurrence.
It gives a somewhat surprising answer.
It was surprising to me the first time I worked it out.
So, how many leaves does this recursion tree have?
Well, we can write a recurrence.
The number of leaves in a problem of size N, it's going to be the number of leaves in this problem plus the number of leaves in this problem plus zero.
So, that's another recurrence.
We'll call this L of N. OK, now the base case is really relevant.
It determines the solution to this recurrence.
And let's, again, assume that in a problem of size one, we have one leaf.
That's our only base case.
Well, it turns out, and here you need to guess, I think.
This is not particularly obvious.
Any of the TA's have guesses of the form of this solution?
Or anybody, not just TA's.
But this is open to everyone.
If Charles were here, I would ask him.
I had to think for a while, and it's not linear, right, because you're somehow decreasing quite a bit.
So, it's smaller than linear, but it's more than a constant.
OK, it's actually more than polylog, so what's your favorite function in the middle?
N over log N, that's still too big.
Keep going.
You have an oracle here, so you can, N to the k, yeah, close.
I mean, k is usually an integer.
N to the alpha for some real number between zero and one.
Yeah, that's what you meant.
Sorry.
It's like the shortest mathematical joke.
Let epsilon be less than zero or for a sufficiently large epsilon.
I don't know.
So, you've got to use the right letters.
So, let's suppose that it's N to the alpha.
Then we would get this N over five to the alpha, and we'd get three quarters N to the alpha.
When you have a nice recurrence like this, you can just try plugging in a guess and see whether it works, OK, and of course this will work only depending on alpha.
So, we should get an equation on alpha here.
So, everything has an N to the alpha, in fact, all of these terms.
So, I can divide through my N to the alpha.
That's assuming that it's not zero or something.
That seems reasonable.
So, we have one equals one fifth to the alpha plus three quarters to the alpha.
This is something you won't get on a final because I don't know any good way to solve this except with like Maple or Mathematica.
If you're smart I'm sure you could compute it in a nicer way, but alpha is about 0.8, it turns out.
So, the number of leaves is this sort of in between constant and linear.
Usually polynomial means you have an integer power.
Let's call it a polynomial.
Why not?
There's a lot of leaves, is the point, and if we say that each leaf costs a constant number of memory transfers, we're in trouble because then the number of memory transfers has to be at least this.
If it's at least that, that's potentially bigger than N over B, I mean, bigger than in an asymptotic sense.
This is little omega of N over B if B is big.
If B is at least N to the 0.2 something, OK, or one seventh something.
But if, in particular, B is at least N to the 0.2, then this should be bigger than that.
So, this is a bad analysis because we're not going to get the answer we want, which is N over B.
The best you can do for median is N over B because you have to read all the element, and you should spend linear time.
So, we want to get N over B.
This algorithm is N over B plus one.
So, this is why you need a good base case, all right?
So that makes the point.
So, the question is, what base case should I use?
So, we have this recurrence What base case should I use?
Constant was too small.
We have a couple of choices listed up here.
Any suggestions?
B, OK, MT of B is?
The hard part.
So, if my problem, if the size of my array fits in a block and I do all this stuff on it, how many memory transfers could that take?
One, or a constant, depending on alignment.
OK, maybe it takes two memory transfers, but constant.
Good.
That's clearly a lot better than this base case, MT of one equals order one, clearly stronger.
So, hopefully, it gives the right answer, and now indeed it does.
I love this analysis.
So, I'm going to wave my hands.
OK, but in particular, what this gives us, if we do the previous analysis, what is the number of leaves?
So, in the leaves, now L of B equals one instead of L of one equals one.
So, this stops earlier.
When does it stop?
Well, instead of getting N to the order of 0.8, whatever, we get N over B to the power of 0.8 whatever.
OK, so it turns out the number of leaves is N over B to the alpha, which is little o of N over B.
So, we don't care.
It's tiny.
And, if you look at the root cost is N over B in the recursion tree, the leaf cost is little o of N over B, and if you wave your hands, and close your eyes, and squint, the cost should be geometrically decreasing as we go down, I hope, more or less.
It's a bit messy because of all the things terminating, but let's say cost is roughly geometric.
Don't do this in the final, but you won't have any messy recurrences like this.
So, don't worry.
Down the tree, so you'd have to prove this formally, but I claim that the root cost dominates.
And, the root cost is N over B.
So, we get N over B. OK, so this is a nice, linear time algorithm for order statistics for cache oblivious.
Great.
This may turn you off a little bit, but even though this is like the simplest algorithm, it's also probably the most complicated analysis that we will do.
In the future, our algorithms will be more complicated, and the analyses will be relatively simple.
And usually, it's that way with cache oblivious algorithms.
So, I'm giving you this sort of as the intuition of why this should be enough.
Then you have to prove it.
OK, let's go to another problem where divide and conquer is useful, our good friend, matrix multiplication.
I don't know how many times we've seen this in this class, but in particular we saw it last week with a recursive matrix multiply, multithreaded algorithm.
So, I won't give you the algorithm yet again, but we're going to analyze it in a very different way.
So, we have C and we have A, and actually up to you.
So, I could cover standard matrix multiplication, which is when you do it row by row, and column by column.
And, we could see why that's bad.
And then, we could do the recursive one and see why that's good.
Or, we could skip the standard algorithm.
So, how many people would like to see why the standard algorithm is bad?
Because it's not totally obvious.
One, two, three, four, five, half?
Wow, that's a lot of votes.
Now, how many people want to skip to the chase?
No one.
One, OK. And, everyone else is asleep.
So, that's pretty good, 50% awake, not bad.
OK, then, so standard matrix multiplication.
I'll do this fast because it is, I mean, you all know the algorithm, right?
To compute this value of C; in A, you take this row, and in B you take this column.
Sorry I did a little bit sloppily.
But this is supposed to be aligned.
Right?
So I take all of this stuff, I multiply it with all of the stuff, add them up, the dot product.
That gives me this element.
And, let's say I do them in this order row by row.
So for every item in C, I loop over this row and this column, B, multiply them together.
That is an access pattern in memory.
So, exactly how much that costs depends how these matrices are laid out in memory.
OK, this is a subtlety we haven't had to worry about before because everything was uniform.
I'm going to assume to give the standard algorithm the best chances of being good, I'm going to store C in row major order, A in row major order, and B in column major order.
So, everything is nice and you're scanning.
So then this inner product is a scan.
Cool.
Sounds great, doesn't it?
It's bad, though.
Assume A is row major, and B is column major.
And C, you could assume is really either way, but if I'm doing it row by row, I'll assume it's row major.
So, this is what I call the layout, the memory layout, of these matrices.
OK, it's good for this algorithm, but the algorithm is not good.
So, it won't be that great.
So, how long does this take?
How many memory transfers?
We know it takes M^3 time.
Not going to try and beat M^3 here.
Just going to try and get standard matrix multiplication going faster.
So, well, for each item over here I pay N over B to do the scans and get the inner product.
So, N over B per item.
So, it's N over B, or we could go with the plus one here, to compute each c_ij.
So that would suggest, as an upper bound at least, it's N^3 over B. OK, and indeed that is the right bound, so theta.
This is memory transfers, not time, obviously.
That is indeed the case because if you look at consecutive, I do this c_ij, then this one, this one, this one, this one, keep incrementing j and keeping I fixed, right?
So, the row that I use stays fixed for a long time.
I get to reuse that if it happens, say that that fits a block maybe, I get to reuse that row several times if that happens to fit in cache.
But the column is changing every single time.
OK, so every time I moved here and compute the next c_ij, even if a column could fit in cache, I can't fit all the columns in cache.
And the columns that I'm visiting move, you know, they just scan across.
So, I'm scanning this whole matrix every time.
And unless you're entire matrix fits in cache, in which case you could do anything, I don't care, it will take constant time, or you'll take M over B time, enough to read it into the cache, do your stuff, and write it back out.
Except in that boring case, you're going to have to pay N^2 over B for every row here because you have to scan the whole collection of columns.
You have to read this entire matrix for every row over here.
So, you really do need N^3 over B for the whole thing.
So, it's usually a theta.
So, you might say, well, that's great.
It's the size of my problem, the usual running time, divided by B.
And that was the case when we are thinking about linear time, N versus N over B.
It's hard to beat N over B when your problem is of size N. But now we have a cubed.
And, this gets back to, we have good spatial locality.
When we read a block, we use the whole thing.
Great.
It seems optimal.
But we don't have good temporal locality.
It could be that maybe if we stored the right things, we kept them around, we could them several times because we're using each element like a cubed number of times.
That's not the right way of saying it, but we're reusing the matrices a lot, reusing those items.
If we are doing N^3 work on N^2 things, we're reusing a lot.
So, we want to do better than this, and that's the recursive algorithm, which we've seen.
So, we know the algorithm pretty much.
I just have to tell you what the layout is.
So, we're going to take C, partition of C_1-1, C_1-2, and so on.
So, I have an N by N matrix, and I'm partitioning into N over 2 by N over 2 submatrices, all three of them times whatever.
And, I could write this out yet again but I won't.
OK, we can recursively compute this thing with eight matrix multiplies, and a bunch of matrix additions.
I don't care how many, but a constant number.
We see that at least twice now, so I won't show it again.
Now, how do I lay out the matrices?
Any suggestions how I lay out the matrices?
I could lay them out in row major order.
I'll call it major order.
But that might be less natural now.
We're not doing anything by rows or by columns.
So, what layout should I use?
Yeah?
Quartet major order, maybe quadrant major order unless you're musically inclined, yeah.
Good idea.
You've never seen this order before, so it's maybe not so natural.
Somehow I want to cluster it by blocks.
OK, I think that's about all.
So, I mean, it's a recursive layout.
This was not an easy question.
It's OK. Store matrices or lay out the matrices recursively by block.
OK, I'm cheating a little bit.
I'm redefining the problem to say, assume that your matrices are laid out in this way.
But, it doesn't really matter.
We can cheat, can't we?
In fact, it doesn't matter.
You can turn a matrix into this layout without too much linear work, almost linear work.
Log factors, maybe.
OK, so if I want to store my matrix A as a linear thing, I'm going to recursively defined that layout to be recursively store the upper left corner, then store, let's say, the upper right corner.
It doesn't matter which order I do these.
I should have drawn this wider, then store the lower left corner, and then store the lower right corner recursively.
So, how do you store this?
Well, you divide it in four, and lay out the top left, and so on.
OK, this is a recursive definition of how the element should be stored in a linear array.
It's a weird one, but this is a very powerful idea in cache oblivious algorithms.
We'll use this multiple times.
OK, so now all we have to do is analyze the number of memory transfers.
How hard could it be?
So, we're going to store all the matrices in this order, and we want to compute the number of memory transfers on an N by N matrix.
See, I lapsed and I switched to lowercase n. I should, throughout this week, be using uppercase N because for historical reasons, any external memory kinds of algorithms, to level algorithms, always talk about capital N. And, don't ask why.
You should see what they define little n to be.
OK, so, any suggestions on what the recurrence should be now?
All his fancy setup with the recurrence is actually pretty easy.
So, definitely it involves multiplying matrices that are N over 2 by N over 2.
So, what goes here?
Eight, thank you.
That you should know.
And that the tricky part is what goes here.
OK, what goes here is, now, the fact that I can even write this, this is the matrix additions.
Ignore those for now.
Suppose there weren't any.
I just have to recursively multiply.
The fact that this actually is eight times memory transfers of N over 2 relies on this layout.
Right, I'm assuming that the arrays that I'm given are given as contiguous intervals and memory.
If they aren't, I mean, if they're scattered all over memory, I'm screwed.
There's nothing I can do.
So, but by assuming that I have this recursive layout, I know that the recursive multiplies will always deal with three consecutive chunks of memory, one for A, one for B, one for C, OK, no matter what I do.
Because these are stored consecutively, recursively I have that invariant.
And I can keep recursing.
And I'm always dealing with three consecutive chunks of memory.
That's why I need this layout is to be able to say this.
OK, Now what does addition cost?
I'll just give you two matrices.
They're stored in some linear order, the same linear order among the three of them.
Do I care what the linear order is?
How should I add two matrices, get the output?
Yeah?
Right, if each of the three arrays I'm dealing with are stored in the same order, I can just scan in parallel through all three of them and just add corresponding elements, and output it to the third.
So, I don't care what the order is, as long as it's consistent and I get N^2 over B. I'll ignore plus one here.
That's just looking at the entire matrix.
So, there we go: another recurrence.
We've seen this with N^2, and we just got N^3.
But, it turns out now we get something cooler if we use the right base case.
So now we get to the base case, ah, the tricky part.
So, any suggestions what base case I should use?
The block size, good suggestion.
So, if we have something of size order B, we know that takes a constant number of memory transfers.
It turns out that's not enough.
That won't solve it here.
But good guess.
In this case, it's not the right answer.
I'll give you some intuition why.
We are trying to improve on N^3 over B.
If you were just trying to get it divided by B, this is a great base case.
But here, we know that just the improvement afforded by the block size is not enough.
We have to somehow use the fact that the cache is big.
It's M, so however big M is, it's that big.
OK, so if we want to get some improvement on this, we've got to have M in the formula somewhere, and there's no M's yet.
So, it's got to involve M. What's that?
MT of M over B?
That would work, but MT of M is also OK, I mean, some constant times M, let's say.
I want to make this constant small enough so that the entire problem fits in cache.
So, it's like one third.
I think it's actually, oh wait, is it the square root of M actually?
Right, this is an N by N matrix.
So, it should be C times the square root of M. Sorry.
So, the square root of M by square root of M matrix has M entries.
If I make C like one third or something, then I can fit all three matrices in memory.
Actually, one over square root of three would do, but who cares?
So, for some constant, C, now everything fits in memory.
How many memory transfers does it take?
One?
It's a bit too small, because I do have to read the problem in.
And now, I mean, here was one because there's only one block to read.
Now how many blocks are there to read?
Constants?
No.
B?
No.
M over B, good.
Get it right eventually.
That's the great thing about thinking with an oracle.
You can just keep guessing.
M over B because we have cache size M. There are M over B blocks in that cache to read each one, OK?
This is maybe, you forgot what M was because we haven't used it for a long time.
But M is the number of elements in cache.
This is the number of blocks in cache.
OK, some of was saying B, and it's reasonable to assume that M over B is about B.
That's like a square cache, but in general, we don't make that assumption.
OK, where are we?
We're hopefully done, just about, good, because we have three minutes.
So, that's our base case.
I have a square root here; I just forgot it.
Now we just have to solve it.
Now, this is an easier recurrence, right?
I don't want to use the master method, because master method is not going to handle these B's and M's, and these crazy base cases.
OK, master method would prove N^3.
Great.
Master method doesn't really think about these kinds of cases.
But with regression trees, if you remember way back to the proof of the master method, just look at the recursion tree as geometric up or down where everything is equal, and then you just add them up, every level.
The point is that this is a nice recurrence.
All of the sub problems are the same size, and that analysis always works, I say, when everything has the same size, all the children.
So, here's the recursion tree.
We have N^2 over B at the top.
We split into eight subproblems where each one, the cost is one half N^2 over B. I'm not going to write them all.
There they are.
You add them up.
How much do you get?
Well, there's eight of them.
Eight times a half is two.
Four.
[LAUGHTER] Thanks.
Four, right?
OK, I'm bad at arithmetic.
I probably already said it, but there are three kinds of mathematicians, those who can add, and those who can't.
OK, why am I looking at this?
It's obvious.
OK, so we keep going.
This looks geometrically increasing.
Right?
You just know in your heart that if you work out the first two levels, you can tell whether it's geometrically increasing, decreasing, or they're all equal, or something else.
And then you better think.
But I see this as geometrically increasing.
It will indeed be like 16 at the next level, I guess.
OK, it should be.
So, it's increasing.
That means the leaves matter.
So, let's work out the leaves.
And, this is where we use our base case.
So, we have a problem of size square root of M. And so, yeah, you have a question?
Oh, indeed.
I knew there was something.
I knew it was supposed to be two out here.
Thanks.
This is why you're here.
It's actually N over two squared over B.
Thanks.
I'm substituting N over 2 into this.
OK, so this is actually N^2 over 4 B.
So, I get two, because there are eight times one over four.
OK, I wasn't that far off then.
It's still geometrically increasing, still the case, OK?
But now, it actually doesn't matter.
Whatever the cost is, as long as it's bigger than one, great.
Now we look at the leaves.
The leaves are root M by root M. I substitute root M into this: I get M over B with some constants.
Who cares?
So, each leaf is M over B, OK, lots of them.
How many are there?
This is the only, deal with recursion trees, counting the number of leaves is always the annoying part.
Oh boy, well, we start with an N by N matrix.
We stop when we get down to root N by root N matrix.
So, that sounds like something.
Oh boy, I'm cheating here.
Really?
That many?
It sounds plausible.
OK, the claim is, and I'll cheat.
So I'm going to use the oracle here, and we'll figure out why this is the case.
N over root N^3 leaves, hey what?
I think here, it's hard to see the tree.
But it's easy to see in the matrix.
Let's enter the matrix.
We have our big matrix.
We divided in half.
We recursively divide in half.
We recursively divide in half.
You get the idea, OK?
Now, at some point these sectors, let's say one of these sectors, and each of these sectors, fits in cache.
And three of them fit in cache.
So, that's when we stop the recursion in the analysis.
The algorithm goes all the way.
But in the analysis, let's say we stop at M. OK, now, how many leaves or problems are there?
Oh man, this is still not obvious.
OK, the number of leaf chunks here is, like, I mean, the number of these things is something like N over root M, right, the number of chunks.
But, it's a little less clear because I have so many of these.
But, all right, so let's just suppose, now, I think of normal, boring, matrix multiplication on chunks of this size.
That's essentially what the leaves should tell me.
I start with this big problem, I recurse out to all these little, tiny, multiply this by that, OK, this root M by root M chunk.
OK, how many operations, how many multiplies do I do on those things?
N^3.
But now, N, the size of my matrix in terms of these little sub matrices, is N over root M. So, it should be N over root M^3 subproblems of this size.
If you work it out, normally we go down to things of constant size and we get exactly N^3 of them.
Now we are stopping at this short point in saying, well, it's however many there are, cubed.
OK, this is a bit of hand waving.
You could work it out with the recurrence on the number of leaves.
But there it is.
So, the total here is N over, let's work it out.
N^3 over M to the three halves, that's this number of leaves, times the cost at each leaf, which is M over B.
So, some of the N's cancel, and we get N^3 over B root M, which is a root M factor better than N^3 over B.
It's actually quite a lot, the square root of the cache size.
That is optimal.
The best two level matrix multiplication algorithm is N^3 over B root M memory transfers.
Pretty amazing, and I'm over time.
You can generalize this into all sorts of great things, but the bottom line is this is a great way to do matrix multiplication as a recursion.
We'll see more recursion for cache oblivious algorithms on Wednesday.
The last lecture of 6.046.
We are here today to talk more about cache oblivious algorithms.
Last class, we saw several cache oblivious algorithms, although none of them quite too difficult.
Today we will see two difficult cache oblivious algorithms, a little bit more advanced.
I figure we should do something advanced for the last class just to get to some exciting climax.
So without further ado, let's get started.
Last time, we looked at the binary search problem.
Or, we looked at binary search, rather.
And so, the binary search did not do so well in the cache oblivious context.
And, some people asked me after class, is it possible to do binary search while cache obliviously?
And, indeed it is with something called static search trees.
So, this is really binary search.
So, I mean, the abstract problem is I give you N items, say presorted, build some static data structure so that you can search among those N items quickly.
And quickly, I claim, means log base B of N. We know that with B trees, our goal is to get log base B of N. We know that we can achieve that with B trees when we know B.
We'd like to do that when we don't know B.
And that's what cache oblivious static search trees achieve.
So here's what we're going to do.
As you might suspect, we're going to use a tree.
So, we're going to store our N elements in a complete binary tree.
We can't use B trees because we don't know what B is.
So, we'll use a binary tree.
And the key is how we lay out a binary tree.
The binary tree will have N nodes.
Or, you can put the data in the leaves.
It doesn't really matter.
So, here's our tree.
There are the N nodes.
And we're storing them, I didn't say, in order, you know, in the usual way, in order in a binary tree, which makes it a binary search tree.
So we now had a search in this thing.
So, the search will just start at the root and a walk down some root-to-leaf path.
OK, and each point you know whether to go left or to go right because things are in order.
So we're assuming here that we have an ordered universe of keys.
So that's easy.
We know that that will take log N time.
The question is how many memory transfers?
We'd like a lot of the nodes near the root to be somehow closer and one block.
But we don't know what the block size is.
So are going to do is carve the tree in the middle level.
We're going to use divide and conquer for our layout of the tree, how we order the nodes in memory.
And the divide and conquer is based on cutting in the middle, which is a bit weird.
It's not our usual divide and conquer.
And we'll see this more than once today.
So, when you cut on the middle level, if the height of your original tree is log N, maybe log N plus one or something, it's roughly log N, then the top half will be log N over two.
And at the height of the bottom pieces will be log N over two.
How many nodes will there be in the top tree?
N over two?
Not quite.
Two to the log N over two, square root of N. OK, so it would be about root N nodes over here.
And therefore, there will be about root N subtrees down here, one for each, or a couple for each leaf.
OK, so we have these subtrees of root N, and there are about root N of them.
OK, this is how we are carving our tree.
Now, we're going to recurse on each of the pieces.
I'd like to redraw this slightly, sorry, just to make it a little bit clearer.
These triangles are really trees, and they are connected by edges to this tree up here.
So what we are really doing is carving in the middle level of edges in the tree.
And if N is not exactly a power of two, you have to round your level by taking floors or ceilings.
But you cut roughly in the middle level of edges.
There is a lot of edges here.
You conceptually slice there.
That gives you a top tree and the bottom tree, several bottom trees, each of size roughly root N. OK, and then we are going to recursively layout these root N plus one subtrees, and then concatenate.
So, this is the idea of the recursive layout.
We sought recursive layouts with matrices last time.
This is doing the same thing for a tree.
So, I want to recursively layout the top tree.
So here's the top tree.
And I imagine it being somehow squashed down into a linear array recursively.
And then I do the same thing for each of the bottom trees.
So here are all the bottom trees.
And I squashed each of them down into some linear order.
And then I concatenate those linear orders.
That's the linear order of the street.
And you need a base case.
And the base case, just a single node, is stored in the only order of a single node there is.
OK, so that's a recursive layout of a binary search tree.
It turns out this works really well.
And let's quickly do a little example just so it's completely clear what this layout is because it's a bit bizarre maybe the first time you see it.
So let me draw my favorite picture.
So here's a tree of height four or three depending on how you count.
We divide in the middle level, and we say, OK, that's the top tree.
And then these are the bottom trees.
So there's four bottom trees.
So there are four children hanging off the root tree.
They each have the same size in this case.
They should all roughly be the same size.
And the first we layout the top thing where we divide on the middle level.
We say, OK, this comes first.
And then, the bottom subtrees come next, two and three.
So, I'm writing down the order in which these nodes are stored in an array.
And then, we visit this tree so we get four, five, six.
And then we visit this one so we get seven, eight, nine.
And then the subtree, 10, 11, 12, and then the last subtree.
So that's the order in which you store these 15 nodes.
And you can build that up recursively.
OK, so the structure is fairly simple, just a binary structure which we know and love, but store it in this funny order.
This is not depth research order or level order, lots of natural things you might try, none of which work in cache oblivious context.
This is pretty much the only thing that works.
And the intuition as, well, we are trying to mimic all kinds of B trees.
So, if you want a binary tree, well, that's the original tree.
It doesn't matter how you store things.
If you want a tree where the branching factor is four, well, then here it is.
These blocks give you a branching factor of four.
If we had more leaves down here, there would be four children hanging off of that node.
And these are all clustered together consecutively in memory.
So, if your block size happens to be three, then this is a perfect way to store things for a block size of three.
If you're block size happens to be probably 15, right, if we count the number of, right, the number of nodes in here is 15, if you're block size happens to be 15, then this recursion will give you a perfect blocking in terms of 15.
And in general, it's actually mimicking block sizes of 2^K-1.
Think powers of two.
OK, that's the intuition.
Let me give you the formal analysis to make it clearer.
So, we claim that there are order, log base B of N memory transfers.
That's what we want to prove no matter what B is.
So here's what we're going to do.
You may recall last time when we analyzed divide and conquer algorithms, we wrote our recurrence, and that the base case was the key.
Here, in fact, we are only going to think about the base case in a certain sense.
We don't have, really, recursion in the algorithm.
The algorithm is just walking down some root-to-leaf path.
We only have a recursion in a definition of the layout.
So, we can be a little bit more flexible.
We don't have to look at our recurrence.
We are just going to think about the base case.
I want to imagine, you start with the big triangle.
That you cut it in the middle; you get smaller triangles.
Imagine the point at which you keep recursively cutting.
So imagine this process.
Big triangles halve in height each time.
They're getting smaller and smaller, stop cutting at the point where a triangle fits in a block.
OK, and look at that time.
OK, the recursion actually goes all the way, but in the analysis let's think about the point where the chunk fits in a block in one of these triangles, one of these boxes fits in a block.
So, I'm going to call this a recursive level.
So, I'm imagining expanding all of the recursions in parallel.
This is some level of detail, some level of refinement of the trees at which the tree you're looking at, the triangle, has size.
In other words, there is a number of nodes in that triangle is less than or equal to B. OK, so let me draw a picture.
So, I want to draw sort of this picture but where instead of nodes, I have little triangles of size, at most, B.
So, the picture looks something like this.
We have a little triangle of size, at most, B.
It has a bunch of children which are subtrees of size, at most, B, the same size.
And then, these are in a chunk, and then we have other chunks that look like that in recursion potentially.
OK, so I haven't drawn everything.
There would be a whole bunch of, between B and B^2, in fact, subtrees, other squares of this size.
So here, I had to refine the entire tree here.
And then I refined each of the subtrees here and here at these levels.
And then it turned out after these two recursive levels, everything fits in a block.
Everything has the same size, so at some point they will all fit within a block.
And they might actually be quite a bit smaller than the block.
How small?
So, what I'm doing is cutting the number of levels and half at each point.
And I stop when the height of one of these trees is essentially at most log B because that's when the number of nodes at there will be B roughly.
So, how small can the height be?
I keep dividing at half and stopping when it's, at most, log B. Log B over two.
So it's, at most, log B, it's at least half log B.
Therefore, the number of nodes it here could be between the square root of B and B.
So, this could be a lot smaller and less than a constant factor of a block, a claim that doesn't matter.
It's OK.
This could be a small square root of B. I'm not even going to write that it could be a small square root of B because that doesn't play a role in the analysis.
It's a worry, but it's OK essentially because our bound only involves log B.
It doesn't involve B.
So, here's what we do.
We know that each of the height of one of these triangles of size, at most, B is at least a half log B.
And therefore, if you look at a search path, so, when we do a search in this tree, we're going to start up here.
And I'm going to mess up the diagram now.
We're going to follow some path, maybe I should have drawn it going down here.
We visit through some of these triangles, but it's a root-to-node path in the tree.
So, how many of the triangles could it visit?
Well, the height of the tree divided by the height of one of the triangles.
So, this visits, at most, log N over half log B triangles, which looks good.
This is log base B of N, mind you off factor of two.
Now, what we worry about is how many blocks does a triangle occupy?
One of these triangles should fit in a block.
We know by the recursive layout, it is stored in a consecutive region in memory.
So, how many blocks could occupy?
Two, because of alignment, it might fall across the boundary of a block, but at most, one boundary.
So, it fits in two blocks.
So, each triangle fits in one block, but is in, at most, two blocks, memory blocks, size B depending on alignment.
So, the number of memory transfers, in other words, a number of blocks we read, because all we are doing here is reading in a search, is at most two blocks per triangle.
There are this many triangles, so it's at most, 4 log base B of N, OK, which is order log base B of N. And there are papers about decreasing this constant 4 with more sophisticated data structures.
You can get it down to a little bit less than two I think.
So, there you go.
So not quite as good as B trees in terms of the constant, but pretty good.
And what's good is that this data structure works for all B at the same time.
This analysis works for all B.
So, we have a multilevel memory hierarchy, no problem.
Any questions about this data structure?
This is already pretty sophisticated, but we are going to get even more sophisticated.
Next, OK, good, no questions.
This is either perfectly clear, or a little bit difficult, or both.
So, now, I debated with myself what exactly I would cover next.
There are two natural things I could cover, both of which are complicated.
My first result in the cache oblivious world is making this data structure dynamic.
So, there is a dynamic B tree that's cache oblivious that works for all values of B.
And it gets log base B of N, insert, delete, and search.
So, this just gets search in log base B of N. That data structure, our first paper was damn complicated, and then it got simplified.
It's now not too hard, but it takes a couple of lectures in an advanced algorithms class to teach it.
So, I'm not going to do that.
But there you go.
It exists.
Instead, we're going to cover our favorite problem sorting in the cache oblivious context.
And this is quite complicated, more than you'd expect, OK, much more complicated than it is in a multithreaded setting to get the right answer, anyway.
Maybe to get the best answer in a multithreaded setting is also complicated.
The version we got last week was pretty easy.
But before we go to cache oblivious sorting, let me talk about cache aware sorting because we need to know what bound we are aiming for.
And just to warn you, I may not get to the full analysis of the full cache oblivious sorting.
But I want to give you an idea of what goes into it because it's pretty cool, I think, a lot of ideas.
So, how might you sort?
So, cache aware, we assume we can do everything.
Basically, this means we have B trees.
That's the only other structure we know.
How would you sort N numbers, given that that's the only data structure you have?
Right, just add them into the B tree, and then do an in-order traversal.
That's one way to sort, perfectly reasonable.
We'll call it repeated insertion into a B tree.
OK, we know in the usual setting, and the BST sort, where you use a balanced binary search tree, like red-black trees, that takes N log N time, log N per operation, and that's an optimal sorting algorithm in the comparison model, only thinking about comparison model here.
So, how many memory transfers does this data structure takes?
Sorry, this algorithm for sorting?
The number of memory transfers is a function of N, and B_M of N is?
This is easy.
N insertions, OK, you have to think about N order traversal.
You have to remember back your analysis of B trees, but this is not too hard.
How long does the insertion take, the N insertions?
N log base B of N. How long does the traversal take?
Less time.
If we think about it, you can get away with N over B memory transfers, so quite a bit less than this.
This is bigger than N, which is actually pretty bad.
N memory transfers means essentially you're doing random access, visiting every element in some random order.
It's even worse.
There's even a log factor.
Now, the log factor goes down by this log B factor.
But, this is actually a really bad sorting bound.
So, unlike normal algorithms, where using a search tree is a good way to sort, in cache oblivious or cache aware sorting it's really, really bad.
So, what's another natural algorithm you might try, given what we know for sorting?
And, even cache oblivious, all the algorithms we've seen are cache oblivious.
So, what's a good one to try?
Merge sort.
OK, we did merge sort in multithreaded algorithms.
Let's try a merge sort, a good divide and conquer thing.
So, I'm going to call it binary merge sort because it splits the array into two pieces, and it recurses on the two pieces.
So, you get a binary recursion tree.
So, let's analyze it.
So the number of memory transfers on N elements, so I mean it has a pretty good recursive layout, right?
The two subarrays that we get what we partition our array are consecutive.
So, we're recursing on this, recursing on this.
So, it's a nice cache oblivious layout.
And this is even for cache aware.
This is a pretty good algorithm, a lot better than this one, as we'll see.
But, what is the recurrence we get?
So, here we have to go back to last lecture when we were thinking about recurrences for recursive cache oblivious algorithms.
I mean, the first part should be pretty easy.
There's an O.
Well, OK, let's put the O at the end, the divide and the conquer part at the end.
The recursion is 2MT of N over two, good.
All right, that's just like the merge sort recurrence, and that's the additive term that you're thinking about.
OK, so normally, we would pay a linear additive term here, order N because merging takes order N time.
Now, we are merging, which is three parallel scans, the two inputs and the output.
OK, they're not quite parallel interleaved.
They're a bit funnily interleaved, but as long as your cache stores at least three blocks, this is also linear time in this setting, which means you visit each block a constant number of times.
OK, that's the recurrence.
Now, we also need a base case, of course.
We've seen two base cases, one MT of B, and the other, MT of whatever fits in cache.
So, let's look at that one because it's better.
So, for some constant, C, if I have an array of size M, this fits in cache, actually, probably C is one here, but I'll just be careful.
For some constant, this fits in cache.
A problem of this size fits in cache, and in that case, the number of memory transfers is, anyone remember?
We've used this base case more than once before.
Do you remember?
Sorry?
CM over B. I've got a big O, so M over B.
Order M over B because this is the size of the data.
So, I mean, just to read it all in takes M over B.
Once it's in cache, it doesn't really matter what I do as long as I use linear space for the right constant here.
As long as I use linear space in that algorithm, I'll stay in cache, and therefore, not have to write anything out until the very end and I spend M over B to write it out.
OK, so I can't really spend more than M over B almost no matter what algorithm I have, so long as it uses linear space.
So, this is a base case that's useful pretty much in any algorithm.
OK, that's a recurrence.
Now we just have to solve it.
OK, let's see how good binary merge sort is.
OK, and again, I'm going to just give the intuition behind the solution to this recurrence.
And I won't use the substitution method to prove it formally.
But this one's actually pretty simple.
So, we have, at the top, actually I'm going to write it over here.
Otherwise I won't be able to see.
So, at the top of the recursion, we have N over B costs.
I'll ignore the constants.
There is probably also on additive one, which I'm ignoring here.
Then we split into two problems of half the size.
So, we get a half N over B, and a half N over B. OK, usually this was N, half N, half N. You should regard it as from lecture one.
So, the total on this level is N over B.
The total on this level is N over B.
And, you can prove by induction, that every level is N over B.
The question is how many levels are there?
Well, at the bottom, so, dot, dot, dot, at the bottom of this recursion tree we should get something of size M, and then we're paying M over B.
Actually here we're paying M over B.
So, it's a good thing those match.
They should.
So here, we have a bunch of leaves, all the size M over B.
You can also compute the number of leaves here is N over M. If you want to be extra sure, you should always check the leaf level.
It's a good idea.
So we have N over M leaves, each costing M over B.
This is an M. So, this is N over B also.
So, every level here is N over B memory transfers.
And the number of levels is one N over B?
Log N over B. Yep, that's right.
I just didn't hear it right.
OK, we are starting at N. We're getting down to M. So, you can think of it as log N, the whole binary tree minus the subtrees log M, and that's the same as log N over M, OK, or however you want to think about it.
The point is that this is a log base two.
That's where we are not doing so great.
So this is actually a pretty good algorithm.
So let me write the solution over here.
So, the number of memory transfers on N items is going to be the number of levels times the cost of each level.
So, this is N over B times log base two of N over M, which is a lot better than repeated insertion into a B tree.
Here, we were getting N times log N over log B, OK, so N log N over log B.
We're getting a log B savings over not proving anything, and here we are getting a factor of B savings, N log N over B.
In fact, we even made it a little bit smaller by dividing this N by M. That doesn't matter too much.
This dividing by B is a big one.
OK, so we're almost there.
This is almost an optimal algorithm.
It's even cache oblivious, which is pretty cool.
And that extra little step, which is that you should be able to get on other log B factor improvement, I want to combine these two ideas.
I want to keep this factor B improvement over N log N, and I want to keep this factor log B improvement over N log N, and get them together.
So, first, before we do that cache obliviously, let's do it cache aware.
So, this is the third cache aware algorithm.
This one was also cache oblivious.
So, how should I modify a merge sort in order to do better?
I mean, I have this log base two, and I want a log base B, more or less.
So, how would I do that with merge sort?
Yeah?
Split into B subarrays, yeah.
Instead of doing binary merge sort, this is what I was hinting at here, instead of splitting it into two pieces, and recursing on the two pieces, and then merging them, I could split potentially into more pieces.
OK, and to do that, I'm going to use my cache.
So the idea is B pieces.
This is actually not the best thing to do, although B pieces does work.
And, it's what I was hinting at because I was saying I want a log B.
It's actually not quite log B.
It's log M over B. OK, but let's see.
So, what is the most pieces I could split into?
Right, well, I could split into N pieces.
That would be good, wouldn't it, at only one recursive level?
I can't split into N pieces.
Why?
What happens wrong when I split into N pieces?
That would be the ultimate.
You can't merge, exactly.
So, if I have N pieces, you can't merge in cache.
I mean, so in order to merge in cache, what I need is to be able to store an entire block from each of the lists that I'm merging.
If I can store an entire block in cache for each of the lists, then it's a bunch of parallel scans.
So this is like testing the limit of parallel scanning technology.
If you have K parallel scans, and you can fit K blocks in cache, then all is well because you can scan through each of those K arrays, and have one block from each of the K arrays in cache at the same time.
So, that's the idea.
Now, how many blocks can I fit in cache?
M over B.
That's the biggest I could do.
So this will give the best running time among these kinds of merge sort algorithms.
This is an M over B way merge sort.
OK, so now we get somewhat better recurrence.
We split into M over B subproblems now, each of size, well, it's N divided by M over B without thinking.
And, the claim is that the merge time is still linear because we have barely enough, OK, maybe I should describe this algorithm.
So, we divide, because we've never really done non-binary merge sort.
We divide into M over B equal size subarrays instead of two.
Here, we are clearly doing a cache aware algorithm.
We are assuming we know what M over B is.
So, then we recursively sort each subarray, and then we conquer.
We merge.
And, the reason merge works is because we can afford one block in cache.
So, let's call it one cache block per subarray.
OK, actually, if you're careful, you also need one block for the output of the merged array before you write it out.
So, it should be M over B minus one.
But, let's ignore some additive constants.
OK, so this is the recurrence we get.
The base case is the same.
And, what improves here?
I mean, the per level cost doesn't change, I claim, because at the top we get N over B.
This does before.
Then we split into M over B subproblems, each of which costs a one over M over B factor times N over B. OK, so you add all those up, you still get N over B because we are not decreasing the number of elements.
We're just splitting them.
There's now M over B subproblems, each of one over M over B the size.
So, just like before, each level will sum to N over B.
What changes is the number of levels because now we have bigger branching factor.
Instead of log base two, it's now log base the branching factor.
So, the height of this tree is log base M over B of N over M, I believe.
Let me make sure that agrees with me.
Yeah.
OK, and if you're careful, this counts not quite the number of levels, but the number of levels minus one.
So, I'm going to one plus one here.
And the reason why is this is not quite the bound that I want.
So, we have log base M over B.
What I really want, actually, is N over B. I claim that these are the same because we have minus, yeah, that's good.
OK, this should come as rather mysterious, but it's because I know what the sorting bound should be as I'm doing this arithmetic.
So, I'm taking log base M over B of N over M. I'm not changing the base of the log.
I'm just saying, well, N over M, that is N over B divided by M over B because then the B's cancel, and the M goes on the bottom.
So, if I do that in the logs, I get log of N over B minus log of M over B minus, because I'm dividing.
OK, now, log base M over B of M over B is one.
So, these cancel, and I get log base M over B, N over B, which is what I was aiming for.
Why?
Because that's the right bound as it's normally written.
OK, that's what we will be trying to get cache obliviously.
So, that's the height of the search tree, and at each level we are paying N over B memory transfers.
So, the overall number of memory transfers for this M over B way merge sort is the sorting bound.
This is, I'll put it in a box.
This is the sorting bound, and it's very special because it is the optimal number of memory transfers for sorting N items cache aware.
This has been known since, like, 1983.
OK, this is the best thing to do.
It's a really weird bound, but if you ignore all the divided by B's, it's sort of like N times log base M of N. So, that's little bit more reasonable.
But, there's lots of divided by B's.
So, the number of the blocks in the input times log base the number of blocks in the cache of the number of blocks in the input.
That's a little bit more intuitive.
That is the bound.
And that's what we are aiming for.
So, this algorithm, crucially, assume that we knew what M over B was.
Now, we are going to try and do it without knowing M over B, do it cache obliviously.
And that is the result of only a few years ago.
Are you ready?
Everything clear so far?
It's a pretty natural algorithm.
We were going to try to mimic it essentially and do a merge sort, but not M over B way merge sort because we don't know how.
We're going to try and do it essentially a square root of N way merge sort.
If you play around, that's the natural thing to do.
The tricky part is that it's hard to merge square root of N lists at the same time, in a cache efficient way.
We know that if the square root of N is bigger than M over B, you're hosed if you just do a straightforward merge.
So, we need a fancy merge.
We are going to do a divide and conquer merge.
It's a lot like the multithreaded algorithms of last week, try and do a divide and conquer merge so that no matter how many lists are merging, as long as it's less than the square root of N, or actually cubed root of N, we can do it cache efficiently, OK?
So, we'll do this, we need a bit of setup.
But that's where we're going, cache oblivious sorting.
So, we want to get the sorting bound, and, yeah.
It turns out, to do cache oblivious sorting, you need an assumption about the cache size.
This is kind of annoying, because we said, well, cache oblivious algorithms should work for all values of B and all values of M. But, you can actually prove you need an additional assumption in order to get this bound cache obliviously.
That's the result of, like, last year by Garrett Brodel.
So, and the assumption is, well, the assumption is fairly weak.
That's the good news.
OK, we've actually made an assumption several times.
We said, well, assuming the cache can store at least three blocks, or assuming the cache can store at least four blocks, yeah, it's reasonable to say the cache can store at least four blocks, or at least any constant number of blocks.
This is that the number of blocks that your cache can store is at least B to the epsilon blocks.
This is saying your cache isn't, like, really narrow.
It's about as tall as it is wide.
This actually gives you a lot of sloth.
And, we're going to use a simple version of this assumption that M is at least B^2.
OK, this is pretty natural.
It's saying that your cache is at least as tall as it is wide where these are the blocks.
OK, the number of blocks is it least the size of a block.
That's a pretty reasonable assumption, and if you look at caches these days, they all satisfy this, at least for some epsilon.
Pretty much universally, M is at least B^2 or so.
OK, and in fact, if you think from our speed of light arguments from last time, B^2 or B^3 is actually the right thing to do.
As you go out, I guess in 3-D, B^2 would be the surface area of the sphere out there.
OK, so this is actually the natural thing of how much space you should have at a particular distance.
Assuming we live in a constant dimensional space, that assumption would be true.
This even allows going up to 42 dimensions or whatever, OK, so a pretty reasonable assumption.
Good.
Now, we are going to achieve this bound.
And what we are going to try to do is use an N to the epsilon way merge sort for some epsilon.
And, if we assume that M is at least B^2, the epsilon will be one third, it turns out.
So, we are going to do the cubed root of N way merge sort.
I'll start by giving you and analyzing the sorting algorithms, assuming that we know how to do merge in a particular bound.
OK, then we'll do the merge.
The merge is the hard part.
OK, so the merge, I'm going to give you the black box first of all.
First of all, what does merge do?
The K way merger is called the K funnel just because it looks like a funnel, which you'll see.
So, a K funnel is a data structure, or is an algorithm, let's say, that looks like a data structure.
And it merges K sorted lists.
So, supposing you already have K lists, and they're sorted, and assuming that the lists are relatively long, so we need some additional assumptions for this black box to work, and we'll be able to get them as we sort.
We want the total size of those lists.
You add up all the elements, and all the lists should have size at least K^3 is the assumption.
Then, it merges these lists using essentially the sorting bound.
Actually, I should really say theta K^3.
I also don't want to be too much bigger than K^3.
Sorry about that.
So, the number of memory transfers that this funnel merger uses is the sorting bound on K^3, so K^3 over B, log base M over B of K^3 over B, plus another K memory transfers.
Now, K memory transfers is pretty reasonable.
You've got to at least start reading each list, so you got to pay one memory transfer per list.
OK, but our challenge in some sense will be getting rid of this plus K. This is how fast we can merge.
We'll do that after.
Now, assuming we have this, let me tell you how to sort.
This is, eventually enough, called funnel sort.
But in a certain sense, it's really cubed root of N way merge sort.
OK, but we'll analyze it using this.
OK, so funnel sort, we are going to define K to be N to the one third, and apply this merger.
So, what do we do?
It's just like here.
We're going to divide our array into N to the one third.
I mean, it they should be consecutive subarrays.
I'll call them segments of the array.
OK, for cache oblivious, it's really crucial how these things are laid out.
We're going to cut and get consecutive chunks of the array, N to the one third of them.
Then I'm going to recursively sort them, and then I'm going to merge.
OK, and I'm going to merge using the K funnel, the N to the one third funnel because, now, why do I use one third?
Well, because of this three.
OK, in order to use the N to the one third funnel, I need to guarantee that the total number of elements that I'm merging is at least the cube of this number, K^3.
The cube of this number is N. That's exactly how many elements I have in total.
OK, so this is exactly what I can apply the funnel.
It's going to require that I have it least K^3 elements, so that I can only use an N to the one third funnel.
I mean, if it didn't have this requirement, I could just say, well, I have N lists each of size one.
OK, that's clearly not going to work very well for our merger, I mean, intuitively because this plus K will kill you.
That will be a plus M which is way too big.
But we can use an N to the one third funnel, and this is how we would sort.
So, let's analyze this algorithm.
Hopefully, it will give the sorting bound if I did everything correctly.
OK, this is pretty easy.
The only thing that makes this messy as I have to write the sorting bound over and over.
OK, this is the cost of the merge.
So that's at the root.
But K^3 in this case is N. So at the root of the recursion, let me write the recurrence first.
Sorry.
So, we have memory transfers on N elements is N to the one third.
Let me get this right.
Yeah, N to the one third recursions, each of size N to the two thirds, OK, plus this time, except K^3 is N. So, this is plus N over B, log base M over B of N over B plus cubed root of M. This is additive plus K term.
OK, so that's my recurrence.
The base case will be the usual.
MT is some constant times M is order M over B.
So, we sort of know what we should get here.
Well, not really.
So, in all the previous recurrence is, we have the same costs at every level, and that's where we got our log factor.
Now, we already have a log factor, so we better not get another one.
Right, this is the bound we want to prove.
So, let me cheat here for a second.
All right, indeed.
You may already be wondering, this N to the one third seems rather large.
If it's bigger than this, we are already in trouble at the very top level of the recursion.
So, I claim that that's OK. Let's look at N to the one third.
OK, there is a base case here which covers all values of N that are, at most, some constant times N. So, if I'm in this case, I know that N is at least as big as the cache up to some constant.
OK, now the cache is it least B^2, we've assumed.
And you can do this with B to the one plus epsilon if you're more careful.
So, N is at least B^2, OK?
And then, I always have trouble with these.
So this means that N divided by B is omega root N. OK, there's many things you could say here, and only one of them is right.
So, why?
So this says that the square root of N is at least B, and so N divided by B is at most N divided by square root of N. So that's at least the square root of N if you check that all out.
I'm going to go through this arithmetic relatively quickly because it's tedious but necessary.
OK, the square root of N is strictly bigger than cubed root of N. OK, so that means that N over B is strictly bigger than N to the one third.
Here we have N over B times something that's bigger than one.
So this term definitely dominates this term in this case.
As long as I'm not in the base case, I know N is at least order M. This term disappears from my recurrence.
OK, so, good.
That was a bit close.
Now, what we want to get is this running time overall.
So, the recursive cost better be small, better be less than the constant factor increase over this.
So, let's write the recurrence.
So, we get N over B, log base M over B, N over B at the root.
Then, we split into a lot of subproblems, N to the one third subproblems here, and each one costs essentially this but with N replaced by N to the two thirds.
OK, so N to the two thirds log base M over B, oops I forgot to divide it by B out here, of N to the two thirds divided by B.
That's the cost of one of these nodes, N to the one third of them.
What should they add up to?
Well, there is N to the one third, and there's an N to the two thirds here that multiplies out to N. So, we get N over B.
This looks bad.
This looks the same.
And we don't want to lose another log factor.
But the good news is we have two thirds in here.
OK, this is what we get in total at this level.
It looks like the sorting bound, but in the log there's still a two thirds.
Now, a power of two thirds and a log comes out as a multiple of two thirds.
So, this is in fact two thirds times N over B, log base M over B of N over B, the sorting bound.
So, this is two thirds of the sorting bound.
And this is the sorting bound, one times the sorting bound.
So, it's going down geometrically, yea!
OK, I'm not going to prove it, but it's true.
This went down by a factor of two thirds.
The next one will also go down by a factor of two thirds by induction.
OK, if you prove it at one level, it should be true at all of them.
And I'm going to skip the details there.
So, we could check the leaf level just to make sure.
That's always a good sanity check.
At the leaves, we know our cost is M over B. OK, and how many leaves are there?
Just like before, in some sense, we have N/M leaves.
OK, so in fact the total cost at the bottom is N over B.
And it turns out that that's what you get.
So, you essentially, it looks funny, because you'd think that this would actually be smaller than this at some intuitive level.
It's not.
In fact, what's happening is you have this N over B times this log thing, whatever the log thing is.
We don't care too much.
Let's just call it log.
What you are taking at the next level is two thirds times that log.
And at the next level, it's four ninths times that log and so on.
So, it's geometrically decreasing until the log gets down to one.
And then you stop the recursion.
And that's what you get N over B here with no log.
So, what you're doing is decreasing the log, not the N over B stuff.
The two thirds should really be over here.
In fact, the number of levels here is log, log N. It's the number of times you have to divide a log by three halves before you get down to one, OK?
So, we don't actually need that.
We don't care how many levels are because it's geometrically decreasing.
It could be infinitely many levels.
It's geometrically decreasing, and we get this as our running time.
MT of N is the sorting bound for funnel sort.
So, this is great.
As long as we can get a funnel that merges this quickly, we get a sorting algorithm that sorts as fast as it possibly can.
I didn't write that on the board that this is asymptotically optimal.
Even if you knew what B and M were, this is the best that you could hope to do.
And here, we are doing it no matter what, B and M are.
Good.
Get ready for the funnel.
The funnel will be another recursion.
So, this is a recursive algorithm in a recursive algorithm.
It's another divide and conquer, kind of like the static search trees we saw at the beginning of this lecture.
So, these all tie together.
All right, the K funnel, so, I'm calling it K funnel because I want to think of it at some recursive level, not just N to the one third.
OK, we're going to recursively use, in fact, the square root of K funnel.
So, here's, and I need to achieve that bound.
So, the recursion is like the static search tree, and a little bit hard to draw on one board, but here we go.
So, we have a square root of K funnel.
Recursively, we have a buffer up here.
This is called the output buffer, and it has size K^3, and just for kicks, let's suppose it that filled up a little bit.
And, we have some more buffers.
And, let's suppose they've been filled up by different amounts.
And each of these has size K to the three halves, of course.
Halves, these are called buffers, let's say, with the intermediate buffers.
And, then hanging off of them, we have more funnels, the square root of K funnel here, and a square root of K funnel here, one for each buffer, one for each child of this funnel.
OK, and then hanging off of these funnels are the input arrays.
OK, I'm not going to draw all K of them, but there are K input arrays, input lists let's call them down at the bottom.
OK, so the idea is we are going to merge bottom-up in this picture.
We start with our K input arrays of total size at least K^3.
That's what we're assuming we have up here.
We are clustering them into groups of size square root of K, so, the square root of K groups, throw each of them into a square root of K funnel that recursively merges those square root of K lists.
The output of those funnels we are putting into a buffer to sort of accumulate what the answer should be.
These buffers have besides exactly K to the three halves, which might not be perfect because we know that on average, there should be K to the three halves elements in each of these because there's K^3 total, and the square root of K groups.
So, it should be K^3 divided by the square root of K, which is K to the three halves on average.
But some of these will be bigger.
Some of them will be smaller.
I've drawn it here.
Some of them had emptied a bit more depending on how you merge things.
But on average, these will all fill at the same time.
And then, we plug them into a square root of K funnel, and that we get the output of size K^3.
So, that is roughly what we should have happen.
OK, but in fact, some of these might fill first, and we have to do some merging in order to empty a buffer, make room for more stuff coming up.
That's the picture.
Now, before I actually tell you what the algorithm is, or analyze the algorithm, let's first just think about space, a very simple warm-up analysis.
So, let's look at the space excluding the inputs and outputs, those buffers.
OK, why do I want to exclude input and output buffers?
Well, because I want to only count each buffer once, and this buffer is actually the input to this one and the output to this one.
So, in order to recursively count all the buffers exactly once, I'm only going to count these middle buffers.
And then separately, I'm going to have to think of the overall output and input buffers.
But those are sort of given.
I mean, I need K^3 for the output.
I need K^3 for the input.
So ignore those overall.
And that if I count the middle buffers recursively, I'll get all the buffers.
So, then we get a very simple recurrence for space.
S of K is roughly square root of K plus one times S of square root of K plus order K^2, K^2 because we have the square root of K of these buffers, each of size K to the three halves.
Work that out, does that sound right?
That sounds an awful lot like K^3, but maybe, all right.
Oh, no, that's right.
It's K to the three halves times the square root of K, which is K to the three halves plus a half, which is K to the four halves, which is K^2.
Phew, OK, good.
I'm just bad with my arithmetic here.
OK, so K^2 total buffering here.
You add them up for each level, each recursion, and the plus one here is to take into account the top guy, the square root of K bottom guys, so the square root of K plus one.
If this were, well, let me just draw the recurrence tree.
There's many ways you could solve this recurrence.
A natural one is instead of looking at K, you look at log K, because here at log K is getting divided by two.
I just going to draw the recursion trees, so you can see the intuition.
But if you are going to solve it, you should probably take the logs, substitute by log.
So, we have the square root of K. plus one branching factor.
And then, the problem is size square root of K, so this is going to be K, I believe, for each of these.
This is square root of K squared is the cost of these levels.
And, you keep going.
I don't particularly care what the bottom looks like because at the top we have K^2.
That we have K times root K plus one cost at the next level.
This is K to the three halves plus K. OK, so we go from K^2 to K to the three halves plus K. This is a super-geometric.
It's like an exponential geometric decrease.
This is decreasing really fast.
So, it's order K^2.
That's my hand-waving argument.
OK, so the cost is basically the size of the buffers at the top level, the total space.
We're going to need this.
It's actually theta K^2 because I have a theta K^2 here.
We are going to be this in order to analyze the time.
That's why it mentioned it.
It's not just a good feeling that the space is not too big.
In fact, the funnel is a lot smaller than a total input size.
The input size is K^3.
But that's not so crucial.
What's crucial is that it's K^2, and we'll use that in the analysis.
OK, naturally, this thing is laid out recursively.
You recursively store the funnel, top funnel.
Then, for example, you write out each buffer as a consecutive array, in this case.
There's no recursion there.
So just write them all out one by one.
Don't interleave them or anything.
Store them in order.
And that, you write out recursively these funnels, the bottom funnels.
OK, any way you do it recursively, as long as each funnel remains a consecutive chunk of memory, each buffer remains a consecutive chuck of memory, the time analysis that we are about to do will work.
OK, let me actually give you the algorithm that we're analyzing.
In order to make the funnel go, what we do is say, initially, all the buffers are empty.
Everything is at the bottom.
And what we are going to do is, say, fill the root buffer.
Fill this one.
And, that's a recursive algorithm, which I'll define in a second, how to fill a buffer.
Once it's filled, that means everything has been pulled up, and then it's merged.
OK, so that's how we get started.
So, merge means to merge algorithm is fill the topmost buffer, the topmost output buffer.
OK, and now, here's how you fill a buffer.
So, in general, if you expand out this recursion all the way, in the base case, I didn't mention you sort of get a little node there.
So, if you look at an arbitrary buffer in this picture that you want to fill, so this one's empty and you want to fill it, then immediately below it will be a vertex who has two children, two other buffers.
OK, maybe they look like this.
You have no idea how big they are, except they are the same size.
It could be a lot smaller than this one, a lot bigger, we don't know.
But in the end, you do get a binary structure out of this just like we did with the binary search tree at the beginning.
So, how do we fill this buffer?
Well, we just merge these two child buffers as long as we can.
So, we merge the two children buffers as long as they are both non-empty.
So, in general, the invariant will be that this buffer, let me write down a sentence.
As long as a buffer is non-empty, or whatever is in that buffer, and hasn't been used already, it's a prefix of the merged output of the entire subtree beneath it.
OK, so this is a partially merged subsequence of everything down here.
This is a partially merged subsequence of everything down here.
I can just merge element by element off the top, and that will give me outputs to put there until one of them gets emptied.
And, we have no idea which one will empty first just because it depends on the order.
OK, whenever one of them empties, we recursively fill it, and that's it.
That's the algorithm.
Whenever one empties -- -- we recursively fill it.
And at the base case at the leaves, there's sort of nothing to do.
I believe you just sort of directly read from an input list.
So, at the very bottom, if you have some note here that's trying to merge between these two, that's just a straightforward merge between two lists.
We know how to do that with two parallel scans.
So, in fact, we can merge the entire thing here and just spit it out to the buffer.
Well, it depends how big the buffer is.
We can only merge it until the buffer fills.
Whenever a buffer is full, we stop and we pop up the recursive layers.
OK, so we keep doing this merge until the buffer we are trying to fill fills, and that we stop, pop up.
OK, that's the algorithm for merging.
Now, we just have to analyze the algorithm.
It's actually not too hard, but it's a pretty clever analysis.
And, to top it off, it's an amortization, your favorite.
OK, so we get one last practice at amortized analysis in the context of cache oblivious algorithms.
So, this is going to be a bit sophisticated.
We are going to combine all the ideas we've seen.
The main analysis idea we've seen is that we are doing this recursion in the construction, and if we imagine, we take our K funnel, we split it in the middle level, make a whole bunch of square root of K funnels, and so on, and then we cut those in the middle level, get fourth root of K funnels, and so on, and so on, at some point the funnel we look at fits in cache.
OK, before we said if it's in a block.
Now, we're going to say that at some point, one of these funnels will fit in cache.
Each of the funnels at that recursive level of detail will fit in cache.
We are going to analyze that level.
We'll call that level J.
So, consider the first recursive level of detail, and I'll call it J, at which every J funnel we have fits, let's say, not only does it fit in cache, but four of them fit in cache.
It fits in one quarter of the cache.
OK, but we need to leave some cache extra for doing other things.
But I want to make sure that the J funnel fits.
OK, now what does that mean?
Well, we've analyzed space.
We know that the space of a J funnel is about J^2, some constant times J^2.
We'll call it C times J^2.
OK, so this is saying that C times J^2 is at most, M over 4, one quarter of the cache.
OK, that means a J funnel that happens at the size sits in the quarter of the cache.
OK, at some point in the recursion, we'll have this big tree of J funnels, with all sorts of buffers in between them, and each of the J funnels will fit.
So, let's think about one of those J funnels.
Suppose J is like the square root of K. So, this is the picture because otherwise I have to draw a bigger one.
So, suppose this is a J funnel.
It has a bunch of input buffers, has one output buffer.
So, we just want to think about how the J funnel executes.
And, for a long time, as long as these buffers are all full, this is just a merger.
It's doing something recursively, but we don't really care.
As soon as this whole thing swaps in, and actually, I should be drawing this, as soon as the funnel, the output buffer, and the input buffer swap in, in other words, you bring all those blocks in, you can just merge, and you can go on your merry way merging until something empties or you fill the output.
So, let's analyze that.
Suppose everything is in memory, because we know it fits.
OK, well I have to be a little bit careful.
The input buffers are actually pretty big in total size because the total size is K to the three halves here versus K to the one half.
Actually, this is of size K. Let me draw a general picture.
We have a J funnel, because otherwise the arithmetic is going to get messy.
We have a J funnel.
Its size is C times J^2, we're supposing.
The number of inputs is J, and the size of them is pretty big.
Where did we define that?
We have a K funnel.
The total input size is K^3.
So, the total input size here would be J^3.
We can't afford to put all that in cache.
That's an extra factor of J.
But, we can afford to one block per input.
And for merging, that's all we need.
I claim that I can fit the first block of each of these input arrays in cash at the same time along with the J funnel.
And so, for that duration, as long as all of that is in cache, this thing can merge at full speed just like we were doing parallel scans.
You use up all the blocks down here, and one of them empties.
You go to the next block in the input buffer and so on, just like the normal merge analysis of parallel arrays, at this point we assume that everything here is fitting in cache.
So, it's just like before.
Of course, in fact, it's recursive but we are analyzing it at this level.
OK, I need to prove that you can fit one block per input.
It's not hard.
It's just computation.
And, it's basically the way that these funnels were designed was so that you could fit one block per input buffer.
And, here's the argument.
So, the claim is you can also fit one memory block in the cache per input buffer.
So, this is in addition to one J funnel.
You could also fit one block for each of its input buffers.
OK, this is of the J funnel.
It's not any funnel because bigger funnels are way too big.
OK, so here's how we prove that.
J^2 is at most a quarter M. That's what we assumed here, actually CJ2.
I'm not going to bother with the C because that's going to make my life even harder.
OK, I think this is even a weaker constraint.
So, the size of our funnel proves about J^2.
That's at most a quarter of the cache.
That implies that J, if we take square roots of both sides, is at most a half square root of M. OK, also, we know that B is at most square root of M because M is at least B squared.
So, we put these together, and we get J times B is at most a half M. OK, now I claim that what we are asking for here is J times B because in a J funnel, there are J input arrays.
And so, if you want one block each, that costs a space of B each.
So, for each input buffer we have one block of size B, and the claim is that that whole thing fits in half the cache.
And, we've only used a quarter of the cache.
So in total, we use three quarters of the cache and that's all we'll use.
OK, so that's good news.
We can also fit one more block to the output.
Not too big a deal.
So now, as long as this J funnel is running, if it's all in cache, all is well.
What does that mean?
Let me first analyze how long it takes for us to swap in this funnel.
OK, so how long does it take for us to read all the stuff in a J funnel and one block per input buffer?
That's what it would take to get started.
So, this is swapping in a J funnel, which means reading the J funnel in its entirety, and reading one block per input buffer.
OK, the cost of the swap in is pretty natural.
The size of the buffer divided by B, because that's just sort of a linear scan to read it in, and we need to read one block per buffer.
These buffers could be all over the place because they're pretty big.
So, let's say we pay one memory transfer for each input buffer just to get started to read the first block.
OK, the claim is, and here we need to do some more arithmetic.
This is, at most, J^3 over B. OK, why is it, at most, J^3 over B?
Well, this was the first level at which things fit in cache.
That means the next level bigger, which is J^2, which has size J^4, should be bigger than cache.
Otherwise we would have stopped then.
OK, so this is just more arithmetic.
You can either believe me or follow the arithmetic.
We know that J^4 is at least M. So, this means that, and we know that M is at least B^2.
Therefore, J^2, instead of J^4, we take the square root of both sides, J^2 is at least B. OK, so certainly J^2 over B is at most J^3 over B.
But also J is at most J^3 over B because J^2 is at least B. Hopefully that should be clear.
That's just algebra.
OK, so we're not going to use this bound because that's kind of complicated.
We're just going to say, well, it causes J^3 over B to get swapped in.
Now, why is J^3 over B a good thing?
Because we know the total size of inputs to the J funnel is J^3.
So, to read all of the inputs to the J funnel takes J^3 over B.
So, this is really just a linear extra cost to get the whole thing swapped in.
It sounds good.
To do the merging would also cost J^3 over B.
So, the swap-in causes J^3 over B to merge all these J^3 elements.
If they were all there in the inputs, it would take J^3 over B because once everything is there, you're merging at full speed, one per B items per memory transfer on average.
OK, the problem is you're going to swap out, which you may have imagined.
As soon as one of your input buffers empties, let's say this one's almost gone, as soon as it empties, you're going to totally obliterate that funnel and swap in this one in order to merge all the stuff there, and fill this buffer back up.
This is where the amortization comes in.
And this is where the log factor comes in because so far it we've basically paid a linear cost.
We are almost done.
So, we charge, sorry, I'm jumping ahead of myself.
So, when an input buffer empties, we swap out.
And we recursively fill that buffer.
OK, I'm going to assume that there is absolutely no reuse, that is recursive filling completely swapped everything out and I have to start from scratch for this funnel.
So, when that happens, I feel this buffer, and then I come back and I say, well, I go swap it back in.
So when the recursive call finishes, I swap back in.
OK, so I recursively fill, and then I swap back in.
And, at the swapping back in costs J^3 over B. I'm going to charge that cost to the elements that just got filled.
So this is an amortized charging argument.
How many are there?
It's the only question.
It turns out, things are really good, like here, for the square root of K funnel, we have each buffer has size K to the three halves.
OK, so this is a bit complicated.
But I claim that the number of elements here that fill the buffer is J^3.
So, if you have a J funnel, each of the input buffers has size J^3.
It should be correct if you work it out.
So, we're charging this J^3 over B cost to J^3 elements, which sounds like you're charging, essentially, one over B to each element.
Sounds great.
That means that, so you're thinking overall, I mean, there are N elements, and to each one you charge a one over B cost.
That sounds like the total running time is N over B.
It's a bit too fast for sorting.
We lost the log factor.
So, what's going on is that we're actually charging to one element more than once.
And, this is something that we don't normally do, never done it in this class, but you can do it as long as you bound that the number of times you charge.
OK, and wherever you do a charging argument, you say, well, this doesn't happen too many times because whenever this happens, this happens.
You should say, you should prove that the thing that you're charging to, Ito charged to that think very many times.
So here, I have a quantifiable thing that I'm charging to: elements.
So, I'm saying that for each element that happened to come into this buffer, I'm going to charge it a one over B cost.
How many times does one element get charged?
Well, each time it gets charged to, it's moved into a new buffer.
How many buffers could it move through?
Well, it's just going up all the time.
Merging always goes up.
So, we start here and you go to the next buffer, and you go to the next buffer.
The number of buffers you visit is the right log, it turns out.
I don't know which log that is.
So, the number of charges of a one over B cost to each element is the number of buffers it visits, and that's a log factor.
That's where we get an extra log factor on the running time.
It is, this is the number of levels of J funnels that you can visit.
So, it's log K divided by log J, if I got it right.
OK, and we're almost done.
Let's wrap up a bit.
Just a little bit more arithmetic, unfortunately.
So, log K over log J.
Now, J^2 is like M, roughly.
It might be square root of M. But, log J is basically log M. There's some constants there.
So, the number of charges here is theta, log K over log M. So, now this is a bit, we haven't seen this in amortization necessarily, but we just need to count up total amount of charging.
All work gets charged to somebody, except we didn't charge the very initial swapping in to everybody.
But, every time we do some swapping in, we charge it to someone.
So, how many times does everything it charged?
Well, there are N elements.
Each gets charged to a one over B cost, and the number of times it gets charged is its log K over log M. So therefore, the total cost is number of elements times a one over B times this log thing.
OK, it's actually plus K. We forgot about a plus K, but that's just to get started in the very beginning, and start on all of the input lists.
OK, this is an amortization analysis to prove this bound.
Sorry, what was N here?
I assumed that I started out with K cubed elements at the bottom.
The total number of elements in the bottom was K^3 theta.
OK, so I should have written K^3 not M. This should be almost the same as this, OK, but not quite.
This is log based M of K, and if you do a little bit of arithmetic, this should be K^3 over B times log base M over B of K over B plus K. That's what I want to prove.
Actually there's a K^3 here instead of a K, but that's just a factor of three.
And this follows because we assume we are not in the base case.
So, K is at least M, which is at least B^2, and therefore K over B is omega square root of K. OK, so K over B is basically the same as K when you put it in a log.
So here we have log base M. I turned it into log base M over B.
That's even worse.
It doesn't matter.
And, I have log of K. I replaced it with K over B, but K over B is basically square root of K. So in a log, that's just a factor of a half.
So that concludes the analysis of the funnel.
We get this crazy running time, which is basically sorting bound plus a little bit.
We plug that into our funnel sort, and we get, magically, optimal cache oblivious sorting just in time.
Tuesday is the final.
The final is more in the style of quiz one, so not too much creativity, mostly mastery of material.
It covers everything.
You don't have to worry about the details of funnel sort, but everything else.
So it's like quiz one but for the entire class.
It's three hours long, and good luck.
It's been a pleasure having you, all the students.
I'm sure Charles agrees, so thanks everyone.
It was a lot of fun.
Algorithms is an interesting area, because it's very creative to come up with new algorithms.
There's often many ways to solve the same problem.
But unlike art say, in algorithms there's a quantifiable measures for how good you did.
What is the running time of the thing you ended up with?
How much space does it use?
It is efficient?
So it's nice in that it's easy to evaluate whether you kind of succeeded.
Did you get the n log n algorithm?
Did you get the fastest that's possible?
But it's hard to measure a student's understanding because it's like, did you get the creative trick that we had in mind, or find another one that's just as good?
Students may find a different creative trick that doesn't end up with as good an algorithm in the end.
So we penalize that someone, but for the most part we are happy when people get correct algorithms.
When they understand the techniques well enough that they get an algorithm, it's pretty good.
May not be the best, but we consider that much better than they were using the right technique, but they ended up using it wrong and so they end up with something doesn't even work.
And that's much more discouraging, I guess.
But in any case, in a timed exam, it's really hard to expect students to be creative.
So in that setting, it's hard.
We don't expect a lot of creativity.
We try to keep the questions relatively simple, so you need just one simple idea.
If you're familiar with the material, it should come quickly.
It Doesn't always, but-- So problem sets are definitely an easier way for students to express creativity.
It gives them a little more time to think about the problem.
Because some problems are going to come easy to some students and hard to others.
It's going to be different problem set, so they have a week to think about it.
And also to talk to others, we allow collaboration in the problem sets, so that makes it easier to find solutions but still learn a lot from it.
In the exams, it's a lot harder.
In past years we've tried take home exams as a way to more accurately gauge how creative someone is in applying these techniques to design new algorithms.
We didn't do that this year.
It's not clear what the right answer is.
It's definitely a challenge in teaching algorithms, is how to evaluate whether someone can really apply the techniques in new and creative ways.
I mean, there are a couple of different approaches to teaching this class.
One is to make it fun.
I really like to make material as engaging and as accessible as possible.
So, especially if you have relatively little experience in algorithms-- you've just taken 6006-- and some of these algorithms get really complicated.
And they're hard to understand and it makes you wonder, how did they ever come up with that?
So I like to talk about that.
In some of the lectures, I'll kind of give an idea of how they might have gotten there.
It's like, oh we could start with this really simple step.
It's really inefficient, but at least it solves a problem.
But then we have this extra thing and this extra thing, and eventually you get to the really fast solution, so it's less of the mystery of how you got there, or how the original researchers got there.
And I also try to remind students, especially in the harder topics, that this is not easy stuff and you should think about it a little more.
Do the problem sets and you'll learn it-- figure it out at a deeper level.
But a lot of this material is hard, and I think being honest about that makes it more reasonable.
Students don't feel bad for thinking that it's hard, because the first time you learn it, it is hard.
But the whole algorithmic way of thinking of setting up a problem-- what should the input and the output be-- and then thinking about all different solutions for how to fill in the middle I think is a really powerful perspective, and it can shape pretty much your entire life, but especially if you're doing computer science.
It's a really useful way of thinking, and I've gotten lots of letters later from students where they're doing some job, some computer thing and they ran this problem and then they discover-- and then they looked back at their notes and said, oh, this is the right data structure I should use now.
And they solve it, and they tell me about it.
And it's kind of exciting to see years later the way of thinking and lots of the solutions and techniques you see in this class really are useful out there in the world, so it's a useful class to take.
It's a relatively hard class.
The exams are challenging.
We try to make them not super challenging, but it's hard, especially whenever creativity is involved.
It's hard to do that live in an exam.
So students may not perform super well my class, but still they get the idea that this is a way to approach problems, and they understand some of the techniques.
And over time, they can fill in whatever holes they left behind, and say, you know, it seems like a problem where dynamic programming would be really useful.
It's time I really learned dynamic programming.
Dynamic programming is one that we actually teach both in 006 and 046, because it's really conceptually weird.
It's actually really easy once you understand it, but getting to that point of understanding is really tough.
So we spend a lot of time on it so that the ideas-- and more exposure you get to it, the clearer it will become.
I think by the end of 6046, they have a pretty good idea.
But if not, years later, if they come back it, I'm sure it'll be a lot easier this third time around.
So a lot of these concepts are-- I mean, the ideas are actually really simple.
They're just hard to shift your mind into that kind of mindset, but once you do, things become really clear.
So hope is, by the end of the semester, they get there.
Co-teaching with Srini, Professor Srinivas Devadas is really a lot of fun.
We've taught before.
We taught the 6006 that's on OCW together.
And Srini is I would say one of the most humorous professors that I know at MIT.
So it's really a lot of fun to teach with him.
And I mean, sometimes it's like a comedy routine in his lectures.
And I like to do that too in lectures, tell lots of jokes and play together.
So it's really nice working together.
We have a good dynamic.
We're both super flexible, like, oh, I need you to cover this lecture.
Oh, no problem, I love that stuff.
In fact, we were both kind of fighting over which lectures we wanted to teach.
Because I was like, I really want to teach this-- no, no, I taught it.
I really want to teach it this time.
So that's a nice dynamic to have.
It's not like oh, you taught one too few lectures, it's not balanced.
Totally not like that.
And both in 6006 and 6046, in the last lecture we do a kind of actual comedy routine where we're somehow fighting each other.
It's all in good humor, and it's pretty fun.
The students really like it.
He's also-- in both classes he has creative ideas on how to reward students for answering questions.
So in 6006, we were in a relatively uncomfortable lecture hall.
And so we had these pillows that we had custom printed and had the name of the class and a little Rubik's cube on it.
And so if anyone ever answered a question in class or asked a good question, then we would throw them a pillow, kind of like a Frisbee.
And so for 6046, Srini was like, you know what we have to do this time is get actual Frisbees.
And so we got these blue and purple Frisbees, also custom printed with the class number and a little diagram that is in lecture 20 or something.
And so whenever anyone asked or answered a question, we would throw them a Frisbee.
We'd make jokes about whether blue was better, or purple was better.
There was no better.
It was just-- So this was tough for me, because I had to really learn how to throw Frisbees.
But I think by the end of the semester, I got pretty good.
To me, the key to teaching algorithms well is just be excited about the material.
I think if you personally think these algorithms are really cool and like, wow, how did they come up with this, and wow, I mean, look how this thing interacts with this thing and magically it all runs super fast, I think communicating that excitement, students pick up on it really quickly and themselves get engaged in the material and care-- care more about it.
So I just try to have fun with the material and encourage the students to similarly have fun learning it.
So it all comes together.
Occasionally, we'll do a demonstration.
But a lot of the material in this class is pretty theoretical.
So it's mostly board work and trying to draw representative figures and things like that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So welcome to 6046.
My name is Srinivas Devadas.
I'm a professor of computer science.
This is my 27th year at MIT.
I'm teaching this class with great course staff, with co-lecturers, Eric Demaine over here and Nancy Lynch, who's over there, and a whole bunch of TAs, who you will meet through the term.
We just signed up our last TA yesterday, so at this point, even I don't know their names.
But we hope to have a great semester.
I'm very excited to be teaching this class with Eric and Nancy.
I recognize some of you folks from 006 from a year ago, so hello again, and from other classes.
And so let's get started.
I mentioned 006.
006 is a prerequisite for this class, so if by chance you've skipped a class-- MIT or EECS has allowed you to skip that-- make sure you check in with us to see that you are ready for 6046 because we will assume that you know the 6006 material.
And by that, I mean basic material on that data structures, classical algorithms like sorting, algorithms for dynamic programming, or algorithms that use dynamic programming I should say, algorithms for shortest paths, et cetera.
6046 itself, we're going to run this course pretty much off the Stellar website in the sense that that'll be our one-stop shop for getting everything including lecture handouts, problem sets-- turning in your problem sets, et cetera.
And I should mention that this course is being taped for OpenCourseWare, and while it'll take a little bit of time for the videos to be put online, we hope to do that perhaps in clumps before the quizzes that you will have as we have to have in our class.
So let me just say a couple more things about logistics, and then we get started with technical content.
As I mentioned, we're going to be running this course off Stellar.
Please sign up for recitations section by going to the stellar website and choosing a section that works for your schedule.
Sections go from 10:00 AM all the way to 3:00 I think, and we've placed a limit on the number of students per section.
We wanted the sections to be manageable in size, but there's plenty of room for everybody, and the schedule flexibility should allows you to choose a section pretty easily.
We have a course information document and an objectives document on the website.
That has a lot of details on the grading policy, the collaboration policy, et cetera.
Please read it very carefully from the first page all the way to the end.
And I will mention one thing that you should be careful about, which is that while problem sets are only 30% of the grade, we do require you to attempt the problems.
And there's actually a penalty associated with not attempting problems and not tuning problem sets in that is way more than 30%, so keep that in mind, and please read the collaboration policy as well as the grading policy, carefully.
And feel free to ask us questions.
You can ask us questions anonymously through Piazza, or you can certainly send us email.
All the information is on Stellar.
So that's all I really had to say about course logistics.
Let me tell you a little bit about how the content of this course is structured.
We have several modules, and Eric, Nancy, and I will be in charge of each of these different modules as the term goes.
Our very first module is going to start really next time.
Today is really an overview lecture.
But it's a module on divide and conquer, and you learned about this divide and conquer paradigm in 006 or equivalent classes.
It's breaking of a problem into smaller problems and getting efficiency that way.
Merge sort is a classic algorithm that follows the divide and conquer paradigm.
If you're going to take it to a new level.
And I guess that's sort of the team of 046.
Take the material in 006 and raise the stakes a little bit-- raise the level of sophistication-- and you'll see things like fast Fourier transform.
Finding an algorithm for a convex hull, we'll do that next time.
That uses the divide and conquer paradigm.
We're going to do a ton of optimization.
Divide and conquer can obviously be used for search and also for optimization.
In particular, we'll look at strategies corresponding to greedy algorithms, Dijkstra, which hopefully you remember the shortest path algorithm from 006 is an example of a greedy algorithm.
We'll see a bunch of other examples, and we'll look at one today.
And dynamic programming, it's a wonderful algorithmic hammer that you can apply to a wide variety of problems, certainly to shortest paths as well.
We'll look at it in many different contexts.
And then really quickly network flow, which is a problem that's associated with-- here's a network.
This capacity is associated with the network.
The capacities could respond to the width of the roads in a highway system or the number of lanes, the amount of traffic that can go through.
How do I maximize the set of commodities, or the amount of commodities that I can push through the network?
That it turns out is, again, a problem that has many different applications, so it's really a collection of problems.
You're going to spend some time, a little bit today, but a little more than in 6006, talking about intractability.
So a lot of algorithms that we're going to talk about are efficient in the sense that they're polynomial time solvable.
And first, polynomial time solvable doesn't imply efficiency in the practical sense, so if you have an n raised to 8 algorithm, it's polynomial time.
But really, it's not something that you can use on real world problems where n is relatively large, but generally in a theoretical computer science class, we'll think about tractable problems as being those that have polynomial time algorithms that can solve them exactly or optimally.
But intractability then corresponds to problems that, at the moment, we don't know of a polynomial time algorithm to solve them, and the best algorithms we have take worst case exponential time.
And so the question is, what happens with those problems?
And we'll look at things like approximation algorithms that can get us, in the case of optimization problems, get us to within a certain fraction of optimal, guaranteed, and run in polynomial time.
So you can't get the absolute best, but you can get within 10% or we can get within a factor of 2.
That may be enough for a particular instance of a problem or a set of instances of a problem.
And what we do a bunch of advanced topics.
I think we have distributed algorithms plan.
Nancy works in that area, and we'll also talk about cryptography.
There's a deep connection between number theory algorithms and cryptography that towards end of the lecture, or, I should say, towards the end of the course, I will look at a little more closely.
So much for overview, let's get started with today's lecture for real.
And here's the theme of today's lecture.
I talked a bit about tractability and intractability.
And what is fascinating about algorithms is that you might see a problem that has a fairly obvious polynomial time solution or a linear time solution, then you change it ever so slightly, and the linear time algorithm doesn't work.
Maybe you can find a cubic algorithm.
And then you change it a little more, and you end up with something that you can't find a polynomial time algorithm for.
You can't prove that the polynomial time algorithm or polynomial monomial time algorithm gives you the optimal solution in all cases.
And then you go off into complexity theory.
You maybe discover that, or show that this problem is NP-complete, and now you're in the intractability domain.
So very small changes in problem statements can end up with very different situations from a standpoint of algorithm complexity.
And so that's really what I want to point out to you in some detail with a concrete example.
So I want to get a little bit pedantic here with respect to intractability and tractability.
You've seen, I think, these terms before in the one lecture in 006, but we'll go over this in some detail today and more later on in the semester.
But for now, let's recall some basic terminology associated with tractability and intractability or complexity theory, broadly speaking.
Capital P is a class of problems solvable in polynomial time.
And think of that as big O, n raised to k for some constant k. Now you can have long factors in there, but once you put a big O in there, you're good.
You can always say, order n, even if it's a logarithmic problem, and big O lets you be sloppy like that.
And there are many examples of polynomial time algorithms, of course, for interesting problems like shortest paths.
So the shortest path problem is order V square, where V is the number of vertices in the graph.
There's algorithms for that.
You can do a little bit better if you use fancier data structure, but that's an example.
NP is another class of problems that's very interesting.
This is the class of problems that whose solution is verifiable in polynomial time.
So an example of a problem in NP that is not known to be NP is the Hamiltonian cycle problem.
And the Hamiltonian cycle problem corresponds to given a directed graph, find a simple cycle.
So you can repeat vertices, but you need the simple cycle to contain each vertex in V. And determining whether a given cycle is a Hamiltonian cycle or not is simple.
You just traverse the cycle.
Make sure that you've touched all the vertices exactly once, and you're done.
Clearly doable in polynomial time.
So therefore, Hamiltonian cycle is an NP, but determining whether a graph has a Hamiltonian cycle or not is a hard problem.
And in particular, the notion of NP completeness is something that defines the level of intractability for NP.
The NP complete problems are the hardest problems in NP, and Hamiltonian cycle is one of them.
If you can solve any NP complete problem in polynomial time, you can solve all problems in NP in polynomial time.
So that's what I meant by saying that NP complete problems are, in some sense, the hardest problems an NP because solving one of them gives you everything.
So the definition of NP completeness is that the problem is in NP and is as hard-- an informal definition-- as any problem in NP.
And so Hamiltonian cycle is an NP complete problem.
Satisfiability is an NP complete problem, and there's a whole bunch of them.
So going back to our theme here, what I want to show you is how for an interval scheduling problem, that I'll define in a couple of minutes, how we move from linear time, therefore P, to something that's still in P. But it's a little more complicated if I change the constraints of a problem a little bit.
And finally, if I add more constraints to the problem, generalize it-- and you can think of it as adding constraints or generalizing the problem-- you get small changes to something that becomes NP complete.
So this is something that algorithm designers have to keep in mind because before you go off and try to design an algorithm for a problem you like to know where in the spectrum your problem resides.
And in order to do that, you need to understand algorithm paradigms obviously and be able to apply them, but you also have to understand reductions where you can try and translate one problem to another.
And if you can do that, and the first problem is known to be hard, then you can make arguments about the hardness of your problem.
So these are the kinds of things that we'll touch upon today, the analysis of an algorithm, the design of an algorithm, and also the complexity analysis of an algorithm, which may not just be an asymptotic-- well, this is order n cubed or order n square but more in the realm of NP completeness as well.
So so much for context, let's dive into our interval scheduling problem, which is something that you can imagine doing for classes, tasks, a particular schedule during a day, life in general.
And in the general setting, we have resources and requests, and we're going to have a single resource for our first version of the problem.
And our requests are going to be 1 through n, and we can think of these requests as requiring time corresponding to the resource.
So the request is for the resource, and you want time on the resource.
Maybe it's computation time.
Maybe it's your time.
It could be anything.
Each of these requests responds to an interval of time, and that's where the name comes from.
si is start time time.
fi is the finish time, and we're going to say si is strictly less than fi.
So I didn't put less than or equal to there because I want these requests to be non-null, non-zero, so otherwise they're uninteresting.
And we're going to have a start time, and we're going to have an end time, and they're not equal.
So that's the first part of the specification of the problem and then the second part, which is intuitive is that two requests-- we have a single resource here remember-- i and j are considered to be compatible, which means you can satisfy both of these requests.
They're compatible.
Incompatible requests, you can't satisfy with your single resource simultaneously-- Provided they don't overlap.
And an overlapping condition might be that fi is less than or equal to sg, or fj less than or equal to si.
So again, I put a less than or equal to here, which is important to spend a minute on.
What I'm saying here in this context is that I really have open-ended intervals on the right-hand side corresponding to the fi's.
So pictorially, you could look at it this way.
Let's say I have intervals like this.
So this is interval number 1.
That's interval number 2.
Right here I have s of 1, f of 1 out here, s of 2 out here, and f of 2 out here.
So this is f of 1 for that and s of 2 for this.
I'm allowing s of 2 and f of 1 to be exactly equal, and I still agree that these two are compatible requests.
So this is-- I guess it's terminology.
It's our definition of compatibility.
So you can imagine now an optimization problem that is associated with interval scheduling where, in a different example, I have this interval corresponding to s1 and f1.
I might have a different interval here corresponding to 2, then corresponding to 3.
And then maybe I've got 4 here, 5, and 6.
So those are my six intervals corresponding to my input.
I have a single resource.
I'm just drawn out in a two-dimensional form.
There's six different requests that I have, the six different intervals.
Intervals and requests are synonyms.
And my goal here-- and it's kind of obvious in this example-- is to select a compatible subset of requests, or intervals, that is of maximum size.
And I'd like to do this efficiently.
So we'll always consider efficiency here, but in terms of the specification of the problem as opposed to a requirement on the complexity of the algorithm, I want maximum size for this subset.
So as I showed you, or I mentioned earlier, in this case, it is clear from the drawing that I put up there that the maximum size for that six requests example that I have is three.
So that's the set up.
Now we're going to spend the next few minutes talking about a greedy strategy for solving this particular problem.
If you don't know of it, the greedy strategy is going to always produce the maximum size or not.
In fact, it depends on the particular greedy heuristic, the selection heuristic that a greedy algorithm uses.
So that's going to be important, and we'll take a look-- and hopefully you can suggest some-- at a few different greedy heuristics.
But my claim, overall claim, that I'm going to have to spend a bunch of time here justifying and eventually proving is that we can solve this problem using a greedy algorithm.
Now what is a greedy algorithm?
You've seen some examples.
As the name implies, it's something that's myopic.
It doesn't look ahead.
It looks to maximize the very first thing that you couldn't maximize.
It says-- traffic is a good example-- don't let anybody cut in front of you.
You've got some room up there.
Get up there.
Generally, people are greedy when it comes to getting to work, trying to minimize the time and, in this case, on the time that they spend on the road.
But we've had other examples.
For example, when you look at interval scheduling, you might say, I'm going to pick the smallest request.
And I'm going to pick the smallest request first, and I'm going to try and collect together as many requests as possible.
And if the requests are small in the sense that si and fi, for the two requests, are close to each other, then maybe that's the best strategy.
So that's an example of a greedy strategy for our particular example.
But just to give you a slightly better definition of greedy than what I've said so far, a greedy algorithm is a myopic algorithm that does two things.
It processes the input one piece at a time with no apparent look ahead.
So what happens is that greedy algorithms are typically quite efficient.
What you end up doing is looking at a small part of the problem instance and deciding what to do.
Once you've done that, then you're in a situation where the problem has gotten a little bit simpler because you've already solved part of it, and then you move on.
So what would a template for a greedy algorithm look like for our interval scheduling problem?
Here's a template that probably puts it all together and gives you a good sense of what I mean by greedy, at least in this context.
So before we even get into particulars of selection strategies, let me give you a template for greedy interval scheduling.
So step 1, use a simple rule to select a request.
And once you do that, if you selected a particular request-- let's say you selected 1.
What happens now once you've selected 1?
Well, you're done.
You can't select 2.
You can't select 3.
You can't select 4.
You can't select 5.
You can't select 6.
So if you have selected 1 in this case, you're done, but we have to codify that in a step here.
And what that means is that we have to reject all requests that are incompatible with i.
And at this point, because we've rejected a bunch of requests, our problem got smaller.
And so you now have a smaller problem, and you just repeat-- go back to step 1-- until all requests are processed.
All right, so that's a classical template for a greedy algorithm.
You just go through these really simple steps.
And the reason this is a template is because I haven't specified a particular rule, and so it's not quite an algorithm that you can code yet because we need a rule.
So with all of that context, let me ask you.
What is a rule that you think would work well for an interval scheduling problem?
Yeah, go ahead
Select one with the earliest finish time
Select one with the earliest finish time.
All right, well, I did not want that answer.
[LAUGHTER] But now that you've given me the answer, I have to do something about this.
So I want a different answer, so we'll go to a different person.
But before I do that, let me reward you for that answer I did not want with a limited edition 6046 Frisbee, OK?
[APPLAUSE] You need to stand up because I don't want to take people's heads off.
[LAUGHTER] Yeah sorry.
All right, so here you go.
All right?
Good.
[APPLAUSE] So people do cookies and candy.
I think Eric, Nancy and I are cooler.
[LAUGHTER] So we do Frisbees.
All right, good, so the fact of the matter was that this class was scheduled for 9:30 to 11:00 on Tuesdays and Thursdays.
That's when we decided to do Frisbees.
And then it got shifted over to 11:00 to 12:30, but then we bought all these Frisbees, so we said, whatever.
It's not like we could use all of them All right, good.
So I don't like that answer, and I want a different one.
Give me another one.
Yeah, go ahead
Just carry it through in numerical order
I'm sorry?
Just carry it through in numerical order
Carry it through in numerical order.
Is that going to work?
And what's an example that it didn't work?
The one right there, right?
Should I get her a Frisbee?
We should.
I'm going to be generous at the beginning.
You can just-- But that's an answer I liked.
Yeah, there you go.
So entering through a numeric order isn't going to work.
This is a great example right there.
Give me another one.
[LAUGHTER] There are no Frisbees right here.
Over there, yeah?
Try the one with the shortest time
Ah, try the one with the shortest time.
OK, so the shortest time in this case might be this one.
The shortest time might be this one, and, hey, that might work in this case because you pick this one, which is the shortest, or maybe it's five, which is the shortest.
Either way, you could get 2, 5, and 6, looking at this picture, seems to work.
Maybe 4, 5, and 6 if you pick 5 first, et cetera, right?
I'll give you a Frisbee if you can take that same algorithm and give me a counter example
Let's say you have two requests which don't overlap, and then there's--
--there's one right in the middle, exactly right.
Yep, so let's see.
What do I do?
Oh, here.
So pictorially, a really you can look at this, and you can actually figure out whether your heuristic works or not.
But this, I think, what you were thinking about.
There you go, right?
So you get one.
So that clearly doesn't work.
So this one was smallest, doesn't work.
The suggestion here was a numeric.
It doesn't work.
Here's one that might actually work.
For each request, find the number of incompatible requests.
So you've got a request.
You can always intersect the other requests with it and decide whether the second request is compatible or not, and you do this for every other request.
And you can collect together numbers associated with how many incompatible requests a particular request has, and you say, well, let me use that as a heuristic.
So each request, find number of incompatible requests and select the one with the minimum number of incompatibles.
So just to be clear, in this case, you would not select 1 because clearly 1 is incompatible with every other request, so that clearly is not numeric order.
In this case, you would not select this one because it's incompatible with this one and that one.
So you'd select that one which has the minimum number of incompatibles.
So you think this is going to produce the correct answer, the maximum answer, in every possible case?
No
No, who said, no?
Well, anybody who said, no, should give me a counter example.
Yeah, go for it
If the one that it selects has mutually incompatible collection of intervals with which it's compatible
Right, so that's a good thought.
We'll have to [INAUDIBLE] that.
And I think this particular example, that's exactly what you said, which just instantiates your notion of mutual incompatibility.
So here's an example where I have something.
It's a little more complicated.
As you can see, this is a pretty good heuristic.
It's not perfect as you can see from this example, where I have something like this.
So if you look at this, what I have here is I have just a bunch of requests which have-- this is incompatible with this and that and these two, so clearly a lot of incompatibilities for these, a lot incompatibilities for these.
Which is the minimum?
The one in here, but what happens if you select that?
Well, clearly you don't get this solution, which is optimal.
The one on top, so this is a bad selection.
And so this doesn't work either, OK?
There you go.
So as it turns out, the reason I didn't like that first answer was it was correct.
[LAUGHTER] It's actually a beautiful heuristic.
Earliest finish time is a heuristic that is-- well, it's not really a heuristic in the sense that if you use that selection rule, then it works in every case.
In every case, it's going to get to you the maximum number, OK?
Earliest finished time so what does that mean?
Well, it just means that I'm going to just scan the f of i's associated with the list of requests that I have, and I'm going to pick the one that is minimum.
Minimum f of i means earliest finish time.
Now you can just step back, and I'm not going to do this for every diagram that I have up here, but look at every example that I've put up.
Apply the selection rule associated with earliest finish time, and you'll see that it works and gets you the maximum number.
For example, over here, this has the earliest finish time.
Not this, not this, it's over here.
So you pick that, and then you use the greedy algorithm step 2 to eliminate all of the intervals that are incompatible, so these go away.
Once this goes away, this one has the earliest finish time and so on and so forth.
So this is something that you can prove through examples.
That's not really a good notion when you can prove to yourself using examples.
And this is where I guess is the essence of 6046, to some extent 006 comes into play.
We will have to prove beyond a shadow of a doubt using mathematical rigor that the earliest finish time selection rule always gives us the maximum number of requests, and we're going to do that.
It's going to take us a little bit of time, but that's the kind of thing you will be expected to do and you'll see a lot of in 046.
OK?
So everyone buy earliest finish time?
Yep, go ahead
So what if we consider the simple path example of there's one request for the whole block, and there's one small request that it mentioned earlier
Well, you'll get one for-- if there's any two requests, your maximum number is 1.
So you pick-- it doesn't matter-- it's not like you want efficiency of your resource.
In this particular case, we will look at cases where you might have an extra consideration associated with your problem which changes the problem that says, I want my resource to be maximally utilized.
If you do that, then this doesn't work.
And that's exactly-- it's a great question you asked.
But I did say that we were going to look at the team here, which I don't have anymore, but of how problems change algorithms.
And so that's a problem change.
You've got a question
I have a counter example.
You have three intervals that don't conflict with one another.
You have one interval that conflicts with the first two and ends earlier than the first one
OK, so are you claiming that there's going to be a counter example to earliest finish time?
Yes
All right, I would write it down on a sheet of paper.
And get me a concrete example, and you can just slide it by.
And if you get that before I finished my proof, you win, OK?
[LAUGHTER] So I would write it down.
Just write it down, so good.
All right, so this is a contest now.
[LAUGHTER] All right, so we are going to try and prove this.
So there's many ways you could prove things, and I mean prove things properly.
And I don't know if you've read the old 6042 proof techniques that are invalid, which is things like prove by intimidation, proof because the lecturer said so, you know, things like that.
This is going to be a classical proof technique.
It's going to be a proof by induction.
We're going to go into it in some detail.
Later on in the term we are going to put out sketches of proofs.
We are going to be skipping steps in lecture that are obvious or maybe not so obvious, but if you paid attention, then you can infer the middle step, for example.
And so will be doing proof sketches, and proof sketches are not sketchy proofs.
[LAUGHTER] So keep that in mind.
But this particular proof that we're going to do, I'm going to put in all the steps because it's our first one.
And so what we're going to do here is prove a claim, and the claim is simply that-- whoops, this is not writing very well.
What is going on here?
OK. [LAUGHTER] Back to the white.
Given a list of intervals l, our greedy algorithm with earliest finish time produces k star intervals where k star is minimal.
So that's what we like to prove
[INAUDIBLE]
Sorry, what happened?
[INAUDIBLE]
Oh, right.
Good point.
Maximum.
What we're going to do is prove this by induction, and it's going to be induction on k star.
And so the base case is almost always with induction proofs trivial, and it's similar here as well.
And in the base case, if you have a single interval in your list, then obviously that's a trivial example.
But what I'm saying here for the base is slightly different.
It says that the optimal solution has a single interval, right?
And so now if your problem has one interval or two intervals or three intervals, you can always pick one, and it's clearly going to be a valid schedule because you don't have to check compatibility.
And so the base case is trivial even in the case where you're not talking just of intervals that have cardinality 1, but the optimal schedule has cardinality 1.
So that's a trivial case.
So the hard work, of course, in the induction proofs is assuming the hypothesis and proving the n-plus-1, or in this case, the k-star-plus-1 case.
And that's what we'll have to work on.
So let's say that the claim holds for k star, and we are given a list of intervals who's optimal schedule is k star plus 1.
It has k-star-plus-1 intervals in the optimal schedule, so L may be some large number, capital L, maybe in the hundreds.
And k star, there may be 10 of what have you.
They're different.
I want to point that out.
So our optimal schedule, we're going to write out as this, s star.
So usually if you use star for optimal in 046 and it's got k-star-plus-1 entries, and those entries look like sf pairs-- so I'm going to using the subscript j1 through j k star plus 1 to denote these intervals.
So the first one is sj1, fj1.
That's an interval that's been selected and is part of our optimal solution.
And then you keep going and we have sj k star plus 1 comma fj k star plus 1.
So no getting away from subscripts here in 046 So that's what we have in terms of this is what the optimal schedule is.
It's got size k star.
Of course, what we have to show is that the greedy algorithm with the earliest finish time is going to produce something that is k star plus one in size.
And so that's the hard part.
We can assume the inductive hypothesis, and we'll have to do that.
But there's a couple of steps in between.
So let's say that what we have is s1 through k is what the greedy algorithm produces with the earliest finish time.
So I'm going to write that down sik fik, so notice I have k here, and k and k star, at this point, are not comparable.
I'm just making a statement that I took this particular problem that has k star plus 1 in terms of its optimal solution size, and for that problem, I have k intervals that are produced by the earliest finish time greedy heuristic.
And so that's why the subscripts here are different.
I have i1 here and ik, and then over here I have the j's, and so these intervals are different.
If I look at f of i plus f of i1, and if I look f of j1, what can I say about f of i1 and f of j1?
Is there a relationship between f of i1 and f of j1?
They're equal?
Do they have to be equal?
Yeah?
Less or equal to
Less than equal to, exactly right, so they're less than equal to.
It's possible that you might end up with a different optimal solution that doesn't use the earliest finish time.
We think earliest finish time is optimal at this point.
We haven't proven it yet, but it's quite possible that you may have other solutions that are optimal that aren't necessarily the ones that earliest finish time gives you.
So that's really why the less than or equal to is important here.
Now what I'm going to do is create a schedule, s star star, that essentially is going to be taking s star and pulling out the first interval from s star and substituting it with the first interval from my greedy algorithms schedule.
So I'm just going to replace that, and so s star star is si1 fj1.
And then I'm going to be going back to sj2 fj2 because I'm going back to s star and all the other ones are coming from s star.
So they're going to be sj k star plus 1 comma fj k star plus 1.
So I just did a little substitution there associated with the optimal solution, and I stuck in part of the greedy algorithm solution, in fact, the very first schedule
So the 1 should be i1
Oh, this should be-- i1,
Right?
i1, thank you.
Yep, good.
So we've got a couple of things to do, a couple of observations to make, and we're going to be able do prove some relationship between k and k star that is going to give us the proof for our claim.
So clearly, s star is also optimal.
All I've done is taken one interval out and replaced it with another one.
It hasn't changed the size.
It goes up to k star plus 1, so s double star is also optimal.
s star is optimal.
s double star is optimal.
Now I'm going to define L prime as the set of intervals with s of i greater than or equal to f of i1.
So what is L prime?
Well, L prime is what happens in the second step of the greedy algorithm, where in the second step of the greedy algorithm, once I've selected this particular interval and I've pull it in, I have to reject all of the other intervals that are incompatible with this one.
So I'm going to have to take only those intervals for which s of i is greater than or equal to f of i1 because those are the ones that are compatible.
So that's what L prime is.
And I'm going to be able to say that since s double star is optimal for L, s double star 2 to k star plus 1 is optimal for L prime.
So I'm making a statement about this optimal solution.
I know that's optimal, and basically what I'm saying is subsets of the optimal solution are going to have to be optimal because if that's not the case, I could always substitute something better and shrink the size of the k star plus 1 optimal solution, which obviously would be a contradiction.
So s double star is optimal for L, and therefore s double star 2 through k star plus 1 is optimal for L prime.
Everybody buy that?
Yep?
Good.
And so what this means, of course, is that the optimal schedule for L prime has k star size.
And I'm starting with 2.
I've taken away 1.
So now I have L prime, which is a smaller problem.
Now you see where the proof is headed, if you didn't already.
I have a smaller problem, which is L prime.
Clearly, it's got fewer requests, and I have constructed an optimal schedule for that problem by pulling it out of the original optimal schedule I was given.
And that size of that optimal schedule is k star.
And now I get to invoke my inductive hypothesis because my inductive hypothesis says that this claim that I have up there holds for any set of problems that have an optimal schedule of size k star.
That's what the inductive hypothesis gives me.
And so by the inductive hypothesis, when I run the greedy algorithm on L prime, I'm going to get sk schedule of size k star.
Now can you tell me, based on what you see on the board, by construction, when I run the greedy algorithm, what am I getting on L star?
By construction, when I run the greedy algorithm on L prime-- there's too many superscripts here-- when I run the greedy algorithm on L prime, what do I get?
Someone?
Yeah?
We get s of i sub 2, s of i sub 2 interval
Exactly right, I get everything from the second thing here all the way to the end because that's exactly what the greedy algorithm does.
Remember, the greedy algorithm picked si1 fi1, and then rejected all requests that are incompatible and then move on.
When you rejected all requests that are incompatible here, you got exactly L prime.
And by construction, the greedy algorithm should have given me all the way from si2 too sik.
Thank you.
So by construction, the greedy on L prime gives s2 to k, right?
And what is the size of this?
2 to k gives me a size of k minus 1.
This is k minus 1.
So if I put these two things together, what is the next step?
I have the inductive hypothesis giving me a fact.
I have the construction giving me something.
Now I can relate k and k star.
What's the relationship?
k star is equal to k minus 1, right?
Do people see that?
So size k star or just k minus 1.
So what that means is given that s2k is a size k star, it means that s1k is of size k star plus 1, which is exactly what I want.
That's optimal because I said in the beginning that we had k star plus 1 in our inductive hypothesis this case as being the optimal solution.
So this last step here is all you need to argue now that s of 1k, going back up here, this is optimal because k equals k star plus 1.
There you go, so that's the kind of argument that you have to make in order to prove something like this in 046.
And what you'll see in your problem sets, including the one that's going to come out on Thursday, is that different problem that you have to have proof for a greedy algorithm for.
I forget exactly what technique you'll have used there, perhaps induction, perhaps contradiction.
And these are the kinds of things that get you to the point where you've analyzed the correctness of algorithms, not just the fact that you're getting a valid schedule, but you're getting a valid maximum schedule in terms of the maximum number of requests.
Any questions about this?
Do people buy the proof?
Yep.
Good.
So that was greedy for a particular problem.
I told you that the team of our lecture here was changing the problem and getting different algorithms that had different complexities.
So let's go ahead and do that.
So the rest of this lecture, we'll just take a look at different kinds of problems and talk a little more superficially about what the problem complexities are.
And so one thing that might come to mind is that you'd like to do weighted interval scheduling.
And what happens here is each request has weight wi, and what we want to do is schedule a subset of requests with maximum weight.
So previously, it was just all weights were 1, so maximum cardinality was what we wanted.
But now we want to schedule a subset of requests with maximum weight.
Someone give me an argument as to whether the greedy algorithm earliest finish time first is optimal for this weighted case, or give me a counter example.
Yep, go ahead
Oh, well, you know like your first example you have your first weight of the first interval, it took the whole time, [INAUDIBLE] would have three smaller ones?
Well, if the weight of the first one was 20 and then--
Exactly, exactly right.
All right, I owe you one too.
So here you go.
So it's a fairly trivial example.
All you do is w equals 1, w equals 1, w equals 3, so there you go.
So clearly, the earliest finish time would pick this one and then this one, which is fine.
You get two of these, but this was important.
This is, I don't know, sleep party, 6046.
[LAUGHTER] So there you go.
So the weight it is, we should make that infinity.
Most important thing in the world at least for the next six months.
So how does this work now?
So it turns out that the greedy strategy, the template that I had, fails.
There's nothing that exists on this planet that, at least I know of, where you can have a simple rule and use that template to get the optimum solution, in this case, maximum weight solution, for every problem instance, so that template just fails.
What other programming paradigm do you think would be useful here?
Yeah, go ahead

DP, right.
So do you want to take a stab at a potential DP solution here?
Yeah, so either include it in your [INAUDIBLE] or discard it and then continue with set of other intervals
Yeah, that's a perfect divide and conquer.
And then when you include it, what do you have to do?
Eliminate all conflicting intervals
Right, how many subproblems do you think there are.
I want to make you own your Frisbee, right?
[LAUGHTER]
2 to the power of the number of intervals you have because--
Well, that's a number of subsets that you have.
So you have n intervals, then you have two [INAUDIBLE] subsets
Yeah
But remember, you want to go-- you want to be smarter than that, right?
You want to be a little bit smarter than that.
So here, you get a Frisbee anyway.
[LAUGHTER] No, not anyway, here you go.
Right.
So anybody else?
So what I want to use is dynamic programming.
We've established that.
I want to use dynamic programming.
And the dynamic programming-- you have some experience with that in 006-- the name of the game is to figure out what the subproblems are.
The subproblems are kind of going to look like a collection of requests.
I mean, there's no two things about it.
They're going to be a collection of requests, and so the challenge here is not to go to the 2 raised to n, because 2 raised to n is bad if you want efficiency.
So we have to have a polynomial number of subproblems.
So someone who hasn't answered yet, go ahead
[INAUDIBLE] so [INAUDIBLE] subset [INAUDIBLE] So from interval i to interval j [INAUDIBLE]
So you're looking at every pair of i's and j's, and, well, not all of them are going to be valid.
There won't be intervals associated with that, but that's a reasonable start.
Someone else, someone who hasn't answered?
Yeah, back there
You could go the best term to start to some even point, and so there'd n of those
Ah, best from the start to any given point.
All right, well, you got close, Michael.
There you go.
You need to stand up.
Ew, bad throw.
That's a bad throw.
I've got to practice.
OK, so as you can see with dynamic programming, the challenge is to figure out what the subproblems are.
The fact of the matter is that there's going to be many different possible algorithms that are all DP for this weighted problem.
There's at least two interesting ones.
We're going to do a simple one, which is based on the answer that the gentleman here just gave.
But it turns out you can be a little smarter than that, and most likely you'll hear the smarter way in the section, but let's do the simple one because that's all I have time here for.
And the key is to define the subproblems, and then once you do that, the actual recursion ends up being a fairly straightforward and intuitive step.
So let's look at dynamic programming, one particular way of solving this problem, using the DP paradigm.
So what I'm going to do is define subproblems R star, so R is the total number of requests that we have, and the subproblems are going to correspond to-- I'm going to request j belonging to R such that-- oh, I'm sorry.
This is R of x-- such that sj is greater than or equal to x.
So what I'm doing here is, given a particular x, I can always shrink the number of requests that I have based on this rule.
And then you might ask, what is x?
And now you can apply the same subsetting property by choosing the x's to be the finishing times of all of the other requests.
All right, so x equals f of i.
So what this means is-- then I put f of i over here-- it means all of the requests that come after the i-th request finished our part of R of fi.
So R of fi would simply be requests later than f of i.
And there's something subtle here that I want to point out, which is R of fi is not the set of requests that are compatible with the i-th request.
It's not exactly that.
It's the set of requests that are later than f of i.
So keep that in mind because what happens here is we're going to solve this problem step by step.
We're going to construct the dynamic programming solution essentially by picking a request, just like in the greedy case, and then taking the request that comes after that.
So we're going to pick an early request, and then we're going to subset the solution, pick the next one just like we did with the greedy.
And so the subproblems that we will actually solve potentially bottom up if we are doing recursion are going to correspond to a set of requests that come later than the particular subset that we're looking at, which is defined by a particular interval.
So requests that are later than f of i, not necessarily all of the requests that are compatible with the i-th request.
And so if you do that, then the number of subproblems here are small n, where n is the number of requests.
So if n is the number of requests in the original problem, the number of sub problems equals n because all I do is plug-in an appropriate i, find f of i for it, and generate the R of f of i for each of those.
So there's going to be n of those subproblems.
And we're going to solve each subproblem once and then memoize.
And so the work that we have to do is the basic rule corresponding to the complexity of a DP, which is number of subproblems times the time to solve each subproblem, or a single subproblem, and this assumes order 1 for lookups.
So you can think of the recursive calls as being order 1 because your assuming you're doing memoization.
So I haven't really told you anything here that you haven't seen in 006 and likely applied a bunch of times.
Over here, we've just defined what our subproblems are for our particular DP, and we argued that the number of subproblems that are associated with this particular choice of subproblems corresponds to n if you have n requests in the original problem instance that you've given.
So the last thing that we have to do here to solve our DP is, of course, to write our recursion and to convince ourselves that this actually all works out, and let's do that.
And so what we have here is our DP guessing.
And we're going to try each request i as a plausible first request, and so that's where this works.
You might be thinking, boy, I mean, this R of fi looks a little strange.
Why doesn't it include all of the requests that are compatible with the i-th request?
I mean, I'm somehow shrinking my subsequent problem size if I'm ignoring some requests that are earlier that really should be part of-- or are part of the compatible set, but they're not part of the R of fi set.
And so some of you may be thinking that, well, the reason this is going to work out is because we are going to construct our solution, as I said before, from the beginning to the end.
So we're going to try each request as a plausible first request.
So even though this request might be in our chart all the way to the right, it might have a huge weight, and so I'm going to have to try that out as my first selection.
And when I try that out as my first selection, then the definition of my subproblem says that this will work.
I only have to look at the request that comes later than that because the ones that came the earlier, I've tried them out too.
So that's something that you need to keep in mind in order to argue correctness of this recursion that I'm going to write out now.
And so the recursion, and I have opt R, what is the first thing that I'm going to have on the right-hand side of this recursive formulation?
What mathematical construct am I going to have to do here?
And you see something like guessing and seeing something like try each request as a possible first, what mathematical construct am I going to have to put up here?
Max
Max, who said max?
No one wants to take credit for max?
It's max, right?
So I'm going to have max 1 less than equal to i less than or equal to n. And I'm going to-- does someone want to tell me what the rest of this looks like?
Someone else?
A couple Frisbees left, guys.
[LAUGHTER] What does the rest of this look like?
Yep?
1 plus the optimal R f of--
Not 1, just what kind of problem do we have here?
It's not 1 anymore
Oh--
The weight
Right
The weight, yep, so Wi plus the optimal R fi.
OK, so we got Wi plus optimum of R of fi.
And you said "1," close enough.
If it was 1, you'd use greedy.
And so that's why we were in that Wi mode, and we end up getting this here.
So that's it.
You try every request as a possible first.
Obviously, you pick that request so it's part of your weight in terms of the weight for your solution.
When you do that, because it was the first request, you get to prune the set of requests that come later corresponding to R of fi that you see here.
And then you go ahead and simply find the optimum for a smaller problem, clearly has fewer requests.
And as long as you maximize over the set of guesses that you've taken, and there's n guesses up at the top level.
Obviously in the lower levels, you're going to have fewer requests in your R of fi's, and you'll have fewer durations of the max, but it's n at the top level.
So one last question, what is the complexity of what we see here?
n square
n square, and the reason it's n square is you simply use-- you can be really mechanical about this-- you say, if this was order 1, I'm doing a max over n items.
And therefore, that's order n time to solve one subproblem.
And since I have n subproblems, I get n times order in, which is order n squared.
So the last thing I'll do-- and I just have one more minute-- is give you a sense of a small change to interval scheduling that puts us in that NP complete domain.
So so far, we've just done two problems.
There's many others.
We did interval scheduling.
There was greedy linear time.
Weighted interval scheduling is order n squared according to this particular DP formulation.
It turns out there's a smarter DP formulation that runs an order n log n time that you'll hear about in section on Friday, but it's still polynomial time.
Let's make one reasonable change to this, which is to say that we may have multiple resources, and they may be non identical.
So it turns out everything that we've done kind of extrapolates very well to identical machines, even though there's many identical machines.
But if you have non-identical machines, what that means is you have resources or machines that have different types.
So maybe your machines are T1 to Tm.
And it's essentially a situation where you say, this particular task can only be run on this machine or this other machines, some subset of machines.
So you can still have a weight of 1 for all requests, but you have something like A of i belonging subset of T is a set of machines that i runs on.
OK, that's it.
That's the change we make.
Q of i is going to be specified for each of the i's.
So you could even have two machines.
And you could say, here's a set of requests that could run on both machines.
Here's a set that only runs on the first machine, and here's another set that runs on the second machine.
That's a simple example of this generalization.
If you do this, this problem has been shown to be NP complete.
And by that I mean, NP complete problems are decision problems.
And so you say, can some specific number k less than and requests be scheduled.
This decision problem is NP complete.
And so what happens when you have NP complete problems?
Well, we're going to have a little module in the class that deals with intractability.
We're going to look at cases where we could apply approximation algorithms, and maybe in the case of the optimization problem, if the optimum for this is k star, I will say that we can get within 10% of k star.
The other way is to just deal with intractability by hoping that your exponential time algorithm runs in a reasonable amount of time for common cases.
So in the worst case, you might end up taking a long time.
But you just sort of back off after an hour and take what you get from the operative algorithm.
But in many cases, the algorithm might actually complete, and they give you the optimum solution.
So done here.
Make sure to sign up for a recitation section.
And see you guys next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
Let's get started.
So we're going to start 6.046 in earnest today.
We're going to start with our first module on divide and conquer.
You've all seen divide and conquer algorithms before.
Merge sort is a classic divide and conquer algorithm.
I'm going to spend just a couple minutes talking about the paradigm, give you a slightly more general setting than merge sort.
And then we'll get into two really cool divide and conquer problems in the sense that these are problems for which divide and conquer works very well-- mainly, convex hall and median finding.
So before I get started on the material, let me remind you that you should be signing up for a recitation section on Stellar.
And please do that even if you don't plan on attending sections.
Because we need that so we can assign your problem sets to be graded, OK?
So that's our way of partitioning problem sets as well.
And then the other thing is problem set one is going to go out today.
And that it's a one week problem set.
All problem sets are going to be a week in duration.
Please read these problem sets the day that they come out.
Spend 5, 10 minutes reading them.
Some things are going to look like they're magic, that they're-- how could I possibly prove this?
If you think about it for a bit, it'll become obvious.
We promise you that.
But get started early.
Don't get started at 7:00 PM when we have 11:59 PM deadline on Thursday, all right?
That four hours or five hours of time may not be enough to go from magical to obvious, OK?
So let's get started with the paradigm associated with divide and conquer.
It's just a beautiful notion that you can break up the problem into smaller parts and somehow compose the solutions to the smaller parts.
And of course, the details are going to be what's important when we take a particular problem instance.
But let's say we're given a problem of size n. We're going to divide it into a sub problems-- I'll put that in quotes so you know it's a symbol-- a sub problems of size n over b.
And here, a is an integer.
And a is going to be greater than or equal to 1.
It could be two.
It could be three.
It could be four.
This is the generalization I alluded to.
And b does not have to be two or even an integer.
But it has to be strictly greater than one.
Otherwise, there's no notion of divide and conquer.
You're not breaking things up into smaller problems.
So b should be strictly greater than one.
So that's the general setting.
And then you'll solve each sub problem recursively.
And the idea here is that once the sub problems become really small, they become constant size, it's relatively easy to solve them.
You can just do exhaustive search.
If you have 10 elements and you're doing effectively a cubic search, well, 10 cubed is 1,000.
That's a constant.
You're in great shape as long as the constants are small enough.
And so you're going to recurse until these problems get small.
And then typically-- this is not true for all divide and conquer approaches.
But for most of them, and certainly the ones we're going to cover today, the smarts is going to be in the combination step-- when you combine these problems, the solutions of these sub problems, into the overall solution.
And so that's the story.
Typically, what happens in terms of efficiency is that you can write a recurrence that's associated with this divide and conquer algorithm.
And you say t of n, which is a running time, for a problem of size n is going to be a times tfn over b-- and this is a recurrence-- plus the work that you need to do for the merge operational or the combine.
This is the same as merge.
And so you get a recurrence.
And you're not quite done yet in terms of the analysis.
Because once you have the recurrence, you do have to solve the recurrence.
And it's usually not that hard and certainly it's not going to be particularly difficult for the divide and conquer examples that we're going to look, at least today.
But we also have this theorem that's called the master theorem that is essentially something where you can fairly mechanically plug in the a's and the b's and whatever you have there-- maybe it's theta n, maybe it's theta n square-- and get the solution to the recurrence.
I'm actually not going to do that today.
But you'll hear once again about the massive theorem tomorrow in section.
And it's a fairly straightforward template that you can use for most of the divide and conquer examples we're going to look at in 046 with one exception that we'll look at in median finding today that will simply give you the solution to the recurrence, OK?
So you've see most of these things before.
That's a little bit of setup.
And so let's dive right in into convex hull, which is my favorite problem when it comes to using divide and conquer.
So convex hull, I got a little prop here which will save me from writing on the board and hopefully be more understandable.
But the idea here is that in this case, we have a two dimensional problem with a bunch of points in a two dimensional plane.
You can certainly do convex hull for three dimensions, many dimensions.
And convexity is something that is a fundamental notion in optimization.
And maybe we'll get to that in 6046 in advanced topics, maybe not.
But in the context of today's lecture, what we're interested in doing is essentially finding an envelope or a hull associated with a collection of points on a two dimensional plane.
And this hull obviously is going to be something, as you can guess, that encloses all of these points, OK?
So what I have here, if I make this string taut enough-- this is not working so well, but I think you get the picture.
All right, so that's not a convex hull.
This is not a convex hull for the reason that I have a bunch of points outside of the hull.
All right, so let me just-- that is a convex hull.
And now if I start stretching like that or like this or like that, that's still a convex hull, OK?
So that's the game.
We have to find an algorithm.
And we look at a couple of different ones that will find all of these segments that are associated with this convex hull, OK?
So this is a segment that's part of the convex hull.
That's a segment that's part of the convex hull.
If, in fact, I had something like this-- and this was stretched out-- because I have those two points outside the convex hull, this may still be a segment that's part of the electronics hall but this one is not, right?
So that's-- the game here is to find these segments.
So if you're going to working with segments or tangents-- they're going to be used synonymously-- all of the tangents or segments associated with the entirety of the convex hull and we have to discover them.
And only input that we have is the set of pointx-- xiy coordinates.
And there's just a variety of algorithms that you can use to do this.
The one that I wish I had time to explain but I'll just mention is what's called a gift wrapping algorithm.
You might not have done this, but I guarantee you I said you probably have taken a misshapen gift, right, and tried to wrap it in gift wrapping paper.
And when you're doing that, you're essentially-- if you're doing it right you're essentially trying to find the convex hull of this three dimensional structure.
You're trying to tighten it up.
You're trying to find the minimum amount of gift wrapping paper.
I'm not sure if you've ever thought about minimizing gift wrapping paper, but you should have.
And that's the convex hull of this three dimensional shape.
But we'll stick to two dimensions because we'll have to draw things on the board.
So let me just spec this out a bit.
I've been given endpoints in a plane.
And those set of points are s, xi, yi such that i equals 1, 2 to n. And we're just going to assume here, just to make things easy because we don't want to have segments that are null or segments that are a little bit different because they're discontinuous.
But we're going to assume that no two have the same x-coordinate.
This is just a matter of convenience.
And no two have the same y-coordinate.
And then finally, no three in a line.
Because we want to be able to look at pairs of points and find these segments.
And it just gets kind of inconvenient.
You have to do special cases if there of them are on a line.
And so the convex hull itself is the smallest polygon containing all points in s. And we're going to call that ch of s-- convex hull of s
Smallest convex polygon
The smallest convex polygon-- thank you.
And so just as an example on the board, when you have something like this, you're going to have your convex hull being that.
This one is inside of it.
These two points are inside of it.
And all the other ones form the hull.
And so we might have p, q, r, s, t, u.
And v and x are inside of the hull.
They're not part of the specification of ch of s, which I haven't quite told you how we're going to specify that.
But the way you're going to specify that is simply by representing it as a sequence of points that are on the boundary on the hull in clockwise order.
And you can think of this as being a doubly linked list in terms of the data structure that you'd use if you coded this up.
So in this case, it would be p to q to r to s. You're going to start with t in this case.
It's a doubly linked list.
So you could conceivably start with anything.
But that's the representation of the convex hull.
And we're going to use clockwise just because we want to be clear on as to what order we're enumerating these points.
It's going to become important when we do the divide and conquer algorithm.
So let's say that we didn't care about divide and conquer just for the heck of it and I gave you a bunch of points over here.
Can you think of a simple-- forget efficiency for just a couple of minutes.
Can you think of a simple algorithm that would generate the segments of the convex hull?
For example, I do not want to generate this segment vx.
If I think of a segment as being something that is defined by two points, then I don't want to generate the segment vx because clearly the segment is not part of the convex hull.
But whereas the segment pq, qr, rs, et cetera, they're all part of the convex hull, right?
So what is the obvious brute force algorithm, forgetting efficiency, that given this set of points will generate one by one the segments of the convex hull?
Anybody?
Did you have your head up?
No?
Go ahead.
Yep
Draw the line and check how many other lines intersect with it
Draw the line and check how many lines it intersects with
Yeah
Is there-- I think you got-- you draw the line.
That's good, right?
[LAUGHS]
[LAUGHING]
Well-- but you want to do a little more.
Yeah, go ahead
For every pair of points you see, make a half-plane and see where they complete all of their other points.
[INAUDIBLE]
Ah, so that's good.
That's good.
That's good.
All right, so the first person who breaks the ice here always gets a Frisbee.
Sorry man.
At least I only hit the lecturer-- no liability considerations here.
OK, now I'm getting scared.
Right, so I think there's a certain amount of when I throw this, am I going to choke or not, right?
But it's going to get higher when one of you guys in the back answers a question.
So you're exactly right.
And you draw a line.
And then you just look at it.
And you look at the half plane.
And if all the points are to one side, it is a segment of the convex hull.
If they're not, it's not a segment-- beautiful.
All right, are we done?
Can we go and enjoy the good weather outside?
No, we've got ways to go here.
So this is not the segment whereas one-- let me draw that.
I should draw these in a dotted way.
This is not a segment.
This is not a segment.
This is a segment.
And I violated my rule of these three not being in a straight line.
So I'll move this over here.
And then that's a segment and so on and so forth, OK?
Right?
It's no longer a side with the ones below it
I'm sorry?
It would have to go directly to the bottom one from the left one
Oh, you're right.
That's a good point.
That's an excellent point.
SO what happened here was when I moved that out-- exactly right.
Thank you.
This is good.
So when I moved this out here, what happened was-- and I drew this-- well, this one here, my convex hull, changed.
The problem specification changed on me.
It was my fault.
But then what would happen, of course, is as I move this, that would become the segment that was part of the convex hull, OK?
So sorry to confuse people.
But what we have here in terms of an algorithm, if I leave the points the same, works perfectly well.
So let me just leave the points the same and just quickly recap, which is, I'm going to take a pair of points.
And I'm going to draw-- and let me just draw this in a dotted fashion first.
And I'm going to say that's the segment.
And I'm going to take a look at that line and say this breaks up the plane into two half planes.
Are all about points on one side?
And if the answer is yes, I'm going to go ahead and, boom, say that is a segment of my convex hull.
If the answers is no, like in this case, I'm going to drop that segment, OK?
So now let's talk about complexity.
Let's say that there are n points here.
And how many segments do I have?
I have O n square theta n square segments.
And what is the complexity of the test?
What is the complexity of the test that's associated with, once I've drawn the segments, deciding whether the segment is going to be a tangent which is part of the convex hull or not?
What is the complexity?
O n
O n-- exactly right.
So on test complexity-- and so we got over theta n cubed complexity, OK?
So it makes sense to do divide and conquer if you can do better than this.
Because this is a really simple algorithm.
The good news is we will be able to do better than that.
And now that we have a particular algorithm-- I'm not quite ready to show you that yet.
Now that we have a particular algorithm, we can think about how we can improve things.
And of course we're going to use divide and conquer.
So let's go ahead and do that.
And so generally, the divide and conquer, as I mentioned before, in most cases, the division is pretty straightforward.
And that's the case here as well.
All the fun is going to be in the merge step.
Right, so what we're going to do, as you can imagine, is we're going to take these points.
And we're going to break them up.
And the way we're going to break them up is by dividing them into half lengths.
We're going to just draw a line.
And we're going to say everything to the left of the line is one sub problem, everything to the right of the line is another sub problem, go off and find the convex hull for each of the sub problems.
If you have two points, you're done, obviously.
It's trivial.
And at some point, you can say I'm just going to deal with brute force.
If we can go down to order n cubed, if n is small, I can just apply that algorithm.
So it doesn't even have to be the base case of n equals 1 or n equals 2.
That's a perfectly fine thing to do.
But you could certainly go with n equals 10, as I mentioned before, and run this brute force algorithm.
And so at that point, you know that you can get down to small enough size sub problems for which you can find the convex hull efficiently.
And then you've got these two convex hulls which are clearly on two different half planes because that's the way you defined them.
And now you've got to merge them.
And that's where all the fun is, OK?
So let's just write this out again.
You're going to sort the points by x-coordinates.
And we're going to do this once and for all.
We don't have to keep sorting here because we're just going to be partitioning based on x-coordinates.
And we can keep splitting based on x-coordinates because we want to generate these half-lengths, right?
So if we can do those once and for all-- and for the input set S, we're going to divide into the left half A and right half B by the x-coordinates.
And then we're going to compute CH of A and CH of B recursively.
And then we're going to combine.
So the only difference here from what we had before is the specification of the division.
It looked pretty generic.
It's similar to the paradigm that I wrote before.
But I've specified exactly how I'm going to break this up.
So let's start with the merge operation.
We're going to spend most of our time specing that.
And again, there's many ways you could do the merge.
And we want the most efficient way.
That's obviously going to determine complexity.
So, big question-- how to merge.
So what I have here, if I look at the merge step, is I've created my two sub problems corresponding to these two half planes.
And what I have here is-- let's say I've generated, at this point, a convex hull associated with each of these sub problems.
So what I have here is a1, a2.
I'm going to go clockwise to specify the convex hull.
And the other thing that I'm going to do is in the sub problem case, my starting point is going to be for the left sub problem, the coordinate that has the highest x value, OK?
So that's a1 in this case-- the highest x value going over.
x is increasing to the right.
And for the right half of the problem, it's going to be the coordinate that has the lowest x value.
And I'm going to go clockwise in both of these cases.
So when you see an ordering associated with the subscripts for these points, start with a1 or b1 and then go clockwise.
And that's how we number this-- so just notational, nothing profound here.
So I got these two convex hulls-- these sub hulls, if you will.
And what I need to do now is merge them together.
And you can obviously look at this and it's kind of obvious what the overall convex hull is, right?
But the key thing is, I'm going to have to look at each of the pairs of points that are associated with this and that and try to generate the tangents, the new tangents, that are not part of the sub hulls, but they're part of the overall hull, right?
So in this case, you can imagine an algorithm that is going to kind of do what this brute force algorithm does except that it's looking at a point from here and a point from here.
So you could imagine that I'm going to do a pairwise generation of segments.
And then I'm going to check to see whether these segments are actually tangents that are part of the overall convex hull or not.
So what I would do here is I'd look at this.
And is that going to be part of the overall hull?
No, and precisely why not?
Someone tell me why this segment a1 b1 is not part of the overall hull?
Yeah, go ahead
If we were to draw a line through the whole thing there would be one on both sides
Exactly right-- that's exactly right.
So here you go.
So that's not part of it.
Now, if I look at this-- well, same reason that's not part of it.
In this case-- and this is a fairly obvious example.
I'm going to do something that's slightly less obvious in case you get your hopes up that we have this trivial algorithm, OK?
This is looking good, right?
That's supposed to be a straight line, by the way.
So a4 b2-- I mean, that's looking good, right?
Because all the points are on one side.
So a4 b2 is our upper tangent.
Right, so our upper tangent is something that we're going to define as-- if I look at each of these things, I'm going to say they have a yij.
OK, what is yij?
yij is the y-coordinate.
of the segment that I'm looking at, the ij segment.
So this yij is for ai and bj.
So what I have here is y42 out here.
And this is-- for the upper tangent, yij is going to be maximum, right?
Because that's essentially something which would ensure me that there are no points higher than that, right?
So if I go up all the way and I find this that has the maximum yij, that is going to be my upper tangent.
Because only for that will I have no points ahead of that, OK?
So yij is upper tangent.
This is going to be maximum.
And I'm not going to write this down, but it makes sense that the lower tangent is going to have the lowest yij.
Are we all good here?
Yeah, question
So I am just wondering, I couldn't hear what she said why we moved out a1 b1
OK, so good.
Let me-- that reason we moved out a1 b1 is because if I just drew a1 b1 like this-- and I'm extrapolating this.
This is again supposed to be a straight line.
Then you clearly see that there are points on either side of the a1 b1 segment when you look at the overall problem, correct?
You see that on a1 b1, b2 is on this side, b3 is on this side if I just extend this line all the way to infinity in both directions.
And that violates the requirement that the segment be part of the overall hull, OK?
That make sense?
Good.
So, everybody with me?
So clearly, there's a trivial merge algorithm here.
And the trivial merge algorithm is to look at not every pair of points-- every ab pair, right?
Every aibj pair.
And so what is the complexity of doing that?
If I have n total points, the complexity would be-- would be in square, right?
Because maybe I'd have half here and half here, ignore constants.
And you could say, well, it's going to be n squared divided by 4, but that's theta n squared.
So there's an obvious merge algorithm that is theta n square looking at all pairs of points.
And when I mean all pairs of points, I mean like an a and a b.
Because I want to pick a pair when I go left of that dividing line and then right of the dividing line.
But either way, it's theta n square, OK?
So now you look at that and you go, huh.
Can I do a better?
What if I just went for the highest a point and the highest b point and I just, no, that's it?
I'm done-- constant time.
Wouldn't that be wonderful?
Yeah, wonderful, but incorrect, OK?
Right, so what is an example.
And so this is something that I spent a little bit of time last night concocting.
So I'm like you guys too.
I do my problem set the night before.
Well, don't do as I do.
Do as I say.
But I've done this before.
So that's the difference.
But this particular example is new.
So what I have here is I'm going to show you why there's not a trivial algorithm, OK, that-- I got to get these angles right-- that you can't just pick the highest points and keep going, right?
And then that would be constant time.
So that's my a over here.
And let's assume that I have my dividing line like that.
And then what I'm going to do here-- and I hope I get this right-- is I'm going to have something like this, like that.
And then I'm going to have b1 here clockwise-- so b2, b3, and b4.
So as you can see here, if I look at a4-- a little adjustment necessary.
OK, so if I look at that, a4 to b1 versus-- I mean, just eyeball it.
A3 to b1-- right, is a4 to b1 going to be the upper tangent?
No, right?
So now a3 is lower than a4.
You guys see that?
And b1 is lower than b2, right?
So it's clear that if I just took a4 to b2 that it will not be an upper tangent.
Everybody see that?
Yep, all right, good.
So we can't have a constant time algorithm.
We have theta and square in the back.
So it is there something-- maybe theta n?
How would we do this merge and find the upper tangent by being a little smarter about searching for pairs of points that give us this maximum yij?
I mean, the goal here is simple.
At some level, if you looked at the brute force, I would generate each of these things.
I would find the yj intercepts associated with this line.
And I just pick the maximum.
And the constant time algorithm doesn't work.
The theta n squared algorithm definitely works.
But we don't like it.
So there has to be something in between.
So, any ideas?
Yeah, back there
So...
I had a question.
[INAUDIBLE]
No, you're just finding-- no, you're maximizing the yij.
So for once you have this segment-- so the question was, isn't the obvious merge algorithm theta n cubed, right?
And my answer is no, because the theta n extra factor came from the fact that you had to check every point, every endpoint, to see on which side of the plane it was.
Whereas here, what I'm doing is I've got this one line here that is basically y equals 0, if you like, or y equals some-- I'm sorry, x equals 0 or x equals some value.
And I just need to, once I have the equation for the line associated with a4 b1 or a4 b2, I just have to find the intercept of it, which is constant time, right?
And then once I find the intercept of it, I just maximize that intercept to get my yij.
So I'm good, OK?
So it's only theta n squared, right?
Good question.
So this is actually quite-- very, very, very clever.
This particular algorithm is called the two finger algorithm.
And I do have multiple fingers, but it's going to work a lot better if I borrow Eric's finger.
And we're going to demonstrate to you the two finger algorithm for merging these two convex hulls.
And then we'll talk about the complexity of it.
And my innovation again last night was to turn this from a two-finger algorithm.
Not only did I have the bright idea of using Eric-- I decided it was going to become the two finger an string algorithm.
So this is wild.
This is my contribution to 046 lore-- come on.
So the way the two finger algorithm works-- this pseudo code should be incomprehensible.
If you just look at it and you go, what, right?
But this demo is going to clear everything up.
Right so here's what you do.
So now we're going to do a demo of the merge algorithm that is a clever merge algorithm than the one that uses order n square time.
And it's correct.
It's going to get you the correct upper tangent and what we are starting at here is with Erik’s left finger on A1, which is defined to be the point that's closest to the vertical line that you see here, the one that has the highest x-coordinate.
And my finger is on B1, which is the point that has the smallest X-coordinate on the right hand side sub-hull.
And what we do is we compute, for the segment A1 B1, we compute by Yij, in this case Y11, which is the intercept on the vertical line that you see here that Erik just marked with a red dot.
And you can look at the pseudocode over on, to my right if I face the board.
And what happens now is I'm going to move clockwise, and I'm going to go from B1 to B4.
And what happened here?
Did the Yij increase or decrease?
Well, as you can see it decreased.
And so I'm going to go back to B1.
And we're not quite done with this step here.
Erik’s going to go counterclockwise over to A4.
And we're going to check again, yeah, keep the string taught, check again whether Yij increased or decreased and as is clear from here Yij increased.
So now we move to this point.
And as of this moment we think that A4 B1 has the highest Yij.
But we have a while loop.
We’re going to have to continue with this while loop, and now what happens is, I’m going to go from B1 clockwise again to B4.
And when this happens, did Yij increase or decrease?
Well it decreased.
So I'm going to go back to B1 and Erik now is going to go counterclockwise to A3.
And as you can see Y31 increased a little bit, so we're going to now stop this iteration of the algorithm and we're at A3 B1, which we think at this point is our upper tangent, but let's check that.
Start over again on my side B1 to B4, what happened?
Well Yij decreased.
So I'm going to go back to B1.
And then Erik’s going to try.
He’s going conterclockwise, he's going to go A3 to A2 and, well, big decrease in Yij.
Now Erik goes back to A3.
At this point we've tried both moves, my clockwise move and Erik’s counterclockwise move.
My move from B1 to B4 and Erik’s move from A3 to A2.
So we've converged, we're out of the while loop, A3 B1 for this example is our upper tangent.
All right.
You can have your finger back Erik.
So the reason this works is because we have a convex hull here and a convex hull here.
We are starting with the points that are closest to each other in terms of A1 being the closest to this vertical line, B1 being the closest to this vertical line, and we are moving upward in both directions because I went clockwise and Erik went counterclockwise.
And that's the intuition of why this algorithm works.
We're not going to do a formal proof of this algorithm, but the monotonicity property corresponding to the convexity of this subhull and the convexity of the subhull essentially can give you a formal proof of correctness of this algorithm, but as I said we won't cover that in 046.
So all that remains now is to look at our pseudocode which matches the execution that you just saw and talk about the complexity of the pseudocode.
So what is the complexity of this algorithm?
It's order n, right?
So what has happening here, if you look at this while loop, is that while I have two counters, I'm essentially looking at two operations per loop.
And either one of those counters is guaranteed to increment through the loop.
And so since I have in this case p points, in one case p plus q equals n-- so let's say I had p points here and I have q points here.
And got p plus q equals n. And I got a theta n merge simply because I'm going to be running through and incrementing-- as long as I'm in the loop, I'm going to be incrementing either the i or the j.
And the maximum they can go to are p and q before I bounce out of the loop or before they rotate around.
And so that's why this is theta n. And so you put it all together in terms of what the merge corresponds to in terms of complexity and put that together with the overall divide and conquer.
We have a case where this is looking like a recurrence that you've seen many a time t of n. I've broken it up into two sub problems.
So I have 2.
And I could certainly choose this l over here that's my line l to be such that I have a good partition between the two sets of points.
Now, if I choose l to be all the way on the right hand side, then I have this large sub problem-- makes no sense whatsoever.
So what I can do-- there's nothing that's stopping me when I've sorted these points by the x-coordinates to do the division such that there's exactly the same number, assuming an even number of points n, exactly the same number on the left hand side or the right hand side.
But I can get that right roughly certainly within one very easily.
So that's where the n over 2 comes from, OK?
In the next problem that we'll look at, the median finding problem, we'll find that trying to get the sub problems to be of roughly equal size is actually a little difficult, OK?
But I want to point out that in this particular case, it's easy to get sub problems that are half the size because you've done the sorting.
And then you just choose the line, the vertical line such that you've got a bunch of points that are on either side.
And then in terms of the merge operation, we have 2t n over 2 plus theta n. People recognize this recurrence?
It's the old merge sort recurrence.
So we did all of this in-- well, it's not merge sort.
Clearly the algorithm is not merge sort.
We got the same recurrence.
And so this is theta n log n-- so a lot better than theta nq.
And there's no convex hull algorithm that's in the general case better than this.
Even the gift wrapping algorithm that I mentioned to you, with the right data structures, it gets down to that in terms of theta n log n, but no better.
OK, so good.
That's pretty much what I had here.
Again, like I said, happy to answer questions about the correctness of this loop algorithm for merge later.
Any other questions associated with this?
Question.
Yeah, back there
If the input is recorded by x coordinates, can you do better than [INAUDIBLE]?
No, you can't, because-- I mean, the n log n for the pre-sorting, I mean, there's another theta n log n for the sorting at the top level.
And we didn't actually use that, right?
So the question was, can we do better if the input was pre sorted?
And I actually did not even use the complexity of the sort.
We just matched it in this case.
So theta n log n-- and then you can imagine maybe that you could do a theta n sort if these points were small enough and you rounded them up and you could use a bucket sort or a counting sort and lower that.
So this theta n log n is kind of fundamental to the divide and conquer algorithm.
The only way you can improve that is by making a merge process that's even faster.
And we obviously tried to cook up a theta one merge process.
But that didn't work out, OK?
But are there algorithms that [INAUDIBLE] ?
First-- if you assume certain things about the input, you're absolutely, right?
So one thing you'll discover in algorithms in 6046 as well is that we're never satisfied.
OK, so I just said, oh, you can't do better than theta n log n. But that's in the general case.
And I think I mentioned that.
You're on the right track.
If the input is pre sorted, you can take that away-- no, it doesn't help in that particular instance if you have general settings.
But if you-- the two dimensional case-- if the hull, all the segments have a certain characteristic-- not quite planar, but something that's a little more stringent than that-- you could imagine that you can do improvements.
I don't know if any compelling special case input for convex hull from which you can do better than theta n log n. But that's a fine exercise for you, which is in what cases, given some structure on the points, can I do better than theta n log n?
So that's something that keeps coming up in the algorithm literature, if you can use that, OK?
Yeah, back there-- question
Where's your [INAUDIBLE] step?
You also have to figure out which lines to remove from each of your two
Ah, good point.
And you're exactly, absolutely right.
And I just realized that I skipped that step, right?
Thank you so much.
So the question was, how do I remove the lines?
And it's actually fairly straightforward.
Let's keep this up here.
And we don't need this incomprehensible pseudo code, right?
So let's erase that.
And thank you for asking that question.
So it's a little simple cut and paste approach where let's say that I find the upper tangent ai bj.
And I find the lower tangent.
Let's call it ak bm.
And in this particular instance, what do I have?
I have a1, a2, a3, a4 as being one of my sub hulls.
And then I have b1, b2, b3, b4 as the other one.
Now, what did we determine to be the upper tangent?
Was it a3 b1?
Right, a3 b1?
So a3 b1 was my upper tangent.
And I guess it was a1-- a1 b4?
A1 b4 was my lower tangent.
So the big question is, now that I've found these two, how do I generate the collect representation of the overall convex hull?
And so it turns out that you have to do this-- and then the complexity of this is important as well.
And you need to do what's called a cut and paste that's associated with this where we're going to just look at this and that.
So if we're going to have these two things, then we've got to generate a list of points.
Now, clearly a4 is not going to be part of that, right?
A4 is not going to be part of the overall hull.
What is it that we want?
We want something like a1, a2, a3, b1, b2, b3, b4, right?
But there's a point that we have to discard here.
Agree?
And so the way we do this is very mechanical.
That's the good news here.
I mean, you don't have to look at it pictorially.
I just made that up looking at-- eyeballing it.
Clearly, a computer doesn't have eyeballs, right?
And so what we're going to do is we're going to say the first link-- in general, the first link is ai to bj.
Because that's my upper tangent, OK?
And in this case, it's going to be a3 d1, OK?
And then I'm going to go down the b list until you see bm, which is the lower tangent.
You're on the b list.
So you're looking for the lower tangent point.
And then you're going to jump until you see bm.
You link it to ak, OK?
You link it to ak and continue until you return to ai.
And then you have your circular list, OK?
So what you see here is you have a3 here.
So I'm going to go ahead and write out the execution of what I just wrote here.
So I have a3.
And I'm going to go jump over to b1.
So I'm going to write down b1.
Then I'm going to along the b's until I get to b4.
In this case, I'm going to include all of the b's.
So I got b1, b2, b3, b4.
And then I'm going to jump from b4 to a1 because that's part of my lower tangent.
And I got a1 here, a2.
And then I'm back to a3, which is great.
Because then I'm done, OK?
And so exactly what I said happened, thank goodness, which is we dropped a4 but we kept all the other points.
Does that answer your question?
Good.
What is the complexity of cut and paste?
It's order n. I'm just walking through these lists.
So there's no hidden complexity here, OK?
Good, good-- thank you.
You definitely deserve a Frisbee.
In fact, you deserve two, right?
Where are you?
I-- oh, could you stand up?
Yeah, right-- two colors.
All right.
Oh, so he-- well, you can give it to him if you like.
So good, thank you.
So are we done?
Are we done with convex hull?
OK, good.
So let's go on and do median finding.
Very different-- very different set of issues here.
Still on divide and conquer, but a very different set of issues.
The specification here is, of course, straightforward.
You can think of it as I just want a better algorithm than sorting and looking for the median at the particular position-- in over two position, for example.
Let's say n is odd.
And it's floor of n over 2.
You can find that median.
Right, so it's pretty easy if you can do sorting.
But we're never satisfied with using a standard algorithm.
If we think that we can do better than that.
So the whole game here is going to be I'm going to find the median.
And I want to do it in better than theta n log n time.
OK, so that's what median finding is all about.
You're going to use divide and conquer for this.
And so in general, we're going to define, given a set of n numbers, define rank of x as the numbers in the set that are greater than-- I'm sorry, less than or equal to x. I mean, you could have defined it differently.
We're going to go with less than or equal to.
So in general, the rank, of course, is something that could be used very easily to find the median.
So if you want to find the element of rank n plus 1 divided by 2 floor, that's what we call the lower median.
And n plus 1 divided by 2 ceiling is the upper median.
And they may be the same if n is odd.
But that's what we want.
So you can think of it as it's not median finding, but finding elements with a certain rank.
And we want to do this in linear time, OK?
So we're going to apply divide and conquer here.
And as always, the template can be instantiated.
And the devil is in the details of either division or merge.
And we had most of our fun with convex hull on the merge operation.
It turns out most of the fun here with respect to median finding is in the divide, OK?
So what I want is the definition of a select routine that takes a set of numbers s. And this is the rank.
So I want a rank i.
And that i might be n over 2-- well, floor of n plus 1 over 2, whatever?
And so what does the divide and conquer look like?
Well, the first thing you need to do is divide.
And as of now, we're just going to say you're going to pick some element x belonging to s. And this choice is going to be crucial.
But at this point, I'm not ready to specify this choice yet, OK?
So we're going to have to do this cleverly.
And then what we're going to do is we're going to compute on k, which is the rank of x, and generate two sub arrays such that I want to find the fifth highest element.
I want to find the median element.
I want to find the 10th highest element.
So I have to keep track of what happens in the sub problems.
Because the sub problems are going to determine, depending on how many elements are inside those sub problems, which I can only determine after I've solved those sub problems.
I'm going to have to collect that information and put it together in the merge operation.
So if I want to find the 10th highest element and I've broken it up relatively arbitrarily, it's quite possible that the 10th highest element is going to be discovered in the left one or the right one.
And I have to show that it's the 10th highest.
And it might be that there's four elements in the left and five on the right that are-- let's see.
If I defined the rank as less than or equal to x, there's four on the left and five on the right that are smaller.
And that's why this is the 10th highest element.
And that's essentially what we have to look at.
So b and c are going to correspond to the sub arrays that you can clearly eliminate one of them.
You can count the number of elements in b, count the number of elements in c. And you can eliminate one of them in this recursion as you're discovering this element with the correct rank-- in this case, i.
So let me write the rest of this out and make sure we're all on the same page.
What I have here pictorially is I've generated b here and c. So this is all of b and that's all of c. I have k minus 1 elements here in b.
And let's say I have n minus k elements in c. And I'm going to do-- essentially take-- once I've selected a particular element, I'm going to look at all of the elements that are less than it and put it into the array b. I'm going to look at all the elements that are better than it.
Let's assume all elements are unique.
I'm going to put all of them into c. And I'm going to recur on b and c. Those two are my sub problems.
But what I have to do is once I recur and I discover the ranks of the sub problems, I have to put them together.
So what I have here is if k equals i-- so I computed the rank and I realized that if k equals-- equals i, I should say-- if k equals i, then I'm going to just return x. I'm done at this point.
I got lucky.
I picked an element x and it magically ended up having the correct rank, OK?
Not always going to happen.
And so in other case, if k is greater than i, then going to return select bi.
So what I've done here is if k is greater than i, then I'm going to say, oh, so now I'm going to have to find the element in b. I know that it's going to be in b because k is greater than i.
And I've got to find the exact position depending on what i is over here.
But it's going to be somewhere between 1 and k minus 1.
And then the last case is if k is less than i, then this is a little more tricky.
I'm going to turn on c of i minus k, OK?
So what happens here is that my k is-- the rank for the x that I looked at over here is less than i.
So I know that I'm going to find this element that I'm looking for in c. But if I just look at c, I don't want to look at c and look for an element of rank i within c, right?
That doesn't make sense because I'm looking for an element of rank i in the overall array that was given to me.
So I have to subtract out the k elements that correspond to x and all of the k minus 1 elements that are in b to go figure out exactly what position or rank I'm looking for in the sub array corresponding to c, OK?
So, people buy that.
So that's just a small, little thing that you have to keep in mind as you do this.
So that's pretty straightforward, looking pretty good.
And you say, well, am I done here?
And as you can imagine, the answer is no, because we haven't specified this value.
Now, can someone tell me, at least from an efficiency standpoint, what might happen, what we're looking for here?
As you can imagine, we want to improve on theta n log n. And so you could you say, well, I'm happy with theta n. That theta n complexity algorithm is better than a theta n log n complexity algorithm, which is kind of in the bag.
Because we know how to sort and we know how to index.
So we want a theta n algorithm.
Now, if you take this and if I just picked, let's say, the biggest element-- I kept picking x to be n or n minus 1 or just picked a constant value.
I picked x to be in the middle.
I picked the index.
I can always pick an element based on its index.
I can always go for the middle one.
So what is the worst case complexity of this algorithm?
If I don't specify or I give you this arbitrary selection corresponding to x belonging to s, what is the worst case complexity of this algorithm?
Yeah, go ahead
N squared
N squared-- why is that?
Because if you [INAUDIBLE] take like the least element
Yep
How do you compare like N o against the other analysis?
Exactly right.
That's exactly right.
So what happens is that you're doing a bunch of work here with this theta n work.
Right here, this is theta n work, OK?
So given that you're doing theta n work here, you have to be really careful as to how you pick the x element.
So what might happen is that you end up picking the x over here.
And given the particular rank you're looking for, you have to now-- you're left with a large array that has n minus 1 elements in the worst case.
You started with n. You did not go to n over 2 and n over 2, which is what divide and conquer is all about-- even n over b, OK?
You went to n minus 1.
And then you go to n minus 2.
And you go to n minus 3 because you're constantly picking-- this is worst case analysis.
You're constantly picking these sub arrays to be extremely unbalanced.
So when the sub arrays are extremely unbalanced, you end up doing theta n work in each level of the recursion.
And those theta n's, because you're going down all the way from n to one, are going to be theta n square when you keep doing that, OK?
So thanks for that analysis.
And so this is theta n squared if you have a batch selection.
So we won't talk about randomized algorithms, but the problem with randomized algorithms is that the analysis will be given a probability distribution.
And it'll be expected time.
What we want here is a deterministic algorithm that is guaranteed to run in worst case theta n. So we want a deterministic way of picking x belonging to s such that all of this works out and when we get our recurrence and we solve it, somehow magically we're getting fully balanced partitions-- firmly balanced sub problems in the sense that it's not n minus 1 and 1.
It's something like-- it could even be n over 10 and 9n over 10.
But as long as you guarantee that, you're shaking things down geometrically.
And the asymptotics is going to work out.
but the determinism is what we need.
And so we're going to pick x cleverly.
And we don't want the rank x to be extreme.
So this is not the only way you could do it, but this is really very clever.
There's a deterministic way.
And you're going to see some arbitrary constants here.
And we'll talk about them once I've described it.
But what we're going to do is we're going to arrange s into columns of size 5, right?
We're going to take this single array.
And we're going to make it a two dimensional array where the number of rows is five and the number of columns that you have is n over 5-- the ceiling in this case.
And then we're going to sort it each column, big elements on top.
And we're going to do this in linear time.
And you might say, how did that happen?
Well, there's only five elements.
So it's linear.
You could do whatever you wanted.
You could do n raised to four.
But it's five raised to four and it's constants.
Don't you love theory?
So then we're going to find what we're going to call the median of medians.
So I'm going to explain this.
This works for arbitrary rank, but it's a little easier to focus in on the median to just explain the particular example.
Because as you can see, there's an intricacy here associated with the break up.
And so here we go.
I'm going to draw out a picture.
And we're going to try and argue that this deterministic strategy that I'll specify gives you fairly balanced partitions in all cases, OK?
So what we see here is we see-- pictorially, you see columns of length five.
Each of these dots corresponds to a number.
This one dimensional array got turned into a two dimensional right.
So I got four full columns.
And it's suddenly possible, given n, that my fifth column is not full, right?
So that's certainly possible.
So that's why I have that up here.
It so what I've here is I'm going to lay them out this way.
And I'm going to look at that.
I'm going to look at the middle elements of each of these n over five columns.
That's exactly what I'm going to look at.
Now, if I look at what I want, what I want over here is this x.
If I want to find-- I'm going to find the median of medians.
So is x.
Now, it is true the first that these columns-- I'm just putting that up here imagining that that's x.
That's not guaranteed to be x because the columns themselves aren't-- well, these columns are sorted.
And what I'm going to have to guarantee, of course, is that when I go find this median of medians is that it ends up being something that gives me balanced partitions.
So maybe say a little bit more before I explain what's going on.
Each of these columns is sorted.
And s is arranged into columns of size 5 like I just said here.
These are the medians, OK?
If I look at determining the medians and I say that once I've determined this x, which I've discovered that it's the median, then this is right there in the middle.
There's going to be a bunch of columns to the left of it, a bunch of elements to the left of it, and a bunch of elements to the right of it.
And in this case, I have five columns.
I could have had more.
It happens to be the third one.
So the idea is that once I find this median of medians, which corresponds to this x number, I can say that all of the columns-- these all correspond to columns that have their median element greater than x.
These correspond to columns that have their median element less than x, OK?
So what I have here in this picture is that these elements here are going to be greater than x.
And these elements here are going to be less than x.
So let me clear.
What's happened here is we've not only sorted all of the columns such that you have large elements up here.
Each of these five columns have been sorted that way.
On top of that, I've discovered the particular column that corresponds to the medians of medians.
And this is my x over here.
And it may be the case that these columns aren't sorted.
This one may be larger than that or vice versa-- same thing over there.
I have no idea.
But it's guaranteed that once I find this median that I do know all of the columns that have elements in this position that are less than this x.
And I know columns that in this position have elements that are greater than x, OK?
Yep
Shouldn't the two elements below x also be computed [INAUDIBLE] less than x
You're exactly right.
I would have probably been able to get the same asymptotic complexity if I dropped those because I had a constant number.
But you're absolutely exactly right.
So the point that-- the question was-- I just redrew it.
These two are clearly less than x as well because they're part of the sorting.
And that's essentially I have here.
Now, my goal here-- and you can kind of see from here as to where we're headed.
What I've down here by this process of sorting each column and finding the median of medians is that I found this median of medians such that there's a bunch of columns on the left.
And roughly half of those elements in those columns are less than x.
And there are a bunch of columns on the right.
And roughly half of those columns have elements that are greater than x.
So what I now have to do is to do a little bit of math to show you exactly what the recurrence is.
And let me do that over here.
So that's the last thing that we have to do.
I probably won't solve the recurrence, but that can wait until tomorrow.
The recurrence will be something that's not particularly difficult to solve.
So I want to now make a more quantitative argument that the variable being n as to how many elements are guaranteed to be greater than x.
And essentially what I'm saying, which is I'm writing out what I have on that picture there, half of the n over 5 groups contribute at least three elements greater than x except for one group with possibly less than five elements, which is the one that I have all the way to the right, and one group that contains x.
So for all the other columns, I'm going to get three elements that are greater than x.
And so if you write that out, this says there are at least three n over 10, because I have half of all of those groups, minus 2.
And I'm not counting perfectly accurately here, but I have an at least.
So this should all be fine.
3n over 1d-- 3 times n over 10 minus 2 elements are strictly greater than x.
And that comes from that picture.
I'm going to be able to say the same thing for less than x as well.
I can't count the one.
Depending on how things go, maybe I could have played around and subtracted 1 instead of a 2 in the latter case.
But I'm just being conservative here.
It is clear that I'm going to have a bunch of columns that are full columns, that are going to be contributing three elements that are greater than x.
And in this case, I have, well, two of them here for the less than x.
And I got one for the greater than x.
So that's all that I'm seeing over here with respect to the balance of the partitions.
And it turns out that's enough.
It turns out all I have to do with this observation is to go off and run the recurrence.
And we're going to get an efficient algorithm.
Yep
Should it not be like greater than or equal to, because there's... [INAUDIBLE]
No, there's nothing that's equal
So you are saying, that's all you need
Yeah.
Yeah, I assume that-- so, convenience, yeah.
There's always a little bit of convenience thrown in here.
We will assume that the a has unique elements.
So there's nothing that's x, OK?
Good.
So the recurrence, once you do that, is t of n equals-- we're going to just say it's order one for n less than or equal to 140.
Where did that come from?
Well, like 140.
It's just a large number.
It came from the fact that you're going to see 10 minus 3, which is 7.
And then you want to multiply that by 2.
So some reasonably large number-- we're going to go off and we're going to assume that's a constant.
So you could sort those 140 numbers and find the median or whatever rank.
It's all constant time once you get down to the base case.
So you just want it to be large enough such that you could break it up and you have something interesting going on with respect to the number of columns.
So don't worry much about that number.
The key thing here is the recurrence, all right?
And this is what we have spent the rest of our time on.
And I'll just write this out and explain where these numbers came from.
So that's our recurrence for n less than or equal to 140.
And else, you're going to do this.
So what is going on here?
What are all of these components corresponding to this recurrence?
Really quickly, this is simply something that says I'm finding the median of medians.
I'm finding some element that has a certain rank.
So this median of medians is going to be running on n over 5 columns.
So I've got this-- there are n over 5 columns here.
And I'm going to be calling this algorithm recursively, the median finding algorithm, to do that-- finding the median of medians.
This thing over here is-- I'm going to be discarding at least regardless of what I do.
Because I have these two statements here, I take the overall n. And I'm going to discard.
In my paradigm over here, I'm either going to go with b or I'm either going to go with c depending on what I'm looking for.
And given that b and c are not completely unbalanced, I'm going to be discarding 3n over 10 minus 6 elements, which simply corresponds to me ignoring the ceiling here and multiplying the 3 out.
So that's 3n over 10 minus 6.
So then I have 7n over 10 plus 6.
That's the maximum size partition that I'm going to recur on.
It's only going to be exactly one of them, as you can see from that.
It's either else.
It's not recurring on both of them.
It's recurring on one of them.
So that's where the 7n over 10 plus 6 comes from.
And then you ask where does this theta n come from.
Well, the theta n comes from the fact that I do have to do some sorting.
It's constant time sorting for every column, OK?
Because it's only five elements.
So I'm going to do constant time sorting.
But there's order n columns.
Because it's-- then it's n over 5 columns.
So this is the sorting of all of the columns, all right?
So that's it.
And I'll just leave you with-- you cannot apply the master theorem for solving this particular recurrence.
But if you make the observation-- and you'll see this in section.
You make the observation that n over 5 plus 7n over 10 is actually less than n. So you get 0.2n here and 0.7n there.
That's actually less than n. This thing runs in linear time.
And you'll see that in section tomorrow.
So this whole thing is theta n time.
See you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
I am Erik Demaine.
You can call me Erik.
Today we're going to do another divide and conquer algorithm called the fast Fourier transform.
It's probably the most taught algorithm at MIT.
It's used in all sorts of contexts, especially digital signal processing like MP3 compression, and all sorts of things.
But we're going to think about it today in the context of divide and conquer and polynomials.
So let me remind you-- I mean, this class is all about polynomial time, but usually with polynomial time we only care about the lead term.
Today-- and today only, pretty much-- we're going to be thinking about all the terms in a polynomial.
So I'll talk about polynomials mostly a and b.
You have a constant term, and then a linear term, and a quadratic term, and so on up to-- I will say that there are n terms-- which means the last one is a n minus 1.
Normally the degree the polynomial here is n minus 1.
I wish the degree was defined to be n here, but whatever.
That's-- I'm not-- I can't change the definitions in algebra.
So this is the traditional algebraic way of thinking about a polynomial.
Of course, you can write it with summation notation.
akx to the k, k equals 0 to n minus 1.
We'll jump back and forth between them.
I'm also going to introduce a vector notation for polynomials because-- so the ai's are real numbers, typically.
We might change that at some point, but usually-- a common reason-- maybe you don't care much about polynomials, but you definitely care about vectors.
Any kind of one-dimensional data set is a string of real numbers, like if you're sampling audio-- like right now we're recording this microphone-- you're seeing lots of different-- the movement of the membrane in this microphone over time.
You're sampling-- whatever-- 40,000 times the second.
Each one you're measuring a real number about where that thing is.
That is a sequence of real numbers.
Now, you can convert it into a polynomial if you want.
They're the same thing.
x is not necessarily meaningful here.
We really care about other coefficients.
Now, given such a polynomial there are three typical things we want to do.
So these are the operations on polynomials.
I'll say why we want to do them in a second.
So the obvious one, if you do care about x, is some kind of evaluation.
So maybe I give you a polynomial-- a of x-- and I give you a number-- let's call it x0-- and I want to compute what is a of x0.
So if I plug in-- x here is a general variable, but if I give you an actual real number, say, for the x's, what does that add up to?
So, how would you solve evaluation before we go to the other operations?
There's an obvious way.
We, you know-- you compute all the terms and add them up, but if you do that naively, computing x to the k-- in this form, maybe-- computing x to the k maybe takes k multiplications, and so the total runtime of the quadratic, but we can do better than that.
I know it's 11 o'clock-- too early in the morning think.
Yeah
When you've already calculated x to the i, you can calculate x to the i plus 1 by multiplying it
Good.
Once you've computed x to the k, you can compute x to the k plus 1 with one multiplication, and so you can compute all x to n's in linear time and then you're basically doing a dot product between x to the k's and the a vector.
Cool, my first Frisbee.
So, that's one way to do it.
There's a slightly slicker way to write it called Horner's Rule, but it's doing exactly the same thing that you said.
It's a-- this is just a nice algebraic way of writing it.
a of x is the same as a0 plus x times a1 plus x times a2 plus, and so on.
x times a n minus 1, and then lots of-- close parentheses.
So this is, of course, equivalent to that expression by the law of distribution, and this is essentially doing the x-products one at a time.
So, this is clearly order n additions and multiplication, so we get order n time.
In this lecture, time is the number of arithmetic operations.
That's our model.
Assume it takes constant time to multiply or add two real numbers.
OK.
Cool.
So, evaluation-- that's easy.
Linear time, for today, is good.
Quadratic is bad.
We want to beat quadratic.
OK, second thing you might want to do with polynomials is add them.
The third thing is multiply them.
So, we're given two polynomials-- a of x and b of x-- and we want to compute a new polynomial-- c of x-- that is the summation.
How do you define the summation?
Well, you would like c of x to equal a of x plus b of x for all x.
That's the definition.
Of course, we can do it algebraically as well because these are numbers in the end-- for any x this evaluates to a number-- so if we add two polynomials of this form-- one with ai's, one with the b i's-- all we're doing is adding corresponding ai's and b i's.
So, this is easy.
We just need ck to equal ak plus bk for all k. So, again, linear time-- no problem.
Third operation is the exciting one, the hard one to get good, otherwise this lecture would be over in a couple more minutes.
So multiplication-- same deal.
We're given a of x and b of x, and we want to convert that into some c of x that, for all x, evaluates to the product of those two polynomials.
How do we do this?
We can't just multiply corresponding ak's and bk's.
In fact, if you take a big thing like this and you multiply it by corresponding big thing-- eh, let's do it.
This doesn't look like fun.
We get-- let's see.
So, the constant term is just the product-- that's easy, the constant terms-- but then, if I take this product or this product, I get linear terms.
So it's going to be a1b0 plus a0b1 times x, and then there's a quadratic term which I get from-- switch colors-- this and this and this.
So there's three things times x squared, and that's where I get tired.
I'm going to switch to the summation notation.
I didn't go to high school, but I assume in high school algebra you learn this.
ck is the sum of j equals 0 to k. ajbk minus j.
That's the general form because aj came from an x to the j term, bk minus j came from x to the k minus j term.
When you multiply those together, you get x to the k, so this is the coefficient of x to the k. Cool?
So, that's what we'd like to compute.
Given a and b we want to compute this polynomial c. How long does it take?
We have to do this for all k. So, to compute the k-th term takes order k time, so the total time is n squared.
So, that's not good for this lecture.
We want to do better.
In fact, today we will achieve n log n. That's our goal-- polynomial multiplication in n log n. Why do we care about polynomial multiplication?
Because it's equivalent to another operation which we use all the time in digital signal processing, image editing, all sorts of different things, which is convolution.
Convolution is usually thought of as an operation on vectors.
So, remember this vector notation, where we're just thinking about the coefficients.
x's are kind of irrelevant.
We're just thinking about a sequence of real numbers.
So, maybe that sequence of real numbers for a represents waveform.
Maybe this is the audio I'm speaking now.
And then I take some other waveform.
Here I have a Gaussian function-- e to the minus x squared-- and I want to take-- for all possible shifts of this Gaussian I want to compute the dot product between the blackboard and the piece of paper.
That's some kind of smoothing function, if I wanted to clean up noise or something like that.
You can do the same thing on a two-dimensional image.
It's a little harder to think about, but you can map a two-dimensional image to a one-dimensional vector.
And you have a two-dimensional Gaussian-- if you ever do Gaussian blur in Photoshop this is what you're doing-- a convolution-- and it's used to pretend that your lens is out of focus when you do that.
It's done in audio processing, and all sorts of things.
So, formally, you're given two vectors and you want to take all possible shifts of one vector and take the dot product with the other one.
I have that written down.
I just-- dot product, same thing as inner product, which just means multiply corresponding positions and add them up.
And, if you ignore this minus sign, that's exactly what this is doing.
This is taking aj versus bk, pretend it's plus j.
So, that's the bj vector, but with all possible shifts k. We compute this for all k. That's really cool.
We're going to compute it in n log n time.
All different n shifts of b will take the dot product with a.
It's kind of magical because it looks like you're doing n squared work, but we will do it in n log n time.
The only issue is we have to reverse b.
Then the minus signs turn into plus signs.
And there's some boundary conditions, but it's basically the same.
If we can solve polynomial multiplication, we can solve convolution.
Cool?
So, that's why we care about multiplication.
So, how are we going to solve this?
What I'd like to do is talk about alternate representations of polynomials.
That's the next thing here.
We just did operations.
Let's talk about different representations.
So, we talked about this one representation-- it's one way to represent polynomial-- but it's not the only one.
You probably know others.
So, on the one hand we have representation a is a coefficient vector.
So we can write down the ai's.
That was just one way to represent a polynomial.
Can anyone give me another way to represent a polynomial?
I have two ways in mind.
Yeah
Generating function?
Generating function.
Isn't that the same-- oh, I guess in principle you could imagine writing a recurrence on the generating function or something.
It's plausible.
In general, generating functions are polynomials if you know what that-- they are cool.
So it doesn't quite answer the question.
Yeah?
[INAUDIBLE]
Sure
Representation
Sorry?
Points?
Point representation, yeah.
I call them samples.
I'm going to put that under C. A bunch of samples, a bunch of points on the polynomial.
So like xk, yk for-- how many do we need-- I think n minus 1 should do it.
We'll check.
Yep.
And if we are told that a of xk equals yk, and we're told that all the xk's are distinct, then this uniquely determines the polynomial.
You have a degree n minus one polynomial, and you have n samples.
There's only one polynomial that passes through all those points.
It's a consequence of the fundamental theorem of algebra that gives you uniqueness.
Existence, I think, was proved by Legendre in 1700s, 1800s-- a long time ago.
This is good.
This is what we're going to use.
There's another answer.
I should give you a Frisbee first.
You look so excited.
Just wait till I hit you, then you'll be less excited.
OK, so.
Samples, coefficients, anything else?
Yeah
Roots?
Roots, yeah.
Roots is the other answer I was looking for, but it's not going to be so good algorithmically, as we'll see.
Sorry.
So, I can give you a sequence of roots.
This is the fundamental theorem of algebra that every polynomial is uniquely determined by its set of roots.
If you allow roots with multiplicity then every polynomial of degree n has exactly n roots.
So, this would be some sequence of r1 up to rn minus 1, and the polynomial would be given as-- you actually need a constant multiplier-- but x minus r0, x minus r1.
That would be polynomial.
The trouble with roots is that if I give you a coefficient vector and I want to compute the roots, not only is it hard to do, it's impossible to do in our model.
If you're only allowed to add, subtract, multiply, divide, take square roots, take k-th roots, there is no way to solve a polynomial of degree 5 or larger.
There's the quadratic formula, cubic formula, quartic formula.
There is no quintic formula.
That's an old result.
1800s.
So, going from coefficient vector to roots takes infinite time.
It's not so good.
And, in particular, if we think about our operations, addition becomes really difficult.
Multiplication is easy if I have two polynomials represented as a sequence of roots.
I want to multiply them.
That's just concatenating the vectors-- taking union the vectors of their root lists.
So that's cool.
I'm multiplying the c's, I guess.
But addition is really hard because addition is sort of fundamentally about coefficient vectors.
And then, once you go there-- you can go from roots to coefficient vectors and add them up, but then there's no relation between the roots of the sum of the polynomials versus the roots of the original.
I don't know for sure that that's impossible.
It's definitely very, very hard, probably impossible.
Let me draw a little table.
Each of these representations has some advantages and a disadvantage in terms of these three operations.
So, on the one hand, we have the algorithms we care about-- evaluation, addition, and multiplication-- and on the other axis we have our representations, which are coefficient vectors, roots, and samples.
You'll see why I chose this order in a moment.
It makes for a nice, pretty matrix.
We've talked about almost every cell in this matrix, but let me just summarize.
We started out just thinking about coefficient vector, and evaluation was linear time.
Addition was linear time.
Multiplication, so far, was quadratic, although our goal is to make n log n. For roots, I just said multiplication is easy.
That's linear time, addition is really hard-- like, infinite time-- and evaluation, I guess that's linear time.
In fact, the way it's written, you only have a linear number of subtractions and multiplications, so it's really easy to evaluate.
And then sample vectors-- we haven't talked much about that.
The idea is, suppose you're given two polynomials with the same xk's.
We're going to fix xk's.
All we need is that they're distinct.
So xk could equal k, for example, just a bunch of integers.
And then we are told what polynomial a evaluates to at every xk, and we're told what polynomial b evaluates to at every xk.
So, we're given some yk's and some zk's, and then we want to compute, say, the sum or the product of those two vectors.
What do we do?
Just add or multiply the corresponding yk and zk's, because if we're told we want c of x to equal a of x times b of x, or c of x to equal a of x plus b of x for all x, Well, now we know what x's we care about.
We just do it at the xk's.
That's what we're told for a and for b, and so to compute c of xk it's just the sum or the product of yk and zk.
So multiplication is really easy in the sample view, and this is why we are going to use this view.
We're also going to use this view because, as we'll see, there's a problem.
Addition is easy, multiplication is easy, evaluation is annoying.
I can evaluate a of x at xk for any k, but I can't evaluate it at some arbitrary value of x.
That's annoying.
I'm told at these finite sample points, but now I have to somehow interpolate.
This is called polynomial interpolation, well studied in numerical analysis and so on.
You can do it, but it takes quadratic time, in general.
The best known algorithms are quadratic.
So, this is bad, this is bad, and this is bad, so no representation is perfect.
Life sucks.
What we'd like is to get the best-- now this one is really hard to work with because converting inter roots is impossible in an arithmetic model, so we're going to focus on column A and column C. We kind of like to take the min of those two columns.
We won't quite get that.
What we will get is an algorithm for converting between these two representations in n log n time-- it's not quite linear, but close-- and once we can do that, if we want to multiply two things in the coefficient land we can convert to sample land, do it in linear time, and then convert back.
So that's the magical transformation we're going to cover, and it is called the fast Fourier transfer.
Fast Fourier transform is the algorithm.
Discrete Fourier transform is that transformation mathematically.
Cool.
So the whole name of the game is converting from coefficient representation to samples, or vice versa.
Turns out they're almost the same, though that won't be obvious for a long time-- till the end of the class.
Any questions before we proceed?
Yeah
[INAUDIBLE] multiply repetitions, why not we evaluate a and b first then multiply?
Ah.
So, OK, the question is, if I want to-- we'll get there in a second-- but if I want to multiply, and it's so easy to do in sample land, why don't I just sample a and b and then multiply them?
That's right, but [?
effect ?]
sampling is not so easy.
It takes quadratic time.
Let's go there now.
Because we have n samples to do, each one will cost linear time.
Remember, to evaluate a polynomial takes linear time.
If you want to think of it in a matrix-- let's enter the matrix-- then we get a big matrix.
So we're given the xi's and we just want to evaluate a given polynomial whose coefficients are given by a0, a1, a2, and minus 1.
Our goal is to compute the yi's-- y0, y1, y2, to yn minus 1-- and if you know a matrix-vector product, you take this row with that column.
You take the dot product that multiplies corresponding entries.
You get y0.
That is the definition of the polynomial evaluation.
I'm just going to write a bunch of these rows so you get the pattern.
Pretty simple.
This is called the Vandermonde matrix.
I'll call it V. And in general-- I don't have room for it, so let me go-- In general, if we look at V ij, it's just-- sorry.
You may notice I'm not using the letter i.
We will get to why in a moment.
V jk.
Row j, column k. That's going to be x sub j to the power k. That's the Vandermonde matrix.
We can compute it in quadratic time.
It has quadratic entries.
We can use the trick we suggested earlier-- compute each term from the previous one by multiplying by xj-- and then we want to compute this matrix-vector product, and you can clearly do it in quadratic time-- I'm just computing each thing correspondingly-- and that's sort of the best you can do without any further assumptions.
So this takes-- if I want to compute this product, that's the coefficients to samples problem.
This is the same thing as computing V times the A vector, so this is a matrix-vector multiplication, which takes n squared time.
OK?
On the other hand-- so, that's a problem because we're trying to beat quadratic multiplication, so if we spend quadratic time to convert over here it doesn't matter if this is linear time.
There are two problems.
One is that conversion costs too much.
The other is we don't yet know how to convert backwards.
But this matrix field gives us also the reverse transformation.
If we want to convert samples to coefficients, this is-- the best notation I know is from MATLAB.
How many people know MATLAB?
A bunch.
So for you, it's V backslash A, but usually in linear algebra like 18.06 you see you have some matrix V times some unknown vector-- usually it's called x, here it's called a-- and you know the right-hand side.
You want to solve for this.
How do you do it?
[INAUDIBLE]
Sorry?
Multiply by the inverse?
Multiply by the inverse.
Yeah.
How do you do it in computer science?
Gaussian elimination
Gaussian elimination.
Turns out inverse is the right answer here, but Gaussian elimination would be the standard way to solve a linear system like that.
The trouble with Gaussian elimination is it takes cubic time in its normal form.
In this case it's a little bit special because this matrix is essentially fixed.
The xi's don't need to change.
It could be xi is just i or something, so we can-- in this case it's a little better to compute the inverse first.
So we could also just do v inverse times a.
From a numerical analysis standpoint this is very bad, but don't worry about it for now.
We're going to get a better algorithm today.
Anyway, it doesn't involve matrices at all, but the nice thing is, if computing the inverse, that takes n cubed time, but you only have to do it once.
So if you have to do this many times you can do this product in n squared time.
You just have to maintain that v inverse once and for all.
OK, great.
So, we've got quadratic algorithms to go back and forth between these representations.
That, at least, tells us it's doable, but we, of course, need better than quadratic to improve on the naive multiplication algorithm, so that's what we're going to do.
In general, we can't do any better, but we have one freedom, which is we have said nothing about the x case.
We can choose them to be whatever we want them to be.
I keep saying xk equals k. That seems fine.
It's actually really bad choice for a reason we will get to, but there is a choice where, magically, this transformation becomes easy and you can do it in n log n time.
Before we get there, I want to give you some motivation for how this could possibly work.
As you might expect, even just from that n log n running time, we're going to be using divide and conquer, so let's just think about how divide and conquer could work.
I'm going to show you an idea and then we'll figure out how that idea could possibly work.
It doesn't work at the moment, but we will be able to choose the xk so that it works.
So, let's say the goal-- I mean, what we want to do is compute this v times a. I'm going to convert-- think of that back into polynomial land.
So our goal is to compute a of x for all x in some set x.
This is taking a bunch of samples.
Set x is just a set of the xk's, but I'm going to change that set in a moment using recursion.
So the input to this algorithm is a polynomial a of x, and it's a set capital X of positions that I'd like to evaluate that polynomial at.
This is clearly more general than the problem we're trying to solve, and I'm going to solve it with divide and conquer.
In divide and conquer there are three steps.
Divide, conquer, and combine.
Let's start with divide.
Here's the big idea.
I would say there are two natural ways to divide a vector.
One is in the middle, that's what we've always seen with merge sort and convex hull from last time, but there's another way which will work better here, which is the even entries and the odd entries.
So I'm going to divide into even and odd coefficients.
Let me write that down.
One of them is called a sub even of x-- that's a polynomial.
It's going to have half the degree, so it's going to be sum from k equals 0 to-- I wrote n here, but I think I want something like n over 2 minus 1, n minus 1 over 2, one of those things-- of a to k x to the k. So, really, what I want-- which is easier to write-- is, in the vector notation, I want all the even entries.
I won't try to figure out what the last one is, but it's roughly n over 2-ish.
I'll just write that.
You can go a little bit extra.
It's fine if you define a sub n to be 0, and a sub n plus 1 to be zero, and all those to be 0, those terms will disappear.
So the key thing here is I'm taking the even entries, but I don't have 2k up here.
This is the x to the 0 term.
This is the x to the 1 term.
This is the x to the 2 term.
So there's a difference between x to the k's and the sub 2k, but, I mean, just think about it in vector form.
Don't worry about the algebra for now.
We're going to have to worry about it in a second, but, intuitively, what I want to do is extract from the vector of all the ai's these two vectors-- the odd coefficients in order and the even coefficients in order-- but I'm going to need the algebraic form in a moment for the combined step.
It should be 2k plus 1 x to k. That's step one.
Easy to do.
Linear time, of course.
Let's jump ahead to step three, combine.
In order to compute a of x from-- what I'd like to do is recursively compute a even of x and a odd of x for some values x.
It's not going to be x and x.
It's going to be some other set.
Let's think about how I can compute a of x given some solutions to a even of x and a odd of x.
So this is step three, combine.
So I would like a of x over here, and I want a even of something and a odd of something and something in here.
Anyone see the algebra?
Maybe start with this?
I hear a mumble.
Yeah
x squared
x squared.
Exactly.
Time for a purple one.
I'm getting better.
Why x squared?
Because we have this mismatch here.
We have a sub 2k.
We want x to the 2k.
How could we do that?
Well, we could put x squared to the 2k, and x squared to the k is the same thing as x to the 2k.
And so magically this transforms into the even entries of a of x.
That's half of them.
We do the same thing for the odd ones and we're almost there.
Now we have a sub 2k plus 1 times x to the 2k, no plus 1.
So how can I add a plus 1?
Multiply by x
Multiply by x here.
Take the whole thing, multiply by x, then I get all of the odd terms of a of x. I add these together.
I get a of x. I mean, you could prove this more carefully, but that's just algebra to see that this is correct.
Once you have this, it tells you what I need to do is compute a even of x squared for all x in x, so this is 4x and x.
There's a for loop for you.
So that's going to take linear time.
If I already know this value and I already know this value, I do one multiplication, one addition, and boom.
I get a of x.
So in the conquer step I want to recursively compute-- I think I'll call it-- a even of y, and a odd of y for y in x squared.
x squared is the set of squares of all numbers in x.
So I'm changing my set x. I started with a polynomial a and a set x. Recursively, I'm doing a different polynomial of half the degree-- half the number of terms-- but with a different set of the same size.
I started with x. x squared has the same size as x, right?
So let's try to figure out how fast or slow this algorithm is.
But that isn't divide and conquer.
That is going to be our golden ticket.
It's pretty simple, but we're going to need another trick.
So I'm going to write a recurrence.
Now, this recurrence depends on two things.
One is how many terms are there in a that we've been calling n?
And the other is how many numbers are there in x?
How many different places do I have to evaluate my polynomial?
So we've got t of n, and I always call it size of x.
So divide and conquer goes hand-in-hand with recurrences.
Generally you've got the recursive part, so that's just how big are the subproblems.
How many are there?
There are two subproblems.
They have half the size in terms of n, but they have the same size in terms of x.
So, all right, 2 times-- because there's two subproblems-- each with size n over 2 and size x, plus what goes here is however much it costs to do the divide step, plus however much it costs to do the combined step-- all the non-recursive parts.
So this is just partitioning the vectors-- linear scan, linear time.
This is-- we talked about it-- it's a constant number of arithmetic operations for each x.
So this cause-order x time, this cause-order n time.
So, in general, we get n plus x.
Now, this is, again, not a recurrent solvable by the master method because it has two variables.
So usually when you're faced with this sort of thing, you want to do back of the envelope picture.
Draw a recursion tree.
That's a good way to go.
So at the root-- now, I know that initially x equals n. When I start out I have n coefficients, I have n different positions I want to evaluate them at because that's what I want to do to do this conversion from coefficients to samples.
So at the root of the recursion tree I'm just going to write n. Order n work to get started and to do the recursions.
There are two recursive calls.
One has som-- both have size n over 2 in terms of a, and they have the same x-- which is also known as n-- in those two recursions.
So in fact, the linear work here will be n and n, and then we'll get n and n. x never goes down, so x always remains n-- the original value of n. This is a bad recurrence.
There are n-- sorry, there are log n levels.
That's the good news.
Once we get down to constant size we can kind of stop.
When there's only one coefficient I know how to evaluate the polynomial with just a 0.
That's easy.
So down at the bottom here, at the last level-- this is the height log n-- the last level is just, again, going to be a whole bunch of n's-- all the same n. Here n is the original value of x because we haven't changed that.
How many n's are there down here?
2 to the log n
2 to the log n, also known as n. Good.
So we had-- because we had log n levels, we had binary branching, so it's 2 to the number of levels, which is just n. So this is n squared.
All this work, still n squared.
Clearly what we need is for x to get smaller, too.
If x-- if, in this recursion-- let me, in red, draw the recursion I would like to have.
If x became x over 2 here, that's the only change we'd need.
Then n and x change in exactly the same way, and so then we can just forget about x-- it's going to be the same as n. Then we get 2 times n over 2 plus order n. Look familiar?
It is our bread and butter recurrence-- merge sort recurrence.
That's n log n. That's what we need to do.
Somehow, when we convert our set x to x squared, I want x to get smaller.
Is that at all plausible?
Let's think about it.
What's the base case?
To keep things simple let's say the base case when x equals 1, I'll just let x be-- let's say I want to compute my a at 1.
Keep it simple.
What if I want two values in x?
I'd like to have the feature that when I square all the values in x-- so I want two values, but when I square them I only have one value.
Solve for x. Yeah
Negative 1 and 1?
Negative 1 and 1.
Whew, tough one.
Ehh.
Not bad.
Good catch.
Negative 1 and 1.
That could work.
What are negative 1 and 1?
They're the square roots of 1.
Negative 1 squared is 1, 1 squared is 1.
There's two square roots for every number.
Two square roots for every number.
Interesting.
So that means, if I just keep taking square roots, when I square them it collapses by a factor of 2.
Let me go to another board and define something.
You're all anticipating what's going to happen, but I'm going to say, a collapsing set x-- or, a set is collapsing if either the size of x squared is the size of x divided by 2, and, recursively, x squared is collapsing.
So I need this to work all the way down the recursion.
Or I need a base case, which is just x equals 1.
There's a single item in x.
So I happened to start with x equals the item 1.
It didn't have to be 1.
It could have been 7.
It couldn't be 0 because 0-- you won't get two numbers.
There's only one square root of 0.
OK, so I lied a little bit.
Other than 0, every number has exactly two square roots, so what's the square root of negative 1?
i
i.
So complex numbers.
If I take square roots of these guys I get i and negative i, and again I get minus 1 and 1.
That's when x equals 4.
Turns out this is only going to work for powers of 2, but, hey, if n isn't a power of 2 just round up to the next power of 2.
That only hurts me by a factor of 2.
Complex numbers.
Every time I said real number in the past, pretend I said complex number.
Everything I said is still true.
Actually, the root thing is only true when you allow complex numbers.
Some polynomials have complex roots, so pretend I said complex.
We're going to need complex numbers.
This is why.
Because, when we'd start taking square roots, we immediately get complex numbers.
Next would be x equals 8.
Square root of i.
Square root of i-- let's see if I can do it-- should be root 2 over 2 times 1 plus i.
And then this one is just the-- square root of negative i is going to be root 2 over 2 times 1 minus i, and then we have all our old guys.
Oh, and then I could write plus or minus in front of each of these.
I was like, there weren't enough terms there.
Now I've got eight-- sorry, I've got four numbers here.
I've got four numbers there.
How in the world could I remember this?
Maybe I memorized it.
No, I didn't memorize it.
It's actually really easy to figure this out if you know geometry.
Geometry.
Let's do geometry over here.
It's convenient.
I'm actually a geometer.
You know, complex numbers have two parts, right?
The real part and the complex part.
I'm going to draw that in what's called the complex plane, where we draw the real part here, and I guess it's usually called the imaginary part on the y-axis.
Every point in this plane is a complex number, and vice versa-- the same thing.
So what did we start with?
We started with the number 1.
Number 1 will be here.
It has no imaginary part, so it's on the x-axis-- this is the real line down here-- and it's at position 1, which I'm going to just define to be right there.
Then we've got negative 1.
That's over here.
Then we got i.
That's here, 1 times i.
Then we've got negative i.
That's here.
Then we've got root 2 over 2 times i plus 1.
That's here.
Root 2 over 2 by root 2 over 2.
What is a property of root 2 over 2 comma root 2 over 2?
It has distance exactly 1 to the origin.
If I draw this triangle, root 2 over 2, root 2 over 2.
I square this.
I get a half.
I square this I get a half.
I add them together, I get what?
Take the square root, I still get 1, so this distance is 1.
Interesting.
And then I got the negative of that, which is over here, and a negative of that.
Then this is the-- with a negative-- did I get it wrong?
Yep, sorry.
It doesn't matter, but I'll think of it as i minus-- killed a chalk-- i minus 1.
It's the same because I have the plus and minus, but I like the geometry.
So this point is negative root 2 over 2 by root 2 over 2, and then there's the negative over here.
What property do these points have?
I heard a word
Unit circle?
Unit circle.
Good.
Who said unit circle?
Nice.
It's a unit circle, right?
Clearly that deserves a Frisbee end.
Unit circle.
Hm.
Circle.
It seem good.
What's going on here is I took this number.
I claimed it was the square root of i because it turns out, if you take points on the unit circle in the complex plane, when you square a number it's like doubling the angle relative to the x-axis.
This is angle 0.
This is angle-- what do you call it-- 45 degrees.
This is angle 90 degrees.
So when I square this number I get 90 degrees.
That's why this number is a square root of i. I probably should have labeled some of these.
This is i.
This is minus i.
This is minus 1, and this is 1.
In general, we get something called-- so these are called the n-th roots of unity.
Unity is just a fancy word for 1.
1 is here, and first we computed the square roots of 1.
They were minus 1 and 1.
Then we computed the fourth roots of 1.
All of these numbers, if you take the fourth power, you get 1.
Then we computed the eighth roots of 1.
All of these numbers, if you take the 8th power, you get 1 again.
So, in general, n-th roots of unity.
We're going to assume n is a power of 2, but this notion actually makes sense for any n. And they're just uniformly spaced around the circle, and if you know some geometry and how it relates to trig, you know that a general point on the circle is cos theta comma sine theta.
x-coordinate is cos theta.
y-coordinate is sine theta.
This is also a funny notation for complex numbers-- not so funny, this is the geometric interpretation of cos theta plus i sine theta.
And if I want them uniformly spaced around the circle, and I want to include this point-- also known as theta equals 0-- because when theta equals 0, cos theta is 1, sine theta 0.
So I want to say 4 theta equals to 0, and then-- here I'm going to get fancy-- tau over n to tau over n up to n minus 1 over n times tau.
What's tau?
2pi
2pi, thank you.
This is modern notation just over the last couple years.
I believe in tau so much I got it tattooed on my arm.
Tau is the fundamental constant.
Screw pi.
None of that 3 and change.
Six and change is where it's at.
So tau.
Clearly this is much nicer.
Tau over n, not 2pi over n. Tau is a whole circle.
This is one n-th of a circle, 2n-ths of a circle, n minus 1 over n-ths of a circle.
I didn't do n n-ths of a circle because that's also the same as 0.
Now, why did I introduce this notation?
Because there's this other great thing called Euler's formula, which is that this equals e to the i theta.
Double check.
It's so rare that I get to do real calculus.
This is Euler's formula-- e for Euler-- another number-- 2 and change.
e to the i-- this is funny because it's complex-- times theta is equal to cos theta plus i sine theta.
This is the relation between exponentials and trigonometry.
That's a big thing Euler did.
Cool.
So what?
Because this lets us understand how squares work-- not squares the shape, squares the operation.
When I take squares-- so if I take e to the i theta, this is one of my roots of unity.
Let me expand out what theta is.
So theta is some k times tau over n. Let's do it this way first.
So in reality we have k tau over n, but when I square it, that's the same thing as putting the 2 right here.
This is the same thing as e to i times 2 theta.
Bingo.
I get what I was claiming, that if I start at some angle theta relative to the x-axis, when I square the number I just double the angle.
This is why.
This is obvious just from regular algebra.
And then this thing, Euler's formula, tells me that corresponds to doubling the angle on a circle.
So it only works with points on a circle.
So when I go here I get e to the i2k times tau over n-- twice as far around circle.
All right.
Fine.
What happens if I take this number and I square it?
This has a really big angle.
This is 1/2 plus 1/8, whatever that is-- 5/8 times tau.
When I double that angle I go to here.
Now, this, you might call it, 10/8, or you might also call it 1/4 because when you go around the circle you stay on the circle.
So there's another thing going on, which is really-- this is e to the i times 2 theta mod tau.
Usually we think of mods relative to integers, but what I mean is every time I add a multiple of tau nothing changes.
If I go around the circle five times and then do something, it's the same as just doing the something.
So I kind of need that.
And this is true because e to the i tau equals 1.
You may know it as e to the i pi equals negative 1, but clearly this is a superior formula-- so superior I got it tattooed on my other arm.
[LAUGHTER] It's amazing what you can do with a laser printer and a temporary tattoo kit.
Sadly, these won't last, but definitely try it at home.
Cool.
So e to the i tau equals 1, so going around in circles-- same thing as not.
All right.
So you can draw on this picture for every number, what is its square?
And, in general, if you look at these four guys, their squares are just going to be these two guys.
If you look at these four guys, their squares are going to be among these four guys.
So I started with 8 guys.
I square them, I get 4.
I square them, I get 2.
I square them, I get 1.
That's how we constructed it, but you can see it works not only for this 8-point set that we constructed sort of by hand, but it works for the n-th roots of unity.
As long as n is a power of 2 this set of points will be collapsing.
So if n is the power of 2 for some integer k, then n-th roots of unity are collapsing according to this definition.
And that's what we want.
Then this divide and conquer algorithm runs in n log n time because every time we square the set x, we reduce its size by a factor of 2.
We get t of n equals 2 times t of n over 2 plus n. Plus order n and that's order n log n. And so this whole thing we compute-- in other words, we set xk to be e to the ik tau over n. Now, I guess I should say how to compute that, but let's just say that's given to you for free.
It takes constant time to compute each roots of unity.
In fact, again, we only need to do this once and for all for each value of n, so you can think of it as just being part of the algorithm.
These are the xk's we use.
I said xk could be anything we want as long as they're all different, so I'm going to choose them to be n uniformly spaced points around the complex unit circle.
And then magically this algorithm runs in n log n time.
It's pretty cool.
Intuitively, you think of real numbers x squared, of course it has the same sides as x.
But once you go to complex numbers there's this nifty trick where you start with one n-th of the circle and you'd uniformly space points after the first level of recursion.
You only have to be dealing with n over two of those points, namely the even ones, also known as the n over [?
2th ?]
roots of unity.
And after the next level of recursion is the n over fourth roots of unity.
And so on.
So this recursion is well defined.
When you have a vector of size n you just have to deal with the n-th roots of unity, and you're happy.
That is fast Fourier transform.
That algorithm with these xi's is FFT.
So let me write this somewhere.
It kind of snuck up on us.
This is the algorithm we were aiming to find.
Fast Fourier transform is that divide and conquer algorithm on the right.
I'll write it abstractly.
For something called the DFT-- the discrete Fourier transform-- is the corresponding mathematical transformation.
The fast is about an algorithm.
Discrete is about discrete.
So DFT is this thing that we wanted to compute, which was basically the product v times a.
Remember, v was the Vandermonde matrix, and it depended on all these xk's.
And we're going to set xk to be this e to the ik tau over n. So if you remember the vjk here was xj to the k-th power, so this just becomes e to the ijk tau over n. It's a little funny these are all consecutive because this is a totally different, but-- that's the matrix.
If you take that matrix times a vector, that is called a discrete Fourier transform of the vector, and FFT is a way to compute it in n log n time.
You just run this algorithm and for those xk's it'll just work in n log n time.
Cool.
But if you remember way back to the beginning, what we needed is a way-- this converts a coefficient vector into a sample vector.
Then we can multiply the polynomials, then we need to transform the sample vector that results back into a coefficient vector.
So we're only half done.
I only have 15 minutes.
Luckily, the other half is almost identical to this half, so let's do that next.
Maybe over here.
So we've got our great divide and conquer algorithm, what we need now-- let me give you the polynomial multiplication algorithm.
Let's call this-- sound more exotic-- fast polynomial multiplication, fast meaning n log n. Let's say that we're given two polynomials, a and b, represented in coefficient form.
What we need is-- I'll call it a star, which is the result of running FFT on a.
It's the discrete Fourier transform of a. b star, do the same thing.
So we know what this means is convert a from a coefficient vector into a sample vector.
Convert b from a coefficient vector into a sample vector.
Now I have the samples of a star at the n-th roots of unity-- these guys-- and I have the samples of b at the exact same points-- the n-th roots of unity-- so I can compute c star, the transform version-- the DFT of c-- is just the, I mean, c star k equals a star k times b star k. This is the multiplication algorithm for sample vectors.
We started with that.
That's linear time.
And then the missing piece is I need to re-compute c, which is the inverse fast Fourier transform of c star.
So this is the missing link, so to speak.
We need to be able to go backwards in this transformation.
Good news is the algorithm is not going to change.
What we're computing isn't going to change.
All that's going to change are the xk's.
Why?
Because, remember from the top-- right now we know how to compute v times a, but what we now need to compute is the inverse times a.
So the only question is, what is the inverse?
This matrix has a super special structure-- it's symmetric, lots of points on a circle-- maybe the inverse has a nice structure.
And it does.
Claim is the inverse is v complex conjugate divided by n. What's complex conjugate?
For a geometer, it's reflection through the x-axis.
For an algebraic person, a plus ib, the complex conjugate is a minus ib.
So just apply that to every entry in the matrix, and then divide all the entries by n. You get the inverse.
Cool.
Very cool because what this tells us is we run exactly the same algorithm and do exactly the same transformation.
If we want to do the inverse we can actually just use v, but with a different choice of xk.
Namely, for the inverse, we'll call it xk inverse.
We just take the complex conjugate of this thing, which turns out to be e to the minus ijk tau over n, and then divide the whole thing by n. I'm using a fact here, which is that the complex conjugate of this number is actually, just, you put a minus sign here.
Why does that hold?
Because geometry.
Theta is usually measuring the counterclockwise angle from the x-axis.
If you take the complex conjugate you go from up here to down here-- the reflection to the x-axis.
That's the same thing as if you measure theta as a clockwise angle from the x-axis.
That's the same thing as negating the angle.
So that's just a little geometry.
You can prove it algebraically, although I don't know how off-hand.
It's not hard.
So if I want to take some angle and then flip it through the x-axis, it's the same as the negative of the angle.
So the complex conjugate of this number is the same thing with a minus sign.
And then we have to divide by n. If I just-- did I lie?
I can't divide by n, sorry.
It's in the wrong spot.
I use these xk's and then I apply the transform.
I take this thing.
I get not quite the inverse, but I end up getting n the inverse.
I multiply that by my a star.
That's going to give me n times n. So then I just take that vector and divided by n. Boom, I've got the inverse transform.
So this is how you do inverse fast Fourier transform.
You just flip the sign and the exponent in the xk's, then you do the same matrix-vector product, and then you divide by n. And then you've done the inverse.
So that's how you do-- if you believe this claim-- that's how you do this last step.
Question?
What's the j in the x j [INAUDIBLE]
Whoops, no j, sorry.
I was imagining these guys.
Just k. Thank you.
Other questions?
So what remains is to prove this claim.
I'd be very happy if I-- I should probably have better-- anyway, that's not the best.
I'm going to call this xk prime.
Then when I plug that in, I get a different matrix, v prime.
And what this claim says is that v prime is equal to n times the inverse.
So I apply the same f of t algorithm, but with v prime instead of v, then I get not quite the product I want, but just n times the product I want.
Divide every term by n and we get the inverse.
This is the cool thing about complex numbers.
Here we get another cool thing.
All right, but we have to prove this claim.
So let's do a little bit of algebra.
No pain, no gain.
Good.
So let's look at vjk prime.
Sorry.
So let's look at p, the product of v, and v complex conjugate-- what I was calling v prime up there.
I claim that this thing is n times the identity matrix, with 1's down the diagonal and 0's everywhere else.
So let's look at this product.
In general, let's look at the the jk-th item in the product.
That's going to come from row j of V, dot product with column k of the complex conjugate.
Now the matrices here are symmetric, so actually rows and columns are the same, but that's the general definition of the cell and the product of two matrices.
So let's write it out and a summation.
We have-- from m equals 0-- it's so hard not to use i for my summations.
It's the only class I have to do it because i is already taken, but I guess I can still use capital I, but we'll use m because i is complex number today.
So we have e to the i tau jm over n times e to the minus i tau mk k over n. I don't know why I changed the order.
I put the tau here instead of there, but same thing.
This is just the for every position m in the cell and corresponding position m in-- sorry, m in the row, corresponding position m in the column, I have jm and I have mk-- again, the order doesn't matter.
It's symmetric, but I'm getting it right here.
This is-- we're using this formula.
I put a minus sign here because this is the complex conjugate.
So now I just do some algebra.
These share a lot.
They share i.
They share m, and they share the divided by n. So this is sum m equals 0, n minus 1 e to the i-- oh, they also share tau-- tau m over n times j minus k. Please correct me if I make any mistakes.
Cool.
So it depends how and k relate, of course.
If j equals k, that's also known as something on the diagonal.
I want the matrix n, n, n, n, 0, 0.
So j equals k. That's the diagonal.
That's where I want to get n, and, indeed, if j equals k, this becomes 0.
So all this becomes 0. e to the 0 is 1, and so I'm summing up 1 n times.
I get n. Cool.
That's one case.
This is n if j equals k, and somehow I claim that everywhere else I get 0's.
So let's prove that.
So if j does not equal k-- I'm going to rewrite this a little bit.
j and k are fixed.
m is changing in the sum, so I want to write this as sum m equals 0 to n minus 1, e to the i tau j minus k, over m, times m. In other words, this thing raised to the n-th power.
What is this series?
One word
Geometric
Geometric.
Thank you.
This is a geometric series, and I guess I should have waited to give you the Frisbee until you tell me how do you solve a geometric series with a finite term?
So think of this as z to the m. Do you know the formula?
That's the e to the i tau--
Just z
Oh, z to the n minus 1
Z to the n minus 1.
Almost
Over z minus 1
Over z minus 1.
Yep.
That's-- If you have sum of z to the k, k equals 0 to n minus 1.
That's that.
It's in the appendix of your textbook.
So we just plug that in and we get-- oh boy-- e to the-- better chalk-- e to the i tau j minus k over n to the n-th power minus 1 over e to the i.
Actually, the denominator doesn't really matter because this is supposed to be 0, so it's all about the numerator, but it's the same thing.
Minus 1.
Where's my red?
Red.
The n cancels with the n. This is an integer not equal to zero, so we have e to the i tau to an integer power.
What's e to the i tau?
1.
Convenient that I have that there.
1 minus 1 is 0, so we get 0.
Satisfying.
So that proves this claim.
We prove that, on the diagonal, we get n because we had n copies of 1.
Off the diagonal we get 0, like this.
Therefore, v complex conjugate times n is the v inverse.
Therefore, this algorithm actually computes the inverse fast Fourier transform.
So that's it for algorithms today, but let me quickly tell you about some applications.
You've probably taken other classes that use applications of fast Fourier transform, so I will just summarize.
If you've ever edited audio, you probably did it in-- unless you're just pasting audio clips together-- you probably did it in what's called frequency space.
So you know that-- as I talked about in the beginning-- when we're measuring where the membrane on this microphone goes over time, that is in the time domain for every time we sample where, physically, this thing is.
If you apply-- I think the way I defined it here is the inverse fast Fourier transform, usually it's called Fourier transform-- to that time domain vector, you get a new vector.
Now it's a complex vector.
You may have started with real numbers.
You get complex numbers.
So for every position in the vector-- what it corresponds to-- the x-axis is no longer time.
Now it's frequency.
For every frequency you're measuring, essentially, you're viewing this vector-- the waveform-- as a bunch of trigonometric functions-- say, sine of something times theta.
If you look at one of the entries in the vector, and it's a complex number-- if you compute the magnitude of the complex number-- the length of the vector, of the length of that two-coordinate vector-- that is how much stuff-- of that frequency-- you have.
And then the angle of the vector, in 2D, is how that trigonometric function shifted in time.
So if you take a pure note, like if I was playing a bell and it's exactly C major, it looks really wavy.
It's actually a nice perfect sine curve-- some offset depending on when I hit it-- and then if you apply the Fourier transform what you get is 0's everywhere except for the one frequency that's appearing, and there you get 1, and everywhere else you get 0.
Well, 1 possibly rotated, depending on the phase.
And you can take any audio stream, convert it by a Fourier transform, do manipulations there.
For example, you've probably heard of high-pass filters that removes all the high frequencies, or low-pass filters remove all of the low frequencies.
You just convert to this space and zero out the parts you want, and then you convert back with inverse Fourier transform.
If you've used Adobe Audition or Audacity, they can all do these things.
And there are tons of contexts where converting to Fourier space makes life easy.
And, in general, if you have any time-based signal you should always think about what do you get with FFT when you transform to frequency-based signal, and you can do lots of cool things you couldn't do otherwise.
And it only takes n log n time.
Plus people do it in hardware.
There's a fast implementation called FFTW-- the fastest Fourier transform in the west-- which is made here at MIT a bunch of years ago, and is still the best software implementation of FFT.
So people use it everywhere.
Your noise-cancelling headsets probably use it, MP3 uses it.
It's a cool algorithm.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Welcome back to 6046
Woohoo
Are you guys ready to learn an awesome data structure?
Woohoo
Yeah, let's do it.
This is a data structure named after a human being, Peter van Emde Boas.
I was just corresponding with him yesterday.
And he, in the '70s, he invented this really cool data structure.
Its super fast It's amazing.
It's actually pretty simple to implement.
And it's used a lot, in practice, in network routers, among other things.
And we're going to cover it today.
So let me first tell you what it does.
So it's an old data structure.
But I feel like it's taken us decades to really understand.
Question
You're mic's not on
In what sense?
It's not amplified.
It's just for the cameras.
So it's taken us decades, really, to understand this data structure, exactly how it works and why it's useful.
The problem it's solving is what you might call a predecessor problem.
It's very similar to the sort of problem that binary search trees solve.
But we're going to do it faster, but in a somewhat different model, in that the elements we're going to be storing are not just things that we know how to compare.
That would be the comparison model.
We're storing integers.
And the integers come from a universe, U, of size little u.
And we'll assume that they're non-negative, so from 0 to u minus 1.
Although you could support negative integers without much more effort.
And the operations we want to support, we're storing a set of n of those elements.
We want to do insert, delete, and successor.
So these are operations you should be familiar with.
You should know how to solve these in log n time per operation with a balanced binary search tree, like AVL trees.
You want to add something to the set, delete something from the set, or given a value I want to know the next largest value that is in the set.
So if you draw that as a one dimensional thing, you've got some items which are in your set.
And then, you have a query.
So you ask for the successor of this value.
Then you're asking for, what is the next value that's in the set?
So you want to return this item.
OK, predecessor would be the symmetric thing.
But if you could solve successor, you could usually solve predecessor.
So we'll focus on these three operations, although, in the textbook, you'll see there are lots of operations you could do with van Emde Boas.
So far so good.
We know how to do this in log n time.
We are going to do it in log log u time.
Woah, amazing.
So an extra log, but we're cheating a little bit, in that we're replacing n with u.
Now in a lot of applications, u is pretty reasonable, like 2 to the 32 or 2 to the 64, depending on what kind of integers you usually work with.
So log log of that is usually really tiny, and often smaller than log n. So in particular, on the theory side, for example, if u is a polynomial in n, or even larger than that, you can support n to the polylog n. Then log log u is the same as log log n, up to constant factors.
And so this is an exponential improvement over regular balanced binary search trees.
OK, so super fast, and it's also pretty clean and simple, though it'll take us a little while to get there.
One application for this, as I mentioned, is in network routers.
And I believe most network routers use the van Emde Boas data structure these days, though just changed in the last decade or so.
Network router, you have to store a routing table, which looks like, for IP range from this to this, please send your packets along this port.
For IP range from this to this, send along this port.
So if you mark the beginnings of those ranges as items in your set, and given an actual IP address, you want to know what range it's in, that is a predecessor or a successor problem.
And so van Emde Boas lets you solve that really fast.
u, for IPV4 is only is only 2 to the 32.
So that's super fast and practical.
It's going to take like five operations to do log log 2 to the 32.
So that's it.
And as you may know, network routers are basically computers.
And so they used to have a lot of specialized hardware.
These days it's pretty general purpose.
And so you want nice data structures, like the one we'll cover.
OK, so we want to shoot for log log u.
We're going to get there by a series of improvements on a very simple idea.
This is not the original way that van Emde Boas got to this concept.
But it's sort of the modern take on it.
It's one that's in the textbook.
So the first question is, how might we get a log log u bound?
Where might that come from?
That's a question for you.
This is just intuition.
Any suggestions?
We see logs all the time.
So, yeah
You organize the height of a tree into a tree
Ah, good.
You organize the height of the tree into a tree.
So we normally think of a tree, let's say we have u down here.
So the height is log u.
So somehow, we want a binary search on the levels of this tree.
Right, if we could kind of start in the middle level, and then decide whether we need to go up or down, I'm totally unclear what that would mean.
But in fact, that's exactly the van Emde Boas will do.
So you can binary search-- I think we won't see that until the very end-- but on levels of a tree.
So at least some intuition.
Now let's think about this in terms of recurrences.
There's a recurrence for binary search, which is usually you have k things, t of k is t of k over 2 plus order 1.
You spend constant time to decide whether you should go left or right in a binary search, or in this case up and down somehow.
And then you reduce to a problem of half the size.
So this solves to log k. In our case, k is actually log u.
So we want a recurrence that looks something like t of log u equals t of log u/2 plus order 1.
OK, even if you don't believe in the binary search perspective, this is clearly a recurrence that solves to log log u. I'm just substituting k equals log u here.
So that could be on the right track.
Now, that's in terms of log u.
What if I wanted to rewrite this recurrence in terms u?
What would I get?
If I wanted to have this still solve to log log u, what should I write here?
If I change the logarithm of a number by a factor of 2, how does u change?
Square root
Square root.
OK, So I've changed what the variable is here.
But this is really the same recurrence.
It will still solve to log log u.
The number of times you have to apply square root to a number to get to 1 is log log u.
So this is some more intuition for how van Emde Boas is going to achieve log log u.
And in fact, this is the primary intuition we'll be using.
So what we would like is to some take our problem, which has size u, and split it into problems of size square root of u, so that we only have to recurse on one of them.
And then, we'll get this recurrence.
OK, that's where we're going to go.
But we're going to start with a very simple data structure for representing a set of n numbers from the universe 0 up to u minus 1.
And let's say, initially, our goal is for insert and delete to be constant time.
But let's not worry about successor.
Successor could take linear time.
What would be a good data structure for storing items in this universe?
I want u to be involved somehow.
I don't just want to, like, store them in a linked list of items or assorted array of items.
I would like u to be involved, insert and delete constant time.
Very simple.
Yeah
Simply an array
In an array, yeah.
What's the array indexed by?
It would be index n
Sorry?
By the index of n
The index of n, close
The value
Sorry?
The value
The value, yeah.
Good.
So I want-- this is normally called a bit vector, where I want array of size u, and for each cell in the array, I'm going to write 0 or 1.
0 means absent.
1 means present.
It's in the set.
So let me draw a picture, maybe over here.
Let me take my example and give you a frisbee.
Let me put it in the middle.
So this is an example of a set with-- if I maybe highlight a little bit-- here's 1.
Here's a 1, and a one, and a one.
So there are 4 elements in the set.
The universe size is 16. n equals 4, in this particular example.
If I want to insert into this set, I just change 0 to a 1.
If I want to delete from the set, I change a 1 to a 0.
So those are constant time.
Good.
If I want to do a successor query, not so good.
I might need to spend order u time.
Maybe I asked for the successor of this item, and the only thing to do is just keep jumping until I get to a 1.
And the worst case, there's almost to u 0's in a row, or u minus n. So that's really slow.
But this, in fact, will be our starting point.
It may seem really silly.
But it's actually a good starting point for van Emde Boas.
So the second idea is, we're going to take our universe and split it into clusters.
van Emde Boas, the person, likes to call these galaxies.
I think that's a nice name for pieces of the universe.
But textbook calls it clusters.
Because they used to call it clusters.
So now, it's question of how big the cluster should be.
But I gave you this picture, and I want to think about these galaxies as separate chunks, and I ask for the successor of this, how could I possibly speed up the successor search?
Yeah
You could form a tree for each cluster and connect--
You could form a tree here and store what at the-- [INTERPOSING VOICES]
Could store an or between the two bits
Cool.
I like this.
So I could store the or of these two bits-- clean this up a little bit-- or of these two bits, or of these two bits, and so on.
The or is interesting, because this 0 bit, in particular, tells me there's nothing in here.
So I should just be able to skip over it.
So you're imagining a kind of binary search-ish thing.
It's a good idea.
So each node here, I'm just writing the or of its two children.
And in fact, you could do this all the way up.
You could build an entire binary tree.
But remember, what we're trying to do is a binary search on the levels of the tree.
And so, in particular, I'm going to focus on this level.
This is the middle level of that tree if I drew out the whole thing.
And that level is interesting, because it's just summarizing-- is there anybody in here, is there anybody in this cluster, is there anybody in this cluster, is there anybody in this cluster.
So we call this the summary vector.
So we'll come back to your tree perspective at some point.
That is a good big picture of what's going on.
But at this level, I'm just going to say, well let's store the bit vector.
Let's also store this summary vector.
And now, when I want to find the successor of something, first I'll look inside the cluster.
If I don't find my answer, I'll go up to the summary vector and find where is the next cluster that has something in it.
And then I'll go into that cluster and look for the first one.
OK, that's a good next step.
So this will split the universe into clusters.
How big should the clusters be to balance out?
There's three searches I'm doing.
One is within a cluster.
One is in the summary vector.
And one is within another cluster.
Yeah
Square root u
Square root u. Yeah.
That will balance out.
If there's square root of u size, then the number of clusters will be square root of u.
So the search in the summary vector will be the same as the cost down here.
Also we know that we kind of want to do square root of u recursion somehow.
So this is not yet the recursive version.
But square root of u is exactly right.
And I owe some frisbees, sorry.
Here's one frisbee.
And yeah, cool.
And I think also you one.
Sorry.
So clusters have size square root of u, the square root of u of them.
And, cool.
So now, when I want to do an insert or a delete, it's still-- let's not worry about delete.
That's a little tricky.
To do an insert, it's still easy.
If I insert into here, I set it to 1.
And I check, if this is already 0, I should also set that to 1.
Now deleting would be tricky.
To delete this guy and realize that there's nothing else, eh.
Let's not worry about that until we do a lot more work.
Let's just focus on insert and successor.
So insert, with this strategy, is still constant time.
It's two steps instead of one, but it's good.
Successor does three things.
First, we look, let's say, successor of x.
First thing we do is look in x's cluster.
Then, if we don't find what we're looking for, then we'll look for the next 1 bit in the summary vector, and then, we'll look for the first 1 in that cluster.
So there are two cases.
In the lucky case, we find the successor in the cluster that we started in.
So that only takes root u time.
If we're unlucky, we research in the summary.
That takes root u time.
And then we find the first 1 bit.
That takes root u time.
Whole thing is square root of u, which is, of course, not very good, compared to log n. But it's a lot better than u, which is our first method, the bit vector.
So we've improved from u to square root of u.
Now of course, the idea is to recurse.
Instead of just doing a bit vector at each of these levels, we're going to recursively represent each of these clusters in this way.
This is where things get a little magical, in the magic of divide and conquer.
And then, we'll get t of square root of u instead of square root of u.
And then we'll get a log log cost.
So before I get there, let me give you a little bit of terminology and an example for dealing with clusters.
OK, in general, remember the things we're searching for are just integers.
And what we're talking about is essentially dividing an integer, like x, by square root of u.
And so this is, whatever, the quotient.
And this is the remainder.
So I want j to be between 0 and strictly less than square root of u.
Then this is unique, fundamental theorem of arithmetic, or something.
And i is the cluster number.
And then j is the position of x within that cluster.
So let's do an example like x equals 9.
So I didn't number them over here.
This is x equals 0, 1, 2, 3, 4, 5, 6, 7, 8, 9-- here's the guy I'm interested in-- 10, 11, 12, and so on.
So 9 is here.
This is cluster number 0, 1, 2.
So I claim 9 equals 2 times square root of u.
Here is 4.
I conveniently chose u to be a perfect square.
And it is item 0,1 within the cluster.
And indeed, 9 equals 2 times 4 plus 1.
So in general, if you're given x, and I said, ah, look in x's cluster, what that means is look at x integer divided by square root of u.
That's the cluster number.
And I'll try to search in there.
And I look in the summary vector, starting from that cluster name, the name of the cluster for this guy, finding the next cluster.
Then I'll multiply by square root of u to get here, and then continue on.
In general, because dividing to multiplying-- I don't want to have to think about it too hard.
I'm going to say, define some functions to make this a little easier, more intuitive.
So when I do integer division by square root of u, which is like taking the floor, I'll call that high of x, the high part of x.
And low of x is going to be the remainder.
That's the j up here.
And if I have the high and the low part, the i and the j, I'm going to use index to go back to x.
So index of ij is going to be i times square root of u plus j.
Now why do I call these high and low?
I'll give you a hint.
Here's the binary representation of x.
In this case, high of x is 2.
And low of x is 1.
Yeah
So the high x corresponds to the first two, which is the first 2 bit.
And the low x corresponds to [INAUDIBLE]
Right.
High of x corresponds to the high half of the bits.
And low of x corresponds to the bottom half of the bits.
So these are the high order bits and the low order bits.
And if you think about it, remember when we take square root of u in logarithm, it takes log u and divides it in half.
So it's exactly, in the bit factor, which is log u bits long, we're dividing in half here, and looking at the high bits versus the low bits.
OK?
So that's another interpretation of what this is doing.
And if you don't like doing division, as many computers don't like to do, all we're actually doing is masking out these bits, or taking these bits and shifting them over.
So these are very efficient to actually do.
And maybe get some intuition for why they're relevant.
So let's recurse, shall we?
I think now we know how this splitting things up works.
So I'm going to call the overall structure v, or a van Emde Boas structure I'm trying to represent is v. And v is going to consist of two parts.
One is an array of all of the clusters.
I'm going to abbreviate van Emde Boas as VEB.
And recursively, each of those clusters is going to be represented by a smaller VEB structure, of size square root of the given one.
OK, and i ranges from 0 to square root of u minus 1.
OK, so there's square root of u of them.
Total sizes is u.
And then, in addition, we're going to have a summary structure.
And this is also a size square root of u VEB.
OK, you should think about inserts and successors.
Those are the two operations I care about for now.
Let's start with insert.
That's easier.
So if I want to insert an item, x, into data structure v, then first thing I should do is insert into its corresponding cluster.
So let's just get comfortable with that notation.
We're inserting into the cluster whose number is high of x.
That is where x belongs.
The name of its cluster should be high of x.
And what we're going to be inserting recursively into there is low of x.
That is the name of x local to that cluster.
x is a global name with respect to v. This cluster only represents a small range of square root of u items.
So this gets us from the big space of size u to the small space of size square root of u within that cluster.
So that's basically what high and low were made for.
But then, we have to also update the summary structure.
So we need, just in case-- Maybe it's already there.
But in the worst case, it isn't.
So we'll just think of that as recursively inserting into v dot summary the name of the cluster, which is high of x.
High of x is keeping track of which clusters are non-empty.
We've just inserted something into this cluster.
So it's non-empty.
We better mark that that cluster, high of x, is non-empty in the summary structure.
Why?
So we can do successor.
So let's move on to successor.
Actually, I want to mimic the successor written here on the bottom of the board.
So what we had in the non-recursive version was three steps.
So we're going to do the same thing here.
We're going to look within x's cluster.
We now know that is the cluster known as high of x.
And either we find, and we're happy, or we don't.
Then we're going to look at v dot summary search for this the successor of high of x.
Right, finding the next 1 bit, that is successor.
And then, I want to find the first 1 bit in that cluster.
Is that a successor also?
Yeah.
That's just the successor of negative infinity.
Finding the minimum element in a cluster is the successor of -1, or 0, or not zero.
But -1 would work, or negative infinity, maybe more intuitively.
That'll find the smallest thing here.
So each of these is a recursive call.
I can think of it as recursively calling successor.
So let's do that.
I want to find the successor of x in v. First thing I'm going to do is do the ij breakdown.
I'll let i be high of x and j be-- I could do low of x.
But what I'm going to try for is to search within this cluster, high of x.
So I'm going to look for the successor of cluster i, which is cluster high of x, of low of x. OK, so that's this first step of looking in x's cluster.
This is x's cluster.
This is x's name in the cluster.
I'm going to try to find the successor.
But it might say infinity.
I didn't find anything.
And then I'll be unhappy if j equals infinity.
So that's line one.
Well, then we're in the wrong cluster.
High of x is not the right cluster.
Let's find the correct cluster, which is going to be the next non-empty cluster.
So I'm going to change i to be the successor in the summary structure of i.
So i was the name of a cluster.
It may have items in it.
But we want to find the next non-empty thing.
Because we know the successor we're looking for is not here.
OK.
So this is the cluster we now belong in.
What item in the cluster do we want?
Well, we want to find the minimum item in that cluster.
And we're going to do that by a recursive call, which is j is the successor within cluster i of minus infinity, I'll say.
-1 would also work.
So this will find the smallest item in the cluster.
And then, in both cases, we get i and j, which together in this form describe the value x that we care about.
So I'm just going to say, return index of ij.
That's how we reconstruct an item name for the structure v. We knew which substructure it's in.
And we know its name within the substructure, within the cluster.
Is this algorithm clearly correct?
Good.
It's also really bad.
Well, it's better than everything we've done so far.
The last result we had was square root of u.
This is going to be better than that, but still not log log u.
Why?
Both of these are bad.
Yeah
You make more than one call to [?
your insert.
?]
Right.
I make more than one recursive call to whatever the operation is here.
Insert calls insert twice.
Here, successor calls successor potentially three times.
This is a good challenge for me.
Let's see.
Eh, not bad.
Off by one.
OK, that's a common problem in computer science, right?
Always off by one errors.
OK, so let's think of it in terms of recurrences, in case that's not clear.
Here we have t of u is 2 times t of square root of u.
Right, to solve a problem of size u, I solve two problems of size square root of u plus constant.
Because high of x and low of x, I'm assuming, take constant time to do.
It's just, I have an integer.
I divide it in half.
Those are cheap.
What does this solve to?
It's probably easier to think of it in terms of log u.
Then we could apply the master method.
Right, this is the same thing as t prime of log u is 2 times t of log u divided by 2 plus order 1.
This is not quite the merge sort recurrence.
But it's not good.
One way to think of it, is we start with the total weight of log u.
We split into log over 2, but two copies of it.
So we're not saving anything.
And we didn't reduce the problem strictly.
In terms of the recursion tree, we have, you know, log u-- well, it's hard to think about because we have constant total cost.
You could just plug this in with the Master method, or see that essentially we're conserving mass.
We started with log u mass.
We have two copies of log u over 2.
That's the same total mass.
So how many recursions do we do?
Well we do do log log u recursions.
The total number of leaves in that recursion tree is log u.
Each of them, we pay constant.
So this is log u, not log log u.
To get log log u, we need to change this 2 into a 1.
We can only afford one recursive call.
If we have two recursive calls, we get logarithmic performance.
If we have three recursive calls, it's even worse.
Here, I would definitely use the Master method.
It's less obvious.
In this case, we get log u to the log base 2 of 3 power, which is log u to the 1.6 or so, so both worse than log n. This is strictly worse than log n. This is maybe just a little bit worse than log n, depending on how u relates to n. OK, so we're not there yet.
But we're on the right track.
We have the right kind of structure.
We have a problem of size u.
We split it up into square root of u sub problems of size u.
From a data structures perspective, this the first time we're using divide and conquer for data structures.
It's a little different from algorithms.
So that's how the data structure is being laid out.
But now we're worried about the algorithms on those data structures.
Those, we can only afford t of u equals 1 times [?
t of ?]
squared of u plus order 1.
Then we get log log u.
So, here we have two recursive calls.
Somehow we have to have only one.
Let's start by fixing insert.
Insert?
No.
Let's start by fixing successor.
I think that will be more intuitive.
Let's look at successor.
Because successor is almost there.
A lot of the time, it's just going to make this call, and we're happy.
The bad cases is when we need that make both of these calls.
Then there's three total, very bad.
How could I get rid of this call?
I was being all clever, that the minimum element is the successor of negative infinity.
But that's actually not the right idea.
Yeah.
[?
Catching ?]
the minimum element in cluster i
Store the minimum element of cluster i. Yeah.
In general, for every structure v, let's store the minimum.
Why not?
We know how to augment structures.
Here in 006, you took an AVL tree, and you augment node to store the sub-tree size of the node.
In this case, we're doing a similar kind of augmentation.
Just for every structure, keep track of what the minimum is.
So that will be idea number four.
I'm going to add something here.
But for now, let's store the minimums.
So to do an insert into to structure v, item x, first thing we'll do is just say, well, if x is-- let's see if it's the new minimum.
Maybe x is smaller than v dot min.
If that's the case, let's just set v dot min to x. OK?
And then, the rest is the same, same insertion algorithm as over here, these two recursive calls.
I just spent constant additional time.
And now every structure knows it's minimum.
Again, ignore delete for now.
That's trickier.
OK, now every structure knows its minimum, which means we can replace this call with just v dot cluster i dot min.
One down.
OK, so if we look at successor, of v comma x. I'm going to replace the last line, or next to last line with j equals v cluster i dot min.
So now, we're down to log u performance.
We only have, at most, two recursive calls.
So that's partial progress.
But we need another idea to get rid of the second one.
And the intuition here is that really, only one of these call should matter.
OK, let's draw the big picture.
Here's what the recursive thing looks like.
We've got v dot summary.
Then we've got a cluster 0, cluster 1, cluster square root of u minus 1.
Each of those is a recursive structure.
And we're also just storing the min over here as a copy.
So when I do a query for, I don't know, the successor of this guy, there's kind of two cases.
One situation is that I find the successor somewhere in this interval.
In that case, I'm happy.
Because I just need this one recursive call.
OK, the other case is that I don't find what I'm looking for here.
Then I have to do a successor up here.
And then I'm done.
Then I can teleport into whatever cluster it is.
And I've stored the min by now.
So that's constant time to jump into the right spot in the cluster.
So either I find what I'm looking for here, or I find what I'm looking for here.
What would be really nice is if I could tell ahead of time which one is going to succeed.
Because then, if I know this is not going to find anything, I might as well just go immediately up here, and look at the successor in the summary structure.
If I know I'm going to find something here, I'll just do the successor here.
And I'm done.
If I could just get away with one or the other of these calls, not both, I'd be very happy.
How could I tell that?
Yeah
Store the max
Store the max.
Store the min and the max.
Why not?
OK, I just need a similar line here.
If x is bigger than v dot max, change the max.
So now, I've augmented my data structure to have the min and max at every level.
And what's going on here is, I won't find an answer if I am greater than or equal to the maximum within this cluster.
That's how I tell.
If I'm equal to the max, or if I'm beyond the max, if all the items are over here, the max will be to my left.
And then I know I will fail within the cluster.
So I might as well just go up to summary and do it there.
On the other hand, if I'm less than the max, then I'm guaranteed I will find something in this cluster.
And so I can just search in there.
So all I need to do-- I'll probably have to rewrite this slightly.
If x is-- not x, close.
I'm going to mimic this code a little bit, at least the first line is going to be i equals high of x.
And now, that's the cluster I'm starting in.
And I want to look at the maximum of that cluster.
So I'm looking at v dot cluster i dot max.
And I want to know, is x before that?
Now within that cluster, x is known as low of x.
So I compare low of x to cluster i's maximum element.
If we're strictly to the left, then there is a successor guaranteed within that substructure.
And so, I should do this line.
I wish I could copy paste.
But I will copy by hand.
Successor within v dot cluster i, of low of x. OK, then I've found the item I'm looking for.
Else, I'm beyond the max, I know this is the wrong cluster.
And so I should immediately do these two lines, well, except I've made the second line use the min.
So it will only be one recursive call, followed by a min.
OK, so this is going to be i equals the successor within v dot summary of high of x.
And then j is that line successor within-- oh, sorry-- the line that I used to have here, which is going to be v cluster i dot min.
OK, and then, in both cases, I return index of ij.
OK, so we're doing essentially the same logic as over here.
Although I've replaced the step with the min, to get rid of that recursive call.
But I'm really only doing one or the other of these, using max to distinguish.
If I'm left of the max, I do the successor within cluster high of x.
If I'm right of the max, then I do the successor immediately in summary structure.
Because I know this won't find anything useful.
And then I find the min within that non-empty structure.
And in both cases, ij is the element I'm looking for.
I put it back together with index.
Clear?
What's the running time of successor now?
Log log u.
Awesome.
We've finished successor.
Sadly, we have not finished insert.
Insert still takes log u time.
But, b progress.
Maybe your routing table doesn't change that often, so you can afford to pay some extra time for insert, as long as you can route packets really fast, as long as you can find where something belongs, the successor in log log u time.
But for kicks, let's do insert in log log u as well.
This is going to be a little harder, or I would say a more surprising idea.
This may be-- I don't have a great intuition for this step.
I'm thinking.
But again, most of the time, this should be fine, right?
Most of the time, we insert into cluster high of x, low of x, and we're done.
As long as there is something already in that cluster, we don't need to update the summary structure.
As long as high of x has already been inserted into the summary structure, we can get away with just this first step.
The tricky part is detecting.
How would we know?
Well, that's not enough just to detect it.
If high of x is not in v dot summary, we have to do this insert.
We can't get away with it.
But that's kind of rare.
That only happens the very first time you insert into that cluster.
Every subsequent time, it's going to be really cheap.
We just have to do this.
It's easy enough to keep track of whether a cluster is empty.
For example, we're storing the min.
We can say v dot min is none, special value, whenever the structure v is empty.
But we still have this problem, that the first time we insert into a cluster, it's expensive.
Because we have to do this.
And we have to do this.
How could we avoid, in the case where a cluster is empty-- remember, an overall structure looks like this.
We can tell that it's empty by saying min equals none, let's say.
What could I do?
Sorry, there's also a max now.
What could I do to speed up inserting into an empty cluster?
Because I'm first going to have to insert into the empty cluster.
Then I'm going to have to answer into the summary.
I can't get away from this.
So I'd like this to become cheap, in the special case when this cluster is empty.
Yeah
Lazy propogation
Lazy propagation-- you want to elaborate?
Yeah.
We mark the place we want to insert in.
And then we will take it down whenever we [?
insert ?]
there
Good.
So when I insert into an empty structure, I'm just going to have a little lazy field, or something.
And I'll put the item in there.
And then the next time I insert into it, maybe I'll carry it down a little bit.
That actually works.
And that was the original van Emde Boas structure, [?
I ?]
[?
learned ?]
[?
recently.
?]
So that works.
But it's a little more complicated than the solution I have in mind.
So I'm going to unify that lazy field with the minimum field.
Say, when I insert into a structure, if there's nothing here, I'm just going to put the item there, and not recurse.
I just am not going to store the minimum item recursively.
Definitely frisbee.
So that's the last idea, pretty much.
Idea number five is, don't store the min recursively.
This is effectively equivalent to lazy.
But we're actually just never going to get around to moving this guy down.
Just leave it there.
First, if the min field is blank, store the item there.
Yeah
What do you mean by moving the guy down?
Don't worry about moving the guy down.
We're not going to do it
[INAUDIBLE]
But in general, moving down means, when I want to insert an item, I have to move it down into its sub cluster.
So I want to insert x into the cluster, high of x with low of x, that recursive call.
That's moving it down.
I'm not going to do that.
If the structure is empty, I'm going to set v dot min equal to x, and then stop.
Let me illustrate with some code, maybe over here.
Here's what I mean.
If v dot min is special none value-- use Python notation here-- then I'm just going to set v dot min equal to x. I should also set v dot max equal to x.
Because I want to keep track of the maximum.
And then, stop.
Return.
That's all I will do for inserting into an empty structure, is stick it in the max field.
OK, this may seem like a minor change.
But it's going to make this cheap.
So the rest of the algorithm is going to be pretty similar.
There's a couple annoying special cases, which is, we have to keep the min up to date.
And we have to keep the max up to date, in general.
This one is easy.
We just set v dot max equal to x.
Because we're not doing anything fancy with max.
Min is a little special.
Because if we're inserting an item smaller than the current minimum, then really x belongs in the slot.
And then whatever was in here needs to be recursively inserted.
OK, so I'm going to say swap x with v dot min.
So I'm going to put x into the v dot min slot.
And I'm going to pull out whatever item was in there and call it x now.
And now my remaining goal is to insert x into the rest of the structure.
There's only one item that gets this freedom of not being recursively stored.
And it's always going to be the minimum one.
So this way, the new value x goes there.
Whatever it used to be there now has to be recursively inserted.
Because every item except the minimum, we're going to recursively insert.
So the rest is pretty much the same.
But we're going to, instead of always inserting into the summary structure, we're going to see whether it's necessary.
Because we know how to do that.
We just look at a cluster high of x.
And we see, is it empty?
Cluster high of x-- and empty means its minimum is none.
So we're going to-- in fact, the next line after this one is going to be insert v cluster high of x, comma low of x.
All right, that's this line.
We're always going to do that.
And in the special case, where there was not previously nothing in v cluster high of x, we need to update the summary structure.
And we do that with this line.
So I'm going to insert into v dot summary high of x.
But I'm only doing that in the case when I need to.
If it was already non-empty, I know this has already happened.
So I don't need to bother with that insertion.
OK, this is a weird algorithm.
Because it doesn't look much better.
In the worst case, we're doing two recursive calls to insert.
But I claim this runs in log log u time.
Why?
Yeah
Because when we update the v dot summary, we [?
just ?]
[?
have the ?]
[?
first ?]
[?
line.
?]
Good.
Yeah.
In the case when I have to do this summary insertion, I know this guy was empty.
Cluster high of x was empty.
So this call is just going to do these two lines.
Because I optimized the case of empty-- when a structure is empty, I spend constant time, no recursive calls.
That means in the case when cluster high of x is empty, and I have to pay to insert into the summary structure, I know my second call is going to be free, only take constant time.
So either I do this, in which case this takes constant time, or I don't do this, in which case I make one recursive call.
In both cases, I really am only making one recursive call.
OK, so this runs in log log u.
Because I get the t of u equals 1 times square root of t of u plus order 1 recurrence.
All the work I'm doing here is constant time, other than the recursive calls.
Question?
So when we insert the first time, we don't update v dot summary?
When I insert into a completely empty structure, we don't update summary at all.
That's right.
We just store it in the min, and we're done
Oh.
So then, if you were to [?
call ?]
the successor, and you--
Good.
Yeah.
The successor algorithm is currently incorrect.
Thank you.
Here's some frisbees for that question and the last answer.
Yeah.
This code is now slightly wrong.
Because sometimes I'm storing elements in v dot min.
And successor is just completely ignoring them.
So it's not going to find those items.
Luckily, it's a very simple fix.
Out of room, but please insert right in here.
If x is less v dot min, return v dot min.
That's all we need to do.
The min is special.
Because we're not storing it recursively.
And so, we can't rely on all of our recursive structures.
We can't rely on cluster i.
We can't rely on summary, on reporting about v dot min.
v dot min is just a special item sitting there.
It's represented nowhere else.
But we can check.
Because it's the minimum element, and we're looking for successors, it's really easy to check for whether it's the item we're looking for.
Because it's the smallest one.
If we're smaller than it, then that's clearly the successor.
OK, so in that case, we just spent constant time.
So it actually speeds up some situations for successor.
We're not exploiting that here.
It doesn't help much in the worst case.
But now, it should be correct.
Hopefully, you're happy.
Any other questions?
So at this point, we have what I will call a van Emde Boas.
This last version-- we can do insert and successor in log log u time.
Yeah, sorry.
I modified the wrong successor algorithm, didn't I?
I meant to modify this one.
This is the fast one.
So please put that code here.
That's the log log u version of successor.
We just added this constant time check.
And now this runs in log log u time.
The key idea here was if we store the max, then we know which of the two recursive calls we need to do.
If we store the min, this doesn't end up being a recursive call.
So that's very clean.
With insert, we needed this trickier idea that the min, we're not even going to recursively represent.
We'll just keep it there.
That requires this extra little check for successor.
But it allows us to do insert cheaply in all cases-- cheap meaning only one recursive call.
Either we need to update the summary structure, in which case that thing was empty, and so we can think of that cluster-- so we have this special case of inserting into an empty cluster, which is super cheap, or most of the time, you imagine that the cluster was already non-empty.
And so we don't need to update the summary structure.
And then we just do this recursion.
So in all cases, everything is cheap.
Now the one thing I've been avoiding is delete.
Yeah, question
[INAUDIBLE] If x is greater than [?
v ?]
max, [?
we ?]
[?
swap ?]
[?
x ?]
[?
with ?]
[?
v ?]
[?
max?
?]
So if x is greater than v max, I'm just going to update v max.
V max is stored recursively.
We're not doing anything fancy with v max.
And we had, at some point, a similar line.
So this is just updating v max.
Yeah, nothing special there.
In your problem set, you'll look at a more symmetric version, where you don't recursively store min and max.
It works about the same.
But in some ways, the code is actually prettier.
So you'll get to do that.
Other questions?
All right.
So, delete.
We have insert and successor.
And through all these steps, it would actually be very hard to do delete.
It turns out, at this point, delete is no problem.
So let me give you some delete codes.
It's a little bit long.
Maybe I'll start with a high level picture, sort of the main cases.
Deleting the min is a little bit special, as you might imagine.
That element is different from every other element.
So if x equals min, we're going to do something else.
But let me specify that later.
Let's get to the bulk of the code, which is we're going to delete low of x from cluster high of x.
That's the obvious recursion to do.
This is essentially the reverse of insert over here.
The first thing we do is undo this.
In all cases, insert was doing that.
So in all cases, delete has to do that, other than the special case of the min.
And then, we need to do the inverse of this.
So if that was the last item, then we need to delete from the summary structure.
So it's actually pretty symmetric, other than the tiny details.
So after we delete, we can check, is that structure empty.
Because then, the min would equal none.
OK.
If that's the case, we delete from the summary structure.
OK.
Cool.
And there is a bit of a special case at the end, which is when we deleted the maximum element.
OK, so I need to fill these in.
And it's important that these are filled in right.
Because in some situations here, we are making two recursive calls.
But again, we'd like it to be, when we do both calls, we want one of them to be cheap.
Now this one's hard to make cheap.
So when we delete from the summary structure, we want this to delete to have taken only constant time, no recursions.
And that's going to correspond to this case.
Because if we made the cluster empty, that means we deleted the last item.
What's the last item?
Has to be v dot min.
If you have a size 1 structure, it's always because that item is in v dot min, everything else is empty.
So that's the case of deleting v dot min.
So we want this case to take constant time when it's the last item we're deleting.
So let's fill that in a little.
Let's see if I can fit it in.
This is code that turns out to work in this if x equals v dot min.
It's a little bit subtle.
But the key thing to check here is, we want to know, is this the last item.
And one way to do that is to look at the summary structure, and say, do you have any non-empty clusters?
If you don't have any non-empty clusters, that means your min is none.
And that means, the only thing keeping the structure non-empty is the minimum element.
That's stored in v dot min.
So in that case, that's the one situation when v dot min becomes none.
We never set v dot min equals none in the other algorithms.
Because initially everything is none.
But when we're inserting, we never empty a structure.
Now we're doing delete.
This is the one situation when v dot min becomes none from scratch.
In that case, no recursive calls.
So that means this algorithm is efficient.
Because if I had to delete from the summary structure, this only had a single item, which is this situation.
Then I just set v dot min equals to none.
And I'm done.
So this will, overall, run in log log u time.
Now, it could be we're deleting the min, but it was not the only item.
So that's this situation.
In that situation, we want to find out what the min actually is.
Right?
We just deleted the min.
We want to put something in v dot min.
We can't set it to none.
Because that indicates the whole structure is empty.
So we have to recursively rip out the new minimum out item.
Because it should not be recursively stored anymore.
And then we're going to stick it into v dot min.
So now, finding minimum items is actually pretty easy.
We just looked at the first non-empty structure.
And we looked at the-- I think I'm missing-- oh, v dot cluster i min, I guess, closed parenthesis.
That is the minimum item in the first cluster.
So I want to recursively delete it.
So I'm setting x to that thing.
And then I'm going to do all this code, which will delete x from that structure.
And then-- I mean, I'm doing it all right here.
But then, I'm going to set v dot min to be that value.
So then v dot min has a new value.
Because I deleted the old one.
And it's no longer recursively stored.
I don't want two copies of x floating around.
So that's why I do, even in this if case, I do all these steps.
Cool?
You can see delete-- is that a question?
[INAUDIBLE]
Oh, why did I set v dot max to none?
Because [?
that's the ?]
all [?
these ?]
[INAUDIBLE] [?
x ?]
equals v dot max, the last time
[?
Do you ?]
[?
find v dot max?
?]
Oh, right.
I'm not done yet.
I haven't specified what to do here.
OK, you really want to know?
OK. Let's go somewhere else.
I have enough room, I think.
Eh, maybe I can squeeze it in.
It's going to be super compact.
So, when x equals v dot max, there are two cases.
So max is a little different.
We just need to keep it up to date.
So it's not that hard.
We don't have to do any recursive magic.
Well, I need another line.
Sorry.
Let me go up to the other board.
OK, I think that's the complete delete code.
You asked for it.
You've got it.
So, at this point, we have just deleted the max, which means we need to find, basically, the predecessor of x.
But we can't afford a recursive call.
I mean, that's OK.
It's just, we're trying to find the max in what remains.
Imagine v dot max is just wrong.
So we've got to set it from scratch.
It's not that hard to do.
Basically, we want to take the last non-empty structure.
That would v dot summary dot max, and then find the last item in that cluster.
OK, so cluster i is the last one for v dot summary.
And then we look v dot cluster of i dot max.
And we combine it with i.
That gives us the name of that item in the last cluster, the last non-empty cluster.
But there's a special case, which is maybe this returns none.
Maybe there actually is nothing in v dot summary.
That means we just deleted the last item, I guess.
Or there's only one left.
We deleted the next to last time.
Now there's only one item left, namely v dot min.
So we set v dot max equal to v dot min.
So that's a special case.
But most the time, you're just doing a couple dot max's, and you're done.
So that's how you maintain the maxes, even when you're deleting.
And unless I made an error, I think all these algorithms work together.
You're going to insert, delete, and successor.
And symmetrically, you can do predecessor in log log u time per operation, super fast.
Let me tell you a couple other things.
One is, there's a matching lower bound.
Log log-- maybe you wonder, can I get log log log time, log log log log time, or whatever?
No.
In most reasonable choices of parameters-- it's a little bit more complicated than this-- but for most of the time that you care about, log log u is the right answer.
This was proved in 2007.
So it took us decades to really understand.
It's by a former MIT student.
So I'll give you some range where it holds, which will raise another issue.
But, OK.
So this range is the range I talked about before.
This is when log log u equals log log n. So that's kind of the case where you care about applying it.
If log log u is more like log n, it's not so interesting.
But as long as u is not too big, this is a little bit bigger than polynomial n. Then this is the right answer.
Now technically, you need another assumption, which is the space of your data structure is not to super linear.
Now this is a little awkward.
Because the space of this data show structure is actually order u, not n. So the last issue is space.
Space is order u.
Let me go back to this binary tree picture.
So we had the idea of, well, there's all these bits at the bottom.
We're building a big binary tree above those.
The leaves are the actual data.
And then we're summarizing, by for every node, we're writing the or of the two nodes below it, which is summarizing whether that thing is non-empty.
What van Emde Boas is doing-- so first of all, you see that the total number of nodes in this tree is order u.
Because there's u leaves.
The total size of a binary tree with u leaves is order u, 2u minus 1, right?
And you can kind of see what van Emde Boas is doing here.
First, it's thinking about the middle level.
Now it's not directly looking at these bits.
It says, hey look, I know my item, the thing I'm doing a successor of, let's say, is three.
I want to know the successor of this position.
First, I want to check, should I recurse in this block, or should I recurse in the summary block-- which I didn't draw.
But it's the part of the tree that would be up here.
And that's exactly what we're doing with successor.
Should we recursively look within cluster i?
Or should we look within the summary structure?
We only do one or the other.
And that's the sense in which we are binary searching on the levels of this tree.
Either we will spend all of our work recursively looking for the successor within the summary structure, which is like finding the next 1 bit in this row, the middle row, or we will spend all of our time doing successor in here.
And we can do that.
Because we have the max augmented.
OK, but that's the sense in which, kind of, you are binary searching in the levels of this tree.
So that's that early intuition for van Emde Boas is kind of what we're doing.
The trouble is, to store that tree takes order u space.
We'd really like to spend order n space.
And I have four minutes.
So you'll see part of the answer to this.
My poor microphone.
Let me give you an idea of how to fix the space bound.
Let's erase some algorithms.
The main idea here is only store non-empty clusters, pretty simple idea.
We want to spend space only for the present items, not for the absent ones.
So don't store the absent ones.
In particular, we're doing all this work around when clusters are empty, in which case we can see that just by looking at the min item, or when they're non-empty.
So let's just store the non-empty ones.
That will get you down to almost order n space, not quite, but close.
To do this, v dot cluster is no longer an array.
Just make it a hash table, a dictionary in Python.
So v dot cluster-- we were always doing v dot cluster of i.
Just make that into dictionary instead of an array.
And you save most of the space.
You only have to store the non-empty items.
And you should know from 006, hash table is constant expected.
We'll prove that formally in lecture eight, I think.
But for now, take hashing as given.
Everything we did before is essentially the same cost, but an expectation, no longer worst case.
But now the space goes way down.
Because if you look at an item, when you insert an item, it sort of goes to log log u different places, in the worst case.
But, yeah.
We end up with n log log u space, which is pretty good, almost linear space.
It's a little tricky to see why you get log log u.
But I guess if you look at the insert algorithm, even though we had two recursive calls in the worst case.
One of them was free.
When we do both of them, we insert here.
This one happens to be free.
Because it was empty.
But we still pay for it.
We set v dot min equal to x.
And so that structure went from empty to non-empty.
So this costs 1.
And then we recursively call insert v dot summary on high of x.
So we might, when we insert one item x, if lots of things were empty, actually log log u structures become non-empty, and that's why you pay log log u for each item you insert.
It's kind of annoying.
There is a fix, which is in my notes.
You can read it, for reducing this further to order n. But, OK, I have 30 seconds to explain it.
The idea is-- you're not responsible for knowing it.
This is just in case you're curious.
The idea is, instead of going all the way down in the recursion, at the very bottom, you say, well, normally if you stop the recursion when you have u equals 1, just stop the recursion when n is very small, like log log u.
When I'm only storing log log u items, put them in a linked list.
I don't care.
You can do whatever you want on log log u items in log log u time.
It's just a tiny tweak.
But it turns out, it gets rid of that log u in the space.
So it's a little bit messier.
And I don't know if you'd want to implement it that way.
But you can reduce to linear space.
And that's van Emde Boas.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Welcome back to 6046.
Today we continue our theme of data structures but this time, instead of doing a fancy cool data structure, we're going to look at fancy cool analysis techniques for data structures.
And these are useful for tons of different data structures, especially in the context when you're using a data structure to implement an algorithm.
For example, in Dijkstra, when you learn Dijkstra's algorithm, you had lots of different heap structures you could use for the priority queue in Dijkstra, and they gave different running times with Dijkstra.
But the key thing in that context is that you cared about the total running time of the algorithm, less than you cared about the individual running time of each operation.
That's what amortization is about.
It's, I guess, a technique from financial analysis, but we've appropriated it in computer science as an analysis technique to say, well, let's not worry about every single operation worst case cost, let's just worry about the total operation, the sum of all the operations cost.
That's the whole idea of amortization, but there's a lot of different ways to do it.
We're going to cover four different methods for doing it, and three-ish examples of doing it today.
You've seen some essentially in recitation last time, and you've seen a little bit in 6006, so let me first remind you of the example of table doubling from 6006.
This came up in the context of hash tables.
As you may recall, if you store n items in a hash table of size m-- there are m slots in the table, let's say, using chaining-- hashing with chaining-- then we got an expected cost constant plus the load factor, size of the table divided by the number of items.
So we wanted to get constant expected, and so we wanted this to always be, at most, a constant.
I guess we could handle a larger table size, although then we are unhappy about our space, but we definitely want m to be at least around n so that this works out to order one.
And the solution for doing that was table doubling.
Whenever the table is too big, double it-- or sorry-- whenever the table is too small and we have too many items, double the size of the table.
If n-- n is the thing that we can't control.
That's the number of items somebody is inserting into the table.
If n grows to the value to match m, then double m. So m prime equals 2m, and to double the table size, you have to allocate a new array of double the size and copy over all the items, and that involves hashing.
But overall this will take order, size of the table, work.
Doesn't matter whether I'm using m or m prime here, because they're within a constant factor of each other, and that's bad.
Linear time to do an insertion is clearly bad.
This is all during one insertion operation that this would happen, but overall it's not going to be bad, because you only double log n times.
And if you look at the total cost-- so maybe you think, oh, is it log n per operation, but it's not so bad because total cost for n insertions starting from an empty structure is something like 2 to the 0-- this is a big theta outside-- 2 to the 1, 2 to the 2.
If we're only doing insertions, this is great.
2 to the log n. This is a geometric series and so this is order n. Theta head I guess.
So to do n insertions, cost theta n, so we'd like to say the amortized cost per operation is constant, because we did n operations.
Total cost was n, so sort of on average per operation, that was the only constant.
So this is the sense in which hash tables are constant, expected, amortized.
And we'll get back to hashing in a future lecture, probably I think lecture 8, but for now we're just going to think about this as a general thing where you need table doubling, then this gives you a fast way to insert into a table.
Later we'll think about deleting from a table and keeping the space not too big, but that's a starting point.
This is an example of a general technique called the aggregate method, which is probably the weakest method for doing amortization but maybe the most intuitive one.
So the aggregate method says, well, we do some sequence of operations.
Let's say, in general, there are k operations.
Measure the total cost of those operations, divide by k, that's the amortized cost per operation.
You can think of this as a definition, but it's not actually going to be our definition of amortized cost.
We're going to use a more flexible definition, but for simple examples like this, it's a fine definition, and it gives you what you want.
When your sequence of operations is very clear, like here, there's only one thing you can do at each step, which is insert-- that's my definition of the problem-- then great, we get a very simple sum.
As soon as you mix inserts and deletes, the sum is not so clear.
But in some situations, the sum is really clean, so you just compute the sum, divide by a number of operations, you get a cost, and that could be the amortized cost.
And that's the aggregate method, works great for simple sums.
Here's another example where it-- no, sorry.
Let me now give you the general definition of amortized bounds, which becomes important once you're dealing with different types of operations.
I want to say an insert costs one bound amortized and maybe a delete costs some other bound.
So what you get to do is assign a cost for each operation.
I should call it an amortized cost, such that you preserve the sum of those costs.
So what I mean is that if I look at the sum over all operations of the amortized cost of that operation, and I compare that with the sum of all the actual costs of the operations, the amortize should always be bigger, because I always want an upper bound on my actual cost.
So if I can prove that the amortized costs are, at most, say, constant per operation, then I get that the sum of the actual cost is, at most, constant per operation.
I don't learn anything about the individual costs, but I learn about the total cost.
And in the context of an algorithm like Dijkstra's algorithm, you only care about the total cost, because you don't care about the shortest paths at time t, you only care about the shortest paths when the algorithm is completely finished.
So in a lot of situations, maybe not a real-time system, but almost everything else, you just care about the sum of the costs.
As long as that's small, you can afford the occasional expensive operation.
So this is a more flexible definition.
One option would be to assign the average cost to each operation, but we have a whole bunch more operations.
We could say inserts cost more than deletes or things like that.
In fact, let me do such an example.
A couple weeks ago, you learned about 2-3 trees.
This would work for any structure though.
So I claim I'm going to look at three operations on 2-3 trees.
One is create an empty tree, so I need to think about how we're getting started in amortization.
Let's say you always start with an empty tree.
It takes constant time to make one.
I pay log n time-- I'm going to tweak that a little bit-- for an insertion, and I pay 0 time per delete in an amortized sense.
You can write big O of 0 if you like.
Same thing.
So deletion you can think of as a free operation.
Why?
This is a bit counter-intuitive because, of course, in reality the actual cost of a deletion is going to be log n. Yeah
You can never delete more elements than you've already inserted
You can never delete more elements than you've already inserted.
Good
Can you cap the cost of [INAUDIBLE]
Yeah.
So I can bound the deletion cost by the insertion cost, and in the context of just the aggregate method, you could look at the total cost of all the operations.
I guess we're not exactly dividing here, but if we look at the total cost, let's say that we do c creations, i insertions and d deletions, then the total cost becomes c plus i times log n plus d times the log n. And the point is d is less than or equal to i, because you can never delete an item that wasn't already inserted if you're starting from an empty structure.
And so this is i plus d times log n, but that's just, at most, twice i times log n, so we get c plus i log n. And so we can think of that as having a d times 0, 0 cost per deletion.
So this is the sum of the actual costs over here.
This is the sum of the amortized costs, where we say 0 for the deletion, and we just showed that this is an upper bound on that, so we're happy.
Now, there's a slight catch here, and that's why I wrote star on every n, which is not every operation has the same cost, right?
When you start from an empty structure, insertion cost constant time, because n is 0.
When n is a constant, insertion is constant time.
When n grows to n, it costs log n time.
At different times, n is a different value, and n I'm going to use to mean the current size of the structure.
For this argument to work at the moment, I need that n is not the current value, because this is kind of charging work.
Some insertions are for large structures, some are for small structures, some deletions are for small, some are for large.
Gets confusing to think about.
We will fix that in a moment but, for now, I'm just going to define n star to be the maximum size over all time.
OK, if we just define it that way, then this is true.
That will let me pay for any deletion, but we'll remove that star later on once we get better analysis methods, but so far so good.
Two very simple examples-- table doubling, 2-3 trees with free deletion.
Of course, that would work for any structure with logarithmic insertion and deletion, but we're going to be using 2-3 trees in a more-- analyzing them in a more interesting way later on.
So let's go to the next method, which is the accounting method.
It's like the bank teller's analysis, if you will.
These are all just different ways to compute these sums or to think about the sums, and usually one method is a lot easier, either for you personally or for each problem, more typically.
Each problem usually one or more of these methods is going to be more intuitive than the others.
They're all kind of equivalent, but it's good to have them all in your mind so you can just think about the problem in different ways.
So with the accounting method, what we're going to do is define a bank account and an operation can store credit in that bank account.
Credits maybe not the best word, because you're not allowed for the bank account to go negative.
The bank account must always be non-negative balance, because otherwise your summations won't work out.
So when you store credit in the bank account, you pay for it.
It's as if you're consuming time now in order to pay for it in the future.
And think of operations costing money, so whenever I do a deletion, I spend actual time, log n time, but if I had log n dollars in the bank, and I could pull those out of the bank, I can use those dollars to pay for the work, and then the deletion itself becomes free in an amortized sense.
So this is, on the one hand, operation-- and when I do an insertion, I'm going to physically take some coins out of myself.
That will cost something in the sense that the amortized cost of insertion goes up in order to put those coins in the bank, but then I'll be able to use them for deletion.
So this is what insertion is going to do.
I can store credit in the bank, and then separately we allow an operation to take coins out of the bank, and you can pay for time using the credit that's been stored in the bank.
As long as the bank balance remains non-negative at all times, this will be good.
The bank balance is a sort of unused time.
We're paying for it to store things in there.
If we don't use it, well, we just have an upper bound on time.
As long as we go non-negative, then the summation will always be in the right direction.
This inequality will hold.
Let's do an example.
Well, maybe this is a first example.
So when I do an insertion, I can put, let's say, one coin of value log n star into the bank, and so the total cost of that insertion, I pay log n star real cost in order to do the insertion, then I also pay log n star for those coins to put them in the bank.
When I do a deletion, the real cost is log n star, but I'm going to extract out of it log n star coins, and so the total cost is actually free-- the total amortized cost is free-- and the reason that works, the reason the balance is always non-negative, is because for every deletion there was an insertion before it.
So that's maybe a less intuitive way to think about this problem, but you could think about it that way.
More generally-- so what we'd like to say is that we only put log n without the star, the current value of n per insert and a 0 per delete amortized.
So we'd like to say, OK, let me put one coin worth log n for each insertion, and when I delete, I consume the coin.
And, in general, the formula here is that the amortized cost of an operation is the actual cost plus the deposits minus the withdrawals.
OK.
So insertion, we just double the cost, because we pay log n to the real thing, we pay log n to store the coin.
That's the plus deposit part, so insertion remains log n, and then deletion, we pay log n to do the deletion, but then we subtract off the coin of value log n, so that hopefully works out to zero 0.
But, again, we have this issue that coins actually have different amounts, depending on what the current value of n was.
You can actually get this to work if you say, well, there are coins of varying values here, and I think the invariant is if you have a current structure of size n, you will have one coin of size log 1, log 2, log 3, log 4, up to log n. Each coin corresponds to the item that made n that value.
And so when you delete an item at size n, you'll be removing the log nth coin, the coin of value log n. So you can actually get this to work if you're careful.
I guess the invariant is one coin of value log i for i equals 1 to n, and you can check that invariant holds.
When I do a new insertion, I increase n by 1 and I make a new coin of log that value.
When I do a deletion, I'm going to remove that last coin of log n. So this does work out.
So we got rid of the end star.
OK, let's use this same method to analyze table doubling.
We already know why table doubling works, but good to think of it from different perspectives.
And it's particularly fun to think of the coins as being physical objects in the data structure.
I always thought it would fun to put this in a programming language, but I don't think there is a programming language that has coins in it in this sense yet.
Maybe you can fix that.
So let's go back to table doubling.
Let's say when we insert an item into a table, and here I'm just going to do insertions.
We'll worry about deletions in a moment.
Whenever I do an insertion, I'm going to put a coin on that item, and the value of the coin is going to be a constant.
I going to give the constant a name so we can be a little more precise in a moment-- c. So here's kind of the typical-- well, here's an array.
We start with an array of size 1, and we insert a single item here, and we put a coin on it.
Maybe I'll draw the coin in a color, which I've lost.
Here.
So I insert some item x, and I put a coin on that item.
When I do the next insertion, let's say I have to double the table to size 2.
I'm going to use up that coin, so erase it, put a new coin on the item that I just put down.
Call it y.
In general-- so the next time I double, which is immediately, I'm going to go to size 4.
I erase this coin, then I put a coin here.
When I insert item, of course, letter after z is w. Then I put another coin when I have to double again, so here I'm going to use these coins to charge for the doubling, and then in the next round, I'm going to be inserting here, here, and here, and I'll be putting a coin here, here, here, and here.
In general, you start to see the pattern-- so I used up these guys-- that by the time I have to double again, half of the items have coins, the other half don't, because I already used them.
You have to be careful not to use a coin twice, because you only get to use it once.
You can't divide money into double money unless you're doing stocks, I guess.
As soon as I get to a place where the array is completely full when n equals m, the last half of the items will have coins.
I'm going to use them in order to pay for the doubling, so the number of coins here will be n over 2.
So this is why I wanted to make this constant a little explicit, because it has to be bigger than 2 in some sense.
However much work-- let's say it takes a times n work in order to do doubling, then this constant should be something like two times a, because I need to do the work to double, but I only have n over 2 coins to pay for it.
I don't get coins over here.
So when we double, the last n over 2 items have coins, and so the amortized cost of the doubling operation is going to be the real cost, which is sum theta n minus the number of coins I can remove and their value.
So it's going to be minus c times n over 2 and, the point is, this is 0 if we set c large.
It only has to be a constant.
It needs to be bigger than 2 times that constant.
And usually when you're working with coins, you want to make the constants explicit just to make sure there's no circular dependence on constants, make sure there is a valid choice of c that annihilates whatever cost you want to get rid of.
So this is the accounting method view of table doubling.
Any questions so far?
So far so good.
Pretty simple example.
Let's get to more interesting examples.
You also think about the amortized cost of an insert.
It costs constant real time.
Actual cost is constant.
You have to also deposit one coin, which costs constant time so the amortized cost of the insert is still constant.
So that's good.
Still we don't know how to deal with deletions, but let me give you a kind of reverse perspective on the accounting method.
It's, again, equivalent in a certain sense, but in another sense may be more intuitive some of the time for some people.
It's actually not in the textbook, but it's the one I use the most so I figure it's worth teaching.
It's called the charging method.
It's also a little bit more time travel-y, if you will, so if you like time travel, this method is for you, or maybe a more pessimistic view is blaming the past for your mistakes.
So what we're going to do is allow-- there's no bank balance anymore, although it's essentially there.
We're going to allow operations to charge some of their cost retroactively to the past, not the future.
I actually have a data structures paper which proves that while time travel to the past is plausible, time travel to the future is not computationally.
So you're not allowed to time travel to the future, only allowed to go to the past, and say, hey, give me $5.
But you've got to be a little bit conservative in how you do it.
You can't just keep charging the same operation a million times, because then the cost of that operation is going up.
At the end of the day, every operation had to have paid for its total charge.
So there's the actual cost, which it starts with, and then there's whatever it's being charged by the future.
So from an analysis perspective, you're thinking about the future.
What could potentially charge me?
Again, you can define the amortized cost of an operation is going to be the actual cost minus the total charge to the past.
So when we charge to the past, we get free dollars in the present, but we have to pay for whatever the future is going to do.
So we have to imagine how many times could I get charged in the future?
I'm going to have to pay for that now in a consistent time line.
You will have to have paid for things that come in the future.
So let's do an example.
Actually it sounds crazy and weird, but I actually find this a lot more intuitive to think about even these very examples.
Let's start with table doubling.
So we have this kind of picture already.
It's going to be pretty much the same.
After I've doubled the table, my array is half full and, again, insertion only, although we'll insertion and deletion in the moment.
In order to get from half full to completely full, I have to do n over 2 insertions.
It's looking very similar, but what I'm going to say is that when I double the array next time, I'm going to charge that doubling to those operations.
In general, you can actually say this quite concisely-- whenever I do a doubling operation, I'm going to charge it to all the insertions since the last doubling.
That's a very clear set of items.
Doublings happen, and then they don't happen for a while, just all those insertions that happened since the last doubling charged to them.
And how many are there?
Well, as we've argued, there are n over 2 of them, and the cost of-- in order to make this doubling free, I need to charge theta n. So this doubling cost theta n, but there's n over things to charge to.
I'm going to uniformly distribute my charge to them, which means I'm charging a constant amount to each.
And the key fact here is that I only charge an insert once.
Because of this since clause, I never will charge an item twice as long as I'm only inserting for now.
If you look over all time, you will only charge an insert once.
That's good, because the inserts have to pay for their total charge in the future.
There's only one charge, and it's only a constant amount, then amortized cost of insert is still constant, amortized cost of doubling is 0, because we charged the entire cost to the past.
So same example, but slightly different perspective.
Let's do a more interesting example-- inserts and deletes in a table.
Let' say I want to maintain that the size of the table is always within a constant factor of the number of items currently in the table.
If I just want an upper bound, then I only need to double, but if I want also a lower bound-- if I don't want the table to be too empty, then I need to add table halving.
So what I'm going to do is when the table is 100% full, I double its size, when the table is 50% full, should I halve it in size?
Would that work?
No, because--
[INAUDIBLE] have to have it inserted in place of linear [INAUDIBLE]
Right.
I can basically do insert, delete, insert, delete, insert, delete, and every single operation costs linear time, because maybe I'm a little bit less than half full-- sorry, yeah, if I'm a little bit less than half full, then I'm going to shrink the array into half.
Get rid of this part, then if I immediately insert, it becomes 100% full again.
I have to double in size, and then if I delete, it becomes less than half full, and I have to halve in size.
Every operation would cost linear time, so amortized cost is linear time.
That's not good.
So what I'll do is just separate those constants a little bit.
When I'm 100% full, I will double.
That seems pretty clear, but let's say when I'm a quarter full, then I will halve.
Any value less than 50 would work here, but-- just halve, like that.
This will actually work.
This will be constant amortized per operation, but it's-- especially the initial analysis we did of table doubling isn't going to work here, because it's complicated.
The thing's going to shrink and grow over time.
Just summing that is not easy.
It depends on the sequence of operations, but with charging and also with coins, we could do it in a pretty clean way.
I'm going to do it with charging.
So this particular choice of constants is nice, because when I double a full array, it's half full, and also when I have an array that's a quarter full, like this, and then I divide it-- and then I shrink it-- I get rid of this part, it's also half full.
So whenever I do a double or a halve, the new array is half full, 50%.
That's nice.
That's nice, because 50% is far away from both 25% and 100%.
So our nice state is right after a doubling or a halve, then we know that our structure is 50%.
In order to get to an under-flowing state where we have to halve, I have to delete at least a quarter of the items, a quarter of m. In order to get to overflowing where I have to double, I have to insert at least m over 2 items.
Either way, a constant fraction times m, that's what I'm going to charge to.
Now, to be clear, when I'm 50% full, I might insert, delete, insert, delete, many different inserts and deletes.
At some point, one of these two things is going to happen though.
In order to get here, I have to do at least m over 4 deletions.
I might also do more insertions and deletions, but I have to do at least that many, and those are the ones I'm going to charge to.
So I'm going to charge a halving operation to the at least m over 4 deletions since the last resize of either type, doubling or halving.
And I'm going to charge the doubling to the at least m over 2 insertions since the last resize.
OK, and that's it.
Because the halving costs theta m time, doubling costs theta m time, I have theta m operations to charge to, so I'm only charging constant for each of the operations.
And because of this since last resize clause, it's clear that you're never charging an operation more than once, because you can divide time by when the resizes happen, grows or shrinks, halves or doubles.
And each resize is only charging to the past a window of time.
So it's like you have epics of time, you separate them, you only charge within your epic.
OK, so that's cool.
So you only get a constant number of charges per item of a constant amount, therefore insertions and deletions are constant amortized.
Halving and doubling is free amortized.
Clear?
This is where amortization starts to get interesting.
You can also think of this example in terms of coins, but with putting coins on the items, but then you have to think about the invariance of where the coins are, which I find to be more work.
We actually had to do it up here.
I was claiming the last half of the items had coins.
You have to prove that really.
With this method, you don't.
I mean, what you have to prove is that there are enough things to charge to.
We had to prove here that there were n over 2 items to charge to.
Kind of the same thing, but it was very clear that you weren't charging to the same thing more than once.
You were never trying to use a coin that wasn't there because of the since clause.
To each their own.
I think either way would work.
I think I will skip this example, but I'll just mention it.
So for 2-3 trees, we said deletions were free, and we did that with the coin invariant, that there was one coin of size log i for each i.
You could instead say, when I delete an item, I'm going to charge it to the insert that made n this current value, because that insert paid log n the actual cost, so it can afford to pay another log n to pay for the deletion of some other item, the one we're currently deleting.
And that works, that you don't double charge to an insert, because you're decreasing n right now.
So for n to get up to that value again, you would have had to do another insert.
So same thing, slightly different perspective.
Let's go to something even more interesting and in some sense more powerful, the last method on the list, which is potential method.
This is a good exercise in how many ways can you skin a cat?
So potential method, I like to call it defining karma in a formal way, is more like the counting strategy.
We're going to think about there being a bank account with some balance, but we're going to define that balance as a function of the data structure state.
So that's called the potential function, but you can think of it as a bank balance.
You can think of it as kinetic potential, I guess.
Potential energy.
Just like the bank account, we want this function to always be non-negative.
We'll also make it an integer.
That would be convenient.
The potential function is basically trying to measure how bad is the data structure right now?
It's, again, like saving up for a rainy day.
We want that whenever we have to do an expensive operation, like a double or halve, that this potential has grown large enough that we can charge that cost to a decrease in the potential.
So it's like this is storing up energy, and whenever we have some free time, we'll give some of that time to the potential function.
It's just like the accounting method, in a certain sense, but we're defining things differently.
Over here, we explicitly said, hey look, I'm going to store some credit right now.
So we were basically specifying the delta, and here we're saying I'm going to consume some credit right now.
Over here, we're going to define this magical function of the current state.
From that you can compute the deltas, but also from here you can integrate and compute the potential function.
So they're interchangeable, but usually it's easier to think about one perspective or the other.
Really often, you can just look at what's going on with the data structure and say, hey, you know, this aspect of the data structure makes it bad, makes costly operations, and you can just define the potential function, then just check that it works.
But it's a little bit of black magic to come up with these functions, so you depends how you like to think about things.
So, as before, we can define an amortized cost.
It's going to be the actual cost plus the change in the potential.
So change of potential is just the potential after the operation minus the potential before the operation.
I highlight that, and it's kind of obvious from the way we set things up, but what I care about is the sum of the amortized costs.
I care about that, because it's supposed to be an upper bound on the sum of the actual costs.
And if you just look at what that sum is, on the right-hand side I have amortized cost plus the fee after the operation minus the fee before the operation.
If I add all those up, this part telescopes or you get cancellation from each term with the previous term.
The sum of the amortized costs is equal to the sum of the actual costs plus phi at the end minus phi at the beginning.
So a slight catch with the potential method.
When you define things this way, you also have to pay for phi at the beginning, because we want the actual cost to be, at most, amortized cost.
So we need to take this apart and put it over here so it's, at most, some of amortized cost plus phi of the beginning.
This part becomes negative, so we usually just ignore it.
It can only help us.
So when you define a potential function, you'd really like it to be 0 at the beginning.
It's funny, but you pay phi of the beginning state at the beginning of time, and when you've done 0 operations, you really like the cost to be 0, and you don't want to have to have stored stuff in the bank, so this should be a-- constant would probably be OK, or whatever the cost of your first operation is but should be constant or 0.
Usually we do this by saying, look, let's start with an empty structure and work from there.
Usually phi of an empty structure is 0, and all is well.
So when you're defining things with potential function, you have to be careful about your initial state.
You have to make sure it's non-negative just like you did over here, but you didn't have to worry about this part over there.
All this infrastructure, what's it good for?
Let's do some examples.
These are going to be the most interesting examples.
A kind of classic example of amortization is incrementing a binary counter.
So when you have some binary value like this one and you increment it, many bits change, but only a constant number are going to change in an amortized sense.
If I start with a 0 vector, 0 bit vector, and I increment-- well, the very first increment costs 1, the next increment costs 2, the next increment costs 1, next increment costs 3, then 1, then 2, then 1, then 4, then it's a fractal.
But instead of thinking about that fractal and working hard to prove that summation is linear for an operation, let's use the potential method.
And the intuition here is actually pretty easy, because an increment has a very clear cost.
It's just the number of trailing 1s plus 1.
That's what it is in actual cost.
We'd like that to be constant so, intuitively, what is making an increment bad?
If you had to name one thing?
If I just look at a configuration, is this bad?
Is this bad?
How bad is the configuration?
Yeah
The more trailing ones you have, the worse the state is?
The more trailing ones, the worse the state is.
So that's one natural definition.
Turns out, it won't work.
Let's see why.
I think here's an example.
So near the end of our increment stage, we have a whole bunch of 1s but no trailing 1s, number of trailing 1s is 0.
If I do a single increment, now the number of trailing 1s is n, so if you look at the amortized cost, it's the actual cost plus the change in phi and so I actually pay n for that operation in the amortized sense, and that's no good.
We only want to pay constant, but it's on the right track.
So number of trailing 1, it is the natural thing to try, but it doesn't quite work for our definition of phi.
Other ideas?
Yeah
The total number of [INAUDIBLE]
The total number of 1s.
Yeah.
Let's define phi, could be the number of 1 bits.
That will work, but you both get a Frisbee
Oh, [INAUDIBLE]
Sorry.
Good thing I missed.
Number 1 bits.
Intuitively, 1s are bad, and this is a good definition, because when I increment I only create one 1, so I'm not going to have this issue that delta phi goes up by a lot-- sorry, that phi goes up by a lot, that delta phi is really large.
Because even in this scenario, if I increment, I only add one 1.
In this scenario, I destroy three 1s and add one.
In general, if there are, let's say, t trailing bits, then an increment destroys t 1 bits, and it creates one 1 bit.
That's always what happens.
T could be 0, and then I have a net positive of 1, but most of the time actually I destroy 1 bits-- well, more than half the time I destroy 1 bits, and I just create a single 1 bit, in terms of the total number of 1s.
So the amortized cost is the actual cost, which is this 1 plus t. I'm actually going to remove the-- well, yeah.
I'd like to remove the big O if I could.
I'm going to count-- I want to be a little bit precise about my counting, because I have to do a minus sign here.
If I just wrote minus t, that doesn't quite work out, because there's a constant here that I have to annihilate.
If I count the number of bits that change, then that's exactly 1 plus t in an increment.
And now the change of potential is that I decrease by t, I increase by 1, I get 0.
That seems a little bit too small, 0 time per operation
You're adding a 1, you're not subtracting [INAUDIBLE].
Sorry, you're not subtracting [INAUDIBLE].
Just subtracting something else
Oh, right, sorry.
That's 2.
Thank you.
I just can't do the arithmetic.
I wrote everything correct, but this is a plus 1 and a plus 1.
T minus t is the key part that cancels.
Now, if you were measuring running time instead of the number of changed bits, you'd have to have a big O here, and in that case you'd have to define phi to be some constant times the number of 1 bits.
So you could still set that constant large enough so that this part, which is multiplied by c, would annihilate this part, which would have a big O. I guess I'll write it in just for kicks so you've seen both versions.
This would be minus c see times t plus 1 times c. So that would still work out.
If you set c to the right value, you will still get 2.
So binary counters, constant amortize operation.
So I think this is very clean, much easier than analyzing the fractal of the costs.
Now, binary counter with increment and decrements, that doesn't work.
There are other data structures to do it, but that's for another class.
Let's go back to 2-3 trees, because I have more interesting things to say about them.
Any questions about binary counters?
As you saw, it wasn't totally easy to define a potential function, but we're going to see-- if see enough examples, you get some intuition for them, but it is probably the hardest method to use but also kind of the most powerful.
I would say all hard amortizations use a potential function.
That's just life.
Finding them is tough.
That's reality.
I want to analyze insertions only in 2-3 trees, then we'll do insertions and deletions, and I want to count how many splits in a 2-3 tree when I do an insertion.
So remember, when you insert into a 2-3 tree, so you started a leaf, you insert a key there.
If it's too big, you split that node into two parts, which causes an insert of a key into the parent.
Then that might be too big, and you split, and so on.
So total number of splits per insert?
Upper bounds?
Log n
Log n. OK. Definitely log n in the worst case.
That's sort of the actual cost but, as you may be guessing, I claim the amortized number of splits is only constant, and first will prove this with insertion only.
With insertion and deletion in a 2-3 tree, it's actually not true, but for insertion only this is true.
So let's prove it.
A 2-3 tree, we have two types of nodes, 2 nodes and 3 nodes.
I'm counting the number of children, not the number of keys, is one smaller than the number of children.
Sorry, no vertical line there.
This is just sum key x, sum key and y.
So when I insert a key into a node, it momentarily becomes a 4 node, you might say, with has three keys, x, y, and z.
So 4 node, it has four children, hence the 4, and we split it into x and z.
There's the four children, same number, but now they're distributed between x and z.
And then y gets promoted to the next level up, which allows us to have two pointers to x and z.
And that's how 2-3 trees work.
That's how split works.
Now, I want to say that splitting-- I want to charge the splitting to something, intuitively.
Let's say y was the key that was inserted, so we started with x z, which was a 3 node.
When we did an insert, it became a 4 node, and then we did a split, which left us with two 2 nodes and something.
So what can you say overall about this process?
What's making this example bad?
What's making the split happen, in some sense?
I mean, the insert is one thing, but there's another thing we can charge to.
Insert's not enough, because we're going to do log n splits, and we can only charge to the insert once if we want constant amortized bound.
Yeah?
Number of 3 nodes?
Number of 3 nodes, exactly.
That's a good potential function, because on the left side of this picture, we had one 3 node.
On the right side of the picture, we had two 2 nodes.
Now, what's happening to the parent?
We'll have to worry about that in a moment, but you've got the intuition.
Number of 3 nodes.
I looked at just a single operation here, but if you look more generally about an expensive insert, in that it does many splits, the only way that can happen is if you had a chain of 3 nodes all connected to each other and you do an insert down here.
This one splits, then this one splits, then this one splits.
So there are all these 3 nodes just hanging around, and after you do the split, the parent of the very last node that splits, that might become a 3 node.
So that will be up here somewhere.
You might have made one new 3 node, but then this one is a couple of 2 nodes, this becomes a couple of 2 nodes, and this becomes a couple of 2 nodes.
So if you had k 3 nodes before, afterwards you have one.
Sound familiar?
This is actually exactly what's going on with the binary counter, so this may seem like a toy example, but over here we created, at most, one 1 bit.
Down here we create, at most, one 3 node, which is when the split stops.
When the split stops, that's the only time we actually insert a key into a node and it doesn't split, because otherwise you split.
When you split, you're always making two nodes, and that's good.
At the very end when you stop splitting, you might have made one 3 node.
So in an insert, let's say the number of splits equals k, then the change of potential for that operation is minus k plus 1, because for every split there was a 3 node to charge to-- or for every split there was a 3 node that became two nodes, two 2 nodes.
So the potential went down by one, because you used to have one 3 node, then you had 0.
At the very end, you might create one 3 node.
That's the plus 1.
So the amortized cost is just the sum of these two things, and we get 1.
That's k minus k plus 1 which is 1.
Cool, huh?
This is where a potential method becomes powerful, I would say.
You can view this as a kind of charging argument, but it gets very confusing.
Maybe with coins is the most plausible use.
Essentially, the invariance you'd want is that you have a coin on every 3 node.
Same thing, of course, but it's I think easier to think about it this way.
Say, well, 3 nodes seem to be the bad thing.
Let's just count them, let's just see what happens.
It's more like you say I want to have this invariant that there's a coin on every 3 node.
How can I achieve that?
And it just works magically, because A, it helps it was true and, B, we had to come up with the right potential function.
And those are tricky and, in general with amortization, unless you're told on a p set prove order t amortize, you don't always know what the right running time is, and you just have to experiment.
Our final example, most impressive.
Let's go over here.
It's a surprise, I guess.
It's not even on the list.
I want to do-- this is great for inserts, but what about deletes?
I want to do inserts and deletes.
I'd like to do 2-3 trees, but 2-3 trees don't work.
If I want to get a constant amortized bound for inserts and deletes, I've got to constant advertised here for inserts-- I should be clear.
I'm ignoring the cost of searching.
Let's just say searching is cheap for some reason.
Maybe you already know where your key is, and you just want to insert there.
Then insert only costs constant amortize in a 2-3 tree.
Insert and delete is not that good.
It can be log n for every operation if I do inserts and deletes, essentially for the same reason that a binary counter can be n for every operation if I do increments and decrements.
I could be here, increment a couple times, and then I change a huge number of bits.
If I immediately decrement, then all the bits go back.
In increment, all the bits go back.
Decrement, all the bits go back.
So I'm changing end bits in every operation.
In the same way, if you just think of one path of your tree, and you think of the 0 bits as 2 nodes and the 1 bits as 3 nodes, when I increment by inserting at the bottom, all those 3s turn to 1, except the top I make a 3.
That's just like a binary counter.
It went from all 1s to 1 0 0 0 0 0, and then if I decrement, if I delete from that very same leaf, then I'm going to have to do merges all the way back up and turn those all back into 3 nodes again.
And so every operation is going to pay log n. Log n's, not so bad, but I really want constant.
So I'm going to introduce something new called 2-5 trees, and it's going to be exactly like b trees that you learned, except now the number of children of every node should be between 2 and 5.
All the operations are defined the same.
We've already talked about insert.
Now insert-- when you have six children, then you're overflowing, and then you're going to split in half and so on.
So actually I should draw that picture, because we're going to need it.
So if I started with a 5 node, which means it has four keys, and then I insert into it, I get a 6 node.
That's too many.
Six children.
OK, that's too much, so I'm going to split it in half, which is going to leave a 3 node and a single item, which gets promoted to the parent, and another 3 node.
OK, so we started with a 5 node, and the result was two 3 nodes.
OK, that split, and we also contaminate the parent a little bit, but that may lead to another split, which will look like this again.
So if we're just doing insertions, fine, we just count the number of 5 nodes, no different, right?
But I want to do simultaneously insert and delete.
So let's remember what happens with a delete.
So if you just delete a key and a leaf, the issue is it may become too empty.
So what's too empty?
Well, the minimum number of children we're allowed to have is two, so too empty would be that I have one child.
So maybe initially I have two children, and I have a single key x, then maybe I delete x, and so now I have 0 keys.
This is a 1 node.
It has a single child.
OK. Weird.
In that case, there are sort of two situations.
Maybe your sibling has enough keys that you can just steal one, then that was really cheap.
But the other case is that you-- yeah.
I'm also going to have to involve my parent, so maybe I'm going to take a key from x and merge all these things together.
So that's y, then what I get is an x y. I had two children here, three children here.
OK. Also messed up my parent a little bit, but that's going to be the recursive case.
This is a sort of merge operation.
In general, I merge with my sibling and then potentially split again, or you can think of it as stealing from your sibling, as you may be experienced with doing.
I don't have siblings, so I didn't get to do that, but I stole from my parents, so whatever.
However you want to think about it, that is merging in a b tree.
We started with a 2 node here.
We ended up with a 3 node.
Hmm, that's good.
It's different at least.
So the bad case here is a 5 node, bad case here is a 2 node.
What should I use a potential function?
Yeah
Number of nodes with two children and number of nodes with five children?
Number of nodes with two or five children, yeah.
So that's it.
Just combine with the sum.
That's going to be the number of nodes with two children plus the number of nodes with five children.
This is measuring karma.
This is how bad is my tree going to be, because if I have 2 nodes, I'm really close to under flowing and that's potentially bad.
If I happen to delete there, bad things are going to happen.
If I have a bunch of 5 nodes, splits could happen there, and I don't know whether it's going to be an insert or delete next, so I'm just going to keep track of both of them.
And luckily neither of them output 5s or 2s.
If they did, like if we did 2-3 trees, this is a total nightmare, because you can't count the number of 2 nodes plus the number 3 nodes.
That's all the nodes.
Potential only changes by 1 in each step.
That would never help you.
OK?
But here we have enough of a gap between the lower bound and the upper bound and, in general, any constants here will work.
These are usually called a-b trees, generalization of b trees, where you get to specify the lower bound and the upper bound, as long as a-- what's the way-- as long as a is strictly less than b over 2, then this argument will work.
As long as there's at least one gap between a and b over 2, then this argument will work, because in the small case, you start with the minimum number of children you can have.
You'll get one more in the end, and in the other situation, you have too many things, you divide by 2, and you don't want dividing by 2 to end up with the bad case over here.
That's what happened even with 2-4 trees-- 2-3-4 trees-- but with 2-5 trees, there's enough of a gap that when we split 5 in half, we get 3s only, no 2s, and when we merge 2s, we get 3s, no 5s.
So in either case, if we do the split-- if we do an insert with k splits, the change in potential is minus k plus 1.
Again, we might make a single five-child node at the top when we stop splitting, but every time we split, we've taken a 5 node and destroyed it, left it with two 3 nodes, so that decreases by k, and so this k cost gets cancelled out by this negative k and change potential, so the amortized cost is 1 just like before.
But now, also with delete, with k merge operations where I'm treating all of this as one operation, again, the change of potential is minus k plus 1.
Potentially when we stop merging, because we stole one key from our parent, it may now be a 2 node, whereas before it wasn't.
If it was already a 2 node, then it would be another merge, and that's actually a good case for us, but when the merges stop, they stop because we hit a node that's at least a 3 node, then we delete a key from it, so potentially it's a 2 node.
So potentially the potential goes up by 1.
We make one new bad node, but every time we do a merge, we destroy bad nodes, because we started with a 2 node, we turned it into a 3 node.
So, again, the amortized cost is the actual cost, which is k, plus the change in potential, which is minus k plus 1, and so the amortized cost is just 1.
Constant number of splits or merges per insert or delete.
So this is actually really nice if you're in a model where changing your data structure is more expensive than searching your data structure.
For example, you have a lot of threads in parallel accessing your thing.
You're on a multi-core machine or something.
You have a shared data structure, you really don't want to be changing things very often, because you have to take a lock and then that slows down all the other threads.
If searches are really fast but splits and merges are expensive, then this is a reason why you should use 2-5 trees instead of 2-3 trees, because 2-3 trees, they'll be splitting emerging all the time, log n. It's not a huge difference, log n versus constant, but with data structures that's usually the gap.
Last class we were super excited, because we went from log to log log.
Here we're excited we go from log to constant.
It's a little better, but they're all small numbers, but still we like to go fast, as fast as possible.
In a real system, actually it's even more important, because splitting the root is probably the worst, because everyone is always touching the root.
In a 2-5 tree, you almost never touch the root, almost always splitting and merging at the leaves, whereas in a 2-3 tree, you could be going all the way to the root every single time.
So that's my examples.
Any questions?
[INAUDIBLE]
For free minutes.
Cool.
That's amortization.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
Let's get started.
A new module today-- we're going to spend a few lectures on randomized algorithms.
And so not only will we look at slightly different ways of solving old problems like sorting, we'll also look at how we can analyze this new kind of algorithm that generates random numbers in order to actually make decisions as it's executing and that we'll end up obviously with the analysis that gives us the expected run time of the algorithm-- for example, whether the algorithm is going to produce a correct result or not, with what probability will this algorithm produce a correct result.
So I'll talk a little bit about why we're interested in randomized algorithms in a couple of minutes, but let me define what a randomized algorithm, or a probabilistic algorithm, is to start things off.
And so randomized algorithm is something that generates a random number.
Now, this would be a coinage flip, but more often than not, you're generating a real number that comes from a sudden range.
Sometimes you're generating a vector.
You'll see a couple of different examples here in today's lecture and in section.
And it's going to make decisions based on this value, based on r's actual value.
Now, you can imagine that an algorithm would be recursive, and at every level of recursion, it's going to generate a random r. So when you're executing at a particular level of recursion, you may be doing different things based on r. And not only that, if you re-run the algorithm again on the same input, the execution will be different because you're resuming a true random number generator as opposed to a pseudo random one.
And the r's that you're going to get at different levels of recursion or through the execution of the algorithm are going to be different from the first time to the second time.
So on the same input on different executions, two things might happen.
The algorithm may run for a different number of steps.
So you might get lucky on the first execution, and the algorithm finishes, let's say at 100 time units.
The second time around, it takes a long time.
It takes 700 time units.
Our goal here is to try and analyze what this probabilistic runtime would be to ask for an expectation, to be able to compute an expectation for the runtime, or-- if you're talking about a different scenario where different executions-- I could actually produce different outputs.
And in this case, it's possible that one or more of these outputs are incorrect.
You actually get the wrong answer.
And obviously, that's going to happen with a certain probability.
You're going to have to decide or analyze what that probability is.
And generally speaking, we won't be happy with a high probability of error, as you can imagine.
And we'd like to set up an algorithm such that you can reduce that probability of incorrect output to be something really, really small.
And it might take you longer to get to that low level of incorrect output in one case for a certain set of inputs versus another case.
So that's this set up here in terms of randomized.
You're going to have algorithms that-- you can think of them as probably correct.
So these are algorithms-- you want to think of them as probably correct, and they do have a name.
They're called Monte Carlo algorithms.
And then you have algorithms that are probably fast.
So-- indicates a probably correct-- you could have a constant probability that they're going to give you the correct answer, 99%.
And you could obviously try and parametrize that.
In the case of probably fast, you say things like, it runs an expected polynomial time.
And really what that means is that you may have to run it for more information.
So rather than taking 100 iterations or 100 steps to sort something, it might take you 110.
But in the case of probably fast, you do get the sorted result at the end.
And when the algorithm has finished execution, you do get that sorted result at the end.
So it's correct and probably fast or probably correct and deterministically fast.
OK. And this is Las Vegas.
So you have Monte Carlo versus Las Vegas here.
So yesterday, it occurred to me-- and I've taught this class a bunch of times-- but it occurred to me for the first time last night that there should be algorithms that are probably correct and probably fast, which means that they're incorrect and slow some of the time.
Right?
So what do you think those algorithms are called?
Sorry.
What?
T?
The T?
Oh.
Oh!
That deserves a Frisbee.
Oh my goodness!
[LAUGHS] All right.
There you go.
There you go.
All right.
Now, they're not called the T. So we should write that down so everyone knows.
Probably correct and probably fast, which is I guess they don't get you anywhere.
I don't know what that means-- incorrect and so in the case of the T. So the MB/TA.
Any guesses?
I mean, think about what we have for Monte Carlo, Las Vegas.
Extrapolate.
These are the kinds of questions you're going to get on your quiz.
I guess you guys don't gamble you.
Go ahead
Atlantic City
Atlantic City.
That deserves a Frisbee.
Yeah.
Absolutely right.
That It turns out Atlantic City isn't a name that's really caught on, but it was in terms of being used in this context.
Most of the time, if you do have a probably correct probably fast algorithm, you can convert it into a Monte Carlo algorithm or a Las Vegas algorithm.
There are some prime testing algorithms to test whether a particular number is a prime or not that run in probabilistic polynomial time, and they may incorrectly tell you that the number is a prime.
So that's an example of an Atlantic City algorithm.
We won't actually do Atlantic City.
What we'll do is we'll take a look at a couple of different algorithms, and both of these will motivate why randomized algorithms are interesting.
The Monte Carlo example is checking matrix multiply.
So you've gotten a couple of square matrices.
Both of them are n by n matrices, and you multiply them out-- A times B, and you produce C. And so you got the C matrix.
And rather than re-multiplying and checking the result, you'd like to do something better.
You'd like to verify with some probability that you can parametrize that the output matrix is in fact the product of the two input matrices.
And so that's a randomized algorithm that's a Monte Carlo because you're not guaranteeing that that output matrix is in fact the product of the first two matrices or the operand matrices, but you're getting a good sense of how likely that is.
And you can kind of squish that probability of error down to however low you want it to be except you have to run the algorithm for longer.
So that's an example of Monte Carlo.
Now, quicksort.
It doesn't make sense to say-- I guess you could-- but it doesn't make too much sense to say that you have an almost sorted array.
What does that mean exactly?
You have to categorize that.
So quicksort is an example where you're guaranteed to get a sorted array at the end of it.
So it's correct.
You will get a sorted ray.
That's what you wanted-- descending order, ascending order.
But it might not run in order n log n time.
That's expected time.
Order n log n is expected time.
And so that's what probably fast would correspond to.
All right?
So that's the set up.
You can kind of see why these are interesting because you could imagine that in practical scenarios, you might want to do some checking in a probabilistic way.
And you want to do that without having to redo all the work.
Obviously you don't want your checker for matrix multiply to be as slow as multiplying two matrices.
Otherwise it makes no sense.
So let's dive into matrix product and our first example of a probably correct algorithm, or Monte Carlo algorithm.
So what I want to do here is C equals A times B.
And the simple algorithm-- I guess, those of us who went to high school, myself included, did my four years-- know of an n cube algorithm-- or learned it back then.
It simply corresponds to taking rows and columns, and you get an entry.
You have n square entries that you need to compute corresponding to the output matrix C. And you're going to do order n multiplications and additions, but we're really going to consider multiplications here.
When I talk about n here, it's not the total number of operations.
It's the number of multiplications.
And the reason for that is-- this may have gone away a little bit, but it's still probably true-- that multiplication, in computers, it takes longer to multiply two numbers, integers are floating point numbers, than adding numbers.
It used to be much more dramatic, the differences between multiplying and add in computers.
But thanks to pipelining and lots of optimizations, multiplies are actually very fast.
But they are, obviously, a more sophisticated operation than addition.
So we'll be counting multiplies.
So when you've seen Karatsuba divide and conquer for multiply, back end in 006.
Remember that we were counting multiplications, and we were actually trading off multiplications for additions.
We were trying to shrink that number associated with the complexity of the algorithm when counting the number of multiplies.
And we actually counted the number of additions that were going up-- at least from a constant factor standpoint, not necessarily from an asymptotic complexity standpoint.
And so that's simple algorithm.
You probably heard of Strassen.
Some of you might have seen it.
Essentially what happens with Strassen is you multiply two two by two matrices using seven multiplications as opposed to eight.
Now, if you do that-- and this is similar to the Karatsuba analysis-- you can do this in n raised to log 2 7 time, which is essentially n raised to 2.81 time.
And so rather than n cubed, you can go down to n raised 2.81.
Now it turns out people have obviously not stopped with this.
You can go to n raised to 2.70 by doing something of the order of 143,000 multiplications for 70 by 70 matrices.
So you can play around.
Just like you had Toom-Cook.
I don't know if you remember that or it got covered-- but Karatsuba could get generalized into this thing called Toom-Cook.
And the same thing, Strassen-- you could go off and divide and conquer whose base case is not two by two, but 70 by 70, and that improves things.
But it turns out there's other arithmetic series summation ways.
And so a famous algorithm that's up until 2010 was the best complexity algorithm known.
It's Coppersmith-Winograd, which is 2.376.
And then at some point, we had a faculty candidate here who either shrunk this from 2.376 to 2.373.
And it turns out that there were two different researchers who came up with a 2.373, but this particular candidate in the sixth decimal place won.
So she had an eight.
Person had a nine or something.
But anyway, all of these are impractical.
OK. You don't want to use them.
The constant factors associated with these things are much larger than what you have here.
The constant factors here, I guess it's one, right?
Makes sense that it would be one, forgetting the additions of course.
So if you have large constant factors, then you need a billion by billion matrix in order to win.
And if you have billion by billion matrices that you want to multiply, do something else.
OK. You don't want to go there.
Even in the day of the internet, it's not going to work.
So what we'd like to do now is do something better.
So we will try-- given that theoretical computer science class, it makes sense to say that our verification algorithm should be better than n raised to 2.376 or 2.3-whatever.
Right?
Otherwise, it doesn't feel good.
So what we'd like to do-- and we can do this-- is try and get an order n square algorithm-- that's this.
So it's probably correct Monte Carlo algorithm where if you have A times B equals C, then the probability of the output equals yes is 1.
So in fact, if you got it right, then the verifier is going to not give you a false negative.
It's not going to say-- no, you got it wrong-- when you got it right.
But it could give you up a false positive with some probability where you have the probability of output equals yes, and that's a false positive.
But you can bound that to be less than half.
OK.
So it's going to say, yes.
So obviously, if the verifier kept saying yes all the time, you wouldn't have this.
It wouldn't be very interesting.
I would be constant time, but it wouldn't be very interesting.
What is interesting here is that when they're not equal, you're going to get an incorrect result with some operand on the probability.
So you say, about one half seems kind of high-- 50% flipping a coin.
The good news is that these algorithms, you can run them over and over.
You can run this checker over and over.
And as long as executions are independent, and you can certainly ensure that they're independent by ensuring that the randomness from one execution to another-- the flipping of the coins-- are independent.
OK. And so that's relatively easy to do, in certainly all of the scenarios we'll be looking at.
In 046, it's relatively easy to do.
You can now drive this probability down to one quarter with two executions because you'll just check different things.
And then one eighth with three and so on and so forth.
So that's what's cool about it.
And now, if you can look at the runtime, you say, well, runtime still order n square.
That's the beauty of this because I'm just putting an extra constant factor here where I have k n square, where k is a constant.
And effectively, I have this nice relationship in terms of the probability of error going down to 1 divided by 2 raised to k. And what I have here is a k n square.
So that's what's cool about it.
And obviously, k n square is 2 order n square in a polynomial time, and but the probably correct aspect of it gets better and better.
OK. Any questions so far?
All right.
Good.
So what we're going to do is this algorithm actually works for arbitrary matrices-- the structure at least.
We're going to assume that the matrix entries are Boolean.
They're going to work in the finite field mod 2.
And it's just is an easier proof.
It's easier to see.
So the complexities are all the same.
You're still multiplying numbers.
They happen to be small.
And multiplication cost you one operation.
And you need to do n cubed multiplies to actually get the C matrix, and you have to verify it order n square time.
And so the number of multiplies, again, that you want to use in your verification algorithm has to be order n square.
We're ignoring the additions.
So that's what we'd like out of our matrix product checker, and the algorithm we're going to look at is called Freivald's algorithm, cute little algorithm, that does the following.
So the algorithm itself is very straightforward, in couple of lines, a minute or so to describe, and the interesting aspect of it is the analysis-- the fact that you can show this.
That's the cool part.
If you couldn't show that, there's nothing cool about this algorithm.
So we're going to choose a random binary vector.
So there you go.
Here's your randomness.
And this binary vector, every time you run it, as the k increases here, the random binary vector is different from one to another.
That's important.
You can't run the same thing again and then expect a different result.
That's called insanity.
But you are going to assume that given that we are working in the binary space and this is a binary vector, you're going to assume that r i equals 1 is half independently for i equals 1 through n. And the algorithm essentially is this-- we're going to do a bunch of matrix vector multiplies.
An n by n matrix multiplied by an n by n matrix gives you an n by n matrix, and that's your n cubed.
So these are all-- I think I said this, but I should've written this down-- these are all square matrices that are n by n. And that's where you get your n cube.
A matrix vector would be something where you have-- typically we'd have a column vector here.
You're going to get something like that, and you have n square multiplications here.
You're going to grab one of these and then multiply it by that and get an entry here.
And that obviously is n multiplications, but you only have n elements to produce here in this vector.
So you only got n square.
That make sense?
And so what we're going to do is, we're going to take this r, and we're going to compute A times B r. And so the brackets are important because it says that you're going to compute what's inside the brackets first.
Otherwise, it would be a problem because you'd be multiplying A times B.
And obviously, that's order n cube.
Right?
You don't want that.
So A times Br equals Cr.
OK.
So r, remember, is a column vector.
And C is an n by n matrix as our A and B.
We're going to output yes.
Else-- if these two are not equal, you're going to output no.
OK?
And so that's it.
That's one run of the algorithm, generator random r and do the multiplication as you see here.
So let's be clear about complexity, and let's make sure we understand the simpler aspects of the algorithm before we get into the analysis associated with bounding the false positive probability.
The hard part is going to be bounding the false positive probability.
But the easy part is first, the complexity.
So how many matrix vector products am I doing here?
How many matrix vector products am I doing here in this check on one iteration of algorithm?
Yeah
Three
Three.
All right.
All right.
You need to stand up.
This is fun.
This is the hardest throw I've had to make in 6046.
I got to put this down.
Warm up a little bit.
It's kind of cold.
Whoa.
Terrible!
All right.
The person who gets up and gets that owns it, and we're going to do this again.
All right.
Let's see how long this takes
Is this part of my trial?
Yes.
Well, the first one failed.
False whatever, right?
[LAUGHTER] I got a few more.
[LAUGHS] All right.
Let me see.
I think I need to go here.
This is good.
And I need to be-- all right.
Number three.
Thank you.
Thank you.
[CLAPPING] So it was three.
Three.
Perfect.
Three matrix vector products because I got to do this.
That's the matrix vector product.
Remember I'm getting a column vector out of this, which is important, and then I'm going to multiply this matrix with a column vector, matrix vector product number two.
And then there's a matrix vector product over here.
So then at that point-- do you remember I have a vector and a vector.
And checking the equivalence of two vectors is simply checking the equivalence of each element in the vector 1 by one.
So first one, same as the first one.
Second one, same as the second one.
Et cetera.
And so this is order n square, but three is something that is worth thinking about simply because every once in a while, we're interested in constant factors.
And the other thing that's interesting about this-- make sure I write this-- let me write this over here-- that you are going to-- if A B equals C, then there's no issue associated with error here.
So there's no notion of a false negative because if AB equals C, then you know, thanks to the associativity of matrix multiplication-- be it whether they're n by n matrices or columns-- you have this relationship here.
And I hope you can read it at the back.
Essentially what I have here is if AB equals C. So if in fact, the matrix multiply happened correctly, I'm in a situation where it is clear that A Br equals this, thanks to this associativity of matrix multiply.
And that of course, is exactly the same as Cr.
So that should convince you, thanks to associativity of matrix multiply that you don't have any false negatives in this algorithm.
Make sense?
So we're all good.
All we have to do, given what we have with respect to Frievald is to do this part here, which is going to take a little bit of doing.
And the challenge always with simple algorithm is you don't quite know why they work.
And then of course, you have sophisticated algorithms, and you don't quite know why they work.
So this will take a few minutes.
It's not super complicated, and there's a little insight, as always, with these things that are not immediately obvious.
But we'll have to look at the number of r's.
So you have an r vector that you've generated randomly, and it may be a bad vector.
It may be a vector that doesn't show you that the product matrix has an incorrect entry.
Remember there's n square entries in this matrix.
Exactly one of them may be wrong, and you need to find it.
Right?
So there may be a lot of entries which are all correct, but you've got to find that one entry that's incorrect.
And so you could miss it.
A given r vector might miss it, and of course, if you keep generating the r's, you'd like to find it and declare that the matrices weren't multiplied correctly and that probability is what we have to compute.
So we want to get this result where we are analyzing the correctness in the case.
You've already analyzed the correctness in the case where AB equals C, but now we have to analyze the correctness in the case where AB is not equal to C. Right?
And so the claim is that if AB is not equal to C, then the probability of ABr not equal to Cr is greater than or equal to half.
So this is greater than or equal to.
Over there, I'm just talking about the false negative probability where I'm actually getting an incorrect yes when you have the matrices being multiplied wrongly, incorrectly.
And so that's why I get-- this is what I want.
I want there to be a greater than one half probability for r to have discovered that, for r to have discovered that.
OK?
I'll stop for questions in a second, but let me do a little bit more.
I'm going to compute the difference matrix, and I'm not computing this because obviously this would take a while to compute.
It's just for the purpose of analysis.
I'm going to look at the difference matrix D equals AB minus C because you want D to be 0.
And that we're going to do some analysis that says-- we are going to try and find these non-zero entries in D because, clearly, the non-zero entries in D tell you if there's non-zero entry in D, you got a problem here.
The matrices weren't multiplied properly.
So that's why we have D here.
Don't think of it as we're actually computing that.
So what we'd like is to, as I said, discover these entries where our hypothesis now is that D is not equal to 0 because that's the case we're considering.
We know that D is not equal to 0 if the matrices were multiplied incorrectly.
And when I say D is not equal to 0, it means that there are n square entries in D, and one of them is not 0.
They all have to be identically 0.
That's all it means.
D not equal to 0 means one entry at least is not 0.
So now what we need to do is we need to show that there are many r-- it's a binary vector of length n, and you can obviously think about two ways to n possibilities with respect to r. And what we really want to show is that there's a large fraction-- more than half of the r's are going to actually discover that the matrices were multiplied incorrectly.
OK.
So we want to show that there are many r's such that Dr is not equal to 0.
And because if Dr is not equal to 0, then you're obviously going to discover that ABr is not equal to Cr.
So if ABr is not equal to Cr, that's identical to saying the Dr is not equal to 0.
That make sense?
So specifically, if you look at the claim and writing it in terms of Dr, you want to say that the probability of Dr not equal to 0 is greater than or equal to half for a randomly chosen r. And so that's it.
That's the setup that we have to show.
We have to do a counting argument corresponding to these r vectors that are being generated randomly.
So let's do that.
So the general argument we're going to make here is simply that we're going to-- roughly speaking-- if you're going to look at a bad r-- what's a bad r?
And bad r is something that doesn't discover the incorrect multiplication.
That's what a bad r is.
So you're D is not equal to 0, but Dr equals 0.
OK. That's a bad r, right?
It's quite possible that that would be the case.
And so you want to try and figure out how many of these bad r are there because those are the ones that are causing the false negatives.
Right?
So that counting argument is the crux of the proof of the claim.
So let's look at that.
And what we're going to do is, we're going to pick a bad r, and we're going to say I there are these good r's that are associated with this bad r. And for every bad r, there's a good r. And a good r is something that actually discovers the incorrect multiply.
And given that for every bad r there's a good r, half of the arts are good r. That's it.
So I'll write it down.
That's the essence of the argument.
And I'm go a little more slowly so hopefully you'll get that.
So let's look at the case where Dr equals 0 case because that's the interesting case.
That's the case where the r is bad even though we had an incorrect multiply, and you get this-- I should have said you get a false positive.
So I'm sorry.
I think just before, I said false negative, but I meant false positive.
So you have a false positive in this case, and D equals AB minus C not equal to 0 implies there exists an i and j such that Dij is not equal to 0.
OK?
And there's just one entry at least-- if you say the matrix is not equal to 0, there's got to be an entry that's not equal to 0.
So let's take a look at that entry, and let's just draw it out.
That's my D matrix.
And there's going to be an i and a j.
So that's my ith row and my jth column.
And there you go.
I have an entry here which is Dij, and I'm just picking that.
I don't care what i and j are, but there's got to be an entry that's not equal to 0.
Now I'm going to create a vector, v. So this vector is not r. It's a vector v that is chosen deterministically given the Dij where it's got 0's everywhere except it at vj.
So if this the jth entry column-wise, everywhere else you got 0.
And you just got the one associated with the-- going downward-- the jth entry.
OK?
So it's a one one-hot vector, if you will.
It's got one, one.
So now, if you multiply these two things out, you know that you're going to get something, and we're going to can call this Dv.
So you take D and you multiply it by v-- matrix multiplied by a vector.
You're guaranteed, given that all of these are 0, when I do my this times that plus this times this plus this times this, all of these are going to produce 0.
This times 1 is going to produce something that's non-zero, and then all of the other ones are going to produce 0.
So I'm just adding a bunch of 0's to this non-zero multiplied by one.
So I'm going to get something that's non-zero.
Right?
All make sense?
So I'm going to see something here, which is the jth entry that's not equal 0.
And so that implies that Dv is not equal to 0.
And in particular, what I'm saying is Dv of j-- so if I just look at that entry that is identically Dij, which is not equal to 0.
Because I'm multiplying it by 1 and I'm adding a bunch of 0's to it.
That's it.
OK. Yeah.
A question
Is it Dv of j or of i?
So I picked the j here.
So I think I'm going j, j, right?
That make sense?
If j was 7 and this is 7 down, then it would be the seventh [INAUDIBLE] because this is going to turn into that.
OK?
Now, either way, if I picked it in the middle, it doesn't really matter.
The point is there's going to be one entry.
So hang in there.
There's going to be one entry that's nonzero if you didn't quite get that.
So Dij is not 0.
And this is one more observation we're going to make in order to do the counting of these bad r's because this is a bad r that we're looking at if you say that Dr equals 0.
You've created a v that has nothing to do with r, but we're going to use the v to go from a bad r, which is our example here, to a good one.
That's pretty much it.
That's the last step here.
So what we're going to do, is we're going to take any r that can be chosen by our algorithm such that Dr equals 0 because that's the case that we're looking at.
And we're going to compute r prime, which is r plus v. And just remember this is mod 2 arithmetic.
You're only going to get 0's and 1's.
So if you have 1 plus 1, it gives you 0.
Obviously 0 plus 0 gives you 0.
And the other cases are clear.
And this plus here, remember, is also-- the other thing that's important is this is not only mod 2.
These are all vectors.
So r is a vector, and you can think of it as a column vector.
That's how I drew it.
You're adding up a column vector with a v. That's the column vector the way I drew that.
You could do it with rows if you like, but it's just notation.
And you're going to compute an r prime here.
What can you say about Dr prime?
Someone?
Yeah.
Go ahead
It's not 0
It's not 0.
And I'll give you a Frisbee, but then you can explain-- can you stand up a little?
I don't want to take this lady's head off.
So can you explain why?
Because r prime is r plus v and Dr gives you Dr plus DB is not 0
Absolutely right.
So essentially what we have is this is simply Dr plus v. 0 plus Dv, not equal to 0.
Do we like yellow or do we like white?
Yellow's fine.
So that's pretty much it.
So what's cool about this- is this final step, which I think you've gotten, but I'm just going to say it out loud now, which is that r to r prime is 1 to 1 for any given r such that Dr equals 0, given the situation where capital D is not equal to 0, and there's some Dij-- and there could be many Dij's.
I just need one.
I've constructed, based on that Dij, this v vector which has the jth entry corresponding to the v vector being a 1 with all of the other entries being a 0.
But I can now create an r to r that is 1 to 1 in the sense that if r prime equals r plus v and that equals r double prime plus v-- so if I ever have a situation where in order to show there's one-to-one, I want to say that it's not too many-to-one or even two-to-one.
So if I have an r prime that equals r plus v and you tell me that r prime also equals r double prime plus v, I can make the argument that r and r double prime are exactly the same.
So then r equals r double prime.
So what am I saying there?
I'm just saying that for any given r that has Dr equals 0, I can twiddle the jth element of that r and go from 0 to 1 or 1 to 0.
If you tell me that there's a Dij somewhere in that matrix that is nonzero and I do that little twiddle-- remembers it's all 0's and 1's Boolean matrices-- so if I do one twiddle, it's one-to-one.
If I do two twiddles and I go 1 to 0, I'm back to 1 again.
And that's all this says.
Because you have mod 2.
That's all that says.
So one little tweak-- and I'm going to be able to take a bad r and turn it into a good r because the good r, the r prime in this case, had Dr prime not equal to 0.
And that's it.
That's my counting argument, and all that remains is to essentially close this by sayiny-- just to write this out to get to the final claim and get the one half-- the one-to-one essentially gives you the one half.
At least half of these things are going to be good r's.
If you had I Dr that's not equal to 0-- and that's the case that you have here-- then we're going to discover an r prime such that the Dr prime is not equal to 0 and r to r prime is a one-to-one mapping.
So the number of r prime for which Dr prime is not equal to 0 is greater than or equal to the number of r for which Dr equals zero.
And so that implies that the probability of Dr not equal to zero-- so if you just choose an r, this is now a randomly chosen r. Not that others weren't, but I'm treating it a little bit differently here.
This was a specific r for which Dr was equal to 0.
I made an argument that you can always get this r prime one-to-one such that Dr prime is not equal to 0.
And now going back to what I had initially with respect to the claim here where the r here was a randomly chosen r, I'm saying, thanks to this little argument-- this line up top-- I'm going to be able to say this is greater than or equal to one half.
OK?
Cool.
Any questions?
Yeah
I think the Dr squared times column equal column on the board
Yeah
On the last column, it should be i, not j
This should be i?
Yes
People agree with that?
Majority vote.
All right.
I'm good.
Let's make that an i
The iteration of that as well, Dv sub j
Oh, yeah.
Of course.
Yeah.
Once you do that, you have to have an i there.
Good.
So you're looking at a particular entry-- it makes a difference whether you used a column or a row.
If I'd done-- now that I remember-- if you turn this into a row matrix and this becomes a row matrix, you'll essentially get the Dvj.
So it depends on which way you look at it, but thanks for pointing that out.
The specifics of i and j weren't particularly important to the proof itself.
The key thing is you zoom in on a particular entry that is not equal to 0, and then you tweak that entry corresponding to the r. So once you tweak that-- you make that 0 or 1 or 1 to 0-- you can get this result.
I'm sorry.
I'm pointing to the wrong spot.
This result-- and then get your claim.
OK?
So summarize-- we have a bound.
We run it over and over, and we get it to the point where we can have a 0.0001 probability that, if the matrices were multiplied incorrectly, that you wouldn't discover that because you ran it for enough hours independently chosen that that probability becomes as low as possible.
OK?
So that was Monte Carlo.
Let's do a Las Vegas algorithm.
And you guys are probably thinking, my goodness.
Another sorting algorithm after, I don't know, 17 different sorting algorithms.
This all sorting algorithms that you've ever learned so far.
Right?
So merge sort doesn't work, and the reason it doesn't work in practice-- if you're really into performance-- is because of the auxiliary space that merge sort requires.
So if you recall there's the notion of in-place sorting.
So let's move onto the next thing here, which is quicksort, which is a new sorting algorithm.
And I want to motivate it for just a couple of minutes.
And the primary motivation really is practical performance, not asymptotic complexity.
So I'll be upfront about that.
It's all about practical performance corresponding to quicksort.
And quicksort is a divide and conquer randomized algorithm invented in '62.
Unlike merge sort, it's got two interesting properties.
The first is that it's in place, like I just said.
So no auxiliary space.
In Mozart, you can try and get around this.
I should say order n auxiliary space.
You need a little temporary variable in order to do a swapping.
But you don't have the order n auxiliary space.
So you don't have to constantly allocate.
And remember, n could be large.
It could be in the billions or trillions.
So, from that standpoint, quicksort ends up winning simply because of relatively mundane things like memory allocation in your computer.
And the other interesting thing about quicksort in relation to merge sort is that all the work is in the divide step.
So in merge sort, remember we just split, and we recurse.
And what happens when you come back is you have to do the finger emerging algorithm by looking at the two sorted arrays and looking at what the new merger is going to look like.
So the work is in the merge.
But in quicksort, the work is going to be in the divide because we're going to have to do a bunch of work associated with figuring out how to keep the partitions balanced-- a little bit like we had to do when we did median finding back a couple of weeks ago.
I'm going to talk about three different variants of quicksort.
The.
variant that we're going to spend the most time on is the Las Vegas quicksort where we'd like to show that it's probably fast and make a statement about the expected runtime.
But we'll get to that by talking about a couple of other interesting variants, and this'll be elaborated on to some extent in section tomorrow.
So before we get to variants of course, let's try and set up the structure corresponding to quicksort.
And as always, we have an n element array A.
You have divide that corresponds to picking a pivot element, x in A.
And then we're going to partition the array into sub-arrays.
And what we have here-- this little picture should make things clearer.
And you kind of saw this in the median finding, but here we go again.
Let's assume all the array elements are unique.
We have L, E, and G. L is less than.
G is greater than.
And so your pivot element is going to break this array up into L and G, where you got all the elements that are less on the left and all the elements that are greater on the right.
And you're going to recurs on the L and G. So recursively sort sub-arrays L and G. Combine is trivial-- or merge is trivial-- because you've already broken things up thanks to the pivoting.
And you just concatenate those arrays.
And that's why you can do this in place.
There's no issues.
You're really recursively sorting sub-arrays.
You are moving things around a little bit when you do the partition.
Obviously, the initial array may have all the elements.
You may pick the pivot such that the pivot is all the way on the right-hand side in the sense that it's a very large element.
That is not necessarily a good thing.
I will talk about that.
But if you pick an interesting pivot or a good pivot, you're going to have to move the elements in the array to the left of the pivot if they're less than the pivot, and you got to move the elements to the right if they're to the right of the pivot.
Nontrivial piece of code, not super complicated, but you can look at the CLRS page 171 to look at in-place partitioning where you don't have to use another order n space to move these elements around such that they look like that picture that I have up there, starting from some random starting point.
So you want to have the picture that you have here, and you need to go from-- the very same array, it needs to-- and x is somewhere here, and you got x plus 1 here and x minus 1 here, for example.
And you need to move those things around so they look like L, E, and G, and that's something that you can do in place.
And you can look at the code for that in the CLRS.
I won't cover that here.
So let's look at a bunch of different variants responding to quicksort.
And there's some real simple ones.
Each of these, we can knock off with respect to complexity and runtime fairly easily with the one exception that we'll spend some time on, which is the Las Vegas quicksort.
But we'll call these different names.
Let's talk about the basic quicksort, which is also a useful algorithm that people use in practice.
And amazingly, this algorithm is simply something that says, I'm just going to constantly pivot on either the first entry or the last entry.
So I'm going to pick my pivot to be A1.
And when I pick my pivot to be A1, it's a value that I'm talking about here.
X is a value.
It's not an index.
The A1 value-- maybe that's 75.
Then I'm going to create my L matrix corresponding to this pivot where all the entries are strictly less than 75, and G would be strictly greater than 75.
And I could do that for A1.
I could do that for An.
So remember that the pivot is a value.
Now, if I look at this, I'm going to do the partition, given x, just like you saw there.
And this is going to be done in order n time.
It makes sense that you're going to look at every element, and you're going to move it to an appropriate location to the left of x, which is the e array, or to the right.
And you'll do that.
That takes order n time.
And, as I mentioned, you can look at this to see how this is done in place.
So let's take a look at the analysis of basic quicksort, and what I'm interested in, of course, is the worst case analysis.
And I asked this question, I think, before when we were doing median finding, but what is the worst case complexity of the basic quicksort algorithm that chooses the pivot as A1?
What is the complexity?
[INTERPOSING VOICES] Order n square.
It's order n square.
And the reason for that is that you may have an array that is sorted or reverse sorted-- depending on whether you're picking A1 or An.
You can have a worst case situation where one side, L or G, has n minus 1 elements, and the other has 0 elements.
And so if you look at our recurrence associated with this, you could have Tn, which is T0, plus T n minus 1 plus theta n. And why do I have a theta n here?
Well, remember that I still have to do this divide step or this partition step in order to compute this up unbalanced array.
So I do have to look at each of these elements and do the comparison.
And maybe I don't actually have to move them, but I have to do the comparison with the A1, which is the x pivot.
And in some cases, if I'm doing the wrong thing reverse sorted, I also have to do the move.
Either way, I have a theta n complexity associated with the divide step.
And so if you go off and you look at what happens with this, well, you've got Tn equals Tn minus 1 plus theta n, which ends up with theta n square complexity.
So a hand waved a little bit two weeks ago for a similar analysis, but you can kind of look at it a little more precisely here by writing the actual recurrence out.
And you see that you get the recurrence Tn equals Tn minus 1 plus theta n, which is an n square result, or the solutions is n square.
OK?
So basic quicksort look bad.
It's got a worst case complexity of theta n square.
It works well on random inputs and practice.
And it turns out that it's a fashion of algorithm, partly because it's in place and it's easy to code, that what people do is they take their inputs, and they shuffle them.
You might get a bad input, and it might take you a long time to run.
But if you take an input and you shuffle it and you do that in theta n time, you just move things around and randomize the input.
Then effectively, you have a random input, and this thing works pretty well in practice.
Now, what is pretty well?
Well, we're going to do an analysis that is going to not be exactly the analysis that you'd have to do on basic quicksort on random inputs, but essentially, you can say that basic quicksort on random inputs is going to run in expected theta m log n time.
OK?
It's something that you'll see a little bit of how to do that today and in section, perhaps for median finding in section tomorrow.
But that's all I wanted to say about basic quicksort.
It's a practical algorithm.
It does require a little bit of shuffling up at the beginning, and then you can simply use the pivot A1.
And because you've done the shuffle, generally you get balance partitions.
The L and G's look balanced, and you don't end up with theta n square.
If you have any sort of balance associated with the two partitions L and G, you're going to get a nice divide and conquer, which is going to give you your theta n log n. OK?
So that's basic quicksort.
There's another way to do this, and so this is a question for you guys.
Suppose I wanted to use the quicksort strategy and get a worst case theta n log n through an intelligent pivot selection.
So I want to do a pivot selection intelligently.
So how would I get under the structure of quicksort that you see up there on the left there?
How would I select a pivot such that I get worse case theta n log n complexity?
Go ahead
Linear median finding
Linear medium finding.
Perfect.
That's exactly right.
There's a gentleman at the back who'd raised his hand, and I decided I'd chicken out.
I think one time to the back of the room is enough for a day.
I'll have a Frisbee left.
Hopefully you can get one.
So the intelligence pivots selection algorithm is the median finding algorithm because that's going to guarantee me that I'm going to get balanced partitions.
If you tell me that A1-- and remember, we're talking medians of values-- so don't get confused with indices.
When I say something is a median, I'm talking about the value, that given its value, there are all these other n over 2 values that are less than it, roughly speaking and n over 2 values that are greater than it.
And so A1, I have no idea whether it's large or small.
So I couldn't say much about it.
But if I want to be worst case and I want to guarantee that I have balanced partitions, I can choose the median.
And if I choose the median every time, I'm going to get perfectly balanced partitions.
They're going to half on the left and half on the right.
And we do know a way of getting balanced partitions.
We can guarantee that balanced L and G using median selection that runs in theta n time.
And we showed that a couple of weeks ago.
Now, that median selection algorithm was nontrivial.
OK?
It had this weird thing where you broke things up into five sub-arrays of size 5, and you found a median of medians et cetera, et cetera.
But we argued that the whole thing ran in theta n time, which is important.
And so now, if you look at what happens with quicksort and if I write the recurrence for quicksort, thanks to selecting a median, I effectively have balanced partitions.
So I have 2T n over 2.
This is thanks to the median based pivoting.
That's important.
Otherwise it won't work.
And then, just to be very clear here, I got two theta n terms.
OK?
The first theta n term is the recursive median selection.
And then the second theta n term is of course the divide, or partition.
But it's important to realize that now I have a lot of work in the divide.
A lot of work.
I have to do an intelligent selection using this recursive median finding algorithm.
And I also have to do the moves comparing and then generate and the G arrays.
OK?
So those are the two theta n's.
They're obviously theta n, but I wanted to make it clear that there's two things going on here.
And we all know that that is theta n log n worst case.
All right?
So there is a way of using the quicksort structure template and getting a theta n log n worst case algorithm, which doesn't work in practice because it's just too complicated.
What's going on here is at every level of recursion, you're calling another recursive algorithm to find the median.
So if you go code this up, it loses to merge sort in practice.
You can do all of this in place, but because of all these recursive calls, it doesn't work well in practice.
But it's good to know.
And so this is a good example I think, which we don't do a lot of in 046, but you get a sense of the difference between asymptotic complexity and performance.
So while the median finding algorithm has better asymptotic complexity worst case, it really loses in practice to the basic quicksort, which essentially is a bit of a hack, where you take an input and you randomize the input and you run it with A1 as the pivot or An at the pivot.
Is there a different way that you can actually get to a Las Vegas algorithm?
And it turns out randomized quicksort is something that you can build and use, which is a bit different from basic quicksort and certainly different from median finding.
But it kind of has a little bit in common with them, and it's our example of a Vegas algorithm.
So what happens at randomized quicksort?
An x is chosen at random from the array, A.
So you're not choosing A1 or An.
You might just flip-- well, effectively an n-sided die-- and pick a particular index, and then go grab the pivot corresponding to the value at that index.
You're not going to randomize over values.
You don't know what these values are, but you can pick a random index and then grab the pivot based on the value at that index.
And so at each recursion, a random choice is made.
And the expected time-- so now we're saying something different.
We're making a stronger theoretical statement that the expected time, when you do this, for all inputs arrays A is order n log n. And so now, this is not worst case time.
It's expected time.
So this is going to be our analysis in the last few minutes here to analyze not randomized quicksort, but a slight variant of randomized quicksort that is going to show you that you can run randomize quicksort and this variant in order n log n time.
So not quite sure what's going to happen in section tomorrow, but the full analysis is in the book.
You should read it.
As you can see, it's a couple of pages that includes the description of a quicksort that I have already.
But what we're going to do here is analyze a variant quicksort, which is a little bit easier to analyze, and it gives you the sense of why in fact the randomized quicksort is going to run in expected time.
And this analysis is easy to do it in a few minutes.
So we'll do that.
And tomorrow, you'll see either a median finding analysis that's similar to that analysis in CLRS or precisely that analysis, depending on what your TAs want to do.
So this particular variant, we're going to call paranoid quicksort.
And so this quicksort is paranoid in the sense that it's going to be afraid of getting unbalanced partitions, and it's going to keep trying to get balanced partitions.
So it's going to try to get a balanced partition.
It's going to check, and then if it fails, it's going to try again.
And so at the end of it, there's obviously an expectation associated with the number of tries that you need in order to get a balanced partition.
But it just sort of flips the problem on its head and says, you know what?
I'm just going to guarantee a balanced partition from a probabilistic standpoint and it might take me a little bit longer to get there.
But that's what Las Vegas algorithms are all about.
They're probably fast.
And once I get a balanced partition, I'm in good shape because I can go do my recursion, and I get my divide and conquer working properly.
So what is paranoid quicksort?
Absolutely straightforward.
You could probably guess given my description.
Let's just choose a pivot to be a random element of A.
Perform the partition, and then values will repeat.
So we're going to go off, and we say until the resulting partition is such that the cardinality of L less than or equal to 3/4 of cardinality of A.
And the cardinality of G is less than or equal to 3/4 the cardinality of A.
So I'm allowing you a certain amount of imbalance, but not a lot.
Right?
And that's it.
That's paranoid quicksort.
You obviously are doing that in each level of the recursion.
And at each level of the recursion, your L and G are going to be, at most, a factor of three apart.
So you might get 1/4 and 3/4.
If you're lucky, you'll get 1/2 and 1/2.
But the worst case, given that you're going to be exiting out of this loop, is 1/4 and 3/4.
OK?
So, as always, you have a simple algorithm, and it's not completely clear how you're going to get to expected n log n time.
But it's not difficult.
Basically, what we had to do is we have to try and figure out what the probability of a good call is, over here, a good pivot choice, and what the probability of a bad pivot choice is.
And we have to obviously-- given the potential imbalance, we have to write the recurrence associated with that, but let's take a look at the pivots here.
And what can we say about the size as of L and G if you just did a random pivot?
Well, a bad call is when you get something in L or G that is less than 1/4.
And a good call is when you get somewhere between 1/2-- well, roughly, if you look at the choice of the pivot.
So what I have up here, is the choice of the pivot.
If my pivot is out here, I have a very small L, and all of the thing on the right is G. If the pivot is here, I have a relatively small L and a large G. The pivot is over here, I'm good.
I got 1/4 and 3/4.
If the pivot is over here, I got 1/2 and 1/2 and so on and so forth.
And so this part is bad, this part is bad, and the middle part is good.
So that's all that this picture shows.
So a call is good with what probability?
Given that picture, a call is good with what probability?
It's greater than or equal to 1/2.
And so what you can now write simply is if Tn is the time required to sort the array, essentially you can say Tn is T of n divided by 4 plus T of 3n divided by 4 plus expected number of iterations in terms of getting a good partition times C times n. And there is a reason why I'm putting C in here as opposed to theta.
That will become clear in just a second.
Because I can't really apply the master theorem to this given what I have with respect to Tn over 4 and 3n over 4.
So what I have here is, I'm looking at the case where I could get an imbalanced partition, but the imbalance is bounded.
So I'd have n over 4 on one side and 3n over 4 on the other side.
But I'm not going to have n over 5 and 4n over 5 or what have you.
And so that's the two recursive calls.
So that's hopefully easy to see.
The part that is new here is simply the complexity of this code that you see here, which is obviously the randomized algorithm.
That's exactly where the randomness comes in because you're picking a random pivot, and you're checking it.
And so this is going to run a certain number of times.
And we can figure out what the expectation is in just a minute.
But I have C times n because this is constant time to choose a random number.
We'll assume that performing the partition is C times n or theta n, and that's why I have this up there.
So this, we're going to call this Cn.
And so expected number of iterations given what I have-- what can I say about the expected number of iterations using simple probability rules?
What is that?
2, right?
1 over p. All of them are independent.
So this is 2.
So what I have here is something that I think you might have seen this before, but it's worth drawing the tree out and seeing it one more time in case it didn't fully registered the first time or you didn't actually see it in 006 or recitation.
But what I now have is T of n. I want to solve T of n equals T of n over 4 plus T of 3 n over 4 plus 2 times Cn.
And, again, like I said, I didn't put theta n in here because, as you'll see, when I draw this tree out-- because it's not a massive theorem invocation-- it's worth looking at it from a constant factor standpoint to really get the sense of how all of this works out.
And so if I draw that tree of execution and I start counting, basically what I have is 2Cn up at the top.
I have 1 over 4 times 2Cn over here.
I have 3 over 4 times 2Cn over here.
And then this 1 over 4 might go 1 over 16 times 2Cn over here.
And this might go 3 over 16 times 2Cn over here.
And this would go, I guess it would be 3 over 16 times 2Cn.
And then 9 over 16 times 2Cn et cetera.
So this is an unbalanced tree because you have an unbalanced partition up on top, and now you want to count up all the work that this tree does.
If you collect up all of the operations, then that's going to tell you what T of n is because that's all the work that you have to do in order to finish up the top level of recursion.
And what you can say is, if you look at this side here are all the way to the right-hand side, you're going to have log to the base 4 over 3 times 2Cn levels.
So that's just simply every time you're multiplying by 3 over 4, when you get down to the number 1, and that's log of 4 over 3.
And then over here, it's a little bit easier to think about because it's a power of 2.
You're going to have log of 4 to the base 4 times 2Cn levels.
And really, it doesn't really matter honestly when we go to asymptotics.
But is worth seeing, I think, just to get a sense of why it all works out, regardless of whether it's n over 4 or a different constant here or whether it's balanced or unbalanced.
The tree looks a little bit different.
It's sort of weird.
It's got fewer levels here and more levels there.
So it's sort of tilted this way.
But eventually, you get down to theta 1 constants down below.
And basically what you can see-- if you just add it up-- is 1 over 4 plus 3 over 4 is 1.
1 over 16.
3 over 16.
Obviously, those all end up being 1.
So you have 2Cn work at each level.
And if you just go ahead and be pessimistic about it, there's a maximum of log 4 over 3 times 2Cn levels.
And that's pretty much it.
Obviously, now you can start ignoring the constants.
You just keep the log here.
You don't care what the base is.
You got an n here.
So drop the 2C.
Drop the 4 over 3.
Drop the 2C.
And you get your n log n. OK?
So that's pretty much it.
I'll stick around here for questions.
But you got an example of a Monte Carlo algorithm.
You got an example of a Las Vegas algorithm.
And tomorrow in section, you'll see a slightly more involved analysis for something that looks a lot closer to the randomized quicksort.
So see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
And let's get started.
Today's lecture is about a randomized data structure called the skip list.
And it's a data structure that, obviously because it's randomized, we'd have to do a probabilistic analysis for.
And we're going to sort of raise the stakes here a little bit with respect to our expectation-- the pun intended of this data structure-- in the sense that we're not going to be happy with just doing an expected value analysis or to get what the expectation is of the search complexity in a skip list.
We're going to introduce this notion with high probability, which is a stronger notion than just giving you the expected value or the expectation for the complexity of a search algorithm.
And we're going to prove that under this notion, that search has a particular complexity with high probability.
So we'll get to the with high probability part a little bit later in the lecture, but we're just going to start off doing some cool data structure design, I guess, [INAUDIBLE] pointing to the skip list.
The skip list is a relatively young data structure invented by a guy called Bill Pugh in 1989, so not much older than you guys.
It's relatively easy to implement as you'll see.
I won't really claim that, but hopefully you'll be convinced by the time you're done describing the structure.
Especially in comparison to balanced trees.
And we can do a comparison after we do our analysis of the data structure as to what the complexity comparisons are for search and insert when you take a skip list and compare it to an AVL tree, for example, or a red black tree, et cetera.
In general, when we have a data structure, we want it to be dynamic.
The skip list maintains a dynamic set.
What that means is not only do you want to search on it-- obviously it's uninteresting to have a static data structure and do a search.
You want to be able to change it, want to be able to insert values into it.
There's a complexity of insert to worry about.
You want to be able to delete values.
And the richness of the data structure comes from the operations and the augmentations you can do on it, and the skip lists are no exception to that.
So if you want to maintain a dynamic set of n elements, and you obviously know a ton of data structures to do this, each of which has different characteristics.
And this is, if you ignore hash tables, this is your first real randomized data structure, if you're just taking 6006 and this class might have seen randomized structures in other classes.
So we're going to try and do this in order log n time.
As you know with balanced binary trees, you can do things in order log n time, a ton of things, pretty much everything that is interesting.
And this, given that it's randomized, it's a relatively easy analysis to show that the expectation or the expected value of a search would be order log n expected time.
But we're going to, as I said, raise the stakes, and we're going to spend a ton of time the second half of this showing the with high probability result.
And that's a stronger result than just saying that search takes expected order log n time.
All right, so that's the context.
You can think of a skip list as beginning with a simple linked list.
So if we have one link list and that link list-- let's first think of this as being unsorted.
So suppose I have a link list which is unsorted and I want to search for a particular value in this link list.
And we can assume that this is a doubly-linked list, so the arrows go both ways.
You have a pointer, let's say, just to the first element.
So if you have a list that's unsorted and you want to search for an element, you would want to do a membership query.
If there's n elements, the complexity is?
Order n
Order n. So a linked list, the search takes the order n time.
Now let's go ahead and say that we are sorting this list, so it's a sorted linked list.
So your values here, 14, 23, 34, 42, 50, 59.
They're sorted in ascending order.
You still only have a pointer to the front of the list and it's a doubly-linked list, what is the complexity of search in the sorted link list?
Log n
Log n. Oh, I wanted to hear that.
Because it is?
Order n
It's order n. log n is-- yeah, that was a trick question.
Because I liked that answer, the person who said log n gets a Frisbee.
This person won't admit.
[LAUGHTER] Oh, it was you.
OK, all right.
There you go.
All right.
So log n would imply that you have random access.
If you have an array that's sorted and you can go [?
AFi, ?]
and you can go [?
AFi ?]
divided by 2, or [?
AF2i ?]
and you can go directly to that element, then you can do binary search and you can get a log n, order log in.
But here, the sorting actually doesn't help you with respect to the search simply because you have to start from the beginning, from the front of the list, and you've got to keep walking.
The only place that it helps you is that if you know it's sorted and you're looking for 37, you can stop after you see 42, right?
That's pretty much the only place that it helps you.
But it's still order n because that could happen-- on average is going to happen halfway through the list for a given membership query.
So it's still order n for a sorted link list.
But now let's say that we had two sorted link lists.
And how are these two link lists structured?
Well, they're structured in a certain way, and let me draw our canonical example for skip list that I'm going to keep coming back to.
So I won't erase this, but I'll draw one out-- 1, 2, 3, 4, 5, 6, 7, 8, 9-- 9 elements.
So that's my first list which is sorted, and so I have 14, 23, 34, 42, 50, 59, 66, 72, and 79.
What I'm going to have now is another list sort of on top of this.
I can move from top to bottom, et cetera.
But I'm not going to have elements on top of each bottom element.
By convention, I'm going to have elements on top of the first element, regardless of how many lists I have.
We only have two at this point.
And so I see a 14, which is exactly the same element duplicated up on the top list.
And that list is also sorted, but I won't have all of the elements in the top list.
I'm just picking a couple here.
So I've got 34, 42-- they're adjacent here-- and then I go all the way up to 72, and I duplicate it, et cetera.
Now, this looks kind of random.
Anybody recognize these numbers?
No one from the great City of New York?
No?
Yup, yup
On the subway stops?
Yeah, subway stops on the Seventh Avenue Express Line.
So this is exactly the notion of a skip list, the fact that you have-- could you stand up?
Great.
All right.
So the notion here is that you don't have to make a lot of stops if you know you have to go far.
So if you want to go from 14th Street to 72nd Street, you just take the express line.
But if you want to go to 66th Street, what would you do?
Go to 72nd and then go back
Well, that's one way.
That's one way.
That's not the way I wanted.
The way we're going to do this is we're not going to overshoot.
So we want to minimize distance, let's say.
So our secondary thing is going to be minimizing distance travel.
And so you're going to pop up the express line, go all the way to 42nd Street, and you're going to say if I go to the next stop on the Express Line, I'm going too far.
And so you're going to pop down to the local line.
So you can think of this as being link list L0 and link list L1.
You're going to pop down, and then you're going to go to 66th Street.
So search 66 will be going from 14 to 42 on L1, and then from 42, let's just say that's walking.
42 to 42, L1 to L0.
And then 42 to 66 on L0.
So that's the basic notion of a skip list.
So you can see that it's really pretty simple.
What we're going to do now is do two things.
I want to think about this double-sorted list as a data structure in its own right before I dive into skip lists in general.
And I want to analyze at some level, the best case situation for worst case complexity.
And by that I mean I want to structure the express stops in the best manner possible.
These stops are very structured for passengers because they figured fancy stops on 42nd Avenue, whatever-- fancy stores.
Everybody wants to go there and so on and so forth.
So you have 34 pretty close to 42 because they're both popular destinations.
But let's say that things where I guess more egalitarian and randomized, if you will.
And what I want to do is I want to structure this double-sorted list so I get the best worst case complexity for search.
And so let's do that.
And before I do that, let me write out the search algorithm, which is going to be important.
I want you to assimilate this, keep it in your head because we're going to analyze search pretty much for the rest of the morning here.
And so I'll write this down.
You've got a sense of what it is based on what I just did here with this example of 66, but worth writing down.
We're going to walk right in the top linked list, so this is simply for two linked lists, and we'll generalize at some point.
So we want to walk right in the top linked list, L1, until going right would go too far.
Now, there was this answer with 72, which I kind of dismissed.
But there's no reason why you can't overshoot one stop and go backwards.
It would just be a different search algorithm.
It's not something we're going to analyze here.
It turns out in analyzing that with high probability would be even more painful than the painful analysis we're going to do.
So we won't go there.
And then we walk down to the bottom list.
And the bottom list we'll call L0.
And walk right in L0 until the element is found or not.
And you know that if you've overshot.
So if you're looking here for route 67, when you get to 72 here-- you've seen 66 and you get to 72 and you're looking for 67, search fails.
It stops and fails.
Doesn't succeed in this case.
So that's what we got for search.
And that's our two linked list argument.
Now, our analysis essentially says what I have is I'm walking right at the bottom list here, and my top list is L1, so I start with L1.
And my search cost is going to be approximately the length of L1.
The worst case analysis, I could go all the way on the top list-- it's possible.
But for a given value, I'm going to be looking at only a portion of the bottom list.
I'm not going to go all the way on the bottom list ever.
I'm only going to be looking at a portion of it.
So it's going to be L0 divided by L1, if I have interspersed my express stops in a uniform way.
So there's no reason-- if I have 100 elements in the bottom list, and if I had five, just for argument sake, five in the top list, then I'd put them at, let's say, the 0 position, 20, 40, 60, et cetera.
So I want to have roughly equal spacings.
But we need to make that a little more concrete, and a little more precise.
And what I'm saying here simply is that this is the cost of traversal in the top list, and this is the cost of traversal in the bottom list, because I'm not going to go all the way in the bottom list.
I'm only going to go a portion on the bottom list.
Everybody gets that?
Yup?
All right, good.
So if I want to minimize this cost, which is going to tell me how to scatter these elements in the top list, how to choose my express stops, if you will-- I want to scatter these in a uniform way, then this is minimized when terms are equal.
You could go off and differentiate and do that.
It's fairly standard.
And what you end up getting is you want to get L1 square equals L0 equals n. So all of the elements are down at the bottom list, and so the cardinality of the bottom list is n. And roughly speaking, you're going to end up optimizing, if you have this satisfied, which means that L1 is going to be square root of n. OK?
So what you've done here is you've said a bunch of things, actually.
You've decided how many elements are going to be in your top list.
If there's n elements in the bottom list, you want to have the square root of n elements in the top list.
And not only that, in order to make sure that this works properly, and that you don't get a worse case cost that is not optimal, you do have to intersperse the square root of n elements at regular intervals in relation to the bottom list on the top list.
OK, so pictorially what this means is it's not what you have here.
What you really want is something that, let's say, looks like this where this part here is square root of n elements up until that point, and then let's say we go from here to here or square root of n elements, and maybe I'll have a 66 here because that's exactly where I want my square root of n. Basically, three elements in between.
So I got 66 here, et cetera.
I mean I chose n to be a particular value here, but you get the picture.
So the search now, as you can see if you just add those up you get square root of n here, and you got n divided by square root of n here.
So that's square root of n as well.
So the search cost is order square root of n. And so that's it.
That's the first generalization, and really the most important one, that comes from going from a single sorted list to an approximation of a skip list.
So what do you do if you want to make things better?
So we want to make things better?
Are we happy with square root of n?
No
No.
Well, what's our target?
Log n
Log n, obviously.
Well, I guess you can argue that our target may be order 1 at some point, but for today's lecture it is order log n with high probability.
We'll leave it at that.
And so what do you do if you want to go this way and generalize?
You simply add more lists.
I mean it seems to be pretty much the only thing we could do here.
So let's go ahead and add a third list.
So if you have two sorted lists, that implies I have 2 square root of n. If I want to be explicit about the constant in terms of the search cost, assuming things are interspersed exactly right.
Keep that in mind because that is going to go away when we go and randomize.
We're going to be flipping coins and things like that.
But so far, things are very structured.
What do you think-- we won't do this analysis-- the cost is going to be if I intersperse optimally, what is the cost going to be for a search when I have three sorted lists?
Cube root
Cube root.
Great guess.
Who said cube root?
[INAUDIBLE]
You already have a Frisbee.
Give it to a friend.
I need to get rid of these.
So it's going to be cube root, and the constant in front of that is going to be?
3
3.
So you have-- right?
So let's just keep going.
You have k sorted lists.
You're going to have k times the k-th root of n. That's what you got.
And I'm not going to bother drawing this, but essentially what happens is you are making the same number of moves which corresponds to the root of n, the corresponding root of n, at every level.
And the last thing we have to do to get a sense for what happens here is we have log n sorted lists, so the number of levels here is log n. So this is starting to look kind of familiar because it borrows from other data structures.
And what this is I'm just going to substitute log n for k, and I got this kind of scary looking-- I was scared the first time I saw this.
Oh, this is n. It's the log n-th root of n, OK?
And so it's kind of scary looking.
But what is the log n-th root of n-- and we can assume that n is a power of two?
2
2, exactly.
It's not that scary looking, and that's because I'm not a mathematician.
That's why I was scared.
So 2 log n. All right.
So that's it.
So you get a sense of how this works now, right?
We haven't talked about randomized structures yet, but I've given you the template that's associated with the skip list, which essentially says what I'm going to have are-- if it was static data items and n was a power of two, then essentially what I'm saying is I'm going to have a bunch of items, n items, at the bottom.
I'm going to have n over 2 items at the list that's just immediately above.
And each of them are going to be alternating.
You're going to have an item in between.
And then on the top I'm going to see n over 4 items, and so on and so forth.
What does that look like?
Kind of looks like a tree, right?
I mean it doesn't have the structure of a tree in the sense of the edges of a tree.
It's quite different because you're connecting things differently.
You have all the leaves connected down at the bottom of this so-called tree with this doubly linked list, but it has the triangle structure of a tree.
And that's where the log n comes from.
So this is would all be wonderful if this were a static set.
And n doesn't have to be a power of 2-- you could pad it, and so on and so forth.
But the big thing here is that we haven't quite accomplished what we set out to do, even though we seem to have this log n cost for search.
But it's all based on a static set which doesn't change.
And the problem, of course, is that you could have deletions.
You want to take away 42.
For some reason you can't go to 42nd Avenue, or I guess art-- you can't go to [INAUDIBLE] would be a better example.
So stuff breaks, right?
And so you take stuff out and you insert things in.
Suppose I wanted to insert 60, 61, 62, 63, and 64 into that list that I have?
What would happen?
Yeah, you're shaking your head.
I mean that log n would go away, so it would be a problem.
But what we have to do now is move to the probabilistic domain.
We have to think about what happens when we insert elements.
We need an algorithm for insert.
So then we can start with the null list and build it up.
And then you start with a null list and you have a randomized algorithm for insert, it ain't going to look that pretty.
It's going to look random.
But you have to have a certain amount of structure so you can still get your order log n. So you have to do the insertion appropriately.
So that's what we have to do next.
But any questions about that complexity that I have up there?
All right, good.
I want a canonical example of a list here, and I kind of ran out of room over there, so bear with me as I draw you a more sophisticated skip list that has a few more levels.
And the reason for this is it's only interesting when you have three or more levels.
The search algorithm is kind of the same.
You go up top and when you overshoot you pop down one level, and then you do the same thing over and over.
But we are going to have to bound the number of levels in the skip list in a probabilistic way.
We have to actually discover the expected number of levels because we're going to be doing inserts in a randomized way.
And so it's worthwhile having a picture that's a little more interesting than the picture of the two linked lists that I had up there.
So I'm going to leave this on for the rest of the lecture.
So that's our bottom, and that hasn't changed from our previous examples.
I'm not going to bother drawing the horizontal connections.
When you see things adjacent horizontally at the same level, assume that they're all connected-- all of them.
And so I have four levels here.
And you can think of this as being the entire list or part of it.
Just to delineate things nicely, we'll assume that 79, which is the last element, is all the way up at the top as well.
Sort of the terminus, termini, corresponding to our analogy of subways.
And so that's our top-most level.
And then I might have 50 here at this level, or so that looks like.
I will have 50, so the invariant here, and that's another reason I want to draw this out, is that if you have a station at highest level, then you will have-- it's got to be sitting on something.
So if you've got a 79 at level four, or level three here if this is L0, then you will see 79 at L2, L1, and L0.
And if you see 50 here, it's not in L3 so that's OK, but it's in L2, so it's got to be at L1 as well.
Of course you know that everything is down at L1, so this is interesting from a standpoint of the relationship between Li and Li plus 1 where i is greater than or equal to 1.
So the implication is that if you see it at at Li plus 1, it's going to be at Li and Li minus 1 if that happens to exist, et cetera.
And so one last thing here just to finish it up.
I got 34 here, which is an additional thing which ends there.
So the highest level is this second level or L1.
This is 66.
And then that's it.
So that's our skip list.
So if you wanted to search for 72, you would start here, and then you'd go to 79, or you'd look and say, oh, 79 is too far, so I'm going to pop down a level.
And then you'd say 50, oh, good.
I can get to 50.
79 is too far, so I'm going to pop down a level.
And then you go to 66-- 79 is too far-- and at 66, you pop down a level and then you go 66 to 72.
So same as what we had before.
Hopefully it's not too complicated.
So that's our skip list.
It's still looking pretty structured, looking pretty regular.
But if I start taking that and start inserting things and deleting things, it could become quite irregular.
I could take away 23, for example.
And there's nothing that's stopping me from taking away 34 or 79.
You've got to delete an element, you've got to delete an element.
I mean the fact that it's in four levels shouldn't make a difference.
And so that's something to keep in mind.
So this could get pretty messy.
So let's talk about insert, and I've spent a bunch of time skirting around the issue of what exactly happens when you insert an element.
Turns out delete is pretty easy.
Insert is more interesting.
Let's do insert.
To insert an element x into a skip list, the first thing we're going to do is search to figure out where x fits into the bottom list.
So you do a search just like you would if you were just doing a search.
You always insert into the appropriate position.
So if there's a single sorted list, that would pretty much be it.
And so that part is easy.
If you want to insert 67, you do all of the search operations that I just went over, and then you insert 67 between 66 and 72.
So do your pointer manipulations, what have you, and you're good.
But you're not done yet, because you want this to be a skip list and you want this to have expected search over any random query as the list grows and shrinks of order log n, expectation, and also with high probability.
So what you're going to have to do is when you start inserting, you're going to have to decide if you're going to what is called promote these elements or not.
And the notion of a promotion is that you are going up and duplicating this inserted element some number of levels up.
So if you just look at how this works, it's really pretty straightforward.
What is going to happen is simply that let's say I have 67 and I'm going to insert it between 66 and 72.
That much is a given.
That is deterministic.
Then I'm going to flip a coin or spin a Frisbee.
I like this better.
I'm not sure if this is biased or not.
It's probably seriously biased.
[LAUGHTER] Would it ever go the other way is the question.
Would it ever?
No.
All right.
So we've got a problem here.
I think we might have to do something like that.
[LAUGHTER] I'm procrastinating.
I don't want to teach the rest of this material.
[LAUGHTER] All right.
Let's go, let's go.
So I'd like to insert into some of the lists, and the big question is which ones?
It's going to be really cool.
I'm just going to flip coins, fair coins, and decide how much to promote these elements.
So flip fair coin.
If heads, promote x to the next level up, and repeat.
Else, if you ever get a tails, you stop.
And this next level up may be newly created.
So what might happen with the 67 is that you stick it in here, and it might happen that the first time you flip you get a tails, in which case, 67 is going to just be at the bottom list.
But if you get one heads, then you're not only going to put 67 in here, you're going to put 67 up here as well.
And you're going to flip again.
And if you get a heads again, you're going to put 67 up here.
And if you get a heads again, you're going to put 67 up here.
And if you get a heads again, you're going to create a new list up there, and at this point when you create the new list, it's only going to be 67 up there.
And that's going to be the front of your list, because that's the one element that you're duplicating.
So you're going to keep going until you get a tails.
Now, that's why this coin had better be fair.
So you're going to keep going and you're going to keep adding.
Every time you insert there's a potential for increasing the number of levels in this list.
Now, the number of levels is going to be bounded in expectation with a high probability of regular expectation, but I want to make it clear that every time you insert, if you get a chain of heads, you're going to be adding levels.
And so the first time you get a tails, you just stop.
You just stop.
So you can see that this can get pretty messy pretty quick.
And especially if you were starting from ground zero and adding 14, 23-- all of those things, the bottom is going to look exactly like it looks now because you're going to put it in there.
It's deterministic.
But the very next level after that looked pretty messy.
You could have all of them chunked up here, and a big gap, et cetera, et cetera.
So it's all about randomized search cost.
The worse case cost here is going to be order n. Worst case cost is going to be order n, because you have no idea where these things are going to end up.
But the randomized cost is what's cool about this.
Any questions about insert or anything I said?
Yeah, go ahead
Is worse case really order n?
What if you had a really long, like a lot of lists on top of each other, and you start at the top of that and you had to walk all the way [INAUDIBLE]?
Well, you go n down and n this way, right?
You would be checking so it would be order n
So it's [?
bounded ?]
by n?
Yeah, the worst case
Worse case is infinity
Worse case is infinity.
Oh, in that sense, yeah.
OK. Well, n elements, Eric is right.
So what is happening here is that you have a small probability that you will keep flipping heads forever.
So at some level, if you somehow take that away and use Frisbees instead or you truncate it.
Let's say at some point you ended up saying that you only have n levels total.
So it's not a-- I should have gone there.
The question has to be posed a little more precisely for the answer to be order n. You have to have some more limitations to avoid the case that Eric just mentioned, which is in the randomized situation you will have the possibility of getting an infinite number of heads.
Yeah, question back there
[INAUDIBLE]
Yes, you can certainly do capping and you can do a bunch of other things.
It ends up becoming something which is not as clean as what you have here.
The analysis is messy.
And it's sort of in between a randomized data structure, a purely randomized data structure, and a deterministic one.
I think the important thing to bring out here is the worst case is much worse than order log n, OK?
Cool.
Good.
Thanks for those questions.
And so what we have here now is an insert algorithm that could make things look pretty messy.
I'm going to leave the insert up here, and that, of course, is part of that.
Now, for the rest of the lecture we're going to talk about why skip lists are good.
And we're going to justify this randomized data structure and show lots of nice results with respect to the expectation on the number of levels, expectation on the number of moves in a search, regardless of what items you're inserting and deleting.
One last thing.
To delete an item, you just delete it.
You find it, search, and delete at all levels.
So you can't leave it in any of the levels.
So you find it, and you have to have the pointers set up properly-- move the previous pointer over to the next one, et cetera, et cetera.
We won't get into that here, but you have to do the delete at every level.
Yeah, question
So what happens if you inserted 10s and you flip off a tail?
So that's like your first element is not going to go up all the way, and then have you do search
So typically what happens is you need to have a minus infinity here.
And that's a good point.
It's a corner case.
You have to have a minus infinity that goes up all the way.
Good question.
So the question was what happens if I had something less than 14 and I inserted it?
Well, that doesn't happen because nothing is less than minus infinity, and that goes up all the way.
But thanks for bringing it up.
And so we're going to do a little warm-up Lemma.
I don't know if you've ever heard these two terms in juxtaposition like this-- warm up and Lemma.
But here you go, your first warm-up Lemma.
I guess you'd never have a warm-up theorem.
It's a warm-up Lemma for this theorem, which is going to take a while to prove.
This comes down to trying to get a sense of how many levels you're going to have from a probabilistic standpoint.
The number of levels in an n element skip list is order log n. And I'm going to now define the term with high probability.
So what does this mean exactly?
Well, what this means is order log n is something like c log n plus a constant.
Let's ignore the constant and let's stick with c log n. And with high probability is a probability that is really a function of n and alpha.
And you have this inverse polynomial relationship in the sense that obviously as n grows here, an alpha-- we'll assume that alpha is greater than the 1-- you are going to get a decrease in this quantity.
So this is going to get closer and closer to 1 as n grows.
So that's the difference between with high probability and just sort of giving you an expectation number where you have no such guarantees.
What is interesting about this is that as n grows, you're going to get a higher and higher probability.
And this constant c is going to be related to alpha.
That's the other thing that's interesting about this.
So it's like saying-- and you can kind of say this for using Chernoff bounds that we'll get to in a few minutes, even for expectation as well.
But what this says is that if, for example, c doubled, then you are saying that your number of levels is order 4 log n. I mean I understand that that doesn't make too much sense, but it's less than or equal to 4 log n plus a constant.
And that 4 is going to get reflected in the alpha here.
When the 4 goes from 4 to 8, the alpha increases.
So the more room that you have with respect to this constant, the higher the probability.
It becomes an overwhelming probability that you're going to be within those number of levels.
So maybe there's an 80% probability that you're within 2 log n. But there's a 99.99999% probability that you're within 4 log n, and so on and so forth.
So that's the kind of thing that with the high probability analysis tells you explicitly.
And so you can do that, you can do this analysis fairly straightforwardly.
And let me do that on a different board.
Let me go ahead and do that over here.
Actually, I don't really need this.
So let's do that over here.
And so this is our first with high probability analysis.
And I want to prove that warm-up Lemma.
So usually what you do here is you look at the failure probability.
So with high probability is typically something that looks like 1 minus 1 divided by n raised to alpha.
And this part here is the failure probability.
And that's typically what you analyze and what we're going to do today.
So the failure probability is that it's not less than c log n levels, is the complement of what we just looked at, which is the probability that it's strictly greater than c log n levels.
And that's the probability that some element gets promoted greater than c log n times.
So why would you have more than c log n levels?
It's essentially because you inserted something and that element got promoted strictly greater than c log n times, which obviously implies that you had the sequence of heads, and we'll get to that in just a second.
But before we go to that step of figuring out exactly what's going on here as to why this got promoted and what the probability of each promotion is, what I have here is I have a sequence of inserts potentially that I have to analyze.
And in general, when I have an n element list, I'm going to assume that each of these elements got inserted into the list at some point.
So I've had n inserts.
And we just look at the case where you have n inserts, you could have deletes, and so you could have more inserts, but it won't really change anything.
You have n inserts corresponding to each of these elements, and one of those n elements got promoted in this failure case greater than c log n times.
That's essentially what's happened here.
And so you don't know which one, but you can typically do this in with high probability analysis because the probabilities are so small and they're inverse polynomials, polynomials like n raised to alpha.
You can use what's called the union bound that I'm sure you've used before in some context or the other.
And you essentially say that this is less than or equal to the probability that a particular element x.
So you just pick an element, arbitrary element x, but you pick one.
Gets promoted greater than c log n times.
So you have a small probability.
You have no idea whether these events are independent or not.
The union bound doesn't care about it.
It's like saying you've got a 0.001 probability that any of these elements could get promoted greater than c log n times, and there's 10 of those elements.
You don't know whether they're independent events or not, but you can certainly use the union bound that says the overall failure probability is going to be less than or equal to n equals 10, in my example, times that 0.001.
That's basically it.
Now you can go off and say, what does it mean for an element to get promoted?
What actually has to happen for an element to get promoted?
And you have n times 1 over 2, because you're flipping a fair coin, and you are getting a c log n heads here.
You flip and you get one promotion.
There's two levels associated with a promotion, the level you came from and the level you went to.
And so a promotion is a move, so you're going to have one more level.
If you count levels, then you have the number of promotions, right?
That's just simply corresponds to taking this 1/2 and raising it to c log n, because that's essentially the number of promotions you have.
And you got n 1/2 c log n, and what does that turn into?
What is n times 1/2 c log n?
1 over 2 raised to log n would give you?
2 raised to log ns?
Is n, right?
So you got n divided by n raised to c, which is 1 divided by n raised to c minus 1, which is 1 divided by n raised to alpha where alpha is c minus 1.
So that's it.
That's our first with high probability analysis.
Not too hard.
What I've done is done exactly what I just told you that the notion of with high probability is.
You have a failure probability that is related.
Inverse polynomial and the degree of the polynomial alpha is related to c. And so that's what I have out there, but c equals-- what did it have?
Alpha equals c minus 1 or c equals alpha plus 1.
So what I've done here is done an analysis that tells you with high probability how many levels I'm going to have given my insert algorithm.
So this is the first part of what we'd like to show.
This just tells us how big this skip list is going to grow vertically.
It doesn't tell us anything about the structure of the list internally as to whether the randomization is going to cause that pretty structure that you see up here to be completely messed up to the point where we don't get order log n search complexity, because we are spending way too much time let's say on the bottom list or the list just above the bottom list, et cetera.
So we need to get a sense of how the structure corresponding to the skip list, whether it's going to look somewhat uniform or not.
We have to categorize that, and the only way we're going to characterize that is by analyzing search and counting the number of moves that a search makes.
And the reason it's more complicated than what you see up there is that in a search, as you can see, you're going to be moving at different levels.
You're going to be moving at the top level.
Maybe at relatively small number of moves, you're going to pop down one, move a few moves at that level, pop down, et cetera, et cetera.
So there's a lot of things going on in search which happen at different levels, and the total cost is going to have to be all of the moves.
So we're going to think about all of the moves-- up moves, down moves, and add them all up.
They all have to be order log n with high probability.
There's no getting around that because each of them costs you.
So that's the thing that we'll spend the next 20 minutes on.
And the theorem that we like to prove for search is that-- this is what I just said-- any search in an n element skip list costs order log n w.h.p.
So it doesn't matter how this skip list looks.
There's n elements, they got inserted using the insert algorithm-- that's important to know if you're going to have to use that.
And when I do a search for an element, it may be in there, it may not be in there.
Doesn't really matter.
We'll assume a successful search.
That is going to cost me order log n with high probability.
And the cool idea here in terms of analyzing the search in order to figure out how we're going to add up all of these moves is we're going to analyze the search backwards.
So that's a cool idea.
So what does that mean exactly?
Well, what that means is that we're going to think about this b search, which think of it as the backward search, starts-- it actually ends, so that's what I'm writing in brackets here, at the node in the bottom list.
So we're assuming a successful search, as I mentioned before.
Otherwise, the point would just be in between two members.
You know that it's not in there because you're looking for 67 and you see 66 to your left and 72 to your right.
So either way it works, but keep in mind that it's a successful search because it just makes things a little bit easier.
Now, at each node that we visit, what we're going to do is we're going to say that if the node was not promoted higher, then what actually happened here was that when you inserted that particular element, you got a tails.
Because otherwise you would have gotten a heads, that element would have been promoted higher.
Then you go-- and that really means that you came from the left-hand side, so you make a left move.
Now, search of course makes down moves and right moves, but this is a backward search so it's going to make left moves and up moves.
What else do I have here?
Running out of room, so let me-- let's continue with that.
All right.
And now the case is if the node was promoted higher, that means we got heads here in that particular insertion.
Then we go, and that means that during the search we came from upstairs.
And then lastly, we stop, which means we start when we reach the top level or minus infinity if we go all the way back.
So that's it.
A lot of writing here, but this should make things clear.
So let's say that we're searching for 66.
I want to trace through what the backwards path would look like, and keep that code in mind as I do this.
So I'm searching for 66, and obviously, we know how to find it.
We've done that.
But let's go backwards as to what exactly happened when we look for 66.
When we look for 66, right at this point when you see 66, where would you have come from?
[INAUDIBLE]
You'd have come from the top.
And so if you go look at what happens here, the node when it got inserted was promoted one level.
So that means that you would go up top in the backward search first.
Your first move would be going up like that.
Now, if there's a 66 up there, you would go up one more.
But there's not, so you go left.
You go to 50.
And when you have a 50 up here, would you stay on this level?
No
No.
You'd go up to 50 because the first chance you get you want to get up to the higher levels.
And again, this 50 was promoted so you go up there, and you go to 14, and pretty much that's the end of that.
So this would look like you go like that, you have an up move, then you have a left move-- different colors here would be good-- then you have an up move, and a left, and then an up.
So that's our backward search.
And it's not that complicated, hopefully.
If you're looking for 66 or 59, you do that.
So it's much more natural, and you just need to flip it.
Why am I doing all this?
Well, the reason I'm doing all this is that I have to do some bounding of the moves, and I know that the moves that correspond to the up moves are probabilistic in the sense that the reason I'm making them is because I flipped heads at some point.
So all of this is going to turn into counting how many coin flips come out heads in a long stream of coin flips.
So that's what this backward search is going to allow us to do.
And that crucial thing is what we'll look at next.
So the analysis itself is a bit painful, but there's a bunch of algebra.
But what I want to do is to make sure that you get the high level picture, number one, and the insights as to why the expected value or the with high probability value is going to be order log n. But the key is the strategy.
So we're going to go off and we're going to prove this theorem.
Our backward search makes up moves and left moves.
We know that.
Each with probability 1/2.
And the reason for that is when you go up is because you got a heads, and if you didn't get a heads in you got a tails, that meant you go left.
Because of the previous element, every time you're passing these elements that are inserted, and they were inserted by flipping coins.
So that's key point number one.
All of that, if you look at what happens here when I drew this out, you got heads here and you got tails there.
So each of those things for a fair coin is happening with probability 1/2.
And it's all about coin flips here.
Now, the number of moves going up is less than the number of levels-- the number of levels is one more than that.
And we've shown that that's c log n with high probability by the warm-up Lemma.
That's what this just did.
The number of up moves-- I mean you can't go off the list here.
This list is now you're not inserting anymore, you're doing a search.
So it's not like you're going to be adding levels or anything like that.
So the number of up moves we've taken care of.
So this last thing here which I'm going to write out here is the key observation, which is going to make the whole analysis possible.
And so this last thing it says that the total number of moves-- so now the total number of moves has to include, obviously, the up moves and the left moves, and there's no other kind.
The total number of moves is going to correspond to the number of moves till you get c log n up moves.
So what does that mean?
There's some sequence of heads and tails that I'm getting, each of them with probability 1/2.
Every time that I got a heads, I moved up a level.
The fact of the matter is that I can't get more than c log n heads because I'm going to run out of levels.
That's it.
I'm going to run out of room vertically if I keep popping up and keep doing up moves.
So at that point I'm forced to go left.
Maybe I'm going left in the middle there when I still had a chance to go up.
That corresponds to getting a tails as opposed to a heads.
But I can limit the total number of moves from a probabilistic standpoint by saying during that sequence of coin flips I only have a certain number of heads that I could have possibly gotten.
Because if I got more heads than that, I would be up top.
I'd be out of the skip list, and that doesn't work.
So the total number of moves is the number of moves till you get c log n up moves, which essentially corresponds to-- now, forget about skip lists for a second.
Our claim is the total number of moves is the number of coin flips, so these are the same, because every move corresponds to a coin flip.
Until-- it's a fair coin, probability 1/2-- until c log n heads have been obtained.
So the number of coin flips until c log n heads is the total number of moves.
This equals that.
And what we now want to show, if you believe that, and hopefully you do because the argument is simply that you run out of levels, that this is order log n w.h.p.
That's why it's a claim.
So the observation is that the number of coin flips, as you flip a fair coin, until you get c log n heads will give you the number of moves in your search, total number of moves in your search.
It includes the up moves as well as the left moves.
And now what we have to show is that that is going to be order log n with high probability.
OK?
And then once you do that you've done two things.
You've bounded the number of levels in the skip list to be order log n with high probability.
And you've said the number of moves in the search is order log n with high probability assuming that the number of levels is c log n, obviously.
So it's not that the bottom one subsumes the top one.
It's the last thing to keep in mind as we get all of these items out of the way.
This assumes that there are less than or equal to c log n levels.
That's the only reason why I could make an argument that I've run out of levels.
So if I have this event A here-- if I call this event A, and I have this event B, what I really want is-- I've shown you that event A happens with high probability.
That's the warm-up Lemma.
I need to show you that event B happens with high probability.
And then I have to show you that event A and event B happen with high probability, because I need both.
Any questions?
We're stopping a minute here.
The rest of the analysis, a bunch of algebra, we'll get through it, you can look at the notes.
This is the key point.
If you got this, you got it.
Yeah
Can you just say that because the probability of drawing an up move instead of a left move is 1/2, that the expected number of left moves should be equal to the number of up moves, [INAUDIBLE] bound the up moves?
So the argument is that since you have 1/2, can you simply say that the expected number of left moves is going to be the same as the same as the up moves?
You can make arguments about expectation.
You can say that at any level, the number of left moves that you're going to have is going to be two in expectation.
It's not going to give you your with high probability proof.
It's not going to relate that to the 1 divided by n raised to alpha.
But I will tell you that if you just wanted to show expectation for search is order log n, you won't have to jump through all of these hoops.
At some level you'll be making the assumptions that I've made explicit here through my observations when you do that expectation.
So if you really want to write a precise proof of expected value for search complexity, you would have to do a lot of the things that I'm doing here.
I'm not saying you waved your hands.
You did not.
But it needed more to than what you just said.
OK?
So this is pretty much what the analysis is.
With high probability analysis we bounded the vertical, we bounded the number of moves.
Assuming the vertical was bounded, we got the result for the number of moves.
So both of those happen with high probability.
You got your result, which is the theorem that we have somewhere.
Woah, did I erase the theorem?
[INAUDIBLE]
It's somewhere.
All right.
Good.
So let's do what we can with respect to showing this theorem.
There's a couple ways that you could prove this.
There's a way that you could use a Chernoff bound.
And this is kind of a cool result that I think is worth knowing.
I don't know if you've seen this, but this is a seminal theorem by Chernoff that says if you have a random variable representing the total number of tails, let's say-- it could be heads as well-- in a series of m-- not n, m-- independent coin flips where each flip has a probability p of coming up heads, then for all r greater than 0, we have this beautiful result that says the probability that y, which is a random variable-- a particular instance when you evaluate it-- that it is larger than the expectation by r is bounded.
So just a beautiful result that says here's a random variable that corresponds to flipping a coin.
I'm going to flip this a bunch of times, and I know what the expectation is.
If it's a fair coin of 1/2, then I'm going to get m over 2-- expected number of heads is going to be m over 2.
Expected number of tails is going to be m over 2.
If it's p, then obviously it's a little bit different-- p times m. But what I have here is if you tell me what the probability is that I'm 10 away from the expectation and that would imply that r is 10, then that is bounded by e raised to minus 2 times 10 square divided by m. So that's Chernoff's bound.
And you can see how this relates to our with high probability analysis.
Because our with high probability analysis is exactly this.
This is the hammer that you can use to do with high probability analysis.
Because this tells you as you get further and further away from the average or you get further and further away from the expectation, what the probability is that you're going to be so far away.
What is the probability that in 100 coin flips that are fair, you get 50 heads?
It's a reasonably large number because the expected value corresponds to 50.
So r is 0.
So that just says this is a-- well, it doesn't tell you much because this says it's less than or equal to 1.
That's all it's says.
But if you had 75, what are the probability that you get 75 heads when you flip a coin 100 times?
Then e of y for a fair coin would be 50, r would be 25, and you'd go off and you could do the math for that.
So it's a beautiful relationship that tells you how the probabilities change as your random variable value is further and further away from the expectation.
And you can imagine that this is going to be very useful in showing our with high probability result.
And I think what I have time for is just to give you a sense of how this result works out-- I'm not going to do the algebra.
I don't think it's worth it to write all of this on the board when you can read it in the notes.
But the bottom line is we're going to show this little Lemma that says for any c, invoking this Chernoff bound, there's a constant d, such that with high probability, the number of heads in flipping d log n. So I have a new constant here.
d log n fair coins, or a single fair coin, d log n times, assuming independence, is at least c log n. So what does this say?
A lot of words.
It just says, hey, you want an order log n bound here eventually.
The beauty of order log n is that there's a constant in there that you control.
That constant is d. So you tell me that c log n is 50.
So c log n is 50.
Then what I'm going to do is I'm going to say something like, well, if I flip a coin 1,000 times, then I'm going to have an overwhelming probability that I'm going to get 50 heads.
And that's it.
That's what the Lemma says.
It says tell me what c log n is.
Give me that value.
And I will find you a d, such that by invoking Chernoff, I'm going to show you an overwhelming probability that for that d you're going to get at least c log n heads.
So everybody buy that?
Make sense from what you see up there?
Yup?
So this essentially can be shown-- it turns out that what you have to do is-- and you don't have to choose 8, but you can choose d equals 8c.
Just choose d equals 8c and you'll see the algebra in the notes corresponding to what each of these values are.
So e of y, just to tell you, would be m over 2.
You're flipping m coins, fair coin with probability 1/2.
So you got m over 2.
And then the last thing that I'll tell you is that what you want in terms of invoking that, you want r-- remember we were talking about tails here-- so r is going to be d log n minus c log n. So you just invoke Chernoff with e of y equals m over 2.
And what you're saying here is you want c log n heads.
You want to make sure you get c log n heads, which means that the number of tails is going to be d log n minus c log n. And typically we analyze failure probability, so what this is is this is going to be a tiny number.
So the failure is when you get fewer than c log n heads.
So the failure is when you get fewer than c log n heads.
And so that means that you're getting more than d log n minus c log n tails as you're flipping this coin.
Fewer than c log n heads means you're getting at least d log n minus c log n tails.
So that's why this is your r here.
And then when your r gets that large, and you can play around with the d and the c and choose d equals 8c, you realize that this is going to be a minuscule probability.
And you can turn that around to a polynomial-- again, a little bit of algebra.
But you can show this result on here that says that the number of coin flips until c log n heads is order log n with high probability by appropriately choosing the constant d to be some time/number over c. So I'll let you do that algebra.
But this one last thing that-- we're not quite done.
So you thought we were done, but we're not quite done.
And why is it that we're not quite done?
Real quick question worth five Frisbees.
Why is it that we're not quite done?
What did I say?
I have done event A and event B, right?
[INAUDIBLE]
I haven't done the last thing which is to show that probability of event A-- this is with high probability happens-- and I need to show that probability of event A and event B happens-- or this is with high probability.
Or I should just say event A and event B happen with high probability.
And you can see that.
It turns out it's pretty straightforward, but you got the gist of it.
Thanks for being so patient.
And there you go guys.
Woah.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
Today we're going to continue the theme of randomization and data structures.
Last time we saw skip lists.
Skip lists solve the predecessor-successor problem.
You can search for an item and if it's not there, you get the closest item on either side in log n with high probability.
But we already knew how to do that deterministically.
Today we're going to solve a slightly different problem, the dictionary problem with hash tables.
Something you already think you know.
But we're going to show you how much you didn't know.
But after today you will know.
And we're going to get constant time and not with high probability.
That's hard.
But we'll do constant expected time.
So that's in some sense better.
It's going to solve a weaker problem.
But we're going to get tighter bound constant instead of logarithmic.
So for starters let me remind you what problem we're solving and the basics of hashing which you learned in 6006.
I'm going to give this problem a name because it's important and we often forget to distinguish between two types of things.
This is kind of an old term, but I would call this an abstract data type.
This is just the problem specification of what you're trying to do.
You might call this an interface or something.
This is the problem statement versus the data structure is how you actually solve it.
The hash tables are the data structure.
The dictionary is the problem or the abstract data type.
So what we're trying to do today, as in most data structures, is maintain a dynamic set of items.
And here I'm going to distinguish between the items and their keys.
Each item has a key.
And normally you'd think of there also being a value like in Python.
But we're just worrying about the keys and moving the items around.
And we want to support three operations.
We want to be able to insert an item, delete an item, and search for an item.
But search is going to be different from what we know from AVL trees or skip lists or even Venom [INAUDIBLE] That was a predecessor-successor search.
Here we just want to know-- sorry, your not searching for an item.
Usually you're searching for just a key-- here you just want to know is there any item with that key, and return it.
This is often called an exact search because if the key is not in there, you learn absolutely nothing.
You can't find the nearest key.
And for whatever reason this is called a dictionary problem though it's unlike a real dictionary.
Usually when you search for a word you do find its neighbors.
Here we're just going to either-- if the key's there we find that, otherwise not.
And this is exactly what a Python dictionary implements.
So I guess that's why Python dictionaries are called dicts.
So today I'm going to assume all items have distinct keys.
So in the insertion I will assume key is not already in the table.
With a little bit of work, you can allow inserting an item with an existing key, and you just overwrite that existing item.
But I don't want to worry about that here.
So we could, of course, solve this using an AVL tree in log n time.
But our goal is to do better because it's an easier problem.
And I'm going to remind you of the simplest way you learn to do this which was hashing with chaining in 006.
And the catch is you didn't really analyze this in 006.
So we're going make a constant time per operation.
It's going to be expected or something and linear space.
And remember the variables we care about, there's u, n, and m. So u is the size of the universe.
This is the all possible keys.
The space of all possible keys.
n is the size of the set your currently storing.
So that's the number of items or keys currently in the data structure.
And then m is the size of your table.
So say it's the number of slots in the table.
So you remember the picture.
You have a table of slots.
Let's say 0 to m minus 1.
Each of them is a pointer to a linked list.
And if you have, let's say over here is your universe of all possible keys, then we have a hash function which maps each universe item into one of these slots.
And then the linked list here is storing all of the items that hash to that slot.
So we have a hash function which maps the universe.
I'm going to assume the universe has already been mapped into integers 0 to u minus 1.
And it maps to slots.
And when we do hashing with chaining, I think I mentioned this last week, the bounds you get, we achieve a bound of 1 plus alpha where alpha is the load factor n/m.
The average number of items you'd expect to hash to a slot is the number of items divided by the number of slots.
OK. And you proved this in 6006 but you assumed something called simple uniform hashing.
Simple uniform hashing is an assumption, I think invented for CLRS.
It makes the analysis very simple, but it's also basically cheating.
So today our goal is to not cheat.
It's nice as a warm up.
But we don't like cheating.
So you may recall the assumption is about the hash function.
You want a good hash function.
And good means this.
I want the probability of two distinct keys mapping to the same slot to be 1/m if there are m slots.
If everything was completely random, if h was basically choosing a random number for every key, then that's what we would expect to happen.
So this is like the idealized scenario.
Now, we can't have a hash function could choosing a random number for every key because it has to choose the same value if you give it the same key.
So it has to be some kind of deterministic strategy or at least repeatable strategy where if you plug in the same key you get the same thing.
So really what this assumption is saying is that the key's that you give are in some sense random.
If I give you random keys and I have not-too-crazy hash function then this will be true.
But I don't like assuming anything about the keys maybe.
I want my keys to be worst case maybe.
There are lots of examples in the real world where you apply some hash function and it turns out your data has some very particular structure.
And if you choose a bad hash function, then your hash table gets really, really slow.
Maybe everything hashes to the same slot.
Or say you take-- well yeah, there are lots of examples of that.
We want to avoid that.
After today you will know how to achieve constant expected time no matter what your keys are, for worst case keys.
But it's going to take some work to do that.
So this assumption requires assuming that the keys are random.
And this is what we would call an average case analysis.
You might think that average case analysis is necessary for randomized algorithms, but that's not true.
And we saw that last week with quicksort.
Quicksort, if you say I will always choose a of 1 to be my partition element, that's what the textbook calls basic quicksort, then for an average input that will do really well.
If you have a uniform random permutation of items and you sort with the method of always choosing the first item as your partition, then that will be n log n on average if your data is average.
But we saw we could avoid that assumption by choosing a random pivot.
If you choose a random pivot, then you don't need to assume anything about the input.
You just need to assume that the pivots are random.
So it's a big difference between assuming your inputs are random versus assuming your coin flips are random.
It's pretty reasonable to assume you can flip coins.
If you've got enough dexterity in your thumb then you can do it.
But it's not so reasonable to assume that your input is random.
So we'd like to avoid average case analysis whenever we can, and that's the goal of today.
So what you saw in 006 was essentially assuming the inputs are random.
We're going to get rid of that unreasonable assumption today.
So that's, in some sense, review from 006.
I'm going to take a brief pause and tell you about the etymology of the word hash in case you're curious.
Hash is an English word since the 1650's, so it's pretty old.
It means literally cut into small pieces.
It's usually used in a culinary sense, like these days you have corned beef hash or something.
I'll put the definition over here.
It comes from French, hacher, which means to chop up.
You know it in English from the word hatchet.
So it's the same derivation.
And it comes from old French-- I don't actually know whether that's "hash-ay" or "hash" but-- which means axe.
So you can see the derivation.
If you look this up in OED or pick your favorite dictionary or even Google, that's what you find.
But in fact there's a new prevailing theory that in fact hash comes from another language which is Vulcan, la'ash, I mean you can see the derivation right?
Actually means axe.
So maybe French got it from Vulcan or vice versa but I think that's pretty clear.
Live long and prosper, and farewell to Spock.
Sad news of last week.
So enough about hashing.
We'll come back to that in a little bit.
But hash functions essentially take up this idea of taking your key, chopping up into pieces, and mixing it like in a good dish.
All right, so we're going to cover two ways to get strong constant time bounds.
Probably the most useful one is called universal hashing.
We'll spend most of our time on that.
But the theoretically cooler one is called perfect hashing.
Universal hashing, we're going to guarantee there are very few conflicts in expectation.
Perfect hashing , we're going to guarantee there are zero conflicts.
The catch is, at least in its obvious form, it only works for static sets.
If you forbid, insert, and delete and just want to do search, then perfect hashing is a good method.
So like if you're actually storing a dictionary, like the OED, English doesn't change that quickly.
So you can afford to recompute your data structure whenever you release a new edition.
But let's start with universal hashing.
This is a nice powerful technique.
It works for dynamic data.
Insert, delete, and search will be constant expected time with no assumptions about the input.
So it will not be average case.
It's in some sense worse case but randomized.
So the idea is we need to do something random.
If you just say, well, I choose one hash function once and for all, and I use that for my table, OK maybe my table doubles in size and I change the hash function.
But there's no randomness there.
We need to introduce randomness somehow into this data structure.
And the way we're going to do that is in how we choose the hash function.
We're going to choose our hash function randomly from some set of hash functions.
Call it h. This is going to be a universal hash family.
We're going to imagine there are many possible hash functions we could choose.
If we choose one of them uniformly at random, that's a random choice.
And that randomness is going to be enough that we no longer need to assume anything about the keys.
So for that to work, we need some assumption about h. Maybe it's just a set of one hash function.
That wouldn't add much randomness.
Two also would not add much randomness.
We need a lot of them.
And so we're going to require H to have this property.
And we're going to call it the property universality.
Generally you would call it a universal hash family.
Just a set of hash functions.
What we want is that-- so we're choosing our hash function h from H. And among those choices we want the probability that two keys hash to the same value to be small.
I'll say-- and this is very similar looking to simple uniform hashing.
Looks almost the same here except I switched from k1 and k2 to k and k', but same thing.
But what we're taking the probability over, what we're assuming is random is different.
Here we're assuming k1 and k2 a are because h was fixed.
This was an assumption about the inputs.
Over here we're thinking of k and k' as being fixed.
This has to work for every pair of distinct keys.
And the probability we're considering is the distribution of h. So we're trying all the different h's Or we're trying little h uniformly at random.
We want the probability that a random h makes k and k' collide to be at most 1/m.
The other difference is we switch from equals to at most.
I mean less would be better.
And there are ways to make it less for a couple pairs but it doesn't really matter.
But of course anything less than or equal to 1/m will be just as good.
So this is an assumption about H. We'll see how to achieve this assumption in a little bit.
Let me first prove to you that this is enough.
It's going to be basically the same as the 006 analysis.
But it's worth repeating just so we are sure everything's OK. And so I can be more precise about what we're assuming.
The key difference between this theorem and the 006 theorem is we get to make no assumptions about the keys.
They are arbitrary.
You get to choose them however you want.
But then I choose a random hash function.
The hash function cannot depend on these keys.
But it's going to be random.
And I choose the hash function after you choose the keys.
That's important.
So we're going to choose a random h and H. And we're assuming H is universal.
Then the expected number of keys in a slot among those n keys is at most 1 plus alpha.
Alpha is n/m.
So this is exactly what we had over here.
Here we're talking about time bound.
But the time bound followed because the length of each chain was expected to be 1 plus alpha.
And here the expectation is over the choice of h. Not assuming anything about the keys.
So let's prove this theorem.
It's pretty easy.
But I'm going to introduce some analysis techniques that we will use for more interesting things.
So let's give the keys a name.
I'll just call them-- I'll be lazy.
Use k1 up to kn.
And I just want to compute that expectation.
So I want to compute let's say the number of keys colliding with one of those keys, let's say ki.
So this is of course the size of the slot that ki happens to go.
This is going to work for all i.
And so if I can say that this is at most 1/alpha for each i, then I have my theorem.
Just another way to talk about it.
Now the number of keys colliding with ki, here's a general trick, whenever you want to count something in expectation, a very helpful tool is indicator random variables.
Let's name all of the different events that we want to count.
And then we're basically summing those variables.
So I'm going to say-- I'm going to use I ij to be an indicator random variable.
It's going to be 1 or 0.
1 if hash function of ki equals the hash function of kj.
So there's a collision between ki and kj j and 0 if they hash to different slots.
Now this is, it's a random variable because it depends on h and h is a random thing.
ki and kj are not random.
They're given to you.
And then I want to know when does h back those two keys to the same slot.
And so this number is really just the sum of Iij over all j.
This is the same thing.
The number in here is the sum for j not equal to i of Iij.
Because we get a 1 every time they collide, zero otherwise.
So that counts how many collide.
Once we have it in this notation, we can use all the great dilemmas and theorems about in this case, E, expectation.
What should I use here?
What?
What's a good-- how can I simplify this formula?
The linearity of expectation
The linearity of expectation.
Thank you.
If you don't know all these things, read the probability appendix in the textbook.
So we want to talk about expectation of the simplest thing possible.
So linearity let's us put the E inside the sum without losing anything.
Now the expectation of an indicator random variable is pretty simple because the zeros don't contribute to the expectation.
The 1's contribute 1.
So this is the same thing as just the probability of this being 1.
So we get sum of j9 equal to I of the probability that Iij equals 1.
And the probability that Iij equals 1, well, that's the probability that this happens.
And what's the probability that that happens?
At most 1/m our universality.
So I'm going to-- I'll write it out.
This is sum j not equal to I. Probability that h maps ki and kj to the same slot.
So that's the definition of Iij.
And this is at most sum j not equal to i of 1/m by universality.
So here's where we're using it.
And sum of j not equal to I, well that's basically n. But I made a mistake here.
Slightly off.
From here-- yeah.
So this line is wrong.
Sorry.
Let me fix it.
Because this assumption only works when the keys are distinct.
So in fact-- how did I get j-- yeah.
, Yeah, sorry.
This should have been this-- actually everything I said is true, but if you want to count the number of keys-- I really wanted to count the total number of keys that hash to the same place as ki.
So there's one more which is ki itself.
Always hashes to wherever ki hashes.
So I did a summation j not equal i but I should also have a plus Iii-- captain.
So there's the case when I hashing to the same place which of course is always going to happen so you get basically plus 1 everywhere.
So that makes me happier because then I actually get with the theorem said which is 1 plus alpha.
There is always going to be the one guy hashing there when I assume that ki hashed to wherever it does.
So this tells you that if we could find a universal hash family, then we're guaranteed insert, delete, and search cost order 1 plus alpha in expectation.
And the expectation is only over the choice of h, not over the inputs.
I think I've stressed that enough times.
But the remaining question is can we actually design a universal hash family?
Are there any universal hash families?
Yes, as you might expect there are.
Otherwise this wouldn't be very interesting.
Let me give you an example of a bad universal hash family.
Sort of an oxymoron but it's possible.
Bad.
Here's a hash family that's universal.
h is the set of all hash functions.
h from 0,1 to u minus 1.
This is what's normally called uniform hashing.
It makes analysis really easy because you get to assume-- I mean this says ahead of time for every universe item, I'm going to choose a random slot to put it.
And then I'll just remember that.
And so whenever you give me the key, I'll just map it by h. And I get a consistent slot and definitely it's universal.
What's bad about this hash function?
Many things but--
[INAUDIBLE] That's just as hard as the problem I'm solving
Sort of.
I'm begging the question that it's just as hard as the problem I'm solving.
And what, algorithmically, what goes wrong here?
There are two things I guess.
Yeah?
It's not deterministic?
It's not deterministic.
That's OK because we're allowing randomization in this algorithm.
So I mean how I would compute this is I would do a four loop over all universe items.
And I assume I have a way to generate a random number between 0 and m minus 1.
That's legitimate.
But there's something bad about that algorithm
It's not consistent
Not consistent?
It is consistent if I precompute for every universe item where to map it.
That's good.
So all these things are actually
It takes too much time and space
It takes too much time and space.
Yeah.
That's the bad thing.
It's hard to isolate in a bad thing what is so bad about it.
But we need u time to compute all those random numbers.
And we need u space to store that hash function.
In order to get to the consistency we have to-- Oops.
Good catch.
In order to get consistency, we need to keep track of all those hash function values.
And that's not good.
You could try to not store them all, you know, use a hash table.
But you can't use a hash table to store a hash function.
That would be-- that would be infinite recursion.
So but at least they're out there.
So the challenge is to find an efficient hash family that doesn't take much space to store and doesn't take much time to compute.
OK, we're allowing randomness.
But we don't want to much randomness.
We can't afford u units of time of randomness.
I mean u could be huge.
We're only doing n operations probably on this hash table.
u could be way bigger than n. We don't want to have to precompute this giant table and then use it for like five steps.
It would be really, really slow even amortized.
So here's one that I will analyze.
And there's another one in the textbook which I'll mention.
This one's a little bit simpler to analyze.
We're going to need a little bit of number theory, just prime numbers.
And you've probably heard of the idea of your hash table size being prime.
Here you'll see why that's useful, at least for this family.
You don't always need primality, but it's going to make this family work.
So I'm going to assume that my table size is prime.
Now really my table size is doubling, so that's a little awkward.
But luckily there are algorithms given a number to find a nearby prime number.
We're not going to cover that here, but that's an algorithmic number theory thing.
And in polylogarithmic time, I guess you can find a nearby prime number.
So you want it to be a power of 2.
And you'll just look around for nearby prime numbers.
And then we have a prime that's about the same size so that will work just as well from a table doubling perspective.
Then furthermore, for convenience, I'm going to assume that u is an integer power of m. I want my universe to be a power of that prime.
I mean, if it isn't, just make u a little bigger.
It's OK if u gets bigger as long as it covers all of the same items.
Now once I view my universe as a power of the table size, a natural thing to do is take my universe items, to take my input integers, and think of them in base m. So that's what I'm going to do.
I'm going to view a key k in base m. Whenever I have a key, I can think of it as a vector of subkeys, k1 up to kr minus 1.
There are digits in base m because of this relation.
And I don't even care which is the least significant and which is the most significant.
That won't matter so whatever, whichever order you want to think of it.
And each of the ki's here I guess is between 0 and m minus 1.
So far so good.
So with this perspective, the base m perspective, I can define a dot product hash function as follows.
It's going to be parametrized by another key, I'll call it a, which we can think of again as a vector.
I want to define h sub a of k. So this is parametrized by a, but it's a function of a given key k as the dot product of those two vectors mod m. So remember dot products are just the sum from i equals 0 to r minus 1 of a1 times ki.
I want to do all of that modulo m. We'll worry about how long this takes to compute in a moment I guess.
Maybe very soon.
But the hash family h is just all of these ha's for all possible choices of a. a was a key so it comes from the universe u.
And so what that means is to do universal hashing, I want to choose one of these ha's uniformly at random.
How do I do that?
I just choose a uniformly at random.
Pretty easy.
It's one random value from one random key.
So that should take constant time and constant space to store one number.
In general we're in a world called the Word RAM model.
This is actually-- I guess m stands for model so I shouldn't write model.
Random access machine which you may have heard.
The word RAM assumes that in general we're manipulating integers.
And the integers fit in a word.
And the computational assumption is that manipulating a constant number of words and doing essentially any operation you want on constant number of words takes constant time.
And the other part of the word RAM model is to assume that the things you care about fit in a word.
Say individual data values, here we're talking about keys, fit in a word.
This is what you need to assume in [INAUDIBLE] that you can compute high of x in constant time or low of x in constant time.
Here I'm going to use it to assume that we can compute h sub a of k in constant time.
In practice this would be done by implementing this computation, this dot product computation, in hardware.
And the reason a 64-bit edition on a modern processor or a 32-bit on most phones takes constant time is because there's hardware that's designed to do that really fast.
And in general we're assuming that the things we care about fit in a single word.
And we're assuming random access and that we can have a raise.
That's what we need in order to store a table.
And same thing in [INAUDIBLE], we needed to assume we had a raise.
And I think this operation is actually pretty-- exists in Intel architectures in some form.
But it's certainly not a normal operation.
If you're going to do this explicitly, adding up and multiplying things this would be r is the log base m of u, so it's kind of logish time.
Maybe I'll mention another hash family that's more obviously computable.
But I won't analyze here.
It's analyzed in the textbook.
So if you're curious you can check it out there.
Let's call this just another.
It's a bit weird because it has two mods.
You take mod p and then mod m. But the main computation is very simple.
You choose a uniformly random value a.
You multiply it by your key in usual binary multiplication instead of dot product.
And then you add another uniformly random key.
This is also universal.
So H is hab for all a and b that are keys.
So if you're not happy with this assumption that you can compute this in constant time, you should be happy with this assumption.
If you believe in addition and multiplication and division being constant time, then this will be constant time.
So both of these families are universal.
I'm going to prove that this one is universal because it's a little bit easier.
Yeah?
Is this p a choice that you made?
OK, right.
What is p?
P just has to be bigger than m, and it should be prime.
It's not random.
You can just choose one prime that's bigger than your table size, and this will work
[INAUDIBLE]
I forget whether you have to assume that m is prime.
I'd have to check.
I'm guessing not, but don't quote me on that.
Check the section in the textbook.
So good.
Easy to compute.
The analysis is simpler, but it's a little bit easier here.
Essentially this is very much like products but there's no carries here from one.
When we do the dot product instead of just multiplying in base m we multiply them based on that would give the same thing as multiplying in base 2, but we get carries from one m-sized digit to the next one.
And that's just more annoying to think about.
So here we're essentially getting rid of carries.
So it's in some sense even easier to compute.
And in both cases, it's universal.
So we want to prove this property.
That if we choose a random a then the probability of two keys, k and k' which are distinct mapping via h to the same value is at most 1/m So let's prove that.
So we're given two keys.
We have no control over them because this has to work for all keys that are distinct.
The only thing we know is that they're distinct.
Now if two keys are distinct, then their vectors must be distinct.
If two vectors are distinct, that means at least one item must be different.
Should sound familiar.
So this was like in the matrix multiplication verification algorithm that [INAUDIBLE] taught.
So k and k' differ in some digit.
Let's call that digit d. So k sub d is different from k sub d'.
And I want to compute this probability.
We'll rewrite it.
The probability is over a. I'm choosing a uniformly at random.
I want another probability that that maps k and k' to the same slot.
So let me just write out the definition.
It's probability over a that the dot product of a and k is the same thing as when I do the dot product with k' mod m. These two, that sum should come out the same, mod m. So let me move this part over to this side because in both cases we have the same ai.
So I can group terms and say this is the probability-- probability sum over i equals 0 to r minus 1 of ai times ki minus ki prime equals 0.
Mod m. OK, no pun intended.
Now we care about this digit d. d is a place where we know that this is non-zero.
So let me separate out the terms for d and everything but d. So this is the same as ability of, let's do the d term first, so we have ad times kd minus kd prime.
That's one term.
I'm going to write the summation of i not equal to d of ai ki minus ki prime.
These ones, some of them might be zero.
Some are not.
We're not going to worry about it.
It's enough to just isolate one term that is non-zero.
So this thing we know does not equal zero.
Cool.
Here's where I'm going to use a little bit of number theory.
I haven't yet used that m is prime.
I required m is prime because when you're working modulo m, you have multiplicative inverses.
Because this is not zero, there is something I can multiply on both sides and get this to cancel out and become one.
For every value x there is a value y.
So x times y equals 1 modulo m. And you can even compute it in constant time in a reasonable model.
So then I can say I want the probability that ad is minus kd minus kd prime inverse.
This is the multiplicative inverse I was talking about.
And then the sum i not equal to d whatever, I don't actually care what this is too much, I've already done the equals part.
I still need to write mod m. The point is this is all about ad.
Remember we're choosing a uniformly at random.
That's the same thing as choosing each of the ai's independently uniformly at random.
Yeah?
Is the second line over there isolating d [INAUDIBLE]?
Second from the top
Which?
This one?
No up
This?
Down.
That one.
No.
The one below that
Yes
Is that line isolating d or is that--
No.
I haven't isolated d yet.
This is all the terms.
And then going from this line to this one, I'm just pulling out the i equals d term.
That's this term.
And then separating out the i not equal to d
I get it
Right?
This sum is just the same as that sum.
But I've done the d term explicitly
Sure.
I get it
So I've done all this rewriting because I know that ad is chosen uniformly at random.
Here we have this thing, this monstrosity, but it does not depend on ad.
In fact it is independent of ad.
I'm going to write this as a function of k and k' because those are given to us and fixed.
And then it's also a function of a0 and a1.
Everything except d. So ad minus 1, ad plus 1, and so on up to ar minus 1.
This is awkward to write.
But everything except ad appears here because we have i not equal to d. And these ai's are random variables.
But we're assuming that they're all chosen independently from each other.
So I don't really care what's going on in this function.
It's something.
And if I rewrite this probability, it's the probability over the choice of a. I can separate out the choice of all these things from the choice of ad.
And this is just a useful formula.
I'm going to write a not equal to d. All the other-- maybe I'll write a sub i not equal to d. All the choices of those guys separately from the probability of choosing ad of ad equal to this function.
If you just think about the definition of expectation, this is doing the same thing.
We're thinking of first choosing the ai's where i is not equal to d. And then we choose ad.
And this computational will come out the same as that.
But this is the probability of a uniformly random number equaling something.
So we just need to think about-- sorry.
Important.
That would be pretty unlikely that would be 1/u, but this is all working modulo m. So if I just take a uniformly random integer and the chance of it hitting any particular value mod m is 1/m.
And that's universality.
So in this case, you get exactly 1/m, no less than or equal to.
Sorry, I should have written it's the expectation of 1/m, but that's 1/m because 1/m has no random parts in it.
Yeah?
How do we know that the, that this expression doesn't have any biases in the sense that it doesn't give more, more, like if you give it the uniform distribution of numbers, it doesn't spit out more numbers than others and that could potentially--
Oh, so you're asking how do we know that this hash family doesn't prefer some slots over others, I guess
Of course like after the equals sign, like in this middle line in the middle.
Middle board
This one?
Oh, this one
Middle board
Middle board.
Here
Yes.
So how do we know that if you give it--
This function
--random variables, it won't prefer certain numbers over others?
So this function may prefer some numbers over others.
But it doesn't matter.
All we need is that this function is independent of our choice of ad.
So you can think of this function, you choose all of these random-- actually k and k' are not random-- but you choose all these random numbers.
Then you evaluate your f. Maybe it always comes out to 5. Who knows.
It could be super biased.
But then you choose ad uniformly at random.
So the chance of ad equalling 5 is the same as the chance of ad equaling 3.
So in all cases, you get the probability is 1/m.
What we need is independence.
We need that the ad is chosen independently from the other ai's.
But we don't need to know anything about f other than it doesn't depend on ad.
So and we made it not depend on ad because I isolated ad by pulling it out of that summation.
So we know there's no ad's over here.
Good question.
You get a bonus Frisbee for your question.
All right.
That ends universal hashing.
Any more questions?
So at this point we have at least one universal hash family.
So we're just choosing, in this case, a uniformly at random.
In the other method, we choose a and b uniformly at random.
And then we build our hash table.
And the hash function depends on m. So also every time we double our table size, we're going to have to choose a new hash function for the new value of m. And that's about it.
So this will give us constant expected time-- or in general 1 plus alpha if you're not doing table doubling-- for insert, delete, and exact search.
Just building on the hashing with chaining.
And so this is a good method.
Question?
Why do you say expected value of the probability?
Isn't it sufficient to just say the probability of [INAUDIBLE]?
Uh, yeah, I wanted to isolate-- it is the overall probability of this happening.
I rewrote it this way because I wanted to think about first choosing the ai's where i does not equal d and then choosing ad.
So this probability was supposed to be only over the choice of ad.
And you have to do something with the other ai's because they're random.
You can't just say, what's the probability ad equaling a random variable?
That's a little sketchy.
I wanted to have no random variables over all.
So I have to kind of bind those variables with something.
And I just want to see what the-- This doesn't really affect very much, but to make this algebraically correct I need to say what the ai's, i not equal to d are doing.
Other questions?
Yeah
Um, I'm a bit confused about your definition of the collision in the lower left board.
Why are you adding i's [INAUDIBLE]?
Yeah, sorry.
This is a funny notion of colliding.
I just mean I want to count the number of keys that hash to the same slot as ki
So it's not necessarily like a collision [INAUDIBLE]
You may not call it a collision when it collides with itself, yeah.
Whatever you want to call it.
But I just mean hashing to the same slot is ki.
Yeah.
Just because I want to count the total length of the chain.
I don't want to count the number of collisions in the chain.
Sorry.
Probably a poor choice of word.
We're hashing because we're taking our key, we're cutting it up into little bits, and then we're mixing them up just like a good corned beef hash or something.
All right let's move on to perfect hashing.
This is more exciting I would say.
Even cooler-- this was cool from a probability perspective, depending on your notion of cool.
This method will be cool from a data structures perspective and a probability perspective.
But so far data structures are what we know from 006.
Now we're going to go up a level, literally.
We're going to have two levels.
So here we're solving-- you can actually make this data structure dynamic.
But we're going to solve the static dictionary problem which is when you have no inserts and deletes.
You're given the keys up front.
You're given n keys.
You want to build a table that supports search.
And that's it.
You want search to be constant time and perfect hashing, also known as FKS hashing because it was invented by Fredman, Komlos, and Szemeredi in 1984.
What we will achieve is constant time worst case for search.
So that's a little better because here we're just doing constant expected time for search.
But it's worse in that we have to know the keys up in advance.
We're going to take the linear space in the worst case.
And then the remaining question is how long does it take you to build this data structure?
And for now I'll just say it's polynomial time.
It's actually going to be nearly linear.
And this is also an expected bounds.
Actually with high probability could be a little more strong here.
So it's going to take us a little bit of time to build this structure, but once you have it, you have the perfect scenario.
There's going to be in some sense no collisions in our hash table so it would be constant times first search and linear space.
So that part's great.
The only catch is it's static.
But beggars can't be choosers I guess.
All right.
I'm not sure who's begging in that analogy but.
The keys who want to be stored.
I don't know.
All right, so the big idea for perfect hashing is to use two levels.
So let me draw a picture.
We have our universe, and we're mapping that via hash function h1 into a table.
Look familiar?
Exactly the diagram I drew before.
It's going to have some table size m. And we're going to set m to be within a constant factor of n. So right now it looks exactly like regular-- and it's going to be a universal, h1 is chosen from a universal hash family, so universal hashing applies.
The trouble is we're going to get some lists here.
And we don't want to store the set of colliding elements, the set of elements that hash to that place, with a linked list because linked lists are slow.
Instead we're going to store them using a hash table.
It sounds crazy.
But we're going to have-- so this is position 1.
This is going to be h2,1.
There's going to be another hash function h2,0 that maps to some other hash table.
These hash tables are going to be of varying sizes.
Some of them will be of size 0 because nothing hashes there.
But in general each of these slots is going to map instead of to a linked list to a hash table.
So this would be h2, m minus 1.
I'm going to guarantee in the second level of hashing there are zero collisions.
Let that sink in a little bit.
Let me write down a little more carefully what I'm doing.
So h1 is picked from a universal hash family.
Where m is theta n. I want to put a theta-- I mean I could m equals n, but sometimes we require m to be a prime.
So I'm going to give you some slop in how you choose m. So it can be prime or whatever you want.
And then at the first level we're basically doing hashing with chaining.
And now I want to look at each slot in that hash table.
So between 0 and m-1.
I'm going to let lj be the number of keys that hash, it's the length of the list that would go there.
It's going to be the number of keys, among just the n keys, Number of, keys hashing to slot j.
So now the big question is, if I have lj keys here, how big do I make that table?
You might say, well I make a theta lj.
That's what I always do.
But that's not what I'm going to do.
That wouldn't help.
We get exactly, I think, the same number of collisions if we did that, more or less, in expectation.
So we're going do something else.
We're going to pick a hash function from a universal family, h2,j.
It again maps the same universe.
The key thing is the size of the hash table I'm going to choose which is lj squared.
So if there are 3 elements that happen to hash to this slot, this table will have size 9.
So it's mostly empty.
Only square root fraction-- if that's a word, if that's a phrase-- will be full.
Most of it's empty.
Why squared?
Any ideas?
I claim this will guarantee zero collisions with decent chance.
Yeah
With 1/2 probability you're going to end up with no collisions
With 1/2 probability I'm going to end up with no collisions.
Why?
What's it called?
Markov [INAUDIBLE]
Markov's inequality would prove it.
But it's more commonly known as the, whoa, as the birthday paradox.
So the whole name of the game here is the birthday paradox.
If I have, how's it go, if I have n squared people with n possible birthdays then-- is that the right way?
No, less.
If I have n people and n squared possible birthdays, the probability of getting a collision, a shared birthday, is 1/2.
Normally we think of that as a funny thing.
You know, if I choose a fair number of people, then I get immediately a collision.
I'm going to do it the opposite way.
I'm going to guarantee that there's so many birthdays that no 2 of them will collide with probability of 1/2 No, 1/2 is not great.
We're going to fix that.
So actually I haven't given you the whole algorithm yet.
There are two steps, 1 and 2.
But there are also two other steps 1.5 and 2.5.
But this is the right idea and this will make things work in expectation.
But I'm going to tweak it a little bit.
So first let me tell you step 1.5.
It fits in between the two.
I want that the space of this data structure is linear.
So I need to make sure it is.
If the sum j equals 0 to m minus 1 of lj squared is bigger than some constant times n-- we'll figure out what the constant is later-- then redo step 1.
So after I do step 1, I know how big all these tables are going to be.
If the sum of those squares is bigger than linear, start over.
I need to prove that this will only have to take-- this will happen an expected constant number of times.
log n times with high probability.
In fact why don't we-- yeah, let's worry about that later.
Let me first tell you step 2.5 which is I want there to be zero collisions in each of these tables.
It's only going to happen with probability of 1/2 So if it doesn't happen, just try again.
So 2.5 is while there's some hash function h2,j that maps 2 keys that we're given to the same slot at the second level, this is for some j and let's say ki different from ki prime.
But they map to the same place by the first hash function.
So if two keys map to the same secondary table and there's a conflict, then I'm just going to redo that construction.
So I'm going to repick h2,j.
h2,j was a random choice.
So if I get a bad choice, I'll just try another one.
Just keep randomly choosing the a or randomly choosing this hash function until there are zero collisions in that secondary table.
And I'm going to do this for each table.
So we worry about how long these will take, but I claim expected constant number of trials.
So let's do the second one first.
After we do this y loop there are no collisions with the proper notion of the word collisions, which is two different keys mapping to the same value.
So at this point we have guaranteed that searches are constant time worst case after we do all these 4 steps because we apply h1, we figure out which slot we fit in.
Say it's slot j, then we apply h2j and if your item's in the overall table, it should be in that secondary table.
Because there are no collisions you can see, is that one item the one I'm looking for?
If so, return it.
If not, it's not anywhere.
If there are no collisions then I don't need chains coming out of here because it is just a single item.
The big question-- so constant worst case space because 1.5 guarantees that.
Constant worst case time first search.
The big question is, how long does it take to build?
How many times do we have to redo steps 1 and 2 before we get a decent-- before we get a perfect hash table.
So let me remind you of the birthday paradox, why it works here.
As mentioned earlier this is going to be a union bounds.
We want to know the probability of collision at that second level.
Well that's at most the sum of all possible collisions, probabilities of collisions.
So I'm going to say the sum over all i not equal to ij of the probability.
Now this is over our choice of the hash function h2,j.
Of h2,j of ki equaling h2,j of ki prime.
So union bounds says, of course.
The probability of any of them happening-- we don't know about interdependence or whatnot-- but certainly almost the sum of each of these possible events.
There are a lot of possible events.
If there 's li things, that there are going to be li choose 2 possible collisions we have to worry about.
We know i is not equal to i prime.
So the number of terms here is li choose 2.
And what's this probability?
[INAUDIBLE]
1/li at most because we're assuming h2,j is a universal hash function so the probability of choosing-- sorry?
li squared.
Thank you.
The size of the table.
1/m but m in this case, the size of our table is li squared.
So the probability that we choose a good hash function and that these particular keys don't hit is at most 1/li squared.
This is basically li squared/ 2.
And so this is at most 1/2.
It's a slightly less than li squared/2.
So this is at most 1/2.
And this is basically a birthday paradox in this particular case.
That means there is a probability of at least a half that there is zero collisions in one of these tables.
So that means I'm basically flipping a fair coin.
If I ever get a heads I'm happy.
Each time I get a tails I have to reflip.
This should sound familiar from last time.
So this is 2 expected trials or log n with high probability.
We've proved log n with high probability.
That's the same as saying the number of levels in a skip list is log n with high probability.
How many times do I have to flip a coin before I get a heads?
Definitely at most log n. Now we have to do this for each secondary table.
There are m equal theta and secondary tables.
There's a slight question of how big are the secondary tables.
If one of these tables is like linear size, then I have to spend linear time for a trial.
And then I multiply that by the number of trials and also the number of different things that would be like n squared log n n. But you know a secondary table better not have linear sides.
I mean a linear number of li equal n. That would be bad because then li squared is n squared and we guaranteed that we had linear space.
So in fact you can prove with another Chernoff bound.
Let me put this over here.
That all the li's are pretty small.
Not constant but logarithmic.
So li is order log n with high probability for each i and therefore for all i.
So I can just change the alpha my minus 1 n to the alpha and get that for all i this happens.
In fact, the right answer is log over log log, if you want to do some really messy analysis.
But we just, logarithmic is fine for us.
So what this means is we're doing n different things for each of them with high probability li is of size log n. And then maybe we'll have to do like log n trials repeating until we get a good hash function there.
And so the total build time for steps 1 and 2.5 is going to be at most n times log squared n. You can prove a tighter bound but it's polynomial.
That's all I wanted to go for and it's almost linear.
So I'm left with one thing to analyze which is step 1.5.
This to me is maybe the most surprising thing that it works out.
I mean here we designed-- we did this li to li squared so the birthday paradox would happen.
This is not surprising.
I mean it's a cool idea, but once you have the idea, it's not surprising that it works.
What's a little more surprising is that squaring is OK from a space perspective.
1.5 says we're going to have to rebuild that first table until the sum of these squared lengths is at most linear.
I can guarantee that each of these is logarithmic so the sum of the squares is at most like n log squared n. But I claim I can get linear.
Let's do that.
So for step 1.5 we're looking at what is the expectation of the sum of the lj squareds being more than linear.
Sorry.
Expectation.
Let's first compute the expectation and then we'll talk about a tail bound which is the probability that we're much bigger than the expectation.
First thing is I claim the expectation is linear.
So again whenever we're counting something-- I mean this is basically the total number of pairs of items that collide at the first level with double counting.
So I mean if you think of lj and then I make a complete graph on those lj items, that's going to have like the squared number of edges, so, if I also multiply by 2.
So this is the same thing as counting how many pairs of items map to the same spot, the same slot.
So this is going to-- and that I can write as an indicator random variable which lets me use linearity of expectation which makes me happy because then everything simple.
So I'm going to write Ii,j.
This is going to be 1 if each 1 of ki, I guess, equals h1 if kj and it's going to be 0 if h1 otherwise.
This is the total number of pairwise colliding items including i versus i.
And so like if li equals 1, li squared is also 1.
There's 1 item colliding with itself.
So this actually works exactly.
All right, with the wrong definition of colliding.
If you bear with me.
So now we can use linear of expectation and put the E in here.
So this is sum i equals 1 to n sum j equals 1 to n of the expectation of Ii,j.
But we know the expectation of the Ii,j is the probability of it equaling 1 because it's an indicator random variable.
The probability of this happening over our choice of h1 is at most 1/m by universality.
Here it actually is m because we're at the first level.
So this is at most 1/m which is theta n. So when i does not equal j, so it's a little bit annoying.
I do have to separate out the Ii terms from the i and different i not equal to j terms.
But there's only-- I mean it's basically the diagonal of this matrix.
There's n things that will always collide with themselves.
So we're going to get like n plus the number of i not equal to pairs double counted.
So it's like 2 times n choose 2.
But we get to divide by m. So this is like n squared /n.
So we get order n. So that's not-- well, that's cool.
Expected space is linear.
This is what makes everything work.
Last class was about getting with high probability bounds when we're working with logs.
When you want to get that something is log with high probability, you have to use, with respect to n, you have to use a turn off bound.
But this is about-- now I want to show that the space is linear with high probability.
Linear is actually really easy.
You can use a much weaker bound called Markov inequality.
So I want to claim that the probability of h1 of this thing lj squareds being bigger than some constant times n is at most the expectation of that thing divided by cn.
This is Markov's inequality.
It holds for anything here.
So I'm just repeating it over here.
So this is nice because we know that this expectation is linear.
So we're getting like a linear function divided by cn.
Remember we get to choose c. The step said if it's bigger than some constant times n then we're redoing the thing.
So I can choose c to be 100, whatever.
I'm going to choose it to be twice this constant.
And then this is at most half.
So the probability of my space being too big is at most a half.
We're back to coin flipping.
Every time I flip a coin, if I get heads I have the right amount of space at less than c times n space.
If I get a tails I try again.
So the expected number of trials is 2 at most not trails, trials.
And it's also log n trials with high probability.
How much time do I spend for each trial?
Linear time.
I choose one hash function.
I hash all the items.
I count the number of collision squared or the sum of lj squared.
That takes linear time to do.
And so the total work I'm doing for these steps is n log n. So n log n to do step 1 and 1 prime and log squared n to do steps 2 and 2 prime.
Overall n Polylog or polynomial time.
And we get guaranteed no collisions for static data.
Constant worst case search and linear worst case space.
This is kind of surprising that this works out but everything's nice.
Now you know hashing.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
Today we have another data structures topic which is, Data Structure Augmentation.
The idea here is we're going to take some existing data structure and augment it to do extra cool things.
Take some other data structure there we've covered.
Typically, that'll be a balanced search tree, like an AVL tree or a 2-3 tree.
And then we'll modify it to store extra information which will enable additional kinds of searches, typically, and sometimes to do updates better.
And in 006, you've seen an example of this where you took AVL trees and augmented AVL trees so that every node knew the number of nodes in that rooted subtree.
Today we're going to see that example but also a bunch of other examples, different types of augmentation you could do.
And we'll start out with a very simple one, which I call easy tree augmentation, which will include subtree size as a special case.
So with easy tree augmentation, the idea is you have a tree, like an AVL tree, or 2-3 tree, or something like that.
And you'd like to store, for every node x, some function of the subtree, rooted at x.
Such as the number of nodes in there, or the sum of the weights of the nodes, or the sum of the squares of the weights, or the min, or the max, or the median maybe, I'm not sure.
Some function f of x which is a function of that.
Maybe not f of x, but we want to store some function of that subtree.
Say the goal is to store f of the subtree rooted at x at each node x in a field which I'll call x.f.
So, normally nodes have a left child, right child, parent.
But we're going to store an extra field x.f for some function that you define.
This is not always possible, but here's a case where it is possible.
That's going to be the easy case.
Suppose x.f can be computed locally using lower information, lower nodes.
And we'll say, let's suppose it can be computed in constant time from information in the node x from x's children and from the f value that's stored in the children.
I'll call that children.f.
But really, I mean left child.f, right child.f, or if you have a 2-3 tree you have three children, potentially.
And the .f of each of them.
OK.
So suppose you can compute x.f locally just using one level down in constant time.
Then, as you might expect, you can update whenever a node ends up changing.
So more formally.
If some set of nodes change-- call this at s. So I'm stating a very general theorem here.
If there is some set of nodes, which we changed something about them.
We change either their f field, we change some of the data that's in the node, or we do a rotation, loosen those around.
Then we count the total number of ancestors of these nodes.
So this subtree.
Those are the nodes that need to be updated because we're assuming we can compute x.f just given the children data.
So if this data is changing, we have to update it's parents value of f because it depends on this child value.
We have to update all those parents, all the way up to the root.
So however many nodes there are there, that's the total cost.
Now, luckily, in an AVL tree, or 2-3 tree, most balanced search structures, the updates you do are very localized.
When we do splits in a 2-3 tree we only do it up a single path to the root.
So the number of ancestors here is just going to be log n. Same thing with an AVL tree.
If you look at the rotations you do, they are up a single leaf to root path.
And so the number of ancestors that need to be updated is always order log n. Things change, and there's an order log n ancestors of them.
So this is a little more general than we need, but it's just to point out if we did log n rotation spread out somewhere in the tree, that would actually be bad because the total number of ancestors could be log squared.
But because in the structures we've seen, we just work on a single path to the root, we get log n. So in a little more detail here, whenever we do a rotation in an AVL tree.
Let's say A, B, C, x, y.
Remember rotations?
Been a while since we've done rotations.
So we haven't changed any of the nodes in A, B, C, but we have changed the nodes x and y.
So we're going to have to trigger an update of y.
First, we'd want to update y.f and then we're going to trigger the update to x.f.
And as long as this one can be computed from its children, then we compute y.f, then we can compute x from its children.
All right.
So a constant number of extra things we need to do whenever we do rotation.
And because the rotations lie on a single path, total cost that-- once we stop doing the rotations, in AVL insert say, then we still have to keep updating up to the root.
But there's only log n at most log n nodes to do that.
OK.
Same thing with 2-3 trees.
We have a node split.
So we have, I guess, three keys, four children.
That's too many.
So we split to two nodes and an extra node up here.
Then we just trigger an update of this f value, an update of this f value, and an update of that f value.
And because that just follows a single path everything's log n. So this is a general theorem about augmentation.
Any function that's well behaved in this sense, we can maintain in AVL trees and 2-3 trees.
And I'll remind you and state, a little more generally, what you did in 006, which are called order statistic trees in the textbook.
So here we're going to-- let me first tell you what we're trying to achieve.
This is the abstract data type, or the interface of the data structure.
We want to do insert, delete, and say, successor searches.
It's the usual thing we want out of a binary search tree.
Predecessor too, sure.
We want to do rank of a given key which is, tell me what is the index of that key in the overall sorted order of the items, of the keys?
We've talked about rank a few times already in this class.
Depends whether you start at 0 or 1, but let's say we start at one.
So if you say rank of the key that happens to be the minimum, you want to get one.
If you say rank of the key that happens to be the median, you want to get n over 2 plus 1, and so on.
So it's a natural thing you might want to find out.
And the converse operation is select, let's say of i, which is, give me the key of rank i.
We've talked about select as an offline operation.
Given an array, find me the median.
Or find me the n over seventh rank item.
And we can do that in linear time given no data structure.
Here, we want a data structure so that we can find the median, or the seventh item, or the n over seventh key, whatever in log n time.
We want to do all of these in log n per operation.
OK.
So in particular, rank of selective i should equal i.
We're trying to find the item of that rank.
So far, so good.
And just to plug these two parts together.
We have this data structure augmentation tool, we have this goal we want to achieve, we're going to achieve this goal by applying this technique where f is just the subtree size.
It's the number of nodes in that subtree because that will let us compute rank.
So we're going to use easy tree augmentation with f of subtree equal to the number of nodes in the subtree.
So in order for this to apply, we need to check that given a node x we can compute x.f just using its children.
This is easy.
We just add everything up.
So x.f would be equal to 1.
That's for x.
Plus the sum of c.f for every child c. I'll write this as a python interpolation so it looks a little more like an algorithm.
I'm trying to be generic here.
If it's a binary search tree you just do x.left.f, plus x.right.f.
But this will work also for 2-3 trees.
Pick your favorite data structure.
As long as there's a constant number of children then this will take constant time.
So we satisfied this condition.
So we can do easy tree augmentation.
And now we know we have subtree sizes.
So given any node.
We know the number of descendants below that node.
So that's cool.
It lets us compute rank in select.
I'll just give you those algorithms, quickly.
We can check that they're log n time.
Yeah.
So the idea is pretty simple.
You have some key-- let's think about binary trees now, because it's a little bit easier.
We have some item x.
It has a left subtree, right subtree.
And now let's look up from x.
Just keep calling x.parent.
So sometimes the parent is to the right of us and sometimes the parent is to the left of us.
I'm going to draw this in a, kind of, funny way.
But this funny way has a very special property, which is that the x-coordinate in this diagram is the key value.
Or is the sorted order of the keys, right?
Everything in the left subtree of x has a value less than x.
If we say all the keys are different.
Everything to the right of x has a value greater than x.
If x was the left child of its parent, that means this thing is also greater than x.
And if we follow a parent and this was the right child of that parent, that means this thing is less than x.
So that's why I drew it all the way over to the left.
This thing is also less than x because it was a, I'll call it a left parent.
Here we have a right parent, so that means this is something greater than x.
And over here we have a left parent, so this is something less than x.
Let's say that's the root.
In general, there's going to be some left edges and some right edges as we go up.
These arrows will go either left or right in a binary tree.
So the rank of x is just 1 plus the number of nodes that are less than x.
Number of keys that are less than x.
So there's these guys, there's these guys, and there's whatever's hanging off-- OK.
Here I've almost violated my x-coordinate rule.
If I make these really narrow, that's right.
All of these things, all of these nodes in the left subtrees of these less than x nodes will also be less than x.
If you think about these other subtrees, they're going to be bigger than x.
So we don't really care about them.
So we just want to count up all these nodes and all of these nodes.
So the algorithm to do that is pretty simple.
We're just going to start out with-- I'm going to switch from this f notation to size.
That's a little more natural.
In general, you might have many functions.
Size is the usual notation for subtree size.
So we start out by counting up how many items are here.
And if we want to start at a rank of 1, if the min has rank 1, then I should also do plus 1 for x itself.
If you wanted to start at zero you just omit that plus 1.
And then, all I do is walk up from x to the root of the tree.
And whenever we go left from, say x to x prime.
So that means we have an x prime.
It's right child is x.
And so when we went from x to its parent we went to the left.
Then we say rank plus equals x prime.left.size plus 1 for x prime itself.
And maybe x prime.left.size is zero.
Maybe there's no nodes over there.
But at the very least we have to count those nodes that are to the left of us.
And if there's anything down here we add up all those things.
So that lets us compute rank.
How long does it take?
Well, we're just walking up one path from a leaf to a root-- or not necessarily a leaf, but from some node x to the root.
And as long we're using a balance structure like AVL trees.
I guess I want binary here, so let's say AVL trees.
Then this will take log n time.
So I'm spending constant work per step, and there's log n steps.
Clear?
So that's good old rank.
Easy to do once you have subtree size.
Let's do select for fun.
This may seem like review, but I drew out this picture explicitly because we're going to do it a lot today.
We'll have pictures like this a bunch of times.
Really helps to think about where the nodes are, which ones are less than x, which ones are greater than x.
Let's do select first.
This you may not have seen in 006.
So we're going to do the reverse.
We're going to start at the root and we're going to walk down.
Sounds easy enough.
But now walking down is kind of like doing a search but we don't have a key we're searching for, we have a rank we're searching for.
So what is that rank?
Rank is i. OK.
So on the other hand, we have the node x.
We'd like to know the rank of x and compare that to i.
That will tell us whether we should go left, or go right, or whether we happen to find the item.
Now one possibility is we call rank of x to find the rank of x.
But that's dangerous because I'm going to have a four loop here and it's going to take log n iterations.
If at every iteration of computing rank of x, and rank costs log n, then overall cost might be log squared n. So I can't afford to-- I want to know what the rank of x is but I can't afford to say rank, open paren, x.
Because that recursive call will be too expensive.
So what is the rank of x in this case?
This is a little special.
What's that?
Number of left children plus 1
Number of left, or the size of the left subtree plus 1.
Yep.
Plus 1 if we're counting, starting at one.
Very good.
I'm slowly getting better.
Didn't hit anyone this time.
OK.
So at least for the root, this is the rank, and that only takes us constant time in the special case.
So we'll have to check that it's still holds after I do the loop.
But it will.
So, cool.
Now there are three cases.
If i equals rank.
If the rank we're searching for is the rank that we happen to have, then we're done, right?
We just return x.
That's the easy case.
More likely is that I will be either less than or greater than the rank of x. OK.
So if i is less than the rank, this is fairly easy.
We just say x equals x.left.
Did I get that right?
Yep.
In this case, the rank.
So here we have x.
It's at rank, rank.
And then we have the left subtree and the right subtree.
And so if the rank were searching for is less than rank, that means we know it's in here.
So we should go left.
And if we just said x equals x.left you might ask, well what rank are we searching for in here?
Well, exactly the same rank.
Fine.
That's easy case.
In the other situation, if we're searching in here, we're searching for rank greater than rank.
Then I want to go right but the new rank that I'm searching for is local to this subtree.
I'm searching for i minus this stuff.
This stuff is rank.
So I'm going to let i be i minus rank.
Make sure I don't have any off by 1 errors.
That seems to be right.
OK. And then I do a loop.
So I'll write repeat.
So then I'm going to go up here and say, OK. Now relative to this thing.
What is the rank of the root of this subtree?
Well, it's again going to be that node .left.size plus 1.
And now I have the new rank I'm searching for, i.
And I just keep going.
You could write this recursively if you like, but here's an iterative version.
So it's actually very familiar to the select algorithm that we had, like when we did deterministic linear time median finding or randomized median finding.
They had a very similar kind of recursion.
But in that case, they were spending linear time to do the partition and that was expensive.
Here, we're just spending constant time at each node and so the overall cost is log n. So that's nice.
Any questions about that?
OK.
I have a note here.
Subtree size is obvious once you know that's what you should do.
Another natural thing to try to do would be to augment, for each node, what is the rank of that node?
Because then rank is really easy to find.
And then select would basically be a regular search.
I just look at the rank of the root, I see whether the rank I'm looking for is too big, or too small, and I go left or right, accordingly.
What would be bad about augmenting with rank of a node?
Updates.
Why?
What's a bad example for an update?
If you add new in home element
Right.
Say we insert a new minimum element.
Good catch, cameraman.
That was for the camera, obviously.
So, right.
If we insert, this is off to the side, but say we insert, I'll call it minus infinity.
A new key that is smaller than all other keys, then the rank of every node changes.
So that's bad.
It means that easy tree augmentation, in particular, isn't going to apply.
And furthermore, it would take linear time to do this.
And you could keep inserting, if you insert keys in decreasing order from there, every time you do an insert, all the ranks increase by one.
Maintaining that's going to cost linear time per update.
So you have to be really careful that the function you want to store actually can be maintained.
Be very careful about that, say, on the quiz coming up, that when you're augmenting something you can actually maintain it.
For example, it's very hard to maintain the depths of nodes because when you do a rotation a whole lot of depths change.
Depth is counting from the root.
How deep am I?
When I do a rotation then this entire subtree went down by one.
This entire subtree went up by one.
In this picture.
But it's very easy to maintain heights, for example.
Height counting from the bottom is OK, because I don't affect the height of a, b, and c. I affect it for x and y but that's just two nodes.
That I can afford.
So that's what you want to be careful of in the easy tree augmentation.
So most the time easy tree augmentation does the job.
But in the remaining two examples, I want to show you cooler examples of augmentation.
These are things you probably wouldn't be expected to come up with on your own, but they're cool.
And they let us do more sophisticated operations.
So the first one is called level linking.
And here we're going to do it in the context of 2-3 trees, partly for variety.
So the idea of level linking is very simple.
Let me draw a 2-3 tree.
Not a very impressive 2-3 tree.
I guess I don't feel like drawing too much.
Level linking is the idea of, in addition to these child and parent pointers, we're going to add links on all the levels.
Horizontal links, you might call them.
OK.
So that's nice.
Two questions-- can we do this?
And what's it good for?
So let's start with can we do this.
Remember in 2-3 trees all we have to think about are splits and merges.
So in a split, we have, for a brief period, let's say three keys, four children.
That's too many.
So we change that to-- I'm going to change this in a moment.
For now, this is the split you know and love, maybe.
At least know.
And if we think about where the leveling pointers are, we have one before.
And then we just need to distribute those pointers to the two resulting nodes.
And then we have to create a new pointer between the nodes that we just created.
This is, of course, easy to do.
We're here.
We're taking this node.
We're splitting it in half.
So we have the nodes right in our hands so just add pointers between them.
And key thing is, there's some node over here on the left.
It used to point to this node, now we have to change it to point to the left version.
The left half of the node.
And there's some node over on the right.
We have to change it's left pointer to point to this right half of the node.
But that's it.
Constant time.
So this doesn't fall under the category of easy tree augmentation because this is not isolated to the subtree.
We're also dealing with it's left and right subtrees.
But still easy to do in constant time.
Merging nodes is going to be similar.
If we steal a node from our parents or former sibling, nothing happens in terms of level links.
But if we have, say, an empty node and a node that cannot afford any stealing.
So we have single child here, two children, and we merge it into-- We're taking something from our parent.
Bringing it down.
Then we have three children afterwards.
Again, we used to have these level pointers.
Now we just have these level pointers.
It's easy to maintain.
It's just a constant size neighborhood.
Because we have the level links, we can get to our left and right neighbors and change where the links point to.
So easy to maintain in constant time.
I'll call it constant overhead.
Every time we do a split or merge we spend additional constant time to do it.
We're already spending constant time.
So just changes everything by constant factor.
So far, so good.
Now, I'm going to have to tweak this data structure a little bit.
But let me first tell you why.
What am I trying to achieve with this data structure?
What I'm trying to achieve is something called the finger search property.
So let's just think about the case where I'm doing a successful search.
I'm searching for key x and I find it in the data structure.
I find it in the tree.
Suppose I found one-- I search for x, I found it.
And then I search for another key y.
Actually I think I'll do the reverse.
First I found y, now I'm searching for x.
If x and y are nearby in the tree, I want this to run especially fast.
For example, if x is the successor of y I want this to take constant time.
That would be nice.
In the worst case x and y are very far away from me in the tree then I want it to take log n time.
So how could I interpolate between constant time for finding the successor and log n time for finding the worst case search.
So I'm going to call this search of x from y.
Meaning, this is a little imprecise, but what I mean is when I call search, I tell it where I've already found y.
And here it is.
Here's the node storing y.
And now I'm given a key x.
And I want to find that key x given the node that stores key y.
So how long should this take?
Will be a good way to interpolate between constant time at one extreme.
The good case, when x and y are basically neighbors in sorted order, versus log n time, in the worst case
Distance along the graph
Distance along the graph.
That would be one reasonable definition.
So I have a tree which you could think of as a graph.
Measure the shortest path length from x to y.
Or we have a more sophisticated graph over here.
Maybe that length.
The trouble with the distance in the graph, that's a reasonable suggestion, but it's very data structure specific.
If I use an AVL tree without level links, then the distance could be one thing, whereas if I use a 2-3 tree, even without level lengths, it's going to be a different distance.
If I use a 2-3 tree with level lengths it's going to be yet another distance.
So that's a little unsatisfying.
I want this to be an answer to a question.
I don't want to phrase the question in terms of that data structure
Difference between ranks of x and y?
Difference between ranks between x and y.
That's close.
So I'm going to look at the rank of x and rank of y.
Let's say, take the absolute difference.
That's kind of how far away they are in sorted order.
Do you want to add anything?
Log?
Log.
Yeah.
Because in the worst case the difference in ranks could be linear.
So I want to add a log out here to get log n in that worst case.
Add a big o for safety.
That's how much time we want to achieve.
So this would be the finger search property that you can solve this problem in this much time.
Again, difference in ranks is at most n. So this is at most log n. But if y is the successor of x this will only be constant and this will be constant.
So this is great if you're doing lots of searches and you tend to search for things that are nearby, but sometimes you search for things are far away.
This gives you a nice bound.
On the one hand, we have, this is our goal.
Log difference of ranks.
On the other hand, we have the suggestion that what we can achieve is something like the distance in the graph.
But we have a problem with this.
I used to think that data structure solved this problem, but it doesn't.
Let me just draw-- actually I have a tree right there.
I'm going to use that one.
Suppose x is here and y is here.
OK.
This is a bit of a small tree but if you think about it long enough, this node is the predecessor of this node.
So their difference in ranks should be 1.
But the distance in the graph here is two.
Not very impressive.
But in general, you have a tree of height log n. If you look at the root, and the predecessor of the root, they will have a rank difference of one by definition of predecessor.
But the graph distance will be log n. So that's bad news, because if we're only following pointers there's no way to get from here to there in constant time.
So we're not quite there.
We're going to use another tweak that data structure, which is store the data in the leaves.
Tried to find a data structure that didn't require this and still got finger search.
But as far as I know, there is none.
No such data structure.
If you look at, say, Wikipedia about B-trees, you'll see there's a ton of variations of B-trees.
B+-trees, B*-trees.
This is one of those.
I think B+-trees.
As you saw, B-trees or 2-3 trees, every node stored one or two keys.
And each key only existed in one spot.
We're still only going to put each key in one spot, kind of.
But it's only going to be the leaf spots.
OK. Good news is most nodes are leaves, right?
Constant fraction of the nodes are going to be leaves.
So it doesn't change too much from a space efficiency standpoint.
If we just put data down here and don't put-- I'm not going to put any keys up here for now.
So this a little weird.
Let me draw an example of such a tree.
So maybe we have 2, and 5, and 7, and 8, 9, let's say.
Let's put 1 here.
So I'm going to have a node here with three children, a node here with two children, and here's a node with two children.
So I think this mimics this tree, roughly.
I got it exactly right.
So here I've taken this tree structure.
I've redrawn it.
There's now no keys in these nodes.
But everything else is going to be the same.
Every node is going to have 0 children if it's a leaf, or two, or three children otherwise.
Never have one child because then you wouldn't get logarithmic depth.
All the leaves are going to be at the same depth.
And that's it.
OK. That is a 2-3 tree with the data stored in the leaves.
It's a useful trick to know.
Now we're going to do a level linked 2-3 tree.
So in addition to that picture, we're going to have links like this.
OK. And I should check that I can still do insert and delete into these structures.
It's actually not too hard.
But let's think about it.
I think, actually, it might be easier.
Let's see.
So if I want to do an insert-- OK.
I have to first search for where I'm inserting.
I haven't told you how to do search yet.
OK.
So let's first think about search.
What we're going to do is data structure augmentation.
We have simple tree augmentation.
So I'm going to do it and each node, what the functions I'm going to store are the minimum key in the subtree, and the maximum key in the subtree.
There are many ways to do this, but I think this is kind of the simplest.
So what that means is at this node, I'm going to store 1 as the min and 7 as the max.
And at this node it's going to be 1 at the min and 9 at the max.
And here we have 8 as the min and 9 as the max.
Again min and max of subtrees are easy to store.
If I ever change a node I can update it based on its children, just by looking at the min of the leftmost child and the max of the rightmost child.
If I didn't know 1 and 9, I could just look at this min and that max and that's going to be the min and the max of the overall tree.
So in constant time I can update the min and the max of a node given the min and the max of its children.
Special case is at the leaves.
Then you have to actually look at keys and compare them.
But leaves only have, at most, two keys.
So pretty easy to compare them in constant time.
OK.
So that's how I do the augmentation.
Now how do I do a search?
Well, if I'm at a node and I'm searching for a key.
Well, let's say I'm at this node.
I'm searching for a key like 8.
What I'm going to do is look at all of the children.
In this case, there's two.
In the worst case there's three.
I look at the min and max and I see where does 8 fall?
Well it falls in this interval.
If I was searching for 7 1/2 I know it's not there.
It's going to be in between here.
If I'm doing a successor then I'll go to the right.
If I'm doing predecessor I'll go to the left.
And then take either the maximum item or the minimum item.
If I'm searching for 8 I see, oh.
8 falls in the interval between 8 and 9, so I should clearly take the right child among those two children.
In general, there's three children.
Three intervals.
Constant time.
I can find where my key falls in the interval.
OK.
So search is going to take log n time again, provided I have these mins and maxs.
If you stare at it long enough, this is pretty much the same thing as regular search in a 2-3 tree.
But I've put the data just one level down.
OK. Good.
That was regular search.
I still need to do finger search, but we'll get there.
And now, if I want to do an insert into this data structure, what happens.
Well I search for the key let's say I'm inserting 6.
So maybe I go here.
I say because 6.
Is in this interval.
6 is in neither of these intervals.
But it's closest to the interval 2, 5, or the interval 7.
Let's say I go down to 2, 5.
And well, to insert 6 I'll just add a 6 on there.
Of course, now that node is too big.
So there's still going to be a split case at the leaves where I have let's say, a,b,c, too many keys.
I'm going to split that into a,b and c. This is different from before.
It used to be I would promote b to the parent because the parent needed the key there.
Now parents don't have keys.
So I'm just going to split this thing, roughly, in half.
It works.
It's still the case that whoever was the parent up here now has an additional child.
One more child.
So maybe that node now has four children but it's supposed to be two or three.
So if I have a node with four children, what I'm going to do, I'm suppose to use these fancy arrows.
What do I do in this case?
It's just going to split that into two nodes with two children.
And again this used to have a parent.
Now that parent has an additional child, and that may cause another split.
It's just like before.
Was just potentially split all the way up to the root.
If we split the root then we get an additional level.
But we could do all this and we can still maintain our level links, if we want.
But everything will take log n. I won't draw the delete case, as delete is slightly more annoying.
But I think, in this case, you never have to worry about where is the key coming from, your child or your parent?
You're just merging nodes so it's a little bit simpler.
But you have to deal with the leaf case separately from the nonleaf case.
OK.
So all this was to convince you that we can store data in the leaves.
2-3 trees still work fine.
Now I claim that the graph distance in level link trees is within a constant factor of the finger search bound.
So I claim I can get the finger search property in 2-3 trees, with data in the leaves, with level links.
So lots of changes here.
But in the end, we're going to get a finger search bound.
Let's go over here.
So here's a finger search operation.
First thing I want to do is identify a node that I'm working with.
I want to start from y's node.
So we're supposing that we're told the node, a leaf, that contains y.
So I'm going to let v be that leaf.
OK. Because we're supposing we've already found y, and now all the data is in the leaves.
So give me the leaf that contains y.
So that should take constant time.
That's just part of the input.
Now I'm going to do a combination of going up and horizontal.
So starting at a leaf.
And the first thing I'm going to do is check, does this leaf contain what I want?
Does it contain the key I'm searching for, which is x?
So that's going to be the case.
At every node I store the min and the max.
So if x happens to fall between the min and the max, then I'm happy.
Then I'm going to do a regular search in v's subtree.
This seems weird in the case of a leaf.
In the case of a leaf, this is just to check the two keys that are there.
Which one is x. OK.
But in general I gave you this search algorithm which was, if I decide which child to take, according to the ranges, that's a downward search.
So that's what I'm calling regular search here.
Maybe downward would be a little better.
This is the usual log n time thing.
But we're going to claim a bound better than log n. If this is not the case, then I know x either falls before v.min or after v.max.
So if x is less than v.min then I'm going to go left.
v equals v. I'll call it level left to be clear.
You might say left is the left child.
There's no left child here, of course.
But level left is clear.
We take the horizontal left pointer.
And otherwise x is greater than v.max.
And in that case I will go right.
That seems logical.
And in both cases we're going to go up.
x equals x.parent Whoops.
v equals v.parent.
X is not changing here.
X is a key we're searching for.
v is the node.
V for vertex.
So we're always going to go up, and then we're going to go either left or right, and we're going to keep doing that until we find a subtree that contains x in terms of key range.
Then we're going to stop this part and we're just going to do downward search.
I should say return here or something.
I'm going to do a downward search, which was this regular algorithm.
And then whatever it finds, that's what I return.
I claim the algorithm should be clear.
What's less clear is that it achieves the bound that we want.
But I claim that this will achieve the finger search property.
Let me draw a picture of what this thing looks like kind of generically.
On small examples it's hard to see what's going on.
So I'm going to draw a piece of a large example.
Let's say we start here.
This is where y was.
I'm searching for x.
Let's suppose x is to the right.
'Cause otherwise I go to the other board.
So x is to the right.
I'll discover that the range with just this node, this node maybe contains one other key.
I'll find that range is too small.
So I'm going to go follow the level right pointer, and I get to some other node.
Then I'm going to go to the parent.
Maybe the parent was the parent of those two children so I'm going to draw it like that.
Maybe I find this range is still too low.
I need to go right to get to x, so I'm going to follow a level pointer to the right.
I find a new subtree.
I'll go to its parent.
Maybe I find that this subtree, still the max is too small.
So I have to go to the right again.
And then I take the parent.
So this was an example of a rightward parent.
Here's an example of a leftward parent.
This is maybe the parent of both of these two children.
Then maybe this subtree is still too small, the max is still smaller than x.
So then I go right one more time.
Then I follow the parent.
Always alternating between right and parent until I find a node whose subtree contains x.
It might have actually, x may be down here, because I immediately went to the parent without checking whether I found where x is.
But if I know that x is somewhere in here then I will do a downward search.
It might go left and then down here, or it might go right, or there's actually potentially three children.
One of these searches will find the key x that I'm looking for because I'm in the case where x is between v.min and v.max, so I know it's in there, somewhere.
It could be x doesn't exist, but it's predecessor or successor is in there somewhere.
And so one of these three subtrees will contain the x range.
And then I go follow that path.
And keep going down until I find x or it's predecessor or successor.
Once I find it's predecessor I can use a level right pointer to find its successor, and so on.
So that's kind of the general picture what's going on.
We keep going rightward and we keep going up.
Suppose we do k up steps.
Let's look at this last step here.
Step k. How high am I in the tree?
I started at the leaf level.
Remember in a 2-3 tree all the leaves have the same level.
And I went up every step.
Sorry.
I don't know what this is, like the 2-step dance where, let's say every iteration of this loop I do one left or right step, and then a parent step.
So I should call this iteration k. I guess there's two k steps, then.
Just to be clear.
So in iteration k, that means I've gone up k times and I've gone either right or left k times.
You can show if you start going right you keep going right.
If you initially go left you'll keep going left.
Doesn't matter too much.
At iteration k I am at height k, or k minus 1, or however you want to count.
But let's call it k. So when I do this right pointer here I know that, for example, I am skipping over all of these keys.
All the keys down-- the keys are in the leaves, so all these things down here, I'm jumping over them.
How many keys are down there?
Can you tell me, roughly, how many keys I'm skipping over when I'm moving right at height k?
It's not a unique answer.
But you can give me some bounds.
Say again.
Number of children to the k power.
Yeah.
Except we don't know the number of children.
But it's between 2 and 3 Closer one should be easy but I fail.
So it's between two and three children.
So there's the number-- if you look at a height k tree, how many leaves does it have?
It's going to be between 2 to the k and 3 to the k. Because I have between 2 and 3 children at every node.
And so it's exponential in k. That's all I'll need.
OK.
When I'm at height k here, I'm skipping over a height k minus 1 tree or something.
But it's going to be-- So in iteration k I'm skipping, at least, some constant times 2 to the k. Maybe to the k minus 1, or to the k minus 2.
I'm being very sloppy.
Doesn't matter.
As long as it's exponential in k, I'm happy.
Because I'm supposing that x and y are somewhat close.
Let's call this rank difference d. Then I claim the number of iterations I'll need to do in this loop is, at most, order log d. Because if, when I get to the k-th iteration, I'm jumping over 2 to the k elements.
How large does k have to be before 2 to the k is larger than d?
Well, log d. Log base 2 The number of iterations is order log d, where d is the rank difference.
d is the absolute value between rank of x and rank of y.
And I'm being a little sloppy here.
You probably want to use an induction.
You need to show that they're really, these items here that you're skipping over that are strictly between x and y.
But we know that there's only d items between x or y.
Actually d minus 1, I guess.
So as soon as we've skipped over all the items between x and y, then we'll find a range that contains x, and then we'll go do the downward search.
Now how long does the downward search cost?
Whatever the height of the tree is.
What's the height of the tree?
That's the number of iterations.
So the total cost.
The downward search will cost the same as the rest of the search.
And so the total cost is going to be order log d. Clear?
Any questions about finger searching with level linked data at the leaves, 2-3 trees?
Sir, I'm not sure why [INAUDIBLE] d, why is that?
I'm defining d to be the rank of x minus rank of y.
My goal is to achieve a log d bound.
And I'm claiming that because once I've skipped over d items, then I'm done.
Then I've found x.
And at step k I'm skipping over 2 to the k items.
So how big is k going to be?
Log d. That's all.
I used d for a notation here.
Cool.
Finger searching.
It's nice.
Especially if you're doing many consecutive searches that are all relatively close to each other.
But that was easy.
Let's do a more difficult augmentation.
So the last topic for today is range trees.
This is probably the coolest example of augmentation, at least, that you'll see in this class.
If you want to see more you should take advanced data structure 6851.
And range trees solve a problem called orthogonal range searching.
Not orthogonal search ranging.
Orthogonal range search.
So what's the problem?
I'm going to give you a bunch of points.
Draw them as fat dots so you can actually see them.
In some dimension.
So this is, for example, a 2D point set.
OK. Over here I will draw a 3D point set.
You can tell the difference, I'm sure.
There.
Now it's a 3D point set.
And this is a static point set.
You could make this dynamic but let's just think about the static case.
Don't want the 2D points and the 3D points to mix.
Now, you get to preprocess this into a data structure.
So this is a static data structure problem.
And now I'm going to come along with a whole bunch of queries.
A query will be a box.
OK.
In two dimensions, a box is a rectangle.
Something like this.
Axis aligned.
So I give you an x min, x max, a y min, and a y max.
I want to know what are the points inside.
Maybe I want you to list them.
If there's a lot of them it's going to take a long time to list them.
Maybe I just want to know 10 of them as examples.
Maybe this is a Google search or something.
I just get the first 10 results in the first page, I hit next then want the next 10, that kind of thing.
Or maybe I want to know how many search results there are.
Number of points in the rectangle.
Bunch of different problems.
In 3D, it's a 3D box.
Which is a little harder to draw.
You can't really tell which points are inside the box.
Let's say these three points are all inside the box.
I give you an interval in x, an interval in y, and an interval in z, and I want to know what are the points inside.
How many are there?
List them all.
List 10 of them, whatever.
OK.
I want to do this in poly log time, let's say.
I'm going to achieve today log squared for the 2D problem and log cubed for the 3D problem, plus whatever the size output is.
So let me just write that down.
So the goal is to preprocess n points in d dimensions.
So you get to spend a bunch of time preprocessing to support a query which is, given a box, axis aligned box, find let's say the number of points in the box.
Find k points in the box.
I think that's good.
That includes a special case of find all the points in the box.
So this, of course, we have to pay a penalty of order k for the output.
No getting around that.
But I want the rest of the time to be log to the d. So we're going to achieve log to the d n plus size of the output.
And you get to control how big you want the output to be.
So it's a pretty reasonable data structure.
In a certain sense we will understand what the output is in log to the d time.
If you actually want to list points, well, then you have to spend the time to do it.
All right.
So 2D and 3D are great, but let's start with 1D.
First we should understand 1D completely, then we can generalize.
1D we already know how to do.
1D I have a line.
I have some points on the line.
And I'm given, as a query, some interval.
And I want to know how many points are in the interval, give me the points in the interval, and so on.
So how do I do this?
Any ways?
If d is 1.
So I want to achieve log d, sorry, log n, plus size of output.
I hear whispers.
Yeah?
Segment trees?
Segment tree?
That's fancy.
We won't cover segment trees.
Probably segment trees do it.
Yeah.
We know lots of ways to do this.
Yeah?
Sorted array?
Sorted array is probably the simplest.
If I store the items in a sorted array and I have two values, I'll call them x1 and x2, because it's the x min and x max.
Binary search for x1.
Binary search for x2.
Find the successor of x1 and the predecessor of x2.
I'll find these two guys.
And then I know all the ones in between.
That's the match.
So that'll take log n time to find those points and then we're good.
So we could do a sorted array.
Of course, sorted array is a little hard to generalize.
I don't want to do a 2D array, that sounds bad.
You could, of course, do a binary search tree.
Like an AVL tree.
Same thing.
Because we have log n search, find successor, and predecessor, I guess you could use Van Emde Boas, but that's hard to generalize to 2D.
You could use level links.
Here's a fancy version.
We could use level linked 2-3 trees with data in the leaves.
Then once I find x min, I find this point, I can go to the successor in constant time because that's a finger search with a rank difference of 1.
And I could just keep calling successor and in constant time per item I will find the next item.
So we could do that easily with the sorted array.
BST is not so great because successor might cost log n each time.
But if I have the level links then basically I'm just walking down the link list at the bottom of the tree.
OK.
So actually level linked is even better.
BST would achieve something like log n plus k log n, where k is the size of the output.
If I want k points in the box I have to pay log n. For each level linked I'll only pay log n plus k. Here I actually only need the levels at the leaves.
Level links.
OK. All good.
But I actually want to tell you a different way to do it that will generalize better.
The pictures are going to look just like the pictures we've talked about.
So these would actually work dynamically.
My goal here is just to achieve a static data structure.
I'm going to idealize this solution a little bit.
And just say, suppose I have a perfectly balanced binary search tree.
That's going to be my data structure.
OK.
So the data structure is not hard, but what's interesting is how I do a range search.
So if I do range query of the interval, I'll call it ab.
Then what I'm going to do is do a binary search for a, do a binary search for b, trim the common prefix of those search paths.
That's basically finding the lowest common ancestor of a and b.
And then I'm going to do some stuff.
Let me draw the picture.
So here is, suppose here's the node that contains a.
Here's the node that contains b.
They may not be at the same depth, who knows.
Then I'm going to look at the parents of a. I just came down from some path here, and some path down to b. I want to find this branching point where the paths to a and the paths to b diverge.
So let's just look at the parent of a.
It could be a right parent, in which case there's a subtree here.
Could be a left parent in which case, subtree here.
I'm going to follow my convention again.
That x-coordinate corresponds roughly to key.
Left parent here.
Maybe right parent here.
Something like that.
OK.
Remember it's a perfect tree.
So, actually, all the leaves will be at the same level.
And, roughly here, x-coordinate corresponds to key.
So here is a.
And I want to return all the keys that are between a and b.
So that's everything in this sweep line.
The parents of the LCA don't matter, because this parents either going to be way over to the right or way over to the left.
In both cases, it's outside the interval a to b.
So what I've tried to highlight here, and I will color it in blue, is the relevant nodes for the search between a and b.
So a is between a and b.
This subtree is greater than a and less than b.
This node, and these nodes.
This node, and these nodes.
This node and these nodes.
The common ancestor.
And then the corresponding thing over here.
All the nodes in all these blue subtrees, plus these individual nodes, fall in the interval between a and b, and that's it.
OK.
This should look super familiar.
It's just like when we're computing rank.
We're trying to figure out how many guys are to our left or to our right.
We're basically doing a rightward rank from a and a leftward rank from b.
And that finds all the nodes.
And stopping when those two searches converge.
And then we're finding all the nodes between a and b. I'm not going to write down the pseudocode because it's the same kind of thing.
You look at right parents and left parents.
You just walk up from a.
Whenever you get a right parent then you want that node, and the subtree to its right.
And so that will highlight these nodes.
Same thing for b, but you look at left parents.
And then you stop when those two searches converge.
So you're going to do them in lock step.
You do one step for a and b.
One step for a and b.
And when they happen to hit the same node, then you're done.
You add that node to your list.
And what you end up with is a bunch of nodes and rooted subtrees.
The things I circled in blue is going to be my return value.
So I'm going to return all of these nodes, explicitly.
And I'm also going to return these subtrees.
I'm not going to have to write them down.
I'm just going to return the root of the subtree, and say, hey look.
Here's an entire subtree that contains points that are in the answer.
Don't have to list them explicitly, I can just give you the tree.
Then if I want to know how many results are in the answer, well, just augment to store subtree size at the beginning.
And then I can count how many nodes are down here, how many nodes are down here, add that up for all the triangles, and then also add one for each of the blue nodes, and then I've counted the size of the answer in how much time?
How many subtrees and how many nodes am I returning here?
Log.
Log n nodes and log n rooted subtrees because at each step, I'm going up by one for a, and up by one for b.
So it's like 2 log n. Log n. So I would call this an implicit representation of the answer.
From that implicit representation you can do subtree size.
Augmentation to count the size the answer.
You can just start walking through one by one, do an inter traversal of the trees, and you'll get the first k points in the answer in order k time.
Question?
Just a clarification.
You said when we were walking up, you want to get all the ancestors in their right subtrees.
But you don't do that for the left parent, right?
That's right.
As I'm walking up the tree, if it's a right parent then I take the right subtree and include that in the answer.
If it's a left parent just forget about it.
Don't do anything.
Just keep following parents.
Whenever I do right parent then I also add that node and the right subtree.
If it's a left parent I don't include the node, I don't include the left subtree.
I also don't include the right subtree.
That would have too much stuff.
It's easy when you see the picture, you would write down the algorithm.
It's clear.
It's left versus right parents
Would you include the left subtree of b?
I would also-- thank you.
I should color the left subtree of b. I didn't apply symmetry perfectly.
So we have the right subtree of a and the left subtree of b.
Thanks.
I would also include b if it's a closed interval.
Slightly more general.
If a and b are not in the tree then this is really the successor of a and this is the predecessor of b.
So then a and b don't have to be in there.
This is still a well defined range search.
OK. Now we really understand 1D.
I claim we've almost solved all dimensions.
All we need is a little bit of augmentation.
So let's do it.
Let's start with 2D.
But then 3D, and 4D, and so on will be easy.
Why do I care about 4D range trees?
Because maybe I have a database.
Each of these points is actually just a row in the database which has four columns, four values.
And what I'm trying to do here is find all the people in my database that have a salary between this and this, and have an age between this and that, and have a profession between this and this.
I don't know what that means.
Number of degrees between this and this, whatever.
You have some numerical data representing a person or thing in your database, then this is a typical kind of search you want to do.
And you want to know how many answers you've got and then list the first hundreds of them, or whatever.
So this is a practical thing in databases.
This is what you might call an index in the database.
So let's start.
Suppose your data is just two dimensional.
You have two fields for every item.
What I'm going to do is store a 1D range tree on all points by x.
So this data structure makes sense if you fix a dimension.
Say x is all I care about.
Forget about y.
So my point set.
Yeah.
So what that corresponds to is projecting each of these points onto the x-axis.
And now also projecting my query.
So my new query is from here to here in x.
And so this data structure will let me find all these points that match in x.
That's not good because there's actually only two points that I want, but I find four points in this picture.
But it's half of the answer.
It's all the x matches forgetting about y.
Now here's the fun part.
So when I do a search here I get log n nodes.
Nodes are good because they have a single key in them.
So I'll just check for each of those log n nodes.
Do they also match in y?
If they do, add it to the answer.
If they don't forget about it.
OK.
But the tricky part is I also get log n subtrees representing parts of the answer.
So potentially it could be that your search, this rectangle, only has like five points.
But if you look at this whole vertical slab there's a billion points.
Now, luckily, those billion points are represented succinctly.
There's just log n subtrees saying, well there's half a billion here, and a quarter billion here, and an eighth of a billion here.
Now for each of that big chunk of output, I want to very quickly find the ones that match in y.
How would I find the ones matching in y?
A range tree.
Yeah.
OK.
So here's what we're going to do.
For each node, call it x. x is overloaded.
It's a coordinate.
So many things.
Let's call it v. In the, this thing I'm going to call the x-tree.
So for every node in the x-tree I'm going to store another 1D range tree.
But this time using the y-coordinate on all points in these rooted subtree.
At this point I really want to draw a diagram.
So, rough picture.
Forgive me for not drawing this perfectly.
This is roughly what the answer looks like for the 1D range search.
This is the x-tree.
And here I've searched between this value and this value in the x-coordinate.
Basically I have log n nodes.
I'm going to check those separately.
Then I also have these log n subtrees.
For each of those log n sub trees I'm going to have a pointer-- this is the augmentation-- to another tree of exactly the same size.
On exactly the same data that's in here.
It's also over here.
But it's going to be sorted by y.
And it's a 1D range tree by y.
Tons of data duplication here.
I took all these points and I copied them over here, but then built a 1D range tree in y.
This is all preprocessing.
So I don't have to pay for this.
It's polynomial time.
Don't worry too much.
And then I'm going to search in here.
What does the search in there look?
I'm going to get, you know, some more trees and a couple more nodes.
OK.
But now those items, those points, match in x and y because this whole subtree matched in x and I just did a y search, so I found things that matched in y.
So I get here another log n trees that are actually in my answer.
And for each of these nodes I have a corresponding other data structure where I do a little search and I get part of the answer.
Every one.
Sounds huge.
This data structure sounds huge, but it's actually small.
But one thing that's clear is it takes log squared n time, because I have log n triangles over here.
For each of them I spend log n to find triangles over here.
The total output is log squared n nodes, for each of them I have to check manually.
Plus, so over here, there's log n, different searches I'm doing.
Each one has size log n. So I get log squared little triangles that contain the results that match in x and y.
How much space in this data structure?
That's the remaining challenge.
Actually, it's not that hard, because if you look at a key.
So look at some key in this x-tree.
Let's look at a leaf because that's maybe the most interesting.
Here's the x-tree.
x-tree has linear size.
Just one tree.
If I look at some key value, well, it lives in this subtree.
And so there's going to be a corresponding blue structure of that size that contains that key.
And then there's the parent.
So there's a structure here.
That has a corresponding blue triangle.
And then its parent, that's another triangle.
That contains-- I'm looking at a key k here.
All of these triangles contain the key k. And so key k will be duplicated all this many times, but how many sub trees is k in?
Log n. Each key, fundamental fact about balanced binary search trees, each key lives in log n subtrees.
Namely all of its ancestors.
Awesome.
Because that means the total space is n log n. There's n keys.
Each of them is duplicated at most log n times.
In general, log to the d minus 1.
So If you do it in 3D, each of the blue trees, every node in it has a corresponding pointer to a red tree that's sorted by z.
And you just keep doing this, sort of, nested searching, like super augmentation.
But you're only losing a log factor each dimension you add.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
Thanks for coming on quiz day.
I have to have someone to throw Frisbees to.
Empty chairs would be difficult.
So we're going to be doing dynamic programming, a notion you've learned in 6006.
We'll look at three different examples today.
The first one is really at the level of 006, a cute little problem on finding the longest palindromic sequence inside of a longer sequence.
Sometimes it's called longest palindromic subsequence.
And as we'll talk about, subsequence means that it can be non-contiguous.
So you could skip letters in a sequence of letters and you would still have a subsequence corresponding to that.
Don't have to be contiguous.
And then we'll raise the stakes a little bit.
So each of these problems gets progressively more complicated, more sophisticated.
And you'll probably see problems here, at least alternating coin game, that are beyond 006 in the sense that it wasn't covered.
Those kinds of notions weren't covered in 006.
So just in terms of review, I wrote this up here because I don't want to spend a whole lot of time on it.
This is something that you should have some familiarity with from the recitation, for example, that you had on Friday and from 006.
Dynamic programming is this wonderful hammer.
It's an algorithmic technique that you can use to solve problems that look exponential in complexity.
But if you can find this optimum substructure associated with the problem and its connection to it's subproblems, and if you can characterize that, then you can do a recursive decomposition of the problem where you show that you can construct the optimum solution from the subproblems.
And that's really the key step in dynamic programming, is step two.
Once you've made this characterization, you write this recurrence out that relates the optimal value of a bigger problem to the optimal values of subproblems.
And you compute the value of the optimal solution through a recursive memoization.
And that memoization is really what gives you the efficient algorithm, because you don't repeat the solution of subproblems.
You can also do this in an iterative fashion.
Essentially, you're going to be computing things bottom-up.
You might want to think about it as top-down when you write your recurrence.
But ultimately, when you actually execute the program, you'll be computing things bottom-up, and you'll be checking the memo table to see if you actually solved this problem before.
And that would be the recursive memoized case, which, in some sense, is a little bit easier to think about and translate directly from the recurrence.
But the other way to do it is to do it iteratively.
And we'll take a look for this first problem as to how you'd do the two different ways at least from a conceptual standpoint, even though I might not write out the code for each of those cases.
So a couple of choices here, recurse and memoize, or essentially do it iteratively.
And the smaller subproblems would have to get computed first in both approaches.
The one thing that sometimes we don't spend a whole lot of time on is this last step, which is getting the exact solution.
So a lot of the time, you stop with saying, I can compute the value of the optimum solution, the value in terms of the length of the longest palindromic sequence is 7 or 9.
But what is that sequence or subsequence?
That requires some additional coding, some additional accounting.
The construction of this optimal solution typically requires some back-tracing and some information to be kept track of during the recurse and memoize step or during the iterative step.
So that's something to keep in mind.
You're not quite done once you've found the value of the optimum solution.
More often than not, you want the solution.
So we'll talk about that as well a little bit.
But let's just dive in and look at this cute, little problem of longest palindromic sequence.
And palindromes, of course, read the same front to back or back to front.
Radar is a palindrome.
Just as a trivial example, a single letter is a palindrome.
Maybe I should have used a since that's actually a word.
But here I got bb.
That's definitely not a word, at least not a word that-- acronym, maybe.
Radar is a palindrome.
Able was I 'ere I saw Elba, right?
That's a palindrome.
That's, of course, not a single word.
But it's a famous palindrome, days of Napoleon.
But what we're trying to do here is, given that we have this notion of a palindrome, we'd like to discover palindromes inside longer words or longer sequences.
So what we have is a string, and we'll call it x1 through n, n greater than or equal to 1.
And we want to find the longest palindrome that is a subsequence.
And so here's an example to get you guys warmed up.
We'll have a couple of puzzles here in a second.
So character.
And you want to find the longest palindrome.
And so you go, I'll pick c, I'll skip h, I'll pick a, I'll pick r, a, and c. And carac, which I guess is not a word either-- but it's the longest palindrome that corresponds to a subsequence of character.
Right?
So the game here, as you can see, is to pick the letters that form the palindrome and drop the ones that don't.
OK?
And we're going to have to use dynamic programming to do this.
The answer will be greater than or-- 1 in length because we've defined a single letter as a palindrome.
So it has to be-- if you have one letter in the input, well, you just pick that letter.
But regardless of how many letters you have on the input, greater than or equal to 1, you know that you're going to get at least a one letter palindrome at the output.
So here we go.
Let's say I have under-- and this is thanks to Eric here.
I've got a couple of nice words-- underqualified.
The person who gets me the longest palindrome wins a Frisbee.
And if you want to code dynamic programming in the next two minutes and run it on your laptops, that's perfectly fine with me.
That's not cheating.
So underqualified.
So u is a palindrome.
So we got that.
Right?
So one letter for sure.
What else?
What's the longest palindrome?
Shout it out.
Go ahead.
Shout it out.
[STUDENTS RESPOND]
D-E-I-- wow, that was quick.
Deified.
That's right.
So right?
Well, deified is to make somebody a deity.
So that was you, a couple of you guys?
Do you have a Frisbee yet?
No
All right.
All right.
This is a little bit more difficult.
I tried this on my daughter yesterday, so I know it's a little more difficult.
Turboventilator.
We'll call that a word.
Turboventilator.
Yell it out
Rotor
Sorry?
Rotor
Rotor.
OK, well, that's five.
Can anybody beat that?
Rotator
Rotator.
So rotator.
Rotator.
So R-O-T-A-T-O-R, rotator.
All right.
Who is the rotator?
You already have one?
I want to throw this one.
Right.
Good practice with the quiz, guys.
Good practice with the quiz.
No, no, no.
These quiz jokes never go over well.
I've been teaching for 27 years, and I still haven't learned that you don't joke about exams.
But so nothing like this on the quiz.
I don't want you studying the thesaurus as opposed to the textbook for the next few hours.
OK?
Nothing like this on the quiz.
All right.
Those of you who are missing Python, who loved 6006 in Python, let's talk about how you would actually solve this using dynamic programming.
And so what we want is lij, which is the length of the longest palindromic subsequence for xij.
And we're going to have i less than or equal to j.
So that's essentially what we'd like to compute.
And essentially when I've said this here, when I have lij, I have decomposed the overall problem into subproblems, and it was kind of fairly obvious in this case, because I'm going to have to go in order.
Right?
I mean that's the constraint.
A subsequence does maintain the ordering constraint.
It's not like I can invert these letters.
That would be a different problem if I allowed you to do that.
So I'm going to start somewhere, and I'm going to end somewhere.
And I want to have a non-null subsequence, so I'm going to have i less than or equal to j. I'm good with i being equal to j, because I still have one letter, and, well, that happens to be a palindrome, and it'll have a length of 1.
So that's my lij.
And what I want to do is define a recursive algorithm that computes lij.
So we'll just try and figure out what the recurrence looks like, and then we can talk about memoization or iteration.
So if i equals equals j, then I'm going to return 1, because I know that that's a palindrome by default.
So that's easy.
And what do you think the next check should be?
If I look at this x sequence, and I have i as the starting point and j as the next point, what do you think the next check is going to be once I have-- if i is not equal to j?
If x of i equals x of j, then j equals as well
Sorry.
What was that again?
If x of i equals x plus--
Beautiful.
You're just checking to see-- you're just checking to see whether the two endpoints are equal or not.
Because if they're equal, then you can essentially grab those letters and say that you're going to be looking at a smaller subsequence that is going to get you a palindrome.
And you're going to be able to add these two letters that are equal on either side of the computed palindrome from the subsequence.
So if x of i equals equals x of j, then I'm going to say, if i plus 1 equals equals j, I'm going to go ahead and return 2, because at that point, I'm done.
There's nothing else to do.
Else I'm going to return 2 plus L of i plus 1 j minus 1.
So I'm going to look inside.
And I've got these two letters on either side that are equal.
So I can always prepend to the palindrome I got from here the letter, and then append the same letter.
And I got 2 plus whatever value I got from this quantity here.
So so far, it's not really particularly interesting from a standpoint of constructing the optimum solution.
But this last line that we have here, where we have the case that the two letters are not equal is the most interesting line of code.
That's the most interesting aspect of this algorithm.
So does someone tell me what this line is going to be out here?
Yeah, go ahead
If x of L i plus 1j or L i [INAUDIBLE]
Beautiful.
That's exactly right.
So what you're going to do is you're going to say, I need to look at two different subproblems, and I need to evaluate both of these subproblems.
And the first subproblem is I'm going to-- since these two letters are different, I'm going to have to drop one of them.
And I'm going to look inside.
I'm going to say-- in this case, I'm going to drop the i-th letter, and I'm going to get L i plus 1j.
And in the second case, I'm going to drop the j-th letter, and I'm going to get ij minus 1.
And that's it.
So it's a max.
And there's nothing that's being added here, because those two letters, one of them had to get dropped.
They weren't equal, so one of them had to get dropped.
So you're not adding anything to this.
So that's good.
And at this point, you're kind of done.
Right?
Whoops.
Oh, nice catch.
But you did drop something.
All right.
So the thing that we've done here is gotten to step three.
So just to be clear, we're not done-done, in terms of this chart here, because we don't have the code there that corresponds to actually computing the sequence.
So it's not that hard.
I'm not going to go over the code here.
You can certainly look at it in the notes.
But you need a little bit of tracing backwards in this recursion to actually compute things.
What is the complexity of what I wrote up there, though?
Yeah?
Theta n squared?
Theta n squared.
Do people agree that the complexity is theta n squared?
Or is this gentleman an optimist?
Is the complexity theta n squared?
Tell me why the complexity is theta n squared?
Because each subproblem is-- that code right there just executed by itself is constant.
But then there's theta n squared subproblems
Absolutely.
But if you actually implemented this code, and you ran it, and n was 100, how long would you wait for this code to complete?
Look at it.
What's missing?
The cache
The cache, exactly.
Well, you fixed your own little error there.
It was a trick question.
So there's no recursion-- I'm sorry, there's recursion here, but no memoization.
So this is exponential complexity.
You will recur.
In fact, the recurrence for that is something like T of n equals 1 if n equals 1, and 2T n minus 1 if n greater than 1.
And this would be 2 raised to n minus 1 in complexity.
Now there's a single line of code, and you all know this, that would fix this.
And that single line of code is simply something that says, right here, look at the lij-- and I'm writing this differently.
I'm calling this now a 2D array.
So that's why I have the open brackets and close brackets.
So I'm overloading l here.
But it's a 2D array that is going to essentially be a cache for the subproblem solution values.
And then if you want to do the backtracing to actually compute the solution, you could certainly have that as an additional record that's connected to this very same value.
But that's implementation, and we won't really go there.
So look at lij and don't recurse if lij already computed.
OK.
So that's important to remember.
Now, if you actually put that, the cache lookup, hash table lookup, array lookup, whatever you want to call it, out there, then what you said is exactly correct.
So our formula for computing the complexity of a DP, that you've seen a bunch of times and I mentioned in the very first lecture as well, is number of subproblems times time to solve each subproblem assuming or given that smaller ones are solved or the lookup is Order 1.
So lookup.
Now you could say that hash table lookup is Order 1 on average, et cetera, et cetera.
So what actually happens in the worst case, in this particular case and in most DPs, you can do things like perfect caching or, something that's even simpler in this case, is just a 2D array.
There's not going to be any collisions if you just use i and j as the indices to the array.
So you will definitely get an Order 1 lookup in this case, in most problems we'll look at in 046.
So if we just do that computation, which my friend over here just described, you do get your theta n squared, because you have theta n squared subproblems.
And time to solve each subproblem, given that the smaller ones are solved, is simply going to be a computation of a max and an addition.
So all of that is theta 1, because you're not counting the recursive calls.
So this is our first example.
I'm done with it.
Any questions about it?
Any questions about DP in general?
All right, good.
So little bit of review there.
Not a particularly complicated question.
Let's go to a different question corresponding to optimal binary search trees.
It's a very different question.
I don't think I need this anymore.
And it's kind of cute in its own way.
One of things that's interesting about this question is it seems like a greedy algorithm should work.
And we'll talk about that, as to why the greedy algorithm doesn't quite work.
So it's kind of similar to the interval scheduling and the weighted interval scheduling problem that we had back in February, where the regular interval scheduling problem, greedy worked, earliest finish time worked.
But when it came to weights, we had to graduate to dynamic programming.
So here's our second problem, optimal BSTs.
So what is an optimal BST?
We have a bunch of keys that we want to store in the BST, K1 through Kn.
And we'll assume that the way this is set up is that K1 is less than K2, da, da, da, less than Kn.
And just to make our life easier in our examples, we just assume that Ki equals i.
That's not necessarily required for anything we're going to do next.
It's just for the sake of examples and keeping things manageable.
So I got a bunch of keys.
And clearly there are many different binary search trees that can come, whether they're balanced or unbalanced.
Many different binary search trees can be consistent with a given set of keys.
If I choose the root to be Kn, then I'm going to get this horribly unbalanced tree.
If I chose the root to be somewhere in the middle, then I'm going to get something that looks a little better, at least at the top level.
But again, if I messed it up at the next level, I'd get something that's unbalanced.
So there's clearly many different BSTs.
I'm not talking about balanced BSTs here.
But we're going to define an optimality criterion that's a little bit different from balanced BSTs, because it's going to have this additional cost function associated with it that corresponds to the weight of the keys.
So what is that?
Well, I'm going to have weights associated with each of these keys corresponding to W1 through Wn.
And the easiest way to motivate you to think that these weights are an interesting addition to this problem is to think about these weights as being search probabilities.
So what you have, for argument's sake, is a static structure that you've created.
I mean, you could modify it.
There's nothing that's stopping you from doing that.
But let's just pretend for now that it's a static structure corresponding to this BST-- has a particular structure.
And chances are you're going to be searching for some keys more frequently than others.
And the Wi's tell you what the probabilities are in terms of searching for a particular key Ki.
So you can imagine, and we won't have this here, that you take-- the Wi's all sum up to 1, if you want to think of them as probabilities.
Or I'm just going to give you numbers.
I don't want to deal with fractions, don't particularly like fractions.
So you can imagine that each probability corresponds to Wi divided by the sum of all the Wi's, if you want all of the probabilities to sum up to 1.
So think of them as search probabilities, because then you'll see what the point of this exercise is.
And the point of this exercise is to find the BST T-- so we're actually constructing a binary search tree here.
So it's a little more interesting than a subsequence, for example-- it has a richer structure associated with it-- than an exponential number of possible binary search trees that are associated with a given set of n keys that are all binary search trees.
They're consistent, unbalanced, balanced-- unbalanced in one way versus another way, et cetera, et cetera.
So we want to find a binary search tree T that minimizes sigma i equals 1 through n. Obviously, we're going to have this Wi that's the game here.
Depth in that T-- so this depth is for that T. What is the depth of the node?
And K of i plus 1.
And I'll explain exactly what this and tell you what precisely the depth is.
But roughly speaking, the depth of the root-- not roughly speaking.
The depth of the root is 0.
And the depth of 1 below the root is 1, and so on and so forth.
And so, as you can see, what you want to do is collect in-- roughly speaking, you want to collect the high weight nodes to have low depth.
If Wi is high, you want the multiplicative factor corresponding to depth T of that node, i, i-th node, to be small.
And if Wi is small, then you don't mind the depth number to be higher.
You only have a certain number of low-depth nodes.
You only have one node of depth 0.
And you have two nodes of depth 1, which means that you only have one node where that quantity there is going to be 1, because you're doing 0 plus 1.
And you have two nodes where the quantity is going to be 2, and so on and so forth.
So you have some room here to play with corresponding to the BST structure, and you want to minimize that quantity.
Any questions so far?
All right, good.
So the search probabilities would be an example.
So in terms of a more concrete application, you could imagine you had a dictionary, English to French, French to English, what have you.
And there are obviously words that are more common, let's say, in common speech than others, and you do want to do the translation in a dynamic way using this data structure.
And you could imagine that in this case, the search probability of a word is associated with the occurrence of the word, the number of times over some normalized number of words that this particular word occurs.
And that would be the way.
So it makes sense to create a structure that minimizes that function, because it would minimize the expected search cost when you want to take this entire essay, for example, and convert it from English to French or vice versa.
So this can-- if these are search probabilities, then this would minimize-- this cost function here would minimize expected search cost.
Make sense?
Yeah.
So that's the definition of the problem.
And now we have to talk about why this is complicated, why this requires dynamic programming.
Why can't we just do something fairly straightforward, like a greedy algorithm?
And so let's look into that.
Let me give you a really good sense for what's going on here with respect to this cost function and maybe a little abstract by giving you a couple of concrete examples.
First off, we got exponentially many trees, exponential in n. Let's say that n equals 2.
So the number of nodes is 2.
Then remember that I'm assuming that the Ki's are all i's, that 1 and 2 would be the case for n equals 2 and 1, 2, 3 for n equals 3, et cetera.
So I could have a tree that looks like that.
And I could have a tree that looks like this.
That's it.
n equals 2, I got 2 trees.
So in this case, my cost function is W1 plus 2W2, and in this case, my cost function is 2W1 plus W2.
Just to be clear, what this is the K, and it's also the i.
So the numbers that you see inside, those are the i numbers, which happen to be equal to the Ki numbers.
And the weight itself would be the Wi.
So I put W1 here, and the reason it only gets multiplied by 1 is because the depth here is 0, and I add 1 to it.
The depth here is 1, and I add 1 to it, which is why I have a 2 out here.
Being a little pedantic here and pointing things out, because you're going to start seeing some equations that are a little more complicated than this, and I don't want you to get confused as to what the weight is and what the key number is and what the depth is.
There's three things going on here.
So over here you see that I'm looking at W1 here, which is this key.
So the 1 corresponds to the 1.
And this has a depth of 1, so I put a 2 in here, and so on.
So so far, so good?
When you get to n equals 3, you start getting-- well, at this point, you have 1, 2, 3, 4, 5-- you have five trees.
And the trees look like this.
I'll draw them really quickly just to get a sense of the variety here.
But I won't write the equations down for all of them.
There you go.
So those are the five binary trees associated with n equals 3, the binary search trees.
And this is kind of cool.
It's nice and balanced.
The other ones aren't.
And I'm not looking at-- Should I have another one here?
I guess I should have.
This is a-- oh, no, no, no.
So I'm not doing mirrors.
So there are a bunch of other trees that have the same equation associated with two-- no, that's not true.
Because these are it.
I was going to say that you could put 3 in here, but that wouldn't be a binary search tree.
So this is it.
And we've got a bunch of trees.
This one would be 2W1 plus W2 plus 2W3 by the same process that I used to show you that W1 plus 2W2 for the n equals 2 case.
You just go off, and that's the equation.
And so your goal here in an algorithm-- clearly this is not the algorithm you want to enumerate all the exponentially many trees, compute the equations for each of those trees, and pick the minimum.
I mean, that would work, but it would take exponential time.
But that's what you have to do now.
That's the goal.
You absolutely want the best possible three that has, for the given Wi's that you're going to be assigned as constants, you do want to find the one tree that has the minimum sum, and you want to do that for arbitrary n, and you want to do that in polynomial time.
So the first thing that you do when you have something like this is forgetting about the fact that we're in a dynamic programming lecture or a dynamic programming module of this class, when you see a problem like this in the real world, you want to think about whether a greedy algorithm would work or not.
And you don't want to go off and build this dynamic program solution, which is, chances are, going to be more inefficient then a greedy solution, which, if it produces the optimum answer, is the best way to go.
So an obvious greedy solution would be to pick K of r in some greedy fashion.
And what is the obvious way of picking K of r to try and get a greedy solution that at least attempts to minimize that cost function that we have there?
Highest weight
Highest weight, right?
So just pick K of r to be highest weight.
Because that goes back to what I said about if Wi is high, you want this value here to be small.
So that's essentially your greedy heuristic.
So K of r should be picked in a greedy fashion.
So what you do is you pick K of r as the root in this particular problem.
And you could certainly apply this greedy technique recursively.
So you do that for every subproblem that you find because when you do this choice of the root for Kr in the current problem that you have, you immediately split the keys into two sets, the sets that have to go on the left-hand side of Kr and the sets that have to go on the right-hand side of Kr.
So in this case, you know that you're going to have a bunch of keys here that correspond to Ki to K4, and over here, you're going to have-- oh, Kr minus 1, I should say, because you already have Kr out there.
And the keys here are going to be Kr plus 1 to Kj, if overall I'm going to say that eij is the cost of the optimal BST on Ki, Ki plus 1, all the way to Kj.
So I'm kind of already setting myself up here for dynamic programming.
But it also makes sense in the case of a greedy algorithm, where this greedy heuristic is going to be applied recursively.
So initially, you're going to have E(1, n)-- so if you want to compute E(1, n), which is the cost of the optimal BST on the original problem, all the way from 1 to n, you're going to look at it, and you're going to say, I have a bunch of keys here of different weights that correspond to K1 through Kn.
I'm going to pick the Kr that corresponds to the maximum weight, and I'm going to use that as the root node, and I'm going to stick that up there.
And the reason I did that is because I believe that this greedy heuristic is going to work, where this maximum weight node should have the absolute minimum depth.
So I stick that up there.
And then my BST invariant tells me that Ki through Kr minus 1-- remember, these are sorted, and they go in increasing order, as I have up here.
You're going to have those on the left, and you're going to have those on the right.
And then you've got to go solve this problem.
And you could certainly apply the greedy heuristic to solve this problem.
You could go essentially find the K that has the highest weight.
So you look at Wi through Wr minus 1, and you find the K that has the highest weight.
Actually, you're not going midway here.
It's quite possible that Kr is K of n. It just happens to be Kr is K of n. So that means that you have the highest weight node up here, but it's also the biggest key.
So all of the nodes are going to be on the left.
So this greedy heuristic-- and now you're starting to see, maybe there's a problem here.
Because if you have a situation where the highest weight node is also the largest node, you're going to get a pretty unbalanced tree.
So I can tell you that greedy doesn't work.
And when I gave this lecture, I think it was a couple of years ago, I made that statement, and this annoying student asked me for an example.
And I couldn't come up with one.
And so I kind of bailed on it, went on, completely dissatisfied, of course.
And by the end of the lecture, the same student came up with a counter-example.
So I'm going to have put that up here and thank the student who sent me an email about it.
And so here's a concrete example.
It turns out I was trying to get a tree node example for a few minutes and failed.
It's, I think, impossible-- I haven't proved this-- to find a tree node example with arbitrary weights for which the greedy algorithm fails.
But four nodes, it fails.
So that's the good news.
So here's the example, thanks to Nick Davis.
And it looks like this.
So I claim that the greedy algorithm for the problem that is given to me would produce this BST.
And the reason for that is simple.
The highest weight node happens to be node 2, which has a weight of 10.
So I would pick that, and I would stick that up here, which means, of course, that I'm going to have to have 1 on this side, and I'm going to have to have 4 and 3 on the other side.
And since 4 has a higher weight than 3, I'd pick that first.
And then 3 would have to go over here.
So if I go do the math, the cost is going to be 1 times 2-- and I'll explain what these numbers are.
I did tell you it was going to get a little cluttered here with lots of numbers.
But if you keep that equation in mind, then this should all work out.
So what do I have here?
I'm just computing-- this is W1.
So you see the first numbers in each of these are the weights.
And so this would be W2 and et cetera.
And you can see that this is at depth 1, which means I have to have 2 here, because I'm adding 1 to the depth.
So I have 1 times 2, 10 times 1, because that's at the root, 9 times 2, et cetera.
And I get 54.
It turns out the optimum tree-- you can do better than that if you pick 3, and you go like this.
And so what you have here is the cost equals 1 times 3 plus 10 times 2 plus 8 times 1 plus 9 times 2, and that's 49.
So I'll let you look at that.
The bottom line is because of the way the weights are set up here, you really want to use the top three spots, which have that the minimum depth, for the highest weight nodes.
And so you can see that I could make these weights arbitrary, obviously, and I could break a greedy algorithm as this little example shows.
So we got no recurs here.
We got to do some more work.
We're going to have to use DP to solve this problem.
The good news is DP does solve this problem.
So let's take a look at the decomposition.
And as with anything else, the key is the step that corresponds to breaking up the original problem into two parts or more that essentially give you this sort of decomposition.
And we don't quite know.
So this is really the key question here.
We don't quite know what the root node is going to be.
So what do we do when we don't know what to do?
What do we do in DP when we don't know what to do?
This is a very profound question.
What do you do?
Guess
You guess.
And not only do yo guess, you guess all outcomes.
You say, is it going to come up heads?
Is it going to come up tails?
I'll guess both.
You have a die.
It's going to get rolled, a 1, 2, 3, 4, 5, 6.
Guess them all, right?
So if you're going to have to guess that the root node could be any of the keys, and it still-- I mean there's a linear number of guesses there.
That's the good news.
It's going to stay polynomial time.
So we're going to have to guess.
And once you do that guess, it turns out that decomposition that you see up there is exactly what we want.
It's just that the greedy algorithm didn't bother with all of these different guesses.
And the DP is different from the greedy algorithm because it's going to do each of those guesses, and it's going to pick the best one that comes out of all of those guesses.
So it's really not that different from greedy.
So we'll keep that up there, leave that here.
Let's go over here.
So the recursion here is not going to be that hard, once you have gotten the insight that you just have to go make a linear number of guesses corresponding to the root node for your particular subproblem, and obviously this happens recursively.
So there's a little something here that that I'll point out with respect to writing these equations out.
So what I have here, just to be clear, is what I wrote up there.
Even when I was talking about the greedy algorithm, I had the subproblems defined.
So the original problem is E1 through n. The subproblems correspond to eij.
And given a subproblem, once I make a root choice, I get two subproblems that result from the root choice.
And each of those two subproblems is going to be something-- and this is something to keep in mind as we write these equations out-- they're going to be one level below the root.
So keep in mind that the original eij subproblem corresponded to this level, but that Ei r minus 1 problem and the E r plus 1j problem are at one level below.
So keep that in mind as we write these equations up.
And so what I want to do here is just write the recurrence.
And after that, things become fairly mechanical.
The fun is in the recurrence.
So I got Wi if i equals j.
And so I'm at this level.
And if I just have one node left, then at that level, I'm going to pay the weight associated with that node, and I'm going to have to do a certain amount of multiplication here with respect to the depth.
But I'm just talking about eij as the subproblem weight, just focusing in on that problem.
So if I only have one node, it's the root node.
And that weight is going to be Wi, because the root node has depth of 0, and I'm going to add 1 to it.
So for that subproblem, I got Wi.
Keep that in mind, because that's the only subtlety here with respect to these equations, making sure that our actual cost function that we're computing has the correct depth multiplicands associated with it.
So then we have our linear guessing.
And I might have said max at some point.
We want to get the min here.
And I had minimize here, so I think I wrote things down correct, but at some point, I think I might have said we want to maximize cost.
But we do want to minimize cost here corresponding to the expected search.
So we're doing a linear, and we're doing a min.
And we're going to go off and look at each of the different nodes as being the root, just like we discussed.
And what I'm going to do here is I'm going to simply say, first, that I'm going to look at Ei r minus 1, which corresponds to this, and I'm going to have plus E of r plus 1, j.
And at this point, I don't quite have what I want because I haven't actually taken into account the fact that the depth of the Ei r minus 1 and the r plus 1j is 1 more than the eij.
So it's 1 more than that.
And I also haven't taken into account the fact that, in this case, I definitely need to add a Wi as well.
Because the root node is part of my solution in both cases.
In one case, it ended my solution, the top case.
But in this case, it's also the root node, as you can see over on the left, and I've got to add that in there.
So I definitely need to add a Wi here.
And what else do I need to add?
From a standpoint of the weights, what other weights do I need to add to this line?
Yeah?
Go ahead
First of all, shouldn't that be a Wr?
First of all, that should be a Wr, you're exactly correct
And you want to add all the weights
And you want to add all the weights, right.
You're exactly right.
I have two more Frisbees, but I need to use them for something else.
So you get one next time.
Or do I have more than that?
No, no.
These are precious Frisbees here.
Sorry, man.
And you corrected me, too.
Shoot.
This is sad.
So that needed to be a Wr.
But you also need to add up all of the nodes that are in here, because they're one more level.
And now you see why I made the mistake.
I don't usually make mistakes.
But what I really want-- actually it's more like-- I don't know, a few per lecture.
Constant order one mistakes.
I'm going to say this is Wi, j, where Wi, j is simply the sum of all of the weights from i to j.
And this makes perfect sense, because the nice thing is that I don't even need to put an r in there.
I could choose a particular r. Wr will be in there.
But all of the other nodes are going to be in there anyway.
So it doesn't really matter what the r selection is corresponding to this term here.
I will put it inside the minimization.
This bracket here is closed with that, but you can pull that out, because that doesn't depend on r. It's just going to be there for all of the cases, all of the guesses.
So that's it.
That's our recurrence relationship corresponding to the DP for this particular problem.
You can go figure out what the complexity is.
I have other things to do, so we'll move on.
But you can do the same things, and these are fairly mechanical at this point to go write code for it, trace the solution to get the optimum binary search tree, yadda, yadda, yadda.
Any questions about this equation or anything else?
Do people get that?
Yeah, go ahead
What is the depth input?
So the depth is getting added by the weights.
So basically what's happening is that as I go deeper into the recursion, I'm adding the weights and potentially multiple times, depending on the depth.
So if you really think about it, this Wij, I added all of these weights.
But when you go into this recursion, into the ei of r minus 1, well, you're going to see i to r minus 1 in the next level of recursion.
So you would have added the weight again.
So it's not the case that the weights are only appearing once.
They're, in fact, appearing this many times.
So that's what kind of cute about this, the way we wrote it.
Any other questions?
All right, good.
So we're done with that.
So one last example.
This is, again, a little bit different from the examples of DP we've looked at up until now, because it's a game.
And you have an opponent, and you have to figure out what the opponent is going to do and try to win.
I suppose you could try to lose as well.
But let's assume here, same thing with minimization and maximization, most of the time you can invert these cost functions, and DP will still work.
But let's assume here that you want to win this game.
So the game is an alternating coins game where we have a row of n coins of values V1 through Vn.
These are not necessarily in any particular order.
And n is even.
And the goal here is select the outer coins.
So select either the first our last coin from the row, and then the opponent plays.
Remove permanently, but add it to your value and receive the value.
So I need two volunteers to play this game.
And you want to maximize the value.
The winner gets a blue Frisbee.
The loser gets a purple Frisbee, because blue is greater than purple.
And you might ask, why is that.
Well, if you went to a beach, and you saw water this color, and you went to another beach, and you saw water this color, which beach would you pick?
Right?
This is Cozumel.
This is Boston Harbor.
So blue is greater than purple.
Don't use this proof technique in the quiz.
So do I have a couple of volunteers to play this game?
Two of them over there.
Are you waving for him?
Yeah.
I see one.
You can come down.
Another volunteer?
Yeah.
Over there.
You don't get your Frisbees yet.
So we're going to make this really fair.
What's your name?
Josiah
Josiah
Tessa
Tessa.
Josiah and Tessa, I'm going to write out a bunch of-- it's going to be a fairly short game.
And we're going to be really fair, and we're going to flip a coin to decide whether Josiah goes first-- you can pick heads or tails-- or whether Tessa goes first.
Actually if you win, you can let her go first if you want.
But you get to choose
All right
So pick
Heads
That's heads.
Do you want to go first, or do you want to let Tessa go first?
[LAUGHTER]
You should go first
All right.
So Tessa, you get to go first
OK, 6
6, OK.
So let's just say T. So you get to choose either 25 or 4, Josiah
I think I'd pick 25
You think you'll take 25.
So that's j.
So it's not on the 4 and 19 over here because those are gone.
So your turn again
OK. Can I take a minute, or should I just go?
Well, you can take 30 seconds
19
19,
4.
[LAUGHTER]
All right, 4
4.
All right.
And 42?
Yeah
All right, 42.
This is one strange game.
Now, we get to add up the numbers.
This is going to be tight.
4 plus 39 is 43.
43 plus 25 is 68.
42 plus 19 is 61.
61 plus 6 is 67.
Ooh.
Well, you get Frisbees.
Well, blue for you.
I mean, you could give her blue if you like
I prefer purple
You prefer purple.
Thanks.
Good job.
[APPLAUSE] No offense is intended to Josiah and Tessa, but that was a classic example of how not to play the game.
First off, Josiah could have won regardless of the coins that were up there, regardless of the values of the coins, if he'd chosen to go first.
So this game, the person who goes first is guaranteed to not lose.
And by that, I mean you may have a situation where you can have a tie in terms of the values, but you're guaranteed to not lose.
Now, he ended up winning anyway, because there were other errors made during this, and I don't have the time to enumerate all of them.
[LAUGHTER] So we're just going to go move on and do the right thing.
So let me first tell you, outside of DP, just in case you play this game over Spring Break or something, how you can win this game without having to compute complicated recursive memoization DP programs for this case.
So let's say I have V1, V2, V3, all the way to V n minus 1 and Vn.
And remember n is even.
So you've got to pick n over 2 coins if you're the first player and n over 2 if you're the second player.
So what the first player does is simply compute V1-- take the odd numbers-- and n is even, so you've got V n minus 1 being odd.
Compute V1 plus V3 all the way to V n minus 1, and compare that with the even positions, V2 plus V4 all the way Vn.
So in this particular instance, Josiah, who went second, if you just look at the odd positions, which is, in fact, what he ended up picking, it was 4 plus 39 plus 25, which was 68.
So you can do this computation beforehand.
You compute this to be 68, in our case.
Compare that with 67.
And you don't give up your first-player advantage.
So you're going with the first player.
But now you say, I know that I can set this up-- and that wasn't the order that he did this-- but I know I can set this up so I always have a chance to get V1, V3, V5, et cetera.
Because let's just say that the first player Josiah started first.
And he decided, in this case, that the odd values are the ones that win.
So let's say he picks V1.
At this point, Tessa sees V2 through Vn.
So you're just looking at that.
Now Tessa could either pick V2 or she could pick Vn.
In that case, Josiah, who's the first player, could pick V3 or Vn minus 1 and stick with his rule of picking the odd coins.
So regardless of what Tessa does, the first player, Josiah, could pick odd if odd was the way to go or if the values were such that, even with the way to go, he could go even.
So no reason for the first player to lose.
But let's just say that you're a nasty person, Michael Jordan nasty.
You want to just press your opponent.
You want to make sure they never want to play you again.
So now you have a maximization problem.
So this is clearly not DP.
You don't need DP to add up a bunch of numbers.
But let's say that you want to, given a set of coins, V1 through Vn, you want a dynamic strategy that gives you the maximum value.
This odd versus even is a bit constraining, because you're stuck to these positions.
And it's good to be in that situation if you just want to win and you don't care how you win.
But if you want to maximize, then it's a more complicated problem.
And so we have to talk about how you would do something that would give-- maybe it would be V1 and V4 and something else, depends on the values.
How would you get to a situation-- if you're the first player.
We'll just stick with you're the first player.
You know you can't lose using this strategy.
But not only do you want to not lose, you want to get as many coins as possible-- let's say this is money.
More money is better, and that's your goal.
So people understand that.
A little trick there with, I guess, a greedy algorithm you can think of.
Or it's not really even something that you might want to call an algorithm.
It's a little computation.
And our goal now is to maximize the amount of money when assuming you move first.
And so this is going to be a little bit different from all of the other problems we've looked at, because we have to now think about what the opponent would do.
And there's going to be a sequence of moves here.
You're going to move first, so that one is easy.
You've got the whole problem.
But now you have to say, the opponent is going to move and pick a coin.
And then you have to think of your subproblems as the potential rows of coins that are going to be different, depending on what the opponent does.
And through this process, you have to maximize the value.
So it's really pretty different from the other couple of examples that we looked at.
So we're going to have to do a little bit of set-up before we get to the point where we can write something like this, which is the solution to our DP.
And so let me do that set-up.
But it's not super complicated.
And part of this is going to look kind of the same as previous problems we've looked at.
So Vij is the max value we can definitely win if it is our turn.
And only coins Vi through Vj remain.
And so we have Vii.
Then there's only one coin left, and you just pick i.
Now, you might say, but that's never going to happen if there's an even number of coins and I'm playing first, because the other player is going to be at the end of the-- is going to be the person who picks the last coin.
But we need to categorize Vii because we need to model what the other player is going to do.
So we can't just say that we're going to be looking at an even number of coins in terms of the row of coins that you look at, which is true, that when you get to move, you only see an even number of coins if you're the first player.
But you do have to model what the opponent does .
So we need the Vii.
And what you might see, of course, is a board that looks like Vi i plus 1, for some arbitrary i.
So that's why I have it up there.
But remember that you're only picking coins on the outside, so it's not like you're going to have a gap.
I mean, you're not going to have V3 left and V7 left.
There's no way that's going to happen.
You're just going to keep shrinking, taking things from the left or the right.
So it's going to be i and i plus 1.
That make sense?
And so in this case, what would you pick?
Vi and i plus 1.
You just pick the max.
Because at the end of this, either you did it right or you did it wrong.
Either way, you're going to improve your situation by picking the max of Vi or Vi i plus 1.
So there's no two things about it.
So here, you're going to pick the maximum of the two.
And you might have Vi i plus 2, which is an odd number of coins that your opponent might see.
It gets more complicated for Vi and i plus 2.
We're going to have to now start thinking in more general terms as to what the different moves are.
But we've got the base cases here.
All I did here was take care of the base case or a couple of base cases associated with a single coin, which is what your opponent will see and pick, or two coins, which is your last move.
So with DP, of course, you always have to go down to your base case, and that's when things become easy.
So we have to talk about two things and put up our recurrence together.
The two things we have to talk about are what you do when you move, and that's actually fairly easy, and the second thing is what the model of the opponent looks like when you're waiting for him or her to move.
So let's take a look at your move.
And I've got V1.
Let's look at Vi.
Vj here, dot dot dot, Vn.
So that's what you see.
And you're looking at i and j.
And at this point, you're seeing all those other coins, the outer coins have disappeared into people's pockets.
And you're looking at this interval.
So I'm going to write out what Vij should be.
And keep in mind that we want to maximize the amount of money.
And we want to say that you should be able to definitely win this amount of money, regardless of what the opponent does.
So I want to do a max, obviously.
And I have two choices here.
I can pick Vi or I can pick Vj.
It's not like there's a lot of choices here.
So if I pick Vi, then-- let me go ahead and say I pick Vi here.
And here, I pick Vj.
Let me draw this out a little bit better.
So I've got to fill in these two arguments to my max.
So this is easy.
I'm going to have a plus Vi here.
Going to have a plus Vj here because I picked the appropriate value.
And now, this is also not that difficult.
What exactly happens?
What can I put in here if I pick Vi?
Vi plus 1
Yeah.
Vi plus 1 to j.
So the range becomes i plus 1 j. I have to be a little careful here in terms of whether I can argue that it's actually the V that I put in here.
So the subtlety here is simply that the Vi plus 1 j is not something that I see in front of me.
This is the complication.
Vi plus 1 j is never a board that I see in front of me, whereas Vij was a board that I saw in front of me.
So I have to model the boards that I see in front of me, because those are the boards that I have control over that I can maximize.
I cannot would be Vi plus 1 j in there, simply because I don't quite know what that is, because of what I get eventually is not something I control.
It's going to be the board after my opponent has moved.
So all I'm going to do is I'm going to say the range becomes-- range is i plus 1 j.
And I'm going to say something else in a second here.
The range is i j minus 1.
And in both of these cases, the opponent moves.
So in order to actually write out my DP, I'm going to have to now look at the worst case situation in terms of the board I get back, because the only times I'm adding values is when I see a board in front of me and I pick a coin.
And I'm going to have to say now the opponent is going to see an i plus 1 j-- or an i j minus 1, excuse me, and might do something.
And I'm going to get something back.
I'm going to assume that the opponent is just as smart as I am, knows DP, taken 6046, et cetera, et cetera.
And I still want to get the maximum value that I can definitely win.
And so we need to look inside of that a little bit.
And it's not that hard if you just make the assumption that I just did, which is the opponent is going to do-- I mean might not necessarily do the right thing.
But you have to assume that the opponent knows just as much as you do and is going to try and do as well as possible.
So let's do that.
That's the last thing that we have to do here.
And it's just one more equation.
So here's the solution.
We now have Vi plus 1 j subproblem with the opponent picking.
And the simple observation is that we are guaranteed the min of Vi plus 1 j minus 1 or Vi plus 2 j.
In this case, the opponent picks Vj, and in this case, the opponent picks Vi plus 1.
And you're guaranteed the minimum of these two, because the opponent can only pick one coin.
So that's the simple observation that lets you jump ahead to your next move, which is really what the DP is interested in, because that's the only time that you're actually executing something in terms of picking coins from the board and adding to your value.
But we did have to model that as to what the maximum value was that you could win, jumping ahead of this move.
And you have the min in here because you're assuming that this is a definite guarantee.
That's what we want.
It's possible that the opponent plays a different game from the one that we think he or she is going to play, which maximizes his or value.
But the min is guaranteed.
So now that you've made this observation, it turns out we just have to take that, and we'll be done.
So let's see.
Let me erase this.
It's the last set of equations.
And it's just plugging in that observation into what I wrote before.
So we have Vi j equals max.
This is the outer max that I had up there already, so that's the same max.
And put these giant brackets here.
And inside, I'm going to plug in this min, which is something that corresponds to the value that I would win in the worst case after the opponent plays the best possible move.
And that would be Vi plus 1 j minus 1, V1 plus 2 j plus Vi.
This Vi is the same as this one up here.
And you've got a plus Vj here.
And I didn't actually do this, but in terms of writing out what the opponent would do in other subproblem case, but it's really pretty straightforward.
You have a problem that corresponds to the i j minus 1 problem.
And the opponent could pick the i-th or could pick the j minus 1th.
If the opponent picks the j minus 1th, you get Vi j minus 2, and you need to take the min of that.
In the other case, where the opponent picks i, in which case you get i plus 1 j minus 1.
And that's our DP.
That's our DP.
So the big difference here was the modeling that you had to do in the middle.
We didn't actually have to do that in any of DPs we've covered in 046 at least up until this point.
Before we talk about complexity, that should just take a minute, people buy that?
See that?
Good.
So with all of these problems, the complexities are fairly straightforward to compute.
The complexity here is simply, again, as before, the number of subproblems times the time it takes to solve the subproblem.
You can see here are that these are all constant time operations.
So assuming that the Vij's have been computed, it's theta 1 time to compute a subproblem.
So that's your theta 1.
And as before, there are n squared subproblems.
Sorry.
Let's just do number of subproblems times theta 1.
It's just theta n squared.
So that was good.
All right.
So good luck for the quiz.
And don't worry too much about it.
See you guys next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started.
Welcome back to 6046.
Today, we start an exciting series of algorithms for graphs.
We've done a lot of data structures.
We're starting to get back into algorithms with dynamic programming last week.
And today and the next few lectures, we're going to see lots of cool algorithms about graphs.
First a bit of recall-- we're starting with shortest paths, which you've seen in 6006 in the context of single-source shortest paths.
So typically, like you do a Google Maps query, you think of, I want to go from A to B.
But what you solved in 6006 was a harder problem, which is I give you a point A-- here it's called S for the source.
I give you a source vertex.
And capital V is a set of all vertices, capital E is a set of all edges, remember graph notation.
Let's say it's a directed graph.
You've got edge weights, like the time it takes to traverse each road.
And you want to know how long's it take me to get from S to V for all V. So this is from one given point to everywhere.
Today, we're going to solve a harder problem, which is all-pairs.
I want to go from all A to all B.
But what you saw in 6006 was single-source, where I just give you one of the vertices, and I want to know how to get to everywhere.
The reason you saw this version and not the A to B version is because the best way we know to solve A to B is to solve this problem.
So at least from a theory standpoint, we don't know how to beat Dijkstra's algorithm and Bellman-Ford's algorithm for the A to B problem.
So you get a little bit more than what you asked for sort of for the same price.
So let me remind you in a few different scenarios what algorithms we have, and how long they take.
So the scenarios of interest are the unweighted case, a non-negative weighted case, the general case, arbitrary weights, positive and negative, and DAGs acyclic graphs.
These are some interesting special cases.
And you should have seen in 006 algorithms for each of them.
Let's see if you remember.
So what's a good algorithm for single-source shortest paths in an unweighted graph?
BFS, good, Breadth-first Search, that takes how long?
V plus E, good.
That's-- for graphs, V plus E is considered linear time.
That's how long it takes to represent the input.
So you got to look at the input, most algorithms, and the BFS is optimal against that.
But we're going to start getting worse as we-- well, for these two situations.
So for non-negative edge weights, what do you use?
Dijkstra.
Ah, everyone's awake this morning.
That's impressive.
And that takes how long?
This is a tricky question.
V log V plus E, wow, nice.
So this answer kind of depends on which heap structure you use, but this is the best we know.
If you use a Fibonacci heap, which we don't actually cover but it's in the textbook, you achieve log V for extract key and constant amortized for each decreased key operation.
So sorry, this is for extracted min, and this is for decrease key.
And so this is the best we know how to do with a Dijkstra-type approach.
If you use other heaps, you get slightly worse, maybe you get a log factor here.
But this is good.
This is almost as good as V plus E. For moderately dense graphs, if E is bigger than V log V, then these are the same.
But if your graph is sparse, like E is order V, then you lose a log factor.
But hey, it's just a log factor, not too bad.
We're going to get worse.
So for general weights, what do you use?
Bellman-ford.
OK.
Which takes how long?
VE, that's the usual statement.
Technically, you should assume VE is at least V for this bound to hold.
But that's the way to think of it.
So this is not nearly as good.
This is a lot slower.
If you think of-- we can think of two situations.
One is when E is theta V, so a very sparse graph like a tree or planar graph or something.
And we could think of when E is quadratic.
That's the dense case.
So here we get, whatever, V and V squared for BFS.
For non-negative edge weights we get V log V in the sparse case.
And we get V squared in the dense case.
And for Bellman-Ford, we get V squared in the sparse case, and V cubed in the dense case.
So this is like a V factor, a linear factor larger than non-negative edge weights-- makes a huge difference.
And finally for acyclic graphs, what do you do?
Dynamic programming
Dynamic programming is one answer, yeah.
That works.
In some sense all of these algorithms are-- especially Bellman-Ford is a dynamic program.
We'll see that little bit.
Another interpretation?
Topological sort, and then Bellman-Ford, yeah-- say, one round of Bellman-Ford.
So Bellman-Ford actually works really well if you know the order you should relax the edges.
And if in an acyclic graph, you can do a topological sort, meaning you visit all the vertices, so that whenever you visit the right endpoint of an edge, you've already visited the left endpoint.
If you do Bellman-Ford in that order, then you only have to do one pass and you're done.
Whereas normally, here, you had to do it V times.
So the total cost of this is just linear.
Good thing to remember, especially on quizzes and so on.
If your graph is acyclic, you can achieve linear time.
But in the general case, Bellman-Ford is your answer for single source.
Now, these are the best algorithms we know for each of these cases.
So I'm not going to improve them today.
You saw the state of the art 006.
But for all-pair shortest paths, we can in some sense do better, sort of.
So let me just quickly define the problem, and then tell you all of the results we know.
And also, the results we're going to cover today.
I didn't remind you of the delta definition.
I want to go over this briefly.
So delta of s comma v is the weight of the shortest path from S to V. The weight is well-defined.
Even though there may be many shortest paths, there's one best weight.
But there's some special cases.
It could be infinity, if there's no path.
That's sort of by definition.
Say, well, it's infinite costs to get-- if there's no path, then we said there's infinite weight one.
And it could be minus infinity in the presence of negative weight cycles.
So let's say, if there's a negative weight cycle on the way, if you could reach a negative weight cycle from s, and then still get to V from there, then the best way to get there is to go to that cycle loop around infinitely many times, and then go to V. OK, so the algorithms you saw probably didn't actually compute correctly in this case.
They just said, negative weight cycle-- I don't know what to do.
But it's actually not that hard.
With a little bit more effort, you can figure out where the negative infinities are.
We're not going to rely on that, but I'm just throwing it out there to make this a well-defined definition.
Once you have the shortest path weights, you can also store parent pointers, get the shortest path tree, then you can actually find shortest paths.
But again, we're not going to talk about here.
We'll focus on computing delta, but with the usual techniques you saw in 006, you could also reconstruct paths.
So for all-pairs shortest paths, we have a similar set-up.
We have a directed graph, V,E.
And we have an edge weight function w-- in general, could have negative weights.
And our goal is to find delta of u comma v for all u and v. OK. Single-source shortest paths is the sort of thing that you might want to do a few-- just given a graph, and you want to find a shortest path from A to B. I said, this is the best way we know how to do A to B, essentially.
But all-pairs shortest paths is what you might want to do if you're pre-processing.
If you're Google Maps, and you want to be able to very quickly support shortest path queries between major cities, then you may want to first compute all-pair shortest paths for all major cities, because road networks don't change very much, the large scale.
This is ignoring traffic and so on.
Pre-compute this, and then given a query of two vertices, come in-- probably get a million queries a second-- you could very quickly know what the answer is.
And this is the basis for real world shortest paths.
Typically, you don't compute shortest paths from A to B every single time.
You use waypoints along the way.
And you have pre-computed all-pair shortest paths between waypoints.
So that's the motivation.
Yeah, I guess in some sense, internet routing is another situation where at any moment you may need to know the shortest path to get to-- the fewest hop path say to get to an internet site.
You know the IP address.
You need to know where to go.
You don't need to know the whole path.
You need to know the next step.
But in some sense, you're computing all-pair shortest paths.
That's a more dynamic situation.
OK.
So here are the results we know for all-pair shortest paths.
I think I'm going to cheat, and reuse this board.
So same situations, except I won't think about acyclic graphs here.
They're a little less interesting.
Actually now, I'm curious, but I didn't intend to talk about acyclic.
And so the obvious thing to do to solve all-pairs shortest paths is just run the single source algorithm V times, once from each source.
So I could do V times Breadth-first Search, V times Dijkstra, V times Bellman-Ford.
And now, I just need to update my bounds.
OK so VE becomes V squared plus VE.
If you're a little bit clever, or you assume E is at least V, that becomes VE.
If I run Dijkstra V times, I'm going to get V squared log V plus V times E. And if I run Bellman-Ford V times, I get V squared E. OK. And over here, everything's just going to increase by a V factor.
So a little more intuitive is to think about the sparse case.
I get V squared, V squared log V, and V cubed.
Check that those match over there-- 1 equals V. And over here, I get V cubed, V cubed, and V to the fourth.
OK.
So pretty boring so far.
The interesting thing here is that we can beat the last result.
The last result, which is the slowest one, could take as long as V to the fourth time.
We can shave off a whole V factor.
So a better general case algorithm is called Johnson's algorithm.
That will be the last algorithm we cover today.
And it achieves this bound, which is the same as running Dijkstra V times.
So it's between V squared log V and V cubed.
And that's cool, because this is the best algorithm we know for all-pairs, non-negative edge weight, shortest paths, just running Dijkstra V times.
Not very smart-- but it's the best thing we know how to do.
And what this says is, even when we have negative edge weights, actually we can achieve the same bound as running Dijkstra.
This is a bit counter intuitive because in 006 you're always told, if you have negative edge weights, can't use Dijkstra.
Turns out, in the all-pairs shortest paths case, you kind of can.
How can that be?
Because this is a harder problem.
If you could solve all-pairs shortest paths, of course you could solve single-source.
And that's actually the luxury.
Because it's a harder problem, we have this VE term in the running time, which lets us do things like run Bellman-Ford once.
And running Bellman-Ford once will let us run Dijkstra V times.
That's the reason we can achieve this bound.
But we won't be seeing that for a while.
For starters, I want to show you some connections between all-pairs shortest paths, and dynamic programming, and matrix multiplication, which turn out to give-- for dense graphs, we're just achieving V cubed in all situations.
So our first goal is going to be to achieve V cubed time for general edge weights.
So we're going to first achieve this bound.
That will be a lot easier.
And then eventually, we will achieve this bound.
So the Floyd Warshall algorithm and some of these will get very close to V cubed.
All right.
So we're going to start with our first approach to solving all-pairs shortest paths-- that is not using an existing single source algorithm-- is dynamic programming.
Someone mentioned that already.
It's a natural approach.
Shortest paths is kind of dynamic programming.
In fact, most dynamic programs, you can convert to single-source shortest paths, typically in a DAG-- not all, but a lot of them.
So we could try dynamic programming.
Now, I'm going to preach to you a little bit about my way of thinking about dynamic programs.
If you watched the 006 OCW version, you've seen the five easy steps to dynamic programming.
And if you haven't, this will be new, otherwise, a reminder.
First thing I like to think about in a dynamic program is, what are the subproblems?
The second thing I like to think about is, what am I guessing?
I'm going to guess some feature of the solution.
Third thing is, I want to write a recurrence relating subproblems solutions.
Then, I'm basically done, but there's a couple wrap up things, which I'm going to have to use another board for.
So number four is, I need to check that I can actually resolve these subproblems in some order that's valid.
Basically, this is saying that the constraint graph on some problems should be acyclic.
Because if there's a cycle in the constraint graph, you take infinite time.
Even if you memoize, if you do infinite recursion-- bad news.
You'll never actually finish anything, so you never actually write anything in the memo table.
So I want to make sure that it's acyclic.
I mean, this is really the same thing we're talking about in this erased row, which is topological ordering.
Personally, I like to-- you could argue that it's acyclic.
I like to just write down, here's a topological order.
That's a nice proof that it's acyclic.
If you write that down as for loops, then you actually have a bottom up dp.
If you just take the recurrence, stick it inside the for loop, you're done, which we'll do in a moment.
I guess I need a row for that.
And finally, you need to solve the original problem.
And then, there's analysis and so on.
But for specifying the algorithm, these are the key things you need to know.
The hard part is figuring out what the subproblems should be, so that your dp becomes fast.
Running time is going to be number of subproblems times time per subproblem.
For each subproblem, usually we're going to want to guess some feature of the solution to that problem.
Once we do that, the recurrence becomes pretty trivial.
Just for each guess, you say what it should be.
So these are the really hard two steps.
And then, OK, we checked that it's acyclic.
And we make sure that we can actually solve our original problem using one of the subproblems.
Sometimes, our original problems are some of the subproblems.
I think that will happen here.
But sometimes you need to do a little bit of post-computation to get your answer.
All right.
So what am I going to do for subproblems?
Well obviously, I have a bunch of different problems involving pairs of vertices.
I want to find delta of u,v for all u and v. That, I erased.
But that's the problem.
So I want to know what is the weight of the shortest path from u to v?
If I stop there and said that was my subproblems, bad things are going to happen, because I will end up-- there's no natural way to make this thing acyclic.
If I want to solve u to v using, I don't know, u to x, and then x to v, something like that-- there's no way to get out of the infinite recursion loop.
OK?
So I need to add more subproblems to add more features to my solution, something that makes it so that when I try to solve my subproblem, I reduce it to other subproblems.
Things get smaller, and so I could actually make progress.
So there are actually two natural ways to do this.
I'll call them method one a method two.
Method two is actually Floyd Warshall.
But any suggestions on how we might do this?
This is a harder problem.
This is in some sense, a kind of guessing, but it's like I'm going to guess ahead of time that somehow there's an important feature of the shortest path.
I'm going to parameterize by that, and somehow it's going to get smaller.
Yeah?
[INAUDIBLE]
Right, shortest path-- it uses at most a given number of edges.
Let's parameterize by how many edges.
I think I'll use m. So using at most m edges.
Very good.
So good, I think it deserves a purple Frisbee.
All right, I'm getting better, slowly.
By the end the semester, I'll be pro at frisbee.
I should enter a competition.
So this is, of course, an additional restriction.
But at the end of the day, the problem I want to solve is going to be essentially duv, let's say, n minus 1.
If I want a shortest path that does not repeat any vertices, then certainly it has at most n minus 1 edges.
So in fact, the claim would be that duv equals duv n. I mean, and so on.
If you go larger than n minus 1, it shouldn't help you.
If you know that your shortest paths are simple-- if you know that shortest paths don't repeat vertices.
So this would be if there are no negative weight cycles.
If there are no negative weight cycles, then we know it never helps to repeat vertices.
So in that situation, we would be done, if we could solve this for all m. Now, slight catch-- well, how do we know there's no negative weight cycles?
You know, we could run Bellman-Ford, I guess.
That's a little tricky, because that only finds reachable negative weight cycles.
In fact from this picture, we will end up knowing whether there are negative weight cycles.
So there will be no negative weight cycles, if and only if there's no negative diagonal entry, say dvv n minus 1.
So it turns out, this algorithm will detect there's a negative weight cycle by finding that the distance from v to v is negative.
Initially, it's going to be 0.
If it turns out to be negative, we know there's negative weight cycle.
With more work, you could actually find all the reachable pairs, and so on.
But I'm not going to worry.
I'm just going to say, hey, a negative weight cycle.
I'm going to throw my hands up in the air and give up for today.
OK.
Cool.
So I can solve my original problem, if I can solve these subproblems.
And now, things are easier, because we can essentially assume in solving this problem that we've solved smaller subproblems for however we define smaller.
That's what's given by this topological order.
Obvious notion of smaller is smaller m. Presumably, we want to write this with an m in terms of this with an m minus 1 or smaller.
So this gets to our guessing part.
What feature of a shortest path could we guess that would make it one edge shorter?
There's probably two good answers.
Yeah?
The next edge, which I guess you mean the first edge?
Sure.
Could guess the first edge, or you could guess the second edge.
Or no, that would be harder.
Or I mean, the last edge, that would also work.
Okay, this is a harder one.
Uh, nope.
Good thing there's students everywhere to catch it.
Cool.
So I'm going to guess the last edge.
That's just how I've written the notes.
But first edge would also work fine.
So I'll call the last edge x comma v. We know we end by going into v. So let's guess the vertex previous to it in the shortest path.
Now of course, we don't know what that edge is.
The guessing just means try them all as you saw last time.
So now, it's really easy to write the recurrence-- see if I can do it without looking at my notes.
So we've got duv of m. We want to write-- we want to find the shortest path, so it's probably going to be a min on the outside.
And we're going to consider the paths of the form-- d go from u to x using fewer edges.
Right, if this is the last edge, then we use m minus 1 edges to get to x.
And then, we follow the edge x comma v. So I'll just add on the weight of the edge, x comma v. If x was the right answer, this would be the cost to get from u to v via x, where xv is a single edge at the very end.
We don't know what x should be, so we're just going to do for loop over x for x in v. So this is using Python notation.
And that will find the best answer.
Done, easy.
Once you know what the subproblems are, once you know what the guessing is, basically, I'm just adding in a min and a for loop to do the guessing.
So that's my recurrence, except I should also have a base case.
Here it's especially important.
So base case is going to be when m is the smallest.
So that, let's say, is 0.
What is the weight of getting somewhere using 0 edges?
Well, typically it's going to be infinity.
But there is an interesting situation, where at 0 namely when u equals v. Hey, there is a way to get from u to itself with 0 edges.
And that costs 0.
But anywhere else is going to cost infinity.
There's no path.
So the sort of a definition, I should say.
If it exists, otherwise it's infinity.
So then I get those infinities.
But this is kind of important, because maybe I actually use fewer than m edges.
I wrote less than equal to m here.
This is also less than equal to m minus 1 edges here.
But in some sense, I'm including the case here, where x equals v. So I just stay at v at the very end.
So it's because I have a 0 here, I'm implicitly including a situation where actually, I just use duv m minus 1.
That's in that min here.
So it's important to get the base case right.
Cool.
Almost done.
I need an acyclic ordering.
So as I said, things get smaller when m is smaller.
So all that means is do the for loop for m on the outside.
Then do the for loop for u in v. For those, it doesn't matter what order you do it, as long as you've done all of m equals 0, before you do all of m equals 1, before you do all of m equals 2.
So that's the nested for loops that gives you the right order.
And so, I guess I take this line and put it on the top.
And I take this line, the induction of the currents, put it inside the for loops, that is my bottom-up dp.
OK, I'm actually going to write it down explicitly here for kicks.
Should I bother?
Uh, what the hell.
So first we do a for loop on m. Then, we're going to do the for loops on u in V. Now, inside the for loop, I want to compute this min.
I could use this single line, but I'm going to rewrite it slightly to connect this back to shortest paths.
Because this type of statement should look familiar from Dijkstra and Bellman-Ford.
This is called the relaxation step.
OK.
It probably would look more familiar if I wrote here w of x v. You could also write wxv.
That's an alternative for both of these-- probably should write the same way.
But in either case, we call this a relaxation step, because-- it's kind of a technical reason but-- what we'd like to satisfy-- we know that shortest paths should satisfy the triangle inequality.
If you look, there's three vertices involved here, u, v, and x.
We're looking at the shortest path from u to v compared to the shortest path for u to x, and the shortest path from x to v. Certainly, the shortest way to get from u to v should be less than or equal to the shortest path of u to x plus x to v, because one way to get from u to v is to go via x.
So this if condition would be a violation of the triangle inequality.
It means we definitely do not have the right distance estimates if duv is greater than ux plus xv.
OK.
So if it's greater, we're going to set it equal.
Because here, we know a way to get from u to v via x.
We know that it's possible to do this, assuming our d values are always upper bounds on reality.
Then, this will be an upper bound on the best way to get from u to v. So this is clearly a valid thing to do.
Relaxations are never bad.
If you start high, these will always improve your shortest paths, so you get better estimates.
And that's exactly what Dijkstra and Bellman-Ford did, maybe with w instead of d, but they're all about fixing the triangle inequality.
And in general and optimization, there's this notion of, if you have a constraint, an inequality constraint like the triangle inequality, and it's violated, then you try to fix it by the successive relaxations.
So that's where the term comes from-- doesn't really matter here, but all of our shortest path algorithms are going to do relaxations.
All the shortest path algorithms I know do relaxations.
So this is familiar, but it's also doing the same thing here.
I just expanded out the min as a for loop over x, each time checking whether each successive entry is better than what I had already.
And if it is, I update.
So in the end, this will compute a min, more or less.
I cheated also because I omitted the superscripts here.
If I put m here, and m minus 1, here and 1 here, it would be exactly the same algorithm.
I'm omitting all the superscripts, because it can only help me.
Relaxing more can only get better.
And if I was guaranteed correct over there, I'll still be guaranteed correct over here.
You have to improve the invariant, but you never-- relaxation is always safe.
If you start with upper bounds, you always remain upper bounds.
You're doing at least the relaxations over there.
And so you will in the end compute the correct shortest path weights.
The advantage of that, mainly, is I save space.
And also, it's simpler.
So now I only need quadratic space.
If I had a superscript, I'd need cubic space.
Right?
So I did a little simplification going from the five step process to here-- both of the polynomial time and space, but this is a little bit, a little bit better.
But how slow is this algorithm?
How long does it take?
Yeah?
V cubed, that would be great.
V to the fourth, yeah, good.
Sadly, we're not doing so great yet-- still V to the fourth.
V to the fourth, I guess I already knew how to do.
That was if I just run Bellman-Ford V times, I already knew how to do V to the fourth.
So I haven't actually improved anything.
But at least you see, it's all dynamic programming in there.
So n here is the size of V. That's probably the first time.
Cool.
I omitted 0, because there was the base case.
That's done separately in the line that I didn't write.
So that was dp one.
Time for dp two, unless there are questions?
Everyone clear so far?
Yeah?
When you iterate over x, why do you do you every [INAUDIBLE]
As opposed to--?
Like, just adjacent vertices [INAUDIBLE]
Oh, yeah.
OK. Good, fair question-- why do I iterate over all vertices, not just the incoming?
If I'm writing w of xv, I could afford to just say, just consider the incoming vertices.
And that would let me improve probably from V to the fourth to V cubed times-- V squared times E, I think, if you do the arithmetic right.
You could do that.
It would be better.
For dense graphs, it's not going to matter.
For sparse graphs it will improve V squared to E, basically.
But we're going to do even better, so I'm not going to try to optimize now.
But, good question.
When I say this at the moment, if there is no edge from x to v, I'm imagining that w of xv equals infinity.
So that will never be the minimum choice to use a non-edge.
I should say that.
If there's no edge here, I define the weight to be infinity.
That will just make algorithms cleaner to write.
But you could optimize it the way you said.
So, where were you?
Yeah.
Ah, perfect.
OK. Other questions?
More Frisbee practice?
No?
OK.
So that was dp one.
Let me do dp two.
Not yet, sorry-- diversion.
Diversion is matrix multiplication.
Before I get to dp two, I want to talk about matrix multiplication.
This is a cool connection.
It won't help us directly for shortest paths, but still pretty good.
And it will help-- it will solve another problem especially fast.
So shortest paths is also closely linked to matrix multiplication, a problem we've seen a couple of times, first in the FFT lecture, and then in the randomization lecture for checking matrix multiplies.
So you remember, you're given two matrices, A and B.
And you want to compute C equals A times B.
You've seen Strassen's algorithm to do this.
There's also these-- here A and B are squared.
OK.
So the n by n, product will be n by n. So standard approach for this is n cubed.
With Strassen, if I can remember, you can get n to the 2.807.
And if you use CopperSmith Winograd, you get 2.376.
And then, if you use the new Vassilevska-William's algorithm, you get n to the 2.3728 and so on.
And that's the best algorithm we know now.
There's some evidence maybe you can get 2 plus epsilon for any epsilon.
Turns out, those are going to help us too much here.
But what I want to show is that matrix multiplication is essentially doing this, if you redefine what plus and dot mean.
We redefine addition and multiplication-- talk about whether that's valid in the moment-- so remember what is matrix multiplication?
Cij is a dot product of a row and a column.
So that's aik with bkIj.
K equals 1 to n. OK. Now that sum looks a lot like that min.
Actually, more like the way I wrote it over here, with the d's instead of the w's.
This-- right-- x is the thing that's varying here.
So this is like, aik plus bkj, except, I have plus here, whereas I have times over here.
And I have a min out here, but I have a sum over here.
So it sounds crazy, but let's define-- be very confusing if I said define dot equals plus, so I'm going to define a new world called circle world.
So if I put a circle around a dot, what I mean is plus.
And if I put a circle around a plus, what I mean is min.
OK?
So now, if I put a circle around this dot, I mean circle everything.
So I've got to circle the summation, circle this thing.
So then I get shortest paths.
Crazy.
So all right, I'm going to define d to the m-th power.
I should probably circle-- whatever.
Slightly different.
OK, I want to define, like, three things at once.
So let me write them down, and then talk about them.
So many infinities.
All right.
OK.
I guess, I should write this too.
OK.
If I define the vert-- suppose I number the vertices 1 through n, OK?
I just assume all vertices are an integer between 1 and n. So then, I can actually express things in a matrix, namely the weight matrix.
This kind of defines the graph, especially if I say wij is infinity if there is no edge.
Then, this is the matrix of all pairwise edge weights.
For every i and j, I have a weight of ij.
That gives me a matrix, once I set V to be 1 through n. Now, I'm also defining this distance estimate matrix.
So remember, we defined duvm-- I'm going to now call it dijm, because the vertices are integers.
That is, the weight of the shortest path using, at most, m edges.
If I define it that way, then I can put it into a matrix, which is for all pairs of vertices ij.
What is the distance, shortest pathways that uses it at most m edges?
That gives me a matrix, d parenthesis m. Then, I claim that if I take circle product between d of m minus 1 and w, that is exactly what's happening here, if you stare at it long enough.
This is the inner product between row u of d to the m minus 1, and column v of w. And that's exactly what this circle product will compute.
So this is dp.
But when you look at that statement, that's saying that d to the parentheses m is really w to the circle power m, right?
This is a definition in some sense of power, of exponentiation, using circle product.
So when I circle the exponent, that means I'm doing circle exponentiation in circle land, OK?
OK so far?
So this is circle land.
So you might say, well, then I should compute these products using matrix multiplication.
Now, just to see how good we're doing, if I execute this operation n times, because I have to get to d to the n minus 1-- so it's basically d to the n. If I do this product n times, and for each one of I spend n cubed time, then I get an n to the four algorithm.
Same algorithm in fact, exactly the same algorithm-- I've just expressed it in this new language.
OK, there are two ideas on the table though.
One is, maybe I could use a better matrix multiplication algorithm.
Let's shelve that for a moment.
The other possibility is, well, maybe I can exponentiate faster than multiplying by myself n times, or multiplying by w n times.
How should I do it?
Repeated squaring, good.
You've seen that probably in 006.
Repeated squaring idea is, we take-- to compute w-- well, I take w to the 0.
I multiply it by w to the 0.
Sorry, circle 0-- that's this thing.
Oh, that seems weird.
Let's start with 1.
1 seems better.
I'm not going to get much if I multiply that by itself.
I should get exactly the same matrix.
So I take the circle product between w to the 1, w to the 1.
That gives me w to the 2.
And then, I take w to the 2 times w to the 2.
Everything's circled.
I get w to the 4.
Oh cool, I doubled my exponent with one multiplication.
If I take w to the 4 by w to the 4, I get w to the 8, and so on.
My goal is to get to n, so I have to do this log n times.
Log n squaring operations, each squaring operation is an n cubed thing.
So this is repeated squaring.
And I get V cubed log V-- finally, an improvement.
So we went from V to the 4, which was in the dense case the same performance as Bellman-Ford, running it V times.
But now in the dense case, I'm getting V cubed log V, which is actually pretty good.
It's not quite V cubed, but close.
All right.
I'm pointing at V cubed.
This is actually the one result that is not optimal.
This is the one we want to improve.
But we're kind of-- we're in this space right now.
We're getting close to as good is this algorithm, the Johnson's algorithm.
But we still a log V Factor.
So this is great, just by translating into matrix multiplication.
Now technically, you have to check that repeated squaring actually gives you the same result.
Basically, this works because products are associative.
Circle products of matrices are associative, which works because circle land is a semi-ring.
If you want the abstract algebra, a ring is something that you wear on your a finger.
No.
A ring is an algebra where you define plus and times, and you have distributivity.
Semi-ring, there's no minus, because min has no inverse.
There's no way from the min to re-compute the arguments, right?
No matter what you apply to it, you can't-- you've lost information.
So that's the semi-ring.
Normally, you have a minus.
But semi-ring is enough for the repeated squaring to give you the right answer.
However, semi-ring is not enough for all these fancy algorithms.
So if you look at Strassen's algorithm, the one you've seen, it uses minus.
There's no way to get around that, as far as we know.
So if you have no minus, n cubed is the best we know how to do.
So sadly, we cannot improve beyond this with this technique.
It sucks, but that's life.
However, we can do something.
If we just change the problem, there is another problem which this is the best way to do.
So let me briefly tell you about that problem.
It's called transitive closure.
Transitive closure is, I just want to know is there a path from i to j.
So it's going to be 1, if there exists a path from i to j.
And it's going to be 0 otherwise.
OK.
I guess it's kind of like if you set all the weights to 0 or infinity.
Then, either there's going to be as 0 way path, or there's no path, meaning there's an infinite way path.
So it's not quite the same.
Here, I want 1 and 0.
I flipped.
It used to be, this was infinity, and this was 0.
This is one saying there is a path from i and j, 0 otherwise.
If I write it this way, and then I think about what I need to do here, it is still in some sense plus and min, but not really.
Because I just want to know, is there a path?
So if I have a way to get there and a way to get there, instead of adding up those values, really I'm taking some other operator.
So I want to know OR.
Yeah, exactly-- who said OR?
Yeah, all right, tough one.
Close, close, close.
So here, we have basically a circle product is OR, and circle sum is AND.
OK?
I mean plus and min would work, but it's a little bit nicer over here.
Sorry, it's the other way around, I think.
It's definitely Booleans.
We want to know there is a way to get to x, and then from x to where we're going.
That's an AND.
And then, to get a path in general, it has to work for some x.
So that's the OR.
And this is a ring.
And once you're ring, you have negation.
You can apply Vassilevska-Williams.
And you solve this problem in n to the 2.3728.
And if I just make a little change in the dot dot dot, I can absorb the log.
So you could put a log n here.
And it's log n if you get the exponent exactly right.
But if you just tweak the exponent by 0.00000001, that's bigger than log n. So we usually omit the log there.
Cool.
Transitive closure-- so it's a problem you didn't know you want to solve, but it is actually a common problem.
And this is the best way we know how to solve it for dense graphs.
OK, it beats, you know, V cubed.
This is the algorithm we're aiming for for dense graphs.
For sparse graphs, we can do better.
But for dense graphs, this is better.
Finally, we get to go to dynamic programming number two, also known as the Floyd-Warshall algorithm.
So we had this dp in V the fourth.
If we forget about transitive closure, we've now are down to V cubed log V. Our next goal is to achieve V cubed, no log V. Let's do that.
So again, I'm going to express it in my five steps.
First step is, what are subproblems?
And this is the key difference, and the key insight for Floyd-Warshall is to redefine the dij problems.
To avoid conflict, I'm going to call them cij, or in this case, cuv, because here, the matrix product view will not work, I think.
Yeah, it won't work.
So it's a totally different universe.
I'm still going to assume that my vertices are numbered 1 through n. And now, the idea is, first I'm going to think about the graph formed by the vertices 1 though k, roughly.
And I want to know for every vertex u and every vertex v, what is the shortest path from u to v, or the weight of the shortest path from u to v that only uses intermediate vertices from 1 through k. So actually, u and v might not be-- they might be larger than k. But I want all the vertices in the path to be 1 through k. This is a different way to slice up my space, and it's the right way.
This is going to do a factor of n better.
It turns out, and that's just an insight you get from trying all the dp's you could think of.
And eventually, Floyd and Warshall found this one, I think in the '70s.
So it was easier back then to get a new result.
But I mean, this is very clever-- so very cool idea.
So now the question is, what should I guess?
Before I guessed what the last edge was.
That's not going to be so useful here.
Can anyone think of a different thing to guess?
We're trying to solve this problem where I get to use vertices 1 through k, and presumably I want to use subproblems that involve smaller k, that say involve vertices 1 through k minus 1.
So vertex k is relevant.
What should I guess about vertex k?
Yeah?
Guess that vertex k is the [INAUDIBLE]
You want to guess vertex k is the i-th intermediate vertex.
That would work, but I would need to parameterize by i here, and I lose another factor of n if I do that.
So I'd like to avoid that.
That is a good idea.
Yeah?
[INAUDIBLE] visit k, before you visit v
You're going to guess that I visit k, and then I go to where I'm trying to go.
OK. That's not a-- OK. That's a statement.
But to guess, I should have multiple choices.
What's my other choice?
[INAUDIBLE]
Yes.
So either I use vertex k, or I don't.
That's the guess-- is k in the path at all from u to v?
So that's a weaker thing than saying, k is at position i in the path.
Here I'm just saying, is k in the path at all?
And that's nice, because as you say, I already know how to get there without using k. Because that's cuvk minus 1.
And then, you just also have to consider the situation where I go to k, and then I leave.
So the recurrence is going to be cuvk is the min of two things.
One is when k is not in the path, that's cuvk minus 1.
And the other option is that I go to x first-- or sorry, I go to k first.
It used to be x.
Now, I've renamed it k. I don't know why.
k minus 1-- and then I go from k to the v-- k minus 1.
That's it.
Min of two things-- before, I was taking the min of n things.
Before, there were n choices for my guess.
Now, there are two choices for my guess.
Number of subproblems is the same, still V cubed.
But the guessing part and the recurrence part is now constant time instead of linear time.
So I'm now V cubed time-- progress.
OK?
This is pretty cool.
The old dp led us to this world of matrix multiplication.
That's why I covered it.
This new dp is just a different way of thinking about it-- turns out to be faster, just by log factor, but a little bit faster.
I need some base cases-- cuv of 0 is going to be-- now it's the weight of the edge uv.
It's a different base case.
Before, I was using 0 edges.
Now, it's not using any intermediate vertices.
So that's how the weights come into the picture, because actually there are no weights of edges up here.
So that's a little weird.
The only place the weights come in is when k equals 0.
This is in some sense still relaxation, but it's a little bit weirder, little bit different order.
I mean the key thing here is, because the way we set up these subproblems with the intermediate vertices, we know k is the only vertex in question.
Before it's like, well, I don't know where you go at the end.
But now we know that either k is in there, or it's not.
And in each case, we can compute it using smaller subproblems and so we save that linear factor
Is this only for [INAUDIBLE] graphs, or is does it also [INAUDIBLE]
This is for directed graphs.
u and v are ordered here.
And this is the weight from u to v-- will work just as well.
It's probably a little bit instructive to write this down as nested for loops again.
Why not?
Because then, you'll see it's just relaxation again.
So I'll even write the base case here, because it's very simple.
We're doing k in order, let's say.
These are really the same kinds of for loops.
But I'll write them slightly differently, because here we care about the order slightly.
Here, we do care about the order.
Here, we don't care about the order.
Vertices, and all we're saying is-- almost exactly the same code as before.
This is, again, just a relaxation step.
We're just relaxing different edges in different orders, basically, because k is evolved in here.
We do that for k equals 1.
Then for k equals 2, and so on.
But in the end, it's just relaxations, so you can use that to prove that this actually computes the right shortest paths.
I won't do that here.
But clearly, cubic time instead of quartic.
Pretty cool.
That's Floyd-Warshall.
It's very simple.
And so a lot of people-- if you need to solve all-pairs shortest paths in dense graphs, this is the best we know how to do.
So this is what you should implement.
It's like, five lines of code.
And you achieve this bound.
But for our sparse graphs, we can do better.
And the rest of lecture is going to be about Johnson's algorithm, where for sparse graphs we're going to get closer to quadratic time.
We're going to match running Dijkstra, and it's V squared log V plus E times V, sorry.
So when E is small, that's going to be close to quadratic.
When E is big, it's going to be cubic, again.
So we'll never be worse than this Floyd-Warshall algorithm.
But for sparse graphs it's better.
OK. Johnson's algorithm.
I was going to make some joke about Johnson and Johnson, but I will pass.
So Johnson's algorithm-- I mean, dp is five steps, but Johnson's algorithm's only three steps-- clearly simpler.
It's actually much more complicated, but it's all about what the steps are.
So here's the crazy idea in Johnson's algorithm.
We're going to change the weights on the edges.
And to do that, we're going to assign weights to vertices.
We're going to choose a function h. Think of it as a height function, I guess, that maps vertices to real numbers.
And then, we're going to define w sub h of u,v.
This is a new way to think about edge weights that depends on h that's defined in a simple way.
It's the old edge weight plus h of u minus h of v. You could define it the other way, but it's better to be consistent here.
So this is a way to tweak edge weights.
For every edge-- this is for directed graphs clearly.
For u, u is the beginning, the head of the-- I don't know if it's the head or the tail-- the beginning of the edge.
v is the end of the edge.
I'm going to add on the height of h, and subtract out the height of v. OK?
Why?
Because that's the definition.
I want this to be greater than or equal to 0.
That's the such that.
I want to assign a function h, so that these new weights are all greater or equal to 0.
This is for all u and v. Why would I do that?
To use Dijkstra instead of--
To use Dijkstra instead of Bellman-Ford, exactly.
So that's step 2.
Run Dijkstra on, I guess, the usual graph.
But now, this new weight function, w sub h, if all the weights are non-negative, I can run Dijkstra.
So this will give me what I call the shortest path sub h of u comma v for all u and v. It doesn't give me the actual shortest path weights I want.
It gives me the shortest path weights using this wh.
But I claim that's almost the same.
I claim that this re-weighting preserves which paths are shortest.
Because-- so in particular, I claim that delta of u,v is delta sub h of u,v-- should be the other way-- minus h of u plus h of v. OK.
If this was a single edge, you can see I'm just cancelling off these terms.
But in fact, I claim for a whole path, every path from u to v gets changed by exactly the same amount.
So this is a claim about the shortest path-- in effect, a claim for every path from u to v, shortest or not.
If I measure it in regular weights w, versus weights w sub h, the only difference is this fixed amount, which depends only on u and v-- does not depend on the path.
And therefore, which paths are shortest are preserved.
And so when we compute these shortest path weights, we can translate them back to what they should be in the original weighting function.
And furthermore, if you have parent pointers, and you actually find the paths, the paths will be the same.
Shortest paths will be the same.
OK.
So let's prove that claim.
It's actually really simple.
Let's look at a path from u to v. So I'm going to label the vertices along the path.
V0 is going to be u.
That's the first one, then V1, then V2, and so on.
Let's say path has length k. And Vk is v. OK, that's just a generic path from u to v. And now, I want to compute the w sub h of that path.
Excuse me.
So the weight of a path is just the sum of the weights the edges.
So I could write this as a sum from i equals 1 to k of w sub h of Vi minus 1 comma Vi.
I think that works, got to be careful not to get the indices wrong.
OK, now, w sub h is defined to be this thing-- w plus h of u minus h of v. So this is the sum i equals 1 to k of w Vi minus 1 Vi plus h of Vi minus 1 minus h of Vi.
What does the sum do?
Telescope.
So success-- this Vi is going to-- this negative h of Vi is going to cancel with the plus h of Vi minus 1 in the next term, except for the very first one and the very last one.
So this is going to be this sum, which is just the weight of the path according to regular weight function, plus h of V0 minus h of Vk.
And that is just the weight to the path plus h of u minus h of v. Did I get it right?
Nope.
Yes?
[INAUDIBLE] subtract [INAUDIBLE]
But it's not-- it's opposite of what I claimed.
So right, because it's the other side, good.
This has h on the right hand side.
This has not h on the left hand side.
But here, I have h on the left hand side, and not h on the right hand side.
So if I flip it around, if I take these two terms, put them on the left hand side, then I get this with the right sign.
Cool, whew.
Self-consistent.
OK.
So this was talking about are arbitrary path.
And so this is proving the stronger thing I said, that every path gets lengthened by this function, which is purely a function of the endpoints.
So in particular, that means the shortest path in w land will still be the shortest path in w sub h land-- slightly less cool name than circle land, but oh well.
All right, so this means shortest paths are preserved.
Shortest paths are still shortest.
And therefore, if I look at the delta function, which is about the shortest path weights, this claim holds.
So that's the proof of the claim.
Cool.
There's one gaping problem with this algorithm, which is how in the world do you find this h?
If we could find h, then we know we could run Dijkstra, and we can do this thing.
And Dijkstra is going to cost the VE plus V squared log V. I didn't say it, but we run V times Dijkstra.
All right, we run it V times.
That's going to take V squared log V plus VE to do.
This is just going to take quadratic time, V squared to update all the weights, update all the delta functions.
The missing step is how do we find this weight function?
I claim this problem of finding h that has this property, is very closely related to shortest paths.
It's weird, but we're going to use shortest paths to solve shortest paths.
So let's do it.
Step 1, finding h. What I want to do, so I want to have w of u,v-- let me just copy that down-- plus h of u plus h of v to be greater than or equal to 0.
Whoops, minus.
I'm going to put the h's on to the right hand side, and then flip it all around.
So this is like saying h of v minus h of u is less than or equal to w of u,v for all u and v. This is a problem we want to solve, right?
w's are given.
h's are unknowns.
This is called a system of difference constraints.
If you've heard about linear programming, for example, this is a special case of linear programming.
Don't worry if you haven't heard, because this is an easy special case.
We're going to solve it much faster than we know how to solve lineal programs.
It's a particular kind of thing.
This is actually useful problem.
You could think of, these are maybe times that various events happen.
And these are constraints about pairs of them.
Says, well, the start time of this event minus the end time of that event should be less than or equal to 1 second.
You can use this to do temporal programming, if you could solve these systems.
We're going to solve these systems, when they have a solution.
They don't always have a solution, which is a bit weird, because we're relying on them always having a solution.
How can that be?
Negative weight cycles.
This is all going to work, when we don't have negative weight cycles.
And that's exactly going to be the case when this system of difference constraints has no solution.
So let me show you that in a couple of steps.
First theorem is that if the graph V,E,w has a negative weight cycle, then that system has no solution-- no solution to the difference constraints.
This is going to be, again, an easy proof, kind of similar to last one actually.
So consider a negative weight cycle.
Let's call it V0, to V1, to V2, to Vk, back to V0.
So the claim is the sum of these weights is negative.
And now, I'm just going to write down these constraints, which are supposed to have a solution, or maybe they won't.
So if it has a solution, then this must be true, where u and v are plugged into be Vi and Vi minus 1, because those are all edges.
So I'm going to write h of V1 minus h of V0 is less than or equal to w of V0 wV1.
And then h of V2 minus h of V1 less than or equal to w of V1 V2.
Repeat that k times, I'm going to get h of Vk minus h of Vk minus 1.
And then, the last one, the wrap around h of V0 minus h of Vk w of V-- did I get it right-- Vk V0.
What do I do with these inequalities?
Sum them.
Time for a Good Will Hunting moment-- do you remember?
I hope we've all seen Good Will Hunting.
I don't have a janitor here, so I have to do all cancels by hand-- and this.
So I end up with 0 at the bottom.
Everything cancels, and then, over here I have less than or equal to the weight of the whole cycle.
I'm just adding up the weight of the cycle.
I didn't give the cycle a name-- call it C. Now, the cycle has negative weight.
So this is less than zero, strictly less than zero.
So we're saying that 0 is strictly less than 0.
That's not true.
So that means there's no way to get all of these constraints simultaneously true-- proof by a contradiction.
So that establishes a connection in the direction we don't want it.
What we want is they're converse, which is if there's no negative weight cycle, then there is a solution.
Luckily, that is also true.
But this is a little easier to see.
So now, we do the other half.
OK. And this will-- I mean, it's going to be constructive proof.
So we're going to actually know how to solve this problem with an algorithm.
So it's going to be-- there is a negative weight cycle if and only if there's no solution.
So in particular, the case we care about is if there's no negative weight cycle, then there is a solution.
We kind of care about both, but this is the more practical direction.
So let's prove it.
You can already see-- you've seen that there's a connection in negative weight cycles.
Now, I'm going to show there's a real connection to shortest paths.
Negative weight cycles are just kind of a symptom of the shortest paths being involved.
So now, we're going to use shortest paths.
Suppose we have some graph.
I'm going to draw a simple little graph with weights.
What I'd like to do is compute shortest path from a single source in this graph.
The question is, which source?
Because none of the vertices-- I guess in this case, this would be a pretty good source, because it can reach.
From here, I can get to every node.
But in general-- maybe there's another vertex here-- draw a more complicated picture.
It could be, there's no one vertex that can reach all the others.
For example, it may be the graph is disconnected.
That's a good example.
So there's no single source that can reach everywhere.
I really want to reach everywhere.
So what am I going to do?
Add a new source.
Call it s. I'm going to add an edge to every other vertex.
Now, I can get everywhere from s. OK?
What are the weights?
0.
0 sounds good.
I don't want to change the weights, in some sense.
So I put 0, and add 0 to everything.
That's not going to change much.
Now, notice I add no cycles to the graph.
So if there were no negative weight cycles before, still no negative weight cycles, because the cycles are the same as they were before.
But now, from s I can reach everywhere.
If there's no negative weight cycles, that means there's a well-defined, finite value for delta of s comma v for all V. And that is h. What?
It's crazy man.
All right, so we add s to V. We're going to add s comma v to e for all V. That's the old V. And I'm going to set the weight of s comma v to be 0 for all V. OK, that's what i just did.
And so now, delta of s comma v is finite for all V. It's not plus infinity, because I know there is-- it's got to be less than 0, right?
I can get from s to everywhere.
So it's less than positive infinity.
It's also not negative infinity, because I've assumed there's no negative weight cycles anywhere.
So I'm going to let h of v be delta of s,v.
I claim that just works, magically.
That's insane.
Every time I see it, it's like, got to be crazy man-- crazy but correct.
That's Johnson.
It's like you just pray that this happens, and it works.
Why would it happen?
Why would it be that-- what do we want to say-- w of u,v plus h of u minus h of v-- we want this to be greater than or equal to 0.
I guess I had already rewritten this way.
Neither way is the right way, so it doesn't matter.
So let's see.
We have a weight of u,v.
We have the shortest path from s to u.
And we have this the shortest pathway from s to v. We want that to be greater than or equal to 0.
Why?
Put this over there, and I get delta s,v is less than or equal to delta of s,u plus w of u,v, which is?
Triangle inequality, which is true.
It turns out, this thing we've been staring at for so long is actually just triangle inequality.
So of course we want to compute shortest paths, because shortest paths satisfy triangle inequality.
The whole name of the game in shortest paths is to find a place where you don't satisfy triangle inequality and fix it.
So if it makes sense, if that's possible to do, Bellman-Ford will do it.
So how we're going to do step 1?
We're going to run Bellman-Ford once.
We're going to add this source vertex, so that there is a clear source to run Bellman-Ford from, and then, run Bellman-Ford from there only.
That will give us a weight function for the vertices, namely how long does it take to get from s to those vertices.
Those weights will actually all be negative.
But then, we're going to modify all the edge weights according to this formula, which negates some of them.
So some of them are going to go up some, some of them are going to go down.
It's kind of weird.
But when we're done, all of the weights will be non-negative because we had triangle inequality.
And now, we can run Dijkstra from every vertex.
So it's like we bootstrap a little bit.
We run Bellman-Ford once, because we know it handles negative weights.
It will also tell us if there are any negative weight cycles.
That's why we want this theorem.
Maybe Bellman-Ford says, I can't satisfy triangle inequality, because there's a negative weight cycle.
I don't know what to do.
Then, we know, well actually, then there was no solution.
OK, that's kind of interesting.
But then, we'll have to deal with the shortest paths-- sorry-- deal with those negative weight cycles.
I won't cover how to do that here.
But you can.
And otherwise, there's no negative weight cycles, then Bellman-Ford finds valid h. Then, we plug that h into here.
Then, we have non-negative weights.
So in VE time, we've reduced to the non-negative all-pair shortest paths.
And then, we run Dijkstra V times.
Then, we get almost our answers, but we have to modify them to get back the correct weights on our shortest paths.
And so we computed shortest paths in V squared log V plus VE, because this is how much Dijkstra costs, and because Bellman-Ford takes less time.
We're good.
That's the magic.
And that's all-pairs shortest paths.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
Today, we have another cool graph algorithm or problem.
Actually, we'll have two algorithms.
The problem is called minimum spanning tree.
You can probably guess from the title what it's trying to do.
We'll see two algorithms for doing it.
Both of them are in the category of greedy algorithms, which is something we've seen a couple of times already in 6.046, starting with lecture 1.
This is the definition of greedy algorithm from lecture 1, roughly.
The idea is to always make greedy choices, meaning the choice is locally best.
For right now, it seems like a good thing to do, but maybe in the future it will screw you over.
And if you have a correct greedy algorithm, you prove that it won't screw you over.
So it's sort of like Cookie Monster here, always locally seems like a good idea to eat another cookie, but maybe it'll bite you in the future.
So today we will embrace our inner Cookie Monster and eat as many-- eat the largest cookie first, would be the standard algorithm for Cookie Monster.
I don't know if you learned that in Sesame Street, but-- all right.
So what's the problem?
Minimum spanning tree.
Can anyone tell me what a tree is?
Formally, not the outside thing.
In graph land.
Acyclic graph, close.
Connected acyclic graph, good.
That's important.
This is 604.2 stuff.
OK, so how about a spanning tree?
Sorry?
It contains all the vertices
It contains all the vertices.
Yeah.
So let me go over here.
Spanning means it contains all the vertices, so implicit here, I guess, is subtree or subgraph.
You're given a graph.
You want a spanning tree of that graph.
It's going to be a tree that lives inside the graph.
So we're going to take some of the edges of G, make a tree out of them, make a connected acyclic graph.
And that tree should hit all the vertices in G. So this is going to be a subset of the edges, or subgraph.
Those edges should form a tree.
And, I'll say, hit all vertices of G. OK, if I just said they should form a tree, then I could say, well, I'll take no edges, and here's a tree with one vertex.
That's not very interesting.
You want a vertex-- you want, basically, the vertex set of the tree to be the same as the vertex set of the graph.
That's the spanning property.
But you still want it to be a tree, so you want it to be connected and you want it to be acyclic.
Now if G is disconnected, this is impossible.
And for that, you could define a spanning forest to be like a maximal thing like this, but we'll focus on the case here as G is connected.
That's the interesting case.
And so we can get a spanning tree.
All right?
So what is this minimum spanning tree problem?
Minimum spanning tree.
We're given a weighted graph, just like last time, with shortest paths.
We have an edge weight function W giving me a real number, say, for every edge.
And we want to find a spanning tree of minimum total weight.
So I'm going to define the weight of a tree T to be the sum over all edges in T, because I'm viewing a spanning tree as a set of edges, of the weight of that edge.
OK, so pretty much what you would expect.
Minimum weight spanning tree.
It's a relatively simple problem, but it's not so easy to find an algorithm.
You need to prove a lot to make sure that you really find the right tree.
I guess the really naive algorithm here would be to try all spanning trees, compute the weight of each spanning tree and return the minimum.
That sounds reasonable.
That's correct.
But it's bad, because-- n to the fourth, that would be nice.
It's larger than that.
Maybe not so obvious, but it can be exponential.
Here's a graph where the number of spanning trees is exponential.
This is a complete bipartite graph with two vertices on one side and n vertices on the other, and so you can-- let's say we put these two edges into the spanning tree.
And now, for each of these vertices, we can choose whether it connects to the left vertex or the right vertix.
It can only do one, but it could do either one independently.
So maybe this guy chooses the left one, this one chooses the right one.
This one chooses the left one, and so on.
If I have n vertices down here, I have 2 to the n different spanning trees.
So there can be an exponential number.
So that algorithm is not so good.
Exponential bad.
Polynomial good.
So today, we're going to get a polynomial algorithm.
In fact, we will get an almost linear time algorithm as fast as Dijkstra's algorithm.
But we can't use Dijkstra's algorithm, there's no shortest paths here.
Plus, one of the algorithms will actually look pretty similar.
Two lectures ago, the dynamic programming lecture, we saw an example where we tried to do greedy, and it gave the wrong answer, and so we fell back on dynamic programming.
Today, we're going to try to do dynamic programming, it's going to fail, and we're going to fall back on greedy.
It's like the reverse.
But the way it's going to fail is we're going to get exponential time initially, and then greedy will let us get polynomial time.
This is actually a bit unusual.
I would say more typically, dynamic programming can solve anything, but, you know, with n to the seventh running time, something slow.
And then you apply greedy, and you get down to like n or n log n running time.
So that's more common.
But today, we're going to go from exponential down to polynomial.
And that's pretty nice.
Cool.
So let me tell you a little bit about greedy algorithm theory, so to speak.
This is from the textbook.
If your problem can be solved by greedy algorithm, usually you can prove two properties about that algorithm.
One of them is called optimal substructure.
And the other is called the greedy choice property.
Optimal substructure should be familiar idea because it's essentially an encapsulation of dynamic programming.
Greedy algorithms are, in some sense, a special form of dynamic programming.
So this is saying something like, if you can solve subproblems optimally, smaller subproblems, or whatever, then you can solve your original problem.
And this may happen recursively, whatever.
That's essentially what makes a recurrence work for dynamic programming.
And with dynamic programming, for this to be possible, we need to guess some feature of the solution.
For example, in minimum spanning tree, maybe you guess one of the edges that's in the right answer.
And then, once you do that, you can reduce it to some other subproblems.
And if you can solve those subproblems, you combine them and get an optimal solution to your original thing.
So this is a familiar property.
I don't usually think of it this way for dynamic programming, but that is essentially what we're doing via guessing.
But with greedy algorithms, we're not going to guess.
We're just going to be greedy.
Eat the largest cookie.
And so that's the greedy choice property.
This says that eating the largest cookie is actually a good thing to do.
If we keep making locally optimal choices, will end up with a globally optimal solution.
No tummy ache.
This is something you wouldn't expect to be true in general, but it's going to be true for minimum spanning tree.
And it's true for a handful of other problems.
You'll see a bunch more in recitation tomorrow.
This is sort of general theory, but I'm actually going to have a theorem like this for minimum spanning tree and a theorem like this for minimum spanning tree.
This is the prototype, but most of today is all about minimum spanning tree.
And for minimum spanning tree, neither of these is very obvious.
So I'm just going to show you these theorems.
They're fairly easy to prove, in fact, but finding them is probably the tricky part.
Actually, I guess optimal substructure is probably the least intuitive or the least obvious greedy choice.
You're probably already thinking, what are good greedy choices?
Minimum weight edge seems like a good starting point, which we will get to.
But there's even a stronger version of that, which we will prove.
And first, optimal substructure.
So here, I'm going to think like a dynamic program.
Let's suppose that we know an edge that's in our solution.
Suppose we know an edge that lives in a minimum spanning tree.
We could guess that.
We're not going to, but we could.
Either way, let's just suppose than an edge e-- I should mention, I guess I didn't say, this graph is undirected.
A minimum spanning tree doesn't quite make sense with directed graphs.
There are other versions of the problem but here, the graph is undirected.
So probably, I should write this as a unordered set, u, v. And there are possibly many minimum spanning trees.
There could be many solutions with the same weight.
For example, if all of these edges have weight 1, all of these trees are actually minimum.
If all the edges have weight 1, every spanning tree is minimum, because every spanning tree has exactly n minus 1 edges.
But let's suppose we know an edge that's guaranteed to be in some minimum spanning tree, at least one.
What I would like to do is take this, so let me draw a picture.
I have a graph.
We've identified some edge in the graph, e, that lives in some minimum spanning tree.
I'm going to draw some kind of tree structure here.
OK.
The wiggly lines are the tree.
There are some other edges in here, which I don't want to draw too many of them because it's ugly.
Those are other edges in the graph.
Who knows where they are?
They could be all sorts of things.
OK?
But I've highlighted the graph in a particular way.
Because the minimum spanning tree is a tree, if I delete e from the tree, then I get two components.
Every edge I remove-- I'm minimally connected.
So if I delete an edge, I disconnect into two parts, so I've drawn that as the left circle and the right circle.
It's just a general way to think about a tree.
Now there are other unused edges in this picture, who knows where they live?
OK?
What I would like to do is somehow simplify this graph and get a smaller problem, say a graph with fewer edges.
Any suggestions on how to do that?
I don't actually know where all these white edges are, but what I'd like to do is-- I'm supposing I know where e is, and that's an edge in my minimum spanning tree.
So how could I get rid of it?
Yeah
Find the minimum weight spanning tree of the two edges
I'd like to divide and conquer.
Maybe find the minimum weight over here, minimum weight over here.
Of course, I don't know which nodes are in what side.
So that's a little trickier.
But what do I do but E itself?
Let's start with that.
Yeah
You remove it?
You could remove it.
That's a good idea.
Doesn't work, but worth a Frisbee nonetheless.
If I delete this edge, one problem is maybe none of these red edges exist and then my graph is disconnected.
Well, maybe that's actually a good case.
That probably would be a good case.
Then I know how to divide and conquer.
I just look at the connected components.
In general, if I delete the edge, and I have these red edges, then I maybe find a minimum spanning tree on what remains.
Maybe I'll end up including one of these edges.
Maybe this edge ends up in the spanning tree, and then I can't put E in.
So it's a little awkward.
Yeah?
Can you merge the two nodes into one?
Merge the two nodes into one.
Yes.
Purple Frisbee.
Impressive.
This is what we call contracting the edge.
It just means merge the endpoints.
Merge u and v. So I will draw a new version of the graph.
So this was u and v before.
You've got to put the label inside.
And now we have a new vertex here, which is uv.
Or you can think it as the set u, v. We won't really need to keep track of names.
And whatever edges you had over here, you're going to have over here.
OK?
Just collapse u and v. The edge e disappears.
And one other thing can happen.
Let me-- go over here.
We could end up with duplicate edges by this process.
So for example, suppose we have u and v, and they have a common neighbor.
Might have many common neighbors, who knows.
Add some other edges, uncommon neighbors.
When I merge, I'd like to just have a single edge to that vertex and a single edge to that vertex.
And what I'm going to do is, if I have some weights on these edges, let's say a and b, and c and d, I'm just going to take the minimum.
Because what I'm about to do is compute a minimum spanning tree in this graph.
And if I take the minimum spanning tree here, and I had multiple edges-- one weight a, one weight b-- do you think I would choose the larger weight edge?
It does-- they're exactly the same edge, but one is higher weight.
There's no point in keeping the higher weight one, so I'm just going to throw away the higher weight one.
Take them in.
So this is a particular form of edge contraction and graphs.
And I claim it's a good thing to do, in the sense that if I can find a minimum spanning tree in this new graph-- this is usually called a G slash e, slash instead of negative, to remove e. I'm contracting e. So this is G slash e. This is G. If I can find a minimum spanning tree in G slash e, I claim I can find one in the original graph G just by adding the edge e. So I'm going to say if G prime is a minimum spanning tree, of G slash e, then T prime union e is a minimum spanning tree of G. So overall, you can think of this as a recurrence in a dynamic program, and let me write down that dynamic program.
It won't be very good dynamic program, but it's a starting point.
This is conceptually what we want to do.
We're trying to guess an edge e that's in a minimum spanning tree.
Then we're going to contract that edge.
Then we're going to recurse, find the minimum spanning tree on what remains, and then we find the minimum spanning tree.
Then we want to decontract the edge, put it back, put the graph back the way it was.
And then add e to the minimum spanning tree.
And what this lemma tells us, is that this is a correct algorithm.
If you're lucky-- and we're going to force luckiness by trying all edges-- but if we start with an edge that is guaranteed to be in some minimum spanning tree, call it a safe edge, and we contract, and we find a minimum spanning tree on what remains, then we can put e back in at the end, and we'll get a minimum spanning tree of the original graph.
So this gives us correctness of this algorithm.
Now, this algorithm's bad, again, from a complexity standpoint.
The running time is going to be exponential.
The number of sub problems we might have to consider here is all subsets of edges.
There's no particular way-- because at every step, we're guessing an arbitrary edge in the graph, there's no structure.
Like, we can't say well, it's the first k edges, or some substring of edges.
It's just going to be some subset of edges.
There's exponentially many subsets, 2 to the e, so this is exponential.
But we're going to make a polynomial by removing the guessing.
This is actually a really good prototype for a greedy algorithm.
If instead of guessing, trying all edges, if we could find a good edge to choose that's guaranteed to be in a minimum spanning tree, then we could actually follow this procedure, and this would be like an iterative algorithm.
If you-- you don't guess-- you correctly choose a good-- you take the biggest cookie, you contract it, and then you repeat that process over and over, that would be a prototype for a greedy algorithm and that's what's going to work.
There's different ways to choose this greedy edge, and we're going to get two different algorithms accordingly.
But that's where we're going.
First, I should prove this claim, cause, you know, where did edge contraction come from?
Why does it work?
It's not too hard to prove.
Let's do it.
Question?
Oh.
All right.
I should be able to do this without looking.
So-- Proof of optimal substructure.
So we're given a lot.
We're told that e belongs to a minimize spanning tree.
Let's give that spanning tree a name.
Say we have a minimum spanning tree T star, which contains e. So we're assuming that exists, then we contract e. And then we're given T prime, which is a minimum spanning tree of G slash e. And then we want to analyze this thing.
So I want to claim that this thing is a minimum spanning tree, in other words, that the weight of that spanning tree is equal to the weight of this spanning tree, because this one is minimum.
This is a minimum spanning of G. And this is also supposed to be a minimum spanning tree of G. OK.
Sounds easy, right?
I'm going to cheat, sorry.
I see.
Right.
Duh.
Easy, once you know how.
So what we're going to do is think about contracting e. OK, we already know we're supposed to be thinking about contracting e in the graph.
Let's look at how it changes that given minimum spanning tree.
So we have T star, minimum spanning tree of the whole graph, and then I'm going to contract e. What I mean is, if that edge happens to be in the spanning tree-- it is, actually.
We assumed that e is in there.
So I'm basically removing, I'm just deleting that edge, maybe I should call it minus e. Then that should be a spanning tree of G slash e. So when I contract the edge in the graph, if I throw away the edge from this spanning tree, I should still have a spanning tree, and I don't know whether it's minimum.
Probably, it is, but we won't prove that right now.
I claim it's still a spanning tree.
What would that take?
It still hits all the vertices, because if I removed the edge, things would not be connected together.
But this edge was in the spanning tree, and then I fused those two vertices together, so whatever spanning-- I mean, whatever was connected before is still connected.
Contraction generally preserves connectivity.
If these things were already connected directly by an edge when I contract, I still have a connected structure, so I'm still hitting all the vertices.
And also, the number of edges is still exactly right.
Before, I had n minus 1 edges.
Afterwards, I'll still have n minus 1 edges, because I removed one edge and I removed one vertex, in terms of the count.
So that proves that it's still a spanning tree, using properties of trees.
Cool.
So that means the minimum spanning tree, this thing, T prime, the minimum spanning tree of G slash e, has a smaller weight than this one.
Because this is a spanning tree, the minimum is smaller than all spanning trees.
So we know the weight of T prime is less than or equal to the weight of T star minus e. Cool.
And now we want to know about this thing, the weight of T prime plus e. Well, that's just the weight of T prime plus the weight of e, because the weight of a tree is just the sum of the weights of the edges.
So this is less than or equal to w of T star minus e plus e, which is just the weight of T star.
So we proved that the weight of our proposed spanning tree is less than or equal to the weight of the minimum spanning tree in G, and therefore, T prime union e actually is a minimum spanning tree.
OK?
This is really easy.
It actually implies that all of these inequalities have to be equalities, because we started with something minimum.
Clear?
That's the easier half.
The More interesting property is going to be this greedy choice property.
This is sort of where the action is for greedy algorithms, and this is usually the heart of proving greedy algorithms are correct.
We don't yet have a greedy algorithm, but we're thinking about it.
We need some way to intelligently choose an edge e, and I'm going to give you a whole bunch of ways to intelligently choose an edge e. So here's a really powerful lemma, and we're going to make it even stronger in a moment.
So I'm going to introduce the notion of a cut, that's going to be a similar picture to what I had before.
I'm going to look at some set of vertices.
S here is a subset of the vertices, and that leaves in the graph, everything else.
This would be V minus S. OK, so there's some vertices over here, some vertices over here, there's some edges that are purely inside one side of the cut.
And then what I'm interested in are the edges that cross the cut.
OK, whatever they look like, these edges.
If an edge has one vertex in V and one vertex not in V, I call that edge a crossing edge.
OK, so let's suppose that e is a least-weight edge crossing the cut.
So let's say, let me be specific, if e is uv, then I want one of the endpoints, let's u, to be in S, and I want the other one to be not in S, so it's in capital V minus S. And that would be a crossing edge, and among all the crossing edges, I want to take one of minimum weight.
There might be many, but pick any one.
Then I claim that edge is in a minimum spanning tree.
This is our golden ticket, right?
If we can guarantee an edge is in the minimum spanning tree, then we plug that in here.
Instead of guessing, we'll just take that edge-- we know it's in a minimum spanning tree-- and then we'll contract it and repeat this process.
So the tricky part-- I mean, it is true that the minimum weight edge is in a minimum spanning tree, I'll give that away.
But the question is, what you do then?
And I guess you contract and repeat but, that will be Kruskal's algorithm.
But this is, in some sense, a more general tool that will let us identify edges that are guaranteed to be in the minimum spanning tree, even after we've already identified some edges as being in the minimum spanning tree, so it's a little more powerful.
Let's prove this claim.
This is where things get particularly cool.
And this is where we're going to use something called a c and paste argument.
And if you are ever trying to prove a greedy algorithm correct, the first thing that should come to your mind is cut and paste.
This is almost universally how you prove greedy algorithms to be correct, which is, suppose you have some optimal solution which doesn't have the property you want, like that it includes e here.
And then you modify it, usually by cutting out one part of the solution and pasting in a different part, like e, and prove that you still have an optimal solution, and therefore, there is an optimal solution.
There is an MST that has the property you want.
OK, so we're going to do that by starting from an arbitrary minimum spanning tree.
So let T star be a minimum spanning tree of G, and if the edge e is in there, we're done.
So presumably, e is not in that minimum spanning tree.
We're going to modify T star to include e. So again, let me draw the cut.
There's S and V minus S. We have some edge e which crosses the cut, goes from u to v, that's not in the minimum spanning tree.
Let's say in blue, I draw the minimum spanning tree.
So you know, the minimum spanning tree connects everything together here.
I claim it's got to have some edges that cross the cut, because if it has no edges that cross the cut, it doesn't connect vertices over here with vertices over here.
So it may not use e, but some of the edges must cross the cut.
So here's a possible minimum spanning tree.
It happens to have sort of two components over here in S, maybe.
Who knows?
But there's got to be at least one edge the crosses over.
In fact, the minimum spanning tree, T star, has to connect vertex u to vertex v, somehow.
It doesn't use e, but there's got to be-- it's a tree, so in fact, there has to be a unique path from u to v in the minimum spanning tree.
And now u is in S, v is not in S. So if you look at that path, for a while, you might stay in S, but eventually you have to leave S, which means there has to be an edge like this one, which I'll call it e prime, which transitions from S to V minus S. So there must be an edge e prime in the minimum spanning tree that crosses the cut, because u and v are connected by a path and that path starts in S, ends not in S, so it's got to transition at least once.
It might transition many times, but there has to be at least one such edge.
And now what I'm going to do is cut and paste.
I'm going to remove e prime and add an e instead.
So I'm going to look at T star minus e prime plus e. I claim that is a minimum spanning tree.
First I want to claim, this is maybe the more annoying part, that it is a spanning tree.
This is more of a graph theory thing.
I guess one comforting thing is that you've preserved the number of edges, so it should still be if you get one property, you get the other, because I remove one edge, add in one edge, I'm still going to have n minus 1 edges.
The worry, I guess, is that things become disconnected when you do that, but that's essentially not going to happen because if I think of removing e prime, again, that disconnects the tree into two parts.
And I know, by this path, that one part contains this vertex, another part contains this vertex, and I know that this vertex is connected to u and this vertex is connected to v. Maybe I should call this u prime and v prime.
I know u and u prime are connected by a path.
I know v and v prime are connected by a path.
But I know that by deleting e prime, u prime and v prime are not connected to each other.
Therefore, u and v are not connected to each other, after removing e prime.
So when I add in e, I newly connect u and v again, and so everything's connected back together.
I have exactly the right number of edges.
Therefore, I'm a spanning tree.
So that's the graph three theory part.
Now the interesting part from a greedy algorithm is to prove to this is minimum, that the weight is not too big.
So let's do that over here.
So I have the weight of T star minus e plus-- minus e prime plus e. By linearity, this is just the weight of T star minus the weight e prime plus the weight of e. And now we're going to use this property, we haven't that yet, e is a least-weight edge crossing the cut.
So e prime crosses the cut, so does e, but e is the smallest possible weight you could have crossing the cut.
That means that-- I'll put that over here-- the weight of e is less than or equal to the weight of e prime, because e prime is a particular edge crossing the cut, e was the smallest weight of them.
So that tells us something about this.
Signs are so difficult.
I think that means that this is negative or zero.
So this should be less than or equal to w of T star, and that's what I want, because that says the weight of this spanning tree is less than or equal to the optimum weight, the minimum weight.
So that means, actually, this must be minimum.
So what I've done is I've constructed a new minimum spanning tree.
It's just as good as T star, but now it includes my edge e, and that's what I wanted to prove.
There is a minimum spanning tree that contains e, provided e is the minimum weight edge crossing a cut.
So that proves this greedy choice property.
And I'm going to observe one extra feature of this proof, which is that-- so we cut and paste, in the sense that we removed one thing, which was e prime, and we added a different thing, e. And a useful feature is that the things that we change only are edges that cross the cut.
So we only, let's say, modified edges that cross the cut.
I'm going to use that later.
We removed one edge that crossed the cut, and we put in the one that we wanted.
OK so far?
There's a bunch of lemmas.
Now we actually get to do algorithms using these lemmas.
We'll start with maybe the less obvious algorithm, but it's nice because it's very much like Dijkstra.
It follows very closely to the Dijkstra model.
And then we'll get to the one that we've all been thinking about, which was choose a minimum weight edge, contract, and repeat.
That doesn't-- well, that does work, but the obvious way is, maybe, slow.
We want to do it in near linear time.
Let's start with the Dijkstra-like algorithm.
This is Prim's algorithm.
Maybe I'll start by writing down the algorithm.
It's a little long.
In general, the idea-- we want to apply this greedy choice property.
To apply the greedy choice property, you need to choose a cut.
With Prim, we're going to start out with an obvious cut, which is a single vertex.
If we have a single vertex S, and we say that is our set capital S, then you know, there's some images coming out of it.
There's basically S versus everyone else.
That's a cut.
And so I could take the minimum weight edge coming out of that cut and put that in my minimum spanning tree.
So when I do that, I put it in my minimum spanning tree because I know it's in some minimum spanning tree.
Now, I'm going to make capital S grow a little bit to include that vertex, and repeat.
That's actually also a very natural algorithm.
Start with a tiny s and just keep growing it one by one.
At each stage use this lemma to guarantee the edge I'm adding is still in the minimum spanning tree.
So to make that work out, we're always going to need to choose the minimum weight edge that's coming out of the cut.
And we'll do that using a priority queue, just like we do in Dijkstra.
So for every vertex that's in V minus S, we're going to have that vertex in the priority queue.
And the question is, what is the key value of that node stored in the priority queue?
So the invariant I'm going to have is that the key of v is the minimum of the weights of the edges that cross the cut into v. So for vertex v, I want to look at the-- I'm not going to compute this every time, I'm only going to maintain it.
I want the minimum weight of an edge that starts in S and goes to v, which is not in S because v in Q-- Q only stores vertices that are not in S-- I want the key value to be that minimum weight so if I choose the overall minimum vertex, that gives me the edge of minimum weight that crosses the cut.
OK?
I've sort of divided this minimum vertex by vertex.
For every vertex over here, I'm going to say, what's the minimum incoming weight from somebody over here?
What's the minimum incoming weight from someone over here to there?
To here?
Take the minimum of those things.
And of course, the min of all those will be the min of all those edges.
OK, that's how I'm dividing things up.
And this will be easier to maintain, but let me first initialize everything.
OK, I guess we're going to actually initialize with S being the empty set, so Q will store everybody, except I'm going to get things started by setting for particular vertex little s. I'm going to set its key to zero.
It doesn't matter who little s is.
That's just your start vertex.
Just pick one vertex and set its key to zero.
That will force it to be chosen first because for everyone else, for v not equal to S, I'm going to set the key to infinity, because we haven't yet seen any edges that go in there, but we'll change that in a moment.
OK, so that was the initialization, now we're going to do a loop.
We're going to keep going until the Q is empty, because when the Q is empty, that means S is everybody, and at that point, we'll have a spanning tree on the whole graph, and it better be minimum.
OK, and we're going to do that by extracting the minimum from our priority Q.
When we remove Q-- we remove vertex u from the queue Q, this means that we're adding u to S. OK, by taking it out of Q, that means it enters S, by the invariant at the top.
So now we need to update this invariant, that all the key values are correct.
As soon as we move a vertex into S, now there are new edges we have to consider from S to not S, and we do that just by looking at all of the neighbors of u. I haven't written this in a long time, but this is how it's usually written in Dijkstra, except in Dijkstra, these are the outgoing edges from u and v are the neighbors.
Here, it's an undirected graph, so these are all of the neighbors v of u.
This as an adjacency list.
OK, so we're looking at u, which has just been added to S, and we're looking at the edges.
We want to look at the edge as they go to V minus S, only those ones.
And then for those vertices v, we need to update their keys, because it used to just count all of these edges that went from the rest of S to v. And now we have a new edge uv that v needs to consider, because u just got added to S. So the first thing I'm going say is if v in in Q.
So we're just going to store a Boolean for every vertex about whether it's in the queue, and so when I extract it from the queue, I just set that Boolean to false.
Being in the queue is the same as being not in S, this is what Q represents.
So Q is over here, kind of.
So if we're in the queue, same as saying v is not in S, then we're going to do a check which lets us compute the minimum.
This is going to look a lot like a relaxation.
Sorry.
A couple things going on because I want to compute not just the value of the minimum spanning tree, I actually want to find the minimum spanning tree, so I'm going to store parent pointers.
But this is just basically taking a min.
I say, if the weight of this edge is smaller than what's currently in the key, then update the key, because the key is supposed to be the min.
OK, that's all we need to do to maintain this invariant, this for loop.
After the for loop, this property will be restored, v dot key will be that minimum.
And furthermore, we kept track of where the minimums came from, so when you end up extracting a vertex, you've already figured out which edge you added to put that into the set.
So in fact, u already had a parent, this would be u dot parent, and we want to add that edge into the minimum spanning tree when we add u to S. Overall, let me write why this is happening.
At the end of the algorithm, for every vertex v, we want the v dot parent.
And that will be our minimum spanning tree.
Those are the edges that form the minimum spanning tree.
Let's prove that this works.
Actually, let's do an example.
We've done enough proofs for a while.
Let's do it over here.
I need a little break.
Examples are fun, though easy to make mistakes, so correct me if you see me making a mistake.
And let me draw a graph.
OK, weights.
14, 3, 8, 5, 6, 12, 7, 9, 15.
10.
OK.
Colors.
So I want to start at this vertex just because I know it does an interesting thing, or it's a nice example.
Here's my weighted undirected graph.
I want to compute minimum spanning tree.
I'm going to start with a capital S being-- well actually, I start with capital S being nothing, and all of the weights-- all of the key values are initially infinity.
So I'm going to write the key values in blue.
So initially everything is infinity for every vertex, except for S the value is zero.
So all of these things are in my priority queue, and so when I extract from the queue, I of course get S. OK, that's the point of that set up.
So that's when I draw the red circle containing little s. The red circle here is supposed to be capital S. So at this point, I've added capital S-- little s to capital S, and then I look at all of the neighbors v of S. And I make sure that they are outside of S. In this case, they all are.
All three neighbors, these three guys, are not in S. And then I look at the weights of the edges.
Here I have a weight 7 edge.
That's smaller than infinity, so I'm going to cross out infinity and write 7.
And 15 is smaller than infinity, so I'm going to cross out infinity and write 15.
And 10, surprise, is smaller than infinity.
So I'm going to cross out infinity rate 10.
So now I've updated the key values for those three nodes.
I should mention in the priority queue, to do that, that is a decrease-key operation.
This thing here is a decrease-key.
You need to update the priority queue to say, hey look, the key of this node changed.
And so you're going to have to move it around in the heap, or whatever.
Just like Dijkstra, same thing happens.
OK, so I've decreased the key of those three nodes.
Now I do another iteration.
I look at all of the key values stored.
The smallest one is 7, because this node's no longer in there.
So I'm going to add this node to capital S. So capital S is going to grow to include that node.
I've extracted it from the queue.
And now I look at all the neighbors of that node.
So, for example, here's a neighbor.
9 is less than infinity, so I write 9.
Here's a neighbor.
12 is less than infinity, so I write 12.
5 is less than infinity, so I write 5.
Here's a neighbor, but s is in big S, so we're not going to touch that edge.
I'm not going to touch s. OK?
I will end up looking at every edge twice, so no big deal.
Right now, who's smallest?
5, I think.
It's the smallest blue key.
So we're going to add 5 to the set.
Sorry, add this vertex to the set S, and then look at all of the outgoing edges from here.
So 6 is actually less than 12, so this edge is better than that one was.
Then, what's that, an 8?
8 Is less than 10.
14 is definitely less than infinity.
And we look at this edge, but that edge stays inside the red set, so we forget about it.
Next smallest value is 6.
So 6, we add this guy in.
We look at the edges from that vertex, but actually nothing happens because all those vertices are inside capital S, so we don't care about those edges.
Next one is 8, so we'll add in this vertex.
And there's only one edge that leaves the cut, so that's 3, and 3 is indeed better than 14.
So never mind.
Stop.
So good, now I think the smallest key is 3.
Notice smallest key is smaller than anything we've seen before, other than 0, but that's OK.
I'll just add it in, and there's no edges leaving the cut from there.
And then over here, we have 9 and 15.
So first we'll add 9.
There's no edges there.
Then we add 15.
OK, now s is everything.
We're done.
Q is empty.
Where's the minimal spanning tree?
I forgot to draw it.
Luckily, all of the edges here have different numbers as labels.
So when I have a 3 here, what I mean is, include 3 in the minimum spanning tree, the edge that was labeled 3.
OK, so this will be a minimum spanning tree edge.
5 will be a minimum spanning tree edge.
These are actually the parent pointers.
6 will be a minimum spanning tree edge.
7, 9, 15, and 8.
Every vertex except the starting one will have a parent, which means we'll have exactly n minus 1 edges, that's a good sign.
And in fact, this will be a minimum spanning tree.
That's the claim, because every time we grew the circle to include a bigger thing, we were guaranteed that this edge was in the minimum spanning tree by applying this property with that cut.
Let me just write that down.
OK, to prove correctness, you need to prove an invariant that this key, the key of every vertex, always remains this minimum.
So this is an invariant.
You should prove that by induction.
I won't prove it here.
But we have another invariant, a more interesting one from an MST perspective, you know, it's just a sort of algorithm implementation detail, that the tree T sub S, within S is always contained in a minimum spanning tree of G. So over here, we have this way of computing minimum spanning tree for all vertices v, but what I'd like to do is just look at v that's currently in S. By the end, that will be the whole thing, but if I look at v in S, and I always look at the edge from v to v dot parent, that gives me this tree TS.
I claim it will be contained in a minimum spanning tree of the entire graph, proof by induction.
So by induction, let's assume-- induction hypothesis will be that, let's say there is a minimum spanning tree T star, which contains T sub S, and then what the algorithm does, is it repeatedly grows S by adding this vertex u to S. So let's suppose that it adds u to S. So I'm actually going to look at the edge that it adds.
So we have S and V minus S, and we do this thing, like we just saw, of growing by one.
We add one new vertex over here to S, and that vertex has a parent edge, has a parent pointer.
So this edge, I'm going to call e. So we're adding some vertex u that we extract at the minimum, and we also added an edge e to this TS, because we grew S by 1.
OK, when I do that, all I do is say, look, greedy choice property guarantees there's a minimum spanning tree that contains e. Because we extracted the min from the queue, and the key values are this, as I was arguing before, that is the minimum overall edge that crosses the cut.
e is a minimum weight edge that crosses the cut, and so by greedy choice property, there is some minimum spanning tree that contains e. But actually, I need that the minimum spanning tree not only contains e, but also contains all the other spanning tree edges that we had already said were in T star.
OK, so here's where I'm going to use the stronger property.
I can modify T star to include e and T sub S. So we already assumed that T star includes T sub S. I just don't want to break that.
And if you remember the proof of this greedy choice property, we said, well all we need to do is remove one edge that crosses the cut and replace it with e. So here what I'm saying is there's some edge, yeah, maybe there's some edge over here in T star that we had to remove, and then we put e in.
And then we get a minimum spanning tree again, T star prime.
OK, this edge that I remove cannot be one of the TS edges because the TS edges are all inside S. So because I'm only removing an edge that crosses the cut, I'm not disturbing TS.
TS will remain inside T star, but then I get the new property that e is inside T star, and so I prove this invariant holds.
OK?
I keep changing T star, but I always preserve the property that all of the spanning tree edges that are inside S are contained in some minimum spanning tree of G. Maybe I'll add in some for emphasis.
Cool?
So that's how we use the greedy choice property to get correctness of Prim's algorithm.
What's the running time of Prim's algorithm?
Same as Dijkstra, good answer.
I guess it depends what priority queue you use, but whatever priority queue you use, it's the same as Dijkstra.
And so in particular, if we use Fibonacci heaps, which, again, we're not covering, we get V log V plus E. In general, for every edge, we have to do a decrease-key.
Actually, for every edge we do two decrease-key operations, potentially, if you think about it.
But this for loop over the adjacency, the cost of this stuff is constant.
The cost of this is the degree of the vertex u.
And so we're basically doing the sum of the degrees of the vertices, which is the number of edges times 2.
That's the handshaking lemma.
So for every edge, we're potentially doing one decrease-key operation, and with Fibonacci heaps, that's constant time.
But we're also doing V extract mins those cost log V time, cause the size of the queue is at most V, and so that is actually the right running time.
Just like Dijkstra, so easy formula to remember.
All right, let's do one more algorithm, Kruskal's algorithm.
Kruskal's algorithm is a little bit weirder from the S perspective, I guess.
We'll see what cuts we're using in a moment, but it's based around this idea of, well, the globally minimum weight edge is the minimum weight edge for all cuts that cross it, or for all cuts that it crosses.
The globally minimum weight edge is going to be a valid choice, and so, by this theorem, you pick some S that partitions the endpoints of e, therefore e is in a minimum spanning tree.
So let's choose that one first, and then repeat.
Conceptually, what we want to do is that DP idea of contract the vertex, sorry, contract the edge and then find the minimum weight edge that remains.
But the way I'm going to phrase it doesn't explicitly contract, although implicitly, it's doing that.
And there's a catch.
The catch is suppose I've picked some edges out to be in my minimum spanning tree.
Suppose this was the minimum weight and this was the next minimum, next minimum, next minimum, next minimum.
Suppose that the next lar-- at this point, after contracting those edges, the minimum weight edge is this one.
Do I want to put this edge in my minimum spanning tree?
No.
That would add a cycle.
Cycles are bad.
This is the tricky part of this algorithm.
I have to keep track of whether I should actually add an edge, in other words, whether this vertex and this vertex have already been connected to each other.
And it turns out you've already seen a data structure to do that.
This is what I call union-find and the textbook calls it disjoint-set data structure.
So it's in recitation.
Recitation 3.
So I want to maintain for my MST so far, so I'm adding edges one at a time.
And I have some tree-- well, it's actually a forest, but I'm still going to call it T, and I'm going to maintain it in a union-find structure, disjoint-set set data structure.
Remember, this had three operations, make set, union, and find set.
Tell me given an item which set does it belong to?
We're going to use that, the sets are going to be the connected components.
So after I've added these edges, these guys, these vertices here, will form one connected component, and, you know, everybody else will just be in its own separate component.
So to get started, I'm not going to have any edges in my tree, and so every vertex is in its own connected component.
So I represent that by calling make-set v for all vertices.
So every vertex lives in its own singleton set.
OK, now I'd like to do the minimum weight edge, and then the next minimum weight edge, and the next minimum weight edge.
That's also known as sorting, so I'm going to sort E by weight, increasing weight, so I get to start with the minimum weight edge.
So now I'm going to do a for-loop over the edges, increasing order by weight.
Now I want to know-- I have an edge, it's basically the minimum weight edge among the edges that remain, and so I want to know whether I should add it.
I'm going to add it provided the endpoints of the edge are not in the same connected component.
How can I find out whether two vertices are in the same connected component, given this setup?
Yeah?
Call find-set twice and then--
Call find-set twice and see whether they're equal, exactly.
Good answer.
So if you find-set of u is from find-set of v, find-set just returns some identifier.
We don't really care what it is, as long as it returns the same thing for the same set.
So if u and v are in the same set, in other words, they're in the same connected component, then find-set will return the same thing for both.
But provided they're not equal, then we can add this edge into our tree.
So we add e to the set T, and then we have to represent the fact that we just merged the connected components of u and v, and we do that with a union call.
And if you're ever wondering what the heck do we use union-find for, this is the answer.
The union-find data structure was invented in order to implement Kruskal's algorithm faster, OK?
In fact, a lot of data structures come from graph algorithms.
The reason Fibonacci heaps were invented was because there was Dijkstra's algorithm and we wanted it to run fast.
So same deal here, you just saw it in the reverse order.
First you saw union-find.
Now, union-find, you know you can solve v in alpha of n time, the inverse Ackermann function, super, super tiny, slow growing function, smaller than log log log log log log log.
Really small.
But we have this sorting, which is kind of annoying.
So the overall running time-- we'll worry about correctness in a moment.
We have to sort-- to sort E by weight.
So I'll just call that's sort of E. Then we have to do some unions.
I guess for every edge, potentially, we do a union.
I'll just write E times alpha of v. And then we have to do, well, we also have to find-sets, but same deal.
So find-set and union cost alpha amortized, so the total cost for doing this for all edges is going to be the number of edges times alpha, and then there's like plus v, I guess, but that's smaller.
That's a connected graph.
So other than the sorting time, this algorithm is really good.
It's faster.
But if you're sorting by an n log n algorithm, this is not so great.
That's how it goes.
I think you can reduce this to sorting just v things, instead of E things, with a little bit of effort, like doing a select operation.
But when this algorithm is really good is if your weights are integers.
If You have weights, let's say weight of e is 0 or 1 or, say, n to the c, for some constant c, then I can use rate x sort, linear time sorting, and then this will be linear time, and I'm only paying E times alpha.
So if you have reasonably small weights, Kruskal's algorithm is better.
Otherwise, I guess you prefer Prim's algorithm.
But either away.
I actually used a variation of this algorithm recently.
If you want to generate a random spanning tree, then you can use exactly the same algorithm.
You pick a random manage that you haven't picked already, you see, can I add this edge with this test?
If you can, add it and repeat.
That will give you a random spanning tree.
It will generate all spanning trees uniform leap likely.
So that's a fun fact, useful thing for union-find.
Let me tell you briefly about correctness.
Again, we proved correctness with an invariant.
Claim that at all times the tree T of edges that we've picked so far is contained in some minimum spanning tree, T star.
T start is going to change, but I always want the edges I've chosen to be inside a minimum spanning tree.
Again, we can prove this by induction.
So assume by induction that this is true so far, and then suppose that we're adding an edge here.
So we're converting T into T prime, which is T union e. By the data structural setup, I know that the endpoints of e, u, and v are in different connected components.
In general, what my picture looks like, is I have some various connected components, maybe there's a single vertex, whatever.
I've built a minimum spanning tree for each one.
I built some tree, and I actually know that these trees are contained in one global minimum spanning tree.
OK, and now we're looking at an edge that goes from some vertex u in one connected component to some vertex v in a different connected component.
This is our edge e. That's our setup.
Because the union-find data structure maintains connected components, that's another invariant to prove.
We're considering adding this edge, which connects two different connected components.
So I want to use the greedy choice property with some S. What should S be?
I want e to cross a cut, so what's a good cut?
Yeah?
The connected component of u and then everything else
Connected component of u and everything else?
Yeah
That would work, which is also the opposite of the connected component containing v. There are many choices that work.
I could take basically this cut, which is the connected component of you with everything else versus the connected component of v. I could take this cut, which is the connected component of u only versus everybody else.
Either of those will work.
Good.
Good curve, all right.
So let's say S equals the connected component of u, or connected component of v. e crosses that, all right?
Because it goes from u to v, and u is on one side, v is on the other side.
I wanted to include an entire connected component because when I apply the greedy choice property, I modify T star, and I don't want to modify, I don't want to delete any of these edges that are already in my connected components, that I've already put in there.
But if I choose my cut to just be this, I know that the edge that I potentially remove will cross this cut, which means it goes between connected components, which means I haven't added that yet to T. So when I apply this greedy choice property, I'm not deleting anything from T. Everything that was in T is still in T star.
So that tells me that T prime is contained in T star prime.
The new T star that I get when I apply the cut and paste argument, I modify T star potentially by removing one edge and putting e in.
And the edge that I remove was not already in T, which means I preserve this part, but I also get that my new edge e is in the minimum spanning tree.
And so that's how you prove by induction that at all times the edges that you've chosen so far are in T star.
Actually, to apply the greedy choice property, I need not only that e is cut-- sorry, that e crosses the cut, I also need that e is the minimum weight edge crossing the cut.
That's a little more argument to prove.
The rough idea is that if you forget about the edges we've already dealt with, e is the globally minimum weight edge.
OK, but what about the edges we've already dealt with?
Some of them are in the tree.
The edges that are in these-- that are in T, those obviously don't cross the cut.
That's how we designed the cut.
The cup was designed not to cross, not two separate any of these connected components.
So all the edges that we've added to T, those are OK.
They're not related to the edges that cross this cut.
But we may have already considered some lower weight edges that we didn't add to T. If we didn't add an edge to T, that means actually they were in the same set, which means also those are-- I'm going to use my other color, blue.
Those are extra edges in here that are inside a connected component, have smaller weight than e, but they're inside the connected component.
So again, they're not crossed.
So they don't cross the cut, rather.
So e is basically the first edge that we're considering that crosses this cut, because otherwise we would have added that other edge first.
So here, we have to do sort of the greedy argument again, considering edges by weight and e is going to be the first edge that crosses this particular cut, which is this connected component versus everyone else.
So e has to be the minimum weight edge crossing the cut, so the greedy choice property applies.
So we can put e in the minimum spanning tree, and this algorithm is correct.
OK?
So we've used that lemma a zillion times by now.
That's minimum spanning tree and nearly linear time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
Welcome back from spring break.
Hope you had a nice time off and couldn't wait to get back to 6.046.
So exciting second half.
Going to do flow networks today on Thursday.
Next week, it's linear programming, on to complexity, distributed algorithms, cryptography.
Topics going to come fast and furious.
Hopefully, they'll all be interesting.
So there's a lot of setup associated with network flow, max flow.
It's an optimization problem.
And so I'm going to spend like the first hour here-- and hopefully, it won't be too boring-- setting up flow networks, describing the max flow problem.
As you can see, this outline is fairly involved, talking about cuts in a network, residual networks.
And we'll, more or less, end the lecture with the statement, though not the proof-- we'll save that for next time-- of the mas-flow min-cut theorem, which is really an iconic theorem in the literature, and suddenly, the crucial theorem for flow networks.
And we'll take the max-flow min-cut theorem and use that to get to the first ever max-flow algorithm, which was due to Ford and Fulkerson.
And that should be, pretty much, at the end of today's lecture.
And next time, we'll talk about the proof of max-flow min-cut, talk about some of the issues with Ford-Fulkerson, and then use max flow as a hammer to solve interesting problems in bipartite matching, baseball playoff elimination, and things like that.
So just like shortest paths, can be used not just to compute the shortest distance from point A to point B, you can imagine that other problems, for example, scheduling problems, time problems, could be solved using Dijkstra-- and you've probably seen examples of that-- max flow is another algorithmic hammer that's being used to solve a wide variety of problems.
We won't really touch on that aspect today, but we'll spend a bunch of time on Thursday talking about that.
So let's get started.
We're going to start with defining what a flow network is.
And at some level, it's really simple.
I mean, it's a graph.
A graph is going to have vertices and edges.
We're going to only look at directed graphs.
And we're going to have a couple of constraints associated with these directed graphs that are going to make the algorithms, and the proofs, and the notation a lot simpler.
I'll get to that in a few minutes.
And the key thing with the flow networks-- I mean, just like you have a source and a destination for shortest paths, in a flow network as well, you're going to have a source, and you're going to have a sink.
So we're going to have distinguished vertices, two distinguished vertices.
And we're going to call them the source, s, and a sink, t. And so the basic idea here-- and just to sort of set things up right up front-- is that there's going to be some flow coming out of s. And it's going to have to obey some constraints associated with capacities of the edges, in order to make its way to t. And along the way-- this is flow.
This is water.
You can think of it as water, or cars, or what have you.
It's a rate.
You're not going to allow accumulation in the intermediate nodes.
So you're going to have law of conservation associated with this commodity that's flowing, be water, or cars, or people.
And that essentially comes down to, for any vertex, other than s or t, everything entering the vertex has to leave the vertex.
So that's like Kirchoff's current law, for example.
So there's a lot of analogies here with respect to real life, or electricity in this case.
And you'll see that, I think, over and over as we go along.
So let's take a look at a flow network.
I'm not going to draw it out here.
I'm going to keep this one example all through the lecture.
But just to go off and talk about edges and capacities before we actually draw an example, we're going to have edges, directed edges, u, v. So you have an edge from u to v. And it's going to belong to E, obviously.
That's the reason the edge is in the network.
And this edge is going to have a non-negative-- each edge is going to have a non-negative capacity, C(u,v).
Right?
And if perchance theres' no edge from, let's say, s1 to s2-- these are different vertices from the source or the sink, just as an example, you can say it's between a and b-- then, we can assume that the capacity is 0.
So if u, v does not belong to E, then assume that C(u,v) is 0.
So there's no way of getting from u to v. This rate, wall water, cars-- you know, there's no road that gets you from u to v, OK?
So let's draw an example of a flow network.
We'll talk a little bit about what the flow is and what the max flow is in example-driven algorithm design, if you will.
So I've got a source, s. And I've got a sink, t. And then, I've got a bunch of nodes in the middle.
Just draw them out.
I'm not going to bother naming those nodes.
Well, that might be a mistake.
We'll see.
So this is not like it's an acyclic graph.
You're allowed cycles in this graph.
But we will have a couple of constraints that are associated with this graph that I'll get to in just a minute.
So first, that's all we've got.
You've got g,v,e, directed graph.
Can't have cycles in it.
And I'm going to draw a couple of numbers here.
The numbers I'm putting up here are the capacities associated with each of these edges.
So those are just the capacities.
Those are C(u,v).
And I said that, if u,v does not belong to E, then you're going to assume that C(u,v) is 0.
So there's no edge, for example, between this node and that node.
So there's no way that you can directly get from here to there.
You can go like this, like this, and like that, and get from here to there.
And you can obviously go like so, and like this, and like that, to get from the one on the top left to the one on the bottom right, OK?
So that's essentially the set up, except for the fact that we've just talked about capacities.
And this is sort of the bandwidth, if you will.
This is the amount of traffic that can go through this road.
And now, we have to talk about the specific case where we're actually going to shove things through the network.
So we're going to have another number.
So typically, we're going to have two numbers associated with each edge.
One of them is going to be the capacity, and the other one is going to be the flow that goes through the edge.
And as you can imagine, we're going to have this constraint that says that the flow can never exceed the capacity.
And that's the local constraint associated with each edge, OK?
That doesn't mean that there aren't variations possible with respect to the overall flow of the network.
Things can change.
You obey the edge capacities.
And there could be different flow coming out of s and going into t, et cetera.
And so there is going to be an optimization here associated with the exact numbers that all obey these edge constraints, so these edge capacity constraints, OK?
So let's take a real simple example of a flow in this network.
And ah, I have some colored chalk here.
So I can put down the flow over here.
And so this is flow.
Wow.
That is an ugly color.
[LAUGHTER] But now, we're stuck for the rest of this lecture.
So I've got 2, 2, 1, colon, 3.
So the first number is the flow, and the second number is the capacity.
Let me just write that out.
This is-- and then, this is 1, colon, 3.
This is 1, colon, 2.
So that meets the capacity.
This is 2, colon, 3.
And then 1, colon, 3, 2, colon, 3, and 1, 2, right?
So that's my first example of a flow.
And we want to make sure that this flow makes sense, OK?
And we're going to write this down a little more precisely in just a minute, but I've sort of given you the intuition already.
I've talked about it's OK to have a flow from the source.
It's the mountain spewing water, or that's the source of the river.
And this is the sea, for example.
And the river flows into it.
But along the way, you really can't have accumulation, OK?
Because when you think about it as a rate-- and so this is sort of gallons per second-- then maybe you'll have a little bit of accumulation allowed.
But over a huge period of time, you can't have infinite accumulation.
So that's where the conservation law kicks in, which says, anything that goes into a node that is not marked s or t has to leave the node.
So you look at this, and you go, well, there's two things coming in here.
There's 1 and 1, which adds up to 2.
And there's two 2 leaving.
So we're good there.
OK?
And then pick another one.
This one, for example.
You've got 2 coming in, and you've got 1 and 1 leaving.
We're good there, right?
For t, you've got 3 coming in, 2 plus 1.
And another check to do is, well, you've got 2 coming out of the source, right?
So hopefully, that all will make sense.
And ask questions if you're confused.
One other thing to look at, which is interesting, is that you could have flows that are essentially cyclic.
You could have commodities that are flowing in a little cycle.
And can you see that?
Can anyone see that over here?
Yep?
Go ahead
The bottom-right triangle [INAUDIBLE]
Bottom-right triangle?
This thing over here?
Yeah
Like that?
Good.
Right?
So you see this one going over this way, this way, and that way?
And so there's nothing that's stopping us from taking this and-- I had 1 over here, so I'm going to make that 0.
I had 2 over here, and I'm going to make that 1.
And I had 2 over here, and I'm going to make that 1
[INAUDIBLE]
[INAUDIBLE]
Oh.
What is it?
It was 1
It was 1 before?
Oh, you're right.
It was 1 before.
So I should make that 0 then.
Good.
I wanted to subtract 1 from what I had before, right?
And I screwed that one up, the simple thing.
All right?
Wow.
No wonder I didn't major in mathematics.
[LAUGHTER] OK. Computer Science is this forgiving field.
So you can go 0-- well, this is 0.
So now, let's take a look at what we have here and check that nothing went wrong, right?
We want to check that nothing went wrong.
And so 2 coming in, 1 going out, 1 going out.
And nothing coming in here, so we're all good.
And so on and so forth, OK?
So there's interesting things happening here with respect to the conservation laws and having to obey them.
In general, you can imagine that what we're interested in is simply maximizing the flow from the source, and getting as much flow out of the source as possible, and pushing it into the sink.
OK?
Now, the flow here into t, as you can see, is 2 plus 1, which is 3, OK?
Can anyone take this particular flow network and increase the flow?
I mean, it's easy to decrease the flow.
You can make everything 0, and that'd be valid, right?
But can you increase the flow?
Do you think that this flow network allows for a larger flow than 3, given that the capacities of those edges are fixed and I'm not going to change them?
Let's see.
I think all of you have Frisbees, right?
Yeah.
Yeah, back there
I don't think you can, because you're maximizing two edges, which would have to be increased.
Like, the bottom one coming out of s would have to be increased.
Or the one on the very top would, too, also have to be increased
People agree with that?
Over there?
Yep?
You can have a path going along the top with a flow of 2, so that s is the one on the top of it has 2.
That, from that, to the right, has 2, like it has now.
From that node, the t has 2.
And then, along the bottom, we have another path that has 2, from s to the 1 [?
plot of it.
?]
And then [INAUDIBLE]
OK.
So-- [LAUGHTER] That made perfect sense-- to me.
[CHUCKLES] It did.
So one of the crucial observations that [?
Rajesh ?]
made here-- and let me just focus in on that-- is-- and that's why I think the other gentleman said no-- is that you can actually decrease the flow in particular edges.
And that's going to help you increase the overall flow, right?
And that's why this problem is challenging.
That's why we need so much of a setup, all right?
So one thing that I could do is I could essentially say, I'm going to take this and make this 0.
So when I do that, essentially what I have is I get to push more flow out from here, right?
So I get to push more flow.
I can turn this into a 2.
And I can turn this into a 2, right?
Am I done?
No
Not quite.
I mean, I've got one little bug here, but I can fix that, right?
I've got one little bug here which says, I've got 2 coming out here.
I made this 0, so I have to push something more.
But hey, I'm lucky.
I've got a s here, which is giving me as much as I want, correct?
So I can just make this a 2.
And so now we get a flow of 4, right?
You both get Frisbees.
Shall we do blue and purple?
Here you go.
Could you stand up?
Oh, OK. Over there.
So that actually kind of summarizes, at some level, our task ahead, right?
So we have to find ways of increasing the flow.
And sometimes, we have to take a step backwards in the sense that we decrease the flow on a particular edge, right?
So it's not this monotonic increase that Dijkstra would do or a greedy algorithm, like MST that Eric talked about.
There's going to be something a little more interesting here.
We eventually are going to end up doing things in a monotonic way, in terms of the overall flow.
So the max flow that we're trying to get at is going to start with the current flow.
And we're going to improve the current flow constantly.
But that doesn't mean that the edges are going to look monotonic in terms of the flows on a particular edge in relation to the capacity.
You'll never exceed the capacity.
But as you saw in this little example already, we increase from 3 to 4, by taking that edge that was vertical up there, which says 0, colon, 3, and that was 1, and we shrank the flow on it.
And so how are we going to discover these paths, especially if we have a 5,000-node network and-- I don't know-- 10,000 edges?
And so that's essentially what we have to do for the rest of this lecture.
Any questions so far?
OK.
So we've done flow networks.
I kind of defined what the max flow problem is.
And let me just write that out more precisely.
Given a flow network, G, find the flow with maximum value on G. And for that, the max flow is 4, right?
So that's another thing we haven't actually done, which is obviously important for us to do, which is we have to show that the 4, in this case, is the max flow, right?
So now, of course, I've given that away.
And so I'm not going to ask you can you push this over to 5.
But you might think that 5 is a possibility, simply because the capacities of the edges that are coming out of the source are 3 plus 2, which is 5, right?
So it's certainly possible that you could push at least, if you only look at those two edges, that you could push 5 units from the source.
But in this case, in this example, if you obey the laws of conservation, you cannot obey those laws and get 5 units from the source to the sink, t. But we have to prove that.
And we have an algorithm that says this is the best that you can possibly do.
And the algorithm terminates when that happens.
And that's our Ford-Fulkerson algorithm, right?
Good.
So that's what we have so far.
I want to talk about flow network assumptions.
And I can do that over here.
This is going to make our life easier.
One of the things that's a little bit confusing sometimes is the circular flow and the fact that we're going to potentially have flows that correspond to edges coming in and edges going out.
So for example, if I had something like s and u, for example-- and s could be the source in this case, or it could be another node-- suppose I had a little subnetwork that looks like this.
And I'm just giving you what the capacities are.
This is a bit strange in the sense that you could have a situation where you essentially have zero flow, really, because you have one unit coming in here and one unit leaving.
All right?
And you can think of this-- let's just call this s1, to make it clear that it doesn't have to be the source, right?
And so you could have the circularity that you saw over there.
And now you're talking about, well, I might have 1, colon, 1 here, and 1, colon, 2 here, which is fine for this subnetwork.
But if I have stuff going out, what happens with that?
Well that's got to be a 0.
I could have a 1 and a 2.
And if I have a 2 here, then-- if I had a 1 and 2 here, then that doesn't work, because maybe I need something else coming in.
So you can see that, pretty quickly, it gets kind of confusing, if we end up having these cycles that are such simple cycles, especially the ones where you have Su and Us, OK?
And so we're going to disallow cycles of two kinds, right?
The first cycle we're going to disallow is this simple one where we say, if I have a, then no self-loop edges allowed.
So that would involve accumulation at a particular node.
And it's going to make things really confusing.
And CLRS disallows that.
And most flow network algorithms assume that you're just going to discard these cycles.
This particular transformation that I'm going to describe here is something that is going to be forced on you.
And this is for your benefit in over 6 lectures and sections.
But it's not actually something that CLRS follows.
And it's going to make things simpler, though.
And what we're going to do is we're going to take any pair of vertices that has this characteristic, where you have s1, u, and u, s1, and they have non-zero capacities.
So s1, u has a capacity of 1. u, s1 has a capacity of 2.
Both of these edges exist.
And if these edges exist, that means they have non-zero capacity, positive capacity.
And we're going to transform that, very simply, into-- this is not changing the generality of the algorithm, but all I'm going to do is transform it-- call this s1-- into something that satisfies this restriction.
So I could have 2 here, 1, and 1, OK?
So all I've done is introduce u prime, right?
Then I can always do this-- if this is trivial for any pair of vertices, I can introduce one other vertex.
It all works out.
Linear expansion, constant factors, ignore them.
Life is wonderful.
So all I've done is taken away the situation where I have-- you can think of it as two-way streets, right?
Two-way streets are annoying.
You don't quite know what the rate of traffic is from one end of the street to another, because cars are going in both directions.
And you have to do subtraction.
Subtraction is painful, so we don't want that.
So we're just going to assume that this is u prime.
And now, you're all set.
We allow this.
So we have to allow for generality reasons.
As we saw in that very first example there, we are going to have cycles here, OK?
But we just don't want cycles to be of length 1 or 2, OK?
So that's essentially what we're going to disallow.
All right?
The good news is that, if you do this, then we'll only have a single notion of flow.
Whereas, if you go read CLRS, you'll see that there's two notions of flow in CLRS.
There's positive flow, which is different from net flow.
And the positive flow is different from net flow in graphs that have this particular structure, or have nodes with these properties.
But if you disallow them, then you can just talk about flow, and it doesn't matter.
Positive flow is the same as net flow, all right?
So for the purposes of [?
6 over 6, ?]
for this semester, we're going to simply think about flow and equate that to positive flow, equate that to net flow.
And it's all going to work out, assuming your graphs satisfy these two properties of the cycle lengths.
All right?
Cool.
Good.
So let's keep going here.
So we're up to finished up on max flow.
Let me just give you some sense of notation.
I've talked a lot about constraints.
But we've got to write some stuff out, because we're going to be getting more precise and proving things in just a few minutes.
So what is a flow?
Well, to be precise, it is going to be a function that satisfies the following properties.
It satisfies the capacity constraint.
This is the obvious capacity constraint, intuitive capacity constraint.
And then we've got flow conservation.
And the important thing here is that I don't have it for all V, but I do have it for vertices, V, that are not the source or the sink.
And I'm going to require f(u,v) equals 0, right?
And the last one, which I haven't talked about, but becomes easy to talk about, given this constraint, is skew symmetry.
So if you take-- this doesn't have to be an edge between u and v. Now, I'm talking about the flow, f, between u and v. And so the u could be s, which is the source.
v could be t, which is the sink.
So in general, I'm not talking about a flow.
And there obviously has to be a path from u to v, in order for there to be a non-zero flow, f(u,v), right?
If there's no path, there's no way of getting there.
But having said that, the definition of a flow here in our network is the straightforward definition, which simply says, if there's a flow from u to v, regardless of what u and v are, then the value of that flow is simply the negation of the value of the flow from v to u, which makes perfect sense, right?
And this all works out under the definition of net flow.
And so this is essentially what the definition of net flow is in the textbook.
But for our purposes here, we don't have to add that adjective.
We just are going to be talking about flows.
Positive or net, they're the same, all right?
All right.
Good.
So one of the things you can do with this notation-- and we're going to use what's called implicit summation notation on top of this-- is to prove some interesting things, interesting theorems, that give you some intuition as to how algorithms on flow networks are going to work.
And in particular, we're going to use this notation when we talk about the value of a flow.
So the value of a flow, f, is denoted-- you can think of it as a cardinality of f. And f is v, belonging to capital V, f(s,v).
And that is f(s,V), so what I have written here.
Well, given a flow network, I want one particular quantity that I want to maximize.
And that particular quantity is going to be defined, based on how much I can push from s, how much can I push outward from s. That's the crucial quantity that I want to maximize.
That quantity is-- you think about everything that is going out of s, and you add it all up together.
So from s, you look at any other vertex, every other vertex, and you say, what is f(s,v)?
And I'm not talking about just the edges that come out of s. A vertex, v, small v, can be any vertex.
If I add up all of the flows that come out of s, then that is the flow that responds to my flow network.
That is everything that's getting pushed out of s, OK?
Now, it may be the case that-- remember, I'm talking about flow here-- so it may be the case that you have an edge from s coming in from v4.
And there may be a flow associated with that.
This is maybe something like 1, colon, 2, all right?
So what this means is that f(s,vr) is-- this is f, remember-- so this is minus f(v4,s).
And in this particular case, this is 1.
So this is minus 1, OK?
So keep that in mind.
When I talk about the flow of the network, I'm going to be looking at the source.
And I'm going to be looking at all of the flows that are going outward.
And I have to keep in mind the skew symmetry relationship.
I obviously have to obey capacity constraints and the conservation laws, all right?
So given that, let's use this implicit summation notation and show some simple properties of flow.
So let's look at-- one thing I want to emphasize is what I've done here is use this implicit summation notation, which simply says, if I see a capital letter here, that's a set.
And I'm going to have to enumerate all of the members of that set.
And it's implicit summation.
So as I enumerate those members, I'm going to add up all of these quantities, right?
So that's really what this means.
So the sigma here gets embedded into this capital V. So two things going on.
The small v turned into capital V, because I'm looking at the entire set.
And the sigma gets in there too.
And that's why it's implicit summation, not just implicit set notation.
So some simple properties.
I can say, f(x,x) is 0, where x is an arbitrary set.
All that says is, let's say, x has a single member in it, which is a.
Then f(a,a) is always 0, because if you don't allow self-loop edges, and that's pretty much all you need.
If you have a pair of vertices here, a and b, then what you're saying is f(a,b) plus f(b,a) is 0.
And that's true, because of skew symmetry.
Right?
We just wrote that out.
So f(x,x) is 0.
And in general, you can say, even though X and Y are sets of vertices, I'm going to be able to use skew symmetry to say that f(X,Y) is minus of f(Y,X), all right?
Similar argument.
And then, lastly-- there's any number of these, we just do three of them here-- X of f(XUY, to Z) is f(X,Z) plus f(Y,Z)-- we've got to use these properties to prove our first theorem here on flow networks-- if X of X intersection Y is null.
So you don't want to double-count.
So that's all this is.
Make sure you're not double-counting.
You've got f(XUY), and you want to look at that entire set, the union, and then look at the flow from any member in XUY to any member in Z.
And you can do that by breaking it up, provided you're careful about double-counting.
And the fact that the two sets, X and Y, do not have an intersection, or they have a null intersection, implies that you're OK with the [?
write ?]
inside.
All right?
So you might be going, why are we doing this?
Well, here's a good reason to like implicit set notation.
You can prove some interesting theorems in a very elegant way, using this notation.
So let's do one example of that.
You'll probably see others in section.
So one of the things that we'd like to do is prove a pretty important theorem, which I think all of you probably can assume in your heads, given all of the properties of flow networks that we have.
And it's a very simple theorem that simply says, I have the law of conservation that is applied on all of these intermediate vertices, and I've got a bunch of commodities, I've got a flow going out of s, right?
So where can this flow go?
Where does this flow end up?
It ends up at the sink, at t, right?
So the point is that, if you have all of these properties that we have up here, you're going to be able to show-- and you want to show this, you want to prove this-- that the value of a flow, which is defined as what gets pushed out from the source, is exactly what goes into the sink, right?
If that's not the case, there's been a violation of some property, perhaps a capacity constraint, perhaps, more likely, a conservation constraint, OK?
So the theorem that we'd like to prove is simply that f is f(v,t), right?
That's the theorem.
And there's a lot going on here, so it's worth spending 30 seconds looking at what exactly this means.
What I have here is that, if you just look at this and that, I'm saying that f is what gets pushed out of the source, OK?
And now what I'm saying here is that f, the value, is exactly what gets pushed into the sink, OK?
So this is what I have to prove, right?
And I should be able to prove that, by invoking my laws.
That's it.
I mean, that's my axiomatic system.
I've got those laws.
I've got a definition of a flow that may not necessarily be the max flow.
It might be something much less than the max flow, it might be the max flow.
Regardless, what gets out of the source has to get into the sink, right?
So how are we going to do that?
And the implicit summation notation is going to give you, essentially, a three or four-line proof, which is very intuitive, right?
So let's do that.
And maybe you can help me.
So we're going to start with what we know.
So that's the proof.
f equals f(s,V).
Right?
So that's what we've got.
That's the definition of cardinality of f, or a value of f, OK?
What I'm going to do is I'm going to say this is the same as-- I'll give you the first step, and then let's see if you can help me with the remaining-- is the same as f of v minus s capital V, right?
So if you're having trouble differentiating between my cap V's and small v's, holler.
I'm trying to write them as big as possible.
Yeah?
Could you maybe put little hats on the top of them?
Put little hats on them.
Yes, I will put little hats on them.
I'd put little Frisbees on them, if I could, but-- I like Frisbees much better than hats.
All right.
That's good.
That's good to do.
So yeah.
So I think, hopefully, I'll keep doing this, and it won't be confusing.
So what I've done here is invoke, essentially, this, except it's not exactly that, in the sense that it's written a little bit differently.
But if you see what's going on here, what I've done is look at this s, and I've said, think of this s as being cap V minus s. Right?
So that gives you s. And those are clearly disjoint, right?
Those are clearly disjoint sets.
There is this one and this one are disjoint sets.
That's what I mean to say.
I mean, these two aren't disjoint, but this and that are disjoint.
And that's what you need, in order to invoke the little property that you have here.
And so that all make sense?
You see why I did that?
OK?
What can I say about either of these two quantities?
Can I say something about either of these two quantities?
Yeah?
f(V,V) is 0
f(V,V) is 0.
That's exactly right.
f(V,V) is 0.
There you go.
Yep.
So this is simply I'm going to invert that.
I'm going to write this as f of V hat minus s, OK?
I'll just flip this.
I had a negative sign here, but I've flipped that.
And skew symmetry tells me I can do that, right?
All right.
So I'm up to this point here.
Now, what I'm going to do is I'm going to do f(V,t).
And the reason I want to do this is because this is where I want to get at, right?
Eventually, I want to show something that corresponds to f(V,t), right?
And what I have here is V,t.
But now, I could do plus f(V,V minus-- cap-- minus s minus t) right?
So what I've done here is taken V minus s and pulled out t from it.
Remember, t is part of cap V. Cap V contains all of the vertices.
So I've pulled out t from it, but that implies that I have to do a V minus s minus t over here.
And again, they're disjoint, so it's all good.
What can I say about this?
Yeah?
It's 0 because of flow conservation?
It's 0 because of flow conservation.
That's exactly right.
We didn't quite write it that way.
But if you look at what the implicit summation notation would mean for that, you look at it and you say, maybe one more step would be, let me think about this as being, f(V,t).
It'll become more obvious if I write it this way.
f(V-- I'm putting a minus in here-- V minus s minus t and cap V again, right?
So all I've done here is flip these two.
Skew symmetry allows me to do that.
And now look at what I have here.
I'm talking about a flow that corresponds to some-- for any vertex, I pick-- and it's not an s vertex, it's not a t vertex, so it's an intermediate vertex.
And if I look at an intermediate vertex and look at the flow that goes out to all vertices, conservation says that has to be 0, right?
So that's exactly what this says.
For any u that's neither s nor t but in V, the sum has to be 0.
So this is zero, and we're done.
All right.
Oh, you-- a Frisbee?
Who is that?
Ah.
Here.
So that's the power of implicit summation notation.
So we could invoke these different properties.
It was fairly straightforward.
Your first example of this.
You'll probably see a few more.
All right?
Any questions so far?
OK.
So as you can see, as I promised, or threatened at the beginning, but followed through on my threat, we have a lot of notation, a lot of baggage here before we get to algorithms.
But we're slowly getting there.
The next major concept is the concept of cuts.
So a cut, you think of a cut as being, well, a cut through paper, a cut through the air, whatever.
It turns out that notion of a cut in a network is more general than that, right?
A cut is basically a partition.
A cut is a partition of nodes.
And a partition means that you can't have a node in both sides, right?
So a cut is going to give you two disjoint components at the end of it.
But the cut doesn't have to be something contiguous.
It doesn't have to be a line through the network.
And everything to the left of the line is in one half of the cut, and everything on the right of the line is in a different half of the cut.
I can just break up these nodes into two disjoint parts.
And the only constraint that I'm going to ask for is that s, which is the source, is on one side of the cut, and t, which is the sink, is another side of the cut, OK?
That's it.
And given that, I'm going to say interesting things, really interesting things, about the flow through a cut, OK?
And so let's do that.
Let's define a cut.
So a cut is (S,T) of a flow network, G, is a partition of V, such that small s belongs to cap S, and small t belongs to cap T. I don't know.
Do you want hats on the T too?
I'll just write them large.
If a flow on G-- if f is a flow on G, then the flow across the cut is f(S,T).
OK?
So again, implicit summation notation here.
The flow across the cut is as the sum of the flows corresponding to each pair of vertices, such that the source vertex is part of capital S. And the destination vertex is part of capital T. All right?
That's it.
I'm just going to add them all up.
That's the flow across the cut.
So what I can do now is just talk about-- let's just go up here back to this.
And I'm going to look at exactly what I have here.
Is that right?
Not exactly.
I'm going to change this a little bit, because I want to make sure I don't have to add up numbers and do that incorrectly.
So I need a 1.
Yup.
That's all I need to do is change it.
So I'm going to change our example here, not the topology of the example, but the actual numbers.
And you'll need to verify that what I have here satisfies our flow network properties.
And there's one more.
OK?
So I think I'm good.
All right.
So this is going to be an example of a cut.
I haven't defined the cut yet.
Let's get rid of that.
Holler if you think there's something wrong with this flow.
All right?
I think I got it right.
It satisfies capacity constraints.
It satisfies flow conservation constraints.
The flow that is going into t is 4.
This happens to be a max flow.
Doesn't really matter.
So what we're going to talk about with respect to cuts, it doesn't require the flow to be maximum.
Keep that in mind.
What do I mean by the flow across a cut via an example?
I'm going to simply say that the shaded nodes, two of them, are part of capital S, OK?
So as you can see, I just arbitrarily picked a couple of nodes.
And that not necessarily something that can be easily partitioned using an actual cut line, a physical cut line.
I just picked that one over there and the one over here with S. And so, I can now look at this, and I can compute numerically, for this example.
And it's worth doing at least once what the flow across this particular cut is, defined by the particular choice of cap S and cap T, OK?
And that's what we're going to do.
So f(S,T) is-- I'm going to have to look at pairs of nodes, such that I've got a shaded node on the left-hand side and non-shaded node on the right-hand side.
And I'm going to have to go through all of the combinations, right?
So if I look at this, I can first knock off this one, and that one, and that one.
Let me actually put in-- let's call this a, b, and c here.
And we can call that d. So we have s and d as being part of the cut, in terms of s, capital S. And the other ones are in cap T. And so what I have is I could do Sa and Sb.
So I've got 2 plus 2, all right?
And this would correspond to Sa and Sb.
So those are going out, right?
So far, so good.
And then, I'm going to write out a bunch of numbers here, minus 2 plus 1 minus 1 plus 2.
And the minus 2, where would the minus 2 come from?
Well, an a,d, for example, is a minus 2, right?
Because d is part of-- it would be d, a.
So a, d has a flow of 2, correct?
And so d, a has a flow of minus 2, right?
And d, a is part of what I have here, because d is part of capital S, and A is part of capital T. You guys see that?
So this is not trivial, so pay attention.
So this would be, for example, the minus 2 would correspond to d, a.
That's what I need here.
And I could also have-- what do I have here?
I have something is going into d. So a c, d is 1.
So d, c is minus 1, right?
Make sense?
d, c is minus 1.
What about the plus 1?
Where do I get a plus 1 from?
d, b
d, b is going out.
That's exactly right, d, b.
And the plus 2, it would be d, T, right?
And so you have to do the enumeration.
It's worthwhile doing once.
And then it gets kind of boring.
We won't to do it again.
But you have to realize that you have to absolutely look at every pair of vertices.
And you have to use skew symmetry and ensure that, even though there's actually no edge going out, if there's an edge coming in, you've got to count that.
And that's going to get a negative.
Whatever is coming in, you've got to subtract, OK?
So it's not that complicated.
Yeah, go ahead
Do we not consider S, c?
I'm sorry?
Do we not consider S, c?
So the beauty of this is that, when you don't have a particular edge from S to c, you can use skew symmetry to argue that S, c and c, S cancel out each other, all right?
So that's the good part, right?
And thanks for asking the question.
That's a good question.
All right.
Here you go.
So you can do that by just looking at the edges.
And you can add up the numbers, all right?
And so I don't think this is going to be absolutely crucial to understand the rest of the lecture.
Keep this in mind, that there's a process by which you define the value of a cut.
And we're going to get back to this, when we prove the max-flow min-cut theorem next time.
But at this point, I want to say something actually much more straightforward, which is going to be important when we look at residual networks, which is the last concept that we need to get at before we get to an algorithm.
And that is simply that the capacity of a cut and the relationship between the capacity of the cut and the flow of a cut.
So the capacity of a cut is c(S,T).
Oops.
I didn't draw that properly.
Open brackets, capital S, capital T. And we can do it exactly the same way, except this is a lot simpler, because you only look at edges and you only have positive quantities.
So in this case, you'll simply say it's 3 plus 2, corresponding to-- what did I have here?
I had d, a-- S, a and S, d. And then, the capacity is you only need to look at the edges that go from a node in S to a node in capital T. And so those are 1 plus 3.
And this simply would be the 1 would be d, b.
And the 3 is d, t. So you don't care about the other flows.
This is not about flows, this is simply about capacity.
So this adds up to 9, OK?
And so we have, at this point, we have defined the flow through a cut.
And we know the capacity of a cut, OK?
It's more or less obvious-- though you could certainly prove a theorem which is going to take a couple of lines-- to say that the value of any flow is bounded by the capacity of any cut.
And sorry, I lied.
It's not a trivial proof.
And that is actually something profound going on here.
And so I'll have to explain exactly what this means.
And then we'll take a look at how we could prove something like this.
So what's cool about this is that you're saying that it's the value of any flow is bounded by the capacity of any cut, OK?
And so that's an upper-bound on the maximum flow value, right?
So I'm saying there's all these cuts that are possible in the network.
And I'm making a statement about what the maximum flow can be, based on the values corresponding to the capacities of any cut, right?
So why is that the case?
Well, we're not going to be able to prove that fully today.
That's the max-flow min-cut theorem.
But you can certainly get a sense of it, by looking at a different characterization of the flow value.
So I'm going to give you one half of the proof, at least, and intuition about the other half.
And we'll finish it next time.
But here's another characterization of the flow value.
So our lemma here, which is going to lead us to this statement, is that, for any flow, f, and any cut, (S, T), we have a really powerful dilemma.
Maybe you should call it a theorem.
But it essentially says, look, it doesn't matter what cut you choose, you've got a flow on the network.
And when you look at the flow on the network, it's going to equal the flow across the cut.
And the only reason for this is simply because you've got the source on one side of the cut.
And you've got the sink on the other side of the cut.
That's it.
That's the only thing that you need, right?
You dump these vertices these into two bins.
You know, dump the source on the left, and dump the sink on the right.
And you compute the flow the way we've defined it.
That's the flow.
It doesn't matter how you partition these vertices, as long as you've got the source on the left and the sink on the right, OK?
And so we can prove this using implicit summation notation.
We'll do that.
And that'll give you a really good sense of why this statement is true, because we know that, for any given cut, the flow that cut is bounded by the capacity of that cut, right?
You know that.
But to show this, here's how we could show that, f(S, T) is f(S, V) minus f(S, S), OK?
So I'm playing around, just like I did before.
I had taken the cap T-- I know that S union T is cap V, right?
This is a partition.
So I know that S union T is cap V. So I can put a V here and an S here, right?
And that's a subtraction over there, of course, right?
So put that up here and finish this, a couple more lines.
And what can I say about either of these?
I could say something about one of these terms.
Yep?
The one on the right is [INAUDIBLE]
The one on the right is 0.
So call this f(S,V).
Right?
And now, I'm going to break it up again, make it small s, big V, plus f(S minus s, cap V).
So broken this up into small s, which is just joined from cap S minus s, clearly.
And what can I say about this?
This is a little more subtle than, perhaps, the previous question.
What can I say about that quantity?
I mean, the answer is not subtle, but-- yeah, go ahead
That that is equal to 0
And why?
Because the cap S doesn't contain t
Ah.
Beautiful.
That's right.
So that's what I wanted.
So this does not contain t. And so, now you can use flow conservation, right?
And that's the important thing.
You can use flow conservation, because this does not contain t. And then, it clearly does not contain small s, because I just took it out of it, right?
So that goes to 0.
And voila.
That's simply f(S, V), which we know is f. We proved that.
Our first implicit summation proof was showing that-- well, this is a definition.
Excuse me.
So we did it for the sink.
But this is simply the definition of the flow value, right?
So this is beautiful.
I mean, it's like fantastic, right?
Why aren't people excited?
[LAUGHTER] Because I put people to sleep before, in the hour before.
But this is absolutely fantastic, because this says that I have ways of figuring out what the maximum flow of the network would be, by making arbitrary cuts through this network and looking for capacities of these cuts, right?
Because I know that the capacity of any cut-- and now you see why min cut is interesting-- but you know that the capacity of any cut is going to bound the flow of the network, because the flow through a cut is the flow through the network.
So if I go through and look at the min cut corresponding to the minimum capacity associated with the flow network, that's going to point me to my max flow, because that going to be an upper bound on the max flow, right?
And so now you see why the min-- not the min flow, sorry-- the min-cut max-flow theorem is an interesting one.
But it relates-- and this is the beginning of that relationship, we're not quite done to prove it-- but the beginning of the relationship is that you can look at any cut, and you can look at the flow through the cut as being the flow through the network.
And then you use the second part of it, which is the capacity bounding-- of course, in a very simple way, because of edge capacities-- the flow through the cut.
And you can put those two things together, all right?
We still don't quite know how to find these cuts, right?
So we don't quite know how to find these cuts.
And that's the last thing that we're going to do today, give you a sense of how we're going to find these min cuts, so we can find the max flow.
All right?
Cool.
So the one last notion that we have here that is going to allow us to go into the algorithm domain, as opposed to the analysis domain-- all we've done so far is analysis, analysis, analysis-- is the notion of a residual network.
OK?
And a residual network, as its name implies, is something that has residual capacities.
It's the network that points you to places where you can increase the flow, because there's capacity left.
Your flow is less than the edge capacity.
It's a local notion, so it's easy to compute.
There's a capacity of 3 on this edge, there's a flow of 2.
The residual capacity is 1, right?
3 minus 2.
And so the residual network Gf (V, Ef), right?
So the actual network is G(V, E).
And the vertices are going to be the same.
The graph is obviously different, but the edges are going to be different.
There's going to be a different set of edges in the residual network, as opposed to the flow network, OK?
And you have strictly positive.
That means greater than 0, strictly greater than 0, residual capacities.
So Cf ) equals c(u,v) minus f(u,v).
And that's strictly greater than 0.
I'm going to put an edge in there, if there is a residual capacity.
0 doesn't mean there's any residual capacity.
Edges in Ef admit more flow, OK?
And one last thing.
If (V, u) does not belong to E-- so we are talking about the original network here, E means the original network-- then we know c(V, u) equals 0.
That was our definition.
If you don't have that edge, the capacity is 0.
But we are talking now about f, which is a flow, which doesn't necessarily require that there be an edge.
And we are simply going to use our skew symmetry relationship.
And you'll see that this may not be completely clear as to why I wrote this at this moment.
But as I draw the residual network, you'll see why that is important.
It's going to be the case that we're going to have extra edges in the residual network that don't exist in the original network, because of that last line there, right?
So let me clear that up.
So we're going to draw a residual network for-- I'm going to change that yet again-- but we're going to draw the residual network for our example up there.
Topology is going to stay the same.
Numbers are going to change, because I want something slightly more interesting than what we have there.
So I'm going to take this out-- 2, 1.
1, 2-- go back to what it was before, I think.
1-- all right.
Good.
So I want a 1 over here.
I want a 1 over here, a 2 over here.
And keep checking to make sure I'm not messing up here, in terms of flow constraints.
But that's pretty much all I got.
OK?
So the flow here and the s and t don't particularly matter.
The max flow is 4.
The flow that you see up there is 3.
This is what we had right at the beginning, all right?
So what I want to do now is give you what the residual network is for this particular flow.
Remember, the residual network is defined, based on a flow.
That's why you have Gf, G subscript f. f is a flow.
So you're going to have a different residual network, if the flow is different.
So that original example that I had would have a different residual network.
This one is going to have the one I'm going to draw, all right?
So the residual network has the same set of vertices.
So I can go ahead and draw these vertices.
I'll just mark T and S over here.
Those are exactly the same as before.
And this is Gf, OK?
That's a residual network.
And the edges are going to be different.
All I have to do is look up there and say, look, I'm going to have a residual capacity of 2-- let me use a different color, since I have them-- corresponding to that edge from S to a, because I clearly have a capacity of 3, and I only have a flow of 1, right?
So that's all there is to it.
Now, the interesting thing is that, because of this line over here, I'm actually going to define an edge in Ef, in Gf or Ef, that corresponds to this edge that didn't exist in E, because I can shrink the flow from 1 to 0.
And that essentially says that that shrinkage is represented in the residual network by an edge that goes from this node, a, up there, to s, right?
And that is going to have a residual capacity of 1, right?
So that's it.
That's the only extra thing that you have to remember when you draw the residual network.
You not only can increase the flow, you can also shrink it.
You have to represent the shrinkage of the flow by an edge in the residual network.
And you obviously represent the increase of the flow by an edge in the residual network.
And now you see why this is all going to make sense.
Remember, way back, eons ago, only an hour ago, but we had this example where we had to shrink the flow in a particular edge, in order to get the overall flow to increase, right?
The residual network is going to point us in the direction of, potentially, those edges whose flow has to shrink.
But they're going to be represented, in effect, as these reverse edges with positive numbers associated with them.
So there's a 1 here.
It's positive, because it goes from this node, call it a, back to s. And if I shrink this from 1 to 0, that is, in effect, taking what I have up here and making this 1 as 0, OK?
That's the way you want to think about this.
So that's pretty much it.
I could draw out the rest of this, and it should all make sense.
I have an edge like that.
So the edges that are at capacity up there aren't going to show up here.
The edges that are not at capacity end up with two edges down below, if they have a flow that is non-zero, right?
If you have a 0 flow up there, you're only going to get one edge.
Obviously, you'll get one edge, because the capacity is non-zero.
But the ones that are not quite at capacity end up to two edges, right?
Think of that as being the simple rule.
And make sure that I'm following this rule.
Good exercise is to check for bugs, lecture bugs.
Best way of understanding the material.
OK?
So that's my residual network.
And let me just point-- I'll put in some numbers.
Or maybe you can tell me what some of these numbers are.
What is this number?
2
2.
Right?
Now, this is a little more tricky.
Take a look at the number that goes up, versus the number that comes down.
What is this number?
Goes up
2
That's 2, because I can go from 1 to 3.
And this number would be?
1
Beautiful.
All right, you guys got it.
Did my job.
All right, so that's our residual network for this particular flow.
So a mechanical way of computing it.
And you should be able to do that.
Now what exactly can we do with this residual network?
It turns out that the algorithm now can be described in a couple of sentences, right?
Essentially our Ford-Fulkerson algorithm-- I'm not going to bother writing this out, because I'm going to have to prove the max-flow min-cut theorem next time.
And we're going to talk about the Ford-Fulkerson algorithm and issues with it next time.
But I'm going to show you how the Ford-Fulkerson algorithm works on this particular example.
And it's only going to have one step, all right?
So it's going to converge in one step.
So it's going to be relatively easy.
But the bottom line is the the Ford-Fulkerson algorithm is going to look for augmenting paths in Gf.
Augmenting paths are defined in the residual network.
What is an augmenting path, you ask?
Well, an augmenting path is simply a path from s in Gf to t in Gf, OK?
That's it.
That's all there is to it.
If you can find a path, you could use depth-first search, you could use breadth-first search, you could use whatever you wanted.
You find a path from s to t, OK?
If you find such a path, if this path exists, it means that the flow is not maximum, OK?
If no path exists, the flow is maximum, and you're done.
If such a path exists, you will be able to increase the flow, in this case, because we have integral quantities, by at least one, OK?
And you will be able to increase the flow by one.
And not only that, the augmenting path is going to tell you exactly what to do with respect to what edges to change, sometimes subtract the flow from, sometimes increase, right?
So the augmenting path is going to take care of it.
But you've flipped it, so everything is positive.
You know, life is great.
Positive numbers.
Just look at s to t and give me any path from s to t here in the residual network.
So let me just call them a, b, d, and c. And we don't care about partitions or cuts at this point.
That's required for proofs.
But give me a path from s to t Yeah, go ahead
S to a, a to b-- All right a little bit more slowly.
s a, OK. And then?
a to b
a to b.
Beautiful
b to c
b to c
c to t
And c to t. Wonderful.
And I have a capacity, a residual capacity, of 2 here.
I have a residual capacity of 1 here, 1 here, and 1 here.
And so the value of this augmenting path is 1, right?
It's not 2, because the minimum value is what I can push through the network.
And I just need to take the min of all of the residual capacities corresponding to the edges that I traversed, that correspond to this particular path.
And that min value of 2, 1, 1, 1, is 1, right?
So what this means is I've discovered an augmenting path of residual capacity, 1.
Now I can go back to this thing over here.
And these edges are going to point me, in some cases, to complimentary edges, in some cases to the direct edges that are going to have to have their flows changed, either increased or decreased, to increase the flow in the original network.
So you're guaranteed now that f, which caused this residual network to have an augmenting path in it was not maximum.
You're guaranteed that, because you found this path, OK?
So what happens here?
Well, you go back up and you say, remember, I'm going to increment by plus 1, because that's the residual capacity.
So I'm going to go up here, and I have s to a.
You'll need to help me out again.
So I'm going to make this 2, because I needed to add a one to it.
What happens here?
This is the key step
Subtract
Subtract and make this a 0, right?
We're not quite done.
What's next?
I have b to c, right?
Right?
You're still on the hook
[LAUGHS]
b to c becomes?
3
3.
Right.
And then, lastly, this becomes 2.
OK?
And at this point, if you create a residual network for this new flow, you will not be able to find a path in that residual network from s to t. And you know you're done, right?
So all of this works.
We haven't quite shown that it works, because we haven't done enough of the proofs.
But you have a sense as to why this works.
You could probably code this up.
It would all work.
A couple of issues in applications.
See you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning everyone, let's get started.
So part two of our two lecture sequence on network flow.
So all the pain involved in the concepts and notation from Tuesday turns into algorithmic fun today.
But we will do a little bit of a review just to make sure we're all on the same page with respect to all of these notions that we talked about on flow networks.
So we call that a flow network is simply a directed graph, GVE, and each edge is going to have two numbers associated with it.
If you see something like 1:3, this means that this is the flow, and that's the capacity.
We have a basic constraint dissociated with the flow not exceeding the capacity on any given edge.
And we also have the laws of conservation that say that other than the sink in the source, which are the two distinguished vertices in G V E, you're going to have a situation where for any intermediate vertex, all of the flow entering the vertex has to leave the vertex.
And with those two constraints you need to find the max flow in your flow network.
The flow value corresponding to this potential max flow, or some flow, is F equaling f of s of V. And so s is the source.
V corresponds to all of the vertices.
And you're looking at pushing flow from the source vertex to any of the vertices, and thus the V includes t, as well.
And that this is essentially a flow value, and we showed-- this is implicit summation notation, we showed, using a little bit of algebra, that this equals f of V t. And that's going to hold because of flow conservation primarily.
So that was not particularly surprising.
Something that was a little more surprising, or is a little more surprising, is that you can have an arbitrary cut in the flow network, and a cut corresponds to any partition.
S, T, such that s belongs to S, the source belongs to S, and the sink belongs to T, and that's essentially a cut.
And so obviously there could be exponentially many cuts in a given flow network.
But the amazing thing is that you can show a lemma that says that, regardless of what the actual flow values are in any of these edges-- across any cut you're going to see this flow.
And this is true for any cut S, T. The corollary to this lemma is that the flow is going to be less than or equal to the capacity of any cut.
And that simply comes from the edge constraints.
You know that the flow value corresponding to any particular edge has to be less than the capacity of that edge.
So you obey that constraint and you just look at the edges that go from S to T, sum up those capacities, and your maximum flow or any flow is bounded, [INAUDIBLE], by that quantity.
And that's our c S T. So you now have a relationship between the minimum capacity cut and the max flow.
We didn't actually exploit that in the sense of the algorithm itself, the operation of the algorithm.
But we're going to actually exploit the notion of a cut when we prove this algorithm that I got to late on Tuesday's lecture that corresponds to the Ford-Fulkerson algorithm.
I showed you the execution of it.
In order to prove that it's correct we're going to need this notion of that bound on the flow value given any cut in the network.
So the max flow related to min-cut.
But before we get to that.
Just to catch you up on the algorithm that we actually got to, the Ford-Fulkerson algorithm, that was based on the notion of a residual graph.
This is now a new graph, G f, which is V, E f, so it depends on the flow.
You computed based on G, based on the given flow f. And this graph has edges that have strictly positive residual capacities which are defined as the C f u, v equals C u,v minus f u, v. And this needs to be strictly greater than 0 in order for the edge to exist in G of f. So if this is zero then you don't have an edge between u and v in G of f. If it's one or greater assuming integral flow values, you will.
And in the residual graph-- two more quick definitions.
One of which you saw last time, and another that I just verbalized, is the notion of an augmenting path.
And this is any path from s to t in G f. So if there's a path from S to T in G f you do not have a maximum flow.
There is an augmenting path.
You're going to be able to increase the flow corresponding to the f value, that gave you your residual network, G S of f. And how much can you increase this flow by?
How much can you increase f by?
That is termed the residual capacity.
And the residual capacity of an augmenting path is simply C f of P, that's what we're going to call it, C f of P, equals the minimum value of the residual capacities along the path.
So this would be C f u,v, and that's pretty much it.
And some examples should make this clearer.
If we look at what the actual algorithm is, and go through a simpler example than we did last time, but it'll give you a sense of how augmentations are going to change the flow.
How augmenting paths appear and eventually disappear as the flow increases and reaches the maximum flow.
So this simply corresponds to, as I said, walking through the augmenting path, looking at each of the residual capacities, and picking the Min value.
So the Ford-Fulkerson algorithm, which is the first algorithm for max flow continues to be used, sets the flow on each of the edges to be zero.
So you just set everything to zero.
Clearly satisfies flow conservation, satisfies all the edge capacities.
And you have literally three lines of code here, while an augmenting path in G f exists.
So there's a lot going on here, you have to compute G of f, I mean it's three lines of code, but the sub routines calls, which are fairly complicated, you have to compute g of f, given f, and initially it's all zeros.
And then eventually it's going to change, or immediately after you discover an augmenting path.
And now you have discovered an augmenting path.
So you have to now do either breadth-first search or depth-first search, in order to discover whether this is a path from s to t because that's the definition of an augmenting path.
And then the last line of code says, augment f by C f of p. And again this is fairly involved.
You have to take the path in G of f. You're going to increase the values of the edges that correspond to G of f translating those edges back to G. And there's possibly an inversion in direction that is associated with that translation.
And so if you take a really straightforward example.
Let's say if you have G is s, and this one vertex here, a, and then you have t. And right now I have 1:2 and 1:4.
I now am going to compute G of f. So G of f is going to be-- I'm just going to write down s, a, and t because the vertices are all the same.
The edges are obviously different.
Am I going to have an edge from s to a?
Yes.
And what number should I put on that edge?
1, and that's because it's 2 minus 1, not because it's just plain 1.
And this edge here is going to have a residual capacity of 1, and that just comes directly from that 1.
And what that means is, and this is important, what that means is that, in effect, this residual capacity says in this direction you still have a capacity of 1.
Which means that in this direction you could reduce by one.
OK, that's the important thing to remember.
And over here I'm going to have-- this way I'm going to have 3 and coming back I'm going to have 1.
All right, so straightforward example but I think evocative in the sense that it applies the concepts that we've seen so far other than cuts.
And we'll get to that.
And what happens here, is there a path from s to t?
Absolutely.
And what is the residual capacity of the path from s to t?
It's one because you've got a 1 here, and you got a 3 here, you got to take the Min, and it's 1.
Right, I guess that's the number of choice today, one.
But once you do that and you find that, you say, OK, what I'm going to do is-- and this is going to be pretty straightforward-- I'm going to look at these edges, and I know that I have c of t being a one corresponding to this value.
And I'm going to go add one to those corresponding edges because these are the edges that actually shows in terms of the augmenting path.
So the augmenting path is that.
And so this turns into s to a to t, and that becomes 2:2.
This becomes 2:4 and that's it.
So I now have G with the new f. So I can now think about what is G of f1 correspond to.
The G is separate from f, so in some sense the G stayed the same, but the f changed, as you saw here.
And now G f1 looks like s, a, t. What do I do between s and a?
Well, I only have something from a to s. I don't have anything from s to a because the residual capacity in this direction, thanks to the fact that the flow has saturated the edge capacity, is actually zero.
But I do have an edge this way.
And what is the value for that edge?
It's 2.
And over here, what are the numbers for the top and the bottom?
2 and 2.
And now I look at G f1, and I've gone through one iteration of this pseudo code, and I now try to find an augmenting path in G f1.
And is there a path from s to t?
No, which means I'm done.
This is the max flow.
You believe me?
For this example, you believe me.
You believe me, like, for every example?
No, absolutely not because you haven't done a proof!
We haven't done a proof!
That's why you shouldn't believe me.
So we got to do a proof.
So how is it that we know, given all of this, it's not because I told you so, that when we've converged.
When we get out of this while loop, that we have a max flow.
Looks pretty good.
Worked for that example and it's going to work for every example, but that's not a proof.
So now we have to sort of take in all of the notions that we've talked about here, including primarily the notion of cuts.
And the proof, the max flow, min-cut theorem, which is going to show this key result that we require, which is that when we terminate in the Ford-Fulkerson algorithm, we're going to have a max flow.
And that's the reason why it's a maximum flow algorithm.
If you don't have that proof, you don't have a max flow algorithm.
So hopefully all of that is clear.
Pipe up if you have questions.
And let's write out the max flow min-cut theorem, which I mentioned of it last time but never really got to even stating, but today we're going to state it and prove it.
So this is an interesting theorem.
I mean it's a legendary theorem.
And it's not your usual theorem in the sense that it makes a simple statement.
It actually makes three statements.
It says the following are equivalent, and it's got three things which are the following.
F equals c S T for some cut, S,T.
This says that some cut is saturated.
Right, so that's just the statement.
The second statement, which is equivalent to the first one, says that f is a maximum flow.
So f being a maximum flow means there's some cut that's saturated.
If there's some cut that's saturated for a given flow, you have a max flow.
And then the last one, which is important for our Ford-Fulkerson algorithm, is that f admits no augmenting paths.
And so if I want to show that the Ford-Fulkerson algorithm terminates with a max flow, tell me, based on implications-- like i implies j, for numbers i and j, what it is that I need to prove.
We're going to prove a bunch of other things along the way but crucially what is the i implies j that I want if I want to claim that the Ford-Fulkerson algorithm terminates with the max flow.
Yeah, you over there
3 implies 2
3 implies 2.
That's right, 3 implies 2, exactly right.
3 implies 2.
So we need to do 3 implies 2.
Now of course, the theorem says that, 1 implies 2, 2 implies 3, 3 implies 1, et cetera.
There's a lot of implications here.
What we are going to do is we're going to show that 1 implies 2, 2 implies 3, and 3 implies 1.
And that pretty much takes care of everything.
And it turns out the reason we do this is simply because it makes for the simplest proofs.
1 implies 2 and 2 implies 3 are one- liners, and that 3 implies 1 is a little more interesting and involved, but it's a little bit easier than directly doing 3 implies 2.
So that's the way we're going to do this.
You could certainly play around and do other things.
So any questions so far?
OK, so let's go ahead and do this.
All right, we should be able to knock these two, 1 implies 2 and 2 implies 3, in just about a minute each.
So I want to show 1 implies 2, and essentially what I want to say here is, if I've saturated a particular cuts capacity then I have a max flow.
And really, I mean this comes from the definitions, since I'm going to have f less than or equal to the c S, T, and this is simply because of edge capacity constraints.
This is just edge capacities for any cut S T, the assumption that f equals c of S T implies f is a maximum flow because f can be increased.
And that's basically it.
Right, so this is pretty easy.
Next one is easy as well.
2 implies 3.
If the rare and augmenting path-- so I'm going to do this by contradiction-- if the rare and augmenting path, the flow value can be increased.
Because remember the augmenting path corresponds to a path with strictly positive residual capacities.
The only reason there's an edge in there is because you have a greater than zero residual capacity.
So that means that the flow value could be increased by some small amount corresponding to C f u v, and each of these quantities that are in here-- I'm sorry, corresponding to min C f u v, that's the residual capacity.
But you know that each of these capacities, C F u v, are strictly greater than zero.
So the min clearly is strictly greater than zero.
It might be some tiny quantity but it's greater than zero.
Which means that the flow value could be increased contradicting the maximality of f. So a little proof by contradiction gives you 2 implies 3.
So, so far, so good.
This is primarily based on the definitions of augmenting paths, residual capacities, et cetera.
Now the fun starts.
Now we want to show really, I mean this is truly what the max flow min cut is because I really didn't have cuts up there.
So what's min-cut about that, right?
So I had a question at the end of Tuesday's lecture on how come the algorithm didn't use cuts.
And, well, the algorithm doesn't use cuts, but showing that the algorithm converges to the max flow uses the notion of cuts.
So what we want to do is do 3 implies 1, and that's going to take care of what we want.
And we're going to say, well, suppose f admits no augmenting paths.
So what does that mean?
Well, that means that you can't reach t from s. If you reach t from s, there's an augmenting path, so you can't make that move or make that path.
So let's go ahead and, sort of, look at this a little more carefully and figure out what's causing this lack of connectivity.
There's a lack of connectivity between s and t. You can't actually get there.
So there's like a chasm, there's a gap.
And so what's causing that?
And what we're going to do is figure out what the cut of the partition is that could be the reason why you have this lack of connectivity.
So we're going to define S which is exactly going to be our S from our favorite s t cut definition, but bear with me for just a second.
Because I will tell you what S is.
There exists a path in G f from s to u.
So what I'm doing is I'm collecting the reachable vertices from S. And I know that I'm not going to have t, small t, in this set because f admits no augmenting paths so I can't possibly reach t from s. But I can collect all of the vertices that that can be reached from s in this given G of f and put them all into S. And then my T definition is simply V minus S. And the key observation which I already have made but important to emphasize is that small t belongs to T, and obviously s belongs to S, therefore S T is a cut.
It satisfies the definition of a cut.
So we've set ourselves up.
And then we have a key observation to make that I'm going to try and extract out or you.
Let me move over here.
All right, so what I want to do is I want to draw this out, and I want to look at this as this is S, that is T. I know that S is over here, are s is over here, t is over here.
I'm going to pick some vertex v that's over here, some vertex u that's over here.
I know that there's a path, and I'm writing this is a dotted path because this is a path in G of f. We're going to actually move between a G of f and G in this proof.
So that's the only hard part of this proof.
The movement between g and G of f. The edges are different between g and G of f. Got to keep that in your head.
Because we're going to make arguments about that.
I know that all of the vertices that are on the left hand side are reachable from s, so I just picked an arbitrary vertex u, and I know that there's a path in G of f because that's the way I defined it from s to u. I also know that there is no path in G of f from u to v because if there was a path v would have been on this side, correct?
So there's clearly no path in g of f from u to v. That's why v is over here.
But let's say that I'm picking something where there's an edge in G, the original flow network from u to v. So the edge that I've drawn here, the dark edge, is an edge between u and v in G. And that edge had a certain capacity.
It existed.
The reason it existed is because it had a nonzero capacity, correct?
Because we originally defined our flow network to be something where-- if you only put edges in there, in the original flow network if the capacities were greater than zero.
So that edge in there, in G had some capacity c u v which was greater than zero.
But this edge does not exist in G of f, correct?
So what can I say?
Go ahead
[INAUDIBLE]
Exactly.
In the original graph, we have saturated this edge-- that was my best throw of the term.
I like that.
And you guys didn't notice, but maybe it'll be on video.
Actually, it won't be on video unfortunately.
But so I got this edge here in the original graph and I didn't get this dotted edge here because the residual capacity was zero.
The original capacity was nonzero.
The residual capacity is zero.
The only reason that happens is because C f u v is zero since if C f u v greater than zero then v would belong to S, not v belonging to T as assumed.
So that's essentially what we've determined here.
So this means that f of u v equals c of u v simply because c f u v equals c u v minus f u v which is 0.
All right, so that's the statement I can make.
Now, I did not have any constraint on u and v. I just said u belongs to s and v belongs to t. And for each of those things, I know that I can't have edges in the residual network between u and v which means that any edge that exists-- maybe there's no edge in the original network, but if there was an edge in that original network it's saturated according to this argument.
So every edge from S to T in the original network is saturated, and that essentially means that I've gotten to the capacity of my cut.
That's essentially what that means.
If I've saturated every edge, I've reached that capacity.
OK, so that's it.
All you do once you recognize that you had an arbitrary choice of u and v, you just say summing over all u belonging to s and v belong to t yields f of S T equals c of S T. All right, I'm on a roll not just with Frisbees.
I finished the proof with a finger to spare.
So f of S T equals c of S T. All right, so that's exactly what we want.
We are saying f of S T is obviously a cardinality of f so I've shown this thing over here.
So that's why the Ford-Fulkerson algorithm works.
It's because of this analysis that the Ford-Fulkerson algorithm works.
So are we done?
What are we missing in algorithm design, our algorithm analysis?
Not you, yet
[INAUDIBLE]
Sorry?
A runtime?
Runtime, good runtime.
I know you have too many Frisbees.
You probably do, too.
But so we haven't done-- we've done correctness, we've done conversions.
We haven't done a complexity analysis.
And so there's some bad news here.
And erasing a pivotal moment in 6 over 6.
Why is this a pivotal moment in 6 over 6?
It's on the Frisbee, exactly.
This picture-- maybe not exactly that picture.
It's not quite that picture but it's the same graph.
It's a pivotal moment because this picture was famous because it was in a textbook but now it's iconic because it's on a Frisbee.
We immortalized this picture.
So this is an example of really a failure of the Ford-Fulkerson algorithm.
So it's kind of a bad example on this Frisbee but it's going to lead us to better algorithms.
So what I have here is a weird flow network that has these strange capacities.
I mean it's a fairly straightforward flow network.
But at the as of the moment it obviously satisfies a flow conservation because all of the flows are 0 and therefore edge capacities.
But we're going to now take a pathological execution of the Ford-Fulkerson algorithm.
Which, by the way, did not specify how you are going to select the augmenting path.
It just said, find an augmenting path.
And in passing, I said, overuse depth-first search, we'll use breadth-first search, and kind of hand wave my way through that.
Now it turns out you can't hand wave when you want to run algorithms, right.
You eventually have to code them up, you got to pick something.
So people did.
People picked different ways of selecting augmenting paths in the Ford-Fulkerson framework.
And they discovered that some of them would work beautifully and some of them would fail miserably on networks that were as small as this, and they'd just take forever.
And this is before complexity analysis, kind of maybe before asymptotic complexity really came into vogue.
So they just had empirical analysis to tell them that some techniques for discovering augmenting paths work better than other techniques on an empirical basis.
So what could go wrong with Ford-Fulkerson on this example?
What selection strategy corresponding to the augmenting path could cause Ford-Fulkerson to have a huge number of iterations?
Go ahead
So each time you pick an augmenting path you pick one that goes to the center
Exactly, and so let's walk through a couple here.
So the first one, let's just say this was a and b, to make it easy.
So what is the first path that we picked?
What is it--?
Go for it.
You're not off the hook yet
You pick s to a to b to t
Yeah, and then when you pick s to a to b to t, I'm not going to draw G of f for this, it's pretty clear that G of f is going to have-- I'm sorry, you end up picking s to a to b to t corresponding to the G of f that you've created.
Right, so the G of f might have something like s to a-- there's only going to be an edge this way that's going to have 10 raised to 9.
And then this way it's going to have 1, and that's going to b, and over here to t it's going to have 10 raised to 9.
It's also going to have 10 raised to 9 this way, and it's going to have 10 raised to 9 this way.
And the reason the gentleman was laughing is because we ended up picking this path.
S in G of f-- this is G of f just to be clear-- you pick s a b t. And so what you end up doing is you end up making this 1, making this 1, and coming over here making this 1.
Wow!
OK, you did increase the flow.
Very good, made progress.
Now what happens?
When you do this, certainly you'll get a different G of f. And the G of f is going to turn into-- well, this gets a little more interesting here.
This is 10 raised to 9 minus 1.
This is 10 raised to 9-- no, that doesn't change.
This is still 10 raised to 9.
This is 10 raised to 9 minus 1.
And that this edge here also changes because basically what ends up happening is you end up going this way with 1.
Did I get that right?
Yeah?
OK, good.
So now what's the bad path?
s b a t. So s b a t would now say, oh, what I'm going to do is, I'm going to go ahead and make this 1.
What do I do with that?
Make it 0
Make it 0, exactly.
Make it 0, and then this 1 becomes a 1.
Shall we keep going?
No.
So 2 billion iterations for a graph with four vertices.
Now that's performance for you.
So this particular, potentially a depth-first Ford-Fulkerson on this example, depending on the ordering of vertices it's certainly quite possible that you would get to this particular order in the depth-first search on G of f. The computers is dumb, it does what you tell it to do, it's doing depth-first search.
It's producing these paths, and you end up with 2 billion iterations for this.
And how many iterations do you really need if you did it right?
Two.
So that's a billion factor slowdown.
So this is a pathological example, a simple pathological example, to just show you what the problem is.
But you can imagine that if you use depth-first search you might be a factor of five slower on average than if you use some of the technique.
And a factor of 5 is nothing to be scoffed at, especially if you're running for minutes.
And, you know, back in the day computers were horribly slow.
So how is this problem fixed?
Well, any number of ways.
But the first real way that took into account asymptotitc complexity, did analysis, and did all of that was due to Edmonds and Karp, which is a few years after Ford-Fulkerson.
In fact, several years after Ford-Fulkerson.
And their contribution was not as much a new algorithm, though it is called Edmonds-Karp algorithm.
It's really a small change to Ford-Fulkerson.
You might want to think about it as the Edmonds-Karp analysis, which was kind of important, and we're not going to do that here.
But you'll see it in section on Friday.
But the Edmonds-Karp algorithm slash analysis is that, if you use a breadth-first augmenting path, it's obviously just as easy to discover complexity-wise as the depth-first path.
Then you could get a polynomial bound that did not depend on the capacities for the complexity and therefore iteration count of Ford-Fulkerson.
So that was their contribution.
And so the breadth-first path is the shortest path in G f from s to t if you count where each edge has weight 1.
So in that case you would get your two iterations.
You'd get the shortest path of weight 2.
Whenever I say weight, I mean just counting the number of edges.
And these algorithms they actually recognized that breadth-first implementations of Ford-Fulkerson were the ones that ran faster than all the other ones.
And then they said, wait, what's going on here?
This looks pretty good.
And so they went off and did a fairly sophisticated analysis that showed that you had, and as I said you'll see that in section, that you only need order VE augmentations in the worst case.
So this is what we know and love with respect to algorithmic complexity.
Order VE augmentations, in the worst case, provided you used breadth-first augmentation.
And so what is the overall complexity of the algorithm, let's call it the Edmonds-Karp algorithm, with this breadth-first search it would be-- order?
VE square, you could think about it as, let's assume that e is greater than V, and so we just say order V squared.
So that was the Edmonds and Karp's contribution.
This mean that an amazing amount of work over the past 30, 40 years on improving that bound.
[?
Denic ?]
independently arrived at a similar analysis in bound, along with Emonds and Karp.
They were the first to give polynomial bounds on max flow.
And there was an algorithm that came out of MIT partly, or at least MIT alums, where you improve that.
And so until about 2011-- and there's a lot of different algorithms depending on whether you're taking into account the value of the capacity, and you want to think about this is as some max value, and if you want to include that in the complexity then things change a bit.
If you can limit that.
But assuming that those capacities are essentially unbounded, then the fastest algorithm in 2011 was King, Raul, and Tarjan.
And that runs in order VE log of E divided by V log V V. Obviously, we need something for the log here.
So that's kind of, pretty good here, right, because this is pretty small.
If you really think about it, if E's large for dense graphs this is going to be a small constant.
So that's pretty good.
So that was the fastest until 2011.
Very recently, Orlin, who's at MIT, came up with an order v e algorithm and there are many variants here that use fast matrix multiply.
And then just this past September, Aleksander Madry joined our department, and he has a variant algorithm that is better in some sense, not in the pure sense of V and E but there's a couple of constraints associated with it that are very mild.
But it's in that sense better than order VE and I won't get to that.
In the context of 6 over 6 we are obviously going to stick with Edmonds-Karp and Ford-Fulkerson in terms of what you're responsible for, but this as been a rich area of research for decades.
All right, yeah.
[INAUDIBLE] Oh, simply because the breadth-first augmentation takes order e. I'm just saying it takes order E time.
So this was the order VE augmentations, in the worst case, each augmentation takes order E time, because breadth-first search takes order E time assuming that E is greater than V and I just added the E square factor.
Thanks for asking.
I'm sure a lot of other people had the same question.
Yes, I just sort of skipped over that a little too quickly.
That's why you get the order VE square.
And so that's a significant reduction if you assume that constant factors don't explode going from Edmonds-Karp to King, Raul, and Tarjan, and certainly Orlin, So now we got algorithms, we've proven them correct, we've done some analysis.
At tomorrow, in section, you're going to see, at the analysis that gives you the order VE.
Which is a half hours worth of pain, should I say.
Uh, no, a half hours worth of excitement!
That corresponds to the order VE, that bound that we have.
All right, so it's not that difficult.
It's about the level of this thing over here, and maybe a little bit more involved.
So, yeah?
[INAUDIBLE]
Breadth-first search takes order v plus e time.
On a graph, breadth-first search is order v plus e. And I'm getting a little lazy here, and I'm paying for it, by putting in the v e square.
But I should really have been-- if I wanted to be a stickler, I would have put in v plus e, thank you.
So where was I?
So we got an algorithm now.
You know, we've got a bunch of algorithms.
Network flow is this amazing hammer that is being used in a wide variety of things.
And it's been used for matching, bipartite matching, all sorts of things.
My favorite example is something that is related to sports, I should say, which is a baseball elimination.
So what is this application about?
You see charts that correspond to how your team is doing.
Standings, right?
Go to ESPN and click on standings for baseball, for example.
It doesn't-- now you see pre season standings but let's assume it's August or something-- and you click on standings.
And you see a chart that corresponds to games played, games won, games lost, games remaining, and so on and so forth.
And back in the day when they were just winners corresponding to each division you had to win your division to make the playoffs.
So you had to be the best.
Now if you ended up being tied, you ended up having to play an elimination game.
But the whole purpose of this analysis is deciding whether you still have a chance to make the playoffs or not.
So the goal here is you want an algorithm that is going to look at the standings and decide if your team is alive or not.
Is there a chance of God is great, whatever, everything goes your way, angels in the outfield, right?
Are you going to make the playoffs?
And so you can be very optimistic.
And you might think that this is a straightforward, I'm going to add up a couple numbers and figure this out.
It turns out it's not quite that simple though sportswriters would love to think that.
That it's as simple as adding up a bunch of numbers.
And obviously, there's network flow lurking in here somewhere.
Otherwise, I wouldn't be talking about it here.
So this is not quite historical in terms of the standings.
I tweaked the numbers a little bit in order to make this example a little more interesting.
But roughly speaking, August 30th, 1996, this was the standings corresponding to the American League East.
And now this is 1996 so the teams are a little bit different from what you have.
So New York, unfortunately, was leading the division even back then.
Still had Baltimore.
Then there was Boston.
And then was Toronto.
And then this is back in 1996, and Detroit was part of the [INAUDIBLE] back then.
This was before realignment.
So I just called these one, two, three, four, and five.
Those are the positions.
All right, so Boston was in third place.
And so, I'm going to say these are w i, which are the wins, so far.
l i, which are the losses.
And r i, which are remaining games to play.
And This is usually what you see.
I mean this is the simple version of the standings and I'm just going to give you most of these, I should say.
And I'm not going to write down what Detroit is, simply because I'm going to vary that number.
Because it gets interesting as that number varies.
And each of these teams had 28 games to play.
OK so that's usually what you see when you see a snapshot.
OK now I'm going to also add rij, which are the games that these teams play against each other.
And the reason is simple.
This linear correlation between win loss column, specifically corresponding to games that Boston plays with New York.
No ties in baseball, so Boston wins, New York loses and vice versa.
So I want those numbers as well and these are things that you typically don't see but they're going to be important, because they do determine whether your team is alive or eliminated.
So 5 The reason I have dashes here is because the team doesn't play itself in the diagonal
Is that the number of games?
Yes that is absolutely the number of games that they're going to play against each other.
All right so I've got small number.
I mean 5 plus 7, 12.
Plus 4.
What is that?
16.
16 plus 3 is 19.
There's 28 games left to play, there's obviously games that are being played against other teams outside the division.
And so this is just rij, OK?
So that's the story here.
And I should say that when I talk about this, what are these corresponding to?
Well this corresponds to NY.
This column corresponds to Baltimore this corresponds to Boston, same order, this corresponds to Toronto.
This column corresponds to Detroit.
So otherwise it's under specified.
Any questions about the table here?
If you want to know, the Bo Sox did not make the playoffs that year.
So all that most sportswriters do, most people do, is say Team i is eliminated.
If wi plus ri is strictly less than wj for sum j.
So you're just going to say, OK you know what?
If I've won all my games, I'm still not going to make it to this team that's already won 75 games, right?
And so clearly I'm done.
That team has already won more games than I could possibly win.
So that's an easy one.
And so this is strictly less, because we're just talking about elimination, total elimination.
You get your summer vacation, or your fall vacation.
So let's just say that you had Detroit, and you're talking about w 5 years.
You're talking about this number here, is 46. l5 is 88.
So they're having a bad season here.
And you're going to say 46 plus 28 equals 74, which is less than 75.
Therefore in this scenario, Detroit is eliminated.
All makes sense?
So this is sufficient, but I claim it's not necessary.
So you might actually have false hope if you are a Mirror sportswriter, who only knows to add up these numbers and compare it with others.
But if you were a 6046 student, and you went through these wonderful lectures on network flow, you know much more.
And we're not quite there yet in terms of requiring the power of network flow.
But I ask, what if w5 were 47?
Can you make an argument which clearly doesn't work here, with this naive argument, you have 75 equals 75.
So you still have hope.
But can you make an argument that if w5 were 47, given the whole table-- that's the hint, the whole table, that Detroit was actually eliminated.
Worth the Frisbee for sure, right?
Over there.
I saw you first
It looks like the only way that Detroit can get more wins than New York is if they win all their games and New York loses all their games?
Yes
and if New York loses all their games, they're going to lose 7 games to Boston And Boston will have 76 wins
Exactly right.
So that's a great argument.
All right, could you stand up?
All right.
You catch well too.
So that's basically the essence of the issue.
There's stuff going on which corresponds to teams playing each other that can affect the outcome.
OK so let me write out what the gentleman just said.
Still eliminated, boohoo, because w5 plus r5 equals 75 now.
47 plus 28 is 75.
But either New York or Baltimore-- I mean there's many ways you could show this.
But we'll just take-- this is not exactly what the gentleman said.
But this is what I had here, so let me just write that out.
Either New York or Baltimore will win 76 games.
Since they play each other 5 times.
OK?
So now I'm going to do some more sophisticated analysis and I got what I wanted.
But let's say that w5 was 48.
So this was 76 and it kind of worked out for both of the examples.
The one with Boston and the 7 games, that I got as an answer.
And this one that I just wrote down.
But what if w5 equals 48?
Is Detroit eliminated?
How many people think Detroit is eliminated?
All right, a bunch of you.
You want to explain why?
So New York, Baltimore, and Boston are guaranteed to get a total of at least 14 more wins.
Because of their total number of wins that they're allowed to have without any of them getting to 77 is only going to be 13.
So it will be 1 plus 5 so that's too many.
But they're not actually eliminated, because this is from the Wild Card round anyway
I didn't like the last part of your answer.
Did, but the other question is, did people understand the first part of his answer?
That was complicated.
I mean it was the right analysis.
Great job.
So, we want a systematic way so we can run an algorithm so we need to bother with this type of analysis.
And you're exactly right.
It turns out that I skipped this little detail, because it was inconvenient, which is that in '95, a wild card was introduced.
So we could think about this as elimination from being the division winner.
So that's an aside.
Thanks for pointing out, it's important to know sports history.
Really important.
Almost as important as 046.
But basically it gets more and more complicated as you get closer and closer to these edge cases.
And so you can do an analysis, and I'll actually describe roughly what your analysis was.
What's your name?
Alexander
Alexander's analysis was.
But I don't want to do that yet.
I'll do that at the end if you have time.
I want to give you a general purpose technique that obviously is going to use max flow to solve this problem.
All right?
So we're going to set this up.
And so that's actually one of the nice things about maximum flow.
How you can translate.
You can sort of create these networks from tables or what have you, and add capacities to edges, find max flows, and solve these problems.
So this is a good example of that.
So let's go ahead and do that I am going to draw a flow network based on this table.
It's going to have a source and a sync that are basically these dummy nodes that are essentially going to source about infinite flow.
But the capacities of the edges are going to come from that table.
And the edges themselves are straightforward in the sense that the graph is going to look the same based on the number of teams, and the particular team that you're analyzing.
So this graph is trying to determine-- this is a flow network to determine if team 5, Detroit, is eliminated.
This is just the flow network.
I'm going to have a bunch of edges.
I'm going to just draw this once and we'll be moving things around on this.
2 to 3, 1 to 4, 2 to 4, and lastly 3 to 4.
And the reason you don't see 5 here is because this is an analysis for team 5.
So the other 4 teams show up here.
They have edges going to t. s is over here, and it's got a bunch of edges going to all of these nodes.
What are these pairs?
As you can imagine, these pairs correspond to the games that each of these teams that are inside the circle play against each other.
So 3 plays 4 a certain number of times.
According to that table it's 4 times.
So I'm going to put a 4 in here.
This is 4 as well.
4, 5, 7, 2.
OK?
And the edges in between here are going to have capacities of infinity and how are these edges structured?
So far I've just explained how the left-hand side works.
These edges have a capacity of infinity.
1 2 goes to 1, 1 2 goes to 2.
1 3 goes to 1, 1 3 goes to 3.
And that's pretty much it.
So that's where these edges are.
That's how these edges are introduced.
And all of these edges have a capacity of infinity.
2 to 4, 3 to 4, and that.
So far, it's pretty straightforward.
There's one last thing that we need to do, which is add capacities to these edges.
This is actually crucial.
It turns out that we have to add capacities such that this max flow is going to represent elimination.
So these capacities have to be chosen.
And in particular the equation for this is simply w5 plus r5 minus w1.
In this case if it's 48, I'm asking the question for 48 for w5 plus 28 minus 75.
So that's 1.
And then this thing here would be w5.
I'll just write that out.
w5 plus r5 minus w2, hopefully you can read that, and that's a 5 and then similarly this is a 7, and that's 13.
And you might ask, what does this represent?
Well it kind of represents what you think it represents if you just look at that equation.
It says how much can I push through here before I get greater than w1.
So what is the capacity?
What's the difference here?
if I have a 1 here, that means that I still see w5 plus r5, I'm 1 greater than w1.
So in particular, if I want to write this out, the capacities are the number of games team i can win, and not have more wins than team 5.
So this is a statement about team 1 when I see that over there.
Our team 2 for the other edge.
And I'm just trying to figure out how much can I allow, with respect to this other team, to determine elimination for the particular team that I'm looking at, team 5 or not.
So in particular, let's say that team i wins one particular game.
So it goes up to it goes up to 76.
And if it goes up to 76, it does not necessarily have more wins then team 5, because team 5 can go up to 76 as well.
That's pretty much it, and the same thing for team 2, team 3, and team 4.
So the intuition here is the following.
So what we're going to do is compute the max flow and look for a certain property on that network.
And that's going to tell us if team 5 is eliminated or not for the particular choice of wi of 48, because that's how I constructed this particular example.
So the intuition is, assume team 5 wins all remaining games.
That makes perfect sense.
This is the best case scenario.
You're trying to figure out, in the best case scenario, are you eliminated or not?
Is team 5 eliminated or not?
And what you want to do is divvy up the remaining games.
And that corresponds to sending flow through these edges.
So flow through these edges corresponds to assigning wins to teams 1, 2, 3, and 4.
That's the key.
Divvy up the remaining games, so all teams have less than or equal to w5 plus r5 wins.
So what does it mean if all teams have less than or equal to w5 plus r5 wins?
If you can divvied up such that that's the case, what does that mean for team 5?
Team five is still in the game.
Team 5 is not eliminated.
So if you can do this, then team 5 is not eliminated If you can't team 5 is eliminated.
Because some team is going to get 77 wins.
And so that analysis that Alexander carried out wasn't a flow analysis.
But somehow, that's got to be in here.
That type of analysis has got to be in here to show that some team is going to get to 77 wins.
Because you know, well thanks to Alexander, that Detroit is going to be eliminated here.
But we want to show that in terms of the max flow on this network.
So there is an associated theorem that I'll write.
We've got enough time here.
We just have to find the max flow for this.
And that should take just a minute.
But I want to set it up so that's pretty much the end.
Finding the max flow, what that means.
But the theorem is, which is essentially a more precise restatement of this observation, is that team 5-- obviously you can do this for any team-- is eliminated.
If and only if the max flow does not saturate all edges leaving the source.
I.e.
in this case, the max flow is strictly less than 26, which corresponds to the sum of those things.
5 plus 7 plus, yadda yadda yadda.
And so what does it mean to saturate all of the edges?
It means that all the games have been played.
All the relevant games here that affect team 5 have been played.
So you want to play all the games, because that's the end game, if you will.
But basically what that intuition here is that saturation of those edges corresponds to playing all remaining games.
And the argument is not a proof if you can't play all the remaining games, without exceeding the capacity of i to t edges, team 5 is eliminated.
So you have to saturate, because you have to play all the games.
And when you do that saturation, obviously your flow network has to satisfy its laws corresponding to the capacity that we have in there.
And it's quite possible that this is going to cause a restriction over here that corresponds to requiring that not all the games we play in order for Detroit to survive.
and that's basically the game.
So saturating the network means playing all the games.
And if you get a max flow, to be more concrete, if you can get a max flow here that's less than 26, then you've saturated all the edges and this means that you played all the games.
And you found this team that beats you.
That beats team 5.
So what happens over here?
Let's take a look at this and all you have to do here, is to find the max flow and then we can go back to the statement of the theorem.
So I want to find the min cut corresponding to this, which is going to tell me what the max flow is.
I could go around for focus in on this, and start with 0 and augment, and so on and so forth.
But there's another way which is not necessarily algorithmic, but eyeballs.
Use your eyeballs and find me the min cut here.
Which has the minimum capacity.
The min cut, the minimum capacity is going to tell me what my max flow is.
We know that from the max-flow min-cut theorem.
So I want to cut this network into capital S and capital T and find the max flow.
So any ideas?
It's a little hard.
So I see some people waving their hands.
All right, let me just do that since we're running out of time.
It's kind of cool.
So I'm going to use a different color.
We're going to go like that, yeah.
and then veer over this way, and then jump up like that, and then come over like that.
And this is S. And on this side is T. So I got small s and capital S, small t and cap T. But I got this 4 in cap T. Notice that I got 3 to 4, 2 to 4, 1 to 4 in cap T and I got these other ones in cap S. So all I've done here, forget baseball elimination, is find a min-cut.
And if you look at what the value of this min-cut is, the value of the min-cut is simply the capacities of the edges that go from S to T. So it's simply the S to T edges and summing over the capacities.
And if you take a look, obviously S to T, you've got 4 plus 4 plus 4.
So you've got 4 plus 4 plus 4, corresponding to this one, this one, and that one.
Do I need to add this edge in here?
Over here?
Where does this edge go?
From this to over there?
That goes from T to S. So that's good because that has a capacity of infinity.
That would cause trouble.
And so the other edges are I got 1, 5, and 7.
So I need this one, that one, and that one.
So 4 plus 4 plus 4, plus 1 plus 5 plus 7 equals 25.
And so this implies elimination, because what I've done is found the max flow, which is strictly less than 26.
Which means that I have not saturated all of the edges that come out of s. I have not been able to play all the games.
And so if I'd played all the games, I'd be exceeding some capacity constraint.
And if I push more flow in there, I'd be exceeding some capacity constraint on this side.
And that would imply that Detroit is eliminated, because some team would have 77 wins.
All right, so hopefully you've got the gist of it.
I'll put this in the notes.
Take a long, hard look at it.
That particular values aren't important.
The framework of translation is important.
And you can certainly ask questions of your TAs on Friday.
And I'll stick around here for questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning, everyone.
So we have a singleton lecture today on linear programming, which is general purpose optimization technique that you can use to solve a whole bunch of problems, including ones that we've seen in 6.046 and previously in 006.
And most recently, we looked at max flow.
We wouldn't have had to go through all of that pain we went through to derive a max flow algorithm if you had a linear programming package handy and all you wanted to do was find the optimum solution.
You could have just run the linear program with an appropriate input, of course, that is derived from the flow network.
And you'd get your optimal solution.
And we'll spend a couple of minutes on that as we look at the power of linear programming in today's lecture.
But it's not just max flow.
You could do shortest paths.
You could do multi commodity max flow, which is more complicated than max flow and a variety of other problems.
So that's that good news.
The bad news is that the algorithms for linear programming are a heck of a lot more complicated than max flow.
And you can imagine that that would be the case, because it's a more general purpose and more powerful technique.
The history really is that it was an open problem.
Up until 1979, people did not know if linear programming was polynomial-time solvable until Khachiyan came up with this ellipsoid method, and then there's been progress sense.
But the algorithm we're going to describe today and execute on a example input is a simplex algorithm-- the simplex algorithm-- that runs in worst case exponential time.
But it's a very efficient in practice.
And it's held its ground, even with the advent of more efficient, from a theoretical sense, polynomial-time algorithms, namely the ellipsoid method, which actually is not that efficient in practice and new interior point methods.
So a little bit of context, let's just dive into an example of optimization in the context of politics and see how you could formulate this particular problem as a linear program.
So how does politics work?
You buy elections, right?
So you don't want to spend a lot of money, so you want to minimize the amount of money that is required to buy an election.
And the way you buy an election is, well, campaign, but you advertise.
And it doesn't matter facts that are relevant.
As long as you get to the right demographic with the right message, let's assume that you're going to win the election.
So that's our mathematical abstraction pf campaigning and politics all in 30 seconds.
So how to campaign to win an election.
And as I said, we're going to advertise.
But you do have a little bit of work to do here.
That's why you need your campaign manager.
And this manager is going to estimate votes obtained per dollar spent.
But that dollar is spent advertising in support of a particular case, or a particular issue.
And contradictions are allowed.
As long as you're sending different messages to different demographics, you're all good, right?
You're assuming that people don't watch more than one channel.
So you're a Fox News guy or you're a MSNB guy.
You don't do both.
So now, we get at this estimate.
And it turns into a table.
And so you have your policy, and you've got your demographic.
So you got urban, now think Detroit, suburban, I guess, you could Lexington where you live, and rural-- I really don't have any idea what that means.
But presumably, there's places.
And here's our policy.
You want to build roads-- kind of boring, but some people are interested in roads-- gun control, of very sensitive, farm subsidies-- you know who's interested in that-- and gasoline tax-- kind of more, this hits your pocket, so more or less everybody's interested in that.
So you tell the urban guys you want to build roads, and they don't like you.
So you get a minus 2 there.
So this can go, you advertise, and it hurts you.
You lose votes.
Tell the suburban people-- well, typically, it's a situation where you have these nice cars, and you don't like potholes, so you like roads if you live in suburbia.
Rural people have 4-by-4's.
They don't particularly care.
They don't care as much.
Gun control-- well, you can imagine that urban people like that.
Suburban people are, hm, OK, meh.
And the rural people hate it, right?
You do not want to advertise on gun control in the rural areas.
Farm subsidies-- so like, don't want to deal with that.
What is a farm?
And the rural people love it, right?
And then gasoline tax, well, urban people are commuting.
And, well, they typically don't have a lot of money.
So there you go.
And those are the numbers.
I'm not going to justify every number here.
But you could put whatever numbers you what, I mean.
So let's move on.
This is just a table.
It could have positive numbers.
It could have negative numbers.
And you still want to win this election.
Regardless of how crazy the demographics are, how crazy your electorate is, you want to win the election.
So as long as you have a great campaign manager, who can get you this table, it's all mathematical here on out.
You've just got to figure out how you're going to win a majority.
And you could argue that all you want is to win the election.
We're going to do something slightly different, which is something that's obviously going to guarantee victory.
But you want to win a majority in every demographic.
As the tables may be off by little, you want to be careful.
So the last thing, of course, in order to estimate how much money you need is the population here.
So that's votes obtained per dollar spent.
So you're getting-- it's $10 a vote, it's $5 a vote, et cetera.
And so we need to translate that to votes, because that's the dollars.
And you got your population here corresponding to each of these areas.
And that's what you got here.
Majority, we'll assume you win in the case of ties, just to keep these numbers easy, so that's just divided by 2.
So that's what you got so far.
And you want to win by spending the minimum amount of money.
So that's our optimization problem.
So we can take this, and we can convert it to a set of linear equations.
And that's going to create our linear program-- our first linear program.
And this is our algebraic setup.
And so let-- we need some variables here, so let x1, x2, x3, x4 denote the dollars spent per issue.
So you got those four issues up there.
So let me write that out.
It's important to make sure you know what I'm talking about with respect to a particular issue.
So those are our four issues.
And those are our four variables.
So this linear program has four variables.
You're trying to discover the values of these variables to optimize, minimize your cost function.
The second thing that a linear program has-- and pretty much the only other thing a linear program has-- are constraints.
And these constraints are also linear.
It gets much more complicated if you have quadratic constraints, and we won't go there.
These constraints that I'm going to write correspond to this statement here that says you want to majority in each demographic.
So you can imagine that because you have three demographics, you're going to have three constraints.
You could have written this differently.
There's just any number of variants here.
And you'll get the sense of that as we go to other examples.
But we'll just stick to one variant here.
So now, I want to translate everything that I've written in English over there into algebra.
And so I got my minimization criterion-- minimize x1 plus x2 plus x3 plus x4-- subject to minus 2x1 plus 8x2 plus 0x3 plus 10x4 greater than or equal to 50,000.
And this represents the requirement that I want a majority in the first demographic, namely the urban demographic.
And so I want at least 50,000 votes there.
And I need to spend the money corresponding to the values of x1 through x4 in such a way that I get those 50,000 votes.
And that represents that, and it's just reading off the minus 2, 8, 0, and 10 from the urban column.
So those numbers that you see here correspond to the column, because I'm talking about the urban demographic.
And you can imagine that the next constraint is going to correspond to the middle column, and the third to the third.
So I'll just write that out.
I will call this constraint the constraint number 1-- I might refer to it later-- 5x1, plus 2x2 plus 0x3, plus 0x4 greater than or equal to 100,000.
Call this number two.
And then finally, 3x1 minus 5x2 plus 10x3.
And that's our set of constraints, but there's one more little issue that we have to be careful about, if you're being precise, and that is that there's no notion of un-advertising.
And so you're going to spend positive dollars.
And so x1 through x4 is greater than or equal to 0.
So that's our first linear program.
And it came from this particular problem.
It'd be wonderful-- and that's exactly what we're going to do for the rest of the lecture-- if we could solve this linear program and any possible linear program in an efficient way.
And so the number of variables is small n. And you can imagine that the number of constraints here, just talking about these constraints, are m constraints.
And you certainly want at run time that is polynomial in n. That's our goal here.
And as I mentioned early on, it was unclear for the longest time-- well, at least not until 1979, but people had been thinking about it for a long time before that-- as to whether there was a general algorithm that would solve any linear program in time-polynomial in n. And that was resolved in '79 by Khachiyan.
We'll look at a algorithm as simplex that in the worst case runs exponentially in n but is simpler to describe and is very efficient in practice.
So in our particular problem, this one, it turns out you-- and I'm actually going to come back to this in a second-- but I will just tell you that the optimum for this particular linear program with these particular numbers correspond to these numbers here.
So you want to spend something of the order of $20,000 for-- so there's 100 here, so take away the two 0's-- so spend something of the order of $20,000 for the first issue, building roads, spend a bit of money for the second issue, ignore the third corresponding to farm subsidies, and spend a bit of money for the gasoline tax issue.
And these numbers aren't important, other than the fact that they happen to be optimum.
So if you add up these numbers, then x1 plus x2 plus x3 plus x4 is something of the order of $21,000-- $27,000, excuse me, though I'm writing it out as this fraction.
So important consideration here is that these values xi are real numbers.
That's it.
It's not that they have to be integral.
Clearly, there were fractions here for the optimum, some of them anyway.
But in general, linear programming says the variable values are real.
There's also integer linear programming, which is NP complete, which adds the additional constraint that the xi values are integral.
So it turns into a harder problem.
You got polynomial-time solvable if the xi are real.
You got NP complete, which Eric is going to talk about on Thursday, if the values are forced to be integral.
So this extra constraint makes things worse from a complexity standpoint.
We won't talk about ILP anymore for the rest of this lecture.
So I will come back to this.
And I'll talk about how we can show that this is optimum without actually going into a deep algorithmic dive.
But what I want to do just before that is to give you the general formulation of a linear program.
It's called the standard form in CLRS, also called the general form.
In some cases, we'll look at the standard form for LP.
And I want to pop up a level about this example and give you the general setting.
And we'll focus in on the general setting for the most part.
But what I have here is I can either minimize or maximize-- we had a minimization problem-- for the political problem, minimize the linear objective function subject to linear inequalities or equations.
And the variables, think of x as a vector, it's a column vector, or x1, x2, all the way to xn.
And the objective function is c times x, so that's c1x1 dot, dot, dot, cnxn.
And we just had all the coefficients being a 1 over there.
And the inequalities, they're the fun part, you can represent them as a matrix A, so A times x less than or equal to b.
And notice that this is the standard form that I'm talking about.
And now, I have diverged from what I had here, because I had greater than or equal to over here.
So it turns out, you'll see linear programs in different settings.
Sometimes, you'll have minimization.
Sometimes, you'll have maximization.
Sometimes, you'll have less than constraints, less than or equal to constraints.
Sometimes, you'll have greater than or equal to constraints.
Sometimes, you'll have equality constraints.
We'll spend a little bit of time talking about how you can transform any given linear program into a standard form.
So our standard form is going to be something that maximizes the objective function.
So these are our inequalities, and they're represented as less than or equal to.
That's the standard form.
And you want to maximize c times x-- again, max for standard-- such that these set of inequalities told Ax less than or equal to b and x greater than or equal to 0.
So for each of the values that correspond to the variables, you want these variables to be non-negative in the standard form.
And you want less than or equal to corresponding to each of the inequalities-- not equal to, not greater than or equal to, but less than or equal to.
And you have this linear cost function, where you could have arbitrary coefficients, but you're maximizing it.
So that's it.
So it's all about polarities, not much more than that.
It's just about polarities.
And if you get a linear program, a specific linear program that doesn't conform to this-- we'll spend a few minutes talking about conversions-- and it's going to be fairly straightforward.
May not be immediately obvious, but we'll get to that.
Any questions so far?
So I want to go back to this claim here, where I said this is optimum.
Now without actually describing an algorithm to you, I'm going to be able to show you, convince you, that this value here corresponding to whatever it is, 3 million, 3.1 million, is in fact optimum.
And this is something I could do, because linear programming has this powerful notion of duality.
So what is that?
Well, let's just first look at our specific example here.
And I'll give you a very specific observation.
I'm going to give you what you can think of as a certificate of optimality.
I'm going to give you a certificate of optimality for that set of numbers.
And here's how I'm going to do it.
So is there a short certificate?
I can imagine giving you a long proof that a particular linear programming algorithm always gives you the minimum answer, the optimum answer, walk through that, executive that algorithm on this particular example, and then you're convinced, of course, that the solution is going to be optimum.
But for this specific example, I want to give you a certificate.
This certificate isn't going to work for other examples.
It's going to short, because it only works with this example.
But it won't work for other ones.
And so how do I do that?
So the answer is, in fact, there is a certificate that shows that the LP solution is optimal.
And consider that I compute this particular algebraic quantity, where all I've done is I've taken these three equations and I've multiplied them by these magical constants.
And so I'm not going to tell you how we get this certificate of optimality.
But I'm going to give you the certificate.
And it's going to be clear that it's a certificate of optimality.
And if I take these three equations here, 1, 2, and 3-- actually, I refer to 1, 2, and 3, they refer to the equations.
These are equations or constraints.
And so I take that.
And obviously, if I have a bunch of equations and I multiply them out, I can certainly add them up, and I get another equation at the end of it.
And it's all going to be linear.
And if I do that, I get x1 plus x2 plus 140 divided by 222 times x3 plus x4.
So that's what happens to the left hand side.
And the right hand side is-- you'll recognize this quantity-- five 0's divided by 111.
That's what you get.
So now, can someone tell me why in the last step, why this is a certificate of optimality-- the fact that obviously, this is all algebra, once I've discovered the coefficients-- so now that I've done this, why have I just shown you that 3.1 million divided by 111 is, in fact, optimum?
Can someone tell me this?
Look at what you have on the left hand side.
No?
Yeah
Any other solution would cost more than the amount put in spent
Any other solution, but I want you to relate that to-- what am I spending?
You're spending 3,100,000-- like, it's the same thing
Yeah, but I mean, this was a claim.
This was a claim-- and at this point, an unproven claim.
It was an unproven claim.
Yeah, go ahead
You know that the left hand side of that inequality is less than or equal to x1 plus x2 plus x3 plus x4
Correct.
And what is x1 plus x2 plus x3 plus x4, to be clear?
The thing you're trying to minimize?
Yeah, exactly, the thing you're trying to minimize.
Exactly.
You're almost there.
but the key observation here is that x1 plus x2 plus x3 plus x4 is greater than or equal to x1 plus x2-- because all of these are positive quantities, remember-- 140 divided by 222, that's less than 1, x3 plus x4, correct?
So given that, I can say that this is greater than our equal to 3,100,000 111.
It's because of this observation that it's a certificate of optimality.
She has her head down, OK. Great.
So that's pretty cool.
Just cooked up these coefficients from somewhere, pulled them out of a hat, you're all convinced now that the value we got was optimum.
Did not run an algorithm.
Maybe I ran an algorithm-- of course, you ran an algorithm to get those coefficients, right?
Well, how did those coefficients appear?
So we're not going to spend a whole lot of time on this.
You'll see this likely in a problem set or perhaps in section.
But in general, I won't worry too much about duality, other than knowing the concept.
And this notion of LP duality essentially says that what just happened wasn't a coincidence.
You can do this all the time.
There will always be, for a linear program, a short certificate of optimality that corresponds to some set of coefficients that you can do this particular math with by taking these linear constraints, multiplying them out, adding them up together, and showing that you have a lower bound on-- in the case of this problem-- you can't get lower than this.
And therefore, for a minimization problem, when you reach that, you clearly found the optimum.
And that's the notion of LP duality.
And the basic theorem-- and this is really more as an FYI, we won't prove this theorem-- is that if you had the standard form for the LP, which I'm just writing down again here, where you had Ax less than b, x vector greater than or equal to 0.
So that's identically what I had up here, or done here, corresponding to the standard LP form.
Well, there's a dual-- this is what's called the primal form.
Usually, if you don't say it, you think of it as the primal form.
And if it's dual, you call it a dual.
And this is primal form of LP.
This is a dual form of LP, or dual LP.
And the dual LP flips everything.
And it's not just negation, but transposed, and the actual variables also swap functionalities.
So it's really pretty cool.
So your max turns into a min.
The c gets replaced by the b, which is on the right hand side of the inequalities.
And your constraints are A transpose, y greater than or equal to c. So there's a flip there as well.
And y is greater than or equal to 0.
So there's a bunch of things that's going on here.
And these two problems end up being equivalent-- the primal and the dual, you can always do this.
And essentially, what is happening here is that you're solving these two problems simultaneously.
And there's lots of algorithms that keep flipping between these two forms for efficiency.
But ultimately, what ends up happening is you see that the actual constraints that you had here corresponding to the b constraints turn into-- the b ends up in the cost function here.
And that's essentially what's happening out here with respect to multiplying these equations out with particular coefficients.
So as I said, this is really more as an FYI.
This is an obviously interesting and an interesting proof of optimality, which is a different kind of proof from proving an algorithm correct and applying that proof to a particular instance.
That's the kind of thing that happens in LP, especially when you flip from primal and dual forms.
So I'll leave it at that.
What I'd like to do is give you a sense of how we can convert to standard form, so you can apply an algorithm that-- for example, you have a program and it only requires standard form.
It runs on standard form.
Let's go over it really quickly.
This is not going to take very long-- a translation from different kinds of LPs-- and we had a slightly different here for our political problem that had a minimization-- and how would we convert that to standard form.
So it's probably just one conversion here that's tricky.
So suppose I want to minimize minus 2x1 plus 3x2, and I want to convert it to standard formal.
All I have is a standard LP solver.
What do I do?
It should be easy.
What do I do if I had a solver that was maximizing, but I want to minimize a quantity?
Just switch the signs, right?
So negate to 2x1 minus 3x2 and maximize.
So that was easy.
This is a tricky one.
Suppose xj does not have a non-negativity constraint.
So it just happens to be the case that it's not dollars but it's some other quantity that can go negative.
It might be profit or loss.
So that quantity represents profit and loss, so it could go negative if it's loss.
So I don't have this constraint in my original problem specification.
But my standard form and my LP solver requires this entire vector to be non-negative.
So I got a problem here.
I can't use my standard solver, because of this non-negativity constraint.
So how do I fix that?
How do I turn it into a problem that allows the standard solver to be used?
Yeah, go ahead
You can break it up into two variables, like xj1 and xj2, so xj1 minus xj2 equals xj, and both could have [INAUDIBLE]
Perfect, great, that's good.
Here you go.
So what you do here is take xj and replace it with, let's say, xj prime minus xj double prime.
And you have xj prime greater than or equal to 0, xj double prime greater than or equal to 0.
But depending on the particular values in whatever solution you're exploring are the final solution, you may have an actual xj value that's negative or positive.
So you added an extra variable here to your linear program.
And a couple more real quick, suppose that I have an equality constraint corresponding to x1 plus x2 equals 7.
What do I do with an equality constraint where I have x1 plus x2 equals 7?
Yeah, go ahead
You can say x1 plus x2 is greater than or equal to 7, and x1 plus x2 is less than or equal to 7
No, you can't do less than or equal to
But then you can flip the signs to--
Ah, then you could flip the signs.
So you have two steps there, good.
So your less than or equal to needs another multiplier.
So what you end up doing is something like x1 plus x2 greater than or equal to 7.
And then you need-- if you do what the gentleman just said-- and flip the sign, you get minus x1 minus x2 greater than or equal to minus 7.
Is that right?
No?
I messed up?
Oh, I want less than or equal to.
You're right, you're right.
So I need less than or equal to-- that's right, of course, thank you.
So I need less than or equal to in both places.
So that's the standard form.
I needed less than or equal to.
Good.
What you've done is increased the number of constraints by one.
Did I get this right the second time?
All right.
So that's pretty much it.
The last thing, which I won't really write out, is, which we've done here already, greater than or equal to constraint translated-- I won't give you an example of this; we have this already-- translates to less than or equal to by minus 1 multiplied.
So we have to invoke that in order to do the equality anyway.
So you're in business.
If you have a standard LP solver, you can take pretty much any optimization problem that is linear in terms of its objective function and has linear constraints, and you can transform it into LP.
If you had non-linear constraints, there's lots of work that goes on in linearizing those constraints and using LP solvers.
It's a very practical thing to do.
It may be something you'll end up doing, invoking these powerful LP solvers-- typically, they're commercially available; the best ones are commercial-- and use it to solve your particular problem.
It turns your algorithm design problem into a reduction.
And so you'll spend really the next couple of weeks thinking about reductions.
We'll start that up right now, where we'll take existing combinatorial problems, for which we already know algorithms for, and you're going to reduce them to LP.
Just to give you a sense of what the power of LP is, but this notion of reduction is very powerful, you can use it to do complexity proofs.
Here, we're just using it as a convenience in today's lecture to use our LP hammer.
So let's say that I have our favorite problem of the week, namely max flow.
And I want to convert that to LP.
So go back a week ago, and right about this time a week ago, and we'd set up the max flow problem.
And let's assume that we went back there.
And we didn't talk about augmenting paths, and we didn't talk about residual capacities or min-cut or anything like that, but we knew LP already.
And we just want to solve max flow using LP.
So let's do that.
So this is maximum flow.
I'm not going to bother with converting to standard form.
We know how to do that, given what we just did here, over there.
So I'll just do whatever I want to keep things simple.
Max flow is obviously a maximization problem.
And using the same notation we've used, it's not going to look like Ax and b, just because I want you to recall what max flow is.
And we're going to translate that.
And the values of these variables-- or the names of these variables, whether they're x or f, it should really matter.
We know how to do LP at this point-- we know how to formulate LP, I should say, at this point.
And we're assuming that we have an LP solver.
So what I want to do here with the maximum flow problem is maximizing the flow value.
And it's simply, you grab the source, you have a variable associated with the flow from the source to every other vertex of v. And you have to maximize that.
So that was the setup for max flow.
I'm not changing that.
What do you think the three constraints, or whatever set of constraints that we have here, are going to correspond to the LP?
You spent a week on max flow, looked at the problem set, what constraints am I going to have to put up there?
I'm going to have to put up capacity constraints.
That's an obvious one.
What is another one?
Conservation constraints.
All flow entering a node that is not the source or the sink has to leave it.
In the original network, is there a concept of negative flow?
No, you will define it going in the other direction.
So we did talk about negative numbers, et cetera, but you're going to have positive quantities, especially if you look at net flow, the version of the flow that we zoomed in on in the Tuesday lecture from last week.
And you also have-- in the general setting, you're going to have these skew symmetry constraints as well.
So the three things that you need here are skew symmetry, conservation, and capacity.
So you have such that f u, v equals minus f v, u for all u, v belonging to V. And depending on the kind of network that you have, if you constrain it to a sudden type that you don't have these two-way edges, you could certainly remove some, if not all, of these skew symmetry constraints.
Important ones are conservation and capacity.
And this should seem pretty familiar to you.
But the key-- the reason I'm writing these all out is primarily to ensure that you understand that these are all linear constraints.
So that's pretty much the only thing that you need to observe here.
Obviously, these constraints you've seen many a time from the two lectures last week.
But notice that they're all linear.
And finally, this one is f u, v less than or equal to c of u, v for all u, v belonging to cap V. So this is f. That's a variable that's less than another constant, clearly linear.
Doing a bunch of sums here.
I could obviously have multipliers, scalar multipliers.
In this case, for conservation, I don't have scalar multipliers, but clearly linear.
Skew symmetry, got two variables in here.
One of them is a negation of the other, clearly linear, so that's why this is an LP.
And so you might say, well, I know better, max flow is much more efficient than any LP solver that's out there.
And you would be right.
If you have a max flow problem of this variety, it's difficult to imagine that you would get performance, empirical performance from running an LP solver.
But this generalization of max flow that's a multi-commodity max flow, where you just don't have one commodity flowing through.
You may be counting cars and trucks on a road, or there's two different kinds of liquid flowing through the same pipe, whatever, gas or liquid.
And so when you have multiple commodities, you may have a linear but more complicated cost function that's a function of the flow of each of the commodities.
And they may have a certain weight associated with them.
So there's a lot of things that could be more general-- there could be more general settings corresponding to max flow.
And I'll just leave you with the thought that you could simply have two commodities.
And we'll just call them 1 and 2.
And so now, you have the f1's and the c1's and the f2's and the c2's.
Each commodity has to be conserved.
But what about the capacity?
What do you think happens with the capacity?
Let's just assume these are two different kinds of cars.
So what would the capacity constraint look like?
Yeah
You can say either c1 or f1 plus f2 is-- for each edge, you can add them together or you might take the linear [INAUDIBLE]
Exactly, that's right.
So good point.
It may be the case that I have distinct capacities.
And in fact, if you have completely disjoint problems, you're right in that you can solve them separately.
But actually, the more interesting case would be that you have a single capacity c, so you'll have-- let me just write this out here.
If in fact you had two distinct things, so if you had f1, c1, f1, c2, the question is, do you have two distinct, disjoint optimizations, in which case you just use max flow twice.
On other hand, what's more interesting really-- and I should've used this example for starters-- but here's a better one.
You have two commodities and a single capacity.
So the road is a good example.
Both the cars and the trucks share the same road.
It has a certain capacity.
And now, your capacity constraint is looking like f1 plus f2 over here.
And that's the flow through the particular edge uv.
So you have something like f1 u, v plus f2 u, v is less than or equal to c u, v for this total capacity.
And that's pretty much it.
So that is linear.
The nice thing is that it's linear.
You could put weights on it.
If you wanted to claim that a particular commodity 1 uses up-- because it's a truck, it uses up more space on the road.
And you can accommodate fewer of them.
You could put a multiplier in there.
Still say it's linear.
So that's the power of by having an LP engine.
You could translate problems that are not exactly max flow, that are multi-commodity flow.
You may have additional linear constraints that you could add, and you could still use your LP package.
So that's the reason why this is interesting and powerful.
So that is kind of an obvious, corresponding to max flow.
Let's look at something that's a little less obvious.
And it's going to be a little tricky to convert the shortest path problem to LP, not a lot of work but one little observation that's going to be important to make in order for the whole thing to flow through or actually work out.
So we all know the shortest path problem.
We want to find-- let's just call it the single source shortest path problem.
You have a specific source.
That's going to turn into the point from which you're going to start computing the distance.
That's what Dijkstra does, and that's Bellman-Ford does.
And so this is from vertex x-- s, excuse me, s. And what I want to do here is obviously set it up as a set of linear constraints.
If I have dv corresponding to the distance from the source-- so dv represents the distance from the source-- and eventually I want dv to be the shortest distance from the source.
That's our notation for shortest paths.
dv represents an existing path-- it may not be the shortest path-- from s to v, the value of that.
But dv monotonically decreases as you run through.
It's initially infinity in Dijkstra going back to Dijkstra.
And then we shrink it through a process of relaxation.
Now I want to try and model that-- I want to try and model all of this as an LP.
So it's not immediately obvious-- the thing that the flow networks had, where we had these constraints.
We have capacity constraints and conservation constraints.
And we could turn that constraint into an inequality.
And it was pretty smooth.
It's pretty easy.
So what I need to do here with shortest paths is something that's a little more subtle.
So what basic constraint do I have in a shortest path algorithm?
What's an inequality-- you remember an inequality from shortest paths that we kept talking about?
The triangle inequality.
So we're going to have to go with the triangle inequality and take the triangle inequality and use that to create an LP formulation of shortest paths.
In particular, what we have here is that I could write dv minus du is less than or equal to w u, v for all u, v belonging to E. And that's the triangle inequality.
And I'm going to have d of s equals 0.
That's the only thing that I start with.
And so what's happening out here is simply that there's different ways of getting to v. And my shortest path is going to be the best way of getting to v. So in particular, the way you want to think about this is that if I have a v and I can get to it from, let's just say, u1 and u2.
And maybe the source is over here.
And these are the only two edges that can get to v. So I'm just looking at a fairly limited setting.
u1 and u2 are going to have to be the two vertices.
One of these two is going to get me to v. And I got w u1, v here.
And I got w u2, v over here.
And so what this is saying is, I'm going to have to write this out for each of these edges.
For each of these edges, I'm going to have this constraint.
And that's says that the dv value, if I want the shortest path, should obey both of these constraints.
And if I want to obey both of these constraints, one of them is going to be my limiting constraint.
And I'm going to get the min of those two.
Correct?
So in effect what this translates to is that it's an AND, right?
So dv have minus du1 is less than or equal to w u1, v. dv minus du2 is less than or equal to w u2, v. That's an AND, because I'm putting both of those constraints in here.
And that essentially means that dv is going to be the min of the two quantities-- the du1 quantity plus the w u1, v, and the du2 quantity plus the w u2, v. That make sense?
Ask me questions if this is unclear.
So that simply corresponds to the fact that I'm doing an AND over here.
I'm adding all of these constraints in there.
So I'm applying the triangle inequality to every edge, to every relationship between a vertex that has a path ending at it.
And you're pushing it forward to this vertex v, all the different ways that you can get to v. In this case, there is two sets of ways-- one from u1 and one from u2.
And the last step is a minimization step.
So you think you're done or we're done, but we're not quite done, because what's missing here in terms of my formulation of LP?
What else do I have to do here?
Well, sure, non-negative-- let's do that.
Sorry?
Objective
Objective, who said objective?
You again?
So we are missing an objective function.
Now shortest path is what kind of problem again?
Short means minimum?
Minimum, height whatever.
So do I put a-- what happens if I put a minimum in here?
And let's say that I do something like sigma over V, cap V, dv, because I want to minimize-- or I could pick a particular one.
I could pick a particular single source, single destination, and I put a minimum there.
What happens?
Does this work?
What's the solution to this problem, if I minimize the distance?
0, because the zero value is going to work.
So there's something-- I haven't put in the constraint that I do want a path.
I do want a path from s to v for any v that matches one of the quantities in the min, because the min says I have equality.
The big issue here is, this is a less than or equal to, and that's why the min doesn't work in the objective function.
It's a less than or equal to.
But this min over here, which is the definition of a shortest path, is saying that it's either equal to this quantity or equal to that quantity.
There's an equality over here that is missing from this side.
And that's the key observation.
Once you observe that, that need equality for one of these constituent quantities of the min, then you'll see that what you have to do is simply change this min to a max.
So you say, well, how the heck did a min get changed into a max?
And I'm not sure I'm going to convince every on of you in the next minute or so.
But the bottom line is, it comes down to I do have a min already in my inequalities, because I'm ANDing each of these inequalities, and I'm putting down each of those inequalities in there.
So each of them is going to force me to find the best solution, because they're going to constrain me to not go via u2 if u1 is better, because the other constraint corresponding to u1 is going to force me down.
So there's an additional min in there, because the ANDing of the less than or equal tos.
And then in order to actually force the equality for one of those, I need to push up as hard as I can, or as high as I can.
So think about it.
Play around with a couple of examples.
Choose a simple example for starters.
And you'll see that this is the correct formulation.
So you can see that it's not completely clear in some cases how to transform problems to LP.
But even in those cases, sometimes you can.
So there's just a ton of different problems, a good skill to have to be able to take combinatorial optimization engines, like LP or even max flow, and be able to translate problems to them.
It's something that you'll probably do if you stick to algorithms in your careers or exploiting available algorithm packages.
So the last thing I want to do here for the rest of the time is to give you some sense for how an LP program is actually optimized.
How can you possibly take the standard LP formulation, which is a general setting.
You know nothing about shortest paths, let's assume, nothing about max flow.
It's not about a specific problem.
It's about the general setting.
How can we solve the general setting, because that was the theme here anyway.
You had this engine, and you want to use this engine.
But now, how do you build this engine?
So what we're going to do is look at a fairly simple example of the simplex algorithm.
And this algorithm is in the textbook.
And it'll be in my notes.
So I'll get as far as I can.
It's not that complicated to describe, especially from an example standpoint.
But I may not get through all of the steps to get you the optimum for this particular example given how much time we have.
The most important concept in simplex is yet another form of representation for simplex, which says that you can represent the LP, not in standard form, but in slack form.
So I'm going to tell you what slack form is.
And then what we're going to do is, the flow of simplex, algorithmic flow, is to convert one slack form into and equivalent.
Obviously, you don't want to do something that's incorrect, but it has to be an equivalent slack form, whose objective value has not decreased and has likely increased.
So you're guaranteed that the objective value has not decreased.
You're not guaranteed that it's increased.
And then we're going to keep going till the optimal solution becomes obvious.
And you might say, how is this obvious?
That's the reason why I talked about the short certificate of optimality.
It's definitely a relationship between the termination of simplex and the fact that you can now say, hey, I know I'm done here, it's kind of obvious that I can't do any better.
And hopefully, you'll see that by the end of this lecture in this simple example.
So that's it.
It's an iterative algorithm.
It's exponential, unfortunately, because this takes m plus n, choose n iterations in the worst case, where n is the number of variables and m is the number of constraints.
Most of the time, it does a lot better, but that's the only bound that you can actually prove in the worst case.
And so you're stuck with an exponential algorithm if you're using simplex worst case.
We won't actually do much analysis on simplex.
It's really out of scope for 046 in terms of the analysis.
The actual algorithm is certainly within scope.
So what I want to do is give you some sense for what the slack form looks like.
And we'll do a couple of iterations of simplex.
And we'll get as far as we can before the end the lecture.
So we'll take a different example from our political example.
It's similar in size.
And I want to explain to you what the slack form is and why it's interesting.
So what I want to do is maximize 3x1 plus x2 plus x3 subject to the constraints that x1 plus x2 plus 3x2 is less than or equal to 30, 2x1 plus 2x2 plus 5x3 is less than or equal to 24, 4x1 plus x2 plus 2x3 less than or equal to 36, and then non-negativity constraints, x1, x2, x3 greater than or equal to 0.
So that's our example problem.
I'll leave it up there.
You're going to convert this to slack form.
And so what is the slack form?
We're going to introduce an additional number of variables that correspond to the number of equations that we have.
So we're going to introduce, in this case, three new labels, because I have three equations.
And the slack for this problem looks like this.
I'm going to have z equals 3x1 plus x2 plus x3, same as before.
And then I'm going to have variables that represent-- these are called basic variables.
And the original variables are called non-basic variables.
So I'm going to add three basic labels, x4, x5, and x6, corresponding to these three constraints.
And they're going to represent slack in the sense that they're going to correspond to how much slack you have in the inequalities that you have in the original problem.
So if x4 happens to be 0, then you're jammed up.
You have no slack, because x1 plus x2 plus 3x3 equals 30.
And increasing any one of them will violate the constraint.
So that's just simply the notion of slack.
It's how much room do you have.
x5 is 24 minus 2x1 minus 2x2 minus 5x3.
And the last one is 36 minus 4x1.
This is very mechanical up to this point.
And so I'll call this set of equations, equation I.
And what I'm going to do is I'm going to now work on a space that corresponds to x1, x2, x3, x4, x5, x6.
So I'm going to have these solutions that now have six values associated with them, as opposed to just three values, because I've added three variables, the basic variables, to my non-basic set.
So the original variables are non-basic, just to differentiate.
So so far it's just set up its definitions.
We can think about up running through iterations of simplex.
It takes about three iterations here in order to get to the point where the optimum is obvious.
So you're going to convert through three slack forms.
And then finally, when you get to this fourth slack form, you see that you have an optimal solution.
And how does that work?
You're going to have the notion of a basic solution, where we set all non-basic variables to 0.
So in this case, what we're going to have-- and then we're going to compute values of the basic variables.
So our objective function here is going to be 3 times 0 plus 1 times 0 plus 2 times 0, which is obviously 0.
And the values x4, of course, is going to be-- because all of these are 0-- is going to be 30. x5 is going to be 24.
And x6 is going to be 36.
So this is kind of a trivial starting point.
So you can think of this as 0, 0, 0, the solution that you're looking at, 30 24, and 36.
So that's our starting point, which doesn't really tell you much.
But now comes the key step, where we're going to do something that's called pivoting.
And in pivoting, you're going to swap a basic able with a non-basic variable.
It is a step that requires some intelligence.
But the basic step is a swap.
So one of the basic variables is going to become a non-basic variable and vice versa.
And how do we select this?
Initially, you can kind of do this in an arbitrary way.
It gets a little more subtle as you go along.
You don't want to do things in a random way.
But let's just start with what pivoting actually does in a more generic setting.
The basic step is select a non-basic variable-- let's call it x sub e-- who's coefficient in the objective function is positive.
You can always redefine things if you can't find something like that.
But we won't go there.
And then what we're going to do is we're going to increase the value of x as much as possible.
And we always have constraints, of course.
So we can do that without violating any of the constraints.
And then at this point, variable xe becomes basic.
So it's going to turn into the left hand side.
It's going to move over.
You're going to swap-- the x1 might be over here, over here, and you got to rewrite these equations so the x1 becomes basic, for example, over to the left hand side, and the variable that it replaced goes over to the right hand side.
So you can think of this as Gaussian elimination, except with inequalities.
There's definitely relationships there, if you recall your Gaussian elimination.
If you don't, don't worry about it.
So xe becomes basic, and some other variable becomes non-basic.
The values of other basic variables and the objective function may change.
So we'll do one pivot step at least, so you get a sense for the algebra involved.
And it becomes a little more concrete.
To motivate you further, you'll be doing this in problem set 8 on a different example.
So what did I do here is we're going to select a non-basic variable.
So let's just select x1, lexicographic order or numeric order, let's select x1 is selected.
That's the non-basic variable.
And what I want to do is increase the value of x1.
So I want to increase it without violating constraints.
Now, which of these constraints do you think is going to cause trouble first with respect to increasing the value of x1?
x1 is now 0.
We're at ground level, we have all 0, things are feasible.
Now, as I start increasing x1, remember, you have non-negativity constraints associated with each of the x4 values as well.
That's what these basic variables represent.
So don't forget the fact that the constraints are violated.
You need all the xi's to be greater than or equal to 0.
That was true for the non-basic variables.
It's also true for the basic variables.
So a violation of a constraint implies that one of the currently basic variables goes negative.
That's exactly equivalent to the original inequality not being satisfied.
So which of these constraints, do you think, is going to cause trouble here?
Just look at it, and should be able to look at the three equations and tell me, as I increase the value of x1, where am I going to hit my limit?
x6, yes, absolutely correct.
So that's because of the minus 4 here.
This is a big multiplier.
I got a minus 1 here, a minus 2 here, and a minus 4 here.
And if we just look at 4, the magnitudes, 4 is bigger than 2 and is bigger than 1, so it's going to be that third constraint.
So third constraint-- you can obviously compute this numerically or mechanically.
It was just easier to do this in this example by eyeballing it.
And so third constraint is the tightest one.
And it limits how much we can increase x1.
So I'm going to do my second step up there, which corresponds to rewriting x1 as these other variables.
And now, I got x1 on the left hand side and x6 on the right hand side.
And now, it's just merely a matter of substitution.
Once I've done this, I'm just going to jam through and go in and I'm going to rewrite the other equations with x6 on the right hand side.
And that is, I'm going to replace x1 with the eyeball equation.
And that's really a simple substitution.
So at this point, what's happened is, because the third constraint representing x6 was the one that was chosen, what's happened is that x1 and x6 are going to interchange roles.
x1 was non-basic, it's going to become basic.
x6 was basic and is going to become non-basic.
And that's basically the essence of the simplex algorithm.
The iteration and then the convergence and all of that is, as I mentioned, going to require getting to a point where the optimality is obvious, but we won't be doing any proofs corresponding to conversions for simplex or any other specific LP techniques in this class.
Maybe some constraint techniques, I take back what I said, but certainly not for simplex.
But I just want to give you a sense for the flow here.
And so let's just go through this last thing here in terms of finishing off the pivot xi's.
And I want to show you what the equations look like.
And if you just keep doing that, at some point, you'll converge.
And so what happens here is you have z equals 27 plus x2 divided by 4 plus x3 divided by 2 minus 3x6 divided by 4. x1 equals-- so there's a bunch of algebra here that I'm obviously skipping over, but it's simple algebra.
And I have x4 over here, 21 minus 3x2 divided by 4 minus 5x3 divided by 2 plus x6 divided by 4, and then one last thing here.
So that's my pivot step, that I've flipped x1 and x6.
So now you ask, what was the point?
What was the point of this?
Well, the point of this was that you actually increased the objective function value and, in this particular case, quite significantly, while maintaining correctness.
And so let me just make these observations and conclude here, because then that gets us to the point where you've seen the details of one pivot step.
And you can imagine applying it over and over to a convergence.
And let's just look at the original basic solution, which as you recall was 0, 0, 0, 30, 24, and 36.
And this is simply x1 through x6.
This original basic solution suddenly satisfies these equations-- equations II, if you just put them in there.
And it makes sense that it will have the objective value of 0, given equivalents, but you can verify that by saying that you have 27-- the original had the objective value of 0, because all the x sub 1, x2, x3 were 0, so that was an easy check in the set of equations corresponding to the first set, which I've erased at this point.
But no matter.
And if you look at what I have here, I have 36 equals 0.
So the objective value here is 0.
It matches what you had before.
But the basic solution for II, I'm going to set the non-basic values.
So what is a solution here?
The non-basic values are 0.
So the solution is going to be 9, because 9 is non-basic.
x1 is now non-basic.
x2 and x3 are still basic.
And then I have 21 and 6.
And x6 now has become basic.
So the way I get this solution is simply by plugging in 0's on this aside.
And I get 9, 21, and 6, because I just plugged in 0's on the right hand side.
So that's how I got a new solution.
And if you look at the objective value for this, the objective value, you can look at this objective value simply by looking at the original problem.
And the original problem had 3x1 plus x2 plus x3 as the objective value.
And so if you go off and you see, well, that you had 0's for the other ones, but you have 3 times 9, so we have an objective value of 27.
So this flip of our pivot basically got you from an objective value of 0, while maintaining correctness, to an objective value of 27.
And you can look at this in the notes or in CLRS, but you have to do two more pivots corresponding to two other variables-- the same grungy stuff that I went through here, substitution after selection.
And the objective value is going to increase.
And you might ask, how do I know that I'm done?
And so that was the last thing here, which is increase the value until it becomes obvious-- no, this was pivoting, I'm sorry-- increase the value pivot until it becomes obvious what the optimum is.
And what ends up happening is you end up with an objective function.
In this case, the objective function mind you, this thing over here is the objective function.
And notice that it has a negative coefficient on the variable that was actually the part of the first pivot.
So x1 and x6 were a part of the first pivot.
And this got a negative coefficient.
So what ends up happening here is you end up getting something like 30 minus something minus something minus something, where you have xi values over here.
And when you set these to be 0, that's the best you can do, because these are all negative quantities.
So I'll just leave you with that.
Hopefully, you understood how we do the pivoting-- sub through it three times, and then you get that objective function, and the optimum value is 30.
See you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, today we do NP completeness, an entire field in one lecture.
Should be fun.
I actually taught an entire class about this topic last semester, but now we're going to do it in 80 minutes.
And we're going to look at lots of different problems, from Super Mario Brothers to jigsaw puzzles, and show that they're NP -complete.
This is a fun area.
As Srini mentioned last class, it's all about reductions.
It's all about converting one problem into another, which is a fun kind of puzzle in itself.
It's an algorithmic challenge.
And we're going to do it a lot.
But first I'm going to remind you of some of the things you learned from 006, and tell you what we need to do in order to prove all of these relations, what exactly we need to show for each of those arrows, and why it's interesting.
So this is generally around the P versus NP problem.
So remember, P is all the problems we know how to solve in polynomial time.
Well not just the ones we know how to solve, but also the ones that can be solved, which is pretty much-- which is the topic of 6.006, and 6.046 up till now.
But for now, in the next few lectures, we'll be talking about problems that are probably not polynomially solvable, and what to do about them.
Polynomial, as you now, is like n to the some constant.
Polynomial good exponential bad.
What is n?
I guess n is the size of the problem, which we'll have to be a little bit careful about today.
And then NP is not problem solvable not in polynomial time, but it's problem solvable in nondeterministic polynomial time.
And in this case we need to focus on a particular type of problem, which is decision problems.
Decision just means that the answer is either yes or no.
So it's a single bit answer.
We will see why we need to restrict to that kind of problem in a moment.
So this is problems you can solve in polynomial time.
Same notion of polynomials, same notion of n, but in a totally unrealistic model of computation.
Which is a nondeterministic model.
In a nondeterministic model, what you can do is say instead of computing something from something you know, you could make a guess.
So you can guess one out of polynomially many options in constant time.
So normally a constant time operation, in regular models, like you add two numbers, or you do an if, that sort of thing.
Here we can make a guess.
I give the computer polynomially many options I'm interested in.
Computer's going to give me one of them.
It's going to give me a good guess.
Guess is guaranteed to be good.
And good means here that I want to get to a yes answer if I can.
So the formal statement is, if any guess would lead to a yes answer, then we get such a guess.
OK, this is weird.
And it's asymmetric.
It's biased towards yes.
And this is why we can only think about decision problems, yes or no.
You could bias towards no.
You get something else called coNP.
But we'll focus here just on NP.
So the idea is I'd really like to find a guess that leads to a yes answer.
And the machine magically gives me one if there is one.
Which means if I end up saying no, that means there was absolutely no path that would lead to a yes.
So when you get a no, you get a lot of information.
When you get a yes, you get some information.
But hey, you were lucky.
Hard to complain.
So in 006, I often call this the lucky model of computation.
That's the informal version.
But nondeterminism is what's really going on here.
So maybe it's useful to get an example.
So here's a problem we'll-- this is sort of the granddaddy of all NP-complete problems.
We'll get to completeness in a moment.
3SAT-- SAT stands for satisfiability.
So in 3SAT, the input to the problem looks something like this.
I'm just going to give an example.
And in case you've forgotten your weird logic notation, this is an and.
These are ORs.
And I'm using this for negation, not.
So in other words, I'm given a formula which is and of ORs.
And each or clause only has three things in it.
These things are called literals.
And a literal is either a variable x sub i, or it's the negation of a variable, not x sub i.
So this is a typical example.
You could have no negations.
You could here have one negation, two negations, any number of negations per clause.
These groups of three-- these or of three things, three literals, are called clauses.
And they're all ANDed together.
And my goal is, this should be a decision question, so I have a yes or no question.
And that question is, can you set the variables-- So they're x1 to true or false?
So each variable I get to choose a true or false designation such that the formula comes out true.
I use T and F for true and false.
So I want to set these variables such that every clause comes out true, because they're ANDed together.
So I have to satisfy this clause in one of three ways.
Maybe I satisfy it all three ways.
Doesn't matter, as long as at least one of these should be true, and at least one of these should be true, and at least one of each clause should be true.
So that's the 3SAT problem.
This is a hard problem.
We don't know a polynomial time algorithm.
There probably isn't one.
But there is a polynomial time nondeterministic algorithm.
So this problem is in NP because if I have lucky guesses, it's kind of designed to solve this kind of problem.
What I'm going to do is guess whether x1 is true or false.
So I have two choices.
And I'm going to ask my machine to make the right choice, whether it should be true or false.
Then I'll guess x2.
Each of these guess operations takes constant time.
So I do it for every variable.
And then I'm going to check whether I happen to satisfy the formula.
And if it comes out true, then I'll return yes.
And if it comes out false, I'll return no.
And because NP is biased towards yes answers, it always finds a yes answer if you can.
If there's some way to satisfy the formula, then I will get it.
If there's some way to make the formula come out true, then this algorithm will return yes.
If there's no way to satisfy it, then this nondeterministic algorithm will return no.
That's just the definition of how nondeterministic machines work.
It's a little weird.
But you can see from this kind of prototype of a nondeterministic algorithm, you can actually always arrange for your guessing to be at the beginning.
And then you do some regular polynomial time checking or deterministic checking.
So when you rewrite your algorithm like this with guesses up front and then checking, you can also think of it as a verification algorithm.
So you can say, your friend claims that this 3SAT formula is satisfiable, meaning there's a way to set the variable so that it comes out true.
So this is called a satisfying assignment.
Satisfying just means make true.
And you're like, no, I don't believe you.
And your friend says no, no, no, really, it's true.
And here's how I can prove it.
You set x1 to false.
You set x2 to true.
You set x3-- basically they give you the guesses.
And then you don't have to be convinced that those are the right guesses, you can check that it's the right guess.
You can compute this formula in linear time, see what the outcome is.
If someone tells you what the xi's are, you can very quickly see whether that was a satisfying assignment.
So you could call this a solution, and then there's a polynomial time verification algorithm that checks that solutions are valid.
But, you can only do that for yes answers.
Your friend says no, this is not satisfiable, they have no way of proving it to you.
I mean, other than checking all the assignments separately, which would take exponential time, there's no easy way to confirm that the answer to this problem is no.
But there is an easy way to check that the answer is yes, namely I give you the satisfying assignment.
So this definition of NP is what I'll stick to.
It's this sort of-- I like guessing because it's like dynamic programming.
With dynamic programming we also guess, and guessing actually originally comes from this world, nondeterminism.
In dynamic programming, we don't allow this kind of model.
And so we have to check the guesses separately.
And so we spend lots of time.
Here, magically, you always get the right guess in only constant time.
So this is a much more powerful model.
Of course there's no computers that work like this, sadly, or I guess more interestingly.
So this is more about confirming that your problem is not totally impossible.
At least you can check the answers in polynomial time.
So that's one thing.
So this is an equivalent definition of NP because you can take a nondeterministic algorithm and put the guessing up top.
You can call the results of those guesses a certificate that an answer is yes.
And then you have a regular old deterministic polynomial time algorithm that, given that certificate, will verify that it actually proves that the answer is yes.
It's just that certificate has to be polynomial size.
You can't guess something of exponential size.
You can only guess something of polynomial size in this model.
So seems a little weird.
But we'll see why this is useful in a little bit.
So let me go to NP completeness.
So if I have a problem X, it's NP-complete if X is in NP and X is NP-hard.
But I haven't told you what NP-hard is.
Maybe you remember from 006, but let me remind you.
So, I need to define reduce.
So maybe I'll do that as well, then we can talk about all these.
OK, a lot of definitions.
But the idea of NP hardness is very simple.
If problem X is NP-hard, it means that it's at least as hard as-- sorry, that is a Y-- it's at least as hard as all problems in NP.
Intuitively, X means it's at least as hard as everything in NP.
Whereas being in NP is a positive statement.
That says it's not too hard, at least there's a polynomial time verification algorithm.
So being in NP is good news.
It says you're no harder than NP.
NP-hard says you're at least as hard as everything in NP.
And so NP-complete is a nice answer because this says you're exactly as hard as everything in NP-- no harder, no easier.
If you draw, in this vague sense, computational difficulty on one axis-- which is not really accurate, but I like to do it anyway-- and you have P is all of these easy problems down here.
And NP is some larger set like this.
NP-hard is from here over.
And this point right here is NP-complete.
Being in NP means you're left of this line, or on the line.
And being NP-hard means you're right of this line, or on the line.
NP-complete means you're right there.
So that's a very definitive sense of hardness.
Now there is this slight catch, which is we don't know whether P equals NP.
So maybe this is the same as this, but probably not.
Unless you believe in luck, basically, unless you imagine that a computer could engineer luck and always guess the right things without spending a lot of time, then P does not equal NP.
And in that world, what we get is that if you have an NP-complete problem, or actually any NP-hard problem, you know it cannot be NP.
So if you have that X is NP-hard, then you know that X is not in P unless all of NP is in P. So unless P equals NP.
And most reasonable people do not believe this.
And so instead they have to believe this, that your problem is not polynomially solvable.
So why is this true?
Because if your problem is NP-hard, it is at least as hard as every problem in NP.
And if you believe that there is some problem in NP-- we don't necessarily know which one-- but if there is any problem out there in NP that is not in P, then X has to be at least as hard as it.
So it also requires nonpolynomial time, something larger than polynomial time.
What does at least as hard mean though?
We're going to define it in terms of reductions.
Reduction from one problem to another is just a polynomial time algorithm, regular deterministic polynomial time, that converts an input to the problem A into an equivalent input to problem B.
Equivalent means that it has the same yes or no answer.
And we'll just be thinking about decision problems today.
So why would I care about a reduction?
Because what it tells me is that if I know how to solve problem B, then I also know how to solve problem A.
If I have a, say, a polynomial time algorithm for solving B and I want one for A, I just take my A input.
I convert it into the equivalent B input.
Then I run my algorithm for B, and then it gives me the answer to the A problem because the answers are the same.
So if you have a reduction like this and if say, B, has a polynomial time algorithm, then so does A, because you can just convert A into B, and then solve B.
Also this works for nondeterministic algorithms.
Not too important.
So what this tells us is that in a certain sense-- get this right-- well this is saying, if I can solve B, that I can solve A.
So this is saying that B is at least as hard as A. I think I got that right, a little tricky.
So if we want to prove the problem is NP hard, what we do is show that every problem in NP can be reduced to the problem of X.
So now we can go back and say well, if we believe that there is some problem Y, that is in NP minus P, if there's something out here that is not in P, then we can take that problem Y, and by this definition, we can reduce it to X, because everything in NP reduces to X.
And so then I can solve my problem Y, which is in NP minus P, by converting it to X and solving X.
So that means X better not have a polynomial time algorithm, because if it did, Y would also have a polynomial time algorithm.
And then in general, P would equal NP, because every problem in NP can be converted to X.
So if X has a polynomial time algorithm, then every problem Y does.
Question?
For the second if statement, why can't you say that if A is in NP, B is in NP?
So you're asked us about the reverse question.
If is A in NP, can we conclude that B is in NP?
And the answer is no.
Because this reduction only lets us convert from A to B.
It doesn't let us do anything for converting from B to A.
So if we know how to solve A and we also know how to convert A into B, it doesn't tell us anything.
It could be B is a much harder problem than A, in that situation.
That's, I think, as good as I can do for that.
Other questions?
All right.
It is really tricky to get these directions right.
So let me give you a handy guide on how to not make a mistake.
So maybe over here.
What we care about, from an algorithmic perspective, is proving the problems are NP-complete.
Because if we prove NP-completeness-- I mean, really we care about NP-hardness, but we might as well do NP-completeness.
Most of the problems that we'll see that are NP-hard are also NP-complete.
So when we prove this, we prove that there is basically no polynomial time algorithm for that problem.
So that's good to know, because then we can just give up searching for a polynomial time algorithm.
So all the problems we've seen so far have polynomial time algorithms, except a couple in your problem sets, which were actually NP-complete.
And the best you could have done was exponential, unless P equals NP.
So here's how you can prove this kind of lower bound to say look, I don't need to look for algorithms any more because my problem is just too hard.
It's as hard as everything in NP.
So this is just a summary of those definitions.
The first thing you do is prove that X is in NP.
The second thing you do is prove that X is NP-hard.
And to do that, you reduce from some known NP-complete problem-- or I guess NP-hard, but we'll use NP-complete-- to your problem X.
Maybe I'll give this a name Y. OK, so to prove that X is in NP, you do something like what we did over here, which is to give a nondeterministic algorithm.
Or you can think of it as defining what the certificate is and then giving a polynomial time verification algorithm.
So sort of two approaches.
You can give a nondeterministic polynomial time algorithm, or you give a certificate and a verifier.
There's no right or wrong certificate.
I mean, a certificate, you can define however you want, as long as the verifier can actually check it and when it says yes, then the answer to the problem was yes.
So it's really the same thing.
Just want to say there's some certificate that a verifier could actually check.
So that's proving that your problem is in NP.
It's sort of an algorithmic thing.
The second part is all about reductions.
Now the definition says that I should reduce every problem in NP to my problem X.
That's tedious, because there are a lot of problems in the world.
So I don't want to do it for every problem in NP.
I'd like to just do it for one.
Now if I reduce sorting to my problem, that's not very interesting.
It says my problem is at least as hard as sorting.
But I already know how to solve sorting.
But if I start from an NP-complete problem, then I know, by the definition, that every problem in NP can be reduced to that problem.
And if I show how to reduce the NP-complete problem to me, then I know that I'm NP-complete too.
Because if I have any problem Z in NP, by the definition of NP-complete of Y I can reduce that to Y.
And then if I can build a reduction from Y to X, then I get this reduction.
And so that means I can convert any problem in NP to my problem X, which means X is NP-hard.
That's the definition.
So all this is to say the first time you prove a problem is NP-complete in the world-- this happened in the '70s by Cook.
Basically he proved that 3SAT is NP-complete.
That was annoying, because he had to start from any problem in NP, and he had to show that you could reduce any problem in NP to 3SAT.
But now that that hard work is done, our life is much easier.
And in this class all you need to think about is picking your favorite NP-complete problem.
3SAT's a good choice for almost anything, but we'll see a bunch of other problems today from here.
And then reduce from that known problem to your problem that you're trying to prove is NP-hard.
If you can do that, you know your problem is NP-hard.
So we only need one reduction for each hardness result, which is nice.
And this picture is a collection of reductions.
We're going to start from 3SAT.
I'm not going to prove that it's NP-complete, although I'll give you a hint as to why that's true.
We're going to reduce it to Super Mario Brothers.
We're going to reduce it to three dimensional matching.
We're going to reduce three dimensional matching to subsets sum, to partition, to rectangle packing, to jig saw puzzles.
And we're going to do all those reductions, hopefully.
And that's proving NP-hardness of all those problems.
They're also all in NP.
So 30 second intuition why 3SAT is NP-hard.
Well, if you have any problem in NP, that means there is one of these nondeterministic polynomial time algorithms, or there is some verifier given a polynomial size certificate.
So that verifier is just some algorithm.
And software and hardware are basically the same thing, right?
So you can convert that algorithm into a circuit that implements the algorithm.
And if I have a circuit with like ANDs and ORs and NOTs, I can convert that into a Boolean formula with ANDs, ORs, and NOTs.
Circuits and formulas are about the same.
And if I have a formula-- fun fact, although this is a little less obvious-- you can convert it into this form, an AND of triple ORs.
And once you've done that, that formula is equivalent to the original algorithm.
And the inputs to that verification algorithm, the certificate, are represented by these variables, the xi's.
And so deciding whether there's some way to set the xi's to make the formula true is the same thing as saying is there some certificate where the verifier says yes, which is the same thing as saying that the problem has answer yes.
So given an NP algorithm, one of these nondeterministic funny algorithms, we can convert it into a formula satisfaction problem.
And that's how you prove 3SAT is NP-complete.
But to do that can take many lectures, so I'm not going to do the details.
The main annoying part is being formal about what exactly an algorithm is, which we don't do in this class.
If you're interested, take 6.045, which is some people are actually in the overlap this semester.
Cool.
Let's do some reductions.
This is where things get fun.
So we're going to start with reducing 3SAT to Super Mario Brothers.
So how many people have played Super Mario Brothers?
Easy one.
I hope if you haven't played, you've seen it, because we're going to rely very much on Super Mario Brothers physics, which I hope is fairly intuitive.
But if you haven't played, you should, obviously.
And we're going to reduce 3SAT to Super Mario Brothers.
Now this is a theorem by a bunch of people, one MIT grad student, myself, and a couple other collaborators not at MIT.
And of course this result holds for all versions of Super Mario Brothers so far released, I think.
The proofs are a little bit different for each one, especially Mario 2, which is its own universe.
What I'm going to talk about the original Super Mario Brothers, NES classic which I grew up with.
Now the real Super Mario Brothers is on a 320 by 240 screen.
It's a little bit small.
Once you go right, you can't go back left, except in the maze levels anyway.
So I need to generalize a little bit.
Because if you assume that the screen size of Super Mario Brothers is constant, in fact you can dynamic program your way through and find the optimal solution in polynomial time.
So I need to generalize a little bit to arbitrary board size, arbitrary screen size.
So in fact, my entire level will be in one screen, no scrolling.
Never mind this is a side scrolling adventure.
And so that's my generalized problem.
And I claim this is NP-hard.
If I give you a level and I ask you, can you get to the end of this level?
That problem is NP-hard.
Also no time limit.
The time limit would be OK, but you have to generalize it.
Instead of 300 seconds or whatever, it has to be an arbitrary value.
So how are we going to do this?
We're going to reduce from 3SAT to Super Mario Brothers.
So that means I'm given-- I don't get to choose.
I'm given one of these formulas.
And I have to convert it into an equivalent Super Mario Brother instance.
So I have to convert it into a level, a hypothetical level of Super Mario Brothers.
Given a formula, I have to build a level that implements that formula.
So here's what it's going to look like.
I'm going to start out somewhere.
Here's my drawing of Mario.
Mario-- or you could play Luigi.
It doesn't matter.
First thing it's going to do is enter a little black box called a variable.
This is supposed to represent, let's call it x1.
And so it's some black box.
I'm going to tell you what it is in a moment.
And it has two outputs.
There's the true output and the false output.
And the idea is that Mario has to choose whether to set x1 to true or false.
Let me show you that gadget.
So here's the-- whoops, upside down-- here is the variable gadget.
So here's Mario.
Could enter from this way or that way.
We'll need a couple of entrances in a moment.
And then falls down.
Once Mario is down here, if you check the jump height, you cannot get back up to here.
So this is like a one way.
Once you're down here, you have a choice.
Should I fall to the left or fall to the right?
And if you make these falls large enough, once you fall, you can't unfall.
So once you make a choice of whether I leave on the true exit or the false exit, that's a permanent choice.
So you can't undo it, unless you can come back to here.
But we'll set up so that never happens.
I mean, if you're trying to solve the level, you don't know which way to go.
You have to guess.
Can I go fall to the left or fall to the right, or do something.
So the existence of a play through, this level, is the same as saying there is a choice for the x1 variable.
Now we have to do this for lots of variables.
So there's x2 variable, x3 variable, and so on.
Each one has a true exit and a false exit.
So the actual level will have n instances of this if we have n variables.
Now, what do I do once Mario decides that this is a true thing?
What I'm going to do is have-- this is called a gadget by the way.
In general, most NP-hardness proofs use these things called gadgets, which is just saying, we take various features of the input, and we convert them into corresponding features on the output.
So here I'm taking each variable, x1, x2, x3, and so on, and building this little gadget for each of those variables.
Now the other main thing you have in 3SAT are the clauses.
We have triples of variables or their negations.
They have to come together and be satisfied.
One of them has to be true.
So down here I'm going to have some clause gadgets, which I will show you in a moment.
OK, and I think I'll switch colors.
This is about to get messy.
So the idea is that some of the clauses have x1 in them.
The true version of x1, not x1 bar.
So for those clauses, I want to connect.
I'm going to dip into the clause briefly.
So from this wire going to dip into the clause here.
And then I'm going to go to the next clause that has x1.
Maybe it's this one, and the next one, and so on.
All the clauses that have x1 in it, I dip into.
The other ones I don't.
And then once I'm done, I'm going to come back and feed into x2.
Next, I look at this false wire for x1.
So all the clauses that have x1 bar in them, I'm going to connect.
So I don't know which ones they are.
Maybe this one, or this one, something.
And then I come here.
And so the idea is that Mario makes a choice whether x1 is true or false.
If x1 is true, Mario is going to visit all of the clauses that have x1 true in them.
And then it's going to go to the x2 choice.
Then it's going to choose whether x2 is true or false, and repeat.
Or Mario decides x1 should be false.
That will satisfy all the clauses that have x1 bar in them.
And then again, we feed back into x2.
So this is why we have two inputs into the x2 gadget.
One of them is when the previous variable was true.
The other is when the previous variable was false.
The choice of x2 doesn't depend on the choice of x1.
So they feed into the same thing.
And you have to make your choice.
So far, so good.
Now the question is, what's happening in these clauses.
And then there's one other aspect, which is after you've set all of the variables, at the very end, after this last variable xn, at the very end, what we're going to do is come and go through all the clauses.
And then this is the flag.
This is where you win the level.
Sorry, I drew it backwards.
But the goal is for Martin to start here and get to here.
In order to do that, you have to be able to traverse through these clauses.
So what do the clauses look like?
This is a little bit more elaborate.
So here we are.
This is a clause gadget.
So there are three ways to dip into the clause.
It's actually upside down relative to that picture, but that's not a problem.
So if Mario comes here, then he can hit the question mark from below.
And inside this question mark is an invincibility star.
And the invincibility star will come up here and just bounce around forever.
We checked.
The star will just stay there for as long as you let it sit.
Unfortunately, all of these are solid blocks, so Mario can't actually get up to here to get the star.
But as long as Mario can visit this question mark or this question mark or this question mark, then there will be at least one star up here.
So the idea is that each of these represents one of the literals that's in the clause.
And if we choose-- so let's look at this first clause, x1 or x3 or x6 bar.
So if we choose x1 to be true, then we'll follow the path and we'll be able to hit the star.
Or if we choose x3 to be true, then we'll come in here and hit this star.
Or if we choose x6 to be false, then that path will lead to here and we'll be able to hit this question mark and get the star up here.
So as long as we satisfy the clause, there will be at least one star.
Won't help if you have multiple stars.
Then the final traversal part-- so that was this first clause.
And now we're traversing through.
Actually in this picture, it's left to right.
Just turn your head.
And so now Mario is going to have to traverse this gadget from left to right on this top part.
And if Mario comes in here and you can barely jump over that.
If there's a star, you can collect the star and then run through all of these flaming bars of death.
If there's no star, you can't.
You'll die if you try to traverse.
So in order to be able to traverse all these clauses, they must all be true.
And them all being true is the same is their AND being true.
So you will be able to survive through all these clauses if and only if this formula has a satisfying assignment.
The satisfying assignment would be given to you by the level play.
The choices that Mario makes in this gadget will tell you whether each variable should be true or false.
So to elaborate just a little bit more in general, when you have a reduction like this, to prove that it actually works, you need to check two things.
You need to check that if there is a way to satisfy this formula, then there is a way to play this level.
And then conversely you need to show that if there's a way to play this level, then the formula has a satisfying assignment.
So for that latter part, in order to convert a level play into a satisfying assignment, you just check which way Mario falls in each of these gadgets, left or right.
That tells you the variable assignment.
And because of the way the clauses work, you'll only be able to finish the level if there was at least one star here.
And stars run out after some time.
So you can barely make it through all the flaming bars of death.
Then you get to the next clause.
You need another star for each one.
Conversely, if there is a satisfying assignment, you can actually play through the level, you just make these choices according to what the satisfying assignment is.
So either way it's equivalent.
We always get a yes or no answer here whenever we get a corresponding yes or no answer to the 3SAT process.
You also need to check that this reduction is polynomial size.
It can be computed in polynomial time.
So there's an issue.
Given this thing, you have to lay this out in a grid and draw all these wires.
And there's one problem here, which is, these wires cross each other.
And that's a little awkward, because these wires are basically just long tunnels for Mario to walk through.
But what does it mean to have a crossing wire?
Really, if Mario's coming this way, I don't want them to be able to go up here.
He has to go straight.
Otherwise this reduction won't work.
So I need what's called a crossover gadget.
And everywhere here I have a crossing, I have a crossover.
And this gadget has to guarantee that I can go through one way or the other way, but there's no leakage from one path to the other path.
Actually, if I first traverse through here, and then I traverse through here, it's OK if I leak back.
Because once I visit a wire, it's kind of done.
But I can't have leakage if only one of them is traversed.
So this is the last gadget, the most complicated of them all.
So this took a while to construct, as you might imagine.
So this is what we call a unidirectional crossover.
You can either go from left to right or from bottom to top, but you cannot go from bottom to right or bottom to left or left to bottom, that kind of thing.
So I'm told that Mario is only going to enter from here to here, because all of these wires, I can make one way wires.
I only have to think about going in a particular direction.
I can have falls to force Mario to only go one way along these wires.
And so let me show you the valid traversals.
Maybe the simplest one is from here.
So let's say Mario comes in here, falls.
So I can't backtrack, can jump up here.
And then if Mario's big, he can break this block, break this block.
But if he's big-- there should be a couple more zig zags here.
Let's try to run.
You can crouch slide through here.
But then you'll sort of lose your momentum, and you won't be able to go through all these traversals as big Mario.
So you can break these blocks and then get up to the top and leave.
Or, if big Mario comes from over this way, you can first take a damage, become small Mario.
Then you can fit through these wiggly blocks.
But you cannot break blocks anymore as small Mario.
So once you've committed to going small, you have to stay small, until you get to here.
And then there's a mushroom in this block.
So you can get big again, and then you can break this block and leave.
But once you're big, you can't backtrack because big Mario can't fit through these tiny tubes.
See it clear, right?
So slight detail, which is at the beginning, we need to make Mario big-- so there's a little mushroom.
I think they have three spots-- at the beginning.
And also at the end, there has to be something like this that checks that you actually have a mushroom.
So the only time you're allowed to take damage is briefly in this gadget you take damage.
If you tried to backtrack, you would get stuck.
There's a long fall here.
And then you have to get the mushroom so you can escape again.
So at the end there's like a mushroom check.
Make sure you have it.
So most of the time Mario is big.
And just in these little crossovers you have to make these decisions.
This would make a giant level, but it is polynomial size, probably quadratic or something.
Therefore Super Mario Brothers is NP-hard.
So if you want more fun examples like this, you should check out 6.890, the class I taught last semester, which has online video lectures, soon to be on OpenCourseWare.
So you can play with that.
Any questions about Mario?
All right, I hope you all play.
So the next topic is a problem you probably haven't heard about, three dimensional matching.
This is a kind of a graph theory problem.
We're going to call it 3DM for short.
And you've seen matching problems based on flow.
Matching problems are usually about pairs of things.
You're pairing them up, which you might call two dimensional matching.
That can be solved in polynomial time.
But if you change two to three and you're tripling things up, then suddenly the problem becomes NP-complete.
So it's a useful starting point, similar to 3SAT.
So you're given a set X of elements, a set Y of elements, a set Z of elements.
None of them are shared.
But more importantly, you are given a bunch of triples.
These are the allowable triples.
So we'll call the set of allowable triples T. And so we're looking at the cross product.
This is the set of all triples X, Y, and Z, or X is in X, Y is in Y, and Z is in Z.
But not all triples are allowed.
Only some subset of triples is allowed.
And your goal is to choose among those subsets-- sorry, among those triples a subset of the triples.
So we're trying to choose a subset S of T such that every element-- so the things in X, Y, and Z are called elements.
So I'm just taking somebody in the union XYZ.
It should be in exactly one triple s in big S. This is a little weird, but you can think of this problem as you have an alien race with three genders-- male, female, neuter I guess.
Those are the X, Y, and Z's.
There's an equal number of each.
And every triple reports to you whether that is a compatible matching.
Who knows what they're doing, all three of them?
So you're told up front-- you take a survey.
There's only n cubed different triples.
For each of them they say, yeah, I'd do that.
So you were given that subset.
And now your goal is to permanently triple up these guys.
And everybody wants to be in exactly one triple.
So it's a monogamous race, imagine.
So everybody wants to be put in one triple, but only one triple.
And the question is, is this possible?
This is three dimensional matching.
Certainly not always going to be possible, but sometimes it is.
If it is, you want to answer yes.
If it's not possible, you want to answer no.
This problem is NP-complete.
Why is it in NP?
Because I can basically guess which elements of T are in S. There's only at most n cubed of them.
So for each one, it is guess yes or no, is that element of T in S?
And then I check whether this coverage constraint holds.
So it's very easy to prove this is in NP.
The challenge is to prove that it's NP-hard.
And we're going to do that, again, by reducing from 3SAT.
So we're going to make a reduction from 3SAT to three dimensional matching.
Direction is important.
Always reduce from the thing you know is hard and reduce to the thing you don't know is hard.
So again, we're given a formula.
And we want to convert that formula into an equivalent three dimensional matching input.
So the formula has variables and clauses.
For each variable, we're going to build a gadget that looks like this.
And for each clause we're going to build a gadget.
So here's what they look like.
If we have a variable x1, we're going to convert that into this picture.
Stay monochromatic for now.
Looks pretty crazy at the moment, but it's not so crazy.
This is not supposed to be obvious.
You have to think for a while.
It's a puzzle to figure out this kind of thing.
But I call this thing a variable gadget because locally-- so there's basically a wheel in the center here.
And then there's these extra dots for every pair of dots, consecutive pairs of dots in a wheel.
And what I've drawn is the set of triples that are allowed.
There's tons of other triples which are forbidden.
The triples that are in T are the ones that I draw as little triangles.
And two color them because there are exactly two ways to solve this gadget locally.
Now these dots are going to be connected to other gadgets.
But these dots only exist in this gadget, which means they've got to be covered.
They've got to be covered exactly once.
So either you choose the blue triangles, or you choose the red triangles.
Each of them will exactly cover each of these guys once.
You cannot mix and match red and blue, because you either get overlap if you choose two guys that share a point, or you'd miss one.
If I choose like this blue and this red, then I can't cover this point because both of these would overlap those two.
And over here you have to choose [INAUDIBLE] triples.
They can't overlap at all.
And everybody has to get covered.
So just given those constraints, locally you can see you have to choose red or blue.
Guess what?
One of them is true, the other one is false.
Let's say that red is true and blue is false.
In general, when you're trying to build a variable gadget, you build something that has exactly two solutions, one representing true, one representing false.
Now how big do I make this wheel?
Big enough.
You could make it as big as the number of clauses.
I'm going to make it into two and x1.
So wheel-- and this number is the number of occurrences of x1 in the formula.
So this is the number of clauses that contain either xi or xi bar.
That's in xi.
I'm going to double that.
Because what I get over here is basically xi being true for those guys.
Actually, yeah, that's actually right.
It looks backwards.
And false for these guys.
One way or the other, we'll figure it out.
So in order for xi to appear in, say, five different clauses, I want five of the true things and five of the false things.
And so I need to double in order to get-- potentially I have twice as many as I actually need, but this way I'm guaranteed to have false or true, whichever I need.
In reality I have some true occurrences.
I have some false occurrences, some x1's, some x1 bars.
This will guarantee that I have enough of these free points to connect into my clause gadgets.
How do I do a clause gadget?
It's actually really easy.
So these would be pretty boring by themselves.
So a clause always looks like this.
Maybe there's some negations.
Yeah, let's do something like that.
I'm going to convert it into a very simple picture.
It's going to be xi dot, and xj bar dot, and xk dot.
And then-- well maybe I'll stick to these colors.
Again, these two points only appear in this clause gadget.
These dots are actually these dots.
So there's one of these pictures for x1.
There's another one for x2, x3.
And so xi has one of these wheels.
I want this dot to be one of these dots of the wheel.
And then I want this dot to be one of the dots in the xj wheel with the false setting, one of the red dots.
I want this one to be xk true setting in the xk wheel.
So these things are all connected together in a complicated pattern.
But the point is that within this gadget, I only have three allowed triples.
And these points only appear in this gadget, which means they have to be covered in this gadget.
They can be covered by this triple or this triple or this triple.
But once you choose one, you can't choose the others.
What this means is if I set x1 to be true, it leaves behind these points marked true.
If I choose the red things, then it's the blue points that are left behind.
Leaving points behind in this case is going to be good, because this clause, in order to satisfy this clause, in order to choose one of these three triples, at least one of these must be left behind by the wheel.
If all of these are covered by their wheels, then there's no way.
I can't choose any of these guys.
But if at least one of these is left behind by the wheel, then I can choose the corresponding triple and cover these points.
So I'll be able to cover these points if and only if at least one of these is true.
And that's a clause.
That's what a clause is supposed to do in 3SAT.
If at least one of these is true, then the clause is satisfied.
I need all the clauses to be satisfied because I need to cover of these points for all the instances of these clauses.
And that's how it works.
Now, slight catch.
If you do this, not all the points will be covered, even so.
Maybe all of these are true.
And so they're all left behind.
And I can only cover one of them with the clause.
It's a little messy.
You need another gadget, which is called garbage collection.
I don't want to spend too much time on it.
But you have two dots.
And then you have every single xi-- these dots, all true and false dots.
And you're going to have this triple, and this triple, and this triple, and this triple, and so on.
It looks an awful lot like a clause.
But this is like a clause that's connected to everybody in the entire universe.
And you repeat this the appropriate number of times, which is something like sum of nx minus the number of clauses.
OK, why?
Because if you look at a wheel, it has size 2 times nx for a variable x.
And half of the points will be left uncovered.
So that means nx of them will be uncovered.
Then the clause, if everything works out correctly, the clause will cover exactly one of those points.
So for each clause we cover one of the points.
That means this difference is exactly how many points are left uncovered.
And so we make this gadget exactly that many times.
And it's free to cover anybody.
So whatever is left over, this garbage collector will clean up.
And if we use exactly the right number of them, this garbage collector won't run out of things to collect.
So this makes the proof messy.
But I want to move on to somewhat simpler proofs and for other problems.
Yeah?
Real quick, what about the t or f points that we didn't cover because we didn't actually need that many?
Right.
So this also includes the points that weren't even connected to clauses.
I think this is the right number no matter what, because this is counting the total number of uncovered guys, whether they're connected to clauses or not.
Each clause will, in a satisfied situation, it will cover exactly one of those points.
The ones that are connected to the clauses won't be covered at all, but that will still be in this difference.
So yeah, it's good to check that.
The first time I wrote this down I forgot about those points and got it wrong.
But I think this is right, hopefully.
I did not come up with this proof.
Garey and Johnson I think-- or no.
This is-- I forgot.
Yeah, this is a Garey and Johnson proof.
There's a cool book from the late '70s by Garey and Johnson, does a lot of NP-completeness, if you're curious.
All right, so hopefully you believe three dimensional matching is hard.
Now I'm going to use it to prove that some very different types of problems are hard.
This is a kind of graph theory problem.
You'll see more graph theory problems in recitation.
This one, I can erase 3SAT and Mario.
So in the world, most NP-hardness proofs are reductions from 3SAT, or some variation of 3SAT.
In some sense, you can think of three dimensional matching as kind of like a version of 3SAT, but it's a little bit more stringent.
And that stringency helps us to do other reductions.
So here's another problem where we'll reduce from three dimensional matching.
It's called subset sum.
So you're given n integers, a1 up to an.
And you're given a target sum, also an integer.
Call it t. What you'd like to know is, is there a subset of the integers that adds up to that target.
Can you choose a sum of the integers so that-- I'll write it the sum of S. But what this means is the sum over the ai's that are in S of the value ai.
I want that to equal t. So this is the definition.
This is the constraint.
So I give you a bunch of numbers.
Do any subset of them add up to t?
That's all this is asking.
This problem is NP-hard.
It's NP-complete, in fact, when you can guess which integers should go in the subset, and then add them up to see if you got it right.
It is NP-hard, but it's something special we call weakly NP-hard.
And why don't I come back to the definition of that in a moment?
Let me first show you the proof.
It's actually really easy now that we have this three dimensional matching problem.
It's pretty cool.
So these numbers are going to be huge.
What we're going to say is, let's view-- so again, we're given a three dimensional matching instance.
Get the directions, right?
We're given a set of triples.
We want to solve this problem by reducing it to a subset sum.
So we get to construct integers that represent triples.
That's what we're going to do.
So here we go.
We get to choose a number.
So I'm going to think of them in a particular base, b, which is going to be 1 plus the max of the mxi's.
So again, this is the number of occurrences of variable xi in a true or false form.
So I take the maximum occurrence of any variable, add 1.
That's my base.
It just has to be large enough.
And this is basically the entire reduction, is one line.
If I have three triples-- if I have a triple xi, xj, xk, I'm going to convert that into a number that looks like this where the one positions are-- I don't really know the order, but they are i, j, and k. Everything else is zero.
And this is in base b, not base 2.
It's a little weird.
All my digits are 0 or 1, but I'm in base b.
And three of the digits are 1.
And the rest are zero.
Why?
Because of my target sum.
Target sum is going to be 1111111111.
So this number, in algebra, you're write this as b to the i plus b to the j plus b to the k. This you would write as the sum of b to the i for all i.
Do you see why this works?
It's actually really simple.
For this instance, my goal is to choose a subset of these numbers that add up to this number.
How could that possibly happen?
Well, I've got to choose-- every time I choose one of the numbers, those three digits get set to 1 in my sum.
If I ever have a collision, if I add two 1s together, I'm going to get a 2.
That's not good, because once I get a 2, I'll never be able to get back to a 1, because my base is really big.
This base is designed so that the total-- this is the total number of colliding 1s.
So we set it one larger than that, which means you'll never get a carry when you're adding up in this base.
That's why I set the base to be something large, not base 2.
Base 2 might work, but this is much safer.
So what that means is for each of these 1s in the target sum, I've got to find a triple that has those 1s.
And those triples can't overlap.
So that means choosing a set of numbers that add up to this is exactly the same as choosing a set of triples that covers all of the elements.
Done, super easy once you have the right problem.
OK, good.
Now why do I call this weekly NP-hard?
Because these numbers are giant.
If I have n elements in X, Y, Z over there-- I guess here they're called xi, yk, zk.
Sorry, maybe I should've called them that here.
Doesn't matter.
If I have n of those elements in X union Y union Z, the number of digits here is n. So the number of digits in order n. This is fine from an NP-completeness standpoint.
This is polynomial size.
The number of digits in my numbers is a polynomial.
And this base is also pretty small.
So if you wrote it out in binary, it would also be polynomial.
So just lost a log factor.
But the size of the numbers, the actual values of the numbers, is exponential.
With weak NP-hardness, that's allowed.
With strong NP-hardness, that's forbidden.
In strong NP-hardness, you want the values of the numbers to be polynomial.
So in this case, the number of bits is small, but the actual values are giant, because you have to exponentiate.
It would be cool.
And this problem is only weakly NP-hard.
Maybe you actually know a pseudo-polynomial time algorithm for this.
It's basically a knapsack.
If these numbers have polynomial value, then you can basically, in your subproblems in dynamic programming, you can write down the number t and just solve it for all values of t. And it's easy to solve it in polynomial time, polynomial in the integer values.
So we call that pseudo-polynomial, because it's not really polynomial.
It's not polynomial in the number of digits that you have to write down the number.
It's Polynomial in the values.
Weak NP-hardness goes together with pseudo-polynomial.
That's kind of a matching result.
Say look, pseudo-polynomial is the best you can do.
You can't hope for a polynomial because if you let the numbers get huge, then the problem is NP-complete.
But if you force the numbers to be small, this problem is easy.
So subset sum is a little funny in that sense.
Cool.
Let me tell you about another problem, partition.
So partition is pretty much the same set up.
I'm given n integers.
Let's say they're positive.
And I want to know, is there a subset-- I'm not given a target sum t. Target sum is basically forced.
What I would like is the sum of all the values in S to equal the sum of all the values not in S. That's A minus S, which in other words is going to be the sum of all values in A divided by 2.
So this is called partition because you're taking a set, you're splitting it into two halves of equal sum.
Every element has to go in one of the two halves.
And they're called S and A minus S, like cuts in the flow stuff.
And you want those two halves to have exactly the same sum, which means they will be the sum divided by 2.
So that better be even, otherwise it's not going to be possible.
So again, you want to decide whether this is possible or impossible, yes or no.
I claim this problem is also weakly NP-complete, and we can reduce from subset sum to partition.
This is a little interesting because partition is actually a special case of subset sum.
It is the case where t equals this.
Subset sum, you're trying to solve it no matter what t is.
t is a given input.
So there's more instances over here.
Some of them, some of these instances are the case where t equals the sum over 2.
Those are partition instances.
So this is like a subset of the possible inputs as over there, which means this problem is easier than this one-- no harder anyway.
In other words, I can reduce partition to subset sum.
I just compute this value and set that to t, and then leave the a's alone.
That will reduce partition to subset sum.
But that's not the direction I want.
I want to reduce from subset sum, a problem I can prove is NP-complete, to partition, because I want to prove that partition is NP-complete.
So in this case, there's an easy reduction in both directions.
This direction is a little harder.
So reduction from subset sum.
So I'm given a bunch of ai's.
I'm not going to touch them.
And I'm given a target sum t. And I basically want to make that target sum into this half.
To do that, I'm going to add two numbers to my set.
So I'm going to let sigma be the sum of the given a's.
And then I'm going to add-- so I'm given a1 through an.
I'm going to add an plus 1, is going to be sigma plus t. And I'm going to add an plus 2 to be 2 sigma minus t. Why?
So these are two basically huge numbers.
Because sigma is bigger than-- I mean, it's the sum of all the numbers, so it's bigger than all of them.
And so imagine for a moment that I put these two in the same side of the partition.
I put them both in S or I put them both out of S. Their sum by themselves is 3 sigma.
The t's cancel.
Whereas all the other items, their sum is sigma.
So I'm host.
If I have 3 sigma on one side and sigma on the other, I'm not going to make them equal.
So in fact, these two elements have to be on opposite sides.
So there's a side that has sigma plus t. There's a side has 2 sigma minus t. And then there's all the other n items, and some of them are going to go to this side, some of them are going to go to this side.
Their total value is sigma.
Right now this is close to sigma.
This is close to 2 sigma.
So they have to kind of meet in the middle.
In fact, what you'll have to do is add sigma minus t over here and add t over here.
Think about it for a second.
If I add sigma minus t, this comes out to 2 sigma.
If I add t to this, this comes out to 2 sigma.
That would be good because they're equal.
And notice that this is sigma minus t. This is t. Their sum is sigma.
So in fact, it has to be like this.
You add something over here, and sigma minus something over here for all the other ai's.
And the something has to be t in order for these two values to equalize.
So in order to solve this slightly larger partition problem, you have to actually solve the subset sum problem because you have to construct a subset that adds up to t. t was an arbitrary given value.
So this is pretty nifty.
We're adding some values so that the new target sum is the 50/50 split when we're given some values that have an arbitrary target sum.
So partition is weakly NP-complete.
Let me go to rectangle packing.
So rectangle packing-- I'm going to draw a picture.
I give you a bunch of rectangles of varying sizes.
And I give you a target rectangle.
Let's call it T. These are the Ri's.
I want to put these rectangles into this picture without any overlaps.
Each of these rectangles here corresponds to one of the rectangles over here.
So I'll tell you that the sum of the areas of these rectangles is equal to the area of T. And the question is, can you pack those rectangles into T without any overlaps, and therefore without any gaps, because the areas are exactly the same.
I claim this problem is weakly NP-hard-- I guess NP-complete by reduction from partition.
This will be super easy if you followed what the definition of partition is.
We're given some integers ai.
And we're going to take each of them and convert them into a, let's say, 1 by 3ai rectangle.
Three is to avoid some rotation we'll see.
And then we're also given the targets.
Oh no, target sum is given.
Target sum is the sum over 2.
But anyway, we're going to build our target rectangle to be-- it's actually going to be really big.
It's going to be 2 by 3 times t. So this is that thing.
So this is 3/2 sum of the a's.
OK, that's about it.
In order to pack these rectangles into here, because each of them is at least three long, you cannot pack them vertically.
They have to be horizontal.
So in fact what your packing will look like is they'll be the top half and the bottom half.
And the top half, the total length of those rectangles has to add up to 3/2 sum of A.
Everything was scaled up by 3, so that's 1/2 of A on the top and the bottom.
That's a partition.
In order to pack the rectangles into here, you have to solve the partition problem, and vice versa.
Easy.
OK, let me show you one more thing, jigsaw puzzles.
This is not the jigsaw puzzles you grew up on, somewhat more generalized.
So a piece is going to look something like this.
I drew them intentionally different.
So on each, you have a unit square.
Some of the sides can be flat.
Some of them can be tabs.
Some of them can be pockets.
Each tab and pocket has a shape.
And they're not in a perfect matching with each other.
So there could be seven of these tabs and seven of these pockets, all the same shape.
This is what you might call ambiguous jigsaw puzzles.
Plus, there is no image on the piece, so this is like hardcore jigsaw puzzles.
This is NP-complete.
And what I'd like to do is to simulate a rectangle with a bunch of jigsaw pieces.
So it would look something like this.
If I have a 1 buy something rectangle, I'm going to simulate it with that same something, little jigsaw pieces.
And I'm going to make these shapes only match each other.
And so for every rectangle, they're going to have a different shape.
This one will be squares.
At that point I ran out of shapes I can easily draw, but you get the idea.
Each rectangle has a different shape.
And so these have to match to each other.
You can't mix the tiles, which means you have to build this rectangle.
You have to build this rectangle.
And then if the jigsaw problem is, can you fit these into a given rectangle, then you get rectangle packing.
But this is not a valid reduction.
You can't reduce from partition.
Why?
Because these numbers are huge.
Remember, the values of the numbers in my partition instance are exponential.
So if I have a value ai and it's exponential in my problem size, and I tried to make ai have little tiles, that means a number of jigsaw pieces will be exponential in n. That's not good.
That's not allowed.
This is why weak NP-hardness is annoying.
So instead, we need a strong NP-hard problem.
This is a problem that's NP-hard even when the numbers are polynomial in value, not just in size.
And it's called 4-partition.
4-partition, you're given n integers, as usual.
Say set is A.
And you want to split those integers into n over 4 quadruples of the same sum.
So this would be the sum of A divided by n over four.
That's your target sum.
So before we had to split into two parts that had the same sum.
That was partition.
Now we have to split into n over 4 parts.
Each part will have exactly four numbers, four integers.
And they should all have the same sum.
This problem is hard even when the integers have polynomial value.
So the values are at most some polynomial in n. I won't prove it here, but it's in my lecture notes if you're curious.
It's like this proof, but harder.
You end up, instead of having n digit numbers, you have five digit numbers.
Each digit only has a polynomial in n different values.
So the total value of the numbers is only polynomial.
It's like n to the fifth or something.
Good news is that this reduction I just gave you is also a reduction from 4-partition because it's the same set up.
Again, I'm given integers.
Each integer I'm going to represent by that many tiles.
Now the number of tiles is only polynomial, so this is a valid reduction.
And again, if I have to pack all of these tiles into a rectangular board, that's exactly the same as packing these integers.
Well, I guess I should do rectangle packing again.
So this is a proof rectangle packing was weakly NP-hard.
But in fact it's strongly NP-hard.
You just change these dimensions.
You say well, I need whatever, n over 4 different parts, each of size the sum over n over 4.
You need some scale factor here.
Three doesn't work.
Use n or something-- n and n. That will prove that rectangle packing is actually strongly NP-hard because we're reducing for 4-partition instead of partition.
And then you can reduce rectangle packing to jigsaw puzzles because you have strong hardness over here.
Over here we don't have numbers.
We just have these pieces.
So whenever you convert from a number problem to a non-number problem, if you're representing the numbers in unary, which is what's going on here, you need strong NP-hardness for it to work.
Weak NP-hardness isn't enough.
Then we get jigsaw puzzles, which we know and love, are NP-complete.
That's it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started.
Thanks for coming to lecture.
Know there's a quiz coming up.
There will be a tangible benefit of attending this lecture.
And it's not Frisbees.
OK?
We'll figure it out soon.
So, two lectures on approximation algorithms.
One today and one, I guess a week and two days from today on Thursday after the break.
Eric will give that one.
So this is more of an introductory lecture.
Eric talked about NP complete problems and NP hard problems.
He talked about how you could show that problems are NP complete or NP hard.
So what happens when you discover that a problem is NP complete or NP hard?
Well, there's a variety of strategies.
You could just kind of give up, and say this is intractable.
I want a different job.
You could say that I'm just going to do the best I can without any theoretical guarantees.
I'm going to use a heuristic.
I'm going to think of the simplest, greedy heuristic.
I'm going to code it up and I'm going to move on.
Or you could do approximation algorithms.
You could say, I'm going to think up an interesting, greedy heuristic.
But I'm going to prove that this greedy heuristic, in every conceivable situation with respect to the inputs, is going to be within some factor of optimal.
Right?
And that's what we're going to do today.
We're going to take a bunch of NP complete problems, and we're going to essentially create simple heuristics with these problems, simple strategies that are polynomial time, to quote, "solve these problems."
And what does it mean to solve these problems?
Well, you know that if it's polynomial time, you're not guaranteed to get the optimum answer every time.
But you'll call it a solution-- an approximate solution-- because you're within a factor of two for every possible input.
That's one example.
Or, you have a more complicated approximation factor that we'll get to in a second, where it's not quite a factor of two.
It might be a factor of two for small size problems.
Might be a factor of 10 for larger problems.
And so on and so forth.
And then the last thing is, it'd be great if you could spend more time and get better solutions.
And those are approximation schemes.
And we'll look at approximation schemes as well.
So, just dive in each of these problems, depending on whether it's decision or optimization is NP complete or NP hard.
I'll define these problems as we go along.
But basically, the name of the game here is, grab a problem, define it, think of an interesting heuristic, do a proof.
OK?
And that's essentially what we're going to do three times.
The good news is the proofs aren't tortuous.
They're not 30-minute proofs.
And they should be pretty intuitive.
And we'll see if we can extract them out of you.
A painful extraction process.
I went to the dentist yesterday.
Not for extraction.
I should've said, I went to the dentist yesterday and I'm now going to take it out on you, right?
OK, so what's an approximation algorithm?
An algorithm, for a problem of size n, and so it's going to be parameterized.
And the approximation factor may also be parameterized.
It'd be nice if it weren't, if it were a constant factor approximation.
But sometimes you can't do that.
And in fact, sometimes you can prove that constant factor approximation algorithms don't exist.
And if they do, then p equals NP.
All right?
So it's gets very interesting.
So, in this case, approximation algorithms or schemes exist for these three problems, which is why we are looking at them today.
But you got a problem of size n. And we're going to define an approximation ratio, row of n, for any input-- if for any input-- excuse me.
The algorithm produces a solution with cost C that satisfies this little property, which says that the max of C divided by Copt divided by-- oh, sorry-- and Copt divided by C is less than or equal to row n. And the only reason you have two terms in here is because you haven't said whether it's a minimization problem or a maximization problem.
Right, so of it' a minimization problem, you don't want to be too much greater than the minimum.
If it's a maximization problem, you don't want to be too much smaller than the maximum.
And so you just stick those two things in there.
And you don't worry about whether it's min or max in terms of the objective function, and you want it to be a particular ratio.
OK?
Now I did say row of n there.
So this could be a constant or it could be a function of no.
If it's a function of n, it's going to be an increasing function of n. OK?
Otherwise you could just bound it, and have a constant, obviously.
So, you might have something like-- and we'll see one of these-- log n approximation scheme, which says that you're going to be within logarithmic of the answer-- the minimum or maximum.
But if it's a million, then if you do log of base two, then you're within a factor of 20, which isn't that great.
But let's just say if you're happy with it, and if it goes to a billion, it's a factor 30, and so on and so forth.
Actually, it could grow.
So that's an algorithm.
If these terms are used interchangeably, we'll try and differentiate.
But we do have something that we call an approximation scheme.
And the big difference between approximation algorithms an approximation schemes is that I'm going to have a little knob in an approximation scheme that's going to let me do more work to get something better.
OK?
And that's essentially what a scheme is, where we're going to take an input-- an additional input-- epsilon, strictly greater than zero.
And for any fixed epsilon, the scheme-- it's an approximation scheme as opposed to an algorithm-- is a 1 plus epsilon approximation algorithm.
And so here we just say that this is a row n approximation algorithm if it satisfies this property.
And here we have a family of algorithms that are parameterized by n in terms of run time, as well as epsilon.
And so you might have a situation where you have order n raised to q divided by epsilon running time for an approximation algorithm.
And what this means is that if you're within 10% of optimal, then you're going to put 0.1 down here.
And this is going to be an n raised to 20 algorithm.
Polynomial time!
Wonderful!
Solve the world's problems.
Not really.
I mean, n raised to 20 is pretty bad.
But it's not exponential.
Right?
So you see that there is a growth here of polynomial degree with respect to epsilon.
And it's a pretty fast growth.
If you want to go to epsilon equals 0.01, I mean, even n raised to 20 is probably untenable.
But certainly n raised to 200 is completely untenable.
Right?
So this is what's called a PTAS, which is probabilistic time approximation scheme.
And don't worry too much about the probabilistic.
It's a function of epsilon.
That's the way you want to think about it.
The run time.
And we'll look at a particular scheme later in the lecture.
But clearly this is polynomial in n, OK?
But it's not polynomial in epsilon.
All right?
So a PTAS is going to be poly in n, but not necessarily in epsilon.
And I say not necessarily because we still call it a PTAS.
We just say, fully polynomial time approximation scheme, FPTAS, if it's polynomial in both n and 1 over epsilon.
Right?
So fully PTAS is poly in n and 1 over epsilon.
So a fully PTAS scheme would be something like n divided by epsilon square.
OK?
So as epsilon shrinks, obviously the run time is going to grow because you've got 1 over epsilon square.
But it's not anywhere as bad as the n raised to 2 divided by epsilon in terms of its growth rate.
OK?
So there's lots of NP complete problems that have PTAS's and that some of them have FPTAS's as well.
And so on and so forth.
Question?
[INAUDIBLE]
So we won't get into that today.
But you can think of it as the probability over the space of possible solutions that you have.
You can distribution of inputs.
The bottom line is that we actually won't cover that today.
So let's shelve for the next lecture, OK?
So just worry about the fact that it's polynomial in n and not in epsilon for the first part.
And it's polynomial in n and epsilon for the second part.
OK?
[INAUDIBLE]
Oh I see.
So, it's a good point.
So for the purposes of this lecture, thank you so much.
I'm glad I didn't have to get into that.
Polynomial time.
Good.
Better answer.
Either way, we're not going to cover that.
All right?
Good.
So, it's polynomial.
I mean, you could have probabilistic algorithms that have this kind of behavior, of course.
But we're not going to cover that in today's lecture.
But thanks for pointing that out Eric.
So, that's our set up.
We essentially have a situation where it'd be great if we could tackle NP complete problems using this hammer, right?
Any questions so far?
All right.
Vertex cover.
So let's dive right in.
Let's talk about a particular problem, very simple problem.
What you have is an undirected graph, G(V,E).
And all we want is a set of vertices that cover all of the edges.
So a set of vertices that cover all edges.
What does it mean to cover?
It's the obvious thing.
If I have something like this.
As long as I have a vertex in my set of vertices that I'm calling a cover that touches one endpoint of an edge, we're going to call that edge covered, OK?
So, in this case it's pretty clear that the vertex cover is simply that.
Because that vertex touches all of the edges at least at one of the endpoints.
A vertex is going to touch one endpoint of an edge.
But this vertex cover that I've shaded touches every edge.
So that's a vertex cover.
If, in fact, I had an extra edge here, then I now have to pick one-- or this one of that one in order to complete my cover.
OK?
That's it.
That's vertex cover.
Decision problem, NP complete to figure out if there's a certain number that is below a certain value that do the covering.
You obviously have an optimization problem associated with that.
And so on and so forth.
So that's our simple set up for our first hard problem.
All right?
And so, just to write that out, find a subset V prime, which is a subset of capital V, such that if (U,V) is an edge of G-- belongs to E-- then we have either U belonging to V prime, or V belonging to V prime, or both.
And it's quite possible that your vertex cover is such that, for a given edge, you have two vertices that touch each of the endpoints of the edge.
And the optimization problem, which is what we'd like to do here, is find a V prime so the cardinality is minimum.
OK?
So that's it.
So, we don't know of a polynomial time algorithm to solve this problem.
So we resort to heuristics.
What is an intuitive heuristic for this problem?
Suppose I wanted to implement a poly time, greedy algorithm for this problem.
What would be the first thing that you'd think of?
Yeah, go ahead
[INAUDIBLE]
Find the max degree.
I love that answer.
It's the wrong answer for this problem.
But I love it because it sets me up for-- ta-DA!
All this work I did before lecture, OK?
All right.
So, it turns out, that it's not an incorrect answer.
It's really not the best answer in terms of the heuristic you apply to get an approximation algorithm.
So we're still in the context of an approximation algorithm, not an approximation scheme.
And what we have here is a perfectly fine heuristic that, who knows?
It might actually work better in practice than this other approximation algorithm that I'm going to talk about and prove.
But the fact of the matter is that this approximation algorithm that has, as a heuristic, picking the maximum degree continually.
And completing your vertex cover by picking the maximum degree continually is a log n approximation algorithm.
And what that means is that I can construct-- and that example is right up there-- an example where, regardless of what n is, this particular heuristic-- the maximum degree heuristic-- might be log n off from optimal.
OK?
Whereas, this other scheme that we're going to talk about is going to be within a factor of two of optimal regardless of the input that you apply.
Right?
So, you have a domination here with respect to the two approximation algorithms.
You've got one that is log n. Row n is log n, as I've defined over there.
And on the other side, you've got two, right?
So if you're a theoretician, you know what you're going to pick.
You're going to pick two.
It turns out, if you're a practitioner, you might actually pick this one, right?
But this is a theory course.
So what is going on here?
Well, this is a concocted example that shows you that a maximum degree heuristic could be as far off as log n, right?
And so if you look at what's going on here, you end up with something where you have a bunch of vertices up on top, OK?
And you end up with case k factorial vertices up on top.
So k equals three in this case.
I have six vertices up there.
I got two down here because this is 6 divided by 3, because k is 3.
And then I got 6 divided by 3 here, so that's 2 and 6 divided by 1 here.
And so that's 6.
OK?
And so these edges are set up in such a way that it's a pathological example.
And I misspoke in terms of the approximation algorithm.
I will correct myself in just a second, in terms of log n. It does grow with the size of the graph.
Well, I'll precisely tell you what this approximation algorithm is in terms of the row n factor in just a minute.
But let's just take a look at this problem here and see what happens when you apply this maximum degree heuristic, right?
And we have to take into account the fact that, if you have ties, in terms of maximum degree, you may end up doing the wrong thing.
Because you haven't defined what the tiebreak is when you have two nodes that have the same degree.
You could do the wrong thing and pick the bad node for this particular problem, right?
You have to do a worst case analysis.
So in the worst case, when you create a vertex cover using maximum degree, what is the worst case in terms of the number of vertices that we picked for this particular example?
Someone?
What is the worst case in terms of the number of vertices?
Yeah, back there
Eleven?
Eleven.
And where did you get that from?
[INAUDIBLE]
You grab all of the ones on the bottom.
Fantastic.
All right, there you go.
Could you stand up?
Whoa.
All right.
It was the dentist yesterday.
So, that's exactly right.
That's exactly right.
So what could happen is you could pick this, because that's degree 3.
Notice that the maximum degree here is 3, of any node.
Right?
So if I pick something of degree three, I'm good.
I'm in keeping with my heuristic.
I could pick all the ones at the top, right?
And then I'm done, right?
That's a good-- that's a good does solution.
That's a good trajectory.
But all I've said is, the heuristic is maximum degree.
So there's nothing that's stopping me from picking this.
And then once I pick that, I could pick this one.
And then I'm down to, once I've taken away these two, remember that now the maximum degree in the entire graph is two.
Right?
Because each of these things is losing the degree-- losing one from its degree-- as I go along.
So then I could pick this one, this one, this one, et cetera.
And so I could end up with 11.
OK?
So if you go do the math really quickly-- and this is where I'll correct what I said before-- the algo could pick all the bottom vertices.
And so the solution and the top vertices are optimal.
Top optimal.
So that's k factorial, right?
According to my parameterized graph.
That's k factorial in terms of the optimal solution for this graph.
But if I pick the ones that are the bottom, then it's k factorial divided by k, plus 1 over k minus 1, plus da da da da, plus 1.
Which is our harmonic number.
And that's approximately k factorial log k, OK?
And this is where I misspoke.
I kept saying log n, log n. But that's not completely correct.
Because if I think of n as being the size of the input, k factorial is n, right?
And so if you see that I have log k here, then remember that this is log k where k factorial equals n. So this is another log factor, roughly speaking.
Right?
So think of it approximately as log log n approximation, OK?
Which is pretty good.
But it does grow with n, right?
The point is this does grow with n. So it's not the best approximation scheme that you can think of.
Because the approximation factor grows with the size of your problem.
So it'd be great if you could come up with a constant factor approximation scheme that would beat this one, certainly from a theoretical standpoint, right?
But this one, maximum degree, chances are, if you're a practitioner, this is what you'd code.
Not the one I'm going to describe.
OK?
But we're going to analyze the one I've described.
I've just shown you that there is an example where you have this log k factor.
We haven't done a proof of the fact that there's no worse example than this one.
OK?
So I'm just claiming, at this point, that this is at best, a log k approximation algorithm.
We haven't actually shown that it is, in fact, a log k approximation algorithm.
At best, it's that.
OK?
Any questions?
All right.
So what's another heuristic?
What's another heuristic for doing vertex cover?
We did this picking the maximum degree.
Nice and simple.
But it didn't quite work out for us.
Any other ideas?
So, I picked vertices.
What else could I pick?
I could pick edges, right?
So, I could pick random edges.
It turns out that actually works better from a theoretical standpoint.
So, what we're going to do here is simply set the cover to be null.
Go ahead and set all of the edges to be E prime.
And then we're going to iterate over these edges.
I'm not even specifying the way I'm going to select these edges.
And I still will be able to do a proof of 2 approximation.
OK?
Oh!
I forgot the best part.
This is on your quiz.
That was the tangible benefit of attending the lecture.
So copy that down.
So this is very simple.
It's not a complicated problem.
This is not simple heuristics that are going to be particularly complicated.
You just do some selections, and then you iterate over the graph.
And you take away stuff from the graph.
Typically, you take away vertices as well as edges.
And you keep going until you got nothing left, right?
And then you look at your cover, and you say, what is the size of my cover?
And here we return C, all right?
So I won't spend any time on that.
You can read it.
It's simple iterative algorithm that fixed edges randomly and keeps going.
So, now comes the fun part, which is we need to show that that little algorithm is always going to be within a factor of 2 of optimal, OK?
And you can play around with this example.
In fact, in this case, you have 6 and 11.
So that's a factor of 2, of course.
So even this algorithm is better than a factor of 2.
But it won't be if I expanded the graph and increased k. But that algorithm that I have up there is always going to be within a factor of 2.
And we want to prove that, all right?
So how do we go about proving that?
We want to prove, in particular, that a prox vertex cover is a 2 approximation algorithm.
So, any ideas?
How would I prove something like this?
Where do you think this factor of 2 comes from?
Someone who hasn't answered yet.
You answered a lot of questions in the past, all right?
No?
Someone?
All right
[INAUDIBLE]
I'm sorry?
[INAUDIBLE]
That's an excellent observation.
The observation is that the edges we pick do not intersect each other.
So, I gave you a Frisbee for the wrong answer.
So for this correct one, I won't give you one?
That's fair, right?
No.
That's unfair.
Right?
There you go.
Sorry.
So, the key observation was in this algorithm, I'm going to be picking edges.
And the edges will not share vertices, right?
Because I delete the vertices once I've picked an edge, correct?
So there's no way that the edges will share vertices.
So what does that mean?
Well, that means that, I'm getting-- let's say, I get A edges.
So let A denote the edges that are picked.
So I'm going to get edges that look like that.
OK?
And I got continuality of A edges.
I know that in my vertex cover, that, obviously, I have to pick vertices that cover all the edges.
Now I'm picking edges, and what's happening, of course, is that I'm picking 2 A vertices.
So my C-- and remember, my cost was C. And I had Copt, that corresponds to the optimum cost.
And so the cost that this algorithm produces is 2 times A, right?
Make sense?
Because I'm picking vertices, we are picking edges.
There's no overlap.
And therefore, the cost is 2 times A, right?
So as long as I can now say that Copt, which is the optimum, is less than or equal to A, right?
I have my factor of 2 approximation algorithm.
So that's it.
It's a simple argument that says now show that Copt is at least A. Copt, I'm minimizing.
Copt should be at least A.
Right?
I hope I said that right before.
But I wrote that down here correctly.
So if I say that Copt is at least A, then I got my proof here of 2 approximation.
Because I'm getting 2 A back, right?
So if you go look at C divided by-- so this means, of course, that C is less than or equal to 2 Copt, if I can show-- make that statement.
And it turns out that's a fairly easy statement to argue simply because of the definition of vertex cover.
Remember that I'm going to have to cover every edge, correct?
So I'm going to cover-- need to cover every edge, including all edges in A.
A is a subset of edges.
I have to cover all the edges.
Clearly, I have to cover the A edges, which are a subset of all of the edges, right?
How am I going to cover all of the A edges that happened to all be disjoint in terms of their vertices?
I'm going to have to pick one vertex for each of these edges, right?
I mean, I could pick this one or that one.
But I have to pick one of them.
I could pick this one or that one.
And so on and so forth, right?
So it is clear that, given this disjoint collection of edges corresponding to A, that Copt is greater than or equal to A, OK?
And that's it.
So I had to cover all.
This requires, since no two edges share an endpoint, this means that I need to pick a different vertex from each edge in A.
And that implies that Copt is greater than or equal to A.
All right?
Any questions about that?
We all good here?
Yup?
Understood the proof?
So that's our first approximation algorithm where we actually had a proof.
And so, this is kind of cool.
It obviously a pretty simple algorithm.
You're guaranteed to be within a factor of 2.
It doesn't mean that that's the best heuristic you can come up with.
It doesn't mean that this is what you'd code.
But this is the best approximation algorithm that you're going to cover for vertex cover, OK?
So, what about other problems?
What's the state of the world with respect to approximation?
There's lots of NP complete and NP hard problems for which we know approximation schemes.
And we like to move towards approximation schemes slowly.
But I'd like to look at a problem that perhaps is a little more compelling than vertex cover before we get to approximation schemes.
And that's what's called set cover.
So set cover tries to cover a set with subsets.
And it's very useful in optimization where you have overlapping sets, maybe it's schedules, it's tasks, it's people getting invited to dinner, et cetera, and you want to make sure everybody gets invited.
You want to invite families, and there's overlapping families, because people have relationships.
And you want to eventually minimize the number of dinners you actually have to have.
And that's, I don't know, hopefully a motivating example.
If it wasn't, too bad.
So you do have a family of possibly overlapping subsets.
S1, S2, Sm, subset of equal to x.
So that's that big set that we have.
Such that I want to cover all of the elements.
So that's what this little equation corresponds to.
The union of all the selected SI's should equal x. I need to cover it all.
And I do want to minimize.
It's called a C. Find C subset 1, 2, m. So I'm selecting a bunch of these things.
So C is simply-- capital C here is some subset of the indices.
And the only reason I do that is to say that I want to do this while minimizing.
I wanted to I equals 1 through m while minimizing C. OK?
Let me get this right.
So this is what I have.
Find C, subset of these, such that-- I'm sorry.
There's one more.
Union of I belonging to C, SI equals x. OK?
Sorry for the mess here.
But this last line there-- so this is simply a specification of the problem.
I'm going to be given x, and I'm going to be given a large collection of subsets, such that the union of all of those subsets are going to cover x.
And now I'm saying, I want to look inside and I want to select all of them.
I want to select a bunch of these things.
You know, C, which is some subset of these indices, so 1 may not be in it.
2 may be in it.
4 may be in it, et cetera.
And such that those subsets that are in this capital C set add up to x. OK?
So pictorially, may make things clearer.
You have, let's say, a grid here corresponding to x.
So each of these dots that I'm drawing here are elements that need to be covered.
So that's my x.
And I might have S1 corresponding to that.
This is S3, right?
And S2 is this thing in the middle.
That's S2.
S5 is this one here.
And I got a little S4 over here.
And finally, I got-- let's see-- S6, yup.
Which is kind of this funky thing that goes like that.
So this thing here is S6.
All right?
OK. What's the optimum?
You got 30 seconds.
What's the optimum cover?
Yeah, go ahead.
You had your hand up
[INAUDIBLE]
Yep.
That's exactly right.
So the optimum S3, S4, S5.
All right
[INAUDIBLE]
Oh.
S3.
Yeah, S4 is this one here.
S6, S5, OK good.
S3.
And then-- oh!
You know what?
You're right.
Let's make you right.
Don't erase that.
Here you go.
So that's 3, right?
So C would be-- cardinality of C would be 3.
So it's a nontrivial problem.
It's not clear how you're going to do this.
I've got to use a heuristic.
Hard in terms of optimization.
Optimal requires exponential time, as far as we know.
And we're just going to go off and say, hey let's design an approximation algorithm, right?
So let's think of a heuristic.
What's a good heuristic?
What's a good heuristic for this problem?
I hope I haven't scared you.
What's a good heuristic for this problem?
What's the obvious heuristic?
Yeah?
[INAUDIBLE]
The largest subset.
And in this particular case, it's actually the best also in terms of theory.
So, approximate set cover-- at least the best that we're concerned about in this lecture.
And what is approximates set cover?
It's pick largest SI.
Remove all elements in SI from x, and other Sj.
So you're constantly shrinking.
And then keep doing that.
So you'll have a new problem.
And you're going to specify that new problem on every iteration, just like we did for vertex cover and we've done many a time.
If you do that over here, notice that what you end up with is picking S1, because S1 is the big boy here, in the sense that it's got six elements in it.
Right?
Up on top.
And so you'd pick approx or heuristic algo would pick S1, S4, S5, and S3 in that order.
I won't go over it.
It's not that important.
The point is it doesn't get you the optimum for this problem.
And in general, you could always concoct examples where any heuristic fails, of course, right?
Because this problem is hard.
But it's four as opposed to three.
And the big question, again, as always, is, what's the bound?
What's the bound if you applied this heuristic?
And what can you show with respect to the approximation algorithm?
What is row n here?
So that's what we have to do here, in terms of what the bound is.
And we're actually going to do an analysis here that is pretty straightforward.
It's got a little bit of algebra in it.
But if you go look at that this, it's covered in CLRS, the textbook.
But the analysis in there uses harmonic numbers, and is substantially more complicated for, in my mind, no reason.
And so we have a simpler analysis here that is simply going to be a matter of counting.
We are picking the maximum number of elements every time.
The best we can do.
It's a greedy heuristic.
We're trying to shrink our problem as much as possible.
Initially we have x.
And then we're going to get a new problem, let's call it x0 first, for the initial problem.
You're going to get a new problem, x1.
And we're maximally shrinking x1 in relation to x0, in the sense that we're going to remove as many elements as we can.
Because that is precisely our heuristic.
So the big question is, as we go from the biggest problem that we have, the original problem to smaller and smaller problems, when do we end up with nothing?
When we end up with nothing, that's when the number of iterations that corresponds to the number of SI's that we picked is going to be the collection of SI's in our solution.
And the cardinality of that is our cost, right?
So that's all pretty straightforward, hopefully.
So what we need to do, of course, is show a proof.
And the way we're going to do this is by a fairly straightforward counting argument.
Assume there's a cover Copt, since that Copt equals t. OK?
So the cardinality of Copt equals t. So I'm just assuming that this t subset's in my optimum cover, OK?
t subset's in my optimum cover.
Now let x of k be the set of elements in iteration k. And let's assume that x0 equals x.
So initially, I'm at 0.
And I want to subscript this because I want to point to each of the problems that I'm going to have as I shrink this set down to nothing.
Now I know that for all k, including of course x0, xk can be covered by t sets.
OK?
I mean, that's kind of a vacuous statement because I assumed that x0 could be covered by t sets.
And x0 is only shrinking to x1, to x2, et cetera.
And I'm just saying, all of these things could be covered-- each of those intermediate problems as well, can be covered-- by t. In fact, in the solution that we have, the optimal solution, if these x0's are coming from my heuristic.
But if they were coming from an optimum solution, then x0 would be covered by t. x1 would be covered by t minus 1.
And t minus 2, and so on and so forth.
But I don't have an optimum algorithm here.
I just have my heuristic algorithm.
And I'm just making a fairly vacuous statement, as I said, based on my definition, that the original problem can be covered using t. And therefore, a smaller problem, when I remove elements from it, could also be covered using t. OK?
So far, so good?
So now, one of them covers what?
What can I say about one of them?
What principle can I use to make a claim about the number of elements that are covered by one of these t sets?
Remember a principle from way back?
6042?
My favorite principle?
Flapping.
Think flapping.
Pigeon hole.
Pigeon hole principle, right?
See, you have to remember all of the material that you learned at MIT for the rest of your life.
OK?
You never know when it's going to be useful.
OK, so here you go, pigeon hole.
My favorite principle.
It's such a trivial principle.
So one of them covers at least xk divided by t. I mean, why is this even a principle?
Right?
Elements.
OK, so that's it, right?
That's the observation.
And, now that implies that an algorithm-- because it's going to pick the maximum of these, right?
The algorithm is going to pick the maximum of these.
So, it's going to pick a set of current size greater than or equal to xk divided by t. Otherwise, the algorithm would be incorrect.
It's not doing what you told it to do.
Right?
It's got to pick the maximum.
So keep chugging here.
And one real observation, and then the rest of it is algebra.
So, what I can say is that for all k, xk plus 1, which is shrinking, is less than or equal to 1 minus 1 over t, xk.
OK?
That's the way I'm shrinking.
Right?
This is again, a fairly conservative statement.
Because the fact of the matter is that I'm putting t in as a constant here, right?
But t is actually, in effect, changing, obviously, halfway through the algorithm.
I don't need t sets to cover the x-- whatever it is-- xk over 2 or whatever it is that I have.
I need the t for x0.
Maybe I did a bad selection with my heuristic, and I still would need t, which is optimum, remember, for x1.
But halfway through the algorithm after I picked a bunch of sets, I'm still saying I need t, OK?
Because I just need that for my proof.
That's all I need for the proof.
In CLRS, this actually varies and it turns into a harmonic series.
We won't go there.
OK?
You can do the natural logarithm of n plus 1, a proof, just doing the simpler analysis.
OK?
So you see what's happening here.
That's my shrinkage.
That's the recurrence, if you will, that tells me how my problem size is shrinking.
And when do I end?
What is my stopping point?
What's my stopping point?
Mathematically?
When do we end this lecture?
When you give me the answer.
No, not quite.
So I end when?
I got nothing to cover, right?
So when one of these things gets down to xk equals 0, right?
When xk equals 0, I'm done.
I'm constantly taking stuff away.
When xk equals 0, I'm going to be done.
OK?
And I'm going to move a little bit between discrete and continuous here.
It's all going to be fine.
But what I have is, if I just take that, I can turn this.
This is a recurrence.
I want to turn that into a series.
So I can say something like 1 minus 1 over t, raised to k, times n. And this is the cardinality of x, which is the cardinality of x0.
So that's what I have up there.
And that's essentially what happens.
I constantly shrink as I go along.
And I have a constant rate of shrinkage here.
Which is the conservative part of it.
So keep that in mind.
But it doesn't matter from an analysis standpoint.
OK?
So if you look at that, and you say, what happens here?
Well, I can just say that this is less than or equal to e raised to minus-- you knew you were going to get an e, because you saw a natural algorithm here, right?
And so, that's what we got.
And basically, that's it.
You can do a little bit of algebra.
I'll just write this out for you.
But I won't really explain it.
You're going to have Xk equals 0.
You're done.
The cost, of course, is k. Right?
The cost is k, because you've selected k subsets.
All right, so that's your cost, all right?
So, when you get to the point, you're done.
And the cost is k. So what you need is, you need to say that e raised to minus kt divided by n is strictly less than 1.
Because that is effectively when you have strictly less than 1 element.
It's discrete.
So that means you have zero elements left to cover.
That means you're done OK?
So that's your condition for stopping.
So this done means that e raised to minus kt times n is strictly less than 1.
And if you go do that, you'll get k over t, just about greater than natural logarithm of m(n).
The algorithm terminates if we just do a little manipulation.
And that implies that k over t less than or equal to natural logarithm of n plus 1 is valid.
Right?
k over t is going to be less than or equal to natural logarithm of n plus 1.
Because the instant it becomes greater, the algorithm terminates.
Right?
And that's how you got your proof over here.
Because this is exactly what we want.
k is our C from way back where it's the cost of our heuristic or the cost of our approximation.
t is the optimum cost.
That's what I defined it as.
And this is a bound on k over t. All right?
Cool.
Any questions on this?
OK.
So this approximation ratio gets worse for larger problems, just like this other approximation's algorithm that we didn't actually prove from the very first problem.
That also got worse.
We had a log k factor for that.
And as your problem size increased, obviously the approximation factor increased.
This is a little clearer as to what the increase looks like in relation to the original size of n. So it's just natural logarithm of n plus 1.
So, so far, we've done approximation algorithms, a couple of different varieties.
We had a constant factor one, and then we had a row of n that actually had a dependence on n. Now let's move, and we'll do one last example on partition, which it turns out has a trivial constant factor approximation scheme.
And this obvious thing, and we'll get to that in just a second.
But what is nice about partition is you can actually get a PTAS, right?
Polynomial time approximation scheme, and FPTAS, fully polynomial time approximation scheme, that essentially give you with higher and higher run times.
They're going to give you solutions that are closer and closer to optimal, right?
If you want to do the FPTAS, we'll do the PTAS.
So partition is a trivial little problem to define.
It's just you have a set, and you want to partition it into two sets such that they're not unbalanced.
So your cost is the imbalance between the two sets.
And you want to minimize that cost.
You'd love to have two sets that are exactly the same weight.
But if one of them is extremely unbalanced with respect to the other, then it's bad.
Either way.
So, here we go with partition.
Set S of n items with weights S1 through Sn, assume S1 greater than S2, Sn, without loss of generality.
So this is just an ordering thing.
I mean, obviously, there's some order.
I'm not even claiming uniqueness here.
I'm just saying just assume that this is the order.
The analysis is much better if you do this-- if you make this assumption.
And I want to partition into A and B to minimize max of sigma I belongs to A. Si sigma I belongs to B Si.
And this is the weight of A.
And this is the weight of B.
And so there's only so much you can do.
If you have 2L equals sigma I equals 1 through n Si-- so I'm just calling that 2L-- the sum of all the weights.
Then my optimum solution is what?
What is the lower bound on the optimum solution?
If it's 2L?
What's the trivial lower bound?
Just L, right?
So I could have L here and L there.
And if I had 2L here and 0 here, then oh!
That's right.
I want to minimize.
Remember, don't-- maybe that's what threw you off.
It threw me off right here.
I see a max here, and I got a little worried.
But I want to minimize the maximum of these two quantities, OK?
And so the best I could do is to keep them equal.
And if I get L for both of them, that would minimize the maximum of those two quantities.
If I had 2L and 0, then the maximum of those two quantities is 2L, and obviously I haven't done any minimization.
So now you see why there's a trivial optimum solution is greater than or equal to L. Right?
And now you see why there's a trivial two approximation algorithm.
Because the worst I could do is 2L, right?
I could dump all of them on one side, constant time, and the other one is 0.
And my cost is 2L.
So I'm within a factor of 2 in this problem trivially.
So, unfortunately that's not the end of the lecture, OK?
So, we've got to do better.
I mean, clearly, there's more here.
You'd like to get much closer.
This is a different kind of problem.
And it would be wonderful if we could get within epsilon.
Right?
I'm within 1%.
How long does it take me?
I'm within 0.01%, guaranteed.
How long does it take me?
Right?
That's what an approximation scheme is, as opposed to just a plain algorithm.
So if you're actually going to talk about epsilon here, and we're just doing a PTAS.
So we're going to see something that is not polynomial in 1 over epsilon.
It's polynomial in n. But not polynomial in 1 over epsilon.
But there's an FPTAS with this problem that you're not responsible for.
So this is going to be an interesting algorithm simply because we now have to do something with epsilon.
It's going to have an extra input.
It's not going to be the simple heuristic, where I'm going to do maximum degree or maximum number of elements or anything like that.
I want to take this epsilon and actually do something with it.
All right?
So how does this work?
Well, basically what happens in PTAS's, or in a bunch of them, is that you essentially do an exponential amount of work given a particular epsilon to get a partial optimum solution.
So you can think of epsilon as essentially being 1 divided by m plus 1, where m is some quantity.
And as m grows, the complexity of your algorithm is going to grow.
But obviously, as m grows, you're getting a tighter and tighter epsilon.
You're getting guaranteed closer and closer to your optimum.
And so we're going to have two phases here in this particular approximation scheme.
The first phase is find an optimal partition, A prime, B prime, of S1 through Sm.
And we're just going to assume that this exhaustive search, which looks at all possible subsets, and picks the best one.
OK?
And how many subsets are there for a set of size m?
It's 2 raised to m. So this is going to be an exponential order, 2 raised to m algorithm.
OK?
I'm just going to find the optimum partition through exhaustive search for m. Right?
m is less than n. So I'm picking something that's a smaller problem.
I'm going to seed this.
So, the way this scheme works is, I'm seeding my actual algorithm-- my actual heuristic-- with an initial partial solution.
And depending on how much work I do to create the seed, I'm going to end up having higher complexity.
And obviously that's a function of small m, or 1 over epsilon.
And so this takes-- this optimal takes order to raise to m time.
And you can think of that as order 2 raised to 1 over epsilon.
And so that's why it's a PTAS, and not an FPTAS.
OK?
So this is PTAS.
What else do we need to do?
Well, I don't actually have a solution yet.
Because if m is really small-- and by the way, m can be 0 as well.
Right?
Epsilon would then be at 2 approximation, 1 divided by-- this would be one half.
And so 1 over epsilon is 2.
So then you've got your trivial algorithm that we had, the 2 approximation scheme.
So that makes sense?
So the second phase is you're going to start with your seed corresponding to A and B.
You're going to set them to A prime and B prime.
And what I'm going to do is, for I equals m plus 1 to n, if w(a) less than or equal to w(b), A equals A union I-- running out of room-- else B equals B union I. OK?
So it's not that hard to see, hopefully.
All I'm doing here is, I'm just going in a very greedy way.
I got my initial A prime and B prime.
I set them to A and B.
And I say, oh, I got this element here.
Which one is bigger?
This one is bigger?
I'm going to put the element over here.
And then I look at it again.
I got another element.
Which one is bigger?
And I go this way, that way.
That's pretty much it.
So, again, all of these algorithms are really straightforward.
The interesting part's-- the fun part's-- in showing the approximation guarantee.
So we're good here?
Yup?
All right.
So back to analysis.
One last time.
So, let's see.
We want to show that a prox partition-- ah, you know how to spell partition-- is PTAS.
I guess I don't, but you do.
Let's assume that w(a) is greater than or equal to w(b) to end with.
Right?
So I'm just saying, at the end here, I'm just marking the one that was larger that came out of the max as A.
Right?
Without loss of generality.
Just to make things easier, I don't want to keep interchanging things.
So our approximation ratio is w(a) divided by L, right?
w(a) could at best be L if I got a perfect partition, perfectly balanced.
But it could be a little bit more.
And that's my approximation ratio, OK?
So I need to now figure out how the approximation ratio reflects on the run time, and is related to m and, therefore, epsilon.
All right?
So what I'm going to do here is, I'm going to look at a point in time where I have A and B-- and remember that w(a) is defined to be greater than w(b).
But here, I'm looking at some point in time, which is not necessarily at the very end here.
It could be.
But you can think of this as being, I'm just going to say B or intermediate B.
And this won't matter too much.
But I'm just being a little bit of a stickler here.
I'll explain why I said that in just a minute.
But the point is, I have a situation where I know that w(a) is greater than w(b) or greater than or equal to w(b) because I assumed it.
That's how I marked it.
And I'm going to look at k is the last element added to A.
It's been added.
Now this could have been added in the first phase or the second phase.
It's quite possible that for a given m, that if it's large, for example, that A prime that you end up with is your A to begin with here, and that you never execute this statement OK?
It's quite possible, right?
You got your initial seed and never added to it.
And that was it.
Because your m was large, for example, right?
So what I'm saying here is, k is the last element added to A. OK?
So there's clearly a last element.
I'm just marking that.
And we know that A is greater than or equal to B.
Now it may be true that if I'm looking at the snapshot when just after I add the k-th element to A, I may not be done yet, in terms of I still have a few elements.
And I may be adding elements to B.
But regardless, given my definition, I know that the rate of B is less than the rate of A.
Because even though the last element overall may be added to B, w(b) is less than w(a), and I'm only looking at the last element added to A here.
OK?
So why all of this skulduggery?
Well, there's a method here to the madness.
We're going to analyze what possibly happens in the first phase and the second phase, and get our approximation ratio.
Shouldn't take too long.
So there's two cases here that we need to analyze.
The first one is easy.
The second one is a little more involved.
I'm going to now assume that k was the last element and was added in the first phase, OK?
If k is added to A, what can I say?
If k was added in the first phase to A, and that's the last element added throughout the algorithm, by the time you get a partition, what can you say?
What can you say about the solution?
What strong statement can you make about the solution?
What's the only interesting strong statement that you can make about a solution?
So I got to A.
Remember what the first phase is.
What's the first phase?
Well, it's optimal, right?
So, after that, what happened to A?
Well it didn't change.
Right?
So, what you got is optimum.
Right?
Because w(a) was defined to be greater than or equal to w(b).
w(a) was optimum for the smaller problem, whatever m was.
You never added anything else to it.
So you're done.
It's optimum.
So in this case, your approximation ratio is 1 because you got the optimum solution, right?
So if k is added to A in the first place, this means A equals A prime.
We have an optimal partition.
Since we can't do better than w(a) prime when we have n greater than m items.
And we know w(a) prime is optimal for the m items.
OK?
So that's cool.
That's good.
So we got an approximation ratio of 1 there.
And remember that, this is not taking overall exponential time necessarily.
It's just a case where I've picked some arbitrary m, and it happens to be the case that A equals A prime at the end of the algorithm.
So I am taking exponential time in m-- m as in Mary-- but I'm not taking exponential time in n-- right?
n as in Nancy.
So the second part is where the approximation ratio comes in.
k is added to A in the second phase.
So here, what we're going to do is, we're going to say we know w(a) minus Sk is less than or equal to w(b).
This is the second phase we're talking about here.
The only reason Sk was added to A was because you decided that A was the side that was smaller, right, or perhaps equal.
So that's the reason you added into it.
So you know that w(a) minus Sk is less than or equal to w(b) at that time.
So think of A and B here as being variables that are obviously changing, right?
But what I'm saying here is, even if you look-- this is the algorithm where A and B, you constantly look at them and decide which way to go.
But if you look at the last step, then you look at the final values.
Then you could certainly make the statement for those final values, that the w(a), which is the resultant value, minus Sk, should have been less than or equal to w(b).
And you had a smaller w(a) to begin with.
And you added Sk to it.
And that happened in the second phase.
OK?
So this is why k was added.
This is why k was added to A. I want to be a little careful here given that we're overloading A and B without trying to point out what each of these statements actually means.
And ask questions if you're confused.
I know that w(a) minus Sk is less than or equal to 2L minus w(a).
That's just a substitution.
Because w(a) plus w(b) equals 2L.
And then, last little trick, again it's algebra.
Nothing profound here.
We have our assumption that we ordered these things.
So you had S1 through-- Sn, excuse me.
The whole thing was ordered.
And so we can say that S1, S2, all the way to Sm, is greater than or equal to Sk.
We are actually doing this in order.
We are taking the bigger elements and then deciding where they go.
So we sorted those things initially.
And so what we end up with when we look at Sk, we have taken care of values prior to Sk that are all greater than or equal to Sk.
Right?
That's, again, using our initial assumption.
So what that means is 2L is greater than or equal to m plus 1 Sk since k is greater than m. Again, this is not particularly tight.
Because m could be really pretty small in relation to n. But I do know that I can make the statement that since the values are decreasing, that 2L, which is the sum of all of those, is greater than or equal to m plus 1 times Sk, regardless of what m is.
Right?
And so, that's pretty much it.
Once you do that, you have your approximation ratio.
Let's leave that up there because that's the algorithm.
Finish this off with a little algebra.
So w(a) divided by L is less than or equal to L plus Sk divided by 2, divided by L. I'm basically substituting.
I have this and I have that.
And I'm playing around with it.
And I got 1 plus Sk divided by-- 1 plus Sk divided by 2L, which is-- now this I could say is equal.
That's simply equal.
This, I have a less than or equal to.
And then I can go less than or equal to 1 plus Sk divided by m plus 1 Sk.
And then I got 1 plus 1 divided by m plus 1, which is, of course, 1 plus epsilon.
So all I did here was use this fact and essentially relate 2L to Sk.
And once I could relate 2L to Sk, substituting for Sk in here, I ended up with the approximation ratio that I want, w(a) over L, is L plus Sk divided by 2.
That simply comes from here.
And plug that in, divided by L. And then I have Sk divided by 2L.
And then this part here, 2L is going to get substituted by m plus 1 Sk and voila.
I'm over here.
OK?
So the story behind this particular problem was it was in the quiz.
And Eric, I guess took the quiz.
Did you actually take the quiz?
Or edit the quiz?
And he says, this problem is impossible.
It was a problem.
He said, this problem is impossible.
I had to Google it to find out the answer.
Or something like that.
And I said, well, I'm going to give the answer in lecture, right?
So there you go.
So remember this.
Write it down.
Four points for coming to lecture today.
See you guys next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
All right.
Welcome to our second lecture on what to do when you have an NP-hard problem.
So two lectures ago we saw how to prove a problem is NP-hard.
Last lecture we saw if you want polynomial time but you're willing to put up with a not perfect solution, but you want to get within some factor of the best solution, that's approximation algorithms.
Today we're going to do a different thing called fixed parameter algorithms.
These are going to run an exponential time in the worst case.
But not so bad in a certain sense, which we'll get to.
In general, the theme of these last two lectures and this one is that we'd really like to solve hard problems.
We'd like to solve them fast, meaning polynomial time.
And we would like correct solutions, also known as exact solutions.
OK. We'd love to solve NP-hard problems in polynomial time exactly.
But that's not possible unless P equals nP.
So pick any two.
That's the general idea.
This is a bastardization of a joke which is-- sleep, friends, work-- pick any two.
That's the MIT motto.
Here in algorithms-- hard, fast, exact-- pick any two.
So most of this class is about these two-- polynomial time algorithms give you exact things.
That's the class P. Last lecture was about hard problems.
We drop exactness.
We still want polynomial time.
We still want to solve hard problems.
So this is approximation algorithms.
And what we're doing today is the other combination.
So we want exact, but we're going to sacrifice how fast things are.
They're not going to be polynomial time in a strict sense, but it's going to be somewhere in between polynomial and exponential.
This is an area called FPT for fixed parameter tractability.
So what's this parameter business?
In general, the idea is that we really want an exact solution to an NP-hard problem, which means it has to take exponential time in the worst case.
But we want to confine the exponential dependence to something called a parameter.
OK. We actually use parameters all the time.
For example, on a graph, there's two typical parameters you think about-- the number of vertices and the number of edges.
If you're sorting an array, the usual parameter you think about is the size of the array.
OK. That's all I mean.
A parameter, in general, is just some kind of size or complexity measure.
So in general, a parameter is going to be-- we're going to call it k of x-- should be a non-negative integer-- and x is the input.
So you're thinking about some problem, like a problem we'll be looking at today is vertex cover, which we saw in the last lecture.
Vertex cover-- you're given a graph.
And based on that graph, we're going to define some function of that graph.
So this is the input to the problem, and k is just going to be some non-negative integer, which is a function of that input.
Just some measure of how tough your problem is.
OK. And what we would like is a running time that is exponential in k, but polynomial in everything else.
Polynomial in the size of the problem, in v and E. OK.
So that's the general goal is-- polynomial in the problem size-- which we usually call n-- and exponential in the parameter-- which I'm calling-- just going to call k-- in general, you could consider more parameters, but we're just going to think of two-- the overall size of the problem, and some particular parameter that we look at called k. So if you can achieve this, which we'll call fixed parameter tractability-- I'll define it formally in a little bit, because there's more than one way you might think of defining it.
Some are right.
Some are wrong.
If you can achieve this, what you get is an exact algorithm for your problem that runs really fast provided k is small.
So this is sort of a way of saying, well, you know the problem is NP-hard in general, but as long as this measure k is reasonably small, I'm still able to solve it really fast.
So it's a way of characterizing a wide family of-- a big subset of the problem that you can solve.
You know that in general you're going to need exponential time, but this gives you a measure of how hard your input is.
May not be the only measure.
May not be the best one, in any sense.
But if you can define a parameter, and you know that in your practical scenarios that parameter will be small, then you're golden.
Then you can actually solve the problem in a reasonable amount of time and get an exact solution.
No approximation here.
So that's the idea.
So that was a parameter.
We're also going to define a parameterized problem.
This is just a problem plus a parameter.
OK.
So we already have some notions of problems.
We can take any problem that we've looked at before, like vertex cover.
And if we just define some parameter, then we get a parameterized problem when we put these things together.
And usually we would write it as something like, oh, take this problem, and then consider it with respect to this parameter.
And in general, for a single problem, there may be several natural parameters that you want to care about.
Usually there's actually one obvious parameter.
So let's do that for vertex cover.
But in general, we can talk about a problem with respect to different parameters.
And some of them may be feasible to solve in this sense.
Some maybe not.
All right.
So I'm going to define k vertex cover.
This is almost the same as vertex cover.
It just has a k in front.
But the k means that it's a parameterized problem instead of just a general problem.
So as in vertex cover, we're given a graph G. And I'm going to think of the decision version of vertex cover.
So we're given a non-negative integer k. And we want to know-- is there a vertex cover of size k-- say, less than or equal to k-- is there a vertex cover?
Remember a vertex cover is a set of vertices that cover all the edges.
And we want the size of S to be less than or equal to k. OK.
So every for every edge, we need to choose one of the two end points.
We want the total number of chosen vertices to be, at most, k. And so that's a regular decision problem.
But for parameterized problem, we also want to define a parameter function.
And that parameter function-- guess what?
k. Most obvious thing, given that I wrote the letter k here.
That's going to be our parameter.
OK. And most problems, a lot of problems, especially decision versions of optimization problems-- like before, we were minimizing the vertex cover-- this is the decision version where we want to decide whether there's one of size of most k. If you can solve this, of course, you can binary search on k, like you did in your quiz.
Hopefully.
So that's all good.
And a lot of problems have this some non-negative integer floating around.
And that's the, kind of the, obvious choice for the parameter.
Doesn't have to be the only one.
But today, we're just going to look at vertex cover with this parameterization.
In your problem set, you'll look at another problem with another natural parameter.
This is usually called the natural parameter.
But there's no formal definition of natural.
That's just intuition.
All right.
So that's the set up.
Let's do some algorithms.
I guess the first note is that k can actually be small.
Nice example is a star graph.
So you have the vertices, but what's the smallest vertex cover?
1.
Everyone's holding up one finger.
You choose this guy-- that in the center, that covers all the edges.
So it can be that k is much smaller than v, and our goal here is that we're going to get some polynomial dependence in the size of the graph, but we're going to get an exponential dependence on k. Now there are many different ways you could think of exponential dependence, but let's start with-- what would be the really obvious brute force solution to vertex cover?
OK.
I want exact.
I'm not going to be clever.
What's the obvious algorithm to solve this?
Yeah
Try any combination of k vertices, and see if it's a vertex cover
Try any combination of k vertices.
See if it's a vertex cover.
How many combinations of k vertices are there?
And choose k. Good.
Let's see.
I'm a little out of practice.
It's been awhile.
Close.
Off by one.
So try all and choose k. I guess, v choose k, subsets of k vertices.
If I wanted to match this definition exactly, I should try all subsets of less than or equal to k vertices.
But, hey, if I choose fewer than k vertices, why not add in a few extras until I get up to k. So it's enough to look at v choose k subsets, because-- subsets of size exactly k-- because that will end up giving the same answer as this question.
OK.
So for each-- test each of those choices for coverage.
So that just means we loop over-- I guess for every vertex in our set, we mark all of the incident edges as covered.
And then we go through all the edges, and see whether every one got marked covered.
If not, we reset and try the next subset.
OK.
This is like not smart dynamic programming.
You just guess what the subset is.
And see if it covers.
This is how you would prove that this problem is in NP.
Right?
But now we're actually making it an exponential algorithm.
So what's the running time of this algorithm?
Yeah
E times v to the k
E times v to the k. Good.
Must be.
So that's obviously exponential.
In a certain sense, the dependence on E and v is in the bottom, which is good.
And the k is in the exponent, which makes sense.
So this is not surprising.
We also don't think of it as good.
We've defined this to be bad.
OK.
In general, we think of a running time like n to the f of k, where n is sort of the overall problem size here.
Here n is basically v plus E. And that's the overall input size for a graph.
If we have a running time where the exponent of n depends on k, in a nontrivial way, we think of that as a bad running time.
This is a slow algorithm.
It's slow because even when k equals 2-- if you have a large graph-- this is probably not something you want to run.
Definitely when k is 10, you're completely hosed.
This is a very impractical.
And the formal sense in which it is impractical is that the exponent in n depends on k. In general, you cannot say-- so I'd like to-- I mean fixed parameter.
The whole point is to think of the parameter as being fixed, like a constant.
OK. Now if the parameter is fixed.
If you think of it as at most 100, then, indeed, this will be at most n to the 101 or something.
So it is polynomial for any fixed k. The catch is that the exponent of the polynomial depends on k. As you increase k, as you increase your bound on k, the exponent increases.
I can't say this is an n squared algorithm for any fixed k. OK.
So exponent depends on k. That's the bad case.
So the good case, we're going to define, is that the exponent doesn't depend on k. That may seem like a small change.
It is a small change.
But it's a big one.
It's a small change with a big effect.
So I'm going to define, let's say, a parameterized problem is fixed parameter tractable-- which, given how many letters that is, we're going to abbreviate to FPT-- if it can be solved in f of k times polynomial in n. OK. Because this means that the exponent here doesn't depend on anything.
The exponent of n doesn't depend on k. OK.
So for this definition-- just to be explicit-- I want the constant here to be independent-- of course, it should be independent of n, and it should be independent of k. This can be any function.
It's presumably an exponential function, because if this is an NP-hard problem, something's got to be exponential.
This clearly is not exponential.
So it's got to be here.
So this is a sense in which we're exponential in k, polynomial in n. But it's much better than this kind of running time.
OK. We can think about what-- in the sense in which it is much better once we have an actual algorithm of this type.
So let's do-- let's try to solve vertex cover in this kind of time.
I claim vertex cover is fixed parameter tractable.
There is such an algorithm.
And the algorithm is going to look familiar.
Very similar to the 2-approximation algorithm that we had last class for vertex cover.
So-- but I'm going to give it a different name, which is bounded-search-tree.
OK.
This algorithm is also going to feel like dynamic programming.
Or we're going to use guessing.
In general, exponential algorithms, naturally is guessing.
But here, when I guess, I have to try all the possibilities.
Here this was one way of trying all the possibilities.
We're going to be a little bit more sophisticated in how we try all the possibilities that actually exploits the properties of vertex cover.
First line is just like the 2-approximation algorithm.
Look at any edge in the graph.
OK.
Here it is.
From u to v. What do I know about that picture?
Yeah
One of those vertices has to be in the cover
One of those vertices has to be in the cover.
Either u or v or both are in S for that edge to be covered.
Now for the 2-approximation, we just put those both in.
Here we can't afford to do that because we want an exact solution.
So we'll try both options.
We don't know which one belongs.
Let's guess.
So we know either u is in S or v is in S. Don't know which.
So guess.
Sorry-- I should, to be clear, mention or both.
So we're going to guess, which means we need to try both options.
We're going to try putting u in, and then we're going to try putting v in.
So let's just see what happens when we try that.
So in the first guess, we say, let's put u in S. OK. Well, if we put u in S, that means we cover all of the edges incident to u.
So I'd like to use recursion.
I'd like to simplify my problem.
Get another vertex cover instance.
So in order to do that, I'm just going to delete u and all of its incident edges.
We do the similar thing in the approximation algorithm but for u and v simultaneously.
So delete u as incident edges.
Now we have a vertex cover instance.
There's one other thing.
There's a new graph we have.
But we also need to update k. Because we just used one of those-- we just added something to S, and then we deleted that from the graph.
Which means, in our new graph, effectively k has gone down by 1.
OK.
So I'll say decrement k. Now I have a new instance.
I have a new graph and a different value of k. Recurse this algorithm.
I would say-- I'll call the new graph G prime and the integer k prime.
k prime equals k minus 1.
And then the second case is do the same thing for v. I won't write the code, exactly the same, but I delete v and it's incident edges.
I still decrement k by 1.
And I recurse.
And then I just return the or of these two answers.
So if this one finds a solution, great.
I found a solution to the overall problem.
This one finds a solution, great.
Maybe both return yes.
Doesn't matter.
In general, I just take the inclusive or of those two Boolean values.
That gives me an overall yes no answer to k vertex cover.
Cool?
So next question is what the running time is.
But you can think of this as a dynamic program.
It's just, here we recurse, and we don't bother memoizing.
Because, in general, memoization will never help us here.
And you may have even thought of algorithms like this in the dynamic programming world.
And we just say, well, that's not good enough, because in dynamic programming we want polynomial time.
This is like a dynamic program, but the running time is exponential.
But it turns out it will be fixed parameter tractable.
That's the good news.
Let's think about the running time.
So if I draw-- let's draw a recursion tree.
Right?
This is a divide-and-conquer algorithm in a very weak sense.
We start up here with a problem of size n and a parameter k. And we make two recursive calls.
OK. We deleted a vertex and maybe some edges.
So let's say, we have a new problem of size something like n minus 1.
But what really saves us is that k went down by 1.
And we have two recursive calls.
Each of them k is 1 smaller.
OK. And then each of those has two recursive calls.
I don't really know what happens to n. It probably doesn't get that much smaller, but k goes down by another 1.
OK.
So I'm writing here the size of the problems and the parameters of the problems.
n minus 2. k minus 2.
OK. How much time do I spend in each of these nodes?
How much work am I doing-- non-recursive work am I doing in this algorithm?
Yeah
o of E, right?
[INAUDIBLE]
o of E. Yeah.
Certainly at most order E. Probably at most order v, because there's only at most v incident edges to each vertex.
Yeah?
Linear time.
Doesn't really matter how careful we are here, but I will say-- each of these nodes-- we spend, at most, let's say, order v time.
OK.
It happened that v went down by 1.
As you can see at these levels.
But certainly an upper bound is the original v. In each of these nodes, we spend at most the original v. When does this recursion stop?
I didn't write a base case.
Help me out.
What's a good base case for this algorithm?
Yeah
When k equals 0, check if there are any edges
When k equals 0, check if there any edges.
When k equals 0, I can't put anything into my vertex cover.
So if there any edges, they're not going to be covered.
That's bad news.
OK.
So over here we have base case, k equals 0, check-- or let's say, return whether size of E is not 0.
If it's not 0-- sorry-- whether it equals 0-- get it right-- if it equals 0, then the answer is yes.
There's a vertex cover.
I can cover all of those 0 edges using 0 vertices.
That's good.
But when E does not equal 0, there's no way I can cover that non-zero number of edges using 0 vertices in my vertex cover.
OK.
So that's the base case, which means this recursion keeps going until we get down to k equals 0.
We start at k. We end up with 0.
So the number of levels here is k. OK.
The height of this tree-- this recursion tree is k. So how many nodes are there in this tree?
2 to the k. So total running time is v times 2 to the k. I guess I should write 2 to the k times v. Hey, that is exactly what I wanted.
I got a function of k-- namely 2 to the k. Exponential-- that makes sense.
And I got a polynomial in n. Here it's n. The exponent is 1. v is at most n. n us v plus E. Wow.
Big improvement.
This seems equally simple of an algorithm as this one, but actually it runs a lot faster.
OK. Let me give you a feeling-- I mean this is what we would call a linear time algorithm for fixed k. The exponent here doesn't depend on k. If k is 10, it's a linear time algorithm.
If k is 100, it's a linear time algorithm.
If k is 100, that might be a little bit beyond what we can run.
But you know, k equals 32, 40 maybe, that would probably be reasonable running time, in practice.
OK. That's a lot better than before where like k equals 2 or 3.
This is probably unreasonable.
v is like a billion, say, big graph.
Also from a theoretical perspective, this works even up to k equals log n. If k equals log n, this'll be n squared.
That's nice.
k equals 2 log n, it's n cubed.
OK.
So it grows.
But we can handle k equals order log n. And this will still be polynomial.
In general, with fixed parameter algorithms, it's not always going to be up to log n, it's going to be up to whatever the inverse of this f of k is.
That's where we can still be polynomial.
So that's nice.
I consider this a good running time.
Good in the sense that it follows that definition of fixed parameter tractable.
So bounded-search-tree algorithm is good.
Brute force algorithm is bad.
In this case.
Bounded-search-tree is a general technique.
You can use it for lots of problems.
We're going to see another technique today called kernelization.
But-- Let's see-- before I get there, I want to question this definition.
So this definition is nice.
It's natural in the sense that it gives you-- it distinguishes between the exponent of n depending on k and not depending on k, which is a natural thing to do.
But there's another natural definition of fixed parameter tractability.
So let's-- vertex cover-- I think you remember the problem by now.
So let's see-- we have this definition, which is f of k times polynomial n. But I would say that the first time I saw fixed parameter tractability, I thought, well, why do you define it that way?
I mean, maybe it would be better to do f of k plus polynomial n. That would be better, right?
That would be faster, seems like.
So I mean, this is nice in that we achieved this bound, but could we hope for this even better bound.
OK?
It turns out these notions are identical.
This is weird.
The first time you see it.
So theorem-- you can solve a problem in this kind of time, if and only if you can solve the problem in this kind of time.
So of course, f is going to change.
And why don't I label these constants.
So we have c up here and some c prime up here.
But you can solve a problem in this multiplicative time, if and only if you can solve it in an additive time with a different function and a different constant.
This is actually really easy to prove.
The longer you think about it, the more obvious it will be.
If you have an instance of size n with parameter k, there are two cases.
Either n is less than or equal to f of k, or n is greater than or equal to f of k. Right?
It's got to be one of those, maybe both.
If n is less than or equal to f of k that means that this running time, f of k times n to the c-- let's see-- n is at most f of k. So this is at most f of k to the c plus 1 power.
Right?
I multiply f of k to the c times f of k. When n is greater than f of k, then I know that this running time, f of k times n to the c, well, now I know an upper bound of f of k. I know this thing is at most n. And so this is at most n to the c plus 1.
OK.
So really I have two scenarios, either I'm bounded by some purely function of k, or I'm bounded by some purely polynomial of n. Which means, in both cases, the running time f of k times n to the c is bounded above by the max of those two things, max of f of k to the c plus 1, n to the c plus 1.
OK. And the max is always, at most, the sum.
I'm assuming everything here is non-negative.
So I take f of k, c plus 1, plus n to the c plus 1.
Boom.
That is an additive function of k plus polynomial in n. OK. Rather trivial.
This a funny area where you think, ah, this is deep question.
Are these things the same?
And ends up, yeah, they're the same for obvious reasons.
So for example, we have this linear, basically n times 2 to the k algorithm.
If you apply this argument, you get this is, at most, so this n times to the k bound, is, at most, n squared plus 4 to the k. OK.
I'm basically just squaring both of the terms.
OK.
Probably you prefer this time bound, but if you really like an additive time bound, the exact same algorithm satisfies this.
OK.
So not that exciting.
And in practice, n squared-- it looks like a bad thing, so you'd probably prefer this kind of running time.
But there is a sense-- there's a quadratics thing going on here in that we have an n and then we have a function of k multiplied together-- OK-- whatever.
All right.
So this justifies the definition.
This is kind of robust to whether I put a dot here or plus, so clearly this is the right definition.
We're going to use dot.
You could also use plus, but-- all right.
But there's another thing called kernelization, which, in an intuitive sense, matches this idea of plus.
And it also matches an idea that's common practice called pre-processing.
If I have a giant graph, and I'm given some number k, and I want to find a vertex cover, well, maybe the first thing I could do is simplify my graph.
Maybe there's some parts that are really easy to solve.
I should throw those away first.
And that will make my problem smaller.
So if I'm going to have an exponential running time, presumably, I want to first make the problem as small as I can.
Then deal with one of these algorithms.
OK.
So we're going to do that.
First, I'm going to tell you about it generically.
And then we'll do it for vertex cover.
So first, let me give you a definition of what we'd like out of this pre-processing procedure.
It's going to be called a kernelization procedure.
Kernelization algorithm is a polynomial time algorithm.
Head back to polynomial time land.
You can think of it as a reduction, but with NP-hardness, we reduced from one problem a to another problem b.
Here we're going to reduce from the problem a to the same problem a.
It's a self reduction, if you will.
But the input to the problem is going to get smaller.
So we're going to convert an input.
So this is for a parameterized problem.
So an input consists of some regular input x and a parameter k. And we want to convert it into an equivalent small input x prime k prime to the same problem.
OK.
The problem is fixed, say vertex cover.
So we're given an arbitrary input.
This would be a graph and a number k. And we want to convert it into an equivalent small input, which is another graph G prime, and another parameter k prime.
So equivalent means that the answer is going to be the same.
OK. And I want the answer to the problem-- let's say, answer of x comma k to be equal to the answer to x prime and k prime.
Again, same problem, but different input.
I'm trying to be a little generic here.
It could be-- we're going to think here about decision problems, but this makes sense even for non-decision problems.
Whatever the answer is here, it should be the same as the answer is here, because I want an exact solution.
I want to solve exactly the problem.
I want to compute this answer exactly correctly.
So if I can reduce it to some x prime k prime with the same answer, well, now I can just solve x prime k prime.
So that's good.
Now what does small mean?
We need to define both of these.
Small means that the size of x prime, which you might call n prime, should be, at most, some function of k. Cool.
So this is interesting.
So we started with probably a giant problem, x excise n, and we have a parameter k, which we presume is relatively small.
And we convert it into a new input x prime that's very small.
Its size is a function of k. No more dependence on n. n has disappeared from the problem.
OK. We start with something a size n. We produced something the size as function of k. And then-- OK-- there's some other parameter k prime-- doesn't matter much what it is.
It's going to also be a function of k. So we started with something of size n. We produced something of size k. In polynomial time.
Wow.
This would be big if we could do it.
Because we start with giant problem small k, and we kernelize it down-- so here's the picture with this big thing.
And the intuition is that the hardness of the problem is just from this thing of size k. But there's no thing of size k. Or at least we haven't found it yet, right?
The k is the size of the vertex cover we're looking for.
But we don't know where that is.
It's hiding somewhere in this instance, in this amorphous blob.
So it's k. But somehow, magically, this kernelization procedure produces a new problem that's only a little bit bigger than k. OK.
Some function of k. So we take the big problem, we make it down to this small thing.
What do you do now?
You can run any algorithm you want, any finite algorithm applied to this instance will run in some function of k time.
Doesn't matter as long as this is a correct algorithm.
If your problem is in NP, there is an exponential time algorithm.
You just try all the guesses.
So we could use-- we have two of them here.
We could run either of these after we've kernelized.
And we would get an FPT time.
And, indeed, the FPT would mimic this kind of running time.
We do a polynomial amount of pre-processing.
That's the kernelization procedure.
That's the only dependence on n. After we've done that, the new problem is entirely a function of k. And then you apply any algorithm to that, you'll get f of k running time.
Now if we want a good f of k, we should use the best algorithm we have.
But, in general, you could use anything.
All right.
So far, so good.
So we had this theorem that product is the same as plus.
In fact, kernelization is the same thing.
So these are all the same thing.
I guess this one is equivalent to being FPT.
that's the definition of FPT.
So I'll just write it here.
The problem is FPT, if and only if it has a kernelization.
This is crazy.
I keep introducing stronger and stronger notions of good.
And they all turn out to be the same.
That, again, gives you a sense of robustness of this definition.
And why this is a natural thing to study.
So this sounds crazy.
How could I put all of the easy work at the beginning in this polynomial time algorithm and then in the end produce something that is a reasonable size?
Again, this proof is going to be trivial.
I think everything in this field is either really hard or trivial.
I guess that makes sense.
So let's first-- so I'm just looking at this inequality-- this implication-- we already did the other one.
The easy direction, of course, is this way.
I didn't even do it in this case.
If I have an additive running time, it certainly, at most, the product, assuming both those numbers are at least 1.
OK. And again, if I have a kernelization, as I said, I could run the kernelization algorithm in polynomial time, and then run any finite algorithm to solve the problem, and I would get some f of k running time.
So this is easy.
Kernelize and then run any algorithm on the kernel.
Kernel is the produced output x prime k prime.
OK. Let's do the other direction.
That's the interesting part.
So suppose I have it an algorithm that runs, let's say, in this running time.
I claim that I can turn it into a kernel.
And the proof is going to look just like before.
OK.
So there are two cases.
One is that f of k, let's say, is less than or equal to n. Actually I want to do the other case first.
I think it's a little more natural.
Well, it doesn't really matter.
They're both easy but for different reasons.
the then parts are going to look different.
So the first case, if n is at most f of k, what do I do in that situation, in other words, kernelize of this thing of size n, I want to kernelize into something of size f of k?
Nothing.
I'm done already.
So this is the already kernelized case.
That's great.
So the other cases, and it's big.
That's the more interesting case, of course.
n is greater than or equal to f of k. What happens here?
Well, just like last time, that means that this running time, f of k is now at most n, that means this running times is at most n to the c plus one.
Right?
So that means the FPT algorithm that I'm given, because we're assuming here we're given an FPT algorithm, we want to produce a kernel-- in that case, that algorithm runs in n to the c plus 1 time.
Which means I can actually run it.
That's polynomial time.
OK.
So over here I needed a polynomial time kernelization algorithm.
If it happens that f of k is at most n, then I can actually run the FPT algorithm, and that would be a valid kernelization procedure.
Now the FPT algorithm actually solves the problem.
Let's say, it says yes or no, whether the answer to my original question.
The kernelization procedure has to output an input to the problem.
So what I need to add one thing here which is just-- output a canonical yes or no input accordingly.
OK.
If the FPT algorithm-- here I'm thinking about decision problems-- if the FPT algorithm says yes, I'm going to output one instance, one input of the problem where the output is yes.
I know that one exists, because this algorithm said yes.
So in a constant amount of space, I'm able to write a yes input.
Or in a constant amount space, I write a no input.
So the new kernel will have constant size, which is smaller than f of k. OK. That's it.
So either I output the same input that I was given.
Or I output something of constant size that encodes a yes or no.
Kind of trivial again.
The catch is that size of the kernel, the size of the output, in general, here is going to be exponential in k, because f of k presumably is exponential in k. That's annoying.
This is what you might call an exponential size kernel.
So an interesting question.
And exponential size kernels are equivalent to FPT.
Something that may not be equivalent is a polynomial size kernel.
It would be nice if I start with something that's polynomial in n, and I reduce it to something that's polynomial in k. And then I run something that's exponential on that.
But it will only be singly exponential in k, hopefully.
Whereas, if I use this kernelization procedure, if I apply to vertex cover with one of these algorithms, I'm going to get a new thing that's size is exponential in k. If I run one of these brute force algorithms, I'm going to get something that's like doubly exponential in k. That's not so hot, because I know how to do exponential in k. All right.
But this is the general idea of kernelization, and why it's not surprising that you can do it.
I have one catch here which is-- this is an algorithm.
It compares n to f of k. In order to do this, you have to know what k is.
Minor technicality.
If you don't know what k is, you can basically run this algorithm with a timer, a stopwatch, and if it's running time exceeds n to the c plus 1, then you know you're not in this case.
If it finishes within that time bound, great.
You found the answer.
If it doesn't finish, then you know you must be in this case, and then you just quit, and output your original input and say, I'm kernelized.
Done.
Easy.
OK. That's a technicality.
All right.
So much for general theory.
Let's go back to algorithms.
Yeah, all this work I want to write down over here.
We have a v times 2 to the k algorithm.
On the one side.
And here, we have an E as v to the k algorithm.
Just want to keep a running-- we're going to get a faster algorithm than both of those through kernelization.
So I claim that we can find a polynomial kernel-- polynomial-sized kernel-- it's going to be quadratic for vertex cover.
These are hard to find.
And there's a whole research industry for finding polynomial kernels.
So I'm going to give you some methods.
But they're specific to vertex cover.
So here's the first thing.
This is hard to draw.
So here I have a vertex u, and suppose I have an edge connected from u to u.
This is called a loop.
What can I do from a vertex cover perspective?
What can I conclude about this picture?
Yeah
[INAUDIBLE]
Right.
You must be in the vertex cover, because this edge really only has one endpoint.
OK.
So far, so easy.
So what I can do in this case is say, OK, u is in the vertex cover, and then delete u at it's incident edges.
So-- and then we have to decrement k. OK.
Cool.
Seems-- feels familiar.
But in this case, there's no guessing.
We just know you must be in the cover.
All right.
Here's another case.
Suppose I have u and v and there are many edges connecting them.
This is called a multi-edge.
So maybe you just assume your graph is simple.
But if you don't assume your graph is simple, we might have these kinds of situations.
What can I do in this case?
What can I guarantee?
Yeah
You can remove all but one edge
You can remove all but one of the edges.
Let's just delete all but one.
If I cover one of them, I cover them all.
Easy peasy.
See if you get the other rules.
So delete all by 1.
In general, we're going to have a bunch of these kinds of simplification rules are guaranteed correct.
They don't change the output.
But magically, we're going to end up with something the size of function of k. We haven't done much yet.
But now we know that the graph is simple, meaning it has no loops and it has no multi-edges.
Cool.
All right.
Next thing I want to think about is a vertex of degree greater than k. k is the current value of k. So suppose I have a high degree vertex, more than k, edges out going from it.
What can I say then?
Yeah
You must pick it
You must put it in the cover.
Why?
Because then you need to cover all the remaining error vertices
Right.
Proof by contradiction.
If I don't put it in the cover, that means all of these guys are in the cover.
There's more than k of them.
And the whole goal was to find a vertex cover of size at most k. So you better not put all of these in your vertex cover, because that's more than k. So, therefore, this one has to be in there.
This is a cool argument.
Simple, but cool.
OK.
So any vertex of degree greater than k must be in the vertex cover, which I was calling S. OK.
So delete that vertex and its incident edges, decrement k, because we just used something.
OK.
So just keep doing this.
Every time I see a vertex of degree more than k, delete it, decrement k. Now I have a new graph and a new value of k. Look for any vertices whose degree is more than k. If I find one, delete it, repeat, repeat.
Keep repeating until you can't anymore.
How much time is this going to take?
I don't know.
Most quadratic, right?
I look at all the vertices.
Look at their degrees.
I could look over all the edges, increment the degrees.
In linear time, I can find whether there's any vertex of degree more than k. Then delete it in linear time.
Then try again.
And this will happen to most linear time many times because I can only delete a vertex once.
So I delete at most v vertices.
So overall running time here is like at most v times E, polynomial.
Probably if you're clever, use a data structure to update degrees.
You could do this in order v time.
But let's not be clever yet.
All right.
So now, after I've done all of these reductions, I have a graph where every vertex has degree at most k. So it's like a bounded degree graph.
Why do I care about a bounded degree graph?
Remember, I drew this example, which was a star graph, where n was large, but k was very small.
n was n, n was v, and k was 1.
Now a star graph is special because it has a very high degree vertex.
In general, if I have a vertex of some degree, say k, and I put it in the vertex cover, it covers k edges.
OK.
So each vertex in S covers at most k edges, wherever the degree is.
So that means you don't get much bang for your buck anymore.
We've already taken care of all the high degree vertices where you get a lot of reward for putting one vertex in the cover.
Now this is the new value of k. It may have decremented from before.
Every vertex that we could possibly put in the set will only cover k edges.
Now we know that we only get to put k more vertices into the set.
We know-- we're supposing that sides of S is at most k. So that means that the number of edges, size of E, must be at most k squared.
All right.
Because every one I put into S covers at most k edges.
All of them have to be covered.
And so k times k is k squared.
Hah, interesting.
That means my graph is small.
Now slight catch.
There might be a whole bunch of vertices that have no edges incident to them.
So I need to delete isolated vertices.
Let's say, degree 0 vertices.
OK.
Degree 0 vertices-- you really don't want to put those into your vertex cover.
No point.
They don't cover any edges.
So delete those.
And now, I still may not have a connected graph, but, in the worst case, I have a matching.
I know the total number of edges is at most case squared.
That means the total number of vertices is at most twice that, 2k squared.
So after all of these operations, I assumed that S was size at most k. And then if I do all of these operations, I get a graph with at most 2k squared vertices, and at most case k squared edges.
So the total size of the graph, which I'm calling n, n which is size of v plus size of E is order k squared.
3k squared.
And I assumed that there was a vertex cover size at most k throughout this.
So what I do is I run this kernelization algorithm.
And I see-- is the graph that I produced size at most 3k squared?
If it is, output it.
That's a kernelized thing because it's small.
If it isn't, if the graph I've produced is still too big, that must mean that this assumption was wrong, which means the answer to the vertex cover problem is no, there is no vertex cover of size at most k. And so then I just output a canonical no instance, like-- so I mean, this is sort of outside-- but if the newly produced graph-- I'll call it v prime plus E prime is greater than 3 times k squared, then output-- and here I can actually give you one-- let's say, I'm going to output the graph which is a single edge with two vertices and k equals 0.
The answer to vertex cover in this instance is no.
So this is an example of a constant size, no representative.
So either I get something that's small and I output that, or it's big, in which case I output this thing, which is to say, nope, can't be done.
That's kernelization.
So I've produced a quadratic size graph, quadratic in k in polynomial time.
Question?
No.
Wow.
So this is kernelization at its finest.
A polynomial kernel.
Polynomial time.
We get that down to something the size of polynomial in k. This is how you should-- if you want to solve vertex cover, you might as well do these reductions first, because they will simplify your thing.
And now, if you happen to know your vertex cover is small, then the graph will be small.
So now we could run either of these algorithms.
OK.
Presumably, we should run the better one.
But for fun, let's analyze both of them.
OK.
So I'm going to leave the running times here.
We're going to get a faster vertex cover algorithm from a fixed parameter tractability perspective.
So here's a new FTP algorithm.
Two of them.
First we kernelize.
OK. We spent-- I guess-- order vE time.
Again, I think you can get that down to order v without too much effort.
Obvious.
It's not totally obvious.
It's a good exercise.
Be a good problem set problem.
It's not on the problem set.
Don't worry.
It could be a good final exam problem.
Probably a little long.
All right.
So now we could-- let's say, option one-- let's use the brute force algorithm after that.
The running time of that is E times v to the k. But now E is k squared.
And v is also order k squared.
Let's not worry about-- actually I do have to worry about constants here, because it's in the base of an exponent.
So I do.
So we're going to get k squared for the E term.
And then v term is going to be 2 times k squared.
And that's going to be raised to the k-th power.
OK.
So I'll simplify a little bit.
This is like 2 to the k times-- I guess-- k to the 2k.
OK.
It's k to the k squared.
Not bad.
Overall running time is vE plus this.
It's a function of k. It's exponential.
Good.
We have a better algorithm.
We have this v times 2 to the k running time, so we might as well use that one.
But the point is-- once you kernelize, you can use pretty stupid algorithms, and you still get really good running times.
OK. We'll get a slightly better running time using the bounded-tree-search.
So if we use bounded-tree-search, we have v. v is 2k squared.
So here the constant doesn't matter, because there's no exponent.
And then we have times 2 to the k. So we're going to get k squared times 2 to the k algorithms.
Kind of funny symmetry here.
2 and k are switching roles.
Of course, the 2 to the k is the big term.
But now it's only singularly exponential in k. This thing is like 2 to the k log k. This thing is only 2 to the k. So it's better.
And this is like k factorial.
And this is just 2 to the k. So it's a big improvement.
This will be a much more practical algorithm.
So we run the kernelization, then we run the bounded-tree-search algorithm.
And so the total running time is vE plus k squared 2 to the k. The story doesn't end here.
There are dozens of papers about how to solve vertex cover from fixed parameter tractability perspective.
The best one so far-- I'm not going to cover-- but it is based on kernelization.
Just more rules.
And you get k v plus 1.274 to the k. And some cover's better than 2, but very similar.
That's vertex cover.
If you have a vertex cover instance, and you know that it's going to have a relatively small vertex cover, these are the algorithms you should use.
Any questions?
This ends our vertex cover story.
But the last thing I want to do is connect up these two areas.
Last class we talked about approximation algorithms.
This class we talked about fixed parameter algorithms.
They're actually closely related.
And so, for example, we will get a fixed parameter algorithm to subset sum, using what we already had last lecture.
So, so far today, I've basically been talking about decision problems.
But let's think a little bit about optimization problems.
So take your favorite optimization problem.
Like any of the ones from last lecture.
And let's assume that the optimal solution value-- the thing we're trying to optimize, minimize, or maximize-- is an integer.
Assume that OPT is an integer.
OK. Now let's look at the decision problem.
Whenever you have an optimization problem, you can convert it into a decision problem.
You can convert it into a few.
For example, OPT less than or equal to k, or OPT greater than or equal to k. They're all going to turn out to work the same.
OPT equal k would also work.
Now that's a decision problem, but what I want is a parameterized decision problem.
What should my parameter be?
k. All right.
That's the obvious parameter.
In some sense, we're parameterizing by OPT, but we're adding a layer of indirection.
We're saying, well, OPT, but we want to decide whether OPT is less than or equal to k. And let's parameterize by k. That's similar flavor to what we had with vertex cover.
If we started with minimum vertex cover, and converted it.
Cool.
Here's the theorem.
This is not going to be as strong as the other things we've seen.
No equivalence here.
So it's a one way implication.
And I haven't defined this term yet, but it's similar to one we saw last class.
If the optimization problem that we started with has an efficient PTAS, an efficient Polynomial Time Approximation Scheme, then the decision problem-- you get from here-- is fixed parameter tractable with respect to k. OK.
So what does EPTAS mean?
It's going to look familiar.
We're going to take an arbitrary function of 1 over epsilon times a fixed polynomial in n. So last time we talked about PTAS we could have-- you could have something like n to the f of 1 over epsilon.
I'm going to consider that bad, as you might imagine, from a fixed parameter tractability perspective.
Better would be some function, possibly exponential, if 1 over epsilon times polynomial in n. This is going to be good from a fixed parameter perspective.
Although it's about approximation algorithms, not about exact algorithms.
Of course, even better is the FPTASs we saw last time, which is polynomial 1 over epsilon times polynomial in n. That's ideal.
If you have an FPTAS, it is also an EPTAS.
You just remove-- or you just add one more stroke to the first letter.
And you got an EPTAS.
And last class we actually saw an EPTAS.
For subset sum, we saw 2 to the 1 over epsilon times n. Now, in fact, for that problem, there's an FPTAS.
Even better.
But you can see from last lecture why it's nice to have an exponential dependence on 1 over epsilon.
And what this is saying is you can do that as long as that exponential dependence is separated from the n part.
If it's multiplicatively or additively separated, as you might imagine, it's the same thing, from n, then we call this an efficient PTAS.
Not fully, not quite as good as an FPTAS, but close.
And as long as you have such a thing, you can convert it into an FPT algorithm for the decision problem.
The way this is typically used-- so this tells us we get an FPT algorithm for subset sum.
In fact, because there's an FPTAS, we get a pseudo polynomial time algorithm, which is in some sense better.
Anyway.
The way this theorem is usually used is in the contrapositive.
What this tells us is that if we can find a problem that is not FPT-- and there's a whole theory like NP completeness for showing the problems are almost certainly not fixed parameter tractable-- then we know that there is not an EPTAS.
And this is the state of the art for proving that these kinds of algorithms do not exist.
Typically, you look at it from a fixed parameter perspective, and show that probably doesn't exist.
Then you get that this probably doesn't exist.
OK. Let's prove this theorem.
It's, again, really easy.
But a nice connection between these two worlds.
All right.
So there are two cases-- the optimization problem we're thinking about could be a minimization or maximization problem.
Let's say it's maximization, just to be concrete.
It won't make too much difference, but it will make a tiny difference in order of-- or the inequality directions.
OK.
So what we're going to do.
So we're given an EPTAS, and we want to solve FPT.
We want an FPT algorithm.
So what do we do?
Well, an algorithm is going to be to run that EPTAS.
That's sort of the only thing we can do.
Now the EPTAS-- this is an approximation scheme.
It has an extra input which is epsilon.
We need to choose epsilon, because we're not-- we're trying to solve it exactly.
But there's no epsilon in that problem, so we got to make one up.
Let's run the EPTAS with-- anyone have good intuition?
What should epsilon be?
Yeah.
Remind you of this.
Tricky.
We want epsilon to be small.
Yeah
It should be 1 over k
1 over k is almost right.
Anything less than that would work.
So I'll use 1 over 2k, but 1 over k plus 1 would also work.
Or anything a little bit-- 1 over k-- yeah-- 1 over k plus .00001 something.
Anything a little bit less than 1 over k will turn out to work.
So, why?
So first of all, how much time does this take?
Well, we were given this running time.
1 over this is 2k.
So this is going to take f of 2k time times polynomial in n. OK. We need to connect E and k, because we're given-- sorry-- epsilon and k, because we're given something whose running time depends on epsilon not k. Now we're setting epsilon in terms of k, so now the running time is a function of k, not epsilon.
And then times n. So this is good.
This looks like an FPT running time.
I claim we found that the answer.
OK.
This is maybe the surprising part.
You had good intuition here.
And the intuition is just that-- if you're this close to optimal, and optimal is actually an integer, and you found an integer, then you're going to be less than 1 away, which means you're actually the same thing.
OK.
But let's do it more formally.
So we're within a 1 plus epsilon factor.
I'm going to call the epsilon part relative error.
All right.
That's how much it gets multiplied by OPT in order to compute the error bound.
So the relative error is epsilon.
Epsilon is-- I guess is-- at most epsilon-- epsilon is 1 over 2k, which all I'm going to need is that this is strictly less than 1 over k. So this means if I look at absolute error-- so in case you're not familiar-- relative error is I take my approximate solution-- I subtract off, let's say the optimal solution, did I get this right?
This is a maximization problem.
So yeah, my solution's presumably-- no, it's going to be the other way around.
For maximization problem, it's going to be-- the optimal could be bigger than me, so I take that difference-- this is called absolute error.
OK. And relative error is when I just divide that by OPT.
That's relative error.
So I have this one part already.
So usually you state it in terms of 1 plus epsilon.
If you state it in terms of relative error, the 1 disappears.
You just get epsilon.
The absolute error which is OPT minus APX is I take the relative error and I multiply it by OPT.
OK.
So relative error is going to be less than 1 if OPT is-- I guess-- greater than or equal to k?
I have less than in my notes.
But if OPT is greater than or equal to k, then absolute error is less than 1.
Right?
I hope.
Let's check.
The relative error is actually OPT divided by k. Oops.
No, I've got it the wrong way around.
It's correct in my notes.
OK.
Relative error is this thing.
It's going to be strictly less than OPT divided by k. This thing times OPT.
OK.
So as long as OPT is less than or equal to k, this thing will be strictly less than 1.
That's good.
OPT error less than 1 for an integer means that we actually have the same value.
I have that written down more formally.
So let's go here.
So if we find an integral solution-- of value-- values the objective function we're trying to maximize.
Let's say we achieve something value less than or equal to k. Which it better be about if OPT is less than or equal to k. Then OPT-- OK-- this is basically doing the computation again in another way.
We had 1 plus epsilon.
Epsilon's chosen to be 1/2 1 over k. And then k was the solution value that we found.
And so we have this relation between OPT and the thing.
And therefore-- and this works out to exactly k plus 1/2.
So this is, again, strictly less than k plus 1.
And so if we found-- we assumed that OPT was less than or equal to k. And so now it must actually be equal to k, because there are no integers between k and k plus 1/2.
OK.
I probably could have done that shorter.
So when we have an EPTAS, we exactly get an FPT algorithm.
And that's-- the reverse does not hold.
There are some problems that have FPT algorithms but do not have EPTASes.
But, it's something, and it connects these two fields.
And that's all we'll say about fixed parameter algorithms.
Any final questions?
Cool.
See you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK so today, you're going to see something new.
In fact all this week, you're going to see something that's quite different from what you've been studying in this course.
These are algorithms.
But they're for a completely different sort of model.
Distributed algorithms, OK, so what are they?
So now instead of having algorithms that run on a typical computer, you're going to have algorithms that run on a network of processors.
Or it could be on one machine that has multiple processors, multi processors that memory.
Much of computing is distributed algorithms now.
They solve problems like communication on the internet, data management over a network, allocating resources in a network setting, synchronizing, reaching agreement among different agents at remote locations.
So these are all distributed problems, not things that you just solve on one computer.
The kinds of algorithms you design for these settings have to work under extremely difficult platforms because what you have is concurrent activity that's going on at many locations, many processors doing things at the same time.
And you don't know exactly when everybody's going to perform their activities.
You can have different sorts of timing uncertainty.
The order of events isn't clear.
There could be inputs that arrive at different locations.
And then you also have to deal with failure and recovery of some of the processors or some of the channels involved in the computation.
You don't think of any of this when you're just trying to run an algorithm on one computer.
So distributed algorithms can be pretty complicated.
It's not easy to design them.
And after you design them, you still have to make sure they're correct.
So there are issues involved in proving them correct and analyzing them.
A little bit of history, the field pretty much started around the late '60s.
Edsger Dijkstra was one of the earliest leaders in the field.
He won of the first Turing Awards.
Leslie Lamport won the Turing Award last year.
Although he actually started as a very young guy, way back in the early days of the field.
If you want to look at some sources, I have a book.
There's another textbook by Attiya and Welch.
There's a new series of monographs that basically try to summarize many of the important research topics in distributed computing theory.
And the last two lines have a couple of the main conferences in the field.
OK so I can't do that much in one week.
What I'll do is just introduce the area, by showing you two common models for distributed networks.
And just introduce a very few fundamental algorithms, and you'll see along the way some techniques for modeling and analyzing them.
OK the two models here are synchronous distributed networks, and asynchronous distributed networks.
The problems I'll look at in the synchronous setting are a simple problem of leader election, which is a symmetry breaking problem, basically.
Maximal independence set problem, and then a couple of problems that should look familiar to you from the settings of this class, establishing structures like breadth-first spanning trees and shortest paths trees.
In the asynchronous case I'll revisit these last two problems, setting up breadth-first and shortest path trees.
OK so I mentioned something about modeling in proofs and analysis.
Turns out, getting the formal models right and getting real proofs tends to be pretty important for distributed algorithms because with all the stuff going on, they're complicated.
And it's easy to make mistakes.
The kinds of models that we use are interacting state machines, inputs and outputs.
They send each other messages.
But the kinds of proofs you do typically use invariants, a technique that you're very familiar with from this class.
You can still use them in a distributed setting.
And you still prove them the same way, by induction.
Something else that comes up a lot in the distributed setting is modeling and proofs using levels of abstraction.
You might want to give an abstract description of an algorithm and prove that that works.
And then you have a very detailed, complicated, lower level description that you can prove implements the higher level description.
That's another popular technique.
You use different kinds of complexity measures.
For time complexity, you would measure rounds if it's the synchronous model, or some approximation to real time, if it's the asynchronous model.
You also count communication, either the number of messages you send, or the total number of bits that you send in an algorithm.
So throughout these two lectures, we'll be looking at distributed networks.
So you start with a graph.
Let's just look at undirected graphs this week.
We use n in this field for what you're calling v, the total number of nodes in the network or vertices in the graph.
We use the notation gamma of u to mean the neighbors of u in the graph.
So every vertex of the graph has a set of immediate neighboring vertices.
That's gamma of u.
And the degree of u is the size of the neighborhood, the number of neighbors of the vertex.
OK so we start with the graph.
But now we're going to plunk down a process, some kind of active entity at each vertex of the graph.
So this is some kind of automaton.
If you've taken automata theory, it's not really finite state machines, it's more like infinite state automata that can interact with each other.
So we usually talk about vertices in a graph, processes at the vertices of a graph.
But sometimes we get sloppy and just say nodes.
And we could mean either the vertex or the active thing running at the vertex.
Can't keep them straight all the time.
OK and then with the edges of the graph, we would put communication channels, one in each direction, so that the processes can communicate over the edges.
This week I'm not going to talk about what happens when you introduce failures because we just don't have time.
A lot of distributed computing theory deals with what happens when some of the components in your system fail.
How do you cope with that?
So we'll start right in with synchronous distributed algorithms.
A source for that, if you're interested is the first technical chapter in my book.
OK so you have processes at the nodes of a graph, like I just said.
They communicate using messages.
So think of each process as not knowing who his neighbors are, not knowing anything about the graph.
So what do they have?
They have ports.
You could say they have output ports, on which they could send a message, and then some input ports on which messages can come in.
So in general, the process doesn't know who the ports are connected to.
It just has local names for the ports, like one, two, three, up to the degree.
If you have any questions just stop me and ask, if something's not clear.
Otherwise I'll go pretty fast.
And I know that none of this is familiar.
So in general, the processes don't have to be distinguishable at all.
So they don't have to have special unique identifiers so you could tell the processes apart.
They could be completely identical.
Well if they have different numbers of ports, they're not exactly identical.
They certainly know how many ports they have, and release the local names for the ports.
Good so these are processes sitting at the nodes of the graph.
What do they do?
So they execute.
And we talk about an execution of this network.
It goes in synchronous rounds, and every round, every process looks at its state, and decides what messages it's going to send on all of the ports.
So it could send different messages on different ports.
So then what happens is all the messages that the processes decide to send get put onto the channels and they get delivered to the process at the other end.
So the process of the other end is in some state.
All these messages come in.
It updates its state, based on the arriving messages.
So it changes state in response to whatever comes in.
And this is completely different from this semester so far.
We're going to completely ignore the costs of the local computation.
So each node can compute some complicated algorithm of the sort you've been studying in this class, and we usually don't consider that cost.
We're more worried about the communication costs.
And so we'll be focusing on the number of rounds that it takes, in the synchronous case, and the number of communication messages or bits.
OK so far?
So let's start on the first problem.
Here's a graph.
The nodes start out possibly identical, but you want to somehow distinguish one of them to be a leader.
So you have this arbitrary, connected, undirected graph.
And exactly one process is supposed to elect itself the leader.
That means it outputs a special leader signal.
so exactly one should do that.
So why do you want a leader?
Well in practice, leaders can coordinate things.
They can take charge of communication, and inform other nodes when they're allowed to send messages.
They can coordinate the processing of data.
Basically it allows you to centralize some of the computation.
It can schedule the other processes.
It can allocate the resources.
It could help to reach agreement among the processes, if they start out with different opinions about what is supposed to happen.
All right so let's start out with a very simple case.
You have a clique.
Here's a four clique, where all the vertices are directly connected to all the other vertices, with two directional channels.
And the processes are identical.
So I should have asked you, instead of just giving the answer here, but are they able to elect a leader?
So this theorem says that in general, that's impossible.
Or it's not possible, in the most general case.
If you have, no matter what n is, let's just say we have an n vertex clique for some n. It's not possible to have any algorithm that you can have all the processes run, if it's deterministic and the processes start out all indistinguishable.
There's no way that they can elect a single node as a leader.
So do you have an intuition for why that might be the case?
Yeah
They're all connected, and the cross-problem communication in one round is equal, then to be equal likely to select each one of them.
It would be--
It's deterministic there's no likelihood here.
And nobody is doing any selecting.
You're talking as if there's somebody who's choosing a process to do something.
There isn't anyone in charge.
So this is a really different way of thinking
So every node is essentially the exact same.
So if it says, OK, let's assume I'm going to be leader, everyone is going to assume they're going to be leader
That's exactly the right intuition.
They can't distinguish themselves because they're always going to do the same thing.
Let's look at a very simple case.
Suppose we have just two nodes, two node clique, two nodes connected by channels.
These are identical.
They're deterministic.
What can they do?
Well you could try to design algorithms for one of them to elect itself as the leader.
But you can show, by using induction, that the processes are actually going to remain in the same state as each other forever, however many rounds you execute.
So let's slow down.
We can work by contradiction.
Suppose you have an algorithm that solves this problem.
Both of the processes, they're identical.
They start in the same start state.
Let's say there's a unique start state.
So we could prove by induction on the number of rounds that after any number of rounds, say r rounds, the processes are still in identical states.
So the inductive step is, all right, they're in identical states after some number of rounds.
Let's look at the next round.
They're in the same state.
So they generate the same messages.
So they each other the same messages.
They receive the same message.
And then they make the same state change.
So they stay in the same state.
And you can tweak this, and say how this works for-- yeah, question?
So in what ways is the proof a contradiction?
I'm not finished.
You're exactly right.
We have to finish by using the requirements of the problem.
Since the algorithm has to solve the leader election problem, the requirements say that eventually, one of them has to output leader.
And what happens when he does?
Anyone?
Yeah
You have node also outputting the leader signal
Yeah the other one would also do the same thing.
We're saying round by round, they stay in the same state.
So as someone said before, when one guy outputs leader, at the same round the other guy will output leader as well.
So that's a contradiction to the problem requirements.
Notice we didn't assume anything at all about exactly how the algorithm works.
We're just saying, however it works, it can't solve this, under the assumptions that the nodes are indistinguishable and deterministic.
So as you can see, this will extend if you have larger cliques of size n. So now the process has not just one output port, it has n minus 1 output ports to connect to all the other nodes.
Let's say they're numbered 1 through n minus 1.
And one of the possibilities, and one I'll use in this proof is that the ports happen to be numbered consistently.
So that if you have output port number k at one node, it's connected to input port number k at the other end.
So that's one way things can match up.
All right if that's the case, we could do the same proof we just did.
Show by induction that all the processes in the clique remain in the same state forever.
So same proof.
Suppose you have an algorithm that's solves it.
They all began in the same state.
You show by induction that they all remain the same state.
Well so now we slow down a little bit.
Each process sends a possibly different message on each port.
But everybody sends the same message on port k because they're all indistinguishable.
And then because the way the ports match up, everybody receives the same message on port k. And then they make the same state changes
Does this proof imply that there's a kernel for simplifying the graph when you find a clique?
No because if you have a graph that consists of a clique and then let's say, some other stuff, maybe the leader could be somebody outside the clique.
So you can't just say because there's a clique that you can't elect a leader because you could break the symmetry of the graph with other stuff in the graph.
Yeah?
What assumptions do we make to know that for each k, they receive the same message?
Because everybody is going to send the same message on the same numbered port, because they're identical.
And one way the ports can be hooked up, and we have to tolerate all ways they could be hooked up-- say an adversary hooks them up-- is that port k, somebody's output port, is the other end's input port numbered k. So then they all receive the same message on their port number k. Yeah?
Is it actually possible to always hook up the boards that way.
I mean, it's like wrapped with three vertices
Well I'm just doing it for cliques.
Yeah it is.
Yeah you could do it.
I mean you could have port one always going clockwise, and port two going counterclockwise, I mean, there's always a way to do that in a clique.
I checked that.
So what you've just seen is one of the very basic problems for distributed algorithms, which is breaking symmetry among identical processes.
And you see that deterministic, indistinguishable processes just can't do it.
So we have to have something more.
So what do you think we could add to make this problem solvable?
[INAUDIBLE] processes
I can't hear
Probability.
Probability, OK, anything else?
So we could have the processes actually distinguishable.
The common way in this area is to say that each process has an identifier.
Like, you buy a chip and it's got some identifier burned in.
OK so you have some kind of unique identifiers.
Or you can use randomness.
OK for unique identifiers, you assume everybody has some number or some identifier that it knows what it is.
It's built into its state, let's say, a special state variable.
They're totally ordered, generally.
They could be integers, or from some totally ordered set.
When you say unique identifiers, is it means that different identifiers could appear any place in the graph.
But each identifier can appear at most once.
You can have a huge identifier space in a small graph.
But you're Just selecting some identifiers to put in the processes in the graph.
So that's one set up.
And the other one, of course, is using randomness.
So let's look at the unique identifiers first.
Now the problem becomes easy.
Let's look at the clique again.
Suppose there's an algorithm-- well, let's construct an algorithm that consists of deterministic processes with unique identifiers.
And we're going to guarantee to elect a leader in the graph.
And moreover, it's just going to take one round of communication.
And it's only going to use n squared messages.
How could that work?
Everybody in this click has a unique identifier.
What would they do?
Send it out, right?
So you can just send it on all your ports.
Everybody would send its unique identifier on all its output ports.
And then they collect the unique identifiers from everyone else.
So everybody sees the same set of identifiers.
And so the process with the maximum unique identifier knows that it's the only one with that identifier.
And it's the biggest one.
So it can elect itself the leader.
So all you is unique identifiers and the ability to exchange them reliably.
And you can elect somebody easily.
Randomness, well, various ways to do it.
But one idea is the processes could just choose identifiers randomly.
You take a sufficiently large set of possible identifiers, and so if they just choose uniformly at random, they're likely to choose all different identifiers.
Once you have these randomly chosen identifiers you could use them like the really unique identifiers.
The only thing is you might, there's a small chance that you'll have a duplicate.
In which case, you want to be able to detect that and repeat this.
So first of all, how big the a set do you need?
Well here's an example.
Suppose that you have the n processes choosing at random, independently from a space of size r. Identifiers are the numbers one through r. OK and r is going to depend on n. It's going to be like n squared, but it's also going to depend on epsilon, which is the error probability that you're interested in.
Turns out that n squared over 2 epsilon is good enough.
OK so you have your IDs space at least that large.
And then you can guarantee that with probability at least 1 minus epsilon, all the numbers that everybody chooses are different.
It's a very easy proof.
The probability-- just look at two particular processes-- what's the probability that they choose the same number?
It's just 1 over r, right.
Because they're both choosing at random.
The first one chooses something.
The probability that the second one chooses the same thing is just 1 over r. But now you can take a union bound, just add up the probabilities of any pair having a duplicate.
And so you have n square around n squared over 2 pairs.
And so multiplying 1 over r by n squared over 2 still keeps your probability less than or equal to epsilon, your error probability.
So you can choose identifiers using randomness.
With large enough space, with very high probability, you can get them to be all different.
And now here's how the algorithm works.
So you get an algorithm that would finish in only one round, with probability 1 minus epsilon.
But it will be correct.
And it will have repeated rounds, in case the first round doesn't work.
But the expected time is just 1 over 1 minus epsilon, not very big.
What's the algorithm?
Well processes just choose the random IDs from the big space, like we just said.
They exchange their Ids.
And now, everybody can see everyone's ID, but they also can tell if there's a duplicate.
if the maximum is not unique.
So if the maximum is unique, find the maximum wins.
And everyone knows that.
Otherwise you have a problem.
And you have to repeat.
And you just keep doing that until you succeed.
So this can just continue, but it's likely to finish very fast, if you have a high likelihood of having no duplicates.
Questions about the leader election?
So the story was, it's impossible without something to help you distinguish some processes.
You can do it with unique identifiers.
You can do with randomness.
Second problem is called maximal independent set.
So you have a picture of a maximal independent set in a graph here.
Let's try this.
Yeah cursor.
So the maximal independent set in the graph is here.
But this is something I'll come back to a minute.
This is actually a use of the maximal independent set to model what happens in a certain kind of biological system.
What's a maximal independence set?
So you start with a general, undirected graph network.
And the problem is to choose a subset of the nodes so that they form what we call a maximal independent .
Set let's break that down.
What does this mean?
Independent means you don't have any two neighbors that are both in the set.
So you don't want to get two neighbors in the set.
Maximal means that whatever set you choose, you can't add any more nodes without violating independence.
So now this should look something like a couple of homework problems that you had from the beginning and recently.
But I'm not saying that it's maximum independent set.
I'm not saying you have to have the global, largest number of nodes.
I'm just saying it has to be a local optimum, in the sense that you can't add any more nodes to your set without violating the independence property.
Make sense?
There's two examples, the same graph, two different maximal independent sets.
The green nodes, here we have four green nodes that are independent, not neighbors of each other.
And they're maximal, in that I couldn't add any of the red nodes into a set without violating the independence property.
But then over here, we have a second maximal independent set for the same graph.
Now we just have two nodes.
And you can't add any of the red nodes without violating the independence property.
In other words, every node is either in the MIS, or has a neighbor in the MIS.
There's nothing else you can do to add notes to the MIS So the notion of maximal independence, that make sense?
All right, so to make this a distributed problem, let's start out assuming we have no unique identifier.
Actually, for this whole problem, we're not going to have unique identifiers.
They're all going to be identical.
The processes do need one piece of information, which is some approximation to n, the size of the network, the total number of vertices.
So we would like to have these nodes somehow cooperate to compute an MIS of the entire network graph.
What that means is every process should find out whether it is in the MIS or not.
If it is, it should output n. And if it's not, it'll just output out.
So you don't have to actually compute this, like you're used to solving problems like this, where somebody has to gather all the information in one place.
Nobody gathers anything.
Everybody just has to know whether or not they're in the MIS.
So as you can imagine, this is going to be unsolvable in certain graphs by deterministic algorithms, by the same kind of symmetry breaking problems that you saw for leader election.
So we're going to move right to randomized algorithms for this problem.
Some applications of distributed MIS, well they come up in communication networks, where you want to choose-- let's say you have a very dense network of processes.
You want to choose just a few nodes, which would be like an overlay network.
You would choose some nodes who could take charge of communication that you can communicate on this overlay network, and then in the end, each node can take care of communicating with its many neighbors.
So that's a common sort of application.
But it also comes up in other places.
A great example is in developmental biology, where a couple of years ago, there was a paper in Science by Afek, Alon-- there's like eight authors on that.
But Ziv Bar-Joseph was the lead author of this paper.
So the idea is you have a bunch of cells in a fruit fly.
And during development, some of those cells are supposed to distinguish themselves as being what's called sensory organ precursor cells.
The properties that you want it that actually, you would like a maximal independent set of the cells to become distinguished in this way.
So they wrote a paper about it, got published in Science.
They basically designed a new distributed algorithm that closely mirrored what happened in the fruit fly during development.
So what I'm going to show you is a very well-known algorithm, a classical algorithm for MIS.
This is by Michael Luby.
Very simple algorithm, it executes in phases.
Each phase has two realms.
So you start out with all the nodes being active.
They're all involved.
They don't know what they're going to end up with.
And at each phase, some of the active nodes are going to decide they're in the MIS.
Some others will decide they're out of the MIS.
And some others won't know yet.
So then you just continue to the next phase, with all the remaining nodes and the edges between them.
So you're basically going to settle what happens with some subset of the nodes, and then reduce the graph and continue.
So that's the algorithm.
So what do you do in each phase?
Here's what an active node does at a phase.
Two rounds.
The first round, it picks a random value in a large space, the same kind of idea as before.
This time it's 1 up 2 n to the fifth.
It sends that random value to all its neighbors, receives the values from all its still active neighbors, and then it just looks to see if its value is greater than all the values it received.
So then it's a local maximum.
It has chosen a value that's strictly greater than the values chosen by all its neighbors.
So then it decides to join the MIS and it outputs in.
But now you want to make sure none of its neighbors-- you know that none of its neighbors are going to join the MIS at round one.
Because you know this guy's chosen value is larger, strictly larger, than all its neighbors.
But now you want to tell them that they should not join.
They should be out.
So if you join the MIS you're going to announce that by sending messages to all your neighbors.
And then anybody who receives an announcement can decide it's not going to be in the MIS and it outputs out.
Because it knows it has a neighbor that's in the MIS.
So if you decided in or out at this phase, you're done.
You become inactive.
And only the remaining active guys continue to the next phase.
Make sense?
any questions about how the algorithm works?
And animation.
All right so all the nodes start out identical.
They all pick IDs.
So here's some numbers that they pick.
So which nodes are going to now join the MIS?
16, and the one that chose 13.
Good, so they're in the MIS.
And then at the same phase, all of their neighbors, those for red nodes, are going to decide to be out of the MIS And now you're left with the remaining four nodes.
We don't keep going with the same IDs.
we start over.
We want the rounds to be independent.
So if they choose again, they get new IDs.
And now the guy with the 12 and the guy with the 18 going to join the MIS at this phase.
And their neighbors will decide not to be in the MIS.
That leaves us with just one mode, the guy who had four.
Next phase, he chooses another ID.
But he has no neighbors so by default, he's bigger than all the neighbors.
So he just joins the MIS.
So that's how this works.
Very simple algorithm, and it actually works to find an MIS very quickly.
Why does this give you independence?
How do we know that if this ever terminates, if everybody decides, how do we know that we don't ever have two neighbors that decided to be in the MIS?
Yeah
Because once a node joins the MIS, it broadcasts to its neighbors that--
Right.
The only way you join the MIS is if you have the unique maximum value in your neighborhood.
And when you do, all your neighbors become inactive.
So you're certainly going to have independence.
Maximality, if it terminates, the final set is not going to allow you to add any more nodes.
Why?
Because a node is only going to become inactive if it joins the MIS, or a neighbor joins the MIS.
And we just continue this algorithm until all the nodes become inactive.
So either the node is in the MIS or a neighbor is in the MIS.
So you can't possibly add any more.
Yes?
So this has the basic correctness properties, but what you're probably wondering, is why is this efficient enough?
Why is it efficient?
Well we could say that with high probability, of probability 1, it will eventually terminate.
More quantitative, we can state this theorem that says, with probability at least 1 minus 1 over n, all the nodes decide within four log n phases.
Since n is the number of nodes, this doesn't tell us that you get probability 1 of eventually terminating.
But we can repeat this and get the same sort of bound repeatedly for successive phases.
But let's just focus on getting probability at least 1 minus 1 over n that all nodes decide within about four log n phases.
So let's see what this is saying.
You have this big complicated graph.
And in one round, for this to be like log n behavior, what has to happen at each phase?
You have to reduce it by some constant fraction.
The number of nodes, say, should go down.
So it's sort of how the proof will go.
So we start out with a Lemma saying, you're choosing these IDs at random.
You want a high probability that they're all different.
So we have a lemma like the one we had before.
It says, the probability at least, we use 1 minus 1 over n squared, in each phase.
All these phases up to four log n, everybody's choosing a different random value.
All the nodes choose different values at each phase.
So this lets us ignore the possibility that you have repeats.
So we'll come back to that at the end.
All right, so we're going to pretend that in each phase, all the random numbers are different.
So the key idea of this is to show that the graph has to shrink enough at each phase.
So the way we're going to say that is not in terms of the nodes, but in terms of the number of edges.
We're going to say at each phase, the expected number of edges that are live-- why is that shaking?
OK.
The expected number of edges that are live at the end of the phase is at most half the number that were live at the beginning of the phase.
So an edge is live, if its endpoints are still live.
So instead of talking about reducing the number of nodes by a constant fraction, I'm going to reduce the number of remaining edges by constant fraction of each phase.
So this is what I'm going to prove.
So now I've got only three slides, but the only three slides today that have calculations on them.
So probably have to pay attention, if you want to follow the calculations online.
So let's see why.
But the goal is clear?
We have to reduce the number of edges that remain by a factor of two.
So this is actually a new proof of this algorithm's performance.
The proof in the original papers is pretty complicated.
This is a very intuitive, neat proof.
So the first line of the proof says if you have a node that has a neighbor that chooses a value that's bigger than all of its own neighbors-- so u has a neighbor w. W chooses a value that's bigger than all w's neighbors.
But let's say more.
Let's say it's also bigger than all of u's other neighbors, besides w. So w is really big, bigger than all w's neighbors, bigger than all of u's other neighbors.
If that happens, then what happens to u?
Well we know that w is going to decide to join the MIS.
And u is going to definitely die, is not going to join the MIS.
Right?
OK?
I don't want to lose people in the first line.
Question?
Here's a picture.
Here's u.
And it has a neighbor w. And let's say that w's chosen value is greater than all of w's neighbors, but also greater than all of u's other neighbors.
Yes?
If w has that, w is going to join the MIS, and u is going to definitely not join the MIS.
It's going to decide out in this phase.
OK so far?
Why does you need w to have value greater than u's neighbors?
Because if w is greater than all of its neighbors then it's--
--be in the MIS and u will not be in the MIS.
And that seems like it ought to be enough.
But look at the next line.
Well the line after this one.
What's the probability that w chooses a value like that?
So if it's going to be bigger than all u's neighbors, and all of w's neighbors, and keeping in mind that they are each other's neighbors, turns out that there is degree u, at most degree u plus degree w nodes involved here.
W has to have the biggest of all of those, so it's going to have the probability 1 over the number of nodes of being the biggest one.
So it's just 1 over the degree of u plus the degree of w, the probability that w will choose a big enough value.
But you ask, this is pessimistic.
Why don't I just say that w is bigger than its own values?
Because I want to do this next step.
I want to say the probability that node u gets killed by one of its neighbors, any one of its neighbors in this phase.
I can calculate that as the sum.
The probability that node u is killed by a neighbor is at least the sum over all of its neighbors.
You look at all the vertices in the neighbor set, and you add up this fraction.
So why did I need to make that additional assumption before?
That w is greater than all of u's neighbors, as well as all of its own neighbors.
Yeah?
So you can add a problem to--
Yeah because otherwise these would be overlapping events.
But this way I know they're definitely disjoint events.
We can't have-- if we have w and w prime, you can't have both of those holding because the requirement for w is saying that its ID is bigger than w prime's ID.
Because you have these disjoint events, you can just add the probabilities.
And you know that the probability that u gets killed by some neighbor is at least this summation.
OK so far?
So now I'm going to calculate.
But I wanted to focus on the edges.
So let's see, this tells us a way that a node can get killed.
But let's look at what happens for an edge getting killed.
This is the probability that a node is killed.
So the probability that an edge dies at this phase is at least the maximum of the probability that either of its two endpoints die.
And let's just write it as the average.
The probability that an edge dies is at least the average of the probability that it's two endpoints are killed, in this way.
So for an edge, an edge is definitely going to die, if one of its endpoints dies.
And then the edge dies if it dies in this particular way.
So the probability an edge dies is at least the probability that one of the-- half the sum of the probabilities that the two end points die.
It's the average probability.
Makes sense?
You might have to read this later.
So now we can go from that to the expected number of edges that die.
What is that?
You just add up, over all, the edges, the probability that the edge dies.
The expected number of edges that die is at least the sum over all of the edges of the probability that the two endpoints die.
So you have the sum, over all of the edges.
You add up for all the edges.
The probability that one endpoint is killed, and the probability the other endpoint is killed.
So what we have is this great big summation involving now the kill probabilities for vertices.
So we have the kill probability for each vertex.
How many times does that occur?
If you have a vertex, u, it appears once for every edge that u is an endpoint of.
So you have the kill probability for each node occurring exactly it's degree number of times.
So that lets me rewrite this in terms of vertices.
This sum is just 1/2 the sum over all the nodes of the probability that the node gets killed times its degree.
So I'm calculating by replacing the description in terms of edges, by description in terms of vertices.
More or less OK so far?
So now what do I do?
Well, I know the probability that u is killed.
I have a bound for that up on the first line.
So I'm just going to plug that in.
So I get 1/2 the sum over all the nodes, the degree of the node times this summation that gives me the kill probability for that node.
And now I play around with the sum.
I can move the degree inside the second summation, and I get this.
So now let's stare at this again.
I have the sum over all nodes of the sum over all of its neighbors of some expression.
But if I'm considering a node, every note and every one of its neighbors, that's like considering all the directed edges.
I look at every u, and I look at every edge that connects u to something else.
So I can write it as the sum over all the directed edges of this expression.
So I get half of the sum over all the directed edges of this expression.
But we were talking about undirected edges.
And the undirected edges are being twice here, once for each direction.
I can change this sum to a sum over undirected edges.
But now I have the two endpoints to deal with.
So I get the degree of u and the degree of v in the numerator because I'm looking at it from the point of view both of the endpoints of each edge.
Well something drops out, so I have 1/2 the sum over all the undirected edges of 1.
So that's 1/2 of the number of undirected edges.
So I don't expect you to get every step of this, but it's on three slides, so you can stare at this when you go home and make sure the steps work.
But remember the point of this is to show that you reduce the number of edges by a factor of two, and it's done and sort of a clever way by counting the kill probabilities of vertices.
So we get this, reducing the number of edges.
And now we can just plug that back in to get our complexity bound for the entire algorithm.
Remember the original theorem you're we were to prove is a probability bound for deciding within log n phases.
Well you should have a pretty good idea of why that works because if at each phase, you're going to reduce the number of edges by around a factor of two, then it's going to take something like log n phases to finish.
And I just put a proof sketch.
The number of edges that are still alive after four log n phases, well you divide by 2 four log n times, so you get down to practically nothing.
The probability any edges are alive at the end is very small.
So you get a small probability the algorithm doesn't terminate within four log n phases.
There's an extra little term I threw in here.
You might have forgotten.
There was a term that I needed for the small probability, that somebody chose duplicate IDs.
So I'm bringing them back in at the end, in a little union bound.
And we get our 1 over n probability this way.
But the key idea is you reduce the number of edges by half at each stage.
Enough for you to look at later, I guess to figure this out or you have any questions about this?
So that's the last equations and calculation.
I'm going to go onto a new idea, more conceptual stuff.
Familiar problem, breadth-first spanning trees, setting up breadth-first paths to every node, but we're going to study it in our new setting.
We have a connected graph.
This time, let's suppose that it has a distinguished vertex, like it already has a leader.
So it has a distinguished vertex in the graph that's going to become the root of the BFS tree.
And the processes don't need any knowledge about the graph for this one.
For the rest of the time today and Thursday, we'll assume the processes have unique identifiers, and I don't think we're using any probabilities.
So this is just going to be using the unique identifiers to solve our problems.
So everybody knows its own unique identifier.
The root has a distinguished, generally known, unique identifier say i0.
And the process that has i0 knows hey, I'm at the root of the graph.
So the set up make sense?
We might as well assume that everybody knows the unique identifiers of their neighbors because they could easily exchange information now, and match up who's connected on which port by a unique identifier.
We'll just do deterministic.
There'll be a little bit of non-determinism here.
I'll say more about that.
But I'm not going to worry about probabilities for this.
Well that told you about the general setup.
What are the processes supposed to do?
Well they're supposed to compute a breadth-first spanning tree, rooted at vertex v0.
The branches are going to be directed paths in this undirected graph, coming from v0.
Spanning means they should reach all the vertices.
And breadth-first means that if a vertex is at a distance d from v0, it will appear at depth d in this spanning tree.
So everybody should get a shortest path from the root.
Now how are we going to compute this in a distributed setting?
Well now the output of a process is just going to be its parent in the tree.
So we're not actually going to compute this tree anywhere as a whole.
Everybody's just going to know its parent in the tree.
Questions?
Problem make sense?
So this is just an example of a spanning tree, breadth-first spanning tree.
This gives you shortest paths to all of the nodes, , shortest in terms of the number of hops.
So we can have a very, very simple algorithm.
We're going to let the processes mark themselves as they get included in the tree.
Starts out only the first process, i0, is marked.
So do you want to give an idea, maybe, of how this might work?
Sketch out-- yeah?
The root will send out to its neighbors.
And they will then mark themselves as the parent of whoever they heard from.
Then they will--
This is all synchronous.
So that's great.
They'll be doing this in synchronous rounds.
So everybody will, at the certain distance, is going to get the message at the right number of rounds to mark their distance.
OK so in round one, process i0 will send a special message, say search, to all of its neighbors.
And anybody who receives a message in round one will mark itself, decide i0 is its parent, could output that i0 is my parent, parent i0.
And then it can get ready for the next round, when it's supposed to send to continue this.
So at later rounds, if you decided you're going to send, if you know you're supposed to send from the previous round, then you send a search message to all of your neighbors.
Now the process is sitting there and it receives a search message.
If he's already marked, then he should just ignore the message.
Once you're included in the tree, you don't care if you get other messages, search messages on other paths.
So you only do anything if you're not yet marked and you receive a message.
And in that case, then you mark yourself.
Then you mark yourself, and then you choose one of your neighbors as to be your parent.
Now because this is synchronous, you have several nodes that could be sending at the same time.
So one node could be receiving search messages from several different neighbors at once.
Well, it wants to choose one of them as its parent, doesn't matter which one it chooses.
So it can just choose nondeterminstically just arbitrarily.
And then it decides that it will send the next round.
Is the algorithm clear?
So there's, I mentioned, a little bit of nondeterministic here, only in that a process can choose arbitrarily among several possible parents.
And then we could put in a default, saying that it chooses the one with the smallest ID, if we really want to make it deterministic.
But it's also OK to leave distributed algorithms nondeterministic.
And here I should make a remark that shows how differently nondeterminism is regarded in the distributed setting, from the way it is for sequential algorithms.
For distributed algorithms, there can be many options.
And maybe they're all OK.
But the algorithm is supposed to work correctly, no matter how you resolve the nondeterministic choices.
So think about like np, and the other ways that you've seen nondeterminism so far.
There you say you're lucky if there is a path to a choice.
Here when you make a nondeterministic choice, or when the algorithm behaves nondeterministically, all the choices are supposed to work.
It's like all the paths have to come up with correct answers.
Do you have a question?
Yes, whenever there's a sub- [INAUDIBLE], whenever there's a race condition, we locally assume that there wasn't a difference in local computation time.
But if there is, even in the slightest, then they would get a parent [INAUDIBLE] before another one, it would still be a valid--
So the synchronous model is more abstract than that.
You don't model the local computation time.
You're moving more toward an asynchronous model, where the steps can take differing amounts of time.
Here we just assume you have an abstract model, where everybody does stuff at once, in each round.
But you still have nondeterminism because they can all arrive at the same round somewhere.
But it's OK. You can pick any one and it still works.
So it should be not hard to see that this does give you a BFS tree because you're creating all the branches synchronously.
And you're growing one hop at each round.
It reaches all the nodes eventually because the graph is connected.
And everybody sends messages once a node get marked.
It sends messages to its neighbors.
So eventually, the markings are going to reach all the neighbors, all the nodes in the graph.
So here's how you get the example I showed before, simple breadth-first search.
That's a search message sent by this guy.
I put it to the right of the edge to indicate-- it's kind of hard to distinguish.
But I put them on the right of the edge from the point of view of the sender.
So he sends a search message.
it gets there.
This arrow just indicates that it reached the other end.
And this guy has chosen the sender, which is the other direction on the arrow, as its parent.
Now the recipient is going to send some search messages.
So he sends four of them.
They all get to the other end.
And OK, so all these guys now get marked.
They're included in the BFS tree.
And now the next round, they all send some messages.
I'm not putting in the messages where somebody would send back to a guy who sent to him.
But I put in all the others.
Some of them are going to be ignored.
But you do get to a few new nodes this way.
That's round three.
Round four, everybody sends.
And now you have all the nodes included.
So this gives you the spanning tree that I showed at the beginning of this topic.
This is not a very complicated algorithm.
But I think you can see that things can get worse.
And you want to argue about why the algorithms work correctly.
So as I said before, a popular method of reasoning about the algorithms is to state invariance.
So here, suppose I want to describe the state of the entire network, after some number, r, of rounds.
what could you say about that?
What's the case after r rounds of this algorithm?
Yeah
All nodes at distance r from the root have been marked
All the nodes at distance r from the root have been marked.
In fact, only those by round r, only the ones with distances up through r have been marked.
So to state the invariance, if you want to state invariance, I have to say what's in the state of the processes.
So all right, what can we say?
So the process has a Boolean that says whether or not it's marked.
It has a place to record a parent.
And it has someplace where it puts information about whether it's supposed to send a message at the next round.
And we also should know its UID, so I'll put that in another state variable.
So here is something I can say in invariance.
At the end of r rounds, as you said, at the end of r rounds exactly the processes at distance at most r from the source node, the root node, are marked.
I can say a little more.
I can say a process has its parents defined if and only if it's marked.
So it doesn't just get market.
It also computes a parent, and the parent gets computed at the point where it gets marked.
Then I should say that the parent is correct.
So for any process that's at distance d from the source, if the parent is defined, then it's in fact the UID of a process at distance d minus 1 from the source.
So that says it's actually getting a correct breadth-first tree.
It's getting the parent on a shortest path.
Yeah?
Do these invariants [INAUDIBLE] for i0?
Distance 0 is marked.
i0 doesn't ever-- I see what you're saying.
i0 doesn't have a parent.
So I guess that we should say for i not equal to i0 in this case.
So this would be a process other than i0.
It would have its parent defined, if and only if it's marked.
Well as I think you just noticed, the root node is marked but it doesn't have a parent.
So it's an exception.
But this should be, this doesn't involve i0.
So the second one, I can fix that a bit.
Other comments, questions?
So if somebody wanted to do a formal correctness proof of an algorithm like this one, you would use these invariants.
You prove it by induction.
In fact there's quite a few people who use interactive theorem provers to do proofs like this because the algorithms can get pretty complicated, with a lot of variables.
So you have to do some bookkeeping.
You keep track of all these invariants, and then you want to prove that they're all true by induction.
They all hold through an inductive step.
So you can use an interactive theorem prover to help you do the bookkeeping.
But even a manual proof in a research paper would use invariance in this style.
OK complexity.
So the number of rounds until everybody outputs their parent would be the maximum distance of any node from v0.
So we can say that's at most the diameter of the graph.
It could be less.
It's just is the maximum distance from this particular node.
Message complexity?
Well how many messages are sent in this algorithm?
So everybody is going to send messages only once on all of its edges.
So that means all the edges get a message sent in each direction just once.
So it's order of the number of edges.
All right, so we can play around with this.
So this algorithm just tells everybody who his parent is.
But maybe when you're finished, you'd like to who your children are as well.
For many uses of these trees, you'd like to have a parent be able to talk to its children in the tree.
So how to do that?
Well you can add a child pointer because anybody who gets a search message and selects its parents could send back a message to that parents saying, hey, I'm your child.
And if you get a search message, and you decide that that's not your parent, you can help that guy out by sending a message saying you're not my parent.
In the synchronous case, he would just know that, if he didn't get a parent message.
But things are going to get more complicated.
So we'll send parents or non parent responses to the search messages.
Suppose we want to compute the distances from v0, not just to the parents are.
Well that's easy.
Everybody can just record its distances, as well as its parent and the mark.
And then you just include your own distance value in your search message.
And when somebody receives a search message, it sets its own distance to the received distance plus 1.
So we can just keep track and add one to the distance.
It's easy to augment this algorithm to get this extra information.
All right, now how do the processes know when this is all finished?
So everybody was able to output parent.
I know who my parent is.
But how does anybody know when the entire tree has been produced?
Not so obvious.
So in some settings, you might know an upper bound on the depth of the tree.
And then you could just wait for that number of rounds.
But what if you don't know that?
You don't know anything about the graph.
Nobody knows.
So let's come up with an algorithm for process i0, the root, to know definitively that the tree has been completely constructed.
Ideas?
You're creating this by search messages.
How is i0 going to know when its done?
Yeah
Every time you mark a node, the node can send a message back to its parent, saying hi, I've been marked.
Then you can probably get all the way back to the root.
And then the root can count the number of-- actually, no if the root doesn't--
Root doesn't know the number of nodes.
So that's a good idea
If you don't have a child, you can tell your parent that you don't have a child
That's a good start.
Was there another?
Yeah
More generally, you just send a signal when you know your sub-tree is done
When you know you're sub-tree is done, so that means you're going to be communicating something up the tree.
Right, so that's the idea that you're working toward.
So a termination algorithm to inform i0 when the tree is completely constructed.
So let's say that the search messages get their responses.
So everybody knows which nodes are their, which neighbors are its children, and which are not.
So suppose a node has gotten responses to all of its search messages, knows who all its children are.
Now the leaves in this tree are going to know that they're leaves.
How do they know that?
Propagating all these search messages, and I'm a leaf.
How do I know I'm a leaf?
You can't have children
Yeah, you send all these search messages, and everybody says, sorry you're not my parent.
So you know you have no children because of the kind of responses you get.
So now we're going to use what we call a convergecast strategy.
Broadcast is sending things out.
Convergecast is fanning in information back to the top of the tree.
So the convergecast would say, all right, so the leaves would send a message to their parents saying they're done.
Now if I'm some node in the middle of the tree, how do I know I'm done?
Well it's what you said.
You know that you can figure out when your entire sub-tree is done.
Well first of all, you have to know your children are.
It's kind of a two stage process.
You have to know who your children are, by having received responses to all your search messages.
And you wait to receive done messages from all of your actual children.
So if I'm sitting in the middle of the tree, and I've got done messages from all my children, I know my whole sub-tree is done.
Then I can send the done message to my parent.
Got that?
That's how convergecast works.
And when it reaches the top, if i0 knows who its children are, and it receives done messages from all its children, it knows the whole tree is done.
So it can output that the tree construction is complete.
And it could tell the others by sending a message down the tree, so they all know as well.
Questions?
Wouldn't i0 be the last one to know?
He'd be the last one.
No, he'd be the first one to know that the whole tree is complete.
Everybody else knows when their sub-tree is complete.
So i0 still has to now send another message down the tree to tell everyone else the entire tree is complete.
Is there another question?
All right so this isn't showing that.
This is just showing done messages, which are actually going in the opposite direction from these edges, going up the tree.
But you can just see how they propagate up until the roots says done.
No big deal.
Complexity for termination.
Well it just takes at most diameter rounds and n messages for this done information to come up to the top, once the tree actually is finished.
Because now you're just sending messages on the paths in this tree, which are only, at most, diameter in length.
And this is just the process i0 can tell everybody else.
It doesn't take very long either.
Applications, well suppose you construct a tree like this.
And process i0 now wants to use it to communicate.
It wants to send a whole batch of messages to all the other nodes.
It can just send them now on the tree.
It's an easy way to make sure messages reach everybody else in the network.
Just send them on the edges of the breadth-first spanning tree.
So now the messages, each individual message takes at most n message instances along the edges of the tree, because you only have to traverse the tree edges.
No more dependence on the total number of edges in the network.
And in fact, you can save time by pipelining a series of messages.
So you can send them one round after the other.
The other way, suppose you want to compute something globally.
Suppose everybody starts with some initial value.
And process i0 is going to try to determine the value of some function of everybody's initial value, like the minimum or maximum or the sum or anything.
Well you can do this while convergecasting on an already built BFS tree.
So everybody can just send their information up the tree, and i0 can collect it all.
In general, you can accumulate, you can do data aggregation as you go up the paths of the tree.
So the message size doesn't blow up.
So if you want, for example, the sum of everybody's values, everybody just sends their values up in a convergecast.
And each node computes the sum of all the values in its sub-tree.
So this is pretty efficient.
Make sense?
I'm going to skip this.
But you could do leader election in a general graph, If you don't have a leader, already, i0 by having everybody run a breadth-first search in parallel.
But we'll skip that.
Because I just wanted to have a couple of minutes to start the last topic, and we'll pick it up next time.
So it's the obvious extension.
Instead of just breadth-first search trees, let's put weights on the edges and try to compute shortest paths trees in terms of the total weight of the path.
So we're going to add weights.
It's an undirected graph.
So it's just a weight for each undirected edge.
I'll still have a starting node, vertex v0 with process i0.
Still have unique identifiers.
And I'll assume the processes know who their neighbors are.
And they know the weights of the incident edges, their adjacent edges.
But otherwise they don't need to know anything else about the graph.
So again, this is a familiar problem.
But we're looking at it in a very different way, by distributing it.
so the processes are supposed to compute a shortest paths tree, in the sense that everybody should output its parent in the tree.
And let's say they output the distance as well, the weighted distance from the root node.
So this is called Bellman-Ford's algorithm.
Again it's got the same name in the distributed setting.
The Bellman-Ford shortest paths algorithm.
So everybody is keeping track of their current best distance that they know, and their parent.
And they know their unique identifier.
And here's how the algorithm works.
This will look familiar from when you had Bellman-Ford earlier.
At every round, everybody is going to send its distance to its neighbors.
Instead of just sending a search message, now it will send its actual distance information.
And you receive the messages from your neighbors.
And now you do a relaxation step, as you've seen before.
You look at the current distance you have.
And you see if you've gotten a new distance from a neighbor, such that if you add the new distance you receive to the weight of the edge between yourself and that neighbor, you get something better than what you had before.
If you get that, then you're going to improve your distance.
And if you improve your distance, then you're going to reset your parent to the sender of this new, better distance information.
So does this algorithm make sense?
It's like what you saw before.
But there's no running through all the nodes.
Each node is doing its own thing.
It's waiting to get better distance information and re-computing.
And then it's going to be sending out its better information at the next round.
Question?
So this is kind of a jump in the way of thinking.
All right, so now I'm just going to end basically with an animation that'll show you the kinds of things that happen here.
All right so you start out with the initial node.
And what's recorded in the circle is the best distances.
The rest of these, the best distance they know is infinity.
So I didn't write that.
So this guy knows 0 After one round, he sent two messages.
The best distance each of these guys knows is just the weight of the edge between v0 and itself.
So this guy's now estimating it's distance at 16 and this guy at 1.
16 is not very good because it's actually very roundabout routes that can get there.
But it's going to take us some time to make that adjustment.
After two rounds, everybody is sending their distance information.
But now we get a correction here.
This used to say 16.
But now we have a two hop path that gives you a better distance.
So you get the 1 plus the 14.
So he's going to here, about the distance of 15 as a result of what 1 sends.
And some new guys get their distance is calculated And then after three rounds, it gets a little bit complicated.
So maybe I'm just going to flip through it quickly and let you study later.
But you see that you keep getting improvements, as you perform relaxation steps.
As information gets to somebody by better paths that happen to have more hops, they're going to be reducing their estimates.
I'm going to flip, and you see that this guy's estimate is going down.
And in the end, after eight rounds of this, you end up with a very roundabout path that actually gives this guy a much better estimate.
So you can see how that works.
So the claim is that eventually, every process will have its distance being a correct minimum weight of the path, and its parent will be correct.
I think maybe this is a good place to stop.
We'll pick up with this algorithm and its analysis.
Most of next time is going to be spent on asynchronous algorithms, which is a whole other level of complication.
So I'll see you on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, welcome back.
This is the second half of our two lectures on distributed algorithms this week.
I could turn that on, OK. All right, I'll start with a quick preview.
This week, we're just looking at synchronous and asynchronous distributed algorithms, not worrying about interesting stuff like failures.
Last time we looked at leader election, maximal independent set, breadth-first spanning trees, and we started looking at shortest paths trees.
We'll finish that today, and then we'll move on to the main topic for today, which is asynchronous distributed algorithms where things start getting much more complicated because not everything is going on in synchronous rounds.
And we'll revisit the same two problems, breadth-first and shortest paths spanning trees.
Quick review, all of our algorithms are using a model that's based on undirected graph.
Using this notation, gamma, for the neighbors of a vertex.
We'll talk about the degree of a vertex.
We have a process, then, associated with every vertex in the graph.
And we associate communication channels in both directions on every edge.
Last time we looked at synchronous distributed algorithms in the synchronous model.
We have the processes of the ground vertices, they communicate using messages.
The processes have local ports that they know by some kind of local name, and the ports connect to their communication channel.
The algorithm executes in synchronous rounds where, in every around, every process is going to decide what to send on all of its ports, and the messages then get put into the channel delivered to the processes at the other end.
And everybody looks at all the messages they get at that round all at once, and they determine a new state.
They compute a new state based on all of those arriving messages.
We started with leader election.
I won't repeat the problem definition, but here's the results that we got last time.
We looked at a special case of a graph that's just a simple clique.
And if the processes don't have any distinguishing information, like unique identifiers, and they're deterministic, then there's no way to break the symmetry and you can prove an impossibility result that says that you can't-- in a deterministic, indistinguishable case, you can't guarantee to elect a leader in this kind of a graph.
Just as an aside, I should say that distributed computing theory is just filled with impossibility results, and they're all based on this limitation of distributed computing where each node only knows what's happening to itself and in its neighborhood.
Nobody knows what's happening in the entire system.
That's a very strong limitation, and as you might expect, that would make a lot of things impossible or difficult.
Then we went on and got two positive results, a theorem that-- well, an algorithm that is deterministic, but the processes have unique identifiers, and then you can elect a leader quickly.
Or if you don't have unique identifiers but you have probability, so you can make random choices, you can essentially choose an identifier, and then it works almost as well with those identifiers.
Then we looked at maximal independent set.
Remember what it means, no two neighbors should both be in the set, but you don't want to be able to add any more nodes while keeping these nodes independent.
In other words, every node is either in the set or has a neighbor in the set.
And we gave this algorithm-- I'm just including it here as a reminder-- Luby's algorithm, which basically goes through a number of phases.
In each phase, some processes decide to join the MIS and their neighbors decide not to join, and you just repeat this.
You do this based on choosing random IDs again.
And we proved that the theorem correctly computes an MIS, and with a good probability, all the nodes decide within only logarithmic phases, and here is the number of nodes.
All right, then went on to breadth-first spanning trees.
Here, we have now a graph that already has a leader.
It has a distinguished vertex.
And the process that's sitting there knows that it's the leader.
The processes now are going to produce a breadth-first spanning tree rooted at that vertex.
Now, for the rest of the time, we'll assume unique identifiers, but the processes don't know anything about the graph except that their own ID and their neighbors' IDs.
And we want the processes to eventually output the ID of their parent.
And here is, just to repeat, the simple algorithm that we used.
Basically, it's processes just mark themselves as they get included in the tree.
It starts out with just the root being marked.
He sends a special search message out to his neighbors, and soon as they get it they get marked and pass it on.
Everybody gets marked in a number of rounds that corresponds to their depth in-- their distance from the root of the tree, [INAUDIBLE] from the tree.
We talked about correctness in terms of invariance.
What this algorithm guarantees is, at the end of any number r of rounds, exactly the processes at distance up to r are marked, and the processes that are marked, have their parents defined.
And if your parents defined-- it's the UID of a process that has distance d minus 1 from the root.
If you're a distance d, your parent should be somebody who's correct-- has the correct distance d minus 1.
We analyze the complexity.
Time is counting the number of rounds.
And that's going to be, at worst, the diameter of the network.
It's really the distance from a particular vertex, v 0.
And the message complexity-- there's only one message sent in each direction on each edge, so that's going to be only order of the number of edges.
And we talked about how you can get child pointers.
This just gives you parents.
But if you want to also find out your children, then every search message should get a response either saying you're my parent or you're not my parent.
And we can do a termination using convergecast, and there's some applications we talked about as well.
All right, and then at the end of the hour, we started talking about a generalization, a breadth-first spanning trees, which adds weights so you have shortest paths trees.
Now you have weights on the edges.
Processes still have unique identifiers.
They don't know anything about the graph, except their immediate neighborhood information.
And they have to produce a shortest path spanning tree rooted at vertex v 0.
You know what a spanning tree is and the shortest paths are in terms of the total weight of the path.
Now we want each node each, process, to output its parent in the shortest path.
And also the distance from the original vertex v 0.
At the end of the hour, we looked at a version of Bellman-Ford, which you've already seen as a sequential algorithm.
But as a distributed algorithm, everybody keeps track of their best guess, their best current guess, of the distance from the initial vertex.
And they keep track of their parent on some path that gave it this distance estimate, and they keep their unique identifier.
The complete algorithm now is, everybody's going to keep sending their distance around-- I mean we could optimize that, but this is simple.
At every round, everybody sends their current distance estimate to all their neighbors.
They collect the distance estimates from all their neighbors, and then they do a relaxation step.
If they get anything that's better than what they had before, they look at the best new estimate they could get.
They take the minimum of their old estimate-- stop shaking, good-- and the minimum of all of the estimates they would get by looking at the incoming information plus adding the weight of the edge between the sender and the node itself.
This way you may get a better estimate.
And if you do, you would reset your parent to be the sender of this improved information.
And again, you can pick-- if there's a tie, you can pick any of the nodes that sense-- the information leading to the best new guess, you can set your parent to any of those.
And then this just repeats.
At the very end of the hour, we showed an animation, which I'm not going to repeat now, which basically shows how you can get lots of corrections.
You can have many paths that are set up that look good after just one round, but then they get corrected as the rounds go on.
You get much lower weight paths by having a roundabout, multi-hop path to a node.
You can get a better total cost.
Here's where we got to last time.
Now why does this work?
Well what we need is eventually every process should get the correct distance.
And the parent should be its predecessor on some shortest path.
In order to prove that-- you always look for an invariant, something that's true, at intermediate steps of the algorithm that you can show by induction will hold-- and that will imply the result that you want at the end.
Here, what's the key invariant?
At the end of some number r of rounds, what do processes have?
After r rounds have passed, in this kind of algorithm, what do the estimates look like?
Well, after one round, everybody's got their best estimate that could happen on a single-- that could result from a single hop path from v 0.
After two rounds, you also get the best guesses from two hop paths, and after r rounds in general, you have your distance and parent corresponding to a shortest path chosen from among those that have at most our hops.
Yes?
That makes sense?
No?
Yeah?
OK. All right, and if there is no path of r hops or fewer to get to a node, there's still going to have their distance estimate be infinity after r rounds.
This is not a complete proof, but it's the key invariant that makes this work.
You can see that after enough rounds corresponding to the number of nodes, for example, everybody will have the correct shortest path of any length.
OK?
The number of rounds until every estimate stabilize-- all the estimates stabilize, it's going to be n minus 1.
All right?
Because the longest simple path to any node is going to be n minus 1, if it goes through all the nodes of the network before reaching it.
Makes sense?
If you want to make sure you have the best estimate, you have to wait n minus 1 rounds to make sure the information has reached you.
The message complexity-- well, since there's all these repeated estimates it's no longer just proportional to the number of edges, but you have to take into account that there can be many new estimates sent on each edge.
In fact, the way I've written this, you just keep sending your distance at every round.
It's going to be the number of edges times the number of rounds.
You can do somewhat better than this, but it's worse than just the simple BFS case.
This is more expensive, because BFS just had diameter rounds and this now has n for n minus 1 rounds, and BFS just had one message ever sent on each edge, and now we have to send many.
Comments?
Questions?
Is it clear that the time bound really does depend on n and not on the diameter?
For breadth-first search, it was enough just to have enough rounds to correspond to the actual distance and hops to each node, but now you need enough rounds to take care of these indirect paths that might go through many nodes but still end up with a shorter-- with a smaller total weight.
Is it clear to everybody why the bound depends on n?
Actually the animation that I had last time showed how there are lots of corrections, and you had enough-- you had rounds that depended on the total number of nodes.
Yes?
OK. Yeah?
But could you keep track of each round if the values-- if any of [INAUDIBLE]?
If you-- if everything stops changing after less than n rounds, then you might not have to--
OK, so you're asking about termination?
Yeah
Ah, OK, well so that's probably the next slide.
First, let's deal with the child pointers, and then we'll come back to termination.
First, this just gives you your parents.
If you want your children, you can do the same thing that we did last time.
When a process gets a message, and the message doesn't make the sender its parent, this is not giving it an improved distance.
Then the node can just respond, non-parent.
But if a process receives a message that doesn't improve the distance, it says, OK, you are my parent, but it might have also told another node-- it might have another parent, another node that previously it thought was its parent.
It has to do more work, in this case, to correct the erroneous parent information.
It has to send its previous parent a non-parent message to correct the previous parent message.
Things are getting a little bit trickier here.
On the other end, if somebody is keeping track of its children, it has to make adjustments too because things can change during the algorithm.
Let's say a process keeps track of its children in some set, it has a set children.
If it gets a non-parent message from a child, even if that child might be from a neighbor.
That neighbor might be in its set children, and this could be a correction, and then the process has to take that neighbor out of the set of children.
And suppose the process improves its own distance.
Well, now, it's kind of starting over.
It's going to send that distance to all of its neighbors again and collect new information about whether they're children or not.
The simple thing to do here is just empty-- zero out your children set and start over.
Now you send your new messages to your neighbors and wait for them to respond again.
There's tricky bookkeeping to do to handle corrections as the structure of this tree changes, so getting child pointers is a little more complicated than before.
Make sense?
All right, so now back to your question, termination.
How do all the processes know when the tree is complete?
In fact, we have a worse problem.
With this problem we hit for breadth first search, but we have an even worse problem.
Now what is that?
Yeah?
[INAUDIBLE]
Yeah, before we had each individual node.
Once it figured out who its parent was, it could just output that.
And now, you can figure out your parent, but it's just a guess and you don't know when you can output this.
How does a process even-- an individual process even figure out its own parent and distance?
There's two aspects here in termination.
One is how do you know the whole thing is finished, but the other one is even how do you know when you're done with your own parent and distance?
Well, if you knew something about the graph, like an upper bound on the total number of nodes, then you could just wait until that number of rounds and be done.
But what if you don't have that information about the graph?
What might you do?
Yeah?
You want to BFS in parallel and filter down the information when you've reached the size of the graph
Maybe.
I think-- what is the strategy we use for termination for BFS?
Let's start with that one.
It's a little easier.
You did the subtree thing that we call convergecast the information.
When we had a leaf he knew he was a leaf, and he could send his done information up to his parent and that got convergecast up to the top of the tree.
Can we convergecast in this setting?
Turns out we can, but since things are changing, you're going to be sending done messages and then something might change.
You might be participating in the convergecast many times.
Since the tree structure is changing, the main idea is anybody can send a done message to it's current-- the node he believes is his parent, provided he's received responses to all of its distance messages so it thinks it knows who all its children are.
It has a current estimate of all the children and-- so if it knows all its children and they have all sent him done messages.
For all your current children, your current belief or who your children are, if they've all sent you done messages, then you can send a done message up the tree.
But this can get a little more complicated than it sounds, because you can change who your children are.
What this means is that the same process can be involved several times in the convergecast, based on improving the estimates.
Here's an example of the kind of thing that can happen.
Let's say you start out, you have these huge weights and then you have a long path with small weights.
i 0 starts out and sends its distance information on its three nodes, to its three neighbors.
And this guy at the bottom now has a distance estimate of 100, and it's going to decide it's a leaf.
Why?
Because when it sends messages to its children, to its neighbors, they're not able to improve their estimates based on the new information that he's sending.
This guy decides that he's a leaf right away, and he sends that information back to node i 0.
On the other hand, this guy has the same estimate of 100.
And he sends out his messages to try to find out if he's a leaf, but he finds out when he sends a message this way that he's actually able to improve that neighbor's estimate, because that was infinity till then.
He doesn't think he's a leaf.
We have this one guy who thinks he's a leaf and responds, so this i 0 is sitting there.
He has to wait to hear from his other children.
OK so far?
All right, in the meantime, the messages are going to eventually creep around and this node is going to get a smaller estimate based on the length of that long many hop path.
Then he's going to decide he is not a child of i 0.
He's going to tell i 0 I'm really not your child, which means that i 0 stops waiting for him.
But also, this guy decides he's not a child as well.
He becomes a child of the node right above him.
So i 0 now will realize he only has one child, but this guy believes he's a leaf again after trying again to find children.
And now the done information propagates all the way up the tree, which is now just this one long path.
They start trying to convergecast, but then, oops, they were wrong.
They have to make a correction, and they are forming a new tree.
Eventually, the tree is going to stabilize, and eventually the done information will get all the way up to the top, but there could be lots of false starts in the mean time.
It's sort of confusing, but that's the kind of thing that happens.
Yeah?
There may be a process in which [INAUDIBLE] and then it just switches its mind at the very end of it.
How do you make sure that, that propagation [INAUDIBLE]?
Yes, so the root node isn't going to terminate until it hears from everybody.
You kind of have to close out the whole process.
It's always pending, waiting for somebody, if it hasn't heard from someone.
If things switch, they'll join in another part of the tree.
I think the best thing to do here is sort of construct some little examples by hand.
I mean we're not going to get into it how you do formal proofs of things like this.
We don't even have right now a simulator you can use to play with these algorithms.
Although if anybody's interested, some students in my class last term, actually, wrote a simulator that might be available.
OK, all right, so that's what you get for a synchronous-- some example synchronous distributed algorithms.
Now let's look at something more complicated when you get into asynchronous algorithms.
OK.
So far complications that you've seen over the rest of this course, you have now processes that are acting concurrently.
And we had a little bit of non-determinism, nothing important, but now things are about to get much worse.
We don't have rounds anymore.
Now we're going to have processes taking steps, messages getting delivered at absolutely arbitrary times, arbitrary orders.
The processes can get completely out of sync, and so you have lots and lots more non-determinism in the algorithm.
The non-determinism has to do with who's doing what in what order.
Understanding that type of algorithm is really different from understanding the algorithms that you've seen all term and even synchronous distributed algorithms, because there isn't just one way the algorithm is going to execute.
The execution can proceed in many different ways, just depending on the order of the steps.
You can't ever hope to understand exactly how this kind of algorithm is executing.
What can you do?
Well, you can play with it, but in the end, you have to understand is some abstract properties.
Some properties of the executions, rather than exactly what happens at every step, and that's a jump.
It's a new way of thinking.
We can look at asynchronous stuff, if you want, from my book.
Now we still have processes at the nodes of a graph.
And now we have communication channels associated with the edges.
Now, the processes are going to be some kind of automata, but the channels will also be some kind of automata.
We'll have all these components, and we'll be modeling all of them.
Processes still have their ports.
They need not, in general, have identifiers.
What's a channel?
A channel is-- it's a kind of an automaton, infinite state automaton, that has inputs and some outputs.
Here, this is just a picture of a channel from node u to node v. Channel uv is just this cloud thing that delivers messages.
It has inputs.
The inputs are-- messages get sent on the channel.
You can have one process at one end sending a message m and the outputs at the other end are the deliveries, let's say receive message m, at the other end, node v. To model this, the best thing is actually to-- instead of just describing what it does, to give an explicit model of its state and what happens when the inputs and outputs occur.
If you want these messages to be delivered-- let's say in FIFO order, fine.
You can make the state of the channel be an actual q. mq would just be a FIFO queue of messages.
Starts out empty, and when messages get sent, they get added to the end, and get delivered they get removed from the beginning.
All that we need to describe-- this is a sort of a pseudo code.
All we need to describe-- to write in order to describe what this channel does, is what happens when a send occurs, and when can this channel deliver a message, and what happens when it does that.
A send, which can just come in at any time, and the effect is just to add this message to the end of the queue.
The recieve-- stop moving.
The receive-- that cannot be construction.
We'd hear noise.
It's gremlins.
We have-- a receive can occur only when this message is at the head of the queue, and when it occurs, it gets removed from the queue.
Does this make sense as a description of what a channel does?
Messages come in, get added to it, and then messages can get delivered in certain situations, and they get removed.
That's a description of the channel.
to be using an asynchronous system.
A process-- the rest of the system consists of processes associated with the graph vertices.
Let's say pu is a process that's associated with vertex u.
But I'm writing that just sort of as a shorthand, because u is the vertex in the graph, and the process doesn't actually know the name of the vertex.
It just has its own unique ID or something, but I'm going to be a little sloppy about that now.
The process that at vertex u can perform send outputs to put messages on the channel, and it will receive inputs for messages to come in on the channel.
But the processes might also have some external interface where somebody is submitting some inputs to the process, and the process has to produce some output at the end.
There can be other inputs and outputs.
And we'll model it with state variables.
Process is supposed to keep taking steps.
The channel is supposed to keep delivering messages.
It's a property called liveness.
You want to make sure that your components in your system all keep doing things.
They don't just do some steps and then stop.
Here's a simple example.
A process that's remembering the maximum number it's ever seen.
There's a max process automaton.
It receives some messages m, some value, and it will send it out.
It keeps track of the max-- that's max state variable.
it starts out with its own initial values, so x for u. x of u is its initial value.
And then it has-- for every neighbor, it has a little queue-- here, it's just a Boolean-- asking whether it's supposed to send to that neighbor.
This is the pseudocode for that max process.
What does it do when it receives a message?
Well, it sees if that value is bigger than what it had before, and if so, it resets the max.
And it also makes a note that it's supposed to send this out to all its neighbors.
Whenever it gets new information, it will want to propagate it to its neighbors.
You see how this is written?
You just say, reset the max and then get ready to send to all your neighbors, and the last part is just sort of trivial code.
It says, if you are ready to send, then you can send, and then you're done and you can set the send flags to false.
Yeah?
What study?
For every neighbor.
Oh, I wrote neighbor v before, and then I-- yeah.
I wrote neighbor-- oh, I know why I wrote w. Here, I'm talking about if you receive a message from a particular neighbor v, then you have to send it to all your neighbors.
Before, I used v to denote a generic neighbor, but now I can't do that anymore, because v is the sender of the message.
Just technical-- OK?
We have these process automata.
We have this channel automata.
Now, we want to build the system.
We paste them together.
How we paste them is just we have outputs from processes that can match up with inputs to channels and vice versa.
If a process has a send output, let's say send from u to v, that will match up with the channel that has send uv as an input.
And the receive from the channel matches up with the process that has that receive as an input.
All I'm doing is matching up these components.
I'm hooking together these components by matching up their action names.
Does that make sense?
I'm saying how I build a system out of all these components, and I just have a syntactic way of saying what actions match up in different components.
Questions?
This is all new.
When this system takes a step, well, if somebody's performing an action, and someone else has that same action-- let's say a process is doing a send.
The channel has a send.
They both take their transitions at the same time.
The process sends a message, it gets put into the channel at the end of its queue.
Make sense?
OK. How does this thing execute?
Well, there's no synchronous rounds, so the system just operates by the processes and the channels just perform their steps in any order.
One at a time, but it can be in any order, so it's a sequence model.
You just have a sequence of individual steps.
There's no concurrency here.
In the synchronous model, we had everybody taking their steps in one big block.
And here, it's just they take steps one at a time, but it could be in any order.
And we have to make sure that everybody keeps taking steps.
That every channel continues to deliver messages, and every process always performs some step that's enabled.
For the max processes, well, we can just have a bunch of processes, each one now starting with its initial value.
And what happens when we plug them together with all their channels between them?
Corresponding to whatever graph they are in.
They just have channels on the edges.
What's the behavior of this?
If all the processes are like the ones I just showed you, they wait till they hear some new max and then they send it out.
Yeah?
All processes eventually have a globally maximum value
Yeah, they'll all eventually get the global max.
They'll keep propagating until everybody receives it.
Here's a animation if everybody starts with the values that are written in these circles.
Now, remember, before they were all sending it once, now no more.
Let's say the first thing that happens is the process that started with five sends its message out on one of its channels, so the five goes out.
The next thing that might happen is the other process with the seven might send the seven out on one of its channels.
And these are three more steps.
Somebody sends a 10.
Somebody sends a seven.
Somebody received a message and updated its value as a result, and we continue.
I'm depicting several steps at once, because it's boring to really do them one at a time, but the model really says that they take steps in some order.
Everybody is propagating the largest thing it's seen, and eventually, you wind up with everybody having the maximum value, the 10.
All right, that's how an asynchronous system operates.
We can analyze the message complexity of this.
The total number of messages sent during the entire execution, at worst, on every edge.
You can send the successively improved estimate, so that's, again, what are n times the number of edges.
Time complexity is an issue.
When we had synchronous algorithms, we just counted the number of rounds and that was easy.
But what do we measure now?
How do we count the time when you have all these processes and channels taking steps whenever they want?
Yeah, so this really isn't obvious.
There's a solution that's commonly used, which is-- OK, we're going to use real time.
And we're going to make some assumptions about certain basic steps taking, at most, a certain amount of time.
Let's say that local computational-- time for a process to perform its next step, is little l. You just give a local time bound.
And then you have d for a channel to deliver one message.
The first message that's currently in the channel.
If you have assumptions like that, you could use those to infer a real time upper bound for solving the whole problem.
I mean if you know it takes no longer than d to deliver one message, then you can bound how long it takes to deliver-- to empty out a queue, a channel, and how long it takes for messages to propagate through the network.
It's tricky, but this is about the only thing you can do in a setting where you don't actually have rounds to measure.
Then for the max system, how long does it take?
Well, let's just ignore the local processing time.
Usually that's assumed to be very small, so let's say it's 0.
We can get a very simple, pessimistic really, upper bound that says the real time for finishing the whole thing is of the order of the diameter of the network times the number of nodes times little d is the amount of time for a message queue to deliver its first message.
As a naive way of analyzing this, you just consider how long it takes for the max to reach some particular vertex u along the shortest path.
Well, it has to go through all the hops on the path, so that would be the diameter.
And how long does it have to wait in each channel before it gets to move another hop?
Well, it might be at the end of a queue.
How big could the queue be?
Well, at worst end for the improved estimates.
Let's say it's n times the delivery time on the channel, just to traverse one channel.
What I'm doing is modeling possible congestion on the queue to see how long it takes for a message to traverse one channel.
Yeah?
Are we just assuming that processes process things, and messages are delivered as soon as they can possibly have them?
Good.
Yeah, we normally have-- I'm sort of skipping over some things in the model-- but you have a liveness assumption that says that the process keeps taking steps as long as it has anything to do, and so we would be putting time bounds on how long it takes between those steps.
That'll be the local processing time, here.
I'm saying that's 0
You do that-- wouldn't you be able to get some amount of information about what were things happening?
Ah, OK.
Here is-- this is a very subtle point.
What do you think is that if I'm making all these timing assumptions, the processes should be able to figure that out.
But actually, you can't figure anything out just based on these assumptions about these upper bounds.
Putting upper bounds on the time between steps does not in any way restrict the orderings.
You still have all the same possible orderings of steps.
Nobody can see anything different.
These times are not visible in any sense.
They're not marked anywhere for the processes to read.
They're just something that we're using to evaluate the cost, the time cost, of the execution.
And you're not restricting the execution by putting just upper bounds on the time.
If you are restricting the execution, like you had upper bounds and lower bounds on the time, then the processes might know a lot more.
They might, in fact, be acting more like a synchronous system.
These are times that are just used for analyzing complexity.
They're not anything known to the processes in the system.
OK?
All right.
Now let's revisit the breadth-first spanning tree problem.
We want to now compute a breadth-first spanning tree in an asynchronous network.
Connected graph, distinguish root vertex, processes have no knowledge of the graph.
They have you UIDs.
All this is the same as before in the synchronous case.
Everybody's supposed to output its parent information when it's done.
Here's an idea.
Suppose we just take that nice simple synchronous algorithm that I reviewed at the beginning of the hour where everybody just sends a search message as soon as they get it and they just adopt the first parent they see.
What happens if I run that asynchronously?
I just send it and I get a search message.
Then I send it out to my parents whenever and everybody's doing this.
Whenever they get their first search message, they decide the sender is its parent.
Yeah?
Could you possibly have a front tier that doesn't keep expanding-- not obeying the defining property of the BFS?
Yeah it could be that, because we don't have any restriction on how fast the messages might be sent and the order of steps, it could be that some messages get sent very quickly on some long path.
Someone sitting at the far end of the network might get a message first on a long path and later on a short path, and the first one to gets it decides that's its parent, then it's stuck.
This is not an algorithm that makes corrections.
OK?
All right, well, before we had a little bit of non-determinism when we had this algorithm in the synchronous case because we could have two messages arriving at once, and you have to pick one to be your parent.
Now that doesn't happen, because you only get one message at a time, but you have a lot more non-determinism now.
Now you have all this non-determinism from the order in which the messages get sent and processes take their steps.
There's plenty of non-determinism, and remember, the way we treat non-determinism for distributed algorithms is it's supposed to work regardless of how those non-deterministic choices get made.
How would we describe?
I'll just write some pseudo code here for a process that's just mimicking the trivial algorithm.
It can receive a search message, that's the inputs, and it can send a search message as its output.
It can also output parent when it's ready to do that.
What does it have to keep track of?
Well, it keeps track of its parent, keeps track of whether it's reported its parent, and it has some send buffers with messages it's ready to send to its neighbors, could be search messages or nothing.
This bottom symbol is just a placeholder symbol.
What happens when the process receives a search message, just following the simple algorithm?
Well, if it doesn't have a parent yet, then it sets its parent to be the sender of that search message, and it gets ready to send the search message to all of its neighbors.
That's really the heart of the algorithm that you saw for the simple algorithm for that is the synchronous case so that's the same code.
The rest of the code is it just sends the search messages out when it's got them in the send buffers, and it can announce its parent once the parent is set, and then it doesn't-- this flag just means it doesn't keep announcing it over and over again.
It makes sense that this describes that simple algorithm.
It's pretty concise, just what does it do in all these different steps.
OK?
Now, if you run this asynchronously, as you already noted, it isn't going to necessarily work right.
You can have this guy sending search messages, but some are going to arrive faster than others.
And you see you can have the search messages creeping around in an indirect path, which causes a spanning tree like this one to be created, which is definitely not a breadth-first spanning tree.
The breadth-first tree is this one-- a breadth-first tree.
You have these roundabout paths.
This doesn't work.
What do we do?
Yeah?
You could have a child [INAUDIBLE]
You're going to try to synchronize them, basically
[INAUDIBLE]
That is related to a homework question this week.
Very good.
We're going to do something different.
Other suggestions?
Yeah?
We could keep a variable in each process that-- you can do something like Bellman-Ford
Yeah, so you can do what we did for Bellman-Ford, but now the setting is completely different.
We did it for Bellman-Ford when it was synchronous and we had weights.
Now, it's asynchronous and there are no weights.
OK?
Oh, there's some remarks on a couple of slides here.
This is just belaboring the point, that the paths that you get by this algorithm can be longer than the shortest paths, and-- yeah, you can analyze the message and time complexity.
The complexity here is order of the diameter times the message delay on one link.
And why is it the diameter even though some paths may be very long?
This is a real time upper bound that depends on the actual diameter of the graph, not on the total number of nodes.
Why would an upper bound on the running time for the simple algorithm depend on the diameter?
Yeah?
Because you only have a longer path if it's faster than--
Exactly.
The way we're modeling it-- it's a little strange, maybe-- but we're saying that something can travel on a long path only if it's going very fast.
But the actual shortest paths still move along, at worst, at d time per hop.
At worst, the shortest path information would get there within time d. Something is going to get there within time d, even though something else might get there faster.
OK?
All right.
Yes, we can set up a child pointers, and we can do termination using convergecast.
It's just one tree.
There's nothing changing here.
And applications, same as before.
But that didn't work, so we're going to-- back to the point we were talking about a minute ago-- we're going to use a relaxation algorithm like Bellman-Ford.
In the synchronous case, we corrected for paths that had many hops but low weight, but now we're going to correct for the asynchrony errors.
All the errors that you get because of things traveling fast on long paths, we're going to correct for those using the same strategy.
Everybody is going to keep track of the hop distance.
No weights now just the hop distance, and they will change the parent when they learn of a shorter path.
And then they will propagate the improved distance, so it's exactly like Bellman-Ford.
And eventually, this will stabilize to an actual breadth-first spanning tree.
Here's a description of this new algorithm.
Everybody keeps track of their parent, and now they keep track of their hop distance, and they have their channels.
Here's the key, when you get new information-- you receive some new information, this m is a number of hops that your neighbor is telling you.
Well, if m plus 1, which would be your new estimate for a number of hops-- if that's better than what you already have, then you just replace your estimate by this new number of hops.
And you set your-- to a new parent.
You set your parent pointer to the sender, and you propagate this information.
It's exactly the same as what we had before, but now we're correcting for the asynchrony.
We get shorter hop paths later.
Makes sense?
And the rest of this is just you send the message.
And notice we don't have any terminating actions.
Why?
Because we have the same problem that we had before with having processes know when they're done.
If you keep get getting corrections, how do you know when you're finished?
And how do you know this is going to work?
In the synchronous case, we could get an exact characterization of what exactly the situation is after any number of rounds.
And we can't do that now, because things can happen in so many orders.
We have to, instead, state some higher level abstract properties, either in variance or you'll see some other kinds of properties as well.
We could say, for instance, as an invariant, that all the distance information you get is correct.
If you ever have your distance set to something, it's the actual distance on some path and your parent is correctly set to be your predecessor on such a path.
This is just saying whatever you get is correct information.
This doesn't say that eventually you're going to finish, though.
It just says what you get is correct.
If you want to show that, eventually, you get the right answer, you have to do something with the timing.
You have to say something like by a certain time that depends on the distance.
If there is-- and at most our hop path to a node, then it will learn about that by a certain time, but that depends on the length of the path and the message delivery time.
And the number of nodes, because they can be congestion.
You have to not only say things are correct, but you have to say, eventually, you get the right result, and here it will say by a certain time you get the right result.
Makes sense?
This is how you would understand an algorithm like this one.
Message complexity.
Since there's all these corrections, you're back in number of edges times possibly the number of nodes.
And the time complexity, till all the distances and parent values stabilize, could be-- this is pessimistic again-- the diameter times the number of nodes times d, because there can be congestion in each of the links because of the corrections.
How do you know when this is done?
How can a process know when it can finish?
Idea?
Before we had said, well, if you knew n, if you knew the number of nodes, you could weight that number of rounds.
That doesn't even help you here.
Even if you know-- have a good upper bound on the number of nodes in the network, there's no rounds to count.
You can't tell.
Even knowing the number of nodes you can't tell, so how might you detect termination?
Yep?
It could bound on the diameter of [INAUDIBLE]
Yeah, well, but even if you know that, you can't count time.
See this is the thing about asynchronous algorithms, you don't have-- although we're using time to measure how long it's termination takes, we-- the processes in there don't have that.
They're just these asynchronous guys who just take their steps.
They're ignorant of anything to do with time.
Other ideas?
Yeah
Couldn't you use the same converge kind of thing--
Same thing.
We're just going to use convergecast again, same idea.
You just compute and your repute your child pointers.
You send a done to your current parent, after you've gotten responses to all your messages, so you think you know your children.
And they've all told you they're done and-- but then you might have to make corrections, so as in what we saw before, you can be involved in this convergecast several times until it finally reaches all the way to the root.
Once you have these, you can use them the same way as before.
You now have costs that are better than this simple tree that didn't have shortest paths, because it now takes you less time to use the tree for computing functions or disseminating information because the tree is shallower.
Finally, what happens when we want to find shortest path trees in an asynchronous setting?
Now, we're going to add to the complications that we just saw with all the asynchrony.
The complications of having weights on the edges.
All right, the problem is get a shortest path spanning tree, now in an asynchronous network, weighted graph, processes don't know about the graph again.
They have UIDs, and everybody's supposed to output its distance and parent in the tree.
We're going to use another relaxation algorithm.
Now think about what the relaxation is going to be doing for you.
We have two kinds of corrections to make.
You could have long paths that have small weight.
That showed up for Bellman-Ford in the synchronous setting, so we have to correct for those.
But you could also have-- because of asynchrony, you could have information travelling fast on many hops, and you have to correct for that as well.
There's two kinds of things you're going to be correcting for in one algorithm.
This is going to-- and it's pretty surprising-- it's going to lead to ridiculously high complexity, message and time complexity.
If you really have unbridled asynchrony and weights, this is going to give you a very costly algorithm.
You're going to see some exponential is going to creep in there.
Here's the algorithm for the asynchronous Bellman-Ford algorithm.
Everyone keeps track of their parent.
Their conjecture distance, and they have, now, messages that they're going to send to the neighbors.
Let's say you have a queue because there could be successive estimates.
We'll have a queue there.
The key step, the relaxation step, is what happens when you receive a new estimate of the best distance.
This is weighted distance, now, from a neighbor.
Well, you look at that distance plus the weight of the edge in between, and you see if that's better than your current distance, just like synchronous Bellman-Ford.
And if it is, then you improve your distance, reset your parent, and send the distance out to your neighbors.
It's exactly like the synchronous case, but we're going to be running this asynchronously.
And since you're going to be correcting every time you see a new estimate, this is actually going to handle both kinds of corrections, whether it comes because of a many hop path with a smaller weight, or whether it just comes because of the asynchrony.
Whenever you get a better estimate, you're going to make the correction.
Is it clear this is all you really need to do in the algorithm, just what's in this code?
That's the received, and then the rest of the algorithm is just you send it out when you're ready to send.
And then we have the same issue about termination, there's no terminating actions.
We'll come back to that.
It's really hard to come up with invariants and timing properties, now, for this setting.
You can certainly have an invariant like the one that we just had for asynchronous breadth-first search.
You can say that at any point, whatever distance you have is an actual distance that's achievable along some path, and the parent is correct.
But we'd also like to have something that says, eventually, you'll get the right distance.
You want to state a timing property that says, fine, if you have an at most r hop path, by a certain time, you'd like to know that your distance is at least as good as what you could get on that path.
The problem is what are you going to have here for the amount of time?
How long would it possibly take in order to get the best estimate that you could for a path of at most r hops?
A guess?
I was able to calculate something reasonable for the breadth-first search case, but now this is going to be much, much worse.
It's not obvious at all.
It's actually going to depend on how many messages could pile up in a channel, and that can be an awful lot, an exponential number-- exponential in the number of nodes in the network.
I'm going to produce an execution for you that can generate a huge number of messages, which then will take a long time to deliver and delay the termination of the algorithm.
First, let's look at an upper bound.
What can we say for an upper bound?
Well, there's many different paths from v 0 to any other particular node.
We might have to traverse all the simple paths in the graph, perhaps.
And how many are there?
Well, as an upper bound, you could say order n factorial for the number of different paths that you can traverse to get from v 0 to some particular other node.
That's exponential in n. Certainly, it's order n to the n. This says that the number of messages that you might send on any channel could correspond to doing that many corrections.
This can blow up your message complexity into n to the n times the number of edges, and your time complexity n to the n times, n times d, because on every edge you might have to wait for that many messages, corrected messages, sitting in front of you.
That seems pretty awful.
Does it actually happen?
Can you actually construct an execution of this algorithm where you get exponential bounds like that?
And we'll see that, yes, you can.
Any questions so far?
Here's a bad example.
This is a network, consists of a sequence of, let's say, k plus 1-- k plus 2, I guess, nodes, in a line.
And I'm going to throw in some little detour nodes in between each consecutive pair of nodes in this graph, and now let me play with the weights.
Let's say on this path, this direct path from v 0 to vk plus 1, all the weights are 0.
That's going to be the shortest path, the best weight path, from v 0 to vk plus 1.
But now I'm going to have some detours.
And on the detours I have two edges, one of weight 0 and the other one of weight that's a power of 2.
I'm going to start with high powers of 2, 2 to the k minus 1, and go down to 2 to the k minus 2, down to 2 of the 1, to the 0.
See what this graph is doing?
It has a very fast path in the bottom, which you'd like to hear about as soon as you can.
But, actually, there is detours which could give you much worse paths.
Let's see how this might execute to make a lot of messages pile up in a channel.
My claim is that there's an execution of that network in which the last node, bk, sends 2 to the k messages to the next node vk plus 1.
He's really going to send an exponential number of messages corresponding to his corrections.
He's going to keep making corrections for better and better estimates.
And if all this happens relatively fast, that just means you have a channel with an exponential number of messages in it, emptying that will take exponential time.
You have an idea how this might happen?
How could this node, bk, get so many successively improved estimates, one after the other?
Well, what's the biggest estimate it might get?
Yeah?
2 to the k minus 1
It could get 2 to the k minus 1?
Well, let's see
Or [INAUDIBLE]
It could do that
Then it's 2 to the k
And it could do that
Plus 2 to the k
Plus 2 to the k minus 2, plus all the way down to plus 2 to the 0.
You could be following this really inefficient path, just all the detours, before the messages actually arrive on the edges on the spine
[INAUDIBLE]
Yeah, so you follow all-- oh, you said 2 to the k, minus 1, parenthesis.
Yeah, that's exactly right
It's all right.
[INAUDIBLE]
We can't parenthesize our speech.
All right, so the possible estimates that bk can take on are actually 2 to the k-- as you said, 2 to the k minus 1, which you would get by taking all of the detours and adding up the powers of 2.
But it could also have an estimate, which is 2 to the k minus 2 or 2 to the k minus 3.
All of those are possible.
Moreover, you can have a single execution of this asynchronous algorithm in which node bk actually acquires all those estimates in sequence, one at a time.
How might that work?
First, the messages travel all the detours.
Fine, bk gets 2 to the k minus 1.
Then, there's a message traveling-- well this guy sends a message on the lower link.
This guy has only heard about the messages on the detours up to that point.
But he sends that on the lower link, which means you kind of bypass this weight of one.
bk gets a little bit of an improvement, which gives it 2 to the k minus 2 as its estimate.
What happens next?
Well, we step one back, and the node from node k minus 2 can send a message on the lower link, which has weight 0, to bk minus one.
But that corrected estimate might then traverse the detour to get to node bk.
If you get the correction for node bk minus 1, but then you follow the detour, you haven't improved that much.
You've just improved by one.
This way, you get 2 to the k minus 3 as the new estimate.
But then, again, on the lower link, the message eventually arrives, which gives you 2 to the k minus 4.
You see the pattern sort of?
You're going to be counting down in binary by successively having nodes further to the left deliver their messages, but then they do the worst possible thing of getting the information to bk.
He has to deal with all those other estimates in between.
If this happens quickly, what you get is a pile up of an exponential number of search messages in one channel.
And then that information has to go on to the next node or the rest of the network, whatever.
It's going to take an exponential amount of time, in the worst case, to empty that all out.
This is pretty bad, but the algorithm is correct.
And so how do you learn when everything is finished, and how does a process know when it can output its own correct distance information?
How can we figure out when this is all stabilized?
Same thing as before, we can just do a convergecast.
I mean this is more corrections, but it's still the same kind of corrections.
We can convergecast and, eventually, this is going to convergecast all the way up to the root.
And then the root knows it's done and can tell everyone else.
A moral here-- you've had a quick dose of a lot of asynchrony-- yeah, if you don't do anything about it and you just use unrestrained asynchrony, in the worst case, you're going to have some pretty bad performance.
The question is, what do you do about that?
There are various techniques.
And if you want to take my course next fall, we'll cover some of those.
I'll say a little bit about the course.
It's a basic-- it's a TQE level, basic grad course.
We do synchronous, asynchronous, and some other stuff where the nodes really know about something about time.
Here's some of the synchronous problems-- some like the ones you've already seen.
Building many other kinds of structures in graphs, and then we get into fault tolerance.
There's a lot of questions about what happens when some of the components can fail, or they're even malicious, and you have to deal with the effects of malicious processes in your system.
And then for asynchronous algorithms, we'll do not only individual problems like the ones you've just seen, but some general techniques like synchronizers, notion of logical time-- that's Leslie Lamport's first and biggest contribution.
He one the Turing Award last year-- other techniques, like taking global snapshots of the entire system while it's running.
In addition to talking about networks, as we've been doing this week, we'll talk about shared memory, multi-processors accessing shared memory.
And solving problems that are of use in multiprocessors, like mutual exclusion.
And again, fault tolerance.
Fault tolerance then gets us into a study of data objects with consistency conditions, which is the sort of stuff that's useful in cloud computing.
If you want to have coherent access to data that's stored at many locations, you need to have some interesting distributed algorithms.
Self stabilization-- if you plunge your system into some arbitrary state, and you'd like it to converge to a good state, that's the topic of self stabilization.
And depending on time, there's things that use time in the algorithms, and the newer work that we're working on in research is very dynamic.
You have distributed algorithms where the network itself is changing during the execution.
That comes up in wireless networks, and lately we're actually looking at insect colony algorithms.
What distributed algorithms do ants use to decide on how to select a new nest when the researchers smash their old nest in the laboratory?
That kind of question.
That's it for the distributed algorithms week.
And we'll see you-- next week is security?
OK, yeah.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
Good morning everyone.
I see a lot of tired faces.
I'm not tired.
Why are you tired?
[LAUGHTER] I only lecture half the time.
You guys take the class all the time.
So today's lecture is about hash functions.
And you may think that you know a lot about hash functions, and you probably do.
But what we're going to do today is talk about really a completely different application of hash functions, and a new set of properties that we're going to require of hash functions that I'll elaborate on.
And we're going to see a bunch of different applications to things like password protection, checking the integrity of files, auctions, and so on and so forth.
So a little bit of a different lecture.
Both today and on Thursday I'm going to be going to be doing cryptography and applications, not too much of algorithms.
But we will do a little bit of analysis with respect to whether properties are satisfied, in this case by hash functions or not.
So let's just dive right in.
You all know what hash functions are.
There's no real change in the definition.
But the kinds of hash functions that we're going to be looking at today are quite different from the simple hash functions, like taking a mod with a prime number that we've looked at in the past.
And the notion of collisions is going to come up again, except that again we're going to raise the stakes a little bit.
So a hash function maps arbitrary strings-- let me do this right.
So you're not making a statement about the length of the string.
You will break it up, even if you had a string of length 512, or maybe it was 27, you do want to get a number out of it.
In a specific range there's going to be a number of bits associated with our hash functions.
And previously we had a number of slots associated with the output of the hash function.
But the input could be arbitrary.
And these arbitrary strings of data are going to get mapped, as I just said, to a fixed length output.
And we're going to think about this fixed length as being a number of bits today, as opposed to slots in the hash table.
Because we really aren't going to be storing a dictionary or a hash table in the applications we're going to look at today.
It's simply a question of computing a hash.
And because the fixed length output is going to be something on the order of 160-bits, or 256-bits, there's no way that you could store two arrays to 160 elements in a hash table, or even two arrays to 64 really.
And so we're going to just assume that we're computing these hashes and using them for certain applications.
I just wrote output twice I guess.
So map it to a fixed length output.
We want to do this in a deterministic fashion.
So once we've computed the hash of a particular arbitrary string that is given to us, we want to be able to repeat that process to get the same hash every time.
We want to do this in a public fashion.
So everything is public.
There's no secrecy.
There's keyed hash functions that we won't actually look at today, but maybe in passing I'll mention it next time.
We're not looking at keyed hash functions here.
There's no secrets in any of the descriptions of algorithms or techniques I'm going to be describing today.
And we want this to be random.
We want it to look random.
True randomness is going to be impossible to achieve, given our other constraints.
But we're going to try and approximate it with pseudo-randomness.
But we'd want it to look random, because we are interested-- as we were in the case of dictionaries and the regular application of hash functions-- we are interested in minimizing collisions.
And in fact we're going to raise the stakes really high with respect to collisions.
We want it to be impossible for you, or anyone else, to discover collisions.
And that's going to be an important property of collision resistance that obviously is going to require randomness.
And those are the three things we want, deterministic, public, and random.
And so just from a function description standpoint you have 0, 1 star here, which implies that it's an arbitrary length strength.
And we want to go to 0, 1 d. And this is a string of length d. So that means that you're getting d-bits out from your hash function.
And here the length is greater than or equal to 0.
So that's it.
Not a lot that's new here.
But a few things that are going to be a little bit different.
And there's some subtleties here that we'll get to.
I want to emphasize two things, one of which I just said.
There's no secrecy, no secret keys here in the hash functions that we are describing.
All operations are public.
So just like you had your hash function, which was k mod p, and p was a prime and p was public and known to everyone who used the dictionary, everything here we are going to be talking about is public.
So anyone can compute h. And we're going to assume that this is poly-time computation-- not too surprising-- but I'm being quite flexible here.
When you look at dictionaries, and you think about using dictionaries, and using it to implement efficient algorithms, what is the assumption we kind of implicitly made-- are perhaps explicitly in some cases-- with respect to computing the hash?
Anybody?
Yeah?
Constant time?
Constant time.
We assumed-- so this is not necessarily order 1, right?
So that's important.
So we're going to-- I want to make sure you're watching.
So you're going to raise the stakes even with respect to the complexity of the hash.
And as you'll see, because of the desirable properties, we're going to have to do that.
We're going to ask for really a lot with respect to these hash functions.
Nobody can find a collision, right?
And if you have something as simple as k mod p, it's going to be trivial to find a collision.
And so these order 1 hash functions that you're familiar with aren't going to make the grade with respect to any of the properties that we'll discuss in a few minutes.
All right, so remember this is poly-time computation.
And there's lots of examples of these hash functions.
And for those of you who are kind of into computer security and cryptography already, you might have heard of examples like MD4 and MD5.
These are versions.
MD stands for message digest.
These were functions that were invented by Professor Rivest.
And they had d equals 128 way back when-- 1992, if I recall-- when they were proposed.
And these algorithms have since been broken in the sense that it was conjectured that they had particular properties of collision resistance that it would take exponential time for anybody to find collisions.
And it still kind of takes exponential time, but 2 raised to 37 is exponential at one level, but constant in another level.
So you can kind of do it in a few seconds now.
So a little bit of history.
I'm not going to spend a lot of time on this.
MD5 was used to create what was called a secure hash algorithm.
This is 160-bits.
And this is not quite broken at this point.
But that people consider it broken, or soon to be broken.
Right now the recommended algorithm is called SHA-3, secure hash algorithm version three.
And there was a contest that ran for like 18 months, or maybe even longer, that eventually was won by what turned into the SHA-3.
And they had a different name for it that I can't recall.
But it turned into SHA-3.
And what happened along the way, as we went from MD4, MD5, SHA-1 to SHA-3, is that this amount of computation that you had to do increased.
And the complexity of operations that you had to do in order to compute the hash of an arbitrary string increased to the point where-- you want to think about this as 100 rounds of computation.
And certainly order d computation, where d is the number of bits.
And perhaps even more.
So it's definitely not order 1.
So as I said a little bit of context with respect to the things that are out there.
At the end of the lecture I'll give you a sense for how these hash functions are built.
We're not going to spend a lot of time on creating these hash functions.
It's really a research topic onto itself and not really in the slope of 6.046.
What is in the scope of 6.046, and what I think is more interesting, which is what we'll focus our energy and time on, is the properties of these hash functions.
And why these properties are useful in a bunch of different apps.
And so what is it that we want?
We want a random oracle.
We want to essentially build something that looks like that, deterministic, public, random.
And we're going to claim that what we want is this random oracle which has all of these wonderful properties that I'm going to describe.
I'm going to describe the random oracle to you, and then I'm going to tell you about what the properties are.
And then unfortunately this is an ideal world and we can't build this in the real world.
And so we're going to have to approximate it.
And that's where the MD4's and the MD5's and the SHA-1's came in, OK?
So this is not achievable in practice.
So what is this oracle?
This oracle is on input x, belonging to 0,1 star.
So that could be an arbitrary string.
If x not in the book-- so there's this the book, all right?
And there's this infinite capacity book that has all of the computations that were ever done prior.
And they're always stored in the book.
And that's how we're going to get determinism.
Because this book initially gets filled in.
All of the entries in the book are filled in using pure randomness.
So you flip a coin d times to determine h of x.
So that's basically it.
And you just keep flipping.
You have to flip d times.
And so if x was 0, you flip d times, d was 160.
You flipped a coin 160 times and got a string.
If x were 1, flip 160 times, you get a different string with very high probability, obviously.
And so on and so forth.
But what you do is you have this book.
So you're going to record x h of x in the book, OK?
So at some level your hash function is this giant look-up table in the sky, right?
Actually not giant, infinite capacity look-up table in the sky.
Because you can put arbitrary strings into this.
And if it's in the book-- this is obviously the important part that gives you determinism-- then you return y, where x and y are in the book, OK?
So you get a random answer every time, except as required for consistency with previous answers.
So the very first time you see a string, or-- and the whole world can create this book.
It's public.
So if I created the book at first with a particular string, let's say Eric.
I was the string.
And I'm the one who put the entry-- x equals Eric, and h of x, h of Eric equals some random 160-bit string-- into the book, I get credit for that, right?
But if you come a nanosecond later and ask for h of Eric, you should get exactly what got put into the book when I asked for h of Eric.
And so on and so forth.
So this is true for everybody.
So this is like-- I mean basically impossible to get.
Because not only can anybody and everybody query, you have to have this ordering associated with people querying the book.
And you have to have consistency.
All right.
So everyone convinced that we can't build this?
All right.
If you took anything out of this lecture, that's what you should take.
No, no.
There's a lot more.
So we want to approximate the random oracle.
And we're going to get to that.
Obviously we're going to have to do this in poly-space as well.
So what's wrong with this?
Of course this picture is I didn't actually say this, but you'd like things to be poly-time in terms of space.
You don't want to store an infinite number-- this is worse than poly-time, worse than exponential time, because it's arbitrary strings that we're talking about here, right?
So you can't possibly do that.
So we have to do something better.
But before I get into how we'd actually build this, and give you a sense of how SHA-1 and MD5 were built-- and that's going to come a little bit later-- I want to spend a lot of time on the what is interesting, which are the desirable properties.
Which you can kind of see using the random oracle.
So what is cool about the random oracle is that it's a simple algorithm.
You can understand it.
You can't implement it.
But now you can see what wonderful properties it gives you.
And these properties are going to be important for our applications, OK?
And so let's get started with a bunch of different properties.
And these are all properties that are going to be useful for verification or computer security applications.
The first one, it's not ow, it's O, W. It's one-wayness, all right?
So one-way, or one-wayness.
And it's also called-- you're not going to call it this-- but perhaps this is a more technical term, a more precise term, pre-image resistance.
And so what does this mean?
Well this is a very strong requirement.
I mean a couple of other ones are also going to be perhaps stronger.
But this is a pretty strong requirement which says it's infeasible, given y, which is in the-- it's basically a d-bit vector, to find any x such that h of x equals y.
And so this is x is the pre-image of y.
So what does this say?
It says that I want to create a hash function such that if I give you a specific-- we call it a 160-bit string, because we're talking SHA-1 here, and that's the hash-- I'm going to have, it's going to have to be impossible for me to discover an x that produced that 160-bit string, OK?
Now if you go look at our random oracle, you realize that if you had a 160-bit string, and perhaps you have the entire book and you can read the entire book.
It's an infinite capacity book.
It's got a bunch of stuff in it.
And know that any time anyone queried the book the first time for a given x, that there was this random 160-bit number that was generated and put into the book.
And there's a whole lot of these numbers, right?
So what's going to happen is, you're going to have to look through the entire book, this entire potentially infinite capacity book, in order to figure out if this particular y is in the book or not.
And that's going to take a long time to do, potentially, OK?
So in the case where you have a random oracle you'd have to go through and find-- looking at the output hash corresponding to each of the entries in the random oracle, you're going to start matching, match, match, match, match, it's going to take you exponential time.
Well actually worse than that, given the infinite capacity of the book.
So this clearly gives you that.
Now you may not be a completely satisfied with that answer because you say well, you can't implement that.
But we'll talk a little bit, as I said, about how you could actually get this.
But what's-- I should be clear-- is that the simple hash functions that we've looked at in the past just to build dictionaries do not satisfy this, right?
So suppose I had h of x equals x square mod p. Is this one-way, given a public p?
No of course not, right?
Because I'm going to be-- it's going to be easy for me to do something.
Even though this is discrete arithmetic I could do something like, well, I know that what I have here-- actually let's do it with something that's simpler, and then I'll talk about the x squared.
If I had something as simple as x mod p, I mean that's trivially broken in terms of one-wayness.
Because I know that h of x could be viewed as the remainder.
So anything-- if this is h of x, and let's just call that y for a second, because that's what we had it out there.
Something that's a multiple of y plus the remainder-- so I could have a-- is that right?
Is that what I want?
Yeah.
No, plus y.
So I want a of-- well since I can't figure it out, why can't you?
What do I need to put in there in order to discover an x that would produce a y?
Can I write an equation?
Yeah?
Could you just write y itself?
Just y itself.
That's right.
Good point.
Just y itself in this case.
Good.
I knew you guys were smarter than me.
This proves it.
So if you just take y-- and y remember is going to be something that's 0 to p minus 1, right?
And that's it.
It just goes through, right?
So that's a trivial example, right?
Now if I put x squared in here, obviously it's not y, but I could start looking at-- what I have here is I'm going to get y that looks like x squared.
But I could take the y that I have, take the square root of that, and then start looking for x's that give me the y that I have.
Actually it's not a complicated process to try and figure out, through trial and error potentially, what an x is that produces a particular y for the kinds of hash functions that we've looked at, all right?
Now as you complicate this equation it gets harder.
Because you have to invert this set of equations.
And that's what the game is going to be when you go create one-way hash functions.
The amount of computation that you do in order to compute the y is going to increase to the point where, as I mentioned, you have 80, 100 rounds of computation, things getting mixed in.
And the hope is that you create this circuit, if you will, that has all this computation in that.
Going forwards is easy, because you've specified the multiplications and the mods and so on and so forth.
But not all of these operations have simple inverses.
And going backwards, which is what you need to do in order to break one-wayness, or discover the x given a y, is going to be harder and harder as the computations get more complex, OK?
So everyone have a sense of what one-wayness is?
So that's one-wayness.
There's four other properties, two of which are very related.
CR and TCR.
So CR is collision resistance.
It's infeasible to find x and x prime such that x not equal to x prime, and h of x equals h of x prime, which is of course a collision.
OK?
And that just says you have this crazy hash function where you can't discover collisions.
Well it would be absolutely wonderful.
In fact that's what we wanted when we built dictionaries.
But why don't we use SHA-3 in dictionaries?
Why don't we use SHA-3 in dictionaries?
Yeah?
Because it's more complicated than we need
Yeah, it's horribly slow, right?
It would take longer to compute the hash than access the dictionary, when you actually had a reasonable dictionary that maybe had some collisions.
I mean you just go off and you have a linked list, you can afford a few collisions, what's the big deal, right?
So it just doesn't make any sense to use this level of heavyweight hash function, even if it satisfies collision resistance-- which some of these are conjectured to do-- for the applications we've looked at.
But there'll be other apps where collision resistance is going to be important.
So that's collision resistance.
And then there's-- TCR is target collision resistance.
It's a weaker form-- so sometimes people CR strong collision resistance, and TCR weak occlusion resistance.
We'll use CR and TCR here.
And this says it's infeasible, given x-- so there's a specific x that you want to find a collision for, as opposed to just finding a pair that goes once to x and x prime.
And any pair would suffice to break the collision resistance property.
But TCR says is I'm going to give you a specific x.
And I want you to find an x prime who's hash collides with the hash of x, OK?
That's TCR.
OK that's TCR for you.
And that just to be clear, I think you probably all got this, obviously we want this here because we have a deterministic hash function.
And it's a trivial thing to say that if you had x, and you had x again, that you get the same hash back from it.
That's a requirement, really.
So we want two distinct x and x primes that are not equal that end up colliding.
That's really what a collision is.
And so you see the difference between CR and TCR?
Yup?
Yeah?
Are we to assume that given an x it's very easy to get the h of x back?
So the question was, given an x, it's poly-time computation to get h of x.
Absolutely.
Public poly-time computation given an x to get h of x.
So going this way is easy.
Going this way-- I ran out of room-- hard.
OK?
So does that mean that TCR is basically the same as 1?
No, no, no, absolutely not.
TCR says it's OK. You can compute this.
You can get x.
And you can get h of x.
So given x, you know that you can get h of x. I didn't actually put that in the definition.
And maybe I should have.
So given x you can always get h of x.
Remember that.
It's easy to get h of x.
So any time I say given x, you can always add it, saying given x and h of x.
So I'm given x. I'm given h of x. I obviously need to map-- I need to discover an x prime such that h of x prime equals h of x, OK?
Now you have situations where for-- it may be the case that for particular x's you can actually do this.
And that's enough to break TCR.
So you have to have this strong property that you really don't want to find collisions are for some-- even if there's a constant fraction of x's that break the TCR property, you don't like your hash function, OK?
Because you might end up picking those and go build security applications using those properties.
I want to talk a little bit about the relationship between OW, CR, and TCR.
So I'm going to get back to that.
And we're going to talking about hash functions that satisfy one property but don't satisfy the other.
And I think your question will probably be answered better, OK?
Thanks for the question.
So those are the main ones.
And really quickly, if you want to spend a lot of time on this-- but I do want to put up-- I think I'll leave these properties up here for the duration.
Because it's important for you to see these definitions as we look at the applications where we require these properties, or a subset of these properties.
But that we have pseudo randomness.
And this is simply a function of the fact that-- so this is PRF-- we know we can't build a random oracle.
And so we're going to have to do something that's pseudo-random.
And basically what we're saying here is the behavior is indistinguishable from random.
So we're going to have to use non-linearity, things that are called non-linear feedback shift registers, to create pseudo-random functions.
There's many ways that we can create pseudo-random functions.
We won't really get into that.
But obviously that's what we want.
And then the last one is a bit tricky.
And we will have an app that requires this way at the end.
But this is infeasible given h of x to produce h of x prime, where x and x prime are-- and it gets a little bit fuzzy here-- are related in some fashion, right?
So a concrete example of this is, let's say that x prime is x plus 1.
So this is a reasonable example of this.
So what this says is you're just given h of x.
It doesn't actually say anything about one-wayness yet.
But you could assume, for example, that if this was a one-way hash function, that it would be possible to get x from h of x, correct?
And let's keep that though.
Hold that thought, all right?
We're going to get back to it.
So if I'm just given the hash through some computation, it may be possible for me to create another hash, h of x prime, such that there's some relationship that I can prove or argue for between the strings that created the hashes, namely x and x prime, OK?
That's what malleability is, right?
Now you might just go off and say here's an x, here's a y, here's h of x, and here's h of y.
These look completely random.
And you might go off-- I'm being facetious here-- I say that y is x's third cousin's roommate's brother-in-law or something, right?
I mean just make something up, right?
So clearly there's got to be a strong, precise relationship between x and y.
If in fact you could do this and get y equals x plus 1, that'd be a problem, right?
But if you are-- and then you can do this sort of consistently for different x's and y's, that would absolutely be a problem, right?
But what you're really asking for-- and typically when you want non-malleability-- it's things where you have auctions, for example, where you are to be careful about making sure that you don't want to expose your bid.
And so maybe what you're doing is exposing h of x.
You don't want somebody to look at your h of x and figure out how they could beat your bid by just a little bit.
Or in case of Vickrey auctions, where the second highest bidder wins, now just be a little bit below you, right?
So that's the kind of thing that you want to think about when it comes to non-malleability, or malleability, where you want a strong relationship between two strings that are related in some ordered fashion, like x equals-- x prime equals x plus 1, or just x prime equals 2 times x.
And you don't want to be able to-- you don't want the adversary to be able to discover these new strings.
Because that would be the system, all right?
So any questions about properties?
Are we all good on these properties?
All right, because I'm going to start asking you how to use them for particular applications, or what properties are required for certain applications, OK?
One last thing before we get there.
I promised a slightly more detailed analysis of the relationships between these properties.
So let's do that.
Now if your just look at it, eyeball it, and you look at collision resistance and TCR, what can I say about the relationship between CR and TCR?
If h is CR, it's going to be TCR, right?
It's got to be TCR.
It's a strictly stronger requirement.
But not reverse.
And you can actually give a concrete example of a particular hash function that is TCR.
I'm not going to go there.
It's actually a little more involved than you might think it is, where a TCR hash function is not collision resistant.
But you can see that examples such as these should exist, simply because I have a more stringent property corresponding to collision resistance as opposed to TCR, right?
So if you're interested in that particular example, you're not responsible for it, get in touch with me and I'll point you to a, like a three-page description of an example.
So I didn't really want to go in there.
But what I do want to do is talk about one-wayness and collision resistance.
Because I think that's actually much more interesting, all right?
So if h is one-way-- any conjectures as to what the question mark is in the middle?
Can I make strong statements about the collision resistance of a hash function, if I'm guaranteed that the hash function I have is a one-way hash function, or vice versa?
Another way of putting it is, can you give me an example of, just to start with, a hash function which is one-way but not TCR, not target collision resistant?
So I'm going to try and extract this out of you.
This is somewhat subtle.
But the way you want to think about this is, let's say that h of x is OW and TCR, OK?
And so I have a bunch of inputs.
And this is the output.
And I get d-bits out.
And I've got x1, x2, to xn, OK?
Now I've given this h-- I've been given this h which is one-way and TCR.
It satisfies those properties that you have up there.
In the case of one-way, I give you an arbitrary d-bit string.
You can't go backwards and find a bunch of the xi's that produce exactly that d-bit string, all right?
So it's going to be hard to get here.
But you're allowed now to give me an example.
So this is some hash function that you can create, which may use h as well.
And h is kind of nice because it has this one-way property.
So let's say that we want to discover something where one-way does not imply TCR.
So I want to cook up a hash function h prime such that h prime is one-way, but it's not TCR, OK?
The way you want to think about this is you want to add to h. And you want to add something to h such that it's still hard-- if you add h it's still hard to go from here to there.
Because you've got to go deeper.
If you add to, for example, the inputs of h. Or you could add to the outputs of h as well, or the outputs of the current h. But you can basically go deeper, or need to go deeper in order to find the break one-wayness, in order to find an x, whatever you have, that produces the d-bit string that you have, right?
So what's a simple way of creating an h prime such that it's going to be pretty easy to find targeted collisions even, not necessarily collisions, it's pretty easy to find targeted collisions, without breaking the one-way property of h?
Yeah?
So if you have x sub i, if i odd then return h of x of i.
So that's minus 1.
So return the even group
Sure.
Yep
Given x any x of i, you can usually find another x of i that was the same output?
You can go backwards
You can't go backwards.
Yeah, that's good.
That's good.
I'm going to do something that's almost exactly what you said.
But I'm going to draw it pictorially.
And what you can do, you can do a parity, like odd and even that was just described.
And all I'll do is add a little [?
XNOR ?]
gate, which is a parity gate, to one of the inputs.
So you have and b here.
So I've taken x1, and I have a and b here.
So I've added-- I can add as many inputs as I want to this function.
Oh I should mention by the way, h of x is working on arbitrary strings.
And obviously I put in some number here that corresponds to n, which is a fixed number.
So you might ask, what the heck happened here with respect to arbitrary strings?
And there's two answers.
The first answer is, well, ignore arbitrary.
And assume that you only have n-bit strings.
And n this is really large number, right?
And that may not be particularly satisfying.
The other answer is, which is more practical, which is what's used in practice, is that typically what happens is, you do have particular implementations of hash functions that obviously need to have fixed inputs, n, for example.
And n is typically 512.
It's usually the block size.
And you chunk the input up into five 12-bit blocks.
And typically what you do is, you take the first five 12-bits, compute the hash for it.
And then you can do it for the remaining blocks.
And then you can hash all of them together, all right?
So there's typically more invocations.
I don't really want to get into it.
But there's typically more invocations of h when the input would be 2 times n, or 3 times n, all right?
So we don't really need to go there for the purposes of this lecture.
But keep that in mind.
So we'll still stick with our arbitrary string requirement.
So having said that, take a look at this picture.
And see what this picture implies.
I have an h prime that I've constructed, right?
Now if I look at h prime, and I give you an output for h prime-- so h prime now has, it's a function of a and b, and x2 all the way to xn, right?
So it's got an extra input.
If I look at h prime, and I look at the output of h prime that is given to me, and I need to discover something that produces that, it is pretty clear that I need to figure out what these values are, all right?
And I need to know what the parity of a and b is.
And maybe I don't need to know exactly what a and b are, but I absolutely need to know what the parity of a and b are, because that's x1.
And the one-way I'd break would require me to tell you what the value of x1 is, and the value of x2, and so on and so forth.
So it's pretty clear that h prime is one-way, right?
Everybody buy that?
h prime is one-way.
But you know what?
I've got target collisions galore, right?
All I have to do is flip-- I have a equals 1 and b equals 1.
And I have a equals 0 and b equals 0.
They're going to give me the same hash, right?
So trivial example, but that gets to the essence of the difference between collision resistance and one-wayness, target collision resistance and one-wayness, all right?
So this is one-way but not TCR, simply because a equals 0, b equals 0 for arbitrary x's produce the same thing as a equals 1 and b equals 1, right?
So those are collisions.
So admittedly contrived.
But it's a counterexample.
Counterexamples can be contrived.
It's OK. All right.
So that was what happens with that.
Let's look at one more interesting thing that corresponds to the other way, right?
So what I want to show is that a TCR does not imply one-wayness.
OK, so now I want an example where it is clear that I have target collision resistance, because I can just assume that.
And we're going to use the same strategy.
I'm just going assume that I have an h that's target collision resistant.
And I'm going to try and cook up an h prime that is not one-way.
So I'm going to assume that in fact h is TCR and OW.
And I'm going to take away one of the properties.
And if I take it one of the properties I have a counterexample, right?
So think about how you could do this.
You have h as before.
And I want to add some stuff around it such that it's going to be easy to discover-- for a large, for a constant fraction of hashes that I've given to me, not for any old hash.
Because you can always claim that one-wayness is broken by saying I have x, I computed h of x, now I know what-- given h of x I know what x is.
I mean you can't do that, right?
So that's not breaking the one-wayness of it.
It's when you have an h of x and this is the first time you've seen it, you're trying to find what x is, right?
So how would you-- how would you set it up so you break the one-wayness of h without necessarily breaking the target collision resistance of the overall hash function that you're creating?
And you have to do something with the outputs, OK?
You have to do something.
This is a little more involved.
It's not as easy as this example.
It's a little more involved.
But any ideas?
Yeah, go ahead
So x is less than b returns x.
If x is greater than b, return [INAUDIBLE]
Beautiful.
Right.
What color did you get last time?
Blue
You got a blue last time?
All right.
Well you get a purple.
You have a set.
Actually we have these red ones that are precious, that are-- no, we don't.
We chose not to do red.
I don't know.
There was some subliminal message I think with throwing red Frisbees that we didn't like.
But OK.
So thank you.
And h of x is simply something where I'm going to concatenate a zero to the x value and just put it out.
And clearly this is breaking one-wayness because I'm just taking the input, I'm adding a zero to it, and shipping it out.
So it's going to be easy to go backwards, right?
And this only happens if x is less than n, as the gentleman just said.
Less than or equal to n in terms of the input length, OK?
Otherwise I'm going to do h of x.
So this is good news.
Because I'm actually using the hash function in the case where I have a longer input string.
This is bad news for one-wayness because I'm just piping out the input.
And so if I get an x, and I see what the x is out here, and let's just say for argument's sake that-- you could even say that n is going to be something that is less than d, which is the final output, which has d-bits.
And so if you see something that h prime produces that's less than d-bits you instantly know that you can go backwards and discover what input produced that for the h prime, right?
Because you just go off and you go backwards.
This is what it tells you.
Now on the other hand if it's larger obviously you can't do that.
But there's a whole lot of combinations that you can do that for.
So this breaks one-wayness, OK?
Now you think about TCR.
And what you want a show of course is that this maintains TCR.
So that's the last thing that we have to show.
We know that it breaks one-wayness.
But if it broke TCR we don't quite have our example.
So we want to show that it actually maintains TCR, which is kind of a weakish property that we need to maintain.
And the reason this maintains TCR is that there's really only two cases here obviously, corresponding to the if statement.
And it's pretty clear that if x is less than or equal to n, clearly different x's produce different h prime x's, correct?
Because I'm just passing along the x out to the output.
So if x is less than n I am going to get different hashes at the output.
I'm just passing them out.
So that's easy.
And for the other case, well I assume that h of x was CCR, correct?
Because that was the original assumption, that I had h, which was CCR.
So in both cases TCR is maintained because else h of x maintains TCR, all right?
So again, a bit of a contrived example to show you the difference between these different properties so you know not to mix them up.
You know what you want to ask for, what is required when you actually implement an application that depends on particular properties.
All right?
Any questions so far about properties or any of these examples?
We're going to dive in to using them.
OK.
So start thinking computer security.
Start thinking hackers, protecting yourself against the bad guys that are out there who are trying to discover your passwords, trying to corrupt your files, generally make your life miserable.
And we'll start out with fairly simple examples, where the properties are somewhat obvious, and graduate to this auction bidding example which should be sort of the culmination of at least this part of the lecture.
And depending on how much time I have I'll tell you a little bit about how to implement hash functions.
But I think these things are more important from a standpoint of giving you a sense of cryptographic hashes.
All right.
Password storage.
How many of you write your password in an unencrypted text file and store it in a readable location?
There you go, man.
Thank you for being honest.
And I do worse.
Not only do I do that, I use my first daughter's name for four passwords.
I won't tell you what the name is.
So that's something that we'd like to fix, right?
So what do real systems do?
Real systems cannot protect against me using my first daughter's name as a password, right?
So there's no way you can protect against that.
But if I had a reasonable password, which had reasonable entropy in it-- so let's assume here that we have reasonable entropy in the password.
And you can just say 128-bits.
And it's not a lot, right?
128-bits is 16 characters, OK?
And you don't have to answer this-- how many of you have 16 characters in your password?
Oh I'm impressed.
OK.
So you've got 128-bits of entropy.
But the rest of you, forget it.
This is not going to help you, OK?
But what I want, assuming you have significant entropy in your password-- because otherwise, if there's not enough entropy you can just enumerate all possible passwords of eight letters.
And it's not that much.
It's 2 raised to 50, what have you.
And you can just go off.
And none of these properties matter.
You just-- you have your h of x.
It's public.
We'll talk about how we use that in a second.
But clearly if the domain is small you can just enumerate the domain.
So keep that in mind.
I talked about h of x, and it's obviously going to be relevant here.
But suppose I wanted to build a system, and this is how systems are built, ETC slash password file, assuming you have long passwords it does it this way, otherwise it needs something that's called a salt.
But that's 6, 8, 57 and we won't go there.
So we just assume a large entropy.
What is it that a system can do?
What can it store in order to let you in, and only let you in when you type your password, and not let some bogus password into the system?
Or somebody with a bogus password into the system.
Yeah, go ahead
If you capture the password when you enter it and compare it to what's stored--
Yes
If it's a one-way hash you know you have what the correct password is
That's exactly right.
That's exactly right.
So it's a really simple idea, a very powerful idea.
It, as I said, assumed that the entropy-- and I'm belaboring the obvious now-- but it is important when you talk about security to state your assumptions.
But you do not store password on your computer.
And you store the hash of the password.
Now why do I store my password on the computer?
Because this is so inconvenient, right?
So this is what the system does for me.
But the fact of the matter is, if I lose my password, this doesn't help me.
Because what the system wants you to do is choose a password that is long enough, and the h is one-way.
So anybody who discovers h of PW that is publicly readable cannot discover PW, all right?
That's what's cool about this.
How do you let the person log in?
Use h of PW to compare against h of PW prime, which is what is entered, where PW prime is the typed password.
And clearly what we need is the disclosure of h of PW should not reveal PW.
So we definitely need one-wayness.
What about-- what about collision resistance?
Our target collision resistance?
Think practitioner now, right?
Are we interested in this hash function being collision resistant?
What does that mean in this case?
Give me the context in this particular application?
Yeah, go ahead
It means that someone entering a different password will have the same hash [INAUDIBLE]
Exactly.
So it means that what you have is a situation where you do not reveal-- and so what might happen is that h of PW prime equals h of PW.
But h of PW equals h of PW prime.
But PW is not equal to PW prime.
What you have is a false positive.
Someone who didn't know your password but guessed right-- and this is a 128-bit value, and they guessed right-- is going to get it.
You don't particularly care of the probability of this occurrence.
It's really small.
Typically you're going to have systems that lock you out if you try 10 tries that occurs one, two, wrong passwords, right?
So really in systems you do not require-- you do want to build systems that have minimal properties with respect to the perimeters that are used.
So from a system building standpoint just require OW.
Don't go overboard.
Don't require collision resistance or TCR, OK?
Let's do a slightly different example.
Also a bit of a warm-up for what's coming next, which is a file modification detector.
So for each file F, I'm going to store h of F. And as securely.
So you assume that this means that h of F cannot be modified by anybody, h of F itself.
And now we want to check if F is modified by re-computing h of F. Which could be, this could be modified.
So this could actually be F prime.
You don't know that.
You have a file.
It's a gigabyte.
And somebody might have tampered with one of the bits in the file.
All you have is a d-bit digest that corresponds to h of F that you stored in a secure location.
And you want to check to see, by re-computing h of F, the file that is given to you, and comparing it with what you've stored, the h of F that you've stored.
And so what property do we need in order to pull this off?
Of hash functions.
What precisely do we need to pull this off?
What is the adversary trying to do?
And what is a successful break?
A successful break is if an adversary can modify the file and keep h of F the same, right?
That would be a successful break, right?
Yup.
Go ahead

TCR?
Yeah, absolutely.
You need TCR.
So you want to modify the file.
So you're given that the file-- the adversary is given the file, which is the input to the hash, and is going to try and modify-- modify the file, right?
So let's do a couple more.
And we're going to advance our requirements here a little bit.
So those two are basic properties.
I want to leave this up there.
We're going to do something that corresponds to digital signatures.
So digital signatures are this wonderful invention that came out of MIT in a computer science laboratory-- again, Ron Rivest and collaborators-- which are a way of digitally signing a document using a secret key, a private key.
But anybody who has access to a public key, so it could be pretty much anybody, could verify the authenticity of that signature, right?
So that's what a digital signature is.
So we're going to talk about public cryptography on Thursday, in terms of how you could build systems or encryption algorithms that are public key algorithms.
But here I'll just tell you what we want out of them.
Essentially what we have here in the case of signatures, we actually want to talk about encryption here, are-- there's two keys associated with a public key system.
Anybody and everybody in the system would have a public key that you can put on your website.
And you also have a secret key-- that's like your password-- that you don't want to write down, you don't want to give away, because that's effectively your identity.
And what digital signatures respond to are that you have two operations.
You have signing and verification.
So signing means that you create a signature sigma that is the sign using your private key, your secret key, off a message M. So you're saying this is this message, it came from me, right?
That's what signing means.
You have this long message and you sign it at the bottom.
You're taking responsibility for the contents of that message.
And then verification is you have M sigma and a public key.
And this is simply going to output true or false.
And so the public key should not reveal any information about the secret key.
And that's the challenge of building PKI systems, that we'll talk about in some detail next time.
But we don't need to think about that other than acknowledging it today.
So the public and private key are two distinct things, neither one of which reveals anything about the other.
Think of them as completely distinct passwords.
But they happen to be mathematically related.
That's why this whole thing works.
And that mathematical relationship we'll look at in some detail on Thursday.
But having said that, take a look at what this app is doing for us, right?
This is a security application.
And I haven't quite gotten to hash functions yet.
But I'll get to it in just a minute.
But what I want to do is emphasize that there's two operations going on.
One of which is a signature, which is a private signature, in the sense that it's private to me, if I'm Alice.
Or private to Alice.
And you're using secret information on this public message, M, because that's going to be publicized.
And you're going to sign the public message.
And then anybody in the world who has access to Alice's public key is going to be able to say, oh I'm looking at the signature, which is a bunch of bits.
I'm looking at the message, which is a whole lot of bits.
And I have this public key, which is a bunch of bits.
And I'm going to be able to tell for sure that either Alice signed this message, or Alice did not sign this message.
And the assumption here is that Alice kept her private key secret.
And of course, what I just wrote there, that the public key does not reveal anything about the secret key, OK?
So that's digital signatures for you, in a nutshell.
And when you do MIT certificates you're using digital signatures a la Rivest-Shamir-Adleman, the RSA algorithm.
So you're using this all the time, when you click on 6.046 links, for example.
And what happens is M is typically really large.
I mean it could be a file, right?
It could be a large file.
And you don't necessarily want to compute these operations on large files.
So for convenience, what happens is you end up hashing the file.
And for large M it's easier to sign h of M. And so replace the M's that you see here with h of M, all right?
So now that we're given that we're going to be doing h of M in here, think about what we wanted to accomplish with M, right?
I told you what we wanted to accomplish with M. There's a particular message.
I'm Alice.
I'm going to keep my secret key secret.
But I want to commit to signing this message M, all right?
And I want to make sure that nobody can pretend to be me who doesn't know my secret key.
And nobody does.
So if I'm going to be signing the hash of the message, now it comes down to today's lecture.
I'm signing the hash of the message h of M. What property do I require of h in order for this whole thing to work out?
Yeah, go ahead
Is it non-malleability?
Non malleability, but even before that-- suppose-- absolutely, but non-malleability is kind of beyond one of these properties over on the right.
You're on the right track, right?
So do you want to give me a different answer?
You can give me a different answer
Oh, I'm not sure
OK. What?
Yeah, back there
I think you wanted to one-way because otherwise you could take that signature and find another message that you could credit
I can make M public.
I can make M-- M can be public.
And h of M is public.
So one-wayness is not interesting for this example if M is public.
And we can assume that M eventually gets public.
Because that's the message I'm signing, right?
I can also put M out.
So I want the relationship-- I want you to focus on the relationship between h of M and M and tell me what would break this system.
And you're on the right track.
Yeah, go ahead.
Or way back there.
Yeah, sorry about that

TCR.
Why TCR?
[INAUDIBLE]
So I have M. So what happens here-- I should write this out.
I'm given-- as an adversary I have M and h of M. It is bad if Alice signs h of M, but Bob claims Alice signed M prime.
Because h of M equals h of M prime, right?
That is bad.
So the M is public-- could you stand up?
M is given.
There's a specific M, and a specific h of M in particular, that has been exposed.
And h of M is what was used for the signature.
So you want to keep h of M the same.
It's a specific one.
So it's not collision resistance, it's target collision resistance, because that's given to you.
And you want to keep that the same.
But you want to claim that oh, you promised me $10,000, not $20, right?
If you can do that, you signed saying you want to pay $10,000, not $20, then you've got a problem.
So your thing is very close.
It's just that it doesn't need to be a strong relationship between the 10,000 or the 20.
I mean I give you a concrete example of that.
But it could be more, it could be less.
Anything that is different from what you signed, be it with the numerical relationship or not, would cause a problem and break this scheme, all right?
Are we good?
All right, one last example, the most interesting one.
And as I guessed I'm probably not going to get to saying very much about how cache functions are implemented.
But maybe I'll spend a minute or two on it.
So let's do this example that has to do with commitments.
Commitment is important, right?
You want to commit to doing things.
You want to keep your promises.
And in this case we have a legal requirement that you want to be able to make people honor their commitments, and not weasel their way out of commitments, right?
And we want to deal with this computationally.
And let's think about auctions.
So Alice has value x, e.g.
an auction bid.
Alice computes what we're going to call C of x, which is a commitment of x, and cements it, right?
C of x, C of x is-- let's assume that the auctioneer, and perhaps other auctionees as well, see C of x.
You have to submit it to somebody, right?
So you can assume that that's exposed.
And what is going to happen is, when bidding is over Alice is going to open-- so this is-- C of x can be thought of as sealing the bid.
So that's the commitment.
You're sealing the-- you're making a bid and you're sealing it in an envelope.
You've committed to that.
That's obviously, what happens in real life without cryptography, but we want to do this with cryptography, with hash functions.
And so now Alice opens C of x to reveal x.
So she has to prove that in fact x was her bid.
And that it matches what she sealed.
When you open it up, think about it conceptually from a standpoint of what happens with paper, and then we have to think about this computationally and what this implies, right?
So again I'll do a little bit of set up.
And then we have start talking about the properties that we want for this particular application.
So there are a bunch of people who are doing bidding for this auction.
I don't-- I want to be the first-- I don't want to spend a lot of money.
But I want to win.
All of us are like that, right?
If I know information about your bid, that is obviously a tremendous advantage.
So clearly that can't happen, right?
If I know one other person's bid I just do plus 1 on that.
If I know everybody else's I just do plus 1 on the maximum.
So clearly there's some secrecy that's required here, correct?
So C of x is going to have to do two things.
It can't reveal x.
Because then even maybe the auctioneer is bad.
Or other people are looking at this.
And you can just assume that C of x is-- the C of x's are all public.
But I also need a constraint that's associated with C of x that corresponds to making sure Alice is honest, correct?
So I need to make Alice commit to something, right?
So what are the different properties of the hash function that if I use h of x here, that I'd want h to satisfy in order for this whole process to work like it's supposed to work with paper and envelopes?
Yeah, go ahead
It has to be one-way [INAUDIBLE]
It has to be one-way.
And explain to me-- so I want a description of it has to be one-way, because why?
Because you want all the c x's to be hidden from all the other options
Right.
C of x should not reveal x, all right?
All right.
That's good.
Do you have more?
It has to be collision resistant.
OK.
I guess.
A little bit more.
You're getting there.
What-- why is it collision resistant?
Because you want to make sure that Alice, when she makes a bid that she commits that bid.
If she's not going to resist it then she could bid $100 and then find something else
That's exactly right.
So CR, because Alice should not be able to open this in multiple ways, right?
And in this case it's not TCR in the sense that Alice controls what her bids are.
And so she might find a pair of bids that collide, correct?
She might realize that in this particular hash function, you know $10,000 and a billion dollars collide, right?
And so she figures depending on what happens, she's a billionaire, let's assume.
She's going to open the right thing.
She's a billionaire, but she doesn't necessarily want to spend the billion, OK?
So that's that, right?
But I want more.
Go ahead
You don't want it to be malleable.
Assuming that the auctioneer is not honest because you don't want to accept a bribe from someone and then change everyone else's bid to square root of whatever they bid
That's exactly right.
Or plus 1, which is a great example, right?
So there you go.
I ran out of Frisbees.
You can get one next time.
So yeah, I don't need this anymore.
You're exactly right.
There's another-- it turns out it's even more subtle than what you just described.
And I think I might be able to point that out to you.
But let me just first describe this answer, which gives us non-malleability.
So the claim is that you also want non-malleability in your hash function.
And the simple reason is, given C of x-- and let's assume that this is public.
It's certainly public to the auctioneer, and it could be public to the other bidders as well.
Because the notion of sealing is that you've sealed it using C of x.
But people can see the outside of the envelope, which is C of x.
So everyone can see C of x.
You still want this to work, even though all other bidders can see C of x.
So given C of x, should not be possible to produce C of x plus 1.
You don't know x is.
But if you can produce C of x plus 1, you win, all right?
And so that's the problem.
Now it turns out you now say OK, am I done?
I want these three properties.
And I'm done, right?
There's a little subtlety here which these properties don't capture.
So that's why there's more here.
And I don't mean to titillate, because I'll tell you what is missing here.
But let's say that I have a hash function that looks like this.
And this here is non-malleable.
It is collision resistant.
And it's one-way, all right?
So h of x has all these wonderful properties, all right?
I'm creating an h prime x that looks like this, which is a concatenation of h of x, and giving away the most significant bit of x, which is my bid, right?
I'm just giving that away, right?
The problem here is that we haven't really made our properties broad enough to solve this particular application to the extent that there's contrived cases where these properties aren't enough, OK?
And the reason is simple.
h prime x is arguably NM, CR, and OW.
And I won't go into to each of those arguments.
But you can think about it, right?
If I'm just giving you one bit, there's 159 others, there's a couple of hundred others, whatever it is that I have in the domain.
It's not going to be invertible.
h prime x is not going to be invertible if h of x is not invertible.
h prime x is not going to be breakable in terms of collision resistance if h of x is not breakable, and so on and so forth.
But if I had a hash function like that, is it a good hash function for my commitment application?
No, obviously not.
Because if I publicize this hash function-- remember, everything is public here with respect to h and h prime-- you are giving away the most significant that corresponds to your bid in this particular hash function, right?
So you really need a little bit more than these for secrecy, for true secrecy.
But in the context of this example, I mean it's common sense that you would not use the hash function like that, right?
So it's not that there's anything profound here.
It's just that I want to make sure that you understand the nuances of the properties that we're requiring.
We had all the requirements corresponding to the definitions of NM and CR and OW.
And you need a little bit more for this example, where you have to say something, perhaps informally, like the bits of your auction are scrambled in the final hash output, which most hash functions should do anyway, and h of x will definitely do.
But you kind of unscrambled it by adding this little thing in here, corresponding to the most significant thing, all right?
So I'll stop with that.
Let me just say that the operation-- or sorry, the work involved in creating hash functions that are poly-time computable is research work.
People put up hash functions and they get broken, like MD4 was put up in '92 and then got broken, SHA-1 and so on and so forth.
And so I just encourage you to look up SHA-3 and just take a quick scan and what the complexity of SHA-3 is with respect to computing the hash given an arbitrary string, all right?
I'll stick around for questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Good morning, everyone.
So more of the same in terms of cryptography and cryptographic techniques similar to Tuesday's lecture.
So if you liked it, you'll like this one.
If you didn't like it, well, it's going to be more of the same, so sorry.
But what we're going to do today is do a couple of things a little bit differently.
We're going to talk about encryption.
So we talked about hashing, which, of course, you know about from the use of dictionaries.
We haven't really talked about encryption in 046 or even in 006 previously.
But we look at two different kinds of encryption algorithms.
We spend a little bit of time on symmetric key encryption, which is really something that was used in encoding machines including the enigma from World War II that you probably saw if you saw The Imitation Game.
So there you have a shared secret.
And it's that single, what's called, symmetric shared secret.
Both parties know the secret.
And very quickly, we'll talk about what it means to actually exchange a secret.
So we'll talk about key exchange.
And then I'll move to asymmetric key encryption, which I alluded to a little bit when we talked about digital signatures last time.
I talked about public keys and private keys.
But we're going to actually look at a couple of different public key encryption algorithms today.
The classical algorithm, the very first algorithm really that stayed secure, the RSA algorithm.
It stands for Rivest, Shamir, and Adleman, the initials of the three inventors, invented at MIT in 1977 and still in use today.
And the last part of today's lecture is going to be looking at hardness.
So in encryption, if you don't know the secret key, it should be hard for an adversary to discover what the message corresponds to.
The adversary sees what's called the cipher text, which is the encrypted text.
The adversary does not know the secret key.
If he or she knows the secret key, game over.
But assuming that adversary does not know the secret key, it should be computational hard to discover the message, right?
That makes sense.
So there's clearly a relationship between this hardness and NP-complete problems, which are computationally hard.
And it's not surprising that people have tried to build what are called cryptosystems, which are essentially cryptographic techniques, based on the hardness of NP-complete problems, including graph coloring and knapsack and so on and so forth.
But it turns out there's a real subtle difference between the kinds of problems that have been useful in terms of building secure cryptosystems like RSA and NP-complete problems.
And we'll talk about that at the end of lecture.
We'll probably spend a bunch of time on that.
So let's get started with the symmetric key encryption, which is, at some level, kind of boring from a mathematical standpoint.
So we want to spend a lot of time on it.
It's definitely very useful.
It essentially assumes that there's a secret key k. And you can think of this as a 128-bit number.
Some people want it to be larger, 256.
But it's suddenly at 64.
At this point, it's probably not enough, even though 2 raised to 64 is still a fairly large number.
With parallelism and with fast computers, it's a little worrisome that the adversary only requires 2 raised to 64 work to enumerate all possible secret keys of 64-bits right?
So 122 raised to 128 is much better.
And this is shared between Alice and Bob.
So we'll think about the protagonists here as being Alice and Bob.
And they want to exchange information.
And typically, the adversary is mal for malicious.
But there's other ways you can obviously name adversaries.
So the basic equation here is really straightforward.
It's C, which is the cipher text.
So a little terminology here, cipher text.
m is the plain text or the message.
And plain text means you can just read it.
Cipher text means it's scrambled.
K is, of course, the secret key that's up here.
And the e is the encryption function.
Now, in symmetric key cryptography, basically the requirement is that you have to be able to get back the plain text from the cipher text given a public decryption function and knowledge of the secret key.
And this should be a straightforward operation, right?
So by the way, e of k m, this is a polytime computation.
It's not constant time.
It's typically linear time.
And you don't really want it to be quadratic time even, because you want to do this vary fast, streaming.
You need to send streams of messages, many gigabytes potentially, that are all encrypted.
And this is actually what happens when you get stuff from your satellite and you're downloading movies.
That's exactly what happens.
It's symmetric key encryption of a lot of data.
So backwards would be dk c. And the only difference here is that everything else stays the same.
This is our decryption function.
All right.
So that's symmetric encryption is.
And there's a requirement of reversibility here.
So it's a lot different from one way hashes that we talked about last time.
Because here, while you want going from c to m to be hard, it's only hard if the adversary doesn't know k. If anybody knows k, it should be easy.
In fact, that e and d are going to be virtually identical in terms of complexity and sometimes implementation.
It's just you run it in reverse, and you get back what you encrypted.
That's the way you want to think about it.
So really what happens is that you need reversible operations in order to build e or d, which are the encryption and the decryption functions.
So permutation, for example, is reversible.
You can always take something and permute it, and you can go backwards.
So you could do something like that.
And the permutation, the reverse, looks like that.
What is this supposed to be?
It's the fact that I have 3 bits here.
And they turn into 3 bits over there.
And obviously, if I reverse the permutation and if I just add a simple thing where it was abc and I'm going to convert it to cba, then cba can go back to abc through the reverse permutation.
So clearly, this is a reversible operation.
But there are other operations that are reversible as well.
Plus can be reversed by a negation.
And exclusive OR is simply exclusive OR.
Because if you do A exclusive OR B, then you get C. And imagine if you did B again to that, you get A back.
That's what I mean by reversal.
Because you have, essentially, A exclusive OR A cancels out.
And so it's something like a exclusive OR B exclusive OR B again would give you A.
So if you go look at AES, for example, which is the Advanced Encryption Standard, is really only about four lines of code, maybe it's eight lines of code.
But this is a well used, it has been around for a while, symmetric key cipher that runs in 128-bit mode as well as 256-bit mode.
And it's, like I said, a few lines of code.
And you'll see operations like this, permutations.
And you'll see hat symbols if you see a C program, which is exclusive OR.
And you can see the encryption and the decryption are identical in implementation.
You're just going to run it one way, and you run it again.
And you get back the result.
Because these permutations are reversible.
And the hats are reversible.
And they're all signed integers, so you're just sort of adding them up.
And it's just complement arithmetic, so that looks the same as well.
So I encourage you to go take a look at AES.
I'm not going to spend more time on it.
The key idea here is of symmetry and reversibility.
We're going to move away from that.
Clearly, we didn't talk about that when we talked about one way hash functions, et cetera.
That was a different situation where we had no secrecy.
But here, we wanted symmetry.
The big question that you would ask and you should ask yourself when you see this, you say, OK great.
I can build symmetric key encryption algorithms.
They're actually, I would say, somewhat easier to do than to build hash functions, which have a whole lot of more interesting properties and hard to obtain properties like collusion resistance.
But the question really is how do Alice and Bob share the secret key, k?
So you need that.
You need this 128-bit number in order for there to be a channel, a secure channel, that Alice can communicate to Bob with and vice verse.
And so now you could say Alice sends Bob a letter.
But you know, mal could intercept that letter.
And even worse, what wants to do is look at the letter, actually deliver the letter, which is the best thing for him, and Alice and Bob think that they have a secure channel.
That's sort of best case scenario for mal, right?
So you could have mal in the middle here.
And you've got to worry about stuff like that.
So key exchange, let's move on to talk about key exchange.
And how does the secret key k get shared?
You can't put this out on a website, right?
So it has to be the case, sharing is something that has to be secure in the sense that there can't be any eavesdroppers.
So here's my favorite example of a puzzle that most of you have probably heard about.
But those of you who haven't, it's really pretty cool.
And those of you who have, it's still pretty cool and worth recalling.
And there's actually something that you probably haven't thought about very much even if you've heard about this puzzle.
That's the mathematical assumption made in the solution of this puzzle.
That will be interesting to you.
But the puzzle is a pirate puzzle.
So you've got Alice and Bob.
Let's call it the Caribbean, because that's my favorite ocean.
And Alice and Bob are in two different islands.
And we all know there are pirates in the Caribbean, right?
And so Alice and Bob want to communicate with each other.
And what Alice has are a bunch of boxes and locks and keys.
She's got the keys for her locks and nothing else and the same thing with Bob.
So Bob has boxes, locks, keys for his locks that he can put on his boxes.
And in this case, Alice wants to send a message to Bob.
And Alice wants to exchange a key with Bob, so they can eventually communicate in a secure way regardless of the pirates or whoever else is listening.
So the problem here, of course, is that if you just send a message on a boat-- so the pirates are kind of nice, in a way, that they will deliver messages.
But they are curious, right?
So they're very curious.
And they will open up boxes.
And so that's supposed to be a boat, by the way, not a box, a little mast there.
So I'm not a sailor.
But you have these boxes that are going to get delivered.
And the deal is this.
If there's an open box, the pirate's will open the box.
And if there's any message in it, they might read it, they might throw it away.
So clearly, a secret key can be exchanged by just putting a box with a message in it if you don't lock it.
They will not be able to open, they'll not touch, and they will deliver a locked box.
But if they ever see a key, they'll keep it.
If they ever see kind of a key on the boat, they'll just grab it and keep it.
And then the next time around, they see a locked box, they'll stick the key in and try and open it.
All right.
So how do Alice and Bob securely exchange a secret where security is based on this notion of piracy, I guess, with the pirates having a certain amount of capability in terms of storing keys, and opening up the locks, but they will not touch a locked box?
All right.
So that's the puzzle.
How many of you have heard of this puzzle before?
So all of you keep quiet.
Someone who hasn't heard of this puzzle before, think for a few seconds here and see if a solution occurs to you with respect to going back and forth-- so that's the hint, going back and forth-- and being able to securely exchange a message, which could be a 128-bit secret key written on a little note between Alice and Bob.
Yeah?
Back there
Are they allowed to pass locks such that--
So if you see it locked, they'll throw away the lock, too.
If the lock is on the box, then sure, they will deliver that box.
But if the key is outside of that box, then they'll keep the key.
No-- this is a difficult puzzle.
Yeah?
Alice could lock the box and send it to Bob
Yeah
And then Bob could also lock the box
Yeah.
And then?
At that point-- no, you're on the right track.
Keep going.
Just push it a little more.
Couple of more boat rides and we're done.
Yeah
And then if Bob sends--
So there's now two locks on the box
Yes
And Bob has two locks on the box.
And so what's Bob going to do next?
What's the logical thing for Bob to do next?
Send it back to Alice
Send it back Alice.
And now the key's inside.
Alice now sees two locks on the box.
And one of the locks is Bobs.
And other is?
Hers
Is hers
She can unlock it
She can unlock the box and send it back to Bob.
And all of this time, the only thing that's gone in transit is a locked box.
No keys have gone into in transit, whether they're inside the box or wherever.
But the only thing that's moved is a locked box.
So that's good.
You're exactly right.
So that gets you a Frisbee.
Whoops, sorry.
We need a secure exchange there.
Yeah, that's good.
So that is exactly right.
So just to recap, Alice locks box with KA, and that's the key for the lock, sends it to Bob.
Bob locks box with KB, sends it to Alice.
And Alice unlocks-- oh, I should've said that Alice puts message in box.
And that message has the secret key inside of it.
Alice unlocks KA and sends box to Bob.
And then Bob unlocks KB and reads message.
So that's good.
That's all good.
Let's look at it a little more deeply and think about it from a mathematical standpoint, not a physical standpoint.
You could think about it from a physical standpoint as well.
What is the relationship between this locking, this sequence, in this case a pair of locks, that we require here in order for this to physically make sense?
s I mean, there's different ways that you could add two locks to a box.
There's many different ways.
One way is to have a box that is locked if I look it over here.
And it doesn't open, because the lid doesn't open.
I've got suitcases like that.
And then there could be another spot here that looks it as well.
So that could be one way.
Well, what's another way of having two locks?
Yeah
Putting a box within a box
Yeah, putting a box within a box.
That's great.
So if you put a box within a box, then does this work?
It doesn't work, right?
So nested locks actually don't work.
Because what happened here is that KA was put first, then KB.
And if, in fact, you had KA, and then KB out there, then you can't remove KA without removing KB.
And this would be the case where you have nested locks.
So the mathematical operation that we require is commutativity between the locks.
And the locks need to commute.
I want to put KA in first.
Like I described, the physical realization could simply be a suitcase with two different positions for the two locks.
And any one of those positions locks the suitcase.
So you put KA here, KB here.
And then you can take KA out, right?
And it's still locked, because you have KB.
So this commutativity between the locks and essentially the keys is what's required.
And so now, let's move away from pirates and go into the cryptography domain, pure mathematical domain, and see how this turns into what's called the Diffie-Hellman key exchange, which is a key exchange algorithm or a protocol that under certain conditions give you exactly what you see here.
It gives you a secure key exchange.
And there's one issue associated with it, and that'll be kind of clear once we write it out here, that we'll get back to.
But Diffie-Hellman key exchange assumes that you have commutative locks.
And this is how it works.
You'll see what commutes when I give you the equations associated with Diffie-Hellman key exchange.
And this is also described in the '70s.
So what we're going to do is we're going to work in a finite field Fp*.
And the finite field means that we're going to be doing mod p, where p is prime.
And the star means we're going to be only looking at invertible elements only.
So we drop the nonvertible elements.
These things aren't particularly important.
And so we'll drop 0.
And we'll be looking at 1, 2, to p minus 1.
So all the numbers that you see are going to be 1 through p minus 1.
Now, what is the analog all this protocol that Alice and Bob, in our pirate puzzle, operated on or ran?
What is the analog in the mathematical domain or in the finite field domain?
Well, here's what happens.
Alice is going to select a random a.
And we're going to assume the g is public.
So she just shouts that out to-- Alice can see Bob from her house.
So she shouts out g and shouts out p. They're all public.
She doesn't care if the pirates can hear this.
And Alice is going to select a, which is random, and compute g of a.
And this is in the finite field, capital G. So you're going to do your mods.
And she's just going to send over g of a over to Bob.
Now, what Bob does is select b and computes g raise to b and sends that over.
So g raised to a is being sent over.
g raised to b is being sent over, sent over to Alice.
So Alice gets g raised to b.
And the key realization here is that Alice can compute g raised to b raised to a, because she knows a.
This is all going to be mod p. And we're going to call that K. And thanks to the fact that exponentiation commutes, Bob computes g raised to a raised to b, because Bob knows b, which is also exactly K-- I should say mod p over here.
Everything is mod p. So that's it.
That's Diffie-Hellman key exchange.
You have now created a shared secret based on the commutativity of exponentiation.
And the part that's still missing here with respect to the analogy is the fact that g of a is essentially the locked box.
So what g of a is hiding is a.
Because you want a to be hidden here and the same thing with g of b.
It needs to hide b.
So the problem that the pirates had was they couldn't open up the box.
The problem that the adversary, let's call them mal, has is that he has to invert g of a in order to discover a.
And in this particular finite field, and many such finite fields, you can think of this as being what's called a discrete logarithm problem.
So if you don't know how to computer logarithms in that continuous domain.
And there's tables.
And it's pretty easy to do.
But this is what's called a discrete logarithm problem, because we're in a finite field.
And we obviously want integers.
a is an integer.
So we need to discover what does that integer is.
Because we're doing the mod p, et cetera, and p is typically on a large number, it's actually a hard problem computationally to do a discrete log.
So when you see g of a, you know g, you know p, but trying to figure out what produced that g of a is a difficult problem.
And people have looked at this for 30, 40 years and there's not great algorithms to solve this problem, certainly not anything that's polynomial time solvable.
They're all kind of subexponential.
And you can make the numbers large enough such that g of a is secure in the sense that it doesn't give away what a is.
So that's the insight here that Diffie and Hellman had, which comes down to the discrete log problem is hard.
And what this simply means is given g of a, the discrete log problem is compute a.
And the same thing for b, of course.
There's one other thing that you want to say to be precise to sort of just cover the spectrum with respect to how this could break.
And that is what's called the Diffie-Hellman problem, for want of a better names and since these are the folks who first came up with it.
And the Diffie-Hellman problem is simply that given g of a and g of b, which is what the pirates see and what the adversary mal sees, we should not be able to compute g of a times b, which is exactly what we get here.
g of a raised to b is g of a times b.
So given those two things, if there's a way of computing g of a times b, even though you potentially haven't discovered a and b precisely, g of a times b is just the secret key that Alice and Bob exchanged.
So you're host.
Alice and Bob are host if mal can do this.
So there's two things going on here.
You want the Diffie-Hellman problem to be hard.
And you want the discrete log problem to be hard.
OK So generally, this is how cryptography works.
You set up protocols.
And there's some information that's bound to be exposed.
You want this information to be hard to reverse to get the crucial information.
That requires the computational hardness assumption like the two that we've made here.
And then you're off and running.
Your system will break if your computational hardness assumptions are incorrect.
And they may be correct for a particular time, for example, the 1970s for particular parameters.
But they may end up being incorrect assumptions, at least for those parameters, at a later point of time, simply because computers got faster.
It's like 2 raised to 40 was this huge number when I was your age.
Now, it's like nothing.
So that's basically part of the game.
But the good systems are those that scale where you increase the parameter size and the system and the protocol stays the same.
And so you just increase p, for example, in this case.
And the discrete long problem is still hard for modern computers.
So those are the good protocols and the good cryptosystems that stand the test of time, not necessarily the ones that have exactly particular parameters.
That's hard to do.
Because as I said, Moore's law and computers have been getting really exponentially faster.
All right.
So there's one main problem with the Diffie-Hellman protocol and the solution to our pirate puzzle.
And so can someone tell me-- and it could be just for the sake of the Diffie-Hellman problem or just in the context of the Diffie-Hellman problem, but also in the context of the pirate puzzle-- what assumption are we making here that's as yet unstated with respect to this secure key exchange being actually secure?
Someone.
Yeah
If I were to intercept a message from [INAUDIBLE] something of their own back?
What does that mean?
The pirates see a locked box.
So the first step of the protocol, Alice is sending a message inside a locked box with a single lock in it.
You're kind of on the right track.
And what could the pirates do in order to break this protocol?
We've kind of made an assumption here.
And I might have said it explicitly.
Yeah, go ahead
[INAUDIBLE] throw the box away
They could just throw the box away.
But that doesn't break the security of the protocol.
That breaks the functionality of the protocol, right?
Yeah, go ahead
Put their old lock on [INAUDIBLE]
Ah, that's exactly right, put their own lock on it.
You know, if they had locks-- so these pirates don't have locks, right?
We're making that assumption.
If they had their own lock with a key, if they just had a lock that locks and they didn't have the key for it, they will wouldn't be able to break the security of the protocol.
But if they had a lock and a key for that lock, then they can pretend to have delivered this to Bob.
And there's no authenticity here with respect to Alice doesn't quite know whether she's communicating with Bob or not.
She's at the mercy of the pirates to deliver these boxes.
So if the pirates had a lock and the key for the lock, then we're in a situation where Alice may have exchanged the key with the pirates.
And she thinks she's exchanged it with Bob.
And she's, in fact, communicating with the pirates.
So there's a man in the middle attack, which corresponds to the pirates having their own locks and keys.
And it's even more trivial in the case of our picture here.
Because assuming the pirates know mathematics, they can generate a random number-- and the number could be c, for example-- and just send back g raised to c. And for all you know, they could actually get Bob to send g raised to b back, but they would intercept it and replace it with g raised to c. And they know what c is.
So what's happening now is so you can [INAUDIBLE].
I won't to go through all of the math here.
But you can kind of see it, I hope.
You end up, if you're Alice, exchanging a secret key with the pirates as opposed to Bob.
And the way you can set this up is the pirates actually will get into a situation where Alice and Bob think that they're communicating with each other in a secure fashion, but the pirates can listen to all of the messages.
They can decrypt all of the messages, because they know what they secret key, k, is.
OK. And remember that the secret key, k, is something that is probably going to be used in a symmetric key encryption scheme eventually to send real messages.
So you're going to have ciphered text with that secret key, capital K, over there.
And if the pirates or mal get to know what the secret key, k, is through a man in the middle attack, you've got problems.
All right?
So the man in the middle attack is something that we have to worry about.
What we're going to talk about next is something that addresses this problem.
And it may not seem like it's directly addressing the problem.
But fundamentally, what's going on here is you need to have authenticity in who you're communicating with.
Alice has to somehow authenticate Bob.
And Alice has to know somehow that g raised to b is something that came from Bob.
It's not g raised to c that came from somebody else is in the middle.
And that's where asymmetric key cryptography and public keys come in, where you have a certified public key associated with yourself.
And maybe you need VeriSign or you need the DMV, or the Registry of Motor Vehicles, RMV, to do this for you.
But you create a certified public key, which is associated with your identity.
And it's public.
You can put it on a website.
And everyone can access it using HTTPS, so they know they're going to the exact website that you've put up.
And that gives you a way of identifying yourself.
And if you can do that, you can protect against the man in the middle attack using asymmetric key cryptography.
So that's kind of the final part of this puzzle that's associated with authentication and secret key exchange and all of that.
Once we do that, you'll know what the functionality is that we require.
And then we'll have to talk about how we can build a subsystems.
Cool.
Any questions so far?
How we doing?
OK.
So public key encryption, let me just do a little set up.
I said some of this last time.
But to make sure we're on the same page, what we have here is we really want a message plus a public key.
And you want to obtain ciphered text using this operation.
And this plus is not arithmetic addition.
It's just that we're putting these two things together into an algorithm, a public key encryption algorithm, that produces ciphered text.
All right.
And this public key, just to reiterate what I just said, is going to be if Alice is producing the message and Bob is getting the ciphered text, this is going to be Bob's public key.
And the fact that it's Bob's public key is something that Alice should be able to authenticate using VeriSign, using the Register of Motor Vehicles, what have you.
That's what's going to protect against the man in the middle attack.
OK. We're not going to talk a lot about how you can get a certificate.
Your MIT certificate is something which corresponds to your MIT ID.
It's got information, what year you are, what your name is.
And when you generate that certificate, you are getting a certificate of authenticity that you are you.
And of course, you give that away and you hand it to someone else, someone can pretend to be you as well.
But that's what's happening when we talk about public keys and you owning public keys.
We're not going to, as I said, get into that very much more.
I'm more interested in describing this algorithm for public key encryption.
We'll look at a couple that produces a ciphered text given a message and a public key.
Now, of course, what Bob needs to do is to take the ciphered text.
And this is what Bob's doing.
And Bob has a private key that is distinct from the public key and needs to get back exactly the message using a decryption algorithm that corresponds to the message that Alice sent.
OK. And this whole thing is going to work provided knowing the public key, let's call it PK, and the private key.
We can't call it PK as well, obviously.
So we call it SK.
Knowing the PK does not reveal anything in a mathematical sense about SK.
But obviously, in order for this whole thing to work, PK and SK have to have some mathematical relationship.
And the different cryptosystems including RSA, and we look at a knapsack cryptosystem, all have different algorithms for encryption and decryption.
And they have different mathematical relationships between PK and SK.
And for each of these relationships, you have to show that the adversary has to solve a computationally hard problem in order to discover SK given PK.
And it turns out that for most of these systems, it's symmetric in the sense that these algorithms, at least for RSA, you could use either one of these interchangeably.
And there's issues associated with that.
So we really won't go too deep into that.
But what I said you should hold, which is you have one, it shouldn't tell you anything about the other.
There has to be a computationally hard problem associated with discovering one of these only given the one other.
And we'll talk about what those hardness assumptions are certainly for RSA and also for another cryptosystem, a knapsack.
So we're going to present RSA, which is this real magical algorithm.
It's amazing it works.
Every time I prepare for this lecture, I got to relearn somethings.
And that's because there's one subtle aspect of this.
It's all about number theory.
Number theory can get pretty subtle.
But it's also intricate enough that I forget the details.
So let's get started on that.
Basically, RSA is based on primes and factoring numbers into primes and using number theory to make sure that you can actually accomplish what this is trying to do.
The functionality of RSA should be distinct from the security of RSA.
When we talk about the functionality of RSA, we are saying for any message, if Alice uses Bob's public key to encrypt it, the ciphered text resulting from that should be decryptable into exactly the message that Alice sent given Bob's private key.
That's the functional requirement of a public key encryption algorithm or a public key cryptosystem.
The security requirement of a public key cryptosystem is what I wrote up there.
It's the knowledge of SK should be hidden even given the knowledge of PK.
And there's precise computational hardness assumptions that are associated with each cryptosystem.
So let's separate out functionality from security.
We're going to talk about functionality for the next few minutes.
Alice is going to pick two large secret primes, p and q.
So what I'm going to describe here are Alice generating her public key and her private key.
She's going to then publish her public key and keep her private key secret.
Bob does the same thing.
And then after that, they have to register.
And this is not something we'll spend time on beyond me saying it one more time.
They have to register their public keys with the VeriSign or the RMVs like I talked about.
So everyone knows that Alice's public key is this long number.
But no one knows Alice's private key.
So Alice picks two large secret primes.
So these are actually going to result in the creation of our private key.
And then Alice computes N equals pq.
So she just multiplies those out.
She chooses an encryption exponent, e, which satisfies this little equation, which says that it's relatively prime in relation to p minus 1 times q minus 1.
And she knows p and q.
So she can compute p minus 1 times q minus 1.
So the gcd of e and p minus 1, q minus 1 is 1.
And you can certainly accomplish this simply by choosing e to be a prime.
Because then a gcd of a prime with anything else is 1.
And it turns out that RSA uses-- this is all going to be public, by the way.
e is going to be public.
So you can just fix that.
And most RSA algorithms just fix that to be a small number.
The encryption exponent is a small number.
And the reason it's a small number is because you're worried about performance.
And we're going to exponentiate using e. And the smaller it is, the faster the encryption is going to go.
So if you want to encrypt fast and decrypt more slowly, unfortunately, that's the trade off here.
You would pick a small e. And then we're going to compute our decryption exponent, which obviously is going to have to be private.
Because that's part of our private key.
But that's going to be bigger if e is small.
And that's just a trade off.
It's symmetric in the sense that while it's an asymmetric algorithm, it's kind of symmetric in the mathematical sense that the private keys and the public key operations are symmetric.
So what is Alice's public key?
Well, Alice's public key, which she can then publish, is simply m, e. OK. Now, the fun starts.
We have to figure out what the private key is going to correspond to.
And it turns out-- and this is one of those things where how did they ever think of this?
And that's still true 40 years later.
You get the decryption exponent using the extended Euclidean algorithm.
And this is done by Alice secretly, where what you want is to have the relationship that e times d is 1.
And this is mod p minus 1, q minus 1.
And there's algorithms out there that would find the inverse that corresponds to d for e or vice versa.
And they're polytime algorithms.
As long as you know this number here, mod p minus 1, q minus 1, and you know that Alice knows that, you can get your decryption exponent.
And typically, if a is small, as I said, d is going to be large.
By the way, the numbers here, p and q, are going to be the roughly 1,000 bits long.
So that's essentially-- we're talking about huge primes here.
And so n would be 2048 bits in that case.
So the private key, Alice's private key, you can think of as d, p, q.
So now, it's clear as to what's public and what's private.
n and e are public.
d, p, and q are private.
So that's the set up for RSA.
And it's not at all clear that RSA accomplishes either of the two things that we need, the first of which is functionality, the fact that encrypting, and then decrypting a message should get you back that message.
So that's the first thing we need to look at.
And the security part is actually a little bit easier.
Because you can see we're going to have to make assumptions about factoring primes and so on and so forth.
Right here, you can just see that immediately.
The biggest assumption made by RSA from a computational hardness standpoint is simply that if the adversary sees n, that they should not be able to factor it into p and q.
Because if they can do that, it's over.
So that's actually easier than the functionality argument.
So why does this work?
And amazingly, we can actually do this in about 10 minutes.
I'm going to explain to you why this works in 10 minutes.
And the only theorem that we'll require on top of this, which I will not approve, because Fermat proved it centuries ago, is Fermat's Little Theorem that says that when you have p being a prime-- you can think of this as a special case.
You take m, and m is an arbitrary number.
And if p is a prime, then this relationship holds.
So you raise it to the p minus 1 power, and you get 1, mod p. So that's Fermat's Little Theorem that's going to be required.
And that's pretty much the only thing that you have to invoke beyond sort of standard mod arithmetic to show that RSA works.
So what's going on here?
Let's call phi p minus 1 times q minus 1.
Obviously, that's showed up a couple of times.
And you may as well represent it by using a smaller, simpler symbol.
So we'll call that phi.
And we are going to say that d equals 1.
Mod phi is given to us.
And therefore, we can say that ed equals 1 plus k phi.
So that's it.
All I'm saying is the remainder.
Then you took to the mod with respect to phi was 1.
So the actual number was 1 plus k times phi.
k is some integer.
Think of it as a positive integer.
Remember that we now have the p and the q are over there.
And p and q are primes.
So p and q are primes.
And given that that's the case, we really have two cases to analyze.
Oh, I'm sorry.
I missed one crucial point, which I should have told you, which is I gave you this.
But I didn't actually tell you what is going on.
I mentioned it in passing, exponentiation.
But I didn't tell you exactly what the encryption algorithm was and the decryption algorithm was.
And obviously, you need that in order to prove their correctness.
I mean, it'd be wonderful if you could prove correctness of this.
There exists an algorithm that is such that RSA works or public key encryption works.
So it turns out it's extremely straightforward.
c equals m raised to e. And that's part of the usefulness and the power of RSA, which is you take m and you exponentiate it.
And you take c and you exponentiate it.
And the first is the encryption.
You get the c through encryption, as you can see over there, the ciphered text.
And m is the plain text.
And you get that back.
So our goal here is to show that you have essentially something where when you exponentiate m raised to ed, it should give you m. For these choices that we have, of e and d, we've set up the d in such a way that m raised to ed-- because if you just go do encryption followed by decryption, you are doubly exponentiating.
That make sense?
Ask me questions if this doesn't make sense.
This is important.
m raised to ed should give you back m. And if you can show that for any m, you're done.
That's the functionality of RSA.
All right.
So that's encryption and decryption.
And so now let's go back to here.
Now, ed equals 1 mod phi.
Because I've set that up.
This is how I discovered d given e. So that's a given to me.
That's part of what's called the key generation phase of RSA.
And that's the mathematical relationship that I keep harping on in terms of the relationship between the public and the private key.
So given that p and q are primes, I have two cases.
The first case is that gcd of m, p is exactly 1, which means that the message m. So what I'm comparing here, the two cases, is I have the message.
And I'm going to break up the messages into two different categories.
That's it.
There are all kinds of messages that are possible.
These are arbitrary numbers.
I'm going to break them up into two categories, one of which where the message is relatively prime in relation to the prime, p. And it's not a multiple of p. That's the way you want to think about it.
Obviously, gcdmp would be 2 if M were 2p.
So the case I'm looking at is gcd of mp equals 1.
And then the another case is going to be trivial, actually, which is gcd of mp equals p. Did I say 2 when I said gcd of 2p, p equals 2?
Come on.
Wake up.
Wow, that's it.
Perfect.
Wake up.
OK.
So gcd of 2p and p is p. So those are the two cases.
All right.
So by Fermat's Little Theorem, this is really Fermat's theorem.
And because he had a last theorem, for some reason, some people call this the Little Theorem.
But it's Fermat's theorem.
And we know what that is.
I just wrote that out there.
You can now write something that says what I'm going to do is I'm going to just take m raised to p minus 1, which is 1.
So this thing is 1.
And then I'm going to raise it to k times q minus 1.
And you'll see why I'm doing this in a second.
Because I want to get the 1 plus k phi factor here.
So I'm taking 1 and I'm raising it to this power, which obviously is going to give me 1 back.
So all of that is straightforward.
And then I'm going to multiply this by m. OK. And this is clearly the same as m mod p. Because all I've done is multiply it by 1.
All right.
So why did I do this?
Well, I did this, because I want to group together these two exponents.
And since I've run out of room here, let me just erase and finish this properly.
The other case is easy anyway.
And so I can write this as 1 plus k p minus 1 q minus 1.
And that, of course, is exactly m raised to ed, right?
And so what I've done here is, because 1 times m is clearly m, but if I look at this, this is m raised to ed.
So that's clearly m. That's it.
Just figured out that when I have k phi here, that's going to turn into 1, basically, when you exponentiate it.
So that's the hard part, actually, as it turns out, of proving RSA's correctness, just introducing this 1 raised to something.
And then the easier part is simply the case where the m is actually a multiple of p. So you have a gcd m, comma p equals p. And in this case, you know that m mod p is actually 0.
Because m is a multiple of p. So you're basically exponentiating 0.
What do you do with 0?
You're going to get 0 on both sides.
So you're sending a message that's essentially 0 mod p And when you decrypt it on that side, you get 0.
But one last line here is simply that m raised to ed is m trivially all of this mod p if m is 0, right?
So that was the easy case.
So it was really pretty straightforward.
In one case, we took 1 and we exponentiated it and showed the result in a couple steps.
In the other case, you had messages that were 0 mod p. So it's very pretty.
It works.
And so what happened here?
I said that m raised to ed equals m. And what I did here, of course, was I did everything.
So it's not quite done, a little slight of hand here as I switched over and talked about mod p. So I had mod p over here.
And I said p and q were primes.
And I looked at p first.
But what I need to do, just to finish this off, let me do this over here, is I have to do the same argument for q.
And I'm going to put it together for n. And the reason for that is simply that I have n over here.
So remember n equals p times q.
The encryption and that decryption are going to be done mod o.
Certainly, the encryption has to be done mod n, because n is only public number that you have that you can mod with, correct?
So what I've done here, this analysis is for p, can do the same for q.
It's exactly the same, because p is a prime and q is also a prime.
But I have to just do one last thing, which is put these two things together and say that n equals p times q, so the math is all going to work out.
Let me just write that out.
It's not too difficult to explain, once I have this up here.
So in both cases of p and q, so when I say both cases, I mean the pk's and qk's.
I have m raised to ed equals m mod p. And m raised to ed is the same as m mod q.
And since p and q are distinct primes, we can say that m raised to ed equals m mod N. And that essentially says c raised to d, if you really want to put it all together, which is m raised to e raised to d, equals m mod N, which of course is what we want here.
And this thing was also mod N. This is mod N, mod N, mod N. All right.
So that's RSA.
That's your first public key algorithm, the first public key algorithm, at least that stood the test of time, still in use today.
From a standpoint of computation, this is the hardest part.
You have to exponentiate, and you have these large numbers.
And as the years have rolled by, RSA, as I've said, withstood the test of time.
But the parameters have increased.
Way back then in the '70s, they were thinking about 512-bit primes.
In fact, I can't recall whether n was 512-bits or p and q were 512-bits.
But if p and q were 512-bits, then n would be 1024.
And now, NSA recommends 8192-bits for n. So there's been an increase.
But the nice thing is that it's not like there's an exponential increase in the computation.
Because the computation is polynomially related to the number of bits.
So if you double it, I think, if I recall correctly, decryption is going to be the cube of that.
Or actually, verifying signatures is probably the cube of that.
But don't worry too much about that.
The bottom line is that as you double the size of the exponent, you're going to have a fairly small increase in the time required to decrypt or to verify a signature, et cetera.
But it has grown from 512 or 1024 to 8192.
And so hopefully, you all understand how RSA works to some extent.
I just will leave it at the hardness assumptions here are, like in the case of Diffie-Hellman, two-fold.
And the first one is kind of immediately obvious.
Just like it was the case with Diffie-Hellman, where you had g raised to a flying about and obviously that has to hide a, here you got N, capital N, being published.
And if anybody could take N and factor it, there may be multiple factorizations.
But you're going to get a unique prime factorization.
So that's what you want, that unique prime factorization of N. And if you get that, then you've broken the System because you know what p and q are.
And so this is all public in the sense that this algorithm is public.
If you're using RSA, this is what you're following.
So the person is trying to figure out what the two primes are that together get multiplied to form capital N. And so that's a factorization problem.
And so RSA hardness assumptions are given N, hard to factor into p, comma q.
And this is factoring.
And then one other thing, which is given e-- so you're not actually breaking the entire cryptosystem.
But you're breaking the privacy associated with a particular message.
And so you could break the privacy associated with a particular message.
You're given e, because that's public.
And you don't know what p and q are.
But you know that e is relatively prime with respect to p minus 1, q minus 1, because that's RSA algorithm.
And that's a publicly known.
And you also know c, which is the ciphered text.
And so what you're doing is you're trying to discover m. So you're trying to break a particular encryption that was created by the RSA algorithm.
And you haven't discovered the private key here.
That's only discoverable through the factoring problem.
But you could break security if you can find m such that m raised to e is c mod N. So you're doing the searching for an m. So you're trying to discover an m that you can exponentiate to get what you have on the right-hand side.
Because clearly, you can compute what's on the right-hand side.
So this is simply called RSA problem.
So those are the two computational assumptions that you need to make in order for RSA to be secure.
Cool.
Any questions?
[INAUDIBLE] center?
So that's anonymity.
Yes.
There's ring cryptography.
And there's a whole host of protocols.
I actually did some of them based on RSA, where you can, by collecting a bunch of private keys together, essentially set it up so it can be verified that the message came from a collection of people, but you can't tell which person the message came from.
So there's just a whole host of things.
There's thousands of papers written.
There's a wonderful field.
I encourage you to look into it if your interests are inclined this way.
And it's just gone on and on.
It's become more important with the internet.
RSA, the company, was probably founded in the late '70s.
And they struggled for a while.
And then eventually, they were used for Secure Sockets Layer in Netscape, which was their big break.
And then, of course, Netscape meant the internet was around.
And so really, the internet made RSA what it is today.
And so just a whole host of wonderful algorithms out there, some of which are based in RSA and some that are broken.
And so let's talk for the last few minutes about all of the fits and starts that occurred in cryptography.
And precisely what I'd like to focus on for the time we have left is hardness.
So we spent a lot of time talking about hard problems.
And we talked about NP-complete problems that are hard.
But they're hard in the worst case.
So you have a situation where you have NP-complete problems.
And I'd like to talk a little bit about the relationship between NP-completeness and crypto.
Because we've made these assumptions about hardness.
Now, what's interesting here is that N composite is clearly in NP, but unknown if NP-complete.
So this is very interesting.
The tried and trusted algorithm for public key encryption relies on a computational assumption where the problem associated with that assumption is not even known to be NPC.
All right.
So that's kind of wild.
So how does this work?
Or why does this work?
Now, if you take other problems, like, is a graph 3-colorable?
And so what does that mean?
Well, you have three colors.
And you're not allowed to reuse the same color on two ends of an edge.
So if you put red over here, you can put red here, but you can't put red here and there.
And so that graph is 3-colorable.
But if you had a click, then this would not be 3-colorable.
Because you have all these edges.
You have three edges coming out.
And so clearly, the degree from a vertex is going to tell you what you have.
So if you have a 4-click over there, immediately it's not 3-colorable.
But checking whether a graphic is 3-colorable is NPC.
You can use a three set as a way of showing that.
So you can say, oh, wow, maybe I shouldn't be worried about RSA.
I should just be building cryptosystems based on 3-colorability.
Because it seems like a much simpler problem than all these grungy map that you have out there-- actually, beautiful map that you have out there.
OK.
So that's something that's a perfectly reasonable question to ask.
And then we have Knapsack.
Knapsack is simply you've got a bunch of items and you just want to figure out whether you can get this particular sum S. Knapsack is NPC as well.
And you got a bunch of weights given to you.
And the BI are going to have to be 0, 1.
So you just want to discover a particular assignment of the BIs, such that you pick the appropriate items to put into the knapsack, right?
That's it.
That's a perfectly reasonable problem to potentially use as a basis for computational hardness to go build cryptosystems.
And people did that.
People did that for years.
They tried and they tried.
And they produced cryptosystems, public key cryptosystems, based on Knapsack that look fantastic.
And they work from a functionality standpoint in the sense that you would use this Knapsack-- and I'll give you a sense of how this is done in a minute-- problem to encrypt.
And then you'd use a different kind of Knapsack problem to decrypt.
And when you encrypt it and decrypt it, you did get that same message back, except the whole world knew what the message was.
Because the problem associated with the Knapsack wasn't hard enough.
So the computational hardness was what broke the Knapsack schemes.
And then you come down to asking why is it that this problem that's not an NPC seems to have stood the test of time, but all these other problems, like Knapsack and 3-colorability, which is even worse than Knapsack, when people have built cryptosystems based on this, they've all been broken very quickly?
And so why is that?
What do you think the reason is, sort of at a high level?
What does NP-completenss say?
When we talk about complexity, what are we worried about?
Most of the time, what are we talking about when we talk about complexity of an algorithm?
Or in this case, in the case of a problem, what adjective do we put in front of runtime, for example, then we compute complexity?
Worst [INAUDIBLE]
Worst case, right?
Worst case.
In the worst case, you're going to be able to create random graphs where it takes exponential time to discover whether they're 3-colorable or not.
But in the average case, all you do is if you have a large graph, if there's one little 4-click in the graph and you can find it, instantly you know that it's not 3-colorable, right?
So it turns out that's 3-colorability is just the worst thing ever when it comes to cryptography.
Because the larger the graph-- and you need this graph to be large.
Because anything that's small, is constant time, right?
Because so what if it's exponential?
It's constant time.
So you need the graph to be large.
When you have a random graph that's large, the chances you're going to find a 4-click in a 2,000 vertex graph is pretty high.
And so if you just go scan and look for a 4-click, instantly you know that this graph is not 3-colorable, right?
So in the average case, 3-colorability is easy.
It's easy to solve in the average case.
And the wonderful thing about factoring is as long as the numbers are large, doesn't matter what the numbers are, it's hard to factor in the average case.
So that's the big difference.
If you're going to take anything away from the rest of this, it's the difference between problems that cryptographics systems are based on.
The systems that have stood the test of time, they're based on problems that are hard on the average.
And the NP-complete problems, like the simple ones here, are hard in the worst case.
And this is also true for Knapsack.
So that's the essence of it.
I'll just give you a sense.
You can read the notes.
There's a way of generating secret keys and public keys using Knapsack that I think is kind of interesting that is worth looking at, even though all of these systems are broken.
It's just kind of cool.
You know, how would you get encryption out of a knapsack?
I mean, you're putting things in a knapsack and taking things out.
How can you set it up so you get an asymmetric key system, a public key system, through a Knapsack problem?
So I'll just give you a sense of that.
And you can read.
I won't finish.
But I'd like to do what we did here for RSA in the time that I have left for Knapsack.
That is kind of cool.
You get a sense of the variety of different public key cryptosystems that are out there by looking at something that is very different from RSA.
So in the Knapsack problem, the general Knapsack problem, what's hard is NPC.
There's a super increasing Knapsack problem that's easy, that can be solved in linear time.
What is a super increasing knapsack?
Well, a super increasing knapsack is something where I Wj has this property.
So an example of that is 2, 3, 6, 13, 27, 52.
So the rates are super increasing.
2 plus 3 is less than 6.
2 plus 3 plus 6 is less than 13 and so on and so forth.
Do you see why super increasing knapsack is easily solvable?
I mean, what is a knapsack?
I got a limit on the amount of stuff I can put into the knapsack.
And I want to make to be able to say yes or no in terms of whether it fits exactly or not, just in terms of our definition.
So what I do here in super increasing knapsack?
Yep
[INAUDIBLE] from the biggest to the smallest [INAUDIBLE]
Exactly.
That means that there's a linear time algorithm that basically solves the problem.
And you know that you can do that, and you would get the correct answer.
So that's pretty much what you've got.
That'll give you the highest weight.
If you have 13 exactly, you know you can't put 52 and 27.
You get 13.
There's no point in putting 2 and 3 and 6, because that's not going to give you 13.
So that's clearly easy.
So we've got an interesting case here, assuming this is all going to work out from an adversarial standpoint, which unfortunately it doesn't, you can look and say, ah, I want encryption to be the super increasing knapsack.
Because that should be easy to do.
And I want the decryption, not knowing the private key, to be as hard as knapsack.
OK.
So that's the kind of thing that you could do if you built a cryptosystem.
And people did, Merkle and Hellman.
Hellman is the same guy, the second name, in Diffie-Hellman.
They proposed this particular system that ended up being broken soon after.
But the idea is that you create a private key.
And the private key is a super increasing knapsack.
And then you use a private transform in order to get a-- and this is really I put it in quotes, because this was the bug-- "hard" Knapsack problem.
And this corresponded to the public key.
And so what you do is you won't actually have to solve Knapsack for encryption, the hard problem.
The encryption would just simply take the public key, which is completely public, and you would create the encryption of a message using this particular public key in a polynomial time.
But the inversion, not knowing the private key, you would force the adversary to solve what you think was hard general Knapsack problem to actually break the scheme or to get the decryption.
And so let me show you really quickly how this works with numbers.
And we won't have you worry about symbols and things like that.
So just give me a couple of extra minutes here.
So let's say that I had a message.
Oh, before I do that, let me look at-- let's say, that we chose N equals 31 and M equals 105.
This is actually the message.
No, I'm sorry.
Capital M is not the message.
These are public parameters.
And we're going to take-- this is a transform.
Oh, I'm sorry.
These are not public.
These are private.
My bad.
So what I'm going to show you is I'm going to take a super increasing Knapsack.
And that's exactly what I have up there.
So that corresponds to an easy Knapsack problem.
I'm going to convert it using these private parameters, N equals 31 and M equals 105.
And so our private key is our super increasing knapsack, which is 2, 3, 6, 13, 27, and 52.
And the public key, what I'm going to do is simply multiply each of these, 2 times N. And I'm going to take mod M. So for each of those values, I multiplied by N, which is 31, and take the mod of 105, and I end up getting 62, 93, 81, 88, 102, and 37.
So you can get a private key and a public key using this private transform.
I'll let you look at the notes.
But basically what happens is when you take a particular message M, what you end up doing is you want to encrypt it using the public key, which is this quantity over here.
And the way you encrypt that is simply by taking a particular message.
And let's say that the message is written as 011000.
Then all you do is add up 93 and 81, because those are those two.
And you say this is going to get encrypted by 174.
So the message encryption is simply a simple operation where you add up weights of the knapsack.
So you end up getting 174 out here.
And the hope is, of course, that when adversary sees 174-- and this is the part where things get a little iffy-- that it's hard-- you have to think about lots of numbers here, of course-- for the adversary to figure out that that 174 is actually 93 plus 81.
OK.
So the diverse is not necessarily an easy problem.
And that's exactly what Knapsack is, right?
I tell you what the sum is over there, which is S. And I tell you what the weights are.
And it's hard for you to figure out what the BIs are.
So now you see why this didn't work.
So you have to have a situation where in the average case whatever you produce for the ciphered text here, you're sending out the ciphered text according to this cryptosystem, which is 174, and you want to make sure that the adversary can't figure out that this is actually 93 plus 81.
Amazingly, people thought they could build systems using this assuming these numbers were much larger than they are here.
But that certainly wasn't the case.
Because in the average case, you end up being able to break these systems.
The last thing is, of course, you don't want to necessarily solve the hard Knapsack problem associated with this.
So what ends up happening is you end up using N equals 31.
So if you want to decrypt, you have N equals 31 and M equals 105.
And what you're going to do is take this and multiply it by N inverse mod M. So rather than doing times N mod M, you divide by N mod M. And you can do this operation relatively simply.
And you can go back from 174 to figuring out what the actual message was by computing this quantity.
So I'll stop there.
I didn't quite get to everything that I wanted to cover.
But take a look at the notes.
Get a sense for why the difference exists between NP-complete problems and problems that were used in cryptosystems.
And happy to stick around and answer questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to the final week of 6.046.
Are you excited?
[CHEERING] Yeah, today--
Oh
Well, and sad, I know.
It's tough.
But we've got two more lectures.
They're on that one topic, which is cache oblivious algorithms.
And this is a really cool concept.
It was actually originally developed in the context of 6.046, as sort of an interesting way to teach cache-efficient algorithms.
But it turned into a whole research program in the late '90s, and now it's its own thing.
It's kind of funny to bring it back to 6.046.
The whole idea is in all of the algorithms we have seen, except maybe distributed algorithms, we've had this view that all of the data that we can access is the same cost.
If we have an array, like a hash table, accessing anything in a hash table is equally costly.
If we have a binary search tree, every node costs the same to access.
But this is not real.
Let me give you some idea of what a real computer looks like.
You probably know this, but we've not yet thought about it in an algorithmic context.
These are caches, what are typically called caches, in your computer.
Then you have what we've mostly been thinking about, which is main memory, your RAM.
And then there's probably more stuff.
These days you probably have some big flash.
If you have a fancier computer, you have flash, which is maybe caching your disk, which is huge.
And then maybe there's the internet at the end, if you like.
So the point is all the data in the world is not on your CPU.
And there's this big thing which is called the memory hierarchy, which dictates which things are fast and which things are slow, not exactly which data items; that's up to you.
But the idea is that on board your CPU you have probably, these days, up to four levels of cache.
As I've tried to draw them, they get increasingly big.
Typical values-- a level one cache is something on the order of 10, 32 K, whatever.
Level four cache these days, as introduced by like Haswell Architectures, has about 100 megabytes.
Main memory you know; this is the thing you usually think.
About.
It's in the gigabytes.
These days you can buy computers with a terabyte of RAM.
It's not crazy.
Flash gets bigger.
Disk-- these days you can buy 4-terabyte single disk, but if you have a whole RAID of disks, you can have petabytes of data on one computer.
So things are getting bigger as we go farther to the right.
But they're also getting slower.
.
And the point of cache efficient algorithms is to deal with the fact that things get slow when they get far away.
And this makes sense from a physics standpoint.
If you think about how much data can you store in a cubic inch or something and how much could possibly be near your CPU, at some point, you're just going to run out of space, and you've got to go farther away.
And to go farther away is going to take more time.
So you can think of it-- I mean, there's the speed of light argument, that things that are farther away in your computer are going to take longer.
Typical computers are not anywhere near the speed of light, so there's a more real issue, which is how long are your traces.
And then when you have physical moving parts, like a disk, I don't know if you know, but disks actually spin, and there's a head, and it has to move around.
And that's called seek time.
Moving a head around on the disk is really slow, on the order of milliseconds.
Whereas reading from on-chip cache, that's on the order of nanoseconds, whatever your clock rate is, so a few billion times a second.
So there's a big spread of like a factor of a million or 10 million from level one cache to disk speed.
That sucks.
And so you might think, well, if your data's big, you're just screwed.
You've got to deal with disk, and disk is slow.
But that's not true.
Life is not so bad.
So, in general, there's two notions of speed, and I've been kind of vague on them.
One notion is latency, which is if right now I have the idea that I really need to fetch memory location 2 billion and 73, how long does it take for that data-- say, one word of data-- to come back?
That's latency.
But there's another issue, which is bandwidth; how fat are these pipes?
What's my rate of information that I could pump?
If I said, please give me all of main memory in order, how fast could it pump it back?
And that's actually really good.
So latency is like your start up cost.
When I ask for something, how long does it take for that one thing to come?
But then there's a data rate.
And bandwidth you can generally make really large.
For example, in disk, bandwidth of a disk is pretty big.
But even if it weren't big, you could just add 100 more disks.
And then when you ask for some data, all 100 disks could give you data at the same speed, and provided you don't overload your bus, so you've got to also make more buses and so on.
You can actually really huge amount of data per second, but still the time to get there and the time for all the disks to seek their heads, that's slow.
It doesn't add up, actually, because they're all doing it in parallel.
So you can't reduced latency, but you can increase bandwidth.
And let's say-- it doesn't match physics, but we can get pretty close to arbitrarily high bandwidth.
And so in a well-designed computer, the fatnesses of these pipes are going to increase, or could increase, if you want.
So you can move lots of data around.
But latency we can't get rid of, and this is annoying because from an algorithmic standpoint, when we ask for something, we'd like it immediately.
In a sequential logarithm, we can't do anything until that date arrives.
So cache efficiency is going to fix this by blocking.
This is an old idea, since caches were introduced.
There's the idea of blocking.
So when you ask for a single word in main memory, you don't get one word.
You get maybe 32 kilobytes of information, not just 4 bytes or 8 bytes.
And we're kind of free to choose these block sizes however we want, when we designed the system.
So we can set them, in a certain sense, to hide latency.
So if you think of amortizing the cost over the block, then you have something like amortized cost over block.
This is per word.
Essentially, we divide the latency by the block size.
And we have to pay one over bandwidth.
Bandwidth is how many words a second you can read, say, from your memory.
So one over bandwidth is going to be your cost.
So this we can't change but.
By adding enough disks or adding enough things at making these pipes fat enough, you can basically make this big.
Latency is the thing we can't control.
But if this block is sort of useful, then we're paying the initial start up time, say, hey, give me this block, and then waiting for the response.
That latency we only pay once for the entire block.
So if there are block-size words in that block, per item, we're effectively dividing latency by block size.
This is kind of rough, but this is the idea of how to reduce latency.
Now, for this actually work, we need better algorithms.
Pretty much every algorithm you see in the class so far works horribly in this model.
So that's the point of today and next class is to fix that.
For this kind of amortization to work, I'm using "use" in a vague sense so far.
We'll make it formal in a moment.
When I fetch an entire block, all of the elements in that block should be useful.
We should be able to compute something on them that we needed to compute.
Otherwise, if I if I only needed the one item that I read out of the block, that's not going to help me so much.
So I really want to structure my data in such a way that when I access one element, I'm also going to access the elements nearby it.
Then this blocking will actually be useful.
This is a property normally called spatial locality.
And the other thing we'd like-- these caches have some size, so I can store more than just one block.
It's not like I read one block, and I just finish processing it, and then I read the next block and go on.
Some of these caches are actually pretty big.
If you think of main memory as a cache to your disk, that can be really big.
So ideally, the blocks that I'm using here relate to each other in some way, or when I access the block, I'm going to access it for awhile, along with other blocks.
So the way this is usually said is that we'd like to reuse the existing blocks in the cache as much as possible.
And this you can think of as temporal locality.
When I access a particular block, I'm going to access it again fairly soon.
That way it's actually useful to bring it into my cache, and then I use it many times.
That would be even better.
I don't have to have both of these, and exactly to what extent I have them is going to dictate what the overall time it's going to take to run my algorithm.
But these are so the ideal properties you want in a very informal sense.
Now, in the rest today, we're going to make this formal, and then we're going to develop some algorithms for this model.
But this is the motivation.
In reality, we're free to choose block size in the system.
Though, in a moment, I'm going to assume that it's given to us.
You'd normally set the block size so that these two terms come out roughly equal.
Because if you're spending the latency time to go and get something, you might as well get a whole chunk of something, according to whatever your bandwidth is.
If it only cost you, say, twice as much to fetch an entire block than to fetch one word, that seems like a pretty good block size.
So for something like disk, that block size is on the order of megabytes, maybe even bigger-- hundreds of megabytes.
So think of the block sizes as really big.
We really want all that data to be useful in some way.
Now it's really hard to think about a memory hierarchy with so many levels.
So we're going to focus on two levels at a time-- the sort of the cheap and small cache versus the huge thing, which I'll call disk, just for emphasis.
So I'm going to call this two-level model the external memory model.
It was originally introduced as a model for main memory versus disk.
But you could apply it to any pair of levels.
In general, you have your problem size N, choose the smallest level that fits N. Typically that's main memory.
Maybe it's disk.
And just think of the level between that and the previous, so the last level and the next to last level.
Often that's what matters.
Like if you run a program, and you run out of RAM, and you start swapping the disks, that's when everything just slows to a crawl.
You can see that difference at each of these levels, but it's probably most dramatic at disk just because it's so slow-- a million times slower than RAM, or at least 1,000 times slower than RAM, I should say.
Anyway, so we have just two levels.
So let me draw a more precise picture.
We have the CPU.
This is where all of our operations are doing this, where we add numbers and so on.
We'll think of it as having a constant number of registers.
Each register is one word.
And then we have a really fat pipe, low latency pipe, to the cache.
Cache is going to be divided into blocks.
So let's say there's B words per blocks.
Instead of writing block size, I'll just write capital B.
And the number of blocks.
I'm going to call M over B.
So the total size of your cache is capital M. And then there is a relatively thin and slow connection-- this one's fast.
This one's slow-- to your disk.
Disk we'll think of as huge, essentially infinite size.
It's also divided into blocks of size B, so same block size.
So this is the picture.
And so, initially, all of the input is over here, all of your end data items, whatever.
So you want to sort those items.
And in order to access those items, you first have to bring them into cache.
That's going to be slow, but it's done in a blocked manner.
So when I can't access an individual item here, I have to request the entire block.
When I request that block, it gets sent over here.
It takes a while.
And then I get to choose where to store it.
Maybe I'll put it here.
And then maybe I'll grab this block and then store it here and so on.
Each of those is a block read, so these are new instructions the CPU can do.
And eventually, this cache will get full.
And then before I bring in a new block, I have to kick out an old lock.
Meaning I need to take one these blocks and write it to some position, maybe to the same place.
I think, in fact, we will always assume that you write to the same place, overwrite what was on the disk.
You made some changes here, send it back.
And, in general, what we're going to do is count how many times we read and write blocks.
Question?
When you talked about how fast the connection is, you're just talking about latency, right?
Yes, sorry, this is latency
Yeah, so like the [INAUDIBLE] connections [?
just don't have ?]
[INAUDIBLE]?
Right, this could have huge bandwidth.
So in this model, we're assuming the block size is fixed, and then the latency versus bandwidth is not-- we're not going to think about bandwidth.
We'll assume the block size has been chosen in some reasonable way.
And then all we need to do is count the number of blocks.
But underneath, yeah, you have some kind of bandwidth.
Presumably you set the block size to make these two things roughly equal, and so then latency and bandwidth are kind of the same thing.
That's the idea.
But really, we're just going to think about counting latency, which is how many times do I have to request to block and wait for it to come over, and how much does it cost to write a block?
How many times do I write a block?
I'm not going to worry about how much physical time it takes me to do either of those things; I'm just going to count them and assume that that is what I need to minimize.
So I'm going to count-- we call these memory transfers-- transfers of blocks between levels, between these two levels.
This is the number of blocks read from or written to disk.
We're going to view accesses to the cache as free.
I'm not going to count those.
You don't need to worry about that so much because we can still count the number of operations that we do on the computer, on the CPU.
We still can think about how much time, regular time, it takes to do the computation-- how many comparisons, how many additions, things like that.
And that would include things like reading and writing elements from cache-- individual things.
But we're going to view this connection-- let's say, these are on the same ship.
So reading cache is just as fast as reading from registers.
So we're not going to worry about that time.
What we're focusing on, for the purpose of this model, is between these two levels.
So these are essentially one level combined.
I'll change that in a little bit.
But for now, just think about the two levels.
And we're counting how many memory transfers do we have between these two levels, cache and disk.
So we want to minimize that.
Now, just like before, we want to minimize the running time in the usual traditional measure.
And we want to minimize space and all the usual things we minimize.
But now we have a new measure, which is number of memory transfers, and we want our algorithm to minimize that too, for a given block size and for a given cache size.
And at this point-- I'm going to change this in a moment-- the algorithm that we would write in this external memory model explicitly manages the blocks.
It has to explicitly read and write blocks.
And there's a software system that implements this model, particularly for disk, and lets you do this in a nice controlled way, maintain your memory, maintain reading and writing disk.
The operating system tries to do this, but it usually does a really bad job with swapping.
But there are software systems that let you take control and do much better.
So that's a good model.
External memory model is especially good for disk.
It's not going to capture the finesse of all these other levels, and it's a little bit annoying to write algorithms in this way-- explicitly reading and writing blocks.
Today I will not write any such algorithms.
Although, you could think about them.
I personally love this other model, which is cache obviousness.
It's going to lead to, in some sense, cleaner algorithm.
Although, it's more of a magic trick to get them to work.
But writing the algorithms is very simple.
Analyzing them is more work.
And it will capture, in some sense, all of these levels.
But, in fact, it is basically exactly this model, almost the same.
We're going to change one thing, which is where the oblivious comes from.
We're going to say that the algorithm doesn't know the cache parameters.
It doesn't know B or M. So this is a little weird.
We're going to have to make some other changes to make it work.
From an analysis perspective, I want to count memory transfers and analyze my algorithm with respect to this memory hierarchy.
But the algorithm itself isn't allowed to know what that member hierarchy looks like.
Another way to say this is that the algorithm has to work simultaneously for all values of B and all values of M. As you might imagine, this is not so easy.
But there are some simple things where this is easy, and more complicated things where this is possible.
And it gives you all sorts of cool things.
Let me first formalize the model a little bit.
The other nice thing about cache oblivious algorithms is it corresponds much more closely to how these caches work.
When you write code on your CPU, you may have noticed you don't usually do block reads and block writes, unless you're dealing with flash or disk.
All of this is taking care for you.
It's all done internal to the processor.
When you access a word, behind the scenes, magically, the system, the computer, finds which word to read or which block to read.
It moves the entire block into a higher level cache, and then it's just serving you words out of that block.
And you don't have explicit control over that.
So the way that works is when you access a word in memory-- and I'm going to think of memory as everything; this is what's stored in the disk, say.
This is the entire memory system, the entire memory hierarchy.
And, as usual in this class, we're going to think of the entire memory as a giant array of words.
Each of these squares is one word.
But then also, the memory is now divided into blocks.
So let's say every four.
Let's say B equals 4.
Every four words is a block boundary, just for the sake of drawing a figure.
So this is B equals 4.
When you access a single word, like this one, you get the entire block containing the word.
Let's say, to emphasize, it's not you personally; the system somehow fetches the block containing that word.
It has to do this automatically.
We can't explicitly read and write blocks in this model because we don't know how big the blocks are.
So it couldn't even name them.
But internally, on the real system and in your analysis, you're going to think of whenever you touch something, you actually get all this into the cache.
So you hope that you will use things nearby because you've already read them in.
Ideally, they're useful.
But you don't know how many you've read in.
You've read in B, and you don't what B is.
The algorithm doesn't now.
One more detail-- the cache is going to get full pretty quickly.
And so then, whenever you read something, you have to kick something out.
In steady state, cache might as well always stay full-- no reason to leave anything empty.
So which block do you kick out?
Any suggestions?
Which block should I kick out?
If I've been reading and writing some blocks, reading and writing to words within these blocks.
Yeah?
[INAUDIBLE]
The block that was fetched farthest in the past?
Yeah that is usually called First In, First Out.
That's FIFO.
And that is a good strategy.
Any other suggestions?
Yeah
[INAUDIBLE]
The block has been least recently used.
So maybe you fetched it a long time ago, but you use it every clock cycle.
That one you should probably not throw away because you use it a lot.
That's called LRU, and that is also a good strategy.
Other suggestions?
Those are two good ones.
If you go beyond that, I'm worried I won't know.
But there are some bad strategies.
Yeah?
Just random
Random-- yeah, random is probably pretty good.
I don't know offhand.
There are some randomized strategies that beat both of those.
But from this perspective, both are good.
We've got lots of Frisbees to go through, so.
That's a good answer.
Random is definitely a good idea.
I know there's a randomized strategy called [?
bit, ?]
that in certain senses is a little bit better.
But from my perspective, I think all of those are good.
Random, I have to double check whether you lose a log factor.
And expectation should be fine.
So all of those strategies will work.
You could define this model with any of them.
I think it would work fine, except randomize, you'd get an expectation bound.
So the system evicts, let's say, the least recently used page.
The least recently loaded page would also work fine.
That's FIFO.
Sorry I'm switching to page, but I've been calling them blocks.
Blocks and pages are the same thing for this lecture.
And either at the end of this lecture or beginning of next, I'll tell you why that's an OK thing.
But let's not worry about it at this point.
So now we have a model-- cache flow oblivious.
We have two models, actually.
But I think now that the cache flow oblivious model is complete, we're going to analyze.
Again, we're still counting the number of memory transfers in this thing.
The algorithm's just not allowed know B and M, and so we had to change the model to make the reading and writing of blocks automatic because the algorithm's not allowed to do it.
So someone's got to.
The cool thing about cache oblivious model is every algorithm you see in this class, or most of the algorithms you see in this class, are in a certain sense cache oblivious algorithms.
They weren't aware of B and M before, still not.
What changes is now you can analyze them in this new way, in this new model.
Now, as I said, all the algorithms we've seen are not going to perform well in this model-- almost all.
But that makes things interesting, and that's why we have some work to do.
I have some reasons why cache obliviousness-- why would you tie your hands behind your back and not know B or M?
Reason one, it's cool.
I think it's pretty amazing you can actually do this.
I guess that's reason two is you can actually do it for a lot of problems we care about.
Cache oblivious algorithms exist that are just as good.
So, I mean, of course they exist.
But there are ones that are optimal.
They're within a constant factor of the best algorithm when you know B or M. So that's surprising.
That's the cool part.
In general, the algorithms are easier to write down because we can use pseudo code just like before.
We don't need to worry about blocking in the algorithm.
The analysis is going to be harder, but that's unavoidable.
In some sense, it makes it easier to write code.
And it's also a little easier to distribute your code because every computer has different block sizes that matter.
Also, as you change your value of N, a different level in the memory hierarchy's going to matter.
And so it's annoying-- each of these levels, I didn't mention, has a different block size and, of course, has a different cache size.
So tuning your code every time to a different B or M is annoying.
The big gain here, though, I think, is that you capture the entire hierarchy, in a sense.
So in the real world, each of these pipes has its own latency.
And let's just think about latency.
And you'd like to minimize the number of block transfers between here and here.
You'd like to minimize the number block answers here here.
Well, OK, I can't minimize all of them.
That's a multi-dimensional problem.
What I'd like to minimize is some weighted average of those things-- latency times number of blocks here, plus the latency times the number of blocks here, plus latency times the number of blocks here, and so on.
If you can find an optimal cache oblivious algorithm and analyze it just with respect to two levels, because the algorithm's not allowed to know B and M, it has to work for all levels.
It has to minimize the number of block transfers between all these levels, and so, in particular, will minimize the weighted sum of them.
It's a bit hand wavy.
You have to prove something there.
But you can prove it.
So there's a paper about this from 1999 by Frigo, Leiserson, Prokop, and Ramachandran.
It's old enough that I remember all the names.
After about 2001, when I became a professor, I can't remember anything.
But before that, I can remember everything.
So Frigo, we've talked about him in the context of FFTW.
That was the fastest Fourier Transform in the West.
So he was a student here.
And FFTW uses a cache oblivious Fast Fourier Transform algorithm.
Leiserson, you've probably seen on the cover of your textbook or walking around Stata.
Professor Leiserson here at MIT.
And Prokop, this is actually his [?
M Enge ?]
thesis.
So pretty awesome [?
M Enge ?]
thesis.
All right, so cool, I think I said all the things I wanted to say.
So if you want to see the proof that you can solve the entire memory hierarchy, you can read their paper.
You have to make a couple of assumptions, but it's intuitive.
Cache oblivious has to work for all B and M, so it's going to optimize all the levels simultaneously.
Doing that explicitly, with all the different B's and M's, that would be really messy code, probably also slower.
Cache oblivious is just going to do it for free with the same code.
All right, let's do some algorithms.
There's one easy algorithm which works great from a cache oblivious perspective, which is scanning.
Let we give you some Python code.
For historical reasons, in this field, N is written with a capital letter.
Don't ask, or don't worry about it.
So here's some very simple code.
Suppose you want to accumulate an array.
You want to add up all of the items in the array or multiply them or take them in or whatever.
This is a typical kind of thing.
Again, an array, we're going to think of-- so here was my memory.
We're going to think of the array as being stored as some contiguous segment of that array, let's say, this segment.
So this is important.
Assume array is stored contiguously, no holes, relative to how it's mapped on to memory.
And this is a realistic assumption.
When you allocate a block of memory, the promise by the system is that it's essentially a contiguous chunk of memory or disk, or whatever.
And when Python makes an array, it does this.
It guarantees that these things will be stored contiguously.
If you use a dictionary, this would not be true.
But for regular [?
array's ?]
list, this is true.
So I'm accessing the items in the array in order, and so I start here at item zero.
I end up with item N minus 1.
That seems good because I read this one.
I get the whole block.
Then I read this one.
I already had that block.
It's free.
This one's free.
This one's free.
Here I have to read a new block.
But then this one's free.
So the first item I access in each block costs one, but as long as my cache store's at least one block, that's enough.
And let's say the sum is a register; that's enough to remember that block so that the next operation I do will be free.
So the cost is going to be-- actually, be a little more precise-- ceiling of N over B almost.
Without the big O here, this is right in the external memory model, but not quite right in the cache oblivious model.
Can someone tell me why?
Yeah?
If N is two, you could have it beyond a border [INAUDIBLE]
Good, N could be two.
But it could span a block boundary.
Remember, the algorithm has no idea where the block boundaries are.
And again, in reality, there are block boundaries all over the place, and there's no way to know.
You can't request that when you allocate an array it always begins in a block boundary.
So great, you can span block boundaries in-- oh, way off.
I just spanned a block boundary, sorry.
So it's going to be, at most, ceiling over N over B plus 1 cache obviously.
So it's just going to hurt you by one.
But I want to point out, there's a slight difference between the two models, even with this very simple algorithm.
In general, I'm just going to think of this as big O N over B plus 1.
There's some additive constant.
I guess you could even say it's N over B plus big O 1, but we won't worry about constant factors today.
So that's scanning, cache oblivious external memory, both great.
Slightly more interesting--
[INAUDIBLE]?
Yeah, in the external memory algorithm, because you're explicitly controlling the blocks, you're explicitly reading and writing them.
And you know where the block boundaries are.
You could, if you wanted to, you don't have to, but you could choose the array to be aligned, to be starting at a block boundary.
So that's the distinction.
In the cache oblivious, you can't control that, so you have to worry about the worst case.
External memory you could control it, and you could do better, and maybe you'd want to.
It will hurt you buy a constant factor.
And in disks, for example, you want things to be track aligned because if you have to go to an adjacent track, it's a lot more expensive.
You've got to move the head.
Track is a circle, what you can read without moving the head, so great.
So slightly more interesting is you can do a constant number of parallel scans.
So that was one scan.
Here's an example of two scans.
Again, we have one array of size N. Python notation, that would be the whole thing.
And what I want to do is swap Ai with-- this is not Python, but it's, I think, textbook notation.
But you know what swap means.
What does this do, assuming I got my minus ones right?
Yeah?
It reverses the array
It reverses the array, good.
We'll just run through these Frisbees.
So this is a very simple algorithm for reversing the array.
It was originally by John Bentley, who was Charles Leiserson's adviser-- PhD adviser-- back in the day.
So very simple, but what's cool about it, if you think about the array and the order in which you're accessing things, it's like I have two fingers-- and I should have made this smaller.
So here, we'll go down here.
I start at the very beginning of the array and the very end of the array.
Then I go to the second element, next to last element, and I advance like this.
So as long as your cache M, the number of blocks in the cache is at least two, which is totally reasonable.
You can assume this is at least 100, typically.
You've got at least 100 blocks, say.
So for any fixed constant, we're going to assume N over B is bigger than a constant.
We'll only need like two or three or four for the algorithms we cover.
Then great, when I access this item, I will load in the block that contains it.
I don't know how it's aligned, but don't care so much.
And then I load in the block that contains this item.
And then the next accesses are free until I advance to the next block.
But once I advance to the next block on the left or the right, I'll never have to access the old ones.
And so again, the cost here is just going to be equal to the number of blocks, which is big O of N over B plus 1.
So a constant number of parallel scans is going to be basically the number of blocks in the array.
So N is smaller than B, this is a bad idea or not so hot.
But when N is bigger than B, this is just N over B.
That's how much it takes to read in the data-- big deal.
So these are boring cache oblivious algorithms.
Let's do interesting ones.
And I would say the central idea in cache oblivious algorithms is to use divide-and-conquer.
This goes back to the first few lectures in this class.
And so we will go back to examples from there.
Today we're going to do the median finding, in particular, which we did in lecture two, so really a blast from the past.
But it's good review because the final covers everything, so you've got to remember that.
Matrix multiplication, we've talked about, but not usually-- well, I guess we did actually use divide-and-conquer for Strassen's algorithm.
We're going to use -and-conquer even for the boring algorithm today.
And then next class, we're going to go back to van Emde Boas, but in a completely different way.
So if you don't like van Emde Boas, don't worry; it's much simpler.
So let's do median finding.
Or actually, sorry, let me first talk about divide-and-conquer in general.
You know what divide-and-conquer is.
You take your problem.
You split it into non-overlapping subproblems, recursively solve them, combine them.
But what I want to stress here is what it's going to look like in a cache oblivious context.
So the algorithm is going to look like a regular divide-and-conquer algorithm.
So, in particular, the algorithm will recurse all the way to, let's say, constant size problems, whatever the base case is.
So same as usual, but what's different is the analysis.
When we analyze a cache oblivious algorithm, then we get to know what B and M are.
In some sense, we're analyzing for all B an M. But let's suppose B and M is given to us, then will tell you how many memory transfers you need.
This kind of bound, you need to know what B is to know what the value of this bound is.
But you learn it as a function of B and, in general, a function of B and M, and that's the best you could hope for as a complete characterization.
So in the analysis, let's just look at one value of B and one value of M. So analysis knows B and M, and it's going to look at, let's say, the recursive level, where one of two things happens.
Either the problem size fits in order one blocks.
So meaning it's order B size.
That's an interesting level.
Another interesting level, the more obvious one probably, is that it fits in cache.
So that means that the size is less than or equal to capital M. Everything here is counted in terms of words.
This is the more obvious one.
For a lot of problems, the cache size isn't so relevant.
What really matters is the block size.
For example, scanning, you're only looking through the data once.
So it doesn't matter how big your cache is, as long as it's not super tiny.
As long as it has a few blocks, then it's just a function of B and N, no M involved.
So for that kind of problem this would be more useful-- constant number of blocks.
Because I think of the cache M as being larger than any constant times B, this is strictly smaller, or this is smaller or equal to problem fitting in cache.
So when M is relevant, we'll look at this level and maybe the adjacent levels in the recursion.
So the algorithm doesn't know what B and M are, so it's got to recurse all the way down-- turtles all the way down.
But the analysis, because we're only thinking about one value B and M at a time, we can afford to just consider that one level, and that will be like the critical place where all the cost is.
Because once things fit in cache and you've loaded things in, the cost will be zero.
So below that, the base case is kind of trivial.
So basically what this is going to do is make our base cases larger.
Instead of our base case being constant, it's going to be order B or M. What don't I need?
So now let's going to median finding.
Median finding, you're given an unsorted array.
You want to find the median.
And in lecture two, we had a linear time worst case algorithm for this.
And so my goal today is to make it this running time.
This is what you might call linear time in the cache oblivious model because that's how long it takes just to read the data.
It turns out basically the same algorithm works.
First, you've got to remember the algorithm.
So let me write it down quickly.
This is the sort of five by in N array.
So think of the array as being partitioned into, I'll call them, five columns.
So this picture of five dots by N over 5 dots-- this is dot, dot, dot.
So this is five.
Now, we didn't talk about it then, and there's a few different ways you could actually implement it, but let's say these-- the actual array is one-dimensional.
Let's say these are the first five items.
These are the next five items.
So, in other words, this matrix is stored column-by-column.
This is just a conceptual view.
So we can define it either way, however we want.
So I'm going to view it that way.
And then what the rest of the algorithm did was for sort each column, it's only five items, so you can sort it in constant time, each one.
But, in particular, what we care about is the median of those five items.
Then we recursively found the median of the medians.
This is the step we're going to have to change a little bit.
Then we-- leave a little bit of space.
Then we partition the array by x.
Meaning we split the array into items less than or equal to x and things greater than x.
We probably assumed there was only one value equal to x, but it doesn't matter.
And finally, we recurse on one of those two halves.
So this is a pretty crazy divide-and-conquer algorithm, one of the more sophisticated ones.
You don't need to know all the details here, just that it worked and it ran in linear time.
What's crazy about it is there are two recursive calls.
Usually, like in merge sort, where you do two recursive calls and spend linear time to do the stuff, like this partition, you get n log n time, like merge sort.
Here, because this array is a lot smaller, this is a size N over 5.
And this one was reasonably small; it was like [?
M of ?]
7/10 N. Because 7/10 plus 1/5 is strictly less than 1, this ends up being linear time instead of n log n. That's just review.
Now, what I'd like to do is the same thing, same analysis, or same algorithm, but now I want to analyze it in this two-level model.
So actually, I will erase this board.
So now my array has been partitioned into blocks of size B, like this picture.
In fact, it's quite similar.
Here, we're partitioning things into blocks, but they're size five.
That's different.
Now someone has partitioned my array into blocks of size B. I need to count how many things I access.
Well, let's just look line-by-line at this code and see what we do.
Step one, we do absolutely nothing.
This is a conceptual picture, so zero cost, great.
Step one is zero, my favorite answer.
Step two, we sort each column.
How long does this take?
What am I doing?
It's right above me
N over
N over B because this is a scan.
It's a little bit weird of a scan.
We look at five items, and then we look at the next five items, and then the next five items.
But it's basically a scan.
You could think of it as almost five parallel scans, I suppose, or you could just break into the case where maybe if B is a constant, then it doesn't matter what you do.
But if B bigger than a constant, then reading five items, those are all probably going to be in one block, except the ones that straddle the block boundaries.
So in all cases, for step two-- maybe I should rewrite step one-- zero cost.
Step two, is order N over B plus 1, to be careful.
That's a scan.
Actually, it's two parallel scans because we have to write out these medians somewhere, so we'll have to.
Step three is recursively find the medians.
Now, before, we had in T of N is T of N over 5 plus T of 7/10 N plus linear.
In this new world-- this is regular running time.
In this new world, I'm going to use a different notation for the recurrence, MT of N for memory transfers.
This is a good old-fashioned time, and this is our new modern notion of time-- how many block transfers do I need to do for problem size N. So this is a recursion, and should be MT of N over 5.
But, and this is important, for this to be a same problem of the same type, I need to know that the array that recursing on is stored contiguously.
Before, I didn't need to do that.
I could say, well, let's put the medians in the middle.
So now every fifth item in this array is my new subarray.
And so I could recursively call this thing and say, OK, here's my array, but really only think about every fifth item.
That's like a stride in the array.
And then the next recursive level, oh, only worry about every 25th item.
And every 5-cubed item, I'm going to stop computing, and so on.
And that would be fine for regular running time.
But when I get my stride gets bigger and bigger, at some point, every item is going to be in a different block.
That's bad.
I don't want to do that.
So when I find these medians, or when I recurse, I need that the medians that I'm recursing on are stored in a contiguous array.
Now, this is easy to do.
But we didn't have to do before.
That's the key difference.
Make sure they are stored contiguously.
I can do that because when I sort each column in one scan, I can have a second scan which is the output, which is the array of medians.
So as I'm scanning through the input, I'm going to output the median.
It's going to be 1/5 the size.
Then I've got all the medians nicely stored in a contiguous array.
So with order-one parallel scans, same time here, this is actually a legitimate recursive call.
Then we partition.
Partition, again, is a bunch of parallel scans, I think, three.
You've got one reading scan, which is you're reading through the array, and you've got to writing scans.
You're writing out the elements less than or equal to x, and you're writing out the elements greater than x.
But again, all of those are scans.
You're always writing the next element right after the previous one.
So if you already have that block in memory and if you assume that the number of blocks in cache is at least three, then three parallel scans is fine.
It's different from the CLRS partition algorithm.
That one was fancy to be in place.
We're not trying to be in place or fancy at all.
Let's just do it with a bunch of scans.
So now we have two arrays-- the element less than x, the elements greater than x.
Then we recurse on one of them, and those elements are consecutive already, so good.
This is a regular recursive call.
Again, we're maintaining the variant that the array is stored contiguously.
And by the old analysis, that array is sized at most 7/10 N. So I get a new recurrence, which is MT of N is MT of N over 5 plus MT-- this analysis feels very "empty--" plus N over B-- sorry, bad joke-- N over B plus 1.
So basically the same recurrence, but now N over B plus 1 for what we're doing here.
But I had to change the algorithm a little bit for this recurrence to be correct, for it to correctly reflect the number of memory transfers.
Now all we need to do is solve the recurrence.
And actually, in some sense, more importantly, we need to figure out what the base case is.
Because we could say, all right, here's the usual base case.
If I have a constant-sized problem, well, that's going to be constant.
This is our base case for every recurrence we've ever done.
And that's enough usually.
It's going to give us a really bad answer here.
So let's go off to the side here and solve that recurrence.
So if that's my base case, well, in particular-- so this is some recursion tree.
It's very uneven, so it's kind of annoying to draw.
But what I know with this base case, this overall MT event is going to at least the number of leaves in the recursion tree.
So let's say MT of N is at least L of N, number of leaves in the recursion.
So this is really if I run the algorithm, how many base cases of constant size do I get?
And that satisfies-- so it's not obvious what that is.
There's no plus here.
Number of leaves is just how many leaves are over here, how many leaves are over here, and L of 1 equals 1, say, or some constant equals constant.
I happen to know, because I saw lots of recurrences, this solves to some N to the alpha.
I claim that L of N is N to the alpha for some constant alpha.
Why?
I'll just prove that it works.
So this is now N over 5 to the alpha, and this is 7/10 N to the alpha.
If it's going to work, this recurrence should be satisfied.
And now, if you look at this equation, there's a lot of N to the alphas, and they all cancel.
So I get 1 equals 1/5 to the alpha plus 7/10 to the alpha.
It's confusing because I was just watching the TV show Alphas, but no relation.
So this is now something purely in terms of alpha.
You just need to check that there is a real solution.
There is one.
You have to plug it into Wolfram Alpha or something, no pun intended.
Wow, they're just coming out today.
And then alpha is... next page...
I can't do this by hand.
Something like .83978.
So we get L of N is say at least N to the 0.8th bigger.
It's sublinear and that was enough when we cared about time but now it's bad news because N over B... our goal was to get N over B+1.
If B is huge, if B is bigger than N to the 0.2, then we are not achieving this bound.
Right.
We are always are paying at least N to the 0.8.
For example B is roughly N. We are way off!
But that's because we used the wrong base case.
Turns out if you use a better base case, things just work.
So let's do that.
I think its going to be smaller.
So... the next base...
I mean...
When you are doing cache full release analysis you never use this base case.
The first one you should think about is this one.
If you have a problem of size that fits in a constant number of blocks.
Well of course that's going to take... once they are read into the cache, you are not going to pay anything.
How long does it take to read a constant number of blocks into cache?
Constant number of memory transfers.
Okay, this is obviously a strictly better base case than this one.
Because we have the same thing on the right hand side as a constant but we've solved a larger problem.
So clearly you should cut here, instead of there.
Then the number of leaves in this recursion...
So same recurrence- different base case.
So we'd stop recursing conceptually in the analysis, the algorithm goes all the way down, but in the analysis we stop recursing when we reach a problem of size B.
The number of leaves in that new recursion tree will be N over B to the alpha.
That's good!
That's smaller than N over B. OK, now I'm going to wave my hands a little bit and say, MT of N is order N over B plus 1.
I guess to do that, you want to prove it the same way we did before when we solved this recurrence, which is by substitution.
You assume this is true, you plug it in, verify it can actually be done with some constants.
The intuition of what's going on is, in general, this recurrence is dominated by the root.
The root cost for this recursion is N over B plus 1.
So this is the root cost.
I claim that, up to constant factors, that is the overall cost.
Roughly because, as you go down the recursion tree, the cost is decreasing geometrically.
But that's not obvious for this recurrence because it's so uneven.
But it's kind of like the master method, a little fancier.
Intuitively, this should be obvious.
There's the root cost and then there's the other ones.
But to actually prove it, you should do substitution method.
I want to go to more interesting algorithms instead, but any questions before we continue?
All right.
So next algorithm, that was median, now we're going to do matrix multiplication via divide and conquer.
So what we just saw was an example where, in divide and conquer, in the analysis we think about the case where things fit in a constant number of blocks.
That was sort of case one.
The next example, matrix multiplication, will be the other case.
So you get to see both types.
So multiplying matrices, something we've done many times.
For example, in the FFT lecture and in the Strassen's algorithm, just to remind you.
I'm just thinking about the square case, although this generalizes.
We have two square matrices, N by N. Normally, I would say C equals A times B, but I realized we used B for block side.
So this is going to be s equals x times y. Hopefully that doesn't conflict with anything else, but no B's.
All right, so standard matrix.
Let's start with the standard algorithm.
Let's start by analyzing that.
Because if you're reasonably clever, this the standard algorithm is not so bad.
So in general, this won't matter too much.
Let's suppose we're doing z row-by-row, and let's say we're currently computing this product cell.
So that product cell is the dot product this ZIJ here is the dot product of this row with this column.
How do I compute dot products?
Two parallel scans.
Right?
I scan through this row and I parallel scan through this column.
Now, it depends the order in which you store x and y, but let's suppose we can store x in row major order, meaning row-by-row, and we store y in column major order, meaning column-by-column.
Then this will be an honest-to-goodness scan of a contiguous array.
Again, the order we store things in memory really matters.
So let's make our life ideal.
Let's say that this is row-by-row and this one is column-by-column, then hey, this is two parallel scans so order N over B to compute this cell.
OK, I claim that computing ZIJ costs N over B, so maybe plus 1.
Again, these are end-by-end matrices, so total size N squared, which means the total cost is what?
N cubed over B plus N squared, I guess.
Seems pretty good.
I mean, we had a running time of N cubed before and we divided by B.
How could you possibly do better?
Well, by being smarter.
This is not optimal, you can do better.
It's not obvious, but let me just spend a little more time convincing you this is the right answer.
Not only is this big O, but for appropriate settings-- in the worst case this is going to be theta.
Because if you think of the order in which we're-- see, we look at these rows several times.
And if you look at, when I compute this cell and this cell and this cell of the z matrix, or the product matrix, each of them uses the same row of x.
So maybe you could reuse that.
You could reuse that row of x.
That might actually be free, depending on how B and N relate.
But the columns of y, those are different every time.
When I compute this one, I use the first column of y, when I compute this one I use the second column of y.
Unless the cache is so big that it can store all of y, which is like, you could store the entire problem in cache that's unrealistic.
So unless M is bigger than N squared, in this algorithm at least, you have to read a new column of y every single time.
So that's why it's theta N over B plus 1.
You need to spend N over B, assuming M is less than N squared.
OK. And I claim this is not the best you can do because we're going to do better.
And we're going to do better by divide and conquer.
Now, you've already seen divide and conquer used for matrix multiplication to get Strassen's algorithm, and the idea there is to use blocks.
So this is sort of an algorithm you've already seen.
I'm going to divide the matrix z into N over 2 by N over 2 sub-matrices.
Each of these ZIJs is an N over 2 by N over 2 matrix.
And I do the same thing for x and y.
Numbers are right.
1, 2, y, 2, 1, and so on.
And you can write this out explicitly.
I prefer not to do all of it, but let's do one of them.
You can just think of these as two-by-two matrices, because matrix multiplication is associative and good things happen.
I can just take these two elements-- but they're actually matrices, sorry.
I might take these two and dot product with these two.
And I get x1,1 y1,1 plus x1,2 y2,1, and that's what I should set z1,1 to.
So this is a formula, but it's also a recursive algorithm.
It says, if I want to compute z I'm going to say, well, there are four sub-problems.
The first one is to compute z1,1, and I'm going to do that by recursively computing the product of x1,1 and y1,1, recursively computing the product of x1,2 y2,1 and then adding them together.
This is not recursive.
Addition is easy.
OK. And there's two products here, two products here, two products here, two products here, a total of eight products, so we're going to have eight recursive calls in size N over 2.
If we look at the number of memory transfers, this is 8 times recursive call on N over 2 by N over 2 sub-matrices plus the cost of addition.
And I claim the cost of addition is at most N squared over B plus 1, because addition is basically parallel scans.
I can scan through x, scan through y.
As long as they're stored in the same order, I just am adding them element by element, and there's a third scan, which is writing out the z vector once things are linearized.
Now, for this to work, for this to be a true recursion, I need that, say, x1,1 and y1,1 are stored as contiguous things in memory.
So this means that the layout of a matrix, let's consider the matrix z, is going to be like the following.
I'm going to recursively lay out 1,1-- so when I say lay out, I mean what order do I store the elements in memory?
What order do I store the cells in memory?
And what I'm going to say is, recursively lay out the pieces-- there's four pieces-- recursively call layout of those and then concatenate them together.
That's my layout.
So I'm going to store all of these items, then I'm going to store all of these items, and then all of these items, then all these items.
How do I store these items, in what order?
Recursively.
So I'm going to divide them like this, store these before these before these before these, how do I store these?
Recursively.
OK, same recursion.
So it's a really weird order, it's a divide and conquer order.
There's only four things here.
In what order should I combine the four things?
Doesn't matter.
All that matters is that this is consecutive, this is consecutive, and this is consecutive, so that when I recurse, I'm recursing on consecutive chunks of memory.
Otherwise the analysis just won't work.
So for this to be right, got to have this layout.
OK. Now we just need to solve the recurrence, and we're done.
I already told you, the base case we're going to use is this one.
We're going to use this one because it's stronger and better, and we'll need it, in this case, to get a better analysis.
You could solve it using the weaker base cases, you'll get larger numbers.
But if you use the strongest base case, MT of-- it's not M. Got to be a little careful.
Because N here is actually just one side length.
This is an end-by-end matrix, so the total size is N squared-- actually the total size is 3N squared, so this is going to be the square root of M over 3, at some constant, times the square root of N. It actually doesn't matter what the constant is.
But this is the size of-- this is the value of N for which all three matrices will fit in cache.
So I claim we know this costs at most M over B memory transfers, because we were kind of stroke here because we know that all of these guys fit in cache and because we know that they can store it consecutively in memory, well three consecutive chunks.
Once, no matter what I do, there are only M over B blocks there, and so at worst I read them all in.
But once the cache is filled with them, for the duration of this recursion, I won't be reading any other blocks, and so the cache will just stay full with the problem.
And so I never pay more than this.
So that's the base case.
Easy, but you have to think about it for a second.
Cool.
Now we have a recurrence and a base case, and now we have a good old fashioned recursion tree.
This one I can actually draw, because it's-- well, partly because it's nice and uniform.
It just explodes rather fast.
So at the top we have a cost of N squared over B plus 1, and we have eight recursive calls.
And the recursive calls are to something in size N over 2 squared over B, also known as N squared over 4B.
OK, so if I add up everything on this level, I get N squared over B, and if I add up everything on this level I'm going to get 8 times N over 4-- is that right?
Yeah.
So 2 times N squared over B. OK.
I did that in order to verify that the cost per level is increasing geometrically, so all that will matter is the leaf level.
This is the proof of the master theorem.
When things are doubling at every step-- and this was just a special case, but every level would look the same-- every level of recursion, if you add them all up, you're getting twice as much as you had at the previous level.
So all that will matter is the leaf level.
OK, the leaf level.
Actually, maybe I'll do it over here.
First question is how many leaves are there?
The leaves are this thing.
So the way I would think about this is, because everything is nice and uniform, is 8 to the power of the number of levels.
What's the number of levels?
Well, we're dividing by 2 each time, so it's going to be log of something, but it's no longer log N because we're stopping early.
We're stopping when N reaches this value.
So it turns out that is N divided by that value.
This is, how many times do I have to multiply by 2 before I get to this, which is the same thing as how many times do I have to divide N by 2 before I get that?
Think about it.
OK, but 8 to the log.
This is 2 to the 3 times log.
2 to the log is just the thing.
So this is N over root M over B-- so many overs-- to the third power.
OK, this is starting to look familiar.
This is N cubed, that should appear somewhere, divided by square root of M over B.
This is the number of leaves.
Now, for each leaf we're paying this cost, so the overall cost of MT of N is going to be this times this.
So let's do that and simplify.
So MT of N is going to be big O, because we're taking the leaf level but there's some other things that's just going to lose us a factor of 2.
We have this thing multiplied by this thing.
So we've got N cubed over square root of M over B times M over
You--
I made a mistake.
Yea, thank you.
This was supposed to be cubed.
So this was M over B to the 1/2, so now we have, down here, M over B to the 3/2.
Thank you, thought that looked weird.
All right.
M over B to the 3/2
[INAUDIBLE]
Yeah.
What was I doing here?
This is supposed to be M over 3.
I was not missing a stroke, thank you.
M over 3, this is supposed to be M over 3.
Wow.
OK, so this is M over 3.
I'm just going to drop the-- well, I'll put it here.
But then I'm just going to write theta so I can forget about the 3, because that's just a square root of 3 factor.
So now this is going to be M to the 3/2.
That makes me much happier.
Did I get it right this time?
Let's double-check.
So this is square root of M to the 3 power, so that's M to the 1/2 cubed M to the 3/2.
I think that's good, this base case was square root of M. OK, get it right.
So now this is M to the 3/2.
There is a square root that's going to come back, there's M to the 3/2 and there's an M upstairs, so the one cancels.
We're going to be left with N cubed over square root of M times B. OK.
There was a lower order term because I dropped this plus 1, but let's not worry about that right now.
Here we had N cubed divided by B, that was the standard algorithm.
Now we've got M cubed divided by B divided by square root of M. That's big.
I mean, this is basically, you're dividing by-- well, square root of your cache size.
Wow.
So who knows how big that is, but say, between memory and disk, we're talking gigabytes.
So this is like billions.
Square root of a billion is still pretty big, like 10 to 100,000, so this is a huge amount faster than the standard algorithm.
You can do way better than scans.
Basically because we're reusing the same rows and columns over and over.
Now, this is standard matrix multiplication.
You might ask, what about Strassen's algorithm?
Well, same thing works.
You can do the same analysis Strassen, of course.
You get a similar improvement over Strassen.
You can do this for non-square matrices and all those good things.
And one minute left.
And it's going to be enough, I think, to cover LRU block replacement.
So here's what I want to say about LRU block replacement.
So in the beginning, we said the model is LRU, or it could have been FIFO.
Remember that?
And this algorithm will work just fine from an LRU perspective or a FIFO perspective if you think about it, but how do we know that LRU is as good as anything?
I claim, if you look at some sequence of block axises-- so suppose you know what B is-- and you count, for a cache of size M, how many memory transfers does LRU do, it's going to be within a factor of 2 of the optimal.
But not the optimal for a cache of size M, the optimal for a cache of size M over 2.
This is a bit of a weird statement.
I have a factor of 2 here and a factor of 2 here.
This is a cool idea called resource augmentation, fancy word for a simple idea.
This we're used to.
This is approximation algorithms.
OK, but this is an approximation in cost.
Here we're approximating the resources available to the algorithm.
We're changing the machine model, dividing M by 2, and we get a nice result.
Why is this OK?
Because, if you look at a bound like this, if you change M by a factor of 2, it will not change the bound by more than a factor of square root of 2.
So as long as you have at most, say, a linear or polynomial dependence on M, changing M by a constant factor will not change the overall running time of the cache for the previous algorithm.
This is why we can assume it's LRU.
The same is true for FIFO, it's probably true in expectation for random sequences.
And I will leave it at that.
If you want to see the-- do you want to see the proof of this theorem?
Tomorrow?
Or, Thursday?
Yes.
OK, we'll cover it on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, welcome to the final lecture of 6046.
Today we continue our theme of cache oblivious algorithms.
We're going to look at two of the most basic problems in computer science-- searching and sorting, a little bit of each.
And then I'll tell you a little bit about what class you might take after this one.
So brief recap of the model, we introduced two models of computation although one was just a variation of the other.
The base model is an external memory model.
This is a two-level memory hierarchy.
CPU and cache, we view as one.
So there's instant communication between them, which means what you're computing on can involve this cache of size m-- total size of the cache is m-- words.
The cache is divided into these blocks of size b each.
So they're m over b blocks.
And your problem doesn't fit here, presumably, or the problem's not very interesting.
So your problem size n is going to require storing your information on disk.
So the input is really provided over here.
Disk is basically infinite in size.
It's also partitioned into blocks.
And you can't access individual items here.
You can only access entire blocks.
So the model is you say, I want to read this block and put it here.
I want to write this block out and put it here.
That's what you're allowed to do in the external memory model.
And what we count is how many block-memory transfers we do.
We call those memory transfers.
So you want to minimize that.
And usually you don't worry too much about what happens in here, although you could also minimize regular running time as we usually do.
The cache oblivious variation is that the algorithm is not allowed to know the cache parameters.
It's not allowed to know the block size.
Sorry, it's also block size b in the disk.
So they match.
And you're not allowed to know the cache size, m. Because of that, all the block reads and writes are done automatically.
So the model is, whenever you access an item, you view the disk as just written row by row-- sequentially block by block.
So in linear eyes, it looks like this, partitioned into blocks.
And so whenever you touch an item, the system automatically loads that block.
If it's not already in cache, it loads it in.
If it's already in cache, it's free.
When you load a block in, you probably already have something there.
So if the cache is already full, you have to decide which one to evict.
And we had a couple of strategies.
But the one I defined was the least-recently-used block.
So whichever one in the cache that's least recently been used by the CPU, that's the one that gets written out, back to disk where it originally came from.
And that's it.
That's the model.
OK, this is a pretty good model of how real caches work.
Although this last part is not how all real caches work.
It's close.
And at the very end, I mentioned this theorem that Why LRU is good.
And if you take the number of block evictions-- the number of block reads, equivalently-- then LRU has to do on a cache of size m. Then that's going to be, at most, twice whatever the best possible thing you could do is given a cache of size m over 2.
So we're restricting OPT.
We're kind of tying OPT's hands behind his back a little bit by decreasing m by a factor of 2.
But then we get a factor of 2 approximation, basically.
So this was the resource augmentation.
And this is regular approximation algorithms.
In general, this is a world called online algorithms, which is a whole field.
I'm just going to mention it briefly here.
The distinction here is LRU, or whatever we implement in a real system, has to make a decision based only on the past of what's happened.
The system, we're assuming, doesn't know the future.
So in a compiler, maybe you could try to predict the future and do something.
But on a CPU, it doesn't know what instruction's going to come next, 10 steps in the future.
So you just have to make a decision now, sort of your best guess.
And least recently used is a good best guess.
OPT, on the other hand, we're giving a lot of power.
This is what we call an offline algorithm.
It's like the Q in Star Trek: Next Generation or some other mythical being.
It lives outside of the timeline.
It can see all of time and say, I think I'll evict this block.
This is like the waste of Q's resources.
But I'll evict this block because I know it's going to be used for this in the future.
LRU is evicting the thing that was used farthest in the past.
There's a difference there.
And it could be a big difference.
But it turns out they're related in this way.
So this is what we call an online algorithm, meaning you have to make decisions as you go.
The offline algorithm gets to see the future and optimize accordingly.
Both are computable, but this one's only computable if you know the future, which we don't.
What I haven't done is prove this theorem.
It's actually really easy proof.
So let's do it.
I want to take the timeline and divide it into phases.
Phases sounds cool.
So this is going to be an analysis.
And in an analysis, we're allowed to know the future because we're trying to imagine what OPT could do relative to LRU.
So we're fixing the algorithm.
It's obviously not using the future.
When we analyze it, we're assuming we do know the future.
We know the entire timeline.
So all the algorithms we covered last time, all the ones we covered today you can think of as just making a sequence of accesses.
They're making sequences of accesses to elements.
But if we assume we know what b is, that's just a sequence of accesses to blocks.
OK so you can just think of the timeline as a sequence of block IDs.
And if you access a block that's currently stored in cache, it's free.
Otherwise, you pay 1.
All right, so I'm just going to look at the timeline of all these accesses and say, well, take a prefix of the accesses until I get to m over b distinct blocks, block IDs.
Keep going until, if I went one more, I'd have m over b plus 1 distinct blocks.
So it's a maximal prefix of m over b distinct blocks.
Cut there.
And then repeat.
So start over.
Start counting at zero.
Extend until I have m over b distinct block accesses.
And if I went one more, I'd have m over b plus 1 and so on.
So the timeline gets divided.
Who knows?
It could be totally irregular.
If you access the same blocks many times, you could get along for a very long time and only access m over b distinct blocks.
Who knows?
The algorithm definitely doesn't know because it doesn't know m or b.
But from an analysis perspective, we can just count these things.
So each of these has exactly m over b distinct accesses, distinct block IDs.
So I have two claims about such a phase.
First claim is that LRU with a cache of size m on one phase is, at most, what?
It's easy
M over b
M over b.
The claim is, at most, m over b basically because the LRU is not brain dead.
Well, you're accessing these blocks.
And they've all been accessed more recently.
I mean, let's look at this phase.
All the blocks that you touch here have been accessed more recently than whatever came before.
That's the definition of this timeline.
This is an order by time.
So anything you load in here, you will keep preferentially over the things that are not in the phase because everything in the phase has been accessed more recently.
So maybe, eventually, you load all m over b blocks that are in the phase.
Everything else you touch, by definition of a phase, are the same blocks.
So they will remain in cache.
And that's all it will cost, m over b memory transfers per phase.
So this is basically ignoring any carry over from phase to phase.
This is a conservative upper bounds.
But it's an upper bounds.
And then the other question is, what could OPT do?
So OPT-- remember, we're tying its hands behind its back.
It only has a cache of size m over 2.
And then we're evaluating on a phase.
I want to claim that OPT is, well, at least half of that if I want to get a factor of 2.
So I claim it's at least 1/2 m over b.
Why?
Now we have to think about carry over.
So OPT did something in this phase.
And then we're wondering what happens in the very next phase.
So some of these blocks may be shared with these blocks.
We don't know.
I mean, there's some set of blocks.
We know that this very first block was not in the set, otherwise the phase would have been longer.
But maybe some later block happens to repeat some block that's over there.
We don't really know.
There could be some carry over.
So how lucky could OPT be?
At this moment in time, at the beginning of the phase we're looking at, it could be the entire cache has things that we want, has blocks that appear in this phase.
That's the maximum carry over, the entire cache.
So sort of the best case for OPT is that the entire cache is useful, meaning it contains blocks in the phase that we're interested in-- the phase we're looking at-- at the start of the phase.
That's the best case.
But because we gave up only m over 2, that means, at most, one half m over b blocks.
This was cache size.
This is the number of blocks in the cache.
At most, this many blocks will be free, won't cost anything for OPT.
But by definition, the phase has m over b distinct blocks.
So half of them will be free.
The other half, OPT is going to have to load in.
So it's a kind of trivial analysis.
It's amazing this proof is so simple.
It's all about setting things up right.
If you define phases that are good for LRU, then they're also bad for OPT when it has cache at half the size.
And so OPT has to pay at least half what LRU is definitely paying.
Here we can forget about carry over.
Here we're bounding the carry over just by making the cache smaller.
That's it.
So this is a most twice that.
And so we get the theorem.
I mean, changing the cache size could dramatically change the number of cache reads that you have to do or disk reads it you have to do into cache.
But in all of the algorithms we will cover, we're giving some bound in terms of m. That bound will always be, at most, some polynomial dependence in m. Usually it's like a 1 over m, 1 over square root of m, 1 over log m, something like that.
All of those bounds will only be affected by a constant factor when you change m by a constant factor.
So this is good enough for cache oblivious algorithms.
All right, so that's sort of review of why this model is reasonable.
LRU is good.
So now we're going to talk about two basic problems-- searching for stuff in array, sorting an array in both of these models.
We won't be able to do everything cache obliviously today.
But they're are all possible.
It just takes more time than we have.
We'll give you more of a flavor of how these things work.
Again, the theme is going to be divide and conquer, my glass class.
So let's say we have n elements.
Let's say, for simplicity, we're in the comparison models.
So all we can really do with those elements is compare them-- less than, greater than, equal.
And let's say we want to do search in the comparison model, which I'll think of as a predecessor search.
So given a new element x, I want to find, what is the previous element?
What's the largest element smaller than x in my set?
I'm thinking of these n elements as static, let's say.
You can generalize everything I say to have insertions and deletions.
But let's not worry about that for now.
I just want to store them somehow in order to enable search.
So any suggestions in external memory model or cache oblivious model?
How would you do this?
[STUDENT COUGHS] This may sound easy, but it's not.
But that's OK. You know, I like easy answers, simple answers.
There's two simple answers.
One is correct, one is wrong.
But I like both, so I want to analyze both.
Yeah?
Store them sorted in order?
Store them sorted in order, good.
That's how we'd normally solve this problem.
So let's see how it does.
I thought I had solution, too, here.
But that's OK. Binary search in a sorted array, sort the elements in order.
And then to do a query, binary search on it.
So you remember binary search.
You've got an array.
You start in the middle.
Then let's say the element looking for is way over here.
So then we'll go over this way and go there, this way, there, this way, log n time.
I mean, binary search is, in a certain sense, a divide and conquer algorithm.
You only recurse on one side, but it's divide and conquer.
So divide and conquer is good.
Surely binary search is good.
If only it were that simple.
So sort of orthogonal to this picture-- maybe I'll just draw it on one side-- there's a division into blocks.
And in a cache oblivious setting, we don't know where that falls.
But the point is, for the most part, every one of these accesses we do as we go farther and farther to the right-- almost all of them will be in a different block.
The middle one is very far away from the 3/4 mark.
It is very far away from the 7/8 mark, and so on, up until the very end.
Let's say we're searching for the max.
So this will hold for all of them.
At the end, once we're within a problem of size order b, then there's only a constant number of blocks that we're touching.
And so from then on, everything will be free, basically.
So if you think about it carefully, the obvious upper bound-- this is our usual recurrence for binary search-- would be constant.
And what we hope to gain here is, basically, a better base case.
And I claim that all you get in terms of base case here is t of b equals order 1.
And, if you think about it, this just solves to log n minus log b, which is the same thing as log n over b, which is a small improvement over just regular log n but not a big improvement.
I claim we can do better.
You've actually seen how to do better.
But maybe we didn't tell you.
So it's a data structure you've seen already-- b tree, yeah.
So because we weren't thinking about this memory hierarchy business when we said b tree, we meant like 2-4 trees or 5-10 trees or some constant bound on the degree of each node.
But if you make the degree of the node b-- or some theta b, b approximate-- so you allow a big branching factor.
It's got to be somewhere, let's say, between b over 2 and b.
Then we can store all of these pointers and all of these keys in a constant number of blocks.
And so if we're doing just search, as we navigate down the b tree we'll spend order 1 block reads to load in this node and then figure out which way to go.
And then let's say it's this way.
And then we'll spend order 1 memory transfers to read this node then figure out which way to go.
So the cost is going to be proportional to the height of the tree, which is just log base b of n up to the constant factors because we're between b over 2 and b.
But that will affect this by a factor of 2.
So we can do search in a b tree in the log base b of n memory transfers.
OK, remember, log base b of n is log n divided by log b.
So this is a lot better.
Here we had log n minus log b.
Now we have log n divided by log b.
And this turns out to be optimal.
In the comparison model, this is the best you can hope to do, so good news.
The bad news is we kind of critically needed to know what b was.
B trees really only make sense if you what b is.
You need to know the branching factor.
So this is not a cache oblivious data structure.
But it has other nice things.
We can actually do inserts and deletes, as well.
So I said static, but if you want dynamic insert and deleting elements, you can also do those in log base b of n memory transfers using exactly the algorithms we've seen with splits and merges.
So all that's good.
But I want to do it cache obviously-- just the search for now.
And this is not obvious.
But it's our good friend van Emde Boas.
So despite the name, this is not a data structure that van Emde Boas.
But it's inspired by the data structure that we covered.
And it's actually a solution by Harold [INAUDIBLE], who did the m-edge thesis on this work.
In the conclusion, it's like, oh, by the way, here's how you do search.
It seems like the best page of that thesis.
And then I think we called it van Emde Boas because we thought it was reminiscent.
So here's the idea.
Take all the items you want to store.
And you're really tempted to store them in sorted order, but I'm not going to do that.
I'm going to use some other divide and conquer order.
First thing I'm going to do is take those elements, put them in a perfectly balanced binary search tree.
So this is a BSTT-- not a b tree, just a binary tree because I don't know what b is.
So then maybe the median's up here.
And then there's two children and so on.
OK, the mean's over here.
The max is over here, a regular BST.
Now we know how to search in a tree.
You just walk down.
The big question is, in what order should I store these nodes?
If I just store them in a random order, this is going to be super bad-- log n memory transfers.
But I claim, if I do a clever order, I can achieve log base b of n, which is optimal.
So van Emde Boas suggests cutting this tree in the middle.
Why in the middle?
This was n nodes over here.
And we're breaking it up into a square root of n nodes at the top because the height of this overall tree is log n. If we split it in half, the height of the tree is half log n. 2 to the half log n is root n. I'm losing some constant factors, but let's just call it root n. Then we've got, at the bottom, everything looks the same.
We're going to have a whole bunch of trees of size square root of n, hopefully.
OK, that's what happens when I cut in the middle level.
Then I recurse.
And what am I recursing?
What am I doing?
This is a layout.
Last time, we did a very similar thing with matrices.
We had an n by n matrix.
We divided it into four n over 2 by n over 2 matrices.
We recursively laid out the 1/4, wrote those out in order so it was consecutive.
Then we laid out the next quarter, next quarter, next quarter.
The order of the quarters didn't matter.
What mattered is that each quarter of the matrix was stored as a consecutive unit so when recursed, good things happened.
Same thing here, except now I have roughly square root of n plus 1.
Chunks, little triangles-- I'm going to recursively lay them out.
And then I'm going to concatenate those layouts.
So this one, I'm going to recursively figure out what order to store those nodes and then put those all as consecutive in the array.
And then this one goes here.
This one goes here.
Actually, the order doesn't matter.
But you might as well preserve the order.
So do the top one, then the bottom ones in order.
And so, recursively, each of these ones is going to get cut in the middle.
Recursively lay out the top, then the next one.
Let's do an example.
Let's do an actual tree.
This is actually my favorite diagram to draw or something.
My most frequently drawn diagram, complete binary tree on eight children, eight leaves.
So this is 15 nodes.
It happens to have a height that's a power of 2, so this algorithm works especially well.
So I'm going to split it in half, then recursively lay out the top.
To lay out the top, I'm going to split it in half.
Then I'm going to recursively lay out the top.
Well, single node-- it's pretty clear what order to put it in with respect to itself.
So that goes first, then this, then this.
Then I finish the first recursion.
Next, I'm going to recursively lay out this thing by cutting it in half, laying out the top, then the bottom parts.
OK, then I'm going to recursively layout this-- 7, 8, 9, 10, 11, 12, 13, 14, 15.
It gets even more exciting the next level up.
But it would take a long time to draw this.
But just imagine this repeated.
So that would be just the top half of some tree.
Cut here, and then you do the same thing here and here and here.
This is very different from in-order traversal or any other order that we've seen.
This is the van Emde Boas order.
And this numbering is supposed to be the order that I store the nodes.
So when I write this into memory, it's going to look like this-- just the nodes in order.
And when I'm drawing a circle-- wow, this is going to get tedious.
And then there's pointers here.
Every time I draw a circle, there's a left pointer and a right pointer.
So 1's going to point to 2 and 3.
2 is going to point to 4 and 7.
So just take the regular binary search tree, but store it in this really weird order.
I claim this will work really well, log base b of n search.
Let's analyze it.
Good first time, this is a cache oblivious layout.
I didn't use b at all.
There's no b here.
Start with a binary tree.
And I just do this recursion.
It gives me a linear order to put the nodes in.
I'm just going to store them in that order.
It's linear size, all that.
Now in the analysis, again, I'm allowed to know b.
So let's say b is b.
And let's consider the level of recursion.
Let's say the first level of recursion, where the triangles have less than or equal to b nodes.
So I'm thinking of this picture.
I cut in the middle.
Then I recursively cut in the middle of all the pieces.
Then I recursively cut in the middle.
I started out with a height of log n and size n. I keep cutting, basically square rooting the size.
At some point, when I cut, I get triangles that are size, at most, square root of b.
So the tree now will look-- actually, let me draw a bigger picture.
Let's start down here.
So I've got triangle less than or equal to b, triangle less than or equal to b.
This is some attempt to draw a general tree.
And first we cut in the middle level.
Then we cut in the middle levels.
And let's say, at that moment, all of the leftover trees have, at most, b nodes in them.
It's going to happen at some point.
It's going to happen after roughly log n minus log b levels of recursion.
The heights here will be roughly log b.
We keep cutting in half, still with height log b.
Then we know the size of it's b. OK, so this is a picture that exists in some sense.
What we know is that each of these triangles is stored consecutively.
By this recursive layout, we guarantee that, at any level of recursion, each chunk is stored consecutively.
So, in particular, this level-- level b-- is nice.
So what that tells us is that each triangle with, at most, b elements is consecutive, which means it occupies at most two blocks.
If we're lucky, it's one.
But if we're unlucky in terms of-- here's memory.
Here's how it's split into blocks.
Maybe it's consecutive, but it crosses a block boundary.
But the distance between these two lines is b and b.
And the length of the blue thing is b.
So you can only cross one line.
So you fit in two blocks.
Each of these triangles fits in two blocks.
Now, let's think about search algorithm.
We're going to do regular binary search in a binary search tree.
We start at the root.
We compare to x.
We go left to right.
Then we go left to right, left to right.
Eventually we find the predecessor or the successor or, ideally, the element we're actually searching for.
And so what we're doing is following some root-to-node path in the tree.
Maybe we stop early.
In the worst case, we go down to a leaf.
But it's a vertical path.
You only go down.
Over here, same thing.
Let's say, because these are the ones I drew, you go here somewhere.
But in general, you're following some root-to-node path.
And you're visiting some sequence of triangles.
Each triangle fits in, basically, one block.
Let's assume, as usual, m over b is at least two.
So you can store at least two blocks, which means, once you start touching a triangle, all further touches are free.
The first one, you have to pay the load in, maybe these two blocks.
Every subsequent touch as you go down this path is free.
Then you go to a new triangle.
That could be somewhere completely different.
We don't really now, but it's some other two blocks.
And as long as you stay within the triangle, it's free.
So the cost is going to be, at most, twice the number of triangles that you visit.
MTN is going to be, at most, twice the number of triangles visited by a root-to-node path, a downward path in the binary search tree.
OK, now to figure that out we need not only an upper bound on how big the triangles are but also a lower bound.
I said the height of the tree is about log b.
It's close.
Maybe you have triangles of size b plus 1, which is a little bit too big.
So let's think about that case.
You have b plus 1 nodes.
And then you end up cutting in the middle level.
So before, you had a height of almost log b-- slightly more than log b.
Then, when you cut it in half, the new heights will be half log b.
And then you'll have only square root of b items in the triangle.
So that may seem problematic.
These things are, at most, b.
They're also at least square root b.
The height of a triangle at this level is somewhere between half log b and log b.
Basically, we're binary searching on height.
We're stopping when we divide it by 2.
And we get something less than log B in height.
Luckily, we only care about heights.
We don't care that there's only root b items here.
That may seem inefficient, but because everything's in a log here-- because we only care about log b in the running time, and we're basically approximating log b within a factor of 2-- everything's going to work up to constant factors.
In other words, if you look at this path, we know the length of the path is log n. We know the height of each of these triangles is at least half log b.
That means the number of triangles you visit is log n divided by half log b.
And the length of the path is log n. So the number of triangles on the path is, at most, log n divided by how much progress we make for each triangle, which is half log b-- also known as 2 times log base b of n. And then we get the number of memory transfers is, at most, twice that.
So the number of memory transfers is going to be, at most, 4 times log base b of n. And that's order log base b of n, which is optimal.
Now we don't need to know b.
How's that for a cheat?
So we get optimal running time, except for the constant factor.
Admittedly, this is not perfect.
B trees get basically 1 times log base b of n. This cache oblivious binary search gives you 4 times log base b of n. But this was a rough analysis.
You can actually get that down to like 1.4 times log base b of n. And that's tight.
So you can't do quite as well with cache oblivious as external memory but close.
And that's sort of the story here.
If you ignore constant factors, all is good.
In practice, where you potentially win is that, if you designed a b tree for specific b, you're going to do really great for that level of the memory hierarchy.
But in reality, there's many levels to the memory hierarchy.
They all matter.
Cache oblivious is going to win a lot because it's optimal at all levels simultaneously.
It's really hard to build a b tree that's optimal for many values of b simultaneously.
OK so that is search.
Any questions before we go on to source?
[STUDENTS COUGHING] One obvious question is, what about dynamic?
Again, I said static.
Obviously the elements aren't changing here.
Just doing search in log base b of n, it turns out you can do insert, delete, and search in log base b of n memory transfers per operation.
This was my first result in cache oblivious land.
It's when I met Charles Leiserson, actually.
But I'm not going to cover it.
If you want to know how, you should take 6851, Advanced Data Structures, which talks about all sorts of things like this but dynamic.
It turns out there's a lot more to say about this universe.
And I want to go in to sorting instead of talking about how to make that dynamic because, oh, OK, search log base b of n, that was optimal.
I said, oh, you can also do insert and delete in log base b of n. It turns out that's not optimal.
It's as good as b trees.
But you can do better.
B trees are not good at updates.
And if you've ever worked with a database, you may know this.
If you have a lot of updates, b trees are really slow.
They're good for searches, not good for updates.
You can do a lot better.
And that will be exhibited by sorting.
So sorting-- I think you know the problem.
You're given n elements in an array in some arbitrary order.
You want to put them into sorted order.
Or, equivalently, you want to put them into a van Emde Boas order.
Once their sorted, it's not too hard to transfer into this order.
So you can do search fast or whatever.
Sorting is a very basic thing we like to do.
And the obvious way to sort when you have, basically, a-- let's pretend we have this b tree data structure, cache oblivious even.
Or we just use regular b trees.
Let's use regular b trees.
Forget about cache oblivious.
External memory, we know how to do b trees.
We know how to insert into a b tree.
So the obvious way to sort is to do n inserts into, if you want, a cache oblivious b tree or just a regular b tree.
How long does that take?
N times log base b of n. It sounds OK.
But it's not optimal.
It's actually really slow compared to what you can do.
You can do, roughly, a factor of b faster.
But it's the best we know how to do so far.
So the goal is to do better.
And, basically, what's going on is we can do inserts.
In this universe, we can do inserts and deletes faster than we can do searches, which is a little weird.
It will become clearer as we go through.
So what's another natural way to sort?
What means to sorting algorithm that we've covered are optimal in the comparison model?
Merge sort
Merge sort, that's a good one.
We could do quick sort, too, I guess.
I'll stick to merge sort.
Merge sort's nice because A, it's divide and conquer.
And we like divide and conquer.
It seems to work, if we do it right.
And it's also cache oblivious.
There's no b in merge sort.
We didn't even know what b was.
So great, merge sort is divide and conquer and cache oblivious.
So how much does it cost?
Well, let's think about merge sort.
You take an array.
You divide it in half.
That takes zero time.
That's just a conceptual thing.
You recursively sort this part.
You recursively sort this part.
That looks good because those items are consecutive.
So that recursion is going to be an honest to goodness recursion on an array.
So we can write a recurrence.
And then we have to merge the two parts.
So in merge, we take the first element of each guy.
We compare them, output one of them, advance that one, compare, output one of them, advance that guy.
That's three parallel scans.
We're scanning in this array.
We're scanning in this array.
We're always advancing forward, which means as long as we store the first block of this guy and the first block of this guy who knows how it's aligned-- But we'll read these items one by one until we finish that block.
Then we'll just read the next block, read those one by one.
And similarly for the output array, we first start filling a block.
Once it's filled, we can kick that one out and read the next one.
As long as m over b is at least 3, we can afford this three-parallel scan.
It's not really parallel.
It's more like inter-leaf scans.
But we're basically scanning in here while we're also scanning in here and scanning in the output array.
And we can merge two sorted arrays into a new sorted array in scan time, n over be plus 1.
So that means the number of memory transfers is 2 times the number of memory transfers for half the size, like regular merge sort, plus n over b plus 1.
That's our recurrence.
Now we just need to solve it.
Well, before we solve it, in this case we always have to be careful with the base case.
Base case is MT of m. This is the best base case we could use.
Let's use it.
When I reach an array of size m, I read the whole thing.
And then that's all I can pay.
So I won't incur any more cost as long as I stay within that region of size m. Maybe I should put some constant times m because this is not in place algorithm, so maybe 1/3 m something.
As long as I'm not too close to the cache size, I will only pay m over b memory transfers.
So far so good.
Now we just solve the recurrence.
This is a nice recurrence, very similar to the old merge-sort recurrence.
We just have a different thing in the additive term.
And we have a different base case.
The way I like to solve nice recurrences is with recursion trees.
This is actually a trick I learned by teaching this class.
Before this, cache oblivious was really painful to me because I could never solve the currencies.
Then I thought the class and was like, oh, this is easy.
I hope the same transformation happens to you.
You'll see how easy it is once we do this example.
OK, this is merge sort.
Remember recursion tree, in every node you put the additive cost so that, if you added up the cost of all of these nodes, you would get the total value of this expands to because we're basically making two children of size n over 2.
And then we're putting, at the root, this cost, which means, if you add up all of these nodes, you're getting all of these costs.
And that's the total cost.
So it's n over b at the top.
Then it's going to be n over 2 divided by b and so on.
I'm omitting the plus 1 just for cleanliness.
You'd actually have to count.
And this keeps going until we hit the base case.
This is where things are a little different from regular merge sort, other than the divided by b.
We stop when we reach something at size m. So at the leaf level, we have something of size m, which means we basically have m over b in each leaf.
And then we should think about how many leaves there are.
This is just n over m leaves, I guess.
There's lots of ways to see that.
One way to think about it is we're conserving mass.
We started with n items.
Split it in half, split it in half.
So the number of items is remaining fixed.
Then at the bottom we have m items.
And so the number of leaves has to be exactly n over m because the total should be n. You can also think of it as 2 to the power log of that.
We have, usually, log n levels.
But we're cutting off a log m at the bottom.
So it's log n minus log m as the height.
I'll actually need that.
The height of this tree is log n minus log m, also known as log n/m.
OK, so we've drawn this tree.
Now, what we usually do is add up level by level.
That usually gives a very clean answer.
So we add up the top level.
That's n over b.
We add up the second level.
That's n over b, by conservation of mass again and because this was a linear function.
So each level, in fact, is going to be exactly n over b cost.
We should be a little careful about the bottom because the base case-- I mean, it happens that the base case matches this.
But it's always good practice to think about the leaf level separately.
But the leaf level is just m over b times n over m The m's cancel, so m over b times n over m. This is n over b.
So every level is n over b.
The number of levels is log of n over m. Cool.
So the number of memory transfers is just the product of those two things.
It's n over b times that log, log n over m. Now let's compare.
That's sorting.
Over here, we had a running time of n times log base b of n. So this is n log n divided by log b. Log base b is the same as dividing by log b.
So n log n divided by log-- we had regular sorting time.
And then we divided by log b.
Over here, we have basically regular sorting time.
But now we're dividing by b.
That's a huge improvement-- a b divided by log b improvement.
I mean, think of the b being like a million.
So before we were dividing by 20, which is OK.
But now we're dividing by a million.
That's better.
So this way of sorting is so much better than this way of sorting.
It's still not optimal, but we're getting better.
We can actually get sort of the best of both worlds-- divide by b and divide by log b, I claim.
But we need a new algorithm.
Any suggestions for another algorithm?
Divide into block size b
I want to divide into block size b.
So, you mean a merge sort?
Yes
So merge sort, I take my array.
I divide it into blocks the size b. I could sort each one in one memory transfer.
And then I need to merge them.
So then I've got n divided by b sorted arrays.
I don't know how to merge them.
It's going to be hard, but very close.
So the answer is indeed merge sort.
What we covered before is binary merge sort.
You split into two groups.
What I want to do now is split into some other number of groups.
So that was n over b groups.
That's too many because merging n over b arrays is hard.
Merging two arrays was easy.
Assuming m over b was at least 3, I could merge these guys just by parallel scans.
So you have the right bound?
B way
B way, maybe
Square root of b
Square root of b?
That's what I like to call root beer.
[LAUGHTER] Nope.
I do call it that.
Yeah?
M over b?
M over b, that's what I'm looking for!
Why m over b?
I was just thinking of the bottom layer of the [INAUDIBLE] binary merge sort
Because m over b is up here?
Nice.
[LAUGHTER] Not the right reason, but you get a Frisbee anyway.
All right, let's see if I can do this.
Would you like another one?
Add to your collection.
All right, so m over b is the right answer-- wrong reason, but that's OK.
It all comes down to this merge step.
So m over b way means I take my problem of size n. Let's draw it out.
I divide into chunks.
I want the number of chunks that I divide into to be m over b, meaning each of these has size n over m over b.
That's weird.
This is natural because this is how many blocks I can have in cache.
I care about that because, if I want to do a multi-way merge, you can mimic the same binary merge.
You look at the first item of each of the sorted arrays.
You compare them.
In this model, comparisons are free.
Let's not even worry about it.
In reality, use a priority queue, but all right.
So you find the minimum of these.
Let's say it's this one.
And you output that, and then you advance.
Same algorithm, that will merge however many arrays you have.
The issue is, for this to be efficient like it was here, we need to be able to store the first block of each of these arrays.
How many blocks we have room for?
M over b.
This is maxing out merge sort.
This is exactly the number of blocks that we can store.
And so if we do m over b way merge sort, merge remains cheap.
An m over b way merge costs n over b plus 1, just like before.
It's m over b parallel scans.
M over b is exactly the number of scans we can handle.
OK, technically we have, with this picture, m over b plus 1 scans.
So I need to write m over b minus 1.
But it won't make a difference.
OK, so let's write down the recurrence.
It's pretty similar.
Memory transfer's size m. We have m over b sub problems of size n divided by m over b.
It's Still conservation of mass.
And then we have plus the same thing as before, n over b plus 1.
So it's exactly the same recurrence we had before.
We're splitting into more problems.
But the sums are going to be the same.
It's still going to add up to n over b at each step because conservation of mass.
And we didn't change this.
So level by level looks the same.
The only thing that changes is the number of levels.
Now we're taking n. We're dividing by m over b in each step.
So the height of the tree, the number of levels of the recursion tree now is-- before it was log base 2 of n over n. Now it's going to be log base m over b of n over m. If you're careful, I guess there's a plus 1 for the leaf level.
I actually want to mention this plus 1.
Unlike the other plus 1's, I've got to mention this one because this is not how I usually think of the number of levels.
I'll show you why.
If you just change it by one, you get a slightly cleaner formula.
This has got m's all over the place.
So I just want to rewrite n over m here.
Then we'll see how good this is.
This is just pedantics.
Log base m over b of n-- I really want n over b.
To make this n over b, I need to multiply by b, divide by m. OK, these are the same thing.
M over m, b's cancel.
But I have a log of a product.
I can separate that out.
Let's go over here.
This is log base m over b of n over b-- this is what I like-- and then, basically, minus log base m over b of m over b
It's b over m
I put a minus, so it's m over b.
If I put a plus, it would be b over m. But, in fact, m is bigger than b.
So I want it this way.
And now it's obvious this is 1.
So these cancel.
So that's why I wanted the 1, just to get rid of that.
It doesn't really matter, just a plus 1.
But it's a cooler way to see that, in some sense, this is the right answer of the height of the tree.
Now, we're paying n over b at each recursive level.
So the total cost is what's called the sorting bound.
This is optimal, n over b times log base m over b of n over b. Oh my gosh, what a mouthful.
But every person who does external memory algorithms and cache oblivious algorithms knows this.
It is the truth, it turns out.
There's a matching lower bound.
It's a weird bound.
But let's compare it to what we know.
So we started out with n log n divided by log b.
Then we got n log n divided by b.
Let's ignore-- I mean, this has almost no effect, the part in here.
Now we have n log n divided by b and divided by log m over b.
It's not quite dividing by log b.
But it turns out it's almost always the same.
In some sense, this could be better.
If you're cache is big, now you're dividing by log m, roughly.
Before, you were only dividing by log b.
And it turns out this is the best you can do.
So this is going to be a little bit better than merge sort.
If your cache is 16 gigabytes, like your RAM caching your disk, then log m is pretty big.
It's going to be 32 or something, 34 I guess log m. OK, I have to divide by b.
So it's not that good.
But still, I'm getting an improvement over regular binary merge sort.
And you would see that improvement.
These are big factors.
The big thing, of course, is dividing it by b.
But dividing by log of m over b is also nice and the best you can do.
OK, obviously I needed to know what m and b were here.
So the natural question next is cache oblivious sorting.
And that would take another lecture to cover.
So I'm not going to do it here.
But it can be done.
Cache obliviously, you can achieve the same thing.
And I'll give you the intuition.
There's one catch.
Let me mention the catch.
So cache oblivious sorting-- to do optimal cache oblivious sorting like that bound, it turns out you need an assumption called the tall-cache assumption.
Simple form of the tall-cache assumption is that m is at least b squared.
What that means is m over b is at least b.
In other words, the cache is taller than it is wide, the way I've been drawing it.
That's why it's called the tall-cache assumption.
And if you look at real caches, this is usually the case.
I don't know of a great reason why it should be the case.
But it usually is, so all is well.
You can do cache oblivious sorting.
It turns out, if you don't have this assumption, you cannot achieve this bound.
We don't know what bound you can achieve.
But we just know this one is not possible.
You can get a contradiction if you achieve that without tall cache.
So it's a little bit weird.
You have to make one bonus assumption.
You can make a somewhat weaker form of it, which is m is omega b to the 1.000000001.
That will do.
In general, 1 plus epsilon.
Any epsilon will be fine.
We just mean that the number of blocks is at least some b to the epsilon, where epsilon's a constant bigger than zero.
OK, then you can do cache oblivious sorting.
Let me tell you how.
We want to do m over b way merge sort.
But we don't know how to do-- we don't know what m over b is.
So instead, we're going to do something like n to the epsilon way merge sort.
That's a so-so interpretation.
This is back to your idea roughly.
We're dividing into a lot of chunks.
And then we don't know how to merge them anymore because we can't do regular merge with n to the epsilon chunks it could be n to the epsilon's too big.
So how do we do it?
We do a divide and conquer merge.
This is actually called funnel sort because the way you do a divide and conquer merge looks kind of like a funnel.
Actually, it looks a lot like the triangles we were drawing earlier.
It's just a lot messier to analyze.
So I'm not going to do it here.
It would take another 40 minutes or so.
But that's some intuition of how you do cache oblivious merge sort.
That's what I want to say cache oblivious stuff.
Oh, one more thing!
One more cool thing you can do-- I'm a data structures guy.
So sorting is nice.
But what I really like are priority queues because they're more general than sorting.
We started out by saying, hey, look.
If you want to sort and you use a b tree, you get a really bad running time.
That's weird because usually BST sort is good in a regular comparison model.
It's n log n. So b trees are clearly not what we want.
Is there some other thing we want?
And it turns out, yes.
You can build a priority queue, which supports insert and delete min and a bunch of other operations.
Each of those operations costs 1 over b log base m over b of n over b amortized memory transfers-- a bit of a mouthful again.
But if you compare this bound with this bound, it's exactly the same.
But I divided by n, which means if I insert with this cost n times, I pay the sorting bound.
If I delete min with this bound n times, I get the sorting bound.
So if I insert n times and then delete all the items out, I've sorted the items in sorting bound time.
So this is the data structure generalization of that sorting algorithm.
Now, this is even harder to do.
Originally, it was done external memory.
It's called buffer trees.
Then we did it cache obliviously.
It's called cache oblivious priority queues.
We weren't very creative.
But it can be done.
And, again, if you want to learn more, you should take 6851, Advanced Data Structures, which leads us into the next topic, what class you should take next-- classes, that's what I mean to say.
So a lot of bias here.
And well I'm just going to give a lot of classes.
There's a lot of them.
I believe this is in roughly numerical order almost.
It changed a little bit-- so many classes.
Are you OK with numbers, or do you want titles?
The obvious follow-on course to this class is 6854, which is Advanced Algorithms.
It's the first graduate algorithms class.
This is the last undergraduate class, roughly speaking, with the exception of 6047.
But in terms of straight, general algorithms, this would be the natural class.
It's only in the fall-- sadly, not next fall.
But in general, it's a cool class.
It's a very broad overview of algorithms but much more hard core, I guess.
It's an intense class but covers a lot of fields, a lot of areas of algorithms.
Then all the other ones I'm going to list are more specialized.
So 6047 is Computational Biology.
So if you're interested in biology, you want algorithms applied to biology.
That's a cool class.
It's also an undergrad class.
Everything else here-- I mean, you know the story.
You take grad classes all the time, or you will soon if you want to do more algorithms.
So 6850 is computational geometry.
I think it's called Geometric Algorithms.
So we've seen a couple examples, like the convex hull divide-and-conquer algorithm and the range trees.
Those are two examples of geometric algorithms where you have points and lines and stuff-- maybe in two dimensions, maybe in three dimensions, maybe log n dimensions.
If you like that stuff, you should take computational geometry.
This is the devil that lad me into algorithms in the first place.
Cool stuff.
6849 is my class on folding algorithms.
This is a special type of geometric algorithms where we think about paper folding and robotic arm folding and protein folding and things like that.
So that's a bit of a specialized class.
6851, I've mentioned three times now-- Advanced Data Structures.
Then we've got 6852, its neighbor.
This is Nancy's Distributed Algorithms class.
So if you liked the week of distributed algorithms, there's a whole class on it.
She wrote the textbook for it.
Then there's 6853.
This is Algorithmic Game Theory.
If you care about algorithms involving multiple players-- and the players are each selfish.
And they have no reason to tell you the truth.
And still you want to compute something like minimum spanning tree, or pick your favorite thing.
Everyone's lying about the edge weights.
And still you want to figure out how to design a mechanism like an auction so that you actually end up buying a minimum spanning tree.
You can do that.
And if you want to know how, you should take 6853.
What else do we have?
6855 is Network Optimization.
So this is like the natural follow on of network flows.
If you like network flows and things like that, there's a whole universe called network optimization.
It has lots of fancy, basically, graph algorithms where you're minimizing or maximizing something.
OK, this is fortuitous alignment.
6856 is kind of a friend of 6854.
These are both taught by David Carter.
This is Randomized Algorithms.
So this is a more specialized approach.
I don't think you need one to take the other.
But this is the usual starting class.
And this is specifically about how randomization makes algorithms faster or simpler.
Usually they're harder to analyze.
But you get very simple algorithms that run just as well as their deterministic versions.
Sometimes you can do even better than the deterministic versions.
Then there's the security universe.
This is a great numerical coincidence-- probably not a coincidence.
But there's 6857 and 6875.
I have to remember which is which.
6857 is Applied Cryptography.
6875 is Theoretical Cryptography, at least as I read it.
So they have similar topics.
But this is more thinking about how you really achieve security and crypto systems and things like that.
And this one is more algorithm based.
And what kind of theoretical assumptions do you need to prove certain things?
This is more proof based.
And this is more connecting to systems, both great topics.
And I have one more out of order, I guess just because it's a recent addition.
6816 is Multicore Programming.
That has a lot of algorithms, too.
And this is all about parallel computation.
When you have multiple cores on your computer, how can you compute things like these things faster than everything we've done?
It's yet another universe that we haven't even touched on in this class.
But it's cool stuff, and you might consider it.
Then we move on to other theory classes.
That was algorithms.
Some more obvious candidates, if you like pure theory, are 6045 6840.
This is the undergrad version.
This is the grad version.
Although, by now the classes are quite different.
So they cover different things.
Some of you are already taking 6045.
It's right before this lecture.
These are general theory of computation classes, atomita, complexity, things like that.
If you like the brief NP completeness lecture, then you might like this stuff.
There's so many more complexity classes and other cool things you can do.
If you really like it, there's advanced complexity theory.
There's, basically, randomized complexity theory-- how randomness affects just the complexity side, not algorithms.
Then there's quantum complexity theory if you care about quantum computers.
As Scott says, it's proving things you can't do with computers we don't have.
[LAUGHTER] It's complexity.
It's all about lower bounds.
And then there's coding theory, which is another universe.
It's actually closely related to-- it comes out of signals and systems and electrical engineering.
But by now it's closely related to complexity theory.
You can use bounds on codes to prove things about complexity theory.
Anyway, choose your own adventure.
Now I have one last topic, which was not on the outline.
This is a bit of a surprise.
It's a boring surprise.
I want to remind you to fill out student evaluations.
[LAUGHTER] Because, you know, we want to know how we did and how we can continue to improve the class.
But really we want to know who's the better teacher.
But more importantly than who is the better teacher, I think we all have a dying question, which is who is the better Frisbee thrower?
So I want to invite Srini Devadas, our co-lecturer here, to a duel.
[LAUGHTER AND APPLAUSE]
I think you mean, no contest.
[LAUGHTER]
Not so sure.
Maybe-- actually, I'm pretty sure.
[LAUGHTER] I want to take you on, man.
Blue or purple?
Blue
Good choice
Blue's better, remember?
[LAUGHTER]
Purple's better, remember?
[LAUGHTER] All right, so how are we going to do this?
So, you guys get to cheer and bet
Bet?
I don't think we can condone them betting money.
I think maybe they can bet their Frisbees.
Anyone got a Frisbee on them?
We can bet those
Yeah, all right
All right, maybe not
Put your Frisbees on me here
All right
All right, so some rules here-- we actually talked about this.
So the way this is going to work-- I mean, it's going to be algorithmic, obviously.
And we get to choose our algorithm, maybe with a little game theory here.
We're going to toss the coin.
And we're going to decide who goes first.
So won't spin the Frisbee.
Remember what happened with that?
So you get to call heads or tails while it's spinning
Oh, while it's spinning
While it's spinning.
This is our Super Bowl
Heads!
Oh, a trick.
Tails
Tails, all right.
You're going to throw first
OK, that's your choice
And I don't know if you've heard the legend of William Tell.
How many of you have heard the legend of William Tell?
All right.
So that was a 14th century Swiss legend where there was this archer who was renowned for his skill.
And he was forced by this villainous king to shoot an apple off the top of his son's head
Yikes
So we're going to reenact that.
[LAUGHTER]
Did you bring your daughter?
[LAUGHTER AND APPLAUSE]
I was thinking TAs
Our "sons."
But there's a big difference between the 21st century and the 14th century.
What is that?
You get sued.
[INTERPOSING VOICES]
You get sued, yeah
Now there's many more lawsuits in the 21st century.
So we want to avoid lawsuits
Genetically modified apples
And genetically modified apples, also
Electronically modified Apples, yeah that's going to be another difference.
So we decided we'd just throw Frisbees at each other
So I'm going to throw to you and try to hit an apple off of your head
Yeah, well you might want to tell them what we decided about the apple
Well, I brought an easy-to-hit apple, a nice big apple, the cowboy hat
Cowboy hat
That should be a little easier
So I get to wear that had first because you're going to throw first

And this is really simple, guys.
Knock the hat off, I guess from the furthest distance, and win.
In your case, lose but yeah
Now for the PETA people in the audience, I want your assure no humans will be harmed during this performance, only professors.
[LAUGHTER] And maybe egos, pride
Seven.
I think seven is a good number
Seven for--
You get to--
I'm going to grab purple
You get to pick.
You can stand right here.
That's probably what's good for you.
[LAUGHTER] Or you can go all the way up there.
And after you knock this hat off, I'm going to have to match you
All right
So furthest away wins
I think I'll try from about here
Right.
Ah!
I've got to look good here, man
I have to do it without looking
I'm going to stand right here

OK. Ah!
[LAUGHTER] I've got to gear up for this

Look, I know how well you throw
Are you scared?
That's embarrassing
Ah!
I can't deal with this.
I just have no confidence in the way you throw.
So I borrowed this.
[LAUGHTER] Since this is mine, it's going to cost you a throw to wear this
Hey!
No, all right.
Whatever, fair.
All right.
Now I'm feeling much better about this.
I won't claim to have a pretty face.
But I like it just the way it is, just the way it is
All right.
Well, since we have a little more protection, maybe I'll start from farther back
All right.
I think I'll hold this up here like that
OK.
So I just have to hit the hat off, right?
Easy
Yup.
[LAUGHTER]
You can keep that.
Oh right, so I'm going to get closer.
One step closer.
Luckily, my steps are much bigger than your steps.
OK, throw 2.
[LAUGHTER] Throw three.
[LAUGHTER] Throw four
That didn't even hurt.
Come on, man!
Throw a little harder here!
Oh!
[CHEERING AND APPLAUSE]
All right, mark it
All right, I marked my spot
You've got a couple more throws here to do better
More throws?
You can do better
Wait, to go back
You've got a couple more, yeah
You almost hit me.
[LAUGHTER] [GROANING] I'm getting better!
Not much-- can I go back now?
Yeah, sure
I don't know the rules.
[APPLAUSE]
Goodness
No contest, right?
I'm getting a little worried here
One more Frisbee.
Two more Frisbees.
Hey, that was your Frisbee
One more.
All right
I think we've done binary search here.
[APPLAUSE]
You need this.
And you don't really need this, but I'll give it to you
So much confidence.
Well, I have so much confidence in you I brought some extra Frisbees.
[LAUGHTER]
I get to use all of them, huh?
You need it, man
All right.
No, we're going to be fair.
You threw seven.
I'm going to throw seven.
1, 2, 3, 4, 5, 6, 7.
You've got a bit of a big head here
Do this.
All right, so I put the hat on the head.
OK. Where were you standing, by the way?
Way back here, right?
No, nope.
I was right there

Right there
All right, right here.
Can I hold onto the hat?
No!
You can hold on to your helmet!
All right
Wow
Ah!
[LAUGHTER] How many throws--
Maybe I should start from right here.
[GROANING]
Phew.
That was close
Oh, I grazed it!
But it's supposed to fall off
What, my head?
[LAUGHTER]
Getting kind of tight here, guys.
Wow.
[YELLING AND GROANING] Does that count?
No!
It does count
No!
It's a tie.
So far, it's a tie.
So far, it's a tie!
All right, if I knock it off, I win.
[LAUGHTER] [GROANING] There you you
Is that it?
[APPLAUSE]
This was fair and square.
We want the world to know that we did not deflate these Frisbees.
[LAUGHTER] [APPLAUSE] So not only did we do a bad job of throwing Frisbee to you guys, we didn't throw enough Frisbees, as you can see, through the term.
So if you want a Frisbee, pick one up.
And if you're embarrassed about throwing Frisbees with this lettering on it, I've got two words for you-- paint remover.
All right, have a good summer.
And have fun on the final exam.
[APPLAUSE]
Hi, I'm Erik Demaine, and I'm a professor of computer science here at MIT.
And I'm one of three professors in charge of this class this semester-- 6.046 So I taught about half of the lectures in the class.
Srini taught the other half.
And Nancy prepared new problem sets.
And we all worked together on everything in the class pretty much.
We had, I think, 10 TAs and then another 5 graders.
So it's a big teaching infrastructure to deliver algorithms to 300 students.
My entire career at MIT, all I have done is taught different forms of algorithms.
So 6.046 holds a place particularly dear to my heart.
It's a class that's evolved over time, so there's an older version of 6.046 on OCW when algorithms in the undergraduate level was all taught in one semester, which was exciting, but also pretty intense and didn't really go into a lot of depth.
So several years ago, we split that class into two-- 6.006, which is introductory algorithms.
And the new 6.046, which is this class, which is intermediate algorithms beyond what you could implement in an hour-- things that you have to think more about and prove theorems about.
So 6.046 is sort of the more theoretical counterpart and gets into much more deeper algorithms, and it's exciting.
And all of my research is also around algorithms.
So this is me living the dream, teaching the topic that I love.
And it's an exciting class.
This as an undergraduate class, so the material is stuff that I should know really well.
But every time I teach it, I find that I learn it at an even deeper level and understand it better.
Also, because I try to add in new topics that I don't know so well, so I learn them even better.
And that, in turn, influences my research.
So once I understand the basics really well, I can apply them in new and interesting ways to solve problems I didn't know how to solve before.
Algorithms are just so much fun.
Sometimes I start the class with I'm Erik Demaine, and I love the algorithms!
It's just such a-- it's really fun material.
It's very creative.
It's almost like an art form to come up with a beautiful algorithm.
And this class teaches so many different algorithms that others have developed.
It's kind of like the cream of the crop.
There's zillions out there, so we can't cover all of them.
So we take out representative examples and find really cool, really interesting, or really beautiful algorithms.
And it's really fun to cover them all and to share that beauty with the students.
I think I've taught this class 10 times in some version.
So it's been an adventure.
Every time is different.
And, of course, the last time is always the best.
Next time will be even better, hopefully.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, welcome everyone.
This is 6046 presentation.
Just make sure you're in the right place.
My name is Ling Ren.
I'm one of the 10 TAs for this class.
We do have a second TA for this section.
I think he is not here right now, but, basically, [INAUDIBLE] and I will be switching every other week.
I want to remind you, just a heads up, this section is recorded for OCW purpose.
But I think he's only recording us, the TAs.
You're not in a camera.
All right, so, the only purpose we are here is to help you learn this very interesting, and also very useful, class.
So don't hesitate to ask any questions or give us any feedback, like whether I'm going too fast or too slow, whether you want us to cover something that's not in the posted schedule, or just anything we can help.
All right, so let's get started.
The two lectures in this week, in the first week, focus on divide and conquer.
It is a class of algorithm that usually involves recursion in the algorithm description.
And Professor Devadas worked through several algorithms, including weighted interval scheduling and a bunch of others, and he left several open problems.
So we will answer those open questions in this section, and we'll also show you a new algorithm, and analyze a bunch of other algorithms.
So just to remind everyone what weighted interval scheduling is.
In this problem, we are given a bunch of requests, each with a start time and a finish time.
And our goal is to find a subset of them that are compatible, meaning they do not overlap, and that have a largest combined weight.
OK, are we clear?
Everyone clear about that?
So an easier case is when the problem is unweighted, meaning that every task has the same weight.
In that case, we can just solve it using our [?
Grady ?]
algorithm.
But when the problem becomes weighted, we have to use dynamic programming, or recursion, and Srini introduced a simple one, a basic version, in a class.
Can someone remind us how that algorithm works?
Any volunteers?
[INAUDIBLE]
Can you speak louder?
[INAUDIBLE] so we find that it's not conflicting [INAUDIBLE]
So what's your name?
[INAUDIBLE]
Yeah, I think the version you described is for the unweighted case.
In unweighted case, we just schedule the 1 with the earliest finish time, and, then, we remove all the incompatible ones, and we keep going, right?
That solved the unweighted version.
If it's the weighted version, we need to use recursion.
And remember, we break the problem into many sub-problems, and each one can potentially be an optimal solution.
Does anyone remember that?
Care to give it a try?
Sure.
So we break it into subproblems.
We took the best solution from a certain point, and then we calculate that subproblem starting at all the different finish times.
[INAUDIBLE]
Great.
What is your name?
[?
Amin.
?]
[?
Amin?
?]
OK. [?
Amin ?]
said, let's just try every one as our potential first request.
So if we request the j as our first, we get its weight.
And then, we're going to solve a subproblem.
So let me call the original problem.
We did interval scheduling with all the incoming requests.
Now, we choose j, request as our first.
Now, we are left with a subproblem that starts after request to j finishes.
So I'll write that as Rj, Where I define Rj to be the set of requests where their stop time is later than the finish time of the j-th request.
OK, so just to repeat, we choose a request, has the potential to be the first request, and then, we look at all the requests that start after it, and solve a subproblem of that case.
Then, we take a max of all the candidate we have, and that's going to give us the optimal solution.
Any question about this algorithm?
So this algorithm runs in n square time.
Now, we're going to try to optimize that, and come up with a better algorithm.
So in order to improve anything, we first want to identify the inefficiency in this algorithm.
So which part in the algorithm do you think is inefficient, or silly, unnecessary?
Go ahead
So it's inefficient to look through every previous subproblem, when we're trying to find maximum [INAUDIBLE]
I was saying that we don't need to go through every of this case
Yeah, we shouldn't Well, we should be able to efficiently query for the right one
OK, I think you are definitely correct.
So let me just go through what this algorithm does, and it will be more clear.
So what this means is, I'll choose the first request.
I'll request 1 as my first request.
Then, I'm going to consider only the requests that start after 1 finishes, right?
That only leaves request a 5.
Potentially some other's already been drawn there.
That's my first candidate in that max.
My second candidate is actually request 2 as my first request.
Then, I have to remove request 1, because it starts too early, and then, I'm left with all the remaining requests.
I'll solve that subproblem.
That's candidate 2.
Candidate 3, I chose request 3 as my first request, and then I have to remove 1 and 2, because they start too early.
Oh, 4 as well.
It also starts too early, right?
Before 3 finishes.
Everyone following that?
So we're left with the remaining request.
Is it more clear now?
Go ahead
So candidate, if you start with 3, that's actually a subproblem of 2.
[INAUDIBLE]
Great point.
What's your name?
Andrew
Andrew said, we can potentially be solving many repeated subproblems, like this.
We definitely don't want to do that.
And that's actually the core idea, the one crisp idea of dynamic programming.
Andrew, can you tell me what's the definition of dynamic programming?
You remember?
Anyone remember that?
Go ahead
Just memorize subproblems, then look them up
Exactly.
So dynamic programming says, we will break a problem into subproblems, and subproblems into even more subproblems, but whenever we solve one, we should memorize, or just remember its result and store it somewhere.
And if you need it again, we'll just retrieve it, without resolving the problem.
That's definitely a great point.
So we can analyze the complexity of this algorithm later, because I want to touch on this more efficient algorithm first.
And we'll see that, even after Andrew's optimization, its runtime is n square.
So without that observation, if we are solving repeated subproblems, these will be a lot worse than that.
Yeah, go ahead
You can also trim ones that get to the same place, so you don't need to explore two paths
Sorry, say that again?
You don't need to explore two paths that [INAUDIBLE] both exploring 3.
Once they're both looking at 3, then you only need to do the ones more efficient to that point
OK, cool.
Go ahead.
Oh, same thing?
Well, I have another one as well.
So I think to get better than n squared, we need to make the observation that it's always fine to start a subproblem later.
So if you've decided you're taking a certain sequence of intervals from your first interval, and then you want to see how to compute from there, it's always valid to start later than, maybe, you had to.
So that means that if we can efficiently starting at any particular point, query for the maximum of any of the subproblems starting after that point, then we can [INAUDIBLE]
OK, great.
I think we are on the same page.
So when I describe the steps of this algorithm, remember this third candidate.
I choose this as my first, right?
It makes zero sense, because, if I do that, I might as well put in 2, as well.
Doesn't hurt, right?
Everyone get that?
So the idea is that we shouldn't try every possible W request as my first.
Some requests are just better, more suited to be the first request.
And how we're going to do that?
So, apparently, 1 can potentially be the first request.
2 can also be.
But it doesn't make any sense for any requests come after that, because there's and earlier request.
So the efficient algorithm, let's first sort them by their start time.
We're going to consider the request that comes early first.
So I have my entire problem, here.
Now, I'm going to ask a question.
Should I include request 1 in my solution or not?
That's only two cases.
So if I do not select 1 in my solution, what subproblem am I left with?
Any idea?
If I decide I will not include 1 in my solution
[INAUDIBLE]
Yeah, exactly.
There's no conflicts anywhere.
I'll solve a subproblem from 2 to n. If I do decide to put my request 1 in a solution, I get this weight.
So what's the subproblem I'm left with?
Yeah, in this example, it's five.
Correct.
But more generally--
[INAUDIBLE] So every request that starts afterward
Exactly, every request that starts after 1 finishes.
Now, suddenly, we are not breaking the original problem into n subproblems.
We only have two subproblems.
So let me draw a recursion tree, which is a powerful tool in analyzing these sort of things.
So we start with our original problem, from 1 to n, and we have two subproblems, 2 to n and R1 to n. This one, we'll also break them into subproblems.
So what is this one?
OK, I have my sub problem 2 to n, and I need to further reapply the same trick
[INAUDIBLE].
The max of Wi has to be--
Oh, sorry.
These are the two cases.
Either I do not schedule my first, or I schedule it, and this is my first request.
Now, here, I'm left with this problem.
Either I do not schedule it, or I schedule it as my first request.
So what is this one?
Probably be 3 to the n
3 to n, because now I'm asking the same question for 2.
And, here, I'll have R2 to n, so on and so forth.
Now, let me point out the big difference for this version, and for the basic version.
So I'm starting with the request that starts first.
If I do not do that, say, if here, I am asking the question for 5, do I schedule 5 as my first or not?
Then, what happens?
Then, these two branches do not cover all the cases, because I can potentially schedule it, but not as my first, in the optimal solution, if I'm asking a question for a random request.
However, if I start with the first request, first meaning it starts earliest, it will either be not scheduled, or it will be scheduled, and as the first.
Because it cannot be the second request in my solution.
Any questions about this algorithm?
OK, now, this is the algorithm.
Let's analyze its complexity.
So what's the overall complexity.
when we go all the way down, solve the entire original problem?
Any guesses?
What do you think?
What do you think?
Go ahead
It should be log n, because [INAUDIBLE] sort the intervals, and we're going to need to, for every interval, find the next interval that starts after it, and do that for [INAUDIBLE]
Cool.
So the first step is sorting, which is n log n. Now, after I sort everything, the question I need to answer is, how many unique subproblems are there, in this entire recursion tree?
So I'm not going to solve the same problem twice.
How many unique problems exist, here?
n of them, right?
Just this left branch.
All the others will be one of these.
So I can start from the bottom of the tree and work my way up, and when I want to take this step, I'll look up the result of this in one of the subproblems I've already solved.
So, actually, the recursion itself is only O of n
I missed when you said, what is the sorting referring to?
OK, so we need to start with the request that starts first, right?
We need to decide whether we schedule it or not.
And, then, we need to do the same thing for this request 2.
It's the start earliest request in this subset.
We always need to do that for all the subproblems.
So the overall complexity is n log n. But if we only focus on this recursion step, our improvement is actually larger than that, because it went from n square to O of n. So why is the original algorithm n squared?
I think it also has only n unique subproblems, right?
So do you agree that the original algorithm is n squared, or do you think it's also O of n?
Just focus on the recursion step
Didn't the original algorithm solve some subproblems up to n times?
So that's why it's still of n squared, because it solved the same thing?
But assuming I do not do that, assuming, whenever I solve it once, I store the result somewhere, and I directly get it.
Assuming I do that, what's the complexity?
Go ahead
O of n
You think it's O of n?
So anyone think it's n squared?
Because I think Srini said it's n squared.
Go ahead
I think the original algorithm isn't [INAUDIBLE] squared, because we're still doing [INAUDIBLE] for every subproblem [INAUDIBLE]
Exactly right.
So, here, whenever we go up one step, I'm doing constant number of work.
Just comparing two numbers and taking the max.
However, in the original algorithm, which is here, whenever I want to go one step up, there are n branches of the tree.
So my total amount of work is 1 plus 2 plus 3 plus n, right?
Every step becomes harder when I go up.
And this is n squared.
Any questions about that?
OK, this is so much for our weighted interval scheduling.
Now, I'm going to transition into the next topic.
So any question on the side, in general, for the scheduling problem?
Do you have a question?
No?
OK, now, let's turn to a second topic of this section, which is Strassen algorithm.
Strassen algorithm is an efficient algorithm for matrix multiplication, and matrix multiplication is a really useful primitive.
It has applications in almost every area I can think of.
Circuit simulation, climate simulation, and physics, basically everything.
Now, I actually had some experience with matrix multiplication, because my undergrad research was improving matrix algorithms.
And, actually, many matrix algorithms, including inversion, solving equations, they all use multiplication as a primitive.
So it actually comes down to improving matrix multiplication.
And I tried very hard to just optimize this basic matrix multiplication.
We'll take a row, you take a column, and then you get your answer for this spot.
Everyone's familiar with that, right?
I tried very hard, but it's still 100 x slower than the best algorithm out there.
So I finally look it up, and I was completely mind-blown when I know that matrix algorithm complexity is not n cubic.
It's actually smaller than that.
Is this a surprise to you?
Anyone expect that before?
And the technique the more efficient algorithm uses is, exactly, Strassen algorithm.
Now that we're talking about divide and conquer, you can guess it must be a divide and conquer algorithm.
And, so, does anyone have an idea how to divide the original problem?
Anyone want to give it a try?
So are you familiar with tiled matrix multiplication, or blocked matrix multiplication?
OK, can you tell us what that is?
Yeah.
You can break it into [INAUDIBLE]
OK, so say this is our A and B, and we want C. We can break each matrix into four parts.
I'll call this A11, A12, A21, A22.
B11, B12, B21, B22.
Same thing for C. Now, I would like someone to tell me, what is C11, in this case?
Yes
A11 [INAUDIBLE]
A12, B21, yep.
And C12 is-- Just speak up, don't be shy.
This is also my first section ever, in my life, teaching a recitation.
I'm more nervous than you guys.
What is C12?
A11, B12?
A11, B12.
A12, B22, right?
So the rule is the same as before.
Matrix multiplication.
Compute this, we take this row, this column, and it gives us this
So the first one [INAUDIBLE]
Right, thank you.
A12, B21.
OK. And, same thing, C21, we're going to take this row and this column.
A21 B11 plus A22 B21.
C11 is A21 B12 plus A22 B22.
OK. And everyone understands this?
OK, great.
So, now, we've broken up the original problem into several subproblems.
We only need to do matrix multiplication here eight times.
And each of this matrix is half in size.
If the original algorithm is n cubic, now each sub problem is half n cubic.
Then we have eight of them, so the complexity is still n cubic.
No improvement at all.
So, actually, to be more precise, we can further break up these matrices into smaller blocks.
So, to be precise, its complexity should be given by a recursion.
Eight subproblems, each one half the size, plus-- can anyone tell me?
What's the merging complexity, once I get all this?
[INAUDIBLE] Go ahead
Is it just constant because you're adding?
Yeah, because I'm adding.
But is it constant?
I'm adding these two matrices
[INAUDIBLE]
Pardon?
For each level it can base case?
Yeah, for base case, it's, of course, constant.
OK, so maybe I'm not clear about this.
What's the subproblem, in this case?
It's this multiplication operation, OK?
And, so, these are the eight subproblems I will solve.
After solving them, I need to add them together.
And adding two matrices, and each is size ha;f of n. The complexity is--
n squared 2 over 2?
Yeah, it's n squared.
Basically, n squared.
So, to get a precise complexity, we should solve this recursion, but it will end up being the same thing as this intuition, n cubic.
OK, so now, this is the magic.
So Strassen, in 1969, came up with this algorithm
[INAUDIBLE]
Each of these is a half n by half n matrix
[INAUDIBLE]
So Strassen came up with this algorithm.
He somehow defined M1 through M7, seven matrices, in this way.
I can't provide any intuition, because I didn't come up with this.
And, somehow, with those seven matrices, he can reconstruct, he can compute all the four submatrices in C. And it's not very just interesting to check it, because the algorithm is definitely correct.
But let's just do one of them.
OK, how about this one.
So C21 is M2 plus M4.
So M2 plus M4.
So M2 will have A12 B11, A22 B11.
So M4, there's A22 minus B11, so that cancels out.
So we're left with A21 B11 plus A22 B21.
That's the correct answer.
So this, I guess, is a very clever algorithm.
You have to work in that area for 10 years to come up with this so that's not our concern.
Our goal is to analyze this algorithm.
What's the complexity of it?
So does anyone understand this recursion?
Can someone tell me, what's the recursion for this part, for this Strassen algorithm?
We have the original problem, and we have some-- go ahead
So since each of the M 1 through 7 only require one multiplication, you'll need to solve seven subproblems, so 7T n over 2 plus O n squared
That's absolutely correct.
Everyone gets this?
So what Strassen did, is he came up with the seven matrices.
Each one requires only one multiplication.
So we have seven subproblems, instead of eight, and that's going to give us a benefit, an improvement.
So the question now becomes, how do I solve this recursion?
Given this recursion, how do I know its complexity?
And same question there.
Anyone want to give it a try?
So that's going to be covered in a third topic, which is Master Theorem.
So Master Theorem does exactly that.
All it does is, given the recursion, a and T of n over b, plus some work for merging, where a and b are constants, it directly tell you what T of n is, in some cases.
So I'll first write the formula.
Master Theorem actually has three cases.
The first case, fn, is order n raised to c, where c is less than log b of a.
Then, Master Theorem says, it's complexity is theta log b of a.
Second case, fn is theta nc log K, where c is, log b is equal to b of a, then it's complexity is n raised to c log K plus 1 n. You don't necessarily have to copy them, because you just find them anywhere.
And the third case, you can imagine, it's the only remaining case, which is, fn is large, it's omega n raised to c, where c is greater than log b of a.
Then, Master Theorem says, its complexity, the complexity of Tn, is theta fn.
So, intuitively, if fn is not too much work, then it's basically this recursion, what recursion gives you.
If fn dominates, fn is the biggest component, then Tn is, roughly, on the order of fn, and there's a case in the middle.
Now, let's see why that is the case.
I'll only cover one case, here.
So, again, we are going to draw a recursion tree, because that is very useful in all the recursion problems.
So we start with a problem of size n, and we break them into problem size n over b.
So on and so forth.
What's the size of this subproblem?
So that recursion represents a class of recursive algorithm.
Every time it breaks the problem, it reduced the problem size by a factor of b, so what do I have here?
Go ahead
[INAUDIBLE]
n over b squared.
So on and so forth.
So what is a, in this graph?
3
3, think so?
3. a is just a branching factor of this tree.
I keep going.
Finally, I will reach my base case.
So my next question is, after how many levels of recursion will I reach a base case of size 1?
Go ahead
Log b of [INAUDIBLE]?
Log b of-- OK, so, here is n over b, n over b square, next one n over b cubic, so on and so forth.
So, say, this last level it Tth level, than the problems size is n over-- we raise to T, and we want that to be a constant, so what is T?
Log b on n
Log b of n. Now, this is the recursion tree, and we have that fn among the emerging work to do.
So, here, we have to do fn work, to merge these a results to the solution of our problem, n. Then, we have f- what's the emerging work for this level, for this part of the tree?
This is my problem size
[INAUDIBLE]
n over b, right.
And we have a of them.
OK, so on and so forth.
So, then, we know what is Tn.
Let's just enumerate all the work we have to do.
So on the first level, we have to do fn.
OK, on second level, af n of b-- sorry, n over b.
And what's the next level?
We have how many subproblems?
Speak louder
a squared
a square subproblems, and each of them is n over b squared.
And, finally, I reach my last level.
They are all base cases, so I have a raised to T of them, because I defined T to be my depth of the tree.
And each of them is T of 1.
OK, so that's Tn.
I'm not entirely happy with this formula, because I have this beautiful pattern, here, except for that last guy.
It's add one a and divide one b, blah, blah, blah.
So I'm going to change this T into f. Can I do that?
Because it's [INAUDIBLE] is the same, right?
T1 is a constant, f1 is also constant.
Then, I get my beautiful form, where it's a sum from i equals 0 to T. What's in the sum?
Go ahead
[INAUDIBLE]
a raised to i, f of n over b1.
Everyone gets that?
Now, you can roughly see why we have three cases.
So let me deal with the first case.
The first case says, fn is order n raised to c. What does that mean?
It means this guy here is sigma a raised to n, then this is what's in my f raised to c. There should be order, here, but everything has order before it, so I just omit that.
So it actually should be this, OK?
So this, because n raised to c is actually independent of the sum, I can pull it out.
And what am I left with?
Is that correct?
Now, this is a sum of geometric sequence.
We know how to solve that, but we need to check whether this ratio is greater or larger than 1, or if it's equal to 1.
And what is this ratio?
The case tells us.
So c is less than log b of a.
That means b raised to c is less than b raised to this guy, which is a, right?
So we know our denominator is smaller than or numerator.
So this is an increasing sequence, right?
So what we have is n raised to c, then that thing raised to t minus 1 divided by this thing minus 1.
But they are all constants.
Are everyone familiar with this formula.
of geometric sequence?
OK, so that's what we have.
Next, we have t equals log b of n. That means b raised to t is n, correct?
So, then, b raised to t is n, then we have raised to c, they cancel out.
What do we have?
I want to make sure everyone's following
is it a over t?
A raised to t. No questions?
Can you do that stuff again?
OK, let me do that again.
It's actually a raised to t, and then n over bt raised to c, right?
How did you get from the line above that?
This one to here?
Yeah
Oh, it's the sum of geometric sequence, so if I have 1 plus q plus q squared, all the way to qt, it's qt plus 1, I guess-- or t, I don't remember very well-- minus 1, then q minus 1.
I guess this should be t plus 1.
So this is what we're doing.
So this is our q minus 1, and divided by q minus 1, they are all constants, so I don't care about them.
So a raised plus this thing raised to c, but we said that b raised to t is equal to n, so we're just left with a raised to t. And what is t?
t is log b of n. I can write it as log a of n over log a of b.
Everyone familiar with that?
That means log a of n, log b of a.
This is when I flip them.
So this, what is that?
n
That is n. OK, we're done.
Are we?
Not exactly, because I have an order here, right?
So everything is ordered.
If you only care about order, big O, then it's fine.
But that theorem says theta, so you have to prove it the other way, that it's no less than that.
I'm not going to do that.
It's not very hard.
Next, I'm going to apply this theorem to the two problems we left here.
So let's apply Master Theorem to this recurrence.
I think you are still looking at that side.
So what is the a, b, c for this?
a is 8, b is 2, right?
And c is 2.
Log b of a is 3, so that's which case of the Master Theorem?
So the theorem says it should be n raised to log b of a, which is 3, OK?
Now, what we have here, can I remind someone to do that for me?
Want to give it a try?
Go ahead
OK, so we have a equals 7, b equals 2, and c equals 2, like before.
And, now, we want to see whether c equals 2 is less than log b of a, which is log 2 of 7.
Pretty sure that's still the case.
So we should just get n to the log 2 of 7
Yeah, exactly.
So, yeah.
Yeah, thank you.
let's give him a round of applause.
So log 2 of 7 is definitely greater than 2.
Why?
Because log 2 of 4 is 2, right?
So this happens to be n raised to 2.80, and many other digits, but it's less than n cubic.
And, just for knowledge purpose, this is no longer the best.
It was the best, when it was proposed, and, well, researchers in that area have got it down to n raised to 2.35.
I think 2.37, first, then 2.35.
I'm not following the literature very closely, so maybe it's 2.34 now.
So I should have one more thing to cover, but I think we are running out of time.
Sorry about that.
I can post it.
So the last thing we should do is, remember, we have this medium finding algorithm, where we have a recurrence, which is, I think, 10 over 5, some ceiling, and then plus this [INAUDIBLE] n plus theta n. And we want to solve this recursion, but we cannot apply Master Theorem.
Apparently, it's not the right form.
So when Master Theorem doesn't apply, we have to study it case by case.
Let me see if I have time to do that.
OK, I think I probably do.
To solve that case, first, can someone tell me, what's the definition of theta?
We have to go back to the definition to solve that.
What does theta even mean?
So if I say fn is theta n, what do I really mean?
Go ahead
It's tightly bounded, so you can move the [INAUDIBLE] to either side
So it means I can find some K1 and K2 such that this holds when n gets sufficiently large.
OK, so, now, we're going to do induction.
Assuming, for all the small n less than capital N, my Tn is bounded by this K2 and K1 thing.
Then, my next step is, T of this capital N would be bounded.
I'll do the right side first.
It will be bounded by K2 n5 ceiling, plus K2.
[INAUDIBLE] in a second term, n plus another theta n. So, we know, that means it's bounded by some other number.
I'll say A2, then.
That's the definition of theta n. Then, I want this to be-- sorry, all of them should be capital N. Capital N, Capital N. I want this to be smaller than K2 capital N. So let me redo this first step.
This is roughly 5 of K2 plus 7 over 10 K2, plus A2 of n, plus a bunch of constants that I don't care.
I want it to be smaller than K2 of n. Can I reach that?
Of course I can, right?
If I select a K2 to be greater than all we have here, what is this?
This is 9 over 10 K2 plus A2, right?
So if I select the K2 to be greater than 10 times A2-- is everyone following that?
When n is sufficiently large, Tn should be bounded by A2 n. That's the induction.
I am assuming, when n is smaller than capital N, I have solved them.
So I can use these two, and I solve the next step.
So there's the other side, which is very similar.
I'm not going to go through that.
All right, so that's all for today.
And just to quickly recap, we went through the weighted interval scheduling, and the Strassen algorithm, Master Theorem and applying Master Theorem, and that case study of a new recursion.
OK, thanks, everyone, for coming.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this is a 2-3 tree.
So as you can see, every node has-- so the 2-3 is either two children or three children.
Every node can have either one key or two keys.
And the correlation is that every-- so if there are n keys in a node, it has n plus 1 children.
So the way that works is similar to binary search trees.
So if you have value here, the two children surrounding it-- so this side is less, this side is more.
So it's essentially sort of going in order reversal, left child, root, right child.
So [INAUDIBLE], it's ordered.
So generally a B tree will have some nodes.
So let's say n and n plus 1 children.
And if you take anything in the middle, look at the two children, all the keys in this sub-tree are smaller than the key here, and all the keys in this sub-tree are larger than the key here.
So that's the general node.
So before we go into more details of the properties and everything, the question is why use B-trees.
So if we do a quick depth analysis, we can see that the depth is to log n rate.
Is that clear to everyone sort of, why the depth is log n?
Because you have branching just like in binary search trees.
In fact, you have more branching.
But in any case, depth is to log n. But why use B-trees over binary search trees?
Anyone have a reason why you would prefer to use B-trees or not?
So all the operations are still log n. Any guesses?
None.
OK. Well, OK, the reason is memory hierarchy.
So normally in [INAUDIBLE], we just assume that the computer has access to memory, and you can just pick up things from disk and constant time and do your operations with it, and you don't worry about caches and everything.
But that's not how computers work.
So in a computer, you have-- so those of you who have taken some computer architecture class [INAUDIBLE] or something, you will know that hierarchy.
So there's a CPU-- so let's draw it somewhere.
So you have your CPU.
And [INAUDIBLE] CPU, you have some registers.
You have your caches, L1, L2, L3, whatever.
You have your RAM.
You have disk after that.
So disk [?
loads.
?]
Then you have your, I don't know, your cloud, whatever.
So each level, your memory size grows and your access time grows as well.
So in the basic memory hierarchy model, we have just two levels of hierarchy, let's say.
So you have cache connected by a high bandwidth channel to the CPU, and you have a low bandwidth channel to disk.
So the difference is-- so essentially you can consider that cache just has infinite speed.
Cache, just like, whatever you can take it.
You don't have any cost for bringing in stuff from cache.
But it's finite size.
So the way cache works is it has a bunch of words, which is a finite number of words.
So each word has size B, and let's say you have m words.
However, hard disk is just, let's say, infinite memory, but it has some cost associated to accessing things.
Also when you access things from hard disk, you copy them into cache.
When you copy a block of size b, you take it up from the hard disk, and you take a block, and you put it into cache.
And you have to get rid of something because it's fine.
So what you want to do is you want to utilize that b block efficiently.
You just want to bring a b block every time you want to access a new node.
In a binary search tree, normal operations are what?
You start in the root and go to a node.
But that's not very easily correlated with this.
Right?
So if you want to utilize an entire block, you would want something like a block which sort of goes down the tree.
But that's not how binary trees are stored.
Binary trees are stored this way.
So that's the nice thing about B-trees.
So this is just a 2-3 tree.
This is not a general B-tree.
A general B-tree will have a bunch of nodes, and we'll come to that number.
But generally you want to make that number of nodes something like the cache-- what is it?
The word size in the cache.
So once you do that, you can get an entire node from disk, like work on that, and then get another [INAUDIBLE], so your height is reduced.
And you can do your operation much quicker, because you're not accessing disk every time you're going down a level.
Sorry.
You are accessing disk every time you go down a level, but you're utilizing the whole block when you're accessing disk.
Good?
Sort of make sense?
OK.
So let's write down the specifications for B-trees now.
All right.
So number of children.
So first of all, a B-tree has something called a branching factor.
So in the 2-3 tree, the branching factor is two.
So what that means is simply that it just balances the number of children.
So the number of children has to be greater than or equal to 2.
Other than the root node.
The root node can have less than B children.
It's fine.
Also it's upper bounded by 2B [?
plus ?]
2B.
Notice that this is a strict upper bound.
So you can have at most 2B minus 1 children from a node.
Also remember that the number of keys, the number of keys is just 1 less than the number of children.
Therefore, these inequalities are just reduced by 1.
So you have minus 1 and you have 2B minus 1.
So the number of keys can be between minus 1 and 2B minus 2.
The rationale for that will become clear-- yeah?
Is B the height of the tree?
No, B is the branching.
B is the branching factor.
So that is the number of children.
It's not the number of children.
It's a bound of the number of children.
So like in the 2-3 tree, B is equal to 2, and this is a 2-3 tree.
So the 2 refers to-- you can have either two children or you can have three children.
And so the upper bound on children is 2B minus 1.
2B minus 1 is equal to 3.
So you can have two or three children.
And correspondingly, you can have either one or two keys in a node.
Make sense?
Yeah
Cool.
OK So coming back to this.
So the root does not have a lower bound.
The root can have one child in any tree.
So you have a B equal to 5 tree, the root can still have one child-- sorry.
Not one child, one key element, two children.
All right.
It's good.
Also it's completely balanced.
So all the leaves are the same depth.
So you can see it here, right?
So you can't have a dangling node here.
This is not allowed.
You have to have a leaf.
You have to have something going down, and everything ends at the same level.
All right.
So that's the thing.
So also the leaves obviously don't have children, so this condition is violated by the leaf.
So that's the basic structure of a B-tree.
So the first operation we'll consider on B-trees is searching.
So that should be relatively straightforward.
So remember how searching is done in the binary search tree.
You bring in a value x compared to the key.
Let's say x is less than K, you go down this path.
Let's say x is greater than K, you go down this path.
So similarly in a B-tree.
So let's say we bring in a value.
Let's say you are looking for 20.
So you bring in 20 compared to this.
20 is less than 30, so you go down here.
Now you have two values.
So where does 20 fit in here?
Not here.
Not here.
It fits here.
OK. Go down this tree.
You find 20, that's it.
So in general, you bring in a key K, you look at this node, and you go through all the values.
So something I forgot to mention, which should be clear.
All the keys in a node, they're sorted, one after the other.
So your values go like this.
So they're increasing in this way.
Make sense?
So you bring in a key.
Look at all the keys in the node you're looking at, pick the place where K fits in, unless it's already in the node.
Then you're done.
You've found it.
Otherwise, let's say K fits in between these two guys.
So you go down this child and continue.
So searching with log n, similar to BSTs.
So searching is not very interesting.
So next is insertion.
So insertion is a little more interesting than searching.
So what you do in insertion is you-- [SIDE CONVERSATION]
So before we resume, does anyone have any questions about the structure of B-trees.
We rushed through that quite fast.
About how B-trees are structured, everyone good with that?
OK, also any questions about searching in a B-tree or a BST?
Go ahead
Just a random question.
So the 38 there, it can only have two children
Yep.
So one value, two children.
So you have some node in the B-tree, and whatever is below it is split into parts by the elements.
So if you have n elements, it splits it up into n plus 1 segments
You said that the root didn't have to follow the root
No
Why is that?
Well, you'll see when we do insertion and deletion why that's necessary.
But essentially you can consider that it's an invariant.
And all we have to do is preserve that invariant.
So the root, it has to still have less than two-- it still has to have the upper bound.
But it doesn't need to have a lower bound
How do you choose B?
Well, the whole [INAUDIBLE] cache size, so something with that.
So you probably want 2B to be about your cache size so you can get the whole block in one go.
I've never implemented a B-tree, so I don't know how it's actually done in practice.
But that is the reason, so I'm assuming it's something to do with the cache length
Is the 14, is it a child of both 10 and 17?
Well, it's not a child of either.
It's a child of this node.
So this node has two elements, so it's being divided-- dividing the interval up into three parts.
So it's in between 10 and 17 is the point here
So then this node has five children?
Sorry?
No, it has three children.
So don't think of every key as a node.
Think of the whole unit as a node.
So it's not necessarily-- in a binary search tree, you have one element, but here every node has multiple elements.
That's the point of it.
Anyone else?
OK, let's start with searching.
So let's leave this here.
Well, you have the formulas up there, so that's good.
Insertion.
Let's start with insertion.
We already did searching.
So insertion is you bring in a new key K, and you want to insert it into the tree.
So what's the problem that could happen?
You can find the location where you want to insert it, just like searching.
You just go down the tree and find where it should be placed.
But once you do place it, you have a problem.
What is the problem?
The problem is that one of your nodes will become overfull.
Whatever.
It'll overflow, and that's not what you want.
So you want some way so you can manage this.
How do you manage this?
So I have this lovely prop here, which I hope to demonstrate.
OK.
So here we have B equal to 4.
So let's first figure out the number of keys.
So what is the minimum number of keys, anyone for B equal to 4?
Three
Three, precisely.
So what is the maximum number of keys?
Six
4 into 2 minus 3, yeah.
Correct.
3, 4.
It's not seven, there's a strictly less than sign somewhere.
Yes.
And you'll see why it's not seven in a minute.
[LAUGHTER] Oh.
Hypocritical of me.
All right.
So as you can see, 1, 2, 3, 4, 5, 6, 7.
So some insertion happened.
Is the writing clear?
Can everyone read the numbers?
49 looks a little skewed.
Anyway, essentially these are all sorted.
This is the parent node.
Doesn't matter what's over here.
All that matters is 8, 56, and whatever's in between.
So what we do when we have an overfull node is something that's called a split operation.
So split.
And there's something which is called a merge, which we'll come to later when we're doing deletion.
But a split is-- very intuitively, it splits the node into two parts.
So what it does is when you have an overfull node-- so the number of elements here is what?
2B minus 1, which is just 1 over the max.
So what do you do is you take the middle element and remove it.
and now you split the node into two parts.
Observe that there are three here and three here, which is perfect.
And now what you do with the middle node-- so now you're actually disrupting the structure of the tree, because there was one pointer going in.
There was one child.
And now you have two children.
So somehow you need to adjust the parent node, because the parent node had only one child.
Well, at least there are other children off to the side.
But here it had only one child, and now it's split apart.
So you do something very simple.
You just insert this guy in here.
And then you say, oh, this points here, and this points here.
Make sense?
I'm going to get rid of these two.
And you can even convince yourself that this preserves all the nice properties.
So your children have nicely fallen back into their interval.
Your sequence is completely correct, because this was the middle element of this.
So this divides this interval properly.
This is also between 8 and 56, because this was in this node.
So all the properties.
But there's one property that is a problem.
So you have just increased the size of the parent node by 1.
So now it's possible that the parent node has overflowed.
So what do you do?
You split it again.
And split it again.
And if at any point, you're fine, you look at the parent node and go, OK, that's fine.
That's in the range.
But every time it overflows, you can keep going.
And how many times can you do this?
You can do this all the way up to the root.
And when you reach the root, either it's fine or the root is too big.
It's reached 2B minus 1.
And then you split the root, and you get one single [INAUDIBLE] up there.
So that, in answer to your question, that is why you need that property in some sense.
Not a very convincing argument, but sort of.
So let's actually do an insertion in this tree we have here.
So we are going to insert 16.
So 16 comes in here.
It's less than 30, it goes to the left.
It's between 10 and 17, it goes in the middle.
16.
And it's greater than 14, so we add 16 here.
All right.
That seems good.
All the properties fine.
This still has two elements, which is the maximum, but it's good.
It doesn't overflow.
Let's insert something else.
Let's insert 2.
So 2 goes to 30, goes down, goes down.
And we have a problem, because 2 has overflowed this node.
So we split.
And the way we split is we take the middle element.
So we split the node here.
And 3 goes up to the parent, so 3 goes here.
And all good, except for the parent has overflowed.
So what do we do with the parent?
We split the parent again.
And this time, it's right down the middle, the 10 goes up.
So OK, let's get rid of this.
So now that we split the parent, the 10 goes up here.
And you're good.
It's a bit cluttered, so let me reposition the 17.
Did those two operations make sense?
Questions?
If your node size [INAUDIBLE] number of--
So just pick the-- first of all-- OK.
If the way we're doing it-- when your node is overflowing, it's returning only one thing at a time, right?
Yeah
So if your node is overflowing, it'll be 2t minus 1, which is an odd number always.
There might be a case where you get an even number if you do something weird.
Maybe you have a-- there are different ways to do B-trees.
But if it does, you can probably pick the one, either of them, and then [INAUDIBLE].
I'm not sure about that.
I'll look into it.
But in general, if you're doing it this way, it's always odd.
So you don't have to worry about that.
Anything else?
If we did reach all the way to the root and then went one more up--
So what you would do is--
That root would have--
That root would have two children, one element and two children, which is fine because we didn't put that restriction on the root.
That's good.
How we doing on time?
OK, we have some time.
Let's jump into deletion, unless anyone else has questions
[INAUDIBLE] any point?
So-- oh, yeah.
That's a good-- thank you.
So you are going down to the leaves at most-- at most of the leaf ones, and you're going back up one.
So it's like log n plus log n, and you're good.
Let's do deletion.
So deletion is more complicated.
So the reason, it'll be clear.
So the problem in deletion will be remove a node and a node is now underfull.
So it has less than B minus 1 keys in it suddenly.
So let's turn this around.
So again B equal to 4.
This node is a problem.
Only two things in it.
So what do we do?
So before we go into that, let's make this assumption that-- there are two steps to deletion.
The first step is making the deletion at a leaf.
How do you do that?
So the way you make a deletion at a leaf is, let's say, you have a key.
You come down in your B-tree, and you add a node.
Oh, this key needs to be deleted.
But it's not a leaf.
So what do you do?
So essentially what you do is you look at these two subtrees.
So it might have only one subtree.
If it's at the end, it will have only one.
Actually, no, that's not true.
Ignore that.
If it's not a leaf, it has two subtrees.
So either take the rightmost element in this subtree, which is a leaf, because you can always keep going down, right, right, right, right till you get to a leaf, or the leftmost element in this subtree.
So that is just the next element after this guy.
So you delete this, and you bring this up to here.
We'll do an example of this, and it'll be clearer.
So you take either the rightmost element in the left subtree or the leftmost element in the right subtree and bring it up here.
So you sort of like move the deletion to the leaf.
And now it's easier to deal with.
So we will come to that.
Also just note that this is not what is done in the recitation.
This algorithm for deletion, I think, is not done in the recitation notes.
This is a different thing, which I'll send out a link for later.
But I believe it works, because I got it from the [INAUDIBLE] reference.
So once you move to the leaf-- so now let's look at this.
So this is a node that is underfull.
And you want to fix it.
So how do you fix it?
So what do is you look at its siblings.
So in this case, it has one sibling.
It can have up to two siblings.
It can have left or right.
So what you do is you look at a sibling.
And this sibling is actually 1 over the minimum.
And if it's 1 over the minimum, then it's really easy.
All you have to do is take the leftmost thing here-- or if it's the sibling on this side, take the rightmost thing here.
And look at its parent.
So you bring the parent down, and you move the sibling up.
And there we go.
So you basically are rotating the thing into place.
So you move the parent down into the underfull node, and you replace the parent by the leftmost thing here.
Everyone see why that preserves everything?
And the child is also shifted.
Make sure you see that.
So the child which was in this subtree is now in this subtree.
But then you can have the situation where you don't have a nice sibling to take care of your problems.
So in this scenario, the sibling is barely full.
It has three things, and it can't donate anything to you.
So what do you do in that case?
So then you do something which is a parallel of the split operation.
You do a merge.
So what do you have?
So here you have B minus 2, and here you have B minus 1.
And you get 2B minus 3.
Well, you've got another element.
You also take the parent.
So how do you do the merge.
I just want to show you the merge first.
So the way you do it is you move the parent down, and you merge these two.
Seems OK?
So you move the parent node down and merge these two.
And, well, now this comes together, and this points into the new node.
Sort of clear what's going on?
Questions?
Yes?
So now the parent is underfull?
Well, so you have-- yeah, exactly.
So you have decreased the size of the parent, so it might be underfull.
So you propagate.
Anything else?
So are these all different techniques for doing that?
So there are two cases.
So either you have a sibling which has extra nodes to donate to you or you don't.
If you don't, then you have to do this
But what about that case?
Or is that just like--
No, that is moving it down to the leaf.
Once you move the deletion down to the leaf, so here we have something now.
And now you move it all the way back up.
So there are two cases.
Let's do an example.
That'll make it clearer.
How are we doing on time?
Five minutes, all right.
So we are going to delete 38.
38 is gone.
But we want to move it down to the leaf.
So let's take an element.
Let's say we take 41.
So we take 41 and move it up here.
41 is the leftmost thing in the right subtree.
So this vacancy doesn't really affect anything, because this node still has the right number of things, because it's still got one thing in it, which is good.
So you're fine.
This is now just 48.
Let's say we now delete 41.
So 41 is gone.
So now that 41 is gone, what do you replace this blank spot with?
Either this or this, right?
Doesn't matter.
So let's just do this one for consistency.
So you have 48 here.
And now you a problem because you have a blank box.
So can you rotate?
Yes, no?
No, right?
Because sibling is barely full.
So what can you do?
So you merge.
And how do you merge?
You move the 48 down, and you combine everything.
So this is kind of hard to understand, but this is like a zero-element node.
So when you merge, you have 32, 48, and nothing, so it's just 32 and 48.
So what you do is-- so this seems weird, but this is just another empty node.
You just propagated the emptiness upwards.
Now you take this empty node, and you look for its siblings.
Again, its sibling is-- well, it's barely full.
So what do you do now?
You bring the 30 down, and you merge this.
So let's do that.
30 comes down, and there we go.
Looks fine?
Does that tree look good?
Questions about the operation?
I'm sure it was not clear, but-- anything?
Make sense?
OK, let's do a deletion where we can actually do a rotation.
So let's go ahead and delete 20.
So you do your searching, go down the tree.
You find the 20 under here.
So now, OK.
So you're left with just-- actually never mind.
We'll do another one.
So this doesn't do anything.
You lost the 20, and you're left with the 24 this time.
So now you delete the 24.
So now that you've got rid of the 24, you have a blank box here now.
But its sibling is not barely full.
It has something to donate.
So anyone, which elements are going to rotate?
17 and 16
16 and 17, right.
Cool.
So 16 goes up, 17 goes down.
And you're done.
You're consistent again.
So that was deletion.
Those are the two cases for deletion.
Does that make sense?
Anyone?
Any questions?
OK.
So that's all the topics we were supposed to cover today.
Any questions about any of the operations, any of the other topics, lecture, anything?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to analyze two random algorithms we've seen in the last lecture, randomized median and randomized quick sort.
But before I go into that, I'd like to make a correction in problem set two.
I think you all got feedback from that.
So it's the second problem, where you're asked to combine B-trees.
You have two B-trees, T1 and T2 each with some children.
And we're giving you another element, k. And we ask you to combine them.
So if the two trees are the same height, how do you do that?
Does anyone want to share his or her answer?
Go ahead
You put k in the middle of T1 and T2.
And then you can split
Yeah, exactly.
So you put k here.
And if this is too full, you split.
So why do I have to do this?
Can I simply just make k my new root and do this?
Go ahead
Because T1 and T2 don't have to be valid like internal [?
modular ?]
nodes
Yeah, exactly.
So for B-tree, the requirement on the root is slightly different from the requirements for the rest of the nodes.
To this may have too few children.
It is not a valid node.
It's not a valid internal node.
But our solution actually made a mistake in the second part.
In the second part, we're saying so T1 and T2, their heights are different by 1.
Our solution says put k here and make a pointer.
Exactly the same problem.
Does everyone see that?
So this may not be a valid internal node.
So what's the correct solution?
You put k here and combine it with the last child of T1.
And then you may have to split and split again.
OK. Everyone happy with that?
Now, today, we're going to look into randomization specifically we have seen two algorithms in class.
I'll just call them quick find and quick sort.
So quick find is a slightly generalized version of medium finding.
In the very first lecture and recitation, we have seen a non-randomized version of quick sort.
So we divide them into groups of five.
And we find a medium of each group.
And then find the median of the median.
Depending on whether you are smaller or larger, we drew a funny subproblem like this.
Anyone remember that?
And we analyzed this runtime, where our recursion was one fifth plus 7 over 10 or something like that.
And we show it's the worst case O of n. So that's a smart algorithm, I'll give you that.
But it's just complicated.
You have to divide them into groups and do, well, several recursive course.
And also, let me digress a little bit, there's a very interesting point regarding this worst case O of n algorithm.
Has anyone wondered why we use groups of five?
Why not groups of three?
Algorithms should work in the same way, right?
If we take out this first row and this last row, we can still find the median, which is just a second element in each group and find the median of the median.
We still have a subproblem that looks like this.
Exercise.
And it turns out if we use groups of three, when we solve the recursion, it doesn't solve to O of n. It solves to something else.
OK. Now, end of digression.
Let's get back to the randomized version.
So how does the randomized version work?
It's that much simpler.
We have an array.
Let me call it Find in array A, which is of size n. And we want to find i-th largest or smallest element in it.
So what we're going to do is that we'll pick a random element, x, in this array.
And then we'll put all the smaller elements on this side and all the larger elements on that side.
Now, because I'm picking the random one, so this x can be anywhere.
If it is the k-th smallest element from the left, then what do I do next?
My goal is to find out the i-th smallest element in this array,
If A is longer than i then [INAUDIBLE]
OK.
If i is less than k, then my element is on this side, right?
So I should find-- OK, let me define this to be the left array, this to be the right array.
I should find in the left array, what is its size?
It's k minus 1.
Make sense?
This is k minus 1.
And that is n minus k plus one element in the middle.
And so what's the last argument in that function called?
It's i.
So on the other hand, if i is greater than k, then I should go to my right half.
Its problem size is n minus k. So what's the last argument?
i minus k
Agree?
i minus k. OK.
So, of course, if i is equal than k, then we just return x.
Now, obviously, this algorithm's runtime depends on our luck, depends on this choice of k. If k is roughly in the middle, then we reduce the problem size by roughly half.
However, if k is 0 or close to n, then we only reduce the problem size by a little bit.
So it's impossible to give a definite runtime.
So what we opt to do in randomized algorithm is that we analyze expected runtime.
What does that mean?
So we can write the recursion of this.
It's T of n equals-- I have two subproblems.
One of them is T of k minus 1.
The other is T of n minus k. So which one should I put into the recursion?
[INAUDIBLE]
Yeah.
I don't know, right?
I don't know whether my element is on a left or on the right.
So I'll be conservative and take a max of these two.
Let me write it down a little bit.
And I have some amount of work to do before I go to my subproblem.
What's the complexity of that work?
Go ahead
[INAUDIBLE]
It's all O of n. O of theta n. Why is that?
Because you have to create the array
Yes.
Because you have to scan the array once to put the smaller elements on one side and the larger elements on the other side.
Now, this recurrence is impossible to solve, because I don't know what k is.
So instead, we'll just calculate its expectation.
So the expectation of T of n is taking average over all the randomness, which means the choice of k. So there is a probability that my k is equal to j.
If my k is equal to j, then I should take the maximum of-- sorry.
If my k is equal to j, then I should take the expectation of the maximum between those two.
And according to the definition of expectation, I should do a sum from j equals 0-- no, not zero.
I'm starting with 1 all the way to n, right?
Any questions so far?
Now, obviously, depending on my choice of j, sometimes this one is larger.
Sometimes that one is larger.
I'll just write it a little verbosely.
So if my j is 1, then I should take the right one.
Plus, if j is 2, then I should take m minus 2, so on and so forth, right?
I think everyone gets this, right?
So I'll just directly jump to the next step.
So when j is smaller than half of n, I will take the right one.
If j is larger than half of n, I will take left one.
And they happen to be symmetric.
And what I have here is-- everyone gets that?
Now, maybe I'm missing the minus 1 here.
OK, I shouldn't sum to there.
Sorry.
I have n minus 1, n, minus 2, n minus 3, all the way to half of n. But from there, I know longer go down.
I go backwards, half of n plus 1 plus 2 plus 3, all the way back to n minus 1.
Any questions so far?
Oh, we forgot our last term, which is a theta n. Now, how do we solve this?
Any thoughts?
This is a recurrence on the expectation of T. So for this type of general recurrence, we don't have a very good way.
Instead, what we'll do is just take a random guess, and then see if it is correct.
So I don't really need to guess in this case, because I know it's O of n. So let's just assume our expectation of Tn is theta of n. What does that mean again?
It means I can find some constant, such that this expectation is bounded by a constant times n. So far so good?
Now, we can use induction, assume that this holds for everything up to n minus 1.
And we're going to show this also holds for n. Then we're done, right?
Now, we'll just plug that in.
The expectation of T n will be less or equal than this sum from half of n to n. Right?
Yeah, I just plugged that in.
Of course, plus our theta n term.
Now, what's the sum of this guy?
Any guesses?
n square?
n cube?
or n?
OK.
It's probably a messy.
More cleanly, I can pull this B out.
It's just the sigma of j if I change my sum, decrease the range of sum by 1.
What is the sigma some of j?
What order first?
n square
n square.
OK. Yeah, definitely n square.
But we need to be a little bit more precise than that.
So what's the coefficient before the n square?
So I claim this coefficient is 3 over 8.
Can anyone see that?
Why did you assume that the expected value is theta n
Oh, that's just a guess.
If it's wrong, we'll have to assume something else, which we'll see in the next example.
But good question.
OK. Yeah.
Let me ask the question again.
I claim this sum is roughly 3 over 8 n square.
Can anyone see that?
Any ideas?
So I don't know how to calculate this term.
But I do know how to calculate sigma from 1 to n, right?
This is easy.
What's that?
It's half of n, n minus 1.
So it's roughly half of n squared.
Now, this term is the sum of this minus the sum to half of n. So it's roughly half of n squared minus one half of one half n squared.
Makes sense?
So this is roughly 3 over 8 n squared plus an order n term or less, or constant.
Any questions?
Does that makes sense?
Then it's very easy if we just plug this in.
Sorry.
There is a mistake.
I just realized.
Can anyone point that out?
So how many terms do I have in total?
One from n, I have [?
a ?]
term.
So each term should appear twice.
Correct?
So I should have a two here.
And so I somehow just throw away this probability.
But this probability is 1 over n. Because I'm choosing a random element, there is 1 over n probability that is equal to 1, equal to 2, 3, 4.
Every of this is 1 over n. Correct?
So I should have 2 over n here, 2 over here.
And if we plug this in, it's 3 over 8 n cubed plus a theta n. Our goal is to show this is less than B times n, which is clearly true.
Because this is 3/4 of n, 3/4 of B times n, plus another term.
We can say this is another constant D times n. And if we choose B accordingly, this can hold.
Any questions?
You look confused or too easy.
OK. Our guess is the latter.
Oh, it is not?
OK?
So then we have solved this expected runtime of quick find.
Now, let's look at quick sort.
Quick sort is very similar.
The only difference is that once I put all the smaller elements on one side and the larger elements on the other side, instead of going into one of them, I have to sort of both.
So the only change is that instead of taking the max here, I need to add them.
Correct?
So now the same thing here.
Instead of taking the max, I should add them up.
Now when it propagates here, so every term appears twice all the way from j equals 1 to j equals n. Is everyone following?
Can you repeat that?
Hm-hmm?
Can you repeat that?
OK. Sure.
Originally, we have a max here.
So first, did everyone get this part?
We have a plus instead of a max here.
We have to solve both the problems.
Now, if it's a max, then what I have is n minus 1, n minus 2, all the way to half of n, and then half of n, half of n plus 1, all the way back to n minus 1, right?
And if I have a sum, then what I have is for j equals 1's case, I have T of 0 and T of n and minus 1.
This is j equals 1.
If j equals 2, I have T of 1 and T of n minus 2.
As j increases, this one goes from 0 to n. And this one goes from n minus 1 to 0.
I'm going to sum all of them up.
Does that answer your question?
OK.
So instead of from half of n to n, we're summing from 1 to n. Now, we also have another good question here.
Why do I guess it's theta n?
Well, it's just a random guess.
It could be wrong.
For example, in this case, it's just incorrect.
Why?
Because every sum, the range becomes a 1 to n. Now what I have is no longer 3 over 8 over n. What do I have again?
What do I have now if it's 1 over n?
One half
Yeah.
It's one half of n. But if I change one half here, what I get is B times n plus D times n. I want to prove that it's smaller than B times n, which is clearly impossible no matter how you choose B.
Did everyone get that?
If we the same assumption, I make the hypothesis and we plug them in, we can no longer prove the induction step.
OK.
So what we do?
We make another guess.
So let me rewrite our recursion.
So what's the next guess?
Any guesses?
How about we just guess n square?
Anyone unhappy with that guess?
So we can do the same thing.
We can plug it in.
And that will be a-- sorry, I missed another term.
This is 1 over 2 here.
If we make that guess, then what we have is the sum from 1 to n minus 1, the sum of j square plus a theta n. And the sum of j squared is roughly n cubed divided by 3.
Is that obvious to everyone?
Maybe not.
OK. Can anyone explain this to us, why can I claim the sum of square term is n cubed over 3?
Go ahead
It's n times n minus 1 times n minus 2 over [INAUDIBLE]
I think you're correct.
There is a formula, which is-- yeah, I don't remember exactly, but it's roughly this.
If you know this, then you definitely see that.
If you don't, we can turn this sum into an integral.
And that is n cubed over 3.
Make sense?
If we plug that in, what do we have?
2 over n 3n cubed plus theta n. Of course there's a B here.
And we want to show this is less than B times n squared.
Does it hold?
Is it true?
It's clearly true, right?
So this-- no.
This is cubed.
Yeah.
Sorry, I am making many mistakes.
But that's actually good to catch your attention.
But it actually worries me that you didn't point this out.
This is 2/3 n squared times B.
It's clearly less than B n square.
OK.
So we claimed the algorithm is n squared.
Correct?
Go ahead
Does that mean that you claim that it's less than n squared, being that it's n squared definitely?
Exactly.
So what I've proved here is that the algorithm is definitely O of n square, but maybe less.
And you can see we still have a lot of room here.
This inequality is not very tight.
So in fact, it's a very good question, how do we make that guess.
So you already know the answer is n log n, right?
So it's not interesting.
But if you don't, then how do we go about and do things?
We have to make these guesses.
So how about we know this n doesn't hold and 2 is too much.
Next guess is n raised to 1 plus epsilon.
Then what will we have?
If we carry out the same integral argument, we have 2 plus epsilon, n raised to 2 plus epsilon over 2 plus epsilon.
Correct?
And if we plug that in, we get this.
And we want to show it's less than n raised to 1 plus epsilon.
Does this hold?
This term is less than 1.
And that's n raised to 1 plus epsilon.
So this is true.
So we can easily prove it's, indeed, n raised to 1 plus epsilon for any epsilon.
Questions?
But is it tight?
We still don't know.
So then what do we do?
We just make another guess.
And let's guess T of n is [INAUDIBLE] n log n. Definitely, you may run into two cases.
You can either prove it or not.
If this doesn't hold, you just go to log n square.
And gradually, you will find the answer.
Well, if you don't know the answer, probably this is how you do things.
Now, if we guess this n log n, then I have a little harder equation here.
Because it's now j log j.
How do I compute that?
Yeah.
It's not the sum of natural number or squares of numbers, so you cannot use a formula like this.
But we can still use the integral argument.
I'm not going to do that, because that's what you should have learned in calculus or other math class.
But it happens that this integral of j log j is roughly half of n squared log n minus some constant times log n. I think you can change this constant.
But it's roughly smaller than that.
If you plug that in, you get n over 2 one half times B plus theta n. And we want it to be smaller than B times n log n, which will be true.
This is exactly that, but we are minus some term.
And the term we are extracting is larger than the theta n term.
So we can prove this algorithm is n log n. But you can ask the same question, how do I know it's n log n?
Or if I don't know it's n log n, maybe I should go about and try log log n. So you're welcome to try.
And it's actually a very good thought.
Because I think is very uninteresting if you already know the answer.
If you don't know, you have to try that.
But when do you stop?
At a reasonable point, you can also prove the other way that it's runtime is larger than something.
Here, if we prove this big O of n log n, if you can show the other way that it's omega n log n, then you know you have arrived at a final answer.
That's the math part.
Any questions about that?
Yeah.
Any questions about everything I have said so far?
If not, so lastly, I just have a few comments, several terminology.
Now, this recitation we focused on expected runtime.
You have already seen amortized runtime.
Or you may have heard of average runtime.
So to be honest, expected and amortized are just too fancier ways of saying average.
But in algorithm analysis, we do mean slightly different things with these terms.
So the difference is that we are averaging over different things.
So if we say average runtime, we usually mean taking the average over input.
We can imagine a quick sort of quick find algorithm that doesn't use randomness, where you'll always select your first element as your favorite.
That's a reasonable algorithm.
If your input is random, then you can carry out the same argument and show that its complexity is over n or n log n. But if your input is pre-sorted or reverse sorted in some special cases, you cannot do that.
So average runtime is usually a very weak argument, because you have to make assumptions about your input.
And expected runtime is definitely better, because we are taking average over the randomness we introduced.
They are independent of the input.
So we're not making any assumptions on the input.
So of course, this comes at a price.
This randomness doesn't come for free.
So in fact, it's very hard to generate high quantity random numbers.
Maybe at the end of the class, you will see that in crypto, a lot of works are just devoted to generate high quantity random numbers.
If you can do that efficiently, you can actually solve a lot of problems.
So amortized runtime are slightly, again, different from these two, because they are taking average over a number of operations.
You're doing many, many operations in a row.
And some of them takes longer.
Some of them take shorter.
And you take average on those.
OK. That's this recitation.
Any questions?
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from 100 of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Everyone, today we're going to look at dynamic programming again.
So I think I have mentioned several times, so you should all know it by heart now, the dynamic programming, its main idea is divide the problem into subproblems and reuse the results of the problems you already solved.
Right?
And, of course, in 6.046 we always care about the runtime.
So those are the two big themes for dynamic programming.
Now, let's start with a warm-up example.
It's extremely simple.
Let's say we have a grid, and there's a robot from, say, coordinate 1,1 and it wants to go to coordinate m,n.
So at every step, it can only either take a step up, or take a step on the right.
So how many distinct paths are there for the robot to take?
Is the question clear?
So we have a robot at coordinate 1,1.
It wants to go to coordinate m,n.
And every step, it can either take a step up, or take a step to the right.
How many distinct path are there that can take the robot to its destination?
Any ideas how to solve that?
Go ahead
So, we define subproblems as the number of distinct paths from some point x,y to m,n.
Then the number of distinct paths from some point is the number of paths if you go up if you're allowed to go up, plus the number of paths if you go right if you're allowed to go right.
So if you were on the edge, [INAUDIBLE]
Yup, yup.
Does everyone got that?
So, it's very simple.
So, I know I have only one way to get to these points.
I need to go all the way right.
And only one way to get to these points.
I need to go all the way up.
So for all the intermediate nodes, my number of choices are-- is this board moving?
Are just the number of distinct paths I can come from my left, plus the number of distinct path I can come from bottom.
And then I can go in.
For every node, I'll just take a sum between the two numbers on my left and on my bottom.
And go from there.
OK. Is that clear?
So this example is very simple, but it does illustrate the point of dynamic programming very well.
You solve subproblems, and ask how many distinct path can I come here, and you reuse the results of, for example, this subproblem because you are using it to compute this number and that number.
If you don't do that, if you don't memorize and reuse the results, then your runtime will be worse.
So what's the runtime of that?
Speak up
[INAUDIBLE]
It's just m times n. Why?
Because I have this many unique sub problems.
One at each point, and I'm just taking the sum of two numbers at each subproblem, so it takes me constant time to merge the results from my subproblems to get my problem.
So to analyze runtime, usually we ask the question how many unique problems do I have.
And what's the amount of merge work I have to do at every step?
That's the toy example.
Now let's look at some more complicated examples.
Our first one is called make change.
As its name suggests, we have a bunch of coins.
s1, s2, all the way to, say, sm.
So each coin has some values, like 1 cent, 5 cent, 10 cent.
We're going to make change for a total of n cents, and ask what's the minimum number of coins do I need to make change of n cents.
So to guarantee that we can always make this change, we'll set s1 to be 1.
Otherwise, there's a chance that the problem is unsolvable.
Any ideas?
Is the problem clear?
How do you find s1 again?
Or si?
What, these numbers?
They are inputs.
They are also inputs.
It could be 1 cent, 5 cent, 10 cent.
Or 3 cent, 7 cent.
Though the smallest one is always 1.
OK.
I need to find a combination of them.
For each of them, I have an infinite number of them.
So I can find two of these, three of that, five of that, such that their sum is n. Is the problem clear?
OK. Any ideas how to solve that?
So let's just use a naive or very straightforward algorithms.
Go ahead
You pick one, and then you do mc of n minus that
OK, great.
Yeah, let's just do exhaustive search.
Let's pick si.
If I pick this coin, then my subproblem becomes n minus the coin value.
And of course, I use the one coin.
That's si.
So then I think the min of this for all the i's, and that's the solution.
So far so good?
OK.
So what's the runtime of this algorithm?
If it's not immediately obvious, then we ask how many unique subproblems are there.
And how much work do I have to do to go from my subproblems to my original problem?
So how many subproblem are there?
So to be clear, for this one, we have to call this recursive call again.
n minus si, probably minus sj.
And if you cannot compute how many subproblems are there, let's just give a bound.
Any ideas?
John, right?
I'm not sure there would be more than n subproblems, because the smallest amount we can subtract from the original is 1.
And if we keep subtracting 1 repeatedly, we get n subproblems, and that will cover everything-- that subproblem
Yeah, correct.
So this may not be a very tight bound, but we know we cannot have more than this number of subproblems.
Actually, I don't need to even put the order there.
I know we can have no more than n subproblems.
They're just make change of n, n minus 1, n minus 2, all the way to make change 1.
And actually, this bound is pretty tight, because we set our smallest coin is 1, so we won't make a recursive call to make change n minus 1, right?
If I pick the 1 coin, the 1 cent coin first.
And then from there, I will pick a 1 cent coin again.
That gives me a subproblem with n minus 2.
So indeed, I will encounter all the n subproblems.
OK, so having realized that, how much work do I have to do to go from here to there?
[INAUDIBLE]
Correct.
Because I'm taking the min of how many terms?
m terms.
So that's our runtime.
Any questions so far?
If not, let me take a digression.
So, make change, this problem.
If you think about it, it's very similar to knapsack.
Has anyone not heard of this problem?
Knapsack means you have a bunch of items.
You want to pack these into a bag, and the bag has a certain size.
So each item has a certain value, and you want to pack the items that have the largest combined value into your bag.
So, why are they similar?
So in some sense, n is our size.
We want to pick a bunch of coins to make the size n. And each coin here actually has a negative value, because we want to pick the min of it.
If you do that, then this problem is exactly knapsack.
And knapsack is NP-complete.
That means we don't know a polynomial solution to it yet.
However, we just found one.
Our input is, m stuff and n. Our solution is polynomial to m, and polynomial to n. If this is true, then I have found the polynomial solution to one NP problem.
So P equals NP.
SO we should all be getting Turing award for that.
So clearly something's wrong.
But there's no problem with this solution.
This covers all the cases.
And our analysis is definitely correct.
So does anyone get what I'm asking?
So what's the contradiction here?
I will probably discuss this later, in later lectures when we get to complexity or reduction.
But to give a short answer, the problem is that when we say the input is n, its size is not n. So I only need log n this to represent this input.
Make sense?
Therefore, for log n length input, my runtime is n. That means my runtime is exponential.
It's not polynomial.
OK. Now that's the end of the digression.
Now let's look at another example.
This one is called rectangular blocks.
So in this problem, we have a bunch of blocks.
Say 1, 2, all the way to n. And each of them has a length, width, and height.
So it's a three-dimensional block.
So I want to put blocks, stack them on top of each other to get the maximum height.
But in order for j to be put on top of i, I require the length of j to be smaller then the length of i, and the width of j is also smaller with width of i.
So visually I just meant this is a block.
I can put another block on there.
They are smaller in width and length.
But I cannot put this guy on top of it because one of its dimension is larger than the underlying block.
And to make things simple, that's not allowed, rotating.
So OK, I can rotate.
It still doesn't fit.
But you see the complication.
So you allow rotate, then there's more possibility.
Length and width are so one of them is north-south, the other is east-west, and you cannot change that.
OK. Is the problem clear?
You want to stack one on top of each other to get the maximum height.
Any ideas?
Again, let's start from simple algorithm.
Say, let's just try everything out.
OK, go ahead
If you try everything else, you have n factorial
Pardon?
It would be O of n factorial?
You're going too fast.
Let's write the algorithm first.
So I want to solve my rectangle block problem, say from 1 to n. What are my subproblems?
Choose one block
OK. Let's choose one block
And then you run RB of everything except that block
So I get its height, and then I have a subproblem.
What is the subproblem?
And then I'll take a max.
So the difficulty here is this subproblem.
So Andrew, right?
So Andrew said it's just everything except i.
Is that the case?
Go ahead
It's everything except i, and anything with wider or longer than i
Do you get that?
Not only do we have to exclude i, we also have to exclude everything longer or wider than i.
So that's actually a messy problem.
So let me define this subproblem to be a compatible set of w i.
And let me define that to be the set of blocks where the length is smaller than the required length, and their which is also smaller than the required width.
So this should remind you of the weighted interval scheduling problem, where we define a compatible set once we have chosen some block.
Question?
What are we trying to do here?
Are we trying to minimize h?
Maximize h. We want to get as high as possible.
I choose a block, I get its height, and then I find out the competitive remaining blocks, and I want to stack them on top of it.
Everyone agrees this solution is correct?
OK, then let's analyze its runtime.
So how do we analyze runtime?
So what's the first question I always ask?
How many subproblems?
Yeah.
I'm not sure who said that, but how many subproblems do we have?
At most n?
At most n. Can you explain why is that the case?
Or it's just a guess?
Because if n is compatible-- nothing in the compatible-- n will not be in the compatible set of anything that is in the compatible set of n
OK, that's very tricky.
I didn't get that.
Can you say that again?
Because for example, if you start with n, then everything that's in the compatible set of n. n won't be in the compatible set of that
OK.
I think I got what you said.
So, if we think there are only n subproblems, what are they?
They have to be compatible sets l1, w1, then l2, w2.
These are the n unique subproblems you are thinking about.
Is there any chance that I will get a compatible set like something like l3 but w5?
If I ever have this subproblem then, well, my number of subproblems are kind of exploding.
Yeah, I see many of you are saying no.
Why not?
Because if we have a subproblem, say, compatible set of l i and w i, and if we go from here, and choose the next block, say t, it's guaranteed that t is shorter and narrower.
That means our new subproblem, or new compatible set becomes-- our new subproblem needs to be compatible with t instead of i.
So, the only subproblems I can get are these ones.
I cannot have one of these.
The number of subproblems are n. And how much work do I have to do at each level?
n
n, because I'm just taking the max, and there are n potential choices inside my max.
So runtime n squared.
OK, we're not fully done, because there is an extra step when we're trying to do this.
We have to figure out what each of these are.
Because once I go into this subproblem, I need to take a max on all the blocks that's in this set.
I have to know what blocks are in that set.
Is that hard?
So how would you do that?
You just check for all of them, and that's O of n
OK.
So, I check all of them.
That's O of n. I'm pretty sure you just meant scanning, scan the entire thing, and pick out the compatible ones.
But that's for this subproblem.
We have to do it for every one.
Or there may be a better way.
So I think the previous TA is telling me there's a better way to do that.
So in order to find the entire compatible stuff, he claims he can do it in n log n, but I haven't checked that, so I'm not sure.
This is a folklore legend here.
Yeah, we'll double check that offline.
But assuming if I don't have this, then figure out all these subproblems will also take n squared.
Then my total runtime is n squared plus n squared, and still n squared.
Question?
Is the n log n solution giving us sorting this by [INAUDIBLE]?
Yeah, I think it should be something along those lines, but yeah, I haven't figured out whether you sort by length or by width.
You can only sort by one of them.
So after sorting, say let's sort by length.
Then after sorting, I may get something like this.
And if I'm asking what's the compatible set of width this guy, I still have to kick all of them out.
Yeah, so it's not entirely clear to me how to do it, but I think you can potentially consider having another, say, binary search tree that's sorted by width, and you can go in and just delete everything larger than a certain width.
So that's the, yeah.
OK, go ahead
Can you convert into a directed graph, where each pair of shapes that's compatible, you do an edge.
And then path find
OK. OK.
But constructing that graph already takes O n squared, correct?
Yeah, OK, let's move on.
I don't have time to figure this out.
So, this problem is remotely similar to interval scheduling, weighted interval scheduling, in a sense that it has some compatible set.
And in the very first lecture and recitation, we have two algorithm for weighted interval scheduling, and one of them is better than the other.
And this one looks like the naive algorithm.
So, does anyone remember what the better algorithm is for weighted interval scheduling?
But instead of checking every one as my potential lowest one, it really doesn't make sense to do that.
Because for the very small ones, I shouldn't put them as my bottom one.
I should try the larger ones first as the very bottom one.
Go ahead.
Oh, you're not--
You could create a sorted list of length n with the width.
So you know that items that are later in the list, they're not going to be in the first level of the tower
Yeah, correct.
So, just in the same line of thought as weighted interval scheduling, let's first sort them.
But then, it's a little tricky because do I sort by length or width?
So I'm not sure yet, so let's just sort by length and then width.
So this means if they have the same length, then I'll sort them by width.
So I can create a sorted list.
Let me just assume that it's in-place sort, and now I have the sorted list.
So once I have that, the potential solutions I should consider is that whether or not I put my first block as the bottom one.
It doesn't make sense for me to put a later one down.
So my original problem becomes taking the max, and whether or not I choose block one.
If I do, then I get its weight-- height, sorry.
And my subproblem is the ones compatible with it.
If I do not choose it, then my sub problem is like what Andrew first said, from 2 all the way to n. So why is this correct?
So I claim this covers all the cases.
Either h1 is chosen as the first bottom one, or it's not.
It's not chosen at all.
It's impossible for h1 to be somewhere in the middle, because it has the longest, largest length.
OK.
So how many subproblems do I have?
Go ahead.
Still n. So there are all of these compatible set of l1 w1, l2 w2.
But it looks like I do have some new subproblems.
These do not exist before.
However, there are only n of them.
They're just a suffix of the entire set.
So I still have O of n subproblems.
And at each step, I'm doing constant amount of work.
There are just two items.
So we found an order n solution.
Are we done?
Is it really order n?
OK, no
You still have to find the c
Yeah.
I still have to find all these c's.
And first, I actually have a sort step.
That sort step is n log n. Yeah, then again, well, if we do it naively, then it's again n squared, because I have to find this compatible set, each of them.
But if there's an n log n solution to find these compatible sets, then my final runtime is n log n. Make sense?
Any questions so far?
OK.
So now we actually have a choice.
So we can either go through another DP example, I do have another one.
But Nancy, one of the lecturers suggested, that it seems that many people have some trouble understanding yesterday's lecture on universal hashing and perfect hashing.
So we can also consider going through that.
Well, of course, the third option is to just call it a day.
So, let me just take a poll.
How many people before we go over the hash stuff?
How many people prefer another DP example?
OK.
Sorry guys.
How many people just want to leave?
It's fine.
OK. Great.
That's it.
OK.
So, so much for DP.
We do have another example.
We will release it in recitation notes.
For those of you who are interested, you can take a look.
So, well, sure you all know that we haven't go into DP in the main lectures yet.
So this is really just a warm up to prepare you to go to the more advanced DP concepts.
And also, DP will be covered in quiz 1.
But the difficulty will be strictly easier than the examples we covered here.
OK?
Now let's review universal and perfect hashing.
So it's not like I have a better way to teach it.
Our advantage here is that we have fewer people, so you can ask questions you have.
So let me start with the motivating example.
So why do we care about hash?
It's because we want to create a hash table of, say, n. It has n bins.
And we will receive input, say, k0, k1, all the way to k n minus 1. n keys.
And we'll create a hash function to each of them to map them to one of the bins.
That the hope is that if n is theta m, or in the other way, m is theta n, then each bin should contain a constant number of keys.
So to complete the picture, all the keys are drawn from a universe that has size u.
And this u is usually pretty large.
Let's say it's larger than m squared.
It's larger than the square of my hash table size.
But let me first start with a negative result.
So if my hash function is deterministic, then there always exists a series of input that all map to the same thing.
We call that worst case.
We don't like the worst case.
Why?
Because in that case, the hash is not doing anything.
We still have all of the items in the same list.
Why is that lemma true?
Because by a very simple pigeonhole argument, so imagine I insert all of the keys in the universe into my hash table.
I would never do that in practice.
It's just a thought experiment.
So by a simple pigeonhole argument, if u is greater than m squared, then at least some bin will contain more than m elements.
Well, if it just so happens that my inputs are these m keys, then my hash will hash all of them to the same bin.
Make sense?
So this is the problem we're trying to solve.
We don't want this worst case.
And it does say that if h is deterministic, we cannot avoid that.
There always exist a worst case.
So what's the solution?
Then the solution is to randomize h. However, I can't really randomize h. If h take some key, if my hash function maps a key into a certain bin, well, the next time I call this hash function, it better give the same bin.
Otherwise I cannot find that item.
So h needs to be deterministic.
So now our only choice is to pick a random h. Make sense?
Every hash function is deterministic, but we will pick a random one from a family of hash functions.
So in some sense, this is cheating.
Why?
Because all I'm saying is I will not choose a hash function beforehand.
I will wait for the user to insert inputs.
If I have too many collisions, I'll choose another one.
If I have too many collisions.
I'll choose another one.
OK.
I think I forgot to mention one thing that's important.
So you may ask why do I care?
Why do I care about that worst case?
What's the chance of it happening in practice?
It's very low, but in algorithms, we really don't like making assumptions on inputs.
Why?
Because if you imagine you're running, say, a website, a web server, and you code has some has table in it.
So if your competitor, or someone who hates you, wants to put you out of business, and if he knows your hash function, he can create a worst case input.
That will make your website infinitely slow.
So what we are saying here is I don't tell him what hash function I'll use.
I'll say I choose one.
If he figures out the wrong input, the worst case input, I'm going to change my hash function and use another one.
Make sense?
Now the definition of universal hash function is that if I pick a random h from my universal hash function family, the probability that any key i mapped to the same bin as any key j should be less or equal than 1 over m, where m is my hash table.
This is really the best you can get.
If the hash function is really evenly distributing things, you should get this property.
So we have seen one universal hash function in the class.
I'll just go over the other example, which is ak plus b modulo p, and then modulo m. So p is a prime number that is greater than the universe size.
We'll see why this is a universal hash function.
So to do that, we just need to analyze the collision probability.
So if I have two key, that k1 and k2 that map to the same bin, that means they must have this property.
After taking the mod m, their difference should be a multiple of m. Because if this is true after taking the modulo m, they will map to the same bin.
Make sense?
Now I can quickly write it as a times the difference of the key equals a multiple of m, mod p. Now, k1 and k2 are not equal, so they are nonzero.
And in this group, based on some number theory, we have an inverse element for it.
So, if this happens, we'll call it a bad a.
How many bad a's do I have?
One of a will make this equation holds with i equals 1.
Another a make the equation holds with i equals 2.
But how many such a's do I have?
At most, because this equation can hold with m, 2m, 3m, all the way to p over m floored m. This is the total number of possible ways this equation can hold.
So how many bad a's do I have?
I have p over m, over the total number of a's, which is p minus 1.
Oh, yeah, I forgot to mention that.
So a is from 1 to p minus one.
OK.
So I can always choose my p to be not a multiple of m. If I do that, this floor-- so, then p and p minus 1 do not cross the boundary of modulo m. Then this is true, and this is less than 1 over m. So this is a universal hash function family.
So what's the randomness here?
The randomness is a. I'll pick an a to get one of my hash, and if it doesn't work, I pick another a
What is b?
What is b?
p is a prime number I choose--
[INAUDIBLE]
b?
Yeah
Oh, b. I think it's also a random number.
Yeah, so, actually it's not needed, but I think there's some deep reason that they keep it in the hash function.
I'm not sure why.
Now once we have that, once we have universal hash, people also want perfect hashing, which means I want absolutely 0 collision.
So how do I do that?
Let me first give a method 1.
I'll just use any universal hash function, but I choose my m to be n squared.
I claim this is a perfect hash function with certain probability.
Why?
Because I want to calculate probability no collision.
Yeah, 1 minus probability I do have a collision.
And I can use a union bound.
That's the probability that any pair has a collision.
Any pair of hx equals hy.
How many pairs do I have?
N choose 2
Yeah.
n choose 2, which is this number.
So if it's a universal hash function, then any collision, any two colliding, the probability is 1 over m. So I choose my m to be n squared, so this one is larger than 1/2.
So what I'm saying, to get a perfect hash function, I'll just use the simplest way.
I select the universal hash function with m equals n squared.
I have a probability more than 1/2 to succeed.
Or if I don't succeed, I'll choose another one until I succeed.
So this is a randomized algorithm, and we can make it a Monte Carlo algorithm or Las Vegas algorithm.
So I can either say if I choose alpha log n times, then what's the chance that none of my choice satisfies perfect hashing?
My failure probability is less than this.
My each chance I have a half success rate, and I try this many times, what's the chance of all of them failing?
This is 1 over n raised to alpha.
Of course, I can also say, I'll keep trying until I succeed.
Then I have a 100 percent success rate, but my runtime could potentially go unbounded.
Make sense?
OK.
This sounds like a perfect solution.
The only problem is that the space complexity of this method is n squared, because I choose my m hash table size to be n squared.
So this is the only thing we don't want in this simple method.
Our final goal, is to have a perfect hash function that has space O of n, and also runtime some polynomial in n, and failure probability arbitrarily small.
And the idea there is this two-level hashing.
So, I choose h1 first to hash my keys into bins.
And for each bin, say I get l1 elements here, l2 elements here, so on and so forth.
I'll choose each of the bins to be a second level perfect hashing.
So we can use the method one to choose this small one.
If I choose m1, which is the hash table size of this guy, to be l1 squared, then I know after alpha log n trial, this one should be a perfect hashing.
After another alpha log n trial, I should resolve all the conflicts in l2 to make it a perfect hashing.
Make sense?
So after n log n trials, I will resolve all the conflicts in my second level hashing.
Question?
It was mentioned in the lecture that this only works if there are no inserts or deletes, or something like that?
Let me think about that offline.
I'm not sure about that.
OK.
So the only remaining problem is we need to figure out whether we achieve this space O of n. What is this space complexity of this algorithm?
It's n plus l i squared, because each table size is the square of the elements in it.
And finally, we have that Markov inequality or I think something like that, to prove this is the case with-- so my space is O of n, also with the probability of greater than 1/2.
I can keep going.
I'll try alpha log n times on my first level hash function, until my space is O of n. Once I get to that point, I'll try choosing universal hash functions for my smaller tables, until I succeed.
OK?
That's it for hashing and DP.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Feel free to populate the front row.
I'm not that scary.
So today, we're going to look at more greedy algorithms.
So I think you went over Kruskal's algorithm and how you do the sorting in the lecture.
So going back to make change from last recitation, so this is sort of a variant on that.
So instead of discrete coins, we now have continuous coins, in the sense so the analogy here is, let's say, you have N metals, and each of the metals has some value given by Ci dollars per kilogram, or whatever units you prefer.
And you want to achieve some value T. You want to give someone T dollars worth of metal.
And you want to do this while minimizing-- oh, so I should mention this.
ki is the weight of every metal that you will give to the person.
So you're taking ki of metal i, and you are going to-- and you have to ensure, so basically, you have to ensure that some summation of ki Ci over all i is equal to T. And in doing so you want to minimize the summation over all ki.
So does that make sense?
So you have a bunch of metals.
Some of them are more expensive than others.
And you want to measure them out and give someone a certain fixed value.
So anyone have any ideas how to do this?
Should be-- should be the first thing that comes to mind.
So you have much of metals-- some of them with certain costs.
And you're trying to create a value T. So which metal would you want to pick?
So it should seem intuitive that if you want to minimize the weight of the metal, you would want to pick the-- have the most expensive one for weight.
So let's start by sort by Ci.
And we want to sort it in decreasing order.
Does make sense?
So if you have the most expensive metal, you want to use as much of that as you can, so that your weight is minimized.
So once you sort by Ci, so let's say, you have your costs right now are-- let's call this one C1, C2, up to Cn.
And these are in sorted order.
So it's increasing this way.
So you now take your value T, and you look at T by C1.
And that is the amount of weight you would need to generate C. So you look at how much you have here.
So the amount of metal-- so a constraint I forgot to mention, you are given a limited amount of every metal.
OK, that's-- it's not that trivial.
So you have-- let's mention that.
So you have-- is that used?
No, it's not-- amount.
Does that make more sense?
So you look at T over Ci.
And if T over Ci is greater than W of i, then you just use the amount you need to construct Wi, and you're done.
Otherwise, you use all of Ci.
So if it's less than Wi, in that case, you-- sorry, other way around.
If it's greater than Wi, you use all of it.
And then you move on to the next one, the next one, and so on.
So that seems pretty intuitive.
Let's actually do a formal proof of that.
So how you go about proving this is that-- so let's say-- so it's what we call the current base method.
So basically what you have is, let's say you're not using the most expensive metal you have at this point.
So let's say your most expensive metal has cost Ci, but instead, you decide to use Cj.
So let's say you decide to use some kj amount of Cj.
So the value you're getting from this is Cj kj.
And instead, if you use Ci, how much metal would you need to get the same value?
You would need Cj kj over Ci.
Does that make sense?
So this is the value you would obtain by using kj kilograms of this metal.
So if you instead used this one, you'd get this value.
And so this is the most expensive one.
And Ci is greater than Cj, this value, so this value is less than kj.
So by using this metal instead of that one, you are decreasing the amount-- the weight you would need, so your minimization goes down.
Make sense?
So that's like a very simple greedy algorithm.
And it's-- the algorithm is exactly what you'd expect, and the proof isn't very hard.
So let's move on to a slightly interesting one.
So this is process scheduling.
So let's say you have a computer, and you're running end processes.
And each of the process has a time-- t1 through tn, again processes.
And you want to order them in some way.
So first, you will do process p1.
Then you'll process p2, and so on, and so forth.
Then you'll define a completion time.
So completion time is simply when does process i end.
So when does process i end?
You just p1 plus-- it's like the time for p1 plus time for p2 up to pn.
So basically, you have all your processes.
So let's says this is p1, this is p2, and so on.
And the completion time for a certain process in the middle is just the sum of all times before it.
That's completion time.
And now what you want to do is you want to minimize the average completion time, which is summation over all the completion times over n. So any ideas what an algorithm for this would look like?
Essentially, you want to minimize the sum of all these times.
So all these times, you want to minimize the average of these.
So what do you want to do?
Do you want to shift the slower-- the processes which take more time, do you want to keep them at the end, or do you want to keep them at the beginning?
So if you have a bunch of small processes, what do you do with them at the end?
What do you do at the beginning?
Completion time is when does-- So let's say this is process pi.
And completion time for process pi is like this distance.
It's like, when does pi get completed?
So it's summation of all the times-- so the time taken for p1, p2, up to the end, see?
So you want to basically minimize the average of these values.
So where do you put the smaller processes-- would you put the shorter processes at the end or the beginning?
Which one would decrease your average?
The beginning?
Makes sense, right?
So yeah, that makes sense.
So you basically want to like scrunch these lines towards the beginning, so your average is smaller.
Note that this total length is always the constant.
It's like summation over all ti.
So let's go about-- so OK, this is strategy.
Again, sort by ti, and this is increasing order, and that's it basically.
So this is your algorithm, sorted by tn, but use the process in that order.
So let's try to prove this.
So the way you prove this is a pretty generic method.
It is often used to prove greedy algorithms.
So let's say that this is not the optimal.
Let's say someone comes up to you and tells you, OK, I have a better sequence.
I have a sequence, let's say, called-- let's say I have a sequence of p1 to pn.
And that sequence does better than a sorted order.
So you're like, OK, so if this is not sorted, then you have some elements in the middle.
Let's say you call them pi is greater than pj, with i is less than j.
So there's some pi here, and there's some pj here, such that this is greater than that.
So it's not in sorted order.
So you can always find a pair like that.
So now I'm going to claim that if you swap these two values-- so you swap pi and pj-- that'll actually decrease whatever current average completion time you have.
So initially, you had something like this.
So-- no, let's not draw a line there.
So let's say you had something like you had this process, so pi-- actually, this is the bigger process, so this is pi, and this is pj.
And now I'm saying that-- and you have some stuff in the middle.
And my claim is that, no, this is not optimal.
You'd do much better if you moved the pj over here.
So you want to go from this to this, and big process.
So let's see what changes when you go from there to there.
So first of all, observe that the completion times of everything behind this is the same.
They all have the same completion time; nothing is affected.
And you're only changing these two things.
Everything after this is also the same-- has the same completion time.
So the only things that are changing are this one, this one, and all the ones up to this one.
Even this one has the same completion time.
Make sense?
So how much is this changing by?
So let's define this.
Delta is equal to t of pi minus t of pj-- so the difference between these two processes.
So the original completion time of pi was this.
And now the corresponding process down here, the completion time is decreased by delta.
So completion time for us goes down like minus delta.
This is a summation of completion time.
This divided by n is a constant.
So you just want to minimize this.
So first it goes-- so this one goes down minus delta.
So let's look at the next process.
The next process is something like this.
So again, these do not change.
You're only swapping these two.
So this completion time also down a minus delta, and so on, and so forth.
So you just get a bunch of minus deltas, which is equal to however many processes you have.
But that's not even important.
What is important is that just by swapping, you're going to get at least one minus delta.
And delta is positive, because assumption-- oops, sorry, t-- because assumption was that t pi minus t pj is positive.
So just by swapping, you're going to always decrease it.
So the claim that that sequence was an optimal solution is wrong.
So you can always do better by swapping two inversions.
So that out of sorted order is called an inversion.
So if you solve an inversion, you always get a better result.
Does that proof make sense?
So that's a slightly more interesting recent algorithm.
So let's move on to the third one we have here.
The third one is event overlap.
So this is how it works.
So you wake up in the morning, and you look at your calendar.
And being an MIT student, your calendar looks pretty full.
So let's say this is what it looks like.
So these are your events.
Let's use some colors, make it a little clearer possibly.
And let's say you have another event over here.
You have something here.
You have something here.
You have something here.
And you have something here.
So OK, let's move this down.
So the problem is that you have this bunch of events planned out.
Now clearly, they're overlapping, so you can't attend all of them.
So the idea is you make a bunch of clones of yourself.
And so in this case, look at the matching colors.
So if you create clone number 1 goes here, and clone number 2 goes to red, and clone number 3 goes to blue.
So then clone number 1 does this.
Clone number 3 does the blue one.
Clone number 2 does red.
I guess, we should move the red back a little or forward a little bit just to make it clear.
Yeah, there we go.
And now, you could easily see that this is optimal.
So you can do this with three clones and no less.
So you make three clones, and then you can go off to spring break.
And your schedule is fine.
So now, how would you approach this problem?
So what is a greedy strategy to, given a number of intervals, how do you find the minimum number of clones you need to cover your day?
Any ideas?
What is a naive thing you could do?
When you say to cover your day, then it's like the number--
So you want to do every event.
But like so this clone can't-- so clone number 1 does this event.
Then he can't do this event or this event
Sort of maximizing your events?
You want to do all the events?
You want to minimize the number of clones
[INAUDIBLE]
So it's like interval scheduling.
But you want to do all the intervals, but you're allowed to use multiple people to do all the intervals.
Yes?
Yeah?
You could sort by end time
By end time, OK. What do you do after you sort by end time?
And then iterate over all the intervals once they're sorted and just count how many intervals there are between the [INAUDIBLE].
That would get complicated
So you're close.
So you do begin by sorting.
But you can actually do it by sorting by end time.
It's easier to visualize if you sort by start time.
So leading from that, anyone want to top in?
It's when your sorting starts, and every time you get a class, you get a new clone
So yeah, essentially, every time you can't add it to one of your current clones, you just create a new one.
You could also do it by end time, because it's symmetrical, right?
So if you sort by end time, then you start with the smallest-- last end time and go backwards, exactly the same thing.
So let's write it down.
So sort by start time, and so actually, let's work out this example.
So in this case, you would go-- OK, actually, if once you sort-- so first you have 1, then you have 2, 3, 4, 5, 6.
So that's sorted by start time.
And then you have-- so first you go for this one.
Then you go for 2, and 2 intersects with 1.
So you put 2 into it.
So this is clone number 1.
And then you have to create a new clone for 2, so you create the new clone.
And there we go.
So then you go to 3.
3 clashes with both 1 and 2, so you have to create a new clone again.
So in that case, you go forth and create 3.
Then you go to 4.
Now, 4, you see, it's starts with 2 and 3, but it is good with 1.
So you just put 4 over here.
And if you continue like this, you essentially get this and this.
Make sense?
So that's how you schedule it.
So does that algorithm make sense?
Let's try to prove its correctness.
So let's look at the instance where you're inserting the m-th clone-- so m-th clone.
So when the m-th is created, you already have some values in here.
So you have 1, 2, all the way up to m minus 1.
So now, you bring in your interval, and you see that it collides with all of these values.
So let's just draw the final interval for all these guys.
So let's say the final interval for this guy was out here.
Let's say the final interval for this guy was out here, and so on, and blah, blah, blah, blah, blah.
And so when you create the m-th clone, you look at the start time.
So what happens is that the start time is somewhere, let's say, here.
And now, you know that because of this-- so you're only adding a new clone when you don't have an available slot.
So that means that there is some interval here, which intersects with this guy.
So how do you show that this is one interval?
Well, it's like consider any level.
But say there is no interval that intersects with it.
So that means that there is either-- So if there were a gap here-- so let's say, at this location, this interval wasn't here.
Let's say if you extrapolate this line outward-- so this is your current starting value.
And let's say you look at this line.
And in this segment, you can't have something which starts after this, because this is the current highest sorting starting time.
So there's no interval that starts after this.
So the only interval that's going to exist have already ended here.
And if they're already ended here, that means you could evaluate here.
Does that make sense?
So basically, then you can show that, OK, so at every existing-- if you're adding a new clone, that means at every existing level, you have something which intersects.
So what that means is that you have a single point of time where there are m minus 1 plus 1 intervals.
That means that you absolutely need m intervals regardless of what your strategy is.
So adding the m-th clone is necessary.
So if you go on, continue the argument-- let's say your total number of clones was m-- so you can just do this argument for m. There you will show that, oh, if I followed all these rules correctly, I can show that the start time for m intersects with m minus 1 other intervals.
So there's no way I can create a scheduling with less than m clones.
Did that argument make sense, or should I go over it again?
So that's somewhat hand wavy, but that shouldn't be-- OK.
In any case, well, that's the three problems.
So I guess we can go back to this one and sort of give the motivation for this.
So this could, for example, be used in scheduling processes for servers, for instance.
So let's say your server gets a request to run n processes, and they have times like that.
So this is like shortest time first.
So you take all the short-- the smallest jobs, and you execute them in the beginning.
And you wait for other jobs.
And this can also be done online.
So you can have an online version of this.
So if you take this algorithm and you do it online-- so let's say your server is running jobs, and you get a new request.
So you get a new request, so you already have some set of t1 to tn.
And let's say, at the current moment, ti is your smallest job.
And you're running it, and you're currently at this point.
And then in the middle of running it, you can get new requests for jobs.
So how would you modify this algorithm to handle that?
So you still want to maintain this lowest average completion time thing.
So how would you handle this situation.
So let's say you're in the middle of a job and you get a bunch of new requests.
So current set is all these existing jobs plus some other things you get in here.
So would you consider switching to a different job here, or would you keep doing this?
Let's say one of the new jobs you get is really small.
So what you would do in that case is that instead of continuing with this, you would switch to current smallest job.
So you would look at the remaining time, so that's important.
So you could forget about the amount of time you already spent on this.
You know what the remaining time is, and that is all that is relevant.
So you can just consider this problem in a different framework.
It's the exact same question.
You just look at remaining time, instead of total time.
So if you're in the middle of a job, and a new one comes in which is smaller, you just switch to that, complete that, and then look at the remaining times for everything.
So at some point of time, you might have a lot of half completed jobs just lying around.
And for all of them, you'll update their ti values to remaining time rather than start time.
And that gives you a nice way to decide which processes to do online.
And that gives you-- So this is assuming that all of your tasks have equal weights.
So all of them have equal reward.
So obviously, that's not always the case.
You might be pushing back a very long job forever, because smaller things keep coming in and that might get important.
But everything is equally weighted, then this is the optimal thing you can do.
And it's a very simple strategy that works.
So those are the three problems I wanted to discuss.
Do you guys have any other questions or comments or anything?
Good?
OK. We finished pretty early, so I guess, have a great spring break.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Morning everyone.
I hope you all had a great spring break.
And during which I hope you can have more of 6.046.
Today we're going to look at network flow and two things in network flow.
The first one is Edmonds-Karp algorithm, and after that, we're going to look at two applications.
In particular, they are called bipartite matching and cover.
OK.
So we'll start with a simpler algorithm, which is Ford-Fulkerson.
Can someone remind us what Ford-Fulkerson does?
Go ahead
Every time you have a augmenting path, it gets rid of it
It gets rid of it?
Yep.
What's your name, by the way?
Michelle
Michelle.
OK.
So, Michelle gave a very brief description.
But let's be a little bit more detailed.
So we have a graph network flow G, and the first thing it's going to do is that given the flow, it will transform G from a residual graph.
Everyone remembers that?
Then F-F algorithm will find a path going from source to sink in this residual graph, and then augment this path.
Augmenting just means increasing the flow on that path.
By how much?
The minimum edge on the path [INAUDIBLE]
Yeah, the minimum edge on the path.
Let me just define that to be the capacity of the path, which is the capacity of the weakest link.
OK. Then what's the last step we're missing here?
So after augmenting, our flow changed from f to f prime.
And then we're going to look back.
We have f prime, then Gf prime, find a path in Gf prime, then continue from there.
So this algorithm, in some sense, is not even a very detailed algorithm, because it doesn't say how to find a path in the second step.
And that's indeed a problem, because we have seen a pathologically bad case where each capacity is a billion, and the correct way to do it is simply augmenting this path and then that path-- oh sorry, there's edge one-- then we'll be done.
But the pathologically bad case is we keep going through this middle edge.
OK.
So Edmonds-Karp is an improvement to this algorithm.
Let me just write it here.
Its first step, third, and fourth steps are actually exactly the same.
No change from Ford-Fulkerson.
The only thing it does is that it finds a special path.
Did Srini cover this in the lecture?
What path Edmonds-Karp finds?
I see some people nodding.
OK. Go ahead
He ran the first search from the source to the sink in Gf, take the shortest path
OK.
Does everyone agree with that?
So Edmonds-Karp finds the shortest path.
From-- I guess, can you see that part?
No?
Oh, I'm sorry about that.
Find the shortest path.
Here, shortest just means least number of hops.
Here, in this pathologically bad example, Edmonds-Karp would do much better, because it would find this path, instead of this weird path.
Because that path, its distance is 2, and the one going across the middle is 3.
OK. Now we're going to prove that this algorithm runs in order VE squared.
So OK.
So one step back.
Do we have to prove that is correct?
That it indeed finds the maximum flow?
No, you don't
Yeah.
I claim I don't have to do that, because it's the same thing as Ford-Fulkerson.
You can find any path.
Now, we want to show-- we want to bound this run time.
So any idea, high-level idea, how we're going to do that?
In which several steps?
OK.
So, OK, a simpler question.
What's the complexity of this second step, find the shortest path from source to sink?
Go ahead
OV plus
OV plus E. And in this case, actually, V is always less than E, so I can simply say O of E. Now, how long does it take to augment that path, if we have found it?
O of
O of E. That's correct.
But our claim, we can also say it's O of V, because the shortest path at most has three hops.
And then, from f to f prime-- OK.
So I think in some sense, these two are the same steps.
Also O of V, and then updating the rescue graph is also O of V. So one of these iterations-- each iteration takes how long?
O of
O of E. Right.
This is the most expensive step in one iteration.
OK.
So what else do we have to do to prove this bound?
Number of iterations
We need to show that the number of iterations is O of V times E. If we can show that, then we are pretty much done.
OK?
OK. Now, to prove that bound, we first need a lemma, which we call monotonic.
So let me define delta f v. This is the length of the shortest path from s, from the source, to v, which is a node in a graph, the length of that in rescue graph of f. Is that definition clear?
And this lemma says delta f v, for any vertex v, does not decrease.
So it can only monotonically increase.
OK. We are going to prove that.
So we'll prove by contradiction.
Assume this is not a case.
That means there is some v that in f prime, which is the new residual graph, its shortest path from the source in the new residual graph is less than the old one.
Right?
We are going to derive a contradiction to that.
So there may be many such vertices that drop.
But if there's any, I'm going to define v to be the one with smallest delta in f prime.
OK. Any questions about this step?
So there could be like v1, v2, v3-- that all make this happen.
I'm going to choose the one among those that has the smallest distance from the source.
OK?
Now here is my source.
We have a path to v in graph Gf prime.
OK.
So this is a path.
Then I can always find a predecessor, which I'll call u here.
If this is the shortest path to v, what can I say about delta of v and delta of u?
What can I say about these two quantities?
Can v decrease?
Can v decrease one of them, at least?
This one is greater than that?
One of them must be--
One of them is greater than the other, and that's always true, right?
For any two?
v is u plus 1
OK. Michelle said v is u plus 1.
Does that make sense to everyone?
That's the shortest path to v. It goes through u. OK. That's correct.
So OK, maybe I wasn't clear about that.
I defined u to be v's predecessor on the shortest path.
OK?
Then definitely delta f prime of v is delta f prime u plus 1.
Now, I'll use u to be my step stone back to the original graph, f. I want to say something about this quantity and delta u in graph f. What can I say about that?
This is a tricky part
It's greater than v?
This one is greater than that?
Or the entire thing is greater than that?
OK.
I'm going to claim this one is greater than delta f u plus 1.
By which I'm claiming this quantity is larger than that.
Can anyone give a reason why I can claim that?
So recall how I defined v
Can you repeat how you defined v?
Good question.
So v-- so there are several, probably several v's that have delta f prime less than delta f. I'm going to define v to be the one with smallest delta f. Among all those nodes that have a job in delta f, in delta.
This is probably a tricky part.
Then by definition, since delta f u is less than delta f v, right, and I defined v to be the one with the smallest delta that satisfies that.
So all the u's-- so u is a predecessor of v, so u shouldn't be one of those nodes that have a drop in delta.
So I know this is probably a tricky part.
Yeah, I'll stop for questions and make sure we resolve this part before we move on.
How many people get it?
OK. Only two.
That's not good
I'm confused about how v can be the one with the smallest delta f if you have a predecessor with a smaller delta f
OK.
So v to be the smallest-- the one with smallest f such that delta f prime is less than delta f

OK, maybe, yeah, that's why I confused you guys.
Yeah.
Sorry about that.
So we have a bunch of nodes who have a drop in delta, and I defined v to be the one among them that has the smallest delta f prime.
Question?
Sorry, I'm lost at what delta f prime is versus delta
OK. Delta f of a node is the shortest path from source to that node in G of f, which is the residual graph given a flow.
So f is, well, some flow, and f prime is the flow after we augmenting a certain path.
So f prime is one step after f. OK. How many people get that now?
Still not everyone.
OK. Any questions about that?
How many people still haven't got that?
OK, so some people-- it's like Schrodinger's cat.
It's in the middle state.
Well, then I'll have to move on, and I'll assume you all get that one.
As the last step, so of course we will ask, what's the relation between this guy and delta f of v?
Because in the end, we want to show a contradiction that delta f v is probably greater or equal to delta f of v. OK.
This is another tricky part.
So let me ask maybe a simpler question now.
So that is our G of f prime.
So we also have G of f. It has the source, some u here, v there, sink there.
Does this edge exist in this graph?
So I know if that edge exists, because I defined u to be the predecessor of v. But u, v is in that graph doesn't necessarily mean u, v is in the old graph.
It could certainly exist, but is it possible that this edge didn't exist?
How many people would say that edge definitely exists?
How many people would say maybe it doesn't exist?
OK.
It doesn't matter, because we can prove both cases.
So let's say case one, u, v is indeed in the original graph, G of f. OK.
In that case, can I say something about that last step?
It's greater or equal to delta f v
Greater or equal to delta v. OK. Why is that?
Because v was on the shortest path.
Well-- yeah.
So the shortest path, either passed through-- yeah.
The shortest path to v either passed through u or didn't
Correct.
Does everyone get that?
So the shortest path in this graph to v is not necessarily this one, right?
But if it's some other one-- OK, so, in case one I'm assuming this edge exists, right?
Then no matter what the shortest path is, it's definitely shorter than I first go to-- from s to u and then u to v. Right?
The shortest path between s to v is definitely shorter than I first go from s to u and then u to v. Make sense?
OK.
So case one, then, is a contradiction, because we showed that delta f prime of v is greater or equal to delta f of v. Any questions about that?
OK. Case two, u, v is not in Gf.
OK, so, how can that even happen?
Is it possible that this edge didn't exist?
It could be a backwards-facing edge that you add [INAUDIBLE]
Great point.
That's exactly right.
So it is possible that this edge didn't exist, but only appeared after we go from f to f prime.
How can that happen?
That must mean we are augmenting a path that goes right through it.
Right?
But this edge doesn't exist, so we cannot be augmenting a path that crosses this way.
We must be augmenting a path that goes like that.
First from s to v, and then v to u, and then u to t. If we are augmenting such a path, then we're going to remove this edge from v to u, but we will add u to v to the next graph.
OK?
Did everyone get that?
So now, here-- well, formally I'm going to claim if we assume u, v is not in G of f, and we also know u, v is in Gf prime.
What does that mean?
These two can only be caused by the fact that u, v is in G of f. Not only in that graph-- it must be on the path we are augmenting.
Make sense?
OK.
So given that, can we say something about delta f of u and delta f of v?
So here is our p. Right?
This entire thing here.
And p is the augmenting path
And f u is just f v plus 1
Correct.
Right.
So if that is the augmenting path in Edmonds-Karp, we are looking for the shortest path.
Right?
So v is the predecessor of u, then delta of u is delta of v plus 1.
That means that quantity, here, is delta f of v plus 2.
OK?
So it's also a contradiction to what we assumed.
OK.
So that proves our lemma of every node's delta monotonically increases.
Now here, I'm going to show that-- our final theorem, that we have at most VE number of iterations.
So the way we're going to show that is we are going to define a capacity of the path.
It must be the capacity of its weakest link.
We're going to define its weakest link is u, v. OK?
And we're going to show that u, v-- we'll call that critical edge-- we're going to show u, v can be critical only O of V times.
If that holds for every edge, then I claim all the edges combined can only be augmented O of VE times.
Now, how do we show that?
So again, let's assume we are augmenting a path that goes through u to v. If we do that, by our algorithm, we will get rid of this path, and have an edge backwards.
We go from f to some other f prime.
Now, when can I augment u, v again?
It can only happen if at some point, I come back to augment the path going from v to u, because that will eliminate this back edge, and and adds back our edge u, v. OK. Let's see what happens there.
Now I'll call this graph f, this graph f prime.
So in f prime, we know we are augmenting the path from v to u.
Right.
The same argument, delta u, is delta v plus 1 in f prime.
Correct?
And we know this one doesn't drop, right?
So it's greater or equal than delta f of v plus 1.
OK?
And we also know in f, v is the predecessor of u.
So this one is equal to delta f of u plus 2.
What does that mean?
It means in between two times u, v is augmented, delta of u must increase by at least 2.
Then how large can delta of a certain node possibly be?
It's definitely bounded by v. So then I claim this particular edge, u, v, can only be involved, can only be critical edge, O of V times.
Actually, half of V times.
Strictly.
So then, every edge combined, there can only be half of V times E number of augmentations.
OK. Any questions about that part?
About the entire proof?
If not, we'll move on.
OK.
This is not part of the required in the recitation, but I'll quickly say a few words about an even better algorithm, Dinic, which was in O of V squared times E. So this is an improvement to Edmonds-Karp, and its idea is that I'm going to find all the shortest paths in one go from s to t. I'm going to augment all of them at the same time.
Then, because they are all shortest paths, when I augment that, each path will be broken.
Maybe I'll get rid of this edge, this edge, and that edge.
And the thesis is that the shortest path will increase.
So here, the shortest path is 3, but I have destroyed all the shortest path of length 3, so I'm going to shortest path of 4.
If they do that, you can bound the number of iterations to O of V, because your shortest path can be, at most, V. But each iteration is slightly more complicated, because you need to find all the shortest paths, and it happens that they can show-- you can find it in V times E. That gives the V square of E. That's not the important part.
The more interesting part is that, actually, the author of this algorithm, his name is Dinitz.
But his algorithm is very hard to understand.
Yeah, nobody got that.
And there's some other guy whose name I think is Even.
He understood the problem and started advertising to people and giving lectures on Dinitz's algorithm.
So he popularized the algorithm, but unfortunately he got the name of the author wrong.
So that's why this algorithm is henceforth called Dinic's algorithm.
This is useful to know.
Why?
Because you can tell this story to other people so they will assume that you know a lot about Dinic's algorithm, while in fact, you probably don't.
And that's exactly what I'm doing here.
OK. Now let's look at one application.
Bipartite matching.
OK.
So the problem is that we have several person and several tasks.
And let me get rid of one person, because I don't want to draw that many stuff.
So we have a graph like that.
And each edge-- if there is an edge, that means this person is capable of performing that task.
And the problem is that-- find the matching, which means the assignment from people to tasks, such that we get as many tasks done as possible.
OK?
So one person can only do one task, and one task only needs one person.
So here you can see a bad matching, which is I assign this first guy to task one, then no one can do task two.
So if I'm smarter, I'll assign one of these guys to that task and have this person handle the other task.
Is the problem clear?
So this is called bipartite graph, because-- well, which means you can partition a graph into two parts, and within each part, there's no edge connecting any pair of vertices.
And you can also define this problem for a general graph.
The goal is the same.
Find a subset of edges such that no two edges are connected to the same vertex.
But we are going to look at bipartite graph.
And we claim in bipartite graph, this can be solved using max flow.
OK.
I'll give you one minute to think about that, how to transform that problem to a max flow problem.
Any ideas?
OK. A hint.
I will add the source here, and the sink there, and I'll have created these edges.
OK. How do I restrict the capacity of all the edges such that I can guarantee no person is taking two tasks, and no task is assigned to two people?
Make all of them weight 1
Make all of them weight 1.
OK. You're right.
We do that-- 1, 1, 1, 1, 1, and everything here is weight 1, then that's definitely correct.
What's your name?
Calvin
Calvin?
OK.
If we do that, then, well, because each person only has one incoming edge-- oh, sorry, maybe I should have drawn the arrows here.
Each person only has one incoming edge, so it cannot take care of two tasks at the same time.
Right?
Same thing for each task.
It only has one outgoing edge, so it cannot be taken care of by multiple people.
OK.
So if we find-- if the max flow is k, that means we can find the max matching is also k. We can get k tasks done.
It's very easy to see because we can have a cut here, and if the max flow is k, that clearly k tasks are taken care of.
OK. Any questions about that?
So, the question is can we do all the assignment [INAUDIBLE]?
Find an assignment from people to tasks to get as many tasks done as possible.
So how many tasks can we handle?
OK.
So I'm doing a different topic from the recitation notes.
In the recitation notes, we are considering another problem called bipartite cover, which is exactly the same thing as that.
So cover, also, bipartite graph, is defined too as, let's find several vertices in this graph such that each node, at least-- OK, let me color several nodes in this graph-- such that each edge is connected to at least one dark node.
OK. And this, the nodes I colored, is called a cover.
I want to find a minimum cover.
You can of course cover all the nodes, that trivially holds, but we are looking for the minimum cover.
And the claim is that min cover is k if and only if max flow-- sorry, max matching-- is k. Why is that?
Because in that matching-- if we have a matching, then we have a set of disjointed edges, right?
No two edges are connected to the same node.
So if we want to cover these edges, we at least have to add one of these guys-- color one of these two, and one of these two.
Right?
So if we have k matching edges, then we at least need k nodes to cover them.
For example, I can cover them like that.
But this is not foolproof, because I also need to show I can indeed find a cover that is k. OK. Let me think about whether I should prove that.
OK.
I'm going to give it a try.
If you don't get that, that's fine
Can you say again what cover is?
OK. Cover--
[INAUDIBLE]?
Cover is a set of nodes such that every edge in the graph is connected to at least one of the nodes, in a cover.
OK?
So this is not a cover, because this edge is not covered.
So in this case I probably have to add that as well.
But clearly there's a better cover, even a smaller cover, which is those two.
Right?
Yeah.
So how are we going to transform that matching to cover?
So let me first give a matching here.
One max matching should be something like this.
I have this edge and one of those two.
That is a matching.
Right?
That is a max matching.
No questions about that?
Let me get rid of all this.
One way to transform this into a cover is that I will first color these two guys.
I colored the nodes on the left where they are connected to a matching edge.
After that, I'm going to start from this one and jump between nodes, taking an alternating path.
Meaning I'll take an unmatched edge, and then take a matched edge, and then take an unmatched edge again, but there's no such thing.
OK. And if I take this jump, I will swap them, make that dark.
OK.
This graph is a little strange.
Ah, OK. Now I claim this is a cover.
Is that?
Yeah, that is one, right?
And if there is another unmatched edge going out, then I'll keep taking that alternating path.
But I have to stop, because there's no such edge anymore.
How can I say-- why I can prove-- why can I claim this is a cover?
So clearly if this is a cover, then it's a cover of size k, right?
So we have proved the entire thing.
Then we're going to consider several things separately.
So I'm going to first claim if I have a matched edge, then it's definitely covered, because one of its end points is dark.
Right.
We only do a switching between dark and white if it's a matched edge itself.
Right.
So I only need to show that this thing doesn't happen.
There's an edge that's unmatched, and two of its endpoints are both white.
OK.
I'm going to first claim this node cannot have a matched edge.
Because if it does, then this is an alternating path, and I'm going to switch this too.
OK. Make sense?
OK.
So this doesn't exist.
So it can only have an unmatched edge.
Now what can I say about this node?
I claim this node needs at least one matched edge, because if it doesn't, right, then I should add this guy into my matching.
It doesn't violate any constraint.
I didn't add that because there's probably a matched edge connecting to this guy.
Then I'm going to-- so, then this guy has to be dark, because this one is not.
Then how did this one become dark?
It must have come from some other guy on the left, right?
So there's an alternating path going back and forth.
Something like this.
There is an alternating path, starting from left, but ends here.
Because we showed that this guy is not connecting to any matched edges.
In that case, what I'm going to do is match this edge, match this edge, match this edge, then unmatch these two edges.
And that's a larger matching.
Because I removed the two, I added three.
That means a max matching of k leads to a minimum cover of k as well, which also means max flow of k in our network flow.
So this part-- this equivalence is not required.
You should know this proof, the proof of Edmonds-Karp, and know that a matching and cover can be solved by network flow, and that Dinitz's name is spelled not as Dinic.
OK. And that's everything for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's start.
So today we're going to do some NP hardness reductions.
So let's just do a quick recap of P and NP.
So let's see.
So P is-- so you have a decision problem.
So I have an input x and you have some algorithm, A, and it spits out an answer, which is either 0 or 1.
So that's a decision problem.
And if a problem isn't P, this algorithm A runs in polynomial time.
So what about NP?
So NP is when the solution is verifiable in polynomial time.
So let's say you have an input x and some oracle, which is [INAUDIBLE] run exponential time or is has infinite computation time gives you an answer, so you get x and you get an answer which is either 0 or 1.
And you also get a certificate.
So let's call that a certificate.
And given these three values, you can verify whether the solution was correct or not in polynomial time.
Make sense?
So clearly if you're going to compute the answer in polynomial time, you can also verify in polynomial time.
So P is a subset of NP.
And so that's it.
So now, NP hard problems are problems that are at least as hard to solve as any problem in NP.
And so now today we're going to be doing some reductions.
So I think we did some of them in class, Nintendo games or something.
Is that what we did?
So today we are going to do some less interesting examples, but let's see.
So how do we do reductions?
So if we know that we have a problem A, which is a hard problem, and we want to show that problem B is hard.
So we want to draw this implication.
So if we want to draw this implication-- so this is equivalent to saying that if B is easy, then A is easy.
Sorry, other way.
This is the counter positive.
So assuming that B has a polynomial time solution, A has a polynomial time solution.
And that statement is equivalent to saying if A is hard, B is hard.
So if you know that A is an NP hard problem, we can say that B is an NP hard problem.
So if we want to show that B is an NP hard problem, first we show that B is an NP, and then we show that B is hard.
So the way do this step is-- so your A looks like this.
You have your algorithm, and it spits out an answer in 0 to 1.
And B looks like this.
Let's say you have an input y and it spits out an answer in 0 to 1.
And you want to find a function R which takes your x and sends it to y.
So basically, if you know how to solve B very fast, then you can take an input to A.
So you take x, you transform it, and then you apply B.
And the condition is that A applied to x is the same as B applied to R of x.
So basically, what you are doing is you're showing that A is easy by showing that you can use-- so let's say B is easy.
We can use B to compute A, so then A must be easy.
But since we know that A is hard, that there's something wrong in our logic, so B must be hard.
Does that makes sense?
Yes?
Sort of?
So let's move onto an actual problem.
So the first problem we're going to reduce is the Hamiltonian path.
So a well-known NP hard problem is the Hamiltonian cycle.
So here our A is-- so it's a Hamiltonian cycle.
So what's a Hamiltonian cycle?
So what's a Hamiltonian cycle?
So a Hamiltonian cycle-- so let's say you have a graph.
So we have this graph.
Let me draw this out.
That's it.
So a Hamiltonian cycle is a cycle in the graph which starts at some vertex, visits all the other vertices, and comes back to the starting vertex.
So in this case, we could do something like go here, and then take this vertex, take this vertex, take this vertex, and come back here.
So that is a valid Hamiltonian cycle.
So this graph is a Hamiltonian cycle.
So the decision problem is here that given the graph, does it have a Hamiltonian cycle?
And that problem is NP-hard, so you can [INAUDIBLE] polynomial [INAUDIBLE].
So now the new polynomial shows NP-hard, which is B is Hamiltonian path.
So the Hamiltonian path is a very similar problem.
Instead of a cycle, you remove the requirement that you have to come back to the starting point.
You can just start anywhere and [INAUDIBLE] all the vertices and stop.
So for example, if you remove this edge, this graph no longer has Hamiltonian cycle, but it has a Hamiltonian path, which is just this line.
Simple.
So this is a simple reduction because the problems are very similar.
So the first step is, of course, showing that Hamiltonian path is an NP.
So that should be pretty clear because-- so what is our certificate here?
So if someone says, OK, I have solved the Hamiltonian path and this is my Hamiltonian path.
And he gives you a certificate which is the actual path.
So you can always look at the certificate, check the path, and see if it's a valid path.
And then you can verify the answer in polynomial time.
So that's the linear time verification.
So this is true.
So now what we're going to show is B is hard.
So the way we do that is we do a reduction.
So the reduction is given an input to the original problem, A.
So let's say an input to A is in the form of a graph.
And now you have transformed this graph somehow to G dash.
And then you have to argue that-- so this is the transformation R-- and you have to argue that the solution, the answer that this will spit out, is the same amount that this would spit out.
So let's look at the transformation first.
Anyone have any ideas?
So you have a graph and you're using that to solve the Hamiltonian cycle problem.
So how do you transform it in a way that lets you cast into Hamiltonian path formulation?
Yes
Remove an edge?
Not exactly.
So which edge would you remove?
You can't find the Hamiltonian cycle.
OK, important point.
So there's no point doing this reduction unless this reduction is itself polynomial because otherwise, this whole strategy of transforming and then using B to find A doesn't work.
Because if the reduction is exponential time, that doesn't help you.
So it--
Try removing every edge
So if you remove some edges, you'll see that there is still a Hamiltonian cycle.
But you remove some edges-- you can't tell.
You don't know which edge to remove.
So a better way to do it is this.
So let's say this is the rest of your graph.
And you just look at one vertex.
So look at one vertex, V. And let's say this is a directed graph.
If it's an undirected graph, you can just like add one edge there and one edge back for everything.
So now you add a directed edge along this.
I'm sorry.
You look at all the directed edges.
So let's say you have some edges coming in and you have some edges going out.
So this is just a vertex and you just look at the rest of the graph and look at all the edges coming in and all the edges going out.
So this is in A.
This is the original problem.
And you transform this into-- you split the vertex into two, so V dash and V double-dash, let's say.
And in one of them, you keep all the incoming edges and the other one contains all the outgoing edges.
Does that transformation make sense intuitively?
So what do you have here?
So here you had-- let's say this graph had a Hamiltonian cycle.
So this graph had some cycle which went up here, did something, something, and came back.
So it would go like this.
It would do the cycle and come back.
So there was some cycle.
And since the cycle, it contains V, so now what you're doing is you're splitting apart V and disconnecting them.
So you'll still have-- if you look at the original path, it's still there, but it's been split up into a path now.
It's no longer a cycle.
Make sense?
So now let's argue this more rigorously.
So what we want to say here is that let's say there was a cycle here.
If there was a cycle here, then is it clear that there is a path here?
Because just take the same edges that you had before.
If you take the same edges, they will now form a path instead of a cycle.
So cycle implies path.
Does that makes sense?
So the other way is a little more tricky.
So let's say you have a path.
So let's say you had a path.
So that means that-- let's redraw this so it's more clear.
So you have this new graph where you have two vertices, V dash and V double-dash.
This has a bunch of incoming edges and this is a bunch of outgoing edges.
So now let's say you have a Hamiltonian path in this graph.
So what does that mean?
So where can the Hamiltonian path start?
Can it start anywhere?
Where can it start?
V double-dash
Right, because V double point doesn't have any incoming edges, so it can't be in the middle of the path.
So it has a start here.
So it starts.
It does something in there.
And where can it end?
It can only end, similarly, in V dash.
Because V dash doesn't have any outgoing edges, so it can't be in the middle of the path.
So it has to end in V dash.
So now, if you have a path like that, and you go back to this graph-- so V dash and V double-dash are now on the same vertex.
And just that path just becomes a cycle now.
So path implies cycle.
So now what we have is previously what we had, right?
So now we know that A of G here is equal to B of G dash.
So G dash was transformation.
Also notice that the transformation was just splitting apart a vertex.
So depending on your representation of your graph, it'll take something, like constant time or linear time or something polynomial, essentially.
So you get a polynomial-time reduction.
After reduction, you show that the answer to the reduced problem is the same as the answer to the original problem.
And that means that this is also an NP-hard problem by the argument given here.
Questions?
Does that make sense?
Yes
So are you creating two vertices for every vertex in the graph?
No, just this one.
Just pick any vertex-- doesn't matter-- because you have a cycle.
So if you take any vertex and split it apart, you get a path
Oh, perfect
Anything else?
OK, let's move on to the next one.
Grab a new board.
So the next problem is-- so given the graph, is there a k-clique?
Do people know what a clique is?
So a clique is this.
So a clique is a set of vertices.
So let's say C subset of V for the set of vertices, such that C is a complete graph.
Let's just draw a diagonal.
That's probably easier.
OK.
So in this example, so you have this graph.
This one.
So look at this set of vertices.
Every pair of them is connected to each other.
What that means is that if you just look at the graph with these vertices, it's a complete graph.
That's what's called a clique.
And in this case, this is a 4-clique.
So you have four vertices, this is a 4-clique.
So the position problem is, given the graph, does there exist a k-clique?
So this is, again, known to be an NP-hard problem.
So now we will use this to show that this problem is NP-hard.
So this problem is independent set.
So again, so given the graph, what is an independent set?
Anyone want to explain?
So what an independent set is this.
So let's say you have a graph which looks like-- so kind of complementary to the definition of a clique.
An independent set is a set of vertices, such that no pair of them has an edge between them.
So in this case, if you took this vertex, you took this vertex, this vertex, and this vertex-- so you can see, none of these vertices have an edge between them, so that is an independent set.
So in this case, you're taking a set of vertices which is a complete graph, so all of them have edges between them.
And in this case, you're taking vertices which are completely disconnected.
So now we're going to find a reduction from this problem to this problem.
So first of all, independent set is an NP.
Is that clear?
How would you show that?
So how would you create a certificate which would tell you that-- so how would someone create a certificate which would convince you, in polynomial-time, that this is correct?
So the certificate would be just give you the independent set.
And you can check if something is independent set.
So given I-- let's call the independent set, I.
So given the set of vertices, you can verify that it is an independent set in polynomial-time.
So just look at all pairs and check if there's an edge-- just an n squared and Q whatever is polynomial.
That's important.
So now let's look at our transformation.
So again, as before, you have A, which is given by a graph, and you want to transform into something.
So the important note here is that in the clique, you have-- so for your clique C, all the pairs of vertices are connected.
In I, no pair of vertices are connected.
So what should be a logical transformation that would map a clique to an independent set?
Anyone?
What do you think you should do to the graph so that the clique becomes-- yeah
Invert the existence of edges so they're [INAUDIBLE]
Precisely
[INAUDIBLE]
Yup.
Exactly.
Great.
So all you have to do is if you want to turn a clique into independent set, you just-- what is it-- complement the adjacency matrix.
So every edge does not exist now exists and every edge that did exist is gone.
So you create a graph, G dash, with the same [INAUDIBLE] vertices, except the edges are now complemented.
So the E bar just means the edges that were not.
So let's just draw an example.
So let's say you had-- and what does this become?
So let's draw the vertices first.
Let's go one by one.
So let's take this vertex.
Let's take this vertex.
And what edges does it have?
So it goes here.
It goes here.
It doesn't go here, so we draw an edge here.
It doesn't go there, so we draw another edge there.
And it doesn't go there, so we draw another edge there.
Now, this vertex.
It's connected to all of these except this one.
So that means that there's an edge there.
Let's take this vertex.
It's connected to everything.
Oh, it's connected to everything.
Let's take this one.
It's connected to these two and nothing else, so we need to connect it to this guy.
And I think that's it.
Yeah.
Similarly proceeding, this is connected to everything except that, so I guess this goes there.
And I think that's it, right?
Or is there more?
Three.
No, OK.
So that is a complementary graph I think.
So you can probably verify that.
So now let's look at the clique in this graph.
So this is the clique.
This is the largest clique, rather, but this is a clique.
You could have other cliques, like, for example, this these three things are also a clique.
So now look at what this is mapping to.
This is mapping to this vertex, this vertex, this vertex, and this vertex.
And you can see that's an independent set.
So does that transformation make sense?
So now the proof should be intuitively clear.
So let's just go through it moderately rigorously.
So let's say you have a clique here.
So your clique here, for every pair of vertices in the clique, there's an edge between them.
And so if that maps to-- so let's say clique C maps to-- let's say this maps to I.
So for all U, V element of C, you have U, V, element of E. So if this is a clique, for every pair of vertices, that edge is in the original graph, which means that U, V is not an element of E for all U, V element of I.
So for every U, V element of I, we have this, which means that-- does that make sense?
So that means that that's the independent set criterion.
So you reduced clique to independent set.
And that means that independent set is now NP-hard.
OK. How are we doing on time?
OK.
So now let's do a more complicated example.
Any questions on these two?
Make sense?
OK.
So let's try this.
So let's start by erasing something.
So this is the next problem.
So as before, our A is k-clique.
So it says there's a clique-- rather, let's add this in this way.
Clique size greater than or equal to k. So that's the decision problem.
Is there a clique of size greater than or equal to k?
And B is, it's called Max-2-SAT.
So what that means is that-- so it is somewhat like normal 2-SAT, except basically you have some clauses and you have some literals.
So each of these literals contain values 1 or 0.
And each of these clauses is something like xi or xj.
And actually, there can be naughts in front of this, so let's say xi 0 xj.
Or it can be other things.
So it's xi 0 xj, or 0 xi 0 xj, or xi xj, and so on and so forth, so just the normal 2-SAT.
So now the decision problem is to-- does there exist an assignment, such that greater than equal to k clauses-- so that the decision problem.
So is there an assignment to literals such that at least k of these clauses are satisfied?
And so now we're going to show that it is with k-clique.
So again, is it an NP?
So what is the certificate?
How would someone convince you their solution to this problem?
Give you the literals
Yeah, so give you an assignment of values to literals.
And then you can go through all the clauses and check them.
So if they give you like x1 equal to 1, x2 equal to 2, x equal to 0, x3 equal to 1, and so on and so forth, you can then go through and check all the clauses and see if greater than k are satisfied.
So this is an NP.
So now let's try the reduction.
So this is how the reduction goes.
Let's see.
So naturally, we have k-clique, or greater than equal to k-clique, rather.
So that means you have a graph with a set of vertices and a set of edges.
So let's say you have a clique, V dash, subset of V, and mod of V dash is greater than equal to k. So now you have to somehow construct literals, construct clauses, which will reflect this behavior.
So first of all, this may not be clear right now, but let's say we take some literals like this.
So let's say we take xi for all i element of V. So for every vertex in the graph, we take a literal.
So if the number of vertices is n, we have n literals because it's corresponding to each vertex.
Also, we take a dummy literal.
Let's call it Z.
So now how do we get our clauses?
So the general idea is that if a vertex is in the clique, you will assign it 1.
If it's not in the clique, we will assign it 0.
Everything outside of the clique is 0.
So this clause, not xi or not xj-- so what is the value of this clause normally?
So let's say xi and xj are both outside the clique.
So i and j are both outside the clique.
That means that both of them are 0 and so this is true.
What if one of them is inside the clique and the other one is outside?
It still is true because one of these naughts is 1 and that is still true.
Let's say both i and j are inside the clique.
So in that case, you have 0 of xi is 0 and 0 of xj is also 0.
That's the only case that this is false.
So the way we take care of-- so we're trying to maximize the number of true clauses in some sense.
It's like you had maybe an explanation of why you're using this clause.
So what we do instead of taking all the xi, xj pairs, is we just take xi, xj, such that i, j is not an element of E. So what does that mean?
So now if you had the graph which looked like this, now it looks like 1, 2, 3, 4.
So you would take 0 x1 and 0 x4.
You would take 0 x1 and 0 x3.
But you would not take 0 x2 and 0 x3.
So what does that do?
That means that if you follow the assignment according to the clique rules-- so if i and j are both in the clique, this clause will not be included.
Does that make sense-- what set of clauses you are taking?
OK, so let's continue and it will hopefully be a little more clear.
So the other sort of clause we're going to take is xi or z.
And the other one is xi or 0 z.
So the reason we're taking these is that if you wanted to Max-2-SAT on this alone, you can just set everything to 0, and that would give you a maximum.
So sort of not do that-- so to sort of minimize the number of things to be set to 0, you were doing this.
So is this just some hand-wavy argument why you're doing this.
So let's actually try to do some analysis on this.
So let's say you do this transformation.
So do the clauses make sense?
Does the first clause sense?
So you have 0 xi, 0 xj for every i, j which is not in the graph.
So how does this work?
So let's say you have V dash such that size of V dash is greater than or equal to k. Actually, let's just make size is V dash equal to k. So if you have a clique of size greater than or equal to k, of course, you have a clique of size equal to k. You can just throw away some of the vertices.
So you take your V dash such that this size is equal to k. And you set xi is equal to 1.
So you set xi equal to 1 if i is element of V dash 0, if i is not an element of V dash.
Make sense?
So you would set everything in your clique to be 1, everything outside to be 0.
And let z equal to 1.
So you're starting with the assumption that-- so you're showing one direction.
You're showing that given that there's a clique of size greater than equal to k. And now you're are going to construct a Max-2-SAT instance which has the satisfied number of clauses greater than equal to k. And then we'd show the other direction.
So now let's look at how many clauses we have to be satisfied.
So the first type of clause was 0 of xi or 0 of xj.
So how many of these clauses are being satisfied?
So first case, i and G are both outside V dash.
Is the clause satisfied in that case?
Yes, because by definition, if they're outside V dash, they're both 0, so their naughts are both 1, so they're 1.
So if the i and j are outside V dash, you're good.
So what about the case when one of them is inside V dash?
Is the clause satisfied.
Yeah, because one of them is 0, which makes the 0 1, and the whole thing is.
It's an [?
arc, ?]
so it's satisfied.
Let's say both of them are inside V dash.
Let's say both i and j are inside V dash.
Then this clause just doesn't exist because of the condition that i, j had 0 elements of V. Because if it's inside the clique, then that edge obviously exists, and therefore this clause is not in the set of clauses we're using.
So essentially, every clause of this form will be satisfied.
And how many clauses of this form are there?
The number of clauses of this form is just E bar, where E is the complementary edge set of that graph.
That's E bar.
Does that make sense?
Next clause is xi or z.
So since we have considered z to be 1, this clause is always satisfied.
So this just gives us mod of V because this is for every i-- I should mention that.
For i, this is also for all i.
So for every i, so they all have a number of xi's are V, so that's it.
So the third type of clause is xi or 0 of v. So 0 of z, since z is 1, 0 of x is 0.
So the only cases where this clause is true is where?
Is when xi is 1.
And xi is one only inside V dash, so the number of clauses satisfied here is mod of V dash.
And mod of V dash is what?
It's just k. You see this?
All three clauses make sense?
Can you see why the first one is it's the size of V bar?
Because all the clauses are satisfied.
The second one, also all the clauses are satisfied because z is 1.
Every clause is just the number of vertices.
And the last one, it's only satisfied for the cases where xi is 1.
And xi is 1 only inside the clique, so that gives you the k things in the clique.
So now we can finish formulating our transformation.
So our transformation was you set-- where was the transformation?
Yes.
So these are our clauses and now we're going to set the k. So the Max-2-SAT was with the condition of k. So here you'll set it to be mod of V bar plus mod of V plus k. So does that make sense?
So basically, our Max-2-SAT problem-- so the reduction to the Max-2-SAT problem-- so we specified the clauses.
We specified the literals.
Literals were xi for all iV and the dummy z.
We satisfied the clauses and we specified the threshold we are trying to achieve.
So those three things completely specify the Max-2-SAT problem.
OK, proceeding.
So we just showed one direction.
We showed that if there's a clique of size greater than or equal to k, we reduce this to a size of exactly k. Then we did this assignment and we showed that the total number of clauses being satisfied is mod of V bar plus mod of V plus k. And that is the first direction.
So now we're showing that-- so if you have a solution to clique, we have a solution to k-SAT, Max-k-SAT.
So now we do the other direction.
So let's say we have a solution to Max-k-SAT.
Let me get this out of the way.
Let's do this.
So let's say-- so this part is a little more tricky.
So we start with Max-k-SAT has a number of satisfied clauses which is greater than or equal to mod of V bar plus V plus k. So we know that max-SAT that accepts.
So know that so we define.
We define a V dash.
This V dash is in the graph for clique as the set of vertices i, such that x of i is equal to 1.
So if our Max-k-SAT has that many things, then there's some assignment to xi which satisfies that.
And we take that assignment.
And for every value that is 1, we put that in a-- it's not a clique yet, but it's, well, it's a clique under construction.
So we make this clique under construction and we assign it values of all the vertices which are currently labeled by 1.
But we don't know that it's a clique, right?
It could not be a clique.
It could be something like, let's say, so this is the current V dash.
So V dash could be something like you have all these vertices and let's say some of them are connected.
But then you have this dangling.
So this is not connected.
This is not connected.
So it's not a clique.
So let's say this is vertex i and this is vertex j, and this whole thing is V dash.
And let's say somehow you have this anomaly.
You have that this guy is not connected to all the vertices.
So let's take one such pair.
And let's just say we remove xi.
So we just take the xi and remove it from V dash.
What that equates to in the Max-k-SAT is setting xi to 0.
So we take the original satisfying assignment and change the value of xi.
So let's see what that does.
So we take and set xi equal to 0.
And xi was originally 1.
Let's actually write it down.
So xi, or rather, i, j is not an element of E, but-- so these i, j were in the supposed to be clique, but the i, j is not in the edge set, so it's not actually a clique.
So the way we resolve that is just we do this.
We say, OK. Let's just forget about this vertex.
Let's say it's just not in the clique.
So we set our xi to 0.
And what does that do?
Let's look at how that affects the number of sat clauses.
So the first one is x [?
over ?]
z.
So now here we had not set z to be 1.
This is something in thing.
If z is equal to 0, we can just replace z by 0 of z.
So the clauses are symmetrical with respect to z.
So it's xi [?
over ?]
z or xi 0 z.
So it doesn't matter which one is set to be 1 or 0.
If one of them is 1, one of them is 0.
So let's just say that z is 1 without loss of generality.
So what that does is-- so initially, this clause was 1 because z is 1.
And now it goes to 1 and nothing changes.
OK, sounds good.
What about the next one?
So now we have xi or 0 of z.
So 0 of z is 0 and xi just went from being 1 to being 0.
So what happens here is that initially, the clause value was 1 and now it goes to 0.
So that's not good because now we are no longer satisfying Max-k-sat clause possibly because we had some number of satisfying clauses, which was above our threshold.
But now we lose a clause and we could be going below the threshold.
But then we look at the third clause.
And what that does is-- so let's say we look at this specific clause, the one which said that xi and xj.
Note that this clause exists because xi, xj is not an edge you need.
And therefore, by that condition, this clause exists in the set of clauses.
So what was the value of this clause initially before?
In the initial assignment, what was the value of this clause?
Note that i and j are both in V dash.
What was the value of this clause in the original assignment?
Before we set xi to 0?
What was the value of xi in the original assignment?
1
Yes, why is it 1?
Because xi was in V dash
Yeah, so xi was in V dash, so it's 1. xj was also in V dash because that's the anomaly we saw it, right?
There was not an edge in the clique.
So this was originally so this was 1 and this was 1.
So this was 9 and this was 0.
And our R0, R0 is-- this used to be 0.
So what happens now though?
Does it change?
It changes to 1 because xi goes to 0.
Now this thing becomes 1 and so it changed to 1.
So there will be other clauses with xi, but realize that 0 of xi-- so if xi is changed to 0, 0 of xi is changing to 1.
So whatever happens here, it will only increase the number of clauses that are being satisfied.
So you lose only 1, but you gain at least 1.
So eventually, it does not change.
It does not change the number of satisfied clauses, so that's important.
So what we did is we started with some satisfying assignment that we assumed existed.
And then we just changed the variable and we said that it's still at least as many clauses being satisfied.
So we did that.
And now we have one less vertex that is violating-- so now we have one less vertex that is violating the clique property.
So now we can take this step-- this setting xi to 0.
We can just repeat this.
So how long do we repeat this?
So we repeat this till there are no longer any violations.
So every time we find a violation-- so once we delete xi-- so let's this is gone.
So now we look for the next violation.
The next violation is this edge.
So there's no edge there.
So we can either take this vertex-- so let's call this k. So we can either remove xk or remove xj.
So we remove one of them.
And once you remove that, you will end up with the 3-clique.
So what happens is once we repeat this, until V dash is a clique.
Does that make sense why you can do that?
So you just keep deleting vertices until this is a clique.
And those clauses plus clause condition is still satisfied.
You will still have greater than or equal to that many clauses.
So now let's look at what we have.
We have V dash, which is a clique and xi is equal to-- so once you have done this process as many times as you need, when is xi equal to 1?
So remember that-- so V dash is also being updated.
Every time you set xi to 0, xi is being removed form V dash.
So that property is always satisfied-- that define V dash equal to xi for xi equal to 1.
That property's always invariant.
That means that even after reviewing these repetitions, you still have xi equal to 1, if and only if i is E V dash and 0 otherwise.
So does make sense why that property is solved?
So you have a clique and you have this assignment.
So that should take you back to this.
So remember where we took a clique of size k and we found that if you go through all the algebra, you will find something which is like you will go through and you will get mod of E bar plus mod of V plus mod of k, right?
So here you have a clique.
So forget about what you did before.
So just consider this is an assignment according to those rules.
And those rules give you that you should get number of satisfied clauses is equal to mod of E bar plus mod of V plus mod of V dash.
Realize that V dash here is not k. Does that make sense?
Because V dash is just-- so you started with some set.
You started deleting some elements.
You throw [INAUDIBLE] randomly, but you ended up with some V dash.
And by this argument, you had E bar plus mod of V plus mod of V dash-- E bar from this clause, V from this clause, V dash from this clause.
So you didn't end up with this.
And you know that because of what we showed here does not change number of satisfied clauses, it's still greater than or equal to mod of E bar plus mod of V plus k. And we cancel this.
You cancel this.
And you get mod of V dash is greater than or equal to k, which means that you have a clique of size greater than equal to k. Questions?
Does that make sense?
Really?
All of it?
OK. That's good.
Any case.
So let's go back and see what we are doing here.
So the way you are doing NP-hard reductions is you take a problem that you already know is hard, you take any arbitrary instance of that problem, and you transform the input into an input to the problem that you're trying to show is hard.
So you take problem A, which you know is hard.
You transform the input into an input for problem B.
And then you show that if you can solve [INAUDIBLE] problem B, you will be able to solve problem A.
But since you know you can't solve problem A in polynomial-time, you know that you can't solve problem B in polynomial-time.
And the other important thing to notice here is that the reduction needs to polynomial-time.
So look at this reduction, for instance.
What are you doing here?
You're taking every vertex.
You're making a clause.
And how many clauses do you have?
Well, you have about n squared clauses here.
You have Rn clauses and you have Rn clauses here.
So time to construct that is like roughly Rn squared.
So you're constructing the clause in polynomial-time.
So you have a polynomial-time reduction.
And if you reduce a known NP-hard problem in polynomial-time to an unknown problem, you can show that it is NP-hard.
OK.
I don't think we have time to do another problem, so we're done.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to look at approximation algorithms for the traveling salesman problem.
So I hope everyone knows what the traveling salesman problem is.
You have a graph, you're trying to visit every vertex.
So you start at your vertex, you visit every single one, and you end at the starting vertex.
And you want to do that in the shortest possible time, or distance, or whatever the metric is.
So, unfortunately, this is NP-hard.
Also, the approximation algorithms for any constant approximation is also known to be NP-hard.
So you should have gone over approximation algorithms in lectures.
So, basically, let's say the optimal solution has some value of e, and if you want to guarantee that your algorithm will end up with a value which is within a constant factor of v, so less than cv, that is an approximation algorithm.
But for the traveling salesman problem, the constant approximation algorithms are also NP-hard.
So instead, we modify it slightly.
So, on the traveling salesman problem, we impose something called a metric.
So the important relation here is this one.
So, first of all, let's go through this.
So you have a distance metric for-- so xy are vertices.
Your distance is always greater than 0, which is reasonable.
You're undirected, which is this relation.
And you have the triangle inequality, which means that if you have 3 vertices, and you have distance like this, this distance is always smaller than the sum of these two distances, which is like real world-ish things that make sense, right?
If this distance was longer, it would just take this path instead.
So the distance by taking other node is always greater than or equal to the direct distance.
So, turns out, the Metric TSP problem is also NP-hard.
So nothing very great there, but you can do a constant approximation here.
And you'll go through two approximations today.
The first, the simpler one, is a 2 approximation, and then, we'll improve this to 3/2.
So let's start with the first one.
So before we begin, let's define a couple of terms.
Let's define c of S. So what is S?
Let X be a path.
Or, rather, let's say S is a set of edges.
So if you have your graph, here, your set of edges could be something like this, this, this, this.
So that's set of edges.
And c of S is defined as the sum of the weights of all the edges.
And this, actually, should be a multiset, because you can count the same edge twice.
So you can count this edge three times, or how ever many times you want.
So that's the definition.
So, now, we want to find a cycle which goes through all the vertices, and we want to minimize the cost of that cycle.
So let's say your optimal-- this should remind you of the Hamiltonian Cycle problem.
So this is, essentially, finding a Hamiltonian cycle in the graph of minimum weight.
So this is worse than the Hamiltonian cycle problem.
This is find the minimum weight Hamiltonian cycle.
So let's say there exists this beautiful Hamiltonian cycle.
You know of minimum weight.
Let's call that H star G. So it's the best Hamiltonian cycle on this graph, and that's the shortest path that your traveling salesman can take.
And so the cost of that is [INAUDIBLE] if H star S G. So how do we go about trying to approximate this problem?
So think about what algorithms you've seen that sort of connect all the vertices of a graph, and minimize costs of edges.
Does it remind you of anything polynomial that you've seen in the class, that connects all the vertices?
Expands all the vertices?
Exactly.
So [INAUDIBLE], minimum spanning tree.
This minimum spanning tree is polynomial time, and it connects all the vertex.
So let's take some graph.
Let's just go with this one.
So you have some vertices, and you make a minimum spanning tree out of them.
Now, clearly, this is not a cycle yet.
But let's try to construct a path out of this minimum spanning tree.
So, first, let's root it.
So let's say we are rooted minimum spanning tree.
Let's say this is the root.
So you have three things going out of there.
I think that's it.
So let's give these labels.
So, now, what we're going to do is, the way we're going to traverse all the vertices, let's just do a DFS.
So, in DFS traversal, you'll first see 1, then you'll go down and see 2, then you'll go down and see 3, then you'll go back up, see 2 again.
Go back down, see 4.
Go back up, see 2.
1, 5, 1, 6.
So, basically, you're ignoring the rest of the graph.
You find your minimum spanning tree, and you follow all the paths.
Follow them back up, and just do a DFS traversal.
And then, you go reach back to 1.
And then, you have this-- well, it visits all the vertices.
It visits some of them more than once, which is a problem, which we'll deal with shortly.
But it visits all the vertices, and it's a cycle.
So, now, the problem is, the traveling salesman problem does not allow you to visit vertices more than once.
Because, if you did not have this restriction, you could shorten your bat length by going to a separate vertex and coming back, or something like that.
So let's make that more concrete.
So, at least in this case, given this triangle inequality, I claim that you can just delete the duplicate vertices.
So let's look at the duplicate ones.
You have 1, 2, 3, this repeats.
So you delete that.
Repeats, repeats, repeats, and that's cycling back.
So how do you delete things?
So, in this case, you had a path, right?
You were going 1 to 2, and 2 to 3, to 2, and so on.
Let's write the 4 in.
So this is [INAUDIBLE] only at tree edges.
So how would you delete this?
So, let's say, you find the first duplicate vertex, and you don't want that.
That's not allowed.
So all you do is, simply, you follow the path, and you bypass it.
And by the triangle inequality, you know that bypassing it will never increase your cost.
It'll decrease it, or it will keep it the same.
So you can remove the duplicate vertex in this path just by bypassing it.
But, also, remember that this is a complete graph.
So the metric is defined on all pairs of vertices.
So every edge exists with some value.
So that also follows with the triangle inequality.
So if an edge does not exist, just make it the sum of the-- so if xz is not an edge, just make it the sum of xy plus yz.
So in any case, you construct the initial path just by going down the tree and doing a naive DFS traversal.
And, then, you correct that path by skipping over the duplicates.
So, finally, we'll end up with [INAUDIBLE] skipping all the duplicates we'll get a 1, 2, 3, 4, 5, 6, 1.
And that's a valid cycle.
So let's call this minimum spanning tree T. So that's your MST.
And you are removing duplicate edges, and getting a cycle, C. So now, actually, let's take another step back.
OK, let's define our cycle, first.
So let's call this cycle, this guy is C. And then, you're going from C, and you're deleting the duplicates, and you're getting C dash.
So now what you have is cost of C dash.
That's a [INAUDIBLE], but anyway.
The cost of C dash is less than equal to cost of C. And what is the cost of C?
If you know the weight of your minimum spanning tree, what is the cost of C?
C is just doing a [INAUDIBLE] of this traversal.
So what is the cost of C, if you know your minimum spanning tree?
[INAUDIBLE]
No.
So you're traversing every edge in the minimum spanning tree twice.
So you're doing a DFS traversal, right?
So you're going down, and you're coming back up.
So you're going down, coming back up, backtracking, coming back down.
So every edge is traversed, right?
So it's exactly twice T. So let's bring up a different board.
So you know that cost of C is twice the cost of T. Does that make sense, why the traversal implies that you have every edge being visited twice, right?
OK.
So, now, our claim is that the C dash cycle is a 2 approximation of the actual cycle.
So why is that?
So we've already [INAUDIBLE], so this also implies that C of C dash is less than equal to 2 of C T. Realize that this is not a valid cycle, but this is a valid cycle.
So, now, we need to show that this is somehow bounded by a star G. So how do you do that?
Well, look at H star G. What is H star G?
H star G is just a cycle, which goes through the optimal cycle, which goes through all the vertices and comes back to the parent vertex.
So this is H star of G. This is the optimal thing.
Now, you can take an edge, e, here, and delete it.
And then you'll get a spanning tree, because this is your optimal cycle.
Remove one edge, and you get a spanning tree.
So let's call that T dash.
Does that make sense, why that is a spanning tree?
Because you had a cycle, and you remove one edge, so it touches all the vertices, and it's a tree.
So it's a spanning tree, but it's not the minimum spanning tree.
So you know that H star G, the cost of H star of G, is greater than equal to the cost of H star of G minus the edge we removed, is greater than equal to the cost of T. Make sense?
So you remove one edge, and then that is still greater than the minimum spanning tree.
So, now, combining this guy and this guy, you get cost of C dash is less than equal to 2 [INAUDIBLE].
We know that cost of C is less than cost of H of G, so you get a 2-approximation.
So does that make sense?
So that was a 2-approximation.
That was pretty straightforward, We just constructed a spanning tree.
You did a DFS traversal and removed duplicates, and you have a nice path.
OK, let's just keep it down.
But here's the thing.
It seems kind of wasteful to go through all the edges when you don't need to.
So you're traveling down the tree, you're going back up, and you're traversing every edge twice.
So it seems kind of ridiculous that you would be doing every edge twice.
So what could you do better?
Well, before we introduce that, let's prove a couple of lemmas.
So, first, we started with this.
So let's say S is a subset of V. So you have a graph, and you make a subgraph.
So you pick out some vertices.
So you pick out this one, and this one, and this one, and this one, and that is your S. And, so, you get a new graph which contains just those vertices.
And, whatever, I just connect them.
So the claim is that that graph also has a Hamiltonian cycle, the minimum cost Hamiltonian cycle, which is also the traveling salesman solution.
So let's call that H star of S. So it's some cycle, which looks like this, I guess, here.
So H star of S. Now, the claim is that the cost of H star of S is less than equal to the cost of H star of G. That should make intuitive sense, because you're taking only some of the vertices and trying to traverse them, and, in this case, you'll try to traverse all the vertices.
However, this is only true because of the triangle inequality, and let's see why that is the case.
So, proof by contradiction.
Say cost of H star of S is actually greater than cost of H star of G. OK, so, look at H star of G. It's a cycle, right?
So you have something.
Your cycle.
And, in this cycle, you have all the vertices of S. So pick them out.
And, now, you have a cycle which has cost less than the optimal cycle in S, but it contains all the vertices in S. So what you can do is, now you can use the skipping lemma from before.
So, last time, we didn't move duplicate vertices.
But, this time, what you'll do is, instead, you'll just skip over this vertex.
We'll skip over this vertex.
And so, every vertex that's not an S, skip over it.
And that can only decrease the cost.
So, now, you've constructed another cycle, which contains only the vertices of S. So it's a Hamiltonian cycle for S, but it has cost less than equal to H star of G, which means that this can never be true.
So the important fact is, there, that if you have a subset of vertices making the restricted graph, the cost of the minimum Hamiltonian cycle is always less than equal to the one in the original graph.
So, intuitively, that should make sense.
OK, next thing is something which might seem unfamiliar right now, perfect matchings.
So you've seen perfect matchings in the content of bipartite graphs, right?
So you find the minimum cost perfect matching.
You do this flow thing, you send all the flow in, and then you connect the vertices with the capacities, and whatever.
So, it turns out, in a complete graph, you can still do perfect matching.
So perfect matching is, you have a bunch of-- so, let's say, you need to have an even number of vertices, right, until we have perfect matching.
So, let's say, you have these varieties.
So this is a perfect matching.
So every vertex has one edge coming out of it, exactly one edge coming out of it.
And it needs to be even, because, otherwise, that doesn't work out.
So that's perfect matching, and the minimum cost perfect matching is the minimum among all such things.
And you did this for bivariate graphs.
It's finite [INAUDIBLE] networks.
So I'm not going to go into the algorithm for this.
It's kind of complicated, but it uses linear programming, and you can find this for a complete graph, and it's polynomial, so that's good.
So, given a complete graph, you can find the perfect matching.
[INAUDIBLE] for now.
OK, one last thing we want to introduce is Euler circuits.
So who has heard of Euler circuits before?
Anyone?
Sort of?
OK.
So the reason we are-- so, OK, let's go back to what we did before.
We had a tree, and the best way we found to traverse it was just going down, and going back up, and going down, and terribly messy.
So what we would, rather, want to do, is sort of traverse the thing without repeating edges.
So, I don't know, you've probably seen this puzzle before.
So you have this graph given to you, and the task is to draw this graph without lifting your pen off the paper.
So first, for example, in this graph, you could start here, go here, go here, and come back, and something, and there you got.
So you can do that.
But if you add on another lobe, here, then you can't make a circuit.
You can still make a path, I believe.
If you add another one, you can't even make a path.
So how does this work?
Let's see.
So, let's say, forget about the graph, for now.
Let's say you're just drawing.
So you start somewhere.
You go to a vertex.
You go to another vertex.
Come back, go to this vertex, leave it.
And so, observe that, whenever you're making this drawing, you go to a vertex, and you leave it.
Every time you hit a vertex, you leave it.
Since there's the circuit, you just loop around, and every time you enter a vertex, you'll have to leave it.
What that means is that, even though this is not directed, if you drew out the actual path, you would see, the number of edges going into a vertex is equal the number of ones leaving, which means that every degree has to be even.
So if you go and look at this graph, which is that of lobes, this degree is not even.
Neither is this, neither is this, neither is this.
They're all 5, I think, yeah.
They're all 5, which means that this can never have an Euler circuit.
So a graph can only have an Euler circuit if it has even degrees for every vertex.
And the other way is also true.
If a graph has even degrees on every vertex, then it must have an Euler circuit.
That's not hard to prove, but there's a constructive algorithm you can use.
So, let's say, you're given this graph, for instance.
You would simply go to the graph, just start at some random node, and then go through, and keep following edges until you can no longer following edges.
So, let's say, you stop here.
Then, you pick another edge, and start, and so on.
And, then, you can splice these cycles together at some point.
So it's kind of a [INAUDIBLE] argument, but it should be sort of intuitive why you can construct another path, given an even degree.
You just perform searches.
You just create cycles, and you splice them together.
But, for now, just take it as fact that a graph is an Euler circuit, as in you can draw it without lifting your pen off, if, and only if, every vertex has even degree, so why is that interesting?
So, let's say, we could add some edges to our tree and turn it into one of those nice graphs.
Right now, this is not, right?
This is degree 1, degree 1, degree 3, degree 1, degree 1, degree 3.
All of them are odd, so that's not good.
But, let's say, you could add some edges in, turn it into an Euler circuit, and then you could do a nice reversal off it, and, maybe, that will give you a better approximation.
So with that hope, let's look at the algorithm.
So what you do is, you go back to your tree.
Let's just leave it at that.
So, now, let's see.
So this is degree 2, that's good.
This is degree 3, that's not good.
Degree 1, this is fine.
This is degree 1.
This also is degree 1.
This is degree 3.
Actually, let's get rid of this, so it does not have degree 3.
That's a lot of vertices.
OK, there we go.
So, in this graph, you see that you have 1, 2, 3, 4, 5 vertices which have degree odd.
So, now, we would like to add some edges to turn this into an Euler circuitable graph.
So how do we do that?
So let's call the set of odd edges S. Odd vertices, sorry.
So you take the set of odd-degree vertices.
Now, go back to perfect matchings.
So what does a perfect matching do?
It adds edges to that graph so that everything gets degree increased by one.
So if you increase the degree of all the odd vertices by 1, everything turns even, right?
So you take the set of odd vertices.
OK, another thing to observe.
Realize that, how many odd vertices can you have?
Can you have an odd number of odd vertices?
Because that would screw up the whole thing where we needed an even number of things.
So why is that not possible?
Why can you not have an odd number of odd vertices?
So the thing is, that, let's say, you have some graph, and what is the sum of the degrees of the graph?
Let's move to a different board for this.
So you have a graph, G, and you want sum of degrees.
So di is the degree, for all V. So what is this equal to?
So, let's say, you have this graph.
So, now, if you count the degrees of every vertex, you're basically counting the number of edges coming out, which means that every edge is counted twice, once for this end point and once for this end point.
So this edge is counted twice, for here and here.
This edge is also counted twice.
So, basically, the sum of the degrees is nothing but 2 times mod of E. Does that make sense?
So this is even.
Now, let's say you take only the odd vertices out.
So the even vertices are good.
This is good.
This is good.
This is not good.
This is also not good.
So these are the odd vertices.
So the sum of the degrees of the even vertices are, by definition, even.
So, if you remove them, the sum of the odd vertices should also remain even.
And so, you have the sum of odd degrees becoming even, which means that you'll need to have an even number of them.
Make sense?
So, going back to this, so there's a lot of branching off, here.
But going back to the main point, here, is that you have an even number of odd vertices.
So consider the restriction of the original graph, G to this set, S. So, now, we're going to the first thing we did, there, where we considered a restriction of the graph to a certain set of vertices.
So we take the set, so 1, 2, 3, 4, 5.
So you take these five vertices.
And you consider the graph that is restricted to these five vertices.
Oh, sorry, there should be six.
Oh, that's interesting.
Oh, there we go.
That's the other one.
So those are the six vertices which have odd degrees.
Now, you find the perfect matching with your polynomial-time algorithm, and you get something.
So you can now add those edges back in here.
So these are your new edges, and, let's say, something, this one.
OK, so you get three new edges.
And, now, realize that all the degrees are now even, so now you can do your Euler circuit.
Also, let's call this matching M. So this is a set of edges, M. Let's call the original tree T, and the new thing that is formed, is T union M. So you're taking all the edges from T, and you're adding all the edges from M. So, realize that you can have multiple edges, but that's fine, because Euler circuits allow you to have multiple edges.
So, now, you take this graph, and you find this Euler circuit.
So that is basically this thing.
So this set of edges in some order, and that order exists.
So the cost of the Euler circuit-- let's call it C-- is equal to the cost of T plus the cost of M, right?
Because you're traversing all the edges in your graph.
So that is the cost for the Euler circuit.
Now, my claim is that-- so this is the difference between all the nodes in the graph.
And, remember, you can do the whole duplication argument, from before.
So, where's that?
Oh, there.
So you can do a duplication argument, so you're visiting all the edges in the graph, all the vertices in the graph.
You remove the duplicates, and you have a valid path.
Now, let's see if this valid path is actually a better approximation.
So, again, you will go from C. So this C, you will go to C dash.
And this will give you, as before, cost of C dash is less than equal to the cost of C, So, now, you're only interested in cost of C. So what are we doing there?
So we know that the cost of C is equal to cost of T plus the cost of M. So, from the previous problem, we know that the cost of T is less than equal to the cost of H star of G, right?
So this is less than equal to cost of H star G. So this guy is less than the optimal Hamiltonian path.
What about this guy?
So let's look at the actual H star G, again.
So, actually, let's look at H star S. So remember that S is the set of odd vertices, and that is where this matching is done.
So H star S is nothing but the optimal Hamiltonian cycle on that restricted graph.
From our previous lemma, which is down here, we know that H star of S is less than H star of G. So we know that cost of is less than equal to cost of whatever, G. So, now, let's just look at H star of S. Now, we construct a matching.
We take every other edge.
We take this one, and we take this one, and we take this one.
And look at the alternate set, also.
So take this one, and this one, and this one.
So look at the blue set and the red set.
Since it's part of a Hamiltonian cycle, they're both matchings, right?
So let's call the red one M1, and the blue one M2.
So I'm not saying they're perfect matchings.
They're not the minimum matchings, but they're definitely perfect matchings.
And because we know the perfect matching is M, or the minimum matching is M. Cost of M is less than equal to cost of M1, and, also, the cost of M is less than equal to the cost of M2.
That make sense?
Because M was the minimum cost matching, you have cost of M is less than any other matching, which is constructed from this.
Again, so this implies that the cost of M is less than equal to half cost of-- OK, let's not write this here.
Let's get a different board.
So cost of n is less than equal to half cost of M1 plus cost of M2.
Since both of these guys are larger than this, their average is also larger than this, right?
But what is cost of M1 cost of M2?
This is nothing but half cost of H star of S, right?
Because the Hamiltonian cycle is constructed from M1 and M2.
And by our lemma, this is less than equal to half cost of H star of G. So, now, we have all we need.
This is less than equal to half cost of H star of G. And then, add them together, you get cost of C dash is less than equal to cost of C, is less than equal to the sum, which is equal to 3/2 cost of H star of G. So that's the proof.
So there's a lot of things going on, here.
Let's try to go back and see what we did.
So we took a minimum spanning tree, and then we tried to remove all the odd degree vertices.
So we took all the odd degree vertices, and made a perfect matching, the minimum cost perfect matching.
So we added edges just to make everything even degree.
Then, we took the Euler circuit on that graph, and we removed duplicates.
So that's fine, that's the easy part.
But you do [INAUDIBLE] circuit on that graph, then you argued that, because all the circuits in that graph is just the sum of the edges in the spanning tree plus the sum of the edges in the matching, which you added later.
So the spanning tree had already bounded, in the previous argument.
The matching was bounded by taking the optimal Hamiltonian cycle, decomposing it into two matchings, arguing that those two matchings are not the optimal matchings-- not necessarily the optimal matchings, but they're less than equal to the optimal.
So they're even, there.
And, so, the cost of the optimal matching, which you were using in our constructive path, constructive Hamiltonian cycle, is less than equal to both of these matching.
You add them up, you get that bound, and it works.
So questions on any of the steps?
This is a lot of branching off and then coming back together.
Yeah
The last one's 3/2, right?
Yes.
That is not 3 over 3.
That would be p equal to np.
Anything else?
You're free to go, or ask questions, or whatever.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Let's get started.
Today's topic would be distributed algorithms.
We will look at two new algorithms.
But they are similar to what you have seen in the lectures.
So it will also be a review of some concepts in lectures.
So our first example would be, again, [INAUDIBLE], the simplest example.
But this time, the network and topology would be a ring.
So in the lectures, the example we see is a click, meaning they are fully connected.
Right.
Every node can talk to every other node.
There, if you remember, our solution for everyone to generate a UID or a random number.
And if you are the maximum, then you output, you're the leader You can do that because, yeah, you are connected to everybody.
So you immediately know what random number everybody generates.
And you can compare whether you are the largest.
Now, the idea is the same if we have a ring.
So you want everyone to generate a ID or random number.
I'll just say ID from now on.
And you want to collect everyone else ID so that you know whether your ID is the largest among all of them
[INAUDIBLE]
Pardon?
[INAUDIBLE]
OK.
The question is where did the comparison happen?
So we first need some way to pass the numbers around such that everyone has everyone else's number.
Right.
You have the number of everyone, then you can compare whether the largest equals yours.
OK.
If the largest equals yours, then you know you are the largest.
And you're going to output, I'm the leader.
OK.
So the difficulty is just how to pass the numbers around so that everyone sees everyone else's number.
Any ideas?
Simple solution.
Well, there's not much you can do.
You are just connect to your two neighbors, the left one and the right one.
So how do you propagate the information?
Go ahead
You can take the maximum of your ID and neighbor's IDs and kind of broadcast that
OK.
So what does broadcast mean here?
Like tell your neighbors what the largest of your value [INAUDIBLE]
OK. And yes.
Let me call them A, B, C, E, F. Then, yes, C can tell D or can tell B, but how does C tell E?
It waited around to I guess propagate the maximum
Yep.
So it would be just, say, everyone talks to its right neighbor.
And then B has A's IDs, C these B's ID, D has C's ID, and then continue and pass around next time.
So just to make it perfect clear.
Say they generate some random IDs that's 5, 10, 20.
And then in the next round, A would send its ID to B, B would send its ID to C, C would send its ID to D. And in the next round, B would pass this information along to C. And C would pass the information to D. And just continue.
And eventually everyone will have everyone else's ID.
So how many rounds do we need?
If they are in nodes in the system
I think probably in just one direction or two neighbors
I think you can do it either way.
If you propagate both ways, it probably 2x faster, yeah
[INAUDIBLE]
Yeah, correct.
It's just O(n).
But to keep it simple, let's say we just propagate in one direction.
That's also fine.
Still O(n).
So how many messages are sent in total?
n squared
Pardon?
n squared
n squared.
Yep.
Is that obvious to everyone?
OK. Give an explanation
Yeah, at each round everyone sends a message to its neighbors
Yeah.
Yeah.
Every round, everyone sends a message.
Or you can think of every message as propagated n times.
And there are n messages in total.
OK. Yeah.
That's definitely a solution.
Well, you can imagine, it's probably the naive solution.
And can we do better than that?
In this case, are we assuming they know that there are n of them?
OK. Good question.
So if they know there are n of them, yep, then they know how to terminate.
So your question is how to terminate, right?
If they don't, eventually you will receive your own ID on the other side.
If we keep sending left, eventually you will receive it on the left port.
And, yeah, that's an indication of termination.
Question?
Are we assuming the trajectory that these are unique?
Yeah.
We usually assume that, yeah.
Either it's a unique user-- sorry, UID.
What does U stand for by the way?
Or if you generate random numbers in a large range, it's very unlikely that they collide.
OK.
So can we do better than that?
You have some idea?
Once something [INAUDIBLE] where you only pass it to the one that's bigger
Hm-mm.
OK.
I think you have two ideas.
And binary search, we'll see how that works later.
So you said only forward something that's larger?
0
Yeah, like only forward things are larger
OK, yeah, that's on the right track.
So one obvious thing we can do is that so we actually don't care what all the IDs are, right.
We only care whether a certain ID's the largest.
So for example, in this case, when A send its ID 5 to B, B knows this 5 won't be the leader.
All right.
Because 5 is too small.
It's even smaller than his ID.
So we can choose to drop this message.
There's no point in passing that message further.
Same thing for this message.
C knows that 10 is too small.
And it doesn't have to pass it along
[INAUDIBLE]
Oh, the ID?
It's just the integer each node chooses at random.
See.
Yeah, the only purpose of the ID is to break symmetry.
So, yeah.
Like we have seen the lecture, if they don't do this, if they don't have any unique identifier, they won't be able to select the leader.
And when they have a unique number, then they can select, yeah, the largest one or smallest one in some way.
OK.
Does this optimization make sense?
We can cut [INAUDIBLE] messages that have no chance of becoming the leader
What is the upper bound of it?
Is it still n squared?
Correct.
Yeah.
That's a very good question and very good answer, by the way.
But how effective is this?
Well in average case, we may be able to drop some messages.
But there pathological case where it actually doesn't help at all.
Say this is the ID we choose.
Then when there's 20, it send around.
B cannot drop it.
Right, because it may be the largest.
And when this 10 is sent to C, C cannot drop it.
And when 20 comes along, it also cannot drop it.
So no node can drop any message.
Its worst case is still n square.
OK.
So let's think about the binary search equivalent, yeah, or how to binary search in this case
So binary is [INAUDIBLE] what?
Yeah, good question.
It's not very obvious binary searching to what.
I Yeah, just some binary idea that will give us a n log end bound.
So actually this is the better logarithm.
But once you have a log n, you can probably get-- OK, go ahead
How about kind of merging, like instead of finding the [INAUDIBLE], finding like log maximum, then [INAUDIBLE]
Yeah.
That's on the right track, yeah
Like clusters of some [INAUDIBLE]
Yeah.
That's definitely the right idea.
Let's detail it a little bit.
How do you carry it out?
Hm-mm
Divide the two parts.
Like kind of the [INAUDIBLE]
And, OK, so then it's let them select their leader respectively and compare which one is larger.
It's interesting thought
First, for example, A, B finds the maximum between these two, the C, D finds the maximum E [INAUDIBLE] maximum.
Kind of merging, considering them as one node
I think the first difficulty I see is that if you cut it by half, it's no longer a ring, right
Also they can't cut themselves in half
Yeah.
Yeah.
Correct.
Yeah, yeah.
Yeah, it's definitely not an easy problem, not a easy algorithm.
And the idea is to, well, you had the right idea.
That we want to-- what's the word?
Let the weak candidates shut up early.
So what I mean by that is say we have several round, so this B and A and C. We will that B only propagate to A and C first.
If B is the local maximum among A, B, C, then B will try to talk further.
Increase its range.
If B is not the local maximum of A, B, C, then, yeah, it can be quiet.
Doesn't need to send messages anymore.
If B succeeds in the next round, then you go further increase its range to try to talk to more people.
OK.
So how does it work in detail?
Well so in round I, we will let a node send this message up to 2 raised to I hops.
In this case, is 1 and in next round is 2 and then 4.
If at any point some mode like along this range decides that you are not the local maximum, then they will reply that, yeah, you no longer need to send messages anymore.
If this message successfully reaches this endpoint that 2 raised to I hops, then this guy will respond.
And this guy, if it still thinks you are the local maximum, then it will respond the message while saying, yeah, continue.
OK. And if the sender receives the continue message on both sides, then they it continue into the next round.
Otherwise, it will go inactive.
Is the algorithm clear?
If the problem reaches the next maximum, like next the node each has to send a message
Say that again
Like after receiving like don't sending it, like do you have to choose one node, which has to send the message?
Oh, OK.
In the first round, everyone send their messages.
OK. Then some of them will go inactive because they learn they are not maximum.
And the remaining, the surviving ones, will, yeah, continue sending messages.
And then, yeah, half of them probably will die in the next round.
And the surviving ones keep sending messages.
Make sense?
How do you return messages through I?
Like if you [INAUDIBLE] its neighbor and not the node that do either
Yeah.
So we will send the message of this form, say, well, some message.
And then we will send the hop and the direction, either left or right.
And this hop will initially be set to 2 raised to I, the number of round.
Then when this guy receives the message, it will increment the hop count and pass it along.
And every node when forwarding the message will decrease the that h by 1.
And finally when it reaches here, that number becomes 0.
And when a node sees a message with 0 hop count, it's going to reverse it, send it in the other direction.
And, again, set it back to 2 raised to I.
This message I should say ID.
And at certain point, a certain node may decide this ID's too small.
It doesn't have a chance.
Then the I can directly send it in the opposite direction, replying a message saying, yeah, you are too small.
OK.
So any more questions on the algorithm itself?
If not, what's the next step?
Time
Yeah, time complexity.
I already claimed it's n log n. Is it?
This is round.
This is message.
This is also message complexity.
OK.
So why is it n log n?
The log n [INAUDIBLE]
Hm-mm.
So we have a certain number of rounds.
So how many rounds do we have?
Two
Yeah, maybe.
But, yeah, I'll just say log n. OK. Because you are increasing your hop length.
And we're going to compute like how many nodes are still active each around and how many messages are sent in the round.
So, well, the number of nodes active will just be this number if we start from 0.
Why?
Because the first time everyone is active.
In the next time, only 1/3 of them will be active.
But we said we are conservative here at we put 1/2.
Right.
Next round is actually 1/5.
If it's local maximum, it means like these two and those two will go inactive.
But we put this as a upper bound.
So this is the number of nodes that are still active.
And they will send the message up to this many hops.
OK. And there are two directions.
And you send a message back.
And then, yeah-- sorry, send the message forward and someone will reply.
But in the end, this is n log n. Yeah, and I think I got 8n log n. So my recitation note says it's 4 log n, not entirely sure what's going on.
Yeah.
But you can double check whether this is correct or the recitation note is correct
What is I again?
I's the number of round in the I-th round.
Yeah.
This many nodes are still active.
And each of them will send a message of this many hops.
And I didn't mention whether the network is synchronous or asynchronous.
And it turns out it doesn't care.
Some of them can work for both synchronous and asynchronous.
Apparently, it works for synchronous networks.
If it's asynchronous, then what changes is that different nodes are in different rounds.
A certain node may be far ahead than the others.
But it's fine.
Eventually, they will converge to the correct result.
OK.
So let's look at the second problem.
Well, problem's defined even simpler.
We just want to count how many nodes are out there.
We want the algorithm to work both synchronously and asynchronously.
By that I just mean we have a network.
Well, say you have a lot of nodes after that.
Just want to count how many nodes are there in this network.
So I'll give you, say, one minutes to let's first come up with a high-level plan.
Is the problem clear?
In the worst case, [INAUDIBLE].
The worst case
OK.
I haven't defined that.
Let's not worry about complexity now for now.
The complexity will depend on number of nodes and the number of edges, E. Let's just get it functionally correct.
OK.
Anyone share a high-level strategy?
Go ahead
So each node will start like IDs of the other devices [INAUDIBLE].
So basically like it's going to send propagates [INAUDIBLE] sets of each node
On what edge?
Furthest one
Yeah.
It wasn't on both edges?
Itself first
OK.
So then maybe send the 1 here and send the 1 here there.
Then this node will think it has a children, it as a child, right, which is this one.
Let's just say this is the entire network.
And what message does it send to this guy?
Reinforced it
Well, you probably need send two here because you have one and this is possibly its child, right.
But then we're double counting this node.
See the problem?
Do nodes have their IDs, like unique IDs?
Oh, yeah.
They have their IDs
We can't send the IDs instead of the IDs through the end node
OK. We are going to send all the IDs
Yeah, neighbors.
And after n steps, it's going to have the [INAUDIBLE]
OK. Yeah, that's an interesting thought.
And then you may still need the root.
And that will have everyone's ID.
And then see how many unique I's are there.
OK.
Interesting.
Does this algorithm work?
Yeah, I don't see any problem.
OK.
But let me still repeat what our algorithm is because it's closer to what's in the lecture.
So we're going to find a spanning tree of this network.
A spanning tree means, well, like I have to cut one of these edges.
If I have a tree, then I can have every child report to its parent.
Like how many offsprings, including myself do I have.
And this node will sum up all its children and report to its parent.
Does everyone get that?
So first, we'll find a spanning tree.
And second, we'll have child reports to parent
How can we find the spanning tree?
Good question.
So in the lecture, we have seen algorithm that find BFS spanning tree for synchronous networks.
OK.
This is review of the lecture.
How does it work?
Each node will, say, we need to first choose a root.
And our root will just send a message to its neighbor and, yeah, saying, you are my child, you are my child, you are my child.
And then every node upon receiving this message from the parent will search among its neighbors.
All right.
So the neighbors that haven't got a parent will acknowledge this sender as the parent.
OK. That's a little messy.
So this node will search to the leaf node.
But then it will also try to search for this guy.
But this guy already had a parent, then, yeah, it will say I already got a parent and blah, blah, blah.
This will give us a BFS spanning tree.
What does it mean?
It's a spanning tree found by BFS.
Does it work for asynchronous network?
Go ahead
This version doesn't.
But there's a different version that contains an edge for relaxation technique
Yeah.
Correct.
And so why does this version not work for a synchronous network?
Because different nodes can be on a different round number so that a longer path could end up going to the node even
Yeah, exactly.
So let me give a concrete example.
What does asynchronous network mean?
Is that, well, messages travel at different speed.
Say this link for some reason is temporarily down.
And this node doesn't receive the message from the root.
And then this message travels very fast.
And this message also travels very fast.
So this message may reach this node earlier.
And then this node will think of it as a child of this node.
OK. And then this message comes along, finally.
But this node says, yeah, I already have a parent.
And I'm going to reject you
So it still makes a spanning tree?
Yes.
Good point.
This is not a BFS spanning tree but it's still a spanning tree.
All right.
So in our problem, we're totally fine with it.
Yeah.
But you do have to know if you really want a BFS spanning tree in an asynchronous network, then you have to like record the distance and do the relaxation and so on and so forth.
OK.
So we will just use this algorithm, the BFS spanning tree algorithm, just run it asynchronously.
It doesn't find the BFS spanning tree but it finds some spanning tree.
OK. How does this algorithm work?
We will have several variables.
The first one is parent.
We will initialize to this undefined single.
OK. Then every node will pass around this search message.
I'll use a slightly more shorthand notation than from the lecture.
OK. Let's say the code I wrote is for a process u. OK.
If we receive a message, a search message, from v to u, that means u-- sorry, v is trying to become a parent.
OK. And if I do not have a parent yet, I should set the parent to v. OK.
So what's the next step?
Now, we got this search message from our parent.
And we have to pass it along
Should we send it to the like add your child [INAUDIBLE]
Oh, yeah.
Great.
Yeah, we first need to respond by saying, OK, I'll use a shorter notation.
Send these a cue that that's the message that will be sent to v at some point.
The message we send is parent 1.
Parent 2.
This is a response to v saying, yeah, you are my parent.
OK. Then else, we also have to respond by saying parent 0.
You are not my parent because I already got some other parent.
OK.
Missing a step here.
If we receive a search message.
Go ahead
We send messages to all the other nodes
Yeah.
We need to pass it to all my potential children.
So I use this comma u, meaning the neighbors of u.
And then I'll send them a message.
What message?
Search.
OK. OK.
So, well, naturally since we have a search message and we know how to deal with it, now we are sending this parent message with better deal with it.
Right.
So then the next chunk of code should be if we receive this parent message, I'll say parent b, meaning this message comes with either true or false.
I received this message from some node w. OK.
I'm still u here.
Because I just send all the message to all the w's, to all my neighbors, and they should give me a response.
OK.
If b is 1 that means this particular w take me as its parent.
Make sense?
OK.
So I'd better have a list of my children.
I want to keep track of that.
How?
We will create a new variable for children.
It's going to be initialized to what?
Yeah, empty set.
And now if this b is 1, then I'm going to put w in this children list.
OK.
I'm leaving some space here because our root should be slightly different.
So every other node will send search messages when it receive a search message.
We need someone to initiate.
Make sense?
So if u equals root, we say the root is v0.
So two things should happen.
First, we should set its parent to some special value.
Just say root.
And then I should copy this blob of code to here.
OK.
It's a little crowded but I hope it's still legible.
OK.
This is already almost the correct algorithm, except that we don't know how to terminate.
If we wait long enough, then everyone will receive all the responses and everyone will know its parent and this child list.
But how do we terminate?
After n rounds
Yeah, that's one way to do, after n rounds.
But the whole point is we are trying to find what n is.
We don't know how many nodes are there
If we receive reject for all, [INAUDIBLE]
Say again
For all the search, if we receive a reject
Oh, yeah.
If you receive a response, either parent 0 or parent 1 from all your neighbors, then your job is pretty much done.
But the others have not.
All right.
So you send a message very fast.
They responded to you.
And they are still working very hard.
OK.
So then we need to use the technique from the lecture that's called converge cast.
So everyone will send a done signal when it is done and all its children are also done.
OK. To do that, I'm going to define a new variable called searched.
Searched means someone has responded.
OK.
In this case, w has responded.
I'll put it into the search the list, no matter whether it accepts me as parent or reject.
OK. Then naturally I need to define this variable here.
OK. As a last step, If we search the list equals my neighbor list, that means everyone has responded and all my children are done, then I need a new variable called done.
That's another list tracking who has finished and who has not
So what's the first one?
Say again
What was the first one?
This one?
Searched means someone has responded to the search message.
OK.
Done means all its children are done.
I haven't write how done is defined.
Give me a minute.
In this case, I'm going to send my parent a message.
This is the step of converge cast.
What do I send?
I'm going to send them I'm done.
OK. Then whenever we create a message, we should deal with that message.
If we receive a message I'm done, what do we do?
OK. From w, then we're going to mark that node as done.
OK.
There's this other point.
So someone has to initiate the done signal.
That's going to be our leaves, right.
Because when this condition check, they don't have any children.
Their children list is empty set.
And their done list is also empty set.
But they're going to send the I'm done signal first.
And then in the median nodes, we'll send a done signal when all its children are done.
All right.
So this is the converged cast version.
Only gives us a termination point of our spanning tree search.
We haven't count the number of nodes in the network.
But that's a small modification.
We're just going to include that number in the I'm done signal.
So then I need to define another variable called total that's initialized to 0.
This variable will attract how many nodes do I have in my subtree, including me and all my children.
Then when I send the I'm done signal, I'm going to send this-- sorry.
That's not right way.
When I'm sending the I'm done signal, I'm going to send my total number of offspring with it.
And when I receive, one of my children reports that I do have t children.
I need to increment my total by that amount.
OK.
I made a mistake again.
Should be total plus 1.
Right.
Because I'm counting all my children and then I should include myself.
So that's the complete algorithm.
Yeah.
One purpose is just to create a different angle to look at distributed algorithm.
Usually just draw that network graphs.
But sometimes it's helpful to think about how the code actually works.
OK. That's all for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So finally, it's our last recitation.
And finally, it's my favorite topic-- cryptography-- because I work in this area.
So I have probably a little bit more than what's required to tell you.
So this recitation is access more primitives, so we'll introduce several more primitives that may be useful to your future work or study.
And so the first one is digital signature.
So we have briefly mentioned digital signatures in the lecture, but mainly as an application of Hatch, so now I'm going to introduce it as a standalone primitive.
So as you may have already known, digital signature is used for verifying message authenticity.
And it's a pair of function sign and verify.
So sign takes a secret key and the message.
It outputs a signature, which we refer to as sigma.
And verify takes a public key, a message, and a signature, and outputs on either true or false.
It either accepts the signature or reject.
So we use secret key to sign and public key to verify.
That means so if I want to send the message, I should be the only one who was able to sign it.
And everyone can verify that this message indeed comes from me.
So what properties do we want from digital signatures?
Any thoughts on that?
Go ahead
[INAUDIBLE] given the signature in a message [INAUDIBLE] to get the secret key
OK, that's definitely one.
I'll put a more general description of what you just said.
Any other answers?
And only one signature [INAUDIBLE], like on any message coming out, can you have one signature?
OK, so what's your name?
Hugo
Hugo says a message should only have one signature.
Let's think about whether that's necessary.
So if my algorithm is a randomized one that for the same message I output many possible signatures.
So why is that bad?
So for any of them, they will all verify if that's how my algorithm works.
I think that's fine.
That's OK.
It's not a bad thing.
Actually, randomized signature is considered more secure.
They are less efficient.
Any other thoughts?
Do you care about speed at all, like how long it takes to sign and verify?
That's definitely one, but we haven't got any scheme yet, so we care about functionality first.
There are faster signatures and slower ones.
So the first one is actually very trivial.
We first want correctness.
What does that mean?
That means if this sigma is indeed generated by the sign function that verified that the output 1.
Otherwise, they should put output 0.
That's actually the first and the most basic property we want.
I don't want to write it because it's-- so the other one.
So your answer is very close that you don't want to extract the secret key.
But to make it more general, what we really want is unforgeability.
That means if I have the secret key and someone else-- an adversary, who does not have the secret key-- should not be able to sign the message to pretend to be me.
So they should not be able to produce m star sigma star, such that it verifies.
Make sense?
So what you said is a special case of this.
So if they can extract, somehow extract the secret key, then, of course they can forge my signature on any other messages.
But we do want to also prevent attack where they cannot extract the secret key, but they somehow can forge another signature.
But usually we want to make the adversary more powerful because then we have higher confidence that we won't be attacked.
So adversary is totally reasonable for it to see a bunch of messages from me.
Because I am signing messages and output it to the world.
So an adversary may have seen some of the message, signature pairs, I generated.
But still we do not want to create a forgery.
Now how is that defined?
Because see you can definitely send one of these back.
That's a valid message signature pair.
So our unforgeability requirement is defined to be-- he should not be able to send such a pair where m star is different from any message he has already seen.
There is no way to prevent the adversary from sending one of the message signature pairs he has seen before.
So far, pretty straightforward.
Now, how can we get digital signatures?
So in the early days, researchers-- and it's actually great computer scientists-- they proposed a digital signature can be implemented as the inverse of public key encryption.
What does that mean?
So I'll use RSA as the example.
So RSA encryption is m to the e mod n. The encryption is c to the d mod n. So the first attempt is we will just use this as our sign function and use this as our verify function.
So now the symbol is a little bit confusing.
So now I'm signing a message [INAUDIBLE] c. Let me actually change it.
This is RSA encryption.
I'm going to transform it into signature scheme where sign signs a message, and verify, raise the signature sigma to the power of e, and checks whether or not I get back my message.
So this actually makes a lot of sense.
Why?
Because think of m as a ciphertext.
Then if I decrypt it, and then re-encrypt it, I should get back my ciphertext.
So correctness-- we have correctness.
And why is it unforgeable?
Because an attacker does not have the secret key, so he should not be able to decrypt this m here.
He cannot run this algorithm.
That's the reasoning behind it.
So far so good?
But, unfortunately, it is broken.
And so I'll give you, say, seven minutes to think about it.
Can you come up with an attack, a forgery?
You can see a bunch of messages and then output a forgery for a message you haven't seen before.
So is the algorithm clear?
[INAUDIBLE]
Can you speak louder?
Is it just the product of any messages?
Exactly.
So if an adversary had seen a bunch of messages-- because RSA has this sometimes good, sometimes nice, sometimes bad property, that is multiplicative, homomorphic, or to use a less fancy word-- malleable.
So if they, an adversary, sees this message, it can set m star to be m1 times m2 and sigma star to sigma 1 times sigma 2.
You can check.
This is a valid signature, message signature pair.
You take this entire thing raised to d. They are raised to d individually and then multiply together, and that's exactly this message here.
Attack one.
OK.
There's actually another attack.
That's even simpler and tells you this scheme is even more broken.
So all I want to do is to come up with a sigma when it's raised to e, that's equal to m. So I'm going to select the sigma, compute, m sigma raised to e, because e is my public key, mod m. I can do that.
And then output sigma m-- oh sorry, m sigma.
I select the signature first, and I raise it to the power of e. I get a very strange message, but it doesn't matter.
That's my forgery.
OK, so now you can see this scheme is basically totally broken.
But they actually come from our, well, several renown scientists.
But why is that a case?
Because actually that definition didn't exist when they were trying to when they were working on this problem.
And so that definition looks obvious today, but it's actually not obvious at all.
And I think this algorithm came in the 70s, '78.
And in '82, Goldwasser and Micali, two professors from MIT, proposed the definition for signature encryption and basically everything in cryptography.
And they won another Turing Award for that.
OK, so let's try to fix it.
Any ideas?
We do not want to change the framework.
Let's still use RSA and combine it with some other primitive you have seen to try to fix it.
What do we want to do?
We want to break this multiplicative property.
And we want to break this, this step, whatever it's called.
Go ahead
Can we change the n, maybe?
Change this n?
Yeah
Right now, just to remind you, it's a product of two primes, OK, pq.
It's a product of two primes.
It's how RSA works.
What's your idea?
Go ahead
Before it raised to the power, we can get the hash function of it
Exactly.
Let's just make a small change.
So sign will be hash of m, raised to d. And verify will be-- just check whether hash of m equals signature is to e. This indeed fixes these attacks.
Why?
Because now you need-- well, if you do this-- hash of m and 1 times hash of m2 is not going to be hash of m star because hash is supposed to be [INAUDIBLE] random.
That's not going to work.
And here, what the attacker needs to do is to find hash of m, such that it's sigma raised to e. It can still do this, but it does not know what this message is because of the one-wayness of hash function.
If we use a good hash function there, then it indeed fixes both the attacks.
But we have seen the lecture that this hash function also needs to be collision resistant.
Remember that?
A question?
Isn't the message public?
Yeah, the message-- oh, OK. Good point.
Oh, no, but, OK, you are talking about this attack, right?
So the attacker needs to find the public message, but all he can do is select the sigma, and raise it to e. That's going to be its hash of m. And then he cannot figure out where this m is.
But what about the other way?
I mean, if he has two messages, he can still get m star, and then get hash of m star
OK, so he then he gets, has of m1.
He gets hash of m2.
But you need to find the m star, such that its hash is the multiplication of these two.
And, yeah, he does not know how to find that message.
So if the hash is not multiplicative, one-way, and collision resistant, then it seems that we have fixed all the attacks we know.
However, how do we know there are no other attacks?
So actually, indeed, this is a good idea.
We have several national standards that just use this but slightly differently.
I can-- this is just for your information.
So there's a standard called [?
NC, ?]
whatever-- X93.1.
It uses RSA, this word padding, so it takes the hash of the message and pad with this hex stream, and prepended and append another hex stream.
Why do they do that?
They don't know either, but they just think it's probably more secure than only using a hash.
There's another standard that has a different steam and a difference stream here, and it doesn't matter.
So that's indeed a weakness of these types of approaches.
So their security is what we call ad hoc.
We do not know how to break them.
But we do not know how to prove they are secure either.
Yet, that's what people do in practice.
So, unfortunately, that's all I can tell you today, so how not to construct the digital signature.
I cannot tell you how to construct the secure digital signature because that's out of the scope of this class.
And it's a major topic in cryptography.
Any questions so far?
Go ahead
The hash function here is the one way, yeah?
Yes, it's one-way, collision resistance, and--
So what is the use of using the RSA?
Couldn't we just use the only hash function then?
OK, good question.
So, OK, let's be clear what you're saying
OK, never mind
Can you answer your own question?
So my question was, well, why do we have to use the RSA?
Why, when we have the hatch function?
You want me-- so [INAUDIBLE] couldn't create the forgery.
[INAUDIBLE]
How does it create a forgery?
Just answer your own question.
Let everyone else know.
Maybe they have the same question.
So answer your own question
So my answer is so adversary can't just choose random message and hash it and [INAUDIBLE]
Yeah.
What's the problem?
Problem is that a hash function is a public function that everybody can compute.
So the attacker just chooses a message, compute as hash, so using a hash is not a signature.
But good point.
I'm actually coming to that.
So far we have seen three major primitives-- private key encryption, public key encryption, and digital signature.
So if we categorize them a little bit-- so these two are asymmetric key.
They are public key and secret key.
This one is symmetric key.
And these two are for secrecy.
They are trying to hide the message.
And this one is for integrity.
Meaning, the message is what the sender sends.
So you can see we are missing one primitive here.
What if the two parties, they do share a secret key, and one party wants to verify the other party?
The other message indeed does come from the other party.
So indeed we do have a primitive for that.
It's called message authentication code.
So its definition is basically exactly the same as digital signature.
I'm just going to change it here.
Except that it has only one key.
So the sign function is replaced by a MAC.
And there's no notion of secret key and public key.
We have only one key.
And how do we verify?
OK, so verify function basically just becomes the other guy also recomputes the MAC of the message and checks whether that's the signature.
So verifier just to recompute and compare correctness-- we also want correctness.
We also want unforgeability And it's defined in exactly the same way.
Now, actually, I would have asked this question here-- is hash a valid MAC?
The answer is still no because MAC is a public function that everyone can compute, and it's trivial come up with a forgery.
So thank you for asking that question.
But the hash is actually very close.
How can we get a message authentication code?
So several ideas.
Can we just hash the key concatenated with the message?
Then some other random attacker who doesn't have the key does not know how to compute this thing.
That's a reasonable idea.
But, well, if we can do it this way, how about we do the message concatenated with the key.
Or if you want, you can do key concatenated with message and then concatenated with the key.
So it turns out this doesn't work for some very advanced reasons.
And this one may or may not.
For SHA1, it doesn't work, unfortunately.
And for SHA3-- that's the replacement for SHA1 and SHA2-- it actually works.
So the simplest MAC we can imagine is just to choose SHA3 as the hash function, and input is the key and then the message.
Not the other way.
It's also, just FYI, purpose.
By the way, there's another reasonable thought.
That is, how about we encrypt the hash?
Now, everyone can compute the hash, but they don't know how to encrypt.
If I use, say, a secret key encryption, this turns out to be wrong as well.
That's digital signature in MAC.
But one caveat here, our unforgeability is defined this way.
A little bit strange, but it makes sense.
But it indeed has some weakness in some applications.
So imagine, say, I send you a message-- today's recitation is canceled.
And it has my signature on it.
So you can verify it indeed comes from me.
But once I send that message, every of you has that message.
So next week, one of you can send that message again, saying, today's recitation is canceled.
Then you have no idea whether it's indeed me sending the message again or someone doing an April Fool's Day joke.
So how do we prevent that?
Well, of course, one thing I can do is if I'm smart I'll say, today, like in parenthesis, May the 8th, recitation is canceled.
Then you cannot repeat that message.
But we want to protect human stability.
That's the whole point of cryptography.
So one thing we could do, let's see.
Very simple modification.
When I sign the message, I'll sign 1, concatenated with my message.
Next time I sign 2, concatenated with my message.
And then 3, 4, and just have this counter that keeps incrementing.
It naturally fixes.
So you can verify if you receive the same message with the same counter, then you know it's someone else who is resending it.
So that's one thing we need to do for signature in practical use.
Now, consider another totally different application.
So say I think everyone uses Google Drive, Dropbox, something like that.
You store a bunch of files on this cloud server.
Now you are here.
You'll have a, say, cell phone, and you can access your files.
But how do you know when you read a file, it is indeed your file unmodified?
How do, maybe Google messes with you, or there's someone in the middle who changes your file?
Usually, most people do not care about that, while in cryptography, we do care about that.
So in that case, MAC and signatures do not help us.
Why?
Because if you just store a MAC alongside each file, what went wrong?
Go ahead
You need to modify the MAC too
But if they modify the file, they do not know how to generate a MAC for their version of the file.
But what they can do is you have this file and then you come and write it, and you generate a new MAC.
When you read it, they give you the old version.
That has the valid signature or MAC on it because you generated that for it.
You all see the problem?
You haven't seen the problem?
What do you mean they give you the old version?
OK, so you have this file.
You generate a MAC, but you-- at some point, you want to update the file.
You want to update this file to this file prime, and generate a new MAC.
Maybe then file double prime, MAC double prime.
In this application, we want freshness.
When you read this file, you want the latest version of the file.
So it should be what you wrote there last time.
But when you are trying to read the file, an attacker can give you this pair.
If you check the MAC, it's going to match.
This is also a valid message MAC pair.
Now, everyone sees the problem.
OK, so what can we do?
Well, one thing we could do is store all these MACs here on your phone.
MAC1, MAC2-- a MAC for every single file.
But if you do that, in fact, we do not need MAC anymore.
We can just use hash.
So I'll say sigma-- I'll use sigmas, but they mean hashes.
This is probably good enough in practice.
I'll say these files are x1, x2, x3, x4.
Now you just create a hash for each of them and store them locally.
And the model here is that an attacker cannot modify files on your own computer or on your own phone.
And then you can download the file.
Match-- compares it with the latest version of the hash, and then you're convinced that it's the latest version.
This is probably a good enough solution.
The only downside is that we do have to store a lot of hashes if you have a lot of files.
Or in our algorithmic terminology, we say your space complexity is o of n. Here, I mean your local space.
So can we somehow reduce the local space complexity?
Well, one thing we could do is to concatenate all the files together, generate a single hash, and store that one hash.
So hash everything in one try.
Then we do have o of 1 space, but there's a bigger problem.
Can anyone tell me?
You don't know which file to modify?
Oh, OK, I think you are thinking in the right thing.
So how do I verify?
I cannot verify a single file.
I have to download all the files and recompute a hash to verify.
So the time complexity is o of n. And, also, if I want to update this file, I have to recompute the hash.
That involves, again, downloading all the files and feed them into that hash.
And we do have a better solution than both of them, which is called a hash tree or a Merkle tree, which was invented by Merkle.
What we will do is so first for every file, we're going to create a hash.
Let me, again, use sigma because h is unclear whether it's a hash value or a hash function.
Sigma 2, sigma 3, sigma 4.
So I said a hash tree.
And guess what's the next step to do?
Cross the hashes
Yeah.
Exactly.
We're going to create a sigma 5, which is the hash of sigma 1, concatenated with sigma 2.
So we do the same thing here.
And so you all know what it is, right?
I don't need to write it.
And keep going until we got a root hash.
And now we're going to store this thing locally on the side.
So what's the local storage complexity?
o of 1-- we're only storing one hash locally.
Now, what's the time complexity?
OK, so how do I verify?
Yeah.
Log in-- how do I verify?
I need to, so, first verify if this hash matches, and then read this hash, and verify whether this link matches, and verify whether this one matches, and then I'm done.
If I want to update, I also need to update this hash, then it causes this hash to change and then that hash to change.
But it's always some path in that tree.
It doesn't affect anything globally.
Question
But you're not storing like sigma 5?
Say again
You're not storing sigma 5
I am not.
I have to go ahead and read it
From where?
From Google Drive or Dropbox
So are we sure that that is secure?
OK, yeah, so that's the next thing we're going to do.
Is this secure?
Or in other words, can the adversary change one of the files, and somehow maintain the same root hash?
That's your question then.
Of course, we assume the hash is collision resistant.
Or I should say if the hash is collision resistant, then this hash tree is collision resistant.
Any intuition?
Or anyone wants to prove it?
Go ahead
So like if the root eventually one of the leaves will be different because it changes
Yep
Now, you want the root to be the same than the other hash has to be different, but there's no collisions.
I mean, it's hard to find the other hash
Correct, so I'll just repeat what you said, but I'll start with the leaf because that's easier for me to think about.
So say I change this one, this block.
Now, I claim this hash here will change.
If it doesn't, then I have found the collision.
Because this x4 prime has the same hash as the original x4.
So if this sigma 4 changes, then sigma 6 will change.
Otherwise, I have found the collision.
Because this sigma 3 concatenate with the new sigma 4 is my collision.
So same argument-- either this one changes, or I have found the collision.
I repeat the argument all the way to the root.
Any question about that?
What if like the adversary changes like two hash options--for example, x1 and x2-- but sigma 1 and sigma 2 changes, but sigma 5 stays the same?
OK, so then we have found the collision that is sigma 1 concatenated with sigma 2.
That's a collision with the new sigma 1 concatenated with the new sigma 2.
Make sense?
If the concatenation stayed the same, like sigma 1 and sigma 2 concatenation
So if the concatenation stayed the same, that means both of them are the same
They had to make sure they are changed?
So I'm not sure I understand your question.
So concatenation is basically just a bunch of bits then followed by another bunch of bits.
If this entire thing is the same, that means this part is the same and this part is the same.
And if your sigma 1, new sigma 1 is the same as your old sigma 1, that means I have found a collision here.
Because we changed it, but your sigma doesn't change.
So lastly, I'm going to do a quick review of the knapsack problem because I think in the lecture, we may run out of time and I didn't mention everything.
So if you recall, the knapsack cryptosystem, it says you have a knapsack problem.
I'll call u1 to un.
And then we're going to transform it.
OK, this is a super increasing sequence.
I'm going to transform into a general one by multiplying n and then mod m. So this is an easy problem, and that is a hard problem.
So how do I encrypt?
I'm going to take a subset sum, which is mi, Wi, where mi is the i-th bit in the message.
So how do I decrypt?
I'll take this, transform this s back to the super increasing domain by multiplying inverse of n. So that's going to be inverse of n multiplied by this mi Wi.
That's how I encrypt it.
And then each Wi is n times ui.
So far so good.
So that gives me mi times ui sigma.
Of course, every step is modulo m. So the first thing I'm going to claim is that m has to be larger than sigma ui.
If that's the case, then the t-- my t is just this subset sum.
So if I solve this knapsack problem, I get the same answer as solving the original, the general knapsack problem.
If my m is not that large, if the m is too small, then I have a problem.
Because then my t will be the subset sum minus sum multiple of m. Then it's a different problem.
I do not get the same message back.
OK, then we have a problem.
So because we defined density to be n over the log of max ui.
Does everyone remember this part?
So each ui is in the range of 1 to m, or maybe 0 to m. If I have a bunch of them, then this is not super rigorous.
If I have a bunch of them, chances are that some of them are very close to m. Because it's unlikely that all of them are small.
So this thing is roughly n over log of m. So then we have a dilemma.
If we set m to be a small number, then my density is fine, but that means all of my ui's needs to be small because m needs to be greater than the sum of them.
If all the ui's are small, then I have a very limited choices of them, then actually an attacker can just guess what ui I chose by a brute force algorithm or something like that.
And if I choose m to be large, or if I choose all the ui's to be large, to choose them from large range, then my m is going to be very large.
And this density is low.
And that's vulnerable to the low density attacks.
And so how low a density is considered low?
So several people proposed that based on heuristics, that if this density is less than 0.45, then it's considered low density, and it can be attacked.
And this threshold had been improved.
So but while most of the knapsack cryptosystems are broken, there are few that have so far stood the test of time.
So they are still interesting because knapsack problems, knapsack cryptosystems will be much faster than RSA or any number theory based, because we are just adding numbers here.
An RSA have this operation where m is a 1,000 bit number, and e is also 1,000 bit number.
And take this exponentiation is actually very slow.
So knapsack cryptosystem are still interesting.
However, the original motivation turned out to be unsuccessful.
The original motivation is to base cryptography on the NP complete problem.
That's not going to work because NP problems are hard, only in the worst case.
And we need cryptography to be hard in the average case.
Because if they are only hard in the worst case, that means there are several instances of this problem that are hard.
So either you pick a secret key that doesn't correspond to a hard problem, or you pick a secret key that's corresponds to a hard problem.
But everyone else picks the same secret key because everyone wants to be secure.
That's the reason why it's unlikely to get cryptography from NP hard problems.
That's all for today's recitation.
And thanks everyone for the entire semester.
Thank you for participation.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to 6.172, Performance Engineering of Software Systems.
This is a fun class.
It's all about going fast.
We all like to go fast-- well, not all of us.
But everybody in this class likes to go fast.
Otherwise you won't like the class very much.
It's all about making stuff go fast.
So a couple of brief announcements.
We're going to be videotaping for OpenCourseWare.
So if you don't want to be videotaped, you should sit it in a place where the camera won't catch you.
Because we're in such a large classroom, I'd like everybody to sort of every day, without me having to draw-- I'm not really an artist.
My sister's an artist, actually.
She's a painter.
But maybe that gene escaped me.
It went that way and not this way.
So I don't want to have to draw it up again.
So everybody, try to confine yourself to this sort of quarter of the classroom.
And that way, it'll feel more like a size room that we ought to be in.
Let me go to our course information sheet.
Did everybody get a copy of the course information handout?
So we're going to the class-- we're going to start at the beginning with the Administrivia stuff, try to run through that as quickly as we can.
And then my co-lecturer, professor Saman Amarasinghe-- if you can't pronounce or spell his last name, you can just say "Saman"-- will give today's lecture.
And we have four teaching assistants.
So teaching assistants, please stand up.
They are Josh, John, Reed, and Eric.
And let me tell you, what they have done in the last few days to get us ready for this course is nothing short of superhuman.
So you really need to accord them your respect, and they're going to really work hard to help you folks do well in this class.
We have, I think, a pretty decent ratio.
I think even with concerns about cutbacks, et cetera, to have four TAs for this is really good.
And we need it, because it's a software class.
And software is, as you know, bugs can take an arbitrarily long time to find.
You guys can sit.
Besides our teaching assistants, we have two support staff people helping us, Mary McDavitt, who works for Saman, and Marcia Davidson, who works for me.
And they're generally pretty good about being able to track us down if we're in some far corner of the world or something.
Our lectures are going to be held on Tuesdays and Thursdays in this room, 2:30 to 4:00 PM.
You should plan to attend regularly.
We dispense all kinds of wonderful gems that you will miss if you think you're going to get it just from reading and even looking at videotapes or whatever.
I'm not sure what our arrangement is going to be about having videos available.
OK, they're not going to be available, except unless there's something really extenuating circumstance.
You have to go into a hospital to have your forebrain removed or something-- something serious, OK?
All material in the lecture, even if it's delivered oral, is fair game for projects and quizzes.
We will post the lecture slides, but they don't generally convey all of the information.
And we're going to be video-recorded, as I mentioned.
If you don't want to appear, sit behind the camera, and you won't appear.
We have no recitations in this class.
However, the teaching assistants will periodically hold primers where we will bring you up to speed on various technologies that you may not be up to speed in, such as C. How many people have programmed in C before?
And how many have not programmed in C before?
A few.
OK. Tools such as VTune.
Who's used VTune before?
Who has not used VTune?
Good.
Pin-- who's used Pin?
OK. Who has used Cilk++?
OK, a couple people.
So you see, we're going to have some primers to bring you up to speed in those kinds of things.
We don't expect that you came in knowing any of those things, particularly.
We're going to be using Stellar for our course management.
And here's the URL.
And you should make sure that you're registered as a member of 6.172, because otherwise, you're not going to get any announcements.
We're going to do all of our administration through Stellar.
So make sure that you're registered there.
At the end of today, I think it is, we'll send out an announcement to the student mailing list.
If you don't get something by the end today-- so you wake up tomorrow morning, you have no email from the course staff.
What that means is you should email us
Well, that mailing list would be the preregistered ones
That'd be the preregistered ones.
And then that way, we'll get all the people.
So if you know you're not preregistered, you can send us email before then.
There are two one-hour quizzes given during class time
One and a half
What's that?
One and a half hour
Oh, you're right.
It's one and a half hour quizzes.
Yes, two one and a half hour quizzes.
We'll correct that in the online version.
Given during class time.
Can somebody, by the way, correct this as we go along?
Great.
The dates are on the course calendar, which is available on Stellar.
We've had mixed results with the ICS feed from Stellar, but you're supposed to be able to download the calendar to your iPhone or other ICS-capable devices.
But we were successful in downloading, but all the times were screwed up.
But anyway, hopefully, maybe they'll allow that.
But anyway, the calendar is there.
The quizzes will generally be closed book and closed notes, but you will be permitted crib sheets, one sheet that you summarize anything that you feel that you need to know.
Generally, everything that we cover in here is fair game for the quizzes, including things that are in prerequisite courses.
There is no final exam because we have a final project.
This is a fast-moving class, so if you get caught behind, it's not a good sign, because it's really hard to work double-hard to keep up in this class.
You really have to keep on top of things.
And so for that reason and others-- because it's really difficult for us-- it's fast moving for us as well-- we generally cannot accept late projects.
If you haven't been able to complete a project, you should just package up your partial solution and hand that in, and you'll get partial credit on it.
Do not just say, oh, I'm not going to hand something in, and expect it to get it done later.
If you do find yourself in an unusual situation which you think there are some extenuating circumstances, like the aforementioned brain surgery or something, let us know in advance.
So if you're going to get in a car accident, it's really helpful if you tell us the day before so that we can plan.
Obviously can't always do that, but if it is something for which there is any reasonable expectation it should be done beforehand, you should let us know.
And we may require, for those kinds of exceptions, some kind of confirmation from a dean or medical professional.
So if that's the route that you're in, you can plan a little bit and make sure that those confirmations are on their way.
The grading, this is essentially approximately how we're going to grade.
As you can see, there's a bunch of content in the projects and a bunch in the quizzes.
And then there's also participation grade.
And the participation is both in-- we will have design reviews and such, so that's part of the participation, as well as asking questions and such in class.
And because we're video-recording everything, we know exactly who's asking questions, unless you choose to sit outside the video zone.
In addition, if there is a significant missing element to your work, it doesn't matter how well you did on the other things, you will receive a failing grade.
You must, essentially, do all the work.
So if you do not get substantial credit on the final project or any other two assignments, for example, you can expect to receive a failing grade.
It goes downhill very quickly.
If you're missing one assignment, generally don't expect to get higher than a C, for example.
It goes downhill very, very quickly, because that's what this is all about is hands on.
What's permissible in this class?
So it's your responsibility to satisfy both the letter and spirit of the rules.
So if you have any questions, let us know.
Here's what the rules are.
Also, let me just say how Saman and I treat this.
We have been involved in cheating incidents off and on over the years.
I've been at MIT almost 30 years.
Saman, you've been here--
13
13 years.
So we've been here long time, longer than you folks.
We've been through this a bunch of times.
It's really a pain when somebody cheats.
It robs me of a huge amount of my time that I could otherwise be spending on teaching and students and my own professional development.
However, we both take cheating incidents extremely seriously.
So we have this thing that is sometimes really inconvenient called integrity.
And I'd rather, if I find somebody's cheating, just give you an F and not deal with it.
Drat.
Instead, when somebody's cheating, I tend to want to take it to the Committee on Discipline, where you face things like expulsion, et cetera.
So I don't like to do that.
It's very time consuming for me.
But I find it hard to just look the other way, because I view the issue of integrity as protecting the vast majority of students who are obeying the rules.
So when a cheating incident occurs, I sort of feel honor bound to push it to its proper conclusion.
So please don't cheat.
We will use a variety of means at our disposal, including technological ones, to detect cheating.
Also, if you think that sharing your results with somebody else means that they're cheating and you're not-- in some cultures that's the way it's treated-- no.
In this culture, in particular the culture in this classroom, they're both equally guilty of academic dishonesty, whether you are the giver or the taker.
Equally, we treat that as equal offense.
So here's the particular things.
You may not share, generally, your solutions with people who are not in your group.
So we will have a bunch of group projects, generally in pairs.
And that includes both people in the class and people outside the class.
So generally, you keep things to yourself.
When you're in a group, of course, you can share ideas within the group.
But we expect that everybody's making a fair contribution.
So you should be concerned if you are not holding up your end of your group participation.
And we're going to ask you to describe the contributions of the people in your group.
Generally, the group, as I say, is two.
So it's basically saying, what did you do?
How did you divide up your project?
What did you do?
What did the other person do?
You generally can't share it with anybody else.
You can't copy or transcribe solutions from other places.
The work you submit must be your own.
Pretty straightforward.
You may go out and use general conceptual material, such that you might get from a good textbook or from Wikipedia or someplace like that.
That's perfectly fine, resources you would ordinarily have available.
But if you do use any material from an external source, you should properly cite it in normal academic that fashion.
Cite where it is so that somebody else can go and find and look to see what it is that you're citing.
If you feel that you have transgressed in some way, let us know.
It always goes way easier.
I generally tend to be kind of a nice guy when somebody comes to me saying, oops, verses I hear from somewhere else or one of our tools detects something else.
Then I'm kind of cranky.
So I'd rather, if you come to me, we could work something out, generally.
Most of your out-of-class time will spent completing six projects.
And there's an outline of what the projects are.
They're all fun projects.
You can ask the TAs.
The TAs will tell you how much fun they are, because they've generally helped to design and built their own solutions to them.
Some will be done in pairs, some individually.
The final project can either be done in pairs or in threes, I think we said
[INAUDIBLE]
Yes, we also have-- we didn't put that in here.
We also have a Project 0, which will be available today after class.
So you should go to Stellar and get Project 0.
Project 0 is not to be handed in.
It's getting warmed up with all of the machines and tools that we'll be using in the class.
We have been very fortunate to have very nice contributions from Dell and Intel of a new Cluster facility that has 16 12-core machines.
And they're Intel Core i7 machines.
Each chip has six cores, and there's two cores per machine, each with 48 gigabytes of memory.
So these are kind of honking modern facility.
So as I say, it's fun.
It's fun when you do something on your laptop, and then you go and you do it on one of these machines and it goes like this.
It's really fun when these things go fast.
So Project 0, you should start on today.
And it's basically the handout called Getting Started.
It will be available on Stellar.
And it will take you through getting access to these machines and so forth.
In particular, in order to use these machines, you must have a CSAIL account.
Who does not have a CSAIL account?
So that is going to be the first thing you need to do, because you've got to get that.
So what you want to do is get registered for a CSAIL account-- and we give instructions in there.
It's basically going to a web page-- pretty much as soon as you get out of the classroom.
Because you won't be able to start anything until you have this CSAIL account.
And hopefully, we'll be able to give you access within a few hours after you register.
Yeah, question?
What operating systems are the tools--
Linux.
We'll be using Linux.
So there are a variety of ways that we describe for how you use these machines remotely.
We have the Andrew File System-- AFS-- which you can use so that you can edit things locally but have an SSH terminal window on the machine, which makes it very convenient.
On the other hand, for some people, often some dorm rooms with poor connectivity, doing things by AFS may be too slow for you.
But there are a variety of other ways that you can-- you can actually edit right on the machine.
And there are a variety of ways of having a repository on your machine, because we'll be using Git as our versioning mechanism.
So you'd be able to have it there and be able to make copies back on the other machine.
So there are a variety of ways to work.
And you're free to make the software run on your own laptops or workstations.
The one thing I'll say is we will not support that.
We don't have the resources to support beyond using the machines that we actually provide.
I think I've got to go faster here.
So let's see.
So software engineering.
So research in programmer productivity has demonstrated that pair programming, where two programmers sit together, one at the keyboard and one looking on, is actually faster than two programmers coding separately.
Why do you suppose that is?
Yeah?
You don't get stuck.
The other person can tell you-- it's like when you're [INAUDIBLE] writing on the blackboard, you get nervous.
You don't know what you're doing.
[INAUDIBLE]
Yeah, you could be stuck.
What else?
You don't procrastinate
Yeah, you don't procrastinate.
That's a good one.
Don't procrastinate
Finding bugs
Yeah, finding bugs.
So the main one is finding bugs, that the person who's looking on finds bugs.
In fact, I was just pair programming with Reed the other day, and we were trying to figure out, why is this thing not working?
And I say, gee, it looks fine.
It says, next y equals x.
And he says, no, next y is supposed to equal y.
And it's like, we had both read over that code, but as soon as I spelled it out, I didn't even notice.
But of course, him in hearing it says, oh, it's supposed to be x and y and y, not x and y.
So anyway, that kind of thing, you pick up very quickly.
It makes it very easy to debug.
Likewise, regression testing demonstrably promotes fast co-development.
So this is where you build a battery of tests that includes tests for every mistake that you've ever made.
And it can also be helpful to write unit tests, where you test individual ones.
And you should also use tools like the assert package.
And some of these things will be taken through in Project 0.
The MIT POSSE.
We have recruited senior software engineers in the region to share with you their invaluable knowledge and experience.
We call them Masters In The Practice Of Software Systems Engineering, which has the great acronym MIT POSSE.
These are people who are not being paid.
And they're senior.
They're doing this because they're volunteering, because they want to help.
You have a lot to learn from these people, and we will have code and design reviews with these people with your assigned master.
You want to treat them nicely.
They're not doing this so that some snotty-nosed kid can say, I don't care about you.
That's not what they're doing.
They're there to help you get a good grade.
And these are some really talented people, people who are very famous in their own areas of expertise.
So be gracious, is the main thing I would say.
Be gracious.
Let's see.
The assignments.
You can read about the assignments.
Basically, generally, what we're going to do is have a beta submission in which we're going to test everything.
And we're also going to take all your tests.
We're going to throw all the tests into a big pool and run everybody's tests against everybody software.
You will lose points if you fail some of the tests, assuming the tests are correct tests.
You will lose points if you have an incorrect test.
You will lose points if somebody passes your test and they have a bug.
You will get points if your test picks up something that very few other groups tested for.
So you have to both submit code and submit tests, which is OK, because you've got to test your own stuff before you hand it in.
But we're also going to be using it to test each other.
After beta, we will have a design and code review with your master.
And let me just say, the master's there not affecting your grade in any way.
They do give us reports about what happened in their meeting, but they have no bearing on the grade.
They're just there to help.
So you don't have to view them as-- they're not part of the course staff.
If you want to get angry at somebody about how things are done, get angry at Saman and me.
Don't get angry at them.
The final submission is generally due a couple weeks after the beta.
And it's after you've gotten feedback on your design, feedback on things.
We'll give you the whole battery of everybody's regression tests, so you can test out your code against everybody's regression test and use that.
And then generally, the scoring is such that people who get fast code get high marks.
People who get slow codes get lower marks.
If you need help, [UNINTELLIGIBLE].
Speed is fun.
Enjoy the class.
Yeah, question?
In terms of the bug testing, I'm thinking of bugs like-- I don't know-- [INAUDIBLE] or something like that.
If you happen to not have made a bug in your own program [INAUDIBLE] would that count against you in the final testing
You have to view your beta test as testing other peoples' code also
But if you [UNINTELLIGIBLE] a test that finds bugs in somebody's program, and if very few people wrote that test, then you will get points.
So you don't have to test for the world, everything.
If you fail somebody's test, but not yours, you will be deducted points.
If your test finds other people's bugs, you will gain points for that.
But on the other hand, if you don't have a test for-- what you said, I think you don't have to go and test for every possible condition in there.
The negative side is, if you miss something, we will deduct points.
Positive side is, if your test finds somebody else's bugs, you'll get points for it.
We'll go into detail exactly how [UNINTELLIGIBLE].
Another thing we did last year, which was kind of interesting-- I think there were mixed-- some people really liked it, some people were not that happy-- is a little bit of competitive grading.
So here, what we are doing is not the full grade-- small part of the grade.
What you do is when you first submit your beta, rerun performance tests of everybody.
And of course, one of you are going to get the fastest code for that.
At that point, we are going to say, OK, this is the fastest code.
And how [UNINTELLIGIBLE] everybody else with that code?
And the person who gets fastest code for that part of the grade gets the full points.
And everybody else will get a partial part of the points.
However, you get a chance in your final submission to basically improve your code.
If you do better than the last time's best code, you won't get any more additional points, because then we don't want to create [?
an arms race.
?]
So you at least had to match the previous times to get full points.
So you get a chance to see how you stand against your peers and get a chance to actually [UNINTELLIGIBLE] math with your peers.
So first time we did it, we said, OK, it's the fraction of your performance against the performance of the best code.
The problem was, there were a couple of codes that 1000x performance improvement, and everybody got 10x.
So you've got, like, 0.25 grade.
That didn't help.
So now we actually are doing a logarithmic log scale.
So we'll get the fraction, and we'll take the log off that.
So there will be a little bit more equality.
So you will know how worse off you are, and you get a chance to [UNINTELLIGIBLE].
Because it is very important, because a lot of times, there might be a crucial thing that you missed.
And instead of trying to be, "aha, got you" in this class, we are trying to give you a chance to actually learn.
So if you missed it, OK, you missed it the first time.
But you get the second chance to come and fix it and see whether you can do well.
And of course, if you do better than the best grade, there are bragging rights, but you don't get extra points.
With that, let me-- where are we?
So again, as Charles pointed out, I'm pretty excited about this class.
This is a really fun class.
Unfortunately, I'm a little bit under the weather.
So I'm going to sit down.
And if I don't sound that enthusiastic, hopefully I'll get enthusiastic later.
And that's one reason we ask you to come forward, because we don't have to come here and shout.
So hopefully people in the back can here.
If you don't, still, there are some seats in the front.
You can move forward.
So for the rest of the class, what we want to do is go through an example and look at why performance matters.
For that, I selected matrix multiply.
And I think your Project 0, you are going to also play with matrix multiply.
And I think that you'll get a feel for what kind of things will happened and how much room you have in doing well in here.
And at the same time, as I go through this, you'll get a feel for what's the process that you might be following through the class and what kind of things-- because linear programming, after taking a bunch of programming classes, probably, you have a methodology set up in your mind, how to go about doing that.
And the same thing after this class, we want you to have a methodology of going through and improving performance.
And this is basically a little bit of a contrite example, but you'll get a feel.
So what's matrix multiply?
It's a very fundamental operation for many computations.
So you are going to produce Matrix A by multiplying Matrix B and C. So you want to calculate each value in here to take a row of B and a column of C, and you get each element of A.
And you do that for every element of A.
And this is the possible simplest code you can write in a language like C, a [?
three-loop nest, ?]
and this matrix multiply.
So, so far, things should be pretty simple.
So what I did was I looked at matrix multiply and said, look, you have taken software engineering class.
You have thought about object-oriented programming, lot of nice techniques to build very useful, portable software.
And let's try to write matrix multiply with some of those concepts.
So the first thing I want to do is matrix multiply write object oriented using immutable classes, and that, in fact, I want matrix class to represent both integers and doubles.
So here's my class structuring here.
And actually, I'm going to show you a bunch of code now, next couple of slides.
I'm not expecting you to follow everything instantaneously.
I'm going to go through somewhat faster.
But you have the slides.
So I always expect you to, perhaps, in some free time, to go and look at some of the slides, get a little bit of better feel for what happened.
So here's what matrix multiply does.
I have a value class that figures out the matrix type.
And if it's integer, it keep integers.
If it's a double, I keep a double.
And here, basically, do this value in here.
And then I have a matrix class.
Basically, given a matrix type and a matrix, it can create a matrix here.
This basically creates your matrix in here by first creating a bunch of rows here, and then instantiating each row in here, whether it's integer or double, depending on the type you want.
And then, of course, there are other things in matrix, basically, in here.
And you can update a matrix.
This is going to be a little bit interesting.
I will give a little bit here, because what I'm going is I'm doing immutable types.
That means I can't change anything in the matrix.
If you are updating, you have to create a new matrix.
So I'll show a little bit about how to go about doing that.
And then there's this row class.
I'm going through this fast, because it's not that important.
If you get a chance, go through this and get a feel for what I'm doing.
But the idea is doing this very software engineering way of implementing a matrix multiply.
So then here's my matrix multiply in here, that what I'm doing is I'm updating each of these matrices by giving a value of B and C and adding it to A and updating A again in here.
So when I run matrix multiply, 1,024 by 1,024 matrix, this is the time I got.
Actually, it took a long time, in fact.
And the key thing, is the performance good?
That's the first question to ask.
Because you run, you get a result.
When the result is correct, OK, great.
You get the correct result.
But how do you do?
And this is sometimes a very hard thing, to know that you have a problem is itself a huge win-- that you figure out you have a problem.
And what I'm going to do is go through a little bit of a calculation to see how we did it.
It took about five hours to do this matrix multiply, and that doesn't look right already.
So if you think about matrix multiply, it's an N-cube operation, basically, what matrix multiply is doing.
That's the algorithmic [?
class.
?]
If you look at Charles' book, that's what it would say.
It's N-cube operation.
So this is the number of operations you want to do.
And for each matrix operation, you only do three index updaters, add, a multiply, and a check-- probably six ops to do, basically.
And that means I need to do this many operations.
Basically, you can think about machine operations to do, basically, because you have to read three values, update one, do a multiply and add, and do a branch.
And if I look at that with my hours, I am doing this many operations per second, which looks high.
Big number.
But my machine, actually, on my somewhat old PC, transfers even much faster.
So if you think about it, I am taking about 8,300 cycles to do any kind of a machine operation.
This doesn't look right.
So at this point, this is what I would call a "backup [UNINTELLIGIBLE] calculation."
I haven't proven anything.
This one worked well in the theory class, but if you can just jot down a couple of these things, you can get a feel for where you stand very fast.
And then you realize, OK, I am a couple of orders of magnitude off here on what best I could do.
And that should give you a feel for what's going on.
OK, so what's going on here?
How can we improve performance?
Because we know we have a problem.
So what we can do is we can look deeply into the program execution.
And at some points, OK, this program runs slow.
[UNINTELLIGIBLE] time spent.
So we can figure out time spent, basically, things like by method-- which method you spend most of the time running.
Sometimes by line.
And we will learn a bunch of profiling tools that will give you this feel.
So instead of reading a million lines of code, looking at every line in there, you can very fast converge onto the culprits where most of the time being spent.
And that's a very easy way of triaging your problem.
And you look at things like time spent, cumulative time spent within that and everything called underneath, number of invocations of the functions.
And a lot times, you have a feel for what it should be.
And when you look at this, if it doesn't look right, you say, wait a minute, this goes against my intuition.
Perhaps there's a problem.
So what you can do is, by doing this, you can identify hot spots.
Basically, if 90% of the time it's it one routine, that's what you want to fix.
And hopefully, that routine is doing something too slow.
Then you can actually get good performance in here.
So here's the profile I ran for a matrix multiply.
And interestingly, what I found was, most of the time spent in basically creating doubles-- number of calls, time spent here, this much time was spending, basically creating new values in here, this initializing a double in here.
So I'm doing matrix multiply.
Why should I be initializing values?
So this is an issue.
And I have the most number of calls to that too.
So we can go look at it and say, hey, that would seem to be a problem here.
So the problem here was kind of obvious.
What happened was I didn't use immutable [?
class.
?]
So this is my matrix representation.
I have a bunch of pointers pointing to each row.
And now assume I won't update this element.
And of course, I can't modify the entire thing.
It's immutable.
So what I had to do is create a copy that updates.
And then, of course, I had to create a new matrix.
I don't have to copy everybody else, because these guys were not changed.
So I can place a pointer to these things and create this new matrix that points to these things.
And then, if you think about what's going on, in fact, and this is my new matrix after I do that, I [?
updated ?]
two end copies for each update, because what I had to do is I had to copy this entire row up to get that.
And I had to create this entire index again for each point in each row.
So just to update one element, I am creating two end copies.
So what that means is now our N cube matrix multiply ends up being N to the power of 4 algorithm.
So suddenly, you realize, wait a minute, I'm doing something bad in here.
I'm actually [UNINTELLIGIBLE] N-cube algorithm, and the way I saw, it becomes n to the power of 4 because I'm making all these copies in there.
So this is a bad thing to do, of course.
And copying is very costly, because you're making duplicates in here.
And of course, you're creating a huge amount of garbage, because you are just copying and leaving the previous value for garbage collection.
So garbage collector's coming up all the time, and huge memory footprint.
So this is no good.
And can we do better?
Hopefully, we can.
So the next thing we did was, OK, just get rid of these immutable types.
That seemed to be a [UNINTELLIGIBLE] here.
And then, I actually have-- I will go faster over the code, so if you are interested, you can go look at the code in here.
So here, what we did was, still we have the issues like object oriented and ability to represent both integers and doubles.
So what that means is each matrix can be the integer matrix or a double matrix.
So this is where you're instantiating with integers or doubles in here.
So we do that.
And how much you think this can improve performance by doing that?
In fact, this was pretty nice.
I actually got a 219x performance improvement because I went from n to the power 4 algorithm to n to the power 3 algorithm.
So here is interesting, algorithm has changed with it, and we have a pretty nice performance improvement.
So if I look at this now, what I find is I'm spending a lot of method time, I'm doing matrix multiply.
That's good.
But I'm spending all this time doing get double and get methods.
I'm trying to do matrix multiply.
Why am I spending all this time trying to do get double and get?
What I found was issue is the method call overhead.
Now, what happened is matrix row has two different matrix types, the integer row and double row.
So every time I want to get a row, I don't know if I have integer row or double row.
I have to do a look-up to figure out my type.
Am I working with the integer matrix or matrix in doubles?
And this is what we call "dynamic dispatch."
So instead of knowing exactly the methods we are to go, I had to actually look up, OK, which method I'm supposed to go.
I have to do that since, if the matrix is integer, I might have to call a different method than matrix is double.
I have to check that.
And then that call, instead of direct call, becomes indirect branch.
I have to check that and basically branch on that.
And these indirect branches are costly, because what that means is I don't where I am going until I do that test.
So no machine-- we'll go into details and pipelines and stuff as we go on.
What that means is normally, a machine can have a lot of instruction in flight.
So that means that I have to know a lot more information to get the instructions going.
But here, until you resolve the instructions, you can't do the next one.
So it's called a, basically, pipeline stall.
So instead of having hundreds of instructions going, basically, suddenly I can't do anything until the previous instruction is done.
So the machine slows down a lot because of this.
Normally, what you can do is keep fetching next instructions here.
We have to fetch, test, then the test result is available, then that's when you can fetch the next instruction.
Let me go through that file.
So direct batch means target address is known.
You can fetch ahead of the targets/ If I know where I am going, even if I don't get there, I can fetch ahead of the target.
I can go start fetching what happens next ahead, basically, into [?
the branch.
?]
Sometimes, you can even fetch both directions, or the direction you might always go might get fetched by the architect.
We'll talk about a lot of architecture details in the future lectures.
Indirect branch means until I calculate the address, I have no idea where I'm going.
That address is only calculated.
And because of that, hardware slows down like crazy.
So this seems to be not a good thing to have these integers and doubles instead of having one nice class of just [UNINTELLIGIBLE] might be nice.
I just basically create a double class.
It's all matrices are doubles.
If I want integers, I will just rewrite my matrix class or copy and replace.
Now my matrix class becomes much more simpler.
This is what I got, and this is my row in here.
And voila, minute I do that, I basically went up about a 2.4x performance improvement, because now I don't have to keep doing that.
And altogether now, I have improved my performance by about 500x here.
And for ops per cycle, we have gone down from 8,000 to 38 to 16.
So we do still better.
And now we look at, again, my profile data.
And what I see is, OK, I am spending a lot of time in matrix multiply.
That's OK.
But also, I am spending time in this matrix get method.
I am trying to get each, basically, row and data elements of the matrix.
So here is the summary of these things.
I'm going to just keep that one, because I gave you [INAUDIBLE].
So the problem in this object-orientedness is I [?
handle ?]
each row is represented separately-- nice objects.
These objects overload the memory.
And in fact, if I want to get to my row, I have to appoint a [?
chase ?]
to go get that.
It's not contiguous in memory.
And to get to every row, I have to actually go through this [?
point ?]
indexing in here.
[UNINTELLIGIBLE] matrices are one, nice, simple objects.
So we can say, OK, wait a minute.
This method call overhead actually keeps dominating.
And because I have done this nice object-oriented thing in here, so I can get to the object-oriented file here and say, OK, get to the objects.
And I write this nice loop of [UNINTELLIGIBLE].
And I have two-dimensional matrices in here.
And voila, I got another 2.2x performance improvement in here.
OK, now I am down to about seven cycles for each operation in here.
That's good.
So what can we do here?
So we did all of these things in Java, because it's supposed to be a nice language.
But Java has a lot of overhead.
It's like you do memory-bound checks.
You have to do this bytecode interpreter and stuff like that.
On the other hand, C can run much faster.
You don't have to do any kind of memory-bound checks.
The compilation happens before you run, so it is not part of your cost of doing anything.
So this can work very much like C. You can just almost cut and paste.
And if you know, C is [?
index chain, ?]
[?
and this ?]
to C is just changing a few lines.
And I did that, and this is, basically, the C code in here.
This is the matrix multiply.
This is the kind of allocation in here.
C can be a little bit painful to allocate.
This is my matrix multiply.
It will look similar.
And I got 2.1x just by going from Java to C. This, I was surprised, because that piece of Java code looked very much close to C. But by just doing that, I actually got another 2.1x performance gain.
Now this is kind of the high-level stuff.
Now we have to go to nitty-gritty details in the hardware.
So in modern hardware, there are these things called "performance counters" that hardware [UNINTELLIGIBLE] that we can look at what's happening inside hardware.
We can get a feel for what happens inside hardware.
And there are things called CPI, which clock cycles per instruction.
So that means if the instructions are running very slow, you will have multiple clock cycles per instructions.
That means instructions are stalling.
Something is wrong.
Cache misses.
So data [UNINTELLIGIBLE] [?
pay attention to ?]
caches and stuff like that.
If data are supposed to be in cache and have cache hits all the time, but if you are getting cache misses, something is wrong.
Accesses are not good.
And then also the instructions retired.
That means, how many instructions are being executed and retired?
If you are doing too much work, the number of instructions retired will go high in here.
This will give you a few how many [?
work been done.
?]
So if you look at this one, I have a CPI of 4.
That means for every instruction, need about almost close to five clock cycles, which is not good.
I have pretty high cache miss rate, too-- L1 cache miss rate and some L2 cache miss rate.
This shows how many instruction are in what they call [?
vector in the ?]
instruction form called SSE instructions.
And here's the number of instructions got retired.
So we just use that baseline case.
So one thing I can look at was my cache miss rates seem to be pretty darn high.
What might be happening here?
So if you look at matrix multiply-- so I have Matrix A in here and Matrix B-- what happens is, to calculate this value in [?
Matrix A ?]
here, basically, I'm going to go through this row in here, and I'm going through column in here.
Problem with the column is it's all [?
low in ?]
the cache.
This is your linear memory.
So because this is here two dimensional, but inside your hardware, there's no two-dimensional memory.
You have a linear memory, so this is your linear memory.
This is how the memory look like.
And this is how the memory look like.
So each row and column is in these areas.
But here, to calculate this value, this is nice.
I go through memory that is contiguous.
But C, I am jumping all over my memory.
And jumping on memory is not that great cache behavior.
And contiguous can do much better, because what you do is, when you get a cache, you get a cache line that has more than one word.
So in here, what you're doing is you are getting large cache lines only using one word before you go to the next one.
So that's not great in cache behavior.
So what you can do is you can make this memory contiguous.
So what you can do is, in matrix multiply, normally, n to the cubed computation to the squared data.
So to help on n to the cubed, it's OK to do some n to the squared processing.
That means you can do a data transport before you start matrix multiply.
Do some pre-processing.
So this will also come in handy.
A lot of these problems you'll do, you'll say, wait a minute, can I do a pre-process that's cheaper that will really improve my main processing?
This is a theme that comes again and again.
And then we can get this n to the cubed running faster since this n to the cubed is much larger than n to the squared.
You can amortize the cost of doing that.
And basically, to get that, we can transpose some matrix going into the squared operations.
And then now n to the cubed might run much faster.
And in fact, what I have done here is do the transpose of this matrix.
And now we run this one, and then everything will be in contiguous memory.
And voila, I got another 3.4x performance improvement.
Now I am running almost at one cycle per op, which is really nice.
But modern superscalars actually can do better, so we can actually end up being even better.
So let's see.
So we are doing that.
So here is the interesting thing.
Now I look at my transpose [?
spot.
?]
So if you look at that number of retired instructions, it's more or less the same.
Number of SSE instructions, more or less the same.
I have a 2x improvement in my L1 cache miss rate.
But more than that, I have a 5x improvement in the CPI.
That means my instructions are stalling a lot less now than before.
So this is where I got all my performance [UNINTELLIGIBLE], because my instructions are running much faster.
So by looking at data-- this is the kind of things you're learning during the class-- then you can get a feel for what actually is happening and why you are getting that.
You can go do more in the cache, in memory system.
So remember, the memory system, if you're [UNINTELLIGIBLE], if you look at cache, what they're doing is we have a small amount of memory called cache that are very fast access.
And you have a large amount of memory that is sitting out that has slow access.
So what you want to give user feel as if they have a lot of memory, everything is very fast access.
So hardware does a lot of crazy things to give you the illusion that you have very fast memory and you have a huge amount of that.
Of course, that's not true, because you have these caches.
And what happens is, if you do things wrong, these get to slow down.
So in the cache system, you have cache [UNINTELLIGIBLE] You have each multicore going to its own L1 cache that goes L2 cache, which goes to L3 cache, which goes to the main memory system.
And you want all of the data to be here.
If that's the case, things run very fast.
If not, you have to go here.
It will be a lot more slow, here, even slower, here, very slow.
So the key thing is to get the data as much as possible here.
So here's the kind of cycles, basically.
One cycle to get here-- three cycles.
If you go to L2 to L3, you have to do 14 cycles [UNINTELLIGIBLE].
If you go to L3 to main memory, you might sometimes get hundreds of cycles delaying here, so it's very costly.
So the caches are very temperamental beasts.
They work beautifully, but when they don't work, these go really haywire.
So a lot of times, if there's a performance issue, look at the cache miss rate and say, aha, the caches are not behaving.
And can I help in here?
So the interesting way to look at that is how much data I touch.
These graphics didn't show up.
Let me explain.
So if you do matrix multiply, I am trying to calculate one value in here.
So this is one value.
I have to calculate row and the column.
This slide got screwed up a little bit.
So just look at this calculating [?
row bar.
?]
So if I want to calculate a row of values here, what I have to do is I have to get this row in B, and I have to use entire C to calculate one row of A.
That means I have touched this much data items just to get one row of A.
So that means I am calculating 1,024 values of A. I am touching this much data items to get that.
On the other hand, if I do block matrix multiply, that means I calculate, basically, a block in here.
This block is also 32 by 32.
It has 1,024 elements in here.
But I only need a block here and block here.
Voila, I only have looked at 25,000 elements.
I got the same number of output calculated.
But I have only touched much less elements altogether to calculate that.
Do you see that?
And nice thing is, if this thing fits in the cache, I am much better than this might not fit in the cache.
So I can calculate blocks in matrix multiply by looking at a lot less number of data, that has a good chance at fitting in cache-- and of course, when you calculate.
you make sure it fits in cache-- than trying to just calculate row by row.
So one thing you can do is-- this is a lot of common things we do.
There's many ways to get same results.
You can change the execution order.
You can change the algorithm.
You contain data structures.
And some of them actually might do better than the other.
And here, we can actually say we can do the execution of the change and make sure we are only looking at small amount of data, you get the same number of data calculated.
And of course, sometimes you have to be careful, because if you do things in different order, you might not get the exact same results.
So if you are a numerical [UNINTELLIGIBLE], you say, OK, of course, if you're doing A plus B plus C, what order you do will make a difference [UNINTELLIGIBLE].
So if you're really careful, you can't do too many things.
But most times, you can do some of these things.
And in fact, in our final thing, we might say-- the final project-- we will give you something that you can change the algorithm as long as the outcome of the [?
image ?]
is the same.
If you look at the [?
image, ?]
and the [?
image ?]
outcome is the same, you can go change things.
So a lot of times, you might have a lot of freedom to change the amount of errors you get, the precision, and, in fact, doing that, that you can actually get a lot of good performance.
So in here, we are not trying to change the matrix multiply, of course.
But we are going to change the way order of operations is being done.
So some people say you can't do that.
It would make matrix multiply different.
But in most cases, it's OK.
So what we have done is done what we call block matrix multiply.
So instead of calculating rows, you calculate blocks at a time.
So calculating each block, you only need a few amount of data.
So this is block matrix multiply.
And voila, by doing block matrix multiply, I got even 1.7x performance gain from that.
Now, it's very interesting.
Now with each block cycle, I am doing two operations.
So my [?
superscale ?]
is finally coming to being.
So every clock cycle, I am doing multiple operations here.
So this is very interesting, the number here.
So now when you look at what happens here, I am actually executing a little bit more instructions here.
Why?
Because I did block matrix multiply.
That means I am doing a little bit more overhead in here.
I have more loops in here.
My inner loops are small.
So I'm doing more instructions to do execute there.
So I'm doing more instructions, but I have a huge cache performance gain.
My cache miss rate now going down to very, very little-- 0.02%.
And I have an 8x improvement in cache, and that means I got a 3x improvement in my CPI in here.
That's very nice.
That means even though I'm doing a little bit more work, my cache behavior really, really helped me in here.
So kind of says why I am getting this performance gain.
So what else can we do?
We can go look at trying to do a lot more crazy optimizations because if you look at the modern Intel, they have this thing called [?
retro ?]
instructions.
We'll get a little bit more detail into that later.
And what we can do sometimes is you can nudge the compiler to take advantage of these instructions.
Of course, if nothing helps, you can actually go write to assembly [UNINTELLIGIBLE].
But hey, I don't want you guys to write assembly code, at least in this class, even though you can get really good performance sometimes.
But you can kind of give enough hints to compiler and pray and kind of [UNINTELLIGIBLE] there's no nice way on these things, and hope for the compiler to do the right thing.
So in here, we can play with, a lot of times, compiler flags.
And this is not exact size.
We keep trying those things, and sometimes we can even get information back and say, OK, what did the compiler do?
It might give you some good hints and say, I couldn't do something because of this part of the code.
And worst comes to worst, you can actually generate assembly and stare at it.
And sometimes in this class, you might actually want to stare at assembly and see what goes on in here.
So here is the assembly for here.
And this is my loop body in here.
And then what I can see, I'm not going to go through that.
Everything is converted into SSE instructions.
At least I know what SSE instructions mean, so this seems to be doing well, so I'm happy.
And in fact, I'm really happy because I got a 2.8x improvement by doing this.
Now what I am actually doing each clock cycle about [?
five of my ?]
operations.
So I'm going really fast in here.
So now, incidentally, I'm running my program about 30,000x faster than when I started.
So if you haven't noticed, these things slowly add up in here.
And now, if we look at what's going on here, it's very interesting.
What I have done is actually, my each instruction is now CPI has gone down.
That means each instruction is running slower because I'm doing these SSE instructions.
And in fact, my cache miss rate has also gone up.
But I am doing a lot less instructions, because SSE means in one instruction, I am doing a vector.
So I'm actually getting four operations done in one instruction.
So the number of instructions I execute has drastically gone down.
So even though my instructions are running a little bit slower, I can basically amortize that, and I get good performance in here.
And of course, this one says, OK, now 88% of the instructions are SSEs.
So I have [UNINTELLIGIBLE] most of the things to run on vector mode in here.
And of course, you can keep doing more and more and more.
And there are these four things [UNINTELLIGIBLE] matrix multiply, there are people who spend their entire lifetime trying to run it faster.
And they have libraries like that, and this thing called a BLAS library and this thing called Intel Math Kernel.
So they have done a lot more other things, like pre-fetching, getting everything right.
So here what I do is basically call their library, don't do anything myself, and this library will do it for me.
And in fact, that library itself gave me another 2.7x performance gain in there.
Now I'm actually doing 11 instructions every clock cycle by doing that.
And if I look at it carefully, what they have done is they are also very into SSE heavy, so they are running almost the same amount of instructions.
But they managed to actually get, again, a better miss rate.
They probably figured out some stuff in here, things like pre-fetching instructions.
And that means they actually brought down the CPI back to where it was before.
And that's why they gave that performance [INAUDIBLE].
And finally, we are doing a little bit of parallel execution.
So as you know, multicores are here.
And you're actually going to these 12-core machines, which is really nice.
Fastest possible machines you can get your hands on these days.
And it's kind of fun to work with them.
And we can basically get concurrency for parallel performance.
And a large part of this class at the end is trying to get good parallel performance.
But there are a lot of issues with parallelism.
I will go a couple of issues now.
Of course, when we get to that part, we will go in detail.
One thing is called Amdahl's Law.
Because if you're trying to run something parallel, you can't run everything parallel.
When you start something, there's part of the code that basically starts the computation.
And then there are the parts you can run parallel, and then other parts you have to also wind down, measure, things like that.
So what Amdahl's Law says is, even if you have an infinite number of processors, the maximum speedup you can get is this-- this the part of the code that are sequential and [?
parts of ?]
the code are parallel.
That means if you have only 90% of the code can be parallelized, the maximum speedup you can get is 10, even if you have an infinite number of processors.
So 99% of the code is parallel, the maximum speedup you can get is 100.
So this is very interesting, because even if you have huge [UNINTELLIGIBLE], if your code has a certain amount of sequential part, there is a limit of what you can do.
[UNINTELLIGIBLE] also think of load balancing.
That means when you get parallel, what you are assuming is all processes are busy all the time.
That's very hard to achieve.
Sometimes some processes are busy.
Sometimes others are not.
Sometimes something's finished early.
So if that happens, even though you have parallels, you might not get really good performance in here.
And then, of course, there's a thing called granularity of parallelism.
Normally, what happens in programming, you run sequential part, you run parallel part, you run sequential parallel part.
To start parallelism is a big task.
You have to tell all the other processors, OK, look, you guys have some work.
Go do that, and finish that, and come back.
And that itself is very expensive.
And the work you give is very small.
You give the work.
The start-up and tear-down cost of parallelism itself might be too much than the speedup you get.
In fact, you can make programs parallel and slow them down really drastically.
So you have to be very careful in that, because you can say, oh, I'm running in parallel, but it's running slow.
Why?
Because you might have a granularity issue, because we are giving so little work that even though you're getting parallelism, you might not get that much performance.
So if you look at matrix multiply, what happens here is you can divide the matrix into two different parts in here and calculate as two different matrix multiplies in a different core or a different process.
Or in here, I can make it two or whatever in matrix multiplies to do that.
So this is nicely parallelized.
No big issue in here.
And when I parallelize, I got rather 3.5x performance improvement.
Of course, I could have gotten more here, but because I had to start here, since I didn't want to wait for a week for this to finish, I had to come up with a smaller matrix size.
But now, when I get here, this is so fast, my granularity is too small.
So that's why I didn't get-- because this was an eight-core machine, I didn't get 8x because the minute I get parallelism, it just is over because it's running so fast in here.
Now, at this point, using eight cores, I am actually getting 36.
So this is not that interesting, because this is not running on one core, every clock cycle, I am trying to get 36 operations done.
So the interesting thing to follow in here is that this example might be somewhat contrived, because I started with this immutable-type matrix multiply.
Hopefully, you and your colleagues who haven't taken this class probably won't go that far.
But even if you start somewhere here, still, there's a huge gain to be made.
So to put this in perspective, OK, so if you look at here, even if I start here somewhere out in C, I already got 131x performance improvement.
That's huge.
So that means in this class, we are not talking about small things.
And in fact, this is why I had to go log scale in doing absolute grades, because we had people who figured out exactly how, in some problems, to change data structures, change execution ordering, do some really cool thing, get 1,000x performance improvement.
And there are other people who did little bit things and got 2x improvement.
So the fact that what you can do the best here can be really, really high.
So that's why we actually started giving you the second chance to also go basically see, if you missed a critical point, to go back and learn that.
So to put this in perspective-- so in summary, what we have done is-- matrix multiply might be an exception.
Every time you go, you might not get that much, but in here, going from this very software-engineered, nice [UNINTELLIGIBLE] to a BLAS [UNINTELLIGIBLE], I got a 296,000 times performance improvement.
This is huge.
If you put this in perspective, if you look at miles per gallon between these two-- a supertanker and this scooter-- it's only 14,000x, basically.
So what that means is you have to have 20 supertankers to basically match this much improvement.
So if somebody can tell you, look, I can't get a supertanker to run at the same miles per gallon as your scooter, that would be revolutionary.
But you can do something similar in your computer.
And in fact, if you look at these large data centers, the computers are actually burning a lot of energy, a lot of cost.
And in fact, if you can reduce that, that can have impact.
So what you get out of this class is to think through these programs this way.
Learn about correctness before [UNINTELLIGIBLE] 6.170, you hopefully are good at writing correct programs.
But that's not good enough in a lot of times.
And how to think about programs, how to make it run faster, there are multiple [UNINTELLIGIBLE] necessary.
So for example, in here, it's basically direct energy.
So can you get the data center to burn 10% less energy, that's huge.
For example, if you look at today's supercomputing centers, they spend about $10 million a year on energy, just for the power grid.
So if you can say, I'm [?
playing with ?]
10%, that's huge.
Other thing, one thing Charles says a lot of times is, what performance gives you is currency, because performance itself might not be that directly useful, but it gives you currency to buy something else.
What it is?
So assume you have a nice application.
Your GUI is doing OK.
It's up to the point that people don't feel stalled.
But if you want to do something addition, if you add a feature, you don't want to slow down the program too much, because your users are going to complain.
So what performance gives you is, if you improve the performance, it gives you currency, the ability to buy some additional features into your system.
Or you have a system that you created a start up, and you created this nice server somewhere.
Suddenly, it became very popular, and a million people want to use your software.
And the software might not be ready to do that.
It might just crash and burn, or you might have to pay a lot of money for Amazon for the [?
cycles, ?]
so by actually making programs faster, you can make it scalable, you can make it efficient.
So this gives you the ability, this currency, to go do a lot more other interesting things to your software because now you can run it faster.
It gives you this margin to play with.
So in this class, we are going to learn a lot about these things.
We are going to learn about architecture, how looking at the architectural issues, how to identify when something is wrong.
We are going to look at algorithmic issues in here, how to look at things.
We are looking at architectural things like memory systems, parallelism.
And also we will talk about all the tools, so you will actually know something has gone wrong.
Because today, you run the program, it runs, it runs correctly, is it good enough or not?
How do you identify that?
That's, I think, the very interesting place to start.
So that's all I have today.
Make sure you get a Project 0.
And especially people who haven't done C, we will-- Charles, are we going to do a C remediation?
We did last time, didn't we?
OK, guys, one more thing.
Last year, we did an evening C remediation class.
So if you know Java, if you want to know C, we will send out a time with TAs that will give you kind of a one-hour crash course into C in there.
And we will send information.
So this class moves very fast.
Get on with Project 0, and see you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to 6.172, Performance Engineering of Software Systems.
This is a fun class.
It's all about going fast.
We all like to go fast-- well, not all of us.
But everybody in this class likes to go fast.
Otherwise you won't like the class very much.
It's all about making stuff go fast.
So a couple of brief announcements.
We're going to be videotaping for OpenCourseWare.
So if you don't want to be videotaped, you should sit it in a place where the camera won't catch you.
Because we're in such a large classroom, I'd like everybody to sort of every day, without me having to draw-- I'm not really an artist.
My sister's an artist, actually.
She's a painter.
But maybe that gene escaped me.
It went that way and not this way.
So I don't want to have to draw it up again.
So everybody, try to confine yourself to this sort of quarter of the classroom.
And that way, it'll feel more like a size room that we ought to be in.
Let me go to our course information sheet.
Did everybody get a copy of the course information handout?
So we're going to the class-- we're going to start at the beginning with the Administrivia stuff, try to run through that as quickly as we can.
And then my co-lecturer, professor Saman Amarasinghe-- if you can't pronounce or spell his last name, you can just say "Saman"-- will give today's lecture.
And we have four teaching assistants.
So teaching assistants, please stand up.
They are Josh, John, Reed, and Eric.
And let me tell you, what they have done in the last few days to get us ready for this course is nothing short of superhuman.
So you really need to accord them your respect, and they're going to really work hard to help you folks do well in this class.
We have, I think, a pretty decent ratio.
I think even with concerns about cutbacks, et cetera, to have four TAs for this is really good.
And we need it, because it's a software class.
And software is, as you know, bugs can take an arbitrarily long time to find.
You guys can sit.
Besides our teaching assistants, we have two support staff people helping us, Mary McDavitt, who works for Saman, and Marcia Davidson, who works for me.
And they're generally pretty good about being able to track us down if we're in some far corner of the world or something.
Our lectures are going to be held on Tuesdays and Thursdays in this room, 2:30 to 4:00 PM.
You should plan to attend regularly.
We dispense all kinds of wonderful gems that you will miss if you think you're going to get it just from reading and even looking at videotapes or whatever.
I'm not sure what our arrangement is going to be about having videos available.
OK, they're not going to be available, except unless there's something really extenuating circumstance.
You have to go into a hospital to have your forebrain removed or something-- something serious, OK?
All material in the lecture, even if it's delivered oral, is fair game for projects and quizzes.
We will post the lecture slides, but they don't generally convey all of the information.
And we're going to be video-recorded, as I mentioned.
If you don't want to appear, sit behind the camera, and you won't appear.
We have no recitations in this class.
However, the teaching assistants will periodically hold primers where we will bring you up to speed on various technologies that you may not be up to speed in, such as C. How many people have programmed in C before?
And how many have not programmed in C before?
A few.
OK. Tools such as VTune.
Who's used VTune before?
Who has not used VTune?
Good.
Pin-- who's used Pin?
OK. Who has used Cilk++?
OK, a couple people.
So you see, we're going to have some primers to bring you up to speed in those kinds of things.
We don't expect that you came in knowing any of those things, particularly.
We're going to be using Stellar for our course management.
And here's the URL.
And you should make sure that you're registered as a member of 6.172, because otherwise, you're not going to get any announcements.
We're going to do all of our administration through Stellar.
So make sure that you're registered there.
At the end of today, I think it is, we'll send out an announcement to the student mailing list.
If you don't get something by the end today-- so you wake up tomorrow morning, you have no email from the course staff.
What that means is you should email us
Well, that mailing list would be the preregistered ones
That'd be the preregistered ones.
And then that way, we'll get all the people.
So if you know you're not preregistered, you can send us email before then.
There are two one-hour quizzes given during class time
One and a half
What's that?
One and a half hour
Oh, you're right.
It's one and a half hour quizzes.
Yes, two one and a half hour quizzes.
We'll correct that in the online version.
Given during class time.
Can somebody, by the way, correct this as we go along?
Great.
The dates are on the course calendar, which is available on Stellar.
We've had mixed results with the ICS feed from Stellar, but you're supposed to be able to download the calendar to your iPhone or other ICS-capable devices.
But we were successful in downloading, but all the times were screwed up.
But anyway, hopefully, maybe they'll allow that.
But anyway, the calendar is there.
The quizzes will generally be closed book and closed notes, but you will be permitted crib sheets, one sheet that you summarize anything that you feel that you need to know.
Generally, everything that we cover in here is fair game for the quizzes, including things that are in prerequisite courses.
There is no final exam because we have a final project.
This is a fast-moving class, so if you get caught behind, it's not a good sign, because it's really hard to work double-hard to keep up in this class.
You really have to keep on top of things.
And so for that reason and others-- because it's really difficult for us-- it's fast moving for us as well-- we generally cannot accept late projects.
If you haven't been able to complete a project, you should just package up your partial solution and hand that in, and you'll get partial credit on it.
Do not just say, oh, I'm not going to hand something in, and expect it to get it done later.
If you do find yourself in an unusual situation which you think there are some extenuating circumstances, like the aforementioned brain surgery or something, let us know in advance.
So if you're going to get in a car accident, it's really helpful if you tell us the day before so that we can plan.
Obviously can't always do that, but if it is something for which there is any reasonable expectation it should be done beforehand, you should let us know.
And we may require, for those kinds of exceptions, some kind of confirmation from a dean or medical professional.
So if that's the route that you're in, you can plan a little bit and make sure that those confirmations are on their way.
The grading, this is essentially approximately how we're going to grade.
As you can see, there's a bunch of content in the projects and a bunch in the quizzes.
And then there's also participation grade.
And the participation is both in-- we will have design reviews and such, so that's part of the participation, as well as asking questions and such in class.
And because we're video-recording everything, we know exactly who's asking questions, unless you choose to sit outside the video zone.
In addition, if there is a significant missing element to your work, it doesn't matter how well you did on the other things, you will receive a failing grade.
You must, essentially, do all the work.
So if you do not get substantial credit on the final project or any other two assignments, for example, you can expect to receive a failing grade.
It goes downhill very quickly.
If you're missing one assignment, generally don't expect to get higher than a C, for example.
It goes downhill very, very quickly, because that's what this is all about is hands on.
What's permissible in this class?
So it's your responsibility to satisfy both the letter and spirit of the rules.
So if you have any questions, let us know.
Here's what the rules are.
Also, let me just say how Saman and I treat this.
We have been involved in cheating incidents off and on over the years.
I've been at MIT almost 30 years.
Saman, you've been here--
13
13 years.
So we've been here long time, longer than you folks.
We've been through this a bunch of times.
It's really a pain when somebody cheats.
It robs me of a huge amount of my time that I could otherwise be spending on teaching and students and my own professional development.
However, we both take cheating incidents extremely seriously.
So we have this thing that is sometimes really inconvenient called integrity.
And I'd rather, if I find somebody's cheating, just give you an F and not deal with it.
Drat.
Instead, when somebody's cheating, I tend to want to take it to the Committee on Discipline, where you face things like expulsion, et cetera.
So I don't like to do that.
It's very time consuming for me.
But I find it hard to just look the other way, because I view the issue of integrity as protecting the vast majority of students who are obeying the rules.
So when a cheating incident occurs, I sort of feel honor bound to push it to its proper conclusion.
So please don't cheat.
We will use a variety of means at our disposal, including technological ones, to detect cheating.
Also, if you think that sharing your results with somebody else means that they're cheating and you're not-- in some cultures that's the way it's treated-- no.
In this culture, in particular the culture in this classroom, they're both equally guilty of academic dishonesty, whether you are the giver or the taker.
Equally, we treat that as equal offense.
So here's the particular things.
You may not share, generally, your solutions with people who are not in your group.
So we will have a bunch of group projects, generally in pairs.
And that includes both people in the class and people outside the class.
So generally, you keep things to yourself.
When you're in a group, of course, you can share ideas within the group.
But we expect that everybody's making a fair contribution.
So you should be concerned if you are not holding up your end of your group participation.
And we're going to ask you to describe the contributions of the people in your group.
Generally, the group, as I say, is two.
So it's basically saying, what did you do?
How did you divide up your project?
What did you do?
What did the other person do?
You generally can't share it with anybody else.
You can't copy or transcribe solutions from other places.
The work you submit must be your own.
Pretty straightforward.
You may go out and use general conceptual material, such that you might get from a good textbook or from Wikipedia or someplace like that.
That's perfectly fine, resources you would ordinarily have available.
But if you do use any material from an external source, you should properly cite it in normal academic that fashion.
Cite where it is so that somebody else can go and find and look to see what it is that you're citing.
If you feel that you have transgressed in some way, let us know.
It always goes way easier.
I generally tend to be kind of a nice guy when somebody comes to me saying, oops, verses I hear from somewhere else or one of our tools detects something else.
Then I'm kind of cranky.
So I'd rather, if you come to me, we could work something out, generally.
Most of your out-of-class time will spent completing six projects.
And there's an outline of what the projects are.
They're all fun projects.
You can ask the TAs.
The TAs will tell you how much fun they are, because they've generally helped to design and built their own solutions to them.
Some will be done in pairs, some individually.
The final project can either be done in pairs or in threes, I think we said
[INAUDIBLE]
Yes, we also have-- we didn't put that in here.
We also have a Project 0, which will be available today after class.
So you should go to Stellar and get Project 0.
Project 0 is not to be handed in.
It's getting warmed up with all of the machines and tools that we'll be using in the class.
We have been very fortunate to have very nice contributions from Dell and Intel of a new Cluster facility that has 16 12-core machines.
And they're Intel Core i7 machines.
Each chip has six cores, and there's two cores per machine, each with 48 gigabytes of memory.
So these are kind of honking modern facility.
So as I say, it's fun.
It's fun when you do something on your laptop, and then you go and you do it on one of these machines and it goes like this.
It's really fun when these things go fast.
So Project 0, you should start on today.
And it's basically the handout called Getting Started.
It will be available on Stellar.
And it will take you through getting access to these machines and so forth.
In particular, in order to use these machines, you must have a CSAIL account.
Who does not have a CSAIL account?
So that is going to be the first thing you need to do, because you've got to get that.
So what you want to do is get registered for a CSAIL account-- and we give instructions in there.
It's basically going to a web page-- pretty much as soon as you get out of the classroom.
Because you won't be able to start anything until you have this CSAIL account.
And hopefully, we'll be able to give you access within a few hours after you register.
Yeah, question?
What operating systems are the tools--
Linux.
We'll be using Linux.
So there are a variety of ways that we describe for how you use these machines remotely.
We have the Andrew File System-- AFS-- which you can use so that you can edit things locally but have an SSH terminal window on the machine, which makes it very convenient.
On the other hand, for some people, often some dorm rooms with poor connectivity, doing things by AFS may be too slow for you.
But there are a variety of other ways that you can-- you can actually edit right on the machine.
And there are a variety of ways of having a repository on your machine, because we'll be using Git as our versioning mechanism.
So you'd be able to have it there and be able to make copies back on the other machine.
So there are a variety of ways to work.
And you're free to make the software run on your own laptops or workstations.
The one thing I'll say is we will not support that.
We don't have the resources to support beyond using the machines that we actually provide.
I think I've got to go faster here.
So let's see.
So software engineering.
So research in programmer productivity has demonstrated that pair programming, where two programmers sit together, one at the keyboard and one looking on, is actually faster than two programmers coding separately.
Why do you suppose that is?
Yeah?
You don't get stuck.
The other person can tell you-- it's like when you're [INAUDIBLE] writing on the blackboard, you get nervous.
You don't know what you're doing.
[INAUDIBLE]
Yeah, you could be stuck.
What else?
You don't procrastinate
Yeah, you don't procrastinate.
That's a good one.
Don't procrastinate
Finding bugs
Yeah, finding bugs.
So the main one is finding bugs, that the person who's looking on finds bugs.
In fact, I was just pair programming with Reed the other day, and we were trying to figure out, why is this thing not working?
And I say, gee, it looks fine.
It says, next y equals x.
And he says, no, next y is supposed to equal y.
And it's like, we had both read over that code, but as soon as I spelled it out, I didn't even notice.
But of course, him in hearing it says, oh, it's supposed to be x and y and y, not x and y.
So anyway, that kind of thing, you pick up very quickly.
It makes it very easy to debug.
Likewise, regression testing demonstrably promotes fast co-development.
So this is where you build a battery of tests that includes tests for every mistake that you've ever made.
And it can also be helpful to write unit tests, where you test individual ones.
And you should also use tools like the assert package.
And some of these things will be taken through in Project 0.
The MIT POSSE.
We have recruited senior software engineers in the region to share with you their invaluable knowledge and experience.
We call them Masters In The Practice Of Software Systems Engineering, which has the great acronym MIT POSSE.
These are people who are not being paid.
And they're senior.
They're doing this because they're volunteering, because they want to help.
You have a lot to learn from these people, and we will have code and design reviews with these people with your assigned master.
You want to treat them nicely.
They're not doing this so that some snotty-nosed kid can say, I don't care about you.
That's not what they're doing.
They're there to help you get a good grade.
And these are some really talented people, people who are very famous in their own areas of expertise.
So be gracious, is the main thing I would say.
Be gracious.
Let's see.
The assignments.
You can read about the assignments.
Basically, generally, what we're going to do is have a beta submission in which we're going to test everything.
And we're also going to take all your tests.
We're going to throw all the tests into a big pool and run everybody's tests against everybody software.
You will lose points if you fail some of the tests, assuming the tests are correct tests.
You will lose points if you have an incorrect test.
You will lose points if somebody passes your test and they have a bug.
You will get points if your test picks up something that very few other groups tested for.
So you have to both submit code and submit tests, which is OK, because you've got to test your own stuff before you hand it in.
But we're also going to be using it to test each other.
After beta, we will have a design and code review with your master.
And let me just say, the master's there not affecting your grade in any way.
They do give us reports about what happened in their meeting, but they have no bearing on the grade.
They're just there to help.
So you don't have to view them as-- they're not part of the course staff.
If you want to get angry at somebody about how things are done, get angry at Saman and me.
Don't get angry at them.
The final submission is generally due a couple weeks after the beta.
And it's after you've gotten feedback on your design, feedback on things.
We'll give you the whole battery of everybody's regression tests, so you can test out your code against everybody's regression test and use that.
And then generally, the scoring is such that people who get fast code get high marks.
People who get slow codes get lower marks.
If you need help, [UNINTELLIGIBLE].
Speed is fun.
Enjoy the class.
Yeah, question?
In terms of the bug testing, I'm thinking of bugs like-- I don't know-- [INAUDIBLE] or something like that.
If you happen to not have made a bug in your own program [INAUDIBLE] would that count against you in the final testing
You have to view your beta test as testing other peoples' code also
But if you [UNINTELLIGIBLE] a test that finds bugs in somebody's program, and if very few people wrote that test, then you will get points.
So you don't have to test for the world, everything.
If you fail somebody's test, but not yours, you will be deducted points.
If your test finds other people's bugs, you will gain points for that.
But on the other hand, if you don't have a test for-- what you said, I think you don't have to go and test for every possible condition in there.
The negative side is, if you miss something, we will deduct points.
Positive side is, if your test finds somebody else's bugs, you'll get points for it.
We'll go into detail exactly how [UNINTELLIGIBLE].
Another thing we did last year, which was kind of interesting-- I think there were mixed-- some people really liked it, some people were not that happy-- is a little bit of competitive grading.
So here, what we are doing is not the full grade-- small part of the grade.
What you do is when you first submit your beta, rerun performance tests of everybody.
And of course, one of you are going to get the fastest code for that.
At that point, we are going to say, OK, this is the fastest code.
And how [UNINTELLIGIBLE] everybody else with that code?
And the person who gets fastest code for that part of the grade gets the full points.
And everybody else will get a partial part of the points.
However, you get a chance in your final submission to basically improve your code.
If you do better than the last time's best code, you won't get any more additional points, because then we don't want to create [?
an arms race.
?]
So you at least had to match the previous times to get full points.
So you get a chance to see how you stand against your peers and get a chance to actually [UNINTELLIGIBLE] math with your peers.
So first time we did it, we said, OK, it's the fraction of your performance against the performance of the best code.
The problem was, there were a couple of codes that 1000x performance improvement, and everybody got 10x.
So you've got, like, 0.25 grade.
That didn't help.
So now we actually are doing a logarithmic log scale.
So we'll get the fraction, and we'll take the log off that.
So there will be a little bit more equality.
So you will know how worse off you are, and you get a chance to [UNINTELLIGIBLE].
Because it is very important, because a lot of times, there might be a crucial thing that you missed.
And instead of trying to be, "aha, got you" in this class, we are trying to give you a chance to actually learn.
So if you missed it, OK, you missed it the first time.
But you get the second chance to come and fix it and see whether you can do well.
And of course, if you do better than the best grade, there are bragging rights, but you don't get extra points.
With that, let me-- where are we?
So again, as Charles pointed out, I'm pretty excited about this class.
This is a really fun class.
Unfortunately, I'm a little bit under the weather.
So I'm going to sit down.
And if I don't sound that enthusiastic, hopefully I'll get enthusiastic later.
And that's one reason we ask you to come forward, because we don't have to come here and shout.
So hopefully people in the back can here.
If you don't, still, there are some seats in the front.
You can move forward.
So for the rest of the class, what we want to do is go through an example and look at why performance matters.
For that, I selected matrix multiply.
And I think your Project 0, you are going to also play with matrix multiply.
And I think that you'll get a feel for what kind of things will happened and how much room you have in doing well in here.
And at the same time, as I go through this, you'll get a feel for what's the process that you might be following through the class and what kind of things-- because linear programming, after taking a bunch of programming classes, probably, you have a methodology set up in your mind, how to go about doing that.
And the same thing after this class, we want you to have a methodology of going through and improving performance.
And this is basically a little bit of a contrite example, but you'll get a feel.
So what's matrix multiply?
It's a very fundamental operation for many computations.
So you are going to produce Matrix A by multiplying Matrix B and C. So you want to calculate each value in here to take a row of B and a column of C, and you get each element of A.
And you do that for every element of A.
And this is the possible simplest code you can write in a language like C, a [?
three-loop nest, ?]
and this matrix multiply.
So, so far, things should be pretty simple.
So what I did was I looked at matrix multiply and said, look, you have taken software engineering class.
You have thought about object-oriented programming, lot of nice techniques to build very useful, portable software.
And let's try to write matrix multiply with some of those concepts.
So the first thing I want to do is matrix multiply write object oriented using immutable classes, and that, in fact, I want matrix class to represent both integers and doubles.
So here's my class structuring here.
And actually, I'm going to show you a bunch of code now, next couple of slides.
I'm not expecting you to follow everything instantaneously.
I'm going to go through somewhat faster.
But you have the slides.
So I always expect you to, perhaps, in some free time, to go and look at some of the slides, get a little bit of better feel for what happened.
So here's what matrix multiply does.
I have a value class that figures out the matrix type.
And if it's integer, it keep integers.
If it's a double, I keep a double.
And here, basically, do this value in here.
And then I have a matrix class.
Basically, given a matrix type and a matrix, it can create a matrix here.
This basically creates your matrix in here by first creating a bunch of rows here, and then instantiating each row in here, whether it's integer or double, depending on the type you want.
And then, of course, there are other things in matrix, basically, in here.
And you can update a matrix.
This is going to be a little bit interesting.
I will give a little bit here, because what I'm going is I'm doing immutable types.
That means I can't change anything in the matrix.
If you are updating, you have to create a new matrix.
So I'll show a little bit about how to go about doing that.
And then there's this row class.
I'm going through this fast, because it's not that important.
If you get a chance, go through this and get a feel for what I'm doing.
But the idea is doing this very software engineering way of implementing a matrix multiply.
So then here's my matrix multiply in here, that what I'm doing is I'm updating each of these matrices by giving a value of B and C and adding it to A and updating A again in here.
So when I run matrix multiply, 1,024 by 1,024 matrix, this is the time I got.
Actually, it took a long time, in fact.
And the key thing, is the performance good?
That's the first question to ask.
Because you run, you get a result.
When the result is correct, OK, great.
You get the correct result.
But how do you do?
And this is sometimes a very hard thing, to know that you have a problem is itself a huge win-- that you figure out you have a problem.
And what I'm going to do is go through a little bit of a calculation to see how we did it.
It took about five hours to do this matrix multiply, and that doesn't look right already.
So if you think about matrix multiply, it's an N-cube operation, basically, what matrix multiply is doing.
That's the algorithmic [?
class.
?]
If you look at Charles' book, that's what it would say.
It's N-cube operation.
So this is the number of operations you want to do.
And for each matrix operation, you only do three index updaters, add, a multiply, and a check-- probably six ops to do, basically.
And that means I need to do this many operations.
Basically, you can think about machine operations to do, basically, because you have to read three values, update one, do a multiply and add, and do a branch.
And if I look at that with my hours, I am doing this many operations per second, which looks high.
Big number.
But my machine, actually, on my somewhat old PC, transfers even much faster.
So if you think about it, I am taking about 8,300 cycles to do any kind of a machine operation.
This doesn't look right.
So at this point, this is what I would call a "backup [UNINTELLIGIBLE] calculation."
I haven't proven anything.
This one worked well in the theory class, but if you can just jot down a couple of these things, you can get a feel for where you stand very fast.
And then you realize, OK, I am a couple of orders of magnitude off here on what best I could do.
And that should give you a feel for what's going on.
OK, so what's going on here?
How can we improve performance?
Because we know we have a problem.
So what we can do is we can look deeply into the program execution.
And at some points, OK, this program runs slow.
[UNINTELLIGIBLE] time spent.
So we can figure out time spent, basically, things like by method-- which method you spend most of the time running.
Sometimes by line.
And we will learn a bunch of profiling tools that will give you this feel.
So instead of reading a million lines of code, looking at every line in there, you can very fast converge onto the culprits where most of the time being spent.
And that's a very easy way of triaging your problem.
And you look at things like time spent, cumulative time spent within that and everything called underneath, number of invocations of the functions.
And a lot times, you have a feel for what it should be.
And when you look at this, if it doesn't look right, you say, wait a minute, this goes against my intuition.
Perhaps there's a problem.
So what you can do is, by doing this, you can identify hot spots.
Basically, if 90% of the time it's it one routine, that's what you want to fix.
And hopefully, that routine is doing something too slow.
Then you can actually get good performance in here.
So here's the profile I ran for a matrix multiply.
And interestingly, what I found was, most of the time spent in basically creating doubles-- number of calls, time spent here, this much time was spending, basically creating new values in here, this initializing a double in here.
So I'm doing matrix multiply.
Why should I be initializing values?
So this is an issue.
And I have the most number of calls to that too.
So we can go look at it and say, hey, that would seem to be a problem here.
So the problem here was kind of obvious.
What happened was I didn't use immutable [?
class.
?]
So this is my matrix representation.
I have a bunch of pointers pointing to each row.
And now assume I won't update this element.
And of course, I can't modify the entire thing.
It's immutable.
So what I had to do is create a copy that updates.
And then, of course, I had to create a new matrix.
I don't have to copy everybody else, because these guys were not changed.
So I can place a pointer to these things and create this new matrix that points to these things.
And then, if you think about what's going on, in fact, and this is my new matrix after I do that, I [?
updated ?]
two end copies for each update, because what I had to do is I had to copy this entire row up to get that.
And I had to create this entire index again for each point in each row.
So just to update one element, I am creating two end copies.
So what that means is now our N cube matrix multiply ends up being N to the power of 4 algorithm.
So suddenly, you realize, wait a minute, I'm doing something bad in here.
I'm actually [UNINTELLIGIBLE] N-cube algorithm, and the way I saw, it becomes n to the power of 4 because I'm making all these copies in there.
So this is a bad thing to do, of course.
And copying is very costly, because you're making duplicates in here.
And of course, you're creating a huge amount of garbage, because you are just copying and leaving the previous value for garbage collection.
So garbage collector's coming up all the time, and huge memory footprint.
So this is no good.
And can we do better?
Hopefully, we can.
So the next thing we did was, OK, just get rid of these immutable types.
That seemed to be a [UNINTELLIGIBLE] here.
And then, I actually have-- I will go faster over the code, so if you are interested, you can go look at the code in here.
So here, what we did was, still we have the issues like object oriented and ability to represent both integers and doubles.
So what that means is each matrix can be the integer matrix or a double matrix.
So this is where you're instantiating with integers or doubles in here.
So we do that.
And how much you think this can improve performance by doing that?
In fact, this was pretty nice.
I actually got a 219x performance improvement because I went from n to the power 4 algorithm to n to the power 3 algorithm.
So here is interesting, algorithm has changed with it, and we have a pretty nice performance improvement.
So if I look at this now, what I find is I'm spending a lot of method time, I'm doing matrix multiply.
That's good.
But I'm spending all this time doing get double and get methods.
I'm trying to do matrix multiply.
Why am I spending all this time trying to do get double and get?
What I found was issue is the method call overhead.
Now, what happened is matrix row has two different matrix types, the integer row and double row.
So every time I want to get a row, I don't know if I have integer row or double row.
I have to do a look-up to figure out my type.
Am I working with the integer matrix or matrix in doubles?
And this is what we call "dynamic dispatch."
So instead of knowing exactly the methods we are to go, I had to actually look up, OK, which method I'm supposed to go.
I have to do that since, if the matrix is integer, I might have to call a different method than matrix is double.
I have to check that.
And then that call, instead of direct call, becomes indirect branch.
I have to check that and basically branch on that.
And these indirect branches are costly, because what that means is I don't where I am going until I do that test.
So no machine-- we'll go into details and pipelines and stuff as we go on.
What that means is normally, a machine can have a lot of instruction in flight.
So that means that I have to know a lot more information to get the instructions going.
But here, until you resolve the instructions, you can't do the next one.
So it's called a, basically, pipeline stall.
So instead of having hundreds of instructions going, basically, suddenly I can't do anything until the previous instruction is done.
So the machine slows down a lot because of this.
Normally, what you can do is keep fetching next instructions here.
We have to fetch, test, then the test result is available, then that's when you can fetch the next instruction.
Let me go through that file.
So direct batch means target address is known.
You can fetch ahead of the targets/ If I know where I am going, even if I don't get there, I can fetch ahead of the target.
I can go start fetching what happens next ahead, basically, into [?
the branch.
?]
Sometimes, you can even fetch both directions, or the direction you might always go might get fetched by the architect.
We'll talk about a lot of architecture details in the future lectures.
Indirect branch means until I calculate the address, I have no idea where I'm going.
That address is only calculated.
And because of that, hardware slows down like crazy.
So this seems to be not a good thing to have these integers and doubles instead of having one nice class of just [UNINTELLIGIBLE] might be nice.
I just basically create a double class.
It's all matrices are doubles.
If I want integers, I will just rewrite my matrix class or copy and replace.
Now my matrix class becomes much more simpler.
This is what I got, and this is my row in here.
And voila, minute I do that, I basically went up about a 2.4x performance improvement, because now I don't have to keep doing that.
And altogether now, I have improved my performance by about 500x here.
And for ops per cycle, we have gone down from 8,000 to 38 to 16.
So we do still better.
And now we look at, again, my profile data.
And what I see is, OK, I am spending a lot of time in matrix multiply.
That's OK.
But also, I am spending time in this matrix get method.
I am trying to get each, basically, row and data elements of the matrix.
So here is the summary of these things.
I'm going to just keep that one, because I gave you [INAUDIBLE].
So the problem in this object-orientedness is I [?
handle ?]
each row is represented separately-- nice objects.
These objects overload the memory.
And in fact, if I want to get to my row, I have to appoint a [?
chase ?]
to go get that.
It's not contiguous in memory.
And to get to every row, I have to actually go through this [?
point ?]
indexing in here.
[UNINTELLIGIBLE] matrices are one, nice, simple objects.
So we can say, OK, wait a minute.
This method call overhead actually keeps dominating.
And because I have done this nice object-oriented thing in here, so I can get to the object-oriented file here and say, OK, get to the objects.
And I write this nice loop of [UNINTELLIGIBLE].
And I have two-dimensional matrices in here.
And voila, I got another 2.2x performance improvement in here.
OK, now I am down to about seven cycles for each operation in here.
That's good.
So what can we do here?
So we did all of these things in Java, because it's supposed to be a nice language.
But Java has a lot of overhead.
It's like you do memory-bound checks.
You have to do this bytecode interpreter and stuff like that.
On the other hand, C can run much faster.
You don't have to do any kind of memory-bound checks.
The compilation happens before you run, so it is not part of your cost of doing anything.
So this can work very much like C. You can just almost cut and paste.
And if you know, C is [?
index chain, ?]
[?
and this ?]
to C is just changing a few lines.
And I did that, and this is, basically, the C code in here.
This is the matrix multiply.
This is the kind of allocation in here.
C can be a little bit painful to allocate.
This is my matrix multiply.
It will look similar.
And I got 2.1x just by going from Java to C. This, I was surprised, because that piece of Java code looked very much close to C. But by just doing that, I actually got another 2.1x performance gain.
Now this is kind of the high-level stuff.
Now we have to go to nitty-gritty details in the hardware.
So in modern hardware, there are these things called "performance counters" that hardware [UNINTELLIGIBLE] that we can look at what's happening inside hardware.
We can get a feel for what happens inside hardware.
And there are things called CPI, which clock cycles per instruction.
So that means if the instructions are running very slow, you will have multiple clock cycles per instructions.
That means instructions are stalling.
Something is wrong.
Cache misses.
So data [UNINTELLIGIBLE] [?
pay attention to ?]
caches and stuff like that.
If data are supposed to be in cache and have cache hits all the time, but if you are getting cache misses, something is wrong.
Accesses are not good.
And then also the instructions retired.
That means, how many instructions are being executed and retired?
If you are doing too much work, the number of instructions retired will go high in here.
This will give you a few how many [?
work been done.
?]
So if you look at this one, I have a CPI of 4.
That means for every instruction, need about almost close to five clock cycles, which is not good.
I have pretty high cache miss rate, too-- L1 cache miss rate and some L2 cache miss rate.
This shows how many instruction are in what they call [?
vector in the ?]
instruction form called SSE instructions.
And here's the number of instructions got retired.
So we just use that baseline case.
So one thing I can look at was my cache miss rates seem to be pretty darn high.
What might be happening here?
So if you look at matrix multiply-- so I have Matrix A in here and Matrix B-- what happens is, to calculate this value in [?
Matrix A ?]
here, basically, I'm going to go through this row in here, and I'm going through column in here.
Problem with the column is it's all [?
low in ?]
the cache.
This is your linear memory.
So because this is here two dimensional, but inside your hardware, there's no two-dimensional memory.
You have a linear memory, so this is your linear memory.
This is how the memory look like.
And this is how the memory look like.
So each row and column is in these areas.
But here, to calculate this value, this is nice.
I go through memory that is contiguous.
But C, I am jumping all over my memory.
And jumping on memory is not that great cache behavior.
And contiguous can do much better, because what you do is, when you get a cache, you get a cache line that has more than one word.
So in here, what you're doing is you are getting large cache lines only using one word before you go to the next one.
So that's not great in cache behavior.
So what you can do is you can make this memory contiguous.
So what you can do is, in matrix multiply, normally, n to the cubed computation to the squared data.
So to help on n to the cubed, it's OK to do some n to the squared processing.
That means you can do a data transport before you start matrix multiply.
Do some pre-processing.
So this will also come in handy.
A lot of these problems you'll do, you'll say, wait a minute, can I do a pre-process that's cheaper that will really improve my main processing?
This is a theme that comes again and again.
And then we can get this n to the cubed running faster since this n to the cubed is much larger than n to the squared.
You can amortize the cost of doing that.
And basically, to get that, we can transpose some matrix going into the squared operations.
And then now n to the cubed might run much faster.
And in fact, what I have done here is do the transpose of this matrix.
And now we run this one, and then everything will be in contiguous memory.
And voila, I got another 3.4x performance improvement.
Now I am running almost at one cycle per op, which is really nice.
But modern superscalars actually can do better, so we can actually end up being even better.
So let's see.
So we are doing that.
So here is the interesting thing.
Now I look at my transpose [?
spot.
?]
So if you look at that number of retired instructions, it's more or less the same.
Number of SSE instructions, more or less the same.
I have a 2x improvement in my L1 cache miss rate.
But more than that, I have a 5x improvement in the CPI.
That means my instructions are stalling a lot less now than before.
So this is where I got all my performance [UNINTELLIGIBLE], because my instructions are running much faster.
So by looking at data-- this is the kind of things you're learning during the class-- then you can get a feel for what actually is happening and why you are getting that.
You can go do more in the cache, in memory system.
So remember, the memory system, if you're [UNINTELLIGIBLE], if you look at cache, what they're doing is we have a small amount of memory called cache that are very fast access.
And you have a large amount of memory that is sitting out that has slow access.
So what you want to give user feel as if they have a lot of memory, everything is very fast access.
So hardware does a lot of crazy things to give you the illusion that you have very fast memory and you have a huge amount of that.
Of course, that's not true, because you have these caches.
And what happens is, if you do things wrong, these get to slow down.
So in the cache system, you have cache [UNINTELLIGIBLE] You have each multicore going to its own L1 cache that goes L2 cache, which goes to L3 cache, which goes to the main memory system.
And you want all of the data to be here.
If that's the case, things run very fast.
If not, you have to go here.
It will be a lot more slow, here, even slower, here, very slow.
So the key thing is to get the data as much as possible here.
So here's the kind of cycles, basically.
One cycle to get here-- three cycles.
If you go to L2 to L3, you have to do 14 cycles [UNINTELLIGIBLE].
If you go to L3 to main memory, you might sometimes get hundreds of cycles delaying here, so it's very costly.
So the caches are very temperamental beasts.
They work beautifully, but when they don't work, these go really haywire.
So a lot of times, if there's a performance issue, look at the cache miss rate and say, aha, the caches are not behaving.
And can I help in here?
So the interesting way to look at that is how much data I touch.
These graphics didn't show up.
Let me explain.
So if you do matrix multiply, I am trying to calculate one value in here.
So this is one value.
I have to calculate row and the column.
This slide got screwed up a little bit.
So just look at this calculating [?
row bar.
?]
So if I want to calculate a row of values here, what I have to do is I have to get this row in B, and I have to use entire C to calculate one row of A.
That means I have touched this much data items just to get one row of A.
So that means I am calculating 1,024 values of A. I am touching this much data items to get that.
On the other hand, if I do block matrix multiply, that means I calculate, basically, a block in here.
This block is also 32 by 32.
It has 1,024 elements in here.
But I only need a block here and block here.
Voila, I only have looked at 25,000 elements.
I got the same number of output calculated.
But I have only touched much less elements altogether to calculate that.
Do you see that?
And nice thing is, if this thing fits in the cache, I am much better than this might not fit in the cache.
So I can calculate blocks in matrix multiply by looking at a lot less number of data, that has a good chance at fitting in cache-- and of course, when you calculate.
you make sure it fits in cache-- than trying to just calculate row by row.
So one thing you can do is-- this is a lot of common things we do.
There's many ways to get same results.
You can change the execution order.
You can change the algorithm.
You contain data structures.
And some of them actually might do better than the other.
And here, we can actually say we can do the execution of the change and make sure we are only looking at small amount of data, you get the same number of data calculated.
And of course, sometimes you have to be careful, because if you do things in different order, you might not get the exact same results.
So if you are a numerical [UNINTELLIGIBLE], you say, OK, of course, if you're doing A plus B plus C, what order you do will make a difference [UNINTELLIGIBLE].
So if you're really careful, you can't do too many things.
But most times, you can do some of these things.
And in fact, in our final thing, we might say-- the final project-- we will give you something that you can change the algorithm as long as the outcome of the [?
image ?]
is the same.
If you look at the [?
image, ?]
and the [?
image ?]
outcome is the same, you can go change things.
So a lot of times, you might have a lot of freedom to change the amount of errors you get, the precision, and, in fact, doing that, that you can actually get a lot of good performance.
So in here, we are not trying to change the matrix multiply, of course.
But we are going to change the way order of operations is being done.
So some people say you can't do that.
It would make matrix multiply different.
But in most cases, it's OK.
So what we have done is done what we call block matrix multiply.
So instead of calculating rows, you calculate blocks at a time.
So calculating each block, you only need a few amount of data.
So this is block matrix multiply.
And voila, by doing block matrix multiply, I got even 1.7x performance gain from that.
Now, it's very interesting.
Now with each block cycle, I am doing two operations.
So my [?
superscale ?]
is finally coming to being.
So every clock cycle, I am doing multiple operations here.
So this is very interesting, the number here.
So now when you look at what happens here, I am actually executing a little bit more instructions here.
Why?
Because I did block matrix multiply.
That means I am doing a little bit more overhead in here.
I have more loops in here.
My inner loops are small.
So I'm doing more instructions to do execute there.
So I'm doing more instructions, but I have a huge cache performance gain.
My cache miss rate now going down to very, very little-- 0.02%.
And I have an 8x improvement in cache, and that means I got a 3x improvement in my CPI in here.
That's very nice.
That means even though I'm doing a little bit more work, my cache behavior really, really helped me in here.
So kind of says why I am getting this performance gain.
So what else can we do?
We can go look at trying to do a lot more crazy optimizations because if you look at the modern Intel, they have this thing called [?
retro ?]
instructions.
We'll get a little bit more detail into that later.
And what we can do sometimes is you can nudge the compiler to take advantage of these instructions.
Of course, if nothing helps, you can actually go write to assembly [UNINTELLIGIBLE].
But hey, I don't want you guys to write assembly code, at least in this class, even though you can get really good performance sometimes.
But you can kind of give enough hints to compiler and pray and kind of [UNINTELLIGIBLE] there's no nice way on these things, and hope for the compiler to do the right thing.
So in here, we can play with, a lot of times, compiler flags.
And this is not exact size.
We keep trying those things, and sometimes we can even get information back and say, OK, what did the compiler do?
It might give you some good hints and say, I couldn't do something because of this part of the code.
And worst comes to worst, you can actually generate assembly and stare at it.
And sometimes in this class, you might actually want to stare at assembly and see what goes on in here.
So here is the assembly for here.
And this is my loop body in here.
And then what I can see, I'm not going to go through that.
Everything is converted into SSE instructions.
At least I know what SSE instructions mean, so this seems to be doing well, so I'm happy.
And in fact, I'm really happy because I got a 2.8x improvement by doing this.
Now what I am actually doing each clock cycle about [?
five of my ?]
operations.
So I'm going really fast in here.
So now, incidentally, I'm running my program about 30,000x faster than when I started.
So if you haven't noticed, these things slowly add up in here.
And now, if we look at what's going on here, it's very interesting.
What I have done is actually, my each instruction is now CPI has gone down.
That means each instruction is running slower because I'm doing these SSE instructions.
And in fact, my cache miss rate has also gone up.
But I am doing a lot less instructions, because SSE means in one instruction, I am doing a vector.
So I'm actually getting four operations done in one instruction.
So the number of instructions I execute has drastically gone down.
So even though my instructions are running a little bit slower, I can basically amortize that, and I get good performance in here.
And of course, this one says, OK, now 88% of the instructions are SSEs.
So I have [UNINTELLIGIBLE] most of the things to run on vector mode in here.
And of course, you can keep doing more and more and more.
And there are these four things [UNINTELLIGIBLE] matrix multiply, there are people who spend their entire lifetime trying to run it faster.
And they have libraries like that, and this thing called a BLAS library and this thing called Intel Math Kernel.
So they have done a lot more other things, like pre-fetching, getting everything right.
So here what I do is basically call their library, don't do anything myself, and this library will do it for me.
And in fact, that library itself gave me another 2.7x performance gain in there.
Now I'm actually doing 11 instructions every clock cycle by doing that.
And if I look at it carefully, what they have done is they are also very into SSE heavy, so they are running almost the same amount of instructions.
But they managed to actually get, again, a better miss rate.
They probably figured out some stuff in here, things like pre-fetching instructions.
And that means they actually brought down the CPI back to where it was before.
And that's why they gave that performance [INAUDIBLE].
And finally, we are doing a little bit of parallel execution.
So as you know, multicores are here.
And you're actually going to these 12-core machines, which is really nice.
Fastest possible machines you can get your hands on these days.
And it's kind of fun to work with them.
And we can basically get concurrency for parallel performance.
And a large part of this class at the end is trying to get good parallel performance.
But there are a lot of issues with parallelism.
I will go a couple of issues now.
Of course, when we get to that part, we will go in detail.
One thing is called Amdahl's Law.
Because if you're trying to run something parallel, you can't run everything parallel.
When you start something, there's part of the code that basically starts the computation.
And then there are the parts you can run parallel, and then other parts you have to also wind down, measure, things like that.
So what Amdahl's Law says is, even if you have an infinite number of processors, the maximum speedup you can get is this-- this the part of the code that are sequential and [?
parts of ?]
the code are parallel.
That means if you have only 90% of the code can be parallelized, the maximum speedup you can get is 10, even if you have an infinite number of processors.
So 99% of the code is parallel, the maximum speedup you can get is 100.
So this is very interesting, because even if you have huge [UNINTELLIGIBLE], if your code has a certain amount of sequential part, there is a limit of what you can do.
[UNINTELLIGIBLE] also think of load balancing.
That means when you get parallel, what you are assuming is all processes are busy all the time.
That's very hard to achieve.
Sometimes some processes are busy.
Sometimes others are not.
Sometimes something's finished early.
So if that happens, even though you have parallels, you might not get really good performance in here.
And then, of course, there's a thing called granularity of parallelism.
Normally, what happens in programming, you run sequential part, you run parallel part, you run sequential parallel part.
To start parallelism is a big task.
You have to tell all the other processors, OK, look, you guys have some work.
Go do that, and finish that, and come back.
And that itself is very expensive.
And the work you give is very small.
You give the work.
The start-up and tear-down cost of parallelism itself might be too much than the speedup you get.
In fact, you can make programs parallel and slow them down really drastically.
So you have to be very careful in that, because you can say, oh, I'm running in parallel, but it's running slow.
Why?
Because you might have a granularity issue, because we are giving so little work that even though you're getting parallelism, you might not get that much performance.
So if you look at matrix multiply, what happens here is you can divide the matrix into two different parts in here and calculate as two different matrix multiplies in a different core or a different process.
Or in here, I can make it two or whatever in matrix multiplies to do that.
So this is nicely parallelized.
No big issue in here.
And when I parallelize, I got rather 3.5x performance improvement.
Of course, I could have gotten more here, but because I had to start here, since I didn't want to wait for a week for this to finish, I had to come up with a smaller matrix size.
But now, when I get here, this is so fast, my granularity is too small.
So that's why I didn't get-- because this was an eight-core machine, I didn't get 8x because the minute I get parallelism, it just is over because it's running so fast in here.
Now, at this point, using eight cores, I am actually getting 36.
So this is not that interesting, because this is not running on one core, every clock cycle, I am trying to get 36 operations done.
So the interesting thing to follow in here is that this example might be somewhat contrived, because I started with this immutable-type matrix multiply.
Hopefully, you and your colleagues who haven't taken this class probably won't go that far.
But even if you start somewhere here, still, there's a huge gain to be made.
So to put this in perspective, OK, so if you look at here, even if I start here somewhere out in C, I already got 131x performance improvement.
That's huge.
So that means in this class, we are not talking about small things.
And in fact, this is why I had to go log scale in doing absolute grades, because we had people who figured out exactly how, in some problems, to change data structures, change execution ordering, do some really cool thing, get 1,000x performance improvement.
And there are other people who did little bit things and got 2x improvement.
So the fact that what you can do the best here can be really, really high.
So that's why we actually started giving you the second chance to also go basically see, if you missed a critical point, to go back and learn that.
So to put this in perspective-- so in summary, what we have done is-- matrix multiply might be an exception.
Every time you go, you might not get that much, but in here, going from this very software-engineered, nice [UNINTELLIGIBLE] to a BLAS [UNINTELLIGIBLE], I got a 296,000 times performance improvement.
This is huge.
If you put this in perspective, if you look at miles per gallon between these two-- a supertanker and this scooter-- it's only 14,000x, basically.
So what that means is you have to have 20 supertankers to basically match this much improvement.
So if somebody can tell you, look, I can't get a supertanker to run at the same miles per gallon as your scooter, that would be revolutionary.
But you can do something similar in your computer.
And in fact, if you look at these large data centers, the computers are actually burning a lot of energy, a lot of cost.
And in fact, if you can reduce that, that can have impact.
So what you get out of this class is to think through these programs this way.
Learn about correctness before [UNINTELLIGIBLE] 6.170, you hopefully are good at writing correct programs.
But that's not good enough in a lot of times.
And how to think about programs, how to make it run faster, there are multiple [UNINTELLIGIBLE] necessary.
So for example, in here, it's basically direct energy.
So can you get the data center to burn 10% less energy, that's huge.
For example, if you look at today's supercomputing centers, they spend about $10 million a year on energy, just for the power grid.
So if you can say, I'm [?
playing with ?]
10%, that's huge.
Other thing, one thing Charles says a lot of times is, what performance gives you is currency, because performance itself might not be that directly useful, but it gives you currency to buy something else.
What it is?
So assume you have a nice application.
Your GUI is doing OK.
It's up to the point that people don't feel stalled.
But if you want to do something addition, if you add a feature, you don't want to slow down the program too much, because your users are going to complain.
So what performance gives you is, if you improve the performance, it gives you currency, the ability to buy some additional features into your system.
Or you have a system that you created a start up, and you created this nice server somewhere.
Suddenly, it became very popular, and a million people want to use your software.
And the software might not be ready to do that.
It might just crash and burn, or you might have to pay a lot of money for Amazon for the [?
cycles, ?]
so by actually making programs faster, you can make it scalable, you can make it efficient.
So this gives you the ability, this currency, to go do a lot more other interesting things to your software because now you can run it faster.
It gives you this margin to play with.
So in this class, we are going to learn a lot about these things.
We are going to learn about architecture, how looking at the architectural issues, how to identify when something is wrong.
We are going to look at algorithmic issues in here, how to look at things.
We are looking at architectural things like memory systems, parallelism.
And also we will talk about all the tools, so you will actually know something has gone wrong.
Because today, you run the program, it runs, it runs correctly, is it good enough or not?
How do you identify that?
That's, I think, the very interesting place to start.
So that's all I have today.
Make sure you get a Project 0.
And especially people who haven't done C, we will-- Charles, are we going to do a C remediation?
We did last time, didn't we?
OK, guys, one more thing.
Last year, we did an evening C remediation class.
So if you know Java, if you want to know C, we will send out a time with TAs that will give you kind of a one-hour crash course into C in there.
And we will send information.
So this class moves very fast.
Get on with Project 0, and see you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare@mit.edu
Any issues with the project so far?
Have you gotten your repositories set up and the compiler works and all this?
So I'm glad to see a lot of people are getting a early start.
In fact, if you haven't, a lot of other people in the class are getting an early start so you're behind already.
So go there, get the project started, because these performance projects can run for long time.
And in fact, last year a lot of people asking, when is my project done?
Because a lot of times when you have a project there's a certain set of-- we ask you to implement.
Once that's implemented, the project is over.
But when is a performance project done?
And the only answer I L when the deadline comes because you can always shave 10% here, you can get a better data structure, you can keep like kind of tweaking for a long time.
And a lot of you will do.
In fact, to give you a warning, last year, the performance difference between the best project won, versus the worst one, even the worst one got about 50% better than the code we gave.
But the best one got 1,000x better than the worst one because they figured out right data structures, they changed algorithms, they did some precomputation, they did a lot of cool stuff, and so there's a lot to get if you do this project right.
So just don't be happy if you just got 10%, 50%, and say, yeah, I got some performance.
Think carefully.
There's a lot of room in this project for you to gain, so work hard.
I'm not saying this yields 1,000x.
I don't know.
I haven't exactly seen all the things you can do, but there could be a huge thing you can win.
So think through not just small, low hanging fruit, but there could be interesting things you can do that can have impact.
So today what I am going to do is talk a little bit about basics of performance engineering.
Kind of things you guys would probably know, rules to follow.
So to start that if you look at performance engineering, it can fit into a couple of classes.
One is you want to get a maximum out of the compiler you have, the processor you have, the system you have, you want them running efficiently.
So if you look at the matrix multiple examples we did this in the first class it's saying, OK, we have a system, we have this very well known algorithm, very long piece of code, and we want to get the best out of that thing, how do you go about doing that?
That's mainly the focus there.
Then when you we looked at the bithacking stuff that Professor Leiserson did last time, it's mainly about really changing the algorithms.
We basically said, OK, look, if we're thinking about this as bits there might be fundamental different ways to approach this problem.
By doing that we can actually get really good performance.
OK, we'll be doing more of that.
Today we are going to focus on this two middle criteria.
What we are going to do is we are going to stay within the spirit of the program that was given.
Even though we are going to change the program a little bit, we are not going to fundamentally change what the program does.
So the same computations would be done basically on the same day but what we will do is we will change certain aspects how it's being done to gain some performance advantages.
So the nice thing about this one is you don't have to really understand the algorithm, you don't have to understand lot of what's going on in, what's implementation.
This is more-- you can do this as a mechanism sub out.
The unfortunate thing is there's not theory in these type of engineering.
There's not something we can prove, here's the optimal you can do.
[UNINTELLIGIBLE] nice there.
So normally performance programming basically knows that you need to understand everything that's in modern performance.
That means all the way from high level algorithms, every layer in the systems, the compiler, the hardware, the disks, all those things can impact performance.
So it's normally when in performance to understand everything that [UNINTELLIGIBLE].
This is very different from a lot of what you guys are used to.
It is staying within one layer.
You say, OK, this is my layer, I don't care what's on any of it.
I don't know care much about-- I'm going to work on that.
And the thing that normally comes and bite you is the other layers that you just take for granted.
In performance it's basically going through end to end.
That's the normal thing.
And normally, if you are an experienced performance engineer, you know when the problems show up.
And you can kind of have a basic understanding, here are the kind of places where they have performance problems.
And so a good performance this is where experience comes in.
So instead of saying looking at the code and say I have no idea what's going on, you can say, let me think about this, this is where normally something goes wrong.
And then you have to have the skill to zoom in and identify the problem fast.
And most of performance engineering is basically a good dose of common sense.
So have kind of a good person who can do really good debugging.
Is you have kind of a built in algorithm how do you go about doing something.
And a lot of times you know your strengths, kind of mistakes you make, you look at those things first, and the same way performance engineering there has to be kind of a back of your head algorithm.
And hopefully at the end of this class you're going to develop that.
And in this lecture what I'm trying to do is come up with a set of rules, that kind of patterns that occur regularly, mistakes lot of people make, and that can have a substantial performance impact, or performance gain if you do that.
If you have done the design patterns lectures in 005, yes, you do, you will see some similar ideas.
What you're doing is kind of say, OK, here's a bunch of patterns in there, what we can do.
So what I'm doing is I am following some very interesting set of rules that was developed by this guy named John Bentley.
He wrote a book called Programming Pearls, I guess.
It's a very old book, last published in 1990.
It talks about a lot of the small, interesting things you can do to get really good performance.
It's out of print.
This is probably one reason to probably go to library probably.
Libraries have this copy but you can't get it on Amazon or online so that's unfortunate.
But what I'm going to do is I'm going to kind of read through it and some of these rules, modified to suit the current program and practices.
So he looked at it and said there are basically two kinds of things you can do, we can modify data, you can modify code.
And then went down more into details, if you modify data you can do three things.
You can pay one against other, you can pay by space and gain time, you can pay from time and gain space, and there's sometimes, if you're lucky, you have a win-win situation when you can get both space and time advantage.
So let's go through a couple of those things.
So if you [UNINTELLIGIBLE] by space for time, there are four things.
I'm going to go through all this four data structure augmentation, storing precomputed results, caching, and lazy evaluation.
So what's data structure augmentation?
So here what you want to do is you have a data structure that you keep the data.
You add some extra information to your data structures.
That can mean the common operations run much quicker.
OK?
And when this is good, because what do you want to do is this adding this information has a clear benefit.
Calculating additional information has to be cheap and easy.
So you don't want a situation where we have this information, you've spent all this time calculating this additional information and you're not using it that much, or the use doesn't give you that much advantage.
So you have amortize the cost in here.
And then also keeping that information current can't be too difficult, because sometimes you can add information into a data structure that makes it very hard to make sure that that information is kept up to date, and then you're spending more time and you're [UNINTELLIGIBLE] that.
So a couple of quick examples in here.
Something like if you have a single link list, if you are doing a lot of deletion it's a very expensive proposition because you might had to walk through the entire list to delete an element.
But if you are in a doubly linked list, a division operation is basically order one instead of [UNINTELLIGIBLE].
So suddenly you realize, I'm doing a lot of deletions, suddenly I can say I'm adding this additional information to my link list, I'm maintaining point as to both directions, make this one operation run much faster.
This is a very classic example of adding additional information.
Another one is what we call reference counting.
So assume I have objects somewhere and I want to figure out who points to me.
In terms of garbage collection you want to see if anybody points to me.
The normal way to do that is look for every other thing saying is anybody pointing to that person, had to walk through every possible object out there to find if anybody's pointing to this object.
That's what normal garbage garbage collectors do.
But there's something easier, what you call reference counting.
That means if each object keep a count on the amount of pointing coming to itself then to ask that question, is this really true?
And you look and say, OK, what's my count?
If there's count to zero nobody's pointing to me.
But of course every time you point to that object to update the counter, every time you point away or take away an object that points you have to subtract the counters, you had to keep that information additional.
But by doing that discussion about asking whether anybody points to you becomes a very trivial question.
So here's a case you add a little bit more information and that makes some questions, some things you want to know trivial and very easy to do, and you have to figure out where the [UNINTELLIGIBLE] is.
So here's one interesting thing.
Go a little bit further, one thing you can do is precompute, result and storage.
What you are doing is you want some calc computation and instead of every time you want that in recalculating it, say, OK, I'm going to compute them early and I'm going to store it somewhere, and then every time I want then I just do a look up.
So when is this going to be useful?
So this is-- the rest of my slides are going to have this kind of form, I'm going to give you a problem, tell you when it's going to be useful, and use some examples.
First of all, the function you are calculating has to be somewhat expensive, otherwise if it is just a simple single operation look up is probably too slow to compile.
And the function has to be very heavily used, because if you are calling it many, many times it's worth storing it.
Another very important thing is the argument space has to be small.
You might be asking for calling the function millions and millions of time, but every time you call you give a different set of arguments.
If the arguments are very large it can be many, many things here, and then it's not easy to store everything.
And it might take more time to find all the solutions for all the arguments.
So if the arguments have to be small.
And of, course, a couple additional things, results only depends on the arguments.
So assume the function results actually look time of day to the computation.
OK, that doesn't work, because then every time you call the function, you get different result.
That doesn't work that much.
And functions shouldn't have side effects.
What that means is if you call the function, if it is modifying some global variable, global state.
OK. Then if you keep the call, if you don't call the function that might not happen, so you want to make sure that this function is something can not call and give the results and the state of the system won't change.
And another interesting, nice thing to have is function determinants, that means every time you call the function you get the same four arguments, you get the same results.
So this is a long list.
Some of the things I will go, so for an example, OK, so instead of me getting there getting all the things I want, you [INAUDIBLE] when I give a prescription like the thing, what kind of things can I do in this way that might be helpful?
Can anybody come up with an interesting example?
[UNINTELLIGIBLE] would precompute and save
Anytime you do dynamic programming
Anytime you dynamic program.
This is dynamic programming, it's a very good way, later we probably go get a little bit into that.
Where you precompute and save results.
OK, that's very sophisticated answer.
Good.
Any other things you can think of?
That's almost a meta answer that's in that class of things you can do.
Any problem there that you have encountered that you'd rather precompute and save that would be a big, big win
Operations with large primes that [INAUDIBLE]
Operations with?
Large primes
Large primes, operations in large primes, yes.
If you are doing that, especially if this fits into this argument space small apart.
If the large primes are, any large primes then you might not call it multiple times.
So here I want to give you kind of a template that what happens here.
So what you want to do is at initialization time I get this function initialized, what I do is I precompute for all arguments what the results are stored at, and then every time you go apply the function I just basically, instead of calling the function, I just look at this table.
OK.
So then would this be a bad idea?
Because you know this early ahead of time you know that you're going to be calling the function for all of your arguments
So if the argument space is very large or if you don't call the function that many times, or as I said, it might only call for very small number of arguments, [UNINTELLIGIBLE] this might not be a good idea.
OK?
So we will go.
As we improve, you'll see some other things that actually address that.
So here's an interesting way of doing a use case in here.
So what I have here is Pascal's Triangle.
How many people know Pascal's Triangle?
OK, good, so I don't have to explain that.
So here is a simple computation for Pascal's Triangle.
So what I'm doing is in order to calculate this value I am calling these two values and adding that.
OK?
But the problem is that this will call- so it kind of call exponentially more and more doing that.
So you'd say exponential rhythm.
However, what I can do in Pascal's Triangle is I can basically compute it at the beginning by storing the previous role, or previous set of values in and then previous role, and every time I'm calling I just basically have to look up the two elements in the previous role.
Trying to get my mouse, OK. Only have to look at the two elements on the previous role.
I added up, I will never have to compute beyond that.
So, in fact, by just doing this one I have taken exponential algorithm to basically lead a path through the data, basically.
This actually become a square, because if you have some depth, I had to do i squared-- become a square, pass through the data.
If I'm looking for n, I don't know how to do exponential look up.
Everybody saw this?
So this is a nice way, simple way I'm just doing a simple storing this data, however in here I had to calculate, I need to know how much maximum of that, and calculate and I just look up.
And here the [UNINTELLIGIBLE] if you calculate a little bit extra, who cares, I'm OK. And another interesting thing is Fibonnaci.
And again, here, if you do keeping the previous value, in here, you're actually changing from exponential algorithm, to in here basically a linear calculation.
OK?
So this is almost an algorithmic change here.
I kind of lied when I said it's just a data structure change.
But just a simple data structure change made an algorithm difference.
So the other interesting thing you can do is caching.
So what that means is if you have a heavily used function you can keep some number of previous basically results from calling that function.
This has a lot of caveats so let's go through this carefully.
Again the function has to be expensive, otherwise it's not worth doing all that work.
Functions have to be heavily used, otherwise you're calculating and if you don't reuse it it's not worth doing all the worth.
Here the arguments space can be large, OK, because you are not calculating everything, you can have a large argument space, and that's OK. And then there's another thing called temporal locality.
Anybody knows what temporal locality means?
OK what
The same arguments are called [UNINTELLIGIBLE]
Yeah, so a lot of times we realize that if you go into the caching and stuff like that.
We really get into that in hardware when you talk about hardware lecture.
So what happens is yes, we have observed that if I have a set of arguments and if I calculate most of the times I will have the same arguments very quickly time-wise.
So if I calculate something, then for a short time I might reuse those arguments again and again, so that's called temporal locality.
So caching works when there's a very good temporal locality.
And then there's two other things we want to do because we won't have hash function that means because a lot of arguments from that we only get one value to look up where I store my cache, have to have a hash function, but to calculate it by argument.
Sometimes if the arguments are too complicated you might not be able to calculate a good hash function and the hash function has to be good.
What do you mean by a good hash function?
[INAUDIBLE]
Yeah, no [UNINTELLIGIBLE].
That means they tend to be a good distribution.
So given most of the arguments it shouldn't be ending up as the same value again, so you should have good distribution when you call with different arguments.
And the final results only depend on the arguments that's again otherwise you have a problem and the function [UNINTELLIGIBLE] if your [UNINTELLIGIBLE] this important.
And there's another thing called coherence.
So what that means is I keep some precalculated values in here.
And sometime this if results are only dependent on the arguments you don't have that much of an issue.
But sometimes you might in the situation that the results are dependent on the arguments and some other things like the time of the day or whatever.
And if you want full coherence, what you want to make sure is-- many of you know that the results might be varied.
I need to understand that, I need to invalidate all the cache.
And suddenly the data or the user changed so that thing I memorized only works for User A, now went to User B, the results are wrong and I had to get rid of all the results.
Or there are some cases where you could say, yeah, little bit of things I can tolerate, like for example, a really far fetched example is something like if you're looking at a web page on a search.
OK, you don't have a good guarantee that all the web pages in the world are in that search.
If you're a little bit late getting a new web page, then you're OK.
So you can do tolerate sometimes, a lot of times, if you can tolerate these kind of things you can have huge performance impact.
You don't have to be exact.
So some of these projects we look we look at these kind of ideas, of tolerate a little bit of a slack and getting a good performance out of that.
So here's the kind of code, I put a lot of code in my slides because not I won't explain everything but since these online if you want to figure out what the right template is you can look at it.
I have no idea why this is moving on me.
OK, OK, here we go.
So what you first have to say is every time you cache a value you have to know, I had to know old arguments and results, I had to store that, and then I'm keeping this many precomputed values in here.
And so every time I call the function with arguments first I have to see I had to get the get the has with these arguments because that might be the place, this buffer disk just might be the place I have stored a previously computed [UNINTELLIGIBLE].
Then you say, OK look, is my [UNINTELLIGIBLE] stored, has the same argument.
If that means I already have that precomputed, at that point I just return it, I have no issue in there.
Otherwise you're to call the function because I haven't stored it, and then I will go about storing the value because I calculated I need a storage because the next time I call these arguments I want to get the same results, and then I return the result.
OK. Is this kind of clear, how this caching works?
OK, what's a good place for caching versus precomputing?
Any idea what can make a good scenario where actually caching can have a big impact?
You have heard a lot of things that have the word cache associated with it, and you can say, OK, what that means
Web browsing
Web browsing OK, you web browse actually that's a cache, because there's a good probability that if you look at a page and you might be refreshing that page or you might be looking at that page, so a lot of times what you do is you keep a copy locally, and the next time you ask for that and if you have a local copy you can give that local copy back.
And then of course there's some issues about how to make the local companies stay, the coherence issues and stuff like that it can get complicated but that's a situation where actually you can use caching.
So finally, in this category, we are going to talk about lazy evaluation.
So lazy evaluation means basically a lot of times you might ask for value but you might not really need it at that point.
Just keep the computation to the side without doing it and then do the computations only when you really, really need the result.
So basically this is useful, a lot of times you might try to compute a lot of values but only a certain of that actually is used.
And at that point you just basically further that calculation and only use it when it's actually done.
The nice part is when can be done in a function call, and also results can be calculated kind of incrementally.
And also, the arguments or what's needed to calculate the results can be nicely packaged up so it can be recalculated later.
So here what you do here is in the precomputing is something like precomputing but you don't precompute early, at the beginning nothing is precomputed.
When you only ask to apply a function then if there's no precomputed value you'll recompute, otherwise you just return the value.
So if you compare this one Pascal's Triangle beforehand I just computed, there's a bunch of values in here and kept it.
OK assuming OK up to PDMAX I will have the results in there.
But you might not do that.
PDMAX might be very large and you might be looking only for small Pascal Triangle.
So instead of doing that, what you can do is you can do this way, you can keep the array for results, assume everything is normally in [UNINTELLIGIBLE] global array.
and then what happens is you look at the results if the results is greater than zero we are what you're looking at, that means, ah-ha.
I have calculated it.
I have a new value, I just iterated it.
I'm happy.
Otherwise I will call Pascal by recalculating values because I don't have the current results.
And once I get it I will store it.
Once I get the value I will basically, I will calculate the Pascal, I will store it and I will return the value.
So what that means is this value of the Pascal yx will be only calculated once fully, and afterward every other time you lose that you actually use the previous one.
This works very nice, this works both very nice about precomputation but you don't pay that extra overhead.
So you are doing the minimum amount of computation.
You can never exceed the computation of the normal computation.
So if you only ask for Pascal very small one, that's exactly the work we [UNINTELLIGIBLE] do.
But it will also really use this kind of exponential blow up that happens in the normal computation.
So this is kind of the you can have the cake and eat it too in this work.
OK good.
So now let's switch from space for time, for time for space.
OK?
So here, one thing you can do is if you have a huge amount of data what you can do is you can store the data in some kind of reduced form and then what you can do is then you can get a big compression of data basically and then as you need the data you can do some more computation to get the data you want.
So what you're doing is you are really reducing the space you need but now you are paying by time to get access to that data.
So this is why [UNINTELLIGIBLE] storage is at a premium.
If you look at systems before 1980s everybody had to do it because storage was a premium and all the memory and disk space was so complicated everything got compacted in there.
Now, most your laptop and desktops, you don't care, you have enough space.
However, if you got things like embedded devices and things like that, this still becomes a issue.
Or if you have a very large data set it becomes an issue.
And what you can do is by compressing, drastically reduce the data size.
And you can do it in different ways, sometimes you can do it in batch.
That means you keep the data compressed, expand it all, and process, hopefully you have room for that.
Or sometimes if you don't have room you do it by streams, that means you're looking at a small amount of data expanded, work at it and then either throw it or again compress it and put it back in.
And hardest thing is if you actually do random access.
That means you'll randomly go to some data, expand and understand that sometimes there's some complications.
So there are a bunch of packing methods and some simplest things are use a small data size.
So right now you're dealing with 64 bit words, you can say wait a minute, why does this data only need 8 bits?
Why do you keep storing everything 64-bit if my data maximum can be only 0 to 256.
I can use 8 bits.
Or I can look at that and I can use small data size, that's something.
And a lot of people realize a lot of data we are storing are just zeros.
There are many different data structures, you have huge data structures and most of it's zeroes.
And eliminating zeroes in many instances can you get a long way in many, many, many data storage.
And something like LZ77 eliminates repetition.
So a lot of times what you find in there you have repetitive patterns you can eliminate that and then, of course, you can go into a lot more complicated.
heavy-weight compression.
So to just give you a feel for what compression is like I found this cute animation so I will show it to you.
So here is the input stream, but it has this corrector LA, and then it has some notion about how much a repetition it has, corrector O and 0 3.
So what happens if it's just a character?
It will get taken in, so these two correctors go.
Now these two forces from the point I point in here go to inwards, and repeat it four times.
What that means is basically in here it will get repeated four times because that 2 4.
So that means instead of having these four correctors in there, what it had was it had two things, and then all get repeated three times and then it'd get shifted in here.
So instead of having ten correctors before, we had 2, 3, 4, 5, 6, 7.
Seven, out of seven correctors you compress you stored information off ten correctors because of repetition.
OK, so this is a very simple scheme that people use to do this LZ77 compression.
More complicated thing you can do a lot of times is build an interpreter.
What that means is you have some complex data, instead of storing the data you have a summary of the data stored.
OK. And then what you can do is you can do this abstract representation.
I mean you were already tested.
Like, for example, you [UNINTELLIGIBLE] exactly doing that.
What you do is you start off storing all the beats of machine instructions.
You store it in abstraction called add register names or something like in smd.
What that means is you're building a very simple abstraction which that would be more complex sometimes to store than the actual data.
So in here, things like if you look at a java byte cord, that's a nice interpreter.
So instead of storing all this complex instructions you interpret, you stored some higher level set of instructions, that byte code, that is much more compressed and that you run you map it into the machine instructions.
There's a lot of advantages other than just compressing storage, it's a nice abstraction, you can change it at the highly level easily.
OK, so those are something that you can do, you give up one thing for the other.
And if you're lucky you can have the cake and eat it too.
Which is you get both space and time at the same time.
So we looked at it in last lecture sometime.
This is bithack.
Bithacks kind of gave you that because instead of having 64 bit word to represent you got 64 of them into one big stream, if you're only representing zeroes and ones.
And by doing that you get a lot more compressed storage.
Also now you can operate that entire 64 in parallel.
We can generalize it a little bit, so instead of just doing 64 Boolean, sometimes you can do in a 64 bit word, you can store two 32 bit words, or four 16 bit words, so eight 8 bit words.
OK.
In [UNINTELLIGIBLE] is called a Memex SSE extensions.
Normally it's called SIMDs, a single instruction multiple data, because what you do to that, all the bits, are the same thing for everybody.
So if you're adding you're doing same add for both of the data.
And because of that you basically going to get both the compression.
Because now for 64 bit you are keeping two thing, or four things, or eight things.
Also, now you're operating.
In a single operation you can get the same operation happening to all the data.
So you're getting both storage and fast operations in here.
Of course if you only look at one you have to do a little bit more work, you take it out of that data word, so if you're just looking at [UNINTELLIGIBLE] thing can be expensive.
So when is it viable?
You'll do the same operations all the data.
OK, if each day you do something different it's not that great and items can be stored in contiguous memory, so when you load you can say I'm getting nice in one word, if I'm getting two 32 bit, so four 16 bit, so I can do that.
And you don't end up picking each operations and doing individual things too much.
So if you are to pick each individual elements a lot, then you have to actually now pick apart that word and that can be multiple operations and expensive.
So here's a simple example.
This, I think, from after last bithacks class, this is a little bit, you probably know, can come up with enough examples here.
What I'm doing is I'm doing a Battleship board game.
So you represent the board, and the interesting thing in this board is to know whether that location has something or not, it's empty or full, that's all I need to know.
And so normally if I do this board game I can have two boards in here.
But what I'm doing is having this overlap calculations, saying of the two boards, how many things overlap.
I can just have each location represented by integer, but then only the representing one value is not good enough.
One thing I can do is I can just put each row into one 64 bit word, and then instead of doing the two loops I can, in one loop, I can just do the ending off each row, and I get the result.
So after the the bithack lecture, this should be kind of trivial.
OK, everybody get it?
Anybody have questions so far?
Are we all on the same page?
OK, good.
There's a lot of blank faces so I want to help, I'll start asking more questions or something.
OK, so that's modifying data.
Now we're going to modifying code, where you will change the programs to actually get performance and that can be done in loops, logic rules, procedures, expression, and parallelism rules.
When we get to parallelism rules we will actually do a lot more on them in future lectures, but I just want to address some basic stuff now.
Bunch of different loop rules.
I will go each of them individually.
So first of all, why loops?
Why do you think loops are so important?
OK, somebody who hasn't answered the question.
Why is loops so important?
Why are we like really focused on loops all of the time?
Anyone want to answer here, this side?
[INAUDIBLE] in terms of maintainability, [INAUDIBLE]?
Yeah, so loops give you a nice, simple abstractions it's easier to maintain instead of having multiple computed times.
Yes, instead of having same thing repeated millions of times for the same thing, if I write a loop, it's much more concise representation.
That's why I think we do a lot of loops, because it makes programs easier.
In fact, assume this word, loops didn't exist, OK?
If loops doesn't exist that means you're going at each instruction get executed only one time.
If you have a 3 gigahertz machine that requires even if you do one instruction per cycle in 32 bit instruction you need 12 gigabytes of instruction per second, if you only execute one instruction once.
That means if you have 100 gigabyte disk full of a program you go through each instruction the entire 100 gigabytes in 8 seconds.
OK. Of course this can never be done, that means you cannot feed 100 gigabytes into the processor in 8 seconds.
That won't work because of the disk access time, caches, and all those.
Even if it is doable, what that means is the entire reason your Pentium is running at 3 gigahertz and actually showing like it's running as 3 gigahertz is because you are using instructions many, many times.
You have loops that run millions of times.
If you don't have loops you can use whatever the data machine you build in '004 and that probably as fast you could get because of this instruction thing.
So loops are critical, and loops are what makes our programs run fast.
Repeated execution of same instruction again and again and again.
OK.
So in the world, in that sense, if you think about this, unless there are some instruction that's run millions of times, you can never keep this ferocious beast's appetite full because there will always be new instructions.
So because of that looking at loops, if you want to get good performance just looking at the most important loops, the inner loops, and just doing only that we'll probably get you most of way in many, many cases.
And basically I could even say 99 of the program execution time is in 10% of the code, even, more than that in many cases.
Because otherwise [UNINTELLIGIBLE] back of the envelope calcuation you can do, you realize how much code you need to feed the beast to keep it active otherwise.
So first loop optimization, loop and invariant code motion.
So normally in a loop you run millions of times, and if you do the same thing millions of times and get the same results you are just doing useless computation.
And so here the key thing is most of the time the compiler, a good compiler will come help you.
So in this class you're not trying to replace the compiler.
We, as humans, we want to the least amount of work.
The compilers [UNINTELLIGIBLE] dumb things sitting in there, it can do most of the hard work.
The key thing is first trying to get the compiler to do the work.
Only do things yourself if the compiler cannot do.
So most of the time compiler, if it can analyze the code, and prove the results are the same in each iteration, it's going to get rid of it, it will do that for you.
But there are cases it's not possible because sometimes the computation might be too costly and also there might be cases the computation is too cheap if you hoist it out, you have to keep the results in some [UNINTELLIGIBLE] and if you keep too many things in too many registers, that cost might be too expensive.
So there might be case that you might not do, but most of the time what happens because the compiler can't do anything.
So here's a good example.
I'm doing, I don't know why I did that, but square root and take exponential off that.
Compiler look at that as functions.
I have no idea what the function are most of the time.
The compiler can't tell what each function does because the function call.
It can do any arbitrary thing as far as the compiler knows.
It says, OK, function call, I don't know what happened inside so I'm just giving up and I'm just leaving it, but you, on the other hand, knows that the square root function, exponential function doesn't have any what you call the side effects.
That means those functions doesn't go changing anything else.
It just calculates [UNINTELLIGIBLE] the value, you'll know this basically calculate the same value again and again and you can just basically take it out, do it twice, happy.
So lot of times make sure the compiler can do it.
If the compiler cannot do then you have to go in and interfere at that point.
So here's another interesting thing the compilers cannot do.
A lot of times you're going through this array or data structure looking for some value.
OK, once you've found the value, you will say OK, found it, I return.
OK.
Normally what that means is when you're going to add it, there's two things.
First of all you had to test whether you found the value that you're trying to exit.
Second, you have to make sure that you don't overrun on the data structure, you're not overrunning that.
you have to make sure you're not at the end of that because that value might not be in that area.
If that's the case, you run to the end of that.
So pretend I don't find it.
So you do two tests.
One thing you can do is you start doing the two test at the end of that array, how about you add the value you are testing for?
So you know when you reach to the end you will always find the value because you added there, so you know it's there already.
So by doing that you can reuse the test by just looking for the value because now when it comes to the end, you don't have to do anymore tests because the value you're looking is already there and you found that, you go out.
But you should be able to add that value to the end of that.
So what you do is this is your normal test that's what you're doing is you're, if I can get my mouse, you're going through this array, looking for a value.
A value I return otherwise I return minus 1 in here.
Now I do two tests.
I do this test to make sure that I'm not at the end of the array.
I'm [UNINTELLIGIBLE] this one.
So, in fact, [UNINTELLIGIBLE] loop I just can put the two tests next to each other so this is what you're doing.
But instead of doing that one thing I can do is put that at the end of the array, after all the data that I care about, I can put the value I'm looking for and then I just only do one test.
I check for that value.
So if you find the real value in the area, at that point I'm done.
Or I ran out of the array, and after I ran out the real data I just found the value because I, myself, put it there.
And so I have to find it, and the minute I find it, if I realize that's the case I say, OK, that's the value I put and I [UNINTELLIGIBLE].
Here I have done something which is not really good programming practice.
Can anybody put your 005 hat and say, that's bad programming?
You modified [INAUDIBLE]
I modified my input array.
I know.
That's not that great.
I mean if I did it really well, if I want to make sure what I should have done is kept the old value of the n and at the end, it back so it looks the pristine thing I got.
I mean, if I was really careful I would have done that, and then my invariant that my input array haven't changed would hold nicely.
because here I have mucked with my input array.
So even though you are constantly doing performance programming you should still have a good eye for doing good correct programming and keeping nice programming variants around than mucking it all over the place.
So if you wonder putting a new hat in here, you should still have the 005 hat, at least partially.
What else can we do?
A lot of times what you find is we have these loops, and loops have a couple of features.
First of all, you have to keep checking the loop condition, that's expensive.
And if the loop body is very small, the compiler doesn't have that much opportunity to do anything useful in the body, just basically compile as is.
So one thing you want to do is if the loop bodies are too small and the number of iterations is small, you want to unroll the loop.
So in here, instead of doing this calculation again and again, I want to actually do a computation unrolled, and this gives you two things.
First of all I get rid of this disk, so I actually [UNINTELLIGIBLE] in here.
And second, this computation can be done much faster, the compiler can do this much faster than this one.
Do you see why?
Can anybody see why this computation would happen much faster than this doing in the loop iteration?
Because you don't have to write the final result of the variable until all the numbers [INAUDIBLE]?
So that's one thing.
Even the final results in the register what happens in here?
So that's a good assertion but that's not exactly what's back there
[INAUDIBLE]
OK, he got 90% of the way there.
First of all here I had increment i.
And I have it check [UNINTELLIGIBLE] the loop condition.
That's another point in here
[INAUDIBLE]
The one on right can be run in parallel.
In here I am doing sum equals-- some plus A I, so that means I have to calculate [UNINTELLIGIBLE].
Once that is done and we get the results only then I can go add the two, only then I can add day three.
So the last [UNINTELLIGIBLE] using, you can show 16 instructions at every cycle.
OK what six things you can do in here?
Nothing.
Because you're just waiting.
You're twiddling your thumb for the first one to finish, to do that.
In here, you can say, look, I don't have to add A1A to it.
I can just do these things in parallel.
The first cycle I can add is 0 to A of 1. that's one instruction 2 3 2 4 5 3 6 7.
I can add all those things together as a tree.
I can do tree addition instead of one after another, that can happen in one go.
And I can get a six editions done and then the next instruction I add the [UNINTELLIGIBLE] and at the later stages do not have too much to do but I could get a lot of [UNINTELLIGIBLE].
So the compiler can actually [UNINTELLIGIBLE] all of that.
So that works very nicely if I have small number of iterations.
But if you have a large number of iterations, you can still do some stuff.
So in here what you can do is it goes to one and you can say, OK, I don't know.
I don't know how to unroll fully because I don't know what n is, but I can unroll partially.
So here what I can do is say, look, I have been running it n times, I will unroll my loop four times.
OK.
So I at least get enough of unrolling here.
That way you can choose.
That is, you unroll enough that the meshing can get busy.
It's not just crawling.
And then I haven't blown up my code like crazy in here.
And of course you had to clean up and then you have to know how many full iterations.
So how do you choose how much to unroll?
What impacts my number of unrolls?
I unroll four times.
You can get greedy, and say, I'm going to unroll 8 times, 16, 32, 64, 128.
Because a lot of time these are not zero-sum some gains.
Some stuff impact some other way.
So what should I use normally for my amount of iterations to unroll?
Anybody want to take a guess what helps?
It's probably the number [UNINTELLIGIBLE]
OK, so in one sense you can look and say number of instructions I can execute in one cycle, that's a good thing to keep, unroll enough to keep the process of [UNINTELLIGIBLE] most of the time.
But sometimes we can say oh that's hard to figure that one out, I don't know that because I'm at the compiler.
I'm at high level.
I don't know what happens at low level.
I'm going to unroll 1,000 times.
So why shouldn't I unroll a huge amount?
What happens if I unroll too much?
[INAUDIBLE]
You might--
You might to be [INAUDIBLE]
OK, define it more closely.
What do you mean?
Perhaps there's some [UNINTELLIGIBLE]
That's the one interesting thing.
Assume your number of iterations are not always millions.
Sometimes you might [UNINTELLIGIBLE] hundred times, or 200 times.
So if you unroll 500 times, you would never get to the unroll loop.
You end up in this clean up loop.
And so clean up loop, of course, is not unrolled and so you're back to your square one.
And so a lot of times if you unroll too much and if your number of iterations doesn't fit so you end up in here, that's not good.
Another thing is when you unroll loop here there's a huge amount of instructions now.
That instructions might be too much, it might not fit in the cache, it might just overwhelm the memory system.
So you have already now created a loop that doesn't fit in the instruction cache and that's not good either.
So that will limit down roll, so don't get too greedy, just sometimes these are the things you kind of play with in the problems you're looking at and some of it is architecture dependent.
Some of them are things like too much unroll where the number of iterations not that high is problem dependent.
Another thing you can do there are many places if you have two different very same looking loops we can put them together into one loop and that gives [UNINTELLIGIBLE] all the loop tests and stuff and also the array access.
At least done once, so that can give you a benefit in here.
Here's interesting one.
Here you have this weird loop that the j loop run from I to J in minus I, I to n minus I.
How can I improve this one?
Can anybody tell me what might happen in here?
So I have outer loop I goes from 0 to N, inner loop goes from I to N minus I.
What happen?
Somebody's getting it
[INAUDIBLE]
Yes, what happens is because what the loop is at the beginning you're going from basically 0 to N, and then the lower bound is going like this, upper bound is going like this.
After N over 2, this is N, N over 2, there's no iterations left.
OK, you're trading and trading and trading and after it iterates, so you're running outer loop, N over 2 to end with nothing running on the inner loop.
And so if you have this kind of funky loops you can just look at it a little bit careful and say, wait a minute, that's not useful.
So therefore I can just get rid of this new situations.
And then sort of free trading to nothing.
So that's loops rules.
Loops are very important.
also you can do a lot of interesting logic rules.
So first of all, can anybody shout out what each of these things can be done?
Square root of x greater than 0.
Square of [UNINTELLIGIBLE] zero, sorry.
x [UNINTELLIGIBLE].
Actually, not [UNINTELLIGIBLE] sorry.
x not equal to zero.
Because x squared of x squared non-zero.
OK.
Here's something your compiler probably doesn't know about the transformation.
We can do that.
How about the next one?
How can I get to input the next one?
Square root of base climb, looking at the distance between x and Y and
[INAUDIBLE]
Eliminate the square root.
It doesn't help anything you do except do too much computation.
OK, next one.
Yeah, I can basically multiply and then do take a one [UNINTELLIGIBLE] final one
[INAUDIBLE]
One.
OK, and then you should go and find that program who wrote that, but believe me you'll find these kind of code sitting somewhere because you didn't think through, you just start writing and suddenly you have this kind of nice axiom.
So when do that this is something compiler can do, of for these kind of patterns.
That was a stupid mistake but a lot of other things are things that a compiler cannot do but you know more than the compiler does.
OK. Other interesting thing in here is if you're looking something like [UNINTELLIGIBLE] increasing function- OK, let me look at this carefully.
OK, this is very similar to what we talked before with the [INAUDIBLE] ending.
So one thing you can do is I can put the cut off at the end and then I don't have to take check both the bounds and basically [UNINTELLIGIBLE] through something.
Basically this is the [UNINTELLIGIBLE] check we did done a little bit differently.
OK. And here's another interesting thing.
Reordering test.
So what can you do here?
What do I do here?
I'm looking at is, I am looking at two different a circles, see whether they are [UNINTELLIGIBLE] in here.
This is expensive test because I had to do something in a square root and check whether it's less than or equal to radius.
So here's what I had to do.
How can I make this faster?
What's the [UNINTELLIGIBLE] in here?
So assume I have this bunch of balls running around in my graphics program.
I want to make sure that the balls haven't collided.
They are actually moving with no collision with other balls and I'm doing this test.
What's a good [UNINTELLIGIBLE] in here?
[INAUDIBLE]
So he's getting there because most of the time what happens is if you're looking at two balls collision most times these balls are farther apart and if you just found the square around the ball, testing these two squares are colliding it's very cheap, OK?
And then if the squares are all happy, then and only then you might want to check if actually all the balls are all happening.
OK and by doing this square test you can do very expensive operations in here.
You can basically do some very simple thing, in fact I have even reduced it more by just doing first doing the x side and then if x is [UNINTELLIGIBLE] I don't even have to check for Y.
And only if those two checks fail that I realize, look, the square might be now there's a good chance it's actually the color id.
And there's a really good chance most of the test, after doing the first two things, will be done.
You'll never have to go through this very expensive operations.
So here's something you have a little bit of intuition on what's going on out there in the problem, and use that intuition to look like more work, but in reality really reduce the amount of work you had to do because the test is very expensive.
OK, here what I have done, OK this is a little bit of a contrived example, what I want to do is assume I have basically a 4 bit word in here, OK. And I am checking- this would be 8 I think, so I have 8 bit word in here.
OK, I'm checking whether the 8 bit word is a palindrome.
So the way I check the word is a palindrome is first of all I set up some bits in here, bit masks, OK?
And I have a bit mask for the two ends.
I said, OK, these two are same, then these two same, these two same, these two same, I just kind of go down the bit mask.
Did everybody see how this work?
And then in these two I kind of shift my bit mask to the side, these two, basically shifting.
So I'm checking the most two bits are the same, the next two bits are the same, next two bits are the same, going on [UNINTELLIGIBLE].
OK, so every time I do that I have [UNINTELLIGIBLE] this bit mask do this operation and stuff like that.
Assume I am checking whether my [UNINTELLIGIBLE] correctors.
If I call this a lot of times what can I do?
Kind of out of the box thinking.
So the one thing is if I am only looking for correctors, how many different correctors are there?
[INAUDIBLE]
Yeah, because all the correctors in everything I look up, I have 8 bits, two to date.
What's two to date?
[UNINTELLIGIBLE].
OK, so you have 256 possibilities in here, OK.
So I'm doing this to 256 why don't I precompute and store?
So if I just calculate for 256 different possibilities, we see the palindrome or not, I just store a bit for each of them, saying it's a palindrome or not, and then my palindrome test is very simple.
I just look up my look up table and say, hey, is this a palindrome, no?
Yes?
No?
Done.
So if I'm doing this computation a lot of times I suddenly realize I can precompute the kind of things we did before.
I am very safe, I can do that.
So sometimes you have this programs that has very complicated control structures but when you look at this here you realize [UNINTELLIGIBLE] use multiple times in here.
There's S1, if v S2, else S3.
S4 again, if v S5.
And so there re a lot of conditions going on in here.
And something like that you can sometimes look at that and say, like OK, look, because my conditions are simple.
I mean in here, I only do one, I just do that one and I will create the path depending on what the Boolean expressions are.
And even if you have two or three conditions you can create a couple of parts depending on that.
So I'm replicating some stuff, like S1 is in both, S4 is in both.
So I have expanded my code a lot, but now instead of having this spaghetti code that my compiler and my architecture is going to stop all the time because [UNINTELLIGIBLE] prediction and stuff, as Professor Leiserson pointed out, can have issues.
I created this nice single branch, lot of code to execute, lot of chances for my compiler to do a lot of optimization.
I can say things like that.
And compiler can do it most of the time and sometimes you might want to help the compiler.
Procedure rules.
So one interesting thing is a lot of times when you have a lot of small procedure calls.
A couple of things.
OK, let me ask you, what's the problem with having a lot of small procedure calls?
You cause a lot of small methods all over the place.
What's the problem with small methods?
[INAUDIBLE]
[UNINTELLIGIBLE] yes, because every time you do a method call you have to basically go change everything [UNINTELLIGIBLE], put it back in memory, stack, and then go do that.
I have a basically calling [UNINTELLIGIBLE] expensive part.
OK, so if I'm doing a very small thing your calling context might have a lot more than overhead than what the procedure does, method does.
What else?
What's a more subtle thing that can happen?
So if you're thinking more, I'm a compiler guy so I think like compilers, what would the compiler do at this point?
[INAUDIBLE]
Compiler [UNINTELLIGIBLE] oops, barrier, I have no idea what's happening in here.
I had retrieved this as just bit stop.
I will do a bunch of things above that, I will do a bunch things below that.
But I can't go through that kind of thing, so the compiler basically treats that as, I can't touch that barrier.
And that [UNINTELLIGIBLE] means the amount of stuff that the compiler can do is [UNINTELLIGIBLE].
So one way to do that is basically inline.
On a max function, it's pretty simple to inline.
Basically you can define a macro that the CE front end will go inline for you.
So this is simple.
Or you can define inline method in here and put it in the same file, and then we will tell the compiler, look, go and try doing inline listening, and by doing that we are forcing the compiler, or telling it the ability to inline, and they're not only getting rid of the overhead of the method calls, but also you're letting the compiler a lot more opportunity to go and do something interesting.
So here's something you'll find in many streaming type programs.
What you do is you run through a loop, reading through a bunch of data, do some processing and writing it back.
And then you want to get the data you wrote back, read it again, do some processing and write it back.
The problem in that this one is the first loop which stretches through all data.
So by the time I'm done with the first loop I have written all the data and the thing I wrote early is probably all the way down in my cache hierarchy.
Because I wrote a huge amount and it doesn't count it in my cache so it goes down, down, down, and the next guy has to read it all the way from bottom up.
So [UNINTELLIGIBLE] means instead of doing that what you can do is run for a small amount of data.
So don't run the entire data set.
Run a small amount of data, write it into a buffer in the middle that we know sit in the cache, and the disk guys picks up the buffer and finish it up and write it back.
So by keeping the data in the buffers what they have done is between these two I actually had some nice locality.
I can keep the data up.
So as we go into memory systems and stuff like that these kind of patterns come up a lot.
Ah ha.
Tail recursion elimination.
So here there are many functions that the last thing you do is call itself.
OK?
This is pretty expensive because what normally happens is when you call a function you set up a calling contest, you do all this context [UNINTELLIGIBLE] put things into the stack and go, but you don't need to keep instead because you're basically done with the previous guy.
You're just calling just to go create another stack.
In a lot of these cases you can reformulate the problem into nice loop that has no function calls.
So, in here, when you're doing factorial, you can actually formulate the factorial in a way that you don't have to do a function call.
So a lot of times if you end up in a situation that the last thing you do in some part is the call itself and if you're not doing anything afterwards, you can always end up in a situation where you don't know how do that, you can actually convert it into [UNINTELLIGIBLE] loop.
And by doing this I got rid of all my calling overhead and stuff that I'm just starting a nice tight loop.
OK?
So if you have patterns that look like that you can think, OK, look, this is nice, I thought through [UNINTELLIGIBLE].
I got this there.
But now I can [UNINTELLIGIBLE] transform it to more iterative way of executing.
In many cases you will find a way of doing that.
OK, then there are a bunch of expression rules.
So a lot of times you have this computation that if you think carefully you can let the preprocessor do in something like calculate something like that, if I know what [UNINTELLIGIBLE] are, stuff like that, I can just put it in as a constant and voila I don't even have to pay any cost to compute it because by the time it's compiled all the computing is done.
And there are a lot of good cases you can just get rid of work by giving it to the preprocessor.
I'm just calling sin(a) and sin(a) twice in here, and the sin(a) squared I just basically, the compiler probably won't know that the sine every time you give it the same number, you give it the same same input, it's a function, I think the compiler doesn't know that, and this is something you probably have to do for the compiler.
Especially when you know the function has no side effects.
This is a little bit more funky, so assume I do two similar computations, you might want to do them together as a single computation that gives the compiler more opportunity, but that means you relay the function like a sine, cosine, s of 1 function.
It's a little bit of funky thing that in this list of interesting things to do.
But I put it there because there might be cases you might find that's interesting, like if you're doing taking min and max, it might be calling min and max separated.
You might be able to call it together.
So the last set of rules is parallelism rules.
Of course, we are going to visit parallelism like crazy in this class, but today we will just go through some very high level stuff to give you a little bit of feel for what can be done.
So I have this rule, this program.
So what's wrong with this program?
Why would this program run very slow?
Can anybody tell me?
Why can't, in this program, I can't do the full utilization of my machine.
The problem in this program is every iteration until this condition is calculated I can't start calculating this.
And so every iteration I had of it, this is finished, condition calculated, and [UNINTELLIGIBLE] and new calculate that.
And once that's done, you go to the next iteration.
We go to next iteration.
So by doing this I can't do too much work.
On the other hand, if I can unroll the loop like two times and have two different conditions, now I can do this condition and these conditions, and these two can be done in parallel and when these two get resolved I can do these two operations.
So instead of doing xmax, I'm operating two different variables and at the end I can resolve that.
So what I'm doing is I'm calculating two things in there.
And they go to four things, whatever, to keep the machine busy.
And each of them can be calculated independently.
OK, so I can get parallelism and at the end of the day I can do that to help when I'm summing something or whatever.
Do you see this?
So to get the six instruction issued at a time, something like that will give you more abilities because there's more things that can run in parallel.
So most of the time it will be good to see whether you can eliminate these kind of chains like here xmax is a chain because xmax-- until the previous is calculated I can't calculate the next one, until that so I have this chain that until this is calculated I can't do this, until this is done I can't do the next iteration, so xmax is kind of keeping me very sequential.
By doing that I have two things I can calculate.
So other interesting things is if you are doing through this kind of data structure link list.
Sometimes if you have I believe the bypass have another point that points couple of things ahead then I can keep multiple of them going on.
So I can do sort of waiting for the next one I can basically switch two elements, do that so the next one I can get some parallelism in here so I can do things like that.
So to get some basic parallel performance going on.
And this is basic kind of a little bit of a data structure augmentation.
You add additional data make it run faster.
And you want to get all your loops vectorized.
That means you want to run things like assembly instruction.
If you're using 32 bits, you want to put a couple of them together, run in one SIMD unit.
But I don't want any of you guys to go write assembly to do this.
It's going to be a big pain to write all the SIMD assembly.
We are not talking about assembly but most the compiler, [UNINTELLIGIBLE] compiler is somewhat good at doing it.
So what this means is you're kind of really nudge the compiler, look at bunch of compiler frags to get the compiler to do it.
Sometimes it's-- when we go more into it, we'll figure out how we can do it looking at given the right flags to compiler, sometimes chaining the program a little bit makes it easier for the compiler.
So sometimes just wrestling with the compiler, that's much more easier, believe me, than trying to do it by yourself.
By doing that you can get some interesting performance.
And we will talk a lot more about this coarse grain parallelism and we have a bunch of lectures on that.
Trying to figure out how to get multicores, each core do something parallel, and get all your [UNINTELLIGIBLE] in the machine working for at a given time then only one guy.
And so here's an interesting situation.
So I have this problem here.
Can I run this parallel?
So what I'm doing is I'm doing these two loops and I'm adding all the elements of an array to total.
Is this parallel?
Can I run anything or any of these things parallel?
Somebody says, yes, no, whatever, what do you think?
Take a stand.
OK, what do you think?
I think yes
Somebody says yes, good.
Yes, takers, no takers.
Yes, everybody says yes.
[UNINTELLIGIBLE] is parallel.
But look at here, what happens is previous total is needed to calculate the next total, so how can I run this parallel?
[INAUDIBLE] for loop jumps every two
OK, so she's somewhat onto it, so let me show you what we can do.
One thing you can do is instead of calculating one total for each basically column, you can calculate its own total.
OK?
So what happens is so for this i's row, j's column.
So for each row, so each row gets a different total, a temp total, and so what that means is you can calculate all the columns total separately.
You can total all the columns, and then total up those values to get the full total.
So totalling up columns can be done parallel because now all of these things can be done parallel because it's totalling up two different, multiple different values.
So it's kind of what she said but you do it for each row separately.
And so here's something, you made the program a little bit more complicated.
Actually for a sequence it run a little bit slower but at the end of the day you actually get the program parallelizable.
So a lot of times when you're parallelizing is to also looking at that algorithm little bit changes to the program that at the beginning might look like you're slowing down the program but ending up getting good parallel performance.
So a lot of parallelism looks like this.
So here's the entire list of things we talked about.
It's out there so you can go look at them and see if you understand them better.
And I'm leaving you with this example that I'm not going to have time to go through.
which is my program of a traveling salesman problem.
So what you're doing is you're looking at a way for a tradesman to travel to all of the city's with a minimum cost.
Of course the minimum shortest thing is exponential algorithm so you don't do that.
There's a good heuristic, that the greedy heuristic every time you go to the city you go to the cheapest city that's not visited and you just jump around and that's seem to be pretty good heuristic.
And so I call this up, and I will go to the end and then just finish it up.
Oops, I went too far.
So what I did was I have eight different stages basically, each stage I did some changes and then I got this program to run about five times faster than from where I started.
So you will see at each stage what I did.
So this is basically out of the menu of things I showed you I found an instance I can do and in this program I got five times faster.
This is more normal than the 300 [UNINTELLIGIBLE] I got [UNINTELLIGIBLE] and that's a real extreme case, but enjoy looking at this code.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I'm going to talk a little bit about computer architecture.
And how computer architecture impacts performance engineering.
So the main part of this is going through a long overview of the Pentium, the Nahalem architecture that in the machines that you guys are using, which is hot off the press.
Six core processor that's out there now.
And then I will talk a little bit about profiling a program.
And then the next lecture, the TAs are going to demonstrate how to use some of those profiling tools that you guys are supposed to use for the project two.
And I guess the project two will probably appear, since we had a delay, so probably be up here 24 delayed.
We'll release the project, too, up there.
And then at the end of the talk, end of the lecture, I'm going to talk about a little bit of example.
Go to some program and show what you can gain by looking at things like profiling, and what information that you gain.
Just kind of do a high level view.
So let's start something.
So how many of you have had a chip in your hand.
Microprocessor in your hand.
Some.
There's some people who haven't.
So I brought some show and tell things.
So what I'm going to do is pass them around.
And there are two things that are actually very valuable that I want to tell you and then make sure that they don't get damaged.
So this is a Pentium 3 that's already in a big heat sink in there.
So you can't see that much, but they're really cute and they created a hologram of the chip and put it there so you can catch, I guess, people who are trying to use counterfeit and stuff like that.
So this is MDK 6 with the packaging.
I'm going to send it around in there.
And this is a Pentium Pro.
Normally what happens is that when the first dye comes out, this is humongous, giganiticized.
But after a couple of generations, what you do is you take the same circuit and you shrink it and shrink it.
This is probably what two shrinks-- so the actual dye right now is very small because this is copper [UNINTELLIGIBLE PHRASE] later process that things have been shrunk.
This is a Core Two Duo So I think there's a two cores in there.
So I didn't want to bring anything newer because those chips are darn expensive, and if you touch it, basically just static will basically destroy the process.
So these are something that came out of things that doesn't work.
The next two things.
These are-- I really need to get this [UNINTELLIGIBLE] because these are like the only thing that's the one available in this world.
Basically there's only one of item.
So at MIT, we build a process a couple of years ago called a row processor.
So here's a row dye before it's being mounted.
So this is what comes out after all the fabrication, before it goes into this mounting with all the pins.
If you look at the top of this one, what do you see?
It's all these dots, which are basically where all the pins go.
So you don't actually even see inside the dye in this one.
However, then you pay a bunch of money to these people who go and grind that chip.
You take a real working chip and you grind it, and expose insides of the chip.
So this is a ground chip that's exposed the metal layering.
In fact, this you can see real circuits in the chip.
So I don't think anybody has the mission that's required to see the transistors or even the circuits.
But it's kind of interesting to see.
So here are these two things, so some show and tell like that.
Ah-ha.
And this.
This is entire reason that microprocessor not getting faster anymore.
Because this is one humongo heap thing that you put in the modern machines, and there's a huge fan that goes above that.
And you can't keep building larger and larger heat things to get the heat out.
And so that this why the heat is paramount, and that's why we can't run 4 gigahertz, 10 gigahertz process.
We are thick into the current kind of gigahertz train, and then we [UNINTELLIGIBLE] multiple cores.
So it's a big, humongo block in there, and then some actually might even have a bigger block.
So while this go around, let's start the lecture.
So computer architecting includes many different things.
You are doing instructions, memory, IEO Bus, disk systems, GPU, graphics, all of those things.
But we are not getting into beyond instruction and memory system.
So we are going to focus on that, but if you really want to understand the full end to end performance, you have to worry about all those other things.
So that said, let's go into instructions and memory.
So here is the Nehalem processor.
And there's a beautiful picture of the processor in the left.
This as kind of the role chip that I'm showing.
They have ground out the upper layers and showed some of the layers in there.
So that's what you get when you buy the Nehalem.
And this very complicated diagram in the right side, it's an abstract notion of what's happening inside.
So what you do is go a little bit into this [UNINTELLIGIBLE] diagram, and trying to understand what impact it would have on you.
This is not architecture class, but you, as trying to get performance, has to make sure all these components works pretty well.
That said, it's very hard to understand in a modern microprocessor exactly what's going on.
Not even Intel understand what's going on most of the time.
And so a lot of things [UNINTELLIGIBLE].
So you can't ask tell me exactly what happened or get there.
So you can be fussy.
On the other hand, being fussy means there's a level of abstraction that you have to live with.
And what that means is when something hit their head very hard, and have a really bad situation you can see that and you can react to that.
But there are a lot of small things that happen.
These things interact in very complex ways.
So you don't understand exactly the minor detail what's going on in these microprocessor for a given application.
A lot of different things might work different ways.
So that's it.
So if we look at what you learn from 004, so what that means is instruction.
You have here two instructions.
One after the other.
So if the instructions take [UNINTELLIGIBLE] cycle.
So we see the non-[UNINTELLIGIBLE] method.
So you look at why it's probably taking five cycles.
So that's because it's doing different things.
You are doing instruction fetch, instruction decode, after that execute.
Then you do some memory [UNINTELLIGIBLE] and then find the right packet in there.
So you do all those things [UNINTELLIGIBLE] instruction, and then you start the next [UNINTELLIGIBLE] cycle.
Of course, this is very necessary.
So the first two people [UNINTELLIGIBLE].
After I do instruction fetch, that logic that's doing instruction fetch is not doing anything for a while.
Why don't I start the next instruction phase immediately because you are doing the same circuit, and then recycle, that's so it can go do the next thing, and next thing, and next thing.
And by doing that, well I can basically recycle, I can get the instruction through the system.
So this looks very nice and simple and you can get.
So what are they choosing here?
Is the world this nice and simple?
No.
OK, what might happen?
[INAUDIBLE PHRASE]
You have a lot of issues that normally cause hazards.
So there could be three different type of hazard.
There's a thing called structural hazard.
What you are trying to do is attempt to use the same hardware, do two things, but there's only one hardware.
You can't get two things done.
One has to be after another.
And there's a thing called data hazards.
That means you're trying to run these things, one now another, but in logically.
They have to run sequence serially.
So that means that's a data dependence, I'll get into, that makes it impossible to make these things run in the pipeline fashion.
And finally, this control has things like branches and things like that, [UNINTELLIGIBLE] interfere with that.
So let me get a little bit detail down here.
So the first thing we have is what we call data hazard.
I will talk about two different hazards.
So before I go there, I will share a little bit about this assembly representation.
In two lectures, you're going to get a more deep drilling into the how to go from C to Assembly.
But before that, so what this instruction says, this is the normal x86 form.
We are doing add long.
Add in to the values in this rbx and rax and put the results back into rax.
So you are doing rbx plus rax and get the result into rax.
You are subjecting rax with rcx and put the results into rcx.
So basically all the rest of the results are in the right-hand side.
Two [UNINTELLIGIBLE] are basically the first and second.
So the last one gets read and modified.
So that's the way it's weighted.
So while you have two is-- question?
[INAUDIBLE PHRASE]
Instruction, yeah.
It's different concept.
The second instruction is data dependent, of course.
What's happening is I am writing this value here, which will be read by this guy here.
So I write something and the next instruction is reading that.
The problem is it's the right thing must not be available until very late in the pipeline.
So they cannot execute simultaneously and all that thing basically of this dependence.
This is called basically read after write, because I'm going to read after it's been written.
And if you look at the pipeline here, what happens is the write happens here, and the next feed has to happen somewhere down here.
So until this one, this slide [UNINTELLIGIBLE] so this status has to get delayed after this point.
This make sense?
Because I'm trying to in here read something that's not going to be produced for two more clock cycles.
So that's not available and I can't do that.
Make sense?
OK.
So this next cycle dependence is called name dependence.
That's the dependence and anti-dependence.
Basically what it's doing is two instructions are using the same register.
And because of that I can't start to see the radius there until the other one switches off with the register.
So in here, what's happening here is I am subtracting rax and rbx.
I'm putting the value in rbx.
And here I am adding rcx to rax, and basically start putting the value in rax.
The problem here is I cannot modify rax before this guy has [UNINTELLIGIBLE] the value in here.
So that means I'm trying to go modify it, and say no, no, no.
You can't touch it because somebody will still want to get the value, and I can't go destroy the value.
And I had to [UNINTELLIGIBLE].
That's the first type of what you call anti-dependence.
And because of this, you both are using rax.
There's no real data movement, but it basically has the same space.
I had to wait till the other person can be evicted before I can read that register.
So these are called write after read hazard.
The other type of dependence is called output dependence.
So I am updating rax twice, and of course in this simple example we can probably even drop this instruction because it doesn't matter because I'm re-writing it.
But if there's something in between also reading, what happens is I can't do this in wrong order.
The last value that updated has to be this instruction.
So we had to make sure this instruction happens after this instruction.
So we have a dependence, so there's some ordering in here that has to be maintained.
And this is call write after write hazard.
So instructions that have medium dependence, we can actually get rid of the dependent by what we call register renaming.
So everybody uses rax, so that's kind of artifact of good old inter [UNINTELLIGIBLE] architecture sometimes [UNINTELLIGIBLE] when register, so you have to use the same register for many, many things.
So in a modern-- inside the hardware there's what we call a register renaming.
You have lots more slots for the same registers, and even when you're using rax, this is an old one, this is a new one, use a different location for the new one.
So I can do renaming.
And then basically hardware can get rid of that.
The next interesting thing is control hazard.
So here what we see, if you had this kind of a loop, s1, it's control dependent on p1.
So we can't do s1 until p1 is done.
Or s2 until p2 is done.
So we are [UNINTELLIGIBLE] the p1 condition is [UNINTELLIGIBLE] before you can do s1 and [UNINTELLIGIBLE] for the next one.
So the interesting thing is control dependence also we can get rid of it by doing speculation.
So the idea there what hardware does is it will say wait a minute.
I know I have to wait till p1 is [UNINTELLIGIBLE] to do s1.
But I'm going to do s1 anyway.
I will just go to s1, speculatively.
And then at the end of doing that, at some time when p1 is calculated, and I know what the p1, I said did I do it right?
If I did it right, I'm good.
If I did something wrong, I have done some [UNINTELLIGIBLE] that is not useful for that.
Question?
[INAUDIBLE PHRASE]
So what happens is there's this thing called a write buffer.
The complex thing is at the end of the day, the instructions have to be get committed in the order they arrive.
So before committing, you keep the state in buffers without [UNINTELLIGIBLE] into the main one
[INAUDIBLE PHRASE]
So right back into memory doesn't happen until the commit point.
But when you read something, before reading the memory, say OK, I have updated something, is it in the right buffer?
So I read things from the right buffer in there.
So you had to go in order commitment.
Because you can't do out of order commitment, that can do crazy things.
You do in order commitment, but inside the metrics things can go easier.
I'm just kind of jumping the gun a little bit.
So in a modern Nehalem processor, these are kind of Intel [UNINTELLIGIBLE], so you don't know exactly what it is.
In some place it says it has 16 clock cycles.
So it says it might be 16 stages of pipeline.
And another place it says 20 to 24 stages of pipeline.
OK, if you work for Intel and sign the NDA you will get the exact number.
But it doesn't matter.
It's a lot of pipeline stages, so that's all you need to know.
So what happens is if you are doing this in sort of five stage, like what you did in probably beta.
You do instruction decode, instruction queue, pre-record queue, decode, register rename, and allocate registers.
And then there's thing called reservation and stations that wait for when data's available to execute that instruction.
And then execute, and then basically go there.
This is what normally instruction life would be, if everything goes one after another after another.
So what to get out of this is the pipelines are long.
So pipeline stores and stuff can be expensive.
So the other thing you can do in abstract way is do multiple issue.
So if you do pipeline, what happens is you do one at a time so you can use the pipeline stages nicely.
How about instead of having one unit, if you have integer unit and a floating point unit.
And every clock cycle you can basically abstract.
You could say look, I'm taking interger instruction, I'm taking floating point instruction, and take them both together [UNINTELLIGIBLE].
And then whala, at the end of the day in one clock cycle I can [UNINTELLIGIBLE] two instructions, one integer, one floating point, if you have two different sets in there.
So this is also very nice.
You can get a lot of what we call super scaler performance.
However, it's called instruction level parallelism.
There's a lot of problems, also things that you worry.
Well first of all, you have to have enough instruction in level parallel.
Because the instruction you get is the sequence history, and you see one after another.
I mean you write your program and compile.
It looks like you run one instruction, finish it, run another, run another.
So in order to form parallely, you're to find things.
You can actually do parallel.
That takes time.
And of course, between hardware and software you are to maintain that preserve order of instructions, because you want to make sure that it looks like and it feels like you ran one after another after another.
You don't want things to run in a haphazard way and create arbitrary result.
And so you have to, again, things like this, data dependences, control dependences have to be satisfied when you're getting this parallelism.
So data dependency, there's a hazard, and determining which order you can run things.
So for example, output dependence.
Say look, I can't just run them parallel.
I have to make one after another after another.
You're to get all those things done.
And also if you're lot of dependency kind of gives you bounds, how much parallelism you can get.
If things are all one dependent against one after another, you can't run them parallel.
You have to wait to run things to another one.
So by looking at data dependence, as you can figure out, OK look, did I get good performance, good ILT or not?
So what we want to do is exploit this parallelism like we see in the program order.
And basically make sure that you always get the same result on that.
One way of getting parallelism that is in modern process is called multimedia instruction.
It's called SIMD in academic circle, something like Single Instruction Multiple Data, and it's called data level parallelism, and Intel, of course, has to give the [UNINTELLIGIBLE] name they call SSE, they just call it MMX.
So the idea there is look, they are building this wide measure.
They can easily build 128 bit wide register.
But you most probably don't need 128 bit data, because that's too big.
But they can build this large.
And most will tell you about happy with about 32 bit data.
So what?
[UNINTELLIGIBLE] wait a minute.
You build this wide thing.
But you can chop it into four pieces or eight pieces or two pieces, what you want.
So here we chop that large thing into four pieces.
And then what you can do is in a single instruction, you can instead of doing one large add, the same type of adder, you can add four separate things.
Four small parts.
So you're assuring add, but instead of adding 228 bit data, what you are doing is adding here, four 32 bit data.
The entire architecture looks very identical to an 228 bit data.
It's basically to remember your double 004, it's like a carry.
Then just have to come the carry.
You don't carry from here to here, and do that.
So this is you're loading this large chunk, and you are just working on these things.
And whala, by doing that, having the same kind of circuitry, you get much better parallelism.
And if you have like 8 bit data, boy, you can get lots of parallelism.
So there's all these multiple instructions in there.
And so what you can do is you can have a loop like this.
So you can, in every situation, you can add the I to BI, and create pretty good [UNINTELLIGIBLE].
So you can do one at a time.
But assume you have 32 bit values.
Instead of doing-- single instruction you can load 0 to 8 3 in one go.
B0 to BC in one go.
Just do the add of all four in one, go [UNINTELLIGIBLE].
So in single time you get four things done.
So this is really cool.
And if you remember my first lecture, we showed this matrix multiplier.
If you get [UNINTELLIGIBLE] in there, you get a lot less instructions executed.
We got a nice [UNINTELLIGIBLE].
So I'm going down, so the top path I'm showing, the C code-- for that loop, and then the bottom half in there, I'm showing the normal assembly that we [UNINTELLIGIBLE], what you see.
And once you start understanding assembly a little bit more, you can go ahead and read this and see what's going on.
And then if you [UNINTELLIGIBLE], you get a humongous beast like this.
This has unrolled [UNINTELLIGIBLE], whatever.
And this is the kind of code that we generated by [UNINTELLIGIBLE].
And that's just much, much faster than the one on the left.
Any questions so far?
OK.
So when we have this superscalar [UNINTELLIGIBLE], with multiple execution, so what you can do is in a clock cycle, you can do a lot of things.
So in the [UNINTELLIGIBLE] processor, you have basically six things, six execution things that can happen in one clock cycle.
It's not the same thing, so you can do a data store.
You can store address, load address from there, and then you can do three different type of executions.
There are three units in here.
In here you can do either integer or SSE, add or move instruction, this unit can do this kind of instruction, you can have a floating point add-in around here.
This one needs to also have this bunch of different things you can do.
So this complex hardware, we'll kind of figure out, OK, if you have these instructions, figure out what of these [UNINTELLIGIBLE] are available.
If the instruction is able to execute, it will put it into one of these and het the results out.
So if you're lucky, what you can see is, you might get about six instructions going each cycle.
So if you look at the mesh here, what we call clock [UNINTELLIGIBLE] instruction, it could be 1/6.
If you're lucky.
If you can run the mesh into the metal, that means everything is working all the time.
It's very hard to find that kind of program, but [UNINTELLIGIBLE] can get something like that.
That's really cool.
So what happens is to get this thing, they do things called out-of-order instruction.
So what that means is, issuing this varying number of instructions in the clock, in here, in the Nehalem processor, you can store 128 instructions in this core instruction buffer.
[INAUDIBLE] better name than instruction buffer.
If you can see that, [INAUDIBLE] very far.
But [INAUDIBLE] up there, sorry.
Here.
It's called reservation stations.
So when you put it in, all those [UNINTELLIGIBLE], reformation [UNINTELLIGIBLE], ready to get, and the minute it becomes available, that means all the data is ready for it to run, you can basically issue that and run.
So what they're doing is, in these hundreds of instructions, you're figuring out the patterns you can run.
And a minute some part is available, the data becomes available, you can run that.
And so that means the program might have instructions in certain order.
It gets executed in very different orders than what the program has.
So that's what call you can give things like [UNINTELLIGIBLE] register, [UNINTELLIGIBLE] dependencies, [UNINTELLIGIBLE] execution.
And sometimes you might run speculator.
You might not know.
You might have the data, but there might be controlled dependence.
You might not know that it's actually is supposed to happen, but it says wait a minute, I have execution available.
It's not doing anything.
OK, let's run it and keep the results, and if it is actually useful, we'll use it.
Otherwise, we'll throw it away.
You can do things like that.
And that's a speculation.
So another type of speculation-- speculation mostly means, you want to do something, but you don't know whether you can do it.
So how do you go about doing that?
The way that interprocess and modern processor do, is they do a lot of predictions.
It has like this is [UNINTELLIGIBLE] say, look at the crystal ball and say, aha, this is the future.
And it does that [UNINTELLIGIBLE] really many things.
So it does things like branch prediction, value prediction, prefetching.
So it says, aha, I don't know where you're going, but I think you are going home after the class.
And then probably if I said something like this, it's probably right for most of the people.
And actually, no, in this class, most of you, again, will probably go to sleep after the class, because of the project.
So if I do that probably I do a very good prediction.
How many of you are going to go to sleep after the class?
Huh, my prediction is not that good.
OK. We'll see.
So you can go all these kinds of things, identify normal behavior programs to deal with that.
Ad what prefetching says is, if I look at what you have been taking from the money, there are some patterns, and I'm expecting you to [UNINTELLIGIBLE] that pattern, and I will basically get the data, even though you don't ask for it, because I'm expecting you to be doing that.
It's like a very good butler who can really think, you ask something, have that ready for you, basically.
So by doing that, you can get much better parallelism, because now you're not waiting for that to be sure that you can do it.
You're just kind of doing this, assuming you might be using it, and if you do it really well, you're actually going to use it.
And that will be useful.
So for example, even things like value prediction.
Value prediction says, [UNINTELLIGIBLE] to get out, you know when you do this complex calculation, most of the time, data is zero.
You multiply and add a lot of zeroes, and if you're doing spreadsheets, all it is [UNINTELLIGIBLE] calculations.
So if I don't know the value, let me assume it's 0.
And that actually works most of the time.
Or you look at the last couple of times, what has happened to that [UNINTELLIGIBLE], and there, let me predict something.
And people to find these kind of things that are a lot of patterns in [UNINTELLIGIBLE] program.
And if it works, that's really great.
So in speculation, what you do is first issue and execute.
As if the branch was-- I already knew where it is going, just by using the prediction.
And if I just know dynamic scheduling, I am only doing [UNINTELLIGIBLE] issues, I am not going to execute until I know everything is [UNINTELLIGIBLE].
So it's speculation goes one step beyond.
It's what we call a data flow execution model, the dynamic schedule.
And that means, the minute the data is ready and everything is ready, you can execute, even though it might not mean the same mode of the program.
It might be much later instruction.
But implicitly, you can execute that.
So again, the [UNINTELLIGIBLE] pipeline, you can do 20 to 24 stage or 16, whatever is correct.
The other thing it does is this crazy thing called micro-ops that puts everybody out to a loop.
So what happened was-- this is kind of historical-- Intel, a long time ago, came up with the x86 architecture.
They came up with this instruction set.
That instruction set is called a CISC instruction set.
It's a complex instruction set architecture.
What that means is there are these instructions there are that are really, really complex.
You can move an entire string from one place to another in one instruction.
The problem is, most instructions cannot be done in one cycle.
So what modern [UNINTELLIGIBLE] did was, they built a [UNINTELLIGIBLE] compiler into the hardware.
So what happens inside this process is, take this CISC instruction and compile them down to these particular micro-operations, that each operation is small can be done in basically one cycle.
So it's doing this mapping in there.
And then inside the computer, it's basically dealing with micro-ops.
So up to here, it's dealing with these long instructions, and here it's precoding and decoding into micro-ops.
So after that, so what that means, every cycle, it can issue 6 micro-ops.
So there might be 6 instructions you know, 3, or even 1, depending on how many micro-ops [UNINTELLIGIBLE].
So if you look at instruction manual in the [UNINTELLIGIBLE], you can see how many micro-op [UNINTELLIGIBLE] instruction it's going into, and if it's large, that means those instructions are slow.
And you can have this 120 micro-op waiting to be executed, sitting in this reservation station.
Basically, they are waiting there.
The minute the data is available, and then a slot available, it will go down, get executed, and come back.
So branch prediction, basically-- the problem with branches is if you have this very nice pipeline, what happens is, branch target is not known until this target detection time.
Where is my mouse?
OK. Until this point, I don't know whether the branch, where I am supposed to go.
And so how do I go ahead and get the instruction, because I don't know where I'm going until this point.
Even worse, I might not even know address.
Some [UNINTELLIGIBLE] you are going AOB, you are taken or not taken.
But sometimes there might be [UNINTELLIGIBLE] branches.
I don't even, if I'm returning the return address in the stack.
Until I go fit the stack and load it and all those things, I don't know where I'm going, so I had to wait until this point.
So if I do it, basically I have to wait until this pipeline.
All these things are going to stalled.
Nothing happens in here.
This is first stage.
You can have a interested pipeline.
You are waiting 20 stages, just stalled, doing nothing.
And then there's superscalar plus 20.
So that means 20 cycles without the [?
6 issues ?]
you can do.
So 6 plus 20 is 120 possible instructions wasted, and it's a lot of waste.
So you don't want to do that.
So what you do is, you build a predictor to figure out which direction the brand is going.
And depending on what the predictor do, and the neat thing is, these days, these predictions are really, really good.
They can predict up to 99 points plus [UNINTELLIGIBLE] where you're going.
So it's like me telling you, you guys are all going to sleep afterwards, and this is going to sleep on me.
So what you can predict, I will say, aha!
I tell you, you are going in that direction.
This is like going to [UNINTELLIGIBLE].
I think you're going that direction, and then you just go there.
And 99% of the time, you're OK, and when [UNINTELLIGIBLE] finally decided, you can say, oh, that was slight.
If you're wrong, you squash everything and restart.
And so modern predictors are this very complicated beast.
It looks at things like what happened previously.
It looks at [UNINTELLIGIBLE] call stack where are the call stack, what went into the call stack.
And then when you're returning, you can predict where we might be returning.
A lot of different things go in there, and in fact, we see the core microarchitecture that came before Nehalem's predictor, and Nehalem [UNINTELLIGIBLE], oh, it even has more things, and so it's even more different.
So these guys are doing these complex things.
What that means is, sometimes it affects program in a way that you think, oh, huh, this is something that I don't know what might happen.
[UNINTELLIGIBLE] this condition, I might not know what might happen.
But model prediction sometimes surprises you.
Aha, I know there's a pattern, I recognize that pattern, and I go about that.
And every architecture, I try different kinds of patterns, and I find the architectures get better and better.
So for example, at some point, if you go odd-even, odd-even branches, if you assume I have a loop, I say, if i is odd, going one direction, otherwise not.
Pentium 3 type thing, OK.
It just screw up [UNINTELLIGIBLE] with every time it's doing something different.
I don't know what's going on.
[UNINTELLIGIBLE] Pentium I'd probably say aha, there's a pattern, odd-even, odd-even, it figure out.
I core 2 figures out things like every third, every fourth type pattern it figures out.
And then things get better and better.
There's very complicated patterns you do, these guys manage to figure out.
Which is kind of fun, to see what it can figure out, what it can't.
By writing a very small program, you can get that.
I show some of that.
So the next thing that you have to worry about is the memory system.
So the memory system, if you want to be the computer, what you want to do is have a lot of very fast memory very close to you.
You can do that.
It's going to be very expensive.
Sometimes through SQL you can't even attain that, because by the time you build a lot of things, it's actually already far away from you, because we are running very fast here.
So the way you deal with that is building caches.
That means you keep a small amount of things close to you in there, and you put things in that cache in a way that hopefully, those are the things you will be using anyways.
So it works very nicely.
So the reason that can be done is in programming, when you run a program, there are two types of behaviors that it [UNINTELLIGIBLE] to take advantage of.
One thing is called temporal locality.
What that means is, most programs, if you use you some data, there's a good chance you are going to use the same data very soon.
Because normally, there's a thing called a working set.
I am working with a certain set of data.
I'm basically touching these things again and again.
So if I use that, I'm going to use that data again.
So that means I want to keep the data that I used close by.
Other one is called special locality.
That means, if I use some data, I have a very good chance that I might be using a neighboring data item.
Because if I am accessing a structure or an array, if I'm accessing something, I might be accessing a [UNINTELLIGIBLE] neighbor thing next.
So there's what's called partial locality.
And taking advantage of these two, you can build this very fast memory system that feels like you have a huge amount of memory very close to you.
Of course, the opposite is true.
If your program doesn't behave like that, it looks like you have lots of memory very far from you, and the program becomes very, very slow.
Because memories didn't speed up as fast as processors.
So what that means is, every generation, the memory [UNINTELLIGIBLE] further and further away, slower and slower and slower.
And this is how we manage to keep the machines running fast.
So if you look at what's going on every level, I just gave you a diagram.
I won't go through detail.
You can look at it later.
There are caches, and higher levels you can only keep very small.
And one like [UNINTELLIGIBLE], you can only keep a couple of hundred bytes in the registers.
But very fast exercise.
Then you go to cache.
I see on-chip stuff.
We can keep a little bit more, access time go down, and then finally when you go to main memory, you can keep a huge amount, slow access.
And even at the end, something like tape, you can keep a huge amount of stuff.
Almost infinite amount of stuff.
But it takes hours to get the tape access is not [UNINTELLIGIBLE].
So there's this hierarchy, and there are a certain amount things you want.
You are using [UNINTELLIGIBLE] very fast.
You don't want to put it in tape.
But you know something that you might access in a year, that's probably a good place to put it.
So there's this hierarchy there.
And then you have cache, what happens is, you want to give the illusion everything is there, but when you go to access it, the data might not be there.
There are many reasons it might not be there, so let's go through some of the reasons.
So one thing is called cold miss.
That means it's the first time I've seen that data.
So it cannot be in the cache.
If it's the first time I was asking the disk, it's probably sitting very back in main memory, or somewhere in the disk, and I have to go get it.
One way to get around this problem is prefetching.
So if I keep accessing this I say, aha.
You're accessing this in this pattern.
And there's a good probability the next pattern is here, and OK, go get it very fast, because you're coming in that sort of direction.
So by doing prefetching, you might be able to get rid of [UNINTELLIGIBLE] is happening.
Then there's a thing called capacity miss.
What that means is, you're accessing a lot of data.
At some point, you have accessed enough data, everything you accessed cannot be fit in the cache, and you have to throw out some of the data to put the next one.
So most caches use the a policy called least recently used.
That means the data that you unpacked for the longest time gets thrown out of the cache.
But the problem is, at some point, you're going to come back to it.
And if you come back to it after a long time, that thing has been out of that level of cache.
This happened at every level.
So what that means is, you have a thing called a working set, that you keep working again and using.
If the working set is a little bit larger than the cache, when you come back to that, the data is gone.
It's not in the cache.
So you want to kind of create a working set that fits in the cache nicely, at ever level, basically.
You can do something like pre-fetching and gets around it, but if you can avoid [UNINTELLIGIBLE], that's really nice.
Then there's a thing called a conflict miss.
One way of cache work-- it doesn't let you store everything in every location.
Sometimes there are some locations in cache [UNINTELLIGIBLE] associated with it.
Did you get [UNINTELLIGIBLE] associated in 004??
Associativity?
OK. Again, I will talk about cache associated [UNINTELLIGIBLE], but I'm going to talk about memory system later and [UNINTELLIGIBLE] there.
And what that means is, this is a really bad behavior, because you are only accessing a small amount of things, but all of the data, all the aggregate can't fit into the cache.
The pattern makes it only fit into a small part of the cache, so you had to throw the data out.
I'll get back to this.
I'm just going to put it there.
And then in a multiprocess system, [UNINTELLIGIBLE] multicourse, there's a thing called true sharing.
That means I [UNINTELLIGIBLE] data [UNINTELLIGIBLE], and the next time [UNINTELLIGIBLE] want to trust the data.
So I have to give the data to you, so you're getting cache.
And then when I want to use my data, it's [UNINTELLIGIBLE].
I have to get it back to me.
So I'm kind of using data back and forth, back and forth.
So that's called true sharing.
And a more hideous form of that is called false sharing.
That means most of the data is in a cache line.
When you move data you move the cache line.
So what I'm doing, is I'm using my data myself, and you are using some other data, but unfortunately, they're in the same cache line.
So when I say, OK, I want my data, I get the entire cache line, including your data, and then you want to use your data.
You say, oops, it's not with me.
And I then that means you [UNINTELLIGIBLE] go to you, and at that point, I don't have my data in my cache.
So this data is going back and forth.
Even though I never touch your data, we have two separate data, but it's going back.
I will get this things in a lot more detail in later lectures, so this is kind of giving [INAUDIBLE].
So there are all of these different ways that the data can be not in the cache.
So here is what the processor you are working with looks like.
There are 6 cores, and there are L1 separate instruction and data caches, and then there's L2 unified cache, [UNINTELLIGIBLE] L3 cache or [UNINTELLIGIBLE], and then there's main memory.
So it's even this deep, deep, deep cache hierarchy going on then.
And then right-hand side, I kind of showed the difference.
And the interesting thing to realize is, the L1 cache delays about 4 nanoseconds.
It gets a little bit, 2 1/2 times slower when you go to L2 cache, and [?
12 ?]
times slower when you go to L3 cache, and even more slow when you go to main memory.
So every time you go down, it gets slower and slower and slower.
That's basically the gist of it in here.
Question?
Each core has two L1 cache for instructions and [INAUDIBLE]?
And instruction data.
So instruction goes one direction, data goes in one direction.
So next I want to talk a little bit about profiling.
So predictor who is going to be-- the first part of the predictor is all about profiling.
Profiling is very important, because you run a program.
It doesn't do well.
So how do you know what's going on?
First of all, you do know, even if it's not doing well.
And if it's not doing well, what's the reason?
So just having one number, that then saying it's [UNINTELLIGIBLE] in 10 minutes, [UNINTELLIGIBLE] 5 minutes, doesn't tell you too much.
You know what to look if you have a large program.
The profiling means going, looking at what the program is doing to get an understanding in there.
So what you want to do was collect this performance data while running the application, and then if you're a large application, you want to organize and display data in that a variety of ways, and a lot of times, relate that data to source code, and see what that means in the [UNINTELLIGIBLE] code.
And hopefully by looking at this one, you can identify, ha, there's a problem here.
So there are a bunch of tools.
Intel Vtune, gprof, oprofile, perf.
So you guys are going to use mainly perf, and we will probably talk a little bit about gprof.
And next time when you come, next lecture we'll talk about this a lot more.
So profiling is mainly to find where in an application or a system there is a significant amount of activity.
And when the significant amount of activity you want to know whether those activity can be avoided, or there's a problem here.
So there are some [UNINTELLIGIBLE] significant to what's there, it could be anywhere.
It might be addressed in the memory, something might be happening.
It might be in the [UNINTELLIGIBLE] system, some kind of process in the operating system might be happening in one thread, it might be happening in an executable file or a module.
If you know the symbol of something, you can say, oh, huh, that mode, that actually is this function, and if you know even more information about the program, you can say, aha, that's false, and in fact it's this line of the program.
So you get this information from the application when you compile that you can say debug information, say aha, this happens in this line of the program that's having this problem.
So you can break it down and get that info like that.
[PHONE RINGING] Somebody's trying to call me.
Hold on.
OK. Secondly, we care about significant activity.
If the activity just happened only a few times, happened only nanoseconds of execution time, who cares?
You don't want to do that.
You want to focus on things that matter.
And the key thing about this [UNINTELLIGIBLE].
If you spend most of your time in insignificant things, you would get insignificant performance improvement.
If you found the significant part and work on that, you can get significant gains.
So the key thing is to find the significant ones.
[PHONE RINGING] Excuse me.
OK. And the final activity.
So activity means time is spent in doing something, and some activities are bad ones.
Like if you actually do running instruction, that's good.
If your running is useful instruction, a long time, you're OK.
But you might doing actually [UNINTELLIGIBLE], [UNINTELLIGIBLE] [?
misprediction ?
], stuff like that, that's bad activity.
You [UNINTELLIGIBLE] say, aha.
I am spending a lot of time doing something that I can avoid, and how do I avoid that?
So that's what you want to look at.
And you have two ways of doing that, and I want to give you an analogy to figure out what it is.
So assume you are going, visiting a bunch of different places.
You're on a city tour.
You are visiting different parts of the city.
And I want to figure out, where do you spend most of your time?
And that's a hard problem, because I am sitting in my office, and say, OK, you're going [UNINTELLIGIBLE] the city.
I want to figure out where you spend most of the time.
I have two ways of doing that.
I'm a busy person.
What I can do is every 30 minutes, I can call you and say, where are you?
And you say, OK, I'm in this library, I'm in this-- and then at that point, I can have a histogram and say, he's still in the library.
And I can call again, where are you?
And I can log that.
And at some point, at the end of the day, I'll have a histogram saying, every time I call, you found something.
If you spend a lot of time in the science museum, I will have a bunch of [UNINTELLIGIBLE].
We'll see [UNINTELLIGIBLE] become then, and if you are not [UNINTELLIGIBLE], if you have only one [UNINTELLIGIBLE] find out, OK, you're not spending time there.
That's one way to do that.
Let's go [UNINTELLIGIBLE] saying, for every landmark, I create a telephone booth.
And every time you enter a landmark, you [UNINTELLIGIBLE] the telephone booth, you've got to call an operator and say what time it is, so [UNINTELLIGIBLE].
And then you call me and say, aha, I'm entering this landmark, the time now is 5:50, and I write it down.
And every time you exit the landmark, you call me and say, I'm exiting this landmark.
Time now is-- I write it down.
OK?
These are kind of two ways of doing that.
That's like an instrumentation solution.
So the sampling collector-based periodically interupt the processor and look at where you are, and depending on where you are, you can mark that off.
And it's called time-based sampling means every time, every 100 milliseconds or something, in [UNINTELLIGIBLE] processing, you stop and say where you are.
Event-based sampling means you are counting number of events like cache meters.
Every hundred cache meters, you stop and say, OK, where has this happened?
Of course, if the cache misses happen in a regular pattern, you might be in trouble, because every hundred, you might be the same place, then you would get a skewed number.
But most probably, there's a statistical variation in there.
You can get [?
account index ?]
and figure out where these things happen.
So if you're looking at where all the cache [UNINTELLIGIBLE] is happening, you don't look at every miss, because there are millions of miss.
Every 10,000 miss, you figure out where you are, and statistically, one is missing many times, that will adapt in that column.
And nice thing about that is, there's nothing you need to do.
Now installation, no changes to application you need to do.
[UNINTELLIGIBLE] here, you know how to go in changes, phone booth everywhere.
Wide coverage.
You can cover everything.
So that means, assume you install phone booth in all the fixed landmarks, but there's a service [UNINTELLIGIBLE] down.
I didn't see.
I don't have a phone booth there.
But here, if I'm calling you, I know I'm [UNINTELLIGIBLE] you, anyplace you are, I will cover, even though I haven't anticipated that point.
Very low overhead, because I can decide to call you every 30 minutes or call you every 1 minute, if I really care or worry about you, and I can control the overhead that I'm looking at, basically.
The problem is its approximate position.
Because if you spend 5 hours at MIT and 30 seconds at Harvard, I will never know that you went to Harvard, because I might have not called you while you were there at Harvard.
You might think it's boring and come back, and I never knew that you did that.
And also, the other thing is, I don't know exactly how many times you went to a place, because I might have called you 10 times, and I found you in the museum of the science 10 times, but you might have gone there 20 times, and I might have missed you under the times I didn't call, or you might have gone there only five times and stayed for a long time, and I might have called multiple times.
I don't know that, and I don't know the count of times you are gone, which is hard to know.
So the main thing is there might be things that are not that statistically significant that you might miss.
And if you care about that, it's not a good bet.
But most of the time we care about the really statistically significant things, so this is a really good method.
The other part is perfect accuracy, because every time you go somewhere, you have to call me, and I know how many times you went to the museum of science.
I know exactly the time you enter and exited a [UNINTELLIGIBLE].
I know all of those things [INAUDIBLE].
The problem of that is kind of low granularity.
I can't put phone booth at every corner, and if you had called me at every corner, it's going to be way too expensive.
So I have low granularity, very good information, and also high overhead.
Because if you're going in and out a lot to a building, you have to call, you have to stop and call, it's very high overhead.
And there's not much of a way for me to control that.
And it's also high touch.
That means I have to build all that infrastructure, I have to go and modify your application, basically, in something [UNINTELLIGIBLE] compile time application, get modified to basically have all these [UNINTELLIGIBLE] installed into the application.
So you can look a lot of different types of events.
So in Intel, if you look at core performance counters, there are hundreds of performance counters.
And some of these performance counters have no sense whatsoever.
So for example, Intel has this counter called number of bogus branches.
What's a bogus branch?
I mean, why should Intel be doing anything bogus?
It's some [UNINTELLIGIBLE] came out.
But there are some counters that are useful, so you're sort of getting [UNINTELLIGIBLE] by thousands of things available, we focus on things that we care about.
Things like branch events, load store events, cache meter, prefetchers.
Those are the important things, and some multicore events that we can look at, and get some interesting information.
And a lot of times, just by looking at numbers doesn't make too much sense.
You know you had $5 billion, $300 million branches missed.
Is it a good high or low?
You have no idea.
The right thing is, OK. [UNINTELLIGIBLE] to number of instruction executed how much it would be.
The most important numbers are ratios.
So from the branches executed, how many were missed?
That's a lot more important than, you have 5 billion branch misses.
That doesn't say anything to me.
So most of the time, you have to figure out the right ratios, and what makes sense to look at those ratios.
So now what I want to do is to go through a couple of examples and show what I can see in these different program behaviors.
Any questions so far?
Next lecture we are going to go through hands-on examples, going through some of these profiling tools.
So it's kind of fun-- what I did was finally discover what architecture is doing, and what is the modern multicore is capable of doing.
In fact, some of these examples, so this set of examples were done on a Core Two.
So some of the numbers might be very different in [UNINTELLIGIBLE], because the architecture might do better in some special prediction stuff.
I don't know.
It might be fun to move it it again.
So what I have is a program that I first created two interesting arrays to access this array.
One array has numbers 1 to n, MAXA, is stored in this array.
This has 1 to MAXA in random order stored in this array.
So when I use this as a way to access the main array, and if I use this in [UNINTELLIGIBLE] 0 to n, I will actually access A [UNINTELLIGIBLE] 0.
It's almost like saying AI.
But I use this here.
But if I use this one, I'm doing random [UNINTELLIGIBLE].
Well, the first program I'm doing is just nothing but going through this one.
I will have autoloop that will go through many, many times, so I can actually [UNINTELLIGIBLE] time.
So I just go through-- just nothing but trying to just go through that.
Second thing I did, I just put a condition.
Say, j is than MAXA half.
That means halfway through the program, I will go and update this one.
Another half, I'm not doing anything.
Then I said, OK, look.
I divide by every fourth [UNINTELLIGIBLE].
So what I is j and with 03 in there.
That means 3 means [?
2b ?]
in the 1 1.
So we study 0.
That means last two [UNINTELLIGIBLE] has to be 0.
To form in delta every fourth element, I will update.
Otherwise I won't.
And the next thing I did was I updated-- OK. Let me ask you a question.
So this program, the output-- that means what happened to A-- the output of A is equivalent to some other program, one of these three programs.
Which one is the [?
outward ?]
equivalent?
See what you can make up.
So first three programs or each will have separate different outputs.
A at A would be different if you run it.
Except the fourth program will produce the exact same results as some other program, on the first three programs.
Which one?
Wake up, wake up, wake up!
I hear something.
I hear some-- second one.
Somebody said the second one.
How many people agree with that?
How many people disagree?
OK.
This is not a statistical example.
It can be exact.
Why say the second one?
Second one is right.
So what that means is, in fact, instead of a j, I'm using inc j. Inc j is exactly j, because it goes, inc 0 is 0, inc 1 is 1, inc 2 is 2.
I just created the same [UNINTELLIGIBLE] array in here.
This is something to get that.
And finally, I am doing the [UNINTELLIGIBLE] A basically from x.
So what that means, I'm using the same amount of data will get updated, but it's very different order in there.
So now, before I go to the next slide-- question?
Maybe you said this, but what is the A array?
Yeah, it's some numbers.
Some random set of numbers.
I don't care about that.
I just updated this one.
So in here, which program do we think run the faster?
So let me ask you this-- which program updated the least amount of updates for A?
Third one, yeah, because it's only doing every fourth element.
Which did the most?
First one is doing every element, and everybody else will do probably half of the updates.
So knowing that, which program will run faster?
First one is updating everything.
[UNINTELLIGIBLE], OK. [UNINTELLIGIBLE] plus 1, OK.
Which program will run slow?
Last one.
You guys are onto something.
You have seen my slide, you actually know what the hell is going on.
So what I show here is, you can't read this one, but I [UNINTELLIGIBLE] on time, and then I create a ratio.
So if you look at the ratio-- so aha.
The first program, I always normalize to that, runs 1, and that runs the fastest, if you tried, and the last program ran the slowest.
And the interesting thing is, the [UNINTELLIGIBLE], the other, the two, the second and fourth one, that wrote about half [UNINTELLIGIBLE], basically ran all of the same.
But the one that wrote every fourth element was actually slower than others.
Why is it going on?
So what you can do is, first look at how many instructions would it add up, and how many instruction were executed?
So if you look at that, what you see is, the first one executes the least amount of instructions, because it doesn't have to do all this [UNINTELLIGIBLE] when it's updating.
The interesting thing is, the one that wrote half has the most amount of instructions.
One that wrote only 1/4 has middle amount of instructions.
That kind of makes sense in here.
But this still doesn't explain the slowdown.
That means if this is the only case, the divide by 4 should run faster than the other three.
That doesn't make sense.
And then what you look at, look at [UNINTELLIGIBLE] branches in there, and if you look at the branches, what you see is-- Let me skip this one, actually.
I want to get this one.
This is called clocks per instruction.
What this is is, if the instructions ran slow, these instructions will take a lot more clock cycles.
That means the instructions are not doing well.
That means they are stalling, they are taking a lot more time.
Normally, we should be able to run [UNINTELLIGIBLE] about four instruction [UNINTELLIGIBLE] cycles should get a CPA of 0.25, but that's very rare, and if you look at that, if I normalize, I get 1 in here normalized.
And then what you see is second and fourth test, almost the same CPI, and the third has worse, and the last one is even worse.
So why is that?
Why are those instructions doing bad?
And then you can go look at that misprediction instruction, and this is exactly what happens.
Divide by 4.
This is a condition.
The branch predictor couldn't predict it that well, because [UNINTELLIGIBLE] predicted in the first three, and it's [UNINTELLIGIBLE] predicted, OK, after three, next one is also going 1 direction, oops, it's in another direction.
OK, going first three predictions, going one direction, knowing that information.
So divide by 4, looks really bad, then predict 3.
And then the random one was just completely off.
Random one, basically predictions of three [INAUDIBLE].
That's why this is.
So in fact, if you just want to calculate the run time, if you think every misprediction cost you 21 instructions, you kind of get the exact total first.
You can't help it.
So after you meet the instruction, take one cycle, and everybody misprediction instruction, take 21 cycles, and you just calculate that.
And this [UNINTELLIGIBLE], I just came, took it out of a hat, and then voila, I get a number that's very close to run time.
So what that means is, I can kind of understand what the behavior, this explains what's going on.
So I can have a model in my head for what's going on.
So it's all about misprediction in here.
I did two other experiments in here.
I have the slide sitting here, but I don't think I have too much time to go.
I want to go to the numbers in there.
Are you ready, [UNINTELLIGIBLE]?
OK. Why don't you go put your laptop in.
You have the dongle?
[INAUDIBLE]
OK.
So voila.
So something you guys waited all this time, and were up all these days developing.
So let's see how everybody did.
OK, without further ado.
I haven't even seen these numbers.
OK, what happened?
He's trying to-- OK, good.
This is [UNINTELLIGIBLE] screen, OK. Do you have it?
OK. Something to keep in mind is, we haven't actually investigated what's causing people to have build failures or crashes yet.
So it could be your fault, it could be ours.
We'll figure that out
So if smaller, better or higher, better, what's--
[INAUDIBLE] better.
This is run time.
That
This is really good!
This is actually--
[INAUDIBLE] The baseline is the flat code that you see in the middle
So this is really good.
That means there were kind of two groups.
Most of the people got everything right, and they're now the bottom.
And there's another group that missed something, and it's in the second camp
I think some of these projects are from people who dropped the class at some point.
So there's probably [INAUDIBLE] might not be actually representative of the class
So last year, we got like almost exponential curve, actually.
Last year, what happened was there were a couple of people at the bottom, and everybody worked at the top, basically.
Which is actually good.
So you guys actually figured out what's going on
So next one is even more interesting
Wow!
So this is [INAUDIBLE] that we gave you guys.
Most of you optimized it down to 0 seconds.
So I coded in a harder test case, and here's the results on that.
Still pretty good.
[INAUDIBLE] There's some very specific optimizations [INAUDIBLE].
Yeah
So exactly.
There are people who are climbing up, I don't know exactly what, but it's very clear that [UNINTELLIGIBLE], probably data representation tag that people missed in here.
Which is really now interesting.
Yes
And for Pentominos, this is very incomplete, because the set of tests that we wanted to run are still currently running.
So I just picked a random test case, just to give everybody an idea of how everybody did, and set the time out very low so that I could actually finish it during this lecture.
And what's [INAUDIBLE]?
Yes.
Somebody solved it instantaneously, and it searches for the first 5,000 solutions to some random puzzle that I pulled out
So [UNINTELLIGIBLE]?
That I haven't verified yet
Aha.
So it might be, there might be just the scratchings, so we don't know that, and then that could be correct answer, but--
Which one is the baseline?
The baseline is actually the 90-second mark.
[INAUDIBLE]
[UNINTELLIGIBLE] the baseline?
No.
10 seconds and above are either timeout or [INAUDIBLE]
Oh, they're probably mostly timeouts
[INAUDIBLE]
Yes.
So there you are.
This one, people would have some work to do, I guess, to get the performance down.
But this is pretty good.
This is not like [UNINTELLIGIBLE].
I mean, last couple of years, we had people, [UNINTELLIGIBLE] someone who was 1,000 [UNINTELLIGIBLE], you had to plot it in a log thing.
So in fact, if I do a square root within the grid, this is pretty nice.
So [UNINTELLIGIBLE] is not that bad.
So this is great.
This is where you get [UNINTELLIGIBLE] performance wise And if you are in the bottom, you can go have a beer.
Otherwise, you might have to go back and figure out what you missed
[INAUDIBLE]
Oh, if you're [UNINTELLIGIBLE].
Yes, exactly yes.
I should say that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
GUEST SPEAKER 1: All right.
So we're going to talk about performance engineering and pretty much how to do that with profiling tools.
So first, we're going to talk about what profiling tools are and how you use them in general, what the theory of that is.
And then, more importantly, we're going to do an interactive walk-through of two examples that we picked out.
The first one is the Matrix Multiply that you guys saw in Project 0, the cache ratio measurements and how to do those measurements and how to do the cycles per instruction measurements, these kinds of things.
And then we're going to look at doing some of the bit hacks that Charles talked about in the second lecture.
We're going to work on a branchless sorting algorithm, which is pretty interesting.
So the code we gave you in project one, it was very obvious where the hot spot was.
But real life doesn't really work that way.
And code gets really, really big.
It's not always obvious where the part is that you want to optimize.
And you really don't want to waste time taking what is otherwise understandable, good code, and beating it over the head trying to make it go faster.
So that's why Donald Knuth said, "premature optimization is the root of all evil."
You really want to focus on the hot spots.
So that's what profiling's for.
It basically gives you visibility into the execution of your code, because it would be too complicated to just sit down and look at it and try to understand where it's spending its time.
And there are many possibilities for where it could be spending its time.
In 6-172 we're mostly going to be focusing on things that are CPU-bound and things that are using the memory hierarchy ineffectively.
But in the real world, a couple of other common things are network requests.
So if you've got your web application, it's making a whole bunch of AJAX requests.
It's going to be wasting a lot of your users' time.
If you've got your desktop application, it's hitting the disk all the time.
That's also going to slow you down.
I think SQL database, if anybody's ever done web development, they know that can be a huge bottleneck.
So basically, the tool that you're going to use is going to depend a lot on what the problem you're solving is.
But the basic ideas are all the same.
There are all kinds of profiling tools for measuring these things and figuring out where it's coming from and where the time's going.
So for trying to figure out CPU and memory, there's a couple things you can do.
The first and most basic and sort of most obvious to you guys is you can do manual instrumentation.
And what that is is basically where you go in and you put in prints and try to figure out where it's spending its time, what codes could you have executed more.
Or you can be a little bit more clever.
You can try to insert logging calls that fill up a buffer with checking the time and actually doing that yourself.
And one of the big advantages of that is that, when you actually go into the code and you instrument yourself, you can cut down the information that's coming back out at you.
If you have an intuition for how the program works, sprinkling little calls to timing functions around your code in the places that you think matter is going to give you maybe a better picture.
But it requires more work on your part.
So that's the drawback.
The other kind of instrumentation is static instrumentation, which is basically-- and I should say that instrumentation is basically a little snippet of code that is gathering information, gathering data about the code execution.
So static instrumentation is basically code that's inserted by the compiler when you build your code.
And gprof is an example of this.
Basically, what it does is it will, at the prologue and the epilogue for every function, insert a little piece of code that increments a counter every time the function gets executed.
I think it also figures out who called it, that kind of thing.
One of the drawbacks to static instrumentation is that you have to have the source, and you have to be able to rebuild your project.
If you've ever had a DLL binary blob of data library handed to you, and you want to know where the time's going in that library, you're not going to be able to instrument it.
So you're not going to get any data.
So that brings us to the next thing, which is dynamic instrumentation, which solves that problem.
Basically, what happens there is you take the binary as it is just before you're about to run it.
And before you start executing, you actually just take a look at the code and translate it.
And you go in, and you insert the instrumentation.
And then you have this new snippet of code that you then run instead.
And so that works for binary blobs.
And it doesn't require a recompile, which is great.
But it introduces a lot of overhead, because you're not actually running the original code, right?
You're running this translated thing, which might be much more inefficient.
And I guess Valgrind has a couple of tools that are sort of classic examples of that, Callgrind and Cachegrind.
I think Callgrind gives you a call graph.
And Cachegrind tries to measure the cache effects, tries to make a prediction of when you do memory accesses, are these things in cache or not.
Another thing you might want to look at, or the thing that we're going to focus on in this course are performance counters.
And two tools to get these are OProfile and perf.
And basically, the CPU has a bunch of counters-- we've talked about them-- for branch misses, cache misses, these kinds of things.
And this will give access to those.
The last tool that I guess I'm going to talk about for profiling is heap profiling.
A lot of times, if memory's your problem, you really want to figure out who's doing all the allocation.
And so these tools will give you a nice breakdown of who's allocating what, who's calling them, and who can I stop calling so that I'm allocating less memory.
And like I said, you can find more tools for profiling different kinds of problems and resources.
But today we're mostly going to focus on perf, which is a new tool for getting access to performance counters on Linux.
I guess I should also mention that, as someone talked about in the last lecture, the way that you want to gather data from performance counters is you want to do event sampling.
There's too much data.
As you execute along, you don't want to record, OK, I executed this instruction, then I executed that instruction, and I'm going to count all the instruction executions and then store that data somewhere.
That would be way too much data.
So what you want to do is you want to sample every so often, every 1,000th event, or some other frequency for whatever your interesting event is.
And basically, what happens is that when that counter has hit the threshold, an interrupt will fire, and the kernel will catch it.
And then it will record the data that you want, the context, like where in the execution it was, what the call stack looked like, these kinds of things.
I think we've covered most of the performance counters that we mostly care about.
But I think it's worth mentioning that there are a lot of performance counters.
And you can go look them up in this part of the manual.
They cover a lot of interesting things, not just the ones we've talked about, including sleep states, power consumption.
And if you want to look at contention, say you have a parallel program, and you have two cores accessing the same data.
If they're writing back to it, they're going to be fighting over the same cache lines.
And so there are counters for things like, how many times did this core have to go off core to get cache line, and all kinds of interesting things like that.
But mostly, we're going to focus on the cache misses, branch misses.
I also wanted to mention that the tool we're using, perf, it kind of replaces OProfile and PerfMon, which are tools that do similar things.
I guess it's different in that it's a unified interface to monitoring two kinds of events.
The first are software kernel events, so things like contact switches.
So how many times did your application make a system call and then get contact switched out?
And how many page faults did your application generate?
These kinds of things that the kernel knows about, but aren't necessarily hardware events.
And the other kind is hardware events.
And those are sort of the ones we're going to focus on.
We've already talked about those.
If you want to get this tool and try it out yourself, if you have any recent Linux distribution, anything with a kernel that's less than a year old, you can install Linux tools.
And the command is perf.
So now we're going to do a demo of the matrix multiply from Project 0.
GUEST SPEAKER 2: All right.
So for the demos, I guess I'll narrate what the code's doing, so Reid can focus on typing.
This code should look familiar.
All right.
Can everybody hear me?
All right, cool.
So this code should look very familiar.
It's Project 0's matrix multiply code, the original version.
And it's a triply-nested four loop for multiplying matrices.
So what we're going to do is I'm going to switch to a terminal that Reid is typing on.
And we're going to run matrix multiply.
And it takes about eight seconds to run.
And now we're going to run it, but have perf output some performance statistics for it, basically, CPU counters.
So you use -e to request specific counters.
You can use perf list to take a look at all the counters that perf supports.
In this case, we're asking for the number of CPU cycles, the number of instructions, the number of L1 cache loads, and then the number of L1 cache misses.
Perf stat.
That's better.
So it takes a little bit longer to execute, I think, with perf.
But it's not supposed to be a big difference.
You can see, 8.31 versus 8.34.
So it's relatively low overhead sampling, which is good, because you don't want to change the behavior of your program when you sample it.
Important things to see.
So we're looking at this cache miss rate, which is 18%, almost 19%, L1 cache miss rate, which is pretty bad.
So as a result of that, if you look at the number of instructions, the number of cycles, perf calculates the IPC for you.
We previously talked about CPI.
But perf uses IPC, which is instructions per cycle.
So basically, every cycle, you only executed half an instruction on average, because you're waiting for the cache.
So that's no good.
So can anybody remember from Project 0 what you did to work around that?
Yes
You reordered the loops.
GUEST SPEAKER 2: Yes.
So the before and after.
You just swap out the inner two loops.
And we will run it again.
OK, so remember, the previous runtime was 8 seconds.
Now it's down to 2.5 seconds, which is a lot better.
Let's run the perf stat command again and look at the performance counters.
And as you can see, the IPC jumped from 0.49 to 1.4.
So it tripled.
And if you look at the new L1 cache miss rate, it's 1% rather than 20%, which is a lot better.
Yes?
Can you point to where the cache miss rate percent is?
Because I just see zero per second.
GUEST SPEAKER 2: Oh, sorry about that.
So L1 D-cache loads is all of the times that the CPU asked the L1 cache for something.
And then L1 D-cache load misses is the number of cache misses.
So you divide this over this to get the ratio, which is basically what Reid did over here.
Does that make sense?
Yes.
GUEST SPEAKER 2: OK.
So this is a ridiculously simple example of how to use perf, just so that you can see the command set.
But that's not really a very interesting example.
So the next one that we'll look at is actually something that Charles, Reid, and I did yesterday afternoon for fun.
It started, we were testing your Project 2.1, one which is-- I don't know how many people have looked at the handout yet.
But it deals with a bunch of mystery sorting algorithms that you have to figure out what they're doing.
And so we took one of them.
And we were running sorting.
And Charles asked, what happens if you have a branchless sorting algorithm?
And we said, we don't know.
So we played around with it.
And in the process, we optimized our branchless sorting algorithm by using perf to generate insight into what the CPU is doing.
And as you'll see, we generated quite a good speed-up.
So the first thing we'll do is, let's start with the baseline, quicksort, everybody's favorite sorting algorithm.
So I'll switch over to the terminal.
And Reid is going to run quicksort on 30 million integers one time, which it says takes 4.14 seconds.
Now, let's use perf stat to see what it's doing.
So this time, we will ask for slightly different events.
We'll still ask for the cycles and instructions to get our IPC.
But we'll also ask for branches and branch misses, because we kind of intuitively know that quicksort is doing branching.
So OK.
It's doing 0.8 IPC, so 0.8 instructions per cycle.
And the branch miss rate, so branch misses over total number branches, is 11.5%.
That's pretty bad in terms of branch missing.
So let's take a look at the code and see why that's the case.
So here's the part of the code in quicksort where you select a pivot recursively, and then you loop, and then you do your swapping.
So these branches are more or less unpredictable.
And looking at them, there's no way that we could think of to get rid of the unpredictable branches.
They're unpredictable by nature.
So this would not be an interesting example of which to do branchless sorting.
So let's switch to a different algorithm, mergesort.
So here's the relevant code in mergesort that takes a list, C, and two input lists, A and B.
And you merge the contents of A and B into C by taking the smallest element from either A or B, putting it in C, and then repeating that process until the lists are empty.
So the "if else" branches, as the comment says, tries to place the min of the value either at A or B into C, and then properly increment the pointer and do those housekeeping tasks so you don't crash.
So we're going to try running this and profiling it.
So Reid is running mergesort.
Took 5.04 seconds.
Well, the last one took 4.8 seconds.
So already, it doesn't look too good.
But let's try to see what happened.
So we'll issue the same perf stat command, asking for branches, branch misses, cycles.
Reid is being lazy.
OK, so the IPC is 1.1, which is a little bit better.
But if you look at the branch misses over branches, they're off by roughly an order of magnitude just like before, so 10%.
So OK. Mergesort is currently slower than quicksort.
And the reason is still, roughly, you still have all of these branches that you're not predicting correctly.
OK, so why branching, and why not caching?
Well, we did some quick back-of-the-envelope calculations, or research, rather.
And it turns out that in the Nehalem, the processes that you're using on the clouds, one of the design changes that Intel made is that they made the L1 cache and, actually, also the L2 cache faster, so fewer number of clock cycles to go out to the cache.
But at the same time, they also made the pipeline deeper.
So an L1 cache miss is maybe three or four cycles.
An L2 cache miss is 15 cycles.
By the time you miss the L2 cache, you're going out into L3 cache, which is 12 megabytes.
And if you're sorting integers, most of them are probably going to be in there anyway
John, those are actually hits [INAUDIBLE].
GUEST SPEAKER 2: Oh, they are.
OK, so misses might be a little bit bigger than that.
But
L1 miss is a L2-- GUEST SPEAKER 2: Right, is an L2
L1 misses
It was just a typo.
GUEST SPEAKER 2: OK, so the next number is probably, like, 30 to 50, somewhere on that order.
50-something.
OK.
But a branch mispredict will also cost you somewhere between 16 and 24 cycles, depending on how many pipeline stages are actually in the Nehalem, which we don't know.
So bad branch predictions might just be as costly as bad memory access patterns.
And plus, Charles is going to go into detail about memory access patterns and how to optimize sorting for good memory access.
So it wouldn't be interesting if I talked about that right now.
So let's optimize branching.
So to do that, we'll use a bit hack that Charles introduced in an earlier lecture.
So remember, all we want to do is we want to put the minimum of either A or B into C. So right now, we're using branches to do it.
But there's no reason why we have to.
So the trick that will apply is the XOR trick for calculating min without a branch.
Does everybody kind of follow what that code is doing?
You can refer to the bit hacks lecture later to check.
But trust us that it's correct.
GUEST SPEAKER 1: [INAUDIBLE] GUEST SPEAKER 2: All right.
I guess we can go over it.
Do I have a mouse pointer?
Cool.
OK, so inside, you're doing a comparing A is less than B, the value of A is less than the value of B.
And you store either a 0 or 1 into a flag called CMP.
And then you take CMP, and you negate it.
So then you end up with a bit mask of either all 0's or all 1's, depending on which one was smaller.
And then you mask A XOR B with that.
So this inner expression, you either get A XOR B out of it, or you get all 0's.
And then you XOR B into it.
So in one case, B XOR this expression cancels out the B.
You'll just get A out.
And the other way, B XOR 0 is going to give you B.
So that should be convincing motivation that this is actually a min function.
And then, once you have that, you can do cute tricks to recycle the flags, to do the A++ and B++ and so on, using CMP.
So next, let's run this code and see what happens.
OK, so we went from 5.04 seconds to 4.59 seconds.
And we'll do a profiling run just to get some stats on it.
OK, so now look at the IPC.
We went from 1.1 to 1.7-- 1.7.
And branch misses went down by a factor of 10, which is really awesome.
But overall, the performance didn't seem to improve by much.
Well, if you want to look at why, look at the total number of instructions executed.
That's that number versus that number.
We seem to be executing considerably more instructions.
So to see why that's the case, we decided to use another mode of perf called report, or record and then report.
So what this does is that it logs where all of those interrupts actually happen.
So it logs which point in your code the CPU was executing when these performance counters tripped.
And then it generates a nice report that we can call using perf annotate.
And the top section gives you a sorted summary, where it shows you the percent of times that perf caught your program executing that line of code.
So 11% of the times that perf triggered its interrupt, your code was executing mergesort.c line 32.
So let's go to mergesort.c line 32 and see what it's doing.
So this annotated view shows you the lines of C-code, which are in black.
C-code is in black.
And then line numbers are annotated here.
And then it actually goes ahead and shows you the assembly instructions that it's executing.
And it puts the timing actually on the assembly instructions corresponding to every C expression.
Yes?
Why is this [INAUDIBLE] is the annotated version-- GUEST SPEAKER 2: Yeah, you call perf annotate and then a function name.
The Project 2.1 will walk you through doing this a couple of times.
So you'll be able to see what it's doing.
OK, so the assembly here is actually pretty important.
I guess everybody's taken 6004, or some variant, where you had to code some form of assembly?
OK, it's a prereq.
Cool.
So Saman will talk more about assembly.
But in this case-- or Charles, actually, now will talk more about assembly.
But anyway, in this case, it was actually kind of useful to look at the assembly.
We won't expect you to know how to write assembly for this class or expect you to write high-performing assembly.
But sometimes taking a look at what the compiler outputs can really help you.
Well, in this case, what caught our eye was this cltq thing, because none of us in the room knew what the heck cltq was It wasn't an assembly instruction that you usually see.
So that caught our eye.
And we wondered what was this assembly doing.
So we spent a little bit of time tracing through this, working out what the compiler was doing.
So in PowerPoint, I pulled out this code.
And OK, so I looked up cltq in my 600-page Intel manual.
Yes, your processor does come with a manual.
So cltq, it says, sign extends the EAX register to 64 bits and then places that into RAX.
Why is it sign extending?
OK, so let's try to trace through what the assembly's doing.
So here, we wanted to do comp equals star A less than star B.
So it moves the value at the address in register 14 into RCX and does the same thing for register 13.
So assume that R14 and R13 are A and B.
And so RCX and RDX are star A and star B, right?
And then it XORs ESI with ESI, which is 0.
So it zeroes out the register ESI.
And then it does a compare between RCX and RDX, which is the value at A and the value at B.
So it does that comparison.
And then, depending on the result of that comparison, it writes that result out into register SIL-- GUEST SPEAKER 1: Which is the low-byte-- GUEST SPEAKER 2: SIL, as Reid said, is the low-byte of the register ESI, which is used here.
But before that, the compiler then executes this expression here, B XOR a, which it puts into register RDX.
And then, finally, it takes ESI, which contains the same value as SIL, and then it moves that into EAX.
And then it negates EAX.
OK, so that kind of looks like negative CMP, right?
And then, after negating it, it then sign extends it to a 64-bit register.
And then it does the en masse that we asked it to.
Well, why is it jumping through so many hoops to do this?
Well, it turns out, when you declare a value of type int, you say that it's a 32-bit value.
And the compiler took you very literally in saying that you requested a 32-bit value.
So it did a 32-bit negation.
And then it extended that to 64-bits.
So it wasted a couple of instructions doing this, instead of just flat-out doing a 64-bit negation to begin with.
Now, the problem with this is usually you don't care if it generates one XOR assembly instruction.
But this is a tight loop that's called in the recursion process for every single merge that it's doing.
So the time that it wastes here actually really adds up.
So how do we fix that?
Well, instead of declaring it as int, let's just try replacing it with long, and then recompiler the code and see if things are any different.
So here we start our process of trial and error.
So we compiled mergesort long.
And the runtime is 4.05 seconds.
Before, it was, like, 4.5 or so.
So the runtime definitely improved.
Let's compare it.
GUEST SPEAKER 1: Yeah, just to verify.
GUEST SPEAKER 2: So it went from 4.59 to 4.05, which definitely is an improvement.
So let's figure out, well, let's see if it generated better assembly.
So we'll run perf record in order to look at perf annotate again.
And OK, once again, it's spending 14% of the time somewhere in mergesort.c, which is hardly surprising.
We'll go to line 27 and, yeah, use long.
OK, so that's roughly the assembly.
And Reid, although he wasted time to find it, I have a nicely prepared screen shot of the same section of code.
So it looks cleaner.
It loads A and B into different registers this time, actually, RSI and RCX.
And then it clears out EDX.
And then it does the compare between RSI and RCX, which is A and B.
And then it sets percent DL.
Now, percent DL is the lower byte of EDX.
So it used a different choice of registers.
And then it did the A XOR B.
And then it moved that compare into RAX.
GUEST SPEAKER 1: [INAUDIBLE] GUEST SPEAKER 2: OK, and Reid actually has a longer snippet of the code.
And the compiler, actually, now that we told it that it was a 64-bit flag, the compiler realized that there's a couple optimizations that it could do.
So it switched around the order of things a little bit.
And it's recycling the value of RDX for loading, I think it's A+ equals comp.
It's recycling the value RDX that it generated out of a compare.
And now, down there, it's negating RAX as a 64-bit number and directly using it without that cltq instruction.
So that was kind of nice.
So in just nudging the compiler a little bit, we got it to produce code that ran roughly about 10% faster, which was surprising to us, actually, that changing a single int to a long could reduce the runtime by that amount.
So next thing that we did was we scrolled down a little bit more on this window.
And we looked at the next thing that the compiler was doing.
And we kind of caught the compiler again doing something silly.
So here's the code that we were at before, the compare code.
And now let's look at how the compiler did B+ equals CMP.
Once again, the reason that we stopped and looked at this code was we saw an instruction that we didn't recognize, SBB.
Well, we looked at the manual again.
And SBB of R1 and R2 is a subtract with a borrow bit.
So it does subtracting the two arguments and then subtracting a carry flag from that.
So OK, why's it doing that?
Well, let's walk through what the compiler tried to do.
Let's see, where's a good place to start?
So it's RDX in this compare.
So it compares RDX to 1.
RDX is where CMP was stored.
So it checks whether or not it equals to 1.
And then the result of that check is actually stored in the carry flag, CF.
And subtract with borrow, RAX, RAX.
Before, RAX held the value of B.
But now, since it did this, RAX minus RAX is 0, right?
And then minus the carry flag is either 0 or negative 1, depending on the value of the compare.
So once again, it generates a register that's either all 0's or all 1's, 64 bits.
Now, what did it go and do with that?
Well, jump down a little bit later on in the code.
And it ends EAX, which is the 32-bit version of RAX, with the value of 8 in hexadecimal.
So now it's either all 0's, or it's 8.
And then it adds that value, EAX, which is the same as RAX-- RAX is a 64-bit version-- it adds that value to RBP, which stored the address of B.
So it jumped through that many hoops just to increment B by either 8 or 0.
Now, this seems a little bit unnecessary.
So we thought, how else could we express B+ equals not CMP, and try to change the code around a little bit so that it does the same thing, but hope that the compiler treats it differently.
Well, in this case, not CMP, when CMP is either 1 or 0, is the same thing as 1 minus CMP.
Well, we compiled a version of this code, and then we tried to run it.
Refreshing our memory, mergesort minus, where we just did the minus sign.
So we took the runtime down from 4.04 seconds to 3.77 seconds.
And once again, this is just from replacing a not CMP to a 1 minus CMP.
That's all the code change that we made.
And we will look at the annotation again, just to make sure that the compiler generated different code.
Rather, we were curious of what the compiler generated this time.
So it's roughly this section of code.
Now, one thing you'll notice is, because we're compiling with-- this is already compiling with GCC's maximum optimization level, mind you.
So keep that in mind when you assume that the compiler is going to help you out and do smart things.
So the instructions are a little bit out of order compared to what you might expect.
All right, so I'll go to my prepared screen shot of the code.
So what did it generate this time?
All right, so already, this code looks shorter than the previous one, right?
A little bit less hairy than that.
So what did it do?
Well, SIL, once again, is the lower byte of RSI.
So it does the compare RCX with RDX, which is actually this comparison.
And then it saves that into SIL.
And now, later, it does the XOR.
So this was the code from before.
And then it does a move of RSI, which contains the same value as SIL, which is CMP's either 1 or 0.
It moves that into RAX.
And then, later on, it uses RAX for one calculation.
And then it also uses the value that's already in RSI.
And it subtracts that from register 14, which is where it decided to store the value of B.
So that's a much more direct way to either subtract 0 or 1, actually get just 0 or 1, and then use it for a subtraction.
So the end result, and something else to point out, is that the move and subtract commands have a little bit of instruction level parallelism, because right after you set [?
LE ?]
SIL, as soon as that's done, you can execute both the move and the subtract at the same time, since they use the same source register and put in different destination registers.
And the other thing to notice in the comparison is the number of ALU operations.
I'll go back to the original one.
You see negate, and, XOR, add.
So move is not an ALU.
It's not an arithmetic operation.
But everything else is either XOR, and, add, or subtract, or compare, which all use the ALU.
Now, if you look at the code over here, you have move, move, move, which don't need the ALU.
And then the other or four or five need the ALU.
But having fewer ALU op is a good thing.
If you remember from the previous lecture on CPU architecture, the Nehalem, has six execution ports.
So it can execute six micro-ops in parallel per CPU clock cycle.
Actually, only three of those execution ports have ALUs on them, so that they can actually do XOR, at, and, subtract, add, and so on.
The rest of them can do register moves and a couple of other things, but not arithmetic operations.
So that's a good thing.
So that's an explanation of why the code actually sped up.
Now, just as an overview of what we were able to do, we ran perf stat, the same thing that we did for mergesort, or for the original two mergesorts.
And we made a table of the improvements that we made.
So originally, quicksort ran in four seconds.
And it had an IPC of 0.8 and 11% branch missing.
So when we switched from quicksort to standard mergesort, we increased execution time by 20%, which is not good.
But the instructions per clock jumped up a little bit.
Branch miss rate is still pretty bad.
Now, as soon as we switched it to branchless, we got an 8% improvement over the previous runtime.
But note that we were still worse off than quicksort at that point.
But the instructions per clock jumped quite a bit from last time, which is kind of what you would expect with reducing the branch misses by a factor of 10.
So the CPU isn't spending extra time stalling the pipeline, trying to backtrack on its branch mispredicts.
And now the more interesting thing to me is, if you look at the two silly little compiler optimizations we made, switching from int to long reduced the runtime by 11% over the previous branchless implementation.
And then instructions per clock and branch miss both didn't change at this point.
At this point, we were just reducing the number of instructions.
And then going from there to changing not CMP to 1 minus CMP reduced our runtime by another 7% from before.
So overall, branchless mergesorting is 10.8% faster than quicksorting by the time we were done optimizing, which is a lot better than negative.
At this point, we were still behind.
And at the end, we ended up ahead of quicksort.
And then, overall, switching to branchless, with all the performance tweaks that we made, was 33.6% better over the branching version.
So the non-branching ran a third faster than the branching version, which is pretty cool.
And that's something that we learned at the end of the day yesterday.
So to conclude, what did we learn when we were doing this?
Well, I think a lot of that applies to the PSETS that you're doing and how you should go about optimizing performance on the code that we give you.
And it's profile before you optimize.
So before you spend too much time just reading through source code and making wild guesses as to why the code is slow, profile the code.
See why it's actually slow.
Get some performance counter data.
Do some perf annotate, so that it tells you what assembly or what line of code that it's actually spending the most time on.
And then, optimize iteratively.
Don't try to do everything at once.
Do things one at a time to see whether or not they make a difference.
Now, before we arrived at the not, there are so many different ways that you could have expressed not and 1 minus and negative and so on.
So if you try them all at once, you really don't know what made the improvement.
So you've got to try things one at a time and see what it causes the compiler to do, because you can see in the previous two examples, by the time we made that one optimization, it wasn't just that one assembly instruction that the compiler ended up changing.
The compiler, in fact, realized that it could do additional optimizations on top of what we hinted to it.
So it did even more optimizations.
But the bottom line is, you should optimize iteratively.
Try something, and see what effect it has before moving on and trying something else.
And then, as much as I don't like coding assembly, and I don't expect that anybody here codes assembly for the class, looking at the assembly is always a good thing.
Just skim through it for your performance bottlenecks.
And try to get an idea of what the compiler is asking the CPU to do.
And see if there's a way that you can nudge the compiler to do the right thing.
Sometimes we tell you the compiler will optimize certain things, will get rid of dead code, and so on.
And most of the time, that's actually true.
But it doesn't hurt to double-check in the assembly and make sure that it's actually doing that.
And the output from perf is pretty helpful to doing this.
It's not a monumental task of disassembling.
It does all of the annotation and interleaving for you.
And finally, learn through practice.
That's the only way to learn how to do this performance optimization.
You gain a lot of experience from just going out and trying the tools.
And Project 2 will have you working with these tools.
Project 2.1 does not have a code submission.
It's just a write-up, where you get to just play around with the tools on code that we give you, so you can see how to use the tools.
And then Project 2.2, which is going out next week, will actually give you a chance to try to optimize code that we give you, using these tools to help you out along the way.
So with that said, any questions about--
[INAUDIBLE] a write-up.
how do you submit write-ups for projects?
GUEST SPEAKER 2: On Stellar.
So the handouts for the write-ups are Stellar homework assignments.
Just upload a PDF to Stellar
OK.
GUEST SPEAKER 2: Yes?
[INAUDIBLE PHRASE] GUEST SPEAKER 1: Yeah, you just have to build the binary with debug info.
GUEST SPEAKER 2: Yeah, minus G is enough.
As long as GDB's break points can see your code, then perf can do it.
And it's actually pretty nice in that, even if your code is inlined, it can identify where the inlined portions came from and pull in the proper lines of code for that
I think what's really interesting and important here is not that they found this interesting thing, but the process they went about.
A lot of times, when you are coding, you are in control.
You have very planned structure how you're going about doing that.
And you follow these through and produce the code you want.
And sometimes, some pesky bugs show up.
You go through that.
Here is, basically, you basically let the system and data drive what you are doing.
And when you start, there's no planned part, saying, I'm going to do A, B, C, D. Basically, you look at the results, and you figure out what might change.
And you can go through that.
And I was actually talking with Charles, saying, in my first lecture, I talked about Matrix Multiply.
It looks really cool.
Every time I do something, it improved by a factor of 2, 50%, stuff like that.
What you didn't see is all the things I did that didn't have any impact on the program.
You saw this nice tree going down there, all these leaves that end up in dead ends, that had nothing happened.
So if you feel like when you go there, you do a bunch of things, nothing happens, that's normal.
Of course, when you tell somebody, you're not going to say, oh, that [UNINTELLIGIBLE].
Say, I did this [UNINTELLIGIBLE], and I did that.
And at the end, it looks like these other people are very smart, and they're getting through all these optimizations.
No, they spend a lot of time going through these dead end parts.
And the interesting thing in here is, one thing is, don't trust anything.
[UNINTELLIGIBLE PHRASE] Intel, they have hundreds of engineers working at it for years.
And OK, that should be good.
Not really.
So a lot of times, it's an end-to-end thing.
With performance, it's end to end.
So the problem can be at any level.
So it could become a compiler, it could be [UNINTELLIGIBLE] network, it can be network, it can be an operating system, it can be in the memory system, it can be in the processor, it can be anything.
So if you say, oh, yeah, that shouldn't be the case, probably that's where it is.
So all the time, basically, it's interesting.
The nice thing about these kind of tools is that sometimes even the tools might be wrong.
So sometimes I have been in situations where you look have to look at the wall clock, OK, wait a minute, does the tool say what I actually see what's happening?
Because you get into very skeptical situations.
Because a lot of times, in these kind of situations, the problem can be hidden in many different things.
So that's a very good way of going through this.
Of course, what you get handed to you, we have set it up in a way that there cannot be too many surprises.
But when you're going through a system and try to optimize and look at performance, this is a really good way of doing it, because I don't think these guys thought, when they started, they'd be looking at compiled bugs or compiler problems.
They were thinking about, OK, I'm going to do a little bit of code rewriting and get there.
But at the end of the day, what they disclosed was very surprising to them.
And it even surprised me today.
GUEST SPEAKER 2: Our plan basically ended at row three, as far as when we first set out.
We had no idea was to follow after that.
So one thing that I remember is that some students came to us and asked questions, like, yeah, I tried to implement this bit hack here, and it actually slowed the program down.
Or I changed my pentomino board representation to this also.
But it didn't seem to speed anything up.
Well, in those cases, yeah, you need to do more detective work to figure out what happened
So a lot of times, the performance is something like a max of a bunch of different things.
So you have done something, that's great.
It should give you a huge performance improvement.
But something is holding.
And a lot of times, you have to do three or four things.
And once you do three or four things, suddenly everything starts kicking off.
A good example is loop unroll.
You unroll a loop and say, ha, I should get all this parallelism.
No, because you have some dependence.
So you have to fix that dependence, fix that dependence.
Suddenly, you remove everything.
Then you get all the parallelism and start running fast.
So a lot of these things, it's basically max.
You have to get all the things right.
And so if you stop halfway through, that means you might be almost there, and you are not up to that.
So that's why it just can be a really frustrating process.
But it's kind of fun, too, when you get there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to take a little bit closer look at what's happening under the covers when you compile a C program.
But before we get into that, we did a little interesting correlation on your scores for the first problem, for the every bit one.
So this is basically plotting.
It's a scatter plot of how you did in your test coverage score versus how you did in your correctness score and performance, correctness and performance together.
And what's interesting is that if you did better in your test coverage, you did better in your performance and correctness.
OK, that's a pretty good correlation, right?
There's some outliers here.
But that's a pretty good correlation.
Yeah, John?
Yeah?
The--
Do we have a handheld?
Here we go.
Just a second.
The only thing is we have to figure out how to turn it on.
There we go
Yeah, so just to clarify, the people who seemingly got no test coverage but really good performance scores, what actually happened was that we tested for things that we didn't expect students to cover for like on feeding invalid values to better a set and better a get or testing for private functions to their implementations.
So in reality they had better test suites than the coverage score would indicate
So what are the lessons that one draws from this?
So professional engineers know what the lessons are.
So the lessons are that it is actually better, if you have a coding problem to do, to write tests first.
Before you code you write your tests.
And that actually speeds the development of fast correct code.
It's actually faster.
You get to the end result much faster.
Because whenever you make an error in your program, you instantly know that you may have a problem rather than thinking that you're doing something OK and then discovering that, oh, in fact your code is, in fact, incorrect, and you're working away optimizing something that's not working So before coding, it's highly recommended that you write test.
Also if you find a bug, when you find a bug, the first thing you should do is write a test for that bug if it wasn't already covered.
Then you fix the bug.
And then you make sure that your test now, that your new implementation, passes that particular one.
Professional engineers know this.
Professional software developers know this.
It comes hard.
And if you want a job at any top flight software firm, they're going to expect that you know that you write tests first before you do coding.
The second lesson isn't quite so obvious from this.
But it's the second lesson that I think some people experienced in the class which was the idea of putting you in groups, in particular in pairs, was not so that you could do divide and conquer on the code.
It was to do pair programming.
And what we found was that a bunch of groups divided up the work.
And they said, OK, it'll go faster if you do this one and I do that one.
Once again that's probably a mistake.
If you can sit together and take turns at the keyboard making the changes, it may seem like it's going slower to begin with, but it's amazing how many errors you catch and how quickly you find your errors because you're just talking with each other.
And it's like, oh, duh.
So good programmers know this.
That it really helps to have more than one person understand what's going on in the code.
So the people who had difficulty with their partners one way or another often did not, it was partly because they just divided up the work, you're responsible for that, oh, we got a bad grade on that, that's your fault.
No, both partners own that grade 100%.
And the best way to ensure is to work together.
Now, this sometimes flies in the face of people who believe that they are clever or more experienced than somebody, than their partner, oh, I can do this much better on my own.
Usually, that's true for little projects, but as the projects get bigger that becomes a much harder situation to deal with.
It becomes the case that you really want two brains looking at the same thing, four eyes as opposed to two eyes looking at things.
But I think, in particular, before coding write test.
And we are right now working on improving the infrastructure.
One of the things that they have in most companies is, at the very minimum, they have what's called a nightly build.
Nightly build says they take all the software, they build it, and then they run regression tests against it all night while everybody's home sleeping.
Come in the next morning, here's the things that broke.
And if you broke the build, you got some work to do that morning.
And it's generally not a good idea to break the build.
What has been demonstrating, in fact, is that continuous build is even better.
This is where, whenever you make a change to the program, you run the full suite of tests on it.
And we're going to look into it.
We have to see what our resources are.
As you know, our TAs are a limited resource.
But we're going to look into seeing whether we can provide more of that kind of infrastructure on some of the later projects for you folks.
So you can sort of see the matrix that we eventually got to you.
You can see that develop in real time.
How am I doing against other people's tests?
How are they doing against my tests, et cetera?
So we'll see whether we can do that.
But what's funny is you think that it'd be faster to just code and do it.
Computer science is full of wonderful paradoxes.
And one of them is that doing things like writing the extra code to test is actually faster than not writing it, surprisingly.
It really gets you to the end result a lot faster.
Any questions about that?
Any comments about that?
Let's talk about our today.
So today we're going to talk mostly about single threaded performance.
This is one instruction stream that you're trying to make go fast.
But if you look at today's computing milieu, how all of the computers are used, what do you have?
You've got networks of multi-core clusters.
It's parallelism everywhere.
You've got shared memory among processors within a chip.
You've got message passing among machines in a cluster.
You've got network protocols among clusters so that you can do wide area things.
Yet we're saying, no, let's take a look at what happens on one core on one machine.
So why is that important to focus first on what one core can do?
Why study single threaded performance at all?
Let's just go do the parallel stuff.
That's more fun anyway.
Well, there are a couple of reasons that I can think of.
The first one is at the end of the day, even if you've got something running widely in parallel, the code is running in each core in a single threaded manner.
You just have a bunch of them.
And so if you've given up a factor of two or a factor of four in performance or even more, as you're aware, you can sometimes make it orders of magnitude, but in performance what you're saying is that you're going to end up using much more resources to do your particular job in parallel.
And resources is money.
So if I can do the job with a cluster of 16 processors and somebody else can do it in a cluster with only four processors, hey, they just spent a quarter the amount on not just the capital investment in that hardware but also the operating costs of what it cost to actually cool and provide electricity to and maintain and so forth.
All that gets much cheaper.
So if you get good single thread performance, it translates.
That's kind of the direct reason.
The indirect reason is, for studying it, is that many of the lessons will generalize.
So things that we'll see for single core, there is an analogy when you start looking at parallel and distributed systems.
So that's a little less concrete.
But as you'll see as you gain experience, you'll see that there's a lot of lessons that generalize to how do you think about performance no matter what the context.
So what about a single threaded machine?
What's it like?
So some of this is going to be a little bit review, but we're going to just sort of go deeper.
We've sort of been taking layers off the onion.
And today we're going to take a few more layers off the onion.
So you have inside a processor core.
You've got registers.
You've got the functional units to do your ALU operations, floating point units, vector units these days, and all the stuff to do instruction, execution, and coordination, scheduling, out of order execution, and so forth.
In addition then, you have a memory hierarchy.
Within the core, you typically have registers and L1 and L2 caches.
And then outside the core often is the L3 cache.
DRAM memory, you may have a solid-state drive these days and disk.
And so in that context, you're trying to make your code run fast.
So when you compile the piece of code, so here I have a piece of code.
I'm always amused when I put up a Fibonacci as the example.
Because this is a really terrible way to compute Fibonacci numbers.
So this is an exponential time algorithm for computing Fibonacci numbers.
And you may be aware you can do this in linear time just by adding up from the bottom.
In fact, if you take the algorithms course, you learn that you can actually do this in logarithmic time by matrix, recursive squaring of matrices.
So it's sort of interesting to put up something where we say we're going to optimize this.
And, of course, we'll get a constant factor improvement on something like this.
But, in fact, really this is a terrible program to write for optimization.
But it's good didactically.
And Fibonacci numbers are fun anyway.
So typically what happens is when you run GCC on your .C file and produce a binary, what happens is it produces the machine code, which is basically a string of bytes, zeros and ones.
And that goes, when you run the program, that goes through the hardware interpreter.
So the hardware of the machine is doing an interpretation of these very simple instructions and produces an execution.
But, in fact, there's actually four stages that go on inside of GCC if you type a command like this.
The first thing is what's called preprocessing.
And what that does is it does any macro expansion and so forth, things that are just basically on the level of textual substitutions before you get into the guts of the compiler.
Then you actually do the compiler.
And that produces a version of machine code called assembly language, which we'll see in just a minute.
And from that version of assembly language it then goes into a process called linking and loading, which actually causes it to produce the binary that you can then execute.
So all four stages are included here.
And there are switches to GCC that let you do only one or all of these things.
You can, for example, run the preprocessor GCC.
You can tell it to run the preprocessor alone and see what all your macros expanded to.
Yes?
Question?
What's the difference between compiling and assembling?
So compiling reduces it to essentially assembly language.
And then assembling is taking that assembly language and producing the machine binary
I was going to say, there's a one-to-one correspondence between machine code and assembly, but there's not a one-to-one correspondence between C code and assembly
Yeah.
So there's actually not quite a one to one, but it's very close.
It's very close.
So you can think of it as one to one between assembly machine code.
But assembly is, in some sense, a more human readable and understandable version of machine code.
In fact, that's what we're going to talk about.
So let's go directly to assembly code.
To do that I can use the minus S switch.
Now it turns out it's also helpful to use the minus G switch.
Minus G says give me all the debugger symbol tables.
And what that makes it is so that you can actually read the assembly language.
If you don't have that information, then you don't know what the programmer wrote as the symbols.
Instead you will get computer generated symbol names for things.
And you don't have any meaning to those.
So it's really a good idea to use minus G and minus S together.
And this basically provides a convenient symbolic representation of the machine language.
And this is sort of the type of thing that you'll get, something coming out that looks like this.
It's basically an Ascii.
It's in text, characters, rather than being in the binary executable.
And if you want, you can find out all the vagaries of it.
This is one site that has some reasonable documentation on the GNU assembler.
It's actually not as good on the instructions, but it's really good on all the directives, which we'll talk about in a minute like .global and .type and all that stuff.
It's very good on that stuff.
There's another thing that you can do.
And once again, it's also helpful if you've produced a binary that has the symbol table.
And that is to do a dump of the object code.
And when you do a dump of the object code, what it does is you basically give it an executable and it goes backwards the other way, take this executable and undo one step, disassemble it.
And what's good about object dump is that it gives you, first of all, these are all the byte codes of the instructions.
Also if you've got the minus S says interleave the source code, so you can see, here's the source code interleaved.
So you can see which regions of code depend on which things.
And so it basically tells you where in memory it's being loaded, it's been loaded, what the instructions are.
And then it gives you the assembly interpretation of that machine binary.
And this is where you can see it's almost one to one what's going on here.
Here we have a push of an operand.
And that notice is just a one byte code.
Whereas here we've got an opcode and two arguments.
And it has three bytes as it turns out.
So you can see there's sort of a correspondence.
Yeah, question?
How does [?
logic then take the ?]
machine language code and go to-- how does it know the function names and stuff?
It knows the function names because when you compile it with -g, it produces, in addition to producing the binary, it produces a separate segment that's not loaded in that has all that information that says, oh, at this location is where this symbol is.
And it produces all that as stuff that's never loaded in at run time but which is there in order to aid debuggers mainly.
Question?
To compile something not using the gflag and then you do an object dump, how would that work?
Then what happens is, first of all, you would not be able to get this stuff interleaved.
And then things like here where it says fib, well, fib may be an external name so you might know it anyway.
But if it were an internal name, you would not be able to see what it was.
Yeah, if you're going to respond let's get you on mike here.
Why don't you just hold this?
Yeah, so you'll generally get the function names so you know roughly a huge blob of assembly corresponds to a function.
But you won't be able to get any information about what variables are in which registers or what position the sixth line of assembly corresponds to in terms of your source code
Then the other thing that you can do is you can actually take the assembler, the assembly code, if you produce just the assembly code, and if you tell GCC to take a .s file, which is the assembly code, you can produce the machine code from it.
And so one thing that you can do is you can produce a .s file and then edit it in Emacs or VI or whatever your favorite text editor is and then assemble it with GCC.
So you can actually modify what the machine code is directly.
And that's what we're going to spend a little bit of time doing today.
Let's go in and see what the compiler generates and then let's twiddle it a bit.
So here's what we're going to expect that you do, that you're able to do.
We expect in this class that you're going to be able to understand how a compiler implements the C linguistic constructs using x86 instructions.
We're going to expect that you can read x86 assembly language with the aid of a manual.
We don't expect that you know all the instructions, but the basic ones we expect that you know what those are.
We expect that you're going to be able to make simple modifications to the assembly language generated by a compiler, and that you would know, if push came to shove, how to write your own machine code on your own.
That's not something we're going to expect that you do, but you would know how to get started to do that if at some point you said, oh, I really have to write this in assembler.
So this is, as I say, really we're going to take off some layers of the onion today, try to get down what's going on.
It turns out this is actually kind of fun.
Now, the part that's not fun at some level is the x86 64 machine model.
The x86 is what's called a complex instruction set computer.
And these, long ago, were demonstrated to be inferior to so-called reduced instruction set computers.
But that hasn't mattered in the marketplace.
What's mattered in the marketplace is who could build better and faster chips.
And also the amount of people who started using the x86 instruction set has produced a huge legacy and inertia.
It's sort of like some people might argue that Esperanto is a better language for everybody to learn than English.
But how come English with all its complexities and so forth, and I'm sure for some of you who have learned English as a second language, it's like it's a crazy language.
Who do you learn English?
Well, it's because that's what everybody's learning.
That's where the legacy is.
And so x86 is very much like the English of machines these days.
So in this model there's basically a flat 64-bit address space.
There are 16 64-bit general purpose registers, and then what are some segment registers, a register full of flags, an instruction pointer register, rest in peace.
They're eight 80-bit floating point data registers, some control status registers, an opcode register, a floating point instruction pointer register, and a floating point data pointing register, some MMX registers for the multimedia extensions, and a 128-bit XMM registers for the SSE instructions, which are the ability to have an opcode run over several pieces of data at once, short vectors, vector instructions, and a 32-bit register that frankly I don't have a clue as to what it does.
So, fortunately, we don't have to know all these.
You can look at the architecture manual if any of these become important.
So our goal is not to memorize the x86 instruction set.
That would be a punishment probably worse than death.
The only thing worse would be learning all of C++.
So here's the general registers.
So there are basically 64-bit registers.
And here's the mnemonics that they have.
So you can see is all very mnemonic, right?
We got some of them that are numbered.
How come they're all just not numbered?
I mean come on, right?
I know why.
I know why.
Don't tell me.
So what you get to do is look at and remember that there are all these fun registers.
And what they did is the x86 64 architecture grew out of the x86 which was 32-bit.
Well, in fact, originally it was 16-bit.
And it's been extended twice to have more bits in the instruction word so that now it's a 64-bit instruction word.
And what they did in order to make it so that they could run legacy code more easily, which might have been written with a smaller word size, is they've overlap so that the EAX register, for example, is the low order 32-bits of the RAX register.
So what you do is you'll see that R is the prefix that says, hey, that's a 64-bit register.
E is the prefix that says that it is-- Whoops, I made a mistake there.
Those should all be Es.
Oh, no, sorry, no, there's are correct.
These are R and then with D because these are the extended ones, yes.
So these are D. So R and D, that means also that it's 16.
So you can see just how easy this is to remember without a cheat sheet, right?
And then you go down to 15, et cetera.
And so you can go all the way down to byte naming, the low order byte of the registers.
In addition, it turns out that that's not all.
But the high order byte of the 16-bit registers are also available as independently named registers.
When you're using this in a C program, there's a convention that C has.
And it's actually different on Windows from on Linux.
Because there's no reason they should make those things compatible.
That would be too easy.
So instead they have different ones.
But the ones on Linux, this is essentially the structure.
What happens when you call a subroutine is generally you're passing the arguments to the subroutine in registers.
And in fact the first six arguments are passed in these registers.
RDI, you'll get very familiar with RDI.
Because that's where the first argument is always passed.
And almost all your functions will have a first argument, except for the ones that have no arguments, and then the second arguments, the third, and so forth, and then fifth and sixth.
If you get more than six, then it turns out, then you start passing arguments through memory.
But otherwise the convention is that the arguments are passed through registers.
There are a couple of other important registers.
One here is the return value always comes back in RAX.
So when a function returns, boom, that's where, RAX is where the value of the return is.
There is a base pointer and a stack pointer which give you the stack frame so that when you do a push and want to push local variables those are telling you the limits of your local variats.
And we'll talk more about that a little bit.
And then there are a variety of other ones.
Some are callee saved and some are caller saved.
And you can refer to this chart.
And there are others similar to it in the various manuals.
Now, it gets pretty confusing, if this isn't confusing enough for the naming.
Let's go on to how you name data types.
And I think some of you have already experienced this a little bit, the beauties of the data types.
So in C, they have all these different data types such as I'm listing here.
And if you want to generate a constant of that size, so sometimes the compiler will coerce a value from one type to another.
But sometimes it won't.
And so if you want to have a constant, and I've just given a couple things here, for example, if you want it to be just an int, you can just write the number.
But if you want it to be unsigned, you have to put a U after it.
Or if you want it to be a long, you have to put an L after it.
And for many things it'll get coerced automatically to the right type because if you do an operator with another argument it will be coerced to that type.
But some of you got burned on some of the shift things to begin with because it wasn't clear what exactly the sizes.
Well, you can be explicit in C and name them using this particular convention.
This tells you how many bytes are being allocated for that type in the x86 64 size.
So it's [?
veted ?]
here for four.
Now, long double is a funny one.
It's actually allocate 16 bytes, but only 10 of them are used.
So basically there are six bytes that get unused by that.
And I think that's for future expansion so that they can have even wider extension.
This is generally used, of course, for floating point and so forth.
Now, in the assembly language, each of the operators has a suffix.
And sometimes, if it's a two operand instruction, it may, where it's taking things of different sizes, it may have two suffixes.
But it has a suffix which is a single character that tells you what the size is that you're working with.
So, for example, B is for byte.
W is for word because originally the words were 16 bits.
L is for long except that it's not a long so don't get confused.
L is not long.
Long is a quad word, or Q, four bytes.
And then a float is an S. A double is a D. And a long double is a T. So these you will get familiar with.
And they're not so hard.
But that doesn't mean you know them right off the bat.
And it helps to have a cheat sheet.
As I say, the main one not to get confused about is the Ls.
L means something different in x86 than it means in C. So, for example, here we have a move of, and because it's a Q, I know that it is an eight byte or a 64-bit operator.
And you can tell that also because it's using RBP and RAX, both of which are 64-bit registers.
In fact, in assembly, you can actually write it without the Q, because the assembler can infer when the Q isn't there that, oh, this is a 64-bit register, that's a 64-bit register, I bet he means move 64-bits.
So it actually fills that in sometimes.
But sometimes you need to be explicit.
Question?
What happens when you actually put 64-bit registers but you only, and you just put move [?
b ?]
or something?
Would it complain?
Would it [UNINTELLIGIBLE]?
Yeah, it would complain.
Yeah, it would complain.
it'll say it's an improperly formed instruction, so, yeah.
And the other thing you can do, of course, is just try it out.
What happens if?
That's the great thing about computers.
It's easy to do what happened if.
Now, the instruction format is typically an opcode followed by an operand list.
So the opcode is a short mnemonic identifying the type of instruction that includes typically the single character suffix indicating the data type.
However, for some instructions it turns out you can have two suffixes if the two-- Most instructions operate on data types of the same size.
But some of them operate on two different sizes in which case you'll have two suffixes.
If the suffix is missing, it can generally be inferred, as I mentioned.
Then the operand list is from zero, two, and very rarely three operands separated by commas.
Now, in the architecture manual, in fact, they say if you look at it, they'll show you fourth operand.
And I said, four operands?
This documentation says there's only three.
This one says there's four.
I went through the whole architecture manual last night.
Every time it says four operands, it says N/A, not applicable.
So I think it's just there reserved or something.
But anyway there is no fourth operand as far as I can tell.
Now, one of the operands is the destination.
And here's where we start to get into some differences.
There's actually two standard formats for assembly language that are generally called Intel and AT&T.
So AT&T was the original Unix system.
And Intel is what Intel uses for their assembler.
They do the destination operand in the opposite order.
So AT&T, it puts the destination last.
In Intel it puts the destination first.
So when you're reading documentation, you can read the Intel documentation.
You just have to remember to flip it around if you're actually writing it as we will be using the AT&T format.
Almost everybody uses AT&T as far as I can tell except Intel.
So Intel's assembler does it the other way around.
And actually now GCC will actually, you can give it a directive to say I'm now switching to writing it in Intel assembler.
So you can actually go back and forth between the two if you happen to borrow some assembly language code from somebody else.
So one of them is the destination.
The other operations are read-only, so const in the C++ terminology.
They're read-only.
So it's always the case that only one of them is going to be modified.
And that's the one that's the destination of the operation.
In addition in assembler, there are what are called directives.
Besides the instructions, there are directives.
So first of all there are things like labels.
You can take any instruction and put an identifier and a colon, and that becomes then a way of naming that place in your code.
So, for example, jump instructions want to know to where they're jumping.
And rather than having to know upfront what the address is, the assembler will calculate what that address is and everywhere you put x, it'll put in the right value.
And you get to name it symbolically rather than as an absolute machine location.
There are storage directives.
So, for example, .space 20 says allocate 20 bytes at location x.
.long says store the constant 172 at y.
It's being stored at y because I said y is here.
And asciz gives you a string that's zero terminated.
So the standard for strings is zero terminated.
You can also, there's one that says give me a nonterminated string.
So you can have fun with that if you like that.
The align directive says make sure that as you're going through, so what's happening is the assembler is going through there, is it's laying these things out in memory typically sequentially, the way you wrote it down in the program, in the assembly language program.
If you say align eight, it says advance whatever the pointer is of where the next thing is going to be put to be a multiple of eight.
And that way you don't run the risk of where you declare a character and then you say, OK, and now I want a long or something, and it's not aligned in a way that the eight bytes correspond to a multiple of eight the way you need to in order for the instructions to properly work on them.
So generally, although, we have byte pointers, most instructions only work on aligned values.
And for some of them that work on unaligned values, they're generally slower than the ones that work on aligned values.
There are also segment directives.
So in memory when you run your program, the executing program starts with the program text down at the bottom of memory.
And then it has fixed data that's not going to change, static allocation of data.
And then it's got heap, which is dynamically allocated data.
And then it's got stack.
The stack grows downward, and the heap grows upward.
By saying something like text, it says make sure that the next stuff I'm putting goes into the text segment.
So that's generally where you put your code.
Saying it's in data says make sure it goes in here.
So you may want to have a table, for example.
So, for example, from pentominos you might have a table there.
There's going to be a fixed table.
You're never going to change it during the running of the program.
Put it in the data segment.
And then there's also things like scope and linkage directives.
So saying .global, and you can either spell it incorrectly, as I have here, or with the a, it's the same thing for the GNU assembler anyway.
It says make the symbol fib externally visible.
And that makes sure that it goes into the symbol table so that debuggers and things can look at it.
And there's a lot more of these in the assembler manual, that link that I showed you to before, tells you what all the directives mean.
So that when you're looking at code, which mostly you'll be reading it, making a few changes to it, you can know what things mean.
So the opcode examples, here's some examples.
There are things like mov, push, and pop.
So, for example, here movslq, this is an interesting one because it's moving.
The s says extend the sign because I'm moving from a long from a 32-bit word to a 64-bit word, from 4-bits to 8-bits.
So that's why this one takes two suffixes moving from, you'll notice, a 32-bit register to a 64-bit register.
So you have to be careful.
This is something I got caught up in the other day.
The results of 32-bit operations are implicitly extended to 64-bit values.
So if you store something into EAX, for example, it automatically zeroes out the high order 32-bits of RAX.
Because that's the one that it's embedded in.
However, that's not true for the eight and 16-bit operations.
If you store into an 8-bit field, an 8-bit part of the register, it does not zero out the high order bits of the remainder.
So you just have to be careful when you're doing that.
Most of these are things, by the way, that is more cryptic when you're looking at stuff.
It's like, oh, how come it's, gee, I thought I had, I'm returning a double word, but it looks here like it's returning a 32-bit word, how come I thought I was returning 64-- Well, the answer is because it knows the high order bits are zero.
So it's using the shorter instructions.
And yet it still is having the impact on the 64-bit register.
They're all the arithmetic and logical operations.
So subtracting, once again, the destination is second.
So typically these are two operator things.
So you always have the destination occurs both at the beginning and on the left hand side and the right hand side, things like shifts and rotates, control transfer, so call which does a subroutine jump, return from a subroutine, a jump instruction that just says make the next instruction the thing that you're pointing to, and very important, the jump conditionals where the condition is a whole bunch of keys that are things like greater than, less than, and so forth, and different ones for signed and unsigned and so forth.
So typically the condition is computed by using a compare instruction.
I probably should have put CNP on here as well, but I didn't.
But the CNP instruction is usually what you use to compare two things and then you separately jump on what the condition is.
There's a pretty nice website that has most of these opcodes.
However, they only deal with the old x86 without the 64-bit extension.
And they use the Intel syntax.
But it's really convenient.
Because they've done a nice job of making a quick jump table where you can just go, look up the opcode, and pop it up.
Otherwise you can just look at them in the manual.
Anyway, that's kind of a convenient place.
And, as I say, just beware because it's 32-bit only, and it's Intel syntax.
Most of the instructions got extended.
I mean it's like, OK, if you do it for eight and 16 and 32, the operation is not going to change that much to go to 64.
A few of them do, however.
Now, the operands, Intel supports, the x86, which is Intel and AMD, typically, support all kinds of addressing modes.
The rule is that only one operand, however, can address memory.
So you have to pick which is the operand that's going to address memory if you have multiple operands.
You can't have both operands.
So you can't add two things in memory.
You always have to take something for memory and bring it into a register and then store it back out.
So the simplest one is two register instructions.
Here I've basically marked the-- What have I marked here?
I guess I marked-- I don't know.
Down here I was marking memory.
I'm not sure what I was marking up here.
Because they're both registers.
But in any case, this is just adding RBX into RAX.
And so it takes the contents of RBX adds it in the contents of RAX.
There's something that's called direct.
So this is, it says, where you move, x is some constant value, and you move it, the contents of it, into RDI.
So if x, for example, is a location that you've stored a value in, you can say move whatever is the value at that location into RDI.
Immediate says, which usually is preceded by a dollar sign, says move the address of it, move that as a constant.
So x has a value, move that value.
So if you say $3, then you'll move the constant three into RDI.
If you said mov3 this, you're going to move the contents of location three in memory.
So that's the difference between direct and immediate.
So the dollar sign says you're taking that as a literal constant.
And the direct says you're actually going to memory and fetching it.
Then things start getting interesting.
Register indirect says, in this case, the thing that you're going to access is the thing pointed to by that register.
So don't move, in this case, RBX into RAX, move it to the memory location that RAX is pointing to.
Then you can do register index which says, well, it's pointing to it, but I want displaced 172-bytes off of that location, of whatever this is pointing to.
So, for example, if you have a pointer to a record, you can then have just a single pointer and address all the fields just by doing register indirect to the different fields using that same register.
Then there is, it actually-- I skipped, actually, a few in here that are subsets of this.
This is, I think, the most complicated one that I know.
It's base index scale displacement where base and index are registers, the scale is two, four, eight, and if it's not there, it implies one.
The displacement is eight, 16, or a 32-bit value.
And it says take RD-- oh, I had put the math on here, and then I guess I lost it-- it says take RDX, multiply it by eight, add RDI, and add 172.
[WHISTLE].
So, anyway, you can look in the manual.
So you'll see some of these instructions being generated.
Generally, you're not going to generate these instructions.
So when you see them generated, you can see it.
And then, and this is actually new, it's not in the x86.
It has this instruction pointer where you can actually access where the current program counter is pointing, where it is in the code, and store that value, in this case indexed by six, into RAX.
So you can do it relative where this has to be a 32-bit constant.
And what's good about that is it allows you then to write code where you can do things like jump to something that's relative to the program counter.
And that lets you put the code anywhere in memory, and it still has the same behavior.
Because you're going relative to where that code is rather than to an absolute location.
So it allows the code to be relocatable.
So here's-- Yeah, questions, yeah sure
Why was it the index registers, when you have the numbering for the register, what does that mean again?
The number before the register?
In the instruction [UNINTELLIGIBLE], that's 60 for
OK, or whatever, whenever it's here, it's basically saying, add that value to the contents of RAX.
And so the same thing here, add six to the contents of the instruction.
So this is six bytes ahead of me in the instruction stream.
OK?
So you can actually say, well, what's that instruction ahead of me in the instructions stream?
OK?
So here's some examples of essentially the same code and how it gets compiled.
So here we're going to have a fou1, fou2, fou3.
And in this case we declare x, y, and z to be unsigned integers.
We set them to some values.
And we just simply say return x plus y or z, bitwise OR with z.
If you look at what the code is that's generated, it says move the constant 45 into EAX.
Why does it do that?
Well, let's just see.
Well, the compiler figures out that it knows what 35, seven, and 45 are.
It computes x plus y.
That's 41.
If you take 41 bitwise OR with 45, it turns out it's masking the same bits, that's 45.
So the compiler actually can figure this out that all it has to do is return 45 in a 64-bit register.
Ah, but here it's returning it in a 32-bit register.
What happened?
It's not obeying the type.
The type is supposed to be 64-bits, but that's a 32-bit register.
Oh, yeah, that's this thing where it automatically zeroes out the high order bits.
And it uses this instruction, because this is a shorter instruction than if it did the RAX.
It could do the same thing with RAX, but it would be more bytes of instruction.
So they saved a couple bytes of instruction by doing that.
So people follow what happened there?
Let's take a look at the next one.
Here it's the same code just let's pass those things as arguments.
Well, if you remember the calling convention, parameter one is in RDI, parameter two is in RSI, and parameter three is in RDX.
So I don't expect you to remember that off the top your head.
But we have the cheat sheet, and you can figure that out.
So here's what it does is, oh my goodness, what is that instruction?
This is actually a computation of effective address.
So the effective address is basically saying, and it's using one of these funny indexing modes.
So what this is actually doing is it's actually adding these two numbers together, the values stored in those locations together, and storing it into RAX.
And then it's then OR-ing RDX, what's in RDX, with RAX and then returning.
Remember that RAX is where the result is always going to be.
So the result is always returned in RAX.
So you can see you have to do a little bit more complicated addressing in order to pull them out as parameters than if it could actually figure out what the numbers are.
Last example here is I declared these things before I ever got their globals.
And so I declared them before I ever got in.
So that means since they're globals, they have a fixed place in memory.
And so the code that's generated is moving, it turns out it allocates them right nearby the instructions here.
And so what it does is it has actually a relative offset for x, relativity instruction pointer, put that in RAX, add the offset of x into it, and then OR it with the offset of z, and then return.
And so there the constants are actually stored right nearby in the code so that they can use this relative offset.
And the compiler figures out, or the assembler figures out, exactly what the offset is that it actually needs to substitute for y so that it can be a relative offset from the current instruction pointer.
Notice that, for example, that's going to change depending upon the value of y here.
It's going to change compared to if I accessed y down here.
It would be a different instruction pointer at this point.
So it actually just goes but it computes what the difference is so it knows what the distance is.
It can compute that at compile time, and then at execution time it just uses whatever constant goes in there.
So the important thing here is just to notice that the code depends upon where x, y, and z are allocated.
So the first thing to actually look at good code is to understand the calling convention that's used by the compiler.
And here are the basics of it.
So the register RSP points to the function call stack in memory.
And the call stack grows downward in memory, like in that little map I showed you before, so that as you push things onto the stack they're getting lower numbered not higher numbered.
The call instruction pushes the current instruction pointer onto the stack, jumps to the call target operand, which is basically the address of the thing you're calling.
So when you do a call, it saves your return address on the stack.
The return instruction pops the return address off the stack and returns to the caller.
It basically says, oh, I know where the return address is.
I slam that into the current instruction pointer, and that becomes the next instruction that's executed.
Now, there are some software conventions that are used that's helpful to know.
Besides those instruction registers, some of the registers are expected to be saved by the caller, some are expected to be saved by the callee.
You're free to violate this in your own little piece of code as long as if you're calling something else, you're obeying it.
So you don't have obey this convention in the code you write unless you want to interoperate with other stuff.
So if you, for example, have a leaf procedure, you can decide for that leaf procedure, oh, I'm going to make something callee saved that was caller saved or whatever as long as by the time you return you've cleaned everything up for the rest of the world.
So these are conventions.
But for the most part, you're not going to violate these, and the code that the compiler generates doesn't violate these because it expects everything to interoperate.
So here's how the subroutine linkage works.
We're going to do an example here where function A calls function B which will call function C. And right now, we're at the point we're executing B.
And so on the stack are the arguments that were passed from A to B that did not fit within the registers.
So normally most of the arguments are within registers.
But if you exceed the six registers then, because you have a long argument list, then it gets passed on the stack.
And here's where it gets passed.
The next thing is B's return address.
This is the thing that got smashed in there when you did the call.
It got pushed onto the stack.
And then there's what's called a base pointer for A.
And this is the way that A ends up accessing its local variables.
And then there's a separate region here where it's going to put arguments from B to B's callees if they exceed the six registers, if any of the things that B is calling require more than the six registers.
So let's just take a look.
So function B can access its nonregister values by indexing off of RBP.
So these we say, these are in a linkage block.
And the reason is because it's actually part of A's frame as well.
It's a shared part of the frame where A stores it into memory and then B's going to fetch it out of memory.
And that's the linkage block.
So this is positive in memory.
So if I use a positive offset, I then go up to getting the arguments from A.
Then it can access its local variables from the base point with a negative offset because we're growing down in memory.
Now, if it wants to call C, what it does is it places the nonregister arguments into the reserved linkage block here, which are arguments from B to B's callees.
And that once again acts just as if they're local variables.
It's positive index off of RBP, sorry, negative offset off of RBP.
So it pushes those things into the argument, into that region, if it needs to use that region.
Then we actually, once it's done that, we have the call.
So B calls C which saves the return address for B on the stack, so it saves it on the stack, and then transfers control to C. So now it starts executing C's code.
And what does C do?
So C is going to have to advance these pointers to refer to its region rather than B's.
It does it by saving B's base pointer on the stack.
So it saves this pointer here so that it can restore it when it returns.
It advances, it sets its new base pointer to be where the stack pointer is now and then advances the stack pointer to allocate space for C's local variables and linkage blocks.
Watch, here we go.
So that ends up being C's frame.
So notice that B's frame and C's frame are overlapping in the linkage block between them.
Now, if a function never performs stack allocations except during function calls, there's a great compile time optimization that the compiler will often do.
And what it will do is realize that this distance is constant.
So, therefore, it doesn't need RBP.
It can just do the math and index everything off of RSP as long as RSP is always the same, for example for C, when C is executing.
There's certain C commands like [?
alaka ?]
which changed the stack pointer.
If you use those, the compiler can't do that optimization.
But if the storage on the stack never changes for a given frame, then it's free to make this optimization.
So you'll see code where RBP has been optimized away.
How about some questions before we go on and do an example.
Yeah?
[INAUDIBLE] should there be A's return address, where you try to [INAUDIBLE]?
Up?
Oh, this should be, sorry, this should be A's return address.
Yes, you're right.
OK, good, typo.
Is somebody catching my typos to-- OK, yep, a good one, that should be A's return address.
Sorry about that.
This is B's return address.
Any other questions?
That's good.
That means you understand something.
Hooray.
So let's do an example.
So here's my fib example.
And I compiled this with minus oh zero.
Because when I compiled it with minus oh three, I couldn't understand what was going on.
So I compiled this with minus oh zero, which gives me really unoptimized code.
And that lets me be the compiler optimizer.
So here's the code that it generates.
So we can take a look at a few things here.
First of all is declaring fib to be a global.
And it's got some other things here.
I actually took out some of the directives that were in here that were irrelevant for our purposes.
If you actually compile it, there's a lot more directives that are stuck in there and a lot more labels and things that you don't need to understand it.
There are two labels here.
And so you can see here basically what's going on is we're first of all doing the advancing of the base pointer and advancing the stack pointer here.
That's doing that operation that I showed you, those of moving the base and stack pointer up.
And then at the end here this is equivalent to doing, a leave instruction is equivalent to undoing that.
So Intel lets you do one leave instruction rather than making you put these instructions in every time.
It's exactly the same thing.
But in any case, let's just sort of see what's going on here.
So we're pushing some storage.
This is saving a register here.
We're then advancing the stack pointer to store 24 bytes of temporary storage.
And then we're start to do some computations here.
This looks like we're comparing one with something and then doing a ja.
So this is a jump above.
This is the unsigned version.
What you're looking is to see, here we say if n is less than two, in fact, what it's doing is saying if n is greater than one go to L4.
So it's actually doing the other one.
So you can see then L4 is, what happens is the one that has the two calls to fib, recursive calls, so that's this part of the code, and it's doing that if it's greater than one.
And otherwise it's going to execute these instructions, which are basically returning n. And so it basically does some computations.
And then both of them converge here where it moves the results and then pops it off and so forth.
So that's sort of the outline of what's going on there.
So let's dive in here a little bit and sort of see what's going on, see if we can read this a little bit more closely and whether we can optimize it.
So the first thing that I noticed in looking at this is look at all this memory addressing that we're doing.
What do you suppose this thing is, minus 16% RBP?
So this is the base pointer.
So this is a local variable because it's a negative offset off of the base pointer.
What do you think it's doing there?
What's stored in here?
Yeah, this is where n is being stored.
Because what are we doing?
We're trying to compare n with one here even though it says two up there.
We're comparing it with one here.
And so I look at that, and I say, look, I'm comparing it with one, then I'm jumping to L4, then I jump to L4 or not.
And then let's say I don't.
Well, then the first thing I do is I move n into RAX.
But wait a minute, I just compared it with that.
So I'm accessing n again.
I'm accessing it a third time.
How about if I try to store that stuff in a register instead?
So what I did is I picked the RDI register, because that one happens to be available, and I said do they, if you look here, what did we do?
We stored RDI, which is the first argument, into memory.
And then we compared with it in memory.
Why don't we compare with it in RDI?
Right?
Duh, stupid compiler, well, because I had minus oh zero.
OK, so I can do that improvement.
So what I did was I edited it to put RDI there and RDI here and RDI here.
And I went up and I said, what about RDI here?
Why didn't I replace that one?
No, there's no loop going on here.
It's recursion
[INAUDIBLE PHRASE]
Yeah, the problem is that when I call fib, RDI gets garbaged on me.
Because RDI is going to be the first argument to-- See it's being garbaged here?
It's garbage as far as my use of it for n. It's being used to pass n minus 1 as the argument to the recursive call.
So I can't replace this one after fib because RDI no longer has it.
Because I had to leave it.
But even so I went from 5.45 seconds for the original code to 4.09 seconds when I compile that just with that little change.
I felt pretty good.
I felt pretty good.
So then I wanted more.
That was fun.
I wanted more.
So what was the next thing I noticed?
I noticed that-- And by the way almost all the things, that stuff I did last night.
This is what I did an hour before class so we'll see whether it-- So then I noticed that, look, we're moving this stuff here.
We keep using minus 24.
And once again, memory operations are expensive compared to register operations.
Let me try to get rid them.
What do you suppose is in here?
So look, we move RAX into the local variable minus 24.
And then we jump to L5.
And we move minus 24 into RAX.
That seems kind of unnecessary.
Here we move RBX into minus 24.
Then we move minus 24 into RAX.
What is this value first of all that I'm storing there?
What's going to be in RAX at the very end?
RAX is the return value.
So I'm trying to save, here in this case, this is the branch where I just want to return n. I just want to put RAX to have it return, be in RAX when I return.
So I've got the value here.
It's just n. It was in RDI.
It's now in RAX.
But that's clearly unnecessary.
Why go put it into memory and then take it back out again?
And here just put it in RAX directly.
So that's what I did.
I basically, instead of moving it here, I changed this instruction that said add it and put in RBX, I said, no, don't put it in RBX.
Let's just add RBX into RAX, and then it's right there.
And this one, get rid of those so that it's now moved into RAX and it's in RAX.
So I did that.
I dropped to 3.9 seconds.
That felt pretty good, too.
In addition, I got rid of this extra variable.
So now I could actually reduce my storage requirements.
However, when I measured it with this being 24 and not being 24, it was the same speed.
So it's like, eh, but I didn't want to waste the storage anyway.
So then I looked a little bit further.
And I noticed that I want to get rid of this access to n here.
So basically I'm subtracting it, and I'm storing n. How can I get rid of it?
And this took me a little while to figure out.
What I realized is, look, we're storing stuff away in RBX.
We have RBX as an available register because I saved the value of RBX with this push instruction there.
So RBX is an available register.
We're using it to keep the return value of the first call to fib.
So I'm going to use it for their first call to fib so that when I make the second call to fib I can then add the two things together.
Well, how about if before the first call the fib, why don't I use it to store the value of n and then use it to store the value of the return value of fib of n minus 1?
So I did that.
And that took a little bit of moving things around a little bit.
But I managed to get rid of it by using RBX for two different purposes, one to store the temporary, the value of n, and the other to store the return value when I need it.
And when I did that, I got it all the way down to 3.61 seconds.
I actually ran it with minus oh three, took about two seconds.
So I think I can keep my day job.
But kind of fun to go in and sort of see what are the things that can be done.
And you can get a very good sense of what's going on.
The more important thing is when you look at compilers generating your code, as we saw on the last lecture on profiling, you can see, oh, it did something silly here.
So you can actually go and say, oh, it's doing something silly.
We can do a better job than that.
Or, oh, I didn't realize I'd declared this an int when in fact, if I declared it a unsigned int 64, it actually would produce faster, better code.
Yeah?
Question?
Sorry.
So when you said that when you run it with minus oh three, what you're just saying is even though you optimized the [INAUDIBLE]?
As the compiler, that's why I said I can keep my day job.
So simple optimization strategies, if you're playing with things, is you try to keep values and registers to eliminate excess memory traffic.
You can optimize naive function call linkage.
And the most important thing probably is constant fold.
Look to see where you've got constants that can be combined together.
There are other optimizations that compilers do like common subexpression elimination and so forth.
But these are sort of the ones, if you're doing it by hand, these are sort of things to focus on, particularly number one, just get rid of excess memory traffic.
Let me say, by the way, in doing this I also went down a bunch of dead ends, things that I said, oh, this should definitely save, and then it was slower.
And then we look at it, and it turns out, oh, my branch misprediction rate is going way up and so forth.
That's why you have a profiler.
Because you don't want to do this blind.
Now, how does the compiler compile some common high level structures?
So if you have a conditional, for example, if p, do the ctrue clause, else do the cfalse clause, what it does basically is it generates instructions to evaluate p. And then it does a jump with the condition to see if p is false to the else clause and executes those instruction.
And then otherwise it passes through, does the true clause, and then jumps to the end.
And you'll see that pattern in the code when you look at it.
So that's a very common pattern for doing conditionals.
Compiling while loops is kind of interesting because most while loops start out with a jump.
So here are the instructions for the body of the while loop.
And here's the test.
And what they usually do is they jump to the test, they evaluate the condition, and if it's true, they jump to the loop.
Otherwise, they fall through.
And then they go back, do the loop sometimes for the first time, et cetera.
So that's kind of the pattern for a while loop.
For a for loop, they basically just convert it into a while loop.
You basically take the initialization code.
You execute that.
Then while the condition is true, you do the code followed by whatever the next code is.
And so it ends up converting for loops into while loops.
Now, arrays are, how do we go about implementing data types?
Arrays are just blocks of memory.
So you can have basically three different types of array depending upon where it gets allocated, either allocated in the data segment, allocate on the heap, or allocated on the stack.
Sometimes even the static arrays these days can be allocated in the code segment, if you're not going to change them.
So one thing is to understand that arrays and pointers are almost the same thing.
If you have an array, that's a pointer to a place in memory where the array begins.
And a zero is the same as the value you get when you dereference the pointer to a.
A pointer, if you think about it, is actually just an index into the array of all memory.
And the hardware allows you to index into the array of all memory.
Well, it can also allow you to index into any subregion of that memory.
And that's why arrays and pointers are basically the same thing.
Here's a little quiz.
What is eight of a?
The a elements?
Yeah, it's basically a of eight.
It's basically a of eight because the addressing that's going on is essentially the same.
Even though we prefer to write it-- If you start writing code like this, I guarantee that at some companies they'll get very angry at you even though you say, it's the same thing.
But what's going on is you're actually taking the base, the address of a, you're adding eight to it, and then dereferencing that value.
And indeed, even in C, they actually do all the coercions, even if eight is a different type, it actually does the coercions properly so that they are actually the same thing.
Because it does them after it's converted it into a dereference of a plus 8.
So it's kind of interesting that it works right even though when I looked at that I say, well, what if it's bytes verses words and so forth?
Yeah?
Question?
Will there be a performance difference between putting data on those three different arrays?
Yes, there can be.
In particular, static array, it knows exactly where the base pointer is as a constant.
Whereas the others it has to actually figure out where it is in the heap you need a pointer to it to dereference it.
It can't put it right into the instruction stream itself
[INAUDIBLE] those would be just constant [INAUDIBLE], right?
So I can either put that in the static array or--
Yes, generally it's faster to have it in a static array if you can.
I want to finish up here so that-- We have structs.
Structs are just blocks of memory also.
So you can have a bunch of things here.
This is a bad way to declare a struct because the fields are stored next to each other generally in the order you give them.
So here it says it's x and then i and double.
What happens here is you have to be careful about alignment issues.
So if you do [?
char, ?]
it's then got to pad it out to get to the next alignment for an int.
And it's got to pad that out to get the next alignment for a double.
Whereas if you do it in the opposite order, it starts packing them.
So generally it's best to declare longer fields before shorter fields because then you know when you're finished with the longer fields, you're already aligned for shorter fields.
Like arrays, there are static, dynamic, and local structs.
So that's all they are.
And so you'll see in the indexing-- There's also stuff that-- and actually this is important for one of the binary puzzles we gave to figure out what it does-- there are what are called SIMD instruction.
This is single instruction multiple data instructions where a single instruction operates on multiple pieces of data.
They operate on smaller vectors.
And there are 16 128-bit XMM registers, which you can view as two 64-bit values or four 32-bit values.
And you can do an operation on it.
So this is used for multimedia, for streaming applications, and so forth where you're trying to shove a lot of data through and you're doing the same repeated stuff on the things at time.
So there are instructions that operate on multiple values.
For example, here we're moving four 32-bit ints into this particular XMM register.
And similarly here's another one, we're adding it.
And you can look at the manual for these.
So you may come across these because they're using up parts of the machine.
Mostly those are the kinds of things we can say look it up in the manual, because nobody's going to remember all those instructions.
Of course, if you get a job with one of the graphics companies, then you may become very familiar with these kinds of instructions.
There's a lot more C and C++ constructs that we don't have to go into.
You can have arrays of structs versus structs of arrays.
And there can be a difference in performance.
If you have an array of structs, then it makes it, if you're accessing one struct, you can access the other structs very easily.
But if you're using things like the SSC instructions, then it may be better to have structs of arrays.
Because then when you're access an array, you can stream what's called a stride of one, a regular stride of just one memory location after the next, to do the processing.
So the hardware doesn't work as well if you skip by seventeens to gather things compared if you just get one thing after the next.
Because there's prefetching logic that tries to fetch things from memory faster.
There are things like function pointers.
So you can have store function pointer into something and then call that function indirectly.
There's things like bit fields in arrays.
There are objects, virtual function tables.
We'll get into some of these when we do C++.
And there's a variety of stuff having to do with memory management that we'll talk about.
But this is mainly to get you folks sort of at the level where you can sort of understand and feel comfortable with dealing with the assembler.
And you'll see that those resources are pretty good resources.
But the basics are relatively simple, but it's hard to do it without a manual or some online reference material.
Any questions?
What are the first two lessons I taught you today?
Number one is--
[INAUDIBLE]
--write tests before you write code.
And what's the second lesson?
Pair programming
Pair programming, not divide and conquer, I teach algorithms where divide and conquer is a fabulous technique.
With programming, pair programming is going to have you generally get where you want to get faster than if you're programming alone.
OK, thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so by this time, most of you should have had your meetings with your masters.
How many of you had meetings with your masters?
Who didn't yet?
Today
Today?
And Friday.
Who hasn't scheduled a meeting?
So are you talking to them, and do you know--
[INAUDIBLE]
OK, make sure you get masters meetings scheduled because this is a very important part.
We are getting feedback from the masters.
I think they are giving you guys a lot of good advice, and so make sure you schedule, you go, you meet, get that result, use that result.
It's very hard to arrange that kind of a senior person spending a lot of time with only two of you at a given time.
That's a really good ratio to have, so take advantage of that.
OK, so also, I think at this point, most of the beta issues for project one, we have been probably worked with, and they have some issues some people looked at that they are performance code.
They were not happy because the thing is this.
You can always run everything instantaneously if it doesn't have to produce a correct answer.
You can just return, and so part of performance is also it has to have some correctness.
So certain amount of correctness has to be passed before we can give performer grade.
So some people have this question.
Why are you giving me a correctness grade because it seemed to run?
But if you haven't done the right checks and stuff like that, then it's unfair for other people because there's no way people who do the correct implementation can match that performance because we're using that as upper bound.
So make sure you do that.
And another issue people had was when you go to masters, you're going to work with one or two other students.
And in your group, you're going to share all the material with your group member and nobody else.
But between the beta and final, you get opportunity to basically get input from your masters, and also see what the other people who are coming to a project in your discussion have done.
So it's OK for you to see those code.
You can't take the code home.
You can't just say give me a print-out, I am going to take home and re-implement.
That doesn't count, but if you see some clever things they have done, it's all here to learn.
Class is all the mostly about learning.
However we have to grade you, so it's not all about grading, it's about learning.
So we are trying to get a balance where we opportunity for you to learn from your peers.
So if you go look at other people's code, if you find something cool they have done, learn.
And also if you [UNINTELLIGIBLE] for you [UNINTELLIGIBLE] say, look, I did something cool.
It's a good learning experience in there, so that's one opportunity you have to talk to other people on other groups, to basically learn something or find some interesting tidbits of how to get good performance.
And hopefully, you learn some interesting things and able to implement that in your final.
OK?
So today we are going to talk about memory systems and performance engineering.
So if you look at basic idea of what memory systems, you want to build a computer that seemed to have a really fast memory.
You don't want things to be slow.
For example, a long time ago, people build these computers where memory's always slow, everything [UNINTELLIGIBLE] further away.
It doesn't help anybody.
The way we know how to do that is to build small amount of memory very close to the processor.
You can't build a huge amount of things all close to you.
That doesn't work, doesn't scale like that.
And we made a cache just like that, and you build larger and larger memory [UNINTELLIGIBLE] going down, and the illusion is you want to give the illusion of you have huge amount of memory.
Everybody's very close to you.
OK, so it looks like you have millions of friends, that you talk to everybody, and that doesn't happen.
But how do you give that illusion?
And the way that you give that illusion is when you use normal programming practice, when you normally using data, people have found there are two properties.
The first thing is called temporal locality.
That means if I use some data item, there's a good chance I use that data item again very soon.
There is something I haven't used for a long time, probably has a good chance I will not use it for a very, very long time.
So can I take advantage of temporal locality?
So that mean some data will be get for use a lot of time, in a very quick [UNINTELLIGIBLE], so those things should be very near to you because you might.
Other one is spatial locality.
That means if I use some data item, there's a good chance you'll be using data items next to that, closer to that.
So can we take advantage of that?
So those two properties help you help the compiler to fool the system, fool everybody, thinking that it has this huge amount of memory very close to you, but internally doesn't.
Unfortunately, since it's trying to fool you, it doesn't work all the time, and when it doesn't work, it's good to recognize and able to fix those things.
So I showed this picture sometime ago, too.
Memories in a big hierarchy.
L1, L2, in fact, we have L3 cache and a memory, disk, tapes, it can go up and down.
And the key thing is when you go here, you're starting from registers, you have very small amount of things, very fast access.
Here, we have almost infinite amount of things, very slow access.
And the two reasons why this has to be smart because first of all, those things are very expensive.
And second of course, you can't [UNINTELLIGIBLE] too much, and when you go down, things get somewhat cheaper.
So that was one of the kind of economic incentive here.
So I talked about cache issues, and I'm going to go through this again because we are really going to dig deep into these things.
So when you have a cache, that means you have a small memory close to you that's basically shadowing something sitting behind you.
So when you ask for the data in the cache, there are many reasons why it's not there.
First thing is cold miss.
What that means is you're asking for something that you have never asked before.
It's the first time you're asking, so the data is probably sitting way behind in your memory where, [UNINTELLIGIBLE] in a disk.
We haven't even loaded it in there.
And the first time you have to get it, so that get pulled in, and so this seems to be a place where there's nothing you can do.
But another part of locality is a thing called prefetching.
These modern processors have a crystal ball.
They carry most of the things that branch prediction and stuff is trying to predict what you want to do.
You can do the same thing in memory.
You can look at what you have done.
And Intel has this huge amount of circuitry that they don't tell anybody what they do, but they're trying to predict what you might fetch next, what data you might look for next.
And if it is working well, something you had never, ever looked before, it might deduce that you might need it and go get it for you.
So if that works, great.
So then there's thing called capacity miss.
What that means is as I said, I am trying to keep my best friends, or the people I will be working with a lot-- Oh, nice.
What's going on here?
Please start later.
OK. --the people I'm going to work a lot with, close to me.
The problem is I can have only certain number of friends, and if you have more than that, I can't keep everybody.
So hopefully, your program is going to use a lot what we call the working set that can fit in the cache.
And at that point, you get all of those people, like I said, get everybody into one room, and the problem is if your party spills over from the room, then there's a lot of issues.
But if everybody fits in the room, things are very nice.
You can have a good interaction, continuous interaction, with them.
And what happens is, if not, the worst case is the caches have eviction policy, normally is called least recently used.
That means if I don't have room in the cache, I figure out the cache line that I haven't touched for longest, and I get rid of that, bring the next one.
So that's a good policy.
It fits in your locality thing, but when will did not work?
[UNINTELLIGIBLE] going to figure out when least recently used will create a really bad problem in capacity
[INAUDIBLE] just one.
Say you're [INAUDIBLE] set, and that set is, by one, bigger than the [INAUDIBLE] missing
Very good.
So what you're going through, you're going round some data again and again, and that amount of data is one bigger than your cache line.
You are going to have no locality because [UNINTELLIGIBLE], and when you go to, just before you use that data, they said, oops, I need to bring one more data.
I need to evict something, and who am I going to evict?
The guy I'm going to use next because it was the least recently used thing.
So it goes, and then you go there, and you bring that.
Who's it going to evict?
You're going to evict the one I just brought.
You're going to evict the next one that I am just about to use and bring that one.
And you go use that, and then to use the next one, you're going evict the one you're just about to use.
So by going and doing that, you basically get no locality.
So that's a really bad problem in here.
So another interesting thing is called a conflict misses.
Conflict misses says normally, when you have a cache-- so what we have is you have this large storage, which [UNINTELLIGIBLE] the memory, you're going to map into smaller storage in here, which is cache.
So one way to say is any of this value line can be anywhere here.
That's called fully associative cache.
Implementing that is somewhat complicated, so the other extreme is called direct map cache.
What that means is this segment, the same size here, can map into here, and then the next segment also can map into here.
That means if you get this cache line here, it can be only mapped into one place in this here.
This cache line can only be here, and also this cache line can also only be here if it is the same offset.
So that means for every cache line, there's only one location in the cache.
So here, what would be really sad scenario?
[INAUDIBLE]
Yeah, I mean I assume I have a lot of these slots.
I read something here, and the next time I read something here, next time I read something here, and I go round and round this way.
I might be touching only very few number of data items, but they also mapping into one cache line even though my working set is very small than the cache.
I still don't have any cache usage because I am having conflicts.
And then there are two other misses when you're going to multi-process.
We'll go through them in later lectures.
One is called true sharing.
That means there are caches private to each core, and what happens is if one core uses some data, you've have to get the cache line there.
If the other core wants to use that data, you have to bring the cache line back there, and the cache lines can ping-pong [UNINTELLIGIBLE] that.
False sharing is even a worst way.
So what happens is if the first processor here touches this value of the cache line, second processor touches this value of the cache line.
We're not sharing anything, but unfortunately, the two things I'm using sits next to each other, so when I ride this thing, I get the cache line, he doesn't.
When he need to ride this thing, he has to get the cache line.
So I am seemingly using independent data, but I am bouncing cache line in between, and if that happened, that going to have a huge big performance impact.
So these other things.
The last two, we'll get to a little bit later.
So today I am going to start with modeling how caches work with a very simplistic cache.
So here's, assume, my cache.
I have 32 kilobytes in here.
This is a direct map cache.
That means that in my memory, 32 kilobyte chucks get mapped to direct [UNINTELLIGIBLE].
This 32 kilobyte get mapped here.
That 32 kilobyte get mapped here.
And the cache line inside is 64 bytes, 64, and that means I have basically [UNINTELLIGIBLE PHRASE].
32 kilobyte in here.
OK?
So just remember this as we go on doing this, we will use this as a formula.
OK?
And we assume if you getting the cache, [UNINTELLIGIBLE] single cycle, if you miss 100 cycles.
OK, so it's nice numbers to do that.
And so the first thing is we have a reference like this.
OK?
You go from [i] equals 0 to [i] less than very large number, A[i].
I just go to accessing memory like one after another after another after another.
So you see how this is going on in here?
So also I am accessing integers, so that means four bytes at a time in cache line.
So what happens here?
So I assume size of int is four, so you're getting four bytes in here.
So yeah, you're doing s reads to A. I'm accessing s elements in here.
And you have 16 elements of a per cache line.
OK, you see why 16 if there?
Because I have 64 in here, 64 bytes in here.
Each time axises four bytes, so I have 16 of integers in cache line
[INAUDIBLE] bits or bytes?
Bytes.
Byte.
Did I say bits?
No, bytes.
So what happens is if it axises the same element, 15 of every 16 is in the cache because when we are going one after the first guy, cache miss, I had to go bring it, and the next 15, I have brought to it.
It's in the cache.
So what should be my cost?
[UNINTELLIGIBLE] cost of memory access.
I am accessing s data, and 1/16th of that is a cache miss, 15/16th is a cache hit.
So basically total [UNINTELLIGIBLE] is a 15/16th of s is a cache hit, and other 1/16th I have 100 times cycles I need because it's a cache miss.
Everybody good so far?
OK, so what type of locality do we have here?
First of all, do we have locality?
Who thinks we have locality?
OK, lot of people thinks we have locality.
What type?
[INAUDIBLE]
Spatial locality because what we are doing is we are accessing the nearby elements even though we are not getting back any of the data yet, so we get some spacial locality axis here.
That's good, so this is our very -- most simple thing we can do.
So what kind of misses are we going to have in the cache?
Cold misses.
I'm missing because I have never seen that data before, so it's a cold miss in here.
OK, so that's that.
Let's look at this one.
I'm accessing A[0] s times.
OK?
What's the total access time?
How many cache misses am I going to get?
One, so I basically have 100 time for the first cache miss, and s minus 1 for the rest basically it's a hit.
OK?
That's good, so what kind of locality's this?
Spatial or temporal.
There's not too many choices.
Or none.
How many people think spatial?
How many people think temporal?
How many people said there's none locality?
OK, that's a lot of temporal.
OK, you want to get [UNINTELLIGIBLE].
That's temporal locality here because you're accessing to the same thing again and again and again, same data.
So this is-- Oh, OK. Want to restart, OK.
Wait a little.
So this is only time we notice in a hurry.
Every time, I go to do something, I get hourglass.
So what kind of misses are we getting?
Trick question.
I got one cold miss, and that's about it.
And the rest I don't have any misses.
So if I have a miss, it's a cold miss.
OK, so here I am doing something interesting.
So what the heck is in this list?
So I'm accessing A[i] [UNINTELLIGIBLE] this 2 to the power number, one shifted to the entire miss, 2 to the power N. I shifted between 4 and 13.
What this 13?
Why is 13?
Less than 13.
What's 2 the power of 13?
I think I got this right
8,192
What's 2 to the power of 13?
8,192
Yeah, 8K.
And I have 32K.
So 8K of 32 [UNINTELLIGIBLE].
8K of integers, which is each as 4 bytes is how much?
32K.
So what this says is everything I access should fit into the cache.
I am accessing data like that.
I'm going back and accessing data like that.
I'm going back and accessing data like that, and it should all fit into the cache.
Do you see what I do?
I access up to 2 to the power of 18, and I go back and access that again.
I'm just going back and back because of the model operation.
Everybody see what's going on?
So how many cache misses should I have?
There are four cache [INAUDIBLE]
OK, how many cache lines I would be accessing if I am doing i1 2 to the power N?
[i] mod 2 to the power
[INAUDIBLE]
So one miss for each axis line the first time around.
Afterward, it's only in the cache.
So how many axis lines?
2 to the power N. Is that right?
I think this is wrong.
Oh, yeah, this is right because every axis-- So in the cache line, how many--
[INAUDIBLE]
This is four.
So there are 16 entries here.
16 entries in here.
OK, only one of them basically misses that, so that means if 2 to the power N axises--
[INAUDIBLE]
You are doing 2 to the power N axises before you go back in here.
So what you're doing is you are-- you make this many axises before you go back again.
OK?
And how many axises assigned to be in the same cache line?
You have 64 bytes in the cache.
OK each axis is four.
16
[INAUDIBLE]
16.
So that's true [UNINTELLIGIBLE]
[INAUDIBLE]
Yeah, but what I'm saying is I might not access the [UNINTELLIGIBLE] because if any small.
I only access a certain amount of cache line, so I'm accessing a part of the cache.
I might not go through the-- can everybody see this, how this is working?
Because if I'm only accessing, we'll say, accessing one, if this is, we'll say, 2 to the power of 5.
OK, I am not going to access the entire cache line.
[UNINTELLIGIBLE] cache is I'm going to [UNINTELLIGIBLE] go through the cache.
So this many cache lines I'm accessing.
OK, and the first time I access that, I get a cache miss, and after that, it say everything is in the cache.
Everybody's following me?
Or are we like lost in here?
How many people are lost?
OK, let's do this.
So what happens is if you have a modular 2 to the power N. What that means is I'm going to keep accessing one to somewhere 2 to the power N, and the next one, I am going back here again.
That's my axises, basically, because it goes [UNINTELLIGIBLE] accessing to the power of N. I nicely said n is between 4 and 13.
So n is 13 means the maximum I can do is 8K.
8K increase.
8K increase equals 32K kilobytes of memory.
So that means you don't have any option of overwriting the cache.
So you go to cache, [UNINTELLIGIBLE].
You don't wrap around in the cache.
Do you see that?
OK, you know wrap arounding is so bad, that means all these things is going to fit in the cache, and then when you get [UNINTELLIGIBLE], this is going to be in the cache because the data you access is smaller than the entire cache.
OK, so what that means is only the first line has to have some cache misses.
So how many cache misses you are going to have in the first line?
Or any of the lines going to have cache misses because the first line still fit in the cache.
[UNINTELLIGIBLE] anything.
We don't need everything.
How many cache misses have in here?
The interesting thing is because these are four bytes, we have a 64-byte cache line, and each is four bytes.
So that means I can do 16 axises every time I have a cache miss.
OK, so up to here to here is 2 to the N. My cache misses I do [UNINTELLIGIBLE] basically.
My cache misses is basically this much.
Everything else is in the cache, and after the first line, everything else is in the cache again, [UNINTELLIGIBLE] nicely, got my working set to fit in the cache.
I'm really happy.
Everybody see this?
How many people don't see it now?
OK, good.
So let's move to something a little bit more complicated.
What kind of locality do we have?
Temporal and spatial
Good.
We have temporal and spatial.
You have spatial because every time you go, you only get the first line of the cache.
Rest is in the cache.
From that, I got a 16x improvement because I have temporal locality.
And since I only access when I go back to the data, after accessing this, since it's already in the cache, I get spatial locality because of the 16.
16 things are in the cache, I am going through that.
I get spatial locality.
I get temporal locality because next time I access, it's still in the cache.
I haven't taken anything out of the cache.
OK, what kind of misses?
Should be cold misses again.
Cold misses because the first time I [UNINTELLIGIBLE] things.
I haven't seen that data.
After I get it, everything is in the cache.
So now, here's interesting case.
Now, I am doing 2 to the power N, where N is great than 14.
Now, what happens?
Now, what happens in our picture here is I am going-- So as you might cache is somewhere here.
At this point, I fill out the cache.
I still keep going, and minute I access something [UNINTELLIGIBLE] cache, what's going to happen?
So I went through accessing 34 kilobyte.
If I access the next one, what happens?
OK, no, no, no shutting down, please.
OK, what happened?
So I am accessing more than I can fit into the cache.
Next time access, what's going to happen?
Anyone want to take a wild guess?
[INAUDIBLE]
[UNINTELLIGIBLE PHRASE] before that.
[UNINTELLIGIBLE] in there.
So what happens is then am I ever going to have any temporal locality?
No.
OK, so now, first access to each line basically misses forever because by the time I get back to the [UNINTELLIGIBLE], it's not there.
So basically, it's gone out of the cache.
So what that means is my total access time is basically every 16th element, I am getting cache miss forever.
So what that means, every 15th of 16, I have a hit.
So what I have is 15 out of 16, I have a cache hit.
1/16th of the time, I have a cache miss forever.
So what kind of locality do I have now?
I have spatial locality.
I don't have any temporal locality.
I don't ever go back to something in the cache again.
So what I have is spatial locality, so what type of misses now do I have?
[INAUDIBLE]
Cold, right
Is it like shared misses?
[INAUDIBLE]
It's not shared.
When you fill out the cache, you go back to [UNINTELLIGIBLE].
You fill the cache.
So what type of miss is that?
Capacity?
Capacity miss.
OK, so you have basically cold and capacity misses happening now.
OK?
One question.
[INAUDIBLE] that you multiplied whenever you're trying to load from memory, is that like an arbitrary number, or is that the actual cost of going--
Every machine, you can get this beautiful table.
I will go off what your machines have.
We still have a number saying this is your miss [UNINTELLIGIBLE], this is the thing-- I mean, so some of these things you realize because of all of the complexity.
It's not that nice and simple, but this, I just pull out of hat, and it's kind of normal.
So what do I do, now?
I am doing the mod, but I am multiplying [i] by 16.
So now, I am accessing this value, this value, this value, this value in here.
I'm accessing this value, this value, this value, this value.
One value in each cache line.
OK?
How much cache misses do you think I'm going to get?
[INAUDIBLE]
100 for every access is basically beginning a cache line, you're taking one value and if I go back to that value, I have already filled up the cache.
So basically, I have first access in the cache.
And what's your total access time?
Anyone will take a guess?
[INAUDIBLE]
Yep, 100 times x because I have no-- OK, so this is a [UNINTELLIGIBLE] locality.
That was clear.
What kind of misses am I getting now?
[INAUDIBLE]
Now, I'm getting conflict misses.
I get a cold miss at the beginning, first time around, and then every time I do something, I'm getting conflict miss because the thing is, now, this N, I am only accessing small amount of data.
It doesn't fit in the cache, but I'm not still getting any kind of sharing, so I'm having conflict misses.
[UNINTELLIGIBLE] second time, I guess I will probably jump over this.
I just did OK, if I go random access, what happens?
So let me jump over that.
Actually can do a calculation to figure out how many, probabilistically, how much you might have and stuff like that.
OK.
So now, if you look at what's going on, when you have no locality, you'd have no locality.
That's a pretty obvious statement.
And then if you have spatial locality and no temporal locality, we are streaming data.
That means you are going through the data, but you're not getting back to it fast enough, so I have temporal locality.
So I stream through data, so that means I go through data in a streaming fashion.
OK?
So what we have is if working set fits in cache, you have InCache mode.
If working set is too big for cache, you can get some streaming mode and still get some locality because we are bringing the lines in here.
And you can do other things like optimizers like prefetching we'll get to.
And other issues [UNINTELLIGIBLE] this last one, but to deal with cache axises.
So if you have more than one axis-- so here what I have is too nice arrays.
[UNINTELLIGIBLE] is 2 to the power size array 2 to the 19 power arrays.
OK, so now what happens when I put this in the cache?
Look what happens in here.
So what happens is this array get mapped in this one.
This array get maps this nicely in here.
So what happens is this line can only be in this cache line, and this line also only can be in this line of the cache.
Do you see what's going on, now?
So if you do this one, basically every time you access something-- See, assume I'm accessing this data item, so I'm doing A, B. I access A, I get this line.
I access B, what happens?
[INAUDIBLE]
So this [UNINTELLIGIBLE] is gone.
It is gone.
Now, I am accessing the next one in that cache line we made
How do you know what cache line-- or how [INAUDIBLE] cache?
[INAUDIBLE]
Normally, in language like C, if people two arrays next to each other, they should be mapped next to each other in memory.
And if you want to be more adventurous, you can go to the assembly and then see what their locations is.
So if you put A, A next to each other, you normally kind of get it next to each other.
So what this one is this is pretty bad.
I should have gotten at least some spatial locality.
I'm getting nothing.
Do you see what's going on here?
Do you know what's going on here?
Know why I'm not getting any spatial locality?
Because I am bouncing.
When I access this one, this one is gone even though I have more data in the same cache line, so I'm kind of bouncing two cache lines because of that.
What's a good solution for this?
Back there
[INAUDIBLE] A and B could be mapped to the same cache line
The thing A is a nice 2 to the power.
A size is a multiple of the cache size
[INAUDIBLE]
The size of A is a multiple of the cache size.
So what happens is if you have memory in here, you map something like A, C here.
If this is a multiple of cache line, so assume this is 32K times some number, and you map B here, and then this is normally what happens in C. You put all adjacent map allocations next to each other in memory.
Do you got A here, and you start having B in here, next to it.
OK, so what happens is now if A starts to assume address, we'll say 100, this is 100 plus 32K times N, basically.
And you do this mod 32K, this mod 32K, this same number.
It maps into the same line.
OK, so I kind of gave you the problem.
What can be the solution?
[INAUDIBLE] allocate one line
Yes, it's called padding.
OK, so I have no locality in here, let me go to the next slide, what kind of misses?
I have cold and conflict.
What I can do is just basically add a little bit of padding.
I did 16 here.
Normally, you do a prime number so it will not clash with anything.
Add a little bit of padding at the end of array, and that means the next array will not be conflicted most of the time.
So a lot of times when you declare things next to each other, it's always good to add a little bit of a padding in between.
Normally, a smaller prime number would be a good thing to add a padding.
Now, I start getting back my nice locality because the two things are mapping the two cache lines.
What type of locality do I get here?
Anybody want to take any guess what type of locality I have here?
Question, sorry.
[INAUDIBLE] you actually added just an extra 16
Yes
Why would that just make the A and B [INAUDIBLE]?
Because what happens is normally it doesn't make A and B into leaving the cache, but it makes A[0] not exactly matching to B[0].
A[0] will interleave with something like B[16] or something.
This thing might map into the same place
I mean, but I [INAUDIBLE]
Yes, normally in a computation, people do that, A[i] equals B[i] plus something
Oh, OK, so [INAUDIBLE].
Like I understand what you're saying, but I don't understand how that [INAUDIBLE] because they're both accessing [i] at the same time.
Why would they [INAUDIBLE]?
So here's the problem.
Assume I have [UNINTELLIGIBLE] 32K.
I'm assuming [UNINTELLIGIBLE].
OK, so first one is, we'll say, you start with 100, 100 plus [i], and the other one is 100 plus 32K plus [i].
OK, then you take a mod 32K, and this end up being 100 plus [i] with another 100 plus [i].
So this is basically [UNINTELLIGIBLE].
You still have one or one element in the cache it's all going to map into.
You see that?
But now, if I add 16 more to this one, then this map into 116 plus [i]
I don't see that reflected in the code.
You just [INAUDIBLE], so how is B different from A in your code?
Because [UNINTELLIGIBLE] my S. I added 16 to S, so my A is a little bit bigger, now
But isn't B also [INAUDIBLE]?
Yeah, B's also bigger here.
I don't care.
B goes down here.
B, I add a little bit more to the B.
It doesn't matter.
I padded B, too.
So what that means is the first A[i] and B[i] are not in the same cache line
How do we allocate A and B right next to each other?
So normally in C, if you declare something like int A [100], int B [100], more or less, they will be allocated next to each other.
If you look at the assembly listing, it'll say code allocated next to each other in memory.
Even in the stack, if you do that, if you allocate things, compiler has no incentive to go and move things around
This is only for global and [INAUDIBLE]?
Global variable, local variables
Does any of it still apply if you use malloc?
The thing about malloc is--
[INAUDIBLE]
It might do something [UNINTELLIGIBLE], but on the other hand, if it is fitting on the same page, it might also do something like that, too.
So it depends on how we do that.
At the beginning, if you keep mallocing things, it might be [UNINTELLIGIBLE].
They might be a little bit off because malloc put some metadata with each status.
If you ask for [UNINTELLIGIBLE], you're not getting exactly 100.
You're getting a little bit of a bigger size.
But you might have some configuring to do, but after some time, when you have done a lot of [UNINTELLIGIBLE], we have no idea where it's going to go.
It's random.
Anymore questions?
OK, good, that is a good question.
So what kind of locality do I have here?
[UNINTELLIGIBLE] two lines, I am accessing each, and I got 16 times 15 out of 16 hits.
What's that?
Spatial?
Spatial locality.
And of course, if you have spatial locality, you get cold misses, basically.
And I think [UNINTELLIGIBLE], you get what other type of misses?
[INAUDIBLE]
[UNINTELLIGIBLE] conflict here.
Basically, capacity.
Capacity miss, exactly.
So in order to avoid this, what we have done is we make cache-- instead of mapping into one place before, and what we said was we had this memory, and assume we have 32 sets.
And if you have a cache like this, we make this entire map into this one, and this map into this one.
What people have done is instead of doing that, what you can make is make the cache, instead of one big block, two small blocks.
So what that means is this can happen to this one.
My drawings are not that great, but I hope you get the picture.
This map into this one, so what that means is each reference here, so maps here, and these two basically conflicts.
But now with that [UNINTELLIGIBLE] too, I can put this one here or here.
I have two places to put the value, basically, and that [UNINTELLIGIBLE].
OK, so that means if I access something conflicting, [UNINTELLIGIBLE], you cannot just two things.
Still, there are places in the cache for them to go.
And one way to do people is fully associative cache.
It can go anywhere, but that's very hard to do in hardware, but you can do some small number of ways.
So in here, if you do two basic associative cache, in the first axis in here, even though these are conflicted, on can go to one [UNINTELLIGIBLE] set, another one can go to the other set.
OK, so two associative cache [UNINTELLIGIBLE].
So total access time, again, I have nice locality in here, spatial locality, and I have both cold and, I guess, capacity misses.
But if you have two associative cache, what's the next problem?
How about if you have three different things?
So we have two different things like that.
OK, now that I only have two associated [UNINTELLIGIBLE], it doesn't work because, in a [UNINTELLIGIBLE], I will replace the last thing, and you might still end up with nothing in here because you are replacing nothing, and the next thing replaced the other thing.
And basically after you gave B, C, then you go to A.
A Is not there.
When you go to B, B is not there, C is not there.
You kind of go back to the same problem in here, and then basically have to access everything again.
There is some spatial locality but cache can't use it.
Basically, you have conflict misses.
OK, so you see that?
So associative is good because normally people assume you don't need that much associativity, but it's limited.
So if you have too many things, you might run into this problem again
[INAUDIBLE]
You mean capacity?
Yeah, [INAUDIBLE] capacity
Less
[INAUDIBLE] elements in a cache before you pick [INAUDIBLE] because it looks as though the matrix is is using half the cache by itself
Because it's associated, you mean?
You have multiple there?
I mean, the thing is since it's associated, no, because you have enough room.
I mean, if you're going [UNINTELLIGIBLE] everything, you have two places to put the name because it maps.
That's a good question.
So what happens is now, assume this many things fit into the cache before.
Now, cache has two banks, and in each bank, only this many things fit in, but the next thing can go to the other bank.
So if you go through [UNINTELLIGIBLE], you have the same behavior even if you have associativity.
OK, you can go and-- because if the cache is the same size, the size is what matters for if you are going through [UNINTELLIGIBLE] axis
So far, [INAUDIBLE] machines that we're working on, do they have a fixed kind of cache, or can we decide like--
The fixed kind of cache.
It's built into the hardware.
I will show the table to say what type of associativity is in there
Is there not a way to say [INAUDIBLE] three-way associative?
Can you have something on top of it so that it looks like it's two-way associative or something?
I mean, you can do things like that in software, but they're very expensive.
I mean, think about cache is trying to fool the cache in that direction cannot be that easy because they are like really hard-baked into hardware.
that's a good question, but there are things you can do, not exactly modeling something, but kind of putting offsets and stuff like that so it doesn't get into these kind of conflicts.
So yeah, I will answer this thing.
Why don't you have more?
Because if you go back to your [UNINTELLIGIBLE] lectures.
Perhaps it might have explanation in it's hardware why it's much harder to build.
And if you have the linked lists now-- so assume I have linked lists we are going through in here.
And what's my [UNINTELLIGIBLE]?
Basically, depending on allocation and using some things, so best case is everything is in cache, so I have a small list.
And I go through again and again, round and around my list, everything's in cache and [UNINTELLIGIBLE] S. That's really great.
Next best is everything is adjacent.
OK, everything might not be in the cache, but everything at least in adjacent memory.
So basically, first time I allocate my malloc did a nice thing for me, and at that time, I basically get a nice basic spatial locality since my size is 16 in here.
So I get this much.
And the worst case is random.
Malloc puts it all over the place.
Question?
[INAUDIBLE] cache, does the thing behind it [UNINTELLIGIBLE] in front of it [UNINTELLIGIBLE] also going to?
Do those both have a spatial locality?
What happens is normally, we assume 64 element cache, so it's at that boundary, so basically 2 to the 8 boundaries is what happens.
So what happens is cache line begins at 0 address, 64 address here.
So if you get 63, that means this value is not in the cache, so you get this cache line.
So the cache line doesn't shift.
It's fixed chunks in here, so that if you get address 0, you get this one, and all these things.
If you get something in the middle, you'll get the left and right [UNINTELLIGIBLE], but if you get here, you only get the right.
So this basically 0 and 64, they are boundaries [INAUDIBLE].
If you like 65, that's it.
But that said, if you're accessing [UNINTELLIGIBLE], that's this thing called prefetcher that say you are going through linear pattern, and you might need that next, and it'll bring this up for you.
And what you find, I'll show later, that in fact, the machine you're using has a really powerful prefetcher, so it can hide a lot of these things.
So we'll get to that.
So this is my total access time in here.
Random case.
Random case is basically, every time I access something, it's not in the cache.
And then if you have a bigger struct in here-- but assume I have this big struct in here, but I am only accessing one element of this struct in here.
I do a lot.
I created this large struct, and I am [UNINTELLIGIBLE] things in the struct, and I just basically looking at my data.
Only one element.
What's the problem here?
[INAUDIBLE]
Exactly, so I bring the entire cache line, but I'm only using a little bit of it.
Because I bring all these things into the cache, it comes with axis and that, but I'm only using data.
So basically, what happens is, if everything in the cache, I have access time S. That's great.
That's not a problem, [UNINTELLIGIBLE] streaming.
Even though I'm getting spatial locality, I only get half of it because even though I'm bringing the cache line, only two have datas because this is 32 bytes long.
It's in my cache.
Even though I am bringing all this data, I only two of them in the cache because rest of this other data is kind of hanging in there with me, and so I don't get that.
And random, of course, you have no locality.
So how can I basically make this better?
How can I make this better?
OK, I just showed it very fast
[INAUDIBLE]
So instead, if I ask you, I have this large data structure, but most of time, I'm only accessing a small part of it, this is kind of a little bit of a hack.
I mean, this is kind of going against, I guess 6005 kind of building, but this actually works.
What you can do is, you can go back and instead of doing structs, what you can do is you can do arrays of each data item.
And now what happens is that I am basically bringing these, and then the data I am accessing is basically right now next to each other.
So you might have this very large data structure representing something you have a lot of, then.
But most of the time, you're only using few of them, so you don't have to be structs of arrays.
You might have couple of different data structures.
One is the one that you access regularly hopefully you get them next to each other.
And the nice thing about arrays is it guarantees things are next to each other.
Of course, was managing it is harder
[UNINTELLIGIBLE]
You can do both.
So that means instead of doing the structs, you use arrays of each field
[INAUDIBLE]
Instead of.
So instead of doing this one, you just do this one
[INAUDIBLE] instead of being able to pass stuff over.
Now you have to kind of be able to know [UNINTELLIGIBLE], so what that is where because you cannot just pass the struct [UNINTELLIGIBLE]
No, you can't pass the struct.
You basically have to pad the index in here for this structure, basically.
So you have to know something in there.
So this is a lot messier, but it can actually give you a performance if you do this type of access, so it's not something you want everywhere.
That's why you still want to use structures and nice classes and stuff.
But this, if you're doing very large data, but most of the you're accessing only small amount, this kind of representation can help.
So I want to get a little bit into matrix multiplying.
So what we're looking at is this four by four matrix, and you can represent it using this representation.
But to make it clear, what I'm going to do is I'm just going to allocate this entire thing as a single array, and I'm going to do that [UNINTELLIGIBLE] calculation myself because if you go a little bit down, you realize this is allocated in a little bit of a weird way.
And so this is what we call row-major because all rows are next to each other, and you can access allocated column-major.
Basically that means now A001, instead of that, 1, 0 is next to 0, 0, 1, 0, 2, 0.
And then basically saying same thing, [UNINTELLIGIBLE] instead of saying 4i plus j, it's just 4j plus i.
You can do both.
So this is my nice matrix multiply function, because the reason I did it this way, it's just very clear how many axis I'm going to use.
So if you look at C, let's look at only the inner loop.
Most of the mapping matters in the inner loop.
I go for k in 0 to size.
Only first axis on C is going to miss because it's going up [UNINTELLIGIBLE] same K, da-da-da-da-da, so C is very nice in here.
Only get one miss in here.
You have a question?
OK. A has a streaming pattern because every time I update K, I go to the next element, next element, next element, next element.
[INAUDIBLE].
What type of a pattern do I have in B?
Looks like [INAUDIBLE]
Yeah, exactly.
I have jumping on memory.
I'm jumping one to another.
So if you look at that, I am going through A like that, and B, and going through C element at a time, basically.
So if you look at doing this in order to calculate one element of A, [INAUDIBLE] calculating these, what you have to do is I am getting B and C this way.
I'm going through row here and column, and this row [UNINTELLIGIBLE] fit into the cache line through that because this is the memory.
But C is just jumping all over the memory, and I have no locality in here.
So what can we do here?
What's the simplest thing in here?
How do we get some locality into the C?
[INAUDIBLE]
So yeah, you're coming, you're getting it.
That's the next thing that's most important.
That's the more advanced thing you can do.
We'll get there.
What was a simple thing you can do for C?
[INAUDIBLE]
[UNINTELLIGIBLE] So here's the [UNINTELLIGIBLE] execution time, original, and voila, when you transpose that, you can actually get a huge benefit in here because, now, I get spatial locality in both sides in here.
And so now when you're transposed, I am going through like this.
So now what I did was I am giving this CPI, and what happens is when you transpose, I have a 5x improvement in sync CPI.
That means instructions actually not waiting, not doing nothing.
I have a pretty good reduce of cache misread.
because, now, I'm actually going through cache very nicely.
So the next thing we can do is, what she just pointed out, blocking.
So instead of matrix multiply, just multiplying this matrix at a time, you can multiply submatrices and calculate small submatrices.
And if you do that, the submatrix can nicely fit in the memory, so let me show you that.
This is the blocking code.
Let me show you what that means.
So what we are doing is you are multiplying this matrix by this matrix in there and getting this matrix.
So what that means is, normally, what you do is, to get one element, you have to multiply a row and a column.
Normally, we are calculating a row of A at a time.
In order to get a row of A, I had to get a row of B [UNINTELLIGIBLE] entire C has to participate into calculate A.
So by the time I have calculated 1024 elements, I have gone through this many axises to new elements basically because I am getting B again and again, but I had to go through the entire C in here.
So if this much data is bigger than the cache, I have no locality in here.
But if, instead of calculating a row, if I calculate a 32 by 32 block, which is again 1024 elements I am calculating.
If I do that, I need a block here and a block here, and I only type this much data.
OK, I'm getting the same amount of output, but because of reuse, I touch a lot less inputting here.
And just by doing this transformation, I can really reduce the amount of data.
Basically, I can create a smaller working set, go through that much more, and reduce the amount of cache misses you get.
So by doing that, I basically go out another 3x improvement in my CPI, really brought down my L1 cache misses, and basically eliminated L2 cache misses.
OK, so I got a really good speed up in here.
So you can block multiple levels, L1, L2, L3, because every time you figure out the workings and size, you block so the data in that will be safe in the cache in here.
And this is kind of nasty to do by hand.
We'll see.
So the other interesting thing is call stack locality.
So if you do something like fib in here, traditional fib, if you do fib(4), you calculate a fib(4), fib(3), fib(2), you kind of build a stack, go up back again in here.
So if you keep doing that, the nice thing is you're going up, but its-- Hopefully, one of these days, I am going to hit the wrong button, and-- What kind of locality do we have if we are accessing data like this in my call stack?
What kind of locality do we have?
I mean, I access here, fib(4), and I get back to fib(4) here.
So within each frame, I am accessing data next to each other, so I can have a little bit of a spatial locality because, each frame, all the data is next to each other.
But across frames, normally, I will have some temporal because I will get back to that data some time.
OK?
OK, before I summarize, this one.
If you are reading a lot of data, processing through stage one, [UNINTELLIGIBLE] and producing [UNINTELLIGIBLE], that means you're reading a huge amount of data through stage one, save, save data again, do stage two, you have really bad cache locality.
One thing you can do is read a small chunk, do all the stages on that small chunk, save it, and go to another chunk.
So instead of doing the entire data, doing something [UNINTELLIGIBLE] producing, you can do this chunk-ifying, and by doing that, you can get a lot of locality.
So once you bring in data, you like to do as many things as possible to that data before it gets out of the cache.
That's the entire thing about locality, and you can restructure code through that.
So if you look at that, what we have done today, we have looked at things like associativity, cache lines, cache sets in here, and how to address and map to a cache, we kind of did this thing, and we look at [UNINTELLIGIBLE] streamings versus InCache versus no locality type patterns.
And then when you look at data structure transformation, we look at how to basically replace structured arrays and split data structures to make sure that you get a lot of nice locality if you access a certain amount of data, and pack data to get smaller cache footprints kind of things.
And we also look at computer transformations, things like transpose copying, compute copy, outtype things you can do, so you can actually get nice locality, so you can do both data and compute transformations.
And [UNINTELLIGIBLE] axises and [?
re-cast ?]
calculation stages, so this is kind of summary.
Next thing I want go is look at what machines you guys are using.
So this is the machines we used last year.
These are quad cores, and it had L1 cache, so these [UNINTELLIGIBLE].
So this has 32 kilobyte L1 cache, data cache [UNINTELLIGIBLE] two caches in here for data and instructions, each core, and then the cache line size is normally 64 bytes [UNINTELLIGIBLE].
And here, latency [UNINTELLIGIBLE], you get three cycles.
If not, you get 14 cycles to go to to L2 in here if data doesn't meet there.
And associated with this [UNINTELLIGIBLE PHRASE].
So that was what we did last year.
This year, we have a bigger machine that has multiple levels of caches.
So each core has L1 instruction and data cache.
It's kind of the same, 32 kilobyte cache, four nanosecond latency in here.
Eight, three and four [UNINTELLIGIBLE], they kind of reduced that, but they added a 256-kilobyte L2 cache in here, and then a pretty large humongo 12-megabyte L3 cache in here.
The interesting thing is here, this pretty fast access to these ones.
If you miss in here, this is about 2 and 1/2 times cost from going from here to here.
From here to [UNINTELLIGIBLE], about a five times cost, so here to here is a five times cost.
The interesting thing that actually surprises a lot us was here to here, it's cost is not that different.
[UNINTELLIGIBLE PHRASE] So actually memory, even though it's sitting out in there, they have managed to get it pretty cheap access.
Normally, my intuition say this should be much higher because you're going off-chip, so this is the structure here.
It's a lot more complicated.
So here's something I produced last year.
So what I did was I ran this program.
And in these machines, there's a way to turn off the prefetch of the previous [UNINTELLIGIBLE].
So when you've turned on the prefetch and then you run, what you are doing is I am accessing data with a working set, small working set, little bit bigger working set.
So this is basically when you consider working set size.
So you get a graph like this.
I am hitting my L1 cache, and if it is more than that, I [UNINTELLIGIBLE] L1 cache, I go to my L2 cache.
And if L2 misses, I have to go to main memory.
Very traditional kind of a graph in here.
Another a little bit of an interesting thing, which is instead of accessing next item, I kind of skipped in here.
So this trying to force how to get conflict misses.
And so if you line up things like this exactly, what happens is you get these conflict misses, and you get this number.
So I have interesting story to tell.
When I was a graduate student, we were running this bunch of benchmarks, and normally when you run a benchmark, you try it for an equal [UNINTELLIGIBLE], stuff like that.
And we got this graph like this.
I mean, we expect a graph going like this.
OK, that how a benchmark [UNINTELLIGIBLE].
Then at one point, by accident, instead of typing an equal 64-something, I typed an equal 63, and suddenly, I got a performance like this.
Wait a minute, what's going on?
So in here, what I got was not a line.
I got 16, 32, 64, 128, nice graph like that.
And then I just did 63, like down to here.
I'm like, geeze, that can't be right.
And then I start looking at 62, which is here.
61 is here.
So I realize if I enumerate all those things, I got a graph like this, basically.
Kind of something like this.
And the thing is, if I had never tried the 63, I would have basically thought my performance was like this, so far worse.
Never realized because it was all having conflicts in memory because I'm [UNINTELLIGIBLE] 2 to the power numbers.
So this is what happens if you just look at 2 to the power numbers, you'll think, yeah, this is the performance you're getting, without realizing that, in fact, you can get this far.
So this is good.
So I ran it on the new machine.
The problem with our new machine is there's no way to turn off prefetching.
OK, so this is what you're getting.
Everything, because I am accessing this nice linear, looks same.
This is exactly what you want to happen.
What it's doing is, minute you go to access, it realizes you accessing linearly.
It start fetching, fetching, fetching data in front you, it's going to catch up.
It's not feeding you data, so you're not going to have any cache missed because you're going through this nice, linear pattern, and you just basic end up with this beautiful graph, which gives the feeling your working set could be, now, gigabytes long.
At this point, I am fitting into the entire L3 cache, and still look like this works beautiful because this is what you want for it to figure out.
So now I'm like, geeze, this is great.
This means my cache works, it's very nice, but I want to show you guys what's going on, so that doesn't help.
So I came up with this program.
So this program, what I did was I created a working set, but I'm accessing randomly within that working set.
I calculated this address.
This is my poor man's random number generator, so basically I'm adding number in here and multiplying this, and then the masking these to get the right [UNINTELLIGIBLE] so I get within that set in here.
And then I just access data, so I'm getting [UNINTELLIGIBLE].
So I get a number like this, and this jump up here, and then I had this kind of behavior.
This is a lot more complicated.
I need to figure out what the hell is going on in here, so I said, OK, let's now look at performance counters.
I look at L1 cache miss.
OK, up to 32K, I have no cache misses.
Boom, cache misses jump up.
Perfect.
And I have this little bit of a kink in here, this beautiful describes because I have 32K L1 cache, and until then, I have no things, and then when the cache start missing, I'm jumping up.
Perfect.
I got that description here.
Then I look at my L2 cache misses.
This is, I think, the L2 cache in this machine might have a little bit more complex [UNINTELLIGIBLE] telling us because this is not going like here had jumping up.
That's a 256K [UNINTELLIGIBLE] get.
Is this 256?
Yeah.
[INAUDIBLE] 250 is [UNINTELLIGIBLE] these things.
OK, so we have 32 256 2L megabytes in here.
Even if you have 256 in here--
[INAUDIBLE]
Hm?
Should it be 256 or 32--
[UNINTELLIGIBLE] because everything's backed up, it's hierarchy [UNINTELLIGIBLE], so it's not added to, it's not next to each other, because at this point, L1 is now useless.
It'll run everything is just going to L2, so L2 is the one who was serving it.
Come on.
So what happens in here, this normally should start jumping here, 256, but it start jumping earlier, so there might be some other interesting thing going on here.
It might be interesting to investigate.
So this is OK, so still I am pretty close.
I realized this jump is basically because of L2 happen, this jump in here.
L2 start missing.
Then I look at L3.
It's here.
What's happening here?
So this was a big mystery the two of us were trying to debug what one hour ago.
Not one hour.
Class is going.
Like two hours ago.
We was trying to figure out, OK, wait a minute, this doesn't make sense.
My performance is here.
L3 should be just basically a nice flattening out here, and just jumping.
It's not doing that, and we are like what's going on here.
So then we were scratching our heads, and we start looking.
I say, ha, there were other things going on.
We looked at this thing called TLB.
Let me tell you what TLB is.
What we realize is at this point, we are [?
seeing ?]
TLB misses.
How many people know what a TLB is?
OK, good, so let me spend the last couple of minutes explaining what a TLB this, and that kind of explain what's going on in this graph.
So TLB means-- Normally, data is stored in memory using real addresses.
That means there's a memory size in here, there's an address format, there's certain [UNINTELLIGIBLE].
And then the program for it use virtual memory.
That means programs want to feel like they have a lot more memory than actual, physical memory available.
So what happens is you have virtual memory address, but somebody has to map virtual memory to physical memory.
So the way it work is there's a thing called translation lookaside buffer, TLB, that says if you give this virtual address, here's the right physical address.
So of course you can't do it for each address by address.
What you have is you have 4K pages, so the address space is broken down into 4K pages.
Each page will have an entry in this TLB, saying if you get this virtual address, here's the right physical address.
So normally what happens is when the operating system they created this large table, for every place in memory, you use this table, and it loads into memory.
Then the hardware will fetch and cache some of those entries in this TLB cache in there.
So normally what happens is in here, it caches 52L entries.
It can keep 52L entries to 52L pages, saying here's the mapping.
The problem with 52L pages is in a 4K byte page size, 52L pages can only hold two megabytes of data.
If you're accessing more than two megabytes of data, you have to go more than 52L pages, and at that point, that mapping is not in the TLB, and that means that you have to go do a mapping translation.
The problem for this is these 4K pages is a very ancient artifact.
There's the new machines, you can I ask for what we call large-pages or super, but you can ask for 2-megabyte or 4-megabyte pages.
If you ask for super-pages, then of course, 52L entry's good enough.
It can hold enough things not to have TLB miss, at least while you're in the cache.
But since Linux, without doing anything, only asks for 4K pages, before you run out of your L3 cache, you run out of the TLB table.
And it has to start fetching and [UNINTELLIGIBLE] data table, that means you had to actually fetch the table entries, and so you have to do more fetchings.
So that is why this is even running higher, because it's trying to fetch this TLB entry and stuff like that.
Question?
Is there a way to then actually fiddle with the TLB so that you have to fit your pages rather than--
So in allocation, you can ask for super-pages or large-pages.
But in Linux, normally just ask for 4K pages, so that the operating systems haven't kind of caught up to the current hardware.
And this is was built [UNINTELLIGIBLE] doesn't make sense TLB to run out before you run out the cache.
I mean, that's kind of mismatch in here.
But it's built assuming you'll get large-pages, but if you don't, you're going to run out of the TLB in there.
So this is why sometimes performance engineering is interesting, exciting, and frustrating, because you assume you know where the hell this is going.
You assume this is going to go here and that, and suddenly, this miss shows up, and so we have to scratch our heads.
And luckily, we found what's going on, but we could have spent hours or days trying to figure out what's going on here because this is one [UNINTELLIGIBLE] say OK, let's try TLB, and it worked.
That was really good, but I have, many cases, spent days trying to figure out why something is going on because performance is kind of a [UNINTELLIGIBLE] of everything.
There's no nice abstractions and stuff like that.
If you think abstraction wise, somewhere outside the abstraction, that's going to kill you.
So that is why it can be interesting, because you have to know everything.
And the fun thing is, once you understand performance, you basically understand end-to-end, soup to nuts, what's going on.
OK, with that encouraging thought, let's see how you guys do in the next part of project two.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good, we're going to take a detour today into the realm of algorithms.
So when you're trying to make code go fast, of course, there's no holds barred.
You can use whatever you need to in order to make it go fast.
Today we're going to talk a little bit in a more principled way about the memory hierarchy.
And to do that we're going to introduce what we call the ideal-cache model.
So as you know most caches are hacked together to try to provide something that will cache well while still making it easy to build and fast to build.
The ideal-cache model is a pretty nice beast if we had them.
It's got a two-level hierarchy.
It's got a cache that has m bytes that are organized in to b byte cache-lines.
So each block is b bytes.
And it's fully associative.
So you recall, that means that any line can go anywhere in cache.
And probably the most impressive aspect of an ideal-cache is that it has an optimal omniscient replacement algorithm.
So what it does, is it figures out when it needs to kick something out of cache, it says, what is the absolutely best thing you could possibly kick out of cache.
And it does that one.
Looking into the future if need be.
It says, oh is this going to be accessed a lot in the future?
I think I'll keep this one.
Let's throw out this one.
I know it's never going to be used again.
So it has that omniscient character to it.
The performance measures we're going to look at in this model, the first one is what we call the work.
And that's just the ordinary serial running time if you just ran the code on one processor and counted up essentially how many processor instructions you would do.
That's essentially the work.
The second measure, which is the one that's much more interesting, is cache misses.
So the work has to do with the processor.
The cache misses has to do with what moves between these two levels of memory.
So in this case, what we're interested in doing is, how often do I try to access something.
It's not in the cache.
I have to go to main memory and bring it back into cache.
And so that's what we'll be counting in this model.
So it's reasonable to ask how reasonable ideal caches are.
In particular the assumption of omniscient replacement, that's pretty powerful stuff.
Well it turns out there's a great lemma due to Slater and Tarjan that says essentially the following.
Suppose that you have an algorithm that incurs q cache misses on an ideal cache of size n. So you ran the algorithm on your machine, you had a cache of size n. Then, if instead of having an ideal cache, you have a fully associative cache of size two m and use the least recently used replacement policy.
So you always, whenever you're kicking something out of cache, you kick out the thing that has been touched the longest ago in the past.
Then it incurs at most 2Q cache misses.
So what that says is that LRU is to with it constant factors essentially the same as optimal.
Really quite a remarkable result.
Who's taking 6046?
You've just seen this, right?
Yeah, OK. Just seen this result in 6046.
See, I do talk to my colleagues occasionally.
So then something about how this is proved.
And what's important here is that really it just says, OK, Yeah you could dither on the constants, but basically whether you choose LRU or choose ideal cache with the omniscient replacement, asymptotically you're not going to be off at all.
So for most asymptotic analyses, you can assume optimal or LRU replacement as convenient.
And the typical way that you do convenience is if you're looking at upper bounds.
So you're trying to show that a particular algorithm is good, then what you do is you assume optimal replacement.
If you're trying to show that some algorithm is bad, then what you do is assume that it's LRU to get a lower bound.
Because then you can reason more easily about what's actually in memory.
Because you just say, oh we'll just keep the least recently used one.
So you tend to use the two for upper bounds and lower bounds.
Now, the way this relates to software engineering is as follows.
If you're developing a really fast algorithm, it's going to start from a theoretically sound algorithm.
And from that then you have to engineer for detailed performance.
So you have to take into account things like real caches are not fully associative.
That loads and stores, for example, have different cost with respect to bandwidth and latency.
So whether you miss some a load or miss on a store, there's a different impact.
But these are all the tuning.
And as you know, those constant factors can sometimes add up to dramatic numbers, orders of magnitude.
And so it's important to do that software engineering.
But starting from a good theoretical basis means that you actually have an algorithm that is going to work well across a large variety of real situations.
Now, there's one other assumption we tend to make when we're dealing with ideal caches, and that's called the tall-cache assumption.
So what the tall-cache assumption says, is that I you have at least as many lines of cache, essentially, in your cache, as you have bytes in the line.
So it says the cache is tall.
In other words, this dimension here is bigger than this dimension here.
And in particular, you want that to be true for where we have some constant here of slop that we can throw in.
Yes, question
Does that [INAUDIBLE] associatively make the cache shorter here
Yes, so this is basically assuming everything is ideal.
We're going to go back.
When you engineer things, you have to deal with the fact that things aren't ideal.
But usually that's just a little bit of a tweak on the actual ideal algorithm.
And for many programs, the ideal algorithm you don't actually have to tweak at all to get a good practical algorithm.
So here is just saying the cache should be tall.
Now, just as an example, if we look at the machines that we're using, the cache-line length is 64 bytes.
The L1 cache size is 32 kilobytes.
And of course, for L1 it's is 32 kilobytes and for L2 and L3, it's even bigger.
It's even taller.
Because they also have 64k line.
So this is a fairly reasonable assumption to make, that you have more lines in your cache then essentially the length, the number of items you can put on a cache line.
Now why is this an important assumption?
So what's wrong with short caches?
So we're going to look at, surprise, surprise, matrix multiplication.
Which, by the end of this class you will learn more algorithms than matrix multiplication.
But it is a good one to illustrate things.
So the idea here is, suppose that you have an n by n matrix here.
And you don't have this tall-cache assumption.
So where your cache is short.
You have a lot of bytes in a line, but very few lines.
Then even if the size of your matrix fits, in principle, in the cache.
In other words, n squared is less than m by more than a constant amount.
So you'd say, Oh gee, that ought to fit in.
If you have a short cache it doesn't necessarily fit in because your length n here is going to be shorter than the number of bytes on a line.
However, if you have a tall cache, it's always the case that if the matrix size is smaller than the cache size by a certain amount, then the matrix will fit in the cache.
OK, question?
Why wouldn't you fit more than one row per cache line?
Well the issue is you may not have control over the way this is laid out.
So, for example, if this is row-major order, and this is a submatrix of a much bigger matrix, then you may not have the freedom to be using these.
But if you have the tall-cache assumption, then any section you pull out is going to fit.
As long as the data fits mathematically in the cache, it will fit practically in the cache if you have the tall-cache assumption.
Whereas if it's short, you basically end up with the cache lines being long and you not having any flexibility as to where the data goes.
So this is sort of a-- So any questions about that before we get into the use of this?
We're going to see the use of this and where it comes up.
So one of the things is that, if it does fit in, then it takes, at most, size of the matrix divided by the cache line size misses to load it in.
So this is linear time in the cache world.
Linear time says, you should only take one cache fault for every line of cache.
And so that's what you'll have here if you have a tall cache.
You'll have n squared over b cache misses to load in n square data.
And that's good.
OK, good.
So let's take on the problem of multiplying matrices.
We're going to look at square matrices because they're easier to think about than rectangular ones.
But almost everything I say today will relate to rectangular matrices as well.
And it we'll generalize beyond matrices as we'll see next time.
So here's a typical code for multiplying matrices.
It's not the most efficient code in the world, but it's good enough to illustrate what I want to show you.
So the first thing is, what is the work of this algorithm?
This is, by the way, the softball question.
What's the work?
So the work, remember, is just if you're analyzing it just like processor forget about caches and so forth
n cubed
n cubed, right.
Because there's a triply nested loop going up to n and you're doing constant work in the middle.
So it's n times n times 1. n cubed work.
That was easy.
Now let's analyze caches.
So we're going to look at row major.
I'm only going to illustrate the cache lines on this side because B is where all the action is.
So we're going to analyze two cases when the matrix doesn't fit in the cache.
If the matrix fits in the cache, then there's nothing to analyze, at some level.
So we're going to look at the cases where the matrix doesn't fit in the cache.
And the first one is going to be when the side of the matrix is bigger than m over b.
So remember, m over b is the height of our cache, the number of lines in our cache.
So let's assume for this, now I have a choice of assuming optimal omniscient replacement or assuming LRU.
Since I want to show this is bad, I'm going to assume LRU.
Could somebody please close the back door there?
Because it's we're getting some noise in from out there.
Thank you.
So let's assume LRU.
So what happens in the code is basically I go across a row of A, while I go down a column of B.
And now if I'm using LRU, what's happening?
I read in this cache block and this one, then this one et cetera.
And if n is bigger than M/B and I'm using least recently used, by the time I get down to the bottom here, what's happened the first cache line?
First cache block?
It's out of there.
It's out of there if I used LRU.
So therefore, what happens is I took a miss on every one of those.
And then when I go to the second one, I take a miss on every one again.
And so as I keep going through, every access to B causes a miss throughout the whole accessing of B.
Now go on to the second row A and I had the same thing repeats.
So therefore, the number of cache misses is order n cubed since we miss on matrix B on every access.
OK, question
I know that you said it's e. Does B push out due to conflict misses or capacity misses?
So in this case they're capacity misses that we're talking about here.
So there's no conflict misses in a fully associative cache.
Conflict misses occurs because of direct mapping.
So there's no conflict misses in what I'm going to be talking about today.
That is an extra concern that you have for real caches, not a concern when you have a fully associative cache
[INAUDIBLE] n needs to be bigger than B?
So the number of lines to fit in my cache is m over b, right?
So can't you put multiple units of data--
Well there are multiple units of data.
But notice this is row major, what's stored here is B11, B12, B13.
That's stored here.
The way I'm going through the access, I'm going down the columns of B.
So by the time to get up to the top again, that cache block is no longer there.
And so when I access B12, assuming indexing from one or whatever, this block is no longer in cache.
Because LRU would say, somewhere along here I hit the limit of my size of cache, let's say around here.
Then when this one goes in, that one goes out.
When the next one goes in, the next one goes out et cetera using the least recently used replacement.
So my--
Spatial locality
I'm sorry?
You don't have any spatial locality here
I'm just wondering why they can't hold units.
I guess this is the question, why can't they hold multiple addresses per cache line.
So why is it even pushed out?
It's being pushed out [UNINTELLIGIBLE] one per cache line, right?
No, so it's getting pushed out because the cache can hold M/B blocks, right?
So once it's accessed m over b blocks, if I want to access anything else, something has to go out.
It's a capacity issue.
I access m over b blocks, something has to go out.
LRU says, the latest thing that I accessed, well that was the first one, gets knocked out.
So what happens is every one causes a miss.
Even though I may access that very nearby in the future, it doesn't take advantage of that.
Because LRU says knock it out
[INAUDIBLE]
Is it row major that's the confusion?
This is the way we've been dealing with are matrices.
So in row major, there's a good-- that's nice, there's no chalk here.
Oh, there's a big one there, great.
Yeah, so here's B.
So the order that B is stored is like this in memory.
So basically we're storing these elements in this order.
So it's a linear block of memory, right?
And it's being stored row by row as we go through.
So actually if I do it like this, let me do this a little bit more.
So the idea is that the first element is going to be here, and then we get up to n minus 1, and then we get to n minus 2 is stored here.
n plus 1, n plus 2, 2n minus 1.
So that's the order that they're stored in linear and memory.
Now these guys will all be on the same cache line if it's in row-major storage.
So when I'm accessing B, I'm going and I'm accessing zero, then I'm accessing the thing at location n. And I'm going down like this.
At some point here I reach the limit of my cache.
This is M/B.
Notice it's a different B-- script B verses-- so when I get to m over b.
Now all these things are sitting in cache, that's great.
However, now I go to one more, and it says OK, all those things are sitting in cache, which one do I kick out?
And the answer is the least recently used one.
That's this guy goes out
Do you only use one element per cache link?
And I've only used one element from each cache line at this point.
Then I go to the next one and it knocks out the second one.
By the time I get to the bottom and then I go up to the top to access 1 here, it's not in cache.
And so it repeats the same process, missing every single time.
We have a question
Yeah,so my question is why does the cache know where each row is?
To us, we draw the matrix, but the computer doesn't know it's a matrix.
To the computer, its a linear array of numbers
That's correct
So why would it load the first couple elements in the first row, and the second column is an extended row the second time
So the cache blocks are determined by the locality and memory
So my assumption would be the first cache line for say--
Let's say 0 through 3
So yeah, 0 through 3
Let's say we have four items on that cache line
4 to 6
The next one the hold 4 to 7, I think
4 to 7, yeah.
So that is a--
So that would be the next one, right?
4 to 7
When you get cache line.
You are not using the fully cache line.
There is no spatial locale.
You are using one from the cache line
So this code is using this one and then this one.
It's not using the rest.
So it's not very efficient code
So the cache line is holding the 0 to 3 and the 4 to 7.
[INAUDIBLE] n plus 2 just reading-- [INTERPOSING VOICES]
Right.
And those are fixed.
So if you just a dice up memory in our machine into 64 byte sizes, those are the things that come in together whenever you access anything on that line
And on this particular axis, you never actually get the 4 to the 7 in the--
Well we eventually do.
Until we get there, yes, that's right.
Until we get there.
Now, of course, we're also accessing A and C, but turns out to this analysis it sufficient to show that we're getting n cubed misses just on the matrix B.
In order to say hey, we've got a lot of misses here.
So this was the case where n was bigger than the size of a cache.
So the situation is a little bit different if n is large but still actually less than m over b.
So in this case, we suppose it n squared is bigger than m. So the matrix doesn't fit in memory.
So that's what this part of the equation is, m to the 1/2 less than n is the same as n squared is bigger than memory.
So we still don't fit in memory, but in fact it's less than some constant times m over b.
And now let's look at the difference with what happens with the caches as we go through the algorithm.
So we essentially do the same thing.
Once again, we're going to assume LRU.
And so what happens is we're going to go down a single row there.
But now, notice that by the time I get down to the bottom, basically I've accessed fewer than some constant times m over b locations.
And so nothing has gotten kicked out yet.
So when I go back to the top for the next access to B, all these things are still in memory.
So I don't take a cache fall, a cache miss in those cases.
And so we keep going through.
And basically this is much better because we're actually getting to take advantage of the spatial locality.
So this algorithm takes advantage of the spatial locality.
If n is really big it doesn't, but if n is just kind of big, then it does.
And then if n is small enough, of course, it all fits in cache and there's no misses other than those needed to bring it in once.
And then the same thing happens once you go through the next one.
So in this case, what's happening is we have n squared over b misses per run through the matrix B, and then we have n times that we go through.
Once for every row of A.
So the total then is n cubed over b the cache block size.
So depending upon the size, we can analyze with this, that this is better because we get a factor of B improvement.
But it's still not particularly good.
And it only works, of course, if my side of my matrix fits in the number of lines of cache that I have.
Yeah, question
Can you explain in-- I don't understand why you have n cubed over b?
OK, so we're going through this matrix n times.
And for each one of those, we're running through this thing.
So this thing basically, I get to go b times through, because all these things are going to be in memory when I come back to do them again.
And so it's only once every B columns that I take a miss.
I take a miss and then I get to the other b minus 1 access is that I get cache hit.
And so the total here is then n squared over b.
So therefore a total of n cubed over b.
So even this is not very good compared to what we can actually do if we exploit the cache well.
So let's go on and take a look.
We saw this before.
Let's use tiling.
So the idea of tiling is to say, let's break our matrix into blocks of s times s size.
And essentially what we do is we treat our big matrix as if we're doing block matrix multiplications of things of size s by s. So the inner loop here is doing essentially the matrix multiplication.
It's actually matrix multiply and add.
The inner three loops are just doing ordinary matrix multiplication, but on s-sized matrices.
And the outer loop is jumping over matrix by matrix for each of those doing a matrix multiply as its elemental piece.
So this is the tiling solution that you've seen before.
We can analyze it in this model to see, is this a good solution.
So everybody clear on what the code does?
So it's a lot of four loops, right?
Yeah
There should be less than n somewhere?
There's like an and something
Oh yeah.
That must have happened when I coped it.
That should be j less than n here.
It should just follow this pattern.
i less than n, k less than n, that should be j less than n there.
Good catch.
I did execute this.
That must have happened when I was editing.
So here's the analysis of work.
So what's going on in the work?
So here we have, basically the outer loop is going n over s times, each loop.
So there's cube there times the inner loops here which are going each s times.
So times s cubed.
Multiply that through, n cubed operations.
That's kind of what you'd expect.
What about cache misses?
So the whole idea here is that s becomes a tuning parameter.
And whether we choose s well or poorly influences how well this algorithm works.
So the idea here is we want tune s so that the submatrices just fit into cache.
So in this case, if I want a matrix to fit into cache, I want to be about the size of the square root of the cache size.
And this is where we're going to use the tall-cache assumption now.
Because I want to say, it fits in cache, therefore I can just assume it all fits in cache.
It's not like the size fits but the actual data doesn't, which is what happens with the short cache.
So the tall-cache assumption implies that when I'm executing one of these inner loops, what's happening?
When I'm executing one of these linear loops, all of the matrices are going to fit in cache.
So all I have is my cold misses, if any, on that submatrix.
And how many cold misses can I have?
Well the size of the matrix is s squared and I get to bring in b bytes of the matrix each time.
So I get s squared over b misses per submatrix.
So that was a little bit fast, but I just want to make sure-- it's at one level straightforward, and the other level it's a little bit fast.
So the point is that the inner three loops I can analyze if I know that s is fitting in cache.
The inner three loops I can analyze by saying, look it's s squared data.
Once I get the data in cache, if I'm using an optimal replacement, then it's going to stay in there.
And so it will cost me s squared over b misses to bring that matrix in for each of the three matrices.
But once it's in there, I can keep going over and over it as the algorithm does.
I don't get any cache misses.
Because those all fitting in the cache.
Question?
Everybody with me?
OK.
So then I basically have the outer three loops.
And here I don't make any assumptions whatsoever.
There's n over s iterations for each loop.
And there's three loops.
So that's n over s cubed.
And then the cost of the misses in the inner loop is s squared over b.
And that gives me n cubed over bm to the 1/2 if you plug in s being m to the 1/2.
So this is radically better because m is usually big.
Especially for a higher level cache, for an L2 or an L3.
m is really big.
What was the value we had before for the best case for the other algorithm when it didn't fit in matrix?
It was n cubed over b. b is like 64 bytes.
m is like the small L1 cache is 32 kilobytes.
So you get to square root the 32 kilobytes.
What's that?
So that's 32 kilobytes is 2 to the 15th.
So it's 2 to the 7.5.
2 to the 7 is 128.
So it's somewhere between 128 and 256.
So if we said 128, I've got a 64 and a 128 multiplier there.
Much, much better in terms of calculating.
In fact, this is such that if we tune this properly and then we say, well what was the cost of the cache misses here, you're not going to see the cost of the cache misses when you do your performance analysis.
It's all going to be the work.
Because the work is still n cubed.
The work is still n cubed, but now the misses are so infrequent, because we're only getting one every-- on the order of 64 times 128, which is 2 to the 6th times 2 to the 7th is 2 to the 13th is 8K.
Every 8,000 or so accesses there's a constant factor in there or whatever, but every 8,000 or so accesses we're getting a cache miss.
Uh, too bad.
If it's L1, that cost is four cycles rather than one.
Or that cost us 10 cycles if I had to go to L2 rather than one, or whatever.
So is to the point is, that's a great multiplier to have.
So this is a really good algorithm.
And in fact, this is the optimal behavior you can get for matrix multiplication.
Hong and Kung proved back in 1981 that this particular strategy and this bound was the best you could do for matrix multiplication.
So that's great.
I want you to remember this number because we're going to come back to it.
So remember it's b times n to the 1/2, b times square root of m in the denominator.
Now there's one hitch in this story.
And that is, what do I have to do for this algorithm to work well?
It says right up there on the slide.
Tune s. I've got to tune s. How do I do that?
How do I tune s?
How would you suggest we tune s?
Just run a binary [INAUDIBLE]
Yeah, do binary search on s to find out what's the best value for s. Good strategy.
What if we guess wrong?
What happens if, say, we tune s, we get some value for it.
Let's say the value is 100.
So we've turned it.
We find 100 is our best value.
We run it on our workstation, and somebody else has another job running.
What happens then?
That other job starts sharing part of that cache.
So the effective cache size is going to be smaller than what we turned it for.
And what's going to happen?
What's going to happen in that case?
If I've tuned in for a given size and then I actually have to run with something that's effectively a smaller cache, does it matter or doesn't matter?
Is it still tall?
Still tall
[INAUDIBLE]
So if you imagine this fit exactly into cache, and now I only have half that amount.
Then the assumption that these three inner loops is running with only s squared over b misses is going to be totally out the window.
In fact, it's going to be just like the case of the first algorithm, the naive algorithm that I gave.
Because the size of matrix that I'm feeding it, s by s, isn't fitting in the cache.
And so rather than it being s squared over b accesses, it's going to be much bigger.
I'm going to end up with essentially s cubed accesses if the cache, in fact, gets enough smaller.
It's also one thing you have to put in there is what I like to call voodoo.
Whenever you have a program and you've got some parameters that, oh good we've got these parameters we get to tweak to make it go better.
I call those voodoo parameters.
Because typically setting them is not straightforward.
Now there are different strategies.
One, as you say, is to do binary search by doing it.
There are some programs, in fact, which when you start them up you call an initialization routine.
And what they will do is automatically check to see what size is my cache and what's the best size should I do something on and then use that when you actually run it later in the program.
So it does an automatic adaptation automatically when you start.
But the more parameters you get, the more troublesome it becomes.
So let's take a look.
For example, suppose we have a two-level cache rather than a one-level cache.
Now I need to have something that I tune in for L1 and something that I tune for L2.
So it turns out that if I want to optimize s and t, I can't do it in more with binary search because I have two parameters.
And binary search won't suffice for figuring out what's the best combination of s and t. And generally multidimensional searches are much harder than one-dimensional searches for optimizing.
Moreover, here's what the code looks like.
So now I've got, how many four loops?
1,2,3,4,5,6,7,8,9 nested for loops.
So you can see the voodoo is starting to make this stuff run.
You really have to be a magician to tune these things appropriately.
I mean, if you can can do it that's great.
But if you don't do it, OK.
So now what about three levels of cache?
So now we need three tuning parameters.
Here s, t and u, we have 12 nested four loops.
I didn't have the heart to actually write out the code for the 12 nested for loops.
That just seemed overhead.
But our new halo machines, they have three levels of caches.
So let's tune for all the levels of caches.
And as we mentioned, in a multi-program environment, you don't actually know what the cache size is, what other programs are running.
So it's really easy to mistune these parameters
[INAUDIBLE] don't you have a problem because you're running the program for a particular n, and you don't necessarily know whether your program is going to run faster or slower--
Well what you're usually doing, is you're tuning for s not n, right?
So you're assuming--
No, no a particular n
But the tuning of this is only dependent on s. It doesn't depend on n. So if you run it for a sufficiently large n, I think it's reasonable to assume that the s you get would be a good s for any large n. Because the real question is, what's fitting in cache?
Yeah--
How long does it take to fill up the cache relative to the context each time?
Generally you can do it pretty quickly
Right.
So why does it matter if your have multiple users, if you can fill it [INAUDIBLE]
No because, he may not be using all of the cache, right?
So when you come back, you're going to have it polluted with a certain amount of stuff.
I think it's a good question
[INAUDIBLE]
OK, so anyway, so this is the-- yeah question
So if n is really large, is it possible that the second row of the matrix never loaded?
If n is really large--
Because n is really large, right?
Just the first row of the matrix will fill up all the caches
It's LRU and in B you're going down this way.
You're accessing things going down.
OK, good.
So let's look at a solution to these alternatives.
What I want to in particular take a look at is recursive matrix multiplication.
So the idea is you can do divide and conquer on multiplying matrices because if I divide each of these into four pieces, then essentially I have 8 multiply adds of n over 2 by n over 2 matrices.
Because I basically do these eight multiplies each going into the correct result.
So multiply A11 B11 and add it into C11.
Multiply A12 B21 add it into C11 and so forth.
So I can basically do divide and conquer.
And then each of those I recursively divide and conquer.
So what's the intuition by why this might a good scheme to use?
[INAUDIBLE]
Well, we're not going to do parallel yet.
Just why is this going to use the cache well?
[INAUDIBLE]
Yeah eventually I get down to a size where the matrix that I'm working on fits into cache, and then all the rest of the operations I do are all going to be cache hits.
It is taking something and it it's doing what the tiling is doing but doing it blindly.
So let's take a look.
Here's the recursive code.
So here I have the base case if n is 1, I basically have a one by one matrix and I just simply update c, with a times b.
And otherwise what I do, is I'm going to do this by computing offsets.
So generally when you're dealing with matrices, especially if you want fast code, I usually don't rely on two-dimensional addressing, but rather do the addressing myself and rely on the compiler to do common subexpression elimination.
So, for example, here what I'm going to do is compute the offsets.
So here's how I do it.
So first of all, in practice what you do, is you don't go down to n equals 1.
You have some cutoff.
Maybe n is 8 or something.
And at that point you go into a specialized routine that does a really good 8 by 8 multiply.
And the reason for that is you don't want to have the function call overheads.
This function call is expensive to do two floating point operations here.
So you'd like to have a function call and then do 100 floating point operations or something.
So that you get a better balance.
Do people understand that?
So normally to write recursive codes you want a course in the recursion.
Make it so you're not going all go the into way down to n equals 1.
But rather are stopping short and then doing something that doesn't involve a lot of overhead in the base case of your recursion.
But here I'll explain it as if we went all the way down to n equals 1.
So then what we do is, if this is a submatrix, which is basically what I'm showing here.
We have an n by n submatrix.
And it's being pulled out on a matrix of size row size, of width row size.
So what I can do is, if I want to know where the elements of the beginning of matrices are, well the first one is exactly the same place that the input matrix is.
The second one is basically I have to add n over 2 to the location in the array.
The third one here, 21, I have to basically add n over 2 rows to get the starting point of that matrix.
And for the last one I have to add n over 2 and n over 2 rows and n over 2 plus 1 rows to get to that point.
So I compute those and now I can recursively multiply with sizes of n over 2 and perform the program recursively.
Yeah--
So you said it rightly.
You're blindly dividing the matrix up til you get something to fit the cache.
So essentially--
Well and you're continuing.
The algorithm is completely blind all the way down to n equals 1
This could never be better if the other one-- your computer's version is well-tuned.
Because the applications are the same, but this one you have all the overhead from the [INAUDIBLE]
Could be
At the end, you still need to make a multiplication and then go back and look at all of the--
Could be.
So let's discuss that later at the end when we talk about the differences between the algorithms.
Let's at this point, just try to understand what's going on in the algorithm.
Question--
[INAUDIBLE]
n over 2 times row size plus-- plus n over 2.
It should be row size plus 1.
You're right.
Good, bug.
Should be n over 2 times row size plus 1.
So let's analyze the work assuming the code actually did work.
So the work we can write a recurrence for.
So here we have the work to solve an n by n matrix problem.
Well if n is 1, then it's just order one work-- constant amount of work.
But if n is bigger than 1, then I'm solving eight problems of size n over 2, plus doing a constant amount of work to divide all those up.
So everybody understand where I get this recurrence?
Now normally, as you know, when you do algorithmic work, we usually omit this first line because we assume a base case of constant if it's one.
I'm actually going to keep it.
And the reason is because when we do caching the basic cases are important.
So everybody understand where this recurrence came from?
So I can use the master theorem or something like that to solve this.
In which case the answer for this is what?
Those of you who have the master theorem in your hip pocket.
What's the solution of this recurrence?
People remember?
Who's has heard of the master theorem?
I thought that was kind of a prerequisite or something of this class, right?
So you might want to brush up on the master theorem for the quiz next week.
So basically it's a n over b, so it's n to the log base 2 of 8.
So that's n cubed, n to the log base 2 of 8 is n to the n cubed.
And that's bigger than the order one here, so the answer is order n cubed.
Which is a relief, right?
Because if weren't order n cubed we would be doing a lot more work than one of the looping algorithms.
However, let's actually go through and understand where that n cubed comes from.
And to do that I'm going to use the technique of a recursive tree, which I think all of you have seen.
But let me go through it slowly here to make sure, because we're going to do it again when we do cache misses and it's going to be more complicated.
So here's the idea.
I write down the left hand side the recurrence, w of n. And now what I do is I substitute, and I draw it out as a tree.
I have eight problems of size n over 2.
So what I do is I replace that with the thing that's on the right hand side, I've dropped the theta here, but basically put just a constant one here.
Because I'll take into account the thetas at the end.
So I have a one here, and then I have, loops that should be a w. Should be w n over 2.
That's a bug there.
And then I replace each of those.
OK, wn over 2, sorry that should be wn over 4.
Ah, more bugs.
I'll fix them up on after lecture.
So this should be w of n over 4.
And we go all the way down to the bottom to where I hit the base case of theta 1.
So I built out this big tree that represents, if you think about it, that's exactly what the algorithm is going to do.
It's going to walk this tree doing the work.
And what I've just simply put up here is to work it does at every level.
So the first thing we want to do is figure out what's the height of this tree.
Can somebody tell me what the height of the tree is?
It is a log n. What's the base?
Log base 2 of n, base because at every level if I hadn't made a mistake here, I'm actually having the argument.
So I'm having the argument at each level.
So the height is log base 2 of n. So LG is notation for log base 2.
So if I have log base 2 of n, I can count how many leaves there are to this tree.
So how many leaves are there?
Well I'm branching a factor of eight at every level.
And if I'm going log base 2 levels, the number of leaves is 8 to the log base 2.
So 8 to the log base 2 of n. And then with a little bit of algebraic magic that turns out that's the same as n to the log base 2 of 8.
And that is equal to n cubed.
So I end up with n cubed leaves.
Now let's add up all the work that's in here.
So what I do is I add across the rows.
So the top level I've got work of one.
The next level I work of eight.
The next I have work of 64.
Do people see the pattern?
The work is growing how?
Geometrically.
And at this level I know that if I add up all the leaves I've got work of n cubed.
Because I've got n cubed leaves, each of them taking a constant.
And so this is geometrically increasing, which means that it's all born in the leaves.
So the total work is order n cubed.
And that's nice.
It's the same work is the looping versions.
Because we don't want to increase that.
Questions?
Because now we're going to do cache misses and it's going to get hairy, not too hairy, but hairier.
So here we're going to cache misses.
So the first thing is coming up with a recurrence.
And this is probably the hardest part, except for the other hard part which is solving the recurrence.
So here what we're doing is, we have the same thing is that I'm solving eight problems of size n over 2 and to do the work in here.
I'm taking basically order one cache misses.
However I do, those things work out.
Plus the cache misses I have in there.
But then at some point, when I'm claiming is that I'm going to bottom out the recursion early.
Not when I get to n equals 1, but in fact when n squared is less than some constant times the cache size.
For some sufficiently small concept.
And what I claim, at that point, is that the number of cache misses I'm going to take at that point, I can just, without doing any more recursive stuff, I can just say it's n squared over b.
So where does that come from?
So this basically comes from the tall-cache assumption.
So the idea is that when n squared is less than a constant times the size of your cache, constant times the size of m, then that means that this fits into-- the n by n matrices fit within m. I've got three of them.
I've got C, A and B.
So that's where I need a constant here.
So they're all going to fit in memory.
And so if I look at it, all I have to do is count up the cold misses for bringing in those submatrices at the time that n hits this threshold here of some constant times m. And to bring in those matrices is only going to cost me n squared over b cache misses.
And once I've done that, all of the rest of the recursion that's going on down below is all operating out of cache.
It's not taking any misses if I have an optimal replacement algorithm.
it's not taking any more misses as I get further down.
Questions about this part of the recurrence here?
So people with me?
So when I get down to something of size n squared, where the submatrix is size n squared, the point is that I'll bring in the entire submatrix.
But all the stuff that I have to do in there is never going to get kicked out, because it's small enough that it all fits.
And an optimal algorithm for replacement is going to make sure that stuff stays in there, because there's plenty of room in the cache at that point.
There's room for three matrices in the cache and a couple of other variables that I might need and that's basically it.
Any questions about that?
So let's then solve this recurrence.
So we're going to go about it very much the same way.
We make draw a recursion tree.
So those of you are rusty in drawing recursion trees, I can promise you there will be a recursion tree on the quiz next Thursday.
I think I can promise that.
Can I promise that?
Yeah, OK I can promise that.
The way I like to do it, by the way, is not to try to just brought out all at once.
In my own notes when I do this I always draw it step by step.
I copy over and just do a step by step.
You might think that that's extensive.
Gee, why do I have to draw every one along the way?
Well the answer is, it's a geometric process.
All the ones going up to the last one are a small amount of the work to draw out the last one.
And they help you get it correct the first time.
So let me encourage you to draw out the tree iteration by iteration.
Here I'm going to just do replacement.
So what we do is we replace with the right hand side to do the recursion.
And replace that.
And once again I made the bug, that should be in over 8.
Sorry, n over 4 here.
n over 4.
And then we keep going down until I get to the base case, which is this case here.
Now comes the first hard part.
How tall is this tree?
Yeah--
[INAUDIBLE] square root of n over b.
You want n squared to be cm, not [INAUDIBLE]
So here's the thing, let's discuss, first of all, why this is what it is.
So at the point where n squared is less than cm, that says that it's going to cost us n squared over b.
But n squared is just less than cm, so therefore, this is effectively m over b.
Good question.
So everybody see that?
So when I get down to the bottom, it's basically costing me something that's about the number of lines I have in my cache, number of misses to fill things up.
The tricky thing is, what's the height?
Because this is crucial to getting this kind of calculation right.
So what is the height of this tree?
So I'm having every time.
So one way to think about it is, it's going to be log bas 2 of n, just as before, minus the height of the tree that is hidden here that I didn't have to actually go into because there are no cache misses in it.
So that's going to occur when n is approximately m, cm, sorry when n is approximately square root of cm.
So I end up with log of n minus 1/2 log of cm.
That's the height here.
Because the height at this point of the tree that's missing because they're no cache, I don't have to account for any cache misses in there, is log of cm to the one half, based on this.
Does that follow for everybody?
People comfortable?
Yeah?
OK, good.
So now what do we do?
We count up how many leaves there are.
So the number of leaves is 8, because I have a branching factor of 8, 2 whatever the height is.
Log n minus 1/2 log of cm.
And then if I do my matrix magic, well that part is n cubed, the minus becomes a divide, and now 8 to the 1/2 log of cm is the square root of n cubed, which is m to the 3/2.
Is that good?
The rest of it is very similar to what we did before.
At every level I have a certain number of things that I'm adding up.
And on the bottom level, I take the cost here, m over b, and I multiply it by a number of leaves.
When I do that I get, what?
I get n cubed over b times m to the 1/2.
This is geometric.
So the answer is going, in this case, just going to be the sum of a constant factor times the large thing.
And why does this look familiar?
That was the optimal result we got from tiling.
But where's the tuning parameters?
No tuning parameters.
No tuning parameters.
So that means that this analysis that I did for one level of caching, it applies even if you have three levels of caching.
At every level you're getting near optimal cache behavior.
So it's got the same cache misses as with tiling.
These are called cache-oblivious algorithms.
Because the algorithm itself has no tuning parameters related to cache.
Unlike the tiling algorithm.
That's a cache-aware algorithm.
The cache-oblivious algorithm has no tuning parameters.
And if it's an efficient one.
So, by the way, our first algorithm was cache-oblivious as well.
The naive one.
It's just not efficient.
So in this case we have an efficient one.
It's got no voodoo turning of parameters, no explicit knowledge of caches, and it passively autotunes itself.
As it goes down when it fits things into cache it fits them and uses things locally.
And then it goes down and it fits into the next level of cache and uses things locally and so forth.
It handles multi-level caches automatically.
And it's good in multi-programmed environments.
Because if you end up taking away some of the cache it doesn't matter.
It still will end up using whatever cache is available nearly as well as any other program could use that cache.
So these are very good in multi-programmed environments.
The best cache-oblivious matrix multiplication, in fact doesn't do an eight way split as I described here.
That was easier to analyze and so forth.
The best one that I know work on arbitrary rectangular matrix.
And what they do, is they do binary splitting.
So you would take your matrix, i times j, So if you take a matrix, let's say it's something like this.
So here we have i, k, k, j.
And you're going to get something of shape.
i times j, right?
What it does, is it takes whatever is the largest dimension.
In this case k is the largest dimension.
And it partitions either one or both of the matrices along k. In this case, it doesn't do that.
And then it recursively solves the two sub-rectangular problems.
And that ends up being a very, very efficient fast code if you code that up tightly.
So it does binary splitting rather than-- and it's general.
And if you analyze this, it's got the same behavior as the eight way division.
It's just more efficient.
So questions?
We had a question about now comparing with the tiled algorithm.
Do you want to reprise your question?
What I was saying was, I guess this answers my question.
If you were to tune the previous algorithm properly, and you're assuming it's not in a multi-program environment, the recursive one, it will never be the one that is locked.
[INAUDIBLE]
So at some level that's true, and at some level it's not true.
So it is true in that if it's cache-oblivious you can't take advantage of all the corner cases that you would might be able to take advantage of in a tiling algorithm.
So from that point of view, that's true.
On the other hand, these algorithms work even as you go into paging and disks and so forth.
And the interesting thing about a disk, if you start having a big problem that doesn't fit in memory and, in fact, is out of core as they call it, and is paging to disk, is that the disk sizes of sectors that can be brought efficiently off of a disk, vary.
And the reason is because in a disk, if you read a track around the outside you can get two or three times as much data off the disk as a track that you read near the inside.
So the head moves in and out of the disk like this.
It's typically on a pivot and pivots in and out.
If it's reading towards the inside, you get blocks that are small versus blocks that are large.
This is effectively a cache line size that gets brought in.
And so the thing is that there are actually programs in which, when you run them on disk, there is no fixed size tuning parameter that beats the cache-oblivious one.
So the cache-oblivious one will beat every fixed-size tuning parameters you put in.
Because you don't have any control over where your file got laid out on disk and how much it's bringing in and how much it isn't varies.
On the other hand, for in-core thing, you're exactly right.
That, in principle, you could tune it up more if you make it more cache aware.
But then, of course, you suffer from portability loss and from, if you're in a multi-programmed environment and so forth.
So the answer is, that there are situations where you're doing some kind of embedded or dedicated type of application, you can take advantage of a lot of things that you want.
There are other times where you're doing a multi-programmed environment, or where you want to be able to move something from one platform to another without having to re-engineer all of the tuning and testing.
In which case it's better to use the cache oblivious.
So as I mentioned, my view of these things is that performance is like a currency.
It's a universal medium of exchange.
So one place you might want to pay a little bit of performance is to make it so it's very portable as for the cache-oblivious stuff.
So you get nearly good performance, but now I don't have that headache to worry about.
And then sometimes, in fact, it actually does as well or better than the one.
For matrix multiplication, the best algorithms are the cache oblivious ones that I'm aware of
[INAUDIBLE] currency and all the different currencies.
Single currency
You want a currency for-- so in fact the performance for this is people who have engineered it to take advantage of exactly the cache size, we can do just as well with the cache oblivious one.
And particularly, if you think about it, when you've got three levels hierarchy, you've got 12 loops.
And now you're going to tune that.
It's hard to get it all right.
So next time we're going to see a bunch of other examples of cache-oblivious algorithms that are optimal in terms of their use of cache.
Of course, by the way, those people who are familiar with Strassen's algorithm, that's a cache-oblivious algorithm.
Takes advantage the same kind of thing.
And in fact you can analyze it and come up with good bounds on performance for Strassen's algorithm just the same.
Just as we've done here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so I'm sure everybody's kind of curious about Project 2.2 Beta, so here's the preliminary performance results.
There's still a bit more work to do on finalizing these numbers, but here's how they look.
But before I show you guys that, I'd like to yell at you guys for a little bit.
So this is a timeline of the submission deadline and when people submitted things.
So from this zone to this zone is an hour, and I see like 50% of the commits made during that period.
So a little bit of background, the submission checking system automatically clones a repository up to the deadline and not a second after.
So if you look at from this to this, some people took quite a jump back.
So just as a warning, please try to submit things on time.
11:59 means 11:59, and to drive that point further, this is an example of a commit that I saw in somebody's repository, who's name I blanked out.
Obviously, it's seven seconds past the deadline, so the automatic repository cloner didn't grab it.
And the previous commit to that was like 10 days ago when Reid pushed out pentominoes grades.
So don't do that either.
A little break down on how often people commit.
About a quarter of the class only made one commit to their repository.
Half of you guys did three to 10 commits, which seems to be an up.
Under 21+, there was somebody who did 100 some commits, which was pretty impressive.
Yeah, very smart dude.
Committing often is a good idea, so that you don't run into a situation like before.
And next time definitely, there is going to be next to zero tolerance for people who don't commit things on time for the deadlines.
Now, the numbers that people want.
So for rotate, just as an interesting data point, this is the 512 by 512 case.
It seems like not everybody remembered to carry over their optimizations for the 512 case over to the final, otherwise you would expect the numbers to be a little bit more similar and not off by a factor of eight.
And for rotate overall, that's the distribution.
The speed up factor is normalized to some constant that gives everybody a reasonable number.
And there were a lot of groups that had code that didn't build.
Yes?
Does the speedup only include the rotate dot 64
Yes.
Performance was only tested on the rotate dot 64, which is what we said in the handout also.
So there were a lot of groups that didn't build, which really surprised me, and I think it's probably because half of you guys pushed things after the deadline, and I presume those things contained important commits towards making your code actually work.
But in this case, there was no header files, there was no cross-testing.
All we did was run your make file on your code, and we replaced your test bed dot c, and your k timing, so I'm not quite sure why people have code that doesn't build.
For sort, this was the maximum size input that we allowed you guys to run.
One group did really well.
So all of these are correct.
The correctness test is built in, and I replaced that with a clean copy that contains a couple additional checks, by the way.
So I tried the other extreme, a relatively small array
Is the top the same person?
I'm not sure whether or not the top is the same person.
But the distribution wise seems like people didn't quite remember to optimize for the smallest case.
And this is the current overall speedup factors for sort averaging.
We did about 10 to 15 cases for rotate and sort.
And that's the overall speedup distribution
OK, so now you're done with individual project, so you already did the last project individually, and then we are moving into, again, a group project.
So the first thing we have, setting up automated system for you to say who your group members are.
So we will send you information, and with that, what you have to do is run a script saying who your group members are, both group members have to do it, and then we will basically clear that account in there.
That said, a lot of you didn't know, in the first project, how to work and what's the right mode of operations with the group.
OK if we gave you to write 100,000 lines of code, it makes sense to say, OK, I'm going to divide the problem into half, one person do one, the other person do the other half.
The reason for doing the group is to try to get you to do pair programming, because talking to a lot of you, getting a lot of feedback, it looks like most of you spent a huge amount of time debugging.
And since you're only writing a little amount of code, it makes a lot more sense to sit with your partner next to the screen, one person types, other person looks over, and then you have a much faster way of getting through the debugging process.
So the next one, don't try to divide the problem by half.
Just try to find some time, sit with each other.
Then the other really disturbing thing is there have been a couple of groups that were completely dysfunctional.
We get emails saying, OK, my group member didn't talk to me, or they didn't do any work, or they were very condescending.
And that's really sad, because from my experience with MIT students, when you guys go to company, you guys probably will be the best programmers there.
There's no question about it.
I have seen that.
To the point that some people might even resent it, to have this best programmer.
But what I have seen is the fact a lot of you cannot work in the group, if you haven't developed that skill, you will not be the most impactful person.
I have seen that again and again in my experience with doing a start-up.
Our MIT students, their way of impacting is put all night do the entire project by themselves.
Doable when you're doing a small change to a large project, but if you want to have a big change, you can't do that.
You have to work with the group, figure out how to impact, how to communicate.
This is more important learning than, say, trying to figure out how you can optimize something.
So being individual contributors and able to do amazing things is important, but the fact that you can't work with a group is going to really make the impact you can make much less.
So please, please learn how to work with your group members.
Some of them might not be as good as you are, and that will be probably true in real life, too, but that doesn't mean you can be condescending towards them, make them feel inferior.
That doesn't cut it.
You have to learn how to work with these people.
So part of your learning is working with others, and that's a large part of your learning.
Don't consider that to be this external thing, even though you might think you can do a better job.
Just work with the other person, especially in pair programming, because where both of you are sticking together, it's much easier.
Because there are four eyes on your project, they might see something you don't see, and see whether you can work together like that.
And I don't want to hear any more stories saying, look, my partner was too dumb, or my partner didn't show up, he couldn't deal with that.
Those are not great excuses.
And so we wonder.
Some of them, we will pay attention to it, because it might be one person's unilateral actions that lead to that.
Still, please, try to figure out how we can work with your partners.
And hope you have a good partner experience.
Use pair programming.
Use a lot of good debugging techniques, and the next project will be fine
Great.
We're going to talk more about caches.
Whoo-whoo!
OK.
So for those who weren't here last time, we talked about the ideal cache model.
As you recall, it has a two-level hierarchy and a cache size M bytes, and a cache line length of B bytes.
It's fully associative and is optimal omniscient replacement strategy.
However, we also learned that LRU was a good substitute, and that any of the asymptotic results that you can get with optimal, you could also get with LRU.
The two performance measures that we talked about were the work, which deals with what the processor ends up doing, and the cache misses, which is the transfers between cache and main memory.
You only have to count one direction, because what goes in basically goes out, so more or less, it's the same number.
OK, so I'd like to start today by talking about some very basic algorithms that you have seen in your algorithms and data structures class, but which may look new when we start taking caches into account.
So the first one here is the problem of merging two sorted arrays.
So as you recall, you can basically do this in linear time.
The way that the algorithm works is that it looks at the first element of the two arrays to be sorted and whichever is smaller, it puts in the output.
And then it advances the pointer to the next element, and then whichever is smaller, it puts in the output.
And in every step, it's doing just a constant amount of work, there are n items, so by the time this process is done, we basically spent time proportional to the number of items in the output list.
So this should be fairly familiar.
So the time to emerge n elements is order n. Now, the reason merging is useful is because you can use it in a sorting algorithm, a merge sorting algorithm.
The way merge sort works is it essentially does divide and conquer on the array.
It divides the array into two pieces, and it divides those each into two pieces, and those into two, until it gets down to something of unit size.
And then what it does is it merges the two pairs of arrays.
So for example here, the 19 and 3 got merged together to be 3 and 19.
The 12 and 46 were already in order, but you still had to do work to get it there and so forth.
So it puts everything in order in pairs, and then for each of those, it puts it together into fours, and for each of those, it puts it together into the final list.
Now of course, the way it does this is not in the order I showed you.
It actually goes down and does a walk of this tree.
But conceptually, you can see that it essentially comes down to merging pairs, merging quadruples, emerging octuples, and so forth, all the way until the program is done.
So to calculate the work of Merge Sort, this is something you've seen before because it's exactly what you do in your algorithms class.
You get a recurrence that says that the work, in this case is-- well, if you have only one element, it's a constant amount of work, and otherwise, I solve two problems of half the size doing order n work, which is the time to merge the two elements.
So classic divide and conquer.
And I'm sure you're familiar with what the solution to this recurrence is.
What's the solution to this recurrence?
n log n. I want to, though, step through it, just to get everybody warmed up to the way that I want to solve recurrences so that you are in a position, when we do the caching analysis-- we have a common framework for understanding how the caching analysis will work.
So we're going to solve this recurrence, and if the base case is constant, we usually omit it.
It's assumed.
So we start out with W of n, and what we do is we replace it by the right hand side, where we put the constant term on the top and then the two children.
So here I've gotten rid of the theta, because conceptually when I'm done, I can put a big theta around the whole thing, around the whole tree, and it just makes the math a little bit easier and a little bit clearer to see what's going on.
So then I take each of those and I split those, and this time I've got n over 4.
Correct?
I checked for that one this time.
It's funny because it was still there, actually, just a few minutes before class as I was going through.
And we keep doing that until we get down to something of size one, until the recurrence bottoms out.
So when you look at a recursion tree of this nature, the first thing that you typically want to do is take a look at what the height of the tree is.
In this case, we're taking a problem of size n, and we're halving it at every step.
And so the number of times we have to halve the argument-- which turns out to be also equal to the work, but that's just coincidence-- is log n times.
So the height is log base 2 of n. Now what we do typically is we add things up across the rows, across the levels.
So on the top level, we have n. On the next level, we have n. The next level, hey, n. To add up the bottom, just to make sure, we have to count up how many leaves there are, and the number of leaves, since this is a binary tree, is just 2 to the height.
So it's 2 to the log n, which is n. So then I add across all the leaves, and I get the order 1 at the base times n, which is order n. And so now I'm in a position to add up the total work, which is basically log n levels of n is total of order n log n. So hopefully, this is all review.
Hopefully this is all review.
You haven't seen this before.
It's really neat, isn't it?
But you've missed something along the way.
So now with caching.
So the first thing to observe is that merge subroutine, the number of cache misses that it has is order n over B.
So as you're going through, these arrays are laid out continuously in memory.
The number of misses-- you're just going through the data once-- is order n data, all going through contiguously.
And so every time you bring in data, you get full spatial locality, there are n elements, it costs n over B.
So is that plain?
Hopefully that part's plain, because you bring in things of each axis, you get the same factor of B.
So now merge sort-- and this is, once again, the hard part is coming up with a recurrence, and then the other hard part is solving it.
So there's two hard parts to recurrences.
OK, so the merge sort algorithm solves two problems of size n over 2, and then does a merge.
So the second line here is pretty straightforward.
I take the cache misses that I need to do, and I take a merge.
I may have a few other accesses in there, but they're going to be dominated by the merge.
OK, so it's still going to be theta n over B.
Now the hard part, generally, of dealing with cache analysis is how to do the base case, because the base case is more complicated than when you just do running time, you get that to run down to a base case of constant size.
Here, you don't get to run down to base case of constant size.
So what it says here is that we're going to run down until I have a sorting problem that fits in cache, n is going to be less than some constant times n, for some sufficiently small constant c less than 1.
When in finally fits in cache, how many cache misses does it take me to sort it?
Well, I only need to have the cold misses to bring that array into memory, and so that's just proportional to n over B, because for all the rest of the levels of merging, you're inside the cache.
That make sense?
So that's where we get this recurrence.
This is always a tricky thing to figure out how to write that recurrence for a given thing.
Then, as I say, the other tricky thing is how do you solve it?
But we're going to solve it essentially the same way as we did before.
I'm not going to go through all the steps here, except to just elaborate.
So what we're doing is we're taking, and we have n over B at the top, and then we divide it into two problems, and for each of those-- whoops, there's a c there that doesn't belong.
It should just be n over 2B on both, and then n over 4B, and so forth.
And we keep going down until we get to our base case.
Now in our base case, what I claim is that when I hit this base case, it's going to be the case that n is, in fact, a constant factor times n, so that n over B is almost the same as m over B.
And the reason is because before I hit the base case, when I was at size twice n, and that was bigger than m. So if twice n is bigger than m-- than my constant times m-- but n is smaller than m, then it's the case that n and m are essentially the same size to within a constant factor, within a factor of two, in fact.
And so therefore, here I can say that it's order m over B.
And now the question is, how many levels did I have to go down cutting things in half before I got to something of size m over B?
Well, the way that I usually think about this is-- you can do it by taking the difference as I did before.
The height of the whole tree is going to be log base 2 of n, and the height of this is basically log of-- what's the size of n?
It's going to be log of the size of n when this occurs.
Well, n at that point is something like cm.
So it's basically log n minus log cm, which is basically log of n over cm.
How about some questions.
Yeah, question
--the reason for why you just substituted on the left, [INAUDIBLE] over B, but on the right [INAUDIBLE]
Here?
No
Or you mean here?
[INAUDIBLE]
On the right side
On the right-most leaf, it's n over B.
Is that all of the leaves added up because [INAUDIBLE]?
No, no, no, this is going to be all of the leaves added up here.
This is the stack I have on the right hand side.
So we'll get there.
So the point is, the number of leaves is 2 to this-- so that's just n over cm-- times m over B.
Well, n over cm times m over B, the m's cancel, and I get essentially n over b with whatever that constant is here.
And so now I have n over b across every level, and then when I add those up, I have to go log of n over cm levels, which is the same as log of n over m. So I have n over B times log of n over m. Yeah, question?
Initial assumption that c is some sufficiently small number, so 1 over c would be a rather large factor
It could potentially be a large factor, but it's a constant.
In other words, it can't vary with n. So in fact, typically for something like merge sort, the constant is going to be very close to-- for most things, the constant there is typically only a few.
Because the question is, how many other things do you need in order to make sure it fits in?
In this case, you have n. Well, you have both the input and the output, so here, it's going to be, you have to fit both the input and the output into cache in order not to have the cache it.
So it's basically going to be like a factor of 2 for merge sort.
For the matrix multiplication, it was like a factor of three.
So generally a fairly small number.
Question?
I guess that makes sense.
But I [INAUDIBLE].
So on the order of the size of the leaves, can you assume that n is more than cm, so you can substitute--
Yeah, because basically, when it hits this condition--
Right.
I understand that.
Then you're also saying, why isn't there more or less just one B, because there's n over cm, n is the same as cm.
That's [INAUDIBLE].
At the bottom level, there should be--
Oh, did I do something wrong here?
Number of leaves is--
[INAUDIBLE]
Right, right.
Sorry.
This is the n at the top.
You always have to be careful here.
So this is the n in this case, so this is some n, little n. So this is not the same n. This is the n that we had at the top.
This is the notion of recurrences, like the n keeps recurring, but you know which ones-- so it can be confusing, because if you're-- so yeah.
So this is not the n that's here.
This is the n that started out at the top.
So we're analyzing it in terms of the n. Some people write these things where they would write this in terms of k, and then analyze it for n, and for some people, that can be helpful to do, to disambiguate the two things.
I always find it wastes a variable, and you know, those variables are hard to come by.
There's only a finite number of them.
OK, so are we good for this?
So here, we ended up with n over b, log of n over m cache misses.
So how does that compare?
Let's just do a little thinking about this.
Here's the recurrence, and I solved it out to this.
So let's just look to see what this means.
If I have a really big n, much bigger than the size of my cache, then I'm going to do a factor of B log n less misses than work, because the work is n over B log n. So if n compared to m-- let's say n was m squared or something-- then n over m would still be n, if n was as big as m squared.
So if n was as big as m squared, this log of n over m would still be log n. And so I would basically have n over B log n for a factor of B log n less misses than work.
If they're about the same-- did I get this right?
If they're about the same size, if n is approximately m, maybe it's just a little bit bigger, then the log here disappears completely, and so I basically just have n over B misses
[INAUDIBLE]
Yeah, but if n is like--
[INAUDIBLE]
Yeah.
In fact, for this, you have to be careful as you get to the base cases.
Technically, for some of this, I should be saying 1 plus log of n over m, and in some of the things I do later, I will put in the ones.
But if you're looking at it asymptotically and n gets big, you don't have to worry about those cases.
That just handles the case whether you're looking at n getting large or whether you're trying to handle a formula for all n, even if and n is small.
Question?
[INAUDIBLE]?
The work was n log n, yes.
The work was n log n. So here we basically have n over B log n, so I'm saving a factor of B in the case where it's about the same.
Did I get this right?
I'm just looking at this, and now I'm trying to reverse engineer what my argument is.
So we're looking at n log n versus n over B log n over m
[INAUDIBLE PHRASE].
So that you get a factor of B less misses, because you would be getting n over B like log of n, that's the only way you're getting a factor of B less misses.
So I don't understand how you're saying, for n more or less equal to m. You would want something more like, for n--
Well, if n and m are about the same size, the number of cache misses is just n over B.
And the number of cache misses is n over B, and the work is n log n. So I've saved a factor of B times log n, OK?
What did I say?
[INAUDIBLE]
B log m?
No, I was saying that's for the case when n is much bigger than m. So let's take a look at the case-- let me just do it on the board here.
Let's suppose that n is like m squared, just as an example, big number.
So I'm going to look at, essentially, n over B times log of n over m. So log of n over m, so what is n over m is about m, which is about square root of n, right?
So this basically ends up being approximately n over B log of square root of n, which is the same as log n, to within a constant factor.
I'm going to leave out the constant factors here.
Then I want to compare that with n log n. So I get a factor of B less misses.
So the first one, yes, OK.
So I get a factor of B less misses, you're right.
Then I get a factor of B less misses.
So I think I've got these switched.
So this is the case I'm doing is for n much bigger than m. So let's do the other case.
I think I've got the two things switched.
I'll fix it in the notes.
If n and m are approximately the same, then the log is a constant, right?
So this ends up being approximately n over B.
And now when I take a look at the difference between the number of things, I get B log n. So I've got the two things mixed.
Yeah?
As n approaches m, then the log would approach zero, but were you talking about how it technically should be--
1 plus n, yes
So technically, that approaches one when the log approaches zero
Yeah
These things are really hard for me, because they are really arbitrary.
And then you're like, oh yeah, you can just put a 1 on top of there.
And for example, I always miss those, because I usually try to do the math as rigorously as I can, and those ones generally do not appear, and you're like, oh, sure whatever.
So how am I supposed to know that the log is actually not going to be zero, and I'm going to be like, yeah, you're not going to do any caches
Because generally, what we're doing is we're looking at how things scale, so we're generally looking at n being big, in which case it doesn't matter.
These things only matter if n's-- for example, notice here that if n goes less than m, we're in real trouble, right?
Because now the log is negative.
Wait, what does that mean?
Well, the answer is the analysis was assuming that n was sufficiently large compared with m
Why can't you just be like, oh, when n is for less than one, you can assume, well, n is 2n.
In that case, you get log of two, which is still something or other
Yeah, exactly.
So what happens in these things is if you get right on the cusp of fitting in memory, then these analyses like, well, what exactly is the answer, is dicey.
But if you assume that it doesn't fit in, what's going to happen?
Or that does fit in, what is going to happen?
And then the analysis right on the edge is somewhere between there.
Good.
So I switched these.
I said this the other way around.
That's funny.
I went through this, and then in my notes, I had them switched, and I said, oh my gosh, I did this wrong.
And I've just gone through it, and it turns out I was right in my notes.
Now, one of the things, if you look at what's going on-- let's just go back to this picture here.
What's going on here is each one of the passes that we're doing to do a merge is basically taking n over B misses to do a binary merge.
We're going through all the data to merge just two things, and traversing all the data.
So you can imagine, what would happen if I did, say, a four-way merge?
With a four-way merge, I could actually merge four things with only a little bit more than n over B misses.
In fact, that's what we're going to analyze in general.
So the idea is that we can improve our cache efficiency by doing multi-way merging.
So the idea here is, let's merge R, which is, let's say, less than n subarrays with a tournament.
So here are R subarrays, and here's they're each, let's say, is of size n over R. And what we're going to do is merge them with a tournament, so this is a tournament where we say, who's the winner of these two, who's the winner of these two, et cetera.
And then whoever wins at the top here, we take them and put them in the output, and then we repeat the tournament.
Now let's just look what happens.
It takes order R work to produce the forced output.
So we got R things here.
To playoff this tournament, there are R nodes here.
They each have to do a constant amount of comparing before I end up with a single value to put in the output.
So it costs me R to get this thing warmed up.
But once I find the winner, and I remove the winner whatever chain he might have come along, how quickly can I repopulate the tournament with the next guy?
The next guy only has to play the tournament on the path that was there.
All the other matches, we know who won.
So the second guy only cost me log R to produce the next guy.
And the next guy is log R, and so once we get going, to do an R-way merge only costs me log R work per element.
So the total work in merging is R, to get started, plus n log R. Well, R is less than n, so that's just n log R total to do the merging.
That's the work.
Now, let's take a look at what happens if I now do merge sort with R-way merges.
So what I do is if I have only one element, then it's going to cost me order one time to merge it, because there's nothing to do, just put it in the output.
Otherwise, I've got R problems of size n over R that I'm going to merge, and my merge takes n log R time to do the merge.
So if I look at the recursion tree, I have n log R here starting here, then I branch R ways, and then I have n over R log R to do the R-way branching at the next level, n over R squared log R at the next level, et cetera
You said that the cost of processing is
--is n log
But is--
Upfront, there's an order R cost, but the order R cost is dominated by the n log R, so we don't have to count that separately.
We just have to worry about this guy.
So as I go through here, I basically end up having a tree which is only log base R of n tall, because I'm dividing things into R pieces each time, rather than into two pieces.
So I only go log base R steps till I get to the base case.
But I'm doing an R-way merge, so the number of leaves is still n. But now when I add across here, I get n times log R, and I go across here, I get n times log R, because I got R copies of the same thing.
Now I've got R squared times n over R squared log R, and so forth.
And so at every level, I have n log R. So I have n log R times the number of levels here, which is log base R of n, plus the order n work at the bottom, which we can ignore because it's going to be dominated.
And so what you notice here is, what's log base R of n?
That's just log n over log R. So the log Rs cancel, and I get n log n plus n, which is just n log n. So after all that work, we still do the same amount of work, whether I do binary merging or R-way merging, the work is the same.
But there's kind of a big difference when it comes to caching.
So it's the same work as binary merge sort.
So let's take a look at the caching.
So let's assume that my tournament fits in the cache.
So I want to make sure that R is less than m over B for some constant R. So when I do constant way, when I consider the R-way merging of contiguous arrays of total size n, the entire tournament plus 1 block from each array can fit in cache.
So the tournament is never going to be responsible for generating cache misses, because I'm going to leave the tournament in memory.
So if I'm the optimal algorithm, it's going to say, let's just leave the tournament in memory and bring in all the other things as we do the operation.
Question?
[INAUDIBLE]
Those circles that I had, the tree
Is that a cumulative list of the elements that you've merged in already?
I'm sorry.
Is the--
Is it a cumulative list of the length arrays that you've merged already?
No, no, no.
You haven't merged them.
Let's just go back and make sure we understand the algorithm.
The algorithm says that what we do is we compare the head of this pair and we produce a single value here, for which whatever is-- because these are already sorted to do the merge.
These are already sorted.
So I just have the minimum of these two here, and the minimum of these two here, and the minimum of all four of them here.
So it's repeated.
When we get to the top, we have now the minimum of all of these guys, and that's the guy that's the minimum overall, we put him in the output array.
And now we walk back down the path that he came from, and what we do is we say, oh, this guy had a-- let's walk down the path, let's say we get to this guy.
Let's advance the pointer in here and bring out another element.
And now we play off the tournament here, play off the guy here, and he advances, and he advances, whatever.
And now some other path may be the minimum.
But it only took me log n work.
I'm only keeping copies of the element, if you will, or the results of the comparisons along this in the tree here.
And that tree, we're saying, fits in the cache, plus 1 block, the first block.
Whatever cache block fits in each of these arrays, no matter how much we've gone down, one of those is fitting in memory.
So then what happens here is the entire tournament plus one block from each memory can fit in cache, and so therefore the number of cache misses that I'm going to have when I do the merge it's just essentially the time to take faults on that one cache block whenever I exceed it in each array, plus the one for the output that's similar.
And so the total number of cache misses that I'm going to have is going to be n over B, because I'm just striding straight through memory, and that tournament, I don't have to worry about, because it's sitting in cache.
And there's enough sitting in cache that all the other stuff, I can just keep one block from each of them in memory and still expect to get it.
In fact, you need the tall cache assumption to assume that they all fit in memory.
So therefore, the R-way merge sort is then, if it's sufficiently small, once again, we have the case that it fits in memory so I only have the cold misses to get there, n over B, if n is less than cm.
And otherwise, it's R copies of the number of cache misses for n over R, plus n over B.
Because this is what it took us here to do the merge.
We get only n over B faults when we merge, as long as the tournament fits in cache.
If the tournament doesn't fit in cache, it's a more complicated analysis
--n over B, that's cold misses.
You're getting the stuff--
Yeah, basically, it's the cold misses on the data, yes, basically.
Good.
So now, let's do the recursion tree for this.
So we basically have n over B that we're going to pay at every level, dividing by R, et cetera, down to the point where things fit in cache.
And by the time it fits in cache, it's going to be m over B, because n will be approximately m, just as we had before when we were doing the binary case.
As soon as the subarray completely fits in memory, I don't have to when I'm doing the sorting.
So this is now analyzing not the merging, this is analyzing the sorting now.
This is the sorting, not the merging.
So we get down to m over B, and I've gone now log base R, not log base 2 as we did before, but log base R n over cm.
The number of leaves is n over cm, and so when I multiply this out, I get the same n over B here, and I've got n over B at every level here.
So where's the win?
The win is that I have only log base R of n over cm, rather than log base 2 levels in the tree, because the amount that every level cost me was the same, asymptotically.
So when I add it up, I get n over B log base R of n over m, instead of n over B log base 2 of n over m. So how do we tune R?
Well if we just look at this formula here, if I want to tune R, what should I do to R to make this be as small as possible?
Make it as big as possible.
But I had to assume that R was less than some constant times m so that it fits in cache.
So that's, in fact, what I do, is I say R is m over B.
So it fits in cache, we have at least one block for each thing that we're merging.
And then when we do the analysis now, I take log base m over B here, and that's compared to the binary one, which was log base 2, which is a factor of-- because this is just log 2 over log base-- of log of m over B savings in cache misses.
Now, is that a significant number?
Let's take a look.
So if your l one cache is 32 kilobytes, and we have cache lines of 64 bytes, that is basically the difference in the exponents, 9x savings.
For l two cache, we get about a 12x savings.
For l three, we get about a 17x savings.
Now of course, there are some other constants going on in here, so you can't be absolutely sure that it's exactly these numbers, but it's going to be proportional to these numbers.
So that's pretty good savings to do multi-way merging.
So generally when you merge, don't merge pairs.
Not a very good way of doing it if you want to take good advantage of cache.
May give you some ideas for how to improve some sorts that you might have looked at.
Now it turns out that there's a cache oblivious sorting algorithm, where you don't actually have-- that was a cache aware algorithm that knew the size of the cache, and we tune R to get there.
There is an algorithm called funnelsort, which is based on recursively sorting and n to the 1/3 groups of n to the 2/3 items.
And then you merge the sorted groups with a merging process called an n to the 1/3 funnel.
So this is more for fun, although the sorting algorithm, in my experience, from what others have told me about implementing it and so forth, is probably about 30% slower than the best hand-tuned algorithm.
Whereas with matrix multiplication, the cache oblivious algorithms are as good as any cache aware algorithm as a practical matter, here, they're off by about 20% or 30%.
So interesting research topic is build one of these things and make it really efficient so that it can compete with real sorts.
So the k funnel merges k cubed items in k sorted lists, incurring this many cache funnels.
Here, I did put in the one, for people who are concerned about the ones.
And so then, you get this recurrence for the cache misses, because you solve n to the 1/3 problems of size n to the 2/3 recursively, plus this amount for merging.
And that ends up giving you this bound, which turns out to be asymptotically optimal.
And the way it works is there's basically, a k funnel is constructed recursively.
And the idea is that what we have is, we have recursive k funnels, so this is going to be a merging process, that is going to produce k cubed items by having k to the 3/2 buffers that each are taking the square root of k guys and producing k guys out.
So each of these guys is going to produce k, and the square root of k of them for a total of k to the 3/2, but each of these is going to be length square root of k, so we end up with k cubed.
and And they basically feed each other, and then they get merged with their own k thing, and each of these then recursively is constructed the same way.
And the basic idea is that you keep filling the buffers, I think I say this here, so that all these buffers end up being in contiguous storage.
And the idea is, rather than going and just getting one element out as you do in a typical tournament, as long as you're going to go merge, let's merge a lot of stuff and put it into our buffer so we don't have to go back here again.
So you sort of batch your merging in local regions, and that ends up using the cache efficiently in the local regions.
Enough of sorting.
Let's go on to physics.
So many of you are probably studying in your linear algebra class or elsewhere, the heat equation.
So, people familiar with heat diffusion?
So it's a common one to do, and these were-- I have a former student, Matteo Frigo, who is a brilliant coder on anything cache oblivious.
He's got the best code out there.
So the 2D heat equation, what we do is let's let u(t, x, y) be the temperature at time t at point (x, y).
And now you can go through the physics and come up with an equation that looks like this, which says that basically the partial of u with respect to t is proportional to the sum of the second partials with respect to x and with respect to y.
So basically what that says is, the hotter the difference between two things, the quicker things are going to adjust, the quicker the temperature moves between them.
And alpha is the thermal diffusivity, which has-- different materials have different thermal diffusivities.
Say that three times fast.
So if we do a simulation, we can end up with heats, say, put like this, and after it looks like this.
See if we can get this running here.
So now, let me see.
So I can move my cursor around and make things.
You can just sort of see that it simulates.
You can see the simulation is actually pretty slow.
Now, on my slide, I have a thing here that says-- let's see if this breaks when we do it again.
There we go.
So we're getting around 100 frames per minute in doing this simulation.
And so how does this simulation work?
So let's take a look at that.
It's kind of a neat problem.
So this is what happened when I did 6.172 for a little while.
It basically gave me that after a while, because it just sort of averages things, smears it out.
So what's going on?
Let's look at it in one dimension, because it's easier to understand than if we take on two dimensions.
So assuming that we have, say, a bar which has no differential in this direction, only in this direction.
So then we get to drop the partials with respect to y.
So if I take a look at that, what I can do is what's called a finite difference approximation, which you probably-- who's studied finite differences?
So a few people.
It's OK if you haven't.
That's OK if you haven't, I'll teach it to you now.
And then you're free to forget it, because that's not the part that I want you to understand, but it is interesting.
So what I can do is look at the partial, for example, with respect to t, and just do an approximation the says, well, let me perturb t a little bit-- that's what it means.
So I add delta t minus u of t divided by t plus delta t minus t, which gives me delta t. And I can use that as an approximation for this partial.
Then on the right hand side-- well first of all, let me get the first derivative with respect to x.
And basically here what I'll do is I'll do an approximation where I take x plus delta x over 2 minus x minus delta x over 2, and once again, the differences in the terms there ends up being delta x.
And now I use that to take the next one.
So basically, to take this one, I basically take the partial with respect to delta x over 2, minus the partial with x minus delta x over 2, and take the partial of that, do the approximation.
And what happens is, if you look at it, when I take a partial here I'm adding delta x over 2 twice, so I end up getting just a delta x here, and then the two things on either side combined give me my original one, 2 times u(t, x), and then another one here, and now the whole thing over delta x squared.
And so what I can do is to reduce this heat equation, which is continuous, to something that we can handle in a computer, which is discrete, by saying OK, let's just do this approximation that says that this term must be equal to that term.
And if you've studied the linear algebra that said that there are all kinds of conditions on convergence, and stability, and stuff like that, that are actually quite interesting from a numerical point of view, but we're not going to get into it.
But basically, I've just taken that equation here and said, OK, that's my approximation for this one.
And now what do I have here?
I've got u of t plus delta t, and u things u of t, and then over here, they're all with t, but now the deltas are in-- whoops, that should have been a delta x there.
I don't know how that got there.
That should be a delta x there.
They're all in spatial over here.
So what I can do is take this, and do an iterative process to compute this.
And so the idea is, let me take this and throw this term onto the right hand side, and look at u of t plus delta t as if it's t plus 1.
Let me make my delta be one, essentially.
Throw the delta t over here times the alpha over delta x1, and then I get basically u of tx is based on u of t of x plus 1 and of x and x minus 1.
As I say, there's a typo here.
That should be a delta t. So what that says is that if I look at my one-dimensional process proceeding through time, what I'm doing is updating every point here based on the three points below it, diagonally to the right, and diagonally to the left.
So this guy can be updated because of those.
These we're not going update, because they're the boundary.
So these can be fixed.
In a periodic stencil, they may even wrap around like a torus.
So basically, I can go through and update all these with whatever that hairy equation is.
And this is basically what the code is that I showed you is doing.
It just keeps updating everyone based on three until I've gone through a bunch of time, and that's how the system evolves.
So any questions about how I got to here?
So we're going to now look at this purely computer sciencey now.
We don't have to understand any of those equations.
We just have to understand the structure.
The structure is that we're updating t plus 1 based on stuff on three points with some function that some physicist oracle gave us out of the blue.
And so here is a pretty simple algorithm to do it.
I basically have what's called the kernel, which does this updating, basically updating each one based on things.
And what I'm going to do for computer science is I don't need to keep all the intermediate values.
And so what I'm going to do is do what's called an even-odd trick.
Basically if I have one row, I compute the next row into another array, and then I'll reuse that first array-- it's all been used up-- and go back to the first one.
So basically here, I'm just going to update t plus 1 mod 2, and just allocate two arrays of size n, and just do modding all the way up.
Is that clear?
And other than that, it's basically doing the same thing, and I'm doing a little bit of fancy arithmetic here by passing-- see stuff, where I'm passing the pointer to where I am in the array, so I only have to update it locally within the array.
So I don't have to double indexing once I'm in the array, because I'm already indexed into the part of the array that I'm going to use, and then I am doing flipping.
So this is just a little bit of cleverness.
You might want to study this later.
So what's happening then is I have this double nested loop where I have a time loop on the outside, and a space loop on the inside, and I'm basically going through and using a stencil of this shape, this is called a three point stencil, because you're basically taking three points to update one point.
And now if I imagine that this dimension is bigger, n here is bigger than my cache size, what's going to happen?
I'm going to take a cache fault here, these are all cold misses, et cetera.
But when I get back to the beginning here, if I use LRU, nothing is going to be in memory that I happened to update over here.
So I have to go and I take a cache fault on every cache line.
And so if I'm going t steps into the future from where I started, I basically have n times t updates, and I save a factor of B, because I get the spatial locality because the u of t minus 1, u of t, and u of t plus 1, are all generally on the same-- are nearby, and all within one cache line.
Question?
The x's, what are the x's for?
Sorry.
I should have put the legend on here.
The x's are a miss.
So I do a miss when I update these, and then these I don't miss on, because it was brought in when I accessed that.
And then I do a miss, and I'll do it-- so basically I do it, then I shift over the stencil by one, and then I won't get a miss.
So I'm just looking at the misses on the reads, not misses on the writes.
I should have made that clear, too.
But the point is, the writes don't help you, because it's all out of memory by the time I get up here.
To the second row, if this is longer than my cache size, none of that's there if I'm using
You have also a miss, like you need to get two [INAUDIBLE]
Yeah, but what I'm saying is I'm only looking at the read misses.
Yes, there are write misses as well, but basically, I'm only doing the read misses.
You can look at the write misses as well.
It makes the picture messier.
So we've basically have nt over b.
However this, let me tell you, is the way that everybody codes it.
and And if you have a machine where you have any bandwidth issues to memory, especially for these large problems, this is not a very good way to do it, as it turns out.
So it turns out that what you want to do is, as we've seen, divide and conquer is a really good way to do it.
But in this case, when we're doing divide and conquer, we're actually not going to use rectangles, we're going to use trapezoids.
And the reason is that a trapezoid has the nice property that-- notice that if I have all these points in memory, then notice that I can compute all the guys that are read on the next level, and then I can compute all the guys that are next on the next level.
And so for example, if you imagine that this part here fit within cache, I could actually keep going.
I didn't have to stop here, I could keep going right up to a triangle if I wanted to, and compute all the values without having any more cache misses than those needed to bring in, essentially, one row-- two rows, actually, because I'm reusing the rows as I go up.
So what we're going to do is traverse trapezoidal regions of space-time points such that the points are between an upper limit, T1, and a low one, T0, and between an x0 and an x1, where now I have slopes here that are going to be, in general, this is plus 1 minus 1.
And in fact, sometimes it will be straight, in which case we'll call it 0.
It's really the inverse of the slope, but we'll still call it zero rather than infinity.
So it's 1 over the slope.
There's a name for that, right?
Is that called the run or something?
I forget, I don't remember my calculus.
So that's what we're going to do.
And we're going to leave the upper and right borders undone and include, so it's going to be a sort of half open trapezoid on the left and bottom, closed on the left and bottom, and open on the top and right.
So the width is basically the midpoint here, and the height is the height, because they're always going to have parallel axes here.
So here's how are our recursion is going to work.
If the height is 1, then we can compute all space-time points in any way we want.
I can just go through them if I want, because they're all independent.
None depends on anybody else.
So that's going to be our base case.
If the width is greater than twice the height, however, then what we're going to do is we're going to cut the trapezoid through the middle of the slope of minus 1.
And that will produce two new trapezoids, which we then will recursively compute all the elements of.
So I'll start out with a trapezoid.
Basically, if it ends up that it's a long and wide one, I'm going to make what's called a space cut, and cut it this way, and then I'm going to recursively do this one and then this one.
And notice that I can do that because-- all these guys I can do, but then when I get to the border here, this will already have been done by the time I'm computing these guys.
So the requirement is that I've got to do things according to that map of triples that I showed you before, but I don't have to do them in the same order.
I don't have to do the whole bottom row first.
In this case, I can compute the whole trapezoid here, and then I can compute this trapezoid here, and then all the values that I'll need will have already been computed over here, that are on the boundary of this trapezoid.
The other type of cut I'll do is what happens when a trapezoid gets too tall for me.
So if the trapezoid is too tall, then what we'll do is we'll slice it through the middle, but the other way.
We call that a time cut.
So we won't take it all the way through time, we'll only take it partially through time.
Now you can show, and I'm not going to show this in detail, but you can show that if I do this, my trapezoids are always sort of medium sized.
I never get long, long skinny ones.
If I start with something that's sort of got a good aspect ratio, I maintain a good aspect ratio through the entire code.
So here's the implementation.
This is what Matteo Frigo wrote, and I've modified it a little bit.
So basically, we pass in it the values that let us identify the trapezoid, t0, t1, x0, and then the slope on the left side, x1 in the slope on the right side, where the dx0 and the dx1s are all either 0, 1, or minus 1.
And then what I do is I look at what the height is that my trapezoid is going to operate on.
And if the height is 1, well, then I just run through all the elements, and I just compute the kernel-- that program that I showed you before, that kernel-- on all the elements.
Nothing really to be done there, just go through and compute them individually with a four loop.
Otherwise, if I've got the situation where the trapezoid is big, then I do this comparison, which I promise you-- you can work out the math if you wish-- which I promise you tells you whether or not it's more than twice the height, as I said before, whether the width is more than twice the height.
And if so, I compute the middle point, and then I partition it into two trapezoids, and I recursively call them.
And otherwise, I simply cut the time in half, and then I do the bottom half and then the upper half.
So getting all those parameters exactly right takes a little bit of thinking, makes my brain hurt, but Matteo is brilliant at this kind of coding.
So let's see how well this does.
So I'm not going to do a detailed analysis that I did before, but basically what's going on is at this level, if I'm doing divide and conquering, I'm only doing a constant amount of work managing this stuff.
So my caches that I'm taking in the internal part of the tree are all going to be order one.
Now each leaf is going to represent a trapezoid, which is going to be approximately h times w, where h and w are approximately equal, because they're going to be shaped-- This is assuming I start out with a number of iterations to do that is at least as large as the number of points that I have to go on.
If I start out with something that's really thin and flat, then it's not going to be the case.
But if I start out with something that's deep enough, then I'm going to be able to make progress in an unconventional order into time by moving the time non-uniformly through the space.
So each leaf represents a fairly balanced trapezoid.
Each leaf basically is going to-- if you look that the direction of the trapezoid is in time, so this represents the spatial dimension, and if I have something of size w, I can access it with only w over B misses.
And when that fits in cache, where w is some constant less than m, so w is order m. So each of these things that's a leaf is only going to occur w over B misses.
Now, the total space number of points I have to go after is n times t. N is going to be the full dimension this way, t is the height that way.
And so since each leaf has hw points, I have nt over hw leaves.
And the number of internal nodes is just the leaves minus 1, so they can't contribute substantially, because there's only order one misses I'm taking here, whereas I've got something on the order of w over B misses for this.
So therefore, now I can do my math.
The number of cache misses I'm going to take is-- well, how many leaves do I have?
nt over hw.
And what does each one cost us?
W over B.
And so now, when I multiply that out, well, hw is about m squared, and w over B is about m over B.
And so I get nt over MB as being the total number of savings.
so whereas the original algorithm only got nt over B, we've got this factor of a memory cache size in there showing us that we have far fewer cache misses.
So the cache misses end up not being an issue for this.
Any questions about that?
So I want to show you a simulation of this three point stencil and comparing the two things.
So this is going to be the looping version, where the red dots are where the cache misses are, and this is going to be the trapezoidal one.
And basically, I have an n of 95 and a t of 87, and what I'm going to do is assume a fully associative LRU cache that fits four points on a cache line, where the cache size is 32, two to the fifth as opposed to two to the 15th, it's really little.
If I get a cache hit, I'm going to call it one cycle.
If I get a cache miss, I'm going to call it 10 cycles.
We're going to race them.
So on the left is the current world champion, the looping algorithm.
And on the right is the cache oblivious trapezoid algorithm.
So let's go.
So you can see that it's basically, it's made a space cut there, but it's made a time cut across the top there.
It said, this is too tall, so let me cut it this way.
And that guy's, meanwhile, taking all those-- you can see how many cache misses he's taking.
Let's speed him up.
That's one way you can do it, is make it think.
So let's see what happens if I have a cache of size eight.
So here we go.
I think I'm just doing the same thing.
I know I can show you the cuts.
Can I show you the cuts?
I know.
I think it's because I'm not-- OK, let's try it.
There we go.
Now I'm showing the cuts as they go on.
Let's do that again.
We'll go fast and do it again.
So those are the cuts that it's making to begin with as it's doing the divide and conquer.
So I think this is the same size cache.
So now I think I'm doing a bigger cache.
I think I did a bigger cache, but I'm not sure I gave the other guy a bigger cache.
Yeah, because it doesn't matter for the guy on the left, right?
As long as the cache line is the same length and as long as it's not big enough, he's going to do the same thing.
He didn't get to take advantage of the fact that the cache was bigger, because it was still smaller than the array that he's striping out there.
Anyway, we can play with these all day.
So if you make the cache lines bigger, then of course it'll go faster, because he'll have fewer misses.
He'll get to bring it in.
So let's see here.
So let's now do it for real.
So this is a two-dimensional problem.
You can use the same thing to do what end up being three-dimensional trapezoids.
In fact, you can generalize this trapezoid method to multiple dimensions.
So this is the looping one.
So let's start that one out.
So it's going about 104 frames a minute.
I think by resizing it, the calibration is off.
But in any case, let's switch to the cash oblivious version.
Anybody notice something?
Slower.
Why is that?
I gave code exactly as I had up there.
No, it's not because it's two dimensions
[INAUDIBLE]
I'm sorry?
[INTERPOSING VOICES]
Yeah, so now it's the trapezoiding at only 86 frames.
What do you suppose is going on there?
You have [INAUDIBLE]
Yeah.
So this is a case where if you look at the code I wrote, I went down to a t, a delta t, of one in my recursion.
I recursed all the way down.
Let's see what happens if instead of going all the way down, playing the trapezoid game on little tiny trapezoids, suppose I go down only to, say, when t is 10, and then do essentially the row major ones.
So I'm basically coarsening the leaves of the thing.
So to do that, I do this.
So now we go-- ah.
So I have to coarsen in order to overcome the procedure call overhead.
It has nothing to do with the cache.
It has to the fact that the way that you implement recursion, recursion and function calls have a cost to them.
And if what you're going to do is do a little tiny update of those few floating point operations-- let's go back to the looping just to see.
The looping is going about 107, 108, and trapezoiding at 136.
So unfortunately, you need a voodoo variable, but it's a voodoo variable not to overcome the cache, but rather to deal with what's the overhead in using the processor when you do function calls.
So let's see.
How coarse can we make it?
Let's try five, a coarsening of five?
That's still pretty good.
That's still pretty good.
How about four?
Still doing 131 frames a minute.
How about three?
Oh, we lost something there.
How about two?
So at a coarsening of two, I go 138, whereas the looping goes at about the same.
I can't do 20.
I didn't program that in.
I just programmed up to 10.
So if I go down to one, however, then you see it's not that efficient.
But if I pick any number that's even slightly larger, that gives me just enough that the function call overhead ends up not being a substantial cost of the things.
So let me just wrap up now.
So I just have a couple more things.
So I'm not going to really talk about these, but there are lots of cash oblivious algorithms that have been discovered in the last 10 or 15 years for doing things like matrix transposition, which is similar to rotating a matrix.
You can do that in a cache oblivious fashion.
Strassen's algorithm, which does matrix multiplication using fewer than n cubed operations.
The FFT can be computed in a cache oblivious fashion.
And LUD composition is a popular thing to solve systems.
In addition, there are cache oblivious data structures, and here are just a few of them.
There's cache oblivious B-Trees and priority queues, and doing things called ordered-file maintenance.
There's a whole raft.
There's probably now several hundred papers written on cache oblivious algorithms, so something you should be aware of and understand how it is that you go about designing an algorithm of this nature.
Not all of them are straightforward.
For example, the FFT one does divide and conquer but not by dividing it into two.
It divides it into square root of n pieces of size square root of n each in order to get a good cache efficient algorithm that doesn't have any tuning parameters.
But almost all of them, since they're recursive, do have this annoying thing that you have to still coarsen the base case in order to get really good performance if you're not doing a lot of work in the leaves of the recursion.
So any questions?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to talk about dynamic storage allocation, which is a hugely rich and interesting topic, and hopefully cover some of the basics of dynamic storage allocation today.
I want to start out with perhaps the simplest storage allocation scheme, and that's stack allocation, which I think you're probably fairly familiar with.
So stack allocation is just an array and a pointer into the array, an index or a pointer into the array.
And when you have a stack in a computer memory, you're talking about the array of all memory, and the stack pointer pointing that's used, for example, for calls.
The reason stack allocation is neat is because it's so simple.
If I want to allocate x bytes, then what I do, is I just simply increment my stack pointer by x bytes, and then I return the original position of the stack pointer as being the location of the storage that's just been allocated.
So pretty simple.
Typically if you write a stack allocator, the routines are so simple they get inlined, although it's of course advantageous to say that things are inlined, but most compilers these days will inline.
And as you can see, it's just a couple of operations here to perform this calculation.
To deallocate the bytes, we basically-- oh, yeah, I should mention, this isn't checking for overflow.
So technically, one should check for overflow.
One way to do that is for the assert package, and that way, at run time, you don't have to pay the extra overhead of checking.
And see, these operations are so cheap that actually checking for overflow is perhaps as expensive as doing the rest of the operations themselves.
Checking to see whether you went over the right end of the array here.
So to free x bytes, you basically just simply decrement the stack pointer.
And of course, technically, you should also check for underflow, make sure somebody didn't say to deallocate more bytes than were actually already allocated on the stack.
And so course, check for stack underflow.
So this is pretty good, because allocating and freeing take just constant time.
Very quick.
But you're required to free memory consistent with the stack discipline, which is the last in, first out discipline.
So you don't get to free something that you allocated long ago until you've freed everything else that's been allocated since.
So therefore, it has limited applicability, but it's great when it works.
Now it turns out, sometimes you can actually use the call stack yourself to store stack values, using a function called alloca.
But this function is now deprecated, meaning that people advise you not to use it in any modern programming.
And part of the reason is that the compiler is more efficient with fixed size stack frames.
You remember when we went through how the compiler actually lays out memory and does the calling convention?
If you have a fixed size stack frame, then the frame pointer and the stack pointer are redundant, and so you can reclaim a register, namely, the frame pointer register, and do everything off the stack pointer.
But if you use things like alloca, which are pushing the sides of the stack frame, then you actually have to keep both pointers there, and that's a little bit less efficient in terms of the register usage.
OK?
So that's pretty much stack storage.
Any questions about stacks?
Yeah?
Is there anything that detects overflow or underflow without using inserts?
Like, if you wanted to check--
Yeah, you couldn't check explicitly.
Just say, you know, for this, is before you check that, make sure that SP plus x, is it bigger than the right end?
But how do you know where the right end is?
Well, because I'm assuming here that in this case, you are allocating it.
Now, if you're talking about using the call stack, you mean?
You'll never run out.
We'll do that analysis in a little bit.
But you'll never run out of stack space.
Although as a practical matter, most compilers assume a stack of a megabyte, and so if you overflow that, your program will fail in any of a variety of ways
You can do guard pages
You can do guard pages, which don't always work.
But let me explain guard pages, OK?
So what's sometimes done is, if I have my stack growing down in memory-- OK, so here's my stack, and it's growing this way-- and I want to make sure, for example, the thing that's typically growing up this way is heat, and I want to make sure that they don't collide somewhere in here.
So one thing I can do is put some pages in here that are protected.
These are read-only pages.
Or actually even better is no permissions.
So I can put no permissions so that if you ever try to read or write that page, it automatically generates a fault.
And so the runtime will sometimes do that.
The difficulty is, what happens if you increased your stack so much that you went beyond your guard pages?
But as a practical matter most of the times that people exceed the guard pages is only by small amount.
And so just putting a few guard pages in there will make sure that you can use the memory management hardware, which basically ends up being free from your point of view as a programmer, in order to catch a stack overflow problem.
And you can do that yourself.
You can use the call mmap, which we haven't talked about, to set permissions on pages at user level
[INAUDIBLE]
No.
You can actually handle the error however you want.
You can vector into a user to find routine.
Now, what are you going to do, I don't know, but you can indeed catch the error yourself and decide that there's some way of recovering, as opposed to crashing and burning.
So generally, it's considered poor form for your software to fail, other than in a controlled way.
So you should always have your software exiting or calling error, and it should never segment fault.
Most people consider any program that segfaults to be poorly engineered.
Is there any case where there's a segfaulting program that is not poorly engineered?
I don't know.
I think most people, it's probably an understatement.
OK?
So this is stack memory.
You can build it yourself out of an array and a pointer, or you can I use this the call stack.
And when you have something that's using that kind of discipline, it's great, because it's really cheap.
Now in C and C++, and we're going to start moving and talking more about things like C++, we have what's called heap allocation.
And in C, the routines are called malloc and free, and C++ provides a new directive and delete.
So unlike Java and Python, C and C++ provide no garbage collector.
So in other words, you're responsible as a programmer for freeing allocated storage yourself.
If you fail to do so, you create a memory leak.
So a memory leak means I allocated something, I no longer am in a position to use it, but I failed to free it, so it can't be reused.
And if you keep doing that, especially if you're in a loop, then you have a memory leak.
Where if you look at a program with a memory leak, and you look using [?
Top ?]
or whatever, how big is my program, you watch as it goes and computes and computes, and it keeps getting bigger and bigger and bigger and bigger as it's running, and it just runs away, getting more and more storage.
Also you have to watch out for things like dangling pointers.
That is, you deallocated something, but somebody still wanted to use it.
Or double-freeing.
You have two places in the code where you free the same storage.
Very bad to try to free the same storage.
Yeah?
[INAUDIBLE]
It's basically going to be sitting in the storage structure, which we'll talk about how those are organized in a few minutes-- going to be sitting in the storage structure, but the program that is going to look at that and not recognize that it's in the storage structure, and think that it's something that now all its pointers can be reset.
And so you basically clobber all the invariants of the storage structure, and somewhere down the road, you get a segment fault or something.
OK?
Yeah, John?
[UNINTELLIGIBLE] that comes out of that is--
Just a second.
Why don't you take this?
[INAUDIBLE] One thing that often happens in the real world is when you free something, it adds it to a free, list and if you free something twice, it will add two copies of that pointer to the free list.
So later on, when you malloc, malloc might return the same pointer to you twice.
So the same memory block might be allocated for two completely different functionalities, and attackers use that to do all sorts of interesting things to your program
Yeah.
OK?
So fortunately, there are some really good-- why don't you keep that down there, and then if you guys want to chime in, you'll have some power to do so.
So fortunately, there are now some tools to make that easier to debug these kinds of really nasty bugs.
The one we'll be using is Valgrind, and what you do is you just say, Valgrind minus minus leak check equals yes, and then run your program the way you would normally run it.
And what it will do is, it will monitor the way that the program is doing allocations and frees, and give you a report as to whether everything that was allocated was free, and whether you tried to free something twice, and so forth.
Now, this works with the existing allocators.
If you have your own allocator that you decide you're going to write for a little piece of your code, you say, gee, I'm going to do my own little allocation here, you can actually communicate to Valgrind what that allocator is, and it can check for proper use of that structure as well.
So you can see valgrind.org for details.
And are a lot of other programs on the market, and out there as open software, such as purify and a variety of others.
So let's talk about heap allocation.
I want to start by talking about fixed size allocation.
So suppose the world has only one size of objects that you care about.
Then it turns out that doing an allocator is really simple, and you could program one yourself.
So here's basically how it works.
It's called using a free list.
And what you have is, in your array, say, that you're using for storing all of your values, and here they're basically this size, what you do is you have a certain number that are used.
That means that the program is using them, not the storage allocator.
And what you do is you take advantage of the fact that if it's an unused block of storage, then you can put a pointer in there.
And so what you do, is you string all these unblocked pieces of storage together into a free list.
So we point to there, which points to there, which points to there, which points to there, which points nowhere.
So you just simply linked together all of the things that are free in that list.
And then whenever you need one to allocate an object, what you do is, you grab one off the free list.
So you set the value that you're going to want to take off free list to free-- so here x gets a pointer to this object, the one that's at the start of the list.
Then you update free with free arrow next, which gives you this guy.
So that's the next guy in line.
And then you basically return x.
And that's all there is to it.
You just grab the first element off the free list.
Now, there's a little bit of subtlety in that the thing that you're grabbing off has a garbage pointer in it, because we're not initializing storage.
There are some storage allocators that will always set the value that's returned to the user to all zeros, for example.
But typically malloc doesn't do that.
It basically leaves that as uninitialized storage.
And so you can actually go and see, what were the values that used to be stored in there?
And so basically, you can end up with a garbage pointer there, but you really don't care about that, because it's no longer on the list.
So you can see, for example, if I then went and I-- well, let's do this, to take a look at what happens when a free an item.
So if I want to free an object x, I take some object x here.
Let's say it's this guy that's being freed.
What I do is, I make it point to the first element of the list.
So basically, next equals free, So he basically points there.
And then I say free equals x, meaning move the free pointer up to now point to the new beginning of the list.
So we're basically pushing and popping things off the front of the list.
It's just the stack discipline, except we're using a linked list, rather than using an array.
So we push to free, and we pop to allocate.
Now the tricky thing here-- you can sort of see what might go wrong if I then said, oh, let me free this thing again.
Suppose you said, now that this is free, suppose you came in and say, oh, I'll free that thing again.
Well, now you're going to go through the same code here, and you're going to see, you're going to completely screw up the free list.
So that's why you don't want to do a double free.
So you can see exactly what's going to go on in the free list implementation.
So this is pretty good and pretty cheap.
Allocating and freeing take constant time.
That's great, just as with stack pointer.
It's got good temporal locality, and the reason is because you're tending to allocate the thing that you freed most recently.
The free list operates as a last in, first out data structure.
So from a temporal locality point of view, it's really very good.
You're always using the most recently freed stuff whenever you allocate, and that means you have very good caching behavior from a temporal locality point of view.
However, it has poor spatial locality, and the reason is due to what's called external fragmentation, where you get your storage distributed across virtual memory.
So if you think about this, as you start allocating and freeing in different orders, the order of that free list gets very jumbled up.
And as it increases, and you end up with jumping from one thing to another, you basically can end up causing you to use a lot of the page table, because you're basically using a huge piece of it, rather than-- and in particular, the translation look aside buffer can also be a problem.
So the translation look aside buffer typically works on a page basis.
And if you recently referenced things on a given page, that's very good for the TLB, because it's already got the translation, the virtual address mapping of that page dealt with.
So it basically has poor spatial locality, because it's getting all jumbled up.
Compared to stack allocation, which is always very nice in terms of its behavior, because you're going across consecutive locations in memory, which means you're always allocating and freeing from the same page, unless you trip over a page boundary.
So here's some ways of mitigating external fragmentation.
Of course, this makes the code for free list more complicated.
And then you have to weigh, how much am I losing in having a more complicated allocator versus suffering the fragmentation may occur?
Yes, question?
You specificed before that it was fixed size.
Is that why we don't specify anything about the size?
That's right.
We're just assuming that it's all whatever the unit size is.
So this is really good when you've created your own data structure, and let's say you're doing nodes in a graph.
You know exactly what elements you want in the graph, for each vertex.
And now what you want to do is basically be able to give yourself a note of a graph, free a note of a graph, and manage that storage yourself.
Yes, question?
[INAUDIBLE] allocated across virtual memory [INAUDIBLE] one application?
Why is it allocated across virtual memory if it's all one application?
Well, because the size that you may end up needing can end up being large.
And I'll show you in just a minute how that happens.
So here's the idea.
What you can do is keep a free list per disk page.
So disk page, a traditional page is 4,000 bytes.
These days they talk about super pages that are megabytes, and who knows what size pages various operating systems will support.
I believe the clouds use 4k, as configured right now are using 4k, which leads to some strange anomalies where the TLB can address less stuff than the L3 cache.
But so be it.
So basically, they just keep a free list per disk page, and basically always allocate from the free list for the page that's fullest, the page that you've allocated the most of the stuff off of.
Rather than for a page that's less empty.
So what this means is that when you come to allocate, you have to figure out, which page should I do the allocation off of?
But we'll have a free list on each page.
Whenever you free of a block of storage, you have to free it to the page on which the block resides.
Don't put into a free list of a different page.
You put it into a free list all of that particular page.
Now if a page becomes empty, in other words, there's only the free list on it, there's no actually used elements by the programmer, the virtual memory system will page it out.
And since the programmer's never referencing things on that page, it basically costs you nothing.
You end up not having to have an entry in the TLB, and you may have written to it, but you will end up allocating things preferentially to the more full pages.
And the basic idea is, it's better to have the use of your pages balance 90/10 than 50/50.
So this kind [?
of fig ?]
configuration is better than that.
So why is that?
If you have 90% of your storage on one page and only 10% on another-- yeah, let's say we only of two pages in our system, and 90% of the objects are on one page, and 10% of the objects are on the other page, versus if they're 50-50.
Then what you can do is look at the probability that two accesses are going to go to the same page.
Because if you're going to the same page, then your paging hardware, which is very much like the cache, your TLB, which is very much like a cache-- whenever you get a hit on the same page, that's good, it costs you nothing.
If you hit on a different page, that's bad.
So what happens is, the probability that two hit the same page is, well, for the first one, it's, what's the probability that the first one was on this page, versus on the other?
So it's going to be on the same page if they're both on this page or they're both on this page.
And so that's going to be 0.9 times 0.9 plus 0.1 times 0.1.
Whereas if it's 50/50, then the probability that you get hit on the same page is now this 0.5 times 0.5 plus 0.5 times 0.5, which is only 50%.
So we get 82% hit rate versus the 50% hit rate.
So it's much more likely, if I'm stacking everything out in the extreme, it's better to have everything on one page and nothing on the other.
Then I get 100% hit rate, because whenever I'm accessing something, I always am hitting on the page I had before.
So this kind of heuristic helps it so that you're swinging your accesses to using fewer pages, and the fewer pages, the less likely it is that you're going to stress your virtual memory paging, disk swapping, you're going to stress your TLB and what have you.
Good for that?
We're going to get deeper into this, so.
So that's basically fixed size allocation.
It's really nice, because you can basically use a very simple free list.
But of course what most people need and want is variable sized allocation.
The most popular way of doing variable sized allocation is a scheme called binned free lists, and of that, then there are a zillion variations on it.
But mostly, people use binned free lists.
The idea is, we're going to leverage the efficiency of normal free lists.
Normal free lists I can allocate and free in just one operation, in just a constant amount of time I make.
And what we're going to do is accept some internal fragmentation.
So by internal fragmentation I mean the following.
When somebody asks for something of a given size, what we're going to do is actually give them something which is a little bit bigger, maybe.
And so we'll waste some.
And in particular, what we'll do is, we'll organize our system so that all the blocks that we ever give out are always a size the power of two.
In practice, you don't always give out exactly a power of two, because then you get cache alignment problems.
But this is the basic scheme, is to give out things are powers of two.
And that way, if somebody asks for something that has 13 bytes, you give them 16 bytes.
For somebody who asked for 65 bytes, have to give them 128 bytes.
So you might waste up to as much as a factor of two.
But now, for a given that range of values, there's only a logarithmic number of different sizes that we have to give out, because we're only giving them out in powers of two.
So here's the basic allocation scheme to allocate x bytes.
So what I do is, if I'm allocating x bytes, what I do is, I look in the bin ceiling of log x, because that's where everything of size two to the ceiling of log x is, which is basically rounding x up to the next power of two.
If it's non-empty, I just return a block as if it were an ordinary free list.
And what I'll be doing is returning a block which is at most twice the size of the requested block.
But that's generally OK. Somebody wants at least that many bytes.
The fact that it's a few more bytes won't matter to them.
They don't know that there's more bytes hidden there.
They only take advantage of the fact that there are many bytes as I've asked for.
Otherwise, suppose that that bin is empty.
There's nothing in that bin at this time.
Well, then what you do is you start looking to find a block in the next larger non-empty bin, and you split it up.
So for example here, suppose that the request is x equals 3.
I have to look in bin 2, which is blocks of size 4.
Uh oh, bin 2 is empty.
So what I do is, I start going down, looking in the next bin.
Oops, that's empty too, that's empty-- and then finally, I get to one where there is a block in it.
So what I do then is I dice this up.
I cut it in half, and then cut that one in half, until I get a piece that's exactly the size I want to return.
OK?
And then I distribute all of these guys up to the other bins, like this.
So I return one of this size, and I populate these other guys with one block each of the appropriate size.
And as you can see, they're basically growing in a geometric fashion.
And then I distribute that bin.
Now there's a caveat here.
Suppose that no larger blocks exist in my storage structure.
Well, then I have to ask the operating system for more bytes, to allocate x more bytes of virtual memory.
Now in practice what you do is you don't actually ask for x, since you never ask the operating system for small values.
You always ask the operating system for big values.
Give me another page worth, or give me another five pages, or something.
Because that's an expensive call to the operating system, to ask it to allocate more storage.
Let me explain how that works.
And this gets back to the question you had before, which is, where's the stuff coming from anyway?
What the operating system is allocating is actually not physical pages, but rather parts of virtual memory.
So we looked at this before, about how virtual memory is laid out for a typical program.
So what we do is, on the very high addresses, we have the stack, and the stack grows downwards.
This is the call stack.
At the low address, we typically start out with what's called the text segment, which holds your code.
Then there's a region called the data segment, and this consists of data that needs to be initialized.
So the compiler typically distinguishes between data that needs initialization and data which is going to be zero, because data that needs initialization, it writes into the binary executable file and stores that on disk.
So when you load in your program, it just loads as part of the text here, it loads that data segment, saying what all of the storage is.
The next segment is, for historical reasons, called the BSS segment.
And what BSS consists of is all the variables that at the start of the program should be initialized to 0.
It doesn't bother storing these in the file.
Why not?
Because it's easy enough to just bring it in, understand what that distance there is, and then do a memset of this whole region with zero, to zero it out, rather than trying to read in a whole bunch of zeroes.
An obvious kind of data compression.
Then heap storage is allocated in the space above the BSS segment.
So the idea is, there's a part of heap which has already been allocated stuff.
And whenever you need more heap storage, you ask the operating system-- the operating system moves just as if it were a stack, moves the line for where heap is to allocate, to give you more pages of heap storage that you can now allocate out of.
And what it's doing is allocating pieces of virtual memory.
Now, here's a good mind question, which I think gets directly to your point.
So think about this.
We have a 64-bit address space.
If I try to write at the rate of 4 billion bytes per second, that's pretty fast, because that's the speed at which the registers get written.
Not the speed that memory gets written.
So really, it's something like 4 billion bytes per second.
It takes over a century to write all of virtual memory.
So as a practical matter, that means I never run out of stack space.
I never run out of heap space.
They're never going to cross, no matter how much I'm allocating.
Even if I allocate on every cycle, there's not going to be a program-- unless you're going to set a program running for a millennium sort of thing, you're not going to see that.
Most programs run for a few minutes.
Some may run for days.
Some may run for years.
Very few do people write to run for a century.
So why not just allocate out of free virtual memory and never bother freeing?
Why am I worried about freeing stuff all the time?
Why not just allocate it?
It's virtual memory.
It's virtual.
I just allocate more storage as I need it.
Yeah?
[INAUDIBLE] memory?
So it is tied to real memory, and that's part of the answer.
It is tied to real memory, because anything that you write must find itself a place on disk.
Because it's going to be written to pages, and those pages are going to be paged and then written to what's called the swap space on disk, which allows you to put things there.
Even so, though, writing to disk-- it's still not going to take very long.
I mean, you're never going to be able to use up disk, because disk takes so long to write for.
But that is part of the answer
So you need to basically use [INAUDIBLE] software?
And so--
And the page table
[UNINTELLIGIBLE] on the disk
So the issue is, this is where the external fragmentation would be horrendous.
If you just kept writing across memory, and you only had a couple of words written on every page that were actually used, the rest is just garbage, then what you're saying is that in order to access a given datum, I'm not going to have it in memory with it a lot of other stuff.
What I'm going to have in memory with it is going to be all garbage, this stuff that I would have freed.
And so therefore, you end up having your program distributed across a large region of virtual memory, and that means as we're doing the paging, et cetera, we're getting very poor page locality as I'm accessing things.
And so I have a finite number of pages that I fit in memory, and now I may be blowing that out, and every access, rather than going to something nearby, is instead going to disk.
And so that can cause disk thrashing, where every access causes a disk access.
So how fast are disk accesses?
Order of magnitude?
How fast are disk accesses?
What's a disk access cost?
Fast disk?
[INAUDIBLE]
No, it's slower than that.
Disk access.
So good memory access, DRAM memory is, like, 60 nanoseconds for DRAM access.
It's like 60 nanoseconds.
So the processor is going somewhere around, these days, between 3 and 4 gigahertz.
So memory is like 200 processor cycles away.
How fast is disk?
An ordinary disk, not a solid state device.
Ordinary disk.
10 milliseconds is a good ballpark to give.
That's a good number to know.
10 milliseconds, not 10 microseconds, not 10 nanoseconds.
10 milliseconds.
So you are, what?
6 to 7 orders of magnitude slower if you go to disk.
Really slow.
So one of the things in performance optimization-- we're focusing a lot on the CPU-- is often the case for database things.
How do you minimize those disk accesses?
And all the things that we're learning about, performance for CPUs, can be applied to disk accesses, as well.
So the idea here is we want to avoid-- that's why we don't want to have our stuff strewn across a whole bunch of pages where we only have a fixed amount of internal memory, because then we'll end up paging all the time.
Every access will cause us to do a disk access, because it won't be very likely that we're getting stuff on the same page.
So getting back, for example, to the fixed allocation, that's why it's a good idea to try to keep those pages full, so that we have a small thing.
So the goal of most storage allocators is therefore to use as little virtual memory as possible, and try to keep the used portions very compact.
Because the more compact it is, the more likely it is that when you access one, you'll be able to access something nearby without causing a page fault.
So that's the goal of almost all storage allocators.
Try to keep things compact, try to keep things tight and in very little virtual memory space, even though virtual memory is huge.
Now, that doesn't mean that everything has to be contiguous.
For example, in this organization as I showed before, actually, I have it just on the previous slide, so why don't I go back there-- these are very far away in memory, but there's only a couple of them, and they're tight and local in the regions that they're operating in.
What I don't want to do is have it where I have just little stuff.
It's OK to go into the middle of virtual memory and have a whole bunch of consecutive pages full of data.
That's fine.
What's bad is having just a little bit of data, a little bit of data, and a little bit of data sort of sprinkled all over.
That's bad.
But as long as I can put a whole bunch of data nearby, it doesn't matter where in virtual memory I put it, particularly.
So the idea is, let's try to use as little virtual memory as possible.
Any questions about that?
OK.
So how do we do that?
So the first thing is, binned free list turns out to do a really good job of using relatively little memory.
So suppose that the user program is using, at most, m memory at any point during the execution.
Maybe allocating, maybe freeing.
But if I took the high watermark of everything he says that he needs, we call that m. Then it turns out that if you use a bin free list allocator, the amount of virtual memory consumed is at most m times log n. Order m times log n. And why is that?
Here's kind of a hand-wavy proof sketch.
So whenever I make a request for a block of size x, it may consume at most twice x storage that's actually in use.
And so it turns out that the total amount of memory devoted to blocks of size 2 to the k is therefore at most 2m.
So for any given size, I might, at any given point in time, have the program using all blocks of that given size.
Let's say size 2 to the k. And since there are a logarithmic number of those bins, each bin of which could have a size at most m, the total is order m log m. Now it turns out that there's actually a stronger property.
In fact, you can show that binned free list is order one competitive.
Who knows the term "competitive" in this sense?
The algorithmic meaning of "competitive"?
So what it means is, it's at most a constant factor slower than the optimal omniscient algorithm that could figure out exactly what's going on in every step.
So you're within a constant factor competitive with the optimal allocator, assuming that the optimal allocator does not do what's called coalescing.
If it simply is dividing up things, then it can subdivide, but it can never glue things back together again.
It turns out that in fact, binned free list is really much better than that.
So let's talk a bit about coalescing, because many, many memory allocators use coalescing.
So the idea is, I'm concerned about the case where I end up with a whole bunch of little small objects, and yet the allocations now come from big chunks.
And so I have to go and use more of my virtual memory, or maybe I can glue together those little pieces into a big chunk, and not have to go off to memory.
So that's what coalescing is all about.
So the idea is, if two things are adjacent, let me glue them together.
And there are really clever systems for doing that, including a very famous one called the buddy system.
But using any of these mechanisms for coalescing, you always have to trade off between the simplicity of not coalescing.
Not coalescing so stick it in the free list, I'm done.
Coalescing is, stick it in the free list, oh, am I next to somebody I can merge with?
Now let me merge.
So you always have to weigh, is that really going to be an advantageous thing?
It turns out there are no good theoretical bounds that I'm aware that prove the effectiveness of coalescing.
People have not figured out how to analyze this, or what an appropriate model is for understanding whether coalescing is a good idea.
However, it does seem to work in practice for some things, and I think the reason for that-- and this is just my own take on it-- is that storage tends to be deallocated as a stack.
So you tend to go into a region of a program where you do a lot of heap allocation-- let's say I'm building up a graph, and I keep doing a whole bunch of mallocs.
And when I'm done with the graph, what do I do?
I free it all.
So I tend to have these batch allocations and then deallocations, and they tend to work often as a stack, often in a batch.
And that means that when I do that freeing, I can sometimes take advantage of that and do coalescing.
In some systems, people have what are called memory pools, where you have a region where when you allocate out of it, you know you're allocating out of your own particular region specifically so that you can do coalescing of things in that region.
And so you may have multiple allocators, and you'll call the allocator on the region, say, for a graph, because let's imagine you're creating two graphs, and one of them you're going to keep around, and the other you're going to dump.
Well, if I create them at the same time, the storage gets all intermixed, I'll deallocate one graph.
The other one will be, I'll have fragmentation all over.
These little spots appeared where the other graph used to be-- not so good.
But if I say, use this allocator for this graph, use that allocator for that graph, then when I'm done and I'm allocating those out of two different regions of memory, that when I'm done with the first graph and deallocate everything, now I've created a nice big piece of contiguous storage.
So people will sometimes play with different memory pools for allocators, and C++ has a fairly rich set of libraries and so forth for that, to allow you to do that kind of thing.
So let's go on and talk about garbage collection, because that's something you've probably seen in Java and Python, and what's going on under the covers there.
Because even though C and C++ do not have garbage collection, you can actually implement your own garbage collector.
It's not that hard, except for some details.
It's only hard when you want it to perform well.
So the idea is to free the programmer from freeing objects.
The downside, by the way, is when you free the programmer from freeing objects, they tend to create lots of garbage.
And so what happens in Java programs, for example, is you'll have these-- I remember once having a Java program where, when it was sitting, waiting for user input, it would periodically go, garbage collect.
It was just sitting there, and then it would go, garbage collect.
We weren't doing anything.
It was just periodically, the way they had written the code, they were somehow generating garbage, doing allocations and frees, when you were sitting around doing nothing.
And so it would basically use up all of the space and then do a garbage collection, use up all the space and do a garbage collection.
Naturally that's kind of a slow system.
And so even in something like Java, where you have a garbage collector, you do have to worry about whether you're creating garbage or not, and you can generate faster code by not creating as much garbage, or by finding ways of recycling the garbage yourself.
So even in Java, it's worthwhile saying, give me a big chunk of memory, let me manage it myself, now let me free it.
I don't have to free it, but when I'm done with it, I'll be done with it.
But in the meantime, let me manage it myself so that I don't have it continually generating more garbage.
So the garbage collector can be built in, or you can do it yourself.
And mostly in this class, we do stuff ourselves, so we know how they work.
So that even when you're doing it with built in, you know what's going on under the covers.
So here's some terminology.
The roots in garbage collection are the objects that are directly accessible by the program.
And those typically are things like globals, stacked variables, and so forth.
Things that the program can directly address by name.
The live objects are reachable from the roots by following pointers, and the debt objects are the ones that the programmer can never get to again.
And so what we're interested in doing in garbage collection is finding all the dead objects and recycling them so we can use them again, so that they don't end up taking up space in our virtual memory and slowing us down by making our virtual memory space continue to grow and grow and grow across many disk pages.
So one of the questions, in order for one to be able to do garbage collection, is how do I know where pointers are in objects?
How do I know what's accessible?
And so typically, in order to do that, you have to have a very strong idea of where is a pointer and where is data?
Because you don't want to follow data as if it's a pointer, you want to know which regions of a given block of memory are pointers.
And so strong typing helps a lot with that, which is why mostly people do garbage collection in strongly typed languages, because then you can actually identify it.
It also requires you to prohibit pointer arithmetic.
So the ability to take a pointer and then index off that pointer means I can get to some piece of storage that I can't access directly.
And so typically in these kinds of systems, pointer arithmetic is illegal.
So you can do array indexing, as long as you treat the array as a single object.
You can't treat an array as the component pieces.
You have to treat it as a single object.
So pointer arithmetic.
Well, you folks, who's familiar with pointer arithmetic?
At this point, almost all of you have done stuff with pointer arithmetic, if your code is any good.
Almost all of you have done adding to pointers in order to move through an array or what have you as being somewhat cheaper than doing an array index every time.
It saves you an extra memory reference, typically.
So that's one of their restrictions when you want to do garbage collection in a general setting.
Now probably the simplest and most useful one as a programmer to use for garbage collection is what's called reference counting.
I know many, many programs where what you do is you do reference counting because you can.
But as you'll see, it has a limitation.
So the idea in reference counting is to keep a count of the number of pointers referencing each object.
So here we have a whole bunch of different objects of different sizes, and whenever there's a pointer going to the object, I add one to the reference count shown in green at the top here.
I hope I got the count right.
Whenever the count drops to zero, you free the object.
That means nobody's pointing to this object [UNINTELLIGIBLE].
Let's free it.
So let's see how that goes.
So suppose that, for example, that pointer got moved to over here.
Went away from there, got moved to there.
Well, now there are two fields that have to be updated.
The one that we took the pointer away from and the one that we put it to.
So we update those.
Oops, that guy went to zero.
That means this stuff is garbage.
But I can't just immediately go and collect that piece of storage.
Why not?
To save the value of the pointer, and [INAUDIBLE] pointer later?
Yeah, to save the value of the pointer--
To save the value of the pointer and then use it again later.
The pointer isn't there, but you know where--
Not quite, not quite.
Yes?
[INAUDIBLE] the reference counts for [INAUDIBLE], and also--
Yeah.
If you're going to deallocate this guy, he's got pointers.
They have to follow his pointers and see which ones need to be deallocated.
Good.
So this guy now, we realize, is garbage.
But before we can just free him, I've got to deallocate these pointers.
So this guy also got decremented, but he didn't.
So this guy, we have to decrement these two fields, because we took away those pointers because we're going to garbage collect that.
And when we do that, it turns out this thing turns to garbage, as well.
So when you do reference counting, you have to make sure that when you decrement things and decide you're going to free something, you also then go through and free the things, and you continue that process until everything that has been deallocated can be deallocated.
And then this becomes recyclable, put back into the storage, into your free list or whatever scheme you have.
So this is a very simple way, when you're implementing data structures, of building your own dynamic storage.
Because you basically can allocate stuff, you can free it, and whenever the reference count goes to 0, boom.
You just have it deallocated.
OK?
But you can program that yourself [INAUDIBLE].
Now there's a problem with the scheme, though.
What's the problem?
Cycles.
Yeah, cycles are the problem for reference counting.
So here's an example.
The problem is that a cycle is never garbage collected.
Let's take a look at this structure here.
So everybody's got a pointer from somewhere, but it turns out that there's a cycle here where, notice that they are pointing to each other, we even have a guy coming off here, but nothing points into the cycle.
So there's no way that, from the roots, I can ever access these four guys.
They're garbage, but my reference counting scheme will never find it, because no one's ever going to decrement one of those counters here.
So those are all garbage, but how would I know?
And that's basically the disadvantage of reference counting.
And uncollected garbage, as we all know, stinks.
So we don't want that situation to occur.
So nevertheless, reference counting is great if you've got an acyclic structure.
And the number of times acyclic structures come up where reference counting is the simple and easy way to do the storage management is huge when you get into doing any kind of interesting programming.
Lots and lots of opportunity to use reference counting approach.
So questions about reference counting?
You have a question?
Yeah?
So this means you actually have to do this [UNINTELLIGIBLE] ever do a [INAUDIBLE]?
You can, like, save the work--
That's right.
Whenever you move a pointer, if you're going to move a pointer, you first decrement the counter on the place you're moving it from, and then garbage collect if need be, and then you move the pointer-- that's typically what you do-- you move the pointer and increment the count at that location, which typically doesn't require any extra work to decrement, which is the trick thing.
Sorry?
There's no good way to do this in idle time.
Like I'll have some idle time later, let me do it--
So that's what you use true garbage collection for Yeah?
You could actually, there's techniques where you can defer the work of incrementing it.
Like you basically put the increment and decrement operations that you'd like to do into a buffer, and then you can do them later.
But it's sort of the same
Yeah, you could log.
So there are schemes where you log that, oh, I decremented this, better check it later, type thing.
Or I decrement it to 0.
Let me just put it in there, and I'll go actually do the stuff in batch mode at some later time.
Yep.
So programming is fun, because you can be clever with schemes like that.
And that could be quite a reasonable thing to do, which is wait until you're waiting on the user for input, and at that point, collect your garbage.
Now, the thing you have to risk is, what happens if that takes a long time to get there?
And then there's the complexity of your algorithms and so forth.
So you get into a lot of issues of software maintenance and so forth.
But that can be a very effective thing, is take it out of the critical path of the actual complication you're doing, if that's mission critical, and put it in some part of the computation where it isn't mission critical.
Very good idea.
I'm sorry?
For thread safety.
Like if you have shared reference accounts, like multiple threads updating the counts, you don't want to have them all in the same count at the same time.
So they fill it like a thread local buffer, and--
I see what you're saying.
Yes.
So we're going to get into that as we deal with multi-threading and so forth, that if you've got two guys that are operating on the same data structure, you have to be very careful about the consistency, and that one doesn't do something while the other is doing something else, and so forth.
So for that reason, it can be a good idea to postpone operation.
So general garbage collection is based on a graph abstraction.
So let's move into the graph world.
So I can view objects and pointers as forming a directed graph G equals VE, where V are the objects and E are the pointers from one object to another.
The live objects are reachable from the roots.
And now the thing is that finding all of the live objects is then like doing a graph search.
The common ways of doing a graph search are depth for search and breadth-first search.
It turns out we're going to use breadth-first search for reasons that I'll show you.
So the idea in breadth-first search, to find all the objects.
So the idea is, what we're going to do is go through and find, where are all the objects in my system that are reachable from the root?
And then anything that I didn't find, ah.
That must be garbage.
So here's the idea.
What we're going to use is FIFO queue called Q.
So that's a first in, first out.
It has a head and a tail.
And the mechanism is very much like a stack.
Whenever I need to enqueue something, I put it on the tail end and increment the tail pointer.
Whenever I need to take something off the queue, I increment the head pointer.
So I'm always doing head and tail, pushing on the tail and pulling off the head.
Enqueue and dequeue.
So the way that this works is, I basically go through all my objects, and if they're a route, I put a mark on it.
And I enqueue-- this is pseudocode, by the way, so don't hold me to my C programming standard here.
And I market, and then I enqueue it on the queue Q.
And otherwise, I just simply mark it as zero.
So being marked means whether we've discovered it's reachable from a route.
So to begin with, what we're going to do is mark everything that's reachable from the routes, and we're going to enqueue them on the FIFO.
And everything else, we're going to mark as unreachable, for now.
And now what we're going to do is see which ones we discover.
So what we do is, while the queue is non-empty, we're going to take something off of the queue and then look at all of its neighbors.
So all the edges UV, look at all the things that go from U to V And if it's not marked, then we're going to mark it and put it on the queue.
That's it.
If it's already marked, we don't have to do anything with it, but if it's unmarked, we'll mark it and do it.
And then in the end, everything that's marked is going to be something that is live, and everything that is unmarked is going to be things that are dead.
But it's going to be actually even sexier and more clever than that, as you'll see.
So here's an example.
We start out with my queue being empty, with head and tail pointing to the same place.
And what we do, is we go through and we first-- in this case, I have just one route R, right here.
So we basically mark R and put it on, and that's how we get started.
And everything else is unmarked up here.
So I'm going to mark with green up here, and I'm going to show what's on the queue with the green down here, and dark green when I pop something off.
So the first thing I do is I say, OK. Let me dequeue something.
And in this case, I'm dequeueing R, because it's the only thing on there.
And what I do now is I visit all the neighbors.
So I visit b, so I put that on, mark it and put that on, and then I visit c, so I visit and put that on.
Then I'm done with R. I visited all its neighbors.
So then I go and I dequeue B, and now visit all its neighbors.
Well, in [?
visit ?]
c, there's nothing to be done there, because c is already marked.
So I don't enqueue it again.
I already have observed that.
And there's nothing else adjacent to b, so basically, I then go on and dequeue c. So I have c here, and now we go visit its neighbours, and the neighbors are d and e. So we enqueue d, enqueue e, and now we are done dequeuing all his neighbors and marking them, so now we pop off d. And now d has no neighbors, so there's nothing to be done.
And now I dequeue e, and basically, e has one neighbor f, so we mark f. Then we dequeue f, we mark g, then we dequeue g, and g has-- the only neighbor is e. And now my queue is empty, and so I'm done.
And now you'll see, what did I do?
I marked all the things that were reachable from r, and everything else is garbage.
Now, that's only half the story.
So I've marked them, but more relevantly, it turns out for garbage collection, is look at what I've got in my queue.
What do I have in my queue?
I've got all the live objects, got put in the queue as we were going along.
All the live objects are put in the queue.
And not only that, but notice they're all adjacent to each other in the queue.
They're compact in the queue.
And so the idea is, we're going to take advantage of that property.
That when I do breadth-first search, that the queue, when I'm done, I put every object in there.
So all the live vertices are placed in contiguous storage in queue.
So this is the principle behind the copying garbage collector, which is one of the more popular garbage collectors that's used.
The idea is that as I'm executing, I'm going to have some live objects and some dead objects, and I'm going to have some place that I'm doing my next allocation at.
And as I go through, I allocate more things in what's called the from space, and I keep allocating, and I may do a deallocation, and maybe some more allocation, and maybe another deallocation, and some more allocation, and eventually I run out of my memory that I've assigned for my heap storage.
So when I get to this point, now what I do is I'm going to do breadth-first search on the from space.
And what I'm going to do is I'm going to copy using the live storage here, using [UNINTELLIGIBLE] to a TO space, where the TO space implements the FIFO queue.
So here's the TO space, and when I go through and I walk all that, I've now copied all of the values, all of the objects down to this space over here.
And now I have this amount of storage left.
I make the two space exactly the same size as the from space.
I have to make it at least that big, because I know that it may be that everything here is alive.
Maybe nothing got deallocated.
So I have to make sure that the TO space is large enough to handle everything that might be alive there, and it may be everything.
So the TO space is generally allocated to be exactly the same size as the FROM space.
So basically, the way I compacted is I copied things down using the breadth-first search algorithm.
Now there's one problem with this, and that is that we have pointers in these objects.
And when I copy it down here, how did I make sure that all the pointers now point to the new locations?
So here's how you do that.
So since the FROM address of the object is not generally equal to the TO address, pointers must be updated.
So I have to know, of course, where the pointers are.
So the idea is that when an object is copied to the TO space, what we're going to do in FROM space is store a forwarding pointer in FROM space saying, I copy you to here.
So that whenever I look at something in the FROM space that's already been copied, I can say, oh, here's where it's been copied to.
It's been copied to this region in the TO space.
When an object is removed from the FIFO queue in the TO space, we're going to update all its pointers.
I'm going to give you an example of this.
So if you think about it, when I remove something from the FIFO queue, at that point, I know all of its adjacent vertices have been placed into the queue, into the TO space.
And so at that point, I know where all the final destinations of pointers are.
So here's an example.
So suppose that I'm doing the copy of these guys, and I've got all the pointers around here, and now some of these guys have already made it into the FIFO queue.
Let's say this guy has made it in, because I've got forwarding pointer to there, and this guy has made it in, so I've got forwarding pointer to there.
And this is what my queue currently holds.
So when I remove an item from the queue, I then look at this guy, and what I do is first of all, I visit all of the adjacent neighbors.
So I visit this guy and I say, oh, he's already been copied, because I have a forwarding pointer.
So nothing to be done there.
But this guy, he hasn't been copied yet.
So I have to make sure that that guy is copied by enqueuing the adjacent vertices.
So I copy my neighbor to there.
And part of copying is, if I'm actually making a copy, I'm going to have a pointer from here.
The pointer now has to point to this guy over here, because that's how I'm copying.
Otherwise it isn't a copy.
So in the copy, this pointer is the same as that one, is pointing to the same storage location, namely this place over here.
So I do that for all my adjacent vertices.
People follow so far?
Now I've copied all of the neighbors for this guy.
Now this guy, all of his neighbors are in their final destinations in the TO space.
So that means I can now update these pointers to be the pointers in the TO space.
So I do that as follows.
Oh, first of all, I forgot to place the forwarding pointer here to indicate that he's been copied.
That's how I'm doing the marks.
Sorry about that.
So I copied the values, and that's just a straight copy, because all the pointer values are now pointing into this space.
Then I put a forwarding pointer in.
Now I'm ready to do the update of the pointers here.
So I update the pointers to the removed item.
So here's one pointer.
I get rid of him there and I make him point to the forwarding thing.
He used to point to here, now he's pointing to here.
This one used to point to here, I want to make him point to there.
There we go.
So now he's pointing here.
And now this guy here now has his pointers to his final destinations.
And so the invariance that I'm maintaining is that everything before the head of the queue all have their final pointers in place in the TO space.
Question?
So do you know where to put the pointers-- how do [UNINTELLIGIBLE] pointers go from the TO space to the TO space so that they're actually getting to their final destination?
So my question is, [UNINTELLIGIBLE] by following [UNINTELLIGIBLE].
But how do you know what pointer to follow?
How do you know that [INAUDIBLE]?
If I have a pointer to something in the FROM space, then I update it to be the forwarding pointer in the TO space
Yeah, the question is, how do you know that [INAUDIBLE] it's being pointed to [?
the ?]
actually in the TO space?
Can you check, is this in the TO space?
Well, yeah.
I mean, you know what the range of the two spaces is, so you could check that.
OK.
Question?
So the reason you would be using [?
BFS ?]
instead of [?
DFS ?]
is because [INAUDIBLE]
The reason you're using BFS is because it has this nice property that the queue represents the copying of all the values.
If I use DFS, I need a stack to do it, and then I'm overwriting the locations where the vertices would be stored in the data structure.
By using DFS, there's this just clever thing that the queue, when I'm done with DFS, represents all of the visited vertices.
It's very clever.
Wish I'd thought of that.
But I was, I think, only 10 years old or something when they invented this, and I wasn't that precocious.
OK?
So very, very clever.
So this is the scheme behind the Minsky, Fenichel, Yochelson garbage collector.
So Marvin Minsky, you may have heard of, was one of the inventors of a scheme like this.
One reason he's so famous.
Good.
So this is really nice, because it takes me linear time to copy and update all of the vertices.
It's just one pass of breadth-first search.
Very fast to copy and update all the vertices.
Sometimes, by the way, people call these forwarding pointers, they call them broken hearts.
You'll see that in the literature.
They call them a broken heart if it's-- I don't quite remember why that's the-- it's like, I went there, oh, you're not there, it's over there-- it's like, I don't know.
Now, the last thing you have to do-- and this is something that's not talked about a lot in the literature-- is how do you manage these from and to spaces in virtual memory?
So here's the strategy that works well.
So basically, typically what I'm doing is after I've done a copy, I've emptied everything out of the FROM space, and I've got a portion that's used in the TO space, and then an unused portion that I'm going to be able to do allocations out of for the next pass around.
So what I do at that point is I make the old TO become the new FROM.
So here's the new FROM.
This is available space.
And now what I do is I extend this space, if necessary, to make the new heap space that I'm going to allocate out of equal in size to the used space.
And the reason for doing this is to amortize the cost of going through all of the storage, so that the cost of going the garbage question can be amortized against what's used.
If I didn't extend this, I might have only freed up a tiny little bit of space, and now when I run out of that space, I'll run the garbage collector going over all the elements, but I'll only have gotten a few elements [INAUDIBLE].
So what I want to do is make sure that I've done at least a certain number of allocations proportional to the number that I'm going to have to walk over in order to finish it.
So that's why you make this be the same size.
Now when the space runs out, the new TO is allocated with the same size as FROM.
So if you can allocate it back in the old FROM space here, that's great.
If there's room here to allocate it at the front, super.
That's where you allocate it.
But as in this example, the FROM space here is bigger than the space that goes up to here.
So therefore what I do is I allocate it at the end, and then I go through and copy to here.
Now if this ended up being used, then the next time around, I would be able to reuse this space at the front for the next time that I flip the garbage collection.
The idea is, if things will fit as early in memory as possible, that's where I put them.
But if they don't fit, I put them later.
And you can prove, if you do this, that the virtual memory space used is order 1 times the optimal, where you're getting nearly the best virtual memory space used compared to the amount of items that are in use by the user at any time.
So it keeps everything sort of compact towards the front end here.
So in this case, I had to allocate to here.
If the use had been very small, I could have allocated just a very small amount for the heap.
When I was done, the new TO space would have fit at the beginning.
I would have allocated it at the beginning.
And it's a nice little exercise, for those of you who are analytically inclined, to figure out what the constant is here, because it's a little bit of a game to figure out what the amortized constant, for those who have some algorithmic energy and our confident about working in this kind of thing instead of studying for the quiz.
So dynamic storage allocation.
Hugely interesting area.
Lots is known and lots is unknown.
For example, I mentioned coalescing.
We really don't understand coalescing.
Disaster people have been doing it for 50 years or more.
We do not understand coalescing.
There are a lot of strategies that people have studied, things like the buddy system, I mentioned.
There are other types of garbage collectors called mark and sweep garbage collection.
Those are typically inferior to the copying garbage collection, because they leave you with a memory that isn't compact.
And so you end up using more and more of your virtual memory, because you're leaving things in place.
On the other hand, you leave things in place so there's no need to move pointers.
So that can be a valuable thing.
There's things called generational garbage collection.
Generational garbage collectors realize that things that are allocated typically are freed very soon thereafter, and so therefore, if you especially take care of the things that are allocated and guess that they're going to be used again, and that things that haven't been deallocated for a long time are very unlikely to be allocated, you can get better performance out of a garbage collector.
There's real time garbage collection.
So this is what I've explained, is what's called a stop and copy garbage collector.
Meaning that when you run out of storage, oops!
Stop the world, run the garbage collector.
There are real time garbage collectors which, whenever you do an allocation, it goes and it does a few cycles of garbage collection, and is such that you can always keep up with the allocations and deallocations going on.
By doing a few cycles, you eventually get to garbage collect everything, but you do it intermixed, so that you're slowing everything down by a constant amount, rather than running and suddenly, OK, your robot arm is going like this, oops, garbage collect, garbage collect, garbage collect, [CREAKING SOUND].
So you can have your robot arm move smoothly, because it's paying the price all the way.
It's not stopping and going.
There is multi-threaded storage allocation.
What happens when you have a multi-threaded environment where you have many processors that are all allocating out of the same pool?
How do they organize it?
You can have the interesting things going there where things are allocated on one processor, and then get deallocated on another.
And if you're not careful, you have something like a memory leak going on, which is called memory drift, where the memory gets allocated on one, and one guy's busy creating it, and it keeps getting deallocated on the other, but never gets recycled back to the guy who needs it.
And so how is it that you figure out to make sure that memory drift doesn't end up having you grow your memory?
In fact, many, many garbage collectors that are out there have this memory drift problem for multi-threaded storage allocation.
There's also parallel garbage collection.
How do you, in a parallel environment, have the garbage collector going on as a separate process while you have other things going on at the same time?
So a huge, huge, huge area of storage.
And for our next lab, lab 3, you'll be building your own storage allocator.
So you have a lot of fun with this.
[?
Reid ?]
says this is his favorite lab.
Great.
Any questions?
OK. Good luck on the quiz next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So hopefully by the end of the class, we will show you a histogram for the quiz.
We are very happy.
You guys did really well, so we feel like, actually, you learned something in the quiz, so it makes us happy.
We're hoping to have the histogram ready by now, but we don't.
By end of the class, hopefully we take a break in a middle, and go through rest of that.
So we are going a little bit off from regular programming.
If you look at the class schedule, we are going to have a guest lecture today, but I am the guest lecture, I guess.
So I'm going to talk about what compilers can and cannot do because, as you went on last couple of projects, you tried hard to do weird things out of the compiler, create different piece of code, [UNINTELLIGIBLE] programs.
And so I'm going to kind of, first, walk through some stuff that is very practical, what the current GCC do or don't do.
OK, so we'll go through some stuff, which is interesting, and then I'm going to-- let's get the outline first.
OK, this doesn't work.
OK. Then we will talk a little more about where the normal optimizations happen, what are all the possibilities, just very quickly, to give you a feel.
It's not all static type compilation.
And then go through two things.
One is data-flow analysis and optimization.
That's a big part what compilers does.
And then also instructions scheduling.
Instruction scheduling might not be that important on a superscalar.
but it's always good to know what it does because there are some cases where you had to do it in the compiler the hardware cannot do, so we'll look at some of those cases.
So the first thing that we found a lot of you guys did is you try to inline a lot of code in your projects, and in fact, one way to do that is have you do a macro.
A macro basically make sure that definitely get in line, but macros are ugly.
There are many things you can't really do with a macro.
The other thing is, we can actually do simple max calculation using function.
OK, starting function defined.
And the first one is calling this one.
Second one is calling this one.
So what do you think this is going to do?
How many people think that the code produced between this and this going to be drastically different?
Code produced for this function versus this function going to be drastically different?
Real different.
For some people, [UNINTELLIGIBLE] going to be different.
OK, why do you think it's different?
So the second function calls the first function, and the first [INAUDIBLE] calls the second one
Yes
So since you have the first one calling max1, which is a macro, the compiler just copied [UNINTELLIGIBLE] out of there.
Whereas the second one-- it will try to actually optimize the-- Won't it try to optimize what is inside un64 [INAUDIBLE]?
So what he thinks is that is just going to get copied.
This will be still a function call, try and do some optimizing.
In fact, what it will do is, the first function is going to get inline.
Something interesting here, what you see is even though there's a condition here, there's no branch.
They have the same old instruction that basically can be used to do a conditional [UNINTELLIGIBLE].
So instead of having a branch and having a pipeline [UNINTELLIGIBLE], this managed to convert this into nice, direct call.
The interesting thing is the second one is also identical, so what it did was it said, hi, I know this function, I can inline.
It got inline automatically.
You didn't tell you to do, the compiler actually inlined it for you, and then did all the things that are necessary.
So what that means is you don't have to write some of these ugly macros and hand inline.
You can have nice functions in there, especially if it's in the same file, it can get inline.
Of course, if you define it to different file, and this file doesn't have access to it, it won't.
But if it is the same file, it'll get inline, so you get the same result.
So you can still have this nice [UNINTELLIGIBLE].
You don't have to be with optimizations at that level.
So another thing is we have this entire-- Question?
On the previous slide, other than looking at the assembly code, how do we know when the compiler based this on?
You've got the assembly code.
Question?
Why do you prefer static converge versus inline?
So the reason I did static is basically, I want to make sure it's not visible outside the file, so if you don't make it static, what'll happen is everybody else had the visibility and then you kind of pollute the space with a lot of names.
So if you give static, it's only within you.
If I had inline I can't ask it to forcefully do that, but what I'm showing is you don't even have to say inline.
It'll inline by itself.
Question?
[INAUDIBLE] this function to be inline without actually [INAUDIBLE]?
Actually, gprof, what'll happen is the file will vanish from gprof, isn't it?
The function will vanish because gprof will basically, if you have a function, you go look, you don't see the function.
And it might give, even, a bad impression that OK, that function is not important, so that you had to be careful in that because when you look at gprof, you will see some files.
The inline find some functions, inline function won't be there.
Am I right, or is it doing anything interesting to the samples?
No.
[UNINTELLIGIBLE] vanish and oh, yeah, that function is not important, but in fact, it might be really important, but it got inlined.
So that's one way to, a little bit, worry about.
Does it make sense?
OK.
So we learned bithacks.
So it was really fun.
We are learning all these interesting bithacks, but the interesting thing is, in fact, GCC compiler also knows a lot of bithacks, so it's also a pretty smart compiler.
So if you have something like this, what do you think the smd would be?
[INAUDIBLE]
Yeah, it actually did this one.
This is very interesting because what it did was it didn't do shift.
It did this load effective address quad instruction, leaq instruction.
What it does is multiply these two, and it's add.
It does nothing in here to add, and then it can add a constanant.
So this very complex address mode that is being used for completely different purpose.
So this [UNINTELLIGIBLE] address.
It's doing this multiplied by this, there's nothing to add, there's no offset to add, save it here.
So it's actually doing this multiply-by.
The nice thing is it doesn't affect any things like condition codes and stuff like that, so this is a fast thing to do.
OK?
Actually, that's interesting.
How would [UNINTELLIGIBLE] 43?
That makes it a little bit more complicated.
What do you think?
How to multiply something by 43?
Anybody want to take a guess [UNINTELLIGIBLE] multiply 43 [UNINTELLIGIBLE] mul43?
[INAUDIBLE] multiple by 32, have that multiplied by--
So it did something interesting.
So here's what it did.
I want you guys to stare at it a little bit and see if you can even figure out what's going on here because this is kind of funky.
So what happens after the first leaq?
What's an rax?
What's an rax after first leaq instruction?
Anyone take a wild guess?
Five?
Five.
Yes, exactly.
What it does is it's 4a+a because this is a+a, 5a in here.
And then after here, what happened?
4 times this one plus this one.
Yeah, it's 21 because 4 times this is 20, and you add the whole original rax 21 here, 21.
And then you do this time again 42, and add another one, 43.
So it did all this interesting three instructions to get to 43.
And so this is fun.
You can spend days giving different meaning to this and see what the compiler generates.
And I notice at some point say, when do it give up?
And it doesn't give up for a while.
It start creating some crazy things.
OK, try one more thing.
OK, this has to be hard.
255.
How did it get 254?
There's no easy way to do that.
How did that to that?
This is the very interesting thing in here, so I will show you this one.
So here is that instructions you generated.
So what it did was, it's doing here, getting it 2a, by doing leaq because since it didn't give a multiply, it just multiplied by 1, so it's just adding these two.
You get 2a.
And then it multiplied by 128 by doing a bitshift here, and then it got such a wonderful thing.
Why did it do this instead of doing one instruction to 256?
[INAUDIBLE]
Yeah, then it went and subtracted a 2a again, and got 254.
So it went overshoot, subtract, and subject one more, so [UNINTELLIGIBLE] got calculated here.
It does a really smart way, I don't know what complex logic is there, to basically figure out these combinations of instructions that can do multiplication.
We planned on a bunch of things until we run into a couple of thousand.
It was doing these weird patterns after a couple of thousand.
Especially if it is to close to 2 to the power, it easily find it.
Even primes, it managed to find.
Actually, before we gave a prime, and it found it because it found the closest thing, and a couple of things [UNINTELLIGIBLE], and you can get to the prime.
So it's kind of interesting how find goes in just [UNINTELLIGIBLE] multiplies
[INAUDIBLE]?
So this is a very interesting question.
So what it's doing is, it has realized somehow that doing a direct multiply by 254 is going to be slower, so the multiply instruction-- if you go, I think you can also look at multiply instruction, how many cycles it's going to take to come--
[INAUDIBLE]?
Oh, because it needs to get this 2a again.
So it's [UNINTELLIGIBLE], use the two 2a.
So by calculating that, it kept it.
I don't know, you might be right, because if it [UNINTELLIGIBLE] 2a to 6n2, then there's no dependency, and right now, there's a dependent change here
[INAUDIBLE]?
Twice, yes, something like that.
Or calculate this separately.
2*8 and 256, and then add and subtract them.
Might be interesting.
So there are other ways of doing that, so in fact-- I don't know why, it might be in because [UNINTELLIGIBLE].
I don't know why they didn't do that.
But it's thinking.
It's thinking, look at all instructions, sequence, how long it would take, and it find this interesting sequence.
So sometimes when you are looking into optimized code, they will look like something crazy that you can't read because it does things like this.
So you found a simple multiply [UNINTELLIGIBLE], and end up with a piece of code like this.
And so you need to kind of decipher, seeing what's going on backward.
So that's why reading assembly is sometimes hard, especially optimized assembly.
OK, so I did absolute value, and look, it did the bithack we basically learned in class.
You can go look at that.
This is the entire thing that Charles talked about to find absolute value.
It knew that, so it has attended Charles' lecture.
That [UNINTELLIGIBLE] programmer.
OK, so here's interesting thing.
So what I did was I am doing update, and I have this big large array here, and I'm checking the index to be within 0 and this value to before I updated, because I don't want to write out the bounds on this setting.
OK?
Makes sense, because I want to make sure that I write in the bound.
Interesting thing here is I am doing two checks, here.
I end up doing only one check here.
What happened to my other check?
[INAUDIBLE], how big was the array, and [INAUDIBLE]
No, no, [UNINTELLIGIBLE] something a lot more simpler.
So to give you a hint, this is unsigned value, and I'm doing unsigned compare.
What happens if the value is more than 0?
A signed value that's smaller than 0 is what it is like in unsigned.
It's a huge number.
It [UNINTELLIGIBLE] to be bigger than this one, so because of that, it can just say, OK look, I don't have to check that.
I can unsigned compare, and I will get anything less than zero also in there.
So one more thing.
So before I continue with this one, so if I actually put it in a loop and say I'm going to trade [UNINTELLIGIBLE] to this value in here.
Then what it'll do is, at that point, it'll inline this.
And when it can complete [UNINTELLIGIBLE] near the check, it will know that, in fact, my bound is going from 0 to this one, so I don't have to check the bound.
So what this did was this inlined this function in here and completely get rid of the checks completely, and this is basically the branch condition in here because it said, OK, look.
These things are redundant because I know because I'm trading from this to this value within these bounds [UNINTELLIGIBLE].
So it is smart in that.
Do you see this, how this is going?
Cool stuff the compiler does.
So this is why compilers are smart, when they are smart.
And the interesting thing is, here's another one.
So now see [UNINTELLIGIBLE] imagine, because less than 0, I can do that.
How about if I'm checking from 5,000?
So we generated this funky code.
It subtracted a 6 here in the value, and then checked for [UNINTELLIGIBLE] because it kind of shifted the value to a 0 basis, basically.
And then you can check that thing, and then can basically get two conditions down to one.
See, the thing is there are many places where bound checks is very important.
If you are doing a lot of adding compilation stuff like that, if you don't want to have buffer overflows and stuff, you want it with bound checks.
And so optimizing bound checks is a very important thing.
So having these kind of things can, in many programs, probably give good performance, so that's why compilers are really good at it and spend time trying to do bound checks.
So this is kind of interesting way of doing that.
So the next thing I want to look at is vectorization because all these machines we have have this as the instructions, that can run really fast, and you probably saw it in there, and see what kind of code will get produced after doing something like that.
So here's a simple program.
So I have two arrays, and I'm just copying A to B, something very simple.
And also the other thing to notice, I know exactly from where to where I'm copying, and I also know which arrays I'm copying, so when you look at it, it produces a code like this.
So what it's doing is it's basically making eax0 here by doing xorl.
And then basically, moving the value A into the xmm registers, much larger.
Instead of having to [UNINTELLIGIBLE] 16 of them, and copying it back into B.
So basically, you are doing copying here, an increment by 16.
By now, every refresh, you're coping 16 of them.
Why could I just be done with just putting this small piece of code?
What additional information this is taking advantage of?
Does it know that [INAUDIBLE]?
Exactly, because it knows that it goes from 0 to this value.
In fact, it knows it's a multiple of 16.
So it knows that.
That's why it do that.
It knows exactly, and these things are nicely aligned to the boundaries, word boundaries.
So it knows that.
So I can read that, and I know all those facts, and that is why I can do this computation.
So now, you start doing that, did one simple change.
You start going from value, I went to end.
0 to end.
I know where it starts, and I know where it's ending.
Ending is somewhere at N. I don't know where the end is.
Then this has to do something a little bit difficult.
So the code produced looks like this because, now, I'm not going to go through this code.
The only thing to say, this is actually doing still a memx instruction, but its trying to make sure that because N it might not be a multiple of 16.
You have to take care of the final number of iterations outside that, so you had to go up to the multiple, and then basically do a normal loop one at a time.
So as we produce a little bit of a more complicated piece like that.
So that's [UNINTELLIGIBLE] compiler has to do.
And so then you have a piece of code like this.
The interesting thing here is, now, I created basically a function, where it's not A and B. I'm giving two arrays as arguments, and then I'm giving a size to copy, and I'm copying that.
And I would have an extremely complicated thing that's getting generated.
Why is it complicated?
What do I have to know, when I get this function, to make sure that, first of all, it's still doing xmm somewhere in here.
What that means is it's trying to do this very fast copy of a multiple using [UNINTELLIGIBLE] instruction.
But what else can happen in this function?
Because compilers delete all the cases.
What are other cases tests deal with?
May not be aligned
May not be aligned because, for example, because xmm assumes that they had the word boundaries.
When you read 16 bytes, we assume it's aligned with 16-byte boundary.
It might not be aligned, so you have no idea where these two are coming from.
So that's one thing is they might not be aligned.
What else?
They don't even have to be a [INAUDIBLE].
Because I mean, that thing could just be copying up to N. But it might just be copying partially parts of the array
Yes, yeah, that's true.
So what that means is because arrays [UNINTELLIGIBLE] it, but arrays is somewhere in memory, just you do two-point to starting point.
That is what x and y are there, two starting points in main memory, and start copying there.
So what else can happen because of that?
Because in A and B, we knew they were two separate arrays.
What else can happen?
Back there
[INAUDIBLE]
Yes.
So arrays can start to overlap, and if arrays are overlapping, then you might end up in an interesting situation.
So you have to figure out if that arrays are overlapping, whether they're aligned.
Actually, there are two types of aligned.
One is self-aligning, so assume we took these arrays and start copying from the byte 3.
So then what you know is basically byte 3 to 16, it's not aligned.
You can't start copying chunks in there.
So you run bytes, but when you run up to 13 iterations, then you end up in, again, aligned chunks.
So in that case, you just run the sum preamble to first aligned place, and then you go to aligned chunk.
But if this is starting to at 3, this is starting at 8, then they're never going to be aligned.
The things are copying A and B are not aligned, so then you have to treat it differently.
So there's a lot of different cases you have to do, so if you just give something like this, the problem is the compiler has to deal with these thousands of different cases.
And in this one, since this small, it probably tore through all the possible cases at the [UNINTELLIGIBLE].
So it's dealing with all those, and checking over everything and trying to find optimal case and do that fast, hoping that you get optimal case for it.
But if you have something more complicated, the compiler won't be able to do all of those things, so it might give up.
So the interesting thing to here note is more information to go into the compiler is better.
And then here, compiler has to divide a lot of things, but probably not happen.
Another interesting thing is now, the first time I just copy to where is A to B, in memcpy4, I just call memcpy3 by doing the same thing, A to B, copy 1024 [UNINTELLIGIBLE].
This is the beauty of inlining.
So what it did was, in memcpy4, it inline memcpy3 and substitute X and Y to A and B and end 2,024.
And it realized that it doesn't have to do all these tests like it did.
What it generated is very close to what we got here because after inlining, it should realize, wait a minute, I'm copying A to B. I know we have the start.
I know we have the end.
I know the size.
I know all of these things, and I don't have to do any of these things.
I can actually generate this very simple piece of code.
So I think that is a neat thing.
What this shows you is, in fact, if you can build this general function of things in there, and then you can call them, and if it is done right, the inlining will basically do all optimizations.
So you don't have to have 50 different memcpies for all the different things in your code.
If you wrote a general function, and you call it in a way it can get inline and got that as efficient as possible as hand optimization.
I think it's a real interesting thing, and what does for you, when you're doing projects, you don't have to do all of these very complex and small functions, hand inline stuff like that.
But it's always good to check that, in fact, the compiler's doing that because you don't know.
You assume the compiler's doing that, and there might be cases it might not be.
And I will show you one example here.
So I want you guys to look at this function a little bit.
OK?
I am doing two memcpies.
I am copying 1,024 elements.
One, I'm doing ai+1, a into-- this is XY, this is X get copied into Y. ai+1 to A, so that means I have array like that, array like this.
I am giving ai+1 as the source.
I am giving this as the source, and I'm doing this as the destination.
OK, what does this copy do?
I'm copying 1,024, yes.
So the first one, this one gets copied to here.
Second iteration, this will get copied.
Third iteration, this will get copied to here.
What does it do?
Yeah, I just do one left-shift of the array.
My second example.
I give this as my first element.
This as my source.
This as my destination.
What happens here?
All your copies have the same number
Exactly.
All of them will copy the same number.
OK, so now, if you look at the code that's been produced, so the interesting thing here is it realizes, in this one, I can still do mmx because I can still copy, take a chunk, and copy it one back, take a chunk and copy it one back, take a chunk and copy it one back.
OK, do you see that?
But what does the next one do?
[UNINTELLIGIBLE] mmx is, and what does this one do?
Copying something from dl-- [?
movzdl ?]
array expressed dl.
Reverse dl, is this copied, this one?
I hope I copied it properly.
That doesn't look right to me.
So this is interesting.
So I might have missed [UNINTELLIGIBLE].
I think it takes-- This doesn't look right, does it?
So what's a bound?
It's char, I see.
One byte
Yeah, one byte
So what this is doing is just taking the first byte, and then just moving it
Into edx?
[INAUDIBLE]
Oh, right, this is actually doing the right thing.
It's doing the right thing, so what it does is this move this one into edx, entire thing, and then this calls the dl, it gets the first byte out of edx.
Do you see what's happening here?
So a gets into-- first [UNINTELLIGIBLE] the first byte out in here, and then you keep copying that byte one at a time into this location
So it's not smart enough to [INAUDIBLE]
So this is where we find something interesting in the compiler
[INAUDIBLE]?
So what doing is, in here, you are copying this byte into edx.
So dl is the first bite out of edl.
That's byte, because what--
The first byte
Yes, because except the six address themes, do you do r32e, r64e, 32, and just dl is just first byte [UNINTELLIGIBLE], but [UNINTELLIGIBLE] low byte of that.
d is just the higher byte
So why does it just take the first byte?
Because this byte get copied into everything here.
So do you see that?
This byte is the one that-- because that's what happened when this copy is basically this byte got everything [UNINTELLIGIBLE] got replaced by this first byte in here.
And you incorporate in there, and then it just goes around copying it in here.
So I try, I did it this way, so what I did was basically went from 1 to 1,025.
[UNINTELLIGIBLE] 0.
This is basically what happened.
So one 2,025, I got a 0, and then basically that's what happens in here, same thing here.
OK?
Do you see what's going on?
But I just did something else, so assume [UNINTELLIGIBLE] is doing right here.
What happens if I just use something else?
I use B[0], so it should be the same, isn't it?
If I use b[0] here, instead of doing A[i], this should be B[i], isn't it?
I'm sorry, where is [INAUDIBLE]?
Yeah.
It's a different array.
Sorry, I didn't put it here.
It's a different array.
So instead of A[0] here, I put another array, B[0], here.
Does it matter whether that's A[0] or B[0]?
B[0] is a different array.
It shouldn't matter because a different array, different element that you don't do that, but the interesting thing is if you do that, it managed to convert it into mmx.
So this is where the compiler is basically falling short a little bit because it could have done this for these two.
OK, it's the same thing because what it does is it takes this one element from me and copy it everywhere.
I could have done it, but for some reason, the compiler decided if using a different array, B[0].
I can do it, but I'm doing A[0] [UNINTELLIGIBLE], even though these two are basically identical except these are different.
I'm not doing any kind of [UNINTELLIGIBLE], anything.
Question?
[INAUDIBLE]?
Yes, so what does is when it goes somewhere, it's doing pac.
[UNINTELLIGIBLE] pac instructions in here.
What it does is it takes a byte and kind of makes multiple copies of the byte and create a larger copy than the 16 copies in there, and then it can kind of stamp it everywhere
[INAUDIBLE] because the A[0] would go off of A[1]
So what it's doing is copying A[0] multiple times one at a times slowly.
So what this does is it takes B[0], make 16 copies in there, in registers, not in memory.
I created a template of 16, and I kind of stamp it as I go
That's probably failure of its alias analysis
Yeah, it's failure [UNINTELLIGIBLE].
So this is where the compiler does some magic, kind of very complex things in here.
But somewhere in the compiler it failed.
so what you want in here is a code looking like this, but it produces this one.
So this is where the compilers are great.
It do some amazing things, but it's not infallible.
It can do, there might be corner cases that data analysis fails.
So that's why it's always good, even though you can take advantage of the compiler, to look at what's generating.
And sometimes when you tweak around it, you suddenly realize, wait a minute.
I can get the compiler do something better, and then you can say wait a minute, now how do I work myself back?
Sometimes, you end up changing your SQL a little bit like the examples, the TAs showed when they were doing their demo, that if you tweak it a little, you can actually get the compiler to do that, instead of trying to do these things by hand.
So the compilers are powerful, but you have to be careful.
Another interesting thing is this factorial here.
So normal factorial is basically you call a function call with x-1 and multiply by x.
But you know functions calls are very expensive, and in fact, GCC knows that, too.
So what GCC did, it basically eliminated the function call and converted it into its [UNINTELLIGIBLE] function here.
So if EDA got x in here, and then it goes to a loop, so it first check with [UNINTELLIGIBLE].
If it is one, you go to the end and return it.
You are done if x is 1 or less than 1.
And otherwise, it goes through a loop.
It doesn't go do any function call.
It just basically calculate this fact value inside EAX and keep multiplying [UNINTELLIGIBLE].
So it can take this simple recursive functions, and also convert it into [UNINTELLIGIBLE].
So it does some very, very fancy stuff in the compiler.
So the compilers are fun when they work.
But the key thing is there are many cases it doesn't work.
So next, I want to switch gears.
Any questions so far?
Sometimes it's fun to find breaking points in the compiler
[INAUDIBLE]?
If I use, in some-- this is not static?
Yeah
No, it won't make a difference because what it says is if it is not static, it's visible to the outside world, but within this function, it's the same.
So what it does, it kind of like limiting my pollution because otherwise what happens is everybody outside will see these names.
So if you use that name again somewhere, it might just use this one.
[UNINTELLIGIBLE PHRASE] this is within the file, nobody else should see this.
It's creating a local copy.
It's kind of a poor man's class heierarchy.
In Java, basically, each file is a single class, and you make sure that things inside the class is not visible to outside.
When you make static, you made it only visible within that file, so you kind of make [UNINTELLIGIBLE].
You can think about your file as your class, and so you limit the scope of the variable doing that.
So some benefits of object-orientedness can be [UNINTELLIGIBLE], I guess.
[UNINTELLIGIBLE] static variable, it's not a class variable.
OK, so next, before I get into doing compilers and say what compilers do, I want to give you a [UNINTELLIGIBLE].
There are many different places where you can do optimization.
So if you look at what happens in the program, program first goes through compile time, compiles each file, then it links all the files together.
At some point, the files will get loaded into your machine, and then it'll be running.
So if you load things in a compiler, you have full access to source code, it's very easy to kind of look at the high-level transformation, low-level transformation, you can look at the entire gamut of things to do.
And the nice thing about compilers, compilers can be slow.
Nobody's going to complain.
It's not going to be part of your run-time, so you just would wait, but it's you, not the customer.
But the problem with compilers, it doesn't see the whole programs.
You see a file at a time, so all this inline things and stuff has to be in the file.
You can't put in a different file and get [UNINTELLIGIBLE].
And also don't know the run-time conditions because that's run-time.
It might be having different inputs, different size of load and stuff, that I don't know any of those things.
And also, I don't know about the architecture, so if my compiler have to make sure that it works on AMD machines, Intel machines, stuff like that, of course, you can use special flags and try to comply for one machine, and breaks, you finish on the other one.
But you don't want to do that, so the compiler has to be a lot more general, and this can be sometimes problematic.
So when you're going to link, the nice thing is that this is a place you have the entire program available.
Sometimes, people try to do things like inlining in the linktime, because that means you know everything in there, so you went with couple different file I can inline it because I have access through here.
And still, there might be things that's not available, like dynamically-loaded classes and dynamic-loaded data, and so things like Java might not be available.
And of course, you don't have access to source most of the time
Sorry, sir.
What do you mean [INAUDIBLE]?
Do you have the full program [INAUDIBLE]?
But so how do you say that [INAUDIBLE]?
So dynamic links, if you have something like Java, there might be some data that kind of get dynamically generated or dynamically linked.
So when you're running, if you're running right [UNINTELLIGIBLE] your browser, all those Javascript classes and stuff like that, you don't have access to because those are coming in here.
So there might be places, things that it gets as it runs.
Not in C, but in other languages.
And the load is interesting time.
Here, load time is important because when you double-click, you want your program to appear fast.
You don't want it to take a long time.
But you have kind of access to all that code in here, and you have some idea about the run-time, also, the architecture, and stuff like that, what you have, not the run-time, but the architecture, exactly what machines you are running.
And then, of course, you can do it run-time.
The thing about run-time is you have full knowledge of everything, it's great, but every clock cycle you spend optimizing is one clock cycle you take away from the program.
So things like Java JIT compilers, they try to do minimal things, so very fast things.
It can't do a lot of complicated things because it's too expensive.
OK, so we're not talking about any of these things any more, but it's always good to know, as you go about using Python or Java or JavaScript and stuff like this where is this thing happening to my code?
Because it might not be all compile-time stuff.
It might be happening at different stages.
So you need to know who's actually mucking with your code, and know that there are other people who can muck with your code.
So next, I want to switch into dataflow analysis.
So this is what compilers are good at, and compilers try to do all the time.
So it's basically compile-time reasoning about run-time values and variables, or expressions, within the program at different program points.
OK, so that means compile-time, I need to know I have this program point, what could it be.
So things like which assignment statement produced a value or variable that I am using?
OK, if I use a value, who actually created that value?
Or which variable contain values that are no longer being used by somebody here?
So that means I am trying to analyze the program and watch the range of values that each variable can have.
So the key thing here is this has to be true for every possible input at every possible execution.
Normally, [UNINTELLIGIBLE], and this time, I know why my variable [UNINTELLIGIBLE], but every possible time, this has to be true.
OK, if there's a condition that something can happen, you have to make sure that condition is not going to break your program.
Last thing you want from optimizer is to basically start producing different results.
Even [UNINTELLIGIBLE], it's not good, so you want a compile optimizer to kind of produce the same result that you got without optimizing.
And this is why this has to be [UNINTELLIGIBLE].
So first, I want to go through a little bit of example, what kind of things the compiler do.
You probably have seen this in one of the earlier lectures.
We talked about some of this as hand optimizations, but I'm going to go through some of them by using this program.
It doesn't mean anything what I'm doing here.
I have a loop here.
I'm calculating some function in here.
And then I am adding something else to this x here, and I have some initializations in here, just something that I can demonstrate what it does.
So it has no meaning for this one.
And here's the assembly instructions.
I'm not going to go through assembly, but [UNINTELLIGIBLE] you can actually create and understand why this is happening in [INAUDIBLE].
So I [UNINTELLIGIBLE] into two slides.
The first thing you can do is think of constant propagation.
So what it says is for all possible executions, if a value that has in a variable is the same, and we know that value, that's a constant.
And I don't have to keep that value in that variable.
I can replace that with a constant.
Sometimes, when you look at dataflow optimization, you can say this is done.
As a programmer, I will never do that.
This is something you should be doing, for example, have things like constant variables that constant values or lower.
But sometimes, something looks dumb, but what happens is sometimes when you're in optimization does, one optimization might lead to code that looks like.
That can lead to it.
I will show you something sometimes that you might not find a code that looks dumb, but previous optimization will leave, or change the code in a way that this optimization can take advantage of.
So nice thing about this is you don't need to keep values in the variables because you can free some variable, that means free RAM registers.
Also, most of the time when you do constant propagation it leads to [UNINTELLIGIBLE] optimizations.
So in this program what are the things that can be constant propagated?
So we know x equals 0, x's are constant up to this point.
But since x get modified here, my dataflow say wait a minute, I am going through this loop, and x is constant from here to here.
But after this point, x is not constant because it get modified in here.
So that's what dataflow is going to say, and so I can't do that x.
But [UNINTELLIGIBLE] why it become constant here?
All input that goes into this loop, has to go through here, becomes constant in every path.
And then it doesn't get modified in this loop at all.
OK, so then I can actually, through constant propagation, get to the file.
OK, so now I have a program like that.
So normal compiler optimization is done by pass-by-pass.
A lot of passes get repeated multiple times, so I leave it like this.
So even though this is just simple thing, but we leave it to somebody else to optimize that, which is what we call algebraic simplification.
Basically, it says you go to your, whatever, fourth grade, fifth grade, sixth grade algebra book-- I don't know where you learn, somewhere you learned algebraic --and they have all these very simple rules, like something multiplied by 0 is 0, multiply 1 by that, and all of those rules, and then you can just busy code them up and look for these patterns and replace.
And that's what the compiler does.
And in fact, we look at something like this, a simple shape, but you saw before that, it do much more complicated things.
And it's a lot less work at run-time, and also it leads to more optimization, so it can simplify things in here.
And other thing is, sometimes instead of algebraic simplification, kind of weird things.
If you want exact precise, for example if you're doing floating point, because floating point, a plus b plus c, is not b plus c plus a, are different because you can get small teeny differences in these kind of-- [UNINTELLIGIBLE] and associate duty, and some people care.
Most people don't because it's so small, most people, they don't care.
Others do.
And also sometimes when you do this optimization, things like overflow and underflow, that happens because if I do x plus x minus x, or x is very large, x plus x minus overflow, and then you end of doing minus x because it overflows.
But instead of x plus x minus x, it's just x, you don't overflow anymore.
So you have changed the behavior of the program, but most of the time, compilers think that things like that are special cases.
They are not the normal behavior, so changing them is probably OK.
Sometimes, you can't do anything.
So now here, what are algebraic simplification I can do?
What can I do here?
Yeah, I multiply by 0, [UNINTELLIGIBLE] this, that.
At 0, I leave it here, and then there's another algebraic simplification, I can do that, but now, I am leaving it here because there's no algebraic simplification.
X equals x is-- there's nothing you can do.
That's called copy propagation.
Copy propagation says you're just making a copy of one value to another, just get another copy.
You don't need to do a copy.
Very simple thing in here.
Less instructions, less memory registers because we are not copying.
However, when we [UNINTELLIGIBLE] register location, I will talk, basically.
If I use the same register now, I might have things that was in two registers, x copied to y, now it's all in x.
So that means I might have some variable in the register that you call my interference graph.
I'll talk about this in a little while, so I'm just forward referencing.
That might not be easily register locatable.
And so in here, x equals x. I can get rid of that.
And another interesting thing is common subexpression elimination.
If you do the same thing multiple times, you calculate it once, less computation, Cons is you need to keep this result somewhere between the two users.
So if I have too many of these things, I might just run out of registers to keep these values calculated.
And also interesting thing is, this can hinder things like parallelization.
When we get there, we can see that by adding additional dependencies in there.
So in here, what are the common expressions?
Either you guys are bored, or this slide is way too hard.
You're bored?
[INAUDIBLE]
y plus 1, OK, good.
So there's y plus 1 in here, and I can calculate it once, and then I can just do the multiplication of that, and do that, and voila.
It got rid of two addition and one multiplication to one addition and one multiplication.
OK, next thing is dead code elimination.
So if you're doing something that nobody's using the value, why do you do it?
And less computation, and maybe you release storage because you're not storing these values your computing, and that's really nice.
And there's not much of bad things about dead code.
Dead code is pretty dead.
You can get rid of it.
So here, what are the dead code you have?
I want keep you at least somewhat engaged, so see if you can find my dead code
y
y, yeah.
I got rid of [UNINTELLIGIBLE].
Now, I don't need it, I can just get rid of that, and then I can even get rid of allocating y.
So I got rid of both instruction and some memory-allocated registry that used to keep that value there.
Another interesting thing you can do is loop invariant code [UNINTELLIGIBLE] because loops are very important.
Most of execution time is mainly inside loops, so if you can get something out of a loop, that's really good.
We talked about that previously.
But you have to worry about, basically, two things.
One thing is that when you move too many things out of the loops, you have to keep all those values in registers, so that means you need more registers inside the loop.
Second thing is when you execute that, you have to make sure that it have the same behavior as when you run the program.
How about special cases, the loop never get executed.
First let's look at this.
What other loop invariant expressions in here?
[INAUDIBLE]
Hm?
4 times [INAUDIBLE]--
4 times eta a divided by b. OK, good, I just moved up there.
So I did that, but why am I really wrong?
Why won't the compiler do this?
Give me a case that this would change the program behavior
[INAUDIBLE]
4 times overflow, yeah, that can happen.
That's one case, but there's something that can happen-- overflow happens in very large numbers, people don't care that much, but there's something that can happen a lot more
If B is 0, and then N is less than 0?
Exactly, when B[0] and N is less than 0.
I am going to have a divide by 0 error in here because I am going here, dividing by 0.
That would have never happened because the loop wouldn't have gone and executed it.
So normally, when you do things like that in a loop, the compiler generate a place called a landing pad, which basically is, before you enter the loop, you check whether the loop will ever get executed.
And then go to the landing pad, and then go to the loop.
So the landing pad will be run only when the loop at least has one iteration running, and so you can move all those thing in the landing pad.
So here, you can see there's no landing pad.
The code generated probably would have, and so I did something that is, you would see in fact, the optimized code I did didn't do that.
So GCC minus [UNINTELLIGIBLE] is smart enough not to do this.
So then there's another type of strength reduction, which is saying if I go something like a times i, what I can do is just, instead of doing a times i, I can basically make the first iteration initialize it, and every time you can update the previous value.
OK, so array times i, the first it's 0 and next time it'll be t plus 80 plus this, so I can keep updating that.
OK, so this is really good.
I have this computation because now we just sort of multiply, I just made it add.
But I have a lot of problems that can happen here.
First of all, I have, now, this one.
I didn't have to keep this value anywhere, only when I needed it.
In here, this value has to be varied through out the entire loop because I keep updating that value, so I created another need for a register.
Before now, I only needed it at that point.
I could've [UNINTELLIGIBLE], rarely used it, but now it just to be there throughout the program I created in there.
Also what I fear is what they call a loop-carried dependence.
Every time you run iteration, you use the previous iteration's value.
When we go into a parallelizing loop, you suddenly realize that means I can't run them parallel, so this creates a huge problem in parallelization.
So you do [UNINTELLIGIBLE], strength increase, when you go to parallelize.
And you can undo these things.
So in here, one thing you can do is you look at something like u times i and say wait a minute, I don't have to multiply by i because it [UNINTELLIGIBLE] 0 like this, and I can just allocate a value b and keep it updating by v, and then I did that, allocated a variable in here, allocated 0.
And [UNINTELLIGIBLE] this, I just basically put v times 0.
You see that?
I just basically got rid of a multiplication and convert it into addition, but I paid some cost by, I need now this additional register that is true all throughout the entire thing.
[UNINTELLIGIBLE] I just calculated that expression.
And the big thing a lot you get performances register allocation, so most processes have very few registers.
In fact, one big change that when you went from [UNINTELLIGIBLE] is to get additional registers.
Registers are very important.
I will go through register location a little bit.
So what happens is when you have a program, you have a control goes like this.
So this control going, executing something that defines this variable x, defines variable y.
And here, you use variable x and variable y, and there are different paths the program can go through.
There are two paths can merge in here, here, you can expand in here.
So this is kind of the flow of the program in a small part.
So what you can say is this definition [UNINTELLIGIBLE] here, so this value, this line in between here-- because you can't get rid of it.
When you decided you had to keep it somewhere because somebody's going to [UNINTELLIGIBLE].
And this definition is used here, so when you decided you had to be [UNINTELLIGIBLE] in here.
When you [UNINTELLIGIBLE] is only used here.
Nobody uses here, so this has to be [UNINTELLIGIBLE].
Interesting thing about x is there are two definitions of x that might be used here, and this definition might be used here or here.
So you put all this into what they call a one web because these two definitions might-- either one of them will be used here.
This definition will be used, either one, over here, so this value has to be [UNINTELLIGIBLE] in here, kept somewhere.
So then what we say is we give names to these, so this is s1, s2.
Somebody has to keep this value.
s2 keeps this value, s3 keeps this value, s4 keeps this value.
The interesting thing is how many registers you need to keep all those values.
That's why the entire thing of register allocation.
So what you do is this really cute mapping of this into nice theoretical problem.
So what you can say is each of these regions, we make it vertex of a graph.
If these regions overlap, then we get edge.
S1 and s2 overlap.
That means you can't use the same register to keep s1 one and s2 because before s1 is finished using, s2 has to be free of that.
OK, there's overlap in here, so we've created edge in here.
OK, s2 and s3.
So s2 and s3 overlaps here because at this point, both value s2 and s3 has to be kept.
So I create an edge in here.
OK, so I create an edge.
Every time I say those do values need separate registers.
I can't keep the same register.
And of course, s3 and s4.
s3 is here.
s4 can be in the same register.
So there's no edge here.
os1 and s4 can be in the same register.
os2 and s4 can be in the same register because they are not live at the same time.
They are live at a different time of the program execution.
Now, what you can do is, you have a graph, you have edges, and there's this very famous problem called graph coloring problem.
How many heard of graph coloring problem?
OK, good.
So what happens is now we can figure out how many colors need to color this graph, and that is the number of colors of the number of registers you need.
So if you have a graph like this with no edges, you can color it with one color.
How many colors for this one?
Two colors.
How many colors for this one?
People said two colors.
Yes, you can color it with two colors.
How many colors for this one?
[INAUDIBLE]
It's three-color [UNINTELLIGIBLE].
So there's all these algorithms [UNINTELLIGIBLE] and say no.
You can see by coloring this how many registers I need.
And the interesting is, if you need more colors than the register you have, that means you can't register allocate, and at that point, you need too many things to keep.
You don't have that many registers and that [UNINTELLIGIBLE].
That means you take edge and say, ah-hah, I can't keep both of these guys in the same.
I will take some vertex out and say this vertex can't be in there because I can't put it into register, and I spill this out.
And you can re-color the graphs.
You can spill it, and of course, spilling is costly because now [UNINTELLIGIBLE] value in the register, it's in the memory, so every time you need it, you had to bring it back, send it back, so it's going to be expensive.
The nice thing is to see how much you can keep in the register.
So I have enough registers for this program, so I found registers for all these things instead of putting it in memory.
And now, this is [UNINTELLIGIBLE] register allocation in a pseudo C code, so this is the kind of optimized code, and this is the generated-- Basically, all four of the original program generated.
But in here, I move this one up.
But in this one, actually, the division didn't get moved up, so [UNINTELLIGIBLE] actually inside the loop because it's realized you can't do that.
But interestingly moved the multiplication out, so that it didn't care about the overflow.
It says, hey, overflow, it can have an overflow, but it will worry more about divide by 0.
OK?
Any questions so far?
So here's the optimized code, and if you run it, there's seconds versus 54 seconds.
Just GCC [UNINTELLIGIBLE] 0, GCC os, so it'll produce very compact optimized code.
So the key thing is what's [UNINTELLIGIBLE] these optimizations.
The key thing is you have to guarantee, when you optimize, all that these programs [UNINTELLIGIBLE] from unoptimized, optimized, all the valid input, all the valid execution, and all valid architecture that you're supposed to run, you can't do the same thing.
Otherwise, it's not a good optimizer if it does different things to code.
So there are a lot of things that means you have to be very conservative in [UNINTELLIGIBLE] cases.
So you have to understand both control-flow and data accesses, and make sure that you understand them, and if any of them, the compile-time analysis cannot understand, the compiler give up very fast.
So the thing is, most of the time if that information is not available, compilers reduce the scope of the region [UNINTELLIGIBLE] the transformation.
So we have this point, I don't know beyond that.
I can only do a small amount of transformations here.
Or reduce the aggressiveness of transformations, and sometimes just completely leave code alone as it is because it couldn't, even the things you know, no sane program would do, and of course, your code will never do.
The compiler assume, if it is a valid C semantics, it might happen.
Even though some of them looked really crazy.
If it is a valid possible way of doing it, compiler has to worry about it, and not do that.
So it's here to be careful of that.
So first of all, control-flow.
That means it doesn't work on possible paths of the program when you execute that.
And the way you look at this, you can add this call graphs in the high-level [UNINTELLIGIBLE] the call in here, and control-flow graphs within the metadata function how control goes from.
And what makes it hard for compiler to analysis this?
Bunch of things [UNINTELLIGIBLE] function pointers.
You probably haven't done function pointers, but if you have function pointers in the compiler concepts, I don't know where it's going.
I have to be very careful.
Indirect branches.
so I keep addresses somewhere in that branch, so that I don't know where it's going.
Something computed go to [UNINTELLIGIBLE].
Large switch statement.
It's just spaghetti code.
We have no idea where it would end up and compile at us, and we can't get anywhere in this switch statement.
Either [UNINTELLIGIBLE] you might know some order of going through that, it doesn't work.
If you are looped with [UNINTELLIGIBLE] breaks and very complex things in the compiler, sometimes it'll give up.
When the loop bounds are unknown, you'd assume it could be anything.
Whereas when loop bounds are known, as you saw in the first set of examples, you can take advantages a lot more, and you can do a lot more aggressive things, or not care about cases because I know that.
But in this unknown loop bounds, you have to be a lot more careful of that.
And conditions where branch is not analyzable.
So if you have branch condition, if you don't know what's happening in the branch, I might not be able to take advantages or think how to do the branch well.
So those are the things that I have to worry about.
The other thing is data accessors, so that means who else can read and write the data.
So I am touching the data item, and I need to know that, between the two points I am looking at the data, nobody else go and muck with my data, or use my data.
Because when I look at the data, [UNINTELLIGIBLE] something, I want to make sure that's the only way that data can be accessed because, as you know, most of the things are in memory.
So normally compiler [UNINTELLIGIBLE] is called def-use chains, so defined to use, so we say that thing that defined here is going to get used here, and nothing comes in between that.
And that information is that's how the compiler [UNINTELLIGIBLE].
That's something we call dependence vectors.
We might talk a little bit about that when you go into parallel execution.
So what makes it very hard for compiler to analyze this?
For example, address taken variables, so if you write and hack with C, you can say, OK, there's a variable.
There's a variable here, I'm taking the address of that.
Suddenly, that means somebody else has the address to the variable.
That means anybody else can suddenly jump in and overwrite you, and there's a lot of possibilities of doing that.
And suddenly compiler says wait a minute, that variable, even though I assigned the variable here, I'm using it here, in between.
Somebody else might touch it even though it might not use the same name because somebody has that address to that.
OK, so that's a hard thing.
Global variables, sometimes, because between function, I don't know.
Some other function might go and change it.
Parameters are really hard.
Like for example, remember when we had a program, and we had something like copying same array to the same, even though parameters say X and Y. I might send the same or overlapping regions into two different parameters even though it looks like two different names.
They're not two different things.
They're actually overlapping at some point.
And so you had to assume, even if you have two different parameters point into memory, they might be the same thing.
And that's the worst case, even though a lot of times, nobody does that.
Nobody gives the same things multiple names, but it's possible.
If it is possible, compilers deal with it.
Either it has to generate code to test all these cases, is it overlapping, if not, do something.
If it is overlapping, do something slower, like the code we showed when you are vectorizing.
You treat it like this huge number of different cases, but unless you do something like that, you can't optimize, and complex programs, it's very hard to do that.
A lot of times, pointers create issues in here because the problem with pointers is what you call it point aliasing, because pointers, you can add any value to a pointer and you have no idea if you had a very large value.
It can be anywhere in memory because if you have a pointer, you have a point in the memory you can add anything, subtract anything.
The world is yours, and C gives you this ability to go all over the world and kind of mapping the world, and some programs do that.
And so the compiler says, oh, it's a point.
I don't know where it is.
I just have to leave it alone because some guy, probably 0.001% of the world programmers will do something crazy, and everybody has to pay the price.
So this is what makes programming hard.
And the final thing is there's a thing called [UNINTELLIGIBLE] types.
When you go to parallel programming you realize, because normally compilers keep normal values are in the memory.
Compiler can [UNINTELLIGIBLE] the value into register and keep operating in the register, and at some point, put it back to memory.
But if you're running a parallel program, somebody else might want to look at that value, and if it isn't registered, you don't have that value in the right place.
It's somewhere else, so you get a stale copy because you have moved it.
What I'm trying to say is, look, you have to always keep it in memory.
You can't take it out.
You can't just modify it, but you can move it somewhere else the faster place to do things to it because somebody else might be looking at it.
And so what that means is compilers give up it's hands and say, look, I can't do anything.
So we are a little bit early.
I have yet another huge session in here at-- OK, we have to go through this thing.
Good.
I think now we are going to go about and see how you guys did in the class exam.
OK. And I'm seeing it for the first time, and it looks really nice.
Where do you plug this in?
Where do you plug this in?
OK, so here is the distribution in there.
This was not an easy exam, and in fact, we compared how you guys did last year, and you guys have done a lot better than I think the first exam in last year.
So basically, we have a median about 70, somewhere here, and a nice tight grouping in here, which is really good.
And so what we have is, we have exams back.
Take a look.
And I think--
I'd like to make one comment about [INAUDIBLE]
OK, sure
[INAUDIBLE].
So not surprisingly, I graded the problem on the cache oblivious algorithm doing the recursion tree.
There is a common mistake that many people made, which I wanted to explain why it's wrong because so many people made this mistake.
They got it almost all right, and then they made this mistake.
So it's basically an understanding of recurrence.
So the recurrences I recall was q of r is equal to square root of r over b if square root of r is less than CM for C, et cetera.
OK?
And then otherwise, it was 2q of r over 2 plus theta 1.
Now, what people did in their recursion tree-- first of all, some people didn't recognize that what goes in the recursion tree is this value, the number of cache misses.
So the recursion tree is going to look like theta 1, or you can leave out the thetas if you want to put them in at the end.
Theta 1, theta 1, et cetera.
So many people got this, and then the question is what happens when it hits the leaf.
OK?
So when it hits a leaf, many people correctly got that you can't mess around with constants.
You have to be very careful of constants if they're in an exponent, that you hit the leaf when square root of r becomes less than c over m, in which case the cost is going to be square root of r over b.
So what they did was the incorrect thing, was they put square root of r over b here.
Why is that wrong?
[INTERPOSING VOICES]
It's the wrong r. Right?
OK, it's the wrong r. This r is the r here on the right-hand side.
It's not the one here.
It's the r if r is sufficiently small, that's the value you're taking.
But we're expanding an r from the top here.
So what's the value that should go here?
OK, cm is the value that should go here.
OK?
The value that should go here is cm.
Yes?
[INAUDIBLE] two r's can actually follow the right-side?
And then it's very close to where they're written the same but are spoken differently
Well, when you say-- what do you mean?
[INAUDIBLE]
There's an r here
And it's different from the other r--
No, it's the same r. The question is there, r is a variable.
So it'd be nice if the r were constant, but it's not.
It's a variable.
And so the point is the point where you plug it in here, you've got to plug in, not the variable, you've got to plug in the value
You just said for r [UNINTELLIGIBLE]
It's a variable.
You have to plug in the value of the variable at this point if you're going to solve the recurrence.
Putting an r here, we're trying to I say, this whole thing is q of r. And we started out, if we did the development of the tree, which is the safest thing to do, you get theta 1 plus q of r over 2, and you keep going down until your value for r satisfies this condition.
At that point, what's the value for r?
OK?
You can't then say it's the same r that you started with.
It's not this r, and that's because r is a variable, not because of anything else.
r is a variable, and we're using the r. This is a question of understanding of the recurrence.
So in any case, that was a common mistake that people make.
The other minor error that people made on that problem, that most people made, was in describing where do you get this recurrence, they left out the fact is why is it going to be square root of r over b.
It's really because na is approximately nb because the way that the code works, we're keeping na and nb to within a factor of two of each other.
OK?
And so if you didn't mention that, you lost a point.
It wasn't a big deal, but many people didn't neglect that very important statement.
Overall, people did very well on this problem.
Overall, you'll see people got a lot of partial credit on it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So John is going to present project three, beta
All right.
So here's the performance grades.
In general, the submission went a lot better than last time in that things were on time and nobody failed to build, or forgot to add files to their project, or so on.
We did change the scoring mechanism a little bit.
In the [?
mdriver ?]
that we gave you, if your validator failed you on any of your traces, your score is a zero.
In this one, we decided to be nicer.
We replaced your validator with our correct validator.
And for traces that you failed, you get a zero for the points that those traces contribute.
But you did get an overall partial score, even if you failed a couple traces.
So on that note, the reference implementation does get a 56 on this score.
And there were people who had slower than reference implementations that landed below 56.
So that might be something to think about for your final submission.
The high score was a 96.
And there were actually quite a few groups in the 90s.
So overall, people did really well on this.
With that said, your validators didn't really-- I guess they were OK.
But there's some people whose validators failed projects that were correct, and other people whose validators failed to detect certain situations.
So that's also something to work on for the final.
We won't be releasing the stock validators.
So it'll be up to you guys to find out what's wrong with your validators and fix them.
And along the same lines of correctness, once again, for the final submission, we'll be running-- actually even for the beta, I believe, we're going to Valgrind your projects and look for memory errors.
So do that to your own projects and investigate any messages you get
[INAUDIBLE]
OK.
So the highlighted column number 31 refers to the reference implementation of the validator.
So that's the authority.
If that's green, then your implementation is correct.
And so hopefully, a correct validator would agree with column 31
[INAUDIBLE]
Yes
[INAUDIBLE] question.
How can it be that most of-- so an implementation is vertical, so tests are [UNINTELLIGIBLE]?
No.
The implementations are horizontal, and the tests are vertical
So we want our column to look like column 31?
Or we want--
You want-- your validators correctness score will be determined by whether or not your column number corresponds with column 31.
And then, your implementations correctness will purely be determined by whether 31 marks your row red or green.
Does that make sense?
[INAUDIBLE PHRASE] columns that are all green, our validators are not [UNINTELLIGIBLE]?
Is that what you're saying?
That's right
That's correct
Whereas the rows that are green, that's what we like to see.
We like green rows.
And then, we like columns that match column 31
[INAUDIBLE PHRASE].
The first row should be all red.
And right now, [INAUDIBLE]
Right
That's correct
Whatever error this person had, very few validators seems to have caught them.
Which is very surprising, because what we did for your validator.c is that we removed the line of code that it contained, and we added the comment that explained in English exactly what that line of code did.
So it was kind of interesting to see that not everybody came up with the validator that's identical to reference one
OK--

Yeah.
So please run Valgrind on your code before the final submission.
And we'll be posting your personalized results to your repose sometime probably by the end of the day, either today or tomorrow
Great.
All right, you can take this [UNINTELLIGIBLE].
Or you can [UNINTELLIGIBLE].
Here you go.
You guys can have it here, in case you need to chip in.
OK.
So today, we're going to talk about programming in parallel.
Parallel programming and so forth.
So this is I'm sure what you've all been waiting for.
Oops.
Oh, we have no power here.
There we go.
There we go.
Now I've got power.
OK. Let's see here.
How's that?
Good.
OK.
So we talk about multicore programming.
And let me start with a little bit of history.
So since the mid to late 1960s-- so how many years is that?
50 years.
Wow.
Semiconductor density has been increasing at the rate of about-- it's been doubling about every 18 to 24 months.
OK.
So every year, every one to two years, every year and a half to two years, we get a doubling of density on the chips.
And that's a trend that still is continuing.
OK.
So that's called Moore's law, the doubling of density of integrated circuits.
And so, this is basically a curve showing how transistor count is rising.
OK.
So all these green things are Intel CPUs and what the transistor count is on them.
Yeah, question?
[INAUDIBLE PHRASE] the lines in [INAUDIBLE]?
So there have been some technology changes along the way.
So in particular, the [UNINTELLIGIBLE] transition is back down here I think.
I don't remember which one that is.
Well, this is actually a different one.
What we're looking at right now is the transistors, which have been very smooth.
OK.
So I'll explain this curve in a minute.
So there's two things plotted on here.
One is the Intel CPU density, and the other is what the clock speed of those processes is.
And so these are the clock speed numbers.
And so, the integrated circuit technology has been-- the density has been doubling.
And it's really an unbelievable sort of social and economic process, that this has basically been called a law.
Because what happens is if a-- there's so many people that contribute to making integrated circuits be dense.
There's so many pieces of technology that go into that.
And what happens is if you decide that you're going to try to jump and try to make something that goes faster than Moore's law, what happens is it's more expensive for you to do it.
And none of the other participants in that economy can keep up.
And you're just going to be more expensive.
So people will op for the cheapest thing that gets the factor of two every 18 to 24 months.
Whereas if you're behind, then nobody uses your stuff.
So everybody's got this sort of self-fulfilling prophecy that the rate at which the density is increasing has just been extremely stable for over 50 years.
It's remarkable.
Yeah, question?
[INAUDIBLE PHRASE] every six months.
And somehow, [INAUDIBLE] you would have self-replicated?
No, I'm not saying that.
What I'm saying is that there is some amount of everybody expecting that this is the point that everybody's going to be at.
And so if you try to go more aggressively than that, you can get burned because you'll be more expensive.
If you don't go that fast, you're going to get burned because nobody's going to adopt your particular piece of the technology.
And so, what happens is everybody sort of settles for this regular repeating.
It's a remarkable social and economic phenomenon.
It's got very little to do at some level of technology.
It's just that we know that we can improve things.
But what's amazing is this growth has gone through many transitions.
At one point, they said we aren't going to be able to build integrated circuits any more densely because all of the masks that were made-- it's basically, you make computers with a photographic process of exposing and using masks that you shine light through.
It's the way they used to do it.
And what happened was the wave lengths of light were such that you were just simply not going to be able to get the resolutions.
So what did they do?
They switched to eBeams.
OK.
Electrons rather than photons to expose the silicon wafers and so forth.
And so, they've gone through a whole bunch of transitions and different technologies.
And yet, throughout all of that, it's been just a very steady progress at about the rate of 18 to 24 months per doubling of density.
And that is still going on, and is projected to go on maybe for 10 years more.
It's going to run out, I hope in my lifetime.
And certainly within your lifetimes.
So that has been going.
Then, there's second phenomenon that has been going on since about mid-1980s.
And that is that the clock speed has actually been growing on a similar curve, where basically, we've been getting 30% faster processors, clock speed, since the mid-1980s.
But something happened there, which was in around 2003, it flattened out.
And the reason is, as a practical matter, clock speed for air cooled systems is bounded at somewhere around 5 gigahertz.
If you want to liquid cool it or nitrogen cool it or something, you could make it go faster.
But basically, the problem is that things get too hot.
And they cannot convey the heat out.
So for a while, if you have greater density, the transistors get smaller.
They switch faster.
And you can make the clock speed go faster.
But at some point, they hit the wall.
And so there the vendors were.
People like Intel, AMD, Motorola.
A variety of the semiconductor manufacturers.
And what's happened is they can still make integrated circuits more and more dense.
But they can't clock them any faster.
OK.
So here's what's going on in the circuits.
So here's essentially how much power was being dissipated by a variety of Intel processors along the way, and what they [INAUDIBLE] 2000.
They started getting hot and hotter, until if they just continued this trend, they were going to be trying to have junction temperatures that are as hot as the surface of the sun.
Well, they clearly couldn't do that.
OK.
So you might say, well, let's put it off a few years.
Yeah, but how many years are you going to put this off?
And so, what happened was they got stuck.
They simply could not make chips get clocked in faster.
So what did they decide to do?
They got all the silicon area, but they can't make the processors faster with it.
So their solution was to scale performance to put many processing cores on the microprocessor chip.
So this is an example of a Core i7.
It's a four core.
One, two, three, four cores processor.
We actually have six core machines now.
But I didn't update the figure.
And what's going to happen now is Moore's law is going to continue for a few more years.
And so it looks like each new generation of Moore's law is going to potentially double the number of cores per chip.
So you folks are using 12 core machines.
Two six core chips.
Well, that's going to basically keep increasing.
And so, we're going to get more and more cores per chip.
OK. That's all well and good.
But it turns out that there's a major issue.
And that's software.
Everybody has written their software.
And there's billions and billions and billions of dollars invested in existing legacy software that's written for how many cores?
One.
And moving it to multicore is a nightmare for these companies.
OK. And it's potentially a nightmare for these vendors.
Because if people say, gee, you can't make the processors go any faster, why should I buy a new processor?
My old processor is as good as my new one.
OK. And so, anyway, so that's sometimes been called the multicore challenge.
The multicore menace.
The multicore revolution.
Whatever.
But that's what it's all about.
It's all about the issue of the frequency scaling of the clocks, verses, Moore's law.
Which talks about what the density is.
OK.
So their solution is to do-- and so what we're going to talk about for a bunch of the rest of the term is going to be, how do you actually program multicore processors?
We're going to look at some fairly new software technology for doing that.
So here's an abstract multicore architecture.
It's not precise.
This is only showing one level of cache.
So we have processors connected to a cache.
In fact, of course, you know that there are multiple levels of cache.
Yeah, this is the international symbol for cache if you live in the US.
So the processors have their cache.
Of course, you know that what actually happens is you have multiple levels of cache.
And it's shared cache at some levels.
OK.
So it's more complex than this.
But this is sort of an abstract way of understanding a bunch of the issues.
And then, of course, they only get more complicated as we look at reality, as with all these hardware related things.
And so, this is a chip multiprocessor.
Now there are other ways of using the silicon.
So another way of using the silicon is building things like graphics processors and using silicon for a very special purpose thing.
So that instead of saying, let's build multiple processors, you can say, let's dedicate some fraction of the silicon real estate.
Instead of to general purpose computing, let's dedicate it to some specific purpose, like graphics, or some kind of stream processing, or what have you.
Sensor processing.
A variety of other things you can do.
But one main trend is doing chip multiprocessors.
So we're going to talk a little bit about shared memory hardware.
Just enough to get you folks off the ground to understand what's going on underneath the system.
And then, we're going to talk about four concurrency platforms, which are not the only platforms one can program in.
But they're ones that you should be familiar with.
The last one, Cilk++, is the one we're going to do our programming assignments in.
And then, race conditions, we're going to talk about, because that's the biggest thing that comes up when you do parallel programming compared to ordinary serial programming.
It's the most pernicious type of bugs.
And you need to understand race conditions and need a way of handling it.
So here's basically-- so we'll start with shared memory hardware.
So the main thing that shared memory hardware provides is a thing called cache coherence.
OK. And the basic idea is that you want every processor to be able to fetch stuff out of local caches because that's fast.
But at the same time, you want them to have a common view of what is stored in a given location.
So let's run through this example and see what the problem is.
And then, I'll show you how they solve it in sketchy detail.
So here's a processor.
Says he wants to load the value of x.
And in main memory here, x has got the value of 3, up here in DRAM.
OK.
So x moves through to the processor, where it gets consumed.
And it leaves behind the fact that x equals 3 in its local cache.
Well, now along comes the second processor.
It says, I want x too.
And perhaps the same thing happens.
Very good.
So far, no problem.
So two caches may have the same value of x.
They may both want to use x, and it's both in their local caches.
Now comes along the third processor.
Says load x as well.
Well, it turns out that it's actually-- what I showed you on the second case is not the common case.
If these two processors, these two processing cores, are on the same chip, it's generally cheaper for this guy to fetch it out of one of these guys caches than it is to fetch it out of DRAM.
DRAM is slow.
Getting it locally is much cheaper.
So basically, in this case, he gets it from this processor.
The first processor.
All is well and good.
They're all sharing merrily around.
OK. And then this fella decides if he wants to load it, no problem.
He can just load it.
He loads it locally.
No problem.
OK.
This guy decides, oh, he's going to store some value to x.
In this case, he's going to store the value 5.
So he sets x equal to 5.
OK. fine.
OK, now what?
Now this guy says, let me load x.
He gets the value x equals 3.
Uh-oh.
If your parallel program expected that this guy had gone first and it set x value x equal to 5, these guys are now incorrect.
And so, the idea of cache coherence is not letting this happen, making it so that whenever a value is changed by a processor, the other processors see that change and yet, they're still able most of the time to execute effectively out of their own local caches.
OK.
So that's the problem.
So do people understand basically what the cache coherence problem is?
Yes, question?
If the last processor was to store x and set x equals 5, as soon as that happens, wouldn't that write DRAM x equals 5?
Good.
So there's actually two types of strategies that are used in caches.
One is called write through.
And one is called write back.
What you're describing is write through.
What right through caches do is if you write a value, it pushes it all the way out to DRAM.
These days, nobody uses write through.
You're always going to DRAM.
You're always exercising the slow DRAM versus being able to just write it locally.
But you do have to do something about these guys that are going to have the shared values.
So here's the mechanism that they use.
So what most people do these days is write back caches.
Which basically means you only write it back when you really need to evict or what have you.
You don't always write it all the way through.
And so here's how these schemes work.
So, right.
So that's a bogus value for that kind to be getting.
So let's take a look.
So what they use is what's called-- the simplest is called an MSI protocol.
There are somewhat more complicated ones called MESI protocols, and ones that are MOESI.
"Mo-esi" and "messy".
Anyway, the MESI one is probably the one you'll hear most often.
It's just a little bit more complicated than this one.
But it saves you one extra access when we do a write.
I'll explain it in just a minute.
But let's first understand the simplest of these mechanisms.
So what you do is in each cache, you're going to label each cache line with a state.
And basically, it's because of these states that you associate with a cache line that cache lines end up having to be long.
OK?
Because if you think about, you'd like cache lines to be at some level very short, in that then you have more opportunity to have just the stuff in cache that you want, from a temporal locality point of view.
It's one thing if you want to bring in extra lines, extra data, for spatial locality.
But to insist that it all be there whether you access it or not, that's not clear how helpful that it is.
However, what instead is we have things like, on the Intel architecture, 64 bytes of cache line.
And the reason is because they're keeping extra data with each cache line.
And they want the data to be the larger fraction of what they're keeping compared to the control information about the data.
So in this case, they're keeping three values.
Three bits.
The M bit says this cache block has been modified.
Somebody's written to it.
And what they do is they, in this protocol, they guarantee in the protocol that if somebody has it in the M state, no other caches contain this block in either the M state or S state.
So what are those states?
So the S state is when other caches may be sharing this block.
And the I state is that this cache block is invalid.
It's the same as if it's not there.
It's empty entry.
So it just marks this entry.
There's no data there.
The cache line that's there is not really there, is basically what it says.
So here, you see for example that this fella has x equals 13 in the modified state.
And so, if you look across here, oh, nobody else has that in either the M or the S state.
They only have it in the I state or not at all.
If you have it in the shared state, as these guys have, well, they all have it in the shared state and notice the values are all the same.
And then, if it's in the invalid state, here this guy once again has it in the modified state, which means these guys don't have it in either the S or M state.
So that's the invariant.
So what's the basic idea behind the cache?
The MSI protocol?
The idea is that before you can write on a location, you must first invalidate all the other copies.
So whenever you try to write on something that's shared across a bunch of things or that somebody else has modified, what happens is over the network goes out a protocol to invalidate all the other copies.
So if they're just being shared, that's no problem.
Because all you do is just have them drop it from the cache.
If it's modified, then it may have to be written back or the value brought back to you, so that you're in a position of changing it.
If somebody has it modified, then you don't have it.
So therefore, you need to bring it in and make the change to it.
Question?
[INAUDIBLE] three states?
Three states.
Not three bits.
Two bits.
Right.
OK.
So the idea is you first invalidate the other copies.
Therefore, when a processor core is changing the value of some variable, it has the only copy.
And by making sure that it only has the only copy, you make sure that you never have copies out there that are anything except copies of what everybody else has.
That they're all the same.
OK.
Does everybody follow that?
So there's hardware under there doing that.
It's actually pretty clever hardware.
In fact, the verification of cache protocols is a huge problem for which there's a lot of technology built to try to verify to make sure these cache protocols work the way they're supposed to work.
Because what happens in practice is there are all these intermediate states.
What happens if this guy starts doing this while this guy is doing that, and these protocols start getting mixed, and so forth?
And you've got to make sure that works out.
And that's what's going on in the hardware.
The MESI protocol does a simple optimization.
It says, look, before I store something, I probably want to read it.
It's likely I'm going to read it.
So I can read it in two ways.
I can read it in a way that says that it is-- where it's just going to be shared.
But if I expect that I'm going to write it, let me when I read it instead of getting a shared copy, let me get an exclusive copy.
And that's where the E comes from.
Let me get an exclusive copy.
In other words, go through the invalidation protocols on the read, so that with the expectation that when you write, you don't have to then wait for the invalidation to occur at that point.
So it's a way of reducing the latency of the protocol by getting it exclusively by the read that you do before you do the write.
So rather than doing a read, which would go out and get the value-- but everybody [?
has them ?]
shared-- then doing the write, and then doing a whole invalidation protocol, if I basically get it in exclusive mode on the read, then I go out, I get the value, and I invalidate everybody else.
Now I've just saved myself half the work and half the latency.
Or basically saved myself some latency.
Not half the latency.
OK?
So basically, what you should know is there is invalidation stuff going on behind when you start using shared memory, behind the scenes which can slow down your processor from executing.
Because it can't do the things that it needs to do until it goes through the protocol.
Any questions about that?
That's basically the level we're going to cover the hardware at.
And so, you'll discover that in doing some your problems, that if you're not careful, you're going to create what are called invalidation storms, where you have a whole bunch of things that are red, and they're distributed across the processor.
And then you go in, and you set one value.
And suddenly, vrrrrrruuuum.
Gee, how come that wasn't a fast store?
The answer is it's going through and invalidating all those other copies.
Good.
So let's turn to the real hard problem.
So it turns out that building these things is not particularly well understood.
But it's understood a lot better than programming these beasts.
OK. And so, we're going to focus on some of the strategies for programming.
So it turns out that trying to program their processor cores directly is painful.
And you're liable to make a lot of errors, as we'll see.
Because we're going to talk about races soon.
And so the idea of a current currency platform is to do some level of abstraction of the processor cores to handle synchronization communication protocols, and often to do things like load balancing, so that the work that you're doing can be moved across from processor to processor.
And so, here are some examples of concurrency platforms.
Pthreads and WinAPI threads, we're going to talk more in detail about.
Pthreads is basically for Unix type systems, like Linux and such.
WinAPI threads is for Windows.
There's threading building blocks, TBB, OpenMP, which is a standard, and Cilk++.
Those are all examples of concurrency platforms that make it easier to program these parallel machines.
So I'm going to do, as an example, I'm going to use the Fibonacci numbers, which you have seen before I'm sure, because we've actually even used it in this class.
This is Leonardo da Pisa, who was also known as Fibonacci.
And he introduced-- he was the most brilliant mathematician of his day.
He came basically out of the blue, doing all kinds of beautiful mathematics very early in the Renaissance.
You'll recognize 1202 is very early Renaissance.
But it turns out, for those of you of Indian descent, the Indian mathematicians had already discovered all this stuff.
But it didn't make it into Western culture except for Leonardo da Pisa.
So here's a program as you might write it in C. So Fib int n says, well, if n is less than 2, return n. So if it's 0 or 1, we return, Fib of 0 is 0.
Fib of 1 is 1.
And otherwise, we compute Fib of n minus 1, compute Fib of n minus 2, and return the sum.
Simple recursive program.
Here's the main routine.
We get the argument from the command line, compute the result, and then print out Fibonacci of whatever is whatever.
Pretty simple piece of code.
So what we're going to do is take a look at what happens in each of these four concurrency platforms to see how it is that they make this easy to run this in parallel.
Now just a disclaimer here.
This is a really bad way-- I hope you all recognize-- of computing Fibonacci numbers.
So this is exponential time algorithm.
And you all know the linear time algorithm, which is basically computed up from the bottom.
And some of you probably know there's a logarithmic time algorithm based on squaring matrices.
Two by two matrices.
So in any case, we're all about performance here.
But obviously, this is a really poor choice to do performance on.
But it is a good didactic example, because it's so the structure and the issues that you get into in doing this with a very simple program that I can fit on a slide.
OK.
So when you execute Fibonacci, when you call Fib of 4, it calls Fib of 3 and Fib of 2.
And Fib of 3 calls Fib of 2 and Fib of 1.
And Fib of 1 just returns Fib of 2, calls [UNINTELLIGIBLE] 1, 0, et cetera.
And so basically, you get an execution trace that basically corresponds to walk of this tree.
So if you were doing this in C, you'd basically call this, call this, call this.
Get a value return.
Call this.
Add the two values together.
Return here.
Call this.
Add the two values together.
Call the return there.
And so forth.
You walk that using a stack, a call stack, in the execution.
The key idea for parallelization is, well, gee.
Fib of n minus 1 and fib of n minus 2 are really, in this calculation, completely independently calculated.
So let's just do them at the same time.
And they can be executed at the same time without interference, because all they're doing is basing it on n. They're not using any shared memory or anything even for this particular program.
So let's take a look, to begin with, how Pthreads might do this.
So Pthreads is a standard that ANSI and the IEEE have established for-- and I actually believe this is a little bit out of date.
I believe there's now a 2010 version.
I'm not sure.
But I recall that they were working on a new version.
But anyway, this is a recent enough standard.
It's a standard that has been revised over the years, the so-called POSIX standard.
So you'll hear, Pthreads is basically POSIX threads.
It's basically what you might characterize as a do it yourself concurrency platform.
It's kind of like assembly language for parallelism.
It allows you to do the things you need to do, but you're sort of doing it all by hand, one step at a time.
It's built as a library of functions with special non-C or C++ semantics.
And we'll look at what some of those semantics are.
Each thread implements an abstraction of a processor, which are multiplexed onto the machine resources by the Pthread runtime implementation.
Threads communicate through shared memory.
And library functions mask the protocols involved in interthread coordination.
So you can start up threads, et cetera, and their library function for doing that.
So let's just see how that works.
So here are, basically, the two important Pthread functions.
There are actually a whole bunch of them, because they also provide a bunch of other facilities.
One is pthread_create, which creates Pthread.
And one is pthread_join.
So pthread_create basically is return an identifier.
So when you say create a Pthread, the Pthread system says, here's a handle by which you can name this thread in the future.
OK.
So it's a very common thing that the implementer says, here's the name that you get.
It's called a handle.
So it returns a handle.
It then has an object to set various thread attributes.
And for most of what we're going to need, we're just going to need NULL for default.
We don't need any special things like changing the priority or what have you.
Then what you pass is a void* pointer to a function, which is going to be the routine executed after creation.
So you can name the function that you want to have it operate on.
And then you have a single pointer to an argument that you're going to pass to the function.
So when you call something with Pthreads to create them, you can't say, and here's my list of arguments.
If you have more than one argument, you have to pack it together into a struct and pass the pointer to the struct.
And this function has to be smart enough to understand how to unpack it.
We'll see an example in a minute.
And then, it returns an error status.
So the most common thing people do is they don't bother to check the error status.
OK. And yet sometimes, you try to create a Pthread, there's a reason it can't create one.
And now you keep going thinking you have one, and then your program crashes and you wonder why.
So when you create things, you should check.
I'm not sure in my code here whether I checked everywhere.
But you should check.
Do as I say, not as I do.
OK.
So the other key function is join.
And basically, what you do is you say, you name the thread that you want to wait for.
This is the name that would be returned by the create function.
And you also give a place where it can store the status of the thread when it terminated.
It's allowed to say, I terminated normally.
I terminated with a given error condition or whatever.
But if you don't care what it is, you just put in NULL there.
And then it returns to the error status of the join function.
So those are the two functions that you program with.
Question?
[INAUDIBLE PHRASE]?
It's different.
It's different.
So it's basically, if the error status, if it returns NULL, it just means everything went OK.
The handle is you pass a name, and basically this is *thread.
It stuffs the name into whatever you give it.
OK so you're not saying, here's the name.
This is returned as an output parameter.
So you're giving it an address of some place to put the name.
OK. Let's see an example.
So here's Fibonacci with Pthreads.
So let's just go through that.
So the first part is pretty good.
This is your original code that does Fibonacci.
And now what we do is we have a structure for the thread arguments.
And so we're going to have an input argument and an output argument in this example.
Because Fib takes an input argument in and returns Fib of n. So we're going to call those input and output.
And we'll call them thread_args.
And now, here is my void* function, thread_func, which takes a pointer.
And what it does is when it executes-- so what you're going to be able to do is, as we'll see in a minute--.
Let me just go through this.
This is going to be the function called when the thread is created.
So when the thread is created, you're just going to call this function.
And what it's going to get is the argument that was passed, which is this *star thing.
And what it does in this case is it's basically going to cast the pointer to a thread_arg struct and dereference the input, and stick that into I.
Then going to compute Fib of I.
And then it's going to take, once again, deference the pointer as if it's a thread_arg, and store into the output field the result of the Fib.
And then it returns NULL.
So that's basically the function that's going to be called when the thread is created.
So in your main routine now, what happens is we initialize a bunch of things.
And now, if argc is less than 2, we'll return 1.
That's fine.
Then we're going to get the reading that we fail.
That's actually the reading of the input.
So then, what we do here is we get n from the command line.
And then if n is less than 30, we're just going to compute Fib of n. This is what I evaluated on my laptop was a good number.
So the idea is there's no point in creating the extra thread to do the work if it's going to be more expensive than me just doing the work myself.
So I looked at the overhead of thread creation and discovered that if it was smaller than 30, it's going to be slower to create another thread to help me out.
It's sort of like you folks when you're doing pair programming, which you're supposed to be doing, versus handing it off.
Sometimes, there are some things that are too small to ask somebody else to do.
You might as well just do it, by time you explain what it is, and so forth.
Same thing here.
What's the point in starting up a thread to do something else, because the startup cost is rather substantial.
So if it's less than 30, well, we'll just be done.
Otherwise, what we do is we marshall the argument to the thread.
We basically set args.input to n minus 1.
Because args is going to be what I'm going to pass in.
So I say the input number is n minus 1.
And now what I do is I create the thread by saying, give me the name of the thread that I'm creating.
This was the field that I said you could put to be NULL, which basically lets you set some policy parameters and so forth.
I say, execute the thread_func.
This guy here.
And here's the argument list that I want to provide it, which is this args thing.
Once you do the thread_create, and this is where you depart from normal C or C++ semantics.
And in fact, we're going to be doing more moving in the direction of C++.
We'll have some tutorials on that.
What happens is we check the status.
OK, I actually did check the status to see whether or not it created it properly.
But basically now, what's happening is after I execute this, it goes off and all the magic in Pthreads starts another thread doing that computation.
And control returns to the statement after the pthread_create.
So when the pthread_create returns, that doesn't mean it's done computing the thing you told it to do.
Then, what would be the point?
It returns after it's set up to operate in parallel the other thread.
People follow that?
So now at this point, there are two threads operating.
There's the thread we've called thread.
And there's whatever the name of the thread is that we started on.
So then we, in our own processor here, we compute Fib of N minus 2.
And now, what we do is we go on to join this thread with the thread that we had created.
So let's see here.
And the thing that the join does is if the other thread isn't done, it sits there and waits until it is done.
And it does that synchronization automatically for you.
And this is the kind of thing a concurrency platform provides.
It provides the coordination under the covers for you to be able to synchronize with it without you having to synchronize on your own.
And then, once it does return, it adds the results together by taking the result which came from the Fib of n minus 2 and adds to it the value that this thread has returned in the args.output.
And then it prints the result.
So any question about that?
Wouldn't this be fun to write a really big system in?
People do.
People do.
Yeah, question?
[INAUDIBLE PHRASE]
That's a tuning parameter.
That's a voodoo parameter
Right.
But in this particular case, it makes no difference at all.
It would've made a difference if it was an actual person [INAUDIBLE]?
No, it does make a difference.
For how fast it computes this?
Absolutely does
That's not recursive?
No, that's right.
This is not recursive.
I'm just doing two things and then quitting
[INAUDIBLE] if it's less than 30, then it's going to be [INAUDIBLE], right?
If it's less than 30, it's fast enough that I might as well just return
Then why [INAUDIBLE PHRASE] to do it.
It would return [INAUDIBLE] too
No.
But it would be slower.
It would be wasteful of resources.
Maybe somebody--
Well, because you're using such a bad algorithm, I guess?
Yeah
Oh, I see.
Oh,
OK.
So in any case, that's Pthread's programming.
There are a bunch of issues.
One is that the overhead of creating a thread is more than 10,000 cycles.
So it leaves you to only be able to do very coarse grain concurrency.
There are some tricks around that.
One is to use what's called thread pools.
What I do is I start up, and I create a bunch of threads.
And I have their names.
I put them in a link list.
And whenever I need to create one, rather than actually creating one, I take one out of the list, much as I would do memory allocation.
Which you folks are familiar with.
OK. Ha, ha, ha, ha, ha.
[MANIACAL LAUGHTER] So basically, you can have a free list of threads.
And when you need a thread, you grab the thread.
The second thing is scalability.
So this code gets about a 1.5 speed up for two cores.
If I want to use three cores or four cores, what do I have to do?
Rewrite the whole program.
This program only works for two cores.
It will also work for one core.
but basically, it doesn't really exploit three or four cores.
It's really bad for modulatary.
The Fibonacci logic is no longer neatly encapsulated in the Fib function.
So where do we see if we go back to this code?
Here's the Fib function.
Oh, but now, I've kind of got-- well, this is sort of just marshaling and calling.
But over here, oh my goodness, I've got some arguments here.
If n is less than 30, I give a result.
Otherwise, I'm adding together-- but wait a minute.
I already specified Fib up here.
So I'm specifying my serial implementation, and I'm specifying a parallel way of doing it.
And so that's not modular.
If I decided I wanted to change the Fib, I've got to change things in two places.
If Fib were something I did.
Code simplicity.
The programmers for this are actually marshalling arguments.
This is what I call shades of 1958.
What happened in 1958 that's relevant to computer science?
What was the big innovation in 1958?
Programming language.
Fortran.
So, Fortran.
Before Fortran, people wrote in assembly language.
If you wanted to put three arguments to a function, you did a push, push, push, or passed them in parameters.
Actually, their machines were so much more primitive than that it was even more complicated than you could imagine, given how complicated it is today what the compilers are doing.
But you had marshal the arguments yourself.
What Fortran did was say, no, you can actually write f of a, b, c. Close paren.
And that it will cause a, b, and c all to be marshalled automatically for you.
Well, Pthreads doesn't have that automatic marshalling.
You got to marshall by hand if you're going to use pthreads.
And of course, as you can imagine, that was error prone.
Because there is no type safety.
Are you calling things with the right types and so forth?
And so forth.
And also, one of the things here is that we've created two jobs that aren't the same size.
So there's no way that they have of load balancing.
So this is why pthreads is sort of the assembly language level, so that you can do anything you want in pthreads.
But you have to program at this kind of very protocol-laden level.
Next thing I want to talk about is threading building blocks.
This is a technology developed by Intel.
It's implemented as a C++ library that runs on top of the native Pthreads, typically, or WinAPI threads.
So it's basically a layer on top of the Pthread layer.
In this case, the program specifies tasks rather than threads.
And tasks are automatically load balanced across the threads using a strategy called work-stealing, which we'll talk about a little bit more later.
And the focus for this is on performance.
They want to write programs that actually perform well.
So here's Fibonacci in TBB.
So as you'll see, it's better.
But maybe not ideal for what you might like to express.
So what we do is we declare the computer, the computation, it's going to organized as a bunch of explicit tasks.
So you say that it's going to be a task.
And FibTask is going to have an input parameter, n, and an output parameters, sum.
And what we're going to do is when the task is started, it automatically executes the execute method of this tasking object here.
And the execute method now starts to do something that looks very much like Fibonacci.
It says if n is less than 2, sum is equal to n. That's we had before.
And otherwise.
And now what we're going to do is recursively create two child tasks, which we basically do with this function, allocate_task, giving it the fib task a name, where this is basically a method for allocating out of a particular type of the pool, which is an allocate child pool.
And then similarly for b, we recursively do for n minus 2.
And then what it does is it sets the number of tasks to wait for.
In this case, it's basically two children plus 1 for bookkeeping.
So this ends up always being one more than the things that you created as subtasks.
And then what we do is we say, OK, let's spawn.
So this will only set up the task.
It doesn't actually say, do it.
So the spawn command says actually do this computation here that I set up.
So it actually does b.
Start task b.
And then itself, it executes a and waits for all of the other tasks, namely both a and b, to finish.
And once it's finished, it adds the results together to produce the final output.
So this, notice, has the big advantage over the previous implementation that this is actually recursive.
So in doing Fib, you're not just getting two tasks.
You're recursively getting each of those two more, and two more, and two more, down to the leaves of the computation.
And then what TBB does is it load balances those across the number of available processors by creating these tasks.
And then, it automatically does all the load balancing of the tasks and so forth.
Questions about that?
Any questions?
I don't expect you to be able to program a TBB, unless I gave you a book and said, program a TBB.
But I'm not going to do that.
This is mainly to give you a flavor of what's in there.
What the alternatives are.
So TBB provides many C++ templates that simplify common patterns.
So rather than having to write that kind of thing for everything, for example, if you have loop parallelism.
If you have n things that you want to have that operate parallel, you can do a parallel four and not actually see the tasks.
It covers them over and creates the tasks automatically, so that you can just say, for I gets 1 to n, do this to all I, and do them at the same time essentially.
And it then balances those and so forth.
It also has to things like parallel reduce.
Sometimes what you want to do across an array is not just do something for every element of the array.
You may want to add up all the elements into a single value.
And so it basically has what's called a reduction function.
It does parallel reduce to aggregate.
And it's got various other things, like pipelining and filtering for doing what's called software pipelining, where you have one subsystem that basically is going to process the data and pass it to the next.
So you're going to process it and pass it to the next.
And it allows you to set up a software pipeline of things.
It also collides with some container classes, such as hash tables, concurrent hash tables, that allow you to have multiple tasks beating on a hash table.
Inserting and deleting from the hash table at the same time and a variety of mutual exclusion library functions, including locks and atomic updates.
So it has a bunch of other facilities that make it much easier to use than just using the raw task interface.
OpenMP.
So OpenMP is a specification produced by an industry consortium of which the principal players-- the original principal player was Silicon Graphics, which essentially has become less important in the industry, let's say.
Put it that way.
And for the most part, recently, it's been players from Intel and Sun, which is now no longer Sun, except that it is Sun part of Oracle, and of IBM, and variety of other industry players.
There's several compilers available.
Both open source and proprietary, including gcc, has OpenMP built-in.
And also, Visual Studio has OpenMP built-in.
These are a set of linguistic extensions to C and C++ or Fortran in the form of compiler practice pragmas.
So who knows what a pragma is?
OK. Good.
Can you tell us what a pragma is?
[INAUDIBLE PHRASE]
Yeah, it's kind of like a compiler hint.
It's a way of saying to the compiler, here's something I want to tell you about the code that I'm writing.
And it basically is a hint.
So technically, it's not supposed to have any semantic impact, but rather suggest how something might be implemented by the compiler.
However, in OpenMP's case, they actually have a compiler-- it does change the semantics in certain cases.
It runs on top of native threads and it supports, especially, loop parallelism.
And then, in the latest version, it supports a kind of task parallelism like we saw with TBB.
So, in fact, their task parallelism is fairly to specify.
So here's the Fib code.
So now, this is not looking too bad.
We basically inserted a few lines here.
And otherwise, we actually have the original Fibonacci code.
So the sharp pragma says, here's a compiler directive.
And it says, the OMP says it is an OpenMP compiler directive.
The task says, oh, the following things should be interpreted as an independent task.
And now, the sharing of memory in OpenMP is managed explicitly, because they're trying to allow for programming both of distributed memory clusters, as well as shared memory machines.
And so, you have to explicitly name the shared variables that you're using.
And here, we're basically saying, wait for the two things that we spawned off here to complete.
So pretty simple code.
It provides many pragma directives to express common patterns, such as a parallel for parallelization.
It also has reduction.
It also has directives for scheduling and data sharing.
And it has a whole bunch of synchronization constructs and so forth.
So it's another interesting one to do.
The main downside, I would say, of OpenMP is that the performance is not really very composable.
So if you have a program you've written with OpenMP over here, another one here, and you want to put them together, they fight with each other.
You have to have your concept of what are going to be the programs.
The task parallelism helps a bit with that.
But the basic OpenMP is very much of the model, I know how many cores I'm running on.
I can set that.
And then I can have it automatically parse up the work for those many.
But once you've done that, some other job, some other part of the system that wants to do the same thing, then you get oversubscription and perhaps some [UNINTELLIGIBLE].
Nevertheless, a very interesting system.
And very accessible, because it's in most of the standard compilers these days.
What we're going to look at is Cilk++.
So this is actually a small set of linguistics extensions to C++ to support fork-join parallelism.
And it was developed by Cilk Arts, which is an MIT spin-off, which was acquired by Intel last year.
So this is now an Intel technology.
And the reason I know about it is because I was the founder of Cilk Arts.
It was based on 15 years of research at MIT out of my research group.
And we won a bunch of awards, actually, for this work.
In fact, the work-stealing scheduler that's in it is provably efficient.
In other words, it's not just a heuristic scheduler.
It's actually got a mathematical proof that it's an effective scheduler.
And in fact, was the inspiration for things like the work-stealing in TBB and the new task mechanisms and so forth in OpenMP, as well as a bunch of other people who've done work-stealing.
It in addition provides a hyperobject library for parallelizing code with global variables, which we'll talk about later.
And it includes two tools that you'll come to know and love.
One is the Cilkscreen race detector, and the other is the Cilkview scalability analyzer.
Now, what we're going to be using in this class is going to be the Cilk++ technology that was developed at Cilk Arts and then massaged a little bit when it got to Intel.
There is a brand new Intel technology with Cilk built into their compiler.
And it is due to come out in like, two weeks.
So our timing for this was it would've been nice to have you folks on the new Intel Cilk+ technology.
But we're going to go with this one for now.
It's not going to make too big a difference to you folks.
But you should just be aware that coming down the pike, there's actually some much more cleanly integrated technology that you can use that's in the Intel compiler.
So here's how we do nested parallelism in Cilk++.
So basically, this is Fibonacci.
And now, what I have here is, if you notice, I've got two keywords, cilk_spawn and cilk_sync.
And this is how you write parallel Fibonacci in Cilk.
This is it.
I've inserted two key words, and my program is parallel.
The cilk_spawn keyword says that the named child function can execute in parallel with the parent caller.
So when you say x equals cilk_spawn or Fib of n minus 1, it does the same thing that you normally think.
It calls the child.
But after it calls the child, rather than waiting for it to return, it goes on to the next statement.
So then, the statement y equals Fib of n minus 2 is going on at the same time as the calculation of Fib of n minus 1.
And then, the cilk_sync says, don't go past this point until all the children you've spawned off have returned.
And since this is a recursive program, it generates gobs of parallelism, if it's a big thing.
So one of the key things about Cilk++, is unlike Pthreads-- Pthreads, when you say, pthread_create, it actually goes and creates a piece of work.
In Cilk++, these keywords only grant permission.
They say you may execute these things in parallel.
It doesn't insist that they be executed in parallel.
The program may decide, no, in fact, I'm going to just call this, and then return, and then execute this.
So it only grants permission, and the Cilk++ runtime system figures out how to load balance it and schedule it.
Cilk++ also supports loop parallelism.
So here's an example of an in-place matrix transpose.
So I want to take this matrix and flip it on its major axis.
And we can do it with for loops.
As you know, for loops are not the best way to do matrix transpose.
Right?
It's better to do divide and conquer.
But here's how you could do it.
And here, I made the indices run from 0, not 1, because that's the way you do it in programming.
But if I did it up here, then these things get to be n minus 1, n minus 1, and then it gets too crowded on the slide.
And I said, OK, I'll just put a comment there rather than try to sort it out.
So here's what I'm saying, is this outer loop is parallel.
It's going from 1 to n minus 1.
And saying, do all those things in parallel.
And each one is going through a different number of iterations of j.
So you can see you actually need some load balancing here, because some of these are going through just one step, and some are going through n minus 1 steps.
It's basically the amount of work in every iteration of the outer loop here is different.
I'm sorry?
[INAUDIBLE PHRASE]
No.
i equals 1 is where you want to start.
Because you don't have to move the diagonal.
You only have to go across the top here.
And for each of those, copy it into the appropriate column.
Flip it into the appropriate column.
Flip the two things.
Actually, transpose is one of these functions.
I remember writing my first transpose functions.
And when I was done, I somehow had the identity.
Because I basically made the loops go from 1 to n and 1 to n and swapped them.
So I swapped them.
So I said, oh, that was a lot of work to compute the identity.
No, you've got to make sure you only go through a triangular iteration space in order to make sure you swap-- and then swap.
This is an in-place swap.
So that's cilk_for.
That's basically it.
There are some more facilities we'll talk about.
But that's basically it for parallel programming in Cilk++.
The other part is, how do you do it so you get fast code?
Which we'll talk about.
Now, Cilk has serial semantics.
And what that means is unlike some of the other ones, it's kind of what OpenMP was aspiring to do.
The idea is that if I, for example here, delete these two keywords, I get a C++ code.
And that code is always a legal way to execute this parallel code.
So the parallel code may have more behaviors of its nondeterministic code.
But always, it's legal to treat it as if it's just straight C++.
And the reason for that is that, really, we're only granting permission for parallel execution.
So even though I put in these keywords, I still can execute it serially if I wish.
They don't command parallel execution.
To obtain this serialization, you can do it by hand by just defining a cilk_for to be for, and the cilk_spawn and cilk_sync to be empty.
Or there's a switch to the Cilk++ composite that does that for you automatically.
And it's probably the preferred way of doing it.
But the idea is conceptually, you can sprinkle in these keywords, and if you don't want it anymore, fine.
If you want to compile it with the straight c compilers, it's better to use the Cilk++ compiler to do it.
But if you wanted to ship it off to somebody else, you could just do these sharp defines, and they could compile it with their compilers, and it would be the same as a serial C++ code.
So the Cilk++ concurrency platform allows the program to express potential parallelism in application.
So it says, where is the parallelism?
It doesn't say how to schedule it.
It says, where is it?
And then, it gets mapped onto, at runtime, dynamically mapped onto the processor cores.
And the way that it does the mapping is mathematically provably a good way of doing it.
And if you take one of my graduate courses, I can teach you how that works.
We'll do a little bit of study of simple scheduling.
But the actual schedule it uses is more involved.
But we'll cover it a little bit.
Here's the components of the Cilk++ platform on a single slide.
So let me just say what they are.
The first one is the keywords.
So you get to put things in there.
And if you elide or create the serialization, then you get the C++ code or C code, for which then you can run your regression test and demonstrate you have some good single-threaded program.
Alternatively, you can send it through the Cilk++ compiler, which is based on a conventional compiler.
In our case, it will be GCC.
You can link that with the hyperobject library, which we'll talk about when we start talking about synchronization.
It produces a binary.
If you run that binary on the runtime system, you can also run it to the regression test.
And in particular, if you run it on the runtime system, running on one core, it should behave identically to having run it through this path with just the serial code.
And of course, you get exceptional performance.
These, I think, were originally marketing slides.
However, there's also the fact that you may get what are called races in your code, which are bugs that will come up that won't occur in your serial code, but will occur in your parallel code.
Cilk has a race detector to detect those, for which you can run parallel regression tests to produce your reliable multi-threaded code.
And then, the final piece of it is there's this thing called Cilkview, which allows you to analyze the scalability of your software.
So you can run, in fact, on a single core or on a small number of cores.
And then, you can predict how it's going to behave on a large number of cores.
So let's just, to conclude here, talk about races.
Because they're the nasty, nasty, nasty thing we get into parallel programming.
And then next time, we'll get deeper into the Cilk technology itself.
So the most basic kind of race there is what's called a determinacy race.
Because if you have one of these things, your program becomes nondeterministic.
It doesn't do the same thing every time.
A determinacy race occurs when two logically parallel instructions access the same memory location, and at least one of the instructions performs a write, performs a store, to that location.
So here's an example.
I have a cilk_for here, both branches of which are incrementing x.
This is basically going.
The index is going.
i equals 0 and i equals 1.
And then, it's asserting that x equals 2.
If I run this serially, the assertion passes.
But when I run it in parallel, it may not produce a 2.
It can produce a 1.
And let's see why that is.
So the way to understand this code is to think about its execution in terms of a dependency [?
dag ?].
So here I have my initialization of x.
Then once that's done, the cilk_for loop allows me to do two things at a time, b and c, which are both incrementing x.
And then, I assert that x equals 2 when they're both done.
Because that's the semantics of the cilk_for.
So let's see where the race occurs.
So remember that it occurs when I have two logically parallel instructions that access the same memory location.
Here, it's going to be the location x.
And at least one of them performs a write execution.
So if we actually looked closer, I want to expand this into this larger thing.
Because as you know, X++ is not done on a memory location.
It's not done as a single instruction.
It's done as a load, x into a register.
Increment the register, and then store the value back in.
And meanwhile, there's another register on another processor, presumably, that's doing the same thing.
So this is the one I want to look at.
This is just a zooming in, if you will, on this dependency graph to look a little bit finer grain at what's actually happening one step at a time.
So the determinacy race, recall, occurs-- this is by something, I'm going to say again, you should memorize.
So you should know what this is.
You should be able to say what a determinacy race is.
It's when you have two instructions that are both accessing the same location, and one of them performs write.
And here, I have that.
This guy is in parallel.
He's being stored to here.
This is also a race.
He's been reading it, and this guy is writing it.
So let's see what can happen and what can go wrong here.
So here's my value, x, in memory.
And here's my two registers on, presumably, two different processors.
So one thing is that you can typically-- and this is not quite the case with real hardware-- but an abstraction of the hardware is that you can treat the parallel execution from a logical point of view as if you're interleaving instructions from the different processors.
OK. We're going to talk in three or four lectures about where that isn't the right abstraction.
But it is close to the right abstraction.
So here, basically, we execute statement one, which causes x to become 0.
Now let's execute statement two.
That causes r1 to become 0.
Then, I can increment that.
It becomes a 1.
All well and good.
But now if the next logical thing that happens is that r2 is set to the value x, then it becomes 0.
Then we increment it.
And now, he stores back 1 into x.
And now, this guy stores 1 back into x.
And notice that now, we [UNINTELLIGIBLE] go to the assertion.
And we assert that it's 2, and it's not the 2.
It's a 1.
Because we lost one of the updates.
Now the reason race bugs are really pernicious is, notice that if I had executed this whole branch, and then this whole branch, I get the right answer.
Or if I executed this whole branch, and then this whole branch, I get the right answer.
The only time I don't get the right answer is when those two things happen to interleave just so.
And that's what happens with race conditions generally, is that you can run your code a million times and not see the bug, and then run it once, and it crashes out in the field.
Or what's happened is there have been race bugs responsible for failure of space shuttle to launch.
You have the North American blackout of 2001?
2003?
It wasn't that long ago.
It was like, 10 years ago.
We had big black out caused by a race condition in the code run by the power companies.
There been medical instruments that have fried people, killed them and maimed them, because of race conditions.
These are really serious bugs.
Question?
[INAUDIBLE] when you said, the only time that that code is actually execute serially?
It could execute in parallel if it happened that these guys executed before these guys.
If you think of a larger context, a whole bunch of these things, and I have two routines where they're both incrementing x in the middle of great big parallel programs, it could be that they're executing perfectly well in parallel.
But if those two small sections of code happen to execute like this or like this, then you're going to end up with it executing correctly.
But if they execute sort of at the same time, it would not necessarily behave correctly.
So there are two types of races that people talk about, a read race and a write race.
So suppose you have two instructions that access a location, x.
And suppose that a is parallel to b.
Both a and b are both reads, you get no race.
That's good.
Because there's no way.
But if one is a read and one is a write, then one of them is going to see a different value, depending upon whether it occurred before and after the write.
Or if they both are writing, one can lose a value.
So these are read races.
And this is a write race.
So we say that the two sections of code are independent if they have no determinacy races between them.
So for example, this piece of code is incrementing y, and this is incrementing x.
And y is not equal to x.
Those are independent pieces of code.
So to avoid races, you want to make sure that the iterations of your cilk_for are independent.
So what's going on in one iteration is different from what's going on in another.
That you're not writing something in one that you're using in the next, for example.
Between a cilk_spawn and the corresponding cilk_sync, the code of the spawn child should be independent of the code of the parent.
OK?
Including any code executed by additional spawned or called children.
So it's basically saying, when you spawn something off, don't then go and do something that's going to modify the same locations.
You really want to modify different locations.
It's fine if they both read the same locations.
But it's not fine for one of them to read and one of them to write.
One thing here to understand is that when you spawn a function, the arguments are actually executed serially before the actual spawn occurs.
So you evaluate the arguments, and you set it all up, then you spawn the function.
So the actual spawn occurs after the evaluation of arguments.
So they're evaluated in the parent.
Machine word size matters.
So this is generally the case for races.
By the way, races are not just Cilk stuff.
These races occur in all of these concurrency platforms.
I'm illustrating Cilk because that's what we're going to be using in our labs and so forth.
So it turns out machine word size matters.
And you can have races in packed data structures.
So for example, on some machines, if you declare a char a and char b in a struct, then updating x and x, b in parallel may cause a race, because they're both actually operating on a word basis.
Now on the Intel architectures, that doesn't happen.
Because Intel supports atomic updates of single bytes.
So you don't have to worry about it.
But if you were accessing bits within a word, you could end up with the same thing.
You access bit five and bit three, you think you're acting independently, but in fact, you're reading the whole word or the whole byte in order to access it.
The technology that you're going to be using fortunately comes with a race detector, which you will find invaluable for debugging your stuff.
And so this is kind of like a Valgrind for races.
What's good about this race detector is it provides a rock hard guarantee.
If you have a deterministic program that on a given input could possibly behave any differently from your serial program, from the corresponding serial program, if you got rid of the parallel keywords, this tool, Cilkscreen, guarantees to report and localize the offending race.
It'll tell you, you got a race between this location and that location.
And it's up to you to find it and fix it, but it can tell you that.
It employs regression test methodology, where the programmer provides test inputs.
So if you don't provide test inputs to elicit the race, you still can have a bug.
But if you have a test input that in any way could behave differently than the serial execution, bingo.
It'll tell you.
It identifies a bunch of things involving the race, including a stack trace.
It runs off the binary executable using what's called dynamic instrumentation.
So that's kind of like Valgrind, except it actually does this as it's running.
It uses a technology called PIN, which you can read about.
P-I-N, which is a nice platform for doing code rewriting and analysis on the fly.
It runs about 20 times slower than real time.
So you basically use it for debugging.
So the first part of project four is basically coming up to speed with this technology.
And so, there's some good things.
And that's going to be available tomorrow.
Is that what we said?
Yeah, that will be available tomorrow.
So this is actually-- this is tons of fun.
Most people in most places don't get to play with parallel technology like this.
The following content is provided under a Creative Commons license.
Your support help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to talk a bit more about parallelism and about how you get performance out of parallel codes.
And also, we're going to take a little bit of a tour underneath the Cilk++ runtime system so you can get an idea of what's going on underneath and why it is that when you code stuff, how it is that it gets mapped, scheduled on the processors.
So when people talk about parallelism, one of the first things that often comes up is what's called Amdahl's Law.
Gene Amdahl was the architect of the IBM360 computers who then left IBM and formed his own company that made competing machines and he made the following observation about parallel computing, he said-- and I'm paraphrasing here-- half your application is parallel and half is serial.
You can't get more than a factor of two speed up, no matter how many processors it runs on.
So if you think about it, if it's half parallel and you managed to make that parallel part run in zero time, still the serial part will be half of the time and you only get a factor of two speedup.
You can generalize that to say if some fraction alpha can be run in parallel and the rest must be run serially, the speedup is at most 1 over 1 minus alpha.
OK, so this was used in the 1980s in particular to say why it was that parallel computing had no future, because you simply weren't going to be able to get very much speedups from parallel computing.
You're going to spend extra hardware on the parallel parts of the system and yet you might be limited in terms of how much parallelism there is in a particular application and you wouldn't get very much speedup.
You wouldn't get the bang for the buck, if you will.
So things have changed today that make that not quite the same story.
The first thing is that with multicore computers, it is pretty much just as inexpensive to produce a p processor right now, like six processor machine as it is a one processor machine.
so it's not like you're actually paying for those extra processing cores.
They come for free.
Because what else are you're going to use that silicon for?
And the other thing is that we've had a large growth of understanding of problems for which there's ample parallelism, where that amount of time is, in fact, quite small.
And the main place these things come from, it turns out, this analysis is kind of a throughput kind of analysis.
OK, it says, gee, I only get 50% speedup for that application, but what most people care about in most interactive applications, at least for a client side programming, is response time.
And for any problem that you have that has a response time that's too long and its compute intensive, using parallelism to make it so that the response is much zippier is definitely worthwhile.
And so this is true, even for things like game programs.
So in game programs, they don't have quite a response time problem, they have what's called a time box problem, where you have a certain amount of time-- 13 milliseconds typically-- because you need some slop to make sure that you can go from one frame to another, but about 13 milliseconds to do a rendering of whatever the frame is that the game player is going to see on his computer or her computer.
And so in that time, you want to do as much as you possibly can, and so there's a big opportunity there to take advantage of parallelism in order to do more, have more quality graphics, have better AI, have better physics and all the other components that make up a game engine.
But one of the issues with Amdahl's Law-- and this analysis is a cogent analysis that Amdahl made-- but one of the issues here is that it doesn't really say anything about how fast you can expect your application to run.
In other words, this is a nice sort of thing, but who really can decompose their application into the serial part and the part that can be parallel?
Well fortunately, there's been a lot of work in the theory of parallel systems to answer this question, and we're going to go over some of that really outstanding research that helps us understand what parallelism is.
So we're going to talk a little bit about what parallelism is and come up with a very specific measure of parallelism, quantify parallelism, OK?
We're also going to talk a little bit about scheduling theory and how the Cilk++ runtime system works.
And then we're going to have a little chess lesson.
So who here plays chess?
Nobody plays chess anymore.
Who plays Angry Birds?
[LAUGHTER] OK.
So you don't have to know anything about chess to learn this chess lesson, that's OK.
So we'll start out with what is parallelism?
So let's recall first the basics of Cilk++.
So here's the example of the lousy Fibonacci that everybody parallelizes because it's good didactically.
We have the Cilk spawn statement that says that the child can execute in parallel with the parent caller and the sync that says don't go past this point until all your spawn children have returned.
And that's a local sync, that's just a sync for that function.
It's not a sync across the whole machine.
So some of you may have had experience with open MP barriers, for example, that's a sync across the whole machine.
This is not, this is just a local sync for this function saying when I sync, make sure all my children have returned before going past this point.
And just remember also that Cilk keywords grant permission for parallel execution.
They don't command parallel execution.
OK so we can always execute our code serially if we choose to.
Yes?
[UNINTELLIGIBLE] Can't this runtime figure that spawning an extra child would be more expensive?
Can't it like look at this and be like--
We'll go into it.
I'll show you how it works later in the lecture.
I'll show you how it works and then we can talk about what knobs you have to tune, OK?
So it's helpful to have an execution model for something like this.
And so we're going to look at an abstract execution model, which is basically asking what does the instruction trace look like for this program?
So normally when you execute a program, you can imagine one instruction executing after the other.
And if it's a serial program, all those instructions essentially form a long chain.
Well there's a similar thing for parallel computers, which is that instead of a chain as you'll see, it gets bushier and it's going to be a directed acyclic graph.
So let's take a look at how we do this.
So we'll the example of fib of four.
So what we're going to do is start out here with a rectangle here that I want you think about as sort of a function call activation record.
So it's a record on a stack.
It's got variables associated with it.
The only variable I'm going to keep track of is n, so that's what the four is there.
OK, so we're going to do fib of four.
So we've got in this activation frame, we have the variable four and now what I've done is I've color coded the fib function here and into the parts that are all serial.
So there's a serial part up to where it spawns, then there's recursively calling the fib and then there's returning.
So there's sort of three parts to this function, each of which is, in fact, a chain of serial instruction.
I'm going to collapse those chains into a single circle here that I'm going to call a strand.
OK, now what we do is we execute the strand, which corresponds to executing the instructions and advancing the program calendar up until the point we hit this fib of n minus 1.
At that point, I basically call fib of n minus 1.
So in this case, it's now going to be fib of 3.
So that means I create a child and start executing in the child, this prefix part of the function.
However, unlike I were doing an ordinary function call, I would make this call and then this guy would just sit here and wait until this frame was done.
But since it's a spawn, what happens is I'm actually going to continue executing in the parent and execute, in fact, the green part.
So in this case, evaluating the arguments, etc.
Then it's going to spawn here, but this guy, in fact, is going to what it does when it gets here is it evaluates n minus 2, it does a call of fib of n minus 2.
So I've indicated that this was a called frame by showing it in a light color.
So these are spawn, spawn, call, meanwhile this thing is going.
So at this point, we now have one, two, three things that are operating in parallel at the same time.
We keep going on, OK?
So this guy that does a spawn and has a continuation, this one does a call, but while he's doing a call, he's waiting for the return so he doesn't start executing the successor.
He stalled at the Cilk sink here.
And we keep executing and so as you can see, what's happening is we're actually creating a directed acyclic graph of these strands.
So here basically, this guy was able to execute because both of the children, one that he had spawned and one that he had called, have returned.
And so this fella, therefore, is able then to execute the return.
OK, so the addition of x plus y in particular, and then the return to the parent.
And so what we end up with is of all these serial chains of instructions that are represented by these strands, all these circles, they're embedded in the call tree like you would have in an ordinary serial execution.
You have a call tree that you execute up and down, you walk it like a stack normally.
Now, in fact, what we have is embedded in there is the parallel execution which form a DAG, directed acyclic graph.
So when you start thinking in parallel, you have to start thinking about the DAG as your execution model, not a chain of instructions.
And the nice thing about this particular execution model we're going to be looking at is nowhere did I say how many processors we were running on.
This is a processor oblivious model.
It doesn't know how many processors you're running on.
We simply in the execution model, are thinking about abstractly what can run in parallel, not what actually does run in parallel in an execution.
So any questions about this execution model?
OK.
So just so that we have some terminology, so the parallel instruction stream is a DAG with vertices and edges.
Each vertex is a strand, OK?
Which is a sequence of instructions not containing a call spawn sync, a return or thrown exception, if you're doing exceptions.
We're not going to really talk about exceptions much.
So they are supported in the software that we'll be using, but for most part, we're not going to have to worry about them.
OK so there's an initial strand where you start, and a final strand where you end.
Then each edge is a spawn or a call or return or what's called a continue edge or a continuation edge, which goes from the parent, when a parent spawns something to the next instruction after the spawn.
So we can classify the edges in that fashion.
And I've only explain this for spawm and sync, as you recall from last time, we also talked about Cilk four.
It turns out Cilk four is converted to spawns and syncs using a recursive divide and conquer approach.
We'll talk about that next time on Thursday.
So we'll talk more about Cilk four and how it's implemented and the implications of how loop parallelism works.
So at the fundamental level, the runtime system is only concerned about spawns and syncs.
Now given that we have a DAG, so I've taken away the call tree and just left the strands of a computation.
It's actually not the same as the computation we saw before.
We would like to understand, is this a good parallel program or not?
Based on if I understand the logical parallelism that I've exposed.
So how much parallelism do you think is in here?
Give me a number.
How many processors does it make sense to run this on?
Five?
That's as parallel as it gets.
Let's take a look.
We're going to do an analysis.
At the end of it, we'll know what the answer is.
So for that, let tp be the execution time on p processors for this particular program.
It turns but there are two measures that are really important.
The first is called the work.
OK, so of course, we know that real machines have caches, etc.
Let's forget all of that.
Just very simple algorithmic model where every strand, let's say, costs us unit time as opposed to in practice, they may be many instructions and so forth.
We can take that into account.
Let's take that into account separately.
So T1 is the work.
It's the time it if I had to execute it on one processor, I've got to do all the work that's in here.
So what's the work of this particular computation?
I think it's 18, right?
Yeah, 18.
So T1 is the work.
So even though I'm executing a parallel, I could it execute it serially and then T1 is the amount of work it would take.
The other measure is called the span, and sometimes called critical path length or computational depth.
And it corresponds to the longest path of dependencies in the DAG.
We call it T infinity because even if you had an infinite number of processors, you still can't do this one until you finish that one.
You can't do this one until you finish that one, can't do this one till you've finished that one and so forth.
So even with an infinite number of processors, I still wouldn't go faster than the span.
So that's why we denote by T infinity.
So these are the two important measures.
Now what we're really interested in is Tp for a given p. As you'll see, we actually can get some bounds on the performance on p processors just by looking at the work, the span and the number of processors we're executing on.
So the first bound is the following, it's called the Work Law.
The Work Law says that the time on p processors is at least the time on one processor divided by p. So why does that Work Law make sense?
What's that saying?
Sorry?
Like work is conserved sort of?
I mean, you have to do the same amount of work
You have to do the same amount of work, so on every time step, you can get p pieces of work done.
So if you're running for fewer than T1 over p steps, you've done less than T1 work over and time Tp.
So you won't have done all the work if you run for less than this.
So the time must be at least Tp, time Tp must be at least T1 over p. You only get to do p work on one step.
Is that pretty clear?
The second one should be even clearer, the Span Law.
On p processors, you're not going to go faster than if you had an infinite number of processors because the infinite processor could always use fewer processors if it's scheduled.
Once again, this is a very simple model.
We're not taking into account scheduling, we're not taking into account overheads or whatever, just a simple conceptual model for understanding parallelism.
So any questions about these two laws?
There's going to be a couple of formulas in this lecture today that you should write down and play with.
So these two, they may seem simple, but these are hugely important formulas.
So you should know that Tp is at least T1 over p, that's the Work Law and that Tp is at least T infinity.
Those are bounds on how fast you could execute.
Do I have a question in that back there?
OK so let's see what happens to work in span in terms of how we can understand our programs and decompose them.
So suppose that I have a computation A followed by computation B and I connect them in series.
What happens to the work?
How does the work of all this whole thing correspond to the work of A and the work of B?
What's that?
[UNINTELLIGIBLE]
Yeah, add them together.
You get T1 of A plus T1 of B.
Take the work of this and the work of this.
OK, that's pretty easy.
What about the span?
So the span is the longest path of dependencies.
What happens to the span when I connect two things in a series?
Yeah, it just sums as well because I take whatever the longest path is from here to here and then the longest one from here to here, it just adds.
But now let's look at parallel composition, So now suppose that I can execute these two things in parallel.
What happens to the work?
It just adds, just as before.
The work always adds.
The work is easy because it's additive.
What happens to the span?
What's that?
[UNINTELLIGIBLE]
It's the max of the spans.
Right, so whatever is the longest, whichever one of these ones has a longer span, that's going to be the span of the total.
Does that give you some Intuition So we're going to see when we analyze the spans of things that in fact, we're going to see maxes occurring all over the place.
So speedup is defined to be T1 over Tp.
So speedup is how much faster am I on p processors than I am on one processor?
Pretty easy.
So if T1 over Tp is equal to p, we say we have perfect linear speedup, or linear speedup.
That's good, right?
Because if I put on use p processors, I'd like to have things go p times faster.
OK, that would be the ideal world.
If T1 over Tp, which is the speedup, is greater than p, that says we have super linear speedup.
And in our model, we don't get that because of the work law.
Because the work law says Tp is greater than or equal to T1 over p and just do a little algebra here, you get T1 over Tp must be less than or equal to p. So you can't get super linear speedup.
In practice, there are situations where you can get super linear speedup due to caching effects and a variety of things.
We'll talk about some of those things.
But in this simple model, we don't get that kind of behavior.
And of course, the case I left out is the common case, which is the T1 over Tp is less than p, and that's very common people write code which doesn't give them linear speedup.
We're mostly interested in getting linear speedup here.
That's our goal.
So that we're getting the most bang for the buck out of the processors we're using.
OK, parallelism.
So we're finally to the point where I can talk about parallelism and give a quantitative definition of parallelism.
So the Span Law says that Tp is at least T infinity, right?
The time on p processors is at least the time on an infinite number of processors.
So the maximum possible speedup, that's T1 over Tp, given T1 and T infinity is T1 over T infinity.
And we call that the parallelism.
It's the maximum amount of speedup we could possibly attain.
So we have the speedup and the speedup by the Span Law that says this is the maximum amount we can get, we could also view it as if I look along the critical path of the computation.
It's sort of what's the average amount of work at every level.
The work, the total amount of stuff here divided by that length there that sort of tells us the width, what's the average amount of stuff that's going on in every step.
So for this example, what is the-- I forgot to put this on my slide-- what is the parallelism of this particular DAG here?
Two, right?
So the span has length nine-- this is assuming everything was unit time-- obviously in reality, when you have more instructions, you in fact would make it be whatever the length of this was in terms of number of instructions or what have you, of execution time of all these things.
So this is length 9, there's 18 things here, parallelism is 2.
So we can quantify parallelism precisely.
We'll see why it's important to quantify it.
So that the maximum speedup we're going to get when we run this application.
Here's another example we did before.
Fib of four.
So let's assume again that each strand takes unit time to execute.
So what is the work in this particular computation?
Assume every strand takes unit time to execute, which of course it doesn't, but-- anybody care to hazard a guess?
17, yeah, because there's four nodes here that have 3 plus 5.
So 3 times 4 plus 5 is 17.
So the work is 17.
OK, what's the span?
This one's tricky.
Too bad it's not a little bit more focused.
What the span?
8
8, that's correct.
Who got 7?
Yeah, so I got 7 when I did this and then I looked harder and it was 8.
It's 8, so here it is.
Here's the span.
There is goes.
Ooh that little sidestep there, that's what makes it 8.
OK so basically, it comes down here and I had gone down like that when I did it, but in fact, you've got to go over and back up.
So it's actually 8.
So that says that the parallelism is a little bit more than 2, 2 and 1/8.
What that says is that if I use many more than two processors, I can't get linear speedup anymore.
I'm only going to get marginal performance gains.
If I use more than 2, because the maximum speedup I can get is like 2.125 if I had an infinite number of processors.
So any questions about this?
So this by the way deceptively simple and yet, if you don't play around with it a little bit, you can get confused very easily.
Deceptively simple, very powerful to be able to do this.
So here we have for the analysis of parallelism, one of the things that we have going for us in using the Cilk tool suite is a program called Cilkview, which has a scalability analyzer.
And it is like the race detector that I talked to you about last time in that it uses dynamic instrumentation.
So you run it under Cilkview, it's like running it under [?
Valgrhen ?]
for example, or what have you.
So basically you run your program under it, and it analyzes your program for scalability.
It computes the work and span of your program to derive some upper bounds on parallel performance and it also estimates a scheduling overhead to compute what's called a burden span for lower bounds.
So let's take a look.
So here's, for example, here's a quick sort program.
So let's just see this is a c++ program.
So here we're using a template so that the type of items that I'm sorting I can make be a variable.
So tightening-- can we shut the back door there?
One of the TAs?
Somebody run up to-- thank you.
So we have the variable T And we're going to quick sort from the beginning to the end of the array.
And what we do is, just as you're familiar with quick sort, if there's actually something to be sorted, more than one thing, then we find the middle by partitioning the thing and this is a bit of a c++ magic to find the middle element.
And then the important part from our point of view is after we've done this partition, we quick sort the first part of the array, from beginning to middle and then from the beginning plus 1 or the middle, whichever is greater to the end.
And then we sync.
So what we're doing is quick sort where we're spawning off the two sub problems to be solved in parallel recursively.
So they're going to execute in parallel and they're going to execute in parallel and so forth.
So a fairly natural thing to divide, to do divide and conquer on quick sort because the two some problems can be operated on independently.
We just sort them recursively.
But we can sort them in parallel.
OK, so suppose that we are sorting 100,000 numbers.
How much parallelism do you think is in this code?
So remember that we're getting this recursive stuff done.
How many people think-- well, it's not going to be more than 100,000, I promise you.
So how many people think more than a million parallels?
Raise your hand, more than a million?
And how many people think more than 100,000?
And how many people think more than 10,000?
OK, between the two.
More than 1,000?
OK, how about more than 100?
100 to 1,000?
How about 10 to 100?
How about between 1 and 10?
So a lot of people think between 1 and 10.
Why do you think that there's so little parallels in this?
You don't have to justify yourself, OK. Well let's see how much there is according to Cilkview.
So here's the type of output that you'll get.
You'll get a graphical curve.
You'll also get a textual output.
But this is sort of the graphical output.
And this is basically showing what the running time here is.
So the first thing it shows is it will actually run your program, benchmark your program, on in this case, up to 8 course.
We ran it.
So we ran up to 8 course and give you what your measured speedup is.
So the second thing is it tells you the parallels.
If you can't read that it's, 11.21.
So we get about 11.
Why do you think it's not higher?
What's that?
It's the log
What's the log?
[UNINTELLIGIBLE]
Yeah, but you're doing the two things in parallel, right?
We'll actually analyze this.
So it has to do with the fact that the partition routine is a serial piece of code and it's big.
So the initial partitioning takes you 100,000-- sorry, 100 million steps of doing a partition-- before you get to do any parallelism at all.
And we'll see that in just a minute.
So it gives you the parallelism.
It also plots this.
So this is the parallelism.
Notice that's the same number, 11.21 is plotted as this bound.
So it tells you the span law and it tells you the work law.
This is the linear speedup.
If you were having linear speedup, this is what your program would give you.
So it gives you these two bounds, the work law and span law on your speedup.
And then it also computes what's called a burden parallelism, estimating scheduling overheads to sort of give you a lower bound.
Now that's not to say that your numbers can't fall outside this range.
But when they do, it will tell you essentially what the issues are with your program.
And we'll discuss how you diagnose some of those issues.
Actually that's in one of the handouts that we've provided.
I think that's in one of the handouts.
If not, we'll make sure it's among the handouts.
So basically, this gives you a range for what you can expect.
So the important thing here is to notice here for example, that we're losing performance, but it's not due to the parallelism, to the work law.
Basically, in some sense, what's happening is we are losing it because the Span Law because we're starting to approach the point where the span is going to be the issue.
So we'll talk more about this.
So the main thing is you have a tool that can tell you the work and span and so that you can analyze your own programs to understand are you bounded by parallelism, for example, in particular, in the code that you've written.
OK let's do a theoretical analysis of this to understand why that number is small.
So the main thing here is that the expected work, as you recall, of quick sort is order n log n. You tend to do order n log n work, you partition and then you're solving two problems of the same size.
If you actually draw out the recursion tree, it's log height with linear amount of work on every level for n log end total work.
The expected span, however, is order n because the partition routine is a serial program that partitions up the thing of size n in order n time.
So when you compute the parallelism, you get parallelism of order log n and log n is kind of puny parallelism, and that's our technical word for it.
So puny parallelism is what we get out of quick sort.
So it turns out there are lots of things that you can analyze.
Here's just a selection of some of the interesting practical algorithms and the kinds of analyses that you can do showing that, for example, with merge sort you can do it with work n log n. You can get a span of log qn and so then the parallelism is the ratio of the two.
In fact, you can actually theoretically get log squared n span, but that's not as practical an algorithm as the one that gives you log cubed n. And you can go through and there are a whole bunch of algorithms for which you can get very good parallelism.
So all of these, if you look at the ratio of these, the parallelism is quite high.
So let's talk a little bit about what's going on underneath and why parallelism is important.
So when you describe your program in Cilk, you express the potential parallelism of your application.
You don't say exactly how it's going to be scheduled, that's done by the Cilk++ scheduler, which maps the strands dynamically onto the processors at run time.
So it's going to do the load balancing and everything necessary to balance your computation off the number of processors.
We want to understand how that process works, because that's going to help us to understand how it is that we can build codes that will map very effectively on to the number of processors.
Now it turns out that the theory of the distributed schedulers such as is in Cilk++ is complicated.
I'll wave my hands about it towards the end, but the analysis of it is advanced.
You have to take a graduate course to get that stuff.
So instead, we're going to explore the ideas with a centralized, much simpler, scheduler which serves as a surrogate for understanding what's going on.
So the basic idea of almost all scheduling theory in this domain is greedy scheduling.
And so this is-- by the way, we're coming to the second thing you have to understand really well in order to be able to generate good code, the second sort of theoretical thing-- so the idea of a greedy scheduler is you want to do as much work as possible on each step.
So the idea here is let's take a look, for example, suppose that we've executed this part of the DAG already.
Then there are certain number of strands that are ready to execute, meaning all their predecessors have exited.
How many strands are ready to execute on this DAG?
Five, right?
These guys.
So those five strands are ready to execute.
So the idea is-- and let me illustrate for p equals 3-- the idea is to understand the execution in terms of two types of steps.
So in a greed schedule, you always do as much as possible.
So is what would be called a complete step because I can schedule all three processors to have some work to do on that step.
So which are the best three guys to be able to execute?
Yes, so I'm not sure what the best three are, but for sure, you want to get this guy and this guy, right?
Maybe that guy's not, but this guy, you definitely want to execute.
And these guys, I guess, OK.
So in a greedy schedule, no, you're not allowed to look to see which ones are the best execute.
You don't know what the future is, the scheduler isn't going to know what the future is so it just executes any p course.
You just execute any p course.
In this case, I executed the p strand.
In this case, I executed these three guys even though they weren't necessarily the best.
And in a greedy scheduler, it doesn't look to see what's the best one to execute, it just executes as many as it can this case.
In this case, it's p. Now we have what's called an incomplete step.
Notice nothing got enabled.
That was sort of too bad.
So there's only two guys that are ready to go.
What do you think happens if I have an incomplete step, namely p strands are ready, fewer than p strands are ready?
I just to execute all of them, as many as I can.
Run all of them.
So that's what a greedy scheduler does.
Just at every step, it executes as many as it can and we can classify the steps as ones which are complete, meaning we used all our processors versus incomplete, meaning we only used a subset of our processors in scheduling it.
So that's what a greedy scheduler does.
Now the important thing, which is the analysis of this program.
And this is, by the way, the single most important thing in scheduling theory but you're going to ever learn is this particular theory.
It goes all the way back to 1968 and what it basically says it is any greedy scheduler achieves a bound of T1 over p plus T infinity.
So why is that an interesting upper bound?
Yeah?
That says that it's got the refinement of what you said before, even if you add as many processors as you can, basically you're bounded by T infinity
Yeah
It's compulsory
So basically, each of these, this term here is the term in the Work Law.
This is the term in the Span Law, and we're saying you can always achieve the sum of those two lower bounds as an upper bound.
So let's see how we do this and then we'll look at some of the implications.
Question, do you have a question?
No?
So here's the proof that you meet this.
So that the proof says-- and I'll illustrate for P equals 3-- how many complete steps could we have?
So I'll argue that the number of complete steps is at most T1 over p. Why is that?
Every complete step performs p work.
So if I had more complete steps than T1 over p, I'd be doing more than T1 work.
But I only have T1 work to do.
OK, so the maximum number of complete steps I could have is at most T1 over p. Do people follow that?
So the trickier part of the proof, which is not all that tricky but it's a little bit trickier, is the other side.
How many incomplete steps could I have?
So we execute those.
So I claim that the number of incomplete steps is bounded by the critical path length, by the span.
Why is that?
Well let's take a look at the part of DAG that has yet to be executed.
So that this gray part here.
There's some span associated with that.
In this case, it's this longest path.
When I execute all of the ready threads that are ready to go, I guarantee to reduce the span of that unexecuted DAG by at least one.
So as I do here, so I reduce it by one when I execute.
So if I have a complete step, I don't guaranteed to reduce the span of the unexecuted DAG, because I may execute things as I showed you in this example, you don't actually advance anything.
But I execute all the ready threads on an incomplete step, and that's going to reduce it by one.
So the number of incomplete steps is at most infinity.
So the total number of steps is at most the sum.
So as I say, this proof you should understand in your sleep because it's the most important scheduling theory proof that you're going to probably see in your lifetime.
It's very old, and really, very, very simple and yet, there's a huge amount of scheduling theory if you have a look at scheduling theory, that comes out of this just making this same problem more complicated and more real and more interesting and so forth.
But this is really the crux of what's going on.
Any questions about this proof?
So one corollary of the greedy scheduling algorithm is that any greedy scheduler achieves within a factor of two of optimal scheduling.
So let's see why that is.
So it's guaranteed as an upper bound to get within a factor of two of optimal.
So here's the proof.
So let's Tp star be the execution time produced by the optimal scheduler.
This is the schedule that knows the whole DAG in advance and can schedule things exactly where they need to be scheduled to minimize the total amount of time.
Now even though the optimal scheduler can schedule very officially, it's still bound by the Work Law and the Span Law.
So therefore, Tp star has still got to be greater than T1 over p and greater than T infinity by the Work and Span Laws.
Even though it's optimal, every scheduler must obey the Work Laws and Spam Law.
So then we have, by the greedy scheduling theorem, Tp is at most T1 over p plus T infinity.
Well that's at most twice the maximum of these two values, whichever is larger.
I've just plugged in to get the maximum of those two and that's at most, by this equation, twice the optimal time.
So this is a very simple corollary says oh, greedy scheduling is actually pretty good.
It's not optimal, in fact, optimal scheduling is mP complete.
Very hard problem to solve.
But getting within a factor of two, you just do greedy scheduling, it works just fine.
More importantly is the next corollary, which has to do is when do you get linear speedup?
And this is, I think, the most important thing to get out of this.
So any greedy scheduler achieves near perfect linear speedup whenever-- what's this thing on the left-hand side?
What's the name we call that?-- the parallelism, right?
That's the parallelism, is much bigger than the number of processors you're running on.
So if the number of processors are running on is smaller than the parallelism of your code says you can expect near perfect linear speedup.
OK, so what does that say you want to do in your program?
You want to make sure you have ample parallelism and then the scheduler will be able to schedule it so that you get near perfect linear speedup.
Let's see why that's true.
So T1 over T infinity is much bigger than p is equivalent to saying that T infinity is much less than T1 over p. That's just algebra.
Well what does that mean?
The greedy scheduling theorem says Tp is at most T1 over p plus T infinity.
We just said that if we have this condition, then T infinity is very small compared to T1 over p. So if this is negligible, then the whole thing is about T1 over p. Well that just says that the speedup is about p. So the name of the game is to make sure that your span is relatively short compared to the amount of work per processor that you're doing.
And in that case, you'll get linear speedup.
And that happens when you've got enough parallelism compared to the number processors you're running on.
Any questions about this?
This is like the most important thing you're going to learn about parallel computing.
Everything else we're going to do is going to be derivatives of this, so if you don't understand this, you have a hard time with the other stuff.
So in some sense, it's deceptively simple, right?
We just have a few variables, T1, Tp, T infinity, p, there's not much else going on.
But there are these bounds and these elegant theorems that tell us something about how no matter what the shape of the DAG is or whatever, these two values, the work and the span, really characterize very closely where it is that you can expect to get linear speedup.
Any questions?
OK, good.
So the quantity T1 over PT infinity, so what is that?
That's just the parallelism divided by p. That's called the parallel slackness.
So this parallel slackness is 10, means you have 10 times more parallelism than processors.
So if you have high slackness, you can expect to get linear speedup.
If you have low slackness, don't expect to get linear speedup.
OK. Now the scheduler we're using is not a greedy scheduler.
It's better in many ways, because it's a distributed, what's called work stealing scheduler and I'll show you how it works in a little bit.
But it's based on the same theory.
Even though it's a more complicated scheduler from an analytical point of view, it's really based on the same theory as greedy scheduling.
It guarantees that the time on p processors is at most T1 over p plus order T infinity.
So there's a constant here.
And it's a randomized scheduler, so it actually only guarantees this in expectation.
It actually guarantees very close to this with high probability.
OK so the difference is the big O, but if you look at any of the formulas that we did with the greedy scheduler, the fact that there's a constant there doesn't really matter.
You get the same effect, it just means that the slackness that you need to get linear speedup has to not only overcome the T infinity, it's also got to overcome the constant there.
And empirically, it actually turns out this is not bad as an estimate using the greedy bound.
Not bad as an estimate, so this is sort of a model that we'll take as if we're doing things with a greedy scheduler.
And that will be very close for what we're actually going to see in practice with the Cilk++ scheduler.
So once again, it means near perfect linear speedup as long as p is much less than T1 over T infinity generally.
And so Cilkview allows us to measure T1 and T infinity.
So that's going to be good, because then we can figure out what our parallelism is and look to see how we're running on typically 12 cores, how much parallels do we have?
If our parallelism is 12, we don't have a lot of slackness.
We won't get very good speedup.
But if we have a parallelism of say, 10 times more, say 120, we should get very, very good parallelism, very, very good speedup on 12 cores.
We should get close to perfect speedup.
So let's talk about the runtime system and how this work stealing scheduler works, because it different from the other one.
And this will be helpful also for understanding when you program these things what you can expect.
So the basic idea of the schedule is there's two strategies the people have explored for doing scheduling.
One is called work sharing, which is not what Cilk++ does.
But let me explain what work sharing is because it's helpful to contrast it with work stealing.
So in works sharing, what you do is when you spawn off some work, you say let me go find some low utilized processor and put that worked there for it to operate on.
The problem with work sharing is that you have to do some communication and synchronization every time you do a spawn.
Every time you do a spawn, you're going to go out.
This is kind of what Pthreads does, when you do Pthread create.
It goes out and says OK, let me create all of the things it needs to do and get it schedule then on a processor.
Work stealing, on the other hand, takes the opposite approach.
Whenever it spawns work, it's just going to keep that work local to it, but make it available for stealing.
A processor that runs out of work is going to go looking for work to steal, to bring back.
The advantage of work stealing is that the processor doesn't do any synchronization except when it's actually load balancing.
So if all of the processors have ample work to do, then what happens is there's no overhead for scheduling whatsoever.
They all just crank away.
And so you get very, very low overheads when there's ample work to do on each processor.
So let's see how this works.
So the particular way that it maintains it is that basically, each processor maintains a work deck.
So a deck is a double-ended queue of the ready strands.
It manipulates the bottom of the deck like a stack.
So what that says is, for example, here, we had a spawn followed by two calls.
And basically, it's operating just as it would have to operate in an ordinary stack, an ordinary call stack.
So, for example, this guy says call, well it pushes a frame on the bottom of the call stack just like normal.
It says spawn, it pushes a spawn frame on the bottom of the deck.
In fact, of course, it's running in parallel, so you can have a bunch of guys that are both calling and spawning and they all push whatever their frames are.
When somebody says return, you just pop it off.
So in the common case, each of these guys is just executing the code serially the way that it would normally executing in C or C++.
However, if somebody runs out of work, then it becomes a thief and it looks for a victim and the strategy that's used by Cilk++ is to look at random.
It says let me just go to any other processor or any other workers-- I call these workers-- and grab away some of their work.
But when it grabs it away, what it does is it steals it from the opposite end of the deck from where this particular victim is actually doing its work.
So it steals the oldest stuff first.
So it moves that over and now here what it's doing is it's stealing up to the point that it spawns.
So it steals from the top of the deck down to where there's a spawn on top.
Yes?
Is there always a spawn on the top of every deck?
Close, almost always.
Yes, so I think that you could say that there are.
So the initial deck does not have a spawn on top of it, but you could imagine that it did.
And then when you steal, you're always stealing from the top down to a spawn.
If there isn't something, if this is just a call here, this cannot any longer be stolen.
There's no work there to be stolen because this is just a single execution, there's nothing that's been spawned off at this point.
This is the result of having been spawned as opposed to that it's doing a spawn.
So yes, basically you're right.
There's a spawn on the top.
So it basically steals that off and then it resumes execution afterwards and starts then operating just like an ordinary deck.
So the theorem that you can prove for this type of scheduler is that if you have sufficient parallelism, so you all know what parallelism is at this point, you can prove that the workers steal infrequently.
So in a a typical execution, you might have a few hundred load balancing operations of this nature for something which is doing billions and billions of instructions.
So you steal infrequently.
If you're stealing infrequently and all the rest of the time you're just executing like the C or C++, hey, now you've got linear speedup because you've got all of these guys working all the time.
And so as I say, the main thing to understand is that there's this work stealing scheduler running underneath.
It's more complicated to analyze then the greedy scheduler, but it gives you pretty much the same qualitative kinds of results.
And the idea then is that the stealing occurs infrequently so you get linear speedup.
So the idea then is just as with greedy scheduling, make sure you have enough parallelism, because then the load balancing is a small fraction of the time these processors are spending executing the code.
Because whenever it's doing things like work stealing, it's not working on your code executing, making it go fast.
It's doing bookkeeping and overhead and stuff.
So you want to make sure that stays low.
So any questions about that?
So specifically, we have these bounds.
You have achieved this expected running time, which I mentioned before.
Let me give you a pseudo-proof of this.
So this is not a real proof because it ignores things like independence of probabilities.
So when you do a probability analysis, you're not allowed to multiply probabilities unless they're independent.
So anyway, here I'm multiplying probabilities that are independent.
So the idea is you can view a processor as either working or stealing.
So it goes into one of two modes.
It's going to be stealing if it's run out of work, otherwise it's working.
So the total time all processors spend working is T1, hooray, that's at least a bound.
Now it turns out that every steal has a 1 over p chance of reducing the span by one.
So you can prove that of all of the work that's in the top of all those decks that those are where any of the ready threads are going to be there are in a position of reducing the span if you execute them.
And so whenever you steal, you have a 1 over p chance of hitting the guy that matters for the span of unexecuted DAG.
So the same kind of thing as in theory.
You have a 1 over p chance.
So the expected cost of all steals is order PT infinity.
So this is true, but not for this reason.
But it's kind, the intuition is right.
So therefore the cost of all steals is PT infinity and the cost of the work is T1, so that's the total amount of work and time spent stealing by all the p processors.
So to get the time spent doing that, we divide by p, because they're p processors.
And when I do that, I get T1 over p plus order T infinity.
So that's kind of where that bound is coming from.
So you can see what's important here is that the term, that order T infinity term, this the one where all the overhead of scheduling and synchronization is.
There's no overhead for scheduling and synchronization in the T1 over p term.
The only overhead there is to do things like mark the frames as being a steel frame or a spawn frame and do the bookkeeping of the deck as you're executing so the spawn can be implemented very cheaply.
Now in addition to the scheduling things, there are some other things to understand a little bit about the scheduler and that is that it supports the C, C++ rule for pointers.
So remember in C and C++, you can pass a pointer to stack space down, but you can't pass a pointer to stack space back to your parent, right?
Because it popped off.
So if you think about a C or C++ execution, let's say we have this call structure here.
A really cannot see any of the stack space of B,C,D or E. So this is what A gets to see.
And B, meanwhile, can see A space, because that's down on the stack, but it can't see C, D or E. Particularly if you're executing this serially, it can't see C because C hasn't executed yet when B executes.
However, C, it turns out, the same thing.
I can't see any of the variables that might be allocated in the space for B when I'm executing here on a stack.
You can see them in a heap, but not on the stack, because B has been popped off at that point and so forth.
So this is basically the normal rule, the normal views of stack that you get in C or C++.
In Cilk++, you get exactly the same behavior except that multiple ones of these views may exist at the same time.
So if, for example, B and C are both executing at the same time, they each will see their own stack space and a stack space.
And so the cactus stack maintains that fiction that you can sort of look at your ancestors and see your ancestors, but now it's maintained.
It's called a cactus stack because it's kind of like a tree structure upside down, like a what's the name of that big cactus out West?
Yes, saguaro.
The saguaro cactus, yep.
This kind of looks like that if you look at the stacks.
This leads to a very powerful bound on how much space your program is using.
So normally, if you do a greedy scheduler, you could end up using gobs more space then you would in a serial execution, gobs more stack space.
In Cilk++ programs, you have a bound.
It's p times s1 is the maximum amount of stack space you'll ever use where s1 is the stack space used by serial execution.
So if you can keep your serial execution to use a reasonable amount of stack space-- and usually it does-- then in parallel, you don't use more than p times that amount of stack space.
And the proof for that is sort of by induction, which basically says there's a property called the Busy Leaves Property that says that if you have a leaf that's being worked on but hasn't been completed-- so I've indicated those by the purple and pink ones-- then if it's a leaf, it has a worker executing on it.
And so therefore, if you look at how much stack space you're using, each of these guys can trace up and they may double count the stack space, but it'll still be bounded by p times the depth that they're at, or p times s1, which is the maximum amount.
So it has good space bounds.
That's not so crucial for you folks to know as a practical matter, but it would be if this didn't hold.
If this didn't hold, then you would have more programming problems than you'll have.
The implications of this work stealing scheduler is interesting from the linguistic point of view, because you can write a code like this, so for i gets one to a billion, spawn some sub-routine foo of i and then sync.
So one way of executing this, the way that the work sharing schedulers tend to do this, is they say oh, I've got a billion tasks to do.
So let me create a billion tasks and now schedule them and the space just vrooms to store all those billion tasks, it gets to be huge.
Now of course, they have some strategies they can use to reduce it by bunching tasks together and so forth.
But in principle, you got a billion pieces of work to do even if you execute on one processor.
Whereas in the work stealing type execution, what happens is you execute this in fact depth research.
So basically, you're going to execute foo of 1 and then you'll return.
And then you'll increment i and you'll execute foo of 2, and you'll return.
At no time are you using more than in this case two stack frames, one for this routine here and one for foo because you basically keep going up.
You're using your stack up on demand, rather than creating all the work up front to be scheduled.
So the work stealing scheduler is very good from that point of view.
The tricky thing for people to understand is that if executing on multiple processors, when you do Cilk spawn, the processor, the worker that you're running on, is going to execute foo of 1.
The next statement-- which would basically be incrementing the counter and so forth-- is executed by whatever processor comes in and steals that continuation.
So if you had two processors, they're each going to basically be executing.
The first processor isn't the one that excuse everything in this function.
This function has its execution shared, the strands are going to be shared where the first part of it would be done by processor one and the latter part of it would be done by processor two.
And then when processor one finishes this off, it might go back and steal back from processor two.
So the important thing there is it's generating its stack needs sort of on demand rather than all up front, and that keeps the amount of stack space small as it executes.
So the moral is it's better to steal your parents from their children than stealing children from their parents.
So that's the advantage of doing this sort of parent stealing, because you're always doing the frame which is an ancestor of where that worker is working and that means resuming a function right in the middle on a different processor.
That's kind of the magic of the technologies is how do you actually move a stack frame from one place to another and resume it in the middle?
Let's finish up here with a chess lesson.
I promised a chess lesson, so we might as well do some fun and games.
We have a lot of experience at MIT with chess programs.
We've had a lot of success, probably our closest one was Star Socrates 2.0, which took second place in the world computer chess championship running on an 1824 node Intel Paragon, so a big supercomputer running with a Cilk scheduler.
We actually almost won that competition, and it's a sad story that maybe be sometime around dinner or something I will tell you the sad story behind it, but I'm not going to tell you why we didn't take first place.
And we've had a bunch of other successes over the years.
Right now our chess programming is dormant, we're not doing that in my group anymore, but in the past, we had some very strong chess playing programs.
So what we did with Star Socrates, which is one of our programs, was we wanted to understand the Cilk scheduler.
And so what we did is we ran a whole bunch of different positions on different numbers of processors which ran for different amounts of time.
We wanted to plot them all on the same chart, and here's our strategy for doing it.
What decided to do was do a standard speedup curve.
So a standard speedup curve says let's plot the number of processors along this axis and the speed up along that axis.
But in order to fit all these things on the same processor curve, what we did was we normalize the speedup.
So what's the maximum pot?
So here's the speedup.
If you look the numerator here, this is the speedup, T1 over Tp.
What we did is we normalized by the parallelism.
So we said what fraction of perfect speedup can we get?
So here one says that I got exactly a speedup, this is the maximum possible speed up that I can get because the maximum possible value of T1 over p is T1 over T infinity.
So that's sort of the maximum.
On this axis, we said how many processors are you running on it?
Well, we looked at that relative to essentially the slackness.
So notice by normalizing, we essentially have here the inverse of the slackness.
So 1 here says that I'm running on exactly the same number of processors as my parallelism.
A tenth here says I've got a slackness of 10, I'm running on 10 times fewer processors then parallelism.
Out here, I'm saying I got way more processors than I have parallelism.
So I plotted all the points.
So it doesn't show up very well here, but all those green points, there are a lot of green points here, that's our performance, measured performance.
You can sort of see they're green there, not the best color for this projector.
So we plot on this essentially the Work Law and the Span Law.
So this is the Work Law, it says linear speedup, and this is the Span Law.
And you can see that we're getting very close to perfect linear speedup as long as our slackness is 10 or greater.
See that?
It's hugging that curve really tightly.
As we approach a slackness of 1, you can see that it starts to go away from the linear speedup curve.
So for this program, if you look, it says, gee, if we were running with 10 time, slackness of 10, 10 times more parallelism than processors, we're getting almost perfect linear speedup in the number of processors we're running on across a wide range of number of processors, wide range of benchmarks for this chess program.
And in fact, this curve is the curve.
This is not an interpolation here, but rather it is just the greedy scheduling curve, and you can see it does a pretty good job of going through all the points here.
Greedy scheduling does a pretty good job of predicting the performance.
The other thing you should notice is that although things are very tight down here, as you approach up here, they start getting more spread.
And the reason is that as you start having more of the span mattering in the calculation, that's where all the synchronization, communication, all the overhead of actually doing the mechanics of moving a frame from one processor to another take into account, so you get a lot more spread as you go up here.
So that's just the first part of the lesson.
The first part was, oh, the theory works out in practice for real programs.
You have like 10 times more parallelisms than processors, you're going to do a pretty good job of getting linear speedup.
So that says you guys should be shooting for parallelisms on the order of 100 for running on 12 cores.
Somewhere in that vicinity you should be doing pretty well if you've got parallelism of 100 when you measure it for your codes.
So we normalize by the parallel there.
Now the real lesson though was understanding how to use things like work and span to make decisions in the design of our program.
So as it turned out, Socrates for this particular competition was to run on a 512 processor connection machine at the University of Illinois.
So this was in the mid in the early 1990's.
It was one of the most powerful machines in the world, and this thing is probably more powerful today.
But in those days, it was a pretty powerful machine.
I don't know whether this thing is, but this thing probably I'm pretty sure is more powerful.
So this was a big machine.
However here at MIT, we didn't have a great big machine like that.
We only had a 32 processor CM5.
So we were developing on a little machine expecting to run on a big machine.
So one of the developers proposed to change the program that produced a speedup of over 20% on the MIT machine.
So we said, oh that's pretty good, 25% improvement.
But we did a back of the envelope calculation and rejected that improvement because we were able to use work and span to predict the behavior on the big machine.
So let's see how that worked out, why that worked out.
So I've fudged these numbers so that they're easy to do the math on and easy to understand.
The real numbers actually though did sort out very, very similar to what I'm saying, just they weren't round numbers like I'm going to give you.
So the original program ran for let's say 65 seconds on 32 cores.
The proposed program ran for 40 seconds on 32 cores.
Sounds like a good improvement to me.
Let's go for the faster program.
Well, let's hold your horses.
Let's take a look at our performance model based on greedy scheduling.
That Tp is T1 over p plus infinity.
What component we really need to understand the scale this, what component of each of these things is work and which is span?
Because that's how we're going to be able to predict what's going to happen on the big machine.
So indeed, this original program had a work of 2048 seconds and a span of one second.
Now chess, it turns out, is a non-deterministic type of program where you use speculative parallelism, and so in order to get more parallelism, you can sacrifice and do more work versus less work.
So this one over here that we improved it to had less work on the benchmark, but it had a longer span.
So it had less work but a longer span.
So when we actually were going to run this, well first of all, we did the calculation and it actually came out pretty close.
I was kind of surprised how close the theory matched.
We actually on 32 processors when you do the work spanned calculation, you get the 65 seconds on a 32 processor machine, here we had 40 seconds.
But now what happens when we scale this to the big machine?
Here we scaled it to 512 cores.
So now we take the work divided by the number of processors, 512, plus 1, that's 5 seconds for this.
Here we have the work but we now have a much larger span.
So we have two seconds of work for processor, but now eight seconds of span for a total of 10 seconds.
So had we made this quote "improvement," our code would have been half as fast.
It would not have scaled.
And so the point is that work and span typically will beat running times for predicting scalability of performance.
So you can measure a particular thing, but what you really want to know is this thing this going to scale and how is it going to scale into the future.
So people building multicore applications today want to know that they coded up.
They don't want to be told in two years that they've got to recode it all because the number of cores doubled.
They want to have some future-proof notion that hey, there's a lot of parallelism in this program.
So work and span, work and span, eat it, drink it, sleep it.
Work and span, work and span, work and span, work and span, work and span, OK?
Work and span.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, more parallel programming, as we will do for the next couple lectures as well.
So today, we're going to look at how to analyze multi-threaded algorithms, and I'm going to start out with a review of what I hope most of you know from 6006 or 6046, which is how to solve divide and conquer recurrences.
Now, we know that we can solve them with recursion trees, and that gets tedious after a while, so I want to go through the so-called Master Method to begin with, and then we'll get into the content of the course.
But it will be very helpful, since we're going to do so many divide and conquer recurrences.
The difference between these divide and conquer recurrences and the ones for caching is that caching is all tricky by the base condition.
Here, are all the recurrences are going to be nice and clean, just like you learn in your algorithms class.
So we'll start with talking about it, and then we'll go through several examples of analysis of algorithms.
And it'll also tell us something about what we need to do to make our code go fast.
So the main method we're going to use is called the Master Method.
It's for solving recurrences of the form t of n equals a t of n over b plus f of n, where we have some technical conditions, a is greater than or equal to 1, b is greater than one, and f is asymptotically positive.
As f gets large, it becomes positive.
When we give a recurrence like this, normally if the base case is order one, it's convention not give it, to just assume, yeah, we understand that when n is small enough, the result is constant.
As I say, that's the place where this differs from the way that we solve recurrences for caching, where you have to worry about what is the base case of the recurrence.
So the way to solve this is, in fact, the way we've seen before.
It's a recursion tree.
So we start with t of n, and then we replace t of n by the right hand side, just by substitution.
So what's always going to be in the tree as we develop it is the total amount of work.
So we basically replace it by f of n plus a copies of t of n over b.
And then each of those we replace by a copies so t of n over b squared.
And so forth, continually replacing until we get down to t of 1.
And at the point t of 1, we no longer can substitute here, but we know that t of 1 is order 1.
And now what we do is add across the rows.
So we get f of n, we get a, f of n over b, a squared f of n over b squared, and we keep going to the height of this, which we're dividing by the argument by b each time.
So to get down to 1 is just log base b of n. So the number of leaves is, since this is a regular [?
a area ?]
tree, is a to the height, which is a to the log base b of n. And for each of those, we're paying t of 1, which is order 1.
And so now, it turns out there's no closed form solution.
If I add up all these terms, there's no closed form solution.
But there are three common situations that occur in practice.
So yes, this is just n to the log base b of a, just this term, not the sum, just this term.
So three cases have to do with comparing the number of leaves, which is times order 1, with f of n. So the first case is the case where n the log base b of a is bigger than f of n. So whenever you're given a recurrence to compute n to the log base b of a-- I hope this is repeat for most people.
If not, that's fine, but hopefully it'll get you caught up.
n log base b of a, if it's much bigger than f of n, then these terms are geometrically increasing.
And since it's geometrically increasing, all that matters is the base case.
In fact, actually, it has to be not just greater than, it's got to be greater than by a polynomial amount, by some n to the epsilon amount for some epsilon greater than 0.
So it might be n to the 1/2, it might be n to the 1/3, it could be n to the 100th.
But what it can't be is log n, because log n is less than any polynomial amount.
So it's got to exceed it by at least n to the epsilon for some epsilon.
In that case, it's geometrically increasing, the answer is just what's at the leaves.
So that's case one, geometrically increasing.
Case two is when things are actually fairly equal on every level.
And the general case we'll look at is when it's arithmetically increasing.
So in particular, this occurs when f of n is n to the log base b of a times log to some power, n, for some constant k that's at least 0.
So this is the situation.
If k is equal to 0, it just says that f of n, the amount here is exactly the same as the number of leaves.
And that case, it turns out that every level has almost exactly the same amount.
And since there are log n levels, you tack on an extra log n for the solution.
In fact, the solution is one more log.
Turns out that if it's growing arithmetically with layer, you basically take on one extra log.
It's actually like doing the integral of a as an arithmetic series.
If you're adding the terms of, say, i squared, the result is i cubed.
If you have a summation that goes from i equals 1 to n of i squared, the result is proportional to n cubed.
And similarly, if it's i to any k, the result is going to be n to the k plus 1, and that's basically what's going on here.
And then the third case is when it's geometrically decreasing, when the amount at the root dominates.
So in this case, if it's much less than n, and specifically, f of n is at least n to the epsilon less than log base b of a, for some constant epsilon, it turns out in addition, you need f of n to satisfy a regularity condition, but this regularity condition is satisfied by all the normal functions that we're going to come up.
It's not satisfied by things like n to the sine n, which oscillates like crazy.
But it also isn't satisfied by exponentially growing functions.
But it is satisfied by anything that's polynomial, or polynomial times a logarithm, or what have you.
So generally, we don't really have to check this too carefully.
And then the answer there is just it's order f of n, because it's geometrically decreasing, f of n dominates.
So is this review for everybody?
Pretty much, yeah, yeah, yeah?
You can do this in your head, because we're going to ask you to do this in your head during the lecture?
Yeah, we're all good?
OK, good.
One of the things that students, when they learn this in an algorithms class, don't recognize is that this also tells you where in your program, where in your recursive program, you should bother to try to eke out constant factors.
So if you think about it, for example, in case three here, it's geometrically decreasing.
Does it make sense to try to optimize the leaves?
No, because very little time is spent there.
It makes sense to optimize what's going on at the root, and to save anything you can at the root.
And sometimes, the root in particular has special properties that aren't true of the internal nodes that you can take advantage of, that you may not be able to take advantage of regularly.
But since it's going to be dominated by the root, trying to save in the root makes sense.
Correspondingly, if we're in case one, in case one, it's absolutely critical that you coarsen the recursion, because all the work is down at this level.
And so if you want to get additional performance, you want to basically move this up high enough that you can cut off that constant amount and get factors two, three, sometimes more, out of your code.
So understanding the structure of the recursion allows you to figure out where it is that you should optimize your code.
Of course, with loops, it's much easier.
Where do you spend your time with loops to make code go fast?
The innermost loop, right, because that's the one that's executing the most.
The outer loops are not that important.
This is sort of the corresponding thing for recursion.
Figure out where the recursion is spending the time, that's where you spend time eking out extra factors.
Here's the cheat sheet.
So if it's n to the log base b of a minus epsilon, the answer's n to the log base b of a.
If it's plus epsilon, it's f of n. And if it's n to the log base b of a times a logarithmic factor, where this logarithm has the exponent greater than or equal to 0, you add a 1.
This is not all of the situations.
There are situations which don't occur.
OK, quick quiz.
So t of n equals 4t of n over 2 plus n. What's the solution?
n squared, good.
So this is n to the log base b of a, is n to the log base 2 of 4, which is n squared.
That's much bigger than n. It's bigger by a factor of n. Here, the epsilon of 1 would do, so would an epsilon of 1/2 or an epsilon of 1/4, but in particular, an epsilon of 1 would do.
That's case one.
The n squared dominates, so the answer is n squared.
The basic idea is whichever side dominates for case one and case three, that's the one that is the answer.
Here we go.
What about this one?
n squared log n, because the two sides are about the same size.
It's n squared times log to the 0n, tack on the extra log.
How about this one?
n cubed.
How about this one?
[INAUDIBLE].
Master Theorem [INAUDIBLE]?
Yeah, the Master Theorem does not apply to this one.
It looks like it's case two with an an exponent of minus 1, but that's bogus because the exponent of the log must be greater than or equal to 0.
So instead, this actually has a solution, so it does not apply, it actually has the solution n squared, log log n, but that's not covered by the Master Theorem.
You can have an infinite hierarchy of narrowing in things.
So if you don't have a solution to something that looks like a Master Theorem type of recurrence, what's your best strategy for solving it?
[INAUDIBLE]
What's that?
[INAUDIBLE]
So a recursion tree can be good, but actually, the best is the substitution method, basically proving it by induction.
And the recursion tree can be very helpful in giving you a good guess for what you think the answer is.
But the most reliable way to prove any of these things is using substitution method.
Good enough.
So that was review for, I hope, most people?
Yeah?
OK, good.
OK, let's talk about parallel programming.
We're going to start out with loops.
So last time, we talked about how the cilk++ runtime system is based on, essentially, implementing spawns and syncs, using the work stealing algorithm, and we talked about scheduling and so forth.
We didn't talk about how loops are implemented, except to mention that they were implemented with divide and conquer.
So here I want to go into the subtleties of loops, because probably most programs that occur in the real world these days are programs where people just simply say, make this a parallel loop.
That's it.
So let's take an example of the in-place matrix transpose, where we're basically trying to flip everything along the main diagonal.
I've used this figure before, I think.
So let's just do it not cache efficiently.
So the cache efficient algorithm actually parallelizes beautifully also, but let's not look at the cache efficient version, the divide and conquer version.
Let's look at a looping version to understand.
And once again here, as I did before, I'm going to make the indices for my implementation run from 0, not 1.
And basically, I have an outer loop that goes from i equals 1 up to n minus 1, and an inner loop that goes from j equals 0 up to i minus 1.
And then I do a little swap in there.
And in here, the outer loop I've parallelized, the inner loop is running serially.
So let's take a look at analyzing this particular piece of code to understand what's going on.
So the way this actually gets implemented is as follows.
So here's the code on the left.
What actually happens in the cilk++ compiler is it converts it into recursion, divide and conquer recursion.
So what it does is it has a range from low to high, is sort of the common case.
We're going to call it on from 1 to n minus 1, because that's the indexes that I've given to the cilk_for loop.
And what we do is, if I have a region to do divide and conquer on, a set of values for i, I basically divide that in half.
And then I recursively execute the first half, and then the second half, and then cilk_sync.
So the two sides are going off in parallel.
And then if I am at the base, then I go through the inner loop and do the swap of the values in the inner loop.
So the outer loop is the one that's the parallel loop.
That one we're doing divide and conquer on.
So we basically recursively spawn the first half, execute the second, and then each of those recursively does the same thing until all the iterations have been done.
Any questions about how that operates?
So this is the way all the parallel loops are done, is basically this strategy.
Now here, I mention that this is in fact what happens is this test here is actually coarsened.
So we don't want to go all the way to n equals 1, because then we'll have this recursion call overhead every time we do a call.
So in fact, what happens is you go down to some grain size of some number of iterations, and at that number of iterations, it then just runs through it as an ordinary serial four loop, in order not to pay the function call overhead all the way down.
We're going to look exactly at that issue.
So if I take a look at it from using the DAG model that I introduced last time, remember that the rectangles here kind of count as activation frames, stack frames on the call stack.
And the circles here are strands, which are sequences of serial code.
And so what's happening here is essentially, I'm running the code that divides it into two parts, and I spawn one part.
Then this guy spawns the other and waits for the return, and then these guys come back.
And then I keep doing that recursively, and then when I get to the bottom, I then run through the innermost loop, which starts out with just one element to do, two, three.
And so the number of elements-- for example, in this case where I've done eight, I go through eight elements at the bottom here if this were an eight by eight matrix that I was transposing.
So there's more work in these guys than there is over here.
So it's not something you just can map onto processors in some naive fashion.
It does take some load balancing to parallelize this loop.
Any questions about what's going on here?
Yeah?
Why is it that it's one, two, three, four, up to eight?
Take a look.
The inner loop goes from j to i.
So this guy just does one iteration of the inner loop, this guy does two, this guy does three, all the way up to this guy doing eight iterations, if it were an eight by eight matrix.
And in general, if it's n by n, it's going from one to n work up here, but only one work down here.
Because I'm basically iterating through a triangular iteration space to swap the matrix, and this is basically swapping row by row.
Questions?
Is that good?
Everybody see what's going on?
So now let's take a look and let's analyze this for work and span.
So what is the work of this in terms of n, if I have an n by n matrix?
What's the work?
The work is the ordinary serial running time, right?
It's n squared.
Good.
So basically, it's order n squared, because these guys are all adding up.
This is an arithmetic sequence up to n, and so the total amount in here is order n squared.
What about this part up here?
How much does that cost us for work?
How much is in the control overhead of doing that outer loop?
So asymptotically, how much is in here?
The total is going to be n squared.
That I guarantee you.
But what's going on up here?
How do I count that up?
I'm assuming that each [?
strain ?]
is going to be constant time?
Yeah, well in this case, it is constant time, for these up here, because what am I doing?
All I'm doing is the recursion code where I divide the range and then spawn off two things.
That takes me only a constant amount of manipulation to be able to do that.
So this is all order n. Total of order n here.
The reason is because in some sense, there are n leaves here.
And if you have a full binary tree, meaning every child is either a leaf or has two children, then the number of the internal nodes of the tree is one less than the number of leaves.
So that's a basic property of trees, that the number of the internal nodes here is going to be n minus 1, in this case.
In particular, we have 7 here.
Is that good?
So this part doesn't contribute significantly to the work.
Just this part contributes to the work.
Is that good?
What about the span for this?
What's the span?
Log n
It's not log n, but your heads are in the right place
The longest path is going [INAUDIBLE]
Which is the longest path going to be here, starting here and ending there, which way do we go?
Go all the way down
Which way?
To the right
Down to the right, over, down through this guy.
How big is this guy?
n. Go back up this way.
So how much is in this part going down?
Log n
Going down and up is log n, but this is n. Good.
So it's basically order n plus order log n, order n down here plus order log n up and down, that's order n. So the parallelism is the ratio of those two things, which is order n. So that's got good parallelism.
And so if you imagine doing this in a large number processors, very easy to get sort of your benchmark of maybe 10 times more parallelism than the number of processors that you're running on.
Everybody follow this?
Good.
So the span of the loop control is order log n. And in general, when you have a four loop with n iterations, the loop control itself is going to have log n is going to have to be added to the span every time you hit a loop, log of whatever the number of iterations is.
And then we have the maximum span of the body.
Well, the worst case for this thing in the body is when it's doing the whole thing, because whenever we're looking at spans, we're always looking what's the maximum of things that are operating in parallel.
Everybody good?
Questions?
Great.
So now let's do something a little more parallel.
Let's make both loops be parallel.
So here we have a cilk_for loop here, and then another cilk_for loop on the interior.
And let's see what we get here.
So what's the work for this?
n squared
Yeah, n squared.
That's not going to change.
That's not going to change.
What's the span?
log n
Yeah, log n. So it's log n because the span of the outer control loop is going to add log n. The max span of the inner control loop, well, it's going from log of 1 up to log of i, but the maximums of those is going to be proportional to log n because it's not regular.
And the span of the body now is going to be order 1.
And so we add the logs because those things are in series.
We don't multiply them.
What we're doing is we're looking at, what's the worst case?
The worst case is I have to do the control for this, plus the control for this, plus the worst iteration here, which in this case is just order one.
So the total is order log n. That can be confusing for people, why it is that we add here rather than multiply or do something else.
So let me pause here for some questions if people have questions.
Everybody with us?
Anybody want clarification or make a point that would lead to clarification?
Yes, question
If you were going to draw a tree like the previous slide, what would it look like?
Let's see.
I had wanted to do that and it got out of control.
So what it would look like is if we go back to the previous slide, it basically would look like this, except where each one of these guys is replaced by a tree that looks like this with as many leaves as the number here indicates.
So once again, this would be the one with the longest span because this would be log of the largest number.
But basically, each one of these would be a tree that came from this.
Is that clear?
That's a great question.
Anybody else have as illuminating questions as those?
Everybody understand that explanation, what the tree would look like?
OK, good.
Get So the parallelism here is n squared over log n. Now it's tempting, when you do parallel programming, to say therefore, this is better parallel code.
And the reason is, well, it does asymptotically have more parallelism.
But generally when you're programming, you're not trying to get the most parallelism.
What you're trying to do is get sufficient parallelism.
So if n is sufficiently large, it's going to be way more-- if n is a million-- which is typical problem size for a loop, for example, for a big loop, or even if it's a few thousand or whatever-- it may be just fine to have parallelism on the order of 1,000, which is what the first one gives you.
So 1,000 iterations is generally a small number of iterations.
So 1,000 by 1,000 matrix is going to generate parallelism of 1,000.
Here, we're going to get a parallelism of 1 million divided by about 20, log of 10 or 20, so like 100,000.
But if I have 1,000 by 1,000 matrix, the difference between having parallelism of 1,000 and parallelism of 100,000, when I'm running on 100 cores, let's say, it doesn't matter.
Up to 100 cores, it doesn't matter.
And in fact, running this on 100 cores, that's really a tiny problem compared to the amount of memory you're going to get.
1,000 by 1,000 matrix is tiny when it comes to the size of memory that you're going to have access to and so forth.
So for big problems and so forth you really want to look at this and say, of things that have ample parallelism, which ones really are going to give me the best bang for the buck for reasonable machine sizes?
That's different from things like work or serial running time.
Usually less running time is better, and it's always better.
But here parallelism-- yes, it's good to minimize your span, but you don't have to minimize it extremely.
You just have to get it small enough, whereas the work term, that you really want to minimize, because that's what you're going to have to do, even in a serial implementation.
Question
So are you suggesting that the other code was OK?
We're going to look a little bit closer at the issue of overheads.
We're now going to take a look at what's the difference between these two codes?
We'll come back to that in a minute.
The way I want to do it is take a look at the issue of overheads with a simpler example, where we can see what's really going on.
So here, what I've done is I've got a loop that is basically just doing vector addition.
It's adding for i equals 0 to n minus 1, add b of i into a of i.
Pretty simple code, and we want to make that be a parallel loop.
So I get a recursion tree that looks like this, where I have constant work at every step there.
And of course, the work is order n, because I've got n leaves.
And the number of internal nodes, the control, is all constant size strands there.
So this is all just order n for work.
And the span is basically log n, as we've seen, by going down one of these paths, for example.
And so the parallelism for this is order n over log n. So a very simple problem.
But now let's take a look more closely at the overheads here.
So the problem is that this work term contains substantial overhead.
In other words, if I really was doing that, if I hadn't coarsened the recursion at all in the implementation of cilk_for, if the developers hadn't done that, then I've got a function call, I've got n function calls here for doing a single addition of values at the leaves.
I've got n minus one of these guys, that's approximately n, and I've got n of these guys.
And which are bigger, these guys or these guys?
These guys are way bigger.
They've got a function call in there.
This guy right here just has what?
One floating point addition.
And so if I really was doing my divide and conquer down to a single element, this would be way slower on one processor than if I just ran it with a for loop.
Because if I do a for loop, it's just going to go through, and the overhead it has is incrementing i and testing for termination.
That's it.
And of course, that's a predictable branch, because it almost never terminates until it actually terminates, and so that's exactly the sort of thing that's going to have a really, really tight loop with very few instructions.
But in the parallel implementation, there's going to be this function call overhead everywhere.
And so therefore, this cilk_for loop in principle would not be as efficient.
It actually is, but we're going to explain why it is, what goes on in the runtime system, to understand that.
So here's the idea, and you can control this with a pragma.
So a pragma is a statement to the compiler that gives it a hint.
And here, the pragma says, you can name a grain size and give it a value of g. And what that says is rather than just doing one element when you get down to the bottom here, do g elements in a for loop when you get down to the bottom.
And that way, you halt the recursion earlier.
You have fewer of these internal nodes.
And if you make the grain size sufficiently large, the cost of the recursion at the top you won't be able to see.
So let's analyze what happens when we do this.
So we can understand this vis a vis this equation.
So the idea here is, if I look at my work, imagine that t iter is the time for iteration of one iteration of the loop, basic time for one iteration of the loop.
So the amount of work that I have to do is n times the time for the iterations of the loop.
And then depending upon my grain size, I've got to do things having to do with the internal nodes, and there's basically going to be n over g of those, times the time for a spawn, which is I'm saying is the time to execute one of these things.
So if these are batched into groups of g, then there are n over g such leaves.
There's a minus 1 in here, but it doesn't matter.
It's basically n over g times the time for the internal nodes.
So everybody see where I'm getting this?
So I'm trying to account for the constants in the implementation.
People follow where I'm getting this?
Ask questions.
I see a couple of people who are sort of going, not sure I understand.
Yes?
The constants [INAUDIBLE]
Yes.
So basically, the constants are these t iter and t spawn.
So t spawn is the time to execute all that mess.
t iter is the time to execute one iteration within here.
I'm doing, in this case, g of them.
So I have n over g leaves, but each one is doing g, so it's n over g times g, which is a total of n iterations, which makes sense.
I should be doing n iterations if I'm adding two vectors here.
So that's accounting for all the work in these guys.
Then in addition, I've got all of the work for all the spawning, which is n over g times t spawn.
And as I say, you can play with this yourself, play with grain size yourself, by just sticking in different grain size directives.
Otherwise it turns out that the cilk runtime system will pick what it deems to be a good grain size.
And it usually does a good job, except sometimes.
And that's why there's a parameter here.
So if there's a parameter there, you can get rid of that.
Yes?
Is the pragma something that is enforced, or is it something that says, hey--
It's a hint
It's a hint
Yes, it's a hint.
In other words, compiler could ignore it.
[?
The compiler is ?]
going to be like, oh, that's the total [INAUDIBLE] constant
It's supposed to be something that gives a hint for performance reasons but does not affect the correctness of the program.
So the program is going to be doing the same thing regardless.
The question is, here's a hint to the compiler and the runtime system.
And so then it's picked at-- yeah?
My question is, so there's these cases where you say that the runtime system fails to find an appropriate value for that [INAUDIBLE].
I mean, basically, chooses one that's not as good.
If you put a pragma on it, will the runtime system choose the one that you give it, or still choose--
No, if you give it, the runtime system will-- in the current implementation, it always picks whatever you say is here.
And that can be an expression.
You can evaluate something there.
It's not just a constant.
It could be maximum of this and that times whatever, et cetera.
Is that good?
So this is a description of the work.
Now let's get a description with the constants again of the span.
So what is going to be the constants for the span?
Well, I'm executing this part in here now serially.
So for the span part, we're basically going to go down on one of these paths and back up I'm not sure which one, but they're basically all fairly symmetric.
But then when I get to the leaf, I'm executing the leaf serially.
So I'm going to have whatever the cost is, g times the time per iteration, is going to be executed serially, plus now log of n over g-- n over g is the number of things I have here-- times the cost of the spawn, basically.
Does that make sense?
So the idea is, what do we want to have here if I want a good parallel code?
We would like the work to be as small as possible.
How do I make the work small?
How can I set g to make the work small?
[INAUDIBLE]
Make g--
Square root of n
Well, make g big or little?
If you want this term to be small, you want g to be big.
But we also want to have a lot of parallelism.
So I want this term to be what?
Small, which means I need to make g what?
Well, we got an n over g here, but it's in a log.
It's minus log.
So really, to get this small, I want g to be small.
So I have tension, trade off.
I have trade off.
So let's analyze this a little bit.
Essentially, if I look at this, I want g to be bigger-- from this one I want g to be small.
But here, what I would like is to make it so that this term dominates this term.
If the first term here dominates the second term, then the work is going to be the same as if I did an ordinary for loop to within a few percent.
So therefore, I want t span over t iter, if I take the ratio of these things, I want g to be bigger than the time to spawn divided by the time to iterate.
If I get it much bigger than that, then this term will be much bigger than that term and I don't have to worry about this term.
So I want it to be much bigger, but I want to be as small as possible.
There's no point in making it much bigger than that which causes this term to essentially be wiped out.
People follow that?
So basically, the idea is we pick a grain size that's large but not too large, is what you generally want to do, so that you have enough parallelism, but you don't.
The way that the runtime system does it is it has a somewhat complicated heuristic, but it actually looks to see how many processors you're running on.
And it uses a heuristic that says, let's make sure there's at least parallelism 10 times more than the number of processors.
But there's no point in having more iterations than like 500 or something, because at 500 iterations, you can't see the spawn overhead regardless.
So basically, it uses a formula kind of that nature to pick this automatically.
But you're free to pick this yourself.
But you can see the point is that although it's doing divide and conquer, you do this issue of coarsening and you do want to make sure that you have enough work to do in any of the leaves of the computation.
And as I say, usually it'll guess right.
But if you have trouble with that, you have a parameter you can play with.
Let's take a look at another implementation just to try to understand this issue.
Suppose I'm going to do a vector add.
So here I have a vector add of two arrays, where I'm basically saying ai gets the value of b added into it.
That's kind of the code we had before.
But now, what I want to do is I'm going to implement a vector add using cilk spawn.
So rather than using a cilk_for loop, I'm going to parallelize this loop by hand using cilk spawn.
What I'm going to do is I'm going to say for j equals 0 up to-- and I'm going to jump by whatever my grain size is here-- and spawn off the addition of things of size, essentially, g, unless I get close to the end of the array.
But basically, I'm always spawning off the next g iterations to do that in parallel.
And then I sync all these spawns.
So everybody understand the code?
I see nods.
I want to see everybody nod, actually, when I do this.
Otherwise what happens is I see three people nod, and I assume that people are nodding.
Because if you don't do it, you can shake your head, and I promise none of your friends will see that you're shaking your head.
And since the TAs are doing the grading and they're facing this way, they won't see either.
And so it's perfectly safe to let me know, and that way I can make sure you understand.
So everybody understand what this does?
OK, so I see a few more.
No.
OK, question?
Do you have a question, or should I just explain again?
So this is basically doing a vector add of b into a, of n iterations here.
And we're going to call it here, when I do a vector add, of basically g iterations.
So what it's doing is it's going to take my array of size n, bust it into chunks of size g, and spawn off the first one, spawn off the second one, spawn off the third one, each one to do g iterations.
That make sense?
We'll see it.
So here's sort of the instruction stream for the code here.
So basically, it says here is one, we spawn off something of size g, then we go on, we spawn off something else of size g, et cetera.
We keep going up there until we hit the cilk sync.
That make sense?
Each of these is doing a vector add of size g using this serial routine.
So let's analyze this to understand the efficiency of this type of looping structure.
So let's assume for this analysis that g equals 1, to make it easy, so we don't have to worry about it.
So we're simply spawning off one thing here, one thing here, one iteration here, all the way to the end.
So what is the work for this, if I spawn off things of size one, asymptotic work?
It's order n, because I've got n leaves, and I've got n guys that I'm spawning off.
So it's order n. What's the span?
[INAUDIBLE]
Yeah, it's also order n, because the critical path goes something like brrrup, brrrup, brrrup.
That's order n length.
It's not this, because that's only order one length, all those.
The longest path is order n. So that says the parallelism is order one.
Conclusion, at least with grain size one, this is a really bad way to implement a parallel loop.
However, I guarantee, it may not be the people in this room, but some fraction of students in this class will write this rather than doing a cilk for.
Bad idea.
Bad idea.
Generally, bad idea.
Question?
Do you think you could find a constant factor, not just [INAUDIBLE]?
Well here, actually, with grain size one, this is really bad, because I've got this overhead of doing a spawn, and then I'm only doing one iteration.
So the ideal thing would be that I really am only paying for the leaves, and the internal nodes, I don't have to pay anything for.
Yeah, Eric?
Shouldn't there be a sort of keyword in the b add too?
In the where?
In the b add?
No, that's serial.
That's a serial code
No, but if you were going to call it with cilk spawn, don't you have to declare it cilk?
Is that not the case?
No
Never mind
Yes, question
If g is [INAUDIBLE], isn't that good enough?
Yeah, so let's take a look.
That's actually the next slide.
This is basically, this we call puny parallelism.
We don't like puny parallelism.
It doesn't have to be spectacular.
It has to be good enough.
And this is not good enough for most applications.
So here's another implementation.
Here's another way of doing it.
Now let's analyze it where we have control over g. So we'll analyze it in terms of g, and then see whether there's a setting for which this make sense.
So if I analyze it in terms of g, now I have to do a little bit more careful analysis here.
How much work do I have here in terms of n and g?
It's the same
Yeah, the work is still asymptotically order n. Because I always have n work in the leaves, even if I do more iterations here.
What increasing g does is it shrinks this, right?
It shrinks this.
The span for this is what?
So I heard somebody say it.
n over g plus g. And it corresponds to this path.
So this is the n over g part up here, and this is the plus g. So what we want to do to minimize this, is we can minimize this.
This has the smallest value when these two terms are equal, which you can either know as a basic fact of the summation of these kinds of things, or you could take derivatives and so forth.
Or you can just eyeball it and say, gee, if g is bigger than square root of n, then this is going to be the dominant, and if g is smaller than square root of n, then this is going to be dominant.
And so when they're equal, that sounds like about when it should be the smallest, which is true.
So we pick it to be about square root of n, to minimize the span.
Since g, I don't have anything to minimize here.
So pick it around square root of n, then the span is around square root of n. And so then the parallelism is order square root of n. So that's pretty good.
So that's not bad.
So for something that's a big array, array of size 1 million, parallelism might be 1,000.
That might be just hunky dory.
Question.
What's that?
I don't see where--
We've picked g to be equal to square root of n
[INAUDIBLE] plus n over g, plus g. I don't see where [INAUDIBLE]
You don't see where this g came from?
This g comes from, because I'm doing g iterations here.
So remember that these are now of size g. I'm doing g iterations in each leaf here, if I set g to be large.
So I'm doing n over g pieces here, plus g iterations in my [INAUDIBLE].
Is that clear?
So the n over g is this part.
This size here, this is not one.
This has g iterations in it.
So the total span is g plus n over g. Any other questions about this?
So basically, I get order square root of n. And so this is not necessarily a bad way of doing it, but the cilk for is a far more reliable way of making sure that you get the parallelism than spawning things off one by one.
One of the things, by the way, in this, I've seen people write code where their first instinct is to write something like this, where this that they're spawning off is only constant time.
And they say, gee, I spawned off n things.
That's really parallel.
When in fact, their parallelism is order one.
So it's really seductive to think that you can get parallelism by this, [?
right.
?]
It's much better to do divide and conquer, and cilk for does that for you automatically.
If you're going to do it by hand, sometimes you do want to do it by hand, then you probably want to think more about divide and conquer to generate parallelism, because you'll have a small span, than doing many things one at a time.
So here's some tips for performance.
So you want to minimize the span, so the parallelism is the work over the span.
So you want to minimize the span to maximize parallelism.
And in general, you should try to generate something like 10 times more parallelism than processors, if you want to get near perfect linear speed-up.
In other words, a parallel slackness of 10 or better is usually adequate.
If you can get more, you're now talking that you can get more performance, but now you're getting performance increases in the range of 5% or so, 5% to 10%, something like that.
Second thing is if you have plenty of parallelism, try to trade some of it off to reduce work overhead.
So this is a general case.
This is what actually goes on underneath in the cilk++ runtime system, is they are trying to do this themselves.
But you in your own code can play exactly the same game.
Whenever you have a problem and it says, whoa, look at all this parallelism, think about ways that you could reduce the parallelism and get something back in the efficiency of the work term, because the performance in the end is going to be something like t1 over p plus t infinity.
If t infinity is small, it's like t1 over p, and so anything you save in the t1 term is saving you overall.
It's going to be a savings for you overall.
Use divide and conquer recursion on parallel loops rather than sprawling one small thing after another.
In other words, do this not this, generally.
And here's some more tips.
Another thing that can happen that we looked at here was make sure that the amount of work you're doing is reasonably large compared to the number of spawns.
You could also say this is true when you do recursion for function calls.
Make sure if you're just in serial programming, you always want to make sure that the amount of work you're doing is small compared to the number of function calls are doing if you can, and that'll make things go faster.
So same thing here, you want to have a lot of work compared to the total number of spawns that you're doing in your program.
So spawns, by the way, in this system, are about three or four times the cost of a function call.
They're sort of the same order of magnitude as a function call, a little bit heavier than a function call.
So you can spawn pretty readily, as long as the total number of spawns you're doing isn't dominating your work.
Generally parallelize outer loops as opposed to inner loops if you're forced to make a choice.
So it's always better to have an outer loop that runs in parallel rather than an inner loop that runs in parallel, because when you do an inner loop that runs in parallel, you've got a lot of overhead to overcome.
But in an outer loop, you've got all of the inner loop to amortize against the cost of the spawns that are being used to parallelize the outer loop.
So you'll do many fewer spawns in the implementation.
Watch out for scheduling overheads.
So if you do something like this-- so here we're paralyzing an inner loop rather than an outer loop.
Now this turns out, it doesn't matter which order we're going in or whatever.
It's generally not desirable to do this because I'm paying scheduling overhead n times through this loop, whereas here, I pay for scheduling overhead just twice.
So is generally better, if I have n pieces of work to do, rather than, in this case, parallelizing-- let me slow down here.
So let's look at what this code does.
This says, go for two iterations.
Do something for which it is going to take n iterations for j.
So two iterations for i, n iterations for j.
If you look at the parallelism of this, what is the parallelism of this assuming that f is constant time?
What's the parallelism of this code?
Two.
The parallelism of two, because I've got two things on the outer loop here, and then each is n. So my span is essentially n. My work is like 2n, something like that.
So it's got a parallelism of that, too.
What's the parallelism of this code?
What's the parallelism?
It's not n, because I'm basically going through this serially, the outer loop serially.
What's the theoretical parallelism of this?
So for each iteration here, the parallelism is two.
No, not n. It can't be n, because I'm basically only parallelizing two things, and I'm doing them serially.
The outer loop is going serially through the code and it's spawning off two things, two things, two things, two things, two things.
And waiting for them to be done, two things, wait for it to be done, two things, wait for it to be done.
So the parallelism is two.
These have the same parallelism.
However if you run this, this one will give you a speedup of two on two cores, very close to it.
Because there's the scheduling overhead here, you've only paid once for the scheduling overhead, and then you're doing a whole bunch of stuff.
So remember, to schedule it, it's got to be migrated, it's got to be moved to another processor, et cetera.
This one, it's not even worth it probably to steal each of these individual things.
You're spawning off things that are so small, this may even have parallelism that's less than 1 in practice.
And if you look at the cilkview tool, this will show you a high burden parallelism.
Because the cilkview tool, the burden parallelism tells you what the overhead is from scheduling, as well as what the actual parallelism is.
And it recognizes that oh, gee whiz.
This thing really has very small-- there's almost no work in here.
So you're trying to parallelize something where the work is so small, it's not even worth migrating it to take advantage of it.
So those are some tips.
Now let's go through and analyze a bunch of algorithms reasonably quickly.
We'll start with matrix multiplication.
People seen this problem before?
Here's the matrix multiplication problem.
And let's assume for simplicity that n is a power of 2.
So basically, let's start out with just our looping version.
In fact, this isn't even a very good looping version, because I've got the order of the loops wrong, I think.
But it is just illustrative.
Basically let's parallelize the outer two loops.
I can't parallelize the inner loop.
Why not?
What happens if I tried to parallelize the inner loop with a cilk_for in this implementation?
Why can't I just put a cilk_for there?
Yes, somebody said it
It does that in cij
Yeah, we get a race condition.
We have more than two things in parallel trying to update the same cij, and we'll have a race condition.
So always run cilkview to tell your performance.
But always, always, run cilk screen to tell whether or not you've got races in your code.
So yeah, you'll have a race condition if you try to naively parallelize the loop here.
So the work of this is what?
It's order n cubed, just three nested loops each going to n. What's the span of this?
What's the span of this?
It's order n, because it's log n for this loop, log n for this loop, plus the maximum of this, well, that's n. Log n plus log n plus n is order n. So order n span, which says parallelism is order n squared.
So for 1,000 by 1,000 matrices, the parallelism is on the order of a million.
Wow.
That's great.
However, it's on the order of a million, but as we know, this doesn't use cache very effectively.
So one of the nice things about doing divide and conquer is, as you know, that's a really good way to take advantage of caching.
And this works in parallel, too.
In particular because whenever you have sufficient parallelism, these processors are executing the code just as if they were executing serial code.
So you get all the same cache locality you would get in the serial code in the parallel code, except for the times that you're actually migrating work.
And if you have sufficient parallelism, that isn't too often.
So let's take a look at recursive divide and conquer multiplication.
So we're familiar with this, too.
So this is eight multiplications of n over 2 by 2 matrices, and one addition of n by n matrix.
So here's a code using a little bit of C++ism.
So I've made the type a variable t. So we're going to do matrix multiplication of an array, a, the result is going to go in c, and we're going to basically have a and b, and we're going to add the result into c. We have n, which is the side of the submatrix that we're working on, and we're also going to have an n size, which is the length of the row in the original matrix.
So remember when we do matrix things, if I take a submatrix, it's not contiguous in memory.
So I have to know the row size of the matrix that I'm in in order to be able to calculate what the elements are.
So the way it's going to work is I'm going to assign this temporary d, by using the new-- which is basically memory allocation C++-- array of size n by n. And what we're going to do is then do four of the recursive multiplications, these guys here, into the elements of c, and then four of them also into d using the temporary.
And then we're going to sync, after we get all that parallel work done, and then we're going to add d into c, and then we'll delete d, because we allocated it up here.
Everybody understand the code?
So we're doing this, it's just we're going to do it in parallel.
Good?
Questions?
OK.
So this is the row length of the matrices so that I can do the base cases, and in particular, partition the matrices effectively.
I haven't shown that code.
And of course, the base case, normally, we would want to coarsen for efficiency.
I would want to go down to something like maybe a eight by eight or 16 by 16 matrix, and at that point switch to something that's going to use the processor pipeline better.
The base cases, once again, I want to emphasize this because a couple people on the quiz misunderstood this.
The reason you coarsen has nothing to do with caches.
The reason you coarsen is to overcome the overhead of the function calls, and the coarsening is generally chosen independent of what the size of the caches are.
It's not a parameter that has to be tuned to cache size.
It's a parameter that has to be tuned to function call, versus ALU instructions, and what that balance is.
Question?
I mean, I understand that's true, but I thought-- I mean, maybe I [?
heard the call ?]
wrong, but I thought we wanted, in general, in terms of caching, that you would choose it somehow so that all of the data that you have would somehow fit--
That's what the divide and conquer does automatically.
The divide and conquer keeps halving it until it fits in whatever size cache you have.
And in fact, we have three caches on the machines we're using
Yeah, but I'm saying if your coarsened constant is too big, that's not going to happen
If the coarsened constant is too big, that's not going to happen.
But generally, the caches are much bigger than what you need to do to amortize the cost.
But you're right, that is an assumption.
The caches are generally much bigger than the size that you need in order to overcome function call overhead.
Function call overhead is not that high.
OK?
Good.
I'm glad I raised that issue again.
And so we're going to determine the submatrices by index calculation.
And then we have to implement this parallel add, and that I'm going to do just with a doubly nested for loop to add the things.
There's no cache behavior I can really take advantage of here except for spatial locality.
There's no temporal locality because I'm just adding two matrices once, so there's no real temporal locality that I'll get out of it.
And here, I've actually done the index calculations by hand.
So let's analyze this.
So to analyze the multiplication program, I have to start by analyzing the addition program.
So this should be, I think, fairly straightforward.
What's the work for adding two n by n matrices here?
n squared, good, just doubly nested loop.
What's the span?
[INAUDIBLE]
Yeah, here it's log n, very good.
Because I've got log n plus log n plus order one.
I'm not going to analyze the parallelism, because I really don't care about the parallelism of the addition.
I really care about the parallelism of the matrix multiplication.
But we'll plug those values in now.
What is the work of the matrix multiplication?
Well for this, what we want to do is get a recurrence that we can then solve.
So what's the recurrence that we want to get for m of 1 of n?
Yeah, it's going to be 8m sub 1 of n over 2, that corresponds to these things, plus some constant stuff, plus the work of the addition.
Does that make sense?
We analyze the work of the addition.
What's the work of the addition?
Order n squared.
So that's going to dominate that constant there, so we get 8.
And what's the solution to this?
Back to Master Theorem.
Now we're going to start pulling out the Master Theorem multiple times per slide for the rest of the lecture.
n cubed, because we have log base 2 of 8.
That's n cubed compared with n squared.
So we get a solution which is n cubed-- Case 3 of the Master Theorem.
So that's good.
The work we're doing is the same asymptotic work we're doing for the triply nested loop.
Now let's take a look at the span.
So what's the span for this?
So once again, we want a recurrence.
What's the recurrence look like?
So the span of this is going to be the span of-- it's going to be the sum of some things.
But the key observation is that it's going to be-- we want the maximum of these guys.
So we're going to basically have the allocation as constant time, we have the maximum of these, which is m of infinity of n over 2, and then we have the span of the add.
So we get this recurrence.
m infinity sub n over 2, because we have only to worry about the worst of these guys.
The worst of them is-- they're all symmetric, so it's basically the same.
We have a of n, and then there's a constant amount of other overhead here.
Any questions about where I pulled that out of, why that's the recurrence?
So this is the addition, the span of the addition of this guy that we analyzed already.
What is the span of the addition?
What did we decide that was?
log n. So basically, that dominates the order one.
So we get this term, and what's the solution of this recurrence?
[INAUDIBLE]
What case is this?
[INAUDIBLE] log n squared
Yes, it's log squared n. So basically, it's case two.
So if I do n to the log base b of a, that's n to the log base 2 of 1, that's just 1.
And so this is basically a logarithmic factor times the 1, so we add an extra log.
We get log squared n. That's just Master Theorem plugging in.
So here, the span is order log squared n. And so we have the work of n cubed, the span of log squared n, so the parallelism is the ratio, which is n cubed over log squared n. Not too bad for a 1,000 by 1,000 matrices, the parallelism is about 10 million.
Plenty of parallelism.
So let's use the fact that we have plenty of parallelism to say, let's get rid of some of that parallelism and put it back into making our code more efficient.
So in particular, this code uses an extra temporary d, which it allocates here and it deletes here.
And generally, there's a good rule that says, if you use more storage you're going to use more time, because you're going to have to look at that storage, it's going to take up space in your cache, and it's generally going to make you slower.
So things that use less storage are generally faster.
Not always the case, sometimes there's a trade off.
But often it's the case, use more storage, it runs slower.
So let's get rid of this guy.
How do we get rid of this guy?
Yeah?
[INAUDIBLE PHRASE]
You're going to do this serially, you're saying?
Yeah, you do those serially in add
If you do this serially in add, it turns out if you do that, you're going to be in trouble because you're going to not have very much parallelism, unfortunately.
Actually, analyzing exactly what the parallelism is there is actually pretty good.
It's a good puzzle.
Maybe we'll do that on the quiz, the take home problem set we're calling it now, right?
We're going to have a take home problem set, maybe that's a good one.
Yeah, so the idea is, you can sync.
And in particular, why not compute these, then sync, and then compute these, adding their results into the places where we added these in?
So it's making the program more serial, because I'm putting in a sync.
That shouldn't have an impact on the work, but it will have an impact on the span.
So we're going to trade it off, and the way we'll do that is by putting essentially a sync in the middle.
And since they're adding it in, I don't even have we call the addition routine, because it's just going to add it in in place.
So I spawn off these four guys, putting their results into c, then I spawn off these four guys, and they add their results into c. Is that clear what the code is?
So let's analyze this.
So the work for this is order n cubed.
It's the same as anything else, we can come up with a recurrence, slightly different from before because I only have an order one there, but it doesn't really matter.
The answer is order n cubed.
The span, now this gets a little trickier.
What's the recurrence of the span?
[INAUDIBLE]
What is that?
Twice the span of m of n over 2
Twice the span of m of n over 2, that's right.
So basically, we have the maximum of these guys, the maximum of these guys, and then this is making those things be in series.
So things that are in parallel I take the max, if it's in series, I have to add them.
So I end up with 2m infinity of n over 2 plus order one.
Does that make sense?
OK, good.
So let's solve that recurrence.
What's the answer to that one?
That's order in.
Which case is it?
I never know what the cases are.
I know two, but one and three, it's like-- they're the same thing, it's just which side it's in, so I never remember what the number is.
But anyway, case one, yes.
Case one.
It's the one where this thing is bigger, so that's order n. Good.
So then the work is n cubed, the span is order n, the parallelism is order n squared.
So for 1,000 by 1,000 matrices, I get parallelism on the order of a million, instead of before, I had parallelism on the order of 10 million.
So this turns out way better code than the previous one because it avoids the temporary and therefore runs, you get a constant factor improvement for that, and it's still, on 12 cores, it's going to run pretty fast.
And in practice, this is a much better way to do it.
The actual best code that I know for doing this essentially does divide and conquer in only one dimension at a time.
So basically, it looks to see what's the long dimension, and whatever the long dimension is, it slices it in half and then recurses, and just does that as a binary thing.
And it basically is the same work, et cetera.
It's a little bit more tricky to analyze.
Let me quick do merge sort.
So you know merge sort.
There's merging two sorted arrays, we saw this before.
If I spend all this time doing animations, I might as well get my mileage out of it.
There we go.
So you merge, that's basically what this code does.
Order n time to merge.
So here's merge sort.
So what I'll do in merge sort is the same thing I normally do, except that I'll make recursive routines go in parallel.
So when I do that, it basically divide and conquers down, and then it sort of does this to merge things together.
So we saw this before, except now, I've got the fact that I can sort two things in parallel rather than sorting them serially.
So let's take a look at the work.
What's the work of merge sort?
We know that.
n log n, right?
2t of n over 2 plus order n, so that's order n log n. The span is what?
What's the recurrence of the span?
So we're going to take the maximum of these two guys.
So we only have one term that involves t infinity, and then the merge costs us order n, so we get this recurrence.
So that says that the solution is order n. So therefore, the work is n log n, the span is order n, and so the parallelism is order log n. Puny.
Puny parallelism.
Log n is like, you can run it, and it'll work fine on a few cores, but it's not to be something that generally will scale and give you a lot of parallelism.
So it's pretty clear from this that the bottleneck-- where's all the span going to?
It's going to that merge.
So when you understand that that's the structure of it, now you say if you want to get parallelism, you've got to go after the merge.
So here's how we parallelize the merge.
So we're going to look at merging of two arrays that are of possibly different length.
So one we'll call A, and one we'll call B, with na and nb elements.
And let me assume without loss of generality that na is greater than or equal to nb, because otherwise I can just switch the roles of A and B.
So the way that I'm going to do it is I'm going to find the middle element of A.
These are sorted arrays that I'm going to merge.
I find the middle element of A, so these guys are or less than or equal to a of ma, and these are greater than or equal to.
And now I binary search and find out where that middle element would fall in the array B.
So that costs me log n time to binary search.
Remember binary search?
Then what I'm going to do is recursively merge these guys, because these are sorted and less than or equal to ma, recursively merge those and put this guy in the middle.
So when I do that, the key question when we analyze-- it turns out the work is going to basically be the same, but the key thing is going to be what happens to the span?
And the idea here is that the total number of elements in the larger of these two things is going to be at most what?
Another way of looking at it is in the smaller partition, if n is the total number of elements, the smaller partition has how many elements at least relative to n?
No matter where this binary search finds itself.
So the worst case is sort of going to come when this guy is like at one end or the other.
And then the point is that because A is the larger array, at least a quarter of the elements will still be in the smaller partition.
Of all the elements here, at least a quarter will be in the smaller partition, which will occur when B is equal to in size to A.
So the number, in the larger of the recursive merges, is at most 3/4 n. Sound good?
That's the main, key idea behind this.
So here's the parallel merge.
Basically you do binary search, you spawn, then, the two merges.
Here's one merge, and here's the other merge, and then you sync.
So that's the code for the doing the parallel merge.
And now you want to incorporate that parallel merge into the parallel merge sort.
Of course, you coarsen the base cases for efficiency.
So let's analyze the span of this.
So the span is basically then the span of something of 3/4, at most 3/4, the size plus the log n for the binary search.
So the span of parallel merge is therefore order log squared n, because the important thing is, I'm whacking off a constant fraction here every time.
So I get log squared n as the span, and the work I get this hairy recurrence, that it's t of alpha n plus t1 minus alpha n plus log n, where alpha falls in this range.
This does not satisfy the Master Theorem.
You can actually do this pretty easily with a recursion tree, but the way to verify is-- we call this technically a hairy recurrence.
That's the technical term for it.
So it turns out, this has order n, just like ordinary merge, order n time.
here's You can use the substitution method, and I won't drag you through it, but you can look at it in the notes.
And this should be very familiar to you as having all aced 6006, right?
Otherwise you wouldn't be here, right?
So the parallelism of the parallel merge is something like n over log squared n. So that's much better than having n order n bound.
And now, we can plug it into merge sort.
So the work is going to be the same as before, because I just have the work of the merge, which is still order n. So the work is order n log n, once again pulling out the Master Theorem.
And then the span is n over 2 plus log n, because basically, I have the span of a problem of half the size plus the span that I need to merge things.
That's order log squared n. This I want to pause on for moment.
People get this recurrence?
Because this is the span of the merge.
And so what I end up with is I get another log, log cubed n. And so the total parallelism is n over log squared n. And this is actually quite a practical thing to implement, to get the n over log squared n parallelism versus just a log n parallelism.
We're not going to do tableau construction.
You can read that up, that's on the notes that are online, but you should read through that part of it.
It's got some nice animations which you don't get to see.
This is like when you do longest common subsequence and stuff like that, how you would solve that type of problem in parallel.
OK, great.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Let's get going here.
So this is a lecture that's actually appropriate for Halloween, because it's a scary topic.
Non-deterministic programming.
So we've been looking mostly at deterministic programs.
So a program is deterministic on a given input if every memory location is updated with the same sequence of values in every execution.
So if you look at the memory of the machine, you can view that as, essentially, the state of the machine.
And if you're always updating every memory location with exactly the same sequence of values, then the program is deterministic.
Now it may be that two memory locations may be updated in a different order.
So you may have one location which is updated first in one execution, and another that's second, and then in a different execution, they may be a different order.
That's OK, generally.
The issue is whether or not every memory location sees the same order.
And if they do, then it's for every execution, then it's a deterministic program.
So what's the advantage of having a deterministic program?
Yeah?
It always runs the same way [INAUDIBLE]
It always runs the same way.
So what?
What's that good for?
So you can find bugs easier
Yeah, debugging.
It's really easy to find bugs if every time you run it it does the same thing.
It's much harder to find bugs if, when you run it, it might do something different.
So that leads to our first major rule of thumb about determinism, which is you should always write deterministic programs.
Don't write non-deterministic programs.
And the only problem is, boy is that poor quality there.
So basically, it says, always write non-deterministic programs unless you can't.
So sometimes, the only way to get performance is to do something non-deterministic.
So this lecture is basically about some of the ways of doing non-deterministic programming.
So it's appropriate that we say this is not for those who are faint of heart.
We are treading into dangerous territory here.
So the basic rule is, as I say, any time you can, make your program deterministic.
So we're going to talk about the number one way that people introduce non-determinism into programs, which is via mutual exclusion and mutexes, which are a type of lock, and then look at some of the anomalies that you get.
Besides just things being non-deterministic, you can also get some very, very weird behavior sometimes for the execution.
So we'll start out with mutual exclusion.
So let's take a look, for example, suppose I'm implementing a hash table as a set of bins.
And I'm resolving collisions with chaining.
So here, each slot of my hash table has a chain of all the values that resolve to that slot.
And if I have a value x, let's say it has key 81, and I want to insert x into the table, I first compute a hash of x.
And let's say it hashes to this particular list here.
And then what I do is I say, OK, let me insert x into the table.
So I make the next pointer of x point to whatever is the head of the table.
And then I make the table 0.2x.
And that effectively inserts x into the hash table.
Fairly straightforward piece of code.
I would expect that most of you could write that even on an exam and get it right.
But what happens when we say, oh, let's have some concurrency.
Let's have the ability to look up things in a hash table in different parallel branches of a parallel program.
So here, we have a concurrent hash table now where I've got two values, and I'm going to have two different threads inserting x and y.
So one of them is going to do this one, and one of them is going to do this one.
So let's just see how this can screw up.
So first, we hash x, and it hashes to this particular slot.
So then we do, just as we're doing before, making its next pointer point to the beginning of the array.
Then y gets in the picture, and it decides oh, I'm going to hash.
And oh, it hashes to exactly the same slot.
And then y makes its next pointer point to the same to the head of the list.
And then it sets the head of the list to point to y.
So now y is in the list.
Whoops, now x puts itself in the list, effectively taking y out of the list.
So rather than x and y both being in the list, we have a concurrency bug.
So this is clearly a race.
So it's a determinacy race, because we have two parallel instructions accessing essentially the same location, at least one of which-- in this case both of them-- performing a store to that location.
So that's a determinacy race.
And how things are going to work out depends upon which one of these guys goes first.
Notice, as with most race bugs, that if this code all executed before this code, we're OK. Or if this code all executed before this code, we're OK.
So the bug occurs when they happen to execute at essentially the same time and their instructions interleave.
So this is a race bug.
So one of the classic ways of fixing this kind of race bug is to insist on some kind of mutual exclusion.
So a critical section is a piece of code that is going to access shared data that must not be executed by two threads at the same time.
So it shouldn't be accessed by two threads at the same time.
So it's mutual exclusion.
So that's what a critical section is.
And we have a mechanism that operating systems typically provide-- as well as runtime systems, but you can build your own-- called "mutexes," or "mutex locks," or sometimes just "locks."
So a mutex is an object that has a lock and unlock member function.
And any attempt by a thread to lock an already locked mutex causes that thread to block.
And "block" is, by the way, a hugely overused word in computer science.
In this case, by "block," they mean "wait."
It waits until the mutex is unlocked.
So whenever you have something that's locked, somebody else comes and tries to grab the lock.
The mutex mechanism only allows one thread to access it.
The other one waits until the lock is freed.
Then this other one can go access it.
So what we can do is build a concurrent hash table by modifying each slot in the table to have both a mutex, L, and a pointer called "head" to the slot contents.
And then the idea is that what we'll do is hash the value to a slot.
But before we access the elements of the slot, we're going to grab the lock on the slot.
So every slot in the table has a lock here.
Now, I could have a lock on the whole table.
What's the problem with that?
Sure
[INAUDIBLE] basically can't do anything.
You can't read.
You couldn't be reading from the table
Yeah, so if you have a lock on the whole table--
You would defeat the purpose [INAUDIBLE]
You defeat the purpose of trying to have a concurrent hash table, right?
Because only one thread can actually access the hash table at a time.
So in this case, what we'll do is we'll lock each slot of the hash table.
And there are actually mechanisms where you can lock each element of the hash table or a constant number of elements.
But basically, what we're trying to do is make it so that the odds are that if you have a big enough table and relatively few processors you're running on, the odds that they'll conflict are going to be very low.
So what we do is we grab a lock on the slot, and then we play the same game of inserting ourselves at the head.
And then we unlock the slot.
So what that does is it means that only one of the two threads in the previous example can actually execute this code at a time.
And so it guarantees that the two regions of code will either execute in this order or in this order, and you'll never get the instructions interleaved.
Now, this is introducing non-determinism.
Why is this going to be non-deterministic?
Yes?
[INAUDIBLE] lock first, it'll be [INAUDIBLE]
Yeah, depending upon which one gets the lock first, the length list in there will have the elements in a different order.
So a program that depends on the order of that list is going to behave differently from run to run.
So let's recall the definition of a determinacy race.
It occurs when two logically parallel instructions access the same memory location, and at least one of the instructions performs a write.
So that is, we do have a determinacy race when we introduce locks.
Locks are essentially, we're going to have an intentional determinacy race.
So a program execution with no determinacy races means the program is deterministic on that input.
So if there are no determinacy races, then although individual locations may be updated in a different order in a parallel execution, every memory location will be updated by exactly the same thing at the same time.
The order will be of update of operations on any given location will be the same always.
So that's actually a theorem, which we're not going to prove.
But I think if you think about it, it's fairly straightforward.
If you never have two guys in parallel that could possibly affect the same location, then the behavior always is going to be the same thing.
Things are going to get written in the same order.
So the program in that case always behaves the same on that given input, no matter how it's scheduled and executed.
We'll always have essentially the same behavior, even though it may get scheduled one way or another.
And one of the nice things that we have in our race detection tool Cilkscreen is that if we do have determinacy races that exist in an ostensibly deterministic program-- that is, a program with no mutexes.
If basically it just reads and writes on locations and so forth, then Cilkscreen guarantees to find such a race.
So It's nice that we get a guarantee out of Cilkscreen.
So this is all beautiful, elegant, everything works out great if there are no determinacy races.
But when we do something like a concurrent hash table, we're intentionally putting in a determinacy area.
So that asks sort of a natural question.
Why would I want to have a concurrent hash table?
Why not make it so that my program is deterministic?
Why might a concurrent hash table be an advantageous thing to have in a program that you wanted to go fast?
Some ideas?
Where might you want to use it?
Yeah?
Speed?
Yeah, speed.
But I mean, what's an application?
What's a use case, as the entrepreneurs would ask you?
Where is it that you would really want to use a concurrent hash table to give you speed?
Yeah?
If you started using it [INAUDIBLE] along with your system [INAUDIBLE] values
Yeah, it could be that there's some sort of global table that you want a lot of people to be able to access at one time.
So if you lock down and only had one thread accessing at a time, you reduce how much concurrency that you could have.
That's a good one.
Yeah?
Perhaps most of the time, people are just reading.
So if you had something concurrent, your reading should be fine.
So in that case, a lot more reading high performance [INAUDIBLE]
Yeah, so in fact, there's a type of lock called a reader-writer lock, which allows one writer to operate, but many readers.
So that's another type of concurrency control.
So just another place, a common place that you use it, is when you're memoizing.
Meaning I do a computation, I want to remember the results so that if I see it again, I can look it up rather than having to compute it again from scratch.
So you might keep all those values in a hash table.
Well, if I go in the hash table, now I'm going to have concurrent accesses to that hash table if I've got a parallel program that wants to do memorizing.
And there are a bunch of other cases.
So we have determinacy races.
And we have a great guarantee that if there is a race, we guarantee to find it.
Now, there's another type of race, and in fact, you'll hear more about this type of race if you read the literature than you hear about determinacy races.
So a data race occurs when you have two logically parallel instructions holding no locks in common.
And they access the same location, and at least one of the instructions performs a write.
So this is saying that I've got accesses.
And if they have no locks in common-- so it could be that you have a problem where one of them holds a lock L, and another one holds L prime.
And then they access the location, that's going to be a data race, because they don't hold locks in common.
But if I have L and L being the locks that the two threads hold, and they access the same location, that's not a data race now.
It is a determinacy race, because it's going to matter which order it is, but it's not a data race, because the locks, in some sense, are protecting access.
So Cilkscreen, in fact, understands locks and will not report a determinacy race unless it is also a data race.
However, since codes that use locks are non-deterministic by intention, they actually weaken Cilkscreen's guarantee.
And in particular, in its execution that it does, if it finds a data race, it's going to say, I'm going to ignore that data race.
But now it is only going to follow one of the two paths that might arise from that data race.
In other words, it doesn't follow both paths.
If you could think about it, when one of them wins-- so you have a race between two critical sections.
When one of them wins, you can imagine that's one possible outcome of the computation.
When the other wins, it's another path.
And what Cilkscreen does is it picks one path.
In fact, it picks the path which is the one that would occur in the cereal execution.
So there's a whole path there that you're not exploring.
So Cilkscreen's guarantee is not going to be strong there.
However, if the critical sections, in fact, commute-- that is, they do exactly the same thing, no matter what the order.
So for example, if they're both incrementing a value, then the result, after doing one versus after the other is the same value, then you get a guarantee out of Cilkscreen.
So Cilkscreen could still be very helpful for finding bugs, because typically, when you organize your computation, if it occurs in this order, there's typically some execution or input where you can make things occur in the other order.
So you can actually cover more races than you might imagine on first blush.
But it is a danger.
But what we're talking about today is dangerous programming, non-deterministic programming.
So when you start using mutexes, some of the guarantees and so forth get much dicier.
Any questions about that?
Now, if you have no data races in your code, that doesn't mean that you have no bugs.
So for example, here's a way somebody might fix that insertion code.
So we hash the key, we grab a lock, we set x next to be whatever is the head of the list, and then we do an unlock.
And now we lock it again.
Now we follow the head to set x-- sorry, we set x to be the head of the list and then unlock again.
And now notice that in this case, technically, there is no data race if I have two concurrent threads trying to access these at a time, because all the axis I'm doing, I'm holding lock L. Nevertheless, I can get that same interleaving of code that causes the bug.
So just because you don't have a data race doesn't mean that you don't have a bug in your code.
As I say, this is dangerous programming.
However, typically, if you have mutexes and no data races, usually it means that you went through and thought about this code.
And if you were thinking about this code, you would say, gee, really I'm trying to make these two instructions be the critical section.
Why would I unlock and lock again?
So most of the time, as a practical matter, if you don't have data races, it probably means you did the right thing in terms of identifying the critical sections that needed to be locked and not unlocking things in the middle of them.
So as a practical matter, no data races usually means it's unlikely you have bugs.
But no guarantees.
As I say, dangerous programming.
Non-deterministic programming is dangerous program.
Any questions about that?
Anybody scared off yet?
Yeah?
So what you can do is the opposite.
You don't have any bugs, but you made the critical distinction to [INAUDIBLE]
Yes, so certainly from a performance point of view, one of the problems with locking is that-- and we'll talk about this a little bit later-- with locking is that if you have a large section that you decide to lock, it means other threads can't do work on that section.
So they're spinning, wasting cycles.
So generally, you want to try to lock things as small as possible.
The other problem is, it turns out that there's overhead associated with these locks.
So if there's overhead associated with the locks, that's problematic as well, because now you may be slowing down.
If this is in an inner loop, notice that we've now, even if I just have the lock and unlock without these two spurious ones here, we may be more than doubling the overhead.
In fact, locking instructions tend to be much more expensive than register operations.
They usually cost something on the order of going to L2 cache as a minimum.
So it's not even L1 cache.
It's like going out to L2 cache.
Now, it turns out there are some times where you have data races.
So we say if there are no data races, then you have no guarantee there's no bugs.
If there are data races, your program still may be correct.
Here's an example of a code where you might want to allow a benign data race.
So here we have, let's say, an array A that has these elements in it.
And we want to find, what is the set of digits in the array?
So these are all going to be values between 0 and 9.
And I want to know which ones are present of the digits 0 to 9.
Which ones are not present?
So I can write a little code for that.
Let me initialize an array called "digits" to have all-zero entries.
And now let me go through all the elements of A and set digits of whatever the digit is to be 1.
So set at 1 if that digit is present.
And I can do that in parallel, even.
So what can happen here is I can have, if I've done this in parallel, this particular update of digits of 6 will be set to 1 when this one is being sent to 1.
Is that a problem?
In some sense, no.
They're both being set to 1. Who cares?
But there is a race there.
There is a race, but it's a benign race.
Well, it may or may not be benign.
So there's a gotcha on this one.
So this code only works correctly if the hardware writes the array elements atomically.
So for example, not on the x86-64 architecture we're using.
But on some architectures, you cannot write a byte value.
You cannot write a byte value as an atomic operation.
It implements a right to a byte by reading a word, masking out things, changing the field, masking again, and then writing it back out.
So you can have a race on a byte value.
In particular, even if I were going to do this with bits, I could have a race on bits, although C doesn't let me access bits directly.
The smallest unit I can access is a byte.
So you have to worry about what's the level of atomicity provided by your architecture?
So the x86 architecture, the grain size of atomic update is you can do a single-byte write, and it will do the right, proper thing-- do the right thing on the write.
So we have both things.
No bugs.
No data races doesn't mean no bugs.
Presence of data races doesn't mean you have bugs.
But generally, they're fairly well overlapped.
Now, why would I not want to put in a lock and unlock here just to get rid of the race?
If I run Cilkscreen on this, it's going to complain.
It's going to say, you've got a race here.
Why would I not want to put a lock on here, for example?
Because then we don't have parallelism anymore?
No, well, I'd have parallelism maybe up to 10, for example, right?
Because I have 10 different things that could be going on at a time.
But that's one reason.
That is one reason.
What's another reason why I might not want to put locks in here?
[INAUDIBLE]
It could be that all the numbers-- that's a case where it doesn't get me much speedup.
But what's another reason I might want to do this?
[INAUDIBLE]
I think you're on the right track.
Overhead.
Yeah.
Overhead.
This is my inner loop.
So if I'm locking and unlocking, all this is doing is just doing a memory [?
dereference ?]
and an assignment.
And that may be fairly cheap, whereas if I grab a lock and then release the lock, those operations may be much, much more expensive.
So I may be slowing down the execution of this loop by more than I'm going to gain out of the parallelism of this.
So I may say, I may reason, hey, there is a good reason why not have a data race there.
So I may want to have a data race, and I may want to say that's OK. And if that happens, however, you're now going to get warnings out of Cilkscreen.
And I generally recommend that you have no warnings on Cilkscreen when you run your code.
So the Cilk environment provides a mechanism called "fake locks."
So a fake lock allows you to communicate to Cilkscreen that a race is intentional.
So what you then do is you put a fake lock in around this access here.
And what happens is when Cilkscreen runs, it says, oh, you grabbed this lock, so I shouldn't report a race.
But during execution, no lock is actually grabbed, because it's a fake one.
So it doesn't slow you down it all at runtime, but Cilkscreen still thinks that a lock is being acquired.
Questions about that?
So this is if you want to have an intentional race, this is a way you can quiet Cilkscreen.
Of course, it's dangerous, right?
It's yet another example of what's dangerous here.
Because what happens if you did it wrong?
What happens if there really is a bug there?
You're now telling it to ignore that bug.
So one way that you can make your code-- if you put in fake locks everywhere, you could make it so, oh, Cilkscreen runs just great, and have your code full of race bugs.
So if you use fake locks, you should document very carefully that you're doing so and why that's going to be a safe thing to do.
Any questions about that?
By the way, one of the nice things about some of the concurrency platforms like Cilk is that they provide a layer of abstraction where generally, you don't have to do very much locking.
If you program with Pthreads, for example, you're locking all the time.
So you're writing non-deterministic programs all the time, and you're debugging non-deterministic programs all the time.
Whereas Cilk provides a layer of programming where you can do most of your programming in a deterministic fashion.
And occasionally, you may want to have some non-determinism here or there.
But hopefully you can manage that if you do it judiciously.
Any questions about mutexes and uses for them and so forth?
Good.
So let's talk about how they get implemented.
Because as with all these things, we want to understand not just what the abstractions is but how it is that you actually implement these things so that you can reason about them more cogently.
So there's typically three major properties of mutexes when you look at them.
And when you see documentation for mutexes, you should understand what the difference is of these things.
The first is whether it's a yielding mutex or a spinning mutex.
So a yielding mutex, when you spin, it returns control to the operating system.
And why might you want to do that?
Whereas a spinning one just consumes processor cycles.
Why would you want to do that?
Yeah
[INAUDIBLE] allow other threads
Yeah, it can allow other threads or other jobs that could be running to use the processor while you're waiting.
What's the downside of that?
To speak to the-- either one.
Go ahead
It might be possible that whatever you're trying to do is essential, and you really want to get that done [UNINTELLIGIBLE] everything else executes.
So you really want [INAUDIBLE]
Yeah, context switching a thread out is a heavyweight operation.
And It may be, if you end up context switching out, it may be you only had to wait for a half a dozen cycles and you'd have the lock.
But instead, now you're going and you're doing a context switch and may not get access to the machine for another hundredth of a second or something.
So it may be on the order of 10 to the 6th-- a million instructions before you get access again, rather than just a few.
The second property of mutexes is whether they're reentrant or not.
So a reenttrant mutex allows a thread that's holding a lock to acquire it again.
So I may hold the lock, and then I may try to acquire the lock again.
Java is full of reentrant locks, reentrant mutexes.
So why is this a positive or negative?
What are the pros and cons of this one?
Why might reentrancy be a good thing to want?
Why would I bother doing-- why would I grab a lock that I already have?
It'd be too easy to do a check [INAUDIBLE]
It lets you do what?
It lets you not have to worry about locking when you already have a lock
It lets you not worry about it.
That's right.
But why is that valuable?
It saves you one line in an If statment to check if you have a lock or not
That could be.
Basically, the If statement is embedded in there.
But why would I care?
Why would I want to be acquiring something that I already have?
In what programming situation might that arise?
This seems kind of weird, right?
Could be recursion.
Yeah.
So usually, what it comes from is when you have objects, and you have several methods on the object.
And what you'd like to do is, if somebody's calling the method from the outside, you would like to be able to execute that particular-- I guess in C++ they don't call them "methods."
They call them "member functions."
"Member functions," they call them.
In Java, they call them "methods," and in C++, they call them "member functions."
Doesn't matter.
It's the same thing.
So when you access one of these, normally, from the outside, you want to make sure you grab the lock associated with the object.
However, it may be that what you're doing inside the object is you want to be able-- one of the operations may be a more complex operation that wants to use one of its own implementations.
So rather than implementing it twice-- once in the locked form, once without getting the lock-- you just implement it once, and you use reentrant locks.
And that way, you don't have to worry about, in coding those things, whether or not you've already got it.
So that's probably the most common place that I know that people want reentrant locks.
Naturally, to acquire a reentrant lock, you have to do some kind of If statement, which is a conditional.
And as you know, if it's an unpredictable branch, that's going to be very expensive.
So generally, there is a cost to making it reentrant.
The third property is whether the lock is fair or unfair.
So a fair mutex puts block threads essentially into a FIFO queue.
And the unlock operation unblocks the thread that has been waiting the longest.
So it makes it so that if you try to acquire a lock, you don't have some other thread coming in and trying to access that lock and getting ahead of you.
It puts you in a queue.
So an unfair mutex lets any blocked thread go next.
So the cheapest thing to implement is a spinning, non-reentrant, unfair lock-- mutex.
Those are the cheapest ones to implement.
Very lightweight, very easy to use.
The heavyweight ones are a yielding, reentrant, fair lock.
And of course, you can have combinations, because all of these have, as you can see, different properties in terms of convenience of use and so forth, as well as different overheads.
So there's some cases where the overhead isn't a big deal because it's not in the inner loop of a program or a heavily executed statement.
So let's take a look at one of the simplest locks, which is a simple spinning mutex.
This is the x86 code for how to acquire a lock.
So let's run through this.
So we start out at the top.
And I check to see if the mutex is 0, which is basically, it's going to be 0 if it's free and 1 if it has been acquired.
So we compare it.
If it's free, then I jump to try to get the mutex.
Otherwise, I execute this PAUSE instruction, and this turns out to be a-- it's humorous.
It's x86 hack to un-confuse the pipeline.
So it turns out that in this case, if you don't have a pause here-- which is no-op and does nothing-- x86 mispredicts something or whatever, and it's more time consuming than if it doesn't have that there.
The manual explains very little about this hardware bug except to say, put in the pause.
So if you didn't get it, then you jump to spin mutex, and try again, check to see if it's free.
Now, notice that we're going to spin until it's free, and then we're going to try to get it.
Why not just try to get it first?
Well, think about that while we go through how to get it, and then I'll ask it again.
Think about why it is that you might want to get it first.
So if I want to get the mutex, I first get a value of 1 in my register.
And then I compute this exchange operation, which exchanges the value of the mutex with the value of the-- with the one that I have.
So it exchanges the memory location with the register.
Now, this is an expensive operation-- exchange-- because it's an atomic exchange, and it typically has to go at least out to L2 to do this.
So it's an expensive operation, because it's a read-modify-write operation.
I'm swapping my register value with a value that's in the mutex.
So it turns out that if it's 0, then it means I got it.
So I compare it with 0, and if it's equal to 0, I go onto the critical section.
When I'm done with the critical section, I release the mutex by basically storing 0 in there, because I'm the only one who accesses the mutex at this point.
If I didn't get it, if the value is 1, notice that because I'm swapping a 1 in, even though the 1 got swapped in, well, there was a 1 there before.
So it basically did not affect the value of the mutex.
But I discover, oh, I don't have it.
Then we go all the way back up there to spin mutex.
So here's the question.
Why do I need all this preamble code?
Why not just go straight to Get_Mutex, make the spin mutex here be a jump to Get_Mutex?
Yeah?
Maybe it's because the exchange is expensive
Excuse me?
The exchange is--
Yeah, because the exchange is expensive.
Exactly.
So this code here, I can compare.
And as long as nobody's touching anything, this becomes just L1 memory accesses.
Whereas here, it's going to be at least L2 to do the exchange operation.
So rather than doing that-- moreover, this one actually changes the value.
So what happens when I change the value of the mutex?
Even though I change it to the same value, what happens in order to do that exchange?
Remember from several lectures ago.
What's going to happen when I make an exchange there?
What does the hardware have to do?
What's the hardware going to do on any store to a shared memory location, to a memory location in shared memory that is actually shared?
Yeah?
[INAUDIBLE]
Yeah, it's got to invalidate all the other copies.
So if everybody spinning here-- imagine that you have five guys spinning, doing exchanges-- they're all creating all this traffic of invalidations, what's called an "invalidation storm."
So they create an invalidation storm as they all are invalidating each other so that they can get access to it so that they can change the value themselves.
But up here, all I'm doing is looking at the value.
All I'm doing is looking at the value to see if it's free.
And it's not until the guy actually frees the value that it actually-- actually, this is interesting.
I think I wrote this with Intel syntax, rather than AT&T, didn't I?
The MOV mutex, 0 moves 0 into the mutex, which is Intel syntax.
I probably should have converted this to AT&T, because that's what we're generally using in the class.
I'll fix that before I put the slides up.
Basically, I pulled this out of the Intel manual.
So any questions about this code?
Everybody see how it works?
It relies on this atomic exchange operation.
And I'm going to end up sitting here spinning until maybe I can get access to it.
When I have a chance to get access to it, I try to get it.
If I don't get it, I go back to spinning.
How do I convert this to a yielding mutex?
Instead of having that spinning mutex, you should-- you shouldn't have that.
You should just have something that allows you to just [INAUDIBLE]
Yeah, so actually, the way you do it is you replace the PAUSE instruction.
Exactly what you're saying.
You've got the right place in the code.
We basically call a yield.
And you can use, for example, pthread_yield.
What it tells the operating system is, give up on this quantum.
You can schedule me out.
Somebody else can be scheduled.
Now, if nobody else is there to be scheduled, often you'll just get control back, and you'll jump again and give the operating system another time.
Now, one of the things I've seen in computer benchmarks that use locking is that they all use spin locks.
They never use the yielding, because if you yield, then when the lock comes free, you're not going to be ready to come back in.
You may be switched out.
So a common thing that all these companies do when they're vying for who's got the fastest on this benchmark or fastest on that benchmark is they go through and they convert all their yielding mutexes into spinning mutexes, then take their measurements, when in fact, as a practical matter, they can't actually ship code that way.
So you'll see this kind of game played where people try to get the best performance they can in some kind of laboratory setting.
It's not the same as when you're actually doing a real thing.
So you have a choice here.
There's kind of a tension here.
You'd like to claim the mutex soon after it's released.
And you're not going to get that if you yield.
At the same time, you want to behave nicely and waste few cycles.
So what's the strategy for being able to accomplish both of these goals?
So one of these goals is the spinning mutex does a great job of claiming the mutex soon after it's released.
The yielding mutex behaves nicely and wastes few cycles.
Is there the best of both worlds?
There's certainly the worst of both worlds, right?
What's the best of both worlds?
How might we accomplish both of these goals with small modification to the locking code?
So it turns out you can get within a factor of two of optimal.
How might you do that while wasting few cycles?
So here's the idea.
Spin for a little while, and then, if after a little while you didn't manage to access the mutex, then yield.
So that if the new mutex was right there available to be accessed, you could access it, but you don't spin for an indefinite amount of time.
So the question is, how long do you spin?
So we're going to spin for a little while and then yield.
Yeah?
[INAUDIBLE]
Yeah, exactly.
So what you do is you spin for basically as long as a context switch takes.
So if you spin for as long as it takes to do a context switch and then do a context switch, if the mutex became immediately available, well, you're only going to wait double what you would have waited.
And if in the meantime during that first part where you're spinning it becomes available, you're not waiting at all any longer than you actually have to.
So in both cases, you're waiting at most a factor of two.
In one case, you're waiting exactly the right.
The other, you can actually wait a factor of two.
So this is a classic amortized kind of argument, that you can amortize the cost of the spinning to the context switch.
So spin until you spend as much time as it would cost for a context switch.
Then do the context switch.
Yet another voodoo parameter.
Yeah, so if the mutex is released while spinning, that's optimal.
If the mutex is released after the yield, you're within twice optimal.
Turns out that 2 is not the optimal value.
There's a randomized algorithm that makes it e over e minus 1 competitive where e is the base of the natural logarithm.
So 2.7 divided by 1.7, which is what?
Who's got a calculator?
2.7 divided by 1.7 is-- I should have calculated this out
[INAUDIBLE]
It's about 1.6.
Good.
So it's better than 2.
People analyze these things, right?
So any questions about implementation of locks?
There are many other ways of implementing locks.
There are other instructions that people use.
They do things like compare-and-swap is another operation that's used.
There are some machines have an operation called load-linked/store-conditional, which is not on the x86 architecture, but it is on other architectures.
You'll see a lot of other things of doing some kind of atomic operation to implement a lock.
Uniformly, they're expensive compared to register operations in particular or even L1 accesses, typically, in particular.
Any questions?
So now that we've decided that we're going to use mutexes, and we understand we're writing non-deterministic code and so forth, well, it turns out there are a host of other system anomalies that occur.
So locks are like, they're this really evil mechanism that works really well.
It feels so good that nobody wants to stop using it, even though-- but nobody has better ideas.
One of the most interesting ideas in recent memory is the idea of using what's called "transactional memory," which is basically where memory operates like a database transaction.
And it's allowed to abort, in which case you roll it back and retry it.
Yet, transactional memory has a host of issues with it, and still people use locks.
So let's talk about some of the bad things that happen when you start doing locks.
I'm going to talk about three of them, deadlock, convoying, and contention.
So deadlock is probably the most important one, because it has to do with correctness.
So you can have coded-- in fact, I've seen people with very fast code that has deadlock potential in it.
It's like, if you deadlock, then your average running time is infinite if there's a possibility of a deadlock, right?
Because you're averaging infinity with everything else that you might run.
So it's not good to have deadlock in your code, regardless.
It's kind of like your code seg faulting.
No decent code should seg fault.
It should always catch its own errors and terminate gracefully.
It shouldn't just seg fault in some circumstance.
Similarly, your code should not deadlock.
So here's sort of a classical instance of deadlock.
And deadlock typically occurs when you hold more than one lock at a time.
So here, this guy is going to grab a lock A, going to grab a lock B, then unlock B, unlock A, and in there do a critical section.
Why might I grab two locks?
What's the circumstance where I might have code that looked very similar to this?
Use case
Two objects?
sorry?
You need two objects
You need two objects.
When might that occur?
Account transactions
Yeah, account transactions.
That's the classic one.
You want to move something from this bank account to that bank account.
And you want to make sure that as you're updating it, nothing else is occurring.
Another place this comes up all the time is when you do graph algorithms.
You always want to grab the edge and have the two vertices on each end of the edge not move while you do something across the edge.
So lots of cases there.
It turns out the order in which you unlock things doesn't matter, because you can always unlock something.
You never hold up for unlocking.
The problem with deadlock is generally how you acquire locks.
So in this example, Thread 2 grabs Lock B, then grabs Lock A.
So it might be, for example, that you have some random process that's at the node of a graph.
And now it's going to grab a lock on the other end of an edge.
But you might have the guy at the other end grabbing that vertex and then grabbing the one on your end.
And that's basically the situation.
So what happens is Thread 1 acquires a lock here.
Thread 2 acquires that lock.
And now which one can go?
Neither of them.
You've got a deadlock.
Ultimate loss of performance.
So it's really a correctness issue.
But you can view it, if you really say, oh, correctness, that's for sissies.
We do performance.
Well, it's still a performance issue, because it's the ultimate loss of performance.
In fact, that's probably true of any correctness issue.
No, that's not true.
Sometimes you just get the wrong number.
Here is a correctness issue that your code stops operating.
So there are three conditions that are usually pointed to that you need for deadlock.
The first is mutual exclusion.
Each thread claims exclusive control over the resources that it holds, in this case, the resources being the locks.
So there's got to be some resource that you're grabbing, and that you're the only one who gets to have it.
So in this case, it would be the locks.
The second is non-preemption.
You don't let go of your resources until you complete your use of them.
So that means you can't let go of a lock in a situation.
If you're actually able to preempt-- so this piece of code over there has grabbed locks, and now I can come in and take them away, then you may not have a deadlock potential.
You may have other issues, but you won't have a deadlock potential.
And the third one is circular waiting.
You have a cycle of threads in which each thread is blocked waiting for resources held by the next thread in the cycle.
So let me illustrate this with a very famous story that some of you may have seen, because it is so famous.
It's the dining philosophers problem.
It's an illustrative story a deadlock that was originally told by Tony Hoare, based on an examination question by Edsger Dijkstra.
And the story has been embellished over the years by many retellers.
It's one of these things that if you're a computer scientist, you should know this story just because everybody knows this story.
So here's how the story goes, at least my version of it.
I get to retell it now.
So each of n philosophers needs the two chopsticks on either side of his or her plate to eat the noodles on the plate.
So they're not worried about germs here, by the way.
So you have five philosophers in this case sitting around the table.
There are five chopsticks between them.
In order to eat, they need to grab the two chopsticks on either side.
Then they can eat.
Then they put them down.
So here's what philosopher i does.
So in an infinite loop, the philosopher does thinking, because that's what philosophers do.
Then it grabs the lock of chopstick i and grabs the lock of chopstick i plus 1.
That's the 1.
So if we index them, say, to the left of the plate, this is grabbing the chopstick to the left of your plate.
This is grabbing the chopstick to the right of your plate.
Then you can eat.
Then you release your two chopsticks.
So here, that's the code.
And then you go back to thinking.
I guess they have no other bodily functions.
So the problem is, one day they all pick up their left chopsticks simultaneously.
Now they go to look for their right chopstick.
It's not there.
So what happens?
They starve because their code doesn't let them release-- there's no preemption, so they can't release the chopstick they've already got.
And we have a circular waiting.
They have mutual exclusion.
Only one of them can have a chopstick at a time.
And we have a circular waiting thing, because everyone is waiting for the philosopher on the right.
Is that clear to everybody?
That's the dining philosophers problem.
How do you fix this problem?
What are solutions to fixing this problem?
The problem being that you'd like them to eat indefinitely
You can index the chopstick and say that [INAUDIBLE]
Yeah, you can pick the smaller index first.
So in general, that means everybody would grab the one on their left, then the one on their right, except for the guy who's going between 0 and n minus 1.
They would do n minus 1 and then 0.
They would do n minus 1 first, and then 0.
Sorry.
They would do 0 first, and then n minus 1.
[INAUDIBLE] Let me say that more precisely.
So this is a classic way to prevent deadlock.
Suppose that we can linearly order the mutexes in some order so that whenever a thread that holds a mutex Li and attempts to lock another mutex Lj, we have it that Li goes before Lj in the ordering.
Then no deadlock can occur.
So always grab the resource so if they can all order the resources-- so they're always grabbing them in some subsequence of this order, so they're always grabbing one that's larger and larger and larger, and you're never going back and grabbing one smaller, than you have no deadlock.
Here's why.
Suppose you have a cycle of waiting.
You have a deadlock has occurred.
Let's look at the thread in the cycle that holds the largest mutex that's called Lmax in the ordering.
So whatever is in the ordering.
And suppose that it's waiting on a mutex L held by the next threat in the cycle.
That's the condition.
Well, then it must be that Lmax falls before L, because we're gathering them always in an increasing order.
But that contradicts the fact that Lmax is the largest.
So a deadlock cannot occur.
Questions?
Is this clear?
Who's seen this before?
A few people.
OK. Is this clear?
So if you grab them in increasing order, then there's always some guy that has the largest one, and nobody is holding one larger.
So he can always grab the next one.
So in this case of the dining philosophers, what we can do is grab the minimum of i and i plus 1 mod n and then the maximum of i and i plus 1 mod n. That gives us the same two chopsticks.
And in fact, for most of the philosophers, it's exactly the same order.
But for one guy, it's a different order.
It ends up being the guy who would normally have done n minus 1 and 0.
Instead, he does 0, n minus 1.
So in some sense, it's like having a left-handed person at the table.
You grab your left, then your right, except for one guy does right and then left.
And that fixes it, OK?
That fixes it.
Good.
So that's basically the dining philosophers problem.
That's one way of fixing it.
There are actually other ways of doing it.
One of the problems with this particular solution is you still can have a long chain of waiting.
So there are other schemes that you can use where, for example, if every other one grabs left and then right and then right and then left and then left and then right and then right and left and so forth, you can end up making it so that nobody has to wait to go all the way around the circle.
Yeah?
[INAUDIBLE]
Well, that would be a preemption type of thing, where I grab one, and if I didn't get it in time, I release it and then try again.
When you have something like that, there's an issue.
It's, how do you set the timeout amount?
And the second issue that you get into when you do timeouts is, how do you know you don't then repeat exactly the same thing and convert a deadlock situation into a livelock situation?
So a livelock situation is where they're not making progress, but they're all busily working, thinking they're making progress.
So you timeout.
Let's try again.
What makes you think that the guys that are deadlocking aren't going to do exactly the same thing
[INAUDIBLE]
And exactly.
And in fact, that's actually a workable scheme.
And there are schemes that do it.
Now, that's much more complicated.
Sometimes has more overhead, especially because things become available.
And it's like, no, you're busy raiding some random amount of time before you try again.
So this is, by the way, the protocol that is used on the Ethernet for doing contention resolution.
It's what's called "exponential backoff."
And various backoff schemes are used in order to allow multiple things acquire mutually-exclusive access to something without having to have a definite ordering.
So there are solutions, but they definitely get more heavyweight.
It's not lightweight.
Whereas if you can prevent deadlock, that's really good, because you just simply do the natural thing.
And that tends to be pretty quick.
But yeah, all I'm doing is sort of covering the introduction to all these things.
There are books written on this type of subject.
Any other questions about dining philosophers and deadlock and so forth?
Now let me tell you how to deadlock Cilk++.
So here's a code that will deadlock Cilk++, or has the potential.
You might run it a bunch of times, it looks fine.
Here's what we've done is main routine spawns foo.
Here's foo down here.
All foo does is grab a lock and then unlocks it.
Empty critical section.
It could do something in there.
It doesn't matter.
Then the main grabs a lock, does a cilk_sync and then unlocks it.
So what can go wrong here?
Notice, by the way, this is only one lock, L. There's not two locks.
So you can deadlock Cilk by just introducing one lock.
So here's sort of what's going on.
Let's let this be the main thread and this be foo.
And this will represent a lock acquire, and this is a lock release.
So what happens is we perform the lock acquire here in the parent.
First, we spawned here, then we acquire the lock here.
And now foo tries to get access to the lock, and it can't because why?
The main routine has the lock.
Now what happens?
The main routine proceeds to the sync, and what does it do at the sync?
It waits for all children to be done.
And notice now we've created a cycle of waiting, even though we didn't use a lock.
Main waits, but foo is never going to complete, because it's waiting for the main thread to release it, the main strand here to release it, the main function here.
Is that clear?
So you can deadlock Cilk too by doing non-deterministic programming.
So here's the methodology that will help you not do that.
So what's bad here?
What's bad is holding the lock across the sync.
That's bad.
So don't do that.
Doctor, my head hurts.
Well, stop hitting it.
So don't hold mutexes across Cilk syncs.
Hold mutexes only within strands, only with serially-executing pieces of code.
Now, it turns out that you can hold it across syncs and so forth, but you have to be careful.
And I'm not going to get into the details of how you can do that.
If you want to figure that out on your own, that's fine.
And then you're welcome to try to do that without deadlocking something.
Turns out, basically, if you grab the lock before you do any spawns, and then released it after the Cilk sync, you're OK. You're generally, in that case, OK.
So as always, try to avoid using mutexes, but that's not always possible.
In other words, try to do deterministic programming.
That helps too.
And on your homework, you had an example of where it is that deterministic programming can actually do a pretty good job.
The next anomaly I want to talk about is convoying.
Once again, another thing that can happen.
This one is actually quite an embarrassment, because the original MIT Cilk system that we built had this bug in it.
So we had this bug.
So let me show you what it is.
So here's the idea.
We're using random work-stealing where each thief grabs a mutex on its victim's deck.
So in order to steal from a victim, it grabs a mutex on the victim.
And now, once it's got the mutex, it now is in a position to migrate the work that's on that victim to actually steal the work.
And you want to do that atomically.
You don't want two guys getting in there trying to steal from each other.
So if the victim's deck is empty, the thief releases the mutex and tries again at random.
That makes sense.
If there's nothing there to be stolen, then just released the mutex and move on.
If the victim's deck contains work, the thief then steals the topmost frame and then releases the mutex.
Where's the performance bug here?
[INAUDIBLE] trying to steal from each other.
Like A steals from B, B steals from C, C steals from D, and they all have locks on each other, and then--
No, because in that case, they'll each grab the deck from each other, discover it's empty, and release it.
OK, let me show the bug.
It is very subtle.
As I say, we didn't realize we had this bug until we noticed some codes on which we weren't getting the speedups we were expecting.
Let me show you where this bug comes from.
Here's the problem.
At the startup, most thieves will quickly converge on the worker P0 containing the initial strand, creating a convoy.
So let me show you how that happens.
So here we have the startup of our Cilk system where one guy has work, and all these are workers that have no work to do.
So what happens?
They all try to steal at random.
In this case, we have this guy tries to steal from this fellow, this guy tries to steal from this fellow, et cetera.
So of these, this guy, this guy, and that guy all are going to discover there's nothing there to be stolen, and they're going to repeat the process.
This guy and this guy, there's going to be some arbitration.
And one of them is going to get the lock.
Let's assume it's this one here.
So what happens is, this guy gets the lock.
What does this guy do?
He's going to wait.
Because he's trying to acquire the lock.
He can't acquire the lock, so he waits.
So then what happens?
This guy now wants to steal the work from this fellow.
So he steals a little bit of work.
Then these guys now, what do they do?
They try again.
So this guy tries to steal from there, this guy tries to steal from there, this one happens to try to steal there.
This one sees there's work there to be done, so what does it do?
It waits.
But these guys then try again.
Maybe a little bit more stuff is moved.
They try again.
A little bit more stuff.
They try again.
But every time one tries and gets stuck on P0 while we're doing that whole transfer, they all are ending up getting stuck waiting for this guy to finish.
And now, we've got work over here, but how many guys are going to be trying to steal from this guy?
None.
They're all going to be trying to steal from this one, because they all have done a lock acquisition, and they're sitting there waiting.
So this is called convoying, where they all pile up on one thing, and now resolving that convoy.
So this was a bug in startup.
Why wasn't Cilk starting up fast?
Initially, we just thought, oh, there's system kinds of things going on there.
So the work now gets distributed very slowly, because each one is going to serially try to get this work, and they're not going to try to get the work from each other.
What you want is that on the second phase, half the guys might start hitting this one.
So you get some kind of exponential distribution of the work in kind of a tree fashion.
And that's what theory says would happen.
But the theory is usually done without worrying about what happens in the implementation of the lock.
What's the fix for this?
Yeah?
Can you just basically shove-- when you're transferring, you should also say, I have work, so that people [INAUDIBLE] waiting for that guy to [INAUDIBLE]
You could do that, but in the meantime, it could be that the attempt to steal goes so much faster than the actual getting of the work, you're still going to get half the guys locked up on this one.
And the other half might be locked up on this one.
Good idea.
What other things can we do?
Can you check how many people are waiting on the--
Yeah, so the idea is we don't want to use a lock operation.
So here's the idea.
We use a non-blocking function that's usually called "try_lock," rather than "lock."
try_lock attempts to acquire the mutex.
If it succeeds, great.
It's got it.
If it fails, it doesn't go to spin.
It simply returns and say, I failed.
It doesn't go to spin or to yield or anything.
It just says, oh, I failed, and tells that back to the user.
But it doesn't attempt to block.
So with try_lock now, what can these other processors do?
They do a try_lock-- yeah?
[INAUDIBLE]
Exactly.
Instead of waiting there on the guy that they fail on, they pick another random one to steal from.
So they'll just continually try to get it.
If they get it, then they can do their operation.
If they don't get it, they just look elsewhere for work.
So that's what it does.
It just tries to steal again at random, rather than blocking.
And that gets rid of this convoying problem.
As I say, dangerous programming, because we didn't even know we had a problem.
Just our code was slower than it could have been.
Questions about convoying?
So try_lock is actually a very convenient thing to use.
So in many cases, you may find that, hey, rather than waiting on something with nothing to do, let me go see if there's something else I can do in the meantime.
Contention.
So here's an example of a code where I want to add up some function of the elements of some array.
So here I've got a value of n, which is a million.
And I have a type X.
So we have a compute function, which takes a pointer to a-- did I do this right?
To value V. So anyway, my C++ is not as good as my C, and for those who don't know, my C isn't very good.
So anyway, we have an array of type X of n elements.
And what I do is I set result to be 0, and then I have a loop here which basically goes and adds into result the result of computing on each element of the array.
And then it outputs the result.
Does everybody understand what's going on in the code?
It's basically compute on every element in the array, take the result, add all those results together.
We want to parallelize this.
So let's parallelize that.
What looks like the best opportunity for parallelizing?
Yeah, we go after the for and make it be a cilk_for.
Let's add all those guys up.
And what's the problem with that?
We get a race.
What's the race on?
Result
Result.
They're all updating result in parallel.
Oh, I know how to resolve a race.
Let's just put a lock around it.
So here we have the race.
First, let's analyze this.
So the work here is order n. What is the span?
Log n
Yeah, the span is log n for the control of the stuff here, because this is all constant time.
So the running time here is order n over P plus log n. If you remember the greedy scheduling, it's going to be something like this, because this is the work over P plus the span.
So we expect that if n over P is big compared to log n, we're going to do pretty well, because we have parallelism over log n. So let's fix this bug.
So this is fast code, but it's incorrect code.
So let's fix it by getting rid of this race.
So what we'll do is we'll put a lock before.
We'll introduce a mutex L, and we'll lock L before we add to the result, and then we'll unlock it.
So first of all, this is a bad way to do it, because what I really should do is first compute the result of my array and then lock, add it to the result, and then unlock so that we lessen the time that I'm holding the lock in each iteration.
Nevertheless, this is still a lousy piece of code.
Why's that?
It's still serialized
Yeah, it's serialized.
Every update to result here has to go on serially.
They're n accesses.
They're all going to go one at a time.
So my running time, instead of being n over log n, is going to be something like order n. Believe me, I have seen many people write code where they essentially do exactly this.
They take something, they make it parallel, they have a race bug, they fix it with a mutex.
Bad idea, because then we end up with contention on this mutex.
What's the right way to parallelize this?
Yeah?
Maybe you could have each [INAUDIBLE] have result as an array and have each [INAUDIBLE] one place [INAUDIBLE].
And then at the end, some of all the--
But won't that be n elements to sum up?
[INAUDIBLE]
So basically, have, say.
Eight results, instead of having--
For each thread.
Good.
So that each one could keep it local to its own thread.
Now, of course, that involves me knowing how many processors I'm running on.
So now, if that number changes or whatever-- there's a way of doing it completely processor obliviously
Divide and conquer
Yeah, do divide and conquer.
Add up recursively the first half of the elements, add up the second half of the elements, and add them together.
Next time, we're going to see yet another mechanism for doing that, which gets the kind of performance that you're mentioning but without having to rewrite the For loop as divide and conquer.
We'll see that next time.
So in this case, we have lock contention that takes away our parallelism.
Unfortunately, very little is known about lock contention.
The greedy scheduler, you can show that it achieves T1 over P plus T infinity plus B where B is the bondage, that is, if you add the total time of all critical sections.
That's a lousy bound, because it says, even if they're locked by different locks, you still add up the total time of all the critical sections.
And generally, although you can improve this in special cases, the general theory for understanding contention is not understood very well.
And this upper bound is weak, but little is known about lock contention.
Very little is known about lock contention.
So to conclude, always write deterministic programs, unless you can't.
Always write deterministic programs, unless you can't.
Great.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The last time we talked about nondeterministic programming.
And I think actually the mic is up pretty high.
If we can tone that down just a little bit.
We talked about nondeterministic programming.
And as you recall, the rule with nondeterministic programming is you should never do it unless you have to.
Today we're going to talk about synchronizing with locks.
And it goes doubly that you should never synchronize without locks unless you have to.
There's some good reasons for synchronizing without locks as we'll see.
But it, once again, becomes even more difficult to test correctness and to ensure that the program that you think you've written is, in fact, the program you meant to write.
So we're going to talk about a bunch of really important topics.
The first is memory consistency.
And then we'll talk a little bit about lock free protocols and one of the problems that arises called the AVA problem.
And then we're going to talk about a technology that we're using in the Cilk++ system, which tries to make an end run around some of these problems and allows you to do synchronization without locks, with low overhead.
But it only works in certain context.
So we're going to start with memory consistency.
So here is a very simple parallel program.
So initially a and b are both 0.
And processor zero moves a 1 into a.
And then it moves whatever is in location b into the EBX register.
Processor one does something complementary.
It moves a 1 into b.
And then it moves whatever is in a into the EAX register.
Into the EAX register as opposed to the EBX register.
And the question is what are the final possible values of EAX and EBX after both processors have executed.
Seems like a straightforward enough question.
What values can EAX and EBX have, depending upon-- there may be scheduling of when things happen and so forth.
So it's not always going to give the same answer.
But the question is what's the set of answers that you can get?
Well, it turns out you can't just answer this question for any particular machine without knowing the machine's memory model.
So it depends upon how memory operations behave in the parallel computer system.
And different machines have different memory models.
And we'll give you different answers for this code.
There'll be some answers that you get on some machines, different answers on different machines.
So probably the bedrock of memory models is a model called sequential consistency.
And this is intuitively what you might think you want.
So Lamport in 1979 said, "The result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program."
So what does that mean?
So what it says is that if I look at the processor's program and the sequence of operations that are issued by that processor's program, they're interleaved with the corresponding sequences defined by the other processors to produce a global linear order.
So the first thing is that there's a global linear order that consists of all of these processors' instructions being interleaved.
In this linear order, whenever you perform a load from memory into register, it receives the value that was stored by the most recent store operation in that linear order to that location, i.e.
it's memory.
So you don't get something if you have in this linear order that two processors wrote.
Well, one of them came last.
The most recent one before you read, that's the one that you get.
Now, there may be many different interleavings and so forth.
And you could get any of the values that correspond to any of those interleavings.
But the point is that you must get a value that is represented by some interleaving.
The hardware can then do anything it wants, but for the execution to satisfy the sequential consistency model, for it to be sequentially consistent, must appear as if the loads and storage obey some global linear order.
So let's be concrete about that with the problem that I gave before.
So initially, we have a and b are 0.
And now, we have these instructions executed.
So what I have to do is say, I get any possible outcome based on interleaving these instructions in this order.
So if I look at it, I've got two instructions here, two over here.
So that there are six possible interleavings because 4 choose 2 is 6 for those people who've taken 6042.
So there are six possible interleavings.
So for example, if I execute first move a 1 into a, and then I execute move load into register, the value of b, and then I move 1 into b, and then I load the value of a, I get a value of 1 for EAX and a value of 0 for EBX.
For this particular interleaving of those instructions.
That's what happens if I execute these two before these two.
If I execute these two instructions here before these two here, I get the order 3412.
And essentially, the opposite thing happens.
EAX gets 0 and EBX gets 1.
And then, if I interleave them in some way, where 1 and 3 somehow come first before I do the 2 and 4, then I'll get a value of 11 for each of them.
Those are the middle cases.
So what don't I get?
00
You never gets 00 in a sequentially consistent execution.
Sequential consistent implies that no execution-- whoops, that should be EAX.
That EAX equals EBX equals 0.
I don't ever get that outcome.
If I did, then I would say my machine wasn't sequentially consistent.
So now let me take a detour a little bit to look at mutual exclusion again.
And understand what happens to mutual exclusion algorithms in the context of memory consistency.
So everybody understood what sequential consistency is.
I simply look at my program as if I'm interleaving instructions.
So most implementations of mutual exclusion, as I showed previously, employ some kind of atomic read-modify-write.
So the example I gave you last time was using the exchange operation to atomically exchange a value in a register with a value in memory.
People remember that?
To implement a lock?
So in order to implement a lock, I atomically switch two values.
So we, in particular, use the exchange one.
And there are a bunch of other commands that people can use.
Test-and-set, compare-and-swap, load-linked-store-conditional, which essentially do some kind of read-modify-write on memory.
These tend to be expensive instructions, as I mentioned.
They usually tend to cost something like an L2 cache hit.
Now, the question is can mutual exclusion be implemented with only atomic loads and stores?
Do you really need one of these heavyweight operations to implement mutual exclusion?
What if I don't use our read-modify-write?
Is that possible to do it?
And in fact, the answer is yes.
So Dekker and Dijksra show that it can as long as the computer system is sequentially consistent.
So as long as you have sequential consistency, you in fact, can implement a mutual exclusion with read-modify-write.
We're actually not going to use either the Dekker or Dijksra algorithms, although you can read about those in the literature.
We're going to look at what is probably the simplest such algorithm that's been devised to date, which is devised by Peterson.
And I'm going to illustrate it with these two smileys.
That's a she.
And that's a he.
And they want to operate on widget x.
And she wants to frob it.
And he wants to borf it.
And we want to preserve the property that we are not frobbing and borfing at the same time.
So how do we do that?
Well, here's the code.
So we're going to set up some things before we start he and she operating.
So we're going to have our widget x.
That's our protected variable.
And we're going to have a Boolean set initially to false that says whether she wants to frob it.
So we don't want to make them frob it unless they want to frob it.
And we don't want him to borf it unless he wants to borf it.
And we're going to have an extra auxiliary variable, which is whose turn it is.
So they're going to sort of do a take turn.
But that only is going to come into account if the other one doesn't have a conflict.
If they don't have a conflict, then one of them is going to be able to go.
So here's what she basically does.
She initially sets that she wants to operate on the widget.
And then, what she does is she sets the turn to be his.
And then, while he wants it, and the turn is his, she's going to just spin.
Notice that you're not frobbing it in while loop.
The body of the while loop is empty.
So this is a spinning solution.
So while he wants it, and it's his turn, you're just going to sit there, continually testing the variables he wants and turn equals his until one of them ends up being false.
So if he doesn't want it, or it's not his turn, then she gets to frob it.
And when she's done, she sets she wants to false.
And he does a similar thing.
He sets it to true, says it's her turn.
And then, while she wants it, and the turn is hers, just sits there waiting, continually re-executing this until finally, one of these turns out to be false.
And then he borfs it.
And he sets it to false.
And then, they're doing both of these things sort of in a loop, periodically coming back and executing it.
And what you want to do is you don't want to make it so that it's forced.
That it's one turn, then the other because maybe he never wants to borf it.
And then, she would be stuck not being able to frob it, even though he doesn't want.
So if you think about this-- let's think about why this always is going to give you mutual exclusion.
So basically, what's happening here is if he wants it-- by the way, these things are not easy to reason about.
And usually, as much as I can talk and talk in class, what you really need to do is go home, and sit down with this kind of thing.
And study it for 10 minutes.
And then, you'll understand what the subtleties are as what's going on.
But basically, what we're doing is we're making it so that it's not going to be the case that both she's setting it and she wants it.
And the turn is his.
And then, if there's a race where he wants it also, then that's going to preclude both of them from going into it at the same time.
And then whichever one sets the turn, one of those is going to occur first.
And one is going to occur second.
And whoever ends up coming second, the other one gets to go ahead.
So it's very subtle how that is actually working to make sure that each one is gating the other to allow them to go.
But the way to reason about this is to reason about it is what are the possible interleavings?
And the important interleavings here as you can see are what happens when setting these things.
And once they're set, what happens in testing these things?
And especially because when you go around the loop and so forth, you have to imagine that an arbitrarily long amount of time is gone.
So for example, between the time that you check that the turn is his, he may have already gone around this loop.
And so you have to worry about-- even though, it may look like one instruction from this processors point of view for correctness purpose, you have to imagine that an arbitrary amount of computation could occur between any two instructions.
So any question about this code?
People see how it preserves mutual exclusion and how you use sequential consistency to reason about it by asking what are the possible interleavings?
Questions?
Yeah
So, I don't know if I got it right.
So basically, sets the [UNINTELLIGIBLE] to give him a chance before she goes to loop.
So basically, she waits there until he has been able to go?
That's why on the [UNINTELLIGIBLE]
So on this third line--
Both of them.
Either before actually frobbing or borfing.
And before that while you always give the turn to the other to give them a chance to go
Yeah.
So there are two things you want to show.
One is that they can't both be stalled on the while loop there.
And that can't happen because the turn can't be simultaneously his and hers.
So you know that they're not both going to deadlock in trying to do this by sitting there waiting for the other because of this.
And now, the question is well, how do you know that one can't get through while the other is also going through?
And for that, you have to look and say, oh well, if you go through, then you know that it is either he doesn't want it, or it's not his turn.
And in which case, if he doesn't want it, it's not his turn.
If he does change it to that he wants it, then in fact, it's going to be your turn.
Question?
This only works for exactly two threads, right?
This only works for exactly two threads.
This does not work for three, but there are extensions of this sort of thing to end threads in an arbitrary large number.
However, the data structures to implement this kind of mutual exclusion for end threads end up taking space proportional to n. And so one of the advantages of the built in atomics-- the compare-and-swap, or the atomic exchange, or whatever-- is they work for an arbitrary number of threads with only a bounded amount of resource.
You don't require extra data structures and so forth.
So that's why they put those things in the architecture because in the architecture you can build things that will solve this problem much more simply than this sort of thing.
However, there are going to be lessons here that you may want to use in your programming, depending on what you're doing.
So now, it turns out that no modern day processor implements sequential consistency.
There have been machines that were built-- actually quite good machines-- that implemented sequential consistency.
But today, nobody implements it.
They all implement some form of what's called relaxed consistency, where the hardware may reorder instructions.
And so you have things executing not in program order.
And the compilers may reorder instructions as well.
So both the hardware and the software are going in there.
So let's take a look at that.
So here's the program order for one of the things.
We move 1 into a, and then move the value of b into EBX to do a load.
Here's the program order.
Most modern hardware will switch these and execute it in this order.
Why do you suppose?
Even if you write it this way, the instruction level parallelism within the processor will, in fact, execute it in the opposite order most of the time.
Yeah?
Because loading takes longer
Yeah.
Because loading takes longer.
Loading is going to take latency.
I can't complete the load from the processor's point of view until I get an answer.
So if I load, and I wait for it to go out to the memory system and back into the processor, and then I do a store-- well, as soon as I've done the store, I can move on.
Even if the store takes a while to get out to the memory system.
But if I do it in the opposite order.
I do the store first, and then I do the load, I've ended up wasting essentially one cycle, the cycle to do the store, when I could have been overlapping that with the time it took to do the load.
So people follow that?
So if I execute the load first, I can go right on to execute the store.
I can issue the load, go right on to execute the store without having to wait for the load to complete if I have a multi-issue CPU in the processor core.
So you get higher instruction level parallelism.
Now when is it safe for the hardware compiler to perform this reordering?
Can it always switch instructions like this to put loads before stores?
When would this be a bad idea to put a load before a store?
Yeah?
You're loading the variable you just stored
Yeah, if you're loading the variable you just stored.
Suppose you say store into x and then load from x.
That's different from if I load from x, and then I store into x.
So if you're going to the same location, then that's not a safe thing to do.
So basically, in this case, if a is not equal to b, then this is safe to do.
But if a equals b, this is not safe to do.
Because it's going to give you a different answer.
However, it turns out that there's another time when this is not safe to do.
So this would have been the end of the story if we were running on one processor.
The other time that it's not safe to do it is-- if it's safe, the other assumption is that there's no concurrency.
If there is concurrency, you can run into trouble as well.
And the reason is because another processor may be changing the value that you're planning to read.
And so if you read things out of order, you may violate sequential consistency.
Let me show you what's going on in the hardware so you have an appreciation of what the issue is here.
So here's 30,000 feet of hardware reordering.
So the processor is going to issue memory operations to the memory system.
And results of memory operations are going to come back.
But they really only have to come back when?
If they're loads.
If they're stores, they don't have to come back.
So the processor, in fact, can issue stores faster than the network can handle them.
And the memory system can handle them.
So the processors are generally very fast.
The memory systems are relatively slow.
But the processor is not generally issuing a store on every cycle.
It may do store, it may do some additions, it may do another store, et cetera.
So rather than waiting for the memory system to do every store, they create a store buffer.
And the memory system pulls things out of the store buffer as fast as it can.
And the processor shoves stuff into the store buffer up to the point that the store buffer gets full, in which case it would have to stall.
But for most many codes, it never has to stall because there is a sufficient frequency of other operations going on that you don't have to wait.
So when a store occurs, it doesn't occur immediately on the store buffer.
Now along comes a load operation.
And the load operation, if it's to a different address, you want to have that take priority because the processor can be waiting.
It's next instructions may be waiting on the result.
So you want that to go as fast as possible.
They have a passing lane here where the fast cars or the important cars, the ambulances, et cetera, in this case loads, can scoot by all the other things in traffic and get to the memory system first.
But as we said, we don't want to do that if the last thing that I stored was to the same address.
So in fact, there is content addressable memory here, which matches the address that is being loaded with everything in the store buffer.
And if it does match, it gets satisfied immediately by the store buffer.
And only does it make it out to the network if it's not in the store buffer.
But what you can see here is that this mechanism, which works great on one processor, violates sequential consistency because I may have operations going to two different memory locations, where the order, in fact, matters to me.
So let's see how that works out.
So first of all, let me tell you what the memory can-- so a load can bypass a store to different address.
First of all, any questions about this mechanism?
So this accounts for a whole bunch of understanding of what happens in concurrency in systems.
This one understanding of store buffers.
It's absolutely crucial.
And I have talked, by the way, with lots of experts who don't understand this.
That this is what's going on for why we don't have sequential consistency in our computers.
It's because they made the decision to allow this optimization, even though it doesn't preserve sequential consistency.
There were machines in the past that did support sequential consistency.
And what they did was they used speculation to allow the processor to assume that it was sequentially consistent.
And if that turned out to be wrong, they were able to roll back the processor's state to the point before the access was done.
In fact, the processor is already doing that for branches, where it makes branch predictions and executes down a line.
But it's wrong, it has to flush the pipeline and so forth.
Why they don't do the same thing for hardware is an interesting-- for loads of stores-- is an interesting question.
Because at some level there's no reason they couldn't do this.
Instead, it's sort of been a thing where the software people say, yeah we can handle it.
And the hardware people say, OK. You're willing to handle it.
We won't worry about it then.
When in fact, it just makes life complicated for everybody that you don't have sequential consistency
[INAUDIBLE] you have to do speculation across both [INAUDIBLE]
Well here, you only have to do speculation over what actually is coming out of your memory system.
And if it doesn't match, you could roll back.
The issue, in part, is how many machine states are you ready to roll back to.
Loads come more frequently than branches.
That's one thing.
So no doubt, there are good reasons for why they're doing it.
Nevertheless, definitely loss of sequential consistency becomes a headache for a lot of people in doing a concurrent program.
We had a question here?
Yes, Sara?
So this does not preserve sequential consistency?
But as long as there's only one processor, it should have the same effect, right?
But sequential consistency for one processor is easy because all you do is execute them--
Yeah, I'm just saying--
It should have the same effect, exactly.
So on one processor, this works perfectly well.
If there's no concurrency, this is going to give you the same behavior.
And yet, you've now got this optimization that loads can bypass stores.
And therefore, you can do a store and a load and be able to overlap their execution.
So this definitely wins for serial execution.
Yep, good.
Any other questions about this mechanism?
You could reason about it on the quiz.
That kind of thing, right?
Yeah, OK?
So here's the x86 memory consistency model.
For many years, Intel was unwilling to say what their memory consistency model was for fear that people would then rely on it.
And then, they would be forced into it.
But recently, they've started being more explicit about it.
And this is the large part of it.
I haven't put up all the things because there are a whole bunch of instructions, such as locking instructions and so forth, for which for some of them, it's more complicated.
But this is the basics.
So loads are not reordered with loads.
So if you add a load to one location, a load to another location, they always execute in the same order.
Stores are not reordered with stores.
If you have store and then a subsequent store, those two stores always go in that order.
Stores are not reordered with prior loads.
So if you do a store after a load-- if you do a load and then a store, they're going to go in that order.
However, a load-- and this is what we just talked about-- may be reordered with a prior store to a different location but not with a prior store to the same location.
So that's exactly what we just talked about on the previous slide.
Then, loads and stores are not reordered with lock instructions.
So a certain set of instructions are called lock instructions.
And they include all the atomic updates, the exchanges, comparisons-and-swaps, and a variety of other atomic operations that the hardware supports.
The stores to the same location always respect a global order.
Everybody sees the store to a location in exactly the same order.
And the lock instructions respect a global total order.
So that everybody sees that this thread, or processor, got a lock before that one.
You don't have two different processors disagreeing on what the order was that somebody acquired a lock or whatever.
And then, memory ordering preserves transitive visibility, which is sort of like saying it obeys causality.
In other words, if after doing a, if you had some effect, and then you did b, it should look like to other people like a and then b happened.
Like there's a causality going on.
But that's not sequential consistency, mainly because of four here.
So what's the impact of reordering?
So here, we have our example from the beginning for the memory bottle, where I'm storing a 1 into a and then loading whatever is in b.
And similarly, over here the opposite.
So what happens if I'm allowed to do reordering?
What can happen to these two instructions?
Yeah.
They can execute in the opposite order.
Similarly, these two guys can execute in the opposite order.
So they can actually execute in this order where we do the load and then the stores.
So it executes as if this were the order.
Did I do this right?
Executes as if this were the order.
So I could do 1, 2, 3, 4.
So if then, I do the ordering 2, 4, 1, 3
[INAUDIBLE]
I got the screwed up, I think.
Didn't I?
[INAUDIBLE]
Because I should be swapping these guys, right?
Swapped the wrong [INAUDIBLE]
Ugh.
OK.
So if I did this one 2, 1, 4, 3.
So ignore this thing.
Suppose I do the order 2.
So basically, I load b.
Then, I load a.
Then, I store a.
And then, I store b.
What's the result value that are in EAX and EBX?
You get 00.
Remember 00 wasn't the legal value from sequential consistency.
But in this case, the Intel architecture and many other architectures out there will give you the wrong value for the execution of these instructions.
Any question about that?
So it doesn't preserve sequential consistency.
So that's kind of scary in some way because you got to reason about this.
Let's see what happens in Peterson's algorithm if you don't have sequential consistency.
So here we go.
We have the code where she wants is true, turn is his, et cetera.
How is this going to fail?
What could happen here?
Where will the bug arise?
What's going to happen?
What's the reordering that might happen?
On the while you do loads, right?
[INAUDIBLE] the he_wants and turn is
Sorry?
On the while statement, you do a load, right?
Because [INAUDIBLE]
Right.
He_wants is a load
And so that will get reordered
Where could that be reordered to?
That could be reordered all the way to the top.
Similarly, this one can be reordered all the way to the top.
So the loads could be ordered all the way to the top.
And now, what's going to happen is you're going to set that she_wants is true but get a value of he_wants that might be old.
And so they won't see each other's values.
And so then, both threads can now enter the critical section simultaneously.
Yeah, Reid?
If you swap the order of the loads, does the [INAUDIBLE]?
If you swap the order of the loads--
If you swap-- put the turn equals his on the left, [INAUDIBLE] on the right.
Because according to--
Put the turn equals his over here?
Because the he_wants can't cross the load
Yeah, but that's not what you want to do
Then you can't [INAUDIBLE]
The whole idea here is that when you're saying you want to do something, you give the other one a turn so that whoever ends up winning the race allows just one of them to go through.
Yeah?
I think the point is that if you put turn equals his and he_wants--
You're saying this stuff here
Swap those two [UNINTELLIGIBLE] turn equals his will not be reordered before the store that--
You might be right.
Let me think about that
You both reorder the same [?
word.
?]
But you just stored turn, right?
Yeah.
So if do turn equals his-- I see what you're saying.
Do this turn equals his.
I was looking at this turn equals his
You mean turn equals equals his
So the Boolean expression [INAUDIBLE]
Yeah.
OK, I hadn't thought about that.
Let me just think about that a second.
So if we do the turn equals his--
[INAUDIBLE] and you won't reorder those two [INAUDIBLE]?
Then the-- Yeah.
You got to be-- I have to think about that.
I don't know about you folks, but I find this stuff really hard to think about.
And so do most people, I think.
This is one of these things where I don't think I can do without sitting down for 10 minutes and thinking about it deeply.
But it's an interesting thought that if you did it the other direction that maybe there would be a requirement there.
I'm skeptical that that is true because to my knowledge to do the mutual exclusion, you pretty much have to do what I'm going to talk about next.
But it would be interesting if is true.
Because you also have to worry about this guy getting reordered with respect to this one
The loads can't be reordered with respect to each other
So he_wants and turn equals his.
Yeah.
So the loads won't be reordered.
Yeah.
So that looks OK. And then, you're saying and then therefore, it can't go forward because this one won't get reordered with that one.
You might be right.
That'd be cute.
So I have to update the slides for next year if that's true.
So one way out of this quandary is to use what's called a memory fence or memory barrier.
And it's a hardware action that enforces an ordering constraint between the instructions before and after the fence.
So a memory fence says don't allow the processor to reorder these things.
So why would you not want to do a memory fence?
Then we'll talk about why you do it.
Yeah?
To force a hardware slowdown?
Yeah.
You're forcing the hardware slowdown.
You're also forcing compiler because the compiler has to respect that, too.
You're not letting the compiler do optimizations across the fence.
So generally, fences slow things down.
In addition, it turns out that they have some significant overhead.
So you can issue a memory fence explicitly as an instruction.
So the mfence instruction sets a memory fence.
There's also, it turns out, on x86 an lfence and an sfence, which allow loads to go over but not stores and stores but not loads.
And this one is basically both.
From the point of view of what we're using it for, we're only going to worry about the fences.
They're done by the explicit one.
But it also turns out all the locking instructions automatically put a fence in.
One of the humorous things in recent memory is major manufacturers for whom the lock instruction was actually faster than doing a memory fence, which is kind of weird because a lock instruction does a memory fence.
So how do you think that sort of thing comes about?
So when you looked at performance it would be like-- for this particular machine I'm thinking about-- it was 30 cycles to do a lock instruction.
And it was on the order of 50 cycles to do a memory fence.
And so if you want to do a memory fence, what should you do?
Do a lock
Do a lock instruction instead to get the effect.
But why do you suppose that came up in the hardware?
Why is it that one instruction would be-- It's a social reason why this sort of thing happens.
So I don't know for sure.
But I know enough about engineering to understand how these things come about.
So here's what goes on.
They do studies of traces of programs.
And how often do you think lock instructions occur?
And how often do you think fence instructions occur?
Turns out lock instructions occur all the time, whereas fences, they don't occur so often because usually it's somebody who really knows what they're doing who's using a memory fence.
So then, they say to the engineering team, we're going to make our code go faster.
And lock instructions are going really fast.
So they put a top engineer on making lock instructions go fast.
They put a second-rate engineer on making memory fence operations go fast because they're not used as often, without sort of recognizing that, gee, what you do for one is the same problem.
You can do the same thing for the other.
So it ends up you'll see things in architecture that are really quite humorous like that, where things are sort of like, wait a minute, how come this is slower when well, it probably has to do with the engineering team that built the system.
And actually now I'm aware of two architectures where they did the same kind of thing by different manufacturers.
Where they got these memory fences.
It should be at least as fast because the one is doing-- anyway.
Interesting story there.
Now, you can actually access a memory fence using a built in function called sync synchronize.
And in fact, there whole set of atomics-- I've put the information here for where you can go and look at the atomic operations that include memory fences and so forth to using in the compiler.
It turns out when I was trying to get this going last night, I couldn't get it to work.
And it turns out that's because our compiler had a bug where this instruction was compiling to nothing.
There's a compiler bug.
And so I messed around for far too much time and then finally sent out a help message to the T.A.s.
And then, John figured out that there was a bug.
And he's patched all the compilers so that you guys all have it.
But anyway, it was like, how come this isn't working?
What compiler are we using?
This was GCC.
I was trying 4 1, and I tried 4 3.
And so the one that we're using in class for the most part, is 4 3.
So anyway, John put the patch in.
So now, when you use these instructions, they're all there.
And then, the last thing is that the typical cost of a memory fence operation is comparable to that of an L2 cache access.
So memory fences tend to be on our machine-- and I haven't actually measured in our machine.
I meant to do that, and I didn't get around to it.
It's probably on the order of 10, or 15 cycles, or something, which is not bad.
If it's less than 20, it's pretty good.
So here's Peterson's algorithm with memory fences.
You just simply sticky in the memory fence there to prevent the reordering.
And it's interesting if there's a way that we can play the game with the instruction stream to do the same thing because that would make this code go, generally, a lot faster in terms of overhead.
And so using memory fences, you can restore consistency.
Now, memory fences are like data races.
If you don't have them, how do you know that you don't have them.
It's very difficult to regression test for them, which is one reason I think there was a bug in the GCC compiler.
How do you know that some piece of code is failing because most of the time it will work correctly.
It's just occasionally, they'll be some reordering, and timing, and race condition that causes it not to work out.
In this case, you both have to have the race and the reordering happening at the same time for Peterson's algorithm, for example.
So compilers can be very difficult for things like this.
Really, the way to do it, which is what I was doing, was do an objdump and search for is fence in there.
And in this case, it wasn't in there
And also compiler's self-analyzers, by itself.
And that's this instruction that basically can take code
Right.
It's not doing anything.
Right.
So it says, oop, get out of it.
Yep.
Good.
So any questions about consistency.
So what turns out to be most of the time when you're designing things where you want to synchronize through memory directly, rather than using locks or what have you.
The methodology that I found works pretty well.
Work it out for sequential consistency, and then figure out where you have to put the fences in.
And that's a pretty good methodology for working out where-- here's sequential consistency.
Now, what reorderings do I need to ensure in order to make sure that it works properly.
And that can be error prone.
So once again, big skull and cross bones on whether you actually try this in practice.
It really better make a difference.
Now, the fact that you can synchronize directly through memory has led to a lot of protocols that are called lock-free protocols, which have some advantages, even though, in particular, because they don't use locks.
And so I want to illustrate some of those because you'll see these in certain places.
So recall the summing problem from last time.
So here we have an array.
And what we're going to do is run through all the elements in the array, computing something on every element, and adding into result.
And we wanted to parallelize that.
So we parallelize that with a Cilk 4.
And what was the problem when we parallelize this?
We get a race.
So there's the race.
We get a race on result because we've got two parallel instructions both trying to update results at the same time.
So we can solve that with a lock.
And I showed you last time that we could solve this for lock.
By declaring a mutex, and then locking before we update the results, and then unlock.
And of course, as we argued yesterday, that could cause severe contention.
Now, contention can be an issue.
But if it turns out that the compute here, which I've moved outside the lock notice.
I've put it into temp and then added temp in so I can lock for the shortest possible time.
If this compute is sufficiently large, there may be contention.
But it may not be a significant contention in your execution because the update here could be very, very short compared with the time it takes to compute.
So for example, if computing on array i cost you more than say order n time, then the fact that you have contention there isn't going to matter, generally, because the total amount of time that you're going to be locking is just small compared to the total execution time.
Still in a multiprogram setting, there may be other problems that you can get into, even when you have this and even if you think that contention is going to be minimal.
So can anybody think of what the issues might be?
Why could this be problematic even if contention is not a big issue?
And the hint here is it's in a multiprogram setting.
So what happens in a multiprogram setting.
Yeah
[INAUDIBLE]
Because the resolve is--
[INAUDIBLE PHRASE]
It actually doesn't have to do with resolve here.
It has to do with locking explicitly.
It's a problem with locking in a multiprogrammed environment.
What happens in a multiprogrammed environment?
What do I mean by multiprogrammed environment?
[INAUDIBLE]
Have multiple jobs running, right?
And what happens to the processor when there are multiple jobs running?
[INAUDIBLE]
Contact switches.
So now, what can go wrong here?
What can be really bad here?
Yeah
You aquire the lock and then the contacts switch out
Yeah.
You acquire the lock.
And then, the operating system contact switches you out.
And so what happens?
You hold the lock while some other job is running.
And what are those guys doing.
They go and spin and wait on the lock.
Now, this is a good time where you'd rather not have a spinning lock.
You'd rather have a yielding lock.
But even so, suddenly you're talking about something that's operating at the level of 100 times a second, 10 milliseconds, versus something that is operating on a nanosecond level.
So you're talking six orders of magnitude of performance difference if you end up getting switched out while you hold a lock.
That's the issue.
What happens.
And then, if that happens, all the other loop iterations must wait
[INAUDIBLE] in the large program here [UNINTELLIGIBLE PHRASE].
I don't have a mic.
If one [UNINTELLIGIBLE] crashes or one [UNINTELLIGIBLE PHRASE] the lock [UNINTELLIGIBLE PHRASE]
Can you specify whether those are yielding locks or spinning locks?
Usually, the mutex type will tell you.
I'm just using a simple name of mutex.
I probably should have been using the ones that-- we were using one called Cilk mutex.
And I probably should've used that here rather than just simple mutex
Are they yielding?
There's a good question.
I used to know the answer to this.
I believe that those spin for a while, are competitive.
They spin for a while and then yield.
But I'm not sure.
They may just spin.
They don't just automatically yield.
They're either competitive, or they'll spin and yield.
I believe they spin and yield.
And I believe there's actually a switch where you can tell it-- if you're doing timing measurements-- make it so that it purely spins so that you can get better benchmark results
So my question is does the colonel have power to switch out a spinning lock or not?
Yeah.
Well, the colonel, the scheduler, can come in at any moment and say, whip you're out.
You're out.
That's it.
And wherever it is, it interrupts it at that moment in time.
So one solution to this problem is to use a lock-free method.
And one of the common ways of doing that is with what's called compare-and-swap instruction.
So this is what's called a locking instruction, meaning it's one of these ones that goes out to L2, in terms of timing and so forth.
And what it does is it does the following thing.
It has an address of a location.
And it's got the old value that was stored in the location and a new value.
And it says if the value that is there is the old value, well, then stick the new value in there.
And then return essentially true.
Otherwise return false.
So it's basically saying what you tend to do is you first look to see what's the value.
You then update the value.
And then you say, if it hasn't changed, stick it back in and return true.
If it has changed, return false.
So it's only swaps the value if it is true.
There's actually two versions.
One which says bool and one which says val.
And if you do the bool version, it returns a flag.
If you do the val version, it actually returns the value that was in there.
So it's more like a compare-and-swap.
The main thing about this is this code essentially executes atomically with the single instruction, which is called-- The instruction is cmpxchg.
Is it up there?
Oh, there it is.
Yeah.
So the cmpxchg instruction on x86.
So when you compile this, you should find on your assembly output that instruction somewhere unless the compiler figures out a better way to optimize that.
But generally, you should find that.
Also, one of the things about this is it works on values that are sort of integer type values.
But it doesn't work on floating point numbers, in particular.
So you can't compare-and-swap a value, which is a floating point value.
You can only do it with energy type values.
So let's take a look at how we can use the compare-and-swap for the summing problem.
So what we do is we have the same sort of code.
And now, what I'm going to do is compute my temporary value.
And then, what I'll do is I'll read the value of result into old.
I'll then update my new value for what I think I want the result to be the result plus the thing that I computed.
And now, what I do is I attempt to compare-and-swap as long as the old value is what I read it to be.
Swap in the new value.
If the old value turns out to be different from what is currently in the result location, then it returns false.
And I redo this again.
Then, I have to redo the whole loop again.
So this is a do-while loop.
Do-while is like a while loop, except you do the body first.
And then you test the condition.
So if this fails, I go back.
I then get a new value for the result and so forth.
So let me show you how that works.
Let's see.
So first, I'll show you how this works.
Actually, I'll show it on a more interesting example how this works.
So what happens if I get swapped out in the middle of a loop iteration?
All I do is when I do the compare-and-swap it fails.
So no other instructions can wait.
They can all march ahead and do the thing they need to do.
And then, the one that got swapped out, eh.
It gets some old value.
It discovers that and has to re-execute the loop.
So is that fine?
So what this means is that the amount work that's going on, however, could in fact, be greater, depending upon how much contention there is.
If there's a lot of contention, you could end up having these guys fighting and not re executing a lot of code.
But that's really not much worse than them spinning is what it comes down to.
Any questions?
Let's do a more interesting example.
Here's a lock-free stack.
So what we're going to do is we're going to have a node, which has a next pointer and some data.
All we really care about is the next pointer.
And we have a stack, which has basically a head pointer.
So we have a linked list here.
We want to basically be able to insert things at the front and take things out of the front.
So here's a lock-free push.
So remember, this could be concurrent.
So these guys want to operate on it at a time.
We saw last time how in doing very simple updates on link structures, you could get yourself into a mess if you didn't properly synchronize when we did the insertion in the hash table.
So here's my push [?
up ?]
code.
Well let's walk through it.
It says, basically, here's my node that I want to insert.
It says, first of all, make node.next point to the head.
So we basically have it pointing to 77.
So then what we say is OK. Let's compare-and-swap to make the head point to the node but only if the value of the head has not changed.
It's still the value of the node.next.
And if so, it does the swap.
Question?
You say compare-and-swap.
But you compare it to what?
In this case it's comparing to the-- so this is basically the location that you're doing the compare-and-swap on, the old value that you expect to see in that location, and the new value.
So here, what it says-- when we're at this point here-- we're saying before you do the compare-and-swap, we're saying, I only want you to set that pointer to go to here if this value is still pointing to there.
So only move this here if this value is still 77.
In other words, if somebody else came in-- well, I'll do an example in a second that shows what happens when we have concurrency, and one of them might fail.
But if it is true, then it basically sets it.
And now I'm home free.
So let's take a look at what happens when we have contention.
So I have two guys.
So 33 says, OK I'll come in.
Let me set my next pointer to the head.
But then comes 81.
And it says, OK. Let me try to set my pointer to also be 77 because I look at what the head is, and that's where it's supposed to go.
So now, what happens is we do the compare-and-swap operation.
And they both are going to try to do it.
And one of them is going to, essentially, do it first because the hardware preserves that the compare-and-swaps, their locking operations, they will happen in some definite order.
So in this case, 81 got in there and did its compare-and-swap first.
When it looked, 77 was still a value that it said.
So it allowed that pointer to be changed.
But now what happens when 33 tries.
33 tries to do the compare-and-swap.
And the compare-and-swap fails because it's saying, I want to swap 33 in as long as the value of head is the pointer to 70, the node was 77.
The value is no longer the pointer to the node of 77.
It's now the pointer to the value of the node with 81.
So the compare-and-swap fails.
People follow that?
And so what does 33 have to do?
It's got to start again.
So it goes back around the loop, and now it sets it to 81, which is now the head.
And now, it can compare-and-swap in the value.
And they both get in there perfectly well.
Question?
What if there's [INAUDIBLE]?
What if two nodes have--
Well, notice here, it's not looking at the value of the data.
Nowhere does data appear here.
It's actually looking at the address of this chunk of memory.
There is a similar problem, which I will raise in just a moment.
There is still a problem.
Yeah, question?
So I'm confused about the interface.
So you give it the address of where you want to compare the value of.
And you're giving it what you're pointing at and--
And here's the value that I expect to be stored in this location.
The value I expect to be in there is node dot next.
So if I go back a couple things.
Here.
Here, the guy says, the value I expect to be there is in this case.
the address of this chunk of memory here.
He expects the address of the node containing 77 is going to be in this location.
It's not.
What's in this location is the address of this chunk of memory.
But you're saying, if it's equal to this, then you can go ahead and do the swap.
Otherwise you're going to fail.
And the swap consists of now sticking this value into-- conditionally sticking it in there.
So you either do it or you don't do it.
So let's now do a pop.
So pop you can also do with things.
So here, I'm going to want to extract an element.
And what I'm going to do is create a current value that I want to make point to the element that gets eliminated.
So what I do is I say, well, the element that I want is that guy there.
And now, what I want to do is make the head jump around and point to 94.
So what I do is I say, well, as long as the-- and I want to do that unless I get down to the fact that I have an empty list.
So basically, I say, if the head still has the value of current-- so they're pointing to the same place-- then, I want to move in current arrow next.
And then I'm done.
Otherwise, I want to set current to head, reset it, and go back to the beginning and try to pop again.
And I'm going to keep doing that until I get my pop to succeed or until current points to nil.
If it ended up at the end, then I don't want to keep popping if the list ended up being empty.
So basically, it sets that one to jump over.
And now, once it's done that, I can go, and I can clean up, I can get rid of this pointer, et cetera.
But nobody else who's coming in to use this link list, can see 15 now because I'm the only one with a pointer to it.
So people understand that?
So where's the bug?
Turns out this has a but after all that work.
Each of these individually does what it's supposed to do.
But here's the bug.
And it's a famous problem because you see it all the time when people are synchronizing through memory with lock-free algorithms.
It's called the ABA problem.
So here's the problem.
And it's similar to what some people were concerned about earlier.
So here's the ABA problem.
Thread 1 begins to pop 15.
So imagine that what it does is it sets its current there, and then it reads the value here, and starts to set the head here using the compare-and-swap.
But it doesn't complete the compare-and-swap yet.
The compare-and-swap hasn't executed.
It's simply gotten this value, and it's about to swap it here.
So then, thread 2 comes along.
And it says, oh, I want to pop something as well.
So it comes in.
And it turns out it's faster, and manages to pop 15 off, and set up its pointers.
Now, what would normally happen here is if this completed, what would happen?
The compare-and-swap instruction would discover that this pointer is no longer the pointer to the head.
And so it would fail.
We'd be all hunky dory.
No problem.
But what could actually happen here?
Thread 2 keeps going on.
It says, oh, let me pop 94.
So it does the same thing.
So thread 1 is still stalled here, not having completed its compare-and-swap.
It swaps 94.
Then, thread 2 goes on and says, oh, let's put 15 back on.
So it puts 15 back on because after all, it had 15.
So now, what happens here?
Thread 1 now looks, and it now completes, and does its compare-and-swap, it resumes, splicing out 15, which it thinks it has.
But it doesn't realize that other stuff has gone on.
And now, we've got a mess.
So this is the ABA problem because what happened was we were checking to see whether the value was still the same value, the same chunk of memory.
It got popped off.
But it got popped back on.
But now, it could be in any configuration.
We don't know what it is.
And now, the code is thinking, that oh, nothing happened.
But in fact, something happened.
So it's ABA because basically, you've got 15 there.
It goes away.
Then, 15 comes back.
Question?
Can you compare two things and then swap because that would solve this, right?
That's called a double compare-and-swap.
And we'll talk about it in a second.
So the classic way to solve this problem is to use versioning.
So what you do is you pack a version number with each pointer in the same atomically updatable word.
So that when 15 comes back, you've got the pointer.
But you also have a version on that pointer so that the value has to be the same as the version you had and not the value.
What you do is you increment the version number every time the pointer is changed.
So you just do an increment.
But you do the compare-and-swap on the version number and the pointer at the same time.
Now, it turns out that some architectures actually have what's called a double compare-and-swap, which will do compare-and-swap on two distinct locations.
And that simplifies things even more because it means you don't have to pack and make sure that things fit in one word.
You can keep versioning elsewhere.
And there are a whole bunch of other places where you can, in fact, optimize and get even tighter code than you could if you have to pack.
So that's generally the way you solve this.
And, of course, you can see this gets-- as I say, this week has been skull and cross bones lecture.
It's appropriate it comes right after Halloween because really, you do not want to play these games unless you have to.
But you should know about them because you will find times where you need this, or you need to understand somebody's code that they've written in a lock-free way.
Because remember lock-free has the nice property that hey, the operating system swaps something out, it just keeps running nice and jolly if it's correct.
So the other issue is that version numbers may need to be very large.
So if you have a version number, how many bits to you assign to that version number.
Well, 64 bits, that's no problem.
You never run out of 64 bits.
2 to the 64th is a very, very, very big number.
And you'll never run out of 2 to the 64th.
We did that calculation at the beginning of the term.
How big did we say it was?
It's pretty big, right?
It's like this big.
Or is it this big?
My two-year-old is this big.
So anyway, it's pretty big.
So is it bigger than-- no, it's not bigger than the number of particles in the universe, right?
That's 10 to the 80th, which is much bigger than 2 to the 64th.
But it's still a big number.
I think it's like more than there are atoms in the earth or something.
It's still pretty big.
You never get through it if you calculate it.
I think we calculated it and it was hundreds of years or whatever.
Anyway, it's a long time.
Many, many, many years at the very fastest, updating with biggest supercomputers, and the most processors, et cetera.
Never run out of 64 bits.
32 bits.
Four billion.
Maybe you run out.
Maybe you don't.
So that's one of the issues.
You have to say, well, how often do I have to do that.
Really, you only have to worry about this.
You can wraparound.
But you've got to make sure that then you never have a situation where something could be swapped out for long enough that it would come back and bite you because you're coming around and then eating your tail.
And you've got to make sure you wouldn't have things overlap and get a [?
thing.
?]
So that might be a risk you're willing to take.
You can do an analysis and say, what are the odds my system crashes from this reason or from a different reason?
That can be reasonable engineering trade-off.
So there's an alternative to compare-and-swap.
One is the double compare-and-swap.
Another one is some machines have what's called a load-linked, store conditional instruction.
What those are actually is a pair of instructions.
One is load-linked.
When you load-linked, it basically says, let's set a bit, essentially, in that word.
And if that word ever changes when you do store conditional, it will fail.
So even if some other processor changes it to the exact same value, it's keeping track of whether anybody else wrote it using the memory consistency mechanism.
The MSI type protocol that we talked about.
It's using that kind of mechanism to make sure that if it changes.
And so this is actually much more reliable as a mechanism.
x86 does not have load-linked, store conditional.
I'm not sure why.
I don't know if there's a patent on it or what's going on.
But they don't have it.
Final topic is reducers.
So once again, recall the summing problem.
In Cilk++, they have a mechanism called reducer hyperobjects, which lets you do an end run around some of these synchronization problems.
And the basic idea behind it is we actually could code this fairly easily as we talked about last time by just doing divide and conquer on the array.
We add up the first half of the elements, add up the second half of the elements, when they return, add them together.
But the problem is that coding that is a pain to do.
So the hyper object mechanism sort of does that automatically for you.
What you can do is declare result to be an integer, which is going to have the operation add performed on it.
And what happens then is you can just go ahead and add the values up like this.
And basically, what it does is essentially adds things locally and will combine them on an as needed basis.
So you don't actually have to do any synchronization at all.
In the end, you have to get the result by doing a get value.
So let me show you a little bit more what's going on in this situation.
So the first thing here is we're saying result is a summing reducer over int.
The updates are resolved automatically without races or contention because they're basically doing it by keeping local values and copying them.
And then, at the end, you can get the underlying value.
So the way this works is that when you declare the variable, you're declaring it as a reducer over some associative operation, such as addition.
So it only works cleanly if your operation is associative.
And there are a lot of associative operations.
Addition, multiplication, logical, and, list concatenation.
I can concatenate two lists.
So what does associative mean?
I think I have a slide on this in a minute.
It means a times b times c. I can parenthesize it any way I want and get the same answer.
Associative, right?
It's not associative like associative memory or whatever.
So now, the individual strands in the computation can update x as if it were an ordinary non-local variable.
But in fact, it's maintained as a set of different copies called views.
The Cilk++ runtime system coordinates the views and combines them when appropriate.
And when only one view remains, now you can get the actual value.
So for example, you may have a summing reducer where the actual value at this point in time is 89.
But locally, each processor may only see a different value whose sum is 89.
But locally, I could do something like increment this.
And this guy can independently increment his view and has the effect that it increments whatever the total sum is.
And then, the runtime system manages to combine everything at the end to make it be the value when there's no more parallelism associated with that reducer.
So here's the conceptual behavior.
Imagine I have this code.
I set x equal to 0.
I then add 3.
I then increment.
I had 4, increments at 5.
Fa da da da da.
At the end, I get some value, which I don't think I put down.
Another way I could do this is the following.
Let me do exactly the same here but with a local view that I'll call x1.
For this set of operations, let me start a new view that I start out with the identity for addition, which is 0 and add those guys up.
And then, at the end, let me add x1 and x2.
It should give me the same answer if addition is associative.
In particular, these now can operate in parallel with no races.
So if you don't actually look at the intermediate values-- if all I'm doing is updating them, but I'm not actually looking to see what the absolute value of the thing is, I should get the same answer at the end.
The answer to the result is then determinant.
It's not deterministic because it's going to get done in a different way with different memory state.
But it's determinant, meaning the output answer is going to give you the same no matter how it executes, even if the resulting computation is nondeterministic.
So this is a way of encapsulating, if you will, nondeterminism.
And it worked because addition is associative.
It didn't matter which order I did it.
And once again, I could have broken it here instead of there, and I still get the same answer.
It doesn't matter.
So the idea is as these things are work stealing around.
they're accumulating things locally but combining them in a way that maintains the invariant that the final value is going to be the sum.
So there's a lot of other related work where people do reduction types of things, but they're all tied to specific control or data structures.
And the neat thing about the Cilk++ version is that it is not tied to anything.
You can name it anywhere.
You can write recursive programs.
You can update locally your reducer wherever you want, and it figures out exactly how to combine them in order to get your final answer.
So the algebraic framework for this is that we have a monoid, which is a set, an operator, and an identity, where the operator is an associative binary operator.
And the identity is, in fact, the identity.
So here are some examples.
Integers with plus and 0, the real numbers with times and 1, true and false, Booleans with and, where true is the identity, strings over some alphabet with concatenation, where the empty string is the identity.
You can do MAX with minus infinity as the operation, and so forth.
And you can come up with your own.
It's easy to come up with examples of monoids.
So what we do in Cilk++ is we represent a monoid over a set t by a C++ class that inherits from this base class that's predefined for you, which is parameterized using templates with the types.
So the set that we're going to use is, in fact, going to be a type.
And the member function reduced-- this monoid has to have a member function reduced that implements the binary operator times.
And it also has an identity member function.
So we set up the algebraic framework.
So here's, for example, how I could define a sum monoid.
I inherit from the base with int, for example, here.
And I define my reduced function.
And it actually turns out to be important, you always do the right one into the left.
Otherwise, you won't have it be associative.
And then, you have an identity, which gives you in this case a new element, which is 0.
And so you can now define the reducer as so.
You just say Cilk reducer, the sum monoid you've defined and x.
And now, the local view of x can be accessed as x open close parenthesis.
Now, in the example I showed you, you didn't need to do the open close parenthesis.
And the way you get rid of those open close parenthesis is you define a wrapper class.
So it's generally inconvenient to replace every access with x over brown.
That's one issue.
The other thing is accesses aren't safe.
Nothing prevents a programmer from writing x times equals 2, even though the reducer was defined over plus.
And that will screw up the logic of this code if somewhere he's multiplying when, in fact, it's only supposed to be combined with addition.
So the way you solve that is with a wrapper class.
You can do a wrapper class that will protect all of the operations inside and export things that you can just refer to the variable.
And it will actually call that.
For most of what you're doing, you probably don't need to write a wrapper class.
You'll do fine just operating with the extra parentheses.
In addition, there's a whole bunch of commonly use reducers.
Lists, appends, max, min, adds, an output stream, and some strings, and also you can roll your own using things.
One issue with addition is that, in fact, this doesn't preserve-- for floating point addition-- does not preserve the same answer.
And the reason is because floating point numbers are not associative.
If I had a to b and add that to c, I can get something different because of round off error from adding a to the result of b and c. So generally, floating point operations don't give you-- they'll give you something that is close enough for most things, but it's not actually associative.
So you will get different answers.
A quick example.
I'm sorry to run over a little bit here.
I hope people have a couple minutes.
Here's a real world example.
A company had a mechanical assembly represented a tree of assemblies down to individual parts.
A pickup truck has all these parts and all of these extra subparts all the way down to some geometric description of what the part is.
And what they want to do is the so-called collision detection problem, which has nothing to do with colliding autos.
What they're doing is saying, find collisions between the assembly and a target object.
And that object might be something like a half space because they're computing a cutaway.
Tell me all the things that fall within this.
Or maybe, here's an engine compartment, and does the engine fit in with it?
So here's a code that does that.
Basically, it does a recursive walk, where it looks to see whether it's an internal node or a leaf.
If it's a leaf, it says, oh, let me check to see whether the target collides with a particular element of the tree.
And if so, add that object to the end of a list.
So this is the standard a C++ library for putting something on the end of the list.
If it's an internal node, then go through all of the children recursively.
And walk the children recursively.
So basically, you're going to look through the whole tree.
Does it intersect this particular object, x?
So how do we parallelize this?
We can parallelize the recursion.
We turn the 4 loop here into a Cilk 4.
So it goes through all the children at the same time.
They all can do their comparisons completely the same.
Oops, but we have a bug.
What's the bug?
Is it push back?
Yeah.
The push back here.
We have a race here because they're all trying to push on to this output list at the same time.
So we could resolve it with a lock or whatever.
But it turns out it's much better to resolve it with a-- so we could do this, right?
But now, you've got lock contention.
And also, the list ends up getting produced in a jumbled order.
So it turns out if you use a reducer, you declare this to be a reducer with list append.
And what happens then is turns out list concatenation is associative.
If I concatenate a to b, and then concatenate c, that's the same as concatenating a to the concatenation of b and c. And I can concatenate lists in constant time by keeping a pointer to the head and tail of each list.
So if you do that, and that turns out to be one of the built in functions, then, in fact, this code operates perfectly well with no contention and so forth.
And in fact, produces the output in the same order as the original C++.
It runs fast.
And there's a little description of how it works.
The actual protocol is kind of tricky.
And we'll put the paper-- let's make sure we get this paper up on the web.
I think it was there from last year.
So we should be able to find it.
If you're interested in how the details work.
Here's the important thing to know from a programmer point of view.
So typically, the cost-- it turns out the reduce operations you're only calling when there's actually a steal.
It's actually a return from a steal.
But since stealing occurs relatively infrequently the load balance, the number of times you actually do one of these reduce operations is small.
The most of the cost is actually accessing the reducer to do the updates.
And it's never worse than a hash table lookup the way it's implemented.
If the reducer is accessed several times within a region of code, the compiler can optimize the lookups using common subexpression elimination.
And in the common case, then, what happens is it basically has an access cost equal to one additional level of indirection, which is typically an L1 cache hit.
So the overhead of actually updating one of these things is really just like an extra L1 cache hit for most of these things, for most of the time.
If you have the case that you're accessing a reducer several times within the same block of code.
Otherwise, at the very worst, you have to actually do a hash table lookup.
And that tends to be a little bit more like a function call overhead just in terms of order of magnitude.
Sorry for running over.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started.
So today, I'm going to talk a little bit more about performance issues in parallelization.
A little bit more out of the [INAUDIBLE] to what people are doing otherwise.
So normally what we have done so far, is we looked at Cilk.
It provides a very robust and environment for parallelization.
It hides many issues and eliminates many of the problems out there if you find other areas of parallelization that you deal with.
And in last lectures, we looked at things like cache [UNINTELLIGIBLE] algorithims, algorithmic issues, like looking at work and spend.
And in fact, in your projects, you're going to use all these nice concepts to get you a nice parallel learning CorD.
But if you look at a lot of these CorD [UNINTELLIGIBLE] the very people normally for parallelized CorD outside probably Cilk.
And there are a lot of other issues that arise, things like synchronization issues and memory issues.
Today, I think we are going to focus mostly on memory issues.
And we are going to use open OpenMP instead of [INAUDIBLE].
And most of these issues will be affected on Cilk sometimes, too.
But Cilk tries to hide them from you.
There's a layer of abstract.
And it's hard to kind of get to those issues in there.
So we are going to look at this thing called OpenMP.
So today, we are going to address things like granularity of parallelism.
There are so many things that just went out on the page, I guess.
True sharing, false sharing, load balancing issues, the [UNINTELLIGIBLE].
So from the license and keep talking about that we want to be out of all this not dealing with Voodoo parameter.
Today, we actually are dealing mainly with Voodoo.
So I guess this should be basically the Halloween lecture.
So we are all about Voodoo today and see how we can deal with Voodoo issues.
So if you look at a Cilk program, here is a nice simple matrix multiply, seem to be [INAUDIBLE] example these days.
What you can do is you can put a Cilk formula in these two loops and get a nice parallel performance.
However, [UNINTELLIGIBLE] from where how the memory is arranged is up to the Cilk scheduler.
Cilk scheduler is doing some work stealing.
Depending on how the work gets distributed, the process will get worked, it will happen.
Hopefully, everything will go nicely.
And so what that means is it ties the distribution and load balancing issues.
So it's nice if you have access to Cilk, but many other [UNINTELLIGIBLE] you might not be.
And even within Cilk, some of these issues might show up.
So what we are going to do is step one below the Cilk scheduler.
So there's this system called OpenMP.
It's a more simplified model of parallelism.
So what it tries to do is instead of giving this very [UNINTELLIGIBLE] system, it lets you basically direct access to the processors.
So what that means is there's normally what we call a fork-join model.
[UNINTELLIGIBLE] we have with Cilk, basically.
We can do fork into different workers and join.
And more or less, you can actually bind these workers to [UNINTELLIGIBLE] sometimes or make sure that the number-- I'll give you some techniques how to do that as I go on.
So for parallel loops, you can do data parallelism, different [UNINTELLIGIBLE] parallelism you can do something like fork-join.
And you can see a bunch of static or dynamic scheduling policies.
So for example in OpenMP, you can see for this loop that add a pragma to [UNINTELLIGIBLE] in front of this loop and say this is OpenMP.
Parallel loop in here.
Parallel full loop in this one.
And schedule it using static chunk.
I will tell you what exactly that means.
And that gives you direct access to how each of these parts will be run.
So let me get a little bit in detail.
So assume you have [UNINTELLIGIBLE] courses in there, [UNINTELLIGIBLE] processors.
So now in OpenMP, you are basically opening the entire world underneath.
And you have to kind of see what's going on.
And if you say, schedule a static chunk of four, assume you have 16 iterations.
Here are my 16 iterations.
So each of these dots represent a value for i.
So what it says is you take chunks of four and basically send it across it.
So what happens is the first four iterations will go to [UNINTELLIGIBLE] or core zero.
Next four will go to core one, core two and core three.
So you know exactly which iterations run where.
It's a very static thing.
You have full control of what's going on.
Whereas in Cilk, it's up to the scheduler.
So the nice thing here is you can have full control.
But you get enough room to harm yourself if you do things wrong.
So this is a double-edged sword in that sense.
So instead of doing static five you do static two.
You're assigning chunks of size two.
What it will do is it will assign chunks of size two to the four cores.
And then you're not done yet.
And then you start with again core zero and assign chunks of size two.
This is called block cyclic schedule.
And if you do a chunk of size one, it's called a cyclic schedule.
[UNINTELLIGIBLE] cycles just assigning iterations to cores.
OK.
So far so good?
OK.
So I want to do something.
So I have this program.
This is again your run-of-the-mill matrix multiply.
And I ran a sequential single machine, and I got this performance.
Then, I said look, I want to parallelize the outer loop.
So I parallelize this loop.
What should I get?
[UNINTELLIGIBLE] fast or slow.
I want to just check whether you are awake or sleep.
How do [UNINTELLIGIBLE PHRASE] to run slower.
It's not a trick question.
This is just to make sure that actually participate.
How do people think it's run faster?
[INAUDIBLE]
OK. Good.
What do others think?
OK.
They're probably checking their email or something.
OK.
So actually it ran faster.
The source not run on the common cloud machines.
This was a previous generation that I ran.
So [UNINTELLIGIBLE] was seven times faster.
So this was great.
I parallized outer loop.
What happens if I parallize inner loop?
So this test, this i loop, runs parallel.
Here, I launch the [UNINTELLIGIBLE] parallel.
How much people thinks this runs faster?
[INAUDIBLE]
Compared to this one.
How many people thinks this runs slower?
OK.
There's some consistent answers here.
Why do you think it would run slower?
So OK.
It ran slower, so it can improve that.
And that's a little bit slow.
Why is it slow?
[INAUDIBLE]
Exactly.
So what it's doing here, it's basically spawning many, many times in here.
Because every time you have parallelism, you chunkify into the processor.
Here you are getting a lot more smaller chunks inside.
So let's look at how this is basically run.
So normally, you can think about an OpenMP program as you have one sequential thread.
You run the main program.
And then assume you have, in cores, you might have n minus 1 other thread just waiting for work.
And then, when you finally come to the parallel loop, it says, OK. Set up.
What do you want to run on other basic cores.
And release it.
Release these waiting people.
And let them start working on the parallel work.
And also, I will start doing on my own chunk.
So suddenly, when you say parallel four, it releases all other cores to go run that part of the core.
And once it's done, it's will say, OK, I'm done.
I have to wait until everybody is done.
So even if the main guy is done, it has to wait until everybody is finished.
And then, start executing the sequence [UNINTELLIGIBLE].
So this is the gist of how OpenMP program is run.
And if you realize that it all heads here because you have basically make sure all these cases are broken up.
So there's some things that has to be issued.
And there's a delay between these guys can start if everybody has equal work.
Despite not finishing on time because it may take some time for this to start.
And then, it has to also tell this back OK, I am done.
So there's a lot of synchronization going on.
Locks and unlocks.
Here it's called various synchronization here.
And so if this work is small, this synchronization starts dominating.
So what happens is [UNINTELLIGIBLE] fine grain parallelism.
Do a little work in the parallel region, and synchronization will basically start dominating your time.
So how do you take this?
And sometimes when you run something parallel, it might even run slow because the amount of stuff in the parallel region is so small, [UNINTELLIGIBLE] will start dominating.
And that's not a good way.
And also, sometimes you assume.
And you keep increasing the number of cores.
Hopefully, you want to see a nice parallelism increase, but it doesn't, even though you have enough information.
But that means you're running a lot of small chunks, even though you seem to have a lot of parallelism available.
And also, you can make sure the synchronization in the time in the parallel region.
If the parallel regions are on a very short time, this happens.
We saw this effect when we were doing Cilk.
Remember?
When did we see this granularity affecting Cilk?
And what did he do?
When you write Cilk programs.
You write [UNINTELLIGIBLE] programs.
Where did the granularity start showing up on us?
It may not be exactly this because the scheduling is complicated.
OK.
Yes?
The two by two matrix [INAUDIBLE]
Yeah.
Something like two by-- for example, that's the reason we wanted to have a large base case.
Because if you didn't put a large base case, it keeps dividing into smaller and smaller problems.
And if the schedule is smart, it won't be doing exactly this.
But it's always good to have these large granulated chunks at the bottom.
So how to get [UNINTELLIGIBLE] granulated parallelism.
What we need to do is reduce the number of [UNINTELLIGIBLE] equations.
So you want to always try to look for the outer most loop you can get at all the really large independent regions.
So you go look, and not [UNINTELLIGIBLE] thing you want to parallelize.
You go up, up, up, up until the point you can parallelize.
And that's the best way to get good performance.
OK?
So if you really compare these three programs here, again, what you see-- of course, this has no synchronization.
This has n amount of synchronizations.
Here in [UNINTELLIGIBLE] synchronization, that's obviously a lot more synchronization going on.
And that is where this [UNINTELLIGIBLE] comes from.
OK.
So now, I am switching a little bit in here.
I want you guys to look at this program a little bit.
So what am I doing here?
I have two [UNINTELLIGIBLE].
And I am just basically adding matrix B to matrix A. OK?
And then I have another loop test here, adding matrix C to matrix A.
And what am I doing in here?
I am basically going through matrix A in another direction in here
[INAUDIBLE]
It's not really a transpose.
I'm not transposing.
What I'm doing is I'm actually doing a mirror because the C gets mirrored on ix.
It's because [UNINTELLIGIBLE] ix [UNINTELLIGIBLE] the other direction.
OK?
So it's not really a transpose.
So I do a mirror addition.
And then I'm asking for the two outer most loops to be parallel.
So if you run this sequential-- OK, you get about 30 milliseconds, I guess, to run in here.
So that is in [UNINTELLIGIBLE].
But if you're running parallel, what do you get?
Should you get faster or slower?
OK.
Anyone want to take a guess [UNINTELLIGIBLE] Sometimes some of these questions, you might not have enough information to answer.
But it's still good to just take a stand on one direction or another.
How many people think it runs faster?
How many people think it runs slower?
OK.
Some.
Oops.
What happened?
What happened in here?
Can anybody point out why it might be running slower parallely than running sequentially?
[INAUDIBLE]
Yeah.
There's a cache issue.
Watch the possible cache issue in here
[INAUDIBLE]
Yeah.
If you think about, the first equations of, I guess, the first core-- I have some diagram.
I'll show it to you in there.
And here, only the last data elements we'll get for the first iterations because we are going in the other direction.
So if you look at it a little more deeply into what's going on.
Number of instructions seem to be a little higher.
This one I couldn't actually explain why this might be the case in here.
If anybody has an idea, you can say that.
But this was kind of [UNINTELLIGIBLE].
This might be [UNINTELLIGIBLE] the cycles.
Huh.
OK.
I can explain this.
Because this is a sequential run, this is a sum total of a parallel run.
So because of all the overhead that happens because this was running on, I think, an eight core machine.
So you're running eight times of small companies.
There's a lot of overhead that goes around, synchronization, and stuff like that.
So a number of instructions just blows up.
But for each core, you don't have this blow up
[INAUDIBLE] Cilk?
Because does Cilk have different processor affinity, things that open [UNINTELLIGIBLE]?
Because it seems like if the program, the language--
[INAUDIBLE].
Let's see if we can process the affinity information or if not.
It's just pure [UNINTELLIGIBLE]
[INAUDIBLE]
Yeah.
I mean if you like executed locally if you have good cache [UNINTELLIGIBLE] with them.
But if there's no cache [UNINTELLIGIBLE] you might steal something where data might be somewhere else
But you'll still mimic the cache behavior, considerably, except for when you steal.
[INAUDIBLE]
Yeah.
So OK. We don't have a mic in here.
OK.
There's a mic.
There we go.
But if you have two different of these regions, the way the parallelization happens can be different
In Cilk, what happens is the code is mimicking, for the most part, exactly what the C or C++ code would be doing.
And so you get exactly the same cache hits and misses.
Except when you steal, it's like starting over with an empty cache.
OK?
But as long as you have sufficient parallelism, the steals don't occur very often.
And so therefore, you end up getting the same kind of behavior that you would get out of the serial code
Yeah.
But Charles, in this one, because you had to steal everything here, the between here and here, the parallelism would be different
There would be no affinity between those two
No [?
affinity ?]
will be there.
Yeah, exactly.
So in the sequential one, everything that fits in the cache, so that would be affinity because we are not doing parallelism.
And that's what I think happened here.
Because you had no [?
affinity-- ?]
No in a serial code, there's no [?
affinity.
?]
No.
Serial code-- if this fits in the cache, it's then running on one core.
So if it fits in the one core's cache, you're happy
So the issue is-- right-- is if you only access it once, by the time you fill up the cache-- It takes some time to fill up the cache to get them synchronized
So it fits in the one core's cache, it's OK.
Otherwise, it has no affinity in here.
So the key difference in here is, of course, CPI is slow.
We don't know exactly why.
But in [UNINTELLIGIBLE].
So what you find is that there's a huge amount of cache in [UNINTELLIGIBLE] going on.
So that should give you a feeling of what's going on.
So let's look at what might happen.
So I'm showing this matches [UNINTELLIGIBLE] last year on-- what we had were Cagnode machines that were basically code to quad processor.
So we had eight codes in here.
And I put them-- you don't have to now look at this table.
I put them in the slides so you can look at it later.
And so this is the last year's machine.
And of course, this year's machine is different.
We have two six core processors in here.
So this is what we [UNINTELLIGIBLE] this year.
[UNINTELLIGIBLE] OK. And so right now, I'm showing numbers for this one.
And later, I will show what happened in the [UNINTELLIGIBLE].
So if you look at a cache-- so what happened is each of the data items in the cache can be in multiple states.
This is called MSI protocol here.
What that means is the item might be modified.
If it is modified, it can be only in one cache.
If anybody else wants to touch it, it has to get it out of the modified state.
Or it can be sharing.
Sharing means it's reading.
So that means that item is read by multiple people.
And that can have multiple covers.
So sharing items can be in multiple places.
However if you're modifying, [UNINTELLIGIBLE] items in everybody else.
So that means if I modify something, it can only be in mine.
If other people had that data, I had to go in and validate this.
So if you modify this, I had to go in and validate it.
So that's a sharing state.
That means I'm [UNINTELLIGIBLE] everybody [UNINTELLIGIBLE] read this.
But if I ever want to change that, I have to go in and validate this one.
So what that means is when I start writing, I am validating it from everybody.
So even if everybody kept a copy and they start modifying, I had to get my own copy.
And everybody else will invalidate.
And then, if somebody else wanted to read-- for example, if this guy wants to read-- basically, this has to make this a sharing and back to sharing.
That means I have to get the value 13, propogate it, and this becomes sharing again.
OK. Did you get that?
What's going on in here?
In the cache?
So reads, everybody can keep a copy if they want.
Write-- only one guy can keep a copy.
So what happens then is true sharing.
So you have these two different cores.
So I want to read something.
So I get it from outside probably on main memory, and I put it in my cache in here.
And then, the next guy wants to write the same thing.
Assume I'm writing that.
And once I want to write that, I can keep this copy I had invalidated from here and get a copy here.
And then, if this way, I want to write it again, I have to basically invalidate it from here and get a copy.
If I'm reading, both of us can keep a copy and just kind of keep bouncing back and forth, back and forth.
And so if you bounce too many times, you get all of these invalidations.
So the fact I looked at that I have invalidations basically tells me something like this is going on.
So what's happening in this program?
When I parallelize this four loop, my four cores-- basically since I am doing here [UNINTELLIGIBLE]-- are going to get this nice distribution of data into the caches.
Assume it fits in cache.
OK.
So all this data nicely fits into cache, and now I'm pretty happy when I run this one because I got this data into the cache.
And I write it.
But the minute I go in here, basically this data has to [UNINTELLIGIBLE].
OK, because I am going this is n minus i in here so this data has a flip route.
And by doing this, basically, I incur this huge amount of [UNINTELLIGIBLE], and it slows down.
OK?
So that's why it didn't work well.
So what can you do?
When you have these read, write and write, write conflicts in here.
And you have to actually move the data across in here.
And what you can do is look for this true sharing.
You can look at the [UNINTELLIGIBLE] and see if we have excessive [UNINTELLIGIBLE], we have a problem.
And how do we eliminate that?
You want to make the sharing minimal.
If you want to get some data into a cache, you want to try to keep it there as much as possible.
And if you're using, you'd want to try to align everything across.
So even across different regions, it'll use the same kind of things.
And/or enforce some kind of [UNINTELLIGIBLE] technique to keep the data alive.
So there are a lot of techniques in there, but true sharing can be an interesting problem.
So in here, simple change, yes.
You're, basically, instead of changing A, you change C. So you write A the same way.
But now what I have done is I am doing the mirror by changing the axis to C, is to [UNINTELLIGIBLE] is the same as this axis.
So these two are the same thing.
And the minute I do that, voila!
I get good speed up.
Because if you look at that, my inundations has gone down.
My L1 cache [UNINTELLIGIBLE] has now really, really gone down.
I'm not doing anything.
And of course, I am doing more instructions here than this one because-- I think, the difference between instruction here and here is because a lot of times synchronization operations are dynamic because in the [UNINTELLIGIBLE] miscounted as the instructions, you are busy waiting in there.
So this number is not really a constant number.
OK, question
Not a question.
So another thing one could do here is do loop fusion
Yes.
Yes.
Here is a nice way of putting both of the loops into one and do loop fusion.
And that works.
In this case, you can do that
So loop fusion is where you take two loops, and you convert it into one loop.
So in this case, you could have just written one nest, which has two things going on inside.
And then you would save all the loop overhead and the scheduling overhead.
So rather than doing it twice, you actually have reduced the overhead to just the parallelism of the one loop.
So if you look at that, you'll realize that you could somehow make it just be a single nest with two statements in there, rather than one
So basically, instead of [UNINTELLIGIBLE] entire thing and move plus C into here, basically.
And I could have just done it in one loop nest.
That's what loop fusion would do here.
So we can actually [UNINTELLIGIBLE] much nicer in here.
But just for example purposes, so now I really reduced this one and got that.
So this is great.
Cagnodes really showed this classic problem in the computer.
And so I'm like, OK. Now we have new machines.
Let's try it and see what happens.
What does this show?
This is your nice cloud machines we've got now.
I have no slow down.
I was really disappointed because beforehand, I had this for sharing going on in here and had a really big slow down.
But this one, in fact, the difference is very small.
And when you look at any kind of performance counters, they are pretty comparable.
There's nothing much going on here.
So what do you think is going on in this new architecture now?
Why this might be?
[INAUDIBLE]
That's an interesting observation, but also we have-- yes, core seven basically is two by two in the die.
But we also have two different processors.
So that's there, too.
So in some sense, when you get our two-way process [UNINTELLIGIBLE] So that's there.
That might help.
That's an interesting observation.
What else might be going on here?
Why do you think they manage to get this one?
What might be another answer?
What can hide these kind of delays that can happen?
Load delays, and cache misses, and stuff like that.
What techniques and hardware can hide those?
Just [UNINTELLIGIBLE] a speculation.
Prefetching.
So most hardware has an internal mechanism.
When you start fetching data, you say, aha!
I see a pattern.
I know you want to get this thing.
Let me go forward and bring more data, thinking you are going to follow that pattern.
OK. All or most of the [UNINTELLIGIBLE] for, I think, have [UNINTELLIGIBLE] even a Pentium had something for prefetching going on.
But most of the time, what happens is the prefetching engine can't keep up.
If you are getting there, it's [UNINTELLIGIBLE] a little bit further.
You are going to catch up, and you're going to start because you have more [UNINTELLIGIBLE] here.
I think [UNINTELLIGIBLE] has a really, really good prefetcher.
And then, we saw it in our architecture slides, too.
That a lot of things that used to happen before is gone.
So this is really good.
What that means is a lot of weird stuff that's going on [UNINTELLIGIBLE] making them disappear.
So these kind of problems don't show up.
So that's the nice story.
The other part is, OK. Now if you start really tweaking your programs to one architecture, you wait a generation.
And then now, we have done either the tweaking-- the best case, tweaking has no impact, and it's not affecting anything.
In most of the time, worst case, tweaking actually slows down the program because you are trying to do something complicated.
That's just not needed anymore.
So even though these kind of things showed up in [UNINTELLIGIBLE] architecture, it's not an issue.
But if you go to many of the smaller architectures that have that don't have that much of the very popular prefetchers, this kind of issue you would see.
So for example, if you go to a cell phone [UNINTELLIGIBLE], you would probably see these kind of issues happening.
Any questions here so far?
So that's the good news.
You guys don't have to worry about it too much.
But at least it's good to know the technique because you'll see it in other architectures.
So now, I want to switch a little bit into looking at programs that don't have what we call data parallelism.
That means you can start and say, [UNINTELLIGIBLE] parallels.
Everybody get the different chunk and run.
And we are going a little bit more deeply into looking at programs that are a little bit different.
So I wanted to come up with this little representation to represent the program.
And so if you think about iteration space-- actually before you, I'll go down to dependence.
I'll also do a little bit of load balance.
So here's a loop that in my iterations-- the first one I transformed zero to eight.
But J only runs from one to eight.
So each I, I have less and less amount of J iterations, basically.
OK?
It's a triangular loop.
OK?
OK.
So this is the way to represent iteration space, so I will represent data and get back to this again.
So if you look at a data space, you can assume data iteration space could be this funky, triangular, hyperplane type of thing.
Whereas data is mostly [?
rectangulineum ?
], multi-dimensional rectangle.
So for example, if I have [UNINTELLIGIBLE] and it's a one-dimensional one, this is basically a two-dimensional data.
And you can have three-dimensional cubes and stuff like that.
You can represent data like that.
So this is a way to nicely represent.
And when you start thinking about it, we can look at what's going on.
OK?
So now you have this loop again.
So here's the basic [UNINTELLIGIBLE] iterations.
And here's the data.
Assume this is both A and B.
There will be another one for matrix [UNINTELLIGIBLE] B.
One data into each iteration is going to touch.
So these are the data that need to get touched, and here's the iterations you are going to run.
So we can say OpenMP parallel four.
So what happens when you do parallel four?
So I am going to get parallel.
And so core one gets this one, another core, another core, another core get these iterations running.
So what happens if you do this one?
Do you get really good performance?
Why?
[INAUDIBLE]
It's not balanced.
The load is not balanced in here.
So basically if you run sequential and if you run block distribution, I get about 3x performance in here.
So if I look at closely, here is the number of iterations given to each core.
The first core gets almost nothing, and the last guy gets a lot of work.
Here's where something like the Cilk runtime can come into play because with Cilk runtime, basically, this guy will finish the [UNINTELLIGIBLE] start stealing from somebody else.
And so it would be done nicely.
But whereas if you do a static schedule, you are in this big bind.
You don't have too many things going on.
And basically, this is what we call load imbalance.
So what you can do is figure out a complicated partitioning so you can statically partition this out.
Or you can do something like the dynamic scheduler like the [UNINTELLIGIBLE] scheduler for a solution.
So how to detect load imbalance?
Basically, what you might want to do is for each of the different sections you are running, you want to look at the time mistakes.
And in the [UNINTELLIGIBLE] axis varying, huge varying, that means there's a load imbalance going on.
So you might want to check and make sure each of the parallel regions time is taking.
And that gives you this view.
How to eliminate load imbalance or the use of dynamic scheduler that will deal with that.
Or you can do a different distribution statically.
That will not partition in this large block.
So let me show you a static part because we have already learned the dynamic part before.
So now instead of doing that, we do a cyclic distribution.
We use a static one.
That means if you have a lot more than and a little bit better distribution so what happens to the processor?
Zero gets this one and this one.
One gets this one and this one.
So on and so forth.
So that would be a little bit between balancing there.
But if you have enough of cyclic, the imbalance would be much lower.
So should we run faster?
So here's the iterations each guy gets in here.
This looks very balanced because I had a lot more iterations than this eight one.
This is not that balanced here because this guy gets a lot more than the first one.
The first one gets six.
And the second and last one gets a lot more.
Uh oh.
What do you think is happening here now?
I ran again slower.
See I guess the people in class last year had things worse because they had this old processor that did all these crazy things on them.
and you guys get the fast one that doesn't do that.
So why do you think cyclic distribution is running a lot slower?
What might be going?
[INAUDIBLE]
Spoiling [UNINTELLIGIBLE] it's not that much because if you don't run this and synchronize, what you do is you run the same amount of tread and say, now, instead of running continuously, you run jumping all iterations.
You should run zero and nine or whatever jump over iterations.
Why do you think?
[INAUDIBLE]
Yeah, there's a cache issue.
All this time and the question is not sure, it's probably a cache issue.
What type of cache issue do you think is going on?
[INAUDIBLE]
Yeah.
[UNINTELLIGIBLE].
But let me show you what happens.
So you get off then.
OK, so if you look at-- the data is here so let's look at what happens.
So this is running a [UNINTELLIGIBLE] lower.
It's showing a lot more instructions, but instruction doesn't tell you too much because a lot of them might be missing synchronization costs in here.
So instruction is not that illuminating here.
The big illumination here is this one again.
Invalidations.
I have a huge amount of invalidations going on.
So here is a case of false sharing.
So what happens is now things next to each other, you want to multiply different processors.
We're not touching the same data.
Everybody's looking at somebody else's data.
So what happens is assume I want to write this data item.
I like that data item.
But I get the entire cache line because when I ask for that, I get my synchronization by the cache line.
I get this entire cache line coming in here into this one.
And the next guys [UNINTELLIGIBLE] at me.
This core won't write this data because instead of blocks, I basically give each strips.
There's a lot of overlap between strips.
So this guy says not to write this one, I had to get the entire cache line going back here.
And so if you want to write that again, I had to get the entire cache line going back even though we are writing different data.
Because we are sharing cache lines in here.
This thinking was in back and forth, back and forth, back and forth.
I have a lot of cache [UNINTELLIGIBLE].
Things are really shot.
OK?
And so what happens here is if you look at the cache lines-- there's my animation.
So cache lines basically mess this all up.
You can see that really carefully.
What happens is between these lines, there would be some overlap of cache lines.
And this overlap in cache lines keeps bouncing back and forth, back and forth in here.
And so what happens is basically cache lines are bigger than the data size, or there's overlap in here, and the cache line is shared when the data is not shared.
And so how to detect false sharing in too many conflicts.
You assume this is a nice parallelism, but suddenly, you don't have a speed up, and you have a lot of conflicts here, even though there isn't something to be sharing.
And how to eliminate false sharing.
Make data used by each contiguous in memory.
That's a good way of doing that.
Or pad at the end.
So these kind of at the corners, there's not going to be any overlapping.
So in here, one thing you can do is, you can measure each thing that each of the cores get.
We can make [UNINTELLIGIBLE].
But before what happens was a core used to get this line and this line.
There are different places in memory.
But you can make these two contiguous in memory by basically now, instead of having a two-dimensional array, you made that a three-dimensional or disarrays
Can you say that again?
So before you what just happened was each of them were going to get this line and this line, each core.
All these lines that were in different parts of the memory.
In here, each would get only two lines.
But they're in a different place.
So if you have more cyclic, you'll get a lot more lines or lower memory.
So what we can do is we can arrange the cache.
So if you think about this, you can think the cache, now the data, is instead of two-dimensions is three-dimensional data.
One dimension is this cyclic part in here.
So we can do that.
And then, you can change any way that the cyclic part, the one that I got this line and this line, now become contiguous.
So you think about data as a two-dimension.
You think about it as a cube.
And you kind of change the cube for the inner dimension to be the one that's contiguous.
So you can do data [UNINTELLIGIBLE] transformation and get there.
So now what happens is the role of core zero just gets contiguous in memory.
And core one gets contiguous in memory.
So if you're trying to make it contiguous, that's great.
So between padding and making things contiguous, you can get good performance.
And if you do data transformation, voila!
My invalidations just went down drastically.
I again have a nice load balancing here.
Invalidations went down drastically.
That means my [UNINTELLIGIBLE] increased a little bit and I get really nice speed up.
So here are the kind of crazy things you are to do if you are doing things like algorithms that are not cache obvious.
And if you are doing directly parallizing yourself without letting a nice [UNINTELLIGIBLE] time to help you.
Something like a [UNINTELLIGIBLE] assistant.
So I'm just going to summarize this because this is important.
We looked at a bunch of cache issues.
We looked at cold missiles, capacity missiles, and conflict missiles before.
And today, here are some examples of true sharing missiles.
What happened was I am actually really using data, but I set up my parallelism in such a way that between different executions, my data has to move across.
[UNINTELLIGIBLE] So I am truly sharing data, but the data has to go to somebody else's cache.
So I've got a lot of [UNINTELLIGIBLE] violations here.
More into this one is more like false sharing, where you assume there's no sharing, nice parallelism, everything, except the program runs very slow.
And that can be because of false sharing.
So we just kind of touch on these two topics.
OK?
So let me switch gears a little bit about dependences.
We touched on the dependences a little bit.
And these are two fine programs that are not completely parallel.
So normally, what happens is a true dependence means that I'm writing and reading [UNINTELLIGIBLE] other way out.
And if two guys are both fighting, then the order has to maintiain us out would be dependence.
And did our dependence even loop, because these are single items.
So if you have an error here, this is becoming a lot more complicated.
Because there's no simple thing in here.
Because it's not just using the same iteration.
You might be using data from different iterations.
So what happens is there's a dynamic instance of iterations.
So one iteration writes the data, and somebody else might be reading the data.
And that is basically the order we have to [UNINTELLIGIBLE].
Let me give you an example.
This kind of demonstrates what's going on.
OK?
And when you edit, you say look, this is where you [UNINTELLIGIBLE] complicated.
So in order to give you and example, let me look at this program.
I have a simple program here.
Ai equals Ai plus one.
My iterations-- I'm running five iterations in here.
So this is my iteration space.
I have a large array, so this is my data space.
And now, I keep running this program.
So what happens is this is time going down in here.
So the first situation basically, I first read and then write.
Same in the second iteration, I read and write.
Third iteration read and write.
Fourth iteration, read and write.
Do you see how this is going on these four situations?
Second iteration, third iteration, fourth iteration, [UNINTELLIGIBLE].
OK.
So what happens is first iteration read this value is zero.
And write the value as zero in the menu writing.
Second iteration A1, A1, A2, A2, A3, A3.
So now, when this is writing, that's a dependence between these two.
You see the true and entire output dependence between these two.
What type of dependence do we have?
[UNINTELLIGIBLE] dependence.
True dependence is what?
What to what?
What's the first thing that occurs?
What's the next thing that occurs?
Anybody want to answer?
[INAUDIBLE]
Write to read.
So you have the first thing has to be write to read.
Watch this.
This is a read to write.
So what type of dependence is this?
This is anti-dependent.
So here is ante-dependence in here.
But the nice thing about that is this dependent didn't cross the iteration boundary.
So these black lines are my iteration boundaries.
So these are for situations that [UNINTELLIGIBLE].
So there's no iteration crossing in here.
You can kind of [UNINTELLIGIBLE] it using each of these iterations and my dependencies within the very same iteration.
So the same iteration I have dependency [UNINTELLIGIBLE].
OK?
This is a simpler case.
So let's look at something a little bit more complicated.
So I have Ai plus 1 equals Ai plus 1.
So what happens is first I am reading Ai.
And then, I am writing Ai plus 1 in the same iteration.
The next iteration, I am reading now Ai [UNINTELLIGIBLE] this is A0 and 1.
This is A is 1.
[UNINTELLIGIBLE] I am writing Ai plus 1.
I am writing 2.
So I have a dependence like this now.
What type of dependence is this?
This is a true dependence because I am writing.
And this is actually reading what what it is writing.
So does this look parallel?
No.
Because what happens is if you look at each iteration depends on the previous iteration.
So you have to actually have this dependence going back and forth, back and forth in here.
So let's look at a couple more other things.
So here is Ai equals Ai plus 2.
So I am basically reading Ai plus 2.
So I am reading this one.
I am writing this one.
Reading this one.
Writing this one.
Here is my dependence that's in here.
You see the two are anti in here.
This is anti-dependence because I am going from a reading to a write in here.
Can this loop be parallel?
Can this loop run parallel?
[INAUDIBLE]
So can every iteration run parallel?
There could be basically.
No because what happens is if you look at that, there's a dependence that goes like this.
And of course, there are two chains.
So if you are interested, you can run at least two-way parallelism.
You can run one chain parallel to another chain when you do get that much parallelism.
How about this one?
2i and 2i plus 1.
[UNINTELLIGIBLE] Is there independence in here?
Nope because one is-- you are reading all the elements and even writing elements [UNINTELLIGIBLE] dependence.
So you can have a missing parallel.
OK?
So this is the kind of interesting thing that is going on.
So next, I want to look at something a little bit more complicated.
So let's look at this.
So here's a classic algorithm called successive over relaxation.
So it kind of simulates a lot of times things like heat flow through a plane.
So the idea there is-- let me illustrate what he does.
So assume you have a big metal sheet.
And you put some kind of heat source in one place.
And after sometime, it all reaches a steady state.
The other side might be cold.
And you want to know part of the sheet's temperature.
Because temperature can leak out.
And there are more things like you have a heat source and others that [UNINTELLIGIBLE] to work a glass of water or some kind of a sink.
So what is the heat distribution?
So one way to compare that this is basically the same [UNINTELLIGIBLE].
The heat value here is basically the average around all these other values right now.
Because if something is too hot, the heat is going to propagate something that is too cold.
The heat is going to propagate because it kind of has to be average around that.
Then, you take the average in here.
So what it's doing is calculating the average in here.
And then, you have to do it many, many times.
So if you have a heat source, at that point, it would be very hard.
And then, it will start propagating slowly and kind of propagate down.
And the cold side in this way or after running many times, it basically stabilizes.
And at that point, you have the kind of heat distribution that we [UNINTELLIGIBLE] have.
This is the kind of calculation you do.
So this is the calculation.
So what you're doing is calculating this.
You are creating this, this, this, and this and updating that.
And then, you do it for t time stamps.
So you just go around doing each of these things first and doing it for t time stamps.
OK?
So we would like to run this parallel.
So here's my basically data space.
There's my data items.
So here's my array, two-dimensional array.
So this is how I'm trying to update.
I'm reading all this file.
So here's my iteration space.
So what I have looked at this.
I don't want to-- it's hard to give you a 3D diagram.
I don't have a 3D projector.
So what I'm showing here is three-dimension here.
So this is the previous iteration, first iteration.
So if I still go tij.
So you go through t, and then you go through i in here, and then, when you're done, you go to the [UNINTELLIGIBLE] iteration and you go this way.
So here's how you would iterate.
So you run this one, this one, this one, this one, this one, this one, this one, this one, this one.
And then increase t by 1, and go like this.
And right now, we are here.
We are trying to update this one.
That's what we are trying to do.
And that means we are already finished up to this point.
All these points are finished up.
Now, what we have to do is figure out when I'm reading, who actually wrote this value.
OK?
First of all, let's figure out which iterations might be able to write this value.
So if you look at this value, this relationship in between here.
This one, basically, is ij.
And this is ij, ij, ij.
These three iterations can write this one.
So and these iterations can write this one.
Let me go to this one.
This is a pretty darn complicated [UNINTELLIGIBLE].
So what that means is in this one, this one already wrote something.
This is what I'm reading in here.
This one already wrote something.
This is what I'm reading here.
This iteration wrote something.
I read it here.
OK. Everybody following so far?
How about this, guys?
Who wrote the value I am reading in these iterations?
In this one, I haven't reached there yet.
So who has written that?
So I assume this is t equals 1.
[UNINTELLIGIBLE] somebody has to write those things.
So what that means is this also wrote all of those values because I have done those iterations.
But the interesting thing is some of these values got overwritten.
This value got overwritten , this value got overwritten.
So these two values disappear.
This value got overwritten by this guy.
This value got overwritten by this guy.
OK.
But we haven't overwritten this value, this value, and this value yet.
This one, basically, I've just updated.
But I [UNINTELLIGIBLE] this one.
Do you see this?
Is everybody following me?
Once again, sir.
I got lost.
So what are [INAUDIBLE]
So what happens is I am trying to update in this iteration because this array get rid of multiple times.
But in each iteration, you are only doing one update.
So I am trying to read and write in here.
So I need to read all of these five elements in this iteration.
So I want to figure out who wrote that.
OK?
This one can be written by this guy and this iteration.
Could this iteration write its value in here?
OK?
[UNINTELLIGIBLE] This iteration write because we see it's writing ij.
I mean my diagram is not that great because I have three in here and five in here.
So just bear with me on that.
So assume I am writing ij in here.
So my iterations go from 1 to n, but my data goes from 0 to n plus 1, basically.
1 to n minus 1 iterations.
0 to n plus 1 data.
So data is bigger than iteration space because of [UNINTELLIGIBLE].
So what happens is when I'm in this iteration, we'll say this is 1 2 iteration.
I will write this value.
This iteration will also write this value.
OK?
You see that?
All of these iterations are the same.
This iteration we will also write this value.
But right now, who is the last guy who wrote it?
The last guy that wrote it is this guy.
Because this iteration wrote it, and after that, it got ordered in this one.
But this one hadn't occurred yet, so it hadn't been ordered by this guy.
So the last guy who wrote it was this one.
So that's why I had to eliminate this.
But this data value-- I haven't executed this iteration yet.
So nobody had written this one in this time stamp.
So it has to be from the previous time stamp.
So I read two values from the current time stamp, three values from the previous time stamp.
These three values have to come from the previous time stamp.
These two values that come from the current time stamp.
You see that?
OK. Good.
So what that means is because dependence means-- OK.
This line, this dark, red line.
See.
I am reading a value in a current iteration that was written by this iteration.
So that means I have no dependence between these two iterations.
OK.
This line, this dark, red line.
I am reading a value written by this iteration.
So I have a dependency in here.
This line means I have a dependence between this iteration to the current one.
This line means I have dependence between this iteration and the current one.
This line means I have dependence between this iteration and the current one.
You see that?
OK.
So now, I want to see how we can parallelize this group.
So what can I do?
So I look at all this dependence.
At this point, I don't have to think about all this where who wrote what.
I can say this is dependence.
In order to do this equation, all these iterations have to be done because I am losing the values produced by them.
So these have to be finished before I can patch that.
So the parallelism means I tried to do things in parallel.
So can we parallelize this loop?
Can we run each time stamp separately?
No because I am using these three values from the previous time stamp.
So I can't run this time stamp, B equals 1, until B equals 0 is done.
Or B plus 2 [UNINTELLIGIBLE] B plus 1 is done.
OK?
So I can't parallelize this loop.
OK. Can I parallelize this loop?
Why?
Will dependence stop me from parallelizing this one?
So I'm looking at i.
This is my i dimension.
How many lines, at least, tell me.
How many dependencies are going to stop me from doing that?
OK. Good.
I have [UNINTELLIGIBLE].
Somebody says three.
Somebody says one.
OK. Let's get a vote.
How many people think it's three?
OK.
There's one vote for three.
How many people think it's three?
How many people think it's one?
Wait a minute.
One vote for three and two votes for one?
OK. Where's the rest?
For two?
For 0?
Can't be 0 if the 0 is parallel.
OK.
So we'll start parallelizing.
OK.
So what happens in here?
Right now, this is actually one.
This one.
Because these things don't participate because this has already happened.
When you go to ij iterations, these are already done.
So you're going from t. So you're looking at the current iterations because you're ending in two loops.
So the t is done.
So these all are already done when you go try to parallelize sides.
So I don't have to worry about these three.
In here because actually I'm losing t of something here, I am in trouble.
So when you go look at this one, I have this one.
So every dimension has a dependence in here.
So I can't run it in parallel.
So does this mean that there's no parallelism?
Who think there's no parallelism?
Who thinks there is?
Oh, somebody thinks there's no parallelism.
Who thinks there's parallelism?
OK. More people think there's parallelism.
Let's see what we can do.
Question?
Do you really think [INAUDIBLE] I'm trying to figure out how to word this.
Do you really want to have dependence on the same concept?
[INAUDIBLE]?
Yeah.
I mean you can do-- this is the way this SOR is sitting so there's a dependence between time stamp.
There's another SOR.
What they do is kind of a red, black.
So when you calculate the next time stamp, you calculate it right and complete the new array.
So there's no dependence.
So that's a different algorithm.
This algorithm, basically, uses sum value from [UNINTELLIGIBLE] because the value-- the algorithm you're talking-- you already created the other copies.
You had two copies.
You're bouncing back and forth.
Nice.
No real problem in here.
But then you had to have twice the amount of storage.
Here, you are updating in.
And since this is kind of running enough iterations until it converges, it doesn't seem to matter that the [UNINTELLIGIBLE PHRASE].
OK.
So we cannot find a loop, what we call doall loop.
The doall loop means there's no loop carried dependences.
It's fully parallel.
This is the best case.
So what happens is when you get there, everybody can run parallel.
And when you're done, you can stop and then do that.
So this is the doall loop.
Of course, there's no doall loop.
We can look at every dimension.
We had some kind of dependence.
So there's another choice, what we call doacross loop.
What that means is we have some loop carried dependence.
There's something I have to use for the previous iteration.
But it's only one thing.
I have a lot of other things I can run around that only I just have to wait one thing.
One is done.
I can just keep running.
And if I calculate and send this one early, then I can do my other calculations later.
This is not that great.
If you look at the difference here.
This definitely has very little overhead in here.
This can run slow.
And of course, this thing gets produced very late.
It's [?
almost ?]
sequential.
So I hope you can just-- it the other guy wants something, I can immediately send it very early.
And then I can run there.
So you can get some kind of doacross patterns in here.
So if you want to do this one-- this is a little bit crazy in here.
But they'll do it in here.
And so what first we are to do is you are to say, OK. Look.
I'm running this loop, the i loop in parallel.
But I have to exchange some data.
Before I want to run this one, I have to basically get the previous i value produced.
And when it's done, I can say the next guy can use it.
So this is a very complicated one.
I don't want you to understand it too well.
So the reason I put it is to show that OK, if you want to spend a week trying to really call this up and understand and make sure that it works OK.
So you can do things like that.
OK?
Aha.
So this is the true voodooness
So in Cilk, if you do this with divide and conquer, you can make it be what I called in the Tableau construction.
Each layer here is basically constructing a Tableau.
And so if you do it with divide and conquer, you can do it with a very simple recursive code.
But you can also do it with a loop that goes diagonally
[INTERPOSING VOICES]
Yes.
I'm going to get that.
That's next
Sorry
That's OK.
So the reason that I'm showing that is because this class is not just about how to make the cores exactly run faster.
Think about algorithmic issues and stuff like that.
So sometimes, when you look at a problem, it looks crazy.
And there might be some changes you can do that you can get to run things in parallel.
So I'm actually doing not diagonal.
I'm actually doing something very simple.
So what I have done here is I have all these dependences in here.
OK?
So the problem here is I can't find a single [UNINTELLIGIBLE] that basically has no crossing.
But if you look at this [UNINTELLIGIBLE] diagonal here.
What you see is, in fact, there's nothing that crosses the diagonal.
OK?
So this one basically doesn't depend on this one or this one.
It only depends on the previous one.
So I can run everything in the diagonal parallel in here in this one.
So of course, I can't write anything [UNINTELLIGIBLE] in here, but there's a cute trick you can do.
What you can do is you can take iteration space and skew it.
So what I have done is now instead off the same thing, instead of this being a square, now I skewed it a little bit.
OK?
So what that means is when I'm running first i, I basically don't run any here.
Then, I run this one and this iteration here.
So what I have done is I have kind of moved my iteration space around.
Do you see how this might be?
So now, the interesting thing is when I skew, if I look at this line, I can parallelize in this one because all the dependences come from the previous iteration.
Am I right?
[UNINTELLIGIBLE] Yeah.
I skewed it.
Yes, everything in here, these ones are parallel.
OK?
And any dependence comes from the previous iteration.
There's no current iteration in here.
Everything in this one is parallel.
So I can parallelize this.
So this one doesn't depend on this one or this one.
So this is all parallel.
This is a little bit more complicated.
So if you're interested to go deep, just go stare at the slides.
I have the slides out there to understand how that happens.
So if you think about what I'm running here in parallel is the one basically this diagonal in here.
So what happens is if you run this, this, and this parallel, there's no dependence.
I don't need this one or this one to run this one.
So I can run this, this, this, this, all this diagonal in parallel.
But the trouble with just the diagonal is I don't have a place in here to say [UNINTELLIGIBLE] for a diagonal.
So I basically skewed it and then made a diagonal into one loop.
So then, now what happens is basically j loop I can run parallel.
This one.
So I can do it four [UNINTELLIGIBLE] four.
OK?
So here's something you found a problem that has no nice parallelism.
But you realize there's kind of a what you call a wavefront going on here.
Wave going on here.
So not the given dimension, but there's another dimension that you can parallel.
So you kind of skewed your space to get that nice [UNINTELLIGIBLE] line.
And you run parallel.
So that's all I have for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
We're going to get started now.
For those of you who don't know me, I'm Josh Slocum, one of your TAs.
And today's lecture is going to be a primer on ray tracing to help you get started on the final project.
So first, we're going to start with a little background.
Tell you about ray tracing.
Ray tracing is essentially a physics simulation.
You simulate the way photons bounce around and interact with materials.
And from that, you generate an image of a scene.
It's extremely parallel because you can trace a whole bunch of photons bouncing around at the same time.
And it has a huge capacity for photorealism.
So like this sample seen here, you have all sorts of reflections and light working in ways that you don't see in a normal rendering program.
The final project, as you know if you've read the handout, is going to be groups of two or three people each.
And you'll be given a working ray tracer program.
A very slow one, but one that works.
And your objective is to make it run as fast as possible.
You can do pretty much anything you want with a few restrictions we'll talk about later.
You can make algorithmic improvements, parallelize it, change the way the rendering works slightly.
All sorts of things.
Pretty much anything you can think of.
The only deliverables for the final project are the preliminary report due next week, in which you'll have to show us that you've profiled the project, tell us what hot spots you've identified, and ideas you have for speeding up the ray tracer, as well as any actual work you've already done.
The final project comes with two built-in scenes that can be ray traced.
Scene one, this one here, is rendered much faster than scene two.
And you're going to want to work on that one first.
And then, there's scene two, which is slightly different.
It has water and one less sphere.
These are the restrictions on what you can do for the final project.
The purpose of these restrictions is basically just to make sure that everyone's ray tracer is doing the same thing.
So you're all working on the same problem, and you're not achieving speed ups by simplifying out some functionality of the program.
Does anyone have any questions about the final project before I move on?
All right.
Go ahead
[INAUDIBLE] we're going to do a third scene similar to the first two, right?
Oh.
Yeah.
In order to make sure that you're following these rules, we're going to test your ray tracer on one or more other scenes and make sure that they render properly.
But these are the two scenes that you will be benchmarked on for your performance grade.
All right.
Next, we're going to go over how ray tracing works and the big concepts used in the ray tracer you'll be working on.
So the basic idea behind the rendering is you're going to construct a geometric model of the scene you want to render, and so it will be made of polygons, or spheres, or planes.
And then within that model, you insert a grid of pixels.
They'll form the image you're going to render.
And then, you just project the scene onto the grid of pixels, and you have some image.
The way you do this with ray tracing is you use a method called ray casting where you take some camera and cast rays through each pixel onto the scene.
And then, whatever color object the ray hits is the color that that pixel gets shaded in the scene.
This method of tracing rays is called backwards ray tracing, because you're going in opposite direction that photons go in when you're actually observing something.
So if this was an actual scene, you'd have light bouncing here and into you're eye.
Whereas what you're doing is you're tracing backwards from the eye to the object.
Ray casting-- this image here is a ray image-- is very primitive.
It doesn't really get you much.
So what you can do is you can improve your physics simulation by adding a whole bunch more things on top of it, like shadows, and reflections, and global illumination.
All sorts of fun stuff.
So if you want to start simulating shadows, and reflections, and defraction, you need to start recursively casting rays, which is called ray tracing.
So when your ray bounces off of an object, you can cast a reflection ray at the same angle mirrored from the perpendicular of the object and see what color object that ray hits, and then, shade this object with that color.
You can also cast a refracted ray, and use Snell's a lot to tell which direction it goes in and do the same thing.
And then, you can cast a shadow ray at all the light sources in the scene.
And if the shadow ray hits an object before hitting the light source, then you know that this object is in the shade and make it darker.
And obviously, you have to limit the recursion depth for your recursive tracing, because otherwise, you'll never finish rendering.
So, first way of improving lighting is to start using direct illumination, which is you have a light source at the top of the scene.
And when you hit something like a wall here, you then bounce a ray at the light source.
And depending on how far this intersection A is from the light source, you shade either brighter or darker.
So it might be hard to see in this light, but the wall gets darker the further away it is from the light.
And then, you do shadows the same way, except at point B here, the rate it bounces from the ground hits this ball.
And so, point B is in the shade.
Now, you can improve on shadows by making what's called soft shadows.
Soft shadows are shadows that form a gradient from darker to lighter.
They happen when you have a light source that's not a point light.
So if you look at the shadows around you in the room, you'll see there are gradients, not just a solid block of darkness.
And the way you achieve that is when you get an intersection, when your ray intersects a point, you cast multiple shadow rays at different points along the light source.
And depending on how many of these rays hit something before hitting the light source, you shade some proportion of the light being casted, or the light coming from this light source.
So you can see point D here, none of the rays hit the light source, and so, it's in a very dark area.
Whereas point C, one of the rays hits the light source, and so it's in a slightly brighter area.
And then reflection and refraction work very similarly to shadows, except instead of bouncing from here to the light source, you bounce off this metal ball at the angle of reflection.
And you bounce through this glass ball using the angle of refraction.
You can calculate using Snell's law.
Any questions before we move on?
All right.
So as you can see pretty clearly in this image, there's still some problems.
The ceiling is completely black.
Shadows are mostly black.
And there's no light being transmitted through this glass ball either into the shadow here.
So there's definitely some improvements we can make on our physics simulation.
The first thing we're going to look at is called global illumination, which is pretty much any sort of illumination that doesn't come directly from light sources.
Now the reason this sort of illumination is important is that there's actually a lot of light that gets bounced around the room that doesn't come directly from the light source.
Like that white wall, if I shine this laser pointer at it, as you can see, there's still a lot of light bouncing off of it.
It's not all being absorbed.
So what you can do is you can take your ray intersection, and you can use what's called the Monte Carlo method, which is you take a whole bunch of random samples and interpolate those samples to get some value that approximates what the illuminant should be here.
The problem with that, though, is that when you trace backwards, you get this exponential explosion of recursively cast rays.
And you have no guarantee that any of these paths will ever get to this light source.
So what you have to do is trace forwards, so you start at the light source, and you cast photons in random directions and let them bounce all around the scene until they get absorbed.
Now, when a photon bounces, a bunch of light gets absorbed.
But not all the light.
And so, the rest of the light keeps reflecting.
And depending on the kind of material, different amounts and colors of light will be reflected.
So as you can see, this ray here bounces off the orange wall.
And so, orange light gets reflected.
This ray bounces off the blue wall.
And so, blue light gets reflected.
And this ray bounces off this metal ball, and so it gets completely reflected up towards the ceiling.
So if you remember each bounce in something called a photon map, then later, you can come back and say, oh, there's so many photons in this area.
And that's what these yellow circles are.
So the illuminance is going to be approximately this.
The problem with that is unless you're simulating an extremely large number of photons, you'll have an uneven distribution.
So what you can do is you can hybridize forwards ray tracing and the Monte Carlo method.
So you construct your photon map, which is represented in this image by all the red dots scattered everywhere.
And then you do backwards ray tracing.
Then when you reach a point, you randomly sample different reflection rays and average their illuminance to get the illuminance at this point.
So you have blue illuminance from the wall here.
And then light gray from here and darker gray from here.
And the result is this sort of pale blue, which you can't really see in the image because the light's bad.
Yeah, so backwards trace, randomly sample, illumination.
And any questions before we go on?
So it turns out even with building a photon map and then randomly sampling the photon map, calculating the irradiance can still be very expensive.
So what we've implemented to speed that up is what's called an irradiance cache.
And the way the irradiance cache works is there are certain areas on objects that will have pretty much the same irradiance.
It's pretty much the same irradiance levels.
So you can interpolate from nearby points.
So what we do with the irradiance cache is as the scene is rendering, if there is a cache miss, then we calculate the irradiance for some pixel which corresponds to a point in the scene.
And then, if we then trace a ray very near to that same pixel, you can then interpolate from nearby cached irradiances and get something that looks right.
So if you take a look at this image, where the green dots represent cache misses on these big planer objects here.
There's very few cache misses.
And it's almost entirely cache hits.
Whereas at the interface between different objects, the cache misses become much more common.
But overall, the number of irradiance calculations is greatly reduced.
And then, the third kind of illumination that we implement is called caustics.
Caustics are just light being lensed into patterns of brighter and darker spaces, such as you get from a light pointing towards a glass ball.
Or if you ever see the sort of wavy pattern on the bottom of a pool, that's also from caustics.
Caustics are also implemented with the photon map, except it's created slightly differently.
Instead of bouncing photons everywhere, photons are absorbed as soon as they hit a diffuse object.
So a photon might bounce off of this metal sphere and then hit the wall, and it will be immediately absorbed.
Whereas if it hits this sphere, it'll bounce, and bounce, and then be absorbed.
So the objective of caustics is to trace how photons are grouped based on the optical properties of materials in the scene.
And then, caustics are rendered pretty simply.
Because there's much fewer photons than in the photon map for global illumination, you don't need to use an irradiance cache.
So you simply trace a ray to a point and then sample the number of photons nearby.
And you end up with something like this if you just do caustics shading.
So then at the end, you take your direct illumination, and then add it to caustics, and then add it to global illumination.
And you get something that looks like this.
Any questions before we go to the code overview?
Go ahead
Do we weight them equally?
So the question was, do we weight them equally?
And the answer is yes.
And you just sum them together.
Ideally, you were clever and constructed your ray tracer so that they should be weighted equally, because there's weighting done internally, basically.
So next, we're going to do a quick overview of the code that will be in your repositories just get you started and help you understand how it works.
First, we're going to start by explaining some of the classes you'll find.
First is SceneObject, which is the parent class for all of the objects that will be used to construct the scenes you'll be rendering.
Every scene object has a ray intersection method that's called whenever you trace a ray.
And it'll return where the ray intersection is and the object.
And SceneObject is inherited by these objects here, which all of the objects will inherit from.
Then we've got the Raytracer class, which contains the main method.
So it starts the program.
And it also contains methods for constructing the scene and then rendering the scene once you've constructed the scene first.
We have the LightSource class, which is a base class for any light sources.
In our ray tracer, the only subclass is square photon light, which is this light at the top of the scene here.
And what that does, or what the light class does, is it contains methods for constructing the photon maps and getting illumination from either global, or caustic, or direct sources.
And then there's also the iCache class, which essentially just implements the iCache for you.
Any questions?
So basic execution overview.
You have the main method for Raytracer.
Parses the command line.
Constructs the scene.
And the scene is just a command line switch.
So if you say S1, it'll construct scene one.
If you say S2, it'll construct scene two.
Then for each light source, it tells the light source to make photon maps for the global and caustic illumination
I should mention that we actually moved the main functionality into [INAUDIBLE] so you can replace the data.
You don't have to touch that.
It just constructs the scene, so that we can then add a scene [INAUDIBLE] from that.
So you shouldn't need to do that
OK.
So that's changed then.
Then, main calls the render function of Raytracer.
What the render function does-- excuse me, method does-- is it casts a ray for each pixel in the scene you're going to render, finds the nearest intersection, and then computes the shading at that intersection by recursively bouncing rays and [?
painting ?]
the illuminance cache for illumination.
And that's basically what the Raytracer does.
It's pretty simple.
So we're also going to quickly go over how the light source traces photons for direct and global illumination.
The difference is kind of settled, but pretty important.
With global illumination, you want some total number of photons to be put in the map.
And you put a photon in the map whenever it hits some diffuse surface.
And then, that photon then continues bouncing around until it's absorbed.
For caustic illumination, it's pretty similar.
You have some set number of photons you want to get into the map.
You cast a ray in some random direction.
And then, the ray intersects with a diffuse material.
You store the intersection in the photon map.
And then you break.
You don't keep bouncing the photons around.
So that's basically it for the code overview.
This slide has some resources that you can use if you want to learn more about ray tracing.
Some of these will explain concepts that we haven't implemented.
They're an interesting read, but not needed if you don't want to read them.
And also, there's no standard way of doing global illumination or direct illumination.
So you may come across resources that do things slightly different than our reference implementation.
That doesn't mean either one is wrong.
But you should be aware that there will almost certainly be differences.
Does anyone have any questions?
Go ahead
So we can use one of those alternatives?
Can you say that louder please?
So we can use one of those alternatives?
These aren't implementations.
They're descriptions of how ray tracing works.
You should not change your code to work in this way.
These are just to help with clarification if you want to learn more about what's being done in the code.
Go ahead
[INAUDIBLE PHRASE] what specifically are we allowed to change for the [INAUDIBLE]?
So what specifically are you allowed to change?
Yeah.
Which ever one's smaller [INAUDIBLE PHRASE]
Anything not prohibited by these rules, which are paraphrased from the project handout, you can do.
So basically, as long as you're still performing the same essential calculations, you're fine.
Hmm?
I would say the criteria for correctness is to look at the images, and say that they look more or less the same.
They don't give you a pixel by pixel accurate.
This isn't a graphics class that actually does that.
This should be almost like a [INAUDIBLE]
They should visually look the same.
If you find some approximation that looks almost the same and runs 100 times faster, go for it.
You can't say, for example, simplify your ray tracer though by saying, OK, we're only going to be able to render these two scenes, but we'll be able to do it really fast.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So a couple of things I want to say about the final project.
You guys should start thinking about it.
So of course, you guys should think about the teams first, and submit team information.
Say who you're going to team up with.
By when?
Team information, Josh, by when?
Team information has to be in--
By tomorrow
By tomorrow.
OK, good.
You should know your teams.
Get them together, use the same mechanism that we used before to get the team information before.
So we are going to add one small thing this year I think that would be useful.
Once you have submitted your design documents, we are going to get a design review done by Masters.
So what that means is next week we are going to schedule a design review with your Masters.
And you should send mail to your Masters, hopefully in the middle of next week, to schedule a design review.
The design review will happen the week after Thanksgiving.
So you'll submit your design doc next week, and then the week after Thanksgiving, you'll schedule your design review.
The earlier the better, because you can hopefully get some really good feedback before you go into implementation.
So have an idea what you're doing.
Of course you're write it up for your design document.
And then go to your Masters and say, here's what I'm planning to do.
Get some good feedback, and hopefully it will help your life to do that.
And then beforehand, performance only mattered for your grade.
We did this absolute grading.
This year, we are going to have actually an in-class competition on the final day of class, to figure out who has the fastest ray tracer in the class.
And for that we will actually give [?
you ?]
a little but of a different [UNINTELLIGIBLE] than what I have given you.
And so you don't go too much into really hand coding to that teammate because that might not work.
And so here's something hot off the press.
So for the winning team, there's going to be an Akamai prize for the winning team.
And this prize includes a celebration/demonstration at Akamai headquarters.
You're going to go visit there [UNINTELLIGIBLE] and perhaps show off to their engineers the cool ray tracer you did.
And also every team member of the winning team is going to get the iPod Nano.
Sorry, guys, last year it didn't happen.
First time.
So there's a lot at stake.
So make sure your program is going to run as fast as you can.
OK, with that, let's get your slides on.
So I'd like to introduce Bradley Kuszmaul.
Bradley has been at MIT, in and out of MIT, for a long time doing lots of cool stuff with high-performance-- yeah, make the screen bigger-- high-performance computing.
He has done some really interesting data structure work, performance optimization work, and stuff like that.
And today he's going to talk about an interesting data structure that goes all the way from theory into getting really, really high-performance.
Thank you, Bradley.
OK, you have the mic
So I'm going to talk about a data structure called fractal trees, which in the academic world are called streaming B-trees.
But the marketing people didn't think very much of that, and so a lot of these slides are borrowed from a company that I've started.
So rather than redo that, I'm just going to stick to the terminology "fractal tree."
I'm research faculty at MIT, and I'm a founder at Tokutek.
And so that's sort of who I am.
I'll do a little bit more introduction.
So I have been around at MIT a long time.
I have four MIT degrees.
And I was one of the architects of the Connection Machine CM-5.
And Charles was also one of the architects of that machine.
So at the time, that was the fastest machine in the world, at least for some applications.
And after getting my degrees and being an architect, I went and was a professor at Yale, and then later I was at Akamai.
So I don't know what an Akamai prize is beyond an iPod, but maybe it's like all your content delivered free for a month or something.
And I'm now research faculty in the SuperTech Group, working with Charles.
And I'm a founder of Tokutek, which is commercializing some work we did.
A couple years ago, I started collaborating with Michael Bender and Martin Farach-Colton on data structures for that are suited for storing data on disk.
And we ended up a bit later starting a company to commercialize the research.
And basically, I'll tell you sort of what the background is, and actually go into some technical on the data structure.
So I don't know exactly what you've spent most of your time on, but a lot of high-performance work, especially in academia, focuses on the CPUs and using the CPUs efficiently, maybe getting lots of FLOPS or something.
The Cilk work that Charles and I did, for example, is squarely in the category of how do you get more FLOPS, or more computrons out of a particular machine.
But it turns out often I/O is a big bottleneck, and so you see systems that look a little bit like this.
You have a whole bunch of sensors somewhere, and the sensors might be something like a bunch of telescopes in an astronomy system.
They're sending millions of data items per second, and they have to be stored.
And disk is where you have to store large amounts of data, because disk is orders of magnitude per byte than other storage systems.
And then you want to do queries on that data, and you want to look at the data that's recent.
So it's not good enough just to look at yesterday's data.
You want to know what's going on right now.
If your sensor array is a bunch of telescopes, and a supernova starts happening, you want to be able to find out quickly what's going on, so that you can broadcast the message to everybody in the world so they can all point their telescopes at the supernova while it's fresh.
So that's the picture.
Another example of a sensor system is the internet, where you have thousands or millions of people clicking away on Facebook, for example.
You could view that collection of mice as, abstractly, a bunch of sensors.
And so you see it in science.
You see it in internet.
There's lots of applications.
For example, another one would be that you're looking for attacks on your internet infrastructure in a large corporation, or something.
So trying to reduce this big sensor system to what is the fundamental problem, basically we need to index the data.
So the data indexing problem is this.
Data is arriving in one order, and you want to ask about it in another order.
So typically data is arriving in the order by time.
When an observation is made, the event is logged.
When the next observation is made, the event is logged.
And then you want to do a query.
Tell me everything that's happening in that particular area of the sky over the past month.
So there's a big transposition that has to be done for these queries, abstractly, is the data's coming in in one order, and you want to sort it and get the data out in another order.
So one solution to this problem that a lot of people use is simply to sort the data.
The data comes in.
Sort it.
Then you can query it in the order that it makes sense.
This is basically a simple-minded explanation of what a data warehouse is.
A data warehouse is all this data comes in-- Walmart runs one of these in Arkansas.
All these events which are people scanning cans of soup on bar codes all over the country out in Walmart stores, all that data arrives in Arkansas, in one location.
They sort the data overnight, and then the next morning they can answer questions like what's the most popular food in the week before a hurricane strikes?
Because this is the kind of request that Walmart might care about, because they get a forecast that a hurricane's coming, and it turns out they need to ship beer and blueberry Pop-Tarts to the local stores, which are the things that basically you can eat even if power has failed.
The problem with sorting is that you have to wait overnight.
And for Walmart, that might actually be good enough.
But if you're the astronomer, that application is not so great.
So this problem is called the indexing problem.
We have to maintain indexes.
And traditionally, the classical solution is to use a data structure called a B-tree.
So do you all know what a B-tree is?
[INAUDIBLE] data structures to algorithms?
A B-tree is like a search tree, except it's got some fan-out, and I'll talk about it in a second.
They show up in virtually all storage systems today, and they were invented about 40 years ago, and they show up in databases such as MyISAM or Oracle.
They show up in file systems like XFS.
You can think of what Unix file systems like EXT do as being a variation of a B-tree.
Basically they're everywhere.
Mike might drew this picture of a B-tree.
And I said, I don't get it.
He said, well, there's a tree, and there's bees.
And I said, but those are wasps.
So anyway-- So a B-tree looks like this.
It's a search tree, so that means everything is organized.
It's got left children and right children, and there's actually many children.
And like any other search tree, all the things to the left are before all the things to the right.
That's sort of the property of trees that lets you do more than just a hash table.
A hash table lets you do get and put.
But a tree lets you do next.
And that's the key observation, why you need something like a tree instead of a hash table.
A lot of database queries-- if you go and click on Facebook on somebody's page, there's all these things that have been posted on somebody's wall.
And what they've done when they organized that data is that they've organized it so that each of those items is a row in the database, and they're next to each other so that you fetch the first one, which is like the home page of the person, and then the next and next and next gives each of the messages that they want to display.
And by making those things adjacent to each other, it means that they don't incur a disk I/O every time.
if it were just a hash table, you'd be having to look all over the place to find those things.
So B-trees are really fast, if you have to do insertions sequentially.
And the reason is you have a data structure that's too big to fit in main memory.
If the data structure fits in main memory, this is just the wrong data structure, right?
If it fits in main memory, what should you use to solve this problem?
Any ideas?
What data structure is like a B-tree except it doesn't have lots of fan-out?
Does anybody know this stuff in this class?
Do you people know data structures at all?
Maybe I'm in the wrong place.
Because it's OK. Just a binary tree would be the data structure if you were doing this in memory, right?
A binary tree would be fine.
Or maybe you would try to minimize the number of cache misses or something.
So for sequential inserts, if you're inserting at the end, basically all the stuff down the right spine of the tree is in main memory, and an insertion just inserts and inserts.
You have no disk I/Os, and basically it runs extremely fast.
The disk I/O is sequential.
You get basically performance that's limited by the disk bandwidth, which is the rate at which the disk can write consecutive blocks.
But B-trees are really slow if you're doing insertions that look random.
The database world calls those high-entropy.
And so basically the idea is I pick some leaf at random, and then I have to bring it into main memory, put the new record in there, and then eventually write it back out.
And because the data structure is spread all over disk, then each of those random blocks that I choose, when I bring it in, that's a random disk I/O, which is very expensive.
So here, for this workload, unlike the previous workload, the performance of the system is limited by how fast can you move the disk head around, rather than how fast can you write having placed the disk head.
And you perhaps you can only, on a disk drive, do something like 100 disk head movements per second.
And if you're writing small records that are like 100 bytes or something, you might find yourself using a thousandth of a percent of the disk I/O, of the disk's bandwidth performance.
And so people hate that.
They hate buying something and only being able to use a thousandth of a percent of its capacity.
Right?
New B-trees.
Something's wrong with that title.
So B-trees are really fast in doing range queries, because basically once you've are brought a block in and you want to do the next item, chances are the next item's also on the same page.
So once in while you go over a page boundary, but mostly you just reading stuff very fast.
Oh, I know what this is about.
When a B-tree's new and it's been constructed sequentially, it's also very fast.
When it gets old, what happens is the blocks themselves get moved around on disk.
They're not next to each other.
And this is a problem that people have spent a lot of time trying to solve, is that as B-trees get older, their performance degrades.
This aging problem-- I saw one report that suggested that something like 2% of all the money spent by corporations on IT is spent dumping and reloading their B-trees to try to make this problem go away.
So that's a lot of money or pain or something.
Well, B-trees are optimal for doing lookups.
If you just want to look something up, there's an old argument that says, gee, if going to have a tree structure, which is what you need in order to do next operations, then you're going to have some path through the B-tree which is a certain depth, and you do it optimally by having the fan-out be the block size.
And everything works.
But that argument of optimality is not actually true for insertion workloads.
And this is where the data structures work that I've done with Mike and Martin sort of gets to be an advantage.
To see that B-trees aren't optimal for insertions-- here's a data structure that's really good at insertions.
What is the data structure?
I'm just going to append to the end of a file.
Right?
So it's great.
Basically, it doesn't matter what the keys are.
I can insert data into this data structure at disk bandwidth.
What's the disadvantage of this data structure?
Lookups?
Lookups.
So what is the disadvantage?
Lookups aren't so good.
What is the cost of doing a lookup?
Order N?
Order n. Yeah.
You have to look at everything.
It requires a scan of the entire table.
And we'll get into what the cost model is in a second.
But basically, you have to look at everything.
So it's order n. It turns out the number of blocks you have to read in, which is the thing you care about-- it's order n over b.
So we'll get into a performance model in just a second.
So here we are.
We have two data structures-- a B-tree, which is not so great at insertions.
It's quite good at point queries and quite good at ranged queries, especially when it's young.
And we have this other data structure, which is the append data structure, which is wonderful for insertions and really bad for queries.
So can you do something that's like the best of all possible worlds?
You can imagine a data structure that's the worst of all possible, but it turns out that there are data structures that do well for this.
And I'll show you how one works in a minute So to explain how it works and to do the analysis, we need to have a cost model.
And we got into this just a minute ago, with what is the cost model for a table scan?
Is it order N?
Well, if you're only counting the number of CPU cycles that you're using up, it's order N, because you have to look at every item.
But if what you really care about is the number of disk I/Os, then you just count up the number of blocks.
And so in that model, the cost is order N over B.
And that's the model that we're going to use to do this analysis.
So in this model, we aren't going to care about CPU cost.
We are going to care about disk I/O.
And that's a pretty good place to design in if you're an engineer, because right now the number of CPU cycles that you get for a dollar is going up.
It's been going up.
It's continuing to go up.
You have to write parallel programs today to get that, but you get a lot of cycles in a $100 package.
But the number of disk I/Os that you're getting is essentially unchanged.
It's maybe improved by a factor of two in 40 years.
So that's the one to optimize for, is the one that's not changing.
And use all those CPU cycles, if you can, to do something.
So the model here is that we're going to have a memory and a disk.
And you could use this, and there's some block size, B, which we may or may not know what it is.
And it's actually quite tricky on real disk systems to figure out what the right block size is.
It's not 500 bytes, because that's not going to be a good block size.
It might be more like a megabyte.
And when we move stuff back and forth, we're going to move a block at a time.
We're going to bring in a whole block from disk, and when we have to write a block out, we write the whole block out.
So we're just going to count that up.
There's two parameters.
There's the block size, B, and the memory size, M. So if the memory is as big as the entire disk, then the problem goes away, and if the memory's way too small-- like you can only have one block-- then it's very difficult to get anything done.
So you need to be able to hold several blocks worth of storage.
The memory is treated as a cache for disk.
So once we've brought a block in, we can keep using it for a while until we get rid of it.
So have you guys done any cache-oblivious data structures?
OK.
So you've seen this model.
So the game here is to minimize the number of clock cycles and not worrying about the CPU cycles.
So here's the theoretical results.
We'll start with a B-tree.
So a B-tree which has a block size b-- and here I'm going to assume that the things you're storing are unit sized.
Because you can do the analysis, but it gets more complicated.
So the cost of a lookup, which is the upper right side, is log N over log B.
That's the same as-- you may not be used to manipulating these, but usually people write this as log base B of N. But that's the same as log N over log B.
And I'm going to write it this way, because then it's easier to compare things.
So if B is 1,000 or something, then basically instead of paying-- just as an example, if N is, say, 2 to the 40th-- let's these all to lg's because it basically doesn't matter.
So it's 40 over log base B, and if B is, say, 2 to the 10th, then that means that if you have a trillion items and you have a fan-out of 1,000, it takes you at most four disk I/Os to find any particular item.
An insertion cost is the same, because to do an insertion, we have to find the leaf that the item should have been in, and then put it there.
So the append log-- well, what's the cost of insertion?
Well, we're appending away, right?
And once every B items, we actually have to do a disk I/O.
So the cost of an insertion in the append log isn't 0.
It's one Bth of a block I/O per object.
And the point query cost looks really bad.
It's N over B, which we already discussed.
So the fractal tree has this kind of performance.
It's log N over-- it's not B, which would be really great.
It's something smaller.
It's maybe square root of B for the insertion cost.
And the lookup cost is log N over something, which I'm going to just hide.
Let's set epsilon to 1/2 and work out what that is.
Because epsilon 1/2 is a good engineering point.
So the insertion cost is log N over B to the 1/2, which is log N over root B.
And the other one, the lookup cost-- there's all big 0's around here, but I'm not going to draw those again-- over 1/2-- so I'm going to maybe ignore that-- of the log of the square root of B.
Did I do that right?
B to the 1 minus 1/2.
Put the 1/2 back in to make you happy.
So big-O of that-- well, what's log of root B?
[INAUDIBLE]
I can't quite hear you, but I know the answer.
I can just say that's the same as log B, when I'm doing big 0's Get rid of the halves.
So it's log N over log B.
So if you sort of choose block sizes, if you set this parameter to be something where you're doing something with the square root, you end up having lookups that cost, asymptotically the same as for a B-tree.
But there are these constants in there.
There's a factor of 4 or something that I've glossed over.
But asymptotically, it's the same.
And insertions have this much better performance.
What if B was 1,000?
Then we're dividing by 30 here.
But B isn't really 1,000.
B's more like a million in a modern system.
So you actually get to divide by something like 1,000 here.
And that's a huge advantage, to basically make insertions asymptotically be 1,000 times faster, whatever that means.
When you actually work out the constants, perhaps it's a factor of 100, is what we see in practice.
So this is basically working out those details.
So here's an example.
Here is a data structure that can achieve this kind of performance.
It's a simple version of a streaming B-tree or a fractal tree.
And what this data structure is-- so first of all, we're kind of going to switch modes from marketoid, or at least explaining what it's good for, to talking about what a data structure is that actually solves the problem.
So any questions before we dive down that path?
OK.
So if there are any questions, stop me.
Because I like to race through this stuff if possible.
So the deal here is that you're going to have log N arrays, and each one is a power of two in size.
And you're going to have one for each power of 2.
So there's going to be one array of size 1, one of size 2, one of size 4 and 8 and 16, all the way up to a trillion, 2 to the 40th.
The second invariant of this data structure is each array is either completely full or completely empty.
And the third one is that each array is sorted.
So I'll do an example here.
If I have four elements in the array, and these are the numbers, there's only one way for me to put those in that satisfy all those requirements.
Because there's four items, it has to go into the array of size four.
It has to fill it up.
I can't have any other way of doing that.
And within that array of size four, they have to be sorted.
So those four elements uniquely go there, and that's the end of the story for where four elements go.
If there's 10 elements, you get a little freedom, because, well, we have to fill up the 2 array and we have to fill up the 8 array, because there's only one way to write in binary 10, which is 0101.
But we get a little choice.
It turns out that the bottom array has to be sorted and the top array, the array containing 5 and 10 has to be sorted.
But we could have put the five down here and, say, swapped the 5 and the 6, and that would've been a perfectly valid data structure for this set of data as well.
So we get a little bit of freedom.
OK?
So that's the basic data structure.
So now what do we do?
How do you search this data structure?
Well, the idea is just to perform a binary search in each of the arrays.
The advantage of this is it works, and it's a lot faster than a table scan.
The disadvantage is it's actually quite a bit slower than a B-tree, because if you do the analysis here, which in this class, you probably-- you've done things like master theorem and stuff, right?
So you know what the cost of doing the search in the biggest array is, right?
How many disk I/Os is that in the worst case?
Log
It's log N. It's going to be log base 2 of N, plus or minus a little bit.
Just ignore all that stuff.
I'll just do L-O-G.
So what's the size of doing the second-biggest array?
What's the cost of searching the second-biggest array?
It's half as big, so it's log of N over 2, right?
I can't write.
So this is log N. This is equal to log of N minus 1.
What's the next array?
What's the cost of searching the next biggest array?
Log of N minus 2-- you add that up, and what's the sum?
We don't even need recurrences for this.
We could have done it that way, but what's the sum?
When we finally get down to 1, and you search the bottom array, you have to do one disk I/O in the worst case.
So this is an arithmetic sequence, right?
So what's the answer?
Big-O.
I'm not even going to ask for the-- pardon?
Log squared?
Yes, it's log squared, which is right there in green.
So basically, this thing is really expensive.
Log squared N, when we were trying to match a B-tree, which is log N over log B.
So not only is it not log base B, it's log base 2 or something.
But it's squaring it.
So if you think of having a million items in your data structure, even a relatively small one, log base 1 million is 20.
If you square that, that's 400.
Maybe you get to divide by 2.
It's hundreds of disk I/Os just to do a look up, instead of four.
So this is just sucking at this point.
So let's put that aside and see if we can do insertion, since we are doing so badly at [?
logging.
?]
So to make this easier to think about, I'm going to add another set of temporary arrays.
So I'm actually going to have two arrays of each size.
And the idea is at the beginning of each step, after doing an insertion, all the temporary arrays are empty.
I'm only going to have arrays on the left side that are going to have data in them.
So to insert 15 into this data structure, there's only one place to put it.
I put it in the one array, if I'm trying to be lazy about how much work I want to do.
And it turns out, this is exactly what you want to do.
You have an an empty one array, a new element comes in, just put it in there.
Now I want to insert a 7.
There's no place in the one array, so I'm going to put it in the one array over on the temp side.
And then I'm going to merge the two one arrays to make a two array.
So the 15 and the 7 become 7 and 15 here.
I couldn't put it there because that array already was full.
And then I merge those two to make a new four array.
So this is the final result after inserting those two items.
Does that make sense?
It's not a hard data structure.
So one insertion can cause a whole bunch of merges.
Here we have sort of an animation.
So here I've laid out the one array across the top, and then the temporary array just under it, and then going down, we have a sequence of steps for the data structure over time.
So we have the whole arrays.
We have one and the two and the four and the eight array are all full, and we insert one more item, which causes a big carry.
So the one creates a two, the two twos create two fours, the two fours and the eight create two eights, and so forth.
So here you are.
You're running.
You've built up a terabyte of data.
Your insert one more item, and now you have to rewrite all of disk.
So that also sounds a little unappealing.
But we'll build on this to make a data structure that actually works.
So first let's analyze what the average cost for this data structure is.
I've just sort of explained why-- there are some really bad cases where you're doing an insertion and it's expensive.
But on average, it turns out it's really good.
And the reason is that merging of sorted arrays is really I/O efficient, because the merge is essentially operating on that append data structure.
We're reading two append data structures and then writing the answer into another append data structure.
And that does hardly any I/O.
So if you have two arrays of size X, the cost to merge them is you have to read the two arrays and you have to write the new array.
And you add it all up, and that's order X over B I/Os.
Maybe it's 4x over b or something.
But big-O of X over B.
So the merge is efficient.
The cost per element for the merge is 1 over B, because order X elements were merged when we did that.
And we get to spread the cost.
Sure, we had to rewrite a trillion items when we filled up our disk, but actually, when you divide that out over the trillion items, it's not that much cost per item.
And so the cost for each item of that big operation is only 1 over B disk I/Os.
And each item only has to be rewritten log N times.
So the total average cost for an insertion of one element is log N over B, which is actually better than what I promised here.
But this data structure's going to be worse somewhere else.
So this is a simplified version.
I'll get to within-- ignoring epsilons and things, it'll be good enough.
So does that analysis makes sense?
It's not hard analysis, so if it doesn't make sense, it's not because of you.
It's got to be because I didn't explain it, because it's too easy to not understand.
So if you're going to build something like this, you can't just say, oh, well, your database is great except once every couple days it hangs for an hour while we resort everything.
So the fix of this is that we're going to get rid of the worst case.
And the idea is, well, let's just have a separate thread that does the merging of the arrays.
So we insert something into a temporary array and just return immediately.
And as long as the merge thread gets to do at least log N moves every time we insert something, it can keep up.
You could actually do a very careful dance, where I insert something, and part of the insertion is I have to move something from this array and something from this array and something from this array, and I can keep everything up to date that way.
So it's not very hard to de-amortize this algorithm-- that is, to turn the algorithm from a good average-case behavior to good worst-case behavior.
The worst-case behavior just becomes that it has to do log N work for an insertion, which isn't so bad.
Does that make sense?
Yeah
Does that work if these are [INAUDIBLE] items [INAUDIBLE]?
What if somebody wants [INAUDIBLE]?
Ah.
Well, OK, so the question-- let me repeat it and see if-- so you're in the middle of doing these merges and you have a background thread doing that, say, and somebody comes along and wants to do a query
Yeah.
[INAUDIBLE]
So the trick to there is that you put a bit on the array that says, the new arrays is not ready to query.
Keep using the old arrays, which are still there.
Just don't destroy the old ones until the new one's ready.
So basically you have these two megabyte-sized things.
You're trying to make a two megabyte-sized one.
You leave the one-megabyte ones lying around for a while while you're incrementally moving things down.
And then suddenly, when the big ones done, you flip the bits, so in order one operations, you can say, no, those two are no longer valid, and this one's valid.
So queries should use this one instead of those.
So that's basically the kind of trick you might do.
Or you would just search the partially constructed arrays, if you have locks.
There's lots of ways to do it.
So that's a pretty good question.
Yes.
That's one that we had to think about a little.
So it sounds glib, but it's like, how do we do this?
Any other questions?
OK.
So now we've got to do something about the search, because the search is really bad.
Well, it's not as bad as the insertion worst-case thing.
I'm going to show you how to shave off a factor of log N, and I don't think I'm going to show you how to shave off the factor of 1 over log B. so we'll just get it down to log N instead of log squared N. Because if I actually want to get it down, then I have to give up-- remember, the performance that I had was log of N over B.
If I actually want to get rid of things, I have to do something else.
There's a lower-bound argument.
So the idea here is we're searching-- I'm going to flip those.
We've got these arrays of various sizes.
And I've just done a binary search on here and then here and then here, and I found out the thing I'm looking for wasn't here and it wasn't here and it wasn't here.
That's where it would have been, if it had been there.
It should have been there but it wasn't.
It should have been here but it wasn't.
And then I'm going to start searching in this array.
And the intuition you might have is that, gee, it's kind of wasteful to start a whole new search on this array when we already knew where it wasn't in this array.
Right?
So for example, if the data were uniformly randomly distributed, and the thing was, say, 1/3 of the array here, I might gain some advantage by searching at the 1/3 point over here to see if it's there.
Now, that's kind of an intuition.
I don't know how to make that work.
But I do know how to make something work.
But the intuition is, having done some search here, I should in principal have information about where to limit the search so that I don't have to search the whole thing on the next array.
OK?
And here's basically what you do, is every element, you get a forward pointer to where that element would go in the next array.
So for example, you have something here and something here, which are the two things that are less than and greater than the thing you're looking for.
And it says, oh, those should have gone here in the next array.
So if you maintain that information, it's almost enough.
But let's gloss over the almost part.
If those two destinations of those two pointers are close together, then you've saved a lot of search in the next array.
Does anybody see a bug in this?
There is one.
The almost part.
You don't have to see it, because I've been thinking about this a lot.
The problem is, what if all of these items are less than all of these items, for example?
In which case, these pointers all point down to the beginning, and we've got nothing.
That's a case where this fails.
And that's allowed, right?
In particular, if we were inserting things-- yeah
Then we know the element is in the biggest array, because the element was supposed to go between the two
Ah, but in this array, we found out that it's above the last element, when we did our search, right?
That's one of the possible ways-- the worst-case behavior is we've got something where this array is less than this array.
We're looking for that item.
So we do a binary search and find out, it's over here.
And it doesn't help to special case this or something, because they could be all to the right or they could be all bunched up in funny ways.
There's lots of screwy ways that this could go wrong.
But the simple version, it's easy to come up with an example, which is everything's to the left.
Yeah
Can you still save time by, when you do the binary search on the smallest array-- but I guess you'd want [INAUDIBLE] search.
It will help reduced cost which gives you the next one and so on?
Yeah.
So there is a way to fix it so that the pointers in the smaller array do help you reduce the cost in the next array.
And that is to seed the smaller array with some values from the next array.
Like, suppose I put in every 20th item, and I stuck it in that array with a bit on it that says, oh, this is a repeat, it's going to be repeated again.
So then I could guarantee that there's these dummies that I throw in here, which are evenly spaced, plus whatever else is in there.
So put the other things in there, and they have forward pointers too.
And now I'm guaranteed that the distance between two adjacent items is guaranteed to be a constant.
Does that make sense?
The trick is to make it so that having found two adjacent items that the thing you want-- then on the next array, the image of those two items is separated by at most 20 items.
And so that gets you down to only log of N instead of log squared of n, because you're searching constant items in this array, and there's only log N arrays.
Yeah
Doesn't that slow down the merging of the rays?
Not asymptotically.
Because asymptotically, what this means-- if I'm going to build that array, so I'm going to merge two arrays to make this array, I have to do an additional scan of this other array as I'm constructing this one.
So the picture is I have two arrays, and I'm trying to merge them into this array.
And I'm trying to also insert these dummy forward pointers from the next array, which is only twice as big.
So the big 0's are, if it's X, instead of it being 1, 2, 3, 4X, it's 8X.
So it's only a constant.
So basically, I can read all three of these.
I can read an array and the next one and the next array, which is twice as big, and the next array which is four times as big.
It all adds up to 8 times the size of the original array.
So at least the asymptotics aren't messed up.
Maybe the engineer in you goes, bleh, I have to read the data eight times.
But remember, the game here is not to get 100% of the disk's insertion capacity.
That's not the game, going back to the marketing perspective.
The competition is only getting 0.001% of the disk's capacity.
That's what a B-tree gets in the worst case.
And so we don't have to get 100% to be three orders of magnitude better, which is where we are.
So it turns out that for this kind of thing, we end up getting 1% of the disk's capacity, and everybody's jumping around saying that's great, because it's 1,000 times faster.
And why do we only get 1%?
Well, there's a factor of two here and there's a log N over there, and you divide all that, and it's a constant challenge, because the engineers at Tokutek always are having ideas for how to make it faster.
And right now, making this data structure faster is not the thing that's going to make people buy it.
Because it's already 1,000 times faster than the competition.
What's going to make it faster is some other thing that adds features that make it so it's easy to use.
So I keep having to-- no, you really need to work on making it so that we can do backups, or something.
It turns out, if you're selling a database, you need to do more than just queries and insertions.
You need to be able to do backups.
You need to be able to recover from a crash.
You need to be able to cope with the problem of some particularly heavy query that's going and starving all the other queries from getting their work done.
All those problems turn out to be the problems that, if you do any of them badly, people won't buy you.
And so I suspect that there's another factor of 10 to be gotten over this data structure, if you were to sit down and try to say, how could I make it be the fastest possible thing.
And someday, that work will have to be done, because the competition will have it and we won't.
So let's see.
I mentioned some of these just now.
So some of the things you have to do in order to have an industrial strength dictionary are you need to cope with variable-size rows.
Now we assumed for the analysis that the rows were all unit size.
In fact, database rows vary in size.
And some of them are huge.
Some of them are megabytes.
Or sometimes people do things like they put satellite images into databases.
So they end up having very large rows.
You have to do deletions as well as insertions.
And it turns out we can do deletions just as fast as insertions.
And the idea there is basically, if you want to do a delete, you just you insert the thing with a bit on it that says, hey, this is really a deletion.
And then, whatever you get a chance, when you're doing a merge, if you find something that has the same value, you just annihilate it.
And the delete has to keep going down, because there might be more copies of it further down that were shadowed.
And eventually, when you finally do the last merge, that tombstone goes away.
You have to do transactions and logging.
You have to do crash recovery.
And it's a big pain to get that right, and a lot of companies have foundered when they tried to move from one mode to the other.
How many of you have experienced the phenomena that your file system didn't come back properly after a crash?
You see the difference in age here.
They're all using file systems that have transactional logging underneath them.
When's the last time it happened?
Tuesday
Tuesday.
So the difference is you're paying attention and they're not, right?
[INAUDIBLE] disk failure
Disk failure.
That's a different problem
Cacheing is not [?
finalized.
?]
Yeah.
You say everybody's running with their disk cache turned on.
And on some file systems, that's a bad idea.
So we're still suffering that it's been difficult to switch from the original Unix file system, which is 30 years old and wasn't designed to recover from crash.
You have to run fsck, and it doesn't always work.
We still have file systems that don't recover from crashes.
So you can see why that could be difficult.
It turns out that one common use case is that the data is coming in sequentially, and this data structure just sucks compared to a B-tree in the case where you're inserting things and the data actually is already sorted as it's inserted.
Because this is moving things all around and moving things all around.
And it's like, why didn't you just notice that it's sorted and put it in?
You have to get rid of the log base B to get it down to log base B of N instead of log base 2 of N for search costs.
Because people in fact do a lot more searches than-- if you have to choose which to do better, you want to generally do searches better.
And compression turns out to be important.
I had one customer who had a database which was 300 gigabytes.
He has a whole bunch of servers, and on each server, he had a 300 gigabyte database.
And with us, it was 70 gigabytes, because we compress.
And we just do simple compression of, basically, large blocks.
When we do I/Os, we do I/Os of like a megabyte.
So when we take one of those megabytes, we compress it.
And it's a big advantage to compress a megabyte at a time, instead of what-- a lot of B-trees, they have maybe 16 kilobytes.
And gzip hardly gets a chance to get anywhere when you only have 16 kilobytes.
And it gets down to 12 kilobytes.
But if you have a megabyte to work with and you compress it, particularly if it's sorted-- so this is a megabyte of data that's sorted, so compression works pretty well on sorted data.
So you get factors of 5 or 10 or something.
And so we asked him to dump the data without the indexes, so just the primary table with no indexes, and then run that through gzip.
And it was 50 gigabytes.
So the smallest he could store the raw data was 50 gigabytes, and we were giving him a useful database that was 70 gigabytes that had a bunch of indexes.
So he was like, yeah.
And you have to deal with multithreading and lots of clients and stuff.
So here's an example.
We worked with Mark Callahan, who was at Google at the time-- he's now at Facebook-- on trying to come up with some benchmarks, because none of the benchmarks out in the world do a good job of measuring this insertion performance problem.
So iiBench is an insertion benchmark.
And basically what it does is it sets up a database with three indexes, and the indexes are random.
So it's actually harder than real workloads.
This workload, you basically have a row and then you create a random key to point into that from three different places.
Real databases, it turns out, probably have more of a Zipfian distribution.
Have you talked at all about Zipfian distributions of data?
So this is sort of an interesting thing.
If you're dealing with real-world caches, you should know that data ain't uniformly randomly distributed.
That's a poor model.
So in particular, suppose I have memory and disk, and this is 10% of the disk.
Very simple situation.
Very common ratio.
You'll see, this is how Facebook sets up their databases.
They'll have a 300-gigabyte database and 30 gigs of RAM.
If the queries that you wanted to do were random, it wouldn't matter what data structure you were using.
Let's suppose that God tells you where it is on disk, so you don't have to find it.
You just have to move the disk head and move it.
If they're random, then basically, no matter what you've done, 90% of the queries you have to do are going to do a random disk I/O.
10% are going to already be there, because you got lucky.
So that is not reflecting what's going on on any workload that I know.
What they'll see is more like 99% of the queries hit here, and 1% go out here, or maybe 95% here and 5% go out there.
And it turns out that, that for a lot of things, there is a model of what's going on called a Zipfian distribution.
This would a random uniform distribution.
It's like every item has equal probability of being chosen for a query.
It turns out that for things like what's the popularity of web pages, or if you have a library, what's the frequency at which words appear in the library-- so words like "the" appear frequently, and words like "polymorphic" are less frequent.
So Zipf came up with this model, and there's a simple version of the model, which says that the most popular word has probability proportional to 1.
It's not going to be 1.
It's going to be proportional to 1.
The second most popular word is going to have 1/2.
The third most popular word is 1/3.
And the fourth most popular word is 1/4 the probability of the first word, and so forth.
So if you plot this distribution, it kind of looks like this, right?
It's like 1 over x.
And what would you tell me if I told you that I had an infinite universe of objects that had a probability distribution like this.
Does that seem plausible?
Why?
You're saying no
[INAUDIBLE PHRASE] try adding them all together
If you add them all together, it doesn't converge.
So it's a heavy-tailed distribution.
So it turns out that if you sum up these up to 1, the sum from 1 to n of 1 over i, it's the nth harmonic number.
And that grows over time.
It's basically like the integral under this curve, from 1 to n. It's close to that.
And what is the integral of that?
It's like something you learned seven years ago, and now you've forgotten, right?
You learned it when you were sophomores in high school, or something.
So it's approximately log of n. It's actually like log of n plus 0.57, is a very good approximation for the nth harmonic number.
When you're doing this kind of analysis, boy, it depresses people, because you say, oh, that's H of n, and if you have a million items in your database, then the sum of all those things is H of 1 million, and what's H of 1 million?
Well, the log base 2 of 1 million, I know that, because I'm a computer scientist.
So it's going to be like 20, and because we're doing log base e in that formula, maybe it's 15 or something.
So if you have 1 million items, then the most popular item is going to-- you have to divide by H of n here.
So the most popular item is going to appear 1/15 of the time.
And the next most popular item is going to appear-- emergency backup chalk.
Somebody's been burning both ends of this chalk.
1/30 of the time, and 1/45 of the time, and those add up.
When you go up to 1 over 1 million-- another zero in there-- times 15, that finite series will add up to 1, approximately.
Except to the extent that I've approximated.
So this is what's going on.
So the most popular Facebook page-- they might have 1 billion pages, so how does that change things?
Well, that means the most popular one has a probability 1 in 20, and the second most is 1 in 40.
And this explains why cache works for this kind of workload.
Nobody really knows why Facebook pages and words in libraries and everything else have this distribution, which is named after a guy named Zipf.
But they do.
Everything has this property.
And so you can sort of predict what's happening.
So iiBench should have a Zipfian distribution and it doesn't.
So this is painting a worse picture.
Or a better picture.
It's making us look better than we really are, because the real world is going to have more hits on the stuff that's in memory for a B-tree than this model, where basically you're completely hosed all the time because it's random.
So this is an example in the category of how to lie with statistics.
And it's a pretty sophisticated lie.
If you're going to lie, be sophisticated.
So these measurements were taken in the top graph.
Up is good.
It's how many rows per second we could insert.
And this axis is how many rows have been inserted so far.
And the green one is a B-tree.
According to Mark Callahan, who's essentially a disinterested observer, it's the best implementation of a B-tree ever.
And you can sort of see what happens, is that as you insert stuff, the system falls out of main memory, and the performance was really good at the beginning-- 40,000 per second-- and then boom, you're down to 200 down here at the end, by the time you've inserted a billion rows.
Whereas, for the fractal tree, you can sort of see this noise.
That's because some insertions are a little cheaper than other insertions.
Every other insertion's completely free, right?
You had a free spot.
You just put it in.
One out of four insertions, the ones that weren't free, half of them only had to do a little operation in memory.
So you see this high frequency noise, because some things are cheaper than others.
And that's like a factor of 30 or something.
It turns out it even works on SSD, solid state disk.
You might think-- all this time I've been talking about disk drives.
Solid state disk has a complicated cache hierarchy inside it, and we were surprised to see that basically we're faster on this workload on a rotating disk than a B-tree is on an SSD, which is orders of magnitude faster in principle, but turns out that for various reasons it's not.
One question I get often is, the world is moving away from rotating disk to solid state disk.
A lot of applications-- how many of you have solid state disks in your laptops?
That's a really good application for a solid state disk, because it's not sensitive to being knocked around.
So it's worth it to have a solid state disk even if it were more expensive, which it is.
It turns out it's not that much more expensive for a laptop.
It's a couple of hundred dollars more or something.
But the advantage of it is that if you go up in an airplane and you're sitting and trying to type in the middle of a thunderstorm, flying across-- it doesn't care.
Disk drives, if you do that-- disk drives do not like flying at high altitude, because they work by having a cushion of air that the head is flying on, and in airplanes, which pressurize the cabin to the same altitude as 8,000 feet, that's half an atmosphere.
So there's only half as much air keeping it off.
So if you travel a lot, that's when your disk drive will fail, is when you're flying.
OK.
So it looks like, however that rotating disk is getting cheaper faster then solid state disk is.
So rotating disk is an order of magnitude cheaper per byte than solid state disk today.
Maybe two orders of magnitude cheaper.
It's hard to measure fairly.
But rotating disk, according to Seagate-- they're saying, by the end of the decade, we'll have 70 terabyte drives that are the same form factor.
And so you figure out what the Moore's Law is for that, and it's better than for lithography.
Lithography is not going to be that much more dense in that timeframe.
So at least for the next 5 or 10 years, it looks like disk drives are going to maintain their cost advantage over solid state storage, and maybe even spread that cost advantage.
So for any particular application, for storing your music, SSD will be cheap enough, but for those people that have really big data sets, like these new telescopes they're putting up-- these new telescopes are crazy.
These people are putting up these telescopes.
They're putting up 1,500 telescopes across the Australian Outback.
And each of those telescopes in the first 15 minutes live is going to produce more data than has come down from the Hubble, total.
And there's just no way for them to-- I don't know what they're going to do.
But it's a huge amount of data, and they're going to have to use disks to store whatever it is that they want to keep.
And they don't like throwing away data, because it's so expensive to make.
So if I were a disk maker, I'd make sure that my salesmen had an office somewhere out there.
So the conclusion is you're not going to be able to, at least for those applications, just have an index in main memory.
You're going to have to have a data structure that works well on disk.
The speed trends-- well, seek time is not going to change.
It hasn't changed.
It's not going to change.
The bandwidth of a disk drive grows with the square root of its capacity.
So if you quadruple the storage on the disk because you've made the bits twice as dense in each dimension, then one spin of the disk sees twice as many disks, not four times as many disks.
So that projects out to something like disks that are 500 megabytes per second.
So how long is it going to take to back up a 67 terabyte disk drive?
So there remain systems problems.
And I was explaining to my son that there's all these problems in systems.
Data structures aren't suited, and all these systems suck.
He said, well, isn't that horrible if you're computer scientist?
I said, no, because we make our living off of these problems.
So here are some problems.
There's plenty of living to be made yet.
Power consumption is also a big issue for these things.
If you fill up a room a Google data center, a room which is probably bigger than this room, full of machines.
The Facebook data center is probably a room about this size, full of machines.
And power and cooling is something like half the cost of the machines.
The machines for something like Facebook, the hardware might cost them $10 million or $20 million a year, and the heating and cooling is another $10 million or $20 million a year, which is why they go off and they build these data centers in places like North Carolina, where I guess they're willing to give them power for free or something.
So making good use of disk bandwidth offers huge power savings, because basically you can use disks which are cheaper than solid state for power.
And you want to use that well.
CPU trends.
Well, you've probably talked about this, right?
CPUs are going to get a lot more cores.
I actually have a 48-core machine that cost $10,000 that I bought about a month ago.
And our customers mostly use machines that are like $5,000 machines.
So when I provisioned this machine, I said, well, I should spend and buy a machine that's twice as good as what they're buying, because I'm developing software that they're going to use next year.
So I bought a $10,000 machine, which is 48 cores.
And we're having all sorts of making a living with that machine.
The memory bandwidth and the I/O bus bandwidth will grow.
And so I think it's going to get more and more exciting to try to use all these cores.
Fractal trees have a lot of opportunity to use those cores to improve and reduce the number of disk I/Os.
So the conclusion is, basically, these data structures dominate B-trees asymptotically.
And then B-trees have 40 years of engineering advantage, but that will evaporate eventually.
These data structures ride better technology curves than B-trees do, and so I find it hard to believe that in 10 years that anybody would design a system using a B-tree, because how do you overcome those advantages.
So basically all storage systems are going to use data structures that are like this, or something else.
There's a whole bunch of other kinds of indexes that we haven't attacked, things like indexing multi-dimensional data or indexing data where you have very large keys, very large rows.
Imagine that you're trying to index DNA sequences, which are much bigger than a disk block.
So there's a whole bunch of interesting opportunities.
And that's what I'm working on.
So any questions or comments?
Arguments?
Fistfights?
Where's the mic?
Where is the mic?
That's OK.
I can [INAUDIBLE]
It's on my coat
So actually, this is a very interesting point, because if you think where the world is leading, I think that big data is something that's very, very interesting, because all these people are gathering huge amounts of data, and they're storing huge amounts of data.
And what do with data, accessing them, is going to be one big problem.
I mean, if you look at what people like Google are doing, they're just collecting all those.
Nobody's throwing anything out.
And I believe if you to kind of look at them, analyze them, do cool things with the data, it's going to be very, very important.
So I think that would be very interesting, high-performance end.
It's not just doing number crunching.
Until now, when people look at high-performance, it's about CPU.
It's about how many FLOPS per second can you do?
TeraFLOPS, petaFLOP machines and stuff like that.
But I think one thing that's really interesting is it's not petaFLOPS.
How many terabytes of data can you process through to find something
So I was at a talk by Facebook, and they serve 37 gigabytes per data per second out of their database tier.
And that's a lot of serving.
Out of one little piece of whatever they're doing.
Those guys have three or five petabytes.
And in the petabyte club, they're small potatoes.
There's people who have hundreds of petabytes, people with three-letter acronyms
I mean, some of those three-letter acronym places, the amount of data they are getting and they are processing is just gigantic.
And I think to a point that even some of the interesting things about-- if they keep growing their data centers at the rate they keep growing in the next couple of decades, they will need the entire power of the United States to power their data centers, because they are at that kind of thing at this point.
Even in these big national labs, the reason they can't expand is not that they don't have money to buy the machines, but they don't have money to pay for the electricity, and also they don't have electricity-- that much electricity-- [UNINTELLIGIBLE] them to basically feed it
I've run into people for whom the power issue was a big deal.
I look at it and say, eh, you bought a $5,000 machine.
You spend $5,000 in power over the lifetime of the machine.
It doesn't seem like it's that big a deal.
But they've filled up their data center, and the cost of adding one more machine has a huge incremental cost, because they can't fit one more in.
So that means they have to build another building.
And so almost everybody's facing that problem who's in this business.
And then they try to build a building somewhere where there's natural cooled-- Google's written these papers about, oh, it turns out if you don't air condition your computers, most of them work anyway.
So, well, air conditioning is a quarter of the cost over the lifetime of the computer.
So if you can make more than 3/4 of them give you service, you come out ahead
On that note, MIT is part of a consortium that includes Harvard, Northeastern, Boston University, and University of Massachusetts Amherst, to relocate all of our high-performance computing into a new green data center in Holyoke, Massachusetts.
So the idea is that rather than us locating things here on campus, where the energy costs are high and we get a lot of our energy from fuels that have a big carbon footprint, locating it in Holyoke-- they have a lot of hydro power and nuclear power there.
And they're able to build a building that is extremely energy-efficient.
And it turns out that a bunch of years ago when they were digging up the Route 90, the Mass Pike, they laid a lot of dark fiber down the length.
And so what they're going to do is light up that fiber, which comes right back here to the Boston area.
And so for most people who are using these very high-performance things, it doesn't really matter where it's located anymore, at the level of that.
So instead of just locating some piece of equipment here, we just will locate it out there, and the price will drop dramatically.
And it'll be a much greener way for us to be doing our high-end computing.
Yeah, question?
Isn't someone talking about water-cooled offshore floating data centers?
Sure.
Sure.
So the question is, are people talking about water-coooled offshore floating data centers.
Yeah.
I mean, locating things in some area where you can cool things easily makes a lot of sense.
Usually, they tend to want those near rivers rather than in the middle of the ocean, just because you get the hydropower.
But even in the ocean, you can use currents to do very much the same kind of thing.
So for some of these things, people are looking very seriously at a whole bunch of different strategies for containing large-scale equipment
So one that's very counterintuitive is people are trying to build data centers in the middle of deserts, when it's very hot.
I mean, why do you think people want to build a data center in the middle of the desert?
Solar power?
Solar power is one thing.
No, it's not solar power
It gets really cold at night
No, it's not really cold at night.
That's not it
Cheap property
No, the biggest thing about cooling is either you can do air conditioning, where you're using power to pull heat out, or you can use just water to cool.
And what happens is most of other places is the humidity is too high.
And when you go to the desert, humidity is low enough that you can just pump water through the thing and get the water evaporation going by, and then use that to cool the system.
So sometimes they're looking at data centers in places where it could be 120 degrees, but very low humidity.
And they think that is a lot more efficient to cool than that.
So there are a lot of these interesting nonintuitive things people are looking at.
So what [UNINTELLIGIBLE] they say is that humidity's the killer, not the temperature
What if you're located on the South-- if you're located on the South Pole, then that's both cold and really low humidity
Yeah.
I mean it'll be interesting to see how these things develop.
It's a very so-called hot topic these days, is energy for computing.
And the energy for computing of course matters also not only at the large scale but also at the small scale, because you want your favorite handheld to use very little battery.
So your batteries last longer.
So the issue of energy, using that as a measure-- we've mostly been looking at how fast we can make things run in this class, but many of the lessons you can use to say, well, how can I make this run as energy-efficient as possible?
And what you'll learn is that many of the lessons we've had in the class during the term we focused, as I say, on performance.
But there are many resources in any given situation that you might want to optimize.
And so understanding something about how do I minimize energy, how do I minimize disk I/Os, how do I minimize clock cycles, how do I minimize off-chip accesses-- which tend to be much more energy intensive than on-chip-- all those different kinds of measures end up being part of the mix of what you have to do when you're really engineering these systems
So I think another interesting thing is, because we are in this time where some stuff grows at exponential rates and stuff like that, some of those ratios that made sense at some point just suddenly start making really bad things.
Like, for example, in this, at some point the seek times were normal enough that you didn't care.
And at some point, because the rest of the things took off so fast, suddenly it becomes this really, really big bottlenecks
B-trees were a really good data structure in 1972.
Because, well, the seek time and the transfer time and the CPU-- the CPUs actually couldn't read in the data in one rotation, so people didn't even read consecutive blocks, because the CPU just couldn't handle data coming in that fast.
You would stagger blocks around the disk, so that when you did sequential reads, you'd get this one and then this one and this one.
There was this whole thing about tuning your file system.
It's like--
By the way, back when disks were--
Yeah.
Washing machine
Washing machine size
For 20 megabytes
Oh, yeah, that's the big disk.
So hopefully you guys got a feel for-- we have been looking at this performance on a small multi-core and stuff like that, how it can scale in different directions and the kind of impact performance can have.
And in fact, if anybody has read books on why Google is successful, one of the biggest things for their success is they managed to do a huge amount of work very cheaply, because the amount of work they do, if anybody did in the traditional way, they can't afford that model, to give it for free or give it supplied for advertising.
Because they can get it done because it's about optimization.
Performance, Performance basically relates to cost.
And if the cost is low enough, then they don't have to keep charging a huge amount of money for each search
So let's thank Dr. Kuszmaul for an excellent talk.
And can you hang out for just a little bit, if people want to come down?
OK.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
As we come close to testing, we have shrinkage here.
People probably left home.
Hopefully, everybody who left home finished their report.
So you guys have all looked into how to do the final project.
And have all the ideas how to go and optimize.
How many people have downloaded, compiled, and ran, and you know what's going on?
OK. Good.
Good.
Good.
Exactly.
It's happening right now.
OK. Good.
So I will repeat this what I said last time in here.
We're going to have a design review with your masters.
So just look for us to send you the information.
That means when you come back from Thanksgiving, schedule it early.
So they can help if you have any changes in design process.
And then we have a competition on December 9 in class here, trying to figure out who has the fastest ray tracer created.
And in fact, this year there is Akamai prize for the winning team, including they have a kind of celebration and demonstration in their headquarters.
You get to go get a tour with their knock and stuff like.
Plus, every winning member is going to get a iPod Nano.
So there's a lot more motivation now to get the fastest running ray tracer OK.
So with that, let's switch gears a little bit.
So today, I'm going to talk about distributed systems.
Until now what we looked at was, OK, given a box how to get something running as fast as possible inside that box.
And today we're going to look at going outside the box.
Basically, we want to scale up to clusters of machines.
That means the room can have 10, 15 machines.
In fact, for your class, you guys are using-- how many machines do we have?
16 machines.
So you are doing independently.
But you can use as one gigantic machine if you can and run something.
And data center scale.
This is kind of people like Google, and Amazon, has these kinds of things.
And finally, Planet Scale.
If you want to run something even bigger, larger.
What you have to deal with, and what kind of issues you have to deal with.
It's time to reboot my machine.
And I have to be pressing this button probably four or five times during the day.
So Cluster Scale.
So you want to run a program on multiple machines.
And OK, Let me put it there.
Why the heck do you want to do that?
What's the advantages of-- instead of running on one nice machine, running on a cluster of machines?
What do you get?
It's cheaper
It's cheaper.
That's a good one.
It's cheaper to get a bunch of small machines than to buy a humongo mainframe type machine.
Yes, that's a very good answer.
What else?
It's very slow.
It's slower
So you would run something because it's slower?
But it is a trade-off
Yes, so there's some trade off between speed.
But it might not be that much.
Even when you get a gigantic machine, there are bottlenecks in it.
In a cluster kind of thing, you can avoid the bottlenecks.
But hopefully, you're trying to do it to get some performance in scaling to large number of users and what not.
So basically, what you want to get is-- so get more parallelism.
Because now we have more machines, more calls.
And hopefully get higher throughput.
Definitely, because you are doing it.
Hopefully, it's a little bit of lower latency too, because if you have one gigantic system, if everything has to go through bottlenecks, it might be slower than basically having different system.
So assume, just an example, if you are something like Verizon or Netflix trying to serve your videos.
It makes much more sense to have a bunch of clusters of machines each doing a lot of independent work than trying to send all the videos to one machine.
Another interesting fact is robustness.
So until now you guys didn't care about robustness, because something went wrong, the entire thing collapsed.
There's no half baked machine.
The machine crashed, your program crashed.
Everything died.
So you just have this fatalistic attitude.
OK.
It crashed.
Everything is dead.
So why bother?
But in these clusters, if you have a lot of machines, if one machine dies, there's many others to pick up.
So you can have a system that probably has availability much higher than what you can get on a single machine.
And finally, cost savings.
Because it's cheaper to do this, have a bunch of small machines.
And businesses like Google has really taken advantage of that.
So there are issues we have to deal with in order to program this damn thing.
And if you want to get performance, you have to program in a way to get good performance.
You don't run much slower and load less performance than one box.
You'll get performance and also performance scalability.
That means if you get 10 machines, you want to get some performance as if you have 20.
Hopefully, you want to get a lot more performance than 20.
So how do we keep things scaling in there?
And also the thing's robustness.
So the idea there is if you have one machine, you're fatalistic.
If the machine goes, everything goes.
You don't care.
But if you have a lot more machines, you want to make sure that application runs even if the machine's fails.
Worse, if you have a lot of machines, there's a lot more chance of failure.
So if one goes down, everything crashes still.
Then your application will be a lot less robust even than a single machine, because there too many moving parts to go wrong.
So you want to actually deal with this robustness.
So that adds an entire new dimension in there.
We are not going to go too much deeper into robustness.
But that is one big thing that you have to really worry about when you go to distributed systems.
OK. What's a distributed system?
So this is what we have been working so far?
Can we see if we can reduce the lights?
I guess up there you can't-- OK. We don't go fully dark, we'll see.
Oh, that's you guys.
Don't go to sleep even though light is-- there.
So this should be over there and I don't have any way to darken this side.
So these are the machines we have been thinking about.
We have a memory system.
And more than just having a shared memory, we have cache coherence.
So that means if two people want to communicate to write to this single memory location, and the lot of that data appears lower on all the different cores.
So we can use that information to basically communicate to the processor.
That's really nice.
So a distributed memory machine has no shared memory.
So each memory is-- Now, how are you going to communicate?
Message?
Yeah, this is not software.
You actually need something additional.
Something like a network, or Internet, something behind sitting out that actually let you communicate between each other.
So if you just really look at the kind of cost, this is a back of the envelope type calculation.
Register is probably one cycle.
Cache is about 10 cycles.
If you go to DRAM, you can get about 1,000 cycles.
Remote memory, going somewhere across, is, again, another order of magnitude from that.
So of course, you keep adding.
And that's probably the reason that sometimes things can be slow.
Because now, we have another layer that's even slower.
So we have to think about it, worry about it when you're writing code for these types of machines.
So in shared memory machines, we learn in languages like Cilk.
It's very nice to communicate because we synchronize via locks.
And all communication via memory.
Because when you write something, if you look at that memory location, everybody else will see it.
And if you put the right synchronization, hopefully you will get the value you want.
In distributed memory machines, there's nothing like that.
So what we see is we explicitly sends some data across.
So you have what we call messages.
And that means if you want to send something to-- if another person needs to look at something, we have to send it to that person.
So you have to originate yourself.
Saying, I'm sending.
That other person has to receive it.
And they have to put it wherever you want.
So everybody's address space is separate.
And if you want to synchronize, you would also do it through the message.
So you send a message, then the other person wait for the message to come.
And so this shows you what normally happens in messages.
In the shared memory, there's nothing called message size.
You write a cache line.
The cache line moves.
And you can't keep changing the cache line size.
Hopefully, prefetcher will be good and do something nice.
But you don't have that much choice.
In messages, you can compose any size of message you want.
So what this graph shows is the minimum cost and average cost of different size messages.
So there's a couple of things to get out of this graph.
One is that if the message is even 0 length, or very small, you still have overhead.
You're going to send the darn message.
So if even you send nothing, it cost you some amount.
And the second thing is as the message gets bigger and bigger, the cost keeps increasing, because now you're sending more and more data.
So if you really amortize the overhead cost, you are to send large messages in there.
Another thing this chart shows is that as messages become bigger, the kind of the distribution of overhead is all over the map.
Because now we are sending large things, a lot of other craziness happens to these things.
So sometimes it can go fast, sometimes it can be pretty slow.
I will get why it might be sometimes this kind of distribution shortly.
So the main point is that, that you don't send smaller messages if you can, because the overhead is too high.
So why is this?
Why is sending messages complicated?
Till now, there's nobody sitting between you and hardware.
Once you send the program run, you own the entire hardware, and after figuring out all the weirdness that's on x86 there's nothing in between you.
You probably won't look at the compile code if you look at what assembly is generated you have full view what's going on in here.
Unfortunately, message passing, a lot of other things come into play.
So if you want to send a message and the applications says, aha, I'm sending a message.
And normally, it will do a system call to operating system.
And normally, this message will get copied into the operating system.
It's copying here.
This operating system called the operating system wakes up.
This might be when this scheduled, there's a lot of things going on.
And then the operating system has to send to the network interface card.
And the network will say, OK, I can't send long messages.
I'm going to break into a bunch of small messages.
And put some hardware here.
And it will end up in the other side.
In a bunch of fragmented small pieces that the network interface unit has to reassemble into one message and deliver up.
And this will probably-- it will copy back into the application.
So what that means is that a lot of other things getting involved, each optimize separately, doing a lot of different things.
And so that is why you have this big unpredictable mess happening in message passing.
And so there you not only have to worry about your code.
You have to worry about what the operating system is doing.
You have to worry about what the network is doing.
You have to worry about your network card's doing.
So there's a lot of moving parts in this.
If you want to get really, really good performance, people have to worry about all these things in here.
So let's look at how a message works.
So I hope you can see these diagrams.
Can you see these?
Barely?
So let me say-- So I have a sending process and a receiving process.
Oh, don't reboot please.
And so what happens if-- this is a message-- if we are sending without any buffering of a message, that means I am not copying it anywhere, so assume I want to send a message.
I said I have a message to send.
And then what happens in this model is, OK, until the receiver is ready, you have to wait.
Because there is no place to send the message.
So finally, when the other side says I want to receive something, it will tell this thing it's OK to send.
And it will copy the data.
And then after copying the data, both parts can continue.
So this is what happens if sender wants to send early.
If you're very lucky, the minute you try to send, the receiver says I want it.
And we have very little delay.
And everything gets copied.
And that's in your lucky case.
In other cases, the receiver wants some data.
But the sender is not ready, so your receiver has to wait until the sender wants to send it.
And when this message [?
is ?]
[?
sent ?
], you copy the data in here.
So this is a very naive simple way.
What can we eliminate out of this?
How can we make it a little bit faster?
Buffer
Yeah.
If you buffer, what will eliminate?
What will go away?
Out of-- we have this overhead, this overhead, and this overhead.
Which overheads can get eliminated?
Wait for send can go ahead.
So what happens is-- So here actually what they're showing is buffering also with some hardware support.
That means I am trying to send something.
And the minute I copied it out there, I can keep working in there.
And somewhere in the background where you send the data, it will arrive here.
And if it wants it, the data is there.
Of course, if the receiver comes early and asks for data, you can't do that.
Still you have to wait, because the data is not there.
However, if there's no hardware support, both has to probably wait a little bit, because you have to get the data copied.
So if you have a lot of hardware support, you don't see this copy time.
But if there's no hardware support, you see some copy time going in here.
So what's the advantage of this versus-- OK, tell me one advantage of this method versus this method.
So of course, this one there is a lot of wait time and stuff like that.
We know that.
But is there any advantage of doing this one, this waiting until sending and sending it there versus this kind of a nice sending it in the background
They're synchronized
Hmm?
Sychronized
Synchronized is one advantage.
What else might happen?
So what else are you going to do to get this kind of thing working?
So in order for this to make progress, what do you have to do to data?
It has to copy.
So it has to get multiple copies.
So from application space.
It has to get copied to operating system space.
It has to get copied into the networking stack.
So data keep getting copying, and copying, and copying in there.
And in here you basically don't copy.
You just say, OK, wait.
I'll keep the data and when you're ready, I will send it directly in here.
And you can directly probably even send it to the network.
And send it.
So if you're sending a lot of data, copy my old value.
So this might even be better if you're sending a huge amount of data.
So that's one advantage of having system like that.
And of course, hardware-- if there's no hardware support, basically still you have to do some copying in here.
So this is-- what am I showing here?
So what we are showing in here is non-blocking.
So one way to look at that is when you're sending, when you request for send, what you can say is, OK, I continue but I haven't copied the data.
I have my data in here, but I'm doing that.
But what I must tell you, OK, look, this data still hasn't moved out of my space yet.
So I have to worry.
I can't rewrite the data.
And at some point, when you say I want the data, it will go there and bring the data for you.
And catch you like that.
So between this time since I don't want to make too many copies, I have to make sure that I don't touch that data.
Or I have to copy it.
So that's my request in here.
And of course, if you have no hardware support, you have to put some time into actually doing the copying.
So this is nice.
But we want to have a little bit of high level support to do this.
So this is not as nice as things like Cilk, because you don't have to worry about a lot of other interesting things going on.
So what people have developed is called MPI language, Message Passing Interface language.
It is kind of a bit foggy.
But that's the best people have these days.
A machine independent way of when have the distributed systems to communicate with each other.
So-- [PHONE RINGING] Whoops.
That's not good.
My phone.
Sorry about that.
So what happens is each machine has its own processor, it's own memory.
So there's no shared memory on a thing like that.
Its own thread of control is run.
And each process communicates via messages.
And there is send as is needed.
And that means but you can't send like pointers, because there's no notion of pointers.
You actually have a data structure that's self-contained center of the site.
So here's a small program.
I'm going to walk through that.
So I have main.
And I'm setting a bunch of these variables.
For now, those are not that important.
But for completeness, I have that.
And then, of course, if use something like MPI, there's a bunch of setup things that you have.
And so basically like cut and paste with what people normally do as we set up.
And then I have this piece of code.
This piece of code, what it does is, this same program runs on multiple different machines.
So everyone has the same program.
But then at some point, I want to know in my machine what to do.
So what I do is I check who am I?
Am I machine zero?
If I'm machine zero, do this.
If I'm machine one, do this.
So by doing that, I can cite a piece of code that everybody runs.
And everybody figures out who they are.
And if they are the given thing, what to do.
So here what it says is, OK, if I'm machine zero, my source and destination is machine one.
If I'm machine one, my source and destination is machine zero.
So I'm trying to communicate between each other.
So if you look at what happens is, first, I am sending basically to this machine.
So I'm sending something into this machine, so the syntax-- I'm not going to go through that.
You don't have to know that.
But what you need to know is that I'm trying to send something.
I tell explicitly who to send.
And there has to be matching receiving that data.
Otherwise, sends go somewhere.
And just it goes bad.
Send here, you can send it.
It can probably go bad.
But receive you have to have somebody who sends for that.
So the receive basically has to have matching.
And then you send it that direction.
And then what I do is I receive in here.
And this gets sent to me in here.
So question I did send receive here.
What would happen if I did also send receive here?
If I reorganized these two, what would happen?
If I used the same piece of code, that two pieces of code.
Then I don't even have do a bit to make this two separate code.
I can basically factor this out down here.
I do a send, receive; send, receive here.
And then send the just IDs.
What happen?
It works without a buffer
These things are what you called blocking sends.
If you have blocking sends, it means that until the receiver receives it might be blocked if you are doing a blocking send.
OK. That means if two guys are trying to send, nobody is receiving.
You have what?
Deadlock
You have deadlock.
So that's why I actually had to do this.
This is called blocking send.
So instead of blocking sends-- So of course, those are finalized things and do that up there.
I can do this one.
What this says is-- This is actually a little more complicated.
What I'm doing is I have a bunch of buffers here.
I have how many processors?
I have bunch of buffers in here.
I have, I guess, my ID number of processors-- no, numtask number of processors.
What I am sending, say I'm sending a circular buffer.
I'm sending around to everybody.
Both directions.
So I am sending the previous and next.
So assume something is sitting in numtasks.
I am sending back and forth.
So here what I am doing is basically non-blocking sends and receives.
So first time issuing a receive.
So even if I receive a receive, it says I have intent to receive.
But I am not receiving something.
I am not waiting.
So I can continue.
And then I am doing the same.
So otherwise, if I just do just receive and send, if you do blocking is going to be deadlocked.
But here I do that.
And then in this wait for all.
What it says is, OK, now I issued a receive.
Now wait until that receive is done.
So before I use the data, I have to wait for it in there.
And also, when I do the send, I am wait for all in here.
So why do you think it might be advantageous to do a non-blocking receives and non-blocking sends?
So sends, it makes perfect sense, because once I have sent, I won't do anything because I don't have to wait for anything.
I am done.
So blocking sends is not that useful.
But receives, why do you want to do non-blocking receives, then a blocking receive?
Because then you won't receive.
You have to wait till the data comes to do anything.
Because that's what non-blocking receives means.
I have to receive early and then wait for the receives to happen at this point.
What might be an advantage of doing a non-blocking receive?
Anybody can think of an advantage?
It's harder.
Because now you have to remove the receives from the synchronization point instead of writing one receive
Because when sends come from other machines, we can receive it
That might be one interesting thing because you are expecting multiple receives.
You don't know what's coming first.
If you do non-blocking receive, then you can be-- opt out the first guy then basically work on.
That's a very good point.
OK. What else?
What other advantages you might have?
Having a non-blocking receive?
If the receive fails, we can just have them resend it again
Receive fails?
OK.
Receive fails.
See, that's complicated.
But another interesting thing might be space.
Because when I see the non-blocking receive, I know where the data has to be.
So I already allocated a buffer for that.
So, if the data comes now, I can directly copy into my local buffer if there's already a received issued, if there's already a space allocated.
Normally, other way around, until you see the issue the received, you don't know where the data has to be, so it has to get copied at that point.
So here, you can keep the buffer, and hopefully, if you are lucky, the same hasn't happened yet, so you should set up the buffer, and then, when the data comes, say, aha, here is the matching receive.
Directly put it there by passing it and copying in the middle.
So that's the advantage here.
So, I am here.
I did a wait for the receives here.
Did the work that uses the data.
And wait for sends here, afterwards.
OK. Could I have moved wait for sends before the work?
What happens if I have moved wait for sends before the work?
Is it incorrect?
How many people think this is incorrect?
Is it incorrect to move this wait for sends?
All the sends before the work, to move this item about?
Because work is where all the work happens, I assume, that uses this data.
So wait for sends about what happens
Well, if that's incorrect, then you lose
Yeah, you lose-- you're waiting for something that you don't have to wait.
Of course, you can move these down, because that means you might start using, and try to use data that's not there.
So, this has to be here.
And this, basically, for performance purposes, has to be after.
So, of course you have to worry about a lot of correctness issues.
One is deadlocks.
So, there are two types of deadlocks.
That's blocking sends and receives, what we talked about.
But there's also other types of deadlocks that happen because of resources.
So, let me get to that in the next slide.
And the other interesting thing that can happen is stale data.
In your shared memory machine, need to update the data.
You know everybody's going to see that.
Because the hardware takes care of that.
But, in a message passing machine, it's up to you get the latest data when it's needed.
So, if you don't have the data, you think, aha, I have the data, but it might not be the right value because you haven't gotten something new.
So, it's up to you to basically send the data out and that.
And, also, robustness is a big issue because the fact that you have multiple machines means you can make it robust, but the other flip side is up to you to make it robust.
So that means you have to figure out if a machine fails, how to respond to that.
So, if you're waiting for a machine there for it fails.
OK?
There are a lot of issues, it time out, and then you have to go and deal with that.
So, that can make the programming a lot more complicated.
And if you just don't do that, your overall program would be a lot less robust than a single machine because there could be a lot more failures in the large system.
So, here's a kind of deadlock that can happen.
What I am doing is processor zero is sending and processor one is sending data to each other.
It doesn't have a read write deadlock because I am for sending, sending and then I am receiving, receiving.
The sends here, and the receives from here, the sends here, and then receive.
So normally it looks like I'm sending two things, It should go, and I am receiving that.
But, assuming that I am sending huge amount of data.
OK.
So I start sending, and there's not enough room for received.
Just say, OK, I don't have any room to receive, I have to wait until the data, at least a data start getting consumed to start receiving.
OK.
If you keep sending multiple of send outs, you might do a multiple of send outs in more than one, multiple of sends here, I might get deadlocked, I might get blocked because they can have receive, receive, and he's also trying to send multiple things, I might get blocked in here.
So I might get into this criss-cross situation.
If you were trying to send something, but other guy can't proceed, up to get received.
So even though, if program look like there's a nice matching send and receive, there's no cycles.
There's a cycle created by the resource usage in here.
So, if doing lot of sends before a lot receives, and vice versa, you have to be careful.
If you do too much of that, it's nice to block things, move things up.
But if you have too many things, then you might end up in deadlock situation.
Even though, traditionally, it might not happen.
So, you have host of other performance issues that you'll deal with.
So let me address couple of them and what it might shows up.
So one big thing is occupancy cost.
Because, when you do shared memory, minute you basically showing instructions.
Instruction goes executes, you are done with and that operation is finished.
When you are doing a message passing, each message is very expensive.
It has to do a lot of things.
You have to do a context switch, do a buffer copy, and a protocol stack processing, and then might have to do another context switch for that.
And there's a lot of these copying and stuff happening, and the network controller might interrupt the private system, because either there's data coming, copy the data in there, and you send as ignored application.
So there's this huge amount of things happening just for one message.
If you are sending like one lousy byte, or one even like kilobyte., it just doing a lot of millions of instructions, just on behalf of small amount of data.
So that's a large amount of cost associated with that overhead.
So, setup already is very high.
And, so what you want to do is you want to amortize the cost by sending large messages.
So what you were to say, so look I'm not sending this small thing, I'm going to accumulate a lot of things, I'm going to send everything as bulk if you can.
And then you can basically amortize these costs of doing things.
Other thing is communications is excruciatingly slow.
So even the memory system, it's about probably a couple of hundred plus compared to CPU communicating.
In the cluster interconnect, you can do tens of thousands of instructions in the CPU, by the time it get communicated.
In a grid, or if you are doing through the internet, and then it sits in the seconds now.
You can actually feel it.
And then your processor can run millions of instructions in that time.
And so if you are waiting for something, you had to wait for a very long time.
They might not have enough things to stuff in the middle, to kind of amortize the cost.
And then you have to worry about that.
So what that means is if you start waiting for something to happen, you are waiting for a long time.
So you have to figure out putting things in there.
Not waiting this, non-blocking things is very important because of that.
And so normally what you want to do is you want to have always split operations, that means you want to kind of initiate something at some point, very early on, and then use it later, especially if you're looking for something like a receive.
So if, I want to, especially if I want to get some-- normally in the shared memory, just kind of doing a simple thing, asking something and get replies, very simple.
Here, because if you just do that, there's a huge waiting bit rate, so you want to kind of do speed operations.
So, this code can be very complicated, because you tried to-- if you want to ask something, you ask very early, you do a lot of other things before the reply comes
[INAUDIBLE]
Oops.
OK. Let's see how many times I had to press that.
Before I realize I have to reboot.
So, if you want to rendezvous with-- normally, what have that means that two points has to kind of synchronize at the same point.
So what you want to do is you can do a three-way sender send a request, receiver acks with, OK, it's OK to send, and the senders delivers the data.
So that means I have to do a three-way communication.
Or, this alternative with two-way, you sender doesn't send anything, receiver basically send a request, and then you send the data.
So this could be faster because there's less things to do in here.
There's another method called RMA, or it's another name you might see, it's called active messages.
Where you don't ask the receiver.
When you send, you have some pre-assigned place you can go and dump the data.
So you don't wait for somebody to ask for data.
You said, OK, if I want send, I'll immediately send, and put it somewhere.
And the data that I send the place in here.
So, the first slide I showed you, all at that time, you saw all these big difference in the time, either this can happen, the message can go very fast, or sometime it will be really slow.
There's a big variation in here.
This happen basically because of the network communications.
How many of you know a little bit about TCP?
OK. Let me just explain about five minutes of TCP before I move on.
So TCP is one of the main protocols that we use to communicate over Internet.
And two things, you want to actually send data.
But also, you want to be a good citizen.
You have actually work in a way that it doesn't really take over the entire shared bandwidth you have.
So what TCP does, it has a window.
So what it says is, OK, I can send certain amount of data that's the size of the window, but I can't move beyond that until I get some acknowledgement.
So I send the window amount of data.
And, on the other side, when it's received that data, I said, I have seen this much of the window.
And once it's seen this much of the window this send that acknowledgement back.
And when that acknowledgement comes, you say, aha, that's good.
That means it has seen that.
Then I can send more.
I keep sending more.
And then, the TCP has this very interesting property.
If you are doing a really good communication, things are going very nicely, it starts increasing the window size.
It says, oh, OK, that means that windows size keeps going.
I can do bigger and bigger and bigger window size.
You can keep increasing the window size.
And then, what happens at some point, the system get overloaded, because if you everyone is start-- increase their window size, too many packets start coming into the network.
At some point, the network in the middle, doesn't have enough room.
It drops something.
So, TCP, nice thing about the network is that, even if it doesn't have a guarantee that it will guarantee it can just drop something.
And when it drops, what happens is the other guy is waiting for acknowledgement.
So waiting for data to come, it never shows up.
So then it has this thing, an ack, saying I never got it.
And the problem with that is, so you have a nice bandwidth, and you get increasing the bandwidth, you get faster and faster and faster, and suddenly data get missed.
And suddenly you have this big timeout delay.
And then everybody freezes.
And then get the ack, and you restart with a smaller window and slowly pick up, something like that.
So because of that, there a lot of times what happens, is this packet get dropped, retransmit happens, so you have this kind of sawtooth pattern.
Things get faster and faster, things go down for nothing, for a little while, again start again, in here.
So the other way of communicating is called UDP.
UDP says, OK, if you don't have any kind of acknowledgement, or something like that, I'll just send.
And you, on the other hand, figure this out whether you got something or not.
And send information back.
So the network doesn't participate in any kind of a balancing act of communication.
It's end-to-end.
So of course, you can be a really bad citizen, and say, OK, I don't care.
I just keep sending huge amount of data and then somebody would go well.
But, what people have found is for things like video, you can send UPD and kind of manipulate yourself end-to-end, can get much better than trying to do this TCP.
So the kind of thing is, even though there's not acknowledgment, or there's no real attempt to make sure all the data goes, UDP sometimes can get better bandwidth, because it doesn't drop packets in here.
So, there's a lot of great stuff, I mean you guys can take the networking class and learn all about the protocols and stuff like that.
There's really, really cool stuff in here, so some of you might actually, next couple of semesters, learn all about how these things work.
So I'm just giving you, lot of performance wise, these are the issues that you are to worry about when you are doing network level things.
OK.
So that's kind of talks about a little bit about a small scale, and then if you want to go to next bigger scale-- why you want to go?
There can be lot more uses.
If you are on something like Facebook, or Amazon, you have a lot more users to deal with.
If you are, what's a good one with a lot of data?
Google Earth, or something like that.
You have a lot of data.
And you have to deal with all the data, and that's a good way to do the scale up.
Or you might have huge amount of processing you want to do, for example, the one place a lot of data and processing is things like these new basically telescopes that's coming about, that has arrays of hundreds of different things, so you have huge amount of data coming from the telescopes.
And then you could do a huge amount of processing and that.
So that basically, has broad data and processing, and in things like webs, social networks, and stuff like that, gives me a lot of data.
So here are some examples of some things like the airline reservation system.
It's something, all the airlines have to assign millions of planes, flights, millions of seats that you deal with.
Things like a stock trading system that all the trades has to has to come there, and the prices has to get calculated, and then trades has to get validated.
And, very big analysis, so you form some kind of global understanding of what's going on.
And I'm going to talk about these two, three things too.
Scene completion and web search, which probably everybody knows.
So, yes, this kind of data, now, kind of a web analysis example.
So what these guys were trying to do was, every weekly, troll 151 million web pages, and get about a terabyte of information, and analyze page statistics.
So that's what they are trying to do.
Some come up with some idea about OK, what is the world happening?
How did the pages change the last week?
And then try to get a global view of that.
At this point, you have both huge amount of data and pretty large amount of computation power, that you had to build a system to do that.
This is where you need a larger system.
Here's another interesting system that people built.
So if you have image here, and the image has this nice background, there's unfortunate house sitting in the front.
So what this says is, OK, eliminate the house, search a very large database to find similar images and plop something in there.
OK.
So OK. You can get your face with some nice actual eyes, or something like that.
Just eliminate all the bad parts, and then and then get good parts and put them in there.
And so this one, basically, what they'll do was, that's about 396 gigabytes of images out there.
And so we had to classify images to get the scene detector, do color similarity, and do context matching.
So computation, what they're doing is about 50 minutes doing scene matching, 20 minutes of local matching trying to find right matching, and four minutes composing there, and then you can parallelize that and reduce this time to about five minutes.
So here's something that's huge amount of data.
You'll look a lot of things, you do a lot of processing to figure out we get the right thing and these actually keep increasing these images as we keep asking for more, more flexibility, and more accuracy.
Things can get higher and higher.
So really cool application that really require large data and large processing.
So, of course, the kind of clinical application is probably Google.
So in this research, you'll get some nice results.
So what people say, is this what two thousand process involved getting this query for you.
It takes 200 plus terabytes of data, but this is already old now, this could be even higher now.
And this takes ten to the ten total clock cycles for everything that needs to happen, for you to get to your query.
And you only get one sent for the query.
So that, not only are doing it fast, you are doing a lot of processing, you are doing it very cheap.
And I think one of the biggest things that Google did is figure how to get that done fast and cheap.
And that's why they so successful.
Oops, sorry, I didn't say this.
so it's one second response time, and the cheapest $0.05 average the cost, basically.
If you compute a time that's going to cost more than $0.05, is not worth it.
And you had to do it that.
So, this is Google, spend a lot of time how to figure out, how to do this is cheaply.
So, this is already validated, but Google is very secretive of what they do, so this is the closest I can figure out.
They have three million plus processers in clusters of 2000 plus process, each, in each cluster.
And what they already did was they went for the cheapest thing.
They build entire system out of the cheapest parts we can get.
x86 processors, the cheapest disks, fairly cheap communication, and gain reliability, redundancy though software.
So each part, I mean supposing in Google, this data center, there's somebody who's constantly growing and changing machines and changing disks.
Because there's so much failure.
But that means we have to have the software system to keep the things running in there.
And what they have is a partitioned workload, all those things are nicely partitioned and distributed through Google as this nice file system and stuff do that.
And then you have to do crawling, index generation, index search, document retrieval, ad placement, all those things happen in there.
Of course, other things like Microsoft and Yahoo, and all those other people have systems like that.
So this is kind of what, when you go in here to scale up, there's no other way, you have to actually build this huge system to do that.
So one thing Google does, going a little bit technical, is this have a system called MapReduce.
How many of you have seen, heard of MapReduce?
OK.
So there's all this people who know MapReduce.
Probably more than I do.
So the idea there is you have a bunch of data, a huge amount of data in here.
And, normally, what you have to do is find some similarities in lot of data, and do some processing for that.
And this is programming model set up nicely help doing that.
So, that this borrows lot of functional programming.
What that means is I'm not changing data, I'm always taking some data values and creating something new.
I'm never changing something existing, that's basically meaning of a functional program.
So MapReduce has two components.
First the map.
That means given some input value and a key in there, what you develop generate is some intermediate results and output key.
You get bunch of values coming through, and everybody process each one as separate.
And say, OK.
So here is the output key, and here's some intermediate value.
And then what you do is things with the same output key gets sorted into one list.
And then it's going reduce it.
And the reducer takes the output key in this list, and say, OK, look I'm going to process the entire list down to one element or small data item.
OK?
So let's go through a little bit more, digging deep into that.
And so you map, basically get a huge amount of records from the data source, and it fits into this map function, and it produce intermediate results.
And the reduced function, basically, combines the data, and all the folding-- let me give you an example.
I think that will show you better.
So here is kind of architecture.
So you have a huge amount of data resources.
You have many, many sources in here.
And each of the data comes in to that, and the map will basically distributed by keys and values, so there could be millions and values.
And then, what you have to do is, wait until all the data, has done that.
And then cleared for the number of keys here, number of reducers.
So hopefully you wont have a lot of keys.
If you have more than two keys, you don't get that parallelism because then you would be too huge lists.
And then, again, what happens is these keys get paired to reducers to come the final value in here.
So what's the parallelism here?
What makes the parallelism go high?
Or, not have enough parallelism?
Yeah, I mean, first of all, you need to have enough, hopefully, multiple data stores so you get a lot of parallelism coming in here.
Map is easily parallelizable, because each choosing in here.
Reducer is the problem I think one.
Because if you have too many keys, too little keys, you are in trouble.
The other interesting thing in here is there's a big shuffling between here.
So that means data has to go all over the place.
So it's not something that, you got data and you process to the end you got data you process to end, every data has to kind of cross back, and so that's a huge amount of communication in here that could be bottle-necked too.
That can be bottle-necked, keys can be bottle-necked.
So map function runs parallel, creating different things.
Reduced functions also run parallel for each key.
And all values are basically processed independently because of that.
Also, the bottle-neck is reduce phase can't start until all the map is done, and also all the data gets shuffled around with that.
So here's an interesting example.
What I am trying to do is I am trying to count the number of words in assume huge amount of web pages.
So what I can do is in the map, I get each page in here-- I thread through the page emitting each word as my key and the count as one.
Because I only get one thing.
And then my reducer is basically-- my key is each word.
So if I have a million words, I can have a million reducers.
And the reducer basically takes all those things-- it's not that fun because everything is all at number one.
Because we count at one.
And then basically keep adding up how many things for each word came about and put up the results.
So you can say, OK, look, for the entire corpus of data, I had this many words count, this many word occurrences, this many all for each word.
You get a word count.
So basically trying to create a histogram here and MapReducer provides a very nice interface to do that.
And it's very nice, high level, and it provides this nice infrastructure to run this in parallel in here and do all the communication necessary, figure out how many reducers to run, look at machines to run them, produce the result, and give you the result.
So this is a nice infrastructure Google has built in there.
So in this level, when you go to this-- this is the data center level.
What do you have to do to scale?
You need to distribute data.
And you need to parallelize because if all the data is in one machine, it doesn't help.
And you need to have parallelism to scale everything.
Another interesting thing you can do is approximate.
So what that means is normally when you calculate, when everybody has exactly the same data all the time-- because when you write the memory, everybody sees that memory-- you have the perfect knowledge of the word.
And in a distributed system, getting perfect knowledge is very expensive.
That means every time something changes, you have to send everybody that data.
And one way that people really make these systems run fast, you say, wait a minute, if somebody doesn't have the perfect knowledge, if there's a little bit of discrepancy between something, I am OK.
Assume you have a new-- you changed your-- we'll say-- web page and added a couple of new words in there.
Next second, the search doesn't see it.
Nobody's going to complain.
And then you can deliver that.
If you do a search, you'll find something.
But somebody else doesn't do it because that data haven't propagated that to both of the things.
Nobody's going to complain and say, wait a minute, I found it, but he didn't.
It can have a little bit of a lag.
And that can be really, really useful in these kind of systems.
Because every time something happened, you don't have to keep updating.
But tell me a system that you can't actually do that.
Play it a little bit fast and easy
Stock trading
Stock trading.
Yeah, that's something basically, if you say, yeah, you might get it too, you might get it-- and that doesn't work.
Basically, stock trading has this very particular thing because when it does submit, we'll say, a sale order, within a certain amount of time, it has to get matched up and has to be announced to both people.
And also, there are a lot of other constraints like if the machine goes down.
Either trade has to be everybody saw the trade or nobody saw it.
You can't say-- somebody says, I sold it.
And another guy says, no, I didn't buy it.
And when you have millions of billions of dollars back and forth, that doesn't really work.
So for that, there's this thing called transactions.
So transactions is an interesting way-- a lot of databases have this transaction.
Transactions say, look, I am doing this very complicated thing.
And I cannot have this intermediate state going on.
So what transactions say is, first, tell me everything I want to do in the transaction.
So it might be I want to sell a stock, I want to buy a stock, whatever.
And then at some point when you commit the transaction, you either say, OK, everything worked, good, the entire thing gets committed.
And you are done.
Or it can explicitly reject it.
It can come back and say, look, I can't do this transaction.
Now sorry, you can restart it.
So you can accept and reject.
But then the nice thing about that is then every one single-- it's like atomicity.
Every one action doesn't have to happen immediately or happen as a group.
You can say, OK, I'm doing a bunch of action in the transaction.
And then finally, I can come in and if it works, great.
So what might be a reason you might not be able to commit a transaction if you do a transaction?
Anybody else want to answer?
When you say, I want a transaction, I want to commit something, what might say-- so here's an interesting thing.
In the stock trading type world-- so assume I want to sell something.
Let's look at the airline reservation.
So assume I have an airline seat in here.
And if two people want to try to resell that seat, I can do all this processing in parallel for everybody until I come to the commit point.
That means I can look at everybody after you enter the data, do the price, all those things separately for the same seat.
But then when you come to the commit point, you say, can I commit the transaction?
At that point, only at that point, they have to figure out whether there's a conflict in here.
And at some point, if there's a conflict, it says, oops, can't.
One transaction has to get aborted.
The nice thing about that is most of the time people are not going to fight for the same seat.
And then things can proceed in parallel.
You don't have to wait.
Otherwise, if you do that, there might only one seat assignment at a time you can do.
And that's really not going to scale.
So everybody tried to get their seat.
They go to the end, and they say, can I proceed?
And at that point, you check whether there's a conflict.
And if there's a conflict, one guy backs out.
So that's the transaction.
Oops, I'm going to get rebooted I guess.
So when you go to planet scale, you can get even into more issues, things like-- what could be a planet scale thing out there?
What's an interesting planet scale thing that you can think of?
Single computation that has to happen in the planet scale.
Something like Internet naming system.
It has to work everywhere in the entire planet.
Or something like Internet routing.
There has to be an algorithm that has to work.
The entire world has to cooperate and then make sure that all the traffic actually goes to the right place.
So there's a lot more issues, interesting things show up in here.
So things like Seti@Home type stuff-- these are a little bit dated these days, that happens-- distributed all across the place.
So if you do planet scale, it has to be truly distributed.
There cannot be any global operations, no single bottleneck.
And you have to have distributed view with stale data.
You cannot say, look, everybody has to have the same data.
You have to have everything distributed.
And it has to add up to load distributions because things can keep changing in there.
So what I'm going to do next is trying to give you a little bit of a case study that shows you some interesting properties that show up when you start building at that scale.
And this has some planet scale type properties, some cluster properties, whatever.
And I will probably first describe this interesting problem and then show what kind of solutions that came through, so to give you a perspective for a problem in here.
Any questions up to this far for distributed systems?
It's hard to do distributed systems in one lecture.
There are almost closest for distributed systems.
But this will give you a feel for some of it.
So the case study here is from VMware.
It's called deduplication at global space.
And the problem shows up when you're trying to move virtual machines across the world.
You have this virtual machine.
So what virtualization did was it took a piece of hardware and it converted it into a file.
So each machine is now a file.
When you have a file, like hardware, there are a lot of cool things you can do then.
You can replicate those files.
So suddenly, instead of one machine, you've got tens of hundreds of machines.
You can move those things in here.
And of course, you can start another machines all over.
So once you are able to move these things, the issue becomes how to move those things around and what's the cost of moving something.
And also, you can store it, store those things in a database.
The interesting thing that's happening these days is cloud computing.
Cloud means there's all these providers all over the place, saying, I have processing power, I can give you some of them.
Amazon does something easy too, but Verizon is trying to do, everybody is trying to do that.
So if you want to have the best market, what you want to do is have the elasticity to move from cloud to cloud for many reasons.
So sometimes the cloud might be too small.
You want to get to a bigger cloud in there.
Or you want to be near the users.
So in the daytime in the US, you want probably to move the machines through US.
At night, there might be users in China, so you want to move your compute nearer China.
Because it will be closer to the people who are using it.
So something like that, you can move around there.
Or you want to find the cheaper provider.
If somebody comes and says, look, I can give you compute power $0.10 cheaper than what you are getting, OK, I want to move to that guy.
And also, to amortize the risk of catastrophic failure.
If there's a hurricane approaching somewhere, I might want to move to a data center that might be out of the way.
And I want to do that.
And the interesting thing there is a lot of things.
But when you say, application in the cloud, it's a machine, a machine basically in a virtual machine.
At the same time, a virtual machine has to get moved around, not your small application.
The entire thing has to move around.
And virtual machines are hefty.
Because it has an operating system, it has all the software.
There's so many things now.
Then your data and your state.
There are a lot of things in the machine in here.
And all those things have to get moved around, so that can be expensive.
So yes, interesting experiment in here.
So the idea here is to try to move something from Boston to Palo Alto on a 2 megabytes network.
And there are a bunch of different virtual machines in here, VMs.
And it takes-- whatever-- 3,000 minutes to move these machines from-- this is 500 minutes.
That means what?
Some hours to move them.
Some hours to move these machines around.
And then what you say, look, the machines are heavy, big.
Why can't you first compress the machine?
So you can use something like normal compression.
So blue is basically compress, move the machine, and decompress.
So here is something interesting.
This is a very fast compression.
You did really well, you are moving there.
If you want a better compression, you say, I'm going to do a full, best compression I can do, it's actually slower.
Because the trouble is the compression time is so high, the reduction is not usable in here.
So you try to compress, you spend most of the time compressing.
So actually, this is even slower that just sending without compression.
So compression is important, compression is useful.
So can you do better than a normal compression in here?
How can you do better?
So some key observations in here.
So a large part of these files are executables.
You have your Linux kernel, whatever, to all those executables hitting in there.
And basically, that's monoculturing the world.
There are no million different executables.
You are a Linux kernel, there's only a certain amount of versions.
Microsoft XP, there are certain types of versions.
So even though there are millions of machines, inside the millions of machines, there aren't millions of different applications.
There's only hundreds of different applications.
And so your motion moves.
If you think about it, you are moving the same thing again and again and again.
Can you take advantage of that?
And there's even substantial redundancy in each of these.
So this is very interesting.
If you have a Windows machine, each DLL has three copies.
So you have the copy, and then the Installer has a copy.
And then there's another copy in the next version to basically back out, so undo copy.
So each thing is kept three copies.
So every big thing, they're seeing multiple copies in there.
So that part is there also.
Even within a single disk, there is redundancy in here.
And another interesting thing is many of the disks have a large amount of zero pages.
So if you send something uncompressed, you send a huge amount of zeros.
So you are waiting for zeros to get in there.
Even easy compression can get to those zeros, but this is a large chunk of data in here.
And so the interesting thing is if you take one virtual machine, this is the number of non-zero blocks.
And this is the number of unique blocks.
So unique blocks are smaller than non-zero blocks.
But if you keep adding more and more virtual machines, then of course, the number of total blocks keeps going up.
But the unique blocks doesn't keep increasing.
That means the second Linux box you add, there's not much new in there.
So if you look at that, what happens is the first guy has about 80% things are unique.
When you keep adding things, it's about only 30% is unique after you add.
Because it's the same program.
Only the data is different as you keep adding.
So can you really take advantage of that?
So that is where deduplication comes in.
So deduplication says, I have this data, I have a lot of redundant data.
So A B, A B, A B is redundant.
So what you want to do is break it up to some kind of blocks in here.
And then one easy way to do is calculate a hash.
Because you don't want to compare blocks.
That's too much.
N squared comparison of blocks is a lot of comparison.
You can have some kind of hash calculated for each of these blocks.
And then you can compare the hashes.
And if the hashes are the same, they are the same blocks.
And then what you can do is you can eliminate most of these blocks in there and then keep hashes for each block-- only the hash.
And then what you can do is you can only keep the unique blocks in here.
So even though you have nine in here, only five different unique blocks are there.
So that's a nice way of deduplicating.
So you actually have what you call recipe, a common block store in here.
So one way to do that is you can have a recipe on common block store for each of the systems in here.
That's the tradition of deduplication.
Or what you can do is have everybody keep a recipe and only have one common block store.
Just keep one, single common block store, and everybody have a recipe, or probably cache of a recipe.
So by doing that, you can even reduce a huge amount of the things happening in here.
So the interesting thing is if you are keeping one common block store, who can keep, who can manage?
That's the interesting question in here.
So can you keep instead of common block store for each processor, each computer, can you keep the common block store for the entire world?
So if you find most of the common blocks in the world, keep one store.
And the nice thing about that is then I can go anywhere in the world.
I can ask for the common blocks for the common things.
I can populate it myself.
So here's this interesting system called the Bonsai.
What they did was-- so if you have a block in here, you calculate a hash function in here, get a hash key.
So what that means, the hash key can uniquely access this block.
And then what you want to do is ask that you want to compress this block.
This additional step, I'll explain later why it's needed.
So the other thing you can do is you can get a second hash key and use that as a private key to encrypt this block.
Because you calculate two hash keys.
One is the hash key to identify.
The other one is a private key to encrypt this block.
And then what you can look at is this global store to see whether this hash key exists.
If the hash key exists, then say, I got the page, here is the page.
That's the encrypted page.
And each page will have a unique ID.
And so here's my unique ID.
If you find the page in here, what you can do is you can only store UID and this private key and get rid of my page.
So storing UID and private key is sufficient to get my page and unencrypt it.
Why do you think I have to compress here?
Why do I have to basically do encrypt here?
What's the interesting thing about encryption?
Why encrypt?
Assume this is global.
Because if you don't encrypt, it might be a common page, but it might not be something you want everybody to know.
So assume a large company like Google.
President of Google, Larry Page, sends everybody emails, saying, this is very private, but here is something that's happening in the company.
And it will get into everybody's mailbox.
And suddenly, it becomes-- aha-- a common page.
And it gets sucked in the world because of the common page.
And now everybody can see that, and that's not good.
But now if you have this private key-- if you don't have the private key, I can't decrypt that.
So what happens is-- let me go through what you can do in here.
And then what can happen is if you had these two, UID and private key, you can go to the global storage and say, here's the UID, give me the page.
It has a page.
It better have the page for that UID.
Get the page out of that.
And then now I can use my private key to decrypt it.
And then of course, I can decompress it in the original page.
So by doing that, I can have this global system that keeps common pages in there and store them.
I can basically, really eliminate the storage.
Because the nice thing is this could be, we'll say, 2K pages.
And this is 64 bit, and this is probably 256 bit.
So there's a huge compression of data in there.
So here are the kinds of decisions you are to make.
For example, hash key.
So each page is represented by a hash key.
But you can have two hash keys, two pages mapping to the same hash key.
So my god, you're not unique.
So why is it OK?
Low probability
Very low probability.
And in fact, what they looked at was they calculated the disk failure.
There's a higher failure of disk failing and losing data than hash collision.
So you can say, look, you're keeping that in the hard drive, so there could be more chance of disk failure than hash collision.
So therefore, hash key is-- you can do that, low enough probability, you can get away with that.
Actually, I want to skip the rest of this in there.
So here is the comparison how much compression you can give.
Some of them, you almost can compress 100%.
Because if you have a newly installed Linux box with very little data, everything is in the global.
So by doing that, basically, here is the communication.
The compression is cheap.
You don't have to do too much because you just do this hash comparison to do that.
And then now the communication time is even reduced because you communicate a lot less.
And then you expand.
So all these three are actually now much faster to send a machine across.
So here is a total size of all the VMs.
The interesting thing is if you compress within, if you just do compression, most of this is zero blocks.
You can eliminate the zeros and do a simple compression in here.
And if you do local deduplication, if you eliminate that, and if you do global, you'll get to this point.
So you get to about basically 30% of all your data.
And yeah, there's more data in here, but just-- So that's an interesting global level system that people are building today.
And I think if these kind of systems appear in many different places.
These days, a lot of people are building things like cell phone games and things like that, has this huge back stores, back computation.
All those things have this large scalability in there.
And so the nice thing about performance engineering is if your application doesn't require a huge amount of computation or very fast processing, if you're going to have millions and millions of users, or if you're expecting millions of users, then building these systems, and understanding, and building scalability is really important.
And a lot of the things we learned in this class is directly applicable there.
So I have... Any other questions before you guys can go finish your project report and go have a nice Thanksgiving dinner?
So everybody is thinking that they will be able to get a good handle on what they are doing for the final project?
Oh come on, there are so many cool things you can do with this project
We're working on it
And there are-- whatever-- three or four iPod Nanos waiting.
So it makes a lot of sense to actually really focus this one.
This is a fun project.
This is, in fact, a fun project.
Because talk to these guys what they did last year.
People did a lot of interesting things.
Because we gave you freedom to actually even change the algorithms.
So you can actually look at the algorithm.
If you know a little bit of physics and graphics type stuff, look at them and say, look, can I even reduce computation of how the computation is being done?
So people got a lot of wins by doing things like that.
And of course, parallelization matters.
And there are a lot of optimization possibilities in this piece of code.
So take a look at that.
Just have a plan.
Just don't go blindly into it, just have a plan.
Run it, profile it, get some feedback.
You need to get this for your presentations.
So run, profile, get some feedback, have a good plan.
Go attack it.
So see you in a week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Can everyone hear me?
So I guess it's 2:35, and we might as well get started.
So the calendar, I think, still says that we have a guest lecture today but sorry to disappoint you, there is no guest lecture.
Instead we have a quiz for you on Thursday, which is so much more fun.
Not really.
But I do realize that everybody has a lot to do, even just for this class.
You guys have your final project and so on.
So hopefully this will be helpful.
It won't take the entire 1 and 1/2 hours, I don't think.
And the remaining time will be up in office hours, answering questions about stuff.
So the first thing is I decided to run some stats that we haven't run before for this class.
These are your Project 42 4.2 final submissions.
And in the red are your beta submission run times.
And in the blue are your corresponding final submission run times.
I thought it was just interesting to see some groups did massive improvements, while others didn't really change their programs much.
And just to give you an idea, a 90% for your performance grade is indicated by that green line and below.
So the majority of the class did really well.
All right.
So the thing that everybody's concerned about.
So Quiz 2 will be in lecture on Thursday.
It'll be an 80 minute quiz, similar in format to the previous one.
And you guys once again are allowed a one-page handwritten crib sheet.
And it covers everything that has been done since Quiz 1.
It's not designed to be cumulative, but unfortunately due to the nature of the class, there's going to be some things from the previous part of the class that you would need to know or it would be helpful to know.
Unlike last time, we didn't post a practice quiz on Stellar, because it turns out that the material and the order of the material gets shuffled around quite a bit every year.
And we have different focuses.
So posting a practice quiz would actually be pretty mean, because it would cause you guys to study the wrong things instead.
Instead I try to extract the most useful things from our discussions of what the quiz is going to be this time.
So hopefully all this information is helpful.
Feel free to interrupt me at any time if you have questions.
So the first part, I would roughly call analyzing and possibly writing parallel programs.
And to start off, is everybody familiar with what one of these graphs represents?
So you have work and then each circle is a unit amount of work.
And then the graph shows the dependencies of the things.
So just to start off with a really simple question, what is the total amount of work represented?
[LAUGHTER]
Uh oh.
I heard 17, which is not what I got when I counted this last night at 2:00
26
24
I like 24
[LAUGHTER]
And I'm going to assume it's 24, and not embarrass myself by trying to count it.
Now what is the span?
8
8
8
8 sounds right.
So the longest path-- there's actually a couple different paths that end up being 8, but I think 8's the right answer.
So then parallelism?
3
Cool.
All right.
So now that we're done with that, how about a little bit of background on caches.
So when you run programs in parallel, it causes some more interesting things to happen in cache that did not happen with serial execution.
One of those things is true sharing-- cache misses-- which are caused when multiple processors, like multiple threads, are trying to operate on the same piece of data in memory.
So as a result, your cache would be forced to synchronize that cache line and move it across-- copy it from the freshest processor to the other ones every time that it's requested.
And you end up with a lot of overhead generated by continually doing these copies.
So ideally you don't want to do this.
But it's a lot easier said than done.
False sharing is a little bit trickier because it's not immediately apparent from looking at your program that you're having multiple threads do the-- operate on the same data.
Instead with false sharing, you have multiple processors that want to operate on different locations in memory.
But due to the way that your computer's cache is organized, they just happen to be in the same cache line.
As a result, you end up with the same memory, the same overhead, that you would see with true sharing.
But when you look at the code itself, you don't think that you're doing anything that involves multiple threads operating on the same data.
And so with that, here's a piece of code.
And my question is this true sharing or false sharing?
Let's take a vote.
How many people think it's true sharing?
Nobody.
Well, how many people think it's false sharing?
Come on.
You have to vote one way or another.
Let's try this again.
True sharing?
False sharing?
That's better.
All right, so it is indeed false sharing.
And can someone propose a solution to fix it?
Yes?
Separate A and B, and put them in different shelves
Would that be sufficient?
Putting them in different trucks?
Then, padding the trucks
Yeah.
Maybe with a little bit of padding, depending on your system.
But sure, I'll accept that.
But on the quiz, you might be asked to propose solutions to problems that you find.
And hopefully those will be easier.
Now on to a more fun topic-- synchronization.
Yes?
I have a question on sharing.
So if two programs only access data, does that still count ?
If you only access the data and nothing in the cache line is being modified, then you're safe, because every processor can independently read from their own copy of that
So that doesn't count
That doesn't count, correct.
But the problem is if your cache line is 64 kilobytes, and any thread on any processor changes one piece of data inside that cache, then everybody has to pull new copies.
So the classic example to demonstrate synchronization correctness is the dining philosophers problem, where you have n-- I believe five in this case-- philosophers, and five chopsticks.
And each philosopher has to eat.
So they use mutexes to-- well, generally you need-- each philosopher's code would be pick up two chopsticks and then eat and then put down their two chopsticks.
It's somewhat of a contrived example, but it actually comes up in a lot of cases.
And one of the cases where it comes up is in the system that I use to grade your unit tests.
So I got impatient about running 200 or so unit test in series.
And since I had about 200 cores altogether on the cloud machines, I figured, why not do them in parallel?
But the problem is I need to run-- if I have students A and B, I need to run student A's test against student B's implementation.
So the rough way that I did that was I grabbed a lock on student A's repository, grabbed a lock on student B's repository, and then modified-- swapped in their test files, did all my testing, and then returned the files to the original conditions, and then let go of the locks.
Unfortunately, when you do that, you run into much of the same problem as the dining philosophers problems.
So the naive strategy is for each philosopher, you grab a lock on your left chopstick.
You grab a lock on your right one, with respect to you.
And then you eat and then you unlock them in the same order.
So this has a bug.
Can anyone point out what it is?
You want to unlock in a certain order
Actually, it doesn't matter which order you unlock in in in this case.
So bad things happen if everybody starts at the exact same time.
So everybody grabs their left chopstick.
there's five people, five chopsticks, so that works fine.
Now everybody wants to grab their right one, but it's not there.
Because the person to the right already grabbed it as their left.
So they wait forever for the right chopstick to appear and it never does.
So the situation is called a deadlock.
And it's obvious to spot, because all of your threads are going to be stuck doing absolutely nothing.
Your CPU is not doing anything.
And you're just sitting there.
So a very common work-around that people like for this is to use what's called a try lock pattern.
So what you do is you grab a lock on one of them, like, say, your left one.
And then for the right one, instead of directly going and trying to grab the lock, you ask-- you basically do-- if the thing is unlocked, grab it.
Otherwise, don't grab it.
So it's a non-blocking approach to trying to get a lock.
So it either succeeds and you have the lock, or it instantly fails.
And if it fails, then you switch the order of which-- you put down the chopstick that you grabbed, and then you try the next one.
Maybe you wait a little bit before then.
This is more or less than what I did for your beta test, for your unit test tester.
When it fails to-- if first grabs one student's directory's locks.
And then it tries to grab the second one.
If that fails, then it puts down both locks, and then it waits a little bit, and then it tries again.
So what's the problem with this approach?
Yes?
It can still hold up
What's that?
It can keep waiting and grabbing, and sometimes it will contend
Yeah, pretty much, the same problem happens.
Only now you have the case where everybody grabs their left, and now the right one's gone.
So they put down their left one.
And then they all go to grab the right one.
Now they have the right one, and now they want the left one.
But the left one not there.
So in theory, they can end up doing this for a very, very long time.
But in practice what happens is because of scheduling overhead, and the phase of the moon and whatever else affects computing, you'll end up after a while, this become distinct from this pattern and eventually all the locks go through, and you're OK.
But in the meantime, you spent a lot of time spinning, trying to grab locks.
And so this situation is called a livelock.
And it's a lot more annoying than a deadlock because it happens-- it tends to be less reproducible.
And it's harder to diagnose, because if you're looking at top or your favorite CPU monitor, you see that the program seems to be making progress.
Maybe some of them actually get the lock and go through.
But the rest of them don't.
So it's hard to argue whether or not there's correctness issue.
Now there's various classes of solutions to the dining philosophers problem, but you guys can go look that up.
I'm not going to go over them.
So another issue with synchronization is suppose you have two threads.
Let's say this is a Mars Rover, and it's on Mars.
And the two things that the programmer wanted to do are send back its debugging log, so that the nerdier people at NASA, I suppose, can read them over, or whatever they do with logs.
And the high priority, what everybody actually wants, is for the Rover to send back pictures.
Now both of these things require access to the radio that's supposed to transfer something.
And only one thing can transmit at time.
Otherwise, the transmission is garbled.
So a reasonable approach might be to just lock the radio in each thread before you transmit, and then unlock after you transmit.
So what could be a potential issue with this?
One might get all the time
Let's say, the scheduler interrupts every 10 milliseconds, and then 1 out of the 10 times, it allows 10 logs to run.
And then 9 out of the 10 times, it will allow some pictures to run.
So everybody gets time scheduled.
So I guess the easiest way to illustrate this is what happens if it's currently running said logs, and it grabs this lock.
And then while it's transmitting, let's say on the second megabyte, the scheduler says your time is up.
And then it swaps in this program, and it tries to run.
So now when this program runs, it tries to lock the radio, but the radio's already locked.
And the one that locked the radio is the send locks thread.
And so it could sit there and spin for a while waiting for the lock to be available.
But that's not going to happen.
Because the guy that's holding the lock is a bug.
So in general, this is called starvation, where in the process of having a shared resource, you end up unfairly depriving someone else of a resource when you're not truly using it.
Well, I guess that's not the-- I guess you are using it, but you're using it for longer than is fair for you.
Like in the scheduler you wanted 90% to one process and 10% to the other, but in reality you end up with a completely different set of priorities.
And this inversion of priorities is commonly called priority inversion.
Does that make sense?
Cool.
So on a completely different topic.
So Cilk hyperobjects were covered extensively in lecture.
You guys had a problem set on it, although it was not received very warmly from what I see in the problems sets write ups.
But I think all of the basic material for what a Cilk hyperobjects is and what it does can be found in the lecture material.
But more interesting, let's explore what you can do with hyperobjects that aren't in the documentation.
So I'm going to define a piece of code here.
And it's C++ or Cilk++ code.
And the first part of it defines a very simple structure called a point.
And it has three members.
Or actually it has two members, x and y, and it has three methods for setting the values of x and y and also getting the values of x and y.
Question?
No?
OK. And then there's also a hyperpoint class which contains some extra Cilk junk.
And it also implements the same three methods.
Set, get x, and get y.
And if you look at the monoid that it contains the point monoid it implements a reduce function that just doesn't do anything.
And for the public methods, the setters and getters, they grab the point represented by the local view to the reducer that it contains.
And it calls the method that you called on that point.
So the question is what does this thing do, the hyperpoint object?
Anybody have any wild guesses at this point?
That's a no.
So OK. Let's try using this code.
So to put the example into perspective.
Suppose I have this artificially contrived function that takes an array of 100 points, and it tries to do an in place reversal of its elements.
So one way that it can do that is you have a global temporary point.
And you, at every step of the iteration, first you step through half of the list.
And then at every step of the iteration, you copy the ith element into the temporary variable.
And then you set the ith from the other end to the-- you swap the two.
And then you-- yeah, so if it is a three-variable swap, I guess is the simplest way of explaining it.
It's a swap using a using a temporary variable.
Anyone not get what that code does, despite my attempts to confuse you?
So as far as this space complexity of this procedure, really what it needs at minimum is enough memory to store the temporary point, and then one register to use to put in the temporary values.
You can think of a solution that uses only one register to update values within your point structure.
And so let's suppose I want to do this in parallel.
And so what I do is I call it parallel reverse, and I replace for with Cilk for.
So am I done?
What's wrong with my code?
Race
Yes.
Well first it takes that much-- it takes the same amount of space.
But there is a data race on temp, because multiple threads could be updating temp.
So now what I do is I replace my point with a hyperpoint.
So now what does the hyperpoint do?
So within each iteration of the Cilk for loop, so if that iteration has been-- if the work of that iteration has been stolen or spawned off.
Then the hyperobject, because it has a reducer within it, it'll get a new local view which is a new point object to use.
And if it's not still in that it continues running in the same thread, then it actually just continues using the local view that was previously instantiated.
So no matter what Cilk does in the background, you're guaranteed to not have any data races on temp, because whenever you spawn off a new thread, the hyperobject will instantiate a new temp object for you to use.
And this is an example of what's called thread local storage, so like a temporary storage space for each thread.
Any questions?
Cool.
So on the quiz, you might be asked about various abusing of various other novel hyperobjects that don't do the typical thing, which is take a left and right and reduce them.
They might do something a little bit quirky.
And you might be asked to explain what they do
Maybe?
Yes?
Maybe.
Maybe
But this is purely for illustrating that you can do such a thing with hyperobjects.
Because you can always create a temporary variable there
But suppose point was a much larger object.
Sure, this is a contrived example.
But yes.
Anyone?
Everyone gets this?
So, in that case, why would that problem be more efficient than creating a new copy of it, say another point?
Because if you think about the case where this code runs on a single processor with no work stealing, this version does the exact same thing as the single threaded version.
No extra objects are being created.
And if you're running on however many processors and Cilk decides what to do in the background as far as how many threads to allocate for you, and your code doesn't have to know anything about that
What's the monoid again?
The monoid is the associative operator that has an identity and a reduce operation and some other things that I forgot.
Any more questions?
Cool.
So switching gears, and this time I actually have a placeholder slide for it, next part is more about like the types of things that you might have learned from doing or experience you might have gained from doing your projects.
And it primarily pertains to lab oriented, like in practice hands on things, as opposed to theoretical concepts.
So just as a fun warm up, everybody has hopefully at this point used some STL data structure.
Vectors should definitely look familiar.
So let's compare some operations on these three.
Because, as far as the API is concerned, both list-- all three of them support almost the same set of functions, as far as Insert, at pushback, pop front.
All those operations are supported by these things.
So it's kind of-- You might ask why would you use one over the other?
Well, it turns out that they have slightly different performance characteristics that are useful in noting.
So anyone want to compare and contrast like what data structure a list corresponds to compared to a vector?
So which one would be like a linked list?
List, yep.
And then vector obviously is a dynamically growing array.
And a deque-- how many people have used the deque?
Cool.
One.
How many people know what deque does even if they haven't used it?
Three?
Four?
OK.
So a deque is actually implemented similar to a vector, but it's dynamically growing on both ends.
OK, so let's start with filling in the table for the list.
So inserting and removing at the end of a list.
Constant time?
Linear time?
How many people say constant time?
Linear?
Half and half, wow.
Don't rely on your neighbor during the test.
So it's actually constant time.
If you think about appending to the end of a linked list, it's just a matter of couple pointer operations.
How about inserting and removing at the front?
Constant?
Cool.
Linear?
OK.
I'll stop picking on people now.
So next is inserting and removing an arbitrary offset.
So what I mean by that is you have an iterator to a particular point inside the list.
Now I say insert this element five or n elements away from that iterator.
How long does that operation take?
How many people say constant time?
Three people.
How many people say some sort of linear time?
I agree with you guys.
It is indeed a linear time thing, because you have to traverse-- you have to actually fall the next pointers to the location or the previous pointers to the location that you want to insert.
So although list might appear to support randomish axis, like you can add numbers to the iterator, it doesn't actually support that operation efficiently in the background.
Now the next one is also kind of tricky.
Checking the size.
Like checking how many elements are calling the dot size method of a standard list.
How many people think that's constant?
Cool.
About four or five.
How many people think that's linear?
One.
How many people think it's possible to have a linked list data structure that's forced constant time?
There we go.
Well, unfortunately, the STL only requires you to have a big O of n time for this.
In fact, I peeked inside the code for glibc the lib standard C++ that is used on the cloud machines.
And in fact, what they do is they actually walk through everything from start to end, and they count up the number of times.
So in fact, it is a linear time operation, although I'm sure somewhere out in the world, there exists a C library, C++ library that does linear time, or constant time, size look ups.
So you might want to be careful about that one.
On a different note, there's actually a dot empty method for checking whether or not a list is empty, and for any type of STL collection that is guaranteed to be constant time.
So if you just want to know whether something's empty, don't use dot size equals 0.
What about reversing a linked list in place?
Linear?
More than linear?
It's not a technical term, and Charles is probably going to hurt me now.
Linear?
Yes.
Linear.
What about in place sorting of a linked list?
How many people think it's N log(N)?
One or two daring souls.
Three daring souls.
A well informed daring soul.
How many people think it's more than N log(N)?
One.
OK.
So actually it is N log(N).
And that's actually not specified by the specs.
But it happens to be.
And it turns out that there is a variant merge sort that works on link list that is N log(N).
I didn't know that, but anyway.
So how about let's fill this in for the vector.
Inserting and removing an element at the end-- constant time?
Linear time?
Nobody says linear time.
So it's amortized constant time, I guess.
I'll just call it constant time.
What about inserting or removing at the front of a vector-- constant time?
Linear time?
Come on, guys.
I'm getting like three hands for linear time and zero for constant time.
I guess that's reasonable.
So it does, in fact, insert one element.
It puts the element in and then it grows the vector in.
It copies everything in.
So don't insert at the front, don't insert or remove at the front of a vector.
It's a very bad idea if you do that a lot.
What about inserting or removing at an arbitrary location inside a vector?
Well, that one's fairly obvious to be some sort of linear time too, because it has to shift all of the elements after it.
Checking the size of the vector?
Some people are getting pessimistic about the STL.
That's my personal opinion, but I won't transfer that onto you.
So it turns out that for a vector, yes, it is constant time to check the size.
So that's OK.
In place reversal, linear time.
And sorting is N log(N), which is not surprising by any means.
So a deque?
Inserting or removing at the end-- constant?
Sure, constant.
And inserting or removing at the front?
Cool.
Constant.
So that's the primary difference between a deque and a vector.
And for everything else, it behaves exactly like a vector.
Does that makes sense?
Yes
So is list like a doubly linked list?
A standard list is in fact a doubly linked list.
So it supports both forward and reverse traversal.
There's a standard S list, which is a singly linked list, and it only supports forward traversal.
Any other questions?
Moving on to a slightly different topic, can anyone point out what is bad about this code?
And I mean from a performance standpoint
It's O of n
All right.
Someone said it, so it's O of n. So the vector that's passed in here is passed in by value, which means whenever you call this function, it'll actually make a brand new copy of the vector and copy all of the elements.
So it's linear time with respect to the number of elements in the vector
So how do you fix that?
You can pass in a pointer
So there's two ways.
So on the left is the original code, and on the right are two different ways to pass in just a pointer to the same thing.
So you can do a reference pass, which is the first version above.
And then you can use a dot to access the methods.
Or you can pass in by pointer which everyone is already familiar with.
And that's where you have to use arrows to access the elements.
And then you also have to change the way that you call the function.
So oftentimes if you just catch something in code, like say the final project where something is passed by value, where it should be passed by reference, oftentimes the easiest way to modify the code is to just change it to use a reference rather than a pointer.
So the next thing that I want to cover is a question type that will be on the final, will be on the quiz.
And I'll give some examples of problems of this form.
But most importantly, I want everybody to be familiar with the form of the question, the rules involving the question, so nobody accidentally loses points for not reading the instructions, which nobody seems to do on quizzes.
So you are given two versions of the same function written in C or C++.
And you are to assume that your compiler-- well, the compiler, as far as like what the CPU is executing is a literal interpretation of the code you see before and after.
By this point, everybody is probably familiar with some of the simple tricks a compiler does, like if you say int x equals 1 plus 1 there is no circumstance in which the compiler actually puts two 1s in two different registers and tells it to add it and then put it in x.
So you are assuming that your compiler is doing a literal translation.
And then you are asked to determine three properties about the optimization, so going from the before to the after.
So the first thing is whether or not the optimization is legal.
And when we define legal, we mean that the optimized version always has the same-- preserves the same behavior as the other version.
It should be relatively simple to infer from the code snippets that we give you, what the intended purpose of a function is.
And you are to determine whether or not the optimized version acts like the unoptimized version, as far as the correctness of the result.
Second, you are asked to determine whether or not this optimized version is faster than the previous version.
And by faster, we mean faster in most conceivable cases.
It's a short answer question so you might want to explain what you mean.
But we do expect a yes or no answer to start off the question.
And it should be pretty clear whether the answer is yes or no.
And then finally, I'm sure by this point everybody knows that when you compile with gcc -02 or -03, the compiler does a lot of things for you.
So there's oftentimes no point in writing certain optimizations by hand, when you know when you hit the compile button, the compiler's going to do it in the background anyway.
So this question asks whether or not you can reasonably assume that the optimizing compilers, like the ones that you're using for your projects, would do this optimization for you or should you take the effort to actually do this optimization by hand.
And then, legalese, if the optimization is illegal, let's not answer whether or not it's faster or automatic.
And if the optimization actually makes things slower, then let's not answer whether the compiler does it for you.
Some of us have more cynical views of the compiler than others.
So we'll just disregard that.
Any question on the rules?
How are we supposed to know what types of things the compiler's going to do, can we just read the documentation?
So there was a lecture that Samaan gave called What Compilers Can and Cannot Do
So it's just stuff from that lecture?
I would say most of it is that lecture.
And then other things-- we might throw in a thing or two that you guys hopefully learn from.
So throughout the term, we've been misguiding you on those walk through P-sets, suggesting you to do things that actually don't help.
And you guys have gotten pretty pissed at us about that probably.
But there was a method to the madness.
I mean, there was something to be learned from that, whether or not we actually meant to do it.
So hopefully some level of intuition would answer it, and you would get the bulk of the points for just understanding the points raised in the lecture.
Any other questions about the rules?
All right.
So I have a couple of examples of these types of questions.
So the first one is this.
So it should be fairly obvious what the code does.
It takes your input, and it wants to mod it by 256.
So first question is this a legal optimization?
How many people say yes?
A lot of people.
Anybody say no?
Reed almost scared me for a moment there.
He had his hand up.
So yes, it is indeed legal.
And it's a standard bit hack for doing a modulo by a power of two
What if it's 257?
I was fortunate enough to make it a uint64.
So next question is given that it's legal, is it faster to do the and instead of the mod?
How many people say yes?
How many people say no?
So a majority of the class says yes, and some people say no.
So what would be a reason why you think it's not faster?
Well, I didn't say it's not faster faster.
It's probably the same time, because I assume the instruction for modding is not actually dividing by it
So for the other people who said no, how many people agree with that line of reasoning?
All right.
Seems like everybody.
So unfortunately, it is faster.
And it turns out, I compiled down both of these pieces of code under-- well, actually, the compiler-- but anyway, I'll get to that next.
Not giving away any answers.
But so I executed the assembly instructions corresponding to both of these operations.
And it does turn out that the compiler, the CPU, when you ask it to take the mod of an arbitrary number, usually it's an instruction called div mod.
And what that does is that it does a divided by b, and it puts the dividend in one register and it puts the modulo in another register.
And we do that for, let's say, 2 to the 63rd modulo 256, because it actually goes through all the clock cycles for doing that division process.
And then it dumps out the modulo at the end.
So yes, this actually does tend to be a lot faster.
Now next question is will the compiler do this for you?
How many people say yes?
Most of the class.
Anybody say no?
Good.
We all trust the compiler to some extent.
So yeah, it turns out that gcc, starting from 01, will do this for you.
Any questions?
All right.
Next question.
So you have a list.
And you pushed in.
You pushed at the end.
It's 42, twice.
And then you returned the result of popping the front element and then popping the back element, subtracted.
So first of all, is this a legal operation?
Let me give everybody a chance to think
Legal optimization as opposed to an operation
Yeah.
How many people vote yes, this is going legal?
Roughly half the class.
How many people say no, this is not legal?
Two people.
OK, so why would it not be legal?
You can't do that.
Yes?
There happens to be wastefulness on some problems
Well, that could certainly happen.
But for the purpose of these problems-- see this is why I'm bringing up these problems-- so for the purpose of these problems, assume there's no man behind the curtain doing things.
For any given problem, you could say that's someone attaching a debugger to your program and changing the memory between instructions.
So let's just assume that's happening
Also, who's local, right?
Yeah.
Yea, Foo is actually local, so if anybody else's doing anything to it, they're a very bad person.
So fine.
It's a legal optimization.
Now.
is it faster?
How many people think that returning zero is slower than pushing two things up?
[LAUGHTER]
Depends upon how big zero is
That's true.
So OK, so yes, obviously, it's faster to just directly return zero.
And finally, is this an automatic optimization?
How many people say yes?
Cool.
Nobody said yes.
How many people say no?
People who say yes, if you could work for compiler companies, I would appreciate that a lot.
So unfortunately, the compiler does not make-- cannot reason through how an STL list works, and realize that what you're doing is an no op.
It turns out that for a related set of questions, like have any of your mentors said anything to-- so my mentor said this to me last year.
And I was pretty interested when I tried it out.
Have you ever used a for loop to set everything in an array to equals 0?
Most people seem to have done that.
Who has used the memset or heard of the memset set API call before?
A good portion of the class.
OK, so some people.
So a little bit of background then.
Mem set instruction takes in an array or a pointer rather, and a constant, and the size of the array.
And it sets that many elements of the array to that constant.
So it turns out that depending on the machine that you're using, there is more efficient and less efficient ways of setting all the elements of an array to 0.
And functions like mem set are actually are several pages long of hand optimized assembly to be optimal for your architecture.
So it's always tempting to take a for loop that iterates through an array, setting everything to 0, and changing it to a single mem set instruction.
And it turns out that for both gcc and icc, Intel C compiler, when you compile at sufficient optimization level, the compiler actually does this for you.
It actually looks through the arrays.
It actually realizes that you're setting all the members in the array to a constant, and it does that optimization for you by replacing it with a memset call.
So just as a general question, why could it be advantageous for you to write the four loop as is, and have the compiler do the optimization for you?
Because you later might add code in the loop
True.
Any other reasons?
I didn't hear question
You might want to add code that changes the behavior inside the loop
It's much more readable to have a for loop and you just read it, and I get that's what he's doing
Yes, indeed.
Looking over some of your code submissions, it seems to be a non-trivial task of figuring out the size of something when you call malloc or memset.
And actually, it sometimes is a non-trivial operation to figure out how big something is in bytes, especially for structs and things like that.
So it actually is easier to let the compiler do the optimization for you, and you don't make any embarrassing mistakes in the process.
So next question.
So here you have a helper function.
And in the after version, seems to do the same thing, does it?
So who thinks this is a legal optimization?
Half the class.
Anyone think it's illegal?
So yeah, it is legal.
The only thing that I've done is that I've copy pasted basically the function body in the bottom example.
And it's relatively easy to prove that or infer that the two do the same thing.
So is it faster?
How many people say yes?
How many people say no?
One person said no.
Why is it not faster?
I mean, I've said that it is the same speed, because so the only thing that would make it slower would be the-- you need to actually do like a compass switch.
Because you need to call that function every time.
But I think it could provide the concept for you-- like basically, it's a static function.
It would inline it even though you don't have to declare it
OK.
So if you recall the rules for the question, so basically we want you to interpret the code, as is, like a very literal interpretation as in-- the point was raised that for this function call, what you actually want to do is assume that the compiler is setting up a function called by putting the arguments in the right register and jumping to the address of that function above.
So for the case of this-- for the matter-- for the purpose of this question, yes, it's much faster to just do a multiply compared to setting up a call stack and tearing it down after the function is done.
Now the next question is is this automatic, which was more or less answered.
And that's yes.
So at a sufficient optimization level which is -03 or -02 with dash f inline functions, the compiler will actually use a heuristic that it has for determining whether or not a function is a good candidate for inlining.
And for good candidates, it'll do the inlining for you.
Now the specific heuristics vary from compiler to compiler.
But for this specific problem that we gave you, it's a prime candidate for inlining, because it's declared as static, which means it's local to this C file.
And it does a very simple task, a one-liner, so it almost certainly will be inline for you.
Any questions?
Cool.
So this one.
Question?
I don't get it.
Could you explain again?
I know I've asked this before, like what's static, the static function?
And it's different than in Java?
OK.
So the question was what's a static function in C. And so in C, a static function means when you declare a function as static, it means that the function is local to the current file that you're work-- the current module, the dot-o file, which is almost always your dot-c file.
So like the function only exists within this dot-c file.
A function in another dot-c file cannot call this function.
So it basically allows the compiler to assume that there's no external effects on this function that are not within this file.
Remember the compiler compiles object files one by one, and then the linker is the one that brings it all together
So the important thing is that it knows that it can just delete helper after it's gotten rid of it
Right, right.
That's another good point.
So in this case, after inlining, the compiler actually won't bother to put in the function helper inside the actual object file.
So you save on my code space.
Because the compiler knows that nowhere else this function can be called
What about static in C++?
Static in C++ when used inside a class means that the function is a static member function.
So it's a function that's tied to the class itself, rather than an object, and instance of the class
Just like Java
Just like Java.
More questions?
Cool.
So back to this code.
First of all, is anyone not familiar with the syntax or wants the syntax explained?
OK, cool.
So first question, is it legal?
How may people say yes?
Half the class.
How many people say no.
Roughly half the class.
Well, I and the people grading the quiz will side with the people who say no, because down here when you're doing-- this is the correct version of the XOR swap.
I'm not trying to trick anyone there.
But it turns out-- well, I'd rather not assume that t, the arbitrary type name, has an operator XOR defined for it that has all the properties that you need in order for it to do this.
Like the simplest example is if I pass in a double, this would not work.
Double does not have an operator XOR defined for it.
Some people look upset
[LAUGHTER]
A list too would not have an XOR defined for it.
Anyone disagree with me?
That's not a challenge.
So yeah, some of these questions require you to do a little bit of reasoning, like reasonable reasoning, as far as what to assume for the purposes of the question.
So watch out for that.
So in this case, it's not legal, so you don't have to answer the next two questions.
So OK.
So let's fix this.
And by fix it, I mean not use templates.
So now we're just operating on ints.
And ints do have the proper operator XOR that has all the good properties that you want.
Yes?
What's the and notation of the variable
It's a reference.
So it's a reference to it.
Because you need references, because you actually want to swap the two items that are being called in.
Right?
Can you use like pointeres?
Yes, so you can also use a pointer for it.
Pointers and references, C will treat this use of references the same way as if you use pointers.
So references are syntactic sugar.
And what it basically allows you to do is write code that looks like you're operating on standard intervals, standard integers, but in fact that it's a pointer to the value that the name of the variable that's passed in here.
Otherwise this code would be riddled with a bunch of stars.
Now is this legal?
How many people say yes?
Most of the class.
How many people say no?
One person.
Yes?
If A and B are the same variable, you set it to zero
Yes, exactly.
The XOR law has a nasty side effect where you want to make sure that A and B are not the same variable.
Not they don't contain the same-- they can contain the same value in different variables, but they can't be the same variable or the same register or the same memory location.
Otherwise, when you do A equals Ax or B, that's 0.
And now both A and B point to 0.
And there's nothing you can do to get your old value back
Can you check for equivalence of recordkeeping in C++
Indeed, you can.
If you take the address of the reference, it gives you the address of the original value.
So this would be the proper way, I guess, of doing an XOR swap.
So now that we've gotten over all the trick questions, is this actually faster?
How many people say yes?
One, two, three, four.
How many people say no?
Cool.
Most of the class.
So this is another example of a gray area question.
So in this class, we primarily talk about the cloud machines, as far as the performance analysis that we've done.
So on the cloud machines, and this was mentioned in Charles's bit hacks lecture which was, well, the second lecture in the course.
This trick is not faster on the cloud machines.
Because what you end up doing is you create a lot of dependencies between these operands, so that the pipeline must execute these one by one serially.
So you introduce a lot of delays by that.
And plus now that we actually want you to write a correct program, you have to do this check, and that's an extra branch.
And finally these XORs have to be done by the ALU, and up to five execution ports.
Only three of them are ALUs.
The other two can do memory operations just fine.
So it turns out that for a combination of these reasons, this version indeed with the temporary register actually does run faster.
Any questions?
Yes
For branches, we assume advanced features, like it will preload the exceptions and only invalidates the thing when certain branches fail
Sure, you can assume cool properties about the chip, such as speculative execution, prefetching, and so on.
But the problem is those tricks are fine when you're doing things sparsely as in you're sometimes doing work and other times not doing work.
But like speculative execution, like when you execute both sides of a branch, you're using more or less double your execution or resources.
And if you have all processors running some intense piece of code that's doing swapping, you don't have any extra resources to dedicate towards running code that might not be useful.
Any other questions?
Cool.
So finally, I got tired of making slides.
Well, actually that's not true.
It's more I'd rather concentrate the slides on things that I think are more useful than other things, as far as per preparation for your quiz.
All your time is valuable.
And so there's other topics that are fair game for the quiz, because they've been mentioned in lectures up to this point.
But I haven't prepared any slides on them.
And the fractal trees were covered by the guest lecture by Bradley.
And the slides are up.
I suggest familiarizing yourself with the basic workings of how the data structure that he described works.
And lock free data structures were covered on your take home P set, and also in a lecture.
So be prepared to answer potential questions about that.
And then finally the mechanism behind the Cilk runtime, the work stealing scheduler, how it's implemented on the Linux side, and also the basics of what a Cilk hyperobject is, what a Cilk hyperobject has to have.
Advantages and disadvantages of using them and so on.
Yes
Are the grades for the P-set out?
No, they are not out yet
Are there solutions for the P-set?
Not yet.
But I believe we have something prepared for that, right?
Somewhat?
Yeah, it's somewhat prepared.
If you have any questions about like questions on the pset that you want answer to, feel free to stop by office hours or email us and we can help you.
Yeah, apologies for not getting that P set back to you in time.
Any other questions relating to the quiz.
It's really all the material I have as far as reviewing.
Yes?
Are you going to put these slides up?
Yeah, I'll put these slides up right after the lecture.
So as far as a general notes, I guess you guys turned in your finals yesterday, and you guys have a design report that you also turned in.
Your POSSE members should have contacted you by now, as far as scheduling a design review session.
Since this project is actually due on next Thursday, it might be a good idea to get this done by the end of the week.
Otherwise, the design review won't be a very useful process
There are a couple of students without a mentor right now here.
We are fixing that
Right.
There's a couple of students caught in limbo right now.
Sorry about that
If you were-- previously dropped on Stellar, send us an email.
If you haven't been to drop through this course, there are no longer drops allowed.
Yeah,
So it's your responsibility.
I'm going to put on my mean hat.
It's your responsibility to schedule a meeting with your POSSE member, and make sure that it happens by the end of the week.
And if it doesn't look like it's going to happen, or you're having difficulties getting in touch with your mentor, or you don't have one yet, please drop us an email or come by office hours and make sure that we know.
All right.
That's it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And so today we have Ravi , who is I guess a performance engineer at VMware, where he basically looks at how to get really good performance in many, many different things in VMware.
So Ravi is an MIT undergrad.
I guess a long time ago
Yeah, '92
'92.
There you go.
And then he got his PhD from Stanford and end up in VMware looking at a lot of interesting performance stuff.
So today, he's going to give us a kind of a bird's eye view of how these kind of technologies, what you guys are learning, is useful and how it's used in industry.
OK, Ravi.
Take over
First question, can everybody hear me?
And is the mic on?
OK. Good.
I have a bit of a sore throat.
So I'm sorry if you can't hear me.
The other thing I want to say is I'm actually really glad you're here.
Because I remember very distinctly in 1990 when I was an undergrad, I was taking 6.111.
And I had a bug in my final project.
It just wasn't working.
And I was going to skip the lecture that day.
But I was like, whatever, I'll go.
So went to the lecture.
Turns out Don Troxell pointed out something in that lecture.
I was like, that's my problem.
And I went back, fixed it.
Everything worked.
And I was very happy.
I don't think that's going to happen here.
But it would be nice if it did.
So that's cool.
So instead of just talking about what a performance engineer does, I would-- I'll just-- Power is coming in.
But-- Oh, there it is.
Perfect.
So instead of just talking about what performance engineering is about.
I just wanted to go through 10 bugs that I've actually been involved in.
And hopefully, they'll be interesting to you.
Because each one of them was interesting to me.
So let's start out with-- I just wanted to start this out with a bang.
So for dramatic effect everybody read through this slide.
It's basically an email thread from one of my colleagues.
And raise your hand when you're done.
So I'll know when you're done.
OK.
Anyway, so the point of this is that his question ended up being, why is the percent lock time in the database increasing despite a lighter load for the thing that uses that database?
Now, if you're like me, you read this email, and your first question is what the hell is he talking about?
So that's why titled this slide like that.
So let's actually deconstruct what, a, that email meant.
And, b, what it meant to me.
So in order to do that, I want to first talk about the system that he was talking about and help describe what the actual problem he was seeing was.
So here we have a standard multi-tier set up.
What we've got is that we've got a server, which is our middleware server.
And it talks to a database.
And then we have a bunch of hosts on the other end.
Those are labeled by agent, HAL, whatever.
Anyway, those are hosts running virtual machines.
And here's the setup.
You've got a client.
And that client issues a command to this middleware server.
And that middleware server tries to figure out what host or what virtual machine needs to be contacted.
So the middleware goes ahead and does that.
And it performs an operation on that host.
And after performing the operation, it then archives some of the information that it gained to the database.
And once it archived the information to the database, it went ahead and notified the client that the operation itself was done.
Fairly simple.
Send a request to a server.
Server contacts some other host.
Host comes back to the server and says it's done.
Server persists information to the database and then notifies the client everything is done.
So the actual problem that my colleague was seeing was, he was sending less and less load to this middleware server, which therefore means that less and less goes to the host and less and less goes to the database.
However, the percent of time spent in the database was higher.
So despite a lighter load and less stuff going to the database, the percent of time spent in the database was higher.
Probably, all of you are smart enough to know what the actual problem is.
But we'll go through it step by step.
So the deal is this.
The first thing I did is I examined the amount of time that each lock was held for each of these different operations.
And what I did was I varied the load just as my colleague did.
And what you see are six different lines.
On the x-axis is each of the different locks that we acquire in our system.
And on the y-axis is the latency of those locks in milliseconds.
And basically, the axis for this-- the legend for this chart says Locked-4, which means a certain type of load, all the way down to Locked-128.
OK. And the deal is that when you have lock-128, what that indicates is that there's 128 hosts per thread.
Which, take my word for it, that means that you've got a lighter load.
So basically, the lowest line in the legend is the lightest load.
And the highest line in the legend is the highest load.
And as you can see, although it's a little bit obscured in this chart, the bottom line is that this is about time you're spent holding a lock.
In this chart, lock-4 indicates a heavy load.
Lock-128 indicates a lighter load.
And what this chart indicates is that, in fact, you are spending more time in the locks with a heavier load, lock-4, than you are with a lighter load, lock-128.
So the latency prologue is higher with a higher load, which is totally reasonable.
And it turns out that if we translate my colleague's question into reality, what he's asking is why is the percent time spent in the database higher, even though you have a lighter load?
Well, the answer ends up being that when you have a lighter load, overall, you're spending less time in locks, which means that any time you spend in the database, if that's a fixed cost, is going to be a higher percent of the overall time.
So it turns out, there really wasn't an issue.
There was nothing odd here.
It's exactly what you'd expect, because the database imposes essentially a fixed amount of latency per operation.
And in order to confirm that, I also looked at the time spent contending for locks.
So you really want a lock, but someone else gets it instead.
And you have to wait.
And the trend is exactly the same.
When you have a lighter load, you end up having a much lower overall latency.
But that just means that any latency you spend in this database, its percent impact on the entire operation is quite a bit higher.
So to take a look at this and to do a postmortem on this case study, his first thing was some random email, which kind of was nonsense.
But it was less nonsensical once you actually tried to understand what this setup was.
So step one is figure out what is the experimental setup and what are the parameters that are being studied.
The next thing to understand is what's being measured?
I remember what my colleague said.
He said the percent of time in the database was pretty high.
Why is that?
Well, OK.
So we have to understand what this multi-tier setup is, why you would be going to the database, and what it means if the time spent in that database is high.
And I guess the next thing you want to do is you want to say, well, OK, he's complaining about lock latency.
So I must have some sort of visibility there somewhere.
Let me go and look at the appropriate metric to figure out what's going on and why time is being spent there.
And finally, it's appropriate to-- once you've got all this data, don't make any assumptions-- just draw the appropriate conclusion based on the data.
The problem is basically that with a lighter load the impact of that database becomes larger, even if it were the same under a heavier load.
Because under a heavier load other aspects of the system also get stressed that don't get stressed in a lighter load.
So, and this is basically what I'm saying here, yeah, the percent lock time in the database overall is higher.
But the overall lock time is smaller.
So everything is behaving exactly as you would expect.
So this was one simple study and hopefully it didn't require any virtualization specific knowledge.
What I'm going to do now is go over a bunch of other different case studies that I've looked at and crystallize what lessons I learned from doing those studies.
So I guess the next case study I want to talk about is something that I call garbage in, garbage out.
I think that's something I learned on the first day of 001.
So here's the deal.
You've got a customer.
And he wants to draw this chart.
And this chart is basically, on the x-axis is time and on the y-axis is some metric.
It happens to be CPU usage megahertz.
But it doesn't really matter.
OK.
So he wants to get this chart.
And it turns out-- if you remember the diagram I showed before, we have a server and a bunch of hosts at the end.
So the client simply has to send a request to that server.
And the server will generate the data, send back to the client.
And it turns out that if any of-- all of you, I'm sure, are familiar with Pearl, Python, and all these different scripted languages.
There's another one that's very popular in the Windows community called PowerShell.
And so it turns out that our server has a PowerShell interface.
You can write a PowerShell script to get any statistic that you want.
And then you can write that to some database or some spreadsheet and get this charge.
So here's the deal.
We have a user.
And what he does, as I've indicated at the bottom, is that he wants to get the CPU usage for the last hour.
So he just writes a simple script, using PowerShell.
It doesn't really matter what the syntax is.
But the point is, he does what's called a get-stats call.
And he indicates, OK, I want this virtual machine.
I want this stat, whatever.
And then he gets his information and writes it to a file.
No big deal.
It's just a simple script to get some statistics.
Now, here's what is actually interesting.
I wrote that script using PowerShell.
Now, let's compare a PowerShell implementation to a highly tuned Java implementation of the very same script.
Now, what I've done here is I've shown the performance of that very same script when getting data for one virtual machine, which is the first row, six, about 40.
And then up to a little over 300 virtual machines.
Look at the scaling of this script.
In the middle column, I have the performance for this PowerCLI script that I just showed you on the previous slide.
As you can see, it goes from a healthy 9.2 seconds to over 43 minutes if you increase the size of your inventory.
In contrast, if you use a Java-based implementation, it starts a little slower than PowerCLI at 14 seconds.
But the scaling is far superior.
It ends up at 50 seconds.
So the question is why?
What's different about these scripts at scale versus when I wrote the little toy script?
Now, to give you some color, why this matters to someone like me is that every once in a while, I'll get a call from a system admin, who wants a very simple script that does blah.
And they say, I wrote this simple script in my development environment, and it worked fine.
But then I run it in my production environment, and it completely breaks.
What's going on?
So often, problems like this actually occur.
Now I'm calling this garbage in, garbage out, because you can see that with PowerCLI and Java, we're getting very different results.
The deal is that in PowerCLI, I wrote a very naive script.
And I'll explain how in a moment.
And that very naive script has not performed at all.
Whereas in Java, I wrote a very highly tuned script.
And it ended up taking about an order of magnitude less time.
So garbage in, garbage out.
Good stuff in, good stuff out.
Let's try to understand why.
OK.
Here's the deal.
All of you are probably familiar with the difference between scripting and high level languages, or low level languages or whatever you want to call them.
So the idea is that if you're running something using a shell script or running Pearl, or whatever.
These things are usually meant for ease of use.
But they're not necessarily meant for performance reasons.
And so they hide a lot of details from the users so that it's easier to come to a quick prototype.
You don't have to declare variables.
Do all this kind of junk, right, if you're using scripting language.
In contrast, if you're using, Java, C++, whatever, you have Typesafe and all this nonsense.
You have to worry about what the declarations of your variables are.
Make sure they're correct types, et cetera.
So they tend to be more difficult to get running.
But once you get them running, you can use a lot of advanced tools that just aren't available necessarily in toolkits.
And we're going to talk about how this difference manifested itself in this particular problem.
I said that when I told the customer to write a simple PowerCLI script, they just have to do one call.
Get-stat.
And here's a bunch of arguments to tell it what statistic it wants to grab.
It's a toolkit.
It's doing a lot of work for you.
What's it doing for you?
Well, it turns out it's calling our internal APIs.
Our internal APIs have to work for a variety of different platforms.
So they have to be incredibly general.
Under the covers of that single get-stat call, we have about seven different steps that are going on, ranging from asking, OK, I want a stat for a virtual machine, what kinds of stats does that virtual machine export?
How frequently does it update?
All kinds of information that you probably don't really care about if all you want is a single stat.
But you might care about if you're writing a production quality script.
So the point is get-stat does these seven calls.
And because you're using a toolkit, it hides them all from you.
It spares it.
It's very easy to use.
But it turns out, this is obviously not the most optimal way to do this.
And I illustrate that by showing you conceptually what's actually going on.
On the right, I guess on your left, I've got what the PowerCLI script was doing.
You call this get-stat call, or whatever, and what it does that for every virtual machine that you want this statistic for, it goes through and does all these random steps that were done that I described in the previous slide.
Figuring out what stats are supported, how frequently they updated, and then actually going and getting the stat.
It turns out, as I show on the right, when I do this in Java, I can take advantage of the fact that a lot of those things that I'm asking about in the toolkit, they never change.
You've got a good virtual machine.
It supports things to be refreshed every 20 seconds or whatever.
That never changes.
You want to find out what statistics are supported.
Well, those never change once you've configured it and you don't change anything.
So why do that every single time?
So as we show in Java, you can pull a lot of that stuff out of the main body of the loop.
And if you wanted information for every VM in the system, you only have to make one call, as I show at the very bottom, for every VM.
You don't have to make seven different RPC calls.
In fact, you could further optimize, if you really understand the API, and collapse a few more calls in Java that you can't do in the toolkit.
Because again, the toolkit's trying to talk at a high level, so any random dude can actually write a script.
OK.
So the point is in this PowerCLI world, a very simple toolkit world, we were making five RPC calls for every VM that we wanted stats for.
Whereas in Java, we made one call.
And we can even optimize that further.
And I don't show that here.
But anyway, the bottom line is that's why the performance of the Java script was so much faster.
So the idea is that with PowerCLI what I do is I wrote very simple, a script that anybody could understand and use, but it's not optimized.
I didn't utilize multi-threading.
I didn't realize that the output format of the data was really, really verbose.
Again, that's the kind of thing that if you really understand the APIs you can take advantage of.
I also didn't realize-- as a customer, I wouldn't have realized that I make this one call but it expands it to a bunch of different network requests that I didn't know about.
In contrast, with Java, I was able to take advantage of the power of a language like Java, where I could thread pools.
I could optimize the return format of the data.
And I could reduce the number of RPC calls.
I guess the analogy I would give is if you think about assembly code versus compiler-generated code, nobody here wants to sit down and write x86 Assembly.
So, of course, you write C and you use GCC and whatever.
But maybe there are some things that would actually be faster if you wrote hand assembly.
OK.
So, first example we gave was just understanding what on earth the problem was.
The second example was if I write bad code, I'm going to usually get bad output.
In the third case, I'm going to talk a little bit about the design of an API.
And I'm going to motivate that by an example.
We have an administrator on the right hand side.
And he's talking to the management server.
This management server is responsible for telling that administrator what's going in every host in the system, every virtual machine, whatever.
So let's pretend that the user wants-- so this is a virtualized environment-- so the idea is that the user wants to actually access his or her virtual machine.
They're running a Linux virtual machine.
They want to see its console.
So they can type at the console.
Here's how that roughly works.
They want to see the console of the VM that I've circled.
So the way it works is that this user talks to the management server.
The management server locates that virtual machine, establishes a handshaking connection.
And that connection is directly given back to that user.
So now the user can interact directly with that virtual machine and send commands and do whatever.
So we've all got the setup.
Very simple.
Now here's the deal.
The problem was that our poor little user, poor big user, whatever, they could not start this console when they were managing a large number of hosts.
To take you back to this diagram for just a moment, on the right hand side of the screen, I show a number of hosts.
And I show a number of virtual machines.
So what I'm basically saying is that this administrator could not access the console of his virtual machine as the number of hosts increased.
There should be no link between these two.
But there was.
So I got involved in this problem.
And here I went to the client folks.
And I explained the problem.
And their response was that's obviously a server problem.
Next step, I go to the server folks.
And they said, yeah, it's obviously a client problem.
So then I went back and thought a little bit more about the architecture.
And what I knew is that whenever the client needs to initiate this handshake connection, it has to spawn a certain process.
So that process is called the VMRC, VMware remote console.
So I talked to the client folks again.
And said, well, I think this is happening.
Oh, yeah, it's a VMRC problem.
OK.
So that just passes the buck another way.
So then I talk to the VMRC folks.
And I said, OK, I think I have this problem with starting up.
And they said, oh, it's probably authentication.
It's probably this.
You have to enable verbose log in on your end host, blah, blah, blah.
And at some point, I got so sick of this.
I thought, well, this is complete garbage.
I'm actually going to take a look at this a little more carefully.
And so, once I got involved, I got a little bit more information.
And the information was actually pretty useful.
Here's the deal.
When you're starting the remote console on one of these virtual machines and you want to connect, it turns out the person that was having this problem-- when you ask a few more questions, you get a few more answers.
Some of which are actually useful.
So I said, well, if you do this with 50 hosts, do you see a problem?
So 50 manage hosts, he only wants to view one VM's console.
No problem.
Then I asked him-- and then I actually sat at my desk, and we reproduced this.
So then I said, let's try it with 500 hosts, no problem.
Just a little bit slower, but no problem.
So again, the set up is 500 different hosts, he just wants to look at one VM.
And it's taking longer and longer to view that one VM.
Finally, he gets to 1,001 hosts.
And that's when the problem occurs.
That' when he cannot see this console.
So obviously then, you go back and you think, well, gee, is there some magic number of 1,000 hard coded anywhere, such that we exceeded it, and that was the problem.
And frankly, the answer is no.
So let's figure out actually what did happen.
So I made a bunch of different observations while I was debugging this.
When you have fewer than that magic number of 1,001 hosts, here's what was going on.
Remember the sequence of events.
I'm a client.
I talked to a server.
Server talks to the virtual machine.
And then that virtual machine and I are directly connected.
What I was noticing was that when there were less than 1,000 overall hosts in this setup, when I would initiate the command for this console.
The CPU usage and memory usage of this middle management server would gradually increase, and increase, and increase until the console would actually show up.
So you right click.
You want to pull up a console.
A little window comes up.
But it's totally blank.
And then eventually, after the management server consumes a lot of CPU and memory, eventually that console comes up.
So now your question is what's going on that's causing the CPU usage of this management server.
And why is that scaling with the number of your virtual machines.
So the first thing I did-- so one of the themes of this talk, I suppose, is that there's a lot of great tools for performance debugging.
But sometimes there's no real substitute for common sense and for the things that are easiest to do.
So the first thing I did is I looked at the log file of this server.
Here's the rationale.
If every time I invoke this remote console, I'm doing more work on the server.
First, let me just see what that server is doing.
Now, it turns out that what I'd noticed about the server was that whenever the client would contact the server and say I want the console of this VM, you would see a data retrieval call from the client to the server.
And that data retrieval call got more and more expensive with more hosts and more virtual machines in the system.
So the question that I asked was, why on earth are we doing some stupid data retrieval call if all I wanted is to say, here's a VM.
Tell me where it's located so I can talk to it.
So it turns out-- that was problem number one.
Problem number two, that's what happened when you had 50 hosts, 500, whatever, that you basically spend more and more time in this data retrieval routine on the server.
However, once you hit that magic number of 1,000, you would see this call start.
But you would never see it end.
And in fact, you wouldn't see it end.
You'd see some exceptions somewhere.
So that to me was a clue that I better take a look and see what's going on at that exception.
So here's what I did.
Unfortunately, the exception gave me no information whatsoever.
So I went ahead and I attached a debugger to the server.
And when I did, I found out that it was an out of memory exception.
So here's the deal.
I'm a poor admin.
I send a request to look at a console.
For some reason, some data handshaking code takes place between the server and the client.
It takes longer and longer with more hosts.
Eventually, when I have too many hosts, that thing ran out of memory and silently failed.
And so my client's sitting around looking dumb with a totally blank screen, because it has no idea that its request basically got denied.
So the question is why is that happening?
And aha, this is where the aha moment occurred.
I happen to know that one of the reasons in our setup, that one of the causes for this data retrieval, is that as a client, I want to know certain attributes of the virtual machine that I wanted to view.
For example, if I'm looking at the console for a virtual machine, I might care that that virtual machine has a CD-ROM connected.
And I might care if that CD-ROM got disconnected or whatever.
That's not something you'd necessarily associate with a virtual machine.
But when I go to the server to connect to it, the server wants to give me that information so that the shell, which is showing me this console can appropriately update if the virtual machine gets disconnected from its CD-ROM, whatever.
Anyway, the point is I looked at the code that this magical VMRC routine was doing.
And here's what it was doing.
It was saying, look, I want to start a remote console session.
I need certain information like the CD-ROM connectivity.
I'm going to go ahead and get that information for every host and every virtual machine in the system.
And then once that's done, I'll go ahead and display the console for this one virtual machine that I care about.
Well, this is clearly just a stupid idea.
So I called the guy up and I said, you realize this is stupid idea.
And he said, well, why?
It's not that expensive.
Is it?
And I was like, OK, we need to have a long discussion.
And bring a priest in here.
But that's a terrible idea.
So what we ended up doing is-- And so now-- OK, so that's the problem.
The problem is we drew a huge data retrieval.
None of which we need.
Now why this 1,001 host thing?
Here's the deal.
It turns out that in our code, we had-- for old style clients, of which this was an old style client-- we had a restriction.
We said if you need to serialize more than 200 megabytes worth of data, we're not going to handle that.
We're just going to fail.
It turns out once we got 1,001 hosts, we slightly exceeded that 200 megabyte buffer size, we silently erred with an out of memory exception.
And we never returned to the client.
And everybody was unhappy.
So it had nothing to do with 1,000 hosts.
It has to do with the amount of data you're actually sending.
So luckily, there were several different fixes.
So I told the VMRC folks, the guy that wrote this thing, I said, please, under no circumstances, should you get information for every single host if you only want it for this one guy.
That's a terrible idea.
And then I went back to the server folks, and I said, you have to admit-- first of all, you have to fail correctly.
And second, you have to admit sensible error messages, because this should not take the kind of expertise that it took to figure out.
And thankfully, this made actually a pretty huge difference.
So that was kind of cool.
So the lessons that I took away from this is if you're going to create an API, create an API that's very difficult to abuse and very easy to use.
Not the opposite.
These poor VMRC folks, it's like a three line piece of code to get information for every VM.
It's like eight or nine lines to get information for just that VM.
But those five lines make a boatload of difference.
The other thing is that this was an entirely internal customer.
In other words, it's just another group within VMware.
Imagine you're some external guy that's using the same API.
You're screwed.
So we have to be better at educating the people that are using our API.
And don't just throw it over the fence and say here's an API.
Deal with it.
There needs to be some interaction between them to make sure that this is the right API for its job.
This is the next case study I want to talk about.
This one actually was found by a colleague of mine, who I'm going to abbreviate as RM.
Very interesting example.
For those of you that took the class last year, I'm recycling it.
But it's a good example, so no worries.
So here's the deal.
You've got a benchmark.
And I've got two builds of a piece of software.
When I run the benchmark against that piece of software, in case a, I was getting a throughput of 100 operations per minute.
In case b, I was getting half the throughput.
And what was the difference between those builds?
Well, the first build, the faster build, was a 32-bit executable running on 64-bit hardware.
While in the second case, it was a 64-bit executable running on 64-bit hardware.
Now if you're like me-- which hopefully you're not-- but if you're like me, you think yourself, well, there shouldn't be much difference between 32-bit and 64-bit.
In fact, 64-bit should be a lot faster for a lot of reasons you're going to learn about in 004.
So the question is what's going on?
I can distinctly remember when this check-in was done, somebody said, oh, we're making the switch over, but it shouldn't be a big deal.
And foolishly, like a moron, I actually believed that person.
Very bad idea.
So this came back to haunt us for three months.
So what's the deal?
Remember, a 32-bit executable running on 64-bit code was faster than a 64-bit executable running on a 64-bit application.
So the first thing we do-- first of all, you've got to find the right tool for the right job.
In this case, we use a profiling tool called Xperf.
Xperf is a very powerful profiler, kind of like Vtune, or Valgrind, or Quantifier, or a lot of the other tools you've used.
The nice thing is it's built into the OS.
It runs on Windows 2008.
And it's a sampling based profiler that has a lot of pretty cool attributes.
If you have Windows 2008, I think you ought to just go and download it.
It's free.
And play with it.
This gives a lot of information like stack traces, caller/callee information, et cetera.
It's potentially the perfect tool for this kind of job.
What kind of output does it show?
In this case, remember the setup.
The setup is I run build A.
And I'm getting twice as much throughput as running build B.
So let's take a look at the CPU usage, just as a first cut to see what's going on.
In the 64-bit case, which is what I showed here.
I showed the output of this Xperf for 64-bit.
What you can see, x-axis is just time.
And y-axis is CPU usage.
It's totally saturated.
In the 64-bit case, we're completely saturated.
Take my word for it, in the 32-bit case, you're not.
And this accounts for the difference.
So now let's dig deeper and find out why we're seeing this difference.
So first, we look at the sampling profiler.
Now there's a lot of perils of using sampling profilers.
In this particular case, here's the deal.
I've got a process.
And that process spawns a bunch of threads.
In this particular view, what you're seeing is where is the time being spent in this process.
And as you can see, most of the time is spent at the root, which kind of makes sense.
Because everything in some sense is a child of the root.
But the first thing our poor process does, it spawns a bunch of child threads.
So clearly, all of the time is essentially going to be spent in the routine that calls all of these threads, because that's the first entry point.
But unfortunately, that's remarkably unhelpful.
So the deal is we have to dig deeper and find out, where is this time actually being spent.
So like I said, just to kind of reiterate the point that I made on the previous slide.
The point is that your entry point is spawning all the threads.
And if that's kind of the starting point for accounting, you're not going to get any help if you're just looking at that starting point.
You have to dig a little bit deeper.
And so what I'm basically trying to say, is that even though this thing records stack traces, this stack happens to be the most popular stack.
But it's not actually the problem.
Because it's the most popular, because that's where everybody starts.
So all of the time that is charged gets charged to that stack.
We have to look a little bit deeper.
So in order to look a little bit deeper, let's think about the problem in a slightly different way.
Suppose I've got my root.
That's where everything starts.
That's where all the threads are spawned, whatever.
And suppose I have three paths to a tiny little function at the very end.
Maybe I'm spending all of my time in the path that I've indicated.
But the problem is that-- suppose you're spending all of your time in that tiny little function, and suppose it's equally spent among these other three paths.
Well, it's not any path that's kind of screwed up.
It's that that tiny function is kind of messed up.
So you got to figure out why are you spending time in that tiny function?
And is there something that you can do to fix that?
So let's look from the opposite perspective.
Let's look at the caller view.
In this case, what I did was I looked at a bunch of different routines, a bunch of these different so-called tiny functions.
And I looked at what was calling those tiny functions.
It turns out that here's a call, ZwQueryVirtualMemory some tiny function.
It happens to be-- it turns out that we're spending, as it indicates, 77% of our time in this tiny function.
And just to emphasize that this is not being called from the root.
So you don't see it from your first glance.
You can see that the number of times this tiny function was called from the root is 0.34%.
So if you were just looking from the root and trying to figure out what's being called from the root.
It would be totally useless.
Instead, you have to dig deeper.
And see all the different functions that are being called.
And find out where they're being called from.
So we know that this function, ZwQueryVirtualMemory, is taking like 70% of the CPU, which is an enormously large amount of CPU.
We also know what's calling it are these two routines right here.
This thing, this magical thing called RTDynamicCast and RTtypeid.
So it turns out we have two paths that are going to this tiny function.
And those two paths comprise most of the calls to this tiny function.
And it turns out we're spending a boatload of time in this tiny function.
So let's dig deeper and find out what on earth these two routines are.
What is RTtypeid?
What is RTDynamicCast, whatever?
So let's first look at RTDynamicCast.
What I show here-- in contrast to the previous view.
The previous view said, I have a tiny function.
What are the people that call me?
Let's look at the reverse.
Let's go and start at one of these two routines, RTtypeid.
And figure out what it's calling that is spending so much time, and ultimately leads to us calling that ZwQueryVirtualMemory or whatever.
And as you can see, this RTtypeid, it calls two functions primarily.
One is incrementing a reference counter.
And the other is decrementing a reference counter.
Now you're thinking to yourself, incrementing a reference counter, decrementing a reference counter should be very simple, very low CPU, and basically should cost you nothing.
And all of those three assumptions are dead wrong, unfortunately.
So let's try to understand a little bit as to why that's happening.
Turns out RTtypeid is used in order to figure out the C++ type of an object.
So we're trying to figure this out on the fly.
And as I said, it's kind of mystifying that this routine, which calls our tiny function, that you're spending 39% overall in this routine in that tiny function.
So let's figure out what's going on by looking at a simple example, which would be IncRef.
So it turns out if you're looking at this code, the dreadful line is the fact that we're doing const type info reference tinfo equals typeid of *this.
This is basically, you have to call RTtypeid to get this kind of information.
And in order to do that, you need some runtime type information.
This runtime type information has some pointers associated with it.
So here's the deal.
Whether I'm running a 32-bit-- Remember, we even forgot the original problem, which is, gee, I'm 32-bit, 64-bit executable-- sorry 32-bit executable on 64-bit hardware.
We were slower than a 64-bit-- we were faster than a 64-bit executable on 64-bit hardware.
Let's try to figure out why.
Well, remember, every time I increment or decrement a reference counter, I get some type information by consulting this runtime type information.
It's got a bunch of pointers.
When I run a 32-bit executable on 64-bit hardware, those pointers are just raw 32-bit pointers.
You just look them up, and you're done.
In 64-bit, pretty much the only difference is that instead of those pointers being actual pointers, they're 32-bit offsets.
So you take the offset, you add it to the base address of the DLL of the executable or whatever, where that runtime typeid call is being made.
And then you're done.
Once you've added that offset, you get a 64-bit pointer.
You look up that 64-bit pointer, everybody's happy.
So here's the deal.
Remember, we said 32-bit's faster.
32-bit is just a pointer look up.
And then you're done.
64-bit's slower.
It's a pointer look up-- and it's an addition plus a pointer look up.
So I pause, because you're thinking to yourself, an addition plus a pointer look up versus a pointer look up.
Does that mean that addition is slow?
That sounds really stupid.
And by the way, it is really stupid.
It's not true.
So the deal is this.
It's not that addition to create a 64-bit pointer is slow.
The deal is that-- remember this 2-step process.
You find out the base address of a module.
And you add this offset to that base address.
Determining that base address is really, really slow.
It turns out you have to call a bunch of different random routines that have to walk the list of loaded modules.
And you're doing this every single time you call this routine.
And if you increment a reference whenever you do a memory allocation, you can imagine that this is going to consume a lot of CPU.
So it turns out, this is what's actually slow.
And remember, that we started this off a little by saying, oh, we've got this weird function, ZwQueryVirtualMemory, which seems to be being called a lot from a bunch of different places.
The reason it's called a lot is because all of these routines to get typeid information are calling this routine.
So there's actually two solutions.
And you can look on this blog to actually figure out that this is a known problem that Microsoft eventually fixed.
And it turns out, that the two solutions end up being that you can either statically compute that base address so you're not constantly relooking it up every single time.
But there's actually a much simpler and stupider solution, which is, use the latest runtime library where they have actually fixed this problem.
So just to summarize what was going on.
We were CPU saturated.
We looked from the top down.
We didn't notice anything.
We looked from the bottom up.
And we noticed that some bottom was being called a lot.
We figured out where that bottom was being called by looking at caller/callee views.
Once we figured out where it was being called, we then tried to figure out why that thing was being called.
And then we fixed the problem.
So to me, I guess the take-home lesson from this is that when that developer said to me, three months before this whole debacle started, he said, oh, yeah, we're switching from say 32 to 64-bit.
That's not a big deal.
You can't really take that stuff at face value, because a lot of little things can really add up.
And we could never have really guessed that this would happen beforehand.
So that's four.
Let me go over another one.
Memory usage.
So excessive-- I'm sure actually all of you know that memory usage is a big problem.
Your application is going to slow down if you're inducing a lot of paging.
And at least in 32-bit Windows, if you exceed two gigabytes for the process that you're running, the process is just going to crash.
So you clearly do care about memory usage.
And I want to differentiate between memory leaks and memory accumulation.
You're wondering why would a process take up two gigabytes of memory.
Well, one is that maybe you have a leak.
You allocate something.
You don't remove it.
That might just be an accumulation.
You allocate something somewhere in your code, you actually are freeing it.
But the trouble is that routine never gets called, because in your code logic you never take the time to think and say, OK, can I actually get rid of this at this point.
And that's what I'm going to call a memory accumulation.
There's a lot of different tools out there to analyze memory usage.
And the sad part about this is that almost none of them work for the particular example that we have, because they just didn't scale very well.
It's fine if you've got your toy, stupid little application that does addition in a loop.
Not so fine if you've got an enterprising middleware application.
So we ended up, despite all the tools that we have available, we ended up having to write our own.
I didn't do this, a colleague did.
But anyway, I guess that distinction doesn't really matter.
But anyway, the point is that you do a lot of what these other tools do.
But you do it special to your environment.
In this case, we just hooked a lot of the calls to malloc.
And we figure out what pointers are live.
And then we do stuff from there.
This is how normal tools work.
And of course, this can be unusably slow if you do a ton of allocations.
For example, if you're doing millions of allocations per second, you can imagine you're calling this kind of routine, which itself-- if it only has a 10x overhead over a call without that, you're slowing down your program by 10x for every single memory allocation.
That means performance depends on how many allocations you do, which is really bad.
And just to illustrate what I mean by a leak.
My definition is basically that you've got this routine, foo.
And it mallocs some data.
But it returns without freeing that data.
And nowhere else in the code log did you actually have a free.
That's an actual memory leak.
An accumulation would be if you have some free somewhere.
But it's never actually called.
So a lot of the code that we write happened-- at least in the past, it's changing-- but a lot of it happened to be written in C++.
And all of you, I guess assume, you know about new and delete and all those kinds of things that we do in C++.
And it turns out, that those things are notoriously bad for assigning memory leaks.
So instead, what people typically do is they use reference counted objects.
So every time you use something you increment some reference count.
And every time you delete it, you decrement a reference count.
And when the decremented reference count is zero it automatically gets deleted.
Now of course, this only solves an actual leak.
It doesn't solve an accumulation.
So here was the problem that I noticed.
That was all kind of setup, just saying what's a memory leak, what's an accumulation, why is memory a problem.
Let's get to actually why it was a problem.
I had this issue.
I have our little server application.
And it's running out of memory after several hours.
Now I used Purify on a much, much smaller setup.
It was about 1/100 the size of the actual setup, because larger than that, Purify would crash, because it would run out of memory.
Because by the way, to track all the memory references, it's got to allocate memory, which ends up being a nightmare of unprecedented proportions.
So the deal was, was that I couldn't really detect this leak very easily.
So what I did was I examined the memory that was in use.
Essentially, what I'm trying to say is that, there wasn't so much a leak as there was an accumulation.
Accumulation again being a logical problem.
Well, yeah, all of my code is written correctly.
But it's just that free call is never called.
So I examined the memory in use and I localized it to one operation that whenever we did it, memory would increase.
So since doing it only once, there's a lot of noise.
What I did was I said, well, OK, why don't I just do this operation hundreds of times at the same time.
Because then, if I see some allocation that's 100x from before I started this I can know that this was actually a problem.
And that's exactly what I did.
I had this issue where something was being allocated 64 bytes.
Now 64 bytes is so small that it's pretty much in the noise.
But if you do it a hundred times, and you're seeing 6,400 bytes.
Then you do it a thousand times and you see 64 times 1,000.
Then it becomes something easily that you can then track.
Once I noticed that this particular piece of data was being allocated like this, I went through and I realized that this actually was-- it was an accumulation and not a leak.
So someone would allocate something whenever an incoming message came in.
And there was a free somewhere, but it was never being called.
So the lesson that I learned from this is that in a lot of situations with performance debugging, it's very helpful to try to find a problem-- in order to try to find a problem, it's very helpful to magnify it and make it a lot worse.
If you think something is problematic in the network, slow down your network if that's an option.
If you think something has a problem with memory usage, either do something like what I've described here.
Or maybe reduce the amount of memory.
When I was in graduate school, we were working on a cache coherence protocol.
And I'm going to spare you a lot of the details.
But here's the thing.
We basically stored a linked list for every data in memory.
And we had to make sure that that linked list was rock solid no matter what the size of the inventory was.
And if you exceeded the linked list, you'd store stuff in disk.
So what we did to magnify any possible bugs was we created it so that you could make a linked list of at most one item.
That way no matter what you did it would force you to overflow and go into disk.
And that was a great way to test a lot of overflow conditions that in practice would never have occurred.
Because we do the engineering thing, figure out the capacity, double it, and hope that nothing happens.
So it's good to exaggerate edge conditions.
So that's one memory analysis problem.
Here's another one.
Here's the deal.
You've got a user and they're using this middleware server.
And they complain that with time it gets slower, and slower, and slower.
And if you look at all of the-- if you look at CPU network and disk, you see that nothing is being saturated there.
But if you look at memory, it's increasing pretty dramatically.
And at some point, it increases so dramatically that the server application crashes.
So first thing, let's actually look at the memory utilization and see the timeline.
What I show here is I show versus time, I show a chart of something in Windows that's called private bytes.
Essentially, think of that as memory.
What you see is that memory is growing slowly, whatever.
But at some point, near the very end, near when the server crashes, it starts growing at a really, really fast clip.
Now remember, I call this private bytes.
What that essentially is, in this particular case, it's Windows giving a process some amount of memory.
It's not necessarily what the process or what the application is requesting.
The application will request some amount.
And Windows will give it this much.
Well, Windows keeps giving it more and more and more.
And it eventually just runs out of memory to give.
So the reason I made the distinction between something that an application and something Windows is giving it is that I, as an application, might be asking for four bytes at a time.
Four bytes, four bytes, whatever.
But what if Windows has this problem where, whenever you allocate-- it doesn't, but suppose it does-- suppose that every time you ask for four bytes, it says, well, you know what, I'm just going to give you 16 bytes because that's my smallest increment of memory.
You can imagine if I keep asking for 4 and it's dumb enough to give me 16 every time, we're going to see a huge expansion of memory, even though I, as an application, didn't ask for that much.
So it's important to keep these distinctions in mind.
So the point is what was happening in this case, was my server's going fine, fine, and fine, until near the very end where it starts increasing in memory use at such a dramatic rate, that it eventually crashes.
So the first thing I did-- We just talked a lot about these reference counted objects.
The fact that I keep track of whenever something is allocated and whenever something is not being looked at anymore.
When it's not being looked at anymore, I go ahead and remove it.
If you look at the reference counted objects, they were pretty flat until you get to the very end of the run, exactly the same as when the memory was increasing.
And it turns out, I've highlighted a few of these, and they ended up being-- it doesn't matter what they specifically were-- but in this case, they ended up being related to threads, and mutexes, and stuff associated with keeping track of what was going on in the system.
So some thread-related objects were increasing for some reason.
Now turns out thread state in some sense is very cheap.
So just because these guys were increasing, it doesn't really explain why we're dramatically increasing in memory.
But I'll tell you what does explain it.
If you take a look at a different attribute, which is these are the total number of threads in our system, we have a static pool of threads.
And whenever a work item comes in, we allocate a thread.
Whenever that work item is done, we deallocate the thread.
So we keep it to a minimal number.
And it turns out, we had a cap.
We said, OK, I'm going to allow at most 20 operations to occur in flight.
Anything beyond 20, I'm going to queue it up in memory.
And we'll deal with it later.
Here's the deal.
What you see when you look at the number of threads being used is that-- it's the bottom chart, unfortunately, it's a little hard to see.
But the point is it goes up and down, up and down, up and down with time.
So stuff gets allocated.
Stuff gets used.
Stuff gets removed.
But then when you get to the very end, you see this monotonic increase in the number of threads being used.
It never goes down.
And at some point, you hit the point where all of the threads that you've preallocated are completely taken.
You obviously can't allocate any more threads.
So every incoming request, you go ahead and queue.
This was the root of the problem.
The root of the problem is that in this particular situation, each of the times that it was increasing was because there was again an uncaught exception.
That uncaught exception resulted in a thread not being deallocated.
At some point, we ran out of threads.
And every single request that came in was being allocated in memory and waiting around for a thread to become available before it could be finished.
Well, if no threads are available, these things can never get pulled off the queue.
You see a monotonic increase in the amount of memory.
And you eventually die.
And so, it's actually a correctness issue.
Why are you not catching exceptions properly?
And why are you not recovering properly from them.
But interestingly, it manifested itself in a performance issue.
So in each of these cases, what we've done, is we've looked at customized profiler instead of a tool that we might have gotten, Purifier, Valgrind, or whatever-- it's nice that it's tailored to our application.
And we can make it as arbitrarily fast, because it understands what our application is doing.
And it also-- the other nice thing about this is this can be run in production environments.
If you write your own tool, and a customer calls and complains, you can just say, oh, just twiddle this knob and you're done.
That's kind of convenient.
But of course, the disadvantage is that now you have to-- whenever you rewrite your code or do a new rev, or whatever, you have to make sure that these tools work right.
And you have to recompile the code.
If you have something like Purifier, Quanitify, in a lot of situations, you can run with some sort of preexisting binary-- sometimes, not always.
You can run it without having to recompile.
And that can be very convenient.
So I don't know.
I was kind of stuck for what a lesson this was besides common sense is useful.
So I just said memory profiling is pretty critical.
Don't ignore that when you're writing your applications.
And I think I also want to point out that-- I didn't put this here.
But you might think that when you're using a garbage collected language, like something like Java, that these problems go away.
But they actually don't.
The problem is actually now memory allocations and deallocations are more hidden from you.
So you have to be a little bit more careful about how you analyze them.
Case study number seven.
So we've looked a lot of things related to CPU usage.
We've looked at a lot of things related to memory.
Now let's look a little bit at the network.
Here was the problem.
I have a user, our Canonical admin.
The user wants to perform an operation on some virtual machine somewhere.
Remember the flow.
Issue request, it goes to some server.
The server goes to the host where that VM is.
Does some operation, comes back.
So this user wants to perform an operation on a virtual machine.
I had setup A, In setup A, this operation end to end took about eight seconds.
Fine.
That doesn't mean much until you compare to setup B, where it took twice the amount of time.
It took 16 seconds, not 8 seconds.
So now the question you're asking yourself is what's the difference between setups A and B.
In setups A and B, the only difference is that this middle server is a different version number.
But everything else is exactly the same.
So with this different version you somehow manage to make your performance twice as bad, which is a really bad thing.
We got to get to the bottom of that.
And in addition, it's not like it was some random thing that happened once.
It happened every single time.
So now, let's take a look at what was going on.
So again, a lot of times, I tend to do things in a top down manner, where I say what's the easiest information to collect.
And then, OK. What's more difficult, what's more difficult?
And finally, if you have to use really industrial strength tools, you use them.
But it might not be necessary.
In this case, my first thing was just to use logging and try to figure out where time was being spent.
Remember, we've got a client talking to a server, talking to an end host.
In this case, the latency imposed by that client was exactly the same in these two setups.
So that's not where this eight seconds is being lost.
The amount of time being claimed by the server was exactly the same.
I'm a client.
I go to a server.
It then goes to a host.
However, the time spent on the host was different.
That's where all of the time was being spent.
Now, you're probably looking at-- well, you may not be.
But I'll tell you to look now.
We had setup A.
Setup A was a client and the server were each at some version number.
And the host was at the same version number.
In setup B, the client and the server were at a different version number with respect to the host.
But we just said all of the time is being spent in the host.
So why is that going on?
And this is where understanding the architecture is important.
It turns out in this case, the first thing that happens when you have a difference in version number is that the server helpfully talks to the end host and says you know what, you're in a different version number.
So let me give you some code to run, because you're in a different version number.
So in fact, even though the host is at the same version number, there's a small shim running on that host that makes up for the fact that it's not at the same version number.
So now let's analyze this host and figure out where is this time being spent.
I did a little bit more logging on the host.
The reason I did this, because a lot of the standard tools that you might use, like gprof or whatever, these things didn't exist.
We have our own kernel.
Which therefore, we have our own tools and our own nightmares, and headaches associated with it.
So we had to use these tools instead of commercially available tools.
When you use these tools, we ended up narrowing down the time.
Here's what goes on.
I'm a client.
And I talk to the server.
The server talks to an agent on the host.
It says, agent, this is what I want to do.
The agent then talks to the hardware abstraction layer and says, dude, this is what I want to do.
The hardware abstraction layer comes back to the agent and says, done.
Agent goes back to the server.
Sever goes back to the client, blah, blah, blah.
It turns out that this interaction on the host between the agent that accepted the call from the server and the hardware abstraction layer, that was where all of the time was being spent.
It was 10 milliseconds in one case.
And 20 times that length in the other case.
And so this was a pretty big difference not to be ignored.
So we wanted to look at why that was going on.
So here, it's kind of a dumb thing.
But the next the thing I did was, well, let's take a look at the configuration.
Maybe I did something stupid that would have caused this to happen.
Here's the deal.
In setup A, remember everything is at the same version number.
And it turns out this agent HAL communication occurs over a named pipe.
OK, whatever.
It turns out in setup B-- remember the client and the server were at a different version from the host, so the server has to download some shim code onto that host so that everybody can talk correctly.
It turns out that shim code was communicating with the hardware abstraction layer-- instead of over a named pipe-- it was communicating over TCP/IP.
Now I have to give credit to my colleague on this.
As soon as I showed him this information, he hit a flash of inspiration.
It turns out that when you're using the named pipe and it takes 10 milliseconds, the reason that's different from TCP/IP communication is because of something called Nagle's algorithm.
Nagle's algorithm basically says, you want to accumulate data before you send it so you reduce the number of round trips and the overhead of creating a connection, and whatever.
And the problem was this was biting us.
Essentially, we were waiting within that shim to collect enough data to send it back to that HAL.
And we were waiting fairly needlessly, because what happens is that this algorithm waits for a certain amount of time.
And then if it hasn't gotten more data, it goes ahead and sends it.
So we are basically stuck, waiting for this time out to occur before the data actually got transferred.
And there's a very simple way to get rid of that.
You just tell it, hey, don't use Nagle's algorithm.
We don't care.
And that solves the entire problem.
So the point is once you get rid of that, the performance is exactly the same.
It's a very simple configuration setting.
But that's again, a situation where somebody-- and in fact, I remember asking why this particular change was made.
And again, it was, well, we didn't think it was going to be a big deal.
And that ends up being, again, a very bad way of doing performance engineering.
So definitely, the lesson that I learned here is just like the lesson with 32-bit, 64-bit.
You might think that a little change like this doesn't mean anything.
But it really does.
And here, it's actually important to understand the entire stack end to end.
Because otherwise, you're never going to be able to figure out where the problem actually was occurring.
I already alluded to a previous situation where we had a correctness problem, which masked as a performance problem.
In the previous case, it was because I had a limited thread pool.
And whenever I ran out of threads, I'd queue everyting.
And at some point, I had a correctness problem where I kept killing threads and never resurrecting them.
And so I ended up queuing everything.
And everything went to pot.
Now let me talk about a different situation.
Here I've got a poor little customer, in fact that customer was me.
I was powering on a virtual machine.
Remember it goes to the server, goes to host, powers on the virtual machine, tells me it's done.
Every other time I would do this, it took five seconds, which is pretty OK.
In comparison, every other time I did it, it would take about five minutes.
That's a pretty huge expansion.
And if a customer called me and said this, I would basically be in deep doggy do-do.
So it was important to try to figure what was actually going on here.
So why does powering on a VM have such a variable performance?
Well, the key here, again, is to understand what's the end to end path for things.
In this case, what I know is that when you power on a virtual machine, one of the things you have to do is allocate some space on disk to write swap information and whatever.
So one of the first places I looked was I said, well, let me understand how the disk is performing in each of these cases to try to figure out what's going on.
It's a mixture of common sense and knowing what's going on, and then knowing where to look.
So let me let you in on a little secret.
In the real world, if you're using some sort of disk, if your disk latency is basically 10 milliseconds or less, you're more or less golden.
That's not a problem.
If your latency is between 10 and 20 milliseconds, maybe that's acceptable because you have a complicated topology.
You're some multinational corporation with sites everywhere, whatever.
If your latency is getting to the point where they're 50 milliseconds, then you probably have to start worrying.
That's actually quite a bit high.
If your latency is greater than 51 seconds, that is staggeringly bad.
In this chart, I show you the latency versus time.
And what this chart shows you is that every single time step, it says in the last five minutes, what was the highest disk latency that I saw.
And remember our rules of thumb.
10 milliseconds, good.
20 milliseconds, not so good.
50 milliseconds, not so good.
This one is 1,100 milliseconds.
That's like you could walk to the disk, get the data, and bring it back faster than what it's actually doing.
That's when you've got to call a priest.
OK. That's horrible.
So this was the reason that five seconds versus five minutes.
So now the question is why is this going on?
Why am I seeing such unbelievably bad disk latency?
And remember, the title of this case study was "Correctness Impacts Performance."
So let's take a look.
What I do here is I take a look at all of the events that are happening with respect to that disk.
You're not going to understand this.
It's some junk related to our product.
But I'm going to circle the relevant portion.
You'll notice a little line that says lost access to volume.
Here's the deal.
I send a request to the disk.
It turns out the disk is connected over some channel to some channel, a disk controller, which is talking to some disk.
It turns out, I kept losing access to that disk.
Whenever I would lose access, it would keep retrying.
And that's why ultimately, yes, it would complete.
But it would take over a second per disk request.
In the situations where I was not seeing that loss of connectivity, things were occurring at the order of milliseconds and my power-on would take five seconds.
So the point here is that in this case, it was actually a correctness problem with my controller.
And the controller would basically flake out and not talk to the disk properly.
And this manifested itself in a performance problem.
Once I changed the disk controller, performance problems went away.
So if you're doing this kind of debugging, sometimes you have to eliminate-- I guess it's a Sherlock Holmesism-- that you eliminate the obvious and whatever's left must be it.
A good example of this is my ex-officemate, he used to ask this question whenever he'd interview somebody for VMware.
He'd say, OK, you have a networking problem.
Where is the first place you look?
And everybody that's coming in with a grad degree or whatever, they're like, I would look at the stack.
I'd look at the CPU.
I'd look at a cache counters, or whatever.
And my officemate would just look at them and say, why don't you just check the cable and see if it's plugged in.
And surprisingly, it's so dumb.
But when you're talking to customers, they're like I swear I did that.
I swear I did that.
Their swearing doesn't necessarily mean anything, unless it's directed at you.
So that's case study nine-- eight.
The last two case studies are going to be specific to virtualization.
And so what I'm going to do is talk a little bit about how the CPU scheduler works in the VMware server class product.
And then I'll talk about the last two case studies.
So actually how many of you have used VMware before?
Just as a hand.
Thankfully, Saman has used it.
That's good.
So basically, the idea is that you're multiplexing time on the CPU between different virtual machines that are running on the CPU.
So what I show here is I show ESX, which is our server class product.
Basically, think of that as the hardware.
So you're running virtual machines on hardware.
That hardware has, in this particular case, four physical CPUs.
No problem.
You've got four physical CPUs.
And you want to run virtual machines on this hardware.
So you've got one virtual machine.
Well, that's fine.
There's four CPUs.
One virtual machine.
No problem.
Plenty of CPU to go around.
Now you got another one.
Still no problem.
They're not chained to any given CPU.
And there's plenty of CPU to go around.
No problem.
Same with three and four.
You got all these four virtual machines.
It's not a one to one correspondence in terms of CPU.
We don't work that way.
But there's plenty-- the point is, there's plenty of CPU to go around.
Now what happens, and when VMs are basically available and when there's hardware on which they can run and they're using the hardware, that's called the run state.
So running means I'm golden.
I'm using the hardware.
Now, let's add another VM.
I put it in red, because this poor VM, he wants to run.
But you'll see that every single CPU is actually taken up by another virtual machine.
So even if this virtual machine wants to run, it can't.
It has to wait.
The time that it spends ready to use the CPU, but unable to use the CPU, because there's no CPU available, that's called ready time.
In some sense, it's kind of the demand.
Ready plus signals down.
But anyway, the point is it's sitting around waiting to use the CPU.
But it can't, because other people are using it.
So it has to wait its turn.
It waits its turn.
And eventually a VM, our ESX scheduler is smart enough to realize that it has to fix the situation.
So it deschedules a VM-- in this case, VM 1-- and goes ahead and schedules that other virtual machine that was previously waiting.
Now predictably, the VM that's now ready to run but can't is in the ready state.
And the VM that formerly was in the ready state is now using the CPU.
So it's in the run state.
It's very happy.
The final thing that I should point out is that VMs don't have to be in the ready state.
They don't have to be either using the CPU or ready to use the CPU.
Some VMs are voluntarily descheduled, because they have nothing to do.
Some VMs are descheduled, because they're waiting-- they're basically waiting for an IO to complete.
And so they don't need to use the CPU.
And that's what I depict by VM6.
These are in what's called the wait or idle states.
So hopefully, you now have a fairly rudimentary understanding of what our CPU scheduler is doing.
Let's talk a little bit about a very simple performance problem related to this.
And I call this-- But It's Only a small probe VM.
I'll explain what this means in a minute.
Essentially, you've got two of these hosts, two ESX hosts that both want to run VMs.
And I've got a user, depicted by the user icon.
And he's talking to this one VM.
Let's call this VM the vSphere VM.
Please pardon the nomenclature.
It doesn't really matter.
But anyway, he's talking to this vSphere VM on one of the hosts.
Now, it turns out that that VM that he's talking to, as I depicted in the diagram, is talking to another virtual machine, which I have labeled vSphere database.
So it turns out, this vSphere VM is talking to a database over what's called an ODBC connection.
That doesn't matter.
Don't worry about that.
And it turns out that on that other host where the database virtual machine is running, we have another virtual machine.
It's called the probe virtual machine.
The job of that probe virtual machine is to sniff any traffic going into the database.
So as I've said, the vSphere VM communicates with the database.
This probe VM is monitoring any traffic that's going to that database virtual machine.
And therefore, as you can imagine, the more traffic that's being sent between this vSphere VM and the database, the more work that that probe VM has to do.
Here was the complaint.
My colleague called me up, and he said, hey, I'm having a little bit of a problem with my vSphere VM.
All of a sudden, it's completely unresponsive.
So then I was like, OK, well, this is kind of silly.
Let's try to figure out what's going on.
So here's what I did.
I graphed various metrics, which I just spoke about, the used time and the ready time, on this chart.
The time is on the x-axis.
And the y-axis is in milliseconds.
In what I'm going to call the land before time, the ready time of the database.
So look at the circle on the-- let me think, you're-- on the left.
This is the ready time of that database virtual machine.
Remember what we said, when you're accumulating ready time.
It means you really want to use the CPU.
But you can't because all the CPUs are already in use.
So it turns out that the first thing that happened when my colleague called and said I'm having a problem, is I look at this chart, which is versus time.
And I said, let me guess.
Are you having a problem at about such and such time, which corresponds to the middle of this chart?
And he said, yeah, that's when I'm having a problem.
So you can tell that there are two very clearly demarcated regions here.
In the left, you can see that this ready time, it's around 12%.
Now if you're like me, a number that has no context means nothing.
So 12% means nothing.
However, what I'd like to point out is that when things got really bad, that number spiked up to 20%.
So here's the deal.
I'm a database virtual machine.
I want to use the physical hardware.
I can use it about 90% of the time, because my ready time's about 12%, whatever.
But at some point, something happens where I now can only use it 80% of the time.
And it turns out that that difference was enough to cause this vSphere VM talking to this database to be perturbed.
And that ultimately was the customer's problem.
So the question is why.
And here is the reason.
Turns out, here's the deal.
Let's go back to the diagram so I can explain this.
Remember that the vSphere VM talks to the database VM.
Remember as well, that the more traffic there is, the more work the probe VM has to do.
It turns out that what this user did was at that critical juncture, he started a workload.
In starting that workload, he caused a lot of traffic between the vSphere VM and the database, which therefore caused a lot of work by this probe VM.
Well, what's happening?
The probe VM and the database VM are sharing the same underlying CPUs.
So if one of them is running, the other one can't.
And that's why we saw this gentle increase in the amount of ready time to vSphere database.
Well, this has a cascading effect, because it can't get as much time to use a CPU.
Any request that goes to it gets slowed down.
Where are the requests coming from?
They're coming from this vSphere VM.
Who's initiating those requests?
It's this poor admin.
The bottom line is that this admin is seeing poor responsiveness because some doofus put his probe VM on some other host, because he wanted to see what was going on in the database.
In fairness, I was that doofus.
But that's fine.
We needed that, because we had to debug other performance problems.
So the point is that one of the things that's kind of important in this particular case is that whenever you introduce any sort of monitoring-- it's like a Heisenberg thing-- it's going to have a performance impact.
And understanding that performance impact is pretty key.
It might even shift where the bottlenecks are in your system.
If your thing has a high cost in terms of CPU, it might shift the overhead of whatever's running to the CPU where it wasn't there before.
So it's kind of important to understand what you're monitoring and how you're monitoring it.
OK. We're almost done.
This was a really curious problem.
We have a customer.
And a customer was performing a load test on a server.
And in that load test, what they would do is they would just keep attaching clients to that end server and run some test.
At some point, they would attach clients and their workload suffered, their workload performance suffered even though there was no metric that was saturated.
And so the question was why.
Why you keep adding clients, nothing is saturated, but performance suffers?
Well, this chart's kind of an eyesore.
But let me explain it.
The first thing we do is we've got some profiling information.
On this chart, I show two things.
In purple, I show CPU usage.
And in white, I show disk latency.
As you can see from this chart-- and the y-axis is just 100%.
And they're both normalized.
As you can see at some point, the CPU usage is steadily, steadily increasing.
And at some point, the disk latency takes over and gets to be really large.
You can tell that, because up until about 3/4 of the way in, the disk latency hovers around zero.
And all of a sudden, it jumps much higher.
So my first instinct was to tell the customer you've got a disk problem.
Your disk latencies are going over a cliff.
And that's why you're having a performance problem.
Unfortunately, that's not correct.
Why is that not correct?
It's not correct, because, yeah, disk latency gets worse at 4:00 PM.
It's because of swapping and some other nonsense.
We know that.
However, the application latency actually gets worse at 3:30, not at 4:00.
It's half an hour earlier.
So now the question is what's going on that causes the problem, even before disk kicks in and gets to be another problem.
This is where understanding the difference in metrics becomes pretty critical.
Now, I'm not expecting very many people to understand this chart.
But I will try to explain it.
What I do here is I show a different chart, which essentially shows CPU usage of all the different virtual machines and processes on the underlying host.
Forget about what's in most of this chart.
And just focus on what I've circled.
I have two columns shown there.
One column is called percent used.
And one column is called percent run.
You can think about it and think, OK, well, I can imagine used time means probably how much time I'm using the CPU.
But then what is the runtime?
Isn't runtime the same thing, because I'm using the CPU?
What's the difference?
This is actually the critical problem.
It turns out, that when you talk of a percent used time, you're normalizing that to the base clock frequency.
If I've got a three gigahertz machine, I've normalized it to three gigahertz.
However, we all live in the era of power management.
It turns out this percent run, which nobody understands, is normalized to the clock frequency at the time you were running.
So if I did power management, and I was only running at two gigahertz for a certain amount of time, percent run captures that.
And so in this particular chart what I'm showing is that a bunch of virtual machines are seeing used time different from runtime.
And the runtime, which is normalized to the clock frequency of the time you're using it, is actually greater than used time, which suggests that the user's actually using power management.
And it turns out, that power management was kicking in at some point and causing their application latency to go down even though the CPU was never saturated.
And in this case, when we went back to the customer-- you look at this chart, and you instantly see, OK, I know that because this is normalized and this is normalized to that.
This is what's going on, you have to first of all change this parameter.
They changed it.
And they were unbelievably happy.
It was pretty cool.
Now, I have to admit that there's much more to this story that I'm not showing you.
Which I'm not allowed to show you.
But it becomes a very interesting thing.
But the point is it's very, very important to understand your metrics.
Here just understanding the difference between runtime and used time made all of the difference in diagnosing this customer's performance related issue.
So I just pick a smattering of ten random bugs.
And in each of them, I tried to distill one kind of conclusion you can get from that particular thing.
Because all of you can read, I'm not going to bother repeating all of these things.
But anyway, here's the first five.
And you all get the slides.
And here's the second five.
Oops.
I'll just keep that up for a second.
And in order to conclude this talk, let me just say that it's very important to avoid assumptions.
It's very tempting to say well that's-- I'll give you a great example.
I used to interview people.
And I'd say, OK, you've got a UI.
You've got a problem.
You have a UI.
And you click on something.
And it takes a long time.
Why is it taking a long time?
And the first thing they'd say, well, I'd go and look at-- this is literally, not joking.
They'd say, well, I'd go to look at the cache messages on the server.
And I just looked them and I said, how do you know the client's not the problem?
Oh, the client's not a problem.
And you base this on what knowledge?
Don't use assumptions.
It's a very bad idea.
It's really important to understand the entire system that you're dealing with.
From the client to the server to the hardware to the network to the disk.
You have to understand all of those different pieces, because you have to understand where to look.
And what's really onerous is that-- it is not so much when it's a problem in your code, because you can be blamed for making stupid mistakes-- it's when it's problems with other people's code and you spend a month looking at other people's code, figuring out what's the problem That's when things are really kind of irritating.
And especially when it's a problem with hardware.
And I think the final bullet I'll say is be persistent and be thorough.
I don't have to tell you how to be good workers.
So let's just blah, blah, blah.
We're done.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to the final competition.
So you guys survived this far.
And I don't know how.
I think, probably, with very little sleep.
And knowing all the emails that I have seen and all the check-ins.-- I don't know how many people had-- Who had any sleep within last 24 hours?
Haha
Oh, OK. [LAUGHTER] OK. And also what we looked at was the quality of the results keep improving because he was running tests to check what's going on.
And when we moved to a machine, the things ran very fast.
We're like, wait a minute.
Why is this running that fast?
Because people had checked in more and more, faster and faster versions.
So things improved, not because we checked in faster machines, but you guys actually did a lot of work.
So first before we get started, I want to thank Akamai.
So we and Akamai got this nice place.
And then we are going to give a demo.
And we're going to have a party sponsored by Akamai, which is really cool.
So you guys actually see something-- what the real-life people who care about performance do.
So it will be really fun.
And also, I also want to thank most of our MIT POSSE that help you guys a lot in meeting with you individually.
So we have people from Akamai, [UNINTELLIGIBLE] Digital, Google, Lockheed Martin, MathWorks, Wordica, VMWare and couple of independent consultants who basically put lot of their sweat and edge in helping you guys
And Microsoft
And Microsoft
So let's give a hand for the posse, MIT POSSE.
[LAUGHTER]
And a couple of the sponsors
So here is the process we are going to do today.
Today's about the competition.
So we've set up a very strict set of rules that we are going to go through, which is not going to be the way we are going to grade you guys.
Because we are going to be little bit more lenient.
So this is the set of rules we are going to do.
So we had a lot of these images that you get submitted.
And some images were not exactly same.
People did little bit things.
They made them look little bit different.
So what they said was instead of we're deciding.
We will let you guys work on images which can participate.
OK?
So don't be harsh to your friends.
So what we'll do, we'll first go through the images.
And then well you guys can say which images should be part of the competition.
This is for the competition.
Then at the end, after you get the Akamai sponsored prize-- not for the grading.
So the fact that the images start to count, that doesn't mean you get zero in the grade.
But you won't get to participate in the competition.
So after that, what we have done is we looked at all your projects.
Ran them on our 12 core machine.
And we will show you and seed them.
We'll say here is the order of everything happening.
And there are--
[LAUGHTER]
There are some-- Basically, what we'll do is we'll pick out the top set of seeds in them because we don't have time to run through everybody.
And basically, a lot of these things live are not for the 48 core machine.
And then the-- accepted image that ran faster, got generated faster in the 48 core machine, wins the prize.
So that doesn't mean that when you start grading, we will look at all the images and we will do some averaging.
We will do a lot more fair.
But hey, this is a competition.
We just need to find the top guy so we have a top group.
So we have a much more faster thing
So to get started.
So here are some of the images people generated.
So what we want to do is show the reference image.
That's the reference image.
And what we'll do is we'll first go through everybody's images generated, so you get a feel for all the variations out there.
And then after that, we'll ask you to go out.
We'll ask you if you don't want that image to be part of this.
[LAUGHTER]
You'll see.
OK.
So this is the reference image.
Why don't you go through-- Charles.
Mike
Yep?
Why don't you go through the rest of the images
OK.
So before we we get into the images that we actually rendered in the last few hours, I just wanted to like honorable mention for this.
I think this is the coolest.
I'm sure you guys have been looking at a whole variety of broken ray tracer images But I think this is pretty much the coolest thing I've seen.
So honorable mention for that.
And so now let's look at things that people actually submitted.
First we're going to go through all the images and sort of look at the range.
And then we'll go back and vote on them.
And so I'm just going to flip through the ones on the right and give a couple seconds to each
So see if they can identify the artifacts
Which one?
You got to say when you're going because otherwise people aren't going to realize that you've been through three of them already.
Here's image one
Image--
Here's the next one
Image two
That's five.
Four, five
Image three.
image four, image five, image six, seven, eight, nine, ten, 11,
12, 13, 14, 15, OK, 15.
There are some groups that we could not get the rendering.
Either it crashed, or after ten minutes it was still hadn't produced an image.
So they're not a part of the competition.
And of course we run them for the finals.
OK, now the vote time.
OK, so this image, in or out?
Who thinks out?
Whoa, you guys are a harsh crowd.
OK. [LAUGHTER]
Who thinks in?
Who thinks in?
Who thinks in?
We've got to get you both.
Who thinks in?
So we know--
Do we get screwed for this?
[LAUGHTER]
Imagine that.
OK. That image is out.
You can take notes, OK. [INTERPOSING VOICES]
OK, next image
Number one's out
OK, this one, in or out?
Out
Probably the same
OK, it's out
It's got the funny ring at the bottom, which I guess that's what most of you are picking up on.
I think that's kind of disqualifying for the derby
OK, how about this?
Out?
In or out?
Out
Out.
Whoa
Wow, you guys are vicious
Exactly.
OK, how about this?
How many people think they should be out?
OK, I guess that'll get in here.
I guess-- OK, that image is in.
OK. Make sure it's in.
How about this one?
It's okay
Who says it's out?
[INTERPOSING VOICES]
OK, Who says it's in?
I think it's in
You should look at the water
Water
The water has holes
The water has holes.
That's what's going on.
The water has holes
[INTERPOSING VOICES] It's the only one we're going to be able to reference
OK, so what's the this is in charge?
Let's do it again

OK, you have to point out--
What's the problem?
--what's going on in the image
Yeah.
I think we should-- I mean, do we want to get people to like say why they think it's in or out?
It's helpful to say what's in the image, though
Yeah
OK. Let's take a vote again.
How many people think it should be out?
OK, that's out.
OK, yeah
OK, next image
Number 5, on the slide
OK, in or out?
How many people think this should be out?
Is there any problem?
I mean, what's the-- [INTERPOSING VOICES] [LAUGHTER]
I don't think there's a problem.
OK, good.
Next one.
How many think this should be-- OK. That's unanimous, that's out.
How many think this should be out?
OK, I guess that's in.
This another one?
Yeah, I think it's interesting to flip back and forth and see the difference between these two.
Like, these are very close, right?
Yeah.
It's obvious they're using this one.
[LAUGHTER]
OK, how many think they should be be out?
That group once is going to kill everybody.
[INTERPOSING VOICES]
How about this one?
How many people think this should be out?
There's not enough caustics.
You don't see the floor
Yeah, that's an interesting point.
[INTERPOSING VOICES]
Say what?
We went past ones that look the same.
[INTERPOSING VOICES]
It's really not fair.
When you have an image and then put that image right after that it's not fair
We will keep this in because there's few people want it out, but it looks pretty good.
OK, next image.
OK, how many people think this should be-- OK, that's a clear out.
How about this image?
It looks OK.
It's fine
It's fine.
OK, anybody want it out?
No?
OK, good.
That's in.
Next image.
How about this image?
[INTERPOSING VOICES] [LAUGHTER] [INTERPOSING VOICES]
We really should start the reference--
We need another slide to see how many people want that out
--identical images and people could be voting.
[LAUGHTER]
Is there an explanation why all of the yellows have been so far different from the reference image?
[INTERPOSING VOICES] [LAUGHTER]
So I guess this image is in.
OK, next one.
How about this image?
It's fine
Fine.
OK, anybody want it out?
No?
OK. How about next one?
[INTERPOSING VOICES]
It looks pretty good
It's pretty good
It's messed up on the left and the right
How many people-- We have to count this one.
This is too close.
One, two, three, four, five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19.
19 out.
Who says it should be in?
[INTERPOSING VOICES]
Oh, I guess it's out
Oh, the number of hands is more for out
OK, and that's it
That's it.
So now, next thing we are going to do is, we are going to show you a rank in our seating
Yeah, so we seeded you guys on the 12 core machines.
So we seeded you guys based on the time that it took for scene three on the 12 core machines.
So here is the ranking so far.
Top one is number one team.
And we ranked about the top 10 or so
Show the real run time
I don't have the run time, actually, for the seat
For scene one?
Yeah.
Actually, I can get it real quick
You had the graph, didn't you?
I don't have a graph
So here are the rankings of the top ten.
Do we have that graph?
What?
I don't have a graph
He has numbers
I just have the numbers.
[INTERPOSING VOICES]
There's all kinds of other numbers, too.
[INTERPOSING VOICES]
We'll say unplug this
No, nobody can see it
It's okay
Do we want to get the rankings?
No.
Now, what we're going to do is we're going to start from the bottom and start running it on our 48 core machine.
And while it's running, we want that team to say a little bit about what they did to get their performance.
So--
We need to knock out the one-- [INTERPOSING VOICES]
Either come prepared when you have formed your presentation, or make your image run very slow.
So you don't have to.
So I guess--
First of all we have to knock out the ones whose images did not render
Oh, that's true.
That's true.
That's true
So maybe they all get knocked out.
And then everybody goes home with an iPod
Or nobody.
[LAUGHTER]
Can we just put the next one up?
Here we go.
Now we have time for everybody
OK, there we go.
So this is the time for all the images.
So is there any of these images today getting knocked out, can you check that?
We're pretty close
Which images get knocked out?
So I'm looking at them from bottom to top
This is the-- I think this one's knocked out
Number one, two, and three.
[INTERPOSING VOICES]
I think David Lamb and Victor Wang got knocked out.
Because this is their image here
That got knocked out.
The last image got knocked out.
[INTERPOSING VOICES]
What's the next group?
In between-- the A group-- [INAUDIBLE] [INTERPOSING VOICES]
This one was the water.
19
Yeah, that was.
[INTERPOSING VOICES]
Uh-oh, they're getting knocked out.
[INTERPOSING VOICES]
OK, that got knocked out, too.
[INTERPOSING VOICES]
Oh, man.
Yeah, nobody competing at this point.
[LAUGHTER] [INTERPOSING VOICES]
That survives.
That survives.
Push!
We have a survivor.
So that might be a knockout by default.
[INTERPOSING VOICES]
Oh, that is out.
Oh, man
Ours is so close though.
[LAUGHTER] [INTERPOSING VOICES]
That's all right.
OK, I at least we have two people right now
Wasn't that the one that was really dark?
This is the one that was sort of darker-- [INTERPOSING VOICES]
Like they don't have enough.
[INTERPOSING VOICES]
So here's the two reference images
We already voted.
We can't vote again.
[INTERPOSING VOICES]
No-- leave them knocked out
Some people voted against it, but-- [INTERPOSING VOICES]
Yeah, but it didn't get the quorum.
It didn't get a quorum to get voted out.
[INTERPOSING VOICES]
How about other group?
First one, let's look at that
Who's this?
[INTERPOSING VOICES]
OK, that survives.
That's good.
Mangpo.
OK. That survived, too.
Two.
That's all right.
Li Chen.
[INTERPOSING VOICES]
Ooooo.
[LAUGHTER]
Oh, man.
That's a...
So that is a knockout blow there.
Now what we're going to do is, we're going to run it on from 48 core machine from the bottom.
So we will let other teams also, if we have time, to come and explain what they did.
We are running-- who's now?
This is a Dobs
Dobs.
OK, why don't you guys explain what you did
Come up here
Need the mic?
I think the mic is off
The mic is on, but it's only for the video
Video
You have to use the mic but you can't hear the speaking
So we parallelized the ray tracing part.
The two for loops for going through all the pixels on the screen.
And then we also parallelized the main loop for caustic elimination.
And we changed the-- sort of the data structure-- how that works.
Used some like bounding boxes to make the intersection faster.
I don't know what else we did
We found ways to get rid of--
Mic, please
We found ways to get rid of a lot of trigonometry, which actually helped a lot.
And we got rid of the recursion in locate photons, which was a major roadblock
Question
What did you do for that?
How'd you get rid of the trig?
Repeat the question
They were converting from XYZ coordinates into spherical coordinates and then back again, over and over.
So we just didn't do that.
[INTERPOSING VOICES] [LAUGHTER]
Slow down
It slow down.
Uh-oh
No.
The seating was done on scene one.
Right?
Oh, yeah.
Oh.
Wait-- Apples and oranges.
[INTERPOSING VOICES] I just ran scene three.
[INTERPOSING VOICES] 48.
Scene three.
So there might have been a slip
No.
But this scene one is much more simpler.
OK, good.
So you're on the board now?
[INTERPOSING VOICES]
How is the seating being done in scene one if it doesn't have a surface that's-- [INTERPOSING VOICES]
--caustics.
It might be that some people did really fast on that one and--
Because we ran everybody's scene three.
And we guarantee that the top end teams are on it.
[LAUGHTER]
The results are rigged.
[LAUGHTER]
He detained the tape.
OK, good.
First team is in.
Then the next team
That's Elizabeth
Elizabeth--
Scott, right?
There we go.
Why their thing is being run
Are you running it?
All right.
So I guess for the display surface, the main optimization there was imagining two planes at the top bound and the bottom bound and seeing if a ray passed through them and then only checking that section of the grid.
And then even within like that square bounding box--
Like if there's a square, and then the ray's going through like that, there's a bunch of grid squares--
Could you use the mic, please?
Like if a ray's intersecting a square, there's still a bunch of grid squares above and below it that you don't have to check.
So we kind of imagined it as a-- projected it onto a 2D surface.
So we didn't check additional superfluous grid squares.
We also did the thing where we got rid of the trig by transforming from polar coordinates to XYZ coordinates.
Although, before that, we used the fast sign and cosign approximation, which was almost as fast.
Using a quadratic curve, which was kind of cool
I think those are the main--
We did an SSE matrix multiply, which barely speeded it up, but whatever.
[LAUGHTER]
We moved stuff into
So you are the team that had issue going to Intel to AMD.
[INTERPOSING VOICES]
Yeah, we were complaining about-- [INTERPOSING VOICES]
If you use Intel, certain instruction is not the same in AMD.
OK, good.
So we are waiting for the time to come?
So we ran it once, but we forgot to time it.
[LAUGHTER]
It was fast enough that we could run it again-- [INTERPOSING VOICES]
So by the way, I want to give a chance for the top team that get knocked out, explain what they did.
Even though they are not in competition because they ran much faster in the same run.
Who's the top team?
[UNINTELLIGIBLE]
Who's the top team up there?
[INTERPOSING VOICES]
They're getting sleep.
[LAUGHTER]
OK, 59.
Uh-oh, now we-- Oops
Ooo
Inversion off that.
OK, next team
So our team also did a same trick.
That was done in the previous two teams.
We got rid of the trigonometry.
And also used a bounding box.
Basically described in the previous team.
And we also tried to use SSE.
But as soon as the news about an AMD machine, we got rid of that.
[LAUGHTER]
Oh, my look at that speed
Whoa
Whoa.
Woo!
[LAUGHTER]
If you want to take-- could we take more time to talk about it--
Yeah, sure.
Yes
Because I'm sure you guys have some things that are different than everybody else's.
In fact, I know it
It's on.
It's just on
It's on.
It's on.
It's already recording.
So for the cache I use like a three-dimensional grid.
Because I realized that as the old times, the torrents is like usually one, unless there's some outlier.
But I don't see one in my case.
So I use a three-dimensional grid.
And use a lot of locks to prevent contention.
So I do the same thing for photon map.
I basically implement my own photon map and just get rid of theirs.
And also, the same thing, like using a grid.
And I spent a lot of time debugging.
[LAUGHTER]
And we also do some minor optimization.
We implement the rendering crest as a reducer.
So that we-- a hyper rod object So that every time Cilk steals some other's job, you will create a local view of the rendered object.
And we found there is a artifact that-- because we are seeding the rendered object.
The default that seed is two, right?
And it will create some artifact in our image, so we use a random number to seed that rendered object.
That's what we do.
And also, we do something like we change the-- no, we don't have locate photon.
Yeah OK.
So we have done a lot of thing to improve the original photon med but after that we decide to change our implementation of photon med so we don't use that.
OK.
PROFESSOR 1: OK, good.
Thank you.
OK. OK, the suspense is on.
So the final team.
OK. STUDENT 8: So one of the first things that happened is that we looked at the code and, like, it was all a KD tree.
And we're like, what's that?
So we got rid of that immediately and we turned it into something-- I think the other groups might have talked about it, but we gave it a nifty marketing name, we called it a photon cube.
And so we created lots of these.
We partitioned the large cube environment into many small photon cubes so we could just jump in and look in the relative air vicinity of photons that you're looking for.
STUDENT 9: Yeah.
So this is like, when you do locate photon, you know the reference photon and you can create a sphere.
So we can do constant time to find the boundary of the photon cube.
Like go from x1 to x something, y to y and z to z.
So then we don't have to traverse the tree.
And we basically parallelized two part, so the one in render and another one in the trace photon that when we try to put the photon in the map.
And also when we create a photon to the cube, there is a reused vector to start a photon in each cube.
But then it's not really good, because when you put the photon in the cube, you have to, like, new photon.
Which means that the memory, it's like spurts throughout your program.
So after that we convert that into a block of memory.
And we use that when we call locate photon.
STUDENT 10: And for the last, about surfaces that intersect.
There were four kinds of optimizations.
First one is checking if the origin of ray is outside of the maximum bound and lower bound of surface, then and check whether by direction is outside or if it's toward the outside then we don't have to check, just return false.
And second thing is like the square cube thing.
When rays intersect we can make a boundary by directly mapping to the XYZ at the surface at the maximum boundary and lower boundary of the surface.
And also when you iterate through the x index, we can cut off again because a bitmap.
We only have to check with the indexes which correspond to ray's z index at that cube, at that x location.
And the last thing was for each square we pre-computed the minimum value and maximum value of y and we compared the rays by value within that range.
And we just cut off if it doesn't intersect.
It doesn't overlap.
PROFESSOR 1: OK. Good.
OK, now this is the suspense part.
OK. We are going to run this and see.
The time to beat is 18.7.
OK, right?
Again.
OK.
There we go.
Let's see.
How many people think it's going to beat 18.7.
18.6
8, 9, 10-- PROFESSOR 1: It's the excitement.
Keep counting.
OK.
There we go
13, 14, 15, 16, 17, 18, 19-- PROFESSOR 1: Oh, oh, oh
21, 22, 23, 24, 25, 26.
PROFESSOR 1: Oh, OK.
It came out as 26.41.
OK. [APPLAUSE] PROFESSOR 1: So you guys are the winners.
Come up here
Danielle.
PROFESSOR 1: Where's Danielle?
Uh, Danielle disappeared
Yes, she did.
PROFESSOR 1: OK. Let's see whether she can find her, so OK. Just hold on a second until--
Yep, there we go.
PROFESSOR 1: You can put it down now, though
So this one's the reference.
PROFESSOR 1: Put a text meter in, that's better
Which one's reference?
This one's the reference.
PROFESSOR 1: I think they don't align.
Can you align them?
There we go.
PROFESSOR 1: OK, so we have a winning team.
PROFESSOR 2: We have a winning team.
And so it's my pleasure to introduce Danielle Smith, who is the University relations for Akamai, and she's here to give you guys your prize.
So we have three winners
I just wanted to make an entrance is why I came in.
Thank you.
First of all, welcome to all of you guys for coming in, and we wanted to give you these prizes.
So we have some cool iPod Nanos, which you may or may not have already, but it can't hurt to have another.
They kind of burn out or you lose one or someone steps on it.
[APPLAUSE]
Congratulations.
You guys have worked really hard.
PROFESSOR 1: Congratulations.
Congratulations.
Congratulations.
OK.
So what's next.
That's it.
We are done?
All righty.
So we have plenty of food, it's going to be right on the atrium.
I know that's the most exciting part.
But it should be arriving in the next 10 to 15 minutes, so we have a few minutes.
I thought maybe this would go a little bit longer.
PROFESSOR 1: We do have a couple of things we can chat about
Yeah.
So just a couple of housekeeping things.
If you need to use the ladies' room or the men's room, it's actually right outside that door.
Out the double doors, you'll see it right there.
I have invited some other Akamai employees to come and mingle and say hello to you guys, so I encourage you, if you see any faces that aren't familiar, to kind of go up and talk about your projects and introduce yourself.
And thanks for coming.
I know we wanted to switch it up a little bit because it was the last class.
We wanted to get you out of the classroom and just kind of shake it up.
So I'm glad that you came to join.
PROFESSOR 1: And the knock tours
And the knock tours.
Thank you.
So on the back of your name tags, there's a number, one, two or three.
One, two or three.
So that correlates with the knock tour that you're going to be on.
We have to keep it to relatively small groups.
So the first tour will be at 10 past 4, the second one will be at 4:40, and the last one will be at 5:10.
PROFESSOR 1: I guess if we don't have too many people we might be able to compress some--
Yes.
PROFESSOR 1: So we'll see whether we can do that, compress--
Yes.
So I'll just call the first tour and you guys can go down.
And obviously it's optional.
Don't feel-- if you're not interested, you don't have to go on the knock tour.
I think that's it.
PROFESSOR 1: Good.
Thank you.
[APPLAUSE] PROFESSOR 1: We can talk about a little bit.
So one thing.
We are trying to get HKN feedback up to hopefully 90%, and we are right now at what, 11?
PROFESSOR 2: No, we're a little bit higher.
We're now at about 16 or so.
PROFESSOR 1: 16.
So that's a little more way to go from 16 to 90.
I hope you guys go back and then fill out this survey.
Very important.
PROFESSOR 2: Please take a few minutes to do that.
It really helps us in, even if you want to say bad things, OK?
It really helps overall to give feedback and the department likes to see these things.
Oh yeah, thanks.
Likes to see how it is that people are responding to the courses.
So if you hated the course, if you liked the course, in particular, we have some questions on there that are specific to how we teach the course next year.
One of the ideas we have for next year is to introduce our recitation sections, which would focus on the use of tools.
OK?
So basically the conceptual stuff, much as we have the lectures now, but then also having something where it really takes you through each tool, and teaches how the use of those tools which would be taught once a week.
So if you think that's a good idea, that's going to really help to have that kind of feedback.
And it will certainly help the students who take it next year if we can persuade the department that that's worth staffing the sections next year.
Also if you have any positive things to say about teaching assistants, that's a great place to put it.
OK?
If they helped you get through this term, this is a great place to say thank you and it only takes a few minutes.
OK?
So I'm sure that some of the TAs put in more than a few minutes, OK, on your thing.
So to take a few minutes and say thank you on that form would be-- I'm sure they would very, very much appreciate.
PROFESSOR 1: One thing, as Charles mentioned, what we are trying to do in recitation is do a little bit of programmer life skills.
So I know all of you guys are probably very smart, can think about the complex algorithms and stuff like that.
But there are a lot of tools that, it's not the smartness but you can spend hours trying to figure out.
But if you know it, it will be useful.
So what we are trying to do is have these recitations to say OK, how to use a debugger.
And even how to change a makefile.
Things like that.
So one thing, until the food arrives, I'd like to say is, get some feedback.
Think of what is a tool you'd like to know or like to learn, or something you think is very useful for people coming next year to learn.
That if you have spent a lot of time worrying and you think you spent way too much time trying to understand, it would be easier for us to think about that.
I mean, I would like to get your feedback.
And also some of you probably will open up with some general feedback on class
I don't know if you want to repeat that.
A more in depth thing on how to use git would be really useful.
Because I know it has things to make your life easier, but I don't know how to use any of them
Yeah.
women are always breaking the branches.
And it's like, yeah, we maybe know how to do this and it is really easy.
And then we're like, use this git command.
And it's like, now your branch is broken.
[LAUGHTER] PROFESSOR 2: Good.
So git PROFESSOR 1: Yeah.
I mean that's because those are interesting tools because, if you know everything right, you can do a lot of fun things.
But if you do it wrong, you end up in huge mess that all you do is scored all over the place and you don't know.
So yes.
I mean I can relate to that
If you use git correctly, if you know how to use git-- If you learn how to use git you will find yourself using it even when you're working on a one person project.
git is awesome.
I can highly recommend learning it
What would be helpful is that telling us that you are running on an AMD box and AMD doesn't support SSE 4.1 instructions.
[LAUGHTER] PROFESSOR 1: Yeah, I mean, this is supposed to be X86 compatible, all those things.
But I mean, some people learn the hard way that, OK, the word is not the same even though-- PROFESSOR 2: But that is part of the lesson of the course, is how do you make high performance portable?
OK?
And so what you choose or don't choose to use is-- But I agree.
PROFESSOR 1: That's why we ran, we want to run it the night before because we knew things like this would happen.
So we can at least give some time to say OK, look.
Go change something.
Were you able to fix that?
OK. Good.
So yes.
PROFESSOR 2: Good.
PROFESSOR 1: And sometimes the problem is, a lot of these small changes you don't know until you run into things like this and oh, somewhere in page 256 in the manual that there's a footnote say, oh, by the way.
And that's what happens in there.
I mean if you had listed everything that would be probably and entire encyclopedia of things that you will see.
PROFESSOR 2: So one other thing that people ask about is, what comes after this course?
OK?
So next term Saman is teaching the compiler course.
PROFESSOR 1: It's Martin.
PROFESSOR 2: Are you teaching with him?
PROFESSOR 1: No.
PROFESSOR 2: Oh, you're not teaching.
OK.
I thought you were.
OK.
So in any case, compiler course is a good follow-on course.
For those who haven't done 6.046, that's a good course and I'll be teaching that course next term.
In addition, how many people here have done a UROP?
Few of you.
So let me encourage you to look for UROPs.
You now have some skills that will make you really valuable.
And in particular my group is looking for-- [LAUGHTER] PROFESSOR 2: Probably two more UROPs than I have already.
Already Boone is one of my UROPs, started just a couple weeks ago.
And so we do have some other projects in both architecture and in software-related things.
And so.
if you're interested, you go to my group's web page.
It tells you five things you have to submit if you want to work in my group.
OK?
That includes things like samples of your code.
Well, you ought to have that by now.
And so forth.
So let me encourage you to do that.
And Saman also has active participation of UROPs in his group.
PROFESSOR 1: Yeah.
So my employees already started doing a project in my group.
And I'm also looking for a couple more people to do some compiler, runtime system, and a bunch of high performance type projects I have.
And also for future course, one course I want to also mention is this IAP.
There's a course that is done with collaboration with VMware to look at how to build scalable large systems using this the Spring and Java platforms.
So if they're interested in trying to see how things were running on one machine and run something that can scale up to hundreds of machines.
If you have ideas of creating an I 8 with web 2.0, interesting thing.
Here is an opportunity to do that.
So if you're looking for interesting IAP courses, this is great.
PROFESSOR 2: I really encourage every single person who completed this class to do a UROP with somebody before you leave.
It's really a way of integrating your knowledge.
And it is one of the most valuable experiences that you can have at MIT is to do a UROP for a professor and get involved in the research of a group.
Because in some ways it's like this course, in that it integrates knowledge from a lot of different sources.
And it isn't just one thing.
But even more than this course.
This course we still are dealing with artificial things.
And in the researcher arena, you're dealing with real problems.
So let me just once again encourage you to spend some time getting a UROP and certainly looking for AUPs and MEng.
How many people here are looking at MEng as a possibility?
So a fair number.
Yeah, that's good.
That's great.
PROFESSOR 1: I want to add to that.
So these days there has been a big philosophical discussion saying why do you have to have universities?
You go to OCW and sit home in your bedroom and you can follow all these courses, and you don't have to pay a mighty tuition.
And wouldn't that be good enough?
One part that's not good enough is this class.
So I mean think about it.
I mean you can't go get the OCW catalog and have this kind of a dynamic.
There's no seventh floor.
All in OCW.
And you have from there to go and get motivated.
So that's one part of that.
PROFESSOR 2: There's no John or Reed or Eric or Josh to respond quickly about what's going on.
PROFESSOR 1: That's one.
But I think one other thing that a lot of people miss when they look at all of the curriculum and stuff like that is the UROP ability.
So a lot of these people, that is something, it's very unique to MIT and a lot of people benefit hugely with UROPs.
I have had UROPs who started with me as seniors.
By the time they are graduating with their MEng they have two first out papers in the best conference out there.
And everybody thinks, when they say they are looking for a job, they are finishing their Ph.D and looking for academic jobs.
No.
They are just finishing up their masters.
And I have a couple of students that managed to become really expert in an area by being a UROP.
So they are finishing up at MIT as a generalist, saying give me a job.
But you're finishing up and have some deep knowledge in some area.
Sitting in and at the same level as most Ph.Ds or most really deep practicing engineers in that area.
So that level, even if you don't do that, just seeing what happens in a research group, being participating on that.
Pushing, seeing something to the maximum limit.
I mean in the class, that's a limit.
I mean even though this class is somewhat open, says make it run as fast as possible.
But most of the time in the class, it's like, we give you something, if you made that, OK, you're done with it.
In research, there's no limit.
You really, really push your boundaries.
What's that?
PROFESSOR 2: So what do we have here, John?
I counted in my mailbox how many messages came from 6.172 staff, and it's about 3,500 over the course of this term.
PROFESSOR 2: Wow.
[APPLAUSE] PROFESSOR 1: How much time-- PROFESSOR 2: So, this is also, as Saman mentions, a programmer life skills course.
And one of the things that you should get used to in life is thanking people who help you.
OK?
I think that all of the MIT POSSE members, all the people who devoted, volunteered their time to help you, could really appreciate an email that says thank you and what exactly you feel you learned from them.
OK?
Because it's one thing to just say thanks, so long, et cetera.
In addition, these people are contacts, part of your network of understanding what the world of software developers is all about, and can serve as useful contacts to keep.
How many people use a service like LinkedIn or something to-- You probably all you mostly use Facebook.
Anybody use LinkedIn at this point?
So many professionals use-- I don't want to pitch for LinkedIn, but it is one of the services that lots of people are on.
So what are some of the other services that people use for professional contacts?
The only thing people try to contact me on is LinkedIn.
PROFESSOR 2: Yeah.
So, good idea to start connecting and building a network of people who you know and who can answer questions for you.
OK?
Such as, gee, I'm considering these two kinds of job offers.
What should I be looking for?
It might be good to get a professional software engineer's input to that.
OK?
Because you know these folks.
You've had a few meetings with them.
OK?
And they know you.
And they may be able to say here's my take on that, as well as getting input from your family and your friends and your professors and other people that you know.
But I do think that courtesy goes a long way in life.
And they really have, these are all people, as I say, who spent time looking over your code.
And you should have heard some of the comments we got from them after project one.
Oh my God, they have a single letter variable names for the whole thing.
Or, what are you teaching these students?
And I said no, this is exactly why we want you there.
OK?
So I think that if you feel in particular, whether or not you feel you got anything out of it, OK?
But in particular, if you did get something out of it, it would be really good idea to not forget to send some email off to thank them.
PROFESSOR 1: OK, we want to get some feedback from them.
PROFESSOR 2: Sure.
PROFESSOR 1: Have open mic?
PROFESSOR 2: Open mic?
PROFESSOR 1: Yeah.
PROFESSOR 2: Actually, open mic, let's do it like this.
Because I think it'll be easier and better for the video if we just have people come up and-- PROFESSOR 1: I don't know that they're going to be too intimidating.
PROFESSOR 2: That's OK.
They can handle it.
These are seasoned-- So, comments about the class.
PROFESSOR 1: How we can improve?
We'd really like to hear that.
I mean things you liked and didn't like.
For example, these absolute gradings we did for projects had passionate support and control let's say before.
I mean, did you guys feel giving the top marke for the fastest and having at least a log scale, no, we didn't do a linear scale going down.
I mean, hey.
Was it motivating or demotivating?
PROFESSOR 2: So what do people think was the most valuable thing out of the course?
I've had dinner with a few of you, and you were very opinionated over dinner.
So I don't mind hearing those comments again, but making them for other people.
So I know it's intimidating, but what I want to do is get a line.
As one person's talking I want other people to line up behind.
And just give us some feedback.
PROFESSOR 1: Come on.
PROFESSOR 2: OK, come on, come on, come on, come on.
Who's brave?
Yeah, yeah, yeah yeah.
You got halfway of your seat.
Yeah.
PROFESSOR 1: Yeah, come on.
PROFESSOR 2: I'm sure you have an opinion.
Who here does not have an opinion?
PROFESSOR 1: Sure.
Yeah.
PROFESSOR 2: Yeah, yeah, good.
And then you can line up right-- there you go.
OK?
There you go.
And then others can come up behind as they're motivated
Good afternoon.
First of all, I'd like to thank the professors and the TAs for this opportunity.
This is absolutely wonderful.
And all the students I worked with.
You were all great.
I just wanted to say when you were talking about what the most valuable thing in the class people thought was, I don't even know how to pick.
Of the projects that we worked on, or that they worked on, I looked at every one and just thought, what of the content of this project have I used just in the past year?
And the amount of stuff that I've made significant use of that you guys covered has been at least 75%, maybe even more, of the material of the class.
And that means that if you look over multiple engineers, you probably pretty quickly get coverage of 100% of it.
All these projects, profiling, parallelism, algorithms, bit hacks, these are all things that I use on.
if not a daily basis, then a weekly basis.
PROFESSOR 2: OK. Good.
Thanks.
Thank you.
[APPLAUSE]
So for me, like, I really had no experience with C or C++ or really distributed version control before this class.
So a lot of my time I think was spent kind of learning the basics of that.
I spent a lot of time initially learning the basics like pointer arithmetic and that kind of stuff.
And then a lot of time learning the profiling tools.
So I wish, I really do wish more time was spent kind of-- well, your recitation, I think, would be a good way to really get that out of the way.
Because the algorithmic stuff was more easy for me, which I thought was kind of ridiculous.
You know?
[LAUGHTER]
The fact that these tools are what's holding back, that I have to spend a lot of time learning.
PROFESSOR 1: This is actually something very interesting people have observed.
So if you do physics problems or math problems, if you run into an issue, after a certain amount of time you will just go ask somebody.
But in programming, people seem to go on and on and on trying to debug this simple thing.
And part of that, part of what we try to get is also not just all the cool algorithms and stuff like that.
To give you a process.
Because there are times when you start saying this program is slow, what you do?
And I think that's a missing part sometimes about the process of debugging.
If the program doesn't work, what do you do?
Where do you start?
I think that's why we are trying to do a little bit of, basically, these life skills
It was great to learn but it was more stressful to learn.
I'm not as willing to ask as a lot of people, so it's hard for me to-- PROFESSOR 1: So one thing we're trying to do is probably-- PROFESSOR 2: That's great feedback.
PROFESSOR 2: We will try to formalize next year the seventh floor lounge, because a lot of people who came there benefited.
But have it a little bit more.
And also get a little bit more life skills type things early on.
Like if you find a problem, what will happen?
It could be a lot longer than that.
PROFESSOR 1: Yeah.
PROFESSOR 2: Yes, so that's our goal, is to make it so that people come up to speed more quickly.
But in some sense, another goal but I have here, next year, is that the C students will stay in the class.
OK?
So what happened is most of the C students dropped out.
So in case you want to know, most people are going to get A's and B's.
OK?
So most of the C students dropped out.
And I would like to have people complete the course, even the C students, feeling that it's useful to complete the course
So I think again, as the person before me, I learned a lot.
I hadn't had almost any experience in C or C++ before the class.
I had some experience with git.
But I think that the class has a lot to offer.
There's a lot of content covered.
And I wish that, for example, there are things that I wish that I could have learned more about, how to implement in a formal way.
Things like SSE instructions.
So I feel that like there was a clear cut off of people who had a lot of experience and they could be like, OK, I know how to do assembly, I know how to do other things, and if I want to optimize this function I can, whatever.
Write it down.
And then there's these other people that don't have that much experience that in a class it mentions this is how you use SSE or compiler optimizations.
It's kind of mentioned but not really more in depth.
This is a case of how do you actually do this in real life?
I mean, I think maybe the examples given in class were useful but very superfluous.
Like you don't really know.
I know that I was looking through other projects that were similar to ours for the final.
And I think it's a class that you taught in IAP.
There was a team of people that did a ray tracer in the multi-programming.
And they used SSE and CMD everywhere.
They packaged everything.
But that's something that I said, it'll be really awesome to do, but I have no idea how to start doing it.
And I think that there's still a lot of room to actually apply the concepts that are trying to be taught.
PROFESSOR 2: I think it's our intention actually to increase that part of the course, the data parallel part of the course.
Which I think we'll be able to do.
One of our concerns about adding material has been, OK, that means you guys have to learn it too.
But I think that if we're able to reduce the workload of the learning of the tools by having recitations, we then can look at adding other material, and expect that the load on students won't go up
Right.
I mean even what I want to say.
It's not even adding material, what I think.
I feel that I was going to class, learning all of these cool things.
And then I was looking at my project and it's like well, these are the things that I know how to do.
And I know that they mentioned all these other things in class.
I don't know how to make them in my project.
PROFESSOR 2: Great comment.
PROFESSOR 1: So I think one thing both good and bad about this class is what happened in-- I think it applied to all of you.
When you try to teach computer science, you get people who are for the first time they are writing any piece of code more than 20 lines.
And you get people who are checking in code for Apache.
And you have that entire range of people in the class.
So how do you teach everybody?
And a lot of times when you give a fixed project, either somebody's completely struggling to finish that, or they are like in a day is done.
They are like, so what's this?
What's new?
The thing about performance is, asked lot of people, figured out there's no end to it.
And you can keep doing it, you can do more, you can do more deeper.
So the nice thing about-- one thing about this class is that there's something for everybody in that.
So if you are struggling still, if you're learning, you can still get to someplace, learn a lot.
But if you have done that when you are in the fourth grade, you can still go and do something more, push that.
So we try to keep that balance in there.
So we don't want to bore anybody.
And also we don't want a lot of people dropping out of the class either
So continuing on with what other people said, I also ran into the same problems with that.
Just the syntax, just the tools.
So I guess I'll start off thanking John and Reed, they're really fast and good at what they're doing.
And I feel like I've asked a lot of dumb questions, and they're good at answering those.
[APPLAUSE] PROFESSOR 2: So John and Reed and Josh all took the class last year.
OK?
And it's amazing, if you ask John how he did on the final project-- [LAUGHTER] PROFESSOR 2: He didn't do so well.
OK?
But it's amazing.
You'll discover, it's amazing what you guys are going to learn next semester after being out of the class.
Because you're on a path to learning this stuff for yourself now.
OK?
And you'll find that in all kinds of things you do that all these skills, you can keep building them.
We will definitely be looking for TAs next fall.
And so people who are interested should-- who are going to be MENGs should-- and even if you're undergraduate, you're still allowed to work hourly for the class.
In fact, that's what John is doing.
So if you're interested, please let us know.
Please let us know.
PROFESSOR 1: So you can have this nice experience as a
Yeah, so, and then I want to say something that I really love about this course.
I feel like it's such an integrated experience.
It's, maybe except project one, everything after that is, you get a piece of code, as you would normally, and then everything is within your bounds of-- you can do anything that you want.
So I really like how it's not limited to like, oh, we're on this topic so we're going to write a little algorithm that implements this particular algorithm, and then move on to next unit, and then nothing ever comes together.
So that's something I really love about this course.
And the next thing I want to say is the beta grading.
For project three, I was working with Angela, I feel like from beta to final, because I don't have much experience with C, so it takes me so long to just get the syntax right, to get my own going.
So for beta, we didn't have anything working.
It doesn't even build.
But with two or three more hours, we got it working.
It's not super fast, but we kept working on it.
And by the final we have something I felt pretty proud.
But the beta grading doesn't take that into account.
Like if you are not very experienced with C, it takes you a long time, longer than other people have experienced to get something going in the first place.
But beta kind of penalizes you for just not having enough time to do things you want.
PROFESSOR 1: I think it's a double-edged sword.
Because if you don't do beta, then you have very harsh.
You have one time code, it's in and out.
If you just really don't do, give that much grading for beta and make it optional and stuff like that, people just drop beta.
They don't just do anything in it.
So we try to do something, give everybody a true chance and get everybody a real motivation to actually work for both sides.
Because for a lot of you guys, beta defined and you learned a lot.
Because you got something in beta.
And for most of the classes I have, we had things called checkpoints.
For example, compiler class, you had to do a checkpoint and checkpoint will be used more like subjectivity to grade.
We don't say how many grade percent checkpoint.
But if checkpoint has nothing, then we'll be very harsh grading later.
We always provide that.
But still, people don't take checkpoint that seriously.
But here we have to try to make something to make people get beta seriously, do something there, and then give them the opportunity to improve.
I think there's a balance in that somewhere.
PROFESSOR 2: And I wouldn't worry about your grade.
[LAUGHTER]
Maybe like one idea I was thinking is maybe there's some component of improvement from beta to final.
Like the proportional improvement that you made from beta to final.
PROFESSOR 2: Oh, that's a good idea, actually.
That's a good idea.
Yeah, to say, if you made a-- Although you want to be careful about people sandbagging.
[LAUGHTER] PROFESSOR 1: Exactly
And then one last small thing-- PROFESSOR 2: I want to make sure we get to everybody
OK, sorry.
PROFESSOR 2: So if you want to go around to the end of the line
That's actually mostly all of it.
PROFESSOR 2: OK.
Thank you
So I've said some of this on my evaluation, which I already turned in.
But I'll repeat it for everyone else.
[APPLAUSE]
A couple things.
One relating to the grading which you mentioned.
Honestly, I found it incredibly stressful knowing that it was competitive grading.
Because we had no idea where the benchmark was.
So it's like, well we've improved it.
But did we improve it enough?
Did everyone else not improve it any more than we did?
And it was just like having it be a complete unknown was pretty stressful.
So I don't know if there's some way you could do like a-- like maybe the TAs got some speed up, and that's the benchmark to start with or something.
PROFESSOR 1: I guess what we did this time is basically we gave the best grade as a benchmark for the final.
And if you go above that, it's just bragging rights
I mean, when you're working on the beta, you have no idea what's going on.
But I did like that.
Once you know where the benchmark is, then it's nice to know what you're aiming for.
PROFESSOR 1: So the thing about that is, the flip side of that is, we don't want somebody that a lot of courses they say OK, I'm done.
When you go there something, Oh yeah, I met the thing, OK, I'm getting a good grade, OK, I move on.
I mean, we want people kind of to keep pushing in time.
Because sometimes the thing is-- If I have a minute, I'll tell the story.
There's this guy named Tony Danzig who basically invented the simplex method.
His thing was, he was always late to class.
He went to class one day and there was-- normally the professor writes the problems on the board.
So he copied the problems down and went home.
And they were really hard problems.
And so he was like, damn.
I don't know how to solve this.
And he worked very hard and solved it.
And wrote a note to the professor saying I'm sorry this is late, and put it on his desk.
And a couple of hours later he gets this excited professor running around trying to find him.
Because what the professor had done is written two unsolved problems on the board.
He came late to the class, he didn't know that, he had two problems, he copied them down.
He worked very hard and solved it.
[LAUGHTER] PROFESSOR 1: And so the professor said divide this up, you get your pay at the end.
Then there's lots of it.
So this is something like that.
If you give the thing, if you know where we are going, for some people it will be a hard climb.
But other people, they will get there faster, they'll say OK, I'm done.
And what we want to do is give them the ability to kind of really push to their limits, not some artificial limit.
But not to penalize too much for that.
We try to balance this out.
So I want everybody to go to their best limits
I had one other thing to say.
Have you considered or would it be possible to make the class more than a 12 credit course?
[LAUGHTER]
No, seriously.
Since there is so much material and we keep talking about wouldn't it be nice to add this and add that.
But I don't know about you, but I spend a lot of time in this course already.
And I mean, I think it's the kind of class where you could pack 16 or 18 credits worth of material into it.
PROFESSOR 2: More realistic would be to have a follow-on.
PROFESSOR 1: Try to make it as two classes.
PROFESSOR 2: To make a two-semester sequence.
But I think Saman and I still feel that this, it's like, OK, that's a new development effort.
That this class still-- I think we feel that there's more than incremental improvement we can do to make the class teachable.
For example, by introducing recitations.
And so I think once-- maybe after next semester, next fall we can we can-- But thank you, because I think that that would be good.
And the whole idea of the sections is to try to reduce your load.
There used to be a thing that was on all the evaluations that they no longer do.
And this is actually before I got to MIT.
They had a measure called cums per tooling hour.
OK?
So a cum is how many units of knowledge you're using in whatever measure, and tooling hours, like how hard you work.
And they got rid of it because they said it was not quantitative.
After all, what's a cum?
All right.
Well, regardless of the fact they got rid of that, because I think that it actually was a very useful measure, even if people are subjective about it.
That's our goal in the course.
It's not to make people spend a lot of time and then have learned epsilon.
It's to have them, for the time they put in, learn as much as possible.
And so I think that we recognize it's a lot more work right now that you folks are going through.
We hope that it's rewarding work.
But we hope we can do even better in terms of giving people the opportunity to learn these things.
PROFESSOR 1: So we are thinking about cutting things.
We all are thinking about what to eliminate even if we add something.
It's very hard because we think a lot things are useful-- important.
But something has to give.
So that's the hard decision.
PROFESSOR 2: Yeah, we're thinking of cutting caching and parallelism.
[LAUGHTER] PROFESSOR 1: OK.
PROFESSOR 2: OK. We have to hurry because we gotta be out of the room by 4
I just wanted to say that I agree.
That I didn't really like the way the course was graded.
But I did like the material in the course a lot.
So that was good.
But for one thing, the thing about the competitive grading was that it was way more stressful than necessary.
My luck was good because we had the kind of benchmark thing.
It was a reasonable high performance to attain.
And also I didn't like the way some of the stuff was weighted.
For example, it didn't really correspond to how much time I spent on it.
The pset was worth like 3% or 4% of the grade and it took like hours, I think.
Many, many hours.
That's all.
PROFESSOR 1: OK, thank you.
PROFESSOR 2:
There is one thing that's pretty general about course six, I guess, is that they're really lenient on prereqs.
Since I've only been at MIT for like three months.
[LAUGHTER]
So yes, I'm a freshman.
PROFESSOR 1: One freshman in the class.
There you go
I'm really thankful that Professor Leiserson and Professor Amarasinghe let me into the course without any prereqs.
And I like the breadth of the topics.
But I think with office hours we could have gotten a bit deeper into them.
And one suggestion I'd like to make is that get the nightly builds working.
It was there for one night for project one, and that was it.
[LAUGHTER]
So I decided this is my last-- PROFESSOR 2: Can you talk in the mic?
So this is my last term at MIT.
And I decided to take things easy by not taking that many courses and TAing this course.
[LAUGHTER]
So yeah.
It would be nice to have 48 unit TA shifts too.
PROFESSOR 2: Thank you
I just want to say that this was like-- should I take it?
PROFESSOR 1: Sure
OK. That this was like an amazing class.
I think everyone can agree that we learned so much that is not found in textbooks.
And what I thought was really weird is that every little random thing I've ever read on the internet related to coding, I guess, has come up in this class in like, useful, come up like useful in this class.
Like every little bit of outside knowledge that you take yourself has appeared useful in this class later on.
And I thought was really cool.
And related to the hardness that people are talking about.
I think everyone came into the class bracing for the absolute worst, because we've heard about you guys.
[LAUGHTER]
But, yeah.
I was actually really pleasantly surprised that there were helpful TAs waiting in the G7 lounge for you, even before office hours start.
And so I was actually really pleasantly surprised by-- just like everyone, the class.
I feel like everyone who has survived until this point has some sort of magic skill.
That you really learn a lot by working with them on the group projects.
And so I was just really inspired.
And I thought it was a great class.
PROFESSOR 1: Thank you.
PROFESSOR 2: Thank you.
[APPLAUSE]
Hi.
I love the class.
I appreciate you Professor Leiserson, and everything that everybody else before me said.
I just wanted to mention two things I don't think were mentioned.
Beta programming, to be honest, was amazing because except the final project, everything I did was beta programming in the projects.
And the amount of bugs that we avoided because of that is ridiculous.
Especially when we're doing pointers, and if it doesn't work, make it a pointer, and all those philosophies that we came up with.
That's the first thing.
Second thing is, and I've already mentioned this.
A book or readings for this class or links.
Because as amazing as the lecture slides are, a lot of work done there, so thanks a lot.
They're hard to connect together, especially when you're trying to figure out what our cache-oblivious algorithm is trying to do with those.
So that's pretty much it.
Are you telling me the stack overflow is as much as this course?
Well, thanks a lot again for the TAs.
Most of my learning came from them, to be honest.
[LAUGHTER] [APPLAUSE]
Hands on learning.
Hands on.
Thank you.
[LAUGHTER]
Hi.
my name is Harold Pokob, I'm the head of engineering at Akamai.
So I wanted to use the opportunity to thank all of you for taking the class.
Charles was my adviser when I was at MIT and he told me about his plans for the class.
I thought it was a great idea.
Getting a lot more hands-on experience going.
Back when I spent time at MIT, it was a lot more theory, and Scheme was one of the main programming languages.
So in that context I really like this type of class.
And I'm glad MIT is investing in that and supported from Akamai.
So I'm honored that we could sponsor it.
I'm very happy you're here.
There is, I think, refreshments downstairs in a few minutes.
PROFESSOR 2: Yeah.
I'd also like to ask Harold to say what the topic of his master's thesis was.
[LAUGHTER]
My master's thesis was cache oblivious algorithms.
[LAUGHTER] PROFESSOR 2: So you have Harold to thank for one of the papers that you read during the term
It's nice that something that I did in academia is still used.
Not all now practical business stuff that I do these days.
Great to have you here, and I hope to see some of you downstairs.
And I've got to actually go into a meeting at 4:00 for half an hour, but I'll be back down afterwards.
Thank you.
PROFESSOR 1: Thank you very much for sponsoring this event and getting all the food.
[APPLAUSE] PROFESSOR 1: OK. Now we have food
We also have quizzes here.
[LAUGHTER]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome, everybody.
So this is the one group meeting that we'll have with all you folks.
We really appreciate your willingness to come and help us teach 6-172.
Saman will explain what the course is all about, and then I will give a presentation, a little bit, of what your expected responsibilities are, and then Eric, who is one of our TAs, will present what the first project that they've been working on is all about.
We think we have a pretty good acronym, OK, MIT POSSE.
Masters in the Practice of Software Systems Engineering, OK, and it is very much like a posse, in that we're trying to sort of deputizing you folks to help us deal with this mass of students that we deal with.
There's no way, given the resources that we have available at MIT, that we could possibly give the attention to each of the students examples of code, and so our strategy is to go out and deputize some people to help us teach.
As we'll discuss your responsibility is not to do any grading, OK, or even any understanding of the course content, OK?
But rather to ask good questions of the students so that they understand what it's like to do software development, and you'll see that most of the students are very immature when it comes to coding practices.
They have taken a minimum of one systems course before this, basically
Probably two classes where they write very small programs--
Where they write a very small programs, so they have no sort of sense of writing scale.
Moreover the class that we're teaching because it's on the topic of performance engineering, they are inclined to break all the rules in order to get the best performance.
And that's part of our game but of course what we want is, we want the countervailing-- you still have to make things maintainable, you really should do it this way, yeah it's fine to violate some abstractions here and there, but then you should document that you're violating those abstractions and so forth.
And that's what we're counting on you for, is to insert some experience and common sense.
So, Saman, take it away
OK, good, so to add to what Charles said, when I was an undergrad, when I took something like a science class, or a physics class, or engineering class, most of the problems had kind of right answer and a wrong answer and then it's very easy to grade those things.
But when I took something like an English class, all the papers I wrote, somebody really went through carefully and graded it and gave me lot of insightful comments so I can kind of develop my own style of writing.
And what happens in software is the programming there's a style that each individual has to develop their style, but they can't develop a very haphazard style so there's a certain set of good styles, certain bad styles.
And the problem that is in a large class we can grade for correctness very easily, we can also look at the programs and have a very fast thing, okay, saying, here are no comments, or your comment doesn't seem to be, it's too verbose, or some very high level feedback, but it's very hard to help somebody build their style.
And so what you guys are coming into is to help us get these young students create their own style of programming.
The nice thing about this class-- and also sometimes last year the masters found it a little bit hard was, we are giving them small amount of code.
They are doing performance engineering on a maximum couple of hundred lines of code.
There's no very complicated class structures or something like that.
So we are not looking at the style and software engineering in the macro level, how to structure large classes but more at the micro level.
In each method do they have right style?
Are they is it clean?
Are they commenting them properly, or can they do it better?
So at that level at least the building blocks that they're-- as they go build a larger and larger program they can hopefully use.
So that's what we're looking at, and it helps to instead of dumping you ten thousand lines of code, you only get a couple of hundred lines of code, so you can actually go through it a lot more carefully, and help them out.
So this is starting with a crazy eye chart that is our new curriculum that we introduced in the department.
So what you have is some early courses in here, these are basically very general math courses, early courses that do-- they probably write about ten lines of code type thing in these courses, and then we feed into these foundation courses and then after that we sit at, this says 197, actually what should be 172.
We sit at that level, so we are sitting at that level, and then the prerequisite for people who are coming is that they're taking a computer architecture course so they know a little bit about architecture.
They have taken this introductory software course, so they have written some Java programs, and so they know a little bit about software engineering from that course, and they have taken algorithms course, so they understand algorithms somewhat, so this is the background of the people are coming in.
At the low end there might be some people who have only written couple of thousand lines of code in their lives, and there are other people who are, written half probably-- the Linux kernel.
So there are kind of very diversified group of people.
But there's some people who it's really new to them, and they don't even understand why you had to do good nice programming, why do you have to document these things.
I mean, for them it's a very new concept, because they haven't seen that.
So OK, if you're writing ten lines, OK, why do you need document that, I know how to do this.
So part of that is this is, for a lot of them it's eye opening, and in last year also we talked to students, they actually gained a lot from talking to you guys.
It's easy for us to tell them do it, a lot of them say OK, just to get the grade, I will do it, but after that I'm never going to do these things, and they just go off in very like, mad at us for asking them to do it, but when you guys say this is what happens in industry, there's a lot more impact on what they're going to do.
And so I would go through very fast, so we have done case study on Matrix Multiply show doing how you can basically do things like algorithmic tricks, architecture tricks, and stuff like that get really good performance on Matrix Multiply.
So we started with doing very object oriented, immutable type matrices, Matrix Multiply and then go down to doing blast.
How much a performance gap do you think that has?
It probably depends on how big the matrices are because--
Yeah it is 2k by a 2k matrix so it is, yeah-- So somebody said 200x
10x
Yeah 10x, so actually, in fact we've been-- since I started with the immutable matrix, very object oriented stuff like that, and then did all to optimize I could, I got 300,000x.
So this is kind of extreme case, but it illustrates how much, kind of, performance sometimes is hidden in these applications.
And then we talk bunch about, these are done lectures, talk about some very simple things in performance engineering, how to optimize loops, things like how to get the compiler to do certain things.
Some basic facts, we have bunch of rules in there that we went through fast.
And we are going to do introduce them to performance analysis for a lot of these people.
First of all, they don't even know their code is running bad, because they don't know how to look at it other than getting one number then.
Even if they know that their code is running bad, how do you go about figuring out what's going on?
They know a little bit about GDB but they don't know anything about performance analysis, so we are introducing them to tools in here.
And we are talking about C to Machine Code, lot of them are Java programmers, we already gave them a C primer, saying if you're Java what does C mean, kind of a thing.
And I actually talk about computer architecture today and talk about the entire process they're going to use.
They're using this Nehalem processor and all of the interesting things that happen inside the processor that they had to be aware of.
And the same, maybe I will talk about the memory system optimization, and then Charles is going to talk about algorithms and data structures for caches in there and some about storage allocation.
We'll have a bunch of lectures on parallelism that Charles is going to use Cilk Language to illustrate how to write parallel programs, that's what we are going to use in this class.
And then more on parallelism, some things like data synchronization issues, dig deep into other performance issues in parallelism in there.
And then since their final project is on Ray Tracing, we'll introduce them to a Ray Tracing primer, and see how Ray Tracing is done and so they can go into their final project.
Then I'm going to talk a little bit about compile optimization, what compilers can do for you, and how to get compilers to do those things for you.
And a little bit about distributed systems, so how can you scale out on this point?
It's about getting a small piece of code to run fast, now how do we get millions of users go to this very large scale system, and get them to work, and what issues are involved.
And then we are going to also have about three or four case studies sprinkled around.
We'll get someone from University come and talk specific problems, OK, look I had this problem, and at the beginning it was bad.
Here's how I went about figuring out why and here are the things I tried, these are things worked, these didn't work, and how we went go.
So kind of a very hands on case studies about doing performance because what these people has to do is develop a methodology.
Their own methodology of how to go about solving these problems, and then looking at these case studies will give them a feel how a very experienced person will go about looking at some of those problems.
So these are the kind of lectures we are doing in a nutshell.
So the projects we are doing.
So students' mission, what we do is we write some inefficient piece of code, not that long, and we give them the specific, here so this is functionally correct but it runs darn slow.
And what you can say is, OK, take advantage of right now, the first project is basically looking at algorithmic and data structure inefficiencies.
So OK, trying to find better ways of doing that and change this program to run faster.
So that's their thing, so they don't have deal with thousands of lines of code.
Small program, it's correct, to run them faster, and this we'll also be asking them to write tests, so you can actually test their implementation, and make sure that implementation are also correct.
And basically there's no right answer.
This drives students crazy because they are like, "When am I done?"
I'm like, "You're never done until the deadline," because you can keep doing more and more and you could keep increasing performance in there, and of course if you hit very marginal returns and at some point and you might decide, OK, that's good enough.
And also more times the journey is as important as the outcome.
So at the end of the day you can say I got it very fast, but how did you get there?
What's the process you followed?
Some of these programs are easy to get there, but if you haven't learned that process you can't scale it out.
And so, a lot of times your part is to talk to them and say, "What did you do?"
And say, "Okay look why did you skip that stage, why didn't you profile it?"
And get them to get do that.
And some will say, "Yeah, I just look at it, I realize what it is."
Like, no just do something there might be something hidden in there, so get through the process, understand the process, that's the important.
So the way the project works is, we give them a project, we start them the project, and there might be, some projects have two parts in there, and final and a project request design document for them to give us, saying what they figure out what's bad and what kind of optimizing they are thinking of doing in this project.
And then we have a thing called a beta turn-in.
They said OK look we ran this, we got the project and turned the product in, an we call it a beta, because I'll tell you why.
And then what we do is, we run everybody's project against everybody, basically we run every project, and we figure out who got the fastest implementation.
And we said OK, we announced this and said this is the fastest implementation, and then we figure out how off are you from the fastest, and we have a proportion of the grade that's basically computer graded.
So we figure the fastest, you'll get 100% for something that on the other hand if you are 10x slower you get 10%.
Except, the first time we tried, the fastest was 1,000x better than the slowest, so somebody got like 0.1%, that didn't work, so we are doing square root of the former so they'll be little bit more, in the somebody, got 1,000x there.
So sometimes it's because somebody just missed one critical insight and so and also they might not have done the test properly and stuff like this, so we give them opportunity.
And then after this beta turn-in you get this code, and the TAs get this code to run and do a very fast check over that and that, and then they are going to have a week to find a time with you, sit down for hour or 90 minutes, and go through this code.
And then they will get all the feedback, the performance feedback, and they can talk to people and figure out what they missed and they get another chance to fix all those things, get the performance up to turn in, and there'll be another performance measurement, but now if they match up to that point, they can get a full amount so of course, if people exceed they don't get extra, because we want to limit the competition, but at least you get a chance to kind of match up the best code.
So, what you have is this design review week, that they get to talk to you, and get feedback and, then after that, they fix everything up and then resubmit it
You should mention the beta tests
Yes, so what also we do in this is we're asking them also to create a test suite, and we are going to run everybody's programs against everybody's test suite, and also give them points on coverage of their suite, and also how well their program does against everybody else.
So if somebody finds lot of bugs in other programs, or some there's a test that founds couple of bugs that nobody else found we will give a lot of points for that person, because that means they went and did a really good job in creating a test suite.
So we are actually using that to basically emphasize on testing and testability issues in there.
And then final, when you're turn in, then the TAs will, at that point, will grade for performance and also the coding style.
So that time they will go to the code and assign the grade.
And so if the students have listened to you, and carried out what you said, they should get really good grades on that part, and then they had to also provide a write up in that.
So this is kind of the flow of the projects we have.
And the project one is a simple algorithmic and data structure trick.
So we did things like data rotator, bit flip counter, and this Pentominoes Puzzle that he's going to explain a little bit.
So we give these things, that we have a pretty inefficient representation, where most of them has a much better bit representation, and you can do bit tricks to basically get good performance.
So these are algorithmic and data structure representations that can have huge performance gains.
So this is the first project and this is what next week you're going to get.
So project two is more about memory system performance, things like image rotator and number sorting.
We have two parts, the first part is we give them these problems, and they're supposed to run with different data set sizes, look at performance counter results and identify what's going on with these programs.
So there's one thing that we'll give them bunch of source compiled, and they have to figure out which sort is which by looking at performance counters and different ones.
So you had to identify, so basically give them idea to basically observe a program behavior, and then we ask them to go and fix some of the performance issues as a second part.
So this is a lot about basically getting performance information, understanding a lot of hardware, what's going on and then going and doing some performance fixes.
So there'll be two parts, to this project.
And the project three is implementing efficient storage allocator.
Most of these students find after they've done the Java and Python, they come thinking there are these objects running in memory, with a thing called pointer that points to, and to understand what a pointer is is sometimes hard.
And this is an interesting example that they actually do a lot of pointer manipulations, and you have to get a lot of those things right and you have to have good discipline to do that, as well as getting some performance, so this is interesting project in there.
Here you don't get too much of performance variance but students have to think very carefully about basically aliasing data structures and stuff like that.
So getting some of this dirty little things you have to do, and learning how do it right.
And then we go into parallel programming.
So first part is introducing the concept of parallelism giving them some simple programs to basically parallelize, and then the second part is we give them insertion detection we're inserting a bunch of lines into a image and then they have to figure out when these lines move when they actually intersect, and so it's not embracing a parallel, you have to actually do by intersegmenting the data into regions and basically figuring out where communication is.
So there's a lot of issue in there to get it done.
It's a fairly complex program that they have to to deal with and so they're going to have to think through a lot of parallelism issues, synchronization issues, communication issues, all those things come into play in here.
And then project 5 is just a problem set because we want to get to make sure that they understood some basic material we did, so you don't have to deal with that.
We give them, actually, a write up they don't have to do any coding in there and they turn that in as a normal problem set.
And the project six also you don't have time to critique that because it is delivered the last day of class, but if you're interested in basically giving them some feedback, or, just come in to the finals and see how they do.
So there what they do is, this is a really fun project, we give them a very unoptimized ray tracer, and we okay here's an image that it can generate.
If you generate image that looks the same to us, you'll be OK.
So you can change the algorithms you are doing, come up with other things you can do, where you can still maintain the visual equivalence and then you can do that, you can optimize, you can parallelize, you can do anything you want and build a ray tracer.
So they go and then we explain to them the basics of ray tracing and they do lot of cool things and a lot of these kids actually really change, think through algorithmic issues, even look at a lot of graphics and find different ways, or better ways of doing things, and come up with really cool tricks and then do really well.
So for that we will invite you guys to come to the-- then we will have basically a bake off, everyone will come and figure out who has the fastest ray tracer, and they have the bragging rights, as well as the grade of having the fastest ray tracer, so--
I'm going to go through this
Okay, you go through-- we have the schedule here.
So the most programs are done in C, and very little C++, not very complex C++ classes, or anything like that.
We want them to be very close to the metal.
They have been trained in a lot of abstract programming in Java, Python and stuff like that, now they're going to get to the metal, and understand things like pointers, malloc and free, native data types and stuff like that.
We're not dealing with things like dealing in garbage collection and bound checks, and all the other things, so they're really seeing what's going close to the metal, and sometimes we are even encouraging to look at at a little bit of assembly work, that's not really required
Have these students had exposure to any kind of assembly language programing?
In 004, they are using some assembly programming for in the architecture class, but I think it's a very simple, I think RISC machine inspection they're doing.
And then we will have our course website as you go about you can look at what the course is doing and also if you change this F10 to a F09 you will see last year's code.
So you can actually even see into the future what we have been doing.
So if you want to kind of preface from the front
I took a look at this website and found that most of it was unavailable because of--
We should have Word readable we need to make it if it's not Word readable--
It says it's Word readable.
To us
Some of it is a lot of it was not I've said you need to, I mean it's all available for-
Certificate?
We will check that, because all this things that we have set up in the background to get everything Word-- we want to make everything Word readable, so we'll make sure that happens.
So, OK, so that's all I have about the class and one thing interesting more so this projects, the best to worst can sometimes be 1,000x difference.
So what that means, even for the people who are struggling, they can get something out of the class.
But unlike other classes where you just read the limit and then you're bored, you can really keep pushing the limit, especially things like the final product, the people who did well really pushed the limits, they could have done that for a graphics class type thing.
I mean they pushed algorithms changes, did amazing parallelization techniques, data partitioning, everything.
And where some people just kind of stayed within that and did some small parallelization got some like low hanging fruit in there.
I mean what we're trying to do is, computer science these days, we get some students, as I said, their first time they're coding anything more than ten lines and other people who are already a submitter for open source projects, who have written millions of lines of code, we have these two.
And these codes, I think we are trying to cater to at every level, and keep everybody basically very motivated, and excited to be in there.
And I think the last year it was a pretty good success.
And they really enjoy meeting masters last year, and after we go to discussion we can talk about that a little bit
So we have two websites, as Saman mentioned before, so here they are, and then here's the schedule.
So we'll run through this, in more detail for what is expected of you, in a moment, but basically there's four projects that's sort of are organized along here.
Oops, that typo didn't get fixed.
That's should be November 19 obviously there, instead of October 19 if you want to correct that on your calendars.
OK, so in general it begins with an information session which you're allowed to attend in person but most people just dial in, it's more convenient.
Then we will provide you with the beta code submitted by your four students and then you'll have two meetings that typically are 60 to 90 minutes with each of your two pairs.
OK, so let's go through how this is working.
So the first thing is just to make sure you folks recognize that you're making a commitment.
It's very difficult for us, and it's even more difficult for the students, to cope with something that happens mid-term, OK?
Where suddenly their master disappeared, or whatever, so we're very big on anti-flakiness, if you will.
And so if you believe that there's going to be a problem please let us know as much in advance as you can.
Sometimes things are unavoidable, they come up, we don't fault you for that or anything, OK?
Just let us know, we'll try to deal with the situation.
But for the most part we're really asking you to make a strong commitment to the class, and so if your situation is such that you're unsure it's best to talk with us or politely decline to participate, rather than putting on the rosy glasses, say, "Oh yeah I think I can do this, even though I know that I've got a major deadline that conflicts with one of these projects," or whatever.
It's just really difficult on the students when things change.
They are sort of going I don't know if you remember what it's like being in school as an undergraduate, but pretty much the students go sort of hand to mouth, in terms of like, they look and they say, OK I got this deadline this week I better work on that one.
They don't really look ahead and so forth, and so it's up to us, to some extent, to recognize that they're in that mode and that we need to do things to make that easy for them.
When they get sick or something mid-term it can really throw a student way off for the term, and so certainly when something happens to the staff, that sort of, at that caliber, can really knock them off their trajectory.
The design and code reviews should normally be held at MIT, and the students will have some locations.
If you are within walking distance to campus however, you may suggest an alternative place to meet, such as at your own workplace, if that's more convenient for you, OK?
But if a student balks at doing that you shouldn't pressure them to accept.
The content of the off campus meetings should be professional, it's strictly the review process, no lab tours, no free lunches, or dinners, or what have you, OK?
And we'll talk a little bit more about that later.
But the main thing about them is that they can be scheduled in the evening or you know whatever time you and your two students find convenient to get together, OK?
There's no limits as to when you know what time those can be.
They can be on the weekend they can be, whatever you can work out, very flexible.
So here's the basic guidelines, each master is responsible for reviewing the code produced by four students.
The students typically will work in teams of two or three for the projects, OK?
So what you will have is your four students will be organized into two review groups of two students each, but none of the students will be on the same project.
The reason for that is that we want to make sure that every student who goes to represent their project can represent the whole project.
So they're responsible for the whole project.
They don't have a buddy there on the same team, who can cover for them when you're asking them questions about their project.
You ask them questions they've got to be able to answer about the whole thing, even if they didn't code every piece of that.
We're encouraging them generally to do things like pair programming, so they really should understand what's going on throughout.
The reason for having two groups, rather than meeting them individually, it turns out there's actually some good knowledge of one of them sort of being able to see what the other teams are doing.
So generally they're not allowed to share their solutions, et cetera, with other teams, but in this meeting that's OK.
The ideas and so forth can come but they shouldn't be making copies for each other of the code of the other group.
But they can say, oh is that what the other team did, And so forth, and there can be some learning for them to submit the final project
Also last year, lot of masters very successfully used the other team members to kind critique each other.
So showing something do you understand this, and then they can look at it and say, "Yeah, I can see what's going on, that's very useful," also when we go to masters we can talk about of that type of thing
Also things like documentation.
You can look at the code there and ask the other one, "Do you understand what that documentation says?"
And so forth.
It turns out it's actually easier than that, because usually if the documentation is inadequate, the student himself has trouble remembering what they did, right?
I mean you know how this works with code, so.
So consequently you're responsible for generally for two review meetings for each of the four projects.
Now as it turns out we'll assign things, we're going to have things like students dropping the class, and so there will be some minor reshuffling as we go on.
Some of you may end up with only one team of two students.
Some of you may find that you're reconstituting it.
In some cases if we end up-- we may ask somebody, "Can you cover an extra team this time because somebody is going to be out of town?"
Or something like that.
So there may be some variations on this, but this is basically how things go.
Now, one thing you should understand is that you are not going to grade the students.
So, one of the things I learned long ago was that when you grade somebody they behave differently than if you don't.
So you're not grading them that has a plus and a minus.
The plus is that then this is really pure feedback for them.
This is to help them, they know that you're supposed to have read it.
The downside is some students will say, "Eh, then I don't need to listen, because nobody's holding a stick over my head."
But the reason we do that is twofold.
One is because we think that the quality of interaction is better, and the second reason is because you're not empowered by MIT to give grades.
So you won't be grading them.
You're also not responsible for the technical content of the class.
We're going to be teaching stuff, hopefully which is sufficiently new, some of it, that some of you may not know all of what we're teaching with respect to parallelism, and so forth.
Some of you may that's great, but that's not the role.
We will take responsibility for teaching the content of the class.
What we're really after here is for you to listen as an experienced program developer, and provide feedback to the students about software engineering.
In other words explained to them what's going on.
"Why didn't you write that in your documentation?
How come all your variables have only one letter?
Why is this stuff wrapping around three lines before you-- why no white space?"
This kind of stuff.
So, remember they're very smart students, but they're also very immature many of them.
Some of them you'll discover are like way mature, but there's going to be some of them that are just absolutely this is new to them, and what they need is some encouragement and some suggestions as to what they can do to improve the quality of their code.
And since we're not in a position to review it, why we're deputizing you as members of the MIT POSSE, is to deputize you to help them produce better quality code, which is not something that's easy for us either to give feedback on, or to grade.
So hopefully when we are grading them in the end on their final submission and the quality of things, it reflects that they've learned something from you folks about producing good code.
Also you want to be careful, this is one of the bugs I have when I interact with the students is, I tend to do most of the talking if I don't check myself.
And one of the really good things is to have a guideline that they I don't know how many of you have teenage kids, and if you lecture them how quickly they learn right?
No, they don't.
So the goal here is to help them learn.
And so lecturing usually doesn't work.
What works generally is getting them to do the talking.
And then it's great to pepper your interactions with anecdotes.
I work with one person who did this, or early in my career I did this and here's how it blew up in my face, that kind of thing.
Excellent to have anecdotes that you can give with real world experience.
Why you might think it's like this, but in fact, if you do it that way you're headed down a very bad path.
Let me tell you story about that.
That's fine,
Or if you find a very stubborn person saying, "If you do that I won't hire you."
That counts too.
You would never get a job here if you coded like that.
And in fact that's actually what Saman says, that's actually one of the big benefits of having folks from industry providing that feedback, because we can say that, if you do it like that you wouldn't be hired by XYZ corporation.
You can say that, but if you say, I wouldn't hire you, if you did that, that has a, it's like, oh!
It has a big impact, whereas when we do it it's like eh, you know.
It's like what do they know?
So the guideline is make them do a lot of the talking, so that they're trying to explain and you teach them a little bit what a design and code review is.
Now, in fact what your review is, is both design and code review
To add to that, what we have told them is to prepare for what a filing statement type thing, to walk through and explain what they did to you guys.
So they will come with some preparations stuff that they can do that, let them, both of them kind of do that, and then start them off the instructive part
And maybe they'll do that.
But that's the goal is, you should say, "Look, when you come you should prepare something that brings me up to speed," and teach them a little bit about what's expected of them when you have a design or code review in industry.
So for each of the projects, as we mentioned, there will be a short information session, typically by dialing conferencing, except for this one, which we'll do in a minute.
After the students have submitted their beta releases we will get the code to you.
Last year we had the students responsible for getting the code to the MIT POSSE, and that was basically a nightmare for everybody involved.
So this time we will make sure that we get you their code, and I hope that way the unreliability-- I don't know if you remember but like, these are mostly like juniors and seniors, and when you're juniors and seniors you're not 100% reliable in everything you do and your word is your bond is not necessarily, oh I forgot that my word is my bond, kind of thing, right?
So by us getting it to you, we'll remove one layer hopefully of flakiness.
Next what you'll do is, you'll coordinate to meet with a mutually agreeable time, and then after the review you're going to provide us with a little bit of feed back, it doesn't have to be long, just a short email, not as a grade or evaluation, but to help us understand where the student's are doing well, and what they're missing.
So what we may need to stress in class and so forth.
And then, of course, after the review, they have the opportunity to make changes to their code before they submit their final revision.
Now, that's a little bit different from industry where you would go through typically some kind of feature freeze, and code freeze, and so forth.
Our beta is beta sort of in name, but not necessarily in all aspects, in that they can actually completely rewrite their code between the beta and the final version
So in your feedback you can also tell a little bit about in your interactions what worked and what didn't.
This is the first time, as far as we know anybody's doing this kind of a thing anywhere.
We are also developing a process so good practice is very--
And since your email will go to MIT POSSE and everybody will read it, what hopefully you can do is, some of the practices that you find useful, I tried this at work, I tried this, it didn't work.
That starts to get spread out among you and is becomes very helpful for each other to say, "Oh, well that was an interesting idea let me try that."
And so we hopefully we'll be able then to start compiling some best practices
Will we be identifying students by name in those broadcasting minutes?
Yes, yes, in fact here's the thing is, what you're going to do is give, well it one, names the students who attended, and that helps us keep track of who's actually going to these and who isn't.
The strengths and weaknesses in the student presentations.
Some suggestions for improving the assignment.
That's suggestions for us to improve the assignment for them, and any other comments.
So we got some really good feedback last year that, "Hey the code you're handing out as the initial code that was really sloppily documented."
Last year we were much more behind the eight-ball in terms of trying to do this.
This time we've had more time, and in particular we also have some really outstanding TAs who've worked very hard on the software engineering of the project.
So I think that we'll be better in that regard, this year, than we were last year
I just had a question
Yeah, sure
You mentioned that they're going to give to you and you're going to pass the code out
Yes
Do you mean just the code or are you thinking structured more as a design review, they really will be talking about, a review package.
It would be the code plus I don't know whether you're doing documentation generation or any sort of an intro or front matter?
Yeah, so the question is whether in addition to the code there should there's going to be any other documentation, or package, or whatever.
In fact, most of the final documentation is going to be for the final project and not for the beta, and so they should have things well documented within the code, for example about what their strategies are, and how they work, and so forth, but there won't generally be a write up.
They should be providing you at your meeting with a brief outline of what's going on, and you can ask them after the first meeting for any other thing that you think would be helpful to them and to you in making the design meeting more productive.
One of the things always in these things is how much do you require out of the students and at which time.
They have in this project, we're on a relentless series of deadlines for them.
And so we've tried to do as much as we can while being still relatively minimalistic in asking for too much stuff too early.
And so it would be great if they had a complete package of here's what's going on, the fact is that they were up really late the night before trying to get this stuff done, they've also had to submit beta tests and so we decided that the final write up of how their project worked, et cetera, we would save for the final, and it won't be something, in English written like that will only be in the code, it won't be as a separate package
Right
Any other questions about that?
OK, so one final word, relationships with students.
So one of the things that MIT has, and many other places have, I'm sure that you're aware, that if you're in a power position with somebody, and this in particular applies at MIT with students, you must ensure that all your relationships are strictly educational.
And what that means is like a, no romance, OK, duh.
But it also means in this context, no recruiting, hints at job opportunities, offers of summer internships, lab tours, et cetera, free dinners, and so forth.
Your job has to be completely educational.
However when the term is over you're welcome to continue your friendship in a non-power relationship.
There's no longer a power relationship you're welcome to start a romance with one of the students if you will, but in particular, allowed to seduce them to come to your company as a summer intern, or a job or what have you.
So you're welcome to do that but not during the term, those sorts of things are absolutely off bounds.
Now the students themselves may say, "Hey do you guys have summer internships, and so forth?"
And it's up to you folks to draw the line and say something like, "Yes, we do, but if you're interested in that, we can talk about that after the term is over," for example.
So, put them off in some way, and return the conversation to the educational mission for the term.
Yeah?
So what if the students do a good job?
Yes, but then you run the risk of it being interpreted as a quid pro quo for employment, which is exactly the thing that we're trying to steer away from.
They're smart MIT students, I think they'll figure out that if they're interested in a summer internship they should be impressing their mentor.
Not all of them figure this out, by the way
So just to clarify.
Just to clarify, it's okay to say, "If you write your code this way, I won't hire you."
But it's not OK to say, "I wouldn't hire you."
I just want to clarify
You'd say I wouldn't hire you, OK, my company would not hire somebody who wrote their code like this.
You could say that
But it's not okay to say my company looks for people who would write code this way?
No, I think that would be OK too.
No, the issue is whether you personalize it, right?
If you're interested in a summer job, I can get you a summer job, all you have to do is, dadadadadadadaaa, and now they feel like there's another master to serve.
So what we want to avoid is having them feel like there's another obligation on top of the obligation they have to the class.
They should not feel that in some ways they're obliged to do something for you in order, for example, to get a job
Or get a good grade
Or get a good grade, or--
If I say no to the summer job they're going to badmouth me and I will fail this class or get a bad grade, and then.
So, that put them in a binding situation that they won't work for the class, but they'll feel obliged to something else
You're in a power situation, and you're in a situation where you must not let them have any sense that there's any power relationship there, if you will, in other words--
We shouldn't try to extort meals from them
For example.
Exactly, exactly.
Also if you feel anything awkward is developing, let Saman, or me, this is something for which the staff only email is better than probably the MIT POSSE one
Or even to us directly
Or even to us directly, although I actually think it better to go via the staff only.
It really is much better to go get via staff only, because the TAs need to know what's going on with all the students in class as well.
Question back here
Two logistical questions but I don't want to interrupt you before you finish
No, that's fine
Okay, it's not clear to me do we meet with the same four students throughout the term?
If possible we will do that.
Because otherwise what happens is, they hear it from one and then they go to another.
So this way there's some amount of continuity, but that won't happen perfectly because students will drop and we'll have to reconstitute groups and things
The winners get David and the losers get me.
That's OK.
I just wanted to be clear.
Also you e-mail the code to us before we do the sit down session?
Yes
Are we expected to review it by ourselves beforehand?
I think it helps to take a look at it, yes
As opposed to walk in cold and doing just that
Yeah, yes, that's right, I mean I think you do a better job if you have looked at the code beforehand to see, oh my goodness this is, and take a few notes.
But for this we're relying on you to use your best judgment for what's the best way of giving feedback to the students.
But I think generally the reason we email it to you in advance is specifically so you have a chance to look it over.
Once again we're not so concerned about whether the content of what they're learning in it as much as-- you look at it, it will take you two seconds as soon as you see one piece of undergraduate code, it'll take you two seconds to say, I don't care what this thing does this student needs to learn X. I mean you will see.
Is that the feedback, from people who took this--
You'll know it when you see it
Any other comments from people from last year what--
Yeah, a couple comments.
One very minor sort of procedural things.
Personally, I found it a lot more useful to me to show up in person for the information sessions than do it by conference call, if for no other reason then a lot of people who called in had issues with voice quality.
You know, just getting a good quality voice signal over the phone.
Also I found it very helpful to have each student review the other one's code.
So I would just do that to start with, just hand it to each other and here, what do you think?
That also gives them some experience of actually doing a review, as well as being the person who's work is being reviewed
That's a great technique just because one of the things that you want them to do is empathize with the person who's going to read the code, and so, by putting them in the situation where they have to read somebody else's code they very quickly learn oh, this is, my goodness that's exactly what I did to the other one, so that is a great technique
The reference implementations are we going to get those--
Yes
Soon?
Early?
Yes, those generally we will be able to give you the reference implementations at the information session.
Or actually, we should probably get it to them before the information session
I would suggest before the--
Yeah, let's get those make sure we get those to you before the information session.
We'll get you a reference implementation at least a few days in advance
OK, also something that was a recurring issue last year regarding the reference supplementations which was the... there was some issues with some of the reference implementation-- [INTERPOSING VOICES]
--software engineering
In particular things like functions that should have been local to a file they were not declared static or, that kind of thing, or macros that should have been used or that were used and shouldn't have been
Last year we were running with a much larger class than we expected, with no infrastructure to speak of, and we had two TAs who were flat out, and so we definitely cut corners.
This year the department has been good to us, we have four TAs, and so I think that the quality is going to be higher.
That doesn't mean that the TAs are seasoned 20 years of software development experience or more, either.
So they have some things to learn too, but already based on the first project we're ahead of the game
OK, also if from time to time one of us wants to attend the class?
Feel free to attend the class
OK, where's the schedule?
It is Tuesdays and Thursdays in 26-100 at--
Is it on the website?
It's on the website at 2:30 to 4:00.
So Tuesdays and Thursdays 2:30 to 4:00, you're welcome to come and learn all about performance engineering.
Reed, did you have a question?
No, I guess I was just going to say that in regard to the feedback, we TAs here have got that
During class last week most of the TAs were students last year, so they aren't learning anything new
I'm sorry, where is the class?
It's in 26-100
So it's not here?
No.
So this is the situation, we outgrew our classroom which seated 60, and we had 85 or 90 students, so they put us into the next smaller classroom that we would fit into.
That was 26-100 which fits 500.
Then we found that in 26-100 you needed a microphone for the students to hear you, even though we had them all crowded into one quarter of the auditorium, you still then-- just the structure of it-- you needed a microphone, so they said, "That will be $85 a lecture."
And you say, "What do you mean $85 a lecture?"
Someone has to walk in just to turn on--
To turn on thing thing.
So, anyway we sorted all this out OK, but it was sort of funny that you're charging us for being in a room that we have no choice as to be in?
Anyway it was very funny
Are your lectures video taped are they available--
They are videotaped.
We are not planning to put them up quick because what that does is discourages students from attending the lectures, as you can imagine
Mainly for OCW OpenCourseWare So they basically they could go to OpenCourseWare they will process it and they have to go through a lot of legal stuff and make sure the named are right
Are right and so forth, yeah.
There is a bunch of stuff there.
Good, so it says final words here but they aren't really final, because we still have Eric to talk about the first project.
But the feedback from the students last year was that the MIT POSSE was one thing that made this one of their favorite classes.
It was getting that kind of seasoned feedback in a very small group setting when you look at the student faculty ratio, that's one thing you're always battling.
especially at MIT, where we are in a very popular major, and so the ability to recruit all you folks to help that ratio in favor of the students is really very much appreciated by students and by us, so enjoy the term.
This is really rewarding the people who have done it before we had just tremendously positive feedback from them about just how rewarding it was to work with these young programmers
They are very smart people, you can see the changes, you will see the you impact on them.
You will see you're molding these people into better programmers, so that's the fun part
So Eric is now going to talk a little bit about project one, and this is similar to what we will do, although most of you will probably be on the phone unless you heed Barry's advice, for the future one.
So I'll get out of your way
So the project one this is, I think, the one project that was substantially different from last year.
So, the code for this project we, the TAs, coded up this semester.
Did we give out the hand outs, Saman, or Charles?
Yeah so these are the--
So this is the actual assignment and we'll share this with you, plus the reference implementation, I think
Well we'll send out the reference implementation tonight then.
In fact, now people have already done their betas for project one.
Project one has two parts, it's a part called Every Bit Counts and a part called Tiling a Torus with Pentominoes.
So the first part it's an abstract data types of modeling and bit array, packed bit array and they are a given a couple of tasks to do on long strings of bits.
So you might have eight million bits and there are two functions to rotate the substring of the bits by some amount to the left or right, and the other operation is to count how many flipped bits there are in a given substring.
So the specific interface for this, this an excerpt without the comments from the header file of this C module.
There is a couple of accessors which accesses bits individually those are not, well those accessors are currently used to implement, in the reference implementation, the rotate and count flips operations.
So what they will need to do is to go into the C module, the implementation, and change these functions, to basically be 10,000 times faster, or whatever, I can't remember what they actually got.
In particular the rotate operation is incredibly slow, there's one constraint for them and we're saying that, well imagine that this is on the cell phone or something, where you have a really big bit array, but you don't really have very much more memory to consume beyond that big bit array.
So we're asking them to only do this using a constant amount of memory.
So that's the first problem with these two parts, the rotate and the count flips.
Any problem, questions about that particular part?
Does it seem clear what?
What does the count flips do?
Oh yeah, so I should've swap those two slides.
So, if you have a bit string, suppose nine bits or something, 1, 2, 3, 4, 5, 6, 7, 8, that would have to be 8.
If this is the substring you want to count, then you count transitions from zero to one, or one to zero.
And there's some new tricks you can do by for instance, considering six--
Count the number of runs, of consecutive value
Yeah, the number of runs
Plus plus one-- or minus one, yeah
That's right.
And you don't count the beginning or the end as a transition, so in this particular substring there is one there, one there, and one there.
And it's not a substrings, it's always substrings, it's not the entire string, and you can see that for instance, you give us an argument, the bit array and you say where the starting offset and the number of bits that the substring that the substring is and that's where the operation is done.
So our testing routines will do lots of rotations in different parts of that substring
For the memory constraint, do you have a fixed size that they're allowed to use or?
We decided against the fixed size.
We just say don't use malloc, and you can allocate constant amounts of memory, and the thing is if they choose to have a 16 megabyte or four gigabyte lookup table then their performance will suffer anyway.
So yep?
What is the computer architecture?
What's that?
What is the computer architecture?
We are running on-- we got this donation from both Intel and Dell, we have 16 machine-segments, each with two six core processors and 48 gigs of memory.
So they're doing it on 12 cores so it becomes interesting, and you got a parallel 740 gigs, so we have a lot of memory in these kinds of things and we'll be asking them to run through that.
So these were the latest what's the word?
Core i7
Core i7s
Nehalem
For this first project, we're asking them specifically not to parallelize
It's all on Windows or Linux?
Linux
Linux?
Yeah.
So the idea here is to exploit bit operation and in particular in rotates.
The reference implementation does the really stupid thing of you want sort of 4,000 bits to the right rotates, well I'll do it one at a time, that's the reference implementation.
But they're only allowed to us a constant amount of memory, so there is certain amount of tricks you have to do to do it in a more efficient way
Do you stress in the course about portable data code from one operating system to another, one machine to the other
This is where we want you to stress the importance of it, because that's not something that we're in a position to do or enforce, or what have you, OK?
We let them assume for this problem, for example, that the machine they're working on is Little Endian.
We talked about the issue, but we did not insist that they wrote code that would be that would port to a Big Endian machine, because they're running on the Intel machines.
But this is exactly the kind of thing where code quality would say, portability, maintainability, readability, all these kinds of things, this is exactly the kind of thing we want you to say.
You say "Hey, you did it like this, in fact did you know that without sacrificing anything in performance, you could have made it portable from one operating system to another?"
And then you could talk about those issues, and that's absolutely super, to do that.
But we are not spending a lot of class time doing that, we're really focusing on the performance engineering part of the course
The reference implementation should be portable, so this should be portable, although we did make sure here, for instance, in the reference representation of bit array here, that the order of the bits within each bite happened to coincide with the low end ideas of these machines.
So that if you happen to cast it to a 64-bit integer on each of the bytes in the array then you would get something that was easy to work with on a low end in architecture.
But the reference implementation is portable itself, or should be, so tell us any kind of problems with the reference implementation like that.
The next problem is based on the puzzle, it's a puzzle solver.
It's this puzzle that I'd never heard of before.
Apparently, if you generalize dominoes you get Pentominoes and the five end cases.
So there are this many configurations of dominoes with five squares on them, I guess.
And the pentominoes puzzle is you're given an eight by eight board with four squares crossed out, and now you're supposed to arrange these so that the entire board is filled up, and only-- what's that?
You can do flips of these--
Oh, right.
You can do flips, you rotate
And they will dissipate and all those things
Right so the task is find all the solutions, although I believe in our reference implementation that was too fast, so we decided to generalize the puzzle to the toroidal case, where the board wraps around the edges, just to make it take a little longer, to do
Like space boards right?
You go off the right edge you come in the left edge
Top and bottom also?
Top and bottom also, yeah
So these two are both examples of solutions to this puzzle
I've got a question here
Yep
Are you allowed to put the empty space where ever you want?
Well, that's the input to the solver
Given these empty spaces--
OK, and then it fills in the rest
That's right.
Of course you can put empty spaces so that it can't solved.
And then they have to correctly determine that there's no answer
That there's no answer
Right
And, also I believe the reference solution takes days, for certain.
To find all the solutions of certain configurations of the initial four dots.
So there's an option to the program we give them, to just find the first 2,000 or something
So once again you can see this is conducive to bit tricks, where you represent the board as a 64-bit word and then do masking operations, and so forth, to determine whether a piece fits and do clever shifting.
It's easy to shift a piece down one row, where you're trying it out, because that's just a shift by eight, but how do you shift it horizontally, to come around on the other-- So they have to do some shifting, some masking-- So anyway, the first project is all about thinking about bit representations of things, and so forth
In fact, we give them a solver that we don't really-- well, a searcher, routine to search the space of possible solutions, that we don't really expect them to change, but they can change it if they want to do.
So, the interface here is basically a board data structure that they're free to change the internal representations of.
Or did we somehow prohibit that due to testing?
No?
No, no, no, that's the whole point, right.
So again, you have this basic accessor for use by the testing routines.
But then what you really need to, well, you need to change basically the entire implementation of all of this, if you're changing the representation of the board, for instance.
But this is the function that should be running faster after you're done, so I guess that comment is misleading.
This is the function that is supposed to run a lot faster when you're done.
There is actual comments in the header file.
So right, you give it a function pointer to a call back
Now, one sort of standard thing we decided we're doing is, we're not trying to write both 32 and 64-bit code in the classroom, we're just going to do 64-bit.
So just one more level.
In practice you would want to have stuff that runs both on 32 and 64, but for the most part it's fine if they just have it running on 64-bit
Right, so these by the way are the initial four points that are the input to the puzzle.
I believe that's the problem set one.
Questions about, yeah?
Can students submit more than one answer?
No.
Nope, they got to decide what they're doing
I guess I have a question about the answer
Yeah
Is an answer that is probabilistically correct, I mean like, it will sometimes when you test it, you have it pass all the test cases, but mine isn't guaranteed to necessarily always solve all test cases is that OK?
If they can conceal that to us, then I think they--
They deserve credit
Keep in mind that the other half of the problem set is to write test cases that will catch as many obscure bugs in the other students' code as possible.
So they run the risk of competing against--
They lose extra points if they don't pass their own tests
They each need to write some tests that will be as--
Yeah, so then they--
They exercise somebody else's
--to somebody else's, yeah
We are running all the tests against everybody else
And then for the final version we give them the full regression suite of the whole class, so they have very good confidence, and we good confidence that they're handing in something that is correct
OK, so you want to open up for--
For questions or comments
Especially, I won't even mind the people who did it last time, last time to provide some of your feedback on how we can... [BUZZING NOISE]
Don't want to do that?
Yeah,
I was trying to turn on the lights.
Do we have more lights we can turn on?
Also, if you could turn these on, too?
Good.
Yeah, so comments?
I want to say that's why I was a master last year, and I had a great time, but I figured I'd stay-- [INTERPOSING VOICES]
Yeah, could you stand up so people can hear you?
Thanks
Give him a mic.
Give him a mic because I think it'll be good--
It'll be good to have these-- yeah.
All right?
All right.
Thank you.
So the first thing I found really helpful was to take the reference solution and actually diff what the student provide because they often just provided a solution, and if you don't know what they started from, it was hard to tell what they'd actually done.
So diffing the two, that was a good place to start.
And then I found incredibly helpful just to spend 15 or 20 minutes prior to actually meeting with them, looking at the code and writing like two or three or four things to talk to them about.
And it might sound like it's going to be highfalutin and stuff, but it was like they're using their returning function pointers to local variables, stuff like that.
It was really like what the hell the makefiles were doing, for example, is something that I think they really appreciated me telling them that didn't get talked about in class.
Especially once I got a good rapport with them, they just ask questions, and I spent the whole time answering more practical stuff about the problem.
We had a good discussion, so that's what I would suggest
Thank you, that's great feedback.
Yeah, as I say, don't underestimate-- I mean, most people learn the kinds of things you're saying, but somebody told them that, hey, these are practices that really don't make sense in a real software system
People were, by the time we got to the projects, your standard pattern was highly representative, a bunch of flags you want to encode in a bunch of fields, right?
And you've seen them a couple times, it's just a standard pattern
Yeah, so one of the things you'll find in this first one is that they're very confused about the difference between unsigned and signed, and how shifts work, and what happens when, you know, how come I shifted this thing left 33 bits and I got a zero.
Why?
I thought I was shifting a 64-bit value, but in fact, they hadn't done 1LL, they just did 1.
And so there are things like that, where they don't-- very low hanging fruit, if you will, to talk about
Did you tell them about valgrind by the way?
Like, why would programs crash when they said they spent a huge amount of time debugging these crazy C programs when they're not used to the compiler not protecting them
Actually, that's a good point because we were going to try to introduce-- maybe we should do that as part of the next lecture, spend a little bit of time on valgrind
Yeah
Because the memory checker there is really a pretty useful tool for them to get correct code
I second that.
That thing has saved my butt so many times
So many times, exactly
I've heard that, but more low-hanging fruit for valgrind is to always compile a -w all to just cache all the warning and declare all warnings as errors, and it just makes--
I think that we have that as a default in our makefile.
Is that right?
At least for the first project
The first project?
I think they should understand that in industry practice, if you don't do that, you're a schmuck
Right.
Yep.
Good, so that's good, and why, in particular, the valgrind-- we discussed that last year, and really actually, the upcoming lecture's exactly when we should do that
Because there's also another tool called Dr. Memory
OK, which makes sense, yeah
One issue that I experienced a little last year was that people they specifically intend to write a little test for their code.
They've mostly relied on a few runs by hand, then they wrote a progression producing things that they ran earlier.
I guess that now you're actually encouraging them to actually have a test suite
Yeah, well, now they get points off if they didn't write enough test that somebody else's code passed it that had a bug
And in this case the testing run will be as far as the first turn in, and then the second turn in is just you have a version to run against and then just need to optimize
Right, that's right
So first, about the actual tests they have write a lot of tests because that will help, and the second, then you give them all the tests.
So they will have a lot of test suite to run and then of course we have a lot of number of points we'll deduct of you actually fail those test.
Because they have no excuse
There's no excuse at that point, it's just a question of having--
What performance and measurement tools are you going to be using this year?
So, right now, we are going to give them perf because what we found was, the process of ECUT is so new a lot of performance tools don't have-- like OperaFile didn't work, because they haven't been ported properly.
There's no port for it yet.
So perf we are going to use, and we are thinking of giving them gprof in there.
And then last year gave them couple Pintools we didn't ask them to write anything but for example, account number instruction or couple of tools they do that.
Those are main tools you get, and also we need to go to Cilk, you'll do a bunch of Cilk tools in there
Everyone last year, they were given VTune, I think it was, it was too hard for them to use
Yeah, that was too hard for them to use because they have to use the machines remotely, and VTune and is an interactive tool, and so that just made it very complicated to do things.
So instead we're going to go with a something that has textual output, and is easy, therefore, to run from command line and has all the advantages of running things that you get when you run things from command line
So a logistical question so, meeting on MIT is fine, but where are there designated places to meet?
We will work with student to find places for them.
For example, there are locations for PSS students to meet, and we are going to get them arranged.
So if in the organizing you guys can create probably an online scheduling something.
What's that website?
Online is, online scheduling is--
Doodle
Doodle, Doodle or something like that, and get their times and then they'll work with us to figure out the place to meet, and then they can send it out.
One thing we will have to do is, between four students, because every project we are chaining the pairings, so sometimes we might end up in a way that, two of your students are paired together, and then we will tell you don't put them together in the same group, you'll basically have to mix them in the other way.
If they are not in any kind of a pairing, then you can organize the four in any way you want
Actually I don't even want of the four of them, I don't even want us pairing.
That shouldn't be.
But we'll see what happens.
Barry?
Last year of course was in classroom downstairs and I just met them just outside the classroom.
There's plenty of little space
Also the first floor of STATA, I mean we've had a lot of people just using the first floor of this building.
There's lots of cafe and place there, and so forth, to sit there, get a coffee, and so forth.
So some people just did something that informally, other people wanted a more quiet private place to talk.
And I think that we'll help the students facilitate that
And the key thing is, especially if any students are flaking out, for example, doesn't answer your email, or missed the meeting, or something like that, or late to the meeting, let us know.
Because that's part of their grade, even though you're not grading them, if they are not participating in working with you guys, they will get points deducted.
So we really want to know that, and we have tried to emphasize that they have to act, and actually do it right.
So we really need to know that, if the students are not actually cooperating with you guys
So I hope that this year will have a better-- So the vast majority of people last year had no trouble, but we did have a couple of people for whom it was one or two students who just made life a little bit miserable for their master, and so hopefully we'll provide a few more incentives this year to avoid that situation.
But they're students and so they--
When do we find out which students are assigned to us?
Probably the next two days.
OK, and then who's responsible for contacting whom?
That's a good question.
Who should we contact?
We decided that, didn't we, at the meeting?
What did we decide?
This is what happens when you don't write things down
You have to cover the old stuff
Let's see, so last time I think we had it that the students contacted the mentor, and then--
What we will do is we will set up, we'll send the intro email to everybody, we'll send the intro email to the mentor, Cc the student then saying I am introducing you to each other.
And then one thing you can do is immediately reply and, if you have Peter Doodle Paul, or something like that.
And then say, OK, look, do you know your students?
So you can reply to that and hopefully they will reply back, and proceed.
So when we send you the names you will also see the emails of students
Are people familiar with doodle?
Anybody not familiar with doodle?
So doodle is a website where you can basically enter a whole bunch of times, and then people enter their names and mark off which times they can make things.
It's a poll source
Don't do what I did the last time, I marked off all the times that I cannot participate, so I was basically the exact opposite
It's green and red, is what the colors they use.
So green means you can make it, and red means you can't.
So, anyway you don't have to use that facility, but it is a convenient one to use
Can we expect students to be dropping?
There will be people dropping.
They just started.
This is also a group project and at 2:00 AM we got mails saying OK, I'm dropping the class, and we had to scramble trying to add a 3:00 AM to create another group, and so, Yeah
You shouldn't take dropping as a reflection on our end.
People drop out
You have to know that
You can if you want
I had like three people drop
No, that tends to be more--
I had one student, it was awesome.
So easy
Yeah
To get an idea of the mindset the students are going to be in this sort of a really tough 6-170 type thing, or is this one of the easier classes
No this is a challenging class
So you take 6-170 that has about four projects.
We doubled the number of turn-ins because each project has beta and a final, and some projects has two turn-ins in there.
So basically, they have all their big projects going on, because they are doing the new project and they have submitted final on the previous project.
And they're already completed project zero that was--
Just like real life huh?
We gave them project zero.
That was built to to get used to the system and we can help them with the class, they have to finish this one.
So this class just screams excess very, very--
Yeah, and of course they're students, some of them have learned you put off things to the end, and haven't learned the lessons of well not always.
So as I said, as a group they're relatively immature when it comes to some of these things, but there's a large fraction of them who are incredibly mature about these kinds of things.
So our job is as lecturers in the class is to make it so that the students who start work early don't get penalized.
And that's probably the hardest thing in the software related class because, the classic thing is that they want to get started, they find all the bugs right?
And they waste their time doing it, and if you do it later, then the TAs have worked all the bugs out.
So one of our challenges is to make it so that students who start early are rewarded by being able to have more time not wasting their time
And today was the day that they turn in the project and by looking at the class, you realize they haven't gone to sleep, and I asked and most people haven't slept in about 30 hours.
They have been up entire last night and an email conversation when I went to sleep there was an email train going.
Every five minutes, something pop up.
When I woke up, the train was still going, and amazingly, there were a couple of TAs up and answering all through the night, and that kept continuing.
So these kids they can be flaky sometimes, but they're amazingly hard working.
And this is only one class they're taking.
They might be taking four classes like that.
And I have seen people scheduling their schedule as by hour, scheduling and they say, OK, look I can do the problems set or I have one hour of sleep.
And a lot of times that's their thing, and then sometimes they triage.
That's why sometimes they miss classes, they realize, OK, I can do the problem set, I can sleep, I came do that.
So I think these kids work amazingly hard
I do think it's helpful to also counsel them on, hey, you can't do everything on a death march.
You got to learn at some point what you're capable of, and manage your own time resources and so forth.
Because the joke at MIT is, classes, friends, sleep, pick two.
The other thing, let me just say, is that despite the obvious sleep deprivation we saw in the class today the number of students who came up after class so charged about working on this first project was really quite amazing.
They were all very, very excited.
And very excited we also gave them the initial feedback on the performance, and you could see, there's one group of students who clearly got it, and then there's another tier of students who didn't get it quite this much, and then there's the ones who their code has errors and stuff.
There's no doubt that there are a lot of students who are very charged about this.
This is a fun class but it's a hard class
I think you'll find this is true.
They will be, probably tired, but they will be soaking up this information like a sieve, and sometimes I found last year's-- After the looking at the first set of code, some of those the masters were really upset.
How can you teach you guys, these guys write this kind of bad code, but when they realized after they said something, this is getting into their heads.
I mean, they're improving they're actually changing.
These guys are very smart, and they're looking for you guys for-- They're basically sponge, really empty sponge.
They're going absorb a lot in this class
I have a 2 and 1/2 year old at home, and it's amazing what the progress is she's made in six months.
She talking now in complete sentences, and so forth, it's just really amazing how how she does it, and these students are like that.
They start out and you say, oh my God, they have no skills whatsoever.
By the end of the term the progress that they've made is really astounding.
You just say, gee if you could just keep on that trajectory your whole life.
But really, they really are very smart, and they come up to speed very, very fast, especially given good counseling.
The main thing that they need right now is just the attention so somebody actually can look at their code and say, you know you did it like this it would be better code.
Any other questions or issues?
This is lots of fun.
If you run into any troubles, we're right here, just send us email.
And as I say they're young and immature, but then you get to see them grow, of course some of them will flake out on you, but the vast majority, you'll see, are really very dedicated and well meaning.
They really want to do a good job in the class, and really, really find the class very exciting.
Our numbers, in fact, almost doubled for last year.
So the reputation got around that this was a good class to be doing.
And the other thing is, right now what I really like about the class, the students right now compared with say 10 years ago.
So 10 years ago we had the Internet boom, and there were all kinds of students taking computer science because they thought it was a ticket to fortune and so forth.
And now we're back to what it was pre-2000, 1998, whatever.
Where the vast majority of students who are taking the class really enjoy engineering.
They really enjoy the software.
They really enjoy the engineering.
The fact that you end up with a pretty good job and opportunity sometimes to do entrepreneurship, and opportunity to explore things, it's sort of like, oh that's a fringe benefit.
But what they're really after is just the intellectual excitement and so forth.
So it's really quite fun to work with the students and quite rewarding.
Really very nice type of class to have right now, where the students taking it because they're actually interested in the material, not because it buys them a bread ticket, meal ticket, whatever.
Yeah, question?
The standard, is the coding standard in the reference, the standard is to consider for them provided?
I don't think we have set up a standard, per se.
What we want them to do is kind of create their own style.
It's like in a writing class, you don't give them a standard writing guideline is to follow this one, you don't constrain them to that.
We want to give them the opportunity to grow into their style, but kind of guide them, so that they don't make the classic mistakes.
That's why we need you guys, I mean, if there's a standard, we can come up with a standard, create a script to actually check the standard, and then say, OK, thou shall follow the standard.
And you can do that and everybody will hate it.
You can't make that bad happen.
One thing you want to do is make them lifelong good coders, not something they realize OK, I need to do this to get this grade, and I'm going to abandon everything and convince myself never to that again.
And we have had sometimes, like when we needed all this software engineering.
We went through things like thou shall write all the preconditions and postconditions, and everything will be graded.
And people did it, but half of them got into them and said this is good things.
Other half, they didn't realize why they're doing it, they just hated it.
And I don't think they left assuming I would never do this again.
So the key thing here is, we don't want force something on to them.
We want them to come through a natural process, and I think a lot of them, can, when they understand what they're doing, they'll develop a really good kind of practice
It's also the case, I think, that coding standards vary culturally from place to place, and the what people expect in one place is different from what thing.
So being too prescriptive means, as Saman is saying, that it becomes a grammatical thing, rather than is this code organized.
The real thing, coding standards are very important, especially in large software organizations of course, so that everybody can tell what's going on in any particular piece of code without having been the author.
But in this kind of thing where we have something that's relatively small, what we're mostly interested in is the organizational aspects of coding style, as opposed to the grammatical ones
So also don't impose your standard and say, my company this is what we do, I want to follow that.
Just figure out and kind of guide them towards something while they're maturing, and try to make it better.
That's the thing.
That's why you cannot have a cookie-cutter model
And why it's good to have senior software engineers who understand what's really important, as opposed to what might be required in any particular individual situation
One other topic I want to add is sometimes when you have two students, there might be a case that one person might be very talkative and dominating, and one person might be just a listener, and so you have to keep an eye on that and make sure that you give the other person opportunities to talk, and because it might be just like that.
Encouragement and stuff like that, so make sure that you get both of them to talk.
And also the are supposed to be responsible for all the code.
You should not let them get away by saying, oh my partner wrote it, so I don't understand.
So that's not a good answer.
So get them to actually go through that, and if they don't come prepared tell them that, and you can give them some feedback, as a next time can you please do, this and this.
Because they also don't know what to expect, and we give them a little bit of guideline, but it'll be good, also for you, the first time around, if you find some mistakes also for the next code review you can encourage them what to do, how to prepare and what to do.
Don't give them too much work, because we're already giving them enough work.
Don't ask them to write extra documentation, but you can say it would have been nice if you had this level of preparation, if you could have explained to me, or given me a summary, or something like that.
So something like that is something you can expect.
And the other thing is if you had to figure out how we want to do code review.
So, for example, you might want to bring your laptop, or ask them to bring a laptop in there, if you want each other to look at the code on the computer, so ask them to bring the laptops or something.
So think about the logistics of how you're going to do that, and that'll be useful
And I think any other sort of expectations you have, in your email to them the first time say, here's what I'd like you.
I think it's helpful to say I'd like you to come prepared to give me a five minute update on how you did each of the problems, for example.
We will be telling them that, but to have it come directly from you means-- The difference if I say go do this to that person, versus you're coming to me and expecting it.
Good.
So, I want to once again, I can't thank you folks enough for being willing to give your time.
Time is the stuff life is made out of, so I really appreciate your willingness to volunteer your time for these students, and I really think you have a dramatically meaningful impact on the students.
The students may or may not fully appreciate that, OK, but Saman and I really appreciate that, and we will make sure the students appreciate it
Yeah.
I just have one.
What what are the guidelines for a contact outside of the code reviews with the students.
I mean are they, can we have them email us with questions about the process and just kind of general guidance
It's your level of tolerance we don't want you to have to commit more than about 20 hours across the term, because then it becomes onerous.
But if you're willing I think it's completely open ended, in terms of how much else.
I mean it's one thing if you go into solving their problems for them, that we don't want you doing, but when it comes to, especially the quality, and non-metrical aspects of programming, the more you can give them the better, OK?
Because it's only making them be better programmers.
And I think the fact that there may be some unevenness in that across the masters is irrelevant.
Really, I do not want to hold back some students from getting good mentorship because somebody else is not getting the same level of attention, et cetera.
But really, no expectation there, on our part, to do more than the code reviews themselves
So, that said I want to make sure that you aren't answering their question, at some point I don't want to be a crutch for them.
For example, if they say, OK, before prebeta, they say OK, this is my next problem, can you on a little bit, I didn't understand.
If you have that, so they're trying to get too much into that level, kind of making you their programmer, or something like that, just let us know.
A lot of these things, there will be a lot of cases that arise that we haven't thought through.
The students are ingenious enough they will find a lot of loopholes, a lot of kind of interesting things that we never thought through.
So if you find something that you're not feeling comfortable just send a mail and say OK, take a look
It really is, if their questions are about course content it's not appropriate for you folks to be dealing with, really.
OK, that's what our TAs are for, and course staff.
If it's about coding style and engineering practice, and even time management, or anything that's sort of the non-technical aspects, but things that have a strong impact on technical, that yes.
Anything you can provide there is great
So if you find anything that's outside the norm of what we discussed if you feel a little bit uncomfortable, just send email.
Send an email, and say, hey, can I do this?
Is it OK, for me to do this?
And then we can always provide guidance, so we are responsible
--do something and say after the fact, oh, it went past, let us know.
There's rarely anything you can do on one shot basis that is heinous or unrecoverable from, or whatever, so
It is always good, I mean that's why-- since we see everybody able to and we have done these kind of situation before.
We can just easily tell you what
But, mentor them, for sure, mentor them
I think a lot of times, when you talk to students last time they got a lot more information out from the masters, other than just the coding style.
I mean, they talk about what is it like to be an engineer in a company, what are the things?
And so they get a good feel.
I mean, some of them might not have done a internship in a company.
So they don't know.
I mean, they're just green, and talking to you guys they get a feel for what it's like.
And also from a non-marketing situation too.
I mean you might have ulterior motives to market your company to them, and stuff, but this is not that they're going into a job interview.
So, for a lot of them it's rare.
I mean they might not have retinues of friends that they can go and talk at that kind of casual level.
Most of the time they're on guard because they're on an interview or they assume that information coming to them will be sugar-coated, and here is a way that they can get-- I think it's very good for them
Eight minutes over time
OK, I think we should
So, any last final comments?
Saman and I will hang around for a few minutes, for anybody wants to chat more one one.
And one and once again I thank you for all the time that you're planning to put into the 6-172, thanks
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Welcome.
This is great.
It almost looks like the beginning of the Fall, at least outside.
So, what we are going to do, is it's going to be a very exciting month.
And mainly the reason for doing this course, is I think we are about a software crisis.
So there's an interesting quote I want to start with.
This is somebody very famous in computer science, who spoke during his student award lecture.
His point was, when the computers were small, there's no problem.
When computers got bigger, this is basically when software started having problems.
So, the first software crisis, in fact, happened around the '60s and '70s.
The problem for the first software crisis was people are writing Assembly language program.
When the computers are small, and memory was small, it was completely doable.
No problem.
But as computers became larger and larger, it became very hard to write Assembly language programs.
And what they needed was some kind of abstraction portability.
So you should be able to go every two years, to not have to rewrite the programs in there.
So you need portability and you need some level of abstraction.
So, what that means is instead of writing all Assembly hacking and dealing with every hardware feature, you can erase about that.
And the way that we solved it, in that generation is, we, came up with these high level languages like Fortran and C. And what those did was, they kind of provided a common machine language across these machines.
And so what these did was, they looked at all machines those days, the uniprocessors, and kind of abstracted out: First of all, all the common properties made available for the programmer.
So it didn't really lead the programmer too far away from the machine.
But it abstracted out all the differences.
So by doing that, we avoided, we managed to get these languagese like C and Fortran.
Weren't very fast.
Pretty efficient.
And working through this.
So that's how we avoided the first software crisis.
And I think that a lot of really good work happened, and a few people got Turing Awards for doing these kind of things.
And then, we faced a second software crisis in the '80s and '90s.
The problem there was, C was great if two guys sit in a corner and write a program.
When Microsoft started programming Word, Word suite, with a few hundred programmers, the problem was the bugs never went away.
The project was supposed to finish in a year.
It lasted three years.
And bugs kept coming and coming.
And they had no idea when the project was going to finish.
Because the way you write in a small piece of code really didn't scale.
And, this was a huge crisis for people, who said we will never be able write large programs.
Because these programs will never work.
They're going to be brittle.
And the problem there is we want to be able to build software that's composable, malleable, and maintainable.
So software should be: Multiple people should be able to do it.
Compose it.
And the requirements say it should be easy to change and easy to maintain.
The interesting thing is, we never cared about high performance at that time.
Moore's law.
They started to take care of that.
The way we solved that crisis -- Again, a lot of people got very famous for doing that, is came up with things like object-oriented programming languages.
C#, C++, and now C# and Java, basically made it possible for, now, hundreds if not thousands of people to develop programs.
If you look at something like new Microsoft products coming out, thousands of people are involved.
And still they manage to get it out on time.
And, also we did a bunch of better tools.
Things like component libraries, and tools like Purify.
And we developed a lot of software engineering methodology to develop software.
We made it more of a process than an ad hoc art-form.
This is great.
So here we are today.
We are very happy.
A lot of programmers, yes.
Unfortunately, programmers are pretty oblivious to what's happening in hardware.
We don't care.
Programs don't have to know about what things are running.
You have these nice virtual machines.
And, of course, Moore's law, this great new thing is going to come and take care of all us.
If you want speed, just wait six months, the machines will be faster.
And that's the thing we have been going on.
The nice thing is, these programs, programs written even in the '70s, still run.
Probably two, three, orders of magnitude faster, because processors today have gotten really fast.
And this abstraction gave us huge amounts of power.
Just say, don't care what's underneath.
We are going to sit up there, and just basically program.
We were very happy with that.
The problem is, we are running into an issue here.
So, we are at the beginning of this third software crisis.
What I call the software crisis due to the multicore menace.
So, multicore, for hardware people, it's a good opportunity.
But for software people, its kind of a menace in there.
Problem is, sequential programs are left behind by Moore's law.
And, as you see, as probably a lot of you know, that you're not going to get performance on sequential programs, increased performance, without dealing with this And -- the key thing here is, so why don't we just stop?
OK. Good.
We have the programs.
Machines today run very fast.
Isn't that good enough?
The problem is, we want it all.
We need to support new features.
We need to support things like large datasets in the same programs.
And we kind of keep expecting a certain performance gain every year.
To sustain portability and malleability, we assume that every few years, the machines' performance will double.
Because we believe in that.
For example, Microsoft Word at the beginning, ten years ago, it still works.
But it didn't have Spellcheck running in the background.
We didn't deal with very large images.
We didn't deal with XML formats in there.
And, then of course, next generation, we probably want to have integrated video, audio, voice recognition, all those things in there.
And, we need that power.
We rely on that power.
And this is going to be a big problem, because sequential programs are stuck.
They're not going to provide that power anymore.
And it's critical.
This kind of power is critical to keep up the current rate of evolution.
And, if we get stuck there, we are going to have a big crisis happening.
So, how is this happening?
I talked about Moore's law.
So this the obligatory slide of Moore's law.
So if you look at it, going from even before the 1970's, we had this amazing, amazing growth in the number of transistors available.
So Moore's law doesn't say anything about performance.
It says about the number of transistors available on a die.
Every 18 months, it doubles.
So, this basically -- some people even say -- this is the basis of these last two decades of economic expansion.
Because we got all this amazing power, because of this kind of growth in here.
The problem is, if you look at performance, it's flattening out.
So, this is spec numbers.
Kind of the way you measure performance is to have this set of programs that we can keep running on these machines.
And we keep running on these same machines.
So even though we are getting twice the amount of transistors, this is kind of flattening out.
And then, for the last few years, it's almost constant.
We're not getting anything from there.
And so, we're still running from the fumes of that amazing growth rates we had in this time.
And now we're kind of flattening out in here.
And why is this?
The main reason for this is, I will talk about four things.
One is power consumption.
Wire delay, and DRAM access time, and diminishing returns.
So I'll go a little bit into these details, to kind of give you an overview of why this happened.
The first thing is power.
So as we go about, we keep taking more and more power.
Right now, if you look at the one Moore trend, that, normally, these one point-some volts that these processors run, this draws more current than any of your houses back home.
I mean, that's amazing.
The houses probably draw like 150 amps.
And in one volt, we are drawing about 150 watts.
So there's that much current going to those things.
And, heat is a big issue.
So this still shows, OK, power, we are flattening out.
But another interesting thing is power efficiency.
So, we are trying to use these transistors.
We are trying to do more and more out of these transistors.
And if you look at how many watts, you are spending to get one of those spec rates, that's basically how much to do the same amount of work, we are spending a lot more power.
Because we are building these things that have very marginal return today.
That is the problem in building the current super-scaled type things.
Because it doesn't go.
This power efficiency.
This is why Google is building their shops next to the Grand Coulee dam, because they are sucking in so much power.
Because power efficiency is a big issue for them.
And, today, microprocessors, most the cost of it is more than purchasing.
It's basically feeding it with power and cooling it.
So another problem is wire delay.
If you try to build this monolithic, large process.
So what happened is, in the good old days, within the clock cycle, in the processor [UNINTELLIGIBLE] this side.
You can basically send a wire from one end to another processor, still within one clock cycle.
So if you are building this monolithic thing, it's almost like sitting in a room.
When I talk, the other end of the room will hear it instantaneously.
Because I'm talking slowly, or you're clocking slowly.
And today, we are around this area, So what that means is, in a clock cycle, you can't get a wire for more than a small part of the chip.
So it's almost like trying to communicate in this building.
I want to tell something to everybody, I can't broadcast.
Somebody has to go next-door to tell somebody to get the information through the entire thing.
Takes time.
And as we go more and more, this is going to be harder and harder and harder.
And what that means is, trying to build large, monolithic, processors like we did to keep the Moore's law kind of performance going, is not working.
And we have this problem.
Interesting third issue is DRAM access time.
So what's happening is microprocessors going like this.
Performance gets faster and faster.
DRAM gets doubled only about every 10 years.
And because of that, there's this interesting mismatch.
If you are trying to get to DRAM, if you put things in DRAM, it's almost -- about 10 years ago the kind of latency we had to the disk, is now we are having to DRAM.
So the way we look at this activity, the costs of it are going to get big.
And it's also a power issue because if you're on chip, it's probably two orders of magnitude cheaper to get to a word on chip then go all the way outside, get to the DRAM and get things out.
And so, between power and this, is also creating a huge amount of issue, an impossibility for us to keep this kind of Moore's law type performance to keep going.
The final thing is diminishing returns.
So if you look at the `80s, that is the era of superscalars.
We had amazing performance improvements.
For example, when you started the decade, every CPI speed, clocks per instructions.
How many clocks you had to run to get one instruction going?
At the beginning of the decade, you had to have about 10 clock cycles to execute one instruction on [UNINTELLIGIBLE].
At the end of the decade, we went about one clock cycle [UNINTELLIGIBLE] we get a new instruction graduated from that.
And that gave an amazing performance boost.
And in the '90s, we had an era of diminishing returns.
We kind of went from this two-way, superscaled, almost like six-way out of order, out of issue, branch prediction, everything's predicated.
And this huge pipeline.
Amazingly complex machines.
But we went from one clock per instruction to about half a clock per instruction.
So we just doubled that kind of performance.
And performance was below expectations, and projects got delayed and cancelled.
One thing that failed -- I'm not going to talk about that much.
Is, even at this time, you could get that clock frequency higher and higher.
And because of power and other physical limitations, we are also hitting a wall in clock frequency.
We are not getting -- because at that time, if you realize, everybody [UNINTELLIGIBLE] saying how fast their machine ran.
And they have stopped that, because they can't get machines to double their frequencies any more.
So, between those two, your performance kind of stuck.
So that is why now we are starting the new era of multicores.
We need explicit parallelism.
We can't build these large monolithic things that keep giving you performance every few years.
That stopped.
And we can build a sequential machine where what you get in 10 years' time will almost be same as what you get today.
But, we get more and more silicon.
It's doubling.
Moore's law still keeps going.
So what you do with that silicon is replicate, and build more.
And that's where we are going.
So, if you look at what's happening, the unicores are a dying breed.
The last big one was Intel had this project that tried to build very fast, amazing, superscalar machines.
All those projects had to get cancelled.
Because they could not get the performance.
Power was way too high, they couldn't get the power in there.
And those things got cancelled.
And as you see, there's more and more multicores coming in there.
In fact, if you look at it, multicores are here in a big way.
If you look all these machines, the Pentium line of uniprocessors.
From recent time, there had been four cores, eight cores and even machines that have more than 256 cores.
Small cores, but we are basically going into -- that's a new trend.
And if you look at the trend line, it's an interesting trend line going in there.
So, what I'm going to do now is switch gears.
Because, we had Mike [?
Brown ?]
from IBM.
We are very fortunate to have him here to give us a feel about the cell processor.
Because we are going to spend this month really getting to understand cell processing.
And really taking advantage of that.
And he had done huge amounts of work with cells.
He's at IBM TJ Watson.
And so he's going to, in the next hour, hour and half, he's going to give us the lowdown on cell.
And then I will conclude with a little bit more remarks about how we are going to do the class structure.
OK. Mike.
Hopefully -- he has to catch a flight, so we will kind of go fast.
Hopefully you guys [INAUDIBLE] go without too much of a break.
So if things get too tired, let us know so we'll see if we can put a break in the middle.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I will go into a little about what the class is, and what you're going to do.
I'll just put one slide about requirements and outcomes.
So what are the requirements?
We require you to be a good programmer with a lot of programming experience.
As I said, we are going down to very bare metal.
And there won't be any niceties of just writing Java code.
So you're going to run into a lot of these kind of issues.
Fluent in C is -- I wouldn't call it a 100% requirement, but it really helps if you're fluent in C. If not, you'll have to probably spend a lot of time trying to understand C, C++.
And in this class that means -- just start doing it today, just get a C book and then try to understand.
If you run a lot of Java it might be an easy transition.
But get to the transition fast, don't wait.
What are we expecting at the end?
We expect at the end for you to know --
I have to go.
Thank you very much [INAUDIBLE]
So what do we expect?
We expect at the end you'll probably understand a lot of fundamentals and concepts of parallel programming -- both hardware and software.
Because the nice thing about -- or the nice thing or the bad thing about Cell -- is a lot of hardware conceptuals to expose.
There's no nice abstraction layers in there.
You'll understand a lot of issues about parallel performance.
We started here, you write a program, compile -- it's not just order n you're looking for.
You're looking for all these constants.
All these things matter, basically.
You should be able to synthesize a fairly complex parallel program by the end, because that's what your group project's going to be.
And you'll get a lot of hands-on experience with the Cell processor, so we'll understand this one instance very well.
So that is what we are expecting will happen at the end of this 3.5 weeks.
So the project.
You proposed the projects, we did something different.
So it's all your ideas that we are going to do.
We selected seven groups out of these -- there are seven projects in there -- mainly based on the strength of the project proposals.
And I was pretty impressed with the height of things you guys proposed.
So here are the seven projects.
Let's actually try to figure out who's who.
Who's doing distributed real-time ray tracer?
OK, that's not true.
OK, how about the global illumination?
OK.
Linear algebra pack?
OK, good.
Molecular dynamics?
OK.
Speech synthesizer?
OK. Soft radio?
OK. Backgammon tutor?
OK. And there are some people who are, I think, in this class thinking that they can join a group.
So at the end, there are a few groups that only have one member, so probably go ahead and talk to them and see whether they are willing to work with people.
So I'll give you a little bit of time, because I'm going to finish early.
OK, let's ask -- who are the people who are hoping to join a group?
Is there anybody who came?
I got a few emails, I don't know that anybody's here.
Oh good.
So it's not going to be a problem.
So I did the project characteristics.
You can see that they are ambitious projects, but accomplishable.
I was worried that people would say, "I'm going to build this entire thing", and it's going to take a year.
But I think most of the projects that they proposed are doable.
Another thing is important and relevant things.
I think for a lot of these things, if the code is done right other people might even use it.
I think this can start something, it's not just going to be this project.
And there's a lot of opportunity to sizzle.
I think some people will come up with these things that I think at the end will be so amazing.
And that's what got me excited.
But get them started as soon as possible, because this is not a semester course, this is going to run so fast.
There's only a few all nighters you can pull, basically is what it is.
So there's a note of caution.
Cell processor is very new.
We heard about that today.
I mean this is a really cool thing, but it has its problems in there.
It's not an easy architecture to work with.
There is though this nice abstraction.
So it's just then from this being completely abstracted out, writing Java code, to all the way to metal.
So it's just almost a jump from going to a higher language of assembly.
So there's a huge, big gap in there, and that's going to be a lot of issues in there.
The tool chain is very thin and brittle.
So my pointing out all those things, you wish there's a tool, but there's nothing.
And even the existing tools -- we just got the SDK 2.0 into working I think this morning -- and so there might be issues in there.
Most of the staff have limited experience.
For example, SDK 2.0.
We just got it working, so we haven't used it yet in there.
So that can create problems, especially in a fast moving course.
The projects you are doing are of your own making.
Most of the classes -- if you need to [UNINTELLIGIBLE] classes, we actually come up with the projects, TAs go and implement it once, we know all the gotchas.
And then when you get the project we know there's an existence proof that it's doable, it works, the tools work.
We know at least there's one path through it.
There's nothing like that here.
Everybody's kind of just jumping at it head on, and hoping that we can read them.
And we might find issues that are very hard, and that's why we actually have very close contact with those guys at IBM.
So if things happen, we'll call them and they will be available to help us.
But it's not something you can say -- oh yeah, I know, just go do that.
Or -- here's the way out.
So there might be issues like that, so we have to be very careful of that.
And you will face these kinds of problems.
That's more given than anything else, so be ready for that.
This is not just writing some high level code and getting away with it.
And you're all in this together.
So the staff, you, me, everybody -- we're just doing it for the first time in some sense.
And that's the fun part about it, because as you pointed out, there might be some stuff that you guys come out with that will become more like folklore and the basis for the things people do.
Because you might say -- hey, there's a neat way of doing that.
You will be the first to do this kind of thing.
So there's that time, but on planning the three week course it's a large chunk to bite.
If it is more for a PhD, then you can say -- OK, spend three months figuring it out.
There's no other time limit.
So we will try to manage this process, but you need to try to -- expectations of what you are trying to get done, and then be able to get it done.
So you have to always balance this out.
If you're running into issues, get to us very fast.
Because we need to figure it out.
And we might have to actually also figure it out from somebody else, so we can escalate it up and get the information.
Because three weeks are going to go so fast.
How many people are taking the class for a grade?
So people who are taking it for a grade, here is what we are going to do.
The first thing, what I call mini quizzes, is just a prop so you will come into class.
So the way I am going to do that is, at the beginning of the class I am going to give you a piece of paper with two questions from the previous lecture.
You can do anything you want to answer it.
You can talk to people, look it up, whatever.
And then at the end of the class basically we want it back.
And the only requirement is, you have to come individually to pick your thing up, and you have to come individually to hand it in, and you can't write somebody else's anwers physically, but you can talk to each other.
So we'll try to make it a little bit entertaining, and a little bit thought-provoking, but nothing like you have to spend hours doing it.
It'll be probably five minutes.
Normally we will do a break inbetween, so you probably can talk to each other and come up with some of the answers.
So this is just the fact that the people who are actually getting the grades will be coming to class, more than anything else I'll be testing.
And then we have these lab projects at the beginning, that will get some grade.
And the final group project.
And then we have a final competition.
So since everybody's not doing the same project, we are going to decide on performance -- so how well your code ran, how much MIPS or [?
GOPs ?]
you were able to get from that.
And how raw performance is in there.
And then we will also look at it from a complexity point of view.
So for example, as I pointed out, if you do matrix multiplier you get certain performance, so if you do [?
FF3, ?]
you get certain performance.
So just absolute numbers is not what matters, but how far you can get hard algorithm too.
Completeness.
So were you able to get your project through?
How much do you actually have a compete solution at the end that something works?
That's important.
Algorithmic complexity -- how much of a complexity did you bite in, both from more of a pure algorithm point of view, and also implementing some shared complexity in there.
And demo and presentation.
So OK, those two.
So we are going to select a winning team.
We haven't figured exactly what, but we'll give everybody a gift set depending on -- this is what we are aiming at right now.
Also we are going to invite the winning team to spend a day at IBM TJ Watson.
You get to give a presentation to a bunch of people there, and probably go see what they are doing.
That should be a fun day to spend.
We haven't scheduled a day, depending on the class schedule and stuff like that, the winning team will go spend a day out in there.
That will be a fun thing to do, there's a lot of cool stuff going on there.
They will get to actually interact with some researchers, see what they do, as well as present this work to them.
And since this is new everybody's excited.
I think hopefully we'll come up with a team that will do things that other people will be interested in and will use.
So these kind of come from backward because I wanted to get Mike Perrone first.
So my name is Saman -- Saman Amarasinghe -- I live in CSAIL.
I have had a lot of interest in -- [LAUGHTER] Not as much as you guys probably.
I have a lot of interest in languages, compilers, and computer architecture.
So if you have been at this game for a long time, in the early 1990s there was a huge surge in parallel performance, work on parallelism, and at that time I did the SUIF parallelizing compiler in there.
And then, everybody says -- hey, there's this Moore slope of performance, who cares about parallelism?
So there was this decade when this paralellism kind of died down.
I kind of hid under the radar and kind of survived through that thing, and then again I think parallelism coming up.
So we are doing a bunch of parallel work in here.
So I probably have a perspective that goes somewhere old, and then coming up.
Probably Aaron can also relate to that.
Aaron: There's one of us who did that
Yeah exactly.
Going through that, kind of survived in that.
And Rodric is sitting there.
So Rodric is right now at IBM PJ Watson, but he's probably going to live at CSAIL next month, so he's going to be here more than me in fact.
He was actually in the lab until a few months ago, as a research scientist.
He's also interested in a lot of these kinds of things -- compilers, architecture, parallel architecture, and FPGA type work.
And then we have two TAs.
Where's David and Phil?
OK, get up.
So they actually have some experience.
Probably they are about a week ahead of you on understanding Cell, not too much.
They are going to lead the recitations and be available to help you in there.
So really use those resources as much as possible.
And then we are going to have a huge amount of guest lecturers, and I was pretty happy with the kind of list we got.
Mike Perrone just came and gave you the lecture, and Alan is going to talk in a few weeks.
And Professor Arvind, Brad Kuszmaul, Mike Acton -- that Mike Perrone talk about is going to come at the end.
And Bill Thies, who was one of the PhD students in the Streamit group, is going to talk about the Streamit language.
So the way we have kind of structured the class is, how do you go about extracting parallelism.
There's implicit parallelism.
And sometimes the current way of everything done by hardware.
In the next lecture, I will just touch upon superscalar hardware, what it is and what its problems are.
Implicit parallelism means you can do by compiler.
So you write a C program, and let the compiler deal with the parallelism.
For as long as you are a programmer, you don't have to deal with that thing.
In lectures 11 and 12 I will talk about parallelizing compilers.
So we'll cover how you can do that [?
in C.
?]
So that means kind of abstract it out, and leaving it out of the end programmer.
So the hardware exposes it, but we put a software layer.
And then the other end is explicit parallelism.
That's where, I think, we are going to spend most of the time in this class.
So in the first part of explicit parallelism is the library approach.
What that means is you are using MPI type things, so you are basically at the metal dealing with this parallelism.
I am going to do lecture 4 on concurrency.
How many people took 6.170 last year when I was teaching?
OK.
I did a lecture on concurrency there, so something very similar I'm going to do in here -- to get the basics of parallelism.
In order to write a parallel program, the program has to run concurrently.
There's a lot of issues for writing concurrent programs.
It is necessary to get parallel, it's not sufficient.
There are a lot more other things to worry about.
But let's deal with those kind of issues -- deal with things like synchronization, deadlocks, and issues like that.
So I'm going to give that, and then Rodric is going to have a few lecture on design patterns.
So if you want to write a parallel program, how do you look at it?
Different algorithms will have different ways of parallelizing.
So he's going to go about looking at these kind of parallel patterns.
If you have this kind of program, here is a way to look at it.
Here's another way to look at it.
So kind of go through that in there.
And in concurrency we work with a lot more abstract parallel model, and get slowly down to all the realism needed in dealing with Cell.
And then we are going to have a bunch of guest lectures, mainly dealing with other ways of doing parallelism.
So this is one way.
We are going to have a lecture on Streamit, because you might be able to use that also as a way on Cell.
So a little bit of high abstraction, we are trying to make that available on Cell.
So if that comes to, then you might be able to even have that as a part of your toolbox in there.
The other three talks are about very different approaches for parallelism.
I think they will give you a big picture view, it's not that you're just going through one parallelling down in there.
I think at the end of to the day you'll understand not only how to deal with Cell, but kind of a general way of looking at parallelism, and maybe from that approach [UNINTELLIGIBLE] parallelism in there.
So the lectures are organized as follows.
We are now at the end of the day here.
Tomorrow we'll have a recitation on getting to know Cell.
We are trying to figure out a good room, so just check your email to figure out exactly where we are going to have it, because we are trying to figure it out.
We want to find a room with machines so everybody can actualy get hands-on experience in kind of a lecture format.
And then we will have the architecture, just general parallel architecture, and an introduction to concurrent programming.
And then Rodric is going to have individual meets with individual groups, trying to get the groups' projects going.
So at that point kind of figure out how we are doing it, have some early ideas about it so you can start thinking and start the process of doing that.
You can start working on Cell probably from tomorrow, but really get a day to kind of get familiarized, and then here start the project.
And then we will have some lectures on design patterns in here.
And as we go we'll have more hands-on experience also.
Then we'll have a lecture on Streamit.
So if you are using Streamit for a part of the run program, you'll be able to understand what's going on.
And we'll get to some debugging tools, hopefully we will have to deal with these things before that, but here's something that if you're really stuck, to get a breadth of tools available in here.
And then we'll talk in general about debugging and performance monitoring and performance optimization.
So at this point hopefully you have done the first cart, you have figured out what problems, and start optimizing things in here.
And then we'll going into a little bit more breadth, things like traffic paralleizing compilers that I talk about, getting implicit parallelism in here.
And then we will have a bunch of guest lecturers coming in.
And we'll talk about some hardware and the future, because we are just starting this very new phase in programming that might change almost everybody.
Five years, ten years down the line if you go through computer science, you might have a very different flavor, if what people think of parallelism takes hold.
So let's look at some future, and talk about that.
And it's a nice way to finish the lectures.
A few more days and then we'll have group presentations in here.
So each group will get about 15 minutes time to present what they have done and give a little bit of a demo.
And then finally on Friday we will have awards and reception.
So this is what we got.
Any questions?
For recitations, if we're not in a room with machines should we bring laptops?
So that's a good question.
Who doesn't have a laptop they can bring?
OK.
If that is the case, we can have it in any room we want, and have the network.
And we might have one or two additional machines for people who don't have a laptop.
And that actually will work much better than everybody marching into some kind of a computer room that's not structured properly for doing a class.
OK, that actually helps a lot.
That's a good point, I think that helps a lot.
OK, we'll do that that way then
How do we get access to these machines?
OK. Rodric, you --
Every room has a PS3 dedicated to them.
And every group has accounts on their dedicated PS3.
I have your usernames and passwords.
I will pass them out tomorrow, because I'm going to propagate the new SDK to all the PS3's -- to sort of test it with one and make sure everything's working.
So you'll get them at the first recitation tomorrow.
Unless you really want them today, then come talk to me or send me email
You'll get to take home --
[INAUDIBLE]
Yes
Yes
We couldn't even put it in the machine room, because they were scared somebody was going to take them home.
So we actually have it in a different room -- closed, nicely locked -- or on the network.
So I have been a very popular person lately, a lot of people have tried to bribe me to get access to one of these masters
[UNINTELLIGIBLE]
How much does the PS3 cost?
Officially $600, but then eBay's going up to like $2,500 these days
No
That's gone down?
They're down between $400 and $500
Oh,
They tried [?
to rope me in.
?]
[LAUGHTER]
OK.
But during Christmas it was down for -- a few desperate payers who were trying to bribe me.
[LAUGHTER] So I guess their volume has caught up to --
No, the demand has dropped

[INAUDIBLE]
Will we get to see the Playstation?
Tomorrow we can bring one --
I can bring one [INAUDIBLE]
If you put a game in it's fun.
You can put a game in and play it, but this is going to be Linux so it's a little bit boring.
[LAUGHTER]
So these are things that actually cannot play the games
It can.
Interestingly, mostly these game machines are sold for less than production value.
So Microsoft, Nintendo, and Sony before made sure that nobody can run anything other than the published games that they've charged $25 per pop to recover the cost.
Huh?
No, $60 for the game.
I think they are charging $25 from the game maker just for the privilege of running the game.
That's how they make their money.
But, [UNINTELLIGIBLE] when they made PS3 bootable by Linux.
So in fact, it's much cheaper, you get a Blu-Ray player, and this Cell processor for a lot less than the production cost
Does Linux on the PS3 run on top of the PS3 OS though?
No.
It just boots up PS3.
What it does is in fact -- right now, you don't get access to the graphics processor properly.
So you can run open G11.
You have framebuffer access, but you don't get the full graphics access out of that
Which distribution are we using?
Which one?
We're using Yellow Dog
So in other words, Mario and Luigi are subsidizing what we're going to do.
[LAUGHTER]
Yeah, exactly.
All those shoot them up games are basically what subsidizes.
Actually he said, in one sense, you get the processor down to $200 because they are selling all the other things.
And the other thing, actually if you want to build a high performance Cell farm, it's probably cheaper because I think that the Cell blend would be $2,000.
It should be cheaper to buy a bunch of PS3's, connect them together.
But there might be issues about things like network access, file access, because as you pointed out, the board access is pretty primitive.
Because for games you don't need this high bandwidth connection, you just need to download it once.
So there might be issues using it for cluster.
OK.
Anything else?
OK, good.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
So my name's Michael Perrone.
I'm at the T.J. Watson Research Center, IBM research.
Doing all kinds of things for research, but most recently-- that's not what I want.
There we go.
Most recently I've been working with the cell processor for the past three years or so.
I don't want that.
How's that?
And because I do have to run out for a flight, I have my e-mail here if you want to ask me questions, feel free to do that.
What I'm going to do in this presentation is as Saman suggested, talk in depth about the cell processor, but really it's still going to be just the very surface because you going to have a month to go into a lot more detail.
But I want to give you a sense for why it was created, the way it was created, what it's capable of doing, and what are the programming considerations that have to be taken in mind when you program.
Here's the agenda just for this section, Mike, of this class.
I'll give you some motivation.
This is going to be a bit of a repeat, so I'll go through it fairly quickly.
I'll talk about the design concepts, hardware overview, performance characteristics, application affinity-- what good is this device?
Talk about the software and this I imagine is one of the areas where you're going to go into a lot of detail in the next month because as you suggested, the software really is the issue and I would actually go a little further and say, why do people drive such large cars in the U.S.?
Why do they waste so much energy?
The answer is very simple.
It's because it's cheap.
Even at $3 a gallon, it's cheap compared to say, Europe and other places.
The truth is it's the same thing with programmers.
Why did programmers program the way they did in the past 10, 20 years?
Because cycles were cheap.
They knew Moore's law was going to keep going and so you could implement some algorithm, you didn't have to worry about the details, as long as you got the right power law-- if you got your n squared or n cubed or n log n, whatever behavior.
The details, if the multiplying factor was 10 or 100 it didn't matter.
Eventually Moore's law would solve that problem for you, so you didn't have to be efficient.
And I think I've spent the better part of three years trying to fight against that and you're going to learn in this class that, particularly for multicore you have to think very hard about how you're going to get performance.
This is actually the take home message that I want to give.
I think it's just one or two slides, but we really need to get to these because that's where I want to get you thinking along the right lines.
And then there's a hardware consideration, we can skip that.
All right, so where have all the gigahertz gone, right?
We saw Moore's law, things getting faster and faster and the answer is I have a different chart that's basically the same thing.
You have relative device performance on this axis and you've got the year here.
And different technologies were growing, growing, growing, but now you see they're thresholding.
And you go to conferences now, architecture conferences, and people are saying, Moore's law is dead.
Now, I don't know if I would go that far and I know there are true believers out there who say, well maybe the silicon on the insulator technology is dead, but they'll be something else.
And maybe that's true and maybe that is multicore, but unless we get the right programming models in place it's not going to be multicore.
Here's this power density graph.
Here we have the nuclear reactor power up here and you see pentiums going up now.
Of course, there's a log plot, so we're far away, but on this axis we're not far away.
This is how much we shrink the technology, the size of those transistors.
So if we're kind of going down by 2 every 18 months or so, maybe it's 2 years now, we're not so far away from that nuclear reactor output.
And that's a problem.
And what's really causing that problem?
Here's a picture of one of these gates magnified a lot and here's the interface magnified even further and you see here's this dielectric that's insulating between the 2 sides of the gate-- we're reaching a fundamental limit.
A few atomic layers.
You see here it's like 11 angstroms.
What's that?
10, 11 atoms across?
If you go back to basic physics you know that quantum mechanical properties like electrons, they tunnel, right?
And they tunnel through barriers with kind of an exponential decay.
So whenever you shrink this further you get more and more leakage, so the current is leaking through.
In this graph, what you see here is that as this size gets smaller, the leakage current is getting equivalent to the active power.
So even when it's not doing anything, this 65 nanometer, the technology is leaking as much power as it actually uses.
And eventually, as we get smaller, smaller we're going to be using more power, just leaking stuff away and that's really bad because as Saman suggested we have people like Google putting this stuff near the Coulee Dam so that they can get power.
I deal with a lot of customers who have tens of thousands of nodes, 50,000 processors, 100,000 processors.
They're using 20 gigabytes-- sorry, megahertz.
No, megawatts, that's what I want to say.
It's too early in the morning.
Tens of megawatts to power their installations and they're choosing sites specifically to get that power and they're limited.
So they come to me, they come to people at IBM and they say, what can we do about power?
Power is a problem.
And that's why we're not seeing increasing the gigahertz.
Has this ever happened before?
Well, I'm going to go to this quickly, yes.
Here we see the power outage output of a steam iron, right there per unit area.
And something's messed up here.
You see as the technology changed from bipolar to CMOS we were able to improve the performance, but the heat flux got higher again and that begs the question, what's going to happen next?
And of course, IBM, Intel, AMD, they're all betting this multicore.
And so there's an opportunity from a business point of view.
So now, that's the intro.
Multicore: how do you deal with it?
Here's a picture of the chip, the cell processor.
You can see these 8 little black dots.
They're local memory for each one of 8 special purpose processors, as well as a big chunk over here, which is a ninth processor.
So this chip has 9 processors on board and the trick is to design it so that it addresses lots of issues that we just discussed.
So let me put this in context, cell was created for the Sony Playstation 3.
It started in about 2000 and there's a long development here until it was finally announced over here.
Where was it first announced?
It was announced several years later and IBM recently announced a cell blade about a year back and we're pushing these blades and we're very much struggling with the programming model.
How do you get performance while making something programmable?
If you go to customers and they have 4 million lines of code, you can't tell them just port it and it'll be 80 person years to get it ported, 100 person years more.
And then you have to optimize it.
So there are problems and we'll talk about that.
But it was created in this context and because of that, this chip in particular, is a commodity processor.
Meaning that it's going to be selling millions and millions.
Sony Playstation 2 sold an average of 20 million units each year for 5 years and we expect the same for the Playstation 3.
So the cell has a big advantage over other multicore processors like the Intel Woodcrest, which has a street price of about $2000 and the cell around 100.
So not only do we have big performance improvements, we have price advantages too because of that commodity market.
All right, let's talk about the design concept.
Here's a little bit of a rehash of what we discussed with some interesting words here.
We're talking about a power wall, a memory wall and a frequency wall.
So we've talked about this frequency wall.
We're hitting this wall because of the power really and the power wall people just don't have enough power coming into their buildings to keep these things going.
But memory wall, Saman didn't actually use that term, but that's the fact that as the clock frequencies get higher and higher, memory appeared further and further away.
The more cycles that I have to go as a processor before the data came in.
And so that changes the whole paradigm, how you have to think about it.
We have processors with lots of cache, but is cache really what you want?
Well, it depends.
If you have a very localized process where you're going to bring something into cache and the data is going to be reused then that's really a good thing to do.
But what if you have random gather and scatter of data?
You know, you're doing some transactional processing or whatever mathematical function you're calculating is very distributed like an FFT.
So you have to do all sorts of accesses through memory and it doesn't fit in that cache.
Well, then you can start thrashing cache.
You bring in one integer and then you ask the cache for the next thing, it's not there, so it has to go in and so you spend all this time wasting time getting stuff into cache.
So what we're pushing for multicore, especially for cell is the notion of a shopping list.
And this is where programability comes in and programing models come in.
You really need to think ahead of time about what your shopping list is going to be and the analogy that people have been using is you're fixing something in your house, you're pipe breaks.
So you go and say, oh, I need a new pipe.
So you go the store, you get a pipe.
You bring it back and say, oh, I need some putty.
So you go to the store, you get some putty.
And oh, I need a wrench.
Go to the store-- that's what cache is.
So you figure out what you need when you need it.
In the cell processor you have to think ahead and make a shopping list.
If I'm going to do this calculation I need all these things.
I'm going to bring them all in, I'm going to start calculating.
When I'm calculating on that, I'm going to get my other shopping list.
So that I can have some concurrency of the data load with the computes.
I'm going to skip this here.
You can read that later, it's not that important.
Cell synergy, now this is kind of you know, apple pie, motherhood kind of thing.
The cell processor was specifically designed so that those 9 cores are synergistic.
That they interoperate very efficiently.
Now I told you we have 8 identical processors, we call those SPEs.
In the ninth processor its the PPE.
It's been designed so that the PPE is running the OS and it's doing all the transaction file systems and what not so that these SPEs can focus on what they're good at, which is compute.
The whole thing is pullled together with an element interconnect bus and we'll talk about that.
It's very, very efficient, very high bandwidth bus.
Now we're going to talk about the detail hardware components.
And Rodric somewhere, there you are, asked me to actually dig down into more of the hardware.
I would love to do that.
Honestly, I'm not a hardware person.
I'll do the best I can, perhaps at the end of the talk we'll dig down and show me which slides you want.
But I've been dealing with this for so long that I can do a decent job.
Here's another picture of the chip.
It has lots of transistors.
This is the size.
We talked about the 9 cores, it has 10 threads because this power processor, the PPE has 2 threads.
Each of these are single threaded.
And this is the wow factor.
We have 200 gigaflops, over 200 gigaflops of single precision performance on these chips.
And over 20 gigaflops of double precision and that will be going up to 100 gigaflops by the end of this year.
The bandwidth to main memory is 25 gigabytes per second and up to 75 gigabytes per second of I/O bandwidth.
Now this chip really has tremendous bandwidth, but what we've seen so far-- particularly with the Sony Playstation and I think you may have lots of them here, the board is not designed to really take advantage of that bandwidth.
And even the blades that IBM sells really can't get that type of bandwidth off the blade.
And so if you're keeping everything local on the blade or on the Playstation 3 then you have lots of bandwidth internally.
But off blade or off board you really have to survive with something like a gigabyte, 2 gigabytes in the future.
And this element interconnect bus I mentioned before has a tremendous bandwidth, over 300 gigabytes per second.
The top frequency in the lab was over 4 gigabytes-- gigahertz, sorry.
And it's currently running when you buy them at 3.2 gigahertz.
And actually the Playstation 3's that you're buying today, I think, they only use 7 out of the 8 SPEs.
And that was a design consideration from the hardware point of view because as these chips get bigger and bigger, which is if you can't ratchet up the gigahertz you have to spread out.
And so as they get bigger, flaws in the manufacturing process lead to faulty units.
So instead of just throwing away things, if one of these SPE is bad we don't use it and we just do 7.
As the design process gets better by the end of this year they'll be using 8.
The blades that IBM sells, they're all set up for 8 OK, so here's a schematic view of what you just saw on the previous slide.
You have these 8 SPEs.
You have the PPE here with this L1 and L2 cache.
You have the element interconnect bus connecting all of these pieces together to a memory interface controller and a bus interface controller.
And so this MIC is what has the 25.6 gigabytes per second and this BIC has potentially 75 going out here.
Each of these SPEs has its own local store.
Those are those little black dots that you saw, those 8 black dots.
It's not very large, it's a quarter of a megabyte, but it's very fast to this SXU, this processing unit.
It's only 6 cycles away from that unit.
And it's a fully pipelined 6 so that if you feed that pipeline you can get data every cycle.
And here, the thing that you can't read because it's probably too dark is the DMA engine.
So one of the interesting things about this is that each one of these is a full fledged processor.
It can access main memory independent of this PPE.
So you can have 9 processes or 10 if you're running 2 threads here, all going simultaneously, all independent of one another.
And that allows for a tremendous amount of flexibility in the types of algorithms you can implement.
And because of this bus here you can see it's 96 bytes per cycle and we're at 3.2 gigahertz.
I think that's 288 gigabytes per second.
These guys can communicate to one another across this bus without ever going out to main memory and so they can get much faster access to their local memories.
So if you're doing lots of computes internally here you can scream on this processing; really, really go fast.
And you can do the same if you're going out to the memory interface controller here to main memory, if you sufficiently hide that memory access.
So we'll talk about that.
All right, this is the PPE that I mentioned before.
It's based on the IBM power family of processors, it's a watered down version to reduce the power consumption.
So it doesn't have the horse power that you see in say a Pentium 4 or even-- actually, I don't have an exact comparison point for this processor, but if you take the code that runs today on your Intel or AMD, whatever your power and you recompile it on cell it'll run today-- maybe you have to change the library or two, but it'll run today here, no problem.
But it'll be about 60% slower, 50% slower and so people say, oh my god this cell processor's terrible.
But that's because you're only using that one piece.
So let's look at the other-- OK, so now we go into details of the PPE.
Half a megabyte of L2 cache here, coherent load stores.
It does have a VMX unit, so you can do some SIMD operations, single instruction multiple data instructions.
Two-way hardware multithreaded here.
Then there's an EIB that goes around here.
It's composed of four 16 byte data rings.
And you can have multiple, simultaneous transfers per ring for a total of over 100 outstanding requests simultaneously.
But this slide doesn't-- this kind of hides it under the rug.
There's a certain topology here.
And so these things are going to be connected to those 8 SPEs.
And depending on which way you send things, you'll have better or worse performance.
So some of these buses are going around this way and some are going counterclockwise.
And because of that you have to know who you're communicating if you want have real high efficiency.
I haven't seen personally cases where it made a really big difference, but I do know that there's some people who found, if I'm going from here to here I want to make sure I'm sending things the right way because of that connectivity.
Or else I could be sending things all the way around and waiting
Just a quick question
Yes
Just like you said you could complie anything on the power processor would be slower, but you can.
Now you also said the cell processor is in itself a [INAUDIBLE] processor.
Can I compile it in a C code just for that as well
C code would compile.
There's issues with libraries because the libraries wouldn't be ported to the SPE necessarily.
If it had been then yes.
This is actually a very good question.
It opens up lots of things.
I don't know if I should take that later
Take it later
Bottom line is this chip has two different processors and therefore you need two different compilers and it generates two different source codes.
In principle, SPEs can run a full OS, but they're not designed to do that and no one's ever actually tried.
So you could imagine having 8 or 9 OSes running on this processor if you wanted.
Terrible waste from my perspective, but OK, so let's talk about these a little bit.
Each of these SPEs has, like I mentioned this memory flow controller here, an atomic update unit, the local store, and the SPU, which is actually the processing unit.
Each SPU has a register file with 128 registers.
Each register is 128 bits.
So they're native SIMD, there are no scalar registers here for the user to play with.
If you want to do scalar ops they'll be running in those full vector registers, but you'll just be wasting some portion of that register.
It has IEEE double precision floating point, but it doesn't have IEEE single precision floating point.
It's curiosity, but that was again, came from the history.
The processor was designed for the gaming industry and the gamers, they didn't care if it had IEEE.
Who cares IEEE?
What I want is to have good monsters right on the screen.
And so those SIMD registers can operate bitwise on bytes, on shorts, on four words at a time or two doubles at a time.
The DMA engines here, each DMA engine can have up to 16 outstanding requests in its queue before it stalls.
So you can imagine you're writing something, some code and you're sending things out to the DMA and then all of a sudden you see really bad performance, it could be that your DMA egine has stalled the entire processor.
If you try to write to that thing and then that queue is full, it just waits until the next open slot is available.
So those are kind considerations
You mean [UNINTELLIGIBLE PHRASE]
Yes
It's not the global one?
Right.
That's correct.
But there is a global address space
16 slots each in each
Right.
Exactly.
Each MFC has its own 16 slots.
And they all address the same memory.
They can have a transparent memory space or they can have a partitioned memory space depending on how you set it up
Each SPU doesn't have its own-- the DMA goes onto the bus, [UNINTELLIGIBLE] that goes to a connection to the [UNINTELLIGIBLE]
You can add this data in the SPUs too.
You don't have to always go to outside memory.
You can do SPU to SPU communication basically
Right.
So I can do a DMA that transfers memory from this local store to this one if I wanted to and vice versa.
And I can pull stuff in through the-- yeah, I mentioned this stuff.
Now this broadband interface controller, the BIC, this is how you get off the blade or off the board.
It has 20 gigabytes per second here on I/O IF.
In 10 over here--I'm sorry, 5 over here.
I'm trying to remember how we get up to 70.
This is actually two-way and one is 25 and the other one's 30.
That gets you to 55.
This should be 10 and now, what's going on here?
It adds up to 75, I'm sure.
I'm sure about that.
I don't know why that says that.
But the interesting thing about this over here, this I/O IF zero is that you can use it to connect two cell processors together.
So this is why I know it's really 25.6 because it's matched to this one.
So you have 25.6 going out to main memory, but this one can go to another processor, so now you have these two processors side-by-side connected at 25.6 gigabytes per second.
And now I can do a memory access through here to the memory that's on this processor and vice versa.
However, If I'm going straight out to my memory it's going to be faster than if I go out to this memory.
So you have a slight NUMA architecture and nonuniform memory access.
And you can hide that with sufficient multibuffering.
So I know that this is 25 and I know the other one's 30.
I don't know why it's written as 20 there
Can the SPUs write to the [UNINTELLIGIBLE PHRASE]?
Yes, they can read from it.
I don't know if they can write to it.
In fact, that leads to a bottleneck occurring.
So I happily start a process on my PPE and then I tell all my SPEs, start doing some number crunching.
So they do that.
They get access to memory, but they find the memory is in L2.
So they start pulling from L2, but now all 8 are pulling from L2 and it's only 7 gigabytes per second instead of 25 and so you get a bottleneck.
And so what I tell everybody is if you're going to initialize data with that PPE make sure you flush your cache before you start the SPEs.
And then you don't want to be touching that memory because you really want to keep things-- stuff that the SPEs are dealing with-- you want to keep it out of L2 cache.
Here there's an interrupt controller.
An I/O bus master translation unit.
And you know, these allow for messaging and message passing and interrupts and things of that nature.
So that's the hardware overview.
Any questions before I move on?
So why's the cell processor so fast?
Well, 3.2 gigahertz, that's one.
But there's also the fact that we have 8 SPEs.
Each 8 SPEs have SIMD units, registers that are running so they can do this parallel processing on a chip.
We have 8 SPEs and each one are doing up to 8 ops per cycle if you're doing a mul-add.
So you have four mul-adds for single precision.
So you've got 8, that's 64 ops per cycle times 3.2.
You get up to 200 gigaflops per cycle, 204.8.
So that's really the main reason.
We've talked about this stuff here.
This is an image of why it's faster.
Instead of staging and bringing the data through the L2, which is kind of what we were just discussing and having this PU, this processing unit, the PPE manage the data coming in, each one can do it themselves and bypass this bottleneck.
So that's something you have to keep in the back of your mind when you're programming.
You really want to make sure that you get this processor out of there.
You don't want it in your way.
Let these guys do as much of their own work as they can.
Here's a comparison of theorectical peak performance of cell versus freescale, AMD, Intel over here.
Very nice.
That's the wow chart.
The theoretical peak, this is in practice, what did we see?
I don't know if you can read these numbers but what you really want to focus on is the first and last columns.
This is the type of calculation, high performance computing like matrix multiplication, bioinformatics, graphics, security, it was really designed for graphics.
Security, communication, video processing and over here you see the advantage against an IA 32, a G5 processor.
And you see 8x, 12x, 15, 10, 18x.
Very considerable improvement in performance.
In the back-- question?
[UNINTELLIGIBLE] previous slide, how did it compare to high [UNINTELLIGIBLE PHRASE]?
All right, so you're thinking like a peak stream or something like that?
Any particular [UNINTELLIGIBLE PHRASE].
The design of the SPUs is very reminiscent of [UNINTELLIGIBLE PHRASE]
So I believe, and I'm not well versed in all of the processors that are out there.
I think that we still have a performance advantage in that space.
You know, I don't know about Xilinx and those kind of things-- FPGAs I don't know, but what I tell people this is there's a spectrum.
And at one end you have your general purpose processors.
You've got your Intel, you've got your Opteron whatever, your power processor.
And then at the other and you've got your FPGAs and DSPs and then maybe over here, somewhere in the middle you've got graphical processing units.
Like Nvidia kind of things.
And then somewhere between those graphics processing processors and the general purpose processors you've got cell.
You get a significant improvement in performance, but you have to pay some pain in programming.
But not nearly as much as you have to do with the graphics processors and no where near the FPGAs, which are just every time you write something you have to rewrite everything.
Question?
Somewhat related to the previous question, but with a different angle.
I always figured anyone could do a [INAUDIBLE], so that's why I ask about FFTs.
Are they captured on the front or otherwise [UNINTELLIGIBLE]
Yeah, so this is actually one of the things I spent a lot of time on for FFTs.
I spent a lot of time with the petroleum industry.
They take these enormous boats, they have these arrays that go 5 kilometers back and 1 kilometer wide, they drag them over the ocean, and they make these noises and they record the echo.
And they have to do this enormous FFT and it takes them 6 months.
Depending on the size of the FFT it can be anywhere from a week to 6 months, literally
[UNINTELLIGIBLE]
Sorry?
Is this a PD FFT?
Sometimes I do too, but they do both.
I've become somewhat of an expert on these FFTs.
For cell the best performance number I know of is about 90 gigaflops of FFT performance.
You know, that's very good.
Yeah, it's like 50% of peak performance.
You know, it's easy to get 98% with [?
lynpacker ?]
or [?
djem ?]
on a processor like this and we have.
We get 97% of peak performance, but it's a lot harder to get FFTs up to that
Well, then I'll [INAUDIBLE] the next questions then which is somehow or another you get the FFT performance, you've got to get the data at the right place at the time.
[UNINTELLIGIBLE] So you've personally done that or been involved with that?
Right, so we do a lot of tricks.
I can show you another slide or another presentation that we talk about this, but typically the FFTs that we work with are somewhere from a 1024 to 2048, that's square.
And so it's possible to take say, the top 4 rows-- in the case of 1024, four rows complex, single precision I think is 16 kilobytes.
That fits into the local store very nicely.
So you can stop multibuffering.
You bring in one, you start computing on it.
While you're computing on those 4 in a SIMD fashion across the SIMD registers you're bringing in the next one.
And then when that one's finished you're writing that one out while your computing on the one that arrived and while you're getting the next one.
And since you can get the entire 1024 or 2000 into local store, you're only 6 cycles away from any element in it.
So it's much, much faster.
We also did the 16 million element FFT.
1D, yeah and we did some tricks there to make it efficient, but it was a lot slower
[UNINTELLIGIBLE PHRASE] would have to be a lot slower by the need for the problem.
[UNINTELLIGIBLE PHRASE]
What I remember it was fifteen times faster than a power 5.
It might have been a power 4, I don't remember, sorry.
I might want to skip this one.
I think I'm going to skip this one
[UNINTELLIGIBLE PHRASE]
Right.
Let's talk about what is the cell good for.
You kind of have a sense of the architecture and how it all fits together.
You may have some sense of the gotchas and the problems that might be there, but what did we actually applied to 2?
I mean you saw some of that here.
Here's a list of things that either we've already proven to ourself that it works well or we're very confident that it works well or we're working to demonstrate that it works well.
Signal processing, image processing, audio resampling, noise generation.
I mean, you can read through this list, there's a long list.
And I guess there are a few characteristics that really make it suitable for cell.
Things that are in single precision because you've got 200 gigaflops single and only 20 of double, but that will change as I mentioned.
Things that are streaming, streaming through and so single processing is ideal where the data comes through and you do your compute and then you throw it away or you give out your results and you throw it away.
Those are good.
And things that are compute intensive, where you bring the data in and you're going to crunch on it for a long time, so things like cryptography where you're either generating something from a key and there's virtually no input.
You're just generating streams of random numbers that's very well suited for this thing.
You see FFTs listed here.
TCPIP off load.
I didn't put that there.
There's actually a problem with cell today that we're working to fix that the TCPIP performance is not very good.
And so what I tell people to use is open NPI.
You know, so that over InfiniBand.
The PPE processor really doesn't have the horse power to drive a full TCPIP sack.
I'm not sure it has the horse power to do a full NPI stack either, but at least you have more control in that case.
The game physics, physical simulations-- I can show you a demo, but I don't know that we'll have time where a company called Rapid Mind, which is developing software to ease programmability for cell.
Basically you take your existing scalar code and you instrument it with C++ classes that are kind of SPE aware.
And by doing that, just write your scalar code and you get the SPE performance advantage.
They have this wonderful demo of these chickens.
They've got 16,000 chickens in a chicken yard.
You know, the chicken yard has varying topologies and the chickens move around and all 16,000 are being processed in real time with a single cell processor.
In fact, the Nvidia card that was used to render that couldn't keep up with what was coming out of the SPEs.
We we're impressed with that.
We're happy with that.
We showed it around at the game conferences and the gamers saw all these chickens and were like, this is really cool.
How do I shoot them?
So we said, you can't.
But maybe in the next version.
But the idea that we've designed this so that it can do physical simulations, and this is maybe an entree for some of you people when you're doing your stuff.
I don't know what kinds of things you want to try to do on cell, but I've seen people do lots of things that really have no business doing well on cell and they did very, very well.
Like pointer chasing.
I'm trying to remember.
There are two pieces of work.
One done by PNNL Fabritzio Petrini and he did a graph traversal algorithm.
It was very much random access and he was able to parallelize that very nicely on Cell.
And then there was another guy at Georgia Tech who did something similar for linked lists.
And you know, I expect things to work well on cell if they're streaming and they have very compute intensive kernels that are working on things, but those are two examples where they're very not very compute intensive and not very streaming.
They're kind of random access and they work very well.
Over here, target applications.
There are lots of areas where we're trying to push cell forward.
Clearly it works in the gaming industry, but where else can it work?
So medical imaging, there's a lot of success there.
The sysmic imaging for petroleum, aerospace and defense for radar and sonar-- these are all signal processing apps.
We're also looking at digital content creation for computer animation.
Very well suited for cell.
This is kind just what I just said.
Did I leave out anything?
Finance-- once we have double precision we'll be doing things with finance.
We actually demonstrated that things work very well.
You know, metropolis algorithms, Monte Carlo, Black shoals algorithms if you're familiar with these kind of things from finance.
They tell us they need double precision and we're like, you don't really need double precision, come on.
I mean, what you have is some mathematical calculation that you're doing and you're doing it over and over and over.
And Monte Carlo there's so much noise, we say to these people, why do you need double precision?
It turns out with decimal notation you can only go up to like a billion or something in single precision.
So they have more dollars than that, so they need double, for that reason alone.
But this gets back to the sloppiness of programmers.
And I'm guilty of this myself.
They said, oh we have double.
Let's use double.
They didn't need to, but they did it anyway.
And now their legacy code is stuck with double.
They could convert it all to single, but it's too painful.
Down on Wall Street to build a new data center is like $100 million proposition.
And they do it regularly, all of the banks.
They'll be generating a new data center every year, sometimes multiple times a year and they just don't have time or the resources to go through and redo all their code to make it run or something like cell.
So we're making double precision cell.
That's the short of it.
All right, now software environment.
This is stuff that you can find on the web and actually, it's changing a lot lately because we just released the 2.0 SDK.
And so the stuff that's in the slide might not actually be the latest and greatest, but it's going to be epsilon away, so don't worry about it too much.
But you really shouldn't trust these slides, you should go to the website and the website you want to go to is www.ibm.com/alphaworks
Tomorrow we are going to have a recitation session talking about the environment that we have created.
I think we just got, probably just set up the latest environment and then we increase it through the three weeks we've got.
This is changing faster than a three week cycle.
So [UNINTELLIGIBLE PHRASE] So this will give you a preview of what's going to be
Then you go to alphaworks, you go to search on alphaworks for cell and you get more information then you could ever possibly read.
We have a programmer's manual that's 900 pages long, it's really good reading.
Actually there's one thing in that 800, 900 hundred pages that you really should read.
It's called the cell programming tips chapter.
It's a really nice chapter.
But there are many, many publications and things like that, more than just the SDK in the OS and whatnot, so I encourage you to look at that.
All right, so this is kind of the pyramid, the cell software pyramid.
We've got the standards under here, the application binary interface, language extensions.
And over here we have development tools and we'll talk about each of these pieces briefly.
These specifications define what's actually the reference implementation for the cell.
C++ and C, they have language extensions in the similar way to the extensions for VMX for SSE on Intel.
You have C extensions for cell that allow you to use intrinsics that actually run as SIMD instructions on cell.
For example, you can say SPU underscore mul-add, and it's going to do a vector mul-add.
So you can get assembly language level control over your code without having to use any assembly language.
And then there's that.
There is a full system simulator.
The simulator is very, very accurate for things that do not run out to main memory.
They've been working to improve this so I don't know if recently they have made it more accurate, but if you're doing compute intensive stuff, if you're compute bound the simulator can give you accuracies within 99%.
You know, within 1% of the real value.
I've only seen one thing on the simulator more than 1% off and that was 4%, so the simulator is very-- excuse me-- very reliable.
And I encourage you to use it if you can't get access to hardware.
What else?
The simulator has all kinds of tools in there.
And I'm not going to go through the software stack in simulation.
This gives you a sense for-- you've got your hardware running here.
You can run this on any one of these platforms.
Power PC, Intel with these OS's.
The whole thing is written in TCL, the simulator.
And it has all these kind of simulators.
It's simulating the DMAs, it's simulating the caches and then you get a graphical user interface and a command line interface to that simulator.
THe graphical user interface is convenient, but the command line gives you much more control.
You can treat parameters.
This gives you a view of what the graphical userface looks like.
It says mambo zebra because that was a different project, but now it'd probably say system sim or something like that.
And you'll see the PPC-- this is the PPE I don't know why they changed it.
And then you have SP of zero, SP of 1 going down and it gives you some access to these parameters.
The model here, it says pipeline and then there's I think, functional mode or pipeline mode.
Pipeline mode is where it's really simulating everything and it's much slower.
But it's accurate.
And then the other is functional mode just to test the code actually works as it's supposed to
I guess one point in the class what we'll try and do is since each group has access to the the hardware, you can do most of the things in the real hardware and use the debugger in the hardware that's probably been talked about.
But if things gets really bad and you can't understand use simulator as a very accurate debugger only when it's needs needed because there you can look at every little detail inside.
This is kind of a thing, a last resort type thing
Yeah, I agree.
Like I said, I've been doing this for three years.
Three years ago we didn't even have hardware.
So the simulator was all we had, so we relied on it a lot.
But I think that usage of it makes a lot of sense.
This is the graphical interface.
You know, it's just a Tickle interface.
I'm going to skip through these things.
It just shows you how you can look at memory with this more memory access.
You get some graphical representation of various pieces.
You know, how many stalls?
How many loads?
How many DMA transactions?
So you can see what's going on at that level.
And all of this can be pulled together into this UART window here.
OK, so the Linux, it's pretty standard Linux, but it has some extensions.
Let's see.
Provided as a patch, yeah.
That might be wrong.
I don't know where we are currently.
You have this SPE thread API for creating threads from the PPEs.
Let's see.
What do I want to tell you here?
There's a better slide for this kind of information.
They share the memory space, we talked about that.
There's error event and signal handling.
So there are multiple ways you communicate.
You can communicate with the interrupts and the event and signaling that way or you can use these mailboxes.
So each SPE has its own mailbox and inbox and an outbox so you can post something to your outbox and then the PPE will read it when it's ready.
Or you can read from your inbox waiting on the PPE to write something.
You have to be careful because you can stall there.
If the PPE hasn't written you will stall waiting for something to fill up.
So you can do a check.
There are ways to get around that, but these are kind of common gotchas that you have to watch out for.
Then you have the mailboxes, you have the interrupts, you also have DMAs.
You can do communication with DMAs so you have at least three different ways that you communicate between the SPEs on cell.
And which one is going to be best really depends on the algorithm you're running.
So these are the extensions to Linux.
This is going to show you a bunch of things that you probably won't be able to read, but there's something called SPUFS, the file system that has a bunch of open, read, write, and close functionality.
And then we also have this signaling and the mailboxes that I mentioned to you previously.
And this you can't even read.
I can't even read this one.
What is it?
Ah, this is perhaps the most important one.
It says SPU create thread.
So the SPEs from the Linux point of view are just threads that are running.
The Linux doesn't really know that they're special purpose hardware, it just knows it's a thread and you can do things like spawn a thread, kill a thread, wait on a thread-- all the usual things that you can do with threads.
So it's a lot like P threads, but it's not actually P threads.
So here you could see these things are more useful.
This is SPE create groups.
So you can create a thread and thread group so that threads that are part of the same group know about one another.
So you can partition your system and have three SPEs doing one thing and five doing another.
So that you can split it up however you like.
You have get and set affinity so that you can choose which SPEs are running which tasks, so that you can get more efficient use of that element interconnect bus.
Kill and waits, open, close, writing signals, the usual.
Let me check my time here.
I really don't have a lot more time, so I'm going to say that we have this thread management library.
It has the functionality that I just mentioned.
In the next month or so you're going to go through that in a lot more detail.
The SPE comes with a lot of sample libraries.
These are not necessarily the very best implementation of these libraries and they're not even fully functional libraries, but they're suggestive of first of all, how things can be written to cell, how to use cell, and in some cases how to optimize cell.
Like the basic matrix operations, there's some optimization.
The FFTs are very tightly optimized, so if you want to take a look at that and understand how to do that type of memory manipulation.
So there are samples codes out there that can be very useful.
We'll skip that.
Oh, this is that FFT 16 million.
There's an example, it's on the SDK.
Actually, I don't know if you've got PS3's if all these things can run.
They should run.
Yeah, they should run.
There may be some memory issues out to main memory that I'm not aware of.
There are all kinds of demos there that you can play with, which are good for learning how to spawn threads and things like that.
You have your basic GNU binutils tools.
There's GCC out there.
There's also XLC.
You can download XLC.
In some cases, one will be better than the other, but I think in most cases XLC's a little better.
Or in some cases, actually a lot better.
So you can get that.
I'd recommend that.
There's a debugger which provides application source level debugging.
PPE multithreading, SPE multithreading, the interaction between these guys.
There are three modes for the debugger: stand alone and then attached to SPE threads.
Sounds like two.
That's problematic.
There's this nice static analysis tool.
This is good for looking for really tightly, optimizing your code.
You have to be able to read assembly, but it shows you graphically-- kind of-- where the stalls are happening and you can try and reorganize your code.
And then like Saman suggested, the dynamic analysis using the simulator is a good way to really get cycle by cycle stepping through the code.
And someone was very excited when they made this chart because they put these big explosions here.
You've got some compiler here that's going to be generating two pieces of code, the PPE binary and the SPE binary.
When you go through the cell tutorials for training on how to program cell you'll see that this code is actually plugged into-- linked into the PPE code.
And when the PPE code spawns a thread it's going to take a pointer to this code and basically DMA that code into the SPE and tell the SPE to start running.
Once it's done that, that thread is independent.
The PPE could kill it, but it could just let it run to its natural termination or this thing could terminate itself or it could be interrupted by some other communication.
But that's the basic process, you have these two pieces of code.
OK, so now this is really what I wanted to get to.
So I want lots of questions here.
There are 4 levels of parallelism in cell.
On the cell blade, the IBM blade you have two cell processors per blade.
So that's one level of parallelism.
At chip level we know there are 9 cores and they're all running independently.
That's another level of parallelism.
On the instruction level each of the SPEs has two instruction pipelines, so it's an odd and an even pipeline.
One pipeline is doing things-- the odd pipeline is doing loads and stores, DMA transactions, interrupts, branches and it's doing something called shuffle byte or the shuffle operation.
So shuffle operation's a very, very useful operation that allows you to take two registers as data, a third register as a pattern register, and the fourth register as output.
It then, from this pattern, will choose arbitrarily the bytes that are in these two and reconstitute them into this fourth register.
It's wonderful for doing manipulations and shuffling things around.
Like shuffling a deck of cards, you could take all of these and ignore this or you could take the first one here, replicate it 16 times or you could take a random sampling from these, put into that register
Do you use that specifically for the [UNINTELLIGIBLE]?
We do use it, yeah.
Yeah, you take a look, you'll see we use shuffle a lot.
It's surprising how valuable shuffle can be.
However, then you have to worry now, you've got the shuffle here, if you're doing like matrix transpose, it's all shuffles.
But what's a date matrix transpose?
It's really bandwidth bound, right?
Because you're pulling data in, shuffling it around and sending it out.
Well, where's the reads and writes?
They're on the odd pipeline.
Where are the shuffles?
They're on the odd pipeline.
So now you can have a situation where it's all shuffle, shuffle, shuffle, shuffle and then the instruction pre-fetch buffer gets starved and so it stalls for 15, 17 cycles while I have to load.
Basically, it's a tiny little loop.
But you get stalls and you get really bad performance.
So then you have to tell the compiler-- actually, the compiler is getting better at these things.
Much better than it used to be or by hand you can force it to leave a slot for the pre-fetch.
These are gotchas that programmers have to be aware of.
On the other pipeline you have all your normal operations.
So you have your mul-adds, your bit operations, all the shift and things like that, they're all over there.
There is one other operation on the odd pipeline and I think it's a quad word rotate or something, but I don't remember.
So that's instruction level dual issue parallelism
[UNINTELLIGIBLE PHRASE]
Everything is in order on this processor, yeah.
And that was done for power reasons, right?
Get rid of all the space and all the transistors that are doing all this fancy, out of order processing to save power
[UNINTELLIGIBLE PHRASE]
That's a really good point.
When you're doing scalar processing you think well, you're thinking I'm going to-- kind of conceptually, you want to have all the things that are doing the same thing together right.
That's how I used to program.
You put all this stuff here then you do maybe all your reads or whatever and then you do all your computes and you can't do it that way.
You have to really think about how are you going to interlead these things.
Now the compiler will help you, but to get really high performance you have to have better tools and we don't have those tools yet.
And so I'm hoping that you guys are the ones that are going to come up with the new tools, the new ideas that are going to really help people improve programmability in cell.
Then at the lowest level you have the register level parallelism where you can have four single precision float ops going simultaneously.
So when you're programming cell you have to keep all of these levels of hierarchy in your head.
It's not straight scalar programming anymore.
And if you think of it that way you're just not going to get the performance that you're looking for period.
Another consideration is this local store.
Each little store is 256 kilobytes.
That's not a lot of space.
You have to think about how are you going to bring the data in so that the chunks are big enough, but not too big because if they're too big thing then you won't be able to get multibuffering.
Let's back up a little bit more.
The local store holds the data, but it also holds the code that you're running.
So if you have 200 kilobytes of code then you only have 56 kilobytes of data space.
And if you want to have double buffering that means you only have 25 kilobytes and then as Saman correctly points out there's a problem with stack.
So if you're going to have recursion in your code or something nasty like that, you're going to start pushing stack variables off the register file.
So where do they go?
They go in the local store.
What prevents the stack them overwriting your data?
Nothing.
Nothing at all and that's a big gotcha.
I've seen over the past three years maybe 30 separate algorithms implemented on cell and I know of only one that was definitely doing that.
But you know, if there are 30 in this class maybe you're going to be the one that that happens to.
So you have to be aware of that and you have to deal with it.
So what you can do, is in the local store put some dead beef thing in there so that you can look for an overwrite and that will let you know that either you have to make you code smalller or your data smaller or get rid of recursion.
On SPEs, recursion is kind of anathema.
Inlining is good.
Inlining really can accelerate your codes performance.
Oh yeah, it says stack right there.
You're reading ahead on me here.
Yes, so all three are in there and you have to be aware of that.
Now there is a memory management library, very lightweight library on the SPE and it's going to prevent your data from overwriting your code because once the code's loaded that memory management library knows where it is and it will stop.
The date you from allocating, doing a [?
mul-add.
?]
over this code.
But the stack's up for grabs.
And that was again done because of power considerations and real estate on the chip.
It you want to have a chip that's this big you can have anything you want, but manufacturing it's impossible.
So things were removed and that was one of the things that's removed and that's one of the things you have to watch out for.
And communication, we've talked about this quite a bit.
I didn't mention this: the DMA transactions-- oh, question in the back?
Is there any reasonable possibility of doing things dynamically?
Is it at all conceivable to have [?
bunks ?]
that fetch in new code or an allocator that shuffles somehow?
Or is it basically as soon as you get to that point your performance is going to go to hell
Yes, well if you don't do anything about it, yes your performance will go to hell.
So there are two ways.
In research we came up with an overlay mechanism.
So this is what people used to do 20 years ago when processors were simple.
Well, these processors are simple, so going back to the old technologies is actually a good thing to do.
So we had a video processing algorithm where we took video images, we had to decode them with one SPE, we had to do some background subtraction to the next SPE.
We had to do some edge detection.
And so each SPE was doing a different thing, but even then the code was very big, the chunks of code were large.
And we were spending 27% of the time swapping code out and bringing in new code.
Bad, very bad.
Oh, and I should tell you, spawning SPE threads is very painful.
500,000 cycles, a million cycles-- I don't know.
It varies depending on how the SPE feels that particular day.
And it's something to avoid.
You really want to spawn a thread and keep it running for a long time.
So context switching is painful on cell.
Using an overlay we got that 27% overhead down to 1%.
So yes, you can do that.
That tool is not in the SDK.
It's on my to-do list to put it in the SDK, but the compiler team at IBM tells me that the XLC compiler now does overlays.
But it only does overlays at the function level, so if the function still doesn't fit in the SPE you're dead in the water.
And I think the compiler will say, when it compiles it it'll say this doesn't fit quietly and you'll never see that until you run and it doesn't load and you don't know what's going on.
So read your compiler outputs.
The DMA granularity is 128 bytes.
This is the same, the data transactions for Intel, for AMD they're all 128 byte data envelopes.
So if you're doing a memory access that's 4 bytes you're still using 128 bytes of bandwidth.
So this comes back to this notion of getting a shopping list.
You really want to think ahead what you want to get, bring it in, then use it so that you don't waste bandwidth; if you're bandwidth bound.
If you're not than you can be a little more wasteful.
But there's a guy, Mike Acton-- you can find his website, I think he has a website called www.cellperformance.org?
Net?
Com?
I don't know
Just a quick comment [UNINTELLIGIBLE PHRASE]
Oh, he's good.
He's much better than me.
You're really going to like him.
His belief, and I believe him wholeheartedly, is it's all about the data.
We're coming to a point in computer science where the code doesn't matter as much as getting the data where you need it.
This is because of the latency out to main memory.
Memory's getting so far away that having all these cycles is not that useful anymore if you can't get the data.
So he always pushes this point, you have to get the data.
You have to think about the data, good code starts with the data, good code ends with the data, good data structure start with the data.
You have to think data, data, data.
And I can't emphasize that enough because it's really very, very true for this processor and I believe, for all the multicore processors you're going to be seeing.
The DMAs that you issue can be 128 bytes or multiples of 128 bytes, up to 16 kilobytes per single DMA.
There's also something called a DMA list, which is a list of DMAs in local store and you tell the DMA queue OK, here are these 100 DMAs, spawn them off.
That only takes one slot in the DMA queue so it's an efficient way of loading the queue without overloading the queue.
Traffic controls, this is perhaps one of the trickier things with cell because the simulator doesn't help very much and the tools don't help very much.
Thinking about synchronization, DMA latency handling-- all those things are important.
OK, so this is the last slide that I'm going to do and then I have to run off.
I want to give you a sense for the process by which people-- my group in particular went through, especially when we didn't even have hardware and we didn't have compilers that worked nearly as well as they do now and it's really very ugly knifes and stones and sticks.
You know, just kind of stone knifes.
That's what I'm thinking, very primitive.
But this way of thinking is still very much true.
You have to think about your code this way.
You want to start, you have your application, whatever it happens to be; you want to do an algorithmic complexity study.
Is this order n squared, is this log n?
Where are the bottlenecks?
What do I expect to be bottlenecks?
Then I want to do data layout/locality.
Now this is the data, data, data approach of Mike Acton.
You want to think about the data.
Where is it?
How can you structure your data so that it's going to be efficiently positioned for when you need it?
And then you start with an experimental petitioning of the algorithm.
You want to break it up between the pieces that you believe are scalar and remain scalar, leave those on the SPE and the ones that can be paralellized.
Those are the ones that are going to go on the SPE.
You have the think conceptually about partitioning that out.
And then run it on the PPE anyway.
You want to have a baseline there.
Then you have this PPE scalar code and PPE control code.
This PPE scalar code you want to then push down to the SPEs.
So now you're going to add stuff for communication, synchronization, and latency handling.
So you have the spawn threads.
The [?
RAIDs ?]
have to be told where the data is, they have to get their code, they have to run their code, they have to then start pulling in the data, synchronize with the other SPEs and then latency handling with multibuffering of the data so that you can be doing computing and reading data simultaneously.
Then you have your first parallel code that's running.
Now the compiler, the XLC compiler, GCC compiler-- well, the XLC compiler I know for certain will do some automatic SIMDization.
if you put the auto SIMD flag on.
Does GCC compiler do that?
[UNINTELLIGIBLE PHRASE]
OK, so I don't know if the GCC compiler does that.
So that can be done by hand, but sometimes that works, sometimes it doesn't.
And it really depends on how complex the algorithm.
If it's a very regular code, like a matrix-matrix multiply.
You'll see that the compiler can do fairly well if the block sizes are right and all.
But if you have something that's more irregular then you may find that doing it by hand is really required.
And so this step here could be done with the compiler initially to see if you're getting the performance that you think you should be getting from that algorithmic complexity study.
You should see that type of scaling.
You can look at the CPI and see how many cycles per instruction you're getting.
Each SPE should be getting 0.5.
You should be able to get two instructions per cycle.
Very few codes actually get exactly-- you can get down to 5.8 or something like that, but I think if you can get to 1 you're doing well.
If you get to 2 there's probably more you can be doing and if you're above 2 there's something wrong with your code.
It may be the algorithm.
It may be just a poorly chosen algorithm.
But that's where you can talk to me.
I want to make myself available to everyone in the class or in my department as well.
We're very enthusiastic about working with research groups in universities to develop new tools, new methods and if you can help me, I can help you.
I think it works very well.
Then once you've done this, you may find that what you originally thought for the complexity or the layout wasn't quite accurate, so you need to then go do some additional rebalancing.
Maybe change your block sizes.
You know, maybe you had 64 by 64 blocks, now you need 32 by 64 or 48 by whatever-- some readjustment to match what you have, And then you may want to reevaluate the data movement.
And then you know, in many cases you'll be done, but you're looking at your cycles per instruction or your speed up and you're not seeing exactly what you expected, so you can start looking at other optimization considerations.
Like using the vector unit, the VMX unit on the cell processor, on the PPE.
Looking for system bottlenecks and this is actually, I have found the biggest problem.
Trying to identify where the DMA bottlenecks are happening is kind of devilishly hard.
We don't have good tools for that, so you really have to think hard and come up with interesting kind of experiments for your code to track down these bottlenecks.
And then load balancing.
If you look at these SPEs, I told you they're completely independent.
You can have them all running the same code or they could be running all different code.
They could be daisy chained so that this one feeds, this one feeds that one, feeds that one.
If you do that daisy chaining you may find out there's a bottleneck.
That this SPE takes three times as long as any of the others.
So make that use 3 SPEs and have this SPE feed these 3.
So you have to do some load balancing and thinking about how many SPEs really need to be dedicated to each of the tasks.
Now that's the end of my talk.
I think that gives you a good sense of where we have been, where we are now, and where we're going.
And I hope that if was good, educational, and I'll make myself available to you guys in the future.
And if you have questions--
Thank you.
I know you have to catch a flight.
How much time do have for questions?
Not much.
I leave at 1:10.
So I should be there by 12:00
OK.
So [UNINTELLIGIBLE] at some time
My car is out--
OK, so we'll have about 5 minutes questions.
OK, so I know this talk is early.
We haven't gotten a lot of basics so there might be a lot of things kind of going above your head, but we'll slowly get back to it.
So questions?
You mentioned that SPEs would be able to run kernel.
Is there a microkernel that you could install on them so that you could begin experimenting with MPI type structures?
Not that I'm aware of.
We did look at something called MicroMPI, where we were using kind of a very watered down MPI implementation for the SPEs in the transactions.
I don't recommend it.
What I recommend is you have a cluster say, a thousand node cluster and the code today, the legacy code that's out there runs some process on this node.
Take that process, don't try to push MPI further down, but just try to subpartition that process and let the PPE handle all the communication off board, off node.
That's my recommendation
So MPI is running on [UNINTELLIGIBLE]?
Yeah, Open MPI.
It's an open source MPI.
It's just a recompile and it hasn't been tuned or optimized.
And it doesn't know anything about the SPEs.
You know, you let the PPE do all the communication or handle the communications.
When it finishes the task at hand then it can issue its MPI process
[UNINTELLIGIBLE PHRASE]
Open NP is the methodology where you take existing scalar code and you insert compiler pragmas to say this for loop can be parallelized.
And you know, this data structures are disjoint, so we don't have to worry about any kind of interference, side effects of the data manipulation.
The compiler, the XLC compiler implements open MP.
There's several components that are required.
One was a software cache where they implemented a little cache on the local store.
And if it misses in that local cache it goes and gets it.
I don't know how well that performs yet, but it exists.
There's the SIMDization.
For a while, Open NP wasn't working with auto SIMDization but now it does.
So it's getting there, for so C it's there.
I don't know what type of performance hit you take for that
Probably runs [UNINTELLIGIBLE PHRASE]
It's XLC version that does that.
I don't know if GCC does it.
But my recommendation is if you want to use open NP, go ahead, take your scalar code, implement it with those pragmas, see what type of improvement you get.
Play around with it a little.
If you find something that you expect should be 10x better and it's only 3x take that bottleneck and implement it by hand
[UNINTELLIGIBLE PHRASE] with the memory models and such that the SPEs certainly went back a couple of generations to a simpler [INAUDIBLE].
How come you went so far back rather to just say, segmentation
I don't know the answer.
I'm sorry.
I suspect and most of these answers come down to the same thing, it comes back to Sony.
Sony contracted with IBM, gave us a lot of money to make this thing.
And they said we need a Playstation 3.
We need this, this, this, this.
And so IBM was very focused on providing those things.
Now that that is delivered, Playstation 3 is being sold we're looking at other options.
And if that's something that you're interested in pursuing you should talk to me
Among other things it seems to me that the lightweight mechanism for keeping the stack from stomping on other things --
I think that this is very new area.
Before you put things in hardware, you need to have some kind of consensus, what's the right way to do it?
This is a bare metal that gives you huge amount of opportunity but you give enough rope to hang yourself.
And the key thing is you can get all this performance and what will happen perhaps, in the next few years is people come up to consensus saying, look, everybody has to do this.
Everybody needs MPI, everybody needs this cache.
And slowly, some of those features will do a little bit of a feature creep, so you're going to have they little bit of overhead, little bit less power efficient.
But it will be much easier to program.
But this is kind of the bare metal thing that to get and in some sense, it's a nice time because I think in 5 years if you look at cell you won't have this level of access.
You'll have all this nice build on top up in doing this so, this is a unique positioning there.
It's very hard to deal with, but also on the other hand you get to see underneath.
You get to see without any kind of these sort of things in there.
So my feeling is in a few years you'll get all those things put back.
When and if we figure out how to deal with things like segmentation on the multicore with very fine grain communication and there's a lot of issues here that you need to figure out.
But right now all those issues are [INAUDIBLE].
It's like OK, we don't know how to do it.
Well, you go figure it out OK?
Thank you very much
Thank you.
I don't have that much more material.
So I have about 10, 15 minutes.
Do you guys need a break or should we just go directly to the end?
How many people say we want a break?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we'll get started.
So today we are going to dive into some parallel architectures.
So the way, if you look at the big world, is there's -- just counting parallelism, you can do it implicitly, either by hardware or the compiler.
So the user won't see it.
It will be done behind the user's back, but can be done by hardware or compiler.
Or explicitly, visible to the user.
So the hardware part is done in superscalar processors, and all those things will have explicitly parallel architecture.
So what I am going to do is spend some time just talking about implicitly parallel superscalar processors.
Because probably the entire time you guys were born till now, this has been the mainstream, people are building these things, and we are use to it.
And now we are kind of doing a switch.
Then we'll go into explicit parallelism processors and kind of look at different types in there, and get a feel for the big picture.
So let's start at implicitly parallel superscalar processors.
So there are two types of superscalar processors.
One is what we call statically scheduled.
Those are kind of simpler ones, where you use compiler techniques to figure out where the parallelism is.
And what happens is the computer keeps executing, intead of one instruction at a time, the few instructions next to each other in one bunch.
Like a bundle after bundle type thing.
On the other hand, dynamically scheduled processors -- things like the current Pentiums -- are a lot more complicated.
They have to extract instruction level parallelism.
ILP doesn't mean integer linear programming, it's instruction level parallelism.
Schedule them as soon as operands become available, when the data is able to run these instructions.
Then there's just a bunch of things that get more about parallelism, things like rename registers to eliminate some dependences.
You execute things out of order.
If later instructions the operands become available early, you'll get those things done instead of waiting.
You can speculate to execute.
I'll go through a little bit in detail to kind of explain what these things might be.
Why is this not going down.
Oops.
So if you look at a normal pipeline.
So this is a 004 type pipeline.
What I have is a very simplistic four stage pipeline in there.
So a normal microprocessor, a single-issue, will do something like this.
And if you look at it, there's still a little bit of parallelism here.
Because you don't wait till the first thing finishes to go to the second thing.
If you look at a superscalar, you have something like this.
This is an in-order superscalar.
What happens is in every cycle instead of doing one, you fetch two, you decode two, you execute two, and so on and so forth.
In an out-of-order super-scalar, these are not going in these very nice boundaries.
You have a fetch unit that fetches like hundreds ahead, and it keeps issuing as soon as things are fetched and decoded to the execute unit.
And it's a lot more of a complex picture in there.
I'm not going to show too much of the picture there, because it's a very complicated thing.
So the first thing the processor has to do is, it has to look for true data dependences.
True data dependence says that this instruction in fact is using something produced by the previous guy.
So this is important because if the two instructions are data dependent, they cannot be executed simultaneously.
You to wait till the first guy finishes to get the second guy.
It cannot be completely overlapped, and you can't execute them in out-of-order.
You have to make sure the data comes in before you actually use it.
In computer architecture jargon, this is called a pipeline hazard.
And this is called a Read After Write hazard, or RAW hazard.
What that means is that the write has to finish before you can do the read.
In a microprocessor, people try very hard to minimize the time you have to wait to do that, and you really have to honor that.
In hardware/software what you have to do is you have to preserve this program ordering.
The program has to be executed sequentially, determined by the source program.
So if the source program says some order of doing things, you better -- if there's some reason for doing that, you better actually adhere to that order.
You can't go and just do things in a haphazard way.
And dependences are basically a fact of the program, so what you got.
If you're lucky you'll get a program without too many dependences, but most probably you'll get programs that have a lot of dependences.
That's normal.
There's a lot of importance of the data dependence.
It indicates the possibility of these hazards, how these dependences have to work.
And it determines the order in which the results might be calculated, because if you need the result of that to do the next, you have what you call a dependency chain.
And you have to excute that in that order.
And because of the dependency chain, it sets an upper bound of how much parallelism that can be possibly expected.
If you can say in all your program there's nothing dependent -- every instruction just can go any time -- then you can say the best computer will get done in one cycle, because everything can run.
But if you say the next instruction is dependent on the previous one, the next instruction is dependent on the previous one, you have a chain.
And no matter how good the hardware, you have to wait till that chain finishes.
And you don't get that much parallelism.
So the goal is to exploit parallelism by preserving the program order where it affects the outcome of the program.
So if we want to have a look and feel like the program is run on a nice single-issue machine that does one instruction after another after another, that's the world we are looking in.
And then we are doing all this underneath to kind of get performance, but give that abstraction.
So there are other dependences that we can do better.
There are two types of name dependences.
That means there's no real program use of data, but there are limited resources in the program.
And you have resource contentions.
So the two types of resources are registers and memory.
So linear resource contentions.
The first name dependence is what we call anti-dependence.
Anti-dependence means that -- what I need to do is, I want to write this register.
But in the previous instruction I'm actually reading the register.
Because I'm writing the next one, I'm not really using the value.
But I cannot write it until I have read that value.
Because the minute I write it, I lose the previous value.
And if I haven't used it, I'm out of luck.
So there might be a case that I have a register, that I'm reading the register and rewriting it some new value.
But I have to wait till the reading is done before I do this new write.
And that's called anti-dependence.
So what that means is we have to wait to run this instruction until this is all -- you can't do it all before that.
So this is called a Write After Read, as I said, in the architecture jargon.
The other dependences have what you call output dependence.
Two guys are writing the register, and then I'm reading it.
So I want to read the value the last guy wrote.
So if I reorder that, I get a wrong value.
Actually you can even do better in here.
How can you do better in here?
You can eliminate
Yeah.
You can elimiate the first one, because nobody's using that value.
So you can go even further and further, but this is also a hazard.
This is called a Write After Write hazard.
And the interesting thing is by doing what you call register renaming, you can eliminate these things.
So why do both have to use the same register?
In these two, if I use a different register I don't have that dependency.
And so a lot of times in software, and also in modern superscalar hardware, there's this huge amount of hardware resources that actually do register renaming.
So they realized that anti-dependence is output dependent, and said -- "Wait minute.
Why do I even have to do that?
I can use a different register."
So even though you have -- Intel basically [UNINTELLIGIBLE] accessory only have eight registers.
They are about 100 registers behind.
Hardware registers just basically let you do this reordering and renaming -- register renaming.
So the other type of depencence is control dependence.
So what that means is if you have a program like this, you have to preserve the program ordering.
And what that means is S1 is control dependent on p1.
Because depending on what p1 is, it will depend on this one to get excuted.
S2 is control dependent on p2, but not p1.
So it doesn't matter what p1 does, S2 will execute only if p2 is true.
So there's a control dependence in there.
Another interesting thing is control dependence may -- you don't need to preserve it all the time.
You might be able to do things out of this order.
Basically, what you can do is if you are willing to do more work, you can say -- "Well, I will do this.
I don't know that I really need it, because I don't know whether the p2 is true or not.
But I'll just keep doing it.
And then if I really wanted, I'll actually have the results ready for me."
And that's called speculative execution.
So you can do speculation.
You speculatively think that you will need something, and go do it.
Speculation provides you with a lot of increased ILP, because it can overcome control dependence by executing through branches, before even you know where the branch is going.
And a lot of times you can go through both directions, and say -- "Wait a minute, I don't know which way I'm going.
I'll do both sides."
And I know at least one side you are going, and that will be useful.
And you can go more and more, and soon you see that you are doing so much more work than actually will be useful.
So the first level of speculation is -- speculation basically says, you go, you fetch, issue, and execute everything.
You do the end of the thing without just committing your weight into the commit to make sure that the right thing actually happened.
So this is the full speculation.
There's a little bit of less speculation called dynamic scheduling.
If you look at a microprocessor, one of the biggest problems is the pipeline stall is a branch.
You can't keep even a pipeline going, even in a single-issue machine, if there's a branch, because the branch condition gets resolved.
Not after the next instruction has to get fetched.
So if you do a normal thing, you just have to reinstall the pipeline.
So what dynamic scheduling or a branch predictor sometimes does is, it will say I will predict where the branch is going.
So I might not have fed board direction, but I will speculatively go fetch down one path, because it looks like it which it's going.
For many times, like for example in a loop, 99% of the time you are going in the backage, because you don't go through that.
And then if you predict that you are mostly [UNINTELLIGIBLE].
So the branch predictors are pretty good at finding these kind of cases.
There are very few branches that are kind of 50-50.
Most branches have a preferred path.
If you find the preferred path you can go through that, and you don't pay any penalty.
The penalty is if you made a mistake, you had to kind of back up a few times.
So you can at least do in one direction.
Most hardware do that, even the simplest things do that.
But if you do good speculation you go both.
You say -- "Eh, there's a chance if I go down that path I'm going to lose a lot.
So I'll do that, too."
So that does a lot of expensive stuff.
And basically this is more for data flow model.
So as soon as data get available you don't think too much about control, you keep firing that.
So today's superscalar processors have huge amount of speculation.
You speculate on everything.
Branch prediction.
You assume all the branches, multilevel down you predict, and go that.
Value prediction.
You look at it and say -- "Hey, I think it's going to be two."
And in fact there's a paper that says about 80% of program values are zero.
And then you say -- "OK.
I'll think it's zero, and it'll go on.
And if it is not zero, I'll have to come back and do that."
So things like that
Do you know what percentage of the time it has to go back?
A lot of times I think it is probably an 80-20 type thing, but if you do too much you're always backing up.
But you can at least do a few things down assuming it's zero.
So things like that.
People, try to take advantage of the statistical nature of programs.
And you are mining every day.
So basically there's no -- it's almost at the entropy.
So every information is kind of taken advantage in the program, but what that means is you are wasting a lot of time cycles.
So the conventional wisdom was -- "You have Moore's slope.
You keep getting these transistors.
There's nothing to do with it, so let me do more other work.
We'll predicate, we'll do additional work, we'll go through multipe branches, we'll assume things are zero.
Because what's wasted?
Because it's extra work, if it is wrong we just give it up."
So that's the way it went, and the thing is it's very inefficient.
Because a lot of times you are doing -- think about even a simple cache.
If you have 4-way as a cache.
Every cycle when you're doing a memory fetch, you are fetching on all four, assuming one of it will have hit.
Even if you have a cache hit where only one bank is hit, and all the other three banks are not hit.
So you are just doing a lot more extra work just to get one thing.
Of course because if you wait to figure out which bank, it's going to add a little bit more delay.
So you won't do it parallelly.
You know that's it's going to be one of the lines, so you just go fetch everything and then later decide which one you want.
So things like that really waste energy.
And what has been happening in the last 10 years is you double the amount of transistors, and you add 5% more performance gain.
Because statistically you have mined most of the lower hanging fruit, there's nothing much left.
So you're getting to a point that has a little bit of a statistical significance, and you go after that.
So of course, most of the time it's wrong.
So this leads to this chart that actually yesterday I also pointed out.
So you are going from hot plate to nuclear reactor, to rocket nozzle.
We tend to be going in that direction.
That is the path, because we are just doing all these wasteful things.
And right now, the power consumption on processors is significant enough in both things like laptops -- because the battery's not getting faster -- as well as things like Google.
So doing this extra useless work is actually starting to impact.
So for example, if you look at something like Pentium.
You have 11 stages of instructions.
You can execute 3 x86 instructions per cycle.
So you're doing this huge superscalar thing, but something that had been creeping in lately is also some amount of explicit parallelism.
So they introduced things like MMX and SSE instructions.
They are explicit parallelism, visible to the user.
So it's not hiding trying to get parallelism.
So we have been slowly moving to this kind of model, saying if you want performance you have to do something manual.
So people who really cared about performance had to deal with that.
And of course, we put multiple chips together to build a multiprocessor -- it's not in a single chip -- that actually do parallel processing.
So for about three, four years if you buy a workstation it had two processors sitting in there.
So dual processor, quad processor machines came about, and people started using that.
So it's not like we are doing this shift abruptly, we have been going that direction.
For people who really cared about performance, actually had to deal with that and were actually using that.
OK.
So let's switch gears a little bit and do explicit parallelism.
So this is kind of where we are -- where we are today, where we are switching.
So basically, these are the machines that parallelism exposed to software -- either compiler.
So you might not see it as a user, but it exposes some layer of software.
And there are many different forms of it.
From very loosely coupled multiprocessors sitting on a board, or even sitting on multipe machines -- things like a cluster of workstations -- to very tightly coupled machines.
So we'll go through, and figure out what are all the flavors of these things
Excuse me
Mhmm?
So does it mean that since there being the level parallelism, the processor can exploit the fact that the compiler knows the higher level instructions?
Does that make any difference?
It goes both ways.
So what the processor knows is it know values for everything.
So it has full exact knowledge of what's going on.
Compiler is an abstraction.
In that sense, processor wins in those.
On the other hand, compile time doesn't affect the run time.
So the compiler has a much bigger view of the world.
Even the most aggressive processor can't look ahead more than 100 instructions.
On the other hand, the compiler sees ahead of millions of instructions.
And so the compiler has the ability to kind of get the big picture and do things -- global kind of things.
But on the other hand, it loses out when it doesn't have information.
Whereas when you do the hardware parallelism, you have full information
You don't have to give up one at the loss of the other
The thing is, I don't think we have a good way of combining both very well.
Because the thing is, sometimes global optimization needs local information, and that's not available at run time.
And global optimization is very costly, so you can't say -- "OK, I'm going to do it any time."
So I think it's kind of even hybrid things.
There's no nice mesh in there.
So if you think a little bit about parallelism, one interesting thing is this Little's Law.
Little's Law says parallelism is a multiplication of throughput vs. latency.
So the way to think about that is the parallelism is dictated by the program in some sense.
The program has a certain amount of parallelism.
So if you have a thing that has a lot of latency to get to the result, what that means is there's a certain amount of throughput you can satisfy.
Whereas if you have a thing that has a very low latency operation, you can go much wider.
So if you look at Intel processors, what they have done is the superscalars -- they have actually, to get things faster they have a very long latency.
Because they know they couldn't go more than three or four wide.
So they went like 55 the pipeline, three wide.
Because you can go fast, so they assume the parallelism fits here.
So still you need a lot of parallelism.
So you say -- "Three, why should [UNINTELLIGIBLE] issue machine.
[UNINTELLIGIBLE] three it's no big deal."
But no, if you have 55 the pipeline you need to have 165 parallel instructions on the fly any given time.
So that's the thing.
Even in the moder machine, there's about hundreds of instruction on the fly, because the pipeline is so large.
So if you said 3-issue, it's not that.
I mean this happens in there.
So this gives designers a lot of flexibiilty in where you are expanding.
And in some ways you can have a lot -- there are some machines that are a lot wider, but the latency is -- For example, if you look at an Itanium.
It's clock cycle is about half the time of the Pentium, because it has a lot less latency but a lot wider.
So you can do these kind of tradeoffs.
Types of parallelism.
There are four categorizations here.
So one categorization is, you have pipeline.
You do the same thing in a pipeline fashion.
So you do the same instruction.
You do a little bit, and you start another copy of another copy of another copy.
So you kind of pipeline the same thing down here.
Kind of a vector machine -- we'll go through categories that kind of fit in here.
Another category is data-level parallelism.
What that means is, in a given cycle you do the same thing many many many many -- the same instructions for many many things.
And then next cycle you do something different, stuff like that.
Thread-level parallelism breaks in the other way.
Thread-level parallelism says -- "I am not connecting the cycles, they are independent.
Each thread can go do something different."
And instruction-level parallelism is kind of a combination.
What you are doing is, you are doing cycle by cycle -- they are connected -- and each cycle you do some kind of a combination of operations.
So if you look at this closely.
So pipelining hits here.
Data parallel execution, things like SIMD execution hits here.
Thread-level parallelism.
Instruction-level parallelism.
So before a models of parallelism, what software people see kind of fits also in this architecture picture.
So when you are designing a parallel machine, what do you have to worry about?
The first thing is communication.
That's the begin -- the problem in here.
How do parallel operations communicate the data results?
Because it's not only an issue of bandwith, it's an issue of latency.
The thing about bandwith is that had been increasing by Moore's Law.
Latency, speed of light.
So as I pointed out, there's no Moore's Law on speed of light, and you have to deal with that.
Synchronization.
So when people do different things, how do you synchronize at some point?
Because you can't keep going on different paths, at some point you have to come together.
What's the cost?
What are the processes of going it?
Some stuff it's very explicit -- you have to deal with that.
Some machines it's built in, so every cycle you are synchronizing.
So sometimes it makes it easier for you, sometimes it makes it more inefficient.
So you have to figure out what is in here.
Resource management.
The thing about parallelism is you have a lot of things going on, and managing that is a very important issue.
Because sometimes if you put things in the wrong place, the cost of doing that might be much higher.
That really reduces the benefit of doing that.
And finally the scalability.
How do you build process that not only can do 2x parallelism, but can do thousand?
How can you keep growing with Moore's Law.
So there are some ways you can get really good numbers, small numbers, but as you go bigger and bigger you can't scale.
So here's a classic classification of parallel machines.
This has been [?
divided ?]
up by Mike Flynn in 1966.
So he came up with four ways of classifying a machine.
First he looked at how instruction and data is issued.
So one thing is single instruction, single data.
So there's single instruction given each cycle, and it affects single data.
This is your conventional uniprocessor.
Then came a SIMD machine -- single instruction, multiple data.
So what that means is the given instruction affects multiple data in here.
So things like -- there are two types, distributed memory and shared memory.
I'll go to this distinction later.
So there are a bunch of machines.
In the good old times this was a useful trick, because the sequencer -- or what ran the instructions -- was a pretty substantial piece of hardware.
So you build one of them and use it for many, many data.
Even today data in a Pentium if you are doing a SIMD instruction, you just issue one instruction, it affects multiple data, and you can get a nice reuse of instruction decoding.
Reduce the instruction bandwidth by doing SIMD.
Then you go to MIMD, which is Multiple Instruction, Multiple Data.
So we have multiple instruction streams each affecting its own data.
So each data streams, instruction streams separately.
So things like message passing machines, coherent and non-coherent shared memory.
I'll go into details of coherence and non-coherence later.
There are multiple categories within that too.
And then finally, there's kind of a misnomer, MISD.
There hasn't been a single machine.
It doesn't make sense to have multiple instructions work on single data.
So this classification, right now -- question?
I've heard that [INAUDIBLE]
Multiple instruction, single data?
I don't know.
You can try to fit something there just to have something, but it doesn't fit really well into this kind of thinking.
So I don't like that thinking.
I was thinking how should I do it, so I came up with a new way of classifying.
So what my classification is, what's the last thing you are sharing?
Because when you are running something, if it is some single machine, some thing has to be shared, and some things have to be separated.
So are you sharing instructions, are you sharing the sequencer, are you sharing the memory, are you sharing the network?
So this kind of fits many things nicely into this model.
So let's go through this model and see different things in there.
So let's look at shared instruction processors.
So there had been a lot of work in the good old days.
Did anybody know Goodyear actually made supercomputers?
Not only did they make tires, for a long time they were actually making processors.
GE made processors, stuff like that.
And so a long time ago this was a very interesting proposition, because there was a huge amount of hardware that has to be dedicated to doing the sequence and running the instruction.
So just to share that was a really interesting concept.
So people built machines that basically -- single instruction stream affecting multiple data in there.
I think very well-known machines are things like Thinking Machines CM-1, Maspar MP-1 -- which had 16,000 processors.
Small processors -- 4-bit processors, you can only do 4-bit computation.
And then every cycle you can do 16,000 of them, 4-bit things in here.
It really fits in to the kind of things they could build in hardware those days.
And there's one controller in there.
So it is just a neat thing, because you can do a lot of work if you actually can match it in that form.
So the way you look at that is, to run this array you have this array controller.
And then you have processing elements, a huge amount of them.
And you have each processor mainly had distributed memory -- each has its own memory.
And so given instruction, everybody did the same thing to memory or arithmetic in there.
And then you had also interconnect network, so you can actually send it.
A lot of these things have the [?
near-enabled ?]
communication.
You can send data you near enable, so everybody kind of shifts the 2-D or some kind of torus mapping in there.
And if you can program that, you can get really good performance in there.
And each cycle, it's very synchronous.
So each cycle everybody does the same thing -- go to the next thing, do the same thing.
So the next very interesting machine is this Cray-1.
I think this is one of the first successful supercomputers out there.
So here's the Cray-1, it is this kind of round seat type thing sitting in here.
Everybody know what was under the seat?
Cooling
Cooling.
So here's a photo.
I don't think you can see that -- you can probably look at it when I put this on the web -- was this entire cooling mechanism.
In fact Seymour Cray at one time said one of his most important innovations in this machine is how to cool the thing.
And this is a generation, again, that power was a big thing.
So each of these columns had this huge amount of boards going, and in the middle had all the wiring going.
So we had this huge mess of wiring in the middle -- [UNINTELLIGIBLE] -- and then you had all these boards in there in each of these.
So this is the Cray-1 processor
Do you know your little -- your laptop is way faster than that Cray --
Yeah.
Did you have the clock speed in here?
[INTERPOSING VOICES]
80 MHz
So, yeah.
And that cost like $10 million or something like that at that time.
Moore's Law, it's just amazing.
If you think if you apply Moore's Law to any other thing we have, it can't do the comparison.
We are very fortunate to be part of that generation
But did it have PowerPoint?
So what it had, was it had these three type of registers.
It had scalar registers, address registers, and vector registers.
The key thing there is the vector register.
So if you want to do things fast -- no, fast is not the word.
You can do a lot of computation in a short amount of time by using the vector registers.
So the way to look at that is normally when you go to the execute stage you do one thing.
In a vector register what happens is it got pipelined.
So execute state happened one word next, next, next.
You can do up to 64 or even bigger.
I think it was 64 length, length 64 things.
So you can -- so that instruction.
So you do a few of these, and then this state keeps going on and on and on, for 64.
And then you can pipeline in the way that you can start another one.
Actually, this will use the same executioner, so you have to wait till that finishes to start.
So you can pipeline to get a huge amount of things going through the pipeline.
And so each cycle you can graduate many, many things going on
Can I ask you a quick question?
Something I'm trying to get straight in my head.
My notion -- and I don't think I'm right on this, that's why I'm asking you -- is machines like the Cray, I know you were talking about some of the vector operations, those were by and large a relatively small set of operations.
Like dot products, and vector time scalar.
On the other hand, when you look at the SIMD machines, they had a much richer set of operations
I think with scatter-gather and things like conditional execution, I think vector machines could be a fairly large -- I mean it's painful
[INAUDIBLE]
The SIMD instruction is Pentium.
I think that is mainly targeting single processing type stuff.
They don't have real scatter-gather
And the Cell processor?
Cell is distributed memory
Yeah, but on one the -- what do they call them, the --
I don't think you can scatter-gather either.
It's just basically, you have to align words in, word out.
IBM is always about doing align.
So in even AltiVec, you can't even do unaligned access.
You had to do aligned access.
So if there is no run align, you had to pay a big penalty in there.
So if you look at how this happens.
So you have this entire pipeline thing.
When things get started the first value is at this point done in one clock cycle.
The next value is halfway through that.
Another value is in some part of a -- is also pipelined, the alias pipeline.
And other values are kind of feeding nicely into that.
So if you have one -- this is called one lane.
You can have multiple lanes, and then what you can do is each cycle you get 40 [UNINTELLIGIBLE] And the next ones are in the middle of that, next ones are in middle.
So what you have is a very pipelined machine, so you can kind of pipeline things in there.
So you can have either one lane, or multiple lanes pipeline coming out.
So if you look at the architecture, what you had is you have some kind of vector registers feeding into these kind of functional units.
So at a given time, in this one you might be able to get eight results out, because everything gets pipelined.
But the same thing is happening in there.
Clear how vector machines work?
So it's not really parallelism, it's basically -- especially if you are one -- it's a superpipelined thing.
But given one instruction, it will crank out many, many, many things for that instruction.
And doing parallelism is easy in here too, because it's the same thing happening to very regular data sets.
So there's no notion of asynchronizations and all these weird things.
It's just a very simple pattern.
So the next thing is the shared sequencer processor.
So here it's similar to the vector machines because each cycle you issue a single instruction.
But the instruction is a wide instruction.
It had multiple operations in these same instructions.
So what it says is -- "I have multiple execution units, I have memory in a separate unit, and each instruction I will tell each unit what to do."
And so something you might have -- two integer units, two memory/load store units, two floating-point units.
Each cycle you tell each of them what to do.
So you just kind of keep issuing an instruction that affects many of them.
So sometimes what happens is if this has latency of four, you might have to wait till this is done to do the next instruction.
So if one guy takes long, everybody has to kind of wait till that.
So it's very synchronous going.
So things like synchronization stuff were not an issue in here.
So if you look at a pipeline, this is what happens.
So you have this instruction.
It's an instruction, but you are fetching a wide instruction.
You are not researching a simple instruction.
You decode the entire thing, but you can decode it separately.
And then you go execute on each execution unit.
One interesting problem here was this was not really scalable.
What happened here is each functional unit, if you had one single register file, has to access the register file.
So each function would say -- "I am using register R1," "I am using R3," "I am using R5."
So what has to happen is the register file has to have -- basically, if you have eight functional units, 16 outports and 8 inports coming in.
And then of course, when you build a register file it has a scale, so it had huge scalability issues.
So it's a quadratically scalable register function.
Question?
The sequencer [INAUDIBLE PHRASE]
Yeah.
It's basically you had to wait till everybody's done, there's nothing going out of any order.
And memory also.
Since everybody's going to memory, this is not scalable.
So people try to build -- you can do four, eight wide, but beyond that this register and memory interconnect became a big mess to build.
And so one kind of modification thing people did was called Clustered VLIW.
So what happens is you have a very wide instruction in here.
It goes to not one cluster, but different clusters.
Each cluster has its own register file, its own kind of memory interconnect going on there.
And what that means is if you want to do intercluster communication, you have to go to a very special communication network.
So you don't have this bandwidth expansion register.
So you only have, we'll say two execution units, so you only have to have four out and one in to the register filing cycle.
And then if you want other communication, you have a much lower bandwidth interconnect that you'll have to go through that.
So what this does is you kind of expose more complexity to the compiler and software, and the rationale here is most programs have locality.
It's like everybody always wants to to communicate with everybody else, so there are some locality in here.
So you can basically cluster things that are local together and put it in here, and then when other things have to be communicated you can use this communication and go about doing that.
So this is kind of the state of the art in this technology.
And something like -- what I didn't put -- Itanium kind of fits in here.
Itanium processor.
So then we go to shared network.
There has been a lot of work in here.
People have been building multiprocessors for a long time, because it's a very easy thing to build.
So what you do is -- if you look at it, you have a processor unit that connects its own memory.
And it's like a multiple [UNINTELLIGIBLE] Then it has a very tightly connected network interface that goes to interconnect network.
So we can even think about a workstation farm as this type of a machine.
But of course, the network is a pretty slow one that requres an ethernet connector.
But people build things that have much faster networks in there.
This was designed in a way you can build hundreds and thousands of these things -- nodes in here.
So today if you look at the top 500 supercomputers, a bunch of them fits into this category because it's very easy to scale and build very large
Are you doing SMPs in this list, or some other place?
SMP is mostly shared memory, so shared network.
I'll do shared memory next.
But there are problems with it.
All the data layout has to be handled by software, or by the programmer basically.
If you want something outside your memory, you had to do very explicit communication.
Not only you, the other guy who has the data actually has to cooperate to send it to you.
And he needs to know that now you have the data.
All of that management is your problem.
And that makes programming these kind of things very difficult, which you'll probably figure out by the time you're done with Cell.
So Cell has a lot of these issues, too.
The problem here is not dealing with most of the data, but the kind of corner cases that you don't know about that much.
There's no nice safe way, of saying -- "I don't know where, who's going to access it.
I'll let the hardware take care of it."
There's no hardware, you have to take of it yourself.
And also message passing has a very high overhead.
Most of the time in order to do message, you have to invoke some kind of a kernel thing.
You have to actually do a kernel switch that will call the network -- it's operaing system involves a process, basically, to get a message in there.
And also when you get a message out you have to do some kind of interrupt or polling, and that's a bunch of copies out of kernel.
And this became a pretty expensive proposition.
So you can't send messages the size of one [UNINTELLIGIBLE] so you had to accumulate a huge amount of things to send out to amortize the cost of doing that.
Sending can be somewhat cheap, but receiving is a lot more expensive.
Because receiving you have to multiplex.
You have no idea who it's coming to.
So you have to receive, you have to figure out who is supposed to get it.
Especially if you are running multiple applications, it might be for someone's application.
You had to contact [UNINTELLIGIBLE] So it's a big mess.
That is where people went to shared memory processors, because it became easier message method to use.
So that is basically the SMPs Alan was talking about.
The nice thing is it will work with any data placement.
It might work very slowly, but at least it will work.
So it makes it very easy to take your existing application and first getting it working, because it's just working there.
You can choose to optimize only critical sections.
You can say -- "OK, this section I know it's very important.
I will do the right thing, I will place it properly everything."
And the rest of it I can just leave alone, and it will go and get the data and do it right.
You can run sequentially, of course, but at least the memory part I don't have to deal with it.
If some other memory just once in a while accesses that data that you have actually parallelized, it will actually work.
So you only have to worry about the [UNINTELLIGIBLE] that you are parallelizing.
And you can communicate using load store instructions.
You don't have to get always in order to do that.
And it's a lot lower overhead.
So 5 to 10 cycles, instead of hundreds to thousands cycles to do that.
And most of these messages actually stoplight some instructions to do this communication very fast.
There's a thing called fetch&op, and a thing called load linked/store conditional operations.
There are these very special operations where if you are waiting for somebody else, you can do it very fast.
So if two people are communicating.
So people came up with these very fast operations that are low cost, as a last -- if the data's available it will happen very fast.
Synchronization.
And when you are starting to build a large system, you can actually give a logically shared view of memory, but the underlying hardware can be still distributed memory.
So there's a thing called -- I will get into when you do synchronization -- directory-based cache coherence.
So you give a nice, simple view of memory.
But of course memory is really disbributed.
So that kind of gives the best of both worlds.
So you can keep scaling and build large machines, but the view is a very simple view of machines.
So there are two categories in here.
One is non-cache coherent, and then hardware cache coherence.
So non-cache coherence kind of gives a view of memory as a single address space.
But you had to deal with that if you write something to get there early to me, you had to explicitly say -- "Now send it to that person."
But we're still in a single address space.
It doesn't give the full benefits of a shared memory machine.
It's kind of inbetween distributed memory.
In distributed memory basically everybody's in a different address space, so you had to map by sending a message.
Here, you just say I have to flush and send it to the other guy.
Some of the early machines, as well as some big machines, were no hardware cache coherence.
Things like supercomputers were built in this way because it's very easy to build.
And the nice thing here is if you know your applications well, if you are running good parallel large applications, and you are actually knowing what the communication patterns are -- you can actually do it.
And you don't have to pay the hardware overhead to have this nice hardware support in there.
However, a lot of small scale machines -- for example, most people's workstations are stuffy, it's probably now two Pentium Quad machines -- actually add memory.
Because if you are trying to do the starting things it's much easier to do shared memory.
And also it's easier to bulid small shared memory machines.
And people talk about using a bus-based machine, and also using a large scale directory-based machine in here.
So for bus-based machines, how do you do shared memory?
So there's a protocol, what we call a snoopy cache protocol.
What that means is, every time you modify your location somewhere -- so of course you have that in your cache -- you tell everybody in the world who's using a busing, "I modified that."
And then if somebody else also has that memory location.
That person says, "Oops, he modified it."
Either he invalidates it or gets the modified copy.
If you are using something new, you have to go and snoop.
And you can ask everybody and say -- "Wait a minute, does anybody have a copy of this?"
And some more complicated protocols have saying -- "I don't have any, I have a copy but it's only read-only.
So I'm just reading it, I'm not modifying it."
Then multiple people can have the same copy, because everybody's reading and it's OK. And then there's the next thing -- "OK, I am actually trying to modify this thing."
And then only I can have the copy.
So some data you can give to multiple people as a read copy, and then when you are trying to write everybody gets disinvited, only the person who has write has access to it.
And there are a lot of complicated protocols how if you write it, and then somebody else wants to write it, how do you get to that person?
And of course you have to keep it consistent with memory.
So there is a lot of work in how to get these things all working, but that's the kind of basic idea.
So directory-based machines are very different.
In directory-based machines mainly there's a notion of a home node.
So everybody has local space in memory, you keep some part of your memory.
And of course you have a cache also.
So you have a notion that this memory belongs to you.
And every time I want to do something with that memory I had to ask you.
I had to get your permission.
"I want that memory, can you give it to me?"
And so there are two things.
That person has a directory [UNINTELLIGIBLE] say -- "OK, this memory is in me.
I am the one who right now owns it, and I have the copy."
Or it will say -- "You want to copy that memory to this other guy to write, and here is that person's address or that machine's name."
Or if multiple people have taken this copy and are reading it.
So when somebody asks me for a copy -- assume you ask to read this copy.
And if I have given it to nobody to read, or if I have given it to other people to read, so I say -- "OK, here's a copy.
Go read."
And I add that person is reading that, and I keep that in my directory.
Or if somebody's writing that.
I say sure, "I can't give it to read because somebody's writing that."
So I can do two things.
I can tell that person, saying -- "You have to get it from the person who's writing.
So go directly get it from there.
And I will mark that now you own it as a read value."
Or, I can tell the person who's writing -- "Look, you have to give up your write privilege.
If you have modified it, give me the data back."
And that person goes back to the read or no privileges with that data.
When I get that data, I'll send it back to this person and say -- "Here, you can read."
And the same thing if you ask for write permission.
If anybody has [UNINTELLIGIBLE] I have to tell everybody -- "Now you can't read it anymore.
Go invalidate, because somebody's about to write."
Get the invalidate request coming back, and then when you've done that I say, "OK, you can write that."
So everybody keeps part of the memory, and then all of that in there.
So because of that you can really scale this thing.
So if you look at a bus-based machine.
This is the kind of way it looks like.
You have a cache in here, microprocessor, central memory, and you have a bus in here.
And a lot of small machines, including most people's desktops, basically fit in this category.
And you have a snoopy bus in here.
So a little bit of a bigger machine, something like a Sun Starfire.
Basically it had four processors in the board, four caches, and had an interconnect that actually has multiple buses going.
So it can actually get a little bit of scalability, because here's the bottleneck.
The bus becomes the bottleneck.
Everybody has to go through the bus.
And so you actually get multiple buses to get bottleneck, and it actually had some distributed memory going through a crossbar here.
So this cache coherent protocol has to deal with that.
And going to the other extreme, something like SGI Origin.
In this machine there are two processors, and it had actually a little bit of processors and a lot of memory dealing with the directory.
So you keep the data, and you actually keep all the directory information in there -- in this.
And then it goes -- then after that it almost uses a normal message passing type network to communicate with that.
And they use the crane to connect networks, so we can have a very large machine built out of that.
So now let's switch to multicore processors.
If you look at the way we have been dealing with VLSI, every generation we are getting more and more transistors.
So at the beginning when you have enough transistors to deal with, people actually start dealing with bit-level parallelism.
So you didn't have -- you can do 16-bit, 32-bit machines.
You can do wider machines, because you have enough transistors.
Because at the beginning you have like 8-bit processors, 16-bit, 32-bit.
And then at some point that I have still more transistors, I start doing instruction-level parallelism in a die.
So even in a bit-level parallelism, in order to get 64-bit you had to actually have multiple chips.
So in this regime in order to get parallelism, you need to have multiple processors -- multiprocessors.
So in the good old days you actually built a processsor, things like a minicomputer.
Basically you had one processor dealing with a 1-bit slice.
So in the 4-bit slice, dealing with that amount, you could fit in a chip.
And a multichip made a single processor.
Here a multichip made a multiprocessor.
We are hitting a regime where a multichip -- what [?
it ?]
will be multiprocessor -- now fits in one piece of silicon, because you have more transistors.
So we are going into a time where multicore is basically multiple processors on a die -- on a chip.
So I showed this slide.
We are getting there, and it's getting pretty fast.
You had something like this, and suddenly we accelerated.
We added more and more cores on a die.
So I categorized multicores also the way I categorized them previously.
There are shared memory multicores.
Here are some examples.
Then there are shared network multicores.
Cell processor is one, and at MIT we are building also Raw processor.
And there is another part, what they call crippled or mini-cores.
So the reason in this graph you can have 512, is because it's not Pentium sized things sitting in there.
You are putting very simple small cores, and a huge amount of them.
So for some class replication, that's also useful.
So if you look at shared memory multicores, basically this is an evolution path for current processors.
So if you look at it, what they did was they took their years' worth of and billions of dollars worth of engineering building a single superscalar processor.
Then they slapped a few of them on the same die, and said -- "Hey, we've got a multicore."
And of course they were always doing shared memory at the network level.
They said -- "OK, I'll put the shared memory bus also into the same die, and I got a multicore."
So this is basically what all these things are all about.
So this is kind of gluing these things together, it's a first generation.
However, you didn't build a core completely from scratch.
You just kind of integrated what we had in multiple chips into one chip, and basically got that.
So to go a little bit beyond, I think you can do better.
So for example, this AMD multicore.
Basically you have CPUs in there, actually have a full snoopy controller in there, and can have some other interface with that.
So you can actually start building more and more uni CPU, thinking that you're building a multicore.
Instead of saying, "I had this thing in my shelf, I'm going to plop it here, and then kind of [INAUDIBLE] And you'll see, I think, a lot of interesting things happening.
Because now as they're connected closely in the same die, you can do more things than what you could do in a multiprocessor.
So in the last lecture we talked a little bit about what the future could be in this kind of regime.
Come on.
OK.
So one thing we have been doing at MIT for -- now this practice is ended, we started about eight years ago -- is to figure out when you have all the silicon, how can you build a multicore if you to start from scratch.
So we built this Raw processor where each -- we have 16, these small cores, identical ones in here.
And the interesting thing is what we said was, we have all this bandwidth.
It's not just going from pins to memory, we have all this bandwidth sitting next to each other.
So can we really take advantage of that to do a lot of communication?
And also the other thing is that to build something like a bus, you need a lot of long wires.
And it's really hard to build long wires.
So in Raw processor it's something like each chip, a large amount of part, is into this eight 32-bit buses.
So you have a huge amount of communication next to each other.
And we don't have any kind of global memory because that requires, right now, either do a directory, which you didn't want to build, or have a bus, which will require long wires.
So we did in a way that all wires -- no wires longer than one of the cores.
So we can do short wires, but we came up with a lot of communications for each of these, what we called tile those days, are very tightly coupled.
So this is kind of a direction where people perhaps might go, because now we have all this bandwidth in here.
And how would you take advantage of that bandwidth?
So this is a different way of looking at that.
And in some sense the Cell fits somewhere in this regime.
Because what Cell did was instead of -- it says, "I'm not building a bus, I am actually building a ring network.
I'm keeping distributed memory, and provide to Cell a ring."
I'm not going to go through Cell, because actually you had a full lecture the day before yesterday on this
Saman, can I ask you a question?
Is there a conclusion that I should be reaching in that I look at the multicores you can buy today are still by and large two and four processors.
There are people that have done more.
The Verano has 16 and the Dell has 8.
And the conclusion that I want to reach is that as an engineering tradeoff, if you throw away the shared memory you can add processors.
Is that a straightforward tradeoff?
I don't think it's a shared memory.
You can still have things like directory-based cache coherent things.
What's missing right now is what people have done is just basically took parts in their shelves, and kind of put it into the chip.
If you look at it, if you put two chips next to each other on a board, there's a certain amount of communication bandwidth going here.
And if you put those things into the same die, there's about five orders of magnitude possibility to communicate.
We haven't figured out how to take advantage of that.
In some sense, we can almost say I want to copy the entire cache from this machine to another machine in the cycle.
I don't think you even would want to do that, but you can have that level of huge amount of communication.
We are still kind of doing this evolutionary path in there [UNINTELLIGIBLE] but I don't think we know what cool things we can do with that.
There's a lot of opportunity in that in some sense
[INAUDIBLE]
Yeah, because the interesting thing is -- the way I would say it is, in the good old days parallelization sometimes was a scary prospect.
Because the minute you distribute data, if you don't do it right it's a lot slower than sequential execution.
Because your access time becomes so large, and you're basically dead in water.
In this kind of machine you don't have to.
There's so much bandwidth in here.
Latency was still -- latency would be better than going to the outside memory.
And we don't know how to take advantage of that bandwidth yet.
And my feeling is as we go about trying to rebuild from scratch multicore processors, we'll try to figure out different ways.
So for example, people are coming up with much more rich semantics for speculation and stuff like that, and we can take advantage of that.
So I think there's a lot of interesting hardware, microprocessor, and then kind of programming research now.
Because I don't think anybody had anything in there saying , "Here's how we would take it down to this bandwidth."
I think that'll happen.
Now the next [?
thing ?]
is these mini-cores.
So for example, this PicoChip has array of 322 processing elements.
They have 16-bit RISC, so it's not even a 32-bit.
Piddling little things, 3-way issue in.
And they had like 240 standard -- basically, nothing more than just a multiplier, and add in there.
64 memory tiles, full control, and 14 some special [UNINTELLIGIBLE] function accelerator.
So this is kind of what people call heterogeneous systems.
Where what this is -- you have all these cores, why do you make everything the same?
I can make something that's good doing graphics, something that's good doing networking.
So I can kind of customize in these things.
Because what we have in excess is silicon.
We don't have power in excess.
So in the future you can't assume everything is working all the time, because that will still create too much heat.
So you kind of say -- the best efficiencies, for each type of computation you have some few special purpose units.
So we kind of say if I'm doing graphics, I fit to my graphics optimized code.
So I will do that.
And the minute I want to do a little bit of arithmetic I'll switch to that.
And sometimes I am doing TCP, I'll switch to my TCP offload.
Stuff like that.
Can you do some kind of mixed in there?
The problem there is you need to understand what the mix is.
So we need to have a good understanding of what that mix is.
The advantage is it will be a lot more memory efficient.
So this is kind of going in that direction.
And so in some sense, if you want to communicate you have these special communication elements.
You have to go through that.
And the processor can do some work, and there are some memory elements.
So far and so forth.
So that's one push, people are pushing more for embedded very low power in
Is this starting to look more and more like FPGA, which is [UNINTELLIGIBLE]
Yeah, it's a kind of a combination.
Because the thing about FPGA is, it's just done 1-bit lot.
That doesn't make sense to do any arithmetic.
So this is saying -- "Ok, instead of 1 bit I am doing 16 bits."
Because then I can very efficiently build [UNINTELLIGIBLE] Because I don't have to build [UNINTELLIGIBLE] out of scratch.
So I think that an interesting convergence is happening.
Because what happened, I think, for a long time was things like architecture and programming languages, and stuff like that, kind of got stuck in a rut.
Because things there are so very efficiently and incremental -- it's like doing research in airplanes.
Things are so efficient, so complex.
Here AeroAstro can't build an airplane, because it's a $9 billion job to build a good airplane in there.
And it became like that.
Universities could not build it because if you want to build a superscalar it's, again, a $9 billion type endeavor to do that -- thousands of people, was very, very customized.
But now it's kind of hitting the end of the road.
Everbody's going back and saying -- "Jeez, what's the new thing?"
And I think there's a lot of opportunity to kind of figure out is there some radically different thing you can do.
So this is what I have for my first lecture.
Some conclusions basically.
I think for a lot of people who are programmers, there was a time that you never cared about what's under the hood.
You knew it was going to go fast, and in the air it will go faster.
I think that's kind of coming to an end.
And there's a lot of variations/choices in hardware, and I think software people should understand and know what they can choose in here.
And many have performance implications.
And if you know these things you will be able to get high performance of software built easy.
You can't do high performance software without knowing what it's running on.
However, there's a note of caution.
If you become too much attached to your hardware, we go back to the old days of assembly language programming.
So you say -- "I got every performance out of a -- now the Cell says you have seven SPEs."
So in two years, they come with 16 SPEs.
And what's going to happen?
Your thing is still working on seven SPEs very well, but it might not work on 16 SPEs, even with that.
But of course, you really customize for Cell too.
And I guarantee it will not run good with the Intel -- probably Quad, Xeon processor -- because it will be doing something very different.
And so there's this tension that's coming back again.
How to do something that is general, portable, malleable, and at the same time get good performance with hardware being exposed.
I don't think there's an answer for that.
And in this class we are going to go to one extreme.
We are going to go low level and really understand the hardware, and take advantage of that.
But at some point we have to probably come out of that and figure out how to be, again, high level.
And I think that these are open questions
Do you have any thoughts, and this may be unanswerable, but how could Cell really [INAUDIBLE].
And not Cell only, but some of these other ones that are out there today, given how hard they are to program
So I have this talk that I'm giving at all the places.
I said the third software crisis is due to multicore menace.
I termed it a menace, because it will create this thing that people will have to change.
Something has to change, something has to give.
I don't know who's going to give.
Either people will say -- "This is too complicated, I am happy with the current performance.
I will live for the next 20 years at today's level of performance."
I doubt that will happen.
The other end is saying -- "Jeez, you know I am going to learn parallel programming, and I will deal with locks and semaphores, and all those things.
And I am going to jump in there."
That's not going to happen either.
So there has to be something in the middle.
And the neat thing is, I don't think anybody knows what it is.
Being in industry, it makes them terrified, because they have no idea what's happening.
But in a university, it's a fun time.
[LAUGHTER]
Good question
OK.
So we'll take about a five minutes break, and switch gears into concurrent programming.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, the next part, today's going to be about concurrent programming.
So in this lecture we are going to study concurrent programing with the emphasis for correctness of programs.
Because parallel programs will have the same correctness issues.
So, if you want to get parallel, you'd better get the concurrency right first.
What we're also going do is start with a much simpler machine model.
In a program where we are going to use Java, because I think a lot of people understand Java.
Also, we are going to do some very simple shared [UNINTELLIGIBLE] machine abstraction.
I'm not going to even talk about any machine anymore.
I'm just going to talk about concurrent programming here.
You need to get through this one before you can start to dig in deep into the next level.
So in the next lecture, we will switch from Java to C-C , I guess using MPI primitives in here.
We'll start moving into parallelism with emphasis on performance.
And, of course, you have to get correctness, that's given, but we'll start looking at performance in there.
And we'll start using the distributive memory machine, all the notions and details of Cell, so we'll just kind of go down and down in that direction.
So, what's concurrency?
Sequential program is -- because sequential program opposite.
It's basically single thread of execution, with is a good one.
Finish that, go to the next, go to the next.
That's a very simple abstract model that for about 35 years, 40 years, none of the machines were actually following, that they had things in the back that actually had some parallelism.
A concurrent program is the [UNINTELLIGIBLE PHRASE] because it's a collection of autonomous sequential threads executing logically in parallel.
So you can execute this thing either multi-programming, so we can multiplex different parts on multiprocessing.
Well, multiprocessing basically has [UNINTELLIGIBLE] starting on different machines.
You can distribute, you can actually send it to different places.
Of course, you have to deal with memory issues.
So, concurrency's not only parallel systems.
So you can do interleaved concurrency.
You can have logically parallel, but you run Thread A for a while, contacts with Thread B for a while, contacts with Thread C, so you can have multiple threads on the same machine running.
Or you can actually have running parallel.
You can have three different machines running, A, B and C all the time.
So you can have both in there.
But logically you should not see a difference except for performance and stuff like that.
So what I'm going to do is do a bunch of examples.
Can you read this?
Let's start with a bank.
So you have a bank account.
So in Java you just basically have ID, password, and balance, and you have some way to construct this object in here.
And you can ask, and see the password is correct.
You can get the balance.
And you can post the balance.
So that's a very simple account object.
If you have a bank, you have a bunch of accounts in a hash map, and you create the hash map in here.
Then you can basically [?
figure out ?]
the bank, you actually create a bank in here, and you can get an account, given an ID.
Now, assume you want to build an ATM.
How do you build an ATM?
So, you have a bank -- you need a bank in here, and here's some input and output streams in here.
When you start the ATM, you will set up these input and output streams in here.
In the main function, what you'll do is, you get a bank, create where the input streams are coming from.
Create output goes standard -- system output goes there.
And create an ATM in here, and you will make the ATM run.
So how do you run the ATM?
So, what happens in run is, you run forever?
ATM doesn't stop any time.
What you can do is you can ask when somebody walks into the ATM, you can say what's the account ID.
Type the account ID.
You can get that account, so of course, if the account already is wrong it says, throw exception.
You can say OK, what's the password, get the password.
You take the password, if it's wrong you throw exception.
Then you can say, here's your balance today.
What do you want to do?
If you want to withdraw or deposit.
If you want to withdraw you can do a minus number, if you want to deposit it will be a plus number.
Then you can post that into your balance.
Everybody got the thing for ATMs?
So, assume activity trace.
So somebody comes and gives the account ID, at least gives the password, and say that they have $100,000 and say how much you withdraw, $200 withdraw.
And you get the balance in here.
Looks nice.
It works.
So I need to run multiple ATMs.
Assume I am in a place that I actually want to put two ATMs or four ATMs next to each other.
So how am I going to do that?
So, in order to do that, there's concurrency in Java.
So one way to get Java concurrency is you can extend this class thread and define a method run.
Or you have interface called [?
Runabout, ?]
that you can basically use that interface and has estimated run.
Then when you have made that run and when [UNINTELLIGIBLE PHRASE] start, that will get started.
Very simple way to do that.
Let me give you an example.
Little bit of a digression.
Why do you want concurrent programming?
A lot of times, natural application structure is not sequential.
The world is not sequential.
And then try to sequentialize the world sometimes means it's much more complicated [UNINTELLIGIBLE] So sometimes it's natural to do things in parallel.
A lot of times the sequentiality's an artifact of the programming language, because we use a language like that.
Sometimes doing things in parallel ways, you can really improve things like throughput and responsiveness.
If you are doing IO, if you're doing sequential programming [UNINTELLIGIBLE PHRASE] you're just twiddling your thumb waiting for the IO to come back.
In parallel things actually, you can do parallel IO and you can do a lot cool stuff in here.
Of course, in this class, if you are multicore and multiprocessor multicore, you can get parallel executions.
So there are more than one [UNINTELLIGIBLE PHRASE].
Also, if you are building a very large distributed system, concurrent programming is, you had to deal with, especially dealing with things like client-server type of applications.
So here's our original ATMs.
So to go to multiple ATMs, I am doing a few changes.
I'll go back and forth a few times.
So the first thing I have done is I have sett here number of ATMs to be four.
Can you really read it from back there?
[INAUDIBLE PHRASE]
OK, good.
Then what I have done is, in here, I did four ATMs here, and then I put it in a loop to create this ATM, so I created four ATMs in here and start four ATMs, basically.
Then of course I extended these ATMs so now we will extend [?
up a thread.
?]
And I haven't started that.
And the ATMs [UNINTELLIGIBLE] ATMs, so it's great.
So now what happens is I assume now there's two guys going, both ATMs, at least [UNINTELLIGIBLE] been.
Then enter the account and [UNINTELLIGIBLE PHRASE], that's works really well.
No problem.
So we have two ATMs, two people actually went on parallel.
One then deposited some money, other one took money, great.
Now, as MIT students, they want to do something, they can hack it.
So, [UNINTELLIGIBLE] basically [UNINTELLIGIBLE] went [UNINTELLIGIBLE], and basically what [UNINTELLIGIBLE PHRASE] enter the password, and they said I want to get $100.
I would get $90, basically.
So he had $100 in his account.
Then what he got was, so he actually managed to get $180 out of an account that had $100.
This is not a good ATM, at least from the bank's perspective.
So what went wrong?
If you look at what happened in activity trace, so we print 100 in here.
And then you said, you want to read this value, you both entered 90, right here.
And this account balance [UNINTELLIGIBLE PHRASE] because the account balance was 100.
You saw also the account balance [UNINTELLIGIBLE PHRASE], yes, it is [UNINTELLIGIBLE].
Then each went post, it went to 10 -- this also did a post of the same time, result came both 10.
How could this happen?
So that way it can happen is, so in the ATM, the [UNINTELLIGIBLE PHRASE], what happens is v is minus 90, and this post [UNINTELLIGIBLE] also when you start a v it's minus 90.
Then you treat the balance as 100.
So in this interleaving, and so it is the plus, you get 10.
Also, before you write it out, you read the balance in the other interleaving, you've got the balance as 100, and you do the plus as 10.
So it destroyed the balance, now balance became 10, and also this guy also wrote the balance -- it doesn't matter, it got 10 updated twice, and that's it.
So you can have interleaving in here, that actually did something that's not a signature program.
And you're in big trouble.
So in order to get out of that, problem is all interleaving of threads are not acceptable and current.
What you want is some kind of a sequential-looking performance, even though you'd get parallel, you don't want to do all these interleavings in here.
So in order to do that, Java provides this synchronization mechanism.
That's just strict interleaving.
So, what synchronizations do is, they ensure safety for shared updates.
So if you're sharing something, so it avoids races, basically.
It avoids this old interleaving ordering here.
Also, it allows you to coordinate actions among shared space, basically.
Because at some point people have to coordinate and take that parallel computation.
With notification you can do that.
So, when multiple threads access the shared resource, simultaneously, it's safe only if all accesses have no effect on the resource.
Basically, we're reading variables.
But everybody can read the same variable, because you're not changing anything.
I can do that.
Or all accesses are idempotent.
So you can say, we can do that.
Or only one access at a time.
Which is called mutual exclusion.
So in this case we are changing something.
It's not that important.
So we have to actually do mutual exclusion.
So here's a way to look at safety problems.
Here might be an algorithm that you and your roommate have.
So you arrive home, look in the fridge, no milk.
Leave for grocery, arrive at grocery, buy milk and arrive at home.
The minute you leave for grocery, your roommate arrives and do this.
Then what do you have is you have too much milk.
So here's the problem in a little bit more abstract sense.
And you need a way to synchronize this.
So how about this.
No milk and no note.
So you leave a note before you actually leave the house and buy milk, and then you come back and remove the note.
Does this work?
[INAUDIBLE PHRASE]
I mean here also you can do that, no milk and no note.
So both are started.
We would leave a note, and these things can happen at the same time.
There's a little bit of things saying OK, why didn't you see your roommate.
They go buy milk, he goes to buy milk and you have too much milk too.
So the way to do this in Java is this notion of critical section.
Critical section is where only one thread can be in it at a given time.
The way you can do it with Java is, you can put synchronized in front of the method.
And you do that method, only one person can be executing that method at any one time.
So in here I would say get balance and post so you can synchronize.
So when you do that, what happens is -- so in here you read.
No problem, you can do this parallel.
You can do this in parallel.
And then you say, first take out post in here.
And this [?
takes out ?]
post.
Because of synchronization these things can't have an order, because this has to happen in some order.
Either this happens first and this has to finish before this one.
At that point you can actually -- what happened now?
Are we happy?
[INAUDIBLE]
Yeah.
At least banks realize -- bank's book is correct.
Because it realizes, here is more money.
But actually it let you take more money than your account had.
So at least it got that value right.
But what happened was, why is this happening now?
You want the check covered
OK, you want to check also.
Good.
So the key thing is, here we didn't check.
So you have a negative bank balance happening.
So this is a problem with atomacity.
Because synchronized methods execute the body at atomic units.
So when that happens, the entire thing of body happens without anybody else [?
modifying.
?]
That's the only thing that's happening at any given time.
The code read [UNINTELLIGIBLE] you chose is probably too small in this case.
What we need to do is, we need to basically have synchronizing not on the method but a lot more [UNINTELLIGIBLE] in there, so you have to do block synchronization.
So synchronized keywords actually work like this too.
You can say instead of doing a method, you can just synchronize account and all those things happen synchronously within that block.
So, now what we have done is we have built a bigger atomic unit.
So here's the programming here.
So here's the synchronized unit in here.
So what we did was, we say instead of these synchronized and these synchronize separately, both of these computations have to happen atomically.
So when I check our bank balance, we can't do anything else.
So now what happens?
So yeah, in this situation you're reading, reading, and you get synchronized account in here, and I do account balance plus [?
well ?]
and post the account.
So in here I go to 10, I do that.
If I start the other one here, I have to wait till that entire synchronization is over before I do that.
Of course, I don't have enough balance, so I throw exception.
Are we still happy?
Is there issue on this one?
I mean, I guess -- assume you can do something clever, but I haven't done that.
But there's one issue in here.
Which, when you start it's just, balance is 100.
So in this one, say balance is 100.
You go type it and then voila, you type it and then, sorry, I don't have money.
So that's not nice, because if you've got the balance you should be able to get that.
So how we deal with that?
[INAUDIBLE PHRASE]
So that's probably the best solution to that because we can only log into one.
But in this example I assume what we are doing.
How can we deal with this one?
There are two ways of doing it.
One is to put the whole thing [INAUDIBLE PHRASE] section.
The other way is to notify somebody that the [UNINTELLIGIBLE PHRASE]
[UNINTELLIGIBLE PHRASE].
So what I can do is I can say OK, wait a minute.
I am actually going to make the critical section even bigger.
So now I print the balance before I do that.
So the entire thing is critical section.
I print the balance off and then go ahead and withdraw that.
So what might happen in this case?
[INAUDIBLE PHRASE]
Yeah.
That's the issue of a little bit of waiting.
So what happens is, in here.
You do this one, and you do synchronized account.
And you put the balance and other one do synchronized and you ask the question.
In here.
And you start thinking.
We can start thinking that my machine is not responsive, it's just waiting for the critical section started.
[UNINTELLIGIBLE] and you have this [?
IOUN ?]
sitting in the middle.
That's not good either.
So he has a performance issue.
So that's not a good way of doing that.
So you don't get any response in here.
So you can make this atomic [?
radius ?]
but there's a price you'll pay by making it [UNINTELLIGIBLE PHRASE] large.
So here's another thing we want to do.
I want to do something that can transfer account balance from one account to another.
So I might do that if I have a method in here to transfer [UNINTELLIGIBLE] account, this amount.
So what I do is, I synchronize from account.
I say, I get balance in here.
If the balance is available, I can synchronize the two accounts and force it there.
See any problems?
So let's see what happenes.
So assume I want to transfer 10 to Ben's account and Ben wants to transfer 20 to Alyssa's account.
So what happens is, this goes -- get the value in here, and you synchronize to two and say OK, great.
Now what happens is, in here, in from, I am holding a Alyssa's account.
There I am holding Ben's account.
Now, inside I want to synchronize for Alyssa.
And I'm still [UNINTELLIGIBLE] when I -- wait until Alyssa got released.
And he says I want to wait till Ben got released.
And nobody's going to release, and you're hung.
You are in what you call a deadlock situation.
That's a deadlock.
So you have to be very careful when you do synchronizing.
If you do multiple synchronization, the easiest thing you can do is, you do it in some order.
And end up in a a deadlock situation.
This is a very common way of parallel programs doing that.
So how to avoid deadlock?
Because deadlock is, there's a cycle in locking graph.
So somebody's going to lock somebody, he's going to lock that person, and we have a cycle.
You can end up in deadlock situation.
So standard solution for that is, you take locks in some kind of canonical order.
You don't take in arbitary order.
So it's some kind of a -- you have some base in, OK, if you are taking this lock, you have to have, after that, you can take a higher order lock.
So you have to have some kind of order in here.
Acquire in increasing order and release in decreasing order.
So you have some kind of force in here.
This ensures deadlock freedom most of the time, but it's not that easy to do a lot of the time.
Because your program might not fit into this nice ordering a lot of times, and then sometimes you realize that you had locked something and at that time it's too late when you realize it.
And then it has a different order.
So this is, you have to sometime do some changes to basically make the program work like this.
So in here, what you can do is, in the program you can associate some kind of a rank, and when you put in account, you put the rank to the account number.
So you have some kind of ordering in here.
Then what you have is, you always get the first, highest rank one before you go the next one.
So there's some ordering in here.
So at least then we'll be at least forced into some ordering in here
Is there a way of [UNINTELLIGIBLE PHRASE] deadlock [INAUDIBLE PHRASE]
Not statically.
Because most of the time that means you have to know all the possible control profile, to do that.
And, for example, there are some tools that can -- because you might know that, for example, assume you are trying to enforce some ordering of locks.
But it's not the software, it's the locking software, that doesn't know about those.
You can actually write a locking software that will tell you, like look, you are trying to acquire locking out of order, out of this locking order.
Most of the time you might be OK because it might not hit, but [?
we assume ?]
that if you are doing unsafe thing that might work, so.
So you can put some dynamic checks that might warn you that you might be in a situation, but it doesn't guarantee you.
So deadlock is something, you have to basically -- there's no nice tools for.
Basically, it's almost a software [?
methodology.
?]
So, for example, you can impose a software methodology to say, I'm following this convention and that will guarantee me deadlock freedom.
So one good convention is this, basically some order in here.
So, another interesting thing and hard thing is race conditions.
These are non-deterministic timing dependent, and cause data corruption, crashes that are impossible to detect.
So the problem with race conditions is the minute you put your debug, or put any debugging things, race conditions goes away.
It comes back when you are in it all and you're debugging [UNINTELLIGIBLE PHRASE].
It happens again because it's basically an independent thing.
In fact, I have this interesting experience with myself.
A long time ago I was working at Microsoft and I worked two summers.
In one summer I was working on their LAN manager and network manager, and there's a bug that after you run the network manager for some time it just freezes.
That's not a nice behavior to have if you are running your network.
That bug lasted the entire year.
And at the end they had, I think, a $2,000 bounty on that bug.
Because the minute you do any instrumentation, the [?
bug isn't ?]
[UNINTELLIGIBLE] When you have more instrumentation and have 100 machines running, heavily, hitting another machine.
Once in a while voila.
It freezes.
And you have no idea why it happened.
That was so hard to debug because there's nothing you could do, because any time you do any changes, the bug goes away.
You had to be very careful because these things are not easy to find, and happen intermittently.
And very hard to debug.
So having good discipline and good design really helps to get rid of it.
These are not something you can go through like program debugs, it [UNINTELLIGIBLE] cycle.
If you read that cycle, it's a very slow cycle.
The best way to do that is get the design right first.
So what's a data race?
So I assume I had this program like that.
So I read [?
hit ?]
in there, and then I modify and write in this.
This doesn't have to be in two statements.
If we [UNINTELLIGIBLE] same statement, the compiler might put it in register, read, update and modify and write.
So it might just look hits equals hits plus 1 and hits equals hits plus 1 on the cycle.
Doesn't have to [?
have temporary.
?]
and in your call.
Because the compiler puts a [?
temporary ?]
in there.
And if you execute like this you're happy.
But if you get excluded in this order, I don't get at it two times, I only get it because I read hit the order given values.
This adds once and writes.
And this also adds one to [?
the ordinary value ?]
and write.
So I only get it increased by one and you are in a bad situation.
The problems with data races is this non-determinism.
We ensured [UNINTELLIGIBLE] that this mutual exclusion.
So if you have same data access, make sure that they are in the mutual exclude region.
You can basically see that it has access to old objects.
Before you go there, one interesting thing is this is just a problem with all parallel programs.
So at the beginning you say OK, I'm going to have this nice mutual excluded, lock ordered program.
You write this.
It worked correctly, beautifully, but run dog slow because now we are huge critical sections.
Everybody's waiting in data and then someone says I want to run fast.
I think I don't need this lock.
It doesn't seem to be, so keep removing locks, making critical sections smaller and stuff like that.
That's where all the problems start cropping up, because all this nice design goes to the dogs when you have performance issues.
So when you realize that, you want to write this nice program, nice large critical sections, stuff like that.
The programs will work correctly.
But run like a dog because now it's sequential in many cases because you are doing this.
Then you go and say OK, I want to run parallel.
Eh, this is OK. That's when problems start creeping up.
So make sure that when you get a discipline, as you can go into the performance improvement but you still maintain at least some part of discipline.
That's the hard thing.
So I want to switch gears a little bit to talk about a classic problem.
It's called dining philosophers problem.
So, there are five philosophers sitting around a table.
Between each of the philosophers there's a chopstick.
So each philosopher do two things.
He thinks -- he or she thinks or he or she eats.
So the philosopher thinks for a while.
And then the philosopher is hungry.
She stops thinking and she picks up a left and right chopstick, eats, and puts the chopsticks down.
He cannot eat until they have both chopsticks right in hand because you can't eat with one chopstick.
So you have to wait until you get both chopsticks.
When you are done, you put the chopsticks down.
Then after you're done, you go back to thinking again for a while and come back to eating.
That's the classic problem.
So how to write that, record that?
You can have philosopher extend thread, and [UNINTELLIGIBLE PHRASE] philosopher you have a chopstick in here, and instead of philosopher buy left and right chopstick.
Then what you do is you create a number of philosophers and you get a new chopstick and start to the left and you go to the other philosophers assigning left and right chopsticks in here, and then you start the philosophers going.
So you just set up a chopstick [?
on it, ?]
and then you share the chopstick and do that.
So here is what a [UNINTELLIGIBLE] philosopher does.
So I am here, I'm taking my left chopstick, I'm taking my right chopstick and I'm going to eat and I'm done eating and I'm putting down there.
What will happen in this one?
[INAUDIBLE PHRASE]
In what situation, [UNINTELLIGIBLE PHRASE].
[UNINTELLIGIBLE PHRASE], but right technical though is different
[INAUDIBLE PHRASE]
You end up in a deadlock because [UNINTELLIGIBLE PHRASE] we will pick up the left chopstick suddenly, and they all try to take the right chopstick.
There's no right chopstick and nobody has right chopstick and everybody waiting for somebody to drop the chopstick, that's not going to happen.
So you have a problem.
Second way to solve that is this, and you say OK.
The problem is everybody trying to pick up this chopstick.
I will put unique variable table, unique object table.
If anybody want to eat, I need to own the table.
What will this do?
[INAUDIBLE PHRASE]
It prevents two people who wouldn't normally interact from eating at the same table
Yes.
So what happens is only one person can eat at a time.
Which works perfectly, beautifully, sequential.
So, you wonder if one philosopher eating, the person or [?
posit ?]
can eat.
But you're not allowed to because the chopstick in there.
So one way of doing that is sequentialized large regions, with putting these critical sections in there.
This works.
Not greatly, but it will work.
Another thing is, of course, what I point out to have some kind of ordering.
So you put some position ordering and saying if you are sitting in even position, you're the first to pick the left one, if you are putting an odd position, you're supposed to pick the right one.
So in some sense, it got [UNINTELLIGIBLE PHRASE] go here, the person go here, so only one can get that so you don't have ordering.
At least between those two you can maintain that.
So you can do something but you have to figure out what the right ordering in here.
This is not a linear list, linear ordering for this circuit.
But you can copy ordering and say OK, look, if you do that this new way, you can run into a deadlock situation.
There are a lot of types of synchronizations in Java, and then tomorrow you learn more different type of synchronization with available using [UNINTELLIGIBLE PHRASE], so using MPI.
But there are a lot of potential problems you are worried about.
Deadlock you have to worry about.
Two or more threads stop, wait for each other forever.
Livelock.
What livelock means is two or more threads basically trying to do something but never made progress.
So good example.
So assume I go -- it's like sometimes you try to cross each other on the road and you go into them and say oops, or you both say oops, sorry, [UNINTELLIGIBLE] You get into a situation that you try to go something, both you start to move a little bit and then do that and you keep doing that forever and ever, doing it, right.
So that can happen.
If you program right, you can actually try to avoid deadlock by doing that, but both no one making forward progress, so that's called livelock.
So another thing that's called starvation.
So the ordering is a very good example.
So ordering says the higher order guy always gets the lock for the lower guy.
So assume you have thousands of things and everybody's trying to do something.
If you have an lower number, you probably never get to around to get picked up because always if higher order person has, that person will get the lock.
So if they're ordering constraint in there, you can be in situation that some people always get and others never get -- there's no fairness in that, because when you some ordering constraints.
So again, lack of fairness.
Of course, race conditions.
So you didn't realize that the same object is accessed by multiple people without being in a particular section.
That's the key -- I mean don't try to do fancy things by letting multiple people have access to same thing.
This is not much of issue on distributed memory machines because there's only you access to your memory.
But the problem there is if you keep values, you suddenly start giving it to everybody and say go play, assuming that only one person have access to it.
So multiple people might be modifying it and then what are you going to do.
So that issue is there.
So when you are doing, using data, you got to be very careful who holds it and at what time.
So, concurrency and parallelism are important concepts in comparison beyond what we are doing in here.
Concurrency can simplify programming beyond anything.
It's very hard to understand and debug concurrent programs.
That's the entire reason that we are still doing sequential programming and this is entire reason that multiple people are looking at it in a very -- people are scared because writing and getting concurrent program right is probably an order of magnitude harder than trying to get sequential programs right.
This is issue.
Parallelism is critical for high performance.
I mean, it was huge for supercomputers in national labs and now it's becoming everybody's issue because of multicore.
Basically, you need to understand concurrent and concurrence issues, it's the basis of writing parallel programs.
So, you will run into all these issues, deadlock, you can deadlock on limited access on Cell, you can deadlock on messages.
So everybody is waiting for somebody else to send you a message and nobody's sending a message because that other guy will send you a message.
You can [?
easier ?]
do that in a message in there, and a lot of times you can deadlock in that.
So this lecture we kind of did concurrent programming, how to write a concurrent program.
We are going to switch gears and start going into parallelism next.
But keep these issues in my mind when you are writing parallel programs.
Have things like 617 we had very good discipline on testing and methodology of development.
You probably won't have kind of discipline on how to do parallelism.
So there are many ways -- next few lectures we'll cover many different ways of doing parallelism.
Parallelism's a very powerful tool, but if you don't use it in a disciplined way, you will not be able to debug these [UNINTELLIGIBLE] I mean you run into bugs that are so subtle, so difficult it's very hard to find.
You don't want in that situation.
So having a good design, good disciplining programming will actually get you working correct program.
Good.
That's all I have for today.
You can spend some time filling out this one.
Just put your name down and you're done.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.
mit.edu
So last week, last few lectures, you heard about parallel architectures and started with lecture four on discussions of concurrency.
How do you take applications or independent actors that want to operate on the same data and make them run safely together?
And so just recapping the last two lectures, you saw really two primary classes of architectures.
Although Saman talked about a few more.
There was the class of shared memory processors, you know, the multicores that Intel, AMD, and PowerPC, for example, have today, where you have one copy of the data.
And that's really shared among all the different processors because they essentially share the same memory.
And you need things like atomicity and synchronization to be able to make sure that the sharing is done properly so that you don't get into data race situations where multiple processors try to update the same data element and you end up with erroneous results.
You also heard about distributed memory processors.
So an example of that might be the Cell, loosely said, where you have cores that primarily access their own local memory.
And while you can have a single global memory address space, to get data from memory you essentially have to communicate with the different processors to explicitly fetch data in and out.
So things like data distribution, where the data is, and what your communication pattern is like affect your performance.
So what I'm going to talk about in today's lecture is programming these two different kinds of architectures, shared memory processors and distributed memory processors, and present you with some concepts for commonly programming these machines.
So in shared memory processors, you have, say, n processors, 1 to n. And they're connected to a single memory.
And if one processor asks for the value stored at address X, everybody knows where it'll go look.
Because there's only one address X.
And so different processors can communicate through shared variables.
And you need things like locking, as I mentioned, to avoid race conditions or erroneous computation.
So as an example of parallelization, you know, straightforward parallelization in a shared memory machine, would be you have the simple loop that's just running through an array.
And you're adding elements of array A to elements of array B.
And you're going to write them to some new array, C. Well, if I gave you this loop you can probably recognize that there's really no data dependencies here.
I can split up this loop into three chunks -- let's say I have three processors -- where one processor does all the computations for iterations zero through three, so the first four iterations.
Second processor does the next four iterations.
And the third processor does the last four iterations.
And so that's shown with the little -- should have brought a laser pointer.
So that's showing here.
And what you might need to do is some mechanism to essentially tell the different processors, here's the code that you need to run and maybe where to start.
And then you may need some way of sort of synchronizing these different processors that say, I'm done, I can move on to the next computation steps.
So this is an example of a data parallel computation.
The loop has no real dependencies and, you know, each processor can operate on different data sets.
And what you could do is you can have a process -- this is a single application that forks off or creates what are commonly called the threads.
And each thread goes on and executes in this case the same computation.
So a single process can create multiple concurrent threads.
And really each thread is just a mechanism for encapsulating some trace of execution, some execution path.
So in this case you're essentially encapsulating this particular loop here.
And maybe you parameterize your start index and your ending index or maybe your loop bounds.
And in a shared memory processor, since you're communicating -- since there's only a single memory, really you don't need to do anything special about the data in this particular example, because everybody knows where to go look for it.
Everybody can access it.
Everything's independent.
There's no real issues with races or deadlocks.
So I just wrote down some actual code for that loop that parallelize it using Pthreads, a commonly used threading mechanism.
Just to give you a little bit of flavor for, you know, the complexity of -- the simple loop that we had expands to a lot more code in this case.
So you have your array.
It has 12 elements.
A, B, and C. And you have the basic functions.
So this is the actual code or computation that we want to carry out.
And what I've done here is I've parameterized where you're essentially starting in the array.
So you get this parameter.
And then you calculate four iterations' worth.
And this is essentially the computation that we're carrying out.
And now in my main program or in my main function, rather, what I do is I have this concept of threads that I'm going to create.
In this case I'm going to create three of them.
There are some parameters that I have to pass in, so some attributes which are now going to get into here.
But then I pass in the function pointer.
This is essentially a mechanism that says once I've created this thread, I go to this function and execute this particular code.
And then some arguments that are functions.
So here I'm just passing in an index at which each loop switch starts with.
And after I've created each thread here, implicitly in the thread creation, the code can just immediately start running.
And then once all the threads have started running, I can essentially just exit the program because I've completed.
So what I've shown you with that first example was the concept of, or example of, data parallelism.
So you're performing the same computation, but instead of operating on one big chunk of data, I've partitioned the data into smaller chunks and I've replicated the computation so that I can get that kind of parallelism.
But there's another form of parallelism called control parallelism, which essentially uses the same model of threading but doesn't necessarily have to run the same function or run the same computation each thread.
So I've sort of illustrated that in the illustration there, where these are your data parallel computations and these are some other computations in your code.
So there is sort of a programming model that allows you to do this kind of parallelism and tries to sort of help the programmer by taking their sequential code and then adding annotations that say, this loop is data parallel or this set of code is has this kind of control parallelism in it.
So you start with your parallel code.
This is the same program, multiple data kind of parallelization.
So you might have seen in the previous talk and the previous lecture, it was SIMD, single instruction or same instruction, multiple data, which allowed you to execute the same operation, you know, and add over multiple data elements.
So here it's a similar kind of terminology.
There's same program, multiple data, and multiple program, multiple data.
This talk is largely focused on the SPMD model, where you essentially have one central decision maker or you're trying to solve one central computation.
And you're trying to parallelize that over your architecture to get the best performance.
So you start off with your program and then you annotate the code with what's parallel and what's not parallel.
And you might add in some synchronization directives so that if you do in fact have sharing, you might want to use the right locking mechanism to guarantee safety.
Now, in OpenMP, there are some limitations as to what it can do.
So it in fact assumes that the programmer knows what he's doing.
And the programmer is largely responsible for getting the synchronization right, or that if they're sharing that they get those dependencies protected correctly.
So you can take your program, insert these annotations, and then you go on and test and debug.
So a simple OpenMP example, again using the simple loop -- now, I've thrown away some of the extra code -- you're adding these two extra pragmas in this case.
The first one, your parallel pragma, I call the data parallel pragma, really says that you can execute as many of the following code block as there are processors or as many as you have thread contexts.
So in this case I implicitly made the assumption that I have three processors, so I can automatically partition my code into three sets.
And this transformation can sort of be done automatically by the compiler.
And then there's a for pragma that says this loop is parallel and you can divide up the work in the mechanism that's work sharing.
So multiple threads can collaborate to solve the same computation, but each one does a smaller amount of work.
So this is in contrast to what I'm going to focus on a lot more in the rest of our talk, which is distributed memory processors and programming for distributed memories.
And this will feel a lot more like programming for the Cell as you get more and more involved in that and your projects get more intense.
So in distributed memory processors, to recap the previous lectures, you have n processors.
Each processor has its own memory.
And they essentially share the interconnection network.
Each processor has its own address, X.
So when a processor, P1, asks for X it knows where to go look.
It's going to look in its own local memory.
So if all processors are asking for the same value as sort of address X, then each one goes and looks in a different place.
So there are n places to look, really.
And what's stored in those addresses will vary because it's everybody's local memory.
So if one processor, say P1, wants to look at the value stored in processor two's address, it actually has to explicitly request it.
The processor two has to send it data.
And processor one has to figure out, you know, what to do with that copy.
So it has to store it somewhere.
So this message passing really exposes explicit communication to exchange data.
And you'll see that there are different kinds of data communications.
But really the concept of what you exchange has three different -- or four different, rather -- things you need to address.
One is how is the data described and what does it describe?
How are the processes identified?
So how do I identify that processor one is sending me this data?
And if I'm receiving data how do I know who I'm receiving it from?
Are all messages the same?
Well, you know, if I send a message to somebody, do I have any guarantee that it's received or not?
And what does it mean for a send operation or a receive operation to be completed?
You know, is there some sort of acknowledgment process?
So an example of a message passing program -- and if you've started to look at the lab you'll see that this is essentially where the lab came from.
It's the same idea.
I've created -- here I have some two-dimensional space.
And I have points in this two-dimensional space.
I have points B, which are these blue circles, and I have points A which I've represented as these yellow or golden squares.
And what I want to do is for every point in A I want to calculate the distance to all of the points B.
So there's sort of a pair wise interaction between the two arrays.
So a simple loop that essentially does this -- and there are n squared interactions, you have, you know, a loop that loops over all the A elements, a loop that loops over all the B elements.
And you essentially calculate in this case Euclidean distance which I'm not showing.
And you store it into some new array.
So if I give you two processors to do this work, processor one and processor two, and I give you some mechanism to share between the two -- so here's my CPU.
Each processor has local memory.
What would be some approach for actually parallelizing this?
Anybody look at the lab yet?
OK, so what would you do with two processors?
One has half memory [INAUDIBLE]
Right.
So what was said was that you split one of the arrays in two and you can actually get that kind of concurrency.
So, you know, let's say processor one already has the data.
And it has some place that it's already allocated where it's going to write C, the results of the computation, then I can break up the work just like it was suggested.
So what P1 has to do is send data to P2.
It says here's the data.
Here's the computation.
Go ahead and help me out.
So I send the first array elements, and then I send half of the other elements that I want the calculations done for.
And then P1 and P2 can now sort of start computing in parallel.
But notice that P2 has its own array that it's going to store results in.
And so as these compute they actually fill in different logical places or logical parts of the overall matrix.
So what has to be done is at the end for P1 to have all the results, P2 has to send it sort of the rest of the matrix to complete it.
And so now P1 has all the results.
The computation is done and you can move on.
Does that make sense?
OK.
So you'll get to actually do this as part of your labs.
So in this example messaging program, you have started out with a sequential code.
And we had two processors.
So processor one actually sends the code.
So it is essentially a template for the code you'll end up writing.
And it does have to work in the outer loop.
So this n array over which it is iterating the A array, is it's only doing half as many.
And processor two has to actually receive the data.
And it specifies where to receive the data into.
So I've omitted some things, for example, extra information sort of hidden in these parameters.
So here you're sending all of A, all of B.
Whereas, you know, you could have specified extra parameters that says, you know, I'm sending you A.
Here's n elements to read from A.
Here's B.
Here's n by two elements to read from B.
And so on.
But the computation is essentially the same except for the index at which you start, in this case changed for processor two.
And now, when the computation is done, this guy essentially waits until the data is received.
Processor two eventually sends it that data and now you can move on
I have a question
Yeah?
So would processor two have to wait for the data from processor one?
Yeah, so there's a -- I'll get into that later.
So what does it mean to receive?
To do this computation, I actually need this instruction to complete.
So what does it need for that instruction to complete?
I do have to get the data because otherwise I don't know what to compute on.
So there is some implicit synchronization that you have to do.
And in some cases it's explicit.
So I'll get into that a little bit later.
Does that sort of hint at the answer?
Are you still confused?
So processor one doesn't do the computation but it still sends the data --
So in terms of tracing, processor one sends the data and then can immediately start executing its code, right?
Processor two, in this particular example, has to wait until it receives the data.
So once this receive completes, then you can actually go and start executing the rest of the code.
So imagine that it essentially says, wait until I have data.
Wait until I have something to do.
Does that help?
Can the main processor [UNINTELLIGIBLE PHRASE]
Can the main processor --
I mean, in Cell, everybody is not peers.
There is a master there.
And what master can do instead of doing computation, master can be basically the quarterback, sending data, receiving data.
And SPEs can be basically waiting for data, get the computation, send it back.
So in some sense in Cell you probably don't want to do the computation on the master.
Because that means the master slows down.
The master will do only the data management.
So that might be one symmetrical [UNINTELLIGIBLE]
And you'll see that in the example.
Because the PPE in that case has to send the data to two different SPEs.
Yup?
In some sense [UNINTELLIGIBLE PHRASE] at points seems to be [UNINTELLIGIBLE] sense that if -- so have a huge array and you want to [UNINTELLIGIBLE PHRASE] the data to receive the whole array, then you have to [UNINTELLIGIBLE]
Yeah, we'll get into that later.
Yeah, I mean, that's a good point.
You know, communication is not cheap.
And if you sort of don't take that into consideration, you end up paying a lot for overhead for parallelizing things
[INAUDIBLE]
Well, you can do things in software as well.
We'll get into that.
OK, so some crude performance analysis.
So I have to calculate this distance.
And given two processors, I can effectively get a 2x speedup.
By dividing up the work I can get done in half the time.
Well, if you gave me four processors, I can maybe get done four times as fast.
And in my communication model here, I have one copy of one array that's essentially sending to every processor.
And there's some subset of A.
So I'm partitioning my other array into smaller subsets.
And I'm sending those to each of the different processors.
So we'll get into terminology for how to actually name these communications later.
But really the thing to take away here is that this granularity -- how I'm partitioning A -- affects my performance and communication almost directly.
And, you know, the comment that was just made is that, you know, what do you do about communication?
It's not free.
So all of those will be addressed.
So to understand performance, we sort of summarize three main concepts that you essentially need to understand.
One is coverage, or in other words, how much parallelism do I actually have in my application?
And this can actually affect, you know, how much work is it worth spending on this particular application?
Granularity -- you know, how do you partition your data among your different processors so that you can keep communication down, so you can keep synchronization down, and so on.
Locality -- so while not shown in the particular example, if two processors are communicating, if they are close in space or far in space, or if the communication between two processors is far cheaper than two other processors, can I exploit that in some way?
And so we'll talk about that as well.
So an example of sort of parallelism in an application, there are two essentially projects that are doing ray tracing, so I thought I'd have this slide here.
You know, how much parallelism do you have in a ray tracing program.
In ray tracing what you do is you essentially have some camera source, some observer.
And you're trying to figure out, you know, how to color or how to shade different pixels in your screen.
So what you do is you shoot rays from a particular source through your plane.
And then you see how the rays bounce off of other objects.
And that allows you to render scenes in various ways.
So you have different kinds of parallelisms.
You have your primary ray that's shot in.
And if you're shooting into something like water or some very reflective surface, or some surface that can actually reflect, transmit, you can essentially end up with a lot more rays that are created at run time.
So there's dynamic parallelism in this particular example.
And you can shoot a lot of rays from here.
So there's different kinds of parallelism you can exploit.
Not all prior programs have this kind of, sort of, a lot of parallelism, or embarrassingly parallel computation.
You know, you saw some basic code sequences in earlier lectures.
So there's a sequential part.
And the reason this is sequential is because there are data flow dependencies between each of the different computations.
So here I calculate a, but I need the result of a to do this instruction.
I calculate d here and I need that result to calculate e. But then this loop really here is just assigning or it's initializing some big array.
And I can really do that in parallel.
So I have sequential parts and parallel parts.
So how does that affect my overall speedups?
And so there's this law which is really a demonstration of diminishing returns, Amdahl's Law.
It says that if, you know, you have a really fast car, it's only as good to you as fast as you can drive it.
So if there's a lot of congestion on your road or, you know, there are posted speed limits or some other mechanism, you really can't exploit all the speed of your car.
Or in other words, you're only as fast as the fastest mechanisms of the computation that you can have.
So to look at this in more detail, your potential speedup is really proportional to the fraction of the code that can be parallelized.
So if I have some computation -- let's say it has three parts: a sequential part that takes 25 seconds, a parallel part that takes 50 seconds, and a sequential part that runs in 25 seconds.
So the total execution time is 100 seconds.
And if I have one processor, that's really all I can do.
And if she gave me more than one processor -- so let's say I have five processors.
Well, I can't do anything about the sequential work.
So that's still going to take 25 seconds.
And I can't do anything about this sequential work either.
That still takes 25 seconds.
But this parallel part I can essentially break up among the different processors.
So five in this case.
And that gets me, you know, five-way parallelism.
So the 50 seconds now is reduced to 10 seconds.
Is that clear so far?
So the overall running time in that case is 60 seconds.
So what would be my speedup?
Well, you calculate speedup, old running time divided by the new running time.
So 100 seconds divided by 60 seconds.
Or my parallel version is 1.67 times faster.
So this is great.
If I increase the number of processors, then I should be able to get more and more parallelism.
But it also means that there's sort of an upper bound on how much speedup you can get.
So if you look at the fraction of work in your application that's parallel, that's p. And your number of processors, well, your speedup is -- let's say the old running time is just one unit of work.
If the time it takes for the sequential work -- so that's 1 minus p, since p is the fraction of the parallel work.
And it's the time to do the parallel work.
And since I can parallelize that fraction over n processors, I can sort of reduce that to really small amounts in the limit.
Does that make sense so far?
So the speedup can tend to 1 over 1 minus p in the limit.
If I increase the number of processors or that gets really large, that's essentially my upper bound on how fast programs can work.
You know, how much can I exploit out of my program?
So this is great.
What this law says -- the implication here is if your program has a lot of inherent parallelism, you can do really well.
But if your program doesn't have any parallelism, well, there's really nothing you can do.
So parallel architectures won't really help you.
And there's some interesting trade-offs, for example, that you might consider if you're designing a chip or if you're looking at an application or domain of applications, figuring out what is the best architecture to run them on.
So in terms of performance scalability, as I increase the number of processors, I have speedup.
You can define, sort of, an efficiency to be linear at 100%.
But typically you end up in sort of the sublinear domain.
That's because communication is not often free.
But you can get super linear speedups ups on real architectures because of secondary and tertiary effects that come from register allocation or caching effects.
So they can hide a lot of latency or you can take advantage of a lot of pipelining mechanisms in the architecture to get super linear speedups.
So you can end up in two different domains.
So a small, you know, overview of the extent of parallelism in your program and how that affects your overall execution.
And the other concept is granularity.
So given that I have this much parallelism, how do I exploit it?
There are different ways of exploiting it, and that comes down to, well, how do I subdivide my problem?
What is the granularity of the sub-problems I'm going to calculate on?
And, really, granularity from my perspective, is just a qualitative measure of what is the ratio of your computation to your communication?
So if you're doing a lot of computation, very little communication, you could be doing really well or vice versa.
Then you could be computation limited, and so you need a lot of bandwidth for example in your architecture
Like before, you really didn't have to give every single processor an entire copy of
Right.
Yeah.
Good point.
And as you saw in the previous slides, you have -- computation stages are separated by communication stages.
And your communication in a lot of cases essentially serves as synchronization.
I need everybody to get to the same point before I can move on logically in my computation.
So there are two kinds of sort of classes of granularity.
There's fine grain and, as you'll see, coarse grain.
So in fine-grain parallelism, you have low computation to communication ratio.
And that has good properties in that you have a small amount of work done between communication stages.
And it has bad properties in that it gives you less performance opportunity.
It should be more, right?
More opportunity for --
No.
Less
Sorry.
Yeah, yeah, sorry.
I didn't get enough sleep.
So less opportunities for performance enhancement, but you have high communication ratio because essentially you're communicating very often.
So these are the computations here and these yellow bars are the synchronization points.
So I have to distribute data or communicate.
I do computations but, you know, computation doesn't last very long.
And I do more communication or more synchronization, and I repeat the process.
So naturally you can adjust this granularity to sort of reduce the communication overhead
[UNINTELLIGIBLE PHRASE] two things in that overhead part.
One is the volume.
So one, communication.
Also there's a large part of synchronization cost.
Basically you get a communication goal and you have to go start the messages and wait until everybody is done.
So that overhead also can go.
Even if you don't send that much data, just the fact that you are communicating, that means you have to do a lot of this additional bookkeeping stuff, that especially in the distributed [?
memory ?]
[?
machine is ?]
pretty expensive
Yeah.
Thanks.
So in coarse-grain parallelism, you sort of make the work chunks more and more so that you do the communication synchronization less and less.
And so that's shown here.
You do longer pieces of work and have fewer synchronization stages.
So in that regime, you can have more opportunities for performance improvements, but the tricky thing that you get into is what's called load balancing.
So if each of these different computations takes differing amounts of time to complete, then what you might end up doing is a lot of people might end up idle as they wait until everybody's essentially reached their finish line.
Yep?
If you don't have to acknowledge that something's done can't you just say, [?
OK, I'm done with your salt ?].
Hand it to the initial processor and keep doing whatever?
So, you can do that in cases where that essentially there is a mechanism -- or the application allows for it.
But as I'll show -- well, you won't see until the next lecture -- there are dependencies, for example, that might preclude you from doing that.
If everybody needs to reach the same point because you're updating a large data structure before you can go on, then you might not be able to do that.
So think of doing molecular dynamics simulations.
You need everybody to calculate a new position before you can go on and calculate new kinds of coarse interactions
[UNINTELLIGIBLE] nothing else to calculate yet
Right
But also there is pipelining.
So what do you talk about [UNINTELLIGIBLE] because you might want to get the next data while you're computing now so that when I'm done I can start sending.
[UNINTELLIGIBLE PHRASE] you can all have some of that
Yep.
Yeah, because communication is such an intensive part, there are different ways of dealing with it.
And that will be right after load balancing.
So the load balancing problem is just an illustration.
And things that appear in sort of this lightish pink will serve as sort of visual cues.
This is the same color coding scheme that David's using in the recitations.
So this is PPU code.
Things that appear in yellow will be SPU code.
And these are just meant to essentially show you how you might do things like this on Cell, just to help you along in picking up more of the syntax and functionality you need for your programs.
So in the load balancing problem, you essentially have, let's say, three different threads of computation.
And so that's shown here: red, blue, and orange.
And you've reached some communication stage.
So the PPU program in this case is saying, send a message to each of my SPEs, to each of my different processors, that you're ready to start.
And so now once every processor gets that message, they can start computing.
And let's assume they have data and so on ready to go.
And what's going to happen is each processor is going to run through the computation at different rates.
Now this could be because one processor is faster than another.
Or it could be because one processor is more loaded than another.
Or it could be just because each processor is assigned sort of differing amounts of work.
So one has a short loop.
One has a longer loop.
And so as the animation shows, sort of, execution proceeds and everybody's waiting until the orange guy has completed.
But nobody could have made progress until everybody's reached synchronization point because, you know, there's a strict dependence that's being enforced here that says, I'm going to wait until everybody's told me they're done before I go on to the next step of computation.
And so you know, in Cell you do that using mailboxes in this case.
That clear so far?
So how do you get around this load balancing problem?
Well, there are two different ways.
There's static load balancing.
I know my application really, really well.
And I understand sort of different computations.
So what I can do is I can divide up the work and have a static mapping of the work to my processors.
And static mapping just means, you know, in this particular example, that I'm going to assign the work to different processors and that's what the processors will do.
Work can't shift around between processors.
And so in this case I have a work queue.
Each of those bars is some computation.
You know, I can assign some chunk to P1, processor one, some chunk to processor two.
And then computation can go on.
Those allocations don't change.
So this works well if I understand the application, well and I know the computation, and my cores are relatively homogeneous and, you know, there's not a lot of contention for them.
So if all the cores are the same, each core has an equal amount of work -- the total amount of work -- this works really well because nobody is sitting too idle.
It doesn't work so well for heterogeneous architectures or multicores.
Because one might be faster than the other.
It increases the complexity of the allocation I need to do.
If there's a lot of contention for some resources, then that can affect the static load balancing.
So work distribution might end up being uneven.
So the alternative is dynamic load balancing.
And you certainly could do sort of a hybrid load balancing, static plus dynamic mechanism.
Although I don't have that in the slides.
So in the dynamic load balancing scheme, two different mechanisms I'm going to illustrate.
So in the first scheme, you start with something like the static mechanism.
So I have some work going to processor one.
And I have some work going to processor two.
But then as processor two executes and completes faster than processor one, it takes on some of the additional work from processor one.
So the work that was here is now shifted.
And so you can keep helping out, you know, your other processors to compute things faster.
In the other scheme, you have a work queue where you essentially are distributing work on the fly.
So as things complete, you're just sending them more work to do.
So in this animation here, I start off.
I send work to two different processors.
P2 is really fast so it's just zipping through things.
And then P1 eventually finishes and new work is allocated to the two different schemes.
So dynamic load balancing is intended to sort of give equal amounts of work in a different scheme for processors.
So it really increased utilization and spent less and less time being idle.
OK.
So load balancing was one part of sort of how granularity can have a performance trade-off.
The other is synchronization.
So there were already some good questions as to, well, you know, how does this play into overall execution?
When can I wait?
When can't I wait?
So I'm going to illustrate it with just a simple data dependence graph.
Although you can imagine that in each one of these circles there's some really heavy load computation.
And you'll see that in the next lecture, in fact.
So if I have some simple computation here -- I have some operands.
I'm doing an addition.
Here I do another addition.
I need both of these results before I can do this multiplication.
Here I have, you know, some loop that's adding through some array elements.
I need all those results before I do final substraction and produce my final result.
So what are some synchronization points here?
Well, it really depends on how I allocate the different instructions to processors.
So if I have an allocation that just says, well, let's put all these chains on one processor, put these two chains on two different processors, well, where are my synchronization points?
Well, it depends on where this guy is and where this guy is.
Because for this instruction to execute, it needs to receive data from P1 and P2.
So if P1 and P2 are different from what's in that box, somebody has to wait.
And so there's a synchronization that has to happen.
So essentially at all join points there's potential for synchronization.
But I can adjust the granularity so that I can remove more and more synchronization points.
So if I had assigned all this entire sub-graph to the same processor, I really get rid of the synchronization because it is essentially local to that particular processor.
And there's no extra messaging that would have to happen across processors that says, I'm ready, or I'm ready to send you data, or you can move on to the next step.
And so in this case the last synchronization point would be at this join point.
Let's say if it's allocated on P1 or on some other processor.
So how would I get rid of this synchronization point?
Do the whole thing
Right.
You put the entire thing on a single processor.
But you get no parallelism in this case.
So the coarse-grain, fine-grain grain parallelism granularity issue comes to play.
So the last sort of thing I'm going to talk about in terms of how granularity impacts performance -- and this was already touched on -- is that communication is really not cheap and can be quite overwhelming on a lot of architectures.
And what's interesting about multicores is that they're essentially putting a lot more resources closer together on a chip.
So it essentially is changing the factors for communication.
So rather than having, you know, your parallel cluster now which is connected, say, by ethernet or some other high-speed link, now you essentially have large clusters or will have large clusters on a chip.
So communication factors really change.
But the cost model is relatively captured by these different parameters.
So what is the cost of my communication?
Well, it's equal to, well, how many messages am I sending and what is the frequency with which I'm sending them?
There's some overhead for message.
So I have to actually package data together.
I have to stick in a control header and then send it out.
So that takes me some work on the receiver side.
I have to take the message.
I maybe have to decode the header, figure out where to store the data that's coming in on the message.
So there's some overhead associated with that as well.
There's a network delay for sending a message, so putting a message on the network so that it can be transmitted, or picking things up off the network.
So there's a latency also associated with how long does it take for a message to get from point A to point B.
What is the bandwidth that I have across a link?
So if I have a lot of bandwidth then that can really lower my communication cost.
But if I have little bandwidth then that can really create contention.
How much data am I sending?
And, you know, number of messages.
So this numerator here is really an average of the data that you're sending per communication.
There's a cost induced per contention.
And then finally there's -- so all of these are added factors.
The higher they are, except for bandwidth, because it's in the denominator here, the worse your communication cost becomes.
So you can try to reduce the communication cost by communicating less.
So you adjust your granularity.
And that can impact your synchronization or what kind of data you're shipping around.
You can do some architectural tweaks or maybe some software tweaks to really get the network latency down and the overhead per message down.
So on something like raw architecture, which we saw in Saman's lecture, there's a really fast mechanism to communicate your nearest neighbor in three cycles.
So one processor can send a single operand to another reasonably fast.
You know, you can improve the bandwidth again in architectural mechanism.
You can do some tricks as to how you package your data in each message.
And lastly, what I'm going to talk about in a couple of slides is, well, I can also improve it using some mechanisms that try to increase the overlap between messages.
And what does this really mean?
What am I overlapping it with?
And it's really the communication and computation stages are going to somehow get aligned.
So before I actually show you that, I just want to point out that there are two kinds of messages.
There's data messages, and these are, for example, the arrays that I'm sending around to different processors for the distance calculations between points in space.
But there are also control messages.
So control messages essentially say, I'm done, or I'm ready to go, or is there any work for me to do?
So on Cell, control messages, you know, you can think of using Mailboxes for those and the DMAs for doing the data communication.
So data messages are relatively much larger -- you're sending a lot of data -- versus control messages that are really much shorter, just essentially just sending you very brief information.
So in order to get that overlap, what you can do is essentially use this concept of pipelining.
So you've seen pipelining in superscalar.
Someone talked about that.
And what you are essentially trying to do is break up the communication and computation into different stages and then figure out a way to overlap them so that you can essentially hide the latency for the sends and the receives.
So let's say you have some work that you're doing, and it really requires you to send the data -- somebody has to send you the data or you essentially have to wait until you get it.
And then after you've waited and the data is there, you can actually go on and do your work.
So these are color coded.
So this is essentially one iteration of the work.
And so you could overlap them by breaking up the work into send, wait, work stages, where each iteration trying to send or request the data for the next iteration, I wait on the data from a previous iteration and then I do my work.
So depending on how I partition, I can really get really good overlap.
And so what you want to get to is the concept of the steady state, where in your main loop body, all you're doing is essentially pre-fetching or requesting data that's going to be used in future iterations for future work.
And then you're waiting on -- yeah.
I think my color coding is a little bogus.
That's good.
So here's an example of how you might do this kind of buffer pipelining in Cell.
So I have some main loop that's going to do some work, that's encapsulating this process data.
And what I'm going to use is two buffers.
So the scheme is also called double buffering.
I'm going to use this ID to represent which buffer I'm going to use.
So it's either buffer zero or buffer one.
And this instruction here essentially flips the bit.
So it's either zero or one.
So I fetch data into buffer zero and then I enter my loop.
So this is essentially the first send, which is trying to get me one iteration ahead.
So I enter this mail loop and I do some calculation to figure out where to write the next data.
And then I do another request for the next data item that I'm going to -- sorry, there's an m missing here -- I'm going to fetch data into a different buffer, right.
This is ID where I've already flipped the bit once.
So this get is going to write data into buffer zero.
And this get is going to write data into buffer one.
I flip the bit again.
So now I'm going to issue a wait instruction that says is the data from buffer zero ready?
And if it is then I can go on and actually do my work.
Does that make sense?
People are confused?
Should I go over it again?
[INAUDIBLE]
So this is an [?
x or.
?]
So I could have just said buffer equals zero or buffer equals one.
Oh, sorry.
This is one.
Yeah.
Yeah.
So this is a one here.
Last-minute editing.
It's right there.
Did that confuse you?
No.
But, like, I don't see [INAUDIBLE]
Oh.
OK.
So I'll go over it again.
So this get here is going to write into ID zero.
So that's buffer zero.
And then I'm going to change the ID.
So imagine there's a one here.
So now the next time I use ID, which is here, I'm trying to get the data.
And I'm going to write it to buffer one.
The DMA on the Cell processor essentially says I can send this request off and I can check later to see when that data is available.
But that data is going to go into a different buffer, essentially B1.
Whereas I'm going to work on buffer zero.
Because I changed the ID back here.
Now you get it?
So I fetch data into buffer zero initially before I start to loop.
And then I start working.
I probably should have had an animation in here.
So then you go into your main loop.
You try to start fetching into buffet one and then you try to compute out of buffer zero.
But before you can start computing out of buffer zero, you just have to make sure that your data is there.
And so that's what the synchronization is doing here.
Hope that was clear.
OK, so this kind of computation and communication overlap really helps in hiding the latency.
And it can be real useful in terms of improving performance.
And there are different kinds of communication patterns.
So there's point to point.
And you can use these both for data communication or control communication.
And it just means that, you know, one processor can explicitly send a message to another processor.
There's also broadcast that says, hey, I have some data that everybody's interested in, so I can just broadcast it to everybody on the network.
Or a reduce, which is the opposite.
It says everybody on the network has data that I need to compute, so everybody send me their data.
There's an all to all, which says all processors should just do a global exchange of data that they have.
And then there's a scatter and a gather.
So a scatter and a gather are really different types of broadcast.
So it's one to several or one to many.
And gather, which is many to one.
So this is useful when you're doing a computation that really is trying to pull data in together but only from a subset of all processors.
So it depends on how you've partitioned your problems.
So there's a well-known sort of message passing library specification called MPI that tries to specify all of these different communications in order to sort of facilitate parallel programming.
Its full featured actually has more types of communications and more kinds of functionality than I showed on the previous slides.
But it's not a language or a compiler specification.
It's really just a library that you can implement in various ways on different architectures.
Again, it's same program, multiple data, or supports the SPMD model.
And it works reasonably well for parallel architectures for clusters, heterogeneous multicores, homogeneous multicores.
Because really all it's doing is just abstracting out -- it's giving you a mechanism to abstract out all the communication that you would need in your computation.
So you can have additional things like precise buffer management.
You can have some collective operations.
I'll show an example of for doing things things in a scalable manner when a lot of things need to communicate with each other.
So just a brief history of where MPI came from.
And, you know, very early when, you know, parallel computers started becoming more and more widespread and there were these networks and people had problems porting their applications or writing applications for these [?
came, ?]
just because it was difficult, as you might be finding in terms of programming things with the Cell processor.
You know, there needed to be ways to sort of address the spectrum of communication.
And it often helps to have a standard because if everybody implements the same standard specification, that allows your code to be ported around from one architecture to the other.
And so MPI came around.
The forum was organized in 1992.
And that had a lot of people participating in it from vendors, you know, people like IBM, a company like IBM, Intel, and people who had expertise in writing libraries, users who were interested in using these kinds of specifications to do their computations, so scientific people who were in the scientific domain.
And it was finished in about 18 months.
I don't know if that's a reasonably long time or a short time.
But considering, you know, I think the MPEG-4 standard took a bit longer to do, as a comparison point.
I don't have the actual data.
So point-to-point communication -- and again, a reminder, this is how you would do it on Cell.
These are SPE sends and receives.
You have one processor that's sending it to another processor.
Or you have some network in between.
And processor A can essentially send the data explicitly to processor two.
And the message in this case would include how the data is packaged, some other information such as the length of the data, destination, possibly some tag so you can identify the actual communication.
And, you know, there's an actual mapping for the actual functions on Cell.
And there's a get for the send and a put for the receive.
So there's a question of, well, how do I know if my data actually got sent?
How do I know if it was received?
And there's, you know, you can think of a synchronous send and a synchronous receive, or asynchronous communication.
So in the synchronous communication, you actually wait for notification.
So this is kind of like your fax machine.
You put something into your fax.
It goes out and you eventually get a beep that says your transmission was OK. Or if it wasn't OK then, you know, you get a message that says, you know, something went wrong.
And you can redo your communication.
An asynchronous send is kind of like your --
Most [UNINTELLIGIBLE] you could get a reply too
Yeah, you can get a reply.
Thanks.
An asynchronous send, it's like you write a letter, you go put it in the mailbox, and you don't know whether it actually made it into the actual postman's bag and it was delivered to your destination or if it was actually delivered.
So you only know that the message was sent.
You know, you put it in the mailbox.
But you don't know anything else about what happened to the message along the way.
There's also the concept of a blocking versus a non-blocking message.
So this is orthogonal really to synchronous versus asynchronous.
So in blocking messages, a sender waits until there's some signal that says the message has been transmitted.
So this is, for example if I'm writing data into a buffer, and the buffer essentially gets transmitted to somebody else, we wait until the buffer is empty.
And what that means is that somebody has read it on the other end or somebody has drained that buffer from somewhere else.
The receiver, if he's waiting on data, well, he just waits.
He essentially blocks until somebody has put data into the buffer.
And you can get into potential deadlock situations.
So you saw deadlock with locks in the concurrency talk.
I'm going to show you a different kind of deadlock example.
An example of a blocking send on Cell -- allows you to use mailboxes.
Or you can sort of use mailboxes for that.
Mailboxes again are just for communicating short messages, really, not necessarily for communicating data messages.
So an SPE does some work, and then it writes out a message, in this case to notify the PPU that, let's say, it's done.
And then it goes on and does more work.
And then it wants to notify the PPU of something else.
So in this case this particular send will block because, let's say, the PPU hasn't drained its mailbox.
It hasn't read the mailbox.
So you essentially stop and wait until the PPU has, you know, caught up
So all mailbox sends are blocking?
Yes.
David says yes.
A non-blocking send is something that essentially allows you to send a message out and just continue on.
You don't care exactly about what's happened to the message or what's going on with the receiver.
So you write the data into the buffer and you just continue executing.
And this really helps you in terms of avoiding idle times and deadlocks, but it might not always be the thing that you want.
So an example of sort of a non-blocking send and wait on Cell is using the DMAs to ship data out.
You know, you can put something, put in a request to send data out on the DMA.
And you could wait for it if you want in terms of reading the status bits to make sure it's completed.
OK, so what is a source of deadlock in the blocking case?
And it really comes about if you don't really have enough buffering in your communication network.
And often you can resolve that by having additional storage.
So let's say I have processor one and processor two and they're trying to send messages to each other.
So processor one sends a message at the same time processor two sends a message.
And these are going to go, let's say, into the same buffer.
Well, neither can make progress because somebody has to essentially drain that buffer before these receives can execute.
So what happens with that code is it really depends on how much buffering you have between the two.
If you have a lot of buffering, then you may never see the deadlock.
But if you have a really tiny buffer, then you do a send.
The other person can't do the send because the buffer hasn't been drained.
And so you end up with a deadlock.
And so a potential solution is, well, you actually increase your buffer length.
But that doesn't always work because you can still get into trouble.
So what you might need to do is essentially be more diligent about how you order your sends and receives.
So if you have processor one doing a send, make sure it's matched up with a receive on the other end.
And similarly, if you're doing a receive here, make sure there's sort of a matching send on the other end.
And that helps you in sort of making sure that things are operating reasonably in lock step at, you know, partially ordered times.
That was really examples of point-to-point communication.
A broadcast mechanism is slightly different.
It says, I have data that I want to send to everybody.
It could be really efficient for sending short control messages, maybe even efficient for sending data messages.
So as an example, if you remember our calculation of distances between all points, the parallelization strategy said, well, I'm going to send one copy of the array A to everybody.
In the two processor case that was easy.
But if I have n processors, then rather than sending point-to-point communication from A to everybody else, what I could do is just, say, broadcast A to everybody and they can grab it off the network.
So in MPI there's this function, MPI broadcast, that does that.
I'm using sort of generic abstract sends, receives and broadcasts in my examples.
So you can broadcast A to everybody.
And then if I have n processors, then what I might do is distribute the m's in a round robin manner to each of the different processes.
So you pointed this out.
I don't have to send B to everybody.
I can just send, you know, in this case, one particular element.
Is that clear?
There's no broadcast on Cell?
There is no broadcast on Cell.
There is no mechanism for reduction either.
And you can't quite do scatters and gathers.
I don't think.
OK, so an example of a reduction, you know, I said it's the opposite of a broadcast.
Everybody has data that needs to essentially get to the same point.
So as an example, if everybody in this room had a value, including myself, and I wanted to know what is the collective value of everybody in the room, you all have to send me your data.
Now, this is important because if -- you know, in this case we're doing an addition.
It's an associative operation.
So what we can do is we can be smart about how the data is sent.
So, you know, guys that are close together can essentially add up their numbers and forward me.
So instead of getting n messages I can get log n messages.
And so if every pair of you added your numbers and forwarded me that, that cuts down communication by half.
And so you can, you know -- starting from the back of room, by the time you get to me, I only get two messages instead of n messages.
So a reduction combines data from all processors.
In MPI, you know, there's this function MPI reduce for doing that.
And the collective operations are things that are associative.
And subtract -- sorry.
And or and -- you can read them on the slide.
There is a semantic caveat here that no processor can finish the reduction before all processors have at least sent it one data or have contributed, rather, a particular value.
So in many numerical algorithms, you can actually use the broadcast and send to broadcast and reduce in place of sends and receives because it really improves the simplicity of your computation.
You don't have to do n sends to communicate there.
You can just broadcast.
It gives you a mechanism for essentially having a shared memory abstraction on distributed memory architecture.
There are things like all to all communication which would also help you in that sense.
Although I don't talk about all to all communication here.
So I'm going to show you an example of sort of a more detailed MPI.
But I also want to contrast this to the OpenMP programming on shared memory processors because one might look simpler than the other.
So suppose that you have a numerical integration method that essentially you're going to use to calculate pi.
So as you get finer and finer, you can get more accurate -- as you shrink these intervals you can get better values for pi.
And the code for doing that is some C code.
You have some variables.
And then you have a step that essentially tells you how many times you're going to do this computation.
And for each time step you calculate this particular function here.
And you add it all up and in the end you can sort of print out what is the value of pi that you calculated.
So clearly as, you know, as you shrink your intervals, you can get more and more accurate measures of pi.
So that translates to increasing the number of steps in that particular C code.
So you can use that numerical integration to calculate pi with OpenMP.
And what that translates to is -- sorry, there should have been an animation here to ask you what I should add in.
You have this particular loop.
And this is computation that you want to parallelize.
And there is really four questions that you essentially have to go through.
Are there variables that are shared?
Because you have to get the process right.
If there are variables that are shared, you have to explicitly synchronize them and use locks to protect them.
What values are private?
So in OpenMP, things that are private are data on the stack, things that are defined lexically within the scope of the computation that you encapsulate by an OpenMP pragma, and what variables you might want to use for a reduction.
So in this case I'm doing a summation, and this is the computation that I can parallelize.
Then I essentially want to do a reduction for the plus operator since I'm doing an addition on this variable.
This loop here is parallel.
It's data parallel.
I can split it up.
The for loop is also -- I can do this work sharing on it.
So I use the parallel for pragma.
And the variable x here is private.
It's defined here but I can essentially give a directive that says, this is private.
You can essentially rename it on each processor.
Its value won't have any effect on the overall computation because each computation will have its own local copy.
That clear so far?
So computing pi with integration using MPI takes up two slides.
You know, I could fit it on one slide but you couldn't see it in the back.
So there's some initialization.
In fact, I think there's only six basic MPI commands that you need for computing.
Three of them are here and you'll see the others are MPI send and MPI receive.
And there's one more that you'll see on the next slide.
So there's some loop that says while I'm not done keep computing.
And what you do is you broadcast n to all the different processors.
N is really your time step.
How many small intervals of execution are you going to do?
And you can go through, do your computation.
So now this -- the MPI essentially encapsulates the computation over n processors.
And then you get to an MPI reduce command at some point that says, OK, what values did everybody compute?
Do the reduction on that.
Write that value into my MPI.
Now what happens here is there's processor ID zero which I'm going to consider the master.
So he's the one who's going to actually print out the value.
So the reduction essentially synchronizes until everybody's communicated a value to processor zero.
And then it can print out the pi.
And then you can finalize, which actually makes sure the computation can exit.
And you can go on and terminate.
So the last concept in terms of understanding performance for parallelism is this notion of locality.
And there's locality in your communication and locality in your computation.
So what do I mean by that?
So in terms of communication, you know, if I have two operations and let's say -- this is a picture or schematic of what the MIT raw chip looks like.
Each one of these is a core.
There's some network, some basic computation elements.
And if I have, you know, an addition that feeds into a shift, well, I can put the addition here and the shift there, but that means I have a really long path that I need to go to in terms of communicating that data value around.
So the computation naturally should just be closer together because that decreases the latency that I need to communicate.
So rather than doing net mapping, what I might want to do is just go to somebody who is close to me and available
Also there are volume issues.
So assume more than that.
A lot of other people also want to communicate.
So if [UNINTELLIGIBLE] randomly distributed, you can assume there's a lot more communication going into the channel.
Whereas if you put locality in there then you can scale communication much better than scaling the network
There's also a notion of locality in terms of memory accesses.
And these are potentially also very important or more important, rather, because of the latencies for accessing memory.
So if I have, you know, this loop that's doing some addition or some computation on an array and I distribute it, say, over four processors -- this is, again, let's assume a data parallel loop.
So what I can do is have a work sharing mechanism that says, this thread here will operate on the first four indices.
This thread here will operate on the next four indices and the next four and the next four.
And then you essentially get to join barrier and then you can continue on.
And if we consider how the access patterns are going to be generated for this particular loop, well, in the sequential case I'm essentially generating them in sequence.
So that allows me to exploit, for example, on traditional [?
CAT ?]
architecture, a notion of spatial locality.
If I look at how things are organized in memory, in the sequential case I can perhaps fetch an entire block at a time.
So I can fetch all the elements of A0 to A3 in one shot.
I can fetch all the elements of A4 to A7 in one shot.
And that allows me to essentially improve performance because I overlap communication.
I'm predicting that once I see a reference, I'm going to use data that's adjacent to it in space.
There's also a notion of temporal locality that says that if I use some particular data element, I'm going to reuse it later on.
I'm not showing that here.
But in the parallel case what could happen is if each one of these threads is requesting a different data element -- and let's say execution essentially proceeds -- you know, all the threads are requesting their data at the same time.
Then all these requests are going to end up going to the same memory bank.
The first thread is requesting ace of zero.
The next thread is requesting ace of four, the next thread ace of eight, next thread ace of 12.
And all of these happen to be in the same memory bank.
So what that means is, you know, there's a lot of contention for that one memory bank.
And in effect I've serialized the computation.
Right?
Everybody see that?
And, you know, this can be a problem in that you can essentially fully serialize the computation in that, you know, there's contention on the first bank, contention on the second bank, and then contention on the third bank, and then contention on the fourth bank.
And so I've done absolutely nothing other than pay overhead for parallelization.
I've made extra work for myself [?
concreting ?]
the threads.
Maybe I've done some extra work in terms of synchronization.
So I'm fully serial.
So what you want to do is actually reorganize the way data is laid out in memory so that you can effectively get the benefit of parallelization.
So if you have the data organized as is there, you can shuffle things around.
And then you end up with fully parallel or a layout that's more amenable to full parallelism because now each thread is going to a different bank.
And that essentially gives you a four-way parallelism.
And so you get the performance benefits.
So there are different kinds of sort of considerations you need to take into account for shared memory architectures in terms of how the design affects the memory latency.
So in a uniform memory access architecture, every processor is either, you can think of it as being equidistant from memory.
Or another way, it has the same access latency for getting data from memory.
Most shared memory architectures are non-uniform, also known as NUMA architecture.
So you have physically partitioned memories.
And the processors can have the same address space, but the placement of data affects the performance because going to one bank versus another can be faster or slower.
So what kind of architecture is Cell?
Yeah.
No guesses?
It's not a shared memory
Right.
It's not a shared memory architecture.
So a summary of parallel performance factors.
So there's three things I tried to cover.
Coverage or the extent of parallelism in the application.
So you saw Amdahl's Law and it actually gave you a sort of a model that said when is parallelizing your application going to be worthwhile?
And it really boils down to how much parallelism you actually have in your particular algorithm.
If your algorithm is sequential, then there's really nothing you can do for programming for performance using parallel architectures.
I talked about granularity of the data partitioning and the granularity of the work distribution.
You know, if you had really fine-grain things versus really coarse-grain things, how does that translate to different communication costs?
And then last thing I shared was locality.
So if you have near neighbors talking, that may be different than two things that are further apart in space communicating.
And there are some issues in terms of the memory latency and how you actually can take advantage of that.
So this really is an overview of sort of the parallel programming concepts and the performance implications.
So the next lecture will be, you know, how do I actually parallelize my program?
And we'll talk about that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So in this second lecture we're going to talk about some design patterns for parallel programming.
And to tell you a little bit about what a design pattern is and why is it useful.
And some of you, if you've taken object oriented programming you've probably already have seen design patterns before.
So I ended the last lecture with OK, so I understand some of the performance implications, how do I go about parallelizing my program?
This is a flyer I found quite often in books and talks on parallel programming.
It essentially lays out 4 common steps for parallelizing your program.
So often, you start out with sequential programs.
This shouldn't be surprising since for a long time as you've heard in earlier lectures, people just coded sequential code and that was just good enough.
So now the problem is you want to take that sequential code or you want to still write sequential code just because it's conceptually easier and you want to be able to parallelize it so you can map it down to your parallel architecture, which in this example has 4 processors.
So the first step is you take your sequential program and you divide it up into tasks.
So during the project reviews, for example, yesterday when I talked to each team individually we sort of talked about this and you sort of stumbled on these 4 steps whether you realized it or not.
And so you come up with these tasks and then each one essentially incapsulates some computation.
And then you group them together, so this is some granularity adjustment and you map them down to processes.
These are things you can pose into threads, for example.
And then you have to essentially plot these down onto actual processors and they have to talk with each other, so you have to orchestrate to communication and then finally do the execution.
So step through each one of these at a time.
Sort of composition and recall that just really effects or just really is effected by Amdahl's Law.
And that if there's not a whole lot of parallels in the application your decomposition is a waste of time.
There's not really a whole lot to get.
But what you're trying to do is identify concurrency in your application and figure out at what level to exploit it.
So what you're trying to do is divide up your computation into tasks and eventually these are going to be distributed among processors and you want to find enough of them so that you can keep all the processors busy.
And remember that the more of these that you have this gives you sort of an upper bound on your potential speed up.
And as in the rate tracing example that I showed, the number of tasks that you have may vary run time.
So sometimes you might have lot of arrays bouncing off a lot of things, and sometimes you might not have a whole lot of reflection going on so number of arrays will change over time.
And in other applications, interactions, for example, between molecules might change in a molecular dynamic simulator.
The assignment really effects granularity.
This is where you've partitioned your tasks, you're trying to group them together so that you're taking into account, what is the communication cost going to be?
What kind of locality am I going to deal with?
And what kind of synchronization mechanisms do I need and how often do I need to synchronize?
You adjust your granularity such that you end up with things that are load balanced and you try to reduce communication as much as possible.
And structured approaches might work well.
You might look at the code, do some inspection, you might understand the application, but there are some well-known design patterns which is essentially the thing we're going to get to try to help you with this.
As programmers really, I think, we worry about partitioning first.
This is really independent of an architecture programming model.
Just taking my application and figuring out well, what are different parts that I need to compose together to build my application?
So I'm going to show you an example of that.
And one thing to keep in the back of your mind is that the complexity of how much partitioning work you actually have to do really effects your decision.
So if you start out with some piece of code or you wrote your code in one way and you realize that to actually parallelize it requires so much more work, in some user studies we've done on sort of trying to get performance from code, it really effects how much work you actually do.
And if something requires a lot of work, you might not do it even though it might have really hot payoff.
So you want to be able to keep complexity down, so it pays off to really think well about your algorithm, how you structure it ahead of time.
And finally, the last two stages I've lumped together.
It's really orchestration and mapping.
I have my task, they need to communicate, so what kind of computation primitives do I need?
What kind of communication primitives do I need?
So am I packaging things up into threads?
And they're talking together over DMAs or shared memories.
And what you want to do is try to preserve locality and then figure out how to come up with a scheduling order that preserves overall dependence of the computation.
Parallel program by patterns is meant to essentially give you a cookbook or set of recipes you can follow to help you with different steps: decompose, assign, orchestrate and map.
This could lead to really high quality solutions in some domains.
So in the scientific computations there's a lot of problems that are well understood and well studied and some of the frequently occurring things have been abstracted out and sort of recorded in patterns.
And there's another purpose to patterns too, in that they provide you with the vocabulary for-- two programmers can talk to each other and use the right terminology and that conveys a whole lot of information without having to actually go through and understand all the details.
You instantaneously know if I use a particular model.
It can also help with software reusuability, malleability, and modularity.
All of those things that are software engineer perspective from an engineering perspective are important.
So brief history and I found this in some of the talks that I was researching.
There's a book by Christopher Alexander from Berkeley in 1977 that actually looked at classifying patterns or really listing patterns from an architectural perspective.
He tried to look at what are some patterns that occur in living designs and recording those.
So as an example, for example, there's a 6 foot balcony pattern.
So if you're going to build a balcony you should build it 6 foot deep and you should have it slightly recessed and so on because this is what's commonly used and these are the kinds of balconies that have good properties architecturally.
Now I don't know whether this book actually had a whole lot of impact on how people designed architectures.
Certainly not probably for the Stata Center, but some patterns from object oriented programming, I think, many of you have already seen these by the Gang of Four in 1995-- really sort of organized and classified and came up with different ways of-- or captured different ways of programming that people had been using.
You know, things like the visitor pattern, for example, some of you might know.
So in 2005, not too long ago there was a new book, which I'm using to create some of these slides.
Really came up with or recorded patterns for parallel programming.
And they identified really 4 design spaces.
I think these are sort of structured to express or capture different elements.
So some elements are for the algorithm expression, I've listed those here and some are for the actual software construction or the actual implementation.
So under algorithm expression it's really the thing of decomposition; finding concurrency.
Where are my tasks?
In the algorithm structure, well, you might need some way of packaging those tasks together so that they can talk to each other and be able to use parallel architecture.
On the software construction side you're dealing slightly more low level details.
So what are some things you might need at a slightly lower level of implementation to actually get all the computation that's expressed at the algorithm level to work and run well?
So I'm going to essentially talk about the latter part in the next lecture and I'll cover much of the algorithm expression stuff here-- at least the fine concurrency part in this talk.
And if there's time I'll do algorithm structure.
Otherwise, just talk about it next time.
So let's say you're working with MPEG decoding.
This is a pipeline picture of an MPEG 2 decoder or rather a blocked level diagram of an MPEG 2 decoder.
And you have this algorithm and you say, OK, I want to parallelize this.
Where's my parallelism?
Where's my concurrency?
You know, in MPEG 2 you have some bit screen, you do some decoding on it and you end up with two things.
You end up with motion vectors that tell you here's somebody's head, in the next scene it's moved to this particular location.
So these are captured by the motion vectors.
So this captures or recovers temporal information.
In here you cover spatial information.
So in somebody's head you might have discovered some redundancies so that redundancy is eliminated out so you need to know essentially, uncompress or undo that compression.
So you go through some stages.
You combine the two together.
Combine the motion estimation and now the recovered pictures to reconstruct the image and then you might do some additional stages.
This particular stage here is indicated to be data parallel in that I can do different scenes for example in parallel or I might be able to do different slices of the picture in parallel.
So I can essentially take advantage of data parallelism in the concept of taking a loop and breaking it up as I showed in lecture 5.
So in task decomposition what we're looking for is really independent coarse-grain computation.
And these often are inherent to the algorithm.
so here I've outlined these in yellow here.
You know, so this is one particular task.
I can have one thread of execution doing all the spatial decomposition.
I can have another thread decoding all my motion vectors.
And in general, you're looking for sequences of statements that operate together as a group.
These could be loops or they could be functions.
And usually you want these to essentially just fall out of your algorithm as it's expressed.
And a lot of cases it does, so depending on how you think about the program you might be able to find these quicker or easier.
Data decompositions, which I've highlighted here essentially says you have the same computation applied to lots of small data element.
You know, you can take your large data element, partition it into smaller chunks and do the computation over and over in parallel and so that allows you to essentially get that kind of data parallelism, expansion of space.
And finally, I'm going to make a case for pipeline parallelism, which essentially says, well, I can recognize that I have a lot of stages in my computation and it does help to actually have this kind of decomposition just because you're familiar with pipelining concepts from other domains.
So this type of producer consumer chain is actually beneficial.
So it does help to sort of expose these kinds of relationships.
So what are some guidelines for actually coming up with your task decomposition?
Where do you start?
You have your algorithm, you understand the problem really well, you're writing some code and the hope is that in fact, as I've pointed out, it does happen that you can look for natural code regions that incapsulate your computation.
So function calls, distinct loop iterations are pretty good places to start looking.
And it's easier as a general rule, it's easier to start with as many tasks as possible and then fuse them to make the more coarse-grained than to go the other way around.
It impacts your software engineering decisions, it impacts software implementation, it impacts how you incapsulate things at low level details of implementation.
So it's always easier to fuse than to fizz.
And you want to consider three things.
You want to keep three things in mind: flexibility, efficiency, and simplicity.
So flexibility says if you made some decisions, is that really going to scale well or is it going to allow you to sort of make the decisions, changes?
So you might want to have fixed tasks versus parameterized tasks, for example.
So the loops that I showed in the previous talk, each loop that I parallelized had a hard coded number that said, you're going to do 4 iterations.
That may work well or it may not work well.
You know, I can't reuse that code now if I want to essentially use that kind of data decomposition and work sharing if I have a longer loop and I want a longer array and I want each thread to do more work.
So you might want to parameterize more things in your tasks.
The efficiency, in that you have to keep in mind that each of these tasks will eventually sort of have to talk with other tasks.
There's communication costs that have to be taken into account, synchronization.
So you want these tasks to actually amortize the communication costs or other overheads over to computation.
And you want to keep in mind that there's going to be dependencies between these tasks and you don't want these dependencies to get out of hand.
So you want to keep things under control.
And lastly, which is probably as important as the other two: simplicity.
And that if you start decomposing your code into different chunks and you can't then understand your code in the end, it doesn't help you from debugging perspective, doesn't help you from a software engineering perspective or not being able to reuse you code or other people being able to understand your code.
Guidelines for data decomposition are sort of similar.
And you essentially have to do task and data parallelism to sort of complete your process.
And often your task decomposition dictates your data partitioning.
So if I've split up a loop into two different processes I've essentially implied how data should be distributed between these two threads.
And data composition is a good starting point as opposed to task parallelism as a good starting point.
If you're doing the same computation over and over and over again, over really, really large data sets, so you can essentially use that as your stick to decide whether you do task decomposition first or data decomposition first.
I've just listed two common data decompositions.
I'll talk about more of these later on when we talk about actual performance optimizations.
So you want to decompose arrays for example, along rows or columns.
You can compose arrays into blocks, you decompose them into blocks.
You have recursive data structures.
So a tree, you might partition it into left and right sub-trees in a binary tree.
And the thing you're trying to get to is actually start with a problem, and then recursively subdivide it until you can get to a manageable part.
Do the computation and figure out a way to do the integration.
You know, it's like merge sort, classic example -- tries to capture this really well.
So again, the three theme, key concepts to keep in mind when you're doing data decomposition: flexibility, efficiency, and simplicity.
The first two are really just meant to suggest that the size of the data that you've allocated actually leads to enough work.
Because you want to amortize the cost of communication or synchronization, but you also want the amount of work that's generated by each data chunk to generate about the same amount of work, so load balancing.
And simplicity, just for same reason as task decomposition can get out of hand, data decomposition can get out of hand.
You don't want data moving around throughout and then it becomes again, hard to debug or manage or make changes and track dependencies.
Pipeline parallelism, this is actually classified somewhere else in the book.
Actually, lifted it up and tried to make a case for it in that it's just good nature I think, to expose producer consumer relationships in your code.
So if I have a function that's producing data that's going to be used by another function as with the spatial decomposition or in different stages of classic rate tracing algorithms you want to maintain that producer consumer relationship or that asembly line analogy.
And what are some prime examples of pipelines in computer architecture?
It's like the instruction pipeline and your super scalar.
But there might be some other examples of pipelines, things that you might have used in say, UNIX shell.
So cat processor, pipe it to another-- to a grep word and then word count that.
So I think it's a natural concept.
We use it in many different ways and it's good to sort of practice that at the software level as well.
And there are some computations in specific domains, like in signal processing and graphics that have really sort of-- where the pipeline model is really important part of how computation gets carried out.
You know, you have your graphics pipeline for example, in signal processing.
How much time do I have?
How am I doing on time?
OK, should I stop here?
About how much more?
10 slides
OK, so this is sort of a brief summary, which will lead into a much larger talk at the next lecture on how you actually go about re-engineering your code for parallelism.
And this could come into play if you start with sequential code and you're parallelizing it.
Some of you are doing that for your projects.
Or if you're actually writing code from scratch and you want to engineer that for parallelism as well.
So I think it's important to sort of understand the problem that you're working with.
So you want to survey your landscape, understand what other people might have done, and look for well-known solutions and common pitfalls.
And the patterns that I'm going to talk about in more detail really provide you with a list of questions to sort of help you assess the existing code that you're working with or the problem that you're trying to solve.
There's something that you need to keep in mind that sort of effect your overall correctness.
So for example, is your computation numerically stable?
You might know if you have a floating point computation you might not be able to reorder all the operations because that might effect your actual precision.
And so your overall output might be different and that may or may not be acceptable.
So a lot of scientific codes for example, are things that have to deal with a lot of precision, might have to be cognizant of that fact.
You want to also define the scope of, what are you trying to do and will it be good enough?
You want to do back of the hand, back of the envelope calculations to make sure that things that you're suggesting of doing are actually feasible, that they're actually practical and that will give you the sort of performance expectations that you've set out.
You also want to understand your input range.
You might be able to specialize if there are some cases for example, that you're allowed to ignore.
So these are good things to keep in mind.
You also want to define a testing protocol.
I think it's important to understand-- you started out with some piece of code, you're going to make some changes to it, how you going to go about testing it?
How you might go about debugging it and that could be essentially where you spend a lot of your time.
And then having these things in mind, I think, the parts that are worth looking at are the parts that make the most sense.
Where is your computation spending most of its time?
Is there hot spots in your code?
And you can use profiling tools for that and in fact, you'll see some of that for cell in some of the recitations later in the course.
So a simple example of molecular dynamics simulator.
What you're trying to do is, you have some space of molecules, which I'm just going to represent in 2D.
You know, look, they have water molecules and I have some protein that I'm trying to understand how the different atoms in that molecule are moving around so that I can determine the shape of the protein.
So there are forces, there are bonded forces between the molecules.
So I've just shown for example, bonded forces among my protein and then there are non-bonded forces.
So how are different atoms sort of interacting with each other because of electrostatic forces, for example.
So what you try to do is figure out, on each atom, what are all the forces that are affecting it and what is its current position and then you try to estimate where it's going to move based on Newtonian, in the simplest case, a Newtonian f equals m a type projection.
So in a naive algorithm you have n squared interactions.
You have to calculate all the forces on one molecule from all others.
By understanding your problem you know that you can actually exploit the properties of forces that essentially decrease exponentially, so you can use a cutoff distance.
So if a molecule is way too far away you can ignore this.
And for people who do galaxy calculations, you know you can ignore geometric forces between constellations or clusters that are too far apart.
So in the sequential code, some pseudo code for doing a molecular dynamic simulator, you have your atoms array, your force array, your set of neighbors in a two-dimensional space and you're going to go through and sort of simulate different time steps.
And for each time step you're going to do-- for each atom-- compute the bonded forces, compute who are my neighbors for those neighbors, compute-- so these are things that essentially incapsulate distance.
Compute the forces between them, update the position and end.
So since this is a loop then that might suggest essentially where to start looking for concurrency.
So you can start with the decomposition patterns and they'll be more in depth details about those next.
I'm going to give you some intuition and then you would try to figure out whether your decomposition has to abide by certain dependencies and what are those dependencies?
How do you expose them?
And then, how can you design and evaluate?
How can you evaluate your design?
Screwed up again.
I just fixed this.
OK, so this is the pseudo code again from the previous slide.
And since all you have is a simple loop that essentially says, this is where to look for the computation.
And since you're essentially doing the same computation for each atom then that again, gives you the type of parallelism that we've talked about before.
So you can look for splitting up each iteration and parallelizing those so that each processor for example, does one atom.
Or each processor does a collection of atoms.
But there are additional tasks.
So data level parallelism versus sort of control parallelism.
For each atom you also want to calculate the forces.
You want to calculate long range interactions, find neighbors, update position and so on.
And some of these have shared data, some of them do not.
So you have to factor that in.
So understanding there control dependencies essentially tells you how you need to it lay out your orchestration.
So you have your bonded forces, you have your neighbor list, that effects your long range calculations.
But to do this update position I need both of these tasks to have completed.
And in each one of these tasks there's different data structures that they need.
So everybody essentially reads the location of the items.
So this is good because we want time step that essentially says, I can really distribute this really well, but then there's a right synchronization problem because eventually I have to update this array so I have to be careful about who goes first.
There's an accumulation, which means I can potentially do a reduction on these.
There's some write on the other end, but that seems to be a localized data structure.
So for partitioning example, neighbors might have to be just locals at a different processor.
So coming up with this structure and sort of block level diagram helps you essentially figure out where are your tasks?
Helps you figure out what kind of synchronization mechanisms you need and it can also help you suggest the data distribution that you might need to reduce synchronization costs and problems.
And lastly, you want to essentially evaluate your design.
And you want to keep in mind, what is your target architecture.
Are you trying to really run on shared memory and distributed memory and message passing or are you just doing this for one architecture?
So for your project, you're doing this for self, so you can be very self specific.
But if you're doing this in other contexts, the architecture actually might influence some of your decisions.
Does data sharing have enough special properties like a read only?
There are data structures that are read only.
Are there enough accumulations that you can exploit by reductions?
Are there temporal constraints on data sharing that you can exploit?
And can you deal with those efficiently?
If you can't then you have a problem.
So you need to resolve that.
If the designs OK then you move on to the next design space.
So at the next lecture I'll go through these in a lot more detail.
That's it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So today we're going to continue on with some of the design patterns that we started talking about last week.
So to recap, there are really four common steps to taking a program and then parallelizing it.
Often you're starting off with a program that's designed or written in a sequential manner.
And what you want to do is find tasks in the program -- and these are sort of independent work pieces that you are going to be able to decompose from your sequential code.
You're going to group tasks together into threads or processes.
And then you'll essentially map each one of these threads or processes down to the actual hardware.
And that will get you, eventually when these programs run, the concurrency and the performance speedups that you want.
So as a reminder of what I talked about last week in terms of finding the task or finding the concurrency, you start off with an application.
You come up with a block level diagram.
And from that you sort of try to understand where the time is spent in the computations and what are some typical patterns for how the computations are carried out.
So we talked about task decomposition or sort of independent tasks or tasks that might be different that the application is carrying out.
So in the MPEG encoder, we looked at decoding the motion vectors for temporal compression versus spatial compression.
It does sort of substantially different work.
We talked about data decomposition.
So if you're doing a process -- so if you have some work that's really consuming a large chunk of data, and you realize that it's applying the same kind of work to each of those data pieces, then you can partition your data into smaller subsets and apply the same function over and over again.
So in the motion compensation phase, that's one example where you can replicate the function and split up the data stream in different ways and have these tasks proceed in parallel.
So that's data decomposition.
And then we talked a little bit about sort of making a case for a pipeline decomposition.
So you have a data assembly line or producer-consumer chains, and you essentially want to recognize those in your computation and make it so that you can exploit them eventually when you're doing your mapping down to actual hardware.
But what does it mean for two tasks to actually be concurrent?
And how do you know that you can safely actually run two tasks in parallel?
So there's something I crudely went over last time.
So as to make it more concrete, highlighting Bernstein's condition, which says that given two tasks, if the input set to one task is different from or does not intersect with the output set of another, and vice versa, and neither task sort of updates the same data structures in memory, then there's really no dependency issues between them.
You can run them safely in parallel.
So task T1 and T2, if all the data that's consumed by T1, so all the data elements that are read by T1 are different from the ones that are read by T2, then you have -- you know, if T2 is running in parallel, there's really no problem with T1 because it's updating the orthogonal data set.
Similarly for T2 and T1, any outputs are different.
So as an example, let's say you have two tasks.
In T1 you're doing some basic statements.
And these could be essentially more coarse grained.
There could be a lot more computation in here.
I just simplified it for the illustration.
So you have task a equals x plus y.
And task two does b equals x plus z.
So if we look at the read set for T1, these are all the variables or data structures or addresses these that are read by the first task.
So that's x and y here.
And all the data that's written or produced by T1.
So here we're just producing one data value.
And that's going into location A.
Similarly we can come up with the read set and write set for T2.
And so that's shown on here.
So we have -- task T2 has x plus z in its read set.
And it produces one data value, b.
If we take the intersection of the read and write sets for the different tasks, then they're empty.
I read something completely different than what's produced in this task and vice versa.
And they write to two completely different memory locations.
So I can essentially parallelize these or run these two tasks in parallel.
So you can extend this analysis.
And compilers can actually use this condition to determine when two tasks can be parallelized if you're doing automatic parallelization.
And you'll probably hear more about these later on.
And so what I focused on last time were the finding concurrency patterns.
And I had identified sort of four design spaces based on the work that's outlined in the book by Mattson, Sanders, and Massingill.
And so starting with two large sort of concepts.
The first helps you figure out how you're going to actually express your algorithm.
So first you find your concurrency and then you organize in some way.
And so we're going to talk about that in more detail.
And then once you've organized your tasks in some way that actually expresses your overall computation, you need some software construction utilities or data structures or mechanisms for actually orchestrating computations for which they have also abstracted out some common patterns.
And so I'll briefly talk about these as well.
And so on your algorithm expression side, these are essentially conceptualization steps that help you abstract out your problem.
And you may in fact think about your algorithm expression in different ways to expose different kinds of concurrency or to be able to explore different ways of mapping the concurrency to hardware.
And so for construction it's more about actual engineering and implementation.
So here you're actually thinking about what do the data structures look like?
What is the communication pattern going to look like?
Am I but going to use things like MPI or OpenMP?
What does that help me with in terms of doing my implementation?
So given a collection of concurrent tasks -- so you've done your first step in your four design patterns.
You know, what is your next step?
And that's really mapping those tasks that you've identified down to some sort of execution units.
So threads are very common.
This is essentially what we've been using on Cell.
We take our computation and we wrap it into SPE threads and then we can execute those at run time.
So some things to keep in mind -- although you shouldn't over constrain yourself in terms of these considerations.
What is the magnitude of your parallelism that you're going to get?
You know, do you want hundreds or thousands of threads?
Or do you want something on the order of tens?
And this is because you don't want to overwhelm the intended system that you're going to run on.
So we talked about yesterday on Cell processor, if you're creating a lot more than six threads, then you can create problems or you essentially don't get extra parallelism because each thread is running to completion on each SPE.
And contact switch overhead is extremely high.
So you don't want to spend too much engineering cost to come up with an algorithm implementation that's massively scalable to hundreds or thousands of threads when you can't actually exploit it.
But that doesn't mean that you should over constrain your implementation to where if now I want to take your code and run it on a different machine, I essentially have to redesign or re-engineer the complete process.
So you want to avoid tendencies to over constrain the implementation.
And you want to leave your code in a way that's malleable so that you can easily make changes to sort of factor in new platforms that you want to run on or new machine architecture features that you might want to exploit.
So there are three major organization principles I'm going to talk about.
And none of these should be sort of foreign to you at this point because we've talked about them in different ways in the recitations or in previous lectures.
And it's really, what is it that determines sort of the algorithm structure based on the set of tasks that you're actually carrying out in your computation?
And so there's the principle that says, organize things by tasks.
I'm going to talk to that.
And then there's a principle that says, well, organize things by how you're doing the data decomposition.
So in this case how you're actually distributing the data or how to the data is laid out in memory, or how you're partitioning the data to actually compute on it dictates how you should actually organize your actual computation.
And then there's organize by flow of data.
And this is something you'll hear about more in the next lecture where we're talking about streaming.
But in this pattern if there are specific sort of computations that take advantage of high bandwidth flow of data between computations, you might want to exploit that for concurrency.
And we'll talk about that as well.
OK.
So a design diagram for how can you actually go through process.
So you can ask yourself a set of questions.
if I want to organize things by tasks, then there are essentially two main clusters or two main computations, two main patterns.
If the code is recursive, then you essentially want to apply a divide and conquer pattern or divide and conquer organization.
If it's not recursive.
then you essentially want to do task parallelism.
So in task parallelism -- you know, I've listed two examples here.
But really any of the things that we've talked about in the past fit.
Ray computation, ray tracing.
So here you're shooting rays through a scene to try to determine how to render it.
And really each ray is a separate and independent computation step.
In molecular dynamics you're trying to determine the non-bonded force calculations.
There are some dependencies, but really you can do each calculation for one molecule or for one atom independent of any other.
And then there are sort of the global dependence of having to update or communicate across all those molecules that sort of reflect new positions in the system.
So the common factors here are your tasks are associated with iterations of a loop.
And you can distribute, you know, each process -- each processor can do a different iteration of the loop.
And often you know sort of what the tasks are before you actually start your computation.
Although in some cases, like in ray tracing, you might generate more and more threads as you go along, or more and more computations because as the ray is shooting off, you're calculating new reflections.
And that creates sort of extra work.
But largely you have these independent tasks that you can encapsulate in threads and you run them.
And this is sort of -- it might appear subtle, but there are algorithm classes where not all tasks essentially need to complete for you to arrive at a solution.
You know, in some cases you might convert to an acceptable solution.
And you don't actually need to go through and exercise all the computation that's outstanding for you to say the program is done.
So there will be a tricky issue -- I'll revisit this just briefly later on -- is how do you determine if your program has actually terminated or has completed?
In divide and conquer, this is really for recursive programs.
You know, you can think of a well-known sorting algorithm, merge sort, that classically fits into this kind of picture, where you have some really large array of data that you want to sort.
You keep subdividing into smaller and smaller chunks until you can do local reorderings.
And then you start merging things together.
So this gives you sort of a way to take a problem, divide it into subproblems.
And then you can split the data at some point and then you join it back together.
You merge it.
You might see things like fork and merge or fork and join used instead of split and join.
I've used the terminology that sort of melds well with some of the concepts we use in streaming that you'll see in the next lecture.
And so in these kinds of programs, it's not always the case that each subproblem will have essentially the same amount of work to do.
You might need more dynamic load balancing because each subproblem -- how you distribute the data might lead you to do more work in one problem than in the other.
So as opposed to some of the other mechanisms where static load balancing will work really well -- and to remind you, static load balancing essentially says, you have some work, you assign it to each of the processors.
And you're going to be relatively happy with how each processor's sort of utilization is going to be over time.
Nobody's going to be too overwhelmed with the amount of work they have to do.
In this case, you might end up with needing some things for dynamic load balancing that says, I'm unhappy with the work performance or utilization.
Some processors are more idle than the others, so you might want to essentially redistribute things.
So what we'll talk about -- you know, how does this concept of divide and conquer parallelization pattern work into the actual implementation?
You know, how do I actually implement a divide and conquer organization?
The next organization is organized by data.
So here you have some computation -- not sure why it's flickering
Check your -- maybe your VGA cables aren't in good
So in the organize by data, you essentially want to apply this if you have a lot of computation that's using a shared global data structure or that's going to update a central data structure.
So in molecular dynamics, for example, you have a huge array that records the position of each of the molecules.
And while you can do the coarse calculations independently, eventually all the parallel tasks have to communicate with the central data structure and say, here are the new locations for all the molecules.
And so that has to go into a central repository.
And there are different kinds of sort of decompositions within this organization.
If your data structure is recursive, so a link list or a tree or a graph, then you can apply the recursive data pattern.
If it's not, if it's linear, like an array or a vector, then you apply geometric decomposition.
And you've essentially seen geometric decomposition.
These were some of the labs that you've already done.
And so the example from yesterday's recitation, you're doing an end body simulation in terms of who is gravitating towards who, you're calculating the forces between pairs of objects.
And depending on the force that each object feels, you calculate a new motion vector.
And you use that to update the position of each body in your, say, galaxy that you're simulating.
And so what we talked about yesterday was given an array of positions, each processor gets a sub-chunk of that position array.
And it knows how to calculate sort of locally, based on that.
And then you might also communicate local chunks to do more scalable computations.
And recursive data structure are a little bit more tricky.
So at face value you might think that there's really no kind of parallelism you can get out of a recursive data structure.
So if you're iterating over a list and you want to get the sum, well, you know, I just need to go through the list.
Can I really parallelize that?
There are, however, opportunities where you can reshape the computation in a way that exposes the concurrency.
And often what this comes down to is you're going to do more work, but it's OK because you're going to finish faster.
So this kind of work/concurrency tradeoff, I'm going to illustrate with an example.
So in this application we have some graphs.
And for each node in a graph, we want to know what is its root?
So this works well when you have a forest where not all the graphs are connected and given a node you want to know who is the root of this graph.
So what we can do is essentially have more concurrency by changing the way we actually think about the algorithm.
So rather than starting with each node and then, in a directed graph, following its successor -- so this is essentially order n, because for each node we have to follow n links -- we can think about it slightly differently.
So what if rather than finding the successor and then finding that successor's successor, at each computational step we start with a node and we say who is your successor's successor?
So we can converge in this example in three steps.
So from five to six we can say who is this successor?
So who is the successor's successor of five?
And that would be two.
And similarly you can do that for seven and so on.
And so you keep asking the question.
So you can distribute all these data structures, repeatedly ask these questions out of all these end nodes, and it leads you to an order log n solution versus an order n solution.
But what have I done in each step?
Well, I've actually created myself and I've sort of increased the amount of work that I'm doing by order n. Right there.
Right.
Yes.
Because I've essentially for each node doing n queries, you know, who's your successor's successor?
Whereas in a sequential case, you know, I just need to do it once for each node.
And that works really well.
So most strategies based on this pattern of actually decomposing your computation according to recursive pattern lead you to doing much more work or some increase in the amount of work.
But you get this back in because you can decrease your execution.
And so this is a good tradeoff that you might want to consider
In the first one order n was sequential?
Yeah, yeah.
It's a typo.
Yeah.
So organize by flow or organize by flow of data.
And this is essentially the pipeline model.
And we talked about this again in some of the recitations in terms of SPE to SPE communication.
Or do you want to organize based on event-based mechanisms?
So what these really come down to is, well, how regular is the flow of data in your application?
If you have regular, let's say, one-way flow through a stable computation path -- so I've set up my sort of algorithm structure.
Data is flowing through it at a regular rate.
The computation graph isn't changing very much.
Then I can essentially pipeline things really well.
And this could be a linear chain of computation or it could be sort of nonlinear.
There could be branches in the graph.
And I can use that in a way to exploit pipeline parallelism.
If I don't have sort of this nice, regular structure, it could be events that are created at run time.
So, for example, you're a car wash attendant and a new car comes in.
So you have to find a garage to assign to it and then turn on the car wash machine.
So the dynamic threads are created based on sensory input that comes in, then you might want to use an events-based coordination.
You have irregular computation.
The computation might vary based on the data that comes into your system.
And you might have unpredictable data flow.
So in the pipeline model, the things to consider is the pipeline throughput versus the pipeline latency.
So the amount of concurrency in a pipeline is really limited by the number of stages.
This is nothing new.
You've seen this, for example, in super scaled pipelines.
And just as in this case, as in the case of an architecture pipeline, the amount of time it takes you to fill the pipeline and the amount of time it takes you to drain the pipeline can essentially limit your parallelism.
So you want those to be really small compared to the actual computation that you spend in your pipeline.
And the performance metric is usually the throughput.
How much data can you pump through your pipeline per unit time?
So in video encoding, you know, it's the frames per second that you can produce.
And the pipeline latency, though, is important, especially in a real-time application where you need a result every 10 milliseconds.
You know, your pacemaker for example has to produce a beep or a signal to your heart at specific rates.
So you need to consider what is your pipeline throughput versus your pipeline latency?
And that can actually determine how many stages you might want to actually decompose or organize your application in.
And in the event-based coordination, these are interactions of tasks over unpredictable intervals.
And you're more prone to sort of deadlocks in these applications.
Because you might have cyclic dependencies where one event can't proceed until it gets data from another event.
But it can't proceed until it gets data from another event.
You can create sort of these complex interactions that often lead to deadlock.
So you have to sort of be very careful in structuring things together so you don't end up with feedback loops that block computation progress.
So given sort of these three organizational structures that say, you know, I can organize my computation by task or by the flow of data or by sort of the pipeline nature of the computation, what are the supporting structures?
How do I actually implement these?
And so there are many different supporting structures.
I've identified sort of four that occur most often in literature and in books and common terminology that's used.
And so those are SPMD, loop parallelism, the master/worker pattern, and the fork/join pattern.
In the SPMD pattern, you're talking about a single program, multiple data concept.
So here you just have one program.
You write it once and then you assign it to each of your processors to run.
So it's the same program.
It just runs on different machines.
Now each program or each instance of the code can have different control flow that it takes.
So just because they're running the same program doesn't mean the computation is happening in lock step.
That would be a sort of a SIMD or vector-like computation.
In this model you can actually take independent control flow.
It could be different behavior in each instance of the code.
But you're running the same code everywhere.
So this is slightly different, for example, from what you've seen on Cell, where you have the PPE thread that creates SPE threads.
Sometimes the SPE threads are the same, but it's not always the case that the PPE threads and the SPE threads are the same.
So in the SPMD model there are really five steps that you do.
You initialize sort of your computation in the world of sort of code instances that you're going to run.
And for each one you obtain a unique identifier.
And this usually helps them being able to determine who needs to communicate with who or ordering dependencies.
And you run the same program on each processor.
And what you need to do in this case is also distribute your data between each of the different instances of your code.
And once, you know, each program is running, it's computing on its data, eventually you need to finalize in some way.
And so that might mean doing a reduction to communicate all the data to one processor to actually output the value.
And so we saw in SPMD an example for the numerical integration for calculating pi.
And if you remember, so we had this very simple c loop.
And we showed the MPI implementation of the c loop.
And so in this code, what we're doing is we're trying to determine different intervals.
And for each interval we're going to calculate a value and then in the MPI program we're essentially deciding how big an interval each process should run.
So it's the same program.
It runs on every single machine or every single processor.
And each processor determines based on its ID which interval of the actual integration to do.
And so in this model we're distributing work relatively evenly.
Each processor is doing a specific chunk that starts at say some index i.
And if I have 10 processors, I'm doing 100 steps.
Then you're doing i, i plus 10, i plus 20 and so on.
But I can do a different distribution.
So the first is a block distribution.
I can do something called a cyclic distribution.
So in a cyclic distribution, I distribute work sort of in a round robin fashion or some other mechanism.
So here, you know, each processor -- sorry.
In the block distribution I sort of start at interval i and I go -- sorry.
So each processor gets one entire slice here.
So I start here and I go through to completion.
I start here and go through to completion.
In a cyclic distribution I might do smaller slices of each one of those intervals.
And so I greyed out the components for the block distribution to show you that for a contrast here.
There are some challenges in the SPMD model.
And that is how do you actually split your data correctly?
You have to distribute your data in a way that, you know, doesn't increase contention on your memory system, where each actual processor that's assigned the computation has data locally to actually operate on.
And you want to achieve an even work distribution.
You know, do you need a dynamic load balancing scheme or can you use an alternative pattern if that's not suitable?
So the second pattern, as opposed to the SPMD pattern is loop parallelism pattern.
In this case, this is the best suited when you actually have a programming model or a program that you can't really change a whole lot or that you don't really want to change a whole lot.
Or you have a programming model that allows you to sort of identify loops that take up most of the computation and then insert annotations or some ways to automatically parallelize those loops.
So we saw in the OpenMP example, you have some loops you can insert these pragmas that say, this loop is parallel.
And the compiler in the run-time time system can automatically partition this loop into smaller chunks.
And then each chunk can compute in parallel.
And you might apply this scheme in different ways depending on how well you understand your code.
Are you running on a shared memory machine?
You can't afford to do a whole lot of restructuring.
Communication costs might be really expensive.
In the master/worker pattern, this is really starting to get closer to what we've done with the Cell recitations in the Cell labs.
You have some world of independent tasks and the master essentially running and distributing each of these tasks to different processors.
So in this case you'd get several advantages that you can leverage.
If each of your tasks are varied in nature -- and they might finish at different times or they require different kinds of resources, you can use this model to sort of view your machine as sort of a non-symmetric processor.
Not everybody is the same.
And you can use this model really well for that.
So you can distribute these and then you can do dynamic load balancing.
Because as processors -- as workers finish you can ship them more and more data.
So it has some particularly relevant properties for heterogeneous computations, but it's also really good for when you have a whole lot of parallelism in your application.
So something called embarrassingly parallel problems.
So ray tracing, molecular dynamics, a lot of scientific applications have these massive levels of parallelism.
And you can use this essentially work-queue based mechanism that says I have all these tasks and I'll just dispatch them to workers and compute.
And as I pointed out earlier, you know, when do you define your entire computation to have completed?
You know, sometimes you're computing a result until you've reached some result.
And often you're willing to accept a result within some range of error.
And you might have some more threads that are still in flight.
Do you terminate your computation then or not?
What are some issues with synchronization?
If you have so many threads that are running together, you know, does the communication between them to send out these control messages say, I'm done, start to overwhelm you?
In the fork/join pattern -- this is really not conceptually too different in my mind from the master/worker model, and also very relevant to what we've done with Cell.
The main difference might be that you have tasks that are dynamically created.
So in the embarrassingly parallel case, you actually know the world of all your potential task that you're going to run in parallel.
In the fork/join join model some new computation might come up as a result of, say, an event-based mechanism.
So a task might be created dynamically and then later terminated or they might complete.
And so new ones come up as a result
It almost seems like you are forking the task in the forking model.
And then keep assigning tasks to that.
The fork/join model you just keep forking at first virtual box.
Might not be completely matched to a number of processor available.
fork them out
So the process that's equating all these threads or that's doing all the forking is often known as the parent and the tasks that are generated are the children.
And eventually essentially the parent can't continue or can't resume until its children have sort of completed or have reached the join point.
And so those are really some of the models that we've seen already, in a lot of cases in the recitations and labs for how you run your computations.
And some of you have already discovered these and actually are thinking about how your projects should be sort of parallelized for your actual Cell demos.
Some of the other things that I'm just going to talk about are communication patterns.
So two lectures ago you saw, for example, that you have point to point communication or you have broadcast communication.
So in point to point communication, you have two tasks that need to communicate.
And they can send explicit messages to each other.
These could be control messages that say I'm done or I'm waiting for data.
Or they could be data messages that actually ships you a particular data element that you might need.
And again we've seen this with Cell.
Broadcast says, you know, I have some result that everybody needs.
And so I send that out to everybody by some mechanism.
There is no real broadcast mechanism on Cell.
The concept I'm going to talk about though is the reduction mechanism, which really is the inverse of the broadcast.
So in the broadcast I have a data element I need to send to everybody else.
In the reduction, all of you have data that I need or all of us have data that each somebody else needs.
So what we need to do is collectively bring that data together or group it together and generate an end result.
So a simple example of a reduction, you have some array of elements that you want to add together.
And sort of the result of the collective operation is the end sum.
So you have an array of four elements, A0, A1, A2, and A3.
And you can do a serial reduction.
I can take A0 and add it to A1.
And that gives me a result.
And I can take A2 and add that to it.
And I can take A3 and add that to it.
And so at the end I'll have sort of calculated the sum from A0 to A3.
So this is essentially -- the serial reduction applies when your operation is an associative.
So the addition is associative.
So in this case I can actually do something more intelligent.
And I think we talked about that last time.
I'm going to show you some more examples.
And often sort of the end result follows a broadcast.
It says, here is the end result.
Who are all the people that need it?
I'll sort of broadcast that out so that everybody has the result.
If your operation isn't associative, then you're essentially limited to a serial process.
And so that's not very good from a performance standpoint.
Some of the tricks you can apply for actually getting performance out of your reduction is to go to a tree-based reduction model.
So this might be very obvious.
Rather than doing A0 and A1 and then adding A2 to that result, I can do A0 and A1 together.
In parallel I can do A2 and A3.
And then I can get those results and add them together.
So rather than doing n steps I can do log n steps.
So this is particularly attractive when only one task needs the result.
So in the MPI program when we're doing the integration to calculate pi, you know, one processor needs to print out that value of pi.
But if you have a computation where more than one process actually needs the result of the reduction, there's actually a better mechanism you can use that's sort of a better alternative to the tree-based reduction followed by a broadcast.
So you can do a recursive doubling reduction.
So at the end here, every process will have the result of the reduction without having done the broadcast.
So we can start off as with the tree-based and add up A0 and A1 together.
But what we do is for each process that has a value, we sort of do a local exchange.
So from here we communicate the value to here.
And from here we communicate the value to here.
And so now these two processors that had the value independently now both have a local sum, A0 to A1.
And similarly we can sort of make the similar symmetric computation on the other side.
And now we can communicate data from these two processors here to come up with the end --
It was there.
All right.
Must have been lost in the animation.
So you actually do that the other way as well so that you have the sum A0 to A3 on all the different processors.
Sorry about the lost animation.
OK.
So this is better than the tree-based approach with a broadcast because you end up with local results of your reduction.
And rather than doing the broadcast following the tree-based reduction which takes n steps, you end up with an order n. Everybody has a result in order n versus an order 2n process for the tree-based plus broadcast
On the Cell processor but not in general
Not in general.
It depends on sort of the architectural mechanism that you have for your network.
If you actually do need to sort of, you know, if you have a broadcast mechanism that has bus-based architecture where you can deposit a local value, everybody can pull that value, then, yeah, it can be more efficient.
Or on optical networks, you can broadcast the data and everybody can just fuse it out.
OK.
So summarizing all the different patterns, so here these are the actual mechanisms that you would use for how you would implement the different patterns.
So in the SPMD you would write the same program.
In loop parallelism you have your program and you might annotate sort of some pragmas that tell you how to parallelize your computation.
In the master/worker model you might have sort of a master that's going to create threads and you actually know -- you might sort of have a very good idea of what is the kind of work you're going to have to do in each thread.
In the fork/join model you have more dynamism.
So you might create threads on the fly.
And you apply these sort of based on appeal or what is more suited in terms of implementation to each of the different patterns for how you actually organize your data.
So in the task parallelism model, this is where you have a world of threads that you know you're going to calculate or that you're going to use for your computation.
And really you can use largely any one of these models.
So I used a ranking system where four stars is really good.
One star is sort of bad or no star means not well suited
Sort of in Cell because the inherit master there.
Sometimes master/worker might get a little bit of a biasing than this one
Right, so --
You don't have to pay a cost of having master
Right.
Right.
Although you could use the Cell master to do regular computations as well.
But, yes.
So and the divide and conquer model, you know, might be especially well suited for a fork and join because you're creating all these recursive subproblems They might be heterogeneous.
In the nature of the computation that you do, you might have more problems created dynamically.
Fork/join really works well for that.
And the fact, you know, the subproblem structure that I showed, the graph of sort of division.
And then merging works really well with the fork/join model.
In the recursive, in the geometric decomposition -- this is essentially your lab one exercise and the things we went over yesterday in the recitation.
You're taking data and you're partitioning over multiple processors to actually compute in parallel.
So this could be SPMD implementation or it could be a loop parallelism implementation, which we didn't do.
Less suitable, the master/worker and fork/join, often because the geometric decomposition applied some distribution to the data which has static properties that you can exploit in various ways.
So you don't need to pay the overhead of master/worker or fork/join.
Recursive data structures sort of have very specific models that you can run with.
Largely master/worker is a decent implementation choice.
SPMD is another.
And you're going to hear more about sort of the pipeline mechanism in the next talk so I'm not going to talk about that very much.
Event-based coordination, largely dynamic.
So fork/join works really well.
So one --
When you're buffering them you could do master/worker with pipelining?
Yes, so next slide.
So sort of these choices or these tradeoffs aren't really orthogonal.
You can actually combine them in different ways.
And in a lot of applications what you might find is that the different patterns compose hierarchically.
And you actually want that in various ways -- for various reasons.
So in the MPEG example, you know, we had tasks here within each task and identified some pipeline stages.
You know, here I have some data parallelism so I can apply the loop pattern here.
And what I want to do is actually in my computation sort of express these different mechanisms so I can understand sort of different tradeoffs.
And for really large applications, there might be different patterns that are well suited for the actual computation that I'm doing.
So I can combine things like pipelining with a task-based mechanism or data parallelism to actually get really good performance speedups.
And one of the things that might strike you as well, heck, this is a whole lot of work that I have to do to actually get my code in the right way so that I can actually take advantage of my parallel architecture.
You know, I have to conceptually think about the question the right way.
I have to maybe restructure my computation in different ways to actually exploit parallelism.
Data distribution is really hard.
I have to get that right.
Synchronization issues might be a problem.
And how much buffering do I need to do between different tasks?
So the thing you're going to hear about in the next talk is, well, what if these things really fall out naturally from the way you actually write the program, and if the way you actually write your program matches really well with the intuitive, sort of natural conceptualization of the problem.
And so I'll leave Bill to talk about that.
And I'm going to stop here.
Any questions?
We can take in some questions and then everybody --
You talked about fork and join.
When you have a parent thread that spawns off to a child thread, how do you keep your parent thread from using up the SPE?
So you have a fork/join where you have --
Most of the parents it might be the PPE.
And so if you just do fork/join, might not really use PPE unless you can, you know, you have some time and you let it do some of the task and come back
So for our purposes we shouldn't spawn off new threads by the SPEs?
So, yeah.
So most of the threads that are spawned off are done by the PPE.
So you have these -- in fact a good walk through in recitation yesterday.
You have the PPE.
Essentially it sends messages to the SPEs that says, create these threads and start running them.
Here's the data for them.
And then these threads run on the SPEs.
And they just do local computation.
And then they send messages back to the PPE that says, we're done.
So that essentially implements the join mechanism
On the other hand, if you are doing something like master slave way, and then the SPE can send a message and deliver another job into the PPE who feeds the master.
If SPE see there's some more computers, you can say, OK, look, put this into your keybord and keep sending messages and so the master can look at that and update it.
So, you know, it's not only master who has to fork off but the slaves also.
They still can send information back.
So you can think about something like very confident that way.
There are eight -- like if six SPE is running and you first get something in there and SPE says divide it will take one task and run that until the other one to the master will finish it and here's my ID.
Send me the message when it's done.
And so you fork that end and wait.
So you can assume you can do something like that.
So it's almost master/slave but the coordination is there.
The trouble with normally fork/join is if you create too many threads.
You are in like a thread hell because there are too many things to run.
I don't know, can you SPE?
No
So you can't even do that because of some physical limitation.
You can't get take up 1000 threads you run another master/slave thing yourself is because 1000 threads on top of your SPEs.
And that's going to be locked threads
Yeah.
Contact switching on the SPEs is very expensive.
So on the PlayStation 3 you have six SPEs available to you.
So if you have a lot more than six threads that you've created, essentially each one runs to completion.
And then you swap that out and you bring in -- well, that terminates.
You deallocate it from the SPE and you bring in a new thread.
If you actually want to do more thread-like dynamic load balancing on the SPEs, it's not well suited for that.
Just because the --
The best model there is master/slave.
Because the PPE [UNINTELLIGIBLE PHRASE] the master part.
It will run more sequential code.
And when there's parallel send -- it will give it to you and produce the work queue model type and send stuff into SPE and feed that.
So work queue type models can be used there
Yeah.
And the SPMD model might not work really well because you have this heterogeneity in the actual hardware, right.
So if I'm taking the same program running on the SPE versus the PPE, that code might not be -- so I essentially have to specialize the code.
And that starts to deviate away from the SPMD model
Something I think most of the code you write for Cell will probably be master/worker.
And if you try to do something other than you should think hard why that's the case
You can do fork/join but you know, it's --
I mean you can't -- because you don't have virtualization.
If you fork too much where are you going to put those?
Right.
Sometimes you fork --
Yeah.
but in that sense you -- should you fork too much?
To keep work you want the master.
You can fork things -- you can do virtual fork and send the work to the master and say, here, I forked something.
Here's the work.
I mean, the key thing is do the simplest thing.
I mean, you guys have two weeks left.
And if you try doing anything complicated, you might end up with a big mess that's undebuggable.
Just do simple things.
And I can vouch, parallelism is hard.
Debugging parallel code is even harder.
So you're sort of trying to push the limits on the complexity of messages going all over the world and the three different types of parallelism all trying to compete in there.
Just do the simple thing.
Just get the simple thing working.
First get the sequential code working and keep adding more and more story.
And then make sure that each level it works.
The problem with parallelism is because things that determine if some bugs that might show up.
Data might be hard.
But design absolutely matters.
Another thing I think, especially for doing demos and stuff would be nice, would be to have a knob that basically you can tune.
So you can say, OK, no SPEs.
Everything running in PPE one is two is.
So you can actually see hopefully in your code how how things move for the demo part
You had a question?
All right.
We'll take a brief break and do the quizzes in the meantime.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resource for free.
To make a donation you or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
So let's get started with the second lecture for today.
So I guess one thing multicores did, is really shatter this nice view of writing in your programs and hardwares to take care of, giving you performance.
So hardware just kind of completely gave that up.
But so what you're doing in this class, is you're trying to do it by yourself.
Give all the responsibility back to the program.
And you realize as you go, it's a much harder job.
I mean, this is not simple programming.
So you need to have, you don't have MIT class students on every company to do this, so we need to have some kind of middle ground.
And so some of the stuff we have been doing is trying to figure out are there any middle ground.
Can you actually take some of that load away from the user into things like languages and compilers.
So we will talk about some of those.
So right now we are kind of switching from directly doing what's necessary, to do the Cell project into going breadth So this lecture, and then we will sit back and do a little bit of debugging and performance work and that will be directly helpful.
And then next week we'll have lots of guest lectures to kind of give you breadth in there.
So you'll understand no just Cell programming but parallel programming and parallel processing, what the world is like beyond that.
So today we're going to have Bill talk about streams
OK very good.
So my name is Bill Thies.
I'm a graduate student working with Saman and Roderick, and others here.
And I'll talk about the StreamIt language.
So why do we need a new programming language?
Well we think that languages haven't kept up with the architectures.
So one way to look at this is that if you look back at previous languages, look at C with von-Neumann machine.
Now I grew up in rural Pennsylvania not too far from Amish Country.
And so to me these go together just like a horse and buggy.
OK they're perfectly made for each other.
They basically go at the same rate.
Everything is fine.
But the problem is, in comes the modern architecture.
OK this is an F-16.
you have a lot more that you can do with it then, then with the horse and buggy.
So how do you program these new architectures?
Well architecture makers these days are basically faced with a really hard choice.
On the one hand, you could get a really cool architecture and develop an ad hoc programming technique where you're really just leaving it to the programmer to do something complicated to get performance.
And unfortunately I think that's the route that they took.
I mean fortunately for the industry, but unfortunately for you, I think that's the route they took with cell which means all of you are going to become basically fighter pilots.
You have to learn how to fly the plane.
You have to become an expert.
You're going to become the best people at programming these architectures.
And unfortunately the only other option is to really bend over backwards to support the previous era of languages like C and C .
And you can see what's coming here, it's just hard to get off the runway.
So you don't want this situation.
Out of consideration for whoever is in the buggy, hopefully you'll never take off.
So looking from a more academic perspective, why do we need a new language right now?
So if you look back over the past 30 years, I know you've seen this graph before.
We were dealing with just one core in the machine for all this time.
And now we have this plethora of multicores coming across the board.
So how did we program these old machines?
Well we had languages like C and FORTRAN that really have a lot of nice properties across these architectures.
So it was portable, high-performance, composable-- you could have really good software development-- malleable, maintainable, all the nice things you'd like to see from a software engineering perspective.
And really if you wrote a program back in 1970, you could keep it in C and have it continue to leverage all the new properties of machines over the past 30 years.
So just one fine out of the box.
And looking forward, that's not going to be true.
So for example, we could say that C was the common machine language.
That's what we say for the past 30 years, was common across all the machines.
But now looking forward, that's not going to be true anymore.
Because you have to program every core separately.
So what's the common machine language for multicores?
We really think you need something where you can write a program once today, and have it scale for the next 30 years without having to modify the program.
So what kind of language do you need to get that kind of performance?
Well let's look a little deeper into this notion of a common machine language.
So why did it work so well for the past 30 years?
Well on uniprocessors, things like C and FORTRAN fran really encapsulated the common properties.
So things like a single flow of control in the machine, a single memory image, are both properties of the language.
But they also hid certain properties from the programmer.
So they hid the things that were different between one machine and another.
So for example, the register file, the ISA, the functional units and so on.
These things could change from one architecture to another.
And you didn't have to change your program because those aspects weren't in the programming language.
So that's why these languages were succeeding.
And what do we need to succeed in the multicore era from a language perspective?
Well you need to encapsulate the common properties again.
And this time it's multiple flows of control that you have for all the different cores, and multiple local memories.
There's no more monolithic memory anymore, that everyone can read and write to.
Also you need to hide some of the differences between the machines.
So some cores have different capabilities.
On cell there's a heterogeneous system between the STEs and the PPE.
Different communication models on different architectures, different synchronization models.
So whatever common machine language we come up with, we'll have to keep these things hidden from the programmer.
Now a lot of different researchers are taking different tacts for how you want to invent the next common machine language.
And the thrust that we're really excited about is this notion of streaming.
So what is a stream program?
Well if you look at a lot of the high-performance systems today-- including Powerpoint which is running this awesome animation-- you can basically see that they're based around some stream of data.
So audio, video, like HDTV, video editing, graphic stuff.
I think actually, a lot of the projects in this class that I looked at, would fit into the streaming mold.
Things like the software radio, array tracing, I probably don't remember them all.
But when I looked at them, they all looked like they had a streaming component somewhere in there.
So what's special about a stream program compared to just a normal program?
Well they have a lot of attractive properties.
If you look at their structure, you can usually see that the computation pattern remains relatively constant across the lifetime of the program.
So they have some well-defined units that are communicating with each other.
And they continue that pattern of communication throughout.
And this really exposes a lot of opportunities for the compiler to do some optimizations that it couldn't do on just an arbitrary general purpose program.
And if you saw before, we have basically all the types of parallelism are really exposed in a stream program.
There's the pipeline parallelism between different producers and consumers.
There's the task parallelism basically going from left to right.
And also data parallelism which means that a single one of these stages can sometimes be split to apply to multiple elements in the data stream.
So when you're thinking about stream programming, there's a lot of different ways you can actually represent the program.
So whenever you have a programming model, you have to answer these kinds of questions.
For example do the senders and the receivers block when they try to communicate?
How much buffering is allowed?
Is the computation deterministic?
What kind of model do you have in there?
Can you avoid deadlock?
Questions like these, and we could spend a whole lecture answering these questions, putting them in different categories.
But what I want to just to do, just to give you a feel is touch on kind of three of the major models that you might see come up in different kinds of programming models.
And I'll just touch Kahn process networks, synchronous dataflow, and communicating sequential processes, or CSP.
So just one slide on these models.
So let's compare them a little bit.
First there's the Kahn process networks.
So this is kind of the simplest model.
It's very intuitive.
You just have different processes that are communicating over FIFOs.
And the FIFO size is conceptually unbounded.
So to a first approximation, it's kind of like a Unix pipe.
These processes can just read from the input, and they can push onto their outputs without blocking.
But if they try to read from an input they do block until an input is available.
And the interesting thing is that the communication pattern can actually be dependent on the data.
So for example I could pop an index off of one channel, and then use that index to determine which other channel I'll read from on the next time time step.
But at the same time it is deterministic.
So for a given series of input values on the stream, I'll always have the same communication pattern that I'm trying from the other input.
So if it's a deterministic model, that's a nice property.
Let's see, what else to say here?
There's actually a few recent ventures that are using Kahn process networks.
So there's commercial interest.
For example Ambric is a startup that I think will be based on a Kahn process network for the programming model.
Looking at another model called synchronous dataflow, this is actually what we use in the StreamIt system.
And compared to Kahn process networks, it's kind of a subset.
It's a little bit more restrictive.
So if you look at the space of all possible program behaviors, Kahn process networks are a pretty big piece of the space.
And then synchronous dataflow is kind of a subset of that space where you know more about the communication pattern at compile time.
So for example, in synchronous dataflow, the programmer actually declares how many items it will consume from each of its in put channels on a given execution step.
So there's no more data dependence regarding the communication pattern.
It'll always input some items from some of the channels and produce some number of items to other channels.
And this is a really nice properties because it lets the compiler do to scheduling for you.
So the compiling can see who's communicating to who and exactly what pattern.
And it can statically interleave the filters to guarantee that everyone has enough data to complete their computation.
So there's a lot of interesting optimizations you can do here.
That's why it's very attractive for StreamIt.
And you can statically guarantee freedom from deadlock, which is a nice property to have.
The last one I want to touch on is communicating sequential processes or CSP.
And in the space of program behaviors, it's kind of an overlapping that from Kahn processing networks, and adds a few new semantic behaviors.
So the buffering model is basically rendezvous communication now.
So there's no bothering in the system.
Basically anytime you send a value to another process, you have to block and wait until that process will actually receive that value from you.
So everyone is rendevouzing at every communication step.
In addition to that, they have some sophisticated synchronization primitives.
So you can for example, discuss alternative behaviors that you have.
You can either one thing or another which will introduce the nondeterminism in the model.
Which could be a good or a bad thing depending on the program you're trying to express.
And pretty much the most well-known encapsulation of CSP is this occam programming language invented quite a while ago.
And some people are still using that today.
Any questions on the model computations?
OK.
So now let me get into what StreamIt is.
So StreamIt is a great language.
It's a high-level architecture-independent language.
Oh question the back
With the CSP I'm trying to understand exactly what that means or how it's different.
Is basically what's it's saying is you have a bunch processes and they can send messages to each other
So all these models have that property
They all fit into that.
But it seems like from your explanation of CSP, that that was just sort of the essence of CSP is it more specific?
So CSP is usually associated with rendezvous communication.
That's the side of programs that fit inside Kahn process networks.
It's any communicating model where you basically have no buffering between the processes.
Now the piece that sits outside is usually lumped with CSP, or especially with occam They have a set a primitives that are richer in terms of synchronization.
So for example, you can have guards on your communication.
Don't execute this consumption from this channel until I see a certain value.
So there's some more rich semantics there.
And so that's the things that are usually outside.
They're outside the other models.
Does that make sense?
Other questions?
OK so StreamIt.
OK so StreamIt is architecture-independent.
It's basically a really nice syntactic model for interfacing with these lower level models of computation for streaming.
And really we have two goals in the StreamIt project.
And the first is from the programmer's side.
So we want to improve the programmer's life when you're writing a parallel program.
We want to make it easier for you to write a parallel program then you would have to do in C or a language like Java, or any other language that you know.
And at the same time, we want scalable and portable performance across the multicores.
So an interesting thing these days is you'll find, is it's often very hard a tempt the programmer to switch to your favorite language based solely on performance.
Or at least this has been the story in the past.
It may change, looking forward.
Because it's a lot harder to get performance these days.
But usually you have to offer them some other carrot to get them on board.
And you know the carrot here is that it's really nice to program in.
It's fun to program in.
It's beautiful.
It's a lot easier to program and stream it then it would be in something like C or Java for a certain class of programs.
So that's how we get them on board, and then we also provide the performance.
We're mostly based on the synchronous in dataflow model.
In that when there are static communication patterns, we leverage that from the compiler side.
So I'll also tell you about some dynamic extensions that we have, that is the much richer model of communication.
So what have we been doing in the Streamit Project?
We have kind of a dual thrust within our group building on this language.
So the first thrust is from the programmability side, looking at applications and programmability What can we fit into the streaming model?
And we're also really pushing the optimizations.
So what can you do from both a domain specific optimization standpoint, as kind of emulating a DSP engineer or a signal processing expert in the design flow.
And also architecture specific optimizations.
So we've been compiling for a lot of parallel machines.
And we were hoping we could have a full system for you guys, this IEP, so you could write it then stream it and then hit the button.
And it would work the whole way down to cell.
Unfortunately we're not quite there yet.
But we do have a pretty robust compiler infrastructure.
And you can download this off the web and play with it if you want to.
One of our backends that we've released so far actually does go to a cluster of workstations.
So it's kind of an MPI-like version of C. It uses Pthreads for the parallelism model.
And I mean, depending on what kind of a hacker you are, you actually might be able to lower that down onto cell.
So some of the stuff you might be able to use if you're have some initiative in there.
And of course we'd be willing to work with you on this as well.
so we have lots optimizations in the tool flow.
And actually Saman will spend another lecture focusing on the StreamIt compiler, and how we get performance out of the model.
OK, so let's just jump right in and do the analog of Hello World in StreamIt.
I'm going to kind of walk you through the language and show you the interesting pieces from an intellectual standpoint.
What's interesting about a streaming model that you can take away.
So instead of Hello World, we have a counter.
Since we're dealing with stream programs here, you're not usually doing text processing.
So how do you write counter?
Well there are two pieces to the program.
The first is kind of the interconnect between the different components.
That's what we have up here.
We're saying the program is a pipeline with two stages, it has a source, and it has a printer.
And then we can write the source and the printer as filters.
We call those basic building blocks filters in Streamit.
So the source will just have a variable x that it initializes is zero.
And then we have a work function which is automatically called by our runtime system every time through the steady state.
So this work function well push one item on to the output channel, and it'll increment the value afterward.
Whereas the intPrinter at the bottom here, will input one value.
And its work function here just pops that value off the input tape, and prints it to the output.
Now how do we run this thing?
Well there's no main function here like you see in Hello World.
Is there comment?
Oh, sorry
The two meanings of push and two meanings of pop
Two meanings?
Push 1, 2, 3
Yeah, yeah.
So the first push here is just declaring that this work function will push one item to the output tape.
So this is the synchronous dataflow aspect.
Were associating an output rate with this work function.
So that's a declaration here.
Whereas this push is just actually executing the push onto the output.
So how do we run this thing?
Well we compile it with a StreamIt compiler, store C into a binary.
And then when we run, we run for a given number of iterations.
So you don't just call it once.
Our model is that this is a continuous stream of data going through the program.
And so when you run it, you run it for some number of iterations, or basically input or output items, is what you're running it for.
So if you run this for four iterations, it would print in the 033.
So we can leverage this steady flow of data.
Yeah Amir?
1, 2, 3, 4, pushing X plus plus
I think the plus, plus is executed after the push
Push plus plus?
So it starts at zero.
So I think a PostFix expression executes after the actual obsession.
Yeah.
Yeah.
Other questions?
OK so let's step up a level and look at what we have in StreamIt.
So the first question is, how do you represent this connectivity between different building blocks?
How do you represent streams?
And if you look at traditional programming models, kind of the conventional wisdom is that a stream program is a graph.
You have different nodes that are communicating to each other.
And graphs are actually kind of hard to analyze.
They're hard to represent.
They're a little but confusing.
So the approach we decided to take in StreamIt is one of a structured computation graph.
So instead of having arbitrary inner connections between the stages, we have a higher hierarchical description in which every individual stage has a single input and a single output.
And you can compose these together into higher level stages.
Of course there's some pages that do split and join with multiple inputs.
We'll get to that.
So the analog here is kind of ah analogous to structured control flow, in your favorite imperative programming language.
Of course there was a day when everyone used goto statements instead of having structure control flow.
We've got a fan of goto statements in the audience?
OK, I'll get to you later.
But the problem was, it's really hard to understand the program that's jumping all over the place because there's no local reasoning you can have.
You know you're jumping to this location, you're coming back a different way.
It's hard to reason about program components.
So when people went to structured control flow, there's just if else, four loop statements.
Those are the two basic constructs.
You can basically express all kinds of computation in those simple primitives.
And things got a lot simpler.
And you know people objected at one point even.
You know what about a finite-state machine?
Don't you need goto statements for a finite-state machine, going from one state to another another.
And now everyone writes in FSM with a really simple idiom.
You usually have a while loop around a case statement.
Right, you have a dispatch loop.
So and now whenever you see that pattern you can recognize, oh there's a finite-state machine.
It's not just at set of gotos.
It's a finite-state machine.
So we think there are similar idioms in the streaming domain.
And that's kind of the direction we're pushing from a design standpoint.
So what are our structures that we have?
Well here are our structured streams.
At the base we have a filter.
That's just the programmable unit like I showed you.
We have a pipeline, which just connects one stream to another in a sequence.
So this gives you pipeline parallelism.
There's a splitjoin where you have explicit parallelism in the stream.
So I'll talk about what these splitters and joiners can do.
It's basically a predefined pattern of scattering data to some child streams, and then gathering that data back into a single stream.
So the whole construct still remains single input and single output.
Likewise a feedback loop is just a simple way to put a loop in your stream.
And of course these are hierarchical.
So all of these green boxes can be any of the three constructs.
So that's how you can have these hierarchical graphs.
And again, since everything is single-input single-output, you can really mix and match.
You know choose your favorite components, and they'll always fit together.
You don't need to stitch multiple connections.
So let's dive inside one of these filters now.
And I gave you a feel for how they look before, but here's a little more detail.
So how do we program the filter?
Well a filter just transforms one stream into another stream.
And here it's transforming a stream of floating-point numbers into another floating-point number stream.
I can take some parameters at the top.
These actually fixed at compile time in our model, which allows the compiler to really specialize the filters code depending on the context in which it's being used.
So for example here, we're inputting N in a frequency.
And then we have two stages of execution.
At initialization time-- this runs one at the beginning of the whole program-- we can calculate some weights for example, from the frequency.
And we can store those weights as a local variable.
So you can think of this kind of like a Java class.
You can have some member variables.
You can retains state from one execution to the next.
The work function is the closest thing to the main function.
This is called repeatedly in the steady state.
And here are the IO rates of the work function.
This filter actually peaks at some data items.
That means that it inspects more items on the input channel than it actually consumes on every iteration.
So we'll look at N input items, and we'll push one new item onto the output and pop one item from the input tape every time we execute.
So here we have a sliding window computation.
It means the next time through, we'll just slide this window up by one and inspect the next N items on the input tape.
And inside the work function you can have pretty much general purpose code.
Right now we just allow pointers and a few other things to keep things simple.
But the idea is this is general purpose imperative code inside the work function.
Now what's nice about this representations of a filter is for one thing is this peak function.
So what we really have is a nice representation of the data pattern, that you're reading the data on the input channel.
And if you look at this for example in a language like C, it's a lot messier.
So usually when you're doing buffer management, you have to do some modulo operations.
You have to keep a circular buffer of your live data.
And increase you know, a head or tail pointer and mod around the side with some kind of modulo operation.
And for a compiler, this is a real nightmare.
Because modulo operations are kind of the the worst thing to analyze.
You can't see what it's actually trying to read.
And if you want to map this buffer to a network or to a combined communication with some other actor or filter in the graph, it's pretty much impossible.
So here is we're exposing that all to the compiler.
And you'll see how that can make a difference.
And also it's just a lot easier to program.
I mean, I don't like looking at the code.
So I'm going to go to the next slide.
OK, so how do we piece things together?
Let's just build some higher level components.
So here's a pipeline of two components.
And we already saw a pipeline.
You can just add one component after another.
And add just basically has the effect of making a queue, and just queueing up all of the components that you added, and connecting them one after another.
So here we have a BandPastFilter by connecting a LowPassFilter and feeding it's output into a HighPassFilter.
You end up with a BandPassFilter.
OK what about a splitjoin?
How do we make those?
So a splitjoin has an add statement as well.
And here we're adding components in a loop.
So what this means is now when we say add, we're actually adding from left to right.
So instead of going top to down, we're adding from left to right across the splitjoin.
And we can actually do that in a loop.
So here we're intPrinting a parameter N. And depending on that value, we'll add N BandPassFilters to this splitjoin.
So it's kind of cool, right?
Because you can input a parameter, and that parameter actually affects the structure of your graph.
So this graph is unrolled at compiled time by our compiler, constructing a big sequence of computations.
And it can resolve the structure and communication pattern in that graph, and then map it to the underlying substrate when we compile.
Also to notice here are the splitter and the joiner.
So we have a predefined set of splitter and joiners.
I'll go into more detail later.
But here we're just duplicating the data to every one of these parallel components, and then doing a round-robin join pattern where we bring them back together into a single output stream.
And if you want to do an equalizer, you basically need an adder at the bottom to add the different components together.
And another thing you can notice here is that we have some inlining going on.
So we actually embedded this splitjoin inside a higher level pipeline.
So what this does is it prevents you from having to name every component of your stream.
You can have a single stream definition with lots of nested components.
And the natural extension is, you can basically scale-up to basically the natural size of a procedure, just like you would in an imperative language.
And here is for example, an FM radio where we have a few inline components.
And the interesting thing here is that there's a pretty good correspondence between the lines of the text and the actual structure of the graph.
And that's something that's hard to find in an imperative language.
I mean if you want to for example, stitch nodes together with edges, it's often very hard to visualize the resulting structure of the graph that you have.
But here if you just go through the program, you can see that the AtoD component goes right over to the AtoD, the demodulator to the demodulator, and so on.
And even for the parallel components, you can kind of piece them together.
so that's kind of how we think of building programs.
Any questions so far?
OK so this is kind of how you go about programming in StreamIt.
But programming is kind of a chug n' plug activity right?
Nobody wants to be a code monkey.
The reason we're all here is to see what's beautiful about this programming model.
Right?
Don Knuth said this, a famous computer scientist from Stanford during his Turing Award Lecture you know, "Some programs are elegant, some are exquisite, some are sparkling.
My claim is that it is possible to write grand programs, noble programs, truly magnificent ones!"
Right.
We want the best programs possible.
It's not just about making it work.
We want really beautiful programs.
So what's beautiful about the streaming domain?
What can you go away with and say wow, that was a really beautiful expression of the computation.
Well for me I think one of the interesting things here is the splitjoin contruct.
Splitjoins can really be beautiful.
You know some mornings I just wake up and I'm like, oh I'm so glad I live in a world with splitjoins.
You know?
And and now splitjoins will be part of your world.
You can say this tomorrow.
This is just wonderful.
So OK, what do we having in the splitjoin constructs?
You can duplicate data.
You can do a round-robin communication pattern from one to another, or round-robin join.
Now the duplicate is pretty simple.
You just take the input stream and duplicate it to all the children.
No problem.
What do you do for a round-robin?
Well you path N items from the input to a given child.
So for example, if N is 1, we'll just distribute one at a time to the child streams.
And you get a pattern like this, a round-robin just going across.
And you can do the same thing on the joiner side.
Let's say you're joining with a factor of one.
You're just reading from the children and putting them into a single stream.
OK so the pretty colorful, but nothing too fancy yet.
Let's consider a different round-robin factor.
So a round-robin of 2 means that we peel off 2 items from the input, and pass those items to the first output.
OK there actually is going to be a quiz on this.
So ask questions of this doesn't make sense.
OK pass the next 2 items, and 2 items round-robin like that.
And you can actually have nonuniform weights if you want to.
So on the right let's say we're doing round-robin 1, 2, 3.
That means we pass 1 item from the first stream, 2 items from the next stream, and then 3 items from the third stream.
OK, pretty simple.
We're just doing 1, 2, and 3, and so on.
Does that make sense?
I'm going to build on this so any questions?
OK this was colorful but this totally beautiful yet.
So what's beautiful about this?
well let's see how you might write a matrix transpose.
OK, something you guys have probably all written at one point in your life is transposing a matrix.
Let's say this matrix has M rows, and it has N columns going across.
And we're starting with a representation in which the stream is basically, I think this is row major order.
The first thing that you're doing is going across the rows before you're going to the next the next row.
I'm sorry you're going across the columns, and then down to the next row.
So it's zigzagging like this.
And you want to pass through a transpose, so you zigzag the other way.
You do the first column, up, and then the next column, and so on.
And this comes up a lot in a stream program.
So it turns out you can represent this as a splitjoin.
Oh that's not good.
Just in my moment in glory as well.
OK, yeah slides.
You can download slides.
Actually I can do this on the board.
This is the thinking part anyway.
Yeah, could you just bring up GMail?
I have a backup on GMail.
Can we focus on the board?
So this is going to be a little exercise anyway.
OK, here's what we had.
We had M, M rows, N columns.
We started with an interleaving like this.
Right, and we want to go into a splitjoin.
And this will be a round-robin construct.
And what I want you to do is fill in the round-robin weight, and also the number of the streams.
And you can have a round-robin at the bottom.
And when it comes out, you want the opposite interleaving.
This is M and N. OK so there are 3 unknowns here, what you're doing the round-robin by-- can you guys see that over there-- how many parallel streams there are, and what you're joining the round-robin by.
Ok so I'm going to give you a minute to think about this.
Try to think about this.
See if you can figure out what these constants are.
You just basically want to read from this data in a row major pattern, M rows and N columns, and end up with something that's column major.
What are the values for the constant?
Does it makes sense?
Ask a question if it doesn't make sense.
Yeah?
So we assume that values are going to, based on the line that you drew, across it, tests like a stream line?
Right, right, right.
So those values are coming down the stream.
You have a 1-dimensional stream.
It's interleaved like this.
So you'll be reading them like this.
And then you the output a 1-dimensional stream that is threading the columns.
Does that make sense?
Somebody ask another question.
Yeah
So the actual matrix transpose codes, it's my understand that nobody actually does it sequentially like that because of locality issues.
Instead it's broken up into blocks
So there are ways to optimize this
And after you've sort of serialized it, can you then capture
Guys, [UNINTELLIGIBLE PHRASE] has a blocking segment.
So you can heirarchically do that.
So, normally what happens is you do the blocks and inside the blocks, you can do it again.
You can do it at two levels, basically
Any hypotheses?
Anyone?
Is it N for first one, M for the second one?
OK, what do we have
N for the first one, and M for the second?
N,M and
Same for the M?
Same, M?
Yeah
OK OK, this is a hypothesis.
Other hypotheses?
N, M, 1
OK, anyone else?
Any amendments?
How about 1, 1, 1?
1, 1, 1?
OK lottery is closing.
Yep?
!, M,
1 M, M?
1, M, M, OK and last call?
OK I believe two of the ones submitted are correct.
Is this and, yeah?
OK I think this, N, M, 1, works and 1, N, M, works.
So let me explain the 1, N, M, this is how I like to think about it.
One way to think about this is we just want to move the whole matrix.
You doing OK, yeah?
OK we just want to move the whole matrix into this splitjoin.
So the way we can do that is have M columns of the splitjoin since we have N columns of the matrix.
And what we'll do is we'll just do a round-robin one at a time, from the columns of the matrix into the columns of the splitjoin.
So we'll take the first element, send it to the last, next element, next column, and so on.
So we get the whole matrix here, Now I want to read it out column-wise.
So we'll do a joiner of M. We'll read M from the left stream that'll read all M items from the columns, send it out, and then M items in the next column, and then send it out.
Does that make sense?
How many people understood that?
All right, So if you think about it, you can also do it in M, N, 1.
That's basically, yeah, it's very similar.
OK there we were.
And yes.
We can do 1, N, M. We basically read the matrix down, and then pull it down into column.
And it's very easy to write this as a transpose.
So we just have a transpose filter in which we're doing nothing.
No competition in the actual rows or the actual contents of the splitjoin.
And we just split the data by 1, have N different identity filters, and then join it back together by M. Any questions about this?
An interesting way to write a transpose.
OK so there's one more opportunity to shine here.
And that's a little more interesting permutation called a bit-reversed ordering.
And so this comes up in an FFT and another algorithm.
The permutation here, is that we're taking the data at the index n. And let's say n has binary digits b sub 0, b sub 1, up to b sub k. And we want to rearrange that data, so this data goes to a different index with the reversed bits of its index.
So if it was and index n before, it ends up at b sub k down to b sub 1, b sub 0.
So for example, let's just look at 3-digit binary numbers.
If we have 0, 0, 0, this is 1 input item.
We're reversing those digits, you still get 0, 0, 0.
Item at index 1 will be 0, 0, 1.
We want to reorder that to index 4.
OK, 1, 0, 0, 0, 1, 0, stays the same, 0 1, 1, goes to 1, 1, 0 shifts over.
And from there on, it's symmetric
[UNINTELLIGIBLE PHRASE]
Sorry?
[UNINTELLIGIBLE PHRASE]
OK.
So here I'm writing the indices.
So I'm not writing the data.
So index 0, 1, 2, 3 up through index 8., or index 7.
OK and on the bottom you have indices 0 through 7.
So the data will actually be moved, reordered like that.
Does that make sense?
It's a reordering of data.
Does this transformation make sense?
Other questions?
OK, it turns out you can write this as a splitjoin.
And you just need 3 different weights for the round-robins.
OK round-robin with 1 weight on the top here, and two different round-robin weights on the bottom.
And here I'm assuming you have 3 binary digits in your index.
So you're reordering in groups to 8.
OK, so let me give you a second to think about this.
What are the values for these weights?
I'll give you a hint in just a second, or ask another question.
Yes?
[UNINTELLIGIBLE PHRASE]
So what we're doing here, is we're exposing the communication pattern.
That's the thing.
If you write this in an imperative way, you end up basically having your, you're conflating basically the data dependencies with the reordering pattern.
So what I'm trying to convey here, is how you can use the streaming model to show how you're sending data around.
Because when you're on an architecture-like cell, everything is about the data motion.
You're taking data from one place and you're trying to efficiently get it to the producers or the consumers.
And you really need to-- the compiler needs to understand the data motion.
And also, it's just another way of writing it, which I think it's actually easier to understand once you see it.
It's a way to think about the actual reordering from a theoretical standpoint Any wagers on this?
So what we to do, if you think about the bit-reverse ordering, what we want to do is distribute the data by the low-order bits.
And then gather the data by the high-order bits.
So you want a fine-grained parity when you're shuffling.
You can also do it the other way around.
It's totally symmetrical.
But one way to think about it is, you want a fine-grained parity when you're distributing data, and then a course-grained when you're coming together.
Anyone see it?
Give you ten more seconds.
Come on?
Yeah it is a little bit tricky.
OK well let me explain how it works.
So 1, 2, and 4.
Ok so what these round-robin 1 splitters do, is these are basically the fine-grained parities.
So OK, the first round-robin, that will send all the even bits to the left, and all the odd values to the right.
Right, that's the lowest order bit.
Because it's doing every other one, shuffling it left and right.
So this round-robin is seeing only the even values.
Now it's going to split them up based on who's divisible by 4.
Now we'll go to the left or go to the right.
This is basically shuffling in the order of the bits, from low-order bits to high-order bits.
So these will be ordered in terms of their low-order bits.
And now we just want to read them out from left to right.
Just take the order that we made with those round-robins, and read them out from left to right.
And since they have 8 values, you can do that just by chunking them up.
We'll read 2, 2, and then we'll read these two, 2 and 2, and now put all 8.
Does that make sense?
OK, so yes, it's a bit clever.
So I think it's a nice way of thinking about what a bit-reversed ordering means.
And you can write this in a very general way.
You just have a recursive bit-reversed filter for N values.
And base case, you only have 2.
So there's no reordering to do when you when you think about it.
So you're not doing any computation.
Otherwise yo have a round-robin split in half, and then have a coarse-grain joiner.
So, you get a structure like this.
If you're, as you're building up, just distributing and then bringing back together in a course-grained way.
OK. Let's see how do I don't do this?
OK so one thing to notice, there's one more example of a splitjoin.
Question?
[UNINTELLIGIBLE]
OK so in general, at the base of this hierarchy, we could've added some other filter to do some competition.
Identity just means we're doing no computation.
It's a predefined filter that just does nothing
On complex data
On complex data.
Sorry this is a templated filter.
So we're reordering complex values.
And we're passing the input to the output.
Amir?
[UNINTELLIGIBLE]
In general the language does not have support for templates.
We only do it for these base classes.
That's more of an implementation detail.
Yeah
[UNINTELLIGIBLE]
Right now there isn't, but nothing fundamental there.
Yeah?
Other questions?
Yeah?
How did you know that there are two filters after that?
Two filters, sorry here?
[UNINTELLIGIBLE]
OK. OK.
So we have two add statements between the split and the join.
So that branches to two parallel streams.
Is that your question?
Yeah.
How do you that theorem, that there's not like three
So the compiler will analyze this, the compile time.
And it'll know these values of N and propagate them down at compile time.
So it'll basically symbolically evaluate this code, see there are two branches, and you can unroll this communication pattern
I think another way of thinking about it is, each add statement essentially adds another branch in your splitjoin
That is another box
It's about to get clear actually.
Other questions?
[UNINTELLIGIBLE]
That's one way to think about it, yeah.
Wait say again, counting?
A rating sort
A rating sort, right.
OK, well is it sorting?
It's not really sorting
Well it's not really sorting
It's a permutation
Could you do a rating sort?
You could do a rating sort.
So actually, what I want to show next is how you can morph this program into a merge sort by changing only a few lines.
OK look carefully.
Don't blink.
OK there's merge sort.
So very similar pattern.
This is one of those idioms.
It's a recursive idiom with splitjoins.
But now in the base case, we have a sort.
So we would basically branch down.
What we ended up with was a sort in the base case.
We're just sorting a few values.
And we call [?
merge sort ?]
twice again, and then we do a merge.
So instead of identity at the base case here, we now have a basic sorting routine.
And we merge those results from both sides.
And the only thing I changed in terms of the communication rate, is to be more efficient we just distributed data in chuncks instead of doing a fine-grained splitting.
Actually you do it however you want in a merge sort.
But this is chunked up.
And let's just zoom in here.
This is how a merger sort looks in StreamIt.
So we split the data two ways, both directions, come together, do a sort on both sides, and then you merge.
And so by having the, you know you can interleave pipelines and splitjoins like this.
So you have these hierarchical structures that are coming back together.
Does this make sense?
OK.
I'm going to hold off on messaging actually.
Let me see, what do I want to cover?
OK let me actually skip to the end.
Oh, I can show you this.
Yeah, I'm going to cut short a little bit.
So here's how other programs look written in StreamIt.
OK, you can have a Bitonic sort.
OK so you see a lot of these regular structures.
And the compiler can unroll this and then match it to the substrate.
This is how and FFT looks.
It's quite an ah elegant implementation of an FFT.
It'd be good to go into in more detail.
You can do things like block matrix multiply.
You don't always have to have column or row-wise ordering.
It's natural to split things up like this.
We have a lot of DSP algorithms, the filter bank, FM radio with equalizer, radar array front end.
Here's an MP3 decoder.
And let's see, I'm going to skip this section and just give you a taste for the end here.
I'm skipping a hundred slides.
Yeah.
OK so if I give you a feel.
Our biggest program written in StreamIt so far, is the complete MPEG-2 encoder and decoder.
So here is MPEG-2 decoder.
And I think you've seen block diagrams of this already in the class.
And so it's a pretty natural expression.
You can really get a feel for the high-level structure of the algorithm mapping down.
And for example, here on the top we're doing the spatial decoding looking inside each frame.
Whereas at the bottom we're doing the temporal decoding between two frames, the motion compensation.
And one thing that I didn't have a chance to mention, is that we have a concept of teleport messaging.
What this means is, I showed you how the steady state flow data goes between these actors in the stream.
But sometimes you want to control the stream as well.
For example, this is a variable length decoder at the top.
It's parsing the input data.
It might want to change how the processing is happening downstream.
For example, say that you have-- you know in this case you have different picture types coming in.
And you want to tell other components to change their processing based on a non-local effect.
And that's hard to do if you just want static data rates.
But what we have is this limited notion of limited dynamism, where you're basically poking into somebody else's stream.
And we let you do that very precisely.
I don't have time to go into the details, but you can basically synchronize the arrival of these messages with the data that's also flowing through the stream, And so in this case, were sending through the picture type.
And it really simplifies the program code.
I didn't have time for details, but why don't we put in the slides anyway, if you're interested.
And if you do a similar communication pattern in C, it's a little bit of a nightmare.
You have all these different, basically memory spaces, different files.
And the control information is basically going left and right all over.
So this really helps both the compiler and the programmer as well in StreamIt.
So it's all implemented.
It's about 2,000 lines of code in StreamIt.
Which is about 2/3 of the size of the C code, taking into account similar functionality there.
And it's a pretty big program.
You can write 48 static streams.
And then we expand that to more than 600 instantiated filters.
So this gives you a lot of flexibility when you're trying to get parallelism.
Question?
When a compiler downloads all bytes?
Oh the object code, you mean.
OK, so right now our current implementation, we duplicate a lot of code.
So it end up being bigger than it needs to be.
There's no reason for us to do that.
That's kind of a-- we have a research compiler that make that easy
So object-wise , its not data
Object-wise we still need to do that comparison.
Yeah that's a good question.
Yeah.
Other questions?
OK so let me cut to the end.
OK, so we have the StreamIt language.
And we think it really preserves the program structure.
It's a new way of thinking about how you orchestrate the data reordering with the splitjoins, showing you who is communicating to who, and how you can stitch together different pieces in your program development.
And again, really our goal is to get this scalable multicore performance.
But you can't get people on board just on a performance stat.
You need to show them a new programming model that actually makes their lives easier.
So that's what we're working on.
And thinks with listening.
[APPLAUSE]
Any last questions?
Yes?
So in the anti-decoder, you have a lot of computations size that were not sequential streams.
Like for example, the output of the distinct cosine transform is not a stream of pixel, you are going to have coefficients and things like that.
Which are a logical sort of chunk
Yes so depending on the granularity of the competition, you don't need to pass individual values over the stream.
For example, you can have a stream that inputs the whole array at a time.
And so we basically advocate that if you have something that course-grained, you should be passing an array or a macroblock, in the case of MPEG, or a set of coefficients in a structure.
So when you have coarse-grain parallelism, you write your program in a course-grained way.
The fine-grained things I showed for the bit interleaving and so on, is more for the fine-grained programs
Can you do both?
In the sense that can you stream over an array, so it's stream of stream, so to speak
So there's an interesting multidimensional problem there.
Right now we've taken a 1-dimensional approach.
So far it's basically the programmer has to set an iteration order, and end up with a 1-dimensional stream coming into and out of every filter.
We're working on extending that.
Yeah, but when you have basically streams of 2-dimensional data, you like the freedom to either iterate basically in time or in space, depending on what you're doing.
And so I think that's more of a research problem.
So far we're just been doing a 1-dimensional representation.
Yeah good point.
Other questions?
Yeah?
Why did you decide on this synchronous dataflow model as opposed to something more general?
So our philosophy has been that you want to start with the most kind of basic block of a stream program, and optimize it really well.
And then you can stitch those together into higher level blocks.
So we think of synchronous dataflow as being kind of the basic block of streaming.
You know what's coming in, you know what's coming out.
And even if you have a more general model, they'll be pieces that fit under their synchronous dataflow model.
And so we saw a lot of optimizations opportunities in there.
And really knowing those IO rates can let you do a lot of things that you can't do in a general model.
So I wanted to get the simple case right first.
And actually kind of our focus now is on expanding, and how do you look at the heterogeneous system, and how do you optimize a more dynamic system.
Yep.
Other questions?
OK.
Yes.
You can check out our web page.
Yeah if you Google for StreamIt, I'm sure you can find it.
Yeah, we have a public release.
Yes, send us any problems.
It's actually it's a good test.
We want to make sure it works for everyone.
But I mean, we've had, you know, hundreds of downloads.
There are a lot of people using StreamIt.
It shouldn't break if you download it.
Yeah.
OK good.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, yesterday in the recitation we talked a little bit about how to debug programs on Cell.
Today I'm going to talk a little more about debugging parallel programs in general and give you some common tips that might be helpful in helping you track down problems that you run into.
As you might have gotten a feel for yesterday, debugging parallel programs is harder than normal sequential programs.
So in normal sequential programs, you have your traditional set of bugs which parallel programs inherit.
So that doesn't get much harder, but then you add on new things I can go wrong because of parallelization.
So things like synchronization, things like deadlocks, things like data races.
So you have to get those right.
Now you have to debug your program and figure out how to do that.
One of the things you'll see here is a lot of tools might not be as good as you'd like them to in terms of providing you the functionality for debugging.
Add to that that bugs in parallel programs often just go away if you change one statement in your code -- you reorder things and all of a sudden the bug is gone.
It's kind of like those pointer problems in C, where you might add a word, add a new variable somewhere and problem's gone, or you add a print-off and the problem is gone.
So here it gets harder because those things can get rid of deadlocks, so it makes it really hard to have an experiment that you can repeat and come down to where the problem is.
So what might you want in a debugger.
So this is a list that I've come up with, and if you have some ideas we'll want to throw them out.
I'm thinking in terms of debugging parallel program, what I want is a visual debugging system that really let's me see all the processors that I have in my network in my multi-processor system.
That includes actual computing and the actual network that they're interconnecting all the processors that are going to be communicating with each other.
So I'd like to be able to see what code is running on each processor.
I'd like to see which edges are being used to send messages around.
I might want to know which processors are blocked -- that might help me identify deadlock problems.
For these kinds of scenarios it might be tricky to define step, because there's no global clock, you can't force everybody to proceed through one step.
What's one step on one processor might be different on another, especially if they're not all running the same code.
So how do you actually do that without a global clock.
So that can get a little bit tricky.
It likely won't help with data races, because I'm looking at global communication problems, I'm looking at trying to identify what's deadlocked and what's not.
So if there are data races, this kind of tool may or may not help with that.
In general, this is the tool that I would build for debugging.
I looked around on the web to see what's out there for debugging parallel programs, and I found this called TotalView.
This is actually something you have to buy, it's not free.
I don't know if they have evaluation licenses or licences for academic purposes.
It kind of gets close to some of the things I was talking about.
You have processors that shows your communication between those processors, how much data is being sent through.
This particular version uses NPI, which we talked about in previous lectures.
So it's sort of helpful in being able to see the computation, looking at the communication, and tracking down bugs.
But it doesn't get much better from there.
You know, how many people have used printouts for debugging?
It's the most popular way of debugging, and even I still use it for debugging some of the Cell programs we've been writing.
I know the TAs actually use this is well.
Yesterday you got a hands-on experience with GDB, and GDB is a nice debugger, but it lacks a lot of things that you might want, especially for debugging parallel programs.
You saw, for example, that you have multiple threads, you need to be able switch between the threads, getting the context right, being able to name the variables is tricky.
So there's a lot of things that could be improved.
There are some research debuggers, like something we've built as part of the streaming projects, StreamIt debugger.
I'll show you some screenshots of this so you can see what we can do.
So in the StreamIt debugger, remember we have -- so this is actually built in Eclipse and you can download this off the web as well.
You can look at your stream graph.
Unfortunately, I couldn't get a split join in there, much to Bill's dismay.
So you can't see, for example, the split join in all the communication.
Each one of these is a filter, and if you recall the filter is the computational element in your stream graph and interconnected by channels.
So channels communicate data.
So what you see here -- well, you might not be able to quite see it -- actually see the data that's being passed through from one filter to the other.
You can actually go in there and change the value if you wanted to or highlight particular value and see how it flows down through the graph.
If you had a split join, then you might be able to -- in fact, you can do this.
You can look at each path of the split join independently and you can look at it in sequence.
Because the split join has nice semantics, it's actually you can replicate the behavior because of the static nature is everything is deterministic.
So this is very helpful.
We we did a user study two years ago almost with something like 30 MIT students who use the C bugger and gave us feedback on it.
So we gave them like 10 problems, 10 code snippets, each of them had a bug in it, we asked them to find it.
So a lot of them found to debugger to be helpful in being able to track the flow of data and being able to see what goes wrong.
So if you had, for example, a division that resulted in NaN and not a number, floating point division you can immediately see on a screen, so you know exactly where to go look for it.
Doing that would print-offs might not be as easy.
So sometimes visual debugging can be very nice.
Unfortunately, visual debugging for the Cell isn't that great.
So this is the Cell plug-in in Eclipse.
I've mentioned just to some of you if you want to run it you can run it from a Playstation 3, but if more than one of you is running it then it becomes unusable because of memory issues.
You can install this on other Linux machines and remotely debug on the Playstation 3 hardware.
So the two remote machines can talk through GDB ports.
I can talk to you about how to set that up if you want to, but it doesn't really add anything over E-Max, for example.
It just might look fancier than an E-Max window or a GDB on the command line prompt.
So this is the code from yesterday.
These are the exercises we asked you to do.
You can look at the different threads.
If you have debug Java programs in Eclipse this should look very familiar.
You can look at the different variables.
You still have the naming problem.
So yesterday, remember you had to qualify which control box you were looking at?
Still the same kind of issue -- have to do some trickery to find it here.
It doesn't have the nice visual aspect of showing you which code is running on which SPE, and you might not be able to find mailbox synchronization problems.
Maybe those things will come in the future, and the fact, they likely will.
But a lot of that is sort of still lacking.
So what do you do in the meantime, In the next two weeks as you're writing your programs?
So I've looked around for some tips or some talks and lectures and what people have done in terms of improving the process for debugging parallel codes.
Probably the best thing I've found is this talk that was given at University of Maryland on defect patterns.
So the rest of these slides are largely drawn from that talk.
I'm going to identify just a few of them to give you some examples so you can understand what to look for and what are some common symptoms, what are some common prevention techniques.
So defect patterns, just like the programming patterns we talked about, are meant to help you conjure up to write contextual information.
You had what are things you should look for if you're communicating with somebody else.
What kind of terminology do you use so that you don't have to explain things down to every last detail.
At the end of this course, one thing I'd like to do is maybe get some feedback from each of you as to what are some of the problems that you ran into in writing your programs, and how you actually went about debugging them, and maybe we can come up with Cell defect patterns, and maybe Cell defect recipes for resolving those defect patterns.
So, probably the worst one of all, and the easiest one to fix is that you have new language features or new language extensions that are not well understood.
This is especially true when you take a class of students and they don't really know the language, they don't know all the tools, and you ask them to do a project in four weeks and you expect things to work.
So there's a lot for everybody to pick up and understand.
So you might have inconsistent types that you use in terms of calling a function.
There might be alignment issues, which some of you have run into.
You might use the wrong functions.
You know the functionality you want but you just don't know how to name it and so you might use the wrong function.
Some of these are easy to fix because you might get a compile time error.
If you have mismatch in function parameters then you can fix that very easily.
Some defects -- you know, very natural parallel programs might not come up until run time, so you might end up with crashes or just erroneous behavior.
I really think this is probably the easiest one to fix, and the prevention technique that I would recommend is if there's something you're unfamiliar about or you're not sure about how to use something, ask.
But also, you don't need to know all the functions that are available in something like the Cell language extensions for C. Yes, there are a lot of functions -- you know, the manuals, hundreds of pages, and you can't possibly go through it all and nobody becomes an expert in everything.
But understand just a few basic concepts and features.
So, David identified a bunch that he found useful for writing the programs, and some of the ones that are up on the web page under the recipes for this course list a few more.
And so this might help you in just understanding how these functions work, understanding basic mechanisms that they give you, and that's good enough because it'll help you get by.
Certainly for doing the project under short time constraints, you don't need to know all the advanced features that Cell might have.
Or you can probably just pick them up on the fly as you need them.
So what are some more interesting problems that come up.
I think one that is probably not too unfamiliar is this space decomposition problems.
So, if you remember, space decomposition is really data distribution.
You have a serial program that you want to parallelize.
And what that means if you have to actually send data around to different processors so that each one knows how to compute locally.
Here you might get things like segmentation faults, alignment problems, you might have index out of range errors.
What this comes of is forgetting to change things or overlooking some simple things that don't carryover from the sequential case that are parallel case.
So what you might want to do is validate your distributions and your memory partitions correctly.
So what's an example?
So suppose you had an array or a list of cells, each cell has a number.
What you want to do is at each step of the computation for any given cell, you want add the value to the left of it and the value to the right of it.
So here we have cell zero has the value 2.
We'll assume that the n disconnectors are first, so this is like a circular list, a circular buffer.
So adding the left and right neighbor would get me, in this case, 3 plus 1, 4.
And so on and so forth.
You want to repeat this computation for n steps.
So this might be very common in computations where you're doing unit [?
book ?]
communication.
So what's a straightforward sequential implementation?
Well, you can use two buffers -- one for the current time step, and you do all the calculations in that.
Then you use another buffer for next time step.
Then you swap the two.
So the code might look something like this.
Sequential c code, my two buffers, here's my loop.
I write into one buffer and then I switch the two buffers around.
Any questions so far?
So now, what are some things that can go wrong when you try to parallelize this?
So how would you actually parallelize this code?
Well, we saw in some of your labs, for example, that you can take a big array, split it up in smaller chunks and assign each chunk to one particular processor.
So we can use that technique here.
So each processor, we have n of them, rather size of them, and it's going to get some number of elements.
So each time step, we have to compute locally all the communications, but then there's some special cases that we need to treat at the boundaries, right.
So if I have this chunk and I need to do my new neighbor communication, I don't have this particular cells.
I have to go out there and request it.
Similarly, somebody has to send me this particular data item.
So there's some data exchange that has to happen.
So in the decomposition, you write your parallel code.
Here, each buffer is a different size.
What you do is you have some local, which says this is how much of the data I'm getting, and has a total number of elements, besides the number of processors.
Local essentially tells me the size of my chunk.
I'm iterating through from zero and local and I'm doing essentially the same computation.
So what's a bug in here?
Anybody see it?
Sort of giving you some hints of things highlighted in red or there's something wrong with the things highlighted in red.
There's another hint.
So this is essentially the computations going on at every processor, this is my buffer, and at every step I have to do the calculations, taking care of the boundary edges.
Anybody want to take a stab?
Mark?
Is it that the nextbuffer zero needs to look at data from 1?
Next buffer zero, right.
So what might be a fix to that?
So the next buffer is zero.
So if this is zero then buffer of x minus 1 plus this points to what?
So you need to start at 1 and iterate
Right, exactly.
It's a local plus 1, if you were going to do these.
So that's one bug.
The other thing is the assumption that your data elements might be divisible by the number of processors that you have.
So you pick the decomposition that might not be symmetric across all processors.
So it's more subtle, I think, to keep in mind.
So that's one particular kind of problem that might come up on your decomposing data and replicating among different processors.
So you have to be careful about what are your boundary cases going to be and how are you going to deal with those.
The more difficult one is synchronization.
So synchronization is when you're sending data from one processor to the other and you might end up with deadlock, because one is trying to send, the other's trying to send, and neither can make progress until the other's received.
So your program hangs, you get non-deterministic behavior or output, every time you run your program you get a different result -- that can drive you crazy.
So some of the defects can be very subtle.
This is probably where you'll spend most of your time trying to figure it out.
So one of the ways to prevent this is to actually look at how you're orchestrating your communication and doing it very carefully.
So look at, for example, what's going on here.
So this is the same problem, and what I'm doing is now this is the parallel version and I'm sending the boundary cases, the boundary cells to the different processors.
This is an SPMD program.
So an SPMD program has every processor running essentially the same code.
So this code is replicated over n processors and everybody's trying to do this same thing.
So what's the problem with this code?
We're doing send of next buffer zero.
Here, rank essentially just says each processor has a rank.
So this is a way of identifying things.
So I'm trying to send it to my previous guy, I'm trying to send it to the next guy, and here I'm sending the value at the far extreme of the buffer to the next processor and then to the previous processor.
Anybody see what's wrong here?
So are these blocking things?
Yeah, imagine they're blocking things
Why will that deadlock?
Right.
So this will deadlock, right.
So this will deadlock because this processor is trying to send here, this processor is trying to send here, but neither is receiving yet, so neither makes progress.
So how would you fix it at this point?
You might not want to use a blocking send all the time.
So if your architecture allows you to have different flavors of communication, so synchronous versus an asynchronous, a blocking versus non-blocking, you'll want to avoid using constructs that can lead you to deadlock if you don't need to.
The other mechanism -- this was pointed out briefly in the talk on parallel programming -- you want to order your sends and receives properly.
So alternate them.
So you have a send in one processor, a receive in the other.
You can use that to prevent deadlock and get the communication patterns right.
There could be more interesting cases that come up if you're communicating over a network where you might end up with cyclic patterns leading to loops, and that also can create some problems for you.
The last two I'll talk about aren't really bugs in that they might not cause your program to break or compute incorrectly.
Things might work properly, but you might not get the actual performance that you're expecting.
So these are performance bucks or performance defects.
So side effects of parallelization is often case that you're focusing on your parallel code and you might ignore things that are going on in your sequential code, and that might lead you to, essentially you've spent all this time trying to parallelize your code, but your end result is not getting the performance that you expect because things look sequential.
So what's wrong here?
So as an example, imagine that we're doing instead of reading data from a -- so, in the previous case I didn't show you how we were reading data into the different buffers, but suppose we were getting it from some files, so input buffer.
So now we have an SPMD program again, everybody's trying to read from this buffer.
What could go wrong here?
Anybody have an idea?
So every processor is opening the file and then it's going to figure out how much to skip and it'll start reading from that location.
So everybody's reading from a file, so that's OK, nobody's modifying it.
But what can go wrong here?
[INAUDIBLE PHRASE]
Right.
So essentially, sequentialize your execution because reading from the file system becomes the bottleneck.
So you'll want to schedule input and output carefully.
You might find that not everybody needs to do the input and output.
Only one processor has to do the input and then it can distribute it to all the different processors.
So, in the Master/Slave model, which a lot of you are using for the Cell programming, the Master can just read the data from the input files and distribute it to everybody else.
So this avoids some of the problems with input and output.
You can have similar kinds of problems if you're reading from other devices.
It doesn't have to be the file system.
So here's another one, a little more subtle.
So you're generating data--.
Hey, Allen, what's up?
I somehow missed the distinction between when you're waiting for the master to read all the dat aand distribute it, and waiting for the other [?
processes ?]
to get through so I can read my private data, isn't it going to be about the same time on this?
No.
So here, just essentially, the Master reads the file as part of the initialization.
Then you distribute it.
So distribution can happen at run time.
So, the initialization you don't care about because hopefully that's a small part of the code.
So this code is guarded by rank equals Master, so only it does this code.
Then here you might have the command that says wait until I've received it and then execute, or on the cells, then these might be the SPE create threads that happen after you've read the data.
So hopefully, initialization time is not something you have concern about too much.
So if you're generating data on the fly or dynamically, so here we might use the Srand function to sort of start with a random seeing and then fill in the buffer with some random data.
So what could go wrong here?
So in Srand, when you're using a random function -- sorry, this is the same function.
When you're using a random, a pseudo random number generator, you have to give it a seed, then if everybody starts off with the same seed, then you might end up with the same random number sequence.
If that's something you're using to parallelize your computation, you might, in effect, end up with the same kind of sequence on each processor and you lose all kinds of parallelization.
So there's some hidden serialization issues in some of the functions that you might use that you should be aware of.
The last one I'll talk about is performance scalability defect.
So here you parallelize your code, things look good, but you're still not getting -- you've taken care of all your IO issue, you're still not getting the performance you want.
So, why is that?
You might have -- remember your Amdahl's law, and what you want is an efficiency that's linear.
Every time you add one processor you want a straight line curve between the number of processors and speed up.
This should be a linear relationship.
So you might see sublinear speed ups, and you want to figure out why that is.
Some of the common causes here, and this will be the end up focus of the next talk is, unbalanced amount of computation.
Remember, dynamic load balancing versus static load balancing.
Your work estimation might be wrong and so you might end up with some processors idling, other processors doing too much work.
So the way to prevent this is to actually look at the work that's being done and figure out whether it's actually roughly the same amount of work everywhere.
Here you might need profiling tools to help, and so I'm going to talk about this in a lot more detail in the next lecture.
So in summary, there are lots of different bugs that you might come up with.
There's a few that I've identified here, some common things you should look out for.
So the erroneous use of language features understand only a few basic concepts of the entire language extension set that you have.
Space decomposition, side effects from parallelization.
Don't ignore sequential code.
Last one is trying to understand your performance scalability.
But there are other kinds of bugs, like data races, for example.
So what can you do with those?
So remember, data races you have different concurrent threads and they're trying to update the same memory location.
So depending on who gets to write first and when you actually do your read, you might get a different result.
So with data race detection, these things are actually getting better.
There are tools out there that will essentially generate traces as your program is running.
So for each thread you instrumented and you look at every load stored at executes.
Then what you do is you look at the load in stores between the difference threads and see if there's any intersections, any orderings that might give you erroneous behavior.
So this is getting better, it's getting more automated.
Intel Threadchecker is one example.
There are others.
I really think the trend in debugging will be towards trace-based systems.
You'll have things like checkpointing.
So as your program is running you can take a snapshot of where it is in the execution, and then you can use that snapshot later on to inspect it and see what went wrong.
I think you might even have features like replay.
In fact, some people are working on this in research and in industry.
So you might be able to say uh-oh, something went wrong.
Here's my list of checkpoints, can you replay the execution from this particular stage in the computation.
So it helps you focus down in the entire lifetime of execution on a particular chunk where things have gone wrong.
This is sort of a personal dream.
I think one day we'll have the equivalent of a TiVo for your programs, and you can use it for debugging.
So my program is running, something goes wrong, I can rewind it, I can inspect things, do my traditional debugging, change things maybe even, and then start replaying things and letting the program execute.
In fact, we're working on things like this here at MIT and with collaborators elsewhere.
So, this was a short lecture.
We'll take a break.
You can do the quizzes.
Note on the quizzes, there are two different kinds of questions.
They're very similar, just one word is different, and so you'll want to just keep that in mind when you're discussing it with others.
Then about 5, 10 minutes and then we'll continue with the rest of the talk, lecture 2.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, what I'll talk about here is how to actually understand the performance of your application, and what are some of the things you can do to actually improve your performance.
You're going to hear more about automated optimizations, compile the optimizations on Monday.
There will be two talks on that.
You'll get some cell-specific optimizations that you can do, so some cell-specific tricks on Tuesday in the recitation.
So here, this is meant to be a more general purpose talk on how can you debug performance anomalies and performance problems.
Then what are some way that you can actually improve the performance.
Where do you look after you've done your parallelization.
So just to review the key concepts to parallelism.
Coverage -- how much parallelism do you have in your application.
All of you know, because you all had perfect scores on the last quiz.
So that means if you take a look at your program, you find the parallel parts and that tells you how much parallelism that you have.
If you don't have more than a certain fraction, there's really nothing else you can do with parallelism.
So the rest of the talk will help you address the question of well, where do you go for last frontier.
The granularity.
We talked about how the granularity of your work and how much work are you doing on each processor affects your load balancing, and how it actually affects your communication costs.
If you have a lot of things colocate on a single processor, then you don't have to do a whole lot of communication across processors, but if you distribute things at a finer level, then you're doing a whole lot of communication.
So we'll look at the communication costs again and some tricks that you can apply to optimize that.
Then the last thing that we had talked about in one of the previous lectures is locality, in locality of communication versus computation, and both of those are critical.
So we'll have some examples of that.
So just to review the communication cost model, so I had flashed up on the screen a while ago this equation that captures all the factors that go into figuring out how expensive is it to actually send data from one processor to the other.
Or this could even apply on a single machine where a processor is talking to the memory -- you know, loads and stores.
The same cost model really applies there.
If you look at how processors -- a uniprocessor tries to improve communication, and some of the things we've mentioned really early on in the course for improving communication costs, what we focused on is this overlap.
There are things you can do, for example, sending fewer messages, optimizing how you're packing data into your messages, reducing the cost of the network in terms of the latency, using architecture support, increasing the bandwidth and so on.
But really the biggest impact that you can get is really just from overlap because you have direct control over that, especially in parallel programming.
So let's look at a small review -- you know, what did it mean to overlap.
So we had some synchronization point or some point in the execution and then we get data.
Then once the data's arrived, we compute on that data.
So this could be a uniprocessor.
A CPU issues a load, it goes out to memory, memory sends back the data, and then the CU can continue operating.
But the uniprocessors can pipeline.
They allow you to have multiple loads going out to memory.
So you can get the effect of hiding over overlapping a lot of that communication latency.
But there are limits to the communication, to the pipelining effects.
If the work that you're doing is really equal to the amount of data that you're fetching, then you have really good overlap.
So we went over this in the recitation and we showed you an example of where pipelining doesn't have any performance effects and so you might not want to do it because it doesn't give you the performance bang for the complexity you invest in it.
So, if things are really nicely matched you get good overlap, here you only get god overlap -- sorry, these, for some reason, should be shifted over one.
So where else do you look for performance?
So there are two kinds of communication.
There's inherent communication and your algorithm, and this is a result of how you actually partition your data and how you partitioned your computation.
Then there's artifacts that come up because of the way you actually do the implementation and how you map it to the architecture.
So, if you have poor distribution of data across memory, then you might unnecessarily end up fetching data that you don't need.
So, you might also have redundant data fetchers.
So let's talk about that in more detail.
The way I'm going to do this is to draw from wisdom in uniprocessors.
So in uniprocessors, CPUs communicate with memory, and really conceptually, I think that's no different than multiple processors talking to multiple processors.
It's really all about where the data is flowing and how the memories are structured.
So, loads and stores are the uniprocessor as what and what are to distributed memory.
So if you think of Cell, what would go in those two blanks?
Can you get this?
I heard the answer there.
You get input.
So, DMA get and DMA put.
That's really just the load and a store.
It's just doing, instead of loading one particular data element, you're loading a whole chunk of memory.
So, on a uniprocessor, how do you overlap communication?
Well, architecture, the memory system is designed in a way to exploit two properties that have been observed in computation.
Spacial locality and temporal locality, and I'll look at each one separately.
So in spacial locality, CPU asks for a data address of 1,000.
What the memory does, it'll send data address of 1,000, plus a whole bunch of other data that's neighboring to it, so 1,000 to 1,064.
Really, how much data you actually send, you know, what is the granularity of communication depends on architectural parameters.
So in common architecture it's really the block side.
So if you have a cache where the organization says you have a block side to 32 words, 32 bytes, then this is how much you transfer from main memory to the caches.
So this works well when the CPU actually uses that data.
If I send you 64 data bytes and I only use one of them, then what have I done?
I've wasted bandwidth.
Plus, I need to store all that extra data in the cache so I've wasted my cache capacity.
So that's bad and you want to avoid it.
Temporal locality is a clustering of references in time.
So if you access some particular data element, then what the memory assumes is you're going reuse that data over and over and over again, so it stores it in the cache.
So your memory hierarchy has the main memory at the top level, and that's your slowest memory but the biggest capacity.
Then as you get closer and closer to the processor, you end up with smaller caches -- these are local, smaller storage, but they're faster.
So, if you reuse a data element then it gets cached at the lowest data level, and so the assumption there is that you're gonna reuse it over and over and over again.
If you do that, then what you've done is you've amortized the cost of bringing in that data over many, many references.
So that works out really well.
But if you don't reuse that particular data elements over and over again, then you've wasted cache capacity.
You still need to fetch the data because the CPU asks for it, but you might not have had the cache, so that would have created more space in your cache to have something else in there that might have been more useful.
So in the multiprocessor case, how do you reduce these artifactual costs in communication.
So, DCMA gets inputs on the cell, or just in just message passing, you're exchanging messages.
Typically, you're communicating over a course or large blocks of data.
What you're usually getting is a continuous chunk of memory, although you could do some things in software or in hardware to gather data from different memory locations and pack them into contiguous locations.
The reason you pack them into contiguous locations again to export spatial locality when you store the data locally.
So to exploit the spatial locality characteristics, what you want to make sure is that you actually are going to have good spatial locality in your actual computation.
So you want things that are iterating over loops with well-defined indices with indices that go over very short ranges, or they're very sequential or have fixed striped patterns where you're not wasting a lot of the data that you have brought in.
Otherwise, you have to essentially just increase your communication because every fetch is getting you only a small fraction of what you actually need.
So, intuitively this should make sense.
Temporal locality just says I brought in some data and so I want to maximize this utility.
So if I have any computation in a parallel system, I might be able to reorder my tasks in a way that I have explicit control over the scheduling -- which stripe executes when.
Then you want to make sure that all the computation that needs that particular data happens adjacent in time or in some short time window so that you can amortize the cost.
Are those two concepts clear?
Any questions on this?
So, you've done all of that.
You've parallelized your code, you've taken care of your communication costs, you've tried to reduce it as much as possible.
Where else can you look for performance -- things just don't look like they're performing as well as they could?
So, the last frontier is perhaps single thread performance, so I'm going to talk about that.
So what is really a single thread.
So if you think of what you're doing with parallel programming, you're taking a bunch of tasks -- this is the work that you have to do -- and you group them together into threads or the equivalent of threads, and some threads will run on individual cores.
So essentially you have one thread running on a core, and if that performance goes fast, then your overall execution can also benefit from that.
So, that's single thread performance.
So if you look at a timeline, here you have sequential code going on, then we hit some parallel part of the computation.
We have multiple executions going on.
Each one of these is a thread of execution.
Really, my finish line depends on who's the longest thread, who's the slowest one to complete, and that's going to essentially control my speed up.
So I can improve this by doing better load balancing.
If I distribute the work [?
so that ?]
everybody's doing equivalent amount of work, then I can shift that finish line earlier in time.
That can work reasonably well.
So we talked about load balancing before.
We can also make execution on each processor faster.
If each one of these threads finishes faster or I've done the load balancing and now I can even squeeze out more performance by shrinking each one of those lines, then I can get performance improvement there as well.
So that's improving single thread performance.
But how do we actually understand what's going on?
How do I know where to optimize?
How do I know how long each thread is taking?
How do I know how long my program is taking?
Where are the problems?
So, there are performance monitoring tools that hard designed to help you do that.
So what's the most coarse-grained way of figuring out how long your program took?
You have some sample piece of code shown over here, you might compile it, and then you might just use time -- standard units command say run this program and tell me how much time it took to run.
So you get some alpha back from time that said you took about two seconds of user time, this is actual code, you took some small amount of time in system code, and this is your overall execution, this is how much of the processor you actually use.
So, a 95% utilization.
Then you might apply some optimization.
So here we'll use the compiler, we'll change the optimization level, compile the same code, run it, and we'll see wow, performance improved.
So we increased 99% utilization, my running time went down by a small chunk.
But did we really learn anything about what's going on here?
There's some code going on, there's a loop here, there's a loop here, there's some functions with more loops.
So where is the actual computation time going?
So how would I actually go about understanding this?
So what are some tricks you might have used in trying to figure out how long something took in your computation?
[INAUDIBLE PHRASE]
Right.
So you might have a timer, you record the time here, you compute and then you stop the timer and then you might printout or record how long that particular block of code took.
Then you might have a histogram of them and then you might analyze the histogram to find out the distribution.
You might repeat this over and over again for many different loops or many different parts that are code.
If you have a preconceived notion of where the problem is then you instrument that and see if your hypothesis is correct.
That can help you identify the problems.
But increasingly you can actually get more accurate measurements.
So, in the previous routine, you're using the time, you were looking at how much time has elapsed in seconds or in small increments.
But you can actually use hardware counters today to actually measure clock cycles, clock ticks.
That might be more useful.
Actually, it's more useful because you can measure a lot more events than just clock ticks.
The counters in modern architectures are really specialized registers that count up events and then you can go in there and probe and ask what is the value in this register, and you can use that as part of your performance tuning.
You use them much in the same way as you would have done to start a regular timer or stop a regular timer.
There are specialized libraries that run.
Unfortunately, these are very architecture-specific at this point.
There's not really a common standard that says grab a timer at each different architecture in a uniform way, although that's getting better with some standards coming out from--.
I'll talk about that in just a few slides.
You can use this to, for example, measure your communication to computation cost.
So you can wrap your DMA get and DMA put by timer calls, and you can measure your actual work by timer calls, and figuring out how much overlap can you get from overlap in communication and communication computation and is that really worthwhile to do pipelining.
But this really requires manual changes to code.
You have to go in there and start the timers.
You have to have maybe an idea of where the problem is, and you have the Heisenberg effect.
If you have a loop and you want to measure code within the loop because you have a nested loop inside of that, then now you're effecting the performance of the outer loop.
That can be problematic.
So it has a better effect because you can't really make an accurate measurement on the thing you're inspecting.
So there's a slightly better approach, dynamic profiling.
Dynamic profiling is really there's an event-based profiling and time-based profiling.
Conceptually they do the same thing.
What's going on here is your program is running and you're going to say I'm interested in events such as cache misses.
Whenever n number of cache misses happen, let's say 1,000, let me know.
So you get an interrupt whenever 1,000 cache misses happen.
Then you can update a counter or use that to trigger some optimizations or analysis.
This works really nicely because you don't have to touch your code.
You essentially run your program as you normally do with just one modification that includes running the dynamic profiler, as well as your actual computation.
As far as multiple languages because all it does is just takes your binary so you can program in any language, any programming model.
It's quite efficient to actually use these dynamic profiling tools.
The sampling frequencies are reasonably small, you can make them reasonably small and still have it be efficient.
So some counter examples.
Clock cycles, so you can measure clock ticks.
Pipeline stalls.
This might be interesting if you want to optimize your instruction schedule -- you'll actually see this in the recitation next week.
Cache hits, cache misses -- you can get an idea of how bad your cache performance is and how much time you're spending in the memory system.
Number of instructions, loads, stores, floating point ops and so on.
Then you can derive some useful measures from that.
So I can get an idea of processor utilization -- divide cycles by time and that gives me utilization.
I can derive some other things and maybe some of the more interesting things like memory of traffic.
So how much data am I actually sending between a CPU and a processor, or how much data am I communicating from one processor to the other.
So I can just grab the counters for number of loads and number of stores, figure out what the cache line size is -- usually those are documented or there are calibration tools you can run to get that value, and you figure out memory of traffic.
Another one would be bandwidth consumed.
So bandwidth is memory of traffic per second.
So how would you measure that?
It's just the traffic divided by the wall clock time.
There's some others that you can calculate.
So these can be really useful in helping you figure out where are the things you should go focus in on.
I'm going to show you some examples.
The way these tools work is you have your application source code, you compile it down to a binary.
You take your binary and you run it, and then that can generate some profile that stores locally on your disk.
Then you can take that profile and analyze it by some sort of interpreter, and some cases you can actually analyze the binary as well, and reannotate your source code.
That can actually be very useful because it'll tell you this particular line of your code is the one where you're spending most of your time computing.
So some tools -- have any of you used these tools?
Anybody use Gprof, for example?
Good.
So, you might have an idea of how these could be used.
There are others.
HPCToolkit, which I commonly use from Rice.
Pappy is very common because it has a very nice interface for grabbing all kinds of counters.
VTune from Intel, and there's others that work in different ways and so there are binary instrumenters that do the same things and do it slightly more efficiently and actually give you the ability to compile your code at run time and optimize it, taking advantage of the profiling information you've collected.
So here's a sample of running Gprof.
Gprof should be available on any Linux system, it's even available on Cygwin, if you see Cygwin.
I've compiled some code -- this is MPEG 2D code, a reference implementation.
I specify some parameters to run it.
Here I add this dash Rflag which says use a particular kind of inverse DCDT that's a floating point precise that uses double precision for the floating point computations in DCT -- inverse DCT rather.
So you can see actually where most of the time is being spent in the computation.
So here's a time per function, so each row represents a function.
So this is the percent of the time, this is the actual time in seconds, how many times I actually called this function, and some other useful things.
So the second function that's used here that happens is MPEG intrablock decoding and here you're doing some spatial decomposition, restoring spatial pictures by 5%.
So if you were optimize this particular code, where would you go look?
You would look in the reference DCT.
So, MPEG has two versions of DCT -- one that uses floating point, another that just uses some numerical tricks to operate over integers for a loss of precision, but they find that acceptable as part of this application.
So you omit the Rflag, it actually uses a different function for doing the DCT.
Now you see the distribution of where the time is spent in your computation changes.
Now there's a new function that's become the bottleneck and it's called Form Component Prediction.
Then IDCT column, which actually is the main replacement of the previous code, this one, is now about 1/3 of the actual computation.
So, this could be useful because you can Gprof your application, figure out where the bottlenecks are in terms of performance, and you might go in there and tweak the algorithm completely.
You might go in there and sort of look at some problems that might be implementation bugs or performance bugs and be able to fix those.
Any questions on that?
You can do sort of more accurate things.
So, Gprof largely uses one mechanism, HPCToolkit uses the performance counters to actually give you more finer grade measurements if you want them.
So, in the HPCToolkit, you run your program in the same way.
You have MPEG 2D code, dash dash just says this is where the parameters are to impact 2D code following that dash dash, and you can add some parameters in there that say these are counters I'm interested in measuring.
So the first one is total cycles.
The second one is the L1, so primary cache load misses.
Then you might want to count the floating point instructions and the total instructions.
As you run your program you actually get a profiling output, and then you can process that file and it'll spit out some summaries for you.
So it'll tell you this is the total number of cycles, 698 samples with this frequency.
So if you multiply the two together, you an idea of how many cycles your computation took.
How many load misses?
So it's 27 samples at this frequency.
So remember what's going on here is the counter is measuring events, and when the event reaches a particular threshold it let's you know.
So here the sampling threshold is 32,000.
So whenever 32,000 floating point instructions occur you get a sample.
So you're just counting how many interrupts you're getting or how many samples.
So you multiply the two together you can get the final counts.
It can do things like Gprof, it'll tell you where your time is and where you spent most of your time.
Actually breaks it down into your module.
So, MPEG calls some standard libraries, libsi.
I could break it down by functions, break it down by line number.
You can even annotate your source code.
So here's just a simple example that I used earlier, and each one of these columns represent one of the metrics that we measured, and you can see most of my time is spent here, 36% at this particular statement.
So that can be very useful.
You can go in there and say, I want to do some [?
dization ?
], I can maybe reduce this overhead in some way to get better performance.
Any questions on that?
Yup
[INAUDIBLE PHRASE]?
I don't know.
Unfortunately, I don't know the answer to that.
There's some nice gooies for some of these tools.
So VTune has a nice interface.
I use HPCViewer.
I use HPCToolkit, which provides HPCViewer, so I just grab the screenshot from one of the tutorials on this.
You have your source code.
It shows you some of the same information I had on a previous slide, but in a nicer graphical format.
So, now I have all this information, how do I actually improve the performance?
Well, if you look at what is the performance time on a uniprocessor, it's time spent computing plus the time spent waiting for data or waiting for some other things to complete.
You have instructional level parallels, which is really critical for uniprocessors, and architect that sort of spent massive amounts of effort in providing multiple functional units, deeply pipeline the instruction pipeline.
Doing things like speculation, prediction to keep that instructional level of parallelism number high so you can get really good performance.
You can do things like looking at the assembly code and re-ordering instructions to avoid instruction hazards in the pipeline.
You might look at a register allocation.
But that's really very low-hanging fruit.
You have to reach really high to grab that kind of fruit.
You'll actually, unfortunately, get the experience that is part of the next recitation.
So apologies in advance.
But you'll see that -- well I'm not going to talk about that.
Instead I'm going to focus about some things that are perhaps lower-hanging fruit.
So data level parallelism.
So we've used SIMD in some of recitations.
I'm giving you a short example of that.
Here, I'm going to talk about how you actually get data level parallelism or how do you actually find the SIMD in your computation so you can get that added advantage.
Some nice things about data level parallelism in the form of short vector instructions is that the harder really becomes simpler.
You issue one instruction and that same instruction operates over multiple data elements and you get better instruction bandwidth.
I just have to fetch one instruction and if my vector lens is 10, than that effectively does 10 instructions for me.
The architecture can get simpler, reduces the complexity.
So it has some nice advantages.
The thing to go after is the memory hierarchy.
This is because of that speed gap that we showed earlier on in the course between memory speed and processor speed, and if you optimize a performance usually it's like 1% performance and your cache registry gives you some significant performance improvement in your overall execution.
So you want to go after that because that's the biggest beast in the room.
A brief overview of SIMD and then some detailed examples as to how you actually go about extracting short vector instructions.
So, here we have an example of scaleacode.
We're iterating in a loop from zero to n, and we're just adding some array elements a to b and storing results in c. So in the scaler mode, we just have one add, each value of a and b is one register.
We add those together, we write the value to a separate register.
In the vector mode, we can pack multiple data elements, so here let's assume our vector lens is four, I can pack four of these data values into one vector register.
You can pack four of these data elements into another vector register, and now my single vector instruction has the effect of doing four ads at th same time, and it can store results into four elements of c. Any questions on that?
[UNINTELLIGIBLE]
No.
We'll get to that.
So, let's look at those to sort of give you a more lower level feel for this.
Same code, I've just shown data dependence graph.
I've omitted things like the increment of the loop and the branch, just focusing on the main computation.
So I have two loads, one brings a sub i, the other brings b sub i. I do the add and I get c sub i and then I can store that.
So that might be sort of a generic op code sequence that you have.
If you're scheduling that, then in the first slot I can do those two loads in parallel, second cycle I can do the add, third cycle I can do the store.
I can further improve this performance.
If you took 6.035 you might see software pipelining, you can actually overlap some of these operations.
Not really that important here.
So, what would the cycle or the schedule look like on a cycle-by-cycle basis if this was defector output?
In the scaler case, you have n iterations, right?
Each iteration's taking three cycles so that's your overall execution on time -- n times 3 cycles.
In the vector case, each load is bringing you four data elements, so a sub i to a sub i plus 3.
Similarly for b.
Then you add those together.
So the schedule would look essentially the same.
The op codes are different, and here what your overall execution time be?
Well, what I've done is each iteration is now doing four additions for me.
So if you notice, the loop bounds have changed.
Instead of going from i to n by increments of 1, now I'm going by increments of 4.
So, overall, instead of having n iterations, I can get by with n over 4 iterations.
That make sense?
So, what would my speed up be in this case?
4.
So you can get more and more speed up if your vector lens is longer, because then I can cut down on the number of iterations that I need.
Depending on the length of my vector register and the data types that I have, that effectively gives me different kinds of vector lens for different data types.
So you saw on Cell you have 128 bit registers and you can pack those with bytes, characters, bytes, shorts, integers, floats or doubles.
So each one of those gives you different kinds of a different vector lens.
SIMD is really now, SIMD extensions are increasingly popular.
They're available on a lot of ISAs.
Alt of x, MMX, SSE are available on a lot x86 machines.
And, of course, in Cell, in fact, on the SPU, all your instructions are SIMD instruction, and when you're doing a scaler instruction, you're actually using just one chunk of your vector register and your vector pipeline.
So how do you actually use these SIMD instructions?
Unfortunately, it's library calls or using inline assembly or using intrinsics.
You'll get hands-on experience with this with Cell, so you might complain about that when you actually do it.
Compile technology is actually getting better, and you'll see that one of the reasons we're using an XLC compiler is because it has these vector data types, which also latest versions of GCC have that allow you to express data types as vector data types, and the compiler can more easily or more naturally get the parallelism for you, SIMD parallelism with you having to go in there and do it by hand.
But if you were to do it by hand, or, in fact, what the compilers are trying to automate are different techniques for looking for where the SIMD parallelism is.
There was some work done here about six years ago by Sam Larson, who is now graduated, on super word level parallelism.
so I'm going to focus the rest of this talk on this concept of SIMDization because I think it's probably the one that's most useful for extracting parallelism in some of the codes that you're doing.
So this is really ideal for SIMD where you have really short vector lens, 2 to 8.
What you're looking for is SIMDization that exists within a basic block.
So within a code block, within a body of a loop or within some control flow even.
You can uncover this with simple analysis, and this really has pushed the boundary on what automatic compilers can do.
Some of work that's gone on at IMB, what they call the octipiler, has eventually been transferred to the XLC compiler do a lot of defecniques that build on SLP and expand in various ways to broaden the scope of what you can automatically parallize.
So here's an example of how you might actually derive SIMDization or opportunities for SIMDization.
So you have some code, let's say you're doing RGB computations where you're just adding the r elements, that's a red, green and blue.
So this might be in a loop, and what you might notice it well I can pack the RGB elements into one register.
I can pack these into another register, and I can pack these literals into a third register.
So that gives me a way to pack data together into SIMD registers, and now I can replace this scaler code with instructions that pack the vector register.
I can do the computations in parallel and I can unpack them.
We'll talk about that with a little bit more illustration in a second.
Any questions on this?
Perhaps the biggest improvement that you can get from SIMDization is by looking at adjacent memory references.
Rather than doing one load you can do a vector load, which really gives you a bigger bandwidth to memory.
So in this case, I have a load from I1, I2, and since these memory locations are continuous, I can replace them by one vector load that brings in all these data elements in one shot.
That essentially eliminates three load instructions, which are potentially most heavy weight for one ligher weight instruction because it amortizes bandwidth and exploits things like spatial locality.
Another one, vectorizable loops.
So this is probably one of the most advanced ways of exploiting SIMDization, especially in really long vector codes, so traditional supercomputers like the Cray, and you'll probably hear Simmon talk about this in the next lecture.
So I have some loop and I hvae this particular statement here.
So how can I get SIMD code out of this?
Anybody have any ideas?
Anybody know about loop unrolling?
So if I unroll this loop, I essentially -- that same trick that I had shown earlier, although I didn't quite do it this way.
I change a loop bound from n to -- the increment from -- rather than stepping through one at a time, stepping through four at a time.
Now the loop body, rather than doing one addition at a time, I'm doing four additions at a time.
So now this is very natural for a vectorization, right?
Vector load, vector load, vector store plus the vector add in the middle.
Is that intuitive?
So this gives you another way of extracting parallelism.
Looking at traditional loops, seeing whether you can actually unroll it in different ways, be able to get that SIMD parallelization.
The last one I'll talk about is about partial vectorization.
Either it might be some things where you have a mix of statements.
So here I have a loop where I have some load and then I'm doing some computation here.
So what could I do here?
It's not as symmetric as the other loop
There's no vector and [INAUDIBLE PHRASE]
Right.
So you might omit that.
But could you do anything about the subtraction?
[INAUDIBLE PHRASE]
If I can unroll this again, right?
Now there's no dependencies between this instruction and this instruction, so I can really move these together, and once I've moved these together then these loads become contiguous.
These loads are contiguous so I can replace these by vector codes, vector equivalents.
So now the vector load bring in L0, L1, I have the addition, that brings in those two elements in a vector, and then I can do my scaler additions.
But what do I do about the value getting out of this vector register into this scaler register that I need for the absolute values.
So this is where the benefits versus cost of SIMDization come in.
So the benefits are great because you can replace multiple instructions by one instruction, or you can just cut down the number of instructions by specific factor, your vector lens.
Low stores can be replaced by one wide memory operation, and this is probably the biggest opportunity for performance improvements.
But the cost is that you have to pack data into the data registers an you have to unpack it out so that you can have those kinds of communications between this vector register here and the value here, this value here and this value here.
Often you can't simply access vector values without doing this packing and unpacking.
So how do you actually do the packing, unpacking?
This is predominantly where a lot of the complexity goes.
So the value of a here is initialized by some function and the value of b here is initialized by some function, and these might not be things that I can SIMdize very easily.
So what I need to do is move that value into the first element of vector register, and move the second value into the second element of the vector register.
So if I have a four-way vector register, then I have to do four of these moves, and that essentially is the packing.
Then I could do my vector computation, which is really these two statements here.
Then eventually I have to do my unpacking because I have to get the values out to do this operation and this operation.
So there's an extraction that has to happen out of my SIMD register.
But you can amortize the cost of the packing and unpacking by just reusing your vector registers.
So these are like register allocation techniques.
So if I pack things into a vector register, I find all cases where I can actually reuse that vector register and I try to find opportunities for extra SIMDization.
So in the other case then, I pack one then I can reuse that same vector register.
So what are some ways I can look for to amortize the cost?
The interesting thing about memory operations is while there are many different ways you can pack scaler values into a vector register, there's really only one way you can pack loads coming in from memory into a vector register is because you want the loads to be sequential, you want to exploit the spatial locality.
So one vector load really gives you specific ordering.
So, that really constrains you in various ways.
So you might bend over backwards in some cases to actually get your code to be able to you reuse the wide-word load without having to do too much packing or unpacking because that'll start eating into your benefits.
So simple example of how you might find the SLP parallelism.
So the first thing you want to do is start with are the instructions that give you the most benefit, so it's memory references.
So here there are two memory references.
They happen to be adjacent, so I'm accessing a contiguous memory chunks, so I can parallelized that.
That would be my first step.
I can do a vector load and that assignment can become a and b. I can look for opportunities where I can propagate this vector values within the vector register that's holding a and b.
So the way I do that is I look for uses of a and b.
In this case, there are these two statements.
So I can look for opportunities to vectorize that.
So in this case, both of these instructions are also vectorizable.
Now I have a vector subtraction, and I have a vector register holding new values h and j.
So I follow that chain again of where data's flowing.
I find these operations and I can vectorize that as well.
So, sign up with a vectorizable loop where all my instructions, all my scale instructions are now in SIMD instructions.
I can cut down on loop iterations of total number of instructions that I issue.
But I've made some implicit assumption here.
Anybody know what it is?
Do you actually need that many iterations of the loop?
Well, so you can factor down the cost.
so here I've vectorized by 2, so I would cut down the number of iterations by 2
You could have an odd number of iterations?
Right, so you could have an odd number of iterations.
What do you do about the remaining iterations.
You might have to do scaler code for that.
What are some other assumptions?
Maybe it will be clear here.
So in vectorizing this, what have I assumed about relationships between these statements?
I've essentially reorganized all the statements so that assumes I have this liberty to move instructions around.
Yup?
[UNINTELLIGIBLE] a and b don't change [INAUDIBLE PHRASE]
Right.
So there's nothing in here that's changing the values.
There's no dependencies between these statements -- no flow dependencies and no other kind of constraints that limit this kind of movement.
So in real code it's not actually the case.
You end up with patterns of computation where you can get really a nice case of classic cases you can vectorize those really nicely.
In a lot of other codes you have a mix of vectorizable code and scaler code and there's a lot of communication between the two.
So the cost is really something significant that you have to consider.
This was, as I mentioned, done in somebody's Masters thesis and eventually led to some additional work that was his PhD thesis.
So in some of the early work, what he did was he looked at a bunch of benchmarks and looked at how much available parallelism you have in terms of this kind of short vector parallelism, or rather SLP where you're looking for vectorizable code within basic blocks, which really differed from a classic way of people looking for vectorization.
[?
And you ?]
have well-structured loops and doing kinds of transformations you'll hear about next week.
So for different kinds of vector registers, so these are your vector lens.
So going from 128 bits to 1,024 bits, you can actually reduce a whole lot of instructions.
So what I'm showing here is the percent dynamic instruction reduction.
So if I take my baseline application and just compile it in a normal way and I run it again an instruction count.
I apply this SLP technique that find the SIMDization and then run my application again, use the performance counters to count the number of instructions and compare the two.
I can get 60%, 50%, 40%.
In some cases I can completely eliminate almost 90% or more of the instructions.
So it's a lot of opportunity for performance improvements that might be apparent.
One because I'm reducing the instruction bandwidth, I'm reducing the amount of space I need in my instruction cache, I have fewer instructions so I can fit more instructions into my instruction cache, you reduce the number of branches.
You get better bandwidth to the memory, better use of the memory bandwidth.
Overall, you're running fewer iterations, so you're getting lots of potential for performance.
So, I actually ran this on the AltiVec.
This was one of the earliest generations of AltiVec, which SIMD instructions didn't have I believe double precision floating point, so not all the benchmarks you see on the previsou slide are here, only the ones that could run reasonably accurately with a single precision floating point.
What they measure is the actual speed up.
Doing this SIMDization versus not doing a SIMDization, how much performance you can get.
The thing to take away is in some cases where you have nicely structured loops and some nice patterns, you can get up to 7x speed up on some benchmarks.
What might be the maximum speed up that you can get depends on the vector lens, so 8, for example on some architectures depending on the data type.
Is there any questions on that?
So as part of the next recitation, you'll actually get an exercise of going through and SIMDizing for Cell, and whether that actually means SIMDize instructions for Cell might take statements and sort of replace them by intrinsic functions, which eventually map down to actually assembly op codes that you'll need.
So you don't actually have to program at the assembly level, although in effect, you're probably doing the same thing.
Last thing we'll talk about today is optimizing for the memeory hierarchy.
In addition to data level parallelism, looking for performance enhancements in the memory system gives you the best opportunities because of this big gap in performance between memory access latencies and what the CPU efficiency is.
So exploiting locality in a memroy system is key.
So these concepts of temporal and spatial locality.
So let's look at an example.
Let's say I have a loop and in this loop I have some code that's embodied in some function a, some code embodied in some function b, and some code in some function c. The values produced by a are consumed by the function b, and similarly the values consumed by b are consumed by c. So this is a general data flow graph that you might have for this function.
Let's say that all the data could go into a small array that then I can communicate between functions.
So if I look at my actual cache size and how the working set of each of these functions is, so let's say this is my cache size -- this is how many instructions I can pack into the cache.
Looking at the collective number of instructions in each one of these functions, I overflow that.
I have more instructions I can fit into my cache any one time.
So what does that mean for my actual cash performance?
So when I run a, what do I expect the cache hit and miss rate behavior to be like?
So in the first iteration, I need the instructions for a. I've never seen a before so I have to fetch that data from memory and put in the cache.
So, the attachments.
So what about b?
Then c?
Same thing.
So now I'm back at the top of my loop.
So if everything fit in the cache then I would expect a to be a what?
You'll be a hit.
But since I've constrained this problem such that the working set doesn't really fit in the cache, what that means is that I have to fetch some new instructions for a.
So let's say I have to fetch all the instructions for a again.
That leads me to another miss.
Now, bringing a again into my cache kicks out some extra instructions because I need to make room in a finite memory so I kick out b.
Bring in b and I end up kicking out c. So you end up with a pattern where everything is a miss.
This is a problem because the way the loop is structured, collectively I just can't pack all those instructions into the cache, so I end up taking a lot of cache misses and that's bad for performance.
But I can look at an alternative way of doing this loop.
I can split up this loop into three where in one loop I do all the a instructions, in the second loop I do all the b's, and the third loop I do all the c's.
Now my working set is really small.
So the instructions for a fit in the cache, instructions for b fit in the cache, and instructions for c fit in the cache.
So what do I expect for the first time I see a?
Miss.
Then second time?
It'll be hit, because I've brought in a, I haven't run b or c yet, the number of instructions I need for a is smaller than what I can fit into the cache, so that's great.
Nothing gets kicked out.
So every one of those iterations for a becomes a hit.
So that's good.
I've improved performance.
For b I have the same pattern.
First time I see b it's a miss, every time after that it's a hit.
Similarly for c. So my cache miss rate goes from being one, everything's a miss, to decreasing to 1 over n where n is essentially how much I can run the loop.
So we call that full scaling because we've taken the loop where we've distributed, and we've scaled every one of those smaller loops to the maximum that we could get.
Now what about the data?
So we have the same example.
Here we saw that the instruction working set is big, but what about the data?
So let's say in this case I'm sending just a small amount of data.
Then the behavior is really good.
It's a small amount of data that I need to communicate from a to b.
A small amount of data you need to communicate from b to c. So it's great.
No problems with the data cache.
What happens in full scaling case?
It's not correct to communicate from a to b
What do you mean it's not correct?
Oh, it's not communicating at the same time
Yeah, it's not at the same time.
In fact, just assume this is sequential.
So I run a, I store some data, and then when I run b I grab that data.
This is in sequential
How do you know that the transmission's valid then?
We could use some global variable
Simple case.
There's no global variables.
All the data that b needs comes from a.
So if I run a I produce all the data and that's all that b needs.
So in the full scaling case, what do I expect to happen for the data?
Remember, in the full scaling case, all the working sets for the instructions are small so they all fit in the cache.
But now I'm running a for a lot longer so I have to store a lot more data for b.
Similarly, I'm running b for a lot longer so I have to store a lot more data for c. So what do I expect to happen with the working set here?
Instructions are still good, but the data might be bad because I've run a for a lot more iterations at one shot.
So now I have to buffer all this data for a to b.
Similarly, I've run b for a long time so I have to buffer a whole lot data for b to c. Is that clear?
No
So let's say every time a runs it produces one data element.
So now in this case, every iteration produces one data element.
That's fine.
That's clear?
Here I run a n times, so I produce n data elements.
And b let's say produces one data element.
So if my cache can only hold let's say n by 2 data elements, then there's an overflow.
So what that means is not everything's in the cache, and that's bad because of the same reasons we saw for the instructions.
When I need those data I have to go out to memory and get them again, so it's extra communication, extra redundancy
In this case where you don't need to store the a variables [UNINTELLIGIBLE PHRASE]
But notice these were sequential simple case.
I need all the data from a to run all the iterations for b.
Then, yeah, this goes away.
So let's say this goes away, but still b produces n elements and that overflows the cache.
So there's a third example where I don't fully distribute everything, I partially distribute some of the loops.
I can fully scale a and b because I can fit those instructions in the cache.
That gets me around this problem, because now a and b are just communicating one day data element.
But c is still a problem because I still have to run b n times in the end before I can run c so there are n data elements in flight.
So the data for b still becomes a problem in terms of its locality.
Is that clear?
So, any ideas on how I can improve this?
assuming you have the wrong cache line and you have to do one or two memory acceses to get the cache back
So, programs typically have really good instruction locality just because the nature of the way we run them.
We have small loops and they iterate over and over again.
Data is actually where you spend most of your time in the memory system.
It's fetching data.
So I didn't actually understand why you think data is less expensive than instructions
What I'm saying say you want to read an array, read the first, say, 8 elements to 8 words in the cache box.
Well then you'd get 7 hits, so every 8 iterations you have to do a rewrite
Right.
So that assumes that you have really good spatial locality, because you've assumed that I've brought in 8 elements and I'm going to use every one of them.
So if that's the case you have really good spatial locality and that's, in fact, what you want.
It's the same kind of thing that I showed for the instruction cache.
The first thing is a miss, the rest are hits.
The reason data is more expensive, you simply have a lot more data reads than you have instructions.
Typically you have small loops, hundreds of instructions and they might access really big arrays that are millions of data references.
So that becomes a problem.
So ideas on how to improve this?
That's a loop?
That's a loop.
So what would you do in the smaller loop?
[INAUDIBLE PHRASE]
Something like that?
Yeah
OK.
So in a nested loop, you have a smaller loop that has a small number of iterations, so 64.
So, 64 might be just as much as I can buffer for the data in the cache.
Then I wrap that loop with one outer loop that completes the whole number of iterations.
So if I had to do n, then I divide n by 64.
So that can work out really well.
So there's different kinds of blocking techniques that you can use on getting your data to fit into your local store or into your cache to exploit these spatial and temporal properties.
Question?
Would it not be better to use a small [UNINTELLIGIBLE] size so you could run a, b, c sequentially?
You could do that as well.
But the problem with running a, b, c sequentially is that if they're in the same loop, you end up with instructions being bad.
That would really, this case -- so even if you change this number you don't get around the instructions.
So you're going to see more optimizations that do more of these loop tricks.
I talk about unrolling without really defining what unrolling is or going into a lot of details.
Loop distribution, loop fision, some of the things, like loop tiling, loop blocking.
I think Simmon's going to cover some of these next week.
So this was implemented, this was done by another Master student at MIT who graduated about two years ago, to show that if you factor in cache constraints versus ignoring cache constraints, how much performance you can get.
This was done in the context of StreamIt.
So, in fact, some of you might have recognized a to b to c as being interconnected as pipeline filters.
We ran it on different processors, so the StrongARM processor's really small.
InOrder processor has no L1 cache in this particular model that we used.
But it had a really long latency -- sorry, it had no L2 cache.
It had really long latency to memory.
Pentium, an x86 processor.
Reasonably fast.
It had a complicated memory system and a lot, a lot of memory overlap in terms of references.
Then the Itanium processor, which had a huge L2 cache at its disposal.
So what you can see is that lower bars indicate bigger speed ups.
This is normalized run time.
So on the processor where you don't actually have caches to save you, and the memory communication is really expensive, you can get a lot of benefit from doing the cache aware scaling, that loop nesting to take advantage of packing instructions instead of instruction cache, packing data into data cache and not having to go out to memory if you don't to.
So you can reduce run time to about 1/3 of what it was with this kind of cache optimization.
On the Pentium3 where you have a cache to help you out, the benefits are there, but you don't get as big a benefit from ignoring the cache constraints versus being aware of the cache constraints.
So here you're actually doing some of that middle column whereas here we're doing third columns, the cache aware fusion.
In a Itanium you really get no benefits between the two.
Yep?
Can you explain what the left columns are?
These?
Yeah
So this is tricky.
So the left columns are doing this
OK, sort of assuming that icache is there
Right, and the third column is doing this.
So you want to do this because the icache locality is the best.
So you always want to go to full or maximum scaling.
I'm actually fudging a little just for sake of clarity.
Here you're actually doing this nesting to improve both the instruction and the data locality.
So you can get really good performance improvement.
So what does that mean for your Cell projects or for Cell, we'll talk about that next week at the recitation.
Yeah?
Is there some big reasons [UNINTELLIGIBLE PHRASE]
Well it just means that if you have caches to save you, and they're really big caches and they're really efficient, the law of diminishing returns.
That's where profiling comes in.
So you look at the profiling results, you look at your cache misses, how many cache misses are you taking.
If it's really significant, then you look at ways to improve it.
If your cache misses are really low, you missed rate is really low, then it doesn't make sense to spend time and energy focusing on that.
Good question.
So, any other questions?
So summarizing the gamut of programming for performance.
So you tune to parallelism first, because if you can't find the concurrency, your Amdahl's law, you're not going to a get a whole lot of speed up.
But then once you figured out what the parallelism is, then what you want to do is really get the performance on each processor, the single track performance to be really good.
You shouldn't ignore that.
The modern processors are complex.
You need instructional level parallelism, you need data level parallelism, you need memory hierarchy optimizations, and so you should consider those optimizations.
Here, profiling tools could really help you figure out where the biggest benefits to performance will come from.
You may have to, in fact, change everything.
You may have to change your algorithm, your data structures, your program structure.
So in the MPEG decoder case, for example, I showed you that if you change the flag that says don't use double precision inverse DCT, use a numerical hack, then you can get performance improvements but you're changing your algorithm really.
You really want to focus on just the biggest nuggets -- where is most of the performance coming in, or where's the biggest performance bottleneck, and that's the thing you want optimize.
So remember the law of diminishing returns.
Don't spend your time on doing things that aren't going to get you anything significant in return.
That's it.
Any questions?
OK. How are you guys doing with the projects?
So, one of the added benefits of the central CBS repository is I get notifications too when you submit things.
So I know of only two projects that have been submitting things regularly.
So, I hope that'll pick up soon.
I guess a few minutes to finish the quiz and then we'll see you next week.
Have a good weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started.
So what we are going to do today is go about discovering other alternating methods.
We know you guys are amazing hackers and you can actually do all those things by hand.
But to make multi-core generally acceptable, can we do things automatically?
Can we really reduce a burden from the programers?
So at the beginning I'm going to talk about general parallelizing compilers.
What people have done.
What's the state of the art.
Kind of get your feel what is doable.
Hopefully, that will be a little over an hour, and then we'll go talk about StreamEd compiler, what we have done recently, and how this automation part can do.
So, I'll talk a little bit about parallel execution.
This is kind of what you know already.
Then go into parallelizing compilers, and talk about how to determine if something is parallel by doing data dependence analysis, and how to increase the amount of parallelism available in code loop, what kind of transformation.
Then we go look at how to generate code, because once you see that something is parallel, how you actually get to run parallel.
And finish up with actually how to do communication code in a machine such as a server.
So in parallel execution, this is something -- it's a review.
So there are many ways of parallelism, things like instruction level parallelism.
It's basically effected by hardware or compiler scheduling.
As of today this is in abundance.
In all for scalars we do that, in [OBSCURES] we do that.
Then password parallelism, it's what most of you guys are doing now.
You probably find a program, you divide it into tasks, you get task level parallelism, mainly by hand.
Some of you might be doing data level parallelism and also loop level parallelism.
That can be the hand or compiler generated.
Then, of course, pipeline parallelism is more mainly done in hardware and language extreme, do pipeline parallelism.
Divide and conquer parallelism we went a little bit more than in hardware, mainly by hand for recursive functions.
Today we are going to focus on loop level parallelism, particularly how do loop level parallelism by the compiler.
So why loops?
So loops is interesting because people observed in morse code, 90% of execution time is in 10% of the code.
Almost 99% of the execution time is in 10% of the code.
This called a loop, and it makes sense because running at 3 gigahertz, if only run one instruction one, then you run through the hard drive in only a few minutes because you need to have repeatability.
A lot of time repeatability thing loops.
Loops, if you can parallelize, you can get really good performance because loops most of the time, each loop iteration have the same amount of work and you get nice good load balance, it's somewhat easier to analyze, so that's why the compiler start there.
Whereas if you try to get task level parallelism, things have a lot more complexities that automatic compiler cannot do.
So there are two types of parallel loops.
One is a for all loop.
That means there are no loop carried dependences.
That means you can get the sequential code executing, run everything in parallel, and at the end you have a barrier and when everybody finishes you continue on the sequential code.
That is how you do a for all loop.
Some languages, in fact, have explicitly parallel construct, say OK, here's a for all loop and go do that.
The other type of loop is called a foracross or doacross loop.
That says OK, while the loop is parallel, there are some dependences.
That means some value generated here is used somewhere here.
So you can run it parallel, but you have some communication going too.
So you had to move data.
So it's not completely running parallel, there's some synchronization going on.
But you can get large chunk running parallels.
So we kind of focus on dual loops today, and let's look at this example.
We see it's a for far so it's a parallel loop or for all loop.
When you know it's parallel, in here, of course, the user said that.
What we can do is we can distribute the iteration by chunking up the iteration space into number of process chunks, and basically run that.
If PMD mode, you can at the beginning the first processor can calculate the number of iterations you can run on each process in here, and then you synchronize, you put a barrier there, so everybody kind of sync up at that point.
Or other process of waiting, and at that point, everybody starts, when you reach this point it's running, it's part of iterations, and then you're going to put a barrier synchronization in place.
Kind of obvious, parallel code basically in here, running on shared memory machine at this point.
So this is what we can do.
I mean this is what we saw before.
Of course, instead of doing that, you can also do fork join types or once you want to run something parallel, you can fork a thread and each thread gets some amount of iterations you run, and after that you merge together.
So you can do both.
So that's my hand.
How do you do something like that by the compiler?
That sounds simple enough, trivial enough.
But you don't automate the entire process.
How to go about doing that.
So, here are some normal loops, for loops.
So the for all does this thing that was so simple, which is the for all construct that means somebody could look at that and said this loop is parallel.
But you look at these FOR loops, how many of these loops are parallel?
Is the first loop parallel?
Why?
Why not?
[OBSCURED.]
It's a loop because the iteration, one of that is using what you wrote in iteration zero.
So iteration one has to wait until iteration zero is done, so and so.
How about this one?
Why?
[NOISE.]
Not really.
So it's writing element 0 to 5, it's reading elements 6 to 11.
So they don't overlap.
So what you read and what you write never overlap, so you can keep doing it in any order, because the dependence means something you wrote, later you will read.
This doesn't happen in here.
How about this one?
There's no dependence in there
Why?
[OBSCURED.]
So you're writing even, you're reading odd.
So there's no overlapping or anything like that.
Question?
OK.
So, the way to look at that -- I'm going to go a little bit of formalism.
You can think about this as a iteration space.
So iteration is if you look at each iteration separately, there could be thousands and millions of iterations and your compiler never [COUGHING] doing any work, and also some iteration space is defined by a range like 1 to n, so you don't even know exactly how many iterations are going to be there.
So you can represent this as abstract space.
Normally, most of this loops you look at you normalize to step one.
So what that means is all the integer points in that space.
So if you have a loop like this, y equals 0 to 6, J equals 1i to 7.
That's the iteration space, there are two dimensions in there.
The points that start iteration off because it's not a rectangular space, it can have this structure because j's go in triangular in here.
So the way you can represent that is so you can represent iteration space by a vector i, and you can have each dimension or use two dimension.
This was some i1, i2 space in here.
So you can represent it like that.
It's the notion of lexicographic ordering.
That means if you execute the loop, what's the order you're going to receive this thing.
If you execute this loop, what you are going to do is you go from -- you go like this.
This is lexicographical ordering of everything in the loops.
That's the normal execution order.
That's a sequential order.
At some point you want to make sure that anything we do kind of has a look and feel of the sequential lexicographical order.
So, one thing you can say is if you have multiple dimensions, if there are two iterations, one iteration lexicographical and another iterations says if all outer dimensions are the same, you go to the first dimension where the numbers, they are in two different iterations.
Then that dictates if it's lexicographical than the other.
So if the outer dimensions are the same, that means the next one decides, the next one decides, next one decides going down.
First one that's actually different decides who's before the other one.
So another concept is called affine loop nest.
Affine loop nest says loop bounds are integer linear functions of constants, loop constant variable and outer loop indices.
So that means if you want to get affine function within a loop, that has to be a linear function or integer function where all the things either has to be constant or loop constants -- that means that that variable doesn't change in the loop or outer loop indices.
That makes it much easier to analyze.
Also, array axises, each dimension, axis function has the same property.
So of course, there are many programs that doesn't satisfy this, for example, if we do FFD.
That doesn't satisfy that because you have exponentials in there.
But what that means is at 50, there's probably no way that the compiler's going to analyze that.
But most kind of loops fit this kind of model and then you can put into nice mathematical framework and analyze that what I'm going to go through is kind of follow through some of the mathematical framework with you guys.
So, what you can do here is if you look at this one, instead of representing this iteration space by each iteration, which can be huge or which is not even known at compile time, what you can do is you can represent this by kind of a bounding space of iterations, basically.
So what this is, we don't mark every box there, but we say OK, look, if you put these planes -- I put four planes in here, and everything inside these planes represent this iteration space.
That's nice because instead of going 0 to 6, if you go 0 to 60,000, still I have the same equation, I don't suddenly have 6 million data points in here I need to represent.
So, my representation doesn't grow with the size of my iteration space.
It grows with the shape of this iteration space.
If you have complicated one, it can be difficult.
So what you can do is you can iteration space, it's all iterations zero to six, j's I27.
This is all linear functionns.
That makes our analysis easier.
So the flip side of that is the data space.
So, if m dimension array has m dimensional discrete cartesian space.
Basically, in the data space you don't have arrays that are odd shaped.
It's almost a hypercube always.
So something like that is a one dimensional space and something can be represented as a two dimensional space in here.
So data space has this nice property, in that sense it's a t multi-dimensional hypercube.
And what that gives you is kind of a bunch of mathematical techniques to kind of do and at least see some transformations we need to do in compiling.
As humans, I think we can look at a lot more complicated loops by hand, and get a better idea what's going on.
But in a compiler you need to have a very simple way of describing what to analyze, what to formulate, and having this model helps you put it into a nice mathematical frame you can do.
So the next thing is dependence.
We have done that so I will go through this fast.
So the first is a true dependence.
What that means is I wrote something, I write it here.
So I really meant that I actually really use that value.
There are two dependences mainly because we are finding dependence on some location, is an anti-dependence.
That means I can't write it until this read is done because I can't destroy the value.
Output dependence is there, so ordering of writing that you need to maintain.
So in a dynamic instance, data dependence exist between i and j if Either i and j is a write operation, and i and j refers to the same variable, and i executes before j.
So it's the same thing, one execute before the other.
So it's not that you don't have a dependence when they get there in time, then it become either true or anti.
So it's always going to be positive over time.
So how about other accesses?
So one element, you can figure out what happened.
So how do you do dependence and other accesses?
Now things get a little bit complicated, because arrays is not one element.
So that's when you go to dependence analysis.
So I will describe this using bunch of examples.
So in order to look at arrays, there are two spaces I need to worry about.
One is the iteration space, one is the data space.
What we want to do is figure out what happens at every iteration for data and what other dependences kind of summarize this down.
We don't want to look at, say OK, one iteration depend on second, two depend on third -- you don't want to list everything.
We need to come up with a summary -- that's what basically dependence analysis will do.
So if you have this access, this is this loop.
What happens is as we run down, so iterations we are running down here.
So we have iteration zero, 1, 2, 3, 4, 5.
First do the read, write, read, write.
So this is kind of time going down there.
What you do is this one you are reading and you are writing.
You're reading and writing, so you have a dependence like that.
You see the two anti-dependence.
Read -- anti-dependence, I have anti-dependence going on here.
If you look at it, here's a dependence vector.
What that means is there's a dependence at each of those things in there -- that's anti-dependence going on.
One way to look at summarizes of this, what is my iteration.
My iteration goes like -- what's my dependence.
I have anti-dependence with the same iteration, because my read and write has to be dependence in the same iteration.
So this is a way to kind of describe that.
So a different one.
This one.
I did Ai plus 1 equals Ai So what you realize is iteration zero, you wrote iteration zero, you wrote a zero, you read these and you wrote A1, and iteration 1, you read A1 and wrote A2, basically.
Now what you have is your dependence is like that, going like that.
So if you look at what's happening in here, if you summarize in here, what you have is a dependence going like that in iteration space.
So in iteration that means iteration 1 is actually these two dependence, that uses something that wrote iteration zero, iteration 2 you have something iteration 1, and you have iteration going like that.
Sometimes this can be summarized as the dependence vector of 1.
Because the previous one was zero because there's no loop carry dependency.
In the outer loop there's a dependence on 1.
So if you have this one, I plus 2, of course, it gets carried 1 across in here and then you have a 1 skipped representation in here.
If you have 2I2 by plus 1, what you realize is there's no overlap.
So there's no basically dependency.
You kind of get how that analytic goes.
So, to find data dependence in a loop, so there's a little bit of legalese.
So let me try to do that.
So for every pair of array accesses, what you want to find is is there a dynamic instant that happened?
An iteration that wrote a value, and another dynamic instance happened that later that actually used that value.
So the first access, so there's a dynamic instance that's wrote, or that access, and another iteration instance that also accessed the same location later.
And one of them has to be right, otherwise there are two in anti.
That's the notion about the second one came after the first one.
You can also look at the same arrays.
It doesn't have the be the same as different access, the same array access if you are writing.
If you look at same array access writing you can have output dependences also.
So it's basically between a read and a write, and a write and a write.
Two different writes, it can be the same write too.
Key thing is we are looking at location.
We're not looking at value path and say who's actually in the same location.
Loop carry dependence means the dependence cross a loop boundary.
That means the person who read and person who wrote are in different loop iteration.
If it's in the same iteration, then it's all local, because in my iteration I deal with that, I moved data around.
But what I'm writing is used by somebody else in different iteration, I have loop carry dependence going on.
Basic thing is there's a loop carry dependence, that loop is not parallelized in that.
What that means is I am writing in one iteration of the loop and somebody is reading in different iteration of the loop.
That means I actually had to move the data across, they can happen in parallel.
That's a very simple way of looking at that.
So, what we have done is -- OK, the basic idea is how to actually go and automate this process.
The simple notion is called a data dependence analysis, and I will give you a formulation of that.
So what you can formally do is using a set of equations.
So what you want to say is instead of two distinct iterations, one is the write iteration, one is the read iteration.
One iteration writes the value, one iteration reads the value.
So write iteration basically, writes a item loop plus 1, the read iteration reads AI.
So we know both read and write have to be within loop bound iteration, because we know that because we can't be outside loop bounds.
Then we also want to make sure that the loop carried dependence, that means read and write can't be in the same iteration.
If it's in the same iteration, I don't have loop carry dependence.
I am looking for loop carry dependence at this point.
Then what makes both of the read and write write the same location.
That means access 1 has to be the same.
So the right access point is iw plus 1, and read access function is [?
IEI.
?]
So the key thing is now we have set up equation.
Are there any values for ie and j, integer values, I'm sorry, iw and ir that these equations are true.
If that is the case, we can say ah-ha, that is the case, there's an iteration that the write and read are writing into two different iterations -- one write iteration, one read iteration, writing to the same value.
Therefore that's a different [OBSCURED].
Is this true?
Is there a set of values that makes this true?
Yeah, I mean you can do ir equals 1, iw equals 1, and ir equals 2.
So there's a value in there so these equations will come up with a solution, and at that point you have a dependency
[NOISE]
So that's very easy to make this formulation.
So if the indices is calculated with some thing or loop value, I can't write the formulation.
So the data that I can do this analysis is this indices has to be the constant or indefinite.
This is A of b of I.
So if my array is A of b of i, I don't know how the numbers work if you have A of b i. I have no idea about Ai is without knowing values of B of i.
And B of i, I can't summarize it.
Each B of i might be different and I can't come up with this nice single formulation that can check out every B of i.
And I'm in big trouble.
This is doable, but this is not easy to do like this.
Question?
[NOISE]
Yeah, that's right.
So that the interesting thing that you're not looking at.
Because when we summarized it, because what you are going to do is we are trying to summarize for everything, every iteration, and we are not trying to divide it into saying OK, can I find the parallel groups.
Yes.
You can do some more complicated analysis and do something like that.
Yes.
So other interesting thing is OK, the next thing you want to see whether can find output dependence.
OK, are there two different iterations that they're fighting the same thing.
What that means is the iterations are I1, I2, and I1 not equals I2, and I1 plus 1 equals I2 plus one.
There's no solution to this one because the I1 has to be equal to I2 according to this, and I1 cannot be equal to I2 during this one.
That says OK, look, I don't have output dependence because it can be satisfied.
OK, so here I know I have a loop carried -- I haven't said the two anti depends on which directions this is.
Two anti-dependents, but I don't have a loop carried out to [OBSCURED].
So how do we generalize this?
So what you can do is as integer vector I, so in order to generalize this, you can use integer programming.
How many of you know integer programming or linear programming?
OK. We are not going to go into detail, but I'll tell you what actually happen.
So integer programming says there's a vector of variable I, and if you have a formulation like that, is array, AI is less than or equal to B, A and B are all constant integers, and you can use the integer programming, you can see that there's a solution for IE or not.
This is if you do things like operations research, there's a lot of work around it.
People actually want to know what value is Y.
We don't care that much what values, we just want to know the solution or not.
If there's a solution, we know that there's a dependent.
If there's no solution we know there's no dependent.
So we need to do is we need to get this set of equations and put it on that form.
That's simple.
For example, what you want is AI less than B -- that means you have constnat A1 I1, plus A2 i2, which is less than or equal to B.
So you won't have this kind of a system.
Not equals doesn't really belong there.
So the way you deal with not equals if you do it in two different problems.
You can say IW less than IER is one problem, and W is greater then IER is other problem, and if either problem has a solution, you have a dependence.
So that means one is true and one is anti.
You can see the true dependence or anti-dependence, you can look at that.
This one is a little bit easier.
This is less than, not actually less than -- less than equal.
How do you deal with equal?
So the way you deal with equal is you write in both directions.
So if A is less than B, A less than or equal to B, B is less than or equal to A means actually is equal to B.
So you can actually try two different inequalities and get equal to down there.
So you have to kind of massage things a little bit in here.
So here are our original iteration bounds, and here's our one problem because we are saying write happens before read, so these are two dependents that we are looking at.
This is saying that write location is the same as the read location and this is equal, so I have two different equations in here.
So kind of massage this a little bit to put it in i form, and we can come up with A's and B's.
These are just manual steps, A's and B's, and now we are going to throw it into some super duper integer linear program package and it will say yes or no and your set.
And of course, you had to do another problem for the other side.
You can generalize it for much more complete loop nest.
So if you have this complicated loop nest in here, you had to solve you've got n deepness, you have to solve two end problems with all these different constraints.
I'm not going to go over this.
I have the slides in here.
So that's the single dimension.
So how about multi-dimension dependences?
So I have two dimensional iteration space here, and I have I,J equals AI, J minus 1.
That's my iteration space.
What does my dependence look like?
We have arrows too.
Which direction are the arrows going?
[OBSCURED]
We have something like this.
Yup.
We have something like this because that's J minus 1, the I's are the same.
Of course, if you have the other way around, go other direction, one is anti and one is it two dependence, so you can figure that one out.
And do something complicated.
First one.
So IJ, I minus 1, J plus 1.
Which has to be diagonal.
Which diagonal does it go?
This way or this way?
Who says this way?
Who says this way?
So, this is actually going in this direction.
This is where you have to actually think which iteration is actually write and read in here.
So things get complicated.
This one is even more interesting.
This one.
There's only one dimensional array or two dimensional loop nest.
So what that means is who's writing and who's reading?
If you look at it basically -- actually this actually is a little bit wrong, because the dependence analysis says -- actually, all these things, all this read has to go into all the write, because they are writing any J, just writing the same thing.
So this is a little bit wrong.
This is actually more data flow analysis.
This is a different -- their dependence means I don't care who the guy wrote, because he's the last guy who wrote, but everybody's reading, everybody else is writing the same location
[OBSCURED]
Keep rewriting the same thing again and again and again.
You start depending on -- It's not dependant on J's it's dependant on I.
But location says you used to have iterations right in the same location, different J.
So not matter what J, it's writing in the same location.
You know what I'm saying?
Because J thinks
[NOISE]
This is iteration space.
I am looking at iteration.
I am looking at I and J.s
[OBSCURED]
B is a one dimensional array.
So B is a one dimensional array.
So what that means is -- The reason I'm saying it's the iteration space and array space is a match.
I'll correct this and put it in there because this is a data flow diagram.
It's row independant.
This one writing to what?
[OBSCURED]
Iteration space is I and J.
So, this is writing to what?
I zero is -- This is writing to what?
B1.
All those things are writng to B1.
This is really -- So this is writing to B1, this is reading B zero.
So this iteration is reading B1 again.
So this was B1, this is iteration B1.
So each of these is writing to B1, each of these are reading from B1, so each has to be dependent from each other
So I guess one thing that's confusing here is why isn't it just -- why don't we just have arrows going down the column?
Why do we have all these--?
Arrows going down the column means each is trying to do different location.
So what happens is that this one, arrays going down this way.
Is this one -- what's wrote here is only that location, only this side I accidentally located.
These are all writing to the same location and reading from the same location
Why isn't B iterated?
This is iteration space.
I have two different loops here
But I don't understand why B [NOISE.]
This is my program.
I can write this program.
This is a little bit of a stupid program because I am kind of trying to do the same thing again and again.
But hey, my program doesn't say array dimensions has to match your loop dimension.
It doesn't say that so you can have programs like that.
You can have other way too.
So the key thing is to make -- don't confuse iteration space versus array space.
They are two different spaces, two different number of dimensions.
That's all the point that I'm going to make here.
So by doing dependence analysis, you can figure out -- now you can formulate this nicely -- figure out where the loops are parallel.
So that's really neat.
The next thing I'm going to go is trying to figure out how you can increase the parallelism opportunities.
Because there might be cases where the original code you wrote, there might be some loops that are not parallelizable, assays, and can you go and increase that.
So I'm going to talk about few different possibilities of doing that.
Scalar privatization, I will just go in each of these separating.
So here is interesting program.
To get parallel to the temporary and use the temporary in here.
You might not know you had written that but the compiler normally generates something like that because you always had temporaries in here, so this might be what compiler generate.
Is this loop parallel?
Yup
Why?
[OBSCURED]
Is the loop carry dependence true or anti -- What's the true dependence which to which?
We didn't loop true dependence.
What is the loop carry dependence?
Anti-dependence.
Because I cannot -- you see I equal 1, basically wrote here in this reading.
I can't write I equals 2x until I equals 1 is done and done reading that.
I have one location and everybody's trying to read or write that, even though I don't really use data.
This is the sad thing about this.
That I'm really using this guy's data, but I'm just waiting for the same space to occupy.
So, there's a loop carry dependence in here, and it's anti-dependent.
So what you can do is if you find any anti or output loop carry dependence, you can get rid of them.
I'm not really using that value, I'm just keeping a location in here.
So how can we get rid of that?
[OBSCURED]
Yeah.
That's one thing.
There's two ways of doing it.
One is I assign something local.
So each processor will have its own copy, so I don't do that.
So it's something like this, so that's [OBSCURED].
Or I can look at the array.
In the array you can have either number of process or iterations for each iteration.
But uses a different location.
This is more efficient than this one because we are touching lot more locations in here.
I haven't done one thing here.
I'm not complete.
What have I forgotten to do in both of these?
[OBSCURED]
Yeah, because it was beforehand somebody might use final assignment of the loop nest, so what you had to do is you had to kind of finalize x.
Because I had a temporary variable, so with n, the last value has to go into x.
You can't keep just not calculating value in something.
So in here, also, you just say last value is x.
But after you do that, basically now each of this loop is faster.
Everybody go that?
OK, here's another example.
x equals x plus AI.
Do I have loop carry dependent?
What did the loop-carried dependence?
True or anti?
True dependence.
So this guy is actually creating previous value and adding something in the event.
So of course in true dependence I cannot seem to parallelize.
But there are some interesting things we can do.
That was an associative operation.
I didn't care which order this initial happened, so I'm just keeping a lean bunch of values in here.
And the results were never used in the other loop.
So we just keep adding things and at the end of the loop you get the sum total in here.
I never used any kind of partial values anywhere.
So that gives the idea.
So what you can do is we can translate this into each of the guys doing a temporary addition into its own variable.
So each processor, just do a partial sum.
At the end, once they're done, you basically do the full sum.
Of course, you can do a tree or whatever much more complicated thing then that -- you can also parallelize this part at the tree addition.
But you can do that.
I mean Roderick talked about this in hand parallelization.
But we are doing something very simple in here.
So these compilers can figure out associative operations and do that.
So this is where all the people who are in parallelizing, and all the people who are writing this scientific code kind of start having arguments.
Because they say oh my God, you're doing operations and it's going to have numerical stability issues.
Yes all true.
In compilers you have these flags that say OK, just forget about all these very issues, and most probably it will be right, and in most code it will work.
You might find that problem, too -- you change operation order to get some parallelism and suddenly you are running unstability.
There are some algorithms that you can't do that, but most algorithms you can.
So here's another interesting thing.
So, I have a program like that, 2 to the power I, and of course, most of the time exponentiation is very expensive.
If you have a smart compiler -- I don't have to exponentiate.
This thing called strength reduction.
They say wait a minute -- I will keep variable t. This 2 to the power i means basically every time I multiply it by 2 and I can't keep repeating that.
Do you see why these two are equal there?
This is good.
A lot of good compilers do that.
But now what did I suddenly do?
[OBSCURED.]
Yeah, I reduced the amount of computation, obviously, but I just introduce a loop-carried true dependence here.
Because now I have t dependent on the previous t to calculate the next value, and while order-wise or sequential-wise this is a win, now suddenly I can't parallelize.
Of course, a lot of times what you had to do is you have a very smart programmer.
They say aha, I know this operation is expensive so I am going to do this myself and create you a much simpler program in sequentially.
Then you try to parallelizes this and you can't.
So what you might try to do is kind of do this direction transformation many times to make the program run a little bit slower sequentially just so you can actually go and parallelize it.
So this get's a little bit counterintuitive.
You just look at a program and say yeah there is a loop carried dependence, I can do it a little bit more expensive without the loop carried dependence, and then suddenly my loop is parallelized.
So there might be cases where you might have to do it by hand, and a lot of compilers automatic parallelizing compilers, try to do this also.
Kind of look at these kind of things and try to move in that direction.
Whereas, most of the sequential compiler is trying to find this and move this direction.
So, OK I said that.
So, another thing called array privatization.
So scalars, I show you where when you have anti and output dependence on a variable, you need to privatize.
And in arrays, you have a lot more complexity.
I'm not going to go into that, you can actually do private copies also in there.
You can do bunch of transformation.
Another thing people do is called interprocedural parallelization.
So the thing is you have a nice loop and you start analyzing loop and in the middle of a loop you have a function call.
Suddenly what are you going to do with it?
You have no idea what the function does, and most of the simple analysis says OK, I can't parallelize anything that has a function call.
That's not a good parallelizing compiler because a lot of loops have function calls and you might call it something simple as sine function or some simple exponentiation function and then suddenly it's not parallelizable.
This is a big problem.
There are two things you can do.
One is interprocedural analysis and another inlining.
So the interprocedural analysis says I'm going to analyze the entire program and I have function, I'm going to go and try to analyze the function itself also.
What happens is -- so assume if the functions are used many, many times, so fine function might be used hundreds of time.
So every time you have a call of a sine function, if you keep analyzing, reanalyzing what's happening inside of the sine function, you kind of have exponential blow up.
So if you code size n, you might have an exponential time of a number of lines that need to be analyzed because every call need to go there, call some other functions, you can see the blow up.
And so analysis might be expensive.
Other option is you analyze each function once.
Yeah, OK.
I analyze this function once and every time I use that function I just use that analysis information.
What that means is you have a kind of summary of what that function does for every call.
This is not that easy and this runs into a thing called unrealizable part problem, because you go into function in one part -- assume you call foo from here and return here.
You call it here and return and here.
So when you analyze, normally you can go from here to here, here to here, but if you treat foo as only one thing you might be able to even think that you can go here to here and here to here.
So this looks like one thing in here.
You see that control here goes here, comes here do a function call goes here, because we are not treating this as separate instance.
So why did are we analyzing it once?
This cleared all this additional mess and then can have problems in here.
So these are the kind of researchy things people are working on.
There's no perfect answer, these are complicated problems, so you had to do some interesting balance in here.
Because other thing is every analyst has to deal with that, so you had to kind of an anti-compiler, which is not simple.
Inlining is much more easy.
It's a poor man solution, so every time you have function call, you just bring the function and just copy it in there.
And every time you have function call you bring the function and you can run it through the same compiler, but of course, you can have huge code blow up.
It's not only analysis expense, you might have a function that before had only 100 lines, now we have millions of lines in there and then try and do cache problems, all those other issues.
So can be very expensive too.
So what people do is things like selective inlining and a lot of kind of interesting combinations of these.
Finally, loop transformations.
So i have this loop, so I have Aij equals Aij minus 1, A i minus 1 j So look at my -- my arrowheads look too big there, but look at my dependences.
Is any of this parallel?
[OBSCURED.]
Yeah.
So, assays neither -- you can't parallelize I because there's a loop carry dependence in I dimension.
You can't parallelize J because there's loop carry dependence in J diimension.
She has idea because you can actually pipeline.
So pipelining, we haven't figured out how to parallelize pipeline.
So the way you can do this simply is a thing called loop skewing.
You can kind of -- because iteration space has changed from a data space.
You can come up with a new iteration space that kind of skew the loop in there.
So what it does is normally iteration space, what this J outside, so you go execute like this.
The skill that -- loop basically say I am executing this way, so I'm executing the pipeline, basically pipeline here.
So I'm kind of going like this way, executing that way.
If I could run that loop in that fashion, what I can do is I can run this -- after this iteration, when you go run the next iteration, there's no dependence across here.
If I run here, I don't have dependence, so I can run each of these and I have a parallel set of iterations to run.
So in here, what happens is this inner loop it can be parallel, basically like your pipeline, but it's written in a way that I still have my two loops in here, but I have done this weird transformation.
Another interesting is granularity of parallelism.
Assume I have a loop like that, i and j.
Which loop is that in here?
i or j?
j is parallel.
OK, I do something like that.
I say I run i, every iteration I do a barrier, I run j parallel and I end up doing a barrier again.
What might be a problem in something like this?
I mean inner parallelism can be expensive, because every time I had to do this probably expensive barrier, run a few iterations, a few in this one, probably only like a few cycles.
And write this very expensive barrier again, and everybody communicates -- all of those things.
Most of the time when you do inner loop parallelism it actually slows down the program.
You will probably find it too sometimes, if you define the parallelism inner array to be too small, it actually has a negative impact, because all the communication you need to do, synchronization you need to do all of them out of the program.
So inner loop is expensive.
What are your choices?
Don't parallelize.
Pretty good choice for a lot of cases.
You look at this and this is actually going to win you basically by doing that.
Or can you transform it to outer loop parallelism.
Take inner loop parallelism and you change it to get outer loop parallelism.
This program is actually nice, there are some complex analysis you need to do to make sure that's legal.
So you can basically take this one and transform in other direction.
What that means is kind of do a loop interchange.
So now instead of i, you have a a j outer dimension, i inner dimension, inner loop.
When you do that what you have is your barrier, and then you can run this is parallel and this like this.
Suddenly, instead of having n barriers for that loop, you have only one barrier.
Suddenly you have a much larger chunk you're running, and this can be run.
OK, so this is great.
So I talked to all about all this nice transformation, stuff like that.
So at some point when you know something is parallel you might want to go and generate parallel form.
So the problem is, depending on how you partition, the loop bound has to be changed, and I'm going to talk to you about how to get loop bound.
So let's look at this program.
So I have something in here and there's an inner loop that actually reads, outer loop writes.
Inner loop reads.
And it's a triangular thing.
It's a big mess.
Now I assume I want to run the i loop parallel.
So what that means is I want to run the first process -- there is no for this one, this one on one iteration, two iteration, three, four, whatever, each one's in here.
How do I actually go about generating code that actually does that?
Each processor runs its right number of iteration.
This is a non-trivial thing because triangularly you get something different and you can assume all this complexity.
One thing I did is my iteration space between i and j, this is my iteration space.
So I assume, assume I am running a processor.
Each I iteration run by your processor, you can say you have then another dimension P, and say i equals P. So I can look at now instead of a two dimensional space in a three dimensional space.
So in this analysis, if you can think multi-dimensionally it's actually very helpful because we can kind of keep adding dimensions in here.
So what are the loop bounds in here?
What we can do is use another technique called Fourier-Motzkin Elimination to calculate loop bounds by using projections of the iteration space.
I will go through later a bit to give you a flavor for what it is.
It's also, if you are in to linear programming, this is kind of extension techniques on that.
So the way we look at that is -- A little bit too far.
I didn't realize MAC can be this slow.
[ASIDE CONVERSATION] See this is why we need parallelism if you think this running fast.
So what you can do is you can think about this as this three dimensional space.
i, j and p. And because i is equal to p, if you get i and p, get a line in that dimension and then j goes there.
So this is the kind of iteration space in here, and that represents inequalities here.
So what I want is a loop where outer dimension is p, then the next dimension is i and j.
We can think about it like that.
So what that means is I need to get my iteration ordering -- when it happens, you just go like that.
All right, about doing that.
So this is the kind of loop I want to generate -- let me go and show you how we generate that.
So here's my space in here, so first one I want to do is my inner most dimension is j.
And what I can do is I can look at this thing and say what are the bounds of j.
So, for each of the bounds of j can be described by -- with p and i. I'll actually show you how to do that in little while.
Then I will get j goes from 1 to i minus 1.
Then after that I can basically project it into to eliminate j dimension.
So what I'm doing is I'm going to have a three dimension and I project into two dimensions without j anymore, because now all I have left is i p and I get a line in that dimension.
Then what I have to do is now I had to find i.
What are my bounds of i?
And bounds of i is actually i is equal to p. You can figure that one out because there's a line in there.
Then you eliminate i and now you get this one.
Then what are bounds of p?
p goes from basically 2 to n. You just basically get that.
So you can do this projection in here -- let me go in there, and now what you end up doing is you can get this, and of course, outer loop p is not a true -- like a loop.
You can say you get p, my_pid.
p is with this range.
i equals p. Do this one.
So this one, -- generated that piece of code.
So I will go a little bit detail and show how this happens, pretty much can happen.
So I have my little bit of different space.
I'm doing a different projection.
I'm doing i, j, p. I want to predict first i of a, j of a, and p of a instead of j, i, p before I do anything.
So here's my iteration space, what do I do?
The first thing I do is I find the bounds of i, So I have this thing.
I just basically expanded this, and eliminated the j this one doesn't contribute to the bounds of i, but everybody else.
So there are a bunch of things that i has to be less than that and i have to be greater than these two.
Then what I have is bound of i is, it has to be maximum of this because it has to be greater than all three.
So it has to be max of this, this, and this.
It has to be less than these two, it has to be mean of this one.
Question?
Well why did you have to go through all this.
At least in this case, the outer loop was very simple, you could have just directly mapped that
I agree with you, it's very simple thing, but the problem is that's because you are smart and you can think a little bit ahead in there, and if I'm programming a computer, I can't say find these special cases.
So I want to come up with a mathematical way that is a bullet proof way that will work from the simplest one to very complicated, like for example, finding the loop bounds for that loop transpose that I showed you before -- no, the skew that what we called before
So it's not so much just defining an index to iterate on, it's to find the best index to map, to parellize
Any could be issue, because you have -- for example, if the inner dimension depends on i, and i goes outside, then I can't make it depend on i.
So if I have something like for i equals something, for j equals i to something.
Now if I switch these two I have 4j.
I can't say it's i to something.
I have to get rid of i and I have to figure out in the for i, this has to be something with j, with some function with j in here.
So what is this function, how do you get that?
You need this kind of transformations do that.
Next time I'll talk to you about can you do it a little bit even better.
So I get this bound in here.
Then actually you found this is going from p to p. So I can actually set p because, mean and max in here.
Then after you do that, what you have to do is eliminate I.
The way you eliminate I is you take this has to be always less than n and less than p. So you take this n constraints here and you get a n times m constraints tier in here.
So the first three has to be less than n, again, we repeat it again, has to be less than p. Then, of course, the missing constraint that 1 is less than j.
You put all those constraints together.
Now, nice think is in that one, it's still legal, it still represents that space, but you don't have i there anymore.
You can completely get rid of i.
So, by doing that -- and then of course, there's a lot of redundancy in here, and then you can do some analysis and eliminate redundancy and you end up in this set of constraints.
That's where when you say what's the best, you can be best -- it has to be correct or that means you can't have additional iterations or less iterations.
But best depends on how complicated is the loop bound calculation.
You can come up with a correct solution, and the best is depending on which order you do that.
When you have two redundant thing, which one you eliminate, so you can have a lot of heuristics saying OK, look if this one looks harder to calculate, eliminate that one with the other one.
So you get this set of constraints.
Then you have to do is now find the bounds of j.
So you have this set again.
To find a bound of j only two constraints are there, and you know j goes to 1 to p minus 1, and you find the bound of j.
Getting rid of j means there's only two.
One get rid of p minus 1.
There are two left for p. You put it there, and then you can eliminate the redundance in here, and now you can find the bounds of p which goes from 2 to n. And suddenly you have the loop nest.
So now I actually di parallelization and a loop transpose in here.
I could combine those two, use this simple mathematical way and find loop bounds in here.
So, I'm going to give you something even a little bit interesting beyond that, which is communication code generation.
So if you are dealing with a cache coherent shared memory machine, you are done.
You generate code for parallel loop nest, you can go home because everything else will be done automatically.
But as we all know in something like Cell, if you have a no cache coherent shared memory or distributed memory, you have to do this one first.
Then you write identify communication and then you generate communication code.
This have additional burden in here.
So until now in data dependence analysis, what we looked at was location-centric dependences.
Which location is written by processor one is used by processor two.
That's kind of a location-centric kind of view.
How about if multiple writes the same location?
We show that in example, if multiple people write the same location, which one should I use?
That's not clear.
What you are using in the last last guy who wrote that location before I read that thing, and that's not in these data flow analysis.
No data dependence analysis doesn't get it.
What you want is something of a value-centric.
Who was the last write before my iteration, who wrote that location?
If I know the last write, he's the one I should be getting the value from.
If the last write happened in the same processor, I am set because I wrote the local copy and I don't need to deal with anything.
If the last write happened in a different processor, you need to get that value from the guys who wrote it and say, OK, you wrote that value, give it to me.
If nobody wrote it and I'm reading it, that means the value came from the original array because nobody had written it in my iteration.
Then I'm reading something that has come from the previous iteration.
So I have to get it from the original array.
But I have these three different conditions.
So you know to represent that.
I'm not going to go into detail on into detail on this representation called Last Write Trees.
So what it says is in this kind of a loop nest in here, you have some read access and write accesses in here, and if you look at it location-centrically you get this entire complex graph, because this is the graph that should have been in that example we gave.
So these arrays going in here.
I'm switching notation.
before i was going the other way around.
j was in here.
But if you go look at value-centric, this is what happens.
So you say all these guys basically got the value from outside.
Nobody wrote it.
This got from -- this is the write, this is the last write, this is the last write -- I actually have my last write information.
So where to look at that is there are some part of iteration got value from somewhere, other part go somewhere else.
You can't kind of do a big summary, as you point out that kind of dependence depend on where the iterations are.
So you can represent it using a tree when it shows up.
So you can say if j greater than 1, here's the relationship between reads and writes.
Otherwise relationship means it came from outside.
So I can say for each different places.
So you can think about this tree can be a lot more complicated tree.
So each part of the iteration space, I got data from somewhere else.
So, you get this function here.
I think I'll go to the next slide.
So what you can do is now I have processor who read, processor who write, and iterations that I can reading and writing.
One thing I can do is I can represent i using a huge multi-dimensional space.
So what happens in here is the receive iterations, those are the iterations that actually data has to be received in communication.
Assume that the part I'm actually communicating is also within the loop bound, so I can write that.
And the last write relation is that i send has to be i receive.
We know that.
What you have is the parallel with the processors -- this is i iterations are parallel, so processor, receive processor, is running iteration i, process i.
Send iterations are the same because you want to parallelize that loop basically.
In each iteration get assigned to each process.
Of course, you want to make sure the process communication is non-local.
If it's local I don't have loop communication.
I can represent this as this gigantic system of equalities.
It has one, two, three, four, five, and there's a j receiver also in here, because you've got to remember I think the program I wrote, the original program basically, write happen in outer loop and the read happen inner loop.
So there's only j receive, the i send in here.
I'll show that later.
So I have five dimensions.
So I can't really draw five dimensions, but can I wait until it comes back?
So what I have here is I have this set of complete system of inequalities for receive and in communication.
Of course, since I can't draw five dimensions, and these dimensions are the same, I just wrote it in the same.
So you can actually assume that there's another two dimensions for this one, and that's a line in that dimension.
Actually, this is wrong.
Sorry.
This should be xi here written.
My program is wrong, sorry.
Now what do I do?
One more time it has to go.
It makes me slow down my lectures which is probably a good thing.
There we go.
So what you can do is you can just scan these by predicting different ways to calculate the send loop nest and receive loop nest.
So if you scan in that direction, what you end up is something saying for this processor you need to send, for this iteration, this processor.
For what you need to send will be received by these processors and this iteration and this, and this you can send xi to this iteration at this processor.
Because you had that relationship, you can get the loop nest that actually will do the send.
The send there you can actually get a loop nest do receive and it shows up.
So what that means is, so all these guys have to send all these iterations have to do the receive.
So, if you predicted a different ordering, what you end up is you can say now for this processor has to receive.
All these processors had to receive something send by these guys.
So now you can get that entire loop nest for receiving and entire loop nest for sending, and you have computation loop nest also.
The problem is you can't run them sequentially because you're run in some into the order.
So what you have is something that next slide will show.
So you have this iteration, there's some computation happen from all the one, and I will get a loop nest do some send, I need loop nest do some receive, in a one dimensional, these kind of, you get three seperate things.
But of course, what you had to do is you had to generate code.
So the way to do that is --.
So what you have to do is kind of break this apart into pieces where things happen, so this one you do computation, this one you do computation and receive, and computation send receive and whatever.
Should be probably send here and receive but -- For that one, if you combine this you get a complicated mess like this.
But this all can be done very in an automated fashion by using this Fourier-Motzkin Elimination and this linear representation.
Of course, you can do a lot of interesting things on top of that.
You can eliminate redundant communication, if you're keeping sending the same thing again that have a send unit, eliminate that, you can aggregate communication.
You want to send a word at a time, you can send bunch of things into one packet.
You can do multitask.
So same thing, send to multiple people.
Doesn't have that much in Cell, but assume some machines have multitask support, you can do that, and also you can do some local memory management because if you have distributed memory, you don't have to allocate everybody's memory and only use a part.
You can say OK, look everybody only had to allocate that part.
OK.
In summary, I think automatic parallelism of loops and arrays -- we talked about data dependence analysis, and we talked about iteration and data spaces, a how to do that, and how the formulate assay integer programming problem.
We can look at lot of optimization that can increase parallelism and then do that.
Also, we can deal with tings like communication code generation and generating loop nest by doing this Fourier-Motzkin Elimination.
So what I want to show out of this talk is that, in fact, this parallelization -- automatic parallelization of normal loop can be done by mapping into some nice mathematical framework, and basically manipulating in that map.
So there are many other things that really complicates the life take out of parallelizing programs.
So like C, there are pointers, you have to deal with that.
So this problem is not this simple, but what compiler writers try to do most of the time is trying to find this kind of thing.
Find interesting mathematical models and do a mapping in there and then operating that model and hopefully you can get the analysis needed and even the transformation needed using that kind of a nice model.
So I just kind of gave you a good feel for general parallelizing compilers.
We will take a ten-minute break and talk about streaming.
We'll see if I can make this computer run faster in the meantime.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Let's get started.
So what I'm going to do next is switch gears to one interesting compiler, which is the StreamIt parallelizing compiler.
The main idea about StreamIt is the need for a common machine language.
What we want to do normally is, in a language, you want to represent common architecture properties so you get good performance.
You don't do it at a very high level of abstraction, so you base a lot of cycles dealing with the abstraction.
But you want to abstract out the differences between machines to get the portability, otherwise you are going to just do assembly-hacking for one machine.
Also you can't have things too complex, because a typical programmer cannot deal with very complex things that we ask them to do.
C and Fortran was a really nice common assembly language for imperative languages running on the unicore machines.
The problem is this type of language is not a good common language for multicores, because it doesn't deal with, first of all, multiple cores.
And as you keep changing the number of cores -- for example, automatic parallelizing compilers are not good to basically get really good parallelism out of that, even though we talk about that.
Still a lot of work has to be done in there.
So what's the correct abstraction if you have multicore machines?
The current offering, what you guys are doing, is things like OpenMP, MPI type stuff.
You are hand-hacking the parallelism.
Well, the issue with that.
It's basically this explicit parallel construct.
It's kind of added to languages like C -- that's what you're working on.
And what this does is all these nice properties about composability, malleability, debuggability, portability -- all those things were kind of out of the window.
And this is why this parallelizing is hard, because all these things makes life very difficult for the programmer.
And it's a huge additional program burden.
The programmer has to introduce parallelism, correctness, optimization -- it's all left to the programmer.
So what the program has to do in this kind of world -- what you are doing right now -- you have to feed all the granularity decisions.
If things are too small you might get too much communication.
If things are too large you might not get good load balancing, and stuff like.
And then you deal with all the load balancing decisions.
All those decisions are left for you guys.
You need to figure out what's local, what's not.
And if you make a wrong decision it can cost you.
All the synchronization, and all the pain and suffering that comes from making a wrong decision.
Things like race conditions, deadlocks, and stuff like that.
And this, while MIT students can hack it, you can't go and convince a dull programmer to convert from writing nice simple Java application code to dealing with all these complexities.
So this is what kind of led to our research for the last five years to do StreamIt.
What you want to do is move a bunch of these decisions to the compiler.
Granularity, load balancing, locality, and synchronization -- [OBSCURED] And in today's talk I am going to talk to you about, after you write a StreamIt program -- as Bill pointed out, the nice parallel properties -- how do you actually go about getting this kind of parallelism.
So in StreamIt , in summary, it basically has regular and repeating computation in these filters.
This is called a synchronous data flow model, because we know at compile time exactly how the data moves, how much each produces and consumes.
And this has natural parallelism, and it exposes exactly what's going to happen to the compiler.
And the compiler can do a lot of powerful transformations, as yesterday I pointed out.
The first thing is, because of synchronous data flow, we know at compile time exactly who needs to do what when.
And that really helps transform.
It's not like everything happens run-time dynamically.
So what does that mean?
So what that means is each filter knows exactly how much to push and pop -- that's in a repeatable execution.
And so what we can do is, we can come up to the static schedule that can be repeated multiple times.
So let me tell you a little bit about what a static schedule means.
So assume this filter pushes two, this filter pops three but pushes one, that filter pops two.
So these are kind of rate pushes, it's not everybody producing-consuming at once.
So what's the schedule?
So you can say -- OK, at the beginning it produces two items, but I can't consume that because I need three items.
And then I do two of them, and I can consume the first three and then produce one there.
And I have two left behind.
I do one more that, and now I got three.
And it consumes that and produces that.
And then I can fire C. So the neat thing about this is, when I started there was nothing inside any of these buffers.
And if I ran A, A, B, A, B, C, there's nothing inside the buffers again.
So what I have is, I'm back to the starting positioning.
And if I repeat this millions of times -- I keep the computation running nicely without any buffers accumulating or anything like that.
So I can come up with this very nice schedule that says -- here's what I have to do.
I have to run A actually three times, B twice, and C once, in this order, and if I do that I have a computation that keeps running.
And that gives me a good global view on what can I parallelize, what can I load balance, all those things.
Because things don't change.
One more additional thing about StreamIt is we can look at more elements than I am consuming.
Question?
How common is it in your typical code that you can actually produce a static schedule like that?
In a lot of DSV code this is very common.
A lot of DSV code right now that goes into hardware and software, they have very common properties.
But even things that are not common, what has a very large chunk of the program has this static property, and there are some few places that has dynamic property.
So it's like, when you write a normal program you don't write a branch instruction after every instruction.
You have a few hundred instructions and a branch, a few tens of instructions and a branch type thing.
So what you can think about it is that those instructions without a branch can get optimized the hell out of them, and then you do a branch dynamically.
So you can think about it like this.
What's the largest chunks you can find that you don't have this uncertainty until run-time?
Then you can optimize the hell out of it, and then you can deal with this run-time issues -- basically branches, or control for changes, or direct your rate changes at run-time.
If we have 10-90 rule, if you get 90% of the things are in a nice thing, and if you get good performance on that -- hey, it has a big impact.
So in our language you can deal with dynamism, but our analysis is basically trying to find the largest static chunk and analyze.
So most of the time that basically said we start with empty and end with empty.
But the trouble is, a lot of times we actually can look beyond the number of what we consume.
So what you have to do is kind of do initial schedule that you don't start with empty, you basically consume something -- you start with something like this.
So the next time in something comes -- three things come into this one -- I can actually pick four and pop three.
So you go through the first thing kind of priming everything with the amount of data needed, and then you go to the static schedule.
This kind of gives you a feel for what you'll get.
This is a neat thing, I know exactly what's going on in here.
So now how do I run this parallelism?
This is something actually Rodric pointed out before, there are three types of parallelism we can deal with.
So here's my stream program in here, and I do some filters, scatter-gather in here.
The first site of parallelism is task parallelism.
What that means is the programmer said, there are three things that can run parallelly before I join them together.
So this is a programmer-specified parallelism.
And you have a nice data parallel messenger presentation.
The second part is data parallelism.
What that means is, you have some of these things that don't depend on the previous run of that one.
So there's no invocation, dependency across multiple invocations.
These are called stateless filters, there's no state that keeps changing.
If the state kept changing, you had to wait till the previous one finishes to run the next one.
So if you have a stateless filter -- assume that it's data parallel -- what you can do is you can basically take that, replicate it many, many times, and when the data comes -- parallel is in it every data -- and it will compute and parallelly get out here.
The final thing is pipeline parallelism.
So you can feed this one into this one, this one into this one, and then douse across in a pipeline fashion.
And you can get multiple things execution.
So we have these three types of parallelism in here, and the interesting thing is if you have stateful filters, you can't run this data parallel.
Actually the only parallelism you can get is pipeline parallelism.
So traditionally task parallelism is fork/join parallelism, that you guys are doing right now.
Data parallelism is loop parallelism.
And pipeline parallelism mainly was done in hardware.
If you have done something like Verilog or VHDL you'll do a lot of pipeline parallelism.
So kind of combining these three ideas from different communities all into one, because I think programs can have each part in there.
So now, how do you go and exploit this?
How do you go take advantage of that?
So I'll talk a little bit of baseline techniques, and then talk about what StreamIt compiler does today.
So assume I have a program like this.
The hardest thing is there are two tasks in here.
The programs are given, you don't have to worry anything about that.
And what you can do is assign them into different cores and run it.
Neat.
You can think what a fork /join parallelism is, you come here you fork, you do this thing, and you join in here.
So the interesting thing is if you have two cores.
You probably got a 2x speedup in this one.
This is really neat because there are two things in here.
The problem is, how about if you have a lot more different number of cores?
Or if the next generation has double the number of cores, and I'm stuck with the program you've written for the current generation?
So this not that great, interesting.
So we ran it on the Raw processor we have -- it has 16 cores in there -- that we have been building, and this is actually running a simulator of that.
What you find is, is a bunch of StreamIt programs we have we kind of get performance like basically close to two, because that's the kind of parallelism people have written.
In fact, some programs even slowed down in there, because what happens in here is the parallelism and synchronization is not matched with the target -- because it's matched with the program.
Because you wrote a program because your parallelism in there was what you thought was right for the algorithm.
We didn't want you to give any consideration to the machine you are running, and it didn't match the machine, basically, if you just got the parallelism.
And you just don't do that right.
So one thing we have noticed for a lot of streaming programs, to answer your question, is there are a lot of data parallelism.
In fact, in this filter -- in this program -- what you can do is you can find data parallel filters, and parallelize them.
So you can take each filter, run it on every core for awile.
Get the data back.
Go to the next filter, write on every go-while, get that back.
So what you can do is, if you have four cores in here, you can each replicate all this four times.
Run these four for a while, and then these four, these four, these four, these four.
OK?
So that's the nice way to do that.
So the nice thing about doing that is you have a lot of nice in the load balancing, because each are doing the same amount of work for a while.
And after it accumulates enough data you go to the next one, do for a while, and then like that.
And each group basically will occupy the entire machine -- you just go down this group like that.
And so we ran it, it started even slower.
Why?
It should have a lot more parallelism, because all those filters were data-parallel.
So you sort of gettting stuck with two, now we can easily run a parallelism of 16, because data parallelism you can just put it any amount in there.
But we are running slow
Communication overhead?
Yeah, it could mainly be communication overhead, because what happens is you run this for a small amount of time.
You had to send it all over the place, collect it back again, send it all over the place, collect it back again.
The problem is there's too much synchronization and communication.
Because every person at the end is like this global barrier, and the data has to go shuffling around.
And that doesn't help.
So the other part, what you can do in the baseline is what you call hardware pipeline.
What that means is you can actually do pipeline parallelism.
The way you can do that is you can look at the amount of work each filters contain, and you can combine them together in a way that the number of filters is going to be just about the number of tiles available.
Most programs have more filters than the number of cores.
So you review combined filters, to give us a number of filters, is just either the same, or one or two less than the number of cores available.
In a way that you combine them so each of them will probably have close to the same amount of work.
The problem is if when you combine it's very hard to get the same amount of work.
And if you assume eight cores, you can do this combination and we can say -- aha, if I do this combination, I have one, two, three, four, five, six, seven.
Eight cores, I can get seven of them.
Hopefully each of them have the same amount of work, and I can run that.
And then we assign this to one filter and say -- "You own this one, you run it forever.
You get the data from the guy who owns this one, and you produce at this one."
And if you have more cores you can actually keep doing some of that.
If you have enough filters you can each combine them and do that.
So we perform, and we got this.
Not that bad.
So what might be the problems here?
Hardware locality.
You want to make sure that the communicating filters are close to each other
Yeah, that we can deal with.
It's not a big locality [OBSCURED] What's the other problem?
The bigger problem
[NOISE] load balance
Load balance is the biggest problem, because the problem is you are combining different types of things together, and you are hoping that each chunk you get combined togeher will have an almost identical amount of work.
And that's very hard to achieve most of the time, because dynamically things keep changing.
The nice thing about loops is, most of the time if you have a loop or state if you replicate it many times, it's the same amount of code, same amount of work.
It nicely balances out.
Hardware -- combining different things becomes actually much harder.
So again, parallelism and synchronization are not really matched to the target.
So the StreamIt compiler right now does two, three things.
I'll go through details.
Coarsen the granularity of things.
So what happens is if you have small filters it combines them together to get the large stateless areas.
It data parallelizes when possible.
And it does software pipelining, that's a pipeline parallelism.
I'll go through all these things in detail.
And you can get about 11x's speedup by doing all those things.
So coarsen the stream graph.
So you look at this stream graph and say -- wait a minute, I have a bunch of data-parallel parts.
And before what I did was I take each data-parallel part, when 16 then came or get together, went 16 came together, went 16.
Why?
I have put too much communication.
Can I combine data-parallel things into one gigantic unit when possible?
Of course, you don't want to combine a data-parallel part with a non-data-parallel part.
Then the entire thing becomes sequential, and that's not helpful.
So in here what we found is these four cannot be combined, because if you combime them the entire thing becomes sequential.
So what we have to do is, you can combine this way.
So all those things are data-parallel, all those things are data-parallel.
And even though they are data-parallel if you combine them they become non-data-parallel, because this is actually doing peeking, it's looking at more than one, and so it's looking at somebody else's iteration work.
So you can't combine them.
So what the benefits of doing this is you reduce global communication basically.
And the next thing is you want data parallelizing to four cores.
And this one fits four ways in there.
But the interesting thing is, when you go in this one you realize there's some task parallelism.
We know there are two tasks that have the same amount of work in here.
So facing this four ways, and facing this four ways, and giving the entire machine to this one, and giving the entire machine to this one, might not be the best idea.
What you want to do is you want to face it two ways.
And then basically give the entire machine to all of these running at the same time, because they're load balanced -- because they are the same thing repeated.
And you can do the same thing in here.
OK.
So that's what the compiler does automatically, and it preserves task parallelism.
So if you are task parallelism you don't need -- the thing about that is the parallelism you need, you don't need too much parallelism.
You need enough parallelism to make the machine happy.
If you have too much parallelism you end up in other problems, like synchronization.
So this gives enough parallelism to keep the entire machine happy, but not too much.
And by doing that actually we get pretty good performance.
There are a few cases where this hardware parallelism wins out, these two, but most of them -- actually this last one we can recover -- do it pretty well.
OK.
So what's left here is -- so this is good parallelism and low synchronization.
But there's one thing, when you are doing data parallelism there are places where there are filters that cannot be parallelized -- they are stateful filters.
Because you can't run the data parallelism, and according to Amdahl's Law that's actually going to basically kill you, because that's just waiting there and you can't do too much.
I'm going to show that using this separate program -- so this number is the amount of work that each of them has to do.
So this is actually a lot of work, a lot of work -- this does a little work in each of these filters.
So if you look at that, these are data parallel but it doesn't do any much work.
Just parallelizing this doesn't help you.
And these are data parallel.
And these actually do enough work.
Actually we can go and say I am replicating this four times, and I'm OK.
I'm getting actually good performance in here.
Now what we have is a program like this.
And so if you are not doing anything else that we have data parallelism in.
So what happens in the first cycle you run these two.
And then you run data parallel this one, and then you run these, and then you run data parallel this one.
And if you look at that, what happens is we have a bunch of holes in here.
Because at that point when you are running that part of the program there's not enough parallelism, and you only have two things in there.
And when you're running this you can run this task parallelism in here, but there's nothing else you can do in here.
And so you get basically 21 time steps each -- time minutes basically will run into that program.
But here we can do better.
What we can do is we can take and try to move that there, and kind of compress them.
But the interesting thing is these things are not data parallel.
So how do I do that?
So the way to do that is taking advantage of pipeline parallelism.
So what you can do is you can take this filter in here.
Since each of the entire graph can run only sequentially -- this has to run after this -- you can look at the filters running separately like that, and kind of say, instead of running this and this and this, why don't I run this iterations of this one.
This iterations of this invocation.
And this interations of this one.
And this iterations on the next one.
And I'm still maintaining -- because when I'm running this even though the -- I'm not running anything data parallel here because these ones were already done previously, so I can actually use that value.
And so I can maintain that dependency, but I'm running things from the different iterations.
And so what I need to do is, I need to kind of do a prologue to kind of set everything up in there.
And then I can do that and I don't have any kind of dependence among these things.
So now what I can do is I can basically take thes two and basically lay out anything anywhere in those groups, because they are in different iterations and since I am pipelining these I don't have any dependence in there.
So I end up in this kind of a thing, and basically much compress in here.
And by doing that what you actually get is a really nice performance.
The only place that this actually wins -- hardware pipelining, and this little bit in there.
But the rest you get a really good win in here.
OK.
So what this does is basically now we got a program that when the programmer never thought anything about what the hardware is -- just wrote abstract graph and data streaming.
And given Raw, we automatically actually mapped into it, and figured out what is the right balance, right communication, right synchronization, and got really good performance.
And you're getting something like 11x performance.
If you do hard hand, if you work hard probably you can do a little bit better.
But this is good, because you don't hand-do anything.
The killer thing is now I can probably take this set of programs -- which we are actually working on -- is you can take them to Cell which has, depending on the day, six cores, seven cores, eight cores, and we can basically get to matching the number of cores in there.
So this is it because right now what happens is you have to basically hand code all those things, and this can automate all that process.
So that's the idea, is can you do this -- which we haven't really proved and this is our research -- write once, use anywhere.
So write this program once in this abstract way.
You have to really don't think about full parallelism.
You have to think about some amount of parallelism, how this can be put into a stream graph, but you are not dealing with synchronization, load balancing, performance.
You don't have to deal with that.
And then the compiler will automatically do all these things behind you, and get really good performance.
And the reason I showed this was -- I'll just play one more slide I think -- showed this was it's not a simple thing.
The compiler actually has to do a bunch of work, the work that you used to do before.
Things like figuring out what's the right granularity, what's the right mix of operations, what type of transformations you need to do to get there.
But at some point we did three things -- coarse-grained, data parallel, and software pipelining.
And by doing these three we can actually get a really good performance in most of the programs we have.
So what we are hoping is basically this kind of techniques can in fact help programmers to get multicore performance without really going and dealing in the grunge level of details you guys do.
You guys will appreciate that, and hopefully will think of making -- because now at the end of this class, you will know all the pain and suffering the programmers go through to get there.
And the interesting thing would be to in fact look at the ways to basically reduce that pain and suffering.
So that's what I have today.
So this was, as I promised, a short lecture -- the second one.
Any questions?
So if we've got enough data parallelism we'll have the same software pipeline jumping on each tile?
Is that right?
Yes
OK.
So if you do that how does it scale up to something that has higher communication costs than Raw?
By doing this software pipelining you have to do all of your communication off tile
So the interest in there right now is we haven't done any kind of hardware pipelining.
We are kind of doing -- everybody's getting a lot of data moving in there.
The neat thing about right now is, even with the SP in Cell and even Raw, the number of tiles are still small enough that a lot of communication -- unless way too much communication -- it doesn't really overwhelm you.
Because everybody's nearby, you can send things.
They talk a little bit about in Cell that near enableness helps, but not that much.
But as we go into larger and larger cores, it's going to become an issue.
Near enables become much easier to communicate, and you can't do global things in there.
And at that point you will actually have to do some hardware pipelining.
You can't just assume that at some point everybody's going to get some data and go to something.
So what you need to do is have different chunks that the only communication that would be between these chunks would be kind of a pipeline communication.
So you don't mix data around.
So as we go into larger and larger cores you need to start doing techniques like that.
The interesting thing here is even though what you had to change was the compiler -- hopefully the program stays the same -- right now it's not an easy issue, because our compiler has 10 times more core than the program, so it's easier in the program.
But if you look at something C, the core base is millions of times larger than the size of the compiler.
So at some point they'll be switched.
It's easier to change the compiler to kind of keep up to date.
That's what happened in C. Every generation you change the compiler, you don't ask programmers where to code the application.
So can you make these kind of things as the multicores become different -- bigger, have different features.
You change the compiler to get the performance, but have the same code base.
That's the goal for portability
Have you tried applying StreamIt or the streaming model in general, to codes that are not not very clearly stream-based but using the streaming model to make communication explicit, such as scientific codes.
Or, for example, the kinds of parallelizable loops that you covered in the first half of the lecture
Some of those things, when you have free form simple communication can map into streaming.
So for example, one thing we are doing is things like right now MPEG.
Some part of the MPEG is nicely StreamIt, but when you actually go inside the MPEG and dealing with the frame, it's basically a big array, and you're doing that.
So how do you chunkify the arrays, and basically deal with it in a streaming order?
There's some interesting things you can do.
There will be some stuff that doesn't fit that.
Things like pattern recognition type stuff, where what you want to do is you want to -- assume you're trying to -- good example.
You're trying to feature a condition in a video.
And what happens is the number of features, can you match or connect two features, or match and connect a thousand features.
And then each feature you need to do some processing.
And that is a very dynamic thing.
And that doesn't really fit into streaming order right now.
And so the interesting thing is, the problem we have been doing is we are trying to fit everything into one.
So right now the object-oriented model is it basically -- everything has to fit in there.
But what you're finding is there are many things that don't really fit nicely.
And you'll do these very crazy looking things just to get every program to fit into the object-oriented model.
That doesn't really work.
I think the right way to work is, is there might be multiple models.
There's a streaming model, there's some kind of a threaded model, there might be different ones -- I don't know what other models are.
So the key thing is your program might have a large chunky model, another chunky model.
Don't try to come up with -- right now what we have is we have a kitchen sink type of language.
It tries to support everything at the same time.
And that doesn't really work because then you have to think about and say -- OK done, can I have a pointer here?
And I need to think about all the possible models kind of colliding in the same space
On the other hand, the object-oriented model is much more generalized to me.
It's not the best model for many things, but it's much more generalizable than some models.
And having a single model cuts down on the number of semantic barriers you have to cross --
I don't know but --
Semantic barriers incur both programmer overhead and run-time overhead
See the problem with right now with all the semantic barriers, is object-oriented model plus a huge number of libraries.
If you want to do OpenGL, it's object-oriented but you have no library.
If you want to do something else, you have to learn the library.
What the right thing would be, instead of trying to learn the libraries is learn kind of a subset language.
So you have nice semantics, you have nice syntax in there, you have nice error checking, nice optimization within that syntax.
Because the trouble is right now everything is in this just gigantic language, and you can't do anything.
And in the program you don't even know, because you can mix and match in really bad ways.
The mix and match gives you a lot of power, but it can actually really hurt.
And a lot of people don't need it.
Like for example in C, people doubt it was really crucial for you to access any part of memory anywhere you want.
You just can go and just access any program, anywhere, anytime in there.
If you look at it, nobody takes advantage of that.
How many times do you write the program an say -- "Hey, I want to go access the other guy's stack from this part."
That doesn't work.
You have a variable and you use a variable
It still [OBSCURED]
Yeah, but the thing is because of that power, it creates a lot of problems for a compiler -- because it needs to prove that you're not doing that, which is hard.
And also, if you make a mistake the program is like -- "Yeah, this looks like right.
It still matches my semantics and syntax, I'll let you do that."
But what you realize is that's not something people do -- just stick with your variable.
And if you don't go to variables -- that's what type-safe languages do -- it's probably more for bugs than a feature.
And the same kind of thing having efficiency in language, is you can do everything at the same time.
Why can't you have a language that you can go with this kind of context.
I'm in the streaming context now.
I say this is my streaming context.
I am in a threaded context.
Then what that does is, I have to learn the full set of features, but I restrict what I'm using here.
That can probably realistically improve your program building, because you don't have to worry about --
It gives the programmer time to get to know each language
But right now you have to do that.
If you look at C# it has all these features.
It has streaming features, it has threaded features, it has every possible object-oriented feature
Right, but there's a compact central model which covers most things.
You can pull in additional features and fit them [OBSCURED].
You can do pointer manipulation in C#, but you bracket things into an unsafe block.
And then the compiler knows in there you're doing really bad things
That's a nice thing, because you can have unsafe.
But can you have something like -- this is my streaming part.
OK. Can I do something like that, so I don't have to worry about other?
The key thing is, is there a way where -- because right now, my feeling is if you look at the object-oriented part.
So if you are doing, for example, Windows programming, you can spend about a week and learn all the object-oriented concepts.
And you have to spend probably a year to learn all the libraries on top of that.
That's the old action these days.
It's basically the building blocks have become too low, and then everything else is kind of an unorganized mess on top of that.
Can you put more abstraction things that easy?
Hey, I'm talking about research, this is one possible angle.
I mean there might -- you can think, I know there are messes that I think in there.
My feeling is what we do well is when things get too complicated we build abstraction layers.
And the interesting thing there is, we build this high level programming language abstraction layer.
And then now we have built so much crud on top of that without any nice abstraction layers, I think it's probably time to think through what there could be at the abstraction level.
Things like, it's hitting -- that is where parallelism is really hitting.
Because that layer, the object-oriented layer, doesn't really support that well.
And it's all kind of ad hoc on top of that.
And so that says something.
Yes, it's usable.
We had this argument -- assembly languages programmers -- for two decades.
There are people who were swearing by assembly languages.
They could write it two times smaller, two times faster than anything you can write in high level language.
It's probably still true today.
But at the end there were things that high level languages won out.
I think we are probably in another layer like that.
I don't know, probably will go with that argument.
You can always point to something saying this is something you cannot do.
If there are still things -- like structured programs and unstructured programs, we talked about that.
That argument went for a decade
The question I would pose is can you formulate a kitchen sink language at a parallelizable level of abstraction?
Ah.
That's interesting, because parallelization is -- one of the biggest things people have to figure out is composability.
You can't have two parallel regions as a black box put together.
You start running into deadlocks and all those other issues in there.
Most of the things that you work is the abstraction works, because then you can compose at a higher level abstraction.
You can have interface and say -- here are something interface, I don't know what's underneath, I compose at the interface level.
And then the next guy composes at the higher level, and everything is hidden.
We don't know how to do that in parallelism right now.
We need to combine two things, it runs into problems.
And the minute you figure that one out -- if somebody can figure out what's the right abstraction that is composable, parallel abstraction -- I think that will solve a huge amount of problems
Isn't it Fortress that attempted to do something that's parallelizable and the kitchen sink, but that then leaves all the parallelizable --
I'm saying how terrible [OBSCURED] programmers
But I would say right now is a very exciting time, because there's a big problem and nobody knows the solution.
And I think for industry they lose a lot of sleep over that, but for academia it's the best time.
Because we don't care, we don't have to make money out of these things, we don't have to get production out of it.
But these actually have a very open problem that a lot of people care about.
And I think this is fun partly because of that.
I think this huge open problem that if you talk to people like Intels and Microsoft, a lot of people worry a lot about they don't know how to deal with the future in 5-10 years time.
They don't see this is scaling what they're doing.
And from Intel's point of view, they made money by making Moore's Law available for people to use.
They know how to make it available, but they don't know how to make people use it.
From Microsoft's point of view, their current development methodology is almost at this breaking point.
And if you look at the last time this happening -- so things like Windows 3.0, where their current development methodology doesn't really scale, and they really revamped it.
They came up with all this process, and that had lasted until now.
For the last two Office and Vista, just realized they can't really scale that up.
So they are already in trouble, because they can't write the next big goal is just two times bigger than Vista, and hopefully get it working.
But on top of that, they have it thrown this multicore thing, and that really puts huge amount of burden.
So they are worried about that.
So from both their point of view, everybody's clamoring for a solution.
And things like last time around -- I'll talk about this in the future -- last time around when that happened, it created a huge amount of opportunities, and bunch of people who sold it kind of became famous.
Becuase they say -- we came up with a solution, and that people started using and stuff like that.
Right now, everybody's kind of waiting for somebody to come up and say here's the solution, here's a solution.
And there are a lot of -- Fortress type exports is one, and what we are doing is one.
And hopefully some of you will end up doing something interesting that might solve it.
This is why it's fun.
I think we haven't had this much of an interesting time in programming languages, parallelism, architecture in the last two decades.
With that, I'll stop my talk.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we have Arvind today going to talk about how actually do parallelism much closer to hardware and all the work we have been doing
So first, just a clarification.
This talk is not really about any techniques.
This is some very primary ideas that I have some new ideas on parallel programming.
The second thing is 5, 6 years ago I was going around the country giving a talk, why hardware design can't be left to hardware designers.
Because I think the way they expressed their ideas is very, very low level from software point of view.
So I have worked pretty hard on injecting some fairly sophisticated ideas from software into hardware design.
But it often happens when you deal with powerful people, you get effected by it.
So I have sort of come the full cirlce.
Now I'm going to show you how hardware design can influence software design and that's really the idea behind this talk.
So that's borrowing some ideas from hardware design.
Cell processor is a very good example of a system on a chip and this is what everybody believes the future will look like.
That you'll have fairly symmetric things in some sense and that is needed by hardware because hardware gets out of hand if you have wires running all over the place.
So people would like these things to be fairly regular in some sense, but that doesn't mean that all these styles have to be the same.
So for example, you may have very application specific blocks on your chip.
So for example, in your cell phone you have many, many blocks which are specialized, especially all the radio communication.
Because if you did that stuff in software you may not be able to meet the performance and even if you can meet the performance your battery will drain instantaneously.
I mean, it just takes too much power to do that kind of computation in software.
So for various reasons we need application specific processing units and I'm sure cell processor has quite a few of these.
By the way, these application specific units, they're often called ASICs.
And you can think of them as a separate chip, but in due course of time they become blocks of a bigger thing.
You may have general purpose processors.
Already, systems have more than one such thing.
Cell processor has how many?
Like 8
Depends.
Some have 8, but they don't [OBSCURED] So the current Playstations already list supposedly 7, but we've had only 6.
So, the [OBSCURED]
And high-end cell phones have 2 general purpose processors and 2 DSPs on it, so you certainly have -- multiple processors you will have memory banks, et cetera.
And you will have structured on-chip interconnection networks
So I think this is not controllers.
That this is what the trajectory will look like in the future.
That doesn't mean that all the chips will look the same.
You may have different mixes of things depending upon the application and one of the interesting things is can you put together one of these things fast enough because it's very, very expensive to design and implement these things.
So really one issue that comes up all the time is can we rapidly produce high quality chips and surrounding systems and software for such thing?
So this talk, as I said, it's about ideas and what I'm going to do is I'm first going to tell you how I used to think about parallel programming.
And this way is all about threads.
You know, where are my threads?
And I worked on this problem for a very, very long time.
Like 20 years and this is not necessarily wrong, You know I'm sure in this course itself you have seen quite a bit about this and I think tomorrow's lecture is on Cilk so you will see even more about threads.
And lately, I've been thinking about it differently.
And really the main point I would like to get across somehow to you is now I think of parallel little programming module as a resource.
What do you mean as a resource?
I'll try to make it more concrete, this idea.
And the whole issue here is I think this viewpoint is valuable, but it's not proven yet.
You know, so it may not be right.
So just a warning.
I've been working with this on students, in particular.
Now those of you who have heard of transactional programming, lots of things I will say you'll say oh, you mean a transaction?
Yes, I mean there are very, very strong connections for transactional stuff, but that's not the language I'm going to be using here.
These slides are color-coded.
So when I was talking about old world thinking it's yellow and when it's new thinking then it's white.
So well, I think you probably have already heard this.
That the only reason for parallel programming used to be performance.
This made programming extremely difficult.
You had to know a lot about the machine.
You were programming in code were not portable.
Endless performance tuning, all this still goes on.
I mean, very few of these things have changed.
Parallel libraries were not composable.
And this is one of the main things I worry about a lot and I think Allen was alluding to it too.
Somebody has code, he just wants to run the damn thing.
You know, even at factor of 2 speed that would be what fantastic.
You know, often it runs slower.
Very, very fun, machines.
Very different to deal with heap structures and memory hierarchies.
And then there is always the synchronization cost issue.
In some sense parallel programming doesn't make sense if there is no synchronization going on and that's like this embarrsingly parallel computation you just start off and things never talk to each other.
In even moderately interesting parallel programs there's always some communication between various tasks and when that happens synchronization cost becomes an issue and it is such a major issue right now that you have to think about it out front.
Oh, if I synchronize too much then I know for a fact that my program's going to run like a dog.
So you try to avoid synchronization and make things coarser and so on.
Really the whole issue used to be how to exploit hundreds of threads from software because new hardware will support many, many threads.
But at the same time, it's important to understand that in this world there's always virtualization.
So number of threads in your machine is never going to match exactly what you have in your application.
So any programming model you have, virtualization is a very important aspect of it.
That I have n threads and n may be dynamic, maybe varying with time in my application, but in hardware I have some fixed number of threads.
Which is significantly smaller in general than what you have in your application.
So the thing I was most interested in all those days was what I call implicit parallelism.
And from the questions I was hearing about analysis you're doing of your programs, all those issues come up there.
You know, given some code can I figure out what can be done in parallel and just run it like that?
So expect parallelism from programs written in sequential languages and people like some of the champions in this and I have also spent lots and lots of time on this research problem.
So this is one of those areas which certainly has not suffered because of lack of research.
There's tons and tons of research in this.
You know, you can go to IBM and there will be 50 Ph.D.'s working on Fortran compiler.
Trying to extract parallelism.
But the success is limited.
In fact, the main takeaway from this research is people have learned now how they should write their programs so that compiler has a chance of extracting parallelism.
Because now people look at your code, they say, well, this is a bad code.
Why?
Because compiler can't analyze it.
You become part of the equation in this.
Now the methodology I was pushing for a long time was functional languages.
Programs in functional languages which may not obscure parallelism in the first place.
I'm not going to talk about functional languages.
It's immaterial, you know, whether this is a good idea or a bad idea, the fact of matter is in real life people's reaction is functional languages-- are you kidding me?
So it doesn't get off ground.
It's one of those things I can come and preach to you is good for your soul.
And they leave me alone.
I don't do movies with subtitles
Sorry to temporarily derail you--
Sorry?
Sorry to temporarily derail you, so what kind of parallelism is easier to extract from a functional language?
I can clearly see some task parallelism, can you also extract data parallelism or pipeline parallelism or what not?
It turns out that's actually not that right question to ask in the functional language because functional language doesn't obscure the parallelism that was there in the first place.
So you already have a partial order on operations as opposed to a sequential order.
You see when you write in Fortran even a simple thing like f of g of x, h of y.
It may be obvious to a functional programmer that these things can be done in parallel.
Perhaps, the execution of these can overlap with the evaluation of f, but that's not the semantics of Fortran or any other sequential language.
The semantics is you're supposed to execute the program left to right, top to bottom.
So you will go and execute g first and you'll go inside g and execute it and then you will go and execute h and then you will go and execute f. Why is that important?
Because there are side effects, so it's conceivable that you do something in this which affects this and so on.
So you're given a sequential order and then compiler does deep analysis, the kind you're doing right now it terms of dependence and anti-dependence and so on.
Which says it's OK to do g and h in parallel.
Well, if it was a functional program then you know by the semantics of the language g and h can be done in parallel.
That doesn't mean it's a good idea to do it in parallel.
So when we get to the second point of mapping this computation onto a given substrate off 2 processors or 10 processors then problems become similar.
But the problem of detecting parallelism is very different.
Because you don't talk of detecting parallelism in functional languages, it's already there.
You don't obscure it.
Does that make sense?
Yeah, maybe
And it goes without saying that if your algorithm has low parallelism then functional languages or any other language and there's no magic-- you can't suck blood out of stone.
You have to know your algorithm.
You have to know what operations can be done in parallel.
So language only is an aid.
In some sense that if it was parallel I could express it more easily as a parallel program.
And if it's a bad language, you obscure it and then you try to undo the damage.
Now as I said, there has been a lot of success in this in terms of dense matrix kind of stuff, but more irregular the computation harder it gets to do these things statically.
So when you can't do it then people of course, designed explicitly parallel programming models and some of the most successful ones in this are data parallel and you'll hear about multithreading tomorrow in Cilk.
And then there are very low level models where you are doing actual message passing between various modules that are sitting over there and I can expose you threads and synchronizations that are very low-level fork and joint, et cetera.
So of course, high-level models are preferable because you're likely to make fewer errors in them.
The question is, can we compile them?
And in some cases what turns out is this is a good model, but it is not general enough.
So if you have a data parallel algorithm, fine.
I mean, you should use these things.
But not all applications fit the data parallel model, multithreading in some sense is more general.
This is what I mean by a multithreaded model that you may have a main which spawn off things.
So not only these things can be done in parallel, but execution of these can overlap the execution of f. So parents and children may be executing simultaneously.
So if you look at this as an activation tree, this invokes a loop which has many, many iterations in it.
When we talk of sequential computing you're always sitting on one branch of this.
So that's the stack.
This stack, this stack, stack frames.
When you talk of parallel computing then part of this activation tree is concurrently active.
So all problems get harder.
Storage management gets harder.
And what gets especially harder is that there is this global heap.
You always have heap in interesting programs, global data structures.
Now the problem is if this and this are active at the same time or this and this are active at the same time and they are both reading and writing into the data structure then naturally there's a race condition in this.
Even if only 1 guy's writing and 5 people are reading it.
It's a question of the guy should not read it too soon.
So the moment you do any kind of parallel programming, synchronization issues arise immediately.
You know, you always have to worry about, how do I indicate that it's OK to read this?
Well, you can follow some control ideas.
You say, this guy was writing.
He has finished execution, so it must be OK to write.
Or you can go to very fine-grained synchronization, you can have some bit here which says, oh this data has been updated, now it's OK to unlock it and read it and so on.
So all these ideas have been explored and they're still being explored in this context.
The main takeaway is instead of a stack you will have a tree which is active at the same time.
Many, many things are going on.
And the second thing is that there is a competition.
There is a race in reading the heap data structures and therefore you have to worry about that you don't read it too soon or you don't write it too soon because if you write it too soon you may clobber before somebody else gets a chance to read it.
So these issues are quite serious.
Now it's really possible-- I mean, I claim you know, I have languages which can express computation very efficiently this way.
Efficiently is not the right word.
Very easily you can express computation in this way.
But at the end of the day the goal is to take that stuff and run it somewhere on some parallel machine.
Whether it was cell processor in the old days if would have been parallel machines of various ergs.
And that has proven to be quite difficult, efficient mapping of these things because it turns out that you say, oh, you didn't tell me before you have only 2 processors.
Then I would have expressed the computation differently.
Or you didn't tell me that you have 1000 processors.
So there is this kind of yes, there is virtualization.
But it's not virtualization enough.
Qualitatively matters a lot.
You know, what is the target?
And that effects how you would have expressed the computation.
So I used to go around saying this all the time that parallel programming is so important.
That really sequential programming will be taught as a special case of parallel programming because it will be all parallel.
If you have only one processor then we can teach you some special tricks how to run it efficiently on one processor.
So this as you can see, is largely an unrealized dream
That's an old idea, but not one to be dismissed
Not one to be dismissed.
Absolutely.
So the question is, has the situation changed?
And the situation has certainly changed.
So multicores have arrived.
This is a big deal.
I mean, Microsoft wants to exploit parallelism.
There can be no bigger indication than this.
And there is explosion of cell phones.
And if you don't understand what that means economically, you know there are 100 million PCs that will be sold this year and 950 million cell phones that'll be sold this year.
This is the same kind of transition that is taking place that happened in the early 80s when we went from mainframes and mini- computers to micros.
So what happens there determines what happens everywhere else.
So it's quite possible then what happens on these small devices, hand-held devices will determine the architecture of everything else just because of the numbers.
People are willing to invest a lot more money into this explosion of game boxes
[OBSCURED]
I'm sorry?
The explosion of laptops is also happening
Yeah, but still that number is 100 million.
While cell phone is 950 million, at least in 2000
These are cell processors
Right.
Absolutely.
So look, if I'm talking to you as a teacher I mean, my message to my colleagues in the department is it's no longer good enough to say well we'll teach you parallelism when you're a sophomore.
I mean, it is the thing that people want to deal with.
You say, what do you mean telling me how to program one processor.
I have 10 of them in my pocket.
So I think we have to deal with this.
You know, how do we introduce parallelism?
What method we should have for teaching parallel programming so that is the default programming model in your head?
So it's all about parallelism now.
No longer an advanced topic.
It has to be the first topic.
Just another word of caution in this.
So let's look at cell phones.
So mine has some bugs in it so sometimes it misses a call when I'm surfing the web.
It doesn't rate.
So this is the deep question now, so whose fault is it?
So for example, to what extent the phone call software should be aware of web surfing software or vice versa.
So now you have to understand how these things are done.
Of course, how the phone calls are made, that software has been evolving, right?
You can look at it all in the last 15 years and you'll be able to trace a continuum.
You know how phone calls are made and that's not trivial among the software on your cell phone.
Well, when it comes to web surfing software I can guarantee you it was not written from scratch.
People went to your PC, they say, what do you do on your web?
They take all that software say, let's port it on the cell phone.
Now the guy who wrote the web software never thought that you would be talking on telephone while you're surfing the web or at least not on the same device.
And vice versa, the guy who wrote the phone software never thought you would be surfing the web while you're talking on the phone.
I mean even 1995 or 1998 if somebody had said, you'll be surfing the web at the same time.
It would have sounded pretty bizarre.
Is it merely a scheduling issue?
So this is the answer saying ah, they didn't process the interrupt properly.
I'm sorry, this is not just a scheduling issue.
I don't even know the semantics of this.
So for example, if you're surfing the web and the phone rings do you really want to stop surfing the web and pick up the phone?
I mean, what's so special about phone?
There is nothing being indicated from the language point of view.
It may be application requirement that you want to stop it or not stop it and so on.
Is it a performance issue?
That oh, it's because you have only 2 processors in this.
If you had 4 this problem will go away.
Or if you could just run it faster it will go away.
I don't think any of these answers are right.
I mean, the bottom line in this is sequential modules are often used in concurrent environments with unforeseen consequences.
So your mindset has to be that I'm going to put together a parallel program with lots of existing things.
Because you can't keep giving the answer oh, let's just rewrite it again from scratch.
Because amount of software is so enormous in this.
What we should worry about now is how we should design these modules.
How we should write pieces of software or design hardware so that we can put together an ensemble of them, a collection of such modules so that they do something useful.
And it's all about parallelism.
And how to do all this in a parallel setting.
OK, so new goals as opposed to old goals.
So I don't want to think in terms of decomposition.
I don't want to think in terms of here is my clever algorithm, how do I decompose it so that this runs here and this runs here and so on.
Instead I want to think in terms of synthesis.
That you've already given me lots of modules, which are very useful.
Which perhaps are written by domain experts.
They know something about that stuff.
And can I put together a bigger application very quickly from it?
And you know, another favorite example of mine in this regard is FFTW and linear algebra packages.
Both have been optimized to the hilt, right?
And if somebody gives you an application which calls FFT and uses enough linear algebra I guarantee you that program will not be easy to write in spite of the existence of both these packages, which are extremely well optimized.
So there is something we're not paying attention to is, how do we bring parallel modules together in such a way so that the functionality, the performance, et cetera are unpredictable in some sense?
I'm sorry.
What's your point?
I think I agree with your point, I just want to make sure I understand your point.
That FFTW linear algorithm is not enough and so there will be--
No, no.
I think it's very well done, FFTW.
Linear algebra is very well done.
But we still haven't provided any good way of thinking in terms of two parallel things.
You know, what happens when they interact?
So if my program is calling FFTW often and then it's calling linear algebra often
You weren't here when I mentioned that but I was actually bringing up the point of linear algebra and FFTW working together.
Just working
Exactly.
Absolutely.
I'm saying in that sense, you see, this is a difference way of thinking because the old way of thinking well, give me your application.
Now let's decompose it.
The point is I don't have experts who can write FFTW as well as Matel wrote it.
Or who's linear algebra packages are being written.
I want to use those things.
I want to synthesize large parallel programs quickly.
Yes?
How does the Apple iPhone work with--
I'm sorry, how does?
The iPhone
Apple iPhone
Apple iPhone?
Yes, they have the operating system on there--
I think from our point of view that's the same thing.
Is just that the user interface is guaranteed to be far superior because in my phone when I see the icons I'm a techie and I still can't understand half of them.
So I like Apples idea.
You know, phone, pictures.
[OBSCURED]
The way Apple might be with that is going to not let anybody program it.
It's going to have a very restrictive development program internally so they can basically say, I'm going to take this person from this person and put it together.
They will try to do everything in one unified way.
So to some extent it might work, but then you suddenly realize Apple hired programmers to keep up with all the needs.
So it may even start to have other people running things
Internally their software is--
So is there something parallel going on in there?
Guaranteed there will be parallel stuff going on there.
Guaranteed because all the communication stuff has to go on in parallel with the computation stuff.
So I mean, phones have to lots of things in parallel if for no other reason just because of power.
It takes less power when you do things in parallel as opposed to when you do sequentially because then you have to run much faster in that.
OK, so a method of designing and connecting modules has that functionality and performance are predictable.
And must facilitate natural description of concurrent systems.
A method of refining individual modules into hardware or software for systems on a chip.
So refinement is a highly technical word.
What that means is that you have written a program and you move on to rewrite it, but nobody can tell from outside that you rewrote it.
All right, so you refine it and you can to refine it into hardware.
I mean, you may implement it in hardware in this as opposed to transformation.
Transformation is automatic.
We're doing something, it's guaranteed it's correct.
Refinement you may have to work a little bit more to show that it is correct in some sense.
It's a correct refinement of whatever you were doing.
So basically, just to get the application going, which as Allen is pointing out is the major task and this is true regardless of what people say.
When you get to complex system, if people start talking about performance you're 90% there.
That means the damn thing works.
So even the people may say, oh, performance will be lousy.
Most of the energy is just consumed in getting it to work.
So never forget that.
So a refinement idea becomes very important because you want to take as many things as possible, which are available.
Make it work and then individually refine; modular refinement of things
Maybe just also underscore, just to emphasis what you're saying Arvind, our salesmen will go and say explicitly that they can sell a factor of 2.
Just like you were saying.
Just a factor of 2 on 100 processors or whatever, they can sell that if it works.
If it gives the right answer
It's the ease
The ease and confidence that it's really doing what you expect it to do
So it's really performance second, ease of use first.
We've got it backwards in academia
But everybody set designs on 2 multicores.
So why multicore problems becomes hard and this where hardware and software starts diverging a little bit.
So what happens in hardware design?
Everything is in parallel.
It's sitting right there, this blog does this, this blog does that.
So there's no confusion in your mind what's going on in parallel.
If you have to multiplex some thing in hardware, that first we'll do this then we'll do that, that's also expressed explicitly in the design.
Software is not like that.
You know, software have n threads, but I have only 10 in my hardware.
So there is another layer of software or operating system or whatever you want to call it, runtime system.
Whose sole job is to virtualize or to do time multiplexing of underlying resources and that makes the problem harder.
You know, that makes the problem harder whether you're doing it in a way that preserves some performance goals.
Whether you're doing it so that you want cause deadlocks, so there are many, many issues that come up in this.
And basically, when I said this is an ideas kind of a talk, I do know how to do this and I still haven't worked enough on this problem of how to take this kind of methodology where I can write these parallel modules, compose them.
How to map it onto multicores.
OK, so now let me tell you something technical about this stuff.
So this hardware inspired methodology for synthesizing parallel programs.
Now I'm going to use some fancy words.
So this is a rule-based specifications or what I call guarded atomic actions.
I will explain in a second what that means.
It will let you think about parallel systems in a very different way.
So it's like saying look, if this register is 2 and this register is 3, you can add them and put them here, 5.
And you can do this any time you want.
So this is like a rule.
Invariant.
You're saying, I don't care what's happening in the machine.
This is always safe to do.
Now what I'm going to ask you to do is take a huge intellectual jump from this.
That if you give me enough such rules you would have completely described the hardware, what it does.
Now why am I thinking like this?
Because these invariants are stated properly.
It's always possible to understand the behavior of the system as if one things at a time is happening.
When 2 plus 3 is becoming 5, if you want you can have this mental picture-- the rest of the world is frozen.
Nothing is changing there, only that change is taking place.
And you can think it terms of these small, small changes and all legal behaviors will be explainable as some sequence of these changes.
It's possible to describe all kinds of hardware using things like that.
Composition of modules with guarded interfaces.
So this is the language called Bluespec and it has a very strong echoes of a language that was designed in the 80s by two guys, Chandry and Misra, the language called Unity.
But they never used it for hardware design.
Their goals were different and they didn't really have a concept of a module in that language.
So let me show you 3 examples here.
Greatest common divisor, just to get off ground and then I will show you 2 very different problems: airlines reservation query problem and the video codec H.264.
We won't have time to get into ordered list, but if somebody wants to discuss that I've be happy to show you.
OK, so now let's look at how do I think of my design of my program in Bluespec?
I always think in terms of a bunch of modules.
And what's a module.
A module is going to have some state elements in it.
If you're thinking software just think each module guards some variables, which nobody else can touch.
You're the only one.
So you have a variable xyz, he has some abcd, et cetera.
So each module has some.
That's what I'm showing in red.
If you were thinking hardware, think of thse things as stateful elements like registers or flip flops of memories.
But the main point is if it is here then it's not here.
You own something, this module owns something, this module owns something.
One.
Second thing is every module has internal rules for manipulating the state and the rules are always going to be of the form that if some condition is true on this you're allowed to make the following changes to that.
So that's what the rule looks like.
Some condition, if it holds that by action I mean, change the state.
Change the state of those variables.
And it's all or nothing.
So if you have 2 variables and if you want to change both of them in a rule then the execution of a rule means either both of them will change or neither will change.
There's nothing in between you can see.
That's disallowed by this model.
I mean, the reason I have modules is they're not totally disjoined.
You can actually access the state.
Both read it and write it, but you can't just arbitrarily reach it and do something.
You have to go through some interface methods the moment you talk to some other modules.
So this is the standard information hiding principle.
You know, abstract data types, whatever you want to call it.
You know, so there's an interface through which you will enter and if you're just reading the values I think we call them read methods or value methods and if you're going to effect the state of one of the modules we'll call those action methods.
Because you're actually going to change the state in some module.
So now, very strange execution model here.
Very, very different from sequential execution model.
So repeatedly you do the following.
You pick a rule to execute.
Any rule, I don't care.
And select a rule to execute, compute the state, what updates should be made, make the state update.
And then go and do it again, so repeatedly you do this.
So this is a highly nondeterministic thing, selective rule.
There may be a gazillion rules in your system.
So system privileges are provided to control the selection if you want.
But it's a detail that we won't get into to.
So does everybody get the model right here?
That you have a lot of state elements and you have lot of rules.
Many rules may be applicable, pick one of them to execute.
Execute and that will change the state.
Ask the same question again, which rules are enabled?
No, pick any rule amongst them and keep doing it.
Now semantics say one rule at a time.
In any implementation we're going to do many, many rule in parallel.
But you don't have to worry about that.
That will be done all automatically.
That we can do many rules in parallel.
OK, so now let's look at GCD.
So this is an ordinary GCD program.
You can write it in any language you want if y is zero then x otherwise if this is greater than GCD of y, x.
Otherwise you subtract.
So no problem and I'm sure you know how it executes.
So if I was to take GCD of 6,15 you will see there is a recursive call that'll go on because 6 is less than 15 so it will be this.
You get this and you get this, et cetera.
Ultimately you get 3 as an answer.
Everybody is with me?
You know you all know GCD?
You can all write it in your favorite language.
Now the question is, what does it mean to execute this program in a concurrent setting?
We were not thinking parallel or sequential, right?
This is GCD.
Someone who's teaching this class says run it in parallel.
You say, what do you mean, run it in parallel?
Oh, you mean that maybe I should try to do several GCD calls, recursive calls in parallel?
That also doesn't make too much sense in this.
Perhaps, this is what someone meant.
You know that if he has some program where there are 2 calls to GCD, he would like you to do those 2 in parallel.
I mean, what does it mean to do GCD in parallel?
Now let me contrast this with how you would think about this problem as a hardware person.
So your job is to build a GCD machine.
You're going to make millions of dollars, GCD is very popular.
So you build this GCD machine.
You know, it has 2 inputs, it has an output.
And this will be my module I'm going to sell this intellectual property to everyone.
OK, so now let me talk of parallel invocations of this.
You see in hardware there can be no confusion, either you have 2 GCD boxes or you have 1 GCD box.
I mean, you can have as many boxes as you want, but there is no confusion about that.
You know how many GCDs you have.
So in some sense, if you have many of them, you're talking of independent calls.
Who knows what about recursive calls?
I mean, that's internal story.
You know, that's a different level of questions.
So this question will automatically get asked.
In hardware you will ask the question, does the answer come out immediately?
Does it come out in particular time?
Why's this question important?
Because if it's going to take some time then while it's computing the question I'm going to ask is can I give another set of inputs?
Are you with me?
I mean, that's a legitimate question to ask.
If it's going to take half an hour to compute the GCD can I give it another input while it's thinking?
Can the machine be shared by 2 different users?
Can it be pipelined?
So you agree that all these questions are meaningful in hardware setting?
And I claim all these questions are also meaningful in software setting.
We just don't think like that.
This is exactly the problem we are having right now when we say that we have optimized something and we call it again.
We think off it as starting a fresh, maybe not.
Maybe not.
You know, maybe we should think of it in terms of resources.
And the point I want to make is I want to think of GCD module as a resource.
Even in software I want to think of it as a resource.
So that there is no ambiguity in your mind whether you have 1 GCD or you have 2 GCDs.
If you're going to multiplex it we'll write it differently.
If you're going to have 2 independent ones, which can go on in parallel we'll write it differently.
OK, so that's the idea I want to borrow from the hardware side of this.
So how would you do-- yes?
But how would you decide how many GCDs to instantiate?
Would that be is this model left up to the programmer?
Well, you see that's the big difference between hardware and software.
That in hardware you have no design until you have taken that decision
Right.
I mean, hardware designs once it's built, it's built.
Software designs you want the same module of software to seamlessly run on different kinds of hardware
That doesn't mean that I can't recompile it or can't resynthesize it or something.
So it may be same source description in software
Right, if your source subscription specifies 2 GCD modules and you have 8 cores, you're not necessarily going to take very good advantage of this, so how do you--
No, no.
The way I would like to think about that is that it'll be highly parameterized code and you will plug in some information like that and resynthesize the parallel program
So the software or some sort of macro meta software has to determine the extent to which the problem should be distributed
Well, you can think like that, but really this integration will be much tighter than you think.
So for example, let me just give you some more insight from the hardware side.
So in the hardware side it's standard practice that will have some RTL code, I'll have some code.
And I plug in the library.
What kind of gates have I got?
Have I got 2 input gates, 4 input gates, whatever.
There will be a huge library.
The same code can be compiled using different libraries.
So you choose something at synthesis time.
Another thing that'll happen in hardware is what is called generic statements.
So you may have a loop which assigns a register file with n registers.
Which will conceptually makes sense, it will make sense in simulation, but when it comes to synthesizing it you have to specify n. So I'm going towards that kind of methodology.
That you will have highly parameterized code, but many parameters you have to specify before you will say, now my program is ready to run on this parallel machine.
And this is totally unexplored.
As I said, this is an idea level them being
So this adds a problem that is not in traditional hardware synthesis, mainly picking all the n's.
Ideally you wouldn't want to have to have the programmer to come up with creative algorithms because they're going to have lots and lots of
Absolutely.
We may have lots of defaults.
We may other level of smart programs, which look at this and they instantly go and set many parameters.
You know, so there will be a practical side, many issues like that.
But main point I'm trying to make is that I really want to synthesize.
I want to instantiate a program, synthesize a program for a given configuration of hardware.
Let's look at the GCD in Bluespec.
If I'm going to find the GCD of 2 numbers I need 2 registers, x and y; so that's what this is.
Make me a register.
If you want, make me 2 variables.
You know, make me variable x and y, initial values are zero.
So this is what is called the state of the module.
It knows about and x and y, nobody else knows about x and y.
What are the dynamics of it?
How do I compute this?
So internal behavior is being described and this is exactly what you saw earlier in saying that there is a swap rule.
If x is greater than y and y is not equal to zero then you swap x and y, so this is a parallel composition.
x gets y, y gets x.
In this system all the reads take place instantaneously and then you do all the writes at the end of it.
Don't read this sequentially, you read x and y in parallel and then you go and update x and y.
Does this rule make sense?
I mean, if you know anything GCD-- if x is greater than y than you're going to swap it and subtraction, if x is less than or equal to y than y gets y minus x.
So this is very interesting.
Now you have 2 registers x and y and I've given 2 rules and I'm saying you can apply these rules anytime you want.
Just repeatedly keep doing these rules.
Now what does the outside world know about this?
What do you want to advertise about GCD to the outside world?
You don't want to give away your secret.
This is your intellectual property of how you're computing the GCD.
You just want to be use GCD.
So the outside world can say, oh find me start it.
With a and b. x gets a, y gets b.
It's going to give you 2 parameters, but we may be busy computing so we have a guard here which says, if y is zero-- this is internal, outside world doesn't see this.
If y is zero then only then can you start it, otherwise you can't start it.
Otherwise it means it's busy computing.
OK, similarly when is the result available?
When the y is zero then you have the result and you will return x
[OBSCURED]
That's right.
So that'll be more sophisticated.
And that's exactly the kind of decision I do want the designers to take because you may want to put a five hole.
You may just keep spitting out results.
You can make it as sophisticated as you want.
Tag it if you want.
No predetermined thing.
So first question I ask you, what happened to those recursive calls?
You know, if you go back to the previous slide there was a recursively called GCD, which you were all comfortable with.
There is no GCD call here
[OBSCURED]
Right, so there's a notion of a cycle event or something.
Look at it now, update it, look at it again.
You know, so there's like an infinite loop here, which is always going.
Which is exactly how hardware works.
It doesn't have to be synchronous, but if you want to have a simple model in your head just think, there's the clock.
You know, look at this state, pick one route to execute, update it, then it's the next clock cycle.
Keep doing this repeatedly.
It's details here, but I can take such a description and actually produce that machine for you, which does the GCD.
What I'm much more interested in is, this is how I want to think about my GCD now.
So I've hidden everything inside it.
And it has 2 methods: start and result and there is a very strong notion of when a method is ready-- because it may be busy computing, it doesn't want to listen to you.
So there is this notion of something being ready and if it's an action method then you're only allowed to enable it when it's ready.
So this is a high-level protocol that compiler will enforce.
It'll never ever set this, it'll never execute this unless it is ready to be executed.
And of course, when you're going to enable it then you have to give me 2 arguments that go with it.
And what does this really mean?
The result is valid.
So if are to do types et cetera, this would easily be baby type or something.
Tag union type that validates is being coded here.
So really to the world you are just advertising this interface.
There is a GCD interface and it has 2 methods.
An action method, which is called start and a result method, which is just a value kind of a thing.
And in order to do this you have to give me an int a into b in this.
An end of interface.
Now this is borrowing a lot from modern programming languages so for example, I can easily make it polymorphic.
It doesn't have to be ints here.
You know, what is the meaning of int?
You can specify the type in this.
So it'll be a's of type t, b's of type t et cetera.
What are the examples of type t here?
It could be a 32 bit integer, it could be 16 bit integer, it could be 17 bit integer, whatever you want and whatever makes sense.
So this is the kind of thing that you always take care of when you synthesize things.
It won't synthesize until you specify the type, but the description remains the same.
And you will instantiate it for a given type.
This I'm showing you for one reason.
If you are coming from the hardware side really these ideas and types and sensations are very sophisticated.
I mean, this is getting as advanced as you can have in software.
The other very interesting thing is, and this is the abstract data type kind of thinking, you can go and completely change the implementation of this.
If you have a clever algorithm for doing GCD, fine, go ahead and do it.
You know, it doesn't effect the users of this.
So this is a very important property for composition.
All I insist on is that every method has this ready thing coming on so that I don't make mistakes in wiring it.
I will involve this method only when it's ready.
So in some sense if you forget about the ready thing, I'm just borrowing ideas from object oriented languages.
You know, you have a class, you have methods on it, and all I'm saying is oh, you should manipulate this state in a very systematic manner using these methods.
But then I'm injecting some hardware idea here, which doesn't exist in software today.
That is, oh, a method may not be ready.
And even that can be captured very abstractly and done properly in this.
Yep?
So then in the case of making models I was thinking of spy nature.
So if you want to basically synthesize the hardware you may also better efficiency by having set of synchronized protocol by taking the output [OBSCURED] number of cycles
I think, yes.
That'll be a low-level detail that if it is synchronized then you will actually try to manipulate it like that
[OBSCURED]
OK, right.
So let me just quickly say what the languages is.
So you have modules, then you have state variables, and you have rules and you have action methods and read methods.
What I wanted to show you is what is the language of actions.
So when I write an action here or an action here, what is it?
So simplest action is assignment to a variable, assigment to a register.
This is a conditional action, so if predicate is true then do this action, otherwise no action, no change in state.
This is a parallel composition.
Do a1 and a2 in parallel.
Sequential composition, the effects of this are visible to this, et cetera.
This is a call to a method of one of the module.
And then there is a guard.
And you have an expression language, which is there's nothing special here.
So this is just simply you can read a variable, you can have constants.
These are just names and there's nothing special going on here.
Let me explain you guards.
So people find guards versus if's confusing.
And this is one way to understand it.
So guards affect the surroundings.
If I wrote here a1 when p1 in parallel with a2 think of guard as something to do with resources.
I really don't want you to do a1 unless p1 is true.
p1 made a fact that the module was busy.
Some predicate here.
But I want the effect of this whole thing to be atomic.
So if any part of it can't be done then the whole thing can't be done.
And therefore the affect of guard is as if you wrote a1 in parallel with a2 when p1.
In other words, guard on anything becomes guard on everything in that parallel composition.
Do you understand what I just said?
This is very important for composition of guards.
Because I want the atomicity of the whole thing to be preserved.
So if anything can't be done that means the whole thing can't be done.
On the other hand, conditional action is just conditional action.
So if I have if p1 than a1 in parallel with a2, well that's like saying if p1 was true then I want to do a1 and a2 in parallel, otherwise I just want to do a2.
So there is a very big difference between conditional actions and guards.
Guards are something about resources.
I need it for proper composition of atomic actions
I have question.
So are they entirely equal into.
Because this is maybe a low-level question, but in the first case-- so the first case in the left.
Can't you start doing a2 and then roll it back or something [OBSCURED]
Oh, I think there may be many tricks you can play in implentation.
You can be optimistic, you can-- [INTERPOSING VOICES]
No.
No.
No.
Semantically this is this defining the semantics.
I mean, this is the algebra.
This has to be true because I'm saying this is what a guard is
Right, so the semantically equal but not [OBSCURED]
Well I mean, that's probably not the right way to say it.
I mean, semantics are being defined.
You have choice in implementations.
I mean I'm not telling you how to implement this.
Let me go on with this.
So here is a problem that was posed to me by Jayadev Misra.
This is quasi realistic problem.
Ask for codes from 2 airlines.
If one code is below $300, buy immediately.
$300 is sort of the cheapest ticket you get these dates.
Buy the lower quote if over $300.
But as some-- say your patience runs out, say I can't wait anymore.
So after one minute buy from whosoever has quoted otherwise flag error.
Is this a realistic scenario?
You can express it using threads.
And you can write a scheduler to do this, et cetera.
Those things are not as succint as you would like.
I mean, you'd be surprised, in this simple a problem how complicated and how many questions will arise if you express this as a threaded computation.
Now let me show you what you will do-- in Bluuspec.
OK, so we are going to make a module, which does what?
Make get quotes.
This module's job is to get quotes.
So what kind of state elements do you expect it to have?
Well airline a may be quoting some value, airline b may be quoting some value.
Whether we are done or not there's a timer, which we're going to be bumping, et cetera.
And they'll be rules you know, get a quote from a, which will be when not done do something.
Executes when a responds.
Get b, timeout, et cetera.
There are there ruls of this sort.
Let me just show you one methodd.
So method, this is how you will start it-- you have this module get quotes and you're saying book me a ticket and this is your request r. Now obviously this can only be done when the module is not busy.
If module is busy the you can't do it.
So what will you do when you get such a request?
Tell me this makes sense.
So what is this saying?
Try to get a request from a in parallel.
Try to get a request from b.
Now you are busy.
The module is busy so done equals false here.
And I'm initializing this.
I'm just saying that a quote right now is infinity and b quote is infinity and timer is zero.
Fair?
And when will I get ticket?
Well, when done then you'll return the ticket from this.
Now let's look at example of a rule.
So you may have a rule like this, pick cheapest.
You see the interesting thing in this is you'll be able to read this independently of what else is going on in this system.
And let's see, what does this rule say?
Is it succint?
Rules says when you are not done and if the quote from a is not infinite that means a has quoted.
And b has also quoted, then what should happen?
If a is less than b then buy the ticket from a.
Otherwise you'll buy the ticket from b and you're done.
I'm not going to write all these rules, but I think you should be able to-- I'm certain you can write these rules.
What's happening here that is just separation of concerns.
It's all concurrenct stuff.
Yes?
[OBSCURED]
Parallel?
Or like, for lack of better words the type in the name
Sorry, say it again
So as the ampersand, how does that interact with the--
Can you hold that question first because we will run out of space
So the ampersand after not done that is, what sort of ordering is that?
It's just, I mean, associative, commutative.
Kind of thing.
It's all happening in parallel
[OBSCURED]
This thing, right?
I mean, think of it like this, these are variables and you're checking their values right now.
So it's just a combination kind of a query that you have here
So I guess you're checking there's new activity happening
That's right
Doesn't matter which order you check
So I guess what I'm trying to get is that the whole block at the top is executed in parallel with the block at the bottom?
No.
Those things are not in parallel
[UNINTELLIGIBLE PHRASE]
Now I just want to show you this because it's completely different from airline reservations.
You know, H.264 this is a codec that is being used everywhere and we'll be used even more.
And this is a typical data flow diagram you will find for this kind of thing.
You don't have to understand anything about this except that conceptually you just have five flows for communicating between them and in all such things the goal is that it has to be able to sustain a certain amount of rate at which dat is to be processed and it's usually forgiving in terms of latencies.
So if latency increases a little bit here and there it's usually OK-- when things go in and come out.
A data flow like network.
And another reason why this example is fascinating is because it's done regularly both in hardware and software and any mixture of the two.
So for example, when it runs on your PC it's being done 100% in software and I think on iPods there are special purpose hardware for doing it and therefore the battery can last much longer.
And on cell phones it's kind of in between in this and it's a question of what frame rate you want to process?
If you're procssing very high frame rate then it'll have to be done in hardware.
So when we got into this thing we said, OK, we wanted to build hardware for doing this.
And reference codes that are available, it's 80,000 lines and it sort of gives you a heart attack.
You know, you read this and you say, whoa!
And it doesn't need the performance.
I mean, that reference code, if it ran on your laptop you won't be able to watch a movie with this.
And there is the Linux version of it, which gives you even a bigger heart attack because it's 200,000 lies and many different codecs are mixed in this.
The biggest problem here it is none of these codes reflect that picture I showed you of the previous diagram.
Because the way people approach it in software is they take some input from here and they push it as far as they can.
And that's it.
Then they will take the next input and push it.
So different boxes keep modifying it as they see fit and as a reader you can't tell, oh, this is this part and this is this part.
I mean, there are no 54Q's.
Why?
Because 54Q's are expensive in software.
You know, it will involve copying.
You know, you'll write it here then you'll read it again, which is a bad idea in software.
So in some sense, the software will thinking already entered.
In this style of coding there is no model of concurrency.
So it's not clear to me when you give me that quote what all I can do in parallel in order to preserve the semantics of this.
I mean the problem is much, much harder here.
So you will just do some high-level analysis of the code, but you have lost a lot of information that was all present in the source-- could've been present in the source.
And my claim is it can be done differently.
So this has been done by Chun-Chieh and this I'm showing you a month old slide, but he has done a lot more work since then and the total code in Bluespec is 9000 lines for this contrasting it with 80,000 versus 200,000 kind of stuff and this is telling you the amount of code in various blocks in this.
And since it's done in Bluespec you can take it and you can go and implement it all in hardware, so you can synthesize it.
It actually does now 45 frames per second, actually 90 frames even per second today.
You know, so this is a month old.
This is the main point.
Once you have done this now you can refine it.
You have a different preference.
You say, oh, I want to do this differently.
So you can go and rewrite any module again.
It will be very, very easy to do this in the system.
It is very difficult to do this in the reference code, so that's one point to be made.
The second thing is you don't have to do everything in hardware.
If you wanted you could have implemented any part of it in software, so you can take the same kind of a description and generate software from it as well.
So each module can be refined separately.
Behaviors of modules are composable.
You know, you can take these things and it's predictable what will happen when you compose these things in terms of performance and in terms of resources also, it's very clear what's happening.
In hardware the resources will be area and time and so on in this.
So takeaway.
Parallel programming should be based on well-defined modules and parallel composition of such modules.
Modules must embody a notion of resources and consequently, sharing in time multiplex use.
This is a controversial point.
And guard atomic actions and modules with guarded interfaces provide a solid foundation for doing so
Thank you.
Now we ran a little bit late, so if you have questions you can ask about programs.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I guess [OBSCURED] Let's get going.
OK, should I introduce you?
If you want.
I can introduce myself
We have Bradley Kuszmaul who's been doing articles on Cilk?
He's a very interesting paralleling and also what you can say about the program It's a very interesting project that coming for a while, and there's a lot of interesting things he's developed, and multi core becoming very important
So how many of you people have ever heard of Cilk?
Have used it?
So those of you who have used it may find this talk old or whatever.
So Cilk is a system that runs on a shared-memory multiprocessor.
So this is not like the system you've been programming for this class.
This kind of machine you have processors, which each have cache and some sort of a network and a bunch of memory and when the processors do memory operations they are all on the same address space and it's typically-- the memory system provides some sort of coherence like strong consistency or maybe released consistency.
We're interested in the case where the distance from processors to other processors into a processors to memory may be nonuniform and so it's important to use the cache well in this kind of machine because you can't just ignore the cache.
So sort of the technology that I'm going to talk about for this kind of system is called Cilk.
Cilk is a C language and it does dynamic multithreading and it has a provably good runtime system.
So I'll talk about what those all mean.
Cilk runs on shared-memory machines like Suns and SGIs and well, you probably can't find Alphaservers anymore.
It runs on SMPs like that are in everybody's laptops now.
There's been several interesting applications written in Cilk including virus shell assembly, graphics rendering, n-body simulation.
We did a bunch of chess programs because they were sort of the raison d'etre for Cilk.
One of the features about Cilk is that it automatically manages a lot of the low-level issues.
You don't have to do load balancing, you don't have to write in protocols.
You basically write programs that look a lot more like the ordinary Cilk programs instead of saying first I'm going to do this and then I'm going to set this variable and then somebody else is going to read that variable and that's a protocol and those are very difficult to get right
[OBSCURED]
Yeah, I'll mention that a little bit later.
We had award-winning chess player.
So to explain what Cilk's about I'll talk about Fibonacci.
Now Fibonacci, this is just to review in case you don't know C. You all know C right?
So Fibonacci is the function that each number is the sum of the previous two Fibonacci numbers.
And this is an implementation that basically does that computation directly.
The Fibonacci of n if n is less than 2, it's just n. So Fibonacci of zero is zero, 1 is 1.
2, the Fibonacci's-- well, then you have to do the recursion, so you compute Fibonacci of n minus 1 and Fibonacci of n minus 2 and sum them together and that's Fibonacci of n. One observation about this function is it's a really slow implementation of Fibonacci.
You all know how to do this faster?
How fast can you do Fibonacci?
You all know this, How fast is this one?
[OBSCURED]
So for those of you who don't know-- certainly know how to compute Fibonacci in linear time just by keeping track of the most recent two.
1, 1, 2, 3, 5, you just do it.
This is exponential time and there's an algorithm that does it in logarithmic time.
So this implementation is doubly, exponentially bad.
But it's good as a didactic example because it's easy to understand.
So to turn this into Cilk we just add some key words and I'll talk about what the key words are in a minute, but the key thing to understand about this is if you delete the key words you have a C program and Cilk programs have the property that one of the legal semantics for the Cilk program is the C program that you get by deleting the key words.
Now there's other possible semantics you could get because-- not for this function, this function always produces the same answer because there's no race conditions in it.
But for programs that have races you may have other semantics that the system could provide.
And so this kind of a language extension where you can sort of delete the extensions and get a correct implementation of the parallel program is called a faithful extension.
A lot of languages like OpenMP have properties that if you had these directives and if you delete them, it will change the semantics of your program and so you have to be very careful.
Now if you're careful about programming OpenMP you can make it so that it's faithful, that has this property.
But that's not always the case that it is.
Sure
Is it built on the different
C 77.
No, C 89
OK, so there's no presumption or any alias involved?
It's assumed that the [OBSCURED]
So the issue of restricted pointers, for example?
Restricted pointers
So Cilk turns out to work with C 99 as well
But is the presumption though for a pointer that it could alias?
The Cilk compiler makes no assumptions about that.
If you write a program and the back end-- Cilk works and I'll talk about this in a couple minutes.
Cilk works by transforming this into a C program that has-- when you run it on one processor it's just the original C program in effect.
And so if you have a dialect of C that has restricted pointers and a compiler that--
You're taking the assumptions that if you make a mistake--
If you make a mistake the language doens't stop you from making the mistake
Well, but in C 89 there's not a mistake.
There's no assumption about aliasing, right?
It could alias.
So if I said --
Because of the aliasing you write a program that has a race condition in it, which is erroneous--
It would be valid?
No, it'd still be valid.
It would just have a race in it and you would have a non-determinate result
It may not do what you want
It may not do what you want, but one of the legal executions of that parallel program is the original C program
So there's no extra
At the sort of level of doing analysis, Cilk doesn't do analysis.
Cilk is a compiler that compiles this language and the semantics are what they are, which is you the spawn is its-- and I'll talk about the semantics.
The spawn means you can run the function in parallel and if that doesn't give you the same answer every time it's not the compilers fault
[OBSCURED]
Pardon?
There has to be some guarantee [OBSCURED].
[OBSCURED]
How in a race condition you get some [OBSCURED]
One of the legal things the Cilk system could do is just run this, run that program.
Now if you're running it on multiple processors that's not what happens because the other thing is there's some performance guarantees we get.
So there's actually parallelism.
But on one processor in fact, that's exactly what the execution does.
So Cilk does dynamic multithreading and this is different from p threads for example where you have this very heavyweight thread that costs tens of thousands of instructions to create.
Cilk threads are really small, so in this program there's a Cilk thread that runs basically from when the fib starts to here and then-- I feel like there's a missing slide in here.
I didn't tell you about spawn.
OK, well let me tell you about spawn because what the spawn means is that this function can run in parallel.
That's very simple.
What the sync means is that all the functions that were spawned off in this function all have to finish before this function can proceed.
So in a normal execution of C, when you call a function the parent stops.
In Cilk the parent can keep running, so while that's running the parent-- this can spawn off this and then the sync happens and now the parent has to stop.
And this key word basically just says that this function can be spawned
Is the sync in that scope or the children scope?
The sync is scoped within the function.
So you could have a 4 loop that spawned off a whole bunch of stuff
You could call the function instead of moving some spawns, but then [OBSCURED] in the sync
There's an explicit sync at the end of every function.
So Cilk functions are strict
[NOISE]
You know, there's children down inside here, but this function can't return-- well, if I had omitted the sync and down in some leaf the compiler puts one in before the function returns.
There's some languages that are like this where somehow the intermediate function can go away and then you can sync directly with your grandparent
Otherwise it would stop
So this gives you this dag, so you have this part of the program that runs up to the first spawn and then part of the program that runs between the spawns and the part of the program that runs after-- well, after the last spawn to the sync and then from there to the return.
So I've got this drawing that shows this function sort of running.
So first the purple code runs at it gets to the spawn, it spawns of this guy, but now the second piece of code can start running.
He does a spawn, so these two are running in parallel.
Meanwhile.
This guy started that pff.
This is a base case, so he's going to not do anything.
Just feels like there's something missing in this slide.
Oh well.
Essentially now this guy couldn't run going back to here.
This part of the code couldn't run until after sync so this thing's sitting here waiting.
So when these guys finally return then this can run.
This guy's getting stuck here.
He runs and he runs.
These two return and the value comes up here.
And now basically the function is done.
One observation here is that there's no mention of the number of processors in this code.
You haven't specified how to schedule or how many processors.
All you've specified is this directed acyclic graph that unfolds dynamically and it's up to us to schedule those onto the processors.
So this code is processor oblivious.
It's oblivious to the number of processors
But because we're using the language we're probably have to create, write as many spawns depending on--
No, what you do is you write as many spawns as you can.
You expose all the parallelism in your code.
So you want this dag to have millions of threads in it concurrently.
And then it's up to us to schedule that efficiently.
So it's a different mindset then, I have 4 processors, let me create 4 things to do.
I have 4 processors, let me create a million things to do.
And then the Cilk scheduler guarantees to give you-- you have 4 processors, I'll give you 4 fold speed up
I guess what you'd like to avoid is the mindset of the programmer has to change or find the changing tuning the parameters for the performance
There's some tuning that you do in order to make the leaf code.
There's some overhead for doing function calls.
So it's small overhead.
It turns out the cost of the spawn is like three function calls.
If you were actually trying to make this code run faster you make the base case bigger and do something, trying to speed things up a little bit with the leaves of this call.
So there's this call tree and inside the call tree is this dag.
So it supports C's rule for pointers.
For whatever dialect you have.
If you have a pointer to a stack and then you have a pointer to the stack and then you call, you're allowed to use that pointer in C. So in Cilk you are as well.
If you have a parallel thing going on where normally in C you would call A, then B returns, then C and D. So C and D can refer to anything on A, but C can't legally refer to something on B and the same rule applies to Cilk.
So we have a data structure that implements this cactus stack is what it's called, after the sugauro cactus-- the view of the imagery there and it lets you support that rule.
There's some advanced features in Cilk that have to do with speculative execution and I'm going to skip over those today because it turns out that sort of 99% of the time you don't need this stuff.
We have some debugger support, so if you've written code that relied on some semantics that maybe you didn't like when you went to the parallel world, you'd like to find out.
This is a tool that basically takes a Cilk program and an input data set and it runs and it tells you is there any schedule that I could have chosen-- so it's that directed acyclic graph.
So there's a whole bunch of possible schedules I could have chosen.
Is there any schedule that changes the order of two concurrent memory operations where one of them is right?
So we call this the non-determinator because it finds all the determinacy races in your program.
And Cilk guarantees-- the Cilk race detector is guaranteed to find those.
There's a lot of race detectors where if the race doesn't actually occur you have two things that are logically in parallel, but if they don't actually run on different processors a lot of race detectors out there in the world won't report the race.
So you get false negatives and there's a bunch of false positives that show up.
This basically only gives you the real ones
That might be indicatiors there might be still a data to arrays
So this doesn't analyze the program.
It analyzes the execution.
So it's not trying to solve some MP complete problem or Turing complete problem.
And so this reduces the problem of finding data races to the situation that's just like when you're trying to do code release and quality control for serial programs.
You write tests.
If you don't test your program you don't know what it does and that's the same property here.
If you do find some race someday later then you can write a test for it and know that you're testing to make sure that race didn't creep back into your code.
That's what you want out of a software release strategy
[NOISE]
If you start putting in sync than maybe the race goes away because of that.
But if just put in instrumentation to try to figure out what's going, it's still there.
And the race detector sort of says, this variable in this function, this variable in this function, you look at it and say, how could that happen?
And finally you figured out and you fix it and then you put it-- if you're trying to do software release you build a regression test that will verify that has that input
What if you have a situation where the spawn graph falls into a terminal.
So it's not a radius, but monitoring spawn is there but it spawns a graph a little bit deeper
Yes.
For example, our race detector understands locks.
So part of the rule is it doesn't report a race if the two memory accesses-- if there was a lock that they both held in common.
Now you now you can write buggy programs because you can essentially do the memory at lock, you know, read the memory, unlock, lock, write the memory.
Now the interleave happens and there's a race.
So the assumption of this race detector is that if you put locks in there that you've sort of thought about.
This is finding races that you forgot about rather than races that you ostensibly thought about.
There are some races that are actually correct.
For example, in the chess programs there's this big table that remembers all the chess positions that have been seen.
And if you don't get the right answer out of the table it doesn't matter because you search it again anyway.
Not getting the right answer means you don't get any answer.
You look something up and it's not there so you search again.
If you just waited a little longer maybe somebody else would have put the value in, you could have saved a little work.
And so in that case, well it turns out to be there's no parallel way to do that.
So I'm willing to tolerate that race because that gives me performance and so you have what we call fake locks, which are basically things that look like lock calls, but they don't do anything except tell the race detector, pretend there was a lock held in common.
Yeah?
[UNINTELLIGIBLE PHRASE]
If it says there's no race it means that for every possible scheduling that--
[UNINTELLIGIBLE PHRASE]
Well, you have that dag.
And imagine running it on one processor.
There's a lot of possible orders in which to run the dag.
And the rule is well, was there a load in a store or a store in a store that switched orders in some possible schedule and that's the definition
So, in practice, sorry, one of the [INAUDIBLE] techniques is loss.
Assuming, dependent on the processor, that you have atomic rights, we want to deal with that data [UNINTELLIGIBLE] in the background --
Those protocols are really hard to get right, but yes, it's an important trick
Certainly [INAUDIBLE]
So to convince the race detector not to complain you put fake locks around it.
You've programmed a sophisticated algorithm it's up to you to get the details right.
The other property about this race detector is that it's fast.
It runs almost in liear time.
A lot of the race detectors that you find out there run in quadratic time.
So if you want to run a million instructions it has to compare every instruction to every other instruction.
Turns out we don't have to do that.
We run in time, which is n times alpha of n where alpha's the inverse Ackermann function.
Anybody remember that from the union-find algorithm.
It's got that graded So it's like the almost linear time.
We actually now have a linear timed one that has performance advantages.
So let me do a little theory in practice.
In Cilk we have some fundamental complexity measures that we worry about.
So we're interested in knowing and being able to predict the runtime of a Cilk program on P processors.
So we want to know T sub p, which is the execution time on P processors.
That's the goal.
What we've got to work with is some directed acyclic graph that is for a particular input set and if the program determines it and everything else it's a well defined graph and we can come up with some basic measures of this graph.
So T sub 1 is the work of the graph, which is the total time it would take to run that graph on one processor.
Or if you assume that these things are all cost unit times, just the number of nodes.
So for this graph what's the work?
I heard teen, but something-- 18?
And the critical path is the longest path.
And if these nodes weren't unit time you'd have to weight the things according to actually how much time they run.
So the critical path here is what?
9.
So I think those are right.
The lower bounds then that you know is that you don't expect the runtime on P processes to be faster than linear speedup.
In this model that doesn't happen.
It turns out cache does things.
It's adding more than just processors.
You're adding more cache too.
So all sorts of things or maybe it means that there's a better algorithm you should have used.
So there's some funny things that happen if you have bad algorithms and so forth.
But in this model you can't have more than linear speedup.
You also can't get things done faster than in linear time.
This model assumes basically that these costs of running these things are fixed and the cache has the property that changing the order of execution means that the actual costs of the nodes in the graph change costs.
So those are lower bounds and the things that we want to know are speedups, so that's T sub 1 over T sub p. And the parallelism of the graph is T sub 1 over T sub infinity.
So the work over the critical path and we've been calling this span sometimes lately.
Some people call that depth.
Span is easier to say than critical path, depth has too many other meanings so I kind of like span.
So what's the parallelism for this program?
18/9.
We said that T sub 1 was what?
18.
The infinity is 9.
So on average and if you had an infinite number of processors and you scheduled this as greedy as you good, it would take you 9 steps to run and you would you be doing 18 things worth of work.
So on average there's two things to do.
You know, 1 plus 1 plus 1 plus 3 plus 4 plus 4 plus 1 plus 1 plus 1 divided by 9 turns out to be 2.
So the average parallelism or just the parallelism of the program is T sub 1 over T sub infinity.
And this property is something that's not dependent on the scheduler, it's a property of the program.
Doesn't depend on how many processors you have
[OBSCURED] You're saying, you're calling that the span now?
Is that the one for us [OBSCURED]
That's too long to say.
I might as well say critical path length.
Critical path length, longest trace span is a mathematical sounding name
We just like to steal terminology
Well, yeah.
So there's a theorem due to-- Graham and Brent said that there's some schedule that can actually achieve the sum of those two lower bounds.
This linear speedup is one lower bound of the runtime and the critical path is the other.
So there's some schedule that basically achieves the sum of those and how does that theorem work?
Well, at each time step either-- suppose we had 3 processors.
Either there's at least 3 things ready to run and so what you do is you do a greedy schedule.
You grab any 3 of them.
If there's fewer than p things to run, like here we have a situation where these have all run.
The green ones are ready to go.
Those are the only 2 that are ready to go.
So what do you do then in a greedy schedule?
You run them all.
And the argument goes, well, how many times steps could you execute 3 things?
At most you could do it the work divided by the number of processors times because then after that you've used up all the work.
Well how many times could you execute less than p things?
Well, every time you execute less than p things you're reducing the length of the remaining critical path.
You can't do that more than the span times.
And so a greedy scheduler will achieve some runtime which is within the sum of these 2.
It's actually the sum of these 2 minus 1.
It turns out that there has to be at least one node that's on both work and critical path.
And so that means that you're guaranteed to be within a factor of 2 of optimal with a greedy schedule.
And it turns out that if you have a lot of parallelism compared to the number processors, so if you have a graph that has a million fold parallelism and a thousand fold processors Well, if this is really small compared to the work, if you have a graph with a million fold parallelism that means the critical path is small.
If you only had 1000 processors that means this term's big.
And that means that this term is very close to this term, so essentially the corollary to this is that you get linear speedup, perfect linear speed asymptotically if you have fewer processors then you have parallelism in your program.
So the game here at this level of understanding, I haven't told you how the scheduler actually works-- is to write a program that's got a lot of parallelism that you can get linear speedup.
Well, the work-stealing scheduler we actually use.
The problem is the greedy schedulers can be hard to compute-- especially if you imagine having a million processors in a program with a billion fold parallelism.
Finding on every clock cycle, finding something for each of the million guys to do is conceptually difficult, so instead we have a work-stealing scheduler.
I'll talk about that in a second.
It achieves bounds which are not quite as good as those.
This bound is the same.
It's the sum of two terms.
One is the linear speedup term, but instead of it being T sub infinity it's big O of T sub infinity because you actually have to do communication sometimes if the critical path length is long.
Basically, you can sort of imagine.
If you have a lot of things to do, a lot of tasks and people to do it, it's easy to do that in parallel if there's no interdependencies among the tasks.
But as soon as there's dependencies you end up having to coordinate a lot and that communication costs-- there's lots of lore about adding programmers to a task and it slowing you down.
Because basically communication gets you.
What we found empirically-- there's a theorem for this-- empirically the runtime is actually still very close to the sum of those terms.
Or maybe it's those terms plus 2 times T sub infinity or something like that.
And again, we basically get near-perfect speedup as long as the number of processors is a lot less than the parallelism.
Should be sort of a less than less than.
The compiler has the mode where you basically can insert instrumentations.
So you can run your program, it'll tell you the critical path length.
You can compute these numbers.
Clear how to compute work, you just sum up the runtime of all the threads.
To compute the critical path length, well you have to do some max's and stuff as you go through the graph.
And the average cost of a spawn these days is about 3 on like a dual core pentium.
Three times the cost of a function call.
And most of that cost actually has to do with the memory barrier that we do at the spawn because that machine doesn't have strong consistencies.
So you have to put this memory barrier in and that just empties all the pipelines.
It does better on like an SGI machine, which has strong-- well, traditional.
A MIPS machine that has strong consistency actually does better for the cost of that overhead.
Let me talk a little bit about chess.
And we had a bunch of chess programs.
I wrote one in 1994, which placed third at the International Computer Chess Championship and that was running on a big connection machine CM5.
I was one of the architects of that machine, so it was double fun.
We wrote another program that placed second in '95 and that was running on an 1800 node Paragon and that was a big computer back then.
We built another program called Cilk chess, which placed first in '96 running on a relatively smaller machine.
And then on a larger SGI origin we ran some more and then at the World Computer Chess Championship in 1999 we beat Deep Blue and lost to a PC.
And people don't realize this, but at the time that Deep Blue beat Kasparov it was not the World Computer Chess Champion, a PC was.
So what?
It's running a program.
You know, there's this head and a tape.
I don't know what it did.
So this was a program called Fritz, which is a commercially available program.
And those guys were very good, the PC guys playing were very good at getting on sort of the algorithm side.
We got advantage by brute force.
And we also had some real chess expertise on our team, but those guys were spending full time on things like pruning away sub-searches that they were convinced weren't going to pan out.
Computer chess programs spend most of their time looking at situations that any person would look at and say, ah, blacks won.
Why are you even looking at this?
And it keeps searching.
It's like, well maybe there's a way to get the queen.
So computers are pretty dumb at that.
So basically these guys put a lot more chess intelligence in and we also lost due to what-- in this particular game, we were tied for first place and we decided to do a runoff game to find out who would win and we lost due to a classic horizon effect.
So it turns out that we were searching to depth 12 in the tree and Fritz was searching to depth 11.
Even with all these heuristics and stuff they had in it, they were still not searching as deeply as we were.
But there was a move that was a good move that looked OK at depth 11 and looked bad at depth 11 and at depth 13 it looked really good again.
So they saw the move and made it for the wrong reason, we saw the move and didn't make it for the right reason, but it was wrong and the right move-- if we'd been able to search a little deeper, we would have seen that it was really the wrong thing to do.
This happens all the time in chess.
There's a little randomness in there.
This horizon effect shows up and again, it boils down to the programs are not intelligent.
A human would look at it and say, eventually that knight's going to fall.
But if the computer can't see it with a search, you know?
We plotted the speedup of star Socrates, which was the first one on this funny graph.
So this looks sort of like a typical linear speedup graph.
Sort of when you're down here with few numbers processors you get good linear speedup and eventually you stop getting linear speedup.
That's sort of in broad strokes what this graph looks like.
But the axes are kind of funny.
The axes aren't the number of processors and the speedup-- it's the number processors divided by the parallelism of the program.
And here is the speedup divided by the parallelism of the program.
And the reason we did that is the each of these data points is a different program with different work in span.
If I'm trying to run a particular problem on a bunch of different processors I can just draw that curve and see what happens as get more processors.
I'm not getting any advantage because I've got too many processors.
I've exceeded the parallelism of the program.
But if I'm running, trying to compare two different programs, how do I do that?
Well, you can do that by normalizing by the parallelism.
So down in this domain the number of processors is small compared to the average parallelism and we get good linear speedups.
And up in this the domain the number of processors is large and it starts asymptoting to the point where the speedup approaches the parallelism and that's sort of what happened.
You get some noise out here so one of the things down here, it's nice and tight.
And that's because we're in that domain where the communication costs are infrequently paid because there's lots of work to do.
You don't have to communicate very much.
Up here there's a lot of communication that happens and so the noise is showing up more in the data.
This curve here is the T sub 1 over P plus T sub infinity curve.
The T sub P equals T sub infinity curve and that's the linear speedup curve on this graph.
So I think there's an important lesson in this graph besides the data itself, which is if you're careful about choosing the axes, you can take a whole bunch of data that you couldn't see how to plot it together and you can plot it together and get something meaningful.
So in my Ph.D. thesis I had hundreds of little plots for each chess position and I didn't figure out how-- it's like they all look the same, right?
But I didn't sort of figure out that if I was careful I could actually make them be the same.
That happened after I published my thesis.
Oh, we could just overlay them.
Well, what's the normalization that makes that work?
So there's a speedup paradox that happened.
Pardon?
[OBSCURED]
Yeah, OK.
There was a speedup paradox that happened while we were developing star Socrates.
We were developing this for 512 processor connection machine that was at University of Illinois, but we only had a smaller machine on which to do our development.
We had a 128 processor machine at MIT and most days I could only get 32 processors because the machine was in heavy demand.
So we had this program and it ran on 32 processors in 65 seconds.
And one of the developers said, here's a variation on the algorithm, it changes the dag.
It's a heuristic.
It makes the program run more efficient.
Look, it runs in only 40 seconds on 32 processors.
And so is that a good idea?
It sure seemed like a good idea, but we were worried that we knew that the transformation increased the critical path length of the program, so we weren't sure it was a good idea.
So we did some calculation.
We measured the work and the speedup.
And so the work here-- these numbers have been cooked a little bit to make the math easy, but the numbers-- this really did happen, but not with these exact numbers.
So we had a work which was 2048 seconds and only 1 second of critical path.
And over this new program had only 1/2 as much work to do, but the critical path length was longer.
It was 8 seconds long.
If you predict on 32 processors what the runtime's going to be that formula says well, 65 seconds.
If you predict it on 32 processors this-- well, it's 40 seconds and that looks good, but we were going to be running the tournament on 512 processors where this term would start being less important than this term.
So this really did happen and we actually went back and validated that these numbers were right after we did the calculation and it allowed us to do the engineering to make the right decision and not be misled by something that looked good in the test environment.
We were able to predict what was going to happen on the big machine without actually having access to the big machine and that was very important.
Let me do some algorithms.
You guys probably have done some matrix multipliers over the past 3 weeks, right?
That's probably the only thing you've been able to do would be my guess.
So matrix multiplication is this operation.
I won't talk about it, but you know what it is.
In Cilk instead of doing the standard triply nested loops you do divide and conquer.
We don't parallelize loops we parallelize function calls, so you want to express a loops as recursion.
So to multipliy two big matrices you do a whole bunch of little matrix multiplications of the sub-blocks and then you express those little matrix multiplications themselves and go off and recursively do smaller matrix multiplications.
So this requires 8 multiplications of matrices these of 1/2 the number of rows and 1/2 the number columns an one edition at the end where you add these two matrices together.
That's the algorithm that we do, it's the same total work as the standard one, but it's just expressed recursively.
So a matrix multiply is you do these 8 multiplies.
I had to create a temporary variable, so the first four multiplies the A's and B's into C. The second four multiply the A's and B's into T and then I have to add T into C. So I do all those spawns, do all the multiplies.
I do a sync because I better not start using the results on the multiplies and adding them until the multiplies are done
Which four do you add?
What?
There's parallelism in add.
Matrix addition
Yeah, but it doesn't add spawn extent
Well, we spawn off add.
I don't understand-- [INTERPOSING VOICES]
So you have to spawn Cilk functions even if you're only executing one of them at a time.
Cilk functions are spawned, C functions are called.
It's a decision that's built into the language.
It's not really a fundamental decision.
It's just that's the way we did it
Why'd you choose to have the key word then?
That's just documentation from the caller side?
Yeah, we found we were less likely to make a mistake if we sort of built it into the type system in this way.
But I'm not convinced that this is the best way to do this type system
Can the C functions spawn a Cilk function
No.
You can only call spawn, spawn, spawn, spawn then you can call C functions at the leaves.
It turns out you can actually spawn Cilk functions if you're a little clever about-- there's a mechanism for a Cilk system running in the background and if you're running C you can say OK, do this Cilk function in parallel.
So we have that, but that's not didactic
Sorry, I have a question about the sync spawning.
Is the sync actually doing a whole wave or -- like, in the case of-- maybe not in the case of the add here, but in plenty of other practical functions you get inside the spawn function looking at the tendencies of the parameters, right?
Based on how those were built from previous spawned funcitons.
You can actually just start processing so long as it's guaranteed that the results are available before you actually read them
So there's this other style of expressing parallelism which you see in some of the data flow languages where you say well, I've computed this first multiply, why can't I get started on the corresponding part of the addition.
And it turns out that in those models there's no performance guarantees.
The real issue is you run out of memory.
It's a long topic, let's not go into it, but there's a serious technical issue with those programming models.
We have very tight memory bounds as well, so we simultaneously get these good scheduling bounds and good memory bounds and if you are doing that you could have sort of a really large number of temporaries required and run out of memory.
The data flow machine used to have this number-- there was a student, Ken Traub, who was working on Monsoon when Greg Papadapolous was here and he came up with this term which we called Traub's constant, which was how long the machine could be guaranteed to run before it crashed from being out of memory.
And that was-- well, he took the rate at which it Kahn's divided by the amount of memory and that was it.
And many data flow programs had that property that Monsoon could run for 40 seconds and then after that you never knew.
It might start crashing at any moment, so everybody wrote short data flow programs.
So one of the things you actually do when you're implementing, when you're trying to engineer this to go fast, is you course in the base case, which I didn't describe up there.
You don't just do a 1 by 1 matrix multiplied down there at the leaves of this recursion.
Because then you're not using the processor pipeline efficiently.
You call the Intel Math Kernel Library or something on an 8 by 8 matrix so that it really gets the pipeline a chance to chug away.
So analysis.
This matrix addition operation-- well, what's the work for matrix addition?
Well the work to do a matrix operation on n rows is well, you have to do 4 additions of size n over 2.
Plus there's order 1 work here for the sync.
And that recurrence has solution order n squared.
Well, that's not surprising.
You have to add up 2 matrices which are n by n. That's going to be n squared so that's a good result.
The critical path for this is well, you have to do all of these in parallel.
So whatever the critical path of the longest one is, they're all the same so it's just the critical path of the size n over 2 plus quarter 1, so the critical path is order log n. For matrix multiplication, sort of the reason I do this is I can.
This is a model which I can do this analysis, so I have to do it.
But really, being able to do this analysis is important when you're trying to make things run faster.
Matrix multiplication, well, the work is I have to do 8 little matrix multiplies plus I have to do the matrix add.
The work has solution order n cubed and everybody knows that there's order n cubed multiply adds in a matrix multiplier, so that's not very surprising.
The critical path is-- well, I have to do a add so that takes log n, plus I have to do a multiply on a matrix that's 1/2 the size.
So the critical path length of the whole thing has solution order log squared n. So the total parallelism of matrix multiplication is the work over the span, which is n cubed over log squared n. So if you have a 1000 by 1000 matrix that means your parallelism is close to 10 million.
There's a lot of parallelism and in fact, we see perfect linear speedup on matrix multiply because there's so much parallelism in it.
It turns out that this stack temporary that I created so that I could do these multiplies all in parallel is actually costing me work because I'm on a machine that has cache and I want to use the cache effectively.
So I really don't want to create a whole big temporary matrix and blow my cache out if I can avoid it.
So I proposed the following matrix multiply, which is I first do 4 of the matrix multiplies into C1 then I do a sync and then I do the other 4 into C1 and another sync.
And I forgot to do the add-- oh, no those are multiply adds so they're multiplying and adding in.
And this saves space because it doesn't need a temporary, but it increases the critical path.
So is that a good idea about or a bad idea?
Well, we can answer part of that question with analysis.
Saving space we know is going to save something.
What does it do to the work in critical path?
Well, the work is still the same, it's n cubed because we didn't change the number of flops that we're doing.
But the critical path has grown.
Instead of doing 1 times a matrix multiply, we have to do one and then sync and then do another one.
So it's 2 matrix multiplies of 1/2 the size plus the order 1 and that recurrence has solution order n instead of order log squared n. So that sounds bad, we've made the critical path longer
[OBSCURED]
What?
Yeah.
So parallelism is now order n squared instead of n cubed over log squared n and for a 1000 by 1000 matrix that means you still have a million fold parallelism.
So for relatively modest sized matrices you still have plenty of work to do this optimization.
So this is a good transformation to do it.
One of the advantages of Cilk is that you can do this kind of You could say, let me do an optimization.
I can do an optimization in my C code and I get to take advantage of it in the Cilk code.
I could do this kind of optimization of trading work for parallelism.
If I have a lot of work that sometimes is a good idea.
Ordinary matrix multiplication just is really bad.
Basically you can imagine spawning off the n squared inner dot products here and computing them all in parallel.
It has work n cubed parallelism log n. I mean, critical path log n so the parallelism's even better.
It's n cubed over log n instead of n squared.
That looks better theoretically, but it's really bad in practice because it has such poor cache behavior.
So we don't do that.
I'll just briefly talk about how it works.
So Cilk does work-stealing.
We had did double ended queue-like decque.
So at the bottom of the queue is the stack where you push and pop things and the top is something where you can pop things off if you want to.
And so what's running is all these processors are running each on their own stack.
They're all running the ordinary serial code.
That's sort of the basic situation.
They're pretty much running the serial code most of the time.
So some processor runs.
It pushes.
Well, it doesn't spawn, so what does it do?
It pushes something onto its stack because it's just a function call.
And it does another couple more spawns so things pop off.
Somebody returns so he pops his stack.
So far everything's going on, they're not communicating, they're completely independent computations.
This guy spawns and now he's out of work.
Now he has to do something.
What he does is he goes and picks another processor at random and he steals the thing from the other end of the stack.
So he's unlikely to conflict because this guy's pushing and popping down here, but there's a lock in there, thers's a little algorithm.
A non-blocking algorithm actually, it's not lock.
And so he goes and he steals something and come on, slide over there.
Whoa.
Yes, that's animation, right?
That's the extent of my animation.
And then he starts working away.
And the theorem is that a work-stealing scheduler like this gives expected running time with high probability actually of T sub 1 over P plus T sub infinity on P processors.
And the pseudoproof is a little bit like the proof for Brent's Theorem, which is either you're working or stealing.
If you're working well, that goes against T sub 1 over P. You can't do that very much or you run out of work.
If you're stealing well, each steal has a chance that it steals the thing that's on the critical path.
You may actually steal the wrong thing, but you actually have a 1 in P chance that you're the one who steals the thing that it's on the critical path and then in which case the expected number-- so you had this chance of 1 over P of reducing the critical path length by 1, so after this many steals the critical path is all gone.
So you can only do P times T infinity steals.
This high probability it comes out.
And that gives you these bounds.
OK, I'm not going to give you all this stuff.
Message passing sucks, you know.
You guys know.
There's probably nothing else in here.
So basically the pitch here is that you get some high level linguistics support for these very fine-grained parallelism.
It's an algorithmic programming model so that means that you can do engineering for performance.
There's fairly easy conversion of existing code, especially when you combine it with the race detector.
You've got this factorization of the debugging problem and to debugging your serial code is you run it with all the Cilk stuff turned off.
You allied the program and make sure your program works.
Then you run it with the rate detector to make sure you get the same answer in parallel and then you're done.
Applications in Cilk don't just scale to large number of processors, they scale down to small numbers, which is important if you only have two processors or one.
You don't suddenly want to pay a factor of 10 to get off the ground, which happens sometimes on clusters running MPI.
You have to pay a big overhead before you've made any progress.
And one of the advantages for example is that the number of processors might change dynamically.
In this model that's OK because it's not part of the program.
So you may have the operating system reduce the number of actual worker threads that you have doing that work-stealing and that can work.
One of the bad things about Cilk is that it doesn't support sort of data parallel or program model kind of parallelism.
You really have to think of things as this divide and conquer kind of the world.
And if you have trouble expressing that-- situations where you're doing Jacobi update and you very carefully put things on, had each processor work on its local memory and then they only have to communicate at the boundaries.
That's difficult to do right in Cilk because essentially every time you go around the loop of I have all these things to do.
All the work-stealing happens randomly and it happens on a different processor.
So it's not very good at that sort of thing, although it turns out Jacobi update's not a very good example for that because there are more sophisticated algorithms that use cache effectively that you can express in Cilk and I would have no idea how to no say those in some of these sort of data parallel languages.
Using the cache efficiently is really important on modern processors
Thank you.
Questions?
You can download Cilk, there's a bunch of contributors.
Those are the Cilk worms and you can download Cilk off our webpage.
Just Google for Cilk and you'll find it.
It's a great language, you'll love it.
You'll love it much more than what you've been doing
How does the Cilk play with processor [OBSCURED]?
Well, you have to have a language, a compiler that can generate those.
If you have an assembly command or you have some other complier that can generate those.
So I just won the HPC challenge, which is this challenge where everybody tries to run parallel programs and argue that they get productivity.
For that there were some codes like matrix multiply and LUD composition with pivoting.
Basically at the leads of the computation I call the Intel Math Kernel Library.
Which in turn uses the SSE instructions.
You could do anything you can do in C in the C parts of the code because Cilk compiler just passes those through.
So if you have some really efficient pipeline code for doing something, up to some point it made sense to use that
[OBSCURED]
So I ran it on NASIS Columbia.
So the benchmark consists of-- well, there's 7 applications they have.
6 of which are actually well-defined.
One of them is this thing that just measures network performance or something, so it doesn't have any real semantics.
There's 6 benchmarks.
One of them is LUD composition, one of them is DJEM matrix multiplication and this FFT and 3 others.
So I implemented all 6, nobody else implemented all 6.
It turns out that you had to implement 3 in order to enter.
Almost everybody implemented 3 or 4, but I did all 6 which is part of why I won.
So I could argue that in a weeks work I just implemented--
What is [OBSCURED]?
So the prize has two components.
Performance and productivity or elegance or something and it's completely whatever the judges want that to be.
So it was up to me as a presenter to make the case that I was elegant.
Because I had my performance numbers, which were pretty good and it turned out that the IBM entry for x10 did me more good than I did, I think.
Because they got up there and they compared the performance of x10 to their Cilk implementation and their x10 thing was almost as good as Cilk.
So after that I think the judges said they had to give me the prize.
So basically, it went down to supercomputing and each of us got 5 minutes to present and there were 5 finalists.
We did our presentation and then they gave out the -- So they divided the prize three ways: the people who got the absolute best performance, which were some people running UPC and the people who had the most elegance based on minimal number of lines of codes and that was Cleve at -- what's his name?
The Mathworks guy, MATLAB guy.
Who said, look, matrix, LUD composition.
LU of P. It's very elegant, but I don't think that it really sort of explains what you have to do to solve the problems.
So he won the prize for most elegant and I got the prize for best combination, which they then changed-- in the final citation for the prize they said, most productivity.
That was the prize.
So I actually won the contest because that was what the contest was supposed to be was most productivity.
But I only won 1/3 of the prize money because they divided it three ways
Any other question?
Thank you
Thank you
We'll take a 5 minute break and since you had guest lecturer I do have [OBSCURED]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Mike Acton today who has done a lot of programming on cell and also done a lot of game development.
He came from California just like this today and [OBSCURED]
Yeah, it's really cool.
I'll tell you
He's going to talk about what it's really like to use cell and PS3 and what it's like to program games.
So I think it's going to be a fun lecture
All right, so anyway, I'm the Engine Director at Insomniac Games.
I've only recently taken that position, previously I was working on PS3 technology at Highmoon studios, which is with vin studios.
And I've worked at Sony.
I've worked for Titus, and Bluesky Studios in San Diego.
And I've been doing game development, 11, 12 years.
Before that I was working in simulation.
So, the PlayStation 3 is a really fun platform.
And I know you guys have been working on cell development.
Working with the PS3 under Linux.
Working as developers for the PS3 is definitely a different environment from that.
I think I'm going to concentrate more on the high-level aspects of how you design a game for the cell.
And how the cell would impact the design, and what are the elements of the game.
Just stuff that you probably haven't had as part of this course that you might find interesting.
And you can feel free to interrupt me at any time with questions or whatever you'd like.
So just, I wanted to go over, briefly, some of the different types of game development and what the trade-offs for each one of them are.
Casual games, console games, PC games, blah, blah, blah.
Casual games, basically, are the small, simple games that you would download on the PC, or you would see on Yahoo or whatever.
And those generally don't have really strict performance requirements.
Where a console game, we have this particular advantage of knowing the hardware and the hardware doesn't change for an entire cycle.
So for five, six years, we have exactly the same hardware.
And that's definitely an advantage from a performance point anyway.
In this case, it's PlayStation 3.
As far as develpment priorities, development priorities for a console game-- and especially a PS3 game-- development would be completely different than you might find on another kind of application.
We don't really consider the code itself important at all.
The real value is in the programmers.
The real value is in the experience, and is in those skills.
Code is disposable.
After six years, when we start a new platform we pretty much have to rewrite it anyway, so there's not much point in trying to plan for a long life span of code.
Especially when you have optimized code written in assembly for a particular platform.
And to that end, the data is way more significant to the performance than the code, anyway.
And the data is specific to a particular game.
Or specific to a particular type of game.
And certainly specific to a studios pipeline.
And it's the design of the data where you really want to spend your time concentrating, especially for the PS3.
Ease of programming-- whether or not it's easier to do parallelism is not a major concern at all.
If it's hard, so what?
You do it.
That's it.
Portability, runs on PlayStation 3, doesn't run anywhere else.
That's a non-concern.
And everything is about performance.
Everything we do.
A vast majority of our code is either hand up from IC, or assembly, very little high level code.
Some of our gameplay programmers will write C plus plus for the high level logic, but as a general, most of the code that's running most the time is definitely optimized.
Yeah?
If programming is a non-priority, does that mean to say that if you're developing more than one product or game, they don't share any common infrastructure or need?
No, that's not necessarily true.
If we have games that share similar needs, they can definitely share similar code.
I mean, the point I'm trying to make is, let's say in order to make something fast it has to be complicated.
So be it, it's complicated.
Whether or not it's easy to use for another programmer is not a major concern
So you wish it was easier?
No.
I don't care.
That's my point
Well, it's not as important as performance, but if someone came to you with a high performance tool, you would like to use it?
I doubt they could.
The highest performance tool that exists is the brains of the programmers on our team.
You can not create-- it's theoretically impossible.
You can not out perform people who are customizing for the data, for the context for the game.
It is not even remotely theoretically possible
That didn't come out in assembly programming for general purpose but we'll take this offline?
And there was a day when that was also true for general preferred cleary at the time, but it's no longer true
It is absolutely--
So the average person prefers to go on -- take it offline
Average person.
We're not the average people.
We're game programmers.
Yeah?
So does cost ever become an issue?
I mean--
Absolutely, cost does become an issue.
At a certain pont, something is so difficult that you either have to throw up your hands or you can't finish in time
Do you ever hit that point?
Or you figure out a new way of doing it.
Or do a little bit less.
I mean we do have to prioritize what you want to do.
At the end of the day you can't do everything you want to do, and you have another game you need to ship eventually, anyway.
So, a lot of times you do end up tabling things.
And say, look we can get 50% more performance out of this, but we're going to have to table that for now and scale back on the content.
And that's why you have six years of development.
You know, maybe in the next cycle, in the next game, you'll be able to squeeze out a little bit more.
And the next one you squeeze out a little bit more.
That's sort of this continuous development, and continuous optimization over the course of a platform.
And sometimes, yeah, I mean occasionally you just say yeah, we can't do it or whatever, it doesn't work.
I mean, that's part and parcel of development in general.
Some ideas just don't pan out.
But--
Have you ever come into a situation where programming conflicts just kills a project?
Like Microsoft had had a few times, like they couldn't put out [OBSCURED].
Couldn't release for--
Sure, there's plenty of studios where the programming complexity has killed the studio, or killed the project.
But I find it hard to believe-- or it's very rarely-- because it's complexity that has to do specifically with optimization.
That complexity usually has to do with unnecessary complexity.
Complexity that doesn't achieve anything.
Organization for the sake of organization.
So you have these sort of over designed C plus plus hierarchies just for the sake of over organizing things.
That's what will generally kill a project.
But in performance, the complexity tends to come from the rule set-- what you need to do to set it up.
But the code tends to be smaller when it's faster.
You tend to be doing one thing and doing one thing really well.
So it doesn't tend to get out of hand.
I mean, it occasionally happens but, yeah?
So in terms of the overall cost, how big is this programming versus the other aspect of coming up with the game?
Like the game design, the graphics--
So, for example, do you have--
OK, development team?
So--
So how many programmers, how many artists, how many--
Maybe, let's-- so for example, like, now it's like, what, $20 million to deliver a PS3 game?
Between $10 and $20 million, yeah
So let's develop [OBSCURED]
So artists are by far the most-- the largest group of developers.
So you have animators and shade artists, and textual artists, and modelers, and enviromental artists, and lighters.
And so they'll often outnumber programmers 2:1.
Which is completely different than-- certainly very different from PlayStation and the gap is much larger than it was on PlayStation 2.
With programmers you tend to have a fairly even split or you tend to have a divide between the high level game play programmers and the low level engine programmers.
And you will tend to have more game play programmers than engine programmers, although most-- the majority of the CPU time is spent in the engine code.
And that partially comes down to education and experience.
In order to get high performance code you need to have that experience.
You need to know how to optimize.
You need to understand the machine.
You need to understand the architecture and you need to understand the data.
And there's only so many people that can do that on any particular team
Code size wise, How is the code size divided between game playing and AI, special effects?
Just like, the amount of code?
Yeah, it should be small I guess
Yeah, I mean, it's hard to say.
I mean, because it depends on how many features you're using.
And, you know sort of the scope of the engine is how much is being used for a particular game, especially if you're targeting multiple games within a studio.
But quite often-- interestingly enough-- the game play code actually overwhelms the engine code in terms of size and that is back to basically what I was saying that the engine code tends to do one thing really well or a series of things really well
Game play code also C plus plus?
These days it's much more likely that game play code is C plus plus in the high level and kills performance and doesn't think about things like cache.
That's actually part of the problem with PlayStation 3 development.
It was part of the challenge that we've had with PlayStation 3 development.
Is in the past, certainly with PlayStation 2 and definitely on any previous console, this divide between game play and engine worked very well.
The game play programmers could just call a function and it did its fat thing really fast and it came back and they continue this, but in a serial program on one process that model works very well.
But now when the high level design can destroy performance but through the simplest decision, like for example, in collision detection if the logic assumes that the result is immediately available there's virtually no way of making that fast.
So the high-level design has to conform to the hardware.
That's sort of a challenge now, is introducing those concepts to the high-level programmer who haven't traditionally had to deal with it.
Does that answer that question as far as the split?
You said 2:1, right?
Approximately 2:1, artist to programmers.
It varies studio to studio and team to team, so it's hard to say in the industry as a whole.
So back basically to the point of the code isn't really important.
The code itself doesn't have a lot of value.
There are fundamental things that affect how you would design it in the first place.
The type of game, the kind of engine that would run a racing game is completely different than the kind of engine that would run a first person shooter.
The needs are different, the optimizations are totally different, the data is totally different, so you wouldn't try to reuse code from one to the other.
It just either wouldn't work or would work really, really poorly.
The framerate-- having a target of 30 frames per second is a much different problem than having a target of 60 frames per second.
And in the NCSC territories those are pretty much your only two choices- 30 frames or 60, which means everything has to be done in 16 and 2/3 milliseconds.
That's it, that's what you have-- 432 milliseconds.
Of course, back to schedule and cost, how much?
You know, do you have a two year cycle, one year cycle, how much can you get done?
The kind of hardware.
So taking for example, an engine from PlayStation 2 and trying to move it to PlayStation 3 is sort of a lost cause.
The kind of optimizations that you would do, the kind of parallelization you would do is so completely different, although there was parallelization in PlayStation 2, the choices would have been completely different.
The loss from trying to port it is much, much greater than the cost of just doing it again
[OBSCURED]
I don't know that there's an average.
I mean, if you wanted to just like homogenize the industry, it's probably 18 months.
The compiler actually makes a huge, significant difference in how you design your code.
If you're working with GCC and you have programmers who have been working with GCC for 15 years abd who understand the intricacies and issues involved in GCC, the kind of code you would write would be completely different than if you were using XLC for example, on the cell.
There are studios-- Insomniac doesn't, but there are other studios who do cross platform design.
So for example, write Playstation 3 games and Xbox 360 games and/or PC titles.
At the moment, probably the easiest approach for that is to target the PlayStation 3.
So you have these sort of SPU friendly chunks of processing SPU chunks, friendly chunks of data and move those onto homogenous parallel processors.
It's not the perfect solution, but virtually all cross platform titles are not looking for the perfect solution anyway because they cannot fully optimize for any particular platform.
I wanted to go through-- these are a basic list of some of the major modules that a game is made out of.
I'll go through some of these and explain how designing on the cell impacts the system.
I'm not going to bother reading them.
I assume you all can read.
So yeah, I'm going to go over the major system, a few of the major systems and then we're going to drive a little bit into a specific system, in this case an animation system.
And just talk it through, basically you see how each of these steps are affected by the hardware that we're running on it.
So just to start with when you're designing a structure, any structure, anywhere-- the initial structure is affected by the kind of hardware that you're running.
And in this particular case on the SPU and there are other processors where this is equally true, but in this conventional structure where you say structure class or whatever and you have domain-constrained structures are of surprisingly little use.
In general, the data is either compressed or is in a stram or is in blocks.
It's sort of based on type, which means that there's no fixed size struct that you could define anyway.
So as a general rule, the structure of the data is defined within the code as opposed to in a struct somewhere.
And that's really to get the performance from the data, you group things of similar type together rather than for example, on SPU, having flags that say this is of type A and this is of type B.
Any flag implies a branch, which is-- I'm sure you all know at this point-- is really poor performing on SPU.
So basically, pull flags out, resort everything and then move things in streams.
And all of these types are going to be of varying sizes.
In which case there's very little point to define those structures in the first place because you can't change them.
And the fact that you're accessing data in quadwords anyway.
You're always either loading and storing in quadwords, not on scalars, so having scalar fields in a structure is sort of pointless.
So again, only SPU generally speaking structures are of much use.
When you go to define structures in general you need to consider things like the cache, the TLB, how that's going to affect you're reading out of the structure or writing to the structure.
More to the point of you cannot just assume that if you've written some data definition that you can port it to another platform.
It's very easy to be poorly, a performing platform to platform.
In this case, when we design structures you have to consider the fundamental units of the cell.
The cache line is a fundamental unit of the cell.
Basically, you want to define things in terms of 128 bytes of wide.
What can you fit in there because you read one you read them all, so you want to pack as much as possible into 128 bytes and just deal with that as a fundamental unit.
16 bytes, of course, you're doing load and stores through quadword load and store.
So you don't want to have little scalar bits in there that you're shuffling around.
Just deal with it as a quadword.
And don't deal with anything smaller than that.
So basically the minimum working sizes, in practice, would be 4 by 128 bits wide and you can split that up regularly however you want.
So to that point I think-- here's an example-- I want to talk about a vector class.
Vector class js usually the first thing a programmer will jump onto when they might want to make something for games.
But in real life, it's probably the most useless thing you could ever write.
It doesn't actually do anything.
We have these, we know the instruction set, it's already in quadwords.
We know the loads and stores, we've already designed your data so it fits properly.
This doesn't give us anything.
And it potentially makes things worse.
Allowing component access to a quadword, especially on the PPU is ridiculously bad.
In practice, if you allow component access, high-level programs will use component access.
So if you have a vector class that says get x, get y, whatever, somebody somewhere is going to use it, which means the performance of the whole thing just drops and it's impossible to optimize.
So as a general rule, you pick your fundamental unit.
In this case, the 4 by 128 bit unit that I was talking about and you don't define anything smaller than that.
Everything is packed into a unit about that size.
And yes, in practice there'll be some wasted space at the beginning or end of streams of data, groups of data, but it doesn't make much difference.
You're going to have that wasted space if you are-- you're going to have much more than that in wasted space if you're using dynamic memory, for example, which when I get to it-- I don't recommend you use either.
So some things to consider when you're doing this sort of math transformation anyway is, are you going to do floats, double, fixed point?
I mean, doubles write out.
There's no point.
Regardless of the speed on the SPU of a double, there's no value in it for games.
We have known data, so if we need to we can renormalize a group of around a point and get into the range of a floating point.
It's a nonissue.
So there's no reason to waste the space in a double at all, unless it was actually faster, which it isn't.
So we don't use it.
Sort of the only real problematic thing with the SPU floating point is its format and not supporting denormalized numbers becomes problematic, but again, you can work around it by renormalizing your numbers within a known range so that it won't to get to the point where it needs to denormalize-- at least for the work that you're actually doing.
Yeah?
[OBSCURED]
Every program will write its own vector class.
And I'm saying that that's a useless exercise.
Don't bother doing it.
Don't use anybody else's either.
If you're writing for the cell-- if you're writing in C you have the SI intrinsics.
They're already in quadwords, you can do everything you want to do and you're not restricted by this sort of concept of what a vector is.
If you want to deal with, especially on the SPU where you can freely deal with them as integers or floats or whatever seamlessly without cost, there's plenty that you can do with the floating point number if you treat it as an integer.
And when on either AltiVec or the SPU where you can do that without cost there's a huge advantage to just doing it straight
[OBSCURED]
Well, I'm saying write it in assembly.
But if you have to, use the intrinsics.
But certainly don't write a vector class.
So memory management.
Static allocations always prefer the dynamic.
Basically, general purpose dynamic memory allocation, malloc free, whatever has just absolutely no place in games.
We don't have enough unknowns for that to be valuable.
We can group our data by specific types.
We know basic ranges of those types.
The vast majority of the data is known in advance, it's actually burned onto the disk.
We can actually analyze that.
So most of our allocations tend to calculate it in advance.
So you load the level and oftentimes you just load memory in off the disc into memory and then fix up the pointers.
For things that change during the runtime, just simple hierarchical allocators, block allocators where you have fixed sizes is always the easiest and best way to go.
These are known types of known sizes.
The key to that is to organize your data so that's actually a workable solution.
So you don't have these sort of classes or structures that are dynamically sized.
That you group them in terms of things that are similar.
Physics data here and AI data is separately here in a separate array.
And that way those sort of chunks of data are similarly sized and can be block allocated without any fragmentation issues at all.
Eventually you'll probably want to design an allocator.
Things to consider are the page sizes.
That's critically important, you want to work within a page as much as you possibly can.
So you want to group things, not necessarily the same things, but the things that will be read together or written together within the same page.
So you want to have a concept of the actual page up through the system.
Probably the most common mistake I see in a block allocator, so somebody says-- everybody knows what I mean by block allocator?
Yeah?
OK.
So the most common mistake I see people make is that they do least recently used.
They just grab the most least recently used block and use that when summoning a request.
That's actually pretty much the worst thing you can possibly do because that's the most likely thing to be called.
That's the most likely thing to be out of cache, both out of L1 and L2.
Just the easiest thing you can do to change that is just use most recently used.
Just go up the other way.
I mean, there are much more complicated systems you can use, but just that one small change where you're much more likely to get warm data is going to give you a big boost.
And again, like I said, use hierarchies of allocations instead of these sort of static block allocations.
Instead of trying to have one general purpose super mega allocator that does everything.
And again, if it's well planned, fragmentation is a non-issue, it's impossible.
Cache line, oh, and probably another important concept to keep in mind as you're writing your allocator is the transfer block size of the SPU.
If you have a 16K block and the system is aware of fixing K blocks then there are plenty of cases where you don't have to keep track of-- in the system-- the size of things.
It's just how many blocks, how many SPU blocks do you have?
Or what percentage of SPU blocks you have?
And that will help you can sort of compress down your memory requirements when you're referring to blocks and memory streams and memory
About the memory management for data here, you also write overlay managers for code for the user?
Well, it basically amounts to the same thing.
I mean, the code is just data, you just load it in and fix up the pointers and you're done
I was just wondering whether IBM gives you embedding --
We don't usse any of the IBM systems at all for games.
I know IBM has an overlay manager as part of the
Well, not really.
It's --
Well, they have some overlay support, right?
That's not something we would ever use.
And in general, I mean, I guess that's probably an interesting question of how--
So it's all ground up?
What's that?
All your development is ground up?
Yeah, for the most part.
For us, that's definitely true.
There are studios that, especially cross platform studios that will take middleware development and just sort of use it on a high-level.
But especially when you're starting a first generation platform game, there's virtually nothing there to use because the hardware hasn't been around long enough for anybody else to write anything either.
So if you need it, you write it yourself.
Plus that's just sort of the general theme of game development.
It's custom to your situation, to your data.
And anything that's general purpose enough to sell as middleware is probably not going to be fast enough to run a triple A title.
Not always true, but as a general rule, it's pretty valid.
OK, so-- wait, so how'd I get here?
All right, this is next.
So here's another example of how the cell might affect design.
So you're writing a collision detection system.
It's obvious that you cannot or should not expect immediate results from a collision detection system, otherwise you're going to be sitting and syncing all the time for one result and performance just goes out the window, you may as well just have a serial program.
So you want to group results, you want to group queries and you want potentially, for those queries to be deferred so that you can store them, you can just DMA them out and then whatever process needed then we'll come back and grab them later.
So conceptually that's the design you want to build into a collision detection system, which then in turn affects the high-level design.
So AI, scripts, any game code that might have previously depended on a result being immediately available, as in they have characters that shoot rays around the room to decide what they're going to do next or bullets that are flying through the air or whatever, can no longer make that assumption.
So they have to be able to group up their queries and look them up later and have other work to do in the meantime.
So this is a perfect example of how you cannot take old code and move it to the PS3.
Because old code, serial code would have definitely assumed that the results were immediately available because honestly, that was the fastest way to do it.
So on a separate issue, we have SPU decomposition for the geometry look up.
So from a high-level you have your entire scene in the level of the world or whatever and you have the set of queries in the case of static-- did I collide with anything in the world?
Or you have a RAID that, where does this RAID collide with something in the world?
And so you have this problem of you have this large sort of memory database in main RAM and you have the smallest spew, which obviously cannot read in the whole database, analyze it, and spit out the result.
It has to go back and forth to main RAM in order to build its result.
So the question is how do you decompose the memory in the first place to make that at least somewhat reasonably efficient?
The first sort of instinct I think, based on history is sort of the traditional scene graph structures like BSP tree or off tree or something like that.
Particularly, on the SPU because if TLB misses that becomes really expensive, really quickly when you're basically hitting random memory on every single node on the tree.
So what you want to do is you want to make that hierarchy as flat as you possibly can.
If the leafs have to be bigger that's fine because it turns out it's much, much cheaper to stream in a bigger group of-- as much data as you can fit into the SPU and run through it and make your decisions and spit it back out than it is to traverse the hierarchy.
So basically, the depth of your hierarchy in your scene database is completely determined by how much data you can fit into the SPU by the maximum size of the leaf node.
The rest of the dep is only because you don't have any other choice.
You know, And basically the same thing goes with dynamic geometry as you have geometry moving around in the scene, characters moving around in the scene-- they basically need to update themselves into their own database, into their own leaves and they'll do this in groups.
And then when you query, you basically want it to query as many of those as possible, as you can possibly fit in at once.
So you could have sort of a broad faced collision first, where you have all of the groups of characters that are potentially maximum in this leaf, so bound and box or whatever.
So even though you could in theory, in principle narrow that down even more, the cost for that, the cost for the potential memory miss for that is so high that you just want to do a linear search through as many as you possibly can on SPU.
Does that make sense?
Procedural graphics-- so although we have a GPU on the PlayStation 3, it does turn out that the SPU is a lot better at doing a lot of things.
Things basically where you create geometry for RSX to render.
So particle system, dynamic particle systems.
Especially where their systems have to interact with the world in some way, which will be much more expensive on the GPU.
Sort of a dynamic systems like cloth.
Fonts is actually really interesting because typically you'll just see bitmap fonts in which case are just textures.
But if you have a very complex user interface then just the size of the bitmap becomes extreme and if you compressed them they look terrible, especially fonts.
Fonts need to look perfect.
So if you do do procedural fonts, for example, two type fonts, the cost of rendering a font actually gets significant.
And in this case, the SPU is actually a great use for rendering a procedural font.
Rendering textures is basically the same case as font.
Procedural textures like if you do noise-based clouds or something like that.
And parametric geometry, it's like nurbs or subdivision services or something like that, is a perfect case for the SPU.
Is there a question?
Geometry database, OK. First thing scene graphs are worthless.
Yeah?
So of those sort of differnet conceptualized paths, are you literally swapping code in and out of the SPUs with the data many times per frame?
Or is it more of a static---
OK, that's an excellent question.
It totally depends.
I mean, in general through a game or through, at least a particular area of a game the SPU set up is stable.
So if we decide you're going to have this SPU dedicated to physics for example, it is very likely that that SPU is stable and it's going to be dedicated physics, at least for some period of time through the level or through the zone or wherever it is.
Sometimes through an entire game.
So there are going to be elements of that where it's sort of a well balanced problem.
There's basically no way you're going to get waste.
It's always going to be full, it's always going to be busy.
Collision detection and physics are the two things that you'll never have enough CPU to do.
You can always use more and more CPU.
And basically, the rest can be dynamically scheduled.
And the question of how to schedule it, is actually an interesting problem.
It's my opinion that sort of looking for the universal scheduler that solves all problems and magically makes everything work is a total lost cause.
You have more than enough data to work with and in your game to decide how to schedule your SPUs basically, manually.
And it's just not that complicated.
We have six SPUs.
How to schedule six SPUs is just not that complicated a problem, you could write it down on a piece of paper.
OK, so scene graphs are almost always, universally a complete waste time.
They store way too much data for no apparent reason.
Store your databases independently based on what you're actually doing with them, optimize your data separately because you're accessing it separately.
The only thing that should be linking your sort of domain object is a key that says all right, we'll exist in this database and the database and this database.
But to have this sort of giant structure that keeps all of the data for each element in the scene is about the poorest performing you can imagine for both cache and TLB and SPU because you can't fit it in individual node on the SPU.
I think I covered that.
Here's an interesting example, so what you want to do is if you have the table of queries that you have-- bunch of people over the course of a frame say I want to know if I collided with something.
And then if you basically make a pre-sort pass on that and basically, spatially sort these guys together, so let's say you have however many you can fit in a SPU.
So you have four of these queries together.
Although they might be a little further apart then you would hope, you could basically create a baling box through a single query on the database that's the sum of all of them and then as I said, now you have a linear list that you can just stream through for all of them.
So even though it's doing more work for any individual one, the overhead is reduced so significantly that the end result is that it's significantly faster.
And that's also what I mean by multiple simultaneous lookups.
Basically you want to group queries together, but make sure that there's some advantage to that.
By spatially pre-sorting them there is an advantage to that because it's more likely that they will have overlap in your queries.
So game logic.
Stuff that the cell would affect in game logic.
State machines are a good example.
If you defer your logic lines and defer your results, SPUs are amazingly perfect for defining state machines.
If you expect your logic lines to be immediately available across the entire system, SPU is absolutely horrid.
So if you basically write buffers into your state machines or your logic machines then each SPU can be cranking on multiple state machines at once where all the input and all the output lines are assumed to be deferred and it's just an extremely straightforward process.
Scripting, so scripting things like-- I don't know, lewis script or C script or something like that.
I mean, obviously the first thing to look at is the size of the interpreter.
Will it fit into an SPU to begin with?
Another option to consider is, can it be converted into SPU code, either offline or dynamically?
Because you'll find that most off the shelf scripting languages are scalar, sequential scripting languages.
So all of a P code within the scripting language itself basically defines scalar access.
So not only are you switching on every byte to every two bytes or whatever, so it's sort of poorly performing code from an SPU point of view, but it's also poorly performing code from a memory point of view.
So I guess the question is whether or not you can optimize the script itself and turn turn it into SPU code that you can then dynamically load or come up with a new script that's just much more friendly for the SPUs.
Another option if you have to use a single, sort of scalar scripting language like lua or C script or whatever, if you can run multiple streams simultaneously so that while you're doing these sort of individual offline memory lookups and reads and writes to main memory, that once one blocks you can start moving on another one.
As long as there's no dependencies between these two scripts we should be able to stream them both simultaneously.
Motion control actually turns out to be a critical problem in games in general that's often overlooked.
It's who controls the motion in the game.
Is is the AI?
So is it the controller in the case of the player?
I say, push forward, so the guy moves forward.
Is that really what controls it?
Or is it the physics?
So all the AI does is say, I want to move forward, tells the physic system I want to move forward and the physics tries to follow it.
Or is it the animation?
That you have the animators actually put translation in the animation, so is that translation the thing that's actually driving the motion and everything else is trying to follow it?
Turns out to be a surprisingly difficult problem to solve and every studio ends up with their own solution.
I forget what point I was making on how the cell affected that decision.
But--
[OBSCURED]
I think the point, probably that I was trying to make is that because you want everything to be deferred anyway, then the order does become a clearer sort of winner in that order.
Where you want the immediate feedback from the controls, the control leads the way.
You have the physics, which then follows, perhaps, even a frame behind that to say how that new position is impacted by the physical reality of the world.
And then potentially a frame behind that or half a frame behind that you have the animation system, which in that case would just be basically, a visual representation of what's going on rather than leading anything.
It's basically an icon for what's happening in the physics and the AI.
The limitations of your system are that it has to be deferred and that it has to be done in groups.
Basically, some of these sort of really difficult decisions have only one or two obvious answers.
All right, well I wanted to dig into animation a little bit, so does anybody have any questions on anything?
Any of the sort of the high-level stuff that I've covered up to this point?
Yeah?
So does the need for deferral and breaking into groups and staging, does this need break the desire for the higher-level programmers to abstract what's going on at the data engine level?
Or is that not quite the issue?
Well, let's say we get a game play programmer, right?
Fresh out of school, he's taught in school, C plus plus in school.
Taught to decompose the world into sort of the main classes and that they all communicate through each other maybe through messaging.
All right, well the first thing we tell him is that all that is complete crap.
None of that will actually work in practice.
So in some sense, yes, there is a sort of tendency for them to want this interface, this sort of clean abstraction, but abstraction doesn't have any value.
It doesn't make the game faster, it doesn't make the game cheaper.
It doesn't make--
[OBSCURED]
There's a bit of a religious.
Let's move on.
He has a lot of other interesting things to say.
And we can get to that question--
It sounds like there's two completely different communities involved in the development.
There's the engine developers and there's the higher-level--
That's a fair enough assessment.
There are different communities.
There is a community of the game play programmers and the community of engine programmers.
And they have different priorities and they have different experiences.
So yeah, in that way there is a division
I will let you go on.
You said you had a lot of interesting information to cover

I can bitch about that.
Don't worry
So just to get into animation a little bit.
Let's start with trying to build a simple animation system and see what problems come creep up as we're trying to implement it on the cell.
So in the simplest case we have a set of animation channels defined for a character, which is made up of joints.
We're just talking about sort of a simple hierarchical transformation here.
And some of those channels are related.
So in the case of rotation plus translation plus scale equals any individual joint.
So the first thing, typically that you'll have to answer is whether or not you want to do euler or quaternion rotation.
Now the tendency I guess, especially for new programmers is to go with quaternion.
They're taught that gimbal lock is a sort of insurmountable problem that only quaternion solves.
That's just simply not true.
I mean, gimbal lock is completely manageable in practice.
When you're trying to rotate on three axes and two axes rotate 90 degrees apart and the third axis can't be resolved or 180 degrees apart.
So you can't resolve one of the axes, right?
[OBSCURED]
Yeah, it's where it's impossible to resolve one of the axes and that's the nature of euler sort of rotation.
But sort of a quaternion rotation completely solves that mathematical problem, it's always resolvable and it's not very messy at all.
I mean, from a sort of C programmers perspective, it looks clean, the math's clean, everything's clean about it.
Unfortunately, it doesn't compress very well at all.
Where if you used euler rotation, which basically just means that the individual rotation for every axis.
So x rotation, y rotation, z rotation.
That's much, much more compressible because each one of those axes can be individually compressed.
It's very unlikely that you're always rotating all three axes, all the time, especially in a human character.
It's much more likely that only one axis is rotating on any one given time and so that makes it-- just without any change, without any additional compression-- it tends to make it about 1/3 of the size
[OBSCURED]
The animation data.
So you have this frame of animation, which is all these animation channels, right?
And then over time you have these different frames of animation, right?
If you store-- for every joint, for every rotation you store a quaternion over time, it's hard to compress across time because you're basically, essentially rotating all three axes, all the time.
Well, with-- yeah?
All right.
So let's say, of course, the next step is how do we store the actual rotation itself?
Do we store it in cloth, double, half precision, fixed point precision?
Probably the national tendency at this point would be to store it in a floating point number, but if you look at the actual range of rotation, which is extremely limited on a character, on any particular joint there are very few joints that would even rotate 180 degrees.
So a floating point is overkill, by a large margin on rotation-- for the range.
For the precision, however it's fairly good.
Especially if you're doing very small rotations over a long period of time.
So probably a more balanced approach would be to go with a 16 bit floating point from a half format where you keep most of the precision, but you reduce the range significantly.
There's also the potential for going with an 8 bit floating point format depending on the kind of animation that you're doing.
And I'll probably have this on another slide, but it really depends on how close-- how compressible a joint is depends on how close to the root it is.
The further away from the root the less it matters.
So the joint at your fingertip, you can compress a whole lot more because it doesn't matter as much, it's not going to affect anything else.
Where a joint at the actual root, the smallest change in motion will affect the entire system in animation and will make it virtually impossible for you to line up animations with each other, so that that particular joint needs to be nearly perfect.
And how do you store rotation?
Do you store them in degrees, radians, or normalized?
I have seen people store them in degrees.
I don't understand why you would ever do that.
It's just adding math to the problem.
Radians is perfectly fine if you're using off the shelf trigonometric functions- tan, sine whatever.
But typically, if you're going to optimize those functions yourself anyway, it's going to be much more effective go with a normalized rotational value.
So basically between zero and 1.
Makes it a lot easier to do tricks based on the circle.
Basically you can just take the fractional value and just deal with that.
So normalized rotation is generally the way to go and normalizing a half precision is probably the even bet for how you would store.
So looking at what we need to fit into an SPU if we're going to running to an end machine.
Yeah?
You talked a lot about compressing because of the way it's impacting data, what's the key driver of that?
The SPU has very little space
OK, so it's just the amount of space
Yeah, well OK.
There's two factors really, in all honesty.
So starting with the SPU.
That you have to be able to work through this data on the SPU.
But you also have the DMA transfer itself.
The SPU can actually calculate really, really fast, right?
I mean, that's the whole point.
So if you can transfer less data, burn through it a little bit to expand it, it's actually a huge win.
And on top of that we have a big, big game and only 256 megs of main ram.
And the amount of geometry that people require from a current generation game or next generation game has scaled up way more than the amount of memory we've been given, so we've only been given eight times as much memory as we had in the previous generation.
People expect significantly more than eight times as much geometry on the screen and where do we store that?
We have the Blu-Ray, we can't be streaming everything off the disc all the time, which is to another point.
You have 40 gigs of data on your disc, but only 256 megs of RAM.
So there's this sort of a series of compression, decompression to keep everything-- basically, think of RAM as your L3 cache.
So we look at what we want to store on an SPU.
Basically, the goal of this is we want to get an entire animation for a particular skeleton on an SPU so that we can transform the skeleton and output the resulting joint data.
So let's look at how big that would have to be.
So first we start with the basic nine channels per joint.
That's not assuming and again, you'd probably have additional channels, like foot step channels and sound channels and other sort of animation channels to help actually make a game.
In this case, we just want to animate the character.
So we have rotation times 3, translations times 3, and scales times 3.
So the first thing to drop and this will cover, this will reduce your data by 70%, is all the uniform channels.
So any data that doesn't actually change across the entire length of the animation.
It may not be zero, but it could be just one thing, one value that doesn't change across length of the animation.
So you pull all the uniform channels out.
And most things that's going to be scale, for example, most joints don't scale.
Although occasionally they do.
And translation, in a human our joints don't translate.
However, when you actually animate a character in order to get particular effects, in order to make it look more human you do end up needing to translate joints.
So we can reduce, but in order to that we need to build a map, basically, a table of these uniform channels.
So now we know this table of uniform channels has to be stored in the SPU along with now the remaining actual animation data.
Of course, multiplied by the number of joints.
So now we have what is essentially raw animation data.
So for the sake of argument, let's say the animation data has been baked out by Maya or whatever at 30 frames a second.
We've pulled out the uniform data, so now for the joints that do move we have these curves over time of the entire length of the animation.
The problem is if that animation is 10 seconds long, it's now way too big to fit in the SPU by a large margin.
So how do we sort of compress it down so that it actually will fit?
Again, just first of all, the easiest thing to do to start with is just do simple curve fitting to get rid of the things that don't need to be there that you can easily calculate out.
And again, the closer that you are to the root, the tighter that fits need to be.
Conversely, the further away you are from the root, you can loosen up the restrictions a little bit and have a little bit looser fit on the curve and compress a little bit more.
So if you're doing a curve fitting with the simple spline, basically you have to store your time values in the places that were calculated.
Part of the problem is now you have sort of these individual scalars with time can be randomly spread throughout the entire animation.
So any point where there's basically a knot in the curve, there's a time value.
And none of these knots are going to line up with each other in any of these animation channels.
So in principle, if you wanted to code this you would have to basically say, what is time right now and loop through each of these scalar values, find out where time is, calculate the postition on the curve and then spit out the result.
So one, you still have to have the unlimited length of data and two, you're looping through scalar values on the SPU, which is really actually, horrible.
So we want to find a way to solve that problem.
Probably the most trivial solution is just do spline segemnts.
You lose some compressibility, but it solves the problem.
Basically you split up the spline into say, sections of 16 knots and you just do that.
And in order to do that you just need a table, you need to add a table that says what the range of time are in each of those groups of 16 knots for every channel.
So when you're going to transform the animation, first you load this table in, you say, what's my time right now at time, t?
You go and say which blocks, which segments of the spline you need to load in for each channel, you load those in.
So now you have basically one section of the spline, which is too big probably for the current t, but it covers what t you're actually in.
So one block of spline for every single channel.
So the advantage of this, now that the spline is sorted into sections is that rather than having all the spline data stored, sort of linearly, you can now reorder the blocks so that the spline data from different channels is actually tiled next to each other.
So that when you actually go to do a load it's much more likely because you know you're going to be requesting all these channel at once and all on the same time, t, you can find a more or less, optimal ordering that will allow more of these group things to be grouped in the same cache or at least the same page.
And the advantage of course again, is now the length of animation makes absolutely no difference at all.
The disadvantage is its less compressible because you can only basically compress this one section of the curve, but a huge advantage is it solves the scalar loop problem.
So now you can take four of these scalar values all with a fixed known number of knots in it and just loop through all of the knots.
In principle you could search through and find a minimum number of knots to look through for each one of the scalars, but in practice it's much faster just to loop through all four simultaneously for all 16 knots and just throw away the results that are invalid as you're going through it.
That way you can use the SPU instruction set.
You can load quadwords, store quadwords, and do everything in the minimum single loop, which you can completely unroll.
Does anybody have the time?
It's [OBSCURED]
So I'm
[OBSCURED]
Yeah?
In context do you make like rendering the animation or it seems like there would be a blow to whatever you're doing on the SPUs
Basically the SPUs are taking this channel animation data and baking it out into-- well, in the easiest case baking it out into a 4 by 4 matrix per joint
So the output time's much bigger than the input time?
I mean, you're compressing the input by animation?
No, the output size is significant, but it's much smaller
[OBSCURED]
So the animation data it's [OBSCURED] [INTERPOSING VOICES]
No.
I was just outputting the joint information
[OBSCURED]
Independently we have this skimming problem.
Independently there's a rendering problem.
This is just baking animation.
This is purely animation channel problem
[OBSCURED]
OK, I'm just going to skip through this because this could take a long time to talk about.
Basically, what I wanted to say here was let's take the next step with animation, let's add some dynamic support.
The easiest thing to do is just create a second uniform data table that you then blend with the first one and that one.
In principle, is basically all of the channels and then now a game play programmer can go and individually set any of those.
So they can tweak the head or tweak the elbow or whatever.
And that's definitely compressible because it's very unlikely thet're going to be moving all the joints at once.
You can create a secondary map that says, this is the number of joints that are dynamic, this is how they map to the uniform values.
But then once you add any kind of dynamic support, you have now complicated the problem significantly.
Because now in reality, what you need are constraints.
You need to be able to have a limit to how high the head can move because what's going to happen is although you could just say the head could only move so much, if that movement is algorithmic, so let's say follow a character or whatever-- it is going to go outside of reasonable constraints really quickly.
So it's much cleaner and simpler to support that on the engine side, so basically define constraints for the joints and then let the high-level code point wherever they want
[OBSCURED]
Yeah.
Yeah, you can have max change over time so it only can move so fast.
The max range of motion, the max acceleration is actually a much harder problem because it implies that you need to store the change over time, which we're not actually storing.
Which would probably blow our memory on the SPU.
So as far as impacting animation, I would immediately throw out max acceleration if an animator were to come to me and say, this is a feature that I wanted.
I would say, it's unlikely because it's unlikely we can fit it on the SPU.
Whereas, on the PC, it might be a different story.
And blending information, how you blend these things together.
What's that?
[OBSCURED]
OK.
So as far as mixing, there's plenty of additional problems in mixing animation.
Phase matching, so for example, you have a running and a walk.
Basically all that means is if you were going to blend from a run to a walk you kind of want to blend in basically the essentially same leg position.
Because if you just blend from the middle of an animation to the beginning of the animation, it's unlikely the legs are going to match and for the transition time you're going to see the scissoring of the legs.
Which you see that in plenty of games, but especially in next generation, especially as characters look more complicated they are expected to act more complicated.
Transitions handling either programmatic transitions between animations, so we have an animation that's standing and animation that's crouching and with constraints, move them down; or artist driven animated transitions and/or both.
Translation matching is actually an interesting problem.
So you have an animation that's running and you have an animation that's walking.
They both translate obviously, at different speeds, nonlinearly and you want to slowly run down into a walk, but you have to match these sort of nonlinear translations as his feet are stepping onto the ground.
Turns out to be a really difficult problem to get perfectly right, especially if you have eye key on the feet where he's walking on the ground or maybe walking uphill or downhill and the translation is being affected by the world.
In a lot of cases you'll see people pretty much just ignore this problem.
But it is something to consider going forward and this is something that we would consider how to solve, regardless of whether or not we could get it in.
As far as actually rendering the geometry goes, you now have your sort of matrices of joints and you have-- let's say you want to send those to the GPU along with the geometry to skin and render.
Now the question is, do you single or double buffer those joints?
Because right now basically, the GPU can be reading these joints in parallel to when you're actually outputting them.
So the traditional approach or the easiest approach is just to double buffer the joints.
So just output into a different buffer that the R6 is reading from.
It's one frame or half a frame behind, doesn't much matter.
But it also doubles now the space of your joints.
One advantage that games have is that a frame is a well defined element in the games.
We know what needs to happen across the course of a frame.
So these characters need to be rendered, the collisions of this background needs to happen, physics need to happen here.
So you can within a frame, set it up so that the update from the SPUs and the read from the GPU can never overlap.
Even without any kind of synchronization or lock, it can be a well known fact that it's impossible for these two things because there's actually something in the middle happening that has its own synchronization primitive.
That will allow you to do single buffering of the data.
But it does require more organization.
Especially if you're doing it on more than just one case.
So you have all these things that you want single buffered, so you need to organize them within the frames so they're never updating and reading at the same time.
So I'll make this the last point I'll make.
Optimization, one of the things that you'll hear, save optimization till the end.
My point here being is if you save optimization till the end, you don't know how to do it because you haven't actually practiced it.
If you haven't practiced it you don't know what to do.
So it will take much longer.
You should always be optimizing in order to understand, when it actually counts, what to do.
And the fact that real optimization does impact the design all the way up.
Optimization of the hardware impacts how an engine is designed to be fast does impact the data, it impacts how game play needs to be written, high-level code needs to be called.
So if you save optimization till last, what you're doing is completely limiting what you can optimize.
And the idea that it's the root of all evil certainly didn't come from a game developer, I have to say.
Anyway, that's it.
I hope that was helpful
Any questions?
I think it's very interesting because there is a lot of things you learn at MIT.
Forget everything you learned so I think there's a very interesting perspective in there and for some of us it's kind of hard to even digest a little bit, but Question?
Call of Duty 3 came out on the Xbox and on the PS3, is Call of Duty 3 on the PS3 just running on the GPU then or is it--
No, it's very likely using the SPUs.
I mean, I don't know.
I haven't looked at the source code, but I suspect that it's using the SPUs.
How efficiently it's using them is an entirely different question.
But it's easy to take the most trivial things right, say you do hot spot analysis on your sequential code and say, OK, well I can grab this section of thing and put it on the SPU right and just the heaviest hitters and put them on the SPU.
That's pretty easy to do.
It's taking it to the next level though, and to really have sort of the next gen of game-- now there's nowhere to go from there.
There's nowhere to go from that analysis, you've already sort of hit the limit of what you can do with that.
It has to be redesigned.
So I don't know what they're doing honestly and certainly I'm being recorded so-- yeah?
You guys have shipped a game, on the PS3?
Yeah, it was on action list
OK, so that was more like the [OBSCURED] games and whatever You seem to talk a lot about all these things you've had to redo.
What else is there-- games look better as a console was built on, what else is there that you guys plan on changing as far as working with the cell processor, or do you think you've got it all ready?
Oh, no.
There's plenty of work.
There's plenty more to be optimized.
It's down to cost in scheduling those things.
I mean, we have a team of people who now really understand the platform and whereas a lot of what went into previous titles was mixed with learning curve.
So there's definitely a potential for going back and improving things and making things better.
That's what a cycle of game development is all about.
I mean, games at the end of the lifetime of PlayStation 3 will look significantly better than release titles.
That's the way it always is
The head of Sony computer and gaming said that PS3 pretty soon would be customizable.
You're be able to get different amounts of RAM and whatnot
Well, I think in that case he was talking specifically about a PS3 based, like Tivo kind of weird media thing, which has nothing to do with us
[OBSCURED]
We're not stuck.
That's what we have.
I mean, I don't see it as stuck.
I would much rather have the-- I mean, that's what console development is about, really.
We have a machine, we have a set of limitations of it and we can push that machine over the lifetime of the platform.
If it changes out from under us, it becomes PC development
Are you allowed to use the seven SPUs or are you-- [OBSCURED]
[OBSCURED]
I don't know how much I can answer this just from NDA point-of-view.
But let's say hypothetically, there magically became more SPUs on the PS3, right?
Probably nothing would happen.
The game has to be optimized for the minimum case, so nothing would change.
Anything else?
Yeah?
So what's the development life cycle like for the engine part of the game.
And I don't assume you start by prototyping in higher-level mechanisms.
Then you'll completely miss the design for performance aspects of it.
How do you build up from empty [OBSCURED]
No, you don't start with an empty.
That's the perspective difference.
You don't start with code, code's not important.
Start with the data.
You sit down with an artist and they say, what do you want to do?
And then you look at the data.
What does that data look like?
What does this animation data look like?
Data size matters.
[OBSCURED]
Right.
We have to figure out how to make it smaller.
But it all starts with the data.
It all starts with that concept of what do we want to see on the screen?
What do we even want to hear on the speakers?
What kind of effects do we want.
And actually look at that from the perspective of the content creator and what they're generating and what we can do with that data.
Because game development is just this black box between the artists and the screen.
We're providing a transformation engine that takes the vision of the designers and the artists and just transforming it and spitting it on to screen.
So where you really need to start is with the source of the data
You've been doing game developoment for 11 years now is what you said.
Have you had a favorite platform and a nightmare platform?
I've been pretty much with the PlayStation platform since there was a PlayStation platform and I don't know, it's hard to get perspective because you're getting it and you always really love plarform you're working on.
So it's hard.
I mean, it's hard to get perspective.
In the program where I am today is not the same program where I was 10 years ago.
Personally, right now my favorite plarform is PS3
So when put it in perspective, there are already some platforms that on the first time round, [COUGHING], it's like cost of development.
So one platform that as time goes we have [OBSCURED]
Well, like with the PS3, some of the things that I like about the PS3, which is sort of a different question are the fact that the cell is much more public than any other platform has ever been.
With IBM's documentation, with Toshiba's support and Sony support, I've never had a platform where I can get up on a website and actually talk about it outside of NDA.
And that for me is an amazing change.
Where I can go and talk to other people-- exactly like this group here- that have used the same exact platform I've used.
That's never been able to happen before.
Even on PS2, for quite a long part of the lifespan, even though there was a Linux eventually on the PS2, virtually everything was covered by NDA because there was no independent release of information.
So that's one of the great things about PS3 is the public availability of cell
Thank you very much for coming all the way from California and giving us some insight.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so today's the last lecture day we're going to talk about the raw architecture.
This is a processor that was built here at MIT and essentially trailblazed a lot of the research in terms of parallel architectures for multicores, compilation for multicores, programming language and so on.
So you've heard some things about RAW and the parallelizing technology in terms of StreamIt.
So we're going to cover some of that again here today just briefly and give you little bit more insight into what went into the design of the raw architecture.
So these are RAW chips they were delivered in October of 2002.
Each one of these has 16 processors on it.
I'm going to show you sort of a diagram on the next slide.
It's really a tiled microprocessor.
We'll get into what that means and how it actually-- what does a tiled microprocessor give you that makes it an attractive design point in the architecture space?
Each of the raw tiles-- you can sort of see the outline here sort of replicates-- is four millimeters.
It's four millimeters square.
It's a single-issue 8-stage pipeline.
It has local memory, so there's a 32K cache.
And the unique aspect of the raw processor is that is has a lot of on-chip networks that you could use to orchestrate communication between processors.
So there's two operand networks.
I'm going to get into what that means and what they used for.
But these eventually allow you to do point-to-point communication between tiles with very low latency.
And then there's a network that essentially allows you to handle cache misses and input and output and one for message passings, a more dynamic style of messaging, something similar to what you're accustomed to at the cell, for example, in DMA transfers.
This was built in 180 nanometer ASIC technology by IBM.
It's got 100 million transistors.
It was designed here by MIT grad students.
It's got something like a million gates on it.
Three to four years of development time.
And what was really interesting here is that this was-- because of the tiled nature of the architecture, you could just design one tile and then once you have one tile, you essentially just plop down more and more and more of them.
And so you have one, you scale it out to 16 tiles.
And the design sort of came back without any bugs when the first chip was delivered.
The core frequency was expected to run at 425-- I think lower than 425 megahertz
Designed for 250?
250 megahertz and came back and it ran 425 megahertz.
And it's been clocked as high as 500 megahertz at 2.2 volts.
The chip isn't really designed for low power but the tile abstraction is really nice for power consumption because if you're not using tiles you can essentially just shut them down.
So it'll allow you to sort of have power efficient design just by nature of the architecture.
But when you're using all the tiles, all the memories, all the networks, in a non-optimized design, you consume about 18 watts of power.
So how do you use this tiled processor?
So here's one particular example.
The nice thing about tile architecture is that you can let applications consume as many tiles as they need.
If you have an application with a lot parallelism then you give it a lot of tiles.
If you have an application that doesn't need a lot of parallelism then you don't give it a lot of tiles.
So it allows you to really exploit the mapping of your application down to the architecture and gives you ASIC-like behavior-- application specific processing technology.
So one example is you have some video that you're recording and you want to encode it and stream it across the web or display it on your monitor or whatever else.
So you can have some logic that you map down.
If your chips are here, you do some computation.
You have memories sprinkled across the tile that you're going to use for local store.
So you can parallelize, for example, the motion estimation for encoding the temporal redundancy in a video stream.
You can have another application completely independent running on other part of the chip.
So here's an application that's using four different tiles and it's really isolated.
It doesn't affect what's going on in these tiles.
You can have another application that's running something like MPI where you're doing dynamic messaging, and httpd server and this tile is maybe not used so it's just sleeping or it's idle.
You can have memories connected off the chip, I/O devices.
So it's really interesting in the sense that probably the most interesting aspect of it is you just allow the tiles to sort of be used as your fundamental resource.
And you can scale them up as your application parallelism scales.
This is a picture of the raw board-- the raw motherboard.
Actually you see it in the Stata Center in the Raw Lab.
This is the raw chip.
A lot of the peripheral device, firmware and interconnect for dealing with a lot of devices off the chip are implemented in these FPGAs, so these are Xilinx chips.
There's DRAM.
You have connection to a PCI card, USB stick.
Network interface so you can actually log into this machine and use it.
And there's a real compiler.
It can run real applications on it.
There's actually a bigger chip that we built where we take four of these raw chips and sort of scale them up.
So rather than having 16 tiles on your motherboard, you can have four raw chips.
That gives you 64 tiles.
You can scale this up to a thousand tiles or so on.
Just because of the tile nature, everything is symmetric, homogeneous, so you can really scale it up really big.
So what is the performance of raw?
So looking at the overall application performance, so we've done a lot of benchmarking.
So these are numbers from a paper that was published in 2004, where we took a lot of applications-- some are well-known and used in standard benchmark suites-- and compiled them for raw using various raw compiler that we built in-house.
And we've compared them against the Pentium 3.
So the Pentium 3 is sort of a unique comparison point because it sort of matches raw in terms of the technology that was used to fabricate the two.
And what you're seeing here, this is a lock scale.
The speedup of the application running on raw compared to the application running on a P3.
So the higher you get, the better the performance is.
So these applications are sort of grouped into a few classes.
So the first class is what we call ILP applications.
So these are applications that have essentially instruction level parallelism.
I'm going to talk a little bit more about and sort of explain it.
But you've seen this early on in the lecture-- in some of Saman's lectures.
So here you're trying to exploit inherent instruction level parallelism in the applications.
And if you have lots of ILP then you map it to a lot of tiles and you can get parallelism that way and you get better performance.
These applications here-- what we call the streaming applications.
So you saw some of these in the StreamIt lecture and the StreamIt parallelizer compiler.
Some of those numbers were generated on a raw-like architecture.
And then you have the server or sort of more traditional applications that you expect to run in a server style or throughput-oriented.
And then finally you have bit-level applications.
So doing things at the very lowest level of computation where you're doing a lot of bit manipulation.
So what's interesting here to note is that as you get into more applications that have a lot of inherent parallelism in them, where you want explicit-- where you can extract a lot of parallelism because of the explicit nature of the applications-- you can really map those really well through the architecture.
And because of the communication nature-- because of communication capabilities of the architecture, being able to stream data from one tile to another really fast, you can get really high on-chip bandwidth and that gives you really high performance, especially for these kinds of applications.
There are other applications that we've done.
Some of the students have worked on in the raw group.
So an MPEG-2 encoder where you're essentially trying to do real-time encoding of a video screen at different resolutions.
So 350 by 240 or 720 by 480 where you're compiling down to a number of tiles.
One, 4, 8 sixteen, 16-- 1 and 16 are somehow missing, I'm not sure why.
And what you're looking for here?
Sort of scalability of algorithm.
As you add more tiles, are you getting more and more performance or are you getting better and better throughput?
So you could encode more frames per second for example.
So if you're doing HDTV, it's 1080p, then you really want to sort of get-- there's a lot of compute power that you need.
And so as you add more frames, maybe you can get to sort of the throughput that you need for HDTV.
So this is something that might be interesting for some of your projects as well.
And we've talked about this before.
On the cell, as you're using more and more SPEs, can you accelerate the performance of your application?
Can you sort of show that if you're doing some visual aspect?
And you can sort of demonstrate it.
So there's a demo that is set up and in the lab where you can sort of crank up number of tiles that you're using and you get better performance from the MPEG encoder.
And just looking at number of frames per second that you can get, with 64 tiles-- so the raw chip is 16 tiles, but you can scale it up by having more chips-- so you can get about 51 frames.
These numbers have been improved and there are different ways of optimizing these performances.
At 352 by 4 240, the estimated data rate-- estimated throughput-- of 160 frames per second almost.
So this is really high bandwidth.
Another interesting thing that we've done with the raw chip is taking a look at graphics pipelines and looking at is there anything we can do to exploit the inherent tiled architecture of the raw chip.
So here's a screenshot from Counterstrike and a simplified graphics pipeline where you have some input to the screen you want to render.
You do some vertex shading.
So these are triangles that you want to figure out what colors to make-- what colors to paint them.
The triangle's set up for pixel stage.
And in this screen you'll notice that there are two different things that you're rendering.
There's essentially this part of the screen which has a lot of triangles that span a relatively not-so-complex image.
And then you have these guys here that have fewer triangle span a smaller region of the frame.
And what you might want to do is allocate more computer power to the pixel stage and less compute power to the vertex stage.
So that's analogous to saying, I want more tiles for one stage of the pipeline and fewer tiles for another.
Or maybe I want to be able to dynamically change how many tiles I'm allocating at different stages of the pipeline.
So that as your screens that you're rendering change in terms of their complexity, you can maintain the good visual illusions transparently without compromising the utilization of the chip.
So some demos that were done with the graphics group it at MIT-- Fredo Durand's group-- phong shading.
You have 132 vertices with 1 light source.
So this is what you're trying to shade.
You have a lot of regions black.
So if you're looking at a fixed pipeline where the vertex shader is taking six tiles-- this is on a 64-tile chip-- the rasterizer is taking 15 tiles, the pixel processor has 15 tiles, the alpha buffer operations has 15 tiles, then you might not get the best utilization because for that entire region that you're rendering where it's black there's nothing really interesting happening there.
You want to shift those tiles to another processor, to another stage of pipeline.
Or, if you can't really utilize them, then you're just wasting power, wasting energy, and so you might just want to shut them and not use them at all.
So with a fixed pipeline versus a reconfigurable pipeline where I can change the number of tiles allocated to different stages of the pipeline, I can get better utilization.
And, in some cases, better performance.
So here, fuller bars, and you're finishing faster in time.
So this is indicative also of what's going on in the graphics industry.
So the graphics card used to be very-- well, it had fixed resources allocated to different stage, which is essentially what we're trying model in this part of the experiment, where more and more now you have unified shaders that you can use for the pixel shading and the vertex shading.
So you're getting into more of that programmable aspect.
Precisely because you want to be able to do this kind of load balancing and exploit dynamisms that you see in different things that you're trying to render.
Another example: shadow volumes.
You have 4 triangles, one light source.
And this was rendered in three passes.
So pass 1, pass 2, pass 3, would essentially take the same amount of time because you're doing the same computation map to a fixed number of resources.
But if I can change the number of resources that I need for different passes-- so the rasterizer, for example, and the alpha buffer operations, is really where you need a lot of power.
So if you go from 15 tiles for each to 20 tiles for each, you get better execution time because you were able to exploit parallelism or match parallelism better to the application.
And so you get 40% percent faster in this particular case.
And another interesting application: this is the largest in the world microphone array.
It's actually in the Guinness Book of Records.
It was build in the lab.
And what it essentially has-- each of these little boards has two microphones on it.
And so what you can use this for is eavesdropping for example.
Or you can carry this around if you want.
Pack it in the car and do some spying.
But somewhat more interesting demos that were done with this in smaller scales was that in a noisy room, for example, if you want the sort of hone in.
Let's say everybody here was speaking, but for the camera they want to record only my voice.
They can have a microphone array in the back that focuses on just my voice.
And the way it's done is you can measure the distance from the time it takes for the sound wave to reach each of these different microphones and you can focus in on a particular source of noise and be able to just highlight that.
So there's this demo where's it's a noisy room-- I probably should have had these in here in retrospect-- there's a noisy room, lots of people are talking, then you turn on the microphone array and you can hear that one particular source and it's a lot clearer.
You can also have applications where you're tracking a person in a room with videos as well, so you can sort of follow him around.
So it's a very interesting application.
An now I regret not having the video demo in here.
Actually, should I do it?
It's on the Web.
OK.
So a case study using the beamformer.
So what's being done in the microphone array is you're doing beamforming.
So you're trying to figure out what are the different beams that are reaching the microphone.
You want to be able to amplify one of them.
So looking at the application written natively in C running on a 1 gigahertz Pentium , what is the operation throughput?
So you're getting about 240 MegaFLOPS.
And if you go down to an optimized-- same code but running on single tile raw chip, you get about 19 MegaFLOPS.
So a not very good performance.
But here, what you really want to do, is you have al lot of parallelism.
Because each of those beams that's reaching individual microphones can be done in parallel.
So you have a lot of parallelism in that application.
So taking the C program, reimplementing it in StreamIt that you've seen in previous lectures, and not really optimizing it in terms of doing a lot of the optimizations you saw in the parallelizing compiler talk, you get about 640 MegaFLOPS.
So already you're beating the C program running on a pretty fast superscalar machine.
And if you really optimize the StreamIt code in terms of doing the fission and fusion, increasing the parallelism, doing better load balancing automatically, you can get up to 1.4 GigaFLOPS.
So really good performance and really matching the inherent parallelism to the architecture.
So it was just a big overview of the raw chip and what we've done with it in lab.
There's more in here than I've talked about.
But what I'm going to do next is give you some insights as to what is the design philosophy that went into raw architecture, why was it designed the way it was.
And then I'm going to talk a little bit about the raw parallelizing compiler.
And while the StreamIt language and compiler also has a back end for the raw architecture, we've sort of seen that in previous lectures so I'm not going to talk about that here.
So I'm just going to focus on the first two bullets.
And a few years ago when the project got started, sort of the insight in the wide issue processors and the design philosophy that was being followed in industry for building wider superscalars, faster superscalars, was really going to come to a halt largely because you have scalability issues.
So if you look at sort of a simplified illustration of a wide issue microprocessor, you have your program counter such as instructions.
Goes into some control logic.
Control logic is then going to run.
You're going to read some variables from the register file.
You'll have a big crossbar in the middle that routes to operands like ALUs.
Yell And then you operate on those and you have to send it back to the register file.
Plus you have this really big problem with the network.
So if you're doing some computation-- sorry, I rearranged these slides.
So what you have if you have n ALUs, then the complexity of your crossbar increases as n squared, because you essentially have to have everybody talking to each other.
And in terms of the number of wires that you need out of the register file to support everybody being able to sort of talk to anybody else very efficiently, the number of ports, the number of wires increases n cubed.
So that's a problem because you can't clock all those wires fast enough.
The frequency becomes sort of limited.
It grows even less than linearly.
And this is a problem because operational routing-- operand routing, is global.
So if I have- I'm doing some operations and it's an add, the results of this add is fed to another operation to shift, and these are going to execute on two different ALUs.
So what's going to happen?
I do the add operation.
It's going to produce a result.
But there's no direct path for this ALU to send this result to this ALU.
So instead what has happened is the operand has to travel all the way back around through the crossbar and then back to this ALU.
So that's really just going to take a long time and not necessarily very efficient.
And if you're doing this for a lot of ALU operations, you have a lot of parallelism in your application level, instructional level parallelism, and that's just creating a lot of communication.
But you're not really exploiting the locality of the computation.
If 2 instructions are really close together, you want to be able to just have a point-to-point path, for example, or a shorter path that allows you to exploit where was instructions are in space.
And so this was the driving insight for the architecture in that you want to make operand routing local.
So an idea is to essentially exploit this locality by distributing the ALUs.
And rather than having that massive crossbar, what you want to do is have an on-chip mesh network.
So rather than have one big crossbar, you have lots of smaller ones.
So these become switch processors.
So I can put value from this ALU here and then have that value routed to any other ALU.
Maybe that just cost me more in terms of instructions that says where this operand is going.
We'll get into that.
But here, what this allows me to do is exploit that locality better.
Same instruction chain, I can put the first operation on one ALU, I can put the other operation on the second ALU.
And here, rather than putting it for example here, which would send the operand really far across chip, what I want to do is recognize that there's a producer/consumer relationship here.
I want to exploit that locality and have them close in spaces so that the routes remain fairly short.
You know what I can also do is sort of pipeline this network so that I can have the hardware essential match computation flow.
If one ALU is producing a lot of results at a lot faster rate than for example this instruction can consume them, then the hardware can take care of, for example, blocking or stalling the producing processor so it doesn't get too far ahead.
It gives you a nature mechanism for regulating the flow data on the chip.
Well, this is better than what we saw before because with the crossbar you're not really getting any scalability in terms of your latency transport operands from one ALU to another.
Whereas with on-chip network, if you've taken routing classes, you know that there exists an algorithm that sort of allows it to route things at least the square root of n, where n is the number of things that are communicating in your network.
But if you're doing locality driven placement then it's essentially costing time.
And in a raw chip, it's in fact three cycles.
So you can send one operand from one tile to another in three cycles.
And we'll get into how that number comes about.
So this is much better.
But what it does is increase the complexity on the compiler.
It says, this is my computation, how do you map it efficiently so that things are clustered in space well so that I don't have these really long routes for communication?
But then we can look at what else can we distribute.
Well, we have the register file.
We can distribute that across all the ALUs.
And that essentially decreases that n cubed relationships between ALUs and register file ports to something that's a lot more tractable.
Where it's one small register per ALU.
And this is better in terms of scalability, but we haven't solved the entire problem in that we still have one global program counter, we have one global instruction fetch unit, one global control unified load/store queue for communicating with memory.
And those all have scalability problems.
So whereas we fixed the problem with the crossbar-- that becomes more scalable-- we haven't really fix the problems with the others.
So what's the natural solution to do here?
Well, we'll just distribute everything else.
And so you start off with each ALU here now having it's own program counter, its own instruction cache, it's own data cache.
And it has its register file ALU and everybody-- that same sort of design pattern is repeated for each one of those ALUs.
So now it looks like it's a lot scalable.
I don't have any global wires.
There's no global centralized data structure.
And all of that means I can do things more-- I can do things faster and more efficiently.
And what you start seeing here is this sort of tile processor coming about all.
So each one of those things was exactly the same.
And what was done in the raw processor is that none of those tiles was longer than you can communicate in one clock cycle.
So this solved essentially a wire delay problem as well.
So if this is the distance that a wire-- that a signal can travel in one clock cycle, the tile is smaller.
It can fit within this circle.
So that means that you're guaranteed-- you have better scalability problems.
You're solving the issues that people are facing with wire delay.
And in terms of the tile processor abstraction, Michael Taylor was is a PhD student in the raw group, his thesis sort of identified the tile processor approach and this aspect of the tile processor approach that makes it more attractive, the SON.
Which is the scalar operand network.
And the next two slides, the next part of the lecture, is going to really focus on what that means.
He argues why the tile processor approach is scalable.
And it's scalable for the same reasons as multicores.
You just add more and more cores on a chip.
But the intrinsic difference between the multicore that you see today and the raw architecture is the scalar operand network.
So I'm going to ask you questions about this in a few slides.
But really what you're getting here is the ability to communicate from one processor to another very efficiently.
And the way you do this on raw is you have your instruction fetch d code, register file read stage, WALU-- your competition pipeline.
But part of the registers-- the new register file-- so 24 through 27 are network mapped.
So what that means is, if I write-- if one of the operations that I have in my computation has a destination register that's 24, 25, 26 or 27, that value automatically get sent to the output network.
And if I have a value-- if one of my source operands is registered at 24, 25, 26 or 27, implicitly that means get that value off the network.
And so I can have add 25-- added to register 25-- so this is one of the network map ports, sum two operands.
So this is a picture of the raw chip.
This is one tile.
This is the other tile.
So you can sort of see the computation and the network switch processor here.
So the operand flows into the network and then gets transported across from one tile to the other.
And then gets injected into the other tiles compute networks.
And here this instruction has sort a source operand that that's register map operand.
So it knows where to get its value from.
And then you can do the computation.
An interesting aspect here is that while you've seen instructions like this, just normal instructions, here you also have explicit routing instructions that are executed on the switch processor.
So the switch processor here says take the value that's coming from my processor and send it east.
So each processor can send values east, west, north or south.
So it can go to the tile above it, the tile below it, the tile to the left of it or tile to the right of it.
And so sending it east sends it along this wire here.
And then this particular switch processor says get a value from the west port and send it to my processor.
Now you could have had here, this process could say, this value is not for me, so I want to just pass through to some other processor.
So you can pass it from the west port to the south port or to the north port or just pass it through laterally to the other east port.
So it just allows you to essentially just have an on-chip network and not operand-- you can imagine having an operand that has a data packet and header that's says, I'm going to tile 10 and the switches know which way to send it.
But the interesting aspect here is that the compiler actually orchestrates the communication, so you don't need that extra header that says, I'm going to tile 10.
You just have to generate a schedule of how to write that data through.
So we'll get into what that means for the compiler in terms of that added complexity.
So communication on multicores is expensive for the following reasons.
And this is really sort of going contrast or going to put the scalar operand network into slightly more perspective.
But first, so how do you communicate between multicores on the cell?
You have the DMA transfers from one SPE to another.
You can't really ship an operand single value.
So if I write the value x, and I want to send x from one SPE to another, I can't really do that very efficiently, right?
So this is essentially the contrasting thing between multicore processors that largely exist today and the raw processor.
So I've shown you an empirical-- a quantitative-- an analytical model for communication costs before in earlier slides.
This is an illustration of that concept.
So if I have a processor that's talking to another, that value has to travel across some network and there's some transport costs associated with that.
But there's also some added complexities.
So there were lots of terms, if you remember, in that really big equation I've shown before.
You have some overhead in terms of packaging the data.
And you have some overhead in terms of unpacking the data.
So what does that look?
Well, there are two components we're going to break this down to: the send occupancy and send latency.
And I'm going to talk about each of those.
And similarly on the receive side, you have the receive latency and the receive occupancy.
So bear in mind, this lifetime of a message essentially has to flow through these five components.
It has to go through the occupancy stage, then there's the send latency, transport, receive latency and receive occupancy before you can actually use it to compute on.
So what are some things that you do here?
Well, it's things that you've done on cell for getting VME transfers to work.
You have to figure who the destination is, what is the value, maybe you have an idea associated with it, a tag, things of that sort.
And you have to essentially inject that message into the network.
So there's some latency associated with that.
Maybe your-- on cell you have a DMA engine which essentially hides this latency for you.
Because you can essentially just send the message to the DMA, right into its queue.
And you can especially forget about it unless it stalls because the DMA list is full.
On the receive side, you sort of have a similar thing.
You have to get the network to inject that value into the processor and then you have to depackage it, demultiplex it and put it into some form that you can actually use to operate on it.
So this 5-tuple is gives us a way of sort of characterizing communication patterns on different architectures.
So I can contrast, for example, raw versus the traditional microprocessor.
So this is a traditional superscalar.
A traditional superscalar essentially has all the sophisticated circuitry that allows you to essentially bypass network.
You can have an operand directly flowing to another ALU through all the n squared wires in the crossbar.
And a lot of dynamic scheduling is going on.
So it really has no occupancy, latency, you're not really doing any packaging of the operands.
Your transport cost is essentially completely hidden.
You have no complexity on the receive side.
So it's really efficient.
So this is essentially what you want to get to go: this kind of 5-tuple.
But as we saw before, it's really not scalable because the wire complexity woes-- whether it's n squared or n cubed, that's not good from an energy efficient point of view.
Scalable multiprocessors-- these are on-chip multiprocessors more indicative of things that you have today-- have this kind of 5-tuple where you have about 16 cycles just to get a message out, know roughly 3 cycles are so to transport message.
So maybe this is being done through a shared cache.
Which is how a lot of architecture communicates between processors today.
And you have to sort of demultiplex the message on the receive side.
So that adds some latency.
In raw, because you have these net memory map registers on the input side and the output side, you really can knock down the complexity from the send side in terms of the occupancy and latency to zero.
And you just write the values to the register.
And it looks like a normal register, right?
But it just magically appears on the network.
And then from one tile to another, it's one cycle to ship the value across that one link from one switch processor to the other, as long as it's a near neighbor.
And then two cycles to inject the network into the tile processor.
And then you're ready to use it.
So in this space, where would you put cell is the question?
Anybody have any ideas?
What would the communication panel look like on cell?
So you have to do explicit sends and receives.
So let's look at this.
So can we get rid of this stage on cell which is essentially saying packaging up my message, is it's no, right?
Because you have to essentially say where that DMA transfer is going to go to-- which region of memory?
So you're buildings these control blocks.
And then the send latency here is roughly zero, because you have the DMA processor which allows that kind of concurrency between communication and computation, so you can hide essentially that part of the transport-- that part of communication costs.
Your transport costs here, you have this really massive bandwidth, this really high bandwidth interconnect on the chip.
So this makes it reasonably fast, but it's still a few cycles.
There's no near neighbor?
Yeah, a hundred cycles to go near neighbor communication.
Because you're still-- you don't have that fast mechanism of being able to send things points to point.
You're putting things on the bus and there's some complexity there.
On the receive, you have the same kind of complexity that you had on the send side.
You have to know that the message is coming, that can be done in different ways.
And then you have to take that message and write it into your local store.
Which also adds some overhead in terms of the communication cost.
So the cell would probably be somewhere up here, I would imagine.
I didn't have a chance to get the numbers.
If I do, I'll update the slide later on.
OK, so that's essentially a brief insight into the raw-- yeah?
Where did you get the scalable processor?
So these are from Michael Taylor's thesis.
So I believe what he's done here is just looked at some existing microprocessor and essentially benchmarked communication latency from one processor to another
So this is like going through the cache on the [OBSCURED]?
That's in fact how you-- a lot of these multiprocessors today have shared caches, either L-1 and more so now it's L-2.
So if you have-- L-1s are dedicated to different processors.
But you still have to go the memory to communicate.
So the raw parallelizing compiler-- yeah?
Another question?
You might want to postpone this question.
Two related questions: so raw has-- I guess raw has pretty well optimized nearest neighbor communication.
But we know from, for example, Red's Rule in heuristic and intellectual engineering about the number of wires needed for a given area.
Is that in between-- as I recall, it's the minimum for a good sized circuit is proportional to the perimeter, or roughly the square root of the area.
And it ranges from there to-- not proportional to the area.
There's something in between.
Something with 3 in it.
Like to the 3/2 power I think, perhaps.
No, something like 2/3rds, something like-- yeah, 2/3rds power.
So the area to the 1/2 power or area to the 2/3rds power.
So Red's Rule says the number of wires you need is roughly in that area.
And so that sort of pushes that-- so the minimum you need is the nearest communication.
And often you need more than that.
We know from the FPGA experience that nearest neighbor communication is not-- or, at least, it's good to have move than nearest neighbor, and that often long wires followed across the chip, in extremely high--
So I'm going to actually show you an example where nearest neighbor is good but you might also want some global mechanism for control orchestration for example
Not just for con-- not surely just for control but for broadcast, for arbitrary for the computation to use, not just for the chip to use.
Like why are you scaling out two hops, four hops, fewer and fewer wire--
Yes, in fact what I think is going to happen is a lot of these chip designs are going to be heirarchical.
You have some really global type communication at the highest level.
And then as you get within each one of the processors, then you see things at the lowest level, something that looked like raw.
So you can build sort of a hierarchy of communication stages that allow you to sort of solve that problem.
But all of that adds complexity, right?
First you have to solve the problem of how do you parallelize for just a fixed number of cores and then figure out the communications.
Once we understand how to do that well with a nice programming model then you can build heirarchically on that
On the other hand, it might make the compiler's job easier because it's not as constrained
It might give you a nice fall back rate.
It might save you in cases where there are things that are hard to do.
There are some issues in the last two-- the second to the last three slides.
We'll talk about an example of where that might be the case
Another question which [OBSCURED] so raw, I guess, being simpled and tiled, I guess one of the selling points I think was that it really cuts down on the engineering effort
Oh, absolutely.
This was done a million gates in-house for [OBSCURED]
So a company like Intel has a ridiculous number of engineers.
And to get a competitive edge, they something they want to apply more engineering to it.
And so the question is, where might you apply more engineering to try to squeeze more--
That's the million dollar question that everybody's looking at.
Because if somehow Intel thought they could add more and more engineering.
And then build this very complex full-scale [OBSCURED] But separate vessels.
And so I think there's still a lot of things that is wrong.
Meaning it's [OBSCURED] so at Intel basically they will let you do something like that.
They will put a lot of engineers doing each of these components, finding very few, and they can get a lot more performance, a lot less power and stuff like that.
So depending on what you want, science is not everything.
There are a lot of other things [OBSCURED] So while it makes it easier?
[OBSCURED] And the key thing is, you start something simple and as you go on, you can add more and more complexity.
Just, as there's more things to do
Part of the complexity might be going to-- not making all those [OBSCURED].
OK, so raw pushes a lot of the complexity into the compiler in that the compiler now has to do at least two things.
It has to distribute the instructions.
You have a single program and you have to figure out how to parallelize it across multiple cores.
But not only that, because you have the scalar operand network, you have to figure out how the different cores have to talk to each other.
So you have to essentially generate schedule for the switch processors as well.
So I'm going to talk a little bit about the raw paralyzing compiler.
And this is different from a StreamIT parallelizing compiler which really talks about a different program as an input, using a different language.
This is work again done here at MIT by Walter Lee who graduated two years ago.
We have a sequential program.
You inject it into raw C seed, raw C compiler, and you get fine-grained Orchestrated Parallel execution.
And what the compiler does is worry about data distribution just like you have to do on cell in terms of which memory goes into which local store.
which competition operates on-- the raw compiler has to worry about which computation operates on which data element and can you put that data in the right caches for each of the different tiles.
Instruction distribution: so the way this compiler essentially get parallelism, it's going to look at instruction level parallelism in your application.
And it's going to divide that up among the different cores.
And then the last step is the coordination of communication in control flow.
So I'm just going to briefly step through each one of those.
So the data distribution really has essentially trying to solve the problem of locality.
You have two instructions.
A load into r1 from some address and then you're adding r1.
You're incrementing that value.
And you might write it back for later on.
So where would you put these two instructions?
So to exploit the locality, then you want the data-- if the data is here, then you want these two instructions to be on this tile.
If the data is here, then you want these two instructions to be on this file.
Because it doesn't help you to have the data here and the instructions here.
Because what do you have to do in that case?
You have to send a message that says, send me this data.
And then you have to wait for it to come in and then you have to operate on it.
And then maybe you have to write it back.
So the compiler sort of worries about the data distribution.
It applies some data analysis.
A lot of a thing that you saw in Saman's lecture on classic parallelization technology.
Sort of figure out the interdependencies and then they can figure out how to split up the data across the different cores.
And there's some other work done by other students in the group that tried to address this problem.
The instruction distribution is perhaps as complicated and interesting.
In here, what's going on is-- let's say you have a base block.
So you take your sequential program.
You figure out what are the different basic blocks of computation that you have and within the basic block you have lots of instructions.
So each one of these green boxes is a particular instruction.
And what you're seeing-- these arrows here that connect the edges-- are operands that you have to exchange.
So you might have-- this is an add instruction.
It requires a value coming from here.
Multiply-- subtract instruction requires values coming in from different areas.
So how would you distribute this across a number of cores-- or across a number of tiles?
Any ideas here?
So you can look for, for example, some chains that are not interconnected.
So you can look for clusters that you can use.
And say, OK, well I see no edges here so maybe I can put this on one tile.
And then maybe I can put some of these instructions on another tile.
Because sort of the communication flow is local.
So maybe one strategy might be, look for the longest single chains so you can keep the communication flow.
And then you apply and make and algorithm, come up with a number of clusters.
Something like that does happen.
And keep in mind from the lectures we talked about the parallelizing compiler, you have to worry about parallelism versus communication.
Some the more you distribute things, the more communication you have to get right.
So here we're showing-- what I'm showing is color mapping from the original instructions in the base block to the same instructions, but now each color essential represents a different cluster or essentially code that would map a different thread.
So blue is one thread, yellow is another, green is another, red, purple, and so on.
But I have to worry about communication between the different colors because they're essentially two different threads.
They're going to run on two different processors or two different tiles.
So those arrows that are highlighted in dark black are communication edges.
They have to explicitly send the operands around.
Right?
So then I might look at the granularity.
What is my communication cost?
What is my computation cost?
And I want to worry about load balancing.
As we saw, load balancing can give you how it can better make use of your architecture and give you better utilization, better throughput.
So you might essentially say, it doesn't-- it's not worthwhile to have these running on a different tile because there's a lot of communication going on.
So maybe I'd want to fuse those together.
Keep the communication local.
And essentially eliminate costly communication.
So there are different heuristics that you can apply.
You can use that 5-tuple.
You can use heuristic space on the 5-tuple to determine when it's profitable to break things up and when it's not.
And then you have to worry about placement.
So you don't quite have this on cell in that you create these SPE threads and they can run on any SPE in the raw compiler.
You can actually exploit the spacial characteristics of the chip in the point-to-point communication network to say, I want to put these two threads on tile 1 and tile 2, where tile 1 and tile 2 are adjacent to each other.
Because I have a well-defined communication pattern that I'm going to use.
And map to the communication network on the chip to get really fast, really low latency.
So you can take each one of these colors, place it on a different tile.
And now you have these wires that are going across these tiles which essentially represent communication.
But now the tile has to worry about, oh, I have to essentially send these on fixed routes.
There's no arbitrary communication mechanism.
So if there's data going from this tile to this tile, it actually has to be routed through a network.
And that might mean getting routing through somebody else's tile.
So the next stage would be communication coordination.
You have to figure out which switch you need to go to and what do you do to get that operand to the right switch which then gets it to the right processor.
So here, I believe the heuristic is to do dimension order routing so you send along the x-dimension and then the y-dimension.
I might have those reversed.
I don't know.
And then finally, now you've figured out your communication patterns, you've figured out your instructions, you do some instructions scheduling.
And what you can do here, because the communication patterns are static, you've split up the instructions so you know when you need to ship data around and how.
You can guarantee deadlock freedom by carefully ordering your send and receive pairs.
So what you see here, every time you see an instruction that needs to ship an operand around, there's the equivalent of a route instruction that has route east, west, north, south.
There's an equivalent route instruction on the other processors.
And that allows you to essentially analyze code and say, OK, I've laid these things out carefully, I've orchestrated my send and receive pairs so I can guarantee, for example, there are no overlapping routes.
Or that there are no deadlocks because one is trying to shift the other while the other is also trying to ship, and they both block on the shared network link.
And finally, you have the code representation.
So this is where you package things up into object files, into essentially things like threads.
And then you can compile them and run them.
Now the question that was posed earlier is, well there's one thing we haven't talked about and that's branching.
This is a sequential program, it executes branches.
And now I have this loop that I've split up across a number of tiles, how do I know who's going to do the branch?
And if one tile is doing the branch, how does it communicate with everybody else?
Or if I'm going to repeat the branch on every file, does that mean I'm redoing too much computation on every other tile?
So control coordination is actually quite an interesting aspect of-- adds another interesting aspect to the parallelization for raw.
So what you have to do is figure out-- there are two different ways you can do it.
Because you have no mechanism for a global message on raw, you can't say, I've taken a branch, everybody go to this program counter.
You essentially have to send either the branch result so one tile can do the comparison, it calculates the condition, and then it has to communicate x to each of the different branches-- to each of the different tiles.
Or every tiles has to essentially just replicate the control and redo the computations.
So every tile figures out what is the condition, what are the conditions for the branch.
They redundantly do that computation and then they can all merge at the same time-- at different times.
So that gives you two ways of doing the branching.
If each tile's doing its own control flow calculation, then they can essentially branch at different times.
But if they're all going to wait for the result to compare, then it essentially gives you points where you have to synchronize.
Everybody's going to wait for the result of the branch.
But the latency could be different.
Because if I'm sending the branch condition to one tile versus another file, and if one's closer than the other.
Then the branch that's closer to me-- the tile that's closer to me will take that branch earlier in time.
So you get sort of the effective of a global asynchronous branching in either case.
Does that make sense?
So, in summary, the raw architecture is really a tile microprocessor.
It incorporates the best elements from superscalars in terms of a really low latency communication network between tiles which really cuts down on the communication costs.
And as we saw, and as probably you've been learning, communication is really an expensive part of parallelization on existing multicore chips.
And it's also getting the scalability of multicores in terms of explicit parallelism but also gives you implicit parallelism because the networks are pipelined and they can give you full control.
So you're trying to get to the point where you have a tile processor with scalar operand network that allows you to do communication with a very low cost.
And it might be the case in the future that these chips will especially be-- more complex architectures will sit on top of these so you'll use these as fundamental building blocks.
And there was the 80 chip multicore from Intel: there have been rumors that that might actually be something like a graphics processor that has something like a scalar operand network because you could communicate with a very fast-- with very low latency between tiles.
And in that article which came out a few months ago was the first time I think that I had seen tile architectures used in literature or in publications.
So I think you'll see more of these kinds of designs pattern appear as people scale out to more than 2 cores, 4 cores, 8 cores and so on, where you could still communication reasonably well with caches.
And that's all I prepared for today.
Any other questions?
And this is a list of people who contributed to the raw project.
A lot of students who are led by Anant and Saman
[OBSCURED] view of what happened in our groups and then how it relates to necessary to what you need.
But this is trying to take it to a much finer grain.
Whereas in Cell, of course, the message has to be large, you can do a lot of coarse grain stuff.
But in raw, you try to do much more fine grain stuff.
But we're going to talk about it the next lecture on the future.
[OBSCURED]
[OBSCURED] Don't you need long wires for the clock
There's no global clock
So you have this network that seems to -- So that the network actually requires handshaking?
Or--
The way you can do is, you can in modern processors, [OBSCURED] so since there's no long wire, you can actually carry the clock with the data.
So in the global world, the switching here would happen when the switching here.
But since there's no big wire connecting, then that's OK.
So you can deal with clock ticking
So this is not going to be not clock drift because--
Yeah, that's clock drift.
One end of the process clock is happening at the global instant time at the other end of the processor.
And since the wires also kind of go in the tree, you can deal with that
Drift meaning ticking at different rates, not just--
Yeah, I know.
Basically I don't think I can go back to it.
It has a skew.
There's a clock skew going in between those
So you don't need synchronizers between the different tiles?
No, we don't need synchronizers because tiles are local.
The clock would bring those tiles.
The clock would bring two things that communicate close enough that it fits it in the cycle.
But for example, if you get it two very far away branches of a tree and then if you try to communicate with them then you have a problem.
Another thing is when the tree goes here, you want to use two different branches it's similar to going down.
So you can compress the process.
So there are all these things.
I mean, modern processors really really destable.
The problem occurs when you try to connect directly from the far end of the branch to something that gets clocked there to something that clocks at a very early end of the branch.
If you're trying to connect those two, then the skew might be too long.
Then you can get into clock trouble
[OBSCURED] I was just worried about this local network.
[OBSCURED]
Another question I had was in the mesh, obviously the processors in the middle have further to get to the I/O devices or to the main memory.
What do you see happening as you get to larger and larger processors?
Are they going to just put more and more local memory on the tile and [OBSCURED] it, or are they going to add extra memory buses on it?
It could be a combination of both.
So it's not just memory, I/O devices.
If you're doing I/O then you might to be placed at a part of the chip that has direct access to an I/O device or very close.
It also comes up in the case of the communication orchestration.
So if this guy is doing the branch then you want him essentially centrally located.
So the best patterns for allocating things is essentially across.
It's like a plus sign where it branches in the middle
But that's not [OBSCURED].
You can make them uniform by everybody equally there.
And a lot of times people have done that simple model with everybody equally there Or you try to take advantage of closeness and stuff like that.
So you can't have both ways.
So anytime you try to make me [OBSCURED] very, very close and fast access, you're are doing it by basically making the other parts to have less resources and less access.
On the other hand, there are a lot of people working on [INAUDIBLE] things that, for example, there's a thing called tree space laser.
So what that does is you put a mirror on top of the tile, on top of the processor.
And each of these-- you can embed a small LED transmitter into the chip.
So basically if you want to communicate with someone, you just bounce that laser on top of that and get it to the right guy.
So there are a lot of exotic things that might be able to solve this thing, technological problem.
But in some case, speed of light-- I don't think an engineer has figured out how to break speed of light.
Unless, of course, people go with quantum computing and stuff like that.
So, I mean the key thing is, you have resources, you have certain data and you just have to deal with it.
Getting nice uniformity has a cost
Yeah, I mean, on the [OBSCURED] that are groups here at MIT who are working on optical networks in the third dimension.
So you have a tile chip plus an optical network in the third dimension which allows you to do things like broadcast much more efficiently.
OK?
I guess we'll take a break here and take a small, three-minute break and then we can go on to the next topic.
The following content is provided under a Creative Common License.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
OK.
So today is the final lecture, and I'm going to talk to about the future.
If I can figure out how to get the slides going.
Predicting the Future is always not a simple thing.
And especially this day and age we are -- things are appearing on the internet, so this is getting recorded and on the internet.
And I'm going to make a few comments that in five years time, I'm going to regret.
What I'll try to do is try to have a systematic process through that.
Because I know you guys are great hackers.
You guys are great programers.
You and any problem, you can solve it.
But as researchers, what's hard, and what's very difficult, is to figure out which problem to solve.
Because as Roderick pointed out, when we started on Raw in about '97, there was a raging debate what to build.
And there's group of people who say we can build bigger and bigger superscalars.
In fact, there was this article, very seminal proceedings that came out, that basically asked many people to contribute, saying what do they think will a billion transistor processor look like?
And about 2/3 of them said, we would have this gigantic, fabulous superscalar.
And about three out of about ten, I think about three groups said that, OK, it would be more like tiled architecture.
The Stanford people and us are the main people who look at that.
So even that short time, 10 years ago, it was not sure what's going to happen.
And so we took a stance and spent 10 years working on that.
We were lucky, more than anything else, that ITV had a little bit better insight, hopefully, but also mostly a larger dose of luck we get here.
So part of that is how do you go about doing this?
So future ia a combination of evolution and revolutions.
Evolution is somewhat easy to predict.
Basically, what you can look at is, you can look at the trends going on and you can extrapolate that trend.
And you can say, if this trend continue, what will happen?
If this trend continue what will happen?
It's very interesting in computer science because one thing I do in 6033 is look at the rate of evolution that happens in computer science, and trying to predict it with things like the rate of [OBSCURED] and stuff like that and you get ridiculous numbers.
So even if it is evolution, in computer science it's very close to revolution.
Because every 18 months things double.
That's almost unheard of.
In a lot of areas, every 18 months you get 1% improvement and people are very happy.
This is doubling.
We have that.
THe other part is revolutions.
If some completely new technological solution just come about and completely change the world.
Change how things happen.
These are a lot harder to predict.
But still it's critically important because some of these things can have huge impact.
And so we will more focus on evolution and if we have at the end we can see if we have some interesting revolutionary ideas.
Paradigm shifts occur in both.
You don't have to wait for a revolution to have something different.
Because things like 2x improvement type things that happens, actually brings paradigm shifts on a very regular basis.
So it's not 1% we are a getting.
We are getting huge evolution changes.
And that can keep changing things.
So what I'm going to talk to you about, a little bit of trends.
Some of it is repetitive.
I spent sometime in the first lecture talking about these trends.
And then we'll look at two different parts.
One is architecture side, what can happen next.
And languages, compilers and tools side.
We'll spend a little bit of time on revolution, and just see that we have group brainstorming.
And then I have this little bit of a my preview of what people should be doing, very broad level, and I'll finish off with that.
So look at trend.
I think there are a lot of interesting trends that we can look at.
Moore's law, which is a very trusting trend that keep going for a very long time.
Power consumption, wire delay, hardware complexity, parallelizing compilers, program design methodologies.
So you have all these trends her to look at.
I have this picture, the graph on the right, which shows what happened at every generation, assume this is a generation.
You concentrate on the hardest problem, what's the most taxing thing?
And what happens is as you go, at this time, something like this might be the taxing, and then that generation, that's where all the design going on.
Then some slowly growing things start catching up, and then, after some time, that becomes the most important thing in the design.
So it's a paradigm shift, because you move that focus on that, that becomes the important thing.
And then some fast moving things start catching up and accelerate, and that becomes our most taxing thing.
And then this thing, again, and the different curves start catching up again.
So what happens is you have this cyclic phenomenon, too.
But also you have the discrete So you might get into a revolutionary thing, and really bring something down.
For example, we went from very power-hungry, bipolar devices to SEMOS.
And suddenly everything shifted down a lot.
And then we are back again, we are those power-hungry days again.
So that was kind of a revolutionary change and get evolution.
If you want to do work on something, it's no fun working on something that's very important, hot today.
The people who are making the biggest breakthroughs on the things that are hot today, started 10 years ago.
So your finally figure out that this is the most important technology today, I'm going to start working on doing my Ph.D. By the time you are done, it'll be passe, most of these cyclic things.
So the key thing is, what is going to be hot in 10 years?
That's a hard problem.
So this is the thing you can look at these trends, and say, look, these are the kind of trends.
This can give you some idea where what might be interesting.
So are you going to become a low power person?
For example, six, seven years ago, people who studied low power, they made a lot of significant contributions.
People now still can contribute.
But people who got in early had an easier time, and they had a lot more impact then people who got in late.
So we look at some of these trends, and see what we can come up with.
The first trend, this is a very trust trend so far, is Moore's Law.
Basically, most of our basis for things happening is part of Moore's Law.
Basically it says, you get twice the number of transistors every 18 months.
We have had that trend for a very long time.
What's the impact of function?
Nobody cares about processors who can't use transistors, it's some kind of closed, you don't even know many transistors are in the processor.
What matters is what it can deliver.
So performance is what?
Delivery.
So what people have realized is if you are in the superscalar world, this has been flattening out recently.
If you look at it, this is kind of getting flat and that was a big problem.
So that's kind of the reason for crossover for superscalar, I mean multicore.
So we are trying to get back into again a higher growth curve in here.
And then another trend is power.
Power had been going in the wrong direction and also power density has been going in the wrong direction.
This current keep growing more than this and then people have problems in that.
Something that is really revealing is what's per spec performance.
That means how much power you are consuming to do something.
And what you see is, we are just starting to -- these days we use very little power to get work done, and now we, what superscalars did was be more and more [OBSCURED].
And this is what, I think, led to a big reason for going into multicores.
Because we realized, look, we keep wasting and now the power consumption of computers are non-trivial.
You could talk to Google.
Their power costs, power budgets, is non-trivial.
We can't just keep wasting that resource.
You can keep wasting things when it is not that important, smaller factor, but power becomes a big factor.
So how do we get out of the wasteful mode?
And I think that -- a lot of people are I'm looking at that.
If you talk to people five years ago, people say, this is free.
Transistors are free.
You can just keep doing things.
Just keep them available, [OBSCURED] makes it free.
But what people realize is, wire transistors are free, switching them is not free.
And switching the request file, so power become an important issue.
At that point I want to point out was this wire delay.
As we keep switching them faster and faster, the world looks smaller and smaller and smaller, you can make global decisions.
What the implication of that?
And we're facing that, I think it's going to get a little bit worse.
One saving grace for that is people realize they can keep scaling the clock speeds that much.
This chart I did a few years ago, still is a road map, that's made the silicon people think where the silicon is going.
Thought they can get to 10 gigs, then 15 gigs, and stuff like that.
And now they [OBSCURED], and go to all of these power requirements.
They might not be able to switch that fast.
So there are changes to road map because of that.
Another interesting thing is --
RC line type wire delay, as opposed to -- that's not speed of light delay, that's including scaling of the wires themselves?
This is mainly speed of light delay.
Because what you do is you just look at clock speeds.
I haven't done too much of dialectrical and stuff like that.
That's the thing, you can get closer to the speed of light but right now we are only about -- I mean we are not even a factor of, I don't know, 80%, 90% speed of light.
So best case we can get 10% better.
It doesn't give you that much of room.
And this is a continuous thing, because microprocessors are going much faster than d-RAM.
And this is having a problem.
There are people says there might be revolutionary things happening here.
Things like non-volatile RAM, might come and kind of take over this area.
What happening today is d-RAM now looks a lot more like a disc looked like 10 years ago.
In may ways.
We have to treat a d-RAM like a disc.
That's kind of the way we have of looking at the world.
Here where I look at compilers and parallelizing.
There's no nice number here.
So in somewhere around the 1970s, people did things like vectorization -- question?
I have a question about the last slide.
When people do this plot which is performance, it's not exactly performance that or throughput that matters relative to d-RAM, it is the computation lengths that is how fast from when I know I want something from memory to when I really, really need it that matters.
Is this going to change as we move to a wider architecture, with more parallelism, in the sense that it will no longer be, the lazy gap won't be increasing anymore and we'll start to hear more about bandwidth?
I don't know, because bandwidth is also power.
These are a lot of interesting arguments, this is what you don't know.
Because the thing is, what superscalar people did was, we reduced the problem with latency by predicting.
And doing extra work.
We will try as much as possible to predict what you need and do things in advance.
What happens is as you keep predicting more and more, you're actually doing, your accuracy of predictions go low.
But you still keep increasing -- the more data you at least get a small percentage increase in useful data.
Effecting latency.
But accuracy start pretty low and it start becoming not useful.
So bandwidth problem has an impact, too.
So unless you're using inside the entire thing, you're predicting a huge amount of things, predicting that you might use it.
So if the prediction is wrong, it's a lot of wasteful work.
And the key thing is can you pay by power?
I think latency, at some point, if you can really predict, it becomes hard.
Unless you have very regular patterns that the predictions working there.
If you look at what happened in compiling and programming technology.
In the '70s, you gets vectorization technology.
It did well.
And then some time in 80's we did things that compile for instruction level parallelism.
That got a lot of parallelism in there, very low, low parallelism in there.
And somewhere in 90's we did automatic parellelization compilers for FORTRAN.
I think that was today almost the highlight of that.
Because what happened was we figured out how to get the program language in FORTRAN and do pretty well automatically extract patterns.
Then things were kind of down in front there because people say FORTRAN is interesting, but we don't program in FORTRAN anymore.
We are going to program in things like C, C .
That did two things.
First of all, one is a language problem, which is C, C was unfortunately was not a typesafe language.
It was not be given up the right scientific application.
More like operating system hacking, that required looking at the entire address space.
That really hindered the compiler.
Second, programmers start falling in love with complex data structures.
The world is not fixed size arrays any more.
They want to develop things, have different structures, trees.
These things are much harder to analyze.
So suddenly you can do it nicer, they said, look, people are doing recursions and these very complex structures and suddenly, the more of the things you can go went down pretty drastically.
And you're back to a pretty bad state.
And then we're slowly recovering a little bit because things like JAVA and C Sharp gave a typesafe language, became a little more analyzable.
I think automatically what you can do kind of improves.
And the hope here is a demand driven by multicores There's no supply technology but there's the demand.
You need to do something.
And hopefully there'll be interesting things that come about because of the demand.
So this is my prediction, where things are.
What happened then and what's going to happen next.
Of course, multicores are here.
There's no question about that.
But I don't think programmers are ready for that.
I mean we have established a very nice, solid boundary between hardware and software.
Nice abstraction layer.
Most programmers don't have to worry about hardware.
Programmers didn't have any knowledgeable about the process.
In fact, we are moving in that direction.
JAVA said you don't even have to know about the class of the program.
We are running this thing in a nice high-level byte code.
Write once, run everywhere.
That's a very nice abstraction.
For most types, more slow actually gave you enough performance, that that was good enough.
We can deal with that.
The nice artifact of that is programmers were oblivious to what happened in processors.
A program written in 1970 still works.
Still will run 10 times faster.
That is really great.
If you look at Windows Office that probably [OBSCURED].
We still have code written in 1970, but in 1980.
There's still code in there.
Every time they tout they have this completely new program.
Only probably 10% of new code is there.
But that's not true for the cell phone.
Every time they say I have a new cell phone, most of the time actually rewritten completely.
So that's why in an industry like that it's very hard to evolve like that because we are just churning too much.
And in soft ware we got really enamored by this ability to reuse stuff.
Ability to have this nice abstraction in there.
And the problem is there's probably a lot of freedom for programmers, so they push on complex issues.
So your question about how large companies want complex things, they way people get their computer advantage is pushing the complexity.
So what they did was, instead of pushing the complexity in the processor, they push the complexity in software, features and stuff like that.
So we are still on the brink of complexity.
But now we are adding another complexity that's very hard to deal with.
So this is where things came about, how things move.
Where can we go?
I want to talk about two different parts.
Architecture, and languages, compilers and tools.
Most of the things I talk about, That's no solution.
This is what I think that will happen, should happen.
I think that can hopefully really inspire people to go do that.
And hopefully I don't have to eat what I say in 10 years.
One thing now about novel opportunities in multicores is you don't have to contend with uniprocessors.
Most of the time they are a source of big pain when you are doing research in here.
Everybody says, yeah, that is very nice, but I just wait two years and this will happen for me automatically.
Why do I have to worry about your complex whatever you are suggesting in there.
The other thing is, in the good old days, the people who really wanted performance were these really performance weenies.
In fact, you probably got a good talk from a performance weenie last time, who's [OBSCURED].
And we want just performance at all cost.
There are people like that.
For them, producing anything it's not easy.
They say, I don't want tools.
They just suck my performance out.
And it was very hard to help them, because they don't want help.
But right now what's happening this is becoming everybody's problem.
I think that will lead people to want the things to take advantage of parallelism.
The other thing is, the problem can change.
Because people worked for 20 years or 40 years on this parallel problem, but that's for multiprocessors.
So how does this problem has changed?
What's the impact from going from a multiprocessor, which is multiple chips, sitting on a board, with some kind of interconnecting to multiple cores on the same die.
OK.
I think there are some very substantial things that happen, that can have big impact.
Let's look at two of them.
Which is communication bandwidth and communication latency.
I think this is where you can have revolutionary kind of things happening.
If you look at communication bandwidth, the best you could get, if you put two chips on any kind of a board, it's about 32 gigabits per second.
Because you had to go through pins and pins had a lot of issues.
Today if you try to put two different cores next to each other in the same die, the bisection bandwidth is huge.
It's four orders of magnitude huge, that's what you can get.
That's a huge change.
Even for the lowest 2x improvement, this is four orders of magnitude difference you can do.
So why is that?
Because what changes number of wires in the die.
Because, if you think about it, one name type of architectures is to just just build, to deal with this reduction of communication bandwidth.
The closed resistors communicates the most, the local memory next, and then remote memory.
And then if you had to go remote to the next guy, network.
These things basically reduce the amount of things you can do.
That was really true.
Because if you you're inside the chip, you can do a lot more.
The minute you go to the board, you can do less.
Minute you go to multiple boxes, less.
Stuff like that.
That is the way the processor would decide.
And also the clocks.
Being a processor you can do much faster to go pins, not that fast.
People are trying to change that, but still that's an issue.
And multiplexing, because pins were inherently limited.
If you have thousand pins that's a big processor basically in there.
And within a die, a thousand wires is basically nothing.
So we have four orders of magnitude improvement in here.
And that's amazing.
That can have a huge impact in here.
So you can do massive data exchange.
On big issue right now in parallelism is the minute you parallelize something, you might slow it down.
Because the thing is, you might not get data locality.
Because that pipe is going to get clogged up because you've been trying to take too much data in there.
So if you put wrong things, you're going to get clogged by the communication panel.
This thing says, I might not have to deal with that.
For example, I don't know why you want to do that, but if you want to move your entire cache from one core to another, you can probably do it in a few clock cycles.
That is bandwidth.
I don't know why you want to do that, because that's probably not memory efficient, power efficient.
But you can do it.
We have wireless.
So the key thing is how can you take advantage of these wires.
And how can you make it in a way you can really use the difficulty in programming, because, as you've figured out, parallel programming is hard.
If you make a wrong decision, you're going to pay a huge price.
How can you make that in a way that you can make a few wrong decisions and get everything by taking advantage of that.
I think that will be a big opportunity in here.
Another opportunity is latency.
This is not as big as bandwidth because latency has the inherent speed of light issue.
Again, what happens is now length of wires are a lot shorter.
And no multiplexing, so you can get a direct wire from there.
And on-chip can be much closer.
And the other interesting things you can also think about.
Right now, why is register to register communication just very fast?
In a processor if you send something one way this can use it almost instantaneously is much faster.
Of course, wires are not that, it's very close.
Also there are a lot of other things we do.
We need more register, register before we even come to the data.
In a normal pipeline machine you have a pretty long pipeline.
The data is externally receivable at the end of the pipeline, but I can speculate the same to the next one when you start working on it if you're wrong, I would yank it out and you have give [OBSCURED].
But right now if you actually go across processors, we can use speculation.
You have to wait until the things are committed to memory, and then you can start using it.
That's even longer than ideally, because that's like 10, 15 cycles I have to wait.
Why can't I send something speculative to my neighbor?
I have enough bandwidth to deal with it.
If I do something wrong, can I speculate across multiple cores.
So I can get my data much faster.
I save probably an order of magnitude beyond my wire delay because I have no pipeline stages made until [OBSCURED] happens, very long later.
Things like that, can I do in modern process?
Because no, I'm putting these darn things next to each other, what can I do with that?
And you can do ultrafast [OBSCURED] real time and they can use -- can I do things a lot fine-grain across and take advantage of that?
So the way to look at that a traditional microprocessor basically had this structure.
Cache memory and processor cache memory.
And this is there because to deal with a small amount of wireless power and stuff like that.
Today what we have done is basically taken this structure, and kind of put it in the same die.
OK. We haven't thought beyond that.
And we get a few order, a few factors better.
Because, no, I don't have to go through the pin because now I can have instead of say 128 pins, I can have 512 going in, or 1,024 going in there.
But still, I have 4 orders of magnitude freedom in here.
I am not using any of those.
So one thing Raw tried to do is have a much more tightly coupled, a lot more communication going on there.
So a lot of people say that today.
The biggest problem in multicores is the network.
Because how will I build that scale of a network?
My feeling is that's baloney.
Network is not an issue.
In fact, in Raw, a lot of our experiments we could never saturate a very, very well done network.
Because in a network I have so many pins, I can keep adding bandwidth like crazy.
The problem the way people design processor code today is to minimize communication going out of it.
It has a very thin memory bus going in and out of it.
That memory bus can never saturate any kind of network you can build on a microprocessor.
It matches well if you have to go through the pins, because it partially measures that.
But within the die you have so much more bandwidth.
So I think something interesting would be a fundamental sign of what the microprocessor looks like.
The core looks like.
It's not taking a Pentium and kind of plopping it once, or plumping it four times.
A Pentium is not any kind of a nice processor, it evolved over the years to basically deal with this very thin memory bandwidth.
And suddenly you've got four [OBSCURED] and that problem goes away.
And we're still saying, OK yeah, I have this entire thing in there, but I only want four wires out of you, or 1,024 wires when you can give me a hundred million wires.
That is the question.
So I think they'll be some interesting things that the people can make a big impact trying to figure out how to take advantage of that.
And by doing that, really reducing the burden for the programmer.
Right now, you say, ok look, I am taking advantage of that, but I'm passing on the buck to the programmers.
They're dealing with all the small interconnectary.
Then we say, you can't just that because programming is hard.
How can I reduce the programmer burden by taking advantage of that.
I think this is a very interesting to looked at this problem.
If you're doing any architecture kind of research or are interested in that, here is an open problem.
I have this huge bandwidth out there, and I have evolved something that is completely in the way dealing with very limited amount of bandwidth, but that's why they live for this long.
And the minute you open it up, how can I do with that?
I don't think we have a model.
And I think a lot of interesting can come out there.
OK. Let's switch to language, compilers and tools.
The way you look at it, what's the last big thing that happened in languages?
I think one that happens is object oriented revolution.
The interesting thing about object oriented revolution is it didn't happen in a vacuum.
It happened because a lot of interesting research in academia and outside went into it.
People didn't come up and say, wow, it's a great I'm writing Java.
There were hundreds of languages that developed and explored these concepts.
And the interesting thing about programming is, a lot of things that looks very neat and interesting, when you look at it first time when you start using it in a media-line program, that has been evolve over 10 years, you realize that's a pretty bad concept.
Can anybody think about a concept that looks very interesting at the beginning, but actually very hard to deal with
My last program
I haven't heard that analogy before but what else?
Exceptions
Exceptions, I think people still haven't figured out.
I think one thing people have made assumptions think of multiple integrators.
With multiple integrators, the first step, wow this is the best thing since sliced bread, because the thing object oriented programming does is it forces this one hierarchy, and the world is objects, but there's no one hierarchy in the world.
World has these multiple interconnections in there and so people say, this is great.
This basically now give you objects and you can get a lot of different relationships.
But, after people trying to deal with multiple integrators they realize this is a pain.
It's very hard to understand it, it's very hard to right compilers, there's so many ambiguities around it.
And then at some point you just say, this is too much.
And that's why you went to, like Java went multiple interfaces light by having, you can have multiple interfaces.
Here's a concept that looks very nice, but you had to work -- it took five, 10 years to realize that it's actually not usable.
So if you to look at object oriented languages, there have been many languages.
This is an interesting thing, because in parallel community we haven't had this kind of explosion out there.
Another interesting thing about languages is language has lot of relationship and evolving.
So if you loot at from FORTRAN that's started.
You can see the FORTRAN, so this is FORTRAN in here.
To figure out what's happening here, you can kind of trace over the years how new languages academically and industrially came about.
Got influence from others, and at then end up with things like at the end of the graph there's something you can never read it.
But the key thing is, there's all these influences going across in there.
And cross fertilization.
At the end, you end up with some TCL TK, Python -- I can't even read it carefully.
Java, C sharp, PHP, and stuff like that.
There are a lot of different languages trying to get influence in there.
The key thing is you need to feed this beast.
If you want to have a lot of evolution or even evolutionary changes.
What that means is, people say, yeah, we have Java, C and C , C sharp, we are happy with languages.
No.
I think in the future to deal with that you need to have kind of revolution in there.
So if you look at something like C , almost every language started with FORTRAN.
And then there had been lot of different influences that happened to that.
And I'm going to just go, that's for example, Java, it's acknowledge that a lot of these different languages influenced ideas in there.
So why do we need new languages in there?
I think we have paradigm shifting architecture.
Because sequential to multicore will require a new way of thinking, and we kind of need a new common machine language for these kind of machines.
And the new application domains.
Because streaming is becoming very interesting.
Scripting.
People are using things like Python and PEARL and stuff like that in a much regular basis to do much bigger things.
So those are cobbled together languages.
But can you do better in those things?
Real time.
Things like cell phones and stuff become more important how we deal with those issues.
And also new hardware features.
Right now good language is tied very well with the one [OBSCURED] hardware features in there.
So things like, for example, black when I tell you that we have all this wires available, if you figure out some new way of taking advantage of those wires.
That hardware feature.
What's the software that can really utilize that?
So you can just build a hardware feature, but it's not sufficient.
You have to have a software that do that, too.
So what's the coupling?
A lot of things that has to be that nice interconnection there.
And I think there are new people who want mobile devices.
A great program.
If we would look at parallelizing compilers or parallelization as I said before, was always geared to this high performance weenies, who just wanted to get their global simulation going as fast as possible.
Who has a multi-million dollar supercomputer waiting.
It's very different having things like great programmers who want these things.
How do you cater to them?
They are very different.
The key thing is how can we achieve parallelism without burdening the programmer?
Because we don't, in this class we barely burden you guys and then you realize how hard it is.
And how much complexity it adds on top of the algorithm, just to get it working parallel.
So how do you reduce some of the burden?
Perhaps not completely, but reduce that burden.
I'm going to skip some of this.
I'll make it available on the site.
I want to have some time for discussion.
In streaming, we talked about it, we tried to do some of that.
Tried to come up with the programming model where it will give you some additional benefits for the program, at the same time you can actually get multicore performers.
And, of course, we saw that in compilers we can actually get good performance in then.
One thing that I did a long time ago, is this SUIF parallelizing compiler.
What we did was we can automatically parallelize FORTRAN programs all the way.
This kind of almost worked in the 90's.
We, at some point, spec bench marks were the way to measure how good your world is, and every company had 20 engineers trying to get 5% of spec.
At the time our spec performance was about 200.
Wire registered, we did this compiler, that we could expect bench mark performance at 800, by just parallelizing.
And in fact we made this t-shirt that said, spec red and the number.
That's all the t-shirt said.
And, in fact, I was working in a few places, including the grocery store at Palo Alto, and people stopped me and said, how did you do that?
It's only on this kind of weenie things.
And I had in the grocery story, I had half an hour conversation with [OBSCURED] of how -- he was, like, hmm.
The only thing, spec 823.
That's the only two things the t-shirt said, people knew what it is.
But, there's problems.
The compiler was not robust.
The compiler can be used by the people who wrote the compiler, because we knew all the things in there.
The thing about that is it worked in a way that if program worked, you get 8x improvement.
You change one line of a code somewhere, you went from 8x to 1x.
That, I think, stopped parallelizing around the slow [OBSCURED].
So now, normal programmer can now understand what happens.
We go and analyze it and afterward say aha, this is how to change the program, how to change the compiler.
So we just kind of developing program and compiler at the same time, and we could use this thing.
You cannot ignore the fact that this is happening.
Because there's so much of a difference in performance.
That makes it even a curse.
I mean, if you only get 4x improvement probably things would have been better.
We should have probably capped the performance or something.
And clients, in those days, were impossible to with.
You saw a very good example in last lecture.
OK.
Performance at any cost.
I don't want any of these tools if it slows down at any time.
So how we deal with those kind of people.
It's hard.
In multiprocessor, communication was expensive.
If you make a wrong decision, you pay a huge price.
Basically slow the program down.
You go from being 1x, you actually go from 0.5x.
And worse.
And you had to always deal with Moore's law.
Why do I need to go all these complicated compiling?
Can I just wait six months and won't I get it?
They're probably right at that time.
And another problem, what you call a dogfooding problem.
The reason that object oriented languages is very successful is everybody who did the object oriented language, in order to get accepted as a reasonable language, had to write their own compiler in that language.
And the compiler is a very good way of really pushing object oriented concepts.
And so almost every object orientated language wrote their compiler.
That's a really big thing you have to write.
And by the time you write the compiler you realize all the problems of the language, and you feel equal, because you write a few hundred thousand line programming of language.
Except all the people who were doing high performance, we were catering for somebody else.
What that means is we can deal with this weirdness, and say, yeah it's no problem, nobody -- but those are the things that really kill the users in there.
We didn't have the users perspective.
And another thing that today is these things are really different.
Because you have complex data structures, complex control flow, complex build processes, aliasing type unsafe languages.
All this cool things we used to do, just became a hundred times harder, because of all those.
People fell in love with data structures.
And they say, oh, they were so nice.
We knew exactly the same size thing, it doesn't change, I could analyze it.
But malloc data structures, it's just and very complex trees and doubly linked lists was just impossible to deal with.
We want to go in the days where everything is a nice simple array But I guess the paths for the compiler became much harder in those days.
So I think compilers are a critical thing to deal with.
I think if you have to improve in multicores we had to deal with compilers.
And the sad thing is some of those things people are trying to do it today by hand, we could do it with the compilers 15 years ago.
We kind of lost that technology.
And people are doing this by hand again today.
And go back and dust off this technology and bring it back in there.
Best case, we might automate everything.
And say, don't worry, do what you are doing today and just keep doing it and we'll get multicore performance.
I don't think that's going to be that realistic.
But in worse case, at least we can help the darn programmers.
Tell them what's good, what to do when they do something wrong.
How can you build those kinds of things.
I think that's very important.
I think in tool side, we need tools.
I mean you realize after programming on [OBSCURED] that was really nice to have your tools.
Everybody can probably come up with some tools additions.
And I think tools, on this, is pretty ancient.
How do you come up with really good tools that the normal programmer can use them?
Figure out problems, figure out debugging issues and stuff like that, and get through together.
We need Eclipse type thing for platform for multicores.
We have a lot of nice plug-ins come about that actually help you to do that.
Another interesting thing, this is about this dogfooding problem.
Normally what you come up with language compiler tool, you do implementation, you do evaluation of that and you cycle through that.
And evaluation says, ok, you develop a program, you debug the program for functionality.
Performance debugging, evaluate, and everything kind of go around that.
I mean there's a process here.
And you'd rather realize somethings are very hard to do, somethings are easy.
And you need that to basically evolve that.
And the problem is -- I'm contrasting two things.
If you look at something like CAD tools.
CAD tools were developed by a bunch of guys sitting in places, used by some very different group of people.
Because of that, today's CAD tools are horrendous to use.
Everyone hates them.
Because the people who developed CAD tools never thought about users.
They were indifferent.
And they just have to get the functionality out and they were happy after that.
Whereas, object oriented languages were more thought about as the how the best way to use it.
Very much of a programmer-centric phase.
And in that end, you develop a system that's a lot easier to use, a lot of people use it, more acceptance in there.
The problem with high performance languages is they really sit in the CAD tool way.
Because the people who write high performance languages never really use it.
People who write the compilers and languages are not the ones who are developing the high performance programs.
No high performance language ever wrote the compiler in that language.
They say, oh, that's different domain, we don't know how to do that.
By doing that, you really never expose the problems in there, and I think the key thing is how do you go about doing that?
This is a hard problem.
Because how do get two different groups together?
Perhaps multicores, what you to do is you probably everybody who write the compiling tool, actually have to write it in the thing that you are doing, because you need the parallelism.
At that point you realize all the problems with the tool.
So I'm going to the skip on this side.
Another interesting, very interesting problem, that people don't pay attention is the migration of the dusty deck.
Millions if not billions of lines of code out there, written in old styles way.
You've can't just say, look, Microsoft, just go rewrite everything new with my spiffy new system.
That doesn't work.
How to get these people to migrate out?
That might not mean doing everything automatically, but how can you even help -- the people who wrote the code is probably gone, left the company, probably dead.
How do you help somebody move some of these things to you.
That is the rate of evolution.
You don't have to rewrite everything from scratch.
What kind of tools that you can use?
And these applications are still in use.
Sometimes source code is even available, but programs are gone.
But the interesting thing is, some of those things, applications have bugs that have become features.
So I'll give you a very good example.
And this happened to Microsoft.
So at Microsoft at some point -- if you know Word, Word does algorithmic pagination.
That means how to lay out the pages, and how to break pages and stuff like that.
So they had the algorithm in there.
And they were switching versions, and they said look, this algorithm is now too old.
It's been there for five, six years.
Rewrite it.
And of course you can assume what the spec is.
There's a nice spec to that, a few pages.
They said, go rewrite that.
And someone wrote a very nice pagination algorithm.
But the problem is, when they use that algorithm, a lot of old documents broke.
That means they didn't paginate right.
So the old document, when you open it, looks very different.
And they said, darn, what's going on?
What they realized is, the old implementation had a lot of bugs.
Subtle things.
The guide didn't really conform to the standard.
So it didn't break the word exactly right, if at some point it has a break and space, it would probably put it on the next line.
A few things that didn't really -- And what they had to do is, they had to spend a lot of time discovering bugs and adding them to the spec.
So they did that.
Because otherwise, all the previous program doesn't work, the previous files will not look right.
And they went through this entire process of trying to discover and trying to figure out what [OBSCURED] instead of what the text of the specs said.
They're out of sync.
And so a lot of times, there's that problem.
So you see we have something old, and we want to get it new.
It doesn't mean that you know the functionality.
The functionality that was implemented is the spec, or the functionality that you wrote about.
Because that's a little bit subtly different.
So how do you go and discover that.
I think that's a very interesting and important problem.
And I think there are many reasons that run such things like creating test cases, extracting invariants, things like failure oblivious computing type, if you had listened to professor Rabbah's talk .
Things like that can help take these old things and migrate.
Can use tools to do that.
I did a lot of interesting research on that.
OK, so that's my take on languages and compilers, where interesting things have happened.
I want to talk about revolution, but I want to skip that first and go into this crossing the abstraction boundaries part, and then come back to revolution.
So the way the world works is, you have some kind of class of computing you want to do, and you have some atoms that need to run that computation.
That's basically the end to end of any process.
But if I said, here's some silicon, and here's my algorithm, go do that, no single person can basically build that.
OK?
You can't take silicon, and go through a semiconductor process, build a process, design all those things, language -- it doesn't happen.
Someday probably it was possible to have one person know everything, but you can't.
So what we have done deal with build abstraction boundaries.
We have compilers, languages, instruction, microarchitecture, layout, design rules, process, materials science -- basically, everything is separated out.
Every time one gets too complicated, we kind of divide it in half, and keep adding and adding.
And we have this big stack of things.
So at the beginning, then you break up, people kind of knew both sides, and they went into one side and kind of broke that up.
And so people knew a lot of things.
But the next generation comes, they only know that.
The guy, someone who's going to be expert in microarchitecture.
You only know microarchitecture.
He has no idea what the languages are, because that world of his is complicated.
What we have is a compartmentalized world that was broken up as we went in the last 40 years, as things progressed, the right thing at that time, they figure out what layers to do.
What we have done is create the domain expert.
There's one guy who probably knows about two things.
He might know a little bit of design rules, and probably process a little bit.
Another person will know a little bit of design rules, layout, and microarchitecture.
And there's someone who'll do microarchitecture and ISA.
And somebody knows compilers and ISA, but has no clue what's happening.
So what this has done is kind of entrenched some layering that really made sense about 30 years ago.
The problem now is, this is creating a lot of issues in here.
So what you kind of really need is a way to break some of this layering.
How do you do that?
You need people who can cross multiple of these disciplines.
So this is a thing for you.
I mean, you might think you are an architect.
But hey, you're in this class, you're learning a little bit of programming.
Go learn a little bit of physics, what's in there.
Just have a little bit of broadening in there.
Then a lot of people, there are some people who are a lot more in the process side, but know a little bit about microarchitecture and compilers, somewhere in the middle.
They are expert in the middle, but they are not just sitting happily in their own domain.
Just trying to look at cross-cutting there.
And keep going up, in some sense.
I kind of put myself in there, saying OK, I am in compilers, but I know a little bit of language, and I know a little -- I try to go down and down as much as possible.
What you actually need is someone who knows everything really well.
That is hard, but at least someone who can know the top and the bottom to redo the middle.
Because my feeling is the way the middle is done is probably now too old.
So I don't know that this is possible.
It's not me.
But if someone really understands the top and the bottom, and say OK, I can probably throw the middle out and redo the middle.
And I think that might be an interesting revolution that might happen.
It's hard.
There's so much information there.
Some of the information, there are the layers out there, it doesn't make sense, today, the way things are.
I mean for example, a lot of these layers had issues with wire delay.
Because when wire delay was not an issue, these layers made perfect sense.
But as wire delay came about, a lot of changes had to propagate across layers, and it was very hard, because people were domain experts.
So I think this is where the revolution is going to come.
Somebody's going to [OBSCURED] the way you do compilers, the way you do architecture, the way you decide everything is wrong.
I know the algorithms you want to run.
I know what's available in very low [OBSCURED].
Let me see you put that together.
A person, a group, whatever.
Might have a beginning.
And so with that, I will go to the revolution side.
So the way you look at revolutions, what are the far-out technologies?
And sometimes revolutions just come from wishful thinking.
I wish we can do this, and then if you can push hard, you might be able to get there.
Anybody wants to -- so in this talk I talk way too much.
How about, come in?
What do you think is going to happen?
What do you want to work on?
You guys are the ones who have to make this decision.
Not me.
I've already made a lot of decisions myself, 10, 15 years ago, what I'm going to do.
You are in the process, you have a lot more choices to make.
You haven't narrowed your horizon yet.
So this affects a lot more on you than me at this point.
What do you think is going to change the world?
What do you want to work on?
I mean, you should work on something, I guess, that you think is going to have a huge impact.
Have you thought about that?
A lot of silence
[OBSCURED]
So faster means very little latency.
Latency is a hard problem.
A lot of people who are trying to do latency [OBSCURED] And the revolution side people are looking at quantum can kind of do deal with latency
Yeah
Yes, I think there are a lot of revolutionary things you can do there.
Especially taking advantage of the fact that you have a lot of wires in there.
And I think people haven't really pursued that yet.
And this is where the complexity will come in.
Trying to basically deal with these, take advantage of these kind of things.
I think there are a lot of opportunities there.
And then that's the kind of evolutionary part.
And then some people are looking at revolution, with quantum.
What else?
I expected people to have a lot more interesting insights
[OBSCURED]
I think the world is almost bifurcated.
There are things that you can have with power plugs, and things that don't have power plugs.
And I think people who have power plugs still have some power issues, like what Google is finding out.
You can't just keep wasting power.
But as things start becoming more mobile, going into smaller and smaller devices, power and size becomes a much bigger thing than what more you can do.
You don't become performance, I mean, you want some level of performance; you need to get a video.
But the minute you get the video, you don't want to have HD TV.
You just say, OK, get the media onto my cellphone screen.
I'll be happy with that speed.
Just do it.
As low power as possible, as [OBSCURED] possible.
So I think that's pushing there.
So for example, in for a long time what has happened is there's a trickle down economy.
Things that start happening in a very high end system and kind of trickle down to the low end.
But there might be a diversion here.
There might be a different part coming.
Because that technology might be fundamentally different
So [OBSCURED] also changing from the software side is this notion of perpetual data.
I've been using this mash-up code, you put it up on the web, lots of people use it.
So it's pushing toward scripting light languages or whatever
So this is an interesting obsession I have.
So [OBSCURED] And then a few years ago I had a start up, and I just again had the time to try it, not large , some large piece of code.
What I realized was, what programming had become, is the biggest, best tool for programming at that time was Google.
Because I was trying a bunch of things with Windows.
And everything I want, there was a function.
And if I can find that function, my work is a factor of 100 smaller.
So I spend more time than actually thinking about algorithms, how the lists should be, or how my data structures should be, trying to Google it.
I said, darn, I want to do this.
Where's that function?
Where can I find that function in this entire big library.
So what that means is there's kind of this missing layer in there.
Because what has happened is, our language is a level of abstraction.
And then we have built these libraries with no kind of support at all.
I mean, the compilers' languages don't really support the libraries.
Because it's just built out of basic blocks.
And keeping it, can you have better basic blocks, can you have more nice abstraction?
Instead of saying, OK, here's 5,000 libraries with 250,000 different functions that you can use for compiling.
Can you build it in a nicer way?
[OBSCURED] interface, your input, your output.
And then you publish your code and then Google indexes it for you.
And then your program is just of Google searches
I think, if you think about what all the Windows source base is, it's like that.
I mean, in the company we were doing, during the [OBSCURED], there was this group of people who understood every line of code they wrote, because they were writing a virtualization system that runs on the Windows.
So for them, every line was probably -- Interesting thing is, when that group started, that entire piece of code was about 50,000 lines.
Now it's about 65,000 lines.
But we had about 10 people working on that.
That means these things get revised.
You don't keep at it.
There's another group of people who's doing user interface and stuff like that.
they have alphabet soup of different libraries.
They're using Ajax, this, that -- I don't even know what they are using.
The amount of code they write is immense.
The amount of things they have to know is immense.
Each of them are not that completed.
But if you use a bad thing, you have a bad looking interface, clunky, slow, whatever, if you figure out the right three function calls, things look much better.
That's a very different type of programming in there.
So for them, it's basically knowledge.
How much do you know.
Just like, it's almost like I remember my college days, friends who were doing things like that in medical school.
So you need to know this entire thing about all the different muscles in the body.
And there's this big thick book you just study.
It's almost like that.
You just take this big Windows manual that are a few hundred pounds heavy, and you look at every page, and figure out what you know.
That's a really strange way of coding.
But, I mean, algorithmically, I think what they were doing is pretty simple.
I don't think they every designed a data structure.
Every data structure they used came from something.
But they had to figure out the right data structure
This is going in a totally different direction, but in terms of graphics in video games, perhaps getting artists who are aware of both the hardware and the software, and so can design more for the capabilities of the machine, and can make more visually appealing --
I think that's the are that has a very different set of communities.
Artists [OBSCURED].
Not even trained engineers.
And they're sitting in one, and they have this neat feature, they want fire to look natural, or whatever it is.
And then there's the programmers and hardware people, that who knows what the capability is.
I think that's what we're trying to push in Mike Eckham's talk, and say how do you go about it?
How do you know when a new thing comes, what is doable?
Like for example, what ATI does, ATI has a group called the research group.
What they do is, every time a new chip comes, they actually go and figure out all the things you can do with that.
OK, how can you do rain in this one?
So they write a lot of -- those of a little bit of an artistic bent, but very hard-core programmer who kind of figure out what's doable.
And they make that available and say here, this is what you can do.
And a lot of people kind of emulate that.
They say, wait a minute, how did they do that?
Then kind of figure out, OK, now I have this new hardware, and I'm able to do something completely new.
I think that's, pushing there.
I mean, I think this is where interesting revolutions come about.
One of my firm beliefs is that a lot of good research is not in one area, as I pointed out.
It's kind of looking at two distinct things and kind of bringing them together.
Bringing knowledge from one to another and mixing them together.
I think that can have huge impact.
And that's a place that I think can have huge impact.
What other things that will have big visual impact?
That the next hardware can enable?
You can keep adding more big servers and stuff like that, but it's another thing.
What other things can can have big impact?
[OBSCURED] The same is true for biologists, physicists, chemists, end users.
So maybe the thing to do is work on language generators.
And easy way for me as an end user to express language [OBSCURED] that I want and give me a way to generate it
So that's a lot of things.
For example, I was talking to this computational biologist.
And what he said was, for computer science, the best thing that happened from a biology point of view is Excel only had 32,000 cells.
Because the biologists keep using Excel, and they say that --
[OBSCURED]
65,000 cells.
And they ran into this and said, jeez, I can't do anything more in that.
Now what do I do?
And then suddenly they had to start looking at programming and stuff like that, because they can't have an Excel spreadsheet doing that.
So that's a lot of biologists actually had to switch.
So I think, interesting thing about computer sciences, there are a lot of fundamental sciences still.
But I think we have grown and ran so fast developing that, what's lagging behind that is applications.
How to apply that to biology, gaming, stuff like that.
And I think a lot of interesting things, as computer scientists, we can do is to bring that knowledge in an applied community.
Really, you can have a huge impact and fundamentally change things if you apply that knowledge.
Both theoretical things you have to get out, as well as using the powers of the computers we've got in the right way.
I think a lot of times, people are taking a few days to do two times better, five times better than kind of seeing fundamentals [OBSCURED].
I think this is an interesting topic.
I think I'm going to stop soon.
And I think it's interesting, especially for you guys.
Because you are probably going into an area, looking at a field, trying to get expert at something.
There's a lot of times you get seduced to become an expert in what's hot today.
But the interesting thing is, especially if you are doing a PhD, or if you do something that takes four or five years for you to mature in that area, is it going to be the hardest thing in four or five years?
So multicores are hard today.
So should I recommend that everybody jump and become expert and try to do a PhD in multicores?
Probably not.
Because five or six years down the line, some of these things might be solved.
they'd better get solved, or else we're in big trouble.
But the thing is, what's going to be the problem there?
I see one thing that's really hard, it's almost art, is to kind of picture your [OBSCURED] and then if your lucky And if you're lucky and you do a good job in that prediction, you've come about, and by the time you get to a point that you become expert, you've become expert in the thing that's most important.
It's not an easy thing to do.
And a lot of times, nobody can claim that they have a technique to do that, too.
A lot of people who claim, they discount how lucky they were.
There's a large amount of luck associated with it.
But a key thing is to kind of follow that trend, identify, and do that.
That's the technique that I think every one of us, every one of you should do a lot.
OK. With that, do you have something about the --
Yes.
A couple of things about -- Anybody here want direct access to a Playstation 3?
Just one?
Two?
So come by, actually if you have time now, we'll talk about that and figure out a way of getting you access to it.
The final competition's going to be this Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, great.
So a lot of effort into making this course, and we hope you've had fun.
It's been a lot of support from Sony, IBM, Toshiba.
A lot of on demand help from people at IBM and Sony, trying to debug problems, even up to the last minute.
So we're very thankful for that.
And we hope in return that you really enjoyed the course, and it was really fun.
So looking forward to the course evaluations.
So just want to tell you a little bit about some opportunities ahead.
I've passed out these brochures for the cell broadband engine challenge, Beyond Gaming.
And this is a IBM sponsored event, where you really can compete in two different challenges, and in one of them you can win really seriously cool t-shirts.
That's actually lifted directly from the online link.
So the competition starts February 5 at midnight.
And you have to pass an online quiz, of which I expect all of you should be able to pass.
And hopefully, you will compete and at least get a seriously cool t-shirt.
The second challenge, which is potentially more interesting, just because there's a $10K grand prize-- there's also a $7K second prize, and some serious money for at least three or four prizes.
You can look at information in the brochures.
According to the information, you'll find it requires some serious coding skills on your part.
But I certainly expect all of you to be able to be well primed at this point, especially since you've learned about cell hands on, and you've discovered all the pitfalls on your own.
It would make for a very good competition on your part.
There are different categories.
Sorry, it's three categories, not four.
Application solutions-- so some of things, some of the people have done more application oriented thing could fit, or you can sort of take what you've learned, apply it to financial services if you have interest in that, medical imaging, electronic design, automation, things of that sort.
Operating systems-- I'm not sure if that's particularly relevant to a lot of people here.
I'm not sure if that's in their interest area, but programmability.
So certainly the streaming framework that one of the software radio projects tried to do would fit onto that, some of the work that the Ray Tracer team did, in terms of coming up with their own interface for graphics could certainly fit.
So I urge you to consider competing in these.
There's probably a lot of fame and fortune to be had here.
So we had our own competition.
And what you're going to get are some seriously cool trophies, since we don't have t-shirts.
So Mary was very-- could have had t-shirts.
I didn't want to compete with IBM directly.
So for the winning team, you will get this seriously awesome looking trophy, which has a little imprint on the front that says first place, 6.189.
And you're also going to get a $150 gift certificate, I think we announced this at the beginning it of the course.
I'm also working with people at IBM on arranging a one day trip to IBM T.J. Watson Research Center, for the winning team to present their winning project, do a demo, and then meet with people at IBM who are doing really cool, interesting research.
And it should be a fun day.
And certain people at IBM have been very excited about hearing what's going on in this project, in this class.
And so it'll be a good, fun opportunity, I think, all around.
OK, so I'm going to work my way up to the first prize.
I think in terms of the complexity of the project handled, certainly Speech Synthesis probably took on the most complex-- certainly in terms of the software that they used, and then maybe that bogged them down a little.
So there's probably something to be said in terms of starting from scratch versus taking complex infrastructure and re-engineering it for parallelism.
I think you've discovered firsthand how hard that really is.
And this is a big problem for parallel computing.
There's a lot of legacy software out there that eventually might have to be ported to parallel architectures.
How do you do it?
You pointed out, code is not well documented.
Tools are lacking.
So what do you do?
So you've stumbled on a really seriously important problem facing a lot of people in the industry.
The most synchronization issues.
We believe the guys who are doing the Battery Simulations and modeling sort of the chemical reactions certainly ran into a lot of synchronization issues, just because of the dependent nature of the computation.
Next up on the list would be for the most parallel programming challenges.
So not only did they run into issues with synchronization, but certainly discovered problems with races.
And I think they're the only team who have successfully hung their PlayStation 3 several times, and required a reboot quite a few times.
OK.
So next, best effort awards.
These guys should really be highlighted, because they worked largely on their own.
Is that true?
Yes.
Yeah, and that's certainly true.
And there was certainly a lot of people who were impressed by the Global Illumination stuff yesterday, and sort of mentioned that.
And what I'm hoping to do is take the judges' feedback and sort of summarize that for each team, so you can have some constructive criticism back that might be helpful.
Molecular Dynamic Simulation work that you heard about today, I think certainly a valiant effort on his part, as well.
The most complete project, I think this would probably be Backgammon Tutor's project.
I think in terms of scope it was well defined, in terms of what they actually completed.
I think they came out with a really cool game.
My money was on them all along to actually win the first prize.
But I think the Ray Tracing team actually blew everybody away with their demo.
And I think they were a clear winner.
They really tackled sort of their paralyzation, descendizing.
They had a really cool frame buffer demo, so they figured out how to actually write to the frame buffer and do that simulation, or do the graphics rendering.
And I think some mumbling about a real cool demo that they've gotten working overnight.
So we'll try to set up over in [?
Stoddart, ?]
where we have pizza, and you guys can take a look at that.
So with that, I'll hand out the certificates and the awards offline.
But you've at this point been Cell-ified.
And the cake is in the back says, Congratulations on your Cell-ification.
So-- Resistance is futile.
That's right.
So thanks a lot for taking the course.
I think we've learned a lot as instructors.
I'm hoping to do this again with Saman next year, next IAP, and maybe eventually turn it into a regular semester course.
I think there's a lot of things we can do better, and hopefully you've told us some ideas of how to do that.
So with that, thanks, and start enjoying cake in the back.
The following content is provided under a Creative Commons license.
Your support will help MIT Opencourseware continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT Opencourseware at ocw.mit.edu
That's it.
SAMAN AMARASINGHE (ON PHONE): I'm sorry that I couldn't be there.
Can you guys hear me?
Yes.
SAMAN AMARASINGHE (ON PHONE): OK, guys, sorry that I couldn't be there in person.
I'm in California at this workshop, and [INAUDIBLE] this panel.
But you guys are doing a great job, and I think [INAUDIBLE] last meeting.
I think this is important to carry on [INAUDIBLE] to as judges, and Steve, and [?
Ray ?]
[?
Fayan ?]
and everybody, thanks for coming in and being judges.
And I guess Roderick will coordinate with them to get the information, and try to [INAUDIBLE] on there
Yeah.
SAMAN AMARASINGHE (ON PHONE): OK. Good
Thanks.
SAMAN AMARASINGHE (ON PHONE): I'm hanging up, and I have to run to my conference now
OK. Bye.
SAMAN AMARASINGHE (ON PHONE): Bye
OK, so tomorrow we'll have class in the regular classroom, 34301, same time, 10:00 to 12:00.
There will be course evaluations, so your change to tell us how you actually felt about the course, and what you liked and what you didn't like.
Afterwards, we'll have an awards ceremony for the winning team.
You will get some prizes, and you'll get some cake.
And then that will take us to around 11:00, 11:30, at which time we'll come back to [?
Scoglin, ?]
the seventh floor lounge.
We'll have pizza and more food for you guys to enjoy.
So thank you, and see you tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to have the final presentations from the student projects.
So they did a lot of hard work.
Some of them are still working upstairs and will come down just in time for presentation.
So just a few slides on format and how the judging will happen-- so each team will have 15 minutes during which to tell about their little project and to give us a little demo.
At least two PlayStation 3s will be set up by the end of the class.
One will have audio.
The other will have video and at least two teams will use them.
The others will SSH in and show you their programs running on the PlayStation 3s that we have upstairs on the network.
One of the things that they'll be graded on or scored on is performance.
Some of the things we'll do after class is performance profiling, to try to understand how much scalability you were able to achieve using parallel processor versus two versus three, all the way up to six.
So what we'll do is we'll set up some time with the TAs to do the profiling after class.
And you can update your source code up until 9:00 AM.
So you'll show us how to do the profiling and then we can re-run that at 9:00 AM Saturday morning and give you updated scores to change your grades at the time of grades.
There are six scoring categories.
Performance-- and this first Performance category is essentially what your performance metric is.
You're going to define it, hopefully, and then you'll tell us how well you did.
Project scope-- this is essentially a measure, because each of your different projects were different.
You picked your own projects.
We'll try to get an understanding for how much you tried to cover and what you actually did achieve.
So we'll hopefully give us a good idea of that.
Completeness-- how robust is your code?
How much you actually achieve relative to what you wanted to do?
Are there data races?
Is your code reproducible with every run?
Presentation and Demo-- so we've decoupled it here, just because there are different ways people will be doing a demo.
So you can sort of earn points in different categories.
If you have a really good presentation but you might not be flashy, we'll take that into account.
And then, Performance Scalability, which we'll do off and on.
So each one of these categories score on a 10-point basis from one to 10, 10 being really good, and for a total of 60 points.
So for our judging panel, unfortunately, we couldn't get virtual Saman to really work, so Saman is with us on the phone.
We have Michael Gordon and Bill [?
Theeves, ?]
two graduate students in Saman's group.
We have myself, Professor Steve Ward from MIT, and Professor Weng Fai Wong, who's visiting from the National University of Singapore, doing a sabbatical year here.
Student presentations will dynamically reorder based on who's available and who's ready to go.
This is a rough ordering that was picked randomly.
So [?
Pumuk is ?]
here.
He'll go first.
And then, I see Linear Algebra Pack so they'll go after that.
The Ray Tracing team is still in my office working, so we might have to bump them down.
Soft Radio Team is here?
Yep.
Speech is here.
Backgammon?
Yeah.
Those guys were ready having breakfast at 9:00.
The seventh team, Molecular Dynamic Simulator-- unfortunately, they're not here.
That team is not physically in Boston, so they'll present tomorrow morning but will not be considered as competitors in the competition.
There is a countdown clock.
Bill is running his laptop.
You can see it directly in front of you.
It'll start at 15 minutes and every minute, it'll tick down, 14, 13, all the way down to zero.
And you can go into negative times, but we'll start subtracting points.
So the more you run over, the more points you start losing.
So try to keep under the time limit.
What about questions?
If there are questions, we can take those after the presentation.
If it's a short question-- yeah, let's leave all questions until after presentation today, unless there's something pressing.
I'll try to moderate.
So if we're ready to start, come up here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, my name is Pramook Khungum.
I'm supposed to be one of four, but it remains that I am the only one now.
So my project was about global illumination.
And it's about rendering 3D scenes.
So first, let me tell you about the direct illumination model.
This is something you can achieve with ray tracing.
As you can see, the image above is rendered by shooting a ray from your eyes, and then taking the local lighting properties of the materials, and then see whether there's a light coming to that point or not, and then figure out the color.
So in this model, there's a lot things unnatural about this.
Because as we can see, the light is an area light, but then the shadow is very solid shadow as if it's lit by a point light or something like that.
And you can see that the ceiling is not lit at all.
This means that this model of lighting doesn't take into account the light that bounces between the walls and lighting parts that should have been lit in the real life scene, as you can see.
So global illumination basically tries to fix that.
And the effects you can achieve with global illumination algorithms are basically soft shadows, color bleeding-- you can see that there's a very pale shades of blue here-- and caustics, which means that light from here goes through the glasses and then focus on here.
So at first when I started the project, I wanted to globally illuminate a very simple scene, which is the Cornell Box.
So it's basically this box, and you replace these two spheres with two white boxes.
And then you can observe color bleeding in something like that.
And I wanted to do it in real time with caustics.
However, the things I accomplished are-- well, I globally illuminated the Cornell Box as I promised.
In real time-- no, it's too slow.
With caustic, didn't implement photon mapping.
And sadly, there's still a lot of room for optimization.
So kind of disappointed myself a little bit, but yeah, that's fine.
So the algorithm I used for global illumination is called Instant Radiosity.
It's by some German guy called Alexander Keller.
It's good for scenes with diffuse objects, which means that scenes that doesn't have objects that reflect light or lets light transmit through.
Diffuse objects can be just this table and something like that.
Basically, it means that when light strikes it, it just bounces off randomly.
As such, it can't do caustics, although it is very simple to implement once you have the core.
So let me describe the algorithm.
You start with the light source, and then you emit artificial light particles and let them bounce around the scene.
And then you will have some more light source.
And then you ray trace using those light sources in the scene basically.
So this is the illustration of the algorithm.
First, you have the light, which goes off from this square light.
And then you illuminate the scene with it.
And then you bounce some lights off, and then illuminate the scene.
Then you combine the results, basically.
So in my implementation, at the beginning, I have one SPU scatter all the light particles.
The number of light particles I scattered is about 128.
And as they bounce around, it ends up to be about 250 light sources in total.
And then I rendered a scene by splitting the scene into horizontal slabs, and then just give the slabs to each SPU to render.
In this way, since the scene is very small, it can just preload them to every SPU.
And they all can work independently.
So there's minimal communication going around.
And since I didn't implement any other smart data structure to take care of the geometry, it just used bruteforce ray tracing.
My scenes have only eight objects anyway.
I wasn't maybe it might be a good point to see if there's [?
a question.
?]
So the SPUs are doing rendering or ray tracing?
Ray tracing, yes, basically
But I mean, can a ray move from one slab to another depending on reflection or something?
No.
Basically, if you trace the light there, the SPU has all the ascription of the scene, so it can trace the ray throughout the scene.
But it's responsible for only the pixels in the slab
Oh,
The complexity is kind of bad because the time is you have your image size, the number of pixels times the number of light particles, which is about 200.
And then since I used bruteforce ray shooting, you have to times by the size of objects.
But at first, I thought I have only eight objects anyway so it's not going to be a big performance rack.
But then I came to the realization that this algorithm initially was conceived for using with programmable graphics hardware, which can do really dumb tasks like taking a point light source and illuminating a scene with shadows very fast.
So it turns out that this algorithm is not really suited for the [?
Cell, ?]
basically.
I wish I could have done better on this.
This 512 times 512 scene took 20 seconds to render on the [?
Cell.
?]
But the original paper, they rendered something a little bigger than this in 24 seconds with GPU basically, so it's kind of disappointing
[INAUDIBLE]
Hello?
24 seconds on what type of a machine?
Can you hear me?
Yes, but not very clearly
24 seconds on what kind of a machine?
The original paper took 24 seconds?
24 seconds on a 75 megahertz something machine with a
OK. And did you try without using any SPU, just using CPU?
Just using the CPU and the GPU and that one
His question was, did you try using--
If you just use CPU without any parallelism, how much time does it take?
Oh, times that by 6.
However, the speedup is proportional to the number of processors you used.
I basically graphed the runtime with the number of processors.
This is in [INAUDIBLE] scale.
So as you can see, it-- so that's the best you can do about parallelism there, basically
[INAUDIBLE]
Yes?
So runtime is normalized to just the PPU?
Using the
So what are the units for the y-axis?
The y-axis is runtime in seconds
I see.
So 3.7 seconds, 3.6 seconds for one SPU?
Yes.
3.6 seconds per one SPU.
And as you increase, the times go down.
You just divide that by that.
Is that clear?
And they're all using the PPU as well?
No, the PPU doesn't do anything.
I just used--
Sorry, did you say 20 seconds on your previous slide?
This is different [?
image ?]
slides
128 by 128
Yes
Oh, OK. OK, different--
Sorry, I didn't make that clear
Sure,
This is 128.
So it's about 16 times faster.
I couldn't wait for llike--
Sure
Yeah.
So here are some images I rendered.
There are eight objects in here.
And it took like 20.5 seconds with six SPUs.
Another image that I rendered is a miniature of a Cornell Box.
It took 20.2 seconds and six SPUs
Why is the ceiling still so in black?
Yes, I don't know that as well.
Maybe it's about the distribution of the particle light source.
Probably I think when it goes to the wall or to the floor, it gets severely attenuated that it cannot really light the ceiling basically.
So there's a comparison between my rendering and the real Cornell Box.
Basically, yes, there are still a lot of [?
boxes ?]
around.
Basically, the ceiling is not lit.
And it's kind of strange that I see some gray tinges here instead of green as in this one.
So probably, I have to fix that as well.
But as you can see, we have achieved soft shadows and then some color bleeding
How do you determine the location of the light sources?
How do I determine the--
Are you--
Are you giving this input to the algorithm, the location of the light source?
Yes, basically
And they can be arbitrary positions within the box or?
Yes, basically.
This one is just a square object, which is written in the scene description file.
And I can locate them wherever I want basically.
And the light particles goes down from here.
So there are some things that can be further optimized.
Basically, if I had decided to use accelerated data structure, the complexity of that in that size times objects times the number of light sources, this term will be reduced to [?
log.
?]
And probably, you see about 3x speedup
Can you give an example of an accelerated [?
structure?
?]
Basically, the most popular one is probably the-- what's it called?
For example, [?
the KD tree, ?]
for example
Oh, is it?
[?
OK.
?]
And actually, I learned in class that basically, you can achieve about 2x or 3x speedup by rearranging the data so that you can trace four rays or 16 rays at a time instead of doing one at a time in the approach I'm taking.
And if you do that, it's faster.
In the program, I have a lot of objects.
And each of them has a transform associated to them so that I can locate them wherever I want and the object can be anything.
But there's another approach which says I just take anything to be a collection of triangles.
And if you have triangles, the description is just three points in space and you don't need transform to do location or something like that.
And basically, checking whether a ray hits a triangle or not will be faster.
So thank you very much
A few more questions?
Do you know that work that was done here by Julie Dorsey in illumination?
I have [?
not.
?]
She basically used the trick of pre-computing, doing the ray tracing for some fixed number of source locations and then just rendering, in real time, by taking a linear superposition of these ray-traced images.
So long as you can approximate your light source by a linear combination of some fixed repertoire of light sources-- you didn't think about taking that shortcut presumably
I didn't think about that shortcut basically.
I [INAUDIBLE] know something like that's--
[INAUDIBLE] few minutes left
Good
Let's see.
So [INAUDIBLE]
So is it possible to just connect it directly to a Playstation 3?
So basically, there's this output.gga, which I'm going to remove because I'm going to generate it.
The scene I'm going to render is going to be a Cornell Box with a cube and a sphere in it
Pramook?
Yes?
How easily could you change the light sources?
I can just move the object and the light source
OK, can you do that [INAUDIBLE]?
Sure.
Let's change it a point light source then.
[INAUDIBLE] relax
[INAUDIBLE].
[?
It just ?]
might be too slow
I see.
Here we go.
So right now, the current light source is this square.
I'm going to comment it out and change it to the point light source here.
And then here we go.
That's too quick, but-- basically, there's a point light source located around here.
So the scene is lit with one point light.
So as to expected, the shadow is hard
One more question
One more question
Sure
Just dividing the scene equally seems like it give good load balancing.
If you had like roughly equal complexity across different sections-- if you have a really complex object or if you have reflections that go much deeper in some places in others, couldn't an equal division lead to bad load balancing?
Did you think about other divisions?
I suppose.
But basically, other approach that I think of, which basically probably you can divide the scene in a really fine-grain manner
Oh, you just do like [?
that striping.
?]
Yes
Yeah, OK.
So is there a reason that you chose [INAUDIBLE]?
The communication how hard was it to get this communication working?
So what [?
you've ?]
communicated, how often do you communicate?
Repeat the question
Professor Saman asked how often does the SPU communicate with one another, and the PPU, and how often do I do that?
Yes
Basically, well, before the SPU starts running, the PPU sends it the scene description.
And then it sends it the task that says, I want you to render this pixel to this pixel.
And that's it.
And then the SPU just streams the pixels to the main memory
OK, so you do a static balance, static distribution of work [?
around?
?]
Yes
And then when they're done, everybody sends it back, and then you synchronize?
Yes

Basically
OK, let's thank the speaker.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
My name's James Geraci.
I worked with Sudarshan and John Chu on this project.
And what we did is we took a numerical simulation, electrochemical simulation, of a battery and ported it to the Playstation 3.
So the objective of this project for this situation was to port an electrochemical simulation to the PlayStation 3 and try to take advantage of the unique features of the cell processor to help speed up the computation of the simulation.
So when we talk to everyone, we talk why we'd want to do that.
We're going to discuss the model.
We're going to discuss how we solved it numerically, look at some of our performance results, and then discuss what we wish we could have implemented.
So the justification is basically the world needs energy, and oil has been in the news a lot and hasn't been as a reliable source of energy as we'd like.
So in the automotive industry, hybrids seem to be part of the solution to the energy needs.
Now, one the problems of hybrids is how much energy is in your battery.
So the more sophisticated battery model that we can put in a hybrid, the better that we're going to be off.
So a more sophisticated model requires more computation, so you're looking at having a large superscalar processor in your car, or you could see if you could get a lot of small processors to do the same amount of work.
And the cell represents the first opportunity be able to test that out.
So the simulation that we ran basically consisted of a lead acid battery cell.
So unfortunately, they used the word "cell," and we're also using the word "cell."
So it's sort of like the modern day Smurf.
So in a lead-acid battery cell, you have a lead dioxide electrode, which consists of a number of parts; a lead electrode, which also consists of a number of parts; and an electrolyte, which is 5 molar sulfuric acid, which consists of H2SO4.
When you discharge the battery, you're going to close the circuit.
And basically, what ends up happening is you have electrons flying on the outer loop.
And between the electrodes, you're going to have an ionic current.
So you have a complete circuit around your loop.
And this is the effect that the simulation is trying to simulate.
The simulation works on a very low physical level, and it stimulates the chemical reactions, which at the lead dioxide electrode, it converts lead dioxide into lead sulfate during discharge, reverses it during charge.
During discharge, the lead electrode turns lead into lead sulfate.
And during recharge, it converts that back.
To implement the model, we took basically and discretized the system and just kind of cross-sectioned it-- took small sections and cross-sectioned it.
We're going to skip this slide.
I don't know what that is.
But basically, to simulate the physics, we have a bunch of nasty, nonlinear coupled partial differential equations.
This is an example of two of them.
And there are a number of other ones that are included in the system but not really worth seeing.
Just kind of a feel, you see a diffusion term.
In the lower equation, you see a diffusion term.
You see a reaction term.
And the upper equation, you see changing of the porosity, changing the geometry of the system.
This is kind of the output that you could get from this type of battery simulation.
And what you're looking at here is you're looking at-- if the left-hand side is the electrodes, is the top of the battery, and the height of the battery goes along that way, you see that as you discharge, the concentration at the top of the electrodes is much-- the acid is consumed where you're drawing current out most readily.
So physically, the simulation kind of makes sense, even if my speech didn't.
The battery model that we used is a two-dimensional model.
It has both a lead dioxide electrode and a lead electrode and an electrolyte area.
This is an example of just the lead dioxide electrode.
And if we discretize that in two dimensions, we would get a discretization that looks like that if we were to use a single-centered finite-volume method.
But it turns out that we have so many discontinuities in here that it's much better to use a staggered grid.
And so we're using actually two different meshes to simulate this system.
Well, on a single mesh, you would end up having four corners, four edges and a center.
So you would get nine different types of objects and one electrode alone.
And you're talking about a rather large and nasty simulation.
On a small scale, the simulation would produce a very sparse matrix that looks something like this.
And Sudarshan will now come up and talk to you about the matrices and how we solved this system
[INAUDIBLE] slight context [INAUDIBLE] speakers.
So basically, like John said, the problem, we are essentially trying to parallelize the most computationally expensive part of this whole simulation, this solving for the Newton iterations during each [?
lower ?]
step.
And so basically, in an abstract-- I'm sorry.
Any questions?
Speak louder
Sorry?
Louder
[INAUDIBLE].
The microphone [INAUDIBLE]
I'll try, but you've got to try and listen harder too
That only works for the TV.
You have to yell at them
I got it, yeah.
Is it any better now?
No, you've got to yell at them
Is it any better now?
Yeah
OK, cool
Yeah, I can hear
So-- [LAUGHTER] Yeah, I always like to say I'll try to speak harder.
You should try to listen harder, too.
So we meet somewhere in between.
So basically in an abstract sense, we are trying to solve a system of sparse linear equations.
And for simplicity, for the first pass, we treated the matrix as being dense.
So we did the whole Gauss elimination ignoring all the zeroes of the matrix.
And so the basic idea is that we do Gauss elimination with partial pivoting.
And it's the forward elimination stage of the Gauss elimination that [?
Rn cubed ?]
that is most expensive.
And that is the thing we've tried to parallelize between among the SPUs.
And the partial pivoting and the back subs are done on the PPUs because that turned out to be much faster.
So this is the matrix that we are considering during each step for the random entries and for the random right-hand side.
And then so the way Gauss elimination works is that we start off with one of the rows that you call the base row, which is where the pivots come from.
And then we eliminate the variable corresponding to base row from all the subsequent elimination rows.
And all these eliminations can be done in parallel.
And that's what we have tried to parallelize.
So each SPU grabs hold of the next available elimination row, eliminates it, and writes a result back on to the main memory.
So if, for example, if we consider the situation with the two SPUs, what we do is that we stream the elimination rows across all the SPUs.
Each SPU performs the elimination and writes the result back to main memory, as you can see.
And this happens for every row.
So in this simulation, the first variable has been eliminated.
As you can see, there are zeroes along the first column.
And the process is recursive.
After the first variable is completed, it moves on to the next variable.
And the way the elimination rows are returned, they're delivered in a cyclical fashion.
So there is no serial bottleneck.
It's all perfectly load balanced
Have you--
I'm sorry?
--change?
I'm sorry?
If you eliminate something lower, doesn't the pivot value change?
I'm completely not getting what you are saying.
That's my first question
Yes
Second question is, is this sparse or dense?
What's your matrix representation?
The matrix representation currently is dense

Yeah, so we've--
Doesn't the pivot value keep changing when you eliminate?
So if it's completely parallel, and doesn't that [INAUDIBLE]?
So all the subsequent variables that are eliminated for each variable, all can be done in parallel.
But after one variable has been eliminated, there's a synchronization step where all the SPUs have to synchronize.
There's a barrier after each row has been eliminated

Yeah, so you start it-- so there's a totally [INAUDIBLE], if you write the algorithm down.
And then it's the inner two loops that are totally parallelized with.
And the second loop is what we're trying to parallelize

Is that a little clearer?
Yeah.
So this is the basic algorithm.
And I'll let John talk about some of the performance numbers we've got.
But some of the optimizations that we haven't done are the use of the [?
last three ?]
operations, like extreme multiple rows at a time and use of [INAUDIBLE] operations.
But I'll let John talk about the performance numbers
Computation of the current elimination row with the fetching of the next elimination row, we get roughly a 30% speedup.
So here's a graph to see how both the unbuffered and buffered version perform scattered with a number of SPUs.
So as you can see, both look roughly pretty linear.
And that makes sense because the-- So here's another graph of how we scaled with the size of the matrix that we're working with.
So the smaller the matrix, the higher the communication/computation ratio.
So the communication latency should be more apparent.
But as you can see, even as you get smaller, the computations is still pretty linear.
So--
Yeah, it's linear.
But it's not just [INAUDIBLE].
It's not running at 2x performance improvement.
But [INAUDIBLE] now.
So when you put 16 there, why is it only 30%?
Why is it not leading to 2x performance improvement?
So is it because of underclock?
Because if it's underclocked, your graph shouldn't have this kind of a pattern.
It should have a more exponential type pattern.
[INAUDIBLE]
Did you understand the question?
Yep
So why aren't you getting a 2x speedup?
I mean, your line is straight, but it's not increasing proportional to the number of SPUs you're increasing.
So your efficiency might not be at 100%.
Any idea why?
Yeah, that [?
looks ?]
horrible
The 2x is only-- this is [INAUDIBLE] [?
laptop ?
], so two would only shift it up and down, right?
So it looks like you're not operating at 100% efficiency, right?
Oh, definitely not
Yeah
So I think to--
[INAUDIBLE] conversation going on there.
But this a strange type of a graph because if [INAUDIBLE] as you get a very nice linear [INAUDIBLE].
If you have [?
undersize ?]
coordinate, what you get is you have a good speed at very beginning, and then you slow down.
You get a [INAUDIBLE] type of a graph.
So it's very much-- I haven't seen things like a strange line, but not as [INAUDIBLE] on your graph.
This is kind of a strange thing.
[INAUDIBLE] to see where and why does it happen
OK. Well, yeah, so I'll just move on.
So I'll talk about some of the further work we're trying to do
John?
Yeah?
Can you clip the mic to your-- yeah, that'll work
So we've also tried to triple buffer by pipelining the fetching of the next row with the computation of the current row with the sending of the previous row.
We did this, but it didn't actually give us the speedup that we thought we would get.
But what we're working on how is SIMDizing the computation stage.
So right now, we treat all the doubles in the rows as scalars.
But by SIMDizing, hopefully we can get a speedup by a factor of two.
And also right now, our LU algorithm isn't very smart in that when we fetch the elimination row, as we go on through the algorithm, part of the elimination row becomes zeroes as we move down.
So we don't really need to fetch the entire elimination row.
And so by taking that into account, our DMA transfers are smaller.
And maybe that will help speed up things a bit
So is the whole thing using double precision?
Yes.
It's in double precision
Do you have any megaflops you're getting?
We have gigaflops.
I think we're getting around 2 gigaflops
So here's the gigaflops graph
Do you know what the expected for [INAUDIBLE]?
[INAUDIBLE] algorithm bias, just [INAUDIBLE]
This is a linear scalar on the y-axis?
Yeah, it's [INAUDIBLE]
Any idea why you have this [INAUDIBLE] at 3 SGUs?
Your double buffering result just drops
Actually, we have no idea.
[INAUDIBLE] I don't know if we consistently checked [INAUDIBLE]
You might want to run it a few more times.
There is a recent series of papers by Jack Dongarra's group aimed at double-precision computations.
They used single precision, did an approximate guess, and then did a few iterations of an iterative approach to get a double-precision result
Yeah, those don't work very well for [INAUDIBLE].
And this is conditioned around 10 to the 8.
So that's kind of out of the ballpark of the [INAUDIBLE]
And the particle [INAUDIBLE] that they do, they tried to do [INAUDIBLE] computation.
And the main reason they do [INAUDIBLE] because they say [INAUDIBLE].
And then in doing this, the hope is to generalize this to sparse matrices later on.
And I think they are trying to do some work with sparse matrices, but they haven't gotten there
One additional question.
Just at a high level, is this the right way to tackle this particular problem?
It strikes me that there must be a closed-form high-level microscopic model for battery self-discharge.
And I don't know anything about the chemistry lead acid batteries, but the little write-up on the website mentions Black-Scholes, which I do know something about.
And of course, there are efficient ways of doing that
Sure, there are efficient ways of doing Black-Scholes.
There are different ways of doing this.
This is only one of the equations that's similar to Black-Scholes.
There are multiple equations that go into this.
They're all coupled.
They're all nonlinear.
So solving this system of equations is, in the literature, if you were going to choose this as your model, is--
But if you were actually in the business of building and operating batteries--
The problem with--
--wouldn't you try and come up with an efficient microscopic model rather than--
Well, it depends on what your compute costs are.
I mean, if you're going to run a model on a superscalar processor, that' very expensive processor.
If, however, now you have the ability to use a collection of small, cheap little processors like this, you can maybe have the luxury of using a much more sophisticated model
Is it necessarily more sophisticated?
I mean, is there some fundamental reason why you'd get more information out of this approach than out of a higher level model?
You get a lot more detail as to what's going on within your battery.
A higher level model will give you I/O information.
But if you're trying to find failure mechanisms, you may want to know how much pressure's going on at certain points in your battery.
Especially with lithium ion batteries, when you-- this differs from a lithium ion battery in that we're actually changing the state, the type of matter that we're using.
And in lithium ion battery, we're intercalating and deintercalating.
And so what we're doing is we're creating a lot of stress on a matrix, a crystal.
And that will be a failure mechanism.
A simple input/output relationship will not give you that failure mechanism.
So you may want to know that kind of information.
You may want to know heat information also.
You could include thermodynamical information to this kind of model
OK, that's a good answer.
I take that to mean that the reason for doing this is if your researching the physics of battery discharge as opposed to trying to estimate how much power's left in your battery
Well, if you have the computation power, you could use the same model that does the research as the car.
And so if you-- yeah, if you can use small processors.
So our demonstration-- if you guys want to see print def statements.
Otherwise--
No, I think we're over time
Yeah
Any last questions?
Wait, just so I understand the sparse versus dense issue, if you just do a sparse elimination on a uniprocessor, do you get a bigger speedup?
Oh, you get a much bigger speedup

The LU is about the slowest thing you could possibly choose, but if it works
OK, but what you have would still usable for dense matrices?
It'd still be useful for dense matrices, yeah
And do you compare it all to just PPU performance on a dense matrix?
No, we didn't compare it to the PPU.
We had it compared-- the same model I'm running on a
Yeah
OK, thank you, speakers.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're the real time ray tracing group.
And as we like to call ourselves, Blue Steel.
So one thing you might be asking yourselves and wondering from us is, where did we get the name Blue Steel?
What does it have to do with this thing we call ray tracing?
And this morning when I was sitting up and finishing up the project, I was asking myself the same thing.
We're doing ray tracing.
Where did we come up with this name?
So I'm going to give you a little background on what Blue Steel is.
The name came from the first day we got access to our PS3, we went to the student center like a bunch of excited kids on Christmas.
And what did we do?
We created a mailing list for our group.
Didn't even mess with the PS3.
Just created the mailing list and sat there for about an hour and a half, thinking of a name.
And somehow, we decided on Zoolander's famous face, Blue Steel.
And it didn't have particular significance to us before, but as you'll see, it has a lot of historical value.
Blue Steel is also an alloy, a ferrous alloy.
As you could see, we have blue tempered spring steel that many companies, industries, use.
And Blue Steel is also a warhead made in the Cold War.
But these have nothing to do with our project.
To us, and for the purpose of this project, Blue Steel is a distributed real time ray tracer.
And Natalia is going to give you a little background on what ray tracing really is
Thank you.
So ray tracing, there are two terms that you would generally hear, ray tracing and ray casting.
Basically, the idea is the camera is like the eye.
And from the camera, rays are cast for each pixel in a picture.
And these rays, they intersect with all the virtual objects in the scene.
When an object is hit, the color of it is computed.
The closest object that is intersected is the object that shows up on the screen
We should credit these slides.
They're Fredo Durand's from 6A37
And ray tracing is recursive.
It accounts for shadows, positions of other objects in the scene.
There is reflection and refraction.
For example, if this ray were to hit that object and then reflect it, then another ray is cast from that object in the reflecting direction.
And this can go on for an indefinite number of reflections.
These are normally specified in the program.
Here is an example of the recursion.
So the three planes are reflective.
They're like mirrors, and there are three spheres running between them.
With no recursion, you can see there is no reflection at all.
With one recursion, you can see the rays are reflecting just off the side mirrors.
At the end with two recursions, there is also reflection at the bottom.
So it is a reflection of one of the reflections.
And this is very computationally intensive because for every pixel, you're casting a ray.
And for every object in the scene, you're intersecting that ray with the object and seeing if it's hit.
And then if it is hit and it is reflective or refractive-- so transparent-- then another ray is cast.
This scene here is a resolution 1024 by 1024 and recursion depth is 100.
And the rendering time in our program was 2.6 seconds.
The rendering time for 20 depth recursion was 600 milliseconds, and for 10 was 300 milliseconds
[INAUDIBLE] SPU [INAUDIBLE].
What would be the [INAUDIBLE]
Many of us have taken 6A37, and in that class, we're all about ray tracers.
And a typical scene like the one that is on the PowerPoint right now could take up to a minute or more.
Often, if you were to do 100 bounces, I think mine would probably take over 10 minutes
[INAUDIBLE] you just to [INAUDIBLE] SPU or [?
one SPU.
?]
We did try different SPUs.
This picture, we didn't try with different SPUs, but we have tried other scenes with various numbers.
And we did get exact linear speed up.
So if there were twice as many SPUs, it would be twice as fast
OK.
So I'm going to talk to you about optimizations that we did.
Natalia's told you about ray tracing, and that's only half of our project.
The part that we have to focus on is optimizing.
And meeting that goal of being real time.
And for us, real time, you have to ask, what is real time?
If we go off of like, the American video standard NTSC, they define it to be around 27, 30 frames a second.
So that was pretty much the goal that we set when we set out to do this.
And so we took the code and we think, OK, we can split this.
This task is easily parallelizable.
But we're going to have to do a lot more than that if we want to get better rates.
So we took advantage of the hardware that we were on.
And one thing that on the ray tracing end, we had to go crazy with, was SIMD.
Pretty much, the entire ray tracer never deals with a scalar value.
We use SIMD for everything.
And in the core of it, we do branch avoidance.
So we really try not to take branches, because branch misses are very costly on the cell.
So I believe there's something like three if statements spread across 4,000 lines of code for the ray tracer.
We really took SIMD to hear in this.
Using SIMD, you can process arrays of structures.
So we took--
[INAUDIBLE]
We would trace ray packets of four rays at a time, and from those ray packets, we would generate hit packets.
And then similarly, as you go down the line, we would produce an RGB packet, which is four color values tied into one.
So at every step, we would use SIMD to our advantage.
Now Brian's going to talk to you about pipe lining
So of particular importance on the cell is not only what instructions you execute, but the order that you execute them in, because there is no reorder buffer on the cell.
There is no out of order execution.
And despite the fact that compiler technology is very good, hand optimization and pipe lining instructions is still always better.
For instance, with our triangle intersection routine, we gained on the order of hundreds of milliseconds of render time.
We shaved hundreds of milliseconds of render time off of our render time for some scenes you will see later, just by reordering our calculations in the intersection code.
So it's very important, especially on the SPEs.
And one other optimization we made was the fact that we coded basically everything on the SPEs.
Anything running on the SPEs is coded in C. Not in C++.
And we're not using inheritance when we do use C++ because virtual functions have more overhead in respect to memory, the V table, and also looking up the function pointers from the V table.
So these are just a couple of the optimizations we made.
We'll talk about more later
Just a quick question.
So for the pipeline, you actually reordered instruction statements?
The compiler wasn't very helpful in that respect?
Yeah.
So on that scene, 1024 by 1024 with the spheres, we were getting, Natalia mentioned with a depth of recursion 10, we were getting 300 milliseconds.
That's with optimized.
Without the optimizations, we were getting closer to 500 with a depth of recursion of 10.
And we gained that extra 200 milliseconds just by reordering instructions in the intersection with optimization level 03 in XLC.
So hand optimization, like Mike Acton said in class, is still very important.
And I'm going to hand you off to Mikey now to discuss the regular grid.
MICHAEL D'
So one of the problems, I guess, with ray tracing is that intersections are very expensive.
And when you have scenes with n rays and m primitives, well, you have to generate n times m intersections.
So one common acceleration structure is grid.
And the concept is very simple, as the picture depicts.
You just take your screen, split it up into a number of evenly size box holes, place your primitives in them, and then you march your ray through.
And you only intersect the ray with primitives that are around the area.
And the good thing about a grid is that it's very easy to construct, as opposed to, say, a k-d tree or an octree.
Those take on the order of n log n, where n is number of primitives.
Regular grid is order n, where n is the number of primitives.
And the reason why we did this is because with an interactive scene where things are moving around, you need to rebuild the grid every frame or whenever something moves.
So we favored this choice of acceleration structure because it was quick to build and easy to traverse.
And now Fish is going to tell you about the fun part of the project
I was in charge of building the actual physics engine.
Our original plan was to the 6170 well-known gizmo ball in 3D and ray traced.
And I guess if we want to show that?
I have it running on this laptop.
We didn't quite get it ray traced, but I'm just going to show you a little bit of what I did as a physics engine.
Maybe not
We'll get the demo working and then I think we can use it later
Anyway, it was basically a 2 and 1/2 D version of the example that was just up on the screen.
Flippers work, absorbers work, and due to complications, we never did get it fully ray traced.
And I guess we'll go ahead and give you the real ray trace demo, since we do have a couple of those
So their actual demo will be on a little LCD screen here.
Hopefully everybody can see the ray trace.
Unfortunately, PS3 needs a specific kind of digital connection to display things in the right resolution when you're writing through Frame Buffer.
So we can't project it onto the big screen
[INAUDIBLE]
Can I do it from here?
Perfect
Now you got to tell [INAUDIBLE].
Oh, no.
Wrong way
Really?
By the way, he's still adjusting it
Whoops
Uh oh.
Oh, no
OK. We've prepared a few demos for you.
And these are really just going to show off the ray tracing part of it and the rates that we got after we applied our speed ups.
Thank you.
Here we see both the camera is moving and we implemented full scene functional textures, so you see the checkerboard.
It's generated every frame.
We used no texture maps.
This is completely generated functionally.
Bump mapping, as you can see in the large orange sphere, is also done on the fly, as well as blending between two textures using Perlin noise is done on the fly also.
Showcased in the center sphere right here, you can see both reflection and refraction.
The sphere is transparent.
It acts as a glass sphere.
So you can see the reflection of the orange in it, but also the warping of the checkerboard as it goes through.
This is one of the advantages of-- well, spheres are one of the advantages of ray tracing, because you can use geometrical equations to define your geometry versus in using rasterization methods, you have to actually define the geometry in some concrete numerical way.
Going to go to the next demo.
This should showcase some of the lights that we implemented
So how many frames per second is this?
This should be 15 frames a second.
And let me reiterate.
The refraction step requires at least two bounces, so two steps of recursion.
If we were to turn that off, we would be achieving, I would estimate, 27 frames a second, 28.
But then it doesn't look as pretty, so, you know, what's the point.
You can see-- this will be showcased later-- but you can see the spotlight.
It's kind of moving.
It's waving back and forth, illuminating parts of the scene.
There's also a point light at some distance around here that's illuminating everything.
But it's basically the same set of primitives.
If you are closer, you could probably notice that the checkerboard is fully reflective.
This far wall, you can view the entire scene in it.
Going to go to the next demo.
This is with the lights off.
So you can get a better sense of what the spotlight is actually doing.
If you notice on this orange sphere, we implemented two different forms of BRDFs.
One was generic foam shading.
And then the other was Cook-Torrance, which helps particularly with metals and other materials like that.
It helps have very bright specular highlight.
And you'll notice as the camera's panning back and forth, the speed ups that we get from-- we used little tricks to glean speed here and there.
Like when it's so dark or it's very far away, we stop recursing
We're over, so--
We are?
Yeah.
So if there's any--
Let's see
Do the SPUs from one to six
I can demonstrate-- we actually enabled it so that you can turn it on either one through six SPEs using command line arguments.
Let's go back to the first demo.
If I say just use one SPE, we can watch it slowly trod along.
And if I were SHH-ed in, it would be giving me timing information so I could tell you exactly.
It's really unfortunate that it's not, because it's very linear in its speed up.
Right now it says 500 milliseconds.
But if I turn it on two, it drops literally in half to 250.
You can get a sense of that, that it's moving like tick, tick, tick, tick versus tick, tick, tick.
And then three, it's a similar speedup.
And then I'll jump to five.
It's getting closer.
And then when we turn on the sixth, this should be about 12, 15 frames a second
So you estimate if you had a dozen SPUs, you could go--
Actually, it's funny you mentioned that.
I think it was the Julia re-tracer or the
Both of them
Both of them use two cell blades, which is a full eight?
MICHAEL D'
It's a full eight times--
Eight cores times two.
So they use 16 SPUs.
The PlayStation 3 do have certain factors.
We're only limited to six
The way you're dividing up the work, each SPU taking every sixth raster line, that sounds like-- could you improve on the performance by allocating the rays--
The issue there is one of granularity of our distribution.
And also communication time on the bus.
We've designed it such that we can fine tune it in any way we want
But it seems to me if you had a localized bundle of rays on a single SPU, you might be able to do some optimizations, like treat them as fat rays.
Nearby rays are likely to intersect the same triangles.
MICHAEL D'
Choosing the frosting variety
The entire material system works very much like that.
It actually saves on computations based on which rays in a packet of four have collided with the same element.
And if it does, then it takes full advantage of SIMD to quickly chug through those rays that are the same and then never call that particular shader again, because it's already done the work for them.
MICHAEL D'
Another issue, I think, is--
Sorry, we should try to wrap so we can get through the others.
Any more very short questions?
Did you guys have any last concluding statements?
Maybe we go through this like really fast?
[INTERPOSING VOICES]
Point is we had two pieces
Thank the speaker
Thank you.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I'm Arvind and this is Micah.
And we worked together on a framework for flexible stream processing on the cell.
So we were also oriented towards an application which is, in this case, a software radio.
And we used this application as a case study.
Where is the mic?
I'll try and speak as loud as I can.
So our project was about flexible stream processing framework for the cell.
And we used the software radios as a case study, essentially, of an application you could build using this framework.
The motivation for our project is essentially what we've been discussing in class.
It's been reiterated.
The cell isn't easy to program.
There's no shared memory.
There's just message passing, which is quite messy.
You have to explicitly parallelize your programs if you want to write them as, say, custom C programs.
Some of the groups have describe the challenges they faced when doing that.
Extracting parallelism can be tricky.
For example, if you want to do pipelining, then on DSPs you have to predict what addresses you're going to require, and set up a DMA so that those addresses are fetched in advance.
And as Bill mentioned in his talk in class, stream programming can help alleviate some of these issues for some applications, like software radio where the applications fit naturally into the streaming model.
So what we tried to do for our project was build a light weight-- as light weight as possible because the code has to fit on DSPs, but as expressive as possible, as well, so as to simplify life for developers.
The streaming framework which is targeted specifically at signal processing applications.
The data model, at least, is based on a research project that I have worked on in the past, which is a WaveScope streaming database management system.
It is is a research project with the database group.
The data model is essentially an extension of the streaming model to handle larger blocks of data so as to process them better.
For example, several signal processing operators, like the fast Fourier transform, need to do multiple passes over the data and, therefore, trying to treat streams on a sample by sample basis leads to high cost, scheduling overhead, as well as inefficiency.
So that's what the data model does.
And we tried to port this data model over to the cell processor and see how well we could exploit the features of the cell processor.
In particular, the high on-chip bandwidth between the SPEs to do with streaming.
So our case study was, as we mentioned, a simple software radio application.
It's really simple.
It uses incoherent demodulation, as well as just simple amplitude-shift key modulation.
As I said, the main goals of our framework were to simplify life and modeling as much parallelism as possible-- try and extract pipeline parallelism, data parallelism.
Some of the kinds of parallelism, We'll mention.
The kind of parallelism we were able to finally implement-- so far, it's only pipeline parallelism, but as future work, we'd be interested in the other kinds of parallelism as well.
More about it as I go on.
So in the framework we at least implemented, the programming model is quite simple.
The basic execution unit is what we call an operator.
It's analogous to what in StreamIt would be called a work function, or what in GNURadio, which is a framework for building soft radios, would be called a block.
So these operators can be any arbitrary C++ classes with state, and they implement an iterate method which the developer has to overload in order to process a block of data.
The WaveScope data model also provides a library for managing memory and passing blocks of signal data between these operators.
Applications in this model are built by chaining operators together.
So this is a snippet of some of the code we wrote for the software radio, roughly.
So you create a box, let's say a FIRFilter, which processes elements of type float.
And then, it takes in some arguments, initialize the filters, parameters, and so on.
You want to create a white noise generator and hook up the filter to the white noise generator.
We use this to simulate a simple channel, [INAUDIBLE] channel.
So I'll just describe the components of our framework.
We have a lightweight scheduler on both the PPE, as well as the SPEs.
Right now, it uses a static operator mapping in the sense that you have to specify a static configuration file, where you say this operator name will run on this [?
SPU ?]
number.
But we've not completely implemented dynamically reconfiguring at runtime.
And we haven't yet seen the need for doing that.
So it wouldn't be too hard to add if needed.
But right now, you can easily shuffle around the operator mapping by tweaking the configuration file.
Signal blocks, as I said, were adapted from WaveScope.
They use reference counting and avoid in-memory copies, which can be quite expensive, especially on the cell.
They also provide a convenient API to manipulate signals.
So you don't have to do much of the memory management yourself or debug any of the hard problems to do with memory management.
We also ported this library to ensure that data is aligned for you automatically and transported via queues.
So one of the major things we had to implement was a queuing library and remote heap management-- what amounted essentially to a remote heap management library.
So in some sense, we faced a choice here.
We could either have the PPE control and allocate memory statically and make all the decisions about what memory is allocated where.
Or, we could have the SPEs manage it themselves.
We decided to go for the latter, partly because it was more dynamic and, also, because we weren't sure what the implications of all the control flow passing through the PPE were for this.
So we chose the second approach, which is autonomous memory management.
So when an SPE sends a streaming data element to another SPE, it doesn't have to actually explicitly request the other SPE for allocating memory.
It has a remote heap interface so that it can directly allocate data and write to the SPE.
This is currently of fixed size, but it could be improved by using this heap to share a bunch of queues.
Right now, we have one remote heap for each queue between operators on different SPEs.
So our system automatically handles pipelining streaming data from SPE to SPE using the DMA API.
So Micah will takeover from here and describe the software radio implementation briefly
So our software radio implementation is relatively simple.
We weren't pushing on that too hard, especially because it took the vast majority of our time just to get the framework to work.
Saving the programmer trouble meant that we inherited a lot of trouble ourselves.
It breaks down to about 25 boxes, which we took in our config file which is manually mapped to the SPEs.
About 3,000 lines of code, most of which is framework.
So I guess, if we were more put-together, we'd have a nice diagram to show you.
There's enough time.
I'll try to draw a quick diagram on the chalkboard.
We're simulating both sender, receiver and a channel.
So the computation in question looks something like you have a bitstream.
You need to take these and convert them into some-- can you read anything?
So you take stream of bits in.
Take bits.
Basically, use a lookup mechanism to convert it to an analog waveform, and filter that to produce something that has a narrower spectrum.
Running out of space here.
Means that.
Multiply that against a sine wave
That's for modulation
And so you get-- you've probably seen pictures of this.
It's very much like AM It's basically binary AM.
[INAUDIBLE].
It's one of the simplest things you can do.
Then, this is to simulate a channel, which is a random FIR filter, finite impulse response.
What that means, it's basically taking the copies of the input at different time offsets with random coefficients and just summing them up.
It's a huge multiply add computation
[INAUDIBLE] So this is 80 taps.
And add some Gaussian noise.
And then this, we take over, and then try to figure out what we put in in the first place.
So a bunch of filtering.
Again, more finite impulse response filtering.
There's a little closed loop that tries to estimate the signal amplitude and correct for it.
That's called automatic gain control to sort of keep it constant.
And I'm probably getting these things a little out of order.
This is the incoherent demodulation part.
We square the signal to get rid of the carrier.
Automatic gain control.
Filtering.
There's another loop.
This is called a phase lock loop.
The idea is try to match a sine wave to some input signal.
I don't know how to explain it very well.
It's basically a locking type of detector.
The idea is to lock into the phase of some periodic thing.
This is for recovering when do you sample.
Because you've got this messy waveform, you've got to know when to look and, say, OK, is it high, is it low to get a bit out
[INAUDIBLE]
Yeah.
I think that gives the picture.
I'm probably boring everybody.
Here's a picture generated from the system.
So the green line is the data in.
Hi, low.
[INAUDIBLE].
The red line is the analog signal out right before it's supposed to decide what the heck the input was.
This is after squaring, and filtering, and automatic gain control, and all that.
The little blips are actually because we used a modulation called alternate-mark-inversion.
It basically flips every one.
That's why it's blipping instead of being constant, which is to be [INAUDIBLE].
And then the little blue daggers are the results of the phase lock loop to try and recover when to sample.
And they're kind of off, but they're kind of right.
And so, if you take the little blue blip, if the red line is above 0, it's a 1.
And if it's below 0, it's 0.
And that's how you get your bits out.
This was hard to get right, mostly because of the framework issues.
Implementing distributed objects on a system without real shared memory is hard because you have to serialize everything into a stream of bits and deserialize.
So it really makes pie out of any existing object oriented code.
We did quite a bit of work to get decent lock-free almost zero-copy.
Another day or so, we probably would have gotten zero-copy-- transfer-- streaming of the data from place to place.
And we had to keep the code footprint low.
C++ is bloated.
We don't have an overlay system yet to-- if SPE is not running a particular box, it still has to have the code for it.
So we don't have any infrastructure for--
There's some macros to get around that, right?
Yeah.
It's pretty messy.
So all the code is on all the SPEs.
So code bloat is particularly a issue.
And XLC has this particular penchant for runtime type information and exception handling.
Incredible amount of voodoo is necessary to get them and that 70 K of useless bloat out of there
We pretty much have the--
It works, but not always
We did manage to get it running long enough to get some measurements
Yeah, we did get some decent data out.
Running on the PPE only, we can about 170,000 samples per second through.
And with the scheduling file, that's kind of rule of thumb-- we just roughly said, OK, that looks about this big.
We'll throw it on this SPE, SPE, SPE.
We got roughly four times that using five SPEs
The core foorprints are really large, but-- [INTERPOSING VOICES]
We really had to push down things like our queue.
We just didn't have enough memory.
It sucked.
We basically just said that already
Some performance bottlenecks, I guess.
[INTERPOSING VOICES]
Interesting performance behavior.
We found that the SPEs are ridiculously underutilized.
Most of the algorithms are quite a bit zippier on the SPEs.
And so, they may be running about a third of the time, and the rest of time just waiting for input.
And then the PPE, which is doing only a tiny amount of the computation-- basically just feeding it in and sucking it out-- is spending half of its time all busy and, the other half of the time, stuck in flow control waiting for queue space, which is-- our flow control algorithm sucks.
Better with time.
So it seems like you should be able to do quite a bit better than we did with a bit more work.
We need to cut down the footprint.
And once we have a little bit of breathing room and get rid of the nasty race bugs and such, we can finally investigate what was our original, pie in the sky goal of automatically deciding what goes where, taking measurements of the performance and then feeding that back to producing a better placement of operators, and applying data parallelism by instantiating operators on multiple different SPEs and splitting the data stream.
And just doing more
Since your PPE is at 40% to 50% utilization, did you put actual work on there?
We did put some work on there.
Actually, we put work on there because we couldn't fit all the boxes-- the code for all the boxes under the SPEs.
So we started strategically to put a few things at the beginning and a few things at the end which weren't supposed to be very computationally intensive, and yet they managed to take up half the CPU.
The issue with the other half of the CPU is that it's actually inside an inner loop deep recursive in blocking because, basically, our back pressure isn't online yet.
So if it tries to emit something to a queue, and that queue is full, it just stops.
And it really could be running a whole bunch of other stuff, but it's not smart enough to do that ahead
[INAUDIBLE]
Unlike a model like StreamIt, everything here is asynchronous and code driven.
The SPUs they can decide on the fly how much to emit.
The programmer doesn't have to declare anything, But it means everything's asynchronous.
And so you basically [?
race the issues ?]
galore
But in this a application, do you find any dynamic rates?
In this application, there's not much.
It's a very simple application.
If we actually went to packetization, error correction, compression, things like that, we'd probably see a lot more of that.
This is definitely underutilized in the asynchronous capabilities of the system
In the case of radio, wouldn't it be OK to draw [INAUDIBLE] frame into audio data just because you're spending so much time waiting, it'd be better just to relieve pressure on the queues by just dropping some frames that are unnecessary
It might well be
And interpolating it in the end to try to fix it up a little bit
It might well be.
We decided not to do that as a part of the framework because [INAUDIBLE] policy decision, and we didn't want to make that for all possible applications.
But if we figure out a good way to dispose that, that definitely would be an option.
Just drop a few packets.
Drop a few samples
So what about buffer sizes?
Do you declare the buffer size as well?
Yeah.
The way we have it now-- there's actually just a #define in the code that says all buffers are this size
Do you need any double buffering in there?
You don't need it.
Well, actually, the way it works is that there's this remote heap input queue.
And an upstream SPE just DMAs things in as it feels like.
And then the downstream SPE looks.
Is there something here?
Grab it, use it.
So it just works, however much buffering there is
The ring buffer and the [INAUDIBLE] should get that block [?
free.
?]
That's a benefit of the asynchronous approach.
It just works, if you have memory, which we don't.
Queues are tiny
[INAUDIBLE] is angrily telling me that his connection dropped but he's not picking up the phone.
Yeah, I could see it in there
Any other questions for them?
Did you want to do the demo?
Nah
What did you do to get race conditions?
How did we manage to get ourselves race conditions?
Well, it's a mix of the fact that all the SPUs can essentially operate as independent threads and are sort of asynchronously DMAing things to each other and somewhat poor-- legacy driven architectural decisions on the way the PPU code works.
Because we ported a lot of code from the WaveScope platform.
[?
Which wasn't very ?]
well documented
And that was intended to be multithreaded with Pthreads, which in our original, naive before we actually started the class impression, we thought we were just going to port the whole thing and use Pthreads on the sub, which of course is not possible.
So there were still some threaded components in there.
And we introduced a lot of bugs trying to port the thing.
See if i can--
The surprising thing was the fact that we had to replace array constructors in order to eliminate [?
RTTI.
?]
And it was just a big deal [INTERPOSING VOICES]
We had to get rid of new, we had to get rid of delete.
We had to get rid of array constructors.
We had to get rid of--
Virtual destructors
--pure virtual functions and virtual destructors.
It was a mess.
I guess one more pithy thing I could say about race conditions is it's incredible how many subtle race conditions and bugs we found in the remote keeping queueing library.
Because there's one-- it's a lock free asynchronous data structure.
There are two threads-- two SPUs reading and writing from it at the same time.
And there's all sorts of little subtleties where if you get it a little bit wrong, you end up with one of the queue pointers overrunning the other guy, and you have basically dangling pointers and things like that.
At least three or four times, we spent a few hours until we discovered that was the cause of mysterious behavior
Anything else?
Thank you.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good morning.
My name is Brown Westrick, and I'm going to be talking to you about the speech synthesis project.
Our main goal for the speech synthesis project was to create simulated speech using a model of the vocal tract in which we would model the flow of air over time.
There's existing software called new speech that already does this.
And we want to deport it to cell and then improve the speech quality that it would afford us by using additional computational cycles.
So again, new speech was originally developed for linguistics research but now it's available for free under the new public license.
It already models airflow in the vocal tract in real time.
What this means is that there are no pre-recorded sounds.
Many speech synthesizers nowadays have very large dictionaries of sounds that they just piece together and then try to smooth the transition between them.
However, this model attempts to actually do what the vocal tract is doing instead of just trying to imitate the end result.
The quality of the speech, of this synthesizer, the way that it exists is not as high as the current ones that use the recorded libraries.
But it has potential to be much better because you have so much finer control over all the different parameters.
Our goal was that we have this software that would make an acceptable speech in real time.
We are hoping it would be able to take advantage of the additional computational power of cell to be able to get an increase in speech quality.
And I will it over to Drew who will tell you about the new speech system
So new speech is made up of three different major parts.
The first of which is just called the new speech engine.
The second of which is, which is probably the largest part of it, which is called Monet.
And then the tube resonance model, which is the final part that actually outputs the sound.
And as [INAUDIBLE], the basic processes is you take a text input standard string and the new speech engine will take care of and transform it into basic phonetic information, which they will then deal with and will take that string and eventually convert it into these what we call vocal tract parameters, which are basically parameters that can be sent to the tube resonance model.
And those parameters will define exactly how this tube, which represents the throat and the nasal tract, changes over time to represent speech sounds.
And with those parameters, you can send a signal through it and create a voice.
So the first part of the example will take a perfectly normal string, like "all your base are belong to us," and transform it into what we call the phonetic format of it.
And you can see it highlighted.
The actual sounds are highlighted, whereas various parameters are also included in the output string, like /w, and you can determine where the words are.
And [INAUDIBLE] which determine where sentences and various phrases end.
And basically, Gnuspeech makes uses of dictionary files as well as some basic linguistic models in order to create this phonetic output from the basic input string.
So having created that phonetic model, you can then send it to Monet, which is by far the largest part of the program, which in turn will take the phonetic information, and as I said, use what a basic diphone file, which takes a very large range of sounds and characters and will then transform these phonetics into direct parameters, which can represent the changing of a throat the entire nasal tract as you voice your own speech.
So Monet has to go through a long process of calculating these phrases given the whole-- the rhythm, the intonation of the phrase that's being given to Monet.
And also, a very important part of the Monet process is by taking each phrase and the postures-- is what we call the output.
Looking at the phrase and piece by piece examining the sounds and realizing that as the postures of the throat change, there are important changes being made between there.
And basically having some sort of-- basically a slow change between them, not a gradual conversion as opposed to a sudden change in the actual shape.
So then having outputted the basic postures, you finally send it to the Tube Resonance Model, or TRM, which will take the vocal tract which is divided into eight sections in this model.
And send a signal off of a sine wave through the modified tube resonance model.
And therefore, all the changes that occur as time goes on and the postures which change the width of the tube at various stages will then cause different speech patterns to come out and basically create an actual speech pattern, which is usually recognized.
So basically you have these three parts from a basic string to phonetics to throat postures until finally you get the actual speech out.
So now I'm handing it over to Joyce to talk a little bit about the resources and algorithms
Well, before I talk about the resources and algorithms, I'll talk a little bit about the TRM, which is the tube resonance model.
So we already talked about how Monet outputs like two parameters based on transitions between different words and postures and so on.
And the tube resonance model actually simulates the physics of the vocal tract.
First you have a glottal source.
If you have done any linguistics, you might have heard the little clicking sound the glottal source makes.
There are different ways to simulate the glottal source.
Now, ideally, the way you have a good, natural glottal source is you have a simulation of the physics between two oscillating masses as you air passes through them.
Now, back in the days when, say, people were doing speech research on gnu speech, actually simulating the physics of glottis was not possible.
So what they did instead was, you know, they would try a half sine wave or they would research the most natural sounding glottal pulse shape, initialize a wave table, and do table lookup on it, updating it with the amplitude and so on to change it a little by little.
So one of our goals was possibly to-- because we harnessed the additional computational power and make more natural sounding speech.
And now I'll talk about allocating the resources.
For example, in new speech Monet, there is not as much computation.
Monet has a lot of rules.
For example, between postures and postures, like different shapes of vocal tracts, you can't just do an linear interpolation to smoothly change.
There are different rules beach in order to change between the postures which greatly affects the speech.
This was much harder to improve on, like to parallelize.
Then the tube resonance model.
It had a lot more computation.
In fact, the thing that took probably most computation was after we got our signal data from the mouth end of the simulation, we would have to up sample or down sample it.
And that was something that had a lot of potential to be parallelized.
However, when you were simulating the tube presence model, you could only update the signal inside the vocal tract incrementally.
If you were to break it up, there was a possibility that there would be a lot of pops in between when you try to space them together.
We thought about trying to resolve that with interpolation between forms.
There were nested loops.
The main synthesized thing had nested loops.
You had a posture.
And then you simulate on the posture and between the postures.
And that took the most computation, as well as updating the glottal wave table.
All right.
Now I will hand it off to Omari to explain the challenges.
SPEAKER 5: So the TRM algorithm, which is-- we're most focused on trying to-- so we most tried to focus on parallelizing the TRM algorithm.
Because both Gnuspeech and Monet are almost entirely just dictionary lookups involving large amounts of memory with not that much computation.
So there wasn't really that much potential for parallelizing those.
So we looked at the tasks that were being done on TRM and profiled them.
And you can see what took most of the time was the noise generator part, like the attempt at the glottal source that was being put into the tubes and the actual updates where the tubes are supposed to be as they were shifting.
And so unfortunately, the main loop as this was updating was very, very fast.
It was about 15 microseconds.
So it would be pretty difficult to update, for example, several SPUs as fast as we needed them to, considering how communication costs affect them.
So parallelism was not very simple to use.
Our main, original idea for this exploiting parallelism was to make a pipeline of the various parts of the TRM model, maybe using three or four SPUs, like one for each part of the throat as a sound would go from one to the next.
So that all of them would be engaged simultaneously, like going from one posture to the next in a linear fashion.
But unfortunately, the timing for this was very, very fast, in the order of about 70 kilohertz, which is many times a second for SPUs to be transferring data back and forth to each other with mailboxes and memory.
So that was somewhat difficult
Unfortunately, with this project, we faced a number of challenges, the first and foremost being that Gnuspeech is written in a programming language most of us weren't familiar with.
And it's huge.
Monet, for example, is 30,000 lines.
It's hardly documented.
And it took a fair amount of time just reading through and figuring out what was going on.
Additionally, because it uses Gnustep, which is a GUI library, the calls are asynchronous and it makes it tremendously difficult to debug as well.
I had tried to convert part of it to C++ to try to get the tube running on one of the SPEs, and that took three days in and of itself.
And I ended up having to toss that attempt out.
Another problem we had was dynamic pointer alignment.
In Objective C, most of the objects are stored as dynamic pointers.
And basically in Objective C, there's also no malloc alignment or anything of that nature.
So we couldn't really transfer any of the objects from Objective C memory area to the SPUs to work on the data and then send them back.
So what is working now?
We are able to do line buffered text in the Gnuspeech engine and translate that to utterances so the-- phonetic pronunciations.
And get to the point where we would execute the tube model.
Unfortunately, one of the-- a bug in Gnuspeech potentially is preventing us from properly executing the tube model right now.
So that's one thing we're having problems with.
Additionally, the tube runs on the PPE.
We've been trying to get the tube to run on the SPE, but it's not going well, partly because of the dynamic pointer alignment issue and partly because of some other things we've run into.
Currently not working.
As Drew had mentioned, there are a lot of dictionary lookups in the preprocessing stage of the pipeline.
And there's a bug in Gnustep where it won't parse the dictionary if it's above a certain size.
The dictionary has I believe 70,000 entries, it takes up almost 3 megabytes.
But if there are more than like 3,000 entries in the dictionary, it just doesn't parse.
And we have no idea why.
So to conclude, this was a tremendously difficult problem.
There are a bunch of data dependencies and the synchronization is very, very close.
However, we feel that with more time and with more experience with the code base, we would have been able to parallelize it.
And the parallelization can almost definitely help the vocal quality in terms of naturalness.
Getting, for example, as Joyce mentioned, a higher quality glottal source.
Speaker identification and vowel identification.
For example, when you pronounce different vowels, sometimes quality the glottal source changes.
And lastly, it would be worth the time to re-write the whole thing from scratch, skipping Gnustep, skipping Objective C, and going with C++ or most likely C for the whole thing.
Thank you.
Any questions?
It sounds like you guys are taking a fine-grained approach in which you're splitting the application across different units.
Since you're synthesizing completely independent words, let's say, could you just run the whole application on an SPU?
There's engineering there, but from a parallelization standpoint, can you just take the whole application and run it, for example, on different words?
So I believe you're suggesting we run the tube on different SPEs and then feed data to the separate instances of the tube from the PPE?
Well, including-- the whole processing backline.
I mean, order the sentences from one sentence to the next?
So the big stumbling block for us was that there isn't currently an Objective C compiler for the SPEs, so we can't run the Objective C code on the SPEs at all
But if you did it from scratch, if you were to write your own-- just throw away all your Objective C and start from scratch, would that be a better parallelization strategy than a fine-grained one?
Possibly.
So one of the disadvantages of splitting up words is that there is state that connects the different-- [INAUDIBLE] continuous state that connects the different postures vocal tract all the way through the utterance.
And so we would have to do possibly some prediction, possibly some interpolation to figure out how to connect the different, separate utterances which should been produced consecutively
Permitting one sentence at a time or something?
Yeah.
That might be an option, yes.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, I'm Eddie Scholtz and this is Mike Fitzgerald and we worked on a backgammon tutor.
And to start off, let's see a show of hands.
Who here knows how to play backgammon?
OK, a few.
So to give you an idea of how it works, it involves both skill and luck.
And it's a two player game.
And the object is to move all your pieces past your opponent's pieces and off the board.
So white's trying to move his pieces this way, down and across.
And brown's trying to move his pieces the other way.
And you move by rolling the dice.
Some important rules are that you can not land on a point on which your opponent has two or more pieces.
And if your opponent only has a single piece there, you can land on it and send it back to the beginning.
So the goals of this project when we started off were, first of all to implement the rules of backgammon and get the game going, and that took a good amount of work.
Next we wanted to create or find the function that evaluates or says how good a given board is.
And then we wanted to be able to look into the future, in order to better determine what move to choose now.
And this is the part of the program that we parallelized.
And finally our last goal was to try to teach the player how to improve by suggesting why one move might be better than another in English.
So to start with, the two basic kinds of board evaluation are just a static evaluator and a neural net.
And the static evaluator works by looking at features of the board and creating a score based on that feature, and then multiplying it by weight based on how important that feature is, and summing them up.
So a program like this was created by Hans Berliner at CMU in 1979, and it was actually the first program to beat a ruling world champion.
And they said the program got a little lucky but it did win.
And one of the important features was it adjusted the weights as the game went on, because certain features became less important as the game progressed and other features became more important.
So the next approach that people have used is framing neural nets to determine how good a board is.
And these have been really successful.
They work by basically playing hundreds of thousands, if not millions of games against itself.
And so it improves this way.
And after about a million and half games it reaches its maximum performance.
And one such board evaluator is Pubeval which was released by Gerry Tesauro in 1993.
And this is the evaluation function that we used for our program.
Looks like we lost network connection.
This is the evaluation function that we used for our program.
And he sort of released this as a benchmark evaluator so that other people in the development community could have something to compare their evaluators against.
And it plays an intermediate level if not a pretty strong level.
So our next goal was to implement the search the looks into the future in order to help choose the best move now.
One of the challenges of this is that the branching factor is so large because you don't know what the dice rolls are going to be in the future.
And like in checkers, the branching factor is 10.
In chess it's 35 to 40.
But in backgammon, for a turn it's around 400.
Another challenge that we faced in the search was that the Pubeval function that we used is not zero sum.
So that produced some challenges that you'll see.
And also there's the question of whether searching deeper actually does produce better play.
And most papers say that the board evaluator is more important than searching deeper.
But they do say that searching into the future does improve play, and especially when you're already at a pretty high level of play where everyone's near the same and playing really well.
A slight increase in performance could make a big difference.
So here's an illustration of the search tree to give you a better idea of how it works.
So say it's X's turn.
There might be 20 or so moves that he can commit.
Now these are a series of moves.
So you roll the two dice, and if they're doubles you get four moves.
Certain turns you may not be able to move at all so you might not have any moves.
So that really varies.
So after those 20 moves the other player, O is going to roll.
And there's 21 different dice combinations he can have.
And then for each of those there's about 20 moves.
And then it's X's roll.
And once again he has 21 dice combinations.
And then X's move.
So this is looking two moves into the future.
So the idea is to build this search tree, and then at this point we're already up to 3.5 million boards.
So you have all these leaf nodes that represent your boards, and you're going to evaluate them with your evaluation function to determine how good they are.
Since it's not zero sum, you have to evaluate it for both player and both of these values need to get passed up as you're going to move back up the tree to determine the best move.
So you start at the bottom and you say that since it's X's move he's going to choose the best possible move given that roll.
And then in order to move up further and calculate the value for the parent node, you say it's equal to the probability of the dice roll for the child node times the value that's stored in that child node.
And then you keep doing this and repeating this.
O picks his best move and sort of sums the expected value.
And then get back up the top and X picks his best move.
So now I'm going to hand it over to Mike who's going to talk about the part of the code that we parallelized
There's a question
Quick question.
Can you use alpha beta pruning?
You know about alpha beta pruning?
Yeah.
People have used that and just like a [INAUDIBLE] search of narrowing it down
Typically by down to the square root of n. So you would have a 20 branch effect rather than a 400.
It seems like a real payoff if you could make it work here.
I don't know if there's some reason on the back end why it's hard
Well, we actually didn't get to that point.
We just spent a lot of time just implementing rules of the game
All right, so the first thing we looked at and said this would be perfect for parallelizing is just the brute force evaluation of those million boards that you get down to at the end.
Which if you run through on one processor just takes a while.
Generally an evaluation of a board, regardless of its state takes the same amount of time.
So we just had to split the number evenly between SPUs.
We had to keep it around a multiple of four just to get the DMA calls to line up nicely with the floats that we were returning and things like that.
So if you're evaluating a million boards which was our number for our benchmarks that we did, each SPU has about 170,000 to handle, and it clearly can't take all those over at once.
So each SPU knows where it's supposed to start and how many it's supposed to do.
And it knows how much it can pull over.
So it does a DMA call to pull in as much as it can process.
It processes those, evaluates them, returns them, and gets to the next bunch.
So that is what we were able to implement.
There's also a good opportunity for double buffering there for pulling them in as it's actually evaluating the previous ones.
So that would be actually a pretty easy step to do next.
But each of those six are roughly finished at the same time and then get back to the PPU to evaluate, to run up that tree.
And another opportunity to make it a bit quicker, which we didn't quite get to would be SIMDizing the evaluation function so we could do four boards at once with vector engines instead of just one.
So the performance that we got when looking at the number of SPUs we were using was actually roughly in proportional to the number of SPUs we were using.
It's actually 18.18 at the top there.
So when using two instead of one we got just about half.
When using three instead of one we got just about a third.
If you just look at this speedup graph it's pretty much one to one from the number of SPUs we were using to the amount of time it took.
So it was a pretty efficient algorithm for splitting those up and evaluating them and pulling them back together.
I'll give you a brief demo.
I'll show you quickly how it looks.
We might have to fudge our network connection a little bit here.
So we used just an [INAUDIBLE] Windows setup to display our [INAUDIBLE] we were using.
So I'm just going play a couple rounds of the display, let's say against the computer.
So you can see the computer just took a couple of moves there.
But basically what we get here is a print out of a text representation of the board.
And then it's my turn.
I rolled a six and a two.
It does its own evaluation and says this is what the best move combo would be for me at this point.
In the ideal tutoring situation we don't let the user pick what they thought was the best and then tell them why not, but right now, with the time we had this is how we did it.
And then it lists all the legal possible moves you can make.
Let's say I want to move from 0.6 to 0.4.
It moves my piece up there, and from 8 to 2 for example.
And the computer makes its couple of moves.
That's the general idea.
You can run the computer against itself as many times, just to test different evaluation functions against each other.
You can also do it without the animation on the GUI so that it runs a lot quicker.
But that's the general idea.
Just to wrap up, four more slides.
The way we went about this basically was, we just got sequential code working on just the PPU, got the rules of the game worked out, all the weird conditionals and things that can happen in backgammon and we fixed out our bugs and our memory leaks and things like that.
We just shelled out the parallel code and got that working on the PPU, then on one SPU, and then on six.
So it was a pretty clean process of doing it.
And we ran into a few walls.
We aren't really backgammon experts or even intermediate players, so it was kind of a new thing for us to make sure we had the rules right, and we weren't doing stupid things here and there.
But also managing the search tree was a big deal.
And looking at what the program has to do in running the game, that's the next thing we want to tackle as far as paralyzing goes.
It takes a while, and it's pretty tough.
We ran into some memory management issues with how we were representing our boards.
We were able to pack them down into, I think 24 byte structure, 20 byte structure to be able to pass more to the SPUs at one time
[INAUDIBLE] What's the general idea for [INAUDIBLE]?
I'm sorry, I missed the beginning of your question because it wasn't on speakerphone
[INAUDIBLE]
Are you talking about moving into the future?
Are you talking about looking into the future?
Yeah.
So we built the code for creating that tree and evaluating the leaf nodes.
We didn't quite have time to finish up--
[INAUDIBLE] I can't hear you through this [INAUDIBLE]
OK. We created the code for constructing the tree and evaluating the leaf nodes.
We haven't quite has time to finish up moving back up the tree.
So at this point we're now looking ahead more than just the moves for now, for like one move

But we got the parallel code working for actually evaluating the boards.
It shouldn't be too hard to--
Just extend that tree
To move back up the tree and see what the best move is now
Yeah, so that's basically as far as the road blocks we hit were
Any other questions?
Yeah, just some future ideas.
One of the big ones is for training, the evaluation functions using neural nets.
That takes a lot of time to play hundreds of thousands, millions of games.
So one of the ideas is to parallelize that to help speed that up.
And I think we're about out of time, so that concludes it.
Are there any questions?
I'm just curious, you said looking far into the future doesn't necessarily help as much as the evaluation function.
Does that mean that AI backgammon is-- I mean how computationally intensive is a good player?
I mean, is it a computationally problem?
Would you benefit from the parallelism in a practical setting?
Well, just looking at how long it takes to go through those million boards, if you're playing a computer you don't want to wait 18 seconds for them to make their move as opposed to--
Oh, so that's time scale of the absolute?
If you just ran on a uni-processor for 20 seconds kind of a thing?
Yeah, once you're doing a million boards.
And also, if you're on the level of competing against really good competitors, just having a deep search tree to get that much more of an edge would make a difference in that situation.
Anyone else?
No?
Great.
Thanks.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Greg was not here yesterday.
He was away.
And so this is work he's largely done on his own again.
He's done a molecular dynamics simulator.
So he's going to tell us a little bit about that, and then after that we'll do course evaluations, hand out some awards, have some cake, and then after that some pizza, and then play same PlayStation 3s.
All yours
OK, so I'll talk about the make general molecular dynamics algorithm and then parallelization approaches.
So molecular dynamics is based on a potential energy function, which includes two terms, bonded terms and non-bonded terms.
And bonded terms are split up into three sub terms.
So don't confuse bonded with-- actually, don't think that these are non-bonded.
These are actually all sort of grouped as bonded.
I know it's a little bit confusing, but right now I'm dealing just with the sort of bonded terms.
And what that means is that it deals with atoms that are connected together covalently.
So for example, the bonds term reflects some energy due to a displacement between two atoms, which are certain distance apart.
There is an equilibrium distance that they like to stay at, and if they vary from that distance the potential increases as a squared of that separation distance.
The angles term represents, basically, the deviation from an equilibrium angle between three atoms.
So if you imagine these three nodes as three atoms, if this angle varies away from a certain equilibrium theta or angle, then the potential energy is also a quadratic term, which tends to bring the three atoms to have a certain equilibrium angle.
And then, finally, the dihedral and improper term is probably one of the more complicated ones.
The improper term basically deals with four atoms.
So if you imagine this is one, two, three, and four.
These three atoms, if you put two planes through them, the angle that those two planes make to each other is given a certain equilibrium angle again.
And this should also be a quadratic term, but the energy also varies quadratically as that angle is deviated from.
An actual dihedral is the rotation along a bond between the two central atoms.
And this energy term has actually a cosine form.
So as you rotate it along this bond, say for example, the energy will very sinusoidally.
And there will be a restoring potential that will tend to make it either a line, or sometimes it will make the atom stagger.
So all of these parameters are very sort of molecule and atom dependent.
And all of these parameters are available, or have been built over many years by different methods like ab initio calculations or experimental calculations.
But basically, if you plug those parameters in, for example, here k represents the spring constant, right?
Because they're basically just like spring constraints more or less.
Or in this case, it will tell you how far up the potential, like what this value will be at the maximum, and will, for example, in this case, will specify how many fluctuations you have in the energy function.
And the other parameters are usually the theta, the equilibrium theta, or the equilibrium, or the equilibrium bond length.
So this is kind of what you can expect for it to look like as molecules will tend to move.
The non bonded terms include van-der-Waals terms and electrostatic terms.
So van-der-Waals forces occur between any two atoms, and if they're far enough apart, they will attract to each other according to this formula.
And if they're too close together, so at equilibrium distance between them the energy is a minimum, but as they get any closer there will be a very strong group repulsive force, which is the r to the 12th term.
And as they get further apart, there is an attraction force, which varies as r to the sixth.
And then the electrostatic force is just a simple Coulombic form.
So you basically take two atoms that are, say oppositely charged, will attract each other.
If they're positively charged they will repel each other.
And it's just a matter of multiplying the charges, and then dividing by r to get the energy.
OK, so to do molecular dynamics, basically, you have to take derivatives of all those formulas.
I just showed you.
And I didn't do that all myself.
It was pretty hard.
I just got some code from some current existing molecular dynamic simulators.
But basically, you take the derivative.
And that basically equates to the force, right?
The typical Newton Newtonian model.
And that gives you the force in every atom, and then using that force and integration scheme, the leapfrog Verlet integration scheme just gives you the velocity at the next half time step.
And then using that velocity at the half time step, the positions of each atom is updated based on the positions at the previous time step, and then some value.
So I will show you a little bit of what it looks like on a simple molecule first.
So here's a sort of simple molecule, and it has just an initial configuration that is just sort of random.
And so one technique that is done is first the geometry is minimized, which means that you just take small steps in the direction of the gradient until you reach the energy minimum.
And if I do that, I sort of reach sort of the minimum configuration that this molecule likes to be in.
And then, if I add some thermal noise, which means that I just set the initial velocities of the atoms to some random value based on the temperature, then the molecule starts sort of jiggling around.
And the nice thing about the Verlet integration scheme is that it's actually just right, like the energy is conserved in the system, although it fluctuates a little bit.
So a lot of integration schemes the energy tends to be either decrease or increase, so the system explodes.
But this integration scheme is very stable.
One thing that you saw there is, so I just talked briefly about kinetic energy.
This is kind of how the initial velocities are set based on the Boltzmann constant and the temperature.
And the other thing that molecular dynamics usually do is called Langevin dynamics.
And in this case, some random velocity vector is added to the normal velocity, which just sort of simulates collisions with other molecules in the environment.
So this makes the simulation look a bit more realistic.
So instead of the molecule just sitting there in space, this random sort of factor adds some movement.
And there's also a damping constant, which sort of opposes the velocity by some constant.
So if you didn't add any more sort of randomness into the system, the system would just eventually go to be completely still.
So there would be no more movement, because this is kind of like a damping sort of factor.
And there are some properties of this random vector that sort of make some physical sense.
For example, it's independent of previously picked random vectors, and it also depends on the temperature.
So that if you keep adding this random factor to the simulation, the temperature will always stay constant.
Another thing to consider is that these molecules, if you try to think of them as they are in a sort of typical biological system, is that they'll be solvated in water.
So in that case the Coulombic term is not actually that simple.
You have to include some sort of dielectric constant.
So for example, one that is typically used and is pretty simple is that the constant is just the distance.
So that makes the electrostatic energy dependent on one over r squared rather than 1 over r. So that means that the electrostatic forces drop off a lot faster as the atoms get further and further apart.
So for example, so two molecules that are oppositely charged, if you just put them in vacuum, they will eventually come close together.
But if you add this term, the electrostatic attraction will be much weaker.
So it will simulate them being solvated in water.
And they will come together eventually but probably a lot slower.
Another thing that is typically-- so I'll talk a little bit about the non-bonded forces right now, which are, again, the electrostatic forces and the van-der-Waal's forces.
So generally, what has to be done is, if you consider any single atom like say this one over here, well, so first of all the bonded forces basically include just the atoms that are connected to it, and that's it.
So it's usually the fastest step in the computation.
But for non-bonded forces, you have to consider basically every other atom in the system.
And I didn't include all of the atoms here, but basically just to give an idea, they could be atoms that are really far apart.
So generally, what's done is there is some cut off radius, outside of which those forces aren't considered.
And because the forces drop off really fast, sometimes that's considered to be good enough for a lot of simulations.
So that's a consideration to take into account when doing these simulations.
OK, so here's the basic molecular dynamics algorithm.
So you specify how many steps you want to run it to.
And each time step is one femtosecond.
So very small.
So usually what happens is you can find the atom pairs.
So that means all the non-bonded interactions, you have to make a list, basically an n squared list.
If you have n atoms, you can expect to have about n squared atom pairs.
But if you consider a cut off, then what's usually done is only atom pairs within that cut off are considered and included into that list.
And that computation is pretty expensive, d it's sometimes only done only every, say every ten steps.
And atoms don't really move that much.
So it's usually pretty accurate.
And if you use a cut off of, say 15 angstroms, and then, well, the forces have dropped to almost zero by 12 angstroms, then that's pretty accurate.
So this is usually an optimization.
It optimizes the simulation a little bit.
The next step is to compute the bonded forces.
So that includes the bonds, angles, dihedrals and improper angles.
And then compute the non-bonded forces, so that iterates over all the atom pairs.
And then the integration is done to obtain the new atom positions.
So here's, running on the PlayStation, this is kind of roughly how this is just running on the PPU alone.
So finding the atom pairs, if I don't use a cut off, I can just compute the atom pairs list once, and then never do it again.
So that's why I put here 0.
But otherwise, generally takes a lot of time to compute the atom pairs, because it involves getting the distances between every two atoms.
And this is a pretty big system, maybe about 1,000 atoms.
So that's why.
So these times are not really representative of smaller systems
What is BF [INAUDIBLE]?
BF, so that's done using the brute force approach.
So that means that you check every two atoms.
And this is using a KD tree, so that breaks space up.
So then checking the neighboring atoms is n log n. Or it's actually a constant factor after you've built the tree.
Building the tree is n log n. So using that kind of an algorithm, you get much better improvement.
So this is a scaling of increasing the system size by two in the number of atoms.
So say going from 1,000 to 2,000 atoms.
The time too, using a brute force approach, increases exponentially.
Whereas, using a KD tree, it actually just about doubles.
So that kind of shows that using a better algorithm sometimes for these things is probably much better than trying to parallelize it.
Doing the bonded forces, say about 50 milliseconds, and with cut off it's the same time, because it doesn't really affect the step.
But then computing the non-bonded forces is actually the biggest chunk of time.
So that takes about 1.4 seconds for the system.
And then this is considering a million pairs.
With a cut off there is less pairs to consider, but the time is still the dominant time basically.
So generally, for molecular dynamic simulations this is the dominant time factor.
And that's what people really focus on, trying to speed things up.
And then the integration is really fast, because it just loops over the atoms once.
So there is basically three different parallelization approaches that I've come across in the literature.
And one of them is referred to as force decomposition.
So if you think of a force operation being, say a non-bonded force operation where you consider two atoms, and you compute the force the two, van-der-Waals and electrostatic forces, one easy way to split this up amongst, say six processors, is to just give each processor, say n divided by 6 of these force operations.
So you have to send each processor both atom positions, and then some other properties of the atoms, such as the mass and-- actually not the mass, but the charges and the minimum radius.
And basically, once that computation is done, the processor just has to either store or return the force on both atom.
And it has to come back to sort of like the main processor where all the forces are added up, and then the integration is done.
So what I just described is sort of the approach I took.
And it's sort of an easy approach to sort of think of if you couldn't store, say every atom on every processor.
Because you just have to send each processor a certain number of these force computation units.
And the processor does then and then returns the forces.
And then you can just add up all the forces on every atom and do that.
So basically here is the algorithm how it works roughly.
And so basically, first set up the SPUs, send the control box with, say the addresses where a these force operations will be stored.
The bonded forces are computed on the PPU, just for simplicity for now.
And then the computation for the non-bonded forces, basically, all the non-bonded forces operations remaining are considered.
And then they're split up into blocks.
So each SPU sent a block with, say a number of force operations.
And in this case it was 200.
So the control block is sent to the SPU, the SPU is told to start processing.
And then the PPU waits for each SPU to finish.
And then once the SPU is finished, the forces are added to the two atoms that are involved in that force computation.
And then this loop is done once all the SPUs are finished.
And then it goes, in this repeats until there is non-bonded operations remaining.
So for example here, just to illustrate this, let's say these are all the non-bonded operations I have to do.
And I'm going to use two SPUs.
And each SPU can only store, say three force operations.
So each SPU is sent by a block.
I mean that, say these three operations are sent to each SPU.
The SPU does them.
The next block is sent to the next SPU at the same time, and then that SPU also does it.
And this is iteration 0 for example.
And this is done until all of the force computations are done.
So here's a timing.
So you might imagine that, as I said, there is there is a lot of communication overhead over here.
So here's how this performs.
So on the PPU, the MD simulation per time step, runs in 310 milliseconds.
On one SPU, the time was up to 630 milliseconds.
So there is a lot of communication overhead here.
On two SPUs it goes back down to 390.
And as the number of SPUs is increased, it gets lower and lower.
But it's not that much better than really running it sequentially.
And the reason is that the way I implemented is pretty rough.
One of the limitations, I mean one very obvious optimization that should be done is that SPU basically waits for this whole block to arrive, and then starts computation.
But really what I should have done is once one force computation arrives, you can already do that computation.
Instead of waiting for all of them to be transferred and then starting to do the operation.
So that I think that could've been a lot better
[INAUDIBLE]
No
[INAUDIBLE]
No.
So right, so this is sort of the simplest approach, I mean sort of a simple approach to implement on this architecture.
And it sort of scales well as the system grows.
So if you can't store every item, for example, in every SPU, this doesn't really care.
The system can grow as large as you want.
There's some other approaches that are done sort of in the literature.
I didn't implement these, but another one is called atomic decomposition.
So here, if you have, say a number of atoms, you send each one of these atoms to one SPU.
And then that SPU is solely responsible for computing the forces on it, and also integrating to get the new positions.
And then all you have to communicate is the atomic positions.
But this is a little bit more complicated, because then every SPU has to know about all of the force computations that you want to do.
So for example, in this case it's just a non-bonded, but the SPU also has to know about--
So that would be like the same example that we did in [INAUDIBLE]
Yeah, it would be.
Yeah, the only problem with that approach is that if the system was much larger than the SPU could store, and you couldn't store every single atom on the SPU, then an operation might refer to atoms that are not there.
So yeah, it would be similar to that.
But it would get more complicated in this case as a system grew, because you'd have to keep track of where each atom is and then implement communication.
If you don't have a certain atom, to get to that atom position and use it in the force computation.
So yeah, this approach, OK, I'll talk a little bit more about this.
This is a--
[INAUDIBLE] I mean not SPU, but [INAUDIBLE]
That's true, yeah.
So you only sort of give a certain number of atoms to every SPU, and then, yeah.
That's true.
And then you would have to sort of store every single force that that atom is included in on the
[INAUDIBLE]
Yeah, like when this will become a problem is, say the system was much larger than you had local store.
Then say you had like 1/10 of the system on SPUs and then 9/10 of the system on the
[INAUDIBLE]
Yeah, it would be hard to think of it scaling properly.
So a special case of this atomic decomposition is actually spatial decomposition.
OK, I'm almost done.
So a special case of spatial decomposition.
So what makes this approach sort of not good, and it sort of involves a lot of communication, is just this fact that some of these force computations, or force operations that you have could include atoms on different units.
So if you don't split up these atoms sort of smartly, then you could end up having to communicate a lot while computing these forces.
So spatial decomposition tries to address this.
And basically, you store atoms that are close to each other.
So say I had the system, and I wanted to split this up on six SPUs, then right, I would give each SPU, say a certain area of space.
But you could see right away what the problem with this is is that it's not load balanced.
So some SPUs could not have too much work to do.
And there's still some communication, because you have to communicate between neighboring SPUs, say some certain cases.
But the communication is still probably a lot better than in the previous case, just for atomic decomposition.
So here's the state of the art on parallelization MD.
And it's implemented in this program called NAMD, which stands for Not Another Molecular Dynamics simulator.
So initially, NAMD 1.0 used solely a spatial decomposition.
And they try to address this load balancing by actually creating many more patches than you have processors.
And then you just sort of split these patches up, load balance them amongst the different processors.
That didn't really work.
So NAMD 1 didn't parallelize very well.
So what they added later was they added this idea of patch objects, and also compute objects.
So basically a patch object puts a number of patches on a processor, but then you can also create compute patches, which include all of these force operations.
And these force operations are also equally distributed amongst processors.
And then there's a lot communication between these two.
And it started off with sort message oriented communication.
Then it moved to a dependency oriented communication so that each compute object would sort of signal which atoms it needs.
And it also sort of optimized things by doing communication and force computation at the same time.
So it did that really well.
And actually it had really good parallel performance.
So on this graph it shows the number of processors times the time per time step.
And as the number of processors is increased, I mean optimal parallel performance would be a straight horizontal line, which is pretty close to what they achieve.
And they also did it on many thousands of processors on Blue Gene.
But as you might expect, the more processors you add, the step time actually still sort of levels off at a certain time.
So I guess I'll skip the sort of live demo I was going to do, but I'll just show one more thing.
So here is sort of the kind of systems that I was looking at.
This is actually a simulation that was done on one of the PPUs and then recorded in positions file.
And there is five different molecules here.
So generally, what happens is first they're just placed randomly.
Then they're an overlap just very coarsely, to make sure that huge forces are not entered into the system.
And I'll just color each one differently.
So here's what happens during the simulation.
And here, in this simulation, actually, the positions were recorded every 20 time steps.
So that's why it sort of seems to go a lot faster.
If I did it on a single molecule, I actually done that first.
So here what a single molecule of this looks like.
And this is a protein with eight residues, which is pretty big.
And this is running in real time on my computer right now.
So like on a small system like this, it runs reasonably fast.
But as you add more and more of these things, the computations do get pretty expensive.
So parallelization really helps.
Well, not in what I implemented, but generally.
Generally, as you saw, NAMD can do a pretty good job at doing it.
Oh OK, I didn't minimize the system well enough.
But yeah, that's about it
Great.
[INAUDIBLE]
So basically, on the SPUs, you managed to get them synchronized properly, data in and data out?
Yeah, exactly.
So that's kind of to the extent that I went.
And that still took a long time.
So [INAUDIBLE].
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So Phil is going to do the next three recitations.
So the first one today is really going to focus on how do you actually debug programs on cell.
How many people in here have used gdb, or are familiar with gdb?
OK, good.
So there's going to be a mini tutorial on how to run programs and attach gdb, which is a debugger that's commonly used for debugging programs.
We'll show you how to debug PPU and SPU programs.
And he's going to talk a little bit about some other things that might help you in terms of performance debugging, finding out where things are not going as well or as fast as you might expect them to.
And the next two recitations, which will be next week, will focus on doing some actual performance-specific optimizations.
So you've written your program.
It's working.
Well, the functionality is there.
But it's not performing as fast as you might like.
Or you think you can improve performance a bit better.
So what can you do?
So he'll show you some tricks and some things you can do to get the performance up.
For some of you who have asked about the Eclipse debugger, the Eclipse IDE, it does run reasonably slowly on the PlayStation 3 because of memory constraints.
So if you have lots of people trying to use it on the PS3, it'll probably be unusable.
So for those of you who are interested, we'll just do separate tutorials offline.
Because I don't know how many people will actually end up using it.
Other than that, [INAUDIBLE]
All right.
So today we'll talk about how to run gdb on Cell.
And then we'll also do some static profiling where you can figure out when instructions are going to be executed.
And then I'll talk a little bit about the dynamic profiling tools that we have available.
So you can use gdb to examine the state of your program.
And this can help you figure out bugs that you might have.
There's two ways that you can use gdb.
The first way is when your program crashes, you can figure out what was happening at the time that it crashed.
Another way that you can use gdb is to kind of run it from the beginning.
You attach gdb to your program, and then you step through all the things, all the instructions that are being executed while the program runs.
In order for gdb to be able to provide useful information about the state of your program, you have to give it a little help.
You need to compile your program using gcc -g. xlc -g will also work.
This puts in some extra information into your program so that when gdb is looking at the state of your program, it's able to figure out what line numbers in the source code that corresponds to.
So you can just add dash g manually to your GCC invocations, or you can also set this compiler-- or you can set this option in the makefile, CC_OPT_LEVEL and set that to CC_OPT_LEVEL_DEBUG.
And that will do the same thing as adding dash g. All right?
Now we have a special version of gdb available for use on the Cell.
ppu-gdb is used for debugging ppu and spu programs after they've been linked together.
And how you invoke it if you're going to run your entire program using gdb is you type ppu-gdb and then the name of your program.
And what you get is a gdb prompt where you can issue additional commands to the debugger to control execution of your program, and also ask the debugger to tell you various things about your program.
One thing you can do is export SPU_INFO=1 before you start your program.
And that will print some extra debugging information about when threads are being created and destroyed.
Like I mentioned, you can also use gdb to attach to a program that's already running.
And if you want to do that you still need to provide the executable name.
And you also provide the process ID of the program to attach to
Do people know how to get the process ID?
[INAUDIBLE] How many people don't know?
The easiest way to do it is if your program's running just type "top."
And that'll actually show you what's running on your machine.
And there'll be a column that has a process ID.
Another thing you can do is just type "ps."
We'll add those to the recitation slides.
[INAUDIBLE]
So I forgot to mention.
If you're invoking your program with gdb, then after you type ppu-gdb with your executable name, gdb is just going to sit there and wait for instructions.
And what you can do is type "run" and that will run your program normally.
But the difference is that gdb will pick up and if something goes wrong.
And then that's where you can step in and look at the program state.
So what kind of things can you figure out about your program?
Anytime you have this gdb prompt and your program is running, you can type "bt" to get a stack trace for your program.
And this will tell you which functions are calling which functions.
It's going to give you a list of frames.
And the top one is going to correspond to the deepest level of function nesting, or the deepest function call in the program.
And of course, the last one on the list is going to be the main or the first function that was called.
That make sense?
And for each stack frame gdb will tell you the name of the function.
And if source code is available it will tell you also which file the function is running in right now, and at which line number.
And so using that information you can figure out exactly where it was that this function called the next function and where was that each function called the next function after that.
Now whenever you've halted the program state, you can get information about what the local variables are.
And to do that you can type "info locals" and it will print a list of all the local variables and their values.
gdb, because it gets annotations from the compiled program, it knows about what data types are associated with which variables.
And so it will be able to format the output appropriately.
So for example, it knows that i is an integer.
Whereas if you had i as, for example, a floating point number, it would print that floating point number representation instead.
Instead of looking at single variables you can also ask gdb to evaluate arbitrary expressions for you.
And this is actually a really powerful facility.
You can pretty much type any expression that you would be able to use in C or C++.
When you type "print VARNAME" gdb will look up that variable name for you.
But sometimes if you have more than one variable available in your function under the same name, for example, these variables are in different files or different functions that are all available, then you may need to disambiguate the variable name.
And to do that you just proceed the variable name with either the file name that you want to use or the function that the variable occurs in.
Any questions?
OK. And because gdb knows which line numbers the programs is executing right now, then it's able to show you the source code corresponding to the current place in the program.
And in order to browse the source code, if it's available you just type "list."
In general whenever gdb has stopped in a certain place you can type "list" to get a listing of the source code near that location.
And what list will do, by default, is show 10 lines of source code near where the program stopped.
If you type "list" again it will show 10 lines following that.
And you can keep typing "list" to browse the source code.
You can also get the code that's at a particular line number if you want to look at another place in your program.
All right, so this is how to use gdb to actually control the flow of your program.
If you invoke gdb to start up your program, then you can type "run" and that will run your program with a normal flow of execution.
And it will only stop when errors occur.
But you can use a couple of these commands to actually go step by step through your program.
And wait for one line to be executed, and then decide whether you want to look at the next line or not.
If you want to do this kind of step-by-step execution you start by using the start command instead of the run command, which will run continuously.
Then anytime you want to jump to the next instruction, you can use next to get to the next line in the current procedure.
You can also use step, which, if there are function calls on the current line, will descend into those function calls and show you what's going on over there.
And if you're inside a function call you can type "finish" in order to jump back to right after the function call ends in the caller of that function.
And if you want to stop line-by-line execution you can type "continue" and it will just continue without interruptions.
Any questions?
Basically any time you use one of these line-by-line commands gdb will show you the line that you're on.
And if your program just jumped into a new function or something, then it will also show you what function and what file you're in.
So this is very helpful for following what's going on in your program.
OK.
So in addition to just running through your program step by step, you can also choose to only stop your program when certain things happen.
So this is very useful for debugging certain pieces of code where the code is very deeply nested inside and you don't want to step all the way to there.
You just want to stop when you get there.
And gdb allows you to define breakpoints, which are places in your source code.
And once gdb gets to a point of execution which corresponds to that point in your source code, it will stop and ask you what to do.
You can set a breakpoint that's associated with any particular function in your source code or any particular-- well, what you can do is if you do break function then it will create a new breakpoint.
And it will stop execution any time you get to the beginning of that function.
But you can also set break points in the interior of a function by using line numbers.
All right?
Any questions?
Now sometimes you want a little bit more flexibility than that in setting breakpoints.
If you only want to set a breakpoint to happen when a certain condition is true-- this is helpful if you're trying to track down a bug-- then you can write "break" and any of these things, and then "if expression."
And again, you can use any sort of expression that you'd be able to use in C. And what gdb will do is every time execution runs up to that line it will evaluate the expression.
And only if that expression is true will it stop to ask you for control.
At any time you can see which breakpoints are active by doing info breakpoints at the gdb prompt.
And you can also remove breakpoints at any time.
When each breakpoint is created it's given this number.
And the numbers start from one.
And to remove a breakpoint you just do remove followed by the number that was assigned.
In addition to breaking, you can also set a watchpoint.
What this does is it will halt your program every time a particular value changes.
And this is useful for tracking down certain kinds of bugs.
If you're trying to figure out where a certain value for a variable came from, there could potentially be many, many places in your program where that value is set.
And you don't want to have to set a breakpoint for all those and watch them.
So you can instead set a watchpoint which will just tell you when that value is set.
Watchpoints work a lot like breakpoints.
If you do info breakpoints it will also list the watchpoints.
And I believe you can also remove them the same way with remove.
All right, so I mentioned how you can examine your program state by looking at the values of local variables.
If you want a more low level view of what's going on in your program, you can actually look at the raw memory.
And gdb allows you to specify an address.
And it will tell you exactly what data is stored in the memory at that address.
But because there's not necessarily annotation info telling gdb what kind of data is stored at that address, and of course, there's multiple ways you can interpret any particular piece of data, you'll have to tell gdb exactly how you want that data to be interpreted.
So you can interpret any particular block of memory as, for example, a series of machine instructions.
Then gdb will tell you what the instructions are as if you were looking at an assembly listing.
You can ask gdb to interpret the memory as if it were an array of integers and print them out either in hex or in decimal.
You can ask gdb to display the data as if they were addresses or floating point numbers.
And how you do this is you type x for examine memory and slash.
And then the number following is the number of words you want to look at.
And then the letter corresponds to one of these letters and tells gdb how to format the output.
And then the address will be the starting block of where you want to start examining memory.
All right?
Any questions about this?
OK, so I mentioned that at any time when you stop your program there are going to be multiple function calls which are active, corresponding to main, which called this other function, which called this other function, which called wherever your current point of execution is.
And you can actually examine the state for all of these stack frames separately.
When you do bt it's going to give you the list of frames.
And they're going to be numbered from 0 to however many there are.
And you can jump to any of them by using frame and the appropriate number.
And it's going to default to frame zero, which is the closest to where your program is actually executing.
But you can examine the state in frames which are further away.
So that means when you're evaluating variables, each variable only makes sense in the context of a particular frame.
And you're able to go up the call stack to figure out what the value of a particular variable is in this function, which called this other function.
You can also use the commands up and down to just jump to the immediately adjacent frames.
All right?
OK, so you can use gdb from emacs as well.
And emacs provides a really handy interface for gdb.
If you do M-x gdb emacs will invoke gdb for you.
And when you do this, you're going to want to tell emacs to use ppu gdb instead of regular gdb.
And emacs will just ask you what you want to invoke.
You just want to replace gdb with ppu gdb.
Anytime you're debugging within emacs, if emacs has your source code files open, it will show you the current point of execution by drawing a little arrow next to the particular line in the buffer.
You also get a bunch of keyboard shortcuts that you can use to do some common operations.
You can set breakpoints with Control X Space, just by placing your cursor at the particular line that you want to stop at.
So you don't have to go to gdb and type break whatever whatever.
But when you do m-x gdb in emacs, you get a separate buffer for gdb.
And so you can issue all the commands that you normally would want to.
OK, so the first thing we're going to try-- this should be really quick-- is we're just going to open up a brief program and make sure you can invoke gdb on it.
And you'll have to set Cell top if you don't have that set already.
I'm just going to do it on the same one.
[INAUDIBLE]
How do you exit gdb again?
Oh, you can do Quit or Control D. So once you've attached gdb to a program, when you exit gdb it will quit the program that you were running, unless you do detach first.
For this one it doesn't really matter, because we're looking at a crash anyway
[INAUDIBLE] Anybody [INAUDIBLE]?
How do you evaluate something [INAUDIBLE]?
You can evaluate an expression using Print.
OK, any questions?
All right, so all I did here was run and then the program crashes.
This is just lab one without the alignment in the control block that you need to make everything work correctly.
And so all I did here was run and then the program crashes and then print cb.
And of course, this thing is going to be 0 because the DMA transfer to do that doesn't work.
Questions?
OK so we can exit the debugger with Control D. And it'll ask if you want to kill the program.
So gdb also has features to help you with debugging programs that have multiple threads.
Whenever a new thread is created or a thread is destroyed, gdb will print a brief message to tell you.
And one important thing is it will print this LWP number on the PlayStations.
To get a list of threads you can type info threads.
And gdb always maintains this thing called the current thread.
And by default, when you do many of these other actions they're going to apply to the current thread.
And so to examine things about other threads you're going to have to change the thread.
And to do that you do thread and some number.
When you do info threads, it's going to give you this list of threads which are numbered, for example, 1, 2, 3 on the left.
And the LWP numbers that are displayed are going to correspond to these that came up when the threads were started.
But you're going to have to use these numbers on the left to switch between threads.
And when you do info threads, gdb will mark the current thread with a star on the left.
And it will also show you where each thread is executing right now.
Questions?
Where does it show you [INAUDIBLE]?
It will show the procedure and the file name and line number, if that information is available
If you can't find out which [INAUDIBLE]
That's what I was saying--
Ah.
No
[INAUDIBLE]
I don't believe you can get that information.
We can check.
David, do you know?
Sorry [INAUDIBLE].
So David says maybe you can do it on a simulator, but not [INAUDIBLE]
OK.
So we're going to try this brief exercise, which is just to get you to work with dealing with multiple threads.
And what you're going to do is load the lab 1 program, which is the correctly working solution for lab 1.
And if you'll recall, this program has multiple threads.
The PPU thread maintains an array of control blocks.
And it's going to send the addresses of those control blocks to the SPUs.
So what we're going to do is we're just going to set breakpoints at the right places to verify that the first SPU thread is getting the control block which is the same as one of the control blocks in the PPU program.
All right, any questions about that?
This is in [?
rec ?]
4, lab 1.
So you're going to have to run and set breakpoints at the right places.
And you're going to want to set one break point in each thread to be able to examine the value of CB that's being produced
[INAUDIBLE] right after the recitation.
As another exercise for those of you who did get through, the way you'll actually run into these problems is you'll run your program and you might end up with a bus error.
So what you might want to do is then launch gdb.
And you run to the error.
And then you trace back in the execution to see, uh-oh, is my control block value matching?
So you might want to try that for the second exercise online
Should do the stack profiling thing?
Yeah, just go through that.
[INAUDIBLE]
OK.
So one problem that you may have run into is that gdb will remove breakpoints from threads that exit, which is a problem if you have more than one thread running the same program.
And if you're debugging SPU programs, gdb may complain about not being able to find the source files.
But if you just ignore that message, it seems to find them OK, I think.
You can use SPU gdb to debug the SPU programs by themselves.
You can try that sometime.
OK, so actually figuring out what the errors are, unfortunately you don't get very much information from the actual errors that occur.
But maybe looking at where the errors are occurring can help you figure things out.
If you've run into memory misalignment problems or the problem in the last recitation where we had DMA transfers that were too big, those are all going to be bus errors.
Under various other circumstances you might get segmentation faults.
And if you think your program is running into a deadlock, you can also use gdb to kind of attach to the program and then examine the state of that program to see what's waiting for what.
OK, so static profiling.
We have some tools for the Cell that will allow you to, when you have a sequence of assembly instructions, figure out when those instructions are going to get scheduled and how fast the resulting sequence will run.
So in order to take advantage of this you need to use GCC dash big S to generate your assembly.
And you can also do that with xlc.
If you're using our makefiles you can also use make whatever file named dot s in order to generate the assembly from your source code.
Then once you have the dot s file you can run this utility called SPU timing.
And what SPU timing does is it will take all the instructions that are in your assembly and figure out the dependencies between them.
And then it will figure out when the earliest point is that each instruction can get executed.
And it will print out the schedule that's generated.
So if you provide the dash running count option, then it will also show you how many cycles in all the entire program will take.
And the output goes into file name dot s dot timing.
So for how instructions are scheduled on Cell, if you'll recall, there's two pipelines for different kinds of instructions.
And some instructions can only run on one or the other of the pipelines.
And some instructions can run on both.
Now, how Cell actually schedules those instructions is it's always going to go in order that the instructions are specified in the binary.
And whenever there's two instructions next to each other and the first instruction can run on pipeline zero and the second instruction can run on pipeline one, then Cell will try and schedule those at the same time.
And that's called dual issue.
So if you were taking advantage of that dual issue all the time, then potentially you could schedule two instructions every cycle.
Oh yes, and unlike a lot of other architectures nowadays, Cell does not have dynamic rent branch prediction.
All the branch prediction is encoded inside the assembly that you're using.
So that means for any type of loop or whatever, you have to be sure to get the branch prediction right.
If the branch prediction is wrong, Cell is going to end up pre-fetching instructions along the wrong line.
And it's going to have to stall by about 20 cycles when it figures out that the branch is wrong.
So if you're looking at the generated assembly on Cell, all the instructions are going to be of the form operation, then the destination register, and then two or more sources.
And if you're trying to just kind of orient yourself in the generated assembly, there are sometimes these helpful markers.
So if you're looking at the generated assembly for dist_spu, then it's going to have a header at the top which says which files were included inside this file.
And then where the actual assembly is there'll be these markers that say, for example, location 1 19.
And what this means is that here is the assembly corresponding to file 1 line 19.
So that's kind of helpful if you're trying to get your bearings inside the assembly.
Because otherwise it's really hard to make heads or tails of.
Questions?
All right, so after you actually run your assembly through the static profiler, it will spit out this schedule.
And what the schedule shows is which clock cycles are used for every single instruction.
There's one line for each assembly instruction.
And what it's going to do is it's going to print one digit in this schedule for each cycle that the instruction takes.
All right?
So you can kind of think of this dimension as the passage of time.
And what happens is that these guys, when they get to the right-hand side, they'll wrap around it 50 columns or whatever.
And you'll be able to notice when these instructions are being scheduled.
And sometimes when there's dependencies between instructions, the instructions are not able to get scheduled at the earliest possible time.
And when that happens you'll see these dashes, which mean that the instruction is being stalled to wait for one of the dependencies.
In order to make your code fast you're going to want to eliminate these stalls.
And you can do that to a large extent by reordering your instructions
Can you stop at the previous slide?
Yep
[INAUDIBLE]
Pardon?
The point of instruction scheduling [INAUDIBLE]
So the point of instruction scheduling is going to be to minimize the number of stalls.
And you can do that by, if you have instructions which are going to be dependencies for other instructions, you just want to move those as far up as you can
So ideally you would get to instructions per cycle, or how many instructions can you use per cycle?
Two, because you can have dual issue, two pipelines.
Or how many cycles per instruction [INAUDIBLE] by a half.
Because you're getting two instructions per cycle, so [INAUDIBLE].
So we have an exercise that actually has you understand the assembly code, and then doing the instruction reordering.
So we'll leave that on the slides.
You can do it offline.
But we'll pick up with this next week and go over instruction scheduling, some DMA tricks for improving performance.
And in particular, the thing we'll focus on quite rigorously is [INAUDIBLE].
So Phil's going to walk you through how you actually [?
synchronize ?]
and get performance from the vectorization and the intrinsics.
We'll talk a little bit about [?
heat ?]
dynamic profiling.
And for those of you who are still having problems with gdp, we'll try to resolve those now since it'll probably be very useful for your projects.
And we installed CVS in the main directory on every PS3, for those of you who actually want to use it and go through the trouble of setting up their own CVS.
The CVS that's satisfied actually notified the local users on that machine.
So that would be everybody on your team.
It will send out an email whenever somebody does a check-in or an import, to let them know that there's some new information that they don't want to update.
If you don't know how to use CVS or want a quick tutorial, just stop by.
Some of the TAs will be able to help you.
OK?
[INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, everyone.
Welcome back.
Today we're going to talk about the profiling tools available on Cell to help you evaluate how your programs are running.
In the next hour I'll talk about SIMD, which is how you can take advantage of the hardware support for data parallelization.
First up, I'd like to follow up with some questions that we had last time about gdb, and then I'll discuss some of the various methods for profiling on Cell.
I'll talk about the Cell simulator and using the decrementers, and then we're going to talk about static profiling and instruction reordering, which I started to get into last time.
So first up, we ran into some problems last time with gdb, various error messages.
First thing, sometimes gbd is going to examine the wrong variable, as we saw when the names are ambiguous.
One thing you can do to get around this-- we had problems with ambiguity between variables, which have the same name where one of them was in the spu thread and one of them was in the ppu thread.
And so to work around this, one thing you can do is use spu-gdb, which will just log on to a particular spu thread, then you can just look for variable names in that thread.
You can also, of course, rename your variables.
Second problem is that sometimes gdb will delete your breakpoints before you want it to delete them.
Reason being, it will remove breakpoints whenever the program associated with a breakpoint has been unloaded.
But sometimes you have more than one thread running the same program, and so the second one you won't be able to break into.
So again, you can use spu-gdb to log on to a particular thread if you want to do that.
Another thing you can do is for example put a delay loop at the end of your spu thread in order to keep that thread running for as long as possible.
It seems that as long as gdb doesn't unload that thread, then it won't delete your breakpoint.
And finally, I saw this kind of cryptic error thread event breakpoint gdb should not stop.
Indeed it shouldn't.
One thing, to get around this, it seems that this problem doesn't happen on spu-gdb.
So I'll just recap how to use spu-gdb to debug your programs.
Last time we were using mostly ppu-gdb, which will help you debug kind of both your PPU program and your SPU program together.
And you're able to switch between all the threads that are available.
On the other hand, when you're using spu-gdb, it's just going to log on to a particular SPU thread.
So what you do is you invoke your original people PPU program with SPU debug star equals 1.
And then you can background it.
Now what happens is every time that your-- every time that your SPU-- sorry, every time that your program spawns a new SPU thread, it will print the process ID and then actually that thread, the spawned thread, will stop and wait for a debugger to attach to it.
So at this point, you can run spu-gdb -p and then the process number that they give you.
And when you attach to it, you can continue that thread to resume debugging.
Any questions?
OK, so the Cell simulator is one of the tools that we have available to help you debug.
This is how you invoke it if you have it installed, opt/ibm/systemsim cell/bin/systemsim, and you want to use -g to use the graphical interface.
And a little window will pop up and you'll click Go.
The window looks like this.
It actually, from here, you can have a lot of fine-grained control over how your program is running.
In fact you can advance the state of the simulation cycle by cycle to look at what's going on inside the simulated Cell system.
And when you boot up this simulator, you can actually get a complete Linux environment in there with an x term.
And what you can do in this Linux environment is you can copy over programs and then you can-- you can copy over programs which were compiled for the Cell, and then you can run them.
In order to transfer a program to your simulated system so that you can run it, you can use this function-- or you can use this program call through which is available on the simulated system.
And what it will do is it will grab onto a file which is on the host system and copy it in.
And it will just put it on standard out so you can redirect it to whatever file you want.
And then when you make it executable, then you can run it.
All right, questions?
While you're in the simulator, you can view a lot of different things.
You can get a lot of information about the state of the machine.
In particular, you can look at the registers at any particular time, again with clock step by clock step.
This might not be as interesting now that you have the debugger.
And you can run the debugger on real hardware.
But one thing which the Cell simulator does get you is these dynamic profiling tools.
And what it will do is it will look at the state of the simulation and it will figure out where things are stalling, where the time is being spent by the processor.
And you can get separate stats like these.
Let's see, it tells you, for example, total cycle count stalls for various reasons.
You get all these statistics separately for the PPU and each SPU.
So this can be very helpful if you're trying to do, for example, you're trying to figure out exactly what's going on each SPU, do load balancing, whatever.
And so these statistics will hold-- these statistics will cover whatever length of the program you run.
But you can also get more fine-grain statistics using these profiling functions.
So if you include profile dot H in your program, then you get access to these functions, profile clear, start, and stop.
And what these will do is they actually-- when they get compiled into your program, there are no ops when you run them on a real Cell processor.
But the simulator will be able to latch onto them and you can use those to start and stop regions of interest for profiling.
Now the simulator comes with the SDK, and it's a little bit cumbersome to install.
So we haven't made it available yet on the actual PS3 hardware.
You can run it on x86, and so if your group would like to get set up with that on one of your computers, or we'll try and make it available on our hardware if there's enough interest.
So please let us know.
OK, the next thing we'll cover is using decrementers to profile your program.
This is one way you can use dynamic profiling, which actually doesn't require the simulator.
So when you get information from the decrementers, you can run these programs on your actual Cell hardware.
Basically the decrementer is just a counter that counts down at a constant rate.
So you can use it as a clock to figure out how long different events are lasting.
And the rate at which the decrementer counts down is not that fast.
So you're not going to be able to use it to time things on the order of a few clock cycles.
It's best for timing things which are maybe on the order of thousands of instructions in length.
And how you use it is there are these SPU write channel and SPU read channel functions which give you access to some of the internals of the Cell processor.
And so here's an example setup you can use.
Basically, first you call write channel, and you pass in this MFC_DECREMENTER_EVENT thing.
And what that will do is initialize this decrementer counter for you.
And then you read the value before and after the function you want to profile, and when you subtract, that just gives you the length or the time that was elapsed in some arbitrary units.
And of course, it's counting down, so you want to subtract the start from the end.
And after you're done using it, you can do another write channel to stop the counter from continuing.
All right, any questions?
OK, let's continue talking about instruction reordering, which I started last time.
So I'm going to kind of start from the beginning this time, and maybe it'll be more coherent.
So on the Cell architecture, when you're looking at what instructions are being run, the instructions are mostly going to be of the form evaluate some function of some of the registers and write the result to some other register.
And the assembly file are what you get when you run with GCC-S is just going to be a human-readable representation of these instructions.
And I'll show you how to read that actually later.
And you can think of these instructions just as executing in series just in the order in which they appear in the assembly.
So if you think of one instruction starting and then finishing, the next instruction comes in, finishes, that should be consistent with the actual behavior of the hardware.
Now for real hardware, waiting for one instruction to finish before the next one starts is going to be too slow.
So there are these various optimizations we pull or that they do on the hardware in order to run instructions sooner than when the previous instruction finishes.
So the big one of these is pipelining, where you have multiple stages inside your processor, and you can run each instruction-- you can push each instruction in before the previous one has completed.
And in pipelining, you're going to be subject to these dependencies between instructions.
For example, if one instruction reads from a value that the previous instruction wrote, then clearly you can't start the second instruction until after the first instruction has completed.
The Cell processor also has-- actually has multiple pipelines in which you can insert instructions and another optimization that they do is branch prediction in order to reduce stalls from branching
[INAUDIBLE]
Pardon?
Can you talk about static branch prediction?
Yes, so static branch prediction means-- or, rather the way it's done on the Cell is that what's predicted for the branch is not going to be affected by the history of which branches have been taken.
And some other processors actually do use this information.
All right, now the thing about pipelining is that because we still have to honor these dependencies between instructions, the order in which we evaluate instructions, that's going to make a difference on how long the program runs.
So let's take the simple example where we have three instructions, a, b, and c, and we have this pipeline processor where we can insert instructions at these clock ticks.
So suppose that c depends on b, and there are no other dependencies in the system.
Then this sequence of instructions is going to take five cycles, because we can push b in after the first cycle, but we can't push c in until two cycles after that.
Is that clear?
Questions?
All right, now because c depends on b, it kind of makes sense to push b as far up as we can in order to ease this dependency.
So if we execute b before a instead of after a-- and remember we can reorder b and a however we want because there's no dependencies between them-- then we can start b at the first tick, then we put in a right after that, and then c. And now the sequence only takes four clock cycles.
All right?
So observe that, when we put in c in this third clock cycle, we actually couldn't have put in the third instruction any sooner, no matter what the dependencies were.
And so in general, we're going to want that to be the case where we want instructions to be waiting on the pipeline rather than on dependencies.
So that means in the first picture there's this stall of one clock cycle, and we're going to want to eliminate these kinds of stalls.
All right?
So what we're going to do is we're going to use these static profiling tools to figure out where stalls happen.
Now the first thing we're going to do-- the first thing that we have to do is generate the assembly and the instruction schedule that goes with it.
And so you can use GCC -S, and the same flag works with xlc to generate the assembly.
And then we have this utility called spu_timing, which runs on the Cell, and which will tell you the instruction schedule.
Basically it'll take in your assembly file and tell you exactly when each instruction in that assembly gets scheduled.
If you call it with -running-count, then it'll tell you the running count of clock cycles at each stage, which means if you look at the last number, that will tell you how long your program took to run in clock cycles.
And we'll write the output to a file with the same name as the input but with dot timing at the end.
Now if you're using our Makefile, you can actually do all this, generate the assembly and the timing info, all in one step by doing SPU_TIMING=1 and then make the name of the .s file, all right?
Now if you'll recall on Cell we're going to have only in-order execution.
The instructions are just going to execute in the order that they're specified in the assembly.
And because the Cell has dual pipelines, that means there's two pipelines in which instructions can go, and the pipelines are going to be selected just based on the instruction type.
And of course, two instructions are only going to go in simultaneously when the dependencies allow it
[INAUDIBLE]
Right.
So if you're looking at the assembly, you're going to see lots of lines of this form OP DESTINATION and then the source registers.
So when you're trying to figure out dependencies, it's going to be helpful to be able to look at each line of the assembly and figure out what registers that line is reading from, and what registers that line is spreading to.
And there's also information in the assembly file I went into that last time, which tells you kind of for each chunk of assembly what the corresponding source file line is.
All right, and the actual output of the static profiler, like I said, it will spit out a schedule of when each instruction runs.
And in the schedule, you're going to see one digit for each cycle that the instruction is running.
So however many digits there are, that's how many cycles are being used.
And in front of some of the instructions, you're going to see these dashes which represent stalls.
So these three dashes in front of this instruction fm means that based on the pipeline, you could've scheduled this instruction up here right after the previous instruction finished.
However, because you're waiting on particular dependencies, you're only going to be able to schedule it these three cycles later.
So that's a three cycle stall.
And the static profile output is also going to show other information.
The original assembly appears on the right-hand side.
And which pipeline the instructions are going into is going to show up on the left.
Any questions about this format?
[INAUDIBLE]
Dual-issue just means that these instructions are going to be executed at the same time
So can people figure out, for example, what is the latency for fm instruction?
How many circles does fm take?
[INAUDIBLE]
[INAUDIBLE]
Pardon?
What are the numbers on the right?
Over here?
All right, so these are the arguments to the assembly instructions.
So remember the first one is going to be the destination register and the other ones are going to be all the source registers that that instruction is going to use.
All right, so again, we want to be able to reorder these instructions to minimize stalls.
And what you're going to do is you're going to try and look for instructions that are going to take a long time potentially, for example, loads and floating point instructions are going to take a long time.
And of these instructions have dependencies then those dependencies are going to stall.
So not only do you want to minimize stalls, you also want to dual issue instructions whenever possible, although that's a little bit more difficult to just kind of eyeball.
But anyway, if you get up to the ideal running point, where you're able to dual issue every instruction, then that means you're going to be running two instructions per cycle, or it's going to take 1/2 cycle per instruction.
Anyway, going through these assembly files, and figuring out where the dependencies are, which instructions you can move around, it's kind of tedious, and you're not going to be able to apply these techniques to your entire program.
So if you only have limited time, what you're going to have to do is try these on the most critical code first.
And that's usually going to be loop bodies, because those are going to get executed many, many times.
And any savings that you see there will be multiplied many, many times over.
So I'm going to go through an example of one possible optimization you could do.
So we'll actually go through these same files in the exercise about two minutes from now.
This is from Lab 1.
And this is one of the code snippets or the assembly snippets from the SPU program.
So notice that over here we have these load instructions followed by a, I guess this is a shift instruction.
And there's two pairs of these.
And in each one, the shift instruction is going to depend on the result of the previous load.
Does everyone see that from the arguments?
So now the problem is that because the shift is happening right after the load, it's going to stall for a long time, because the load takes six cycles.
All right, so what can we do over here in order to reduce stalling?
Yep?
Two loads concurrently?
Yeah, we can do the loads concurrently, and in general, we just want to move these loads up as high as we can.
So if I move the two loads up to the top of this-- oops-- the top of this snippet, then we have two loads followed by all the instructions that came after.
Now the first shift is going to stall by only two cycles, and the second shift is not going to stall at all.
All right?
[INAUDIBLE]?
So, pardon?
[INAUDIBLE]
So the question is what would this change look like in code?
The thing is that kind of the order of these generate instructions is going to be dependent on the compiler, and it's really up to the compiler how it wants to order these instructions.
So this change is not going to be reflected in like your C code.
And that's why you have to do these optimizations after the assembly is generated.
Does that make sense?
[INAUDIBLE]
If you have a smart compiler then it might do some kind of instruction scheduling in order to minimize stalls using this methodology.
Yup?
I'm quite impressed with XLC.
I compiled the same program under GCC for SPU with optimization level 3 under XLC.
XLC ran in half the time.
So it was doing a much better job of instruction scheduling--
Than GCC?
Than GCC.
So that's something, if you've got a choice, and the XLC compiler is working for you--
In general, this kind of sort of low-level assembly hacking for instruction ordering is not something you'll do, just because compilers are getting incredibly good at doing instruction scheduling.
But here the exercise is for you to sort of get familiar with the tools and understand a little bit more in detail as to what's going on at the instruction level [INAUDIBLE] pipelines.
Another thing to note here is, well, two things you could consider here, how high up can you push the limits?
And related to that is, why can't you completely hide the latency for the first load, and hence the first shift still takes two seconds to stall.
So that's dependent upon how much slack do you have in your schedule, dependent on dependencies and where the loops starts.
So in some cases, you just simply can't hide the entire latency.
So it's part of considerations you have to consider.
But [INAUDIBLE] and the compilers are getting really good at it
And in case you're wondering, I think this code was compiled using XLC on no optimizations, which is why there's this really easy opportunity for optimization here.
It turns out, if we run the timing utility again, we saved eight cycles.
So notice that on the previous picture, we're going up to seven in that column.
And now we're only going up to the previous nine, all right?
So what we're going to do for the exercise right now is you can try something like this.
What we're going to try and do is just improve performance by rescheduling instructions.
And so if you download the tarball, you'll get the exact same file I was working on before.
And what we're going to have you do is just generate the assembly, practice generating the assembly, modifying it, and then continuing with the rest of the build process to get a new object file that's based on your modified assembly.
All right, any questions?
Now you just need to find, for example, one opportunity for optimization.
And you can do the exact same thing that I did, if you want.
And so the key is just after you've done your reordering, you want to re-run the timing utility to see how many cycles you saved.
Also you want to, of course, continue the build process to get the final executable file.
And you want to run that just to make sure that your code is still correct
Are we able to get this?
[SIDE CONVERSATIONS]
OK, so this static profiling process does have its limitations.
The first is that, and this is a pretty major limitation, the static profiler assumes that none of the branches are taken.
So basically it just zips through the entire assembly file in a straight line.
So that means you're going to have a skewed view of how long code inside conditionals or loop bodies is going to take, because conditionals, if there's savings in there, they may not run at all.
And loop bodies, any savings in there may get multiplied many, many times.
So if, for example, you see that you've saved eight cycles in the static scheduling, you're going to need to know some more context, that is whether it's inside a loop body and how many times you expect that to run.
So you're going to need a little more information to get a good idea of how these instruction reorderings are actually affecting the runtime of your program.
And also, because this is static profiling, which means you're not using an actual program run to get data, you're not going to get behavior-- you're not going to capture any behavior that depends on the inputs to your program.
So for example, you don't get counts of how many times each loop runs, or when each branch is taken.
One other thing to worry about, which isn't going to be manifested in these static profiling predictions, is branch prediction.
Now remember the static profiler is just going to ignore all branches.
And the thing is that branch prediction, because branch prediction is used to reduce the stall after a branch is taken by kind of pre-fetching instructions and possibly doing other things.
So what we'll do is guess the direction that the branch will take.
And then we'll start pre-fetching instructions from the expected place where it's going to resume after the branch.
And the thing is, if your branch prediction is wrong, then you're going to incur a pretty large penalty.
And the penalty is on the order of 20 cycles while it flushes the pipeline and starts fetching instructions from the new place.
On the other hand, if branch prediction is correct, then there's no penalty at all, actually.
So the next instruction will just resume right after the branch.
Anyway, you can give hints to the compiler to tell it what you think the outcomes of branches will be.
And this can help in some circumstances.
Basically there are these macros that you can define.
And what you're going to use is this built-in expect compiler intrinsic, and what that means is if you put that inside an if, it will mean, you know, run the if as if the condition were exp.
But then I'm going to expect that the value of this condition is going to be, for example, true or false.
All right, does that make sense?
Anyway, just one note about the exercise that you did.
I believe the original run time was 469 clock cycles.
Actually, if you run XLC with 05 optimizations, it will reduce that to 188, so almost 280 cycles of savings.
All right, so we'll-- pardon?
The one that we just [INAUDIBLE]
No, so the original code was-- the original assembly was compiled on O 0, or no optimizations.
But then if you're doing an optimization by hand, it saves eight cycles off of O 0.
These are actually pretty easy to find.
And the 05 optimization level will actually shave off 280 cycles.
So we'll break for about 10 minutes, and then after the break, we'll do SIMD on Cell.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this hour we're going to talk about SIMD programming with cell.
First we'll talk a little bit about what SIMD is and then about the facilities that cell and the compiler provide for programming with SIMD.
And then some design considerations that you have to keep in mind when you're doing things.
All right so the situation these days is that most compute bound applications are running through a large piece of data and running the same computations on it over and over again, or rather running the same computations across all the pieces of different data.
And very frequently there'll be no dependence between iterations when you're going through this data.
So that means there's opportunities for you to data parallelize.
So as an example, if we have for example, say we're multiplying a0 and b0 to get c0.
And suppose we want to actually perform this operation across all the elements of arrays a, b and c. So instead of multiplying two elements or two integers together, we're actually going to be taking two arrays an element-wise multiplying each of the-- multiplying each of the pairs element-wise and writing the results to a third array.
So the picture's going to look something like this.
And you would of course represent this using for example, four loop.
Now what we're going to do is instead of-- let's see, so you can think of this as kind of an operation that's abstractly operating on these entire arrays.
And we're not going to go quite that far, but what we're going to do is we're going to think of these operations as acting on these kind of bundles of elements.
So we're going to bundle our arrayed elements into groups of four.
And then each time we're going to take a group and multiply with another group using this element-wise multiplication and write the result to this third bundle.
OK does that make sense?
Now the thing about this kind of model is that cell is going to provide very good hardware support for something that looks kind of like this
Is that actual cell [INAUDIBLE]
Yes, I'll get into this.
In fact, this is what we'll be talking about the syntax and meaning of this kind of thing.
All right?
So for this kind of thing to happen we need the compiler to support two different things.
First is we need to be able to address these kind of bundles of elements and these are going to be called vectors.
And second we need to be able to perform operations on these vectors.
So cell and the XLC compiler give us support for this.
And what they're going to do is provide first, registers which are capable of holding vectors.
Now normally you think of a register as holding on a 32-bit machine, one machine word will hold a 32-bit int, for example.
What we're going to have on the cell is these 128-bit registers which are going to be able to hold for example four ints right next to each other.
So we're going to be able to take this bundle of ints and operate it-- operate on it as a unit.
And the second part is we're going to have operations that act on these vector registers.
So the cell is going to support special assembly instructions and it's going to be able to interpret those as acting on particular vectors.
But also we're going to have C++ language extensions which are called intrinsics.
And those are going to give us access to these special assembly instructions, but not require us to be poking around in the assembly.
All right now the big draw of this is that these operations are going to be pretty much as fast as single operations, which means that if we take advantage of them we can make our code run say four times as fast.
OK so how do we refer to these vectors when we're coding?
XLC is going to provide us with these intrinsics.
And we have these vector data types and each one is just going to specify how to interpret a consecutive group of 128-bits as some sort of vector.
And you can have vectors of varying element sizes and varying number of elements.
So when you're programming on the PPU or the SPU you get these four different kinds of vector data types.
You can declare things as for example, vector signed int, which is what I mentioned in the example.
Which is where you have four ints next to each other each 32-bits.
You could also have vectors which contains 16-bit integers or 8-bit integers.
And you could also have vectors of floating point-- floating point numbers.
I should mention that all of these signed integer types also have unsigned equivalence.
Anyway, so you can just declare these anywhere in your code and use them as if they were a C++ data type.
All right any questions?
On the SPU you also get some additional vector data types.
One is vector signed long-long, which is 64-bit ints and you can fit two of those in 128.
And you can also fit two double precision floating point numbers in 128-bits.
Now the compiler's actually support these types pretty nicely.
So not only can you declare variables of these types pretty much anywhere in your code, you can also declare pointers to these types and arrays of these types.
All right.
And so they look pretty much like natural C++ types, except that they translate directly into these particular types that the hardware supports.
Now in order to manipulate these vector data, we're going to have the-- we're going to have compiler extensions called intrinsics, which are going to provide access to the assembly level features that we want.
Remember we're going to have specific assembly instructions that correspond to for example, multiplying two vectors which contain each four, 32-bit integers.
And instead of writing out-- instead of going into the assembly and actually inserting that instruction ourselves, we just use a compiler intrinsic inside our C++ code.
And what it does is it provides this notation that looks a lot like a function call, but the compiler automatically translates it into the correct assembly instruction.
And again you don't have to worry about going into the assembly and messing around with this instruction that's supposed to apply to these registers.
You don't have to worry about register allocation at all.
The compiler just figures out the right thing for you.
And to use these in your SPU program you're going to want to include SPU_intrinsics.h.
Now what's a little bit confusing is that you're going to have slightly different intrinsics available on the PPU and the SPU, because those are actually going to have different instruction sets.
But anyway as an example, you can declare two variables of type vector signed int and then you can multiply them using this intrinsic called SPU add.
All right and assign them to a third vector signed int.
Questions?
Yep
In what way are they introduced if you're on the SPU or the PPU?
Is it just-- not entirely the same set of operations available?
Or are there actually semantic differences?
Could make a little header file that masks over the differences mostly
There are going to be some operations that are only available on one and not the other.
But in general, if you look at the names, if the names correspond, and I'll go into that in a little bit, then they should perform essentially the same function
[INAUDIBLE] mostly was the [INAUDIBLE] also some name differences where there really don't need to be.
For instance, if you try to do a shift on the PPU I believe it's a [INAUDIBLE] SL, shift logical right, shift logical left or shift arithmetic right.
Sort of things you would remember.
On the SPU it's the acronym for rotate and mask for shift.
So R-O-T-M-A-R or something like that.
So yes there's some differences that don't need to be there
OK so to actually create these vectors there's a couple of different notations you can use.
The first is, you can use this thing which looks like a cast to, for example, vector signed int.
So you do vector signed int in parentheses and then a list of four integers you want to fill in.
And that will create an integer vector and assign it to a.
You can also, I believe you can also use that notation with just one integer and it will fill in that integer in all four positions.
There's also an SPU intrinsic called splats that you can use to basically copy the same integer to all four components
How does it know you're not using a comma operator?
Yeah I don't-- is that right David, with the parentheses in the second part?
Whatever
Another caveat here from someone who's been in the trenches is that XLC likes this notation.
GCC sometimes likes curly brace notations instead
So I'd seen both of those and I do know which to do.
[INTERPOSING VOICES]
[INAUDIBLE]
OK great thanks.
All right.
And after you've assigned some of these variables in order to get back the pieces out, one way you can do it is to use this union trick where you assign or rather you allocate something of vector signed int and then you tell C++ that it can find an array of integers in the same place.
And what that will do is pull out the components.
So if you define this union this way, then you get a type called intVec.
And any time you have an intVec you can either do dot Vec to get at the vector signed int, the vector data type.
Or you can use dot vals to-- with an array index get at the components of the vector.
And you could also use this intrinsic call SPU extract to pick out the same components.
XLC provides a bunch of different vector operations that you can use.
There's integer operations, floating point operations, there are a permutation and formatting operations which you can use to shuffle data around inside vector.
And there's also load and store instructions.
And I believe we have a reference linked off the course website if you want to figure out more about these.
I'm only going to touch on a few of them.
OK so the arithmetic and logical operations like I said, most of these are the same between the PPU and the SPU.
There's some that are named slightly differently and some that are not available on the PPU.
So these are all things you would expect, add, subtract.
Madd is multiply and then add with three-- with three arguments.
Multiply, re is for reciprocal.
You can also do bit-wise and, or xor and I believe there are other logical operations there too.
Now the thing is you have to worry about which PPU or SPU instruction you're using.
But you usually don't have to worry about selecting the right vector type.
The compiler should figure out which vector types you're using and substitute the appropriate assembly instruction that produces a result of the same vector type.
So all these operations are what we call generic.
And they stand in for all the specific instructions, which are-- which only apply to a single vector type.
Does that make sense?
OK so one handy thing is a permutation operation.
And this allows you to rearrange the bytes inside a vector or two vectors arbitrarily.
And so the syntax is SPU shuffle a, b which are your source vectors and pattern which tells you how to shuffle them.
And pattern is going to be interpreted as a vector of 16-bytes.
And each byte is going to tell you-- each byte is going to tell the compiler how to pick out one of these bytes in the result.
And how the byte is interpreted is the low, the low four bits are going to specify which position the source is going to come from.
And the fourth byte is going to specify whether you're going to pull from a or b.
So as an example, here's the pattern VC and if you look at the second byte which is one, which is one, four in hex.
Then that means the destination register is going to contain the fourth byte or the fourth byte of b, all right.
So four means select the element numbered four and one means select from b.
Does that make sense?
And this is very versatile by putting in the right, by putting in the right pattern vector you can arrange for all these bytes to be shuffled around however you want
The pattern is a constant [INAUDIBLE]
Pardon?
The pattern is a constant in intermediate parameter
You can fill in the parameter at run time if that's what you're asking
[INAUDIBLE]
[INAUDIBLE]
OK, also useful are these rotation operations which will let you shift your vector left or right by some amount.
Now one thing to be aware of is that on the SPU you only have these 128-bit registers.
So on the PPU you have different registers which are suitable for holding different types.
For example, there's word-sized registers for holding ints and PPU also has these 128-bit registers.
But the SPU has nothing else.
So that means whenever you're using scalar types on the SPU they're all going to be using these large registers.
No matter what the size of the scalar you're using.
And depending on the size of the scalar you're using it's going to go in a particular position inside this wide register.
It's called a quadword register, because it's 16-bytes.
Now the thing to watch out for is that whenever you load something from memory into-- whenever you load a scalar from memory into one of these registers, there's going to have to be a little extra processing done in order to shift-- in order to possibly shift the scalar into the right place inside this register.
And furthermore, the hardware is always going to want to grab one of these quadwords all at a time.
So loading a scalar is not going to be any cheaper than loading one of these quadword registers.
So one possible you're going to want to load an entire quadword register at a time.
And if you just need a part of it, then you can figure that out later.
But you might as well get the whole thing.
Questions?
So when you-- just a scalar question.
So when you load a scalar value that's not aligned-- it's not aligned with the preferred position is that-- is there overhead associated with that?
I'm not sure how much overhead is associated with that.
Pardon?
Oh do you know?
Well, unlike scalar it's [INAUDIBLE], it can only load on 16-byte boundaries.
So it's going to load the-- load something that includes that and that's going to have to shift to the another position
So do unaligned-- when it has to shift the scalar around, does that actually take longer than went in-- when it's natural?
I don't know if it's-- well what you can do, you can set some flags in XLC that say, align all of my scalars correctly.
And we'll waste 4x overhead.
It'll even say align my array, have my elements so that I can have the scalar array at the back-- I can load and it will waste the overhead's that everything in the array is [INAUDIBLE].
So you can have the compiler trade off space versus time for you off two switches
I see.
OK so we're going to want to look at the sim application from recitation two.
And we want to adapt that to make use of SIMD data types and intrinsics.
So what we've done is, remember we had these x, y, z coordinates that we were manipulating.
What we're going to do is we're going to pad each one.
It was three words before and we're going to pad each one so that it fills a quadword.
And so for each quadword of course the first three words are going to correspond to the x, y, z components.
And we can grab those out using SPU extract or some other intrinsics.
Now when we're doing manipulations with these components for example, we wanted to find the displacement between two locations.
And that's just subtracting two of these coordinates.
So we can do that subtraction which before required three floating point subtractions.
We can replace that with a single SIMD instruction.
Does that make sense?
OK so all this-- most of this has already been done and we're providing most of the implementation of this SIMD version of sim.
And what we want you to do is download this, download the tarball for this recitation and then go in there and what we want you to do is fill in one of the blanks.
All right.
So there's just one function here that's been left unimplemented.
And to see if you know what's going on, see if you can fill in the implementation for this.
Any questions?
So this question-- this function you want to implement is basically going to take a vector float and if that float contains a, b, c and d you want to return a-- you want to return a vector which each of whose elements is a plus b plus c plus d. Questions?
What directory under the--
[INAUDIBLE]
So we're going to go into sim a list
But we can stay around afterwards and help you figure out what's going on
OK so here's one implementation.
Basically we're going to just declare another vector float and that vector float we're going to-- that's basically we're just going to do these swaps.
So notice in this one we're swapping the first and second-- we're swapping the first and second words.
So that means down here we're going to want to carry bits four, five, six, seven and then-- or bytes four, five, six, seven first and then bytes 0, 1, 2, 3.
And then over here we want bytes 12, 13, 14, 15 and then 8, 9, 10, 11.
Everyone see what's going on for the first shuffle?
And then we're going to just add that to our original vector to get this.
And we can do that again, this time now we just want to swap these two halves.
So the shuffle pattern is going to be 8, 9, 10, 11, 12, 13, 14, 15 followed by 0, 1, 2, 3, 4, 5, 6, 7.
Make sense?
OK so the way we translated the program we just used into SIMD was we used a struct of arrays.
Basically each of these structs that we had from our previous implementation just carried over and we just put all those into an array.
So the structs were right next to each other.
Alternatively we could have laid out the, laid out the data in memory in a different way and this is called an array of structs layout.
Instead what we can do is put all the like fields next to each other so that we have, for example, an array of all the x components, then an array of all the y components, then an array of all the z components.
And when you reorder the data this way you get different ways you can use to process it.
So for example, now each quadword instead of containing the data for a single point is going to contain the data for the same component of four consecutive points.
Everyone see that?
And actually we can implement the algorithm from before in this new layout.
But we have to be a little bit more clever in how we're putting together the elements.
Before we were able to just subtract each-- subtract or multiply the quadwords with each other, because those would just correspond to for example, subtracting the coordinates of two points.
Now this time we have to do some additional computation in order to put all the pieces together.
The trick behind this structure of arrays implementation which I'll just gloss over is, if we're storing state for eight objects then we're going to need-- eight objects hold the-- hold 24.
Rather for each object we need the position and the velocity.
And for each of those we have x, y and z.
So that means to store state for each object, for eight objects we need 8 times 6 is 48 words.
And so we can put those in 12 quadwords if we pack them right.
And when we do SIMD operations on these quadwords that we pull out, we can get four pair interactions.
So suppose this is a quadword and it's contained-- and it contains data corresponding to elements a, b, c and d. And over here we have a quadword containing data corresponding to elements one, two, three, and four.
With some SIMD operations we can kind of figure out the pairwise interaction between objects a and one, between b and two, c and three and d and four.
But of course we have to be able to find the interactions between any pair.
It's not just these pairs that lineup.
So what we have to do is rotate the quadword over by one word and then do the same thing again.
We do that four times in all and then we add up the results.
So as you can see this implementation is a little bit more involved and less-- it maps to the original implementation less directly.
On the other hand, it does give us a really dramatic speedup.
Because we're using more of the vector words.
Notice that in the first packing we had.
We had x, y and z and then the fourth blank was unused.
Anyway, this time the structure of a race implementation is actually 7 1/2 times faster than the array of structures implementation.
So choosing this data layout correctly can actually be one of the really big determinants of how your program performs
The scalar version was like what 480, something like that?
Or is it not comparable?
I let's see, David, do you remember?
[INAUDIBLE]
OK so--
[INAUDIBLE]
No that was just on the
[INAUDIBLE] [INTERPOSING VOICES]
[INAUDIBLE]
OK, so something like 400 for the double buffered one and 300 for array of structs.
OK one other thing to worry about is when you're dealing with these-- when you're dealing with these SIMD instructions, you want to make sure that all your data are aligned correctly in memory.
And like I said before, when you're pulling things in from memory you want to make sure that whatever you're pulling in is going to be aligned on a quadword boundary.
And you can use the align compiler directive to tell the compiler, I want this piece of data aligned at a particular place.
And if you do that on all your arrays for example, and make sure that your array-- the array elements are going to fit neatly into quadwords then you should be OK. Again like I said before, you also want to transfer only multiples of 16-bytes on the load and store.
And so when you're doing processing it may help, it may help you if you actually pad the end of your-- pad the end of your array's so that they fill out a multiple of 16-bytes.
Because it's easier to just do that processing with the SIMD instruction rather than just have one or two elements hanging off and have to worry about those
Question
Yep
Is it a good idea to pass parameters 2.2 [INAUDIBLE] I mean, which one is preferred?
[INAUDIBLE]
So you should [INAUDIBLE] for figuring out whether something can scale easily or not.
So you might make [INAUDIBLE].
So in cases where you can avoid using pointers, you should do that
OK. [SIDE CONVERSATION]
So one last thing that I should mention.
I haven't really let on, but compilers can actually generate some of these SIMD instructions by themselves.
If you declare your types to be vector and then use just regular operations apparently GCC and XLC yes, will substitute the correct intrinsics for you.
Of course that doesn't get you all the operations which are available with intrinsics, but anyway automatically simbianizing your code is something that's really worth looking into.
As we saw it can give you a great performance improvement.
And the thing is that compilers are still not very good at automatically doing this transformation.
So unlike instruction scheduling where if your passing 05 your compiler will do a much better job than you would have time to do.
This is something that you should probably reserve some time for.
That's all.
If you have any questions you can stick around and I'll try and help you.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good afternoon.
My name is Joe Leavitt.
My group members are Ben Ayton, Jess Noss, Erlend Harbitz, Jake Barnwell, and Sam Pinto, and we're going to introduce you to the topic incremental path planning.
Listed on screen are just some of the references we used in putting together this lecture.
The top three are the main references we used in developing our material on the D* Lite algorithm, but also take note of the bottom reference, which is a paper done within Professor William's group here at MIT.
It talks about all-pairs, shortest paths incremental search method.
So if you're interested in more incremental search methods than those we're going to talk about today, please get into this.
Especially if you're going to go ahead and do our P set, these references will be very helpful to you.
So today, we're going to first Monday problem, then we're going to introduce the idea of incremental search.
Then we're going to go into detail on the D* Lite algorithm, which is type of incremental path planning, an algorithm used for incremental path planning, run through an example of D* Lite, talk about when it is good to use incremental path planning when other path planners might be better, and then I'll go into some algorithm extensions and related topics, and some applications in mobile robotics.
So in order to motivate the problem, we're going to use a robot living in a grid world.
It's at a start location trying to get to a goal location.
It can move up, down, left, or right, or diagonally.
And it can only see the eight cells around it, as it's moving through this grid world.
And we're going to give some pretty fine map of the world.
This is a map that has the world before it starts moving.
And it's possibly moving through some of these, new obstacles could appear, or obstacles could go away.
And the idea is that it's going to make observations as it's moving to this world and update its plane to get to the goal based on what it sees as it's moving along.
So I must correct myself.
The map that you're seeing now is the map that the robot has ahead of time.
The previous map is what the environment actually looks like.
So as it starts moving, it's going to make a plan to get from the start to the goal, and that's what we want it to do.
And it's going to move to the next position and then realize that there's an obstacle in the path that it previously planned, and now it must replan in order to find a valid path to the goal.
And so now, it replans, and continues to move on the new plan, and now it finds another obstacle that it has to replan around, and it does the same thing.
And then it continues to walk through until it gets to a point where we have a change in the environment.
The change is represented by these two cells going black here, and now its original plane is no longer valid again.
And what we want it to do is to replan to find a new valid optimal path to the goal.
This is just showing here what happened to our environment.
And then the robot replans and continues to march towards the goal So this is the behavior we want from a mobile robot, whether it's a two-dimensional path-planning problem, or we're talking about multi-dimensional robot arms, planning, working with humans on a manufacturing environment, or an activity-planning process.
The idea is that we want the robot to be able to account for new obstacles or changes in its representation of the environment and continue to plan on the fly quickly and optimally.
So there's methods that accomplish this.
One method used to in path planning is rapidly exploring random trees, or RRTs.
RRTs are basically going to plan, replan, on every time step based on changes in the environment.
And one problem in my experience here, though, is that RRTs are suboptimal, even though fast.
See, here you see the robot has come up with a plan.
It's seeing new obstacles, the table in front of it.
The coffee cup.
And its planning for those things, so the plan it came up with is clearly not optimal.
So we can perform some additional computation.
We can use a method that builds upon the RRT, RRT*, that it finds an optimal path, but that requires a pretty significant amount of offline computation time, and then you can accomplish something that looks a little more normal.
Again, we have to allow the robot to perform a lot of the computation off-line.
So how do we accomplish what we want to?
We want to compute quickly.
We want to compute optimally, and then have the robot continue to move as it's received new information coming to the environment.
Some problems we're facing, there's changing environmental conditions.
There's sensor limitations.
You might not be able to see everything in the environment around you, and then we have limited computation time.
And the solution that we're going to explore here is to reuse that from the previous search, in order to speed up the search process, and continue to plan optimally.
And the method we're going to use to accomplish this is incremental search.
So now, we're going to talk about what incremental search means.
First to show you kind of how powerful incremental search can be, this is one example of an incremental search method, incremental all-pairs shortest paths, which allows you to online compute paths between any start and goal location.
The idea is that you have a robot that's trying to move around one obstacle.
That's what you see in the upper sequence of events there.
And another obstacle is added, and now it has to plan around this new obstacle.
In using this algorithm, you have a significant increase in performance and computation time using the all-pairs shortest paths versus an RRT.
RRT can act as another version of an RRT, where you're planning basically from both the goal and start and connecting your paths.
It's on the order of what you get with RRT, but this RRT connect algorithm doesn't give you an optimal result.
And then it's even fast than another algorithm probablistic road mapping, where you're randomly sampling the environment, and connecting points to produce a path.
So the idea here is that we can really leverage this idea of reusing results from previous searches to accomplish fast replanning online.
So the whole idea of incremental search is that we're going to form a normal graph search, just like anybody's used to.
And then we're going to repeat as the robot moves through the environment, or you're executing your plan, you're going to execute the next point in the plan, or the next action in the plan.
We're going to receive changes from the environment, either through our sensors or information from the outside.
And then we're going to use that information to update our previous search results instead of completely replanning, and then continue to form that loop, until you reach your goal.
So before we get into too many of the details of how we're going to accomplish this, first, we want to review the graph search problem.
So in graph search, you have a graph consisting of vertices or nodes.
We're going to use nodes throughout this lecture to talk about the vertices or nodes in the graph that are connected by edges.
And then, there's an edge weighting function that describes the cost going from one node to another.
And then, a heuristic function is an estimate of the cost to get from a current node to the goal node.
And then we a start vertex and a gold vertex.
In the graph search problem, we're also going to have this idea of a g value, or a cost so far.
We're going to use term g value, which is essentially the cost that it's taken to get to some predecessor node, plus the cost of the node that you're actually expanding.
So you would have this W start to S1 cost to get from start to S1.
And then the g value for S goal would be the weight from S1 of the goal plus the S1's g value.
Yes, go ahead
What is W?
W is the weighting function that I talked about previously here.
So the weighting function describes the cost to get from one node to another along any edge in a graph.
And then we'd use those waiting functions to develop the idea of the G value.
So W is just the edge cost.
And then our heuristic value, like, we use our heuristic function to establish a heuristic value from one goal to another, and we're going to call that our h value, or h. And then the total estimated cost to get from a node that we're currently expanding in a graph search is f, which is your cost so far, plus your estimated cost to go.
I think I might have messed that up in there.
Call g value associated cost so far, not cost to go, and then h is an estimate of cost to go.
So in order to reason about a real-world environment, we first have to take that environment and convert it to a graph, so we can form graph search, and then do everything we're talking about.
Some of the examples, we're going to talk about, and some of the ones we've already talked about, we're using this notion of a grid world.
You can have a four connected graph or a grid, where a robot can move up, down, left, or right, basically in x or y directions, or an eight connected graph, where you can also move along the diagonals.
And then had here, we just showed the graphs associated with those, with the nodes and edges.
And then you would have an associated admissible heuristic with those if you're going to form a heuristic search.
So just know up front that any time we're moving from a real world environment, that we're trying to reason about these search methods, we have to develop an associated graph.
And one more thing that is important as far as understanding incremental searches, is the notion or idea of relaxation.
We're going use Dijkstra's algorithm.
Everybody here familiar with Dijkstra's algorithm, I'm assuming, in the past.
We're going to use Dijkstra's algorithm to kind of illustrate the idea of relaxation and remind everybody of what that means, because it's an important concept when we're talking about incremental search algorithms.
So essentially, in Dijkstra's algorithm, it's a single source, shortest path algorithm, a best first search.
So we're going to make eight expanding nodes according to their best g value, or cost so far.
And we have our start node initialized to zero cost so far, because at your start, haven't accrued any cost.
And then, every other node is going to be initialized to infinity.
And the idea is we're going to find the shortest distance to all of the other nodes by relaxing the cost to those nodes from infinity down to their actual shortest or lowest cost value.
So we start by expanding the start node, and then you find the cost so far to each of the children nodes of the start node.
So you have an edge way to one, and so the node on the bottom as a value of 1.
Similarly for the middle node at 5, and the top node for 3.
Now, the next node we're going to expand, can anybody tell me what the next node we're going to expand based on performing best first search here using cost so far?
The 1?
1?
Yes.
So the node labeled 1 here, that has a g value of 1 is the lowest out of 1, 3, and 5, and infinity.
So we're going to go ahead and expand 1 next.
And now, we find that using this value of 1 plus 3 equals 4, which is less than what we previously had here as 4.
So now the center node has been relaxed from a value of 5 down to 4.
So now the shortest path we found to the center node is 4.
And we're going to continue to perform this by going through each node in order of the lowest cost.
And similarly, you saw how the top node was relaxed to 6 by going through 1, 3, and 2, where as previously, it had a lowest cost of 7 going through 3 and 4.
And if we format through the whole graph until all nodes have been unexpanded, then we've relaxed every node in the graph to its shortest path value.
And then, we can just do a greedy search from a chosen goal node, back through the graph to find shortest path
Is the term relaxation standard?
Because the fact that you're bringing numbers from infinity double to small value sounds more like tightening as opposed to relaxation.
So I was just wondering if people use it as like, a standard--
I'm not sure if it is in all areas, but I have seen it in multiple places.
When you're talking about graph search, specifically with respect to Dijkstra's algorithm, that is where the term relaxation is used.
And we're going to use that as we go throughout and talk about incremental search.
We're kind of repairing the results because of new edge weights and things like that.
We're going to figure out which nodes we have to relax back down to their shortest value to be able to find the new shortest path.
So that's the term we're going to use here if it's not used universally.
I can't say for certain.
But we will use relaxation to talk about reducing-- it's kind of related to-- think about like, a tension spring.
As you reduce the constraints on it, it'll slowly relax to its completely relaxed position.
That's kind of the idea here.
So now that we have talked about the ideas of relaxation and graph search, how do we reuse the results from a previous search in a new search field to employ an incremental search method where you have done a search, you found the shortest path, and now, you had some changes to your graph, and you want to use your previous results to come up with your new shortest path.
So the way we're going to do that, is we're going to store our optimal results from a previous search.
This can be done in a number of ways.
It's going to depend on the algorithm.
But you can soar like, a list of shortest paths as an incremental all-pairs shortest paths problem.
The one we're going to talk most about is storing g values, and that's what's done in a D* Lite algorithm, where you're finding the shortest distance from any given node that you've expanded so far in your search to the goal node.
And then, you'll use that data to find out, find the shortest path.
And then when you go to perform your next iteration, you'll look for inconsistencies in that graph.
And then in order to find a new shortest path, we'll leverage that previous data and the inconsistencies for relaxations to come up with a new optimal solution.
And that's all we're showing here.
On the right side of these two graphics is you have the initial search.
You have an obstacle that is now in a place that was wasn't there previously, which causes you to update edge weights in the graph associated with this grid world.
And then using these values which represent the g values in the graph search, you only update the ones you need to update to find a new optimal path.
And so completely reperforming our search.
So what incremental search methods exist, now that we've kind of talked about what they are and what we can do with them?
So the whole idea of this slide here is to show you that incremental search can be used across many domains.
It's not just specific to path planning.
It's used in temporal planning to determine temporal consistency.
It can be used propositional satisfiability.
Instead of every time you're updating another algorithm, you can use your previous results.
And the one we're going to focus on today is D* Lite, which is specific to mobile and robots.
But the ideas and concepts we're going to talk about there are applicable elsewhere, and you can use to help you learn about and expand your knowledge in other incremental search.
So the D* incremental path planning approach.
The whole idea behind it is that we're going to take the idea of incremental search that we just talked it about, and we're going to combine it with heuristic search, which is an optimal solution.
And the idea is that two things are orthogonal so we can combine them, and then we can form efficient incremental path planning, getting both an optimal result, and quickly getting a new path when things change in the environment without having to completely replan.
And so what that looks like, when we have a grid world, which is shown in four instances here up on the board, and we're trying to plan from a start node to a goal node, and showing each of these sections.
And in the upper left or on the left axis, we have-- whether we're performing a complete search or an incremental search.
And on the horizontal axis, whether it's an uninformed search or a heuristic search.
So the upper left is a complete search, just using best-first search and no heuristics.
And what it's showing is all the nodes that are expanded, and trying to find a path from the start to the goal.
In the upper right is a heuristic search, A*.
Employing the heuristic, we're able to collapse the number of nodes expanded by quite a bit.
In the bottom left is an incremental search method that's using uninformed search or best-first search as an underlying search algorithm, so it's going to expand the same nodes as the best-first search.
And then in the bottom right, you see an incremental heuristic search, the lifelong planning A* algorithm, which is kind of the baseline for the D* Lite we're going to talk about, that the initial search will expand the same nodes as the heuristic search, since it's using A* as its baseline.
So now let's say the robot moves.
We get an update to the environment.
There's a bunch of changed edges.
And now best-first search, as you can see, is going to have to expand pretty much the same number, if not more nodes, then our previous search.
And A* also is going to expand about the same number of nodes as it did previously.
The incremental search method using best-first search, by using previous results, reduces significantly the number of nodes that needs to be expanded.
And then, lifelong planning A*, the incremental search plus the heuristic search, reduces that number by even more.
Even when there's quite a few changes in the environment, you have very few nodes that you need to expand on the next search in order to find a new optimal path.
With that, I'll turn it over to Ben to talk about the D* Lite algorithm
Thanks.
So I'll be walking you through the D* Light algorithm today.
Now as we're emphasizing, the D* Lite is not the only incremental algorithm, as we mentioned previously.
But it is a widely used incremental algorithm, and you'll see it's quite efficient in what it does.
D* Lite is an algorithm that searches from a single start node to a single goal node.
But as we see changes occur in the path along the way, we'll adjust that.
So it's nice to think about D* Lite from two perspectives.
One is to think about it as a modification of the principle space.
Now, you'll see that D* Lite in its first iteration or its first run through before any modifications behaves very similar to A*, and that's where it's nice to keep that main model in mind.
And you'll see that in an example further along.
The second perspective to look at A* is from the perspective of Dijkstra's algorithm, from which we see many of the principles of relaxation that we covered beforehand.
From this perspective, we see that when we do reparations to the graph or changes to the graph, our methods of repairs the graph is essentially to relax down our changed edge weights until they reach their truth.
Now, where we differ from Dijkstra's algorithm with something like D* Lite is selecting only the nodes that we want to hit efficiently.
So whereas Dijkstra's algorithm tried to fit the entire graph, we only want to travel from a single start node to a single goal node.
And so D* Lite behaves like Dijkstra's, but a guided Dijkstra's.
So I'd like to just remind you of several concepts from A* first, because it's easier to introduce them in that context.
And then we'll modify them a little bit.
So Joe mentioned before that for A*, we're searching from a start node to a goal node.
We're sorting the nodes in our queue by total cost, which is the sum of what we're calling g, which are costs to get to a single node, and our heuristic, which is the cost to get from a node, s, to the goal node, your s. Now, what we don't usually consider with A*, but is very important for D*, is how to distinguish between ties when two different nodes on the search cube have the same total cost.
So here's what we're doing is we're introducing a couplet, which consists of two values.
The first, which I'm calling f1 here for any given state, is the same as what we're usually used to.
It's the linear combination of your cost to reach the goal node and heuristic function.
But we also introduced a second value for this couplet that's used in the case for only those to draw.
And that is simply the cost to go, that g value.
So the order of expansion of nodes from the q will order them in terms of f1 first, and if two nodes at the front of the cube tie in terms of their f1 value, we select the node with the lowest g value.
So looking at our graph down here, we're looking at our states s1 here and s2 here.
Can anyone tell me the total cost for our node s1 which, remember, is a couplet of two different values?
5, 3.
So five, yes, is our cost to reach this node one, plus the heuristic, which is 2 here.
And also, 3 is our second value, because it's only our g value there.
And so what would the value be by the same logic for s2 down here?
5, 2
5, 2.
That's exactly right.
And so, bearing in mind what I told you before, where should we take off the q first?
S2?
That's exactly right, because the first value, f1, for these two nodes is exactly the same.
But the second value is lower is f(s)2.
I'd also like to introduce the concept of successors and predecessors of any given node.
In general, a graph consists of directed edges.
And we could have a graph with undirected edges, but we can think of as just any edge being directed in both direction.
So here, we define the concept of a successor of a node, which is every node can be reached from a given node, along edges that can be found.
So the successors of this given red node are these two blue nodes here.
We also introduced the concept of predecessors, which is every node from which that node can be reached, meaning nodes from which we can follow our directed edge to our node.
So for our red node, the predecessors are these two nodes.
What I'd like to emphasize here is comparing the two, that this node here was both the processor and the successor of our red node.
And that's fine.
That occurs when we have undirected edges or edges that direct in both directions.
So think about D* Lite in the sense that is a repeated best-first search, which is where the A* principles come in through a graph with changing edge weights as that graph is traversed, meaning that as we travel from our start node to our goal node along a path that we are planning, we can see that edge weights are changing.
And in this example, I'm modeling an obstacle coming in place between this node here and the goal, which means that we have removed that edge.
And that's effectively the same as changing an edge weight.
We're just modeling it as changing this edge weight to infinity, which means that edge can't be traveled along.
And as I mentioned before, we also view this as replanning through relaxation of path costs.
So once we have moved to this point, how do we find the path from here to here when this edge has been removed, essentially?
So we want to make D* Lite efficient.
And it's a repeated search through the graph as we traverse it.
So you'll see that since we're using these g values, if we maintain a formulation where we're measuring the distance to go or to reach a node from our start node, this is not preserved when we move our start node.
And we move our start node when we're updating the position of our vehicle.
So let me show this through an example here.
The g value, or cost to get from one start node to our current node, s, here is the combination of these two edges.
And so, it sums to 4.
Now when we move along that path that we've planned to this next node, the g value to get to that same node is here too.
So this isn't preserved.
And that means that we'd have to potentially reformulate things in our cue, which would be very inefficient.
So what we do is we reformulate our search.
And it's perfectly valid to reformulate our search to search in the opposite direction.
We're essentially measuring for our g values now the cost to reach the goal from a specific.
And instead of heuristics, then measure the costs to get from this specific node, or to get from the start node to our specific node.
And so we search from the goal backwards through the graph.
In this case, we see that regardless of whether our start node, we preserve our g values, and our cost to travel from s to that goal.
That's what we like, and we we'll keep that reformulation to maximize efficiency.
So here, I want to give you a kind of overall understanding of the D* Lite algorithm before we dive into specifics of what it's doing.
First we want to initialize all nodes as unexpanded.
And what exactly, it means expand a node neighbor or cover.
And then we're doing a best-first search from the goal node to the start node until the start node is consistent with its neighbors.
And I'll again say what that means later.
We then move our start node to the next best vertex, essentially our node moving along the plan that we created.
And then we have to see if any of those edge costs change.
We track how our heuristics change, and then update the source nodes of those changed edges.
And we essentially repeat this process until, again, we've converged to a solution, and we keep moving.
So this best-first search, the A* Lite part of the algorithm, is where most of the computation is going to be occurring.
However, the incremental component is actually how can we adjust our edge costs to make sure that we're using most of the information that we previously gathered as efficiently as possible.
So I'm going to walk you through various steps of this.
First, I'd like to walk you through this step of moving to the next best vertex, just so you understand that.
So we can extract a path, given the path costs that have, through this simple argument, which says that we map the successor, by which I mean we restate what the start node is, because we're moving from one state to the next state, which becomes the new start node, by this form, which essentially says that we're taking the best path in the sense that we're minimizing the total cost that would be gained traveling by that path to this node.
So in a sense, the g value to get to this node, to the red from the green, or color stretching back through the graph, by this path here, is 7.
But to get to this node from the lower node is 10.
And so our g value that is optimal is 7 in this case, and know that we pick the successor as the green node up here.
So now to walk through some principles that appear in both of these steps.
I want to show how we handle weight changes locally.
And we do this because we don't want to track weight changes throughout the entire graph.
We want to handle this efficiently and separate them, so we only need to handle the changes that really impact our problem.
So this is the same formula that defines our successor, or our g values for a given node.
But we can see that this may no longer hold when edge weights change.
The example that I have here is that I've changed edge weight cost from 6 to 1.
And so now we can reach the red node, searching back through graph by a new optimal path, coming down from this node here, which would make our new g value five instead.
So this hasn't been preserved.
And then these changes then propagate to our predecessors, because the costs for the processors are each dependent on the g value for their successor.
And so when we hit a node, we have to propagate that change all the way back through the graph.
So to do this efficiently, we take an A* like approach, where we update the lowest cost nodes first, and we don't expand a node until it is the next lowest cost.
To help us do this, we're going to store additional fact.
We'll call this rhs.
Now the meaning behind rhs, it comes from the term right-hand side, which in the paper world, was first introduced and made more sense than it does so here.
What you can think of your rhs value as is essentially you're corrected your g costs.
When edge costs change, how G should be signed so that it's correct for a different graph.
And when your rhs value is not equal to your g value, that means that we have what is called a local inconsistency.
So given the logic that your rhs value should be what your g value would be corrected to be, what would your rhs value be for this graph here?
5?
5, yes.
So this is the same graph that I presented previously, and I walked through how g should be 5. rhs should be 5 here.
And what we're tracking is rhs and g differently.
So g is what it should be, but we haven't got the dated g to be 5 yet.
So we have two different types of local consistencies that we distinguish between.
We have what we call local overconsistency, where g is greater than your rhs value.
This is going to behave more similarly to Dijkstra's algorithm, where the value that we're associating to any given node now is higher than it should be, and we're relaxing down to the true value.
However, we have a lovely underconsistent case that we have to handle slightly differently.
So now I'm going to walk through updating and expanding nodes.
What I mean by updating a node is recomputing the rhs value, and then placing that node on the priority queue, if that node is locally inconsistent.
So to give an example here, I've changed this value, moving from 6 at the bottom to 1, and recomputing rhs, as we did beforehand.
Now we see that the node is locally inconsistent.
In this case, it's locally underconsistent.
And so we place that on the priority queue.
The priority queue values are based on this new formula.
This new formula essentially handles how we see G and rhs.
In essence, we want to take the minimum value of one of these, so that we can handle it the first time that it will cause changes that propagate through the graph.
And so this is essentially our ordering on our priority queue which is essentially the same coupling that we saw for A*.
But here, we just have g of s replaced by the minimum of g of s and rhs of s. And so, here your minimum of g and rhs is 5, plus a heuristic, leads to a total cost associated with this node of 9, 5.
So we expand our five prior nodes by then taking them off the priority queue and changing g. Now I said here that we have an inconsistent.
And so, we update this in much the way that we would expect.
Your g values are relaxing down to your rhs values, and so we just set g to be equal to rhs.
In this case, our node is now locally consistent, and it's going to stay that way.
So can anyone think of some good reasons why I go through the process of changing rhs and keeping track of that new value, only to put on the queue, and then take it off again, instead of just recomputing what g of s should be in the first place?
Does anyone have any ideas?
[INAUDIBLE]
What you can think of is that you're putting something on the queue.
If we update that value immediately, there are multiple nodes that could change the same node that we just considered.
By, which I mean more work changes as they propagate back through the graph, could lead to changes in rhs value of that specific node.
And so, what we would need to do if we can recompute g of s every time, is put it on the queue to signal that we're recomputing g of s, and do that recomputation, and then new changes we've done, perform that again, do that again and again and again and again.
And we want to avoid that situation.
So what we're going to do is we're going to keep our rhs as essentially a temp value that can be adjusted when it's in the queue.
So when something is on the queue, it's on there once, and it can be updated and taken off.
But we know when it's been taken off, that it's been taken for the correct time, and won't need to be handled again
So if I understand this correctly, so if you had a single edge cost that we changed, then you could just do what you just said, in terms of, you could just update the thing, and probably should be predecessor rate
Yes.
So if you knew for sure that nothing else was coming back and down, then we could just do that change and propagate to all the predecessors, because nothing upstream of that node is affected
So am I understanding correctly-- so you're basically running the programming, because you're keeping the costs to goal to devote, and then what you're doing is every time you have an edge that changes to a value that could potentially make that node better than it was before, so if it changes the value of that node to something better than it was before, you would change that node, and then queue all the predecessors, because those be changed as well
Yes
I mean, if those nodes do not change, you don't do anything.
You're moving from the queue.
They also change a better bell and keep propagating until you have nothing else to [INAUDIBLE]
Yeah.
But we only do the propagation, or we queue changes that occur earlier in the graph, but we only actually perform the change resetting the g value, when it's expanded
And then the reason for that is for efficiency when you have more than one edge changing?
Yes
So you could basically have multiple paths affecting the same node.
And in this situation, if you actually allow your changes will occur and multiple edges, that's when the importance of the queue comes
That's exactly
Which is actually, it prevents you from redoing the same work over and over again
That's exactly right.
Because if another change came through, then we'd put all the predecessors back in the queue again, and we'd do all of our updates for all of them as well.
We're avoiding that process
And the queue ensures that we're only updating-- and I think that'll cover it.
But in the manner in which we terminate, we don't necessarily-- we don't expand every node in the queue.
We only expand those we need in order to find the shortest, and then we retain the queue for that search in case we need to update those in the next round of the search
Right.
So I have to correct a mistake in my language that I said before.
I said that the node in my examples that I gave before was highly consistent, [INAUDIBLE] wasn't consistent.
I apologize for that mistake.
Other consistency is the easy case that we're handling here, where when we expand that node, we have to take g of s and set that to be equal to the rhs value for expansion.
And then we propagate that effect to all of the predecessors at that point, which means we update the predecessors and put those on the queue if it hasn't had an impact on them.
This means that the node is not low key inconsistent, and it's going to remain that way in that fashion.
The underconsistent case, which was the opposite of what I showed, is the more difficult case.
In this case, the old path cost was better than the new rhs value.
Rhs value has increased to above [INAUDIBLE]..
In this case, we can no longer relax down to that g value from our Dijkstra's algorithm respective.
So we have to think of a new way to handle this.
What we do is we essentially set our g value to be equal to infinity.
And this is an extra step that's going to cause the node to go back on the queue an additional time.
But what this mechanism assures is that that node goes back on the queue at most one additional time, which means that for D* Lite, we only ever examine any node at most twice on the queue.
Once we've set g of s to be equal to infinity, then we can essentially relax down to the rhs value from the perspective that we had before.
So we set g of s to be equal to infinity, and we update all the predecessors of s and s itself this time.
As I was saying before, we need to put that node node back on the cue so that we handle it when rhs has hit its minimal value.
And this means that that node is now going to locally consistent or overconsistent.
And we handle overconsistency in the case that I've shown you before.
So to give you an idea of how exactly this works, I'm showing you an example here.
We start off with this graph, and we introduce two changes.
We have increased this edge cost to 5.
And so rhs has now increased to a total of 10.
But I've also changed a value here for edges that you can't can see.
But assume that the rhs value of this node has also changed.
So now we can put all of these nodes on a cable ordered in this way, computed using the formula that we've seen for it.
We expand the node s1 first.
And because it's underconsistent, we set g to be equal to infinity.
Then we propagate to all predecessors and s1 itself, meaning those go back on the queue.
Now, assuming that node changes of any relevance happen in these nodes, we recompute the costs for s1, which has now changed to 14, 10, because we're using the minimum of infinity and 10, which is 10 plus 14, going behind the node s2.
Next, we expand this node s2.
And because that was an overconsistent case, we can just set g to be equal to our rhs value.
We then have to propagate that to its predecessors, which includes s1, which has updated its rhs value.
So if we didn't do this, we wouldn't have handled this effect correctly.
When we set our rhs value here, we then have to put this node back on the queue.
So previously, s1 was already on the queue, but the effect has to be propagated to this node, which puts it on the queue again, which would mean adjusting the values, because rhs has been reduced by 1.
Both the costs here have been reduced by 1.
Then we expand that node.
We have an overconsistent case.
We're setting g to be equal to rhs.
And finally, we've completed our graph search problem.
So now what we have to handle is an additional complication that occurs between several steps, tracking how our heuristics have changed.
So we want to carry over prior queue, which was what was mentioned previously, between our different searches as our stock node moves.
But the problem is that we're moving from our start node to a different point.
Recall that our heuristic value is measured from any given node to the stock node.
So when our stock node has changed, the heuristic value is not going to be the same value to the new stock mode.
So we want what's already on the queue to be comparable to what is being computed for our new value, so we can use the math and the algebra that we've already done.
So how we handle this is we introduce something called the key modifier, where we move from the previous start node, which here I indicated as s last, to the new start node, all the heuristics for admissible heuristic are lowered by at most a heuristic from the last start node to the new start node.
So now, when we add new nodes to the queue, instead of subtracting that value from everything that's already on the queue, what we do is we just increase for all nodes that we put on the queue, their values, by this new modifier.
Because essentially when we're taking off of the queue, all that matters is the relative differences between the two.
And so, instead of subtracting a value from everything on the queue, adding that value to all new values that occur on the queue will achieve the same purpose.
And so our key modifier is initialized to 0 at the start of the algorithm, and is increased every time that we update our start by this new value, by the heuristic between the last start node and the new start node.
And then we update our total cost to include the edition of the key modifier fire for our f1 term.
So now we've covered all of the D* Lite algorithm basics.
And we can actually walk through the slide that I showed you before with the math.
So when we start, we initialize all of our g values to be equal to infinity, and all of our rhs values to be equal to infinity, except at the goal node.
And that's to ensure that we always have a node to start from our search.
We best-first search from the goal node to the start node until that's locally consistent and expanded.
We should now know what that means.
And that signal that we have found our best path, we move according to our movement rule, and then if any of our edge costs have changed, we update our key modifier, and update our rhs and queue positions for the source nodes of change in edge costs, meaning the nodes from which those edges originated.
And then, we simply repeat from two, following best-first through the principles that I've walked you through for overconsistent and underconsistent nodes.
And as a reminder down here, we always search or sort by this very long couplet of two values.
So now, we're going to walk through an example of [INAUDIBLE]
OK.
So again, here's the pseudo code that Ben was just presenting.
And we're going to go through a quick example with five nodes, so this isn't going to show all of the properties of why you would want to use incremental path planning, because when you have only five nodes, you're going to end updating a lot of them a lot of the time.
But after the example, you'll see how it makes more of a difference on a larger graph.
So in this graph, we're going to have prior nodes that go A, B, C, D, and G is our goal node.
The robot will start at node A, and its goal is to find the shortest path to node G, except that the graph might change as the robot goes through, or the robot might gain new information.
So initially, the robot thinks that every node is available.
And it knows that the path lengths are all 1, except for from D to G, the path length is 10.
So this is a bi-directional graph.
Every edge goes both directions.
And the heuristic, as Ben described, will be with respect to the start node.
In this case, we'll just say that the heuristic is the number of nodes to the start node.
So we're ignoring the fact that this edge length is 10.
So first, we'll initialize all the g and rhs values.
So we have infinity at every node, except for the goal node, which has an rhs value of 0.
So this means that all of the nodes are correctly locally consistent, because their g and rhs values are the same, except the goal node.
So we put the goal node on the queue using the formula that is on Ben's slide, which I'll put here for reference.
So the key is going to be these two terms.
First, we have the minimum of g and rhs plus the heuristic value plus the key modifier, which is initially 0.
And then we have the second term, which is just the minimum, g of rhs.
So you can see how we would get 3, 0 here, because the 3 comes from the heuristic, and everything else was 0, so we just added a bunch of 0's.
So that's everything what happens for the initialization.
And now we have one node on the queue.
So we'll have to dequeue first.
This is an easy question
The node on the queue
The one node on the queue.
So we dequeue G. So we're going out to update its g value.
So as Ben said, the rhs value is sort of a look ahead.
You can also think of rhs as being the shortest path that we found so far, and G is the guaranteed shortest path to that node.
So this one is trivial.
The shortest path from this node to itself has length 0, because its the same node.
So now we're going to update the rhs values and its neighbors.
So it has two neighbors, C and D, and in each case, the rhs value will be the minimum-- this is not actual math, but its the minimum of the edge cost, the d value of a neighbor plus the edge cost.
Which is the same thing as the edge weight or the weight w. So we rhs is 10 here, because it's the distance from g to do, and then it's one, which is G to C. So again, we can calculate keys for those.
So now we'll dequeue another node.
What should we dequeue next?
C. Because it has a smaller key.
So we dequeue C. We update its g value.
So this is saying, now we know that the shortest path to C actually has length 1.
And we can also compare this to A* star algorithm by drawing a search tree.
So in the beginning, we had A*.
In the beginning, we just had node G, because we were searching from G backward through the goal.
And then we expanded that back to get C and D. And then A* to keep track of the path length to that node plus the heuristic at each node.
So the pathway to C was 1 plus the heuristic 2, give us the value of 3.
So if we go back a moment, we can see-- is that OK?
So you can see that this 3 is the same as the first point of the key, and path length one is the same as the second part of the key.
And then similarly at D, we have the path length, which is 10, plus the 2, give us 12.
So this 12 is the 12 here, and then the 10 is the path length.
And so you can see how if we were doing this the A*, you would also dequeue C next.
OK.
So now, we dequeue C, and we're going to update the rhs and its neighbors.
So B is pretty simple.
Previously, it's rhs was infinity.
Now the shortest path to B that we've found so far has length 2.
So here we have B was a path length 2, plus a heuristic of 1 gives us 3.
But what about D?
Previously, its rhs value was 10, but do we now have a shorter path to D, given that we're expanding C?
Or is 10 still the shortest path?
That is the shortest path, through
Yeah.
So now that we're expanding C, we see that you can actually get to D in only two units instead of 10 units.
So that's better.
So we'll update the rhs value to 2.
And that means that we're going to change D's key accordingly.
And the way that maps to A* is, so if you already have the path length is 2, plus the heuristic is 2, gives us 4.
And the fact that we're changing the key to 4, 2 means that we're never going to expand this D, because A* uses an extended set, so it would always expand this D with a shorter cost before this one.
So that's indicated by-- removed by changing the key on the queue.
So what do we do next?
Which node do we dequeue?
B?
Yeah, because it has the smaller key.
So we'll dequeue B.
We'll update its g value, which is again saying that we know now that the shortest path to B is 2.
So we dequeue B, and then we expand it.
How many neighbors does B have?
Three
Three.
So we're actually expanding to all three of those neighbors.
We have A, C, and D. Can we still see this?
We can.
So at A, it's pretty simple.
We have the rhs value is 3, because it's the shortest path through GCB to A.
So that's 3, plus the heuristic 0 plus 3.
What about-- oh, see, it doesn't change, because-- you could go through B and back to C, but-- oh, C shouldn't actually be on here.
Sorry.
C shouldn't be on here, because we don't want a loop in our path.
OK.
So at D, can we do better than this path length of 2?
No
No.
Because what happened was originally, we had the path length of 10 through G, and then we had the path length of 2, 3 C, and this path through B is not as good.
So we're going to keep the same rhs value and key at D. And we can see that in a lot of he A*, where we would have, if we did do GCBD, we would have a pathway of 3, plus the heuristic of 2 is 5.
But we would never actually want to expand that D, because it's not as good as this one.
OK. Now what would we dequeue?
Looks like we have two nodes in our queue currently

A.
So we dequeue A.
That's where we trying to get to, because that's the current start node.
So we set the g value to 3, which means that that's actually the shortest path.
And we're done.
We found the shortest path.
Well, we're done, except that the path might have to change.
So one thing you notice here is that we never have to actually set D's g value.
So it's possible, maybe there's still a shorter path to D, but it doesn't matter.
So now the robot will move to its next best place, which is B, and we'll update the heuristics accordingly, because this is our new start node.
So now, again, our heuristic is the number of nodes away from that green node.
And we're keeping the same items on the queue from before.
But we also have to update the key modifier.
So now what happens is an obstacle appears.
So there's an obstacle at C that the robot can now see because it's moved closer to node C, but it couldn't see it before.
So now, we're going to indicate that by keeping all the edges, but changing their rates to infinity.
And that means that the robot could try to get to see, but it wouldn't ever get there.
So what we do next in the algorithm is we're going to update all of the rhs values associated with the source nodes of these edges.
And in this case, the edges go both directions, so all four of these nodes are affected by the infinities.
So let's start with node C. What would be its new rhs value, given that rhs is the minimum from one of its neighbors to it of the g plus edge cost?
Infinity
Infinity.
because all of the edge costs going to C are infinity.
So what would be the key?
The tricky part here is the first part of the key is the minimum of g and rhs.
So the minimum of g or rhs is what?
So it'd be 3, 1?
Yeah
Yeah, 3, 1, because the minimum of g and rhs is g, which is 1.
The heuristic is 1, and the key modifier is 1.
OK. How about node B?
What's the new rhs value?
So it has a neighbor here with g as infinity, a neighbor here of g is 1, except the edge length is infinity.
And then this guy with g is 3
So it's infinity, 4?
4.
Yeah, because currently, we still think that you can get to A and B units.
So this is actually going to be 4.
So this is one of the sillier things that D* does, but luckily, it doesn't do this very much, so it doesn't actually really cost too much time.
And then how about D?
What would be the new rhs value?
5, 3?
rhs
So it has been 3 neighbors.
This edge length is infinity is useless, so that's useless.
This one has g of 0 plus 10 would be 10.
Yes, so this is the best one.
G is 2, plus edge length of 1.
So that would give us 3.
So now what do we dequeue?
C?
C. So this is one of those tie breaking cases, where both of them have the first value is 3, but C has a better second value in its key, so we'll dequeue C. Is this over or under consistent?
So the definitions of over and under consistent that we had earlier, we had if g is greater than rhs, then that's overconsistent, and we set g equal to rhs when we're updating g. So that was the case that we've seen so far.
But then if g is less than rhs, it's underconsistent, and we set g to infinity.
So what is this?
Underconsistent
Underconsistent.
So the reason intuitively why we're going to set g to infinity is because we're saying there's some sort of contradiction here.
This one doesn't make any sense.
It's less than the rhs value, so we're going to just reset g and sort of start over with this node.
So we set that to infinity.
And in this case, we don't actually need to add the node back onto the queue, because it's now consistent.
It's now locally consistent.
So next, we'll dequeue the one with the smallest key, which is B.
B is also underconsistent, so we're going to set its g value back to infinity.
So this is sort of resetting the weird thing we did earlier that didn't make any sense.
We're going to update its neighbors' rhs values.
So what would be the new rhs value for A, now that we know that we don't actually have a path B?
Infinity?
Infinity.
Yeah.
And then, how about rhs value to D?
Because currently, D got its rhs value by going from A to B to
10
10.
'Cause now the only way-- rhs is 10.
The key would have that one also.
Is that right?
Yeah.
Because currently, the best path to D, and actually, the best path to D you can see is from G to D, because you can't go through C anymore.
So now, we'll dequeue node A, because it has the smallest key.
It was also underconsistent, so we would set G to infinity.
So we'll propagate that to its neighbors.
And if we continue doing this, then we find-- eventually, we dequeue this node, and in this case, we had to dequeue everything, so now the node's are locally consistent.
So what's the best path that we've found from B to the goal?
B, D, G?
B, D, G. And that's also the only path in this case.
And we had to update these neighbors.
OK.
So here's our new shortest path.
Now, the robot will move to node D. But now, what's going to happen-- I think we updated our heuristics.
But now what happens is it turns out it's a moving obstacle, and it starts following the robot.
So now the obstacle has moved to there.
So as humans, we can see that, obviously, the shortest path is through C, but the algorithm doesn't know this yet.
So what would we do next?
So we just updated our edge weights.
So now all the edges to B are infinity.
But some of the edges to C are no longer infinity.
So we need to update the rhs values accordingly.
So we'll update all of the rhs values that were affected by this change.
In this case, G's rhs value never changes because the distance from G to itself is always 0.
And this one doesn't change because we still use the G value here.
OK.
So now again, we can start dequeuing and planning a new path, so we'll dequeue this node.
We'll propagate to its neighbors.
Dequeue that one.
Update the rhs values.
But in this case, we don't-- let me go back step.
So we dequeue this node, and we update its G value, but we don't actually need to continue dequeuing things.
We should-- oh, this rhs value should be updated.
Oh, it is updated.
Why is this rhs infinity?
All edges to it are infinity
Yeah.
All edges to it are infinity.
You can't get there.
But we don't actually have to bother expanding B, because we've already found the shortest path.
So this gives you sort of a hint at when D* might be efficient.
Because even if there were a whole bunch of nodes over there, I wouldn't need to consider them, because we found the shortest path.
So now, the robot can move up to C. And let's say the obstacle chases the robot again.
Do we need to find a new path?
No.
So intuitively, we don't, because the obstacle's not in the way of our current best path.
But we can also see algorithmically why that's the case.
So first of all, we'll update all of our rhs values accordingly, because we've changed all of the edge weights.
But the reason algorithmically why we don't actually need to consider any of these change values is because this node has the smallest key, which means that this is the node that has the shortest path to G anyway.
It is smaller key than all the other nodes that are on the queue.
So dequeuing things wouldn't actually help us find the shorter path.
So the robot can move to the goal, and we've succeeded.
Any questions on that?
OK.
So now then we'll talk about sometimes when you would want to use incremental path planning, other than this five node graph
Thanks.
So as Jess said, there are sometimes good reasons to use incremental path planning, but there are also good reasons not to in certain circumstances.
In certain circumstances, we can actually formulate graph problems, where a problem is worse by using incremental planning.
Remember that I said that D* Lite puts any node on the queue at most twice.
And so we can compare that to A*, or something like that, that does full replanning whenever a search occurs.
That pus every node on the search queue at most once.
So D* Lite is most beneficial when putting the nodes on the queue, potentially more than once, leads to less nodes being examined overall.
So here is essentially what I just said.
So A* might perform better for certain problems if we only have to consider a small amount of nodes anyway.
So putting nodes on the graph twice would actually lead to more expansions.
And so, to get at an intuition of why that might occur if changes are close to the start node, I want to show you an example here.
So here, we're actually showing an example that's derived from [INAUDIBLE].
So we're searching forwards here from the start node to the goal node.
And that's just a different reformulation, so don't let that confuse you too much.
When we walk forward through A* with a good heuristic, we might move directly from the start node to goal node, and find that path.
Now, if a change occurs somewhere upstream of the start node, when we replanning, we essentially have to replan the sections that are beyond the change that's occurred.
So everything that's here, to an extent, it's relatively insensitive to this change.
It could occur that we would have to take an entirely different route.
But for the most part, it's likely that changes are going to couple to our existing path and so forth.
And so we have an efficient search that will just go around from the path that we've already planned to our goal node.
So in this sense, we have an example where incremental path planning would be very efficient.
In this case, which is a different one, we introduce an obstacle very near our start node.
This means that most of our existing path is to charge.
So where we start out from our start node to our goal node, we have to do a lot of recomputation from scratch.
And so an incremental searching algorithm is going to have to undo all searches that we've already done, and then search through almost the entirety of the graph again.
So intuitively, in this case, just restarting with A* from a blank slate, which we would search directly down to the goal, instead of doing reparations that would send us in one path, would be a better idea in this case.
So since we're powering from the start to the goal, in this example that we're showing, it's worse to use incremental path planning algorithms when your changes happen near your start node.
For something like D* Lite, we search back from the goal node to the start node, it's better if changes occur nearer to the start node, which for our vehicles that we normally see that have limited sensor horizons, is what's going to occur in the most case.
So in those type of situations, incremental path planning is going to be very efficient
So I understand how this could be a problem for LPA* and D* Lite over finding a single path, where you have like a single source or destination or something like that.
But what if it's incremental, all-pairs shortest-paths, where you've computed-- I'd assume you wouldn't experience the same problems if your algorithm was computing choice paths between all pairs of nodes indirect
Yeah.
It already exists.
Well, that depends.
When you're redoing recomputation, between all the nodes in the path, it depends if our obstacle has changed [INAUDIBLE] of something there that we to conceive.
And by that I mean imagining this graph we've computed, D, the paths and path costs, to get from every single node to every single other node.
Now, if introducing an obstacle here only changed the path cost to get from our stock node to our goal node from example, then replanning that specific pair of paths from start node to the goal node would use data that already exists from the other nodes to the goal node.
We already have that.
But if introducing an obstacle would cause changes throughout the entire graph, then we can potentially create what would have to be a very well-connected graph, where again, incremental path planning would probably perform worse in that case
Yeah.
So you could still end up in the same situation [INAUDIBLE]
Now these algorithms are interesting, because they allow for some extensions that allow us to do some some very different things quite easily.
One example is greedy mapping, in terms of how can we design algorithm using what we know about incremental path planning, to hit every single node on our graph.
And what we do is then we introduce a faux goal node, essentially a node that doesn't exist that we can never reach.
And we add to our initial beliefs, our initial understanding of the graph connections between all nodes to this faux goal node.
Now, it's false in the sense that whenever we move our start node to one of the nodes that we've examined before, what happens in terms of the changes of the edge costs is removing that edge that connected that node to the goal node.
So what this means in principle is that our vehicle, our poor vehicle, can never reach the goal node.
Whenever it reaches a node that it thinks it can get to the goal node from, that path disappears.
But what that causes it to do is when it's replanning, it finds a path to a node that we haven't visited before, because it thinks there is a path that leads from that node to the goal node.
And so this means that it's efficient, but replanning a root from his current understanding to a node that it hasn't examined before.
And so it can apply something like the D* Lite or LPA* with this formalism in order to search through the graph in its entirety and make sure that in a greedily efficient manner, we hit every single node in the graph
Is this [INAUDIBLE] TSP problem, or the traveling solutions problem?
I'm not very familiar with the TSP problem
You try to-- and you try to cover all the Cs, and try to come up with a route that is the most efficient, and try to visit every known C value once
OK. Then in that case, yes.
It's not going to be globally efficient.
As you said, it's going to be an approximation for it.
In a sense, from each node that you visited, you're searching for the next best choice.
So it's possible you can corner yourself.
But it's a way that we use what we already implemented and the methods that we've already established to tackle a different type of problem
And all this [INAUDIBLE] usually if you have a strategy which is visit the closest unvisited node, instead of making this whole incremental path from this study.
If your policy is, always move to the closest unvisited node, how worse does that get compared to this?
I think it would behave quite similarly.
What we're just trying to show here is that we can implement that problem as an extension if this happened.
It may be an efficient way to do it
[INAUDIBLE] probably like, zigzag
Also, if you're just going to the nearest connected node, if you corner yourself as we showed here, then you could potentially run into issues if your algorithm doesn't then do an extended search for it.
If it only considers nodes in its immediate neighborhood that hasn't visited, once it corners itself and doesn't know where to go
Just keep going through the graph until you hit the next inhibited node, assuming that you know how many invocations you have to do
Sort of like the random walk type approach?
Well, I mean, kind of like the greedy version of it.
You kind of start from a node, and then you keep searching, you'll see, what is the next unvisited node, and you pick that.
Kind of like-- [INAUDIBLE] some you see what you have covered.
You see what's the next best move.
You make that move.
And you're done.
[INAUDIBLE] So you probably don't have this [INAUDIBLE]
This basically ensures you're going to find the shortest path back to the next unvisited node, I guess.
We also can use these incremental path planning algorithms for anytime planners.
And the benefit of an anytime planner is that we quickly reach a plan which is suboptimal.
And then from there, we are repairing this plan to approach optimality.
And this is useful in the real world where you want to get a solution that comes out quickly, and we want to start along our solution, and then repair it as we go.
And once we start it, we need o save some more time if your reparations might lead to small changes and that type of thing.
And this can be handled quite efficiently using an incremental path planning.
So what happens is we apply something, idea, called inflated heuristics.
And so these inflated heuristics are essentially increasing our heuristic values that exist on our graph, in our true graph, by a cost of scale factor.
And this is typically very large, so differences in heuristics become very large.
This means that once we start along a path, we continue along it.
And that causes us to very quickly find the first path that actually works.
Then, we can view reducing those heuristics back to their original values as essentially a form of changing our [INAUDIBLE] causes or-- so it's a form of replanning.
And then by applying the principles of an incremental planner, we can efficiently use the data that we already have in order to perform the search and repair our solution as we go.
Finally, we want to make explicit to you guys how we can actually apply these algorithms to mobile robotics.
Because it may not immediately clear how we go from a full space to a graph.
Joe on this kind of briefly when he talked about we discretize the world.
But we want to give you guys some ideas of how we might do this cleverly.
So in order to build a graph system out of our world state, we differentiate between holonomic systems and nonholonomic systems, which their difference is just in terms of whether your constraints are differential in nature or not.
Then we are going to outline three methods here very briefly, cell decomposition, using a visibility graph, and sampling-based construction methods.
So for cell decomposition methods, we might to do what you would expect, take the free space in this region and split it up into various different cells.
And each of those cells is going to represent a node in our graph.
And how we can do this is we split the region up based on each of the vertices of obstacles.
So we draw a line vertically from each of our vertices that goes in the direction away from the obstacle.
So at the top here, it's going up and out of the obstacle.
But in the lower left corner there, both up and down, don't intercept the obstacles.
We draw them in its entirely.
So we can continue this throughout the entirety of the state space, and then we can attribute to each essential cell that we've defined through these regions, a single point that represents the midpoint of that cell, which is going to be our graph node, because we attribute a single set of coordinates to each of our graph nodes.
And we also, to make sure that our transitions between these cells are consistent, additional nodes at the midpoint of each of these lines that we've drawn, where we make the distinction that from here, we've actually drawn two lines.
So we draw the midpoint between all of those.
This allows us to essentially create a graph through the entire system, and we connect each node at the center of any given cell to nodes that are on the boundary of any given cell.
This allows us to create a full graph through the system, that we can move through, guaranteeing that none of these paths, if we follow them exactly collide with the obstacles.
Now, this works in environments where the obstacles are 2D polygons, and the path is far from optimal, as we can see, but we can plan an optimal path over the graph that we have defined.
And this only works on holonomic systems.
We can also define what is called a visibility graph, where we essentially draw a boundary between both our vehicle down there at that bottom left, and boundaries around our obstacles.
These boundaries are sufficiently sized so that if the center of the vehicle that we're defining is along the extended edge of the obstacle, that the vehicle will not collide with that obstacle.
Then for each of the vertices of the extended boundary that we've defined, we add in a new node, and we draw all lines between all nodes, which can be connected without intersecting the extended obstacles, which we've drawn here.
And this leads to a graph that we can, again, plan over.
This, again, works in environments where the obstacles are 2D polygons.
But what's nice about this system is that we do get an optimal path, provided that we assume that we assume that our extension of the robot itself is valid, or doesn't act too much-- isn't too excessive for constraint.
Finally, going back to some of the random sampling methods that we mentioned at the very beginning of this lecture, we've introduced sampling-based roadmap construction, wherein we choose nodes randomly or deterministically throughout our environment.
The difference here, though, is that we then create a graph out of this system, and that we reason over that graph using something like an incremental planning algorithm.
And so when we create that graph, after we've sampled, we can use the [INAUDIBLE] neighbors of each node, or we can connect each node to every node within a certain radius, which is what we've drawn here.
Finally, in order to actually move between nodes, we need to have some knowledge about the dynamics of the vehicle, which is what we're discussing here.
And one method that we can do this for nonholonomic systems is we can distinguish between different paths that use turning right, turning left, and traveling in straight lines.
It turns out that the shortest path between two points can be defined using one of these systems.
And so we just have to search over a finite number of options in order to find the best path that can connect any two nodes.
Hopefully, that has given you some intuition into how we can apply these incremental path planning [INAUDIBLE] systems.
Are there any questions?
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, everybody.
Today we're going to talk about semantic localization.
So first I'm going to talk about what is semantic localization and what is the motivation for it.
Then we'll go through an algorithm that will allow us to localize.
And then we'll go into how to actually add semantic information into this algorithm.
So our focus what we're coming up with this was the orienteering Grand Challenge.
So in orienteering, basically you have a map and you have a compass, and you have to go around to various checkpoints.
So as you can imagine, this is sort of difficult, because you don't know where you are on this map except what you can tell from your compass.
And so we weren't sure how to do this either, so we asked some orienteering experts.
So basically the key ideas are, if you are disoriented, that's fine.
You just need to find a reference point so that you can figure out where you are.
You can think about where you last knew where you were, and estimate what your movement has been since you were there.
And that can give you some options as to where you are.
And you can look around for features that you can identify uniquely on the map.
And this is an example of an orienteering map.
If you just have what's over here, it's not very useful.
You can't actually tell what those things are.
You might guess that green is grass, but you don't actually know what they are.
This map isn't useful until you actually have a legend to go along with it so that you can say you have roads, or foot paths, or pits, or different things on the map that you can actually identify.
So this is the difference between how robots typically localize and how humans localize.
So humans think about places in terms of features, in terms of what you can do with them, things like that, whereas robots might measure distances with laser scanners to form a perfect map of the entire space, whereas humans might think about obstructions such as rooms and their relative location to each other.
So we can define semantic information as signs and symbols that contain meaningful concepts for humans.
Basically different sorts of abstractions.
And why might we want this semantic information?
Well, one very important application would be improving human-robot interaction.
So say you want a robot that can make coffee.
It has to understand commands like, go to the kitchen.
Get a mug.
Turn on the coffee maker.
This is the language that humans think in, and it's very useful if a robot can actually understand this sort of language as well.
Know what a coffee maker is, what it does, and where it is.
And additionally, you can get some performance and memory benefits.
You're not storing this full map with the distance to every single surface.
You're just storing locations of key information.
And this means that your search base is a lot smaller too.
So you can prune everything that's not related to kitchens or coffee makers, and get a performance that way.
Finally, the fact that you can use simply a camera instead of a complicated laser scanner means that this is a lot cheaper and more accessible for robots to have.
So we've talked about what semantic information is.
So now let's define what semantic localization is.
So basically, it's localizing based on semantic information rather than metric information, like distances.
So for our Grand Challenge, we have a map with labeled objects and their coordinates.
This is our semantic information.
And we want to basically look around and say, what can we see in our field of view?
And from that, we want to figure out where could we be on this map.
And it's important to note that map building is actually a very different and hard problem with a lot of other research on it.
In our case, we've been given a map, and we're simply talking about the problem of localizing based on that map.
And now I'm going to let Matt tell you about particle filters, which is an algorithm for localization
All right, so my name is Matt Deyo.
Now we're going to talk about particle filters.
So this is a tool that we're going to teach you to use, specifically for state estimation, using the given math and measurements.
So what is localization?
We just went over it.
Specifically, it's the question of, where am I?
For any system to work, the robot needs to know where it is on a map.
This is not so simple of an answer.
Metric localization.
As was said before, it is quantitative.
So how many meters are you from this wall or from your origin?
What's your orientation in degrees?
And here's some examples of that.
You can convert between coordinate frames really easily if you have metric.
So the localization is, well, in our case, mathematical.
The problem statement is, you have a control u.
You're going to observe something about your state.
It might be a camera image.
It might be laser scans.
And then essentially on your map, you're going to use probability to figure out where you are.
So this equation specifically is taking into account the probability of being out of position at your current time based off your previous position, the observation that you're taking right now, the command variable you gave it, and then your map, like I said.
So that's built on the observation noise, actuation, and then the belief.
We're specifically looking at the belief here for the particle filter.
So in our case, we're approximating it by several points, also known as particles.
So we're going to look at an example here.
I have to actually press the Play button.
OK.
Here's a quick demo.
[INAUDIBLE] So a quick YouTube demo of a particle filter in action.
So I'm just going to show it first.
So pause.
The red dot here is our actual robot position.
And it's trying to localize itself in a maze.
There's black walls throughout here.
And it looks like a grid world, but that initial distribution we had was completely random across all the walls.
So it's taking in observations of probably laser range finders.
So it essentially knows that there are walls around it, and it knows that there's not a wall in front of it.
So that centers all of these guesses on the center of hallways.
Obvious it got rid of things that were close to the walls.
There's a low probability that it's right up against the wall because of the observations it's taking.
And we're going to revisit this example at the end of the presentation.
Particle filters.
So the method that we're going to teach you has four easy steps.
One is not repeated.
So the initial step is your first sample of particles.
If you don't know anything about your state at the start, then you can sample just a distribution of your entire space.
And then the repeated steps are updating weights, resampling, and then propagating dynamics.
So we're going to show you a toy example.
This is a really simple example just to get the idea across.
We're focusing on just one dimension.
We have an aircraft at constant altitude.
So take that variable out.
The only thing we're trying to localize to is a distance x over a map.
The sensor is just a range finder down to the ground at this constant altitude.
And then the math is a known mapping of x location to ground altitudes.
And we're about to see what that means.
The goal here is determining where you are in the mountains.
So here we have our plane.
As you see, we know the map below, and we know that it has to be flying at this altitude.
So with a range finder, you can measure different depths down to the mountain.
So here we have long distances, a medium distance, and a short distance if it's directly on top of a mountain.
Constant altitude, unknown x.
That's what we're solving for.
And then we actually have a noisy velocity forward.
So we know how fast the plane wants to be traveling in this direction, but obviously with some noise, it could be travelling a little faster or a little slower.
So the first step, sampling.
Our initial belief here is that we don't know where it is with respect to these mountains.
So we're going to actually sample with a uniform distribution.
Well, this is as uniform as I could get it.
So over this range, those were our initial particles.
Essentially guesses for what the x location is.
We're going to update these weights based on the observation.
So we get our first observation from our range finder, and it's this length.
So it's a long length.
It's, I guess, green in this case.
This is our measured value.
Here are all the values that we expect to see at these particles.
So this is pretty easy to calculate, based on the map that you have.
The position of each particle maps directly to a range finder measurement.
So we're going to compare them.
So the likelihood of getting these.
So measuring the likelihood of each of these occurring.
And we're actually weighting the particles.
So based on what we measured, these are most likely, so they get a larger weight.
And we are most likely not on top of a mountain at this point.
So those get smaller weights.
So we're going to resample, given that.
So we're going keep the same number of samples the entire time.
That's just how this particle filter works.
Except we're going to resample across our new distribution.
So these are the weights that we just came up with.
And we're going to resample over them.
So now it's more likely that it's here, or here, or way over there.
And then the final step is propagating.
So this whole time that we've been filtering and updating our weights, the way forward.
So we need to take that into account.
So we have a forward velocity.
Let's say this range is 0 meters per second, to 5 meters per second, to 10 meters per second that you can see on this plot.
So we're most likely moving five meters per second.
So essentially, this is your change in time between sensor measurements.
Obviously if you're only getting sensor measurements at 10 hertz, then you can propagate in between each of those.
So here we are.
Using our new sample particles, propagating them forward.
For example, this one moving there is a low likelihood.
That's a large distance.
And these moving at a shorter distance is more likely, given our model.
So we have those new ones.
The new weights are now based in the probability of it transitioning to that point.
How likely is it for the plane to move that far, essentially.
And then we repeat.
So we're repeating the steps again.
We're going to get a measurement in from our range finder.
We're going to compare it to the map, to our particles, keep the ones that are most likely and weight them more, and then propagate them.
So as an example, here's time step 2, when we're measuring the uphill of this mountain.
Time step 3, now we're halfway up the mountain.
So positions that we've kept are at similar heights, so here and here.
And then we can slowly get down to differentiating it.
So now that we're on top of a mountain, the only pattern that matches that is here, and maybe over there.
And then eventually, we get down to these two.
And then eventually, as this mountain drops off, it's a unique position, where it goes farther than that one did.
So in the end, our particle filter thinks that we're right around here or here.
And finally, there's a pretty high chance that we're over that valley.
So we'll looking at this, again, now that you know how to do the filter.
And this will make a little more sense now.
So they started with the uniform distribution.
They had no clue where they were at the beginning.
And as the robot moves forward, they are resampling.
And the measurements are changing, obviously.
Because it's seeing these two walls, and eventually, it sees that there's doorway to the left.
And you can keep going forward as well.
So eventually, that geometry doesn't line up with any other spot on the map.
Here, we see it nose into this hallway.
I think this top hallway is the only one on this map that's that long.
But it still don't know which direction it's going.
It hasn't been able to differentiate that yet, but it's about to.
So the only remaining particles are here and here.
It knows it's in the middle of a hallway.
And it knows it's moved about two blocks now without seeing a wall directly in front of it, which that doesn't occur anywhere else, without having another turn-off.
So it's about to solve itself.
Because eventually, it sees this wall over here.
And those observations don't match up with what it's actually seeing.
So there we go.
There's a [INAUDIBLE] particle filter successfully working for a little robot with a rangefinder.
I went too far.
There we go
I'm David Stingley.
And now, after we've gotten an idea of why we want to use semantic localization, and how to use particle filters to get an idea of our guesses for initial positions, we're going to talk about how we can use these two to actually implement semantic localization onto a robot.
So hearkening back to the implementation idea, we have three important parts.
We have a belief representation.
We have the actuation model.
So once we have a location, how are we going to move?
As we said before, if you don't exactly know how fast you're moving, there's a probability you move to a bunch of different locations.
And then finally, we have an observation model, which is the probability that you see some certain observation, given you're in this newly simulated position.
If we continuously solve for our most probable x, then that's our location.
So that particle that is the most probable is the place that we guess we're going to be.
So let's look at a pseudocode for what a semantic localization would look like.
So as long as our robot is moving, we're going to make observations.
And those observations are going to be [INAUDIBLE] z1.
Then for those observations, we're going to start off our particle filter and guess a certain number of probable locations.
We're going to use our actuation to update it.
And then we're going to simulate observations at that location, and compare what we expect to see for that particle on our map versus what we actually saw that we made our observation.
This is going to be our update.
And this is what we're going to focus on understanding, since a lot of the rest of this is covered by a particle filter.
So what kind of models can we pick, because we need to define what our observation looks like.
And you get a lot of choices.
In metric localization, you'd use a labelled laser scan.
You'd have perfect information about the environment.
And so you can see everything.
It might be nice to use a scene of objects at specific locations, but that requires once again, a lot of information.
Now you need to know where the objects are.
You need to have an idea of its exact specific locations and orientations with respect each other.
Maybe it might be nice just use, like, a bag of objects.
I see four things in my view pane versus three.
These are all choices, and we're going to take a look just really quickly at what the facts of these are.
As we've stated before, there's a lot of choices in observation.
It can get complicated really quickly.
So just to impress that upon you, imagine if you used laser scanners when you have three objects here-- for instance, a house, a couple of trees, and a mailbox.
You check each line for an intersection.
And then you have to figure out what counts as your detection, since you're going to have to differentiate what's in your scene.
The problem with that might be is that, well, you saw an entire wall, for instance.
Where do you want the house to be?
So let's say we assume that objects were a point at some point.
You either completely see it or you completely don't.
That way, you don't have to half-see an object.
You just check to see if whatever you want your center of mass to be intersects with your current view plane.
If it does, then you're good.
The issue with that is that, for instance, something center of view isn't inside the scene anymore, you just completely ignore it.
We have the exact opposite problem.
So you might want to make it a bit more complex and have polygons, parts of objects.
Do you see some percentage of something?
How much of it is in the view plane?
It adds in a lot of chance for errors.
And that's, I guess, the big point here to remember, is that depending on how we characterize our observations, we have different ways to get things wrong.
So let's say, for instance, for an observation like distance and bearing to some new scenic object, what can be wrong?
Well, you have noise inside of your sensors.
Your sensors might have limitations.
You can't necessarily see to infinity.
So an object might be too far away for you to identify correctly.
It might be rotated in a way that you can't necessarily interpret it correctly.
How about if you want to have your observations in classes of objects, so of just everything being an obstruction, now you have different types of obstructions?
Trees are different from mailboxes, of course, in which case you have a classification error.
What if you see a tree and you just think it's a really, really big mailbox, or you see a mailbox and you think it's a really small tree with a funny, little, metal top?
These kind of errors can then make your scenes look incorrect.
If you decide to have sets of objects, well, what permutations matter?
If you don't have a way of differentiating elements in the set, then you don't know if two trees and a mailbox with the trees on the left and the mailbox on the right or vice versa aren't the same thing?
So with that in mind, we're going to be talking about how we're going to solve this question of given some position and your synthetic view, how likely is it that it's your actual observation?
And for that, we're just going to change what this probability statement looks like to make it a little more concrete.
In this case, Z is the set of observed objects that you have.
So we're going to say that there are objects in the scene, and we put them inside of this value, Z.
So for instance, you see a house and you see a mailbox.
That would be Z.
Your set of objects is just two things.
We're going to use the bag of objects approximation.
Y of x is going to be the set of objects you expect to see given your map for the position that you're at.
So given a position X, Y is a set of objects that you would see.
So you might be in a position where you can see a house and a mailbox.
Then Y is is a house and a mailbox.
And X is just going to be your position.
That's the element that we're getting from our particle filter.
So in our example, as you can tell by how much I talk about it, we're going to use just two things, trees and mailboxes, because I like them.
So here's my map.
There's going to be a long road, and off to the side, we have trees and mailboxes.
And let's say that your robot wants to be a paper delivery boy.
So he needs to be able to figure out where the mailboxes are and how far down the street he is.
Because if he were to throw the paper wrong, he'd throw a paper into a tree.
So in this world, our robot is going to be here, represented by an orange hexagon.
And it has this field of view.
So given this field of view, what does Z look like, our actual observation, one tree and one mailbox?
Well, for this case, what we're going to say is, we're assuming that as long as the thing intersects with our field of view, we're going to see it.
So simple finding that.
We're just going to say that we see both trees and this mailbox.
Once again, we talked about how difficult it is to do this observation.
So for a simplifying assumption, let's say that we just completely see the tree.
So that's great for our actual robot position.
But when we start spawning particles, we need to figure out what we're going to say they synthetically do.
So say we spawn a particle here, and we spawn one when we're just slightly off the road.
We deviated a little bit.
We're further forward than the actual position.
We drove into somebody's house, another guy's house in the woods.
Then what is each of synthetic observations for these given points?
Determining this is going to determine how we actually get that probability calculation.
So what do we need to consider here?
Well, the first thing we need to consider is classification.
Like we said before, with this set of objects approximation, it's important that you understand if you classify things correctly or not.
Past that, like we asked before, we said, wait, why didn't I just see a tree and a mailbox?
Well, we need to know if we saw everything inside of our field of view.
Maybe we did miss something.
Maybe instead that old scene saw just one tree, one mailbox, even though the other tree intersected it.
And noise can happen in reverse, so we could accidentally see a tree when there actually isn't one.
And finally, we could see things overlap.
We kind of ignored this before, but what if two trees were right on top of each other?
It might make it kind of difficult for you to successfully see that there are two trees there instead of one.
So to start with, we're going to strike this assumption.
It will become more evident later why this is important.
But for now, we're just going to assume that every observation corresponds to only one object being seen.
Otherwise, you could end up infinitely expanding your scene.
Think about it.
You see a tree.
There might be a possibility that that tree is two trees.
Well, you saw two trees, then.
So maybe there's a probability that each of those two trees is also two trees.
And you keep going and keep going, until the entire forest is behind one tree.
It would be kind of bad for doing probability calculation, because you'd eventually have to cut that off at some point, so that your algorithm actually finishes.
So for our purposes, we're just going to cut it off at the start and say that everything's just one object.
It's just error.
So now we need to talk about if we classify it correctly.
If we can solve this equation, what's the probability that our classification is right?
So for now, we're going to make two simplifying assumptions.
They're going to remove the two problems that we had before.
And don't worry, we'll relax them later.
For our first assumption, we're going to assume that we see everything inside of our field of view.
So that means we're not going to have any misconceptions.
And we never see something that doesn't exist.
So we have no false detections.
Everything that's in the scene, we see successfully.
If it's not in the scene, we don't see it.
So given that, and we spawn a robot here, and it has this field of view, What does this robot see?
Three trees
Yes.
It happens to see three trees.
So remember our assumptions.
We're going to say here that our actual observation for wherever our robot is is one mailbox, and two trees.
And we can see that the synthetic robot that we made saw three trees.
So what are some other forms of Y that we can make that would also map to this?
What's the way that we can take our Y and transform it so that it maps this Z?
What kind of action would we have to take on this?
If you were thinking that we'd have to misclassify one of these trees, you're correct.
And remember, this is just a set of objects.
So this doesn't have to be the first tree that got misclassified.
There's three of them.
We could have any permutation of these trees get misclassified.
So it becomes important.
And we're going to introduce this concept of this operator, phi.
And what phi is going to do is it's going to be a way to map the permutations that we could have of misclassifications for Y to look like Z.
That way, we don't have to try and write all the permutations down.
It's possible to do this essentially with a matrix, like a permutation matrix, that just reorders the elements that you have.
So what does this probability look like now?
We're going to use the lower case z, y and i to represent each individual element of those sets.
So for some element in the actual observation, z, what's the probability that some element in our synthetic observation matches it?
Well, for that we need to pick-- well, there's some probability of being wrong or a misclassifying, or classifying correctly.
So we're going to use c to represent a classification matrix.
So there's some probability that we classify correctly, usually a higher probability, and then some small probability that an object becomes another type of object.
So how often we misclassify is represented here.
But we could add another term.
Let's say that our classification engine gave the weighted values of things.
It's common for neural nets and other types of systems that observe images to kind of give weights to their classification.
So you might want to use those weights to represent our confidence.
And there's a problem that that confidence determines that our classification might just be wrong out the get-go.
In that case, we want to know what's the probably that the score is likely to be that type of object.
And then we could have other things.
For instance, let's say that our classification was way better at identifying mailboxes from the front.
And if we turn the mailbox to the side, it gives poor observations.
And it tells us if it thinks that the mailbox is on the side, so it doesn't really know.
In that case, maybe having the bearing of the object inside of our synthetic view allows us to determine another probability for misclassification.
The important thing to notice here is that we can keep adding more terms to this.
The more specific your classification can get, the more terms you can introduce into it.
So add some confusion to it, and make the probabilities of misclassification smaller, and smaller, and smaller.
So with that in mind, we're going to look at what the probability for these entire sets is going to look like.
And what is really is going to be is it's just going to be a product over all these classifications for some selection of phi.
So you're going to have to take all the different permutations that you could possibly have, and for each of them, you're going to multiply all of these probabilities by each other.
In the end, it's going to give you that entire probability.
So it's all the permutations that you have, and then just the probability of each of those objects being classified as that type-- that you just map one set to the other set.
So now let's take a look at another scene.
So we spawned a particle above, fake robot here.
And it has this field of view.
What does this field of view look like?
If you said it looked like two mailboxes and two trees, you're right.
And so we're going to assume that we have the same actual observation.
We still see a mailbox and two trees.
And we're going to remove our old assumption.
We're going to say that it might be possible that we don't see everything in a synthetic robot's field.
In which case, how can we amp this synthetic observation to the actual observation?
Well, we just add in a probability that we miss identifying an object.
If we just didn't see one of these mailboxes, it looks exactly like this.
So for that, we want to capture what's the probability that we see nothing?
So say we have a synthetic view with some number of objects.
What's the probability we just miss all of them?
So the probability that we miss all of them, we're going add an assumption here, which is that we're going to say that there exists some probability that we see an object for a given synthetic view here, and that we don't see it with a probability one minus that probability.
So essentially, there's just two states-- either we see the object or we don't see the object.
And we're going to say that probability of identifying objects is independent.
So if we see one object, then that does not change our chance of seeing the next object.
Both of these assumptions help simplify the math here.
In reality, these might be strongly interlinked.
For instance, if your robot's camera is broken, the probability that it doesn't see one object directly correlates with the other ones, because it won't see any of the objects successfully if it has no camera any more.
If your robot drives into a wall, and can't see anything because it's staring straight at the wall, the same kind of idea holds.
But for the purposes of making it so that you don't have a lot of covariances, and a really, really big conditional statement of a lot of probabilities of seeing things, we make these assumptions for our items to be independent.
Independence means we can multiply our probability successfully.
So if we just don't want to see any of the objects, we just take 1 minus the probability of seeing the object for all the objects in the scene.
So what goes hand-in-hand with not seeing anything?
Think about if you had a robot far off in the distance over here.
What can it see?
Just two trees.
So now, we're going to remove the idea that we can't see things that don't exist.
So to make two trees map to two trees and a mailbox, we just made up a mailbox of some noise.
So what's the probability that we see a full scene when we can't see anything?
And if you're thinking it's going to map to a very similar formula, you're kind of right.
But we need to figure out a way to capture our noise statement.
Specifically, we are going to use noise as a Poisson variable, which means that there's always some probability of seeing an object out of nothing.
And it's going to be coordinated and according to this factor Kz we'll get to on the next slide.
You could choose a lot of different things to represent your noise in this case.
A Poisson variable was just chosen by the specific method that we used and implemented from another paper.
But with testing, you could find that different potential distributions map to your noise for your particular sensor better.
So with a Poisson variable, what we're going to have is we're just going to have the product of all of our Poisson variables times this Kz factor for the given scene that we want to map to.
Essentially, it's just the product of all these independent Poisson variables for each of our different objects that we want to create.
So with that in mind, what's this Kz factor that we're multiplying everything by?
What it's actually going to be is it's going to be a set of uniform distributions.
These uniform distributions are going to be uniform over all the classifications we could get for an object we spawned from the noise, and all the possible scores, and all these possible bearings for this synthetic object.
And if you remember a few slides ago or you rewind the video, you'll notice that these map directly to the categories that we put into that classification engine.
In fact, they should.
So if you added more things or took them out, you changed this uniform distribution.
The idea is that when you synthetically create an object out of noise, you might get any of those types of objects.
But if you needed to create a tree synthetically from seeing it in the noise, you might get a tree or you might get a mailbox.
Either one of them might show up.
If you had a different or more intelligent distribution for how you might misclassify things-- for instance, let's say your noise, whenever it shows up, always identifies trees.
Whenever you accidentally see noise as an object, it's always a tree.
Then you might only want to have a probability of seeing trees over here, and it would change you distribution.
But for simplicity's sake, we're going to assume that it's equally probable that any type of object shows up out of noise.
So now, we're going to put it all together, since we relaxed our assumptions.
So we have a lot of different things that can potentially map to this actual scene.
We have scenes that lack objects.
We have scenes that lack objects and therefore, need to add the object in.
And when they lack objects and have to add the object in, there's a chance they add in a different object.
And then maybe, they just misclassify something, like our misclassification idea.
But wait-- I guess we can make this more complicated.
What if one of them just wasn't seen, so we'd take one of these out?
And then we just see two things from noise.
As you could see, these keep getting more complicated.
But if you think about it, with our previous probabilities, is every single one of these is going to add a lot of probability terms to be multiplied by each other.
You can keep getting as complicated as you want it to.
If we removed our first assumption, we'd be adding in probabilities of two objects becoming one object.
The idea is like higher power terms when you're doing approximations.
The goal is to make sure that any particle you spawn could potentially be what you've actually seen.
But we want the ones that are very low probability to really be very low probability.
So we make sure that these incredibly complicated transformations are going to be so insignificant that they'll usually return to almost 0.
So if we wanted to solve this, we need to start by using a little assumption, which is the fact that we now have to fold in the idea that we could have some missed detection and some false detections.
So false detections would increase the number of objects we've seen over the number of objects that are actually presence.
And missed detections would reduce the number of objects that are actually present.
Notice that this always has to be same size as the number of expected objects, because you're trying to map whatever your synthetic scene is to the actual thing that you observe.
When you do this, it really turns into a large number of multiplications.
So this should look familiar.
This is our term for successfully classifying all of our objects, and we're going to multiply it by the probability that we actually identify it, since we could now miss things.
Then, we have to multiply that by the probability that for whatever objects we didn't see, that we actually missed them.
And then finally, we have to add in the noise, because the noise is going to make the objects that we're missing show up, so that we have a classification of the same size as the thing that we want to see.
Now, one thing you should note is that we added in phi way down here at Kz.
That's because we have to map these false detections that added to the number of objects that we could see, as well as the actual detections.
Essentially, we have to take all the objects that are seen here, real or not, and map them to all the objects we expect to see, because they're going to be one to one.
So let's take a look at a little video that shows what semantic localization can do.
So in this video, for a little heads up, we have essentially a scene where they have mapped out two things inside of a suburban area.
They've mapped out cars and windows on the path of the robot as it drives around an area.
And they're attempting to use just the information of the cars and windows visible on the scene in order to localize where they believe the robot to be during its journey.
[VIDEO PLAYBACK] - This is a video extension to the paper--
So going to mute that.
It's on.
So this is a few steps into the process.
And then it resets.
It's spawn a number of points for potential locations.
And then it very quickly localizes and finds its spectacular rotation.
These are all the identifications that are happening inside of the scene.
You can see the boundary boxes for cars and windows showing up, the cars in red, and the windows in green.
It expands out its distribution of particles.
And then shortly thereafter, it gets a couple of approximations.
And then it settles in on a location.
It happens to use a kind of like a centralized weight for where it is for the set of particles, and then it draws the car as being in that location.
If you let it keep running, it occasionally expands back out to make sure it doesn't fall into a local minima, and then compresses once again to where its strongest belief is.
Notice that it has a couple of seconds of having a very, very spread distribution, and very quickly converges into a singular point.
That's because a lot of these locations become very, very low probability after you've seen a couple of scenes into the future.
And we're going to pause this.
[END PLAYBACK] I welcome you to go see the video, to see the video yourself if you want to.
You can check the title.
It's pretty easy to find on YouTube, because it's pretty much the only one.
So hopefully, it works now.
Step back over here to the other side.
So why would semantic localization be useful in this manner?
In the example that was shown, it was kind of done on post-processed data.
They took a scene.
They drove around it already.
Then they did the identification on the video feed, and used that to do a localization.
We talked about before.
But people kind of use sparse information to do a lot of their-- or if you can't walk into a room-- people don't walk into a room and produce an exact laser scanned map of the entire area themselves.
But they can store important information, like seeing objects and tables that they find with their shins, and use those to move around.
Robots store that perfect map, if they can manage to make it.
But they don't really have a good understanding of what that map might mean to a human.
So that means that we're actually kind of like better at doing tasks with the environment.
If we wanted to go directly to a location, we don't have to look around and figure out on a wall what our distance from the wall is before we can then localize and move ourselves back over to the table.
We just say, oh, I want to go to the table.
And you look over there.
Oh, there's a table.
Table, got it.
So how can we make robots think a little more like humans, so it's easier for us to give them instructions?
If we can make the robot use a map that has just scenic objects like we use a map that just has scenic objects, then order to move between scenic objects are as simple as turning, finding the object, and saying, oh, well, I know where the object is.
So I know roughly where I need to be.
And then we start moving in process towards it.
In conclusion, semantic localization has a lot of-- I'm going to leave the references slide up while I talk, so that if you wanted to go and take a chance to see any of these papers, you definitely can.
Most of the work in this slide comes directly from these sources.
Semantic localization offers us the opportunity to take robots, use sparser information to potentially localize, find ourselves inside of spaces.
It also gives us a chance to have really tweakable factors for how you might understand where you are in a space intuitively.
If you think certain things are important, you can add them in as probabilistic factors.
If you don't, you remove them just as well.
Thank you.
And this is essentially the conclusion of our presentation.
I definitely recommend taking, if you wanted to use something like this, taking a look at this particular paper here.
Notice how it says, "via the matrix permanent?"
As I said before, a lot of these operations are a series of multiplications and some permutations.
There's permutation matrices.
There's matrix multiplication.
And there's a lot of ways to make matrix multiplication faster.
This paper details how you can take the math that was shown before that's pretty inefficient, and turn it into something that you can do an estimate of very quickly.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so as this XKCD comic points out, in CS, it can be very difficult to figure out when something is just really hard or something is virtually impossible.
And until a couple years ago, people thought this idea of image classification would be something that was closer to the impossible side.
But with the advent of deep learning typology, we've made significant strides in image classification, and now the problem's actually quite practical.
So today we'll be going through how the process of image classification with deep learning works.
So we're first going to talk about what deep learning is, and then we'll move into some of the image processing techniques that researchers use, followed by the architecture of the convolutional neural networks, which will be the main focus in our presentation.
We'll also talk about the training process, and then go through some results and limitations of CNNs and image classification.
So what is deep learning?
Well, the term is particularly vague, and it's purposely so for a couple of reasons.
The first is mystery is always good for marketing.
But the second reason is that deep learning refers to a pretty wide range of machine learning algorithms.
They do have some commonalities.
They all seek to solve problems of a complexity that previously, people thought only people could solve.
So these are more sophisticated classification problems than traditional conventional machine learning algorithms can do.
So how do they go about doing this?
Well, all of these deep learning programs tend to take all the processes that need to happen, and split them up.
They've got different parts of their program working on different things, all while performing calculations, and then at the end, it all comes together, and we get a result.
Of course, this isn't unique to deep learning, and lots of distributed systems decentralize their calculations.
But the key thing about deep learning is that every part is performing these calculations.
The calculations are not simple calculations.
They're not we'll do this one simple operation over, and over again on a lot of data, and then we'll get a result at the end.
Each part is performing some particularly complicated process on all the little parts before they come together.
So why is this architecture a good idea?
Why did engineers come up with this sort of decentralized, multi-layered complex process?
Well, we take the example of image classification.
It turns out that the human brain does a pretty similar process.
So here's the human visual system, and it's pretty much a hierarchical process.
So you begin by moving from the retina into the first areas of the brain, and as the information gets processed, it moves from one region of the brain to the other, and each spatial element of your brain is performing an entirely different calculation.
For example, the v1 area over here is picking out edges and corners, and then over here, a couple steps later in v4, you're starting to group those figures together.
And so the brain kind of operates in a way that is very similar to the way these networks operate.
So let's talk about to classify a face.
If I asked you guys how would you classify a face, what is the first thing you might do?
Well, as I mentioned before, the first thing out brain does is it finds these edges.
The first thing to do is identify where the face is versus everything else.
Now, does anyone have any idea as to what we could with the next step?
Julian, you have an idea?
Maybe you could group these edges together
Right.
We could maybe identify some of these features that we're working with.
So these are things like noses, and lips, and eyes.
And then what do we do after we have these individual features?
Steve
Well, maybe we can group some of those together
Exactly.
Yeah, we can organize them into what we know the pattern to be.
We know that a face has to have two eyes, above a nose, and then above the mouth.
So that is precisely what a neural network actually ends up doing, and we'll walk through the process of how it does this later on in the talk.
But as you can see, the intuitive way that we classify a face, and the way our brains are wired to do it, is pretty similar to the way got these neural networks to operate.
So like I said, we're talking a lot about these convolutional neural networks.
There are other types of architectures involved.
Like we mentioned before, deep learning is a pretty wide variety of algorithms, but we're going to focus on these CNNs.
To give you a precursor of how good these CNN are, this results from ImageNet competition.
So the ImageNet competition is basically exactly what it sounds like, a bunch of computer scientists get together, and see how many images they can correctly classify.
And their error rate was pretty high.
Almost a third error rate over here in 2010, 2011, and then in 2011, the CNNs were introduced to the topic, and the error rate plummeted.
As you can see over here in 2015, we've got a significant improvement in these ImageNet competitions.
So clearly, the CNNs have been very effective, and it's definitely been something that is exciting in the field and happening right now.
All right, so now we're going to move into image processing
OK, so Ryan gave us a nice overview of where we get this concept of neural networks, but let's take a time travel, and go into a quick history lesson.
So suppose I had a chair, and I wanted the computer to classify this chair.
I have some a priori knowledge about what sort of things make up a chair.
So I might be interested in looking at arms, and corners of the chair, legs, things like that.
So I would go ahead, and feature engineer my discovery scheme to be looking for specific things.
So I'm going to talk about some techniques that are traditionally used.
For example, chairs, doors, these things have corners.
So I might use an image processing technique called a Harris Corner Detector, where we basically look at large changes in intensity as groups of pixels move from an image to an image that indicate the presence of corners, and you can use common corners to say that OK, all of these images are chairs, or doors, or whatever.
Similarly, I want to say I have a bunch of pictures of chairs of different sizes, but that they all must have so many corners or something.
So typically, we use a sift algorithm to scale invariant feature transforms.
It basically says that across different sizes, I still should be able to extract information about the placement of corners.
Another common technique that's used in image processing is what we call HOG, Histogram of Gradients.
So basically, for example, in this image, if I want to say I want to find all the images that have faces in them, or consist of faces, let's say, I might come up with a template of a face that basically assigns gradients to groups of pixels that form an outline of what looks like a face, and then scan it across my sample images, and say OK, a face is present in this image.
Obviously, there are some errors.
A mead cap, and a logo back here have become a face, but this is the traditional approach.
But here's the problem, what if I don't actually necessarily know what features are the most critical depending on the dataset that I get?
I want the system itself to figure out what techniques to apply without having any a priori knowledge about the dataset.
So this is exactly the idea of CNNs, the convolutional neural networks.
We want the techniques to be learned automatically by the process, by the system.
So if I'm trying to classify faces, I want the system to figure out that eyes, and ears, and nose, these are the most important things.
Or if I'm trying to classify elephants, the ears and trunks are the critical features, without me having to say OK, we're going to do corner detection, so on, and so forth.
So this is the idea.
So to be able to understand this process in greater detail, I'm first going to go into a little bit of math, and the idea is to present the most fundamental operation here, which is the convolution.
So this is the formal definition of the two-dimensional convolution, and since we're working with images, we're only considering the two-dimensional case.
So in a more graphical presentation, which is a little bit easier to understand than just seeing the formula, the idea is that we have a kernel, or a convolutional filter that we seek to apply on another image, and that extracts some information about that image that we can use to help us classify the convolution.
So assume that this is our kernel, or this is our filter, and suppose this-- oh, there it is.
So suppose we're applying the kernel [INAUDIBLE] here to the image that's in green.
So the idea is we want to slide this filter across the image, and what we're basically doing this is a succession of dot products.
So at each placement on the image, we multiply the overlayed numbers, and the sum becomes the output image on the convolt output.
So this is basically the way the process works.
You probably notice that there's a reduction in dimension, and Henry will talk a little bit more about why this is.
Let me get to it, and then [INAUDIBLE] So let's see some examples of what information we get by applying the convolution.
So you see the image of a tiger on the top left.
When we apply a filter that's a low pass filter, basically-- it's a Gaussian-- then we get low spatial frequency information about this image.
So basically, we blurred it, and this tells us something specific that we might want to learn.
So the kernel actually looks like a two-dimensional Gaussian function that's been distributed across this three-by-three kernel.
Similarly, we might be interested in high spatial frequency information.
So in this case, we're looking at sharp features.
So horizontal edges or vertical edges.
So a question for you is if I have this kernel, which of these outputs when this kernel was applied to the original image, which of these outputs do you think it produced?
The third one on the right
Yeah, that's exactly right, and it's probably pretty easy to see why that's the case, given that the numbers are all horizontal bands here.
Lastly, we also may be interested in extracting information at a particular frequency.
So we can take the difference of a high pass filter and a low pass filter, and add it to your frequency you can extract information about that as well.
OK, one last helpful piece of information is that there's another way that you can think of the information that's learned at each stage because a convolution can also be thought of as a Fourier transform in the frequency domain.
You can think of the image transformation that way.
So from an image perspective, what a Fourier transform is is basically a sum of a set of sinusoidal gratings that differ, say, in frequency, in orientation, in amplitude, and in phase.
So you can think about the zebra image here that's actually a composite of different gradients that might look like this, and the Fourier coefficients would be how much of each of these pieces come together to make that final image.
So just to get a sense of what kind of information this could convey, we typically take a Fourier transformation, and break it apart into the magnitude and phase representation.
So you see magnitude, and you see phase.
So those images weren't particularly clear, but this is a really good example for this.
So if we take the Fourier transform of all the horizontal text here, you see how the magnitude reflects this, and you can go back to the math to understand why it's reflected in a vertical marking.
And similarly, if I were to take that same image, and rotate it, and then ask for the Fourier transform, you see how that information is contained very clearly in the magnitude spectrum.
So these might be things that a network would learn at each stage to try to identify this as a text, or as as body of text that's tilted one way or the other, so on and so forth.
So with that, we can now go into what the actual architecture of a convolutional neural is
All right, so as it was said earlier, in order to classify or detect objects, you actually need certain features.
You need to be able to identify these features.
And the way you can identify these features is using certain convolutions or certain filters.
In many cases, we don't know what these features are, and as a result of that, we don't actually know what the filters are to extract these features.
And what convolution neural networks allow us to do is actually determine what these features are, and also determine what the filters are in order to extract these features.
Now, the idea for convolutional neural networks, or the idea for replicating how the brain works started in about 1960s or 1950s after some experiments by Hubel and Wesel.
And what happened in these experiments, as can be seen here, is a cat was actually shown a light band at different angles, and the neural activity of the cat was measured using an electrode.
And the outcome from this experiment show that based on the angle at which the light was shown, the neural response of the cat was different.
As you can see here, the number of neurons, as well as the neurons that were firing were very different based on the angle.
So what you can see also here is really a plot of the response versus the orientation of the light.
And what this has led Hubel and Wesel to is the idea that neurons in the brain are organized in a certain topographical order, and at each filter, it fills a specific role, and the only fires when its specific input is shown, or when the angle is show, or the angle is specified.
Now, the first step to actually replicating how the brain works in code is really understanding how the building block, the neuron, works.
That's a quick reminder of 7012 here.
So a neuron is actually a cell with dendrites, nucleus, axon, and a terminal.
And what the neuron actually does is aggregate the action potentials or the inputs it gets from all the neighboring neurons that are connected to it through the timelines, and it sums these action potentials, and then compares them to a certain threshold that it has internally, and that would determine whether or not this neuron would fire an action potential.
And that very simple idea can actually be replicated in code.
An artificial neuron looks very much like a natural one.
So what you would have is a set of inputs.
Here we have three inputs that are summed inside of a cell, or a neuron.
The sum here is not just a regular sum, it's a weighted sum.
So the neuron specifies some weight, which you can think of as how much it values the input coming from a specific neuron, and then the input is multiplied by its weight, and then the total sum that the neuron computes is then fed into an activation function that produces the output that the neuron then basically produces.
Now, what we just saw here is really a simple neuron, a single neuron.
You can't really do much with just one neuron, so what you would do is combined these neurons in a certain topography, or in that case, we have a network with seven neurons organized in three different layers.
And what you can think of that is really as one big neuron with 12 inputs, and one output.
So for example, in the case of the chair that was previously mentioned, if you're trying to identify whether a specific image has a chair in it or not, these 12 inputs here could be some sub images, or some small areas of the initial image that you feed into the network, and the output here could be a yes or no.
Whether the image has a chair, or doesn't have a chair.
And that is really the concept behind convolutional neural networks, which we'll go into details in a bit.
So what each neuron would be doing in that case, is really just performing a dot product, which if you aggregate that with the dot products computed by each of the other neurons, you would obtain a convolution.
So what we have here is three inputs.
If the input, in that case, is an image or a sub image, then the inputs would be pixels.
The weights that you would be using here would be the filter weights, which is the filter that you use in the convolution.
And then the sum here would be the dot product of the weights and the inputs, and that sum would be computed by a specific neuron in your network.
Then, that would be the convolution step, and then that convolution step would happen at the first layer in the network.
So you would be applying this to the input, but you also would be applying this at the second layer, and third layer.
In that case, we're only showing what happens in the first layer.
The next step after the convolution would be the activation step.
So the dot product computed here would be there's a function that would be applied to the sum, and then that function would produce the output of the neuron.
And this is where the activation layer is.
You also have another activation layer here, and then a final one here.
What we just went through now are convolutions and activations, but this is not the only thing that actually happens in a neural net.
What we also have is a step called subsampling, which we will be talking about next.
For now, we will dig deeper into the activation, and specifically, what activation functions to use.
In that case, we can see that that's a neuron, and what the neuron is doing here is the weighted sum that we talked about, or the dot product.
And then the output from this would be fed into a certain activation function.
Common activation functions are sigmoid, tanh, or rectify linear unit, and we will go through each one independently.
So here, we can see the sigmoid activation function.
So what this function essentially does is map any input to an output in the range of zero to one, and it's defined as one divided by one plus e to the minus x.
The other common activation function is tanh, and that's any input to an output between minus one and one.
And then finally, would be the rectified linear unit, which maps an input to itself if it's positive, or to zero if it's negative.
Now, in theory, you could use any function as an activation function in your network, but that's not what you want to do in practice.
You want your activation functions to be non-linear for one main reason, that the goal of the activation function is actually to introduce non-linearity in your system.
And if all your activation functions are linear, then you would essentially be having a linear system, which prevents you from achieving the level of complexity that you would ideally want to achieve with a neural network.
And there's a formal proof for as to why you need non-linear activation functions.
They don't all need to be non-linear, but you need to have a least a few non-linear activation functions in your network.
And the proof is available in the appendix, or the link to the paper that has the proof.
So after we've discussed what happens at the activation layer, now we want to talk about convolution.
So as I said earlier, an image is obviously a two-dimensional image, but we're using RGB images.
So actually need three channels.
So what this means is that an image is actually three-dimensional, and each 2D matrix represents one channel.
One of them corresponding to R, one of them corresponding to G, one of them corresponding to B.
So a 32 by 32 image would essentially be represented by a 32 by 32 by three matrix, as can be seen here.
So what happens at the convolution layer?
So here we have a nice animation that shows what is happening at each convolutional layer.
So assume we have a five by five by three filter.
So what this is, essentially, would be doing is covering a certain patch in the original image, which is 32 by 32 by three.
So what can see here is that for that five by five by three patch in the original image, we have a neuron that is actually performing the dot product on all the pixels in that specific patch.
So what is happening here is the pixel values, which in that case, we have five by five by three pixels, are being multiplied by the filter values, and this operation is being performed here.
Then, after that dot product is performed, it's fed into an activation function, as can be seen here, and this produces the output of this neuron.
Now, this is what this single neuron is actually doing.
It's just covering that area of the original image.
What you would have in a neural net is many neurons, each covering a certain area of the original image.
And if you aggregate the output of all of these neurons, what you would be doing or performing is, essentially, a convolution on the original image.
And to formalize what happens here, or what's the output that's being produced from that operation, we can look at that from a more mathematical perspective.
So if you have an input of size H1, W1, D1, and you're performing a convolution with a filter, then the output, W2, would be related to W1 with the following formula.
So W2 plus W1 minus filter width plus one, and the same formula applies for the height, and the depth would actually be the same because in that case, we're using a filter that has the same depth, or three, as the original image.
So what this would produce in aggregate is if you have 28 by 28 by one neurons, each one performing a dot product on some pixels in the original image, the output would be an activation map of size 28 by 28 by one, and the output of each neuron would be one pixel in the activation map.
Now, if we go back to the points that we made earlier, one thing we said was that the reason you use a neural network is because you don't know exactly what features you want to extract, and you don't actually have specific filters that you want to apply to the image.
So ideally, what you want to do is have multiple filters being applied to the first image, and perform multiple convolutions, and this is what you can do with multiple neuron layers.
So what we described before was just for one neuron layer.
In that case, we can assume we have five different neuron layers, each one performing a different convolution on the original image.
So in that case, we would have 28 by 28 by one neuron per layer, and then if we aggregate all these neurons together, we need to multiply it by five, and that would be the total number of neurons we have in that specific number.
So this actually leaves us with a pretty complicated system.
It would have many parameters.
The neurons have weights, the number of neurons is also a parameter So how do we actually formalize that?
If we have an input volume of 32 by 32 by three, which is our original image, and a filter size of five by five by three, then the size of the activation map that would be reduced would be 28 by 28 by one.
Then in that case, we also said we have five different neuron layers that perform five different convolutions.
Then the total number of neurons would be 28 by 28 by five, and then the weights per neuron are five by five by three, which is 75.
In that case, we're assuming that the neurons independently keep track of their own weights.
This could be simplified to each layer having their own weight, which would tremendously reduce the number of parameters.
But in that case, just to get an upper bound, this leaves us with a total number of parameters of 294,000.
And this is just using a 32 by 32 by three image.
You can think of this as a pretty small image.
So if you have a bigger image, you will have many more parameters.
Great.
So what we just saw now, and described, were convolutions, activations, and these steps happen sequentially in a convolutional neural network, specifically as can be see here.
One step that also happens occasionally is subsampling, and we'll discuss that step in detail here.
So there are two main reasons why you would actually subsample your input.
One is to obviously reduce the size of your input, and your feature space, but also because you want to keep track of the most important information, and get rid of everything else that you don't think is going to be relevant to your classification.
And the common methods used in subsampling are either max pooling or average pooling.
We will describe max pooling here.
So what happens in max pooling is, essentially, you are dividing the image into different sub images, non-overlapping sub images, and you perform an at max operation.
So in that case, if we consider two by two filters, we would split the image, which in that case is four by four.
We'd split it into four sub images, and for each two by two square, we would take the maximum.
In that case, for the first square it would be six, then eight, then three, then four.
And the reason that actually works is because what you want to do is really keep track of the response of the neurons that-- or the highest response produced by your neurons.
In that case, for example, the first highest response in the first square is six, and that means that if you get that high of a response, it means that something has been detected in the image, or has been detected.
And this is something you want to keep track of as you move forward in your network.
And although this moves around the location of pixels, because you can think of that as subsampling an image, it does keep track of the information you care about because you only care about the fact that something has been detected in the image.
At this point, you don't really care about where it's located in the image, and you want to keep track of all the features that your neurons have detected in order to be able to eventually classify the input correctly.
So if you have multiple feature maps-- so in that case, if you have 224 by 224 by 64, what your subsampling operation would be doing is reducing the height and the width so the depth would remain unchanged.
So in that case, you would go from 224 by 224 by 64 to 112 by 112 by 64, and that would be reducing your output size by a factor of four.
And formally, what this would look like is if you have an input of size H1, W1, D1, the size of your output would be related to your input in the following ways.
W2 would be W1 minus the pool width plus one.
The same applies for H2, and the depth would remain unchanged.
So these are, essentially-- these are the steps that happen in a convolutional neural network.
What you could be doing is repeating these steps on a certain number of times in your network.
But eventually, you have to make a classification, and decide in our case, whether our image has a chair or doesn't have a chair.
So how does that happen?
So after you perform all these steps, there's a step that happens here that would allow you to make that prediction, and that step is usually called a fully connected layer, or a multi-layer perception.
And what this essentially is is layers that are very similar, or exactly the same as what you had before, except that every neuron in the layer is connected to all the previous neurons.
So what it's allowed you to do is really consider everything you currently have about your input, or everything that's left about your input, and compute a dot product on that, rather than focusing on a subset subsample of your input like previous layers do.
In that case, if you're actually trying to classify your input into four classes, you would ideally have four different neurons in your output layer, each one corresponding to one of the classes that you have, and then you would perform the same operation as you would in a previous layer, compute the dot product, and then once you obtain the values at every neuron, you would perform a normalization operation on all the output.
This organization operation is called softmax, or normalized exponential, and what it does is really, put more weight on the highest value.
And by computing the softmax at the output, you're able to compute the posterior probabilities, and allows you to make a more informed-- or basically make a classification decision on your input.
Great.
So that's everything.
And now, the next step will be talking about back propagation
OK.
So now that Henry has given us an overview of the entire architecture of a CNN, I'm going to quickly spend some time, and talk about standard preprocessing tricks and tips that people might use on the image dataset before they actually feet it through a neural net to classify the images.
So let's suppose we have a dataset x, and there are n number of data points in the dataset, and each point has a dimension, D. So they have D features per point.
So in this example, we use these graphs as an example.
Basically, our original data here has just two dimensions, and it spans this range of values.
So for example, if we want to center this data, what we would do is a mean subtraction.
So we basically subtract the mean across all the features of all the points, and we basically center it, and you can see that transformation here.
And then we might, again, go for normalizing the dimension so that you have it.
The data points span the same range of values in both dimensions.
So you can see that transformation, and how it's taken place here.
And we just divide by the standard deviation to do this.
Something that's very commonly done is called PCA, or Principal Component Analysis.
And the idea here is sometimes we have a dataset that has a very, very high dimensionality, and we would like to reduce that dimensionality.
So basically, what our goal is is to project the higher dimensional space onto a lower dimensionality space by taking the subset of those features.
And if you've seen a little bit of 1806 from linear algebra, the way we do this is by generating a covariance matrix, then doing the single variable decomposition.
And I'll gloss over the math now, but that's the idea.
And you can see here how the original data spanned two dimensions.
I would decorrelate it so that it spans a single dimension.
And even with this data, you might want to ensure that it's widened, which is the same deal.
You want the values to span the same range in both dimensions.
So then you would just divide by your Eigenvalues to get the widened data.
This last bit is something that's very commonly done as a preprocessing trick, though people aren't entirely sure why it works very well, or that it really does help, but it's something that people do, and it's called data augmentation.
So basically, if I have a dataset that contains a bunch of images of chairs, a bunch of images of tables, and then a bunch of images of say, trees, I might want to intentionally augment that dataset further by introducing a few variations on these same images.
So I might take the chair image, rotate some, reflect it a few more, scale, crop, remap the color space, or just kind of have a process that does this randomly to create more variation on the same dataset.
And this is a good illustration of why this makes a difference.
I've taken an image here of what looks like a waterfall or some spot of nature, and simply just inverted the colors.
And what I see, if I were to just see this image alone, it maybe looks like a curtain, or a bit of texture, or something.
And the idea is even to a human perception, these images have two very different meanings, and so it's interesting to see what effect they would have on a neural network.
And with that, we'll go over to image classification results
So, so far we've seen how convolutional neural networks are built, and certain image processing techniques we can use on the input images to get them into formats that are there for the classification process, but so far, it seems a bit abstract.
It's good to know how CNNs work, why CNNs work, but why don't we take a look at some of the practical results from CNNs, and what they're used for so that when you're done watching this lecture, you can go home, and try classifying images on your own time?
With that, let's first revisit the ImageNet competition.
I hope you remember the graph at the bottom from the beginning of the lecture, where we used this to motivate the use of CNNs.
CNNs came onto the picture in 2012, but the winning CNN from 2012 was used on the 2010 ImageNet competition as well, and it managed to bring down the top five error rate to 0.17, which is pretty much on the same level as how performed in the 2012 competition when it was first used, which was at 0.16.
So this just goes to show that these convolutional neural networks are the state of the art when it comes to image classification, and that's why we're currently focusing on that.
But you might be wondering what the ImageNet competition exactly looks like, what the images looks like.
So why don't we take a look at that.
As you can see here, these are images from the ImageNet competition.
Underneath each image is a bold caption, which is considered to be the ground truth, or what the competition believes to be the correct classification of the image.
Underneath that ground truth, you see a list of five different labels.
Now, these five labels are produced by a convolutional neural network, and the different bars-- the different lengthened bars, some pink and others blue, represent how confident the CNN is that what it sees in that image is that specific label.
As you can see in certain examples, the CNN is pretty confident in that it has a correct answer.
For example, when we look at the container ship, it's pretty confident that what's in that image is exactly a container ship.
There are certain cases when it doesn't get the correct label on its first try, but it does have in its top five labels.
For example, you can see grill and mushroom.
Now, the funny thing about the mushroom image is that what it thinks the image should be classified as is agaric.
And if you don't know, agaric is actually a type of mushroom, and in fact, it's a mushroom that image.
And it make sense that their confidence levels are pretty much the same.
Agaric is slightly-- it's slightly more complex that what it sees in the image is agaric.
But there are certain cases when the CNN fails to classify the image correctly in its top five levels.
This will be registered as a top five error, as you just saw in the previous slide about the top error rate.
One example here on this slide is cherry.
Now, the ImageNet competition believed that this should be classified correctly as cherry, even though there's also a Dalmatian in the background.
The CNN, on the other hand, is pretty confident that what it sees in this image is the Dalmatian.
But if you look at some of the other results within the top five, although it doesn't guess cherry at all, it does guess certain fruits that it may think look sort of like cherries like grape or elderberry.
So the CNN does actually pick up on two different distinct objects within the image, but as a result of how it's built, or its training set, it ends up classifying it as a Dalmatian.
But it goes to show you that CNNs could also be used not just as an image classification, but also as object detection, which we do not touch up on in this lecture at all.
So I'm not going to go further into that.
Now, this is all fun and all, but what about some real world applications?
So this is a study that they did at Google with Google Street View house numbers, where they used the CNN to classify photographic images of house numbers, as you can see here, of certain examples of these house numbers-- what they look like.
So what the CNN was tasked with doing was that it was supposed to recognize the individual digits within the image, and then understand that it's not just one digit that it's looking at, but it's actually a string of digits connected, and successfully classified as the correct house number.
This can be quite challenging, even for humans sometimes when the image is quite blurry.
You might not exactly know what the house number exactly is, but they managed to get the convolutional neural network to operate around human operator levels.
So that corresponds to around 96% to 97% accuracy, and what that enables Google to do is that they can deploy the CNN such that the CNN automatically extracts the house numbers from the images online, and uses that to geocode these addresses.
And it's gotten to a point were the CNN is successfully able to do this process in less than an hour for all of the street view house numbers in all of France.
Now, you might asking where this could be useful for.
If you don't have access a lot of resources to actually do this geocoding process where you match latitude and longitude to street addresses, then your only resource might be actually photographic images.
So you actually need something, hopefully not human, but some sort of software that can do this successfully.
And so this is, for example, a place in South Africa, a bird's eye view.
Not sure if you can exactly see, but there are these small numbers on top of each of the houses.
All of these numbers were extracted and correctly classified using this previously seen CNN.
Another example from robotics is recognizing hand gestures.
So obviously, robots come equipped with a lot of different hardware.
They can sense sounds, they can also capture images of their surroundings.
And if you're able to classify what you see-- if the robots is able to classify what it sees, then it can actually act upon it, and take certain actions.
That's why it becomes really helpful to successfully classify the images.
So this is what they did using hand gestures, where there were five different classes.
Each class corresponds to the number of extended fingers.
So a, b, c, d, the top row, corresponds to the same class.
They all have two fingers sticking out.
The bottom row has three fingers sticking out.
So that's another class.
And they get the error rate down all the way to 3%.
So 97% of the time, the convolutional neural net correctly classified the hand gesture.
And you can use these hand gestures then to give certain commands to a robot, and it can train the CNN to act upon something else besides hand gestures.
For example, if it's in some sort of terranean and you train it on certain images that you might find in nature, then it can take those classifications, and act upon it once it sees, for example, a tree, or some sort of body of water.
It's all thanks to image classification.
Now, obviously, gestures are not necessarily static.
You could be waving your hand, and so that would require a temporal component.
So it's not just an image you're looking at, but a video.
And so if follows that we can probably extend image classification into video classification.
After all, videos are just images with an added component, specifically time.
Obviously, the added temporal component comes with a lot of additional complexity.
So we're not going to dive into any of that, but in the end, it comes down to the same thing.
You extract features from the videos, and you attempt to classify them using convolutional neural nets.
So why don't we look at a study done, again, at Google, where they extracted one million videos from YouTube, sports videos, with somewhere between 400 to 500 different classes, and they used CNNs to attempt to classify these videos.
Now, they used different approaches.
They used different approaches, different tests, different types of CNNs that I'm not going to go into.
But as you can see here, these are certain stills from these videos where the caption highlighted in blue is what the correct answer should be, and underneath it, the top five labels that the convolutional neural network producers.
The one highlighted in green is supposed to be the correct answer.
So you can see on all of these, it gets it within the top five, and for the most part, within the top two, and it's pretty confident when it does get it correctly.
Now, when I said that they use different types of classifieds, some of them were more stacked classifieds, where they were just trained on stills within these images, while others were what they called fusion ones, where they sort of add the temporal component by fusing in different stills from these photo images together.
Now, the current accuracy rate-- the best one they've achieved so far-- has been around 80% accuracy within the top five label.
Now, 80% accuracy is nowhere near what we saw with the ImageNet classification, where in 2015, they had managed to get it up to 98% or 99% accuracy.
But obviously, there's way more complexity involved in this.
So it makes sense that it's not quite there yet.
But it does provide a good benchmark, and something to improve upon in the future as well.
Now, that being said, convolutional neural networks do come with certain limitations.
They're not perfect.
And so Julian will now talk about the limitations
Thanks, Ali.
So Ali talked about the ImageNet competition, and talked about how the recent winners have been convolutional neural nets.
So before, the best was about 26% top five error rate, but now they've actually gotten it down to a 4.9% top five error rate, and that was-- the winner of that competition, the 2015 one, was actually Microsoft.
They've got the current state of the art implementation, and so because it's the ImageNet competition, that means they can identify exactly 1,000 different categories of images.
So there are few problems, actually, with the implementation, or just in general with convolutional neural nets.
So one of them is that 1,000 categories, well, it may seem like a lot-- ImageNet is actually one of the largest competitions-- that's not actually that many categories.
So it doesn't contain things like hot air balloons, for instance.
So these things that children would be able to classify, the neural nets actually aren't able to, even in the biggest competition.
And each of these categories also requires thousands of training images, whereas you could show a child a couple examples of a dog or a cat, and they'd be able to, generally, get a feel for what a dog or a cat looks like.
It takes thousands of images per category for the neural nets to learn, which means that the total number of images you need to train for the ImageNet competition is over a million.
And so this leads to very long training times, even with all of the heavy optimizations that Ishwari was telling us about like how efficient convolution is, it still takes weeks to train on multiple parallel GPUs working together to train the net.
There's actually a more fundamental problem with neural nets as well.
So here on the left, we have a school bus, some kind of bird, and an Indian temple.
And all of these images on the left side are actually correctly identified by convolutional neural nets.
But when we add this small distortion here in the middle, that doesn't change any of the images perceptively to the human, this actually causes the neural network to misclassify these images, and now all three of them are ostriches.
So that's a little weird.
How does this work?
How did we find those distortions.
So here on the left side, we see how a neural network typically works.
You start with some images, you put it through the different layers of the neural network, and then it tells you a certain probability that it is a guitar, or a penguin.
So it classifies it, and so we can use a modification of that method by applying an evolutionary algorithm, or a hill-climbing or gradient ascent algorithm.
We take a couple of images, and we put them through, it classifies it, and we see what the classification is.
And then we can do some crossover between the images.
So we take the ones that look a lot like what we're training for in the guitars or penguins, in this case, and we take the features of those that identify very strongly as a guitar, and we combine those together in the crossover phase.
This is for the evolutionary algorithm.
Then we mutate the images, which is making small changes to each one, and then we re-evaluate by plugging it back in through the neural network, and only the best images, the ones that looked the most like a guitar or penguin, are then selected for the next iteration.
And this continues until you get to a very high identification rates, even higher than actual images of the objects.
So using gradient ascent, these are some of the images that you could produce if you start with just a flat grey image, and then you run it through this algorithm.
So here on the side, we have a backpack, and we can actually see the outline of what looks like a backpack in there.
And over here, we have what looks like a Windsor tie right here, but all of these objects-- and perhaps, there are things in these other images, but they seem to be lost in the LSD trip of colors here.
So that's kind of strange.
That's definitely not how humans do it.
So let's try a different method.
What if instead of directly encoding, which is where we change individual pixels, what if we change patterns in the images, like different shapes?
Then this is the kind of output that we get.
So in the upper left, we have a starfish.
So you can see that it has the orange and blue of the orange in the starfish, and also the blue of the ocean of the environment that typical images of starfish are taken in.
And you can also see that it has the points, the jagged lines, the triangles that we associate with the arms of a starfish.
But the strange thing here is that they're not arranged in a circular pattern.
They're not pointing outwards like this, like we would expect of an actual starfish.
So clearly, it's not latching onto the same large scale features that humans do.
It's actually just looking at the low down features.
Even though it's a deep neural network, it doesn't grab onto these abstract concepts like a human would.
So the reason for this problem, or at least why we think neural networks aren't as good as humans at things like this is the type of model.
So a human would have more of what's called a generative model, which means if we have examples here, these dark blue dots, say, of lines, a few examples of images of lines, then we could construct a probability distribution, and say that images that fall somewhere in this region are lines.
And over here, we have a few examples of giraffes, say, and so anything that falls in this region would be a giraffe.
And so if you had a red triangle in here, that would be a lion.
But if the red triangle is instead over here, it actually wouldn't classify at all.
We wouldn't know what that is.
We would say that's something other than a lion or a giraffe, but neural networks don't work the same way.
They just draw a decision boundary.
They just draw lines between the different categories.
So they don't say that something really far away from the lion class is necessary not a lion.
It just depends how far away it is from the decision boundary.
So if we have the red triangle way over there, it's very far away from giraffes, and it's just generally closer lions, even though it isn't explicitly very close to it at all, and that will still be identified as a lion.
So that's why we think we're able to fool these neural networks in such a simplistic way or in such a really abstract way.
So the main takeaways from our presentation, and the salient points are that deep learning is a very powerful tool for image classification, and it relies on multiple layers of a network.
So multiple processing layers.
Also CNNs outperform basically every other method for classifying images, and that's their primary use right now.
We're currently exploring other uses, but that's generally where it's at, and this is because convolutional filters are just so incredibly powerful.
They're very fast and very efficient.
Also back propagation is the way that we train neural networks.
Normally, if you were to train a neural network that has a lot of layers, there's actually an exponential growth in the time it takes to train because of the branching when you go backwards because each neuron is connected to a large number of neurons in the previous layer.
You get this exponential growth in the number of dependencies from a given neuron.
By using back propagation, it actually reduces it to linear time to train the networks.
So this allows for efficient training.
And even with back propagation and convolution being so efficient, it still takes a very large number of images, and a long time with a lot of processing power to train neural networks.
Also, if you'd like to get started working with neural networks, there are a couple of really nice open source programming platforms for neural networks.
So one of them that we used for our pset was actually TensorFlow, which is Google's open source neural network platform, and another one would be Cafe, which is Berkley's neural network platform.
And they actually have an online demo where you can plug in images, and immediately get identifications.
So you can get started very quickly with that one.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so today's lecture-- we're going to be talking about probabilistic planning later, and in these cases where you're planning a large state spaces is very difficult.
You do the MVP planning.
It could be stress that activity planning, or the likes.
But you have to be able to figure out how to deal with these state spaces.
So Monte Carlo tree searches is one of the techniques that people can identify, over last five years, is having an amazing performance improvement over other kinds of sample-based approaches.
So entity is very interesting from that standpoint.
And then if we [?
link it to ?]
the last lecture, then the combination of something, we just learn about [INAUDIBLE] and combine it with search, is very powerful, in this case, through the state-of-the-art techniques for that, as much as tree search [INAUDIBLE] later [INAUDIBLE] PROFESSOR 2: Good morning, everyone.
As Professor Williams just said, we are going to be talking about Monte Carlo tree search today.
My name is Eann and I'll be leading the introduction and motivation of this presentation.
By the end of this presentation, you will know not only why we care about Monte Carlo tree searches.
As Professor Williams said, there's so many algorithms out there.
Why do we care about this specific one?
And second, we'll be going through the pros and cons of MCTS, as well as the algorithm itself.
And then lastly, we will have a pretty cool demo on how it's applied to Super Mario Brothers and the latest Alpha Go AI that built the second best leading Go player in the world.
So the outline for today's presentation is, first, we're going to talk about pre-MCTS algorithms.
There are other algorithms that currently exist out there, and just a few of them to lead into why we do care about MCTS and why these other algorithms fail.
And second, we'll talk about Monte Carlo tree searches itself with Yo.
And lastly, Nick will tell you more about the applications of Monte Carlo tree searches.
So the motivation of these kind of algorithms is we want to be able to play games and we want to be able to create programs to play these games, but we want to play them optimally.
We want to be able to win, but we also want to be able do this in a reasonable amount of time.
So these three can train itself leads to different kinds of algorithms, and different algorithms with different complexities and time, or times to search.
And so that's why today we're going to be talking about Monte Carlo tree searches.
And you'll figure out in a few slides why we do care.
So these are the types of games we have.
You have this chart where there's fully observable games, partially observable games, determinstic, and games of chance.
And so today, the games that we care about are the games that are fully observable and deterministic.
And these games are games like chess and checkers and Go.
And we'll also be talking about another example with Tic-tac-toe.
So these pre-MCTS algorithms include deterministic, fully observable games, like we said earlier.
And the idea of this, and the nice thing about these games, is that they have perfect information, and that you have all of the states that you need and there's no opportunity for chance.
And so the idea is that we can construct a tree that contains all possible outcomes because everything is fully determined.
And so one of these algorithms, to address this, is the algorithm Minimax, which you might have heard before.
And the idea of Minimax to minimize the maximum possible loss.
That sounds a little weird in the beginning, but if you take a look at this tree, this red dot, for example, is the computer.
And so in the computer's eyes, it wants to beat its opponent.
And we're assuming the opponent wants to win also, so they're playing their best game as well.
And so the computer wants to maximize his or her points, but also knowing that the opponent, or the human, wants to maximize their own win as well.
And so in the computer's eyes, it wants to minimize the maximum possible lost.
Does that make sense to everyone?
Yes?
OK. And so in the example of Minimax, we're going to start with a connect, or a Tic-tac-toe board, where the computer is this board right here, and the blue Tic-tac-toe boards are the states that the computer finally chooses.
It's anticipating the moves a human could play.
So if you take a look up here, here's the current state of the board.
The current state of the board.
And the possible options for the human are this guy, this guy.
Nope.
Possible options for the computer, we have three different options.
And so you'll notice that this is clearly the obvious winner.
But in the state of Minimax, it goes through the entire tree, which is different from depth-first search.
It goes through the entire tree until it finds the winning move and the minimize of the maximum possible points it could win.
So is there a way we can make this better?
Yes.
I'm sure you've heard about pruning, where, in our human intuition, it makes sense.
Well, why don't we just stop when we win, or when we know we're going to have a game that allows us to win?
And so this idea is the idea of simple pruning.
And so when we combine Minimax and simple pruning, we have-- anyone know?
Alpha, beta.
PROFESSOR 3: Yes.
Our 6.034 head TA knows about this.
We have alpha-beta pruning, where we prune away any branches that cannot influence the final decision.
So in other words, you wouldn't keep exploring the tree if you already knew that a previous term would allow you to win.
And so this idea in alpha-beta pruning, we have an alpha and a beta.
And so the details aren't important for you to know right now, but the idea is that we stop whenever we know we don't need to go on any further.
So in the games that have Tic-tac-toe and Connect 4 and chess, we have relatively low branching factor.
So in the case of Tic-tac-toe, we have 2 to the fourth branching factor.
But what if we have really large branching factors, like Alpha Go?
In Alpha Go, we have 2 to the 250.
Do you see that Mini Max, or even alpha-beta pruning, would be an optimal algorithm for this?
The answer is?
No.
PROFESSOR 3: No.
And this leads us to out next section.
Our goal is going to talk about how we can use the Monte Carlo tree search algorithm for games with really high branching factors, and using the random extension to allow us to see, ultimately, how Alpha Go, which is Google's AI, was able to beat the leading Go player in the world.
PROFESSOR 3: All right, guys.
So this is the part where we re-explain the algorithm itself.
And before we dive into this, I want to make something really clear, which is that because these are technical details and because we actually want you to understand them, and because I definitely didn't understand this the first three times I read the paper.
I really want you to feel free to ask any questions on your mind, with the knowledge that, in my experience, it is very rare that someone asks a question in class that's [INAUDIBLE] OK, so really, whenever you have one.
OK.
So why are we doing this?
Well, the ideal goal behind MTCS is that we want to selectively build up different parts of the tree.
So the depth-first search way, the exhaustive search, would have us exploring the entire koopa tree, and that our depth is limited by looking at all the possible nodes of that level.
But what we want is we want-- because the amount of computation required for that explodes really quickly.
With the number of moves that you're basically looking into the future, we wanted to be able to search selectively in certain parts of the tree.
And so for example, if there are less promising parts over here, then we care less about looking into the future of those areas.
But if we have a certain move-- in chess, for example, there's a certain move where in two moves, you're going to be able to take the opponent's queen.
You're really want to search that region and figure out whether that's going to end up being a significantly positive group for me.
And so the whole goal of our algorithm is going to be growing this asymmetric tree.
How does that sound?
OK, great.
So how do we actually do this?
We're going to go over a high-level outline, but before we do that, let's talk about our tree, which you're going to get very familiar with.
Can people see that this is red and this is blue?
So this is our game state when we start our game.
We can be given a Tic-tac-toe board with a [INAUDIBLE] place, a game of chess with the lose configured a certain way.
And so our player, which is the computer, has three separate moves that it can take.
And so each of those moves are presented by a node.
And each of those moves have response moves by the opponent.
So you can imagine that if one of these is a Tic-tac-toe board with just a circle, that one of these is with that circle and the next place right by it.
And as you go down the this tree, you start understanding basically, it's the way that humans think about playing these games.
If I go here, then what if they go there, and then what if I go right here.
You try to think through the set of future moves and try to evaluate whether your move will be good in the long term sense.
They way that are going to expand our tree, as we said, to create an asymmetric tree is first of all, we're going to descend through the tree.
We're going to start at the top and we're basically, jump down some sequence of branches until we figure out where we're going to place our new node, which seems like a key operation here.
To create an asymmetric tree it's all about how you [INAUDIBLE].
For example, in this case, we're going to pick this sequence of nodes.
And once we get to the bottom and find every location, we're going to create a new node.
It's not very hard.
Then we're going to simulate a game from this new node.
And this is the key part of MCTS.
Once you get to new a location, what you're going to be doing then, is you're going to be simulating a game from that new location.
We're going to talk about how you go about simulating a game from this more advanced game state that what we started out with.
Does anyone have any questions right now?
We will be going in depth into all of these steps, but just in a high level sense
Just a quick question.
PROFESSOR 3: Yeah
To create the new node, is it probabilistic, just creating a new node as the most probable [INAUDIBLE] PROFESSOR 3: No, no.
You're creating some new node.
We'll talk about how we pick that new node, but we're just making a new node and we're not thinking anything about probability.
The next thing is that we're going to update the tree.
So whatever the value of the simulation delta was-- delta, remember-- we're going to propagate that up and basically add that to all of the nodes that are in that parent of that node in the tree and update some information that goes in there and that they're storing.
This is going to be good because it's going to mean that-- it's a lot like in search algorithms where you have trees that then the entirety of the tree remains up to date with the information from every given simulation.
And we're just going to repeat this over and over and over again.
And slowly, our tree will grow out until whenever we feel like stopping.
This is actually one of the nice things about MCTS, is that whenever we decide that we're out of time, like for example, if you're in a competition playing a champion Go player, you can stop the simulation.
And then all you have to do is pick between one of the best first moves that you're going to make.
Because an the end of the day, after you're doing all the simulation, we're still right here.
And we're still only picking between the movies that go immediately where we started.
Yeah
Could this [INAUDIBLE] good tree?
And then on some initial region of interest, or is it arbitrary how you get to create it?
PROFESSOR 3: We'll go through how you pick where to descend right now.
I guess, it's any possible move that starts at your starting game state.
Does that make-- great.
Before we move on to the algorithm itself, let's talk about what we store in each one of these nodes.
So now we've added these numbers.
And these numbers represent is that nk, as in the value of the right, is the number of games that have been played that involve a certain node.
So for example, if I look this node, that means that four games have been played that involve this node.
A game that has been played that involves the node just means that one of the states of the board at some point in the game was the state of the board that this represents.
For example, if I have a game that was played here, if I know that I've played this once, then that guarantees to me that I played this game once because this is a precursor state to this one.
Make sense?
Yeah
How can the two n's below that node not add up to a value of [INAUDIBLE] PROFESSOR 3: That will come when we start expanding our game.
But that's a great question.
And intuitively speaking, it should
You're saying you're storing data from past games about what we've-- PROFESSOR 3: Yes
--done before
If past game's outside of the script simulation?
PROFESSOR 3: No, no, no.
Past game's in the script simulation.
And then the other value is the number of wins associated with a certain node.
And these are going to be wins for player one, which is red in this case.
It would get confusing if we put both of them, but they're complementary.
So for example, three out of the four times that the red player visited this node, they won in that node.
And these are the two numbers that we're going to store.
And we're going to see why they're significant to store later.
So first, descending the key part of our algorithm that we're talking about.
And when descending, there are these two counterbalanced desires that we have.
The first of them is that we want to explore really deeply into our tree.
We want to think about, OK, if they do this then I'll do this.
And then, well, then I'll do that unless I want it to forth.
And we want to think through a long term strategy.
But at the same time, we don't want to get caught in that.
We want to make sure that we're not missing a really promising other movie that we weren't even considering because we were really going down this certain rabbit hole of the move that we had thought about before.
This is illustrated by the x case [INAUDIBLE] SMBC.
The SMBC comic about academia and how someone tells you that a lot of really great work has been done in an area, that means nothing about how promising the future will be.
It's all about expansion and exploration.
And the way that we're going to balance expansion and exploration in order to create our really nice asymmetric tree is the following formula.
And it's fine if that looks really confusing and messy.
But actually, it breaks down quite nicely into two parts.
This formula is known as the UCB.
You don't need to know why it's the Upper Confidence Bound.
Let's just talk about what's inside it.
So first of all, you have this term on the left.
And this term on the left is the extension term.
It's basically proportional to the likelihood that the expected number of times that you're going to win, given that you are in a certain node and that you were a certain player.
It's basically the quality of your state in some abstract level.
If we knew this perfectly, then we would be doing great because that's the thing we're looking for on some grand level, The expected likelihood of winning from a certain state.
On the other hand, you have this exploration term.
And you may not be able to read the font there.
But what this is basically saying is that it looks at the number of games that I have been played through, and it was the number of games that my parent has been played through.
And it tries to preserve those numbers at a certain ratio, at a log ratio.
And what that effectively means, is that the number of times that I have been-- if I have been visited relatively few times, and the denominator is small.
Whereas my parent has been visited many times, which means that my siblings have gotten much more attention, then the likelihood that I will be visited again actually increases.
So this is biased on the one hand, towards nodes that are really promising, and on the other hand, towards nodes that haven't been explored yet, where there's a gold mine and all you need to do is dig a little bit, potentially.
We don't actually have an analytical expression for this.
But we can approximate it because you can think that the expected value from a certain node is, roughly speaking, approximately the ratio of wins at that node to the ratio of times that that node has been visit at all.
Let's talk about actually applying this statement.
Because what the statement is going to give you, is it's going to give you some number for here and some number here, and some number for here, and so on.
When we start descending through the tree, we're going to start at the top node.
And then we're going to look at the three children of that node.
And we're going to compute this UCB value for each of these children and pick whichever one is the highest.
So just as a thought for a moment, what if we ignore this one?
And what if we're just computing the UCB of these two?
Does anyone have any intuition on whether the UCB would be higher for this node or for this node?
The left node.
PROFESSOR 3: The left node?
OK.
So why is that?
It has a win [INAUDIBLE] PROFESSOR 3: Yeah.
It has a win
And they both have a [INAUDIBLE]..
PROFESSOR 3: Exactly.
And so clearly, you think the exploration term is the same because you know it's not that one child has been loved less than the other, but the expansion term is going to be different.
And so it's definitely going to pick this one.
In this case, what we're going to say is actually that this is so much more promising than the others that it's actually going to pick this left node.
And so it's going to expand, and it's going to look down.
And then when it looks down, it's going to compare between these two.
And this time, remember, that this is a parent.
A parent want to minimize the number of wins that we have.
Which means that our opponent is going to want to pick the one that were less likely to win in and they're more likely to win in.
This is the idea of mini-max, minimizing how well my enemy does in this game.
Although again, the expiration term might counterbalance it a little bit because, technically, this has been explored more.
We're going to pick the one on the left again.
And we're going to get to that location that we got to originally.
Now when we're comparing between these two, between a node that has been visited once and a node that has never been visited, can anyone guess which one of these it is going to pick?
Yeah
Never has been visited.
PROFESSOR 3: Yeah, exactly.
Because this number is zero.
And so if the parent has ever been visited but the node hasn't, this is going to be infinite and it's going to have to pick the node that it has never seen before.
So that's how we descend through the tree.
Does anyone have any questions on that.
Really, it's totally fine.
We're going to be talking about this for a while.
Yeah
With the left node that has the four for n sub k, wouldn't that be three because there's two and one below?
PROFESSOR 3: No because of the way that we're going to be updating the tree.
Next, we'll talk about some [INAUDIBLE]
I like the concept.
But if it's a deterministic game, why couldn't it hold it's [INAUDIBLE] pretty strictly?
PROFESSOR 3: That's a great question.
That's really up to computer memory limits.
As I think that Leah mentioned, the number of stakes in the game of Go-- it's a 19 by 19 board, and you can play something at every state.
It's only like 2 to the-- PROFESSOR 2: [INAUDIBLE] PROFESSOR 3: What?
PROFESSOR 2: 250.
PROFESSOR 3: 250.
You could never explore the entire search tree
[INAUDIBLE] over the first few layers or are we going polite.
We try to do this real time where you could have done something offline.
PROFESSOR 3: It's definitely true.
If you know a state that you're going to arrive at ahead of time, then you can totally do that.
But in a game that's large enough that to do that for all the possible states would take that much more time and take that much more memory.
It doesn't end up making that much sense.
Also, something to point out here, is that for most of the games that we're talking about, simulating a run through of the game is really fast.
So if you think about it-- let's actually get to that in next piece.
But the point is that building up this many levels of a tree for a computer takes probably on the order of less than millisecond.
So doing this for a really, really huge tree, it's peanuts because their such simple operations.
But it won't get expensive when we start building up the tree to serious depths
But a game like Go, how many nodes would you have?
PROFESSOR 3: On each level, in the beginning, we have something on the order of 400 nodes.
And we have a depth of about, I think most games have up to 250 steps, or something like that
So just to build, if you go in there blank, without any nodes built, you have to in the computer, like you said, it hasn't visited a node, it has to go there before it descends further.
Basically, like breadth first.
PROFESSOR 3: It's sort of like breadth first but not quite.
There's an important distinction here, which is that it doesn't have to build up this or this node.
It doesn't have to build up all of the nodes at a certain level.
All it has to do is, if it branches down to a certain sub region, then can't descend in that sub region below one of its siblings without having at least looked once at all its siblings.
After it looks once it can do whatever it wants.
And the point is, that it doesn't mean the tree has to be kept at an even level.
All it means is that the tree, in order to descend on a specific part of the tree, it has to have at least visited direct neighbors once before.
Any more questions on this before-- Yeah
What's the advantage necessarily of having to visit every single?
PROFESSOR 3: The advantage of having to visit every single-- the way that I think of it, is that you don't want to be missing out on potentially being interested in some of the things and not others.
It comes back to the exploration versus expectation distinction.
We do want to descend into the region of the tree that is really valuable to us.
But at least have explored a little bit, at least maintaining some baseline, which really isn't that costly compared to the size of the tree.
400 moves is not that bad compared with 400 and 250
Are these simulations, they're just random simulations?
PROFESSOR 3: We're going to talk about that in a minute.
Any more questions before I move onto that?
Next step is expanding.
And this is very simple.
You just create a node and you set the two initial values.
And the initial values are the number of times it's been visited is zero, and then number of times that someone has won from there is zero
[INAUDIBLE] So the easy part is solving it.
PROFESSOR 3: Now, simulating.
Simulating is really hard.
You can imagine that if you get to a single node and you've never seen that node before, and you don't know what to do from this node onward, that if we knew how the game was going to play out, that is exactly what were searching for, and we would be done.
But we don't.
And in fact, we have no idea how to go about simulating a realistic game, and a game that will tell us something meaningful about the quality of a certain state.
And so, as you correctly guessed, we're going to do it randomly.
We're going to be at a certain state.
And then from that state, we're just going to pick random nodes for each of the players until the game ends.
And if we, as player one, win then we're going to add one.
Then we're going to say delta equals plus one.
And if we don't win, or if we tie or lose, then we're going to call it a zero.
You can in this graph, we're descending randomly and not thinking about it.
And it turns out that this is actually great because it's really, really computationally efficient.
If you have a board, even if it has 400 open squares, populating it by a bunch of random moves doesn't take you very long, on the order of not that many machine can
That's why does you don't score-- if you go down a tree randomly, you already have a simulation.
So the node's going to get to someplace.
But you don't store it because it would lose the randomness?
PROFESSOR 3: You're totally right, actually, in this case.
I've thought through this, and I can't come up with a reason why you wouldn't store it, that's it's temporary values that you find all the way down the tree.
But they don't in most of the literature [INAUDIBLE] But you're totally right about that.
Does everyone understand that distinction?
The fact that we only hold onto the result here and don't theoretically make nodes for every place down in the tree just because we could, just because we've seen them before.
We don't, and it doesn't really matter in this case.
But it's theoretically a slight speed up that you could do
But you reduce that question to generalities?
PROFESSOR 3: Yeah, a little bit.
So we can look at an example of simulating out a running game.
We get some intuition for why a random game would be correlated with how good your board position is.
For example, here we have a Detecto game.
Circle is going to move next.
But as hopefully you can see, because you have played Detecto before, this is not a particularly promising board for x.
Because no matter what circle does, if x is an intelligent player x can win right now.
It has two different options for winning.
And so, if you simulated this forward randomly, what you'll get is that 2/3 of the time, x will in fact win, even if the players aren't really thinking of it ahead of time.
Yeah
Then why not do n simulations at a node instead of just a single simulation?
PROFESSOR 3: You totally can do that.
That's in fact, something that make sense to do and that some people do.
Although what you'll find somewhat soon, is that considering that we're going down the tree, and that sometimes soon we're going to explore all of its children, there's a good question of why you end simulations now when you could just descend through the tree n times and thereby do n simulations by going through the thing and also building out the children?
This case is-- yeah
This gives more importance to why you do randomness.
Because if you're doing random simulations you would ignore the possibility of the best one.
When you first ran a simulation here was that o wins.
If I ignore this node-- PROFESSOR 3: Absolutely.
Which is why it matters that we do this so many times that we drown out all the noise that is associated with playing a game out randomly.
Let's talk about that.
If there's a lot of distance between where we are right now and our end result-- For example, in this game, if I were to tell you how good is this board position, if you are one of those people who played out every game of Detecto, you'll know that this is great if you want it to be [INAUDIBLE] Anyway, the point is, that is not easy to do if you are doing random simulations from where you start.
The correlation between your friend's board state and the quality of that state actually drops precipitously.
And this for me is one of the hardest parts to study about Monte Carlo Tree Search.
Although, as Nick will explain to you, it actually works quite well.
And one of the reasons that it works quite well in practice for more complicated applications is they do away with the assumption of random simulation.
Because even the random simulations does allow you to explore all the states, if you have some idea of where a reasonable quality approach would be, then using that, as long as it's not that much more expensive computationally, can help you with your simulation.
Right now we're still talking about total randomness.
How are people doing with that idea?
Now we're going to update the tree with the results of our simulation.
So given that we had some result lambda, we're going to try to get up the parents.
And for each parent we're going to add that the game has been played there once, and that the result of that simulation gets added if it was a one.
So for example, if there was a win in this game, than this becomes one, one because now it's won once and it's been visited once.
And these two get incremented by one, and these two get incremented by one.
That in itself comprises a complete iteration, the complete single iteration of running Monte Carlo Tree Search, which means that now we can keep doing this over and over again, building up the tree and slowly making it deeper, and making it deeper in selective areas.
And having these numbers increase and increase.
And be more and more proportional to the actual expected value of the quality of the state, until-- does anyone have any questions about this idea?-- until we terminate.
And we have to come up with a way to terminate it.
Now again, we said we're going to pick what the best child is going to be, what the best immediate move from the start state is going to be.
That's the move that were actually going to play.
And so, how do we determine what the best is?
Well, the trivial solution is just the highest expected win given k. What that, in our case, is going to be is the ratio of number of times that I've win from a given early state to the number of times that I visited.
However, this doesn't actually work as well as we might hope.
Let's suppose the following scenario, which is that you have the Detecto game like this.
And you have been exploring the tree for a while.
And you're really mostly looking at these two nodes.
One of these nodes, if you think it through, this node is quite promising and you've been exploring it for a while.
There is a winning strategy from this node.
It's that circle goes here, and then x goes here, and then circle loses because x has two options to win.
However, if you explore this a bunch of times, and for some reason, due to the randomness, this is at 11 out of 20.
Whereas this state, which is inherently inferior, is at three out of five because of a bunch of randomness and because it hasn't been explored as much.
And if we had looked at this one as exhaustively we had at this one, that you probably would actually say that this state is actually better.
And so, you can create an alternative criteria, which is that it's the highest expected win value of one of the children.
But also, that value has to be the node that has been most visited so that they aren't explored by different amounts.
What this sacrifice is however, is that this means that we can't terminate on demand.
This is not always going to be true, and therefore, we're going to have to let the algorithm run until that's true for some start state, which means that maybe is not a criteria that we want to apply even though we know that it would be wise to do so.
Are there any questions about how we pick the terminating guide?
That was the whole thing.
And now we're going to do it lots and lots of times until you guys are sick of Monte Carlo Tree Search.
So this our tree.
It's more or less what we've had before.
The first thing we're going to do is we're going to look at the top.
And then we're going to pick one of these children.
Now let's say that we looked at this, and it turns out that the one on the left is really valuable.
I think it's the one.
Nope, yeah.
Never mind.
It's wrong.
The one on the left has been explored a whole bunch of times.
Remember, this term starts becoming larger than the ones that haven't been visited as much.
And so we're going to descend from this one.
And now we're going to descend, and we have these two options.
Given what you know, would you expect that this is going to pick is going to be the one on the right or the one on the left?
[INAUDIBLE] PROFESSOR 3: On the right because it's never been visited before.
And so, this term is going to explode.
And so, we're going to build a node there.
And then we're going to simulate a game.
And the result is a win, which is bad for this player.
That means that he probably didn't want to make that move.
And so we're going to propagate that value up.
And we're going to start the algorithm again.
And it's going to compare between these three.
And now it's going to pick the one on the left.
Now that it picked the one on the left, it going to compare between these two states.
Which of the two is going to have a higher expansion factor?
The left
Don't you invert it, though, because this is the opponent.
PROFESSOR 3: Exactly.
Because two out of three is actually better.
Because it's one out of three for the opponent that's currently making the move.
So the one on the left is going to have a higher expansion factor, and the one on the right is going to have a higher exploration factor.
Does that make sense for people?
It's OK if it doesn't.
So we're actually going to pick the one on the right because the other one was is doing three and has lots of it's mother's love than that one's.
Anyone else need a drink?
We're going to expand that node.
It doesn't matter.
They are both equally likely to be expanded.
We're going to simulate forward, and it's going to be one.
Which means that that was probably a wise countermove.
Yeah
So when it's the opponent's turn versus your turn, the exploration factor is the same but we complement the expansion factor, right?
PROFESSOR 3: Yes.
So the key here being that this takes in both the state that you're talking about and the player that you're talking about
But regardless of the player, the exploration factor will always be like this is.
PROFESSOR 3: Because it's only the number of visits it's.
It has nothing to do with results of exploration
If you win and you have the plus one, double plus one, and you've propagated out, but I'm wondering-- so if the opponent wins do you also propagate out the win increment itself?
If the opponent's winning, wouldn't you want to [INAUDIBLE] node here?
PROFESSOR 3: If the opponent wins then what you do is you propagate up a zero.
Which means that wk is not incremented, but nk is.
Have we seen a zero yet?
There's one soon.
But the idea is that rather than subtract or anything, all you do is propagate up the result of the game, which in this case is zero.
Which means that all of those states seems to become more valuable to the blue and less valuable to the red.
Because these numbers are lower than the other ones were
OK.
PROFESSOR 3: So we propagate this up and this becomes better.
What we've done here is we've figured out a theoretical countermove to blue moving here.
That's how you should think about this whole tree.
It's really a lot like the way the humans think about these things.
If I do this, then what if they do this?
Well, then I'll do this.
And I see that I'm successful when I do that.
We're going to look again at the top.
And we're going to pick the one on the left because it's really promising.
Five out of six is a good number.
And we're going to look at both sides.
And which one is blue going to pick now?
Well, it's going to pick the one that it's going to be more successful in, which is two out of three.
I realize that this is actually not the kind of thing where I could necessarily ask people because I'm the one who's decided which node to stop.
Then we go down here.
And there's an equal likelihood of picking either of those nodes.
And so we're going to pick one at random.
So that's going to be the left one.
And we're going to create an empty node.
Then we're going to play it out.
And it was a success for blue, which is amazing because what this means now is that suddenly, in this tree of this really good move that red could make the blue wasn't find a response to, suddenly there's hope because we're going to propagate this back.
And that means that blue actually has a response move to that sequence of red's moves.
And so it's going to propagate up.
And this state's going to be more promising to blue and less promising of red.
That region of the tree that we had dug into is a little less promising.
We're going to look back up.
And this time, instead, we're going to evaluate the thing that is both promising from the expansion factor, and also promising because we haven't looked at it very much [INAUDIBLE] exploration factor.
We're going to pick between these two.
Which one is going to be picked here?
[INAUDIBLE] PROFESSOR 3: Because the exploration factor is the same but the expansion factor is higher for the one on the left.
And it's going to show us a node.
And the result is going to be a win for a red, which means that red has found a good countermove to the thing that was previously promising for blue.
And we propagate it back up.
And finally, we're going to pick the one furthest on the right.
Because even though it's terrible for red, and even though it's never won when it's tried it, it has to obey his idea of the exploration mode to find out whether maybe there isn't something possible there.
So it explores, and it goes down, and it has to pick the one on the right.
And so it does.
And it plays this game out.
And it's a loss, again.
Which goes to show you, that blue has found yet another superior move to this really bad move of red, where probably this move of red, if this is a game of chess, is like putting my queen directly in front of the opponent's row of pawns, and I just leave it there.
There's nothing good that's ever going to come of it but we have to explore it just to find out whether there isn't some magical way that I should protect.
And as you can see, we've built up this tree over and over and over again.
And it's starting to look asymmetric.
And we're starting to see that there's really this disparity between exploring the regions that are crossing this tree and exploring the regions that are not and that don't really matter to us very much.
And that this is exactly what we wanted from Monte Carlo trees.
That was why we started the whole endeavor in the first place.
The next thing I'm going to talk about is the pros and cons.
But before I do that, does anyone have any more questions about the algorithm?
Yeah
It's still not clear how we're getting nodes with different denominators-- [INAUDIBLE] PROFESSOR 3: The reason for that is because of the way that we're simulating through.
We're actually not holding onto to the results of the simulation as we're going farther down the tree than the lowest node we expand.
For example, when you simulate from here, you're going to propagate that value here and here, and so on.
But then when we expand below, even if in the course of this guy's simulation it happened to go through one of the states that we expanded below, it will not have incremented the values of that state because we weren't keeping track of it.
Theoretically, if we were to keep track of all of the simulations that we have in fact run, the numbers beneath these things would be higher
If you've already run a simulation from that-- if you've already run a simulation from that red node when you first built it, and then when you created those two ones, each of those have [INAUDIBLE] PROFESSOR 3: OK.
I see
So would the denominator always be one more than the sum of the children?
PROFESSOR 3: Yeah, in [INAUDIBLE] Yeah
I understand how you built that.
Is there a rule of thumb, like it's time to choose a move?
And it seems like you have very low numbers here to make a [INAUDIBLE] Is there a rule of thumb on giving games like it's 2 to the 4 or 2 to the 350, whatever it is.
What kind of numbers do you need for that first row before you [INAUDIBLE]?
PROFESSOR 3: What we'll get to soon is that isn't one.
That's one of the problem with MCTS.
But in terms of which of the moves you will choose, there are actually variants of MCTS that suggest that you more selectively age or insert new children based on something more than just the blind look right now.
In terms of, if I'm here and it's creating my next children as the equivalent, then there are some intelligent guesses that you can make in terms of which one you should score first.
Although it doesn't particularly matter
I'm just saying computational time being what it is, you might say, OK, if this is the timeline of this game I can expect to do a million simulations, which will give me if there's 400 nodes, I'm going to have so much use.
In other words, is that enough time to say that I can play through a game?
I couldn't play through a game with 400 options if I've gotten five out of seven [INAUDIBLE] three out of four [INAUDIBLE] PROFESSOR 3: Absolutely.
And I would say that so far as I know, that's something that's basically very high experimentally.
They don't have good balance on it.
[INAUDIBLE] So let's get on the first comment because that is a computer element.
So why should you use this algorithm?
Even though we've seen tremendous breakthroughs in this algorithm, and you're going to have to ignore everything that I tell you and remember that this does actually work quite well in certain scenarios.
Should we use it or not?
The pros are that it actually does the thing that we want it to do.
It grows the tree asymmetrically.
It means that we do not have to explore.
And it doesn't explode exponentially with the number of moves that we're looking into the future.
And that it selectively grows the tree towards the areas that are most promising.
The other huge benefit, if you'll notice from what we've just talked through, is that it never relies on anything other than the strict rules of the game.
What that means is that the only weight of the game that's factored in is that the game is what tells us what the next moves we can take from a given state are, and whether a given state is a victory or a defeat.
And that's kind of amazing because we had no external heuristic information about this game.
Which means that if I took a completely new game that someone had just invented, and I plugged MCTS into it, MCTS would be a slightly or someone competitive player for this game, which is a powerful idea.
It leads to our next two pros.
The first of which is that it's very easy to adapt to new games that it hasn't seen before, or even that people haven't seen before.
This is clearly valuable.
But the other nice thing about it is that even though heuristics are not required to make MCTS work [INAUDIBLE],, it can work [INAUDIBLE].
There are a number of [?
advanced ?]
places in the algorithm that you can actually incorporate heuristics into.
Nick is going to talk about how AlphaGo uses this very heavily.
AlphaGo is not vanilla Go.
It has a lot of external information that's built into the way that it works.
But MCTS is a framework-- you can imagine your heuristics you can apply in the simulation, there are heuristics you can apply in the UCB in the way that we choose the next node.
There are places that it can fit in.
And this services as a nice infrastructure to do so.
The other benefit is that it's an on demand algorithm, which is particularly valuable when you're under some sort of time pressure, when you're competing against someone that's a mathematician, or when something is about to explode and you have to make a decision on which reactor to shut down.
And lastly-- or not lastly, actually, it's complete, which is really nice because you know that if you run this game for long enough it's going to start looking at a lot like a BFS tree.
No, it's actually going to start looking like an alpha-beta tree, if it is what it is converted to.
It's a nice property to have.
Although, this property does slightly get compromised if you remove the red in this idea, and if only simulate these [INAUDIBLE].. Yeah
You made an interesting comment when you said, oh, it looks like -beta tree.
So it looked like a mini-max tree.
But have they also incorporated notions of pruning in the MCTS, which would make it look like an -beta tree?
PROFESSOR 3: Sorry, you're completely right.
It does look like a mini-max tree.
I think I've seen variants where they do pruning, but I haven't looked into it as much.
But I would imagine that they would converge to whatever you know pruning a certain tree [INAUDIBLE]
But people have explored incorporating pruning into MCTS?
PROFESSOR 3: I think so.
I can't say [INAUDIBLE] And then lastly, it's really parallelizable.
You'll notice, none of the regions of this tree, other than the original choice, ever have to interact with each other.
So if you have 200 processors and you decide, OK, I'm going to break up this tree in the first 200 decisions and then have each one of those flesh out one of those decisions, that actually means that they can all combine information right at the end and make a decision [INAUDIBLE],, which is a really nice, powerful principle as you [INAUDIBLE]..
It does have its fair share of problems.
The first problem being that it does breakdown under extreme tree depth.
The main reason for this being that as you increase more moves between you and the end of the game, you're increasing the probability-- you are decreasing the correlation between your game state and whether a random playoff would suggest that you're in a good position or a bad position.
The same goes for branching factors.
One of the things that people sometimes talk about it as if MCTS AI's cannot play first-person shooters because the distance between the number of things that you can do at every given moment, and what would be a successful approach in the long term after meeting many, many, many moves that each have many branching factors, is that never begins to explore the size of the search tree.
For the most part, it's not really coming up with a long term policy.
It's really thinking about what are the next sequence of moves that I should [INAUDIBLE].
Another problem is that it requires simulation to be very easy and very repeatable.
So for example, if we wanted to tell our AI, how do I take over Ontario?
There's not a particularly good way that you can simulate taking over Ontario?
If you try it once, you're not going to have an opportunity to try it again, at least with the same set of configurations.
And actually, one of the things that we really took advantage of, if that random simulation happens really quickly, on the order of microseconds.
On other hand, the bigger your computational resources that you have access to, the better the algorithm works.
That means that I can't run it off my Mac particularly well.
It would be like large games.
It relies on this tenuous assumption of random play be weakly correlated with the quality of our game state.
And this is one of the first assumptions that is going to be thrown out the window for a lot of the more advanced MCTS approaches, which are going to have more intelligent play outs.
But those are going to lose some of the generality that we had before.
Something that goes off of that is that MCTS is a framework.
But in order to actually make it effective for a lot of games it does require a lot of tuning, in the sense that there are a whole bunch of variants.
And that you need to be able to implement whatever flavor is best suited for you.
Which means that it's not quite as nice and black boxy as we would want it to be as far as give it the rules and have it magically come up with a strategy [INAUDIBLE].. And then lastly, as you mentioned, there is not a great amount of literature right now about the properties of MCTS and its convergence, and what the actual proportion of time to quality of your solution is.
This is true of all modern machine learning things, is that there is certainly a lot more work that could be done.
But right now, that's a gap in terms of using this for a simulation that's supposed to be reliable.
Anyone have any questions on the Pros and Cons?
Before we jump dive into applications, let's talk through a few examples of what games could be solved and could not be solved by MCTS.
Do you guys think that checkers is a game that could be solved by MCTS?
Yes.
PROFESSOR 3: It's completely deterministic.
It's two-player.
It satisfies all of the criteria that we've laid out before.
Checkers is definitely a game that can and has been solved by MCTS, although not solved to the extent that you can defeat the thing that actually has the solution [INAUDIBLE].
How about "Settlers of Catan?"
This one's a little bit trickier.
Do you guys think that MCTS is likely to be able to play "Settlers of Catan?"
If not, let's throw out reason why or why not it would be [INAUDIBLE].
Yeah
No because there's randomness.
PROFESSOR 3: So yes, that is absolutely the criticism.
And that's why we can't apply it vanilla.
I put this on here as a trick question, though, because it turns out that MCTS is robust to randomness.
That you can actually play-- and I realize that's just me and we do.
[LAUGHTER] You can actually play through games.
If you think about the simulation, the simulation is actually applicable even if the game is not deterministic because it does give you a sense of the quality of your position.
And the MCTS-based AI to play "Settlers" is, I think, at least 49% competitive with the best AI to play, at least in the autonomous non-scale space.
So it does work.
Let's talk about the war operations plan response.
Who here has seen the movie "War Games?"
OK. Well, it should be more of you.
The idea of "War Games" is that one of the core characters in this world is this computer that has been put in charge of the national defense strategy with respect to Russia.
And that it needs to think through the possible future scenarios and decide whether it's going to launch the nukes or not.
Do you think that WOPR can be MCTS-based?
No.
PROFESSOR 3: No
It could, it just wouldn't be very good.
PROFESSOR 3: Absolutely.
Once you fire the nukes you're not going to get another chance.
So you can't particularly simulate through what the possible scenarios are going to be like.
Yeah
So what if you had-- I agree you can't simulate it in the real world.
But what if you had a really good model and you just simulated based on that model?
PROFESSOR 3: In that case, it probably depends on the quality of your model.
If you have a good model for how World War III is going to [INAUDIBLE].
[LAUGHTER]
It is the case that the military does have simulators and they do war games in simulation.
PROFESSOR 3: Yes, that's true.
They could certainly try it and run MCTS if they wanted.
And that's what happened in the movie.
[INTERPOSING VOICES]
And there you're putting your money in the simulation not in the--
It's like having an MCTS play SOCOM or something like that.
PROFESSOR 3: Yeah.
It's definitely about putting money into the simulation and you get really good simulation.
If you have a really good simulations then you [INAUDIBLE] to play WOPR.
Yeah
Back to "Settlers" for a second.
I'm curious if there's a way for the whole player training resources thing, or would it have to be only purely like using the ports.
PROFESSOR 3: That's a good question.
I haven't looked closely at whether they do that or not.
If it's playing a two-player game, then I would imagine that they wouldn't because you don't really trade in to play a game.
But if they weren't, I bet that you can incorporate it with
Is it limited to two-player games?
PROFESSOR 3: No, not at all.
In fact, there are lots of purchases that do only one-player games, where you think of what's the best movie that you can make
I know.
But I mean, couldn't MCTS handle three- or four-player games?
PROFESSOR 3: Yeah, it absolutely could.
I'm not sure how they computed their head-to-head.
That might be completely flat cursors.
I'm not even sure how the settlers interact.
Yeah
A quick question.
So at first you know if I reduce the chess board to only 4 by 4 or 5 by 5, and I run MCTS versus the traditional algorithm that AlphaGo offered as a tree.
Do you think MCTS will prefer theory and perform this computational requirement.
PROFESSOR 3: The thing about the way that Deep Blue is, which is the AI that ended the Kasparov thing, a bunch of his chess grand master, is that it has a tremendous amount of heuristic information.
There's a lot of external stuff that's incorporated into the system that makes it able to explore the best paths.
What I would say is that knoledgesless MCTS based on randomness, would take a very long computational time to even become competitive with those kinds of algorithms, and probably feasibly never would.
What if you incorporated heuristic information, I think that there's a bunch of hope in terms of getting MCTS to start performing better.
And you can look at what next I'm going to talk about, AlphaGo.
It takes inspiration for how we go about incorporating these new circuits
So only the circuit you [INAUDIBLE] PROFESSOR 3: It definitely seems like if you have a really good heuristic model for what good states in the game are, that if it's a smaller search space, that some other models could perform better.
Although, I'm probably going to eat my foot here because this is going to be on OCW some massive amount, massive chess playing algorithms.
Eat my shoe not my foot.
[LAUGHTER] One last game.
Does anyone know what this game is?
"Total War?"
PROFESSOR 3: Yes.
Nice.
This is "Rome, Total War II."
It's a simulator for this tremendous real time strategy game, where you play, I think, the Roman Empire.
And you're controlling armies and huge infrastructure systems that move and conquer states and continents, and meet in the field, and manage resources, and do all of these incredible diplomacy feats.
And so do you think that this game can be solved by MCTS?
Yes
Yes.
PROFESSOR 3: Lets say no.
But I guess I put it on here.
So that's good on you.
The way that the AI in "Rome, Total War II" is built is that it's built on an MCTS structure.
And it in fact does do resource allocation and a lot of its political maneuvers based on Monte Carlo Tree Search moves.
There are a bunch of reasons that they explain in the game for why they do this, or in papers released about the game.
But one of the nice ones is that it's random, which means that you're never going to play against the same kind of AI twice because every time the set of decisions that it's going to think about is completely different
I have a quick question.
PROFESSOR 3: Yeah
So if I want to model any game with MCTS, does it have to be that the actions in playing a game has to be able to discretize.
PROFESSOR 3: Yes.
So far as I know, I haven't seen many continuous variants in MCTS.
And so, I think that it is about choosing these reactions, which on it's most narrow level does actually bring it down to here.
I think one of the reasons that this is nice is that there are so many different decisions that could be made that MCTS is really the only approach that could even begin to handle the massive branching factor that's associated with the game Rome, Total War.
Yeah
This is also the consequence of this year you get the play off when this game comes.
PROFESSOR 3: That's interesting.
That's probably totally it.
That's cool.
That's everything about how the algorithm actually works.
I'm going to pass it off to Nick, and he's going to talk to us about some actual limitations for this game [INAUDIBLE].
PROFESSOR 3: So as you have said, I'm going to start diving into some applications here.
And not only applications but also some modifications or augmentations of MCTS.
It should hopefully clarify some of the side questions you all have been having on slight tweaks to MCTS.
Now let's get started.
Wait for it.
Now let's get started.
[LAUGHTER] Part III, applications.
First thing we're going to look at is an MCTS-based "Mario" controller.
And "Mario" might seem like some weird thing to test AI on, but there actually is a "Super Mario Bros" AI benchmark, which it used to test a lot of AI on how well they could play this platform.
In case any of you don't know what "Super Mario Bros" is, this is a screenshot.
Basically, you control this one character.
It's a single-player game.
The ultimate goal is to reach this flag at the end.
But along the way there's enemies, there's some bonus shrooms you can get.
If you break open some boxes you might get coins, things like that.
But first, let's just highlight some of the modifications that need to be made, or some of the differences between vanilla MCTS and an MCTS that's going to be able to work for "Mario."
First thing is that it's single-player.
The second is, we use a slightly different simulation strategy than the initial just vanilla simulation.
And someone actually hinted at doing more than one simulation because you, you're watching us to n simulations, I think.
We'll touch on that.
Then this also introduces what I would consider to be domain knowledge.
Then finally, there's a 50 to 40 millisecond computation time.
And that has to do with the frames per second of the game.
So you would think that "Mario" is a continuous game, but if we discretize time into these chunks, then we can use MTTS.
Now let's just think about how we could possibly formulate this problem.
Can anyone think of what each of these nodes would be if we're playing "Super Mario?"
Jump.
PROFESSOR 3: Sorry?
Jump.
It would be like, first node you're going to jump.
PROFESSOR 3: That might be a way to formulate it.
But I think that could get--
Oh, it's not your control at inputs [INAUDIBLE]..
PROFESSOR 3: Right.
So the node itself isn't going to be an action
Equal frames.
PROFESSOR 3: Yeah, basically.
So it's going to be the state of a game, what we'll call a state.
So it's basically just a screen grab.
And it take it, in this case, it's a 15 by 19 grid screen grab of the game.
And it will have information about-- it knows Mario's position, it knows the enemy's position, position of the blocks, et cetera.
And then, as Yo was saying, in MCTS we have values associated with our nodes.
And so it will also have a value.
But we'll get into the value in the next slide because I can't really fit it all in here.
With that being said for our node, that being the state of the game, what makes sense for the edge?
Does anyone know?
How do we transition from one state to another state?
Jump.
PROFESSOR 3: Yeah, exactly.
So this is where the jump and all the action have been played.
So the actions that you take-- I didn't list all the actions.
You can also have a jump left, jump right, all those things.
But basically, the actions are what takes you from state to state.
So I just drew out what a node might look like if you used the jump action.
You might have Mario go up in the sky.
Are there questions?
Does it just run the rest of it?
Because that little thing's moving as they move on?
PROFESSOR 3: Well, it's not moving in this moment in time.
We're discretizing time right now
But I'm saying, if your action is jump, just you would have 1,000 nodes because if you did plan out where that thing's moving, left or right, then it could be-- PROFESSOR 3: Yeah, right.
So in each state we have the enemy position.
And we know the speed and direction.
And so we know when we go from this node to one time step later, we'll know where the enemy's moving.
Any other questions?
Moving on.
Sorry.
Let me just preface this part real quick.
So in our other simulations, at the end of the simulation we would get either a one or a zero, if we'd won tic-tac-toe or we lost tic-tac-toe.
But that won't really work too well here because there's a lot of other factors that go into play when you're playing "Mario."
Also, if you're doing a simulation, more than likely, you're going to end up hitting an enemy and dying or falling into a gap and dying.
So a lot of these simulations might all return zero.
And that is, you can't really distinguish between them.
So this is why I say, this version of MCTS introduces what I would consider to be domain knowledge.
Basically, they're assigning scores to potential things that could happened along the way.
And this is basically telling the AI that collecting a flower is a little bit better than collecting a mushroom.
It's telling it that getting hurt is bad.
Right off the bat, all these things in the score are giving the AI some domain knowledge about "Super Mario Bros," that it's helping it calculate the simulation results.
As it says here, it's just doing a multi-objective weighted sum of all these things.
Throughout the simulation it's just adding up your score.
And then that's the score that is going to be propagated.
Are there questions about the score?
You said that it adds up all these guys and it propagates it over.
Is it possible to just propagate the multi-part sum [INAUDIBLE] as opposed to propagating one value that you create?
Are you essentially propagating all-- what's this?-- 15 values upwards at every node, or are you propagating one value-- PROFESSOR 3: Well, it's one value.
It's the collective--
Then you make them add it together and you got each one of them a sub factor.
PROFESSOR 3: Then also, just one thing to note here, is distance, you get 0.1.
And these are all parameters that have been tuned.
In the initial version, distance was, I think, a reward of five, but probably realized that that made Mario skip past a lot of coins and things.
And so he tweaked the score for that.
And also, time left is two.
So there's some weight there.
You want to get to the very end of the game
If you're pushing up this score, it's no longer a win over losses.
So it's not w over n. What is it affecting?
PROFESSOR 3: You can just use the score
The score is the-- PROFESSOR 3: Yeah.
In MCTS you have this idea of when you're propagating your q value, you could have that to be zero, one
It's like the sum of all the scores and the nodes below over the number of games you win.
PROFESSOR 3: So basically, what you would be getting when you divide by the number of simulations is your average score at that node

When you have killsByFire and [INAUDIBLE] like that, if you have a positive value, then isn't it good to be killed by fire, or something like that?
PROFESSOR 3: This is killing an enemy by fire.
Like Mario could collect a certain flower or mushroom?
I think flower, then you have a fire breath and you [INAUDIBLE]
So that's Mario's status if Mario never dies?
PROFESSOR 3: No.
Mario's status is-- I believe, Mario's status is the fact that you could upgrade Mario by collecting [INAUDIBLE] mushroom from a fire Mario.
So that gives you a lot of points.
Because if you become fire Mario, then you're more likely to not die by running into enemies because you have fire-spewing--
You said they spent a lot of time tuning these parameters.
Isn't it generally, though, just an optimization framework if that's some formula?
So they tuned the parameters just to make behave the way that we think is nice.
But if you change the values, they'll do the right thing for that equation.
PROFESSOR 3: Yeah
OK.
PROFESSOR 3: Yes.
But they were tuning this to make it play how they wanted
[INAUDIBLE] can't just be a reflection of [INAUDIBLE] PROFESSOR 3: That's a strategy.
If you choose that, I don't see why not.
That might affect certain things.
Obviously, you can change these to whatever you want.
It'll slightly tweak which simulations as to working better, in terms of changing which nodes you end up choosing [INAUDIBLE].
So we move on.
So we know about scoring simulations.
Now we're going to look at exactly the simulation type that's used to play this MCTS controller.
So the regular version that Yo talked about is just choosing a random node at each level in your simulation.
But there are some other strategies.
And someone brought one up.
The first is, look at best of n. So in this one, you choose three random nodes at each level, except that you stick with the best of those three.
Choose three random nodes, stick with this one.
Go to the next one.
You would choose n random three, take the best one, and then go to the next level.
You are able to do that in this game because of the way the scoring works, you don't have to get to the end of the game for your score.
You actually could collect a coin along the way.
If this is jump, and then it gets to be a coin versus moving left and right.
That doesn't give you any points.
Then this is the node that would give you the highest scores, so I would choose that one, et cetera.
And then the final one, which is the one that is actually used for this controller, is multi-simulation.
This was brought up by him.
I don't know your name.
Sorry.
But basically, you run multiple random simulations from your node.
And then you propagate up whichever of those simulations give you the highest value.
And the reason to do multiple simulations is to attempt to increase the accuracy of your simulations.
If you just do one simulation you might just get really lucky.
But if you do three then you can take the highest value use that as your value.
Since the whole point of this is to try make moves that get you the highest values, then that will make your random simulation value more accurate.
Are there questions about multi-simulation?
So what do you think about the simulation [INAUDIBLE] how many [INAUDIBLE] PROFESSOR 3: So there's a trade off here.
The more simulations you do the more accurate-- the more representative your simulation will be at the end of the game.
You could run two to the whatever simulations to try to get every single possible action and then take the max of that.
And that would give you the maximum value.
That would be ideal.
But obviously, that takes more time.
So there's a trade off between computation time and the number of simulations you run.
And that's just something that they probably just played around with
Do you use [INAUDIBLE] have to finish the decision losing a couple of minutes or 10 minutes or they're going to take your [INAUDIBLE] away.
PROFESSOR 3: In this competition there is different computation time budgets that you get.
And I believe the reason for the different computation time budgets is the frame per second of the game.
I told you all about the setup, we went over, the scoring, the nodes, what the advantages are, what the simulation strategy is used.
So you probably want to see it in action.
So this is always a risky move trying to get video to play
It's actually in the back up.
Hit Escape.
PROFESSOR 3: OK. Got it
And now, I guess, we-- PROFESSOR 3: And drag it over again?
Yeah.
PROFESSOR 3: Running this full screen
Hit the [INAUDIBLE] PROFESSOR 3: [INAUDIBLE] All right.
Here's this MCTS-based "Mario" playing controller.
You can see he's actually wrecking, so doing some serious damage here.
But those lines that you see, the reason they're different colors it's not showing different players, or anything like that.
It's just using different colors so you can see the different layers of this tree search.
You can see he actually went backwards there.
And that's because in a simulation, when one of the backward ones landed on an enemy-- and in fact gets you points from our scoring system versus if you had just gone forward you would have gotten some distance points but not-- also, he is just [INAUDIBLE] The simulation is quickly being able to figure out that he can jump on all his enemies.
So he's just wrecking all these guys.
Getting lots of points here, collecting the coin, et cetera.
You get the idea.
It's pretty awesome to watch.
There's that flower we were talking about.
So now he's actually a fire-spewing Mario demon.
He's doing some serious damage with that.
Stepping on missiles.
I didn't even know you could step on the missiles.
All right.
You could watch this for a while.
But we'll exit now.
It looks super promising in this video.
I don't know how close max stuff
Just click on back [INAUDIBLE] PROFESSOR 3: There it is.
OK.
The demo looks really cool, looks really promising.
Let's take a look at the charts here because we all want some quantitative stuff.
This is the chart.
The score is on the y-axis.
The bottom is computation budget, which is something that you were talking about.
I just want to highlight to make this a little more visually appealing here.
All of these things that I highlighted, it's labelled as UCT.
That's Upper Confidence Bound Tree.
Remember, Yo talked about upper confidence bounds.
That's essentially what's used in that TTS for guiding your tree search.
So these are all the methods.
But then UCT multi, which is this purple square, that's saying it's using MCTS but it's doing the multiple simulations.
And you can see this multi plus care is also in the top.
Both these use the multi-simulation technique.
And then the plus car is they added an extra scoring mechanism for carries.
I believe that's probably like carrying a shell.
That made it do better.
Then these ones that aren't highlighted are using plain Astar, and then a refined version of Astar.
With increasing time, the do increase scores, but they're even worse than just your UCT with just random simulation, no multi-simulations.
We're running low on time, which is not ideal.
But another thing that I want to point out is down at the bottom here, these are the multi-simulations.
They have the lowest maximal search depth, which at first would seem like, what?
I have the lowest search depth but my score is the most?
But that comes into play when you were saying about the trade off between the simulations and the amount time it takes.
So because I'm doing multiple simulations, I'm taking more time at each node.
But that's giving me a more accurate value assessment.
So that let's me choose my actions more carefully, or with more information.
And so that's what's able to give me this better scores.
That's all "Mario."
So we're going to moving onto AlphaGo.
Are there any questions about "Mario" before I go to AlphaGo?
Yeah
What's the table [INAUDIBLE] inference?
PROFESSOR 3: That's a good question.
I have a feeling it's because if you're doing best of n, that's really heavily relying on your scoring metrics.
Let's say at one step if I jump and collect a coin versus if I go left or right and play, I'll get more points if I get that coin.
But maybe, a missile is going to hit me in the face if I do that.
It gets rid of some of the-- it's forcing you to do certain moves
Is the A* heuristically using the same value, the same value that you're getting by your simulation?
PROFESSOR 3: Yeah.
I'm not exactly sure what the Astar heuristic is.
The whole reason that A* is difficult is because coming up with heuristics for these types of games are.
But this is not his version of Astar.
I believe this is the Astar that was used by-- I forget the name of the guy-- but he won the AI competition a couple of years ago.
I'm going to try to move onto AlphaGo.
Does someone have how many minutes I have left?
Four.
PROFESSOR 3: OK. We're going to power through.
Here's AlphaGo.
Hopefully, you all know the rules.
Just in case, I'll just go through a quick-- 19 by 19.
You alternate black stones and white stones.
You collect enemy stones by completely surrounding them.
You can surround a single stone.
groups of stones.
And your score is your territory plus the number of captive pieces.
So your territory is just the area that you're surrounding, and then you just add the stones you've collected.
The rules aren't super important.
The main emphasis is there's very few rules so you would think it's really simple.
But the complexity of the game is quite extreme.
At each turn you have about 250 options that you can play.
Each Go game lasts about 150 turns.
So that gives you a total of 10 to the 761 games, approximately.
And to put that in comparison, here's chess.
You can read those numbers.
Chess is also pretty complex.
But there's 35 options for turns.
Deep Blue.
I think you were talking about building out the whole tree.
So Deep Blue would build out the tree for six levels.
And then use this hard core chess master inputted heuristic evaluation that it used to find the best move.
Except with Go, you have 250 options, which already is adding a lot more complexity.
So that strategy won't work quite as nicely.
What do we do?
We use a modified version of MCTS.
Well, it's not what we do.
That's what Google's DeepMind team did with Go.
They combined neural networks with MCTS.
Coincidentally, we learned about neural networks last class.
Probably not a coincidence.
PROFESSOR 3: It's not a coincidence.
PROFESSOR 3: The we ordered two policy networks in the AlphaGo, and one value network.
And another big coincidence here, the two policy networks are actually CNN's, which we learned specifically about last class, convolutional neural nets.
And the reason for that is the input to the policy neural networks is an image of the game.
And remember, convolutional neural nets work really well with images.
What it outputs, though, is a probability distribution over the legal moves.
And the idea is, that if a move has a higher probability it will be a more promising move for you to take.
But another key point is that it's not deterministic.
It's not telling you to take this move.
It's just assigning a higher probability to this move.
And this network was generated by doing supervised learning on 30 million positions from human expert games.
Apparently, there's a giant database of Go expert games.
So that came in handy.
And there were two different networks trained.
One of them was a slow policy, the other was a fast policy.
The slow was able to predict an expert move with 57% accuracy, which to me was mind blowing.
Using this neural network, 57% of the time it could pin where the expert would place his move.
That took 3,000 microseconds.
Versus the fast policy, which suffered a bit in the accuracy, but it's 1,500 times faster.
And we'll see where they used each of these different policies later on.
But it could predict the expert move with 57% accuracy.
The other Go team was, that's not our goal.
We don't want to predict an expert move.
We want to predict a winning move.
And so to do that, they took their policy network, and then they would use reinforcement learning.
That's where you play the network against iterations of itself in order to hone in a better policy that's geared towards winning moves.
Then they tested this against Pachi, which uses-- for the camera, I have no idea if that's how you pronounce Pachi.
It might be Patchey.
I'm not sure.
But there's 100,000 MCTS simulations at each turn.
So this is purely MCTS.
If it were playing just the AlphaGo policy network, the policy network won 85% of the game.
So without any sort of trained search or anything involved, it won 85%, which is pretty great.
And that suggests that maybe intuition wins over long reflections in Go.
And interestingly, if you talk to expert Go players and you ask them why they did a certain move, they'll just say, It felt good, or I had a hunch in this.
That's indicative there.
Hopefully, I'm not going overtime.
Sorry.
Those are the two policy networks.
There's also a value network.
What the value network does is it takes in a board, and they'll give you a value, like how good is this board?
They'll give you a win probability number.
So 77%, it would say, 77% of the time you should win from the board.
That's similar to the evaluation that comes from Deep Blue.
But rather than a Go master coming in and telling you, well, if these are connected in this way, and down here we have this certain thing then here's the score we should expect, in chess, they had chess masters, like if the knight is here and the queen is here, all these specific things.
This was actually learned from the reinforcement learning that was happening when the policy networks were playing each other.
The value network was learning about those positions during that time.
And the predictions get better towards the end of the game, which I think Yo mentioned in his talk.
So how do you combine all these into MCTS?
The slow policy network, if you remember, is slower but should give us stronger moves.
It is used to guide our tree search in order to help us decide which nodes to expand next.
When we expand that node to get the value, the value of the state is the simulation, like before, like normal MCTS, except it's not a completely random simulation.
We use our fast policy network to give us a more educated simulation here.
But we're using a fast one, obviously, to save some computation time.
It's giving us probably a more indicative random simulation of what's going to actually happen.
And then we also combine that with our value network output.
So we run our value network on this node, as well.
And we add that to our simulation value and we propagate it.
Interestingly, the AlphaGo team tested out just using the fast policy simulation value and scrapping the value network.
And they also just used the value network and scrapped the simulation value.
And those both performed worse than if it had these.
And another added interesting point here, is that these two factors in our value have about the same weight.
They were both about equally important.
I think I'll get into that later.
But first--
Can I just ask a quick question?
PROFESSOR 3: Yeah
So when you said the policy network is used, is that used when you're navigating to the tree to get to a leaf, or is policy network being used to do the simulation once you're at the leaf, or both?
PROFESSOR 3: The slow policy is done for this part.
Then the fast policy is used for the simulation.
Because the slow policy does take 1,500 faster than-- or the slow takes 1,500 times longer than the fast policy.
You don't want to use that in your simulations.
That would just take way too long.
It's basically just a way of making it so our simulation isn't completely random.
It has some educated moves.
Why use policy and value network synergy?
Why can't we just use the policy network?
Why can't we just use the value network?
If we have the value network alone, we'll actually-- here's a side point.
Remember, the value network learned from the policy network.
And then also, later on, the policy network is improved by our values.
They work hand-in-hand.
But if we had the value network alone, when we're deciding on it the next move, we're going to have to evaluate every single move, which would take forever.
And so, what the policy network does is project the best move with a probably distribution.
And it narrows our search space.
And then, if we had the policy network alone, we'd be unable to compare nodes in different parts of our tree.
The policy network is able to tell us a distribution over which move we should take from a certain node.
But then, if I ask it if I'm in a better position here than in some other place, it won't know.
That's where the value network comes in.
It will give us an estimated number of the value assigned and open an evaluation of that node.
And then these values are later used to direct our tree searches based on updating the policy once it realizes, oh, I thought this would be a good path but the value is this, so update all that.
Then why do we combine neural networks with MCTS?
Remember, the policy network alone played against Pachi, which was purely MCTS, and it did pretty well.
So how does MCTS improve our policy network?
Remember, MCTS did win 15% of those games.
So already, that makes you think there's something there that maybe the policy network is missing.
Also, the policy network is just a prediction.
So by using this tree structure, we're able to use these Monte Carlo rollouts to adjust our policy to move towards nodes that are actually evaluated to be good.
And then, how do neural networks improve MCTS?
The point should probably be clear by now.
We're able to more intelligently lead our tree exploration.
Our simulations are more reflective of actual games.
And the value network and our simulation value are complementary, which I've mentioned before.
And just to highlight that, basically, the value network is going to give us a value that is reflective as if we've played the slow policy the whole time.
And the simulation is if we used a faster policy.
So they are complementary.
And I know I'm over time.
So I just wanted to skim through the stats real quick.
Distributed AlphaGo won 77% of the games against regular AlphaGo.
So it's the only thing that beat regular AlphaGo.
And then distributed AlphaGo won 100% of the games against all these.
In a rematch against Pachi, now that we've added MCTS to our policy network and we have our value network, we slaughtered Pachi 100%.
Then we decided to see how we fare against humans.
And by we, I mean not me, I mean Google.
And they won 4 to 1.
And Lee Sedol rating was 3,520.
Now AlphaGo's rating is estimated to be about 3,586.
So you're like, whoo, we beat the best dude.
Except we didn't because there's another dude who has an even higher score, apparently, 3,621.
This should be the last part.
Here's this timeline.
Basically, tic-tac-toe, checkers were conquered in '50.
About 40 years later, we conquered checkers, chess.
Then we scroll down to 2015, is when AlphaGo was able to beat Fan Hui, who was a two-dan player, which is considered lower down in the tier of professional Go.
But then, Lee Sedol was a nine-dan player.
And he was able to beat him literally last month
So good job.
PROFESSOR 3: We're done.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PRESENTER 1: The basic concepts of reachability-- I'm talking about how we can represent reach sets in continuous spaces.
[INAUDIBLE] is going to take it over and talk about some applications in robust motion planning.
And then Gerard is going to talk about how we compute reach sets and focus on these two, on flow tubes and funnels.
So reachability is a task of figuring out what stage a dynamical system can possibly reach or be at at a certain time.
So for example, if we have these two systems, we have a finite state machine on this reach system and a continuous system.
And these are the starting points.
And these are our axes.
We can move east or north here.
And we can move east or north with these velocities.
[INAUDIBLE] So after one second or one time step, we can be in either of these two states.
Or we can be anywhere in this area-- and after two seconds, after three seconds, and so on.
OK?
So this is what reachability does.
And the motivation for this-- one of the applications for reachability is for verification.
Verification is just the task of making sure that we are never going to reach any bad states.
So for instance, let's say we have a microwave.
A bad state is the microwave being open and on.
We want to make sure that these bad states are never reachable.
So what we do is we compute the reachable sets, and we check for intersection with the bad states to see if the microwave is on and open.
Or like over here, let's say we have these obstacles.
Let's say we have a wall or somewhere else over here.
We want to-- this is bad, right?
Because we can reach this state, and it's the same as a wall.
Any questions so far?
Feel free to interrupt me any time.
Another motivation for reachability is for robust motion planning, which [INAUDIBLE] talk more about.
And I have a video of this.
So what we can do with reachability is stuff like this.
So this is done using flow tubes, which is reach sets, reachability.
Gerard is going to talk more this example.
So now before we talk more about these cool examples, let's just formalize concepts.
Let's define some things.
So let's go back to the same system that we had earlier, this state machine, with a finite set of states, X; a finite set of inputs, U, which in this case are east and north; and a transition function.
And let's say this is just [INAUDIBLE] to keep things simple.
And we have a set of initial states, X0, which I didn't label as red.
So the reach set is defined as the set of states that we can possibly be in at time t. So more formally, there exists a sequence of t control inputs that will take us from this initial state to the corresponding reach set.
So let's say, for example, you have bad state, it's in the reach set at time 3 because there exists a sequence of three inputs.
In this case this is one such sequence.
There is two more that will take us from the initial state to that.
OK?
And then the reachable set, the reach set reachable set is defined as the union of all reach sets for time less than or equal to t. So the reachable set of time 0 is just the initial set of states.
And time 1, we have the unit and so on.
So the reachable set is what we use to test for intersection with bad states because that's all the states that we can possibly reach from time 0 to time something.
OK?
Now as some of you may have figured out by now, there's a simple way for finding segmenting so we can compute reach sets.
So for example, let's say we wanted the reach set of time 2.
One way-- the easiest way we can do this is to break this up into time steps, into single time steps.
So instead of computing it straight for time 2, let's first compute it for time 1.
And now we can use these two states as the set of initial states and compute the reach set for one more time step.
And that gave us this reachable-- the reach set.
OK?
Any questions?
OK, and the reason why these works is because reach sets are semi-groups.
And semi-groups satisfy this equality.
So we can not just break it up into single time sets.
We can break into any two times, s and t. OK, so all of this was for finite state machines.
But for continuous systems, things get a little more complicated.
It's a lot of the same concepts, but we need to handle a very important issue, and that is how to represent reach sets, or just set, regions in space for continuous systems.
Because in a finite state machine, we can just enumerate the list of states in the reach set or reachable set.
In a continuous system, we need some other way.
We can't just say it's every point for which there exists a sequence of control inputs, because there can be infinitely many points.
So what we do, we have two types of representations that we like.
One is convex polytopes.
And that's just a generalization of a convex polygon into n space, into n dimensions.
And we have two basic ways that we can represent these complex polygons.
One is with these hull vertices, like this, AB to F. [INAUDIBLE] And then the convex polytope is defined as the convex hull.
And in case someone doesn't remember what a convex hull is, let's say we have just a list of points.
The convex hull, imagine have a rubber band and you are going to stretch it out, and you let it go.
And what that's going to do is this.
And the convex hull is just the whole area inside this-- the rubber band.
OK?
And then another way we can represent convex polytope is with a set of inequalities, so for example, here we have four lines.
Here's the equation for each line.
So let's say this red line, we want anything below the red line.
This blue line, we want anything to the right, so anything greater than, and so on.
So this area is the intersection of all these inequalities.
OK?
And each of these representations has an advantage over the other.
Vertices are good to test for emptiness because when our set of vertices is empty, then our convex polytope is going to be empty.
And this inequalities is very useful for testing membership, because for any points, we can just plug it in to every inequality.
And if it's true for every inequality, then it is in the whole group.
And convexity, I'll just explain it briefly.
Convexity means that any two points inside our region-- our region is not non-convex is for the line connecting these two points.
All the points in that line are going to be in the region.
So here you can see that this is convex.
And here, this red area is outside the region, so this other region is non-convex.
And convexity is going to be important for the algorithms later on.
Gerard will talk more about them.
OK?
Other than convex polytopes, we can also use ellipsoids to represent reach sets.
So ellipsoids are just the extension of ellipses to n spaces, n dimensions.
And a convex polytope is defined by this.
x is just all the points inside the polytopes, inside the ellipse.
v is the center.
And A just tells us about this, how deform the ellipsis.
Any questions?
OK. And one of the reasons why we like convex polytopes, why we like ellipsis, ellipsoids, is because of their enclosure under linear operators.
So let's say we have P that is convex polytope or an ellipsoid.
Then, if we have a matrix A, then A times P will be defined that way.
It is still going to be a convex polytope or an ellipsoid.
So if we have something like this, then this can maybe getting formed, but it's still going to be convex, a convex polytope.
And the reason why this is nice is that, if we have a linear system defined this way or this way, and then if we start with a-- basically, if we started with a convex set of states, then our reach sets are always going to be convex because we're just multiplying by A.
And this is very nice to represent-- this tells us that our representation is going to be good because we can just use convex polytopes, ellipsoids, and stick with those.
And then just a technical-- OK, so even though the reach sets are going to be convex, the reachable set might not be convex.
But it will always be a union of convex polytopes or ellipsoids.
For example, let's say, in the previous example, we started here.
Then we need, after one time set, we were here.
So these are the two things.
And then our reachable set is the union of both of them.
This is clearly not convex.
But it is a union of convex polytopes.
So we can represent everything within it.
OK?
Any questions before I move on to [INAUDIBLE]??
OK, thank you.
PRESENTER 2: OK, so now that have understood what reachability is and how [INAUDIBLE] represents, and before we dwell into more theoretical aspects of [INAUDIBLE].
Here are some cool applications of reachability.
So there are many applications of reachability to robots are like complex systems [INAUDIBLE] systems.
One of the applications that has been talked about before is the robust motion planning.
The early idea is to more of a glider or an airplane through lots of obstacles, even in the presence of [INAUDIBLE] these and [INAUDIBLE].
They're already crashing into obstacles, even when they are [INAUDIBLE].
[INAUDIBLE] applications of reachability to control complex systems like by parallel working robots.
Here you can see that we are controlling [INAUDIBLE] simple soccer ball with something.
And more importantly, reachability is used for very fine safety properties, for safety reasons, like aircraft collision avoidance.
So in those [INAUDIBLE] we're going [INAUDIBLE],, robust motion planning.
If you're interested in other things, there are papers and there are [INAUDIBLE].
So the goal of motion planning is to find a set of control inputs that take you from the start state to the goal state.
So in this example-- without colliding with any of the obstacles.
So here, this is one possible path from the start state to the goal state.
So it just looks nice.
[INAUDIBLE] collective things of the obstacles.
But if you get the controls to this cart and then give it a real robot, you might find it going-- doing something like this.
So this is because of the path that you-- algorithm that you use for planning this path is-- that's not taking into account any of the uncertain areas in the environment or [INAUDIBLE]..
So there are different kinds of uncertainties.
One of them is environmental disturbances, like there is wind here.
So that can move you over off your planned path.
And there can be modeling errors.
The model that you use to represent your complex system might just be an approximate.
So the real world is different.
And there are state uncertainty.
So the things that you used for measuring your position or velocity are also approximate, so there are uncertainties.
[INAUDIBLE] there can be some randomness in the initial conditions.
You are thinking of starting the robot at 0,0, but in practice, you might actually be starting at [INAUDIBLE] 0, [INAUDIBLE]..
So there is randomness in initial conditions.
The goal of robust motion planning is to have some guarantees with some confidence that the system will reach the goal set without getting on with the obstacles even in presence of these uncertainties.
So this is the problem.
So let's look at some of the solutions that come up for solving robust motion [INAUDIBLE]..
So this is a timeline.
So there are three different systems.
And all of them have a goal concept where they use some representation to represent the reach sets, the set of possible state that the system be.
And then instead of searching for a single trajectory to a state, they search for these representations so that each final self-serviced representations that can go from the start state to the goal state, you won't hit any of the obstacles.
So earlier Bradley and Zhao and Frazzoli came up with this concept called the flow tubes.
It's [INAUDIBLE] presenting makes sense.
And we will know all about the [INAUDIBLE] of the [INAUDIBLE]..
So one approach that we use is that we have a set of initial states and we have a set of goal states.
So flow tube is a combination of all the possible paths that will get you from any point in the initial state and point in goal state.
So this is kind of representing all possible [INAUDIBLE]..
So and then you search for these flow tubes to be a path to get your [INAUDIBLE].. And [INAUDIBLE] constraints so that you can also reason about complex systems and revise spatial and temporal coordinations like the walking robot that you have seen before.
An [INAUDIBLE] about for using funnels for motion planning.
So funnels is also a similar concept to produce.
It's kind of like a [INAUDIBLE] region around your path.
So that if you are inside a funnel, you're guaranteed to be inside the funnel.
So in this picture, we're going to [INAUDIBLE] motion planning using funnels.
Again, there are references for the other people's [INAUDIBLE]..
So instead, you'll be using funnels [INAUDIBLE] for every path, you compute some region around the path that is a stable region.
And that's called a funnel.
[INAUDIBLE] your funnels [INAUDIBLE],, but for this part of the lecture, we are assuming we don't have a hundred of funnels, and that you'll see how to use these funnels to do motion planning.
So the previous example is here.
The [INAUDIBLE] funnel [INAUDIBLE] flied by.
You see that it collides with the trees and [INAUDIBLE].. Then [INAUDIBLE] like these funnels are [INAUDIBLE] uncertainty in the world that you have.
If [INAUDIBLE] everything with uncertainties, [INAUDIBLE] and then you compare funnels for that.
Any questions so far?
Can you please say again definition of funnel?
PRESENTER 2: So it's kind of something that [INAUDIBLE] around [INAUDIBLE] around-- like you have a [INAUDIBLE] around there.
So I think Gerard with explain more about it.
So the [INAUDIBLE] with funnels is that you are inside the funnel at the point.
When you [INAUDIBLE].
It's like [INAUDIBLE]
Yeah, you have an [INAUDIBLE]..
So a funnel is basically-- it's kind of like a reach set around a certain point too.
So the way that you do it is you have some [INAUDIBLE] trajectory.
And then at each node point in that trajectory, you compute some kind of reachable set at that node point.
And so you blow it up, and it becomes some kind of ellipsoid.
And as long as you're within that ellipsoid, no matter what disturbance you get, you're going to go back towards the nominal path.
And then so, when you join all of those ellipsoids over the trajectory, then you get a certain kind of funnel.
And as long as you're inside that funnel, you're going to stay on that funnel.
And you're going to be on the path you want to be at with some disturbance in this [INAUDIBLE].
So that the great thing that there would be an example of a funnel.
As long as you're in the funnel, you're going to stay in the funnel, basically
So is this considered the control, or is it only the dynamics?
PRESENTER 1: It's the dynamics with the control.
So it's the flow tube system.
PRESENTER 2: Yeah, so, you'll learn more of those [INAUDIBLE]
Let me just quickly add.
So a funnel is type of flow tube, right?
I mean, just different people either use the same terminology, use different terminology.
Each of the applications of flow tubes has some invariant, which is different.
And the particular invariant on this one is that if you are you in the funnel and you apply the LQR component, [INAUDIBLE] then you're guaranteed to stay within that tube
But I thought the flow tube is actually like getting smaller and smaller in the funnel [INAUDIBLE]
Not necessarily.
In general, a flow tube is simply a bundle of trajectories, which capture some common features.
And then maybe that they move to a limit of cycle, and maybe they stabilize to a point, and maybe that the move to a goal, maybe that they always maintain.
They're stable within a tube.
In a funnel they focus particularly on stability to disturbance and then staying within the tube.
PRESENTER 2: OK, so the [INAUDIBLE] example is kind of useful to combine the reachability motion planning.
So here is another situation where getting to the goal from the start [INAUDIBLE] and there are a couple of sort of trees lined up.
But there is a path that goes in between the [INAUDIBLE]..
This path looks kind of risky because it's pretty close to the two trees.
But there is another path we can take which travel around the trees.
And so if you don't need the [INAUDIBLE] safe.
But maybe that's [INAUDIBLE] too.
So that's why we do the combined reachability with motion planning.
So if you combine that, you might find that the first path, if you compute a flow tube for the first path, [INAUDIBLE] it does not hit any of the trees.
And it's indeed safe.
But it might happen that the second path is more susceptible to the disturbances in [INAUDIBLE]..
So that's why [INAUDIBLE] and also [INAUDIBLE] combined motion planning.
Because what's intuitive is not actual.
So far it's all about [INAUDIBLE] planning.
So the [INAUDIBLE] everything about the environment, like where the obstacles are.
So that's not always true.
[INAUDIBLE] So you don't always know all the obstacles beforehand.
You can't [INAUDIBLE] as you [INAUDIBLE]..
So one problem with doing online planning, there is that you can not do any expensive computations during runtime.
And all of these funnels can be places that are very expensive.
[INAUDIBLE] solving an optimization problem.
It takes a couple of hours to finish them.
So you can not compute funnels on the fly.
So the idea here is that you compute the funnels offline and do all possible funnels from the start state to the end state, and then use this library to [INAUDIBLE]..
So you have the same [INAUDIBLE] going from the start state [INAUDIBLE] the goal state in the archives.
So these are the set of possible paths from start state to goal state.
And then from each possible trajectory, you compute the funnel.
And the idea of online planning is to now find the sequence of these funnels that you can compose that takes you from the start state to end state without hitting any of the obstacles.
So an important component here is finding a sequential composition of funnels [INAUDIBLE]
So why plan-- you're doing a forward search with funnels.
Why use funnels instead of something like RIT?
PRESENTER 2: I actually don't know what RITs are
Can I guess at the answer?
[INAUDIBLE] thing
Why use funnels instead of RITs
Well, or why use funnels as opposed to anything else
Well, I guess RITs, you have like the classical example, where you want to go through a very-- well, if you want to go through a very narrow region.
Well, first of all, I guess RITs are going to have a hard time finding the narrow region because you'd need like a point to go through that.
And then, RITs by themselves don't really account for errors, maybe some extension.
But funnels, you can be sure that you're always going to stay inside the funnel.
So this if for [INAUDIBLE].
This is to make sure that we don't crash into anything.
Does that answer your question?
Do you want to answer it?
Yeah, I think what he just said, it's-- the point of the funnels is that they provide a control policy.
The actual reference trajectory that you have on the left is the motion planning.
But with the funnels, you get a control policy.
Because when you start executing a motion plan there's always going to be a disturbance.
So you want a control policy that keeps you on that trajectory.
And there's all sorts of control policies.
You could use simple EID controllers.
They work in some cases.
But funnels provide better guarantees for the power plants.
PRESENTER 2: Yeah, so the idea is that you have trajectory [INAUDIBLE] funnels [INAUDIBLE].
You're using this funnels compliance.
You compose them to find your path to those presets that you wanted to go in.
So you can not always compose funnels.
Some [INAUDIBLE] positions that are legal, and some of those are not legal.
So [INAUDIBLE] example, a time going from left to right.
So you have a situation where there is a funnel.
You're trying to compose a bigger funnel with a smaller funnel.
And you have another situation where you're going to compose a smaller funnel with a bigger funnel.
So what if I would have said, [INAUDIBLE]..
So which of these combinations do you think is a legal composition?
[INAUDIBLE] Legal or illegal, I mean that which of these [INAUDIBLE] do not receive the [INAUDIBLE]?
The bottom [INAUDIBLE].. PRESENTER 2: The bottom one is--
--is actually marked because-- with the top one, if you're at the very top [INAUDIBLE].. PRESENTER 2: Yeah, that's true.
So the first one is like illegal.
And the second one is a legal composition.
So why is that?
So if you are inside this funnel, what this guarantees that you will stay inside the funnel.
But it could happen that you can go through the funnel and make your outside [INAUDIBLE] funnel.
And after this step, you can go anywhere.
So you can [INAUDIBLE].
But if you're inside the second case, after you're through the first part, you're still set at the second part then we can decide [INAUDIBLE].
So now let's actually look at those online planning algorithms.
So this the case of your planning [INAUDIBLE].. First you can [INAUDIBLE] with quantum's chronicles.
It makes our analysis easier.
So this is the algorithm that's going through each of the steps one-by-one.
So this is online planning.
You will always one information beforehand.
[INAUDIBLE] And you get to know more about it as we go.
But you still have the initial planned funnel sequence that takes you from the start state to goal state.
And then you check for any new obstacles information when you go out from the sensor.
So this is the region that you can sense some obstacles.
And you also get the current state of your robot.
This is also something concern [INAUDIBLE].. And you check if your current funnel path collides with any of the obstacles you have seen so far.
And in this case, the answer is no.
So then you apply the control corresponding to this funnel for this [INAUDIBLE] location and time.
That take your one [INAUDIBLE] problem.
Then you put Goto and then do this again and again.
So [INAUDIBLE] update from certain information.
And one obstacle is here.
You can differentiate the other one.
When we check at the Replan collides with any of the obstacles-- in this case, yes.
So since it collides with the obstacles, you need to [INAUDIBLE] funnels.
So how do you delete funnels?
You have all this library funnels.
So you go to each of them, and then you find one set of funnels that has-- you find a set of funnels that starts with your current location and [INAUDIBLE] of the obstacles.
So this was a point, first of all, of a new funnel sequence.
And then, again, you apply controls corresponding to the new sequence and doing it two dimensional [INAUDIBLE],, In the position, if it collides, and again collides, so you redirect [INAUDIBLE] apply the control for that funnel.
And get a full solution.
And then it collides, so you have to replan that part.
Then you're OK.
So now you reached the goal.
So those are the planning plan using funnel libraries.
And questions about it?
Can you say more about the connotation of the funnel libraries?
So they generate a set of funnels, which covered the complete set space?
PRESENTER 2: Yeah, so that's like-- there's every possible [INAUDIBLE] end point.
And so then you can make [INAUDIBLE] or least, have it for every initial [INAUDIBLE] you can compose all these [INAUDIBLE]..
So if you're doing it for every initial [INAUDIBLE]..
So you might have a funnel that exactly starts at your initial position.
If there is a funnel that already started somewhere, where it goes through the funnel-- goes through your initial funnel, you can use that.
So it's a truncation of your funnel.
[INAUDIBLE] Any other questions?
Why can't you use-- if it really is an online problem that you're not going to run out of a funnel, that you won't reach a dead end, will it be adaptable enough to make a U-turn at all times, either that there's come latency that would perfectly time it.
And the robustness, that you'll always be able to find a path, even if we have guarantees that you'll always stay within a funnel.
PRESENTER 2: So are you asking what the situation, like the funnel, if you're going [INAUDIBLE] backtrack?
Yeah, just because these obstacles come out of nowhere, seemingly, that you don't have a guaranteed that the funnel is safe through the entire duration of our execution.
PRESENTER 2: Yeah, so that's a valid point.
[INAUDIBLE] you can use the adaptive [INAUDIBLE] backtrack [INAUDIBLE] combined with the [INAUDIBLE]
Even if the robots can't actually backtrack?
PRESENTER 2: If the robots can what?
Come back, or--
I mean, I guess, it's just like a UAB, it couldn't do that.
I'm just saying, what guarantees do you need to have about the obstacles , since it's an online setup, right?
PRESENTER 2: Maybe you'd have some initial information about some kind of obstacles.
You can find a [INAUDIBLE] trajectory-- you can find a trajectory that goes from the start sequence.
It's like [INAUDIBLE]
Well, I'm just going to say.
If you're in some kind of glider that can't necessarily-- like a quart of stop, go backwards, stuff like that, depending on how funnels you can plan backwards.
But if you're in some kind of glider going straight at a wall, there's no guarantee.
You're just screwed up.
I mean, even humans, they make errors.
There's situations that humans can't dynamically get out of.
So if your system is dynamically not going to be able to make a sharp enough turn or go straight into something, no matter how smart your planning algorithm is, no matter if you're like, there's a wall right there.
Stop.
If you can't stop, and you just go straight at it.
There's obviously situations where you can pretty much [INAUDIBLE] algorithm [INAUDIBLE].. You're just going to crash.
Hopefully you're in a situation where you're not [INAUDIBLE]..
It's also based on how far away your sets are received.
Your sensor can only see 5 meters ahead, and you're going 5 meters per second, [INAUDIBLE].. You're going to have make perfect decisions.
But if your sensor has perfect information about the environment, then what seems like the initial best path may not actually be once you know the full [INAUDIBLE]..
So it depends on the [INAUDIBLE] sensors.
PRESENTER 2: Any other questions?
OK, so finally I have some demos showing simulations of [INAUDIBLE].
Let's see how [INAUDIBLE].
So the [INAUDIBLE] applications.
So we have seen how to use funnels to do robust motion planning.
We also have the offline planning and come to do online planning using a sequence funnels and the funnel libraries.
I will talk more about computing these funnels and flow tubes and other kind of reach sets.
PRESENTER 3: Yeah, so I'm going to talk about how to actually compute flow tubes and funnels generally.
I'm going to give an example of what each one is used for.
So we had the question before, how are flow tubes and funnels different?
Flow tubes are-- so easiest way is they're both some kind of way to guarantee that you're going to-- the robustness guarantees.
So say you are here.
You want to be here.
They're both going to-- in most planning algorithms, you've got some kind of path from here to here.
But say there's some disturbance [INAUDIBLE] down here or something.
You don't know if you're going to be able to reach there or not.
What a flow tube is is it's some kind of set of trajectories that will basically go from some initial state, which are here, to some goal states over here.
And basically, [INAUDIBLE] if you're within this flow tube, and you get pushed out on a different trajectory, you know how to get from here to here anyway.
So you don't really care if you're pushed, as long as you're still within here.
A funnel-- let's see.
So a funnel is, say you compute some nominal trajectory around here.
Using the system theory and stability theory, you know that, with your control inputs, you're going to stay at each of these points.
And you're going to stay within here within these circles.
So if you get pushed out here, you know you're going to be able to get back onto your path when you're going to try to converge back to your nominal path.
So you have some sort of funnel.
Yes?
I have a question.
How complete is the funnel?
It sort of related to what Brian was saying.
PRESENTER 3: Completeness as in like these edges are--
Well, if you have a high dimensional state space, does the funnel cover it completely?
There's a set of funnels.
PRESENTER 3: A set of funnels--
So the question wasn't about if the individual funnel, but in the particular [INAUDIBLE]..
When he generates a set of funnels, does the set of funnels completely cover the space?
PRESENTER 3: It depends how much time you want to put into it, generally.
If you only compute two funnels, one of them goes over here and one of them goes here, then you're not you're not going to be covered in these states.
So it depends on how much computation you want to do offline previously.
So you get a set of funnels that's here, another one that's right here, another one that's here.
And you have your big library of funnels that cover the whole space
I think I skimmed that paper.
And, yeah, I think it is probabilistically [INAUDIBLE].. PRESENTER 3: OK, yeah.
All right, yeah, but I mean, as far as computing where you're going to go, if you compute that path of it, you should be good.
But yes, you're right.
The probabilistically-- So here is what I explained.
Normally you have your optimal trajectory.
But if you get pushed off of it, you may not get back to the goal state.
But if you have a set of trajectories that go from some initial states to your goal state, and you stay within that, you know how to get to your goal state, even if you get pushed.
Again, so some ways to approximate flow tubes, you can use some kind of polytope, which is as Gabriel explained previously, as long as it's a convex polytope, you can take that cross-sectional area and you can propagate it down here for the tube.
You can also use ellipsoids or rectangles as cross-sections.
And important point is to know that they're intercross-sectional approximations.
So as you see here, you may-- this would be your actually flow tube.
But your approximation has to be an inter one, because that guarantees that you're with a flow tube.
But it means you may also miss some of the outer trajectories , which also, if you want to compute a more thorough approximation area you get better flow tube representation.
OK, so robust planning, you plan a trajectory from here to here, and that should be good.
But then you get disturbed.
You're still within the trajectory, so you're still good.
But some work done by Professor Williams and Hoffman shows that-- so what do you do when you're actually outside of the flow?
Now what you can do is temporal planning.
So we're using the algorithm that we previously learned in class.
You can take your goal state and move the timeline back to a different time.
So here, this flow tube is, if you're here in a certain amount of time, can you get to here?
And if you get outside of it, then you can't do that anymore.
But you can just use our algorithm and move back the goal time.
And then when you move the goal time, the whole funnel shifts because now you're going to be over here.
And you don't care about getting here at this time.
You can here at this time now.
So then you move over.
And now where you, you get back into it because you're now inside the flow tube again.
OK, so one example is a humanoid footstep planning.
So this take a complex systems.
You have a bunch of different parts of the system.
You have your center of mass dynamics.
You have your leg dynamics, foot dynamics.
So what you're trying to do is plan your footsteps forward and through time.
So you have your original plan of where you want your footsteps to be for different parts.
So there's discrete things that happen in the system, as we've previously seen in a lecture.
So one thing that can happen is your left foot hits the ground, and then your right foot lifts off.
You're left foot lifts off-- or your right foot hits the ground, and then your left foot lifts off.
So those are different things that can happen.
So you have certain temporal constraints on the system that you want to [INAUDIBLE] by.
And so as you start planning, you have flow tubes to get you from one step to another step.
And that guarantees that you're going to stay within those.
Even if you get disturbed and get pushed, your feet can still get from one step to the other.
But then also, with the thing that we saw in the previous slide, you can move the timeline back and forth, as long as you're withing your temporal constraints that you had.
So those are flow tubes for the feet.
And that would guarantee that your feet go where they want to.
But you can decompose the system to also have flow tubes for the center of mass.
So what the center of mass does is to be broken down into separate flow tubes.
So does that make sense?
So this is the humanoid footstep planning that we saw previously.
And now you're going to understand how the feet are planned to go to each of the blocks.
[INAUDIBLE] go video
I have a quick question.
You said that you have a separate flow tube for the center of mass.
But aren't they really just the same flow tube, like a high dimensional-- PRESENTER 3: Yes, but that's-- when you decompose this and then you [INAUDIBLE] flow tube rather than a full system high dimensionality
Is it hard to recouple them?
How do you know that they're consistent [INAUDIBLE]??
PRESENTER 3: Well, it still adheres to the dynamics of the system.
So you can couple how you want, how you want it to move.
And you want to move any basic constraints.
But eventually, as you said, when you move your feet, your center of mass is going to move.
When you step forward, you're center of mass is going to move.
[INAUDIBLE] analysis of it
Can I make a comment about that?
Yeah, there's a technique called feedback linearization that can be used to decompose a complex non-linear dynamics assuming to the set of linear ones.
And that allows you to treat the center of mass as a [INAUDIBLE] separately from a control in the flow tube computations and while guaranteeing that [INAUDIBLE].. PRESENTER 3: So this is work done by Professor Williams experiment, which is actually [INAUDIBLE].
This is the robot doing the same kind of thing on Mars.
This is walking on stones and planning its footsteps through a pretty complicated environment.
The stones are not exactly in the path where you would normally want to walk.
But moving them around, as long as you're within the flow tube, you'll get to wherever [INAUDIBLE] Awesome.
So that was how to compute flow tubes and-- oh, and another way you can also compute flow tubes is by learning.
So you can have some kind of example, move through trajectories from one state to the other.
And you can estimate a flow tube based on where the trajectories are, as long as the flow tube encompasses all the possible trajectories
Is the dynamics always linear?
Because I though if your polytope, and it would be like kind of, is it a complex proposal when you would use a non-linear dynamic, then you move a little bit.
Then you would move a little bit, not being the convex shape then.
PRESENTER 3: I mean, you can plan non-linear systems too.
But as far as I'm concerned, [INAUDIBLE]
So, yeah, the polytopes generally require linearization with the system.
And there's a limit to the validity of the-- the range of the linearization.
And that's concluded in-- that's one of the things that limits the size of the polytopes
But it is the case-- I mean, you've just got to understand.
The thing that you do for non-linear systems, which is a piece-- if you're making a piece-wise linear, but you are-- you are linearizing them at the w step.
And there's other techniques that people have developed with flow tubes where you do require the input system to be linear.
Right here it can be non-linear.
But... it's an approximation.
PRESENTER 3: OK, so funnels, the goal is basically to find a region that guarantees you safety on a bounded uncertainty.
So there are regions of finite time variance throughout all time during that trajectory that you've defined.
And so in practice, there's a trade-off between computation time and guarantees.
So if your trajectory, if you compute more nodes, then you're going to be able to fill in these little gaps between each of the ellipsoid.
Whereas, if you lessen them, it's going to take less time to compute each ellipsoid.
But it's also going to not be as guaranteed.
And especially important when planning is that, if you're planning in some kind of point cloud, then some of the points may be right here, where one ellipsoid doesn't intersect the other.
And you might think that's a safe path, when actually it's not at all.
Yeah, so an example is this little glider, which is given in the paper by Anirudha and Russ.
And so what you start doing is you create your system, your function.
And this is dependent on your state, time, and w, which is some kind of bounded uncertainty.
And you choose the bounded uncertainty, which means that you choose some kind of reasonable estimate of how much can happen to your system that you didn't account for.
So you start it.
Your velocity can be different from what you want.
Say your nominal velocity is 10 meters per second.
You can have some kind of drag, or you can have some inconsistency in your motors.
And so what you expect is that there's a plus or minus 0.5 meters per second to your nominal.
So when you start running your equations, this is what you expect your robot to do.
And then that can vary forward and backwards depending on what your actual was.
But then you add some kind of wind on your system, which-- some kind of cross-sectional wind, [INAUDIBLE] direction, which we see there.
And so what that ends up doing is pushing you system away from where you want it to actually be.
So now you have some kind of velocity uncertainty.
And you also have wind that can push you one way or the other.
So how do you deal with that?
Well, you start with your nominal trajectory that has a plane flying perfectly straight.
There's no uncertainty.
Everything is deterministic.
And it acts like you want it to.
And then, so right there the wind hits it.
So what do you do?
You have to have some kind of control, some kind of controller to bring you back to your nominal trajectory and try to control you around the system.
So a standard way to do that is to have a cause function and create your optimal trajectory.
And this cause function basically, it kind of mirrors a LQR controller.
And by solving a lot of these equations, which is a pretty standard process, and to find out how to do that, you get a controller that tried to bring you back to your nominal trajectory.
And so you get back.
You're happy.
But you don't have any guarantees just with the controller.
You don't have any guarantees on how far you can get pushed.
You don't have any guarantees on where you're going to go and where your system really can be pushed.
So when you compute the funnel, what you do is you choose a Lyapunov function.
And I'll explain what that is in the next slide.
And you try to minimize the space around that function that tried to bring it back.
So what this is is-- so you have some kind of your initial state.
This would be the initial state.
And the whole region, what you're trying to do is minimize it around it.
So does that makes sense why you do that?
So your trajectory is basically going to go anywhere around here.
And what you're trying to do is find the region of space that minimizes-- that encompasses all of the places that your system is going to get to.
So that's easy to do when you it out and you know exactly where your system's going to go.
It's easy to just shrink your ball around it.
But you actually compute that.
No, forget it.
OK.
So when you have that for each point in the trajectory, you know that you're going to stay within here, because you're going to have some kinds of-- from your nominal trajectory, you're going to have different places where your system can go under the given conditions of velocity and wind uncertainty.
You're going to go anywhere withing that.
But as long as you know you're still in that ellipsoid, then you're going to stay within those ellipsoids as you go down the path.
Does it make sense?
Good.
So the reason they use Lyapunov functions is there's different ways to measure stability.
And so I'll move forward to this.
So there's different ways to measure stability in a non-linear system.
So in a linear system, you're either unstable or you're globally exponentially stable.
So you're going to converge back to your-- to whatever stability equilibrium, what you have at an exponential rate.
But there is also, in non-linear systems, there's global exponential stability.
There asymptotic stability.
And there's stability in the sense of Lyapunov, which is what we care about.
So the difference between that is-- so say this is your nominal.
And globally exponentially stable means that no matter where you start anywhere on the board, you're going to be converging back to this equilibrium going at an exponential rate.
So that's the best case scenario.
That's what you want.
Asymptotic stability means that you're going to be going from somewhere on the board, within a certain region, back to this nominal trajectory.
But it doesn't have to be an exponential rate.
So you can do something like that, as long as eventually you sort of converging back to it.
And then there's stability in the sense of Lyapunov, which is-- so, say you're here.
Your system can be doing anything, anything crazy, as long as it stays within some certain thing, which in the 1D case we saw was the little ball going around it.
So imagine there's like a ball for each of these.
and this is where your trajectory could possibly go.
As long as it stays within that, you can say your system is stable in the sense Lyapunov.
Does that make sense?
Cool.
So know this, does that--
I just have a basic question.
So the Lyapunov function is around a stable set point.
But the trajectory has non-zero velocity.
So what is the point that it's stabilizing around?
PRESENTER 3: You can stabilize around a trajectory, which also what the funnels do.
So we have your little point here.
And this is your nominal trick, some kind of state.
And you're wind can knock you this way.
Your velocity difference can knock you this way or this way.
But as long as you're still within that, You've got to converge back to it.
And this is the value for your Lyapunov function.
And these are the different states you can have.
An important thing to notice is the derivative is a negative definite-- or negative semi-definite in the case.
And that means that basically all your trajectories, no matter where you go, they're going to be going down this cone.
And eventually they're going to to down to some value.
So does that make sense?
That's like pretty amazing stuff and your system stuff.
Cool.
So once you know that, it makes sense to use Lyapunov functions as candidates for computing funnels.
So an ellipsoid is actually defined by a quadratic Lyapunov function, which is down here.
[INAUDIBLE] shown the equation for an ellipsoid is some V times S. So that was the equation for an ellipsoid that Gabriel showed previously.
The quadratic Lyapunov function has the same structure, where you have this being x to [INAUDIBLE],, which is basically the error of you state.
So it's some kind of exact mean of the state that you're actually in minus x0, which is the nominal point that you want to reach.
So this defines the region around your-- the nominal point, which would be somewhere in there.
And the ellipsoid is computed by doing that minimization that I showed you on the previous slides.
And then you can figure out the region inside of it by solving that equation.
And if it's less than 1, I believe you inside.
And those are all the red dots that show that you're inside of it.
So that's used when computing trajectories.
You solve for-- say that you're inside a point cloud, you know which points you're going to get inside of this ellipsoid.
So when you're planning, you solve that equation for each of the ellipsoids.
And you know if you're going to hit something or not.
All right, so this is a video that [?
Ani ?]
did.
And this is using the funnel libraries for collision avoidance.
This is showing for just one funnel.
The glider dynamics were shown in one of the previous slides.
They were pre-set by the glider dynamics.
But [INAUDIBLE].
So you see right there that there's objects, and the glider plans a path around them.
So the funnels-- this is one possible one.
Right here, I'm pretty sure they tell it where objects are, even though they're not actually there.
And it finds that funnel being the one the guy wants.
And you see that throughout the entire path, the glider will stay within the funnel.
And then when you're searching with the online planning algorithm that was shown previously, it goes through all of these funnels and eventually finds the best safe one.
You won't always find a safe one, as discussed.
In this case you did.
But it finds the best one around the path.
So one way that it finds the best one is by finding all of-- so if you're planning a point cloud, you find all the points that lie with the ellipsoids.
And the one with the least amount of points is probably going to be the safest one.
So here's a bunch of different obstacles.
And it safely maneuvers all.
And there's some pretty dynamic movements, which is cool too.
So within that there is-- let's see.
I'm going to back to-- is there a way to get back to the full screen [INAUDIBLE]??
[INAUDIBLE] PRESENTER 3: [INAUDIBLE] So with the online planning algorithm, here's one of the possible paths.
You have your point cloud of trees.
And those were shown in the previous, the little round thing with the green foxy tree-looking things.
Using a sensor model, you get a point of that.
And as you're going, you plan your path.
And the way that this works is every five meters, you plan a path.
And then as you're flying through it, you get new sensor information.
And you can replan your path using the algorithm.
And that will guarantee you that you're safe.
And when you're flying, obviously you don't see the funnels.
But right here you can see there's probably one funnel here and then another funnel that plans this way, another funnel that goes to here, and then another one that goes there.
And that's a full path.
And there's also a cool one , because it shows that you go through the tree.
And that's where one of the funnels is rather than going around.
It is possible that you start computing a new funnel-- or you start searching a new library here.
And you may not be able to make it up the way in time, given what speed you're going at.
Does anyone have any questions?
And this is the online planning algorithm that we saw before written out.
And some of the references, the paper from the video that we just saw, that's given here.
And that also go to the example [INAUDIBLE].. Frazzoli was one of the original people that computed a flow tube.
Hoffman and Williams did the temporal planning for footsteps and decoupled humanoid system.
And I did most of the stuff on [INAUDIBLE] for that years ago.
Another important thing to note that was a big thing in my personal research was that even though you may note be able to compute-- or that you may not find a funnel that necessarily goes-- finds a safe path, you can shift that funnel around.
You could even move it slightly up and down, as long as the first ellipsoid is still within-- as long as your initial state is still within better ellipsoid.
And that's easy to do because you can just give it a simple transformation between one state and the other.
And if you transform this S matrix, then that will transform your entire funnel and it will shift it around.
So you can find your best funnel, even though it may hit something.
If it was slightly shifted to the left, you can just shift it using that transformation.
So that's one of the [INAUDIBLE] funnels too.
Does everyone understand funnels and flow tubes now?
So I have two questions.
So the side of that funnel is computed by the transform script.
Is that right?
PRESENTER 3: Yeah, if you want to get it some ridiculous wind, like 1,000 miles per hour, then you're going to get huge funnels.
But that's under some kind of reasonable-- you know in this region of the forest or something there's only three mile per hour winds or something like that.
Yeah, that's based a lot on what your counts are
So do you have any algorithm that in your mind that is not using offline [INAUDIBLE]??
So this is like mostly you do the offline and the you do the online search.
But if you were in a wild environment, you do nothing.
Then do you have any [INAUDIBLE] that isn't-- PRESENTER 3: I mean, there are algorithms that will-- well, just the basic controller will try to get you back to nominal trajectory.
But there's nothing that I know of at least that will guarantee you safety and will give you an all-encompassing region like flow tube or a funnel.
So flow tube you find there's an infinite number of paths that will get you from here to here with your system, basically every little minute change that you have.
And that obviously takes awhile to compute and approximate.
And the funnels, it takes awhile to compute the regions where it can go.
So I don't know anything that will online compute this region.
But online you will be able to compute some trajectory.
And then let your controller will take your state error and control it.
But it won't guarantee [INAUDIBLE]
With the funnels, how do you compute your initial set of states, or the allowed initial set of states?
You have these uncertainties, like for the wind and velocity, but what about the initial state before those disturbances happen?
You'd probably want more than one initial state
Here's and eraser.
PRESENTER 3: Yeah, that'd be good.
Cool.
Thanks.
So with the funnels, so you have your nominal trajectory again, and then your ellipsoids around it.
So when your system starts, you have your initial state.
And your initial state is just-- it's going to be here at time t0.
But over the time computed, you can go over here, or anywhere around here really, depending on what happens
But I just me that you'd want to apply the funnel multiple times.
And you're initial state may not be exactly the same.
So it should be possible to use the same funnel even though the initial state is off by a little bit.
PRESENTER 3: Yeah, OK.
So say that you chose this funnel first.
And eventually what actually happens is you do this and you get pushed around.
And then you end up here instead of this funnel.
As long as your next funnel that you plan-- so you can start a new funnel at any point in this trajectory, as long as the next funnel is also-- as long as you have V intersection between the point and the next funnel.
And if you're within this region, and you plan the next funnel, what do you do is you plan the next funnel with this being the original x0.
So as long as that's you next x0, and then this will take on a new nominal trajectory
Yeah, I'm just saying, asking how you, like for the one all the way on the left, that one, yeah, how do you define how big that initial region is?
PRESENTER 3: Oh, OK, so that one, say, if you start, say, not moving or something and you got pushed?
Is that what you're asking for--
Well, I mean, I understand that you're using the V for uncertainty, and V and W. So that essentially defines how big those funnels are.
Do the also define how big the first one is?
PRESENTER 3: Yes.
Yeah, the do
OK. PRESENTER 3: So what you're doing, because what you want to do is use the funnels and have some funnel library and be able to use those throughout all time.
So you don't want-- so I guess what you're asking, the first funnel, you're going to start at some initial state 0 with no velocity, stuff like that.
And so you're not going to have any uncertainty on the first thing because you know where you start, the initial state
Yeah, I'm just asking independent of the wind and stuff like that, you might just be starting at a point other than what that nominal point is.
So you may want to define that initial funnel size by some other criteria besides wind.
Is there--
So [INAUDIBLE] sensor how sure you are about where initial location is
Yeah, and then-- I mean, there's sensor uncertainty.
But you just may happen to be in the-- if you're doing the glider experiment 10 times, you may not start it from exactly the same position every time.
PRESENTER 3: Yeah, you should-- your funnel, every time you plan a new funnel, you start at the nominal trajectory above that new funnel.
So you're not going to have this funnel.
And you're never going to start over here on the first one.
You're always going to start at this point.
[INTERPOSING VOICES]
I though the point of the funnel was so that you can reuse it without having to replan.
PRESENTER 3: Yeah, you can move the funnels around
Oh, you can move?
OK. PRESENTER 3: Yeah, yeah, that was-- so every time you start a new funnel, you start at this nominal trajectory.
But the next time you use the same funnel, you could be over here and just shift the funnel down to exactly the point.
So the initial state is never going to be anywhere starting at t0, it's not going to be somewhere other than the nominal trajectory.
But at t1 it could be over here.
And so you just shift the whole funnel around.
And as long as your t0 starts at the new funnel, and as long as t0 is within the funnel above t-- last funnel, then you just shift it around.
You can also turn it, as long as you can multiply this and this
It seems like the shifting and turning is a pretty complex search problem in itself.
Is that a hard problem?
PRESENTER 3: No, that was actually the-- [INAUDIBLE] I got rid of the slide.
I might have gotten rid of that slide.
I shouldn't because that was the work that I did for the [INAUDIBLE].
Let's see [INAUDIBLE].
But, yeah, there's a bunch of different ways to do it.
I did a pretty not very smart search, where I just looked for the funnels, the safest funnel being the one with the least number of points in the point cloud within that entire funnel.
And then-- oh, man, I did get rid of this.
But basically what I did was find the safest funnel.
And if there still was a collision within the safest funnel, the I admit I had a minimization problem as you shifted back and forth.
And there was a cost function that-- see if I can bring it up
So you're doing an optimization of the online running of the-- PRESENTER 3: Yeah.
That was an input that was implemented in realtime simulation.
But that wasn't implemented on the actual glider as of [INAUDIBLE].
So let's see, all right [INAUDIBLE].. And you also want-- I also made is so you minimize the amount that you shift it to.
So you don't want to shift it completely somewhere else because it won't be in your path.
And you'll end up doing some dynamic [INAUDIBLE].. Like, you'll be flying, and then if you compute a funnel that's here that's kind of safe, but you shift it all the way down to here.
Then you're gliders going to be doing something crazy.
And that may-- I guess it would be [INAUDIBLE]..
But I minimized that for at least [INAUDIBLE] shift it.
Does anybody have other questions?
Yeah, so does anybody have any questions on--
Can I ask one?
So you've given us models of bounded uncertainty in your dynamics.
So if you use probabilistic models, then your funnels start looking like risk allocations.
Is there an interpretation of one in terms of the [INAUDIBLE]?
PRESENTER 3: I don't know.
No, but I'm sure if you look it up, yeah.
I'm sure somebody's been working on that.
I'm not really sure that fits-- I know I haven't done that
Talk a little bit about the interaction between the flow tube selection or the funnel selections and the underlying [INAUDIBLE].
So let's say you have something that's planning the reference trajectory that's separate from your flow tube selection or your funnel selection.
And then you have to select the funnels and flow tubes that allow you to follow that trajectory the best you can.
How do the interact if you have a flow tube-- or don't have a safe flow tube or funnel [INAUDIBLE] reference trajectory?
PRESENTER 3: So you're asking basically the difference between planning with a funnel laddering and trajectory laddering
Yeah, kind of sort of-- well, because I know the checkoff planner that uses an incremental pathfinder, so the planned path or something like that.
And then you fit-- are you fitting flow tuber to the reference trajectory?
PRESENTER 3: Oh, you do that off and on.
So you compute-- they're called funnel libraries.
Or usually they're trajectory libraries.
So ever computing a trajectory in realtime for a fast-moving dynamic system is going to be expensive.
And then computing a funnel on top of that is going to be even more expensive.
So what a lot of people do is they compute the library offline for a trajectory and plan using that.
But on this one, you compute your trajectory, compute the funnel around that trajectory offline.
And then you search through the funnel library.
You don't search through trajectories and then try to fit a funnel around that one.
You just search through the funnel storage.
PRESENTER 2: If you don't have a very good library, then you will not get a very good trajectory.
PRESENTER 3: Also, a cool thing about funnels versus trajectories is a lot of trajectories will be discretized for speed, so you have your nodes.
And so, say this is what your trajectory does, and you have node points here, and you're trying to fly past a wall or something.
If your wall goes here, you're algorithm is going to search these points, and it's going to say, OK, well, this is a safe path because none of the node point intersect with this wall.
So you fly [INAUDIBLE] a little [INAUDIBLE]..
But a funnel, since you have this region around it, you-- no, I did a bad job of drawing that, but OK.
When we search the algorithm and see if it's less than or greater than 1, you're going to find all these points being in that region also.
I guess that depends on how well you want to discretize it.
If your trajectory is [INAUDIBLE] of really, really small points, then you're not going to run into that issue
Seems like you could-- like when you're building a normal RIT without funnels, from concave points, if you say, yeah, they're illegal, it seem that if you're automatically adding on to that funnel instead of generating an entire trajectory and then making a funnel, you could save computational time for [INAUDIBLE] things like that.
Do you know if that's done?
PRESENTER 3: No, I don't.
I do know that, I mean, computing, you obviously want to minimize computation time.
But it's not a critical thing when computing, I guess, funnels and flow tubes, because you do that all offline.
And you want to-- I don't think it's even close to the point where you go to online to compute a funnel.
It takes like six hours to compute it, so-- depending on how many dimensions your state has.
So, yeah, it's not like you're flying and you can compute one of these things.
You'll crash like [INAUDIBLE].
OK. [APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.MIT.edu
Self-driving cars are pretty cool.
It's pretty hot nowadays, so we're going to use this as an example.
And in the past we have seen how we can give an autonomous vehicle a goal-- some condition we want it to reach.
For example, we might have a car that wants to pick up some passengers along the way, get to some destination, and maybe navigate some worlds.
But let's consider that for something like a self-driving car, we also have a number of other goals or requirements that are sort of implicit along the way, in that we want our self-driving car to obey the rules of the road as it's driving.
And these aren't just goal conditions which is a sort of single goal that you're trying to reach, but these are sort of goals that you're trying to maintain throughout the path that you're traveling.
So one example of this is a traffic light.
So maybe you guys can help me out with this.
Just in plain English, how would you describe the rules of how you would want a car to behave when it comes across a traffic light?
Anything?
It's very simple.
Yes?
Slow down when you see yellow, stop when you see red, go if you see green
Yeah.
So pretty much stop if you see red, go if you see green, and specifically, if you see red and you stop, you're going to want to stop until you see the green, at which point that sort of condition that you're stopping goes away, and you're able to move on.
And in addition to things like obeying traffic lights, there might also be some other rules of the road that we want to follow.
For example, we might want to always stay within the speed limit.
And then there might also be some other practical conditions of driving down the road, which is that at some point we're going to need to refuel.
And if we consider a car that might be driving for a really long time, one logical statement we could say about the path that this car takes, and the sequence of states that it goes through is that at every point in time we want the plan to have some future state when we are going to visit a gas station.
So this isn't just a single goal that we're trying to reach, but over a very long time, even if we consider the car to be driving an infinite amount of time, at every point in the time, we're going to be thinking ahead that there's going to be some time when we reach a gas station and refuel.
So these types of conditions, things like staying at a red light until it turns green, always staying within a certain speed limit, or always having some path in the future-- some state in the future when we reach a gas station, are things that we can express with a type of logic called temporal logic.
And there are two things that we hope you're going to be able to do by the time you leave this lecture.
The first is be able to model these temporally extended goals using a specific type of temporal logic called Linear Temporal Logic, or LTL.
And secondly, we're going to hope that you'll be able to actually apply this to planning, create plans with these temporally extended goals, and in addition just sort of this regular planning we've been dealing with, actually incorporating this idea of preferences into your plans.
So a preference basically is not a required condition, but a desired condition that you can use to select between alternative paths.
So you know, you might have many different ways that you could reach your goal, but we can use some of these temporally extended goals, like maybe you prefer to break as few of the rules of the road as you can, and you can use those a preferences to choose between plans.
So just as an outline, first we're just going to do a little introduction to linear temporal logic.
We're going to talk about what this is-- what LTL is, why we want to use it, and we're going to go through the syntax and the semantics of different LTL operators.
Then we're going to walk you through some example LTL problems, and actually talk about complications to plans-- how you create plans with these linear temporal logic and temporal goals.
Finally we're going to incorporate preferences into this, talk about how you express preferences, and specifically talk about a language called LPP that allows you to plan with temporal logic and preferences.
So now for the introduction to linear temporal logic.
Temporal logic at its core is a formalism for specifying properties of a system that vary with time.
So these aren't just conditions that are true at single state, which is what we've mostly been dealing with prepositional logic.
Right, so we've been saying things like, condition A and B, but not C, are true at a given state, and that's prepositional logic.
Temporal logic is sort of a layer on top of that, when we're dealing not just with what's true at a single state, but actually extending over time as the system moves through a sequence of states, and expressing properties on a temporal level.
So you might have a system that can be represented as a state machine, and a [INAUDIBLE],, and it can go through a sequence of different states.
And while you might be able to represent the whole system like this, if we actually execute the system we're going to get one path through the system.
And that's often called a trace of the system, and you can also think of it as a timeline of the system.
So one example timeline of this system is it might start in state A, and then go to state C, and then go to D, and keep looping around in D forever.
So that's one trace, or one timeline of the system, and that's really just a sequence of states that it goes through.
And in the past we've seen these in some of the problem sets.
For example when we were modeling the warp reactor, we had the valves that could transition between open and closed states.
It could also be the whole system-- the starship going through different planets, transitioning through a larger set of locations, picking up passengers, dropping them off.
So these could be quite complex.
And so as I said, previously in our problems that we've been modeling, we've been going through a system, and we've been searching for a single state that satisfies all of our goal conditions.
Maybe we to reach [?
Lavinia ?]
and save a certain number of people.
But with temporal logic, we can actually express goals that are satisfied over a number of states, and this allows us to do some more expressive goals than we could capture just by looking at a single state at the end.
So for example, what if a problem requires some condition to be met until another condition is met?
And for example, you know, when you see a red light, that implies that we should stop until we see a green light.
And if we look at a timeline, or a trace of one of these systems, that might look something like this.
So we're going along through our system, and at one of the states, we see the red light.
So that adds this condition that we should stop.
And then that condition will hold forever until we see a green light, at which point that condition is dropped, and we can keep moving on.
Another example where we might want to use temporal logic is, what if our problem requires a condition to always eventually be met.
And this is what I was talking about with the gas station.
So we want to be going on forever, and there should always be some point in the future when we do have a state where we're at a gas station.
So we might start our system at a certain state.
And our plan says that at some point we do reach a gas station, so that's good.
But then when we get to the next state, we also want it to be true that from that point on, we're going to reach another gas station, and after that, going on and on and on into the future.
These are things that you can't express very easily with the types of logic that we've been dealing with before.
So one important distinction that we should clear up before we really dive into the syntax of how these linear temporal logics work is the difference between branching time and linear time.
They're really two different models of time that we can be working with.
There's two different types of temporal logic that exist to express these different types of time.
So linear time is more similar to the world that we live in, unless you're living in some sci-fi movie where time is branching into different realities, and all sorts of crazy stuff is going on.
You can think of time as just this linear timeline.
So right now, it's one future state that's going to happen for me.
And you know, regardless of how many options I have, there's one realization of the time.
And what this lets us do is that it gives us a single path for our system, and we can reason very exactly about the conditions that are met within that path.
And this lets us describe what will always happen in a system.
For example, we can guarantee that we always stop at a red light until we see a green light, or we always stay within the speed limit.
This is contrasted with a different model of time, called branching time.
And in this model of time, at each time instant, we consider all possible futures.
So if we have multiple actions that we could perform at a given state, we're going to consider different timelines that branch off for each of those possible behaviors.
And this results in alternate courses of time that we can reason about as a set.
And instead of thinking about this as always conditions-- so, our path always does this-- we can think about this more in possibilities.
And if we're using branching time, we can say, you know, at this state and time, there exists a future path in which we will always stay under the speed limit, or, at this point in time, all our future paths satisfy a certain condition.
So whereas linear temporal logic is more about guaranteeing what will happen on a given path, branching time allows us to reason about what might possibly happen in the system.
So for our purposes with planning, in part because it's more attractable, and in part because it gives us an exact analysis of a given path, we're going to be doing linear time, and for that reason, we use what's called linear temporal logic.
There's a different type of temporal logic.
There's one called CTL and CTL* that deal with branching time.
And those are sort of an extension of linear temporal logic.
They add a few other operators that allow you to reason over multiple paths.
But for our purposes, we're just going to be using the linear time model.
So kind of as a recap, what linear temporal logic involves-- we're using this linear time model.
Another important distinction is that we're actually going to be talking about infinite sequences of states.
So instead of us just reasoning up until a certain goal condition is met, we're thinking about systems that could theoretically keep transitioning over time, over and over and over into the future.
So if you have one of these state machines, you could think, at each time step, it's going to visit the new state.
And if you give it enough time, it's just going to generate an infinite sequence of states.
And we can actually express properties that can hold over an infinite number of states, not just the discrete number of states that we might usually think of to reach a goal.
And linear temporal logic can actually be modified to work with discrete set of states.
It just requires an additional operator.
But the ability to reason about infinite sequences of states gives us a lot of power in talking about guarantees for our system, regardless of how much time we're talking about it.
Finally, all the operators we're going to be looking at in our logic system are what we call forward-looking conditions.
So when we're given a state we're going to reasoning about what's going to happen in the future for a potentially infinite amount of time in the future.
There's sort of a symmetric version of this logic that reasons about the past.
So for example, we can say-- talk about guarantees that something will eventually happen in the future-- there's sort of a symmetric version where we can talk about, given a certain state, we can guarantee that something had eventually happened in the past.
But for us in that we're going to focus on planning-- we're typically planning about what to do in the future, and it makes more sense for us to use this future of updating the set for operators.
And so I guess one little nuance here that I sort of pointed out in the previous slide is that because we're using a linear model of time and not a branching model of time, we're going to be expressing properties that happen over a single path of states, and not properties that happen over several path of states.
So finally I'm just going to talk about a couple of the main application for linear temporal logic before Ellie will dive in, and actually talk more about the semantics and syntax of how we use this logic.
So the first menus is verification and model checking.
We can use this linear temporal logic to create guarantees about a system, and formally verify it for a couple different properties.
For example, we can talk about the safety of the system.
We can say, over the course of this system, we guarantee that it will never enter a state of a certain condition.
We could also talk about the maintenance of properties.
So over the entire lifetime of this system, we guarantee that a certain condition will always hold.
The other main application for temporal logic, the one that we're going to be focusing on, is for planning.
And this planning using these temporally extended goals that I've been talking about, and it actually extends nicely to talking about temporally extended preferences.
So this is given that we have a number of different paths that could reach a goal, we can actually look at all those and see which paths satisfy certain conditions over the sequence of states, and use those conditions as a way of choosing which ones we prefer.
So with that, Ellie is now going to talk about some of the syntax of linear temporal logic
Hi guys.
My name is Ellie, and I'm going to be talking about the syntax of LTL.
So an LTL formula is a series of states.
And at each state, you can have various propositions than can be either true or false.
So for example-- is it OK if I erase this, Steve?
Yeah.
Go ahead
So for example you can have a [INAUDIBLE] animation show that has a series of states that goes on to infinity.
And at each state you can have propositions that are either true or false.
So the traffic could be red could be an example for a proposition.
We could be stopped, et cetera.
And there's various logical operators that we can apply to these formulas-- these sets of states, and we can determine if these logical operators are either true or false about each set of states-- each formula.
So there's the not operator, the or operator, the and, the implies, if and only and, and like I said, the state-- each proposition of [INAUDIBLE] can either be true or false.
So just to go through some examples of these, the logical operator true says that-- essentially what the operator true says is it's not saying, oh, this state is true.
The operator just true says that there are states for all infinity.
So true is always true.
So no matter what you had for your state here, the logical operator true holds.
However, when you talk about the logical operator true with respect to a certain state or a certain proposition, then it can either be true or false.
So for example, here, the logical operator red light is true at the first state.
If this was green, then the logical operator red would be false at the first state.
Does that make sense?
All right.
One important thing to note about the logical o operators is that they just apply to the very first state in your statement.
So when we're discussing logical operators, we haven't gotten to the point where we can express temporal information about our sequence of states.
The next operator is, not.
So as you can see here, the traffic light is read at the first state.
So the traffic light's not green.
Hold, because the proposition green is not true at the first state.
And the logical operator, and, at this first state is we're at the red light, and we're getting gas.
And so both that we're at the red light and that we're getting gas holds for the first state.
And then the operator, or, essentially says-- I guess a lot of this is probably review for you guys-- logical operators-- but essentially if you have red or green then you could either have the traffic light be red or the traffic light be green at the first state.
And as long as one of those two hold, or both of them hold, then it's true.
And one thing to note about or lies in if and only if these can be re-written with just the logical operators, and, and, not.
So another way of expressing red and green is to say that you can't have the scenario where both is not red, and it's not green.
And I'm not going to go over that for, implies, and, if and only if, just for the sake of time.
So that's just a review of the logical operators, and now we're going to go into the temporal operators.
So when we're talking about our autonomous car, what are some useful temporal operators?
Because up until now, we just are describing of the first state in our sequence, which isn't really useful if we want to talk about the entire set of sequences [INAUDIBLE] states.
So what do you guys think are some mutual things that we want to be able to describe?
I know Ben touched on some of them at the beginning, but if you guys could recount some of them.
Yes
State connection
Mm-hmm.
State connections.
So exactly.
So if I'm in a state right now, might there be concerns about the next state?
Is that kind of like what you're saying?
Exactly.
Yeah.
So next is one of the things that we might be concerned about, the next state.
Are there any other tings you guys can think of?
Always or [INAUDIBLE]
Yes, exactly.
Always.
So like Ben said, we always want to follow the speed limit.
We always want to be under 35 miles an hour, or whatever the speed limit is.
Anything else?
Until
Until.
Yep.
So like he said, you want to be stopped until the light turns green.
Yes
Eventually
Eventually.
Yep.
So we eventually want to get gas.
And eventually.
Any other things?
Yes
At this state
At this state.
Yep.
And so, at this state, is kind of what the logical operators were doing at the beginning, but I agree that's something we have to be able to express.
So that's not a specific temporal operator, but it's included in the logical one, so I'll just write it up here for you
Can I ask a question?
Uh huh
So the logical operators, are they acting on the-- so they're acting for a state?
They're not acting on the entire chain
They act for a state.
And so if you just wrote the logical operators, just by the definition of LTL, it means let's just analyze the first state and see if it's true.
And so it won't tell us anything about the third state or the fourth state or the second state, which is why we have to look at the next state.
So yeah.
Any other questions?
OK.
So like you said, next is an important one.
So right here we consider OK, the next light will be green.
And until is the one that she mentioned, so the light will be red until it becomes green eventually.
So the light will eventually at some point in the future turn green.
So you might be waiting at the red light for a long time, but eventually it's going to turn green for us.
The light will always be red, so I guess you're stuck at the traffic light for a long time, or globally-- that's another name for always.
And lastly, this on is a little bit less intuitive.
So what this is saying is the light will always be red until another circumstance coincides with the same state.
So the light will be always red until the car gets gas.
And then after it gets its gas, then the red is released, and the light can be either green or red after that.
So you have red red red red red until we have both red and gas.
And then it can be green or it can be red [INAUDIBLE] it's not constrained to being red.
Does that make sense?
That one's a little bit less intuitive, so-- is that good?
OK. All right.
So now that we've gone over just the intuitive understanding of that, I'm going to give you the actual syntax for that.
So x is representative of next.
So in the first state, we don't care if p is true.
p could be not true, p could be true.
It doesn't matter.
But in the second state, the next one up in the first state, p, has to hold in order for xp to be a true statement about our sequence of states.
So what do you guys think if I wrote this?
What would that mean?
Different state [INAUDIBLE]
Yep.
So this one over here?
Sure.
[INAUDIBLE]
Yeah.
Perfect.
All right.
The until operator.
The until operator essentially says, like Ben said, we're stopped until the light is green.
So if you have p until w, you just p has to be true at every state until w, and after that point, p can be whatever it wants to be.
p eventually, or future operator, says that at some point in the future we will get gas.
However it's important to note-- I know when we talked about getting gas we said, oh, we have to always get gas at some point in the future, but the future eventually operator is just at one point in the future.
So once you get gas, you don't have to get it again.
Yes
On the previous slide you had a not at the bottom that said that w is required to become true.
Does that mean an until operator also implies a future operator on w?
Yeah
All right.
[INAUDIBLE]
Yep.
But it's the added condition that p will be true until the w
But like whenever you use until, you say p until w that you'd also have to add at some point in the future [INAUDIBLE]
Yeah.
Yep, you're right
Well, I mean, it would be possible that this stays p, right?
It could.
Yeah.
But then if your system-- that would be totally fine if you had your system just be p, but then the operator p until w wouldn't hold unless you had a w at some point in that sequence of p's.
Does that make sense
Yeah.
[INAUDIBLE] I guess it does if that's the way to do it, but it doesn't make sense [INAUDIBLE] it
So you're saying that we could have a bunch of p's, right?
Is that what you're saying?
And I'm saying that this is a fine state to have.
You can have whatever you want.
And we're just analyzing if a sentence is true about it.
So this statement, it's not true-- or, sorry.
I'm saying that it's not true that we have p and lw because there's no w in our state.
But if we add in a w somewhere and we keep all the p's, then it's true because we have p up until we get to a w. Did that make sense?
It made sense that that's the rule, but it doesn't make sense to me why.
Because the statement [INAUDIBLE] true and if w never appears.
You have p until w, which never happened
Yeah.
If w never happened, then it wouldn't be true
I mean, technically, for the English language it would be true
I guess it's not--
If you have p out to infinity, and you never have w's.
p until w is true
p until anything is true [INAUDIBLE]
Like if you say that I'm hungry until I eat, and you never eat, then you will always be hungry
Oh, I see.
So that makes sense in a logical sense, but that's not how it's described in
There's also an operator that's sort of unofficial called the weak until, and that's sort of saying p until w, or always p. So that kind of gets rid of the condition that it reaches w at some point
It aligns more with the English, too
Yeah
Yeah
But just the until operator on its own is just sort of like a strong until
Yeah
So it adds the--
Yeah, that's fine

This is the way it works
Yeah.
Sometimes-- yeah, I get confused about this sometimes.
I'll be like, why they necessarily [?
chose any of these, ?]
but, OK. And then the globally one, which we said-- this is the one that you were just describing here [INAUDIBLE] when you were talking about the p at every single state in the future.
And then the release operator is you have to have p until you have w, but p and w also have to share the same state when they switch from-- so in until, you don't have to have the p here, but with release, you have to have the p and w occur in the same state.
Does that make sense?
Yes?
OK.
So just like before, operators can be described using not and and.
The future, released, and the globally operators can also be described using other temporal operators.
And so just as an exercise, can any of you guys tell me which-- these are not in order-- which of these line up with the other-- like, whichever ones they line up with.
And I'll give you guys like three minutes to just think about it, and then we'll work through it together on the board.
All right, so have any of you guys thought about which ones might correlate to which ones?
Does anyone have any suggestions?
Yes
The first one's the top one
This one is the top one.
That's exactly right.
So let's continue this state again.
So the statement, true, doesn't say anything about what the propositions are.
It's just something that's true for all states, right?
So if we're saying, true until pi, that means that you can have whatever you want until p occurs.
So does that make sense that, at some point in the future, p has to occur, exactly because we had the strong until, right.
Does that make sense to everyone?
You guys want to see some [INAUDIBLE]??
All right and now what about this one?
We have not as not p. So this is saying that at some point, if you can't get a p set at some point in the future, you don't have p. So what do you think that would correspond to?
If at some point in the future, you can't have the situation where there's not p
The third one
The third one?
[INAUDIBLE] So that would be if you have p at every single state, then you would never have a situation where you don't have p, right?
And so that leaves this last one.
You don't have a scenario where you have not p until not w. So if you have not-- OK.
It's still a little hard.
But I'll come back to that one at the end, OK?
But it's just important to remember that you can describe these operators with similar to them so that your planner doesn't have to deal with the global operator or these operators-- eventually operator, it can have less [INAUDIBLE] to deal with.
So yeah, you guys were right.
Perfect.
And then this is just a recap of the different operators and the other ways that you can express them.
And so there's a few more really cool things that you can do combining these different operators, and the first is that-- remember when Ben was describing that we need to get gas?
And we need to get it in the future, but even after we get gas, we're going to have to get it again some time in the future.
So it's important that we can describe something that happens infinitely often.
And the way that we describe that is saying that we always have the scenario where you will eventually reach p. So I'm trying to think of another way to describe that, but does that make sense that right now in the future, I'm going to get gas.
And even after I get that gas, I will eventually in the future also need to get gas again, because your car will always be needing to get gas.
Does that make sense?
Cool.
And then eventually forever is another scenario that's important to describe.
So let's say that the traffic light will eventually turn green forever.
We need to be able to say that at some point in the future, although we don't know when, the traffic light will turn green.
So the future operator says that at some point in the future, you will have something come true.
And so at this state, p becomes true, but the global operators that happened from this state forever afterwards, which is saying that you eventually will always have this state B be true.
Do you guys have any questions on that?
Did I explain that good enough
It seems weird to me that the gas tank is becoming this occasional temporal operator instead of just like the state that we actually have gas, would just be the logical thing that [INAUDIBLE]
Oh, yeah, I--
Why is the act of getting gas showing up here like that, versus just, you always need quantity of gas
That's true.
We do always need some quantity of gas, and you would describe that, like you said, just with the always operator, but I guess my action of getting gas is I was thinking of actually driving up to the gas station and filling it up with gas.
And if you didn't do that, you wouldn't be able to always have gas.
Does that make sense?
Right, so I would think that always having gas would make [INAUDIBLE] of you plan [INAUDIBLE]
Yeah, and depending on how you make your model, you definitely could do that, and that definitely would satisfy the condition.
[INTERPOSING VOICES]
--the usual way to deal with it?
Well, it depends on the situation.
Maybe you [INAUDIBLE] want to say at some point in the future, the light will always be green, because you want to make sure that cars can go through, and there's a difference between red, and some point it will be green in the future.
So it really depends on your model and how-- I think the way you're saying it, where you always want to have gas, and the proposition gas is how much gas you have in your tank, and if you have it or don't, then I think that that would be a really good way to model it, too
Because this way, if you said, I have to get gas.
I planned it for next week, but I've run out of gas this week.
Do you know what I mean, like, you have to somehow be tracking the quantity of gas also
I agree.
I agree.
I was just using it as an example to explain the concept, but I agree that that definitely probably could be better with more work
Well, I think in both cases what it could represent is having gas all the time, or explicitly saying that you're going to be in gas stations eventually-- always eventually.
It depends on how-- maybe we'd want to always have gas from a station, and also getting gas from it.
So depending on how you want to model it, but I think both will work
So infinite and often, doesn't really care about how frequently--
Yeah, it doesn't care how frequently
But [INAUDIBLE] for instance something that happens every 60 years.
My system lifetime server is 30 years, which means something happens only once.
So in this case, infinite and often, how can you filter a switch operator could be the same, right?
Can you repeat that again [INAUDIBLE]
So for instance something happens every 60 years.
My system lifetime goes to 30 years.
After 30 years, I have to get a re-up on the system.
Then so during this [INAUDIBLE] only one thing happened.
And so it happens off into infinity, but in the lifetime of the system, it only happened once, and therefore in this condition, the infinite and often should be set the same as the future, right, in the current state.
Something happened in the future, I would say something could never happen again
Kind of.
So when you're talking about a system that's dying after 30 years, the LTL doesn't actual model something that ends.
It would model something to infinity, although there is an extension to LTL that can incorporate that into it, having it have an actual end goal state.
And I think in that case, then infinitely often would be different, and would-- I don't know exactly how that would find that situation.
Do you know, Ben?
I know you had kind of--
So there's something called metric temporal logic, which associates a time scale of each of these operators.
So I think you'd have to use that if you were trying to express that you had to always meet those conditions still within your lifetime.
You could have some sort of qualifier to these operators to specify an actual time [INAUDIBLE] where that operator [INAUDIBLE]
So which would you file for the instance infinitely often [INAUDIBLE] scenario something at a little bit more frequent of a general system by the time, right.
Now we have something you know [INAUDIBLE] 100 years ago for years
You know, it's important that the LTL we're explaining here is only dealing with infinite states.
So I agree, it's a little bit-- you have to think about that process of how do apply something that goes on forever to a real system.
It obviously doesn't model correctly
There has to be a way to-- I mean the future thing, if he has a system that lasts 30 years, and you say future p, somewhere in your design of the system is going to say that the constraint on future p is that it has to happen within 30 years, right?
Because future p here, for getting gas, has to happen before I run out of gas.
It's not just-- this representation says it just happens in the future, but somewhere in your constraint checking, it's going to say, hey, we didn't get gas in this plan soon enough, right?
Yeah
Somehow
Yes, that's definitely true.
And these are just tools to help you model it, but it's definitely not the entire model of your system obviously, and so I agree that we definitely would have to incorporate that in.
And for finance systems you have to account for that.
And that's why they have the extension of LTL which is the finance systems, which we can certainly drop a link of of the papers, and where we got the information, to the class if you guys would like that
Yeah, I think you guys have done a very good job of answering the questions.
The three extensions to LTL that I've found to be the most relevant kinds of things that we're doing is metric temporal logic, which is the first one that you mentioned, also to be able to have it over bounded time, which is related, and you mentioned.
And then the last one is to deal with uncertainty, and is then a problem of linear temporal logic
OK.
Cool.
All right.
So I meant to have this as two separate slides, but what are some true statements about LTL that you guys can tell me about to look at this, or do, and explain why if you look at it.
OK, so why is the next red true?
The next step is slow down
Yep, exactly.
Because the next one is red.
Why is it true that in the future, that it's green?
[INAUDIBLE] Because in the future, it's green.
And why is it true that there's red until green?
It keeps being red until it's green
Exactly.
So it makes sense that if all of these are true, that this culmination of the and statements between all of them is also true, right?
And so you could do that with or as well, or you could do that with-- if you had sequences behind here, you could put a future around this whole thing, and it would be true that at some point in the future, all of this would hold.
Does that make sense?
Yes?
Why keep it red there?
Your red is the next step
Yep.
Exactly.
I was just saying if you had more in the past, and you were considering it from the previous-- All right so now Nadia's going to talk about expressing LTL in PDDL3.
So Ben and Ellie have guided through how expressive LTL formulation is.
I'm going to formulate the LTL in a classical planner like PDDL3.
Now I'm using PDDL3, which is an composed extension of PDDL2.2, which supports some of the LTL operators.
So these are the basic operators that you can use to express the constraints and goals of your plan.
So you have at end, always, sometimes, within, at-most-once, sometime-after, sometime-before, always-within, and hold-during.
So again, now here is a numeric controls that you can specify, and the ellipses represents an already-existing [INAUDIBLE] goal.
This is a goal description-- [INAUDIBLE] So some of the operators may look familiar to you, because they can also be used to express the constraint in STN, the Simple Temporal Network, that we learned in class.
And there are some operators that are unique to LTL or other kinds, like metric temporal network, like always, for example.
So now we are going to take a look at how you can express the operators using the PDDL3 language.
So we can use, within one occurrence to the next, because there is no explicit next in the PDDL3.
And you use always until to express until.
And in the future, you use sometimes after, and globally, you can use always.
And for release, since it can always be omega or always until omega when you see p. You can use all costs, too, although this [INAUDIBLE].. OK, let's see some examples.
So if your goal is to have the traffic light turn red in the next state, you would want to formulate it using within one occurrence, the traffic light would turn red.
And let's take a look at a more complicated example.
So if you want to model a goal saying that the traffic light would be green until it turns red, at which point it would be red forever.
So this is temporal logic of predicates that express this goal.
And then if you want to model that using PDDL, there is a direct mapping.
You can see the direct mapping between the predicates and the PDDL.
So this is basically saying, it would always be green until it turns red.
And turning red implies it will always turn red.
So next, [?
Arleese ?]
will tell us how to map between the LTL to PDDL with a Buchi Automata which is a specialized automata
Cool.
So I'm [?
Arleese.
?]
I have bridged the gap between LTL and the planning world.
So this is kind of the framework for how you would go from taking a problem with temporally extended goals all the way to a plan.
So as you can kind of imagine from what Ben and Ellie were talking about, LTL is pretty expressive.
You can express a lot of different types of goals that include more temporal properties than just a time window for a goal to be completed.
So once we have a kind of defined problem with these temporary extended goals, then you want to model it in a language like LTL or PDDL.
So if you have a language like PDDL, and you have a planner that can do an algorithm called progression, which I'll talk about a little later, you can go directly to a plan.
Basically how progression works is it's kind of an algorithm that tells you when you're analyzing a state, and you're analyzing LTL-like formulas that are true in a specific state, how to push formulas that you need to evaluate later on to the next state, and how to keep track of things that you need to continue to evaluate for satisfaction in future states.
Another way you can do this is by taking LTL formulations and translating them into Buchi Automata.
So Buchi Automata are basically finite state machines that are extended to handle an infinite sequence of states.
And I'll get into more about how Buchi Automata work.
After you have a Buchi Automata, you can then translate that into PDDL2, which you guys are all familiar with, and then that PDDL2 can be used from the classical planner, and get a plan.
So a little bit more about Buchi Automata.
Again, like I said, it's an extension of a finite state machine.
Oh, yes
Why would have a Buchi Automata when you could just use a planner that will go from PDDL3 straight to the plan?
I guess it just depends on how-- what kind of planners we have access to, what other systems you have.
It's done both ways.
So PDDL3 is still kind of new, and so there's not a lot of planners that actually have progression and can handle PDDL3
And it's also the case that the particular algorithms that are trying to either be verification or planning, maybe exploiting particular properties of the Buchi Automaton, as opposed to the properties of the native language
So you guys are all familiar with regular state machines.
Buchi Automata has a very similar formulation.
Some slight differences-- you have a set of states, you have an initial state, and you have a transition relation, and then you have a set of accepting states.
These accepting states are essentially what replaces finite states in the state machine.
We also have a set of symbols that are again simply the transitions between states.
So what an accepting state is, is a Buchi Automata can only be valid if the sequence of transitions visits an accepting state an infinite amount of times.
So I'll get into a couple examples.
So let's say you want to model the ticking-tocking of a clock.
This is just a regular finite state machine.
As you can see, after some number of ticks and tocks, you get into S2, and you're in S2 because you can be in transition only [INAUDIBLE] So you're in S2 basically forever once you get to S2.
So we can model this as a Buchi Automata by making S2 an accepted state.
This example, making it a Buchi Automata, doesn't really do anything for you.
It's just kind of to illustrate what the accepting state does.
So the accepting words are a sequence of transitions.
In this case it would be an infinite combination of ticks and tocks that would make this Buchi Automata valid.
Does that make sense.
Questions about that?
Here's another example, for example, if I have changed what my accepting state was.
So if our accepting state was S1, I now have to visit S1 an infinite amount of times for this Buchi Automata to be valid.
That means the only valid sequence of transitions I could have is tock tick tick tick tick for an infinite amount of times.
So you can kind of represent different things and different sequences of inputs you might want to have based on which accepting state you use.
This is another example.
If you wanted your clock to be tick tock tick tock tick tock, you can have more than one accepting state, and now I have to visit S0 and S1 an infinite amount of times if I want this Buchi Automata to be valid.
So then the sequence of transitions I get is tick tock tick tock tick tock.
Does anyone have any questions to really think about in general with Buchi Automata?
OK.
It might help to see how we model LTL as Buchi Automata.
So here's an example of of these LTL formulas modeled as a Buchi Automata.
I guess take a few minutes and see if you can get which one this one might be
[INAUDIBLE]
Yeah.
Just so that everyone can kind of see that, and future events, we remember that means that eventually p has to be true at least once.
So from any sort of input state, you go to S0, and then I do have amount p for some amount of time, but as soon as I have p, I have to get to p, because I have to be in my subject state an infinite amount of times.
So I have to execute p at some point to get to my accepting state for this Buchi Automata to be valid.
And then once I'm at S1, I can have any other amount of inputs.
I'll always be in S1, and that's what the true label means
And it's also something stronger, right, which is you could-- it's eventually globally?
Right.
Yeah, so actually in this case, after it gets here, my input could be anything.
So this basically just says that for my initial state, I have to execute p at least once to get to this accepting state, and once I'm there, I can be there-- I'm always there.
Any input will believe me in S1
So just to make this clear for myself and maybe for others, the data string executed that's at R state something that's on the loop.
If you go from s0 to s0, that's essentially saying that one of the nodes is not p. If you go from s0 to s1, it's saying that one of the nodes is p
Yeah
If you go from s1 to s1 and take whatever it would be because you set it true
So if you put p where the true is right now, then you you won't have to go through
Right.
That's what I missed.
By the way, you guys have 25 more minutes
So that's what we talked about.
This is future.
So the accepted sequence of propositions would be not to not p. Then you have p. Then it can be anything after you get to p. And then in this case, for Buchi Automata states, which are like [INAUDIBLE],, slightly different than the LTL states we've been talking about.
In this case, you're in s0.
And then once you get to s1, you're forever in s1.
So globally, looks pretty similar to this.
Take a few minutes to think about how I might change this Buchi Automata to [INAUDIBLE] globally after [INAUDIBLE]
[INAUDIBLE]
Yeah, exactly.
So you have initially, your first state by your n because p has to be true forever after you get to this first state.
You have an accepting state in p. You can also model not p. If you want to, you can also just have this single see which would encapsulate exactly what Buchi is.
This is how you go from LTL to Buchi Automata.
In a real-world scenario, we have multiple LTL formulas all combined.
You're Buchi Automata are going to look a little bit more complicated than this.
Try multiple accepting states.
There's a full weighted model of what an LTL is trying to represent
If we have the not p transition out of there, Like it's possible you can come into s0 to accept the Buchi Automata because it would go to s1, right?
Accepting a Buchi Automata after that, it's a little confusing.
A Buchi Automata is only valid if the infinite sequence of states satisfies it.
If I were to go the p instead of not p, my sequence would accept a transition of p, p, not p. Then that would not be a valid sequence of transition for this Buchi Automata because I'm transitioning out of my accepting state, which I need to have been in for at least the amount of times.
And there's no way I can get back to success.
Also, a more defined algorithm [INAUDIBLE] an intuitive way to build a Buchi Automata, or is an algorithm.
The other one that's most popularly used was developed by [INAUDIBLE].
This is a pseudocode version of the algorithm.
We go from LTL to Buchi.
As you've noticed, it actually uses progression, which is something that planners that take PDDL use to generate plans.
An important thing to note, which I won't go into too much detail, this algorithm generates a generalized Buchi Automata, which you then change to a simple Buchi Automata.
The difference is, the generalized Buchi Automata has a set of sets of accepting states.
For a generalized Buchi Automata to be accepted, you need to visit an accepting state in each one of those sets of accepting states.
At least one of those states has to be listed.
For a simple Buchi Automata, you only have one set of accepting states.
And you can visit any one of those for the Buchi Automata to be valid.
On the translation between those two is a little bit more complicated.
We'll have paper in the preference that will walk through how to do that.
But we're not going to do that now.
This is a progression algorithm.
Basically, it just tells you how if have a LTL formula f and some current state N-- and usually they involve some time step because we're in the real world and we can't really model infinite amount of time, and we have to make it discrete.
So we have some time step, which means successive state.
How do you push certain LTL formulas onto next states to be evaluated later?
For example, I'll take this next f as an example.
So if f was a next f of some LTL formula, you need to append that formula to the next state to be evaluated in the next state to see if that's going to be true or not.
Similar things for LTL.
And I won't go through this [INAUDIBLE] it's here for reference.
The next step to this process is going from Buchi Automata, that's a PDDL2.
This process is confusing.
At least it is confusing for me to understand.
The first thing I'll say is that Buchi states are not equivalent to traditional PDDL states.
In a PDDL state, if you have two states, you have the same propositions that are true and false, those states are identical.
In the Buchi Automata, that's not necessarily true.
In the Buchi Automata you also have to encapsulate which transitions you can make out of each Buchi state.
So s1 and s2 couldn't have the same set of propositions that are [INAUDIBLE] hold.
But out of s1-- actually, back up.
This is the Buchi Automata for FutureGlobally, which is what [INAUDIBLE] was talking about.
So if I enter an f1, the star means I can have any transition that I want in s1.
At some point, I make the transition p and whatever the s2, I have to take transition p forever.
However, if I get a sequence of inputs, let's say I get p and then b, and p again, and then p again, it's not clear if I'm in s1 or s2 because in s1 I can execute any kind of state I want.
In s1 I can execute any transition I want in s1.
2 I have to execute only p. So if I got a sequence of p's, I could be in either state.
You need something else to activate PDDL2 to basically to determine which state you're and which transitions you can make from that state.
Does that make sense to everyone?
So there's two ways we can transform PDDL2 to encapsulate what a Buchi Automata encapsulates.
The first thing is you can create new actions that can encapsulate the allowable transitions of each state.
So basically, for every action you have in your traditional PDDL, you have create a repetitive action for each one of these states so you know exactly which transitions you can take out of that state.
This can get long because you have to make a new action for every single one of these states for every action that you want to take.
That is one way you can do it.
Another way you can do it is introducing derived predicates.
Derived predicates are predicates that don't depend the actions at all and just depend on some other formula in your plan.
For example, we can have a derived predicate that explicitly tells you whether you're in s1 or whether you're in s2.
And you can use these predicates in your actions in PDDL2 to make sure, even though which state, x1 or x2, you're in in your Buchi Automata.
And then, in your formulation of your plan, you set your final state to be one of the accepting states.
I guess that answers one of the questions, though, how you go from being infinite to finite.
The accepting state is usually what you place as your final state
Then in the derived predicates, you're purely in the pre-conditions?
Or do they also appear in the effects
They also appear in the effects because you need to know how to transition out of s1 to s2.
So you'd likely have a derived predicate for both s1 and a derived predicate for s2.
And that is also a lot of work to add to the PDDL2 problem.
But it's definitely shorter than the first way.
And it definitely encapsulates everything, by the Buchi Automata [INAUDIBLE] Does anyone have any questions about that?
So you're changing the action then?
Yeah.
You're changing the action but you don't need make a new action for every state.
In the first formulation, let's say I have some traditional action, like move blue block, or something.
I have to create a move block action for each one of these states because I need to know exactly which transitions I can take out of that action
In the second example?
And in the second one I can just have one move block.
But in my move block I'll have a derived predicate that tells me which state I'm in when I execute that action.
The number of actions you have to write is less because you explicitly have predicates that are telling you which state that you're in.
Now I'm going to hand it over to-- if there are no more questions-- hand it over to Mark.
And he's going to talk about preferences
We're getting a little low on time.
Now I'll transition to talking about-- we talked about temporally extended goals.
We said that goals are things that always have to be true.
So now let's talk about preferences.
Preferences are things that don't necessarily have to be true, but are things that you'd like to be true.
In the classical planning problem, we can formulate it like that.
We can set a state S, set a state s0, a set of operators, and a set of goal states.
The problem is transition from the initial states and any state.
But those are goal states using the operators [INAUDIBLE] Preference based planning problem introduces a traditional field R, which is a partial or total relation expressing preferences between plans.
And this is a little preference symbol right there.
That tells you, if there are multiple plans you need to accomplish your goal, which one do you want to use.
And again, preferences are properties that are desired but not necessarily required.
There are two [INAUDIBLE] different types of preference languages, quantitative and qualitative.
As you would expect, in quantitative languages, we actually assign a numeric value to the different plans in order to compare them.
So these plans are a weight of four, and a weight of two, and that tells you which one to prefer.
Here are some languages that do that.
For qualitative languages, they actually just have a plan with this property is preferred to this other plan with this property.
But we don't actually know any information that knows how much we prefer one to the other, or something like that.
An important element of this is when you have quantitative languages, they're totally comparable.
Any two sets of plans you can determine which one is preferred.
Whereas in qualitative languages, you could have situations where plan one is preferred to plan two, and plan one is also preferred to plan 3, but that doesn't give you any information about whether plan two is preferred to plan three.
So it's a little bit less expressive [INAUDIBLE].. To go into how you actually express preferences in PDDL3, like we talked about temporally extended goals, there is a PDDL3 syntax to do this.
There's a preference label here that you can put on fluents to represent things you prefer.
And then there's a function call is-violated that essentially returns the number of times that any fluent that has a preference label was not satisfied in your plan.
For example, if you have a preference, "Traffic light is green until it turns red."
Here's our PDDL3 template.
It extended this kid's preference, in which we label it with a preference label here.
And this is just a name.
And then or plan tries to minimize the number of times that this preference is violated.
That's one way to express preferences directly in PDDL3.
So now we talk about LDP.
LDP is a different language that is quite expressive in terms of types of preferences you can represent.
It is a quantitative language, which means we're going to have weights for each of our plans to express our preferences.
We can actually express the strength of the preference.
So Goal A is preferred twice or three times as much as Goal B.
It's an extension of an older language named PP.
Here is the paper if you want to actually check out these details.
The formulas are constructed hierarchically.
I'll go through quickly how we actually construct these preference formulas.
The lowest level is called a Basic Design Formula, or BDF.
That just expresses our temporally extended proposition.
So this is just a straight up LTL, basically, that we saw in the first section of the presentation.
I'm going to use that I'm cooking dinner example for these slides.
In this case, we always use the future operators.
At some point, I might want to cook this program plan.
Maybe I want to order takeout to eat dinner.
And then I have some eating spaghetti or eating pizza.
Those are two different things, two different options that I could have in my plan.
I don't have to have either of those.
Those are just options that I could have.
That's the lowest level.
The next level is called Atomic Preference Formulas, or APFs.
There are where we express our preferences between the BDFs that we formed in the previous slide.
So in this example I'm using weight, specifically to represent preference, with lower weight being preferred.
Here we're saying, we prefer to cook over ordering takeout.
This is you where you have to cook, this is where you have to order takeout.
And we apply weights to those two expressions.
The first one's preferred, and how much it is preferred over this plan.
And here is you prefer to eat spaghetti over eating pizza.
So that's the second level.
Third levels called General Purpose Formulas, or GPS.
These are where we can do conjunctions or disjunctions, or qualification of our previous formulas.
So for example, if we want to say-- because in the previous slides we had two APFs.
We had prefer to cook and prefer to eat spaghetti.
And here we can express that we really don't care which one of these we want to satisfy.
You can think of this as we actually tried to satisfy APF at the lowest weight.
So if we have this "or" operating here, that means that our planner is going to try and satisfy the lowest weight among all these different options here, which means that he doesn't prefer to cook.
That's its first goal.
You can also use a handoff rigor, for example.
And that's going to minimize the maximum weight against both of those options.
Basically, what's it going to try and do-- oops-- what it's going to try and do is minimize the highest weight among these two options.
So it's going to try and cook, and then it's going to try and eat spaghetti versus doing these other options.
So those are GPFs.
I'm now moving on to Aggregated Preference Formulas, which are the highest level.
So with APFs, these define the order in which our different preferences are relaxed.
You can, of course, express a lot of different preferences for your planner, but not all of them may be achievable, especially if you're trying to achieve a large number of preferences at the same.
You may not actually be able to achieve all of those.
So how do we produce our set in the correct order, in the preferred order so that our plan we end up with isn't even the most preferred plan?
That's what APFs like to do.
You can express, using this preference operator.
First, I want to try and satisfy both of these in the previous slide.
Then if I can't, I want to relax it so I only satisfy G2.
And then if I can't do that, then I want to relax it so I only try and satisfy G1.
So that gives us the order in which things are relaxed.
It's important here that if you have these situations where you can't distinguish from one another, or you don't really care, we can establish some order.
So at the very lowest type of level, we can sort them alphabetically, or something, just to provide some sort of order.
This is just a review of what I've been talking about.
BDFs are the lowest level, which express temporally extended propositions.
We can apply preference to those using APFs.
Then we can combine or join APFs using tell preference formulas.
And finally, we can aggregate those preference formulas to determine the order in which you shouldn't relax the propositions to allow yourself to plan it right.
So using this scheme in LPP, we're using both LTL syntax and rules that we talked about in the first section to express temporally extended preferences.
The plans that's actually used to solve these types of problems is called P Plan.
P Plans can actually handle templates, and the preferences, and a goal at the end, as well.
And it does basically a best first search among all your different options.
It's actually one of the planner that uses the left branch.
But the one we showed uses progression to take LTL formulas at each point in the plan and determine whether you've satisfied those formulas or not.
And if not, it will push them to the next state, so that in the next state you can evaluate whether you've satisfied those formulas.
That's how it prunes the search space and tries to always find the most preferred plan to meet your goal.
All right.
So that's our presentation.
Any questions from anyone?
[INAUDIBLE] question.
I have a question about [INAUDIBLE] presentation.
You guys [INAUDIBLE] and then believes omega.
Would you say that omega has to hold if p happens, and they have a conjunction in one state?
Omega doesn't have to hold until we [INAUDIBLE].. Omega just has to hold at the last state of p. Essentially what it's saying is omega releases p. So once omega happened, p has to happen at the state.
And the omega has to release p. Now it happens, so p, you can don't you want.
You can be true, you can be false
Sorry.
So p has to hold until omega happens
Yep.
And then omega-- [INTERPOSING VOICES]
So the definition is just switched to p [INAUDIBLE]
This is an occasion.
Occasionally, you have pR omega.
But like intuitively--
It's just the definition-- oh-- [INTERPOSING VOICES]
That's all I was wondering because then I saw that and it was confusing.
So that's all.
OK, sorry
That's fine
Any other questions?
Since there are multiple ways to express the same formula using the grammar, is there any notion of canonical forms, or a least complicated form, or something like that?
Yeah, there is.
Typically they actually-- let me find the slide.
So to simplify the algorithms they typically apply the kind of reductions that we showed in the slide to reduce the release and the globally and the future down to just the [INAUDIBLE] of an x.
Otherwise, your algorithms going to be able to handle fewer cases.
And then they apply the usual rules, trying to push nots out to the left-hand side.
All right.
Thanks, everyone.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.MIT.edu.
PROFESSOR 1: So as he gives an introduction, myself along with Catherine, David, [INAUDIBLE],, and Charlotte [?
are going to present ?]
you guys Probablistic and Infinite Horizon Planning.
To give you a brief overview of what we're going to talk about, we're going to start with the Quadrotor motivating example.
We're going to move into planning with Markov decision processses, give you a little bit about value iteration before discussing heuristic guided solvers.
And we're going to go into the more stochastic case, partially observable Markov decision processes and operating in belief space.
So we very often now see quadrotor motion planning as a problem, given, for example, with the Amazon fulfillment center.
We start with a goal configuration, start configuration, set of actions we can take, and some type of reward function or cost function.
So for instance, if we have a quadrotor starting at the Amazon fulfillment center, and we want to get to 77 Mass Ave, and let's say we want to take the shortest path to get there.
We would follow the red dashed line.
But, as we can see, it comes very close to these obstacles.
So we're looking at very higher risk for mission failure, crashes.
If there's any uncertainty in its path, we're going to have a problem.
So one of the ways that we can compensate for that is we can also done a green path, which is adjusted to give us a little bit of space.
Are there any more questions?
So as you can see, the level of uncertainty allows us to determine how easy or difficult the problem's going to be to solve.
On the easier side we have deterministic dynamics and deterministic sensors.
In this case our actions are going to be executed as commanded, and our sensors are going to tell us exactly where we are.
This will be something like dead reckoning validated through sensing, very redundant.
If we moved to a little bit more at a difficult case, we have deterministic dynamics.
Our commands are still being executed, but maybe we have a noisy camera.
So we have some uncertainty in the sensors.
These are the cases where we would see dead reckoning, but we would compensate with Kalman filtering to get rid of that noise level.
Now, down at the-- yes?
Sorry, what is dead reckoning?
PROFESSOR 1: It's essentially-- you are saying I want to execute this option, and it's going to execute it exactly
OK.
PROFESSOR 1: Then towards the bottom, we have stochastic dynamics and deterministic sensors.
So in this case maybe there's some uncertainty in our actions.
But we can validate through sensing.
This is where we're going to spend the next section on Markov decision processes.
And then briefly, later in the lecture, we're going to talk about this most difficult case, which is the stochastic dynamics and stochastic sensors.
Our execution maybe, maybe not, our sensors maybe a little bit of noise.
And this is where we see partially observable Markov decision processes.
PROFESSOR 2: Can we make just a brief clarification of the dead reckoning?
So dead reckoning is where you estimate you position using a probabilistic file, but you don't use any observation stuffs.
That's the thing.
PROFESSOR 1: OK.
So we talked a little bit about action uncertainty, where we're going to focus.
And this is a case where, for instance, even if you tell a quadrotor to stay in its place and a gust of wind comes by, it's not going to stay in that same place, right?
It's going to have a little bit of movement.
And we can see, this causes things in disparity where you have a command of trajectory and your actual trajectory and then you need to associate.
So in order to compensate for that, we want to model things with that uncertainty or else we have these higher situations where we command to follow some line.
We don't incorporate the uncertainty, and we see a crash.
So this allows us to introduce Planning with Markov Decision Processes.
So MDPs have a set of states-- some actions you can take-- a transition model.
So essentially, what's the probability of reaching some state if you take an action?
An immediate reward function and a discount factor.
This discount factor is important because it allows us to prioritize gaining an immediate reward as opposed to an uncertain future award.
So the concept of, bird in the hand is worth two in the bush.
Now we want to find an optimum policy that will essentially back an action-- the best action-- for each state.
And what we hope to get from this is maximized expected lifetime reward.
So we want to maximize the accumulative reward we get over the times.
So let's walk through an example.
If we have a quadrotor with a perfect sensor, and let's put it in this environment [INAUDIBLE] 7x7 bridge.
Our set of states are obviously [INAUDIBLE] space in them.
Can anybody tell me what some of the actions might be?
[INAUDIBLE] PROFESSOR 1: (LAUGHING) Yep.
Up, down, left, right.
In this case, we call them North, South, East, West, or Null.
The next thing we need for this example-- arbitrarily gave ourselves a transition probability.
So we said that you have 2% chance of following your commanded action with a 25% chance of moving to the left or the right.
Next we have the reward function.
And again, we arbitrarily decided that we want it to be a reward.
If you get to the state 6-5, the increment [INAUDIBLE]
Alicia, you had [INAUDIBLE] left or the right, is that clockwise or counter clockwise?
Let's say you had planned to go to the right, would that mean you have a 75% [INAUDIBLE] or 25% chance of [INAUDIBLE]?
What if you just waited [INAUDIBLE]??
PROFESSOR 1: We said clockwise, counterclockwise from the intended direction of action
Thanks.
PROFESSOR 1: Uh-huh.
And finally, we give ourselves a discount factor of 0.9.
So let's assume for a second that we have our optimal policy.
And let's say that our optimal policy says, from this state, we want to take the action North.
Right?
As we discussed, we have a 50% chance of going North and 25% chance of going to the left or right.
So after that time step, these are possible states that we could end up in.
Right?
So now let's assume for a second that we can take our next action.
And our next action says, go North.
Again, we have the same probability distribution.
And these are the states we could end up in after two time steps.
We can see that this starts getting very complicated, right?
And there are increasing amounts of uncertainty.
So does anybody have any ideas on how we could collapse this distribution?
Keeping in mind that our senses, at this point, are deterministic.
Yep
Fly to a corner?
PROFESSOR 1: I'm sorry
Fly to a corner?
PROFESSOR 1: We could do that
You just sense how far away the red box is.
PROFESSOR 1: We could do that
Quick comments.
So the blue state is not actually one.
PROFESSOR 1: I'm sorry
So the problem is you're not actually one
Yeah.
The issue is if you went to (1,3) and then you transitioned to the left, that's where the 0.0625 is coming from
[INAUDIBLE] then, shouldn't those numbers always are actually 1-- your probability distribution always are actually 1?
PROFESSOR 1: Yep
So it's just a point.
It's just off the screen.
PROFESSOR 1: We would add another section to the screen and just move the grid over.
We just cut it off for graphics
OK.
PROFESSOR 1: Yeah.
So those are all great points.
The easiest way to do it is just take an observation.
So at this point we say, after our first time set, we weren't sure which of these three states we were in.
So we took an observation and said, wait, we're actually here with complete certainty.
So to make this a little bit clear, we're going to look at it from a tree view.
Right?
We said that we started at a state.
We took an action.
And this is our possible states we could have ended up in.
We're going to collapse this by taking this observation.
And now have a complete study here.
And we take our next action and see that we have moved out here.
So this allows us to basically ignore the history of states.
We have the same percentage probability from each time set.
This will be really useful in completely collapsing the distribution every single time you take an observation.
Anybody have any questions on that?
OK.
So let's go back now and figure out how he came up with his optimal policy.
They way we do that is through dynamic programming.
There's two different ways.
You can do it either through value iteration or policy iteration.
For this lecture, we're going to focus on value iteration.
So let's take this same example we had.
We still want to maximize the expected reward.
And so to start, we're going to initialize the values of each state to 0.
Let's for example, start at (2,0).
We're going to focus on suite 6-5.
And we're going to say that we're going to take the Null action to start with.
From there you can see with a probability distribution 4, 6, 5, we have a 50% of staying.
We have 25% chance of going to 5-5.
And a 25% chance of going to 7-5.
The next item of information is the values at each of those states that we could end up in.
And currently, they're all set to 0.
And finally, the most important part here will be with the reward [INAUDIBLE] 6-5, taking the null action, regardless of what state you end up, is going to be 1, as we defined in our initial set up.
So let's see how we would calculate the next time step value of the state.
You'd start by taking probabilities, right?
And from there, we would add the reward here.
And because we said that the reward does not dependent on the state we end up in, [INAUDIBLE] should be across all three probabilities.
From there, we're going to add in the discounted value that we had from before.
So a way to look at this in a more generalized form is to say that across all the states that you can end up in, you're going to look at the probability of ending up in that state.
You're going to multiply that by the sum of the reward and the discounted lifetime value that it has.
So we want to make sure that we're getting the best possible values.
So we need to incorporate all the other actions that we can take from that state.
So what's going to happen is we're going to take that general formula, we're going to repeat it over all of the possible actions.
And then we're going to take the maximum of that.
So, for this example, the state is very easy.
All of them are the same for this case.
But we go fast, and we say we get a value of 1.
And we update it by showing the graph.
This gives us what's called the value [INAUDIBLE] Backup-- or Update equation.
This will be really important because it reaches across the entire states space and allows us to provide a history.
So what this would end up looking like is we're going to iteratively calculate the values across the entire state space So at t0, we determine that all the values are 0.
At t1, (6,5) gets a value of 1, and at t2, we see that value start to propagate out.
Make sense so far?
So the way this works is you would repeat those iterations until your changes in value become what we would consider small enough, which would indicate your approximation is close enough to the real value.
From there, you would extract the optimal policy from the lifetime values.
So you see the [INAUDIBLE] in the Bellman equation.
And you're now just taking the action from it.
And you would map those actions to your states.
An example of what this might look like propagating out-- if you say blue was the reward and red is an obstacle, et cetera-- you can see, as that value propagates out, you start seeing your policy by the arrows.
Anybody have any questions?
So one last thing to mention about this, though, is the complexity for each iteration is dependent on the size of the state space squared and the number of actions you can take.
So you can imagine, as your state space expands or you gather more actions it's very complex, in which case time and value depiction becomes very time intensive and costly.
So this allows us to move into the Heuristic-Guided solvers
Thank you.
[INAUDIBLE] transition, one quick question.
Can I just get a show of hands-- how many of you have learned value iteration before versus how many of you have seen it for the first time?
So how many have seen it before?
OK. And then, how many is this their first time?
[INAUDIBLE] PROFESSOR 3: Any questions on value iteration before we jump in?
It's going to be an essential part of how we're going to do [INAUDIBLE] solvers.
Anyone who hasn't [INAUDIBLE].
So the most important thing we said about value iteration is that it's super slow.
It's going to have to go over every possible state and every possible action that it can take.
Our state space is multi-dimensional, and we can take a lot of different actions.
That going to be really costly and really hard to [INAUDIBLE]..
So the approach we're probably going to want to take is using some sort of best first search applet?
Who can tell me some example algorithms that already do that?
A star.
PROFESSOR 3: A star.
So that's very good.
And that's exactly what we're going to base our stuff off of.
The A star is for deterministic graph search.
If we have a graph, we can use it heuristic, and we walk down it and search the space that we're most interested in until [INAUDIBLE] variable.
So going to introduce to new items focusing on the last one.
AO star is an algorithm we can use to search for graphs that have this "and" problem.
AO stands for And Or graphs as opposed to the simple graphs that we have [INAUDIBLE].
The And is a way to express probabilistic coupling between edges.
So if we explore one thing, we might have to explore other functions.
We'll discuss that a little bit more in the next slide.
LAO is a bit more of a generalization.
What it does is allows us to search loopy graphs and deal with the probabilistic coupling.
And allows us to search and find the best path with a heuristic guided algorithm over infinite time horizon, possibly revisiting states using the tools we just had-- valued iterations-- to understand what the next best thing to do is.
So we'll talk about exactly how to get our MDPs that we just saw, these little arrow examples to these And/Or graphs.
We'll talk about two cases.
One simple one where we could apply the AO star algorithm is where you have a qualicopter.
And if you command an action North, you have a high probability of going North, but you might go East.
And vise versa.
If you go East, you might go North.
This is expressed here with the growing tree.
And these And edgers that connect it.
But despite the action of reading my command from 0 to 0.
We might end up in (1,0) or (0,1).
Right?
As we propagate outward, you can see how we're never going to loop back to a statement we've been to.
We're going to be moving in the Northeast direction constantly.
Our [INAUDIBLE] and our probability distribution across this tree explands.
And that's the coupling of the edges.
Because as we explore down the tree, we have to explore all the edges together.
Anyone have any questions on just this kind of conversion formulation?
I have a broader question.
We mentioned that MDPs deal with finite states.
Do we always just discretize a continuous row into a set of planning states?
PROFESSOR 3: Yes.
That's our prerequisite for searching over the state space.
We can do that as finely as possible, but yes, discretization is there.
So now let's look at this case.
Instead of the Northeast, let's say it's not deterministic whether we command an action north or South that we go north or south.
Here we see this loopy structure begin to emerge.
We see that we might-- on our first action commanding North, we might go to plus 1.
And then commanding North again, but we have our And edge and our probability distribution is [?
back ?]
to 0.
What this does is this loopy structure.
And this is exactly what we're going to be exploring.
This is a very real world scenario where it's a very likely we might return to somewhere we just came from, just because of the uncertain dynamics we have.
This is the type of problem [INAUDIBLE]..
So we're going to use the LAO to start out with.
We're going to talk about the three main things that [INAUDIBLE] has a heuristic guided envelope.
And what that means is that we have our large state space here.
But we're only going to look at a portion of it.
This greyed-out portion.
We're only going to look at the portion that is interesting to us-- the portion that provides us with the biggest rewards-- the portion that's reachable if we follow an optimal policy.
We figure this out with some admissible heuristic.
We'll estimate our rewards just like A star.
The idea here was that we'll keep the problem small so we don't have to search the valuation for a giant state space.
What we'll do next is at the state space expands, we get a bigger picture of the states that we're interested in.
We're going to do [?
an audio ?]
[INAUDIBLE].. And then we're going to figure out what the best action is.
And in the ideal case, the states that we would never reach using an optimal policy are never going to be explored.
Because our policy is going to say, no don't go over there.
That's a dead end.
Or that's getting you farther away from [INAUDIBLE]..
So we're going to be searching in a very specific part of the state space that is useful to explore-- that will get us closer and give us a higher reward.
What's important here, the L stands for Loops.
It's an extension of the AO star algorithm, which is by itself is an extension of the A star algorithm.
We can handle infinite horizon problems, like transition in different ways.
And really models the real world scenarios we're interesed in.
Any questions so far on the broad scope of what AO star is going to do?
Yeah
Can you put the [INAUDIBLE] what you're doing over here, but-- PROFESSOR 3: Sure.
The AO stands for And Or graphs.
So that's graphs that we have here edges that are coupled.
If you took time to do this transition, you might end up here or you might end up there.
So that's the notion that of this probability coupling, that we'll see in action as we [INAUDIBLE] we're going to do, we're going to input this MDP or AO graph with transition probabilities or reward function heuristics.
These are all things that we defined prior to figuring out a plan.
What we're going to come out with is an optimal policy for every regional state.
It's a little different than what value iteration is, which is an optimal policy from every possible state.
But we argue that that's all we need.
If we know where we're starting and we follow our optimal policy, we're only going to explore a certain portion of the state space.
And we're going to explore that together.
[INAUDIBLE] plan a little bit more efficiently than iterating for a high [INAUDIBLE] heuristic.
Any questions on that?
So we'll define some terms that we're going to use throughout this.
We've already talked about our state space.
This is just a small portion of it that we're going to work with as we're going to walk through this example.
Next we're going to define something called our envelope.
And that's the sub-portion of our states that we're going to be looking at.
We're going to initialize that to just this zero [INAUDIBLE]..
But as we progress through the algorithm, it's going to grow.
It's going to grow only in the areas that we're interested in.
A subset of this envelope is the terminal states.
Now these aren't goal states, these are just [INAUDIBLE] of our space that we've added to our envelopes.
And this is what it would be in the expanded [INAUDIBLE].. And it's just the nodes that haven't been expanded yet.
Here they haven't drawn anything incredible but you can imagine the state space goes out further because [INAUDIBLE].
You can imagine that this goes out further.
And they're keeping track of the nodes that we haven't expanded yet.
Or likewise, we've showed that we initialize [INAUDIBLE]..
The other few things that we're going to define-- we've already defined the states that are in our envelope.
That's the blue or the red.
We're going to define cost heuristic or reward function, R E, and a transition probability matrix, or set of matrices, that we're going to use our optimal policy search on.
What's important here is that our reward function and transition probabilities are slightly altered to account for the fact that we haven't explored the entire state space.
We see here that if a node is in ST, in other words, it's one of those terminal nodes.
We're going to say we can't transition out of it, because we don't know what's beyond it so far.
And we're going to set it for reward to be a heuristic.
Whenever we think that the reward is going to be when we begin to explore and go further.
Like I said, we're just going to feed this into a policy search just like we discussed with value iterations on the sub problem.
And we're going to search for an optimal policy [INAUDIBLE]..
So far so good?
This is the general steps.
This is very text-y, but we're going to definitely walk through every single step.
We're going to do two full iterations of the algorithm.
So like I said, we're going to create RE and TE.
That's are reward function transition probabilities using the definitions we showed.
We're going to use value iteration to find the optimal policy of the sub space that we're interested in now.
And then this is probably the most important step.
Knowing this optimal policy, we take a look at what new states that aren't in our terminal states-- nodes that we haven't explored yet-- what new states we might visit now.
So let's say we have our policy, and it says we'll go North.
And we know that we haven't explored the North state yet.
We know this is the state that we're going to reach following what we consider now to be already an optimal [INAUDIBLE].
So that's the states we're going to expand next.
We're going to do some bookkeeping, adding them to the terminal states [INAUDIBLE] once we expand it, then adding them to our envelope.
What's important here is that we're only going to add states that we visited yet to our envelope.
And this is basically the little hack that allows us to deal with loopy graphs.
We're not going to continually explore nodes that we might reach a second time, probabilistically.
We're going to let value iteration handle that [INAUDIBLE]
[INAUDIBLE] states are expanded.
I'm the one who got confused on that.
Are you saying that you're going to just repeat until there aren't any more terminals to look at.
And if that's the case, how is that possible if you have an infinite horizon [INAUDIBLE] PROFESSOR 3: Sure.
So if you can imagine-- and we'll talk about termination at the end.
But you can imagine that as we have these terminal states, but you have a policy that guides you to a part of the state space that we've already expanded.
Imagine you've reached your goal.
Your optimal policy is going to say, stay put.
Right?
And it's not going to say, move North again
It's just the goal [INAUDIBLE] PROFESSOR 3: Essentially, the goal state is definitely an example in the more extreme case that there's nothing else you can do that's going to get you closer to our goal.
The idea is that your policy on your sub space never tells you to go to a terminal.
Nobody can [INAUDIBLE] inherently worse than [INAUDIBLE]
Planning optimal policy means running value iteration entirely.
PROFESSOR 3: Yes.
We were going to treat it essentially as a black box.
But the trick here is that we're doing it on a smaller portion of space of a different world.
All right.
I'm going to put the steps up there.
Hopefully you can see this.
But for now, we'll just walk through from the beginning of what we're going to do.
So we said that our envelope and our terminal nodes are just at 0 to start.
So very simply, we use these definitions and say, OK, the transition probability as 0 to any node right now is 0.
Because it's in that terminal.
[INAUDIBLE] That's just so that we don't transition out of it.
We can develop the policy based on only this portion of the space [INAUDIBLE].
And our reward function, we're just going to apply it to be the heuristic [INAUDIBLE].. Let's say that's 20.
And for a move-on from there.
[INAUDIBLE] started to value iteration.
And we'll run this value iteration.
So this is a very basic case.
We're just going to-- we're using this to kind of build up the machinery, understand what we're doing.
It's a very basic case where you only node we have is that zero.
We can't transition out of it.
All we have is some heuristic.
So the only thing we can do is nothing.
So the action we're going to take from S0 is nothing.
Very simple case just to get us warmed up and understand what's happening.
So using this policy and knowing that those 0s in our terminal node are node reset.
We're going to take the intersection between this terminal mode set.
And the nodes that we might reach following our policy.
And we know that we're at S0.
And we know that action that our optimal policy says to take is not.
So we know we're there.
And we know it's in our terminal node set.
So that's the only thing that we could reach so far using our optimal policy.
So far so good?
Clear?
So that's exactly what we're going to expand.
Just expand S0, A, B, and C defined those to our terminal nodes.
So that's up there the symbols [INAUDIBLE] from our terminal nodes added as children.
Then we added the children to the end.
Here we go.
We're going to do a little bit more.
We're going to do the same thing again.
But now we obviously have more nodes and a bigger sub space to explore.
So using these definitions we see our reward function is a tuple of the node that we start from.
And so this S0.
And one of these three actions.
And we'll take A1, A2, and A3.
When we have our instantaneous reward functions, 6,4, and 8 as being our rewards from doing those actions.
From A, B, and C, no matter what action you take, it's just heuristic.
That's part of it.
And likewise, we're going to take a look at this transition probability [INAUDIBLE].
This is for [INAUDIBLE] transitioning from a state S0 for saying that if we take actions A1, A2, and A3, what's the probability of being done in nodes A, B, and C?
If you look so far here, we're going to look at something deterministic.
If we take an action, we'll end up where we say we're going to end up.
And we'll see how this algorithm collapses down to A star if everything's deterministic.
We're also obviously going to look at the probabilistic case where we say [INAUDIBLE] small probability that it might end up with [INAUDIBLE].
That's going to necessitate that we're going to have to look at and expand B together with A if we were to decide we want to try [INAUDIBLE].. And likewise, if we try to take action A3, we have to expand them all three nodes.
So the tighter the probabilistic coupling, the more of the space we're going to have to explore.
So just off this, assuming that if you take action A1, A2, and A3, can someone tell me what the policy is for the rewards here?
And we're interested in a policy from S0 what we're going to do.
Take a look at the rewards and judge what the best action to take is from a purely deterministic sense
[INAUDIBLE] do you add [INAUDIBLE] or do you just [INAUDIBLE]?
You have the reward that you have gotten so far.
PROFESSOR 3: Does that help you answer the question?
Oh, yeah.
[INAUDIBLE] student.
PROFESSOR 3: So the policy preference says, from S0, take action A1.
And that's going to stay there.
[INAUDIBLE] from A to C, there's no action to take [INAUDIBLE].. What that means is that the nodes that we might reach that are in our terminal state set, using our policy, is node A.
All right.
So that's the node going to expand.
And this is where you really see that all we've done is collapse down to A star.
A star would say, OK. What's the best node using some heuristic.
Action a1 one takes us to that best node, and we're going to expand just that.
So when everything is deterministic, basically this algorithm collapses down to A star.
Nothing super interesting.
The interesting case does come up when we start doing more probablistic ones.
That's where nodes are probabilistically helpful to the scanned graph sense.
So now we have our policy.
The policies are politely going to remain the same, because they have very little probability on the edge actions that we might accidentally hit up.
What would I want to look at is what [INAUDIBLE] and how reachable following our optimal policy.
So we talked about that if we were to actually read.
We know some probability of ending up in [INAUDIBLE] nodes.
So taking action A3 makes C, 2, and A all reachable.
What's reachable in taking action A1?
A and B.
PROFESSOR 3: [INAUDIBLE] And that's where we get the notion on this probabilistic algorithm that we're going expand things together.
Explore the part of the state space that we reach both higher optimal polity and that we might accidentally end up in if we [INAUDIBLE] the policy.
And this is what guarantees that we'll have an action to take from any state that we intended to go to or that we might end up [INAUDIBLE].. That's how our state expands, our envelope expands to encompass only the reachable and interesting states that we want to look at.
It's not as simple as just examining not only the heuristic, but with A start because we have this probabilistic [INAUDIBLE].. You can see that if you have a tighter coupling, then you don't get to exploit this optimization as much.
For example, A3 we would have had to expand more as opposed to A1.
We can just stick to A and B and ignore expanding C for now.
[INAUDIBLE]
Our [INAUDIBLE] We know that when people do the statistic [INAUDIBLE].
So in this case, if you take action a because of a and b, [INAUDIBLE] PROFESSOR 3: So in the complete sense of running this algorithm, we shouldn't print it.
Because what if we do?
If we have that 2% chance, [INAUDIBLE].. We need to have a policy for correcting.
So we end up at B.
You can imagine that these are going to try to push back to whatever path that A was looking for.
I suppose that the probability is low enough, you can have some cutoff percentage where you've decoupled [INAUDIBLE]-- PROFESSOR 2: So just a quick point.
So I think that's an excellent point and an excellent answer.
We're going to talk about next week-- exactly what's going to happen and if you can prove lower probability on the paper right here will be lecturing on that.
Good question.
PROFESSOR 3: That also gets us into the sense that if every state in the world were probabilistically coupled-- let's say we had some transporter to go with the Star Trek examples.
If we had this transported that non-deterministically put us in any state in the world, we have to explore the whole world, because we could end up [INAUDIBLE].
So luckily that's not the case yet and we can take advantage of the fact that we ended up most likely where we commanded with some probability of [INAUDIBLE] This is exactly what we just talked about.
We coupled these nodes with this And edge.
And we expand those, too.
Is everyone understanding the holding intuition, and the logic for why both of them have the [INAUDIBLE]??
Even if there's only a small probability that we end up [INAUDIBLE]?
So and this is what we're going to repeat until the states are expanded.
You can imagine that the next time we run our value iteration, we're now running it on all of these colored nodes-- both the blue and the red.
We can imagine that next time, now that we have a little bit more information what lies beyond A and B, that our policy might say, oh, you know what?
Actually from S0 C, action A3 was the best to take.
What that does is say, OK, with a regional map, it's A, B, and C. And we expanded those nodes.
We'll we've already expanded A and B, so we move into the next part of the sub space, that as we gain more information, we run value iteration and we can expand on why.
Steve asked a good question.
When we did our dry run, about whether there is a way to save on the computation you did prior to the value iteration and add these [INAUDIBLE] states.
And we've looked at it a little bit.
I've seen some stuff.
But I haven't found a paper that specifically deals with it.
You can imagine how you've already run value iteration on your previous iteration [INAUDIBLE] and you add the new terminal edges which you'd expand on.
And you run them again until it stabilizes.
And that way you've saved the computation of having to run valuation multiple times [INAUDIBLE] state space
So I'm trying to think of how this is different from something like [INAUDIBLE] for [INAUDIBLE].
PROFESSOR 3: I think I don't know enough about that.
But my basic understanding says that what's useful here is it's this explicit [INAUDIBLE]..
I don't know how much [INAUDIBLE]
And also, as long as your heuristic is admissible, it's guaranteed [INAUDIBLE] not all [INAUDIBLE] algorithms.
Just like A star is optimal, as long as you've got a consistent [INAUDIBLE].
PROFESSOR 3: All right.
So that's definitely the idea here.
We coupled these in the only explored of the portion of the state space that we'll reach an optimal policy [INAUDIBLE]..
So we'll talk about quickly another determination.
We've touched on most of this.
So it's most likely when there are no more states to expand this when we've reached our goal.
It's when our policy that we run on our entire envelope from value iteration doesn't say that we should go to anymore terminal [INAUDIBLE] that we haven't looked at yet, that we haven't seen yet.
We've said that those are only the things that are reachable and needed.
Both, reachable because we're following optimal policy, and needed, only if we might accidentally end up in the probabilistic.
This is gives us the sense of rigorous that we get policy on the entire state space that we can end up in following the optimal policy.
The third bullet here touches on if we don't expand the states that are probabilistically coupled and we do accidentally end up there, we risk getting lost and not having a policy.
We can compute this all off line and have a plan before we even start planning to know exactly where we want to go even if our dynamics aren't [INAUDIBLE].. We're come back to this.
This was our motivating example.
And so we show that these real platforms can be modeled stochastically and then we can pretty easily deal with that.
Search our state space and deal with those probabilities and expand the nodes that we might end up.
Right?
And the heuristic allows us to not have to explore these areas of state space.
I never actually end up there.
We'll always be a commanding toward 77.
We're never going to try to command backward.
Sure, if there's a gust of wind and we have some probability there.
But you can imagine that we're going to only explore a small portion of this.
Because we'll always be trying to correct to get back to the top four blocks.
And using our reward function, we get to determine if we want to fly a quick path or if we want to fly a safer path.
For example, our time times our probability [INAUDIBLE] we want to perhaps reduce that.
All right.
Are there any questions about planning with MDPs, anything like that.
I love this stuff.
So the more questions, the more I get to talk.
Fine.
What I'm going to be talking about for the rest of this lecture is extending beyond MDPs to a broader class of problems called POMDPs, Partially Observable Markov Decision Processes.
I love this stuff.
I think it's really cool.
They're really fun problems.
We're going to talk about why they're so much harder to plan with, to execute-- but why they're important to at least know about so that you can model real world problems with them.
And then we're going to delve into a case study of a specific POMDP solver.
We're not going to go into as much detail as we did for MDPs, but we're going to look at what powerful results we can get by planning with POMDPs.
PROFESSOR 4: So first, I want to rephrase this in the overall talk.
Right?
We have this spectrum of uncertainty.
And coupled with uncertainty is difficulty of planning, of solving, of executing a problem.
And we've killed these first two cases.
That was really easy.
And then we just discussed the bottom [INAUDIBLE],, MDPs.
What I'm going to talk about is the case where both your dynamics and your sensors are stochastic.
Why is that important?
It's because when we first saw this slide-- our motivating example slide, we only saw the left hand side.
We said, our actions are uncertain.
But good news, we have a perfect sensor-- a perfect camera.
But that's unrealistic.
I think, we have all, to some extent, experienced the fact that no sensor is totally perfect.
Your camera might have fluctuated pixel values.
Your laser range finder is going to never read out exactly the right number all the time.
You can have a camera in different lighting conditions that will behave differently.
You might not be able to observe your full state.
That's, in a way, an imperfect sensor, right?
If I'm in this room, I have imperfect eyes.
I can't map out all of MIT's campus because I'm blocked by walls.
How can you deal with the fact that you can't see all your obstacles all the time?
We've already talked about some cases-- that there are some algorithms that can help us with that, like D Star Lite.
But can you reason about these things probabilistically?
And then finally, you might be in a non-unique environment where you cannot resolve your state with certainty no matter how good your sensors are.
Imagine you're in a building with two identical hallways.
You're dropped off in one of them.
How can you figure out where you are?
You can't unless you start exploring.
And so we've got to deal with this uncertainty, right?
it's Part of every single problem.
When observational uncertainty is slowing, you can maybe ignore it.
But it's there.
And so we're going to formulate this as a POMDP, a partially observable Markov Decision Process.
And this next slide is just like the MDP slide.
Hairy, but important.
Right?
We can formulate a POMDP, which is seven elements.
And MDP too five.
Most of them for all those are carried over here.
We've got our set of states where we can be.
We've got a set of actions what we can do.
We've got our transition model which says, given that I started in one state and then I took an action, what's the probability I end up somewhere else?
And like David was talking about, hopefully that distribution is pretty local-- we're not teleporting all over the world.
We've got our reward function.
This is exactly the same as for MDPs.
And we've got our discount factor down here.
The key difference of POMDP is these two elements.
We've got a set of possible observations and a probabilistic model for the probability of making an observation given your state and the action you just took.
Now it's important, I think, it matches up really well with real world sensors having this probabilistic model.
If you have a laser range finder, for example, and you're standing one foot away from the wall-- now a perfect sensor would always say, you're one foot away from the wall.
You're one foot away from the wall.
Every single reading is constant.
But realistically, there might be Gaussian noise, for example.
Or at a more extreme case, it says, your one foot away from the wall.
You're two feet.
You're right there.
There's this distribution.
And so you would ideally characterize this distribution.
And you plug that into this model and that formulates your POMDP.
This sounds really hairy, but if you work through just a sample iteration of living in a POMDP world, that's not too bad.
You start at some state, S. You take an action, A.
With some probability, you're going to end up in a bunch of different states based on your transition model.
At that point, we can use the lessons we learned from MDP land where we said, when we make observations, we reduce our uncertainty.
we collapse into a single state.
So we say, let's make an observation.
But this time, observations aren't guaranteed to resolve all our uncertainty.
So we make an observation.
And that observation is probabilistic based on our current state and the action we just took.
And again, obviously, it depends on your current state.
Because if you're one foot away from a wall, hopefully you'll get a different characterization of observations than if you're 20 feet away from the wall.
Otherwise, your sensor it's totally useless.
Are there any questions about this formulation?
Yep.
Quick question.
So we will take observation, any other observation and then you try to infer which state you're in, is it just a clustering problem?
For instance, the multi-cluster Gaussian [INAUDIBLE] models.
So class A, class B, class C, which are state, then taking an observation there's a high probability [INAUDIBLE] This is what we're tyring to do for each observation over here.
So we're trying to find clustering [INAUDIBLE]..
PROFESSOR 4: So you could, I imagine, implement an algorithm where, yeah, every time you make an observation, you then try to say, all right.
What's my most likely estimate, or maybe my [INAUDIBLE] least cost estimate?
But inherent with that is the risk that you're discarding a lot of information, right?
Because you're going to generate a probability distribution over your state.
And so, yes, you can say, I'm going to stick with the maximum likelihood estimate.
But if you can, you should probably try to maintain that distribution as long as possible.
OK. And we'll see that this is really computationally expensive unless you start making some assumptions.
And in the case study we're going to look into, that's exactly what [INAUDIBLE].
But I've seen a lot in the literature that as much as you can, you want to maintain these distributions for improved accuracy.
Any other questions?
All right.
Well, we're going to compare now the execution of a POMDP to the execution of an MDP.
We started out-- we're living in the same real world.
We've got our same transition model.
Everything is peachy.
We take action one, and we want to go North.
We have this distribution that we generate the states.
And at this point, what did we say?
We said, we hate the fact that we have to deal with three cases.
Three is two too many.
So let's make an observation to collapse this distribution.
Now I've described a lot about noisy sensors, right, where basically it's a true measurement plus some noise, maybe Gaussian distribution.
There's another partially observable sensor you can have in the POMDP which really feeds into the name, Partially Observable.
What if you can only observe part of your state?
For example, if you're living in an x-y grid, maybe you can only observe your y dimension.
This match up, in a real world example, to quadrotor flying down a hallway.
Catherine was working on a DARPA project with quadrotors flying down hallways.
If the hallway is too long, your laser range finder isn't going to be able to determine where you are along one axis.
But it can tell where you are along another.
And so that's what I've said.
I've said, pretend this quadrotor has that sensor.
We can only observe its y component.
And it says, my y component is 3.
I have no idea what my x component is.
Well, this sucks.
Right?
Because we got rid of this state, but then we couldn't decide, are we at state (1,3) or (3,3)?
There's no way of resolving this.
So we can re-normalize.
We can add in the effect from the observation probabilities saying, maybe, in fact, I'm far more likely to observe a y component of 3 if I'm at (1,3).
But in this case, we say it's equally likely to make that observation for those two states.
And so now we've got to deal with these two cases.
And so we can take our next action.
Instead of resetting to a single state, we've got to keep growing this tree.
And what's the key difference between this and when we were executing our MDP?
Except we didn't manage to collapse back to a single state.
We didn't manage to reset the problem.
And this is annoying.
Because you can't execute a policy and say, I'm certain that this is my configuration.
So your policy can't map from exact states to actions, because you never know your exact state.
Does this make sense?
Has everyone lost hope in planning, right?
Yeah
So in here because-- so from the left to the right, you're basically mapping from one belief state to another belief state.
So it's like a one arrow thing.
But and then from that second layer, you should have two arrows.
One with probably 0.5 that observes 3, and it gives rise to that belief state it just showed.
In one, with probability of 0.5 where the sensor is being 4.
Because if you have a 0 probability of looking at (2,4), you might as well just observe 4 as well as your y.
So in this case, I'm not sure if you were just trying to show one branch of your POMDP planning, but basically what you have to do is you would go like this to the second layer.
You would have a branch with a 0.5 probability on either.
One giving you 3, one giving you 4.
For the one that is showed, you've got that relief state.
For the other one, you get the state (2,4) with probably 1.
PROFESSOR 4: Yeah.
So you were perfectly describing planning, right?
I should have made this more clear.
This isn't planning.
This is executing.
We have this policy that we're going to execute.
And so if we were planning, we would have to consider all these branches and say, well, yeah.
There's a 50% chance here I'll end up here, in which case, my y value is going to read 4 and you have to grow this whole thing.
I'm saying, no.
This is real time execution.
yeah.
Great question.
Any others?
Well, this is a great time to transition to, well, we can't just magically be handed these policies.
How do we actually generate them?
How do we start planning in the belief space?
The belief space is the space distributions of possible configurations.
So I'm going to talk about a general class of algorithms.
A lot of planners in POMDP land and the belief space plan with probabilistic roadmaps- PRMs.
The goal is to generate a policy that maps from a belief state to an action.
And I'm going to go into a little more what a belief state is.
But the general algorithm in this graphic illustrates these four steps.
We're going to sample points from the configuration space, as if everything was deterministic.
We're going to connect those points to nearby points.
And define nearby however you want.
It could be your closest neighbors, all neighbors within a radius, whatever.
As long as those edges don't collide with obstacles.
Once you've done that, somehow-- and there's some magic in this-- you're going to transform your configured action space probabilistic roadmap to a probabilistic road map in the belief state.
Great.
Once you've done that, you can just do shortest path depending on whatever cost function you use.
And what's really cool is you get different paths for when you stay in the configuration space and when you go to the belief space.
So the green path, that bottom right figure, seems a lot longer than the red path.
And the reason is the green path was planned in the belief space and followed a lot of landmarks that the quadrotor could take measurements off of.
So it was really confident about its position whereas the red path is the shorter path that had a higher likelihood of a collision.
And we're going to look into that figure more later.
I'm going to segment this algorithm into two parts.
The first part-- these first two steps-- it's just probabilistic road maps.
Who here has heard about probabilistic road maps?
Raise your hand.
OK. 50/50.
I'm so excited for the 50% who haven't heard.
One of the top algorithms, in my opinion.
It's really simple, and it's really powerful.
Here's basically almost a complete implementation of probabilistic road maps.
It's pseudocode so don't copy paste, but it's almost there.
You're going to construct a graph.
You're going to add your start goal.
Your start configuration being the green dot and then goal configuration, the red dot.
And then you're just going to keep sampling notes those from the free space.
You say, how about (2,3)?
You're going to add that to your graph.
You're going to connect that node to a bunch of other nodes nearby.
And then you're just going to keep sampling until maybe you have enough nodes or maybe until you have your complete path.
Should be happening there.
And then once you've got this whole graph, you can just find the shortest path along that.
And there are some really cool results that if you sample in a good way, and so on, asymptotically.
As you start sampling more, you're going to asymptotically approach the best path in a completely continuous space.
The power of probabilistic road maps and a bunch of randomized algorithms though is that they scale pretty well to high dimensions.
So you don't need to actually consider the continuous space.
You can just sample [INAUDIBLE].
Are there any questions about probabilistic road maps?
Really cool.
If you are interested, and you just heard about PRMs.
You probably haven't heard about RRTs.
Those are also really cool
Just a quick [INAUDIBLE] question.
So for any node, [INAUDIBLE] at this uniform example [INAUDIBLE].
PROFESSOR 4: Sorry.
When you're choosing where to place the dot?
Or what to connect to?
[INAUDIBLE] there's a dot there on the side [INAUDIBLE] right?
So when I do the sampling, I just [INAUDIBLE] this node.
I uniformly choose one of them [INAUDIBLE]..
PROFESSOR 4: So in general, probabilistic roadmaps, you can throw in whatever sampler you want.
The way this particular one-- the way I implement this is you sample points uniformly from the entire space.
If it's inside an obstacle and you remove it, once you place it-- this was connect to the K closest-- I think K is 7 in this case.
And if there's an edge that if that edge collides with an obstacle, remove it.
I'd be happy to go into more detail for that PRMs later.
All right.
But that's not enough.
Because what we described as PRMs in the configuration space, but what we need to do is somehow elevate a PRM from the configuration space to a belief space.
And this is really hard.
We don't have access to these raw configurations.
Let's imagine we were in this really simple world where the quadrotor could be in three possible states.
One, two, three.
Really easy, right?
Sample a bunch of points.
They're going to end up in one, two, or three.
You could pretty quickly cover the entire space.
But this simple configuration space will transform to the belief space becomes infinite.
You have infinite possible distributions to consider.
There is the distribution where you have 100% chance probability-- 100% probability that you're in state 1, 100% probability that you're in state 2.
100% probability that you're in state 3.
And then everything in between.
We went from three to infinite.
This is not boding well.
And even if you start saying, well, I'm not going to consider the whole distribution.
I just care about the mean and the variance, it's still not pretty, right?
Well, this is where we have to start making approximations And this is where you start getting differences in POMDP planners-- where they make assumptions, where they make approximations.
So these images are from the belief road map paper from the Robust Robotics group a couple of years ago.
But I'm going to talk about a different planner soon.
But the idea behind a lot of these planners is maybe we can start saying what these distributions are going to look like based on our models.
So to our planning problem, if we know we started at (2,3) and we know our transition distribution, we can start saying, well, this is my probability distribution.
And then when I make and observation, I can build distributions off that.
And so, if you could exhaustively propagate these distributions forward, that would be great.
But it's unrealistic.
And I just want to point in terms of the visual way to represent these distributions.
A really nice way of saying, in the deterministic world, you have these dots and edges.
In the probabilistic world, these circles, these ellipsoids, represent uncertainty.
Typically, it's the one standard deviation or three standard deviations away.
And so you can start building into the map, these are the distributions and the variances I can see in these nodes.
Are there any questions about this stuff.
All right.
Well, we're going to delve now into a specific case study.
Feedback Based Information State Roadmaps-- FIRM.
From now on, that's the only way I'm going to refer to it.
The idea behind this is you're going to sample mean configurations from your configuration space.
Then you want to build an LQR-- that's Linear Quadratic Regulator controller-- around these mean points.
And that will generate what variance you can tolerate.
So LQR controllers, if you don't know, they're really nice.
Around a small region around a point, they can drive a quadrotor, for example, to that figure.
And so if you build these LQR controllers around points, you can say, all right, anytime I end up in this cloud in the belief space-- so any sort of distribution and can bring it back to that mean.
And so now, what we've done is we've generated every point, we just need to connect them.
And the idea is if you have a feedback based, they can get from one cloud to another anywhere in the clouds.
Then you can get from point to point.
All right.
This is how you generate the graph.
And what's cool is the way they formulated the problem is they said, well, set up the cost of executing an edge.
We switched from rewards to costs because we're pessimists now.
Well, the cost is going to be a linear combination of the expected time to execute that edge and the uncertainty along that edge.
Now, this is actually really cool to play with.
What do you think would happen if I set beta to 0 in this?
Like, what sort of [INAUDIBLE] would you get?
I will cold call because I know a few names here.
Anybody?
I don't want to cold call someone who may [INAUDIBLE]
You're going to get the shortest path.
PROFESSOR 4: Shortest path, that's right, right?
This turn goes away.
The cost is a function of the time-- shortest path.
Now, what's cool is one day I was messing around with this code.
I'm like, I wonder what happens if I set alpha to 0, right?
So your cost is purely a function of uncertainty.
In turns out what the quadrotor does it just hangs out where it starts.
It says, I'm in no hurry.
I know where I am.
I'm just going to stay here.
But I find this amazing, right?
Because this almost models behavior even.
You could start saying, do I want to be a risky quadrotor or a safe one?
Like, how important it is for me to get somewhere on time or be safe?
And it's just those two parameters.
Or you could even make it 1, like alpha and 1 minus alpha.
I think this stuff is really cool.
The one detail that I'm really going to into for FIRM is the cost equation, which is based on the Bellman backup equation that we had.
The cost to go, right, the expected cost from a belief state [INAUDIBLE] is-- well, you're going to take the best action.
So you're going to take the mide of this whole thing.
You're going to say, it's the cost of executing a specific action plus the cost of colliding with something, an obstacle, times the probability of colliding, right?
That's this term.
And then you've got to say, well, OK, and then once I reach a state, what's the cost from there?
And then I could weight that by the probability of ending up in that state.
Does this equation make sense to people?
There are a lot of symbols, and honestly, I hate notation, but it works.
You just plug in-- the cost is I'm going to take the best action.
It's going to be the cost of using that action, the cost of colliding times the probability of colliding given that I used that action and I started where I started, and then the cost from where I end up.
And since you end up in a probability distribution, we need to consider all these cases.
Yeah?
When you define like an action from one place to another, do you always think of it as starting from mean that you sampled or from anywhere [INAUDIBLE]?
PROFESSOR 4: So it's from the-- so formally, it was from the belief state, which is the mean, yeah, plus the variance once it stabilized to that point.
And the way that variance is generated, I should have said, is, you're going to have models of these quadrotors.
And so I spent a good time in the ACL with John Howell, in Course 16.
And you just like let the quadrotor hover.
You measure its position for a long time.
And you get a distribution over where it goes
So what does the letter M stand for?
PROFESSOR 4: What is the letter M?
Yeah.
PROFESSOR 4: Right.
So you're summing over-- these are the belief states that you could end up in, right?
So if everything were deterministic, this would just be ignore the sum.
It's where you end up.
Realistically you could end up in some other state we haven't considered
So just a quick question.
So those, if we're operating on a Gaussian space, then you have Gaussian observations.
So those are sums over the observation samples that are generated when [INAUDIBLE].
So that is a finite sum over the possible infinite observation states we might have.
PROFESSOR 4: Yeah.
So there's definitely [INAUDIBLE] in terms of where you could end up.
And even the action space, there's a set feedback controller that you're allowed.
I think the observation, if you modeled it as a Gaussian, if you made some nice assumptions, it can be tractable as continuous
Oh, yeah.
I was [INAUDIBLE].
So for the start, so if you start with Gaussian noise, then you have a linear model.
Then your prediction's going to be Gaussian.
PROFESSOR 4: Yeah
But then you have Gaussian observations, which is great because then your update is Gaussian but the observation space is infinite.
So basically not only you have two sample positions, but you also have two sample potential observations that you might get as you go along.
PROFESSOR 4: Yeah
So that you can basically-- and it's great.
But you end up basically reducing a possibly infinite branching, which is your Gaussian to kind of a like a Monte Carlo Tree search.
You generate a whole bunch of meaningful samples, and those are the ones that you consider.
So is that where the sum is coming from?
PROFESSOR 4: Yeah, basically.
And a fun fact, the theorem that I read and trusted was that if you just sample randomly right from your configuration space, you have zero probability of constructing a graph that without any assumptions will be connected.
Whereas for PRMs, you sample enough and things will turn out nicely, not the case with the belief state.
That's why we need to make these assumptions.
We're killing it.
We know POMDPs.
Now we get to look at some really fun graphics generated from real flights using FIRM.
The big takeaway is that FIRM prefers safer paths.
We've got two images that look really similar.
We're going to talk about one and then show why they're slightly different.
The test flight that we put this quadrotor under was we said, we're going to start at this configuration.
We're going to go this configuration.
And there's this big, blue obstacle.
And there are landmarks that you can take measurements off of.
So those are these red dots.
We want to compare two planners right, a PRM and a FIRM planner.
The PRM planner said, right, I just want to minimize time.
And so it found the first path is stay to the left of the obstacle.
But that's actually a really narrow path between the obstacle and the wall.
On the other hand, the FIRM planner said, right, I want to minimize the linear combination of time and uncertainty.
And so if you ramp up the term the wait for uncertainty, at some point the path sort of pops over the obstacle.
It's really cool.
It snaps.
And then you get this safer path.
But that's longer.
And how do we know it's safer?
Because these ellipsoids that we've drawn are the 3D versions of what we saw for PRM elevated to the belief space version.
It's the uncertainty that the quadrotor has over its true state.
Why is the PRM plan-- why does it have such big ellipsoids?
It's because when it's behind the obstacle, it can't make any of these measurements because it can't see the landmarks.
So it's basically using dead reckoning.
And the transition model, right, is not deterministic.
So it's uncertainty grows.
And so we can see these ellipsoids are bigger for PRM than FIRM.
And then the reason I have two images is they are slightly different.
It's that these landmarks were fictitious landmarks that we just said, you can-- and we would generate fake measurements off of them.
And so we could tune the noise of the landmarks.
Maybe a little bit of cheating, but it allowed us to say, would it increase the noise from some-- the dimension was number 0.05 to 0.15.
You can see the uncertainty ellipsoids from PRM grow when we increased the noise, whereas for FIRM, they stay about the same.
And importantly, these ellipsoids grow enough that they start overlapping with the obstacles.
That represents a very high probability of collision.
Whereas the FIRM, the way it managed to keep these ellipsoids so small is we kept the cost function the same, right-- the same weight on uncertainty, same weight on the time, right?
But the uncertainty was so much higher.
And so we decided, well, I can sacrifice a little bit of time.
I can take the slower path.
I can just hang out by landmark, really make sure I know where I am before continuing.
And so the path-- the duration of the flight took a lot longer for FIRM as time would increase but the uncertainty would stay about constant.
Do people understand these graphics?
Great.
All right.
We've got a graph that just represents a little more formally that growing uncertainly.
As noise increases, the variance along a single dimension, z, y, to x, for PRM, which is that, and FIRM.
The variance for PRM is always higher, and it grows.
For FIRM, it's lower, and stays about constant.
That's the big takeaway.
FIRM minimizes this uncertainty.
And then the final image from these results that I want to show is in simulation.
They said, well, let's actually measure how often it crashes.
We didn't want to do this in the real world because we don't want to crash the quadrotor that many times.
The gist of it is comparing a reactive planner and a deterministic one [INAUDIBLE].. As noise increases-- noise was simulated with wind strength-- the number of crashes increases for PRM, basically.
And for FIRM, it stays constant and low.
The reason there are two lines is there were two planners with different time horizons.
The important thing is FIRM is low and constant.
PRM grows.
We've talked now throughout all probabilistic planning you ever need to know, right?
No, not quite.
But we have covered a lot of stuff.
What are the big takeaways?
We've learned that real-world problems are stochastic, right?
Quadrotors are not these perfect machines that we wish they were.
But it's important to model them as stochastic.
The problem is once you start modeling them as stochastic, it becomes a lot harder to solve.
But if you make some assumptions, or even if you don't, if you're just get smart and you do the heuristics, you can resolve this complexity.
And so I hope you remember these three points, you remember this graphic or that graphic, the idea that if you take uncertainty into account, you get fundamentally different paths [INAUDIBLE].. And that can be a good thing.
What questions do you have, anything?
Yeah?
[INAUDIBLE] so far people have [INAUDIBLE] problems.
So how do the same, you know, [INAUDIBLE]??
So for instance, we suggest that this is the maximum risk we want to take.
Is it possible to integrate each of our constraints into your optimization problem and solve it?
Or [INAUDIBLE]?
PROFESSOR 4: So I imagine one thing that I would like to test is if we go back to this really crude version of our cost equation, we can imagine saying that if we want to come up with a bound for uncertainty that we can tolerate, you could maybe like setting an intercept for this to be that bound and then just ramping this up
Oh, [INAUDIBLE] multiplier [INAUDIBLE]..
PROFESSOR 4: Something like that
[INAUDIBLE]
Yeah, exactly.
So it only comes into effect.
Possibly what I said is a terrible idea.
But it's-- especially in simulation, you can just try it out and see if it works
And just another question, you said propagated probability solution on the networks is hard.
[INAUDIBLE] So therefore we can assume an illustrated area, and then [INAUDIBLE] normal distribution, they are contrary to each other.
Therefore you can [INAUDIBLE] observation and then updating [INAUDIBLE] and form a distribution, right?
PROFESSOR 4: Sure.
So I think that might be making some assumptions that we might not be willing to make about the nature of the distributions that you're trying to propagate.
I need think about this some more.
But I think intuitively, we can understand that any time you're propagating a distribution versus a single discrete value, it's definitely not going to be easier.
And so as your distributions become more complex.
Perhaps if you're modeling a real-world stochastic sensor, you might not be able to perform these efficient updates using conjugate priority.
Any other questions?
Otherwise we-- yeah?
So all these FIRM examples you're showing, this is all planning done offline.
[INAUDIBLE] offline, right?
Have you expanded this at all to like an online case?
Or does that all require re-planning?
And how long does it take?
PROFESSOR 4: Right.
So it's not good, I can tell you that much.
What's nice is FIRM generates a policy.
So if you construct your PRM to say don't just find the shortest path, keep sampling points until you're confident that you're always going to end up near some point.
You can construct a policy and then just online look up what's my belief that matches most closely.
It took for-- I think we sampled 600 nodes in like-- what are the dimensions on this?
So 2 to 3 meter by 8 by 3 meter thing-- it took about 15 minutes.
It was really slow.
Now, granted it was a virtual machine.
But it's not something we want to re-plan on the fly.
With PRM, typically a lot faster
Who was a lot faster in that case?
PROFESSOR 4: So when we just did PRM, I think it was one minute or something, at least an order of magnitude.
And you could use other planners as well, like RRTs.
And there are RRT versions of-- there are POMDT versions of RRTs.
But [INAUDIBLE].
All right, I'm going to turn it over to Steve so we can learn about this project.
[APPLAUSE]
So I'm just going to say a few good words about the Grand Challenge.
So again, the final assignment will be released tonight for this.
And as Professor Warrens mentioned, it's going to be descoped a bit from what we originally had in the syllabus due to time constraints.
But here's a preview of what you'll be doing.
So for an overview, it will be a class-wide collaboration.
So what we're going to do, you guys are going to stay in your advanced lecture teams and each basically apply what you've done, the great work you've done on that, onto our hardwork robot that we have.
so it's not a competition where each team will be competing against each other.
And so again, we're descoping it.
The syllabus originally said it was 20% of your grade.
It's probably going to be more like 10% or 15% in the end.
So this is the robot you guys will be using.
It's called an [INAUDIBLE] robot.
Usually it has sort of a robotic arm on it.
But we're actually not going to be using it for this [INAUDIBLE].
This base made from a company from Spain, Orbonix.
It's a pretty cool robot.
One cool thing about it is that it's actually omnidirectional.
So there are wheels on your wheels here.
And what that means is that it can drive sideways, left to right.
It would make parallel parking your car very easy.
[LAUGHTER] We're not going to be driving it probably this fast because it's actually super heavy.
And we don't want anyone to-- yeah, we forgot to screw in that [INAUDIBLE].. [LAUGHTER] It will be screwed in during the competition.
But it's a pretty fun robot.
So you guys will be working on this.
And so the actual challenge itself, as we've mentioned many times throughout this class, will be a modified orienteering your challenge.
So there's going to be a few different challenge stations that you have to drive to.
And at the stations, there'll be small computational challenges.
And those computational challenges will be a subset of the advanced lecture teams, not all of them.
And the goal is to complete as many of those as you can and try to do that as quickly as possible.
And it's also going to be held indoors in our lab space, where we just showed you.
So that way if it rains, we can still have the Grand Challenge at the end of the semester.
So this is sort of how it's set up as of last night actually.
So basically the robot will be abe to drive around in a small little LEGO maze that we set up.
And there's going to be sort of different things that you have to do with in different places.
So what are you actually going to be doing?
What assignment are you guys going to be doing?
Well, it's actually a little flexible.
Since each of you did different things for your advanced lectures, each team is going to have a bit of a different assignment applied to this.
I have some proposed ideas that I'm going to talk about in the next slide here.
But the big thing is that these are just ideas.
You guys have a lot of flexibility in this.
You'll be probably working with the [INAUDIBLE] a lot to have access to the robot.
We can arrange extra office hours for you guys to come use the hardware to test things.
So we'll be arranging all of that things as sort of on an as-needed basis.
If you want to-- basically it'll sort of depend on your team
And the people who'll be helping out are you, us--
Yes, me and Tiago, who gave a lecture earlier in the semester and possibly also a few other people from our lab.
But we'll be the main contact points for it.
So of course it should go without saying, but all the team members should contribute equally within your team.
So it'll be maybe less structure than in advanced lecture.
But just make sure that everyone's contributing equally in the assignment.
And it's going to involve using this thing called the Robot Operating System, or ROS, which is basically a software framework for communicating and it's used a lot in robotics.
Just a quick show of hands, how many of you have used ROS before?
Oh, wow, so a lot of used ROS.
How many have heard of ROS, if not used it?
OK, so a lot of people.
So that's a good starting point.
So here are the the proposed tasks for each group.
And of course, all of these are up for change.
If you guys want to change it, let me know.
Basically, so some of the groups, it's very clear how it immediately applied to the Grand Challenge.
So incremental path planning-- well, we have a mobile robot, so maybe we can actually print that on the robot and get it to change or plan if something gets in the way.
The semantic globalization group-- obviously very applicable to the Grand Challenge.
You need to know where you are.
So the robot that we have now can actually do normal metric localization.
So it can sort of know where it is.
But what will be interesting to see is compare the semantic localization to that one.
But how would you do semantic localization?
Well, we can use a camera.
And we can choose the visual learning through deep classification-- the visual classification through deep learning group.
So I think these two groups would have a really nice synergy and a really cool way to work together.
So the MCTS group are very cool, but I had a little trouble thinking about exactly how that would apply.
So maybe you look like you have an idea maybe
So to clarify, all these are separate runs of the robot?
So we're actually probably going to run all of them-- so the grid is-- this would be probably one run of the robot.
We're going to decouple these so that if one or a subset these don't work super well, the other groups will still be able to run.
So we're carefully planning that out too.
So the MCTS group, I was thinking maybe could solve-- that could be a really nice way to-- one of those computational challenges.
Maybe you have to play against a human.
And if it wins, great.
You can go faster or you get more points.
So you could implement on a different game other than connect four or possibly and sort of [wrap it around [?
rod ?]
that to call with that.
So each ability group I think would also be a great place to do one of these challenge stations.
So we could give you guys puzzle, say maybe it's some sort of maze-like state space.
And you have to see could we even the reach the goal here?
And if you get the answer right, well, you an move on to the next stage or get more points or something like that
But again, these are just suggestions.
So for example, they can do the reachability for doing motion planning
Yeah, so if you guys have other suggestions on how to implement your team's stuff into the Grand Challenge, definitely send me an email, preferably today or as soon as you think of the things.
And we can change these.
These are just suggestions for right now.
The last two are sort of more planning related.
So planning with temporal logic, which was Monday's lecture, I thought would be cool way to sort of control the robot's high-level action.
So maybe that could involve modeling are Grand Challenge with PDDL.
And maybe with linear temporal logic goals, models that you get [INAUDIBLE].
Then you could compile that and call up to turn around it and execute that plan on the actual robot, do [INAUDIBLE] high-level actions of the robot.
That's a possibility.
And for today's group, the infinite horizon probablistics planning, maybe you could do something actually similar.
But instead of modeling the domain as PDDL model it as an NVP and solve it with LAO star, and get sort of a policy on how to control the robot.
Maybe we can change it a little bit so that certain squares can be more risky than others or so on.
So again, there's flexibility in all of these here.
So these are just some of the suggestions.
Does anyone have any questions before we-- this all I have.
So anyone have any questions?
Yeah?
How are we working on [INAUDIBLE]??
Are you [INAUDIBLE]?
So it's basically up to you guys really divide up the work amongst yourselves.
It's really different for every team.
So we're--
Sorry.
So each team is developing [INAUDIBLE]
Right, yeah.
Each team is really in their own separate package FIRM.
For example, these two teams, there's probably going to be a common interface where the output of this one goes to the input of that one.
So for that one, we're going to give you sort of the interface [INAUDIBLE] message type package for that.
But other than than, like for dividing up the work, you guys [INAUDIBLE] that.
PROFESSOR 2: So your plans will be integration if you're trying [INAUDIBLE] pieces the group is doing.
That we think is, thing that uses soft engineering skills [INAUDIBLE].
It can be really unpleasant to do, take a lot of time.
So we don't want you to have that experience.
So that's why you can...
If not only what we wanted you to do is to be able to get your own capability [INAUDIBLE]..
If you guys choose to integrate with other teams because you're really excited about that and because it looks like the people that you're working for [INAUDIBLE].. Then that's purely your choice.
Is that fair enough?
Sure.
It's fine.
Any more questions?
OK.
Sounds great.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educatiohal resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
As this says, and all the handouts say that this is 6.450.
It's a first year graduate course in the principles of digital communication.
It's sort of the major first course that you take as a graduate student in the communication area.
The information theory course uses this as a prerequisite, uses it in a rather strong way.
6.432, the stochastic process course uses it as a prerequisite.
6.451, which is the companion course which follows after this uses it as a prerequisite.
It's sort of material that you have to know if you're going to go into the communication area.
It deals with this huge industry, which is the communication industry.
You look at the faculty here in the department and you're a little surprised to see that there are so few faculty members in digital communication, so many faculty members in the computer area.
You wonder why this is.
It's sort of this historical accident, because somehow or other the department didn't realize as these two fields were growing, the one that looked more glamorous for a long time was the computer field.
Nobody in retrospect understands why that is.
But at the same time that's how it is.
You will not see as much excitement here.
You also will see that because of the big bust in the year 2000, there's much less commercial activity in the communications field now.
As students, you ought to relish this.
You ought to really be happy about this if you want to go into the communication field, because at some point in the future, all of the failure to start new investments, and all of the large communication companies means a very explosive growth is about to start.
So I don't think this is a dead field by any means.
Looking back historically, every time the communication field has seemed dead, and I can think of three or four such times in history, those have been the exact times where it was great to get into the field.
There was a time in the early '70s when all the theoreticians who worked in communication theory all were going around with long faces saying everything that we can do has been done, the field is dead, nothing more to do, might as well give up and go into another field.
This was really exactly the time that a number of small communication companies were really starting to grow very fast because they were using some of the modern ideas of communication theory and they were using these modern ideas to do new things.
You now see companies like Qualcomm that were started then, Motorola of whom major divisions were started back then.
An enormous amount of activity was starting just then, partly because the field seemed to be moribund and the theoreticians were turning and deciding well, I guess we have to do something practical now.
If you want to find a good practical engineer, if you wind up in industry, if you wind up as an entrepreneur or if you wind up climbing the ladder and being a chief executive, if you want to find somebody who will really do important things, find somebody who understands theory who has decided that they suddenly want to start to do practical things.
Because if you point to the people who have really done exciting things in this field, those are the ones who have done most of it.
OK, so anyway, the big field, it's an important field.
What we want to do is to study the aspects of communication systems that are unique to communication systems.
In other words, we don't want to study hardware here, we don't want to study software here, because hardware is pretty much the same for any problem you want to look at.
It's not that highly specialized, and software is not either.
So we're not going to study either of those things, we're going to study much more the architectural principles of communication.
We're going to study a lot of the theory of communication, and I want to explain why it is that we're going to do that as we move on.
Because it's not at all clear at first why we want to study the silly things that we'll be studying in the course, and I want to give you some idea today of why we're pushed into doing that.
As we start doing these things, I'll give you more of an idea.
I know when I was a student I couldn't figure out why I was learning the things that I was learning.
I just found that some things were much more interesting than others, and the things that tended to be interesting were the more theoretical things, so I started looking at them harder and thought gee, this is neat stuff.
But I had no idea that it was ever going to be important.
So I'm going to try to explain to you why this is important, and something today about what the proper inter-relationship is of engineering and theory.
As another example of my own life, for a long time I thought -- this is rather embarrassing to say -- but I was a theoretician and not much of an engineer for a very long time.
And I kept worrying about this and saying well the stuff I'm doing is fun and I enjoy doing it, but why should anybody pay me for it?
Of course, I was being paid by MIT, I was being paid as a consultant, but I felt I was robbing all these people.
And finally, I decided a rationale for all of this.
Most people felt that theory was silly, and therefore, they would keep asking me well, if you're so smart why aren't you rich?
I didn't have any very good answer to that because I felt I was smart because I understood all this theory, but I wasn't rich.
So I thought gee, what I've gotta do is do enough engineering to get rich, and then when people ask me that question I can say I am.
I would suggest to all you, if you have theoretical leadings, to focus on some engineering also so you can become rich, not because there's anything to do with the money, but just because it lets you hold your head up high in a society that values money a lot more than values intellect.
So it's important.
OK.
This is a part of this two-term sequence.
One comment here is I'm not sure that whether 6.451, which is the second part of this sequence, is going to be taught in the spring or not.
It might not be taught until spring of the following year.
If a large number of you decide that you really want to do this and do it now, make yourselves heard a little bit because the idea of a lot of people want this course will have something to do on whether somebody manages to teach it or not.
As I was starting to say, theory has had a very large impact on almost every field we can think of.
I mean you think of electromagnetism, you think of all these other fields and theory is very important in them.
But in communication systems it is particularly important.
It's partly important because back in 1948, Claude Shannon who was the inventor of information theory -- you usually don't invent theories, but he really did.
This came whole cloth out of this one mind.
He had these ideas turning around in his head for about eight years while the second World War was going on.
He was working on other things, which were, in fact, very important things, and he was doing those things for the government.
He was working on aircraft control and things like that.
He was working on cryptography.
But he was just fascinated by these communication problems and he couldn't get his mind off them.
He took up cryptography because it happened to use all the ideas that he was really interested in.
He wrote something about cryptography during the middle of the war, which made many people believe that he had done his cryptography work before he did his communication work.
But in fact, it was the other way around.
This was purely the fact that he could write about cryptography, and he never liked to write, so he didn't want to write about communication because it was a more complicated field and he didn't understand the whole thing.
So for 25 years it was a theory floating around.
You will learn a good deal of what this theory is.
Most graduate courses on communication don't spend much time on information theory.
This was not a mistake many, many years ago, but it is a mistake now, because communication theory at its core is information theory.
Most people realize this.
Most kids recognize it, most people in the street recognize it.
When they start to use a system to communicate with the internet, what do they talk about?
Do they talk about the bandwidth?
No.
They might call it bandwidth but what they're talking about is the number of bits per second they can communicate.
This is a distinctly information theoretic idea, because communication used to be involved with studying different kinds of wave forms.
Suddenly, at this point, people are studying all these other things and are based on Claude Shannon's ideas, therefore, we will teach some of those ideas as we go.
As a matter of fact, I was talking a little bit about the fact that there's been a sort of a love-hate relationship by communication engineers for information theory.
Back in 1960, a long, long time ago when I got my PhD degree, the people at MIT, the people who started this field, many of them told me that information theory was dead at that point.
There'd been an enormous amount of research done on it in the ten years before 1960.
They told me I should go into vacuum tubes, which was the coming field at that time.
Two points to that story.
If people tell you that information theory isn't important, please remember that story or else tell them to start to study vacuum tubes.
The other point of it -- well, I forgot what the other point of it is so let's go on.
Anyway, information theory is very much based on a bunch of very abstract ideas.
I will start to say what that means as we go on, but basically it means that it's based on very simple-minded models.
It's based on ignoring all the complexities of the communication systems and starting to play with toy models and trying to understand those.
That's part of what we have to understand to understand why we're doing what we're doing.
A very complex relationship between modeling, theory, exercises and engineering design.
Maybe I shouldn't be talking about that now and maybe the notes shouldn't talk about it, but I want to open the subject because I want you to be thinking about that throughout the term.
Because it's something that most people really don't understand.
Something that most teachers don't understand.
It's something that most students don't understand.
When you're a student, you do the exercises you do because you know you have to do them.
There was a sort of apocryphal story awhile ago that somebody told me about a graduate student, not at MIT, thank God, who went into see the person teaching a course on wireless and saying, I don't want to know all that theory, I don't want to have to think about these things, just tell me how to do the problems.
Now, we will explain the enormous stupidity of that as we move on.
Part of the stupidity is that the problems that you work on as a graduate student, even in your thesis, the problems that you work on are not real problems.
The problems that we work on everywhere in engineering are toy problems.
We work on them because we want to understand some aspect of the real problem.
We can never understand the whole problem, so we study one aspect, then we study another aspect.
The reason for all these equations that we have, it's not a way to understand reality.
It really isn't.
It's a way to understand little pieces of reality.
You take all of those and after you have all of these in your mind, you then start to study the real engineering problem and you say I put together this and this and this and this and this -- this is important, this is not so important, here, this is more important.
And then finally, if you're a really good engineer, instead of writing down an equation, you take an envelope, as the story goes, and you scribble down a few numbers, and you say the way this system ought to be built is the following.
That's the way all important systems get built, I claim.
Important systems are not designed by the kinds of things you do in homework exercises.
The things you do in homework exercises are designed to help you understand small pieces of problems.
You have to understand those small pieces of problems in order to understand the whole engineering problems, but you don't do simulations to build a communications system.
You understand what the whole system is, you can see it in your mind after you think about it long enough, and that's where good system design comes from.
Simulations are a big part of all of this, writing down equations are a big part of it.
But when you're all done, if you design an engineering system and you don't see in your mind why the whole thing works, you will wind up with something like Microsoft Word.
That's the truth.
So the exercises that we're going to be doing are really aimed at understanding these simple models.
The point of the exercises is not to get the right answer.
The answer is totally irrelevant to everything.
What's important is you start to understand what assumptions in the problem lead to what conclusions.
How if you change something it changes something else.
That's when you're using system design, because in system design you can never understand the whole thing.
You have to look at what parts of it are important, what parts of it aren't important, and the only way you can do that is having a catalog of the simple-minded things in the back of your mind.
Practical engineers get that by dealing with real systems day-to-day all their lives.
They see catastrophes occur.
Those things tell them what to avoid and what not to avoid.
Students can do this in a much more efficient way.
Obviously, in the practical experience also, but the experience that you get from looking at these exercises and looking at this analytical material in the right way, mainly looking at it is how do you analyze these simple toy models, how do you make conclusions from them is what let's you start being a good engineer.
I'm stressing this because at some point this term, all of you, all of you at different times are going to start saying why the hell am I doing this?
Why am I dealing with this stupid little problem?
Stupid little problems can get incredibly frustrating at times, because you see something that's so simple and you can't understand it.
Then you say well, this simple thing can't be important anyway.
What I really want to do is understand what this overall system is, and you give up at that point.
Don't do that.
Sometimes there's a very difficult question about what you mean by something being simple.
I will sometimes say that something is very simple, and I will offend many you to whom it's not simple.
The point is something becomes simple after you understand it.
Nothing is simple before you understand it.
So there's that point at which the light goes off in your head at which it becomes simple.
When I say that something is simple, what I mean is that if you think about it long enough a light will go off in your head and it will become simple.
It's not simple to start with.
Other things are just messy.
You've all heard of mathematical problems which are just ugly.
There are these things of extraordinary complexity.
There are things where there just isn't any structure.
You see a lot of these things in computer science.
I talked about Microsoft Word.
Part of the reason it's such a lousy language is because the problem is incredibly difficult.
It's incredibly unstructured.
So people come up with things that don't make any sense, because that inherently is not simple.
OK, these other things that we'll be studying here are inherently simple and the only problem is how do you get to the point of seeing these simple ideas.
OK, so that's enough philosophy.
No, not quite enough, a little more of that.
All the everyday communication systems that we deal with are incredibly complex in terms of the amount of hardware, the amount of software in them.
If you look at the whole system and you try to understand it as a whole, you don't have prayer.
There's no way to do it.
These things work and they can be understood because they're very highly structured.
They have simple architectural principles.
What do I mean by architectural principles?
It's a word that engineers use more and more.
It started off with computer systems because it became essential there to start thinking in terms of architecture.
It's exactly the same thing that you mean when you're talking about a house or a building.
Mainly the architecture is the overall design.
It's the way the different pieces fit together.
In engineering, architecture is the same thing.
It's the thing that happens when your eyes fuzz over a little bit and you can't see any of the details anymore, and suddenly what you're doing is you're looking at the whole thing as an entity and saying how do the pieces fit together.
Well, what we're going to be focusing on here.
One of the major keys to making these things simpler is to have standardized interfaces and to have layering.
And we'll talk a little bit about each of these because our entire study of this subject is based on studying the different layers that we'll wind up with.
OK, the most important and most critical interface in a communication system is between the source and the channel.
Today, that standard interface is almost always a binary data stream.
You all know this.
When you talk about your modab, how do you talk about it?
48 kilobits per second if you're an old-fashioned kind of guy.
1.2 megabits if you're a medium technology person, or several gigabits if you're one of these persons who likes to gobble up huge amounts of stuff.
That's the way you describe channels -- how many bits per second can you send?
The way you describe sources is how many bits per second do you need as a way of viewing that source.
When you store a picture, how do you talk about it?
You don't talk about in terms of the colors or any of these other things, picture is a bunch of bits.
That, at the fundamental level, is what we're doing here.
When you deal with source coding, you're talking fundamentally about the problem of taking whatever the source is, and it can be any sort of thing at all -- it can be voice, it can be text, it can be emails, the emails with viruses in it, whatever -- and the problem is you want to turn it into a bit stream.
Then this bit stream gets presented to the channel.
Channels and the channel encoding equipment don't have any idea of what those bits mean.
They don't want to have any idea of what those bits mean.
That's what an interface means.
It means the problem of interpreting the bits, the problem of going from something which you view as having intelligence down to a sequence of bits is the problem of the source coder.
The problem of taking those bits and moving them from one place to another is the function of the channel designer, the network designer and all of those things.
When you talk about information theory, it's a total misnomer from the beginning.
Information theory does not deal with information at all, it deals with data.
We will understand what that distinction is as we go on.
When you talk about a bit stream, what you're talking about is a sequence of data.
It doesn't mean anything.
It's just that a one is different from a zero, and you don't care how it's different, it just is.
So the channel input is a binary stream.
It doesn't have any meaning as far as the channel is concerned.
The bit stream does have a meaning as far as the source is concerned.
So the problem is the source encoder takes these bits, takes the source, whatever it is, with its meaning and everything else, turns it into a stream of bits -- you would like to turn it into as few bits as possible.
Then you take those bits, you transmit them on the channel, the channel designer understands what the physical medium is all about, understands what the network is all about, and doesn't have a clue as to what all those bits mean.
So the picture is the following.
That's the major layering of all communication systems.
Here's the input, which is whatever it happens to be.
Here's the source encoder.
The source encoder has to know a great deal about what that input is.
You have to know the structure of the input.
You have a source encoder for voice and you use it to try to encode a video stream it's not going to work, obviously.
If you try to use a source encoder for English and try to use it on Chinese, it probably won't work very well either.
It won't work for anything other than what it's intended for.
So the source encoder has to understand the source.
When I say understand the source, what do I mean?
It means to understand the probabilistic structure of the source.
This is another of the things that Shannon recognized clearly that hadn't been recognized at all before this.
You have to somehow understand how to view what's coming out of the source, this picture, say, as one of a possible set of pictures.
If you know ahead of time that a communication system is going to be sending the Gettysburg Address or one of 50 other highly-inspiring texts, what do you do?
Do you try to encode these things, all these different texts?
Of course not.
You just assign number one to the Gettysburg Address, number two to the second thing that you might be interested in.
You send a number and then at the output out comes the Gettysburg Address because you've stored that there.
So that's the idea of source coding.
What is important is not the complexity of the individual messages, it's the probabilistic structure that tells you what are the different possibilities.
That's what happens when you try to turn things into binary digits.
You have one sequence of binary digits for each of the possible things that you want to represent.
Then in the channel you have the same sort of thing.
You have noise, you have all sorts of other crazy things on the channel.
You have a sequence of bits coming in, you have a sequence of bits coming out.
The fundamental problem of the channel is very, very simple.
You want to spit out the same bits that came in.
You can take a certain amount of delay doing that, but that's what you have to do.
In order to do that, you have to understand something about the probabilistic structure of the noise.
So both ways you have probabilistic structure.
It's why you have to understand probability at the level of 6.041, which is the undergraduate course here studying probability.
If you don't have that background, please don't take the course because you're going to be crucified.
If you think you can learn it on the fly, don't.
You can't learn it on the fly.
As a matter of fact, if you've taken the course, you still don't understand it, and you will need to scramble a little bit to understand it at a deeper level in order to understand what's going on here.
6.041 is a particularly good undergraduate course.
I think it's probably taught here better than it's taught at most places.
But you still don't understand it the first time through.
It's a tricky, subtle subject you really need to understand enough of it, If you have no exposure to it.
If you've taken a course called statistics and probability, which first teaches you statistics and then tucks in a little bit of probability at the end, go learn probability first because you don't have enough of it to take this subject.
The other part of dealing with these channels is that suddenly we have to deal with 4AA analysis and all of these things, if you don't have some kind of subject in signals and systems.
Some computer scientists don't learn that kind of material anymore.
Again, you can't learn it on the fly because you need a little bit of background for it.
Those two prerequisites are really essential.
Nothing else is essential.
The more mathematics you know the better off you are, but we will develop what we need as we go.
Now, we talked about this fundamental layer in all communication systems, which is between source coding and channel coding.
You first turn the source into bits, and then you turn the bits into something you can transmit on the channel.
Source coding breaks down into three pieces itself.
It doesn't have to break down into those three pieces, but it usually does.
Most source coding you start out with a wave form, such as with voice, or with pictures you start out with what's more complicated than a wave form, some kind of mapping from x-coordinate and y-coordinate and time, and you map all of those things into something.
Our problem here is we want to take these analog wave forms or generalizations of.
It turns out that all we have to deal with here is the analog wave forms because everything else follows easily from that.
So there's one question.
How do you turn an analog wave form into a sequence of numbers?
This is the common way of doing source coding.
Many of you who have been exposed to the sampling theorem, you think that's the way to do it, this is one way to do it.
We will talk a great deal about this.
You will find out that the sampling theorem really isn't the way to do it.
Although again, it's a simple model, it's the way to start dealing with it.
Then you learn the various other parts of that as you go along.
After you wind up with sequences, sequences of numbers, we worry about how to quantize those numbers, because fundamentally we're trying to get from wave forms into bits.
So the three stages in that process are first go to sequences, go from sequences to symbols -- namely, you have a finite number of symbols, which are quantized versions of those analog numbers which can be anything.
Then finally, you encode the symbols into bits.
The coding goes in the opposite order.
Here's a picture which shows it better than the words do.
From the input wave form, you sample it or something more generalized than sampling it.
That gives you a sequence of numbers.
You quantize those numbers, that gives you a symbol sequence.
You have a discrete coder, which turns those symbols into a sequence of bits in some nice way.
Then you have this reliable binary channel.
Notice what I've done here.
This thing is that entire system we were talking about before.
This has a channel encoder in it, a channel, a channel decoder and all of that.
But what we've been saying in this layering idea is when you're dealing with source coding you ignore all of that.
You say OK, it's Tom's job who was designing the channel encoder to make sure that my bits come out here as the same bits.
If the bits are wrong it's his fault.
If the bits are right, and this output wave form doesn't look like the input wave form, it's my fault.
So part of the layering idea is we just recreate this whole thing here as the same bits and we don't worry about what happens when the bits are different.
The other part of this is that all of this breaks down in the same way.
Namely, at this interface between here and here, the idea is the same.
Namely, there's an input wave form that comes in, there's a sequence of numbers here.
The job of this analog filter, as we call it, and you'll see why we call it that later, is to take numbers here, turn them back into wave forms.
These numbers are supposed to resemble these numbers in some way or other.
I can deal with this part of the problem totally separately from dealing with the other parts of the problem.
Namely, the only problem here is how do I take numbers, turn them into wave forms.
There are a bunch of other problems hidden here, like these numbers are not going to be exactly the same as these wave forms because I have quantization involved here.
Since the numbers are not exactly the same, we have to deal with questions of approximation.
How to approximations in numbers come out in terms of approximations in terms of wave forms?
That's where you need things like the sampling theorem and the generalizations of it that we're going to spend a lot of time with.
Then you go down to this point.
At this point the quantizer produces a string of symbols.
Whatever those symbols happen to be, but those symbols might, in fact, just be integers whereas the things here were real valued numbers, and in quantizing we turn real numbers into intergers by saying -- well, it's the same way you approximate things all the time.
You round off the pennies in your checkbook.
It's the same idea.
So the problem here is the quantizer produces these symbols here.
The job of all of this stuff is to recreate those same symbols here.
So the symbols now go into something that does the reverse of a quantizer -- we'll call it a table look-up because that's sort of what it is.
So we can study this part of the problem separately, also.
You don't have to understand anything about this problem to understand this problem.
Finally, we have symbols here going into a discrete encoder.
We have an interesting problem which says how do we take these symbols which have some kind of probabilistic structure to them and turn them into binary digits in an efficient way.
If the symbols here, for example, are binary symbols and the symbols happen to be 1 with probability 999 out of 1,000, and zero with probability 1 in 1,000, you don't really just want to take these symbols and map them in a straightforward way, 1 into 1 and zero into zero.
You want to look at whole sequences of them.
You want to do run length coding, for example.
You want to count how long it is between these zero's, and then what you do is send those counts and you encode them.
So there are all kinds of tricks that you can use here.
But the point is, this problem is separate, and this problem is separate from this problem.
Now what we're going to do in this course is start out with this problem, because this is the cleanest and the neatest of the three problems.
It's the one you can say the most about.
It's the one which has the nicest results about it.
In fact, a lot of source coding problems just deal with this one part of it.
Whenever you're encoding text, you're starting out with symbols which are the letters of whatever language you're dealing with and perhaps a bunch of other things -- ask a code, you've generated 256 different things, including all the letters, all the capital letters, all of the junk that used to be on typewriters, and some of the junk that's now in word processing languages, and you have a way of mapping those into binary digits.
We want to look at better ways of doing that.
So this, in fact, covers the whole problem of source coding when you're dealing with text rather than dealing with wave forms.
This problem here combined with this problem deals with all of these many problems where what you're interested in is taking a sequence of numbers or a sequence of whatjamacallits and quantitizing them and then coding.
Finally, when you get to the input wave forms and output wave forms, you've solved both of these problems and you can then focus on what's important here.
This is a good case study of why the information theoretic approach works and why theory works.
Because if you started out with the problem of saying I want to build something to encode voice, and you try to do all of these together, and you look up all the things you can find about voice encoding on the web and you read all of them, what you will wind up with, I can guarantee you, is you will know an enormous amount of jargon, you will know an enormous number of different systems, which each half work, you will not have the slightest clue as to how to build a better system.
So you have to take the structured viewpoint, which is really the only way to do things to find new and better ways of doing things.
Now there are some extra things.
I have over-simplified it.
I want to over-simplify things in this course.
What I will always do is I'll try to cheat you into thinking the problem is simpler than it is, and then I will talk about all of the added little nasties that come in as soon as you try to use any of this stuff.
There are always nasties.
What I hope you will all do by the end of the term is find those little nasties on your own before I start to talk about them.
Namely, when we start to talk about a simple model, be interested in the simple model, study it, find out what it's all about, but at the same time, for Pete's sake, ask yourself the question.
What does this have to do with the price of rice in China or with anything else?
And ask those questions.
Don't let those questions interfere with understanding the simple model, but you've got to always be focusing on how that simple model relates to what you visualize as a communication system problem.
It usually will have some relation but not a complete relation.
Here the problems are in this binary interface.
The source might produce packets or the source might produce a stream of data.
In other words, if what you're dealing with is the kind of situation where you're an amateur photographer, you go around taking all sorts of pictures, and then you encode these pictures -- what do I mean by encoding the pictures?
You turn them into binary digits.
Then you want to send your pictures to somebody else, but what you're really doing is sending these packets to someone else.
You're not sending a stream of data -- you have 10 pictures, you want to send those 10 pictures.
At the output of the whole system, you hope you see 10 separate pictures.
You hope there's something in there which can recognize that out of the stream of binary digits that come across the channel, there's some packet structure there.
Well we're not going to talk about that really.
Whenever you start dealing with this problem of source encoding, you also need to worry a little bit about the packet structure.
What are the protocols for knowing when something new starts, when something new ends.
Those kinds of problems are dealt with mostly in a network course here, and you can find out all sorts of things about them there.
But there are these two general types of things -- packets and streams of data.
In terms of understanding voice encoding, you can study them both together.
Why can you study them both together?
Because the packets are long and because the packets are long because they include a lot of data, you have a little bit of end effect about how to start it, a little bit of end effect about how to end it, but the main structural problem you'll be dealing with is what to do with the whole thing.
It's an added piece that comes at the end to worry about how do you denote the beginning and the end.
That's a problem you have with stream data also, because stream data does not start at time minus infinity.
Whatever the stream data is coming from, what it's coming from did not at time, t equals infinity.
You just think of it that way because you recognize you can postpone the problem of how do you start it and how do you end it, and hopefully you can postpone it until somebody else has taken over the job.
What the channel accepts is either of these binary streams or packets, but then queueing exists.
In other words, you have stuff coming into a channel -- sometimes I'm sitting at home and I want to send long files, I want to send this whole textbook I'm writing to somebody.
It queues up, it takes a long time to get it out of my computer and into this other person's computer.
It would be nice if I could send it at optical speeds.
But I don't care much.
I don't care much whether it takes a second or a minute or an hour to get to this other person.
He won't read it for a week anyway, if he reads it at all, so what difference does it make?
But anyway, we have these queueing problems, which are, again, separable.
They're dealt with mostly in the network course.
What we're mostly interested in is how to reduce bit rate for sources.
How do you encode things more efficiently?
Data compression is the word usually given to this.
How do you compress things into a smaller number of bits using the statistical structure of it?
Then how do you increase the number of bits you can send over a channel?
That's the problem with channels.
If you have a channel that'll send 4,800 bits per second, which is what telephone lines used to do.
If you build a new product that sends 9,600 bits per second, which was a major technological achievement back in the '70s and '80s, suddenly your company becomes a very important communication player.
If you then take the same channel and learn how to communicate at megabits over it, that's a major thing also.
How did people learn to do that?
Mostly by using all the ideas that the people used in going from 4,800 bits per second to 9,600 bits per second.
It was the people who did that first job who were the primary people involved in the second job.
The point of that is it doesn't really make any difference as an engineer whether you're going from 4,800 to 9,600 or from 9,600 to 19.2 or from 19.9 to 38.4 or whatever it is, or so forth up.
Each one of these is just putting in a few new goodies, recognizing a few new things, making the system a little bit better.
Let's talk about the channel now.
The channel is given -- in other words, it's not under the control of the designer.
This is something we haven't talked about yet.
In all of these communication system design problems, one of the things that gets straight is when you're worrying about the engineering of something, what parts of the system can you control and what parts of it can't you control.
Even more when you get into layering -- what parts can you control and what parts can other people control.
Namely, if I have some source and I'm trying to transmit it over some physical channel and I have two engineers -- one of them is a data compressor, and the other is a channel guy.
What the channel guy is trying to do is to find a way of sending more bits over this channel than could be done before.
What the data compressor is trying to do is to find a way to encode the source into fewer bits.
If the two come together, the whole thing works.
If the two don't come together, then you have to fire the engineers, hire some new engineers or do something else or you get fired.
So that's the story.
The things you can't change here, you can't change the structure of the source usually, and you can't change the structure of the channel.
There's some very interesting new problems coming into existence these days by information theorists who are trying to study biological systems.
One of the peculiar things in trying to understand how biological organisms, which have remarkable systems for communicating information from one place to another, how they manage to do it.
One of the things you find is that this separation that we use in studying electronic communication does not exist in the body at all.
In other words, what we call a source gets blended with the channel.
What we call the statistics of the source get very much blended in with what this organism is trying to do.
If the organism cannot send such refined data, it figures out ways to get the data that it needs which is essential for its survival or it dies and some other organism takes over.
So you find the system which has no constraints in it, and it evolves with all of these things changing at the same time.
The channels are changing, the source statistics are changing, and the way the source data is encoded is changing, and the way the data is transmitted is changing.
So it's a whole new generalization of this whole problem of how do you communicate.
Here though, the problem we're dealing with is these fixed channels, which you can think of, depending on what your interests are, you can view the canonical channel as being a piece of wire in a telephone network, or even view it as being a cable in a cable TV system, or you can view it as being a wireless channel.
We will talk about all of these in the course.
The last three weeks of the course is primarily talking about wireless because that's where the problems are most interesting.
But in any situation we look at, the channel is fixed.
It's given to us and we can't change it.
All we can do fiddle around with the channel encoding and the channel decoding.
The channels that we're most interested in usually send wave forms.
That brings up another interesting question.
We call this digital communication, and what I've been telling you is that the sources that we're interested in are most often wave form sources.
The channels that we're interested in are most often channels that send wave forms, receive wave forms and add noise, which is wave forms.
Why do we call it digital communication?
Anybody have a clue as to why we might call all of this digital communication?
Yeah?
[INAUDIBLE]
Because the analog gets represented in binary digits, because we have chosen essentially, because of having studied Shannon at some point, to turn all the analog sources into binary bit streams.
Namely, when you talk about digital communication, what you're really doing is saying I have decided there will be a digital interface between source and channel.
That's what digital communication is.
That's what most communication today is.
There's very little communication today that starts out with an analog wave form and finds a way to transmit it without first going into a binary sequence.
So here's a picture of one of the noises that we're going to study.
We have an input which is now a wave form.
We have noise, which is a wave form.
And we have an output, which is a wave form.
This input here is going to be created somehow in a process of modulation from this binary stream that's coming into the channel encoder.
So somehow or other we're taking a binary stream, turning it into a wave form.
Now, think a little bit about what might be essential in that process.
If you know something about practical engineering systems for communication, you know that there's an organization in the U.S. called the FCC, and every other country has its companion set of initials, which says what part of the radio spectrum you can use and what part of the radio spectrum you can't use.
You suddenly realize that somehow it's going to be important to turn these binary streams into wave forms, which are more or less compressed into some frequency bands.
That's one of the problems we're going to have to worry about.
But at some level if I take a whole bunch of different binary sequences, one thing I can do, for example, I could take a binary sequence of length 100.
How many such binary sequences are there?
Has length 100, the first bit can be 1 or zero, second bit can be 1 or zero -- that's four combinations.
Third bit can be 1 or zero also -- that makes eight combinations of three bits.
There are 2 to the 100th combinations of 100 binary digits.
So a theoretician says fine, I'm at those 2 to the 100th different combinations of binary digits into evenly spaced numbers between zero and 1.
Then I will take those evenly spaced numbers between zero and 1 and I'll modulate some wave form, whatever wave form I happen to choose, and I will send that wave form.
In the absence of noise, I pick up that wave form and turn it back into my binary sequence again.
What's the conclusion from this?
The conclusion is if you don't have noise there's no constraint on how much data you can send.
I can send as much data as I want to, and there's nothing to stop me from doing it, if I don't have noise.
Yeah
So, is that where something that [UNINTELLIGIBLE] basically comes in.
Like a difference between one signal and the next signal
Somehow I have to keep these things seperate.
I don't necessarily have to separate one binary digit from the next binary digit in time.
I could, in fact, do something like creating 2 to the 100th different wave forms or I could do anything in between.
I could take each sequence of eight bits, turn them into 1 of 256 wave forms, transmit one wave form, then a little bit later transmit another wave form and so forth.
So I can split up the pie in any way I want to, and we'll talk about that a lot as we go on.
Now, I split them up into different wave forms, which I send at spaced times.
I somehow have to worry about the fact that if I send them over a finite bandwidth, there is no way in hell that you can take finite bandwidth wave forms and send them in finite time.
Anything which lasts for a finite amount of time spreads out in bandwidth.
Anything which is in a finite amount of bandwidth spreads out in time.
That's what you ought to know from 6.003.
So we're dealing with an unsoluble problem here.
Well, Nyquist back in 1928 solved that problem.
He said well no, you can't make the wave form separate, but you can make the sample separate.
People earlier had figured out they could do that with the sampling theorem, but the sampling theorem wasn't very practical, but Nyquist's way of doing it was practical.
So, in fact, you can separate these wave forms.
But you still have the problem how do you deal with the noise.
Well, one of the favorite ways of dealing with noise is to assume that it's something called white Gaussian noise.
What is white Gaussian noise?
White Gaussian noise is noise which no matter where you look it's sitting there.
You can't get away from it.
You move around to different frequencies, the noise is still there.
You move around to different times, it's still there.
It is somehow uniformed throughout time and throughout frequency.
Kind of an awkward thing because if I developed a receiver which didn't have any bandwidth constraint on it, and I looked at this noise coming in, which was spread out over all frequencies, the noise would burn out the receiver.
In other words, this white Gaussian noise has infinite power.
That shouldn't bother you too much.
You deal with unit impulses all the time.
A unit impulse has infinite energy also.
You probably never thought about it because most undergraduate courses try very hard to conceal that fact from you because they like to use impulses for everything, but impulses have infinite energy associated with them, and that's kind of a nasty thing.
But anyway, we will deal with the fact that impulses have infinite energy, and therefore, not use them to transmit data.
And we will deal with the fact that this noise has infinite power, and find out how to deal with that.
And we will learn how to understand the statistical structure of that noise.
So we'll deal with all of these things when we start studying these channels.
As we do that we will find out that no matter what you do on a channel like this, there are some fundamental constraints on how much data can be transmitted.
I can take you through a little bit of a thought experiment which will sort of explain that, I think.
It's partly what we were just starting to say over here.
A simple-minded way to look at this problem is I have binary digits coming in.
I take these binary digits and I separate them into shorts sequences of m binary digits each.
So, I take the first m binary digits, then I take the next m binary digits, and the next m binary digits and so forth.
A sequence of m binary digits, there are 2 to the m combinations of this.
So I map these 2 to the m combinations into one of 2 to the m different numbers.
Now, you look at this noise and you say I have to keep some separation between these numbers.
So, with a separation between the numbers, say the separation is 1, let this define the number 1.
That's something we theoreticians do all the time.
So I have 2 to the m different levels.
I have to send this in a certain bandwidth.
The sampling theorem says I can't send things that are more than twice the bandwidth of this system, so I have a bandwidth of 2w.
I can send m binary digits in this bandwidth, w. Therefore, I can send bits at a certain rate.
This is all very crude but it's going to lead us to an interesting trade-off.
I can increase m and by increasing m I can get by with a smaller bandwidth, because I have these symbols, these m bit symbols coming along at a slower rate.
So by increasing m, I can decrease the bandwidth, by increasing m, I can reduce the bandwidth, but by reducing the bandwidth, the power is going up.
You can see that as I change m, the power is going up exponentially with m. In other words, the trade-off between bandwidth and power is kind of nasty in this simple-minded picture.
Let me jump ahead.
For this added of white Gaussian noise with a bandwidth constraint, Shannon showed that the capacity was w times log of 1 plus the power in the system times the noise power times w. Now, I don't expect you to understand this formula now, and there's a lot of very tricky and somewhat confusing things about it.
One thing is that the power and the noise is going up with the bandwidth that you're looking at.
Because I said the bandwidth is everywhere, you can't avoid it, and therefore, if you use a wider bandwidth system, you get more noise coming into it.
If you have a certain amount of power that you're willing to transmit and a certain amount of noise, and the number of bits per second you can transmit is going up with w. But look at this affect with p here, the effect with p is logarithmic, which says as I increase this parameter m and I put more and more bits into one symbol, this quantity is going up exponentially, which means the logarithm of this is going up just linearly with m. This gives you, if you look at it carefully, the same trade-off as the simple-minded example I was talking about.
That simple-minded example does not explain this formula at all.
This formula's rather deep.
We probably won't even completely have proven this term.
What Shannon's results says is that no matter what you do, no matter how clever you are with this noise that exists everywhere, the fastest you can transmit with the most sophisticated coding schemes you can think of is this rate right here, which is what you would think it might be just from the scaling in terms of w and m, where when you increase m the power goes up exponentially.
It's the same kind of answer you get with that simple thought experiment.
The fact that you have a 1 in here is a little bit strange, and we'll find out where that comes from.
The idea here is that this noise is something you can't beat.
The noise is fundamental.
It's a fundamental part of every communication system.
As I always like to say, noise is like death and taxes, you can't avoid them, they're there, and there's nothing you can do about them.
Like death and taxes, you can take care of yourself and live longer, and with taxes, well, you can become wealthy and support the government and make all sorts of payments and reduce your taxes that way.
Or you can hire a good accountant.
So you can do things there also.
But those things are still always there.
So you're always stuck with this.
One of the major parts of the course will be to understand what this noise is in a much deeper context than we would otherwise.
So, I'll save that and come back to it later.
We have the same kind of layering in channel encoding.
When I talk about channeling encoding, I'm not talking about any kind of complicated coding technique.
6.451 talks about all of those.
What I'm talking about is simply the question of how do you take a sequence of bits, turn it into a sequence of wave forms in such a way that at the receiver we can take that sequence of wave forms and match them reliabily back into the original bits.
There's a standard way of doing this, which is to take the sequence of bits, first send them through a discrete decoder.
What the discrete encoder code will do is something like this.
And match bits at one rate into bits at a higher rate, and this let's you correct channel errors.
A simple example of this is you match zero into zero, zero, zero.
1 into 1, 1, 1.
If the rest of your system, namely, if this part of the system makes a single error at some point, this part of the system escapes from it.
I sort of put these on one slide but I couldn't.
This takes a single zero, turns it into three zeros.
These three zeros go into this modulator, which maps this into some wave from responsive to these three zeros.
Wave form goes through here, noise gets added, the modulator, the text, those binary digits, comes out with some approximation to these three zeros.
Every once in awhile because of the noise, one of these bits comes out wrong, and because they come out wrong this discrete decoder says ah-ha, I know that what was sent/put in was either zero, zero, zero or 1, 1, 1.
It's more likely to have one error than to have two errors, and therefore, any time one error occurs I can decode it correctly.
Now, that is a miserable code.
A guy by the name of Hamming became very famous for generalizing this just a little bit into another single error correcting code, which came through at a slightly higher rate but still corrected single errors.
He became inordinately famous because he was a tireless self-promoter and people think that he was the person who invented error correction coding.
He really wasn't.
And his heirs will probably -- I don't know.
Anyway, coding of this type, namely mapping binary digits into longer strings of binary digits in such a way that you can correct binary errors, has always been a major part of communication system research.
A large number of communication systems work in exactly this way.
In fact, the older systems which use coding tended to work this way.
This was a very popular way of trying to design system.
You would put a total layering across here.
Namely, this part of the system didn't know that there was any coding going on here.
At this point where you're doing discrete decoding, you don't know anything about what the detector is doing here.
It doesn't take a lot of imagination to say if you have a detector here and there's this symbol that comes through here, noise gets added to it, you're trying to decode this symbol, and there's this Gaussian noise, which is just sort of spread around, it's concentrated.
Usually when errors occur, errors occur just barely.
What you see at this point is you almost flip a coin to decide whether a zero or a 1 was transmitted at this point.
If this poor guy had that information, this poor guy could work much, much better.
Namely, this guy could correct double errors instead of single errors, because if one bit came through with a clear indication that that's what it was, and the other two bits came through saying well, I can't really tell what these are, then this guy could correct two errors instead of one error.
So this is a good example where this layering in here is really a rotten idea.
Most modern systems don't do that strict layering.
But a whole lot of modern systems do, in fact, do this binary encoding here and they use the principles that came out of the binary encoding from a long time ago.
The only extra thing they do here is they use this extra information from the demodulator.
This is something called soft decoding instead of hard decoding.
In other words, what comes out of the demodulator is soft decisions, and soft decisions say well, I think it was this but I'm not sure or I am sure and so forth.
There's a whole range of graduations.
We will study detection theory and we'll understand how that works and we'll understand how you use those soft decisions and all of that stuff and it's kind of neat.
The modulation part of this is what maps bit sequences into wave forms.
When we map bit sequences into wave form, there's this very simple idea.
If I take a single bit that's either zero or 1, I map a zero into one wave form, I map a 1 into another wave form, I send either wave form a or wave form b. I somehow want to make those wave forms as different from each other as I can.
Now, what do you mean by making wave forms different?
I know what it means to make numbers different.
I know that zero and 3 are more different than zero and 2 are.
Namely, I have a good measure of distance for a sequence of numbers.
One of the things that we have to deal with is how do you find an appropriate measure of distance for wave forms.
When I find the measure of distance for wave forms, what do I want to make it responsive to?
What I'm trying to do is to overcome this noise, and therefore, I have to find the measure of distance which is appropriate for what the noise is doing to me.
Well that's why we have to study wave forms as much as we do in this course.
What we will find is that you can treat wave forms in the same way as you can treat sequences of numbers.
We will find that there's a vector space associated with wave forms, which is exactly the same as a vector space associated with infinite sequences of numbers.
It's almost the same as the vector space associated with finite sequences of wave form, and that vector space associated with a finite sequence of numbers is exactly the vector space that you've been looking at all of your lives, and which you use primarily as a notational convenience to talk about a sequence of numbers.
What's the distance that you use there?
Well, you take the difference between each number in the sequence, you square it, you add it up and you take the square root of it.
Namely, you look at the energy difference between these sequences.
What we will find remarkably is when we do things right, the appropriate distance to talk about on wave forms, is exactly what comes from the appropriate looking at sequences.
In fact, we'll take wave forms and break them into sequences.
That's a major part of this modulation process.
You think of that in terms of the sampling theorem.
Namely, you take a bunch of numbers, you take each number in the sampling theorem and you put a little sin x over x hat around it and you transmit that, and then you add up all of these different samples, which is a sequence of numbers, each of them with these little sin x over x caps around them.
The neat thing about the sin x over x caps around them is they all go to zero with these sample points and it all works.
Therefore, the sequences look almost the same as a wave form.
We'll find out that that works in general, so we will do most of that.
Modern practice often combines all these layers into what's called coding modulation.
In other words, they go further than just this idea of soft decisions, and actually treat the problem in general of what do you do with bits coming into a sequence, into a system, how do you turn those into wave forms that can be dealt with an appropriate way.
And we'll talk just a little bit about that and mostly 6.451 talks about that.
I want to talk a little bit right at the end, I'm almost finished, about computational complexity in these communication systems that we're trying to worry about.
One of the things that you will become curious about as we move along is that we are sometimes suggesting doing things in ways that look very complex.
The word complex is an extraordinarily complex word.
None of us know what it means because it means so many different things.
I was saying that these telephone systems are inordinately complex.
In a sense they aren't because they have an incredible number of pieces, all of which are the same.
So you can understand a telephone system in terms of understanding a relatively small number of principles.
You can understand these Gaussian channels in terms of understanding a small number of principles.
So they're complex if you don't know how to look at them, they're simple if you do know how to look at them.
Here what I'm talking about is how many chips do you need to build something?
How complicated are those chips?
Fundamentally, how much does it cost to build the system?
Well, one of the curious things and one of the things that has made information theory so popular in designing communication systems is that we're almost at the point where chips are free.
Now, what does that mean?
It means if you want to do something more complicated it hardly costs anything.
This is sort of Moore's law.
Moore's law says every year you can put more and more stuff on a single chip.
You can do this cheaply and the chips work faster and faster so you can do more and more complicated things very, very easily.
There are some caveats.
Low cost with high complexity requires large volume.
In other words, it takes a very long time to design a chip, it takes an enormous amount of lead time -- I mean we're not going to study these details in the course, but I want you to be aware of why it is that in a sense complexity doesn't matter anymore.
If you spend a long enough time designing it and you can produce enough of them, you can do extraordinarily complex things very, very cheaply.
I mean you see this with all the cellular phones that you buy.
I mean cellular phones now are actually give-away devices.
You look at them to see what's in them and they're inordinately complicated.
You buy a personal computer today and it's 1,000 times more powerful than the biggest computers in the world 30 years ago.
You can do everything more cheaply now than you could before.
What's the hitch?
Well, you see the hitch when you program a computer, namely, if you look at word processing languages, yes, they become bigger and bigger every year, they become more and more complex, they will do more and more things.
As far as their function of word processing, some of us think there have been advances in the last 30 years.
Others, like myself, feel that there have not been advances and, in fact, we're going backwards.
The problem is we have so much complexity we don't know how to deal with it anymore.
Most of us have become blinking 12's.
You know what a blinking 12 is.
It's all of the electronic equipment you have in your home, whenever you don't program it right you see a 12 blinking on and off, which is where the clock is and the clock hasn't been set, and therefore, it's blinking at you.
Most people who have complicated devices, whether they're audio devices or what have you, are using less than one percent of the fancy features in them.
You spend hours and hours trying to bring that from one percent up to two percent.
So that the cost of complexity is not in what you can build, but it's in this conceptual complexity.
If you design an algorithm and you understand the algorithm, it hardly makes any difference how complicated it is, unless the complexity is going up exponentially with something.
That's the first caveat.
It's actually the second caveat, too.
Complex systems are often not well thought through.
They often don't work or are not robust.
The non-robustness is the worst part of it.
The third caveat is when you have special applications.
Since they involve small numbers, you're not going to build a lot of them, it takes a long time to design them.
The only way you can build special applications is to make them minor modifications of other things that have been done before.
So this question of how complicated the systems we can build are, if you can make enough of them they become cheap.
If you have enough lead time they become cheap.
Which says that this layering of communication system that we're talking about really ultimately makes sense.
Starting next time, we're going to start talking about this first part of source coding, which is the discrete part of source coding.
We will have those notes on the web shortly.
You can look ahead at them, if you would like to, before Monday.
Please review the probability that you are supposed to know because you will need it very shortly.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'm going to spend a couple of minutes reviewing the major things that we talked about last time and then get into discrete source coding, which is the major topic for today.
The first major thing that we talked about last time, along with all of the philosophy and all those other things, was the sense of what digital communication really is.
I said that what digital communication is, is it's communication where there's a binary interface between source and destination.
The source is very often analog.
The most interesting sources are analog.
The channel is often analog, most interesting.
Channels are analog, and we'll say more about what I mean by analog later.
What's important is that you have this binary interface between source and channel coding.
We said a little bit about why we wanted a binary interface, aside from the fact that it's there now, there's nothing you can do about it even if you don't like it.
One reason is standardization, which means it simplifies implementation, which means you can do everything in the same way.
If you have ten different kinds of channel coding and you have ten different kinds of source coding and you have a binary interface, it means you need to develop 20 different things -- ten at the source and ten at the decoder.
If you don't have that standardization with a binary interface between them, you need 100 different things.
You need to match every kind of source with every kind of destination.
That raises the price of all chips enormously.
One of the other things we said is the price of chips is very much the cost of development divided by the number of them that you stamp out.
That's not quite true, but it's a good first approximation.
In other words, standardization is important.
Layering.
Layering is in many ways very similar to standardization because this binary interface is also a layer between source and destination.
But the idea there is not that it standardizes to make things cheaper, but it simplifies the conceptualization of what's going on.
You can look at a source and only focus on one thing.
How do I take that source and turn it into the smallest number of binary digits possible?
We'll talk a good deal about what that means later because there's something stochastic involved in there and will take us awhile to really understand that.
Finally, using a binary interface loses nothing in performance.
That's what Shannon said, it's what he proved.
There's some questions there when you get to networks, but the important thing is the places where you want to study non-binary interfaces, you will never get a clue of what it is that you're looking at or why if you don't first very well understand why you want a binary interface to start with.
In other words, if you look at these other cases, there's exceptions to the rule, and if you don't know what the rule is, you certainly can't understand what the exceptions are.
So for today we're going to start out by studying this part of the problem in here.
Namely, how do you turn a source and put a general source input into binary digits that you're going to put into the channel.
How do I study this without studying that?
Well, one thing is these are binary digits here.
But the other thing is we're going to assume that what binary digits go in here come out here.
In other words, there aren't any errors.
It's an error-free system.
Part of the purpose of studying channel encoding and channel decoding is to say how is it that you get that error-free performance.
You can't quite get error-free performance, you get almost error-free performance, but the idea is when errors come out here, it's not this guy's fault, it's this guy's fault.
Therefore, what we're going to study here is how we do our job over here.
Namely, how we deal with decoding, the same string of bits that went into there and decode them coming out.
So that's where we'll be for the next three weeks or so.
We talked a little bit last time about how do you layer source coding itself.
I want to come back, because we were talking about so many things last time, and emphasize what this means a little bit.
We're going to break source coding up into three different layers again.
You start out with some kind of input wave form or image or video or whatever the heck it is.
You're going to do something like sampling it or expanding it in some kind of expansion, and we'll talk a great deal about that later.
That's not an obvious thing, how to do that.
When you finish doing that, you wind up with an analog sequence.
In other words, you wind up with a sequence of real numbers or sequence of complex numbers.
Those go into a quantizer.
What the quantizer does is to turn an uncountably infinite set of things into a finite set of things.
When you turn an uncountably infinite set of possibilities into a finite set of possibilities, you get distortion.
There's no way you can avoid it.
So that's a part of what happens there.
Then at this point you have a finite alphabet of symbols.
That goes into the discrete coder, goes through what we're now calling a reliable binary channel and comes out here.
What we're going to be studying for the next two weeks or so is this piece of the system right in here.
Again, what we're going to be doing is assuming a reliable binary channel to the right of this, which is what we already assumed.
We're going to assume that these things do whatever they have to.
But this problem here, this isolated problem is important because this is dealing with the entire problem of text, and you know what text is, it's computer files, it's English language text, it's Chinese text, it's whatever kind of text.
If we understand how to do that, we can then go on to talk about quantization because we'll have some idea of what we're trying to accomplish with quantization.
Without that we won't know what the purpose of quantization is.
Without the quantization we won't know what we're trying to accomplish over here.
There's another reason for studying this problem, which is that virtually all the ideas that come into this whole bunch of things are all tucked into this one subject in the simplest possible way.
One of the nice things about information theory, which we're going to touch on I said in this course, is that one of the reasons for studying these simple things first is that information theory is really like a symphony.
You see themes coming out, those themes get repeated, they get repeated again with more and more complexity each time, and when you understand the simple idea of the theme, then you understand what's going on.
So, that's the other reason for dealing with that.
To summarize those things -- most of this I already said.
Examples of analog sources are voice, music, video, images.
We're going to restrict this to just wave form sources, which is voice and music.
In other words, an image is something where you're mapping from two dimensions this way and this way into a sequence of binary digits.
So it's a mapping after you get done sampling from r square, which is this axis and this axis, into your outut.
Namely, for each point in this plane, there's some real number that represents the amplitude at that point.
Video is a three-dimensional to one-dimensional thing, namely, you have time.
You also have this way, you have this way, so you're mapping from r cubed into r. We're not going to deal with those because really all the ideas are just contained in dealing with wave form sources.
In other words, the conventional functions that you're used to seeing.
Namely, things that you can draw on a piece of paper and you can understand what's going on with them.
These are usually samples or expanded into series expansions almost invariably, and we'll understand why later.
That, in fact, is a major portion of the course.
That's where all of the stuff from signals and systems come in.
We'll have to expand that a whole lot because you didn't learn enough there.
We need a lot of other things, and that's what we need to deal with wave forms.
We'll take the sequence of numbers that comes out of the sampler.
We're then going to quantize that sequence of numbers.
That's the next thing we're going to study.
Then we're going to get into analog and discrete sources, which is the topic we will study right now.
So we're going to study this.
After we get done this, we're going to study this also.
When we study this, we'll have what we know about this as a way of knowing how to deal with the whole problem from here out to here.
Finally, we'll deal with wave forms and deal with the whole problem from here out to here.
So that's our plan.
In fact, this whole course is devoted to studying this problem, then this problem, then this problem -- that's the source part of the course.
Then dealing with -- if I can find it again -- with the various parts of this problem.
So first we study sources, then we study channels.
Because of the binary interface, when we're all done with that we understand digital communication.
When we get towards the end of the term we'll be looking at more sophisticated kinds of channels than we look at earlier, which are really models for wireless channels.
So that's where we're going to end up.
So discrete source coding, which is what we want to deal with now.
What's the objective?
We're going to map a sequence of symbols into a binary sequence and we're going to do it with unique decodability.
I'm not going to define unique decodability at this point.
I'm going to define it a little bit later.
But roughly what it means is this.
We have a sequence of symbols which come into the encoding encoder.
They go through this binary channel.
They come out as a sequence of binary digits.
Unique decodability says if this guy does his job, can this guy do his job?
If this guy can always do his job when these digits are correct, then you have something called unique decodability.
Namely, you can guarantee that whatever comes out here, whatever comes in here, will turn into a sequence of binary digits.
That sequence of binary digits goes through here.
These symbols are the same as these symbols.
In other words, you are reproducing things error-free if, in fact, this reproduces things error-free.
So that's our objective.
There's a very trivial approach to this, and I hope all of you will agree that this is really, in fact, trivial.
You map each source symbol into an l-tuple of binary digits.
If you have an alphabet of size m, how many different binary strings are there of length l?
Well, there are 2 to the l of them.
If l is equal to 2, you have 0, 0, 0, 1, 1, 0, and 1, 1.
If l is equal to 3, you have strings of 3, which is 0, 0, 0, 0, 0, 1, 0, 1, 0, blah, blah, blah, blah, blah.
What comes out to be 2 to the 3, which is equal to 8.
So what we need if we're going to use this approach, which is the simplest possible approach, which is called the fixed length approach, is you need the alphabet size to be less than or equal to the number of binary digits that you use in these strings.
Now, is that trivial or isn't it trivial?
I hope it's trivial.
We don't want to waste bits when we're doing this, particularly, so we don't want to make l any bigger than we have to, because for every symbol that comes in, we get l symbols coming out.
So we'd like to minimize l subject to this constraint that 2 to the l has to be bigger, greater than or equal to m. So, what we want to do is we want to choose l as the smallest integer which satisfies this.
In other words, when you take the logarithm to the base 2 of this, you get log to the base 2 of m has to be less than or equal to l, and l is then going to be less than log to the base 2 of m plus 1.
This is the constraint which says you don't make l any bigger than you have to make it.
So in other words, we're going to choose l equal to the ceiling function of log to the base m. In other words, this is the integer which is greater than or equal to log to the base 2 of m. So let me give you a couple of examples of that.
Excuse me for boring you with something which really is trivial, but there's notation here you have to get used to.
You get confused with this because there's the alphabet size which we call m, there's the string length which we call l, and you keep getting mixed up between these two.
Everybody gets mixed up between them.
I had a doctoral student the other day who got mixed up in it, and I read what she had written four times and I didn't catch it either.
So this does get confusing at times.
If you have an alphabet, which is five different kinds of the letter a -- that's one reason why these source codes get messy, you have too many different kinds of each letter, which technical people who like a lot of jargon use all of them.
In fact, when people start writing papers and books you find many more than five there.
In terms of Latex, you get math cow, you get math gold, you get math blah, blah, blah.
Everything in little and big.
You get the Greek version.
You get the Roman version and the Arabic version, if you're smart enough to know that language, those languages.
What we mean by code is alpha gets mapped into 0, 0, 0. a gets mapped into 0, 0, 1.
Capital A into this and so forth.
Does it make any difference what mapping you use here?
Can you find any possible reason why it wouldn't make a difference whether I map alpha into 0, 0, 0, and a into 0, 0, 1 or vice versa?
I can't find any reason for that.
Would it make any difference of instead of having this alphabet I had beta b, capital B, script b, and capital B with a line over it?
I can't see any reason why that would make a difference either.
In other words, when we're talking about fixed length codes, there are only two things of importance.
One of them is how big is the alphabet -- that's why we talk about alphabets all the time.
After you know how big the alphabet is and after you know you want to do a fixed length binary encoding, then you just assign a binary string to each of these letters.
In other words, there's nothing important in these symbols.
This is a very important principle of information theory.
It sort of underlines the whole subject.
I'm not really talking about information theory here, as I said, we're talking about communication.
But communication these days is built on these information theoretic ideas.
Symbols don't have any inherent meaning.
As far as communication is concerned, all you're interested in is what is the set of things -- I could call this a1, a2, a3, a4, a5, and we're going to start doing this after awhile because we will recognize that the name of the symbols don't make any difference.
If you listen to a political speech if it's by a Republican there are n different things they might say, and you might as well number them a1 to a sub n. If you listen to one of the Democratic candidates there are m different things they might say.
You can number them 1 to m, and you can talk to other people about it and say oh, he said a1 today, which is how do we get out of the war in Iraq.
Or he said number 2 today, which is we need more taxes or less taxes and so forth.
So it's not what they say as far as communication is concerned, it's just distinguishing the different possible symbols.
So, you can easily decode this -- you see three bits and you decode them.
Can I?
Is this right or is there something missing here?
Of course, there's something missing.
You need synchronization if you're going to do this.
If I see a very long string of binary digits and I'm going to decode them into these letters here, I need to know where the beginning is.
In other words, if it's a semi-infinite string of binary digits, I don't know how to look at it.
So, inherently, we believe that somebody else gives us synchronization.
This is one of these things we always assume.
When you start building a system after you decide how to do this kind of coding, somebody at some point has to go through and decide where do you get the synchronization from.
But you shouldn't think of the synchronization first.
If I'm encoding 10 million symbols and it takes me 1,000 bits to achieve the synchronization, that 1,000 bits gets amortized over 10 million different symbols, and therefore, it doesn't make any difference, and therefore, we're going to ignore it.
It's an important problem but we ignore it.
The ASCII code is a more important example in this.
It was invented many, many years ago.
It was a mapping from 256 different symbols which are all the letters, all the numbers, all the things that people used on typewriters.
Anybody remember what a typewriter is?
Well, it's something people used to use before they had computers, and these typewriters had a lot of different keys on them and they had a lot of special things you could do with them.
And somebody dreamed up 256 different things that they might want to do.
Why do they use l equals 8?
Nothing to do with communication or with information theory or with any of these things.
It was that 8 is a nice number.
It's 2 to the 3.
In other words, this was a standard length of both computer words and of lots of other things.
Everybody likes to deal with 8 bits, which you call a byte, rather than 7 bits which is sort of awkward or 6 bits which was an earlier standard, which would have been perfectly adequate for most things that people wanted.
But no, they had to go to 8 bits because it just sounded nicer.
These codes are called fixed length codes.
I'd like to say more about them but there really isn't much more to say about them.
There is a more general version of them, which we'll call generalized fixed length codes.
The idea there is to segment the source sequence.
In other words, we're always visualizing now having a sequence of symbols which starts at time zero, runs forever.
We want to segment that into blocks of length n. Namely, you pick off the first n symbols, you find the code word for those n symbols, then you find the code word for the next n symbols, then you find the code word for the next n symbols and so forth.
So it's really the same problem that we looked at before.
It's just that the alphabet before had the number of symbols as the alphabet size.
Now, instead of having an alphabet size which is m, we're looking at blocks of m symbols and how many possible combinations are there of blocks where every symbol is one of m different things.
Well, if you have two symbols, the first one can be any one of m things, the second one can be any one of m things.
So there are m squared possible combinations for the first two symbols, there are m cubed possible combinations for the first three symbols and so forth.
So we're going to have an alphabet on blocks of m to the n different n tuples of source letters.
Well, once you see that we're done because what we're going to do is find a binary sequence for every one of these blocks of m to the n symbols.
As I said before, the only thing important is how many are there.
It doesn't matter that they're blocks or that they're stacked this way or that they're stacked around in a circle or anything else.
All you're interested in is how many of them are there.
So there are m to the n of them.
So, what we want to do is make the binary length that we're dealing with equal to log to the base 2 of m to the n, the ceiling function of that.
Which says log to the base 2 of m is less than or equal to l bar where l bar is going to be the bits per source symbol.
I'm going to abbreviate that bits per source symbol.
I would like to abbreviate it bps, but I and everyone else will keep thinking that bps means bits per second.
We don't have to worry about seconds here, seconds had nothing to do with this problem.
We're just dealing with sequences of things and we don't care how often they occur.
They might just be sitting in a computer file and we're doing them offline, so seconds has nothing to do with this problem.
So, log to the base 2 of m is less than or equal to l over n, which is less than log to the base 2 of m plus 1 over n. In other words, we're just taking this dividing it by n, we're taking this dividing by n, the ceiling function is between log to the base 2 of m to the n, and log to the base 2 of m to the n plus 1.
When we divide by n, that 1 becomes 1 over n. What happens when you make n large?
l approaches log to the base 2 of m from above.
Therefore, fixed length coding requires log to the base 2 of n bits per source symbol if, in fact, you make n large enough.
In other words, for the example of five different kinds of a's, we had m equal to 5.
So if you have m equal to 5, that leads to m squared equals 25, that leads to l equals -- what's the ceiling function of log of this?
It's 5. l bar is equal to -- what's half of 5?
2 and 1/2, yes.
As you get older you can't do arithmetic anymore.
So look what we've accomplished.
We've gone from three bits per symbol down to two and and half bits per symbol, isn't that exciting?
Well, you look at it and you say no, that's not very exciting.
I mean yes, you can do it, but most people don't do that.
So why do we bother with this?
Well, it's the same reason we bother with a lot of things in this course, and the whole first two weeks of this course will be dealing with things where when you look at them and you ask is this important, you have to answer no, it's not important, it doesn't really have much to do with anything, it's a mathematical idea.
What it does have to do with is the principle involved here is important.
It says that the lower limit of what you can do with fixed coding is log to the base 2 of m. You have an alphabet of size m, you can get as close to this as you want to.
We will find out later that if you have equally likely symbols when we get to talking about probability, we will find out that nothing in the world can do any better than this.
That's the more important thing, because what we're eventually interested in is what's the best you can do if you do things very complicated.
Why do you want to know what the best is if you do something very complicated?
Because if you can do that simply then you know you don't have to look any further.
So that's the important thing.
Namely, it lets you do something simple and know that, in fact, what you're doing makes sense.
That's why we do all of that.
But then after we say well there's no place else to go on fixed length codes, we say well, let's look at variable length codes.
The motivation for variable length codes is that probable symbols should probably have shorter code words than very unlikely symbols.
And Morse thought of this a long, long time ago when Morse code came along.
Probably other people thought of it earlier, but he actually developed the system and it worked.
Everyone since then has understood that if you have a symbol that only occurs very, very, very rarely, you would like to do something, make a code word which is very long for it so it doesn't interfere with other code words.
Namely, one of the things that you often do when you're developing a code is think of a whole bunch of things which are sort of exceptions.
They hardly ever happen.
You use the fixed length code for all the things that happen all the time, and you make one extra code word for all these exceptions.
Then you have this exception and paste it on at the end of the exception is a number which represents which exception you're looking at.
Presto, you have a variable length code.
Namely, you have two different possible code lengths -- one of them for all of the likely things and the indication that there is an exception, and two, all the unlikely things.
There's an important feature there.
You can't drop out having the code word saying this is an exception.
If you just have a bunch of short code words and a bunch of long code words, then you see a short code word and you don't know -- well, if you see a long code word starting or you have a short code word, you don't know which it is and you're stuck.
So one example of a variable length code -- we'll use some jargon here.
We'll call the code a script c. We'll think of script c as a mapping which goes from the symbols onto binary strings.
In other words, c of x is the code word corresponding to the symbol x.
So for each x in the alphabet, capital X, and we have to think of what the capital X is.
But as we say, the only thing we're really interested in is how big is this alphabet -- that's the only thing of importance.
So if we have an alphabet which consists of the three letters a, b and c, we might make a code where the code word for a is equal to zero, the code word for b is equal to 1, zero, and the code word for c is equal to 1,1.
Now it turns out that's a perfectly fine code and that works.
Let me show you another example of a code.
Let me just show you an example of a code here so we can see that not everything works.
Suppose c of a is zero, c of b is 1, and c of c is -- this is a script c, that's a little c -- is 1, zero.
Does that work?
Well, all of the symbols have different code words, but this is an incredibly stupid thing to do.
It's an incredibly stupid thing to do because if I send a b followed by an a, what the poor decoder sees is 1 followed by zero.
In other words, one of the things that I didn't tell you about is when we're using variable length codes we're just concatenating all of these code words together.
We don't put any spaces between them.
We don't put any commas between them.
If, in fact, I put a space between them, I would really have not a binary alphabet but a ternary alphabet.
I would have zeros and I would have 1's and I would have spaces, and you don't like to do that because it's much harder.
When we start to study channels we'll see that ternary alphabets are much more difficult to work with than binary alphabets.
So this doesn't work, this does work.
Part of what we're going to be interested in is what are the conditions under why this works and why this doesn't work.
Again, when you understand this problem you will say it's very simple, and then you come back to look at it again and you'll say it's complicated and then it looks simple.
It's one of these problems that looks simple when you look at it in the right way, and it looks complicated when you get turned around and you look at it backwards.
So the success of code words of a variable length code are all transmitted just as a continuing sequence of bits.
You don't have any of these commas or spaces in them.
If I have a sequence of symbols which come into the encoder, those get mapped into a sequence of bits, variable length sequences of bits which come out.
They all get pushed together and just come out one after the other.
Buffering can be a problem here, because when you have a variable length code -- I mean look at what happens here.
If I've got a very long string of a's coming in, I got a very short string of bits coming out.
If I have a long string of b's and c's coming in, I have a very long string of bits coming out.
Now usually the way the channels work is that you put in bits at a fixed rate in time.
Usually the way that sources work is that symbols arrive at a fixed rate in time.
Therefore, here, if symbols are coming in at a fixed rate in time, they're going out at a non-fixed rate in time.
We have to bring them into a channel at a fixed rate in time, so we need a buffer to take care of the difference between the rate at which they come out and the rate at which they go in.
We will talk about that problem later, but for now we just say OK, we have a buffer, we'll put them all in a buffer.
If the buffer ever empties out -- well, that's sort of like the problem of initial synchronization.
It's something that doesn't happen very often, and we'll put some junior engineer on that because it's a hard problem, and seeing your engineers never deal with the hard problems, they always give those to the junior engineers so that they can assert their superiority over the junior engineers.
It's a standard thing you find in the industry.
We also require unique decodability.
Namely, the encoded bit stream has to be uniquely deparsed at the decoder.
I have to have some way of taking that long string of bits and figuring out where the commas would have gone if I put commas in it.
Then from that I have to decode things.
In other words, it means that every symbol in the alphabet has to have a distinct code word connected with it.
We have that here.
We have that here.
Every symbol has a distinct code word.
But it has to be more than that.
I'm not even going to talk about precisely what that more means for a little bit.
We also assume to make life easy for the decoder that it has initial synchronization.
There's another obvious property that we have.
Namely, both the encoder and the decoder know what the code is to start with.
In other words, the code is built into these devices.
When you design a coder and a decoder, what you're doing is you figure out what an appropriate code should be, you give it to both the encoder and the decoder, both of them know what the code is and therefore, both of them can start decoding.
A piece of confusion.
We have an alphabet here which has a list of symbols in it.
So there's a symbol a1, a2, a3, up to a sub m. We're sending a sequence of symbols, and we usually call the sequence of symbols we're sending x1, x2, x3, x4, x5 and so forth.
The difference is the symbols in the alphabet are all distinct, we're listing them one after the other.
Usually there's a finite number of them.
Incidentally, we could have a countable number of symbols.
You could try to do everything we're doing here say with the integers, and there's a countable number of integers.
All of this theory pretty much carries through with various little complications.
We're leaving that out here because after you understand what we're doing, making it apply to integers is straightforward.
Putting in the integers to start with, you'll always be fussing about various silly little special cases, and I don't know a single situation where anybody deals with a countable alphabet, except by truncating it.
When you truncate an infinite alphabet you get a finite alphabet.
So, we'll assume initial synchronization, we'll also assume that there's a finite alphabet.
You should always make sure that you know whether you're talking about a listing of the symbols in the alphabet or a listing of the symbols in a sequence.
The symbols in a sequence can all be the same, they can all be different.
They can be anything at all.
The listing of symbols in the alphabet, there's just one for each symbol.
We're going to talk about a very simple case of uniquely decodable codes which are called prefix-free codes.
A code is prefix-free if no code word is a prefix of any other code word.
In other words, a code word is a string of binary digits.
A prefix of a string of binary digits.
For example, if we have the binary string 1, 0, 1, 1, 1.
What are the prefixes of that?
Well, one prefix is 1, 0, 1, 1.
Another one is 1, 0, 1.
Another one is 1, 0.
Another is 1.
In other words, it's what you get by starting out at the beginning and not quite getting to the end.
All of these things are called prefixes.
If you want to be general you could call 1, 0, 1, 1, 1, a prefix of itself.
We won't bother to do that because it just is -- that's the kind of things that mathematicians do to save a few words in the proofs that they give and we won't bother with that.
We will rely a little more on common sense.
Incidentally, I prove a lot of things in these notes here.
I will ask you to prove a lot of things.
One of the questions that people always have is what does a proof really mean?
I mean what is a proof and what isn't a proof?
When you take mathematics courses you get one idea of what a proof is, which is appropriate for mathematics courses.
Namely, you prove things using the correct terminology for proving them.
Namely, everything that you deal with you define it ahead of time so that all of the terminology you're using all has correct definitions.
Then everything should follow from those definitions and you should be able to follow a proof through without any insight at all about what is going on.
You should be able to follow a mathematical proof step-by-step without knowing anything about what this is going to be used for, why anybody is interested in it or anything else, and that's an important thing to learn.
That's not what we're interested in here.
What we're interested in here for a proof -- I mean yes, you know all of the things around this particular proof that we're dealing with, and what you're trying to do is to construct a proof that covers all possible cases.
You're going to use insight for that, you're going to use common sense, you're going to use whatever you have to use.
And eventually you start to get some sort of second sense about when you're leaving something out that really should be there.
That's what we're going to be focusing on when we worry about trying to be precise here.
When I start proving things about prefix codes, I think you'll see this because you will look at it and say that's not a proof, and, in fact, it really is a proof.
Any good mathematician would look at it and say yes, that is a proof.
Bad mathematicians sometimes look at it and say well, it doesn't look like proof so it can't be a proof.
But they are.
So here we have prefix-free codes.
The definition is no code word is a prefix of any other code word.
If you have a prefix-free code, you can express it in terms of a binary tree.
Now a binary tree starts at a root, this is the beginning, moves off to the right -- you might have it start at the bottom and move up or whatever direction you want to go in, it doesn't make any difference.
If you take the zero path you come to some leaf.
If you take the one path you come to some intermediate node here.
From the intermediate node, you either go up or you go down.
Namely, you have a 1 or a zero.
From this intermediate node you go up and you go down.
In other words, a binary tree, every node in it is either an intermediate node, which means there are two branches going out from it, or it's a leaf which means there aren't any branches going out from it.
You can't, in a binary tree, have just one branch coming out of a node.
There are either no branches or two branches, just by definition of what we mean by a binary tree -- binary says two.
So, here this tree corresponds where we label various ones of the leafs.
It corresponds to the code where a corresponds to the string zero, b corresponds to the string 1, 1, and c corresponds to the string 1, 0, 1.
Now you look at this and when you look at the tree, when you look at this as a code, it's not.
Obvious that it's something really stupid about it.
When you look at the tree, it's pretty obvious that there's something stupid about it, because here we have this c here, which is sitting off on this leaf, and here we have this leaf here which isn't doing anything for us at all.
We say gee, we could still keep this prefix condition if we moved this into here and we drop this off.
So any time that there's something hanging here without corresponding to a symbol, you would really like to shorten it.
When you shorten these things and you can't shorten anything else, namely, when every leaf has a symbol on it you call it a full tree.
So a full tree is more than a tree, a full tree is a code tree where the leaves correspond to symbols.
So a full tree has no empty leaves.
Empty leaves can be shortened just like I showed you here, so we'll talk about full trees, and full trees are sort of the good trees.
But prefix-free codes don't necessarily have to worry about that.
Well, now I'm going to prove something to you, and at this point you really should object, but I don't care.
We will come back and you'll get straightened out on it later.
I'm going to prove that prefix-free codes are uniquely decodable, and you should cry foul because I really haven't defined what uniquely decodable means yet.
You think you know what uniquely decodable means, which is good.
It means physically that you can look at a string of code words and you can pick out what all of them are.
We will define it later and you'll find out it's not that simple.
As we move on, when we start talking about Lempel Ziv codes and things like that.
You will start to really wonder what uniquely decodable means.
So it's not quite as simple as it looks.
But anyway, let's prove that prefix-free codes are uniquely decodable anyway, because prefix-free codes are a particularly simple example of uniquely decodable codes, and it's sort of clear that you can, in fact, decode them because of one of the properties that they have.
The way we're going to prove this is we want to look at a sequence of symbols or a string of symbols that come out of the source.
As that string of symbols come out of the source, each symbol in the string gets mapped into a binary string, and then we concatenate all those binary strings together.
That's a big mouthful.
So let's look at this code we were just talking about where the code words are b, c and a.
So if a 1 comes out of the source and then another 1, it corresponds to the first letter b.
If a 1, zero comes out, it corresponds to the first letter c. If a zero comes out, that corresponds to the letter a.
Well now the second symbol comes in and what happens on that second symbol is if the first symbol was an a, the second symbol could be a b or a c or an a, which gives rise to this little sub-tree here.
If the first letter is a b, the second letter could be either an a, b or a c, which gives rise to this little sub-tree here.
If we have a c followed by anything, that gives rise to this little sub-tree here.
You can imagine growing this tree as far as you want to, although it gets hard to write down.
How do you decode this?
Well, as many things, you want to start at the beginning, and we know where the beginning is.
That's a basic assumption on all of this source coding.
So knowing where the beginning is, you sit there and you look at it, and you see a zero as the first letter as a first binary digit, and zero says I move this way in the tree, and presto, I say gee, an a must have occurred as the first source letter.
So what do I do?
I remove the a, I print out a, and then I start to look at this point.
At this point I'm back where I started at, so if I can decode the first letter, I can certainly decode everything else.
If the first letter is a b, what I see is a 1 followed by a 1.
Namely, when I see the first binary 1 come out of the channel, I don't know what was said.
I know either a b or c was sent.
I have to look at the second letter, the second binary digit resolves my confusion.
I know that the first source letter was in a b, if it's 1 1, or a c, if it's 1 zero.
I decode that first source letter and then where am I?
I'm either on this tree or on this tree, each of which goes extending off into the wild blue yonder.
So this says if I know where the beginning is, I can decode the first letter.
But if I can decode the first letter, I know where the beginning is for everything else.
Therefore, I can decode that also.
Well, aside from any small amount of confusion about what uniquely decodable means, that's a perfectly fine mathematical proof.
So, prefix-free codes are, in fact, uniquely decodable and that's nice.
So then there's a question.
What is the condition on the lengths of a prefix-free code which allow you to have unique decodability?
The Kraft inequality is a test on whether there are prefix-free codes or there are not prefix-free codes connected with any given set a code word lengths.
This is a very interesting enough inequality.
This is one of the relatively few things in information theory that was not invented by Claude Shannon.
You sit there and you wonder why didn't Claude Shannon realize this?
Well, it's because I think he sort of realized that it was trivial.
He sort of understood it and he was really eager to get on to the meat of things, which is unusual for him because he was somebody, more than anyone else I know, who really understood why you should understand the simple things before you go on to the more complicated thing.
But anyway, he missed this.
Bob Fano, who some of you might know, who was a professor emeritus over in LCS, was interested in information theory.
Then he was teaching a graduate course back in the '50s here at MIT, and as he often did, he threw out these problems and said nobody knows how to figure this out.
How kinds of lengths can you have on prefix-free codes, and what kinds of lengths can't you have?
Kraft was a graduate student at the time.
The next day he came in with this beautiful, elegant proof and everybody's always known who Kraft is ever since then.
Nobody's ever known what he did after that.
But at least he made his mark on the world as a graduate student.
So, in a sense, those were good days to be around, because all the obvious things hadn't been done yet.
But the other thing is you never know what the obvious things are until you do them.
This didn't look like an obvious problem ahead of time.
Don't talk about a number of other obvious things that cuts off, because somebody was looking at it in a slightly different way than other people were looking at it.
You see, back then people said we want to look at these variable length codes because we want to have some capability of mapping improbable symbols into long code words and probable symbols into short code words.
You'll notice that I've done something strange here.
That was our motivation for looking at variable length codes, but I haven't said a thing about probability.
All I'm dealing with now is the question of what is possible and what is not possible.
We'll bring in probability later, but now all we're trying to figure out is what are the sets of code word lengths you can use, and what are the sets of code word lengths you can't use.
So what Kraft said is every prefix-free code for an alphabet x with code word lengths l of x for each letter in the alphabet x satisfies the sum 2 to the minus length less than or equal to 1.
In other words, you take all of the code words in the alphabet, you take the length of each of those code words, you take 2 to the minus l of that length.
And if this inequality is not satisfied, your code does not satisfy the prefix condition, there's no way you can create a prefix-free code which has these lengths, so you're out of luck.
So you better create a new set of lengths which satisfies this inequality.
There's also a simple procedure you can go through which lets you construct a code which has these lengths.
So, in other words, this, in a sense, is a necessary and sufficient condition on the possibility of constructing codes with a particular set of lengths.
It has nothing to do with probability.
So it's, in a sense, cleaner than these other results.
So, conversely, if this inequality is satisfied, you can construct a prefix-free code, and even more strangely, you can construct it very, very easily, as we'll see.
Finally, a prefix-free code is full -- you remember what a full prefix-free code is?
It's a code where the tree has nothing that's unused if and only if this inequality is satisfied with a quality.
So it's a neat result.
It's useful in a lot of places other than source coding.
If you ever get involved with designing protocols for computer networks or protocols for any kind of computer communication, you'll find that you use this all the time, because this says you can do some things, you can't do other things.
So let's see why it's true.
I'll give you another funny proof that doesn't look like a proof but it really is.
What I'm going to do is to associate code words with base 2 expansions.
There's a little Genie that early in the morning leaves things out of these slides when I make them.
It wasn't me, I put it in.
So we're going to prove this by associating code words with base 2 expansions, which are like decimals, but decimals to the base 2.
In other words, we're going to take a code word, y1, y2 up to y sub m where y1 is a binary digit, y2 is a binary digit.
This is a string of binary digits, and we're going to represent this as a real number.
The real number is the decimal, but it's not a decimal, it's a becimal, if you will, which is dot y1, y2 up to y sub m. Which means y1 over 2 plus y2 over 4 plus dot dot dot plus y sub m over 2 to the minus m. If you think of it, an ordinary becimal, y1, y2 up to y sub m, means y1 over 10 plus y2 over 100 plus y3 over 1,000 and so forth.
So this is what people would have developed for decimals if, in fact, we lived in a base 2 world instead of a base 10 world.
If you were born without fingers and you only had two fingers, this is the number system you would use.
When we think about decimals there's something more involved.
We use decimals all the time to approximate things.
Namely, if I say that a number is 0.12, I don't mean usually that it's exactly 12 one hundredths.
Usually I mean it's about 12 one hundredths.
The easiest way to do this is to round things down to two decimal points.
In other words, when I say 0.12, what I really mean is I am talking about a real number which lies between 12 one hundredths and 13 one hundredths.
It's greater than or equal to 12 one hundredths and it's less than 13 one hundredths.
I'll do the same thing in base 2.
As soon as I do this you'll see where the Kraft inequality comes from.
So I'm going to have this interval here, which the interval associated with a binary expansion to m digits, there's a number associated with it which is this number here.
There's also an interval associated with it, which is 2 to the minus m. So if I have a code consisting of 0, 0, 0, 1 and 1, what I'm going to do is represent zero zero as a binary expansion, so 0, 0, is a binary expansion is 0.00, which is zero.
But also as an approximation it's between zero and 1/4.
So I have this interval associated with 0, 0, which is the interval from zero up to 1/4.
For the code word zero 1, if I'm trying to see whether that is part of a prefix code, I map it into a number, 0.01 as a binary expansion.
This number corresponds to the number 1/4, and it also corresponds into sub length 2 to an interval of size 1/4.
So we go from 1/4 up to 1/2.
Finally, I have 1, which corresponds to the number 1/2, and since it's only one binary digit long, it corresponds to the interval 1/2 to 1.
Namely, if I truncate thing to one binary digit, I'm talking about the entire interval from 1/2 to 1.
So where does the Kraft inequality come from and what does it have to do with this?
Incidentally, this isn't the way that Kraft proved it.
Kraft was very smart.
He did this as his Master's thesis, too, I believe, and since he wanted it to be his Master's thesis he didn't want to make it look quite that trivial or Bob Fano would have said oh, you ought to do something else for a Master's thesis also.
So he was cagey and made his proof look a little more complicated.
So, if a code word x is a prefix of code word y, in other words, y has some binary expansion, x has some binary expansion which is the first few letters of y.
Then the number corresponding to x and the interval corresponding to x, namely, x covers that entire range of decimal expansions which start with x and goes up to something which differs from x only in that mth binary digit.
In other words, let me show you what that means in terms of here.
If I tried to create a code word 0, 0, 0, 1, 0, 0, 0, 1 would correspond to the number 1/16.
1/16 lies in that interval there.
In other words, any time I create a code word which lies in the interval corresponding to another code word, it means that this code word has a prefix of that code word.
Sure enough it does -- 0, 0, 0, 1, this has this as a prefix.
In other words, there is a perfect mapping between intervals associated with code words and prefixes of code words.
So in other words, if we have a prefix-free code, the intervals for each of these code words has to be distinct.
Well, now we're in nice shape because we know what the size of each of these intervals is.
The size of the interval associated with a code word of length 2 is 2 to the minus 2.
To be a prefix-free code, all these intervals have to be disjoint.
But everything is contained here between zero and 1, and therefore, when we add up all of these intervals we get a number which is at most 1.
That's the Kraft inequality.
That's all there is to it.
There was one more thing in it.
It's a full code if and only if the Kraft inequality is satisfied with a quality.
Where was that?
The code is full if and only if the expansion intervals fill up zero and 1.
In other words, suppose this was 1 zero, which would lead into 0.1 with an interval 1/2 to 3/4, and this was all you had, then this interval up here would be empty, and, in fact, since this interval is empty you could shorten the code down.
In other words, you'd have intervals which weren't full which means that you would have code words that could be put in there which are not there.
So, that completes the proof.
So now finally, it's time to define unique decodability.
The definition in the notes is a mouthful, so I broke it apart into a bunch of different pieces here.
A code c for a discrete source is uniquely decodable if for each string of source letters, x1 up to x sub m, these are not distinct letters of the alphabet, these are just the things that might come out of the source.
x1 could be the same as x2, it could be different from x2.
If all of these letters coming out of the source, that corresponds to some concatenation of these code words, namely, c of x1, c of x2 up to c of x sub m. So I have this coming out of the source, this is a string of binary digits that come out corresponding to this, and I require that this differs from the concatenation of the code words c of x1 prime up to c of xm prime.
For any other string, x1 prime x2 prime, x of m prime of source letters.
Example of this, the thing that we were trying to construct before c of a equals 1c of b equals zero, c of c equals 1 zero, doesn't work because the concatenation of a and b yields 1 zero, c of x1 -- take x1 to be a, take x2 to be b.
This concatenation, c of x1, c of x2 is c of a, c of b equals 1 zero.
C of c equals 1 zero, and therefore, you don't have something that works.
Note that n here can be different from m here.
You'll deal with that in the homework a little bit, not this week's set.
But that's what unique decodability says.
Let me give you an example.
Here's an example.
Turns out that all uniquely decodable codes have to satisfy the Kraft inequality also.
Kraft didn't prove this.
In fact, it's a bit of a bear to prove it, and we'll prove it later.
I suspect that about 2/3 of you will see the proof and say ugh, and 1/3 of you will say oh, this is really, really interesting.
I sort of say gee, this is interesting sometimes, and more often I say ugh, why do we have to do this?
But one example of a code which is uniquely decodable is first code word is 1, second code word is 1, 0, third is 1, 0, 0, and the fourth is 1, 0, 0, 0.
It doesn't satisfy the Kraft inequality with the quality, it satisfies it with inequality.
It is uniquely decodable.
How do I know it's uniquely decodable by just looking at it?
Because any time I see a 1 I know it's the beginning of a code word.
So I look at some along binary string, it starts out with the 1, I just read digits till it comes to the next one, I say ah-ha, that next 1 is the first binary digit in the second code word, the third 1 that I see is the first digit in the third code word and so forth.
You might say why don't I make the 1 the end of the code word instead of the beginning of the code word and then we'll have the prefix condition again.
All I can say is because I want to be perverse and I want to give you an example of something that is uniquely decodable but doesn't satisfy the Kraft inequality.
So it's a question.
Why don't we just stick the prefix-free codes and forget about unique decodability?
You won't understand the answer to that really until we start looking at things like Lempel Ziv codes, which are, in fact, a bunch of different things all put together which are, in fact, very, very practical codes.
But they're not prefix-free codes, and you'll see why they're not prefix-free codes when we study them.
Then you will see why we want to have a definition of something which is more involved than that.
So don't worry about that for the time being.
For the time being, the correct idea to take away from this is that why not just use prefix-free codes, and the answer is for quite a while we will.
We know that anything we can do with prefix-free codes we can also do with uniquely decodable codes, anything we can do with uniquely decodable codes, we can do with prefix-free codes.
Namely, any old code that you invent has like certain set of lengths associated with the code words, and if it satisfies the Kraft inequality, you can easily develop a prefix-free code which has those lengths and you might as well do it because then it makes the coding a lot easier.
Namely, if we have a prefix-free code -- let's go back and look at that because I never mentioned it and it really is one of the important advantages of prefix-free codes.
When I look at this picture and I look at the proof of how I saw that this was uniquely decodable, what we said was you start at the beginning and as soon as the decoder sees the last binary digit of a code word, the decoder can say ah-ah, it's that code word.
So it's instantaneously decodable.
In other words, all you need to see is the end of the code word and at that point you know it's the end.
Incidentally, that makes figuring out when you have a long sequence of code words and you want to stop the whole thing, it makes things a little bit easier.
This example we started out with of -- I can't find it anymore -- but the example of a uniquely decodable, but non-prefix-free code, you always had to look at the first digit of the next code word to know that the old code word was finished.
So, prefix-free codes have that advantage also.
The next topic that we're going to take up is discrete memoryless sources.
Namely, at this point we have gone as far as we can in studying prefix-free codes and uniquely decodable codes strictly in terms of their non-probabalistic properties.
Namely, the question of what set of lengths can you use in a prefix-free code or uniquely decodable code, and what sets of lengths can't you use.
So the next thing we want to do is to start looking at the probabilities of these different symbols and looking at the probabilities of the different symbols.
We want to find out what sort of lengths we want to choose.
There will be a simple answer to that.
In fact, there'll be two ways of looking at it, one of which will lead to the idea of entropy, and the other which will lead to the idea of generating an optimal code.
Both of those approaches are extremely interesting.
But to do that we have to think about a very simple kind of source.
The simple kind of source is called a discrete memoryless source.
We know what a discrete source is -- it's a source which spews out a sequence of symbols from this finite alphabet that we know and the decoder knows.
The next thing we have to do is to put a probability measure on the output of the source.
There's a little review of probability at the end of this lecture.
You should read it carefully.
When you study probability, you have undoubtedly studied it like most students do, as a way of learning how to do the problems.
You don't necessarily think of the generalizations of this, you don't necessarily think of why is it that when you define a probability space you start out with a sample space and you talk about elements in the sample space, what's their sample points.
What do those sample points have to do with random variables and all of that stuff?
That's the first thing you forget when you haven't been looking at probability for a while.
Unfortunately, it's something you have to understand when we're dealing with this because we have a bunch of things which are not random variables here.
These letters here are things which we will call chance variables.
A chance variable is just like a random variable but the set of possible values that it has are not necessarily numbers, they're just events, as it turns out.
So the sample space is just some set of letters, as we call them, which are really events in this probability space.
The probability space assigns probabilities to sequences of letters.
What we're assuming here is that the sequence of letters are all statistically independent of each other.
So for example, if you go to Las Vegas and you're reporting the outcome of some gambling game and you're sending it back your home computer and your home computer is figuring out what your odds are in black jack or something, then every time the dice are rolled you get an independent -- we hope it's independent if the game is fair -- outcome of the dice.
So that what we're sending then, what we're going to encode is a sequence of independent, random -- not random variables because it's not necessarily numbers that you're interested in, it's this sequence of symbols.
But if we deal with the English text, for example, the idea that the letters in English text are independent of each other is absolutely ludicrous.
If it's early enough in the term that you're not overloaded already, I would suggest that those of you with a little time go back and read at least the first part of Shannon's original article about information theory where he talks about the problem of modeling English.
It's a beautiful treatment, because he starts out same way we are, dealing with sources which are independent, identically distributed chance variables.
Then he goes from there, as we will, to looking at Markov chains of source variables.
Some of you will cringe at this because you might have seen Markov chains and forgotten about them or you might have never seen them.
Don't worry about it, there's not that much that's peculiar about them.
Then he goes on from there to talk about actual English language.
But the point that he makes is that when you want to study something as complicated as the English language, the way that you do it is not to start out by taking a lot of statistics about English.
If you want to encode English, you start out by making highly simplifying assumptions, like the assumption that we're making here that we're dealing with a discrete memoryless source.
You then learn how to encode discrete memoryless sources.
You then look at blocks of letters out of these sources, and if they're not independent you look at the probabilities of these blocks.
If you know how to generate an optimal code for IID letters, then all you have to do is take these blocks of length m where you'd have a probability on each possible block, and your generate a code for the block.
You don't worry about the statistical relationships between different blocks.
You just say well, if I make my block long enough I don't care about what happens at the edges, and I'm going to get everything of interest.
So the idea is by starting out here you have all the clues you need to start looking at the more interesting cases.
As it turns out with source coding there's another advantage involved -- looking at independent letters is in some sense a worst case.
When you look at this worst case, in fact, presto, you will say if the letters are statistically related, fine.
I'd do even better.
I could do better if I took that into account, but if I'm not taking it into account, I know exactly how well I can do.
So what's the definition of that?
Source output is an unending sequence -- x1, x2, x3 -- of randomly selected letters, and these randomly selected letters are called chance variables.
Each source output is selected from the alphabet using a common probability measure.
In other words, they're identically distributed.
Each source output is statistically independent of the other source outputs, x1 up to x k plus 1.
We will call that independent identically distributed, and we'll abbreviate it IID.
It doesn't mean that the probability measure is 1 over m for each letter, it's not what we were assuming before.
It means you can have an arbitrary probability assignment on the different letters, but every letter has the same probability assignment on it and they're all independent of each other.
So that's the kind of source we're going to be dealing with first.
We will find out everything we want to know about how we deal with that source.
You will understand that source completely and the other sources you will half understand a little later.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'm going to review what we did with the craft inequality just a little bit, because evidently a number of people were confused about this.
I'm going to put a little more notation in with it.
For some people, notation helps.
For other people, it hinders things.
But after you've thought about it a little bit, a little more notation can certainly be helpful.
What we're trying to do in this craft inequality is, we're thinking of a set of symbols where, supposing that there's a codeword for each symbol, c of x is the codeword for symbol x, which is a string of binary digits.
y1 up to y sub n. In the world of two-toed sloths, the representation of numbers that sloths use is binary, base 2.
And therefore, they use the number associated with some sequence of bits like this would be the sum of y sub i, times 2 to the minus i.
In other words, it's the same thing as a decimal, except it's in a world where people have two fingers instead of ten.
There's an interval associated with this number, also.
And the interval is what you would get if you took an arbitrary real number and rounded it down to l sub i.
Well, in this case to m binary digits.
So that interval, then, is the number itself.
And the other side of the interval is that number itself -- the two-toed sloth didn't like what I was going to say about it, and it changed my slide.
So, is that extra factor the 2 to the minus n, there.
In other words, any number in that range, if you round it down to m significant binary digits is going to the this number here.
Well the point of this, if the number, namely, this base 2 expansion of a number y prime is in this interval, then y is going to be a prefix of y prime.
Let me just give you some examples of that because saying it in words is confusing, the idea is very simple.
Suppose you have a binary string, 011.
That corresponds to the number 3/8.
Namely, 1/4 plus 1/8.
And the interval there is going to be the interval from 3/8, including 3/8, up to 1/4.
But not including 1/4.
Namely, 1/4 will be represented as -- the sloth really is hitting hard this morning.
Maybe I'm the sloth.
OK.
There we go.
From 3/8 to 1/2.
Namely, 1/2 is just 1.
Nothing more than that.
Or it could be 10 or 100, and so forth.
So then, as an example, 011 is going to be a prefix of all of these quantities here.
That's a prefix of 0111 because 0111 is 3/8 plus 1/16, which is in that interval we're talking about.
0110 is a more interesting case.
Because 0110 is itself, the number associated with it, is just 3/8.
But what this is saying is, if you take the same number but expand it to four digits, according to what this says, r of y like is then in this interval.
And therefore this prefix situation holds.
So this is a prefix of this.
Why do I make it so complicated?
I make it so complicated because I want to talk about the length of that interval.
And the length of these intervals is denoted in this diagram here.
Anytime I have a number expressed to n binary digits, it covers an interval of 2 to the minus n. And because of this prefix property, as soon as one number covers an interval, no other number have its base in that interval.
In other words, all of these intervals have to be disjoint, exactly as is indicated here.
So when you add up the size of all these intervals, you have to get something less than or equal to 1.
And that's the proof of the craft inequality.
So, let's go on and talk about discrete source probabilities.
Now the two-toed sloth has gotten his machine out there, he's really mad at me this morning.
If we try to model English text, you know that some letters are far more probable than others.
Namely, if you take an enormous amount of English text and you measure the relative frequency with which each letter occurs, you'll get more or less stable relative frequencies if you take enough text.
And these letters are far more probable than these letters.
So that gives you part of a model.
You can also say that successive letters are going to be very dependent.
Namely, t is very often followed by h, and h is often preceded by t. q u is even more of a case here, because as far as English language words are concerned, u always follows q.
Some letter strings are words.
Other letter strings are not words.
There are constraints on grammar.
And what is really the clincher, which says there's no way you're going to model English in any sensible nice way, is meaning.
And even worse than that, depending on who writes the English, it might have meaning or it might not have meaning.
And for those English texts that don't have any meaning, the entropy is going to depend very much on whether the meaningless text is written by a salesperson or it's written by James Joyce.
And in one case you have -- well, in one case you have an enormous amount of freedom in what this sequence of letters are.
And in the other case you have letters which are very, very constrained.
So what's the point of this?
The point of this is, if you're interested in trying to find the source coding method for English, what you don't want to do is to start out trying to get the best statistical model of English that you can.
Because it's a losing proposition.
And by trying to do that, you'll spend all your time trying to get the model.
And you won't get any insight into what you ought to do as far as source coding is concerned.
This is pretty much true throughout all of technology.
You don't solve problems by first getting too far into the details of what the problem is, before you start thinking about structures of possible solutions.
In other words, we always deal with technological problems by dealing with toy problems first.
Now, there's a difference between engineers who worry about toy problems.
Because engineers, if they hate theory, usually, don't say what the toy problem is.
But they have that toy problem very firmly in the back of their mind, because of all their experience.
Theoreticians, on the other hand, make their models very, very explicit.
They don't often like to say that they're toy models because DARPA doesn't tend to support things that are working on toy models.
So they try to conceal this.
And often they just hide the fact that they're using a model.
So, all of this becomes very complicated.
But, for you, if you're trying to do either engineering or mathematics or teaching, or just be a sensible person.
When you're dealing with these problems, be explicit about what your models are.
Try to understand the toy problems before you understand the more complicated problems.
If you understand that out of this course, it'll be a worthwhile course for you.
And, of course, you won't understand it until you get lots more experience.
But believe me, that's the way it is.
OK, so that's the whole point of this.
You want to start with simple toy models.
And I'm not just justifying the fact that we're going to study this incredibly simple model here, which is a toy model.
But by studying this you will see that everything else follows.
If you read Claude Shannon's work on information theory, this, in fact, is where he started.
He started with a beautiful description of trying to model the English language.
Finally wound up with talking about this.
The conclusions he drew, from studying these discrete memory sources, lead to his general theorems about data compression on sources.
They led to his general theorems about the capacity of channels.
They led to the idea that you want to separate source coding from channel coding, and finally, they led to all of the modern ideas that we have about quantization.
In other words, the simple ideas you get out of this generalize directly to everything that's known about information theory.
So.
Enough philosophy, let's get on with the business of what we're trying to do.
A discrete memoryless sources has the following properties.
The source output has to be an unending sequence, x1, x2, x3, blah, blah, blah, of random letters drawn from a finite alphabet, x.
In other words, we are taking these real sources.
And we're saying, let's make them not real now.
Let's put a probability measure on them.
And in this probability measure, one of the things that the probability measure will do will be to describe the probability on each one of these letters and the sequence in which it's coming out of the source.
Each source output, x1, x2, blah, blah, blah, is selected from a common alphabet.
Namely, if you're using English on one letter of the sequence, you're going to use English on every letter of the sequence.
Going to use a common probability measure, with some probability mass function p sub x of x.
This notation means this is the probability mass function for the chance variable X.
A chance variable is like a random variable, except the objects are not necessarily numbers.
The objects can be anything.
So, a chance variable is a generalization of a random variable.
So this probability mass function talks about the probability of each of the symbols in this alphabet x.
Then the final thing is, each source output, x sub k, is statistically independent of all other source outputs, x1 up x k minus 1, and x k plus 1 on to forever.
This is a nice example, because if you're going to specify a source probabilistically, you have to somehow find a way of explaining what the probability of every possible event within this source is.
This is an easy way of doing it.
You say they're independent.
And then you can find the probability of anything you want to find.
So that's a generic way of putting probability measures on things.
So then, we want to go into the idea of prefix-free codes for these discrete memory of the sources.
We've already talked about prefix-free codes.
We talked about the craft inequality.
You might have thought it was a little bit strange talking about the strictly combinatorial property of codes without talking at all about the probabilities, which are the things that led us into talking about these codes in the first place.
Namely, we want to use unequal length, variable length codes.
Because of the fact that some letters are more likely than other letters.
And eventually we'll be using them because of all these constraints between different words.
So, for notation, let l of x be the length of the codeword for letter x in, the alphabet capital X. ok so that's the same as this y1 y2 up to y sub n. At some strength of binary symbols.
Capital L of x is a random variable, where capital L of x is equal to little l of x, where capital X equal to x.
Now, what the heck does that mean?
It's just notation.
In other words, what we're starting out with is this ensemble of letters.
We have a probability assignment on each letter in that alphabet.
And then what we would like to talk about is a length function on those letters.
So we have little l of x, which is defined for each x.
We then want to talk about this as a random variable.
Because when we choose some random letter, x, little x, of this ensemble, capital X, l of x becomes a random variable.
We will always, in this course, use capital letters to talk about random variables.
And we will always use little letters to talk about things which are not random variables.
Excuse me, not not random variables but random variables or chance variables.
I think we can probably leave it open now.
It seems as if the sloth has gone away.
So then we want to talk about the expected value of the length.
You talk about the expected value of something, you're talking about the expected value of a random variable.
We will also denote the expected value of this random variable L with a bar over it, which is the sum over the letters in the alphabet, of p of x times L of x.
So all this is what you would do anyway if you never thought about this.
Until, at some point, when you're taking a quiz or something and start to get confused and say, what is this stuff all about?
I don't have any idea what this means after you've written five pages of stuff.
So, it's worthwhile spending a little bit of time sorting that out.
So, L bar is the number of encoder output bits per source symbol.
In some strange sense.
Namely, it's this expected value.
Now, to finish this off, if we want to look at the number of binary digits to come out of the source when a long sequence of letters come out of the source -- A long sequence of letters.
x1, x2, x3, and so forth, come out of the source.
They go into the encoder.
The encoder is mapping each letter that comes out of the source into this codeword c of x.
So we have a sequence of codewords which are all concatenated together.
And, therefore, the total number of binary digits which has come out of the source corresponding to these n symbols that have come out of the source, is the sum of L of x1 plus L of x2 plus of x3 plus L of x4, and so forth.
So we have a sum of independent random variables.
Now, what do you know about sums of independent random variables?
Well, the one thing you ought to know about, and which should be stamped on your brain because it's the central thing that makes any probabilistic theory make sense, it's the only way that we can ever understand our environment.
You look at the past.
You try to figure out from the past what's going on in the future.
And the only way you can do that, the only tool you have, really, is this law of large, numbers.
Which says, when you see a long sequence of things, from that long sequence of things, you sort of figure out what's going on.
If you're dealing with a random variable, the thing you do is add up all of these numbers.
You divide by the total number of them that you have.
And that gives you the expected value.
It gives you a typical value.
What the law of large numbers really says is, if you look at the sum of binary digits out of this encoder, over a very long period of time, divide by the total number of symbols, that's a random variable again.
And this random variable is, with high probability, going to be very, very close to this expected value, which is this quantity here.
In other words, the ensemble average, which is this, is going to be very close to the time average.
And the time average, now, is a random variable.
And that's what the law of large numbers says.
You see, the problem that we all have, dealing with real world problems, is that there's nobody to tell us this is what the ensemble is.
Unless you believe somebody that doesn't know.
And the only real evidence that you have is the actual sequence.
And from the actual sequence, you then look at what happens for this particular sequence.
You then build a model.
And your model, by definition, has the expected value of L going to be equal to the expected value in the model that you've chosen.
So.
What's your objective?
Your objective in trying to form a prefix-free code, then, is to find a set of integers, L of x, which satisfy the Kraft inequality.
And they minimize L bar.
In other words, what we're trying to do is, we're trying to choose a code which minimizes the expected length of the code.
Which is really, over a long period of time, going to minimize the number of binary digits that come out of the source encoder.
What we want to do is to choose these integers to minimize this.
So what we're going to do now is, suppose our alphabet is just 1, 2, up to capital N. What am I doing here?
I'm saying, we don't care about what these symbols are called, anyway.
It's totally irrelevant what the names of the symbols are.
So I will name them 1, 2, up to capital N. Probability mass function, then, I can denote as p sub 1 up to p sub capital N. In other words I've gotten rid of all these axes that were lousing up our equations all along.
Now, I'll denote the unknown lengths by l1 up to l sub M. So the problem is, somebody gives you this set of numbers.
p1 to p sub M, which is a PMF.
In other words, these numbers add up to 1.
And tells you, I want a prefix-free code which minimizes this expected length.
Namely, the expected value corresponding to these lengths here.
So, the expected length, to minimize it, what we want to do is to minimize over the choice of l1 up to l sub M. Subject to the craft inequality, we want to minimize this expected value.
So we have a nice, clean, mathematical problem now.
We want to minimize this sum, subject to this constraint.
And the constraint includes the fact that all of these things have to be integers.
Well, for those of you who have studied minimization, there's a funny thing in here.
Because integer minimization problems tend to be very, very nasty.
And, therefore, you look at this and you say, this is probably something I'm going to have trouble solving.
Strangely enough, it isn't.
But you would think it probably is something which will be hard to solve.
So, since integers louse up minimization problems, what do we do?
Well, we say, just for fun let's try to solve this problem without the integer constraint on it.
Let's see what that leads to, and see if we can do anything with that.
So we say, OK, let's try to minimize this function here over the integers l1 to l sub M, subject to this constraint.
So we're minimizing this, subject to this constraint.
Now, an easy way to do that -- yes
Are you saying that side length is not a fixed probability
No, I still have these fixed probabilities.
I still have p1 up to p sub M, as known probabilities.
But I'm going to say, let's suppose I can choose a length which is two point five bits instead of two bits
You're saying the shortest length [UNINTELLIGIBLE]
Well, we're going to wind up there eventually.
But for now, all I want to do is to look at this problem.
If I start out by saying, assign shortest lengths to the biggest probabilities, I have two problems.
One is, it's a little hard to prove to you that I want to do that.
Although we'll do that later today.
And the other is, it doesn't really give you the general properties that we want to know about this.
So, for those two reasons, I want to just attack this as a straightforward mathematical problem.
If you're a computer scientist, this looks strange.
Because computer scientists like to attack problems by algorithms.
Analog engineers like to attack problems by writing a complicated formula, and taking derivatives, and all sorts of things like that.
We're going to be doing both of those things in this course.
And you'll see that both of them lead to certain advantages.
And here, we're taking, where the analog engineer's approach is saying, suppose this is a bunch of numbers.
I want to minimize this function over a set of numbers, l1 up to l sub capital M. So, how do I do that?
Well, this guy Lagrange, he was a great mathematician.
He was also a great mathematician early enough that he could do some really trivial things and become famous for them.
Just like Kraft that we were talking about before.
But, unlike Kraft, Lagrange really did a lot of other very important things.
And what Lagrange said was the following: Well, suppose I want to minimize this sum.
And I want to have this constraint added in.
Sort of what I want to do, then, is to minimize a weighted sum of this, which is what I'm interested in, and this.
In other words, if I minimize this weighted sum here of these two things, I'm going to wind up with some sort of value for this.
And some sort of value for this.
By changing lambda, then, which stands for Lagrange, he was also clever in making himself famous that way, by changing lambda, I can change the balance between how important these two things are.
And as I change the balance between how important they are, when I change it to just the right place, I'm going to have this constraint here satisfied with equality.
So that's the whole idea of Lagrange minimization.
So we take this function.
How do you now minimize a function of multiple variables?
Well, again it's a messy problem.
But the first thing you can try to do is find a stationary point.
So, let's always do the easy thing first.
We take the partial derivative of this function here, with respect to l sub i.
That's what we're trying to minimize.
And what we get is p sub i minus lambda times the natural log of 2, times 2 the minus l sub i. I'm not very good at differentiation any more, so I only differentiate things which are easy.
And that's easy.
I want to find a stationary point, so I set this equal to 0.
That makes the problem worse, because now I have a function of lambda and also all of these l sub i's.
But now I choose l sub i, so that I satisfy the constraint.
Namely, I choose lambda to satisfy this equation here.
When I choose lambda to satisfy this equation here, what I get is p sub i is equal to 2 to the minus l i, and therefore, l sub i is equal to minus log p sub i.
In other words, I have this equation here.
What happens when I sum this equation over i?
Let's look at it.
We sum this over i.
The sum of p i over i is 1 minus lambda times natural log of 2 times sum of 2 to the minus l sub i.
And I want to make this equal to 1.
So what I get is 1 is equal to this times lambda natural log of 2.
So, I hope that choosing lambda equal to 1 over natural log of 2 is what I want to do.
And when I do that, this becomes 1 here.
And I just have 2 to the minus lambda i is equal to 1.
OK, good.
And then, going back to this equation, p sub i is equal to 2 to the minus l sub i. OK, this is this arithmetic.
I mean, if you don't follow what I'm doing, just look at it later and you'll find that there's nothing really there.
So we wind up with these lengths being equal to the negative of the binary logarithms of these probabilities.
It's only a stationary point.
We don't know whether it's a minimum yet.
And, unfortunately, we also have the problem of, they might not be integers.
But, anyway, what we wind up with, then, if we ignore looking at these problems for the time being, is that the lengths are going to be equal to this.
The expected value of the lengths is then going to be equal to the sum, over i, of minus p sub i times the logarithm of p sub i.
When Shannon saw this, and when various other people saw it, they said, gee, this looks like the entropy of statistical mechanics.
So let's call this quantity entropy.
For no better reason than that.
And it would probably have been far better if they called it something else.
Because for years, there were physicists and philosophers trying to figure out what the deep relationship was between statistical mechanical entropy and information theoretic entropy.
And there probably is such a relationship, but the relationship is far more complicated than understanding information theory.
And it's far more complicated than understanding statistical mechanics.
So I advise you to not worry about that one until after you understand what it means in an information theoretic sense.
So h of x is what we call the entropy of the random variable x.
And it really is the entropy associated with these logarithms of p sub i.
So when you take functions of a random variable, a random variable carries along a lot of baggage with it.
Including the probabilities of everything.
And when you take the expected value of a random variable, the individual values of the sample points of that random variable are important.
And the probabilities are important.
Here we have something even stranger.
Because it's only the probabilities that have anything to do with it.
And this makes sense.
We already said that these symbols have nothing to do with this problem we're dealing with.
You can call the symbols whatever you want to call them.
And, therefore, the only thing of any interest to us is these probabilities that we're dealing with.
So H of x is a function only of these probabilities.
It's the expected value of minus log p sub i.
This is called entropy, and in fact we will find out very shortly that it really is the minimum number of bits per source symbol needed to represent the source.
In other words, when we generalize the problem from just plain ordinary garden-variety oh prefix-free codes, we will find that this number is really what characterizes the whole problem for discrete memory-less sources.
So let's go on and say more about that.
Let's say something about bounds on the entropy.
First, what's the relationship between the entropy and this minimum of the expected length that we started to talk about?
And I claim that H of x is less than or equal to L min, which is less than the entropy plus 1.
And why is that?
We already have the machinery to see this.
We almost have the machinery to see this.
Namely, we have solved this minimization problem.
We've only found a stationary point, but we've ignored the fact that we have an integer constraint.
So, if you allow me to say for the time being that, in fact, when we solve the problem without worrying about integers, that it actually gives me a minimum, then in fact this follows very easily.
Because what I'm going to do is to find those optimal lengths, which are non-integers.
And then I can solve the prefix condition and get a code by simply increasing each of those numbers to the next integer up.
In other words, I can take the ceiling function of each of those real numbers to get an integer.
When I take the ceiling function, 2 to the minus l sub i is going to go down.
So the craft inequality is still satisfied.
So the entropy has to be less than or equal to this average.
Has to be less than H of x plus 1.
So the average is equal to H of x if and only if each these probabilities it an integer power of 2 to start with.
In other words, the solution I came up with before is that the length I wanted should be equal to minus the logarithm to base 2, of p sub i.
So if p sub i is already a power of 2 then I'm home free.
Because I just pick that length to be minus log of p sub i, and it happens to be an integer.
And I don't have to round it up.
So if I let l1 to lM be these codeword lengths -- well, here's where I'm going to prove this to you.
And the proof is the following: I want to prove that H of x is less than or equal to l min, which I'll just call here L bar.
So H of x minus L bar is equal to, this is the entropy.
This is the expected length here.
I can rewrite this as the sum of p sub i times logarithm of 2 to the minus l sub i divided by p sub i.
That's just arithmetic.
l sub i is equal to to logarithm of 2 to the l sub i, that's equal to minus the logarithm of 2 the minus l sub i.
So I get this.
There's an inequality.
Hate to call it an inequality, it's so trivial.
Here's the point 1.
If you plot a natural log of x.
And if you compare it with the function x minus 1, you can see that natural log of x is less than or equal to x minus 1.
Now, this an inequality which happens to be very useful in information theory.
I would claim that any inequality that you can prove in information theory, by any means at all, I can prove using this inequality and nothing else.
And I've believed that for 50 years and nobody's proven me wrong yet.
And also, this is something you can draw and remember.
So it's simple.
So the idea, then, is since this sum log of 2 to the minus l over p sub i is less than or equal to the natural log of 2 times the natural logarithm of this.
You just, here we go.
For any u greater than 0, natural log of u is less than or equal to this.
So the logarithm to the base 2 of u is less than or equal to the logarithm to the base 2 of e, which is some number, times u minus 1.
With equality at u equals 1.
So this is less than or equal to this.
And look how nice that is.
The p sub i's cancel out and you get the sum over i, of 2 the minus l i minus p sub i is less than or equal to 0.
An equality occurs if, and only if, p sub i is equal to 2 to the minus l i. OK?
So that's all there is to it.
And that establishes that -- well, establishes part of this theorem here.
And the other part we already established, And if you don't believe me, the notes do it more carefully.
Well, this left us a serious problem unknown.
Which is, how do you actually solve this integer minimization problem.
How do you solve it if you have a big, long, complicated source with lots of probabilities in it?
And everybody thought it was hopeless.
Even Shannon thought it was hopeless.
And Shannon sort of figured out ways to approach this problem.
He said, well, you want to have about half the probability.
Starting with 1, about half the probability starting with 0.
So he would divide up the symbols in the alphabet, so he could come as close is possible to half of them being up here and half of them coming down here.
And he would continue to do that, I mean, I don't usually like to write on the blackboard, but he would start out generating a code like this.
And this would be approximately 1/2.
This is approximately 1/2.
And then he would take these symbols.
Split them, again, in probability.
And everybody was starting the problem over here, and trying to generate a code working their way out.
Well, Dave Huffman was a graduate student at the time.
and he took Bob Fano's graduate course in information theory, I think a year or so later than Kraft did.
And Bob Fano assigned as a homework problem, how do you solve this problem?
Sneaky guy.
And he was very amazed when Dave Hoffman came in next day and said, oh, it's easy, you do it this way.
So the question is, how did he do it?
Well, Huffman, instead of looking at the problem from here out, looked at the problem from here in.
He was -- I mean, this was before there was anything called computer science.
But he thought like a computer scientist did.
In other words, he thought algorithmically.
And he also thought in terms of discrete problems.
And therefore, he looked for properties that these optimum codes should have.
And it was neat.
So, he started out with a limit.
He said, an optimal code has to have the property that a p i is greater than p sub j.
Then the optimal length associated with p sub i, namely the optimal length of the i'th codeword, had to be less than or equal to the length of the j'th codeword.
And you can see this by saying, well, suppose that's not true.
Suppose that p i is greater than p j.
And also, li is greater than lj.
And then you say, OK, take this situation.
We will interchange those two codewords in the code.
And we'll look at what that does to the average.
And if you work that through, you find out that since what you've done is, you've shortened the codeword associated with this and lengthened the codeword associated with this.
You have changed the average length to make it smaller.
Now, let me warn you about something.
When you start looking at these properties, the most confusing thing is what happens when two probabilities are the same, or when two lengths are the same.
And I would advise you to just ignore that problem, until you get an idea of what's going on.
Namely, assume that all lengths are different.
All probabilities are different.
And then it's easy to see what's going on.
And when you get all done, go back and straighten out the cases where things are equal.
And I think the notes do this carefully.
If you read books on information theory, about half of them do it carefully, and about half of them don't.
So you should be suspicious.
But anyway, that's one of those trivialities that you just have to sort out for yourself.
OK.
The next lemma is optimal prefix-free codes are full.
We talked about what a full code is.
When you draw this binary graph for it, you don't have any nodes in a binary graph -- you don't have any leaves that are not associated with codewords.
Because if you do, we showed you that shortened the codeword of the part of the tree on the other side of that leaf.
In other words, if you have something here, if this is a codeword and this is not a codeword, then you just get rid of this, and bring that back here.
But this is a whole tree stemming off here, you do the same thing.
You take this whole tree and you bring it into there, and you throw this away.
So, optimal prefix-free codes are full.
So far there's nothing to this.
The next part of it is the sibling of a codeword.
And what's a sibling?
Well, we used to call it a brother.
But then couldn't do that because we have to call it a brother or sister.
And that got to difficult.
So people invented the word sibling, to talk about a brother or a sister.
So the sibling of a codeword for is the string form by changing the last bit.
In other words, in this family tree here, the sibling of this is this.
The sibling of this is this.
The sibling of this is this.
So, a leaf can have a sibling which is an intermediate node, and vice versa.
So then he said, the sibling of a codeword is a string formed by changing the last bit.
I think he probably said the brother, but, anyway.
For optimality, the sibling of each maximum length codeword is another codeword.
Now, that's a really simple one.
If I make this a codeword, and this is the maximal length codeword in this code I'm talking about, this can't be an intermediate node because then there would have to be longer codewords.
And it can't be empty because these optimal codes are all full.
And therefore, this has to have a sibling which is also a codeword.
So the longest codewords have to have siblings.
Well, that's easy enough.
Incidentally, one of the problems that you have in proving this sort of thing is, what happens if you have zero-probability letters.
Well, we just get rid of that problem and say, well, there aren't any zero-probability letters.
Because if we want to come up with a sensible model for something, we're not going to create a codeword for something that can't happen.
So, there are no zero-probability letters in this alphabet.
I mean, if you want to put them in, it just complicates the whole thing.
And you can do it.
Then, finally, there's this lemma which says, there is an optimal prefix-free code in which, after you order the probabilities of all of the messages, namely you order p1 to be greater than or equal to p2, greater than or equal to p sub m. In other words, we just rename the letters in the alphabet, so that letter m is less likely than letter m minus 1, and so forth.
Back to 1.
1 is the most probable, m is the least likely.
Well, we've already concluded that we want to assign the longest messages to the least probable codewords.
And this says, take the two least probable codewords and we can always make an optimal code in which those two codewords are siblings.
And the reason for that is, one of them is not going to be longer than the other or else you can shorten the code by interchanging things.
So there is an optimal prefix-free code in which the codeword for m minus 1.
And the codeword for m are maximal length and they're siblings.
So the Huffman algorithm first combines these two.
And then looks at the reduced tree with m minus 1 nodes.
Let me show you an example of that.
So it starts out.
Here, I've ordered the probabilities associated with a set of symbols.
The symbols are 1, 2, 3, 4, 5.
The two least likely messages are 0.1 and 0.15.
Obviously, I could've interchanged these two if I want to.
But why interchange them?
So I say, OK, the last digit on this one, I'm going to assign to be a 0.
The last digit on this, I'm going to assign to be a 1.
And the important thing is, I'm going to make them siblings in this tree.
And what I'm going to do now, terribly complicated thing, instead of building a tree from left to right, I'm going to build a tree from right to left.
So when I get all done with the tree it's going to come in like this.
And what I'm doing is starting out at the end, to start to build the end of the tree.
And what happens after I go through this first step is, I say, OK there is an optimal code.
In which these two quantities are siblings of maximal length.
I now want to form an optimal code for these probabilities here.
So, I go back and I iterate again.
And I said, OK, if I have these probabilities here, what's the optimal code.
Well, I could reorder the things.
But now I know that the only thing I'm interested in is the two least likely symbols in this new alphabet here.
Which is 0.2 and 0.15.
So I combine those together.
I tie them together as siblings in this last generation, however it works out.
So then I have an alphabet of size three.
And then down here, I have these two things tied together.
These two things tied together.
So I have a node of probability 0.25.
I have a node of probability 0.35, and I have a node of probability 0.4.
I take the two least likely, and I tie them together.
And then I have two nodes left, one with probability 0.6 and one with probability 0.4.
And I tie them together.
And, presto, I have my whole code, except for flipping it over, to go from left to right if you like.
Codes that go from left and right, instead of right to left.
OK.
I have swindled you.
How have I swindled you?
I mean, I've swindled you a little bit by talking about these things that might be equal or not equal.
And that's not important.
You can sort that out on your own.
There's a very important swindle I pulled.
And what's that?
What's very incomplete in this argument?
This part is fine.
Nothing wrong here.
We have a lemma which says, you can find an optimal code by tying these two things together.
Yeah?
[UNINTELLIGIBLE] combine those two [UNINTELLIGIBLE] combination
You're saying, how do I know to combine these two?
OK, which means what?
Yeah
[UNINTELLIGIBLE] you've just added the probabilities --
I've just added those two probabilities.
So I have a new ensemble where I have four probabilities, 0.25, 0.15, 0.2, and 0.4.
And that's fine.
I still have these things.
No, there's no independence involved here at all.
I mean, I started out with five letters.
Which are disjoined.
I now have four letters that are disjoined.
What have I done?
Yeah
[UNINTELLIGIBLE]
Yes.
Yeah.
I have assumed, now, that once I get these four symbols, if I have those four symbols, I can form an optimal code for those four symbols in which these two symbols get tied together.
But how do I know that an optimal code for this reduced set of probabilities is also an optimal code for the original problem?
I have tied these two things together.
I know there's an optimal code in which these two things are tied together.
I then have four symbols.
I want to find a code for those four symbols.
But I assume that the optimal code for these four symbols, when I break apart these two things, gives me an optimal code for five symbols.
That's the sort of thing I want you people to start catching onto immediately.
I want you to start asking those nasty questions.
And those nasty questions are the things that say, OK, how do I know that this works?
In other words, you're not here to learn these algorithms.
I can tell you what the algorithm is in an instant.
You can do the algorithm.
A computer can do the algorithm about three thousand times faster than you can.
And you can be replaced by a computer, if you only learn the algorithms.
You can program the algorithm.
You can probably find the computer that can program the algorithm too.
And there's no need to program it more than once.
So that after you've done that, you are useless again.
So the only thing that's worthwhile for you is to be able to spot these problems and to understand what's going on.
So.
How do I know that this first optimization leads to the second optimization.
After combining these two least likely codewords, or siblings, we've gotten a reduced set of probabilities.
In this problem here, what we've done, the reduced set of probabilities are 0.4, 0.2, 0.15, and 0.25.
Why does finding the optimal code for this reduced set result in an optimal code for the original set?
That's really the question that we're asking.
Well, it's not hard.
If you take any code for the reduced set, let's call the reduced set x prime, set of probabilities.
Let the expected length of that be l prime.
It's not necessarily an optimal code, but it's any old code that I generate.
Any old code I generate for L prime, I can now take that code for l prime and I can expand it out to a code for L. Namely, I have this code here this, this, this, and that's the expanded -- and now I can expand it into a code for the original set, by adding on this and this, as leaves on this.
This leaf here then becomes an intermediate node.
And I add two extra leaves to it.
OK, well, it's not hard.
The expected length for this code, for these five letters, I claim, is equal to the expected length for this reduced code, this, this, this, and this.
Plus one extra digit for this.
Plus one extra digit for this.
So the expected length for L is the expected length for L prime plus 0.15 plus 0.1.
Which says the following: if I want to minimize this, and I know that this has to be equal to this, and these two numbers are fixed, I can't change them.
I can minimize this, by minimizing this.
And that's the final step in the whole argument.
And what's peculiar is that everybody learns the Huffman algorithm.
And what Huffman did, which was really very smart, was to sort out this issue.
And I can teach this to a hundred classes, and nobody will ever point out to me that there's a logical flaw in the whole argument.
And you can look at most books on information theory and they never point out that there's that logical flaw there, either.
So, anyway, that's the end of Huffman's algorithm.
You can see when you look at this that this is really an extraordinarily easy thing to do.
I mean, you can take an alphabet of several thousand symbols.
All you have to do is order them.
Tie the least two likely together.
Assign a 1 and a 0 to them.
Then, stick that into an ordered list again.
Take the two least probable.
Tie them together.
Stick it into an ordered list again.
And, if you have some minimal knowledge of data structures, you can do this with essentially on the order of one operation for each letter in this alphabet.
So it really isn't a very difficult sum.
So here's an integer problem which is really easy to solve.
And the way to solve it is to look at the problem in the opposite way from what everybody else has looked at it in.
Does this say you want to ignore everything that everybody else has done, and go your own way?
Not quite.
But it says that's one of the things you ought to try, if you find that everybody is doing something one way and you can find another way to look at, that's very rich.
It might turn out to be nothing but it might turn out to be something very worthwhile.
Let's now talk about this quantity, entropy.
And for every chance variable, x, if that chance variable, x, is discrete and has a finite number of elements in it, so I'm talking about a chance variable x, what's a chance variable have tagging along after it?
It has a set of n probabilities tagging along after it.
That's what a chance variable is.
A chance variable is not just the alphabet.
A chance variable is the alphabet plus the probabilities.
That's why you then talk about it having an entropy.
And the entropy is the expected value of, minus the logarithm, of this PMF function.
So, in fact, this is an unusual statistic in the sense that it has nothing to do with the symbol values, and everything to do with just the probabilities of the symbol values.
And as we go on, you'll see that in fact this is a very important property of it.
And dealing with the logarithms of these symbol values is, in fact, a much more worthwhile thing to do than dealing with the probabilities of the symbols.
Now, let me pause again and see if anybody can have any idea of why logarithms of probabilities might be more significant than probabilities.
And think of what we're going to be doing here.
We're taking a sequence of letters.
When I take a sequence of letters, what's the probability of the sequence of letters?
If they're IID.
Namely, we're looking --
[UNINTELLIGIBLE]
It's the product of those probabilities.
Now, if you agree with me that the probability theory is concerned 50% with the law of large numbers and 50% with everything else all put together, why is the logarithm of a probability important?
[UNINTELLIGIBLE]
You change your product to a sum, yes.
If you have a product of probabilities, you can talk about a sum of the logarithms of probabilities.
That's why entropy is important in statistical mechanics.
It also is, fundamentally, the reason why entropy is important in information theory.
Is because what you're almost always interested in is a product of probabilities.
And when you're interested in a product of probabilities and you want to use the law of large numbers, you turn that product of probabilities into a sum of the logarithms of probabilities.
Fundamental idea.
Shannon took eight years sorting all this out.
And Shannon was by far the smartest person I've ever met.
I mean, the problems that we worry about, he just, bip.
Solves it with no effort at all.
This one took them a while to sort out.
It also took him a while to sort out the fact that once he sorted this out, he could sort out all of the other problems.
As far as communications was concerned.
So was quite important.
I mean, I can tell you one of the peculiar things about Shannon.
Just from the first time I ever talked to him about a technical problem.
I'd just become a faculty member here.
And his office was about five doors down from mine.
And one day I screwed up my courage to go down and talk to the guy about a problem I was working on.
And I thought it was a really neat problem.
It had all sorts of pieces to it, all sorts of bells and whistles on it.
And I started to explain it to him.
And he said, well, can we look at a slightly simpler case where you throw out this part of it, you throw out one bell.
Then he'd throw out a whistle.
Then he'd throw out a bell.
And I was going along with this and saying, yeah, I guess we could.
We could.
We can throw out all of these things without really losing the essence of the problem.
And, finally, I started to get discouraged.
Because this really neat research problem, this really important research problem, was turning a toy problem which was almost trivial.
It had nothing to do, it seemed, with anything.
And finally we got down to a certain point.
And I said, yeah, but this is trivial, the solution is this.
And he said, yeah.
And then we started putting back all the pieces.
And his genius was, he knew which things to throw out.
So that each of the things we threw out, we could put them back in again.
When we got done, the research problem was trivial.
And his genius was in finding the right trivial example to look at.
So, in fact, what you always want to look at, in the communications field -- and in most fields, I think -- is finding the really simple way of looking at something.
Which means you have to throw out most of the nonsense.
So, in this case, it's looking at entropy, which is the logarithm of a probability assignment.
And you want to look at that because the logarithm of a probability assignment lets you add the logarithms of probabilities.
Use the law of large numbers.
And then you can talk about sequences of elements.
Properties of entropy.
For a discrete random chance variable.
We have m elements in the alphabet.
First thing is that the entropy is always greater than or equal to 0.
Why is that?
I'll let you figure it out.
Why is the logarithm of a problem, minus the logarithm of a probability, greater than or equal to 0?
Why is it non-negative?
Yeah
[UNINTELLIGIBLE]
Probabilities are always less than or equal to 1, yes.
So this quantity here is always greater than or equal to 0.
Because the logarithm of 1 is equal to 0.
We have a quality here if x is deterministic.
Which is just a special case there.
Where you have an ensemble of one element and it has probability 1.
Or, in fact, at this point you could add in things which have zero probability.
Well, that's a little bit tricky.
Because you add in something that's zero probability.
And the logarithm of 0 is infinity.
So you're dealing with the expected value of a bunch of infinities, which each occur with zero probability.
And you're forced to say, well, I think that 0 times log of 0 is equal to 0.
And in fact, epsilon times log of episilon goes to 0 as epsilon goes to 0.
But you save yourself a lot of worry by just leaving out things of zero probability.
So H of x is greater than or equal to 0.
We have equality if x is deterministic.
H of x is less than or equal to log of m. The quality, if x is equiprobable.
And how do I know that?
I look at this again.
I'm not going to prove it here but, essentially, this follows from saying that the natural logarithm of something is less than or equal to that something minus 1.
And you take the difference of the entropy, and log of m. And, presto, it gives you the result that you want.
So you've got the most entropy, if everything is equiprobable.
For any code satisfying the Kraft inequality, the entropy is less than or equal to L bar.
Well, that's what we already proved.
Mainly in the middle of the lecture, we showed that for any code to satisfied the Kraft inequality, the entropy was always less than or equal to L bar, because the entropy is what you get if you minimize the expected length without the integer constraint.
And L bar is what you get -- well, L bar min is what you get when you minimize it with the integer constraint.
I mean, you don't bother about minimizing it.
You get something bigger than L min.
So this is less than or equal to the length of any codeword.
For the very best codeword, for the very best code, the expected value of the minimum is less than or equal to the entropy plus y.
And you get that just by adding to each non-integer length the ceiling function.
Which gives you, at most, one extra digit for each codeword.
Now, here's the more interesting one.
For independent chance variables, x and y, here's where the nice part about notation comes along.
What's the entropy of x y?
Well, what do I mean by x y first?
I have a chance variable, x.
And this chance variable x has an alphabet associated with it.
x1 up to x sub m. I have a chance variable y.
It has an alphabet associated with it.
What's the sample space, what's the set of events corresponding to the chance variable x y?
By x y, I mean a chance variable whose elements are the possible values of both x and y.
So, I'm talking about the joint ensemble of x and y. I have a bunch of possible values for that.
And those possible values, if I have m possibilities for each, I have m squared possible values for the two of them.
So I'm talking about the expected value of minus the logarithm of the probability of x and y.
In other words, I am trying to take the -- let me write it out.
I'm probably given conniptions to -- no?
OK.
I want to take p of x y of symbol x y, times minus the logarithm to the base 2 of p sub x y of x y.
That's what this means if I write it out.
Well, this probability here is p sub x of little x, times p sub y of little y.
Why is that?
Because I'm assuming that they're independent of each other.
And, therefore, the probability of two of them is the product of the probabilities, times minus log to the base 2.
So if p sub x of x minus logarithm to the base 2 of p sub y of y.
And I'm summing this over all x and all y.
And the more sophisticated way to write this -- things I say in lecture, you don't have to copy down because they're always in the notes.
If they're not in the notes, it's probably wrong anyway, so.
So this expected value is expected value of the logarithm of the probability of x y.
Which is the expected value of the logarithm of p of x times p of y.
And, since I have a logarithm of a product, that's the expected value of minus log p of x minus log p of y, which is the entropy of x plus the entropy of y.
In other words, when I have a joint ensemble of even more independent quantities, The entropy the sequence is equal to the sum of the entropies of the individual elements in that sequence.
Well, that's all I wanted to talk about today.
If any of you have any questions to ask, you should ask them now.
I went through Huffman coding pretty quickly, because it's something where you have to do some exercises on it to sort it out.
And I didn't want to do any more than that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Just to review where we are, we've been talking about source coding as one of the two major parts of digital communication.
Remember, you take sources, you turn them into bits.
Then you take bits and you transmit them over channels.
And that sums up the whole course.
This is the part where you transmit over channels.
This is the part where you process the sources.
We're concentrating now on the source side of things.
Partly because by concentrating on the source side of things, we will build up the machinery we need to look at the channel side of things.
The channel side is really more interesting, I think.
Although there's been a great deal of work on both of them.
They're both important.
And we said that we could separate source coding into three pieces.
If you start out with a waveform source, the typical thing to do, and almost the only thing to do, is to turn those waveforms into sequences of numbers.
Those sequences might be samples.
They might be numbers in an expansion.
They might be whatever.
But the first thing you almost always do is turn waveforms into a sequence of numbers.
Because waveforms are just too complicated to deal with.
The next thing we do with a sequence of numbers is quantize them.
After we quantize them we wind up with a finite set of symbols.
And the next thing we do is, we take that sequence of symbols.
And -- and what we do at that point is to do data compression.
So that we try to represent those symbols with as small as possible a number of binary digits per source symbol.
We want to do that in such a way that we can receive it the other end.
So let's review a little bit about what we've done in the last couple of lectures.
We talked about the Kraft inequality.
And the Kraft inequality, you remember, says that the lengths of the codewords in any prefix-free code have to satisfy this funny inequality here.
And this funny inequality, in some sense, says if you want to make one codeword short, by making that one codeword short, it eats up a large part of this fraction.
Since this sum has to be less than or equal to 1.
If, for example, you make l sub 1 equal to 1, then that uses up a half of this sum right there.
And all the other codewords have to be much longer.
So what this is saying, essentially, and we proved it, and we did a bunch of things with it, and your homework you worked with it, we have shown that that inequality has to hold.
The next thing we did is, given a set of probabilities on a source, for example, p1 up to p sub m, by this time you should feel very comfortable in realizing that what you call these symbols doesn't make any difference whatsoever as far as any matter of encoding sources is concerned.
The first thing you can do, if you like to, is take whatever name somebody has given to a set of symbols, replace them with your own symbols, and the easiest set of symbols to use is the integers 1 to m. And that's what we will usually do.
So, given this set of probabilities, and they have to add up to 1, the Huffman algorithm is this ingenious algorithm, very, very, clever, which constructs a prefix-free code of minimum expected length.
And that minimum expected length is just defined as the sum over i, of p sub i times l sub i.
And the trick in the algorithm is to find that set of l sub i's that satisfy this inequality but minimize this expected value.
Next thing we started to talk about was a discrete memoryless source.
A discrete memoryless source is really a toy source.
It's a toy source where you assume that each letter in the sequence is independent, and equally independent, and identically distributed.
In other words, every letter is the same.
Every letter is independent of every other letter.
That's more appropriate for a gambling game than it is for real sources.
But, on the other hand, by understanding this problem, we're starting to see that we understand the whole problem of source coding.
So we'll get on with that as we go.
But, anyway, when we have a discrete memoryless source, what we found -- first we defined the entropy of such a source as h of x, which is the sum over i of minus p sub i, of logarithm of p sub i.
And that was just something that came out of trying to do this optimization not the way that Huffman did it but the way that Shannon did it.
Namely, Shannon looked at this optimization in terms of dealing with entropy and things like that.
Huffman dealt with it in terms of finding the optimal code.
One of the very surprising things is that the way Huffman looked at it, in terms of entropy, is the way this really valuable, even though it doesn't come up with an optimal code.
I mean, here was poor Huffman, who generated this beautiful algorithm, which is extraordinarily simple, which solved what looked like a hard problem.
But yet, as far as information theory is concerned, he used that algorithm, sure.
But as far as all the generalizations are concerned, it has almost nothing to do with anything.
But, anyway, when you look at this entropy, what comes out of it is the fact that the entropy of the source is less than or equal to the minimum number of bits per source symbol that you can come up with in a prefix-free code, which is less than h of x plus 1.
And the way we did that was just to try to look at this minimization.
And by looking at the minimization, we usually showed it had to be greater than or equal to H of x.
And by looking at any code which satisfied this inequality with any set of length -- well, after we looked at this, this said that what we really wanted to do was to make the length of each codeword minus log of p sub i.
That's not an integer.
So the thing we did to get this inequality is said, OK, if it's not an integer, we'll raise it up the next value.
Make it an integer.
And as soon as we do that, the Kraft inequality is satisfied.
And you can generate a code with that set of lengths.
So that's where you get these two bounds.
This bound here is usually very, very weak.
Can anybody suggest the almost unique example where this is almost tight?
It's a simplistic sample you can think of.
It's a binary source.
And what binary source has the property that this is almost equal to this?
Anybody out there?
[UNINTELLIGIBLE]
Make it almost probability 0 and probability 1.
You can't quite do that because as soon as you make the probability of the 0, 0, then you don't have to represent it.
And you just know that it's a sequence of all 1's.
So you're all done.
And you don't need any bits to represent it.
But if there's just some very tiny epsilon probability of a 0 and a big probability of a 1, then the entropy is almost equal to 0.
And this 1 here is needed because l bar min is 1.
You can't make it any smaller than that.
So, that's where that comes from.
Let's talk about entropy just a little bit.
If we have an alphabet which has size m, which is the symbol we'll usually use for the alphabet, x.
And the probability that x equals i, is p sub i.
In other words, again, we're using this convention so we're going to call the symbols the integers 1 to capital M. Then the entropy is equal to this.
And a nice way of representing this is that the entropy is equal to the expected value of minus the logarithm.
Logarithms are always to the base 2, in this course.
When we want natural logarithms we'll write ln, in other words it's log to the base 2.
So it's log to the base 2 of the probability of the symbol x.
We call this the log pmf random variable.
We started out with x being a chance variable.
I mean, we happen to have turned it into a random variable because we've given it numbers.
But that's irrelevant.
We really want to think of it as a chance variable.
But now this quantity is a numerical function of the symbol which comes out of the source.
And, therefore, this quantity is a well-defined random variable.
And we call it the log pmf random variable.
Some people call it self-information.
We'll find out why later.
I don't particularly like that word.
One, because what we're dealing with here has nothing to do with information.
Probably the thought that this all has something to do with information, namely, that information theory has something to do with information, probably held up the field for about five years.
Because everyone tried to figure out, what does this have to do with information.
And, of course, it had nothing to do with information.
It really only had to do with data.
And with probabilities of various things in the data.
So, anyway.
Some people call it self-information and we'll see why later.
But this is the quantity we're interested in.
And we call it log pmf, we sort of forget about the minus sign.
It's not good to forget about the minus sign, but I always do it.
So I sort of expect other people to do it, too.
One of the properties of entropy is, it has to be greater than or equal to 0.
Why is it greater than or equal to 0?
Well, because these probabilities here have to be less than or equal to 1.
And the logarithm of something less than or equal to 1 is negative.
And therefore minus the logarithm has to be greater than or equal to 0.
This entropy is also less than or equal to log M, log capital M. I'm not going to prove that here.
It's proven in the notes, it's a trivial thing to do.
Or maybe it's proven in one of the problems, I forget.
But, anyway, you can do it using the inequality log of x is less than or equal to x minus 1.
Just like all inequalities can be proven with that inequality.
So there's a quality here if x is equiprobable.
Which is pretty clear, because if all of these probabilities are equal to 1 over M, and you take the expected value of logarithm of M, you get logarithm of M. So there's nothing very surprising here.
Now, the next thing -- and here's where what we're going to do is, on one hand very simple, and on the other hand very confusing.
After you get the picture of it, it becomes very simple.
Before that, it all looks rather strange.
If you have two independent chance variables, say x and y, then the choice where the sample value of the chance variable x, and the choice of the sample value y, together that's a pair of sample values which we can view as one sample value.
In other words, we can view xy as a chance variable all in its own right.
This isn't the sequence x, followed by y, where you can think of it that way.
But we want to think of this here as a chance variable itself.
Which takes on different values.
And the values it takes on are pairs of sample values, 1 from x, 1 from ensemble y, and xy takes on the sample value of little xy with the probability p of x times p of y.
As we move one with this course, we'll become much less careful about putting these subscripts down, which talk about random variables.
And the arguments which talk about sample values of those random variables.
I want to keep doing it for a while because most courses in probability, even 6.041, which is the first course in probability, almost deliberately fudges the difference between sample values and random variables.
And most people who work with probability do this all the time.
And you never know when you're talking about a random variable and when you're talking about a sample value of a random variable.
And that's convenient for getting insight about thing.
And you do it for a while and then pretty soon you wonder, what the heck is going on.
Because you have no idea what's a random variable any more, and what's a sample value of it.
So, this entropy, H, of the chance variable, xy, is then expected value of minus the logarithm of the probability of the chance variable, xy.
Mainly, when you take the expected value, you're taking the expected value of a random variable.
And for the random variable here, in the chance variables, xy and here.
This is the expected value of minus the logarithm of p of x times the probability p of x.
Which is the expected value.
Since they're independent of each other it's the sum.
And that gives you H of x y is equal to H of x plus H of y.
This is really the reason why you're interested in these chance variables which are logarithms of probabilities.
Because when you have independent chance variables then you have the situation that probabilities multiply and therefore log probabilities add.
All of the major theorems in probability theory, in particular the law of large numbers, which is the most important result in probability, simple though it might be, that's the key to everything in probability.
That particular result talks about sums of random variables and not about products of random variables.
So that's why Shannon did everything in terms of these log PMF.
And we will soon be doing everything in terms of log PMF also.
So now let's get back to discrete memoryless sources.
If you now have a block of n chance variables, x1 to xn, and they're all IID, again we can do this whole block as one big monster chance variable.
If each one of these takes on m possible values, then this monster chance variable can take on m to the n possible values.
Namely, each possible string of symbols, each possible string of n symbols where each one is a choice from the integers 1 to capital M. So we're talking about tuples of numbers now.
And those are the values that this particular chance variable, x sub n, takes on.
So it takes on these probabilities, takes on these values with the probability p of x n is the product from i equals 1 to n, of the individual probabilities.
In other words, again, when you have independent chance variables, the probabilities multiply.
Which is all I'm saying here.
So the chance variable x sub n has the entropy H of x n, which is the expected value of minus the logarithm of that probability.
Which is the expected value of minus the logarithm of the product of probabilities, which is the expected value of the sum of minus the log of the probabilities, which is n times H of x.
If you compare this with the previous slide, you'll see I haven't said anything new.
This argument and this argument are really exactly the same.
All I did was do it for two chance variables first.
And then observe.
But it generalizes, to an arbitrary number of chance variables.
You can say that it generalizes to an infinite number of chance variables also.
And in some sense it does.
And I would advise you not to go there.
Because you just get tangled up with a lot of mathematics that you don't need.
So the next thing is, how do we fix the variable prefix-free codes and what do we gain by it?
So the thing we're going to do now, instead of trying to compress one symbol at a time from the source, we're going to segment the source into blocks of n symbols each.
And after we segment it into blocks of n symbols each, we're going to encode the block of n symbols.
Now, what's new there?
Nothing whatsoever is new.
A block of n symbols is just a chance variable.
We know how to do optimal encoding of chance variables.
Namely, we use the Huffman algorithm.
You can do that here on these n blocks.
We also have this nice theorem, which says that the entropy -- well, first the entropy of x n as n times the entropy of x.
So, in other words, when you have independent identically distributed chance variables, this entropy is just n times this entropy.
But the important thing is this result of doing the encoding.
Which is the same result we had before.
Namely, this is the result of what happens when you take a set -- when you take an alphabet, and the alphabet can be anything whatsoever.
And you encode that alphabet in an optimal way, according to the probabilities of each symbol within the alphabet.
And the result that you get is the entropy of this big chance variable x sub n is less than or equal to the minimum -- well, it's less than or equal to the expected value of the length of a code, whatever code you have.
But when you put the minimum on here, this is less than the entropy of the chance variable x super n plus 1.
That's the same theorem that we proved before.
There's nothing new here.
Now, if you divide this by n, this by n, and this by n, you still have a valid inequality.
When you divide this by n, what do you get?
You get H of x.
When you divide this by n, by definition L bar -- what we mean is the number of bits per source symbol.
We have n source symbols here.
So when we divide by n, we get the number of bits per source symbol in this monster symbol.
So l min is equal to this.
When we divide this by n, we get this.
When we divide this by n, we get H of x.
And now the whole reason for doing this is, this silly little 1 here, which we're trying very hard to think of as being negligible or unimportant, has suddenly become 1 over n. And by making n big enough, this truly is unimportant.
If you're thinking in terms of encoding a binary source, this 1 here is very important.
In other words, when you're trying to encode a binary source, if you're encoding one letter at a time, there's nothing you can do.
You're absolutely stuck.
Because if both of those letters have non-zero probabilities, and you want a uniquely decodable code, and you want to find codewords for each of those two symbols, the best you can do is to have an expected length of 1.
Namely, you need 1 to encode 1, and you need 0 to encode 0.
And there's nothing else, there's no freedom at all that you have.
So you say, OK, in that situation, I really have to go to longer blocks.
And when I go to longer blocks, I can get this resolved.
And I know how to do it.
I use Huffman's algorithm or whatever.
So, suddenly, I can start to get the expected number of bits per source symbol.
Down as close to h of x as I want to make it.
And I can't make it any smaller.
Which says that H of x now has a very clear interpretation, at least for prefix-free codes, of being the number of bits you need for encoding prefix-free codes when you allow the possibility of encoding them a block at a time.
We're going to find later today that the significance of this is far greater than that, even.
Because why use prefix-free codes, we could use any old kind of code.
When we study the Lempel-Ziv codes tomorrow, we'll find out they aren't prefix-free codes at all.
They're really variable length of variable length codes.
So they aren't fixed to variable length.
And they do some pretty fancy and tricky things.
But they're still limited to this same inequality.
You can never beat the entropy bound.
If you want something to be uniquely decodable, you're stuck with this bound.
And we'll see why in a very straightforward way, later.
It's a very straightforward way which I can guarantee all of you are going to look at it and say, yes, that's obvious.
And tomorrow you will look at it and say, I don't understand that at all.
And the next day you'll look at it again and say, well, of course.
And the day after that you'll say, I don't understand that.
And you'll go back and forth like that for about two weeks.
Don't be frustrated, because it is simple.
But at the same time it's very sophisticated.
So, let's now review the weak law of large numbers.
I usually just call it the law of large numbers.
I bridle a little bit when people call it weak because, in fact it's the centerpiece of probability theory.
But there is this other thing called the strong law of large numbers, which mathematicians love because it lets them look at all kinds of mathematical minutiae.
It's also important, I shouldn't play it down too much.
And there are places where you truly need it.
For what we'll be doing, we don't need it at all.
And the weak law of large numbers does in fact hold in many places where the strong law doesn't hold.
So if you know what the strong law, is temporarily forget it for the for the rest of the term.
And just focus on the weak law.
And the weak law is not terribly complicated.
We have a sequence of random variables.
And each of them has a mean y bar.
And each of them has a variance sigma sub y squared.
And let's assume that they're independent and identically distributed for the time being.
Just to avoid worrying about anything.
If we look at the sum of those random variables, namely a is equal to y1 up to y sub n. Then the expected value of a is the expected value of this plus the expected valuable of y2, and so forth.
So the expected value of a is n times y bar.
And I think in one of the homework problems, you found the variance of a.
And the variance of a, well, the easiest thing to do is to reduce this to its fluctuation.
Reduce all of these to their fluctuation.
Then look at the variance of the fluctuation, which is just the expected value of this squared.
Which is the expected value of this squared plus the expected value of this squared, and so forth.
So the variance of a is n times sigma sub y squared.
I want all of you know that.
That's sort of day two of a probability course.
As soon as you start talking about random variables, that's one of the key things that you do.
One of the most important things you do.
The thing that we're interested in here is more the sample average of y1 up to y sub n. And the sample average, by definition, is the sum divided by n. So in other words, the thing that you're interested in here is to add all of these random variables up.
Take one over n times it.
Which is a thing we do all the time.
I mean, we sum up a lot of events, we divide by n, and we hope by doing that to get some sort of typical value.
And, usually there is some sort of typical value that arises from that.
What the law of large numbers says is that there in fact is a typical value that arises.
So this sample value is a over n, which is the sum divided by n. And we call that the sample average.
The mean of the sample average is just the mean of a, which is n times y bar, divided by n. So the mean of the sample average is y bar itself.
The variance of the sample variance, -- the variance of the sample average, OK, that's, -- I'm talking too fast.
The sample average here, you would like to think of it as something which is known and specific, like the expected value.
It, in fact, is a random variable.
It changes with different sample values.
It can change from almost nothing to very large quantities.
And what we're interested in saying is that most of the time, it's close to the expected value.
And that's what we're aiming at here.
And that's what the law of large numbers says.
The sample average here, the variance of this, is now equal to the variance of a divided by n squared.
In other words, we're trying to take the expected value of this quantity squared.
So there's a 1 over n squared that comes in here.
When you take the 1 over n squared here, this variance then becomes sigma -- I don't know why I have the n there.
Just take that n out, if you will.
I don't have my red pen with me.
And so it's the variance of the random variable y, divided by n. In other words, the limit as n goes to infinity of the of the variance of the sum is equal to infinity.
And the variance of the sample average as n goes to infinity is equal to 0.
And that's because of this 1 over n squared effect here.
When you add up a lot of independent random variables, what you wind up with is the sample average has a variance, which is going to 0.
And the sum has a variance which is going to infinity.
That's important.
Aside from all of the theorems you've ever heard, this is sort of the gross, simple-minded thing which you always ought to keep foremost in your mind.
This is what's happening in probability theory.
When you talk about sample averages, this variance is getting small.
Let's look at a picture of this.
Let's look at the distribution function of this random variable.
The sample average as a random variable.
And what we're finding here is that this distribution function, if we look at it for some modest value of n, we get something which looks like this upper curve here.
Which is then the lower curve here.
It's spread out more, so it has a larger variance.
Namely, the sample average has a larger variance.
When you make n bigger, what's happening to the variance?
The variance is getting smaller.
The variance is getting smaller by a factor of 1/2.
So this quantity is supposed to have a variance which is equal to 1/2 of the variance of this.
How you find a variance in a distribution function is your problem.
But you know that if something has a small variance, it's very closely tucked in around this.
In other words, as the variance goes to 0, and the mean is y bar, you have a distribution function which approaches a unit step.
And all that just comes from this very, very simple argument that says, when you have a sum of IID random variables and you take the sample average of it, namely, you divide by n, the variance goes to 0.
Which says, no matter how you look at it, you wind up with something that looks like a unit step.
Now, the Chebyshev inequality, which is one of the simpler inequalities in probability theory, and I don't prove it because it's something you've all seen.
I don't know of any course in probability which avoids the Chebyshev inequality.
And what it says is, for any epsilon greater than 0, the probability that the difference between the sample average and the true mean, the probability that that quantity and magnitude is greater than or equal to epsilon, is less than or equal to sigma sub y squared divided by n epsilon squared.
Oh, incidentally that thing that was called sigma sub n before was really sigma squared.
That's mainly the variance of y. I hope it's right in the notes.
Might not be.
It is?
Good.
So, that's what this inequality says.
There's an easy way to derive this on the fly.
Namely, if you're wondering what all these constants are here, here's a way to do it.
What we're looking at, in this curve here, is we're trying to say, how much probability is there outside of these plus and minus epsilon limits.
And the Chebyshev inequality says there can't be too much probability out here.
And there can't be too much probability out here.
So, one way to get at this is to say, OK, suppose I have some given probability out here.
And some given probability out here.
And suppose I want to minimize the variance of a random variable which has that much probability out here and that much probability out here.
How do I do it?
Well, the variance deals with the square of how far you were away from the mean.
So if I want to have a certain amount of probability out here, I minimize my variance by making this come straight, come up here with a little step, then go across here.
Go up here.
And then, oops.
Go up here.
I wish I had my red pencil.
Does anybody have a red pen?
That will write on this stuff?
Yes?
No?
Oh, great.
Thank you.
I will return it.
So what we want is something which goes over here.
Comes up here.
Goes across here.
Goes up here.
Goes across here, and goes up again.
That's the smallest you can make the variance.
It's squeezing everything in as far as it can be squeezed in.
Namely, everything out here gets squeezed in to y minus epsilon.
Everything in here gets squeezed into 0.
And everything out here gets squeezed into epsilon.
OK, calculate the variance there.
And that satisfies the Chebyshev inequality with equality.
So that's all the Chebyshev inequality is.
And it's a loose inequality usually, because usually these curves look very nice.
Usually this looks like a Gaussian distribution function, and the central limit theorem says that we don't need the central limit theorem here, and we don't want the central limit theorem here, because this thing is an inequality that says, life can't be any worse than this.
And all the central limit theorem is, is an approximation.
And then we have to worry about when it's a good approximation and when it's not a good approximation.
So this says, when we carry it one piece further, it's for any epsilon and delta greater than 0.
If we make n large enough -- in other words, substitute delta for this.
And then, when you make n small enough, this quantity is smaller than delta.
And that says that the probability that s and y differ by more than epsilon is less than or equal to delta when we make n big enough.
So it says, you can make this as small as you want.
You can make this as small as you want.
And all you need to do is make a sequence which is long enough.
Now, the thing which is mystifying about the law of large numbers is, you need both the epsilon and the delta.
You can't get rid of either of them.
In other words, you can't say -- you can't reduce this to 0.
Because it won't make any sense.
In other words, this curve here tends to spread out on its tails.
And therefore, there's always a probability of error there.
You can't move epsilon into 0 because, for no finite end, do you really get a step function here.
So you need some wiggle room on both end.
You need wiggle room here, and you need wiggle room here.
And once you recognize that you need those two pieces of wiggle room.
Namely, you cannot talk about the probability that this is equal to y bar, because that's usually 0.
And you cannot talk about reducing this to 0 either.
So both of those are needed.
Why did I go through all of this?
Well, partly because it's important.
But partly because I want to talk about something which is called the asymptotic equipartition property.
And because of those long words you believe this has to be very complicated.
I hope to convince you that what the asymptotic equipartition property is, is simply the week law of large numbers applied to the log pmf.
Because that, in fact, is all it is.
But it says some unusual and fascinating things.
So let's suppose that x1, x2, and so forth is the output from a discrete memoryless source.
Look at the log pmf of each of these.
Namely, they each have the same distribution function.
So w of f x is going to be equal to minus the logarithm of p sub x of x, for each of these chance variables.
w of x maps source symbols into real numbers.
So there's a random variable, capital W of x sub j, which is a random variable.
We have a random variable for each one of these symbols to come out of the source.
So, for each one of these symbols, there's this log pmf random variable, which takes on different values.
So the expected value of this log pmf, for the j'th symbol out of the source is the sum of p sub x of x, namely the probability that the source takes on the value x, times minus the logarithm of p sub x.
And that's just the entropy.
So, the strange feeling about this log pmf random variable is its expected value is entropy.
And w of x1, w of x2, and so forth, are a sequence of IID random variables, each one of them which has a mean, which is the entropy.
So it's just exactly the situation we had before.
Instead of y bar, we have the entropy of x.
And instead of the random variable y sub j, we have this random variable w of x sub j, which is defined by the symbol in an alphabet.
And just to review this, but it's what we said before, if capital X1, this little x1, namely, if little x1 is the sample value for the chance variable x1 and if x2 is a sample value for the chance variable X2, then the outcome for w of x1 plus w of x2 -- very hard to keep all these little letters and capital letters straight.
Is w of x1 plus w of x2 is minus the logarithm of the probability of x1 minus the logarithm of the probability of x2, which is minus the logarithm of the product, which is minus the logarithm of the joint probability of x1 and x2, which is the random variable w of x1 x2.
So the sum here is the random variable corresponding to log pmf of the joint outputs x1 and x2.
So w of x1 x2 is the log pmf of the event, but this joint chance variable takes on the value x1 x2.
And the random variable x1 x2 is the sum of x1 and x2.
So, what have I done here?
I said this at the end of the last slide, and you won't believe me.
So, again this is one of these things where tomorrow you won't believe me.
And you'll have to go back and look at that.
But, anyway, x1 x2 is a chance variable.
And probabilities multiply in log pmf's add, which is what we've been saying for a couple of days now.
So.
If I look at the sum of n of these random variables, the sum of these log probabilities is the sum of the log of pmf's, which is minus the logarithm of the probability of the entire sequence.
That's just saying the same thing we said before, for two random variables.
The sample average of a log pmf's is the sum of the w's divided by n, which is minus the logarithm of the probability divided by n. The weak law of large numbers applies, and the probability that this sample average minus the expected value of w of x is greater than or equal to epsilon is less than or equal to this quantity here.
This quantity is minus the logarithm of the probability of x sub n, divided by n, minus H of x, greater than or equal to epsilon.
So this is the thing that we really want.
I'm going to spend a few slides trying to say what this means.
But let's try to just look at it now, and see what it must mean.
It says that with high probability, this quantity is almost the same as this quantity.
It says that with high probability, the thing which comes out of the source is going to have a probability, a log probability, divided by n, which is close to the entropy.
It says in some sense that with high probability, the probability of what comes out of the source is almost a constant.
And that's amazing.
That's what you'll wake up in the morning and say, I don't believe that.
But it's true.
But you have to be careful to interpret it right.
So, we're going to define the typical set.
Namely, this is the typical set of x's, which come out of the source.
Namely, the typical set of blocks of n symbols out of the source.
And when you talk about a typical set, you want something which includes most of the probability.
So what I'm going to include in this typical set is all of these things that we talked about before.
Namely, we showed that the probability that this quantity is greater than or equal to epsilon is very small.
So, with high probability what comes out of the source satisfies this inequality here.
So I can write down the distribution function of this random variable here.
It's just this w -- this is a random variable w. I'm looking at the distribution of that random variable w. And this quantity in here is the probability of this typical set.
In other words, when I draw this distribution function for this combined random variable, I've defined this typical set to be all the sequences which lie between this point and this point.
Namely, this is H minus epsilon.
And this is H plus epsilon, moving H out here.
So these are all the sequences which satisfy this inequality here.
So that's what I mean by the typical set.
It's all things which are clustered around H in this distribution function.
And as n approaches infinity, this typical set approaches probability 1.
In the same way that the law of large numbers behaves that way.
The probability that x sub n is in this typical set is greater than or equal to 1 minus sigma squared divided by n epsilon squared.
Let's try to express that in a bunch of other ways.
If you're getting lost, please ask questions.
But I hope to come back to this in a little bit, after we finish a little more of the story.
So, another way of expressing this typical set -- let me look at that as the typical set.
If I take this inequality here and I rewrite this, namely, this is the set of x's for which this is less than epsilon plus H of x and greater than H of x minus epsilon.
So that's what I've expressed here.
It's the set of x's for which n times H of x minus epsilon is less than this logarithm of probability is great less than n times H of x plus epsilon.
Namely, I'm looking at this range of epsilon around H, which is this and this.
If I write it again, if I exponentiate all of this, it's the set of x's for which 2 to the minus n, H of x, plus epsilon, that's this term exponentiated, is less than this is less than this term exponentiated.
And what's going on here is, I've taken care of the minus sign also.
And if you can follow that in your head, you're a better person than I am.
But, anyway, it works.
And if you fiddle around with that, you'll see that that's what it is.
So this typical set is a bound on the probabilities of all these typical sequences.
The typical sequences all are enclosed in this range of probabilities.
So the typical elements are approximately equiprobable, in this strange sense above.
Why do I say this is a strange sense?
Well, as n gets large, what happens here?
This is 2 to the minus n times H of x.
Which is the important part of this.
This epsilon here is multiplied by n. And we're trying to say, as n gets very, very big, we can make epsilon very, very small.
But we really can't make n times epsilon very negligible.
But the point is, the important thing here is, 2 to the minus n times H of x.
So, in some sense, this is close to 2 to the minus n H of x.
In what sense is it true?
Well, it's true in that sense.
Where that, in fact is, a valid inequality.
Namely in terms of sample averages, these things are close.
When I do the exponentiation and get rid of the n and all that stuff, they aren't very close.
But saying this sort of thing is sort of like saying that 10 to the minus 23 is approximately equal to 10 to the minus 25.
And they're approximately equal because they're both very, very small.
And that's the kind of thing that's going on here.
And you're trying to distinguish 10 to the minus 23 and 10 to the minus 25 from 10 to the minus 60th and from 10 to the minus three.
So that's the kind of approximations we're using.
Namely, we're using approximations on a log scale, instead of approximations of ordinary numbers.
But, still it's convenient to think of these typical x's, typical sequences, as being sequences which are constrained in probability in this way.
And this is the thing which is easy to work with.
The atypical set of strings, namely, the compliment to this set, the thing we know about that is the entire set doesn't have much probability.
Namely, if you fix epsilon and you let n get bigger and bigger, this atypical set becomes totally negligible.
And you can ignore it.
So let's plow ahead.
Stop for an example pretty soon, but -- If I have a sequence which is in the typical set, we then know that its probability is greater than 2 to the minus n times H of x plus epsilon.
That's what we said before.
And, therefore, when I use this inequality, the probability of x to the n, for something in the typical set, is greater than this quantity here.
In other words, this is greater than that.
For everything in a typical set.
So now I'm heading over things in a typical set.
So I need to include the number of things in a typical set.
So what I have is this sum.
And what is this sum?
This is the probability of the typical set.
Because I'm adding overall elements in the typical set.
And it's greater than or equal to the number of elements in a typical set times these small probabilities.
If I turn this around, it says that the number of elements in a typical set is less than 2 to the n times H of x plus epsilon.
For any epsilon, no matter how small I want to make it.
Which says that the elements in a typical set have probabilities which are about 2 to the minus n times H of x.
And the number of them is approximately 2 to the n times H of x.
In other words, what it says is that this typical set is a bunch of essentially uniform probabilities.
So what I've done is to take this very complicated source.
And when I look at these very humongous chance variables, which are very large sequences out of the source, what I find is that there's a bunch of things which collectively have zilch probability.
There's a bunch of other things which all have equal probability.
And a number of them is enough to add up to y.
So I have turned this source, when I look at it over a long enough sequence, into a source of equiprobable events.
And each of those events has this probability here.
Now, we know how to encode equiprobable events.
And that's the whole point of this.
So, this is less than or equal to that.
On the other side, we know that 1 minus delta is less than or equal to this probability of a typical set.
And this is less than the number of elements in a typical set times 2 to the minus n h of x minus epsilon.
This is an upper bound on this.
This is less than this.
So I just add all these things up and I get this bound.
So it says, the size of the typical set is greater than 1 minus delta, times this quantity.
In other words, this is a pretty exact sort of thing.
If you don't mind dealing with this 2 to the n epsilon factor here.
If you agree that that's negligible in some strange sense, the all of this makes good sense.
And if it is negligible, let me start talking about source coding, which is why this all works out.
So the summary is that the probability of the complement of the typical set is essentially 0.
The number of elements in a typical set is approximately 2 to the n times h of x. I'm getting rid of all the deltas and epsilons here, to get sort of the broad view of what's important here.
Each of the elements in a typical set has probability 2 to the minus n times H of x.
So I've turned a source into a source of equiprobable elements.
And there are 2 to the n times h of x of them.
Let's do an example of this.
It's an example that you'll work on more in the homework and do it a little more cleanly.
Let's look at a binary discrete memoryless source, where the probability that x is equal to 1 is p, which is less than 1/2.
And the probability of 0 is greater than 1/2.
So, this is what you get when you have a biased coin.
And the biased coin has a 1 on one side and a 0 on the other side.
And it's more likely to come up 0's than 1's.
I always used to wonder how to make a biased coin.
And I can give you a little experiment which shows you you can make a biased coin.
I mean, a biased is a little round thing which is flat on the top and bottom.
Suppose instead of that you make a triangular coin.
And instead of making it flat on top and bottom, you turn it into a tetrahedron.
So in fact, what this is now is a coin which is built up on one side into a very massive thing.
And is flat on the other side.
Since it's a tetrahedron and it's an equilateral tetrahedron, the probability of 1 is going to be 1/4, and the probability of 0 is going to be 3/4.
So you can make biased coins.
So when you get into coin-tossing games with people, watch the coin that they're using.
It probably won't be a tetrahedron, but anyway.
So the entropy here, the log pmf random variable, takes on the value of minus log p with probability p. And it takes on the value minus log 1 minus p, with probability 1 minus p. This is a probability of a 1.
This is a probability of a 0.
So, the entropy is equal to this.
Used to be that in information theory courses, people would almost memorize what this curve looked like.
And they'd draw pictures of it.
There were famous curves of this function, which looks like this.
0, 1, 1.
Turns out, that's not all that important a distribution.
It's a nice example to talk about.
The typical set, t epsilon n, is the set of strings with about p n1's and about 1 minus p times n 0's.
In other words, that's the typical thing to happen.
And it's the typical thing in terms of this law of large numbers here.
Because you get 1's with probability p. And therefore in a long sequence, you're going to get about pn 1's and 1 minus p 0's.
The probability of a typical string is, if you get a string with this many 1's and this many 0's, it's probability is p. Namely, the probability of a 1 times the number of 1's you get, which is pn.
Times the probability of a 0, times the number of 0's you get.
And if you look at what this is, if you take p up in the exponent and 1 minus the p up in the exponent, this becomes 2 to the minus n times h of x, just like what it should be.
So these typical strings, with about pn 1's and 1 minus pn 0's, are in fact typical in the sense we've been talking about.
The number of n strings with pn 1's is n factorial divided by pn factorial divided by n times 1 minus p factorial.
I mean I hope you learned that a long time ago, but you should learn it in probability anyway.
It's just very simple combinatorics.
So you have that many different strings.
So what I'm trying to get across here is, there are a bunch of different things going on here.
We can talk about the random variable which is the number of 1's that occur in this long sequence.
And with high probability, the number of 1's that occur is close to pn.
But if pn 1's occur, there's still an awful lot of randomness left.
Because we have to worry about where those pn 1's appear.
And those are the sequences we're talking about.
So, there are this many sequences, all of which have that many 1's in them.
And there's a similar number of sequences for all similar numbers of 1's.
Namely, if you take pn plus 1 and pn plus 2, pn minus 1, pn minus 2, you get similar numbers here.
So those are the typical sequences.
Now, the important thing to observe here is that you really have 2 to the n binary strings altogether.
And what this result is saying is that collectively those don't make any difference.
The law of large numbers says, OK, there's just a humongous number of strings.
You get the largest number strings which have about half 1's and half 0's.
But their probability is zilch.
So the thing which is probable is getting pn 1's and 1 minus pn 0's.
Now, we have this typical set.
What is the most likely sequence of all, in this experiment?
How do I maximize the probability of a particular sequence?
The probability of the sequence is p to the i times 1 minus p to the n minus i.
And 1 minus p is the probability of 0.
And p is the probability of a 1.
How do I choose i to maximize this?
Yeah?
[UNINTELLIGIBLE] all 0's
You make them all 0's.
So the most likely sequence is all 0's.
But that's not a typical sequence.
Why isn't it a typical sequence?
Because we chose to define typical sequence in a different way.
Namely is only one of those, and there are only n of them with only a single one.
So, in other words, what's going on is that we have an enormous number of sequences which have around half 1's and half 0's.
But they don't have any probability.
And collectively they don't have any probability.
We have a very small number of sequences which have a very large number of 0's.
But there aren't enough of those to make any difference.
And, therefore, the things that make a difference are these typical things which have about np 1's and 1 minus pn 0's.
And that all sounds very strange.
But if I phrase this a different way, you would all say that's exactly the way you ought to do things.
Because, in fact, when we look at very, very long sequences, you know with extraordinarily high probability what's going to come out of the source is something with about pn 1's and about 1 minus p times n 0's.
So that's the likely set of things to have happen.
And it's just that there are an enormous number of those things.
There are this many of them.
So, here what we're dealing with is a balance between the number of elements of a particular type, and the probability of them.
And it turns out that this number and its probability balance out to say that usually what you get is about pn 1's and 1 minus p times n 0's.
Which is what the law of large numbers said to begin with.
All we're doing is interpreting that here.
But the thing that you see from this example is, all of these things with exactly pn 1's in them, assuming that pn is an integer, are all equiprobable.
They're all exactly equiprobable.
So what we're doing when we're talking about this typical set, is first throwing out all the things which have to many 1's are or too few 1's in them.
We're keeping only the ones which are typical in the sense that they obey the law of large numbers.
And in this case, they obey the law of large numbers for log pmf's also.
And then all of those things are about equally probable.
So the idea in source coding is, one of the ways to deal with source coding is, you want to assign codewords to only these typical things.
Now, maybe you might want to assign codewords to something like all 0's also.
Because it hardly costs anything.
And a Huffman code would certainly do that.
But it's not very important whether you do or not.
The important thing is, you assign codewords to all of these typical sequences.
So let's go back to fixed-to-fixed length source codes.
We talked a little bit about fixed-to-fixed length source codes before.
Do you remember what we did with fixed-to-fixed length source codes before?
We said we have an alphabet of size m. We want something which is uniquely decodable.
And since we want something which is uniquely decodable, we have to provide codewords for everything.
And, therefore, if we want to choose a block length of n, we've got to generate m to the n codewords.
Here we say, wow, maybe we don't have to provide codewords for everything.
Maybe we're willing to tolerate a certain small probability that the whole thing fails and falls on its face.
Now, does that make any sense?
Well, view things the following way.
We said, when we started out all of this, that we were going to look at prefix-free codes.
Where some codewords had a longer length and some codewords had a shorter length.
And we were thinking of encoding either single letters at a time, or a small block of letters at a time.
So think of encoding, say, 10 letters at a time.
And think of doing this for 10 to the 20th letters.
So you have the source here which is pumping out letters at a regular rate.
You're blocking them into n letters at a time.
You're encoding in a prefix-free code.
Out comes something.
What comes is not coming out at a regular right.
What is coming out, sometimes you get a lot of bits out.
Sometimes a small number of bits out.
So, in other words, if you want to send things over a channel, you need a buffer there to save things.
If, in fact, we decide that the expected number of bits per source letter is, say, five bits per source letter, then we expect over a very long time to be producing five bits per source letter.
And if we turn our channel on for one year, to transmit all of these things, what's going to happen is this very unlikely sequence occurs.
Which in fact requires not one year to transmit, but two years to transmit.
In fact, what do we do if it takes one year and five minutes to transmit instead of one year?
Well, we've got a failure.
Somehow or other, the network is going to fail us.
I mean we all know that networks fail all the time despite what engineers say.
I mean, all of us who use networks know that they do crazy things.
And one of those crazy things is that unusual things sometimes happen.
So, we develop this very nice theory of prefix-free codes.
But prefix-free codes, in fact, fail also.
And they fail also because buffers overflow.
In other words, we are counting on encoding things with a certain number of bits per source symbol.
And if these unusual things occur, and we have too many bits per source symbol, then we fail.
So the idea that we're trying to get at now is that prefix-free codes and fixed-to-fixed length source codes which only encode typical things.
In fact, are sort of the same if you look at them over a very, very large sequence length.
In other words, if you look at a prefix-free code which is dealing with blocks of 10 letters, and you look at a fixed-to-fixed length code which is only dealing with typical things but is looking at a length of 10 to the 20th, then over that length of 10 to the 20th, your variable length code is going to have a bunch of things which are about the length they ought to be.
And a bunch of other things which are extraordinarily long.
The bunch of things which are extraordinarily long are extraordinarily unpopular, but there are an extraordinarily large number of them.
Just like with a fixed-to-fixed length code, you are going to fail.
And you're going to fail on an extraordinary number of different sequences.
But, collectively, that set of sequences don't have any probability.
So the point that I'm trying to get across is that, really, these two situations come together when we look very long lengths.
Namely, prefix-free codes are just a way of generating codes that work for typical sequences and over a very large, long period of time, will generate about the right number of symbols.
And that's what I'm trying to get at here.
Or what I'm trying to get at in the next slide.
So the fixed-to-fixed length source code, I'm going to pick some epsilon and some delta.
Namely, that epsilon and delta which appeared in the law of large numbers.
I'm going to make n as big as I have to make it for that epsilon and that delta.
And calculate how large it has to be, but we won't.
Then I'm going to assign fixed length codewords to each sequence in the typical set.
Now, am I going to really build something which does this?
Of course not.
I mean, I'm talking about truly humongous lengths.
So, this is really a conceptual tool to understand what's going on.
It's not something we're going to implement.
So I'm going to assign codewords to all these typical elements.
And then what I find is that since the typical set, since the number of elements in it is less than 2 to the n times H of x plus epsilon, if I choose L bar, namely, the number of bits I'm going to use for encoding these things, it's going to have to be H of x plus epsilon in length.
Because I need to provide codewords for each of these things.
And it needs to be an extra 1 over n because of this integer constraint that we've been dealing with all along, which doesn't make any difference.
So if I choose L bar, that big, in other words, if I make it just a little bit bigger than the entropy, the probability of failure is going to be less than or equal to delta.
And I can make delta -- and I can make the probability of failure as small as I want.
So I can make this epsilon here which is the extra bits per source symbol as small as I want.
So it says I can come as close to the entropy bound in doing this, and come as close to unique decodability as I want in doing this.
And I have a fixed-to-fixed length code, which after one year is going to stop.
And I can turn my decoder off.
I can turn my encoder off.
I can go buy a new encoder and a new decoder, which presumably works a little bit better.
And there isn't any problem about when to turn it off.
Because I know I can turn it off.
Because everything will have come in by then.
Here's a more interesting story.
Suppose I choose the number of bits per source symbol that I'm going to use to be less than or equal to the entropy minus 2 epsilon.
Why 2 epsilon?
Well, just wait a second.
I mean, 2 epsilon is small and epsilon is small.
But I want to compare with this other epsilon and my law of large numbers.
And I'm going to pick n large enough.
The number of typical sequences, we said before, was greater than 1 minus delta times 2 to the n times h of x minus epsilon.
I'm going to make this epsilon the same as that epsilon, which is why I wanted this to be 2 epsilon.
So my typical set is this big when I choose n large enough.
And this says that most of the typical set can't be assigned codewords.
In other words, this number here is humongously larger then 2 to the l bar, which is in the order of 2 to the nh of x minus 2 epsilon n. So the fraction of typical elements that I can provide codewords for, between this and this, I can only provide codewords for a fraction 2 to the minus epsilon n of the codewords.
We have this big sea of codewords, which are all essentially equally likely.
And I can't provide codewords for even a small fraction of them.
So the probability of failure is going to be 1 minus delta.
The 1 minus delta's the probability that I get something atypical.
Plus, well, minus in this case, 2 to the minus epsilon n, which is the probability that I can't encode a typical codeword that comes out.
And this quantity goes to 1.
So this says that if I'm willing to use a number of bits bigger than the entropy, I can succeed with probability very close to 1.
And if I want to use a smaller number of bits, I fail with probability 1.
Which is the same as saying that I'm using a prefix-free code, I'm going to run out of buffer space eventually if I run long enough.
If I have something that I'm encoding -- well, just erase that.
I'll say it more carefully later.
I do want to talk a little bit about this Kraft inequality for unique decodability.
You remember we proved the Kraft inequality for prefix-free codes.
I now want to talk about the Kraft inequality for uniquely decodable codes.
And you might think that I've done all of this development of the AEP, the asymptotic equipartition property.
Incidentally, you now know where those words come from.
It's asymptotic because this result is valid asymptotically as n goes to infinity.
It's equipartition because everything is equally likely.
And its property, because it's a property.
So it's the asymptotic equipartition property.
And I didn't do it so I could prove the Kraft inequality.
It's just that that's an extra bonus that we get.
And by understanding why the Kraft inequality has to hold for uniquely decodable codes, if is one application for AEP which lets you see a little bit about how to use it.
OK, so the argument is an argument by contradiction.
Suppose you generate a set of lengths for codewords.
And you want this -- yeah?
And the thing you would like to do is to assign codewords of these lengths.
And what we want to do is to set this equal to some quantity b.
In other words, suppose we beat the Kraft inequality.
Suppose we can make the lengths even shorter than Kraft says we can make them.
I mean, he was only a graduate student, so we've got to be able to beat his inequality somehow.
So we're going to try to make this equal to b.
We're going to assume that b is greater than 1.
And then what we're going to do is to show that we get a contradiction here.
And this same argument can work whether we have a discrete memoryless source or a source with memory, or anything else.
It can work with blocks, it can work with variable length to variable length codes.
It's all essentially the same argument.
So what I want to do is to get a contradiction.
I'm going to choose a discrete memoryless source.
And I'm going to make the probabilities equal to 1 over b times 2 to the minus li.
In other words, I can generate a discrete memoryless source for talking about it with any probabilities I want to give it.
So I'm going to generate one with these probabilities.
So the lengths are going to be equal to minus log of b times p sub i.
Which says that the expected length of the codewords is equal to the sum of p sub i l sub i, which is equal to the entropy minus the logarithm of b.
Which means I can get an expected length which is a little bit less than the entropy.
So now what I'm going to do is to consider strings of n source letters.
I'm going to make these string very, very long.
When I concatenate all these codewords, I'm going to wind up with a length that's less than n times H of x minus b over 2, minus log b over 2 with high probability.
And as a fixed-length code of this length it's going to have a low failure probability.
And, therefore, what this says is I can, using this remarkable code with unique decodability, and generating very long strings from it, I can generate a fixed-length code which has a low failure probability.
And I just showed you in the last slide that I can't do that.
The probability of failure with such a code has to be essentially 1.
So that's a contradiction that says you can't have these unique decodable codes.
If you didn't get that in what I said, don't be surprised.
Because all I'm trying to do is to steer you towards how to look at the section in the notes that does that.
It was a little too fast and a little too late.
But, anyway, that is the Kraft inequality for unique decodability.
OK, thanks.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The AEP is probably one of the most difficult concepts we talk about in this course.
It seems simple to start with.
As I said before, it's one of those things where you think you understand it, and then you think you don't understand it.
When Shannon first came out with this theory, there were a lot of very, very good professional mathematicians who spent a long time trying to understand this.
And who, in fact, blew it.
Because, in fact, they were trying to look at it strictly in terms of mathematics.
They were looking for strict mathematical theorems, they weren't looking to try to get the insight from it.
Because of that they couldn't absorb it.
There were a lot of engineers who looked at it, and couldn't absorb it because they couldn't match it with any of the mathematics, and therefore they started thinking there was more there then there really was so, so this is, in fact, tricky.
What we're looking at is a sequence of chance variables coming from a discrete memoryless source.
In other words, a discrete memoryless source is something that spits out symbols and each symbol is independent of each other symbol, each symbol has the same probability mass function as every other symbol.
We said it's a neat thing to look at this log pmf random variable, and a log pmf random variable is minus log of the probability of the particular symbol.
So we have a sequence now of random variables.
And the expected value of that random variable, for each of the random variables, is the expected value of the log pmf which is, as we said before, is this thing we've called the entropy, which we're trying to get some insight into.
So we have this log pmf random variable, we have the entropy, which is the expected value of it.
We then talked about the sequence of these random variables.
We talked about the sample average of them, and the whole reason you want to look at this random variable is the sample average of a lot of these pmf's, since logs add when the thing you're taking the log of multiplies.
What you wind up with is the sum of these log pmf's is, in fact, equal to minus the lof of the probability of the whole sequence.
In other words, you look at this whole sequence as a big, giant chance variable, which has capital X to the n different possible values.
Namely, every possible sequence of length n. When you're talking about source coding, you have to find the code word for each of those m to the n different sequences.
So what you're doing here is trying to look at the probability of each of those m to the n sequences.
We can then use Huffman coding, or whatever we choose to use, to try to encode those things.
OK, the weak law of large numbers applies here, and says that the probability that this sample average of the log pmf is close to the expected value of the log pmf.
The probability that that's greater than or equal to epsilon is less than or equal to the variance of the log pmf random variable, divided by n times epsilon squared.
Now, we are going to take the viewpoint that n epsilon squared, is very small.
Excuse me, it's very large.
Epsilon we're thinking as a small number, and we're thinking of as a large number, but the game we always play here is to first pick some epsilong, as small as you want to make it, and then you make n larger and larger.
And as n gets bigger and bigger, eventually this number gets very small.
So that's the game that we're playing.
It's y n times epsilon squared, we're thinking of it is a large number.
So what this says is the probability that this log pmf of the entire sequence, well the sample value of it, namely the log pmf divided by n, us close to H of X.
The probability of that is less than or equal to this.
We then define the typical set, T sub epsilon of n. This is the set of all typical sequences out of the source.
So we're defining typical sequences in this way.
It's a set of all sequences which are what we put into there.
Well, it's what we put into there, but we're looking at the compliment of this set.
These are the exceptional things.
These are the typical things.
We're saying that the exceptional things here have a small probability when you make n big enough.
This is saying that the exceptional things don't amount to anything, and the typical things fill up the entire probability space.
So what were saying is the probability that this sequence is actually typical is greater than or equal to 1 minus something small.
It says when n gets big enough the probability that you get a typical sequence out of the source is going to wan.
OK. We drew this in terms of a probability distribution, which I hope makes things look a little more straightforward.
This is the distribution function of the sample average of this log pmf random variable.
And what we're saying is that as n gets larger and larger, the thing that's going to happen because of the fact that the variance of the sample average -- the sample average is always a random variable.
The average is not a random variable, it's a fixed number.
And what this is saying is that as n gets larger and larger, this distribution function here gets closer and closer to a stack.
In other words, the thing that's happening is that as n gets big, you go along here, nothing happens suddenly you move up here and suddenly you move across there.
That's what happens in the limit, mainly the sample average is, at that point, always equal to the average.
So as n goes to infinity, a typical set approaches probability 1.
We express that in terms of the Chebyshev inequality in this way, but the picture says you can interpret it in any one of a hundred different ways.
Because the real essence of this is not the Chebyshev inequality, the real essence of it is saying that as n gets bigger and bigger, this distribution function becomes a stack.
So that's a nice way of thinking about it.
Let's summarized what we did with that.
All of the major results about typical sets.
The first of them, is this one.
It's a bound on the number of elements in a typical set, I'm not going to rederive that again, but it comes from looking at the fact that all of the typical elements have roughly the same probability.
All of them collectively fill up the whole space, and therefore you can find the probability of each of them by taking one and dividing by the probability of each of them.
When you do that, you get two bounds.
One of them says that this magnitude is less than 2 to the n times H of X plus epsilon.
The other one says it's greater than 2 to the n H of X minus epsilon.
And you have to throw in this little fudge factor here because of the fact that the typical set doesn't quite fill up the whole space.
What that all says is when n gets large the number of typical elements is approximately equal to 2 to the n times H of X.
Again, think hard about what this means.
This number here really isn't all like 2 to the n H of X.
This n times epsilon, is going to be a big number.
As n gets bigger, it gets bigger and bigger.
But, at the same time, when you look at 2 to the n times H of X plus epsilon, and you know that H of X is something substantial and you know that epsilon is very small, in some sense it still says this.
And in source coding terms, it very much says this.
Because in source coding terms, what you're always looking at is these exponents.
Because if you have n different things you're trying to encode, it takes you log of n bits to encode them.
So when you take the log of this, you see that the number of extra bits you needs to encode these sequences is on the order of n times epsilon.
Which is some number of bits, but the real number bits you're looking at is n times H of X.
So that's the major term, and this is just fiddly.
The next thing we found is that the probability of an element in a typical set is between 2 the minus n times H of X plus epsilon and 2 to the minus n times H of X minus epsilon.
Which is saying almost the same thing as this is saying.
And again, there's this approximation which says this is about equal to 2 to the minus n of H of X.
And this is an approximation in exactly the same sense that that's an approximation.
Finally, the last statement is that the probability that this typical -- the probability that you get a typical sequence is greater than or equal to 1 minus this variance divided by n times epsilon squared, with the same kind of approximation.
The probability that you get the typical sequence is about 1.
So what this is saying is there are hardly any exceptions in terms of probability.
There are a huge number of exceptions.
But they're all extraordinarily small in probability.
So most of the space is all tucked into this typical set.
Most of these, all of these typical sequences, have about the same probability.
And the number of typical sequences is about 2 to the n times H of X.
Does that say that the entropy has any significance?
It sure does.
Because the entropy is really what's determining everything about this distribution, as far as probabilities are concerned.
There's nothing left beyond that.
If you're going to look at very long sequences, and in source coding we want to look at long sequences, that's the story.
So the entropy tells a story.
Despite what Huffman said.
Huffman totally ignored the idea of entropy and because of that he came up with the optimal algorithm, but his optimal algorithm didn't tell him anything about what was going on.
This is not the [UNINTELLIGIBLE] Huffman, I think his algorithm is one of the neatest things I've ever seen, because he came up with it out of nowhere.
Just pure thought, which is nice.
We then start to talk about fixed lenght to fized length source coding.
This is not a practical thing.
This is not something I would advise trying to do.
It's something which is conceptually useful, because it's talks about anything you can do eventually, when you look at an almost infinite sequence of symbols from the source and you look at how many bits it takes you to represent them.
Ultimately you need to turn that encoder on, at some point in pre-history, and pre-history in our present day is about one year ago.
And you have to turn it off sometime in the distant future, which is maybe six months from now.
During that time you have to encode all these bits.
So, in fact, when you get it all done and look at the overall picture, it's fixed length to fixed length.
And all of these algorithms are just ways of doing the fixed length to fixed length without too much delay involved in them.
I didn't say everything there.
What this typical set picture gives us there, is you can achieve an expected number of bits per source symbol, which is about n times H of X, with very rare failures.
And if you try to achieve H of X minus epsilon bits per symbol, the interesting thing we found last time, was that the fraction of sequences you can encode was zilch.
In other words, there is a very rapid transition here.
If you try to get by with too few bits per symbol, you die very, very quickly.
It's not that your error probability is large, your error probability is asymptotically equal to 1.
You always screw up.
So that's the picture.
We want to go onto Markow sources.
Let me explain why I want to do this first.
When we're talking about the discrete memoryless sources, it should be obvious that that's totally a toy problem.
There aren't any sources I can imagine where you would want to encode their output where you can reasonably conclude that they were discrete and memoryless.
Namely, that each symbol was independent of each other symbol.
The only possibility I can think of is where you're trying to report the results of gambling or something.
And gambling is so dishonest that they probably aren't independent anyway.
You could use this is a way of showing they aren't independent, but it's not a very useful thing.
So somehow we want to be able to talk about how do you encode sources with memory.
Well Markow sources are the easiest kind of sources with memory.
And they have the nice property that you can include as much statistics in them as you want to.
You can make them include as much of the structure you can find as anything else will do.
So that people talk about much more general classes of sources, but these are really sufficient.
These are sufficient to talk about everything useful.
Not necessarily the nicest way to think about useful things but it's sufficient.
So a finite state in Markov chain, I assume you're somewhat familiar with Markov chaings from taking a probability courses.
If not, you should probably review it there, because the notes go through it pretty quickly.
There's nothing terribly complicated there, but a finite state in a Markov chain is a sequence of discrete chance variables, in that sense it's exactly like the discrete memoryless sources we were looking at.
The letters come from some finite alphabet, so in that sense it's like the discrete memoryless sources.
But here the difference is that each letter depends on the letter before.
Namely, before the Markov chain changes from one state to another state in one of these steps, it looks at the state it's in and decides where it's going to go next.
So we have a transition probability matrix, you can think of this as a matrix, it's something which has values for every S in the state space, and for everys S prime in the state space.
Which represents the probability that the state at time n is equal the this state S, and the state of time n minus 1 is equal to the states S prime.
So this tells you what's the probabilities are of going from one state to another.
The important thing here is that this single step transition incorporates all of the statistical knowledge.
In other words, this is also equal to the probability that S n is equal to this state S, given that S n minus 1 is equal to the state S prime, and also that S n minus 2 is equal to any given state of time n minus 2, and all the way back to S minus zero being any old state at time S zero.
So it says that this source loses memory, except for the first thing back.
I like to think of this as a blind frog who has very good sensory perception, jumping around on lily pads.
In other words, he can't see the lily pad, he can only sense the nearby lily pads.
So he jumps from one lily pad to the next and when he gets to the next lily pad, he then decides which lily pad he's going go to go to next.
That'll wake you up, anyway.
And we also want to define some initial pmf on the initial state.
So you like to think of Markov chains as starting at some time and then proceeding forever into the future.
That seems to indicate that all we've done is to replace one trivial problem with another trivial problem.
In other words, on step memory is not enough to deal with things like English text and things like that.
Or any other language that you like.
So the idea of a Markov source is that you create whatever set of states that you want, you can have a very large state space, but you associate the output of the source not with the states, but with the transitions from one state to another.
So this gives an example of it, it's easier to explain the idea by simply looking at an example.
In this particular example, which I think is the same as the one in the notes, what you're looking at is a memory of the two previous states.
So this is a binary source, it produces binary digits, just either zero or 1.
If the previous two digits were zero zero, it says that the next digit is going to be a 1 with probability 0.1 and the next is going to be a zero with probability 0.9.
If you're in states 0,0 and the next thing that comes out is a zero, then at that point, namely at that point one advanced into the future, you have a zero as your last digit, a zero as a previous digit, and a zero as the digit before that.
So your state has move from xn minus 2 and xn minus 1, to xn minus 1 and xn.
In other words, any time you make a transition, this digit here, which is the previous digit, has to always become that digit.
You'll notice the same thing in all of these.
When you go from here to there, this last digit becomes the first digit.
When you go from here to here, the last digit because the first digit, and so forth.
And that's a characteristic of this particular structure of having the memory represented by the last two digits.
So the kind of output you can get from this source -- I mean you see that it doesn't do anything very interesting.
When you have two zeroes in the past, it tends to produce a lot more zeroes, it tends to get stuck with lots of zeroes.
If you got a single 1 that goes over into this state, and from there it can either go there or there.
Once it gets here, it tends to produce a large number of 1's.
So what's happening here is you have a Markov chain which goes from long sequences of zeroes, to transition regions where there are a bunch of zeros and 1's, and finally gets trapped into either the all zero state again, or the all 1 state again.
And moves on from there.
So the letter is what's in this case the source output.
If you know the old state, the source output specifies in this state.
If you know the old state and the new state, that specifies the letter.
In other words, one of the curious things about this chain is I've arranged it so that, since we have a binary output, they're only two possible transitions from each state.
Which says that the state plus the sequence of source outputs specifies the state at every point in time, the stayed at every point in time specifies the source sequence.
In other words, the two are isomorphic to each other.
One specifies the other.
Since one specifies the other, you can pretty much forget about the sequence and look at the state chain, if you want to.
And therefore everything you know about Markov chains is useful here.
Or if you like to think about the real source, you can think about the real source as producing these letters.
So either way is fine because either one specifies the other.
When you don't have that property, and you look at a sequence of letters from this source, these are called partially specified Markov chains.
And they're awful things to deal with.
You can write theses about, but you don't get any insight about them.
There's hardly anything that you would like to be true which is true.
And these are just awful things to look at.
So we won't look them.
One of the nice things about engineering is you can create your own models.
Mathematicians have to look at the crazy models that engineers suggest to them.
They have no choice.
That's their job.
But as an engineer, we can only look at the nice models.
We can play and the mathematicians have to work.
So it's nicer to be an engineer, I think.
Although famous mathematicians only look at the engineering problems that appeal to them, so in fact, when they become famous the two groups come back together again.
Because the good engineers are also good mathematician, so they become sort of the same group.
These transitions, mainly the transition lines that we draw on a graph like this, always indicate positive probability.
In other words, that there's zero probability from going to here to here.
You don't clutter up the diagram by putting a line in, which allows you to just look at these transitions to figure out what's going on.
One of the things that you learned when you study finite state Markov chains, is that a state s is accessible from some other state s prime, if the graph has some path from s prime to s. In other words, it's not saying you can go there in one step.
It's saying that there's some way you can get there if you go long enough.
Which means there's some probability of getting there.
And the fact that there's a probability of getting there pretty much means that you're going to get there eventually.
That's not an obvious statement but let's see what that means here.
This state is accessible from this state, in the sense that you can get from here to there by going over here and then going to here.
If we look at the states which are accessible from each other, you get some set of states, and if you're in that set of states, you can never get out of it.
Therefore, every one of those states remains with positive probability and you keep rotating back and forth between them in some way, but you never get out of them.
In other words, a Markov chain which doesn't have this property would be the following Markov chain.
That's the simplest one I can think of.
If you start out in this state you stay there.
If you start out in this state, you stay there, That's not a very nice chain.
Is this a decent model for an engineering study?
No.
Because when you're looking at engineering, the thing that you're interested in, is you're looking at something that happens over a long period of time.
Back at time infinity you can decide whether you're here, or whether you're here, and you might as well not worry a whole lot about what happened back at time minus infinity, as far as building your model.
You may as well just build a model for this, or build a model for that.
There's another thing here, which is periodicity.
In some chains, you can go from this state to this state in one step, well one, two steps.
Or you can go there in one, two, three, four steps.
Or you can go there in one, two, three steps and so forth.
Which says, in terms of m the period of s is the greatest common denominator of path lengths from s back to s again.
If that period is equal to 1, mainly if there's not some periodic structure which says the only way you can get back to a state is by coming back every two steps or every three steps or something, then again it's not a very nice Markov chain.
Because if you're modeling it, you might as well just model things over two states instead of over single states.
So the upshot of that is you define these nice Markov chains, which are aperiodic, which don't have any of this periodic structure, and every state is accessible from every other state.
And you call them ergodic Markov chains.
And what ergodic means, sort of and as a more general principle, is that the probabilities of things are equal to the relative frequency of things.
Namely, if you look at a very long sequence of things out of a Markov chain, what you see in that very long sequence should be representative of the probabilities of that very long sequence.
The probabilities of transitions at various times should be the same from one time to another.
And that's whate ergodicity means.
It means that the thing is stationery.
You look at it at one time, it behaves the same way as at another time.
It doesn't have any periodic structure, which means if you look at it at even times, it behaves differently from looking at it at odd times.
That's the kind of Markov chain you would think you would have, unless you look at these odd ball examples of other things.
Everything we do is going to be based on the idea of ergodic Markov chains, because they're the nicest models to use.
A Markov source then is a sequence of labeled transitions on an and ergodic Markov chain.
Those are the only things we want to look at.
But that's general enough to do most of the things we want to do.
And once you have ergodic Markov chains, there are a lot of nice things that happen.
Mainly, if you try to solve this set of equations, namely if you want to say, well suppose there is some pmf function which gives me the relative frequency of a given state.
Namely, if I look at an enormous number of states, I would like the state little s to come up with some relative frequency.
All the time that I do it.
Namely, I wouldn't like to have one sample path which comes up with a relative frequency 1/2, and another sample path of almost infinite length which comes up with a different relative frequency.
Because it would mean that different sequences of states are not typical of the Markov chain.
That's another way of looking at what ergodicity means.
It means that infinite length sequences are not typical anymore.
It depends on when they start, when they stop.
It depends on whether you start at an even time or an odd time.
Depends on all of these things, that -- all of these things that real sources shouldn't depend on.
So, if you have relative frequencies then you should have those relative frequencies at time n and at time n minus 1.
And probability of a particular symbol s, if the probabilities of the previous symbol s prime were the same values, q of s prime.
We know the transition probabilities, that's q of s given s prime.
The sum over s prime, of q of s prime, given capital Q of s given s prime, what is that?
That's the probability of x.
In other words, if you start out at time n minus 1 with something pmf function on the states at time n minus 1, this is the formula you would use to calculate the probability the pmf function for states at time s. This is the probability mass function for the states at the next unit of time, time n. If this probability distribution is the same as this probability distribution, then you say that you're in steady state, because you do this again, you plug this into here, it's the same thing.
You get the same answer at time n plus 1.
You plug it in again you got the same answer at time n plus 2, and so on forever.
So you stay in steady state.
The question is, if you have a matrix here, can you solve this equation?
Is it easy to solve?
And what's the solution?
There's a nice theorem that says, if the chain is ergodic, namely if it has these nice properties of transition probabilities, that corresponds to a particular kind of matrix here.
If you have that kind of matrix, this is just a vector matrix equation.
That vector matrix equation has a unique solution then for this probability little q, in terms of this transition probability.
It also has the nice property that if you start out with any old distribution, and you grind this thing away for a number of times, this q of x is going to approach the steady state solution.
Which means if you start a Markov chain out in some known state, after awhile the probability that you're in state s is going to become this steady state probability.
It gets closer and closer to it exponentially as time goes on.
So that's just arithmetic.
These steady state probabilities are approached asymptotically from any starting state, i.e.
for all s and s prime and s. The limit of the probability that s sub n, the state at time n, is equal to a given state s, given that the state at time zero was equal to s prime.
This probability is equal to q of s and the limit as n goes to infinity.
All of you know those things from studying Markov chains.
I hope, because those are the main facts about Markov chains.
Incidentally I'm not interested in Markov chains here.
We're not going to do anything with them, the only thing I want to do with them is to show you that there are ways of modeling real sources, and coming as close to good models for real sources as you want to come.
That's the whole approach here.
How do you do coding for Markov sources?
The simplest approach, which doesn't work very well, is to use a separate prefix-free code for each prior state.
Namely, if I look at this Markov chain that I had, it says that when I'm in the state I want to somehow encode the next state that I go to, or the next letter that comes out of the Markov source.
The things that can come out of the Markov source are either a 1 or a zero.
Now you see the whole problem with this approach.
As soon as you look at an example, it sort of blows the cover on this.
What's the best prefix-free code to encode a 1 and a zero?
Where one appears with probability 0.9 and the other one appears with probability 0.1.
What's the Huffman encoder do?
It assigns one of those symols to 1 and one of them to zero.
You might as well encode 1 to this one, and zero to that one.
Which means that all of the theory, all it does is generate the same symbols that you had before.
You're not doing any compression at all.
It's a nice thing to think about.
In other words, by thinking about these things you then see the solution.
And our solution before to these problems was that if you don't get anywhere by using a prefix-free code on a single digit, take a block of n digits and use a prefix-free code there.
So that's the approach we will take here.
The general idea for single letters is this prefix-free code we're going to generate satisfies a Kraft inequality, you can use the Huffman algorithm, you get this property here.
And this entropy, which is now a function of the particular state that we were in to start with, is just this entropy here.
So this is a conditional entropy, which we get a different conditional entropy for each possible previous state.
And that conditional entropy for each possible previous state, is what tells us exactly what we can do as far as generating a Huffman code for that next state.
This would work fine if you had a symbol alphabet of size 10,000 or something.
It just doesn't work well when your symbol alphabet is binary.
If we start out in a steady state, then all of these probability stay in steady state.
When we look at the number of binary digits per source symbol and we average them over all of the initial states, the initial states occur with these probabilities q of s. We have these best Huffman code.
So the number of binary digits we're using per source symbol is really this average here.
Because this is averaging over all the states you're going to go into.
And the entropy of the source output, conditional on the chance variable s, is now, in fact, defined just as that average.
So they encoder transmits s zero, followed by the code word for s 1 using s zero.
That specifies s 1, and then you encode x 2, using s 1 and so forth.
And the decoder is sitting there and the decoder does exactly the same thing.
Namely, the decoder first sees what s zero is, then it uses the code for s zero to decide what x 1 was, then it uses the code for s 1, which is determined by s 1, s one is determined by x 1, and it goes on and on like that.
Let me review a little bit about conditional entropy.
I'm going pretty fast here and I'm not deriving these things because they're in the notes.
And it's almost as if I want you to get some kind of a pattern sensitivity to these things, without the idea that we're going to use them a lot.
You ought to have a general idea of waht the results are.
We're not going to spend a lot of time on this because, as I said before, the only thing I want you to recognize is that if you ever want to model a source, this, in fact, gives you a general way of doing it.
So this general entropy then is using what we had before.
It's the sum over all the states, and the sum over all of the source outputs of these log pmf probabilities.
This general entropy of both a symbol and a state is equal to this combined thing.
Which is equal, not surprisingly, to the entropy of the state.
Namely, first you want to know what the state is that has this entropy.
And then given the state, this is the entropy of the next letter, conditional on the state.
Since this joint entropy is less than or equal to H of s plus H of x, I think you just proved that in the homework, didn't you?
I hope so.
That says that the entropy of x conditional on s, is less than or equal to the entropy of x, which is not surprising.
It says that if you use the previous state in trying to do source encoding, you're going to do better than if you don't use it.
I mean the whole theory would be pretty stupid if you didn't.
That's what that says.
As I told you before, the only way we can make all of this work is to use n-to-variable-length codes for each state.
In other words, you encode n letters at the same time.
If you look at the entropy of the first n states given a starting state, turns out to be n times H of x given s. By the same kind of rule that you were using before.
The same argument that you used to show that this is equal to that, you can use to show that this is equal to H of S 1, given S zero, plus H of S 2, given S 1, plus H of S 3, given S 2 and so forth.
And by the stationarity that we have here, these are all equal, so you wind up with n times times this conditional entropy.
And since the source outputs specified the states, and the states specified the source outputs, you can then convince yourself that this entropy is also equal to n times H of X, given S. And once you do that, you're back in the same position we were in when we looked at n-to-variable-length coding when we were looking at discrete memoryless sources.
Namely, the only thing that happens when you're looking at n-to-variable length coding, is it that one fudge factor becomes a 1 over n. When you have a small symbol space by going two blocks, you get rid of this, you can make the expected length close to H of X, given S. Which means, in fact, that all of the memory is taking into account and it still is this one parameter, the entropy, that says everything.
The AEP holds -- I mean if you want to, you can sit down and just see that it holds, once you see what these entropies are, I mean the entropy you're using log pmf's, you're just looking at products of probabilities, which are sums of log pmf's, and everything is the same as before.
And again, if you're using n-to-variable-length codes, you just can't achieve an expected length less than H of X, given S. So H of X, given S gives the whole story.
You should read those notes, because I've gone through that very, very fast, partly because some of you are already very familiar with Markov chains some of you are probably less familiar with it, so you should check that out a little bit on your own.
I want to talk about the Lempel Ziv universal algorithm, which was rather surprising to many people.
Jacob Ziv is one of the great theorists of information theory.
Before this time, he wrote a lot of very, very powerful papers.
Which were quite hard to read in many cases.
So it was a real surprise to people when he came up with this beautiful idea, which was a lovely, simple algorithm.
Some people, because of that, thought it was Abe Lempel, who spends part of his year at Brandeis, who was really the genius behind it.
In fact, it wasn't Abe Lempel, it was Jacob Ziv who was the genius behind it.
Abe Lempel was pretty much the one who really implemented it, because once you see what the algorithm is, it's still not trivial to try to find how to implement it in a simple, easy way.
If you look at all the articles about it, the authors are Ziv and Lempel, instead of Lempel and Ziv, so why it got called Lempel Ziv is a mystery to everyone.
Anyway, they came up with two algorithms.
One which they came up with in 1977, and people looked at their 1977 algorithm and said, oh that's much too complicated to implement.
So they went back to the drawing board, came up with another one in 1978, which people said, ah, we can implement that.
So people started implementing the LZ78 and of course, by that time, all the technology was much better.
You could do things faster and cheaper then you could before, and what happened then is that a few years after that people were implementing LZ77, which turned out to work much better.
Which is often the way this field works.
People do something interesting theoretically, people say, no you can't do it, so they simplify it, thereby destroying some of its best characteristics.
And then a few years later people are doing the more sophisticated thing, which they should have started out doing at the beginning.
What is a Universal Data Compression algorithm?
A Universal Data Compression algorithm, is an algorithm which doesn't have any probabilities tucked into it.
In other words, the algorithm itself simply looks at a sequence of letters from an alphabet, and encodes it in some way.
And what you would like to be able to do is somehow measure what the statistics are, and at the same time as you're measuring the statistics, you want to encode the digits.
You don't care too much about delay.
In fact, one way to do this -- I mean, if you're surprised that you can build a good universal encoder, you shouldn't be.
Because you could just take the first million letters out of the source, go through all the statistical analysis you want to, model the source in whatever way makes the best sense to you, and then build a Huffman encoder, which, in fact, encodes things according to that model that you have.
Of course you then have to send the decoder the first million digits, and the decoder goes through the same statistical analysis, and therefore finds out what code you're going to use, and then the encoder encodes, the decoder decodes and you have this million symbols of overhead in the algorithm, that if you use the algorithm for a billion letters instead of a million letters, then it all works pretty well.
So there's a little bit of that flavor here, but the other part of it is, it's a neat algorithm.
And the algorithm measures things in a faster way then you would believe.
And as you look at it later you say, gee, this makes a great deal of sense even if there isn't much statistical structure here.
In other words, you can show that if the source really is a Markov source, then this algorithm will behave just as well, asymptotically, as the best algorithm you can design for that Markov source.
Namely, it's so good that it will in fact measure the statistics in that Markov model and implement them.
But it does something better than that.
And the thing which is better is that, if you're going to look at this first million symbols and your objective then is to build a Markov model, after you build the Markov model for that million symbols, one of the things that you always question about is, should I have used a Markov model or should I have used some other kind of model?
And that's a difficult question to ask.
You go through all of the different possibilities, and one of the nice things about the Lempel Ziv algorithm is, in a sense, it just does this automatically.
If there's some kind of statistical structure there, it's going to find it.
If it's not Markov, if it's some other kind of structure, it will find it.
The question is how does it find this statistical structure without knowing what kind of model you should use to start with?
And that's the genius of things which are universal, because they don't assume that you have to measure particular things in some model that you believe in.
It just does the whole thing all at once.
If it's running along and the statistics change, bing, it changes, too.
And suddenly it will start producing more binary digits per source symbol, or fewer, because that's what it has to do.
And that's just the way it works.
But it does have all these nice properties.
It has instantaneous decodability.
In a sense, it is a prefix-free code, although you have to interpret pretty carefully what you mean by that.
We'll understand that in a little bit.
But in fact, it does do all of these neat things.
And there are better algorithms out there now, whether they're better in terms of the trade-off between complexity and compressability, I don't know.
But anyway, the people who do research on these things have to have something to keep them busy.
And they have to have some kind of results to get money for, and therefore they claim that the new algorithms are better than the old algorithms.
And they probably are, but I'm not sure.
Anyway, this is a very cute algorithm.
So what you're trying to do here, the objective, one objective which is achieved, is if you observe the output from the given probability model, say a Markov source, and I build the best code I can for that Markov source, then we know how many bits we need per symbol.
Number of bits we need per symbol is this entropy of a letter of a symbol given the state.
That's the best we can do.
How well does the Lempel Ziv algorithm do?
Asymptotically, when you make everything large it will encode using a number of bits per symbol, which is H of X, given S. So it'll do just as well as the best thing does which happens to know the model to start with.
As I said before the algorithm also compresses in the absence of any ordinary kind of statistical structure.
Whatever kind of structure is there, this algorithm sorts it out.
It should deal with gradually changing statistics.
It does that also, but perhaps not in the best way.
We'll talk about that later.
Let's describe it a little bit.
If we let x 1, x 2 blah blah blah, be the output of the source, and the alphabet is some alphabet capital X, which has size m, let's just as notation, let x sub m super n denote the string xm, xm plus 1, up to xn.
In other words, in describing this algorithm we are, all the time talking about strings of letters taken out of this infinite length string that comes out of the source.
We want to have a nice notation for talking about a sub string of the actual sequence.
We're going to use a window in this algorithm.
We want the window to have a size with the power of 2.
Typical values for the window range from about a thousand up to about a million.
Maybe they're even bigger now, I don't know.
But as we'll see later, there's some constraints there.
What the Lempel Ziv algorithm does, this LZ77 algorithm, is it matches the longest string of yet unencoded -- this is unencoded also, isn't it, that's simple enough -- of yet unencoded symbols by using strings starting in the window.
So it takes this sequence of stuff we haven't observed yet, it tries to find the longest string starting there which it can match with something that's already in the window.
If it can find something which matches with something in the window, what does it do?
It's going to first say how long the match was, and then it's going to say where in the window it found it.
And the decoder is sitting there, the decoder has this window which it observes also, so the decoder can find the same match which is in the window.
Why does it work?
Well, it works because with all of these AEP properties that we're thinking of, you tend to have typical sequences sitting there in the window.
And you tend to have typical sequences which come out of the source.
So the thing we're trying to encode is some typical sequence -- well you can you can think of short typical sequences and longer typical sequences.
We try to find the longest typical sequence that we can.
And we're looking back into this window, and there are enormous number of typical sequences there.
If we make the typical sequences short enough, there aren't too many of them, and most of them are sitting there in the window.
This'll become clearer as we go.
Let's go on and actually explain what the algorithm does.
So here's the algorithm.
First, you take this large W, this large window size, and we're going to encode the first thought W symbols.
We're not even going to use any compression, that's just lost stuff.
So we encode this first million symbols, we grin and bear it, and then the decoder has this window of a million symbols.
We at the encoder have this window of a million symbols, and we proceed from there.
So it gets amortized, so we don't care.
So we then have a pointer, and we set the pointer to W. So the pointer is the last thing that we encoded.
So we have all this encoded stuff starting at time P, everything beyond there is as yet unencoded.
That's the first step in the algorithm.
So far, so good.
The next step is to find the largest n, greater than or equal to 2, I'll explain why greater than or equal to 2 later, such that the string x sub p plus 1 up to p plus n, what is that?
It's the string which starts right beyond the pointer, namely the string that starts here, what we're trying to do is find the largest n, in other words the longest string starting here, which we can match with something that's in the window.
Now we look at a, a is in the window, we look at a b, a b as in the window.
We look at a b a, a b a as in the window.
a b a b, a b a b is not in the window.
At least I hope it's not in the window or I screwed up.
Yeah, it's not in the window.
So the longest thing we can find which matches with what's in the window is this match of length three.
So this is finding the longest match which matches with something here.
This next example, I think the only way I can regard that is as a hack.
It's a kind of hack that programmers like.
It's very mysterious, but it's also the kind of hack that mathematicians like because in this case this particular hack makes the analysis much easier.
So this is another kind of match.
It's looking for the longest string here, starting at this pointer.
a b a b so forth, which matches things starting here.
Starting somewhere in the window.
So it finds a match a b here.
a b a here.
But now it looks for a b a b, a match of four.
Where do we find it?
We can start back here, which is still in the window, and what we see is a b a b.
So these four digits match these four digits.
Well you might say foul ball, because if I tell you there's a match of four and I tell you where it is, in fact, all the poor decoder knows is this.
If I tell you there's a match of four and it starts here, what's the poor decoder going to do?
The poor decoder says, ok, so a is that digit two digits ago, so that gives me the a.
So I know there's an a there.
b is the next digit, so b is there.
And then I know the first two digits beyond the window, and therefore this third digit is a, so that must be that digit.
The fourth digit is this digit, which must be that digit.
OK?
If you didn't catch that, you can just think about it, it'll become clear.
I mean it really is a hack.
It's not very important.
It won't change the way this thing bahaves.
But it does change the way you analyze it So that's the first thing you do, you look for these matches.
Next thing we're going to do is we're going to try to encode the matches.
Namely, we're going to try to encode the thing the we found in the window.
How do we encode what we found in the window?
Well the first thing we have to do -- yeah?
What if you don't find any matches?
I'm going to talk about that later.
If you don't find any matches, I mean what I was looking for was matches of two or more.
If you don't find any matches of two or more, what you do is you just take the first letter in the window and you encode that without any compression.
I mean our strategy here is to always send the length of the match first.
If you say the length of the match is one, than the decoder knows to look for uncompressed symbols, instead of looking for something in the window.
So it takes care of the case where there haven't been any occurrences of symbol anywhere in the window.
So you only look for matches of length two or more.
So then you use something called a unary-binary code.
Theoriticians always copy everybody else's work.
The unary-binary code was due to Peter Elias, who was the head of this department for a long time.
He just died about six months ago.
He was here up until his death.
He used to organize department colloquia.
He was so essential that since he died, nobody's taken over the department colloquia.
He was my thesis adviser, so I tend to think very kindly of him And he was lots of other things.
But anyway, he invented this unary-binary code, which is a way of encoding the integers, which has a lot of nice properties.
And they're universal properties, as you will see.
The idea is to encode the integers, there are infinite number of integers.
What you'd like to do, somehow or other, is have shorter code words for lower integers, and longer code words for longer integers.
In this particular Lempel Ziv algorithm, it's particularly important to have the lenght of the code words growing as the logarithm of n. Because then anytime you find a really long match, and you got a very large n, you're encoding a whole lot of letters, and therefore you don't care if there's an overhead which is proportional to log n. So you don't mind that.
And if there's a very small number of letters encoded, you want something very efficient then.
So it does that.
And the way it does it is, first you generate a prefix, and then you have a representation in base 2.
Namely base 2 expansion.
So the number n, the prefix here, I think I said it here, the positive integer n is encoded into the binary representation of n, preceded by a prefix of integer part of log to the base 2 of n zero.
Now what's the integer part of log to the base 2 of 1?
Log to the base 2 of 1 is zero.
It was a prefix of zero zeros.
So zero zeros is nothing.
So the prefix is nothing, the expansion of 1, in a base 2 expansion or any other expansion, is 1.
So the code word for 1 is 1.
If you have the number 2, log to the base 2 of 2 is 1.
The integer part of 1 is 1, so you start out with a single zero and then you have the expansion, 2 is expanded as 1 zero.
And so forth.
Oh and then 3 is expanded as 1 1, again with a prefix of 1.
Four is encoded as 1 zero zero, blah blah blah and so forth.
Why don't you just leave the prefix out?
Anybody figure out why I need the prefix there?
If without those prefixes, you don't a prefix-free code
Yeah Right.
If I left them out, everything would start with 1.
I would get to the 1, I would say, gee, is that the end of it or isn't it the end of it?
I wouldn't know.
But with this, if I see 1, the only code word that starts with 1 is this one.
If it's 2 or 3, it starts with zero and then there's a 1, which says it's on that next branch which has probably 1/4.
I have a 1 0, 1 1, a prefix of 0 0, followed by a 1, put me off on another branch.
And so forth.
So, yes, this is a prefix-free code.
And it's a prefix-free code which has this nice property that the number of digits in the code word is approximately 2 times log n. Namely, it goes up both ways here.
The number of zeros I need is log to the base 2 of n. The number of digits in the base 2 expansion, is also the integer part of log to the base 2 of n. So it works both ways.
And there's always this 1 in the middle.
Again, it's a hack.
It's a hack that works very nicely when you try to analyze this.
OK so if the size of the match is bigger than one, we're going to encode the positive integer u. u was where the match occurred.
How far back do you have to count before you find this match?
You're going to encode that integer u into a fixed length code of length log of w bits.
In other words, you have a window of size 2 to the twentieth.
You can encode any point in there with 20 binary digits.
The 20 binary digits say how far do you have to go back to find this code word.
So first we're encoding n, by this unary-binary code, then we're encoding w just with this simple minded way of encoding log w bits.
And that tells us where the match is.
The decoder goes back, there finds the match, pumps it out if n is equal to 1, here's the answer to your question, you encode the single letter without compression.
And that takes care of the case, either where you have a match to that single letter, or there isn't any match to the single letter.
The next thing, as you might imagine, is you set the pointer to P plus n, because you've encoded n digits, and you go to step two.
Namely, you keep iterating forever.
Until the source wears out, or until the encoder wears out, or until the decoder wears out.
You just keep going.
That's all the algorithm is.
Yeah?
Can you throw out the first n bits, when you reset the pointer, because you only have w bits that say where n was for the next iteration?
No, I throw out the n oldest bits in the window
Well, those are the first n bits
Yes the first n bits out of the window and I keep all of the more recent bits.
I tend to think of the first ones as the things closest to the pointer, but you think of it either way, which is fine.
So as you do it, the window keeps sliding along.
That's what it does.
Why do you think this works?
/ There's a half analysis in the notes.
i'd like to say a little bit about how that analysis is cheating, because it's not quite a fair analysis.
If you look at the window, there are w different starting points in the window.
So let's write this down.
w starting points.
So for any given n, there are there are w springs of length n. We don't know how long this match is going to be, but what I would like to do, if I'm thinking of a Markov source, is to say, OK, let's make n large enough so that the size of the typical set is about w. ok So choose n to be about w divided by H of X given S. And the size of the typical set is then going to be 2 to the n, wait a minute.
n is equal to log w. So the size of the typical set then is T sub epsilon, is going to be roughly from what we said 2 to the n times H of X given S. So what I'm going to do is to set w equal to T of epsilon.
I'm going to focus on a match length which I'm hoping to achieve, of log w over H of X given S. The typical set, then, is a size 2 to the n times H of x give S. And if the typical set is of this size, and I look at these w strings in the window, yeah, I'm going to have some duplicates but roughly I'm going to have a large enough number of things in the window to represent all of these typical strings.
Or most of them.
If I try to choose an n which is a little bigger than that, let's call this n star.
If I try to make n a little bit bigger than this typical match size, I don't have a prayer of a chance, because the typical set then is just very much larger than w, so I'd be very, very lucky if I found anything in the window.
So that can't work.
If I make n a good deal smaller, then I'm going to succeed with great probability it seems, because I'm even allowing for many, many duplicates of each of these typical sets to be in the window.
So what this is saying is there ought to be some critical length when the window was very large, critical match length, and most of the time the match is going to be somewhere around this value here.
And as w becomes truly humongous, and as the match size becomes large -- you remember for these typical sets to make any sense, this number has to be large.
And when this number gets large, the size of the typical set it humongous.
Which says, that for this asymptotic analysis, a window of 2 to the twentieth, probably isn't nearly big enough.
So the asymptotic analysis is really saying, when you have really humongous windows this is going to work.
You don't make windows that large, so you have to have some faith that this theoretical argument is going to work here.
But that tells you roughly what the size of these matches is going to be.
If the size of the matches is that, and you use log w bits plus 2 log n bits, to encode each match, what happens?
Encode match.
You use log w, plus 2 log n star.
That's the number of bits it takes you, this is the number of bits it takes you to encode what the match size is.
You still have to encode that.
This is the number of bits it takes you to encode where the match occurs.
Now how big is this relative to this?
Well n star is on the order of log w, so we're taking log w plus 2 times log of log of w. So in an approximate analysis, you say, I don't even care about that.
You wind up with encoding a match with log w bits.
So you encode n star bits, you use log w bits to do it, how many bits are you using per symbol?
H of X given S. That's roughly the idea of why the Lempel Ziv algorithm works.
Can anybody spot any problems with that analysis?
Yeah
You don't know the probabilities beforehand, so how do you pick w?
Good one.
You picked w by saying, I have a computer which will go at a certain speed.
My data rate is coming in at a certain speed, and I'm going to pick w as large as I can keep up with.
With the best algorithm I can think of for doing string matching.
And string matching a hard thing to do, but it's not a terribly easy thing to do either.
So you make w as large as you can.
And if it's not large enough, tough.
You got matches which are somewhat smaller -- all this argument about typical sets still work except for the epsilon and deltas that are tucked into there.
So it's just that the epsilons and the deltas get too big when you're strings are not long enough.
Yeah?
So your w is just make your processing time equal the time [UNINTELLIGIBLE]?
Yeah.
That's what the determines w. It's how fast you can do a string search over this long, long window.
You're not going to just search everything one by one, you're going to build some kind of data structure there that makes these searches run fast.
Can anybody think of why you might not want to make w too large?
This isn't a theoretical reason, this is more practical thing
Is it if the probabilities change, it's slow to react to those changes?
If the probabilities change, it's slow to react to them, because it's got this humongous window here, and it's not until the window fills up with all of this new stuff, that it starts to work well.
And before it fills up, you're using an effective small window, but you're using a number of bits which is proportionate to log of a large window, and therefore you're wasting bits.
So another thing that that determines how big you want w to be -- I mean the main thing that's determines it is just that you can't run that fast.
Because you'd like to make it pretty big.
Another question.
How about this matter of wasting w symbols at the beginning to fill up the window?
what do you do about that?
I mean, that's a pretty stupid thing, right?
Anybody suggest a solution to that?
If you're building this yourself, how would you handle it?
It's the same argument as if the statistics change in mid-stream.
I mean, you don't measure that the statistics have changed, and throw out what's in the window.
What?
Can you not assume that you already have a typical sequence?
Yes, and you don't care whether it's right or wrong.
You could assume that the typical sequence is all zeros, so you fill up the window with all zeros.
The decoder also fills it up with all zeros, because this is the way you always start.
And then you just start running a log, looking for matches and encoding things.
And as w builds up, you start matching things.
You could even be smarter, and know that you're window wasn't very big and let your window grow also.
If you wanted to really be fancy about this.
So if you want to encode this you can have a lot of fun, and a lot of people over the years have had a lot of fun trying to encode these things.
It's a neat thing to do.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
To take up the topic of quantization, you remember we're talking about source coding in the first part of the course, channel coding in the last 2/3 of the course.
Source coding, like all, is divided into three parts.
If you have waveforms, such as speech, you start out with the waveform.
The typical way to encode waveforms is you first either sample the waveform or you expand it in some kind of expansion.
When you do that, you wind up with a sequence of numbers.
You put the sequence of numbers into a quantizer, and the quantizer reduces that to a discrete alphabet.
You put the discrete symbols into the discrete encoder.
You pass it through a reliable binary channel.
What is a reliable binary channel?
It's a layered view of any old channel in the world, OK?
In other words, the way that discrete channels work these days is that, in almost all cases, what goes into them is a binary stream of signals, and what comes out of them is a binary stream of symbols, and the output is essentially the same z-input.
That's the whole purpose of how you design digital channels, and they work over analog media and all sorts of things.
OK, so discrete encoders and discrete decoders are really a valid topic to study in their own.
I mean, you have text and stuff like that, which is discrete to start with, so there's a general topic of how do you encode discrete things?
We've pretty much answered that problem, at least in an abstract sense, and the main point there is you find the entropy of that discrete sequence, the entropy per symbol, and then you find ways of encoding that discrete source in such a way that the number of bits per symbol is approximately equal to that entropy.
We know you can't do any better than that.
You can't do an encoding, which is uniquely decodable, which you can get out of with the original symbols again, with anything less than the entropy.
So at least we know roughly what the answer to that is.
We even know some classy schemes like the Lempel-Ziv algorithm, which will in fact operate without even knowing anything about what the probabilities are.
So we sort of understand this block here at this point.
And we could start with this next, or we could start with this next, and unlike the electrician here, we're going to move in sequence and look at this next and this third.
There's another reason for that.
When we get into this question, we will be talking about what do you do with waveforms?
How do you deal with waveforms?
It's what you've been studying probably since the sixth grade, and you're all familiar with it.
You do integration and things like that.
You know how to work with functions.
What we're going to do with functions in this course is a little bit different than what you're used to.
It's quite a bit different than what you learned in your Signals and Systems course, and you'll find out why as we move along.
But we have to understand this business of how to deal with waveforms, both in terms of this block, which is the final block we'll study in source coding, and also in terms of how to deal with channels because in real channels, generally what you transmit is a waveform.
What the noise does to you is to add a waveform or, in some sense, multiply what you put in by something else.
And all of that is waveform stuff.
And all of information theory and all of digital communication is based on thinking bits.
So somehow or other, we have to become very facile in going from waveforms to bits.
Now I've been around the professional communities of both communication and information theory for a long, long time.
There is one fundamental problem that gives everyone problems because information theory texts do not deal with it and communication texts do not deal with it.
And that problem is how do you go from one to the other, which seems like it ought to be an easy thing.
It's not as easy as it looks, and therefore, we're going to spent quite a bit of time on that.
In fact, I passed out lectures 6 and 7 today, which in previous years I've done in two separate lectures because quantization is a problem that looks important.
We'll see that it's not quite as important as it looks.
And I guess the other thing is, I mean, at different times you have to teach different things or you get stale on it.
And I've just finished writing some fairly nice notes, I think, on this question of how do you go from waveforms to numbers and how do you go from numbers to waveforms.
And I want to spend a little more time on that this year than I did last year.
I want to spend, therefore, a little less time on quantization, so that next time, we will briefly review what we've done today on quantization, but we will essentially just compress those two lectures all into one.
In dealing with waveforms, we're going to learn some interesting and kind of cool things.
Like for those of you who really don't -- are not interested in mathematics, you know that people study things like the theory of real variables and functional analysis and all of these neat things, which are very, in a sense, advanced mathematics.
They're all based on measure theory, and you're going to find out a little bit about measure theory here.
Not an awful lot, but just enough to know why one has to deal with those questions because the major results in dealing with waveforms and samples really can't be stated in any other form than in a somewhat measure-theoretic form.
So we're going to find just enough about that so we can understand what those issues are about.
So that's why next time we're going to start dealing with the waveform issues.
OK, so today, we're dealing with these quantizer issues and how do you take a sequence of numbers, turn them into a sequence of symbols at the other end?
How do you take a sequence of symbols, turn them back into numbers?
So when you convert real numbers to binary strings, you need a mapping from the set of real numbers to a discrete alphabet.
And we're typically going to have a mapping from the set of real numbers into a finite discrete alphabet.
Now what's the first obvious thing that you notice when you have a mapping that goes from an infinite set of things into a finite set of things?
[INAUDIBLE]
What?
[INAUDIBLE]
Not really.
It's a much simpler idea than that.
How are you going to get back?
I mean, usually when you map something into something else, you would like to get back again.
When you map an infinite set into a finite set, how are you going to get back?
You're not
You're not.
Good!
There's not any way in hell that you're ever going to get back, OK?
So, in other words, what you've done here is to deliberately introduce some distortion into the picture.
You've introduced distortion because you have no choice.
If you want to turn numbers into bits, you can't get back to the exact numbers again.
So you can only get back to some approximation of what the numbers are.
But anyway, this process of doing this is called scalar quantization, if we're mapping from the set of real numbers to a discrete alphabet.
If instead you want to convert real n-tuples into sequences of discrete symbols, in other words, into a finite alphabet, you call that vector quantization because you can view n real numbers, a sequence of n real numbers as a vector within coordinates and within components.
And I'm not doing anything fancy with vectors here.
You just look at an n-tuple of numbers as a vector.
OK, so scalar quantization is going to encode each term of the source sequence separately.
And vector quantization is first going to segment this sequence of numbers into blocks of n numbers each, and then it's going to find a way of encoding those n-blocks into discrete symbols.
Does this sound a little bit like what we've already done in dealing with discrete sources?
Yeah, it's exactly the same thing.
I mean, we started out by mapping individual symbols into bit sequences, and then we said, gee, we can also map n-blocks of those symbols into bits, and we said, gee, this is the same problem again.
There's nothing different, nothing new.
And it's the same thing here almost except here the properties of the real numbers are important.
Why are the properties of the real numbers important?
Why can't we just look at this as symbols?
Well, because, since we can't do this mapping in an invertable way, you have to deal with the fact that you have distortion here.
There's no other way to think about it.
There is distortion.
You might as well face it.
If you try to cover it up, it just comes up to kick you later.
So we face it right in the beginning, and that's why we deal with these things as numbers.
So let's look at a simple example of what a scalar quantizer is going to do.
Basically, what we have to do is to map the line R, mainly the set of real numbers, into M different regions, which we'll call R1 up to R sub M. And in this picture here, here's R1, here's R2, here's R3, here's R4, R5 and R6, and that's all the regions we have.
You'll notice one of the things that that does is it takes an enormous set of numbers, namely all these numbers less than this, and match them all into the same symbol.
So you might wind up with a fair amount of distortion there, no matter how you measure distortion.
All these outliers here all get mapped into a6, and everything in the middle gets mapped somehow into these intermediate values.
But every source value now in the region R sub j, we're going to map into a representation point a sub j.
So everything in R1 is going to be mapped into a1.
Everything in R2 is going to get mapped into a2 and so forth.
Is this a general way to map R into a set of M symbols?
Is there anything else I ought to be thinking about?
Well, here, these regions here have a very special property.
Namely, each region is an interval.
And we might say to ourselves, well, maybe we shouldn't map points into intervals.
But aside from the fact that we've chosen intervals here, this is a perfectly general way to represent a mapping from the real numbers into a discrete set of things.
Namely, when you're doing a mapping from the real numbers into a discrete set of things, there's some set of real numbers that get mapped into a1, and that by definition is called R1.
There's some set of numbers which get mapped into a2.
That by definition is called R2 and so forth.
So aside from the fact that these intervals with these regions in this picture happen to be intervals, this is a perfectly general mapping from R into a discrete alphabet.
So since I've decided I'm going to look at scalar quantizers first, this is a completely general view of what a scalar quantizer is.
You tell me how many quantization regions you want, namely how big the alphabet is that you're mapping things into, and then your only problem is how do you choose these regions and how do you choose the representation points?
OK, one new thing here: Before, we said when you have a set of symbols a1 up to a6, it doesn't matter what you call them.
They're just six symbols.
Here it makes a difference what you call them because here they are representing real numbers and they are representing real numbers because when you map some real number on the real line into one of these letters here, the distortion is u minus a sub j.
If you're mapping u into a sub j, then you get a distortion, which is this difference here.
I haven't said yet what I am interested in as far as distortion is concerned.
Am I interested in squared distortion, cubed distortion, absolute magnitude of distortion?
I haven't answered that question yet.
But there is a distortion here, and somehow that has to be important.
We have to come to grips with what we call a distortion and somehow how big that distortion is going to be.
So our problem here is somehow to trade off between distortion and number of points.
As we make the number of points bigger, we can presumably make the distortion smaller in some sense, although the distortion is always going to be very big from these really big negative numbers and from really big positive numbers.
But aside from that, we just have a problem of how do you choose the regions?
How do you choose the points?
OK, I've sort of forgotten about the problem that we started with.
And the problem that we started with was to have a source where the source was a sequence of numbers.
And when we're talking about sources, we're talking about something stochastic.
We need a probability measure on these real numbers that we're encoding.
If we knew what they were, there wouldn't be any need to encode them.
I mean, if we knew and the receiver knew, there would be no need to encode them.
The receiver would just print out what they were or store them or do whatever the receiver wants to do with them.
OK, so we're going to view the source value u with the sample value of some random variable capital U.
And more generally, since we have a sequence, we're going to consider a source sequence to be U1, U2, U3 and so forth, or you could consider it a bi-infinite sequence, starting at U minus infinity and working its way up forever.
And then we're going to have some sort of model, statistical model, for this sequence of random variables.
Our typical model for these is to assume that we have a memoryless source.
In other words, U1, U2, U3 are independent, identically distributed, random variables.
That's the model we'll use until we get smarter and start to think of something else.
OK, so now each of these source values we're going to map into some representation point a sub j.
That's what defines the quantizer.
And now since a sub j is a sample value of a random variable U, a sub j is going to be a sample value of some random variable V. OK, in other words, the probabilities of these different sample values is going to be determined by the set of U's that math into that a sub j.
So we have a source sequence U1, U2, blah, blah, blah.
We have a representation sequence V1, V2, blah, blah, blah, which is defined by if U sub k is in R sub j, then Vk is equal to a sub j.
Point of confusion here: It's not confusing now, but it will be confusing at some point to you.
When you're talking about sources, you really need two indices that you're talking about all the time.
OK, one of them is how to represent different elements in time.
Here we're using k as a way of keeping track of what element in time we're talking about.
We're also talking about a discrete alphabet, which has a certain number of elements in it, which is completely independent of time.
Namely, we've just described the quantizer as something which maps real numbers into sample values.
It has nothing to do with time at all.
We're going to use that same thing again and again and again, and we're using the subscript j to talk about that.
When you write out problem solutions, you are going to find that it's incredibly difficult sometimes to write sentences which distinguish about whether you're talking about one element out of an alphabet or one element out of a time sequence.
And everybody has that trouble, and you read most of the literature in information theory or communication theory, and you can't sort out most of the time what people are talking about because they're doing that.
I recommend to you using an element and an alphabet to talk about this sort of thing, what a sub j is or an element and a time sequence to keep track of things at different times.
It's a nice way of keeping them straight.
OK, so anyway, for a scalar quantizer, we're going to be able to just look at a single random variable U, which is a continuous-valued random variable, which takes values anywhere on the real line and maps it into a single element in this discrete alphabet, which is the set a1 up to a6 that we were talking about here.
So a scalar quantizer then is just a map of this form, OK?
So the only thing we need for a scalar quantizer, we can now forget about time and talk about how do you choose the regions, how do you choose the representation points?
OK, and there's a nice algorithm there.
Again, one of these things, which if you were the first person to think about it, easy way to become famous.
You might not stay famous, but you can get famous initially.
Anyway, we're almost always interested in the mean squared error or the mean squared distortion, MSD or MSE, which is the expected value of U minus V. U is this real-valued random variable.
V is the discrete random variable into which it maps.
We have the distortion between U and V, which is U minus V. We now have the expected value of that squared distortion.
Why is everybody interested in squared distortion instead of magnitude distortion or something else?
In many engineering problems, you should be more interested in magnitude distortion.
Sometimes you're much more interested in fourth-moment distortion or some other strange thing.
Why do we always use mean squared distortion?
And why do we use mean-squared everything throughout almost everything we do in communication?
I'll tell you the reason now.
We'll come back and talk about it more a number of times.
It's because this quantization problem that we're talking about is almost always a subproblem of this problem, where you're dealing with waveforms, where you take the waveform and you sample the waveform, where you take the waveform, and you turn the waveform into some expansion.
When you find the mean-squared distortion in a quantizer, it turns out that that maps in a beautiful way into the mean-squared distortion between waveforms.
If you deal with magnitudes or with anything else in the world, all of that beauty goes away, OK?
In other words, whenever you want to go from waveforms to numbers, the one thing which remains invariant, which remains nice all the way through, is this mean-square distortion, mean-square value, OK?
In other words, if you want to layer this problem into looking at this problem separately from looking at this problem, almost the only way you can do it that makes sense is to worry about mean square distortion rather than some other kind of distortion.
So even though as engineers we might be interested in something else, we almost always stick to that because that's the thing that we can deal with most nicely.
I mean, as engineers, we're like the drunk who dropped his wallet on a dark street, and he's searching for it, and somebody comes along.
And here he is underneath a beautiful light where he can see everything.
Somebody asks him what he's looking for you, and he says he's looking for his wallet.
The guy looks down and says, well, there's no wallet there.
The drunk says I know.
I dropped it over there, but it's dark over there.
So we use mean square distortion in exactly the same sense.
It isn't necessarily the problem we're interested in, but it's a problem where we can see things clearly.
So given that we're interested in the mean square distortion of a scalar quantizer, an interesting analytical problem that we can play with is for a given probability density on this real random variable, and we're assuming we have a probability density and a given alphabet size M, the problem is how do you choose these regions and how do you choose these representation points in such a way as to minimize the mean square error, OK?
So, in other words, we've taken a big sort of messy amorphous engineering problem, and we said, OK, we're going to deal with mean square error, and OK, we're going to deal with scalar quantizers, and OK, we're going to fix a number of quantization levels, so we've made all those choices to start with.
We still have this interesting problem of how do you choose the right regions?
How do you choose the right sample points?
And that turns out to be a simple problem.
I wouldn't talk about if it wasn't simple.
And we'll break it into subproblems, and the subproblems are really simple.
The first subproblem is if I tell you what representation points I want to use, namely, in this picture here, I say OK, I want to use these representation points.
And then I ask you, how are you going to choose the regions in an optimal way to minimize mean square error?
Well, you think about that for awhile, and you think about it in a number of ways.
When you think about it in just the right way, the answer becomes obvious.
And the answer is, let me not think about the regions here, but let me think about a particular value that comes out of the source.
Let me think, how should I construct a rule for how to take outputs from this source and map them into some number here.
So if I get some output from the source u, I say, OK, what's the distortion between u and a1?
It's u minus a1.
And the magnitude of that is the magnitude of u minus a1.
The square of it is the square of u minus a1, and then say, OK, let me compare that with u minus a2.
Let me compare it with u minus a3 and so forth.
Let me choose the smallest of those things.
What's the smallest stuff for any given u?
Suppose I have a u which happens to be right there, for example.
What's the closest representation point?
Well, a3 is obviously closer than a2.
And in fact for any point in here, which is closer to a3 than it is to a2, we're going to choose a3.
Now what's the set of points which are closer to a3 than they are to a2?
Well, you put a line here right between a2 and a3, OK?
When you put that line right between a2 and a3, everything on this side is closer to a3 and everything on this side is closer to a2.
So the answer is, in fact, simple, once you see the answer.
And the answer is, given these points, we simply construct bisectors between them, namely, bisectors with -- halfway between a1 and a2, we call that b1.
Halfway between a2 and a3, we call that b2, and those are the separators between the regions, OK?
In other words, what we wind up doing is we define the region R sub j, the set of things which get mapped into a sub j as the region which is bounded by bj minus 1 is the average of aj and aj minus 1, and bj is the average of aj and aj plus 1, where I've already ordered the points a sub j going from left to right, OK?
And that also tells us that the minimum mean square error regions have got to be intervals.
There is no reason at all to ever pick regions which are not intervals because, as soon as you start to solve this problem for any given set of representation points, you wind up with intervals.
So if you ever think of using things that are not intervals for this mean square error problem, then as soon as you look at this, you say, well, aha, that can't be the best thing to do.
I will make my regions intervals.
I will therefore simplify the whole problem, and I'll also improve it.
And when you can both simplify things and improve them at the same time, it usually is worth doing it unless you're dealing with standards bodies or something, and then all bets are off.
OK, so this one part of the problem is easy.
If we know what the representation points are, we can solve the problem.
OK, we have a second problem, which is just the opposite problem.
Suppose then that somebody gives you the interval regions and asks you, OK, if I give you these interval regions, how are you going to choose the representation points to minimize the mean square error?
And analytically that's harder, but conceptually, it's just about as easy.
And let's look at it and see if we can understand it.
Somebody gives us this region here, R sub 2, and says, OK, where should we put the point a sub 2?
Well, anybody have any clues as to where you want to put it?
[INAUDIBLE]
What?
At the midpoint?
At the midpoint.
It sounds like a reasonable thing, but it's not quite right
[INAUDIBLE]
What?
[INAUDIBLE]
It depends on the probability density, yes.
If you have a probability density, which is highly weighted over on -- let's see, was I talking about R2 or R3?
Well, it doesn't make any difference.
We'll talk about R2.
If I have a probability density, which looks like this, then I want a2 to be closer to the left-hand side than to the right-hand side.
I want it to be a little bit weighted towards here because that's more important in choosing the mean square error.
OK, if you didn't have any of this stuff, and I said how do I choose this to minimize the mean square error, what's your answer then?
If I just have one region and I want to minimize the mean square error, what do you do?
Anybody who doesn't know should go back and study elementary probability theory because this is almost day one of elementary probability theory when you start to study what random variables are all about.
And the next thing you start to look at is things like variance and second moment.
And what I'm asking you here is, how do you choose a2 in order to minimize the second moment of U, whatever it is in here, minus a2?
Yes?
[INAUDIBLE]
I want to take the expectation of U over the region R2.
In other words, to say it more technically, I want to take the expectation of the conditional random variable U, conditional on being in R2, OK?
And all of you could figure that out yourselves, if you sat down quietly and thought about it for five minutes.
If you got frightened about it and started looking it up in a book or something, it would take you about two hours to do it.
But if you just asked yourselves, how do I do it?
You'll come up with the right answer very, very soon, OK?
So subproblem 2 says let's look at the conditional density of U in this region R sub j. I'll call the conditional density -- well, the conditional density, given that you're in this region, is, in fact, the real density divided by the probability of being in that interval, OK?
So we'll call that the conditional density.
And I'll let U of j be the random variable, which has this density.
In other words, this isn't a random variable on the probability space that we started to deal with it.
It's sort of a phony random variable.
But, in fact, it's the intuitive thing that you think of, OK?
In other words, if this is exactly what you were thinking of, you probably wouldn't have called this a separate random variable, OK?
So this is a random variable with this density.
The expected value of U of j minus a of j quantity squared, as you all know, is sigma squared of U of j plus the expected value a U of j minus a sub j quantity squared, OK?
How do you minimize this?
Well, you're stuck with this.
This term, you minimize it by making a sub j equal to the expected value of U of j, which is exactly what you said.
Namely, you set a of j to be the conditional mean of U of j conditional on being in the center.
It's harder to say mathematically than it is to see it.
But, in fact, the intuitive idea is exactly right.
Namely, you condition your random variable on being in that interval, and then what you want to do is choose the mean within that interval, which is exactly the sort of thing we were thinking about here.
When I drew this curve here, if I scale it, this, in fact, is the density of u conditional on being in R sub j.
And now all I'm trying to do is choose the point here, which happens to be the mean which minimizes this second moment.
The second moment, in fact, is this mean square error.
So, bingo!
That's the second problem.
Well, how do you put the two problems together?
Well, you can put it together in two ways.
One of them is to say, OK, well then, clearly an optimal scalar quantizer has to satisfy both of these conditions.
Namely, the endpoints of the regions have to be the midpoints of the representation points, and the representation points have to be the conditional means of the points within a region that you start with.
And then you say, OK, how do I solve that problem?
Well, if you're a computer scientist, and sometimes it's good to be a computer scientist, you say, well, I don't know how to solve the problem, but generating an algorithm to solve the problem is almost trivial.
I start out with some arbitrary set of representation points.
That should be a capital M because that's the number of points I'm allowed to use.
Then the next step in the algorithm is I choose these separation points to be b sub j is the midpoint between the a's for each of these values.
Then as soon as I get these midpoints, I have a set of intervals, and my next step is to say set a sub j is equal to the expected value of this conditional random variable U of j where R sub j is now this new interval for 1 less than or equal to j less than or equal to M minus 1.
This means it's open on the left side.
This means it's closed on the right side, and since it's a probability density, it doesn't make any difference what it is.
I just wanted to give you an explicit rule for what to do with probability zero when you happen to wind up on that special point.
OK, and then you iterate on 2 and 3 until you get a negligible improvement.
And then you ask, of course, well, if I got a negligible improvement this time, maybe I'll get a bigger improvement next time.
And that, of course, is true.
And you then say, well, it's possible that after I can't get any improvement anymore, I still don't have the optimal solution.
And that, of course, is true also.
But at least you have an algorithm which makes sense and which, each time you try it, you do a little better than you did before.
Now, this mean square error for any choice of regions and any choice of representation points is going to be nonnegative because it's an expected value of squared terms.
The algorithm is nonincreasing with iterations.
In other words, the algorithm is going down all the time.
So you have zero down here.
You have an algorithm which is marching you down toward zero.
It's not going to get to zero, but it has to reach a minimum.
That's a major theorem in analysis, but you don't need any major theorems in analysis to see this.
You have a set of numbers, which are decreasing all the time, and they're bounded underneath.
After awhile, you have to get to some point, and you don't go any further.
So it has a limit.
So that's nice.
So you have an algorithm which has to converge.
It can't keep on going.
Well, it can keep on going forever, but it keeps going on forever with smaller and smaller improvements.
So eventually, you might as well stop because you're not getting anywhere.
OK, well, those conditions that we stated, the way a mathematician would say this, is these Lloyd-Max conditions are necessary but not sufficient, OK?
In other words, any solution to this problem that we're looking at has to have the property that the representation points are the conditional means of the intervals and the interval boundaries are the midpoints between the representation points.
But that isn't necessarily enough.
Here's a simple example of where it's not enough.
Suppose you have a probability density, which is running along almost zero, jumps up to a big value, almost zero, jumps up to a big value, jumps up to a big value.
This intentionally is wider than this and wider than this.
In other words, there's a lot more probability over here than there is here or there is here.
And you're unlucky, and you start out somehow with points like a1 and a2, and you start out with regions like R1 -- well, yeah, you want to start out with points, a1 and a2.
So you start out with a point a1, which is way over here, and a point a2, which is not too far over here.
And your algorithm then says pick the midpoint b1.
Here the algorithm is particularly simple because, since we're only using two regions, all you need is one separator point.
So we wind up with b1 here, which is halfway between a1 and a2.
a1 happens to be right in the middle of that big interval there, and there's hardly anything else here.
So a1 just stays there as the conditional mean, given that you're on this side.
And a2 stays as the conditional mean, given that you're over here, which means that a2 is a little closer this big region than it is to this region, but a2 is perfectly happy there.
And we go back and we iterate again.
And we haven't changed b1, we haven't changed a1, we haven't changed a2, and therefore, the algorithm sticks there.
Well, that's not surprising.
I mean, you know that there are many problems where you try to minimize things by differentiating or all the different tricks you have for minimizing things, and you very often find local minima.
People call algorithms like this hill-climbing algorithms.
They should call them valley-finding algorithms because we're sitting at someplace.
We try to find a better place.
So we wind up moving down into the valley further and further.
Of course, if it's a real geographical area, you finally wind up at the river.
You then move down the river, you wind up in the ocean and all that.
But let's forgot about that.
Let's just assume that there aren't rivers or anything.
That we just have some arbitrary geographical area.
We wind up in the bottom of a valley, and we say, OK, are we at the minimum or not?
Well, with hill climbing, you can take a pair of binoculars, and you look around to see if there's any higher peak someplace else.
With valley seeking, you can't do that.
We're sitting there at the bottom of the valley, and we have no idea whether there's a better valley somewhere else or not, OK?
So that's the trouble with hill-climbing algorithms or valley-seeking algorithms.
And that's exactly what this algorithm does.
This is called the Lloyd-Max algorithm because a guy by the name of Lloyd at Bell Labs discovered it, I think in '57.
A guy by the name of Joel Max discovered it again.
He was at MIT in 1960, and because all the information theorists around were at MIT at that time, they called it the Max algorithm for many years.
And then somebody discovered that Lloyd had done it three years earlier.
Lloyd never even published it.
So it became the Lloyd-Max algorithm.
And now there's somebody else who did it even earlier, I think, so we should probably take Max's name off it.
But anyway, sometime when I revise the notes, I will give the whole story of that.
But I hope you see that this algorithm is no big deal anyway.
It was just people fortunately looking at the question at the right time before too many other people had looked at it.
And Max unfortunately looked at it.
He was in a valley.
He didn't see all the other people had looked at it.
But he became famous.
He had his moment of time and then sunk gradually into oblivion except when once in awhile we call it the Lloyd-Max algorithm.
Most people call it the Lloyd algorithm, though, and I really should also.
OK, vector quantization: We talked about vector quantization a little bit.
It's the idea of segmenting the source outputs before you try to do the encoding.
So we ask is scalar quantization going to be the right approach?
To answer that question, we want to look at quantizing two sample values jointly and drawing pictures.
Incidentally, what's the simplest way of quantizing that you can think of?
And what do people who do simple things call quantizers?
Ever hear of an analog-to-digital converter?
That's what a quantizer is.
And an analog-to-digital converter, the way that everybody does it, is scalar.
So that says that either the people who implement things are very stupid, or there's something pretty good about scalar quantization.
But anyway, since this is a course trying to find better ways of doing things, we ought to investigate whether it is better to use vector quantizers and what they result in.
OK, well, the first thing that we can do is look at quantizing two samples.
In other words, when you want to generalize a problem to vectors, I find it better to generalize it to two vectors first and see what goes on there.
And one possible approach is to use a rectangular grid of quantization regions.
And as I'll show you in the next slide, that really is just a camouflaged scalar quantizer again.
So you have a two-dimensional region corresponding to two real samples.
So you've got two real numbers, U1 and U2.
You're trying to map them into a finite set of sample values of representation points.
Since you're going to be interested in the distortion between U1 and U2, between the vector U1, U2, and your representation vector, a1, a2, these points, these representation points, are going to be two-dimensional points also.
So if you start out by saying let's put these points on a rectangular grid, well, we can then look at it, and we say, well, given the points, how do we choose the regions?
You see, it's exactly the same problem that you solved before.
If I give you the points and then I ask you, well, if we get a vector U1, U2 that's there, what do we map it to?
Well, we map it to the closest thing, which means if we want to find these regions, we set up these perpendicular bisectors halfway between the representation points.
So all of this is looking very rectangular now because we started out with these points rectangular.
These lines are rectangular, and now I say, well, is this really any different from a scalar quantizer?
And, of course, it isn't because for this particular vector quantizer, I can first ask the question, OK, here's U1, which is something in this direction.
How do I find regions for that?
Well, for U1, I just establish these regions here.
And then I say, OK, let's look at U2 next.
And then I look at things in this direction, and I wind up in saying, OK, that's all there is to the problem.
I have a scalar quantizer again.
Everything that I said before works.
Now if you're a theoretician, you go one step further, and you say, what this tells me is that vector quantizers cannot be any worse than scalar quantizers.
Because, in fact, a vector quantizer has a -- or at least a vector quantizer in two dimensions -- has a scalar quantizer -- has two scalar quantizers as a special case.
And therefore, the amount of square distortion that I wind up with in the vector case, I can always get that -- whatever I can do with the scalar quantizer, I can do just as well with a vector quantizer by choosing it, in fact, to be rectangular like this.
And you can also get some intuitive ideas that if instead of having IID random variables, if U1 and U2 are very heavily correlated somehow so that they're very close together, I mean, you sort of get an engineering view of what you want to do.
You want to take this rectangular picture here.
You want to skew it around that way.
And you want to have lots of points going this way and not too many points going this way because almost everything is going this way, and there's not much going on in that direction.
So you got some pictures of what you want to do.
These regions here, a little bit of terminology, are called Voronoi regions.
Anytime you start out with a set of points and you put perpendicular bisectors in between those points, halfway between the points, you call the regions that you wind up with Voronoi regions.
So, in fact, part of this Lloyd-Max algorithm, generalized at two dimensions, says given any set of points, the regions ought to be the Voronoi regions for them.
So that's that first subproblem generalized at two dimensions.
And if you have an arbitrary set of points, the Voronoi regions look sort of like this, OK?
And I've only drawn it for the center point because, well, there aren't enough points to do anything more than that.
So anything in this region gets mapped into this point.
Anything in this semi-infinite region gets mapped into this point and so forth.
So even in two dimensions, this part of the algorithm is simple.
When you start out with the regions, with almost the same argument that we used before, you can see that the mean square error is going to be minimized by using conditional means for the representation points.
I mean, that's done in detail in the notes.
It's just algebra to do that.
It's sort of intuitive that the same thing ought to happen, and in fact, it does.
So you can still find a local minimum by this Lloyd-Max algorithm.
If you're unhappy with the fact that the Lloyd-Max algorithm doesn't always work in one dimension, be content with the fact that it's far worse in two dimensions, and it gets worse and worse as you go to higher numbers of dimensions.
So it is a local minimum, but not necessarily the best thing.
OK, well, about that time, and let's go forward for maybe 10 years from 1957 and '60 when people were inventing the Lloyd-Max algorithm and where they thought that quantization was a really neat academic problem, and many people were writing theses on it and having lots of fun with it.
And then eventually, they started to realize that when you try to solve that problem and find the minimum, it really is just a very ugly problem.
At least it looks like a very ugly problem with everything that anybody knows after having worked on it for a very long time.
So not many people work on this anymore.
So we stop and say, well, we really don't want to go too far on this because it's ugly.
But then we stop and think.
I mean, anytime you get stuck on a problem, you ought to stop and ask, well, am I really looking at the right problem?
Now why am I or why am I not looking at the right problem?
And remember where we started off.
We started off with this kind of layered solution, which said we were going to quantize these things into a finite alphabet and then we were going to discrete code them, OK?
And here what we've been doing for awhile, and none of you objected.
Of course, it's hard to object at 9:30 in the morning.
You just listen and you -- but you should have objected.
You should have said why in hell am I choosing the number of quantization levels to minimize over?
What should I be minimizing over?
I should be trying to find the minimum mean square error conditional on the entropy of this output alphabet.
Because the entropy of the output alphabet is what determines what I can accomplish by discrete coding.
That's a slightly phony problem because I'm insisting, now at least to start with, that the quantizer is a scalar quantizer, and by using coding here, I'm allowing memory in the coding process.
But it's not that phony because, in fact, this quantization job is a real mess, and this job is very, very easy.
And, in fact, when you think about it a little bit, you say, OK, how do people really do this if they're trying to implement things?
And how would you go about implementing something like this entire problem that we're talking about?
What would you do if you had to implement it?
Well, you're probably afraid to say it, but you would use digital signal processing, wouldn't you?
I mean, the first thing you would try to do is to get rid of all these analog values, and you would try to turn them into discrete values, so that you would really, after you somehow find this sequence of numbers here, you would go through a quantizer and quantize these numbers very, very finely.
You would think of them as real numbers, and then you would do some kind of discrete coding, and you would wind up with something.
And then you would say, ah, I have quantized these things very, very finely.
And we'll see that when you quantize it very, very finely, what you're going to wind up with, which is almost optimal, is a uniform scalar quantizer, which is just what a garden-variety analog-to-digital converter does for you, OK?
But then you say, aha!
What I can't do at that moment, I don't have enough bits to represent what this very fine quantizer has done.
So then I think of this quantization as real numbers.
I go through the same process again, and I think of then quantizing the real numbers to the number of bits I really want.
Anybody catch what I just said?
I'm saying you do this in two steps.
The first step is a very fine quantization, strictly for implementation purposes and for no other reason.
And at that point, you have bits to process.
You do digital-signal processing, but you think of those bits as representing numbers, OK?
In other words, as far as your thoughts are concerned, you're not dealing with the quantization errors that occurred here at all.
You're just thinking of real numbers.
And at that point, you try to design conceptually what it is you want to do in this process of quantizing and discrete coding.
And you then go back to looking at these things as numbers again, and you quantize them again, and you discrete encode them again in whatever way makes sense.
So you do one thing strictly for implementation purposes.
You do the other thing for conceptual purposes.
Then you put them together into something that works.
And that's the way engineers operate all the time, I think.
At least they did when I was more active doing real engineering.
OK, so that's that.
But anyway, it says finding a minimum mean square error quantizer for fixed M isn't the right problem that we're interested in.
If you're going to have quantization followed by discrete coding, the quantizer should minimize mean square error for fixed representation point entropy.
In other words, I'd want to find these representation points.
It's important what the numerical values of the representation points are for mean square error, but I'm not interested in how many of them I have.
What I'm interested in is the entropy of them.
OK, so the quantizer should minimize mean square error for a fixed representation point entropy.
I would like some algorithm, which goes through and changes my representation points, including perhaps changing the number of representation points as a way of reducing entropy.
OK, sometimes you can get more representation points by having them out where there's virtually no probability, and therefore they don't happen very often, but when they do happen, they sure save you an awful lot of mean square error.
So you can wind up with things using many, many more quantization points than you would think you would want because that's the optimal thing to do.
OK, when you're given the regions, if we try to say what happens to the Lloyd-Max algorithm then, the representation points should still be the conditional means.
Why?
Anybody figure out why we want to solve the problem that way?
If I've already figured out what the region should the -- well, before, when I told you what the regions were, you told me that you wanted to make the representation points the conditional means.
Now if I make the representation points something other than the conditional means, what's going to happen to the entropy?
The entropy stays the same.
The entropy has nothing to do with what you call these symbols.
I can move where they are, but they still have the same probability because they still occur whenever you wind up in that region.
And therefore, the entropy doesn't change, and therefore, the same rule holds.
But now the peculiar thing is you don't want to make the representation regions Voronoi regions anymore.
In other words, sometimes you want to make a region much closer to one point than to the other point.
And why do you want to do that?
Because you'd like to make these probabilities of the points as unequal as you can because you're trying to reduce entropy.
And you can reduce entropy by making things have different probabilities.
OK, so that's where we wind up with that.
I would like to say there's a nice algorithm to solve this problem.
There isn't.
It's an incredibly ugly problem.
You might think it makes sense to use a Lagrange multiplier approach and try to minimize some linear combination of entropy and mean square error.
I've never been able to make it work.
So anyway, let's go on.
When we want to look at a high-rate quantizer, which is what we very often want, you can do a very simple approximation.
And the simple approximation makes the problem much easier, and it gives you an added benefit.
There's something called differential entropy.
And differential entropy is the same as ordinary entropy, except instead of dealing with probabilities, it deals with probability densities.
And you look at this, and you say, well, that's virtually the same thing, and it looks like the same thing.
And to most physicists, most physicists look at something like this, and physicists who are into statistical mechanics say, oh, of course that's the same thing.
There's no fundamental difference between a differential entropy and a real entropy.
And you say, well, but they're all these scaling issues there.
And they say, blah, that's not important.
And they're right, because once you understand it, it's not important.
But let's look and see what the similarities and what the differences are.
And I'll think in terms of, in fact, a quantization problem, where I'm taking this continuous-valued random variable with density, and I'm quantizing it into a set of discrete points.
And I want to say what's the difference between this differential entropy here and this discrete entropy, which we sort of understand by now.
Well, things that are the same -- the first thing that's the same is the differential entropy is still the expected value of minus the logarithm of the probability density.
OK, we found that that was useful before when we were trying to understand the entropy.
It's the expected value of a log pmf.
Now the differential entropy is expected value of a log pdf instead of pmf.
And the entropy of two random variables, U1 and U2, if they're independent, is just the differential entropy of U1 plus the differential entropy of U2.
How do I see that?
You just write it down.
You write down these joint densities.
You write down this.
The logarithm of the joint density for IID splits into a product of densities.
A log of a product is the sum of the logs.
It's the same as the argument before.
In other words, it's not only that this is the same as the answer that we got before, but the argument is exactly the same.
And the other thing, the next thing is if you shift this density here.
If I have a density, which is going along here and I shift it over to here and have a density over there, the entropy is the same again.
Because this entropy is just fundamentally not -- it doesn't have to do with where you happen to be.
It has to do with what these probability densities are.
And you can stick that shift into here, and if you're very good at doing calculus, you can, in fact, see that when you put a shift in here, you still get the same entropy.
I couldn't do that at this point, but I'm sure that all of you can.
And I can do it if I spent enough time thinking it through.
It would just take me longer than most of you.
OK, so all of these things are still true.
There a couple of very disturbing differences, though.
And the first difference is that h of U is not scale invariant.
And, in fact, if it were scale invariant, it would be totally useless.
The fact that it's not scale invariant turns out to be very important when you try to understand it, OK?
In other words, if you stretch this probability density by some quantity a, then your probability density, which looks like this, shifts like this.
It shifts down and it shifts out.
So the log of the pmf gets bigger.
The pmf itself is just sort of spread out.
It does what you would expect it to do, and that works very nicely.
But the log of the pmf has this extra term which comes in here, which is a factor of a.
And it turns out you can just factor that factor of a out, and you wind up with the differential entropy of a scaled random variable U is equal to the differential entropy of the original U plus log of a, which might be minus log of a. I'm not sure which it is.
I'll write down plus or minus log of a.
It's one or the other.
I don't care at this point.
All I'm interested in is that you recognize that it's different.
An even more disturbing point is this differential entropy can be negative.
It can be negative or positive.
It can do whatever it wants to do.
And so we're left with a sort of a peculiar thing.
But we say, well, all right, that's the way it is.
The physicists had to deal with that for many, many years.
And the physicists, who think about it all the time, deal with it and say it's not very important.
So we'll say, OK, we'll go along with this unless asked: What happens if we try to build a uniform high-rate scalar quantizer, which is exactly what you would do for implementation purposes anyway.
So how does this work?
You pick a little tiny delta.
You have all these regions here.
And by uniform, I mean you make all the regions the same.
I mean, conceptually, this might mean having accountably infinite number of regions.
Let's not worry about that for the time being.
And you have points in here, which are the conditional means of the regions.
Well, if I assume that delta is small, then the probability density of u is almost constant within a region, if it really is a density.
And I can define an average value of f of u within each region in the following way.
It's easier to see what it is graphically if I have a density here, which runs along like this.
And within each region, I will choose f-bar of u to be a piecewise constant version of this density, which might be like that, OK?
Now that's just analytical stuff to make things come out right.
If you read the notes about how to do this, you find out that this quantity is important analytically to trace through what's going on.
I think for the level that we're at now, it's fine to just say, OK, this is the same as this, if we make the quantization very small.
I mean, at some point in this argument, you want to go through and say, OK, what do I mean by approximate?
Is the approximation close?
How does the approximation become better as I make delta smaller, and all of these questions.
But let's just now say, OK, this is the same as that, and I'd like to find out how well this uniform high-rate scalar quantizer does.
So my high-rate approximation is that this average density is the same as the true density and for all possible u.
So conditional on u and Rj, this quantity is constant.
It's constant and equal to 1 over delta, and the actual density is approximately equal to 1 over delta.
What's that mean?
It means that the mean square error in one of these quantization regions is our usual delta squared over 12, which is the mean square error of a uniform density in a region from minus delta over 2 to plus delta over 2.
So that's the mean square error.
You can't do anything with that.
You're stuck with it.
The next question is what is the entropy as a quantizer output?
I'm sorry, I'm giving a typical MIT lecture today, which people have characterized as a fire hose, and it's because I want to get done with this material today.
I think if you read the notes afterwards, you've got a pretty good picture of what's going on.
We are not going to have an enormous number of problems on this.
It's not something we're stressing so I want to get you to have some idea of what this is all about.
This slide is probably the most important slide because it tells you what this differential entropy really is.
OK, I'm going to look at the probabilities of each of these discrete points now.
p sub j is the probability that the quantizer will get the point a sub j.
In other words, it is the probability of the region R sub j.
And that's just the integral of the probability density over the j-th region.
Namely, it's the probability of being in that j-th region.
p sub j is also equal to this average value times delta.
OK, so I look at what the entropy is of the high-rate quantizer.
It's the sum of minus pj log pj.
I translate that pj is equal to this, so I have a sum over j. I have an integral minus a fU of u, which is that, times log And now I'll use this quantity instead of this quantity.
f-bar of u times delta du.
OK, in other words, these probabilities are scaled by delta, which is what I was telling you before.
That's the crucial thing here.
As you make delta smaller and smaller, the probabilities of the intervals go down with delta.
OK, so I break this up into two terms now.
I take out the log of delta, which is integrated over fU.
I have this quantity left.
This quantity is approximately equal to the density, and what I wind up with is that the actual entropy of the quantized version is equal to the differential entropy minus the log of delta at high rate.
OK, that's the only way I know to interpret differential entropy that makes any sense, OK, in other words, usually when you think of integrals, you think of them in a Riemann sense.
You think of breaking up the interval into lots of very thin slices with delta, and then you integrate things by adding up things, and that integral then is then a sum.
It's the same as this kind of sum.
And the integral that you wind up with when you're dealing with a log of the pmf, you get this extra delta sticking in there, OK?
So this entropy is this phony thing minus log of delta.
As I quantize more and more finely, this entropy keeps going up.
It goes up because when delta is very, very small, minus log delta is a large number.
So as delta gets smaller and smaller, this entropy heads off towards infinity.
And in fact, you would expect that because if you take a real number, which has a distribution which is -- well, anything just between minus 1 and plus 1, for example, and I want to represent it very, very well if it's uniformly distributed there, the better I try to represent it, the more bits it takes.
That's what this is saying.
This thing is changing as I try to represent things better and better.
This quantity here is just some funny thing that deals with the shape of the probability density and nothing else.
It essentially has a scale factor built into it because the probability density has a scale factor built into it.
A probability density is probability per unit length.
And therefore, this kind of entropy has to have that unit length coming in here somehow, and that's the way it comes in.
That's the way it is.
So to summarize all of that, and I'm sure you haven't quite understood all of it, but if you do efficient discrete coding at the end of the whole thing, the number of bits per sample, namely, per number coming into the quantizer that you need, is H of V, OK?
With this uniform quantizer, which produces an entropy of the symbols H of V, then you L-bar bits per symbol to represent that.
That's the result that we had before.
This quantity H of V depends only on delta and h of U. Namely, h of U is equal to H of V plus log delta.
So, in other words, analytically, the only thing you have to worry about is what is this differential entropy?
You can't interpret what it is, but you can calculate it.
And once you calculate it, this tells you what this entropy is for every choice of delta so long as delta is small.
It says that when you get to the point where delta is small and the probability density is essentially constant over a region, if I want to make delta half as big as it was before, then I'm going to wind up with twice as many quantization regions.
They're all going to be only half as probable as the ones before.
It's going to take me one extra bit to represent that, OK?
Namely, this one extra bit for each of these old regions is going to tell me whether I'm on the right side or the left side of that old quantization region to talk about the new quantization region.
So this all make sense, OK?
Namely, the only thing that's happening as you make the quantization finer and finer is that you have these extra bits coming in, which are sort of telling you what the fine structure is.
And little h of U already has built into it the overall shape of the thing.
OK, now I've added one thing more here, which I haven't talked about at all.
This uniform scalar quantizer in fact becomes optimal as delta become small.
And that's not obvious; it's not intuitive.
There's an argument in the text that shows why that is true.
It uses a Lagrange multiplier to do it.
I guess there's a certain elegance to it.
I mean, after years of thinking about it, I would say, yeah, it's pretty likely that it's true if I didn't have the mathematics to know it's true.
But, in fact, it's a very interesting result.
It says that what you do with classical a to z converters, if you're going to have very fine quantization, is, in fact, the right thing to do.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to start out today by reviewing what we covered last time.
We sort of covered two lectures of material, but a little bit lightly last time because I want to spend more time starting today dealing with wave forms and functions.
We will get notes on that very, very shortly.
To review quantization, we started out by talking about scalar quantizers, in other words, the thing that we want to do is to take a sequence of real numbers and each of those real numbers we want to quantizers it into one of a finite set of symbols.
Then, of course, the symbols we're going to encode later on.
So basically what we're doing is we're taking the real line, we're splitting it into a bunch of regions, r1, r2, r3.
The last region of the first region goes off to minus infinity.
The last region goes off to plus infinity.
So that clearly, if there's a lot of probability over in here or a lot of probably over in here, you're going to have a very large distortion.
So, when we talk more about that later and talk about how to avoid it and why we should avoid it and all of these things.
We then talked about these Lloyd-Max conditions for minimum mean square error.
What we said last time was suppose somebody gives you these representation points, which you're going to use to represent the actual number on the real line that comes out.
Then you ask when some particular number occurs should we encode it into this point or into this point?
If our criterion is mean square error, and that's what our criterion is normally going to be, then we're going to minimize the mean square error for this particular point by mapping it here, if that's the most probable -- we're going to map it here if this is the closest point, and we're going to map it here if that's the closest point.
Because by doing this we minimize the squared error between b and a1 or b and a2.
So, that says that we're going to define these regions to have the separations between the regions at the bisector points between the representation points.
So that says that one of the Lloyd-Max conditions for minimum mean square error is you always want to choose the regions in such a way that they're the mid-points between the representation points.
Any minimum mean square error quantizers has to satisfy this condition for each of the j's.
Namely, for each of these points they have to be mid-points.
Then the other thing that we observed is that once we choose these regions, the way we want to choose the representation points to minimize the mean square error is we now have to look at the probability density on this real line and we have to choose these points to be the conditional means within the representation area.
That just comes out by formula to be the expected value of the random variable u, which is this value as it occurs on the real line.
The expected value of that conditional on being in region rj is just the integral of u times the conditional density of u.
The conditional density of u is the real density of u divided by the probability of being in that region.
So all of this is very simple.
I hope you see it as something which is almost trivial, because if you don't see it as something simple, go back and look at it because, in fact, this is not rocket science here, this is just what you would normally do.
So Lloyd-Max algorithm then says alternate between the conditions for the mid-points between the regions and the conditional means.
The Lloyd-Max conditions are necessary but not sufficient.
In other words, any time you find a minimum mean square error quantization, it's going to satisfy those conditions.
But if you find a set of points, b sub j, and a set of points, a sub j, which satisfy those conditions, it doesn't necessarily mean that you have a minimum.
In other words, there are often multiple sets of points which satisfy the Lloyd-Max conditions, and one or more of those is going to be optimum, is going to be the smallest one, and the others are not going to be optimum.
In other words, the algorithm is a local hill-climbing algorithm, which finds the best thing it can find which is close to where it's starting at some very strange sense of close.
The close is not any sense of mean square error, but close is defined in terms of where the algorithm happens to go.
So an example of that that we talked about last time is where you have three spikes of probability.
Two of them are smaller and one of them is bigger.
One of them is at minus 1, one of them is at zero, one of them is at plus 1.
One solution to the Lloyd-Max conditions is this one here where a1 is sitting right in the middle of the spike.
Therefore, any time that the sample value of the random variable is over here, you get virtually no distortion.
The other point is sitting at the conditional mean between these two points.
So it's a little closer to this one than it is to this one -- I hope the figure shows that.
Any time you wind up here or here, you get this amount of the distortion.
Now without making any calculations you just look at this and you say well, this spike is bigger than this spike is.
Therefore, it makes sense if we're going to do this kind of strategy to put a 2 underneath that spike, therefore, getting a very small distortion any time the big spike occurs.
Then a1 is going to be midway between these two points, and you get the larger amount of distortion there but now it's a less probable event.
So you can easily check that both of these solutions satisfy the Lloyd-Max conditions, but one of them turns out to be optimal and the other one turns out to be not optimal.
If you fiddle around with it for a while, you can pretty much convince yourself that those are the only solutions to the Lloyd-Max algorithm for this particular problem.
Yeah?
When there's a region that has zero probability throughout, and the Lloyd-Max algorithm tries to find the mean for that region, it's going to find somewhere outside the region, it will find zero as the expected value.
But that might not necessarily be inside that region, what does it do in that case?
What does it do in that case?
Well, I don't think you can argue that it's going to be at zero.
I think you have to argue that it might be anywhere that it wants to be.
Therefore, what the algorithm is going to do when you start with a certain set of representation points -- well, if you start with a certain set of representation points that picks that separater wherever it happens to be, than this particular point you're talking about is going to be at some completely unimportant place.
You know eventually the thing that's going to happen is that this thing that's in a region of no probability is going to spread out and include something that has some probability, and then you're going to nail that region with some probability.
I can't prove this to you, and I'm not even sure that it's always true, but I think if you try a couple of examples you will see that it sort of does the right thing
But in the algorithm, you replace the point at the point of expected value in that region.
So, the algorithm doesn't know what to do at that point.
It crashes
Well, unless you're smart enough to write the program to do something sensible, yes
[UNINTELLIGIBLE PHRASE]
Yes.
And you have to write it so it'll do something reasonable then.
The best thing to do is to start out without having any of the regions have zero probability
We have that [UNINTELLIGIBLE PHRASE]
All right.
Well then you have to use some common sense on it.
So, after that we say OK, well just like when we were dealing with discrete source coding, any time we finish talking about encoding a single letter, we talk about what happens when you encode multiple letters in the same way.
Somebody is bound to think of the idea of encoding multiple real numbers all together.
So they're going to think of the idea of taking this sequence of real numbers, segmenting it into blocks of n numbers each and then taking the set of n numbers and trying to find a reasonable quantization for that.
In that case, the quantization points are going to be n vectors.
The regions are going to be regions in n dimensional space.
Well, if you think about it a little bit, these n dimensional representation points, if you're given them, the place where you're going to establish the regions then is on the perpendicular bisectors between any two points.
Namely, for each two points you're going to establish a perpendicular bisector between those two points, you're going to do that for all sets of points.
You're going to take regions which are enclosed by all of those perpendicular bisectors, and you call those the voronoi region.
Remember, I drew an example of it last time that looked something like this.
You have various points around.
It's hard to draw it in more than two dimensions.
So these perpendicular bisectors go like this and so forth.
I think you can show that you've never had the situation -- interesting problem if you want to play with it.
I don't think you can have that, but I'm not sure why.
Anyway, you do have these voronoi regions.
You have these perpendicular bisectors that you set up in two dimensional space or in high dimensional space.
Then given those regions you can then establish representation points, which are at the conditional means within those regions.
You really have the same problem that you had before, it's just a much grubbier problem because it's using vectors, it's an n dimensional space.
For this reason this problem was enormously popular for many, many years, because many people loved the complexity of it.
It was really neat to write computer programs that did this.
Back in those days you had to be careful about computer programs because computation was very, very slow, and it was a lot of fun.
When you get all done with it, you don't gain much by doing any of that.
The one thing that you do gain is that if you take square regions, namely, if you take a whole bunch of points which are laid out on a rectangular grid and you take regions which are now little rectangles or little squares, and you look at them for a while, you say that's not a very good thing to do.
A better thing to do is to take all this two dimensional space, for example, and to fill it in to tile it we say with hexagons as opposed to tiling it with rectangles or to tiling it with squares.
If you tile it with hexagons, for given amount of area you get a smaller mean square error.
If you could tile it with circles that would be the best of all, but when you try to tile it with circles you find out there's all this stuff left in the middle, like if you've ever tried to tile a floor with circles you find out you have to fill it in somehow and it's a little bit awkward.
So hexagons work, circles don't.
If you then go on to a higher number of dimensions, you get the same sort of thing happening, you get these nice n dimensional shapes which will tile n dimensional volume.
As n gets larger and larger, these tiling volumes become closer and closer to spheres, and you can prove all sorts of theorems about that.
But the trouble is when you get all done you haven't gained very much, except you have a much more complex problem to solve.
But you don't have a much smaller mean square distortion.
So you can still use Lloyd-Max.
Lloyd-Max still has as many problems as it had before in finding local minima.
With a little bit of thought about it you can see it's going to have a lot more problems.
Because visualize starting Lloyd-Max out where your points are on a square grid and where your regions now are a little square.
So in other words, like this.
Try to think of how the algorithm is going to go from that to the hexagons that you would rather have.
You can see pretty easily that it's very unlikely that the algorithm was ever going to find its way to hexagons, which by looking at it a little further away we can see it's clearly a good thing to do.
In other words, Lloyd-Max algorithm doesn't have any vision.
It can't see beyond its own nose.
It just takes these points and looks for regions determined by neighboring points, but it doesn't have the sense to look for what kind of structure you want.
So anyway, Lloyd-Max becomes worse and worse in those situations and the problem gets uglier and uglier.
Then, as we said last time, we stop and think and we said well gee, we weren't solving the right problem anyway.
As often happens when a problem gets very popular, people start out properly by saying well I don't know how to solve the real problem so I'll try to solve a toy problem.
Then somehow the toy problem gets a life of its own because people write many papers about it and students think since there are many papers about it, it must be an important problem.
Then since there are these open problems, students can solve those open problems and get PhD theses, and then they got a in a university, and the easiest thing for them to do is to get 10 students working on the same class of problems and you see what happens.
I'm not criticizing the students who do that or the faculty members who do it, they're all trapped in this kind of crazy system.
Anyway, the right problem that we should have started with is when we look at the problem of quantization followed by discrete source coding, we should have said that what we're interested in is not the number of quantization levels, but rather the entropy of the set of quantization levels.
That's the important thing because that's the thing that determines how many bits we're going to need to encode these symbols that come out of the quantizer.
So the problem we'd like to solve is to find the minimum mean square error quantizer for a given representation point entropy.
In other words, whatever set of points you have, you want to minimize the entropy of that set of points.
What that's going to do is to give you a larger set of points, but some points with a very small probability.
Therefore, those points with a very small probability are not going to happen very often.
Therefore, they don't affect the entropy very much, and therefore, you get a lot of gain in terms of mean square error by using these very improbable points.
That's a very nasty problem to solve.
And again, we said well let's try to solve a simpler version of it.
A simpler version of it is first to go back to the one dimensional case and then say OK, what happens if we just use a uniform quantizer, because that's what most people use in practice anyway.
If we use a uniform quantizer and we talk about a high rate uniform quantizer, in other words, we make the quantization points close together, what's going to happen in that case?
Well, the probability of each quantization region in that case is going to be close to the size of the representation interval, in other words, of the quantization interval, times the probability density within that interval.
Namely, if we have a probability density and that probability density is smooth, then if you take very, very small intervals you're going to have a probability density that doesn't change much within that interval.
Therefore, the probability of the interval is just going to be the size of that quantization interval -- in a uniform quantizer you'll make all of the intervals the same -- times the density within that integral.
Then we say OK, let's look at what the entropy is of that set of points, of the set of points where the probabilities are chosen to be some small delta times the probability density there.
I'm going through a slightly simpler kind of argument today than I did last time, and I'll explain why I'm doing something simpler today and why I did something more complicated then.
So this entropy is this quantity.
If we now substitute delta times the density for pj here, we get the sum over j of this delta pj, which is the probability density times the logarithm of delta pj.
Well now look, the logarithm of delta pj is just logarithm of delta plus logarithm of pj.
So we're taking the sum over all the probability space of logarithm of delta.
That comes out.
So we get a minus log delta, and what's left is minus the sum of delta pj log pj.
Does that look like something?
That looks exactly like the approximation to an integral that you always talk about.
Namely, if you look at a Reimann integral, the fundamental way to define a Reimann integral is in terms of splitting up that integral into lots of little increments, taking the value of the function in each one of those increments, multiplying it by the size of the increments and adding them all up.
In fact, we're going to do that a little later today when I try to explain to you what the difference is between Reimann integration and Lebesgue integration.
should Don't be frightened if you've never taken any mathematics courses, because if people had taught you Lebesgue integration when you were freshmen at MIT or seniors in high school or whenever you learned about integration, it would have been just as simple as teaching about Reimann integration.
One is no simpler and no more complicated than the other, so we're really going back to study something you should have learned about five years ago, maybe.
So anyway, when we represent this as an integral, we get this thing called the differential entropy, which is the integral of p of u minus p of u times log of p of u.
So the entropy of the discrete representation is minus log delta plus this differential entropy.
The mean square error in this uniform quantizer, the conditional means according to this approximation are right in the middle of the intervals.
So we have a uniform probability interval of width delta, a point right in the middle of it, and even I can integrate that to find the mean square error in it, which is delta squared over 12, which I think you've done at least once in the homework by now.
So I said I was going to tell you why I went through doing it this simpler way this time and put in a lot more notation last time.
If you really try to trace through what the approximations are here, the way we did it last time is much, much better, because then you can trace through what's happening in those approximations, and you can see, as delta goes to zero, what's happened.
Yes?
This may be an obvious question, why did you substitute delta with pj [INAUDIBLE PHRASE]?
Oh, why did I--?
OK, this is the probability of the representation of the j's representation point.
This is the probability density around that representation point.
The assumption I'm making here is that f of u was constant over that interval.
And if the density is constant over the interval, if I have a density which is constant over an interval of width delta, than the probability of landing in that interval is the width times the height
I think there's a typo in your [UNINTELLIGIBLE PHRASE]
A typo?
[UNINTELLIGIBLE PHRASE]
Yes, yes, yes.
I'm sorry, yes.
I'm blind today.
I knew what I meant so well that I didn't -- thank you.
s of uj.
s of uj.
Yes.
Then I take out the delta and what I'm left with is the delta f of uj times the log of f of uj.
Thank you.
When I look at that it's delta times the probability density times the log of the probability density.
If I convert that now into an interval when delta is very small, I get this thing called the differential entropy.
Does that make a little more sense?
So your question was obvious, it was just that I was a total dummy.
So let's summarize what all of that says.
In the scalar case we're saying -- I have said but I have not shown -- that a uniform scalar quantizer approaches an optimal scaler quantizer.
I haven't explained it all in class why that's true.
There's an argument in the notes that points it out.
You can read that there.
It's just another optimization, but it's true if you're looking at a higher and higher rate, a scaler quantizer where delta gets smaller and smaller, then in general what you need is to take a different size delta for each quantization region and then look at what happens when you try to optimize over that and you find out that you want to make all of the deltas the same.
The required number of encoded bits per symbol depends only on h of u and on delta.
This is the most important part of all of this.
It says that as you change this differential entropy, if you try to draw a curve between H of v and MSE, and there's a curve like that drawn in the notes, if you change the differential entropy, it just shift this curve left and right.
For a given value of h of u, this is a universal curve.
In other words, as you change delta, this quantity changes and this quantity changes.
That's the only variable which is left in here at this point.
When you make delta half as big, if you want to get a higher rate quantizer, what happens?
Your mean square error goes down by a factor of four.
Delta squared goes down to 1/4 of its previous value.
What happens here, at log of delta?
Delta has changed by a factor of 1/2.
H of v goes up by one bit.
So you take one more bit in your quantizer and you get a mean square error, which is four times as small.
Any time you think of what kind of accuracy you need on a computer or something, I think this is obvious to all of you, if you put it in terms of something you're already familiar with.
If you use 16 bit quantization with fixed bit numbers and then you change it to 24 bit accuracy, what's going to happen?
Well, everything is going to get better by a factor of 256, and since we're talking about mean square error, it's going to be four times that.
So that's just saying the same thing that you know.
For vector quantization, uniform quantization again approaches optimal for a memoryless source.
If you have a source with memory, vector quantization gains a great deal for you.
But if you don't have any memory, vector quantization doesn't gain much at all.
The only thing that vector quantization gains you is this thing we call a shaping gain now.
We talk about that again when we start talking about modulation.
If you change from a square set of points to a hexagonal set of points and you keep the areas the same, the mean square error goes down by a smidgen -- something like 1.04 or something.
It's not a big deal but there's some gain, so the gain is not impressive.
The big gains come when you look at the memory and when you take that into account.
So now we want to get on to the last part of our trilogy when we're talking about source coding.
Remember, when we were talking about source coding, we broke it up into three pieces.
The first piece we called it sampling, which took a wave form, turned it into a sequence of numbers.
That's what happens here.
We then quantize the sequence of numbers, either one number at a time or with a vector quantizer n numbers at a time.
We just finished talking about that.
The first five lectures in the course were all talking about discrete encoding, and whenever you're going from wave forms to bits, you gotta go through all three of these.
Now, sampling is only one way to go from wave form to sequence, and filtering is only one way to get back.
We're going to talk about sampling.
We're probably going to teach you more about sampling than you ever wanted to know.
But it turns out that it's worth knowing.
After you understand it you never forget it.
There's a lot of stuff to go through to start with, but finally, I hope, it all makes sense.
But anyway, the thing we're going to be talking about today is really the question of how do you go from wave forms to sequences, it's that simple.
How do you in general take wave forms, turn them into sequences?
How do you go back from sequences to wave forms?
We're going to spend quite a bit of time on this.
We're going to spend three lectures talking about it, and probably with today thrown in it'll be closer to three and a half lectures.
It's not only because we want to talk about the problem with source coding, because as soon as we start talking about channels, we're going to have the same truck problem looked at in the opposite direction.
We're going to start out with binary data.
We're then going to go through a modulator, we're going to find symbols.
From the symbols, from the numerical symbols, we're talking about a sequence of things and we have to go from the sequence to wave forms.
So, both of those problems are really the same.
We're talking about it first in terms of source coding, but whatever we learn about wave forms to sequences will be general and will be usable for both.
So I want to review why it is that we want to spend so much time on this analog source to bit stream problem.
I just told you one of the reasons which is not here, which is that it's a good way to get into the question of what do we do with channels.
But the other reasons, and we've talked about them all, and they're all important and you ought to remember them, because often we get so used to doing things in a certain way that we don't know why we're doing them and then somebody suggests something else and we say oh, that's a terrible idea because we've always done it this way.
One of the reasons why we want to go to bits is that a standard binary interface separates the problem of source and channel coding.
This was, in a sense, one of Shannon's great discoveries, and he also showed that you could do it without really any loss.
Another reason is you want to multiplex data on high speed channels.
This is perfectly familiar to you.
I think to everyone today we think of sending data over the web and we're all used to using the web all together.
I send my stuff, you send your stuff, I get my stuff off, you get your stuff off, and this stuff was all going over common channels.
It's going over optical fibers into MIT, and then it splits up at MIT and goes into many places and then it goes many places again.
But this idea of multiplexing data is perfectly straightforward.
If we didn't do any of this, if all of my stuff was really wave forms and all of your stuff was images and if all of somebody else's stuff was data and every piece of the internet had to worry about all those different things, when you start worrying about all those different things you create an awful lot of other things also.
We just wouldn't have any internet today.
So this multiplexing is a big deal, too.
You can clean up digital data at each link in a network.
In other words, if I'm sending analog data from here to San Francisco and I'm sending it over multiple different links, on every link a little bit of noise gets added to it.
That noise keeps adding up because there's no way to clean it up, because nobody knows what I sent.
If I'm sending digital data, at the receiver on each link, nobody knows what I sent, no, but they know that what I sent was one out of a finite collection of things.
There's something called repeating going on there at every channel, which takes what is received as an analog signal and, in fact, knowing what the encoding process was, goes back to cleaning it up to a digital signal again.
If you believe all of that and if you think it's simple, it's not.
We're going to talk about it later.
At this point, it's only plausible, and we're going to justify it as we move on.
We can separate problems of wave form sampling from quantization from discrete source coding.
In other words, we not only have the layering between sources and channels, but we also have this layering for sources, which goes between wave form to sequence separation, then sequence and quantization into a finite set of symbols.
Then a finite set of symbols getting coded.
So, three separate things we've learned about, we can separate them all very nicey.
So we said that in this wave form, the sequence business, sampling is only one way to go.
I'm going to show that to you right away at the beginning by talking about Fourier series.
How many of you have studied Fourier series and say spending more than a couple of hours of your life thinking about Fourier series?
OK, good, quite a few of you, that's nice.
Because we have to assume that you know a little bit about this, but probably not a whole lot.
There's a formula for a Fourier series, which is probably not the formula for a Fourier series that you're used to.
It says the Fourier series of a time-limited function matched the function to a sequence of coefficients.
Here's the formula.
Here's the function.
You can represent the function as a sum of coefficients times these complex exponentials.
You do that over this interval minus capital T over 2 less than or equal to t, less than or equal to capital T over 2.
The complex coefficients satisfy this equation here.
That's just what you've seen before.
The way this is different from what you've probably seen before is that most people think that you use Fourier series for periodic functions.
In other words, if we leave out this part here, leave out this, then this quantity here is a periodic function, because each of the what have you are all squiggling around with a period which is a sub-multiple of capital T. So that, in fact, this is a periodic function with period t. If I think of it as a periodic function I don't have to worry about this, this still works for any periodic function.
The problem is this isn't the way the Fourier series is usually used.
Occasionally, you want to talk about periodic functions, but most often what you want to do is you want to take a function which exists only over some finite interval and you want some way of mapping that function into a set of coefficients.
I take a function only over the interval minus t over 2 to capital T over 2, and I can map that into a sequence of coefficients, I have, in fact, done what I said we're interested in doing right now, which is turning a wave form into a sequence.
The only problem with it is it's a fine duration wave form, which I'm turning into a sequence.
Now how do you do speech coding?
There's an almost universal way of doing speech coding now of turning speech, analog wave forms, into actual data, into binary data.
The way that it always starts, I mean everybody has their own way of doing it, but almost everyone takes the speech wave form and segments it into 20 millisecond intervals.
Each 20 millisecond interval is then encoded into a sequence of coefficients.
You can think of that as taking each 20 millisecond interval, creating a Fourier series for it, and the Fourier series coefficients then represent the function in that interval.
You go on to the next interval, you get another sequence of Fourier coefficients and so forth.
Now, most of these very sophisticated voice coders don't really use the Fourier series coefficients because there's a great deal of structure in voice, and the Fourier series is designed to deal with any old thing at all.
But the Fourier series is a good first order approximation to what's going on when you're dealing with voice coding.
when you're dealing with voice coding you are certainly looking at frequencies, you're looking at formats, which are ranges of frequencies.
If you want to think about those problems, you better start to think in these ways here.
So anyway, this is not just mathematics, this is one of the things that we need to understand how you do actual wave forms to sequences.
We're not going to talk too much about where these formulas come from too much.
It is interesting that this also works for complex functions as well as real functions.
There's a nice sort of symmetry there, because the coefficients are all going to be complex anyway, because of these things here which are complex.
Incidentally, we always use i in this course for the square root of minus 1.
Electrical engineers have traditionally used the letter j for the square root of minus 1 for the rather poor reason that they like to refer to current as i.
In the first two years of an earlier electrical engineering education back 50 years or so ago, you spent so much time talking about voltages and currents that using the letter i for anything other than current was just an abomination.
Well, everybody else in the world uses i for the square root of minus 1.
So in this course we're going to do the same thing.
I would urge you to get used to it because then you can talk to people other than electrical engineers, and you'll probably have to spend a lot of time in your life talking to other people.
You shouldn't expect them to get used to your conventions, you should try to do a little to get used to their conventions.
So there's that peculiarity.
We're also using this complex notation throughout.
You could do this in terms of sines and cosines, which is probably the way you first learned it.
I'm sure for any of you who spent more than a very, very small amount of time dealing with Fourier series, you did enough with it to realize that just computationally going from sines and cosines to complex exponentials just makes life so much easier and makes your formula so much shorter that you want to do it.
So anyway, we want to make this work for complex signals as well as anything else.
I do want to verify the formula for these Fourier coefficients.
Incidentally, the other thing that I'll be doing which is a little bit weird here is that most people when they talk about the Fourier integral and the Fourier series they use capital letters to talk about frequencies and they use little letters to talk about signals.
For us, we really want to use capital letters to talk about random variables, and we do that pretty consistently.
Believe me, when we start talking about random processes, you will get so confused going back and forth between sample values and random of variables, that having a notation way to keep them straight will be very valuable to you.
When you start reading the literature you get even more confused because most people in the literature don't tell you what it is that they're talking about and they go back and forth between sample values and random variables, oftentimes using the same symbol in the same sentence for two different things.
So I think that's a more important thing to keep straight, so we'll always use tildes to talk about frequency type things.
You can see that these coefficients here are, in fact, frequency-like things because they're talking about how much of this wave form is at a certain discrete frequency, and we'll come back to talk about that later.
But anyway, if you want to verify the formula for this, what we're going to do is to start out by looking at -- this is where having smaller data would be a big help.
u of t is equal to this sum here.
So I'm going to replace u of t in this formula by this sum.
I'm going to make the index m because I already have a k over here.
When we have a k over here and you're talking about this, you don't want to get your indexes mixed.
So if I'm trying to see what this looks like, I want to represent as the integral from minus t over 2 to plus t over 2 of this representated as a sum with e to the minus 2 pi i kt over t taken into account over here.
So what happens here?
Here we have an integral of a sum.
Later on we're going to be a little bit careful about interchanging integrals and sums, but for now let's not worry about that at all.
I suggest to all of you, never worry about interchanging integrals and sums until after you understand what's going on.
Because if you start asking about that -- I mean that's a detail.
You look at what's going on in a major way first, and then you go back to check that sort of thing out.
So when we look at this integral here, when we take the sum outside, we have the sum over m of an integral over one cycle of these quantities here, of this times e to the 2 pi i times m minus kt over t. Now, you look at this integral here of a complex exponential as it's rotating around.
In the period of time t, this always rotates around some integer number of times.
If m is equal to k, it doesn't rotate at all, it just sticks where it is, at 1.
If m is unequal to k, it goes around some integer number of times.
If I'm thinking of this as being real and this as being imaginary, I'm just running around this circle here.
So what happens when I run around the circle once?
The integral is zero because I'm up here as much as I'm down here, I'm over here as much as I'm over here.
So this integral is always zero, which says that all of these terms except when m is equal to k disappear.
So that means I wind up with just this one term u hat of k times the integral from minus t over 2 to t over 2dt.
That's another integral I can evaluate, and it's equal to capital T times u sub k. So, u sub k is this quantity here divided by t, which is what we said over here.
In fact, that argument, you can make it precise and it works.
So what this is saying is that, in fact, if you look at these Fourier series formulas, this thing is pretty simple in terms of this.
The question which is more difficult is what functions can you represent in this way and what functions can't you represent in this way.
The easy answer is if you can think of it you can represent it in this way.
But if you stop and think about it for six months, then that might not be true anymore.
So if you become very good at this, you can find examples where it doesn't work, and we'll talk about that as we go on.
Let's define this rectangular function because we're going to be using it all the time.
You probably used it when dealing with the Fourier integral all the time because you all know that a rectangular function is a Fourier transform of a sync function where a sync function is a sine x over x type function.
If you don't remember that, fine.
But anyway, this function is 1 in the interval minus 1/2 to plus 1/2 and it's 0 everywhere else, which is why it's called a rectangular function.
It looks like this.
We do it from minus 1/2 to plus 1/2 so it has area 1.
In terms of that, we can express the formula for a time-limited function as this sum here, uk times these complex exponentials times this rectangular function.
How many of you can see it ought to be rectangle of t over capital T instead of rectangle of t times t?
Good.
I can't.
I always have to take two minutes doing that every time I do it, and if you can see it in your mind you're extremely fortunate.
When we work with these things for a while, you will become more adept at doing things like that.
But anyway, this works.
I want to look at an example now.
And there's several reasons I want to look at this.
One is to just look at what a Fourier series does.
Suppose we expand the function, the rectangular function of t over 2.
Now the rectangular function of t over 2 is going to be 1 from minus 1/4 to plus 1/4, instead of minus 1/2 to plus 1/2, because of the 2 in here.
We want to expand it in a Fourier series over the interval minus 1/2 to plus 1/2.
One of the things this is telling you is that when you're expanding something in a Fourier series, you have to be quite explicit about what the interval is that you're expanding it over.
Because I could also find a Fourier series here using the interval minus 1/4 to plus 1/4, which would be a whole lot easier.
But we're gluttons for punishment.
So we're expanding in a Fourier series over the bigger interval from here to there.
We go through these formulas calculating u sub k. We can easily do it for the first one, which is just u sub zero.
It's just the average value in this interval minus 1/2 to 1/2, which is 1/2.
The next term turns out to be 2 over pi times the cosine of 2 pi t. We can evaluate all of them just going through more and more junk like that.
But look at what's happened.
We started out with a rectangular function.
When we evaluate more and more terms of this Fourier series, the Fourier series terms are all very smooth.
So what we're doing is trying to represent something with sharp corners by a series of smooth functions.
Which means if we're going to be able to represent it, we're only going to be able to represent it by adding on more and more terms, which hopefully are going to be coming closer and closer to approximating this the way it should be approximated.
Now if you look at these terms here, these Fourier series, you will notice that every one of them, except this original one which is at 1/2 -- so this is the first term here in the Fourier series.
The second term is to add on that cosine term.
The first term is sitting here at 1/2.
Every other one of them is zero at minus 1/4 and zero at plus 1/4.
So when we add up all of those terms, what we wind up with is not what we started out with, but something which is 0 from minus 1/2 to minus 1/4.
It's 1/2 at the value minus 1/2.
It's 1 all along here.
It's 1/2 over here, and 0 down here.
Every time you study Fourier series you find out about these bizarre things.
Every time you have a discontinuity in the function, the Fourier series comes out to split the difference on you.
So you like to define your functions at discontinuities as either being here at minus 1/4 or here at minus 1/4.
Then when you come back from the Fourier series, it forces you to be there.
Well, what does this say?
It says that u of t, which we started out defining to have a certain value at minus 1/4 and plus 1/4, is equal to its Fourier series everywhere except here and here.
I have to ask you to take that on faith.
But you can see that it's not equal to it at those discontinuities.
And it shouldn't be surprising that it's not equal to its discontinuities.
I could have defined it as being zero at minus 1/4 or 1 at minus 1/4, and just that one point shouldn't change our integrals too much.
Because of that as engineers, I mean at some level we have to say we don't care.
It's only a modeling issue.
Functions don't have perfectly straight discontinuities in them.
If they do you don't care how you define it, it's a discontinuity.
This Fourier series is sort of coming back and slapping us with that.
And it's saying OK, the function u of t is not the same as its Fourier series because the two are different at these two points.
You say OK, I don't care that they're not different at those two points.
They're the same everywhere else.
A mathematician comes back and says a function is a function is a function, and a function is defined at every value of t, and if u of t is equal to v of t, it means that at every value of t, u of t is equal to v of t. And you say ah.
Well, turns out that by studying Lebesgue theory, all of those problems get resolved.
Lebesgue was a very powerful mathematician.
But you know at some level deep in his heart, he was an engineer.
He was trying to get rid of all this nonsense that people talked about, and he resolved this question about how to talk about these functions in a nice way.
I mean really, good engineers are mathematicians at heart, too.
I mean at some level we all become the same.
What Lebesgue tried to say is that two functions are said to be equivalent in the L2 sense -- I'll talk about this L2 notation later -- if their difference has zero energy.
In other words, Lebesgue said what's really important is not what functions are at each point, but really things about their energy.
So what you would like to have is if u of t and v of t, if the difference between them, you take the magnitude of that and you square it, if that difference is equal to zero, you have to recognize that there's no possible way that you could ever distinguish those two functions, except just by fiat, by saying this is equal to this and not equal to that.
That's the only way you could straighten that out in your minds.
So we say the two functions are L2 equivalent if their difference has zero energy.
Well, we have a couple of problems there.
How do we define that?
At this point, we're sort of already deep in the mathematical soup because, in fact, we're trying to make these small distinctions and make them make sense.
We're also going to see, as we go on to two functions that have the same Fourier series, are L2 equivalent, because if two functions have the same Fourier series, put one of them there and one of them there, and we're going to see that when we expand it in a Fourier series they're both the same, and we're going to see that, in fact, what that means is that their energy difference has to be zero.
Which says that if you don't talk about functions, but if you talk about their Fourier series, all of these confusions go away about things having to be equal point-wise.
So let's go on and try to say a little more about this.
One of the problems that we come up with is that not all time-limited functions, in fact, have Fourier series, even in a sense of L2 equivalents.
You can think of functions which are so awful that they don't have a Fourier series, although it's hard to find them.
We really want to make general statements about classes of functions.
Why do we want to do that?
Well, I can give you two reasons for it.
The two reasons are both, I think, both good reasons.
The first reason is that as we deal particularly with channels, we have to look at things both in a time domain and in a frequency domain.
We look at things in the domain and the frequency domain, we have a function in a time domain, we have a Fourier transform in the frequency domain, and it turns out that nice properties in a time domain are not always carried over to the frequency domain and vice versa.
Give me one example of that.
Suppose you think of a constant.
The constant function, which is equal to 1 everywhere on the real line.
Nice function, right?
It models various things very well.
It doesn't model physical reality, really, because I mean you don't care what this function was before the fourth ice age.
You don't care what it is after we all blow ourselves up, I hope in not too many years.
I mean I hope in more than just a few years.
Therefore, when we model something as a constant, what do we mean?
We mean that over the interval of time that we're interested in, this function is equal to a constant.
And it means we don't want to specify what the time interval is that we're interested in, which is a common thing, because you don't want to set time limits on something.
Now, you take those functions which go on and on forever.
Well, the Fourier series, you don't have any problem with them because we're going to truncate the function anyway before we take a Fourier series.
But if we look at the Fourier integral, and we'll see this as soon as we get into the Fourier integral, the awful thing is that what has happened in the thousand years before the fourth ice age back, is just as important in the Fourier transform as what happens in the thousand years right now.
In other words, everything is important.
Things back in the dim past clobber you in a frequency domain.
Things in the distant future clobber you in the frequency domain.
Therefore, since we have to face the fact that we're dealing with approximations, since we have to face the fact that we want to ignore things -- back there we want to ignore things there.
When we look at constants in the frequency domain, we don't mean that something is constant over all frequency.
We mean it's constant over the range of frequencies that we're interested in and we don't want to specify what that range is.
You have the same problems going from there back to time.
So, as soon as we face the fact that we're really interested in approximations, and the approximations that we deal with normally in time are not the same as the approximations we deal with in frequency, at that point, we start to realize that we have to be able to make general statements about what functions do what kinds of things.
We have to make general statements about what has a Fourier transform, what doesn't, what has a Fourier series, what doesn't have a Fourier series.
Now, one of the things we're aiming at is to define a class of functions called L2 functions.
These are basically functions which have finite energy.
The nice thing about those functions is that every one of them has a Fourier transform, and the Fourier transform is also an L2 function.
All the other things that you deal with -- continuity, things like that -- doesn't carry over at all.
L2 is the only property that I know of that really carries over from time functions to Fourier transform.
So we really want to be able to talk about that.
So we want to talk about these finite energy functions.
We want to be able to talk about representing finite energy functions.
I say here, all physical wave forms have finite energy, but their models do not necessarily have finite energy.
In other words, we look at a constant -- a constant does not have finite energy.
How about an impulse?
Does an impulse have finite energy?
[INAUDIBLE]
What?
Yes?
How many people think the answer is yes?
The hands are going up very slow.
Well, the answer is no.
Let me explain why.
It's something you should know.
But it's something that you get blinded by studying too much signals and systems before you study any communication, because you're talking about all sorts of transforms, all sorts of things that you deal with as functions, which you're dealing with electronically, instead of those functions that you're interested in transmitting.
If you think of a narrow pulse of height 1 over epsilon, and of width epsilon, it has unit area.
So I make epsilon very, very small.
This starts to look like a unit impulse, right?
In fact, you usually define a unit impulse somehow or other as thinking of some limiting process for this kind of rectangular function.
What's the energy of that function?
What?
[INAUDIBLE PHRASE]
Energy is 1 over epsilon, yes.
What happens as epsilon goes to infinity?
Bing.
If you put an impulse into an electrical circuit it'll blow it up.
You usually don't care about that because you don't see impulses in the physical world.
You see things which are so narrow and so tall that in terms of the filters that you put them through, which have smaller bandwidth, those narrow pulses behave very nicely, and as you make those narrow impulses more and more high and more and more narrow, they behave the same way after they go through a filter.
But before that they're ugly and they have infinite energy.
In fact, you could determine that from two of the statements I made earlier, and I'm sure I can't blame any of you for not doing that.
I said that all finite energy functions have Fourier transforms which are finite energy, and you all know that the Fourier transform of a unit impulse is a constant, and the Fourier transform of a constant is a unit impulse.
Therefore, if the constant has infinite energy, the unit impulse has to have infinity energy also.
So anyway, we have all these functions we like to deal with all the time which do not have finite energy.
We don't want to deal with those in this course, not because they aren't very useful in signal processing, but because they aren't useful as wave forms which we will transmit, and they aren't very useful as source wave forms.
Source wave forms do not behave that way.
Source wave forms that we want to encode all have finite energy, and we'll see why as we go on.
So now I want to try to tell you what the big theorem is about Fourier series.
I will do this in terms of a bunch of things that you don't understand yet.
The theorem says we're looking at a function u of t, which is time-limited -- nothing strange there, that's what we've been looking at all along so far.
It's a function that goes from minus t over 2 to t over 2, and we'll let it go into the complex numbers because we said it's just as easy to deal with complex functions as real functions.
We're going to assume that it has finite energy.
Then it says for each index k, the Lebesgue integral, u sub k equals this.
In other words, this is the formula for finding the Fourier series coefficient.
What we're saying here is we have to redefine that to be a Lebesgue integrall instead of a Reimann integral.
But anyway, when you define it that way it always exists and it is always finite, necessarily.
It can't be infinite, it can't not exist.
It just is there.
The other thing is -- now this formula is harder to swallow as a whole.
Let's try to look at it.
What's inside of here is the difference between u of t and a finite approximation to the Fourier series.
Now if you're taking a Fourier series, whether you're taking it on a computer or calculating it or what you're doing with it, you're never going to take the infinite sum.
You're always going to be dealing with some finite approximation.
This says that the different between u of t and these finite approximations, if you take that difference and you find the energy in that difference, says the energy in that difference gets small.
In other words, it says that as you take more and more terms in your Fourier series, you get a function which comes closer and closer to u of t in terms of energy difference.
So that's the kind of statement that we want in this course, because we're aiming towards saying that this in the limit looks like that in terms of having zero energy difference between it.
Namely, this is going to allow this function to converge to one of these strange functions that has bizarre values on discontinuities of u of t, because that doesn't make any difference in terms of energy.
It says also the energy equation is satisfied.
The energy equation -- I didn't say it was the energy equation -- I hope I said what it was.
Blah blah blah.
I have to write it down.
The energy equation says that the integral of u of t magnitude squared dt from minus t over 2, the t over 2 is equal to the sum over k of u hat of k magnitude squared.
And there's a 1 over t in here.
There's either a 1 over t or a t -- I'm pretty sure it's a 1 over t, but I wouldn't swear to it.
I can't believe I didn't have that written down.
At any rate it's in the notes.
So you will find it there.
I can't keep these constants straight.
I think I did it wherever I stopped -- here.
That's where I did it.
That's the energy equation.
Yeah, I got it right, amazing.
The integral of u of t squared dt is equal to t times the sum of all the Fourier coefficients.
I forgot to say this.
This is something I wanted to say.
This energy equation is important because in terms of source coding, if you take a function u of t, if you find this Fourier series, u of k, that's a sequence of coefficients.
If we take that sequence of coefficients and we quantize them to some other set of values, v sub k, and then we recreate the function corresponding to this set up Fourier coefficients, we get sum v of t. So we start out with u of t, we go all the way through all of this chain, going through a channel and everything else, come back with some function v of t. This applied to u of t minus v of t says that the energy difference between u of t, and our re-created version v of t, is exactly the same as t times the sum of the differences between u sub k and v sub k squared.
Now, that is the reason why most people talk about mean square error most of the time, because if you can control the mean square error on your coefficients, you're also controlling the mean square error on the functions.
This formula does not work for magnitude or cubes or fourth powers or anything else.
It only works for these square powers.
That's why everybody uses mean square error rather than other things.
It also makes sense for energy, because we believe in, in some sense, energy ought to be important and energy is important.
So, the final part of the theorem says that finally, if you start out with a sequence of complex numbers and the sequence of numbers has finite energy in this sense, then there's an L2 function, u of t, which satisfies all of this stuff.
In other words, you can go from function to Fourier series, you can go from Fourier series to function.
You can go either way.
So long as you have finite energy this all works.
I want to spend just a couple of minutes talking about the difference between Reimann and Lebesgue integration to show you that, in fact, it isn't really any big deal.
When you're talking about Reimann integration -- I've just showed the integral for a function between zero and 1.
How do you conceptually find the integral of a function -- a Reimann integral, which is what you're used.
You split up the interval on the horizontal axis into a bunch of equal intervals of size 1 over n each.
So you split it into n intervals, each one a size 1 over n. You approximate the value of the function in each interval somehow, as the smallest value, the largest value, the mean value, whatever you want to do.
That doesn't make any difference because as the intervals become smaller and smaller and smaller and you have a function which is sort of smooth in some sense, then this Reimann sum here is going to get close to this interval.
In other words, the Reimann sum is going to approach a limit, and that limit is defined as the Reimann integral.
So that's the thing you're used to.
The Lebesgue integral is similar, oh and in a sense, it's no more complicated.
What you do is instead of quantizing the horizontal axis into regions of size 1 over n and letting 1 over n get small, you quantize the vertical axis into intervals of size epsilon and you're going to let epsilon get small later.
Then the thing that you do is you ask how much of the function is in each of these intervals here?
So the amount of the function that lies between 2 epsilon and 3 epsilon is an interval t2 minus t1 -- there's that interval where the function is in this range.
There's also this interval over here between t3 and t4.
So you say the measure of the function in this interval here is t2 minus t1 plus t4 minus t3.
You say the measure of the function in this region here, vertical region here, is t1, namely, this, plus 1 minus t4, namely, this region over here.
So for any function you can do the same thing.
That's the Lebesgue integral.
Lebesgue integral says you do this and you let these epsilons get very small and you just add them up.
Let me say just -- last slide.
Turns out that whenever the Reimann integral exists, namely, that limit exists, the Lebesgue interval also exists and has the same value.
All of the familiar rules for calculating Reimann integrals also apply for Lebesgue integrals.
For some very weird functions, the Lebesgue integral exists, but the Reimann integral doesn't exist.
For some extraordinarily weird functions, there aren't even any examples in the notes.
I couldn't find an example which I thought was palatable, not even the Lebesgue integral exists.
So the Lebesgue integral is much more general than the Reimann integral.
But the nice thing is you can almost forget about it because everything you know to do still works, it's just that some of the things that didn't work before now work.
Because those things that didn't work before now work, your theorems can be much more general than they were before.
We'll talk more about that next time.
This material we're talking about right now is in the appendix to the lectures that just got passed out.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I really stated everything about discrete coding as clearly as I could in the notes.
I stated it again as clearly as I could in class.
I stated it again as clearly as I could in making up problems that would illustrate the ideas.
If I talked about it again it would just be totally repetitive.
So, at this point, if you want to understand things better, you gotta come up with specific questions and I will be delighted to deal with them.
So we want to go on--.
Oh, there's one other thing I wanted to talk about.
We're not having a new problem set out today.
I don't think most of you would concentrate on it very well.
I'll tell you what the problems are, which will be due on October 14th.
I'll pass it out later.
It's problems 1 through 7, and the end of lectures 8 to 10, and one other one all the way at the end, problem 26.
So, 1, 2, 3, 4, 5, 6, 7 and 26.
So you can get started on them whenever you choose and I'll pass out a traditional problem set form next time.
Last time we started to talk about the difference between Reimann and Lebesgue integration.
Most people tell me this is further into mathematics then I should go.
If you agree with them after we spend a week or two on this, please let me know and I won't torture future students with it.
My sense is that in the things we're going to be dealing with for most of the rest of the term, knowing a little bit extra about mathematics is going to save you an awful lot of time worrying about trivial little things that you shouldn't be worrying about.
In other words, the great mathematicians of the 19th century who developed -- yeah, the 19th century, but partly the 20th century.
These mathematicians were really engineers at heart.
The 20th century and the 21st century, mathematics has very much become very separated from physics and applications.
But in the 19th century and in the early 20th century, mathematicians and physicists were almost the same animals.
You could scratch one and you'd find another one there.
They were very much interested in dealing with real things.
They were like the very best of mathematicians everywhere and the very best of engineers everywhere.
They really wanted to make life simpler instead of making life more complicated.
One way that many people express it is that mathematicians are lazy, and because they're lazy, they don't want to go through a lot of work, and therefore, they feel driven to simplify things.
There's an awful lot of truth in that.
What we're going to learn here I think will, in fact, simplify what you know about Fourier series and Fourier integrals a great deal.
Engineers typically don't worry about those things.
An awful lot of engineers, and unfortunately, even those who write books often state theorems and leave out the last clause of the theorem, and the last clause of most of those theorems the only way to make them true is to add the clause or not, as the case may be, at the end of it, which makes what they state absolutely meaningless, because anything you can add or not, as the case may be, and it becomes true at that point.
The whole question becomes well, what are those cases under which it's true.
That's what we're going to deal with a little bit here.
We're going to say just enough so you start to understand these major things that cause problems, and I hope you will get to the point where you don't have to worry about them after that.
Well, the first thing, we started talking about it last time, we were talking about the difference between Reimann integration and Lebesgue integration.
Reimann was a great mathematician, but he came before all of these mathematicians who were following Lebesgue, and who started to deal with all of the problems with this classical integration, which started to fall apart throughout most of the 19th century.
What Reimann said, well, split up the axis into equal intervals, then approximate the function within each interval, add up all of those approximate values, and then let this quantization over the time axis become finer and finer and finer.
If you're lucky, you will come to a limit.
You can sort of see when you get to a limit and when you don't get to a limit.
If the function is smooth enough and you break it up finely enough, you're going to get a very good approximation, and if you break it up more and more finely, the approximation gets better and better.
If the function is very wild, if it jumps around wildly, and we'll look at examples of that later, then this doesn't work.
We'll see why, in fact, this approach does work.
Lebesgue said, no, instead of quantizing along this axis, quantize on this axis.
So he said, OK, start with a zero, quantize into epsilon, 2 epsilon, 3 epsilon and so forth, and everybody when they talk about epsilon they're thinking about something small.
They're also thinking about making it smaller and smaller and smaller and hoping that something nice happens.
So then what he said is after you quantize this axis, start to look at how much of the function lies in each one of those little windows.
I've drawn that out here, mu 2 is the amount of the function that lies between 2 epsilon and 3 epsilon.
Now the function lies between 2 epsilon and 3 epsilon starting at this point, which I've labeled t1, going up to this point, which I've labeled t2, on over here.
It's not in this interval until we get back to t3.
It stays in this interval until t4.
Lebesgue said, OK, let's say that the function is between 2 epsilon and 3 epsilon over a region of size, t2 minus t1, which is this size interval, plus t4 minus t3, which is this size interval.
Instead of saying size, he said size gets too confusing so I'll call it measure instead.
That was the beginning of measure theory, essentially.
Well, in fact other people talked about measure before Lebesgue.
There are a lot of famous mathematicians who were all involved in doing this.
Anyway, that's the basic idea of what measure is concerned with.
Now, for this curve here, it's a nice smooth curve, and you can almost see intuitively that you're going to get the same thing looking at it this way or looking at it this way.
In fact, you do.
Anyway, what he finally wound up with is after saying what the measure was on each one of those little slices, how much of the function lay in each one of those intervals, he would just add them all up.
He would add up how much of the function was in each slice, he would multiply how much of the function was in a slice by how high the slice was up, and then he'd get an answer.
One difference between what he did and what Reimann did was that he always got a lower bound in doing it this way.
If he was dealing with a non-negative function, his approximation was always a little less than what the function was, because anything that lay between 2 epsilon and 3 epsilon, he would approximate it as 2 epsilon, which is a little less than numbers between 2 epsilon and 3 epsilon.
So this is a lower bound, whereas this is whatever it happens to be -- however you decide to approximate the function there, and there are lots of ways of doing it.
Well, I won't prove any of these things, I just want to point them out so that when you get frustrated with this, you can always rely on this.
Which says that whenever the Reimann integral exists, in other words, whenever the integral that you're used to exists, mainly, whenever it has meaning, Lebesgue integral gives you the same value.
In other words, you haven't lost anything by going from Reimann integration to Lebesgue integration.
You can only gain, you can't lose.
The familiar rules for calculating Reimann integrals also apply for Lebesgue integrals.
You remember what all those rules are, you probably know all of them better even than this fundamental definition of an integral, which is split up the function into tiny little increments, because throughout all of the courses that you've taken, learning about integration, learning about differentiation, learning about all of these things, what you've done for the most part is to go through exercises using these various rules.
So, you know how to integrate lots of traditional functions.
You memorized what the integral of many of them is.
You had many ways of combining them to find out what the integral of many other functions is.
If you program a computer to calculate these integrals, a computer can do it both ways.
It can either use all the rules to find out what the value of an integral is, or it can chop things up finely and find out that way.
As I said before, if you think that being able to calculate integrals is what engineering is about, think again.
I told you before you could be replaced by a digital computer.
It's worse than that.
You could be replaced by your handheld Palm, whatever it's called.
You can be replaced by anything.
After a while we're going to wear little things embedded in our body that will tell us when we're sick and things like this.
You can be replaced by those things even.
So you really want to learn more than just that.
For some very weird functions, the Lebesgue integral exists, but the Reimann integral doesn't exist.
Why do we want to worry about weird functions?
I want to tell you two reasons for it -- I told you about it last time.
One thing is an awful lot of communication theory is concerned with going back and forth between the time domain and the frequency domain.
When you talk about things which are straightforward in the time domain, often they become very, very weird in the frequency domain and vice versa.
I'm going to give you a beautiful example of that probably in the next class.
You can look at it -- it is in the appendix.
You'll see this absolutely weird function which has a perfectly well-defined Fourier transform.
Therefore, the inverse Fourier transform of this nice looking thing is absolutely weird.
The other reason is even more important.
We have to deal with random processes.
We have to deal with noise functions, which are continuously varying functions of time.
We have to deal with what we transmit, which is continuously varying functions of time.
Just like when we were dealing with sources, we said if we want to compress a source it's not enough to think about what one source sequence is.
We have to think about it probabalistically.
Namely, we have to model these functions as sample values of what we call a stochastic process or a random process.
In other words, when we start talking about that, these functions which already look complicated just become sample points in this much bigger space where we're dealing with random processes.
Now, when you're dealing with sample points of these random processes, weirdness just crops up as a necessary part of all of this.
You can't define a random process which consists of only non-weird things.
If you do you get something that doesn't work very well, it doesn't do much for you.
People do that all the time, but you run out of steam with it very, very quickly.
So for both reasons we have to be able to deal with weird things.
The most ideal thing is to be able to deal with weird things and get rid of them without even thinking about them.
What's the nice thing about Lebesgue theory?
It let's you get rid of all the weirdness without even thinking about it.
But in order to understand it to start with so we get rid of all that weird stuff, we have to understand just a little bit about what it's about to start with.
So that's where we're going.
Well, in this picture -- let me put the picture back up again -- I sort of sluffed over something.
For a nice, well-behaved function, it's easy to find out what the measure of a function is.
Namely, what the measure is of the set of values where you're in some tiny little interval, and that's straightforward, it was just the sum of a bunch of intervals.
But now we come to the clincher, which is we want to be able to deal with weird functions too, and for weird functions this measure, the set of values of time in which the function is in some tiny little slice here, the measure of that is going to become rather complicated.
Therefore, we have to understand how to find the measure of some pretty weird sets.
So, Lebesgue went on and said OK, I'll define how to find measure of things.
It's all very intuitive.
If you have any real numbers, a and b -- if I have two real numbers I might as well say one of them was less than or equal to the other one, so a is less than or equal to b. I want to include minus infinity and plus infinity here.
When we say the set of real numbers, we don't include minus infinity and plus infinity.
When we talk about the extended set of real numbers, we doing include minus infinity and plus infinity, Here for the most part when we're dealing with measure theory, we really want to include minus infinity and plus infinity also because they make things easier to talk about.
So what Lebesgue said is for any interval the set of points which lie between a and b, and he started out with the open interval -- namely the set of points not including a and not including b, but including all the real numbers in between -- the measure of that is what we said before, and none of you objected to it.
The measure of an interval ought to be the size of the interval, because after all, all Lebesgue was doing was taking size and putting another name on it because we all thought of it as being size.
So the measure of the interval ab is b minus a.
Now as engineers we all know that it doesn't really make any difference whether you include a or you don't include a when we're trying to find the size of an interval.
So, he said OK, the size of that interval is b minus a, whether or not you include either of the end points.
Then he went on to say something else, the measure of a countable union of disjoint intervals is the sum of the measure of each interval.
We already said when we were trying to figure out what the Lebesgue integral was, that if you had several intervals, the measure of the entire interval, namely the measure of the union of those intervals ought to be the sum of the measure of each interval.
So all he's adding here is let's go on and go to the limit and talk about a countably infinite number of intervals and use the same definition.
Now there's one nice thing about that and what is it?
If we take the sum of a countable set of things, each of those things is non-negative.
So what happens when we take the sum of a bunch of non-negative numbers?
There are only two possibilities when you take the sum of even a countable set of non-negative numbers and what are they?
Either to the limit or it goes to infinity?
It goes to a limit or it goes to infinity, yes.
Lebesgue said well, let's say if it goes to infinity that's a limit also.
So, in other words, there are only two possibilities -- the sum is finite, you can find the sum, and the sum if infinite.
Nothing else can happen.
So that's nice.
Then Lebesgue said one other thing.
A bunch of mathematicians trying to deal with this did different things when they were trying to deal with very small sets.
Some of them said well these very small sets aren't measurable, others, and part of Lebesgue's genius was that he said if you can take a set of points and you can put them inside another set of points, which doesn't amount to anything, then this smaller set of points should have a measure which is less than or equal to the bigger set of points.
So that that says that any subset of something that has zero measure also ought to have zero measure.
Now when we look at some examples you'll see that that's not quite as obvious as it seems.
But anyway, he said that.
An even nicer way to put that is when you're talking about weird intervals, try to cover that weird set with a bunch of intervals.
If you can cover it with a bunch of intervals in different ways and the measure of the bunch of intervals, the countable set of intervals, is you can make it arbitrarily small, then we also say that this set has measure zero.
So any time you have a set s and you can cover it with intervals which have arbitrarily small measure, namely we can make the measure of those intervals as small as we want to make them, we say bingo, s has zero measure.
Now that's going to be very nice, because it let's us get rid of an awful lot of things.
Because any time we have some set which has zero measure in this sense, when we look back at what we did with the Lebesgue integral, it says good, forget about it.
If it has zero measure it doesn't come into the integral at all.
That's exactly the thing we want.
We'd like to get rid of all that stuff.
So if we're going to get rid of it we have these sets which have measure zero, they don't amount to anything, and we're not going to worry about them.
Let's do an example.
It's a famous example.
What about the set of rationals which lie between zero and 1?
Well, rational numbers are numbers where there's an integer numerator and an integer denominator, and people have shown that there are numbers which are approximated by rational numbers but they're not rational numbers.
You can take that set of rational numbers and you can order them.
The way I've ordered them here, you can't order them in terms of how big they are.
You can't start with the smallest rational number which is greater than zero because there isn't any such thing.
Whatever rational number you find which is greater than zero, I can take a half of it, that's a rational number, and that also is greater than zero and it's smaller than your number.
If you don't like me getting the better of you, you can then take half of that and come up with a smaller number than I could find, and we can go back and forth on this as long as we want.
So the way we have to order these is a little more subtle than that.
Here we're going to order them in terms of first the size of the denominator and next, the size of the numerator.
So there's only one fraction in this interval -- oh, I screwed that up.
I really wanted to look at the set of rational in the open interval between zero and 1, because I don't want zero in it, and I don't want 1 in it, but that's a triviality.
You can put zero and 1 in or not.
So, anyway, we'll leave zero and 1 out.
So I start out with 1/2 -- that's the only rational number strictly between zero and 1 with a denominator of 2.
Then look at the numbers which have a denominator of 3, so I have 1/3 and 2/3.
I look then at the rational numbers which have a denominator of 4.
I have 1/4, I have to leave out 2/4, because that's the same as 1/2 which I've already counted, so I have 3/4, Then I go on to 1/5, 2/5, 3/5, 4/5 and so forth.
In doing this I have actually counted what the integers are.
Namely, whenever you can take a set and put it into correspondence with the integers, you're showing that it's countable.
I've even labeled what the counting is.
A sub 1 is 1/2, A sub 2 is 1/3 and so forth.
Now, I want to stick this set inside of a set of intervals, and that's pretty easy.
I stick A sub i inside of the closed interval, which is A sub i on one side and A sub i on the other side.
So I'm sticking it inside of an interval which has zero measure.
Then what I'm going to do is I'm going to add up all of those things.
Whenever you add up an infinite number of things, you really have to add up a finite number of them and then look at what happens when you go to the limit.
So I add up all of these zeroes, and when I add up n of them I get zero.
I continue to add zeroes, I continue to have zero, and the limit is zero.
Now here's where the mathematicians have pulled a very clever swindle on you, because what they're saying is that infinity times zero is zero.
But they're saying it in a very precise way.
But anyway, since we said it in a very precise way, infinity times zero here is zero, the way we've said it.
But somehow that's not very satisfying.
I mean it really looks like we've cheated.
So let's go on and do this in a different way.
What we're going to do now is for each of these rational numbers, we're going to put a little hat around them, a little rectangular hat around them.
Namely a little interval which includes A sub i, and which also goes delta over 2 that way, delta over 2 this way, and also multiplies that by -- my computer knew I was going to talk about computers replacing people, so it made some mistakes to get even with me.
So this is delta times 2 to the minus i minus 1, and delta times 2 to the minus i minus 1.
In other words, you take 1/2 and you put a pretty big interval around it.
You take 1/3, you put a smaller interval around it.
2/3 you put a smaller interval around that and so forth.
Well, these intervals here are going to overlap.
But anyway, the union of two overlapping open intervals is an open interval which has a smaller measure than with two of them.
In other words, if you take this, the measure of this and add it to the measure of this, the union of this and this is just this.
We don't have to be very sophisticated to see that this length is less than this length plus length, because I've double counted things here.
So anyway, when I do this I add up the measure of all of these things and I get delta, and I then let delta get as small as I want to.
Again, I find that the measure of the rationals is zero.
Now if you don't like this you should be very happy with yourselves, because I've struggled with this for years and it's not intuitive, because with these intervals I'm putting in here, no matter how small I make that interval, there's an infinite number of rational numbers which are in that interval.
In other words, the thing we're trying to do is to separate the interval 0,1 into some union of intervals which don't amount to anything but which include all of the rational numbers.
Somehow this argument is a little bit bogus because no matter what number I look at between zero and 1, there are rational numbers arbitrarily close to it.
In other words, what's going on here is strictly a matter of which order we take limits in, and that's what makes the argument subtle.
But anyway, that is a perfectly sound mathematical argument.
You can't get around it.
It's why people objected to what Lebesgue was doing for a long time, because it wasn't intuitive to them either.
It was intuitive to Lebesgue, and finally it's become intuitive to everyone, but not really intuitive.
It's just that mathematicians have heard it so many times that they believe it.
I mean one of the problems with any society is that if you tell people things often enough they start to believe them.
Unfortunately, that's true in mathematics, too.
Fortunately in mathematics we have proofs of things, so that when somebody is telling you something again and again which is false, there are always people who will look at it and say no, that's not true.
Whereas in other cases, not necessarily.
There are also uncountable sets with measure zero.
For those of you who are already sort of overwhelmed by this, why don't you go to sleep for three minutes, it'll only take three minutes to talk about this, but it's kind of cute.
In the set I want to talk about something closely related to what people call the Cantor set, but it's a little bit simpler than that.
So what I'd like to do, and this is already familiar to you.
I can take numbers between zero and 1 and I can represent them in a binary expansion.
I can also represent them in a ternary expansion.
I can also represent them in a decimal expansion, which is what you've been doing since you were three years old or five years older or whenever you started doing this.
Well, ternary is simpler than decimal, so you could have done this a year before you started to deal with decimal expansions.
So I want to look at all of the ternary expansions.
Each real number corresponds to a ternary expansion, which is an infinite sequence of numbers each of which are zero, 1 or 2.
Now, what I'm going to do is I'm going to remove all of the sequences which contain any 1's in them at all.
Now it's not immediately clear what that's going to do, but think of it this way.
If I first look at the sequences, I'm going to remove all the sequences which start with 1.
So the sequences which start with 1 and have anything else after them, that's really the interval that starts at 1/3 and ends at 2/3, because that's really what starting with 1 means.
This is what we talked about when we talked about approximating binary numbers also, if you remember, is the way we proved the Kraft inequality.
It was the same idea.
The sequences which have a 1 in the second position, when we remove them we're removing the interval from 1/9 to 2/9.
We're also removing the interval from 7/9 to 8/9.
We're also removing the 4/9 to 5/9 interval, but we removed that before.
So we wind up with something which is now -- we've taken out this, we've now taken out this, and we've taken out this.
So the only thing left is this and this and this and this, right?
And we've removed everything else.
We keep on doing this.
Well, each time we remove one of these, all of the numbers that's -- when I do this n times, the first time I go through this process, I removed 1/3 of the numbers, I'm left with 2/3 of the interval.
When I remove everything that has a 1 in the second position, I'm down with 2/3 squared.
When I remove everything which has a 1 in the third position, I'm down to 2/3 cubed.
When I keep on doing this forever, what happens to 2/3 to the n?
Well, 2/3 to the n goes to zero.
In other words, I have removed an interval, I have removed a set, the measure 1, and therefore I'm left with a set of measure zero.
You can see this happening.
I mean you only have diddly left here and I keep cutting away at it.
So less and less gets left.
But what we now have is all sequences, all infinite sequences of zeroes and 2's.
So I'm left with all binary sequences except instead of binary sequences with zeros and 1's, I now have binary sequences with zeros and 2's.
How many binary sequences are there when I continue forever?
Well, you know they're an uncountable number, because if I take all the numbers between zero and 1, I represent them in binary zeroes and 1's, I have an uncountable number of them.
Well, because I have to have an uncountable number because we already showed that any countable set doesn't amount to anything.
Countable sets are diddly.
Countable sets all just go away.
So, anything which gets left has to be uncountable.
Again, people had to worry about this for a long time.
But anyway, this gives you an uncountable set which has measure zero.
So, back to measurable functions.
I'm going to get off of mathematics relatively soon, but we need at least this much to figure out what's going on here.
We say that a function is measurable.
Before we were only talking about sets of numbers being measurable.
We had to talk about sets of numbers being measurable because we were interested in the question of what's the set of times for which a function lies between 2 epsilon and 3 epsilon, for example.
What we said is we can say a great deal about that because we can not only add up a bunch of intervals, we can also add up a countable bunch of intervals, and we can also get rid of anything which is negligible.
So, a function is measurable if the set of t, such that u of t lies between these two points is measurable for each interval.
In other words, if no matter how I split up this interval, if no matter what slice I look at, the set of times over which the function lies in there is measurable.
That's what a measurable function is.
Everybody understand what I just said?
Let me try to say it once more.
A function is measurable if for every two values, say 3 epsilon and 2 epsilon, if the set of values t for which the function lies between 2 epsilon and 3 epsilon, if that set is measurable.
In other words, that's the set we were talking about before which went from here to here, and which went from here to there.
In this case for this very simple function, that's just the sum of two intervals.
If I make the function wiggle a great deal more, it's the sum of a lot more intervals.
So, we say the function is measurable if all of the sets are measurable.
Now, what I'm going to do is when I'm trying to define this integral, I'm going to have to go to smaller and smaller intervals.
Let's start out with epsilon, 2 epsilon, 3 epsilon and so forth.
Let's look at a non-negative function--.
Yeah?
Maybe I missed something, but could you tell me [UNINTELLIGIBLE] definition for a set, if measurable
What's the definition for a set is measurable.
I didn't really say, and that's good.
I gave you a bunch of conditions under which a set is measurable, and if I have enough conditions for which it's measurable then I don't have to worry about--.
I said that it is measurable under all of these conditions.
I'm saying I don't have to worry about the rest of them because these are enough conditions to talk about everything I want to talk about
[UNINTELLIGIBLE]
I will define my measure as all of these things.
Unfortunately, you need a little bit more, and if you want to get more you better take a course in real variables and measure theory.
Good.
So, if I want to now make this epsilon smaller, what I'm going to do is do it in a particular way.
I'm going to start out partitioning this into intervals of size epsilon.
Then I'm going to partition it into intervals of size epsilon over 2.
When I partition it into intervals of size epsilon over 2, I'm adding a bunch of extra things.
This thing gets added because when I'm looking at the interval between epsilon and 3 epsilon over 2, the function is in this interval here, over this [UNINTELLIGIBLE].
It's in this interval over this whole thing.
I'm representing it by this value down here.
Now when I have this tinier interval, I see that this function is really in this interval from here to there also, and therefore, instead of representing the function over this interval by epsilon, I'm representing it by 3 epsilon over 2.
In other words, as I add these extra quantization levels, I can never lose anything, I only gain things.
So I gain all of these cross-hatched regions when I do this, which says that when I add up all these things in the integral, every time I decrease epsilon by 2, the integral that I've got, the approximation to the integral increases.
Now what happens when you take a sum of a set of numbers which are increasing?
Well, they're increasing, the result that you get when you add them all up is increasing also, and therefore, as I go from epsilon to epsilon over 2 to epsilon over 4 and so forth, I keep climbing up.
Conclusion, I either get to a finite number or I get to infinity -- only two possibilities.
Which says that if I'm looking at non-negative functions, if I'm only looking at real functions which have non-negative values, the Lebesgue integral for a measurable function always exists, if I include infinite limits as well as finite limits.
Now, if you think back to what you learned about integration, and I hope you at least learned enough about it that you remember there are a lot of very nasty conditions about when integrals exist and when they don't exist.
Here that's all gone away.
This is a beautifully simple statement.
You take the integral of a non-negative function, if it's measurable, and there are only two possibilities.
The integral of some finite number or the integral is infinite.
It's never undefined, it's always defined.
I think that's neat.
A few people don't think it's neat, too bad.
I guess when I was first studying this, I didn't think it was neat either because it was too complicated.
So you have an excuse.
If you think about it for a while and you understand it and you don't think it's neat, then I think you have a real problem.
So now -- I did this.
I'm getting too many slides out here.
Here we go.
Here's something new.
Hardly looks new.
Let's look at a function now, just defined on the interval zero to 1.
Suppose that h of t is equal to 1 for each rational number and at zero for each irrational number.
In other words, this is a function which looks absolutely wild.
It just goes up to here and it's 1 or zero.
It's 1 at this dense set of points, which we've already said doesn't amount to anything, and it's zero everywhere else.
Now you put that into the Reimann integral, and the Reimann integral goes crazy, because no matter how small I make this interval, there are an infinite number of rational numbers in that interval, and therefore, the Reimann integral can never even get started.
For the Lebesgue integral, on the other hand, look at what happens now.
We have a bunch of points which are sitting at 1, we have a bunch of points which are sitting at zero.
The only thing we have to do is evaluate the measure of the set of points which are up in some tiny interval up in here.
What's this measure of the set of t's corresponding to the rational numbers?
Well, you already said that was zero.
Now, that's why Lebesgue integration works.
Any countable set, and in fact, any of these uncountable sets that measure zero get lost in here because you're combining them all together and you say they don't contribute to the integral at all.
That's why Lebesgue integration is so simple.
You can forget about all that stuff.
When we looked last time at the Fourier series for a square wave, you remember we found that everything behaved very nicely, except where the square wave had a discontinuity, the Fourier series converged to the mid-point, and that was kind of awkward.
Well, the mid-points where the function is discontinuous don't amount to anything, because in that case there were just two of them, there were only two points.
If they had measure zero it just washes away.
You all felt intuitively when you saw that example, that those points were not important.
You felt that this was mathematical carping.
Well, Lebesgue felt it was mathematical carping too, but he went one step further and he said here's a way of getting rid of all of that and not having to worry about it anymore.
So you've now gotten to the point where you don't have to worry about any of this stuff anymore.
Now let's go a little bit further.
We're almost at the end of this.
If I take a function which maps the real numbers into the real numbers, in other words, it's a function which you can draw on the line.
You take time going from minus infinity to plus infinity, you define what this function is at each time.
That's what I'm talking about here, a function which you can draw.
The functions magnitude of u of t, and the function magnitude of u of t squared are both non-negative.
Now I'm not going to prove this, but it turns out that the magnitude and the magnitude squared are both measurable functions if u of t is a measurable function.
In fact, from now on we're just going to assume that everything we deal with is measurable, every function is measurable.
I challenge any of you without looking it up in a book to find an example of a non-measurable function.
I challenge any of you to find an example of a non-measurable set.
I challenge any of you to understand the definition of a non-measurable set if you look it up in a book.
You've heard about things like the axiom of choice and things like that, which are very fishy kinds of things -- that's all involved in finding non-measurable function.
So, any function that you think about is going to be measurable.
I hate people who say things like that, but it's the only way to get around this because I don't want to give you any examples of that because they're awful.
Since magnitude of u of t and magnitude of u of t squared are measurable and they're non-negative, their integrals exist.
Their integrals exist and are either a finite number or they're infinite.
So, we define L1 functions, and we'll be dealing with L1 functions and L2 functions all the way through the course, u of t is an L1 function if it's measurable, and if this integrals is less than infinity.
That's all there is to it.
u2 is L2 if it's measurable and the integral of u of t squared is less than infinity.
I could have said that at the beginning, but now you see that it makes a lot more sense than it did before because we know that if u of t is measurable, this integral exists -- it's either a finite number or infinity.
The L1 functions are those particular functions where it's finite and not infinite.
Same thing here.
The L2 functions are those where this is finite.
This is really the energy of the function, but now we can measure the energy of even weird functions which are zero on the irrationals and one on the rationals.
Even things which are zero on the non-cantor set points and 1 on the cantor set points, still it all works.
So those define the set L1 and L2.
Now, a complex function u of t, which maps r into c, why does a complex function map r into c?
What's the r doing there?
Think of any old complex function you can think of.
e to the i, 2 pi t. That's something that wiggles around, the sinusoid.
t is a real number.
So that function e to the i 2 pi t is mapping real numbers into complex numbers.
That's what we mean by something which maps the real numbers into complex numbers.
We always call these complex functions.
I mean mathematicians would say yeah, a function could be anything.
But you know, when most of us think of a function, we're thinking of mapping a real variable into something else, and when we're thinking of mapping a real variable into something else, we're usually thinking of mapping it into real numbers or mapping it into complex numbers, and because we want to deal with these complex sinusoids, we have to include complex numbers also.
So a complex function is measurable by definition if the real part and the imaginary part are each measurable.
We already know when a real function is measurable.
Namely, a real function is measurable if each of these slices are measurable.
So now we know when a complex function is measurable.
We already said that all of the complex functions you can think of and all the ones we'll ever deal with are all measurable.
So L1 and L2 are defined in the same way when we're dealing with complex functions.
Namely, just whether this integral is less than infinity and this integral is less than infinity.
Since these functions -- this is a real function from real into real, this is a real function from real into real.
So those are well-defined.
What's the relationship between L1 functions and L2 functions?
Can a function be L1 and not L2?
Can it be L2 and not L1?
Yeah, it can be both, unfortunately.
All possibilities exist.
You can have functions which are neither L1 nor L2, functions that are L1 but not L2, functions that are L2 but not L1, and functions that are both L1 and L2.
Those are the truly nice functions that we like to deal with.
But there's one nice that you can say, and that follows from a simple argument here.
If u of t is less than or equal to 1, if the magnitude of u of t is less than or equal to 1--.
Let's start out by looking at u of t being greater than or equal to 1.
If u of t is greater than or equal to 1, then u squared of t is even bigger.
You see the thing that happened is when u of t becomes bigger than t, u squared of t becomes even more bigger.
When u of t is less than 1, u squared of t is less than u of t. But if u of t is less than or equal to 1, it's less than 1.
So in all cases, for all t, u of t, magnitude is less than or equal to u of t squared plus 1.
So that takes into account both cases.
It's a bound.
So if I'm looking at functions which only exist between over some limited time interval, and I take the integral from minus t over 2 to the t over 2 of the magnitude of u of t, I get something which is less than or equal to the integral of u squared of t plus the integral of 1, and the integral of 1 over this finite limit is just t. This says that if a function is L2 and the function only exists over a finite interval, then the function is L1 also.
So as long as I'm dealing with Fourier series, as long as I'm dealing with finite duration functions, L2 means L1.
All of the nice things that you get with L1 functions apply to L2 functions also, and there are a lot of nice things that happen for L1 functions.
There are a lot of nice things that happen for L2 functions.
You take the union of what happens for L1 and for L2, and that's beautiful.
You can say anything then.
Can't calculate anything, of course, but we all said, we leave that to computers.
Let's go back Fourier series now, let's go back to the real world.
Any old function we have u of t, the magnitude of u of t and the magnitude of u of t times either the 2 pi ift, for any old f, this thing has magnitude 1, right, a complex exponential.
Real f, real t. This just has magnitude 1.
And therefore, this magnitude is equal to this magnitude.
This says that if the function u of t is L1, then the function u of t times either the 2 pi IFT is also L1, which says if we can integrate one we can integrate the other.
In other words, the integral of u of t, either the 2 pi ift, the magnitude of the fdt is going to be less than infinity.
Since we're taking the magnitude, it's either finite or it's infinite, and since u of t in magnitude when we integrate it is less than infinity, this thing is less than infinity also.
Now, this is a complex number in here.
So you can break it up into a real part and an imaginary part, and if this whole thing, if the magnitude is less than infinity, then the magnitude of the real part is finite.
If you take the real part over the region where this is positive and the region where it's negative, you still get non-negative numbers.
In other words, if we're taking the integral of something which has positive values -- I should have written this out in more detail, it's not enough to--.
I'm taking the integral of something which--.
This is u of t. If I can find another color.
Let me draw magnitude of u of t on top of this.
This thing here is magnitude of u of t. What?
[INAUDIBLE]
I'm making it real for the time being because I'm just looking at the real part.
In other words, what I'm looking is this quantity here.
Later you can imagine doing the same thing out on complex numbers, OK?
So what I'm saying is if I just look at the real part of u of t -- call this real part of u of t, if you like.
If I know that this is finite, this is non-negative, I know that the positive part of this function has a finite integral, I know that the negative part of it has a finite integral.
In other words, the thing which makes integration messy is you sometimes have the positive part being infinite, the negative part being infinite also, and the two of them cancel out somehow to give you something finite.
When you're dealing with the magnitude of u of t, if the magnitude of u of t has an infinite integral, then that messy thing can't happen.
It says that the positive part has a finite integral, the negative part has a finite integral also.
If you take the imaginary part, visualize that out this way, the positive part of the imaginary part has a finite integral, the negative part of the imaginary part has a finite integral also.
It says if the magnitude of u of t, that integral always exists, and if it's finite, then these positive parts of the real, the positive part of the imaginary part, all of those are finite, and it says the integral itself is finite.
Which says that this integral here has to be finite.
Namely, the positive part of the real part, the negative part of the real part, the positive part of the imaginary part, the negative part of the imaginary part, all four have to be finite, just because this quantity here is finite.
Now, if u of 2 is L2 and also time limited, it's L1, and the same conclusion follows.
So this integral always exists if it's over a finite interval.
So at this point we're really ready to go back to this theorem about Fourier series that we stated last time and which was a little bit mysterious at that point.
In fact, at this point we've already proven part of it.
I wanted to prove it because I wanted you to know that not everything in measure theory is difficult.
An awful lot of these things, after you know just a very small number of the ideas, are very, very simple.
Now, you will think this is not simple because you haven't had time to think about the ten slides that have gone before.
But if you go back and you look at them again, if you read the notes, you will see that, in fact, it all is pretty simple.
What this says is if u of t, a complex function real into c, but time limited, suppose it's an L2 function, then it's also L1 over that interval minus t over 2 to plus t over 2.
Then for each k and z, then for each integer k, this integral here, this function here, is now an L1 function, and therefore, this integral exists, is finite.
You divide by t is still finite.
So that Fourier coefficient has to exist and it has to exist as a finite value.
Now, you look at Reimann integration, and you look at the theorems about Reimann integration, and if they're stated by somebody who was stating theorems, the conditions are monstrous.
This is not monstrous.
It says all you need is measurability and L1, which says it doesn't go up to infinity.
That's enough to say that every one of these Fourier coefficients has to exist.
It might be hard to integrate it, but you know it has to exist.
Now the next thing it says is that -- this is more complicated.
What we would like to say and what we tried to say before and what you like to say with the Fourier series is that u of t is equal to this, where you sum from minus infinity to plus infinity.
We saw that we can't say that.
We saw that we can't it for functions which have step discontinuities, because whenever you have a step discontinuity, the Fourier series converges to the mid-point of that discontinuity.
If you were unfortunate enough to try to make life simple and define the function without defining it at the step discontinuity as the mid-point, then the Fourier series would not be equal to u of t. But what this says is that if you take the difference between u of t and a finite expansion, and then you look at the energy in that difference, it says that the energy and the difference goes to zero.
Now that's far more important than having this integral be equal to that, because frankly, we don't care a fig for whether this is equal to this at every t or not.
What we care about is when we add more and more terms onto this Fourier series.
I mean in engineering we're always approximating things.
We have to approximate things.
We talk about functions u of t, but our functions u of t are just models of things anyway, and we want those models to really converge to something, which means that as we take more and more terms in the Fourier series, we get something which comes closer to u of t and it comes closer in energy terms.
Remember when we take this, when we approximate it in this way by a finite Fourier series, and we then quantize coefficients, and then we go back to the function, we have lost all the coefficients and all of these terms for the very high frequencies.
We've dropped them off.
What this says is if we take more and more of them and then we quantize and we go back to a function v of t it says as we add more and more Fourier coefficients and quantize carefully, we can come closer and closer to the function that we started with.
You don't get that by talking about point to point equality, because as soon as we quantize you lose all of that anyway.
You don't have a quality anywhere anymore, and the only thing you can hope for is a small mean square error.
So, this looks more complicated than what you would like, but what I'm trying to tell you is that this is far more important than what you would like.
What you would like is not important at all.
I can give you examples of things where this converges to this everywhere, but in fact, no matter how many terms you take, the energy difference between these two things is very large.
Those are the ugly things.
That says you never get a good approximation, even though looking at things point-wise everything looks great.
But what you're really interested in is these mean square approximations and what this Fourier series thing says is that if you deal with measurable functions then this converges just to that in this very nice energy sense, and energy is what we're interested in.
The final part of this -- I mean I talked about this a little bit last time -- sometimes instead of starting out with a function and approximating it with the Fourier series, you want to start out with the coefficients and find the function.
In fact, when we start talking about modulation, that's exactly what we're going to be doing because we're always going to be starting out with these digital sequences, we're going to be finding functions from the digital sequences.
This final part of the theorem says yes, you can do that and it works.
It says that given any set of coefficients where the coefficients have finite energy, in other words, where the sum is less than infinity, then there's an L2 function, u of t, which satisfies the above in this limiting sense.
Now this is the hardest thing of the theorem to prove.
It looks obvious but it's not.
But anyway, it's there.
It says these approximations are rock solid and you can now forget about all of this measure theoretic stuff, and you can just live with the fact that all of these in terms of L2 approximations, work.
I mean again, it doesn't look beautiful at this point because you haven't seen it for long enough.
It really is beautiful.
It's beautiful stuff.
When you think about it long enough, for 40 years is how long I've been thinking about it, it becomes more and more beautiful every year.
So, if you live long enough it'll be beautiful.
Any time you talk about this type of convergence, we call it convergence in the mean, and then notation we will use is l.i.m., which looks like limit but which really stands for limit in the mean.
We will write that complicated thing on the last slide, which I said is not really that complicated, but we will write this as this.
In other words, when we write limit in the mean, we mean that the difference between these two sides in energy sense goes to zero as k gets large.
That's what this limit means.
It just means the statement that we talked about before.
So the Fourier series, the theorem really says you can find all of the Fourier coefficients in this way, they all exist, they all exist exactly, they're all finite.
The function then exists as this limit in the mean, which is the thing that we're always interested in.
So, every L2 function defined over a finite interval, and every L1 function defined over a finite interval, all of these have a Fourier series which satisfies these two relationships.
Now, what we're going to do with all of this is we're going to segment an arbitrary function, an arbitrary L2 function over the entire time range, and we're going to segment it into pieces, all of width t. Then we're going to expand each of those segments into a Fourier series.
That's what people do any time they compress voice that I said several times.
When you compress voice -- for some reason or other everybody does it in 20 millisecond increments.
You chop up the voice into 20 millisecond increments.
You then, at least, conceptually look at those 20 millisecond increments, think of them in terms of the Fourier series, you expand them in the Fourier series.
So for each increment in time we get a different Fourier series, and we use those Fourier series to approximate the function.
So, each one of the increments now is going to be represented in this way here.
Since we're only looking at the function now over the interval around time, mt, we look at it this way, we calculate the coefficients, again, in terms of this rectangular function spaced off by m. This is saying the same thing that we said before, just moving it from minus t over 2 to t over 2 to t minus mt.
Here's minus t over 2.
The t over 2.
Next we're looking at t over 2 to 3t over 2.
Next we're looking at 5t over 2.
This is m equals zero.
This is m equals 1.
This is m equals 2.
This notation here you should get used to it, it will be confusing for a while.
All it means is that, and it works.
So, in fact, you can take all these terms, you can break them up this way.
So what you've really done is you've broken u of t into a double sub expansion.
These exponentials limited in time, that entire set of functions are talking to each other.
A function living in this interval of time and a function living in this interval of time have to be orthogonal, because I multiply this by this and I get zero at each time.
In one interval these exponentials are orthogonal to each other -- we've pointed that out before.
So we have a doubly exponential, we have a double sum of orthogonal functions, and what we're saying is that any old L2 function at all we can break up into this kind of sum here.
This is a very complicated way of saying what we think of physically anyway.
It says take a big long function, segment it into intervals of length t, break up each interval of length t into a Fourier series.
That's all that's saying.
So it's saying what is obvious.
We're going to find a number of such orthogonal expansions which work for arbitrary L2 functions.
As I said, it's a conceptual basis for voice compress algorithms.
Even more, next time we're going to go into the Fourier integral.
You think of the Fourier integral as being the right way to go from time to frequency, but, in fact, it's not really the right way to go from time to frequency.
When we think of voice or you think of the wave form of a symphony, for example, what going on there?
Over every little interval of time you hear various frequencies, right?
In fact, if you make t equal to the timing of the music, the idea becomes very, very clean because at each time somebody changes a note.
So you go from one frequency to another frequency, so it's a rather clean way of looking at this.
Our notion of frequency that we think of intuitively is much more closely associated with the idea of frequencies is changing in time then it is of frequencies being constant in time.
When we look at a Fourier integral, everything is frozen in time.
As soon as you take the Fourier integral, you've glopped everything from time minus infinity to time plus infinity all together.
Here when we look at these expansions, we're looking at these frequencies changing.
There's an unfortunate part about frequencies changing, and that is frequencies, unfortunately, live over the entire time interval.
These truncated frequencies work very nicely but they don't quite correspond to non-truncated time intervals, but it still does match our intuition and this is a useful way to think about functions that change in time.
If you believe what I've just said, why do people ever worry about the Fourier integral at all?
Well, you see the problem is this quantity t here that I'm segmenting over is unnatural.
If I look at voice, there's no t that you can take which is a natural quantity.
The 20 milliseconds is just something arbitrary that somebody did once and it worked.
Too many other engineers in the field said, well that works, I'll pick the same thing instead of trying to think through what a better number would be.
So they all use the same t. It doesn't correspond to anything physically.
So, in fact, people do try the same thing.
Let me just say what a Fourier transform is and we'll talk about it most of next time because that's the next thing we have to deal with.
Something you're all familiar with are these Fourier transform relationships.
There's the function of time, there's the function of frequency.
You can go from one to the other, mapping.
Well, if you know this you can find this.
If you know this, you can find this.
It looks a little bit like the Fourier series, and usually when people learn about the Fourier integral they start with a Fourier series and start letting t become large, which doesn't quite work.
But anyway, that's what we're going to do.
If the function is well-behaved, the first integral exists and the second exists.
What does well-behaved mean?
It means what we usually mean by well-behaved, it means it works.
So again, this theorem here is another example, or not, as the case may be.
But we'll make this clearer next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
--And go on with lectures 8 to 10.
First I want to briefly review what we said about measurable functions.
Again, I encourage you if you hate this material and you think it's only for mathematicians, please let me know.
I don't know whether it's appropriate to cover it in this class either.
I think it probably is because you need to know a little more mathematics when you deal with these Fourier transforms and Fourier series.
When you're dealing with communication theory, then you need to know for signal processing and things like that.
When you get into learning a little more, my sense at this point is it's much easier to learn a good deal more than it is to learn just a little bit more, because a little bit more you're always faced with all of these questions of what does this really mean.
It turns out that if you put just a little bit of measure theory into your thinking, it makes all these problems very much simpler.
So you remember we talked about what a measurable set was last time.
What a measurable set is, is a set which you can essentially break up into a countable union of intervals, and something which is bounded by a countble set of intervals which have arbitrarily small measure.
So you can take all of these sets of zero measure -- countable sets, cantor sets, all of these things, and you can just throw all of those out from consideration.
That's what makes measure theory useful and interesting and what simplifies things.
So we then said that a function is measurable if each of these sets here -- if the set of times for which a is less than or equal to u of t is less than or equal to b -- these things are these level sets here.
It's a set of values, t, along this axis in which the function is nailed between any two of these points here.
For a function to be measurable, all of these sets in here have to be measurable.
We haven't given you any examples of sets which are not measurable, and therefore, we haven't given you any examples of functions which are not measurable.
It's very, very difficult to construct such functions, and since it's so difficult to construct them, it's easier to just say that all of the functions we can think of are measurable.
That includes an awful lot more functions than you ever deal with in engineering.
If you ever make a serious mistake in your engineering work because of thinking that a function is measurable when it's not measurable, I will give you $1,000.
I make that promise that you because I don't think it's ever happened in history, I don't think it ever will happen in history, and that's just how sure I am of it.
Unless you try to deliberately make such a mistake in order to collect $1,000.
Of course, that's not fair.
Then we said the way we define this approximation to the integral, namely, we always defined it from underneath, and therefore, if we started to take these intervals here with epsilon and split them in half, the thing that happens is you always get extra components put in.
So that as you make the scaling finer and finer, what happens is the approximation into the integral gets larger and larger.
Because of that, if you're dealing with a non-negative function, the only two things that can happen, as you go to the limit of finer and finer scaling, is one, you come to a finite limit, and two, you come to an infinite limit.
Nothing else can happen.
You see that's remarkably simple as mathematics go.
This is dealing with all of these functions you can ever think of, and those are the only two things that can happen here.
So then we went on to say that a function as L1, if it's measurable, and if the integral of its magnitude is less than infinity, and so far we're still talking about real functions.
If u of t is L1, then you can take the integral of u of t and you can split it up into two things.
You can split it up into the set of times over which u of t is positive, and the set of times over which u of t is negative.
The set of times over which u of t is equal to zero doesn't contribute to the integral at all so you can forget about that.
This is well-defined as the function is measurable.
What's now happening is that if the integral of this magnitude is less than infinity, then both this has to be less than infinity, and this has to be less than infinity, which says that so long as you're dealing with L1 functions, you never have to worry about this nasty situation where the negative part of the function is infinite, the positive part of the function is infinite, and therefore, the difference, plus infinity, minus infinity doesn't make any sense at all.
And instead of having an integral which is infinite or finite, you have a function which might just be undefined completely.
Well this says that if the function is L1, you can't ever have any problem with integrals of this sort thing being undefined.
So, and in fact, u of t is always integral and has a finite value in this case.
Now, we say that a complex function is measurable if both the real part is measurable and the imaginary part is measurable.
Therefore, you don't have to worry about complex functions as really being any more than twice as complicated as real functions, and conceptually they aren't any more complicated at all because you just treat them as two separate functions, and everything you know about real functions you now know about complex functions.
Now, as far as Fourier theory is concerned -- Fourier series, Fourier integrals, discrete time, Fourier transforms -- all of these things, if u of t is an L1 function, then u of t times e to the 2 pi ft has to be L1, and this has to be true for all possible values of f. Why is that?
Well, it's just that the absolute value of u of t for any given t, is exactly the same as the absolute value of u of t times e to the 2 pi i ft. Namely, this is a quantity whose magnitude is always 1, and therefore, this quantity, this magnitude is equal to this magnitude.
Therefore, when you integrate this magnitude, you get the same thing as when you integrate this magnitude.
So, there's no problem here.
So what that says, using this idea, too, of positive and negative parts, it says that the integral of u of t times e to the 2 pi i ft dt has to exist for all real f. That covers Fourier series and Fourier integral.
We haven't talked about the Fourier integral yet, but it says that just by defining u of t to be L1, you avoid all of these problems of when these Fourier integrals exist and when they don't exist.
They always exist.
So let's talk about the Fourier transform then since we're backing into this slowly anyway.
What you've learned in probably two or three classes by now is that the Fourier transform is defined in this way.
Namely, the transform, a function of frequency, u hat of f, we use hats here instead of capital letters because we like to use capital letters for random variables.
So that this transform here is the integral from minus infinity to infinity of the original function, we start with u of t times e to the minus 2 pi ift dt.
Now, what's the nice part about this?
The nice part about this is that if u of t is L1, then in fact this Fourier transform has to exist everywhere.
Namely, what you've learned before in all of these classes says essentially that if these functions are well-behaved then these transforms exist.
What you've learned about well-behaved is completely circular, namely, a function is well-behaved if the transform exists, and the transform exists if the function is well-behaved.
You have no idea of what kinds of functions the Fourier transform exists and what kinds of functions it doesn't exist.
There's another thing buried in here, in this transform relationship.
Namely, the first thing is we're trying to define the transform function of frequency in terms of the function of time.
Well and good.
Then we try to define a function of time in terms of a function of frequency.
What's hidden here is if you start with u of t, you then get a function of frequency, and this sort of implicitly says that you get back again the same thing you started with.
That, in fact, is the most ticklish part of all of this.
So that if we start with an L1 function u of t, we wind up with a Fourier transform, u hat of f. Then u hat of f goes in here.
We hope we might get back the sum transform here.
The nasty thing, and one of the reasons we're going to get away from L1 in just a minute, is that if a function is L1, it's Fourier transform is not necessarily L1.
What that means is that you have to learn all of this stuff about L1 functions, and then as soon as you take the Fourier transform, bingo, it's all gone up in a lot of smoke, and you have to start all over again saying something about what properties the transform might have.
But anyway, it's nice to start with because when u of t is L1, we know that this function actually exists.
It actually exists as a complex number.
It exists as a complex number for every possible real f here.
Namely, there aren't any if, and's, but's or maybe's here, there's nothing like L2 convergence that we were talking about a little bit before.
This just exists, period.
There's something more here, that if this is L1, this not only exists but it's also a continuous function, and those don't prove that.
If you've taken a course in analysis and you know what a complex function is and you're quite patient, you can sit down and actually show this yourselves, but it's a bit of a pain.
So for a well-behaved function, the first integral exists for all f, the second exists for all t, and results in the original u of t. But then more specifically, if u of t is L1, the first integral exists for all f and it's a continuous function.
It's also a bounded function, as we'll see in a little bit.
If u hat of f is L1, the second integral exists for all t, and u of t is continuous.
Therefore, if you assume at the output at the onset of things that both this is L1 and this is L1, then everything is also continuous and you have a very nice theory, which doesn't apply to too many of the things that you're interested in.
And I'll explain why in a little bit.
Anyway, for these well-behaved functions, we have all of these relationships that I'm sure you've learned in whatever linear systems course you've taken.
Since you should know all of these things, I just want to briefly talk about them I mean this linearity idea is something you would just use without thinking about it, I think.
In other words, if you'd never learned this and you were trying to work out a problem you would just use it anyway, because anything respectable has to have -- well, anything respectable, again, means anything which has this property.
This conjugate property you can derive that easily from the Fourier transform relationships also, if you have a well-behaved function.
This quantity here, this duality, is particularly interesting because it isn't really duality, it's something called hermitian duality.
You start out with this formula here to go to there and you use almost the same formula to get back again.
The only difference is instead of a minus 2 pi ift, you have a plus 2 pi ift.
In other words, this is the conjugate of this, which is why this is called hermitian duality.
But aside from that, everything you learn about the Fourier transform, you also automatically know about the inverse Fourier transform for these well-behaved functions.
Otherwise you don't know whether the other one exists or not, and we'll certainly get into that.
So, the duality is expressed this way, the Fourier transform of, bleah.
If you take a function, u of t, and regard that as a Fourier transform, then -- I always have trouble saying this.
If you take a function, u of t, and then you regard that as a function of frequency -- OK, that's this -- and then you regard it as a function of minus frequency, namely, you substitute minus f for t in whatever time function you start with.
You start with a time function u of t, you substitute minus f for t, which gives you a function of frequency.
The inverse Fourier transform of that is what you get by taking the Fourier transform of u of t, and then substituting t for f in it.
It's much harder to say it than to do.
This time shift, you've all seen that.
If you shift a function in time, the only thing that happens is you get this rotating term in it.
Same thing for a frequency shift.
You want to have an interesting exercise, take time shift plus duality and derive the scaling and frequency from it -- it has to work, and of course, it does.
Scaling -- there's this relationship here.
This one is always a funny one.
It's a little strange because when you scale here, it's not too surprising that when you take a function and you squash it down that the Fourier transform gets squashed upwards.
Because in a sense what you're doing when you squash a function down you're making everything happen faster than it did before, which means that all the frequencies in it are going to get higher than they are before.
But also when you squash it down, the amount of energy in the function is going to go down.
One of the most important properties that we're going to find and what you ought to know already is that when you take the energy in a function, you get the same answer as you get when you take the energy in the Fourier transform.
Namely, you integrate u of f squared over frequency and you get the same as if you integrate u of t squared over time.
That's an important check that you use on all sorts of things.
The thing that happens now then when you're scaling is when you scale a function, u of t, you bring it down and you spread it out when you bring it down.
The frequency function goes up so the energy in the time function goes down, the energy in the frequency function goes up, and you need something in order to keep the energy relationship working properly.
This is what you need.
Actually, if you derive this, the t just falls out naturally.
So we get the same thing if we do scaling and frequency.
I don't think I put that down but it's the same relationship.
There's differentiation.
Differentiation we won't talk about or use it a whole lot.
All of these things turn out to be remarkably robust.
When you're dealing with L1 functions or L2 functions and you scale them or you shift them or do any of those things, if they're L1, they're still L1 after you're through, if they're L2, they're L2 after you're through.
If you differentiate a function, all those properties change.
You can't be sure of anything anymore.
There's convolution, which I'm sure you've derived many times, and which one of the exercises derives again.
There's correlation, and what this sort of relationship says is taking products at the frequency domain is the same as going through this convolution relationship and the time domain.
Of course, there's a dual relation to that, which you don't use very often but it still exists.
Correlation is you actually get correlation by using one of the conjugate properties on the convolution and I'm sure you've all seen that.
That's something you should all have been using and familiar with for a long time.
Two special cases of the Fourier transform is that u of zero is what happens when you take the Fourier transform and you evaluate it at t equals zero.
You get u of zero is just the integral of u hat of f. u hat of zero is just the integral of u of t. What do you use these for?
Well the thing I use them for is this half the time when you're working out a problem it's obvious by inspection what this integral is or it's obvious by inspection what this integral is, and by doing that you can check whether you have all the constants right in the transform that you've taken.
I don't know anybody who can take Fourier transforms without getting at least one constant wrong at least half the time they do it.
I probably get one constant wrong about three-quarters of the time that I do it, and I'm sure I'll do it here in class a number of times.
I hope you find it, but it's one of these things we all do.
This is one of the best ways of finding out what you've done and going back and checking it.
Parseval's Theorem, Parseval's Theorem is really just this convolution equation which we're applying at tau equals zero.
You take the convolution, you apply it at tau equals zero and what do you get?
You get the integral of u of t times some other conjugate -- the conjugate of the other function is equal to the integral of u hat of f times the complex conjugate of v of f. Much more important than this is what happens if v happens to be the same as u, and that gives you the energy equation here, which is what I was talking about.
It says the integral in a function you can find it two ways, either by looking at it in time or by looking at it in frequency.
I urge you to always think about doing this whenever you're working problems, because often the Fourier integral it's very easy to find the integral and the and the time function is very difficult.
A good example of this is sinc functions and rectangular functions.
Anybody can take a rectangular function, square it and integrate it.
It takes a good deal of skill if you don't use this relationship to take a sinc function, to square it and to integrate it.
You can do it if you're skillful at integration, you might regard it as a challenge, but after you get done you realize that you've really wasted a lot of time because this is the right way of doing it here.
Now, as I mentioned before, it's starting to look like Fourier series and Fourier integrals are much nicer when you have L1 functions, and they are, but L1 functions are not terribly useful as far as most communication functions go.
In other words, not enough functions are L1 to provide suitable models for the communication systems that we want to look at.
In fact, most of the models that we're going to look at, the functions that we're dealing with are L2 functions.
One of the reasons for this is this sinc function, sine x over x function is not L1.
A sinc function just goes down as 1 over t, and since it goes down as 1 over t, you take the absolute value of it and you integrate 1 over t and what do you get when you integrate it from minus infinity to infinity.
A function that's 1 over t, well, if you really want to go through the trouble of integrating it, you can integrate it over limits and you get limits where you have to evaluate the limits on a logarithmic function.
When you get all done with that you get an infinite value.
And you can see this.
You could take 1 over t and you just look at it, as you go further and further out it just gets bigger and bigger without limit.
So, sinc t is not L1.
Sinc function is a function we'd like to use.
Any function with a discontinuity can't be the Fourier transform of any L1 function.
In other words, we said that if you take the Fourier transform of an L1 function, one of the nice things about it is you get a continuous function.
One of the nasty things about it is you get a continuous function.
Since you get a continuous function it says that any time you want to deal with transforms which are not continuous, you can't be talking about time functions which are L1.
One of the frequency functions we want to look at a great deal is a frequency function corresponding to a band-limited function.
When you take a band-limited function you just chop it off at some frequency, and usually when you chop it off, you chop it off and get a discontinuity.
When you chop it off and get a discountinuity, bingo, the time function you're dealing with cannot be L1.
It has to dribble away much too slowly as time goes to infinity.
That's an extraordinarily important thing to remember.
Any time you get a function which is discontinuous in the frequency domain, the function cannot go to zero any faster in a time domain than 1 over t and vice versa in the frequency domain.
L1 functions sometimes have infinite energy.
In other words, sinc t is not L1 -- well, that's not a good example because that's not L1, and it also has infinite energy, but you can just as easily find functions which drop off a little more slowly than sinc t, and which have infinite energy because they--.
Excuse me.
Sometimes you have functions which go off to infinity too fast as you approach time equals zero, things which are a little bit like impulses but not really.
Impulses are awful and we'll talk about them in a minute, because they don't have finite energy as we said before.
We have functions which just slowly go off to infinity and they are L2 but they aren't L1.
Why do we care about those weird functions?
We care about them, as I said before, because we would like to be able to make statements which are simple which we can believe in.
In other words, you don't want to go through a course like this with a whole bunch of things that you have to leave.
It's very nice to have some things that you really believe, and whether you believe them or not, it's nice to have theorems about them.
Even if you don't believe the theorems, at least you have theorems so you can fool other people about it, if nothing else.
Well, it turns out that L2 functions are really the right class to look at here.
Oh, I think I left out one of the most important things here.
Maybe I put it down here.
No, probably not.
One of the reasons we want to deal with L2 functions is if you're dealing with compression, for example, and you take a function, if the function has infinite energy in it, then one of the things that happens is that any time you expand it into any kind of orthonormal expansion or orthongonal expansion, which we'll talk about later, you have coefficients, which have infinite energy.
In other words, they have infinite values, or the sum squared of the coefficients is equal to infinity.
When we try to compress those we're going to find that no matter how we do it our mean square error is going to be infinite.
Yes, we will talk about that later when we get to talking about expansions.
So for all those reasons L1 isn't the right thing, L2 is the right thing.
A function going from the reals into the complexes, in other words, a complex valued function is L2 if it's measurable, and if this integral is less than infinity, in other words, if that has finite energy.
Primarily it means it has finite energy because all the functions you can think of are measurable.
So it really says you're dealing with functions that have finite energy here.
So, let's go on to Fourier transforms then.
Interesting simple theorem.
I think I stated this last time, also.
If a function is L2 and its time limited, it's also L1.
So we've already found that if functions are L1, they have Fourier transforms that exist.
The reason for this is if you square u of t, take the magnitude squared of u of t for any given t, it has to be less than or equal to the sum of u of t plus 1.
In fact, it has to be less than that.
Why?
How would you prove this if you had to prove it?
Well, you say gee, this is two separate terms here.
Why don't I look at two separate cases.
The two separate cases are first, suppose u of t itself as a magnitude less than 1.
If u of t has and magnitude less than 1 and you square it, you get something even smaller.
So, any time u of t has a magnitude less than 1, u squared of t is less than or equal to u of t. Blah blah blah blah blah blah blah blah.
If u of t -- what did I do here?
No wonder I couldn't explain this to you.
Let's try it that way and see if it works.
If you can prove something, turn it around and see if you can prove it then.
Now, two cases.
First one, let's suppose that u of t is less than or equal to 1.
Well then, u of t is less than or equal to 1.
And this is positive so this is less than or equal to that.
Let's look at the other case.
u of t is greater than or equal to 1, magnitude.
Well then, it's less than or equal to u of t magnitude squared.
So either way this is less than or equal to that.
What that says is the integral over any finite limits of u of t dt is less than or equal to the integral of this.
Well, the integral of this splits up into the integral magnitude u of t squared dt plus the integral of 1.
Now, the integral of 1 over any finite limits is just b minus a.
That's where the finite limits come in.
Finite limits say you don't have to worry about this term because it's finite.
So that says at any time you have an L2 function over a finite range, that function is also L1 over that finite range.
Which says that any time you take a Fourier transform of an L2 function, which is only non-zero over a finite range, bingo, it's L1 and you get all these nice properties.
It has to exist, it has to be continuous, it has to be bounded, and all of that neat stuff.
So for any L2 function u of t, what I'm going to try to do now, and I'm just copying what a guy by the name of Plancherel did a long time ago.
The thing that Plancherel did was he said how do I know when a Fourier transform exists or not.
I would like to make it exist for L2 functions, how do I do it?
Well, he said OK, the thing I'm going to do is to take the function u of t and I'm first going to truncate it.
In fact, if you think in terms of Reimann integration and things like that, any time you take an integral from minus infinity to plus infinity, what do you mean by it?
You mean the limit as you integrate the function over finite limits and then you let the limits go to infinity.
So all we're doing is the same trick here.
So we're going to take u of t, we're going to truncate it to some minus a to plus a over some finite range, no matter how big a happens to be.
We're going to call the function b sub a of t, u of t truncated to these limits.
In other words, u of t times a rectangular function evaluated at t over 2a.
Now, can all of you look at this function and see that it just means truncate from minus a to plus a?
No.
Well, you should learn to do this.
One of the ways to do it is to say OK, the rectangular function is defined as having the value 1 between minus 1/2 and plus 1/2 and it's zero everywhere else.
I think I said that before in class, didn't I?
Certainly it's in the notes.
Rectangle ft equals 1 for t less than or equal to 1/2, zero else.
So with this definition you just evaluate what happens when t is equal to minus a, you get rectangle of minus a over 2a, which is minus 1/2.
When t is equal to a, you're up to the other limit, so this function is 1 for t between minus a and plus a and zero everywhere else.
Please get used to using this and become a little facile at sorting out what it means because it's a very handy way to avoid writing awkward things like this.
So va of t by what we've said is both L2 and its L1.
We started out with a function which is L2 and we truncated.
Then according to this theorem here, this function va of t is now time limited.
It's also L2, and therefore, by the theorem it's also L1.
Therefore, it's continue and you can take the Fourier transform of it -- v hat a of f exists for all f and it's continuous.
So this function is just the Fourier transform that you get when you truncate the function, which is what you would think of as a way to find the Fourier transform anyway.
I mean if this is not a reasonable approximation to the Fourier transform a function, you haven't modeled the function very well.
Because when a gets extraordinarily large, if there's anything of significance that happens before year ten to the minus 6 or which happens after year ten to the plus 6, and you're dealing with electronic speeds, your models don't make any sense.
So for anything of any interest, these functions here are going to start approximating u of t, and therefore we hope this will start approximating the Fourier transform of u of t. Who can make that more precise for me?
What happens when u of t has finite energy?
If it has finite energy it means that the integral of u of t magnitude squared over the infinite interval is finite.
So you start integrating it over bigger and bigger minus a to plus a.
What happens is as the minus a to plus a gets bigger and bigger and bigger, you're including more and more of the function, so that the integral of u of t squared over that bigger and bigger interval has to be getting closer and closer to the overall integral of u of t. Which says that the energy in u of t minus the a of t has to get very, very small as a gets large.
That's one of the reasons why we like to deal with finite energy functions.
By definition they cannot have a appreciable energy outside of very large limits.
How large those large limits have to be depends on the function, but if you make them large enough you will always get negligible energy outside of those limits.
So then we can take the Fourier transform of this function within those limits and we get something which we hope is going to be a reasonable approximation of the Fourier transform of u of t. That's what Plancherel said.
Plancherel said if we have an L2 function, u of t, then there is an L2 function u hat of f, which is really the Fourier transform of u of t. Some people call this the Plancherel transform of u of t and say that indeed Plancherel was the one that invented Fourier transforms or Plancherel transforms for L2 functions.
That's probably giving him a little too much credit, and Fourier somewhat less than due credit.
But it was a neat theorem.
What he said is that there is a function u hat of f, which we'll call the Fourier transform, which has the property that when you take the integral of the difference between u hat of f and the transform of b sub a of t, when you take the integral of this dt -- in other words, when you evaluate the energy in the difference between u hat of f and v sub a of f, that goes to zero.
Well this isn't a big deal.
In other words, this is plausible, since this integral has to go to zero for an L2 function, that's what we just said, then therefore, using the energy relation, this also has to go to zero.
So is this another example where Plancherel just came along at the right time and he said something totally trivial and became famous because of it?
I mean as I've urged all of you, work on problems that other people haven't worked on.
If you're lucky, you will do something trivial and become famous.
As another piece of philosophy, you become far more famous for doing something trivial than for doing something difficult, because everybody remembers something trivial.
And if you do something difficult nobody even understands it.
But no, it wasn't that because there's something hidden here.
He says a function like this exists.
In other words, the problem is these functions get closer and closer to something.
They get closer and closer to each other as a gets bigger and bigger.
You can show that because you have a handle on these functions.
Whether they get closer and closer to a real bonafide function or not is another question.
Back when you studied arithmetic, if you were in any kind of advanced class studying arithmetic, you studied the rational numbers and the real numbers you remember.
And you remember the problem of what happens if you take a sequence of rational numbers which is approaching a limit.
There's a big problem there because when you take a sequence of rational numbers that approaches a limit, the limit might not be rational.
In other words, when you take sequences of things you can get out of the domain of the things you're working with.
Now, we can't get out of the domain of being L2, but we might get out of domain of measurable functions, we might get out of the domain of functions at all.
We can have all sorts of strange things happen.
The nice thing here, which was really a theorem by [?
Resenage ?]
a long time ago.
It says that when you take cosine sequences of L2 functions, they converge to an L2 function.
So that's really what's involved in here.
So maybe this should be called the [?
Resenage ?]
transform, I don't know.
But anyway, whatever this says, the theorem says, the first part of Plancherel's theorem says that this function exists and you get a handle on it by taking this transform, making a bigger and bigger, and it says it will converge to something in this energy sense.
Bingo, when you're all done this goes to zero.
We're going to denote this function as a limit and a mean of the Fourier transform.
In other words, we do have a Fourier transform in the same sense that we had a Fourier series before.
We didn't know weather the Fourier series would converge at every point, but we knew that it converged at enough points, namely, almost everywhere, everywhere but on a set of measure zero.
It converges so that, in fact, you get this kind of relationship.
Now, do you have to worry about that?
No.
Again, this is one of these very nice things that says there is a Fourier transform, you don't have to worry about what goes on at these oddball sets where the function has discountinuities and things like that.
You can forget all of that.
You can be as careless as you've ever been, and now you know that it all works out mathematically, so long as you stick to L2 functions.
So, sticking to L2 functions says you can be a careless engineer, you can use lousy mathematics and you'll always get the right answer.
So, it's nice for engineers, it's nice for me.
I don't like to be careful all the time.
I like to be careful once and then solve that and go on.
Well, because of time frequency duality, you can do exactly the same thing in the frequency domain.
So, you start out defining some b, which is bigger than zero which is arbitrarily large, you define a finite bandwidth approximation as w hat sub b of f is u hat of f. We now know that u hat of f exists and it's an L2 function.
u hat of f times this rectangular function, that's f over 2b.
In other words, it's u hat of f truncated to a big bandwidth minus b to plus b.
Since w sub b of f is L1, as well as L2, this always exists.
So long as you deal with a finite bandwidth, this quantity exists.
It exists for all t and r. It's continuous.
The second part of Plancherel's theorem then says that the limit as b goes to infinity of the integral of u of t minus this truncated function, magnitude squared the energy in that, goes to zero.
This now is a little different than what we did before.
It's easier in the sense that we don't have to worry about the existence of this function because we started out with this to start with.
It's a little harder because we know that a function exists, but we don't know that it's u of t. So, in fact, poor old Plancherel had to do something other than just this very simple argument that says all the energy works out right.
He had to also show that you really wind up with the right function when you get all through with it.
But again, this is the same kind of energy convergence that we had before.
Yeah?
Could you discuss non-uniqueness?
Clearly, [INAUDIBLE] u hat f to satisfy Plancherel 1 and Plancherel 2
Yeah, in fact, any two functions which are L2 equivalent.
But you see the nice thing is when you take this finite bandwidth approximation there's only one.
It's only when you get to the limit that all of this mess occurs.
If you take these different possible functions, u hat of f, which just differ in these negligible sets of measure zero, those don't affect this integral.
Sets of measure zero don't affect integrals at all.
So the mathematicians deal with L2 theory by talking about equivalence classes of functions.
I find it hard to think about equivalence classes of functions and partitioning the set of all functions into a bunch of equivalence classes.
So I just sort of remember in the back of my mind that there are all these functions which differ in a strange way.
We'll talk about that more when we get to the sampling theorem later today, because there it happens to be important.
Here it's not really important, here we don't have to worry about it.
Anyway, we can always get back to the u of t that we started with in this way.
Now, this says that all L2 functions have Fourier transforms in this very nice sense.
In other words, at this point you don't have to worry about continuity, you don't have to worry about how fast things drop off, you don't have to worry about anything.
So long as you have finite energy functions, this beautiful result always holds true.
There always is a Fourier transform, it always has this nice property that it has the same energy as the function you started with.
The only nasty thing, as Dave pointed out, is that, in fact, it might not be a unique function, but it's close enough to unique.
It's unique in an engineering sense.
The other thing is that L2 wave forms don't include some of your favorite wave forms.
They don't include constants, they don't include sine waves, and they don't include Dirac impulse functions.
All of them have infinite energy.
I pointed out in class, spent quite a bit of time explaining why an impulse function had infinite energy by looking at it as a very narrow pulse of width epsilon and a height 1 over epsilon, and showing that the energy in that is 1 over epsilon, and as epsilon goes to zero and the pulse gets narrower and narrower, bingo, the energy goes to infinity.
Constants are the same way, they extend on and on forever.
Therefore, they have infinite energy, except if the constant happens to be zero.
Sine waves are the same way, they dribble on forever.
So the question is are these good models of reality?
The answer is they're good for some things and they're very bad for other things.
The point in this course is that if you're looking for wave forms that are good models for either the kinds of functions that we're going to quantize, namely, source wave forms, or if you're looking for the kinds of things that we're going to transmit on channels, these are very lousy functions.
They don't make any sense in a communication context.
But anyway, where did these things come from?
Constants and sine waves result from refusing to explicitly model when very long-lasting functions terminate.
In other words, if you're looking at a carrier function in a communication's system, sine of 2 pi, f carrier times t, it just keeps on wiggling around forever.
Since you want to talk about that over the complete time of interest, you don't want to say what the time of interest is, you don't want to admit to your employer that this thing is going to stop working after one month because you want to let him think that he's going to make a profit off of this forever, and you don't want to commit to putting it into use in one month when you know it's going to get delayed for a whole year.
So you want to think of this as something which is permanent.
You don't want to answer the question at what time does it start and at what time does it end, because for many of the questions you ask, you can just regard it as going on forever.
You have the same thing with impulses.
Impulses are always models of short pulses.
If you put these short pulses through a filter, the only thing which is of interest in them is what their integral is.
And since the only thing of interest is their integral, you call it an impulse and you don't worry about just how narrow it is, except that it has infinite energy and, therefore, whenever you start to deal with a situation in which energy is important, these becomes lousy models.
So we can't use these when we're talking about L2 functions.
That's the price we pay for dealing with L2 functions.
But it's a small price because for almost all the things we'll be dealing with, it's the energy of the functions that are really important.
So as communication wave forms, infinite energy wave forms make mean square error quantization results meaningless.
In other words, when you sample these infinite energy wave forms you get results that don't make any sense, and they make those channel results meaningless.
Therefore, from now on, whether I remember to say it or not, everything we deal with, unless we're looking at counter examples to something, is going to be an L2 function.
Let's go on.
I'm starting to feel like I'm back in our signals and systems course, because at this point I'm defining my third different kind of transform.
Fortunately, this is the last transform we will have to talk about, so we're all done with this.
The other nice thing is that the discrete time Fourier transform happens to be just the time frequency dual of the Fourier series.
So that whether you've ever studied the dtft or not, you already know everything there is to know about it, because the only things there are to know about it are the results about Fourier series.
So the theorem is really the same theorem that we had for Fourier series.
Assume that you have a function of frequency, u hat of f -- before we had a function of time, now we have a function of frequency.
Suppose it's defined over the interval minus w to plus w into c. In other words, a way we often say that a function is truncated is to say it goes from some interval into c. This is a complex function which is non-zero only for f in this finite bandwidth range.
We want to assume that this is L2 and thus, we know it's also L1.
Then we're going to take the Fourier coefficients.
Before we thought of the Fourier coefficients as corresponding to what goes on at different frequencies, now we're going to regard them as time quantities, and we'll see exactly why later.
So we'll define these as the Fourier coefficients of this function.
So they're 1 over 2w times this integral here.
You remember before when we dealt with the Fourier series, we went from minus t over 2 to plus t over 2.
Now we're going from minus w to plus w. Why?
It's just convention, there's no real reason.
So that what's happening here is that this is a Fourier series formula for a coefficient where we're substituting w for t over 2.
We're putting a plus sign in the exponent instead of a minus sign.
In other words, we're doing this hermitian duality bit.
What's the other difference?
We're interchanging time and frequency.
Aside from that it's exactly the same theorem that we established -- well, that we stated before.
So we know that this quantity here, since u hat of f is L1, this is finite and it exists -- that's just a finite complex number and nothing more -- for all integer k. Also, the convergence result when we go back, this is the formula we try to use to go back to the function we started with.
It's just a finite approximation to the Fourier series.
We're saying that as k zero gets larger and larger, this finite approximation approaches the function in energy sense.
So this is exactly what you should mean by a Fourier series anyway.
It's exactly what you should mean by a Fourier transform anyway.
As you go to the limit with more and more terms you get something which is equal to what you started with, except on the set of measure zero.
In other words, it converges everywhere where it matters.
It converges to something in the sense that the energy is the same.
I said that in such a way that makes it a little simpler than it really is.
You can't always say that this converges in any nice way.
Next time I'm going to show you a truly awful function which we'll use in the Fourier series instead of dtft, which is time limited and which is just incredibly messy and it'll show you why you have to be a little bit careful about stating these results.
But you don't have to worry about it most of the time, because this theorem is still true.
It's just that you have to be a little careful about how to interpret it because you don't know whether this is going to reach a limit or not.
All you know is that this will be true, this energy difference goes to zero.
Also, the energy in the function is equal to the energy in the coefficients.
This was the thing that we found so useful with the Fourier series.
It's why we can play this game that we play with mean square quantization error of taking a function and then turning it into a sequence of samples, trying to quantize the samples for minimum mean square error and associate the mean square error in the samples with the mean square error on the function.
You can't do that with anything that I know of other than mean square error.
If you want to deal with other kinds of quantization errors, you have a real problem going from coefficients to functions.
And finally, for any set of numbers u sub k, if the sum is less than infinity, in other words, if you're dealing with a sequence of finite energy, there always is such a frequency function.
Many people when they use the discrete time Fourier transform think of starting with the sequence and taking a sequence and saying well it's nice to think of this sequence in the frequency domain, and then they say a function f exists such that this is true, or they just say that this is equal to that without worrying about the convergence at all, which is more common.
But since we've already gone through all of this for the Fourier series, we might as well say it right here also.
So there's really nothing different here.
But now the question is why do these u of k's -- I mean why do we think of those as time coefficients?
I mean what's really going on in this discrete time Fourier transform.
At this point it just looks like a lot of mathematics and it's hard to interpret what any of these things mean.
Well, the next thing I want to do is to go into the sampling theorem.
The sampling theorem, in fact, is going to interpret for you what this discrete time Fourier transform is, because the sampling theorem and the discrete time Fourier transform are just intimately related, they're hand and glove with each other, and that's the next thing we want to do.
But first we have to re-write this a little bit.
We're going to say that this frequency function is the limit in the mean of this -- this rectangular function is what we use just to make sure we're only talking about frequency between minus w and plus w. The limit in the mean, there's a little notational trick that we use so that we can think of this as just a limit instead of thinking of it as this crazy thing that we just derive, which is really not so crazy.
That means we can talk about this transform without always rubbing our noses in all of this mess here.
It just means that once in awhile we go back and think what does this really mean.
It means convergence in energy rather than convergence point-wise, because we might not have convergence point-wise.
So we're going to write this also as the limit in the mean of the sum over k of u of k. We're going to glop all of this together.
This is just some function of k and a frequency.
So we're going to call that phi sub k of f at some wave form, and the wave form is this.
What happens if you look at the relationship between to phi k of f and phi k prime of f?
Namely, if you look at two different functions.
These two functions are orthongonal to each other for the same reason that the functions that the sinusoid you used in the Fourier series are orthongonal.
Namely, you take this function, you multiply it by e to the minus 2 pi ik prime f over 2w times this rectangular function.
You integrate from minus w to w and what do you get?
You're just integrating a sinusoid -- the whole thing is one big sinusoid -- over one period of that sinusoid or multiple periods of the sinusoid.
Actually, k minus k prime periods of the sinusoid.
And when you integrate a sinusoid over a period, what do you get?
You get zero.
So it says that these functions are all orthongonal to each other.
So, presto, we have another orthongonal expansion just like the Fourier series gave us an orthongonal expansion.
And in fact, it's the same orthongonal expansion.
Now the next thing to observe is that we have done the Fourier transform and we've also done the discrete time Fourier transform.
In both of them we're dealing with some frequency function.
Now we're dealing with some frequency function which is limited to minus w to plus w, but we have two expansions for it.
We have the Fourier transform, so we can go to a function u of t, and we also have this discrete time Fourier transform.
So u of t is equal to this Fourier transform here.
Again, I should write limit in the mean here, but then I think about and I say do I have to write limit in the mean?
No.
I don't need a limit in the mean here because I'm integrating this over finite limits.
Since I'm taking a function over finite limits, u hat of f is over these limits, is an L1 function, therefore, this integral exists.
This function is a continuous function.
There aren't any sets of measure zero involved here.
This is one specific function which is always the same.
You know what it is exactly at every point.
At every point t, this converges.
So then what we're going to do, what the sampling theorem does is it relates this to what you get with a dtft.
So the sampling theorem says let u hat of f be an L2 function which goes from minus ww to c, and let u of t be this, namely, that, which we now know exists and is continuous.
Define capital T as 1 over 2w.
You don't have to do that if you don't want to, but it's a little easier to think in terms of some increment of time, T, here.
Then u of t is continuous, L2 and bounded.
It's bounded by u of t less than or equal to this.
Why is that?
It doesn't make any sense as I stated it.
Now it makes sense.
OK, its magnitude is bounded.
Its magnitude is bounded because if you take u of t magnitude, it's equal to the magnitude of this which is less than or equal to the integral of the magnitude of this, which is equal to the integral of the magnitude of just u hat of f, which is what we have here.
So all of that works nicely.
So, u of t is a nice, well-defined function.
Then the other part of it is that u of t is equal to the sum if its values at these sample points times sinc of t minus kt over t. Now you've probably seen this sampling theorem before.
How many people haven't seen this before?
I mean aside from the question of trying to do it -- do it in a way that makes sense.
OK, you've all seen it.
So good.
What it's saying is you can represent a function in terms of just knowing what its samples are.
Or you can take the function, you can sample it, and when you sample it if you put these little sinc hats around all the samples, you get back to the function again.
So you take all the samples, you then put these sincs around them, add them all up, and bingo, you got the function.
Let's see why that's true.
Here's the sinc function here.
The important thing about sinc t, sine pi t over pi t, which you can see by just looking at the sine function, is it has the value 1 when t is equal to zero.
I mean to get that value 1, you really have to go through a limiting operation here to think of sine pi t when t is very small as being approximately equal to pi t. When you divide pi t by pi t you get 1, so that's its value there and value around there.
At every other sample point, namely, at t equals 1, sine of pi t is zero.
At t equals 2, sine of pi t is equal to zero.
So the sinc function is 1 at zero and is zero at every other integer point.
Now to see why it's true and to understand what the dtft is all about, note that we have said that a frequency function, u hat of f, can be expressed as the sum over k -- and I should use a limit in the mean here but I'm not -- of uk times this transform relationship here.
Well, these are these functions that we talked about awhile ago.
They're these functions.
They're the sinusoids, periodic sinusoids in k truncated in frequency.
So we know that u hat of f is equal to that dtft expansion.
If I take the inverse Fourier transform of that, I can take the inverse Fourier transform of all these functions and I'll get u of t is equal to the sum over k, of uk pk of t. I'm being careless about the mathematics here.
I've been careful about it all along.
The notes does it carefully, particularly in the appendix.
I'm not going to worry about that here.
If I take the function pk of f, which is this truncated sinusoid, and I take the inverse transform of that -- take the transform of this, I get this.
Can you see that just by inspection?
If you were really hot on these things, if you just finished taking 6.003, you could probably see that by inspection.
If you remember all of those relationships that we went through before, you can see it by inspection.
The Fourier transform of erect function is a sinc function.
This exponential here when you go into the time domain corresponds to a time shift, so that gives rise to this time shift here.
The 1 over t is just one of these constants you have to keep straight, and which I would do just by integrating the things to see what I get.
Finally, u of kt, if I look at this, is just 1 over t times u of k, because these functions here are these sinc functions, which are zero everywhere but on their own point.
So if I look at u of kt, it's a sum over k of ck of kt, and ck of kt is only 1 when little t is equal to k times capital T, and therefore, I get that point there.
Therefore, u of kt is just 1 over t times u of k. That finishes the sampling theorem except for really tracing through all of these things about convergence, but it also tells you what the discrete time Fourier transform is.
Because the discrete time Fourier transform is just scaled samples of u of t. In other words, you start out with this frequency function, you take the inverse Fourier transform of it, you get a time function.
You take the samples of that, you scale them, and those are the coefficients in the discrete time Fourier transform, which is what you use discrete time Fourier transforms for.
You think of sampling of function, then you represent the function in terms of those samples and then you want to go into the frequency domain as a way of dealing with the properties of those samples, and all you're doing is just going into the Fourier transform of u of t, and all of that works out.
There's one bizarre thing here, and I'm going to talk about that more next time.
That is that when you look at this time frequency limited function, u hat of f, u hat of f can be very badly behaved.
You can have a frequency limited function which does all sorts of crazy things.
Since it's frequency limited as inverse transform, it's beautifully behaved.
It's just the sum of sinc functions -- it's bounded, it's continuous and everything else.
When we go back into the frequency domain it is just as ugly as can be.
So what we have in the sampling theorem, it comes out particularly clearly.
Is that L1 functions in one domain are nice, continuous, beautiful functions which are not L1 in the other domain.
So you sort of go from L1 to continuous.
Now when we're dealing with L2 functions, they're not continuous, they're not anything else, but you always go from L2 to L2.
In other words, you can't get out of the L2 domain, and therefore, when you're dealing with L2 functions, all you have to worry about is L2 functions, because you always stay there no matter what.
When we start talking about stochastic processes, we'll find out you always stay there also.
In other words, you only have to know about one thing.
We've seen here that to interpret what these Fourier transforms mean, it's nice to have a little idea about L1 functions also because when we think of going to the limit, when we go to the limit we get something which is badly behaved, and for these finite time and finite frequency approximations, we have things which are beautifully behaved.
I'm going to stop now so we can pass out the quizzes.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We started to talk about the discrete time Fourier transform last time.
This is probably something you've been exposed to before, and as in many cases, we're looking at it in a very different way than what you probably looked at it before.
The discrete time Fourier transform is simply the time frequency dual of the Fourier series.
Nothing more than that.
For any L2 function -- now we're talking about a function of frequency, but a function is a function.
So this is a complex valued function of frequency.
If it's limited, if it's truncated to the band from minus w to plus w, then, in fact, it's inverse Fourier transform is given by this limit in the mean of the sum of the coefficients times v sub k of f. This is exactly the same as the Fourier transform replacing times with frequencies, replacing w for t over 2, and in the complex frequency you're replacing a to the plus 2 pi i here with e to the minus 2 pi i in the Fourier series.
So those are the only differences, it's just notational differences, and aside from that, it works exactly the same way.
The coefficients are given by this.
We showed, when we were talking about the Fourier series, that these coefficients exist as complex numbers, they're always finite.
You can calculate them if you want to.
This quantity here can be rather fishy.
This is this limit in the mean which says that you have to calculate this by looking at the sum here over a finite sum, over a finite sum this is well-defined and behaves very nicely.
As you go to the limit funny things can happen, but the thing that we showed is in terms of energy, nothing funny can happen.
I'm going to give you an example of this kind of funny business as we talk about the sampling theorem in just a couple of minutes.
So the u hat of f has to be L1 since it is limited in this way.
It has a continuous inverse transform, which is this.
OK, so you can go from -- blah blah blah blah blah.
The discrete Fourier transform is simply a transform between a function and a sequence of terms.
Now this bit here is something we haven't talked about before.
Because now we've talked about Fourier transforms also, and if you have an L2 function, u hat of f, that L2 function has a Fourier transform and an inverse Fourier transform in this case, which is u of t, which is given by this expression here, the usual expression for the Fourier transform.
Now, the thing that's peculiar about this Fourier transform is that since u hat of f is limited, is truncated, this transform here exists everywhere.
We don't need a limit in the mean here.
You can calculate this for every t this exists.
This is a point-wise convergent thing.
This is the thing that you're used to when you think of functions, because the function is defined everywhere in this case.
And since we're going to go into the sampling theorem, it had damned well better be defined everywhere because otherwise the sampling theorem wouldn't make any sense at all.
So, the inverse Fourier transform of the discrete time Fourier transform, as we said, is this, it's a limit in the mean.
Namely, you take more and more terms here.
You get closer and closer to this in terms of energy.
It doesn't say anything about what happens for particular values of f. This is a sampling expansion with t equals 1 over 2w.
OK, let's go back here.
We're talking about some function of frequency which has a Fourier series, it also has a Fourier transform.
What we're interested in now is what is the relationship between these coefficients here and a discrete time Fourier transform and this function here.
What I want to show you is that, in fact, these two things are very closely related.
You can now go back to the Fourier series itself and relate the Fourier series coefficients to the Fourier transform and you'll get the same kind of sampling representation that we're getting right now.
So, if we -- there's something missing in what I'm trying to say here.
Oh, I see what I'm trying to do, sorry.
What I'm trying to do is to take the inverse Fourier transform of u hat of f, which is given by this expression here.
Temporarily I'm going to forget about the fact that this is a limit in the mean, throw mathematics to the winds, and simply take this inverse transform.
Take the inverse transform of this also.
So, when I take the inverse transform of the sum, what I'm going to get is the sum over k of u sub k, and then in place of this frequency function -- these are orthogonal functions here, the things I listed on the previous page -- I got u of t is the sum of u sub k times these time functions now.
These time functions are just the Fourier transforms of these frequency functions here.
This is a set of orthogonal wave forms here, which are truncated sinusoids.
I want to take the Fourier transform of these and the Fourier transforms of these -- I should have put them both on the same slide so you could see what they are -- but in fact, in your homework you're going to take the inverse Fourier transform of that and show that it is, in fact, this.
That's just a nice exercise in taking rectangular functions and sinc functions and applying shifts in both time and frequency.
When you do this, this turns out to have the inverse transform of this.
So this is u of t is just this, where this comes from here, and then this turns out to be the inverse transform of this function here, which, in fact, is just this.
Sorry for all of that.
So now if we want to try to understand what this means, suppose that you take a function u of t, which is, in fact, the inverse transform of this u hat of f, which is truncated to a finite band limit.
If we then take v of t equal to u of t everywhere except on the sample points, and on every sample point we simply add 1 to v -- in other words, v of kt is equal to u of kT plus 1.
So we take this nice smooth function that we have here, and at every sample point we simply add 1.
So now the question is, is this new function I have still baseband limited or isn't it?
You see you can't answer that question because we weren't careful enough to say what we meant by a baseband limited function.
Usually when you talk about a baseband limited function, you're talking about a function whose Fourier transform is zero, except in the range minus w to plus w. Well, this new function v of t here has the property that its Fourier transform is zero outside of minus w to plus w. And therefore if you define baseband limited as functions whose Fourier transform is zero outside of limits, then the sampling theorem doesn't hold.
So what do we do about this?
Well, the easiest thing to do about it is to change what we mean by baseband limited to what you would have meant if we hadn't going through all of this mathematics.
In other words, the Fourier transform cuts both ways -- u of t has a Fourier transform, u hat of f, u hat of f has an inverse transform, u of t. What we mean now by baseband limited is that u of t is the inverse transform of a frequency function, which is limited to minus w over plus w. In other words, we will regard u of t here as being baseband limited, we will not regard v of t as being baseband limited.
Because we have these two functions, u of t and v of t, which both have the same Fourier transforms, but which are not equal to each other.
We want to, at this point, say that the only one which is really baseband limited is the function which is the inverse transform of u of t. If you read the sampling theorem as it's written in the notes, that's exactly what it says.
It defines this function u of t to which the sampling theorem applies in that way.
Well anyway, this is the sampling theorem at this point.
It says you can take u of t, you can express it in this way.
If I go back to the previous slide, what we did here by comparing u of t with this expression for u sub k. u sub k are the coefficients in the discrete time Fourier transform.
u of t is this inverse transform.
These quantities here are almost the same.
This quantity here is just evaluating this at particular frequencies.
Namely, if for frequency f I substitute in, if we're time t, I substitute in k over 2w, then this formula just becomes that formula.
In other words, 2w times u sub k is equal to u evaluated at k over 2w.
You already saw that when you looked at the Fourier series.
When you looked at the Fourier series, you saw these coefficients, which, in fact, look like the Fourier transform terms.
And which, in fact, were the same as the Fourier transform except for a scale factor at some particular frequency.
Here we have these coefficients, which are, in fact, scaled versions of the inverse transform of particular times.
So the conclusion from that is that, in fact, what the discrete time Fourier transform is is it's simply the Fourier transform of the sampling theorem expansion.
The two of them are duals in a very different way than the Fourier series and the dtft are duals.
The dtft and the Fourier series are duals in the sense that if you take the expressions for the Fourier series and change frequency for time and time for frequency, you get to the dtft.
Here what we're doing is taking the dtft and simply taking the inverse Fourier transform of it, so that the sampling theorem is, in fact, the Fourier transform of the dtft.
It's not the dual, it's the Fourier transform itself.
Well, the discrete time Fourier transform generalizes to arbitrary frequency intervals just as well as to a baseband interval.
Namely, you can do exactly the same thing as what we've just done if you're looking not at the frequency at the range of frequencies from minus w to plus w, but you shift that up to any old place you want to and look at delta minus w to delta plus w. And the fact the discrete time Fourier transform, if you don't put the rectangular function in it is going to be periodic anyway.
It's exactly the same thing that we had with a Fourier series.
With a Fourier series, we could find the Fourier series for a function limited between minus t over 2 to plus t over 2.
Now by duality, we can find the dtft for a function limited between minus w plus delta and plus w plus delta.
That's what we're doing here.
So the dtft in generalized form is now this, and v sub k is now the integral from delta minus w to delta plus w of the same old thing as before.
This is equal to this, which is the same old thing as before, except we now have this shifted frequency in the rectangular function.
So if we take the inverse Fourier transform of this, again, using the same duality we had before, we get v of t is equal to the sum times the sinc function.
And the only difference now that we're expanding this given frequency band not centered around zero but centered around something else, the only difference in the sampling theorem is now we have this rotating term gyrating around up at this frequency, k over 2w.
That's the only way in which this is different from the sampling theorem that we had before.
I wish I could put more things on one slide but you wouldn't see them if I could.
So here's a Fourier transform of the sum 1 over t, u of kt time the sinc function.
Here it's the same thing.
That's one reason for comparing these things sometimes.
1 over t in here.
OK, times, so it's the same thing with this rotating term which is the only difference.
Now, how many of you can see that the Fourier transform of this quantity is equal to this sinc function?
I can't do that.
It's one of the things you're going to do in your homework.
I get confused every time I do it, and I got confused enough to leave out the 1 over t this time when I did it.
You just have to be patient with that and do it a few times and you'll find that it's not -- well, it becomes automatic after awhile.
It's one place where you need plug and chug.
So now that we've generalized the dtft to look at any old frequency band instead of just the frequency band around zero, we can do the same thing that we did with time functions.
Namely, with a time function -- Yes?
[INAUDIBLE]
You don't think it should be a 1 over t?
Well, you very well might be right because I didn't think it should have been either when I wrote it down.
But I don't see --
[INAUDIBLE]
It was the--
[INAUDIBLE]
Oh, that's the difference, yes, of course.
It's not the coefficient that I have here, it's the actual -- yeah.
Yes.
So it should be the same as the-- Oh, I see the problem.
I shouldn't have had the 1 over t here, should I?
No.
I know one of them couldn't be right.
It is right in the notes, so you can sort it out there.
So, the thing we did when we were dealing with a Fourier series is we took an arbitrary function of time, we segmented it into time intervals and then we expanded each one of those time intervals into a Fourier series.
By doing that we could take an arbitrary L2 function and represent it as an orthogonal expansion over this double sum of time shifts and frequency terms.
We call that the truncated sinusoidal expansion, the t spaced truncated sinusoids and we made an expansion out of that that would allow us to express any old L2 function in terms of that.
Here we're going to do the same thing.
We can take an arbitrary frequency function, separate into bands of frequencies.
You often want to do this in digital communication when you're looking at transmitting information in different bands, which you do all the time in radio.
Somebody has a certain part of the spectrum, they transmit a signal there, somebody else has another part of the spectrum, they transmit a signal there.
You can look at those different signals which are in different frequency bands, they're all orthogonal to each other, they don't interfere with each other at all.
We're doing the same thing here.
We're just saying an arbitrary function can be split into different frequency bands, each one of those frequency bands can be represented both by a dtft, which is the thing we just did on the last slide, and by sampling theorem expression, which is what we get when we take the inverse Fourier transform of the dtft.
So when we do that what we get is a perfectly arbitrary frequency function which exists from minus infinity to plus infinity.
We can represent it as the sum of all these separate frequency functions.
I just threw a limit in the mean here because I'm not being careful about what happens where we separate from one frequency function to the next.
Namely, at frequency w do I use one term or do I use the other term or do I use the sum.
But we don't want to worry about that.
We don't want to even think about it.
So we put a limit in the mean here.
So the v sub hat m of f then is going to be the part of u of f which is in this particular frequency range.
That's completely the analogy of taking a time function, looking at that time function over a particular range of time, and here what we're doing is taking a frequency function, segementing it into different frequency intervals so that the end frequency interval is then just this with a rectangular function to truncate it.
If I take the inverse Fourier transform of this what I'm going to get is u of t, take the inverse transform of all of these terms, so I'll get the sum of vm of t. Now, vm of t is the inverse Fourier transform of vm of f. You take that quantity, take the inverse transform of it, and sure enough you get this kind of expression here.
It's a sampling theorem in v sub m of f with this rotating frequency term here, which is just the thing that we had before.
So, all we're doing here is starting out with some arbitrary frequency function, we're segementing it in frequencies.
Each frequency band then has a dtft associated with it.
When we take the inverse Fourier transform of that dtft, what we get is a sampling expansion for that particular frequency band.
So what this is doing here, finally when we get all done with this, is I'm just combining the sampling theorem expansion in each frequency range, which is what u of t is.
So I've taken u of t, I split it up into different frequency ranges, I've expressed what's in each frequency range in terms of a sampling theorem.
The sampling theorem terms are these with these rotating terms in them corresponding to the mth frequency range.
This is completely analogous then to the truncated sinc function expansion we had before.
This becomes a little more sensible if we substitute a sampling time, t, for 1 over 2w.
Namely, all of these expressions here are talking about what happens when you sample these individual frequency bands at intervals 1 over 2w.
So we'll just call that capital T to make the formula look a little simpler.
Then we get u of t is this limit in the mean of this whole expression there.
So it's a double sum, it's a sum over time, over the samples, so there's one term for each time, kT, and there's one term for each frequency interval.
Frequencies are indexed by m, time is indexed by k. So the thing we have here is an expansion now in terms of coefficients -- these are just called coefficients again.
This expansion which looks suspiciously like the t space truncated sinusoids that we had before.
The only difference is that, the terms were truncated in time; here, the terms are truncated in frequency.
So the different terms making up this expansion, these orthogonal terms here, in one case what we have is a sinc function which is translated in time and then translated in frequency.
In another case we have the rectangular function, which is translated in time and then translated in frequency.
So in that sense, these two expansions are almost the same, and you can think of doing expansions perhaps in other things also and we'll talk more about that later.
So, this then is just this thing we're going to call a t spaced sinc weighted sinusoid expansion.
So the only thing we have is this one sinc function which is this hat sort of function.
The terms in here are those functions shifted in time by some number of sampling intervals, t, and then shifted in frequency by some number of frequency bands, 2w.
See, the original frequency bands that we had went from minus w to plus w. The next one goes from w to 3w, the next one goes from 3w to 5w.
So the frequency bands we're talking about here are of width 2w.
The time intervals we're talking about are of width t. Why do people confuse you that way?
Well, because all of this happened a long time before people realized how closely the duality relationship between time and frequency was.
So people wanted to talk about frequencies, baseband frequencies.
You talk about a baseband limited to w and you're talking about positive frequencies, because engineers used to deal with cosines and sine, and there weren't any such thing as negative frequencies.
Then they decided everything was easier when they dealt with complex sinusoids, negative frequencies reared their ugly head, but people didn't want to change their notation for what a frequency band was, which is probably good.
So we're simply stuck with this incompatibility of dealing with frequencies one way and dealing with time the other way.
We can look at that as increments of time t, and increments of frequency, 1 over t, but, in fact, 1 over t is 2w, so the increments in time we're using in both of these expansions, are t, the increments in frequency we're using are 2w.
Now, there's a relatively long section in the notes talking about degrees of freedom, which is a pretty important topic.
It's a little bit fishy mathematically, but it really makes good engineering sense.
It's an idea which is important both in terms of taking source wave forms and representing them in orthogonal expansions, and in taking frequency functions and representing them -- well, it is also important in terms of taking things that we transmit where you have bits coming into an encoder.
We're going to turn those bits into signals, we're going to turn those signals into wave forms.
Those wave forms will usually be thought of as things that we transmit in time, and we also transmit them in frequency, because we often use some kind of multiplexing between different frequency bands.
We want to have a common way of thinking about all of these things, and this is the way that we're going to do it.
Namely, if we're thinking in terms of a particular sampling time, t, and we want to look at a very, very large frequency band, and therefore, look at many multiplex frequency bands, we can say how many coefficients can we send on this channel?
When we look at it in these terms of different frequency bands, the number of coefficients we can send is over a period of time, t zero.
We can send t zero over t different coefficients in time.
We now look at frequency.
We have some very broad frequency band, w zero.
The number of different bands that we have is w zero over 2w.
As a result of all of this when you add everything up, you get 2t zero w zero degrees of freedom over this overall bandwidth of w zero.
Now remember, an overall bandwidth of w zero in terms of these complex frequencies goes from minus w to plus w. Minus w zero to plus w zero.
The time interval goes from minus t zero over 2 to plus t zero over 2.
So this factor 2 here, which we always talk about in terms of number of degrees of freedom, is really a consequence of the fact that we measure time intervals and frequency intervals in a slightly different way.
But anyway, whether we look at it in terms of one expansion or the other expansion the answer we get is the same.
If you take some large time interval, some large frequency interval, tuck as many numbers as you can in that interval, this is what you come up with.
Now, why did I say that this is just slightly fishy?
Well, it's slightly fishy because if you take a function and you truncate it in time -- if we take this function and truncate it to minus t zero over 2 and plus t zero over 2, even though that might be ten years, how can we limit the frequency?
Well, we can't.
Because when we take the Fourier transform of a time limited function, it exists for all frequencies.
The same thing happens if we try to limit it in frequency, it squirts out forever in time.
So you can't get around that.
The thing that saves us is that if t zero and w zero are both large enough, these functions all dribble away quickly enough that it doesn't make any difference.
You know it has to.
If you think in terms of the sampling theorem, and you try to think about it carefully in mathematics, what does it say?
If you want to transmit a function by putting these little sine x over x hats around each of the coefficients in the function, when do you have to start transmitting those wave forms?
You have to start transmitting them at minus infinity.
I mean we turn on our transmitter and we somehow have to have been transmitting for an infinite amount of time before we send the first symbol.
Well that's ridiculous, of course.
So that we always approximate these sinusoids by sinusoids which are truncated, and we always have some engineering faith that what we're throwing away is not important.
The only place that it's important is when you start talking about things which are really zero everywhere and then it becomes important.
But the idea of degrees of freedom is a very sensible idea until you try to express it precisely.
Let's get on to something called aliasing.
We're going to spend most of the rest of today talking about aliasing.
I want to try to explain why it is that we want to spend time on this, because there are really two things going on here.
One of the things that are going on is that if you want to look at a wave form and do some processing on it, the usual way to do it with the digital technology we have today, is to take that wave form and sample it very, very rapidly in time.
Then process the hell out of all those samples.
We do that hardly caring whether it's band limited, hardly caring about the information in it, hardly caring about anything, we just want to sample it so fast that we essentially approximate the function very well.
Now when we do that something's going to get lost, because when we take those samples we're ignoring what happens between the samples.
In some approximate sense, wave forms are always smooth because they always get filtered by something before anybody looks at them.
And because they're smooth, if you'd sample it fast enough, your fast enough samples are going to look like the wave form.
If you just connect them with straight lines you're going to get a very good approximation of what the wave form is.
But then you stop and ask, and when you stop and ask you're in trouble because you say well, how fast do I have to sample, and if I sample that fast, how much error am I going to make?
That's the question for which aliasing gives you the answer.
So we want to explore it for that reason.
The other reason that we want to explore it is when we start talking about modulation, we'll start talking about something called Nyquist criterion, and that's best looked at in terms of aliasing again, and we'll see why that is when we get there.
So for both of those reasons, and also for the reason of trying to understand these expansions in terms of source wave forms, we want to understand what this relationship is between the samples of a function that isn't quite band limited and the function itself.
So the thing we're going to do to try to understand that is instead of studying just the samples, which is what you usually do, if you're just looking at the samples and you're trying to say how much error do I incur by doing that, and we're looking at mean square error, somehow or other we have to get back to wave forms and compare the resulting wave form with the wave form we started with.
So the thing that we're going to do is to take our function u of t, we're going to sample it at some very rapid speed, and then we're going to take those samples and recreate a wave form by the sampling theorem.
So we're going to call the approximation to u of t some approximation s of t -- s of t is, in fact, baseband limited at this point to a frequency w where t is equal to 1 over 2w, and we simply have this sampling theorem applied to u of kt, as if u of kt came from a band limited wave form.
So this is, in a sense, an interpolation formula.
It's a little better than taking these samples and joining them by straight lines, because we're, in fact, joining them by these smoother sinc functions.
The question is how close is s of t to u of t and how do we look at that question?
Well, we now have a nice way of looking at it because after going through this t spaced sinc weighted sinusoidal expansion, we have an exact expression for u of t, which is the limit in the mean of these frequency terms in u of t. Namely, to get this expression, remember the thing we did was to take arbitrary u of t, split it up into little frequency bands, each of width w, then apply the sampling theorem to each of those frequency terms, which we can do exactly.
So this, in fact, is an exact expansion -- don't worry about the limit in the mean right now, we're going to get rid of that in a while.
So we have s of t, which is this.
We have u of t, which is this.
Now, we look at this and we say OK, s of kt, namely, the case sample of s of t, is simply u of kt.
That's what happens here.
You put in any old time which is some integer j times capital T and you look at this expression here, and the sinc function is only non-zero when j capital T over t, when if you take the sinc function of an integer it's zero unless the integer is zero.
The sinc function goes through zero at every integer point except for zero itself where it's equal to 1.
That's the thing which makes the sync function nice.
It dribbles away -- 2, 3 and so forth minus 1.
So it's zero at all those sample times, so that the case sample of s of ts of kt is just equal to u of kt.
Namely, what we're doing in this approximation, which is what one usually does if one samples something, is we're assuming that the approximation is correct at the sample points, and we're arranging it so it's correct at the sample points.
So, s of kt then is equal to u of kt from this.
u of kt from this -- OK, now take t and substitute k times capital T in here.
And if we substitute k times capital T in here -- let's not, let's substitute j times capital T to avoid a conflict with notation here.
Then this T here becomes jT and what we have is sinc of j minus k. So we have a sum here over all k of sync of j minus k. Sinc of j minus k is zero for all integers k, except for j, therefore, this quantity is zero every time k is not equal to j.
Therefore, we just have the sum over m. So, s of kt is equal to u of kt, which is equal to the sum over all frequency bands, m, of v sub m of kt.
Now what is this saying?
The thing this is saying is if you take this function u of t, which has a bunch of different frequency bands in it, each of those frequency bands has a sampling theorem associated with it.
Each one of those frequency bands is represented by its samples at periods of time, k. But as soon as we look at u of kt, if we only have the samples u of kt, there's no way to tell which frequency band it's coming from.
So all of these different frequency bands all get alias together.
If I just look at u of kt, what it is is the sum over all these frequency bands of the samples of the individual frequency functions.
So that if you tell me what u of kt is, I can't tell you what these samples are.
All I know is what the sum of them is.
So, in fact, if you start out with a function which instead of being baseband limited to w, in fact, is sitting between w and 3w, and there's nothing in this baseband, and I look at these samples and then I recreate things this way, what am I doing?
I'm just taking that function at w to 3w and translating down in frequency to minus w to w. You can't tell.
There's no way to tell just from the samples which frequency band we're looking at.
So all these things get mixed together.
So, s of t then, since s of t is this times these sinc functions, and this is the sum of all the vm's, s of t is just this double sum now where all of these are now down at this baseband frequency interval.
There's no way to tell them apart.
We have this double sum here so I'm adding up all of these different coefficients and they're all mixed together all down in this one frequency band.
So, u of t is represented this way, double sum, vm of kt, vm of kt, sinc of t over t minus k, sinc of t over t minus k. This rotating term up here, an s of t, what I've done effectively is to get rid of all these rotating terms.
It simplifies it enormously, but I changed the function.
In fact, this function is low path limited to w and this function has all of the glory of an arbitrary set of frequencies in it.
By just looking at these samples, I've lost all of this stuff and it's just back down to a baseband limited function at this point.
So, if I look at the difference between u of t and s of t, what I'm going to get -- and this is expressed a little differently than the way it is in the notes but it's the same thing -- just the sum over k and m, and it has these sinc functions in it, which both of these terms have.
Then u of t has these rotating terms and s of t doesn't, so s of t just has one in place of the rotating term.
So the difference between u of t and s of t is just this big monster sum here, which is looking at all the different frequency bands at all the different sampling times.
If I now try to look at the energy difference between u of t and s of t, because that's what I'm interested in -- how much error have I accumulated, how much mean square error have I gotten?
By taking u of t and sampling it and then viewing it as a low pass function.
This energy difference, well, we have a set of coefficients here, we have a set of functions here.
These functions are all orthogonal to each other.
Why are they orthogonal?
Well, because sinc functions are orthogonal, which I think we've shown on our homework, and space time functions are orthogonal.
So, all of these functions are orthogonal to -- excuse me -- spaced frequency functions are orthogonal to each other, and the sinc weighted functions are orthogonal to each other.
So all I'm doing here is expanding all of this in terms of these orthogonal functions.
I have to do this separately for these terms -- I know what happens.
This works when it's cold and it doesn't work when it's hot.
Interesting.
So I'm separating this into this term and this term.
Now, look at what this is when m is equal to zero.
When m is equal to zero, e to the 2 pi i, zero t over t minus 1 is zero.
So, all of the m terms here are collapsed into zero, so there isn't any error down at baseband in a sense.
All of the error occurs due to these frequency terms larger than minus w to w. Well, that's the way it should be.
Because we know that if u of t didn't have any terms outside of minus w to plus w, the sampling theorem would be absolutely rock solid with no error.
So the errors are due to two kinds of terms.
One, they're due to these terms, which is this thing.
Two, they're due to these terms, which is this.
The only difference between this is that this is a square of a sum and this is a sum of squares.
Sum of squares are nicer, we can deal with them more nicely, and we understand what that is better.
A sum and then taking the square of it after we take the sum is a good deal dirtier.
The trouble is that this difference doesn't have to be L2.
In other words, the mean square error we got from doing this can be infinite.
One of the problems in the current problem set is a nice simple example of this.
So, s of t need not have finite energy.
That's a real kicker, because all of this theory is very nice until this point.
I mean the sampling theorem works perfectly because when you're dealing with baseband limited functions -- for up baseband limited function, the sinc functions give you a perfect approximation.
Namely, the sampling theorem works with no error at all.
As soon as you get a function which spills out into higher frequencies, you can, in fact, have these samples coming up with infinite energy in them, and at that point this difference here can have infinite energy.
In fact, you would do much, much better if you simply represented u of t by zero.
You'd do infinitely better in terms of mean square error just by throwing it away and saying I'm not going even bother to approximate it.
I'll just call it zero and nothing else.
If you did that, you would only have the error in u of t and you wouldn't have all this generated stuff from all these terms that you're throwing away.
When we start looking at random processes, and really the only thing that we're interested in here is random processes because noise is a random process, signals are random.
So when we start looking at random processes, the thing that we're going to find is that this square of a sum in terms of expected value is going to be approximately the same as this sum of squares.
So that this term and this term are going to be roughly equal for most of the stochastic processes that we deal with, so this problem doesn't occur of s of t having infinite energy.
What that means is that these two terms are of roughly equal magnitude.
Now, we can understand something more about these two terms.
The term here is what you get, namely, these are the parts of u of t which are outside of the frequency range from minus w to w. A reasonable alternative to simply sampling this function would be to filter the function first.
If you filter the function first what's going to happen?
These terms will go away.
These terms will stay.
You still have the aliasing -- you can't get rid of that.
But you can get rid of these terms.
Excuse me.
I take that back.
I seem to have a binary problem today, whatever.
I seem to be mixing up proof and falsehood.
If you filter a function you're clearly going to get rid of all of the aliasing, because after you filtered the function you will have a band limited function sitting there.
So the only error you're going to get is the error in what you've thrown away.
What you've thrown away is, in fact, this quantity here.
Slow down, say it right.
What you have thrown away is all of the extra frequency terms.
So the error is all of those frequency terms that you've thrown away.
These are the frequency terms that you've thrown away.
This is the error that you wind up with after you filter.
These terms here are the aliasing terms.
These are the things which, in fact, could be infinite if you're unlucky enough.
These you can get rid of by filtering.
So now the question is should you filter first or should you just sample and say to hell with it?
Well, if you think about why you want to sample and use digital signal processing, what you're trying to do is to avoid building very complex analog filters.
So if you try to build a very, very sharp filter which is getting rid of all of those out of band terms, you're sort of throwing away a lot of the reason for trying to sample to start with.
So the usual conclusion is no, we're not going to filter first, or we're only going to have a very crude filtering operation first, and we'll just sample a little faster if we have to, because sampling faster is easier.
So we usually just sample faster and avoid these terms.
Now we want to look at aliasing viewed in frequency terms, and we've almost been doing this.
We've been sort of talking around it a little bit.
We're viewing an arbitrary function, u of t, in terms of a sum of these frequency limited functions.
We're just arbitrarily taking a function, splitting it up into these frequency bands.
In terms of frequency what we're doing is just segementing the Fourier transform u of f. As far as the samples are concerned, uf of kt is just equal to the sum of the samples in each of these frequency bands -- that's what we've been saying.
That's really what aliasing is at a fundamental level.
So when you sample you can't tell which frequency band each of these samples come from, and they usually come a little bit from each one.
So, s of t now, I'd like to split up s of t in the same way that I split up u of t. I'd like to split up s of t, even though I can't do this from looking at s of t, I can do it mathematically.
I want to split up s of t into the contributions to s of t from all of these different bands.
So I want to view the mth frequency band as the sum of vm of kt times sinc of t over t minus k. Namely, look at what s of t was to start with, somewhere back here a long time ago.
What we found was that s of t is equal to this quantity here, which is the sum over time terms, the sampling terms, and a sum over frequency term.
What I'm doing now is I'm just defining s sub m of t to be this sum in here for one particular value of m. I'm just splitting this double sum into a number of separate terms.
So this is the contribution to s of t from the mth frequency band.
So, s of t is the sum of all of these different frequency contributions in this quantity.
vm of t, which came from u of t, same quantity here except for the complex exponential here.
vm of t is the same quantity with the rotating term at the end of it.
Namely, this is the actual signal at this mth frequency band.
So the contribution to it and the sampling approximation is this, the actual term is this.
So we look at these two and we say gee, this looks like -- if we look at this in the frequency domain this looks like just a frequency shift.
So the thing we can then say is that v sub m of f, namely the Fourier transform of this, differs from the Fourier transform of this just by a frequency shift.
If we take the frequency shift formula and frequency and look at it in time it just becomes that rotating term there.
Namely, frequency shifts look like complex exponential multipliers.
We already know that d sub m of f is the mth frequency band and u of t. So in fact, it's u of f truncated to the mth frequency band.
That's what this said .
So, u hat of f rect ft minus m is just equal to this term here.
So this says that s hat of f, which is the sum of all these terms, is just the sum over m, sum over all the frequency terms of u hat of f plus m over t times a rectangular function of ft. All this is saying is that s of f in frequency, it's all down at baseband now.
Each of these frequency terms in u of t, up at some band and frequency, when we sample it, we're effectively bringing it down to base then.
When we sample it and multiply by the sinc functions we're bringing it down to baseband.
So this is saying when you look at this approximation, the space band approximation, what we wind up with is this frequency function evaluated at all of these different -- well, just evaluated at different times.
So, s hat of f is, in fact, frequency limited to minus w to plus w, and it has all of these different terms contributing to it.
It's just looking at aliasing and frequency instead of looking at aliasing and time.
In both cases it's the same thing.
In one case, all of the samples get mixed together.
In the other case, all of the frequency bands get mixed together.
I hope this will be clearer in terms of this.
Let's take some arbitrary function here -- a most amazing thing, these things all get.
It seems as if latex -- a thing for drawing pictures is fine on my screen but not fine when I print things.
But anyway, this is fine as it is.
It's slightly different from the one in the notes, which is also goofed up in the same way.
Let's suppose we start out with a frequency function which looks like this.
Actually, it should have something else there but that's OK.
This is minus 1 over 2t, this is plus 1 over 2t.
In other words, minus w to w. Here we have another frequency band from w to 3w, another frequency band from 3w to 5w.
What that formula said is that each frequency band gets picked up and stuck down in the baseband area.
So, in fact, what's happening is that this part of the frequency function is going to be picked up, stuck down there.
So this goes over there.
This quantity here is in the band from 3w to 5w.
It gets picked up and put in here also.
It's gotten picked up and put down there.
So I have this part, this part, and this part just stays where it is -- this is the part that's actually band limited.
So that s hat of f now is going to be the sum of this and this and this, and one disappeared term which was supposed to be here, and that's going to come in there.
When you add up all of these -- this goes over to there.
When you add up all of these you get the total frequency function s hat of f. All of you understand the mechanics of this?
I mean graphically you can just find s hat of f from taking all of these things and folding them all into this baseband approximation.
So that you can look at the error that you get in terms of sampling a function which is not quite band limited in these terms, also.
The aliasing then looks like the translation from something which is in this band stuck into the main band between minus w and w. Translation of this thing stuck into the main band.
Translation of this stuck into the main band.
They're all added together in such a way that you can't tell which one came from where.
So once you look at this, you can't go back to here, which is why people call it aliasing.
These things are all just mixed together and there's no way to get back.
The theorem that corresponds to all of this, and the theorem is pretty much proven in the appendix if you want to read it.
If you're not interested in those mathematical details you're certainly welcome not to read it.
I'm not going to have any problems on it.
I'm not going to have any quiz problems on it, we might have a problem on it.
It turns out that just L1 and L2 is not enough when you're dealing with aliasing.
Aliasing, since you're both sampling and looking at things at arbitrarily large frequencies, the mathematics just gets messy.
So the condition that you need is that the frequency function you're dealing with in order for all of these aliasing results to hold true, is that the limit as f goes to infinity of u hat of f times some function, f to the 1 plus epsilon has to go to zero.
In other words, this is saying that u hat of f has to go to zero with increasing frequency fast enough.
It has to go to zero a little faster than 1 over f. If it went to zero, it was 1 over f, that would be guaranteed by a thing, L2.
That's not enough here.
You need the stronger condition.
So if it goes to zero fast enough as f gets very, very large -- you know any function you're going to deal with you can always model it so that it does this.
Because you simply can't transmit wave forms that have arbitrarily high frequencies in them.
I mean no matter what kind of antenna you use to transmit them, if it's an optical antenna you can get to much higher frequencies, but no matter what kind of antenna you use you're going to be limited somehow in frequency, and it's going to drop off much faster with increasing frequency than this.
So this isn't any sort of practical limitation on aliasing, it's just that it's there and it limits the models you can create somewhat.
If you have this condition then it says that the Fourier transform of u of t has to be an L1 function.
The inverse transform, u of t, has to be continuous and bounded.
In other words, when we go from u hat of f to u of t we get a bonafide function there.
It's not something which is a limit in the mean.
Because as soon as we're dealing with samples of something you really need a function to talk about.
You can't have something which is -- well, you can't have something which has these little extra things on it.
You can't live with that.
So the theorem says that this frequency function has to be L1.
It says the inverse transform has to be continuous and bounded.
We've already seen that L1 functions have continuous and bounded Fourier transforms.
For any t greater than zero, the sampling approximation, namely, s of t, which is this, is going to be bounded and continuous.
And s hat of f satisfies this relationship here.
So this is the frequency version of the aliasing formula.
Here you still have the limit in the mean here.
As soon as you go back to frequencies you can't say anything precise anymore.
In time, everything is precise.
These functions are bonafide functions.
In terms of frequencies they're not quite.
I want to just start talking about these L2 functions.
As you realize, we've not only been talking about L2 functions, we've been talking about L1 functions also, and now these crazy functions which go off to zero fast enough.
The thing we're basically interested in, aside from sampling, is the L2 functions because they're the ones where you can go from function to Fourier transform and back again.
They're the ones which work or these orthogonal expansions, and they're the main things we're going to be interested in.
When we try to go further talking about these functions, it turns out that it's much easier to think about them in terms of vector spaces.
Some of you probably have thought about wave forms before as vectors, some of you probably haven't.
I'm sure all of you are familiar with vectors at least in terms of a notational convenience as a way of representing n tuples of numbers.
You can always take an n tuple of numbers and instead of writing out u1, u2, u3, up to u sub n, you can say vector u, and you can manipulate those vectors, you can add vectors, you can multiply them by scalers, you can do all of the neat things that people do, even without knowing anything about vector spaces.
It's not much of an extension on that to take accountably infinite sequence of numbers and represent it as a vector, especially if you've never done it before.
If you've done it before and if you've thought about it before, you will realize there are some problems there.
But at least conceptually there's no problem.
You can think about a sequence of numbers as being a vector, you can add those sequences, you can multiply them by scalers, you can do all sorts of neat things with them.
Since we've shown how to represent wave forms as sequences of coefficients in orthogonal expansions, it then isn't much of an extension to think about wave forms as being vectors.
But the nice thing about doing this is that you're not really stuck to a particular orthogonal expansion when you do this.
You can think of wave forms as being vectors in just as nice a sense as you can think about these n tuples as being vectors, and that's what we're trying to get at.
These orthogonal expansions we're going to be looking at are really viewed most easily in terms of vector space.
All of the questions about convergence of orthogonal expansions and things like that, limits and all of that, all of these are very natural in vector spaces.
They're so natural that people often forget about the fact that they're taking limits even.
So it just looks nice there.
But as soon as we do this, what we're going to be doing constantly from now on is trying to say what we know when we're looking at two-dimensional vectors, we have all sorts of pretty pictures about how they behave.
We are going to be trying to use those pretty pictures in two-dimensional space to understand what happens with wave forms, and in order to do that we have to know a little bit more about vectors than just vectors are things you can add and multiply scalers by.
So I will bore you for two minutes by quickly going through the axioms of a vector space.
My reason for going through this is when you define a vector space axiomatically, then you can, in fact, prove that wave forms satisfy all those axioms.
Once you prove that wave forms satisfy all those axioms, then everything you know about two-dimensional and three-dimensional vectors and all of that stuff all applies.
Namely, everything which is general for vector spaces applies to these vectors.
So the axioms are the following.
You start out with a set of elements which you call vectors.
In terms of n tuples, the set of elements is just the set of different n tuples.
You also start out -- well, for here we don't have to worry about scalers.
So we have the set of n tuples now, or perhaps it might be a set of sequences or a set of wave forms.
The things we insist on before we will call this set a vector space are that we have an operation which we call addition.
Fortunately for n tuples, addition is what you would think it would be.
It's element-wise addition.
For sequences, it's element-wise addition again.
For wave forms, it's function addition.
You have communitivity -- you can add things in either order.
I mean look, if nobody pointed this out to you, you'd do it anyway, right?
So that it looks like nothing.
But in fact, you can invent things, mathematical objects, which you can deal with which are not communitive.
So this is saying to be a vector space it has to have this communitivity operation.
Associativity -- again, it's hard to see why you would say something like this, but all we're defining is this addition operation, namely, there's a definition of addition, and we know by axiom that the sum has to be in this vector space.
As soon as we assume this associativity axiom, it says OK, this is a vector in the space, this and this are both vectors in the space.
Therefore, this sum has to be in the space.
Therefore this plus the sum has to be in the space.
What associativity says is that that's the same element as this.
Once you see this you leave out the parentheses because you can add as many things as you want to.
Same thing -- well, here there has to be a unique vector, zero -- v plus zero is equal to v for all v, and there's only one vector that has that property.
We're going to see some problems there in a little bit when we start studying these L2 functions, but we'll do that next time.
Finally, for every v there's a unique negative vector so that the sum is equal to zero.
All the operations that you're used to.
We'll go through the other things next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I just started talking a little bit last time about viewing L2, namely this set of functions that are square integrable as a vector space.
And I want to reveal a little bit about why we want to do that.
Because after spending so long worrying about all these questions of measurability and all that, you must wonder, why do we want to look at it a different way now.
Well, a part of it is we want to be able to look at orthogonal expansions geometrically.
In other words, we would like to be able to draw pictures of them.
You can draw pictures of a function, but you're drawing a picture of one function.
And you can't draw anything about the relationship between different functions, except by drawing the two different functions.
You get all the detail there, and you can't abstract it at all.
So somehow we want to start to abstract some of this information.
And we want to be able to draw pictures which look more like vector pictures than like functions.
And you'll see why that becomes important in a while.
The other thing is, when you draw a function as a function of time, the only thing you see it how it behaves in time.
When you take the Fourier transform and you draw it in frequency, the only thing you see is how it behaves in frequency.
And, again, you don't see what the relationship is between different functions.
So you lose all of that.
When you take this signal space viewpoint, what you're trying to do there is to not stress time or frequency so much, but more to look at the relationship between different functions.
Why do we want to do that?
Well, because as soon as we start looking at noise and things like that, we want to be able to tell something about how distinguishable different functions are from each other.
So the critical question there you want to ask, when you ask how different functions are, is how do you look at two functions both at the same time.
So, again, that's -- all of this is coming back to the same thing.
So we want to be able to draw pictures of functions which show how functions are related, rather than show all the individual detail of function.
Finally, what we'll see at the end of today that it gives us a lot of capability for understanding much, much better what's going on when we look at conversions of orthogonal series.
This is something we haven't had any way to look at before we talked about the Fourier series, the discrete time Fourier transform, and things like this.
But we haven't been able to really say something general.
Which, again, is what we want to do now.
I'm going to go very quickly through these axioms of a vector space.
Most of you have seen them before.
Those of you who haven't seen axiomatic treatments of various things in mathematics are going to be puzzled by it anyway.
And you're going to have to sit at home or somewhere on your own and puzzle this out.
But I just wanted to put them up so I could start to say what it is that we're trying to do with these axioms.
What we're trying to do is to say everything we know about n-tuples, which we've been using all of our lives.
All of these tricks that we use, most of them we can use to deal with functions also.
The pictures, we can use.
All of the ideas about dimension, all of the ideas about expansions.
All of this stuff becomes useful again.
But, there are a lot of things that are true about functions that aren't true about vectors.
There are lots of things which are true about n-dimensional vectors that aren't true about functions.
And you want to be able to go back to these axioms when you have to, and say, well, is this property we're looking at something which is a consequence of the axioms.
Namely, is this an inherent property of vector spaces, or is it in fact something else which is just because of the particular kind of thing we're looking at.
So, vector spaces.
Important thing is, there's an addition operation.
You can add any two vectors.
You can't multiply them, by the way.
You can only add them.
You can multiply by scalars, which we'll talk about in the next slide.
But you can only add the vectors themselves.
And the addition is commutative, just like ordinary addition of real numbers or complex numbers is.
It's associative, which says v, u plus w in parentheses.
Now, you see why we're doing this is, we've said by definition that for any two vectors, u and w, there's another vector, which is called u plus w. In other words, this axiomatic system says, whenever you add two vectors you have to get a vector.
There's no way around that.
So u plus w has to be a vector.
And that says that v plus u plus w has to be a vector.
Associativity says that you get the same vector if you look at it the other way, if first adding v and u and then adding w. So, anything what you call a vector space has to have this property.
Of course, the real numbers have this property.
The complex numbers have this property.
All the n-tuples you deal with all the time have this property.
Functions have this property.
Sequences, infinite length sequences have this property, so there's no big deal about it.
But that is one of the axioms.
There's a unique vector 0, such that when you add 0 to a vector, you get the same vector again.
Now, this is not the 0 in the real number system.
It's not the 0 in a complex number system.
In terms of vectors, if you're thinking of n-tuples, this is the n-tuple which is all 0's.
In terms of other vectors, spaces, it might be other things.
Like, in terms of functions, 0 means the function which is 0 everywhere.
And finally, there's the unique vector minus v, which has the property that v plus minus v -- in other words, v minus v -- is equal to 0.
And in a sense that defines subtraction as well as addition.
So we have addition and subtraction, but we don't have multiplication.
And there's scalar multiplication.
Which means you can multiply a vector by a scalar.
And the vector spaces we're looking at here only have two kinds of scalars.
One is real scalars and the other is complex scalars.
The two are quite different, as you know.
And so when we're talking about a vector space we have to say what the scalar field is that we're talking about.
So, for every vector.
And also for every scalar, there's another vector which is the scalar times the vector.
Which means, you have scalar multiplication.
You can multiply vectors by scalars in terms of n-tuples.
When you're multiplying a scalar by an n-tuple, you just multiply every component by that scalar.
When you take the scalar 1, and in a field there's always an element 1, there's always an element 0.
When you take the element 1, and you multiply it by a vector, you got the same vector.
Which, of course, is what you would expect if you were looking at functions.
You multiply a function.
For every t you multiply it by 1, and you get the same thing back again.
For an n-tuple, you multiply every component by 1, you get the same thing back again.
So that's -- well, it's just another axiom.
Then we have these distributive laws.
And I won't read them out to you, they're just what they say.
I want to talk about them in a second a little bit.
And as we said, the simplistic example of a vector space, and the one you've been using for years, partly because it saves you a lot of notation.
Incidentally, one of the reasons I didn't list for talking about L2 as a vector space is, it saves you a lot of writing.
Also, just like when you're talking about n-tuples it saves you writing.
It's a nice, convenient notational device.
I don't think any part of mathematics becomes popular with everyone, unless it saves notation as well as well as simplifying things.
So we have these n-tuples, which is what we mean by r sub n or c sub n. In other words, when I talk about r sub n, I don't mean just a vector space of dimension n. I haven't even defined dimension yet.
And when you talk in this generality, you have to think a little bit about what it means.
So we're just talking about n-tuples now.
r sub n is n-tuples of real numbers.
c sub n is n-tuples of complex numbers.
Addition and scalar multiplication are component by component, which we just said.
In other words, w equals u plus v means this, for all i. Scalar multiplication means this.
The unit vector, e sub i, is a 1 in the i'th position, a 0 everywhere else.
And what that means is that any vector v which is an n-tuple, n rn or n cn, can also be expressed as the equals summation of these coefficients, times these unit factors.
That looks like we're giving up the simple notation we have and going to more complex notation.
This is a very useful idea here.
That you can take even something as simple as n-tuples and express every vector as a sum of these simple vectors.
And when you start drawing pictures, you start to see why this makes sense.
I'm going to do this on the next slide.
So let's do it.
And on the slide here, what I've done is to draw a diagram which is one you've seen many times, I'm sure.
Except for the particular things on it.
Of two-dimensional space, where you take a 2-tuple and you think of the first element in the 2-tuple, v1, a being the horizontal axis.
The second element, v2, as being the vertical axis.
And then you can draw any 2-tuple by going over v1 and then up v2, which is what you've done all your lives.
The reason I'm drawing this is to show you that you can take any two vectors.
First vector, v. Second vector, u.
One thing that -- I'm sure you all traditionally do is you view a vector in two ways.
One, you view it as a point in two-dimensional space.
And the other is you view it as a line from 0 up to v. And when you put a little arrow on it, it looks like a vector, and we call it a vector.
So it must be a vector, OK?
So either the point or the line represents a vector.
Here's another vector, u. I can take the difference between two vectors, u minus v, just by drawing a line from v up to u.
And thinking of that as a vector also.
So this vector really means a vector starting here, going parallel to this, up to this point, which is what w is.
But it's very convenient to draw it this way, which lets you know what's going on.
Namely, you represent u as the sum of v plus w. Or you represent w as a difference between u and v. And all of this is trivial.
I just want to say it explicitly so you start to see what the connection is between these axioms, which if I don't talk about them a little bit, I'm sure you're just going to blow them off and decide, eh, I know all of that, I can look at things in two dimensions and I don't have to think about them at all.
And then, in a few days, when we're doing the same thing for functions, you're suddenly going to become very confused.
So you ought to think about it in these simple terms.
Now, when I take a scalar multiple of the vector, in terms of these diagrams, what we're doing is taking something on this same line here.
I can take scalar multiples which go all the way up here, all the way down there.
I can take scalar multiples of u which go up here.
And down here.
The interesting thing here is when I scale v down by alpha and I scale u down by alpha, I can also scale w down by alpha and I connect it just like that, from alpha v to alpha u.
Which axiom is that?
I mean, this is the canonic example of one of those axioms
[UNINTELLIGIBLE]
What?
Distributed
Distributed, good.
This is saying that alpha times the quantity u minus v is equal to alpha u minus alpha v. That's so trivial that it's hard to see it, but that's what it's saying.
So that the distributive law really says the triangles maintain their shape.
Maybe it's easier to just say distributive law, I don't know.
So, for the space of L2 complex functions, we're going to define u plus v as w, where w of t equals u of t plus v of t. Now, I've done a bunch of things here all at once.
One of them is to say what we used to call a function, u of t, we're now referring to with a single letter, boldface u.
What's the advantage of that?
Well, one advantage of it is when you talk about a function as u of t, you're really talking about two different things.
One, you're talking about the value of the function at a particular argument, t. And the other is, you're talking about the whole function.
I mean, sometimes you want to say a function has some property.
A function is L2.
Well, u of t at a particular t is just a number.
It's not a function.
So this gives us a nice way of distinguishing between functions and between the value of functions for particular arguments.
So that's one more notational advantage you get by talking about vectors here.
We're going to define the sum of two functions just as the point y sum.
In other words, these two functions are defined at each t. w defined at a t is the sum of this and that.
The scalar multiplication is just defined by, at every t, u of t is equal to alpha times v of t. Just what you'd expect.
There's nothing strange here.
I just want to be explicit in saying how everything follows from these axioms here.
And I won't say all of it because there's too much of it.
With this addition and scalar multiplication, L2, the set of finite energy measurable functions is in fact the complex vector space.
And it's trivial to go back and check through all the axioms with what we said here, and I'm not going to do it now.
There's only one of those axioms which is a bit questionable.
And that is, when you add up two functions which are square integrable, is the sum going to be square integrable also.
Well, you nod and say yes, but it's worthwhile proving it once in your lives.
So the question is, is this function here less than infinity if the integral of u of t squared and the integral of v of t squared are both integrable.
Well, it's a useful inequality, which looks like a very weak inequality but it, in fact, is not weak.
It says that u of t plus v of t. This is just at a particular value of t. So this is just an inequality for real numbers and complex numbers.
It says that this u of t plus v of t, quantity squared, magnitude squared, is less than or equal to 2 times u of t squared plus 2 times v of t squared.
You wonder, at first, what's the 2 doing in there.
But then think of making v of t equal to u of t. You have 2 times u of t squared, which is 4 times u of t squared.
Well, that's what you need here to make this true.
So, in that example this inequality is satisfied with equality.
To verify the inequality in general, you just multiply this out.
It's u of t squared plus v of t squared plus two cross-terms and it all works.
This vector space here is not quite the vector space we want to talk about.
But let's put off that question for a while and we'll come to it later.
We will come up with a vector space which is just slightly different than that.
The main thing you can do with vector spaces is talk about their dimension.
Well, there are a lot of other things you can do, but this is one of the main things we can do.
And it's an important thing which we have to talk about before going into inner product spaces, which is what we're really interested in.
So we need a bunch of definitions here.
All of this, I'm sure is familiar to most of you.
For those of you that it's not familiar to, you just have to spend a little longer with it.
There's not a whole lot involved here.
If you have a set of vectors, which are in a vector space, you say that they span the vector space if in fact every vector in this vector space is a linear combination of those vectors.
In other words, any vector, u, can be made up as some sum of alpha i times v sub i.
Now, notice we've gone a long way here beyond those axioms.
Because we're talking about scalar multiplications.
And then we're talking about a sum of a lot of scalar multiplications.
Each scalar multiplication is a vector.
The sum of a bunch of vectors, by the associative law is, in fact, another vector.
So every one of these sums is a vector.
And by definition, a set of vectors spans a vector space if every vector can be represented as some linear combination of them.
In other words, there isn't something outside of here sitting there waiting to be discovered later.
You really understand everything that's there.
And we say that v is finite dimensional if it is spanned by a finite set of vectors.
So that's another definition.
That's what you mean by finite dimensional.
You have to be able to find a finite set of vectors such that linear combinations of those vectors gives you everything.
It doesn't mean you have a finite set of vectors.
You have an infinite set of vectors because you have an infinite set of scalars.
In fact, you'd have an uncountably infinite set of vectors because of these scalars.
Second definition.
The vector v1 to vn are linearly independent -- and this is a mouthful -- if u equals the sum of alpha sub i v sub i equals 0, only for alpha sub i equal to 0 for each i.
In other words, you can't take any linear combination of the v sub i's and get 0 unless that linear combination is using all 0's.
You can't, in any non-trivial way, add up a bunch of vectors and get 0.
To put it another way, none of these basis vectors is a linear combination of the others.
That's usually a more convenient way to put it.
Although it takes more writing.
Now, we say that a set of vectors are a basis for v if they're both linearly independent and if they span v. When we first talked about spanning, we didn't say anything about these vectors being linearly independent, so we might have had many more of them than we needed.
Now, when we're talking about a basis, we restrict ourselves to just the set we need to span the space.
And then the theorem, which I'm not going to prove, but it's standard Theorem One of any linear algebra book -- well, it's probably Theorem One, Two, Three and Four all put together -- but anyway, if it says if v1 and v sub n span v, then a subset of these vectors is the basis of b.
In other words, if you have a set of vectors which span a space, you can find the basis by systematically eliminating vectors which are linear combinations of the others, until you get to a point where you can't do that any further.
So this theorem has an algorithm tied into it.
Given any set of vectors which span a space, you can find the basis from it by perhaps throwing out some of those vectors.
The next part of it is, if v is a finite dimensional space, then every basis has the same size.
This, in fact, is a thing which takes a little bit of work proving it.
And, also, any linearly independent set, v1 to v sub n, is part of the basis.
In other words, here's another algorithm you can use.
You have this big, finite dimensional space.
You have a few vectors which are linearly independent.
You can build a basis starting with these, and you just experiment.
You experiment to find new vectors, which are not linear combinations of that set.
As soon as you find one, you add it to the basis.
You keep on going.
And the theorem says, by time you get to n of them, you're done.
So that, in a sense, spanning sets are too big.
Linearly independent sets are too small.
And what you want to do is add the linearly independent sets, shrink the spanning sets, and come up with a bases.
And all bases have the same number of vectors.
There many different bases you come up with, but they all have the same number of vectors.
Not going to talk at all about infinite dimensional spaces until the last slide today.
Because the only way I know to understand infinite dimensional vector spaces is by thinking of them, in some sort of limiting way, as finite dimensional spaces.
And I think that's the only way you can do it.
A vector space in itself has no sense of distance or angles in it.
When I drew that picture before for you -- let me put it up again -- it almost looked like there was some sense of distance in here.
Because when we took scalar multiples, we scaled these things down.
When I took these triangles, we scaled down the triangle, and the triangle was maintained.
But, in fact, I could do this just as well if I took v and moved it down here almost down on this axis, and if I took u and moved that almost down on the axis.
I have the same picture, the same kind of distributive law.
And everything else.
You can't -- I mean, one of the troubles with n-dimensional space, to study what a vector space is about, is it's very hard to think of n-tuples without thinking of distance.
And without thinking of things being orthogonal to each other, and of all of these things.
None of that is talked about at all in any of these axioms.
And, in fact, you need some new axioms to be able to talk about ideas of distance, or angle, or any of these things.
And that's what we want to add here.
So we need some new axioms.
And what we need is a new operation on the vector space, before the only -- the only operations we have are addition and scalar multiplication.
So that vector spaces are really incredibly simple animals.
There's very little you can do with them.
And this added thing is called an inner product.
An inner product is a scalar valued function of two vectors.
And it's represented by these little brackets.
And the axioms that these inner products have to satisfy is, if you're dealing with a complex vector space.
In other words, where the scalars are complex numbers, then this inner product, when you switch it around, you have Hermitian symmetry.
Which means that this inner product is equal to u v complex conjugate.
We've already seen that sort of thing in taking Fourier series.
And the fact that when you're dealing with complex numbers, symmetry doesn't usually hold, and you usually need some kind of Hermitian symmetry in most of the things you do.
The next axiom is something called bilinearity.
It says that if you take a vector which is alpha times a vector v, plus beta times a vector u, take the inner product of that with w, it splits up as alpha times v w plus beta times u w. How about if I do it the other way?
See if you understand what I'm talking about at all here.
Suppose we take w alpha u plus beta v. What's that equal to?
Well, it's equal to alpha something w u plus beta w v. Except that's wrong, It's right for real vector spaces, it's wrong for complex vector spaces.
What am I missing here?
I need complex conjugates here and complex conjugates here.
I wanted to talk about that, because when you're dealing with inner products, I don't know whether you're like me, but every time I start dealing with inner products and I get in a hurry writing things down, I forgot to put those damn complex conjugates in them.
And, just be careful.
Because you need them.
At least go back after you're all done and put the complex conjugates in.
If you're dealing with real vectors, of course you don't need to worry about any of that.
And all the pictures you draw are always of real vectors.
Think of trying to draw this picture.
This is the simplest picture you can draw.
Of two-dimensional vectors.
This is really a picture of a vector space of dimension two, for real vectors.
What happens if you try to draw a picture of complex two-dimensional vector space?
Well, it becomes very difficult to do.
Because you're really talking about the real part of, if you're dealing with a basis which consists of two complex vectors, then instead of v1, you need a real part of v1 and imaginary part of v1.
Instead of v2, you need real part of v2 and imaginary part of v2.
And you can always draw this in four dimensions.
And I even have trouble drawing in three dimensions, because somehow my pen doesn't make marks in three dimensions.
And in four dimensions, I'm a blinking 12.
And have no idea of what to do.
So you have to remember this.
If you're dealing with rn or cn, almost always you define the inner product of v and u as the sum of the components with the second component complex conjugated.
This is just a standard thing that we do all the time.
When we do this, and we use unit vectors, the inner product of v with the i'th unit vector is just v sub i.
That's what this formula says.
Because e sub i is this vector u, in which there's a 1 only in the i'th position, and a 0 everywhere else.
So v e i is always the v i, and e i v is always v i complex conjugate.
Again, this Hermitian nonsense that comes up to plague us all the time.
And from that, if you make v equal to e sub j or e sub i, you get the inner product of two of these basis vectors is equal to 0 for i unequal to j.
In other words, the standard way of drawing pictures when you make it into an inner product space, those unit vectors become orthonormal.
Because that's the way you like to draw things.
You like to draw one here and one there.
And that's what we mean by perpendicular, which the two-dimensional or three-dimensional word for orthogonal.
So we have a couple of definitions.
The inner product of v with itself is called inner product v squared, which is called the squared norm of the vector.
The squared norm has to be non-negative, by axiom.
It has to be greater than 0, unless this vector, v, is in fact a 0 vector.
And the length is just the square root of the norm squared.
In other words, the length and the norm are the same thing.
The norm of a vector is the length of the vector.
I've always called it the length, but a lot of people like to call it the norm.
v and u are orthogonal if the inner product of v and u is equal to 0.
How did I get that?
I defined it that why.
Everybody defines it that way.
That's what you mean by orthogonality.
Now we have to go back and see if it makes any sense in terms of these nice simple diagrams.
But first I'm going to do something called the one-dimensional projection theorem.
Which is something you all know but you probably have never thought about.
And what it says is, if you have to vectors, v and u, you can always break v up into two parts.
One of which is on the same line with u.
In other words, is colinear with u. I'm drawing a picture here for real spaces, but when I say colinear, when I'm dealing with complex spaces, I mean it's u times some scalar, which could be complex.
So it's somewhere on this line.
And the other part is perpendicular to this line.
And this theorem says in any old inner product space at all, no matter how many dimensions you have, infinite dimensional, finite dimensional, anything, if it's an inner product space on either the scalars r or the scalars c, you can always take any old two vectors at all.
And you can break one vector up into a part that's colinear with the other, and another part which is orthogonal.
And you can always draw a picture of it.
If you don't mind just drawing the picture for real vectors instead of complex vectors.
This is an important idea.
Because what we're going to use it for in a while, is to be able to talk about functions which are these incredibly complicated objects.
And we're going to talk about two different functions.
And we want to be able to draw a picture in which those two functions are represented just as points in a two-dimensional picture.
And we're going to do that by saying, OK, I take one of those functions.
And I can represent it as partly being colinear with this other function.
And partly being orthogonal to the other function.
Which says, you can forget about all of these functions which extend to infinity, in time extend to infinity, in frequency and everything else.
And, so long as you're only interested in some small set of functions, you can just deal with them as a finite dimensional vector space.
You can get rid of all the mess, and just think of them in this very simple sense.
That's really why this vector space idea, which is called signal space, is so popular among engineers.
It lets than get rid of all the mess and think in terms of very, very simple things.
So let's see why this complicated theorem is in fact true.
Let me state the theorem first.
It says for any inner products space, v. And any vectors, u and v in v, with u unequal to 0 -- I hope I said that in the notes -- the vector v can be broken up into a colinear term plus an orthogonal term, where the colinear term is equal to a scalar times u.
That's what we mean by colinear.
That's just the definition of colinear.
And the other vector is orthogonal to u.
And alpha is uniquely given by the inner product v u divided by the norm squared of u.
Now, there's one thing ugly about this.
You see that norm squared, you say, what is that doing there.
It just looks like it's making the formula complicated.
Forgot about that for the time being.
We will get into it in a minute and explain why we have that problem.
But, for the moment, let's just prove this and see what it says.
So what we're going to do is say, well, if I don't look at what the theorem is saying, what I'd like to do is look at some generic element which is a scalar multiple times u.
So I'll say, OK let v parallel to u be alpha times u. I don't know what alpha is yet, but alpha's going to be whatever it has to be.
We're going to choose alpha so that this other vector, v minus v u, is a thing we're calling v perp.
So that that, the inner product of that and u, is equal to 0.
So what I'm trying to do is to find a vector -- strategy here, is to take any old vector along this line and try to choose the scalar alpha in such a way that the difference between this point and this point is orthogonal to this line.
That's why I started out with alpha unknown.
Alpha unknown just says we have any point along this line.
Now I'm going to find out what alpha has to be.
I hope I will find out that it has to be only one thing, and it's uniquely chosen.
And that's what we're going to find.
So v minus this projection term, this is called a projection of v on u, is equal to v u minus a projection inner product with u.
So it's equal to this difference here.
This is equal to the inner product.
Since we have chosen this term to be alpha times u, we can bring the alpha out.
So it's alpha times the inner product of u with itself.
That's the norm squared of alpha.
So this is inner product of v and u minus alpha times the norm squared.
This is 0 if and only if you set this equal to 0.
And the only value alpha can have to make this 0 is the inner product of v and u divided by the norm squared.
So I would think that if I ask you to prove this without knowing the projection theorem, I would hope that if you weren't afraid of it or something, and you just sat down and tried to do it, you would all do it in about half an hour.
It would probably take most of you longer than that, because everybody gets screwed up in the notation when they first try to do this.
But, in fact, this is not a complicated thing.
This is not rocket science.
Now.
What is this norm squared doing here?
The thing we have just proven is that with any two vectors v and u, the projection of v on u, namely that vector, there which has the property that v minus v u is perpendicular to u, we showed, is this inner product divided by the norm squared times u.
Now, let me break up this norm squared.
Which is just some positive number.
It's a positive real number.
Into the length times the length.
Namely, the norm u is just some real number.
And write it this way, as the inner product of v with u divided by the length of u times u divided by the length of u.
Now, what is the vector u divided by the length of u?
[UNINTELLIGIBLE]
What?
[UNINTELLIGIBLE]
It's the same direction of u, but it has length 1.
In other words, this is the normalized form of u.
And I have the normalized form of u in here also.
So what this is saying is that this projection is also equal to the projection of v on the normalized form of u, times the normalized form of u.
Which says that it doesn't make any difference what the length of u is.
This projection is a function only of the direction of u. I mean, this is obvious from the picture, isn't it?
But again, since we can't draw pictures for complex valued things, it's nice to be able to see it analytically.
If I shorten u, or lengthen u, this projection is still going to be exactly the same thing.
And that's what the norm squared of u is doing here.
The norm squared of u is simply sitting there so it does this normalization function for us.
It makes this projection equal to the inner product of v with the normalized form of u, times the normalized vector for u.
The Pythagorean theorem, which doesn't follow from this, it's something else, but it's simple -- in fact, we can do it right away -- it says if v and u are orthogonal, then the norm squared of u plus v is equal to u squared plus v squared.
I mean, this is something you use geometrically all the time.
And you're familiar with this.
And the argument is, you just break this up into the norm squared of u plus the norm squared of v, plus the two cross-products, the inner product of u times the inner product of u with v, which is 0, and the inner product of v with u, which is 0.
So the two cross-terms go away and you're left with just this.
And for any v and u, the Schwarz inequality says that the inner product, the magnitude of the inner product of v and u, is less than or equal to the length of v times the length of u.
The Schwarz inequality is probably -- well, I'm not sure that it's probably, but it's perhaps the most used inequality in mathematics.
Any time you use vectors, you use this all the time.
And it's extremely useful.
I'm not going to prove it here because it's in the notes.
It's a two-step proof from what we've done.
And the trouble is watching two-step proofs in class.
At a certain point you saturate on them.
And I have more important things I want to do later today, so I don't want you to saturate on this.
You can read this at your leisure and see why this is true.
I did want to say something about it, though.
If you divide the left side by the right side, you can write this as the magnitude of the inner product of the normalized form of v with the normalized form of u.
If we're talking about real vector space, this in fact is the cosine of the angle between v and u.
It's less than or equal to 1.
So for real two-dimensional vectors, the fact that the cosine is less than or equal to 1 is really equivalent to the Schwarz inequality.
And this is the appropriate generalization for any old vectors at all.
And the notes would say more about that if that went a little too quickly.
OK, the inner product space of interest to us is this thing we've called signal space.
Namely, it's a space of functions which are measurable n square integrals.
In other words, finite value when you take the square and integrate it.
And we want to be able to talk about the set of either real or complex L2 functions.
So, either one of them, we're going to define the inner product in the same way for each.
It really is the only natural way to define an inner product here.
And you'll see this more later as we start doing other things with these inner products.
But just like what when you're dealing with n-tuples, there's only one sensible way to define an inner product.
You can define inner products in other ways.
But it's just a little bizarre to do so.
And here it's a little bizarre also.
There's a big technical problem here.
And if you look at, it can anybody spot what it -- no, no, of course you can't spot what it is unless you've read the notes and you know what it is.
One of the axioms of an inner product space is that the only vector in the space which has an inner product with itself, a squared norm equal to zero is the zero vector.
Now, we have all these crazy vector we've been talking about, which are zero, except on a set of measures zero.
In other words, vectors' functions which have zero energy but just pop up at various isolated points and have values there.
Which we really can't get rid of if we view a function as something which is defined at every value of t. You have to accept those things as part of what you're dealing with.
As soon as you started integrating things, those things all disappear.
But the trouble is, those functions, which are zero almost everywhere, are not zero.
They're only zero almost everywhere.
They're zero for all engineering purposes.
But they're not zero, they're only zero almost everywhere.
Well, if you define the inner product in this way and you want to satisfy the axioms of an inner product space, you're out of luck.
There's no way you can do it, because this axiom just gets violated.
So what do you do?
Well, you do what we've been doing all along.
We've been sort of squinting a little bit and saying, well, really, these functions of measure 0 are really 0 for all practical purposes.
Mathematically, what we have to say is, we want to talk about an equivalence class of functions.
And two functions are in the same equivalence class if their difference has 0 energy.
Which is equivalent to saying their difference is zero almost everywhere.
It's one of these bizarre functions which just jumps up at isolated points and doesn't do anything else.
Not impulses at isolated points, just non-zero at isolated points.
Impulses are not really functions at all.
So we're talking about things that are functions, but they're these bizarre functions which we talked about and we've said they're unimportant.
So, but they are there.
So the solution is to associate vectors with equivalence classes.
And d of t and u of t are equivalent if the v of t minus u of t is zero almost everywhere.
In other words, when we talk about a vector, u, what we're talking about is an equivalence class of functions.
It's the equivalence class of functions for which two functions are in the same equivalence class if they differ only on a set of measure zero.
In other words, these are the things that gave us trouble when we were talking about Fourier transforms.
These are the things that gave us trouble when we were talking about Fourier series.
When you take anything in the same equivalence class, time-limited functions, and you form a Fourier series, what happens?
All of the things in the same equivalence class have the same Fourier coefficients.
But when you go back from the Fourier series coefficients back to the function, then you might go back in a bunch of different ways.
So, we started using this limit in the mean notation and all of that stuff.
And what we're doing here now is, for these vectors, we're just saying, let's represent a vector as this whole equivalence class.
While we're talking about vectors, we're almost always interested in orthogonal expansions.
And when we're interested in orthogonal expansions, the coefficients in the orthogonal expansions are found as integrals with the function.
And the integrals with different functions in the same equivalence class are identical.
In other words, any two functions in the same equivalence class have the same coefficients in any orthogonal expansion.
So if you talk only about the orthogonal expansion, and leave out these detailed notions of what the function is doing at individual times, then you don't have to worry about equivalence classes.
In other words, when you -- I'm going to say this again more carefully later, but let me try to say it now a little bit crudely.
One of the things we're interested in doing this taking this class of functions, mapping each function into a set of coefficients, where the set of coefficients are the coefficients in some particular orthogonal expansion that we're using.
Namely, that's the whole way that we're using to get from waveforms to sequences.
It's the whole -- it's the entire thing we're doing when we start out on a channel with a sequence of binary digits and then a sequence of symbols.
And we modulate it into a waveform.
Again, it's the mapping from sequence to waveform.
Now, the important thing here about these equivalence classes is, you can't tell any of the members of the equivalence class apart within the sequence that we're dealing with.
Everything we're interested in has to do with these sequences.
I mean, if we could we'd just ignore the waveforms altogether.
Because all the processing that we do is with sequences.
So the only reason we have these equivalence classes is because we need them to really define what the functions are.
So, we will come back to that later.
Boy, I think I'm going to get done today.
That's surprising.
The next idea that we want to talk about is vector subspaces.
Again, that's an idea you've probably heard of, for the most part.
A subspace of a vector space is a subset of the vector space such that that subspace is a vector space in its own right.
An equivalent definition is, for all vectors u and v, in the subspace alpha times u plus beta times v is in s also.
In other words, a subspace is something which you can't get out of by linear combinations.
If I take one of these diagrams back here that I keep looking at -- I'll use this one.
If I want to form a subspace of this subspace, if I want to form a subspace of this two-dimensional vector space here, one of the subspaces includes u, but not v, perhaps.
Now, if I want to make a one-dimensional subspace including u, what is that subspace?
It's just all the scalars times u.
In other words, it's this line that goes through the origin and through that vector, u.
And a subspace has to include the whole line.
That's what we're saying.
It's all scalar multiples of u.
If I want a subspace that includes u and v, where u and v are just arbitrary vectors I've chosen out of my hat, then I have this two-dimensional subspace, which is what I've drawn here.
In other words, this idea is something we've been using already.
It's just that we didn't need to be explicit about it.
The subspace which includes u and v -- well, a subspace which includes u and v, is the subspace of all linear combinations of u and v. And nothing else.
So, it's all vectors along here.
It's all vectors along here.
And you fill it all in with anything here added to anything here.
So you get this two-dimensional space.
Is 0 always in a subspace?
Of course it is.
I mean, you multiply any vector by 0 and you get the 0 vector.
So you sort of get it as a linear -- what more can I say?
If we have a vector space which is an inner product space; in other words, if we add this inner product definition to our vector space, and I take a subspace of it, that can be defined as an inner product space also, with the same definition of inner product that I had before.
Because I can't get out of it by linear combinations, and the inner product is defined for every pair of vectors in that space.
So we still have a nice, well-defined vector space, which is an inner product space, and which is a subspace of the space we started with.
Everything I do from now on, I'm going to assume that v is an inner product space.
And want to look at how we normalize vectors.
We've already talked about that.
If I have a vector in this vector space that's normalized, if its norm equals 1.
We already decided how to normalize a vector.
We took an arbitrary vector, u, divided by its norm.
And as soon as we divide by its norm, that vector, v, divided by the norm of v, the norm of that is just 1.
So the projection, what the projection theorem says, and all I need here is a one-dimensional projection theorem, it says that v, in the direction of this vector phi, is equal to the inner product of u with phi times phi.
That's what we said.
As soon as we normalized these vectors, that ugly denominator here disappears.
Because the norm of phi is equal to -- because the norm is equal to 1.
So, an orthonormal set of vectors is a set such that each pair of vectors is orthogonal to each other, and where each vector is normalized.
In other words, it has norm squared equal to 1.
So, the inner product of these vectors is just delta sub j k. If I have an orthogonal set, v sub j, say, then phi sub j is an orthonormal set just by taking each of these vectors and normalizing it.
In other words, it's no big deal to take an orthogonal set of functions and turn them into an orthonormal set of functions.
The Fourier series was natural to define that, and most people define it, in such a way that it's not orthonormal.
Because we're defining it over some interval of time, t, that has a norm squared of t and, therefore, you have to divide by square root of t to normalize it.
If you want to put everything in a common framework, it's nice to deal with orthonormal series.
And therefore, that's what we're going to be stressing from now on.
So I want to go on so the real projection theorem.
Actually, there are three projection theorems.
There's the one-dimensional projection theorem.
There's the n-dimensional projection theorem, which is what this is.
And then there's an infinite-dimensional projection theorem, which is not general for all inner product spaces, but is certainly general for L2.
So I'm going to assume that phi 1 to phi sub n is an orthonormal basis for an n-dimensional subspace, s, which is a subspace of v. How do I know there is such an orthonormal basis?
Well, I don't know that yet, and I'm going to come back to that later.
But for the time being, I'm just going to assume that as part of the theorem.
Assume I have some particular subspace which has the property that it has an orthonormal basis.
So this is an orthonormal basis for this n-dimensional subspace, s and v. For each vector in v, s now is some small subspace.
v is a big subspace out around s. What I want to do now is, I want to take some vector in the big subspace.
I want to project it onto the subspace.
By projecting it onto the subspace, what I mean is, I want to find some vector in the subspace such that the difference between v and that point in the subspace is orthogonal to the subspace itself.
In other words, it's the same idea as we used before for this over-used picture.
Which I keep looking at.
Here, the subspace that I'm looking at is just the subspace of vectors colinear with u.
And what I'm trying to do here is to find, from v I'm trying to drop a perpendicular to this subspace, which is just a straight line.
In general, what I'm trying to do is, I have an n-dimensional subspace.
We can sort of visualize a two-dimensional subspace if you think of this in three dimensions.
And think of replacing u with some space, some two-dimensional space, which is going through 0.
And now I have this vector, v, which is outside of that plane.
And what I'm trying to do here is to drop a perpendicular from v onto that subspace.
The projection is where that perpendicular lands.
So, in other words, v, minus the projection, is this v perp we're talking about.
And what I want to do is exactly the same thing that we did before with the one-dimensional projection theorem.
And that's what we're going to do.
And it works.
And it isn't really any more complicated, except for notation.
So the theorem says, assume you have this orthonormal basis for this subspace.
And then you take any old v in the entire vector space.
This can be an infinite dimensional vector space or anything else.
And what we really want it to be is some element in L2, which is some infinite-dimensional element.
It says there's a unique projection in the subspace s. And it's given by the inner product of v with each of those basis vectors.
That inner product, times v sub j.
For the case of a one-dimensional vector, you take the sum out and that was exactly the projection we had before.
Now we just have the multi-dimensional projection.
And it has a property that v is equal to this projection plus the orthogonal thing.
And the orthogonal thing, the inner product of that with s equal to 0 for all s in the subspace.
In other words, it's just what we got in this picture we were just trying to construct, of a two-dimensional plane.
You drop a perpendicular two-dimensional plane.
And when I drop a perpendicular to a two-dimensional plane, and I take any old vector in this two-dimensional plane, we still have the perpendicularity.
In other words, you have the notion of this vector being perpendicular to a plane if it's perpendicular whichever way you look at it.
Let me outline the proof of this.
Actually, it's pretty much a complete proof.
But with a couple small details left out.
We're going to start out the same way I did before.
Namely, I don't know how to choose this vector.
But I know I want to choose it to be in this subspace.
And any element in this subspace is a linear combination of these p sub i's.
So this is just a generic element in s. And I want to find out what element I have to use.
I want to find the conditions on these coefficients here such that v minus the projection -- in other words, this v perp, as we've been calling it -- is orthogonal to each phi sub i.
Now, if it's orthogonal to each phi sub i, it's also orthogonal to each linear combination of the phi sub i's.
So in fact, that solved our problem for us.
So what I want to do is, I want to set 0 equal to v minus this projection.
Where I don't yet know how to make the projection, because I don't know what the alpha sub i's are.
But I'm trying to choose these so that this, minus the projection, the inner product of that with phi j is equal to 0 for every j.
This inner product here is equal to the inner product of v with phi sub j. I have this difference here, so the inner product is the inner product of v with phi sub j minus the inner product of this with phi sub j.
Let me write that here.
v minus sum alpha i phi sub i comma phi sub j is equal to v phi sub j minus summation of i alpha i t sub i. phi sub j.
Which is equal to this.
All of these terms are 0 except where j is equal to i.
Where i is equal to j.
So, alpha sub j has to be equal to the inner product of v with this basis vector here.
And, therefore, this projection is equal, which we said was sum of alpha i phi sub i, that's really the sum of v phi sub j, this inner product, times p sub j, Now, if you really use your imagination and you really think hard about the formula we were using for the Fourier series coefficients, was really the same formula without the normalization in it.
It's simplified by being already normalized for us.
We don't have that 1 over t in here, which we had in the Fourier series because now we've gone to orthonormal functions.
So, in fact that sort of proves the theorem.
If we express v as some linear combination of these orthonormal vectors, then if I take the norm squared of v, this is something we've done many times already.
I just express the norm squared, just by expanding this the sum of -- well, here I've done it this way, so let's do it this way again.
When I take the inner product of v with all of these terms here, I get the sum of alpha sub j complex conjugated, times the inner product of v with phi sub j.
But the inner product of v with phi sub j is just alpha sub j times 1.
So it's a sum of alpha sub j squared.
OK, this is this energy relationship we've been using all along.
We've been using it for the Fourier series.
We've been using it for everything we've been doing.
It's just a special case of this relationship here, in this n-dimensional projection, except that there we were dealing with infinite dimensions and here we're dealing with finite dimensions.
But it's the same formula, and you'll see how it generalizes in a little bit.
We still have the Pythagorean theorem, which in this case says that the norm squared of vector v is equal to the norm squared of a projection, plus the norm squared of the perpendicular part.
In other words, when I start to represent this vector outside of the space by a vector inside the space, by this projection, I wind up with two things.
I wind up both with the part that's outside of the space entirely, and is orthogonal to the space, plus the part which is in the space.
And each of those has a certain amount of energy.
When I expand this by this relationship here -- I'm not doing that yet -- what I'm doing here is what's called a norm bound.
Which says both of these terms are non-negative.
This term is non-negative in particular, and therefore the difference between this and this is always positive, or non-negative.
Which says that 0 has to be less than or equal to this, because it's non-negative.
And this has to be less than or equal to the norm of v. In other words, the projection always has less energy then the vector itself.
Which is not very surprising.
So the norm bound is no big deal.
When I substitute this for the actual value, what I get is the sum j equals 1 to n of the norm of the inner product of v, with each one of these basis vectors, magnitude squared, that's less than or equal to the energy in v. In other words, if we start to expand, as n gets bigger and bigger, and we look at these terms, we take these inner products, square them.
No matter how many terms I take here, the sum is always less than or equal to the energy in v. That's called Bessel's inequality, and it's a nice, straightforward thing.
And finally, the last of these things -- well, I'll use the other one if I need it.
The last is this thing called the mean square error property.
It says that if you take the difference between the vector and its projection onto this space, this is less than or equal to the difference between the vector and the other s in the space.
Any other -- I can always represent v as being equal to this plus the orthogonal component.
So I wind up with a sum here of two terms.
One is the difference between -- well, it's the -- it's the length squared of the projection.
Write it out.
v minus v s -- let me write this term out.
v minus s is equal to v, the projection, plus v perpendicular to the subspace s minus this vector s. This is perpendicular to this and this, so -- ah, to hell with it.
Excuse my language.
I mean, this is proven in the notes.
I'm not going to go through it now because I want to finish these other things.
I don't want to play around with it.
We left something out of the n-dimensional projection theorem.
How do you find an orthonormal basis to start with?
And there's this neat thing called Gram-Schmidt, which I suspect most of you have seen before also.
Which is pretty simple now in terms of the projection theorem.
Gram-Schmidt is really a bootstrap operation starting with the one-dimensional projection theorem, working your way up to larger and larger dimensions.
And each case winding up with an orthonormal basis for what you started with.
Let's see how that happens.
I start out with a basis for an inner product subspace.
So, s1 up to s sub n as a basis for this inner product space.
First thing I do is, I start out with s1.
I find the normalized version of s1.
I call that phi 1.
So phi 1 is now an orthonormal basis for the subspace whose basis is just phi 1 itself.
phi 1 is the basis for the subspace of all linear combinations of s1.
So it's just a straight line in space.
The next thing I do is, I take s2.
I find the projection of s2 on this subspace s1.
I can do that.
So I find a part which is colinear with s1.
I find the part which is orthogonal.
I take the orthogonal part, and that's orthogonal, to phi 1.
And I normalize it.
So I then have two vectors, phi 1 and phi 2, which span the space of functions of linear combinations of s1 and s2.
And I call that subspace S2, capital S2.
And then I go on.
So, given any orthonormal basis, phi 1 up to phi sub k of the subspace s k generated by s1 to s k, I'm going to project s k plus 1 onto this subspace s sub k, and then I'm going to normalize it.
And by going through this procedure, I can in fact find an orthonormal basis to any set of vectors that I want to, to any subspace that I want to.
Why is this important, is this something you want to do?
Well, it's something you can program a computer to do almost trivially.
But that's not why we want it here.
The thing we want it here is to say that there's no reason to deal with bases other than orthonormal bases.
We can generate orthonormal bases easily.
The projection theorem now is valid for any n-dimensional space because for any n-dimensional space we can form this basis that we want.
Let me just go on and finish this, so we can start dealing with channels next time.
So far, the projection theorem is just for finite dimensional vectors.
We want to now extend it to infinite dimensional vectors.
To accountably infinite set of vectors.
So I'm given any orthogonal set of functions, status sub i, we can first generate orthonormal functions as phi sub i, which are normalized.
And that's old stuff.
I can now think of doing the same thing that we did before.
Namely, starting out, taking any old vector I want to, and projecting it first on to the subspace with only phi 1 in it.
Then the subspace generated by phi 1 and phi 2, then the subspace generated by phi 1, phi 2 and phi 3, and so forth.
When I do that successively, which is successive approximations in a Fourier expansion, or in any orthonormal expansion, what I'm going to wind up with is the following theorem that says, let phi sub m be a set of orthonormal functions.
Let v be any L2 vector.
Then there exists an L2 vector, u, such that v minus u is orthogonal to each phi sub n. In other words, this is the projection theorem, carried on as n goes to infinity.
But I can't quite state it in the way I did before.
I need a limit in here.
Which says the limit as n goes to infinity of u, namely what is now going to be this projection, minus this term here, which is the term in the subspace of these orthonormal functions.
This difference goes to zero.
What does this say?
It doesn't say that I can take any function, v, and expand it in an orthonormal expansion.
I couldn't say that, because I have nothing to know whether this arbitrary orthonormal expansion I started with actually spans L2 or not.
And without knowing that, I can't state a theorem like this.
But what the theorem does say is, you take any orthonormal expansion you want to.
Like the Fourier series, which only spans functions which are time limited.
I take an arbitrary function.
I expand them in this Fourier series.
And, bingo, what I get is a function u, which is the part of v, which is within these time limits and what's left over, which is orthogonal, is the stuff outside of those time limits.
So this is the theorem that says that in fact you can --
[UNINTELLIGIBLE]
What?
[UNINTELLIGIBLE]
It's similar to Plancherel.
Plancherel is done for the Fourier integral, and in Plancherel you need this limit in the mean on both sides.
Here we're just dealing with a series.
I mean, we still have a limit in the mean sort of thing, but we have a weaker theorem in the sense that we're not asserting that this orthonormal series actually spans all of L2.
I mean, we found a couple of orthonormal expansions that do span L2.
So it's lacking in that.
OK, going to stop there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to spend a little bit of time today finishing up what we were saying last time, about orthonormal expansions.
Just, essentially, to get a better understanding of what it means to go from finite dimension to infinite dimension.
For the most part, you can just ignore the question of being at an infinite dimensional space, and everything is pretty much the same as it is in finite dimensions.
But there are a few things you have to be a little careful about.
So, let's suppose that we have a set of orthonormal vectors.
Like the sinc vectors for example, that we were using for the sinc functions that we were using, to talk about the Fourier series.
And the thing that we said last time at the end of the hour, we quoted this theorem which was almost obvious, I think, in terms of the other things that we did.
Which said that if you start out with some vector v, you can project it successively onto the set of orthonormal functions in such a way that you get closer and closer to the vector that you're looking for.
And the theorem that we had was that the projection on the whole space, which we call u, in fact is the same as the limit of this sum of orthonormal functions as we take the limit adding more and more functions into this sum.
So this was sort of the same kind of thing that we were doing when we were going -- it's the same sort of thing that we did when we were doing the projection theorem for finite dimensions.
You remember the thing that we did there was to first project this function onto a single waveform.
Then we found the part of the waveform that was orthogonal to the first waveform.
That gave us our second waveform.
And the thing that we're doing here, which is sort of different, is we're starting out with the orthogonal sequence to start with.
Anyway, the thing that we showed was that the vector we start with, minus this approximation, which is in the space generated by all of these p sub n's, that this difference has to be orthogonal to each p sub n. Because that was a way we constructed it.
So this difference is -- well, the limit goes to zero, but also the difference is orthogonal to each phi sub x.
This is the thing which is new, which we didn't get from looking at the Fourier series.
We could have gotten it from looking at the Fourier series, but this is the thing which is general for every orthonormal expansion in the world.
Not just a Fourier series, except all of them.
And then we say that an inner product space has accountably infinite dimension.
If accountably infinite set of orthonormal vectors exist, such that only the zero vectors are orthogonal to each other.
This comes back to one of the issues that we faced when we were going through the Fourier series.
Namely, we showed all this nice stuff, which was really about approximating a waveform by the Fourier series.
The thing that we never showed, and the thing which is messy and difficult, is that if you take a time-limited function and expand it in a Fourier series, how do you know that when you get all done, you're actually going to get the function back again?
This is the part of it which we never talked about.
We talked about how you generate all of the Fourier coefficients, and all of that was fine.
And we showed that when you were all done, this representation function that you had was in fact orthogonal the thing that was left over.
But you never showed that the thing that was left over went to 0.
And the thing this is saying is you don't have to worry about that too much.
Because the thing which is left over has to be orthogonal to everything you started with.
And this sum here has to go to some limit.
What is this limit in the case of the Fourier series?
Suppose we start out with an arbitrary L2 function.
We now try to expand that arbitrary L2 function in a Fourier series from minus t over 2 to plus t over 2.
And we can do that.
When we're all done, this series here is in fact going to be the part of the function which lies between minus t over 2 and plus t over 2.
And this difference here -- well v minus u, the difference between the original vector and this representation vector is going to be this part of the function which lies outside of minus t over 2 to plus t over 2.
So the next thing that we did then was to show that the truncated sinusoids and the sinc weighted sinusoids both in fact span L2.
In other words, any function in L2, after you represent it in terms of that series, what you have left over has to be orthogonal.
In other words, if you take this entire set of truncated sinusoids, every non-zero function is orthogonal to all of that.
Every non-zero function in this L2 equivalent sense.
So in fact, the thing you get out of this is this part of looking at functions which we always ignored.
There's another issue that we ignored, when we were looking at functions, which comes out of this.
Since v, by assumption is an L2 vector.
In other words, it's an L2 function aside from this L2 equivalent stuff, and phi sub m is an L2 vector also, this inner product has to be finite, by the Schwarz inequality.
How did we get around this when we were dealing with the Fourier series?
Does anybody remember?
Of course you don't remember.
I hardly remembered that.
The way we got around it was by saying that this orthonormal function which was in a truncated sinusoid.
It was truncated to minus t over 2 and plus t over 2, and the waveform was just e to the i 2 pi f t. The magnitude of that function was always less than or equal to 1.
And because the magnitude was always less than or equal to 1, you could just take the integral of v of t times phi sub n of t, and you could show directly that that integral always existed.
Because of the special property of the sinusoids.
Here what we're saying is, you don't have to worry about that any more.
You can use arbitrary L2 functions as the components of an L2 orthonormal expansion.
And it still works.
So every one of these things has to be finite also.
So, in fact we're buying something out of this.
At the end of the notes on Lectures Eight to Ten, you will notice something that I'm certainly not going to hold you responsible for.
Something called the prolate spiroidal expansion.
And it's just given there primarily as one more example of an orthonormal expansion.
It's a very interesting orthonormal expansion because it has the property that if you start out asking how much energy can I concentrate within a fixed time interval, and a fixed bandwidth interval.
Which is one of the things which make this whole subject a little bit fishy in a whole and certainly very, very messy.
If I start out with a time limited function and go through the Fourier series, these truncated sinusoids spill their energy out of the band enormously.
In fact, one of the problems at the end of Lectures Eight to Ten carries you through the process of just how much energy you can have outside of band by using these truncated sinusoids.
Because truncated sinusoids, when you look at them in a Fourier series, have a lot of energy outside of where it should be.
So that this particular set of prolate spiroidal functions gives you the answer to the following question.
I would like to find that function, which is limited to minus t over 2 plus t over 2, which has the largest amount of energy, largest fraction of its energy, within the band minus w to plus w. What is that function?
Well, that function happens to be the zero order prolate spheroidal function.
It's the nice property that it has.
And it's a nice function, which almost looks like a rectangular function.
But it just trails off to 0.
And I say, OK, if I want to construct an orthonormal expansion, and I want to find another function which is orthogonal to that function and has the next biggest amount fraction of its energy, inside of this -- strictly inside of this time limit, and as much of the energy within the frequency limit as possible, what's that next function?
Well, you solve that problem.
If you're very good at interval equations.
And I certainly couldn't solve it.
But, anyway, it has been solved.
I mean, physicists for a long time have worried about that particular kind of question.
Because it comes up in physics all the time.
If you time-limit something and you take the Fourier integral, how can you also come as close to frequency limiting as possible?
So this particular prolate spheroidal set of orthogonal functions, in fact, exactly solves the problem of how do you generate a set of orthonormal functions which in fact have as much energy as possible both frequency-limited and time-limited?
And what you find when you do that is, when you take 2 w t of them, at that point the energy in these orthonormal functions, the part of it that's inside the band, really starts to cut off very, very sharply, when w and t are large.
So that in fact you get 2 w t of these functions, which have almost all of their energy in this band.
And everything else has almost no energy in the band.
So if you ever get interested in the question of how many waveforms really are there, which are concentrated in time and frequency, that's the answer to the problem.
And don't bother to read it now, but just remember that if you ever get interested in that problem, which I'm sure you will at some point or other, that's where the solution lies.
And I hope I gave a reference to it there.
I think I did.
Anyway, we have these functions which span L2.
And there's at least one other orthonormal expansion that we have.
Which is both time- and frequency-limited.
And either at the end of today or at the beginning of Monday, we're going to find another particularly important sequence of orthonormal functions.
Which we actually use when we're transmitting data.
So to give an example of what we were just talking about, the Fourier series functions span the space of functions over minus t over 2 to t over 2.
And when normalized, these functions become 1 over the square root of t times what we started with before.
Namely, a sinusoid truncated.
Before we were dealing with the orthogonal functions without the 1 over square root of t in it.
If you want to make them orthonormal, you get this square root of 1 over t, because when you take the energy in this function, you get t. If you don't believe me, set k equal to 0 and look at that.
And even I can integrate that.
And when you integrate 1 from minus t over 2 to t over 2, you get 1.
So, then you have to multiply by the square root of 1 over t. When you view the Fourier series functions in this way, it's nice because they're orthonormal.
It's not nice because the square root of 1 over t appears everyplace.
But the nice thing about it is that then when you expand in the Fourier series, you don't find this t anywhere.
Namely, v is equal to the limit of these approximations where the n'th approximation is just the sum from minus m to m of alpha sub k. phi sub k and alpha sub k is just this inner product.
So, again, you got something nicer by looking at these things in terms of vectors You get nice statements about how these things converge, and you also get a very compact way of writing out what the expressions are.
You also get something a little bit fishy, which a lot of people in the communication field, especially ones who do theoretical work, run into problems with.
And the problem is the following: when you start dealing all the time with vectors, and you forget about the underlying functions and the underlying integrals, you start to think that the subject is simpler than it really is.
Because you forget about all the limiting issues.
And when you forget about all the limiting issues, it's fine almost all the time.
But every once in a while, you get trapped.
And when you get trapped, you then have to go back behind all of the vector stuff and you have to start looking at these integrals again and it gets rather frustrating.
So you ought to keep both of these things in mind.
Let's go on, let's get back from mathematics into worrying about the question of how do you send data over communication channels.
And this is just a picture that we saw starting on Day One of this course, which says the usual way of doing this is, you start out with -- you start out with the source.
You break the source down into binary data, and then you take the binary data and you transmit it over a channel.
And this is the picture of what you get when you're trying to transmit binary data over a channel.
And here, we've broken down the encoder, as we called it, into two pieces.
One of which we call the discrete encoder and one of which we call modulation.
Now, this is a little bit fishy also.
Because there are a lot of people who now talk about coded modulation, where it turns out to be nice to combine this discrete encoder with the modulation function.
And when you actually build modern-day full encoder, namely the whole thing from binary digits to what goes out on the channel, you very often combine these two functions together.
What is it that allows people to do that?
The fact that they first studied how to do it when they separate the problems into two separate problems.
And when you separate the two problems, what we wind up with is binary digits coming in.
You massage those binary digits strictly digitally.
And what comes out is a sequence of symbols.
And, usually, the sequence of symbols comes out at a slower rate, and the binary digits come in.
For example, if you take two binary digits and you convert them into one symbol, from a symbol alphabet with force of size four, then for every two in you get one out.
If you take three binary digits coming in, then you need a symbol alphabet of size eight here.
If you move down in rate from four binary digits to one symbol, then of course you need an alphabet of size 16.
So the size of the alphabet here is going up exponentially, as the great advantage that you get is going down.
We will talk about that more as we move on.
You should sort of keep in mind that there's a strange relationship between size of alphabet and the rate at which you can transmit.
As you try to transmit at higher and higher rates here, the size of your alphabet goes up exponentially, with the rate gain that you get.
In fact, when you look at Shannon's famous formula of how fast you can communicate on a channel, you find a logarithm of a signal to noise ratio.
Of 1 plus a signal to noise ratio.
When you look at that signal to noise ratio and you look at this alphabet size in here, you can almost see just from that why in fact that logarithm in these capacity formulas.
And we'll come back to talk about that again more later also.
But, anyway, what we're going to do at this point is to separate this into the problem of discrete encoding and modulation.
And in modulation, you start out with some arbitrary alphabet of symbols.
You turn the symbols in that alphabet into waveforms.
You transmit the waveforms.
You then get the symbol back, and then you put it into a digital encoder, which gets the binary digits back.
Now, what order are we going to study these in?
Well, we're going to study the modulation first.
And we're not going to study this at all, essentially, except for a few very, very simple-minded examples.
6.451, which is the course that follows after this, which might or might not be taught in the Spring term, and incidentally those of you who want it to be taught should send me an email saying you really need to take this next term for some reason or other.
Because otherwise it will be given in Spring of the following year instead of Spring of this year.
There's another side to that issue, which is a wireless course is going to be given in the Spring of this year.
And if you want to take a wireless course and postpone taking the coding course until the following year, then in fact it might be better to do the postponing job.
I would recommend that those of you who are interested in wireless take the course now.
Because if you're looking for research problems in a communication area, wireless is probably the hottest area around, and the area where the most interesting problems occur.
So, you have to make that tradeoff.
If you want to take both of the courses, great.
Send me an email and say you want to take the coding course.
But anyway, there'll be very little coding in this course, very little discrete coding.
And mostly we're going to look at simple ways of taking simple symbol sequences, turning them into waveforms and then transmitting them on channels
So, basically, when you do the source coding, you have to [UNINTELLIGIBLE] binary --
Binary to symbol to waveform.
Yes
[UNINTELLIGIBLE]
OK, why don't I go from waveforms directly to waveforms instead of going from waveforms to symbols to binary digits, and then I just have to go back up again.
A bunch of reasons.
One of the reasons is that some of the stuff that you transmit is already digital to start with.
When you look at what goes over a network today, for example, it's carrying digital data, it's carrying analog data.
It's carrying images.
It's carrying everything you can imagine.
If you want to design a modulation system which goes directly from analog data to channel data, and you have a hundred different kinds of analog source data, and you have a hundred different kinds of channels, then you're going to have to build one encoder for every combination of source and channel.
In other words, you've got to built ten thousand devices.
If your interest is in keeping engineers employed, which I think is a very good interest, that's a very good philosophy to take.
But, unfortunately, most people who build communication equipment say, we would really rather have just a hundred -- well, two hundred different devices.
One hundred devices which turn all these different sources into binary digits, and another hundred devices which turn the binary digits into something that can be transmitted over any channel you want to.
I mean, this is just a general example of what people call layering.
You want to turn systems into systems with a bunch of layers in them, where each layer is standardized in some way.
And only has to take care of a few particular functions.
And, in fact, that's what we're doing here also.
Because we're dealing with one layer, which is doing discrete encoding.
Which in fact is sort of generating, for the most part, binary digits out of the discrete encoder.
Where, as you go through all of this stuff and you come out with binary digits, some of which are wrong, you can do the decoding and get the correct binary digits out.
If you look at an awful lot of coding theory, you'll find out it doesn't pay any attention at all to what's going on in the channel.
Lots of people who've studied coding all of their lives, and decoding all of their lives, live in this mathematical theory of abstract algebra.
And they have no idea of what channels are.
And they survive because of this layering principle.
So if you want to employ engineers, it's nice to have layering also.
Because engineers don't have to know as much then.
Of course, you people should know it all.
So because, then you can do anything.
And you can be part of those very rare people who can put it all together.
I was just at the 70th birthday party of Irwin Jacobs this past week.
And Irwin Jacobs is the CEO of Qualcomm, which is the company that started to build CDMA wireless systems.
For a long time, the CDMA systems were thought of as probably better than TDMA and FDMA, but much more expensive.
And eventually, we're in the situation where all the new systems being designed are all using CMDA.
As a result of this, Irwin Jacobs is a very wealthy person.
We went to a symphony Friday night, out in San Diego, which was given specifically for him.
Partly because he'd just given $110 million to the San Diego Symphony.
So you can figure from that that he's fairly wealthy.
Well the point of all of that is, he started out as a faculty member here.
He was one of the authors of Rosencraft and Jacobs, which is sort of the Bible of digital communications systems from the `60s.
And people still use it, it's still an excellent book.
And in doing that, he really learned the communication trade from soup to -- I guess, soup to nuts is the way we put it now.
And he could do the whole thing.
And because he could do the whole thing, because he understood coding, because he understood modulation, because he understood channels, this is why he could design systems that really work.
I was talking to the Chief Technology Officer out there, and the Chief Technology Officer said his job was really very easy because Irwin did all of that himself.
And he still does it.
So it really makes sense to know something about all of these pieces.
If you want to become a billionaire.
And if you want to become a billionaire without cheating.
Now, many people become billionaires by cheating, and you've heard of many of them but.
Well, anyway.
Enough of that.
I mean, none of us really want to be billionaires anyway, -- I mean, there's nothing you can do with more than a billion dollars, right?
It's all wasted.
Let's move on.
We've gotten rid of digital coding, saying we're not going to deal with that here.
Let me take the PAM out of here because I don't want to even say what it is yet.
If we take this box I called modulation before, which was one of the two main pieces of a channel encoder, we can break it down into two pieces.
Namely, mainly we're going to be layering things again.
We start out with this symbol sequence, which is what comes out of the encoder, which is usually just a short sequence of binary digits.
So the symbols can be thought of as two binary digits in a sequence, or four binary digits, or six tuples of binary digits, or what have you.
And all of those are common.
There's then a signal consolation that turns symbols into signals.
Now, notation in the field is totally non-uniform, because most people use symbols and sequences and waveforms and binary n-tuples, all totally synonymously.
And the distinction we'll try to make here is that symbols are things like binary digits, that don't have any numerical meaning to them.
I mean, a 1 and a 0, the only thing important there is that 1 and 0 are different from each other.
All computer scientists call 1 and 0's Alices and Bobs.
Lots of other people call them plus-1's.
and minus 1's.
Doesn't make any difference what you'd call them, it's just an alphabet with two values in it.
When we talk about signals, we're talking about numerical values.
So in fact, you could have a signal constellation with two elements in it, where what you're doing is mapping 1 and 0 into plus 1 and minus 1.
Now, that's a pretty silly kind of situation, but it still has the value of saying, you're turning things where there's just an alphabeticized 2 into something where you're saying, these are numerical values and you're interested in the difference between the numerical values.
And the difference is measured in the ordinary way of measuring difference for numerical values.
Or, these can be vectors also.
But vectors in an inner product space where, again, you have length.
And you have distance.
So you have numerical values here.
You don't have numerical values here.
So we're going to talk about how you do this.
And then, for the most part, we're going to define modulation as what happens when you go from the signal sequence to the waveform.
You might realize that our notation is lousy here.
Because I'm calling modulation this whole thing.
And I'm also calling modulation just this thing.
And I'm doing that because there doesn't seem to be any other reasonable word for either one of these things.
So, and I'm going to later split this into two pieces also.
But that will come later.
So, anyway, we're going from symbols to signals to waveforms.
Which might look remarkably similar to what we did with sources.
But we just did it backwards with sources.
We went from waveforms to signals to symbols there.
And here we're doing the same thing.
The modulator often converts a signal sequence to a baseband waveform, and then converts the baseband waveform to a passband waveform.
And, just historically, people used to think of modulation as the process of taking something at baseband and converting it up to some passband.
Now it's recognized that the interesting problem is not, how do you go from baseband to passband.
Which is just multiplying by cosine wave, not much more to it than that.
Well, it's a little more to it, but not much.
And the interesting problem is, how do you convert signals into waveforms.
Which is why we went, one of the reasons we went through all of this stuff about L2 waveforms and orthonormal expansions and all of this stuff.
So, for the time being, we're going to look at modulation and demodulation, without worrying about what bandwidth any of this occurs at.
So we'll just look at it at baseband.
So the simplest example of all of this, so simple that it almost looks like it's silly, is to map a sequence of binary digits into a sequence of signals from the constellation 1 and minus 1.
So all you're doing there is mapping 0 into 1 and mapping 1 into minus 1.
In other words, the 0 and 1 binary digits are mapped into 1 and minus 1.
Why do you do it this way, which is a little confusing, mapping 0 into 1 instead of mapping 1 into 1?
Well, primarily, so you can look at multiplication of signals in the same way as you look at modular to addition of binary digits.
It just turns out to be a little more convenient that way.
And it doesn't make a whole lot of difference.
The point is, we're going from 0 1's to signals which are 1 and minus 1.
Then this sequence of signals is mapped into a waveform which is the sum of u of k times the sinc function t over capital T minus k. In other words, you're thinking of each one of these signals, now, think of it as being a sum of impulses, delayed impulses.
And then think of taking a little sinc, sinc t over t and putting it around each one of those impulses.
In other words, think of passing each of those impulses, which is weighted by one of these values, u sub k, into a linear filter whose response is sine of x over x.
Anybody see anything a little bit fishy about that?
Have you ever built a filter whose response was sine x over x?
It's hard to
It's hard to, why?
[UNINTELLIGIBLE]
Because it's not causal, yes.
We're going to talk about that more later today.
Communication engineers hardly ever talk about causality.
They hardly ever talk about whether something is realized or not.
And the reason I want to say this a number of times is that one of the parts of any receiver is a timing recovery circuit.
And what the timing recovery circuit does is, it tries to figure out what the transmitter timing is, in terms of what it's receiving.
And when you're figuring out what the transmitter timing is in terms of what you're receiving, the most convenient way of doing that, if you're sending a pulse or something, you would like the receiver timing to peak up at the same time that the transmitter timing is peaking up, in terms of the pulse.
In other words, you want to get rid of the propagation delay and just ignore that.
The receiver timing is going to be delayed from the transmitter timing exactly by the propagation delay.
Now, if we always do that, causality becomes totally unimportant.
Now, one of the problems with a sinc function is, it starts at time minus infinity.
And even if you delay the receiver timing by an infinite amount of time you can never use the communications system until it's time to tear it down and bring in a new one.
Which, unfortunately, is what happened to the third generation wireless systems.
By the time, people figured out how to build them, everybody was saying, ah, old-fashioned stuff, we're going to go directly onto to fourth generation.
And they might go directly from fourth generation to fifth generation.
Who knows.
Anyway, you can't implement these even with the receiver timing different than the transmitter timing.
But you can always approximate things with enough delay between transmitter and receiver.
So that you can approximate any filter you want to, without worrying about causality at all.
And it's hard enough designing good filters without worrying about casuality that you don't want to bring that into your picture at the beginning.
So communication engineers usually say that those timing issues are not important.
And a little bit of delay is not going to hurt anything.
All of Shannon's theory was as successful as it has been, and transformed the whole communication industry, because he never talked about delay at all.
He just ignored the question of delay from beginning to end.
And when you look at his capacity formulas, they are assuming that you have as much delay as you need between transmitter and receiver.
And, assuming that that isn't important, and in terms of the propagation delays, and the filtering delays in most modern communication systems, those delays are not really important.
The propagation delays you can't avoid.
So you might as well ignore them.
Which is why you build timing recovery circuits.
And the filtering delays are usually unimportant.
Relative to all of the other delays that come into these system.
So for the most part, we're just going to ignore those questions.
If you have no noise, no delay, and no attenuation, the received waveform is going to be the same as the transmitted waveform.
And then what you would like to do is to sample that and convert it back to binary.
So, that's what the receiving side of this trivial system is doing.
Now, why can you ignore attenuation?
Anybody have any idea why we might want to ignore attenuation?
Well, it's like all of these other things.
You don't ignore it, but the question you ask is, can I separate the issue of attentuation from the other issues that I want to look at now.
In other words, can I layer this problem in such a way that we can deal with our problems one by one.
And the problem of attentuation is something that communication engineers have had to deal with from day one.
In dealing with it from day one, they have dealt with all of the different ways of losing power that you have.
Which includes attenuation in the actual medium between transmitter and receiver.
It deals with the attenuation you get in the receiver by building filters and by all the other things you do there.
When you get all done with that, there's only one thing that's important.
Because you can deal digitally with very, very small signals.
And the only thing that's important is what happens to the noise.
You get noise in the communication medium.
That comes into the receiver.
And now, any time you amplify what you receive, they're going to be amplifying the noise as well as amplifying the signal.
An easy way to deal with that is to assume that there isn't any attentuation in what you're calling the signal, because you're going to amplify that to a reasonable value to operate on it anyway.
So in fact you're amplifying the noise.
So the only thing you're interested in is, what's the ratio between the signal power and the noise power.
So, those issues, we can deal with completely separately from these issues of what kind of waveforms do we want to choose.
What should we choose in place of this sinc function, which is impractical because it causes too much delay and it's also hard to build the filters.
So we'd like to deal with that question separately from the question of attenuation.
There's a short section in the notes, which I'm not going to go into, because you can read it just as easily as I can talk about it, about dB.
And why people use dB to talk about all of these questions of energy losses.
And why, in fact, there's a whole set of engineers whose function is to deal with link budgets.
In other words, when you build a communications system, you're losing power everywhere along the way.
You're losing it in the median.
You're losing it by the way you build the antennas.
You're losing some here, you're losing some there.
And what these people do is, they look at all of these attenuations together.
And they multiply together.
So, in fact, it's easier to take the logarithm of all of these terms.
And when you take the logarithm, that's where you get decibels from.
Because these people, instead of taking natural logs, which would seem like a much more reasonable thing, always take the logarithm to the base 10 and then divide by 10.
And what the section in the notes says is that that's a practice which has grown up because it's very easy to do mental arithmetic with that.
The logarithm to the base 10 of the number 2 which is one of the biggest numbers that ever appears.
2's are more important than thousands, OK?
And the logarithm to the base 2 of 10 is 0.3.
And when you divide it by 10 you get 3 dB.
So a factor of 2 is called 3 dB.
This means that a factor of 4, which is 2 squared, is 6 dB.
Factor of 8 is 9 dB, and so forth.
And this gives this table of all these numbers which all communication engineers memorize.
The other reason for it is that if you're talking to a communication engineer, he will recognize you as a member of his fraternity if you let the word dB slip several times.
Don't even have to know what it means.
You just talk about dB.
And you say, my income is 3 dB lower than your income.
And that makes him very happy.
So.
Anyway.
That's all we need for this simple example.
We now want to go into more complicated things.
OK pulse amplitude and modulation.
This is one of the major ways of turning bits into signals, and then turning the signals into waveforms again.
Again, I'm doing this first not because it's the most important scheme to talk about, but because we want to understand these things one by one.
And when we understand PAM, we'll then talk about QAM.
And you'll understand that.
And then we can go on to look at other variations of these things.
So the signals in PAM are one dimensional.
The constellation, the only thing it can be, since it's one dimensional, one real dimension, is a set of real numbers.
So you're going to modulate these real numbers.
And that, we're going to talk about later.
How do you find this function here?
We're just going to take these real numbers, which are coming into the transmitter one by one out of the digital encoder.
You're going to take these numbers.
You're going to view them as being multiplied by delayed impulses, and then pass through a filter.
And the filter's response is just, impulse response is just, p of t. In other words, what we're doing is saying, we don't like the sinc function.
Therefore, simplest thing to do is replace the sinc function by something we do like.
So we're going to replace the sinc function by some filter characteristic, which we like.
And that's the way to modify this previous example into something that makes better sense.
So we're doing two things here when we're talking about PAM.
One, we're talking about generalizing this binary signal set.
And, two, we're talking about generalizing the sinc function into some arbitrary impulse response.
So a standard PAM signal set uses equispaced signals symmetric around 0.
And if you look at the picture, it makes it clear what this means.
It's the same thing that we were using all along when we were talking about quantization.
If you're looking at one dimensional quantization, that's a very natural way to choose the representation points.
Here, we're doing everything in the opposite way.
So we're starting out with these points.
Well, we're starting out with eight symbols and turning them into eight signals.
And when you take eight symbols and turn them into eight signals, perfectly natural thing to do is to make each of these signals equispaced from each other, and center them on the origin.
This is not something we have to see later.
I think you can see that if these symbols all are used with equal probability, and you're trying to reduce the amount of energy that the signals use, which will then go through into reducing the amount of energy in the waveforms that we're transmitting, it's nice to have them centered around 0.
Because if they have a mean, that mean is just going to contribute directly to the expected mean square value of the signal that you're using.
So why not center it around 0.
Later on we'll see many reasons for not centering it around 0.
But for now we're not going to worry about any of those, and we're going to center it around 0.
And the other thing is, why do you want them to be equally spaced?
Well, I'll talk about that in just a second.
Anyway, the signal energy in these equally spaced signals, you can calculate it to be d squared times m squared minus 1 divided by 12.
I think we sort of did this when we were worrying about quantization.
There's a problem at the end of lectures 11 and 12 which guides you by the hand in how to calculate this.
Or you can sit down and just calculate it by hand, it's not hard to do.
But, anyway, that's the mean square value of these signals.
If you assume that they're equiprobable.
Now, if you take m to be 2 to the b, now what's b?
b is the number of binary digits which come into this signal former when you produce one signal out.
Namely, if you bring in b binary digits, you need an alphabet of size 2 to the b.
If you have an alphabet of size 2 to b, then you're going to need m equals 2 to the b different signals.
So, usually when you have a standard PAM system, that number there, 8 PAM, means 8 signals.
8 is usually going to be replaced by 4, or 16, or 32, or 64, or what have you.
And, usually, not anything else.
Because you're usually going to deal with something which is a power of 2.
Because the logarithm of this to the base 2 is the number of bits which are coming into the signal former for each signal that comes out.
This goes up very rapidly as m squared goes up.
In other words, you try to transmit data faster by bringing more and more bits in per signal that you transmit, it's a losing proposition very, very quickly.
It's this business of a logarithm which comes into everything here.
We're going to talk about noise later.
We're not going to talk about it now.
But, we have to recognize the existence of noise enough to realize that when you look at this diagram here, when you look at generating a waveform around this, or a waveform around this, however you receive these things, noise is going to corrupt what you receive here by a little bit.
Usually it's Gaussian, which means it tails off very, very quickly.
With larger amplitudes.
And what that means is, when you send a three, the most likely thing to happen is that you're going to detect a three again.
The next most likely thing is you'll detect either a four or a two.
In other words, what's important here is this distance here.
And hardly anything else.
If you send these signals with equal probability, and the noise is additive, the noise does the same thing no matter where you are along here.
Which says that this standard PAM set is almost the only thing you want to look at.
So long as you assume that the noise is going to be additive.
And the noise is going to affect everything along this line equally.
It says that you just want to make the spacing between points big enough that it will pretty much avoid the noise.
So, the point of all of that is that d is fixed, ahead of time.
You can't play with that.
You can play with m. When you play with m, you're playing a losing game with energy.
So that's why standard PAM is the thing which is used with PAM.
Not much you can do about it.
And say here that the noise is independent of the signal.
We will talk about this later.
The noise being additive.
The noise being independent of the signal are both saying almost the same thing.
Which will be obvious to you in a little while.
But not until we start talking about noise.
We don't want to start talking about noise now.
So, for now, we deal with the noise just by saying that every signal must be separated by some minimum amount.
Again, what we said about delay and attenuation.
Let me say it again, because it's important enough that you have to understand it.
After you go through two or three problem sets, you will not understand it again.
Because you'll get so used to dealing the receive signal as the same as the transmitted signal that you'll forget the weirdness in that.
I mean, people become accustomed to extraordinarily weird things very, very easily.
And especially when you're taking a course and trying to get the problems done, you just take things which are totally ridiculous and accept them if it lets you get through the problem set.
So I want to tell you right up front that there is some weirdness associated here.
It is something you have to think about.
After you think about it once, you then accept this is a layering decision.
You ignore delay, since the timing recovery locks the receiver clock to the transmitter clock plus the propagation delay.
And in fact, it can lock the receive clock to any place that wants to lock it to.
So we're going to lock it in such a way that the receive signal looks like the transmitted signal.
And the attenuation is really part of the link budget.
We can separate that from all the things we're going to do.
I mean, if we don't separate that, you have to go into antenna design.
And all this other stuff.
And who wants to do that?
I mean, we have enough to do in this course.
It's pretty full anyway.
So we're just going to scale the signal and noise together.
And that's a separate issue.
So now we want to look at the thing which is called PAM modulation.
In other words, in this one slide we separated the question of choosing the signal constellation, which we've now solved by saying, we want to use signals that are equally spaced.
So that's an easy one.
From the question of, how do you choose the filter.
So the PAM modulation is going to go by taking a sequence of signals, mapping it into a waveform, which is this expansion here.
We're not assuming that these functions are orthogonal to each other.
Although later, we will find out that they should be.
But for now, p of t is just some arbitrary waveform.
And we will try to figure out how to choose this waveform in such a way as to replace the sinc waveform with something which is better in terms of delay and almost as good in all other ways.
We're not going to worry about the fact that p of t has to be realizable.
Because with our arbitrary time reference at the receiver, it doesn't have to be realizable.
It doesn't have to be causal.
So, p of t could be sinc of t over t. Which would give us a nice baseband limited function.
But it could be anything else at all.
As we said before, sinc t over t a dies out impractically slowly.
And it requires infinite delay at the transmitter.
You can't even send these signals with a sinc t over t signal.
Because to send them, you have to start out at the beginning of this little bit of wiggling.
Now, you say, OK I'm going to truncate that when it's very small.
And I don't worry about that.
The point of what we're starting to look at now, though, is we're saying, OK, you truncate it, you do all these practical things.
But it turns out that this problem of choosing this filter response has a very elegant and a very nice solution.
And when we put noise in, it fits in perfectly with the idea of also choosing this filter in a particular way.
Now, we've talked about many problems here which were solved almost immediately after Shannon came out with his way of looking at communication in 1948.
Guess when this problem was solved?
It was solved 20 years earlier than that by a guy by the name of Nyquist, who was at Bell Labs back when Bell Labs meant something.
I mean, in `28 it was a great place.
It was a great place until seven or eight years ago.
Nyquist is important in feedback theory.
He's done some of the most important things there.
His Nyquist criterion in dealing with how do you choose these filters to avoid intersymbol interference is fairly simple.
But it's a very, very nice result.
So we're going to talk about it.
And then we will use that to say, ok, we don't have to worry about intersymbol interference any more, so all we have to worry about is noise.
So we're getting rid of these problems one by one.
Our main problem is to choose this filter, so that we get some kind of reasonable compromise between time delay and bandwidth.
That's -- that's basically the problem that we're facing.
And Nyquist's solution to this was to say, forget about that also.
Let's look at what set of filters work, and what set of filters don't work.
And then you can take your choice among this set of folders that work.
So our problem is, how do you take the waveform u of t, which you receive, and find these samples out of it?
Now, we already know that if you use sinc functions, all you have to do is sample this and you get these u sub k's back again.
But if this thing is some absolutely wild waveform, maybe that's not what you get.
So we say, OK, how do we retrieve these signals from the waveform that was transmitted and therefore from the waveform that was received.
We're separating this from the noise question.
Even if there's no noise at all, we still have a question and how do you find those input values directly from the function.
What we're going to do is to assume that the receiver filters u of t, with a linear time invariant filter, with impulse response q of t. Now, since we've said that p of t doesn't have to be causal, we might as well say that q of t doesn't have to be causal either.
Because we've already thrown these details of delay to the winds.
So the filtered waveform, then is going to be r of t. Will be the integral of what was transmitted, convolved with q of t. So this is what you receive.
And then what we're going to do is, we're going to sample this waveform.
So Nyquist said, let's restrict ourselves to looking at receivers which first filter by some filter we're going to decide on, and then sample.
Why do you want to do that?
Well, an interesting question.
And Nyquist said, that's what we're going to do.
And since Nyquist was famous, that's what we're going to do.
One of the problems in the homework this week is to show that if you relax this a little bit and you say, well, I don't want to do what Nyquist said, what I want to do is to look at an arbitrary linear receiver which takes this received waveform, goes through any old linear operations that I want to go through, and solves for what -- and from that, tries to pick out these coefficients.
And the question is, can you do anything more than what Nyquist did.
And the answer is, no.
If you look at it in another way, you will find that in fact, if you know what the signal constellation is, you can look at non-linear receivers.
Which in the absence of noise will let you pick out these coefficients in a much broader context than what Nyquist said.
And why don't we do that?
Well, because when noise comes in, that doesn't work at all.
That's a lousy solution.
So what we're going to do is say, OK, Mr. Nyquist, we'll play your silly game.
We will have this filter at the transmitter.
We'll have this filter at the receiver.
We'll have the sampler and we'll look at what conditions we need in order to make the whole thing work.
And then we will fit it in with noise and everything.
It will fit in, in a very nice way.
So we have a nice layered solution when we do that.
And we will find -- I mean, Nyquist had some of Shannon's genes in him, I think.
Because what we find when we're all finished with this is that by avoiding -- by being able to receive these coefficients perfectly, which we'll refer to as avoiding intersymbol interference, it doesn't hurt us at all as far as taking care of the noise.
In other words, you can have your cake and you can eat it too as far as intersymbol interference and noise is concerned.
We wind up with a received waveform, which is the integral of the transmitted waveform times some filter.
And we don't know how to choose this filter.
But let's just -- this is a filter.
This can be represented now in terms of u of t is equal to this transmitted waveform in terms of this other filter that we don't understand.
So we now have two filters that we don't understand.
And we have this integral here.
Well, we can take this sum.
We can bring this sum outside of the integral and have the sum of u sub k times, just some composite filter g of t where g of t is the convolution of p of t and q of t. Now, when you look at the notes, the notes are fairly careful in dealing with all these questions of L2 and convergence and all of this stuff that we've been talking about.
Again, when you're trying to understand something for the first time, ignore all those mathematical issues.
Try to find out what's going on.
When you get an intuitive sense of what's going on, go back and look at the mathematics then.
But don't fuss about the mathematics at this point.
OK, so r of t, then, is just going to be the sum over k of these sample values that are coming in, times this composite filter, which is the convolution of the transmit filter and the recieve filter.
Now, this shouldn't be surprising.
If you think of u of t as being formed by taking a sequence of impulses, and then passing that sequence of impulses through a filter p of t, and then passing the output through a filter q of t, all you're doing is passing this sequence of impulses through the convolution of p of t and q of t. In other words, in terms of this received waveform, it couldn't care less what filtering you do at the transmitter and what filtering you do at the receiver.
It's all one big filter as far as the receiver is concerned.
When we study noise, what happens with the transmitter and what happens with the receiver will become important again.
But, so far, none of this makes any difference.
And this is all we need to worry about.
Then, Nyquist said, a waveform g of t is ideal Nyquist.
He didn't call it ideal Nyquist, he was very modest person.
But since then, people call it ideal Nyquist because he was the guy that sorted it all out.
It's ideal Nyquist with period t, and I usually leave out the period t because that's usually understood, if g of k t is equal to delta of k. In other words, if g of 0 is equal to 1 and g of k times t, for every non-zero integer k is equal to 0.
In other words, if g has the same property that the sine function has, sine function is 1.
It's 0.
And it's 0 at every integer point beyond that.
And Nyquist said, well, gee, all we need to do is make this filter has a property.
And when you look at this, it's fairly obvious that that works, right?
I mean, we want a sample RFT -- say, at j times capital T. Or, if j times capital T, it's going to be the sum here of u sub k times g of j t r of t. r of j t is equal to sum over k of u k times g of j t minus k t. If the waveform is ideal Nyquist, then this quantity is 0 for all integer k except when k is equal to j.
So, this is just equal to u sub j.
And conversely, if this waveform is not ideal Nyquist, you have the problem that you can pick two values.
u sub k and u sub j, in such a way that they interfere with each other.
In other words, that they add up at some sample point to the wrong value.
One of them is going to come in and clobber the other.
So, this is a necessary and sufficient condition for avoiding intersymbol interference.
So long as you're not smart enough to look at what those actual values are.
In other words, so long as you're only going to use a linear receiver, which is what that amounts to.
While ignoring noise, r of t is determined by g of t, p of t and q of t otherwise irrelevant.
That's what we said.
We said this.
If t of t is ideal Nyquist and r of k t equals u of k for all k and z.
If f of t is not ideal Nyquist, and r f k t unequal to u k for some k and some choice of the sequence.
Now, so far there's no big deal here.
This is pretty easy to figure out.
You don't need rocket science to say, once I pose the problem this why, which is where part of Nyquist's genius came from.
The hard thing is always to post the right problem.
It's not to solve it.
Those of you who want to do Ph.D theses, believe me, 80% of the problem is finding the problem.
20% of the problem is doing it.
If you do a really outstanding thesis, 99% is finding the problem and 1% of it is doing it.
And I believe that.
I'm not exaggerating.
An ideal Nyquist g of t implies that no intersymbol interference occurs at the above receiver.
In other words, you have a receiver that actually works.
We're going to see that choosing g of t to be ideal Nyquist fits in nicely when looking at the real problem, which is coping with both noise and intersymbol interference.
And we've also seen that if g of t is sync of t over capital T, that works.
It has no intersymbol interference because that's, one, at t equals 0, and it's 0 at every other sample point.
We don't like that, because it has too much delay.
We want to make g of t strictly baseband limited to 1 over 2t.
And this turns out to be the only solution.
And we'll see that in a little while.
In other words, if you want to do something which has better delay characteristics than the sinc function, your only way of doing it is spilling out into slightly higher frequencies.
So, what Nyquist really wanted to find out, although you won't find that out by reading his paper because it's all this nice mathematical proof, is how much do you have to expand the bandwidth in order to get nice delay things which do the same thing.
Well, we think about it a little bit.
And we have an advantage that Nyquist didn't have.
Because we understand about aliasing.
Nyquist didn't understand about aliasing.
Aliasing hadn't been done at that time.
It was probably done as a result of what Nyquist had done.
And if we look at the reconstruction from the samples, g of k t of g, we get this function s of t, which is the sum of all the samples times sinc of t over t, minus k. That's for an arbitrary filter.
This baseband reconstruction by definition, when we were talking about alising, is just this function.
We've said that g of t is ideal Nyquist, if and only if s of t is equal to sinc of t over capital t. Why is that?
You look at this function here.
You look at s of 0.
And what do you get?
You get the sum of g of k t times sinc of t over t minus k. If we have an ideal Nyquist filter, then all of these terms are 0 except when k is equal to 0. s of t, in general, if you have a ideal Nyquist filter, you only have one term in here.
So s of t is just equal to sinc of t over t. Because all those other terms go away.
If we take the Fourier transform of s of t equals sinc t over capital T, the Fourier transform is s s tilde of f is equal to t times the rectangle function of f times t. A sinc function of a Fourier transform is a rectangle.
And the aliasing theorem then says that this Fourier transform of this low pass representation has to be equal to this sum of different frequency terms.
That's what aliasing says.
It says that this baseband representation is aliased into by all of these other frequency bands.
And each of them come in and add to what you get in frequency here.
Remember that diagram that we drew where we took this arbitrary frequency function and we picked up what was in each band, then we stuck it into the center band and then we added all these things up?
That's what the aliasing thing says.
So it says that g of t is ideal Nyquist if and only if this sum is equal to that.
And that's the Nyquist criteria.
That's what Nyquist did.
But he did it long before anybody had heard of aliasing.
There's a slightly easier way to look at aliasing criterion, which now becomes the Nyquist criterion.
When what you're interested in is a waveform g of t, which is almost band-limited but not quite.
So what we're going to assume is that it's limited to, at most, twice this -- this w here is 1 over 2t, which is called the Nyquist bandwidth.
Everything is called Nyquist here, so.
This value here is the minimum bandwidth you could be using.
It's 1 over capital 2t.
It's what you would get if you were using the sinc function.
If you were using the sinc function, what you would get is something which is a rectangle here.
Cut off, right at this point.
And cut off right at this point.
Nyquist is saying, suppose it's limited to at most 2w.
In other words, suppose you have a slopover into other frequencies but, at most into the next frequency band and no more than that.
Then, if you look at this thing, which is spilling out, and we take the same picture we were looking at before, we take this quantity.
Bring it back down here.
We take this quantity, bring it up here.
And what do we get?
This, added into here.
This thing adds up right there.
In other words -- well, let's take this one here.
This thing here gets translated over to here.
And added to this.
This is assuming that g hat of f is real, and we're ignoring the complex part of it.
So this gets added to this.
This is just enough to turn this into something, which goes across here and down here.
Down here.
My finger is not perfect.
But, anyway, when you add this to this, you get this ideal rectangular shape.
What this is saying, in terms of just this upper side band here, this is going to be the same as this, from symmetry.
So it's saying, if you take what's on the positive side of w, and you rotate it around this way, you rotate it around up here, if it's just enough to fill that in, then you've satisfied the Nyquist criteria.
In other words, you want band edge symmetry here.
You want this to be symmetrical to this.
In that rotated 180 degree sense.
So it says that anything which has this band edged symmetry condition satisfies the property of no intersymbol interference.
So this makes the problem easy for us.
It says, we would like to have a rectangular function.
That has too much delay, because in particular the inverse Fourier transform of that, because we have a discontinuity, can drop off, at most, as 1 over t. Which is pretty slow.
We've gotten rid of the discontinuity.
We've also gotten rid of the slope discontinuity, the way I drew the figure here.
And, therefore, we can wind up with a function which decays as 1 over t cubed.
Which is a whole lot better than 1 over t. And, excuse me for going a little over, but.
The things people build in practice usually have something called a raised cosine filter.
Which is this messy expression here, for the frequency response.
But it really isn't that bad.
In fact, it's just what this is.
This frequency response is t over most of the frequency interval, up to 1 over 2t.
It's t times a raised cosine over this part of the frequency band here.
t times cosine squared, and the cosine squared just raises things up to be average out at 1/2, and it's 0 everywhere else.
So what you're doing here in that formula is simply making things t up to here.
Making it a raised cosine here.
And making it 0 everywhere else.
And, depending on what do choose as alpha, that makes this sharper or less sharp.
And people usually choose it to be about, alpha to be about 15% or so.
Which means these filters chop off very, very rapidly.
Is that hard to build?
Doesn't make any difference.
I mean, these days, anything which you can figure out how to compute, you can put it on a little chip and it costs nothing if you make enough of them.
So you want to raise the cosine filter which cuts off at 15%, fine.
Somebody spends a year designing it.
And then you cookie-cut it forever after.
So it doesn't cost anything.
And, well, g of t also has an inverse transform which we won't worry about.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to pretty much finish up today talking about modulation.
Part of the reason we don't spend much time on this is, I think most of you have seen some kind of undergraduate class in communication or signal processing or something.
Where you do a lot of this because a lot of this is just nice exercises and -- well, nice or not nice, depending on the way you about it -- in multiplying waveforms by cosines and by sines and by complex exponentials.
And you keep doing this, and doing this, and doing this and you get these long expressions.
And and it all means something.
So, most of you have seen a good deal of this.
You probably haven't seen the Nyquist criterion before.
I'm sure you haven't seen the Nyquist criterion done carefully, because I've never seen it done carefully before.
I don't think it is done carefully anywhere else.
And I'm sure you don't care about this.
But, at some point, if you deal with this stuff, you will care about it.
Because at some point you will need it.
But anyway, we were looking at pulse amplitude modulation, you remember.
We were looking at the modulated waveform.
And this is all done down at baseband, now, remember.
And the interesting problems down at baseband are, first, how do you choose a signal set.
Namely, how do you choose those quantities u sub k, out of some set of possible values.
So they have an appropriate distance between them.
And, next, how do you choose this waveform p of t, which is called the pulse, that you're using.
And then, after you've chosen those two things, there's nothing more to be done.
You simply form the modulated waveform as u of t equals the sum of the signals which are coming in regularly at intervals of t. You multiply each one of them by this waveform, p of t. Or, alternatively, you think of the waveform as being a sum of impulses.
Each impulse weighted by u sub k. And you think of passing this string of impulses through a filter with impulse response p of t. Which is usually the way that it's implemented, except of course you're not using ideal impulses.
You're using sharp pulses which have a flat spectrum over the bandwidth of the filter p of t. But anyway, somehow or other you implement this.
And that is this fundamental step we've been talking about for quite a while now, of how do you turn sequences into waveforms.
How do you turn waveforms into sequences, it's a fundamental piece of source coding.
It's a fundamental piece of channel coding.
Here we're doing the channel coding part of it.
Then we did something kind of flaky.
We said that when we received this waveform, what we were going to do is to first pass it through another filter, and then sample it.
If you've done the homework yet, you will probably have looked at what happens when you take an arbitrary linear operation on this received signal to try to retrieve what these signals u sub k are.
And you will have found that this filtering and sampling is in fact not a general linear operation, but it's the only general linear operation that you're interested in as far as retrieving these signals from that waveform.
So, in fact, this is, in fact, more general than it looks.
There a confusing thing here.
If you receive u of t at the receiver, and your question is, how do we get this sequence of samples u sub k out of it, and suppose that this pulse p of t or the -- I mean, suppose for example that the pulse is a narrow bandwidth pulse, and there's just no way you can perform linear operations and get these signals out from it.
Is it possible to do nonlinear operations and get the signals out from it?
And you ought to think about this question a little bit because it's kind of an interesting one.
If I don't tell you what the signals u sub k are, if I ask you to build something which extracts these samples u sub k, without any idea of what signal set they're taken from, then there's nothing you can do better than a linear operation.
And, in fact, if this pulse p of t has a bandwidth that's too narrow, you're just stuck.
If I tell you that these u sub k are drawn from a particular signal set -- for example, suppose they're binary -- then you have an enormous extra amount of information.
And you can, in fact, given this waveform, even though p of t is a very, very narrow band, you can still in principle get these binary signals out.
So what's the game we're playing here?
I mean, we're doing kind of a phony thing.
We're restricting ourselves to only linear operations.
We are restricting ourselves to retrieving these signals without knowing anything about what the signal set is.
Which is not really the problem that we're interested in looking at.
So what are we really doing?
What we're really doing is trying to look at this question of modulation before we look at the question of noise.
And after we start looking at noise, the thing that we're going to find is that this received waveform is a received waveform plus a lot of noise on top of it.
And if there's a lot of noise on top of it, these non-linear operations you can think of are not going to work very well.
And I guess the problem is, you just have to take that on faith right now.
And after we look at random processes, and after we look at how to deal with the noise waveforms, you will in fact see that these operations we're talking about here are exactly the things we want to do.
Now, I don't know whether this is the right way to do it or not.
Dealing with all this phony stuff that you see in elementary courses before dealing with the real stuff.
I think it probably is, because for most of you this is sort of familiar and we'll come back later and make it all right.
But who knows?
Anyway.
what we want to do then is to find some composite filter, g, which is what happens when you take the filter p, or the pulse p, pass it through a filter q of t, what you get is the convolution of p and q.
And, therefore, what comes out after this filtering is done is a received waveform which is just a sum over k of u sub k times g of t minus k t. In other words, these two filters are not doing anything extra for you.
All they are is a way of putting part of the filter at the transmitter, part of the filter at the receiver.
When you look at noise you'll find out that there is some real difference between what's done at the transmitter and what's done at the receiver, because the noise comes in between the transmitter and the receiver.
But for now, it doesn't make any difference so the only thing we're interested in is the properties of this composite waveform, g of t. And what we find is that if we receive r of t, which is this waveform sum of u k g of t minus k t, and if we want to retrieve the coefficient u sub k, it becomes duck soup to do so if in fact this waveform is like a sampling waveform.
In other words, if g of t is equal to 1 at t equals 0, and it's equal to 0 at each other sample point, then all we have to do is take this waveform, simply sample it each capital T seconds, and we get these coefficients out automatically.
Now, I should warn you at this point that in the notes the scaling business is not done quite right.
Sometimes we talk about g of t as a filter whose shifts are orthonormal to each other.
And sometimes as a filter whose shifts are orthogonal to each other.
I advise you not to worry about that, because you have to make changes about five times in the notes to make it all right.
And I will put up a new version of the notes on the web which in fact does this right.
It's not important.
It's just this old question of, do you use a sinc function when you're dealing with strictly band-limited functions, or do you multiply the sinc function by 1 over the square root of t to make it orthonormal.
Or do you multiply it by 1 over t to make it -- I mean, you can scale it in a number of different ways.
And, fundamentally, it doesn't make any difference.
It's just that if you want to get the right answer you have to scale it the right way.
And it's not quite right in the notes.
So I'll change it around and send the thing out to you.
Then it says that T-spaced samples of r then reproduce u sub k without intersymbol interference.
The Nyquist criterion is different from this business of the pulse being ideal Nyquist.
Ideal Nyquist is talking about the time domain.
It simply says the trivial thing you, want a pulse which has 0's at every sample point except for the sample point you're interested in.
The Nyquist criterion translates that ideal Nyquist property, in time, to a property in frequency.
And it says that the frequency, that the Fourier transform of g of t has to satisfy this relationship here.
And there's this added condition on g of f that it has to go to 0 fast enough as f goes to infinity.
But we won't worry about that today.
So the condition is this: the picture that I showed you last time is this.
We defined the nominal band, the Nyquist band, as the base bandwidth w equals 1 over 2t.
That's the bandwidth that a sinc pulse would have if you were using a sinc pulse.
The actual baseband limit, b, should be close to w. And most of the work that people do trying to build filters and things like this, since what we're trying to do is find a waveform that we're trying to transmit.
And we're stuck with the FCC and all these other things that say, you better keep this band-limited.
What we're going to do is to make the actual band of the waveform close to the nominal bandwidth.
So we're going to assume that it's less than twice the nominal bandwidth.
In other words, you can have a little bit of slop, but you can't have too much.
When you try to design a filter that way, and you satisfy the Nyquist criterion, which is talking about all of these bands all the way out to infinity, and the fact they have to add up in a certain way, if you only have this band here and part of the next band -- if this function has to go to 0 before you get out to 2w, then the only thing you have to worry about is what does this waveform look like here.
You take this and you pick it up.
And you put it over here, and you add it up.
You take this, you put it over here and you add it up.
And if the pulse, p of t is real, then what you have over here is the complex conjugate of what you have here.
And if you make this real in frequency also -- in other words, you have symmetry in time and symmetry in frequency, then this band edge symmetry requirement is that when you add this to this, you get something which is an ideal rectangular pulse.
Now, if you think of taking this waveform here -- now you only have to worry about the positive frequency part of it.
And you take that point right there, which is halfway between 0 and t. In other words, this is at t over 2.
And it's also at frequency w. And you rotate this thing around by 180 degrees, then this comes up there.
And it fills in this little slot up here.
That's what the band edge symmetry means.
Yes?
[UNINTELLIGIBLE]
Ah, yes.
We could, if we wanted to, put various notches in this filter.
But we've defined the bandwidth, b, as the largest frequency, f, such that g hat of f is 0 beyond b.
In other words, everywhere beyond b, g hat of f is equal to 0.
Now, if g hat f cuts down to 0, say, back here, there's no way you can meet the Nyquist criterion.
Because there's no way you can build this thing up with all these out-of-band components so that you get something which is flat all the way out to w. So you simply can't have a filter which is band-limited to a frequency less than w. What you need is to use these out-of-band frequencies as a way to help you construct this ideal rectangular pulse.
Through aliasing.
In other words, the point here is when we're doing transmission of data, we know what the data is.
We know what the filter is, and we can use aliasing to our advantage.
When we were talking about data compression, aliasing just hurt us.
Because we were trying to represent this waveform that we didn't have any control over.
And the out-of-band parts added into the baseband parts and they clobbered us.
Because we couldn't get the whole thing back again.
Here, we're doing it the other way.
In, other words we're starting out with a sequence.
We're going to a waveform, and then we're trying to get the sequence back from the waveform.
So it's really the opposite kind of problem.
And here, the whole game, namely, the thing that Nyquist spotted, back in 1928, was you could use these out-of-band frequencies to, in fact, help you to get rid of this intersymbol interference.
Because all you need to do is make these things add up to this so that you have something rectangular.
And then when you do the samples, you have to write samples.
Now, the problem that a filter designer comes to when saying this is to say, OK, how do I design a frequency response which has the property that it's going to go to 0 quickly beyond w. Because the FCC, when we translate this up to passband, is going to tell us we can't have much energy outside of minus w to plus w. And if we can't design a good filter, it means we have to make w smaller.
So we can keep ourselves within this given bandwidth.
And we don't want to do that because that keeps us from transmitting much data.
And then we can't sell our product, so suddenly we have to design something which uses all of this bandwidth that we have available.
So what we want to do, then, is to design something where b is just a little bit more than w, but where also we get from t down to 0 very quickly.
We could just use the square pulse to start with.
And what's the trouble with that?
This rectangular pulse, its inverse Fourier transform is the sinc pulse.
The sinc pulse, because a discontinuity in the Fourier transform, it can't go to 0 any faster than is 1 over t. And suddenly it goes to 0 as 1 over t goes to 0 very, very slowly.
In other words, you have enormous problems over time.
And you have enormous delay.
And since you have so many pulses adding up together, everything has to be done extraordinarily carefully.
So what you want is a pulse which remains equal to t over a y bandwidth here.
Which gets down to 0 very, very fast.
So the problem is, how do you design something which gets from here down to here very, very quickly and very smoothly.
You want it to go smoothly because if you have any discontinuities in g of f, you're back to the problem where g of t goes to 0 as 1 over t. If you have a slope discontinuity, g of t is going to go to 0 as 1 over t squared.
If you have a second derivative discontinuity, it's going to go to 0 as 1 over t cubed.
Now, 1 over t cubed is not bad, and filter designers sort of live with that.
So they design these filters which are raised cosine filters, which over the band here -- someday I'll get a pen that works -- are flat.
Over this band here, from here to here, this is a squared cosine, analytically.
And a squared cosine is just the same as a cosine which you take and displace up so it's centered right there.
Well, excuse me, it's the same as a sine which is centered there.
Which is what you get when you square a cosine pulse as 1/2 plus.
Anyway, it looks like this.
So our problem is, how do you design a filter which gets from here to there quickly but where the inverse transform also goes to 0 relatively quickly?
Now, if you want to do this and you also face the fact that in a Nyquist criterion, any part of g hat of f which is imaginary, the Nyquist criterion says that what do you have to do in-band and what you have to do out of band have to add up there also.
That doesn't help you at all in getting from this real number t here down to 0.
So anything you do as a complex part of g hat of f is just wasted.
I mean, your problem is getting from t to 0 with a smooth waveform.
You would like to make g hat of f strictly real.
You would like to make it symmetric.
Why would you like to make it symmetric?
Because this thing down here and this thing up here are really part of the same problem.
If you find a good way to make a function go to 0 quickly up here, you might as well use the same thing over here.
So you might as well wind up with a function which is symmetric and real.
That leads us into the next thing we want to look at.
That's a slightly flaky argument.
We're going to find a better argument as we go.
What we've said is, the real part of g hat of f has to satisfy this band edged symmetry condition.
Choosing the imaginary part unequal to 0 simply increases the energy outside of the Nyquist band.
You don't get any effect on reducing delay out of that.
Thus, we're going to restrict g of f to be real.
And we're going to also restrict it to be symmetric.
Although that's less important.
Now, when we start to look at noise, we're going to find out something else.
We're going to find out that we want to make the magnitude of p of f equal to the magnitude of g of f. Now, magnitude doesn't make any difference.
So we want the frequency characteristic of p of f to be the same as the frequency characteristic of q of f. In other words, there's no point descending a p of f which is a perfect sinc function and then using a very sloppy q of f. Because that's kind of silly.
There's no point to using it very sloppy p of f and then using a very sharp q of f, because somehow when you start looking at noise, you're going to lose everything.
Because the noise gets added to this pulse that you're transmitting.
So, what we're going to find when we look at this later, is we really want to choose the magnitudes of these to be equal.
Since g hat of f is equal to this product, and since we've already decided we want to make this real, what this means is that q hat of f is going to be equal to p complex conjugate of f. What that means is the filter q of t should be equal to the complex conjugate of p of minus t. You take a p of t, and you turn it around like this.
If p of t is real, this is called a matched filter.
And it's a filter which sort of collects everything which is in p of t, and all brings it up to 1p, which is what we would like to do here.
So, anyway, when we do this it means that g of t is going to be this convolution of p of t. And q of t, which we can now write as the integral of p of tau, times p complex conjugate of t minus tau, p tau.
And what we're interested in is, is this going to be ideal Nyquist or not.
And what does that mean if it is ideal Nyquist?
If g of t is ideal Nyquist, it means that the samples of g of t, times k times this signaling interval, t, have to have the property that these samples are equal to 1 for k equals 0, and 0 for k unequal to 0.
What does that mean?
If you look at this, that kind of looks like these orthogonality conditions that we've been dealing with, doesn't it?
So that what it says is that this set of functions, p of t minus k t. In other words, the pulse p of t and all of it shifts by t, 2t, 3t, and everything else.
This set of pulses all have to be orthogonal to each other.
And the thing which is a little screwed up in the notes is whether these are orthogonal or orthonormal, or what.
And you need to make a few changes to make all of that right.
These functions are all real L2 functions.
But we're going to allow the possibility of complex functions for later.
In other words, if we're transmitting a baseband waveform on a channel, how do you transmit an imaginary waveform?
Well, I've never seen anything in electromagnetics, or in optics, or anything else, that lets me transmit an imaginary waveform.
These are not physical.
We will often think of baseband waveforms that are imaginary.
Real and imaginary complex.
And when we translate them up to baseband, we'll find something real.
But the actual waveforms that get transmitted are always real.
There's no way you can avoid that.
That's real life.
Real life is real.
That's why they call it real, I guess.
That's why they call it real life.
I don't know.
I mean it's more real than something imaginary, isn't it?
So, anyway, what gets transmitted is real.
But we'll allow p of t to be complex just for when we start dealing with something called QAM, which is our next topic.
So in vector terms, the integral of u of tau times q of k t minus tau is the projection of u of t onto this waveform.
And partly for that reason, q of t it's called the matched filter to p of t. In other words, you use this waveform here as a way of selecting out the parts of this waveform u of tau, u of t, that we're interested in.
So that any way you look at it, we're going to use a pulse waveform, p of t, which has this property that its shifts are all orthogonal to each other.
When we start studying noise, you will be very thankful that we did this.
Because when you use pulses that are orthogonal to each other, you can break up the noise into an orthogonal expansion.
And what goes on at one place is completely independent of what goes on at every other place.
And we'll find out about this as we go.
But, anyway, we have the nice property now, that anytime we find a function g of t, that satisfies the Nyquist criterion And any time we choose p of t and g of t so that their Fourier transforms have the same magnitude, then presto, we have freely gotten a set of orthonormal functions.
Which just comes out in the wash. Before we worked very hard to get these truncated sinusoid expansions and sinc weighted sinusoid expansions, and all of this stuff to generate different orthonormal sets of waveforms.
Suddenly, we just have an orthonormal set of waveforms and a very large set of orthonormal waveforms popping up and staring us in the face here.
And in fact these are the waveforms we're going to use for communication, so they're nice things.
Nobody uses sinc functions for communication.
Nobody uses rectangular functions.
You can't use either one of them because rectangular functions have lousy frequency characteristics.
Sinc functions have lousy time characteristics.
These functions p of t are sort of nice compromises.
And they're orthonormal, again.
Let's go on to the rest of modulation.
We've been talking about baseband modulation.
And when we're thinking about PAM, pulse amplitude modulation, we are thinking in terms of this sequence of symbols coming in.
The symbols being turned into signals.
The signals been turned into waveforms.
And what comes out here, then, is some baseband waveform.
That's what the Nyquist criterion is designed for.
How do you make a baseband waveform which is very sharply cut off in frequency?
Usually what we want to transmit is something at passband, so we somehow want to take this baseband waveform, convert it up to passband.
We're then going to transmit it on a channel.
I mean, why do we have to turn it into passband anyway?
Well, if you did everything at baseband, you wouldn't have more than one channel available.
I mean, wireless, you know you have all these different channels.
The way it's done today, you use something called CDMA, where you're not breaking into narrow channels.
But you should understand how to break it into narrow channels before understanding how to look at it as co-division multiple access.
If you're using optics, you want to send things in different frequency bands.
Whether it's optics or electromagnetics simply determines the frequency band you're looking at anyway.
You can't propagate things in every frequency band.
Things don't propagate very well at baseband.
So for all of these reasons, we want to convert these baseband waveforms to passband.
Why don't we generate them originally at passband?
Because things are changing too fast there.
I mean, you want to do digital signal processing to massage these signals, to do all the filtering.
To do most of the other things you want to do.
And you can do those very easily at baseband.
And it's hard to do them at passband.
So the generic way things are done is to first take a signal sequence.
Convert it into a baseband waveform.
Take the baseband waveform, convert it up to passband.
And the passband is appropriate to whatever the channel is.
You send it.
You take it down from passband back to baseband, and then you filter and sample and get the waveform back again.
You don't have to do this.
We could generate the waveform directly at passband.
There's a lot of research going on now trying to do this, which is trying to make things a little bit simpler.
Well, it's not really trying to make things simpler.
It's really trying to trying to pull a fast one on the FCC, I think.
But, anyway, this is being done.
And it doesn't go through this two-step process on the way.
So it saves a little extra work.
So what we're going to do with our PAM waveform, them, we're going to take u of t, which is the PAM waveform.
And I'm sure you've all seen this.
I hope you've all seen it somewhere or other.
Because everybody likes to talk about this.
Because you don't have to know anything to talk about this.
So you take u of t. We multiply it by e to the 2 pi i f c t. In other words, you multiply it by a sine wave.
A complex sine wave.
When you do this, this thing is complex.
You can't transmit it.
So what do we do about that?
Well, this is real.
This is complex.
If we add the complex conjugate of this, this plus its complex conjugate is real again.
So we transmit this times this complex sinusoid, plus this other complex sinusoid, which is the complex conjugate of this.
And you get 2 u of t times the cosine of 2 pi f c t. Which is just what you would do if you were implementing this anyway.
You take the waveform u of t, you multiply it by cosine wave at the carrier frequency.
And bingo, up it goes to carrier frequency.
This is real.
This was real.
And everybody's happy.
And in frequency, what this looks like, since all we're doing here is just, by this shift formula that we have for Fourier transforms, the multiplying a time waveform by an exponential -- by a complex sinusoid, is simply is the same as shifting the frequency response.
So the Fourier transform of this is u hat of that minus f c. The Fourier transform of u f t times this is u hat of f the plus f c. And you start out with this waveform, whatever that shape is here.
This, I tried to draw to satisfy the Nyquist criteria, which it does it.
Satisfies that band edge symmetry condition.
So this gets shifted up.
It also gets shifted down.
And the transmitted waveform then exists in the band which I'll call b sub u. b sub u is the bandwidth of u of t. Namely, it's this baseband bandwidth that we've been talking about.
But, unfortunately, when we do this thing shifted up and this thing shifted down, the overall bandwidth here is now twice as much as it was before.
Now, every communication engineer in the world, I think, measures bandwidth in the same way.
When you talk about bandwidth, you're always talking about positive bandwidth.
Because, back a long time ago, communication engineers didn't know about complex sinusoids.
So everything was done in terms of cosines and sines.
Which was very good, because back then communication engineers didn't have much else to do.
So they had to learn to write everything twice.
And now, since we have so many other things to worry about, we want to use complex sinusoids and only write things once.
Well, in fact we have to write it twice here, but we don't write it twice very often.
But, anyway, when this thing, which exists for minus b u up to plus b u gets translated up in frequency, we have something which exists from f c minus b u to f c plus b u.
And this negative band is down here.
Now, we're going to assume everywhere, usually without talking about it, that when we modulate this up in frequency, that the bandwidth here, this b u here, is less than the carrier frequency.
In other words, when we translate it up in frequency, this and this do not intersect with each other.
If this and this intersected with each other, it would be something very much like aliasing.
You just couldn't to sort out from this, plus this, what this is.
Here, if I drew it on paper at least, if you tell me what this is, I can figure out what that is.
Namely, demodulating it independent of how we design the demodulator is, in some sense trivial, too.
You just take this and you bring it back down to passband again.
Well, anyway, since communication engineers define bandwidth in terms of positive frequencies, the bandwidth of this baseband waveform is b sub u.
The bandwidth of this waveform is 2 b sub u.
You can't get away from that.
You have doubled the bandwidth, and you wind up with this plus this.
And this looks kind of strange.
So let's try to sort it out.
The baseband waveform is limited to b.
If it's shifted up to passband, the passband waveform becomes limited to 2 b.
Might as well put these little u's in here.
Because putting in little u's here is a way of getting around the problem of talking about baseband waveforms for a while.
And then talking about passband waveforms.
And one of them is always twice the other one.
If you filter out this lower band here, now, what's the lower sideband here?
Who thinks the lower sideband is this?
Who thinks the lower sideband is this little thing here?
Well, you all should think that because that's what it is.
So when people talk about sidebands, what they're referring to, it's not, this is one sideband and this is another sideband.
What they're referring to is this is one sideband and this is one sideband.
This is stuff you all know, I'm sure.
But haven't thought about for a while.
If you filter out this lower sideband, then this resulting waveform, which now runs only in this upper sideband here, and since it has to be real it has this accompanying lower sideband down here going with it, but you then have the frequency band b sub u like you had before.
So, in principle, you can design a communication system by translating things up in frequency by the carrier, and then chopping off that lower sideband.
And then you haven't gained anything in frequency.
And everything is essentially the same as it was before.
Now, this used to be a very popular thing to do with analog communication.
Partly because communication engineers felt the only thing they had to study back then was, how do you change things in frequency and how do you build filters.
So they're very good at this.
They love to do this.
And this was their preferred way of dealing with the problem.
They just got rid of that sideband, sent this positive sideband, and then somehow they would get it back down to here.
Single sideband is hardly ever used for digital communication.
It's not the usual way of doing things.
Partly because these filters become very tricky when you're trying to send data at a high speed.
All sorts of noise in here when you try to do this and you don't do it quite right.
Affects you enormously, and people have just seen over the years that those systems don't work as well as the systems which do something else that we'll talk about later.
Namely, QAM, which is what we want to talk about.
If you don't do this filtering, you call this system a double sideband pulse amplitude modulation system.
Which is what happens when you use pulse amplitude modulation.
Namely, this baseband choosing a baseband pulse, which is the thing we're interested in.
Because that's where the Nyquist criterion and all this neat stuff comes in.
And then you translate it up in frequency.
And you waste half the available frequency.
If you don't care about frequency management, this is a fine thing to do.
Nothing wrong with it.
You just waste some frequency.
It's the cheapest way to do things.
And there are lots of cheap communication systems which do this.
But if you're trying to send data, if you're concerned about the frequency efficiency of the system, then you're not going to do this.
So what we're going to do is do something called quadrature amplitude modulation.
QAM, which is what quadrature amplitude modulation stands for, solves the frequency waste problem of double sideband amplitude modulation by using a complex baseband waveform u of t. Before, what we were talking about is these signals which were one-dimensional signals.
We would use these one-dimensional signals to modulate this waveform p of t. And we wound up with a real waveform.
Now what we're going to do is use complex signals, which then have two dimensions.
Use them to modulate the same sort of pulse, p of t, usually.
And wind up with a complex baseband waveform.
And then we're going to take that baseband waveform, translate it up in frequency.
So when we do this, what do we get?
We need a waveform to transmit which is real.
So we're going to take u of t, which is complex.
Translate, shift it up in frequency by the carrier frequency.
So we get u of t times e to the 2 pi i f c t. To make it real, we have to add all this junk down at negative frequencies, which we'd just as soon not think about if we didn't have to.
But they have to be there.
So our total waveform is x sub t equal this sum of things.
When you look at this and you take the real part of this, the real part of this, the imaginary part of this, and the imaginary part of this, as I'm sure most of you have seen before, x of t becomes 2 times the real part of u of t. Times this complex exponential.
Which is equal to 2 times the real part of u of t times this cosine wave minus 2 times the imaginary part of u of t times a sine wave.
Which says, you take this real part of this baseband waveform you've generated.
You multiply it by cosine wave.
You take the imaginary part, and you multiply it by a sine wave.
For implementation, is one thing going to be real and the other thing imaginary?
No.
You can't make things that are imaginary, so you just deal with two real waveforms.
And you call one of them the real part of u of t. You call the other one the imaginary part of u of t. And the imaginary part of u of t is in fact a real waveform again.
So all this imaginary stuff is just in our imagination.
And the actual waveforms look like this.
You take one waveform which is generated.
Multiply it by cosine.
Take another waveform.
Multiply it by sine.
What about these factors of two here?
The factors of two are things that drive everybody crazy.
Everyone I talk to, I ask them how they manage to keep all this straight.
And they all give me the same answer: they say they can't keep it straight.
It's just too hard to keep it straight, and after they're all done, they try to figure out what the answer should be by looking at energy or something else.
Or by just fudging things, which is what most people do.
And part of the trouble is, you can do this two ways.
You can do it three ways, in fact.
You can either I want to view x of t as being some real function times the cosine wave and leave out that 2.
And some other function, imaginary part of u t times the sine, and leave out the 2 there.
And many people do that.
And would that be better?
But when you put the 2 in with the cosines and the sines, you have to put a 1/2 in here and a 1/2 in here.
Most people, when they think about these things for a long time, they find it's far more convenient to be able to think of this positive frequency part of x of t as just u of t translated up in frequency.
In other words, they like this diagram here.
Which says you take this.
You translate it up.
And after you translate it up, you create something else down here, to make the whole thing real.
But what we think of is this going up to this all the time.
So that's one way of doing it.
The other way of doing it is thinking in terms of sines and cosines, removing that 2 here.
And who can imagine what the third way of doing it is?
Just split the difference.
Which means you put a square root of 2 in.
And, in fact, that makes a whole lot of sense.
Because then when you take the waveform u of t, translate it up in frequency.
Make it real, you have the same energy in the baseband waveform as you have in the passband waveform.
I'm not going to show that.
You can just figure it out relatively easily.
I mean, you know that the power in a cosine wave is 1/2, the power in a sine wave is 1/2.
So when you're multiplying things by 1/2 in here -- well, this has a power of 1/2.
This has a power of 1/2.
And when you start looking at power here, you find out that that has to be a square root of 2 rather than 2.
So there are three ways of doing it.
People do it any one of three different ways.
It doesn't make any difference, because any paper you read will start out doing it one way and then, as they go through various equations, they will start doing it a different way.
And these factors of 2 multiply and multiply and multiply.
And in big complicated papers, sometimes I've found that these add up to a factor of 8 or 16 or something else.
By the time people are all done.
And we will explain later why, in fact, you don't really care about that very much.
But you can't just totally ignore those factors, so anyway.
This is the way we will do it.
We will try to be consistent about this, and usually we will be.
The way we want to think about this conceptually is that quadrature amplitude modulation is going to take this complex waveform u of t. It's going to shift it up in frequency to f sub c, and then we're going to add the complex conjugate.
Add it to form the real x sub t. In other words, we're going to think of it as a two-stage operation.
First you take waveform, you translate it up.
Then you take the real part or something, or add the negative frequency part.
And we're going to think both of this double operation of 1 going from u of t to the positive frequency part of things.
And then, of looking at the real waveform that corresponds to that.
What we're going to be doing here in terms of all of this is, we're going to start out with binary data.
From the binary data, we're going to go to symbols.
And we're going to go to symbols by taking a number of binary data -- a number of binary digits.
Framing then into b tuples.
Each b tuple will correspond to a set of 2 to the b symbols.
We're going to map these symbols into complex signals.
We're going to map the complex signals into a baseband waveform u of t. We're going to map the baseband waveform u of t into this positive frequency waveform u of t times this complex sinusoid.
And finally we're going to add on the negative frequency part to wind up with x of t. What do you think we do at the receiver?
As always, we do just the opposite.
Namely, one of the reasons for wanting to think about this this way, is we want to use this layering idea.
And the layering idea says, you start out with the received waveform x of t. And later on we'll have to add the noise to it.
You go from there to the positive frequency part.
You go from the positive frequency part.
You shift it down to u of t. How we got from here to there, I'll explain in a minute.
You go from here down to baseband again.
You go from the baseband to the complex signals, which we're going to do simply by filtering and sampling.
We go from the complex signals to the symbols, which is in fact a trivial operation.
It's just a look-up operation.
And then from there we un-segment things into binary digits again.
So that's the whole system.
And it has all these different pieces to it.
I couldn't draw it as our favorite kind of diagram, because it has too many blocks in it.
So that has to do.
What we're going to do now is look at each of these pieces one at a time.
And the first part is the complex QAM signal set.
And, just for some notation here, so we'll be on the same page, but we use r to talk about the bits per second at which data is coming into this whole system.
That's the figure you're interested in, when everything is done.
How many bits per second can you transmit?
We're going to segment this into b bits at a time.
So we're going to have a symbol set with 2 to the b elements in it.
We're going to map these m symbols, which are binary b tuples, into elements from the signal set.
The signal rate, then, is r sub s, which is r over b.
This is the number of signals per second that we're sending.
In other words, t, this signal interval that we've always been using in everything we've been doing, is one over r sub s. So t is the signal interval.
Every t seconds, you've got to send something.
If you didn't send something every t seconds, the way this stuff coming in from the source would start piling up and your buffers would overflow and it wouldn't work.
The signals u sub k are complex numbers.
Or real 2-tuples.
So we can, when we're trying to decide what signal set we're using, we can just draw our signals on a plane.
The signal set is a constellation, then, of m complex numbers or real 2-tuples.
So the problem of choosing the signal set is, how do you choose m points on a complex plane.
What problem is that similar to?
It's similar to the quantization problem where we were trying to choose m representation points.
And it's very close to that problem.
It's a very similar problems.
Has a few small differences, but not many.
But, before getting into that, we want to talk about a standard QAM signal set.
In a minute I'll explain why people do that.
And, as you might imagine, a standard QAM is just a square array of points.
It's the simplest thing to do, and sometimes the simplest thing is the best.
So it's determined by some distance, d, that you want to have between neighboring points.
And given that distance, d, you just create a square array here.
The square array means that m has to have an integer square root.
This is drawn for m equals 16.
If you look at this, you see that the real part of this it's the standard PAM set.
The imaginary part is a standard PAM set, which says you can deal with the real part and the imaginary part separately.
You take half the bits coming in and you choose your real part signal.
Take the other half of the bits coming in.
You form your imaginary part signal, and bingo, you're all done.
The energy per 2D signal, we can find the energy for 2D signal by looking at it this way.
It's two PAM systems running in parallel to each other.
For the PAM system, the energy in one of these dimensions is then d squared times the square root of n squared, minus 1 divided by 12.
But now we want to look at the energy which we have in both the real part and the imaginary part.
So we need this extra factor.
Well, we need to add together two of these things.
So we wind up with d squared times m minus 1 divided by 6.
Big deal.
Choosing a good signal set is similar to choosing a 2D set of representation points in quantization.
If you like to optimize things, you see this problem and you say, gee, at least there's something I can put my teeth into here.
What's the best way to choose a signal set?
And we found that for quantization that wasn't a terribly nice problem, although at least we had things like algorithms to try to choose reasonable sets.
And we then looked at entropy quantization and things like this, and it was a certain amount of fun.
Here, this problem it's just ugly.
There's no other way to express it.
I had to be convinced of this.
I once spent an inordinate amount of time trying to find the best signal set with eight points in it, in two dimensions.
How do you put eight single points in two dimensions in such a way that every point is distance at least d from every other point, and you minimize the energy of the set of points?
The answer is just absolutely ugly.
It has no symmetry.
Nothing nice about it.
You do the same thing for 16 points, and it's just an ugly problem.
You do the same thing for any number of points.
Except for four points.
For four points, it's easy.
For four points, you use standard QAM and it's the best thing to do.
And that problem is easy.
But you know, that's not much fun.
Because you say, bleugh.
So, partly for that reason, people use standard signal sets.
Partly because you don't seem to be able to gain much by doing anything else.
So that's about all we can say about standard -- oh, with eight signals, you can't use a standard signal set.
That was one reason we had to worry about it.
Back a long time ago, we were trying to design a 7200 bit per second modem.
Back in the days when people did 2400 bits per second.
And we managed to do 4800 bits per second by using QAM, big deal.
And then we said, well, we can pile in an extra bit by using three bits per two dimensions instead of two bits.
And spent this enormous amount of time trying to find a sensible signal set.
I don't even want to tell you what it was, because it wasn't interesting at all.
So, enough for signal sets.
The next thing is, how do you turn the signals into complex waveforms?
Namely, how do you go from the signals in two dimensions, complex signals into a baseband waveform u of t?
Well, fortunately, Nyquist's theory is exactly the same here as it was when we were dealing with PAM.
Everything we said before works here.
The only difference is that you don't have to choose the pulse p of t to be real.
But if you look back at what we did, we didn't assume that p of t was real before, anyway.
We just said, you might as well choose it to be real.
But you don't have to choose it to be real.
Bandedge symmetry requires that g of t be real.
Anybody know why that is?
When you choose g of t to be real, the negative frequency part is the complex conjugate of the positive frequency part.
Which is why, when we took this out-of-band stuff at negative frequencies, piled it into the positive frequencies, we got the same thing as if we simply rotate it around on the positive frequency.
So that bandedge symmetry condition really requires that g of t be real.
The orthogonality of t a t minus k t, this set of waveforms, requires g of t to be real.
Neither of these things require p of t to be real.
You can choose p of t to have any old phase characteristic you want to, but if we're choosing p of t -- if we're choosing p hat of f magnitude to be the square root of a Nyquist waveform, then you can choose this phase to be anything you want to make it.
But you're just restricted in, aside from the phase, you're somewhat restricted in what p of t can be.
OK.
So we're going to make the nominal passband, Nyquist band, with 1 over t. Before we made the passband -- before we made the baseband bandwidth 1 over 2t.
When we go up the passband we double the bandwidth so the Nyquist bandwidth is now 1 over t. The passband bandwidth is 1 over t. That's the only thing that's changed.
Usually people design these filters, which they design at baseband, to go 5-10% over the Nyquist band.
In other words, these filters are very, very sharp, usually.
I mean, once you design a filter, it doesn't cost anything.
You put it on a chip and that's the end of it.
And the cost of it is zero.
So it's just the cost to design it, so you might as well make it small.
So finally, we want to go to base, from baseband to passband.
We talked about this a little bit.
In terms of these frequencies, the baseband frequency b sub u, we want to assume that that's less than the carrier frequency.
This is this condition that we needed to make sure that the positive frequency part -- ah, here it is.
That's the condition that makes sure that this is separated from that, and doesn't cause intersymbol interference between the two.
So, everything we do, we'll make this assumption.
I mean, a part of this is, if you're going to modulate with such a small carrier frequency, you might as well not do it at all.
You might as well just generate the waveform you want directly.
Because you don't gain that much by doing it at baseband.
Because you don't really have a baseband in that case.
So, u of t times e to the 2 pi i f c t is strictly in the positive frequency band, then.
And these two bands don't overlap.
As I said before, we're going to view this as two different steps.
The first step is, I'm going to take this complex waveform, u of t. Multiply it by a complex sinusoid, which shifts me up in frequency.
Just going to call that u passband of t. This is this passband signal that I want to think about.
The thing that's up in positive frequencies.
I'm going to ignore the thing at negative frequencies.
And then I'm going to form the actual waveform x sub t, as this plus its conjugate.
If you think a little bit now, you can see that since these two bands are separated, if you think in terms of complex waveforms, how do you retrieve this band up here by a waveform which has both bands in it?
Well, you filter out what's at negative frequencies, OK?
So we want to design a filter which filters out the negative frequencies and x of t, and only leaves the positive frequencies.
In other words, you want a filter whose frequency response is just 1 for all positive frequencies, 0 for all negative frequencies.
And that filter is called a Hilbert filter.
Have any of you ever heard of a Hilbert filter before?
I don't know of anybody that's ever built one.
And we'll see why they don't built them in a while.
But it's a nice idea.
I mean, if you try to build one you'll find that it's harder to build -- you'll find that you have to implement four real filters in order to implement this filter.
So we'll find out that's not the thing to do.
But it's nice conceptually because it lets us study things like energy, power, and linearity, and all of these things.
So the transmitter then becomes this thing you start out with a complex waveform of baseband.
You shift it up in frequency.
This gives you this high frequency waveform, u sub p of t. You then take 2 times the real part of that to find the real waveform.
We won't worry about how to implement this, you just do it.
This passband waveform, you then pass it through this Hilbert filter, which just chops off the negative frequency part of it.
Gives you a complex waveform again.
You multiply by e to the minus 2 pi i f c t, which takes this positive frequency waveform, shifts it back down to baseband.
So this is a nice convenient way of thinking about, how do you go from baseband up to passband and passband down to baseband again.
Now.
If you want to view these vectors here as vectors, want to view u of t as a vector in L2, there's an important thing here going on.
We'll have to talk about it a good deal later on.
This is a complex waveform.
You want to deal with it as a vector in complex L2.
In complex L2, when we're dealing with vectors, we have scalars, which are complex numbers.
When we start dealing with real parts of these things, we want to view the real parts as being elements of real L2.
Where the scalars are real numbers.
And what this says is that real L2 is not a subspace of complex L2.
It's not a subspace because the scalars are different.
This might sound like mathematical nitpicking.
But, put it in the back of your mind.
Because at some point it's going to come up and clobber you.
And at that point, you will want to think that in fact real L2 is not a subspace of complex L2.
When we start thinking about orthonormal expansions for u of t and orthonormal expansions for x of t, in fact you have to be quite careful about this.
Because you take an orthonormal expansion here, translate it up into frequency.
And you wind up with a bunch of complex waveforms.
And they aren't real waveforms.
And funny things start happening.
So we'll deal with all of that later.
This is just to warn you that we have to be careful about that.
This is not the way people implement these things.
Because these Hilbert filters are in fact for real filters.
So the implementation is what you've seen.
I mean, the implementation is old.
And it's the way you want to build these things.
You start out with two real baseband waveforms.
One which we call the real part of u of t. One which we call the imaginary part of u of t. In this single diagram, one of them is the stuff that goes this way.
And the other one is the stuff that goes this way.
And if, in fact, you're using a standard QAM signal set, the two are completely independent of each other.
So the real part of u of t is just the sum of these shifted pulses times the real parts.
This is the sum of the shifted pulses times the complex parts.
The defined u sub k prime is a real part, and u sub k double prime is the imaginary part.
In the notes, this and this are called a sub k and a sub k double prime.
Which doesn't correspond to anything else.
So, this is the correct way of doing it.
But, anyway, when you get all done, x sub t is 2 times the cosine of this low pass modulated PAM waveform.
Minus 2 times the sine of this low pass PAM modulated waveform.
So, QAM, when you look at it this way, is simply two different PAM systems.
One of them modulated on a cosine carrier, one of them modulated on a sine carrier.
And the picture of that is this.
Aargh.
Can't keep my notation straight.
I'm sure it doesn't bother most of you that much, but it bothers me.
All of those a's should be u's.
They were a's last year, but they don't make any sense as a's.
So the thing we're going to do now is we start out with the sequence of signals.
The real part of the signals and the imaginary part of the signals.
This is why it's called double side band quadrature carrier, because in fact we're doing two different things in parallel.
We generate this as a pulse waveform.
We filter it by p of t. We're thinking of p of t as a real waveform now.
If you want p of t to be complex you have to modify this all a little bit.
But there's no real reason to make p of t complex anyway.
So when you get out of here, what you have is just this low pass real waveform, real PAM waveform.
Here's another low pass real PAM waveform.
You module this up by multiplying by cosine of 2 pi f c t, in fact, by 2 cosine of 2 pi f ct .
You modulate this up by multiplying by minus sine.
And you get the actual waveform that you're going to transmit.
How do you demodulate this?
Well, again, I'm sure you've seen it in one of the undergraduate courses you've taken.
Because if you take this waveform, which is the sum of this and this, and you multiply this by cosine of 2 pi f c t, what's going to happen?
Taking this waveform and multiplying it by cosine is going to take this cosine waveform.
Half of it goes up in frequency by f sub c. The other half goes down in frequency by f sub c of t. When you multiply by sine, the same thing happens.
And all of the stuff at this double frequency term all gets filtered out.
I mean, you have enough filtering to just wash that away.
And you wind up, just with this one waveform which is the result of this.
Another waveform which is the result of this.
You can show that the two don't interfere at all, and you just have to do the multiplication to find this out.
It looks a little bit like black magic when you look at it like this.
Because when you're multiplying by a cosine wave, I mean it's easy to see what cosine squared does here.
But it's a little harder to see what happens to all of this.
And when we look at it the other way, which was this Hilbert filter kind of thing, when you look at in terms of the Hilbert filter it's quite clear that you can filter out the lower sideband and then you can just go back down to baseband again.
So it's very clear that the whole thing works.
Except you wouldn't implement it this way.
Here you have to be more careful to see that works.
But in fact you would.
Well, after you get all done, then you get these baseband PAM waveforms back again.
You sample them after filtering.
And you're all done.
With that, we are almost done with what we want to do with modulating up to passband and down to baseband.
We'll spend a little bit of time reviewing a couple of minor points on this next time.
Like, I guess, the main thing we have to talk about is how do you do frequency recovery, which is kind of a neat thing.
And then we'll go on to talking about random processes and how you deal with noise.
So.
If you want to read ahead, we will probably have the notes on random processes on the web sometime tomorrow afternoon or Sunday.
Thanks.
The following content is provided by a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I just want to be explicit about it because with all of the abstraction of QAM looking at it in terms of Hilbert filters and all of that, you tend to forget that what's really going on in terms of implementation is something very, very simple.
We said that what we were doing with QAM is, we're trying to essentially build two PAM systems in parallel.
So what we do is, we have two different data streams coming in.
Each one there's a new signal, each t seconds.
t is the time separation between times when we transmit something.
So the sequence of symbols come in.
What we form is then -- well, actually this isn't a real operation.
Because everybody does this in whatever way suits them.
And the only thing that's important is getting from here over to there.
And what we're doing in doing that is taking each of these signals, multiplying them by a pulse, to give this waveform the pulses delayed according to what signal it is.
And if you like to think of that in terms of taking the signal and viewing it as an impulse, so you have a train of impulses weighted by these values here.
And then filtering that, fine.
That's a good way to think about it.
But, at any rate you wind up with this.
With the other data stream which is going through here -- yes?
[INAUDIBLE]
You couldn't argue that was band limited also
[INAUDIBLE]
But it's band limited in the same way.
I mean, visualize -- I mean, the Fourier transform of p of t minus k t is simply the Fourier transform of p of t, which is multiplied by a sinusoid in f at this point.
So it doesn't change the bandwidth of that.
In other words, when you take a function and you move it in time, the Fourier transform is going to be rotating in a different way.
But its frequency components will not change at all.
All of them just change in phase because of that time change
[INAUDIBLE]
When you look at it in frequency, well no, because we're adding up all these things in frequency at that point.
But each one of these pulses is still constrained within the same bandwidth
[INAUDIBLE]
The transform of a pulse train does not look like a pulse train in frequency, no.
When you take a pulse train in time, and you take the transform of each one of these pulses, what you get is a pulse which, where in frequency there is a rotation.
In other words, the phase is changing.
But in fact, what you wind up with is a pulse within the same frequency band.
I mean, simply take the Fourier transform and look at it, and see which frequencies it has in it.
In they don't change.
And when you weight it with coefficients, the Fourier transform will be scaled, will be multiplied by a scalar.
But that won't change which frequencies are there, either.
So in fact, when you do this -- I should make that clearer in the notes.
It's a little confusing
[INAUDIBLE]
It'll be band limited in the same way, though, yes
[INAUDIBLE]
It won't look the same.
But the Nyquist criterion theorem remains the same.
I mean, if you look at the proof of it, it's -- well I'll go back and look at it.
I think if you look at it, you see that it actually -- I think you see that this time variation here in fact does not make any difference.
I mean, I know it doesn't make any difference.
But the question is, how to explain that it doesn't make a difference.
And I'll go through this.
And I will think about that.
And I'll change the notes if need be.
But anyway, it's not something you should worry about.
Anyway.
What happens here is, you get this pulse train, which is just a sum of these pulses.
And they all interact with each other.
So at this point we have intersymbol interference.
We multiply this by a cosine wave.
We multiply this by a sine wave.
And what we get then is some output, x of t. Which is in the frequency band.
But if you take the low pass frequency here, which goes from 0 up to sum w, up at passband, we go from f c minus w to f c plus w. Same thing here.
So we have this function, which is then constrained.
But it's a passband constraint to twice the frequency which we have here.
When we take x sub t, and we try to go through a demodulator.
The thing we're going to do is to multiply, again, by the cosine.
Multiply again by the sine.
Incidentally, this is minus sine and minus sine here.
When you implement it, you can use plus sine or minus sine, whichever you want to.
So long as you use the same thing here and there.
It doesn't make any difference.
It's just that when you look at things in terms of complex exponentials, this turns into a minus sine.
Oh, and this turns into a minus sine.
We then filter by q of t. We filter this by q of t. The same filter.
And we wind up then going through the T-spaced sampler.
The T-spaced sampler and the output use of k prime.
And the reason for this is that if you look at the system broken right here, this is simply a PAM waveform.
And when we take this PAM waveform, the only thing we're doing is we're multiplying it by a cosine.
Then we're multiplying it by a cosine again.
When we get a cosine squared term, half of it comes back down to baseband.
Half of it goes up to twice the carrier frequency.
And this filter, in principle, is going to filter out that part at 2 times the carrier frequency.
And it's also going to do the right thing to this filter.
To turn it into something where, when we sample, we actually get these components back again.
So the only thing that's going on here is simply, this is a PAM system where, when we multiply by cosine twice, the thing we're getting is a baseband component again, which is exactly the same as this quantity here.
Except it's -- why, I when I multiply this by this, don't I have to divide by -- don't I have to multiply by 2?
I don't know.
But we don't.
And when we get all through the thing we come out with u sub k prime, and we come out with u sub k double prime.
It is easier to think about this in terms of complex frequencies.
When we think about it in terms of cosines and sines, we're doing it that way because we have to implement it this way.
Because if you implement a Hilbert filter, you're taking a complex function, filtering it by complex waveform, which means you've got to build four separate filters.
One to take the real part with the real part of the filter.
The real part with the complex part of the filter.
Also the imaginary part of the waveform with the real part of the filter, and so forth.
So you don't implement it that way, but that's the easy way to look at what's happening.
It's the easy way to analyze it.
This just does the same thing.
When we use this double sideband quadrature carrier implementation of QAM, the filters p and q really have to be real.
Why do they have to be real?
Well, because otherwise, we're mixing what's going on here with what's going on here.
In other words, you can't take -- this will not be a real waveform, so you can't just multiply it by a cosine wave.
In the other implementation we had, you have this, which is a complex waveform, plus this, which is another complex waveform.
You multiply it by e to the 2 pi i f c t. And then you take the real part of it.
And everything comes out in the wash.
Here we need this to be real and this to be real in order to be able to implement this just by multiplying by cosine and multiplying by sine.
So the standard QAM signal set makes it just parallel PAM systems.
If, in fact, you don't use a standard QAM set; namely, if these two things are mixed up in some kind of circular signal set or something, then what comes out here has to be demodulated in the same way.
So you don't have the total separation.
But the real part of the system is the same in both cases.
I think the notes for a little confusing about that.
Finally, as a practical detail here, we have talked several times about the fact that every time people do complicated filtering, and complicated filtering involves very sharp filters.
Every time you make a very sharp filter at baseband, the way you do it is, you do it digitally.
Namely, you first take this waveform.
You sample it very, very rapidly at a rapid pace.
And then you take those digital signals and you pass those through a digital filter.
And that gives you the waveform that you're interested in.
When you do that, you're faced with having a waveform coming into the receiver which has a band down at baseband.
Which it has width w. You also have a band up at twice the carrier frequency.
And if you sample that whole thing, according to all of the things we've done with aliasing and everything, those samples mix what's up at twice the carrier frequency with what's down here at baseband.
And you really have to do some filtering to get rid of once or twice the carrier frequency.
This really isn't much of a practical issue, because you can hardly avoid doing that much filtering just in doing the sampling.
I mean, you can't build the sampler that doesn't filter the waveform a little bit too.
But you have to be conscious of the fact that you have to get rid of those double frequency terms before you go into the digital domain.
So, you have to think of it as a little bit of filtering followed by a rapid sampler, followed by a careful filtering.
Which does this function q of t here.
OK. Want to talk a little bit about the number of degrees of freedom we have when we do this.
If we look at quadrature amplitude modulation, anytime you get confused about bandwidths, it's probably better to look at the sample time, t, which is something that never changes.
And that always remains the same.
When you look at the sample spacing, the baseband bandwidth, as we've seen many times, the nominal baseband bandwidth is 1 over 2t.
Namely, that's what's called the Nyquist bandwidth.
It's what you get if you use sinc functions, which strictly puts everything in this baseband and nothing outside of it.
And we've seen that by using the Nyquist criterion and building sharp filters, we can make the actual bandwidth as close to this as we want to make it.
When we modulate this up to passband, we're going to have a passband bandwidth which is 1 over t. Because, in fact, this baseband function is going from minus 1 over 2t to plus 1 over 2t.
When we modulate that up, we get something that goes from f c minus 1 over 2t to f c plus 1 over 2t.
Which is really an overall bandwidth of 1 over t. If we look at this over a large time interval, t0, we have two degrees of freedom.
Namely, the real part and the imaginary part.
Namely, we have two different real number signals coming into this transmitter.
Each t seconds.
So that over a period of t0, some much larger period, we have t0 over capital T different signals that we're transmitting.
Each of those signals has two degrees of freedom in it.
So we wind up with 2 t0 times w real degrees of freedom.
Namely, 2 t0 over T. There's 2 t0 times w, where w is the passband bandwidth.
What I want to do now, to show you that actually nothing is being lost and nothing is being gained, is to look at a baseband system where you use pulse amplitude modulation.
But, in fact, you use a humongously wide bandwidth to do this.
So it becomes something like these ultra-wideband systems that people are talking about now.
You use this very, very wide bandwidth, and you see how many signals you can pack into this very wide bandwidth using PAM.
And what's the answer?
There are 2 t0 w 0 real degrees of freedom.
And a baseband bandwidth w 0.
This is what we determine when we were dealing with PAM.
It simply corresponded to the fact that the baseband bandwidth was 1 over 2 t 0.
Therefore we got two signals coming in, each 1 over w 0.
So this was the number of real degrees of freedom we have.
Now, the thing we want to do is to take this humongous bandwidth.
And to say, what happens if instead of using this wide bandwidth system, we use a lot of little narrow bands.
So we make narrow bands send QAM in the narrow bands, where in fact we're packing signals into the real part and the imaginary part, which is what we decided we ought to do.
And how do these two systems compare.
Well, if we take a very wide bandwidth w0, with lots of little bandwidths, w, up at various passbands, all stacked on top of each other.
Like here's zero.
Here's w0 which is up at five gigahertz or whatever you want to make it.
We put lots of little passbands in here, each of width w. And the question is how many different frequency intervals do we get?
How many different signals can we multiplex together here by using different frequency bands?
Which is exactly the way people who use QAM normally transmit data.
Well, the answer is, you get w0 over w different bands here.
So how many degrees of freedom do you have?
Well, we said we had 2 t 0 w real degrees of freedom with each QAM system.
We have w0 over w different QAM systems all stacked on top of each other.
So, when you take w equals w0 over m you wind up with 2 t0 w, 2 t0 w 0 real degrees of freedom, overall using QAM.
Exactly the same thing if you use PAM.
This corresponds to these orthonormal series we were thinking about, where we were thinking in terms of taking segmenting time, and then using a Fourier series within each segment of time.
And looking at the number of different coefficients that we got.
And we then looked at these sinc weighted expansions of the same type.
We got the same number of degrees of freedom.
This is simply saying that PAM and QAM use every one of those degrees of freedom.
And there aren't any more degrees of freedom available.
If you're stuck with using some very large time interval t0 and some very large frequency interval, w0, you're stuck with that many real degrees of freedom, and real waveform.
There's nothing else available, and PAM and QAM are perfectly efficient as far as their user spectrum is concerned.
They both do the same thing.
How about single sideband?
That does the same thing too.
I mean, with single sideband, you're sending real signals but you cut off half the bandwidth at passband.
So, again, you get the same number of signals for unit time and per unit bandwidth.
And most systems, except double sideband quadrature carrier, which is just obviously wasting bandwidth, all do the same thing.
So, it's not hard to use all of these available degrees of freedom.
And the only thing that makes these degrees of freedom arguments complicated is, there's no way you can find the pulse, which is perfectly limited to 1 over w in time and 1 over w in bandwidth.
If you make it limited in time, it spills out in bandwidth.
If you make it limited in bandwidth, it spills out in time.
So we always have to fudge a little bit as far as seeing how these things go together.
Let's talk about one sort of added practical detail that you always need in any QAM system.
In fact, you always need it in a PAM system, also.
And you need to do it somehow or other.
There's this question of, how do you make sure that the receiver is working on the same clock as the transmitter is working on?
How do you make sure that when we talk about a cosine wave at the receiver, the phase of that cosine wave is, quote, the same as the phase at the transmit?
So, if the transmitted waveform at the upper passband -- at positive frequencies is u of t times e to the 2 pi i f c t. This is this is one of the reasons why you want to think in terms of complex notation.
If you try to analyze QAM phase lock using cosines and sines -- I mean, you're welcome to do it, but it's just a terrible mess.
If you look at it in terms of complex signals, it's a very easy problem.
But it's most easy when you look at it in terms of just what's going on at positive frequencies.
So the thing we're doing here is, when we modulate, we're taking u of t. We multiply by e to the 2 pi i f c t. What happens to any arbitrary phase that we have in the transmitter?
Well, nothing happens to it.
We just define time in such a way that it's not there.
At the receiver, we're going to be multiplying again by e to the minus 2 pi i f c t. There's going to be some phase in there because the receiver's clock and the transmitter's clock are not the same.
Now, the thing which is complicated about this is that at the receiver there's some propagation delay from what there is at the transmitter.
So what we're really saying here is that when the waveform u of t gets to the receiver -- in other words, after we demodulate, what do we get?
That's the practical question to ask.
I mean you have an arbitrary sinusoid, complex sinusoid at the transmitter.
You have an arbitrary complex sinusoid at the receiver.
When you multiply by e to the 2 pi i f c t, and then at the receiver you demodulate by multiplying e to the minus 2 pi i f c t, there's some added phase in there just because you're multiplying by some sinusoid in an arbitrary phase.
What do you wind up with when you get done with that?
You get the received waveform, u of t, times e to the i phi.
Now, suppose you're sending analog data.
What happens here?
Well, with analog data, you're absolutely up a creek without an oar.
There's nothing you can do.
But with digital data there is something you can do.
And I think this diagram makes it clear, if you can see the little arrows on it.
What you're transmitting is QAM signals out of a QAM constellation.
So the signals -- so you're either transmitting this, or this, or this, or this, and so forth.
Now, at the receiver we haven't analyzed noise yet.
We're going to start doing that later today.
But if you visualize noise being added up at passband and then coming down to baseband again, we can visualize what's happening as the noise just adds something to each one of these points.
I mean, assuming that we know exactly what the timing is for the time being.
So that we sample at exactly the right time at the receiver.
And that's a separate issue.
But let's assume that we do know what the timing is.
We do sample at the correct time.
And the only question we're trying to find out now is, how do we choose this phase right on this sinusoid that we're using at the demodulator.
Because at the demodulator, there's no way to know whether phi is zero or whether phi is something else.
But after we got all done doing this, if we sample at the right time, what we're going to receive, if there's some small error in phase, is that this diagram is all rotated around a little bit.
Any time you send this point, what you receive is going to be a point up here.
If you send this point it's going to be a point here.
If phi is negative, everything will go the other wa.y How do I know whether it's positive phi or negative phi?
I don't.
I just do the same thing either place.
Everybody I know who worries about phase locked loops ignores the sign and simply makes sure that they're using the same sign at the transmitter and the receiver.
If they don't, when they build it, the thing goes out of lock immediately.
And they try switching the sign, and if it works then, they say uh-huh, I now have the right sign.
I mean, it's like these factors of two.
You always put them in after you're done.
Anyway, I think I've done it right here.
So the received waveform is now going to be u of t times e to the i phi.
It will have a little bit of noise added to it.
So what we're getting, then, if you look at a scope, or whatever people use as scope these days.
And people have done this for so many years that they call this an eye pattern.
And what they see, after looking at many, many received signals, is little blobs around each of these points due to noise.
And they also see the set of points rotated around a little bit.
And if the noise a small, and if the phase error is small, you can actually see this diagram rotated around sitting at some orientation.
And the question is, if you know that's what's happening, what do you do about it?
Well, if you have this diagram rotated around a little bit, you want to change the phase of your demodulating carrier.
So the thing you do is, you measure the errors that you're making when you go from these received signals.
And you make decisions on them.
You then look at the phase of these errors.
If all the phases are positive, then you know that you have to change the phase of this carrier one way.
And if all the phases are negative, you change it the other way.
Now, why does this work?
It works because these clocks are very stable clocks.
They wander in phase slowly.
I mean, you can't lock them perfectly.
And since they wander in phase slowly.
And since you're receiving data quickly, you can average this carrier recovery over many, many data symbols.
Which is what people do.
So, you average it over many symbol.
And you gradually change the phase as the phase is getting off.
How much trouble does that cause in trying to actually receive the data?
Hardly any.
Because if the phase changes are slow enough, then you average over a long enough interval, the actual phase errors are going to be extraordinarily small.
So it's the noise errors that look big and the phase errors that look very small.
When you average over a long period of time, the noise is sort of independent from one period to another.
So the noise averages out.
And the thing you see, then, is the phase errors.
So, so long as the phase is slow and the noise is quick, you first decode the data.
After you decode the data, you average over what the phase errors are.
And then you go and you change the clock.
And people spend their lives designing how to make these phase lock loops.
And there's a lot of interesting technology there.
But the point is, the idea is almost trivially simple.
And this is what it is.
Since the phase is changing slowly, you can change the phase slowly.
You can keep the phase error very, very small.
And therefore, the data recovery due to the noise works almost as well as if you had no phase error at all.
How do you recover from frequency errors?
Well, if I can recover from phase errors, I'm recovering from frequency errors also.
Because I had this clock which is running around.
And I'm slaving the clock at the receiver to the clock at the transmitter.
So, so long as I have very little frequency error to start with, simply by keeping this phase right, I can also keep the carrier frequency right.
Because the phase is just going around at the speed it's supposed to be going around.
So this will also serve as a frequency lock as well as a phase lock.
When you're trying to do both together, you build the phase lock loop a little different, obviously.
What's the problem with this?
You look this diagram, and you say, OK, suppose there's some huge error where for a long period of time, things get very noisy.
What's going to happen?
Well, if you can't decode the data for a long period of time, suddenly these phase errors build up.
The channel stops being noisy.
And what could have happened is that your phase could be off by pi over 2.
Which means what you see, what you should be seeing is this.
What you're actually seeing is this.
You can't tell the difference.
In other words, you decode every symbol wrong.
Because you're making an error of pi over 2 in every one of them.
You decode this as this, you decode this as this, this as this, and so forth.
So every point is wrong.
What will you do to recover from this?
I mean, suppose you were designing a system where in fact you had no feedback to the transmitter at all.
You have feedback from the transmitter, you just, obviously every once in a while you stop and you resynchronize.
You don't have any feedback, what do you do?
Yeah?
[INAUDIBLE]
Differential phase shift key, yes.
I mean, this isn't quite phase shift key, but it's the same idea.
In other words, the thing that you do is, you don't -- usually when the data is coming in, you don't map it directly into a signal point.
The thing that you do is map the data into a phase change from one period of time to the next.
I mean, you map it into an amplitude.
And then you map it into a change of phase in this set of points.
Or change of phase within this set of points.
Or a change of phase within the outer set of points.
And then if, due to some disaster, you get off by pi over 2, it won't hurt you.
Because the changes are still the same, except for the errors that you make when this process starts.
So in fact, a lot of these things that people have been doing for years, you would think of almost immediately.
The only thing you have to be careful about, when you invent new things to build new systems, is you have to have somebody to check whether somebody else has a patent on it.
So, you invent something and then you check whether somebody else patented it.
So that's what this slide says.
Bingo.
It's time to go on to talk about additive noise and random processes.
How many of you have seen random processes before?
How many of you have seen it in a graduate context as opposed to an undergraduate context?
A smaller number.
Many of you, if you got your degrees outside of the US, if you've studied this as an undergraduate, you probably know what students here have learned in their various graduate courses.
So you might know a lot more about it.
What we're going to do here is I'll try to teach you just enough about random processes that you can figure out what's going on with the kinds of noise that we're looking at this term.
Which involves two kinds of noise.
One, something called additive noise, which is what we're going to be dealing with now.
And the other is much later, when we start studying wireless systems, we have to deal with a different kind of noise.
People sometimes call it multiplicative noise, but really what it is, is these channels fade in various ways.
And the fading is a random phenomena.
And therefore we have to learn how to deal with that in some kind of probabilistic way also.
So the thing we're going to do is we're going to visualize what gets received as what gets transmitted plus noise.
Now, at this point, I've said OK, we know how to deal with compensating for phase errors.
So we'll assume that away, it's our usual layered approach.
What kind of other assumptions are we making here?
When I say the received signal is the transmitted signal plus the noise.
Well, I mean, I could look at this and say, this doesn't tell me anything.
This is simply defining the noise.
There's the difference between what I receive and what I transmit.
So when I say this is additive noise, I'm not saying anything other than the definition of the noise, is this difference.
But when we look at this, we're really going to be interested in trying to analyze what this process is.
And we're going to be interested in analyzing what this is as a process also.
Here, I'm just looking at sample functions.
And what I'd like to be able to do is say this sample function, namely, what's going into the channel, is one of a large set of possibilities.
If it weren't one of a large set of possibilities, there would be no sense at all to transmitting it, because the receiver would know what it was going to be.
And you wouldn't have to transmit it, it would just print it out at the output.
So, this is some kind of random process which is determined by the binary data which is coming into the encoder and the signals it gets mapped into, and so forth.
This reviewing of some kind of noise waveform.
What we're going to assume is that in fact these two random phenomena are independent of each other.
And when I write this, you would believe that if nobody told you to think about it.
Because it just looks like that's what it's saying.
Here's some noise phenomena that's added to the signal phenomenon.
If in fact I told you, well, this noise here happens to be twice the input then you'd say, foul.
What I'm really doing is calling something noise which is really just a scaling of the input.
Which is really a more trivial problem.
So we don't want to do that.
Another thing I could try to do, and people tried to do for a long time in the communication field, was to say, well, studying random processes is too complicated.
Let me just say there's a range of possibilities, possible things for what I transmit.
There's a range of possible waveforms for the noise.
And I want to make a system that works for both these ranges of different things.
And it just turns out that that doesn't work.
It works pretty well for the transmitter.
It doesn't work at all for the noise.
Because sometimes the noise is what it should be.
And sometimes the noise is abnormally large.
And when you're trying to decide what kind of coding devices you want to use, what kind of modulation you want to use, and all these other questions, you really need to know something about the probabilistic structure here.
So we're going to view this as a sample function of a random process.
And that means we're going to have to say what a random process is.
And so far, the only thing we're going to say is, when we talk about it as a random process, we'll call it capital N of t instead of little n of t. And the idea there is that for every time, for every instant of time, when you talking about random processes, you usually call instants of time epochs.
And you call them epochs because you're using time for too many different things.
And calling it an epoch helps you keep straight what you're talking about.
So for each epoch t, n of t is a random variable.
And this little n of t is just some sample value of that random variable.
So we're going to assume that we have some probabilistic description of this very large collection of random variables.
And the thing which makes this a little bit tricky, mathematically is that we have an uncountably infinite number of random of variables to worry about here.
x sub t is known at the transmitter but unknown at the receiver.
And what we're eventually concerned about is, how does the receiver figure out what was transmitted.
So we'd better view x of t as a sample function of a random process.
x of t, from the receiver's viewpoint.
Namely, the transmitter knows what it is.
This is this continual problem we run into with probability.
When you define a probability model for something, it always depends on whose viewpoint you're taking.
I mean you could view my voice waveform as a random process.
And in a sense it is.
As far as I'm concerned it's almost a random process, because I don't always know what I'm saying.
But as far as you're concerned, it's much more of a random process than it is from my viewpoint.
If you understand the English language, which all of you do, fortunately, it's one kind of random process.
If you don't understand that, it's just some noise waveform where you need a much more complicated model to deal with it.
So everything we have to do here is based on these models of these random processes, which are based on reasonable assumptions of what we're trying to do with the model.
When we do this in terms of random processes, we have the input random process, and we have the noise random process.
And if we add up two random processes, it doesn't take much imagination to figure out that what we get is another random process.
It's random both because of this and because of this.
Now, we implicitly are assuming a whole bunch of things when we write things out this way.
We're not explicitly assuming them, but what we want to do is explain the fact that we're implicitly assuming them and then make it explicit.
One of those things that we're doing here is, we're avoiding all attenuation in what's been transmitted.
We've been doing that all along.
We've been talking about received waveforms as being equal to the transmitted waveform.
Which means that we're assuming that the receiver knows what the attenuation is.
And if it knows what the attenuation is, there's no point in putting in this scale factor everywhere.
So we might as well look at what's being transmitted as being the same as what's being received, subject to this attenuation, which is just some known factor which we're not going to consider.
There's also this question of delay.
And we're assuming that the delay is known perfectly.
So that assumption is built into here.
y of t is x of t without delay, plus z of t. And now, after studying these modulation things, we can add a few more things which are implicitly put into here.
One of them is that we know what the received phase is.
So that in fact we don't have any phase error on all of this.
So we're saying, let's get rid of that also.
So we have no timing error, no phase error.
All of these errors are disappearing because what we want to focus on is just the errors that come from this additive noise.
Whatever the additive noise is.
But we want to assume that this additive noise is independent of this process.
So we're saying, when we know all of these things about -- or when the receiver knows all of these things about what's being transmitted, except for the actual noise waveform, how do we deal with the problem?
We implicitly mean that z of t is independent of x of t, because otherwise it would be just very misleading to define n z of t as the difference between what you receive and what you transmit.
So we're going to make that assumption explicit here.
These are standing assumptions until we start to study wireless systems.
And another standing assumption which we'll make is the kind of assumption we've been making in this course all along.
Which some people like, usually theoreticians.
And some people often don't like, namely practitioners.
But which if both the theoreticians and practitioners understood better what the game was, I think they would agree to doing things this way.
And what we're going to do here is, we're not going to think of what happens when people make measurements of this noise and then try to create a stochastic model of it from all of these measurements.
What we're going to do is more in the light of what Shannon did then when he invented information theory.
Which is to say, let's try to understand the very simplest situations first.
If we can understand those very simplest situations namely, we will invent whatever noise we want to invent.
To start to understand what noise does to communication.
And we will look at the simplest processes we can.
Which seem to make any sense at all of us.
And then after we understand a few of those, then we will say, OK, what happens if there's some particular phenomenon going on in this channel.
And when you look at what practitioners do, what in fact they do is, everything they do is based on various rules of thumb.
Where do these rules of thumb come from?
They come from the theoretical papers that were written 50 years before, if they're very out of date practitioners.
25 years before if they're relatively up to date practitioners.
Or one year before if they're really reading all the literature as it comes out.
But rules of thumb usually aren't based on what's happening as far as papers are concerned now.
It's usually based on some composite of what people have understood over a large number of years.
So in fact, the way you understand the subject is to create these toy models.
And the toy models are what we're going to be creating now.
Understand the toy models, and then move from the toy models to trying to understand other things.
This is the process we'll be going through when we look at this additive noise channel, which doesn't make much sense for wireless channels, as it turns out.
But it's part of what you deal with on a wireless channel.
And then, on a wireless channel, we'll say, well, are these other effects concerned also?
And we'll try to deal with these other effects as an addition to the effects that we know how to deal width.
We've already done that as far as phase noise is concerned.
Namely, we've already said that since phase effects occur slowly, we can in fact get rid of them almost entirely.
And therefore we don't have to worry about them here.
So, in fact we have done what we've said that we believe ought to be done.
Namely, since we now know how to get rid of the phase noise, we're now looking at additive noise and saying, how can we get rid of that.
Actually, if you look at the argument we went through with phase noise, we were really assuming that we knew how to deal with channel noise as part of that.
And in fact when you get done knowing how to deal with channel noise, you can go back to that phase locked loop argument.
And you'll see a little more clearly, or a little more precisely, what's going on there.
When we started talking about waveforms, if you remember back to when we were talking about compression, we really faked this issue of random processes.
If you remember the thing we did, everything was random while we were talking about taking discrete sources and coding for them.
When we then looked at how do you do quantization on sequences of real numbers or sequences of complex numbers, we again took a probabilistic model.
And we determined how to do quantization in terms of that probabilistic model.
And in terms of that probabilistic model, we figured out what ought to be done.
And then, if you remember, when we went to the problem of going from waveforms to sequences, and our pattern was, take a waveform.
Turn it into a sequence.
Quantize the sequence, then encode the digital sequence.
We cheated.
Because we didn't talk at all about random processes there.
And the thing we did instead is, we simply converted source waveforms to sequences and said that only the probabilistic description of the sequence is relevant.
And, you know, that's actually true.
Because so long as you decide that what you're going to do is go from waveforms to sequences, and in fact you have decided on which particular orthonormal expansion you're going to use, then everything from there on is determined just by what you know about the probabilistic description of that sequence.
And everything else is irrelevant.
I mean, each waveform maps into a sequence.
When you're going back, sequences map into waveforms, at least in this L2 sense, which is what determines energy difference.
And energy differences usually are criterion for whether we've done a good job or not.
So, in fact, we could avoid the problem that way simply by looking at the sequences.
And it makes some sense to do that because we're looking at mean square error.
And that was why we looked at mean square error.
But now we can't do that any more.
Because now, in fact, the noise is occurring on the waveform itself.
So we can't just say, let's turn this into a sequence and see what the noise on the waveform does to the sequence.
Because, in fact, that won't work at this point.
So random process z of t is a collection of random variables, one for each t and r. So, in other words, this is really a collection of an uncountably infinite number of random variables.
And for each epoch t, the random variable z of t is a function.
Capital Z of t and omega.
Omega is the sample point we have in this sample space.
Where does the sample space come from?
We're imagining it.
We don't know any way to construct a sample space very well.
But we know if we study probability theory that we need a sample space to talk about probabilities.
So we're going to imagine different sample spaces and see what the consequences of those are.
I mean, that's the point of this general point of view that we have.
That we start out looking at simple models and go from there to complex models.
Part of the model is the sample space that we have.
So for each sample point that we're dealing with, the collection -- I should have written this differently.
Let me change this a little bit to make it a little clearer.
z of t omega for t and r is a sample function z of t; t and r. So if I look at this, as a function of two variables.
One is time and one is this random collection of things we can't see.
If I hold omega fixed, this thing becomes a function of time.
And in fact it's a sample function, it's the sample function corresponding to that particular element in the sample space.
And don't try to ask what the sample space is.
The sample space -- I mean, the sample point here -- really specifies what this sample function is.
It specifies what the input sample function is.
It specifies what everything else you're interested in is.
So it's just an abstract entity to say, if we're constructing probabilities we need something to construct them on.
So that's the sample space that we can construct these probabilities on.
The random processes -- Oh.
OK. We want to look at two different things here.
Here we're looking at a fixed sample point, which means that this process for a fixed sample point corresponds to a sample function.
If instead I look a fixed time, a fixed epoch, t and r, and I say what does this correspond to as I look at z of t and omega over all omega in this big messy thing, fix t. What do I have there?
I have a random variable.
If I look at a fixed t, and I'm looking over the whole sample space, that's what you mean by a random variable.
If I look at -- if I hold omega fixed and look over the time axis, what I have is a sample function.
I hope that makes it a little clearer what the game we're playing is.
Because this random thing, the random process that we're dealing with, we can best visualize it as a function of two things, both time and sample point.
The sample point in this big sample space is what controls everything.
It tells you what every random variable in the world is.
And what the relationship of every random variable in the world is.
So, a random process is defined, then, by some rule which establishes a joint density for all finite values of k, all particular sets of time, and all particular arguments here.
What am I trying to do here?
I'm trying to admit right from the outset that if I try to define this process for an uncountably infinite number of times, I'm going to be dead in the water.
So I had better be happy with the idea of, at least in principle I would like to be able to find out what the joint probability density is for any finite set of points in time.
And if I can find that, I will say I have defined this random process.
And I'll see what I can do with it.
So we need rules that let us find this kind of probability density.
That's the whole objective here.
Because I don't know how to generate a random process that does more than that, that talks about everything all at once.
You can read mathematics texts to try to do this, but I don't know any way of getting from that kind of general mathematics down to anything that we could make sense about in terms of models of reality.
The favorite way we're going to generate this kind of rule is to think of the random process as being a sum of random variables multiplied by orthonormal functions.
In other words, we will have sample values in time, z of t, which are equal to z sub i phi sub i of t. And for each sample function in time, we will then have, for that particular time, as I vary over capital Omega, I'm going to be varying over the values of z sub i.
These are just fixed functions.
And what that's going to give me is a set of random variables here.
So in a way we're doing very much the same thing as we did with functions.
With functions we said, when we're dealing with these the equivalence classes, things get very, very complicated.
The way we're going to visualize a function is in terms of an orthonormal expansion.
And with orthonormal expansions, if two functions are equivalent, they both have the same orthonormal expension.
So now we're saying the same thing when we're dealing with this more general thing, which is stochastic processes.
This is really what the signal space viewpoint of communication is.
It's the viewpoint that you view these random processes in terms of orthonormal expansions where the coefficients are random variables.
And if I deal with a finite number of random variables here, everything is very simple.
Because you know how to add random variables, right?
I mean, at least in principle, you all know how to add random variables.
You've been doing that any time you take a probability class.
A quarter of the exercises you deal with are adding random variables together and finding various things about the sum of those random variables.
So that's the same thing we're going to be doing here.
The only thing is, we can do this for any real value of t, just by adding up this fixed collection of random variables.
And you add it up at a different arguments here because as t changes, phi sub i of t changes.
And this is sort of what we did when we were talking about source waveforms.
It's very much the same idea that we were looking at.
Because with source waveforms, when we turned the waveform into a sequence we were turning the waveform into a sequence by in fact using an orthonormal expansion.
And then what was happening is that the coefficients in the orthonormal expansion, we were using to actually represent the random process.
So we're doing the same thing here.
The same argument.
So, somehow we have to figure out what these joint densities are.
So, as I said before, when we don't know what to do, we make a simple assumption.
And the simple assumption we're going to make is that these random variables at different points in time, of noise, we're going to assume that they're Gaussian.
And a normal Gaussian random variable; this, I hope, is familiar to you, has the density 1 over the square root of 2 pi, e to the minus n squared over 2.
So it's just density that has this nice bell-shaped curve.
It's an extraordinarily smooth thing.
It has a mean of 0.
It has a variance of 1.
How many of you could prove that the variance of this is 1?
Well, in a while you'll all be able to prove it.
Because there are easy ways of doing it.
Other than looking at a table of integrals, which we'll do.
An arbitrary Gaussian random variable is just where you take this normal variable.
These normalized Gaussian random variables are also traditionally called normal variables.
I mean, they have such a central position in all of probability.
And when somebody talks about a normal random variable -- I mean, there's sort of the sense that random variable should be normal.
That's the typical thing.
It's what should happen.
And any time you don't have normal random variables, you're dealing with something bizarre and strange.
And that's carrying it a little too far, but in fact, when we're looking at noise, that's not a bad idea.
If you shift this random variable by a mean z bar, and you scale it by -- excuse me, scale it by sigma.
Then the density of the result becomes 1 over the square root of 2 pi sigma squared, e to the minus z minus z bar over 2 sigma squared.
In other words, the shifting just shifts the whole thing by z bar.
So this density peaks up around z bar, rather than peaking up around 0, which makes sense.
And if you scale it by sigma, then you need a sigma in here, too.
When you have this probability density, you're going to be scaling it this way.
Scaling it that way is what we mean by scaling it.
If you scale it this way to make sigma squared larger, the variance larger, this thing still has to integrate to 1.
So you have to pop it down a little bit also.
Which is why this sigma squared appears here in the denominator.
So anytime we scale a random variable, it's got to be scaled this way.
And it's also got to be scaled down.
So that's why these Gaussian random variables look this way.
We're going to be dealing with these Gaussian random variables so much that we'll refer to this random variable as being normal with mean z bar and sigma variance sigma squared.
And if we don't say anything other than it's normal, we mean it's normal with mean 0 and variance 1.
Which is the nicest case of all.
It's this nice bell-shaped curve which everything should correspond to.
They tend to be good models of things for a number of reasons.
One of them is the central limit theorem.
And the central limit theorem, I'm not going to state it precisely here.
I'm certainly not going to prove it.
One of the nastiest things to prove in probability theory is the central limit theorem.
It's an absolute bear.
I mean, it seems so simple.
And you try to prove it, and it's ugly.
But, anyway, the central limit theorem says that if you add up a large number of Gaussian random variables, of independent Gaussian random variables, and you scale the sum down to make it variance one, you have to scale it down.
Then when you get all done adding them all up, what you wind up with is a Gaussian random variable.
This is saying something a little more precise than the law of large numbers.
The law of large numbers says you add up a bunch of independent or almost independent random variables.
And you scale them down.
And you get a mean, which is something.
This is saying something more.
It's saying you add them all up, and you scale them down in the right way.
And you get not only the mean, now, but you also get the shape of the density around the mean.
Which is this Gaussian shape, if you add a lot of them.
It has a bunch of extremal properties.
In a sense, it's the most random of all random variables.
Not going to talk about that, but every once in a while, one of these things will come up.
When you start doing an optimization in probability theory, over all possible random variables of a given mean and variance, you often find that the answer that you come up with either when you maximize or when you minimize is Gaussian.
If you get this answer when you maximize, then what you get when you minimize is usually a random variable, which is zero or something.
And vice versa.
So that either the minimum or the maximum of a problem makes sense.
And the one that makes sense, the answer is usually Gaussian.
At least, it's Gaussian for the problems for which it's Gaussian.
But it's Gaussian for an awful lot of problems.
It's very easy to manipulate analytically.
Now, you look at that density, and you say, my God, that can't be easy to manipulate analytically.
But in fact, after you get used to two or three very small tricks, in fact you'll find out that it's very easy to manipulate analytically.
It's one of the easiest random variables to work with.
Far easier than a binary random variable.
If we add up a bunch of binary random variables, and what do you get?
You get a binomial random variable.
And binomial random variables, when you add up enough of them, they get uglier and uglier.
Except they start to look Gaussian.
These Gaussian random variables, if you add up a bunch of them, and what do you get?
You get a Gaussian random variable.
Yes?
Central limit theorem applies to random variables or Gaussian random variables?
Central limit theorem applies to random variables
[INAUDIBLE]
Oh, I'm sorry.
I shouldn't have.
No.
I said, when you add a bunch of Gaussian random variables, you get exactly a Gaussian random -- when you add up a bunch of jointly Gaussian random variables, and we'll find out what that means in a while, you get exactly a Gaussian random variable.
When you add up a bunch of random variables which are independent, or which are close enough to independent, and which have several restrictions on them and you scale them the right way.
What you get as you add more and more of them, tends toward a Gaussian distribution.
And I don't want to state all of those conditions.
I mean, we don't need all those conditions here.
Because the only thing we're relying on is when we look at noise, we're looking at a collection of a very large number of phenomena.
And because we're looking at a very large number of phenomena, we can sort of model it as being an addition of a lot of little random variables.
And because of that, in an enormous number of situations, when you look at these things, what you get is fairly close to a Gaussian shape.
And it's a model that everybody uses for noise.
If you read papers in the communication field, or in the control field, or in a bunch of other fields, and people start talking about what happens when noise enters the picture, sometimes they will talk about their model for the noise process.
Sometimes they won't.
But if they don't talk about what noise model they're using, it's a very safe bet that what they're using is a Gaussian process model.
In other words, this is the kind of model people use because it's a model that makes a certain amount of sense.
Now, there's one other thing you ought to be aware of here.
It's something that we've talked about a little bit in other contexts.
When we're looking at a noise waveform that you have to deal with in a modem or something like that, this modem only experiences one noise waveform in its life.
So, what do we mean by saying that it's random?
Well, it's an act of faith, in a sense.
But it's not so much.
Because if you're designing communication equipment, and you're cookie cuttering cellphones or something like that, or something else, what you're doing is designing a piece of equipment which is supposed to work in an enormous number of different places.
So that each one you're manufacturing is experiencing a different waveform.
So that as far as the designing the device is concerned, what you're interested in is not that one waveform that one of these things experiences, but in fact the ensemble of waveforms that all of them experience.
Now, when you look at the ensemble of waveforms that all of these things experience, over all of this large variety of different contexts; noise here, noise there, noise there, suddenly the model of a large collection of very small things makes even better sense.
Because in each context, in each place, you have some forms of noise.
When you look at this over all the different contexts that you're looking at, then suddenly you have many, many more things that you, in a sense, are just adding up if you want to make a model here.
So we're going to use this model for a while.
We will find out that it leads us to lots of nice things.
And so forth.
So, we said that we'd be happy if we could deal with at least studying a random process over some finite set of times.
So it makes sense to look at a finite set of random variables.
So we're going to look at a k-tuple of random variables.
And we're going to call that -- what do you think we're going to call a k-tuple of random variables?
We're going to call it a random vector.
In fact, when you look at random vectors, you can describe them as being elements of a vector space.
And we're not going to do that because most of you are already sufficiently confused about doing functions as vectors, and adding on the idea of a random process as a vector would be very unkind.
And we're not going to do it because we don't need to do it.
We're simply calling these things vectors because we want to use the notation of vector space, which will be convenient here.
So, if we have a collection of random variables n 1 to n k, and they're all IID, independent identically distributed.
They're all normal with mean zero and variance one, the joint density of them is what?
Well, you know what the joint density is.
If they're independent and they're all the same, you just multiply their densities together.
So it's 1 over 2 pi with a k over 2. e to the minus n 1 squared minus n 2 squared, and so forth.
Divided by 2.
That's why this density form is a particularly convenient thing.
You have something in the exponent.
When you have independent random variables, you multiply densities.
Which means you add what's in the exponent.
Which means everything is very nice.
So you wind up adding up all these sample values for each of the random variables.
With our notation looking at norm squared, this is just the norm squared for this sequence of sample values.
And look at the sequence of samples values is a actual vector, n 1 up to n sub k. So we wind up width the norm of the vector n squared divided by 2.
If you draw a picture of that, you have spherical symmetry.
Every set of sample vectors which have the same energy in them has the same probability density.
If you draw it in two dimensions, you get -- it looks like a target where the biggest probability density is in here, and the probability density goes down as you move out.
If you look at it in three dimensions, if that same thing's looked at in a sphere.
And if you look at it in four dimensions, well, it's the same thing, but I can't draw it, I can't imagine it.
But it's still a spherical symmetry.
In the one minute that I have remaining, I'm going to -- I might derive a Gaussian density.
I'm going to call a bunch of random variables zero-mean jointly Gaussian if they're all derived by linear combinations on some set of normal IID Gaussian in random variables.
In other words, if each of them is a linear combination of these IID random variables, I will call the collection z 1 up to z k a jointly Gaussian random variable.
In other words, jointly Gaussian does not mean that z 1 is Gaussian, z 2 is Gaussian, up to z k being Gaussian.
It means much more than that.
It means that all of them are linear combinations of some underlying set of independent random variables.
In the homework that was passed out this time, you will look at two different ways of generating a pair of random variables which are each Gaussian, but which are not jointly Gaussian.
Which you can't view in this way.
So this is important.
Jointly Gaussian makes sense physically because when we look at random variables, we say it makes sense to look at Gaussian random variables because of a collection of a lot of noise variables.
If you look at two different random variables, each of them is going to be a collection of different small noise variables.
And they're each going to be some weighted sum over a different set of noise variables.
So it's reasonable to expect each of them to be expressible in this way.
I guess I'm not going to derive what I wanted to derive.
I'll do it next time.
It's easier to think of this in matrix form.
In other words, I'll take the vector of sample values, z 1 up to z sub k. Each one of them is a linear combination of these normal random variables, n 1 up to n sub n. So I can do it as z equal to matrix a times n. This is a vector of real numbers.
This is a vector of real numbers, this is a matrix.
And if I make this a square matrix, than what we're doing is mapping each of the unit vectors, each single noise vector, into the i'th column of a.
In other words, if I have a noise vector where n 1 is equal to 0, n 2 is equal to 0, n sub i is equal to 1 and everything else is equal to 0, what that generates in terms of this random vector z is just the vector a sub 1.
So.
What I'm going to do when I come back to this next time is, if I take a little cube in the n space, and I map it into what goes on in the z space, this unit vector here is going to map into a vector here.
The unit vector here is going to map into a vector here.
So cubes map into parallelograms.
Next time, the thing we will say is, if you look at the determinant of a matrix, the determinant of a matrix, the most fundamental definition of it, is that it's the volume of a parallelepiped.
In fact, it's the volume of this parallelepiped.
So we'll do that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu
OK. We are talking about jointly Gaussian random variables.
One comment through all of this and through all of the notes is that you can add a mean to Gaussian random variables, or you can talk about zero-mean random variables.
Here we're using random variables mostly to talk about noise.
When we're talking about noise, you really should be talking about zero-mean random variables, because you can always take the mean out.
And because of that, I don't like to state everything twice once for variables of processes without a mean, and once for variables of processes with a mean.
And after looking at the notes, I think I've been a little inconsistent about that.
I think the point is you can keep yourself straight by just saying the only thing important is the case without a mean.
Putting the mean in is something unfortunately done by people who like complexity.
And they have unfortunately got in the notation on their side.
So anytime you want to talk about a zero-mean random variable.
You have to say zero-mean random variable.
And if you say random variable, it means it could have a mean or not have a mean.
And of course the way the notation should be stated is you talk about random variables as things which don't have means.
And then you can talk about random variables plus means as things which do have means, which would make life easier but unfortunately it's not that way.
So anytime you see something and are wondering about whether I've been careful about the mean or not, the answer is I well might not have been.
And two, it's not very important.
So anyway.
Here I'll be talking all about zero-mean things.
A k-tuple of zero-mean random variables is said to be jointly Gaussian if you can express them in this way here.
Namely as a linear combination of IID normal Gaussian again random variables.
OK.
In your homework, those of you who've done it already, have realized that just having two Gaussian random variables is not enough to make those two random variables be jointly Gaussian.
They can be individually Gaussian but not jointly Gaussian.
This is sort of important because when you start manipulating things as we will do when we're generating signals to send and things like this, you can very easily wind up with things which look Gaussian and are appropriately modeled as Gaussian, but which are not jointly Gaussian.
Things which are jointly Gaussian are defined in this way.
We will come up with a couple of other definitions of them later.
But you've undoubtedly been taught that any old Gaussian random variables are jointly Gaussian.
You've probably seen joint densities for them and things like this.
Those joint densities only apply when you have jointly Gaussian random variables.
And the fundamental definition is this.
This fundamental definition makes sense, because the way that you generate these noise random variables is usually from some very large underlying set of very, very small random variables.
And the Law of Large Numbers says that when you add up a very, very large number of small underlying random variables, you normalize the sum so it has some reasonable variance then that random variable is going to be appropriately modeled as being Gaussian.
If you at the same time look at linear combinations of those things for the same reason, it's appropriate to model each of the random variables you're looking at as a linear combination of some underlying set of noise variables which is very, very large.
Probably not Gaussian.
But here when we're trying to do this, because of the central limit theorem, we just model these as normal Gaussian random variables to start with.
So that's where that definition comes from.
It's why when you're looking at noise processes in the real world, and trying to model them in a sensible way this jointly Gaussian is the thing you almost always come up with.
OK.
So one important point here, and the notes point this out.
If each of these random variables Z sub i is independent of each of the others, and they have arbitrary variances, then because of this formula, the set of Z i's are going to be jointly Gaussian.
Why is that?
Well you simply make a matrix here A, which is diagonal.
And the elements of this diagonal matrix are sigma 1 squared, sigma 2 squared, sigma 3 squared, and so forth.
If you have a vector Z, which is then sigma 1 squared down to sigma k squared times a noise vector N1 to N sub k. What you wind up with is Z sub i is going to be equal to -- I guess I don't want those squares in there.
Sigma 1 up to sigma k -- Z sub i is going to be equal to sigma i, N sub i; N sub i is a Gaussian random variable with variance one and therefore Z sub i as Gaussian random variable with variance sigma sub i squared.
OK.
So anyway, one special case of this formula is anytime that you want to deal with a set of independent Gaussian random variables with arbitrary variances, they are always going to be jointly Gaussian.
Saying that they're uncorrelated is not enough for that.
You really need the statement that they're independent of each other.
OK. That's sort of where we were last time.
When you look at this formula, you can look at it in terms of sample values.
And if we look at a sample value what's happening is that the sample value of the random vector Z, namely little z, is going to be defined as some matrix A times this sample value for the normal vector N. OK. And what we want to look at is geometrically what happens there.
Well this matrix a is going to map a unit vector, E sub i into the i'th column of A.
Why is that?
Well you look at A, which is whatever it is, A sub 1, 1, blah, blah, blah up to A sub k, k. And you look at multiplying it by a vector which is zero only in the i'th position.
And what's this matrix multiplication going to do?
It's going to simply pick out the i'th column of this matrix here.
OK.
So a is going to map e sub i into the i'th column of A. OK.
So then the question is what is this matrix A going to do to some small cube in the n plane?
OK.
If you take a small cube in the n plane from 0 to delta along the n1 line is going to map into 0 to this point here, which is A e sub 1.
This point here is going to map into A times e sub 2.
This is just drawing for two dimensions of course.
So in fact all the points in this cube are going to get map into this little rectangle here.
OK. Namely, that's what a matrix times a vector is going to do.
Anybody awake out there?
You're all looking at me as if I'm totally insane.
OK. Every everyone following this?
OK.
So perhaps this is just too trivial.
I hope not.
OK.
So unit cubes get mapped into rectangles here.
If I take a unit cube up here, it's going to get mapped into the same kind of unit rectangle here.
If I visualize tiling this plane here with little tiny cubes, delta on a side, what's going to happen?
Each of these little cubes is going to map into a parallelogram over here, and these parallelograms going to tile this space here.
OK.
Which means that each little cube here maps into one of these rectangles.
Each rectangle here maps back into a little cube here.
Which means that I'm looking at a very special case of a here.
I'm looking at a case where a is non-singular.
In other words, I can get the any point in this plane by starting with some point in this plane.
Which means for any point here, I can go back here also.
In other words, I can also write n is equal to A to the minus 1 times Z.
And this matrix has to exist.
OK. That's what you mean geometrically by a non-singular matrix.
It means that all points in this plane get mapped into points in this plane.
And get mapped into only one point in this plane, and get mapped into only one point in this plane.
Where every point in this plane is the map of some point here.
In other words you can go from here to there.
You can also go back again.
OK.
The volume of a parallelepiped here, and this is in an arbitrary number of dimensions is going to be the determinant of A.
And you all know how to find determinants.
Namely you program a computer to do it, and the computer does it.
I mean it used to be we had to do this by an enormous amount of calculation.
And of course nobody does that anymore.
OK.
So we can find the volume of this parallelepiped, it's this determinant.
And what does all of that mean?
Well first let me ask you the question.
What's going to happen if the determinant of a is equaled to zero?
What does that mean geometrically?
What does that mean here in terms of this two dimensional diagram?
[INAUDIBLE]
What?
Projection onto a line
Yeah.
This little cube here is simply going to get projected onto some line here.
Like for example that.
In other words it means that this matrix is not invertible for one thing, but it also means everything here gets mapped onto some lower dimensional sub-space here in general.
OK. Now remember that for a minute, and we'll come back to that when we start talking about probability densities.
OK because of the picture we were just looking at, the density of the random variables Z at A times N, namely the density of Z at this particular value here is just the density that we get corresponding to the density of some point here mapped over into here.
And what's going to happen when we take a point here and map it over into here?
If you have a certain amount of density here, which is probability per unit volume.
Now when you map it into here, and the determinant is bigger than zero, what you're doing is mapping a little volume into a big volume.
And if you're doing that over small enough region where the probability density is of such essentially fixed, what's going to happen to the probability density over here?
It's going to get scaled down, and it's going to get scaled down precisely by that determinant.
OK.
So what this is saying is the probability density of the random variable z, which is linear combination of these normal random variables, is in fact the probability density of this normal vector N, and we know what that probability density is, divided by this determinant of a.
In fact this is a general formula for any old probability density at all.
You can start out with anything which you call a random vector N and you can derive the probability density of any linear combination of those elements in precisely this way.
So long as this volume element is non-zero, which means you're not mapping an entire space into a sub-space.
When you're mapping an entire space into a sub-space and you define density as being density per unit volume, of course the density in this map space doesn't exist anymore.
Which is exactly what this formula says.
If this determinant is zero, it means this density here is going to be infinite in the regions where this z exists at all, which is just in this linear sub-space, and what do you do about that?
I mean do you get all frustrated about it?
Or do you say what's going on and treat it in some sensible way?
I mean the thing is happening here.
And this talks about it a little bit.
If a is singular, then A is going to map Rk into a proper sub-space.
Determinant A is going to be equal to 0.
The density doesn't exist.
So what do you do about it?
I mean what does this mean if you're mapping into a smaller sub-space.
What does it mean in terms of this diagram here?
Well in the diagram here it's pretty clear.
Because these little cubes here are getting mapped into straight lines here.
Yeah?
What?
[INAUDIBLE]
Some linear combinations of this are being mapped into 0.
Namely if this straight line is this way any Z which is going in this direction is being mapped into 0.
Any vector Z which is going in this direction has to be identically equaled to 0.
In other words some linear combination of n1 and n2 is a random variable which takes on the value 0 identically.
Why do you try to represent that in a probabilistic sense?
Why don't you just take it out of consideration altogether?
Here what it means is that z1 and z2 are simply linear combinations of each other.
OK.
In other words once you know what the sample value of z1 is, you can find the sample values z2.
In other words z2 is a linear combination of z1.
It's linearly dependent on z1, which means that you can identify it exactly once you know what z1 is.
Which means you might as well not call it a random variable at all.
Which means you might as well view this situation where the determinant is 0 as where the vector Z is really just one random variable, and everything else is determined.
So you throw out these extra random variables, pseudo-random variables, which are really just linear combinations of the others.
So you deal with a smaller dimensional set.
You find the probability density of the smaller dimensional set, and you don't worry about all of these mathematical peculiarities that would arise otherwise.
OK.
So once we do that, A is going to be non-singular.
Because we're going to be left with a set of random variables, which are not linearly dependent on each other.
They can be statistically dependent on each other, but not linearly dependent, OK.
So for all z then, since determinant A is not 0, the probability density at some arbitrary vector Z is going to be the normal joint density evaluated at a to the minus 1z divided by the determinant of A. OK.
In other words we're just working the map backwards.
This formula is the same as this formula, except instead of writing An here, we're writing z here.
And when An is equal to z then little n has to be equal to A to the minus 1z.
And what does that say?
It says that the joint probability density has to be equal to this quantity here.
Which in matrix terms looks rather simple, it looks rather nice.
You can rewrite this.
This is a norm here, so it's this vector there times this vector.
Where in fact when you want to multiply vectors in this way, you're taking inner product of the vector with this cell.
These are real vectors we're looking at.
Because we're trying to model real noise, because we're modeling the noise on communication channels.
So this is going to be the inner product of A to the minus 1z with itself, which means you want to look at the transform of a to the minus 1z.
Now what's the transform of a to the minus 1z?
It's z transform times a to the minus 1 transform.
So we wind up with a to the minus 1 transform times a to the minus 1 times z.
So we have some kind of peculiar bilinear form here.
So for any sample value of the random vector Z we can find the probability density in terms of this quantity here.
Which looks a little bit peculiar, but it doesn't look too bad.
OK. Now we want to simplify that a little bit.
Anytime you're dealing with zero-mean random variables -- now remember I'm going to forget to say zero-mean half the time because everything is concerned with zero-mean random variables.
The covariance of Z1 and Z2 is expected value of Z1 times Z2.
So if you have a k-tuple Z, the covariance is a matrix whose i,j element is expected value of Zi times Zj.
And what that means is that the covariance matrix, this is a matrix now, it's going to be the expected value of z times z transpose.
Z is a random vector, which is a column random vector, Z transpose is a row-random vector, which is this simply turned upside down, turned by 90 degrees.
Now when you multiply the components of this vector together, you can see that what you get is the elements of this covariance matrix.
In other words this is just standard matrix manipulation, which I hope most of you or at least partly familiar with.
OK.
When we then talk about the expected values Z times Z transpose we can write this as the expected value of A times N, which is what z is.
N transpose times A transpose, which is what Z transpose is.
And miraculously here the N and the N transpose are in the middle here, where it's easy to deal with them, because these are normal Gaussian random variables.
And when you look at this column times this row, since all diagonal elements are independent of each other, and all of them have variance one, the expected value of N times N transpose is simply the identity matrix.
All the randomness goes out of this which it obviously has to because we're looking at a covariance which is just a matrix and not something random.
So you wind up with this covariance matrix is equal to a rather arbitrary matrix A, but not singular, times the transpose of that matrix.
OK. We've assumed that A is non-singular and therefore it's not too hard to see the k sub Z is non-singular also.
And explicitly case of Z mainly its co-variance matrix, the inverse of it, is A to the minus 1 transpose times A to the minus 1.
Namely you take the inverse and you start flipping things and you do all of these neat matrix things that you always do.
And you should review them if you've forgotten that.
So that when we write our probability density, which was this in terms of this transformation here, what we get is the density of Z is equal to, in place of determinant of A, we get the square root of the determinant in the k sub Z.
You probably don't remember that, but is what you get.
And it's sort of a blah.
Here this is more interesting.
This is minus 1/2 z transpose times Kz to the minus 1 times z.
What does this tell you?
Look at this formula.
Is it just a big formula or what does it say to you?
You got to look at these things and see what they say.
I mean we've gone through a lot of work to derive something here
[INAUDIBLE]
Well it is Gaussian.
Yes.
I mean that's the way we define jointly Gaussian.
But what's the funny thing about this probability density?
What does it depend on?
What do I have to tell you in order for you to calculate this probability density for every z you want to plug in here?
I have to tell you what this covariance matrix is.
And once I tell you what the covariance matrix is, there is nothing more to be specified.
In other words, a jointly Gaussian random vector is completely specified by its covariance matrix.
And it's specified exactly this way by its covariance matrix.
OK.
There's nothing more there.
So this says anytime you're dealing with jointly Gaussian, the only thing you have to be interested in is this covariance here.
Namely all you need to have jointly Gaussian is somebody has to tell you what the covariance is, and somebody has to tell you also that it's jointly Gaussian.
Jointly Gaussian plus a given covariance specifies the probability density.
OK. What does that tell you?
Well let's look at an example where we just have two random variables, Z1 and Z2.
then expected value of Z1 squared is the upper.
Left hand element of that covariance matrix, which we'll call sigma 1 squared.
The lower, right hand side of the matrix is K22, which we'll call sigma 2 squared.
And we're going to let rho be the normalize covariance.
We're just defining a bunch of things here, because this is the way people usually define this.
So rho will be the normalized cross covariance.
Then the determinant of the Kz is this mess here.
For A to be non-singular, we have to have rho less than 1.
If rho is equal to 1 then this determinant is going to be equal to 0, and we're back in this awful case that we don't want to think about.
So then if we go through the trouble of finding out what the inverse of K sub z is, we find this 1 over 1 minus row square times this matrix here.
The probability density plugging this in is this big thing here.
OK what does that tell you?
Well the thing that it tells me is that I never want to deal with this, and I particularly don't want to deal with it if I'm dealing with three or more variables.
OK.
In other words the interesting thing here is the simple formula we had before, which is this formula.
OK. And we have computers these days which say given nice formulas like this, their standard computer routines to calculate things like this.
And you never want to look at some awful mess like this.
OK. And if you put a mean into here, which you will see in every textbook on random variables and probability you ever look at, this thing becomes so ugly that you were probably convinced before you took this class that jointly Gaussian random variables were things you wanted to avoid like the plague.
And if you really have to deal with explicit formulas like this, you're absolutely right.
You do want to avoid them like the plague, because you can't get any insight from what that says, or at least I can't.
So I say OK, we have to deal with this.
But yet we like to get a little more insight about what this means.
And to do this, we like to find a little bit more about what these bilinear forms are all about.
Those of you who have taken any course on linear algebra have dealt with these bilinear forms.
And played with them forever.
And those of you who haven't are probably puzzled about how to deal with them.
The notes have an appendix, which is about two pages long which tells you what you have to know about these matrices.
And I will just sort of quote those results as we go.
Incidentally those results are not hard to derive.
And not hard to find out about.
You can simply derive them on your own, or you can look at Strang's book on linear algebra, which is about the simplest way to get them.
And that's all you need to do.
OK. We've said the probability density depends on this bilinear form z transpose times Kz to the minus 1 time z.
What is this?
Is this a matrix or a vector or what?
How many people think it's a matrix?
How many people think it's a vector?
You think it's a vector?
OK. Well in a very peculiar sense it is.
How many people think it's a number?
Good.
OK.
It is a number, and it's a number because this is a row vector.
This is a matrix.
This is a column vector.
And if you think of multiplying a matrix times a column vector, you get a column vector.
And if you take a row vector times a column vector, you got a number.
OK.
So this is just a number which depends on little z. OK. Kz is called a positive definite matrix.
And it's called a positive definite matrix, because this thing is always non-negative.
And it always has to be non-negative because this refers to the -- if I put capital Z in here, namely if I put the random vector in here Z, then what this is, is the variance of a particular random variable.
So it has to be greater than or equal to zero.
So anyway K sub z is always non-negative definite.
Here it's going to be positive definite, because we've already assumed that the matrix A was non-singular, and therefore the matrix Kz has to be non-singular.
So this has to be positive definite.
So it has an inverse, K sub z minus 1 is also positive definite which means this quantity is always greater than zero, if z is non-zero.
You can take these positive definite matrices and you can find eigenvectors and eigenvalues for them.
Do you ever actually calculate these eigenvectors and eigenvalues?
I hope not.
It's a mess to do it.
I mean it's just as bad as writing that awful formula we had before.
So you don't want to actually do this, but it's nice to know that these things exist.
And because these vectors exist, and in fact if you have a matrix which is k by k, little k by little k, then there are k such eigenvectors and they can be chosen orthonormal.
OK.
In other words each of these Q sub i are orthogonal to each of the others.
You can clearly scale them, because you scale this and scale this together.
And it's going to maintain equality.
So you just scale them down so you can make them normalize.
If you have a bunch of eigenvectors with the same eigenvalue, then the whole linear sub-space formed by that set of eigenvectors all have the same eigenvalue lambda sub i. Namely you take any linear combination of these things which satisfy this equation for a particular lambda.
And any linear combination will satisfy the same the same equation.
So you can simply choose an orthonormal set among them to satisfy this.
And if you look at Q sub i and Q sub j, which have different eigenvalues, then it's pretty easy to show that in fact they have to be orthogonal to each other.
So anyway when you do this you wind up with this form becomes just the sum over i of lambda sub i to the minus 1.
Namely these eigenvalues to the minus 1.
These eigenvalues are all positive.
Times the inner product of z with Q sub i.
In other words, you take whatever vector Z you're interested in here, you project it on these k orthonormal vectors Q sub i.
You get those k values.
And then this form here is just that sum there.
So when you write the probability density in that way -- we still have this here we'll get rid of that in the minute -- you have e to the minus sum over i, these inner product terms squared divided by 2 times lambda sub i.
That's just because this is equal to this.
It's just substituting this for this in the formula for the probability density.
OK. What does that say pictorially?
Let me show you a picture of it first.
It says that for any positive definite matrix and therefore for any covariance matrix, you can always find these orthonormal vectors.
I've drawn them here for two dimensions.
They're just some arbitrary vector q1; q2 has to be orthogonal to it.
And if you look at the square root of lambda 1 times q1, you got a point here.
You look at the square root of lambda 2 times q2, you got a point here.
If you then look at this probability density here, you see that all the points on this ellipse here have to have the same sum of z times Q sub i. OK.
It looked a little better over here.
Yes.
OK. Namely the points little z for which this is constant are the points which form this ellipse.
And it's the ellipse which has the axes square root of lambda i times Q sub i.
And then you just imagine it if it's lined up this way.
And think of what you would get if you were looking at the lines of equal probability density for independent Gaussian random variables with different variances.
I mean we already pointed out if the variances are all the same, these equal probability contours are spheres.
If you now expand on some of the axes, you get ellipses.
And now we've taken these arbitrary vectors, so these are not in the directions we started out with, but in some arbitrary directions.
We have q1 and q2 are orthornormal to each other, because that's the way we've chosen them.
And then the probability density has this form which is this form.
And the other thing we can do is to take this form here and say gee this is just a probability density for a bunch of independent random variables, where the independent random variables are the inner product of the random vector Z with Q sub 1 up to the inner product of z with Q sub k. So these are independent Gaussian random variables.
They have variances lambda sub i.
And this is the nicest formula for the probability density of an arbitrary set of jointly density of jointly Gaussian random variables.
OK.
In other words what this says is in general if you have a set of jointly Gaussian random variables and I have this messy form here, all you're doing is looking at them in a wrong coordinate system.
If you switch them around, you look at them this way instead of this way, you're going to have independent Gaussian random variables.
And the way to look at them is found by solving this eigenvector eigenvalue equation, which will tell you what these orthogonal directions are.
And then it'll tell you how to get this nice picture that looks this way.
OK. OK so that tells us what we need to know, maybe even a little more than we have to know, about jointly Gaussian random variables.
But there's one more bonus we get.
And the bonus is the following.
If you create a matrix here B where the i'th row of B is this vector Q sub i divided by the square root of lambda sub i, then what this is going to do is the corresponding element here is going to be a normalized Gaussian random variable with variance one and all of these are going to be independent of each other.
OK. That's essentially what we were saying before.
This is just another way of saying the same thing.
That when you squish this probability density around and look at it in a different frame of reference, and then you scale the random variables down or up, what you wind up with is IID normal Gaussian random variables.
OK.
But that says that Z is equal to B to the minus 1 times N prime.
Well so what?
Here's the reason for so what.
We started out with a definition of jointly Gaussian, which is probably not the definition of jointly Gaussian you've ever seen if you've seen this before.
Then what we did was to say, OK if you have jointly Gaussian random variables and they're not linearly independent and none of them are linearly dependent on the others, then this matrix A is invertible.
From that we derive the probability density.
From the probability density we derive this.
OK.
The only thing you need to get this is the probability density.
And this says that anytime you have a random vector Z with this probability density that we just wrote down.
Then you have the property that N prime is equal to BZ, and Z is equal to B to the minus 1 N prime.
Which says that if you have this probability density, then you have jointly Gaussian random variables.
So we have an alternate definition of jointly Gaussian.
Random variables are jointly Gaussian if they have this probability density.
You can somehow represent them as linear combinations of normal random variables.
OK. Then there's something even simpler.
It says that if all linear combinations of a random vector Z are Gaussian, then Z is jointly Gaussian.
Why is that?
well If you look at this formula here, it says take any old random vector at all that has a covariance matrix.
From that covariance matrix, we can solve for all of these eigenvectors.
If I find the appropriate random variables here from this transformation, those random variables are then uncorrelated from each other, they are all statistically independent of each other, And it follows from that, that if all these linear combinations are Gaussian then Z has to be jointly Gaussian.
OK.
So we now have three definitions.
This one is the simplest one to state.
It's the hardest one to work with.
That's life you know.
This one is probably the most straightforward, because you don't have to know anything to understand this definition.
This original definition is physically the most appealing because it shows why noise vectors actually do have this property.
OK.
So here's a summary of all of this.
It says if Kz is singular, you want to remove the linearly independent random variables.
You just take them away because they're uniquely specified in terms of the other variables.
And then you take the resulting non-singular matrix, and Z is going to be jointly Gaussian if and only if Z is equal to AN for some normal random variable N. If Z has jointly Gaussian density or if all linear combinations of Z are Gaussian all of this for 0 mean would mean it applies to fluctuation.
OK.
So why do we have to know all of that about jointly Gaussian random variables?
Well because everything about Gaussian processes depends on jointly Gaussian random variables.
You can't do anything with Gaussian processes without being able to look at these jointly Gaussian random variables.
And the reason is that we said that Z of t is a Gaussian process if for every k and every set of time instance, every set of e, that's t1 to t sub k. If Z of t1 up to Z of tk is a jointly Gaussian random vector.
OK.
So that directly links the definition of a Gaussian process to Gaussian random vectors.
OK.
Supposed the sample functions of Z of t or L2 with probability one.
I want to say a little bit about this because otherwise you can't sort out any of these things about how L2 theory connects with random processes.
OK.
So I'm going to start out by just assuming that all these sample functions are going to be L2 functions with probability 1.
One way to ensure this is to look only at processes of the form Z of t equals some sum of Z sub i times ti of t. OK remember at the beginning we started looking at this process the sum of a set of normalized Gaussian random variables times sinc functions times displaced sinc functions.
You can also do the same thing with Fourier coefficients or anything.
You got a fairly general set of processes that way.
unfortunately they don't quite work, because if you look at the sinc functions and you look at noise, which is independent and identically distributed in time, then the sample functions are going to have infinite energy.
I mean that kind of process just runs on forever.
It runs on with finite power forever.
And therefore it has infinite energy.
And therefore the simplest process to look at, the sample functions are non-L2 with probability one, which is sort of unfortunate.
So we say OK we don't care about that, because if you want to look at that process a sum of Z sub i times sinc functions, what do you care about?
I mean you only care about the terms in that expansion, which run from the big bang until the next big bang.
OK. We certainly don't care about it before that or after that.
And if we look within those finite time limits, then all these functions are going to be L2.
Because they just last for a finite amount of time.
So all we need to do is to truncate these things somehow.
And we're going to diddle around a little bit with a question of how to truncate these series.
But for the time being we just say we can do that.
And we will do it.
So we can look at any processes the form sum of Zi times phi i of t, where the Zi are independent and the phi i of t are orthonormal.
And to make things L2, we're going to assume that the sum over i of Zi squared bar is less than infinity.
OK.
In other words we only take a finite number of these things, or if we want to take infinite a number of them, the variances are going to go off to zero.
And I don't know whether you're proving it in the homework this time or you will prove it in the homework next time, I forget, but you're going to look at the question of why this finite variance condition makes these sample functions be L2 with probability one.
OK.
So if you had this condition, then all your sample functions are going to be L2.
I'm going to get off all of this L2 business relatively shortly.
I want to do a little bit of it to start with.
Because if any of you have start doing any research in this area, at some point you're going to be merrily working away calculating all sorts of things.
And suddenly you're going to find that none of it exists, because of these problems of infinite energy.
And you're going to get very puzzled.
So one of the things I tried to do in the notes is to write them in way that you can understand them at a first reading without worrying about any of this.
And then when you go back for a second reading, you can pick up all the mathematics that you need.
So that in fact you won't have the problem of suddenly finding out that three-quarters of your thesis has to be thrown away, because you've been dealing with things that don't make any sense.
OK.
So, we're going to define linear functionals in the following way.
We're going to first look at the sample functions of this random process Z. OK. Now we talked about this last time.
If you have a random process Z than really what you have is a set of functions defined on some samples space.
So the quantities you're interested in is what is the value of the random process at time t for sample point omega.
OK.
If we look at that and make it for a given omega, this thing becomes a function of t. In fact for a given omega, this is just what we've been calling a sample element of the random process.
So if we take this sample element, look at the inner product of that with some function g of t. In other words we look at the integral of Z of t omega times g of t, dt.
And if all these sample functions are L2 and if g of t is L2, what happens when you take the integral of an L2 function times an L2 function, which is the inner product of something L2 with something L2 which says something with finite energy inner product with something with finite energy.
Well the Schwarz inequality tells you that if this has finite energy and this has finite energy, the inner product exists.
That's the reason why we went through the Schwarz inequality.
It's the main reason for doing that.
So these things have finite value.
So V of omega the results of doing this namely V as a function of the sample space is a real number.
And it's a real number for the sample points of omega with probability one, which means we can talk about V as a random variable.
OK. And now V is a random variable which is defined in this way.
And from now on we will call these things linear functionals which are in fact the integral of a random process times a function.
And we can take that kind of integral.
It sort of looks like the linear combinations of things we were doing before when we were talking about matrices and random vectors.
OK.
If we restrict the random process to have the following form, where these are independent and these are orthonormal, then one of these linear functionals is given by the random variable V is going to be the integral of Z of t times g of t, but Z of t is this.
And at this point we're not going to fuss about interchanging integrals with summations.
You have the machinery to do it, because we're now dealing with an L2 space.
We're not going to fuss about it.
And I advise you not to fuss about it.
So we have a sum of these random variables here times these integrals here.
These integrals here are just projections of g of t on this space of orthonormal functions.
So whatever space of orthonormal functions gives you your jollies, use it talk about the inner products on that space.
This gives you a nice inner products space of sequences of numbers.
And then if the z i are jointly Gaussian, then V is going to be Gaussian.
And then to generalize this one little bit further, if you take a whole bunch of L2 functions, g1 of t g2 of t and so forth, you can talk about a whole bunch of random variables V1 up to V sub j, 0.
And V sub j is going to be the integral of Z of t times gj of tdt.
Remember this thing looks very simple.
It looks like the -- like the convolutions you've been doing all your life.
It's not.
It's really a rather peculiar quantity.
This in fact is what we call a linear functional, and is the integral of a random process times this.
Which we have defined in terms of the sample functions of the random process.
And now we said OK now that we understand what it is, we will just write this all the time.
But I just caution you not to let familiarity breed contempt.
Because this is a rather peculiar notion.
And a rather powerful notion.
OK.
So these things are jointly Gaussian.
We want to take the expected value of V sub i times V sub j, and now we're going to do this without worrying about being careful at all.
We have the expected value of the integral of Z of t, gi of t, td times the integral of Z tau gj of tau, d tau.
And now we're going to slide this expected value inside of both of these integrals.
And not worry about it.
And therefore what we're going to have is a double integral of gi of t expected value of z of t time z of tau times gj of tau dt, d tau, which is this thing here.
Which you should compare with what we've been dealing with most of the lecture today.
This is the same kind of form for a covariance function as we've been dealing with for covariance matrices.
It has very similar effects.
I mean before you were just talking about finite dimensional matrices, which is all simple mathematically in eigenfunctions and eigenvalues.
You have eigenfunctions and eigenvalues of these things also.
And so long as these are defined nicely by these L2 properties we've been talking about.
In fact you can deal with these in virtually the same way that you can deal with the matrices we were dealing with before.
If you just remember what the results are from matrices, you can guess what they are for these covariance functions.
OK.
But anyway you can find the expected value of Vi times Vj by this formula.
Again we're dealing with zero-mean and therefore we don't have to worry about the mean, put that in later.
And that all exists.
OK.
So the next thing we want to deal with, hitting you with a lot today.
But I mean the trouble is a lot of this is half familiar to most of you.
People who have taken various communication courses at various places have all been exposed to random processes in some highly trivialized sense.
But the major results are the same as the results we're going through here.
And all we're doing here is adding a little bit of carefulness about what works and what doesn't work.
Incidentally in the notes which is towards the end of lectures 14 and 15, we give three examples which let you know why in fact we want to look primarily at random processes which are defined in terms of a sum of independent Gaussian random variables time orthonormal functions.
And if you look at those three examples, some of them have problems.
Because of the fact that everything you're dealing with has infinite energy.
And therefore it doesn't really make any sense.
And one of them I should talk about this just a little bit in class.
And I think I still have a couple of minutes, is a very strange process where Z of t is IID.
In fact just let it be normal.
And independent for all t. OK.
In other words you generate a random process by looking at an uncountably infinite collection of normal random variables.
How do you deal with such a process?
I don't know how to deal with it.
I mean it sounds like it's simple.
If I put this on a quiz, three-quarters of you would say oh that's very simple.
What we're dealing with is a family of impulse functions.
Spaced arbitrarily closely together.
This is not impulse function.
Impulse functions are even worse than this, but this is bad enough.
When we start talking about spectral density, we can explain this a little bit better by thinking this kind of process it doesn't make any sense.
But this kind of process, if you look at its spectral density, it's going to have a spectral density which is zero everywhere, but whose integral over all frequencies is one.
OK.
In other words it's not something you want to wish on your on your worse friend.
It makes a certain amount of sense as a limit of things.
You can look at a very broadband process where in fact you spread the process out enormously.
You can make pseudo noise which looks sort of like this.
And you make the process broader and broader and lower and lower intensity everywhere.
But it still has this energy of one.
It still has a power of one everywhere.
And it just is ugly.
OK. Now if you never worried about these questions of L2, you would look at a process like that and say, gee there must be some easy way to handle that because it's probably the easiest process you can define.
I mean everything is normal.
If you look at any set at different times, you get a set of IID normal Gaussian variables.
You try to put it together, and it doesn't mean anything.
If you pass it through a filter, the filter is going to cancel it all out.
So anyway, that's one reason why we want to look at this restricted class of random processes we're looking at.
OK. What we're interested in now is we want to take a Gaussian random process really, but you can take any random process.
We want to pass it through a filter, and we want to look at the random process that comes out.
OK. And that certainly is a very physical kind of operation.
I mean any kind of communication system that you build is going to have noise on the channel.
And one of the first things you're going to do is you're going to filter what you've received.
So you have to have some way of dealing with this.
OK. And the way we sort of been dealing with it all along in terms of the transmitted way forms we've been dealing with is to say OK. What we're going to do is to first look what happens when we take sample functions of this, pass them through the filter, and then what comes out is going to be some function again.
And then we're back into what you studied as an undergraduate talking about functions through filters.
We're going to jazz it up a little bit by saying these functions are going to be L2 the filters are going to be L2.
So that in fact you know you'd get something out that make sense.
So what we're doing is looking at these sample functions V the output at time tau for sample point omega is going to be the convolution of the input at time t and sample point omega.
Remember this one sample point exists for all time.
That's why these sample points and sample spaces are so damn complicated.
Because they have everything in them.
OK.
So there's one sample point which exists for all of them.
This is a sample function.
You're passing the sample function through a filter, which is just normal convolution.
What comes out then.
If in fact we express this random process in terms of this orthonormal sum the way we've been doing before.
Is you get the sum over j of this, which is a sample value of the j'th random variable coming out times the integral of pj of t, h of tau minus t, d tau.
OK. For each tau that you look at, this is just a sample value of a linear functional.
OK.
If I want to look at this at one value of tau, I have this integral here which is a random process.
A sample value of a random process at omega time a function.
This is just a function of t for given tau.
OK.
So this is a linear functional.
As a type we've been talking about.
And that linear functional is then given by this for each tau.
This is a sample value of a linear functional we can talk about.
OK.
These things are then, if you look over the whole sample space omega, V of tau becomes a random variable.
OK. V of tau is the random variable whose sample values are V of t omega, and they're given by this.
So if z of t is Gaussian process, you get jointly Gaussian linear functionals at each of any set of times tau 1, tau 2 up to tau sub k. So this just gives you a whole set of linear functionals.
And if z of t is a Gaussian process, then all these linear functionals are going to be jointly Gaussian also.
And bingo.
What we have then is an alternate way to generate Gaussian processes.
OK.
In other words you can generate a Gaussian process by specifying at each set of k times, you have jointly Gaussian random variables.
But once you do that, once you understand one Gaussian process, you're off and running.
Because then you can pass it through any L2 filter that you want to.
And you generate another Gaussian process.
So for example, if you start out with this sinc type process, I mean we'll see that that has a spectral density, which is flat over all frequencies.
And we'll talk about spectral density tomorrow.
But then you pass it through a linear filter, and you can give it any spectral density that you want.
So at this point, we really have enough to talk about arbitrary covariance functions just by starting out with random processes, which are defined in terms of some sequence of orthonormal random variables.
OK. Now we can get to covariance function of a filter process in the same way as we got the matrix for linear functionals.
OK. And this is just computation.
OK.
So what is it?
The covariance function of this output process V evaluated at one time r another time s. One of the nasty things about notation when you start dealing with a covariance function of the input and the output to a linear filter is you suddenly need to worry about two times at the input and two other times at the output.
OK. Because this thing is then expected value of V sub r times the expected value of V subs.
OK.
This is a random variable, which is the process V of r evaluated at time r. This is the random variable, which is the process V of t evaluated at a particular time s. This is going to be the expected value of what this random variable now is the integral of the random process z of t times this function here, which is now a function of t. Because we're looking at a fixed value of r. So this is a linear functional, which is a random variable.
This is another linear functional.
This is evaluated at some time s, which is the output of the filter at time s. Then we will throw caution to the wind interchange integrals with expectation and everything.
And then in the middle we'll have expected value of z of t times z of tau, which is the covariance function of z. OK.
So the covariance function of z then specifies what the covariance function of the output is.
OK.
So whenever you pass a random process through a filter if you know what the covariance function of input to the filter is, you can find the covariance function of the output of the filter.
That's a kind of a nasty formula, it's not very nice.
But anyway the thing that it tells you is that whether this random process is Gaussian or not.
The only thing that determines the covariance function of the output of the filter is the covariance function of the input to the filter plus of course the filter response, which is needed OK. And this is just the same kind of bilinear form that we were dealing with before.
Next time we will talk a little bit about the fact that when you're dealing with a bilinear form like this, you can take these covariance functions and they have the same kind of eigenvalues and eigenvectors that we had before for a matrix.
Namely this is again going to be positive definite as a function, and we will be able to find its eigenvectors and its eigenvalues.
We can't calculate them.
Computers can calculate them.
People who've spend their lives doing this can calculate them.
I wouldn't suggest that you spend your life doing this.
Because again you would be setting yourself up as a second class computer, and you don't make any profit out of that.
But anyway, we can find this in principle from this.
OK. One of the things that we haven't talked about at all yet, and which we will start talking about next time in which the next set of lecture notes, lecture 16, we'll deal with is the question of stationarity.
Let me say just a little bit about that to get into it.
And then we'll talk a lot more about it next time.
The notes will probably be on the web some time tomorrow.
I hope before noon if you want to look at them.
Physically if you look at a stochastic process, and you want to model it.
I mean suppose you want to model a noise process.
How do you model a noise process?
Well you look at it over a long period of time.
You start taking statistics about it over a long period of time.
And somehow you want to model it in such a way, I mean the only thing you can look at is statistics over a long period of time.
So if you're only looking at one process, you can look at it for a year and then you can model it, and then you can use that model for the next 10 years.
And what that's assuming is that the noise process looks the same way this year as it does next year.
You can go further than that and say, OK I'm going to manufacture cell phones or some other kind of widget.
And what I'm interested in then is what these noise wave forms are to the whole collection of my widgets.
Namely different people will buy my widgets.
They will use them in different places.
So I'm interested in modeling the noise over this whole set of widgets.
But still if you're doing that you're still almost forced to deal with models which have the same statistics over at least a broad range of times.
Sometimes when we're dealing with wireless communication we say, no the channel keeps changing in time.
and the channel keeps changing slowly in time.
And therefore you don't have the same statistics now as you have then.
If you want to understands that, believe me, the only way you're going to understand it is to first understand how to deal with statistics for the channel which stay the same forever.
And once you understand those statistics, you will then be in a position to start to understand what happens when these statistics change slowly.
OK.
In other words, what our modeling assumption is in this course, and I believe it's the right modeling assumption for all engineers, is that you never start with some physical phenomena and say, I want to test the hell out of this until I find an appropriate statistical model for it.
You do that only after you know enough about random processes.
That you know how to deal with an enormous variety of different home cooked random processes.
OK.
So what we do in a course like this is we deal with lots of different home cooked random processes, which is in fact why we've done rather peculiar things like saying, let's look at a random process which comes from a sum of independent random variables multiplied by orthonormal functions.
And you see what we've accomplished by that already.
Namely by starting out that way we've been able to define Gaussian processes.
We've been able to define what happens when one of those Gaussian processes goes through a filter.
And in fact, that gives a way of generating a lot more random processes.
And next time what we're going to do is not to say how is it that we know that processes are stationary.
How do you test whether processes are stationary or not.
But we're just going to assume that they're stationary.
In other words if they had the same statistics now as they're going to have next year.
And those statistics stay the same forever.
And then see what we can say about it.
To give you a clue as to how to start looking at this, remember what we said quite a long time ago about Markov chains.
OK. And now when you look at Markov chains, remember that what happens at one time is statistically a function of what happened at the unit of time before it.
OK.
But we can still model those Markov change as being stationary.
Because the dependents at this time on the previous sample of time is the same now as it will be five years from now.
OK.
In other words you can't just look at the process at one instant of time and say this is independent of all other times.
That's not what stationary means.
What stationary means is the way the process depends on the past at time t is the same is the way it depends on the past at some later time tau.
And that in fact is the way we're going to define stationarity.
It's at these joint sample times that we're going to be looking at.
Which I had a better word for that.
Join sets of epochs that we'll be looking at.
The joint statistics over a set of epics is going to be the same now as it will be at sometime in the future.
And that's the way we're going to define stationarity.
A prelude of what we're going to find when we do that is that this covariance function, if the covariance function at time t and time tau is the same if you translate it up to t plus t1 and tau plus t1, then in fact this function is going to be a function only if the difference t minus tau.
It's going to be a function of one variable instead of two variables.
OK.
So as soon as we get this function being a function of one variable instead of two variables, first thing we're going to do is to take the Fourier transform of this.
Because then we'll be taking the Fourier transform of a function of a single variable.
We're going to call that the spectral density of the process.
And we're going to find that for stationary processes the spectrum densities tell you everything if they're Gaussian.
Why is that?
Well the inverse Fourier transform is this covariance function.
And we've now seen that the covariance function tells you everything about a Gaussian process.
So if you know the spectral density for a stationary process, it will tell you everything.
We will also have to fiddle around a little bit about how we define stationarity.
But at the same time don't have this infinite energy problem.
And the way we're going to do it is the way we've done it all along.
We're going to take something that looked stationary, we're going to truncate it over some long period of time, and we're going to have our cake and eat it too that way.
OK.
So we'll do that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
I want to review zero-mean jointly Gaussian random variables.
And I want to review a couple of the other things I did last time.
Because when I get into questions of stationarity and things like that today, I think it will be helpful to have a little better sense of what all that was about, or it will just look like a big mess.
And there are sort of a small number of critical things there that we have to understand.
One of them is that if you have a non-singular covariance matrix, OK, you have a bunch of random variables.
This bunch of random variables, each one of them has a has a covariance.
Namely, z sub i and z sub j have an expected value of z sub i and z sub j. I'm just talking about zero mean here.
Anytime I don't say whether there's a mean or not, I mean there isn't a mean.
But I think the notes are usually pretty careful.
If the covariance matrix is non-singular, then the following things happen.
The vector, z, is jointly Gaussian.
And our first definition was, if you can represent it as a -- all the components of it each as linear combinations of IID normal Gaussian random variables, namely independent Gaussian random variables -- so they're all just built up from this common set of independent Gaussian random variables.
Which sort of matches our idea of why noise should be Gaussian.
Namely, it comes from a collection of a whole bunch of independent things, which all get added up in various ways.
And we're trying to look at processes, noise processes.
The thing that's going to happen then is that, at each epoch in time, the noise is going to be some linear combination of all of these noise effects, but is going to be filtered a little bit.
And because it's being filtered a little bit, the noise at one time and the noise at another time are all going to depend on common sets of variables.
So that's the intuitive idea here.
Starting with that idea, last time we derived what the probability density had to be.
And if you remember, this probability density came out rather simply.
The only thing that was involved here was this idea of going from little cubes in the IID noise domain into parallelograms in the z domain.
Which is what happens when you go through a linear transformation.
And because of that, this is the density that has to come out of there.
By knowing a little bit about matrices -- which is covered in the appendix, or you can just take it on faith if you want to -- you can break up this matrix into eigenvalues and eigenvectors.
If the matrix is nonsingular -- well, if the matrix is singular or nonsingular, if it's a covariance matrix, or it's -- a non-negative definite matrix, is what these matrices are -- then you can always break this matrix up into eigenvalues and eigenvectors.
The eigenvectors span the space.
The eigenvectors can be taken to be orthonormal to each other.
And then when you express this in those terms, this probability density becomes just a product of normal Gaussian random variable densities, where the particular Gaussian random variables are these inner products of the noise vector with these various eigenvectors.
In other words, you're taking this set of random variables which you're expressing in some reference system, where z1 up to z sub k are the different components to the vector.
When you express this in a different reference system, you're rotating that space around.
What this says is, you can always rotate the space in such a way that what you will find is orthogonal random variables, which are independent of each other.
So that this is the density that you wind up having.
And what this means is, if you look at the region of equal probability density over this set of random variables, what you find is it's a bunch of ellipsoids.
And the axes of the ellipsoids are simply these eigenvectors here.
OK, so this is the third way.
If you can represent the noise in this way, again, it has to be jointly Gaussian.
Finally, if all linear combinations of this random vector are Gaussian, that's probably the simplest one.
But it's the hardest one to verify, in a sense.
And it's the hardest one to get all these other results from.
But if all linear combinations of these random variables are all Gaussian, then in fact, again, the variables have to be jointly Gaussian.
Again, it's important to understand that jointly Gaussian means more than just individually Gaussian.
It doesn't mean what you would think from the words jointly Gaussian, as saying that each of the variables are Gaussian.
It means a whole lot more than that.
In the problem set, what you've done in one of the problems is to create a couple of examples of two random variables which are each Gaussian random variables, but which jointly are not jointly Gaussian.
OK, finally, if you have a singular covariance matrix, z is jointly Gaussian if a basis of these zi's are jointly Gaussian.
OK?
In other words, you throw out the random variables which are just linear combinations of the others, because you don't even want to think of those as random variables in a sense.
I mean, technically they are random variables, because they're defined in the sample space.
But they're all just defined in terms of other things, so who cares about them?
So after you throw those out, the others have to be jointly Gaussian.
So in fact, if you have two random variables, z1 and z2, and z2 equals z1 -- OK, in other words the probability density is on a straight diagonal line -- and z1 is Gaussian, z1 and z2 are jointly Gaussian in that case.
This is not an example like the ones you did in this problem set that you're handing in today, where you in fact have things like a probability density which doesn't exist because it's impulsive on this line, and also impulsive on this line.
Or something which looks Gaussian in two of the quadrants and is zero in both the other quadrants, and really bizarre things like that.
OK?
So please try to understand what jointly Gaussian means.
Because everything about noise that we do is based on that.
It really is.
And if you don't know what jointly Gaussian means, you're not going to understand anything about noise detection, or anything from this point on.
I know last year I kept telling people that, and in the final exam there were still four or five people who didn't have the foggiest idea what jointly Gaussian meant.
And you know, you're not going to understand the rest of this, you're not going to understand why these things are happening, if you don't understand that.
OK.
So the next thing we said is Z of t -- I use those little curly brackets around something just as a shorthand way of saying that what I'm interested in now is the random process, not the random variable at a particular value of t. In other words, if I say Z of t, what I'm usually talking about is a random variable, which is the random variable corresponding to one particular epoch of this random process.
Here I'm talking about the whole process.
And it's a Gaussian process.
If Z of t1 to Z of tk are jointly Gaussian for all k and for all sets of t sub i.
And if the process can be represented as a linear combination of just ordinary random variables multiplied by some set of orthonormal functions, and these Z sub i are independent, and if the sum of the variances of these random variables sum up to something less than infinity.
So you have finite energy in these sample functions, effectively is what this says.
Then it says the sample functions of Z of t are L2 with probability one.
OK. You have almost proven that in the problem set.
If you don't believe that you've almost proven it, you really have.
And anyway, it's true.
One of the things that we're trying to do when we're trying to deal with these wave forms that we transmit, the only way we can deal with them very carefully is to know that they're L2 functions.
Which means they have Fourier transforms.
We can do all this stuff we've been doing.
And you don't need anything more mathematically than that.
So it's important to be dealing with L2 functions.
We're now getting into random processes, and it's important to know that the sample functions are L2.
Because so long as the sample functions are L2, then you can do all of these things we've done before and just put it together and say well, you take a big ensemble of these things.
They're all well defined, they're all L2.
We can take Fourier transforms, we can do everything else we want to do.
OK, so that's the game we're playing.
OK, I'm just going to assume that sample functions are L2 from now on, except in a couple of bizarre cases that we look at.
Then a linear functional is a random variable given by the random variable is equal to the integral of the noise process z of t, multiplied by an ordinary function, g of t, dt.
And we talked about that a lot last time.
This is the convolution of a process -- well, it's not the convolution.
It's just the inner product of a process with a function.
And we interpret this last time in terms of the sample functions of the process and the sample values of the random variable.
And then since we could do that, we could really talk about the random variable here.
This means that for all of the sample values in the sample space with probability one, the sample values of the random variable v are equal to the integral of the sample values of the process times g of t dt.
OK, in other words, this isn't really something unusual and new.
I mean, students have the capacity of looking at this in two ways, and I've seen it happen for years.
The first time you see this, you say this is trivial.
Because it just looks like the kind of integration you're doing all your life.
You work with it for a while, and then at some point you wake up and you say, oh, but my god, this is not a function here.
This is a random process.
And you say, what the heck does this mean?
And suddenly you're way out in left field.
Well, this says what it means.
Once you know what it means, we go back to this and we use this from now on.
OK?
If we have a zero-mean Gaussian process z of t, if you have L2 sample functions, like you had when you take a process and make it up as a linear combination of random variables times orthonormal functions, and if you have a bunch of different L2 functions, g1 up to g sub j 0, then each of these random variables, each of these linear functionals, are going to be Gaussian.
And in fact the whole set of them together are jointly Gaussian.
And we showed that last time.
I'm not going to bother to show it again.
And we also found what the covariance was between these random variables.
And it was just this expression here.
OK. And this follows just by taking vi, which is this kind of integral, times vj, and interchanging the order of integration, taking the expected value of z of t times z of tau, which gives you this quantity.
And you have the g i of t and the g j of t stuck in there with an integral around it.
And it's hard to understand what those integrals mean and whether they really exist or not, but we'll get to that a little later.
We then talked about linear filtering of processes.
You have a process coming into a filter, and the output is some other stochastic process.
Or at least we hope it's a well defined stochastic process.
And we talked a little bit about this last time.
And for every value of tau, namely for each random variable in this output random process, V of tau is just a linear functional.
It's a linear functional corresponding to the particular function h of that tau minus t. So the linear functional is a function of t here for the particular value of tau that you have over here.
So this is just a linear functional like the ones we've been talking about.
OK, if z of t is a zero-mean Gaussian process, then you have a bunch of different linear functionals here for any set of times tau 1 up to tau sub k. And those are jointly Gaussian from what we just said.
And if all of these sets of k sample -- If based a random process, v of tau, is a Gaussian random process, if for all k sets of epochs, v, tau 1, tau 2, up to tau sub k, if this set of random variables are all jointly Gaussian.
So that's what we have here.
So the V of tau is actually a Gaussian process if Z of t is a Gaussian random process to start with.
And the covariance function is just this quantity that we talked about before.
OK, so we have a covariance function.
We also have a covariance function for the process we started with.
And if it's Gaussian, all you need to know is what the covariance function is.
So that's all rather nice.
OK, as we said, we're going to start talking about stationarity today.
I really want to talk about two ideas of stationarity.
One is the idea that you have probably seen as undergraduates one place or another, which is simple computationally, but is almost impossible to understand when you try to say something precise about it.
And the other is something called effectively stationary, which we're going to talk about today.
And I'll show you why that makes sense.
So we say that a process Z of t is stationary if, for all integers k, all shifts tau, all epochs t1 to t sub k, and all of values of z1 to zk, this joint density here is equal to the joint density shifted.
In other words, what we're doing is we're taking a set of times over here, t1, t2, t3, t4, t5.
We're looking at the joint distribution function for those random variables here, and then we're shifting it by tau.
And we're saying if the process is stationery, the joint distribution function here has to be the same as the joint distribution function there.
You might think that all you need is the distribution function here has to be the same as the distribution function there.
But you really want the same relationship to hold through here as there if you want to call process stationary.
OK.
If we have zero-mean Gauss, that's just equivalent to saying that the covariance function at ti and t sub j is the same as the covariance function at ti plus tau and tj plus tau.
And that's true for t1 up to t sub k. Which really just means this has to be true for all tau, for all t sub i and for all t sub j.
So you don't need to worry about the k at all once you're dealing with a Gaussian process.
Because all you need to worry about is the covariance function.
And the covariance function is only a function of two variables, the process at one epoch and the process at another epoch.
And this is equivalent, even more simply, to saying that the covariance function at t1 and t2 has to be equal to the covariance function at t1 minus t2 and zero.
Can you see why that is?
If I start out with this, and I know that this is true, then the thing that I can do is take this and I can shift it by any amount that I want to.
So if I shift this by adding tau here, then what I wind up with is kz of t1 minus t2 plus tau, comma tau, is equal to kz of t1 plus tau, t2 plus tau.
And if I change variables around a little bit, I come to this.
So this is the condition we need for a Gaussian process to be stationary.
I would defy any of you to ever show in any way that a process satisfies all of these conditions.
I mean, if you don't have nice structural properties in the process, like a Gaussian process, which says that all you need to define it is this, this is something that you just can't deal with very well.
So we have this.
And then we say, OK, this covariance is so easy to work with -- I mean no, it's not easy to work with, but it's one hell of a lot easier to work with than that.
And therefore if you want to start talking about processes, and you don't really want to go into all the detail of these joint distribution functions, you will say to yourselves that one thing I might ask for in a process is the question of whether the covariance function satisfies this property.
I mean, for a Gaussian process this is all you need to make the process stationary.
For other processes you need more, but at least it's an interesting question to ask.
Is the process partly stationary, in the sense that the covariance function at least is stationary?
Namely, the covariance function here looks the same as the covariance function there.
So we say that a zero-mean process is wide-sense stationary if it satisfies this condition.
So a Gaussian process then is stationary if and only if it's wide-sense stationary.
And a random process with a mean, or with a mean that isn't necessarily zero, is going to be stationary or wide-sense stationary if the mean is constant and the fluctuation is stationary.
Or wide-sense stationary, as the case may be.
So you want both of these properties there.
So as before, we're just going to throw out the mean and not worry about it.
Because if we're thinking of it as noise, it's not going to have a mean.
Because of it has a mean, it's not part of the noise.
It's just something we know, and we might as well remove it.
OK, interesting example here.
Let's look at a process, V of t, which is defined in this way here.
It's the sum of a set of random variables, V sub k, times the sinc function for some sampling interval, capital T. And I'm assuming that the v sub k are zero-mean, and that, at least as far as second moments are concerned, they have this stationarity property between them.
Namely, they are uncorrelated from one j to another k. Expected value of vj squared is sigma squared.
Expected value of vj vk for k unequal to j is zero.
So they're uncorrelated; they all have the same variance.
Then I claim that the process V of t is wide-sense stationary.
And I claim that this covariance function is going to be sigma squared times sinc of t minus tau over t. Now for how many of you is this an obvious consequence of that?
How many of you would guess this if you had to guess something?
Well, if you wouldn't guess it, you should.
Any time you don't know something, the best thing to do is to make a guess and try to see whether your guess is right or not.
Because otherwise you never know what to do.
So if you have a function which is defined in this way -- think of it this way.
Suppose these V sub k's were not random variables, but instead suppose they were all just constants.
Suppose they were all 1.
Suppose you're looking at the function V of t, which is the sum of all of the sinc functions.
And think of what you'd get when you add up a bunch of sinc functions which are all exactly the same.
What do you get?
I mean, you have to get a constant, right?
You're just interpolating between all these points, and all the points are the same.
So it'd be very bizarre if you didn't get something which was constant.
Well the same thing happens here.
The derivation of this is one of those derivations which is very, very simple and very, very slick and very elegant.
And I was going to go through it in class and I thought, no.
You just can't follow this in real time.
It's something you have to sit down and think about for five minutes.
So I urge you all to read this, because this is a trick that you need to use all the time.
And you should understand it, because various problems as we go along are going to use this idea in various ways.
But anyway, when you have a process defined this way, it has to be wide-sense stationary.
And the covariance function is just this function here.
Now if these V sub k up here are jointly Gaussian, and they all have the same variance, so they're all IID, then what this means is you have a Gaussian random process.
And it's a stationary Gaussian random process.
In other words, it's a process where, if you look at the random variable v of tau at any given tau, you get the same thing for all tau.
In other words, it's a little peculiar, in the sense that when you look at this, it looks like time 0 was a little special.
Because time 0 is where you specified the process.
Time t you specified the process.
Time 2t you specified the process.
But in fact this is saying, no.
Time 0 is not special at all.
Which is why we say that the process is wide-sense stationary.
All times look the same.
On the other hand, if you take these V sub k here to be binary random variables which take one the value plus 1 or minus 1 with equal probability, what do you have then?
You look at the sum here, and at the sample points -- namely at 0, at t, at 2t, and so forth -- what values can the process take away?
At time 0?
If v sub 0 is either plus 1 or minus 1, what is v of 0 over here?
It's plus 1 or minus 1.
It can only be those two things.
If you look at V of capital T over 2, namely if you look halfway between these two sample points, you then ask, what are the different values that v of T over 2 can take on?
It's an awful mess.
It's a very bizarre random variable.
But anyway, it's not a random variable which is plus or minus 1.
Because it's really a sum of an infinite number of terms.
So you're taking an infinite number of binary random variables, each with arbitrary multipliers.
So you're adding them all up.
So you get something that's not differentiable.
It's not anything nice.
I don't care about that.
The only thing that I care about is that v of capital T over 2 is not a binary random variable anymore.
And therefore this process is not stationary anymore.
Here's an example of a wide-sense stationary process which is not stationary.
So it's a nice example of where you have wide-sense stationarity, but you don't have stationarity.
So all kinds of questions about power and things like that, this process works very, very well.
Because questions about power you answer only in terms of covariance functions.
Questions of individual possible values and probability distributions you can't answer the very well.
One more thing.
If these variables are Gaussian, and if you actually believe me that this is a stationary Gaussian process, and it really is a stationary Gaussian process, what we have is a way of creating a broad category of random processes.
Because if I look at the sample functions here of this process, each sample function is bandlimited.
Baseband limited.
And it's baseband limited to 1 over 2 times capital T. 1 over the quantity 2 times capital T. So by choosing capital T to be different things, I can make these sample functions have large bandwidth or small bandwidth so I can look at a large variety of different things.
All of them have Fourier transforms, which look sort of flat in magnitude.
And we'll talk about that as we go on.
And when I pass these things through linear filters, what we're going to find is we can create any old Gaussian random process we want to.
So that's why they're nice.
Yes?
[INAUDIBLE]
Can we make V of t white noise?
In a practical sense, yes.
In a mathematical sense, no.
In a mathematical sense, you can't make anything white noise.
In a mathematical sense, white noise is something which does not exist.
I'm going to get to that later today.
And it's a good question.
But the answer is yes and no.
OK, the trouble with stationary processes is that the sample functions aren't L2.
That's not the serious problem with them, because all of us as engineers are willing to say, well, whether it's L2 or not I don't care.
I'm just going to use it because it looks like a nice random process.
It's not going to burn anything else or anything.
So, so what?
But the serious problem here is that it's very difficult to view stationary processes as approximations to real processes.
I mean, we've already said that you can't have a random process which is running merrily away before the Big Bang.
And you can't have something that's going to keep on running along merrily after we destroy ourselves, which is probably sooner in the future than the Big Bang was in the past.
But anyway, this definition of stationarity does not give us any way to approximate this.
Namely, if a process is stationary, it is either identically zero, or every sample function with probability one has infinite energy in it.
In other words, they keep running on forever.
They keep building up energy as they go.
And as far as the definitions current is concerned, there's no way to say large negative times and large positive times are unimportant.
If you're going to use mathematical things, though you're using them as approximations, you really have to consider what they're approximations to.
So that's why I'm going to develop this idea of effectively wide-sense stationary or effectively stationary.
OK.
So a zero-mean process is effectively stationary or effectively wide-sense stationary, which is what I'm primarily interested in, within two intervals, within some wide interval of time, minus T0 to plus T0.
I'm thinking here of choosing T0 to be enormous.
Namely if you build a piece of equipment, you would have minus T0 to plus T0 include the amount of time that the equipment was running.
So we'll say it's effectively stationary within these limits if the joint probability assignment, or the covariance matrix if we're talking about effectively wide-sense stationary, for t1 up to t sub k is the same as that for t1 plus tau up to tk plus tau.
In other words I have this big interval, and I have a bunch of times here.
t sub k. And I have a bunch of times over here, t1 plus tau up to tk plus tau.
And this set of times don't have to be disjoint from that set of times.
And I have this time over here, minus t0, and I have this time way out here which is plus t0.
And what I'm saying is, I'm going to call this function wide-sense stationery if, when we truncate it to minus t0 to plus t0, it is stationary as far as all those times are concerned.
In other words, we just want to ignore what happens before minus t0 and what happens after plus t0.
You have to be able to do that.
Because if you can't talk about the process in that way, you can't talk about the process at all.
The only thing you can do is view the noise over some finite interval.
OK. And as far as covariance matrix for wide-sense and stationary, well it's the same definition.
So we're going to truncate the process and deal with that.
For effectively stationary or effectively wide-sense stationary, I want to view the process as being truncated to minus t0 to plus t0.
We have this process which might or might not be stationary.
I just truncate it and I say I'm only going to look at this finite segment of it.
I'm going to define this one variable covariance function as kz of t1 and t2.
And kz of t1 and t2 is going to be -- blah.
The single variable covariance function is defined as kz if t1 minus t2 is equal to the actual covariance function evaluated with one argument at t1 and the other argument at t2.
And I want this to hold true for all t1 and all t2 in this interval.
And this square here gives a picture of what we're talking about.
If you look at the square in the notes, unfortunately all the diagonal lines do not appear because of the bizarre characteristics of LaTeX.
LaTeX is better than most programs, but the graphics in it are awful.
But anyway, here it is drawn properly.
And along this line, t minus tau is constant.
So this is the line over which we're insisting that kz of t1 and t2 be constant.
So for a random process to be wide-sense stationary, what were insisting is that within these limits, minus t0 to plus t0, the covariance function is constant along each of these lines along here.
So it doesn't depend on both t1 and t2.
It only depends on the difference between them.
Which is what we require for stationary to start with.
If you don't like the idea of effectively wide-sense stationary, just truncate the process and say, well if it's stationary, this has to be satisfied.
It has to be constant along these lines.
The peculiar thing about this is that the single variable covariance function does not run from minus t0 to plus t0.
It runs from minus 2T0 to plus 2T0.
So that's a little bizarre and a little unpleasant.
And it's unfortunately the way things are.
And it also says that this one variable covariance function is not always the same as the covariance evaluated at t minus tau and 0.
Namely, it's not the same because t minus tau might be considerably larger than t0.
And therefore, this covariance is zero, if we truncated the process, and this covariance is not zero.
In other words, for these points up here, and in fact this point in particular -- well, I guess the best thing to look at is points along here, for example.
Points along here, what we insist on is that t minus tau, k sub z of t and tau is constant along this line.
But we don't insist on kz of t minus tau, which is some quantity bigger than t0 along this line, being the same as this quantity here.
OK, so aside from that little peculiarity, this is the same thing that you would expect from just taking a function and truncating it.
There's nothing else that's peculiar going on there.
OK, so let's see if we can do anything with this idea.
And the first thing we want to do is to say, how about these linear functionals we've been talking about?
Why do I keep talking about linear functionals and filtering?
Because any time you take a noise process and you receive it, you're going to start filtering it.
You're going to take start taking linear functionals of it.
Namely all the processing you do is going to start with finding random variables in this particular way.
And when you look at the covariance between two of them, what you find is the same thing we found last time.
Expected value of Vi times Vj is equal to the integral of gi of t times the covariance function evaluated at t and tau, times gj of tau.
All of this is for real processes and for real functions.
Because noise random processes are in fact real.
I keep trying to alternate between making all of this complex and making it real.
Both have their advantages.
But at least in the present version of the notes, everything is real, concerned with random processes.
So I have this function here.
If gj of t is zero for t greater than T0, what's going to happen here then?
I'm evaluating this where this is a double integral over what region?
It's a double integral over the region where t is constrained to minus T0 to plus T0, and tau is constrained to the interval minus t0 to plus T0.
In other words, I can evaluate this expected value, this covariance, simply by looking at this box here.
If I know what the random process is doing within this box, I can evaluate this thing.
So that if the process is effectively stationary within minus t0 to plus t0, I don't need to know anything else to evaluate all of these linear functionals.
But that's exactly the way that were choosing this quantity, T0.
Namely, we're choosing T0 to be so large that everything we're interested in happens inside of there.
So we're saying that all of these linear functionals for effective stationarity are just defined for what happens inside this interval.
So if Z of t is effectively wide-sense stationary within this interval, you make the intervals large as you want or as small as you want.
The real quantity is that these functions g have to be constrained within minus T0 to plus T0.
Then Vi, Vj, are jointly Gaussian.
In other words, you can talk about jointly Gaussian without talking at all about whether the process is really stationary or not.
You can evaluate this for everything within these limits, strictly in terms of what's going on within those limits.
That one was easy.
The next one is a little harder.
We have our linear filter now.
We have the noise process going into a linear filter.
And we have some kind of process coming out of the filter.
Again, we would like to say we couldn't care less about what this process is doing outside of these humongous limits.
And the way we do that is to say -- well, let's forget about that for the time being.
Let us do what you did as an undergraduate, before you acquired wisdom, and say let's just integrate and not worry at all about changing orders of integration or anything else.
Let's just run along and do everything as carelessly as we want to.
The covariance function for what comes out of this filter is something we already evaluated.
Namely, what you want to look at is v of t and v of tau.
v of t is this integral here.
v of tau is this integral here with tau substituted for t. So when you put both of these together and you take the expected value, and then you bring the expected value in through the integrals -- who knows whether you can do that or not, but we'll do it anyway -- what we're going to get is a double integral of the impulse response, evaluated at t minus t1, times the covariance function evaluated at t1 and t2, times the function that we're dealing with, h, evaluated at tau minus t2.
This is what you got just by taking the expected value of v of t times v of tau.
And you just write it out.
You write out the expected value of these two integrals.
It's what we did last time.
It's what's in the notes if you don't see how to do it.
You take the expected value inside, and this is what you get.
Well now we start playing these games.
And the first game to play is, we would like to use our notion of stationarity.
So in place of kz of t1 and t2, we want to substitute the single variable form, kz tilde of t1 minus t2.
But we don't like the t1 minus t2 in there, so we substitute phi for t1 and t2.
And then what we get is the double integral of kz of phi now.
And we have gotten rid of t1 by this substitution, so we wind up with the integral over phi and over t2.
Next thing we want to do, we want to get rid of the t2 also.
So we're going to let mu equal t2 minus tau.
When we were starting here, we had four variables, t and tau and t1 and t2.
We're not getting rid of the t and we're not getting rid of the tau, because that's what we're trying to calculate.
But we can play around with the t1 and t2 as much as we want, and we can substitute variables of integration here.
So mu is going to be t2 minus tau.
That's going to let us get rid of the t2.
And we wind up with this form here when we do that substitution.
It's just that h of tau minus t2 becomes h of minus mu instead of h of plus mu.
I don't know what that means.
I don't care what it means.
The only thing I'm interested in now is, you look at where the t's and the taus are, and the only place that t's and taus occur are here.
And they occur together.
And the only thing this is a function of is t minus tau.
I don't know what kind of function it is, but the only thing we're dealing with is t minus tau.
I don't know whether these intervals exists or not.
I can't make any very good argument that they can.
But if they do exist, it's a function only of t minus tau.
So aside from the pseudo-mathematics we've been using, V of t is wide-sense stationary.
OK?
Now if you didn't follow all this integration, don't worry about it.
It's all just substituting integrals and integrating away.
And it's something you can do.
Some people just take five minutes to do it, some people 10 minutes to do it.
Now we want to put the effective stationarity in it.
Because again, we don't know what this means.
We can't interpret what it means in terms of stationarity.
So we're going to assume that our filter, h of t, has a finite duration impulse response.
So I'm going to assume that it's zero when the magnitude of t is greater than A.
Again, I'm assuming a filter which starts before zero and runs until after zero.
Because again, what I'm assuming is the receiver timing is different from the transmitter timing.
And I've set the receiver timing at some convenient place.
So my filter starts doing things before anything hits it, just because of this change of time.
So v of t, then, is equal to the integral of the process Z of t1 times h of t minus t1.
This is a linear functional again.
This depends only on what Z of t1 is for t minus A less than or equal to t1, less than or equal to t plus A.
Because I'm doing this integration here, this function here is 0 outside of that interval.
I'm assuming that h of t is equal to 0 outside of this finite interval.
And this is just a shift of t1 on it.
So this integral is going to be equal to zero from when t is equal to t1 minus A to when t is equal to t1 plus A.
Every place else, this is equal to zero.
So V of t, the whole process depends -- and if we only want to evaluate it within minus T0 plus A to T0 minus A -- in other words we look at 10 to the eighth years and we subtract off a microsecond from that region.
So now we're saying we want to see whether this is wide-sense stationary within 10 to the eighth years minus a microsecond, minus that to plus that.
So that's what we're dealing with here.
V of t in this region depends only on Z of t for t in the interval minus T0 to plus T0.
In other words, when you're calculating V of t for any time in this interval, you're only interested in Z of t, which is diddling within plus or minus A of that.
Because that's the only place where the filter is doing anything.
So V of t depends only on Z of t. And here, I'm going to assume that Z of t is wide-sense stationary within those limits.
And therefore I don't have to worry about what Z of t is doing outside of those limits.
So if the sample functions of Z of t are L2 within minus T0 to plus T0, then the sample functions of V of t are L2 within minus T0 plus A to T0 minus A.
Now this is not obvious.
And there's a proof in the notes of this, which goes through all this L1 stuff and L2 stuff that you've been struggling with.
It's not a difficult proof, but if you don't dig L2 theory, be my guest and don't worry about it.
If you do want to really understand exactly what's going on here, be my guest and do worry about it, and go through it.
But it's not really an awful argument.
But this is what happens.
So what this says is we can now view wide-sense stationarity and stationarity, at least for Gaussian processes, as a limit of effectively wide-sense stationary processes.
In other words, so long as I deal with filters whose impulse response is constrained in time, the only effect of that filtering is to reduce the interval over which the process is wide-sense stationary by the quantity A.
And the quantity A is going to be very much smaller than T0, because we're going to choose T0 to be as large as we want it to be.
Namely, that's always the thing that we do when we try to go through limiting arguments.
So what we're saying here is, by using effective stationarity, we have managed to find a tool that lets us look at stationary as a limit of effectively stationary processes as you let T0 become larger and larger.
And we have our cake and we eat it too, here.
Because at this point, all these functions we're dealing with, we can assume that they're L2.
Because they're L2 for every T0 that we want to look at.
So we don't have to worry about that anymore.
So we have these processes.
Now we will just call them stationary or wide-sense stationary.
Because we know that what we're really talking about is a limit as T0 goes to infinity of things that make perfect sense.
So suddenly the mystery, or the pseudo-mystery has been taken out of this, I hope.
So you have a wide-sense stationary process with a covariance now of k sub z of tau.
In other words, it's effectively stationary over some very large interval, and I now have this covariance function, which is a function of one variable.
I want to define the spectral density of this process as a Fourier transform of k sub z.
In other words, as soon as I get a one variable covariance function, I can talk about its Fourier transform.
At least assuming that it's L2 or something.
And let's forget about that for the time being, because that all works.
We want to take the Fourier transform of that.
I'm going to call that spectral density.
Now if I call that spectral density, the thing you ought to want to know is, why am I calling that spectral density?
What does it have to do with anything you might be interested in?
Well, we look at it and we say well, what might it have to do with anything?
Well, the first thing we're going to try is to say, what happens to these linear functionals we've been talking about?
We have a linear functional, V. It's an integral of g of t times Z of t dt.
We would like to be able to talk about the expected value of V squared.
Talk about zero-mean things, so we don't care about the mean at zero.
So we'll just talk about the variance of any linear functional of this form.
If I can talk about the variance of any linear functional for any g of t that I'm interested in, I can certainly say a lot about what this process is doing.
So I want to find that variance.
If I write this out, it's same thing we've been writing out all along.
It's the integral of g of t times this one variable covariance function now, times g of tau d tau dt.
When you look at this, and you say OK, what is this?
It's an integral of g of t times some function of t. And that function of t is really the convolution of k sub z.
This is a function now.
It's not a random process anymore; it's just a function.
That's a convolution of the function k tilde with g. So I can rewrite this as the integral of g of t times this convolution of t. And then I'm going to call the convolution theta of t, just to give it a name.
So this variance is equal to g of t times this function theta of t, which is just the convolution of k with g. Well now I say, OK, I can use I can use Parseval's relation on that.
And what I get is this integral, if I look in the frequency domain now, as the integral of the Fourier transform of g of t, times the complex conjugate of theta star of f times df.
I've cheated it you just a little bit there because what I really want to talk about is -- I mean, Parseval's theorem relates this quantity to this quantity, where theta is theta complex conjugate.
But fortunately, theta is real.
Because g is real and k is real.
Because we're only dealing with real processes here, and we're only dealing with real filters.
So this is real, this is real.
So this integral is equal to this in the frequency domain.
And now, what is the Fourier transform of -- OK, theta of t is this convolution.
When I take the Fourier transform of theta, what I get is the product of the Fourier transform of k times the Fourier transform of g. So in fact what I get is g hat of f times theta star of f. Which is g complex conjugate of that times k complex conjugate -- times -- it doesn't make any difference whether it's a complex conjugate or not, because it's real.
So I wind up with the integral of the magnitude squared of g times this spectral density.
Well at this point, we can interpret all sorts of things from this.
And we now know that we can interpret this also in terms of effectively stationary things.
So we don't have any problem with things going to infinity or anything.
I mean, you can go through this argument carefully for effectively stationary things, and everything works out fine.
So long as g of t is constrained in time.
Well now, I can choose this function any way that I want to.
In other words, I can choose my function g of t in any way I want to.
I can choose this function in any way I want to.
So long as its inverse transform is real.
Which means g hat of f has to be equal to g hat of minus f complex conjugate.
But aside from that, I can choose this to be anything.
And if I choose this to be very, very narrow band, what is this doing?
It's saying that if I take a very narrow band g of t, like a sinusoid truncated out to some very wide region, I multiply that by this process, I integrate it out, and I look at the variance.
What am I doing when I'm doing that?
I'm effectively filtering my process with a very, very narrow band filter, and I'm asking, what's the variance of the output?
So the variance of the output really is related to the amount of energy in this process at that frequency.
That's exactly what the mathematics says here.
It says that s sub z of f is in fact the amount of noise power per unit bandwidth at this frequency.
It's the only way you can interpret it.
Because I can make this as narrow as I want to, so that if there is any interpretation of something in this process as being at one frequency rather than another frequency and the only way I can interpret that is by filtering the process.
This is saying that when you filter the process, what you see at that bandwidth is how much power there is in the process at that bandwidth.
So this is simply giving us an interpretation of spectral density as power per unit bandwidth.
OK, let's go on with this.
If this spectral density is constant over all frequencies of interest, we say that it's white.
Here's the answer to your question: what is white Gaussian noise?
White Gaussian noise is a Gaussian process which has a constant spectral density over the frequencies that we're interested in.
So we now have this looking at it in a certain band of frequencies.
You know, if you think about this in more or less practical terms, suppose you're building a wireless network, and it's going to operate, say, at five or six gigahertz.
And suppose your bandwidth is maybe 100 megahertz or something.
Or maybe it's 10 megahertz, or maybe it's 1 megahertz.
In terms of this frequency of many gigahertz, you're talking about very narrowband communication.
People might call it wideband communication, but it's really pretty narrowband.
Now suppose you have Gaussian noise, which is caused by all of these small tiny noise affects all over the place.
And you ask, is this going to be flat or is it not going to be flat?
Well, you might look at it and say, well, if there's no noise in this little band and there's a lot of noise in this band, I'm going to use this little band here.
But after you get all done playing those games, what you're saying is, well, this noise is sort of uniform over this band.
And therefore if I'm only transmitting in that band, if I never transmit anything outside of that band, there's no way you can tell what the noise is outside of that band.
You all know that the noise you experience if you're dealing with a carrier frequency in the kilohertz band is very different than what you see in the megahertz band, is very different from what you see at 100 megahertz, is very different from what you see in the gigahertz band, is very different from what you see in the optical bands.
And all of that stuff.
So it doesn't make any sense to model the noise as being uniform spectral density over all of that region.
So the only thing we would ever want to do is to model the noise as having a uniform density over this narrowband that we're interested in.
And that's how we define white Gaussian noise.
We say the noise is white Gaussian noise if, in fact, when we go out and measure it we can measure it by passing it through a filter by finding this variance, which is exactly what we're talking about here, if what we get at one frequency is the same as what we get at another frequency.
So long as we're talking about the frequencies of interest, then we say it's white.
Now how long do we have to make this measurement?
Well, we don't make it forever, because all the filters we're using and everything else are filters which only have a duration over a certain amount of time.
We only use our device over a certain amount of time.
So what we're interested in is looking at the noise over some large effective time from minus T0 to plus T0.
We want the noise to be effectively stationary within minus T0 to plus T0.
And then what's the next step in the argument?
You want the noise to be effectively stationary between these very broad limits.
And then we think about it for a little bit, and we say, but listen.
I'm only using this thing in a very small fraction of that time region.
And therefore as far as my model is concerned, I shouldn't be bothered with T0 at all.
I should just say mathematically, this process is going to be stationary, and I forget about the T0.
And I look at it in frequency, and I say I'm going to use this over my 10 megahertz or 100 megahertz, or whatever frequency band I'm interested in.
And I'm only interested in what the noise is in that band.
I don't want to specify what the bandwidth is.
And therefore I say, I will just model it as being uniform over all frequencies.
So what white noise is, is you have effectively gotten rid of the T0.
You've effectively gotten rid of the w0.
And after you've gotten rid of both of these things, you have noise which has constant spectral density over all frequencies, and noise way which has constant power over all time.
And you look at it, and what happens?
If you take the inverse Fourier transform of sz of f, and you assume that sz of f is just non-zero within a certain frequency band, what you get when you take the Fourier transform is a little kind of wiggle around zero.
And that's all very interesting.
If you then say well, I don't care about that.
I just like to assume that it's uniform over all frequencies.
You then take the Fourier transform, and what you've got is an impulse function.
And what the impulse function tells you is, when you pass the noise through a filter, what comes out of the filter, just like any time you deal with impulse functions -- I mean, the impulse response is in fact the response to an impulse.
So that as far as the output from the filter goes, it's all very fine.
It only cares about what the integral is of that pulse.
And the integral of the pulse doesn't depend very much on what happens at enormously large frequencies or anything.
So all of that's well behaved, so long as you go through some kind of filtering first.
Unfortunately, as soon as you start talking about a covariance function which is an impulse, you're in real trouble.
Because the covariance function evaluated at zero is the power in the process.
And the power in the process is then infinite.
So you wind up with this process which is easy to work with any time you filter it.
It's easy to work with because you don't have these constants capital T0 and capital W0 stuck in them.
Which you don't really care about, because what you're assuming is you can wander around as much as you want in frequency, subject to the antennas and so on that you have.
And you want to be able to wander around as much in time as you want to and assume that things are uniform over all that region.
But you then have this problem that you have a noise process which just doesn't make any sense at all.
Because it's infinite everywhere.
You look at any little frequency band of it and it has infinite energy if you integrate over all time.
So you really want to somehow use these ideas of being effectively stationary and of being effectively bandlimited, and say what I want is noise which is flat over those regions.
Now what we're going to do after the quiz, which is on Wednesday, is we're going to start talking about how you actually detect signals in the presence of noise.
And what we're going to find out is, when the noise is white in the sense -- namely, when it behaves the same over all the degrees of freedom that we're looking at -- then it doesn't matter where you put your signal.
You can put your signal anywhere in this huge time space, in this huge bandwidth that we're talking about.
And we somehow want to find out how to detect signals there.
And we find out that the detection process is independent of what time we're looking at and what frequency we're looking at.
So what we have to focus on is this relatively small interval of time and relatively small interval of bandwidth.
So all of this works well.
Which is why we assume white Gaussian noise.
But the white Gaussian noise assumption really makes sense when you're looking at what the noise looks like in these various degrees of freedom.
Namely, what the noise looks like when you pass the noise through a filter and look at the output at specific instance of time.
And that's where the modeling assumption is come in.
So this is really a very sophisticated use of modeling.
Did the engineers who created this sense of modeling have any idea of what they were doing?
No.
They didn't have the foggiest idea of what they were doing, except they had common sense.
And they had enough common sense to realize that no matter where they put their signals, this same noise was going to be affecting them.
And because of that, what they did is they created some kind of pseudo-theory, which said we have noise which looks the same wherever it is.
Mathematicians got a hold of it, went through all this theory of generalized functions, came back to the engineers.
The engineers couldn't understand any of that, and it's been going back and forth forever.
Where we are now, I think, is we have a theory where we can actually look at finite time intervals, finite frequency intervals, see what's going on there, make mathematical sense out of it, and then say that the results don't depend on what t0 is or w0 is, so we can leave it out.
And at that point we really have our cake and we can eat it too.
So we can do what the engineers I've always been doing, but we really understand it at this point.
OK.
I think I will stop at that point.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Review what random processes are a little bit now.
And remember, in what we're doing here, we have pointed out that random processes in general, have a mean.
The mean is not important.
The mean is not important for random variables or random vectors either.
It's just something you add on when you're all done.
So the best way to study random variables, random vectors, and random processes, particularly when you're dealing with things which are based on base things being Gaussian, is to forget about the means, do everything you want to with the fluctuations, and then put the means in when you're all done.
Which is why the notes do most of what they do, in terms of zero mean, random variables, random vectors, and random processes, and simply put in what the value of the mean is at the end.
OK so some of this has the mean put in.
Some of it doesn't.
So to start out with, a random process is defined by its joint distribution at each finite set of epochs.
OK, that's where we started with all of this.
How do you define a random waveform?
I mean, you really have to come to grips with that question before you can see how you might answer the question.
If you think it's obvious how to define a random process, then you really have to go back and think about it.
Because a random process is a random waveform.
It's an uncountably infinite number of random variables.
So that defining what it is, is not a trivial problem.
And people have generally agreed that the way to define it, be the test for whether you have defined it or not, is whether you can find the joint distribution at each finite set of epochs.
Fortunately most processes of interest can be defined in a much, much simpler way in terms of an orthonormal expansion.
When you to find it in terms of an orthonormal expansion, you have the orthonormal expansion which is the sum over k, of a set of random variables, Z sub 1, Z sub 2 and so forth Z sub k, times a set of orthonormal functions.
So all the variation on t is stuck into the set of orthonormal functions.
All the randomness is stuck into the sequence of random variables.
So at this point, you have a sequence of random variables, rather than a waveform.
Why is it so important to have something countable instead of something uncountable?
I mean, mathematicians understand, in a deep mathematical sense, why that's so important.
For engineers the argument is a little bit different.
I think for engineers the argument is, no matter how small the interval of a waveform you're looking at, even if you shrink it, no matter what you do with it, you can't approximate it in any way and get rid of that uncountably infinite number of random variables.
As soon as you represent it in terms of an orthonormal expansion though, at that point you have represented it as a countable sum.
As soon as you represent it as a countable sum, you can approximate it by knocking off the tail of that sum.
OK, and at that point you have a finite number of random variables, instead of an infinite number of random variables.
We all know how to deal with a finite set of random variables.
You've been doing that since you were in 6.041 OK.
So if you have a countable set of random variables, the way to deal with them is always the same.
It's to hope that they're defined in such a way that when you look at enough of them, all the rest of them are unimportant.
That sort of is the underlying meaning of what countable means.
You can arrange them in a sequence, yes.
But like when you count in school, after you name the first hundred numbers you get tired of it, and you realize that you understand the whole thing at that point.
OK in other words, you don't have to count up to infinity to understand what it means to have a countable set of integers.
You all understand the integers.
You understand them intuitively.
And you understand that no matter how big an integer you choose, somebody else can always choose one bigger than you've chosen.
But you also understand that that's not important.
OK?
So this is the way we usually define a stochastic process, a random process to start off with.
The other thing that we have to do, is given a random process like this, we very often want to go from this random process to other random processes which are defined in terms of this random process.
So almost always we start out with a random process which is defined this way.
And then we move from there to various other processes which we define in terms of this.
So we never really have to deal with this uncountably infinite number of random variables.
If we really had to deal with that, we would be in deep trouble.
Because the only way we could deal with it, would be to take five courses in measure theory.
And after you take five courses in measure theory, it's not enough.
Because at that point, you're living in measure theory.
And most of the mathematicians that I know can't come back from measure theory to talk about real problems.
So that every time they write a paper, which looks like beautiful mathematics, it's very difficult to interpret whether it means anything about real problems.
Because that's where things get hard.
OK so the point is, if you define random processes in this way, then you always have some sort of grounding about what you're talking about.
Because you have a countable number of random variables here.
You know that if you're dealing with anything that makes any sense, only a finite number of them are going to be important.
The thing you don't know is how many of them you need to be important.
OK, so that's why we take an infinite sum here.
Because you don't know ahead of time, how many you'd need to deal with.
OK, so we then started to talk about stationarity.
The process Z of t is stationary if Z of t sub 1 up to Z of t sub k and Z of t sub 1 plus tau up to Z of t sub k plus tau have the same distribution.
OK, so you take any finite set of random variables in this process, you shift them all to some other place, and you ask whether they have the same distribution.
How do you answer that question?
God only knows.
I mean, what you need to be able to answer that question is some easy way of finding those joint probabilities.
And that's why you deal with examples.
That's why we deal with jointly Gaussian random variables.
Because once we're dealing with jointly Gaussian random variables, we can write down those joint distributions just in terms of covariance matrices.
Well it means, of course, if you want to deal with things that are not a zero mean.
And after you learn how to deal with covariance matrices, at least you're dealing with a function of a finite number of variables.
So it's not so bad.
OK so the argument is, this is what you need to know if you're going to call it stationary.
But we have easier ways of testing for that.
And in fact this Wide Sense Stationary we said, if the covariance matrix, mainly the expected value of Z of t sub 1 times the expected value of Z of t sub 2 is equal to some function of just t sub 1 minus t sub 2.
In other words, where the expected value of this value times this value is the same, if you shift it over by some amount.
OK, you see the difference between?
I mean the difference between these two things is one, that in the definition for stationarity, you need an arbitrarily large set of random variables here.
You shift them over to some other point here.
And you need the same joint distribution over this arbitrarily large set of random variables.
When you get into dealing with the covariance matrix, all you need for Wide Sense Stationarity is you only have to deal with two random variables here, and the shift of those two random variables here.
Which really comes out to a problem involving just two random variables t sub 1, t sub 2, and the fact that it's the same no matter where you shift the t sub 1 to.
It's only a function of the difference between t sub 1 and t sub 2.
So it's a whole lot easier.
And then we pointed out that since jointly Gaussian random variables, and since the Gaussian random process, it's completely determined by its covariance function, a zero mean Gaussian process.
If you'll forgive me, I'm not going to keep coming back and saying that.
I will just be thinking solely today in terms of zero mean processes.
And you should think solely in terms of zero mean processes.
You can sort out for yourselves whether you'd need a mean or not.
That's sort of a trivial addition to all of this.
OK, so it's Wide Sense Stationary.
Well we already said that.
Well, here I put in the mean.
OK, Wide Sense Stationary implies that it's stationary for Gaussian processes.
OK, so in other words, this messy looking question of asking whether a process is stationary or not, with all of these random variables here and all of these random variables here, becomes trivialized for the case of a Gaussian random process, where all you need to worry about is the covariance function.
Because that's the thing that specifies the whole process.
OK. That's part of what we did last time.
We set an important example of this.
And you don't quite know how important this is yet.
But this is really important because all of the Gaussian processes you want to talk about can be formed in terms of this process.
So it's nice in that way.
And the process is Z of t is the sum of an orthonormal expansion.
But the orthonormal functions now are just the time shifted sinc functions.
We know the time shifted sinc functions are orthogonal.
So we just have this set of random variables.
And we say OK, what we're going to assume here is that these random variables, well here I put in the mean.
Let's forget about the mean.
The expected value of V sub k times V sub i, namely any two of these random variables expected value of the pair, is equal to zero if K is unequal to i. OK, in other words, the random variables are in some sense orthogonal to each other.
But let's save the word orthogonal for functions, and use the word correlated or uncorrelated here for random variables.
So these random variables are uncorrelated.
So we're dealing with an expansion where the functions are orthogonal, and where the random variables are uncorrelated.
And now, what we're really interested in, is making these random variables Gaussian.
And then we have a Gaussian random process with this whole sum here.
We're interested in making these have the same variance in all cases.
And therefore what we have is process which is stationary.
And its covariance function is just sigma squared times this sinc function here.
There's an error in the notes, in lecture 16, about this.
It will be corrected on the web.
This is in the notes.
This quantity here is left out.
So you can put that back in if you want to.
Or you can get the new notes off the web after a few hours or so.
This is not put on the web yet.
I just noticed yesterday.
In fact I noticed it this morning.
OK, the sample functions of a Wide Sense Stationary non-zero process are not L sub 2.
When I talk about a non-zero process I'm not talking about zero mean, I'm saying this peculiar random process which is zero everywhere.
In other words, a random process which really doesn't deserve to be called a random process.
But unfortunately by the definitions, it is a random process.
Because with probability one, Z of t is equal zero everywhere.
And that just means you have a constant, which is zero everywhere.
So you're not interested in it.
OK, if you take the sample functions of any non-trivial Wide Sense Stationary random process, they dribble on forever.
And they have the same-- no matter how far you go out in time-- the V sub k's which are the samples of this process, sample random variables of the process, weigh out.
These things all have the same variance.
So that this is stationary because it just keeps going on forever.
OK and as we said last time, that runs into violent conflict with our whole idea here of trying to understand what L sub 2 functions are all about.
Because what we would like to be able to do is stick the L sub 2 functions as sample values of these processes, have the waveforms that we send the L sub 2.
When you add up two L sub 2 functions, you get an L sub 2 function.
And therefore, all the sample values of all the things that happen with probability one are all L sub 2 functions.
As soon as you do the most natural and simple thing with random processes, you wind up with something which cannot be L sub 2 anymore.
But it's not L sub 2 in only a trivial way.
OK in other words, it's not L sub 2 because it keeps going forever.
It's not L sub 2 for the same reason that sine x is not L sub 2, or 1 is not L sub 2.
OK, and we've decided not to look at those functions as reasonable functions for the waveforms that we transmit or receive.
But for random processes, maybe they're not so bad.
OK.
But we don't know yet.
We started to talk about something called effectively Wide Sense Stationary, which the notes talk about quite a bit.
Which to say, OK we will assume that the process is stationary over some very wide time interval.
We don't know how wide that time interval is, but we'll assume that it's finite.
In other words, we're going to take this random process, whatever it is, you can define it, a nice random process this way.
And then we're going to get our scissors, and we're going to cut it off over here and we're going to cut it off over here.
So it's then, time-limited over this very broad band.
Why do we want to do this?
Because again, it's like the integers which are countable.
You don't know how far out you have to go before you're not interested in something anymore.
But you know if you go out far enough, you can't be interested in it anymore.
Because otherwise you can't take limits.
You can't do anything interesting with sequences.
OK, so here the idea is.
If you go out far enough, you don't care.
This has to be the way that you model things because any piece of equipment that you ever design is going to start being used at a certain time, and it's going to stop being used at a certain time.
And about halfway after when it stops being used, the people who make it will stop supporting it.
And that's to hurry on the time at which you have to buy something new.
I bought a new computer about six months ago.
And I find that Microsoft is not supporting any of the stuff that it loaded into this damn computer anymore.
Six months old, so you see why I'm angry about this matter, of things not being supported.
So in a sense they're not stationary after that.
So you can only ask for things to be stationary over a period of six months or so.
OK.
But for electronic times, when you're sending data at kilobits per second or megabits per second, or gigabits per second as people like to do now, that's an awful long time.
You can send an awful lot of bits in that time.
So stationary means stationary for a long time.
The notes do a lot of analysis of this.
I'm going to do a little of that analysis here in class again.
Not because it's so important to understand the details of it, but to understand what it really means to have a process be stationary, and to understand that doesn't really destroy any of the L sub 2 theory that we built up.
OK.
So the covariance function is L sub 2 in cases of physical relevance.
That's the thing.
That's one of the things we need.
We want the sample functions also to be L sub 2 in cases of physical relevance.
If you have a function here, this is just a function of time now at this point.
There's nothing random about this.
It's just a statistic of this random process.
But it's a nice well-defined function.
We're talking about real random processes here.
So what can you say about this function?
Is it symmetric in time?
How many people think it must be symmetric?
I see a number of people.
How many people think it's not symmetric?
Well it is symmetric.
It has to be symmetric because its expected value of Z of t sub 1 times Z of t sub 2, and if you flip the roles of those two, you have to get the same answer.
So it is symmetric.
It is real.
We're going to talk about the Fourier transform of this before trying to give any relevance or physical meaning to this thing called spectral density.
We'll just say this is the Fourier transform of the covariance function for a Wide Sense Stationary process.
So we have some kind of Fourier transform.
We'll assume this is L sub 2 and everything.
We won't worry about any of that.
So there is some function here which makes sense.
Now if k is both real and symmetric, what can you say about its Fourier transform?
If you have a real function, what property does this Fourier transform have?
[UNINTELLIGIBLE]
It's conjugate symmetric, yes.
And furthermore, if k is symmetric itself, then you can go back from this thinking you have a symmetric transform, and realize that this has to be real.
OK.
So spectral densities of real processes are always both real and symmetric.
We will define what we mean by spectral density later for complex processes.
And spectral densities are always real.
They aren't always symmetric if you have a complex process.
But don't worry about that now.
There are enough peculiarities that come in when we start dealing with complex processes, that I don't want to worry about it at all.
So spectral density is real and symmetric.
And so far it's just a definition.
OK, but the thing we found about this sinc process, is that the sinc process in fact, is a stationary process.
It's a Wide Sense Stationary process for whatever variables you want to put in here.
And if these variables are IID and they're Gaussian and zero mean, then this process is a zero mean, stationary Gaussian random process.
And it has a spectral density.
This turns out to be a sinc function.
The Fourier transform of it is a rectangular function.
So this process has a spectral density which is constant out to a certain point, and then drops off to zero.
So it's nice in that sense.
Because you can make this be flat as far as you want to, and then chop it off wherever you want to.
And we'll see that when we start putting a process like this through filters, very nice things happen.
OK.
So we're familiar, relatively familiar, and conversant with at least one Gaussian random process.
OK, we talked about linear functionals last time before the quiz.
And we said a linear functional is a random variable, V, which is simply this integral here.
We've talked about this integral a lot where Z of t is a function, and g of t is a function.
If the sample values of Z of t are L sub 2, then this integral is very well-defined.
It turns out, also, that if g of t is L sub 2, and these sample values are bounded, then all of this works out very nicely also.
And we'll find other ways to look at this as we move on.
OK but what this means is that for all sample points in this entire sample space, in other words we're dealing with a probability space where we have a bunch of processes running around, we have data coming in.
We're transmitting the data.
We have data that gets received.
We're doing crazy things with it.
All of this stuff is random.
We might have other processes doing something else.
Big complicated sample space here for all of the elements in it, the sample value of the random variable, V, is just the integral of the sample value of the process here.
In other words for each omega, this process is simply a waveform.
So it's this waveform times the function g of t, which we think of like that.
So if g of t is time limited in L sub 2, if g of t is time limited as far as these sample functions are concerned, we don't give a fig about what Z of t is outside of that interval.
And if we're going to only deal with linear operations on this process where those linear operations are constrained to some interval, then we don't care what the process does outside of that interval.
And since we don't care what the process does outside of that interval, we can simply define the process in terms of what it's doing from some large negative time to some large positive time.
And that's all we care about.
And so we wanted to find effective stationarity, as something which obeys the law of stationarity over that interval.
And we don't care what it does outside of that interval, OK?
In other words, if Microsoft stops supporting things after six months, we don't care about it.
Because everything we're going to do we're going to get done within six months.
And if we don't get it done within six months we're going to have a terrible crash.
And we're going to throw everything away and start over again.
So it doesn't make any difference at that point.
OK, so we have a process now we'll assume is effectively stationary within two time limits.
And our definition is if the covariance function, which is a function of two values, t and tau, namely it's the expected value of Z of t times expected value of Z of tau.
And our definition of effective stationarity is that this is equal to a function of t minus tau, whenever t and tau are in this box.
OK, and we drew figures last time for what that meant.
We drew this box, you know, bounded by T sub 0 over 2.
And what this effective stationarity means, is that on all these diagonal lines, the covariance function is constant on those diagonal lines.
And that gives us the same effect whenever we're passing this process any time we're calculating the inner product of a sample value of the process with some function, which is contained within those limits.
All we need to know about is just what the process is doing within minus T sub 0 over 2 to plus T sub 0 over 2.
Nothing else matters.
And therefore, something is effectively stationary within those limits means that we get the same answer here, whether or not it's stationary.
Why is that important?
It's important because the things that you can do, the simple things that you can do with stationary processes are so simple that you'd like to remember them, and not remember all these formulas with capital T sub 0's floating around in them.
Because having a capital T sub 0 in it is just as crazy as assuming that it's stationary, because you never know how long it's going to be until Microsoft stops supporting its software.
I mean, if you knew you'd never buy their products, right?
So you can't possibly know.
So you assume it's going to go on forever.
But then, when we try to ask what does that mean, we say what it really means is over some finite limits, which are large compared to anything we're going to deal with that at all of these results hold.
So eventually we're trying to get to the point where we can leave the T sub 0 out of it.
But we're going through this so we can say, OK there's nothing magical that happens in the limit as T sub 0 goes to infinite.
Mainly we're trying to really derive the fact that all these results can be established over a finite interval of time.
And therefore you don't care.
OK.
So if this covariance function, single variable covariance function, is less than infinity, what does that mean?
What is a single variance, single value, covariance function of zero mean?
It's the expected value of Z of T, times Z of T. In other words, it's the variance of the process at T for any T within minus T sub 0 to plus T sub 0, or T sub 0 over 2 to plus T sub 0 over 2.
OK?
So if that variance is finite, then you can just integrate over the sample values of the process, and you get something finite.
In other words this is what you need for the sample functions to be L sub 2 with probability one.
As soon as you have this, then you're in business with all of your L sub 2 theory.
And what does your L sub 2 theory say?
It really says you can ignore L sub 2 theory.
OK, that was the nice thing about it.
That was why we did it.
OK in other words, the whole thing we're doing in this course is we're going through complicated things sometimes so that you can know how far you can go with the simple things.
And we always end up with the simple things when we're all done.
OK, and that's the whole principle.
I mean we could just do the simple things like most courses do.
Except then, you would never know when it applies and when it doesn't apply.
So, OK.
So there we are.
Let's talk about linear filtering of processes.
The notes don't do quite enough of this.
So when I get to the place where you need it I will-- and some of it's on this slide-- I'll tell you.
We're taking a random process and we're passing the sample waveform of that random process through a linear filter, a linear time and variant filter, so some other random process comes out.
OK, and now the output at some time tau, is just the convolution of the input Z of t times this filter response h of tau minus t. And you can interpret this in terms of these sample functions the way we did before.
But now we already sort of understand that.
So we're just interpreting in terms of if there's a random variable V, which is the value of the process at time at epoch tau, which is given in this way.
It's a random variable when you pass the process through the filter.
OK, if Z of t is effectively Wide Sense Stationary with L sub 2 sample functions, and if h of t is non-zero only within finite limits, minus A to plus A, what's going to happen?
When you pass the sample waveforms through a linear filter, which is bounded between minus A and plus A, that linear filter cannot do anything with the inputs.
Namely the output that comes out of there at some time, T, can't depend on the input anymore than A away from that output.
OK in other words, let me draw a picture of that.
Here's some time where we're observing V of t. And V of t can depend on Z of t only in this region here, Z of t minus A up to Z of t plus A. OK that's what the convolution says.
That's what those filter equations say.
It says that this output, here at this time, depends on the input only over these finite limits.
And again remember we don't care about realizability here.
Because the timing at the receiver is always different from the timing at the transmitter.
OK so what that says then is, what does it say?
It says that we know that Z of t is Wide Sense Stationary over these big limits minus T sub 0 to t sub 0 from six months ago until six months from now.
And this linear filter is non-zero only over one millisecond.
It says that the process which comes out, is going to be Wide Sense Stationary over minus six months plus one millisecond to six months minus one millisecond.
And that's just common sense.
Because the filter isn't moving the process any more than that little bit.
OK.
So V of t then, is going to be Wide Sense Stationary in L sub 2 within those slightly smaller limits.
The covariance function is going to be the same thing that it was before if you had a completely stationary process, if you only worry about what's going on within those limits, T sub 0 minus A to T sub zero plus A.
You can take that big mess there, and view it more simply as a convolution.
Let's see, this part of it is a convolution of h of t with K tilde, which is the covariance, which is a function of one variable.
And then it's convolved with-- and here I put in the complex conjugate of h, because it's easier to do it here.
Because at some point, we want to start dealing with complex random processes.
And for filters it's easy to do that.
For linear functionals it's a little harder.
So I want to stick to real processes.
If we're dealing with filtering, I can simply define this covariance function as the expected value of Z of t times Z complex conjugate of tau.
And when you do that, we're taking this integral, which is the convolution of h of t with K tilde.
And then we're convolving it with h complex conjugate of minus t. Because the t gets turned around in there.
And what that says when you take the Fourier transform of it, at least what I hope it says-- I'm not very good at taking Fourier transforms.
And you people, some of you are very good at it.
We get the spectral density of the process, Z of t times the magnitude of h hat of f. Now this is the formula, which is not in the notes I just found out.
And it's a very important formula.
So you ought to write it down.
It says what the spectral density is on the output of the filter, if you know the spectral density of the input of the filter.
And it's a kind of a neat, simple formula for it.
It says, you pass a random process.
I mean, it's like the formula that you have when you take a waveform and you pass it through a linear filter.
And you know it's kind of easier to look at that in the frequency domain where you multiply, than it is to look at it in a time domain where you have to convolve.
Here things become even simpler because all we have to do is take the spectral density, which is now a function of frequency, multiply it by this magnitude squared, and suddenly we get the spectral density at the output.
Now what happens when you take this nice sinc filter, this sinc Gaussian process that we have, which is flat over as large a bandwidth as you want to make it.
And you pass that process through a linear filter.
What do you get out?
You get a process out which has any spectral density that you want within those wide limits.
OK, so just by understanding the sinc Gaussian process and by understanding this result, you can create a process which has any old spectral density that you want.
And if you can create any old spectral density that you want, you know you can create any old covariance function that you want.
And all of this is good for Wide Sense Stationarity so long as your filter doesn't extend for too far.
Which says all of our theory works starting at some negative time going to some positive time, taking filters that only exist for a relatively small time.
If it's Wide Sense Stationary and it's Gaussian, then everything works beautifully.
You can simply create any old spectral density that you want.
If you have some waveform that's coming in, you can filter it and make the output process look whatever you want to make it look like, so long as you have the stationarity property.
Now you know the secret of why everybody deals with stationary processes.
It's because of this formula.
It's an incredibly simple formula.
It's an incredibly simple idea.
You see now we know something else.
We know that it also applies over finite, but large time intervals.
OK.
So just to spell out the conclusions once more, a Wide Sense Stationary process is Wide Sense Stationary after filtering.
So if you start out with Wide Sense Stationary, it's Wide Sense Stationary when you get done.
If it's effectively Wide Sense Stationary, it's effectively stationary with a reduced interval of effective stationarity after you filter.
OK in other words, if you start out with a process which is effectively stationary from minus T sub 0 to plus T sub 0, then after you filter it with any filter whose impulse response is limited in time, you get something which is effectively stationary within that six months minus one millisecond period of time.
Yes?
[UNINTELLIGIBLE]
What?
Don't you differentiate [UNINTELLIGIBLE]
Effectively Wide Sense Stationary, thank you.
I try to differentiate between them, but sometimes I'm thinking about Gaussian processes, where it doesn't make any difference.
Is effectively Wide Sense Stationary with reduced interval after filtering.
OK, good.
The internal covariance, in other words, the covariance function within these intervals, the spectral density, and the joint probability density aren't affected by this interval of effective stationarity.
In other words, so long as you stay in this region of interest, you don't care what T sub 0 is.
And since you don't care what T sub 0 is, you forget about it.
And for now on we will forget about it.
But we know that whatever we're dealing with, it's effectively stationary within some time period, which is what makes the functions L sub 2.
We don't care how big that is.
So we don't have to specify T sub 0.
We don't have to worry about it.
We just assume it's big enough and move on.
And then if it turns out it's not big enough, if your system crashes when one of Microsoft's infernal errors come up, then you worry about it then.
But you don't worry about it before that.
OK so now we can really define white noise and understand what white noise is.
White noise is noise that is effectively Wide Sense Stationary over a large enough interval to include all time intervals of interest.
Also on the other part of white noise, is that the spectral density is constant in f over all frequencies of interest.
OK in other words, white noise, really you have to define it within terms of what you're interested in.
And that's important to understand.
If you're dealing with wireless channels, you sometimes want to assume that the noise you're seeing is actually white noise.
But sometimes you do things like jumping from one frequency band to another frequency band to transmit.
And when you jump from one frequency band to another frequency band, the noise might in fact be quite different in this band, than it is in this band.
But you're going to stay in these bands for a relatively long time.
So as far as any kind of analysis goes which is dealing with those intervals, you want to assume that the noise is white.
As far as any overall analysis that looks at all of the intervals at the same time, in other words, which is dealing with some very long-term phenomenon, you can't assume that the noise is white.
OK in other words, white noise is a modeling tool.
It's not something that occurs in the real world.
But it's a simple modeling tool.
Because what do you do with it?
You start out by assuming that the noise is white.
And then you start to get some understanding of what the problem is about.
And after you understand what the problem is about, you come back and say, is it really important for me that the noise is going to have a constant spectral density over this interval of interest?
And at that point you say, yes or no.
OK.
It's important to always be aware that this doesn't apply for times and frequencies outside the interval of interest.
And if the process is also Gaussian, it's called white Gaussian noise.
So white Gaussian noise is noise which has a flat spectral density over this effective stationarity period that we're interested in, over some very large period of time, minus T sub 0 to plus T sub 0.
And over some very large frequency interval, minus W up to plus W or even better over some limited frequency band in positive and negative frequencies, the spectral density is constant.
You can't define the spectral density until after you look at this effective period of stationarity.
But at that point, you can then define the spectral density and see whether it looks constant over the period you're interested in.
You can either see whether it is, or if you're an academic and you write papers, you assume that it is.
OK.
The nice thing about being an academic is you can assume whatever you want to assume.
OK let's go back to linear functionals.
The stuff about linear functionals is in fact in the notes.
This will start to give us an idea of what spectral density really means.
So if I have a Wide Sense Stationary process, a linear functional, well this is what a linear functional is in general.
But for a Wide Sense Stationary process, we have the single variance, single variable covariance function.
The expected value of the product of two random variables of this sort is just this integral here.
This is what we did before.
If you don't remember this formula, good.
It's just what you get when you take the definition of V sub i, which is the integral of g sub i of t with Z of t. And then you multiply it by V sub j.
And you take Z of t and Z of tau.
And you take the expected value of the two.
And then you integrate the whole thing.
And it sort of comes out to be that, if I've done it right.
You can express this in this terms, namely as the integral of g sub i of t times a convolution of this function times this function.
You can then use Parseval's identity.
Because this convolution gives you a function of t, OK.
When you integrate this over tau, this whole thing is just a function of t. OK.
So what we're dealing with is the integral of a function of t times a function of t. And Parseval's relation says that the integral of a function of time, times a function of time is equal to the integral of the Fourier transform times the Fourier transform.
OK. You all know this.
This is almost the same as a trick we used with linear filters, just slightly different.
So that says that this expected value is equal to this product here.
And here I've used the complex conjugate for the Fourier transform of g sub j, because I really need it there if g is not a symmetric function.
OK so if these Fourier transforms, g sub i of t and g sub j of t are non-overlapping in frequency, what does this integral go to?
And this is absolutely wild.
I mean if this doesn't surprise you, you really ought to go back and think about what it is that's going on here.
Because it's very astonishing.
It's saying that stationarity says something you would never guess in a million years that it says.
OK it says, that if this and this do not overlap in frequency, this integral is 0.
OK in other words, if you take a linear functional over one band of frequencies, it is uncorrelated with every random variable that you can take in any other band of frequencies.
So long as the function is Wide Sense Stationary, this uncorrelated effect takes place.
If you're dealing with the Gaussian random process, it's also stationery.
Uncorrelated means independent.
And what happens then, is that expected value of V sub i and V sub j is zero.
V sub i and V sub j are statistically independent random variables.
And it says that whatever is happening in one frequency band of a stationary Gaussian process is completely independent of what's happening in any other band.
Now if any of you can give me an intuitive reason for why that is, I would love to hear it.
Because I mean it's one of those-- well after you think about it for a long time, it starts to sort of make sense.
If you read the appendix in the notes, the appendix in the notes sort of interprets it a little bit more.
If you take this process, if you limit it in time and you expand it in a Fourier series, what's going to happen to those coefficients in the Fourier series?
If the process is stationary, each one of them, if you think of them as being complex coefficients, the phase in each one has to be random and uniformly distributed between zero and 2 Pi.
In other words, if you have a phase on any sinusoid, which is either deterministic or anything other than uniform, then that little slice of the process cannot be Wide Sense Stationary.
And there's nothing in any other frequencies that can happen to make the process stationary again.
OK in other words, when you expand it in a Fourier series, you have to get coefficients at every little frequency slice, which have uniform face to them.
In other words, the Gaussian random variables that you're looking at, are things that are called proper complex Gaussian random variables which have the same variance for the real part as for the imaginary part.
When you look at them as a probability density, you find circles.
In other words, you find circularly symmetric random variables at every little frequency interval.
And that's somehow, what stationarity is implying for us.
So it's a pretty important thing.
And it doesn't just apply to white noise.
It applies to any old process with any old spectral density, so long as we've assumed that it's Wide Sense Stationary.
OK.
So, different frequency bands are uncorrelated for Gaussian Wide Sense Stationary processes.
Different frequency bands are completely independent of each other.
OK, as soon as you have stationary noise, looking at one frequency band to try to get any idea of what's going on in another frequency band, is an absolute loser.
You can't tell anything from one band about what's going on in another band.
So long as you're only looking at Gaussian noise If you're sending data which is correlated between the two bands, then of course it's worthwhile to look at both bands.
But if you're sending data that's limited to one frequency band, then at that point, there's no reason to look anywhere else.
We're going to find that out when we start studying detection.
It'll be a major feature of understanding detection at that point.
OK, and one of the things you want to understand with detection, is what things can you ignore and what things can't you ignore.
And this is telling us something that can be ignored.
It's also starting to tell us what this spectral density means.
But I'll interpret that in the next slide I hope.
OK if you take this set of functions g sub j of t, in the notes I now convert that set of functions to V sub j's instead of g sub j's.
If I start out with that set of functions these-- oh dear, this is a random variable-- V sub j is the component of the process in the expansion, in that orthonormal set.
So in other words, you can take this process, you can expand it into an orthonormal expansion using random variables and these orthonormal functions.
If g sub j of t is a narrow band, and S sub Z of f is constant within that band-- in other words we don't need white noise here.
All we need is a spectral density which changes smoothly.
And if it changes smoothly, we can look at a small enough interval.
And then what happens?
When we take the expected value of V sub j squared, what we get is this times this times this-- OK the product of these three terms and since this-- that's supposed to be a j there.
OK so what's happening is these are narrow band, this is just a constant within that narrow band.
So we can pull this out and then all we have is the integral g sub i of f times g sub i complex conjugate of f, which since these functions are orthonormal, is just one.
So what we find is that random variable there is simply the value of the spectral density.
Now we don't use that to figure out what these random variables are all about.
We use this to understand what the spectral density is all about.
Because the spectral density is really the energy per degree of freedom in the process at frequency f. In other words, if we use orthonormal functions which are tightly constrained in frequency, they just pick out the value of the spectral density at that frequency.
S sub Z of f sub 0 is just sort of a central frequency in this band, or integrating over here, OK.
So the interpretation of the spectral density is that it's the energy per degree of freedom in any kind of orthonormal expansion you want to use, so long as those functions are tightly limited in frequency.
So that's what spectral density means.
It tells you how much energy there is in the noise, at all these different frequencies.
It tells you, for example at this point, that if you start out with a sinc Gaussian process like we were talking about before, pass it through a filter, what the filter is going to do is change the spectral density at all the different frequencies.
And the interpretation of that which now makes perfect sense when you're going through a filter, is the amount of energy in each of those frequency bands is going to be changed exactly by the amount that you filtered it.
OK so we have a nice thing there.
S sub Z of f is the energy per degree of freedom in the process at frequency f. For a Gaussian Wide Sense Stationary process, this is the energy per degree of freedom at f. And the noise at all different frequencies is independent.
That's what we were saying before.
And after filtering, this independence is maintained.
In other words, you pass a Gaussian random process through a filter and you still have independence between all the different frequency bands.
It's an absolutely remarkable property.
Just this property of the process acting the same at each time as it acts at each other time, that this leads to this amazing property of things being independent from one frequency to another is really worth thinking about.
And it's one of the things that people use when they actually design systems.
I mean it's almost natural to use if you don't think about it.
If you start thinking about it then it becomes even more worthwhile.
OK should I start detection?
Let me just start to say a few things about detection, then I want to stop in time to pass the quiz back.
OK let's talk about where detection fits in with our master plan of what we've been doing.
We started the course talking about what was over on this side of the channel business, namely all the source coding and all of those things.
We then started to talk about how do you do signal encoding.
In other words, how do you map from binary digits coming in, into signals.
Well we never talked about this problem.
We only talked about this problem, when you didn't have any noise.
We carefully avoided dealing with this problem at all.
Then we talked about baseband modulation for PAM and for QAM.
For PAM it was real, for QAM it was complex.
Then we talked about how you go from baseband frequencies up to passband frequencies.
Now we've been talking about what happens when you add white Gaussian noise, or any other noise at passband.
Then you go from passband down to baseband.
When I do this, we're going to have to come back at some point and deal with the question of what happens when you take passband white Gaussian noise and convert it down to baseband.
Because some kind of funny things happen there.
And we have to deal with it with a little bit of care.
But let's forget about that for the time being, and suppose everything works out all right.
Then we go through our baseband demodulator.
Finally something comes out of the baseband demodulator, which we hope is the same as what came out of the signal encoder.
But what we're trying to do at this point now, is when the noise is here to say, this is in some sense, going to be what we put in plus noise.
And for the time being, in terms of looking at detection, we will just assume that.
We will assume that this thing down here is what went in, plus noise.
And assume that all this process of passing white Gaussian noise through this junk gives us just a signal plus noise.
OK, so what a detector does is it observes a sample value of this random variable V, or it observes a vector, or observes a random process.
It observes whatever it's going to observe.
And it guesses the value of another random variable.
And we'll call the other random variable H. We had been calling it other things for the input.
It's nice to call it H because statisticians always call this a hypothesis.
And I think it's easier to think of what's going on if you think of it as a hypothesis type random variable.
Namely the detector has to guess.
If we have a binary input, it has to guess at a binary output.
It has to flip a coin somehow and say, I think that what came in is a zero, or I think what came in is a one.
The synonyms for detection are hypothesis testing, decision making, and decoding.
They all mean exactly the same thing.
There's no difference whatsoever between them, except the fields that they happen to deal with.
And each field talks about it in a different sense.
The word detection really came mostly from the radar field where people were trying to determine whether a target was there or not.
And hypothesis testing has been around for a great deal longer.
Because all scientists from time immemorial have had to deal with the question of have you collect a bunch of data about something.
And it's always noisy data.
There's always a bunch of junk that comes into it.
And you have to try to make some conclusions out of that.
And making conclusions out of it is easier if you know there's only a finite set of alternatives which might be true.
In a presidential election, after you know who the candidates are, it's a binary decision.
The people who try to guess who people are going to vote for, are really doing binary decision making.
They don't have very good probability models.
And it certainly isn't a stationary process.
But in fact, it is a detection problem, the same as the detection problem we have here.
So all these problems really fall into the same category.
We're fortunate here at having one of the cleanest probabilistic descriptions for detection that you will ever find.
So if you want to study decision making, you're far better off trying to learn about it in this context and then using it in all these other contexts, than you are doing something else.
If you go back and look at old books about this, you will be amazed at the philosophical discussions that people would go through about what makes sense for hypothesis testing, and what doesn't make sense.
And all of these arguments were based on a fundamental misapprehension that scientists had until relatively recently.
And the misapprehension that they had, was that they refused to accept models as taking an intermediate position between physical reality, and then you have models, and then you have doing something with a model.
And you do something with the model instead of doing something with the physical reality.
And people didn't understand that originally.
So they got the analysis of the model terribly confused with trying to understand what the physical reality was.
In other words, statisticians and probabilists were both doing the same thing.
Both of them were trying to simultaneously determine what a reasonable model was, and to determine what to do with a model.
And because of that, they can never decide on anything.
Because they can never do what we've been doing, which says take a toy model, analyze the toy model.
Figure out something from it about the physical reality.
Then go back and look at reality and see what you need to know about reality in order to make these decisions, which is the way people do it now I think.
Particularly decision theory is done that way now.
Because now we usually assume both that we know Apriori probabilities, it's called, for these inputs here.
For the binary communication problem, we usually want to assume that these inputs are equiprobable, in other words, if P sub 0 is equal to a half, and P sub 1 is equal to a half.
And that's the Apriori probability of what's coming in.
And we then want to assume some probabilistic model on v. Usually what we do, is we figure out what v is in terms of a conditional probability density, conditional probability density of v, conditional on either of these two outputs.
You see for this model here, this is just made in heaven for us.
Because this thing is an input plus a Gaussian random variable.
So if we look at the conditional probability density of the output, which is signal plus noise, conditional on the signal is just a Gaussian random variable shifted over by the input.
OK.
So we have these two probabilities, the input probability, this thing, which statisticians call a likelihood, which is the probability of this conditional on this.
And in terms of that, we try to make the right decision.
Now, I think I'm going to stop now so we can pass back the quizzes and so on.
Before Wednesday, if you don't read the notes before Wednesday, spend a little bit of time thinking about how you want to make the optimal decision for this problem as we've laid it out now.
What's surprising is that it really is a trivial problem.
Or you can read the notes and accept the fact that it's a trivial problem.
But there's nothing hard about it.
The only interesting things in detection theory are finding easy ways to actually solve this problem which is conceptional trivial.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK I want to review a little bit what we said about detection at the end of last hour, last hour and a half, because we were going through it relatively quickly.
Detection is a very funny subject.
Particularly the way that we do it here because we go through a bunch of very, very simple steps.
Everything looks trivial-- I hope it looks trivial-- as we go, at least after you think about it for awhile, and you work with it for awhile, you will come back and look at it and you will say, "yes, in fact that is trivial."
Nothing when you look at it for the first time is trivial.
And the kind of detection problems that we're interested in is-- to start out with-- we want to look just at binary detection.
We're sending a binary signal, it's going to go through some signal encoder, which is the kind of channel encoder we've been thinking about.
It's going to go through a baseband modulator, a baseband to passband modulator.
It's going to have white Gaussian noise, or some kind of noise, added to it.
It's going to come out the other end.
We come back from passband to baseband.
We then go through a baseband demodulator.
We then sample at that point.
And, the point is, when you're all done with all of that, what you've done is you've started out sending either plus a or minus a as a one- dimensional numerical signal.
And when you're all through, there's some one- dimensional number that comes out, v. And on the basis of that one- dimensional number, v, you're supposed to guess whether the output is zero or one.
Now, one of the things that we're doing right now is we're simplifying the problem in the sense that we're not looking at a sequence of inputs coming in, and we're not looking at a sequence of outputs coming out.
We're only looking at a single input coming in.
In other words, you build this piece of communication equipment, you get it all tuned up, you get it into steady state.
You send one bit.
You receive something.
You try to guess at the receiver, what was sent.
And at that point you tear the whole thing down and you wait a year until you've set it up perfectly again.
You send another bit.
And we're not going to worry at all about what happens with the sequence, we're only going to worry about this one shot problem.
You sort of have some kind of clue that if you send the whole sequence of bits, in a system like this, and you don't have intersymbol interference, and the noise is white, so it's sort of independent from time to time.
You sort of have a clue that you're going to get the same answer whether you send the sequence of data or whether you just send a single bit.
And we're going to show later that that, in fact, is true.
But for the time being we want to understand what's going on, and to understand what's going on we take this simplest possible case where there's only one bit that's being transmitted.
It's the question, "Are we going to destroy ourselves in the next five years or not?"
And this question is important to most of us, and at the output we find out, in fact, whether we're going to destroy ourselves or not.
So it's one bit, but it's one important bit.
OK, why are we doing things this way?
Want to tell you a little story about the first time I really talked to Claude Shannon.
I was a young member of the faculty at that time, and I was working on a problem which I thought was really a neat problem.
It was interesting theoretically.
It was important practically.
And I thought, "gee, I finally have something I can go to this great man and talk to him about."
So I screwed up my courage for about two days.
Finally I saw his door open and him sitting there, so I went in and started to tell him about this problem.
He's a very kind person and he listened very patiently.
And after about 15 minutes he said, "My god!
I'm just sort of lost with all of this.
There's so much stuff going on in this problem.
Can't we simplify it a little bit by throwing out this kind of practical constraint you've put on it?"
I said, "yeah, I guess so."
So we threw that out and the we went on for awhile later, and then he said, "My god, I'm still terribly confused about this whole thing.
Why don't we simplify it in some other way?"
And this went on for about an hour.
As I say, he was a very patient guy.
And at the end of an hour I was getting really depressed.
Because here was this beautiful problem that I thought was going to make me famous, give me tenure, do all these neat things.
And here he'd reduced the thing to a totally trivial toy problem.
And we looked at it.
And we said, yes this is a trivial toy problem.
This is the answer.
The problem is solved.
But so what?
And then he suggested putting some of those constraints back in again.
And as we started putting the constraints back in, one- by- one, we saw that each time we put a new constraint in-- since we understood the problem and its simplest form-- putting the constraint in, it was still simple.
And by the time we built the whole thing back up again, it was clear what the answer was.
OK, in other words, what theory means is really solving these toy problems.
And solving the toy problems first.
And in terms of practice, some people think the most practical thing is to be practical.
And the whole point of this course, and this particular subject of detection is a wonderful example of this, the most practical thing is to be theoretical.
I mean, you need to add practice to the theory, but the way you do things is you start with a theory-- which means you start with the toy problems, you build up from those toy problems, and after you build up for awhile, understanding what the practical problem is also-- you then understand how to deal with the practical problem.
And the practical engineer who doesn't have any of that fundamental knowledge about how to deal with these problems is always submerged in a sea of complexity.
Always doing simulations of something that he doesn't, he or she doesn't understand.
Always trying to interpret something from it, but with just too many things going on to have any idea of what it really means.
Ok, so that's why we're making this trivial assumption here.
We're only putting one bit in.
We're ignoring what happens all the way through the system.
We only get one number out.
We're going to assume that this one number here is either plus or minus a, plus the Gaussian random noise variable.
And we're not quite sure why it's going to be plus or minus a, plus a Gaussian noise random variable, but we're going to assume that for the time being.
OK?
So the detector observes the sample values of the random variable for which this is the sample value, and then guesses what the value of this random variable, h, which is what we call the input now.
Because we view it from the standpoint of the detector-- the detector has two possible hypotheses-- one is that a zero was sent, and the other that a one was sent.
And on the basis of this observation, you take first the hypothesis zero and you say, "Is this a reasonable hypothesis?"
Then you look at the hypothesis one, say, "Is this a reasonable hypothesis?"
And then you guess whether you think zero was more likely or one is more likely, given this observation that you've had.
So what they detector has, at this point, is a full statistical characterization of the entire problem.
Mainly you have a model of the problem.
You understand every probability in the universe that might have any effect on this.
And what might have any effect on this-- as far as the way we've set up the problem-- is only the question of what are the probabilities that you're going to send one or the other of these signals here?
And conditional on each of these, what are the probabilities of this random variable appearing at the output?
Because you have to base your decision only on this.
So all of the probabilities only give you this one simple thing.
Hypothesis testing, decision making, decoding, all mean the same thing.
They mean exactly the same thing.
And they're just done by different people.
OK, so what that says is we're assuming the detector uses a known probability model.
And in designing the detector, you know what that probability model is.
It might not be the right probability model, and one of the things that many people interested in detection study is the question of when you think the probability model is one thing and it's actually something else, how well does the detection work?
It's a little like the Lempel-Ziv algorithm that we studied earlier for doing source coding.
Which is, how do you do source coding when you don't know what the probabilities are?
And we found the best way to study that, of course, was to first find out how to do source encoding when you did know what the probabilities were.
So we're doing the same thing here.
We assume the detector is designed to maximize the probability of guessing correctly.
In other words, it's trying to minimize the probability of error.
We call that a MAP detector-- maximum a posteriori probability decoding.
You can try to do other things.
You can say that there's a cost of one kind of error, and there's another cost of another kind of error.
I mean, if you're doing medical testing or something.
If you guess wrong in one way, you tell the patient there's nothing wrong with them, the patient goes out, drops dead the next day.
And you don't care about that, of course, but you care about the fact that the patient is going to sue the hospital for 100 million dollars and you're going to lose your job because of it.
So there's a big cost to guessing wrong in that way.
But for now, we're not going to bother about the costs.
One of the things that you'll see when we get all done is that putting in cost doesn't make the problem any harder, really.
You really wind up with the same kind of problem.
OK, so h is the random variable that will be detected, and v is the random variable that's going to be observed.
The experiment is performed.
Some sample value of v is observed, and some sample value of the hypothesis has actually happened.
In other words, what has happened is you prepared the whole system.
Then at the input end to the whole system, the input to the channel, somebody has chosen a one or a zero without the knowledge of the receiver.
That one or zero has been sent through this whole system, the receiver has observed some output, v, so in fact we're now dealing with the sample values of two different things.
The sample value of the input, which is h, the sample value of the output, which is v, and in terms of the sample value of the output, we're trying to guess what the sample value of the input is.
OK, an error then occurs if, after the output chooses a particular hypothesis as its guess, and that hypothesis, then, is a function of what it receives.
In other words, after you receive something, what the detector has to do is somehow map what gets received, which is some number, into either zero or one.
It's like what a Quantizer does.
Mainly, it maps the whole region into two different sub- regions.
Some things are mapped into zero, Some things are mapped into one.
This H bar then becomes a random variable, but is a random variable that is a function of what's received.
So we have one random variable, H, which is what actually happened.
There's another random variable, H hat, which is what the detector guesses will happen.
This is an unusual random variable, because it's not determined ahead of time.
It's determined only in terms of what you decide your detection rule is going to be.
This is a random variable that you have some control over.
These other random variables you have no control over at all.
So that's the random variable we're going to choose.
And, in fact, what we're going to do is we're going to say what we want to do is this MAP decoding, maximum a posteriori probability decoding, where we're trying to minimize the probability of screwing up.
And we don't care whether we make an error of one kind or make an error of the other kind.
OK, is that formulation of the problem crystal clear?
Anybody have any questions about it?
I mean, the easiest way to get screwed up with detection is at a certain point to be going through, studying a detection problem, and then you suddenly realize you don't understand what-- you don't understand what the whole problem is about.
OK, let's assume we do know what the problem is, then.
In principle it's simple.
Given a particular observed value, what we're going to do is we're going to calculate the-- what we call the a posteriori probability, the probability given that particular sample value of the observation-- we're going to calculate the probability that what went into the system is a zero and what went into the system is a one.
OK?
This is the probability that j is the sample value of H, conditional on what we observed.
OK, if you can calculate this quantity, it tells you if I decide that-- if I guess that H is equal to j-- this in fact tells me that this is the probability that guess is correct.
And if this is the probability that guess is correct, and I want to maximize my probability of guessing correctly, what do I do?
Well, what I do is my MAP rule is arg max of this probability.
And "arg max" means instead of trying to maximize this instead of trying to maximize this quantity over something, what we're doing is trying to find the value of j, which maximizes this.
In other words, we calculate this for each value of j, and then we picked the j for which this quantity is largest.
In other words, we maximize this but we're not interested in the maximum value of it at this point.
We're interested in it later because that's the probability that we're choosing correctly.
What we're interested in, for now, is what is the hypothesis that we're going to guess.
OK?
So this probability of being correct is going to the this probability for this maximal j.
And when we average over v we get the overall probability of being correct.
There's a theorem which is stated in the notes, which is one of the more trivial theorems you can think of, which says that if you do the best thing for every sample point you, in fact, have done the best thing on average.
I think that's pretty clear.
If you, well I mean you can read it if you want to form a proof, but if you do the best thing all the time, then it's the overall best thing.
OK, so that's the general idea of detection.
And in doing this we have to be able to calculate these probabilities, so that's the only constraint.
These are probabilities, which means that this set of hypothesis is discrete.
If you have an uncountable infinite number of hypotheses, at that point you're dealing with an estimation problem.
Because you don't have any chance in hell of getting the right answer, exactly the right answer.
And therefore you have to have some criterion for how close you are.
And that's what's important in estimation.
And here what's important is really, do we guess right or don't we guess right.
And we don't, we don't care.
There aren't any near misses here.
You either get it on the nose or you don't.
OK, so we want to study binary detection now to start off with.
We want to trivialize the problem because even that problem we just stated is too hard.
So we're going to trivialize it in two ways.
We're going to assume that there are only two hypotheses.
That it's a binary detection problem.
And we're also going to assume that it's Gaussian noise.
And that will make it sort of transparent what's happening.
So H takes the values zero or one.
And we'll call the probabilities with which it takes those values P zero and P one.
These are called a priori probabilities.
In other words, these are the probabilities that the hypothesis takes to value zero or one before seeing any observation.
And the probabilities after you see the observation are called a posteriori probabilities.
In other words, probabilities after the observation and the probabilities before the observation.
Up until about 1950 statisticians used to argue terribly about whether it was valid to assume a priori probabilities.
And as you can see by thinking about it a little bit, the problem they were facing was they couldn't separate the problem of choosing a mathematical model and analyzing it from the problem of figuring out whether the model was valid or not.
And at that point people studying in that area had not gotten to the point where they could say, "Well, maybe I ought to analyze the problem for different models, and then after I understand what happens for different models I then ought to go back because I'll know what's important to find out in the real problem."
But up until that time, there was just fighting among everyone.
Bayes was the person who decided you really ought to assume that there's a model to start with.
And he developed most of detection theory at an early time.
And people used to think that Bayes was a terrible fraud.
Because in fact he was using models of the problem rather than nothing.
But anyway, that's where we were.
We're also going to assume that, after we get all through with modulation and demodulation, and we really want to look at a general problem here.
There's one discrete random variable, H, one analog random variable, which has a probability density, v, what we want to assume is that there's a probability density that we know, which is the probability density of the observation conditional on the hypothesis.
We're assuming that we know this, and we know this-- we call these things likelihoods-- and in most communication problems anyway, it's far easier to get your hand on these likelihoods than it is to get your hand the a posteriori probabilities, which you're really interested in.
So we find these likelihoods, we can find the marginal density of the observation.
Which is just the weighted sum.
The probability that the hypothesis is zero times a conditional probability of the observation and so forth.
So we're going to assume that those densities exist.
We're going to assume that we know them.
And then with a great feat of probability theory, we say the a posteriori probability is equal to the a priori probability times the likelihood divided by the marginal probability of v. OK?
What was the first thing you learn in probability when you started studying random variables?
It was probably this formula, which says when you have a joint density of two random variables you can write it in two ways.
You can either write, "density of one times density of two conditional and one is equal to the density of two times density of one conditional in two."
And then you think about it a little bit and you say, "A ha!"
It doesn't matter whether the first one is the density or whether it's the probability.
You can deal with it in the same way.
And you get this formula, which I hope is not unusual to you.
OK, so our MAP decision rule then, our MAP decision rule, remember, is to pick the more, is to find the a posteriori probability which is most probable.
Because that is the probability of being correct.
So, in fact, if this probability is bigger than this probability, this is the a posteriori probability that H is equal to zero.
This is the a posteriori probability that H is equal to one.
So we're just going to compare those two and pick the larger.
And if this one is larger than this one, we pick our choice equal zero.
And if it's smaller, we pick our choice equal to one.
So this is what MAP detection is.
Why did I make this greater than or equal and this less than?
Well, if we have densities, it doesn't often make any difference.
Strangely enough, it does sometimes make a difference.
Because sometimes you can have a density, and the densities are the same for both of these likelihoods.
And you can find situations where it's important.
But when the two probabilities are the same, the probability of being correct is the same in both cases, so it doesn't make any difference what you do when you have a quality here.
And therefore we've just made a decision.
We've said, OK, what we're going to do is whenever this is equal to this, we're going to choose zero.
If you prefer choosing one, be my guest.
All of your MAP error probabilities will be exactly the same.
Nothing will change.
It just is easier to do the same thing all the time.
OK, well then we look at this formula, and we say, "Well, I can simplify this a little bit."
If I take this likelihood and move it over to this side, and if I take this marginal density and move it over to this side, and if I take p zero and move it over to this side, then the marginal densities cancel out.
They had nothing to do with the problem.
And I wind up with a ratio of the likelihoods.
And what do you think the ratio of the likelihoods is called?
Somebody got the smart idea of calling that a likelihood ratio.
Somehow the people in statistics were much better at generating notation than the people in communication theory who have done just an abominable job of choosing notation for things.
But anyway, they call this a likelihood ratio.
And the rule then becomes: if the likelihood ratio is greater than or equal to the ratio of p1 to p0, we choose zero.
And if it's less, we choose one.
And we call this ratio the threshold.
So in fact what this says is binary MAP tests are always threshold tests.
And by a threshold test I mean finds the likelihood ratio, compare the likelihood ratio with the threshold-- the threshold in fact is this ratio of a priori probabilities-- and at that point you have actually achieved the MAP test.
In other words, you have done something which actually, for real, minimizes the probability of error.
Maximizes the probability of being correct.
Well because of that, this thing here, this likelihood ratio, is called a sufficient statistic.
And it's called a sufficient statistic because you can do math decoding just by knowing this number.
OK?
In other words, it says you can you can calculate these likelihoods.
You can find the ratio of them-- which is this likelihood ratio-- and after you know the likelihood ratio, you don't have to worry about these likelihoods anymore.
This is the only thing relevant to the problem.
Now this doesn't seem to be a huge saving, because here we're dealing with two real numbers-- well here we've reduced it to one real number-- which is something.
When we start dealing with vectors, when we start dealing with wave forms, this is really a big thing.
Because what you're doing is reducing the vectors to numbers.
And when you reduce accountibly infinite dimensional vector to a number, that's a big advantage.
It also, in terms of the communication problems we're facing, breaks up a detector into two pieces in an interesting way.
Mainly it says there are things you do with the wave form in order to calculate what this likelihood ratio is, and then after you find the likelihood ratio you just forget about what the wave form was and you deal only with that.
What we're going to find out is in this problem we were looking at here-- we're going to find out later when we look at the vector problem-- in fact this thing here is in fact the likelihood ratio if you make an observation out at this point here.
In other words, right at the front end of the receiver, that's where you have all the information you can possibly have.
If you calculate likelihood ratios at that point what you're going to do is to find the likelihood ratio you're going to go through all this stuff right here and wind up with that which is work which is proportional to the likelihood ratio right here.
OK, so one of the things we're doing right now is we're not looking at that problem.
We're only looking at the simpler problem, assuming a one dimensional problem.
But the reason we're looking at it is that later we're going to show that this is, in fact, the solution to the more general problem.
Which was Shannon's idea in the first place.
Of, how do you solve the trivial problem first and then see what the complicated problem is.
OK, so that's what we're trying to do, summarized here for any binary detection problem where the observation has a sample value of a random something.
Namely, a random vector, a random process, a random variable, a complex variable, a complex anything.
Anything whatsoever, so long as you can assign a probability density to it.
You calculate the likelihood ratio, which is this ratio here, so long as you have then cities even talk about.
The MAP rule is to compare this likelihood ratio with the threshold data-- which is just the ratio of the a priori probabilities-- if this is greater than or equal to that, you choose zero.
Otherwise you choose one.
The MAP rule, as I said before, partitions this observation space into two pieces.
Into two segments.
And one of those pieces gets mapped into zero.
One of the pieces gets mapped into one.
It's exactly like a binary quanitizer.
Except the rule you use to choose the the quantization regions is different.
But a quanitizer maps a space into a finite set of regions.
And this detection rule does exactly the same thing.
And since the beginning of information theory people have been puzzling over how to make use of the correspondence between quanitization on one hand and detection on the other hand.
And there are some correspondences but they aren't all that good most of the time.
OK, so you get an error when the actual hypothesis that occurred, namely the bit that got sent was i and if the observation landed in the other subset.
We know that the MAP rule minimizes the error probability.
So you have a rule which you can use for all binary detection problems so long as you have the density.
And if you don't have a density you can generalize it without too much trouble.
OK, so we want to look at the problem in Gaussian noise.
In particular we want to look at it for 2PAM.
In other words for a standard PAM system, where zero gets mapped into plus a and one gets mapped into minus a.
This is often called antipodal signaling because you're sending a plus something and a minus something.
They are at opposite ends of the spectrum.
You push them as far away as you can, because as you push them further and further away it requires more and more energy.
So you use the energy you have.
You get them as far apart as you can, and you hope that's going to help you.
And we'll see that it does help you.
OK, so what you receive then, we'll assume, is either plus or minus a-- depending on which hypothesis occured-- plus a Gaussian random variable.
And here's where the notation of communication theorists rears its ugly head.
We call the variance of this random variable n 0 over 2.
I would prefer to call it sigma squared but unfortunately you can't fight city hall on something like this.
And everybody talks about n 0 and n 0 over 2.
And you got to get used to it.
So here's where we're starting to get used to it.
So that's the variance of this noise random variable.
OK, we're only going to send one binary digit, H, so this is the only, this is the sole problem we have to deal with.
We've made a binary choice.
Added one Gaussian random variable to it.
You observe the sum, and you guess.
So what are these likelihoods in this case?
Well, the likelihood if H is equal zero, in other words if you're sending a plus a, the likelihood is just a Gaussian density shifted over by a.
And if you're sending, on the other hand, a one-- which means you're sending minus a-- you have a Gaussian density shifted over the other way.
Let me show you a picture of that.
We'll come back to analyze more things about the picture in a little bit, so don't worry about most of the picture at this point.
OK, this is the likelihood probability density of the output given that you sent a zero.
Mainly that you sent plus a.
So we have a Gaussian density-- this bell shaped curve-- centered around plus a.
If you sent a one, you're sending minus a-- one gets mapped into minus a-- and you have the same bell shaped curve centered around minus a.
If you receive any particular value of v, mainly suppose you receive the value of v here, you calculate these two likelihoods.
One of them is this.
One of them is that.
You compare them, the ratio with the threshold, and you make your choice.
OK, so let's go back to do it.
To do the arithmetic.
Here are the two likelihoods.
You take the ratio of these two things.
When you take the ratio of them, what happens?
And this sort of always happens in these Gaussian problems.
These terms cancel out.
Well it always happens in these additive Gaussian problems.
These terms cancel out.
You take a ratio of two exponents.
You just get the difference.
So the likelihood ratio-- this divided by this-- is then e to the minus v minus a squared over n0.
and v plus a squared over n0.
OK?
Because normally the Gaussian density is something divided by two sigma squared, and sigma squared here is n0 over 2, so the 2's cancel out.
One nice thing about the notation anyway, you get rid of one factor of two in it.
Well so you have this minus this.
When you take the difference of these two things the v squareds cancel out.
Because one of these things is in the numerator, the other one was in the denominator.
So you have this term comes through as is.
This is-- you're dividing by this-- so when you multiply this turns into a plus sign.
So the v squared here cancels out with the v squared here.
The a squared here cancels out with the a squared here.
And it's only the inner product term that survives this whole thing.
And here you have plus 2va.
Here you have plus 2va.
So you wind up with e to the 4av divided by n0.
Which is very nice, because what it says is this likelihood, which is what determines everything in the world, is just a scalar in multiple of the observation.
And that that's going to simplify things a fair amount.
It's why that picture comes out as simply is it does.
OK, so to do a little more of the arithmetic.
This is the likelihood here, e to the 4av over n0.
So our rule is you compare this likelihood to the threshold-- which is p1 over p0, which we call eta-- and you look at that for awhile and you say, "Gee, this is going to be much easier to deal with.
Instead of looking at the likelihood ratio, I look at the log likelihood ratio."
And people who deal with Gaussian problems a lot, you never hear them talk about likelihood ratios, you always hear them talk about log likelihood ratios.
And you can find one from the other, so either one is equally good.
In other words, the log likelihood ratio is a sufficient statistic, because you can calculate the likelihood ratio from it.
So this is a sufficient statistic.
It's equal to 4av over n0.
And when this is greater than or equal the log of the threshold, you go this way.
When it's less than, you go this way.
So when you then multiply by n0 over 4a, your decision rule is you just look at the observation.
You compare it with n0 times log of eta divided by 4a.
And at this point we can go back to this picture and sort of sort out what all of it means.
Because this point here is now the threshold.
It's n0 times log of eta divided 4eta.
By 4a.
That's what we said the threshold had to be.
So we have these two Gaussian curves now.
Why do we have to go back and look at these Gaussian curves?
I told you that once we calculated the likelihood ratio we could forget about the curves.
So why do I want to put the curves back in?
Well because I want to calculate the probability of error at this point.
OK?
And it's easier to calculate the probability of error if, in fact, I draw curve for myself and I look at what's going on.
So here's the threshold.
Here's the density when H equals one is the correct hypothesis.
The probability of error is the probability that when I send one, the observation is going to be a random variable with this probability density and if it's a wild case, and I got an enormous value of noise, positive noise, the noise is going to push me over that threshold there and I'm going to make a mistake.
So, in fact, the probability of error-- conditional on sending one-- is just this probability of that little space in there.
OK?
Which is the probability that I'm going to say that zero occurred when, in fact, one occurred.
So that's my probability of error when one occurs.
What's the probability of error when zero occurs?
Well it's the same analysis.
When zero occurs-- mainly when the correct hypothesis is zero-- the output, v, follows this probability density here.
And I'm going to screw up if the noise carries me beyond this point here.
So you can see what the threshold is doing now.
I mean, when you choose a threshold which is positive it makes it much harder to screw up when you send a minus a.
It makes it much easier to screw up when you send a plus a.
But you see that's what we wanted to do, because the threshold was positive in this case, because p1 was so much larger than p0.
And because p1 is so much larger than p0-- p1 happens almost all the time-- and therefore you would normally almost choose p1 without looking at v. Which says you want to push it over that way a little bit.
OK, when you calculate this probability of error, it's the probability of the tail of a Gaussian random variable.
So you define this tail variable, q of x, is the complimentary distribution function of a normal random variable.
It's the integral from x to infinity of one over the square root of 2pi, e to the minus c squared over 2.
I guess this would make better sense if this were a z-- one and-- oh, no.
No, I did it right the first time.
That's an x, because x is the limit in there, you see.
So I'm calculating all of the probability density that's off to the right of x.
And the probability of error when H is equal to one is this probability-- which looks like it's the tail on the negative side but if you think about it a little bit, since the Gaussian curve is symmetric, you can also look at it as a q function where now when you have this is equal to zero, this corresponds to changing this plus to a minus here and that's the only change.
OK, so this looks a little ugly and it looks a little strange.
I mean you can sort of interpret this part of it here-- I can interpret this part if I'm using maximum likelihood decoding-- maximum likelihood is mapped decoding where the threshold is equal to one.
In other words, it's where you're assuming that the hypothesis is equally likely to be zero or one-- a priori-- which is a good assumption almost always in communication because we work very hard in doing source coding to make those binary digits equally likely zero or one.
And there are other reasons for choosing maximum likelihood.
If you don't know anything about the probabilities it's a good assumption in a sort of a max/min sense.
It sort of limits how much you can screw up by having the wrong probability, so it's a very robust choice also.
OK, but now this, we're taking the ratio of a with the square root of n0.
Well the square root of n0 over 2 is really the standard deviation of the noise.
So what we're doing is comparing the amount of input we've put in with the standard deviation of the noise.
Now does that make any sense?
The probability of error depends on that ratio?
Well yeah, it makes a whole lot of sense.
Because if, for example, I wanted to look at this problem in a different scaling system-- if this is volts and I want to look at it in milli-volts-- I'm going to divide a by, I'm going to multiply a by 1000.
I'm going to multiply the standard deviation of the noise by 1000.
Because one of the things we always do here-- the way we choose n0 over 2-- n0 over 2 is sort of a meaningless quantity.
It's the noise energy in one degree of freedom in the scaling reference that we're using for the data.
OK?
And that's the only definition you can come up with that makes any sense.
I mean you scale the data in whatever way you please, and when we've gone from baseband to passband, we in fact have multiplied the energy in the input by a factor of two.
And therefore, because of that, we're going to-- n0 at passband is going to be a square root of 2 bigger than it is at baseband.
If you don't like that, live with it.
That's the way it is.
Nobody will change n0, no matter who wants them to change it.
OK, so this term make sense.
It's the ratio of the signal amplitude to the standard deviation of the noise.
And that should be the only way that the signal amplitude or the standard deviation of the noise enters in, because it's really the ratio that has to be important.
Why this crazy term?
Well if you look at the curve you can sort of see why it is.
The threshold test is comparing the likelihood ratio of this curve with the likelihood ratio of this curve.
What's going to happen as a gets very, very large?
You move a out, and the thing that's happening then is this curve-- which is now coming down in a modest way here-- if you move a way out here, you're going to have almost nothing there.
And it's going to be going down very fast.
It's going to be going down very fast relative to its magnitude.
In other words the bigger a gets, the bigger this difference is going to be for any given threshold.
And that's why you get a over square root of n0 here.
And here you get exactly the opposite thing.
That's because for a given threshold, as this signal to noise ratio gets bigger, this threshold term becomes almost totally unimportant.
I mean you get so much information out of the reading you're making, because it's so reliable, that having a threshold is almost completely irrelevant.
And therefore you can sort of forget about it.
If a is very large, this term is zilch.
OK?
So if you want to have reliable communication and you use a large amount of single noise ratio to get it, that's another reason for forgetting about whether the threshold is one or something else.
And we would certainly like to deal with problems where the threshold is equal to one because most people who remember q of a signal to noise ratio.
I don't know anybody who can remember this formula.
I'm sure there's some people, but I don't think anybody who works in the communication field ever thinks about this at all.
Except the first time they derive it and the say, "Oh, that's very nice."
And then they promptly forget about it.
The only reason I think about it more is I teach the course sometimes.
Otherwise I would forget about it, too.
OK, which is what this says.
For communication we assume p0 is equal to p1.
So we assume that eta is equal to one.
So the probability of error, which is also the probability of error when H is equal to one-- in other words when a one actually enters the communication system-- is equal to the probability of error when H is equal to 0.
In other words these two tails here, when the threshold is equal to one you set the threshold right there.
The probability of this tail is clearly equal to the probability of this tail.
Just by symmetry.
So these two error probabilities are the same.
And in fact they are just q of a over the square root of n0 over 2.
It's nice to put this in terms of energy.
We said before that energy is sort of important in the communication field.
So we call e sub b the energy per bit that we're spending to send data.
I mean don't worry about the fact that we're only sending one bit and then we're tearing the communication system down.
Because pretty soon we're going to send multiple bits.
But the amount of energy we're spending sending this one bit is a squared.
At least back in this frame of reference that we're looking at now, where we're just looking at this discrete signal and single variable noise.
And n0 over 2 is the noise variance of this particular random variable, z, so when we write this out in terms of Eb, which is a squared, it looks like this-- it's 2eb over n0.
So the probability of error for this binary communication problem is just the square root of 2Eb over n0.
Which is a formula that you want to remember.
It's the error probability for binary detection when n0 over 2 is the noise energy on this one degree of freedom and Eb is the amount of energy you're spending on this one degree of freedom.
You will see about 50 variations of this as we go on.
If you try to remember this fundamental definition, it'll save you a lot of agony.
Even so, everybody I know who deals with this kind of thing always screws up the factors of twos.
And finally when they get all done, they try to figure out from common sense or from something else, what the factors of two ought to be.
And they reduce their probability of error to about a quarter after they're all done with doing all of that.
OK, so we spent a lot of time analyzing binary antipodal signals.
What about the binary non antipodal signals?
This is beautiful example of Shannon's idea of studying the simplest cases first.
If you have two signals, one of which is b and one of which is b prime, and they can be put anywhere on the real line.
And what I've done, because I didn't want to plot this whole picture again, is I just took the zero out, and I replaced the zero by the point halfway between these two points which is b plus b prime over 2.
And then we look at it and we say what happens if you have an arbitrary set of two points anywhere on the real line?
Well when I send this point the likelihood, conditional on this point being sent, is this Gaussian curve centered on b prime.
When I send b the likelihood is a Gaussian curve centered on b.
And it is in fact the same curve that we drew before.
If in fact I replaced zero with a center point between these two, which is b plus b prime over 2.
And if I then define a as this distance in here, the probability of error that I calculated before is the same as it was before.
Now I would suggest to all of you that you try to find the probability of error for this system here, just not using what we've already done, and just writing out the likelihood ratios as a function of an arbitrary b prime and an arbitrary b and finding a likelihood ratio, and calculating through all of that.
And most of you are capable of calculating through all of it.
But when you do so, you will get a god awful looking formula, which just looks totally messy.
And by looking at the formula you are not going to be able to realize that what's going on is what we see here from the picture.
And the only reason we figured out what was going like the picture is we already solved the problem for the Simpson case.
OK so it never makes any sense in this problem to look at this general case.
You always want to say the general case is just a special case of a special case.
Where you just have to define things slightly differently.
OK, so we're going to do the center point, then, I mean it might be a pilot tone.
It might be any other non information bearing signal.
In other words were sending the one bit.
Sometimes for some reason or other, you need to get synchronization.
You need to get other things in a communication system.
And for that reason, you send other things.
We'll talk about a lot of those later.
But they don't change the error probability at all.
The error probability is determined solely by the distance between these two points which we call 2a.
So probability of error remains the same in terms of this distance.
The energy per bit now changes.
The energy per bit is the energy here plus the energy here.
Which in fact is the energy in the center point plus a squared.
I mean we've done that in a number of contexts, the way to find the energy in a binary random variable is to take the energy in the center point plus the energy in the difference.
It's the same as finding the fluctuation plus the square of the mean.
It's that same underlying idea.
So any time you use non antipodal and you shift things off the mean, you can see what's going on very, very easily.
You waste energy.
I mean it might not be wasted, you might have to waste it for some reason.
But as far as a communication is concerned you're simply wasting it.
So your energy per bit changes, but your probability of error remains the same.
Because of that, you get a very clear cut idea of what it's costing you to send that pilot tone.
Because in fact what you've done is to just increase this energy, which we talk about in terms of db.
If c is equal to a in this case which, as will see, is a common thing that happens in a lot of systems, what you've lost is a factor three db.
Because you're using twice as much energy which is three db more energy, than you have to use for the pure communication.
So it's costing you three db to do whatever silly thing you want to do for synchronization or something else.
Which is why people work very hard to try to send signals which carry their own synchronization information in them.
And we will talk about that more when we get to wireless and things like that.
OK. Let's go on to real antipodal vectors in white Gaussian noise.
And again, let me point out to you again that one of the remarkable things about detection theory is once you understand detection for antipodal binary signals and Gaussian noise, everything else just follows along.
OK, so here what we're going to do is to assume that under the hypothesis H equals zero-- in other words conditional on a zero entering the communication system-- what we're going to send is not a single degree, is not something in a single degree of freedom.
But we're actually going to send a vector.
And you can think of that if you want to as sending a way form and breaking up the waveform into an orthonormal expansion and a1 to ak as being the coefficients in that expansion.
So we're going to use several degrees of freedom to send one signal.
Yes?
[INAUDIBLE]
I'm only sending one bit.
And on Wednesday I'm going to talk about what happens when we want to send multiple bits or when we want to send a large number of signals in one degree of freedom, or all those cases, multiple hypotheses.
You know what's going to happen?
It's going to turn out to be a trivial problem again.
Multiple hypotheses are no harder than just binary hypotheses.
So again, once you understand the simplest case, all Gaussian problems turn out to be solved.
Just with minor variations and minor tweaks.
OK, so I have antipodal vectors.
One vector is a1 to a sub k. Under the other hypothesis we're going to send minus a, which is the opposite vector.
So if we're dealing with two dimensional space with coordinates here and here, if I send this I'm going to send this.
If I send that I'm going to send that, and so forth.
As the opposite alternative.
The likelihood ratio is then, the probability of vector v conditional on sending this.
I'm assuming here that the noise is IID and each noise variable has mean zero and variance n0 over 2.
Namely, we're pretending we're communication people here, using and n0 over 2 here.
So the conditional density-- the likelihood of this output vector given zero-- is just this density of z shifted over by a.
So it's what we've talked about as the Gaussian density.
Just this, which is just the energy in v minus a is what that turns out to be.
So the log likelihood ratio is the ratio of this quantity to the density where H is equal to one.
And when H is equal to one, what happens is the same thing as happened before.
One makes this sign turn into a plus sign.
So when I look at the log likelihood ratio, I want to take the ratio of this quantity to the same quantity with a plus put into it.
And when I take the log of that, what happens is I get this term minus the opposite term of the opposite side.
So I have minus the norm squared of v minus a plus the norm squared of v plus a over n0.
And again, if you multiply this out, the v squareds cancel out.
The a squareds cancel out.
And you just get the inner product terms.
And strangely enough you get the same formula that you got before, almost, except here you have the inner product of v with a instead of just the product of v times a.
So in fact we just have a slight generalization of the thing that we did before.
In other words, the scalar product is a sufficient statistic.
Now what does that tell you?
It tells you how to build a detector.
OK?
It tells you when you have a vector detection problem, the thing that you want to do is to take this vector, v, that you have and form the inner product of v times a.
If in fact v is a waveform and a is a waveform, what do you do then?
Well the first thing you do is to think of v as being a vector-- where it's the vector of coefficients-- in the expansion for that waveform and a in the same way.
You look at what the inner product is then, and then you say, "well what does that correspond to when I deal with L2 waveforms?"
What's the inner product for L2 waveforms?
It's the integral of the product of the waveforms.
And how do you form the integral of the product of the waveforms?
You take this waveform here.
You turn it around and you call it a matched filter to a.
And you take the received waveform.
You pass it through the max filter for a, and you look at the output for it.
Now, let's go back and look at what all of this was doing.
And for now let's forget about the baseband to the passband business.
Let's just look at this part here because it's a little easier to see this first.
So this comes in here.
Now remember what we were saying when we studied Nyquist.
We said a neat thing to do was to use a square root of the Nyquist pulse at the transmitter.
When you use a square root of the Nyquist pulse at the transmitter, what you have is orthogonality between the pulse and all of its shifts.
Well now we don't much care about the orthogonality between the pulse and all of the shifts because we're only sending this one bit anyway.
But it sort of looks like we're going to be able to put that back in in a nice convenient way.
So we're sending this one pulse, p of t, city and what did we do in this baseband demodulator?
We passed this through another filter, q of t, which was the matched filter to p of t. What's our optimal detector for maximum likelihood?
It's to take whatever this waveform was, pass it through the matched filter.
In other words, to calculate that inner product we just talked about.
OK?
So in fact when we were looking at the Nyquist problem and worrying about inner symbol interference, in fact what we were doing was also doing the first part of an optimal MAP detector.
And at this point what comes out of here is a single number, v, which in fact now is the inner product of this waveform at this point.
With the waveform, a, that we sent.
OK?
In other words, we started out by saying, "let's suppose that what we have here is a number.
What's the optimal detector to build?"
And then we go on and say, "OK, let's suppose we look at the problem here.
What's the optimal detector to build now?"
And the optimal detector to build now at this point is this matched filter to this input waveform.
Followed by the inner product here-- which is what the match filter does for us-- followed by our binary antipodal detector again.
OK?
So by studying the problem at this point, we now understand what happens at this point.
And do I have time to show you what happens at this point?
I don't know.
Let me-- let's not do that at least right now-- let's look at the picture of this that we get when we just look at the problem when we have two dimensions.
So we're either going to transmit a vector, a, or we're going to transmit a vector, minus a.
And think of this in two dimensions.
When we transmit the vector, a, we have two dimensional noise.
We've already pointed out that two dimensional Gaussian noise has circular symmetry.
Spherical symmetry in an arbitrary number of dimension.
So what happens is you get these equal probability regions which are spreading out like when you drop a rock into a pool of water.
You see all of these things spreading out in circles.
And you then say, "OK, what's this inner product going to correspond to?"
Finding the inner product and comparing it with a threshold.
Well you can see geometrically what's going to happen here.
You're trying to do maximum likelihood.
And we already know we're supposed to calculate the inner product, so what the inner product is going to do is take whatever v that we receive-- it's going to project it on to this line between 0 and a.
So if I got a v here I'm going to project it down to here.
And then what I'm going to do is I'm going to compare the distance from here to there with the distance from here to there.
Which says first project, then do the old decision in a one dimensional way.
Now geometrically, this distance squared is equal to this distance squared plus this distance squared.
And this distance squared is equal to the same distance squared plus this distance squared.
So whatever you decide to do in terms of these distances, you will also decide to do in terms of these distances.
Which also means that the maximum likelihood regions that you're going to develop, or in fact the maximum a posteriori probability regions are simply planes.
Which are perpendicular to the line between minus a and plus a. OK?
So if you're doing maximum likelihood you just form a plane halfway between these two points.
Yeah?
[UNINTELLIGIBLE]
We got the error probability just by first doing the projection and then turning it into this scale or problem again.
So in fact the error probability-- What?
[UNINTELLIGIBLE]
The probability of error is just the probability of error in the projection.
Did I write it down someplace?
Oh yeah, I did write it down.
But I wrote it down, well I sort of cheated.
It's in the notes.
I mean the likelihood ratio is just a center product here which is a number.
And when you find the error probability, you just use the same q formula that we used before.
And in place of a you substitute-- in place of a you substitute the inner product of v with a.
Which is the corresponding quantity.
So it's q of 4va divided by n0.
OK?
So that's the maximum likelihood error probability.
OK?
In other words, nothing new has happened here.
You just go through the match filter and then you do this same one dimensional problem that we've already figured out how to do.
I think I'm going to stop there and we'll do the complex case which really corresponds to what happens after baseband to passband then passband to baseband.
The following content is provided under a Creative Commons license.
Your support will help MIT OpeCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, we talked about this a little bit last time.
We were talking about the detection of vectors in white Gaussian noise.
When we talk about vectors we'll often refer to white Gaussian noise as noise where each component of the vector is independent of each other when they're all the same variance.
Usually we take the variance to be n0 over 2, capital N0 over 2.
And we'll say more about that as we go on, but that's just-- I guess one thing I ought to say about it now-- people keep wondering why we call this N0 over 2 instead of something else.
As I said before, there's really no reason for it except custom.
The one important thing that you can always remember and which is always true is that any time you're talking about a sequence of real noise variables they all have variants N0 over 2, and the same coordinate system that you're using to measure the signals.
OK, the only thing that ever appears in any of these formulas is a ratio of signal power or signal energy to noise signal power.
And if we're up it passband, we're dealing with the power, which is two times larger than that at baseband.
And because of that-- and this is what really gets confusing-- is that when you talk about N0 over 2 at passband, you are talking about something which is twice as big as the N0 over 2, the same N0 over 2 you were talking about at baseband.
And the reason is since the signal is twice as big there, the noise is also said to be twice as big.
I can't do anything about that.
It's just the way that everybody does things.
The other thing that everybody does-- since everyone gets confused about that-- is after they get all done dealing with anything in a paper they're writing or something, they always look at the signal to noise ratio that they have and they remove all the factors of two that they know shouldn't be there.
So that you shouldn't trust anything in the literature too much as far as factors of two are concerned.
And I try to be careful in the notes about that, but you shouldn't trust the notes too far along those lines, either.
So well eventually I'll get the notes straightened out on all of that but I think they're pretty close now.
But anyway, we were looking at this question of how do you detect antipodal vectors in white Gaussian noise?
And the picture that we can draw is this.
Namely we have two signals.
One is the vector, a, one is the vector, minus a.
It's in some finite dimensional system, but we're viewing it as far as drawing a picture as a two dimensional system.
So a has some arbitrary component in the first direction.
Some arbitrary component in the second direction.
Minus a is, of course, the reverse of that.
This point right in here is the zero point, which is halfway in between them.
And the output that we observe is either plus or minus a, plus this independent zero mean noise, Z, which has the kind of circular symmetry indicated here with these little circles.
Each of the Z sub i have the same variance, they're independent of each other.
And when you write down the likelihood of the probability density of this output, given that the hypothesis was zero.
Namely that plus a was the signal which was chosen.
OK, remember in all of these things there's this process now going on that we usually don't talk about anymore.
But there's an input coming into the communication channel which we're now calling capital H. It's the hypothesis-- which is the thing you're trying to detect when you're all done-- that input which is one up to capital N, or sometimes zero up to capital N minus 1.
Is then mapped into a signal from a single set of capital N different signals.
So they're impossible signals in this signal alphabet.
You map the hypothesis into one of those.
From those, you generally form a waveform.
This waveform might be modulated up to high frequency, back to low frequency again.
Detected or whatever.
You got some vector, v, at that point.
Which is a sequence of samples that you're going to be taking as far as most cases are concerned.
We'll talk more about that later today.
But anyway, v is a vector which is plus or minus a at this point, plus this Gaussian noise.
We can write down the probability density of that vector, v, which is if hypothesis zero occurs.
Namely if a zero enters the communication channel, plus a is the signal which is chosen.
Then what happens is that the output is a plus Z.
Which means that the probability density of the noise is v minus a.
So we have this probability density here.
When we look at the log likelihood ratio, we're taking the logarithm of this probability density divided by the probability density of the alternative hypothesis.
Namely v given one.
Which is the same as this formula except there's the plus a there instead of a minus a.
So you get this thing here.
Now, why I wanted to talk about this today is we're going to talk about the complex case also and something very, very peculiar and funny happens there.
OK so the log likelihood ratio is the scaled difference of the energy of the distance between v and a.
Which is this term here.
This is just a squared distance between the vector, v, and the vector, a.
It's v minus a is that distance there, squared.
And the other term here is the term that comes from the probability density of v given one.
Which turns out to be that distance there squared.
So you have the difference between these two things.
This is just the inner product of v minus a with v minus a.
And if you multiply that all out it's the inner product v with itself, plus the inner product of a with itself, minus the inner product of v and a, minus the inner product of a with v. The only things that don't cancel out between these two things is the inner product of v with a, the inner product of a with v, the inner product of v with a, the inner product of a with v. So those things last because there's a minus sign here and a plus sign here.
There's a minus sign here and a plus sign here.
So the plus and minus signs cancel out, so you just get four of these terms here, which is four times the inner product over n0.
What happens to that geometrically?
What is the inner product of v and a?
Well it's the projection of v on the vector a.
Which happens to be the line between minus a and plus a.
Mainly the fact that it's the line between minus a and plus a is the thing which is valuable whether you're dealing with antipodal communication or any other kind of communication.
You're always looking at this line between two points.
So what this thing says is you form the inner product, which says drop a perpendicular from Z down to here, and in terms of where that perpendicular lands here, you make your decision.
Namely you compare that with the threshold.
OK?
So we have two different ways of doing this.
One of them is compare this distance with this distance.
Or actually here you compare the square distance here with the square distance there.
Which says, if you're using maximum likelihood then the threshold that you're dealing with here is zero and you simply make your decision on whether the log likelihood ratio is positive or minus.
Which means in terms of the projection theorem here, what you're doing is taking a perpendicular bisector of the line between minus a and plus a, and putting a plane there and that's the plane that separates what goes into one and what goes into zero.
This goes into zero.
This goes into one.
OK, so is it clear to all of you that this is really saying the same thing as this?
This inner product just comes from here.
You can either look at this as minimum distance decoding.
You just find the point which is closest to what you receive.
You find the hypothesized signal, which is closest to the actual observation that you make.
You make your decision in terms of that.
Or you do this projection and make your decision in terms of that.
And if you use a triangle thing which says that this squared distance plus this squared distance is equal to that squared distance.
We all remember that from third grade or something.
I don't know when.
But that simply says the same thing that this says.
This just says it more generally.
In terms of an arbitrary finite conventional vector, rather than just the case of where you're looking at two dimensions.
OK that's-- we will probably come back to look at that in a little bit, but now I want to look at complex antipodal vectors in white Gaussian noise.
So the set up there is that the input is some vector.
I usually use u's to mean complex numbers.
So the vector there is u1 up to u sub j where u sub j is a complex number.
So we're dealing with complex vectors at this point.
So we have two points, minus a -- minus u and plus u.
Where instead of being a real vector they're complex vectors.
And if you can't visualize things geometrically, in terms of complex vectors, join the crew.
I can't either.
The only thing I can never do is talk about real vectors, try to get some idea of what's going on from that, and use mathematics for the complex vectors.
Because the reason we use complex vectors is that, analytically they're just as simple as real vectors.
The reason we use real vectors is because we can draw pictures of them.
I defy anybody to draw a picture in four dimensions.
Some books will do it, but I can't understand them.
And anyway.
OK, so under hypothesis zero what gets sent as u?
And under hypothesis one, what gets sent as minus u?
So we're still talking about binary communication and if you're talking about antipodal vectors, you can't do much other than talk about binary communication.
Because if you're sending a, the only vector antipodal to a is minus a.
And then you're stuck and you're done.
So we're still talking about binary vectors.
Remember the reason why we're doing this-- because one of the things we did last time, one of the things that's in the notes and stressed again and again is that once you understand the antipodal case, you can translate those two points anywhere you want to and the maximum likelihood decision and the MAP decision are still the same thing.
You take those two points and you just translate them in space until the mean between them sits on the zero point.
And then you're back to the antipodal case again.
OK, so the reason why we're doing this is really so we can look at the more general case.
But we don't have to have that mean sitting around all the time.
OK, Z then is going to be a vector of j complex IID Gaussian random variables.
IID real and imaginary parts.
Namely the real part of each Gaussian, complex Gaussian random variable has variance n0 over 2 and the imaginary part also has variance n0 over 2.
These complex vectors, if you look at the probability density for them and you draw it in two dimensions, one for the real part one for the imaginary part, you get this circular symmetry that we've always associated.
Those are supposed to be circles and not ellipses.
And those are sometimes called proper complex Gaussian random variables.
Because almost everywhere where you see complex Gaussian random variables, the real and imaginary parts are independent of each other and both have the same variance.
Again, when you look at formulas in papers, formulas everywhere else, they are almost always assuming this kind of circular symmetry.
Or what's often called proper complex random variables.
Sort of accepting the fact that anything else is very, very improper.
It's improper because formulas don't work in that case.
OK, so we have a vector of these complex IID Gaussian random variables.
Under H equals zero, the observation, v, is given by v equals u plus Z.
And under hypothesis one, the observation is given by minus u plus Z.
So I have exactly the same cases as we had before.
OK in other words almost all formulas stay the same when you go from real to complex.
But I don't trust the complex case and you shouldn't either until you at least go through it once.
So what I'm going to do now is translate this complex case to the real case.
In other words, for each complex variable I'm going to make two real variables.
One the real part and one the imaginary part.
We know that the real part and the imaginary part are independent of each other.
And if these Gaussian random variables are independent of each other, then the real and imaginary parts of each of them are independent of the real and imaginary parts of each of the other side.
OK?
So we can go from j Gaussian random variables, independant Gaussian random variables, which are complex.
To 2j Gaussian random variables, which at this point are going to be real.
OK so it's just a translation from j complex variables to 2j real variables.
Again we can't draw pictures of things in this j dimension, we can start to draw pictures in 2j dimensions.
If you talk about a probability density for a complex random variable, what are you talking about?
How do you write the probability density for just a plane Gaussian complex random variable?
What is it?
Anybody know what it is?
Is it one dimensional or is it two dimensional?
What does probability density mean?
It means probability per unit area.
What does area mean when you're talking about complex numbers?
Well you sort of mean what you've drawn there, yes.
And you're looking at areas in this complex plane here.
So that in fact when you write the probability density for a complex random variable, what you have already done whether you want to do it or not is you've converted the problem to real and imaginary part.
That's what the probability densities are.
Excuse me for a belaboring this but, if I don't belabor it I mean there's a catch here that comes along in a little bit.
And you won't understand to catch if you don't understand why these things are almost the same as real variables up until the catch comes.
OK, so we're going to deal with these 2j dimensional real vectors.
The components will be real part of u j, imaginary part of u j for what goes into the channel.
And we'll let capital Y capital Z prime be the two j dimensional real versions of V and Z. OK so that we'll call Y the real part, an imaginary part of Z.
And you notice that what's going on here is the same thing that's going on when you modulate QAM.
You take a complex signal, you multiply it by either the 2pi j carrier frequency times t, and then we started to look at orthonormal expansions, you remember that what we looked at-- as far as the real signals that were actually being transmitted on the channel-- was the real part of that u of t times z to the blah, blah, blah, and the imaginary part of u of t times blah, blah, blah.
So you've got two orthonormal functions in place of one complex orthonormal function.
And that's the same thing that's going on here.
It's just not with immodulation put in, it's just dealing with the real parts and imaginary parts directly.
OK, so if we do that we get a bunch of equations.
They're sort of familiar looking equations by now I hope.
This is just the probability density of this real 2j dimensional random variable.
Which is all this junk that we're used to seeing.
We can collapse that into e to the minus the norm squared of y minus a.
This is the norm squared in this 2j dimensional real space.
It's not the norm squared in this complex space.
What gets confusing is that those two norms are exactly the same.
As we'll see in just a few minutes.
But anyway, what we're dealing with now is this norm in real space.
OK, note that's y-- oh let me translate this one for you.
If we think of this v that we received j complex random variables as being: real part of v1, imaginary part of v1; real part of v2, imaginary part of v2; and so forth, then y2j minus 1 minus a2j minus one.
I can't ever get these formulas right.
That should be a2j minus 1.
There.
This squared, plus this squared-- OK, in other words the real part squared of the difference plus the imaginary part squared of the difference-- is really just the same is vj minus uj squared.
OK, in other words you take the complex number v sub j, you subtract off the real number u sub j.
And the way to do that, you visualize this one complex variable in the complex plane, and what you're doing is you're taking the square of the real part of the distance, you're adding it to the square of the imaginary part of the distance.
OK, all of this is stuff you learned in high school.
Just viewed in a slightly different way.
OK, now when you take the probability density with respect to these complex vectors-- which is what I want to get at-- probability density of these complex variables really means the same thing as that with the real variables.
But then you wind up with the magnitude of vj minus uj squared.
And this term is really exactly the same as these two terms there.
So when you take this probability density, you wind up with these terms the same and with these terms the same.
OK, in other words the complex norm squared is the same as the real norm squared, when you go from complex to real and imaginary parts of, well-- as I said, this is the same as that.
OK so when we look at the log likelihood ratio, then, let's do the log likelihood ratio in terms of the real parts first.
We get the difference between this norm squared of y minus a, and the norm squared of y plus a.
This is the part that comes from the hypothesis zero, this is the part that comes from the hypothesis one.
OK, so we get these two terms here.
When we take the inner products here, we get the same thing that we got before.
Four times the inner product of yna.
Now, the whole reason for going through all of this is this next formula.
When you do this, you wind up with this very bizarre four times the real part of the inner product of v and u, divided by N0.
And you get it in exactly the same way that we got it before.
Namely you take this norm here, which is an inner product squared of v minus u.
Let me write it out.
I'll write it out here.
Norm squared of y minus a, is the norm squared of y plus the norm squared of a, plus the inner product of minus ya, plus the inner product of minus ay.
What's the sum of these two inner products?
This inner product, by definition in terms of integrals, is the integral of minus y times a-- complex conjugate.
This is minus a times y-- complex conjugate.
So in one case the complex conjugate is here.
In the other case it's on the other term.
In other words this and this are complex conjugates of each other.
What happens when you add two complex conjugates?
You get the real part of the two of them.
Okay so when you add these two things you just get that real part there.
OK, and then when you do the other term, the same thing happens.
The same cancellation that we had before occurs.
And these two inner products add up so you wind up with the real part of vu over N0 When we look at the picture here, what does it mean?
Well I suggest you first look at the one dimensional case here.
Namely on this one dimensional case think of V as being a one dimensional complex random variable.
Then we can draw a picture.
The picture make sense.
And what we're dealing with is the real and imaginary parts, and these distances here, when you talk about the norm of V minus a-- namely what corresponds to this line here, the length of this line-- what do you really mean by it?
If you took the inner product, if you took the norm of v, with i times a-- namely that the square root of minus 1 times a-- would you get the same thing or wouldn't you?
If I take a complex number, and I take the inner product of that complex number-- namely the product-- of that, when I take the inner product of ya, is this the same as the inner product of y and i times a?
Not at all.
The two things are totally different.
Namely, inner products are complex things.
Norms are real things.
And these norms, when you're dealing with complex numbers, have real parts in them.
In other words, this distance that we're talking about here is not just the norm squared-- well it is the norm squared-- because the norm has this complex feature built into it.
Because people kept making that mistake all the time.
So they fudged the mathematics to make it come out right.
But after doing that, you have to fudge the mathematics to come back to something that makes sense here.
So you have the real part of this inner product, here.
So in fact, what you're doing when you're taking the inner product of two vectors and you're trying to relate it to this plane here, this separation plane, is you have to look at that separation plane.
You have to look at that projection in terms of real numbers.
Namely, you have to look at the projection.
First as being a complex projection of v onto a, which gives you a complex number.
And then after you do that you have to you visualize yourself in a two dimensional real space.
And you have to project once more from the two dimensional thing to the one dimensional thing.
And here where we're just looking at one dimension to start with, we have to draw it as a two dimensional space.
And suddenly we are dealing with this real part there while we're not dealing with that here.
Which is why when people say minimum distance detection when they're dealing with complex numbers it sounds very, very simple.
But in fact, it's not so simple.
If you view this in the complex plane, is this a linear operation or isn't it?
When you're looking at things as complex vectors.
Is this thing a sub space of the complex vector space?
No, it's not.
It's not a sub space.
Because, to be a sub space you have to be able to multiply by arbitrary scalors-- which includes complex numbers-- and stay in the same space.
And here the complex numbers are important.
OK?
You should go back and think about that.
You will be confused about it for the first ten times you think about it.
For those of you who stick with it and carry through on it, you'll be happy because you'll never be confused about it again.
OK, anyway thats real numbers there.
And the most straightforward way to deal with complex noise is to first turn it into real noise.
If you do that you never get confused.
And otherwise you only have half the analytical work.
You only have half the writing to do.
But you never know whether you've done the right thing until you go back and check.
OK the probability of error for maximum likelihood detection, in other words where the threshold for the log likelihood ratio is zero, is simply the same thing that it was before.
Namely it's the q function.
This tale of the Gaussian normal function.
Of the norm of a divided by the square root of N0 over 2.
In other words it's the length of a divided by the by the length of a one standard deviation of the noise.
When you put that in terms of, well, if you write this as the square root of the norm squared then you get this formula here.
Because e sub b is just the energy of these antipodal signals.
When you look at this in terms of the complex random variables, you get the same thing.
OK?
You get u instead of a because, in fact, in the complex plane and the real plane, distances turn out to be the same.
But again, in all cases it's square root of 2eb over n0 Now, that is true for any vector at all where these norms are appropriate.
When we start dealing with functions what happens?
When we start dealing with functions, what we're going to do is we're going to take this vector, we're going to turn it into a wave form.
We're going to transmit the wave form.
The wave form is going to come back to us.
We're going to demodulate it, get back to a number again.
And that's the next thing that I want to talk about.
Because what I want to convince you of is the property of white Gaussian noise which is so important.
Is that it doesn't make any difference how you modulate.
All modulation systems are the same.
All modulation schemes are the same.
All frequencies are the same.
There is no way you can avoid white Gaussian noise.
There is no way you can get screwed by it.
No matter what you do, the same thing happens.
You can always take all these problems where you're dealing with wave forms.
You can convert them to finite dimensional vector problems.
And when you convert it into a finite dimensional vector problem all of the orthonormal functions that you're using all pass away.
Because none of them are relevant anymore.
OK?
That's the bottom line of all of that.
OK. We haven't really talked about M-ARY hypothesis testing.
So I want to talk about it a little bit now.
I talked about it just a shade, but not much.
When we want to detect between m different hypothesis-- namely in the vector case we're going to now be detecting not between antipodal signals but m signals which are placed any place at all.
We already said what the MAP, optimal MAP test was.
You see an observation, you're trying to guess what the input was, or what the hypothesis was.
And in general, the MAP test says try to find that j, namely that hypothesis, for which the a priori probability of hypothesis j, times the likelihood-- namely the probability that you see v given h of v, given j is maximum.
OK, this is just standard formula for finding a posteriori probabilities.
Where you factor out the marginal on, when you cancel out the marginal on v. In other words, what this rule says is to do MAP testing, what you do is you find the a posteriori probabilities of each of the hypotheses and you choose the a posteriori probability which is largest.
Perfect common sense.
The way we're going to do that, the way which is particularly convenient, is you do it the same way that we've been doing all along for binary hypotheses.
The way to do a MAP test, at least one way to do a MAP test, is you compare every pair of hypothesis and you choose the most likely of each pair.
And when you've got it all done, you have a winner.
OK?
Mainly you can always compare any objects if they're comparable.
And after you compare each pair, you take the one which beats every other one and that's the winner.
OK?
So what you do is you do a pairwise test between each hypothesis.
Namely, the likelihood ratio of m relative to m prime.
Is the likelihood of the output conditional on m, divided by the likelihood of the output conditional on m prime.
You compare it with the a priori probabilities, and the point here is that nothing really has been added.
You have the same problem you had before, OK?
Nothing new.
It's just gotten n square times as complicated.
The computation is free now, so you have exactly the same problem that you had before.
If you have to write it out on paper, yeah it's much more complicated.
But conceptually, it's not.
You have to remember that the signals are not antipodal here.
But what we're dealing with mostly at this point is this Gaussian noise case.
And here, what you observe, is signal plus noise.
And Z is zero-mean jointly Gauss and s is discrete with n possible values.
OK, so let's see what that means.
Here's a picture of it.
If you have three singles which are each two dimensional vectors, suppose one of them is s0, suppose one of them is s1, suppose one of them s2.
OK?
And now you want to pairwise, see which one is most likely.
And let's think of doing this first for the maximum likelihood case.
What do you do?
You set up a perpendicular bisector between s0 and s1.
That's this line here.
And if you weren't to worry about s2, everything on this side would go into H equals zero.
And everything on this side would go into H equals one.
Namely, whatever's closest to this point gets mapped into it.
Whatever's closest to this point gets mapped into it.
If you're doing MAP testing, what happens?
In the test between this and this you had the same orientation for this line, but it just gets shifted a little bit this way or a little bit this way.
OK?
Then you compare this with this and you get this line here.
Same argument as before.
It's just comparing two things are not antipodal they've just been shifted off from the origin a little bit.
But for the maximum likelihood you still take the perpendicular bisector between them.
And then you compare these two and you got a perpendicular bisector between those.
And these perpendicular bisectors, in two dimensions, always come together at one point.
And I don't know why.
And if you looked at it often enough, you probably know why.
And you could probably prove it in about ten minutes.
But in fact these things always come together somehow.
If you do the MAP test, they always come together also.
You can shift each of them in arbitrary ways and somehow they always come together in this point.
OK, the separators between decision regions here are the set of points where the real part of the inner product, vu, is constant.
OK?
Again, for dealing with complex vectors, you got to both do the projection and then do the projection again onto the real part of this projection.
So it's sort of a two way projection.
Because probability densities in j dimensional complex space are really 2j dimensional quantities.
And when you're comparing them, you really have to compare things in terms of that 2j dimensional probability density.
OK so that's why that comes out.
These are best visualized in separate, real, and imaginary coordinates.
And for maximum likelihood detection, the regions are Voronoi regions.
OK?
We talked about Voronoi regions in terms of doing quantization.
And we found out if you wanted to minimize the mean square error, what you did was you set up regions, which are perpendicular bisectors between all the points.
And here you get the same perpendicular bisectors between the points.
And everybody-- because of that-- thinks that quantization has a great deal to do with error probability when you have large sets of signals.
And it probably has something to do with it, but I don't know what other than the fact that you've got Voronoi regions in each case.
Which is what you get.
OK, so that's where we are with both complex vectors and real vectors.
I want to now just restrict attention to real wave forms so I don't have to keep going back and forth between the real and imaginary case.
If you're thinking in terms of QAM, we're now thinking in terms of what goes on at passband.
Why do we want to think of what goes on at passband?
Because that's where the noise hits us.
And in a fundamental sense, all of the stuff about QAM really isn't fundamental.
I mean, it's all done-- all this stuff down at passband-- is all done because people thought it was easier to implement things there.
It's the only reason for all of that mess with dealing with all of these complex signals.
If we really want to deal with the problem in a fundamental way, what we want to do is to choose a signal set up at passband.
Do detection up at passband.
And then after we find out what the optimal detection is up at passband, see if we can actually implement that down at baseband.
So the fundamental problem is looking at single sets up at passband and analyze what they all mean.
OK, so we're going to generalize both PAM and QAM.
And now we're going to look at the general problem where what we're dealing with is a single set.
Which is m signals.
Each of them we're going to visualize as a vector in j dimensional space.
m different signals, j dimensional space.
Don't confuse the dimension of space with the number of signals.
OK?
You can have an arbitrarily large dimensional space and just binary signals.
Or you can have an arbitrarily large set of signals and you can be dealing with it.
In PAM, for example, it's just all done in one dimension.
So j there is equal to 1.
QAM j is equal to 2.
We now want to look at more general things.
Partly because we want to look at orthoginal wave forms.
We want to look at orthoginal wave forms for two reasons.
One is that we would like to show that by using orthoginal wave forms, you can reach what we've called the capacity of a white Gaussian noise channel.
And two, when we get to studying wireless it's very, very useful to base signal sets on orthonormal sets of functions.
And we'll see why each of those things happen as we move on.
OK so we're going to denote the single set as the set of vectors, a1 up to a sub m. And in that signal set, we will denote the vector, a sub m, as a j dimensional vector, a sub m1, a sub m2, up to a sub m capital j.
So j is at the dimension.
m is just a component of these vectors.
I'm going to create a set of capital J orthonormal wave forms.
They can be anything at all.
I don't care what they are.
I'm going to use those orthonormal wave forms in order to modulate the signal-- which is now a vector-- up to some waveform.
This is really the standard way we've been turning signals into waveforms all along.
It's just that now we're looking at the general case instead of all of these specific cases that we've been looking at.
All of the special cases all fit into this same category.
So we have these capital M different waveforms.
And we're going to transmit one of them and then at the receiver we're going to try to decide which one was sent.
OK, well one of the reasons why I'm going through all of this generality is that there's an issue we haven't talked about yet.
All of the stuff we've done on detection so far we have had one hypothesis.
Could be M-ary could be binary.
We have sent something.
We have received something.
We have made a detection.
OK?
In other words, for all of the antipodal stuff we've done, we built a communication system.
We set it all up.
We transmitted one bit.
We've received the one bit.
We've made a decision on it.
Then we've torn down the communication system and we've gone home.
You really want to transmit a whole sequence of symbols or signals or waveforms.
So we want to deal with that now.
So we need some way to transmit a succession of M-ary signals.
And we'll call this succession of signals-- mainly the signals are the things that get chosen from the signal set-- we'll call them x of k, k of z.
Which is what we've been calling them all along.
We've been transmitting a sequence of things when we're dealing with PAM.
And aa in am.
Why do I call them xk instead of ak?
Well I can't call them ak because when I talk about ak I'm talking about the k'th signal in the signal set.
And here what I'm talking about now is transmitting a sequence of choices.
Each one of these choices, the first choice is a choice from this set here.
The second thing that I send is a choice from this set.
The third thing that I send is a choice from this set.
So x1, x2, x3, and x4 and so forth are different choices among these M-ary signals.
If m is 2 to the 6-- OK in other words, every time I transmit a signal I'm transmitting six bits.
OK.
In a communication system we transmit six bits.
Then we transmit another six bits.
Then we transmit another six bits, and so on forever.
OK, so I need to talk about these things as ways of a succession of signals.
The thing that I'm trying to get at is, how do you know when you send one of these signals that you don't have to worry about the other signals?
How do you know that they don't interfere with each other?
Well, we sort of solved the problem of them interfering with each other in dealing with Nyquist, but we haven't dealt with that problem at all since we started to talk about random processes.
So we don't know whether we've really solved it or not.
So at this point we have to solve that problem, and that's what we're aiming at.
So, the one way to be able to transmit a whole sequence of signals is to have these choices of vectors here and to develop a set of orthonormal waveforms, v1 up to v sub j, which all have the property that if you time shift them each by capital T, they're stilll-- if you time shift them by capital T, they have to be orthonormal to each other.
The question you're facing is whether these things that you're transmitting are orthoganol to all of these things that you're transmitting.
Now, in the Nyquist problem, we dealt with the problem of how do you take one waveform here and make it orthonormal to all of its time shifts.
And we solved that problem.
In the quiz you solved the problem-- although you probably didn't recognize it-- of dealing with orthonormal functions both in time and in frequency.
And that's the kind of thing we would like to use here.
If I take a set of orthonormal pulses and then I modulate those orthonormal pulses up to a higher frequency-- which is out of the range of this first frequency-- then I can send one sequence of orthonormal functions down here, another set of orthonormal functions up here in a different frequency range.
Another one up here in a different frequency range and so forth.
So then all of these orthonormal functions are going to be orthonormal to each other.
Yeah?
Are you saying that each x of k is its own frequency?
Because each x of k is infinitely long
Each x of k is going to-- each x of k is just a vector of j components.
I'm going to modulate that vector, x of k, into a waveform, x of t, which might be finite duration or it might be infinite duration.
I mean, it's going to go to zero very, very fast, anyway.
And whether it is absolutely time limited or not is something I don't really care about at this point.
But the point is I can create functions where, in fact, I have a whole sequence of functions here and they're orthonormal to all of the shifts.
One way of doing this, for example, is to make capital J a bunch of little time shifts on functions.
I can pick a function p of t, which is orthonormal to all of its shifts, in terms of t1.
I can send J of those pulses to take care of x of k. And then I can use a capital T in here, which is j times this little t that I was using.
And I can send another set of functions.
So I can do that, I can move up and down in frequency.
I can choose any old set of orthonormal functions that I want to.
But the thing that I want to do is I want to make sure that for each vector, x of k, that I'm sending in time when I modulate it to a waveform, that waveform is orthonormal to the waveforms for every other k. And there are lots of ways of doing that.
OK, mainly there are lots of choices of orthonormal functions.
OK so anyway what I'm going to be doing is making all of these signals orthogonal to each other.
OK, so the transmitting waveform for this sequence of modulated signals is x of t, which is the sum of x of k times t minus kt.
Mainly the same thing we were doing before.
Except now I have also the problem that each of these waveforms, x of k of t, has to be some sum of orthonormal functions.
So the problem becomes a little more difficult than what it was before.
But in fact it's-- I mean this is just standard communication.
Every wireless system in the world uses this kind of scheme.
They don't use QAM or PAM.
They use something much more like this.
OK, so now our problem is you want to detect a generic x from this sequence.
OK, in other words, one of these x sub k in sequence, we want to be able to detect what signal was sent.
We want to detect which hypothesis chose a signal which was then formed into a waveform, x of k of t. And if I can do this for one k, I can do it for all of them.
So I want to solve the problem for one generic value of k. OK, how is this problem different from what I was looking at before?
Before I was looking at the problem where we built a communication system, we tuned it all up, we sent one bit.
We detected it, we tore it all down and we went home.
Now what we're doing is we're building the communication system.
We're tuning at all up.
We're sending a sequence of bits.
And then all I'm interested in at the moment is detecting the k'th of them.
But if I find a way to detect the k'th of them, I can then use it for every k. OK?
So I'm going to build a detector which is going to detect, in some optimal way, each one of these signals that gets sent.
OK?
Is it clear how the problem is different?
Mainly I have to deal with the fact that these other signals are floating around there.
And that's my problem.
Ok, so the input to the channel is hypothesis H. That takes values one up the m. The symbol, m, is mapped into the signal, vector a sub m, it's modulated into x of t equals summation over j, a sub mj, phi j of t. OK, this waveform, now, is a function of which particular signal I'm sending.
Which is a function of which particular hypothesis entered the encoder.
The trouble with this material is all the complication comes in this awful notation, which you can't avoid.
Because you're dealing with sequences, you're dealing with vectors, and you're dealing with wave forms all at the same time.
What's going on, after you understand it, you'll say why was it so difficult to understand this?
Because eventually when you see it, it becomes very, very simple And I understand why there's just too much stuff all going on at the same time.
OK, so what I'm going to do now is I'm going to take these J, capital J, orthonormal waveforms.
And we've already seen that you can start out with any old orthonormal waveforms and if you want to you can extend that set of waveforms into an orthonormal set that spans all of L2.
OK?
So I'm going to imagine that we've done that.
It's taken us a long time, but we've done it.
We're all through with it.
We have this orthonormal set now.
If I'm smart, that orthonormal set, which I generated, will also include easy ways to represent each of the other signals that we're going to send.
But I don't care about that right now.
All I'm dealing with is this one hypothesis that came in.
This one signal, a sub m-- oh, the hypothesis m, the signal a sub m, an the particular time instant, k, and this waveform that gets sent, which can be represented as the first J terms in this orthonormal sequence.
OK, so what I'm going to get then is the received random process.
Is going to be a sum -- and forgot about the j prime now-- I can represent it as a sum of coefficients times these orthonormal waveforms.
OK, that's what we've done for arbitrary sequences, and then we've said we can also do it for at least well defined random processes.
I'm going to make, I mean, instead of making this an infinite dimensional sum, I want to make it a finite dimensional sum where J prime is very, very large.
Say, 10 to the fiftieth if you want to.
I don't want to make it infinite, I want to look at what happens when I let it get bigger or smaller.
So I'm expanding Y of t over an orthonormal expansion.
But I'm not going all the way.
I'm just going to try to do maximum likelihood detection with this finite set of observations.
So I wont do quite as well as if I have all the observations, but I'll still do pretty well.
We hope.
OK, well so Y sub j-- the output that I see in this degree of freedom corresponding to phi sub j of t. Is going to be xj-- what I sent in that degree of freedom-- plus zj.
And there are j degrees of-- capital J degrees of freedom that I'm using.
So the outputs in those degrees of freedom, namely in the phi1 of t, phi2 of t, phi3 of t directions in this L2 space are going to be the signal plus the noise.
For all of these dimensions.
And Yj is just going to be equal to zj for all the other terms.
OK, now I want to add one extra thing here.
What I should be putting in here is all the other signals that are going to be transmitted.
I don't know how to do that.
Notationally it gets very confusing.
So what I'm going to say is, OK Z sub j here is not just Gaussian noise.
Z sub j is Gaussian noise plus all the signals from other time instance that we're sending.
Plus all of the signals that anybody else is sending.
If we're dealing with wireless then we have interference from them.
Plus any old other thing you can think of.
z sub j is everything but in these other degrees of freedom.
In these other coordinates.
This solves another problem for us, because when we defined white Gaussian noise we had this problem.
That we could only say it looked white over the region of interest.
We could only say it looked white over some time span.
Because the earth keeps changing, you know.
And over some frequency span because different frequencies behave in different ways.
So this also allows us to have different Gaussian random variables here.
So when we have arbitrary random variables here, they can be Gaussian or non-Gaussian.
They don't have to have the same variance.
They don't have to have anything.
What I am going to assume is that these out of band, out of sense, out of view random variables are all independent of the things that I am looking at.
And for white Gaussian noise, that's true.
All of these random variables here are independent of all of these random variables here.
For these first capital J different random variables that I'm interested in.
And all these random variables are independent of they input that I'm using.
OK?
In other words, a hypothesis came into the transmitter that generated a signal.
The signal got turned into a waveform, which is defined solely in terms of these J degrees of freedom, these J orthonormal functions.
And now everything everywhere else is independent of these J functions that I'm interested in.
Why is that a shaky assumption?
Anybody think of a situation where that is absolute nonsense?
Yeah?
The stuff from the other message had t--
Mm hmm
You said t of j is not just Gaussian noise--
It also includes all those other signals, yes.
Well, but I want to assume that those other signals are independent of this particular signal that I'm sending.
But in fact that is making a pretty big assumption.
Because one of the things that a lot of people like to do is, when these bits come into a channel, the first thing they do is they encode the bits for error correction.
And then they take those bits that come out of the error correction device, error encoding device, which are as correlated as could be.
And they're all statistically very dependent, because we want to use that statistical dependence later to correct errors.
And this assumption that I'm making here says, "no that's not the case.
I'm assuming that all that other stuff is independent of what I'm transmitting here."
So I'm very specifically assuming at this point that all of that stuff has not been encoded first.
That I'm sending something which is independent of everything else.
Which is going to enter this channel.
We'll just assume that and after we get done assuming it and seeing what the consequence of it is, we'll go back and see what it all means.
OK, so for a little more notation.
I'm going to call the vector, Y, the first J, capital J, outputs.
I'm going to call the vector Y prime all the other outputs.
Now intuitively what were aiming at is, we would like to say this stuff doesn't have anything to do with it.
We're going to base our decision on this.
But I want to prove that to you, and show you why it works and why it doesn't.
And the noise I'm going to break up the same way.
Z is this the first J components of the noise.
And Z prime is the other components of noise.
So what I have is that the output, the output that I want to look at, namely the output this vector of dimension J output.
Which is equal to a vector of dimension J input plus of a vector of dimension J noise is equal to-- well Y is equal X plus Z.
And the out of band stuff, the output, is just these noise and other signals.
OK?
And I want to assume that Z prime, X, and Z are statistically independent.
Question, test your probability.
If I assume that Z prime is independent of Z, and if I assume that Z prime is independent of X, does that mean that Z prime is independant of X and Z?
If a is independent of b, and a is independent of c, is a necessarily independent of the pair bc?
How many think that's true?
Better go back and study a little elementary probability again.
And the notes are occasionally wrong about that, too.
So you shouldn't feel, you shouldn't feel badly about it.
No, the problem is you really need this joint independence between all three of them.
I could, for example, make X plus Y be equal to Z. I could do this with discrete random variables.
Which are equally probably zero and one.
And make the plus equal to a mod 2 operation.
And if I did that, each pair would be independent of each other, and the triple would be very, very highly dependant.
So anyway, I want to assume that Z prime, X, and Z are statistically independent.
In other words, what I'm doing is saying, "If I assume that, what's the consequence of it?"
OK, so the likelihood then, the probability density of the output, Y-- this is the output in the first j dimensions given a particular hypothesis Y-- is equal to the probability density of the noise evaluated at Y minus am.
Where this is the signal that goes with this hypothesis, well with am.
Times the probability density of Y prime for Z prime.
OK?
And I don't even have to assume that this is Gaussian.
All I've done is to assume that these random variables are independent of these random variables.
And therefore this probability density is multiplied by this probability density.
OK, well that's kind of neat.
Because if I put a different i in here in place of m, I get this thing.
Changes all around.
F sub Z of Y minus a sub i.
But this stuff, which is out of band, doesn't change at all.
When I form the likelihood ratio, what I get then is this divided by that.
What has happened to Y prime?
Y prime has disappeared.
In other words, Y1 to Y sub j are a sufficient statistic for this problem.
We've shown that sufficient statistics are the only thing you need to use to do maximum likelihood detection.
OK?
In other words, all those other signals, all that other noise, all that stuff from out of band has disappeared.
Now let's go back to the fact that we were looking at a finite dimensional problem.
What happens now when I make j prime bigger?
When I start enlarging j prime?
What happens to all these probabilities that we're talking about?
This probability density goes ape because of this term here.
We're talking about a probability density here which is involving more and more and more terms.
I can't talk about that.
It doesn't go to any limit.
It goes to absolute nonsense as j prime gets big.
But if I form the likelihood ratio before I go to the limit, then I can go to the limit quite easily.
Because there isn't any limit involved there.
OK, in other words this is the likelihood ratio between hypothesis m and hypothesis i, if in fact I looked at this entire infinite amount of observation.
This is all I need to make the optimal MAP decision.
OK, so there's a theorem here which is called the theorem of irrelevance.
This is something that Wosencraft and Jacobs in their book on communication many years ago stressed a lot in trying to come up with a single space viewpoint of communication.
And you'll see why this really does give you a single point viewpoint of communication.
It says that assume that Z prime is statistically independent of the pair X and Z.
Then the MAP detection of X from the observation of Y and Y prime depends only on Y.
The observed sample value of Y prime is irrelevant.
OK?
So you can do all of detection theory, you can do all of communication, simply forgetting about that irrelevant stuff.
Because of the theorem we can stick to finite dimensional vectors and the other signals can be viewed as part of Z prime.
So you don't have to worry about them.
So long as each signal is independent of each other-- which means that these groups of bits, the first group a bit is used to form a sub x sub 1.
The next group, the form x sub 2, the next group the form x sub 3, and so forth.
So long as those sequences of bits are independent of each other, you're fine.
Now, suppose they aren't?
What happens then?
Interesting question.
We said that if they are independent, I can really do maximum likelihood detection on the whole sequence.
If they aren't independent, suppose I say, "oh I don't care about that."
I'm just going to use this portion of the output to make my decision and not worry about whether this is independent of anything else.
I can do that, this still is going to give me the optimal maximum likelihood detection in terms of the observation y1 up to y sub j.
So in other words, whether I have coding done before this or not doesn't make any difference.
I can still use maximum likelihood detection on the basis of y1 up to y sub j.
What the theorem says is if the out of band stuff-- both these inputs and the noise-- are independent of what I'm trying to detect, maximum likelihood becomes the optimum thing to do for equally likely inputs.
And otherwise, it's a perfectly reasonable thing to do but it's not optimal.
Now, a lot of people-- and we'll see some examples of this when we look at wireless-- in fact, use coding.
Then they use this particular kind of detection where they forget about all of the added information from other signals.
They make a decision on each of these x sub k, namely each of the M-ary signals that goes in, they make a hard decision on it.
It's called a hard decision, not because it's difficult it because they refuse to ever go back and change it.
If they just say likelihoods and try to put things together in the final decoder, it's called soft decoding.
Otherwise it's called hard decoding.
If you do soft decoding, it has to work better.
Because you're making, in a sense, a better decision because eventually you're using more information.
So soft decisions are better than hard decisions.
Used to be that everybody used hard decisions because hard decisions were easy and soft decisions were hard.
Strange, strange thing.
But anyway that's changed.
Why?
Well it ought to be obvious why.
Because anything you build now cost a tenth of what it used to cost to build it.
I heard Irwin Jacobs awhile ago saying that one of the things that they always did when they were designing new pieces of equipment is they would look at how much it would cost to build these devices.
And then, as opposed to most companies which would say that's too expensive let's find the cheaper way to do it, they said OK how long is it going to take for us to do it, and what is the price of components going to be by time we got it done?
And they would usually say, well it's going to cost a year before we can go into mass production.
By that time, everything will cost less than a half of what it's costing now.
So let's go ahead and do it the right way.
So again the argument comes that you can do the hard thing now.
Which is soft decisions, and that's what most people do at this point.
Let me give you one more picture to get ready for what we're doing next time.
Because it's a nice picture of different signal sets.
Because we've just talked abstractly of having multiple signals viewed as vectors, and this will give us some idea of what all of these mean.
I can have two signals, a binary signal set, and I can insist on the signals being orthogonal to each other.
Which is a nice thing to do some times.
But then I can look at it and I can say, "how can I make that a better signal system?"
The trouble with this signal system is it's not antipodal.
It's not antipodal because somehow by alternating between these two orthogonal signals-- there's a mean between them-- and I'm transmitting that mean plus the difference.
And the difference between them is minus 0.7 and plus 0.7 in that direction that way.
If you can, I guess you can't see it that way.
In this direction this way.
Well anyway, OK. A better thing to do than this is this, which is called bi orthogonal.
So you take orthogonal signals and then you have a signal set consisting of four signals and two dimensional space.
We then talk about orthogonal signals and higher dimensional space.
You can talk about three orthogonal signals in three dimensional space here.
There, there, and there.
So that's m equals 3 and j equals 3.
If you do the same thing that we did here-- we're going to make this into an equilateral triangle.
Namely we're going to center it around the center.
If we do the same thing that we did here we're going to turn this into a set of six waveforms which are still using the three degrees of freedom, but at least get us more signals.
For the same number of degrees of freedom.
So you can extend this picture as far as you want to.
You can talk about many, many orthogonal signals going into many more degrees of freedom.
For each one of them you can come up with a simplex set of signals.
The nice thing about the simplex set of signals is that all of the signals are arranged around the center point.
They're all equally distant from each other.
You can get these for every dimension by starting with these and simply taking the mean out, which loses you one dimension and makes this sort of ideal set.
Well tomorrow-- on Wednesday what we're going to do is, we're going to talk about these large sets here, and see what happens.
And we'll see that you in fact get to channel capacity this way.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, last time we were talking about this thing called the theorem of irrelevance.
At one level, the theorem of irrelevance is just another statement of, when you start looking at detection theory, there are things called sufficient statistics.
And what the theorem of irrelevance says is you can ignore things that aren't part of a sufficient statistic.
But more specifically, it says something about what those irrelevant things are.
And in particular, let me repeat what we said last time but with a little less of the detail.
You start out with a signal set.
Ok?
You're going to transmit one element of that single-- signal set-- and you're going to turn it into some waveform-- which we'll call X of t. And X of t is going to depend on which particular signal enters the transmitter at this point.
You're going to receive something where the thing that you receive has noise added to it.
And also if you'll notice I've written the thing that we receive that it's including a whole lot of other stuff.
OK?
In other words, the signal is constrained to j different degrees of freedom.
And what you get out of the channel involves a much larger set of degrees of freedom.
In other words, you're putting things in involving only a certain period of time, in a certain bandwidth, usually.
And you can look at anything you want to at the output.
The only thing is you can't peek at what the input was at the transmitter, because if you could there will be no sense in actually transmitting it.
OK, so the point is, we receive something which, in the degrees of freedom that we know about, Yj is equal Xj plus the noise variables.
And in the other degrees of freedom, Yj is just equal to Zj.
And now the rules of the game are that these out of band things, all of these noise coefficients, which are not part of the phi 1 up to phi sub capital J.
All of these things, the noise there, is independent of the noise in-- is independent of the noise and the signal in what we're actually sending.
Because it's independent it's irrelevant that's relatively easy to show.
But the broader way to look at this, and the way I want-- and the way I'd like to get you used to thinking about this-- is that this other stuff here can include signals sent by other users.
It can include signals sent by you at other times.
It can include anything else in the world.
And what this theorem is saying is, if you're sending the signal in just these j degrees of freedom, then you don't have to look at anything else.
So that all you have to look at is these received components, Y sub j. OK, it says something more than that.
It says a lot more than that.
These actual orthonormal functions that you use, phi 1 up to phi sub capital J, don't appear in that solution at all.
OK?
In other words, you really have a vector problem at this point.
You have a j dimensional vector that you send.
You have a j dimensional vector that you receive.
You can use orthonormal functions, which are anything in the world, so long as the noise in that region that you're looking at and sending things in does not vary.
In other words, what you can send is very broadband signals.
You can send very narrow band signals.
You can send anything and it doesn't make any difference.
What this says is when you're dealing with a white Gaussian noise channel, all signals are equivalent.
OK?
In other words, it doesn't make any difference what modulation system you use.
They all behave the same way.
The only difference between different modulation systems comes in these second order affects, which we haven't started to look at much yet.
How hard is it to retrieve carrier?
How hard is it to deal with things like time synchronization?
How hard is it to build the filters?
How hard is it to move from passband down to baseband if you want to do your operations at baseband?
All of these questions become important, but the basic question of how to deal with the Gaussian noise, it doesn't make any difference.
You can use whatever system you want to, and the analysis of the noise part of the problem is exactly the same at everything.
OK?
And that's fairly important because it says that in fact, this signal space there we're using-- I mean yes, we're all used to the fact that it's a vector space.
L2 is a vector space, fine-- what's important here is you're using a finite part of that vector space, and you can deal with it just as if it's finite dimensional vectors.
You can deal with vectors and matrices and all of that neat stuff and you can forget about all of the analog stuff.
you look at 99 percent of the papers appear both in the information theory transactions and in the transactions of-- oh I guess it-- it's communication technology.
You look at all of these things and 99 percent of the authors don't know anything about analog communication.
All they've learned is how to deal with these vectors.
OK?
So in fact this is the key to that.
And you can now play their game.
But also when their game doesn't work you can go back to looking at the analog waveforms.
But you now understand what that game is that they're playing.
They're assuming white Gaussian noise.
They don't have to deal with the fact that the white Gaussian noise is spread over all time and all frequency.
Because part of this thoerem says it doesn't make any difference how the noise is characterized outside of this region that you're looking at.
That's the other part of the argument that we've been dealing with all along.
It doesn't matter how you model the noise anywhere outside of what you're looking at.
The only thing you need is that the noise is independent outside of there.
OK?
So that's-- I mean this was sort of trivial analytically, but it's really an important aspect of what's going on.
OK. Let's go back to QAM or PAM.
The baseband input to a white Gaussian noise channel-- we're going to model it as u of t-- we're going to look at a succession of j different signals.
In other words, when we study detection, we said we're going to build the system, we're going to send one signal, and then we're going to receive that signal and we're going to tear that system down.
We're not going to send more than just this one signal.
OK, now we're sending a big batch of signals.
Were sending capital J of them.
Where J can be as big as you please.
Now we're going to look at two different alternatives.
OK, one of these alternatives is sending all j signals and building a receiver which looks at all J of them together and makes a joint decision on everything.
It makes a maximum likelihood decision on this sequence of J possible inputs.
This is one of the things we've looked at.
This is what happens when you have non binary detection.
I mean, here you have an enormous number of things you're detecting between.
And you do maximum likelihood detection on it.
And the question is, do you get anything extra from that beyond what you get out of doing what we did before, just forgetting about the fact that other signals existed?
Namely, which is better?
What the notes prove, and what I'm going to sort of indicate here without any attempt to prove it-- I mean it's not a hard proof, it's just-- I mean the hardest part of this is realizing what the problem is.
And the problem is you can detect things in two different ways.
You can detect things one signal at a time.
Or you can detect them the whole sequence at a time.
And you do something different in each of these cases.
OK, so we're going to assume again that these thetas are an orthonormal set.
I'm going to assume that I've extended them so they span all of l2.
I'm going to let v be a sample of this output vector, v1 to v sub j.
You see, I use the theorem of irrelevance here because I don't care about anything beyond v sub j.
Because all those other things are irrelevant.
So I only look at v1 to v sub j.
So the little v is going to be a sample value of this random vector.
And the components of the random vector, the output, are going to be the input variables plus the noise variables.
And the zj here are independent.
I don't even have to assume that they're Gaussian for this argument.
They can be anything at all, so long as they're independent.
OK, now if I'm doing the signal by signal detection, in other words if what I'm doing is I'm saying, "OK, I want to take this little j signal, and I want to decide on what it is just from looking at the v sample of the output."
OK, I can do that.
My observation is v sub j.
That's something we've talked about.
I mean the fact that you might have other information available doesn't make any difference.
You still can make a detection on the basis of this one variable.
OK, so we do that.
Now, we want to compare that with what happens when we make a detection for all capital J of these inputs, conditional on the whole sequence.
OK, you write out the likelihood ratio.
It factors.
And the thing that happens when you do this-- and you have to read the notes for the details on this, and it's not hard-- what you find is that the maximum likelihood decision is exactly the same in both cases.
OK, so it doesn't make any difference whether you detect little u sub j from the observation v sub j.
Or whether you detect the vector, u sub capital j, from the whole set of outputs, v sub 1 up to v sub capital J.
You get the same answer in both cases.
Now, you might say, "What happens if some of these likelihood ratios come out to be right on the borderline, equal to one?"
Well it doesn't make any difference because that's a zero probability thing.
If you want to worry about that, you can worry about it and you get the same answer, you just have to be a little more careful.
Now here is something the notes don't say, and it's also fairly important.
Here we're talking about the case where all these signals are independent of each other.
Which is sort of what we want to look at in communication because for the most part what we're doing is we're taking data of some sort, we're source processing it to make the bits coming up the channel be independent of each other, and then we're going to be sending them.
Well except now we want to say, well suppose that these inputs are not independent.
Suppose, for example, that we pass them through an error correction encoding device before transmitting them.
And the question is, what happens then?
Well the trouble is then you have dependence between u1 and u sub j.
So you can still detect each of these just by looking at the single output.
And it's a maximum likelihood detection based on that observation.
But if you look at the entire observation, v1 up to v sub capital j, you get something better.
How do I know you got something better?
Well I know you get something better because in this case where u1 up to u sub j are dependent on each other, these output variables, v sub 1 up to v sub capital j, depends on each other.
They aren't irrelevant.
If you want to do the best job of maximum likelihood detection, given the observation of v sub 1 up to v sub capital j, then you're going to use all those variables in your detection and you're going to get a better -- And you're going to get a better decision.
In other words, a smaller error probability, then you would have gotten otherwise.
OK?
But the important thing that comes out of here is that this simple minded decision-- where you make a decision on u sub 1 based just on v sub 1-- it's something you can do.
You can do the maximum likelihood decision.
And you know how it behaves.
You can calculate what the error probability is.
But you know now automatically that your error probability is going to be greater than or equal to the error probability that you would get if you base that decision one the whole set of inputs.
OK, this is an argument that, it seems hard for most people to-- that it seems hard for most people to think of right at the beginning.
Which is really to say, if you're doing an optimal detection, it is optimal.
In other words, anything else that you do is worse.
And you can always count on that to get bound between probabilities of error that you get doing something stupid, and probabilities of error that you get doing something intelligent.
And the stupid thing is not always stupid-- I mean the stupid thing is sometimes better because it's cheaper-- but the error probability is always worse there then if you did the actual optimum thing.
OK, so people in fact often do do detection where they have coded systems.
They decode each received symbol separately based on the corresponding observation.
They wind up with a larger error probability than they should.
Then they pass this through some kind of, some kind of error correction device.
And they wind up with a system that sort of performs reasonably.
And you ask, "Well, would it have performed better if in fact what you did was to wait to make a final decision until you got all the data?"
And we talk a little bit about various kinds of error control later when we get into wireless.
We'll see that some kinds of coding systems can behave very easily and can make use of all this extra information.
And other ones can't.
Algebraic kinds of schemes can't seem to make use of the extra information.
And various other kinds of schemes can make use of it.
And if you want to understand what's happened in the error correction field over the last five years-- unfortunately 6.451 won't be given, so you won't get all the details of this-- but the simplest one sentence statement you can say is that the world has changed from algebraic coding techniques to probablistic decoding techniques.
And the primary reason for it is you want to make use of all that extra information you get from looking at the full observation, rather than just a partial observations.
OK. Now, back to various signal sets.
I put this slide up last time.
I want to really talk about it this time because for each number of degrees of freedom, you can define what's called an orthogonal code.
And the orthogonal codes for m equals 2 and for m equals 3 are drawn here.
For m equals 2, it's something that we've seen before.
Namely, if you want to send a zero, you send a one in the first compound, the first degree of freedom, a zero in the second.
And if you want to send a one, you send the opposite thing.
We've all seen that this isn't a very sensible thing to do when we looked at binary detection, because when you use a scheme like this of course the thing that happens is we now know that we should look at this in terms of just looking at this line along here.
Because what we're really transmitting is a pilot tone, so to speak, which is half way in the middle here, which sticks right here.
Plus something that varies from that.
So that when we take out this pilot tone, what we wind up with is a one dimensional system instead of a two dimensional system.
Which we used to call antipodal communication-- and which everybody with any sense calls antipodal communication, even now-- but in terms of this, it's the simplest case of a simplex code.
And a simplex code is simply an orthogonal code where you've taken the mean and moved it out.
And as soon as you remove the mean from an orthogonal code, you get rid of one degree freedom, because one of the signals becomes dependent on the others.
Which is exactly what's happened here.
You've just-- if all your signals are in one degree of freedom here, when you do the same thing down here, well you get this.
And I'll talk about that later.
So, one thing you can do from an orthogonal code is go to a simplex code.
The other thing you can do if you want to transmit one more bit out of this signal set is to go to a bi orthogonal code.
Which says-- along with transmitting zero, one and one, zero-- you look this and you say, "Gee why don't I also put in zero minus one, minus one zero, and zero minus one down here.
Which is exactly what the bi orthogonal code is.
The bi orthogonal code simply says take your orthogonal code and every time you have a one, change it into a minus one and get an extra code word out of it.
What's the difference between a set of code words and a signal set?
Anybody have any idea?
Absolutely none.
They're both exactly the same thing.
And you think of it as being a code, usually, if what you're doing is thinking of generating an error correcting code and then from that error correcting code you think of using QAM or PAM or something else out beyond that.
You call it a signal set if you're just doing the whole thing as one unit.
What a lot of systems now do is they start out with a code, then they turn this into an orthogonal signal set.
I'll tell you in other words.
A code produces bits.
From the bits you group them together into sets of bits.
From the sets of bits you go into a signal-- which is, for example something from one of these three possibilities here-- OK.
The important thing to notice here-- and it's particularly important to think about it for a few minutes-- is because you do so many exercises.
And you've done a number of them already and you will do a few more in this course where you deal with the m equals 2 case.
And you can deal with this biorthogonal set here.
You can shift the biorthogonal set around by 45 degrees.
In which case it looks like this.
OK, so that looks like two PAM sets.
It looks like a standard QAM.
It looks like standard 4QAM.
This is exactly the same as this of course.
When you do detection on this you do detection by saying, when you transmit this does the noise carry you across that boundary?
And then does the noise carry you across this boundary?
The noise in this direction is orthogonal from the noise in this direction and therefore, finding the probability of error is very, very simple.
Because because you look at two separate orthogonal kinds of noise and you can just multiply these probabilities together in the appropriate way to find out what's happening.
The important thing to have stick in your memory now is that as soon as you go to m equals three, life gets harder.
In fact, if you look at this orthogonal set here and you try to find the error probability for it-- you try to find it exactly-- you can't do it like this.
You can't just multiply three terms.
You look at these regions in three dimensional space.
If you want to visualize what they are, what do you do?
What picture do you look at?
You look at this picture.
OK?
Because this picture is just this with the mean taken away.
So the error probability here is the same as the error probability here.
When I send this point the regions that I'm looking at look like this.
And they're not orthogonal to each other.
So to find the probability that this point gets outside of this region looks just a little bit messy.
And that happens for all m bigger than two.
I never knew this because well, I think this is probably a disaster that happens more to teachers than to people working in the field.
Because so often I have explained to people how these two dimensional pictures work that I just get used to thinking that this is an easy problem.
And in fact when you start looking at the problem for m equals three, then the problem gets much more interesting and much more useful and much more practical when n becomes three or four or five or six or seven or eight.
Beyond eight it becomes a little too hard to do.
But up until there, it's very easy.
OK, so we're going to make use of that in a little bit.
As we said orthogonal codes and simplex codes-- if you scale the simplex code from the orthogonal code-- have exactly the same error probability.
In other words, this code here where I've made the distances between the point square root of two over two, which corresponds to the distance between the points here.
This and this have exactly the same error probability.
So you can even, in fact, find the error probability for this or for this, whichever you find easier.
You now think it's easier to find the error probability for this.
I've lead you down the primrose path, because in fact this one is easier to find the error probability for.
This one, again, you can find the error probability if you want.
But the energy difference between this and this is simply the added energy that you have to use here to send the mean of these three signals.
And one signal is sitting out here at 1,0,0 one is sitting at 0,1,0 one is sitting at 0,0,1.
Even I can calculate the mean of those three.
It's one third, one third, an one third.
So you calculate the energy in one third, one third, and one third, and it's three times one ninth, or one third.
Which is exactly what this says.
The energy difference between orthogonal and simplex is 1 minus 1 over m. In other words that's the factor in energy that you lose by using orthogonal codes instead of using simplex codes.
Why do people ever use orthogonal codes?
If it's just a pure loss in energy in doing so?
Well, one reason is when n gets up to be 6 or 8, it doesn't amounts to a whole lot.
And the other reason is if you look at if you look at modulating these things on to sine waves and things like that, you suddenly see that when you're using this it becomes easier to keep frequency lock and phase lock than it does when you use this.
I mean you have to think about that argument a little bit to make sense out of it.
But that, in fact, is why people often use orthogonal signals because, in fact, they can recover other things from it.
Well because they're actually sending a mean, also.
So the mean is the thing that lets them recover all these other neat things.
So that sort of sometimes rules out this and it sometimes rules out that.
OK. Orthogonal and biorthogonal codes have the same energy?
Well, look at them.
These two signal points each have the same energy.
And these two have the same energy as these two.
So the average energy is the same as the energy at each point.
So this and this have the same energy.
What happens to the probability of error?
Well you can't evaluate it exactly.
Except here is a case where just looking at the m equals 2 case gives you the right answer.
I mean here to make an error you have to go across that boundary there.
Here to make an error you have to either go across that boundary or go across that boundary.
The error of probability essentially goes up by a factor of two.
Same thing happens here.
You just get twice as many ways to make errors as you had before.
And all of these ways to make errors are equally probable.
All the points are equally distant from each other.
So you essentially just double the number of likely ways you can make errors.
And the error probability essentially goes up by two.
Goes up a little-- well because you're using a union band it either goes up by a little more or a little less than two-- but it's almost two.
OK, so I want to actually find the probability of error now.
If you're sending an orthogonal code.
Namely, we pick an orthogonal code where we pick as many code words as we want to.
m might be 64, it might be 128, whatever.
And we want to try to figure out how to evaluate the probability of error for this kind of code.
After you face the fact that, in fact, these lines are not orthogonal to each other.
OK, well the way you do this is you say, OK the, even though this is a slightly messy problem, it's clear from symmetry that you got the same error probability no matter which signal point you sent.
Namely, every signal point is exactly the same as every other signal point.
What you call the first signal depends only on which you happen to call the first orthonormal direction.
Whether it's this, or this, or this.
I can change it in anyway I want to and the problem is still exactly the same.
OK, so all I'm going to do is try to find the error probability when I send this signal here.
In other words, when I send 1,0,0-- actually I'm going to send the square root of e and 0,0-- because I want to talk about the energy here.
Why am I torturing you with this?
Well 50 years ago, 55 years ago, Shannon came out with this marvelous paper which says there's something called channel capacity.
And what he said channel capacity was was the minimum rate-- the, was the maximum rate at which you could transmit on a channel and still get zero error probability.
Sort of the simplest and most famous case of that is where you have white Gaussian noise to deal with.
You're trying to transmit through this white Gaussian noise.
And you can use as much bandwidth as you want.
Namely, you can spread the signals out as much as you want to.
You can sort of see from starting to look at this picture that you're going to be a little better off, for example, if you want to send one bit in these two dimensions here with orthogonal signals.
If you can think of what happens down here for m equals 4, you would wind up with four orthogonal signals.
And if you wind up with four orthogonal signals, you're sending two bits, so you're going to use twice as much energy for each of them.
So you can scale each of these up to be 2,0,0,0; 0,2,0,0; and so forth.
So you're sending twice as much energy.
You're filling up more bandwidth because you need more degrees of freedom to send this signal.
But who cares?
Because we have all the bandwidth we want.
Gets more complicated.
But the question is, what happens if we go to a very large set of orthogonal signals?
And what we're going to find is that when we go to a very large set of orthogonal signals, we can get an error probability which goes to zero as the number of orthogonal signals gets bigger.
It goes to zero very fast as we send more bits with each signal.
And the place where it goes to zero is exactly channel capacity.
Now in your homework, you're going to work out a simpler version of all of this.
And a simpler version is something called the union band.
And in the union band you just assume that the probability of error when you send this is the sum of the probability of error, of making an error, to this and the possibility of making an error to this.
The thing that we're going to add here-- and let me try to explain it from this picture.
I'm sending this point here.
OK?
And I can find the probability of error over to that point.
Which is the probability of going over that threshold.
I can talk about the probability of error to over here, which is the probability of going over that threshold.
These are not orthogonal to each other.
And in fact they have a common component to them.
And the common component is what happens in this first direction here.
OK, in other words if you send 1,0,0 and the noise, and your own noise variable clobbers you.
In other words, what you receive is something in this coordinate which is way down here and sort of arbitrary everywhere else-- conditional on the noise here being very, very large-- you're probably going to make an error.
Now if you can imagine having a million orthogonal signals-- and the noise clobbering you on your own noise variable-- you're going to have a million ways to make errors.
And they're all going to be kind of likely.
If I go far enough down here, suppose what I receive in this coordinate is zero.
Then there's a probability of one half that each one of these things is going to be greater than zero.
If I use a union bound, adding up the probabilities of each of these, I'm going to add up a million one halves.
Which is 500,000.
As an upper bound to a probability.
And that's going to clobber my bound pretty badly.
Which says the thing I want to do here is, when I'm sending this I want to condition my whole argument on what the noise is in this direction.
And given what the noise is in this direction, I will then try to evaluate the error probability, conditional on this.
And conditional on a received value here.
In fact, the noise in the direction, w2, is independent of the noise in direction, w3, independent of the noise in direction, w4, and so forth.
So at that point, condition on w1 I'm dealing with m minus 1 independent random variables.
I can deal with independent random variables.
You can deal with an independent random variables.
Maybe some of you can integrate over these complex polycons.
I can't.
I don't want to.
I don't want to write a program that does it.
I don't want to be close to anybody who writes a program that does it.
It offends me.
OK, so here we go.
Where am I?
OK, so the first thing I'm going to do, which I didn't tell you, because I'm going to scale the problem a little bit.
I did say that here.
I'm going to normalize the whole problem by calling my output W sub j instead of Y sub j.
And I'm going to normalize it by multiplying Y sub j by the square root of the noise variance.
OK in other words, I'm going to scale the noise down so the noise has unit variance.
And by scaling the noise down so it has unit variance, the signal will be scaled down in the same way.
So the signal, now, instead of being the square root of e-- which is the energy I have available-- it's going to be the square root 2e divided by N0.
Somehow this thing keeps creeping up everywhere.
This 2e over N0.
Well of course it's the difference between e, which is the energy we have to send the signal, and N0 over 2, which is the noise energy in each degree of freedom.
So it's not surprising that it's sort of a fundamental quantity.
And as soon as we normalize to make the noise variance equal to one, that's what the signal is.
So I'm going to call alpha this so I don't have to write this all the time.
Because it gets kind of messy on the slides.
OK so given that I'm going to send input one, the received variable W1 is going to be normal with a mean alpha which is the square root of 2e over N0 and with a variance of one.
All the other random variables are going to be normal random variables.
Mean zero, variance one.
OK?
I'm going to make an error if any one of these other random variables happens to rise up and exceed W1.
So the thing we have here is W1 is doing some crazy thing.
I have this enormous sea of other code words in other directions.
And then the question we ask is can the noise-- which is usually very small all over the place but it might rise up some place.
And if it rises up someplace, we're asking what's the probability that it's going to rise up to be big enough that it's exceeds W1?
So, I can at least write down what the error probability is exactly at that point.
What I'm going to do is I'm going to write down the probability density for W1.
And W1, remember, is a Gaussian random variable with mean alpha and with variance one.
So we could actually write down what that density is.
And for each W1 I'm going to make an error if the union of any one of these events occurs.
Namely, if any one of the W sub j is bigger than W1, bingo I'm lost.
OK?
So I'm going to integrate that now over all values of W1.
Now, as I said before if W1 is very small, then lots of other signals look more likely than W1.
In other words I'm going to get clobbered no matter what I do.
Whereas if W1 is large, it looks like the union bound might work here.
And the union bound is what you're doing in the homework without paying any attention to W1.
OK, so I'm going to use the union band-- and the union band says the probability that this union of all these different m minus 1 events-- exceeds some value, W1.
W1 is just some arbitrary constant in here.
The probability of this.
I'm going to write it in two ways.
It's upper bounded by one.
Because all probabilities are upper bounded by one.
This is not a probability density, this is a probability now so it's upper bounded by one.
And it's also upper bounded by this union band.
Mainly, the union of m minus 1 events.
Each with probability.
This is the tail of the Gaussian distribution evaluated at w1.
So I have the union band here.
I have one here.
Going to pick some parameter gamma.
I don't know what gamma should be yet, but whatever I pick gamma to be I still have a legitimate bound.
This is less than or equal to this, and it's less than or equal to this.
Just common sense dictates that I'm going to use this when this is less than one.
And I'm going to use this when this is greater than one.
That's what common sense says.
As soon as I start to evaluate this, common sense goes out the window because then I start to deal with setting this quantity equal to one and dealing with the inverse of the Gaussian distribution function.
Which is very painful.
So I'm then going to bound what this is.
And in terms of the bound on this, I'm going to affect gamma that way.
Because all I'm interested in is an upper bound on error probability anyway.
OK, so that's where we're going to go.
So this probability of error then is then exceeded by using this bound for small W1.
I get this integral over all W1 between minus infinity and gamma.
And let's just look at what this becomes.
This is just the tail of the Gaussian distribution.
That's the lower tail instead of the upper tail, but that doesn't make any difference.
So this is exactly Q of alpha minus gamma.
Mainly, alpha is where the mean of this density is and gamma is where I integrate to.
So if I shift the thing down, I get something that goes from-- well you might be surprised as to why this is minus gamma instead of plus gamma, and it's minus gamma because I'm looking at the lower tail rather than the upper tail and asking you to think this through in real time is unreasonable, but believe me if you sit down and think about it for a couple of seconds you'll realize that this integral is exactly this lowered tail of this Gaussian distribution.
The other term is a little more complicated.
It's m minus 1, which is that term there, times Q of w1, which is this term here, times the Gaussian density.
Now this is the Gaussian density for W1, which is one over square root of 2pi.
This is a normalized invariance, but it has a mean of alpha, so it's this.
OK, well next thing we have to do is either fiddle around like mad or look at this.
If you remember one of the things that you did-- I think in the previous homework that you passed in-- you found the bound on Q.
Which looks like this.
OK. That's just the tail of the Gaussian distribution.
And the tail of the Gaussian distribution is upper banded by this for W1 greater than or equal to zero.
It's upper bounded by a bunch of other things which you find in this problem.
The other bounds are tighter.
This is the most useful bound to the Gaussian distribution that there is.
Because it works for W greater than or equal to zero.
And it's exact when W1 is equal to zero.
Because you're just integrating over half of the Gaussian density.
And it's convenient and easy to work with.
But what that says is the Q of W1, when W1 is anything greater than or equal to zero, looks very much like a Gaussian density.
So the thing that you're doing here is taking one Gaussian density and you're multiplying it by another Gaussian density.
And the one Gaussian density is sitting here looking like this with some scale factor on it at zero.
The other Gaussian density is out here at alpha-- was alpha, wasn't it?
Or was it gamma?
Can't keep my gammas and alphas straight-- OK, and it looks like this.
If you take the product of two Gaussian densities of the same amplitude and everything.
And the same variance.
What do you wind up with?
Well you can go through and complete the square.
And you can sort of see from looking at this from the symmetry of it that what you're going to get is the Gaussian.
Which is right there.
Alpha over 2.
OK so these two things, when you multiply them, look like this.
This has the same variance as these two things do.
But it's centered on alpha over 2.
If you want to take the Fourier transform and multiply the Fourier transform.
I mean, you take the Fourier transform of the Gaussian and you got a Gaussian.
So here we're multiplying.
Up there in Fourier transform space you're convolving.
And when you convolve a Gaussian with a Gaussian you got a Gaussian.
When you multiply Gaussian by a Gaussian you got a Gaussian.
When you do anything to a Gaussian, you got a Gaussian.
OK?
So this thing-- and if you don't believe me just actually take these two exponents and complete the square and see what you get-- so the mean is going to be alpha over 2.
So, here you have a term which is one tail of the Gaussian distribution centered at alpha minus gamma.
And here you have another one centered at alpha over 2.
When you see these two terms, something clicks in your mind and says that sometimes this term is going to be the significant term.
Sometimes this term is going to be significant term.
And it depends on whether gamma-- alpha minus gamma- is greater than or less than alpha over 2.
So you can sort of see what's going to happen right away.
Well I hope you can see what's going to happen right away, because I'm not going to torture you with any more of this.
And you can look at the notes to find the details.
But you now sort of see what's happening.
Because you have a sum of two terms.
We're trying to upper bound this.
We don't much care about factors of two, or factors of square root of 2pi or anything.
We're trying to look at when this goes to zero.
When you make m bigger and bigger.
And when it doesn't go to zero.
So when we do this, the probability of error is going to be less than or equal to either of these two terms.
And here are these two alternatives that we spoke about.
Namely when alpha over 2 is less than or equal to gamma, we get this.
When alpha over 2 is greater than gamma, we get this.
And this involves choosing gamma in the right way, so that that union bound is about equal to one at gamma.
OK, well that doesn't tell us anything.
So we say, "OK, what we're really interested in here is, as m is getting bigger and bigger, we're spending a certain amount of energy per input bit.
And that's what we're interested in as far as Shannon's theorem is concerned.
How much energy do you spend to send a bit through this channel?
And this gives you the answer to that question.
You let log m equal to b. m is the size of the signal alphabet, so b is the number of bits you're sending.
So Eb, namely the energy per bit, is just the total energy in these orthogonal waveforms divided by b.
So that's the energy per bit.
OK, you substitute these two things into that equation and what you get is these two different terms.
You got E to the minus b times this junk.
And E to the minus b times this junk.
What happens when m gets big?
When m gets big, holding Eb fixed, so the game is we're going to keep doubling our orthogonal set, being able to transmit one more bit.
And every time we transmit one more bit, we get a little more energy that we can use to transmit that one extra bit.
So we used an orthogonal set, but using a little more energy in that orthogonal set.
So what this says is that the probability of error goes down exponentially with b, if either one these terms are positive.
And this, and looking at the biggest of these terms tells you which one you want to look at.
So, anytime that Eb over 4N0 is less than or equal to log n is less than Eb over N0, you get this term.
Any time Eb over 4N0 is greater than the natural log of 2, you get this term.
Now when you go through the union bound in your homework, I'll give you a clue.
This is the answer you ought to come up with when you're all done.
Because that's what the union bound tells you.
Here, remember we did something more sophisticated the union bound because we said the union bound is lousy when you get a lot of noise on W1.
And therefore we did something separate for that case.
And what we're finding now is depending on how much energy we're using, namely depending on whether we're trying to get very, very close to channel capacity or not.
If you're trying to get very close to channel capacity you've got to use this answer here.
Which comes from here.
And in this case, this says that the probability of error goes to zero exponentially in the number of bits you're putting into this orthogonal code, at this rate.
Eb over 2N0 minus log 2.
Which is positive if Eb over 4N0, well-- I'm sorry.
If you can, if you can trace back about three minutes, just reverse everything I said about this and about this.
Somehow I wrote these inequalities in the wrong way and it sort of confused me.
This is the answer you're going to get from the union bound.
This is the answer that we get now because we use this more sophisticated way of looking at it.
Sob.
No.
Erase what I just said in the last thirty seconds and go back to what I said before that.
This is the answer you're going to get in the homework.
This is the answer that we want to look at.
This thing goes-- is valid-- anytime that Eb over N0 is greater than or equal-- is greater than natural log of 2.
When Eb over N0 is equal to log 2, that's the capacity of this channel.
Namely Eb equals N0 log 2.
Well better to say it Eb over N0 equals natural log of 2 is capacity.
And anytime Eb over N0 is greater than log 2 this term in here is positive.
The error probability goes down exponentially as b gets large and eventually goes to zero.
It doesn't get down as fast here as it does here.
But this is where you really want to be because this is where you're transmitting with absolutely the smallest amount of energy possible.
OK, so that's Shannon's formula.
And at least we have caught up to 50 years behind what's going on in communication.
And actually shown you something that's right there.
And in fact, what we went through today is really the essence of that, of the proof of the channel capacity theorem.
If you want to do it for finite bandwidth it gets much harder.
But for this case, we really did it.
It's all there.
I mean you read the notes and in a little extra detail.
But just with the grunge work left out, that's what's going on.
OK, wireless communication.
That's what we want to spend the rest of the term on.
We want to spend the rest of the term on that because whether you realize it or not, we sort of said everything there is to say about white Gaussian noise.
And when all of that sinks in, what you're left with is the idea the white Gaussein noise.
You're really dealing with just a finite vector problem.
And you don't have to worry about anything else.
The QAM and the PAM, all that stuff, all disappear.
Doesn't matter whether you send things broadband, narrow band, whatever.
It's all the same answer.
Wireless is different.
Wireless is different for a couple of reasons.
You're dealing with the radiation between antennas because you're dealing with the radiation between antennas rather than what's going on on a wire.
I mean, what's going on a wire, the wire is pretty much shielded from the outside world.
So you send something, noise gets added, and you receive signal plus noise.
There's not much fading, there's not much awkward stuff going on.
Here all of the stuff goes on.
As soon as you start using wireless communication, you're allowed to drive around in your car talking on two phones at once.
With your ears and eyes shielded.
You can kill yourself much easier that way than you can with ordinary telephony.
You can be in constant communications with almost anyone.
OK, so you have motion, you have temporary locations.
You have all these neat things.
And if you look at what's happening in the world, the less developed parts of the world have much more mobile communication than we do.
Because in fact they don't have that much wire communication.
It's not that good there.
So they find it's far cheaper to get a mobile phone.
Than like us where we have to pay for both a wire line phone and a mobile phone.
So they have sort of the best of the two worlds there.
Except their mobile phones are like our mobile phones.
They only work three quarters of the time.
And all of the research that's going into sending video over wireless phones, it seems that nobody's spending any time trying to increase the amount of time you can use your wireless phone from 75% to 90%.
And if any of you want to make a lot of money and also do something worthwhile for the world, invent a wireless phone the works 90% of the time.
And you'll clean up, believe me.
And you can even send video on it later if you want to.
OK that's another thing that wireless has turned out to be very useful for.
And I'm sure you all know this.
It avoids mazes of wires.
I mean many people in their homes and offices and everywhere are starting to use local area wireless networks just as a way of getting rid of all of these maddening wires that we have running all over the place.
As soon as we have a computer and a printer and a fax machine and a blah blah blah, and a watch which is connected to our, and a toaster which is connected to our computer.
Argh!
Pretty soon we're going to be connected to our computers.
We're going to have little things stuck in our head and stuck our neck and all over the place.
So it'll be nice to have these, it'll be nice to not have wires when we're doing that.
OK, but the new problem is that the channel, in fact, changes with time.
It's very different from one time to another.
And you get a lot of interference between channels.
In other words, when you're dealing with wireless you cannot think of just one transmitter and one receiver anymore.
That's one of the problems you want to think about.
But you really have to think about what all the other transmitters are doing and what all the other receivers are doing.
It was started by Marconi in 1897.
It took him about three years to get transcontinental communication.
I mean we think we're so great now-- being able to have research move as quickly as it does-- but if you think of the amount of time it takes to create a new wireless system, it's a whole lot larger now than it was then.
I mean he moved very fast.
I mean the technology was very primitive and very simple.
It was not a billion dollar business.
But in fact it was very, very rapid.
But what's happened since, with wireless, has been very fitful.
Businesses have started.
Businesses have stopped.
People have tried to do one thing.
People have tried to do another thing.
They name things by something different all the time.
I mean one sort of amusing thing is back in the early seventies the army was trying very hard to get wireless communication in the field.
And they called this packet radio.
And they had all the universities in the country spending enormous amounts of time developing packet radio.
Writing many papers about it.
They finally got disgusted because nothing was happening.
So they pulled all the funding for that.
And about five years later, when the people at DARPA and NSF and all of that forgot about this unpleasant experience, people started talking about ad hoc networks.
Guess what an ad hoc network is?
Same thing as packet radio.
Just a new name for an old system, and suddenly the money started flowing in again.
We don't know whether it'll be any better this time than it was last time.
But anyway that's the way funding goes.
OK, what we're going to talk about in this class is sort of an old fashioned thing.
It's not as sexy as what all these other systems are.
It's just cellular networks.
It's probably because that's well understood by now, and it's because we can talk about all of these fundamental problems that occur in mobile communication just in the context of this one kind of system that, by now, is reasonably well understood.
When you're doing cellular communication, you wind up with a large bunch of mobiles all communicating with one base station.
OK, in other words you don't have the kind of thing you had in the packet radio network or in the ad hoc network, where you have a huge number of mobile telephones which are all communicating to each other.
And where one phone has to relay things for others.
You wind up with a very complicated network problem.
Here, it's in a sense, a much simpler and more sane structure.
Because you're using mobile for doing the things that mobile does well.
And you're using wires for the things that wires do very well.
Mainly you have lots of mobiles which are moving all over the place.
You have these fixed base stations which are big and expensive, and put up on hills or on buildings or on big poles or something.
And you spend a lot of money on them.
You have optical fibers or cables or what have you running between them or running from them to what's called a MTSO.
Mobile Something Subscriber Office-- and I can never remember what those letters stand for-- Mobile Telephone Subscriber Office.
All I had to do to remember that was thank this was done by telephone engineers.
No, Mobile Telephone Switching Office and telephone engineers think in terms of switching and in terms of telephones.
And mobile and offices just follows along.
So the way these systems work is you go from a mobile to a base station.
From the base station to one of these MTSOs, which is just a big switching center.
From there you're in the wired network.
And from there you can either go back to a mobile or go back to a wire line telephone or go anywhere you want to.
But but the point in that, and I think this is important to remember, is that cellular networks are an appendage of the wire line network.
And you always have this wire line network in the middle.
You probably always will.
Because wire line networks have things like fiber which carries enormous amounts of data very, very cheaply.
And mobile is very limited as far as capacity goes.
And it's very noisy.
OK, so that lets us avoid the question of how do you do relaying.
When you see pictures of this, people draw pictures of hexagon cells
[UNINTELLIGIBLE] turn around
Oh, when I hit this, and it uh, OK.
I mean there's not much information on this picture anyway but, [LAUGHTER] OK, but people think in terms of base stations put down uniformly with nice hexagons around them.
And any time a mobile within one hexagon it communicates with the base station which is at the center of that hexagon.
And in reality what happens is that the base stations are spread all over the place in a very haphazard way.
I shouldn't say haphazard, because people worked very hard to find places to put these base stations.
Because you need to rent real estate, or buy real estate to put them in.
You have to find out somehow what kind of coverage they have.
And it's a very fascinating and very difficult problem.
One thing I'm going to try to convince you of in the next lecture or so is that the problems of choosing base stations are very heavily electromagnetic in nature.
You really have to understand electromagnetism very well.
And and you have to understand the modeling of these physical communication links very, very well in order to try to sort out where base stations should go and where they shouldn't go.
The other part of the problem is the part of the problem dealing with how do you design the mobile phone itself?
How do you design the base station itself?
And these are questions which don't depend so much on the exact modeling of the electromagnetic channel.
They only depend on very coarse characteristics of it.
And very often, when you start to study mobile, you will spend an inordinate amount of time studying all of the details of these electromagnetic channels.
Which in fact are very important as far as choosing base stations are concerned.
And have relatively little to do with the questions of how do you design mobiles.
How do you design base stations?
It has enough to do with it that you have to know something about it, but it's not central anymore.
OK, so let's look at what the problems are.
As I said the cellular network is really an appendage to the wire network.
The problems we're going to have to deal with is when you're outgoing from your own cell phone, there's some kind of strategy that has to be used for you to find the best base station to use.
And it's a difficult question because you're trying to find a base station you can communicate with and one that's not so overcrowded that you can't talk to it.
So that's one big problem.
We won't talk about that much.
Another is the ingoing problem.
Finding a mobile.
If you think about that, it's really a very tricky problem.
Because I run around in my car with my cellphone turned off all the time.
And I only turn it on if I want to talk to somebody.
So I turn it on and somehow the whole network has to suddenly realize where I am.
And you know that happens with all of these cellular networks all over the place.
And every time somebody turns on their cell phone there's a lot of stuff going back and forth that says who is this guy?
Does he have the right to talk?
Has he paid is bill?
And how do I actually find a base station for him to use?
So this is kind of, both these questions are kind of difficult.
And the even worse question is if somebody's calling me and I live say, in Boston, or close to Boston, and I'm out in San Francisco and somebody calls me on my cell phone, the call gets to me.
And if you just imagine a little bit what has to go on in this cellular network in order for the cellular network to realize that I'm in San Francisco instead of Boston.
And then realize how to get calls to me in San Francisco.
I mean there's a lot of interesting stuff going on here.
But we're not going to talk about any of that because that's really sort of an organizational question as opposed to a physical communication question, which is the kind of thing we're interested in here.
OK, when you have these multiple mobiles which are sending to the same base station.
People who are working on mobile communication, sort of the practical side of it, call this the reverse channel.
Why they call this the reverse channel and the other one the forward channel, I don't know.
Forward channel goes from the base station to the mobile.
Reverse station, reverse channel goes from the mobile to the base station.
And what it says is the terminology was chosen by the people designing the base stations.
That's sort of clear.
But if you read about this in any more technical publication, you will see this thing being called a multi access channel.
It's the multi access channel because many, many users are all trying to get into the same base station.
And this one electromagnetic wave-- which is impinging on the various space station antennas-- is carrying all of that stuff all multiplexed together in some way.
And it's not multiplexed together in a sensible way, because it's multiplexed together just by all of these waveforms randomly adding to each other.
So information theorists call these things multi access channels.
When you're going the other way, base station to mobiles, it's called the forward channel by the telephone engineers.
It's called the broadcast channel by information theorists.
For those of you who think about broadcast in terms of TV and FM and all that sort of stuff, this is a little bit confusing.
Because this is not the same kind of broadcast that you're usually thinking about.
I mean the usual kind of broadcast is where everybody gets the same thing whether you want it or not.
But that whole signal is there, and you all get the whole thing.
Here what it is, is you're sending a different message to everyone.
You don't want everyone to be able to tune in and receive what anybody else is getting.
You want a little privacy here.
So it's really broadcasting separate messages and trying to keep them separate.
While the systems, almost all of them, are now digital I think.
In the sense of having a binary interface, this is the same issue we've been talking about all along.
You say something is digital if there's the binary interface on it.
The source is either analog or digital.
Cellular communication was really designed for voice.
Now all the research is concerned with how do you make it work for data also.
One of the things we're going to talk about a little bit is why the problems are so very different.
I mean you would think they're both the same problem.
Because in both cases you're just transmitting a string of bits.
That's all that's going on.
But the big difference is that in voice, you can't tolerate delay in voice.
In data you can tolerate delay.
You can tolerate a lot of delay in data.
And therefore you can do lots of things with data that you can't do with voice.
If you want to have a system that deals with both voice and data, it's got to be able to get the voice through without delay.
And you have to find some way of solving that problem.
OK, let me just say this quickly.
Let me just see if there's anything else of content here.
This is all just boiler plate stuff.
Let me skip that.
And skip this.
The thing we're going to be concerned about here is really these physical modeling issues.
And where we wind up with that is we're typically talking about bandwidths that are maybe a few megahertz wide.
Maybe a few kilohertz wide.
Or maybe a few megahertz.
But we're talking about carrier frequencies which are usually up in the gigahertz range.
And they keep varying depending on which new range of frequencies gets opened up.
They started out a little below a gigahertz.
They only went up to 2.4 and now they're up around five or six.
And things like that.
When we talk about physical modeling, we want to understand what difference it makes what carrier frequency you're at.
And it does make a difference because we'll talk about Doppler shift.
And Doppler shift changes a lot as you go from one range to another.
But for the most part these systems are narrow band.
There's a lot of work now on wide band systems.
And what's does a wide band mean?
Does it mean more than a megahertz?
No, it means a system where the bandwidth that you're communicating over is a significant fraction of the carrier frequency.
If there is a carrier frequency.
Many of these wide band systems are not even done in terms of the carrier frequency.
They're just done in terms of an arbitrary waveform which takes over an enormous amount of bandwidth.
If you're dealing with the narrow band problems, white Gaussian noise is a good assumption for the noise.
But now, along with the noise you have all these other effects.
You have a channel, where the channel is not just a pass through wire with a little attenuation on it.
I mean, remember what we've done all along.
We have absolutely ignored the question of attenuation.
We've just gotten rid of it and say what you send is what you receive.
We've gotten rid of the problem of filtering on the channel.
We've said a little bit about it, but essentially we've avoided it.
Now the problem that you have is this channel that you're transmitting over really comes and goes.
Sometimes it's there, sometimes it's not.
So it's a time varying channel.
It's a time varying channel which depends on the frequency band that we're using.
And one of the things that we have to talk about in order to come to grips with this is questions about how quickly does it change and why does it change.
And how much do you have to change the frequency before you got something that looks like an independently different channel?
So we have to deal with both of those and we're going to do that next time.
And in trying to come to grips with these questions, the first thing we're going to do is to look at very, very idealized models of what goes on in communication.
Like, we're going to look at a point source radiating outwards.
We're going to look at a point source radiating outwords, hitting a barrier, and coming back.
Interesting problem to look at and you ought to read the notes about this.
What happens when you're in a car and you're driving at 60 miles an hour towards the reflecting wall.
And right before you hit the wall, what's the communication look like?
OK, that's a very neat and very simple problem.
You can't do it many times, but we will talk about that next time.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We've just started talking about wireless communication.
We spent a lot of time talking about how do you communicate, essentially on wire lines, where essentially the only problem is white Gaussian noise.
Namely, you transmit a signal, noise gets added to it, you receive the sum of the transmitted signal plus noise and all of the going from base band to pass band.
All of that stuff is all messy analytically, but essentially all that's happening is that you're going from -- you take a signal, you move up to pass band, you add noise, you move back to base band, in which case you get the original transmitted signal plus the noise moved back to -- move back -- moved to base band.
So what we wound up with there was a relatively simple situation.
Wireless communication is not so simple.
As you all know, if you use a cell phone, and I'm sure most of you do, because of something called fading -- what we want to understand is where this fading comes from, how it arises and some of the things you might do about it.
One of the things you should recognize right at the beginning -- and we won't spend much time talking about it -- is if you happen to be trying to use a cellular phone and there's a big wall, which is perfectly reflecting right in front of you and the base station you're trying to communicate with is on the other side of that wall, you're not going to get through.
In other words, there are some situations where no matter how you design a cellular phone, you just can't get any communication.
It's part of life.
What we want to do, however, is to make sure that when you can get communication, you will get it as well as possible.
You will prolong the period, but you're still communicating while the channel's getting worse and worse -- and part of the way to do that is to understand what it is in the physical mechanisms that's making the problem difficult.
OK, so to start out with this, we'll start out kind of easy.
We'll assume an input, which is just a cosine of 2 pi ft at some ficed antenna -- and this is radiating outwards and the electric field -- anywhere in free space, if you go a long distance away.
If you go a very short distance from the antenna, if you study electromagnetism, you know that all sorts of crazy things are going on.
When you get very far away, there's something called the far field and essentially what happens is that all of these disturbances close in just sort of disappear and very far away what's happening is that the field strength -- and this is true for the magnetic field also, which is just in -- if the electric field is this way, the magnetic field is that way.
They're both propogating outwards and they're going down as 1 over r, just like when we deal with linear systems, we never talk about voltage and current anymore.
This is why we use a square root of minus 1 at this point -- because after you learn about voltage and current, you don't have to talk about them anymore.
You just deal with one of them for the function that exists someplace and the other one just follows along from the impedance.
Same thing happens for electric fields and magnetic fields.
You don't have to bother about both of them, unless you're really trying to solve Maxwell's field equations, which is a very challenging endeavor.
I admire any of you who can do this.
I used to be able to do it many, many years ago and I've given up on it because I decided that's for younger people than me.
This far field has a very simple kind of behavior, because what happens is it has to go down as 1 over r. How do you know it has to go down as 1 over r?
Namely, the distance away from this radiating antenna.
If you look at a sphere, which is very, very far away from the radiating antenna, all you find is this field that's radiating outwards and you look at how much energy is radiating outwards.
The energy which is radiating outwards, there's no place to lose it because we're transmitting just an open space now -- or at least that's what we're imagining at this point.
So we're transmitting out in open space.
Energy doesn't get beaten up any place so it just keeps going out until it disappears out of the outer edges of the known universe.
It travels out there with the speed of light.
That's what Maxwell's laws say if you solve them.
Since this sphere has an area which is proportional to r squared, the only thing that can be happening in this wave, which is propogating outward spherically, is it has to be going down as 1 over r, because that's the only way for the power to balance out.
You can't be creating power out in this free space and you can't be losing power.
Whatever you're sending is just radiating outwards, so we have to have this 1 over r dependence any time you're dealing with free space.
The other thing that's happening is we have an antenna pattern.
The antenna pattern is a function of two angles.
We'll think of theta as being the angle around this way and psi as being the angle this way.
If you like to think of angles in some other way, be my guest.
The only thing is, when we're radiating outwards through this sphere, if the antenna is directional, it's going to be radiating more in some directions than it is in other directions and this simply takes that into account.
I'm not going to pay any attention to this at all.
It just exists and it's there.
For people who design antennas, that's an important issue: How do you make antennas which are directional, which will shine their energy in one direction rather than another?
We're not going to go into that at all.
We're just representing the fact that it's there.
This is just a factor which says, how much loss do you have in the antenna and how much of the power that you're radiating goes in each one of these directions and how does it depend on the frequency that you're transmitting at?
Antennas are sometimes designed to be rather frequency dependent, so they're tuned to a certain frequency.
They work very well at that frequency and on other frequencies, they just go to pot.
The other part of this equation is that if I send a signal, as it radiates outward, it's going to be radiating outward at the speed of light.
Therefore, whatever I receive is going to be delayed by the distance that I am away divided by the speed of light.
So this equation is something you don't really have to know any electromagnetics to derive.
If you want to find out what this term is, yes, you'd need some electromagnetics.
All this is saying is the power has to be going down as 1 over r and you have a propogation delay, which has to be going as r divided by the speed of light.
Now, if we look at what happens when we put a receiving antenna out at some distance r away from the transmitting antenna and in this direction theta and psi, that receiving antenna is going to distort this electromagnetic wave locally around the receiving antenna, but it's not going to distort the whole thing.
In other words, its power is radiating outwards.
It doesn't know anything about this little receiving antenna until it gets close to it, then the electromagnetic wave gets distorted somewhat.
The only thing that's happening because of this receiving antenna is that there's some added antenna pattern due to the receiving antenna -- namely, some added attenuation which multiplies by the attentuation in the source antenna.
You have both the source antenna pattern, the receiving antenna pattern.
We put the two together.
We call that alpha and we don't bother about it anymore except recognizing that it might change with frequency also.
Then we have this propagation delay between transmitting antenna and receiving antenna If you're transmitting in free space from one antenna to another antenna, this is what happens, with some arbitrary pattern here for the two antennas, which depends on whether they radiate spherically or whether they radiate in some sort of direction.
That's the received wave form.
That said, if you look at it, at the received field -- this is supposed to be valid for any -- oh, sorry.
Yes, that would help.
This equation is supposed to be valid for any frequency we want to transmit at, at any time, and if we're transmitting two signals, both together, the response is going to be the sum of the response to one signal and a response to the other signal.
In other words, Maxwell's laws are linear and therefore the response that you get when you solve Maxwell's laws -- I've never solved Maxwell's laws for anything this complicated and probably none of you had either.
You might have, but anyway, they are linear and therefore, in fact, what's going on is that this gives you a system function which says what the response is to any given input that you might want to transmit.
It says the response -- at frequency f to a sinusoidal input, which is what we were assuming before.
We were assuming that we transmitted cosine 2 pi ft.
The system function is just this antenna pattern times e to the minus 2 pi i times f times the distance away divided by c divided by r. Namely, this takes into account the propogation delay.
The only thing it doesn't take into account is what the input is.
The received field is then the real part of the system function times what we already assume that we were going to be transmitting.
Namely, the real part -- e to the 2 pi i ft.
So at this point, what we're doing is taking into account the fact that the solution to Maxwell's equations is going to be linear and therefore we can just add up what happens for each frequency of input.
What you notice from looking at that is that when we have a fixed transmitting antenna, a fixed receiving antenna and free space between them, we are right back to the problem that we started with.
Namely, white Gaussian noise on a channel, because nothing is varying with time.
There isn't any fading.
Nothing interesting is going on.
This is sort of like the case of microwave towers.
Microwave towers are set up and they have nice directional antennas, nice horns which are blowing at each other.
Nothing changes except every once in awhile is a rainstorm or something and the communication goes to pot.
Usually, you're just transmitting as if it was white Gaussian noise and you usually view microwave towers sending microwave as being almost equivalent to wire line communication.
So there's nothing very interesting there.
Now if the receiving antenna starts to move -- at this point, we are transmitting from a fixed sending antenna.
We have a receiving antenna, which for example, is in a car where somebody's running along, driving a car with their feet and talking on two cellular phones at the same time -- and the car's driving along at 100 miles an hour and something's going to happen soon, but it hasn't happened yet.
At this point what we're interested in is not the response of some fixed place r, but what we're interested in is the electromagnetic field in the absence of receiver at this point, which is moving.
We're interested in the electromagnetic field at a point r 0 plus v2, where v is the velocity of this vehicle.
We'll assume for the time being that the vehicle is going directly away from the transmitting antenna.
If it's going at some angle, it just changes these equations a little bit.
And the electric field there, before we put the car in the car changes all of the field patterns, but just changes it in a local way again.
Just like when we put the receiving end antenna in a fixed location, it changed all the local field equation, but it didn't change anything globally.
Again, what we have when we put in the receiving antenna -- is now the electric field at a point r, which is varying with time.
There's a time dependence with this now.
1 over r 0 plus v.t.
times the real part of this antenna pattern -- which we'll assume remains fixed -- times e to the 2 pi if times t minus r 0 plus vt over c. This is for a velocity away from the antenna.
That's just what this same equation says, but we can now interpret this nicely if we take the vt over c and combine it with the ft here and then we get something which looks like this antenna pattern again -- e to the 2 pi i f times 1 minus v over c times t. This v over c here is just this term here.
It's coming down to there.
Nothing mysterious has happened here, but what you see is this well known phenomenon called Doppler shift.
If you throw some screaming person over a cliff, what you'll hear coming back to you is a scream in velocity much smaller than the actual scream that the person is actually transmitting to you.
You're all familiar with this.
You're familiar with having planes fly overhead and when you hear them coming towards you, you hear a higher pitched sound.
When it passes by you -- this is a nicer example than throwing somebody over a cliff, obviously -- and you then hear a lower frequency sound as the plane starts moving away from you.
This Doppler shift is a well known phenomena as far as sound is concerned.
The same phenomena exists with electromagnetic radiation.
And there's nothing more to it than just this -- it's just that as you are transmitting from here to a point which is moving away, it keeps taking longer for the electromagnetic wave to get out to there than it takes to get here, so if you look at the wave fronts going along, the peak of the wave as it travels along, the peak of the wave takes a little longer to get out here than it took to get here, which means that from the viewpoint of the receiver, it looks like the receive frequency is much smaller than it was when it was actually being transmitted.
We get this thing called the Doppler shift.
Just to get some idea of the magnitude of this Doppler shift, what you're interested in is the speed of the vehicle divided by the speed of light.
That's the relative change in the frequency that you observe.
The situation here is quite different than it is in sound.
Sound travels rather slowly.
Light travels pretty fast and therefore, you need a really rapidly speeding vehicle to make this be any appreciable fraction of one.
So it looks like this is a very small effect.
The trouble is, what you are multiplying this effect by is the carrier frequency, which can be up in the gigahertz range.
To look at it another way, we are looking at situations where the wavelength is small fractions of the meter.
What this equation says is that any time this receiving vehicle moves by one wavelength, namely a small fraction of a meter, the crest of this wave goes from maximum down to minimum back up to maximum again.
In other words, in a quarter wavelength, it will go from maximum down to zero.
What you are observing at this carrier frequency is very, very different and it keeps changing rather rapidly.
All these other terms are just junk, of course.
This f times r 0 over c -- all that is is just a fixed phase difference, so we don't care about that.
This is just some fixed term.
This quantity here is changing with t also.
If we're thinking of a distance away, it's several kilometers and we're thinking of the amount of time for this to become an appreciable fraction of one.
For this to becoming an appreciable fraction of this, that's a pretty long time.
The amount of time for this to change appreciably is seconds or minutes.
The amount of time for this to go through a wavelength change is milliseconds.
So despite the fact that you see this sitting in an important place down there, this is not important.
Everything that goes on as far as fading is concerned is tied up with that term there.
This is important.
So we now have a system which is linear.
We still have the linear field equation, but it's not time invariant anymore.
It's changing with time.
The response is changing with time.
You send an exponential, what you received.
If you have a linear time invariant system, when you send an exponential of frequency f, you receive an exponential with frequency f. The only thing that a linear time invariant system can do is change the phase of that signal and change the amplitude of it.
Can't do anything more complicated than that.
That's why we love to study it; because it's so simple.
Now we have something more.
We have a system that can also change the frequency of what's getting received.
This small change down here -- and of course, if obstacles get in the way or something then there's this huge shadowing difference and all those important things, but you can't do anything about that.
You can do something about this, which is why we're focusing on that.
Let's go to the next example.
Incidentally, that example is no problem at all for communication.
I'll show you why in a little bit.
You can get around that problem very, very easily and I'll show you why.
This is a problem you can't get around so easily.
Here we have a vehicle, which is travelling, say, at 60 kilometers an hour.
Person's talking on his two cell phones, has his eyes closed because something surprising is happening, there's this big reflecting wall right in front of him and he doesn't see it at all, so he's speeding into this wall.
We're going to analyze this problem right before he hits the wall.
We have two paths here.
We have one path which is the path from here out to the vehicle, which has a length, r of t -- this is the distance away from the sending antenna to the receiving antenna.
We have another path which has a length d, then it gets reflected and this distances is d minus r of t. The total length -- and you're adding up this length with this length -- is 2d minus r of t. The reason is -- do I have an extra picture there?
No, I didn't make my extra pretty picture.
The reason is that one way to deal with electromagnetic radiation is when you see a wall, the thing that happens is that you get a reflection which is coming back this way.
The reflection has a strength which is equal to the radiation that you would get if there weren't any wall -- uhhuh except, of course, that you've changed directions.
In other words, this wall has generated a new plane wave which is going backwards, which is just enough to make the electric field strength on this wall equal to zero, because we're assuming a perfectly reflecting wall.
You can satisfy Maxwell's equations by having this incoming electromagnetic wave.
You would like to have an outgoing electromagnetic wave, but you can't do that because there's no way for the wave to get through it.
The only way you can do it is to generate a new wave, which is moving backwards, which cancels out the incoming wave right at this point.
We really have a path here of length 2d minus r of t and as a result of that, the electric wave has two components.
One is the component we were dealing with before where there's this Doppler shift because this is moving away from the sending antenna.
The other term -- in fact, we are moving closer to the wall and the distance in this path is getting shorter and shorter as doom approaches.
Here we have a positive Doppler shift.
Here we have a negative Doppler shift and we have these junk terms in both places.
One is fr 0 over c. One is 2d minus rc over c. Here we have r0 plus vt.
Here we have 2d minus r0 zero minus v.t.. As we said before, this term and this term are not changing very rapidly.
It makes it a little easier to analyze this if we say, let's suppose that this is equal to this.
In other words, we'd like to look at this right before the car strikes the wall.
That also is where this approximation is best because for those of you who have studied electromagnetism, you know that if a plane wave impinges on a wall, funny things happen.
If you look very far away from the wall, you will find this electromagnetic wave, which looks like a plane wave if the wall is distant from the source.
So this electromagnetic wave coming in, there's this wall in here and what happens to the electromagnetic radiation is outside of the wall that's going to go out past the wall -- and because of Maxwell's equations, it just sort of gathers together beyond the wall and it sort of comes together.
What you find is a disturbance, which is just around the wall.
Far away from the wall, you get the same electromagnetic radiation that you had before and close to the wall you have this disturbance.
If you look at the situation -- here it is.
If you look at what happens here and the wall is not big enough, if the wall is very small, this reflection is really going to look like what happens when you have an electromagnetic wave hitting the wall and the wall then re-radiates an electromagnetic wave, which very far away, this wall just looks like a point source.
What you have is instead of a 1 over r, 1 over 2r minus 1 over 2d minus r attenuation, you have a 1 over d attenuation multiplied by a 1 over d minus r attenuation.
If you didn't get all of that, fine.
Doesn't make any difference.
The point that I'm trying to make is that this analysis is really limited to the case where there the wall is rather small, where the wall is very large, because otherwise you won't have just this plane wave radiation effect.
What we wind up with these two terms instead of one term.
If I throw away all of the phase terms and I assume that the denominators are equal -- I'm going through this for some sort of reason -- I wind up with two sinusoids: e to the blah and e to blah plus.
When I take the real part of the sum of two sinusoids, and I look in all of the high school books I can think of about elementary geometry and playing around with sine waves, what I find is that this collapses into 2 alpha times the sine of 2 pi ftv over c times the sine of 2 pi ft.
In other words, it collapses into a sinusoidal term, which is the major part of this term, ft, and the major part of this term.
In other words, I can cancel out the terms here that are the same as the terms here.
When I cancel out those same terms, that's the term that comes out.
When I look at the other terms, I get an e to the minus vc over t and an e to the plus vc over to t. When we look at e to the minus vc over to t and e to the plus vc over t, even I know how to deal with that.
It looks like either a cosine term or sine term, depending on whether the sines in the same or the sines are different.
What we have at that point is this sinusoid is really a sinusoid at the carrier that we're transmitting at.
This term here is really something which expands and contracts slowly.
So it's a beat, which says that if you're transmitting from this source at this receiver, what you're hearing is something which contracts, expands, then contracts, then expands again.
There's nothing you can do about that problem either.
There's just no energy there part of the time.
This sine term here is running along, sort of changing from maximum to minimum at a few milliseconds time period.
If you have this vehicle traveling at 60 kilometers per hour -- you just work out the numbers there with the velocity of light and all of that stuff.
You find that you really can't communicate over your cellphone in that situation, because of these peak frequencies.
It's too fast.
It's too slow to ignore and ride over it and it's too fast to be able to get all of your data transmitted before it happens.
It sort of is a catastrophe.
So that's what happens because of Doppler shift.
You get a response which is periodically fading at the Doppler frequency.
This is called multipath fading or fast fading.
It's called fast fading because it happens so fast.
It's called multipath fading because it happens because of multiple paths, which each have lengths which are changing relative to each other.
Let's go back and look at the thing we had before, where we just had a moving antenna.
Here you have a Doppler shift also.
Why doesn't it bother you?
Here you have this Doppler shift and you're transmitting, let's say, a gigahertz and what the receiver is getting is the gighertz minus -- perhaps a kilohertz or something.
So why isn't that a problem?
If I demodulate at the carrier frequency, I'm sort of in bad luck because I have a signal then which is changing very rapidly.
I have something which looks like a time varying system.
But what's going to happen?
If I use the same kind of frequency recovery system that we talked about earlier, that frequency recovery system has all the time in the world to track that frequency which is one gigahertz minus a kilohertz.
It can track it perfectly, which says so what happens is we start out with a signal.
We move it up in frequency by one gigahertz.
The Doppler shift moves it down by a kilohertz.
We track that frequency, then we move it down again by a gigahertz minus a kilohertz and everything works fine and nobody even knows that there's any Doppler shift there.
So the problem is not Doppler shift.
The problem is multiple Doppler shifts which are at different speeds relative to each other.
That's an important point and we will come back to it as we move along.
If you put all of those phases that we neglected in the analysis that we just went through, this is the equation that arises.
I write this down not because it's important, but because the notes -- this is in lecture 20 -- have an error.
In the sine term, it fails to put an i in, which should be there and this is the correct term and that is off like ninety degrees.
If you write this down, you will then see what's going on.
Equation 7 in the notes has an e to the 2 pi f.t.
minus f.d.
over c, which is not the right thing.
As we said, the fading is due to Doppler spread between different paths.
The single Doppler shift does not bother us at all.
If you have a vehicle which is traveling away from the sending antenna and you have some kind of reflector, which is not something you're running into, but which is a reflector above, a reflector below or something like that, the thing which is going to happen is that both of those paths then are going to have roughly the same Doppler shift in them and if they both have roughly the same Doppler shift, it's not going to be this kind of beat cancellation that we have here, which is something you really can't get rid of.
The other thing that this points out again is that this variation is going to be in terms of minutes or seconds and anything you're doing to track the signal is going to be adequate for that until you get to the point where there just isn't enough energy anymore and then of course you have to move to a different base station or something else.
Want go through one more example of electromagnetism because it's so surprising -- at least, it was surprising to me when I found this out.
If you have a sending antenna -- think of this as a base station, which is high up at about maybe 15 meters or something.
It's sending to some receive antenna, which is at some height above the ground -- usually quite a bit smaller.
Suppose there is some more or less partly reflecting plane, like a road.
Here we have a vehicle which is travelling along a road and we have ascending antenna, which is also close to the road, which is sending the signal so we have two paths, one which is the direct path from sending antenna to receive antenna.
The other is a reflecting path which goes down here and comes back up again.
The rather surprising thing here, the thing that I couldn't believe when I saw it, because it contradicted all of my intuition, is that when r gets quite big, the difference between these two path lengths goes to 0.
Is that surprising to anybody else?
Anybody awake enough to be surprised?
This difference in path length here really goes down as 1 over r. That's an easy geometric problem to solve.
You just write down what this length is and the sum of these two lengths and you will find when you do it that the difference is proportional to 1 over r. What happens then is that as r gets big, these two path lengths get closer and closer together.
As they get closer and closer together, eventually they're much closer than one wavelength to each other.
When they're much closer than one wavelength to each other, the thing that happens is that we get a reflection in the electric field here, so.
This field and this field are going to be canceling each other.
They'll be canceling each other, except for this phase difference which is proportional to 1 over r, and the phase difference which is proportional to 1 over r is the only thing that gives us any power here at all.
As we move further and further away, that phase difference goes down as 1 over r, which means what's happening is the overall electric field that you receive here, instead of going down as 1 over r, which it would in free space, is going down as 1 over r squared.
Since it's going down as 1 over r squared, it means that the power that we're receiving is proportional to 1 over r 4th instead of 1 over r squared.
The analysis of this is not particularly important.
Why it is that surfaces such as macadam reflects so well is not particularly important to us either.
The point is that when you look at all the problems of electromagnetic radiation, in actual situations, you find some situations which behave like this, you find some situations which behave like this nice plane wave and free space.
You find other things where in fact the radiation goes down as 1 over r 6th, rather than 1 over r squared, so that we wind up with each path has its own particular kind of attenuation in with path lengths and it's kind of hard to figure out what all of those are.
Then you go on and say that things are even worse than this because sometimes you're communicating through a wall which is only partly absorbing and if you're transmitting through a wall which is partly absorbing, then the attenuation is exponential in the width of the wall.
You have a lot of different paths, all of which are very messy electromagnetic radiation problems and all of which have attenuations which range from 1 over r squared to 1 over r 6th, sometimes with an exponential thrown in for good measure, which says that if you really try to find the electromagnetic field at a wireless cell phone, you're in very deep trouble.
It's certainly not something that your cellular phone is going to solve for you and it's certainly not something that you're going to have solved ahead of time and program into your cellular phone or into the base station or anything else.
The question is, what do we do about this?
One thing is, we're not going to study these electromagnetic phenomena any further because it's a losing game.
If the thing that you're interested in is finding out where to place base stations, all of this kind of analysis is useful and you can go much further with it and you should go much further with it.
If your interest is in, how do you build third or fourth or fifth or 20th generation wireless systems?
All the electromagnetics that you study is only going to give you gross ideas of what kind of phenomena you have to deal with.
We already have some idea of the kind of phenomena we have to deal with.
We have to deal with paths which have different attenuations on them, which have different propogation delays on them, and all of these multiple paths are things that we have to somehow deal with without analyzing them in detail.
The question is, how do we do this?
Everything that we've done so far is called rate tracing.
In fact, even with all the complexity that we're dealing with now, we have highly oversimplified it.
Each of these paths that we have is going to give rise to an attenuation.
We'll now call the attenuation beta of j of t and a propogation delay, which we'll call passive j of t and the propogation delay is just what we get by assuming that a plane wave is going from source to destination and we add up the distance from source to reflector to destination and that gives us this propogation delay.
These are going to vary with time, but in our rate tracing approximation, we've assumed that they're independent of frequency.
I originally assumed that the antenna pattern was a function of frequency.
We didn't want to say anything about that.
So we have a total of capital J paths.
we put in an input of cosine 2 pi ft, what's going to come out is an electromagnetic radiation, which is a sum of these different attenuation factors times e to the 2 pi ft minus this propogation delay.
Everything you can do with ray tracing is included in this formula.
What you might be able to do in a wireless system is you might, by looking at the received wave form and knowing things about the transmitted wave form, you might be able to figure out what these attenuation factors are and what these propogation delay factors are.
Just like when we tried to do frequency recovery, we can find out what the transmitted frequency was.
We can play the same sorts of games here, but they're harder and we'll talk about that later.
If you ever hear the term rake receiver, a rake receiver is a receiver that in fact measures all this stuff and responds to it and we'll talk about that probably next Monday.
If we want to look at that the reflecting wall just as an example of what this formula means: beta 1 of t, namely for the direct path.
We have an attenuation, which is the magnitude of the antenna patterns divided by r0 plus vt. For the attenuation on the return path from the wall, it's the same alpha.
Why is it the same alpha?
Because we assumed it was the same alpha to make things simple for ourselves -- divided by 2d minus r0 minus vt.
If you look at these propogation delay terms, the propogation delay terms are r0 plus vt divided by c and this gives us the Doppler shift that we're interested in here.
We're also going to have an extra term here, which is really caused by the phase change at the transmitting antenna and the phase change at the receiving antenna and a phase change at a reflector, if there's any there.
We have the same sort of term at both of these places.
The reason that I talk about that is if you look at the electromagnetic wave that you received for this reflecting wall problem that we've talked about a good deal, the second term is there with a negative sign rather than a plus sign.
Here, everything is put in with plus signs.
You can create negative signs by phase changes of pi.
So the assumption is, we put in a phase change of pi as part of this term here.
So all of these terms can be expressed in this general forum here.
As we said before, you only have two choices with a cellular system.
You cannot solve the electromagnetic field problem at the cell phone.
The person using the cell phone is not going to do it.
The cell phone is not going to do it.
The base station is not going to do it and you're not going to store all those changes because these radiations change remarkably within a period of just small fractions of one meter.
You have a coverage here, which in fact, is at least area coverage of one kilometer times one kilometer.
Then the reflectors are going to be moving also, so you can't deal with them very easily either.
It's a hopeless problem, to try to solve the electromagnetic problem and store it some place.
Electromagnetism helps us to limit the range and likelihood of choices, but it doesn't help in actual detection.
So we're now going to deal with the kind of thing we just talked about, which is this sort of general expression for electric field in terms of attenuation factors and phase changes, as opposed to anything which is wh much more detailed.
We're going to define a channel system function as just this sum of these attenuation terms times phase change terms.
The reason that we're doing this is that if we put in an input -- e to the 2 pi ift, then what we get is this system function here -- times x of t, which is e to the 2 pi ift, so we get this quantity here.
Here's the minus 2 pi if tau -- that's that term coming down there and here is the e to the 2 pi ift coming down here.
So all of this term and this term are both included in this system response term.
This is linear also, so we know what the response is to an exponential that we put in.
I'm cheating you a little bit here by going from real to complex.
And the notes do that a little more carefully, but we ought to be used to that now.
If I put in an input, x hat of f e to the 2 pi ift df and integrate it.
In other words, if I put in an arbitrary input, which I represent in terms of its Fourier transform, I now know what the response is to every x-hat of f. It's given by this response here.
So I just integrate over that and I find that the response y of t to an arbitrary input now is the integral of x-hat of f, h-hat of ft, e to the 2 pi ift df -- namely, the same game that we always play.
Namely, that's just the system analysis way of looking at arbitrary systems in terms of their Fourier transforms.
So the output y of t is just this integral here.
Important point: When you look at this, this says this is the same as any old linear time invariant system.
This is not a linear time invariant system.
If you try to take the Fourier transform of this to get y-hat of f, you're not going to get x-hat of f times h-hat of f. Namely, this is not equal to that.
Why isn't it equal to it?
First reason is, try to take the Fourier transform of this and see what you get over here and when you deal with the fact that there's a t in here, you will find that there's nothing you can do.
You'll find you're stuck.
So you can't derive this equation here.
Next, argument is this quantity here is not a function of t. This quantity here is a function of t. You can't have a quality between something which is not a function of t and something else which is a function of t. It just can't happen.
The final argument is, if you look at what's happening here, when you put in a single frequency -- when you put in x of t equals e to the 2 pi ift, what comes out?
In terms of this reflecting wall example, the thing that came out was not one sinusoid, but two sinusoids.
One sinusoid a little bit above the carrier, the other sinusoid a little bit below the carrier.
In other words, because of Doppler shifts, when you put in a sinusoid, what comes out is not a sinusoid, but a modulated sinusoid.
It's something spread out over a region of frequencies.
So one of your favorite tools for dealing with linear time invariant systems is no longer adequate.
Doesn't work.
Just make a mark of that, because every time you see a problem like this, everytime I see it, the first thing I try to do is go through that most familiar and most favorite form of linear system analysis, which is the Fourier transform of convolution, is the same as multiplication in the frequency domain.
You cannot do that anymore.
However, convolution still work, so that's the next thing we want to look at.
So the thing that we have is the output of the system is now going to be this integral over frequency -- x-hat of f times the frequency response function for the linear but time varying system, times e to the 2 pi ift.
This is what we derived on the last page, on the last slide.
It is the thing which just automatically happens here.
If I let h of tau and t be the inverse Fourier transform of h-hat of ft -- and here what I'm doing is I'm regarding t as a parameter.
So this is a function of tau now and this is a function of tau for a given t, I can take the Fourier transform of this.
Take the Fourier transform of any old thing at all, so long as it's L2.
I will sort of half-pretend it's L2.
We'll worry about that later.
So this is the Fourier transform of this.
So then, the thing that happens is that y of t is going to be equal to this quantity here, except in place of the system function, h and f of t, for a particular value of t, I'm going to put in this inverse Fourier transform.
So it'll be h of tau and t, e to the minus 2 pi i of tau d tau.
This integral here is just h-hat of f and t. If I take this quantity here and I move this term inside and I interchange orders of integration -- incidentally, when we're dealing with wireless, we're going to forget about all of the nice things that we know about L2 functions.
There's just too much new stuff that's going on here to worry about that.
So what you want to do is just take Fourier transforms like while, interchange orders of integration, interchange everything you want to and simply forgot about all the mathematical problems that might arise.
After you understand this, at that point, go back and straighten out the mathematical issues.
This in fact is the way we deal with any problem, or the way you should deal with any problem.
You don't bring the mathematics in unless it's going to help you solve the problem.
You don't bring it in to frustrate yourselves.
So the thing we're going to do now is to interchange these orders of integration.
We're going to integrate over tau on the outside and f on the inside.
So we're going to bring the function of tau outside.
This is an integral in tau.
The function of f is going to go on the inside.
we have e to the 2 pi ft -- that's this quantity here.
We have this term there.
When we look at this, we see something very nice because this is in fact just the Fourier transform of x of t minus tau.
When we take that out, what we get is the integral of x of t minus tau times h of tau and t be tau.
In other words, this is time varying convolution.
Nice, simple equation, makes a lot of sense.
It says you have this impulse response here.
h of tau and t now can be interpreted as the response at time t to an impulse tau seconds earlier.
If you have a system which is changing very, very very, slowly, then this is essentially just a function of tau and it's the usual impulse response that you're familiar with -- namely, this convolution equation gives you the response at time t to an impulse tau seconds early.
Now we just have something which says this is a linear time varying filter and in all the cases, we're interested in this linear time varying filter, changes its impulse response very slowly as time changes.
Relatively fast change with tau, relatively slow change with t. So this is very similar to linear time invariant convolution.
Channel behaves like a slowly time varying filter and that's the bottom line of this.
For these ray tracing models we were looking at, the system function is a sum of terms at the sum of attenuations times phase change terms.
If we take the inverse Fourier transform of this -- remember, we're taking the inverse Fourier transform on f and putting in a tau.
So we're taking this inverse Fourier transform for a particular t. We have a function of f and of t. We take the inverse Fourier transform with respect to the f and get a tau here.
This then becomes this quantity here.
How do I interpret that?
If I look at a single term here, what is it?
A single term here is just an attentuation factor, a constant times a sinusoid.
At a particular value of t, this is just the constant here also.
So for a particular t, all I have is a sinusoid.
What's the inverse Fourier transform of a sinusoid?
I told you all along that it doesn't exist, but for the time being we will assume that it's what you learned early, that the inverse Foruier transform of the sinusoid is an impulse.
So here we are with our impulse there.
This says that the response at time t to an impulse at tau is going to be zero unless tau is equal to one of these propogation delay terms.
In other words, I have a system where I'm putting in an input and this input comes in.
The response to the input is a number of different path delays and at each path delay, what I'm going to get out of the system is just a delayed and attenuated version of what I put in.
Namely, the system isn't a function of frequency at all.
That's what I get through using ray tracing.
I mean, it's one of the consequences of using ray tracing.
So what I wind up with is a system function which is a string of impulses and the output then, the convolution of this with that -- you're probably better at working with impulses than I am -- and it's y of t is just the sum of these attenuation terms times x of t at these various delays.
So what we're doing is we're putting in an arbitrary input.
What's coming out is that attenuated input coming out at various different times, due to these various different paths.
I get various paths that are delaying the input by different amounts and out that delayed input comes at various times.
This is a nice sanity check because if you think about it, that's exactly what ought to come out of here.
On the other hand, you ought to wonder about this impulsive impulse response, because that clearly doesn't make any sense physically.
So what's going on here?
The thing that's going on is that when we started, we said, if we're putting in a narrow band input, we don't care about the frequency response because the frequency response on these different paths cannot change very quickly and therefore, we're just going to have a fixed frequency response term.
Then we've worked with that thing which is not a function of frequency and then finally we get down here, where in fact, what we're doing is looking at the output.
Due to a bunch of delayed input terms, if this input term here -- if x of t is in fact the narrow band term -- I guess the way to see that as to look at -- where do I look at it?
I want to look at this expression here.
If I have a narrow band or maybe -- I guess this one is better here.
Let's look at this expression.
If my input is in fact narrow band, it's only going to be non zero over a small range of frequencies.
If it's only non zero over a small range of frequencies, I don't care what this is, except over that small range of frequencies.
All of this gets filtered out.
In other words, this filters out this, opposite of the usual case.
So I don't care about what this is at different frequency ranges and therefore, we simply have the consequences of this, which is something that you see in linear system theory all the time.
We've sort of ruled out impulses and sine waves because they don't carry information, but in terms of looking at things as intermediate points and going through filters and things like that, they're perfectly fine So here, for simplicity, we assume that these channel filters really do not have any -- don't respond to frequency changes, whereas in fact they do, and all we're doing is modelling them in certain frequency bands, which, when we get all done, is what really gets rid of all our problems, due to this sort of input because this input is smooth now and therefore, the output is smooth also.
The next thing I want to spend a little bit of time on and we'll come back to it next time is, how do you deal with all of this at baseband?
I should warn you here that the notes don't do a terribly good job of this.
They have all of the results that you need.
They don't seem to put them in a very nice, well organized fashion.
I'm not sure there is a nice, well organized fashion to put them in.
But anyway there's a lot of stuff going on when you try to move this down from pass band down to baseband.
I will try to change the notes a little bit to make it clear, but I'm not sure that I can.
The kind of system that we're looking at now is our usual QAM type system, which can be generalized somewhat.
We have a binary input coming in.
We have a baseband encoder.
That baseband encoder is creating baseband signals, which are being added together to give us a baseband complex input to the channel.
This is being frequency modulated up to some function, x of t, which is just the real part of ut times e to the 2 pi if of ct as usual.
So we now have a real part of this signal here, modulated up by the carrier frequency.
This is going through what we'll now regard as a time varying channel filter.
We talked a little bit about channel filters before when we were talking about Nyquist theory, because we said in general, you want to take your input, you want to pass it through a pulse p of t. That goes through another filter, which is some h of t and that goes into another filter, which is at the receiver, which is q of t and you want the product of t and you want the convolution of all of those to satisfy the Nyquist criteria.
So we're back with that in spades because now this is varying with t also.
So this goes through this channel filter, up at pass band.
The channel filter up at pass band, we've seen that one of the things that it can be viewed as doing is putting Doppler shift into this input.
So what comes in at some frequency f is now going to be coming through here at some slightly different frequency.
We then add white noise to it.
We then get y of t out, which has now been shifted around and smudged in frequency a little bit.
We go through a frequency demodulation.
We get down frequency to modulation by this carrier frequency.
We get down to v of t, which is now a baseband complex function again, which is supposed to be the same as this except for the white Gaussian noise and except for the fact we've gone through this filtering operation here.
Then we want to do base band detection at this point.
What we would like to do and what will make life a little easier for ourselves because we all got sick of this business of looking at filters at baseband and also looking at filters at pass band, we would like to be able to take some baseband equivalent of this filter here.
So what we're going to do is look at the baseband equivalent.
The system function at baseband corresponding to this system at carrier frequency will just be this response moved down by s of t. In other words, when you take what comes out of here, you multiply it, and you shift it down in frequency by s sub c, what happens is that this gets shifted down in frequency by s sub c and this channel filter gets shifted down in frequency by f of c and therefore, what happens is the effect of passing y of t through a base band filter -- h-hat of f plus fc and t or 0 for f, less than or equal to minus fc.
So far, this is all kind of straightforward and not too mysterious.
So you wind up then in -- and this is pure analogy to what we did before -- the output is then going to be the integral of this Fourier transform of the input.
The system function for the baseband filter times e to the 2 pi ift df.
Same equation as we had before, but before we did it at pass band and now we're doing it at baseband .
For the ray tracing model that we looked at, this function here down at baseband is going to be the same as it was at pass band, except in place of the 2 pi i tau jft, we now have f plus the carrier frequency times tau jft.
So when we take the inverse Fourier transform of this, we're given parameter t. What we're going to wind up with is this quantity here.
This is the same as we had before, with the difference that now we're stuck with this carrier frequency times propogation delay, which is occurring in time t. We still have the same delta functions here and the output v of t is the same time variant convolution as we had before.
I'm doing this much too fast for you to follow this in real time.
What I'm saying is, this is exactly the same thing as we did before and the only thing new that happens is now this carrier frequency term is coming in here on this term.
If we're using the ray tracing model then, v of t is equal to this quantity here.
The reason I write this down is that this shows you quite simply what's going on in terms of the Doppler shift, because these terms here, these delay terms are changing with Doppler shift and now we're going to see exactly what they do to us.
So we're going to represent the propogation delay on each path.
That's tau j of t, which is tau j prime, which is the propogation delay of time zero minus the Doppler shift times t divided by f. This is just Doppler shift, which is a frequency term, and it's increasing linearly.
The propogation delay is increasing linearly with t. The Doppler shift is a shift in frequency and, as I think we saw before, you need a 1 over f to compensate for this Doppler shift.
Namely, this is in terms of hertz.
This is in terms of hertz.
This is in terms of time.
This is in terms of time.
So you need this frequency here to be dimensionally right at any rate.
If we're using narrow band communication, then v of t is just going to be this difference here, which is just the last equation with the Doppler shift put into it.
So all of this says that when we're now looking at things at base band instead of a pass band, what has happened is that this term has been added.
Before, we just had these delay terms here and we had these attenuation terms.
Now what's happening is because we are modulating up with carrier frequency s sub c, we then get a Doppler shift that moves us down a little bit.
We're then moving down by s sub c. What we wind up with is something which is off by that Doppler shift.
If we only had a single term, we wouldn't be demodulating by s sub c. We'd be demodulating by something which would get rid of this term for us.
Then all we'd have is just some arbitrary phase term here, because this isn't important.
This is just a phase term.
This is the term which is important.
The reason for going through all of this -- and I'm going to come back to this next time -- is that when you use frequency recovery at the receiver and you always use frequency recovery at the receiver, what's going to happen is you're not going to shift down by s of c. You're going to shift down s of c, plus some average value of Doppler shift.
If they have a bunch of terms, each with different Doppler shifts in them, when you try to measure how much Doppler -- when you try to measure what the received carrier is, you're going to be frustrated by all of these different Doppler shifts.
You're going to come up with some average value in your frequency circuit, which means that in place of this quantity, you will get something which replaces each of these D sub j's by not the actual Doppler shift, but by how far this Doppler shift is away from the main Doppler shift.
So these terms here, which are the things that make this system function in here change with time, are in fact changing according to how far away each of these Doppler shifts are from the main Doppler shift, which means that the amount of time this system function in here is going to remain stable depends on how much the Doppler shifts vary from each other on these different paths.
In other words, the important thing is the Doppler spread between the biggest Doppler shifts and the smallest Doppler shifts.
That Doppler spread is going to determine how long you've got something that looks like a linear time invariant system function which says that every once in awhile, if you're trying to measure what's going on at the channel, at intervals of time approximately one over two times the Doppler spread, you're going to have to change those measurements.
So whether you can make cellular telephony work or not depends on whether you can make measurements at a speed which is equal to one over two times the Doppler spread.
I'm going to do more about that next time.
I don't expect you to understand it now.
Maybe after you read about it and we talk about it more, because in fact that's -- if you look at this question of Doppler spread and what its effect is on how long a system looks like it's time invariant, this is one of the key parameters to understanding how any kind of wireless system is going to work.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, I want to spend a fair amount of time, to start out with, talking about this business of, how do you go from pass band to a baseband in a wireless system and some of these questions about time reference and all of these things.
When we were dealing with ordinary channels that didn't have fading in them and didn't have motion or anything like that, it was a fairly straightforward thing to think of a transmitter having its time reference and the receiver locking in on its time reference, which had a certain amount of delay from what the transmitter was doing.
As soon as you start having multiple paths, each with different delays in them, this starts to get confusing.
If you try to write it down carefully, it gets doubly confusing.
If these path lanes are changing dynamically, it gets even more confusing.
So I wanted to spend a little bit of time at the beginning of the hour trying to sort that out.
It's not sorted out that well in the notes.
As you'll see, it doesn't get sorted out too well here either because the problem is, whenever you write everything down, it becomes a nightmare.
So this is just another effort at trying to do this.
Let's start out by just looking at a single path.
This single path, we're going to assume, has a time varying attenuation, which we'll call beta of t and a time varying delay, which we'll call tau of t. Namely, this is the same thing that we did before.
We started out with a fixed transmitter and a fixed receiver.
Now, since we're allowing both the attenuation and the delay to vary, but only having one path.
What we're essentially thinking is, that is the receiver, which is in a vehicle for example, which is moving away from or towards the transmitter.
So these are the, or in a sense an easy way of keeping track of that physical situation.
So the response at passband to a real waveform, x of t, it's going to be just beta sub t times x of t minus tau of t. In other words, here's the delay here.
Here's the attenuation here and we just have a single path.
Now if you start out with a baseband waveform, if you start out with a complex baseband waveform, say u of t, you're going to shift this up first to a positive frequency band, which we'll call u sub p of t, which is u of t e to the 2 pi ifct and you're going to transmit it as x of t, which is equal to two times a real part of that.
Half the time we've been dealing with going through this modulation from baseband to passband in terms of cosines and sines.
Half the time we've been doing it in terms of complex exponentials.
One of the problems that appears as soon as you get into wireless is that if you really like to do things in terms of cosines and sines, you really run into a lot of trouble here because with phases varying all over the place, it just becomes very, very difficult to track what's going on with all of the cosine terms and all of the sine terms.
So what we sort of have to wind up with here is this, in a sense, simpler but more abstract viewpoint where we think of modulation as being a two step process, where we first take the take the low pass waveform, move it up in frequency and then we add on the negative frequency part as an afterthought and then at the receiver, what we're going to do is have a Hilbert filter, which you would never build, of course.
You would always build it in terms of cosines and sines, but conceptually have a Hilbert filter which blocks off that negative frequency part and then you just shift things down in frequency again.
So that's the way we're thinking about this here, so we're going to have u of t shifted up and then transmitted as 2 times the real part of up of t and therefore, it's going to get received -- the received waveform at the passband.
We're still using the timing at the transmitter.
That's the purpose of going through all this analysis, to look at what's going on at transmit time and what's going on at receive time.
So in terms of transmit time, this received waveform at passband is just the attenuation term times x of t minus tau of t, which is two times the real part of theta of t times this positive frequency part.
Now when you write it out in all of its glory, it's 2 times the real part of the attenuation times the input you started with, but now this has been delayed by tau of t and e to the 2 pi ifct, but this was the carrier that was put on at the transmitter, physically at the transmitter and now we're at the receiver, so that carrier has gotten delayed by a factor of tau sub t. The question is, how do you demodulate this?
How do you come down again from transmitter to receiver?
What we've thought all along and what makes a lot of sense until you start putting these delays in, which are really a pain in the neck, is that what we're going to do is just multiply this positive frequency waveform by e to the minus 2 pi i f sub ct, but that's not what we're going to do because the receiver sitting there and the receiver has to recover timing and it has to recover carrier frequency.
So what's it going to do?
It's going to figure out what the timing is and the timing that it wants is at time 0, it wants to be seeing what the transmitter transmitted at transmit time 0.
In other words, if you look at what the transmitter was doing at time 0, it was sending, say, the first bit that was going to be sent.
In receiving this, we want to have our timing so we're looking at that first bit sent.
So our timing is going to be shifted from what it was then.
So we're going to take this new timing.
The receiver clock time is now going to be t prime, which is t minus this delay term.
If you get confused between the minuses and the pluses here, don't worry about it.
I get confused about them too and the only way I can straighten them out as to put down one or the other and then spend ten minutes looking at it and try to figure out -- after I write it down -- whether it should really be a plus or a minus and I think these sines are right, but I'm not going to try to argue why in realtime.
But anyway, this received waveform now, as a function of received time, is going to be the received waveform and in terms of transmit time -- and in place of t now, I'm going to have t prime so that since you t prime is equal to t minus tau of t, this is why it's t prime plus tau of t. In other words, what I'm looking at now, at time 0 at the receiver is what was being transmitted a little while ago.
So that's this term and that's going to be the real part of this, where I've taken account of all of these shift terms.
Now if you look at this carefully, you see that u of t minus t of t is turned into tau of t prime.
This term has turned into 2 pi ifct prime, so everything is fine there.
What I should have done with this term is to compensate for the fact that I'm now looking at it in a different time and at this point, what I'm going to do is to say, this quantity is changing so slowly with time that I don't care about that.
If you really try to adjust this to make it right, it's a terrible mess.
So we're just not going to worry about it.
So what we receive then is approximately equal to this where the approximation is due to the fact that I'm not evaluating this attentuation term at exactly the right time, because it's a pain in the neck to do so.
All of this stuff with wireless, you simply have to make approximations all over the place.
You have to make crazy modeling assumptions all over the place, because the physical medium is so complicated that you can't do much else and the whole question is trying to make the right approximations and get some sense of which thing's very fast and which thing's very slowly.
So that's the equation that we had.
What we receive now at passband, but at the received time scale is this quantity here.
So what's the receiver going to do?
The receiver, at its time is going to demodulate by multiplying by e to the minus 2 pi ifct prime.
First, we're going to take away this 2 times the real sine by going through the Hilbert filter, which just removes this.
At that point, we have a complex positive frequency waveform and then we're going to multiply that positive frequency waveform by e to the minus 2 pi ifct prime, where t prime is the only thing the receiver knows anything about.
So after you shift the baseband with a recovered carrier in receiver time, what you get in receiver time is v of t prime as equal to beta of t prime; namely, the attenuation at t prime -- or this is the approximate part of it -- times what was actually set.
So now, think of this as saying, suppose this delay term is actually changing with time.
Namely, it's some tau 0 minus a velocity term, vt divided by the velocity of light.
In other words, the time delay that you incur, where the change in the time delay that you incur, is due to the time that it takes light to travel from where the receiver was to where the receiver is now.
We can also write that as tau 0 minus the Doppler shift times the time divided by the carrier frequency.
Now here's another approximation, because what we're sending -- I mean, this quantity here is exact.
When you try to convert from the velocity, which you're going to, for what it does in frequency, which is the Doppler shift, it's a function of the frequency.
Here what we're assuming is that all the frequencies we're dealing with are so close to the carrier frequency that we can just ignore everything else.
So we're just writing this as the Doppler shift divided by this carrier frequency.
Now if you view the passband waveform, going into a baseband of modulation and you think of doing this in transmit time, what are you going to do?
In terms of transmit time, you've got to get rid of not this term, but this thing here.
So in terms of transmit time, you're going to be multiplying by e to the minus 2 pi i times carrier frequency minus the Doppler shift.
So the thing that's happening is that time at the transmitter gets spread out a little bit at the receiver.
In other words, as this receiving antenna is moving away from the transmitter, a period of time like this at the transmitter is a period of time like this at the receiver and therefore, a certain number of cycles of carrier at the transmitter looks like a different number of cycles of carrier at the receiver.
Therefore, if you're dealing with transmit time, you're taking account of this Doppler shift.
You're multiplying not by the carrier frequency, but by the carrier frequency minus the Doppler shift.
If you do it in receiver time, you're just multiplying by the by this carrier frequency, because the trouble is, your clock is running a little slow.
Because your clock is running a little slow, you think you're demodulating at the actual carrier frequency, whereas in terms of what the transmitter thinks, you're doing something else.
If you all got confused when you studied relativity, this is exactly the same problem you got confused about there.
There's nothing very mysterious about it or hard about it, except that to most of us, time is a very fundamental quantity and you don't like to monkey with that.
If you can't count on time being what it's supposed to be, you're really in deep trouble.
When you try to write equations that do that, it makes things very, very tough also.
So anyway, the result after we look at this in both ways, is that you get the same answer each way, but the receiver sees the carrier frequency and sees this carrier frequency in transmitter time and it sees this frequency in received time.
Of course, the receiver only sees received time because the receiver sitting there trying to measure from the waveform coming in what that carrier frequency is and what that time base is.
So now it gets to be fun.
We look at multiple paths.
When you look at it in transmit time, what gets received at the receiver, according to the transmit clock -- so here we are at the transmitter and we're peering at what's going on at this distant receiver and you now have the thing that we've talked about in the notes where you have all these different paths.
Each path is associated with a certain attenuation and each path is associated with a time delay.
This can be written in terms of what has gotten modulated up from baseband as 2 times the real part of this sum -- of beta j of t times the positive frequency part of what got transmitted and then it now has this delay in it.
This can be rewritten in the same way that we did before, but now we just have all of these paths in there instead of one path.
If you look at this expression then, we have all of these propogation delay terms.
We have this baseband input, which has been delayed in different ways for each path and you have have this exponential, which has been delayed also.
So at this point, the receiver is in a real pickle because if the receiver is smart enough to see -- yes?
[UNINTELLIGIBLE PHRASE]
The receiver doesn't know anything.
It's like you're driving in your car, you're talking on your cell phone and your cell phone -- I mean, the only way your cell phone knows that you're moving is that the cell phone is getting some waveform coming in and it has to tell from that waveform what's going on.
So it's going to do things like the carrier tracking that we were talking about before and it's not going to have too much trouble because all of these changes are taking place relatively slowly, relative to this very high -- either one gigabits or two gigabits or four gigabits, or what have you, carrier frequency.
So all these changes look like almost nothing, but at the same time, they're fairly important because they keep changing faces.
But anyway, in terms of the transmitter clock, this is what you actually receive.
So at this point, what the receiver has to do is it has to retrieve clock time and the clock time that it retrieves is going to be -- receiver clock time is -- is t minus tau 0 sub t -- so it's t prime equals t minus tau 0 sub t. That's the same thing it was before, except that this tau zero of t here doesn't really amount to anything physical anymore.
It's just whatever the circuitry that we have trying to recover carrier and trying to recover timing -- it's just whatever that happens to come up with, so with this best sense of the timing it should use if it's going to try to detect what the signal is.
So it's some arbitrary value and again, this is varying in time.
So again what we have is now this expression becomes really messy, because instead of having a receiver time variation, which cancels out what the actual variation in path length is, you really have two terms that are different and you have the same thing in terms of this phase, which is changing and this is the more important term.
But anyway, you have that.
You can then demodulate.
What do you do when you demodulate?
You take this thing in receiver time and you multiply by e to the 2 pi i, e to the minus 2 pi i -- this carrier frequency, what you think it is, times this time reference, what you think that is -- So that term has disappeared and what we then wind up with is the carrier frequency times these two terms here.
That's to the left there.
These two terms turn out to have both Doppler shift terms in them and also time spread terms.
So when we write that out in terms of Doppler shift, what we wind up with is the input, which is then delayed by some arbitrary amount of time, where this is our best estimate of the right time to use and this is the actual time on that path.
The thing that's happening here, which you don't see in the notes unfortunately, in the notes, it pretty much looks at things as far as transmit time is concerned.
Therefore, what you see is expressions for impulse response where the impulse response is at some time much later than time 0.
You have a -- I mean, you put in an impulse of time 0, what you see is stuff dribbling out over some very tiny interval, but a good deal later than what got put in.
What we want to do now is to shift that so that in fact, when we look at filtering and things like that, we have a filter which starts a little bit before 0 and ends a little bit after 0 and that's the whole purpose of this.
If you don't think my purposes just to confuse you, it really isn't, although I realize this is very confusing, but you sort of have to go through with it to see why these time shifts appear in the places where they do.
But anyway, up here in the phase, you wind up with the Doppler shift terms and down here, when you're worrying about the input waveform, you're worrying about these time shift terms.
We'll see we'll see how they come in a little later.
To make the expression a little bit easier, let me look at tau j prime of t as tau j minus tau 0 and let me look at the Doppler differential as the Doppler that actually occurs on path j minus what the receiver thinks the Doppler is.
This is this receiver term that we've demodulated by and therefore, we've really gotten rid of this term.
Then when we rewrite this received baseband waveform -- and I've now gotten rid of the t primes because everything we've done up until now, we've just automatically assumed that the receiver timing and the transmit timing, we didn't have to be careful about it and it wasn't until we got to wireless that we do have to be careful about it.
So now we're throwing it out again and what we have is these attenuations, which are very slowly varied.
These terms, which are now delayed by this difference between the actual delay and what the receiver is assuming that the light ought to be.
So these are really just differential delays relative to each other.
These are Doppler shifts relative to each other.
So that's the whole receive baseband waveform then.
If you look at this, it's not apparent where the carrier frequency is coming in and the carrier frequency is coming in because these Doppler shifts have the carrier frequency in them.
Namely, the Doppler shift is the velocity times the carrier frequency divided by the speed of light.
One of the things you have to be very careful with in the wireless business, is that everybody thinks it's a neat idea to be able to be able to go up to higher and higher frequencies.
When you go from one gigahertz up to four gigahertz, one of the things that you have to deal with is that every Doppler shift you're dealing with is now four times higher than it was before.
So there's one big disadvantage in operating at extremely high frequencies.
It's not necessarily a bad thing, but it's there and if you have a bunch of different paths which are all acting in different ways, you suddenly wind up with some real problems.
The received baseband waveform on path j is going to change completely over an interval -- 1 over 4 times the magnitude of this differential Doppler shift.
The Doppler shift might be positive or negative, but if you look at this expression up here, that's the amount of time that it takes for this quantity to change by pi over 2.
When this changes by pi over 2, this exponential goes from its maximum one down to zero or it goes from zero to minus one or it goes from minus one to zero or from zero up to plus one, and that's a pretty major change.
So the interval which is required to do that is 1 over 4 times that Doppler shift.
The Doppler spread now is defined as the difference between the maximum Doppler shift and the minimum Doppler shift.
If we choose this Doppler shift D sub 0 that we had, because of what the receiver was doing, seeing all these different Doppler shifts and trying to come up with something in the middle, that's pretty close to what the receiver thinks it should be using as far as receive time is concerned.
So it's the maximum plus the minimum divided by 2 and then each of these differential Doppler shifts is going to be somewhere between that overall Doppler spread minus Doppler spread over 2 and plus Doppler spread over 2.
It goes from -- minus D over 2 up to plus D over 2 and this difference in here, is strangely enough, D. So what we've done is by adjusting our received timing -- and slowing it down or speeding it up if necessary, is we wind up with Doppler shifts that are sort of spread around from negative Doppler shifts to plus Doppler shifts.
What that means is, if you look at this expression, it takes a period of time, about -- here we had 1 over 4 times the differential Doppler, which is now 1 over 2 times this overall Doppler spread.
So what that says is, there's a coherence time on the channel of about 1 over 2 times the Doppler spread.
That's important because that says the channel is going to look like the same thing for a period of time, which is about 1 over 2 times this Doppler spread term.
I think this will be D instead of D 0.
So you find out what the Doppler spread is and the reason you want to find out what the Doppler spread is is that that tells you how long the channel remains stable.
Why do you want to know how long the channel remains stable?
If you're going to do detection or anything like that, if you're going to filter the received waveform to try to find out what the input waveform is, you would like to know what the channel is doing.
You'd like to be able to measure the channel.
If you're going to measure the channel, you really want to know how long it stays the same before you have to measure it again.
If you look at all these cellular systems and you look at every other wireless system in the world, all of them do one of one of two things: either periodically they measure what the channel is so they can use it and the other thing they do is every once in awhile, they send a pilot tone and they use the pilot tone to try to figure out what kind of channel they have.
This is the thing which tells you how often you have to send that pilots tone.
If you know about estimation theory, it's also the thing that tells you whether you have any chance or not of estimating what the channel is, because it takes you a certain amount of time in the presence of noise to estimate something and if that something you're trying to estimate it's changing faster than you can estimate it, then you don't have a prayer of a chance.
So yes, this is important.
This is the prime coherence.
It's one of the main parameters of a wireless channel you want to understand.
It tells you how long the channel will remain essentially the same before it changes and becomes something different.
Both of these quantities are very approximate.
If anyone wants to tell you that it's twice what I say it is here, I certainly wouldn't quarrel with them.
I mean, it's like talking about the bandwidth of a waveform them that you send.
I mean, you look at the spectrum of what you send and it's going to be something which sort of tails off.
If you want to call it the maximum out there where it's going to zero or the minimum, where it starts to go to zero or what have you, it doesn't make much difference.
Here, it's a worse situation because you don't shape this waveform.
It's just given to you and you find something which is going to be tailing off slowly.
It's going to have little bits of nothingness way out there.
All this is is just one very approximate number, which tells you what that Doppler spread is and therefore, which tells you how long the channel is going to remain stable and yes, in half of that time the channel starts to change and in twice that time, the channel might be still sort of like it was before.
So this is only a very approximate way of looking at these things, but it gives you an idea about what you can do and what you can't do.
It gives you an idea, for example, if you say, can I take this wireless system I've been using and use it at a frequency that's 10 times as high -- and you know when you use it at a frequency 10 times as high that the stability of the channel is going to be 10 times as small.
You know if you were having trouble measuring the channel before, you know you don't have a prayer of a chance when you go up to that higher frequency, so it so it tells you things like that.
There are two other sort of related quantities, which are called multipath spread and coherence frequency.
So we have Doppler spread on the one hand, which is a physical phenomena and that gives rise to coherence time, which says how long the channel will stay the same and in the opposite domain -- and these are almost dual quantities of each other, you have something called multipath spread, which -- and multipath spread is pretty much what it sounds like it is.
You have a bunch of different paths.
You send a very short duration waveform and things start trickling in on the shortest path and then you got a big bunch of stuff in all the paths which are sort of the same lanegth and then they sort of dribble away again.
So the waveform that you see coming in is spread out from what you send because of these multiple paths and the multiple paths having different delays on them.
Because you have all of these different delay paths, you're going to have an impulse response for this channel, which has a certain duration to it.
When you take the Fourier transform of that, you find a certain duration for Fourier transform.
As we're going to see in just a minute, that's the thing that tells you how stable this channel is in frequency.
If you measure it at one frequency, how far do you have to go in frequency before you find something which is totally separate from what you measure down here?
This is something the cuts both ways.
Having a large frequency coherence and having a large time coherence can be good or it can be bad.
If you have a very large time coherence and a very large frequency coherence, you can measure the channel once and you can use it for a long time and you can use it over a broadband width and everything looks very nice.
If the channel happens to be faded over that long period of time and over that long interval of frequency, you're really in the soup.
On the other hand, if you have a very small time coherence and a very small frequency coherence, if the channel doesn't look good there, you move to a different time interval or you moved to a different frequency interval and you can transmit again.
So both of these quantities have good aspects to them and bad aspects to them, but if you want to evaluate what a physical wireless channel is and somebody lets you ask two questions, the two questions you ought to ask are, what is the time coherence and what is the frequency coherence?
Those are the first things you want to find out.
Everything else is sort of secondary.
So trying to claim that the frequency coherence here almost has a matter duality, as 1 over twice the time spread.
Let's take a slide to see why that is.
If I take the impulse response at baseband of this channel, which I wrote down here -- here's the impulse response and you get that directly from the baseband response for giving input waveform.
With all these different paths, what this impulse response is at a particular time t -- namely, this is the response at time t to an impulse tau seconds earlier.
In other words, when I'm using this notation here, I'm really thinking of t as a receiver time and t minus tau as transmit time.
But anyway, this is what it is.
You take the Fourier transform of this for a particular t -- in other words, we talked about this a little bit last time.
You have a function of two parameters here and at any given t at the receiver, what you see is a spread of different inputs coming in from the transmitter.
What we're interested in now is at that particular time t at the receiver, what's the Fourier transform, when you take the Fourier transform on tau to see what kinds of frequencies are coming in?
We take the Fourier transform of a unit delta function and you just get an exponential function.
So the transform of that delta is just this e to the minus 2 pi if times tau sub j prime of t. This is this differential delay that we're faced with and we still have the Doppler shift over here.
So when we look at it this way, we have both the Doppler shift sitting there and we have this delay shift sitting up here.
Now the question we ask is, how much does the frequency have to change before this quantity is going to change materially?
The answer is the same as it was before.
When this quantity here changes by pi over 2, you have a totally different result than you had before.
So the question is, how much does a frequency have to change in order for f times tau j prime of t to the change by a factor of one-fourth.
So the answer that we come up with is, this has to go from minus L over 2 to plus L over 2 and this is where we get the result that we were just talking about, that the frequency coherence is 1 over 2 times the multipath spread of the channel.
The next thing we want to do in this long process -- it's a process that we've sort of been doing all along.
The whole purpose of looking at waveforms and looking at digital communication is to be able to go from sequences to waveforms -- and what do you think the next thing we're going to do is?
We're going to go from these waveforms -- we've now got them down to baseband waveforms and we want to go back to looking at sequences.
So what we would like to do is to take this input waveform, view it in terms of the sampling theorem -- better thing to do would be to view it in terms of input pulses, but that's too complicated for now, so we'll just take the input waveform and view it as a sequence of data that we're sending -- the u sub k's times these sinc waveforms.
Same thing we've been doing all along.
We take a sequence of numbers, we put little sine x over x-hats around them and that gives us a band limited waveform.
The baseband output is then -- you can look at the same way.
The baseband output is going to be limited to the same frequency.
Question here: You have these Doppler shifts.
So when I send a frequency which is right up at the limit of w over 2, what I get back from that is going to be spread out a little bit from that.
So if I send a waveform that's limited to w over 2, I'm going to get back a waveform which is a little bit bigger than that.
What do I do about that?
I worried about it in the notes a little bit and I'm probably going to take it out from the notes, because as a practical matter, the amount of Doppler shift that you get is such a negligible fraction of the kind of bandwidth you would be using for transmission that you just want to forget about that.
So we are going to forget about it and what's coming in is again a band limited function with maybe a little bit of squishiness on it.
We get this baseband output, which we can then view in terms of the sampling theorem.
We can sample the output and what do we get when we're all done?
Here's the nice thing that we can jump to: The output at this free time m is then the sum -- namely, it's the convolution, it's the discrete convolution of what went in at time m, at time m minus 1, m minus 2, m plus 1, m plus 2 -- and what we're doing is, we're designing this recovery time so the main peak is about a time 0, so k goes from minus something to plus something here and then we wind up with these channel terms, which are again, sampled at this point because we can sample them, because it's only these low frequency components of the channel response that make any difference.
They're the only things that give us this output here and this output has already been filtered down to this low frequency.
When I look what these components are, these are big messes again.
They're sums over all of these different paths.
So what we're winding up with is a function -- if I sketch g of tau and t as a function of tau, what I'm going to find is a bunch of different terms.
Now I'm going to filter this so every one of these turns into a little sinc x over x and then I'm going to sample it.
So what I wind up with then is this goes into something which looks like this.
It's going to look like something which goes up and comes down again, and I'm just interested in what it's sample values are and I might wind up with five or six sample values and that expresses the whole thing.
I mean, this is something you have to think about a little bit because it's something that we've done about 10 times already, but now when we do it here it looks very different from what we've done before -- except this is what it is analytically.
This is what it is in pictures and that's where it comes out.
The thing that's going on here then is we started to look at this physical modeling in terms of ray tracing and things like that and what we're doing at this point is we're modeling what's going on at the frequency bandwidth that we're using -- namely, not the carrier frequency anymore, but the baseband bandwidth.
We're viewing this filter as having filtered these things out for us so we wind up with a baseband discrete filter that filters these impulses and aggregates them under discrete caps.
So at this point, when we look at this filter for what the channel is doing, instead of seeing a bunch of impulses, we see a rather smooth waveform.
If we look at what contributes to each tap, we see a large number of different paths, which all contribute to the same tap.
If the multipath spread is not too large relative to 1 over w -- if if the multipath spread is small relative to 1 over w, then you can really represent this whole thing in one path.
In other words, if the multipath spread is very, very small and the sampling time interval is big, everything just appears under one tap and that's called flat fading.
When you have flat fading, the output that comes out is really just a faded version of the input and there isn't any time dispersion on it.
It all just -- it's an undistorted version of what got sent, except it's very slowly fading and coming back up again and fading again and coming back up again.
If you have a slightly wider bandwidth, then you need multiple taps.
If you look at most of the of the systems that are being built today, the largest number of taps they ever used in this kind of implementation is three to five or something in that order, so it's not a huge number of taps that we're talking about, because the bandwidths are not huge.
But anyway, statistically what we wind up with is that if the bandwidth that we're using is relatively narrow and it's not too many times what this term 1 over L is -- namely, this multipath spread, then we wind up with a small number of paths.
We can sort of assume that we have a large number of paths coming in on each tap and then what we're doing is that each tap strength is going to be a sum of a bunch of terms which are essentially random.
If you add up a bunch of complex terms that are essentially random, what do you get?
If you add up a very humongous number of them, you get something which is Gaussian.
If you were modeling these things statistically, you would like them to be Gaussian.
So what do you do?
You assume that they're Gaussian, which is what everyone does.
As a slight excuse for that, what we're really interested in if you're designing cellular systems, you're trying to build cellular receivers which will deal with all of the different circumstances that they come up against and as you walk around or drive around using these things, if you look at the ensemble of different situations that these phones are faced with, over that ensemble of things, each of these taps is pretty much going to look like a Gaussian random variable.
If you look at one sample path of one little cellular phone, then no, it won't look like that -- but as far as a design tool, it looks like something which you can model as Gaussian because you're going to view each of these taps in this filter, in this time varying filter, the sum of lots of unrelated components and therefore, we're going to take the density of these taps and a statistical model as being jointly Gaussian -- a proper Gaussian random variable for each of them.
What happens if different taps is jointly Gaussian?
So what we wind up with is a model for wireless channels then, where the channel itself is a Gaussian random process and as a Gaussian random process both in tau and in t, it's easier to look at if we look at it in terms of these discrete samples.
Namely, we're going to look at the channel now, not in terms of a wavelength type of thing.
We're just going to look at it as a bunch of taps, which the receiver has to add over to get to the received waveform.
We're going to model each of those taps as being Gaussian.
We're going to model in time t as being stationary.
We're going to model them in time tau as coming up and going back down again and existing over a multipath spread that's about L in duration.
So the phase is going to be uniform with this density.
This is this two dimensional independent Gaussian density that we've looked at so many times and if you look at this, it has circular symmetry to it, which means if you look at it in an amplituding phase, the phase is random, uniform between zero and 2 pi.
The energy in it is exponential and the magnitude is this, which comes from that -- which is what you call a Rayleigh density.
So Rayleigh fading channels are simply channels which you have modeled as having taps, which are really random variables.
Real and imaginary parts, each of them have the same distribution.
I mean, you'd be very very surprised if you looked at these taps after the way we've derived them and you didn't find the same distribution on the real part as the imaginary part.
If you look at the real part and imaginary part, they're simply coming as an arbitrary phase, which comes from some arbitrary demodulating frequency that you've used with a phase in there that doesn't have any physical connotation at all.
I mean, what is real is what you choose to call real and what is imaginary is what you choose to call imaginary.
If you made your time reference just a little bit, a smidgen away from where your time reference is, what is real would become imaginary and what is imaginary would become real again and you can't expect the channel to know what time reference you've chosen.
I mean, this is sort of a big philosophical argument, but you certainly wouldn't expect these random variables to have a distribution which is anything other than uniform in phase.
So that's the distribution we get and what we now have is we changed this awful physical situation into a very simple analytical situation where what we're doing is modeling this complex channel as just a bunch of taps in time, at some bandwidth that we're using and each of these taps is now Gaussian and the output is going to be the discrete sum as we take a discrete sequence of inputs, pass it through this discrete filter and then we add up these outputs and at that point, we have to worry about how do we detect things from what's coming out?
So they say here -- this is a really flaky modeling assumption and it's only partly justified by looking at lots of different situations that a given cellular phone would be placed in.
If we look at this whole ensemble, it makes a little bit of sense, but really what we're doing here is saying what we want is a vehicle to let us understand how to receive these waveforms.
We know there is going to be fading.
Since we know there is going to be fading, we have to find some kind of statistical model for it if we're going to find sensible things to do and this is the one we're going to pick.
It's very common instead of using a Rayleigh fading model to use a Rician fading model.
A Rician fading model makes sense because if you have a line of sight directly to a base station, the waveform you get from the base station is going to be kind of strong and all of these waveforms you get reflected from other obstacles and so forth are going to be rather weak.
So in that case, you're going to have one strong received waveform, which is one big peak and everything else is going to be very weak.
The thing that that says is the kind of statistics that you want are not a sum of a lot of little tiny things, but the sum of one big thing and lots of tiny things.
That's not a good model either.
What you would really like is a sum of a bunch of medium sized things most of the time, but about as far as anybody gets talking about statistical models for fading channels is talking about Rician model and talking about Rayleigh models and then worrying about what happens the different paths of these filters.
The trouble with Rician random variables is they're very complicated.
They have vessel functions in them.
If you read the notes I passed out today, you find that when you try to detect things, when they've gone through this kind of Rician fading, everything is just a bloody mess.
That's the way it is.
Since we are talking now about a statistical model for these channels, we would like to have something called a tap gain correlation function, which is going to tell us how each of these taps vary in time.
That's just the expected value of Gkm and G of k prime n. What we're going to assume, and what is really a very good assumption, is if you look at the things that add up as taps under a particular value of k and under some other value of k -- in other words, all of the paths that are coming in in one range of delays and all the paths that come in on another range of delays, those things are pretty much independent of each other.
So almost always, when you start calculating tap gain correlation functions, you assume that this is independent when k is unequal to k prime.
So really the only thing you're concerned about here is how these quantities vary in time.
We've already talked about how they vary in time.
In other words, that was why we talked about all of this business about time coherence and Doppler shifts.
The thing which is causing these things to change in time is these Doppler shifts.
We have already decided that the amount of time the channel remain stable is about 1 over 2 times this Doppler spread.
Here we have an analytical, statistical way of doing the same thing.
In other words, you can look at this correlation function here and you can see how long it takes before it drops essentially to zero.
So you can define the time coherence in terms of this duration here just as well as the way we've already done.
In other words, how big does n have to get before this thing gets small?
What's the advantage of doing it this way instead of in terms of Doppler shifts?
If you want to measure it, you don't have to go out and measure where these paths are.
If you want to measure it, you just take a cellular phone and you measure where these things are coming in and how they change in time and that gives you a direct view of what the time coherence is.
Why that W was there, there's a problem in the problem set that will give you some insight into why it's there, but otherwise it's just arbitrary.
I want to spend the rest of the time talking about Rayleigh fading, because Rayleigh fading is something you can analyze easily.
We're going to do something even simpler.
We're going to do Rayleigh fading where you have a single tap model.
In other words, where the bandwidth that you're using is small enough that all these different paths all fall in the same delay range.
In other words, it's where the multipath spread, L, is small relative to 1 over W. That's what's called flat fading, so we're assuming flat fading and we're assuming that it changes in time according to a Rayleigh model.
Suppose that we do something really simple, like trying the antipodal signalling that we've been using all along, which is really neat when you have Gaussian noise.
It's just about the best kind of signaling that you can use.
It doesn't work at all here and it doesn't work at all because since this channel is Rayleigh fading, you don't know what the phase is of the channel, so you send a bit and what comes in is something which, sure, it has an amplitude to it, but it has a completely random phase to it, so you can't tell the difference between one and minus one.
The only way you can tell is to send first one signal and then another signal, but if you send one signal and another signal, you might as well choose a single pattern which lets you do this sensibly.
So the particular thing that we'll look at is something called post position modulation.
This is essentially the same as about ten other schemes and we'll talk about that a little more later, but what you're going to use is two degrees of freedom, which is two different time instants.
In the first instant, you're going to send an a.
In the second instant, you're going to send a zero where conversely, you're going to send a zero in the first degree of freedom and an a in the second degree of freedom.
So you can do this either with two different frequencies, you can do it with two different time instants.
You can do it in all sorts of different ways.
Any time you have two degrees of freedom, you either use one degree of freedom or you use the other degree of freedom.
It's not like the situation we've talked about before, because these degrees of freedom are now complex degrees of freedom.
If I send the one, it's going to come in as some complex number, which is going to have uniform phase.
So my hypotheses now are one hypothesis is a and zero and the other hypothesis is zero and a.
What's going to happen with these hypotheses is that first, the channel is going to do its thing to them.
In other words, if I send this, the channel is going to take that a.
It's going to multiply it by a circularly symmetric Gaussian random variable, so it's going to come as some blob and in the other dimension, what I send is a zero, so what comes in there is nothing.
I'm going to add noise to this, so when I send a zero, I'm going to get -- from the channel, I get a blob in this first degree of freedom.
The noise adds another blob, which is also circularly symmetric.
What comes in the other degree of freedom is zero on the channel and a blob of noise.
So I'm going to base my detection on either a big blob or a little blob and on the other hypothesis, I got a little blob here and a big blob there.
So the question is, you have to look at this pair of blobs and decide from it what you think was the most likely signals.
Sounds hard, but it turns out that it's easy, because look, under the hypothesis H 0, what you're going to receive -- and these are a times a complex variable, G 0 plus Z 0.
Both of these are Gaussian random variables, both zero mean, both circularly symmetric and V 1 is going to be just Z 1.
So here I have a sum of two Gaussian random variables.
Here I have one Gaussian random variable.
V 0 is complex Gaussian and its variance is a squared plus N 0 times W, because that's what the noise is.
So I look at the probability density of these two complex Gaussian random variables, sample value v 0 and v 1, conditional on H 0.
What I have is some constant here -- I don't even care about it -- times e need to the minus v 0 squared divided by a squared plus wn 0.
This is the variance of V 0, because V 0 was a squared times G 0.
That's the a squared there.
Variance is Z 0 WN 0 and for the other random variable -- that's the little blob random variable -- probability density of that is minus v 1 squared divided by WN 0.
For the alternative hypothesis, I get the same constant out front, which I haven't even bother to write down, times e to the minus v 0 squared over WN 0.
Here are the other variables: The one where we have the big blob, so that's the variance a squared plus WN 0.
I take the log likelihood ratio, which is to take the logarithm of the ratio of these two hypotheses, of these two likelihoods, and when I take this quantity and divide it by this quantity, these are both up in the exponents, so I'm just taking this and subtracting this.
When I subtract this quantity from this quantity and then subtract this quantity from this quantity, what I get is just this quantity here.
I mean, you can see it's simple because this difference is going to be the same as this difference, except they have a different sign.
So it's just algebra to find out what happens when you when you take this minus this and then you take minus this plus this.
This is what you get.
Then you look at that for a bit and you say, how do I find the probability of error from this?
Before we find the probability of error, the first thing we have to do is say, what's the maximum likelihood rule to use here?
Maximum likelihood rule is you take the log likelihood ratio and if it's positive, you choose v0.
If it's negative, you choose v1 and if it's 0 -- if 0 was zero probability, it doesn't matter what you do.
So what we're interested in now is, what is the probability that this quantity here is going to be bigger than zero?
But you see, this isn't hard because when you look at the difference between v 0 squared and v 1 squared, v 0 squared is an exponential random variable.
It's just the energy of a complex Gaussian.
So we're taking one exponential random variable, subtracting off another exponential random variable and we're just looking at it over on the region where these things are positive.
When you integrate that, this is the answer that you get.
I mean, it's not hard to integrate it.
I just don't -- and if you look at the notes, the notes carry it out in all of its gory detail, but it really is about two steps.
The thing we want to look at is this result here, because it looks so different from all of the error probability results that we've gotten before.
I mean, before, error probabilities go down exponentially as a square of the signal amplitudes that we're using and they go down exponentially with the energy that we're using.
Here, this crazy thing is going down as 1 over 2 plus energy to noise ratio.
This is 1 over 2 plus Ed over N 0.
So it really goes down slowly.
So what's going on?
I mean, why is it that with all of these Gaussian random variables where things go down so fast, we're really in such deep trouble here?
The trouble is, every time the fading is significant and the fading is significant whenever -- under hypothesis 0, if you don't have much of a channel -- namely, if both v 0 and v 1 are both very -- if both the real part of G 0 and the imaginary part of G 0 are both very small -- which happens with reasonable probability -- then you don't have a chance of trying to decode correctly because the two signals are both going to look the same, because both of them are just Gaussian random variables.
So as a result of that, when you're dealing with Rayleigh fading, you really have to find something else to do in order to make this work.
I mean, if you have Rayleigh fading and you just try to communicate in the way that we tried to communicate before -- just sending individual bits and hoping for the best, you're in very deep trouble.
So next time we'll start to talk about ways of dealing with this using diversity, using channel measurement, using all sorts of other things, but -- I don't know what happened to the person who was supposed to evaluate the class, but maybe they will come in next week.
I don't Know.
The following content is provided under a Create Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality education resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's get started then.
We went through Rayleigh fading very, very quickly last time.
I want to spend a little more time on it today since it's one of the sort of classical models of wireless channels.
And it's good to understand how it works.
And it's good to also understand what all the assumptions that are made when one assumes Rayleigh fading, because they're really quite a few of them.
OK, so what we're doing is we're assuming flat fading.
In other words when we talk about flat fading, we're talking about fading where if you generate a discrete model for the channel, that discrete model is just going to have one path in it.
In other words, the output is going to look like a faded version of the input.
It'll be shifted in phase because of the unknown phase in the channel.
It'll be attenuated by some random amount.
But if you look at the waveform, it'll look like the waveform that you transmitted except for the noise.
And that's what really is represented by this one tap model that we've been looking at.
In general we've said that you can model a pretty arbitrary channel for purposes of somewhat narrow band communication by using a sequence of taps where usually for want of something better to do, we model those taps as being Gaussian random variables, complex Gaussian random variables with zero mean, and variables which are circularly symmetric.
And we assume for not any very good reason, that the taps are independent of each other.
I mean we have to make some assumptions or we can't start to make any progress on trying to analyze these channels.
But we should realize that all of these assumptions are subject to a certain amount of question.
OK.
When we assume a single tap model, and these tap models are always given with the number of the tap given first, and the time given second.
So what we're assuming here is the only tap is the tap at time 0.
And it's at time 0 because we're assuming the receiver timing is locked at transmitter timing.
And we're just going to get rid of the zero because there's only one tap, and call this G sub m. We're also going to pretty much assume that G sub m stays relatively constant for a relatively long amount of time.
Except as far as this analysis of Rayleigh fading goes, we don't have to assume that.
Because in fact, when we're assuming Rayleigh fading, the analysis that we're going to follow, the receiver doesn't know anything about the channel at all, except that it's a single tap model.
And therefore what the receiver does is it goes through maximum likelihood detection assuming that that single tap is just a complex Gaussian random variable.
OK when you have a complex Gaussian random variable as you've seen in the problem sets and we've noted a number of times, the energy in that complex Gaussian random variable is exponential.
And the magnitude is just a square root of the magnitude squared, namely the energy.
And that has a Rayleigh distribution which looks like this, namely the probability density of how much response you get.
We'll base this law here.
And the phase of course, is equally likely to be anything.
Namely the phase is uniform and random.
This density looks like this.
I wanted to draw this so it would emphasize the fact that the magnitude is in fact, always nonnegative.
But also to emphasize the fact there's a whole lot of probability down here where there's very, very little channel.
And this is in fact what gives rise to the fact that if you try to communicate over Rayleigh fading, and you don't make any use of diversity-- and we'll talk later about diversity-- in fact you can't communicate very well at all.
And that's because of this very bad region here where the channel is very badly faded.
You send a bit on this channel which is very badly faded, and there's nothing much the receiver can do.
And that's the thing we want to try to get a real feeling for.
OK so the output of the channel when you put in an input U sub m, and we'll think of this as being a binary digit for this time being.
So the output is going to be the input times this tap variable, which is this complex Gaussian random variable plus a noise random variable, which we're also assuming is complex Gaussian and circularly symmetric.
OK so what we have, if you're going to make two hypotheses about two possible values of U sub m, look at what this random phase does here.
No matter what U sub m you transmit in one epoch of time, the channel is going to rotate this around by a completely random phase.
It's going to add a noise to it which has a completely random phase.
And the output is going to come out.
And the output has a completely random phase.
Namely the phase of the output cannot possibly tell you anything about what input you're putting into the channel.
OK so in other words, in this model that we're using, the phase is completely useless.
And if we want to talk about anything connected to using likelihoods, the only thing we can use is the magnitude of the output.
OK. Now, why don't we just analyze it in terms of the magnitude of the output?
Well when you analyze these problems, Gaussian things are usually much easier to analyze than things like this.
Not always, I mean we have to get used to analyzing all of them.
But this particular problem of Rayleigh fading is really easier to analyze in terms of these Gaussian random variables.
But it's easier to understand in terms of recognizing that the only thing you can make any use is these magnitudes.
OK if we only use one complex degree of freedom in a signal, namely if we try to send some signal and we only use one input to the channel, then we only get one output.
Namely we sent U sub 0, we get V sub 0.
And we try to decide from V sub 0 what was sent.
We're really in a very bad pickle at that point.
Because the only thing that makes any sense, since we can only use the magnitudes, is to send a very small magnitude or a positive magnitude.
Magnitudes are positive anyway.
So if you're going to send binary signals, this is your only choice-- if it makes any sense-- if you make this larger than zero you're just wasting energy.
So you only have this choice.
And you can choose the amplitude a that you're using.
But that's the only thing that you can do.
OK, this is a very nasty thing to analyze for one thing.
It gives you a very large error probability for another thing.
Nobody uses it for another thing.
And therefore almost all systems of trying to transmit in this kind of Rayleigh fading, always use at least two sample values.
In other words, instead of just putting one complex degree of freedom into the channel, you're going to put two complex degrees of freedom into the channel.
And the thing that we're going to analyze, because it's the easiest thing to do in this discrete time model we've developed, is to think of modeling hypothesis 0 as sending two symbols U sub 0 and U sub 1 will make U sub 0 equal to a, and U sub 1 equal to 0.
And the alternative case, if we're going to try to send input 1, this is binary transmission.
You can talk about more than binary transmission, but binary is awful enough.
You get U sub 0 and U sub 1 is equal to 0 and a.
So what you're going to be doing here in this pulse-position modulation, is choosing one of these two different epochs to put the data in.
So in one case, you put all your energy in the first one.
In the other case, you put all your energy in the second one.
Mathematically, this is completely equivalent to frequency-shift keying, that's completely equivalent to phase-shift keying.
And if we had a little more time, we could talk about that.
And I'll probably put an appendix in which talks about those two systems.
But in fact, it's completely the same thing.
It's just that you're using different complex degrees of freedom than we're using here.
So we're really analyzing FSK and PSK.
And that's where people usually come up with these analyses of Rayleigh fading.
OK when we have input 0, what we receive then, is the 0 is going to be the input a, times the magnitude of the channel of time 0, plus a noise variable.
The noise is complex Gaussian, remember.
The second input is just going to be the noise variable.
Alternatively, if we're sending the second symbol, which means we put our energy into the second degree of freedom, it means that what we're going to get is the 0 was just going to be the noise.
And the second output is going to be the signal plus the noise.
And remember, both this variable and this variable are both complex Gaussian.
The phase doesn't mean anything.
So what we can use is simply the magnitude.
OK, so when we have hypothesis equal to 0, what comes out is going to be, the 0 is going to be a complex Gaussian random variable.
Let me introduce a new piece of notation now.
Because it gets to be a real mess to constantly talk about a Gaussian complex random variable, and talk about it's real part and imaginary part as being independent Gaussian.
So I'll just call this normal complex.
And this first thing is the mean, which is a real and imaginary part, but it's zero in most of the things we deal with.
And the second one is the mean square value of this random variable V sub zero.
So this quantity here is now twice the variance of the real part of V sub 0, and twice the imaginary part of the variance of v0.
We scaled the noise in a peculiar way here.
And I apologize for all of the mess that occurs when we do this.
Because sometimes we think of the noise as having variance N sub 0 over 2 in each real and imaginary degree of freedom.
And therefore N sub 0 in a complex degree of freedom.
And sometimes we think of it as having variance N sub 0 W Where does that difference come from?
It's this infernal problem of the sampling theorem being so critical in most of the models that we talk about.
OK because when you use the sampling theorem, the sinc x over x waveforms that we use are not orthonormal, they're orthogonal.
And this factor of W appears exactly because of that.
They appear because the magnitude of the signal is a, and the energy and the power in the signal is then a-squared.
OK.
In this case the power in the signal is not quite a-squared because we only send energy in one or the other of alternate degrees of freedom.
So therefore, if we look at a time one second, we get W complex degrees of freedom to use.
We only send energy in half of those so that the actual power that we're sending is a-squared divided by 2.
OK. Because of that, when we normalize the noise the same way the signal is normalized, we get this variance W N sub 0.
If you're confused by that, everyone is confused by it.
Everyone I know, when they go through calculations like this, they always start out with some arbitrary fudge factor like this.
And after they get all done, they think it through or more likely they look it up in a book to see what somebody else has gotten.
And then they sweat about it a little bit, and they finally decide what it ought to be.
And that's just the way it is.
It's the problem of having both the sampling theorem and orthonormal waveforms sitting around.
It's also the problem of multiplying the power by 2 as soon as we go to passband.
Because both of those things together generate all of this difficulty.
But anyway, this is the way it is.
And the important thing for us is that what we can have now is under these two hypotheses.
We just have two Gaussian random variables, complex Gaussian random variables.
And in one case, the larger mean square value is in one.
And in the other case the larger mean square value is in the other.
OK.
So just reviewing that.
If H is equal to zero, V sub 0 and V sub 1 are these complex Gaussian random variables.
If H is equal to one, then we have this set of Gaussian random variables.
The probability density of V sub 0 and V sub 1, and now it's more convenient to use the real and imaginary parts for the Gaussian density.
Anytime you're working problems of this type, try both densities using real and imaginary parts, and using magnitude in phase, and see which one is easier.
Here it turns out that the easiest thing is just to use the ordinary conventional density over real and imaginary parts.
And what we wind up with is this Gaussian density.
On V sub 0 the density is V sub 0 squared divided by the variance a-squared plus W N sub 0.
And on V sub 1, it's this Gaussian density V sub 1 squared divided by W N sub 0.
Just because here we have this variance.
Here we have this variance.
OK, on the alternative hypothesis when H is equal to one, you have the same thing but the denominators are switched around.
When you take the likelihood ratio, you want to take the ratio of this, to the ratio of this.
If you look at it and you take the logarithm of that, you're taking the ratio of this to this.
Incidentally the coefficient here, you could write it out if you want to.
It's 1 over the square root of blah, times 1 over the square root of blah.
But if you recognize that the coefficient here has to be the same as the coefficient here, you don't have to worry about it.
So when you take the log likelihood ratio, you get this divided by this.
You have the same form in both cases.
In one case, you have this term minus this term and this term minus this term.
And the other case well, for V sub 0, you have this term minus this term.
And for V sub 1 you have this term minus this term.
Because of the symmetry between the two, this just comes out to V sub 0 squared minus V sub 1 squared times a-squared.
And when you do the algebra, the denominator is a-squared plus W N sub 0 times W N sub 0.
OK. What do we do for making a maximum likelihood decision?
Maximum likelihood is map when the threshold is equal to 1, which is when the logarithm of the correct threshold is equal 0.
Which says that you take this quantity, and if it's nonnegative, you choose H equals zero.
And if it's negative, you choose H equals one.
Which says you compare V sub 0 squared and V sub 1 squared.
And whichever one is larger, that's the one you choose.
And if you go back and look at the problem, it's pretty obvious that that's what you want to do anyway.
I mean you'd be very, very surprised when you're comparing two Gaussian random variables where one of them has a larger variance than the other.
And on the other hypothesis, the absolon has the larger variance.
If you came up with any rule other than to take the magnitude squares and to then compare those two magnitude squares, you would go back and look at the problem again realizing you must have done something wrong.
But anyway when you deal with problems like this, I advise you to take log likelihood ratio anyway.
Because every once in awhile you find something which comes out in a somewhat peculiar way.
But anyway, here there's nothing peculiar.
So what we have to do now is to find the probability of error.
Now what's the probability of error?
OK if we actually transmit zero, then V sub 0 squared is exponential.
It's exponential with this mean.
Namely this is the mean of V sub 0 squared.
And V sub 1 is exponential with this mean.
In other words, this is a big exponential.
And this is a little exponential.
The two of them have probability densities that look like this.
This is not going to work.
The big one has a probability density that looks like this.
And the little one-- this is big-- and the little one has a probability density that looks like this.
And what you want to do is to subtract a random variable with this density from a random variable with this density.
So you're convolving two exponential densities with each other.
And unfortunately, you're taking the differences of two.
So you're convolving the negatives of this with this.
And then you have to integrate the thing.
And it's just something you have to do.
And the answer is, the probability of error is then 2 plus a-squared over W N sub 0 to the minus 1.
OK that is really an awful result.
Because that says that if you increase the energy that you're using, the probability of error goes down very, very, very slowly.
And if you look at this picture you think about it a little bit, it should be clear that that's the only thing that can happen.
OK. Because if you increase a-squared a little bit, it's not going to save you much here.
Because when you have a bigger a-squared, it's just going to move down the value of g bar that's going to give you trouble.
Namely when you double a, the value of magnitude of g that gives you trouble just goes down by a factor of two.
When that goes down by a factor two, this bad part of the curve just goes down in a quadratic way.
Well that's what this is telling us.
OK.
I mean the thing that we see is a quadratic and a.
So we're sort of assured that we're doing the right thing.
And we're sort of also assured that the reason why this result is so awful, is just that sometimes the fading is so bad there's nothing you can do about it.
OK now the signal power as we said before is a-squared over 2.
Since half the inputs are zero.
So we can put twice as much energy into the ones that are non-zero.
And therefore when you put this in terms of the average signal energy that you're sending, what we get is E sub b over N sub 0.
OK.
So that again says exactly the same thing that this does.
It's worthwhile keeping both of these notions around, because we have done something kind of peculiar here.
I should mention it for you.
As soon as you're looking at a fading channel, the power that you're talking about becomes a little peculiar.
Because remember when we were looking at white noise channels?
What we were looking at is the power at the receiver, the signal power as received at the receiver.
Now at this point, we still want to talk about E sub b.
We still want to isolate this problem from the attenuation that occurs just because of distance and things like.
Because of that, when we model g, this original model that we use here was a model in which the magnitude of g had mean one.
And we made it have mean one so that the energy would come out right.
Which is another reason why you get confused with these E sub b over N sub 0 terms.
OK so anyway, that's the answer.
And E sub b is in terms of the received energy using the average value of fading.
OK we next want to look at non-coherent detection.
Non-coherent detection is another thing that communication engineers use all the time, talk about all the time.
And you have to understand what the difference is between coherent transmission and incoherent transmission.
The general idea is that when you're doing incoherent detection, you're assuming that you don't know what the phase of the channel is.
And somehow you want to do your detection without knowing that phase.
The difference between Rayleigh fading on this kind of channel and incoherent detection, is that with incoherent detection the receiver is assumed to know what the magnitude of the channel is, but not the phase.
It's harder to measure the phase of the channel than it is the measure of the magnitude.
Because the phase changes very, very fast.
If you look at these equations we have for what the response of the channel is, you see the phase changing many, many times during the time where the amplitude of the fading changes by just a little bit.
So a very common assumption that people make when trying to do detection is that it's incoherent.
Partly, people get used to analyzing incoherent communication.
And I've seen this so many times.
And they insist on building communication systems using incoherent detection.
They will swear up and down there's no way you can measure the phase.
And what they're really saying is that's the only kind of communication they understand.
And because that's the only thing they understand, they become very, very upset if anyone suggests that you ought to try to measure the phase.
But that's a tale for another day.
OK so now we want to look at the case where we're assuming that we know the magnitude of the channel.
It's just some quantity that we'll call g tilde.
We're assuming that the same magnitude occurs both on U sub 0 and U sub 1.
We're going to use the same transmission system that we used before, namely pulse-position modulation.
We'll either put our energy in U sub 0 or we'll put our energy in U sub 1.
We'll try to detect what's going on.
But we just give the detector this little extra amount of ability of knowing what the channel is.
I'm going to talk more later about how you can use this knowledge of what the channel is, and how you can measure what the channel is.
But for now we just assume that we know it.
So the phase is random and independent of everything else.
So under hypothesis H equals zero, we have the output of the channel.
And times 0 is whatever input level we put in a, times what the channel does to us, times e to this random phase.
And V sub 1 it's just-- plus Z sub 0.
And in the other case we have V sub 1 equals Z sub 1.
And under the other hypothesis V sub 1 is this input with a random phase but a known magnitude and again, a Gaussian random variable.
Phases are independent of the hypothesis.
The phases are independent of the magnitudes which are known.
The phases are independent of everything and therefore, we just want to forget about them.
So the question is, how do we make a maximum likelihood decision on this problem?
Well you look at the problem.
And for the same reason as before you say, it's obvious how to make a maximum likelihood decision just from all the symmetry that you have.
If the magnitude of V sub 0 is bigger than the magnitude of V sub 1, V sub 0 corresponds to this little bit of extra energy that you have.
So if V sub 0, the magnitude of V sub 0 is positive, is bigger than the magnitude of V sub 1, you want to choose H equals 0.
And alternatively you'll want to choose H equals 1.
It's obvious right?
I've tried for years to find a way to prove that.
And the only way I can prove it is by going into Bessel functions which is the way that everybody else proves it.
And this seems like absolute foolishness to me.
And if any of you can find a way to do this, I would be delighted to hear it.
I will be in great admiration of you.
Because I'm sure there has to be an easy way to look at this problem.
And I just can't find it.
OK anyway, we're not going to worry about all these Bessel functions, because that's just arithmetic in a sense.
So we're just going to say well it can be proven using all of this machinery.
So what we really want to find is what is the probability of error when we make that decision.
And when we make that decision, namely what we're looking for is the probability of this magnitude then, is bigger than this magnitude when H equals one is the correct hypothesis.
Because that's the probability of error then.
So you have these two different terms.
You just go through all of the junk that's in the appendix to the notes we passed out last time.
If you want to go through that, I think it's great.
It's an interesting analysis.
Certainly not going to do it now.
When you get done doing that you find out the probability of error is exactly one half times e to the minus a-squared times this known magnitude of the channel.
I mean, a-squared and g tilde have to appear together here.
OK because what's coming out of the channel, the magnitude of what's coming out of the channel without noise is just a times g tilde.
They both come together everywhere.
And therefore, they have to come together anytime you're talking about optimal detection, probability of error, or anything else.
So these two appear together.
We have the same noise term down here as we had before.
Because again we're using a sampling theorem analysis and the noise in each of these random variables is W N sub 0.
OK so that's a little surprising that that's what the noise is.
If you knew the phase also, if the detector knew both the magnitude and the phase of the channel, it would be the conventional Gaussian problem that we've analyzed many times before.
And the solution would be that probability of error is equal to Q of a-squared times g tilde squared divided by W N sub 0.
Now if you remember the estimates we've come up with and the bounds we've come up with on the Q function, the simplest bound that we came up with was this.
Namely you take this thing, you take one half of it, which is the Gaussian density with the coefficient.
You multiply it by one half.
So this is the simplest estimate we can get of this.
On the other hand when this quantity is large, a much better estimate of this is to have that estimate which has a 1 over the square root of pi times W N sub 0 over a-squared g tilde squared in it.
So we have that term extra.
Which says that when this is large, whenever we're communicating at all reasonably, this probability of error is much smaller than this probability of error.
However you talk to any communication engineer, and they'll say when you have a good signal noise ratio, incoherent detection is virtually as good as coherent detection.
And why did they say that?
Well it's because the probability of error goes down so quickly with energy here.
It's going down as a square of an exponent.
Well it's going down as an exponent in the energy.
The question you want to ask is how much extra energy do I have to use?
If I'm using coherent detection, how much more energy does an incoherent detector need at the input in order to get the same results?
And then you see the question is very different.
Because if I increase this quantity just a little bit, this probability of error goes down like a bat.
OK.
So what happens then, when you compare these two terms is that as the signal to noise ratio gets larger and larger, the amount of extra energy you need to make incoherent detection work as well as coherent detection goes down with 1 over a-squared.
Which says that these communication engineers who swear that they like incoherent detection in fact, have something on their side.
because they don't have to assume so much about the channel.
They have something which is more robust.
And in fact what's turning out here, is that even though this error probability is a little bigger than this error probability, there's only a very negligible amount of extra dB required to make the two the same.
So it only costs a little bit of extra energy to be able to use incoherent detection instead of coherent detection.
OK so this is very strange now.
We have a nice error probability which is almost as good as the Gaussian error probability using incoherent detection.
This is assuming that the channel, that the receiver knows what g tilde is.
But now we go back and think about this, and look at our detection rule, which is the optimal detection rule.
And the optimal detection rule is no matter what g tilde happens to be, we compare the magnitude of V sub 0 with the magnitude of V sub 1.
In other words, we have analyzed this assuming that we know what g tilde is.
We know what the gain of the channel is.
But the receiver doesn't pay any attention to it.
OK so now we have this very peculiar situation where incoherent detection with a known value channel is almost as good as coherent detection is.
But at the same time Rayleigh fading gives this awful error probability.
So now you have the final part of the argument take this probability of error, multiply it by the probability density of g tilde squared, integrate it to find out what the average error probability is when we average over the channel fading.
And guess what answer you get?
Well you ought to be able to guess it if you've looked at the homework already.
Because in the homework you actually go through this integration.
And bingo you get the Rayleigh fading result.
Which says that the problem with Rayleigh fading is not any lack of knowledge about the channel.
Knowing what the channel is would not give you, even knowing what the phase is of the channel would not give you a lot of extra help.
The only help in knowing what the phase is, is to get this result instead of this result.
And even that won't help you much.
The problem is anytime you're dealing with Rayleigh fading, the channel has faded so badly a large fraction of the time, that you can't get an acceptable probability of error.
OK so now we have to stop and think.
What do you do about this?
Well you have two general kinds of techniques to use at this point.
OK and one of them is to try to measure the channel at the receiver.
You take the measurement of the channel at the receiver.
You send it to the transmitter.
And the transmitter then does something to compensate for the amount of fading.
One thing that the transmitter can do is anytime the channel is badly faded, it increases the amount of power that it's sending.
That's what typical voice systems do.
And the other thing that you can do is change the rate at which you're transmitting.
You can do all sorts of things with the transmitter if you know what the channel is.
You can respond to it in various ways.
And all these different communication systems have various ways of dealing with that.
And we'll talk a little about that on Wednesday when we talk about CDMA.
The other thing you can do about it is use something called diversity.
And the idea of diversity is that instead of sending this one bit, trying to use as few degrees of freedom as possible, you try to send your bits using as many degrees of freedom as possible.
If you can use a large number of degrees of freedom, and if the fading is independent on these different degrees of freedom, then in fact you gain something.
Because instead of having one random variable which can totally cripple you, you have lots of random variables.
And if any one of them is good, you get through.
So you get a benefit out of diversity.
OK so that's our next topic.
Namely, how do you measure the channel?
Because if you're going to use diversity it's a help to know the channel.
If you're going to use coding, coding is just another way to get diversity.
Again your coding will work better if you know what the channel is.
So somehow we would like to be able to measure the channel and send it back to the transmitter if we want to alter the power, the rate of the transmitter, and to let the receiver use it if the receiver is going to.
Well we have seen that when you use one bit on just a couple of degrees of freedom, knowing what the channel is does not do you much help.
If you use coding or if you use one bit and spread it over a large number of degrees of freedom, then knowing what the channel is gives you a great deal.
This is one of the basic confusions that everyone has when they deal with Rayleigh fading.
Because when you look at a Rayleigh faded channel, the first thing you analyze is this incredibly small number of degrees of freedom.
And you say wow, that's awful.
There's no way to deal with that.
And then you start looking for something.
And you say, well diversity helps me.
But in general, this is the general scheme of things that we're going to use.
OK.
So as we said channel measurement helps if diversity is available.
Why does that help when diversity is available?
OK, think of sending this one bit.
You get one reception, and then you get another reception.
And on this reception, you get one amount of fading.
On this reception you get another amount of fading.
If I don't know how much fading there is it doesn't help me an awful lot.
It helps me some.
But if I know that this channel is faded badly and this channel is not faded, then I'm going to use what comes out here instead of what comes out here.
And then my detector is going to work much, much better.
When you look at diversity results, always ask yourself a couple of questions.
Is the detector using knowledge of what the strength of the channel is on these two diversity outputs?
Is the transmitter using it's knowledge of what those channels are?
You get very different results for diversity depending on the answers to both of those questions.
OK, so if you have a multi-tap model for a channel-- OK remember the multi-tap models that we came up with.
We we're looking at transmission using multipath.
And we had multipath in different ranges of a delay.
We came up with a model which gave us multiple taps for a discrete model of the channel.
You get a large number of taps if you're using broadband communication.
Because using broadband communication 1 over W becomes very small.
And therefore these ranges of delay become very small.
And if you're using very narrow band communication, that's when you have the flat fading.
Namely flat fading is not flat fading, it's fading which is flat over the bandwidth that you're using.
So if you use a broader bandwidth and you have multiple taps, then these taps are going to be independent of each other.
And you automatically have diversity.
So the question is how do you use that.
Well if you're going to use it, you better be able to measure it.
OK so now we're going to try to figure out how to do that measurement.
And the first thing to do is to assume the simplest possible thing.
I mean, suppose you know how many taps the channel has.
Suppose it has k sub 0 channel taps.
So the channel looks like this, G sub 0, G sub 1, and G sub 2.
You're transmitting a sequence of inputs.
OK remember all of this stuff came from trying to model a channel in terms of discrete inputs, where you're sending one input each one over W seconds.
So you put in a sequence of inputs.
You have these three different channel taps here.
And what comes out when you put in a single bit here or a single symbol.
You get something out when this tap right away.
You get something out here one time unit later.
You get something out here, one epoch still later.
So all of these outputs get added up.
And therefore the output here at time m, is the input at time m times this tap, plus the input at time m minus 1 times this tap, plus the input at time m minus 2 times this tap.
Because it takes these inputs that long to go through there.
All this is is just digital convolution, OK.
I'm just drawing it out in the figure so you see what's going on.
Because otherwise you tend to think everything happens at one instant of time.
Then we're adding this white Gaussian noise.
When we're talking about digital systems, white Gaussian noise just means that each of these random variables are independent at each other random variable.
They all have the same variance.
And the real parts and imaginary parts have the same variance.
And they're independent of each other.
Namely these are all normal random variables.
Since we're sending a, or minus a, or something with magnitude a, we want to divide by a out here, if we want to figure out anything about what these taps are.
OK so suppose that what we send now is a bunch of zeros, followed by a single input, followed by a bunch of zeros.
What comes out?
Well the thing that comes out is at the point that this big input comes in, we get a G sub 0 out at the time that you put in a. I mean we're leaving out propagation delay here.
We got a times G sub 1, the next epoch.
Then we get a times G sub 2, the next epoch.
And by that time the input is completely out of the filter.
And we get zeros after that.
So if you put in a bunch of zeros and then a single a, you got a nice measurement of the channel.
There's Gaussian noise added to each of these inputs.
But in fact you do get a reading of each channel output.
When you divide these by the a here, then you get something which is a measurement of the appropriate tap G plus Gaussian noise on it.
OK now you try to make an estimation from this.
And the trouble is we don't want to say much about estimation theory.
But in fact the notes gives you a very brief introduction into estimation.
There are two well-known kinds of estimation.
One of them is maximum likelihood estimation.
And the other one is minimum mean square error estimation.
Maximum likelihood estimation is in fact exactly the same as maximum likelihood detection.
Namely you look at the likelihoods which is the probabilities of the outputs given the inputs.
And what's the input in this problem?
The input is these channel variables.
Because that's the thing we're trying to measure in this measurement problem.
We assume that the probing signal is known.
It's just a bunch of zeros, followed by a, followed by a bunch of zeros.
So we know that.
We're trying to estimate these things.
So these are the variables that we're trying to estimate.
So we try to find the probability density of the output conditional on the knowledge of G sub 0.
Which is just the Gaussian density shifted to a times G sub 0.
You then look at the maximum likelihood estimate of G. So you're looking at the value you can put in to maximize this estimate which comes out here as a times G sub 2.
And then at this appropriate time, you're looking at G sub 2 here, plus a noise random variable.
And since the noise is zero mean, this quantity here is in fact the best estimate in terms of the maximum likelihood that you can get.
If you assume that this is a Gaussian random variable and this is a Gaussian random variable, you can solve a minimum mean square error estimation problem.
It's much like the map problem except these random variables are all continuous here.
But it's a little different from the map problem in the sense that you can't have equally likely inputs where you have a continuous random variable.
You make them all equally probable.
The only possible value you can have is zero.
Because it has to extend forever.
So anyway, maximum likelihood detection is just normalize what you get so that in the absence of Gaussian noise, you would get the variable you're looking for.
And then ignore the Gaussian noise, and that's your estimate.
OK.
If you want to do this and you want to use the strategy, it looks like a very nice strategy.
But what's the problem in it?
If this sequence is somewhat longer, you need a whole lot of zeros in between each probing signal.
And what that means is you're going to be using your energy and clumping it all up into the small number of degrees of freedom.
Which means you're going to be sending a lot of energy at one instant of time and then nothing for a long period of time, then a very big signal for awhile then nothing for a long period of time, and so forth.
If you do that, the FTC is really going to be down on you.
Because you're not supposed to send too much energy in any small amount of time or any small amount of frequency.
So you're supposed to spread things out a little bit.
You say OK, that doesn't work too well.
What am I going to do?
How can I choose a sequence of input so they have relatively constant amplitude, but at the same time so that when I go through this kind of filter, I can sort out what's coming from here, and what's coming from here, and what's coming from here.
Well it turns out that the answer to that question is to use a pseudonoise sequence.
And the next thing I want to do is to give you some idea about why these pseudonoise sequences work.
OK so we'll think in terms of vectors now.
OK so we have a vector input, u sub 1, u sub 2, up to u sub n, a vector of length n. So we're putting these discrete signals in one after the other.
We're passing them through this, which is a digital filter now.
So what comes out here V prime is just a convolution of u and G. We then add the noise to it.
I claim that what we ought to do is use the matched filter here to u.
And if I use a matched filter to u here, that matched filter, if I'm using a pseudonoise sequence, is going to bingo give me the filter that I started out with, plus some noise.
OK why is that?
The property that pseudonoise sequences have, if I choose each of the inputs to have the magnitude of a, and think of it as being real plus or minus a, which is what people usually do.
If you look at the correlation of this sequence, namely the correlation of u sub m with the complex conjugate of u sub m spaced a little bit, PN sequences have the property that this correlation function looks like an impulse.
OK. Now how you find sequences that have that property is another question.
But in fact they do exist.
There are lots of them.
They're easy to find.
And they have this very nice property.
Another way to say this is that is that the vector u has to be orthogonal to all of its shifts.
That's exactly what this is saying.
And another way of saying it is that u, if you pass it through the matched filter to u-- now remember what a matched filter is on an analog waveform.
You take a waveform, you switch it around in time.
You take the complex conjugate of it, and that's the matched filter.
And when you convolve u with this matched filter, what it's doing is just exactly the same operation of correlation.
OK in other words, convolution with one of the sequences turned around this time, is the same as correlation.
And most of you have seen that I'm sure.
So that if we take this matched filter where u tilde sub j is equal to the complex conjugate of u at time minus j, then I pass u through the filter G. Forget about the noise for the time being.
I then pass it through the matched filter u tilde.
What I'm going to get out, I claim, is G. And I'll show you why that is in just a minute.
Let make caution you about something.
Because you can get very confused with this picture.
Because as soon as I take this input, u 1 up to u sub m. This matched filter is going to start responding at time u sub minus m. And it's going to finish responding a time u sub minus 1.
So it responds before it's hit.
Which again is this business of thinking of timing at the receiver being very much delayed from timing at the transmitter.
Which is a trick that we've always played.
Which is why we don't have to think of filters as being realizable.
Still in this example, this becomes confusing.
And I'll show you why in a minute.
OK so I'm going to assume that I picked a good PN sequence.
So when I convolve it with its matched filter I essentially get an impulse function, namely a discrete impulse.
Which is the same as saying that u is orthogonal to all of its shifts.
And that's exactly what you what to do.
You want to think of turning it around and passing it through.
And that's exactly what this is doing.
OK so we have the output of this filter, which is u convolved with G. We're then convolving that with this matched filter u tilde.
And now we use the nice property of convolution, which you probably don't think of very often.
But the nice property that convolution has, is that it's both associative and commutative.
OK. And therefore when we look at V prime times u tilde, it's the convolution of u with G-- that's what the prime is-- all convolved with the matched filter u.
Because of the associativity and the commutativity, you can reverse these two things so you're taking the convolution of u with its matched filter.
When you take the convolution of u with its matched filter, you get a delta function.
And you take a delta function and pass it through G. And what comes out is a delta function weighted by a-squared n, which is just the energy of what we're putting in.
OK so that says that if we can find pseudonoise sequences, all of this works.
And it works just dandy.
If you put noise in, what happens there?
Well let's analyze the noise separately.
The noise is going through this matched filter.
Well if u is a pseudonoise sequence, if it has this nice correlation property and you flip it around in time, it's going to have the same nice correlation property.
So that in fact u tilde is going to have the same property that it's orthogonal to all of its time shifts.
If you now look at what happens when you take Z and send it through this filter, and you find the covariance matrix for Z passed through this filter, what that independence gives you is the correlation function is just all diagonal, all terms, all the same.
Which says that all of these terms and this vector here are all white Gaussian noise variables.
So what comes out is the filter plus white noise.
Which is the same thing that happened when we put in a single input with zeros on both sides.
OK.
So using a PN sequence works in exactly the same way as this very special pseudonoise sequence, which just has one input in it.
Which happens to be a pseudonoise sequence in this term also.
OK so the output then, is going to be a maximum likelihood estimate of G. OK, this is the way that people typically measure channels.
They use pseudonoise inputs.
And the output that comes out, namely the output that comes out.
When we put in a finite duration pseudonoise sequence, what we're going to look for is the output at the exact instant of the last digit as the input goes in.
And the output then is G sub 0, followed by G sub 1, followed by G sub 2, and then silence.
So you see nothing coming out until this big burst of energy, which is all digits of G. OK so now we want to put all of this together into something called a rake receiver.
I wish I could spend more time on the rake receiver because it's a really neat thing.
It was developed in the 50s about the same time that information theory was getting developed.
But it was developed by people who were trying to do radar.
And at the same time trying to do a little communication along with the radar.
And this was one of the things they came up with.
So they wanted to measure the channel and make decisions in transmitting data both at the same time.
And the trick here is about the same as the trick we use in trying to measure carrier frequency, and make decisions at the same time.
Namely you use the decisions you make to measure the frequency.
You use the frequency that you've measured to make future decisions.
And here, we're going to do exactly the same thing.
We make decisions.
We use those decisions as a way of measuring the channel.
We then use the measurements of the channel to create this matched filter G tilde.
And that's what we're going to use to make the decisions.
OK if you have two different inputs, I mean here we'll just look at binary inputs.
You take u sub 0 and u sub 1, and you look at what happens when you have those two inputs.
This is just a vector white Gaussian noise problem that we looked at in quite a bit of detail when we were studying decision theory.
What we want to do is to look at, I mean if these two signals are not antipodal to each other you want to look at the mean of them.
And you'll want to look at u sub 0 minus that mean, and u sub 1 minus that mean as two antipodal signals.
When you go through all of that, you find that the maximum likelihood decision is to take the real part of the output, of the inner product of the output, with u sub 0 convolved with g, and the real part of v convolved with u sub 1 convolved with G. OK in other words, what's happening here is that as far as anybody is concerned, we're not using u sub 0 and u sub 1 in this making a decision.
We know what the channel is.
And therefore what exists right before the white noise is added, is these two signals u sub 0 convolved with g and u sub 1 convolved with g. So we're doing binary detection on those two known signals.
And we're using the output to try to make the best choice between them.
So this is the thing that we do.
So we want to use a filter matched to u sub 0 convolved with g. Now how do we build a filter matched to a convolution of two things?
Well we convolve u sub 0 with g. And then we turn the thing around.
And then we see that after turning it around what we've gotten is the turned around version of u convolved with the turned around version of g. I mean write it down and you'll see that that's what you have.
So what you wind up with is the following figure.
You either send u sub 0 or you send u sub 1.
This is a way to send one binary digit.
We're sending it by using these long PN sequences now.
If u sub 0 goes through g, we got a V prime out, which is the output before noise is added.
We then add noise so we get.
And then we pass to try to detect whether this or this is true.
We take this output V. We convolve it with u sub 1 convolved with g, and with u sub 0 convolved with g. Now you'll all say I'm wasting stuff here.
Because I could just put the g over here and then follow it with u sub 1 or u sub 0.
Be patient for a little bit.
I want to put both of them in.
And I want to put them in this order.
And then I make a decision here.
OK well here comes the clincher to the argument.
Look at what happens right there.
If I forget about this and I forget about this, what I get here is u sub 0 coming in.
It's going through the filter g. It has white noise added to it.
It goes through the matched filter to u sub 0.
And what comes out is a measurement of g. That's what we showed before.
When we were trying to measure g, that was the way we did it.
We started out with a PN sequence, go through g, add noise-- we can't avoid the noise-- go through the matched filter.
That is a measurement of g at that point there.
And if we send u sub 1, that's a measurement of g at that point there.
So finally we have the rake receiver which does both of these things it once.
You either send u sub 0 or u sub 1.
You go through this filter.
You add white noise.
As far as making a decision is concerned, you do what we talked about before.
You compare this output with this output to make a decision.
After you make a decision you go forward in time, because we've done everything backwards in time here.
And you take what is going to come out of here, which hasn't come out yet.
And you use that to make a new estimate of g. You use that estimate of g turned around in time, to alter your estimate of the matched filter to g. And if you read the notes, the notes explains what's going on as far as the timing in here, a little bit better than I can here.
But in fact, this is the kind of circuit that people actually use to both measure channels, and to send data at the same time.
I want stop here because we're supposed to evaluate the class.
The following content is provided under a Creative Commons license.
Your support well help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The CDMA and the cellular standard, the IS95, which is a kind of an old standard by now, but it's still widely used.
There's still three major standards that are used for cellular communication, cellular voice communication.
This is one of them.
This is the one which is primarily being used as a jumping off point for all of the new standards that people are trying to think of.
I want to talk about it for that reason but also because it really is a nice way of pulling together almost everything we talked about in the course.
All of it comes into this one particular way of sending cellular voice.
Let's go through some of the details of it.
Some things here might sound more detailed than what you're interested in but I think it's good to have the numbers on it because it gives you some idea of which things, which factors are really relevant and which ones aren't so relevant.
So it uses a frequency ban from 800 to 900 megahertz.
Most of the new standards that people are thinking up are up in the gigahertz range around two gigahertz or five or six gigahertz.
I don't know the exact bandwidths.
All of these bandwidths depend critically on what the FCC wants to allocate and also depend critically one which particular bands are good for electromagnetic propagation and which ones tend to absorb all of the radiation.
Anyway, you use the band 800 to 850 for the reverse channel for the cell phone to the base and you use the band 850 to 900 for the forward channel.
Why do you want to separate the reverse channels and the forward channels by as much as you can?
If you think for just a minute about the power levels at your sending antenna, either at the base station or at the cell phone itself, and then you think about how much power you're receiving, he see there's an enormous difference between these two.
Because things get attenuated enormously after they got transmitted.
So what you're trying to do is to filter out what you're sending, which is an enormously high power and you have to filter it out by enough so that it doesn't clobber what you're trying to receive.
This means you need very good filters.
How do you build very good filters?
It's a lot easier to build good filters if you're not trying to make them cut off immediately.
If you have 50 megahertz, and you actually have a lot more than that, you have exactly 50 megahertz because this is all separated into a bunch of different channels.
People in wireless, most practical communicators, when they talk about a channel, what they're talking about is a particular bandwidth.
All along what we've been talking about is a particular medium going from transmitter to receiver.
That's what they're talking about is these channels, these channels or sub-bands are 1.25 megahertz wide each.
We'll see what the significance of that is in a while.
So you have a transmit and receive pair which are both 1.25 megahertz wide and which are separated by 50 megahertz.
So you have one channel then, at the bottom of the range, another channel a little bit higher, another channel, a little bit higher, and so forth all the way up.
So you have lots of different channels and you put lots of different cell phones in each channel for each base station.
So that's roughly the overall architecture of it.
It's the same kind of system that we've been talking about all along.
You start out with voice wave forms, you have voice compression.
You then have channel coding, you then have modulation and then it goes out on the channel.
Guess what happens when it comes back?
You have demodulation, channel decoding, and then you turn it back into voice.
Those are the three major steps which we viewed before primarily as two steps.
All of the channel stuff is much harder than all of the source stuff.
Let's talk about the source stuff first.
The input voice gets segmented into these 20 millisecond segments.
Why 20 milliseconds?
Well it sort of seems like it's probably a number that somebody picked out of a hat.
It's kind of striking that all of the standards use the same interval.
If you think about it awhile -- I'll talk about this more on the next slide -- the reason for this interval is that in voice communication, delay is enormously important.
You don't want to have much delay and since you don't want to have much delay, you have to do encoding over very short segments.
Well anyway, what this does is it takes each 20 millisecond segment of voice and it maps it into 172 bits.
What that means is if you multiply the 172 by the 50 segments that you have per second, it comes out to be 8.6 kilobits per second.
That might not sound like much of an achievement.
Think about the old fashioned way that people used to encode voice back in a time when they were first doing digital communication.
What you would was you would think of the voice as having a bandwidth that went from about 400 hertz up to about 3,200 hertz.
You would then view it as a base band wave form zero to 4,000 hertz.
If you were going to try to sample that wave form you'd have to sample it 8,000 times per second.
If you want good quality, you then have to use a large number bits.
You need a large number of bits for each sample.
The standard thing was to use eight bits per sample.
That gave you 64,000 bits per second in the old fashioned way of doing it.
People have worked for many years.
Many engineers at the old Bell labs spent their whole career trying to find out how to compress voice into smaller and smaller numbers of bits.
You know, 8.6 kilobits per second is still sort of a good rate.
It's still a good rate if you insist that you have to do it in 20 millisecond segments.
If you can take much longer segments to encode voice, you can do much better.
Voice really has a lot of constraints over relatively long periods of time.
Well actually, you know this because if you take a person's voice and you map it into text and then you also add a few things about pitch and things like that, you can really come by with a very small number of bits.
Here you're constraining yourself to something that sounds like the actual voice and also has to be done in these 20 millisecond segments because of delay.
So it still is a reasonably good achievement.
All of the standards have figures about like this, in the same order of magnitude.
I expect that all of the new standards will do similar things.
They will probably have slightly smaller numbers of bits.
They will probably do much more computation.
They will achieve a little bit more because this was already -- I think -- relatively close to how far you can go.
The next thing they do is they add 12 parity checks per segment.
They use that for error detection.
I don't want to talk about that a lot because it probably isn't necessary.
All of the standards do something or other like this.
The reason is, if you take this 20 millisecond segment and you go through all of this enormous processing that we're going to go through: namely turning it into an encoded wave form, transmitting it, receiving it, detecting it, doing all the stuff we're going to do to it.
If you get the 20 millisecond segment confused, and usually if you make errors you make a lot of errors.
If at the receiver you wind up with the wrong thing, it is really going to sound awful.
They found that it's much better if you're not quite sure of what that segment is, to just send silence for 20 milliseconds.
The silence is hardly detectable at all; it just sounds like good voice.
I mean for most of us it really sound wonderful to have a little more silence than we usually do.
Well enough of that.
Then there's another thing.
This is eight zeros per segments that are added to this 172 bits as something which is called a terminator for the convolutional code.
I might say a little bit about that but not much.
So what we've done is to add 12 bits and eight bits to this 172 bits.
That brings us up to 192 bits, which brings us up to 9,600 bits per second which is what we're now going to try to transmit.
OK so that's the voice compression part of the thing and a little bits of overhead done for various special functions.
The main reason I want to talk about these overhead factors is that every communication system I know starts out with very nice principles studying very carefully what are the essential things.
By the time you got done with it, there are all sorts of little overhead items that got thrown in to make the thing work and it's frustrating to everybody.
It's usually 10% or 20% of the capacity of the thing, but it's always there.
That's why I bring up these two terms because that's exactly the kind of things they're doing.
They're just patching to make things work.
So you have many details that lose all of the efficiency that you would like to have.
Those are what makes it hard to compare different systems.
Each system, architecturally nice though it might be to start with, always winds up with these little things because of these silly little constraints that say, since you have to have small delay, you have to segment voice into segments which are not much more than 20 milliseconds long.
Those things cause you to do these other things.
That's where all of this comes in.
OK, all of the timing in this IS95 cell phone system, everything is keyed to this 9,600 bits per second and it's all keyed through this 20 millisecond interval.
Everything is done in terms of 20 millisecond intervals.
It's all a completely block system.
The source generates 20 milliseconds worth of data.
You encode that into a 192 bits.
That gets mapped into some code word, gets turned into some high frequency wave form.
At the output you look at 20 milliseconds and that gets turned back into your best hope of the 172 bits that you started with and from there to some wave form.
Every 20 milliseconds is completely independent of every other 20 milliseconds.
You don't even save any knowledge about what the channel is between those periods of time.
Everything is independent from one to the next.
The point is here, we talked a great deal about all of digital communication sort of being generated from people's realization that they ought to separate voice coding from channel coding.
In other words, there's just binary interface between all sources and all channels.
This is a violation of that; all cell phones have this violation.
They don't have this pure separation because all of them face the fact that to do this small delay -- anytime you want to get small delay -- you have to have some mixing of what you do with the source and what you do with the channel.
The delay has to be some combination of what happens in both places.
So, we do have that violation and it's because of interactive voice.
You probably never thought about why it is that you need small delay on voice.
If you try to talk to somebody with a 50 millisecond delay, in between what you're saying and what the other person is saying, you will find it turns you into a stutter in about five minutes, everyone.
The problem is we all have these social conventions that we use to tell us when we can start to talk when we hear silence from the other person.
If you have a break in that routine, even at 50 milliseconds, it totally screws it up.
Both people start to talk at the same time and it's very hard to have a conversation.
Back in the early days of telephony, they found that about as much delay as they could tolerate was about 20 milliseconds.
When it got much longer than that, it gets very objectionable.
If you talk to somebody on the phone who is in Japan, you have a little more delay than that.
You find it's very hard to talk to them.
I mean, you have to practice a little bit, you have to think about it.
If it's somebody for whom English is a second language, or you have Japanese as a second language, it's very easy.
Then you both talk very slowly so you catch up on that delay.
but.
If you're both talking English very rapidly, or both Japanese very rapidly, it becomes hell on wheels.
As far as the channel is concerned, we could also do very well by using longer code words on the channel.
Again, we're stuck by this 20 milliseconds.
Everything has to be blocked into 20 millisecond periods.
OK, so the first thing as far as coding is concerned, is what's called a convolutional encoder.
We unfortunately, have not talked about coding at all in this course; we're not going to talk about it.
I just want to enough about convolutional codes so you get some idea of what they are.
Here's the simplest example of a convolutional code that I think you can think of.
You have a stream of input bits which are coming in one per second.
Each time an input bit comes in, that input bit sits at this dock here.
I guess most people would prefer to put an extra memory element in but it just complicates the diagram.
You have this input bit, you have the previous input bit, and you have the input bit before that.
What comes out, a time J is a linear combination of this bit, that previous bit, and the bit one bit before that.
You call this a convolutional code with a constraint length of two because it has memory of these two bits.
If you think about it a little bit, this is a device that has four states.
OK, because at time J, it needs to remember whether this bit was a one or a zero and whether this bit was a one or a zero.
So it's just a linear modulo two device that happens to have four states.
It produces two bits in each interval of time which are functions of this bit and of the current state.
In this convolutional encoder which is used in IS95, the constraint length is eight instead of two.
In other words, you have 8 of these memory devices here and the rate is one-third instead of one-half Here you have two bits coming out for each bit going in.
There you have three bits coming out for each bit going in.
It's the same principle.
If you think about it a little bit, as you increase the constraint length, the number of possible states that you have is going up very rapidly.
So, if you have a constraint length of eight, which means eight binary digits stored there, you have two to the eighth different states, which is 256 states.
If you added one more state to this thing, you would double the complexity.
Well, it turns out that because of the way the decoder works, if you added one more bit to this block length, it would multiply the complexity of the decoder by a factor of two.
So, when you try to decide how long these constraint lengths should be, you have to take into account that every time you make it one bit longer, yes, the thing is going to work better.
Also, you make it twice as complex.
Therefore, you have to wait another six months before you produce it at the same cost.
OK, we're going to use a Viterbi algorithm for decoding.
If you don't know what the Viterbi algorithm is fine.
If you think in terms of finite state devices, think of it this way; you have a device here which has four states.
Somehow the decoder has to realize at each instant of time, if it's gonna decode, it could either decode the bits here.
A nicer way to think of it is, it decodes the state, at each instant of time.
If it decodes the state at each instant of time, it doesn't really have to have any memory.
As soon as it knows what the state is at one unit of time, then it has a very simple decision to make to figure out what the incoming bit is and what the state is at the next instant of time.
And what a Viterbi decoder is, is something that does maximum likelihood decisions on the state.
There's a nice trellis diagram to talk about how that's done, in the notes.
You can read about it, it's interesting.
Many of you who have probably seen about it, it's sort of the easiest thing to explain, as far as coding theory is concerned.
It's the easiest thing to explain until you put all the notation in.
Then whoever does it, as soon as they put all of the notation in, it becomes a bloody mess.
Anyway, the idea is simple.
So at that point, we can sort of summarize what we've done as far as waste compression and channel coding.
So far, the IS95 standard is pretty much the same as any other cellular standard.
All of them, it turns out, use convolutional codes.
Whether that's a good idea or not is another question, but they all do.
It certainly is a very sensible thing to do.
All of them use the same segmentation.
All of them use similar techniques for doing voice compression.
All of them have all these little overhead terms that come in.
If you look at the numbers of what's going on, we started out with a 172 bits per second, which was what comes out of the voice compressor.
You then have these two overhead items.
Incidentally this eight bit convolutional terminator, we can now get some idea of what that's for.
You put in your 172 bits here.
You're thinking of this thing in terms of the state of each unit of time.
If you're going to figure out what's going on at the end, after you put in your 172 bits, you have to put in a couple of extra zeros to let this thing settle down.
You have to let this bit go all the way through this device so you got enough data out about it that you can decode it.
So that's what that terminator is for, for putting zeros in at the end of the sequence so you can view it all as one block code.
Again, this is overhead, which is caused by the need for small delay.
If you're going to need small delay, that overhead term would be much, much smaller.
So at that point, we're up to 9.6 kilobits per second and we're up 192 bits per segment.
Then we go into this convolutional encoder and the convolutional encoder multiplies things by a factor of three.
In fact, the rate one-third is a little bit different for most of the other standards because one of the things that we're going to be doing here, since we're mapping this input into one and a quarter mega bits, that's a considerably broader band than these other standards used.
The other standards are quite narrow band and therefore they don't want to have a large number of bits per second.
Here we're going to wind up talking about 28.8 kilobits per second at this point which is 576 bits per second, which is three times what we started out with.
We've expanded the number of bits by a factor of three.
If you question whether this makes sense to get a slightly smaller error probability through coding but increasing your rate by a factor of three, that's a perfectly sensible question to ask.
It just turns out that you do get a saving from that and that's what all of coding theory is about, how much savings you actually get.
In fact, the savings are pretty considerable.
So, at this point, we're up here at 28.8 kilobits per second with 578, 576 bits per segment.
We then go through an interleaver.
Don't ask yet what the interleaver is for.
All the interleaver does is takes these 576 bits and it's scrambles them all around.
The receiver knows how they've been scramble so the receiver can unscramble them again.
Why you'd need that is something we will talk about later, a good deal later.
The next thing we're going to do, you thought you were through with coding but no.
In this system, they're actually two stages of coding that you go through.
You can view one of them as modulation instead of coding.
Therefore, you can claim you're only doing one stage of coding and one stage of fairly complex modulation.
You remember when we talked about orthogonal codewords.
We said we could view orthogonal codewords almost as easily, either as a coding technique or as a modulation technique.
Mainly, you'd gather together lots of bits and then you'd form codewords, which we're signals, which had a large number of dimensions in them.
We're going to do the same thing here.
We are going to take this interleaver output; we're going to segment it into six, six bit blocks.
Then each six bit block is going to choose one of a set of 64 orthogonal codewords.
I mean, with six bits, you have two to the sixth, which is 64 different combinations.
So, if you're going to use orthogonal codewords, you need 64 of them.
When we were talking about orthogonal codewords before, we didn't quite know how to generate them.
We then said last time, that may be using PN sequences is a nice way of doing it.
That's not quite exact.
Here using Hadamard matrices is a nice exact way of doing it.
That happens to be the way it's done and it's simple.
I mean, you have a Hadamard matrix for each integer b.
If you start out with b equals one, which is one bit that you want to encode.
You have a matrix which consists of 2 orthogonal codewords one of which is 0, 0 and the other is 0, 1.
There aren't too many choices for that and they're all equivalent.
Then when you want to go to b equals two, the thing you'd do is make a matrix and the rows of the matrix are going to be the codewords.
The columns correspond to the bits in the codeword.
The first codeword here is 0, 0 second codeword is 0, 1.
The first codeword here is 0, 0, 0, 0 the next one is 0, 1, 0, 1.
For orthogonal codes, the block lengths of the code and the number of codewords is going to be the same.
So the thing you'd do is you take this matrix, you put it there, you put it there, and you put it there.
Then you compliment the matrix and you put it there.
OK why does that work?
Well if we just look at this part of it, this was an orthogonal code, this was an orthogonal code.
In other words, each of them differ from each of the others in half the positions, which is why you think of it as being orthogonal.
This is the confusion that runs through all of communication theory.
We might as well get rid of that confusion right now.
When you talk about binary sequences and you say they're orthogonal, what you mean is, if you turn that sequence into a 2PAM signal, those corresponding signals are orthogonal.
In other words, an orthogonal binary sequence, an orthogonal set of binary sequences is a set of binary sequences where each of the sequences differ from each of the other sequences in half of the positions.
Here with these Hadamard matrices, the way that's done is that one of the sequences is always all zero and the other sequences are all half ones and half zeros.
It turns out that when you add any two sequences, add this sequence to this sequence for example, you got something else, which is half ones .
In fact, something else which is in there, is what's called a group code.
When you add any two codewords you got another code.
Anyway, when you look at this, which is mapped into here and here -- I think the argument is difficult here because it's too simple minded, so you don't see it in it's generality -- each time you go from one b to the next b, the thing you do is the same thing.
You start out with this matrix, put it there, you put it there, you put it there, and you put the complement of it there.
What that does is when you look at the first half of the matrix, all of these differ in half the positions because they differ in half the positions here, and they differ in half the positions here.
So, they differ in half the positions over all.
When you look at everything down here, they differ in half the positions for the same reason because complementing this doesn't change any of those distance properties.
When you look at the difference between these and these, it's the same kind of thing if you think about it for a couple of minutes.
So, this is an orthogonal code.
Why do we want to use this orthogonal code rather than just an orthogonal code which uses the first degree of freedom and sends an enormous amount of energy?
Or the second degree of freedom sends an enormous amount of energy and so forth?
That's the same thing we were talking about last time with regard to channel measurement.
You don't want to send enormous amounts of energy in any one degree of freedom.
You want to spread it out over all the degrees of freedom.
That's what all these codes are doing; all these codes are using 2PAM on each degree of freedom to generate a codeword.
So the codewords all have sort of uniform power in them.
OK, so at this point, we have gone into this orthogonal code generator with 28.8 kilobits per second.
We have come out of it with 28.8 kilobits per second times 64 over six.
Namely for each six bit segment in, we've gotten 64 bits out.
So the total number of bits that we have has gone up by a factor of 64 divided by six.
So we're suddenly up to 307.2 kilobits per second.
What we're trying to do is get ourselves -- by hook or by crook -- get ourselves up from these very small bit rates that we need to represent voice, up to these large bit rates, 1.25 megabits per second, which we're going to spread this wave form to.
All the reasons we want to spread the wave form, I'll talk about them later.
For now let's just think about it as saying, somehow or other we want to spread the wave form out and turn a small number of bits into a large number of bits.
So we've done a pretty significant part of that right here.
If we think of turning these binary rows of the Hadamard matrix and modulate them by 2PAM we wind up with wave forms.
The wave forms are orthogonal.
If you wanted to think of taking these wave forms and then transmitting them up to passband, we would wind up with the bandwidth of 307,000 hertz.
That's not enough yet, we want to go up to 1.25 megahertz.
Anyway, these wave forms are called Walsh functions.
So the wave forms that you got from Hadamard matrices are called Walsh functions.
You have to find something to honor everybody that's done a significant amount in this field and Walsh was one of these people.
Walsh functions are quite famous and all they are are just these orthogonal wave forms.
So, we now have an orthogonal code of length 64.
The receiver, I'm going to talk about the receiver more later.
In essence, what it's going to do is take the 64 orthogonal wave forms, only one of them will get transmitted in each of these small units of time.
When the period of time corresponding to six bits into the encoder, we have to decode which of these 64 wave forms was sent.
We're going to do that by using a rake receiver.
That's what's used in IS95.
This rake receiver is different from the one we talked about.
The one we talked about, we were making a decision between two hypotheses.
Here we're making a decision between 64 possible hypotheses.
If you can remember the structure of that rake receiver, it sort of split into two separate sections.
One section was used for the decision on one of the wave forms, the other one, on the other wave form.
When we do this with 64 wave forms, it spreads into 64 different sections, which is why it's called the rake receiver.
If you draw a picture of it looks like a rake.
You start out here with the handle and then you have it spread out into 64 different tines and all of these tines have all of this signal processing going on.
At the output you come back and you change what your model of the channel is depending on the actual decision that you've made.
So what we're using is a rake receiver but a rake receiver which has 64 decision devices instead of two.
Suppose we go into an orthogonal code using seven bits instead of six bits.
Now, at this point, you're talking about real money.
OK, because at this point, you're talking about taking this rake receiver and turning it from 64 arms into 128 arms.
Now, you can do that because these things are essentially free now and computation is essentially free.
At the time when this was designed, there was a very hard constraint in terms of complexity about how many bits they can encode together into orthogonal wave forms.
I'll show you why in a little bit.
It wouldn't have done them much better to have a larger set of orthogonal wave forms anyway.
It also uses incoherent soft decisions as a way of doing its detection.
We talked about incoherent detection.
This rake receiver uses that instead of the coherent detection that we talked about in class.
If you just put together in your mind, the lecture we spent talking about rake receivers and the one we spent talking about incoherent detection and you put them together, you have what this receiver is doing.
I'll talk about just a little more as we go.
OK, if we now imagine these are orthogonal codewords and suppose that each orthogonal codeword has an energy e sub s. Suppose for the time being that the channel is represented perfectly by a single-tap model and we know what the energy in that single-tap model is.
We don't know what the phase so we're using incoherent detection.
What's the error probability?
Well the error probability for an incoherent receiver for making a decision between two possibilities was one-half times e to the minus energy divided by 2 N 0.
Here we have 64 different possibilities.
So we're going to transmit one of these wave forms and we now use the union band to say there's this probability of error to each of the other 63 possible wave forms.
we could make an error to any one of them.
That's equally likely to make an error to any one of them because they're each orthogonal to the one that we're dealing with.
So this is a reasonably good bound on the error probability.
OK, now you go back from this to look at the reason why, one of the main reasons why you want to do this.
E sub s is the amount of energy you use to transmit six bits instead of one bit.
Therefore, when you look at the energy per bit that you're using, it's e sub s divided by six.
So, your probability of error, -- if in fact you were making hard decisions at the output of this incoherent detector -- would be in the order of 63 over 2 times e to the minus 3eb over N0.
This is a considerably larger exponent and therefore, a considerably lower error probability than we were getting when we were talking about transmitting single bits.
In other words, the same thing is going on here as was going on when we were talking about getting up the channel capacity using a large number of orthogonal wave forms.
It's the same idea.
When you encode lots of bits together, you make a joint decision on all of them, you make a gain.
That's exactly what's happening here.
You made a gain because you've taken the amount of energy that you have available for each one of your bits and you've combined it all so that you get this bigger exponent in here.
If you're lucky, the exponent is big enough that it overwhelms this term and your decision comes out with very small error probability.
Well, as it turns out, we're not interested in making these hard decisions.
What we're really interested in doing is just sending a likelihood ratio back to this Viterbi decoder which it can use in making final decisions.
To see how well that's going to work, this is exactly the thing you want to look at.
This is what's telling you, if E b is large enough your decisions, if you had to make them, would be very good.
Therefore, if instead of making decisions, you pass likelihood ratios on, those likelihood ratios are going to have a lot of information in them.
Essentially, those likelihood ratios tell you what those codewords are.
OK, now, the effect of fading is to change this E sub b because when we were talking about this incoherent detection, we were looking at the energy per bit after going through the channel.
So, this is the error probability for a particular received energy per bit.
Sometimes when the channel is badly faded, this number is very small.
This bound is a lot bigger than one but you don't care about it because the bound is no good then.
When the channel is good, this number is large and this probability there is very small and your likelihood ratios give you a lot of information.
If you had multitap diversity instead of the single-tap here, if you have a lot of taps.
If you have a lot of taps you can visualize this as getting independent readings on the channel.
Therefore, you can view it as getting independent information about the bits that were sent and this is what diversity is.
I mean, the trouble with these Rayleigh fading channels is when they're faded there's nothing you can do.
If you have multiple taps instead of one tap, then in fact, you have diversity between all of them.
If any of them are good, then you decode.
Yes?
You were saying earlier that PAM is not ideal [INAUDIBLE].
[INAUDIBLE] Why is the standard using 2PAM instead of pulse position modulation?
Well because it's using these long codewords.
Because back when we were analyzing it, we were analyzing it using two codewords.
Those two codewords were orthogonal to each other.
We could have made them plus one minus one and plus one minus one for the other one if we had in fact known what the channel was ahead of time, OK. That brings up a good point because you have to get to started somehow.
I mean you have to know somehow, what's going on at the channel at the point where you start.
I'm not going to talk about that.
If you have this multitap diversity than E sub b is going to be better than it would be.
Otherwise, this says, if you can make the bandwidth bigger, as you make the bandwidth bigger, the bits are coming in faster.
As the bits are coming in faster, this bit time is the separation between the taps and this digital model.
So that as you make the bandwidth bigger, you automatically get more diversity.
You get more taps in this discrete model.
Looking at it another way, you have a certain frequency coherence in the channel.
If you're sending a broad bandwidth, you get lots of different frequencies.
If you get lots of different frequencies, there's a good chance that one of them is good.
OK, so we are going to get some gain in diversity there.
Typically we're going to get a relatively large amount of energy if the thing works.
Well, now we come to a question that we've been carefully avoiding all term.
It's sort of the question of, if you make codewords orthogonal or if you make codewords have nice structure at the input to a channel and the channel has some strange behavior, what's going to be the output of the channel?
What we're doing here is making our codewords orthogonal at the input of the channel, we can't guarantee that they're still orthogonal at the output.
What we do know is that PN sequences are almost orthogonal anyway.
If you put PN sequence through anything, they still look like PN sequences.
In other words, if you think of a PN sequence as an IID binary sequence, and you have a very long IID binary sequence, and you corrupt it in any way you want to, it still looks like an IID binary sequence.
So, the point is, if you can make these orthogonal codewords look more like PN sequences, you can make them behave better at the output of the channel as well as the input to the channel.
So you'll get something that looks orthogonal whether the channel essentially looks like a one tap channel or a multiple tap channel.
OK, we also have bandwidth to burn because we're only up to 307 kilohertz.
What they decided to do was to take a PN sequence and run it at four times the speed of this bit sequence.
So we have bits coming in at 307,000 bits per second.
We're going to have a PN sequence which is changing four times for each bit time.
So suddenly we're increasing the speed of this whole system by a factor of four.
For every one bit, we multiply it by a sequence of four bits.
So instead of getting a change every one over 307,000 seconds, you now get a change every one over 1.2288 millionths of a second.
So, suddenly you're up to a sequence which is right at the bandwidth you want.
I mean we want it to get up to 1.25 megahertz.
Suddenly, we're up so close to that it's going to be a real challenge to build a filter, which will really make the output look like it's limited to this 1.25 megahertz.
So we need very tight filters here.
If you multiply this PN sequence , you're using the same PN sequence on all of the orthogonal codewords.
If you think just a little bit about it, if these orthogonal codewords -- I'm I mean think of them still as binary sequences -- if each of them differ from each other when in half the places, when you multiply both of them by this PN sequence, they're still gonna differ in half the places.
You still got the same affect out.
They're still perfectly orthogonal before going through a channel but now you have a good chance that they're pretty close to orthogonal after going through the channel.
So, suddenly, we've got it up to the bandwidth we want to be at.
There's another reason for the PN sequence.
I mean, so far we've been thinking in terms of, how do you build a cell phone and a base station so they will work together?
Actually, you want to build a base station and thousands of cell phones so they'll work together.
So you want some way of making different cell phones look different from each other.
What they use this PN sequence for is to make different cell phones look different.
Each cell phone is going to start out on this PN sequence at a different point.
The PN sequence is going to be generated by a 42 bit feedback shift register.
If you don't know what a feedback shift register is, don't worry about it.
Most of you probably do.
A 42 bit feedback shift register, with the right kind of settings in it, is going to generate a periodic sequence with a period of two to the forty second minus one, which is about four times 10 to the twelfth.
It doesn't repeat very often.
I mean, this really looks like an IID sequence and binary digits.
It really goes through every non zero 42 bit combination in this period here.
It has very nice properties to it.
But the property we want for this system is that in fact, when you give a cell phone a particular setting in that shift register, it starts it off generating a PN sequence which looks totally different from that generated by any other cell phone.
That's the only thing that makes these different cell phones different.
It's enough to make each one of them look orthogonal to each of the others.
OK, their basements sequences and thus passband wave forms are now going to be essentially orthogonal to those of other cell phones.
In other words, at this point, we're sort of going into fantasy world.
I mean, we started out generating 64 wave forms which were strictly orthogonal to each other.
Then we said we want these wave forms to still be orthogonal after we go through this unknown channel.
We said, well we can do this by making them look like PN sequences.
But the other thing that PN sequences do, is it all of the possible wave forms for one cellphone will look orthogonal to the wave forms for all of the other cell phones.
So that in fact, what's happening is, if you look at a base station, and the base station is trying to receive information from each cell phone, what it receives from one cell phone is going to be essentially independent of what it receives from the next cell phone.
Now if we think of the base station as trying to receive what's coming from one cell phone, what's the interference from the other cell phone going to look like?
We're going to take thousands and thousands of sequences, which are now turned into wave forms, which look like IID wave forms, IID binary wave forms, and now we're adding together a large number of them.
They fill up the whole spectrum uniformly.
So they have a spectral density -- I mean it's a random process -- and it has a spectral density which is flat over this bandwidth of 1.25 megahertz.
And you've added a lot of them together, so it looks like white noise.
So the effect of these other cell phones is just like a small addition of white noise as far as trying to detect what's going on at one cell phone.
Now, in fact you could be very clever here because, as soon as a base station decodes one cell phone, it can say, I know what that wave form was, subtracts it off from the received wave form, and uses that to help in decoding other wave forms.
These standards do not do anything as sophisticated as that.
Lots of papers talk about doing it.
Information theorists love this idea.
I think it's one of these ideas that sooner or later, ah going to become practical.
As soon as it becomes practical, everyone will use it.
What does it take to become practical?
For someone to use it, because we all know it'll work.
It's just a matter of actually doing it.
But anyway, it's not done now.
All of these interferes are just going to look like white noise as far as this one cellphone detection is concerned.
Now, if we compare this IS95 system with all of the narrow band systems that people use, at this point you can find the main difference between these different systems.
When you use a narrow band system, things are a ranged so that only one cell phone is talking to one base station on one of these narrow bands of frequency.
Namely the channels are very narrow but because they're narrow if you have two cell phones which are using that same bandwidth, they're just going to interfere with each other totally.
You won't get through.
So each cell phone which is using one base station has to be using a different narrow bandwidth.
Cell phones using different adjoining base stations are also going to have to use different bands.
In fact, that's a terribly complicated problem just because these base stations are put in sort of, random locations and trying to program things in such a way that you don't have two cell phones which are using adjacent base stations using the same frequency.
It's a real nightmare.
So, you're going to have cell phones using the same base station, which are a little bit separated, but still because of all sorts of channeling effects and strange electromagnetic effects, base stations still receive data from these far away cell phones.
They're also trying to receive data from the cell phone that's using that frequency.
So you got a lot of interference in these narrow band systems; you got a lot of interference from cell phones using different base stations.
In the IS95 system, you get more interference from cell phones that are using this particular base station because they're all using the same bandwidth.
They all just look like white noise to each other.
So they do interfere with each other, they raise the noise level.
When you look at the interference with cell phones using other base stations, it's not catastrophic like it is with these narrow band systems.
So what you effectively wind up with is that in the IS95 system, you have more interference within a base station but less interference between base stations.
In the narrow band systems you have no interference within a base station but considerable interference between base stations.
Which is better?
Nobody really knows.
Well except for the fact that almost all new systems are planning to use CDMA.
It seems that most people have sort of agreed that you get less noise, overall, if you're using if you're using a CDMA system.
So that let's us sort of summarize where we are with all of this.
We started out with 28.8 kilobits per second.
We then going into this Walsh function, orthogonal wave form and coder, which comes out with 307.2 kilobits per second.
We then multiplied by this long PN sequence, 42 bits long with four bits coming in here for each bit coming in here.
Now we're implementing this and anybody with any sense who was trying to implement PN sequences with multipliers would just say, I don't want to turn things into to PAM sequences yet.
I want to keep them binary and add them mod 2 because that's easier.
So all of this is still mod 2 binary digits.
This is mod 2 binary digits.
This is mod 2 binary digits.
The only thing we haven't talked about yet is now they throw in two other PN sequences.
One on the real part of what's going on here and one on the quadrature part.
The thing they're doing is taking this one sequence, splitting it into two parts; adding a PN sequence here, adding a PN sequence here, then doing the 2PAM, then filtering and sending this stuff out on the cosign channel and the stuff out on the sign channel.
I've never had a very good sense of why you need these two PN sequences here.
I can like sort of explain it to you in the following way.
If you didn't have PN sequences here, what you'd be doing here is simply sending this same sequence twice.
So you be sending it on the cosign channel, you'd be sending it on the sign channel, which would sort of be equivalent to sending it at 45 degrees.
Once you realize that, you say well, I might as well leave this whole thing out and just use the cosign channel and nothing else.
If I'm just using the cosign channel and nothing else it won't make much difference because the cosign just refers to the phase at this transmitter.
I'm talking about many cell phones whose transmitters are all at different places relative to a base station.
Therefore, they're all going to be using different phases so they're all going to fill up the signal space.
The thing that, that doesn't take into account is the fact that if we do that we're not using all the degrees of freedom available to us.
We're still just using the number of degrees of freedom that we got here, which are real degrees of freedom.
When we do this and this, we got twice as many degrees of freedom.
The PN sequences that we're looking at, which are the things that are orthogonal to each other.
We have twice as many bits in a give interval of time when we have these two PN sequences as we do when we only use this one PN sequence.
So in fact, if we view PN as IID binary digits, you get twice as many binary digits this way as you got before.
Therefore, everything works better.
You come much closer to being orthogonal with higher probabilities.
If you didn't understand that's fine.
It's not explained in the notes either.
It's not explained in the IS95 standard.
I think the explanation is right but it needs some fleshing out.
OK, here's another interesting feature about this rake receiver.
It doesn't use this digital model of the channel that we've talked about.
It in fact, uses the analogue base band model.
In other words, the base band model of the channel is the channel is represented by an analog filter.
The analogue filter has impulses or almost impulses at different delays.
Now what you'd like to do in this rake receiver here is instead of having taps at these sample times for the given bandwidth, they're going to adjust the taps so the taps are right on top of these things that look like impulses in all of these path returns.
So the question is, what's the difference between these two strategies?
I mean, we sort of show that the digital model was completely equivalent to the analog model.
Then I cheated you; I didn't realize I cheated you until about nine o'clock this morning.
I cheated you in the following way: I told you that if you model the channel in terms of the discrete set of taps according to the sampling therom, this was completely equivalent to trying to model it -- as far as a band limited input was concerned -- in terms of an analog filter.
That's absolutely right if you use an infinite number of taps for the channel.
If you now say that the impulse response of the channel is about one tap long and then you say, how many tabs do I need to represent it?
It's an interesting question because if the impulse response is one tap long and you filter to look like a sign x over x, the impulse response can look like this.
So you have one tap here, one tap here, which is zero, one tap here, one tap here, which is zero.
If you've gotten your spacing off by half a cycle, what do you have then?
You have a tap here.
You have a tap here.
You have a tap here.
You have these taps which are going down as one over time.
So in fact, modeling this kind of thing with one tap would be terrible.
Now when you use the analog filter and you actually put these taps of the rake receiver at the places where you have actual physical responses you do much better.
You particularly do better if the impulse response duration of the channel is the same order of magnitude as 1 over the bandwidth of the source that you're using.
At that point this becomes an important issue.
That's exactly the situation we have here because of the impulse response of these channels.
The sample time on this discrete model it's about eight-tenths of a microsecond.
If you think of how far light can travel in eight-tenths of a microsecond, it's about the same as the multipath spreads you would find outdoors when you're in a relatively rural area.
If you're in a city area, multipath spreads are going to be even smaller.
So if you want to a rake receiver to work well you should really have a rake receiver which is trying to zero in on the actual physical response times.
This is what this system does.
They did it right and I did it wrong.
OK, final thing.
I told you I wanted to talk about why we did scrambling.
This rake decoder, in other words, this decision device, which every six bits tries to make a decision on these 64 different orthogonal wave forms.
You can just make hard decisions which gives you six bits out each time.
You can produce six bits out also with some likelihood about how likely they are to be right.
So what you'd like is some information which is essentially the logarithm of the probability of being correct over the probability of error.
You can estimate that in your rake receiver.
You can estimate it by knowing whether you're making a close call or whether there's one orthogonal wave form which is far more likely than all of the others.
So if you're making a close call, then you produce an output that says, I think it's this output but I'm not very sure.
This is what goes into this viterbi decoder.
The viterbi decoder, since it's operating on these states -- in other words, it's operating on what it visualizes is being the state of the viterbi decoder -- it can actually deal with these log likelihood ratios as well as anything else.
That's one of the reasons you want to use a viterbi decoder because most coding devices that people have invented don't work very well with analog signals.
This does work well with analog signals.
The problem is, anytime you make a mistake on the 64 orthogonal wave forms, you have a six bit batch of digits, a six bit block, and you're going to be wrong, typically in three of them.
well, you're going to be wrong in a random number of them.
Half the time you're going to be wrong.
What this means is if you put this data into a decoder, what you're going to find is that every time there's an error in the in the rake receiver, you're going to have a burst of errors, some random bunch of errors over six bits.
This is typically going to be in the order of three bits an error all at once.
If you look at the structure of that decoder, anytime you got a large number of errors in all at once, it's going to screw the thing up terribly.
I'll give you one example of that.
You can take a much simpler coding example.
Suppose you transmit either all zeros, seven zeros or seven ones.
At the receiver there's a certain probability that you make an error on each one of these bits.
You first make a decision on each bit and then after that you make a decision on the whole code word.
If the bit errors are independent of each other, then you make an error in the overall detection of these code words only when you make four errors out of seven.
Or five errors out of seven which is negligible.
So you calculate, what's the probability of making four errors out of seven here?
Well it's the number of combinations of seven things taken three at a time which is 35 times p fourth, p to the fourth power which is the probability of making four errors.
If those bits are perfectly correlated mainly, if whenever you make one bit error you make all seven bit errors, then the probability of error is p If p is relatively small, p is always a lot bigger than 35 times p to the fourth.
In other words, the lesson here is that highly dependent errors cause trouble for any kind of decoder.
Now you see why you had the scrambler in here, why you have the interleaver.
You Have the interleaver to take these highly correlated errors which are coming out of the rake receiver where you make blocks of six errors all tend to appear together.
You're scrambling them out so they're widely separated so that the viterbi decoder is able to deal with them.
That's the last piece of the whole thing.
I could talk a little bit about how the viterbi decoder works but I think you're probably exhausted by now.
I think I will stop at this point.
Than you all for your attention all term and good luck on the final.
So many of you have signed up this term.
As most of you know, this course is now on an alternate year basis, but it really depends on how much interest there is in it.
So this is certainly very satisfactory interest.
This morning I'm just going to go through the information sheet logistics a little, tell you a little bit about the course, and then go through a very rapid path through the first three chapters.
So hang onto your seats.
But first of all, the information.
One thing I did not mention in the information sheet is that due to the sponsorship of OpenCourseWare and the department, we are going to have this class on TV or videotaped for OCW in the department archives.
And I hope that's OK with everybody.
I think at some point we'll send out a permission sheet.
Does anyone anticipate having any problem with the camera lingering on them or the back of them from time to time?
I don't think this is going to be an issue.
But if it is, either mention it now or come up and see me privately.
As I say, I think eventually you'll have to sign permission sheets.
Our cameraman is Tom White.
Let's give a hand to Tom.
And he promises to be as unobtrusive as possible.
OK.
There are four handouts this morning.
I hope you have them all.
First of all, the information sheet.
Second, a little personal data sheet that I ask you to each fill out so we know who's in the class and a little bit about you.
The first problem set, which is due a week from today.
In general, problem sets will be due on Wednesdays and handed out on Wednesdays.
And please do pass them in on time.
We won't make any serious attempt to grade them if they don't come in on time.
I'll mention our philosophy about problem sets in a little while.
And then chapters one through three, which are all sort of warm-up chapters.
And problem set one is this kind of warm-up exercises to get you into context, to get you into the mood for this course.
It doesn't have to do -- well, it does.
But at this point, it's more just getting up to speed with the language and the environment.
The teaching assistant is Ashish Khisti.
Ashish, why don't you stand up just so everybody is sure to see you.
I'm delighted to have Ashish.
He was a superb student this course.
Certainly understands it very well.
Ashish will even be a distinguished guest lecturer for lectures two through four, because I will be on vacation during that time.
So you'll get to know Ashish very well.
I think he will be able to help you quite a bit.
His office hours we've tentatively scheduled for Tuesday from five to seven in the basement area of Building 32, the Stata Center.
Is that going to be OK with everybody?
Is there anybody who's --?
It's of course set that way because the homeworks are due on Wednesday.
That's all right?
All right.
We aim to please.
I've already mentioned Tom White.
I want to mention that we've tentatively scheduled the midterm for Wednesday, March 16, which is the Wednesday before MIT vacation week.
And normally I've run it for two hours to try to minimize time pressure.
For 9:30 class, that's easy.
We simply start it at nine.
But again, if anyone has any personal problem with that, please let me know.
You may not know about that yet.
But I want to let you know immediately.
Prerequisite.
I've held to the policy of enforcing the prerequisite, which is 6.450, rigorously.
Not so much because the content is really what's needed.
This course is entitled Principles of Digital Communications two.
But what it really is is a course on coding, modern coding, in a communications wrapper.
Our motivation all along will be how to get to capacity on the additive white Gaussian noise channel.
Within that story, we're going to develop all the principal classes of codes that you would see in any coding course.
But we're always going to be measuring them by how well they do on a particular communications channel, the canonical additive white Gaussian noise channel.
But in fact, we're not going to use very much of what you had in 6.450.
The point of having taken 6.450 is simply to be up to a certain level of speed and sophistication in dealing with rather mathematical concepts.
As you know, 450 is rather mathematical.
Most of the Area 1 courses are rather mathematical.
This one is rather mathematical.
And if you simply aren't used to making logical arguments fairly quickly and in a snap, you're going to have trouble with this course.
So that's really the reason for a prerequisite.
And I have found it's important to enforce it, because we always get, each year, a couple of people who probably shouldn't be here.
So is there anyone here who hasn't taken a version of 450 before?
Yes?
What's your name?
Mesrob.
I emailed you
Oh.
So you emailed me -- Mesrob?
OK. And you had taken a couple of other very rigorous courses, and so I'm happy with your prerequisites.
The registrar told me that Mukul Agarwal and Daniel Wang, that's you, and Andrew Cross have not had the prerequisites.
Daniel?
What's the story?
I audited [INAUDIBLE]
You audited 450?
OK. Perhaps we could talk after class about, in general, what courses you've taken, and why you think you can keep up it
[INAUDIBLE]
OK. Or just let me know how you prefer to handle it.
We can do that immediately after class in a corner of the classroom, if you want.
And are the other two people here?
Agarwal?
No?
Cross?
So they seem to have selected themselves out.
All right.
Very good.
And everyone else here has had 450.
All righty.
There's no text.
Unfortunately there's really isn't any text that comes close to covering all the material we're going to cover.
Many of the individual chapters in this course are covered by various textbooks, of course, in far greater depth than I am able to cover them here.
So partly what you're getting from me is a selection of a thread of what you really need to know, at least first time through.
And over the years, I've developed course notes of my own, which I'm now reasonably satisfied with.
At least up through the last chapters, where we'll spend a little bit more time this year.
And so I hope that that will be satisfactory.
But I do give you some supplementary texts in here, and more are coming out all the time.
And part of being a graduate student is understanding where to find supplementary material if you need to.
So let me know if you ever feel -- if you ever want to know, where can I read more about something, I'd be happy to give you a suggestion.
Office hours I already mentioned.
Problem sets.
They'll be weekly, except before the exam and except in the last week to the course.
The purpose of the problem sets is for you to get practice in getting up to speed.
If you don't use them that way, you're not using them correctly.
I absolutely don't care if you do them together or whatever method is most effective for you to learn.
These are intended for you to learn.
I don't recommend going in -- many of these of have been offered in prior years, and the answers are to be found somewhere on the net or in somebody's library.
But as you've heard in every other class, I don't recommend your going about them that way.
Not much weight is put on them in the grading.
The TA or the grader will simply put a grade of zero, one, or two.
We'll be happy to discuss any of them where you feel you didn't really get it, or don't know how to get your arms around a particular problem.
And in fact, I wouldn't put any weight on the homeworks, except that every year students say, well, if you put 0% on the problem sets, then we won't do them.
And so I put 15% on the problem sets.
But in fact the problem sets -- we do get an idea of how engaged you are in the course and how much you're getting it from the problem sets.
That's important feedback, both generally for the class and individually for each of you.
You know, there have been cases where somebody is just copying the problem set from the previous year every week, and then we can pretty well predict that they'll bomb out on the midterm, and they do.
So anyway.
You've been at MIT a while.
This is pretty much the standard philosophy, I think, at least in Area 1.
Any questions about what I just said?
No?
The midterm.
Scheduled for two hours in the Wednesday, the last class before the vacation.
Counts for 1/3.
The final is scheduled during finals week.
I don't have control over that.
It will count for 1/2.
Basically, the grade comes from adding up your scores on those two things.
Doing a scatter chart and trying to make some intelligent guess.
Usually we get a little more than half A's, a little less than half B's.
But I don't have any fixed number in mind for that.
It's really, I try to do it based on how you do, using pluses and minuses for decoration.
And I don't know.
Again, I think it's like what you see in every other course.
All right.
The topics.
What's going to be different about 6.451 this year from 2003, which is the last time it was offered for previous years?
Not too much.
Basically, this course -- originally there was a single course, which covered 6.450 and 451.
It evolved, got split into two courses.
And this course has pretty much become a coding course.
So you are here because you're interested in coding, in particular for communications.
This is a good first course in coding for, I think, other interests.
Are you Andrew Cross?
Yes.
What is your situation on the prerequisites?
[INAUDIBLE]
OK. Let's talk about it afterwards.
Alright.
So as I said earlier, the course is pretty much stabilized at this point to be a course on coding over the additive white Gaussian channel.
Those of you who are here with other interests, that's not the only place that coding is used, I think will be reasonably satisfied with the course.
We're always putting it in a communication context, but we talk about all the major classes of algebraic block codes, convolutional codes, tail-biting codes, capacity approaching codes, low-density parity-check codes, lattice codes, trellis coded modulation, if you want to get ... and all of their decoding algorithms, which are just as important as the codes themselves.
So I hope that's what you're after, is to get a view of which codes have proved to be most important over the years.
My context is communication, but I believe you'll get exposed to most of the things you would need, regardless of why you happen to be interested in coding.
What is going to be different this year is I'm going to be rather ruthlessly chopping out not perfectly essential topics wherever I can through the course.
And in particular, I'm going to run through chapters one through three today, assuming that -- well, one is just introduction.
Chapter two is basically how you get from continuous time to discrete time on additive white Gaussian noise channels.
You did that fairly well in 6.450, so I assume all I need to do is basically sketch it for you.
And chapter three is proof of the channel capacity formula, which you may or may not have encountered somewhere else.
But even if you haven't encountered it, you're probably reasonably willing to take it on faith.
And so that allows us to get immediately into chapter four in the next lecture, where Ashish will be starting to work up from the smallest, simplest little constellations in Euclidean space, which is where the story begins.
This hopefully will give us more time at the end of the course for more on what's been happening in the last ten years or so.
First of all, a much more analytical treatment of capacity approaching codes than I've been able to give in the past.
The real theory.
How do you design them?
How do they perform?
Some more details.
Enough so that you could go ahead and implement them, I believe, and analyze them, and optimize them.
And that's, I think, really important in this course.
Not just because that's the way people are doing coding these days, but also because that's the end of the story.
After 50 years, we finally did get to channel capacity, and this was the way we got there.
In addition, if we really do well, we'll be able to do at least a week, maybe two weeks, on codes for band limited channels, where you need to go beyond binary.
You need to send codes, at least symbols, that have many levels on them, not binary levels, in order to send it at high spec coefficiencies, many bits per second per Hertz, which is, of course, what you want for most wireless or wireline channels.
You are able to send lots of bits per second per Hertz.
How do we do that?
Again, hopefully getting to capacity.
And finally, I always offer the teaser, and for at least five years, I haven't been able to get there.
We could talk about linear Gaussian channels.
Where you get into equalization, precoding.
Other topics that are very important.
But I'm almost certain we won't get there.
And in addition, this subject, this kind of a scalar version of what's done to a fare-thee-well in the matrix way, in the wireless course, 6.452, which is taught in the next classroom in the next hour and a half.
So I anticipate in future the whole equalization of precoding -- intellectually, it should be merged with Wireless.
Because they do matrix versions of all of that.
On the other hand, it forms a part of this story, which is just basically point-to-point transmission over the additive white Gaussian noise channel.
All right.
I very much encourage questions and feedback, and I'm always willing to be distracted, tell stories, respond to just what's on your mind.
It's much more fun for me, and probably helps you, too.
Does anybody have any questions at this point, or observations?
No?
All right.
We should just get into it.
OK.
So I say I always like to be thinking of this from a communications design point of view.
We are engineers.
I am a communications engineer and theorist.
And what's the design problem that we're really trying to solve in this course?
The design problem is that somebody -- your boss, or the FCC, or somebody -- gives you a channel, which usually involves some set of frequencies and ... over some physical medium and says, OK.
I want you to design a communications device, a modem, to get as much digital information over that channel as you can.
So in chapter one, we talk about a couple of channels which have been very important for the development of coding.
One, for instance, the deep space channel.
The deep space channel -- what's the problem?
You have a highly power-limited satellite way out there.
It's got a little tiny antenna.
It can use whatever frequencies it wants.
It's talking to a huge 140-foot dish or whatever somewhere in Pasadena or somewhere else around the world.
And that's basically the channel.
And you can send any wave form you'd like, subject, really, to power limitation.
This is an extreme example of a power-limited channel.
How do we characterize such a channel?
Well, frequencies are unlimited, but the signal-to-noise ratio is very poor.
You get a very tiny signal, and you get noise in the antenna front end, which they've done everything they can reduce it, they've cooled it to a few degrees above zero and so forth.
Nonetheless, the front end noise -- in the very first stages of amplification are what is the noise.
And you've basically got to transmit through that.
So you have a pure additive white Gaussian noise channel, which we we'll always write as the output of the channel Y of t is the input, X of t, plus the noise, N of t. Which is additive, independent of the signal, and white.
The spectrum can be taken to be flat.
All right.
And what else do we have?
We have that this has some average power P, and this has some noise power spectral density which is characterized by a parameter N_0, which is basically the noise power per Hertz.
Per positive frequently Hertz, as it turns out, because these things were all defined back in the dark ages.
OK. And we also, either because of the channel specification or because ultimately, we decide to signal in a particular bandwidth, we have a bandwidth W. Again, that's just measured over the positive frequencies.
So if it goes from zero to W, that's a baseband channel.
If it goes from W_0 to W_0 plus W, that's a passband channel.
But we're going to find out it doesn't really matter where the passband is, or the baseband.
So those are three parameters that specify an additive white Gaussian noise channel.
And we can aggregate them into a signal noise ratio.
A signal-to-noise ratio is the ratio of a signal power to a noise power.
In this case, the signal power, by definition, is P. The noise power, it's N_0 per Hertz.
We're going across W Hertz.
So by the way we define N_0, the signal-to-noise ratio is just P over N_0 W. That's the amount of noise power in the band.
And there's an implicit assumption here that only the noise in the band concerns you, and it's one of the key results of 6.450 or equivalent courses that out-of-band noise is irrelevant, because you know information about what's in-band, so you may as well filter it out.
And therefore we don't have to count any noise power that's outside the signal transmission band or the signal space, as we would say.
So two key things.
It's easier to keep in mind the signal-to-noise ratio than both P and N_0.
By an amplifier, you could scale both P and N_0 so really all you're interested in is the ratio anyway.
So we have these two key parameters for a additive white Gaussian noise channel.
All right.
A more typical kind of design problem is, say, a -- well, the telephone line modem, which is something on which I've spent a good part of my career.
Your boss says, OK.
I want you to build me a modem that sends as much data as possible over telephone lines.
How would you approach that problem?
OK?
This is your summer project .
What would you do on the first day?
I really would like a little feedback, folks.
Makes it so much more fun for me.
Yeah?
Can we just tell him he's [INAUDIBLE] frequency -- we have [INAUDIBLE] frequency, so it's not like this time it would [INAUDIBLE]
All right.
So you have in mind a certain channel model.
How would you verify that channel model?
Well, you put the channel frequency response, you could measure it somehow.
You could estimate it
Yeah.
You could take a sine wave generator and run it across the frequencies, and see how much gets through at the receiver.
And you could plot some kind of a channel response, which for a particular telephone channel might go something like that.
Turns out telephone channels are designed so that they have no transmission above 4 kilohertz, because you sample them 8000 times a second inside the network.
They're are also designed to have no DC response.
So this might go from -- this is -- well, I'm just drawing some generic response.
And dB, I'll give you my dB lecture later.
300 to 3800 Hertz might be something like that.
And of course, if you measure another channel, would it be exactly the same?
No, but it might be roughly the same.
I mean, all these channels are engineered to pass human voice.
Alright?
In particular, there are now international standards that say it's got to pass at least between here and here with pretty good fidelity.
All right.
So that's the first thing you do.
You develop a model of the channel.
And the key things about it, first order are, it's nearly a linear channel.
What you put in will nearly come out.
It has a certain bandwidth, which is somewhere between 2400, we used to say, way back in the '60s.
Nowadays we might say we can actually get through 3600, 3700 Hertz.
But that's what's the order of the channel bandwidth is.
So we have W equals 3700 Hertz.
And then, of course, we have some noise.
If we didn't have any noise, how many bits per second could we get through here?
Hello?
Infinite, right.
Send one number with infinite precision.
And that would communicate the entire file that we wanted to send.
And so we have a signal-to-noise ratio.
It's questionable whether the noise is totally Gaussian.
You've heard a telephone channel.
You know, it's some weird stuff.
But this is what our kind of normal telephone channel would look like.
I should say, greater than this.
Nowadays.
Used to be not nearly as good... Or I shouldn't say nowadays.
This was true about ten years ago.
Nowadays everybody is using cell phones, and these things have all gotten terrible again.
All right?
So please do not design your modem for a cell phone.
Let's plug a nice wired connection into the wall.
OK.
So that's a first gross characterization of this channel.
What would you then do on the second day of your summer project?
Yeah?
[INAUDIBLE]
OK. That's a good idea.
What do you have in mind in designing a modulations key?
[INAUDIBLE]
A signal constellation.
All right.
What signal constellation are you going to use?
PAM?
QAM?
Yeah?
What would control your choice?
ISI might.
This channel doesn't have a flat response.
We might -- let's say we've actually developed a model where Y of t is X of t going through a linear filter, convolved with some response h of t plus N of t. So what we transmit gets filtered.
We add noise to it.
And that's what we see at the output.
That's just a first order model.
This is basically the way modems were designed for telephone channels for many, many years.
Just with that simple model.
All right.
So if we have a filter here, we might have to worry about intersymbol interference.
But you're not quite back at the basic level.
Let's suppose we decide that the channel is reliable just within some particular frequency band w, and within that band, it's more or less flat.
All right?
When I talk about white Gaussian noise, I'm always going to mean that the channel, the noise spectral density, and the channel response are flat within the given bandwidth.
All right?
So I'm not going to have to worry about ISI at this level.
Of course I will in my telephone line modem.
Yeah?
So even if we want high speed, [INAUDIBLE] so we want a rate that would be greater than [INAUDIBLE]
I suppose.
I mean, do you have enough information to say?
Well, for example, if we [INAUDIBLE] 64 kilobits per second, can we travel 3,700 [INAUDIBLE] of bandwidth that we need use the bandwidth [INAUDIBLE]
OK. Is there any chance of sending 64 kilobits through this channel?
Yes
There is?
How do you know that?
If you use [INAUDIBLE PHRASE]
All right.
So that would be 32 level complement -- that would be five bits per Hertz, basically, talking in terms of spectral efficiency.
Times even 4000 kilohertz, that only gives you 20 --
Then a higher level
Then a higher level.
1024 level QAM.
But what's going to limit you?
I mean, why not make it a million level?
[INAUDIBLE]
The signal-to-noise ratio, all right?
OK. Well.
I suggest what you do on your second day is you start thinking about, on the one hand, simple modulation schemes.
And I would say 32 -bit QAM is simple in the context of this course.
It has no coding.
But just kind of give yourself a baseline of what might work.
But another very good thing to do is to establish an upper limit.
Now, it's kind of amazing that we can do this.
Up to 1948, no one would've thought that it was a reasonable to say, well, there's a certain upper limit on what we could ever transmit through this channel.
The idea was if you transmitted fast, you'd make a lot of errors.
If you slowed down, you'd make fewer errors.
And you know, there's no hard and fast limit.
But of course, what Shannon did was to show that for mathematically well-specified channels, of which this is one, there is a certain limiting data rate called the channel capacity.
And what the capacity is for an additive white Gaussian noise channel, this is the most famous formula in information theory.
If we have time, I'll derive it for you.
Otherwise I encourage you to look it up.
The capacity of this channel is entirely specified by these two parameters, bandwidth and signal-to-noise ratio.
And it's simply W times the binary logarithm of one plus SNR in bits per second.
All right?
And the Shannon proved a strong capacity theorem and a converse.
The capacity theorem says, there does exist a coding scheme -- perhaps involving QAM, perhaps involving lots of other stuff -- that can get you as close as you want in rate, for any rate less than capacity, there exists a -- let's say a coding slash modulation scheme, slash modulation, slash detection, slash decoding scheme -- such that the probability of error is less than or equal to epsilon, where epsilon can be chosen as in any arbitrarily small number.
All right?
So if you want a probability of error of ten to the minus five or ten to the minus ten, however you measure it -- can't be zero.
Very big difference between zero and ten to the minus ten.
But if you want probability of error of less than ten to the minus ten, then Shannon says there exists a scheme that can get there, as long as your rate is less than this capacity or Shannon limit.
All right.
So I suggest it would be a good idea to calculate what that is for this.
Now, signal-to-noise ratio is 37 dB.
Then one plus SNR is about the same as SNR.
Log2 of 37 dB, factor of 2 is 3 dB, so that's about 12 and a third.
This is our first example of why calculating in dB makes things very easy.
So this is about 12 and a third times whatever the bandwidth is.
3700 Hertz, and we go through that calculation, and we get something over 4,2000 bits per second.
All right?
So you have no hope of going more than 42000 bits per second, if this is, in fact, an accurate channel model.
I mean, I'm assuming that these two numbers are correct.
If this turns out to be 50 dB, then you can go a little further.
All right?
Yes?
[INAUDIBLE]
Yes.
And it's really only true for a perfectly flat channel that satisfies the complete mathematical definition of a white Gaussian noise channel within the band.
I'm shocked to find that this is -- lots of people get through MIT Communications course and never see a proof of this formula.
This is about as fundamental as you get.
That's why I include one in this course.
Or perhaps you've never even seen the formula before.
Certainly don't understand where it's valid and where it's not valid.
OK.
So on the other hand, we might try something like 32 QAM.
If we know something about that, we make it so that it fits in here.
We find that we can get maybe five bits per second per Hertz.
That's something called spectral efficiency, which in this course, we'll always designate by rho.
So we can get maybe 3000 Hertz without having to worry about filtering too much.
So we're going to have a system that sends 3000 QAM symbols per second -- 3000 two-dimensional signals per second, each one carrying five bits.
And so with that, we'll get 15000 bits per second.
So these kind of establish the alpha and omega, the baseline and the ultimate limit, of what we could do over this channel.
With a simple uncoded scheme, we would check that we can, in fact, get our desired error rate at the signal-to-noise ratio that we have here.
So we know, without sweating hard, we could get 15000 bits per second.
Shannon says we can get 42000 bits per second.
Maybe we want to put a little coding in there.
How does Shannon say you could get to this marvelous rate?
He says, well, what you need to do is to choose a very, very long code.
And if you choose it the right way and decode it in the right way, then I can prove to you that your probability of error will be very small.
Of course, Shannon's proof is not constructive at all.
If you've ever seen it, it involves just choosing a code at random, decoding it exhaustively by simply looking at all the two the nr code words, and finding which one is closest to the received signal, and that's not very practical.
So there's nothing, in that sense, practical about Shannon's theorem.
But Shannon says, the way you get there is you code.
What is coding?
Coding is introducing constraints into your transmission.
Uncoded is where you just send an independent five bits in every symbol.
And the next symbol is newly, you choose it new.
So there's no dependencies between the symbols.
It's just one shot communication, again and again and again.
Coding is introducing constraints.
From Shannon, it's clear you need to introduce constraints over very, very long sequences.
Basically what we do is we take some alphabet of all the sequences that we could possibly send -- maybe the alphabet is all 1024 QAM sequences that we could possibly send.
And then the code, by the code, we say we weed out a lot of those sequences.
So we only send a small subset of all sequences that we might have sent.
As a result, we make sure that there is a big distance between the sequences that are actually in the code.
The fact that there's the much bigger distance than there is between the signal points in any particular small constellation means that if we do truly exhaustive maximum likelihood decoding, it's extremely unlikely we'll confuse one sequence for another.
And therefore, we can get a very low error probability.
And the whole course is about the history of people's efforts to realize what Shannon said on the particular channel that we're talking about.
The additive white Gaussian noise channel, which, as you might imagine, is the canonical channel.
But it's also proved to be a very good model for things like the deep space channel, the telephone line channel.
In chapter one, I talk about the history and you can get some sense for the time flow.
And it really took until -- well, depending on how you count it, up to about 1995 to when people could get effectively within tenths of a dB of the Shannon limit and you could say the problem was cracked.
And research has continued for a little -- the past ten years, and this has been made more practical, and they've spent more channels refined in many, many directions.
But really, you could say it was a 50 year project to take this existence theorem and make it real in a practical engineering sense.
And that's the story of this course.
I think it's a terrific story.
So that's the way I like to package the course.
How are we going to get to what Shannon said we could do?
Alright, so that's the whole story.
You having a problem hearing me?
It's just the noise in the system
OK. Well, it's probably not an additive white Gaussian noise channel.
One of the things about an additive white Gaussian noise channel is that there's no dependence in it.
Time dependence in it.
So a third channel of very very practical interest these days is a wireless channel, radio channel.
Let's imagine it's a single user channel.
Everybody in this -- everything in this course is going to be single user, point-to-point.
But nonetheless, the FCC has said we can have five Megahertz.
All right?
And we've got to stay within that five Megahertz up to some little slopover that won't bother anybody.
And again, your design problem could be, well, how do you send as much data as possible through a five megahertz wireless channel?
If you take the Wireless course, and I encourage you to do that, or even if you listen to the last couple of lectures, I think, in 450, you know that there's a big difference in the wireless channel and it's time-varying.
In particular, it has fading.
So sometimes it's good and sometimes it's bad.
You have outages.
Not characteristic of the telephone line and the deep space channel.
So that introduces -- well -- a whole lot, many more considerations.
I'm not going to talk about that in this course.
For that course, it's held in the next room immediately following this one.
And I think it's an excellent course.
OK.
So that's the story of the course.
We're going to go from day one knowing nothing about how to signal through this channel, except maybe we've taken some course that's introduced to us words like PAM and QAM.
And by the end of the course, we're going to know how to go at the Shannon limit.
And we're going to know a whole lot of techniques at that point.
We're going to know how to code, we're going to know how to decode at least some representative examples, which I've chosen to be the ones that have been the most useful in practice.
And they also tend to be the most interesting in theory.
OK?
Is that clear?
OK. Just an aside on the first homework set.
As I said, these are just supposed to be warm-up exercises that should get you comfortable with operating in the additive white Gaussian noise channel, which, at least some discrete time, turns out to be trying to code in Euclidean space.
To encode and decode in Euclidean space.
And the very first thing I do here is to give you my dB lecture.
Now, how many people here would say they're comfortable using dB?
Nobody.
OK. Over the years, I've tried to encourage Bob Gallager to talk more about dB in 6.450.
And now I believe he does give the dB lecture, but I don't believe he gives it with very much conviction.
I'm convinced the dB are a very useful thing to know about, and I think Bob had a bad experience with dB early in life is really what the problem is.
Some army sergeant said somebody was three dB taller than somebody else, and he was so revolted, he never wanted to talk about dB again.
OK. dB, which stands for decibel, one tenth of a Bell -- the reason B is capitalized is it is after somebody's name, Alexander Graham Bell -- are very useful whenever you want to use logarithms.
They basically are a system of logarithms.
This gets all confused with ten log ten and 20 log ten and EE tests, but basically it's just a very well-designed, for humans, system of logarithms.
Think about it.
What would be the most convenient system of logarithms?
We're going to use this whenever we have talking about factors, or the multiplication of a bunch of numbers.
First of all, you'd want to have it be based on the base ten number system, because then it's -- you know, we have a decimal system, and we want to be able to very conveniently multiply by ten, 100, 1000, and so forth.
So a very natural logarithm to consider is log to the base ten.
But so if we have log to the base ten of some factor -- let's say log to the base ten of 2 is 0.3010.
And in fact, you know there are lots of tables that give logs to base ten.
But it's just not so easy to remember that the log of base ten of 2 is 0.3.
Basically if we take the log to the base ten of the whole range from, say, one to ten, that's really all we have to know.
Because what we need is one factor of ten.
This goes from zero to one.
OK. What we'd really like to do is to spread this out for human factors engineering.
We'd like to take ten to the base ten, which will now map the range from one to ten into a nice interval from zero to ten.
That's very easy for people to understand.
And ten to the log10 of 2 is 3.01 and so forth.
OK.
So if that's why we say a factor of 2 is equal to 3 dB.
I find it's always helpful to say "a factor of" when talking about dB, just to remind ourselves, we're always talking about a multiplicative factor.
So a factor of alpha is 10 log10 alpha dB.
Now most of you are mathematically sophisticated enough to know this is basically the same thing as saying log to the base ten to the one-tenth of alpha.
So we're really using a base here, which is ten to the one-tenth point 1, which is about 1.25 something or other.
So the number whose one dB, which is beta to the one, is about 1.25.
So we're just using a log to a certain base, which is very convenient.
The base is ten to the one-tenth.
And that's all it is, all right?
Beyond that it's not very mysterious.
So I think it's very useful just to remember a little table.
alpha of one is how much in dB?
It's zero.
Or I write it two ways.
Round numbers, or in any system, it's that.
1.25 is about 1 dB.
2 is 3 dB.
Let's see.
3 is about 4.8 dB.
4 is what?
4 is just 2 squared, so that has to be 6 dB.
5 is what?
5 is 10/2, so that has to be -- ten down here is 10 dB.
That's exactly 10 dB.
And so 5 has to be 7 dB.
And 8 is 9 dB.
And that's consistent, by the way.
You see then ten eighth, which is 5/4, is 1.25, which is 1 dB.
And just, I find it useful in everyday life.
Maybe that makes me an engineering nerd.
But the first problem, for instance, has to do with compound interest.
If you just remember dB values of things, you can do things like compound interest calculations in your head.
Or I gave you a more engineering example over here.
If you want to evaluate log2 of 37 dB, it's very easy.
You just divide by 3.
All right?
So anyway.
I encourage you to memorize this short table.
There's some more things in it.
And use it day by day, and become known as a real MIT person.
Why schemes?
Well, why not?
All right.
The other problems on the first homework set -- one's a quite algebraic construction, using Hadamard matrices of geometrical signal sets, like biorthogonal NS orthogonal and simplex signal sets, with a easy decoding algorithm.
And that's good to know about.
Next one is a QAM problem, asking you to look at various arrangements of signal points in two dimensions, and try to find the best ones for various criteria.
The last one has to do with spherical shaping of large constellations, in many dimensions, with high spectral efficiency.
Many bits per second per Hertz.
OK. And that's chapter one.
Ok. Chapter two we'll do very quickly.
And then I could see we're not going to get to chapter three at all.
And that's fine.
You can read it.
Ashish, if you want to say a word about chapter three next time, that would be fine.
Chapter two is really about given a continuous time additive white Gaussian noise channel, with model Y of t equals X of t plus N of t, and here we have a certain power and we have a certain bandwidth limitation W, the same things we were using before, here we have a certain power spectral density N_0.
So there are the parameters of the channel.
Or I've collapsed them into two parameters, SNR and W. How can we convert this to a discrete time additive white Gaussian noise channel, which will be a channel model like this -- a sequence of symbols Yk is equal to a sequence of transmitted symbols Xk plus a sequence of noise symbols Nk.
Where again, this is going to be IID Gaussian.
That's what "white" is in discrete time.
Independent Identically Distributed Gaussian random variables.
This is going to have some variance S_N, it's going to have some power constraint S_X.
We don't specifically see a bandwidth in here, but the bandwidth is essentially how many symbols we get per second.
And I think all of you should have a sense that a bandwidth of W translates, through the sampling theorem, or through PAM modulation, or something else, is roughly equivalent in discrete time to two W real symbols per second, or to W complex symbols per second.
Now the -- sorry, did I screw you up?
Do I need to readjust?
All right.
Perfect.
There is a particular way we can -- well.
So we're going to show that these two, the continuous time and the discrete time channel -- if we have a continuous time channel, we can get a discrete time channel like that, where the values of the parameters, signal-to-noise ratio, for instance, translate in a natural way and that they're the same.
And furthermore, where we carry as much information on the discrete time channel as we do on a continuous time channel.
So there's no loss of optimality or generality in doing this.
Now you've all, with a couple of exceptions, taken 6.450, so you know how to do this.
One way of doing this in a fairly engineering sense is orthonormal PAM -- Pulse Amplitude Modulation.
And you should know how this goes.
We're going to take in a sequence X, which I might write out as a sequence of real symbol levels Xk, perhaps chosen from a PAM alphabet.
I'm not even going to specify.
And what do I need to know here?
This is going to be at two W symbols per second, real symbols per second.
OK. That's my transmitted sequence that I want to get to the other end.
PAM modulator -- so this is PAM modulator -- is specified by a certain pulse response p of t. and I'm going to require that this be orthonormal in the following sense.
That the inner product between p of T minus kT and p of T minus jT is equal to the chronic or delta.
Delta kj.
So that the time shifts of this single basic pulse by T, where T equals one over two w, the symbol interval, are going to be orthogonal to each other, and what's more, orthonormal.
So in effect, I've got a signal space consisting of all the linear combinations of the time shifts of p of T by integer values of T, and I'm going to send something that's in that signal space simply by amplitude modulating each one that comes along.
All right.
So what I get out here is the continuous time signal X of T, sum of Xk p of T minus kT.
That's what the modulator does.
Then the channel, continuous time channel -- so this is discrete time here.
Now I'm into continuous time.
The channel is going to add white Gaussian noise N of T with N_0 power spectral density to give me channel output Y of T is X of T plus N of T. And now I need a receiver to get back to discrete time.
The receiver will simply be a sampled matched filter, which has many properties which you should recall.
Physically what does it look like?
We pass Y of T through p of minus T. The matched filter's turnaround in time.
What it's doing is performing an inner product.
We then sample at T samples per second.
Perfectly phased.
And as a result, we get out some sequence y equal Yk.
And the purpose of this is so that Yk is the inner product of Y of t with p of T minus kT.
And you should be aware that this is a realization -- this is a correlator-type inner product.
Correlate and sample inner product.
All right?
So what are some of the properties that you developed ad nauseum in 6.450?
First of all, if we take the signal part of this, and we have the sampler phasing right, what do we get out for the signal part of Yk?
We have X of T is this.
If we correlate that against, let me say, p of T minus jT for all j, we're going to get zero for everything except for the desired sample.
For the desired sample, we're just going to get the desired sample out.
So from this, we get Yk is equal to Xk plus the noise term, Nk.
In other words, there's no intersymbol interference.
The noise term -- by taking these inner products, what are these?
These are the coefficients of yK in the signal space, under -- these are an orthonormal expansion of Y of T. Those components which lie in the signal space.
Can we ignore the components that don't lie in the signal space?
Yes.
If it's out of white Gaussian noise by the theorem of irrelevance or whatever it was called.
There is no information about the Xs contained in any part of Y or T that is orthogonal to the signal space, in the orthogonal space.
So we simply take the parts that are in the signal space.
And another property is that this is white Gaussian noise.
Then Nk is simply an IID jointly Gaussian sequence with variance S_N equal to N_0 over 2 if the power of spectral density here was N_0.
And one other property of I should have mentioned here is that again by the orthonormal property, as you would think intuitively, if we sent two W symbols per second, and each one is limited to power S_X, then this will have power 2 W S_X.
The powers are the same in discrete time and continuous time.
So what do I get?
Let me now show the SNR.
Let me compute it in discrete time and in continuous time.
The SNR in discrete time is 2 W S_X over S_N, which was N_0 over 2.
I'm not seeing the two's fall in the right place.
What am I doing wrong?
So this is the power.
Discrete time and signal-to-noise ratio is just S_X over S_N.
Now S_X is P over 2W.
S_N is N_0 over 2.
Now I'm happy.
It's P over W N_0.
Continuous time is p over W N_0.
OK.
The point is, the signal-to-noise ratio is the same.
In discrete time, talking about bandwidth, the bandwidth is really 2W symbols per second.
So that's our measure of bandwidth in discrete time.
And that is approximately equal to W Hertz in continuous time.
Now, I haven't proved that to you yet, have I?
How to prove that we can get a bandwidth as small as W but no smaller.
Again, you've done this in 6.450, so I hope I can just remind you.
If the shifts of p of T are orthonormal, that says something about p of T star p of T, which is the correlation of p of T with itself.
It means g of T has to be 1 to equal 0, and it has to be 0 at all other times, at all other integer times.
Which means the Fourier transform, power spectral density of p of T, it has to satisfy the Aliasing Theorem, or the zero ISI, the Nyquist criterion for zero ISI, which basically means the alias version of the power spectral density of this has to add up to a perfect brick wall response between 1 over 2T minus 1 over 2T and 1 over 2T in the frequency domain.
And there are simple ways to make it do that.
Which is simply to have a roll-off, a real frequency response, with a roll-off that's really sharp as you like above 1 over 2T.
But so we can say the nominal Nyquist bandwidth equals 1 over 2T, or T is 1 over 2W.
So it's W. So the moral of this is that we can design orthonormal pulses.
If we have a similar interval of T equals 1 over 2W, we can design an orthonormal signal set that satisfies this condition for any bandwidth that's not much greater than W. All right?
And it's obvious from the Aliasing theorem that we can't do it if we try to choose some bandwidth that's less than W. So that's what justifies our saying, if we are 2M symbols per second, we're going to have to use at least W Hertz of bandwidth.
But we don't have to use very much more than W Hertz of bandwidth if we're using orthonormal PAM as our signaling scheme.
So we call this the nominal bandwidth.
In real life, there will be a little roll-off -- five percent, ten percent.
And that's a fudge factor in going from discrete time to continuous time.
But it's fair that we can get as close to W as we like.
Certainly in the approaching Shannon limit theoretically, we would say that we can get as close to W as we need to, I should specify.
All right?
So there's one other parameter that Ashish is going to be talking a lot more about in the next lectures which is called spectral efficiency.
So far I haven't told you what information the sequence of symbols is carrying.
But let's suppose it's carrying R bits per second.
All right?
We have a modem.
It's a 64000 bits per second modem or whatever.
R is 64000.
Or whatever it is.
My objective is to make R as high as possible subject to the Shannon limit.
If I am sending R bits per second across a channel which is W Hertz Y, in continuous time, I'm simply going to define, I'm always going to write this as rho.
And I'm going to write it simply as a rate divided by the bandwidth.
So my telephone line case, for instance.
If I was sending 40000 bits per second in 3700 Hertz bandwidth, I'd be sending 12 bits per second per Hertz.
That's why we say that.
That's clearly a key thing.
How much data can I jam in?
We expect it to go linearly with the bandwidth.
Rho is the measure of how much data per unit of bandwidth.
What does that translate into in discrete time?
Well, I'm dividing up this rate into 2W symbols per second.
How many bits am I sending per symbol?
I'm sending R over 2W per symbol.
So simply rho over two bits per symbol, or since I'm always going to think of Euclidean space, I will often write that as dimension-- rho over 2 bits per dimension.
All right.
Well that's not the exact translation I like.
I'd like to talk in terms of rho.
So here, rho can be evaluated at the number of bits I'm sending per two symbols, or per two dimensions.
All right.
So for the discrete time system, our major spectral efficiency is just going to be the number of bits I'm sending every two dimensions.
It turns out if you get into this game, you feel that two dimensions are more fundamental than one dimension.
Which may be a manifestation of the fact that complex is more fundamental than real.
Really we want to talk about the amount of information we're sending per a single complex dimension, or we can say two real dimensions.
Geometrically they amount to the same thing.
Again, in the notes, there are discussions about how you go back and forth, and so forth.
So the bottom line here is that we can talk about spectral efficiency, which is obviously a continuous time concept.
But we can talk about it in the discrete time domain.
And what it's going to mean simply is the rate that we're managing to send, by particular scheme, if it's per two dimensions.
For instance, 32 QAM has a spectral efficiency of-- I didn't make myself clear at all
Five bits per two
Five bits per two dimensions, all right?
So that's the basic spectral efficiency of a 32 PAM modulation scheme.
2 PAM, plus or minus one.
What's the spectral efficiency of that if it's for two dimensions?
It's a trick question.
One bit per one dimension.
It's two bits per two dimensions.
If it's 2 PAM, it's actually the same thing as 4 QAM, because 4 QAM is just doing 2 PAM place in two successive symbols.
So 2 PAM, you could send two bits per two dimensions.
4 QAM is also two bits per two dimensions Ok.
Similarly for any [UNINTELLIGIBLE], we'll be talking about codes which cover many dimensions.
If you send R bits in N dimensions, then the spectral efficiency is going to be 2R over N. So R over N in one dimension, 2R over N in two dimensions.
I guess initially, this catches people up.
Because it's more difficult to normalize for two dimensions than for one dimension.
But come on, suck it up.
You're a graduate student at MIT.
You can do it for two dimensions.
And I assert that it really is the more natural thing.
In particular, makes this nice correspondence between discrete and noncontinuous time parameters.
So the SNR is the same.
The bandwidth has different interpretations, but we're talking with W in both cases.
Spectral efficiency, we can measure it as either discrete time or continuous time.
And oh, by the way.
What is the channel capacity formula in terms of spectral efficiency?
The channel capacity was basically the data rate has to be less than W log2 of one plus SNR.
Find an equivalent way.
So this is writing things in terms of bits per second, or equivalently rho, which is R over W, has to be less than log2 of one plus SNR.
This is in bits per second per Hertz.
Or in discrete time, we have the same formula, except this becomes 50 bits per two dimensions.
And so things have been normalized so that now you only have a single parameter to worry about.
What special efficiencies can you achieve?
It is purely a matter of what SNR you have.
Telephone line channel 37 dB, then you get spectral efficiency of 12 and one-third by Shannon.
OK.
So I'm sorry not to be able to give you the very lovely derivation of this formula as a little bit of culture.
I encourage you to read chapter three.
You will not be held responsible for any of these first three chapters.
They're all more or less just for orientation.
Next time, Ashish will get into the real stuff.
Any final questions?
OK. See you in two weeks.
There is one handout being passed out, it's chapters four and five, so be sure to pick it up.
And just a reminder that there is a homework due on next Wednesday, so we have homeworks due every week in this course as well.
So last class, we covered Chapters One through Three quite quickly rather, because it was mostly a review of 6.450, and one of the key ideas we covered last class was the connection between continuous time and discrete time systems.
So we have continuous time, discrete time.
A specific example that we saw was that of an orthonormal PAM system.
The architecture for a PAM system is as follows: you have Xk, a sequence of symbols coming in, they get into a PAM modulator.
What you get out is a wave form, X of t, is what the PAM modulator produces.
You have a little noise on the channel, and what you receive out is Y of t. So Y of t is the wave form that the receiver receives, and what a canonical receiver's structure to a matched filter followed by sampling.
So what you get out is a sequence of symbols Y of t. So this is a structure for an orthonormal PAM modulator, and the continuous time version of the channel is that you have y of p, which is received at the receiver, plus X of t, plus N of t. The discrete time model is you have the symbol Y of k as the output of the sampler.
It equals X of k plus N of k. Now basically, we say that the two systems are equivalent in the following sense: if you want to make a detection of X of k here, at the receiver, Y of k is a sufficient statistic, given that Y of t is received at the front end of the receiver.
So that's the equivalence between discrete time and continuous time.
And the way we established this fact was by using the theorem of irrelevance.
The noise here is white Gaussian noise, so if we project it onto orthonormal wave forms, the corresponding noise samples would be IID.
The noise will be independent of everything which is out of band, and so there is no correlation among the noise samples, and Y of k is a sufficient statistic to detect X of k. So that is the basic idea.
Now, a similar architecture also holds when your continuous time system operates in fast band rather than base band, except the main difference now is instead of a PAM modulation, you have a QAM modulator, and your symbols, X of k and N of k, will be complex numbers rather than the real numbers.
Now the connection between continuous time and discrete time systems can be made more precise by relating some parameters in continuous time with those in discrete time.
OK?
So we have continuous time parameters, and discrete time parameters.
So a continuous time parameter is a bandwidth, W. A discrete time parameter is given by symbol interval T, and the relation is T equals 1 over 2W.
This is for a PAM system.
OK?
So something is shown in 6.450, and it's shown by using Nyquist's ISI criteria.
Do not want to have ISI in the system, the maximum symbol rate, or rather the minimum symbol rate, would be 1 over 2W.
You cannot send symbols faster than this rate.
A second criteria is power, which is P here, in continuous time.
In discrete time the equivalent parameter is energy per two dimensions.
It's denoted by Es, and Es is related to P by using Es equals 2 times PT.
Note that this is for two dimensions, meaning in PAM we have one symbol per dimension, so this is for two symbols.
This is energy for two symbols for a PAM modulator, and the relation is Es equals 2 times PT.
T is the time for the one symbol.
We also -- we are looking at energy for two symbols, so we multiply P by 2T.
OK?
Noise in continuous time is AWGN, for Additive White Gaussian Noise process, and the power spectral density is flat on positive support of bandwidth, and the height is N_0.
OK, so this is how the spectral density is.
In 6.450, we looked at double-sided power spectral density when the height was N_0 over two, but it was going over both the positive and negative part of the bandwidth.
Here, we are only looking at the positive part of the bandwidth, and we are going to scale noise by N_0.
This is just a convention.
In discrete time, your noise sequence is Nk, and is an IID and you take them as Gaussians with zero mean, and variance of N_0 over 2.
So we have noise which has a variance of N_0 over 2 per dimension.
And this was, again, shown by this theorem that when you project noise onto each of the orthonormal wave forms, you get the variance is N_0 over 2.
OK?
Instead of a base band system, if you had a fast band system, then instead of having a PAM modulator, we would have a QAM modulator.
So if instead of PAM we had a QAM then the main difference now would be that these symbols are going to be complex numbers instead of real.
So what's the symbol interval now going to be?
1 over
It's going to be 1 over W. Instead of sending a real symbol, we are sending one complex symbol, which is occupying two dimensions, and our symbol rate is going to be 1 over W. Well, the energy for two dimensions, is still given by Es equals 2PT, or equivalently, P over W. OK?
Or that's -- The noise samples Nk are still IID, but now they are complex Gaussians, with zero mean, and variance N_0.
Or equivalently, they have a variance of N_0 over 2 per dimension.
So what we see here is that we can have analogous definitions for discrete time and continuous time, and one of the key parameters that comes up over and over again in the analysis is this notion of signal to noise ratio.
The signal to noise ratio is defined as the energy per two dimensions, over the noise variance per two dimensions.
So that's the definition of signal to noise ratio.
Es, well, it's 2PT, or equivalently, P over W. The noise variance for two dimensions is N_0, so the definition is the same as -- SNR equals P over N_0 W. OK, are there any questions?
The other notion we talked about last time is this idea of the spectral efficiency.
In continuous time, the definition is quite natural.
It's denoted by symbol rho.
The units are bits per second per Hertz, and it's basically R over W. You have R bits per second over W hertz.
So it's the amount of information bits that I'm able to send over the amount of bandwidth that I have in my system.
In discrete time, we can also define the same idea of spectral efficiency, but it's usually -- and a good way to think about spectral efficiency from a point of view of a design of an encoder.
What the encoder does, is you have an encoder here, it takes a sequence of bits in-- so say you have b bits coming in-- and it produces N symbols.
So this could be X1, X2, Xn, and you have a sequence of B bits -- b1, b2, b subcapital B.
This is how an encoder operates.
It maps a sequence of bits to a sequence of symbols.
Now where does the encoder fit into this architecture?
It fits right here at the front, right?
You have bits coming in, you encode them, you produce a sequence of symbols, and you send them over the channel.
So if I have this encoder, what's my the spectral efficiency going to be?
Well, you have to ask what the encoder does, right?
So from here, we have a PAM modulator.
So it's basically this from here on, we are back to the system.
So what's the spectral efficiency now for this system?
How many bits do you have per symbol?
[UNINTELLIGIBLE]
You have B over N bits per symbol.
Now how many symbols do you have per dimension, if this is an orthonormal PAM system?
One symbol per dimension
You have one symbol per dimension, right?
So you have B bits per N dimensions, in other words
[UNINTELLIGIBLE]
So in QAM how many -- you have usually how many symbols per dimension?
[UNINTELLIGIBLE]
Half symbols per dimension, right.
So in that case, the spectral efficiency -- the units are bits per two dimensions, is 2B over N. B over N bits per dimension, because the units are bits per two dimensions, you get 2B over N bits per two dimensions.
OK?
Now a natural question to ask is, how is this definition related to this definition here?
This is quite the natural definition, and here we have imposed this encoder structure.
Are the two definitions equivalent in any way?
And in order to understand this, let us take a one-second snapshot.
So now in one second, I can send N equals 2W symbols.
Because this is an orthonormal PAM system, I can send 2W symbols per second, but in one second, I can send N equals 2W symbols.
Because my rate is R bits per second, B equals R, in one second I can send R bits.
So now my definition of rho, which I defined to be 2B over N, is same as 2R over 2W, which is R over W. So the definitions in continuous time and discrete time are equivalent.
OK?
Now, why is spectral efficiency very important?
Well, there is a very famous theorem by Shannon which gives us a nice upper bound on the spectral efficiency.
Perhaps the most important theorem in communications is Shannon's Theorem, it says that if you have an AWGN system with a certain SNR, then you can immediately bound the spectral efficiency by log2 of 1 plus SNR bits per two dimensions.
This is a very powerful statement.
Equivalently, the capacity of an AWGN channel is log2 1 plus SNR bits per second.
So one important observation here is if I have a communication system, and if what I care about is the spectral efficiency of the capacity, there are only two terms that are important.
One is the signal to noise ratio, which is P over N_0 W, which is defined here.
So the individual units of P and N_0 doesn't matter, it's only the ratio of P over N_0 that matters.
And the second parameter is the bandwidth W that we have in the system.
So signal to noise ratio and bandwidth are in some sense fundamental to the system.
An operational meaning of this theorem is that if I look at this encoder, then it gives me an upper bound of how many bits I can put for each symbol.
The number of bits that I can put on each symbol is upper bounded by this term here, log2 1 plus SNR.
I cannot put arbitrarily many bits per each symbol here.
Now in order to make such a statement, there has to be some criteria that we need to satisfy.
And what is the criteria for Shannon's Theorem?
In order to make such a statement, I need to say that some objective function has to be satisfied.
Because in some sense, I could just put any number of bits I could have on upper symbol, right?
The encoder could just put 100 or 200 bits.
What's going to limit me?
Probability of error?
The probability of error.
So this assumes that -- OK?
Now, you're not responsible for the proof of this theorem, it is in Chapter Three of the notes.
Basically, it's just a random coding argument, which is quite standard in information theory.
So if you already taken information theory or otherwise, you would probably have seen that argument involves bounding atypical events that happen.
So the probability of error is an atypical event, and we use asymptotic equipartition property to bound the error event.
There's a standard proof in Cover and Thomas, for example.
Now the main difference in Professor Forney's approach is that he uses this Theory of Large Deviations.
Theory of Large Deviations basically gives you a bound on the occurrence of rare events, and it is well-known in statistical mechanics.
So it's kind of a different approach to the same problem.
The basic idea is same as you would find in a standard proof, but it uses -- it comes from this idea of large deviations theory.
So for those of you who are taking information theory or already have seen it, I urge you, go at some point, take a look at this proof.
It's quite cool.
I already saw somebody reading this last Friday, and it was quite impressive.
So I urge more of you to do that.
So now that we have the spectral efficiency, a natural thing is to plot how the spectral efficiency looks like.
So what I'm going to plot is as a function of SNR the spectral efficiency.
Typically, when you plot SNR on the x-axis, you almost always plot it on a dB scale.
But I'm going to make one exception this time, and I'm going to plot this on a linear scale.
So this point is zero, or minus infinity dB here, and I'm going to plot -- well I should call this, actually, rho Shannon, so I don't confuse the notation.
So we'll define rho Shannon as log2 1 plus SNR.
So this is rho Shannon here, and we want to plot rho Shannon as a function of SNR.
Now if my SNR is really small, log 1 plus SNR is approximately linear, so I get a linear increase here.
If my SNR is large, then the logarithmic behavior kicks into this expression here, so now the spectral efficiency grows slower and slower with SNR.
So this is a basic shape for my spectral efficiency.
And this immediately suggests that there are two different operating regimes we have.
One regime where the spectral efficiency increases linearly with SNR, another regime where the spectral efficiency increases logarithmically with SNR.
So if SNR is very small, then we call this regime as power-limited regime.
And if SNR is large, we call this the bandwidth-limited regime.
These are our definitions.
And let's see what motivates their names.
Suppose I have a 3 dB increase in my SNR, and I am in power limited regime.
How does rho Shannon increase?
[UNINTELLIGIBLE]
A factor of 2, right?
Basically, if I have a 3 dB increase in SNR, my SNR increases by a factor of 2.
In this regime, rho Shannon increases linearly with SNR, so I have a factor of 2 increase in my spectral efficiency.
What about this regime here?
If SNR increases by 3 dB, how does rho Shannon increase?
It increases by one bit per two dimensions.
Units of rho are bits per two dimensions.
I have a logarithmic behavior kicking in, so rho is approximately log SNR.
If I increase SNR by a factor of 2, I get an additional 1 bit per two dimension scale.
OK?
If I increase my bandwidth in the power-limited regime by a factor of 2.
So this bandwidth here increases by a factor of 2, how does my capacity change, if I'm in the power-limited regime?
There's no change, right?
Or is there a change?
I'm in the power-limited regime here, and I increase my bandwidth by a factor of 2
[UNINTELLIGIBLE]
Right.
What is my SNR?
It's P over W N_0.
So what happens if I fix P over N_0?
OK, yeah, so you had it.
Basically, if I double my bandwidth, my SNR decreases by a factor of 2, so this term here is like SNR over 2, W increases by a factor of 2, and there is no change in bandwidth.
So let's do it more slowly.
So we have, say, in the -- and say, my C is now W log2 1 plus SNR.
Instead of SNR, I will write P over N_0 W. So this approximately is W times P over N_0 W times log E to the base 2.
And this is approximately -- this is basically P over N_0 times log E to the base 2.
So in other words, in the power-limited regime, changing bandwidth has no effect on the capacity.
What happens in the bandwidth-limited regime?
Well, in this case C equals W log2.
Well, I can say approximately, and I can ignore this factor of 1, because P over N_0 W is large, because my SNR is large in the bandwidth-limited regime.
I'm operating in this part of the graph.
So this is P over N_0 W. Now suppose I increase my bandwidth by a factor of 2.
C prime will be 2W log2 over P over 2 W N_0.
All right?
This, I can write it as 2W log2 P over N_0 W minus -- this is to the base 2, so I can write a 1 here.
And this term is approximately same -- if I ignore the 1, if I assume P over N_0 W is quite large, then I can just ignore the subtraction of 1.
So I get 2W log2 P over N_0 W, or it's equivalently 2C.
OK, so in other words, in the bandwidth-limited regime, if I increase my W by a factor of 2, the capacity approximately increases by a factor of 2.
So that's what motivates the name bandwidth-limited regime here.
Are there any questions?
OK. All right.
So it turns out that there are two points I wanted to say.
First of all, one might ask, that fine, these seem to be interesting definitions, that SNR is much smaller than one, and SNR is much larger than one, but is there any kind of hard division between the bandwidth-limited regime and power-limited regime?
I mean, the general answer is no.
It's basically -- because there is some point when the capacity appears not strictly logarithmically, or strictly linearly in terms of SNR.
But from an engineering point of view, we take rho equals two bits per two dimension as a dividing point between the two regimes.
And one motivation to see why rho equals two bits per two dimension is a good choice is because this is the maximum spectral efficiency we can get from binary modulation.
OK, if you want to have spectral efficiency more than two bits per two dimensions, you have to go to multi-level modulation, and that's one of the reasons why this choice is often used in practice.
Another point is that the bandwidth-limited regime and power-limited regime, they behave quite differently in almost all criterias that you can think about.
So usually, with the analysis -- we keep them separately and do the analysis differently in the bandwidth and power-limited regime, and that's what we will be doing in the subsequent part of this lecture and next lecture.
So we start with the power-limited regime.
Now, we already saw that in doubling the power doubles rho S, and doubling bandwidth does not change C. OK, the other point I mentioned was binary modulation is sufficient in this regime, because our spectral efficiency is less than two bits per two dimensions, so the idea is to have a strong code followed by binary modulation.
What else?
Right.
Typically, the normalization is done in per information bit.
What does this mean?
When we are wanting to compare different systems, we will look at all the parameters normalized per information bit.
For example, if we want to look at the probability of error, we look at this quantity here, Pb of E, which is the probability of error per information bit, as a function of Eb/N_0.
This is an important trade-off that we wish to study in power-limited regime.
Eb is the energy per bit, P sub b of E is the probability of error per information bit.
So I want to spend some time on this Eb/N_0, because it's an important concept that we'll be using often in the course.
So what is Eb/N_0?
It's sometimes also known as Eb over N_0, so it depends how you wish to call it.
What is energy per bit in terms of Es?
[UNINTELLIGIBLE]
Well, Es over rho, right?
Energy per bit.
Well, Es is energy per two dimensions.
Rho is bits per two dimensions, so Eb is Es over rho, and you have N_0 here.
And Es over N_0 is also our SNR, so this is SNR over rho.
OK, so that's how Eb/N_0 is defined.
Now we know from Shannon that rho is always less than log2 1 plus SNR, or equivalently, 2 to the rho minus 1 is less than SNR.
If I sub in here, this means that SNR is greater than 2 to the rho minus 1 over rho, and then we're just dividing by rho.
So Eb/N_0 is always greater than 2 to the rho minus 1 over rho.
And this is quite an interesting observation.
For example, if you are analyzing the feasibility of a communication system, which has a certain spectral efficiency, and a certain Eb/N_0, then you can immediately check this relation, to see if it is a system in the first place.
What Shannon says is that Eb/N_0 is always going to be greater than 2 to the rho minus 1 over rho.
So if you see that this relation is not satisfied, immediately know something is wrong.
This actually reminds me of an interesting anecdote that Professor Forney once mentioned when I was taking the class.
Well he has been in this field since the '60's, and so he has seen a lot of this stuff.
He was saying when turbo codes -- which is one of the first capacity approaching codes in recent years when they were proposed -- they presented the results at ICC, the International Conference in Communications, and what they saw was that the performance was very close to the limit that we can predict.
Eb/N_0 was very close to the ultimate limit -- the Eb/N_0 achieved by turbo codes was very close to the ultimate limit that one can predict, at least 3 dB better than the best codes that were available there.
So most people just thought that there was something wrong in the simulations, and they told them that there was a factor of 2 missing somewhere, so they should just double check.
But it turned out when they went back and people actually implemented these codes, they were actually very close to the capacity.
So sometimes, you have to be careful.
If you are not going below this limit, it could be that the system is good.
And when it is, it's a really important breakthrough.
Ok So one particular concept that comes here is the idea of ultimate Shannon limit.
And basically, if we are talking about power limited regime, our SNR is very small.
So our spectral efficiency is going to be quite small.
Now notice that this function here it monotonically increasing in rho.
It's easy to show that this function here increases monotonically in rho.
You could just differentiate this, or there's easier ways to do a Taylor expansion of 2 to the rho minus 1, the series expansion and show each term is positive.
So if this term is always going to be greater than this term here.
So I'm going from here to here.
Limit rho tends to zero of 2 to the rho minus 1 over rho.
This is going to be -- it's a simple calculus exercise to show this is the natural log of 2, or in dB, it's minus 1.59 dB.
So no matter what system you design, your Eb/N_0 is always going to be greater than minus 1.59 dB.
That's basically what this calculation shows.
And when is it achieved?
Well, it's only achieved when the spectral efficiency goes to zero.
So if you have a deep space communication system, where you have lots and lots of bandwidth, and you do not care about spectral efficiency, then if your only criteria is to minimize Eb/N_0, then you can design your system accordingly, check how much Eb/N_0 you require for a certain probability of error, and see how far you are from the ultimate Shannon limit.
So in this way, in the power-limited regime, you can quantify your gap to capacity, if you will, through this ultimate Shannon limit.
OK, are there any questions?
OK.
So now let's look at the bandwidth-limited regime.
So we already saw two things in the bandwidth-limited regime.
If I double P, my spectral efficiency increases by one bit per two dimension.
Similarly, if I double bandwidth by C, capacity approximately doubles.
Now, because you want a spectral efficiency of more than two bits per two dimension, in this system you typically do a multi-level modulation.
So for those of you who are familiar, we do things like trellis-coded modulation, and bit-interleaved coded modulation, and so on.
If we have time, we'll be seeing those things towards the very end of the course.
This is not a subject of the course as such.
The normalization in this regime is done for two dimensions.
So if you want to normalize all the quantities, we normalize them for two dimensions here.
And particularly, we will be seeing probability of error as a function of this quantity called SNR norm.
So this is the performance analysis that is done in bandwidth-limited regimes.
Here Ps of E is the probability of error for two dimensions.
What is SNR norm?
It's defined to be SNR over 2 to the rho minus 1.
Why do we divide by 2 to the rho minus 1?
Well, that's the minimum SNR that you require for the best possible system.
That's what Shannon says, right?
So this quantity here is always going to be greater than 1, or 0 dB.
So OK, well, this is the ultimate Shannon limit in the bandwidth-limited regime.
If you have a system that is designed, that operates at a certain SNR and a certain spectral efficiency, you can calculate SNR norm and see how far you are from 0 dB.
If you're very close to 0 dB, then that's great.
You have a very good system in practice.
If not, then you have room for improvement.
And so, in other words, SNR norm defines the gap to capacity.
OK, so let's do an example.
Suppose we have an M-PAM system.
So we have an M-PAM constellation.
So how does it look like?
Well, you have points here on a linear line.
This point is say minus alpha, alpha, three alpha minus three alpha, all the way up to m minus one alpha, and here, minus of m minus one alpha.
Assume m is an even number.
So the distance between two points here is two alpha, any two points.
Now we want to find the SNR norm given that we are using this constellation.
So in other words, if I use this constellation in my communication system, how far am I operating from the ultimate Shannon limit?
OK, that's the question.
So in order to find what we need to find, is first energy for two dimensions.
Does anybody remember the formula for M-PAM?
Well, there was a very nice way of doing it in 450.
One natural way is to simply sum up all of the coordinates here, square them, and divide by M. And because it's for two dimensions, we could do it -- we're to multiply by a factor of two, because it's for two dimensions-- and summation of Xk squared.
And if you work out, you will get some answer here.
Another way that was shown in 450 --
[UNINTELLIGIBLE]
With uniform quantization, exactly.
So the idea here is, you have a source, say, which is uniformly distributed between M alpha and minus M alpha.
So you have a source uniformly distributed, and it can be easily seen by inspection that M-PAM is the best quantizer for this particular source.
Ok.
So the interval regions are just of equal width.
Each interval region is width two alpha, so your mean square error, if you will, is 2 alpha squared over 12, so it's alpha squared over 3.
Well, the variance in your source, which I will denote by sigma square s, is 2M alpha squared over 12, or it's M squared alpha squared over 3.
So your energy per symbol -- it's energy in your quantizer, so I'm denoting it by E of A in order to differentiate it by Es, because Es is two times E of A, OK?
It's going to be m squared minus 1 times alpha squared over 3.
So Es is 2 E of A, and so we get it's 2 times alpha squared, m squared minus 1 over 3.
OK, so can anybody tell me what the spectral efficiency will be for the system, if I use an M-PAM constellation?
Well, how many bits per symbol do I have?
Log2
Log2 M bits per symbol.
So since it's bits per two dimensions, the sum would be 2 logM to the base 2.
Ok?
So right now, we have pretty much everything we need to find SNR norm.
The SNR here is Es over N_0, so it's 2 alpha squared, M squared minus 1, over 3N_0.
But remember, N_0 by definition is 2 sigma squared, because sigma squared is N_0 over 2.
That's how -- that's the noise variance per dimension.
So, I had 3 times 2 times sigma squared, or I will just write this as 6 times sigma squared, and that's going to be -- this is just multiplication, so I should not even write it -- six sigma squared.
So this is alpha squared, M squared minus 1 over 3 sigma squared.
OK, so now SNR norm is SNR over 2 to the rho minus 1.
2 to the rho minus 1 is just M squared minus 1.
It cancels with this M squared minus 1, and so I get alpha squared over 3 sigma squared.
So that's the SNR norm if I use an M-PAM system
Why is [INAUDIBLE]
Well, N_0 -- I've plugged in for SNR 3 here, so I have 3 N_0, but N_0 is 2 sigma squared.
Did I miss anything?
Ok?
Any questions?
OK, so there are two important remarks from this example.
The first remark is that SNR norm is independent of M. So I started with an M-PAM constellation, so it's a different constellation for each value of M, right?
If I look at my spectral efficiency rho, it's different for each value of M, because I can pack more and more bits per symbol as I increase M. If I look at my signal to noise ratio, it's also a function of M. But when I took SNR norm, remarkably, the M squared minus 1 term cancelled out in the numerator and denominator, and what I was left with was something independent of M. So this is actually quite an interesting result.
What it says is suppose I design a system, an M-PAM system, that has this particular spectral efficiency, then my gap to capacity is given by this expression.
If I use a different value of M, my gap to the ultimate Shannon limit is still given by this expression here.
So by increasing value of M or decreasing the value of M, my gap to the Shannon limit is still going to be the same.
For each value of M, I will have a different spectral efficiency, but I'm not getting any kind of coding gain, if you will, here.
OK, so this motivates that M-PAM is an uncoded system.
All of them have the same gap to the Shannon limit, regardless of what the value of M is, and so.
The second point to note is if I look at the value of alpha squared over 3 sigma square, I can make it quite small, right?
I can even, if I decrease alpha really by a great amount, I can make this quantity smaller than 1.
OK?
I told you here that SNR norm is always greater than 1.
So what happened?
Did I lie to you here, or did I do something wrong here?
I mean, I can choose any alpha, right, and make this quantity as small as I please, and then I'm doing better than the Shannon limit
[UNINTELLIGIBLE]
Right.
So basically, what's missing in this calculation is probability of error.
If I make alpha really small, what I have is all these points come closer and closer, and sure, I seem like I'm doing very well at the encoder.
But what happens at the decoder?
There is noise in the system, and so I get too many errors.
This lower bound clearly assumes that you can make your probability of error quite small, arbitrarily small, right?
So in any reasonable system, I should also look at the probability of error.
If I make alpha really small, I'm going to have too much error at the decoder, and so I won't be able to have a practical system.
So the comment is SNR norm cannot be seen in isolation.
We need to couple SNR norm with the corresponding probability of error.
Yes?
[UNINTELLIGIBLE]
What I mean by uncoded syst -- that's a good question.
What do I mean by uncoded system?
All I'm really saying here is M-PAM system is a fairly simple system to implement, right?
Irregardless of what value of M I get, I have a certain gap to the Shannon limit, which is independent of M. So if I start with a simple system, where I have bits coming in, each bit gets mapped to one symbol, and I send it over the channel, I have a fixed gap to the Shannon capacity.
And this, I will call an uncoded system.
My only hope will be to improve upon this system.
OK?
Any other questions?
So, if I'm multiplying [INAUDIBLE]
Right, right.
I'm assuming the fixed alpha
[UNINTELLIGIBLE]
I mean, basically, you will see that alpha is a function of Es, right?
And I want to -- so why do I want -- the question is, why do I want to keep alpha fixed, right?
Yeah
OK, so in order to understand that, I have to look at energy for two dimensions.
If I normalize by M squared minus one -- that doesn't work out
[UNINTELLIGIBLE] so the constellation cannot be expanding [UNINTELLIGIBLE]
Right, so I wanted to always plot by keeping alpha fixed.
And I mean, the point is, typically you want to plot the probability of symbol error -- do I have that expression anywhere?
Right there.
Ps of p is a function of SNR norm, right?
And that will be -- if I increase SNR norm, so it will be some kind of a graph, right?
A trade-off.
Basically, alpha will define the trade-off between SNR norm and Ps of p. So as I slip over larger and larger values of alpha, for different values of M, then I will have a trade-off as Ps of p is a function of SNR norm.
Does that make sense?
So say I fixed a value of M, OK?
I define SNR norm, which is alpha squared over 3 sigma squared.
For this, I will get a certain probability of error now.
So if I fix this value of SNR norm, I get a certain probability of error.
What if I want to increase my SNR norm?
The only way to do that will be to increase alpha.
Does that make sense?
Yeah
[UNINTELLIGIBLE] You're defining SNR norm as the gap to capacity, but it seems like, I mean, obviously as you increase alpha, as you increase your signal energy, you're going to do better and better, right?
Right
Well, in terms of what?
In terms of probability of error, like achievable probability of error, or?
Right, exactly.
So basically, the point is say I plot Ps of E as a function of SNR norm, or as a function of alpha squared over 3 sigma squared.
So my core will look like this
But the gap to capacity is defined all the way over here on the left, is that what--?
Right, that's a very good question.
You are going much ahead then what I thought.
So basically, at zero, seven, SNR norm is zero, right?
This is in linear scale, so when I have one, SNR norm is one.
I have the Shannon system.
Basically, for any SNR norm greater than 1, what Shannon says is your probability of error will be arbitrarily small, and here, it will be large.
So this here is the Shannon limit.
And now, in a practical system, what I want to do is I want to fix the probability of error.
So say I like probability of error of ten to the minus five.
So that's something that the system specifies.
And from that, I know the gap to the capacity.
So if here, if I require, this is my SNR norm, then this is going to be my gap.
I wanted to cover it, but later.
That's a good point
[UNINTELLIGIBLE]
Right, this is a certain, specific system, the M-PAM system.
What Shannon says that, if you're anywhere here, you should be able to make your probability of error arbitrarily small.
So your code should basically look something like this, if you will, or even steeper
What's the difference between that curve and the wider curve?
This curve and this curve?
Yeah
This is basically what we achieved by an M-PAM system.
I don't want to go into too much of this, because we'll be doing this later.
This is the next topic.
Ideally, what we would want is something, basically, this is my Shannon limit, I want something such as the probability of error decreases right away as soon as I am just away from the Shannon limit.
All right
[UNINTELLIGIBLE]
Right.
So that's a good point.
So if I did not do coding, this is what I get.
If I did coding, this is what I would get.
So this is how much I can expect a gain from coding
So in other words, basically, this bound is basically in my channel, the SNR is such that -- I mean, I'm so energy limited that my SNR norm is lower than a certain amount, there's nothing I can do, basically, is what it's saying
Right.
Because if you want a certain spectral efficiency, there's also spectral efficiency, right?
And if your SNR is much lower, you're out of luck
So basically, you have to make your spectral efficiency higher -- or, lower
Lower, right.
Exactly.
So now let us start with the probability of error analysis, and we'll start in limited regime.
So we have Eb of E as a function of Eb/N_0, we want to quantify this trade-off.
Basically, what we are doing now is trying to quantify how this graph will look like, at least for the uncoded systems today.
We'll do coding next class, or two classes from now, as we get time.
So say I have a constellation, A, as binary PAM.
So it takes only two values, minus alpha and alpha.
So I have these two points here, alpha and minus alpha.
My receiveds are Y, symbol Y is X plus n, where X belongs to A, and N is Gaussian zero mean, variance sigma squared, where sigma squared is N_0 over 2.
And now what want to do at the receiver is given Y, you want to decide whether X was alpha or minus alpha.
That's a standard detection problem.
So how will the probability of error look like?
So suppose I transmit X equals alpha, my conditional density of Y -- let me make some room here -- will be a bell shaped curve, because of the Gaussian noise.
So this is P of Y given X equals alpha.
Similarly, if I have set X equals minus alpha, what I get is something like this.
This is p of Y given X equals minus alpha.
Excuse my handwriting there.
And the decision region is at the midpoint, Y equals zero.
This is a standard binary detection problem.
If Y is positive, you will say X equals alpha was sent.
If Y is negative, you will say X equals minus alpha was sent.
And your probability of error is simply the probability of error under this -- graph under these two curves here.
So we want to find what the probability of error is.
Just let me just do quickly the calculation, in order to remind you.
We'll be doing this over and over again.
We'll just do it once.
Without loss in generality, I can say this is probability of error, given X equals minus alpha, where both the points are equally likely.
So by symmetry, I can say that.
So this is the same as probability that Y is greater than zero, given X is minus alpha.
Now Y is X plus N, so this is same as probability that N -- yeah capital N -- is greater than alpha.
Since N has zero mean variance sigma squared, this is a standard Q function of alpha over sigma.
OK, so that's the probability of error.
Now, there is one bit per symbol for each X that is one bit, so probability of error is also same as probability of bit error, so this is also Pb of E. So I have Pb of E equals Q of alpha over sigma, and now, I want to basically express it as a function of Eb/N_0, right.
So what is Eb for the system?
It's going to be alpha squared.
Sigma squared is now N_0 over 2, so alpha squared over sigma squared is 2 Eb over N_0.
So now my probability of bit error is this Q function of root 2 Eb over N_0.
OK?
So let us plot this.
So I'm on the x-axis, I'm plotting Eb/N_0, which is in dB scale.
On the y-axis, I'm going to plot the probability of bit error.
Typically, what you do is you plot the y-axis on a semi-log scale.
So this is ten to the minus six, ten to the minus five.
If you want to do this in MATLAB, you can use this command semi-log y, and that does it.
And so on.
So can anybody say what will be a good candidate for the x-value at this point here?
So what should be the Eb/N_0 at this point?
[UNINTELLIGIBLE]
It should be the Shannon limit, right?
You cannot hope to go below the Shannon limit.
So now, what Shannon says is that -- so I'm going to plot this as my Shannon limit here.
This probability of error will basically use the standard waterfall curve.
This is 2-PAM.
And if you look at the x-coordinates, then the value at ten to the minus five is 9.6 dB.
So as a system designer, if you care about probability of error at ten to the negative five, then your gap to capacity, or gap to the ultimate limit, if you will, will be this, 9.6 minus of minus 1.59 dB.
OK, I should not erase this.
OK, so the first thing is at N to the minus five, our gap to the ultimate limit is 9.6 plus 1.59 dB, and that's approximately 11.2 dB.
OK?
But there is one catch to this.
This particular system has a spectral efficiency of two bits for two dimensions, whereas if you want to achieve something close to the Shanon limit, you have to drive the spectral efficiency down to zero.
So you might say that this is not a fair comparison.
So if you do want to make a fair comparison, you want to fix rho to be two bits for two dimensions.
So if you fix rho for two bits per two dimensions, you will get a limit somewhere, not at 1.59 dB, but at some other point.
This is if you fix rho2, two bits for two dimensions.
Can anybody say what that point will be?
[UNINTELLIGIBLE]
3 over 2.
1.5.
What will it be in the log scale?
[UNINTELLIGIBLE]
1.76.
Good.
So what we do know -- if we do the calculation here.
If you fix rho2, two bits for two dimensions, your Eb/N_0, we know, is greater than 2 to the rho minus 1 over rho, which is 3 over 2.
And that is, if you remember your dB tables, 1.76 dB.
Log of 3 is like 4.7, log of 2 is like 3, so it's 1.76 dB.
So in this case, your gap to the ultimate limit is going to be 9.6 minus 1.76, which comes out to be 7.8 dB.
OK?
So now, let us do the bandwidth-limited regime.
Now in bandwidth-limited regime, the trade-off is Ps of E as a function of SNR norm.
OK, so that's the trade-off we seek for.
And the baseline system we will be using is an M-PAM system.
So now, your constellation is going to be minus alpha, alpha, 3 alpha, minus 3 alpha, and so on, up to M minus 1 alpha -- I always run out of room here -- minus M minus 1 alpha.
This is your constellation.
Now what's the probability of error going to be for this constellation?
Well, what's the probability of error for each intermediate point?
There are two ways you can make an error.
Say we say alpha, your noise is either too small, so it takes you to minus alpha.
Your noise is too high, so it takes you to 3 alpha.
For each one, the probability will be Q of -- this distance is 2 alpha, it would be Q of alpha over sigma, because you will be having your decision regions right here.
So in other words, for each intermediate point -- and there are M minus two of this -- your probability of error would be 2 times Q of alpha over sigma.
OK?
And how many of them there are?
There are M minus 2 of these.
And if all the points are equally likely, and you want to find the probability of error, you divide by M. For the two end points, the noise can only make an error in one direction.
So you get two over M times Q of alpha over sigma.
You work this out, and you have two times M minus 1 over M, times Q of alpha over sigma.
Now, we want to find probability of error for two symbols, because that's what Ps of E is, OK?
So what's Ps of E going to be?
Well, it's -- in terms of Pr of E, it's going to be one minus one minus Pr of E squared.
This is the probability you make error in none of the two symbols, and 1 minus that will be the probability of error you make in at least one symbol.
And that's going to be equal to two Pr of E, or it's four times M minus 1 over M, Q of alpha over sigma.
Good, I did not write on this board.
So now what remains to do is to relate alpha over sigma to SNR norm.
And I had it on this board here.
SNR norm, we just did in the previous example, for an M-PAM system is alpha squared over 3 sigma squared.
OK?
So I plug that in here, so I get Ps of E is 4 times M minus 1 over M times Q of root 3 SNR norm.
And if M is large, this is approximately 4 times Q of root 3 SNR norm.
So we are plotting now probability of error as a function of SNR norm, similar to what we did in that part in the power-limited regime there.
So this is Ps of E as a function of SNR norm.
Now my Shannon limit would be at 0 dB, so this is my Shannon limit point.
This is ten to the minus six, ten to the minus five, ten to the minus four, ten to the minus three, ten to the minus two, and so on.
This is going to be the performance of M-PAM.
And again, I turn to the minus five, which will be the kind of performance criteria we'll be using throughout this course.
You'll see that this y-axis is 8.4 dB.
So in this case, the gap to capacity is going to be 8.4 dB.
So if I want a certain spectral efficiency and I use M-PAM, I'm 8.4 dB away from the Shannon limit.
The idea behind coding, as someone pointed out, was to start from here and do coding to come closer and closer to bridge this gap.
OK?
So are there any questions?
We are almost end time.
OK.
I think -- yes?
A perfect code would just be literally like a step function --
Right
-- all the way down
That's what Shannon coded.
OK, I think this is a natural point to stop.
We'll continue next class.
And what we saw was the performance was quite different in the two regimes.
In power-limited regime, typically our SNR is much smaller than one, whereas in the bandwidth-limited regime, the SNR is large.
And because of this behavior of SNR, what we saw is that the Shannon spectral efficiency in the bandwidth-limited regime, it doubles for every -- if we double our SNR.
For every 3 dB increase in SNR, the spectral efficiency increases by a factor of 2.
In the bandwidth limited regime, if we have a 3 dB increase in SNR, the spectral efficiency increases by one bit per two dimension.
On the other hand, if we double our bandwidth, then the capacity in bits per second is not affected in the power-limited regime.
But if we double our bandwidth, then in the bandwidth-limited regime, the capacity increases approximately by a factor of 2.
And that's really what motivated the name bandwidth-limited and power-limited regime.
If we want a more operational definition, then we say that the flow is less than two bits per two dimensions, we will have this bandwidth-limited regime.
And in this case, rho is greater than two bits per two dimensions.
The number two bits per two dimensions was chosen because it is like the largest spectral efficiency we can get through binary transmission.
If it's an uncoded two-PAM over a channel, then we get two bits per two dimensions.
If we are coding, the only thing we can do is reduce spectral efficiency.
So typically, if we are going to operate in power-limited regime, the operational meaning is we can get away with binary transmission.
In the bandwidth-limited regime, we have to resort to non-binary transmission.
So in other words, binary modulation is done in power-limited regime, whereas we need multi-level modulation in the bandwidth-limited regime.
The baseline system here is 2-PAM.
The uncoded performance was of 2-PAM.
In the bandwidth limited regime, it is M-PAM.
And the way we measure performance in the power-limited regime is the probability of bit error as a function of EbN0.
OK?
In the bandwidth-limited regime, the performance measure is done by probability of error per two dimensions as a function of SNR norm.
And in this case, we saw that the gap to capacity -- or rather, to put it in other words, the ultimate limit on EbN0 is minus 1.59 dB.
And here, the ultimate limit on SNR norm is 0 dB.
OK?
Any questions on this?
Yes
Why do we use Eb over N_0 SNR norm for [INAUDIBLE] bandwidth limited?
That's a good question
Why do we use Eb over N_0 for both regimes?
Or why don't use SNR norm for both regimes?
Yeah
Now if we think about the bandwidth limited regime, what we really care about is spectral efficiency, right?
What SNR norm does, if you remember the definition, is that it compass the amount of SNR we require for a practical system to that of the best possible system.
So in other words, if you do care about spectral efficiency, SNR norm is the right measure to look for.
OK, now what happens in the power-limited regime?
It turns out, probably more for historic reasons, people started with EbN0 in the power-limited regime.
And if you look at the definition of EbN0, it is SNR over rho, right?
So in the power limited regime, our rho is small, the SNR is going to be small, but if you look at -- because we are in the power limited regime so we have lots of bandwidth, so our SNR is going to be small and the spectral efficiency is going to be small.
But if you look at the ratio between the two, it's going to be greater than minus 1.59 dB.
OK. so it turns out that the kind of limit we do take, our EbN0 remains constant as minus 1.59 dB, and that's probably one of the reasons that motivated to use EbN0 in the power limited regime.
On the other hand, one could also argue is that what really happens in the power limited regime is that our bandwidth becomes really large.
So if this [UNINTELLIGIBLE] stick with 2-PAM system, then we do get a spectral efficiency of two bits per two dimensions, but that's just because we are using a particular modulation scheme.
If our bandwidth is really large, we are not really going to care about what spectral efficiency we use.
What really matters is this energy per bit, and that's why this is a reasonable assumption.
Does that answer your question?
Right, it's not completely clear as to why this EbN0 is the best here, and SNR norm is here if you don't take the limit rho going to zero here, but again, you can think of it more as a convention
[INAUDIBLE]
So if you look in the power-limited regime, you are saying rho is less than two bits per two dimensions.
If you use an uncoded 2-PAM, what's your spectral efficiency?
It's two bits per two dimension, right?
Now the idea is, suppose we want to design a system with spectral efficiency greater than two bits per two dimensions?
We cannot really use a 2-PAM system.
Because if you put coding on top of it, all we are going to do is simply reduce the spectral efficiency below two bits per two dimension.
So we have to start with a non-binary modulation, right?
So that's how we distinguish between power-limited and bandwidth-limited
That's just because you're using the 2-PAM as your baseline [INAUDIBLE]?
Right.
OK?
All right.
So let us do an example to finish off this analysis.
Now suppose -- say you are at a summer project, and you're assigned to design some system.
Your boss gives you some specifications, like continuous time specifications.
In particular, you have a baseband system.
The baseband system has a bandwidth of one Megahertz.
You have a power, P, which is one unit, so that's another resource you have.
And if you measure your channel, it can be reasonably approximated as an AWGN channel, so there is no ISI or any filtering going on, just Additive White Gaussian Noise.
And your noise a single sided spectral density of ten to the minus six units per Hertz of the bandwidth.
And what is your goal?
So you have the following goal.
Design a 2-PAM system, with a specified probability of bit error.
And what you want to do is compare this to the ultimate Shannon limit.
So that is your objective.
So since we have to compare it with Shannon limit, and we have already a formula for the Shannon limit, let's just start with that.
So for this problem, we have the Shannon limit.
You have rho is less than log base 2 of 1 plus SNR.
For SNR, I can talk in terms of this continuous time parameters, it's P over N_0 W. My P is one unit, and nought is ten to the minus six, and my bandwidth, W, is one Megahertz here, which is ten to the six.
So my P over N_0 W is basically one.
So I have 1 plus 1, which is 2.
This is log base 2 of 2, or it's one bit per two dimension.
So my capacity in bits per second is rho W, and rho is one bit per two dimension, the bandwidth is one Megahertz.
So I get ten to the six bits per second.
So this is my Shannon capacity for this particular AWGN system.
OK.
The next thing we want to do is compare this with a practical system, and see how close we get to the Shannon limit.
And since you only have to work with 2-PAM, the generic architecture is something we saw last time.
You have input bits coming in, let's call them X sub k, where k is the kth bit.
And they belong to a certain constellation, let's call the -- the constellation points are just -- it's a 2-PAM system, so we have minus alpha and alpha.
And this goes through a PAM modulator, and one parameter to specify for the PAM modulator is the symbol interval.
Right, the time between sending consecutive signals over the channel.
What you get out is X of t, this is the channel model, N of t. There's already noise over the channel, and what you get out is Y of t. So this is the generic architecture.
And now, your goal for the design problem is to select alpha and t in the right way, so that they satisfy this continuous time constraints, and at the same time, you have your probability of error of ten to the minus five.
So what would be an obvious choice for T?
[INAUDIBLE]
Right.
So the first idea is you are given a certain amount of bandwidth, and you clearly want send your signals as fast as possible in order to get excellent data rate.
Now because you have a certain amount of bandwidth, what Nyquist's criteria tells you is that you want to have zero ISI.
And if you want to have zero ISI, what you do know is that the symbol interval should be greater than or equal to 1 over 2W.
You cannot signal at a rate faster than one over T, and so if we look at this, it's 1 over 2 times 10 to the 6.
Now it I do use this particular value of T, then what's my alpha going to be?
Well, alpha is simply the energy per symbol.
So I know alpha squared is the power that I have times the symbol interval, T. It's just a definition.
This comes from orthonormality of the PAM system that we have.
Now P is one, because that's what I specified as a system specification, so this is just T, which is one over times ten to the six.
So I can select this value of alpha and this value of
[INAUDIBLE]
Well, alpha squared is energy per symbol.
So what will that be?
What's the energy per symbol, if you're sending every T seconds, and if you have a power of P?
Es, that I mentioned last time, or energy per two dimensions.
So in PAM, it will be energy per symbols.
In that case, it will be 2P.
OK.
But now if I select these values of alpha and T, will my system work?
Is this a reasonable design, or is there something wrong here?
I'm clearly satisfying my --
[INAUDIBLE]
The probability of error, right, exactly.
So in fact, I do know how to calculate it, right?
What's the probability of bit error?
Well, we saw that last time it was Q of root 2 Eb over N_0.
Eb is same as alpha squared, because we have one bit per symbol.
So alpha squared is this quantity here, 1 over 2 times 10 to the 6.
So this is Q of square root of -- so 2 alpha squared is 10 to the 6, 1 over 10 to the 6.
N_0, I know, is ten to the minus six, so this is actually Q of 1.
And if I do calculate that, it's like 17 percent, which is nowhere close to ten to the minus five.
So any suggestions on how I can improve my system?
Increase T [INAUDIBLE]
Increase T, right?
What's happening right now is -- the reason we selected this value of T in the first place is because we wanted to send our signals as fast as possible avoid ISI, but that's just one of criteria in my system, right?
I have to also satisfy this probability of error criteria, so I want to make sure my probability of error is going to be small.
If I look at the expression for probability of error, it doesn't really look at T. All it looks at is this ratio of Eb/N0, right?
So if I want to reduce my probability of error, I have to increase my energy per bit.
Now my energy per bit is P times T, so the only hope of increasing my energy per bit will be to increase T, which means I have to signal at a slower rate.
OK?
So we have probability of -- let's write the calculation down.
It's ten to the minus five.
Last time, we saw that the best way to solve this is to look at the waterfall curve, and EbN_0 in this case is approximately 9.6 dB.
I will say that that's approximately ten on the linear scale.
So this implies that energy per bit is ten to the minus five.
So energy per bit is P times T, in this case, it's ten to the minus five.
p is one, so this implies that t is ten to the minus five.
So I can send one bit every ten to the minus five seconds.
So my rate that I achieve -- just write it here -- which is ten to the five bits per second.
OK?
If you compare this to the Shannon limit, the Shannon limit is right here, you have ten to the six bits per second.
So you lose by a factor of ten in your data rate if you're going to use an uncoded 2-PAM system.
So what this example tells you is that if you're going to do more sophisticated cording, you can gain up to a factor of ten in your data rate.
So if the 10 dB did not really impress you last time, hopefully this example throws more light on the value of coding.
Are there any questions on this example?
Since the -- since we are signaling at a faster rate now, instead of using sink process, we can use something better
That's a very good point, yes.
Well, if you look at the nominal bandwidth here, it's 1 over 2T, right?
T is 10 to the minus 5 seconds, so this says 1 over 2 times 10 to the minus 5.
So it's going to be 5 times 10 to the 4, or 50 KHz.
OK?
The available bandwidth you have, the system bandwidth, if you will, is 1 Megahertz.
But if you're going to do Nyquist's ideal sinks pulses, then you only need 50 KHz of bandwidth in your system, right?
So one advantage of this system, if you will, is that you're not required to do the complicated sink pulses.
Do not need to send those pulses.
You could simply send, for example, square pulses and, because your bandwidth is such low, you have a very low complexity system.
Of course, the price you pay is you reduce the data rate by a factor of ten.
OK, it's a good point.
In fact, there are many points that will come up in this example if you think about it later on, so feel free to ask me questions if you think about some issues later on.
Ok.
So I think we have motivated the need for coding enough now, so let's look at our encoder design.
So a typical encoder design takes bits in -- we saw this last time in the context of spectral efficiency -- and produces symbols out.
So I can represent my bits by, say a vector b, and I can represent my symbols by a vector x.
So every sequence of b bits gets mapped to a sequence of N symbols.
Now this output sequence of symbols is not any arbitrary sequence, but it lies in a set of all possible sequences, which I denote by C. And this set is essentially a set of permissible output symbol sequences, which I will write by C sub j, which is a vector in Rn, because there are N symbols being produced.
And we can have set up to j, j goes from 1 to M. So we can have up to M symbols.
And note that here, M has to be equal to 2 to the b, in order to be able to map every sequence of b bits to M symbols.
So this C is known as a codebook, and each C sub j is called a codeword.
OK?
The standard definition of an encoder.
Now in today's lecture and half of next week's lecture, we will be seeing at a very specific case when N equals 1 and 2.
In that case, instead of using the letter C, we will be using a different letter, A.
So C, in that case, we'll call it actually a constellation.
So in particular if C is one, it's a PAM constellation.
If N is 2, it's a QAM constellation.
And we'll be denoting it by a letter A instead of C. So A is again, a sequence of symbols a_j, which belongs to Rn, where one is less than j is less than M. OK, in this case a is a constellation.
a_j's are known as symbols, or sometimes they're also known as signal points in the constellation.
There are a number of definitions that follow from this -- number of properties of the constellation, rather, that follow.
So in particular N is known as the dimension of your constellation.
The number N is this, and here, it's the number of symbol sequences you output for a sequence of b bits that are in.
M is the size of your constellation.
The energy per constellation is given by 1 over M times the summation the norm of a_j squared, where j goes from one to M. The minimum distance of your constellation is simply the Euclidean minimum distance between two points in the constellation.
So if you take the norm of a_i minus a_j, and minimize it over all possible values i and j.
The number of nearest neighbors, of the average number of nearest -- K_min of A is the average number of nearest neighbors in A.
In addition to this, there are some orthonormalized parameters that you saw last time.
The spectral efficiency, which is in units of bits per two dimensions is 2b over N. And if you want to eliminate b, we use the relation that b is log M to the base 2 here.
And so we have 2 log M to the base 2 over N. The energy per two dimensions, denoted by Es, is simply 2 over N E(A).
So E(A) is the average energy of your constellation.
If you divide it by the number of dimensions you have, you get energy per dimension, and you multiply it by 2.
And finally, the energy per bit is Es over rho, or it can also be expressed as E(A), which is the energy per symbol over the number of bits per symbol, which is log M to the base 2.
It might seem like a lot of definitions, but you will see very soon that they have a very tight interplay among one another, so it's not nearly as overwhelming as it might seem at the first point
[INAUDIBLE]
Yes
Why is [UNINTELLIGIBLE]?
Because you have two [UNINTELLIGIBLE] b bits coming in, right, which you map to each --
[INAUDIBLE] There are two [INAUDIBLE] possible sequences, but all of them need not be used, right?
Well, we assume that there is no coding going on before the encoder.
So you have the source code, for which there's a sequence of IID bits, and then they mapped to a sequence of symbols.
So we'll see all of our possible input bits coming in here, because it's produced by a source code, like a Huffman code.
Right?
And the idea here is, perhaps what you're asking is -- this did not span the entire space of Rn.
We want to select these sequences carefully here.
Maybe we'll come back to that later on in the course.
OK, so let's do an example.
So the example is, say we have A, which is a 2-PAM system, and you want to look at this constellation, B, which is denoted by A raised to K. The definition of A raised to k is it's a sequence of K symbols where each x_i belongs to A.
So this is also known as the K-fold Cartesian product of
Another question.
So it has been pre-decided that b bits will be encoded [UNINTELLIGIBLE]?
Right.
So this is a specific structure we are imposing on the encoder.
So this is the constellation, and you'll want to study the properties for this constellation.
For this constellation, what's N going to be?
What's the dimension going to be?
[INAUDIBLE]
For B, not A.
It's going to be K. Well, the number of points in this constellation, how many points are there?
There are K coordinates.
Each coordinate can be plus or minus alpha.
So we have 2 to the K possible points in this constellation.
OK?
What's E of A going to be?
[INAUDIBLE]
K alpha squared.
right?
Basically, we have K coordinates.
The energy for each coordinate will simply add up.
The energy across each coordinate is always going to be alpha squared.
So each point in this constellation has an energy of K alpha squared.
So regardless of how many points we have, the average energy is always going to be K alpha squared.
Does everybody see this?
[INAUDIBLE]
You're right.
Maybe that was the confusion.
It's a good thing.
Just getting too used to writing E of A. OK. What's d_min of b going to be?
[INAUDIBLE] 2 alpha
2 alpha.
I think everybody had the right idea.
So the minimum distance here is two alpha for A.
If we look at this point B, we can fix K minus 1 coordinates for two points to be the same, they will only differ in one point.
And so the minimum distance is across that point, which is 2 alpha.
What is K_min of B going to be?
It's going to be K. For each point -- let's say the point which has all alphas, we can fix K minus 1 coordinate and find another point which is different in only one of the coordinates, say the first coordinate.
We can do it for all K different coordinates, so K_min is going to be K for each point, and hence the average number of nearest neighbors is also K. OK, so now in this case, let's first start with the normalized parameters.
That's always good to start with spectral efficiency.
That's 2 log M over N. Well, log of M is going to be K, so N is going to be K. So this is going to be two bits per two dimensions, and this is the same as that of the original constellation, A.
Your energy per two dimensions is going to be 2 over N E(B).
E(B) is K alpha squared, N equals K, so this is 2 alpha squared, and that is the same as the original constellation, A.
Finally.
energy per bit is Es over rho, so it's 2 alpha squared over 2.
So that's alpha squared, and that's same as the 2-PAM constellation.
So why did I go through all of these calculations?
What we see is that the normalized parameters, rho, Es, and Eb, are the same for the Cartesian product as for the original constellation.
And at some level, that should not be too surprising, right?
Because what I'll be doing in this Cartesian product, we are not really doing any coding, right?
In this original constellation, we had one bit coming in, and we are mapping it to one symbol.
All we are doing in the Cartesian product is we are taking K bits in and mapping them to K symbols.
So we still have one bit per symbol.
The noise is IID, so it's optimal to two decisions for each of the coordinates independently, and decide whether that coordinate corresponds to plus alpha or minus alpha.
So in other words, there's nothing gained by doing this Cartesian product.
And we will see, the probability of error expression depends on these normalized parameters, if we want to look at Pb of E, and so we do not gain anything in terms of the probability of error, versus EbN_0, trade-off through Cartesian product.
So I'm making the note here because that's the only space I have.
So the note is if I look at probability of bit error versus EbN_0, the curve we saw last time, it is the same for B and A.
You should be able to convince yourself about this, and so there is really no coding going on here.
Are there any questions on this?
Let's look at this problem a bit more carefully now
[INAUDIBLE]
Yeah
Why [INAUDIBLE]
Right
What does he use?
Energy per two dimensions.
Es will always be energy per two dimensions.
Throughout the course, we'll be using these notations.
Eb is the energy per bit, Es is the energy per two dimensions.
And if you want to say energy per symbol, we'll be using this notation E sub the constellation
Oh.
It's not energy per bit
No, this is energy -- B is my constellation.
So that's why it's energy of that constellation, average energy per symbol in that constellation.
OK, so let us consider the special case when K equals 3.
So in that case, B is A^q.
So if I look at all possible points in B, they are going to lie on the vertices of a three-dimensional cube.
That's a Cartesian product in three dimensions.
And all my constellations points are basically on the vertices of this cube.
The distance here is going to be 2 alpha.
That's the length of each edge in my cube, and that's what B is going to be.
Clearly, the minimum distances is 2 alpha, as we saw before.
Now let me define a different constellation, B prime, and only going to take four vertices from these possible eight vertices.
I'm going to take this vertex here, I'm going to take this vertex here, this one, and this one.
I'm only taking four vertices.
If I want to tell you explicitly what the points are, I need to draw an axis, so I'm simply drawing the x, y, and z axis here.
This is x-axis, this is y-axis, and z-axis.
And B prime is a subset of the points in this three-dimensional Cartesian product.
They will be alpha, alpha, alpha; minus alpha, minus alpha, alpha; alpha, minus alpha, minus alpha; and let's see, minus alpha, alpha, minus alpha.
So two of the coordinates will be minus alpha here, in these three points, and we have one coordinate all alphas.
So this is my B prime.
What is the minimum distance going to be for B prime?
[INAUDIBLE]
2 over 2 alpha, right?
It's basically the length of this edge here.
This is 2 alpha, this is 2 alpha.
So it's 2 over 2 alpha.
So in other words, by simply selecting a subset of points, I have been able to increase my minimum distance.
Because my minimum distance is larger, I hope that the probability of error will be smaller as opposed to the original constellation.
But this comes at the price, right?
And what's the price?
The spectral efficiency is smaller, right?
What if I look at my spectral efficiency?
Well, I'm only sending out two points, two bits per each point.
So two bits, each point takes three dimensions, so my spectral efficiency is 2 times 2 over 3 bits per two dimensions, or it's 4 over 3 bits per two dimensions.
And this is in contrast to the two bits per two dimensions we had for B.
So in other words, there is a trade-off between your spectral efficiency and the minimum distance.
We'll start with a K-dimensional Cartesian product of A, which has all points.
We took a subset of points, and if we chose them smartly, we were able to increase the minimum distance, but the price we had to pay was to reduce the spectral efficiency
Where did this two / three come from?
This two here?
2/3, yes
2/3, I am sending two bits per symbol, right?
Each symbol has three dimensions.
So it's 2/3 bit per dimension, or 4/3 bit per two dimension.
OK, so the point was it seems like there is a trade-off between minimum distance and spectral efficiency.
And indeed, this might seem like a reasonable trade-off, and a lot of coding here that we will be seeing in the early part of the course is indeed motivated by this trade-off.
You want to reduce your spectral efficiency in order to increase your probability of error.
And this has in fact been quite a dominant design principle for a large number of codes that have come up in coding theory.
However, if you look at what Shannon says, Shannon says something quite different.
In Shannon's theorem, all they say is, you have -- if your spectral efficiency is below a certain amount, then your probability of bit error can be made arbitrarily small.
OK, so what Shannon is saying, it's something much stronger than this trade-off.
It's saying if you reduce your spectral efficiency below a certain quantity which is finite, then the probability of error can be made arbitrarily small.
There is no statement of minimum distance in this theorem here.
And indeed, if you look at the most modern codes which are capacity approaching, they are not designed to maximize the minimum distance.
They are designed to work well with some practical decoding algorithms, like the belief propagation of algorithms and so on.
So they are designed on a somewhat different principle than minimum distance.
But nevertheless, this is quite a powerful tool that we will be using in the early part of this course.
We start with a K-dimensional Cartesian product, select a subset of points, and we want to increase the minimum distance at the cost of spectral efficiency.
OK. Now are there any questions?
[INAUDIBLE]
That's a good question.
Suppose I have a 2-PAM constellation, then I can easily write the probability of bit error as a function of Q function.
If it is a more complicated expression, I have to integrate over the decision regions, which we'll be seeing later on in this lecture.
And it's not usually possible to get an exact probability of error expression.
We usually use an in-union bound, to bound it by a pair of [UNINTELLIGIBLE] error probability.
We'll be doing all that just now.
OK.
So now let us -- I have talked now enough now about encoder, and we'll be visiting it very soon, but let us switch gears and talk about the decoder now.
OK, what does a decoder do?
So the goal of a decoder is the following.
You get your received vector Y, which is X plus N, and from Y, you want to estimate X-hat as a point in your signal constellation.
So you receive a noisy version of X, and you want to estimate X-hat at the decoder.
So this is the architecture of your decoder.
And the goal here is you want to minimize the probability of error.
And what's the probability of error?
It's basically probability that X is not equal to X-hat.
So that is your general criteria at the decoder.
Now what we'll doing next is basically going through this exercise to show that this minimum probability of error criteria is equivalent to a bunch of other criteria.
So the first criteria is the MAP criteria: Maximum A-Posteriori Rule.
So our probability of error is basically -- I can track it as an integral of probability of error given Y times the density function of Y.
So if I want to minimize my probability of error, I want to minimize each term in this integral.
So this implies I want to minimize probability of error given Y for each possible value of Y. OK?
Now what's that going to be?
Well, in order to look at what this term is, suppose I make a decision.
I receive Y, and I decide a symbol a_j is sent.
Then what's the probability of error going to be?
My probability of error given Y is going to be 1 minus the probability that I was correct.
Probability that I was correct is probability X equals a_j, given Y.
This follows from the definition.
So if I want to minimize my probability of error given Y, I want to actually choose an a_j that maximizes the probability of a_j given Y.
So this implies, choose a_j.
And this is known as the MAP rule.
So the idea behind the MAP rule is to choose the symbol in the constellation that maximizes the posterior probability, given the received symbol.
Now, this MAP rule is equivalent to the maximum likelihood rule, under the assumption that all the signal points a_j are equally likely.
The proof is not hard, you just use Bayes Theorem for that.
So suppose all a_j's are equally likely.
Then probability of a_j given Y, which by Bayes Theorem is the density of Y given a_j, times the probability of a_j.
But since all a_j's are equally likely, I will just write it as 1 over M, over the density of Y.
Now because Y is fixed, the density of Y is fixed, so this quantity is just proportional to -- the proportionality symbol -- to the density of Y given a_j.
I won't be writing all the vectors, I might be missing some.
But please bear with me.
So this implies we want to choose a_j that maximizes the density of Y given a_j, and this is known as the maximum likelihood rule.
And there is one final rule.
Basically if the noise is additive Gaussian, then the density of Y given a_j is simply proportional to E raised to minus the norm of Y minus a_j squared.
So if we want to maximize this quantity, we want to minimize Y minus a_j squared.
So we want to choose a_j that minimizes the Euclidean distance between Y minus a_j.
And this is known as the minimum distance decision rule, MDD rule.
Yes?
We are ignoring [UNINTELLIGIBLE]?
We are ignoring P value.
Because for a given Y, Py is going to be fixed for all possible choices of a_j.
The goal is I'm given Y, and I want to decide which signal point was set, because that's the probability of error given Y.
This is my criteria now.
So Y is fixed, so the density of Y is fixed
[INAUDIBLE]
It's basically given by -- in order to find this density, we'll just condition it on a_j, and sum up over all possible values of a_j.
Just take the marginal of Y, right?
I mean, to write this explicitly.
I'm writing it here.
It's going to be sigma P of Y given a_j that's the probability of a_j.
And this you can find by the Gaussian
But then you have [INAUDIBLE]
But this is a summation over all possible a_j's.
I should write, sorry -- a_k's.
This is just a summation over all k's, right?
So basically, Py is going to be a mixture of several Gaussians, OK?
And it's fixed
[INAUDIBLE]
Right.
OK.
So we want to choose the Minimum Distance Decision rule, and I should have the variance of noise here.
OK, so what we have so far is we started with the Minimum Probability of Error rule, and that's the criteria of your decoder.
Be sure this is equivalent to MAP rule, and that basically comes just from the definition.
This integral here, we want to minimize each term in the integral, and that basically implies that the Maximum A-Posteriori rule is the best.
This implied Maximum Likelihood rule, and Maximum Likelihood rule comes from the fact that all the signal points are equally likely.
And this implies, then, the Minimum Distance Decision rule, and that comes from the fact that your noise is Additive Gaussian.
So you have an exponential in the -- you have the Euclidean distance as an exponent, and you want to minimize the Euclidean distance.
So this is the story we have so far.
And it turns out that the Minimum Distance Decision rule is actually quite nice, because it gives you a lot of geometrical insights.
So say I have three points.
We not even draw the coordinates.
My constellation, A, has three points, and let me write them as a1, a2, a3.
This is my constellation, for example.
And say I receive a symbol Y.
Then the job of the decoder is to figure out whether I sent a1, a2 or a3.
How will the decoder do that?
Well, it will measure the distance from all the three constellation points and select the one with the smallest Euclidean distance.
More generally, what we want to do is we want to look at the space of all received Y symbols, and partition it into decision regions.
So that if the point falls in a certain decision region, we say that the constellation point corresponding to that decision region was sent.
And how do I find the decision region?
Well, I start drawing hyperplanes between every two pair of constellation points.
Say I want to find the decision region of point a1.
I draw a hyperplane between a1 and a3, which is given by this line.
I draw a hyperplane between a1 and a2 which is given by this point.
And so the region -- the set of points which are closer to a1 than a2 and a3 is basically bounded by this region here.
So this is my R1.
Similarly, if I want to find a decision region for a2 and a3, I will draw this line here.
This will be R2 and this will be R3.
So these are my decision regions.
So if I want to write that formally, Rj is my decision region.
And it is the set of points Y belong to Rn, such that the norm of Y minus a_j squared is going to be less than or equal to -- it doesn't matter if you have less than or equal to, because the point [UNINTELLIGIBLE] bound [UNINTELLIGIBLE] probability zero -- is radius squared.
That's just the definition of Rj.
Now the way to construct Rj was to look at all the half planes which are closer to this point than any other point, and take the intersection of all the half planes.
So in other words, I can also write Rj to be the intersection of these half planes -- the intersections over all points, j prime not equal to j -- of Rj, j prime.
So Rj, j prime is your half plane where -- so norm of Y minus a_j prime squared is greater than or equal to norm of Y minus a_j squared.
Note that this is a_j prime, and this is a_j here.
OK.
So it turns out that this decision region has a somewhat nice structure, because they're intersection of a bunch of half planes, their shape is the convex polytope.
And they're also known by the name Voronoi regions.
OK, so these regions are known as Voronoi regions here.
Now the set of points whose hyperplanes are active in a certain decision region has a special name, too, and it's called the relevant subset.
So in this case, the relevant subset of a1 is a2 and a3, because both of them have hyperplanes that are active in the decision region of a1.
So let me write that down.
So the relevant subset is the set of points, a_j prime, whose hyperplanes are active in this decision region Rj.
There's a theorem which says that the nearest neighbors are always included in the relevant subset.
It's asserted in your notes.
OK?
So now that we have this Minimum Distance Decision rule, let us see if we can get a hang with the probability of error.
Let me see the probability of error, given that I sent a symbol, a_j.
I want the value that probability of error.
That's simply the probability that Y does not belong to Rj, given that I sent the symbol a_j.
That's when an error happens.
That's same as probability that the noise vector -- because Y is a_j plus N now -- does not belong to the Rj minus a_j, and the noise is independent of a_j so I remove the conditioning.
And that is 1 minus the probability that the noise does belong to Rj minus a_j.
If I want to find this integral, find this expression, I will integrate over the region Rj minus a_j, of the density of the noise, dN.
No note that the noise has a spherical symmetry, but unfortunately despite that, the integral is not a straightforward integral, because this region here has sharp edges.
The decision region is a convex polytope, so it's typically something like this, and if this was your point, a_j, your noise does have a spherical symmetry about these spheres, but when it intersects the decision boundary, things get ugly.
And so this decision -- this is not a nice integral in general.
And so unfortunately, there is not much progress we can make beyond this point for the exact probability of error expression, but we can say some nice geometrical properties about the probability of error.
The first property is that probability of error is invariant to translations.
And this should be quite obvious.
You have, say, a constellation with two points here, and say I subtract off the mean.
So I get a different constellation whose points are like this.
This is my constellation, A, and this is my constellation, A prime.
The probability of error will be same for the two constellations because the decision regions will have the same distance from both the points.
This should be quite obvious.
And basically, what this really says is if I have any constellation, I can always subtract off the mean, and get another constellation with the same probability of error, but with smaller average energy.
And so this implies that any optimal constellation will have zero mean.
The second point is, the probability of error is invariant to orthonormal rotations.
So if I had, say, one point, one constellation, with these four points, and I rotate it by 45 degrees, what I get is another constellation with these four points.
And both these constellations are simply rotations of one another, and they have the same probability of error.
And the easiest way to see that is the decision regions here are simply the four quadrants.
And if I want to integrate my probability of error, I will be integrating it over this region.
Here, my decision regions will be these 45 degree lines.
And if I want to integrate the probability of error for this point here, it will be given by noise, which is symmetric about these circles.
Basically, since the noise is invariant to orthonormal rotations, it should be quite obvious that the probability of error is invariant to rotations
[INAUDIBLE]
Any rotation, right
So what do you mean by [UNINTELLIGIBLE]?
Basically, you're preserving the distance.
So you're just rotating the point, not scaling it
[INAUDIBLE]
Unitarily.
OK?
I mean you will be proving these properties in the next homework, which is just handed out
So those [UNINTELLIGIBLE] hold, because Minimum Distance rule is orthonormal, right?
And that's because you assume Gaussian --
Because you assume Gaussian noise
So that's the only assumption we make?
Right
-- for those [INAUDIBLE], right?
I think so
Why [INAUDIBLE] constellation must have zero mean?
Because if I have any constellation, there's a certain probability of error, right?
Can always subtract out the mean from the constellation, I get a new constellation with a smaller average energy with the same probability of error
Oh, so in terms of [INAUDIBLE]
Right.
If you're looking at a trade-off of probability of error versus energy, usually what we look at.
So maybe that's a good point.
I should just mention it.
For probability of error versus -- we're looking at this trade-off here.
Ok.
The next idea is to basically bound the probability of error by a union bound, because we cannot compute an exact expression for the probability of error, so we might as well compute a bound which is tractable.
So we'll look at what is known as the pairwise error probability.
So the idea behind pairwise error probability is suppose I send a point, a_j, what is the probability that instead of a_j at the receiver, I decide that a_j prime was sent.
This is the pairwise error probability.
So geometrically, say a_j and a_j prime are two points here.
Let me draw some coordinate axis here.
And say I sent point a_j, and there is noise on the channel, that takes me to this point.
So this is my Y, and this is the noise vector.
OK?
And now what I want to know is under what conditions will I decide a_j prime over a_j.
What is the probability of deciding a_j prime over a_j?
So let's draw a line joining a_j prime and a_j.
So how would I decide -- suppose I receive this point, Y, and I wanted to decide between a_j and a_j prime.
What would be my decision rule?
[INAUDIBLE]
Uh-huh
[INAUDIBLE]
You select --
[INAUDIBLE] a_j prime
Exactly.
An equivalent way of saying it is to project Y onto this line, a_j prime minus a_j.
We take two projections, one orthonormal to the line, one along on the line, and receive this projection.
This is a straight line like this.
Let's call it n tilde.
I should change my chalk, it's getting too blunt now.
This projection here is closer to a_j prime or a_j.
So in other words, this probability of error is same as the probability that this n tilde, which is the projection of Y onto a_j prime minus a_j, is greater than or equal to the norm of a_j prime minus a_j over 2.
OK, now why did I use the notation n tilde here?
Because the projection Y onto a_j, which is this.
n tilde is same as the projection of the noise onto a line joining a_j prime minus a_j.
So n tilde, I can write it as projection of N onto a_j prime minus a_j over the norm
[INAUDIBLE]
Sorry?
Why, exactly?
You can just see geometrically, right?
This is a 90 degree here.
This is the noise.
If I project the noise, it will be this component here
[INAUDIBLE]
All right.
This should be 90, I'm sorry.
This is 90.
I'm messing things up.
OK, this is the noise here.
This noise, if I project it onto a_j prime minus a_j, it's going to be this component.
This is Y, if I project it, it's the same component.
Now, if the noise is IID with variance sigma squared in each coordinate, we are simply projecting the noise onto one orthonormal vector.
So n tilde is also Gaussian, with zero mean variance sigma squared.
So we can use that to find this probability of error.
So in that case, the probability of error -- I should write this -- probability of a_j prime going to a_j is simply probability that this Gaussian is greater than some distance, and that's Q of norm of a_j prime minus a_j over two sigma.
Yes?
What is sigma?
So the noise vector is IID, in each of the components, and has a variance of sigma squared.
Sigma is basically N_0 over 2, if your noise is flat with -- so let me just write that down, sigma squared is N_0 over 2.
If you have an AWGN channel, and you project it on each orthonormal signal, that's what you get
What if you project the noise vector on [INAUDIBLE] why is [INAUDIBLE] you don't have --
So you have a noise vector.
If you have a Gaussian vector, and you project it onto an orthonormal basis --
Yes.
But [INAUDIBLE] normal?
Right.
[INAUDIBLE] We are only projecting out on one vector which you need now
So then you're saying -- OK, yeah.
The assumption is the noise is symmetric in all dimensions?
Right.
Let's do this algebraically, so you're convinced that there is no magic I'm doing here.
So we can write this as you said, as the probability that the norm of Y minus a_j squared is greater than norm of Y minus a_j prime squared, given that Y [UNINTELLIGIBLE] a_j, so Y is a_j plus the noise vector.
So I sub in for Y.
What I get is probability that Y is a_j plus N. So here I have norm of N squared is greater than or equal to norm of a_j plus N minus a_j prime squared.
And since the only random variable here is this noise, N, I can remove the conditioning [UNINTELLIGIBLE] down there.
Let me expand this second norm term there.
That's basically the norm of a_j minus a_j prime squared plus the norm of N squared minus two times the projection of N -- or rather, the inner product of N -- and a_j prime minus a_j.
So this is the probability that the inner product of N and a_j prime minus a_j is greater than or equal to norm of a_j prime minus a_j squared over 2.
If you divide by the norm of a_j prime minus a_j, you get the same expression as we had.
So it's the same thing.
This was done geometrically, this is done algebraically.
So this is the expression of the probability of error, and this is Q of the norm of a_j prime minus a_j over 2 sigma.
So now that we have the pairwise error probability, we can use it to bound the probability of error given a_j.
Well by definition, the probably of error given a_j is simply the probability of the union of all the possible error events of the a_j goes to a_j prime over all possible j prime, not equal to j.
This by the union bound is less than or equal to the summations of the probability that a_j goes to a_j prime.
That's just using union bound.
And now I can sub that expression over from there.
This is the same as..So the summation is over j prime not equal to j times Q of the norm of a_j prime minus a_j over 2 sigma.
Now let me write the summation in a different way.
I'm going to write the summation over all possible distances which belong to the set of distance, times K_D of a_j, times Q of D over 2 sigma.
Where the set D is the set of all possible distances from a_j.
Ok?
And K_D of a_j is the number of points at distance D from a_j.
That's just a straightforward change of variables.
Now if you look at this expression, then Q of B over 2 sigma basically behaves like an exponential, for an algebra use of the argument.
So recall that Q of X is like half E to the minus X squared over 2, for X much larger than 1.
So what you are really seeing here is that you have a sum of a bunch of exponentials, each written by this term, K_D of a_j.
Now if you think about the argument being large, then when you have a sum of exponentials, the term with the smallest exponent will dominate, because they are all decreasing exponentials.
So this term can be written as approximately K_min of a_j times Q of d_min over 2 sigma.
So what I am doing is I'm only picking up one term from this summation.
So far, we have a strict upper bound here, so this summation is a strict upper bound on the probability of error given a_j, But now what I am doing is I'm only going to keep one term in the summation, the term which has the smallest exponent here.
So I'm looking at the smallest value of D in this set of possible distances from a_j
So you're just looking at the nearest neighbor
You're looking at essentially the nearest neighbor, geometrically speaking.
And this approximation actually works quite well in practice.
It's not a bound on the probability of error given a_j, but it's an approximation.
And why did I do this?
Well, if I want to look at the probability over all error, what's that going to be?
It's going to be the average over all possible a_j's of probability of error given a_j.
Now, so I want to take an average of this quantity.
So this is a constant.
So I will just take the average over this, and that's going to be K_min of the constellation, which is the average number of nearest neighbors, times Q of D_min over 2 sigma.
This is approximate here.
So this is an approximation that will be used, and it's a very useful approximation, and it is known as the Union Bound Estimate.
It's no longer a bound on the probability of error, it's an estimate.
And in fact, there is a homework problem where you will be showing that the Union Bound Estimate is in fact exact for an M-PAM constellation.
And I will let you think why that is the case.
I was going to do it, but then I realized it's a homework problem, so you might as well spend some time on it.
So the last thing that I wanted to do today is find a lower bound on the probability of error.
So if I look at probability of error, it's a union of bunch of the events
[INAUDIBLE]
Yes
[INAUDIBLE] That union should be with a_j prime
The union should be with a_j -- yeah.
It's not what I have?
So I'm taking a union over all possible events, but a_j's confused with a_j prime
[INAUDIBLE] a_j going to union j prime not equal to j, a_j prime
Oh, I see
I think you need parentheses around the --
Brackets around the --
[INAUDIBLE] another set of parentheses behind the event a_j going to a_j prime.
Because that's the event you were talking about there.
At least that's [INAUDIBLE]
So you are saying that --
Put parentheses after the u
After the u.
Like this?
Yeah, right.
That's the event
Right.
That's what I meant.
OK, fine.
Fair enough.
OK, so basically, the lower bound is actually quite simple.
All I'm going to do is only take one event from that union.
I'm only going to take one point, which is the minimum distance from a_j.
So probability of error given a_j is greater than or equal to probability that a_j goes to a_j prime, where now a_j prime is the nearest neighbor of a_j.
And this we know from PAM analysis is simply Q of d_min over 2 sigma.
So this is a strict lower bound on the probability of error, and it has the same exponent as the Union Bound Estimate.
Of course, if I want to find the overall probability of error, I can just take an average of this.
Since this is fixed, it's going to be the same quantity.
So far what we have is a strict upper bound on the probability of error, which is this quantity here, a union bound estimate, and we have a lower bound on the probability of error.
In the next lecture, we will be looking at how to use these bounds to compute a probability of error for small signal constellations, and quantify the performance trade-off of the probability of error versus the EbN_0 and so on.
I think this is a natural point to stop.
It's almost time now.
So the idea behind the encoder is increase the minimum distance at the cost of spectral efficiency.
The idea behind the decoder was the following.
You have a received signal, y.
And the job of the decoder is to find x hat, which belongs to the signal set so that you minimize your probability of error.
And the probability of error x is not equal to x-hat.
That's the probability of error for your decoder.
And we saw a bunch of equivalence rules -- we said that the minimum probability of error rule, which is this rule here, is the same as the maximum a posteriori rule.
And this just follows by the definition.
This was the same as the maximum likelihood rule.
And here we made one assumption that all the signals in your signal set A are equally likely.
And this was shown to be equivalent to your minimum distance decision rule.
And in order to show this equivalence, we use the fact that the noise was White and Gaussian.
So this assumes equi-probable signals.
And this assumes AWGN channel.
OK.
In fact, the minimum distance decisions has very nice geometrical properties.
We basically said that the decision regions had a particular shape.
They were convex polytopes.
And they were also known as Voronoi regions.
However, finding the exact expression for the probability of error seemed quite tedious so we decided to settle with bounds.
And we developed some bounds for the probability of error.
And the main bounds were -- we had a strict upper bound, which was the summation.
OK.
This is a strict upper bound.
Well, the inequality here, say it's a strict upper bound.
We also had a union bound estimate.
Because this is a sum of various Q functions, this is still a very cumbersome expression to evaluate.
So we wanted a simpler expression.
So we then approximated this by K_min of aj times Q of d_min over 2 sigma.
OK, and now if we take the average on both sides, what we get is the probability of error is approximately equal to K_min of the constellation, which is the average number of nearest neighbors, times Q of d_min over 2 sigma.
And this expression is known as the union bound estimate.
We call it an estimate because it's not a bound anymore.
We have made an approximation going from here to here.
So this is what we came up with last time.
Throughout the analysis today, whenever we have a probability of error expression, we will be estimating it by the union bound estimate.
So this union bound estimate is quite important.
Are there any questions on this?
OK, so today we will start by introducing the notion of effective coding gain.
And we'll use the symbol gamma sub f for that.
What do we mean by effective coding gain?
First you have to decide what region you're operating in.
It's defined for both bandwidth-limited regime and power-limited regimes.
So let us start with the power-limited regime.
Remember in the power-limited regime, the trade-off we care about is the probability of bit error versus Eb over N_0.
The baseline scheme is 2-PAM.
And for 2-PAM, we showed two lectures ago that the probability of bit error is given by Q of root 2 Eb N_0.
OK, it should be -- so now the idea is to plot your probability of bit error as a function of Eb N_0.
Eb N_0 is always plotted in dB on the x-axis.
The probability of bit error we also plot on a logarithmic scale.
So this is ten to the minus six.
This is ten to the minus five, ten to the minus four and so on.
The Shannon limit is at -- this is the Shannon limit -- is at minus 1.59 dB.
OK?
For the power-limited regime, the performance of 2-PAM, which is given by this expression, is here.
And we basically go to say that when the probability of bit error is ten to the minus five, the EbN _0 that you require is 9.6 dB.
This is a nine.
So what's the idea behind the effective coding gain?
Well, suppose I give you a certain constellation, OK, a certain signal set.
And you find the probability of bit error for that, and you plot it.
And it comes out to be something like this.
So this is the performance for some given signal set A.
This is your 2-PAM.
And now you want to quantify how much better is the signal set over the baseline system.
Now, so in other words you want to look at the gap between the two curves.
Now clearly the gap is going to be a function of the probability of bit error, right?
But the basic idea now is you want to fix a certain probability of bit error because if you're designing a system, the probability of bit error is something that the system tolerates and is going to be fixed throughout the analysis.
So in this course, we'll be fixing it at ten to the minus five.
You fix this probability of bit error to ten to the minus five, and you look at how much a gap you have between the two curves.
And this gap is going to be your effective coding gain in dB.
So in other words, effective coding gain, I can write here, is the gap between a given signal set and concordant system, 2-PAM, at a fixed probability of bit error.
OK?
So now let us try to quantify this effective coding gain using the union bound estimate that we have here.
So if I have a certain signal set, A, then I know from the union bound estimate that the probability of error is given by K_min of A times approximately Q of d_min over 2 sigma.
So since I want that expression in terms of probability of bit error, I will use the fact that probability of bit error is probability of error per symbols or the number of bits per symobl.
Since I have endpoints in my signal set, I have logM bits per symbol.
And this is then K_min of A over logM base 2 times Q of d_min over 2 sigma.
So I'm going to rewrite my probability of bit error expression now in this -- in fact the right side in this quad.
K_b of A times Q of root 2 gamma_c of A times Eb over N_0.
So I'm just going to define the right hand side in this way.
And I'm going to relate gamma_c of A and K_b of A to the parameters I have on the right hand side there.
And why do I want to do it this way?
Because I want to get an expression as close as possible to the performance of an uncoded 2-PAM system because that way things will be easier to do if I want to calculate the effective coding gain, OK?
So how do the parameters relate?
Well, K_b of A is K_min of A over logM to the base 2.
And this is the average number of nearest neighbors per information bit.
If I compare the arguments inside the Q function, what I have is 2 gamma_c of A times Eb N_0 is the argument d_min over 2 sigma squared, which I will write as d_min squared over 4 sigma squared.
So remember sigma squared is N_0 over 2.
So if I simplify this expression, what I will end up getting is gamma_c of A is d_min squared over 4 Eb.
Gamma_c of A is known as the nominal coding gain.
It's not the same as the effective coding gain, which is the distance between the two curves.
OK, are there any questions so far?
So far what I have done is given a constellation A, I can find two parameters.
I can find the nominal coding gain, and I can find the average number of nearest neighbors per information bit.
And using these two parameters, we will try to get a handle over the effective coding gain.
So that's the idea.
So how can we plot -- so given this constellation A, we want to plot the probability of bit error curve like this.
But instead of plotting the exact curve we will be plotting this curve here, which is the union bound estimate of your probability of bit error.
So we will always be plotting the union bound estimate.
So how do we do that?
So we'll again start with a curve like that.
Probability of bit error as a function of Eb N_0.
This is in dB.
The y-axis is also in dB.
Sorry, not in dB but on the log scale.
And so on.
This is the Shannon limit at minus 1.
59 dB.
This is the 2-PAM performance.
So if we want to plot this union bound estimate, you will be doing two steps.
First, we'll be plotting this Q function, a curve corresponding to the Q function.
And then we'll be scaling it.
So if I just want to plot this Q function, how will it compare to this curve here?
Well, you're just scaling the argument inside the Q function, right?
But now because we are plotting the x-axis on a dB scale, it simply means that we are translating to the left by an amount of gamma_c in dB.
Is that clear?
Well, let's see.
What we are doing is we are going to scale the x-axis by a certain constant.
So this means that we will have to scale x-axis here.
But, our Eb N_0 is being plotted on a dB scale, right?
So multiplying in the linear scale corresponds to an addition on the logarithmic scale.
An addition on the logarithmic scale simply implies a shift on the x-axis so that's the constant shift on the x-axis.
I'm going to plot it here.
This curve is going to be parallel to the original 2-PAM curve to the best of my abilities.
OK, so this is what we get as your Q function.
Now we are going to scale the y-axis by this factor here, K_b of A.
But remember the y-axis is also on a logarithmic scale, rather than the linear scale.
So multiply [INAUDIBLE] by a factor of K_b of A implies a vertical shift by a certain factor.
So I'm going to do a vertical shift here.
So let me make that darker now.
So this is what I end up getting.
OK, now note that the distance between these two curves is not fixed anymore.
If the slope is steeper, then this distance is small, meaning that this distance here is large.
On the other hand, if the slope is going to be smaller here, then this distance is large.
So this means that a distance between these two curves is no longer fixed.
And basically what we're saying here is that if the probability of error is large, then the number of nearest neighbors matters much more.
But if the probability of error is going to be quite small, then the more important factor is your exponent, how fast the slope decays as opposed to the number of nearest neighbors, OK?
This distance here is your effective coding gain.
And this constant horizontal distance here is your nominal coding gain in dB.
Are there any questions on this curve?
[INAUDIBLE] on the right of four 2-PAM because these are the log
So so what I'm really assuming is that gamma_c of A is greater than 1.
So if I'm adding it, then I'm going to shift to the left, right?
OK, so a couple of remarks.
I'm just summarizing a few remarks I made while plotting the curve.
First is the log-log scale is very convenient because any multiplicative factor simply involves a translation along the x and y directions.
The second point is that the accuracy we have in the effective coding gain in this way is determined by the union bound estimate because that's all we are plotting.
We are not really plotting the exact curve.
And the third point is that we have a rule of thumb.
Because the distance between the curves is not fixed, it's still too tedious to find the exact numerical value of effective coding gain.
So what we can find is the value of the nominal coding gain and K_b of A exactly, given a constellation.
So we want to have a rule of thumb for the effective coding gain.
And the rule of thumb is that the effective coding gain is given by the nominal coding gain in dB minus 0.2 log2 of K_b of A.
This is in dB.
What this means is every factor of two increase is K_b results in a loss of 0.2 dB in the effective coding gain.
OK, and this is our probability of bit error of ten to the minus five.
So that's the region that we will be doing our analysis in.
So this rule of thumb is quite helpful.
Are there any questions?
[INAUDIBLE]
This rule of thumb?
Well, it works quite well in practice so you want to use that rule
[INAUDIBLE]
Right.
If things are clear, let us do some examples.
So let us start with a 2-PAM for examples.
The first is 2-PAM system.
What's the effective coding gain going to be for this constellation?
It's going to be 0 dB, right?
It just follows from the definition.
I mean the effective coding gain tells us how much gap we have between this new constellation and 2-PAM.
If your new constellation is 2-PAM itself, we don't have any effective coding gain.
What about the 4-QAM?
It's going to be still 0, because a 4-QAM is a Cartesian product of two 2-PAM constellations.
And we argued last time that there is no coding gain if you use a Cartesian product.
If you want to verify it using the framework that we have just developed, say we have four points.
This point is alpha, alpha.
This point is minus alpha, alpha, minus alpha, minus alpha, and alpha minus alpha.
The nominal coding gain is given by d_min squared over 4 Eb.
The minimum distance is 3 alpha between any two points.
So we have 4 alpha squared.
Then what's the energy per bit?
Well, the energy per symbol is 2 alpha squared.
Then we have two bits per symbol, so the energy per bit is alpha squared.
So we have 4 alpha squared, and so that's 1.
So we do not have any nominal coding gain.
K_b of A, what's that going to be?
Well, we have two nearest neighbors per symbol, but we also have two bits per symbol.
So the number of nearest neighbors per bit is going to be one.
So your nominal coding gain is 1, K_b of A is going to be 1.
And so we do not see any coding gain for the 4-QAM system.
Now let me do a slightly different case.
Let me start with a 4-QAM, but I remove two points.
I remove this point, and I remove this point.
And I only keep the two points here.
So I have a constellation, say, A prime, which only has two points, one point here and one point here.
So this point is alpha, alpha, and this point is minus alpha, minus alpha.
Any idea what the effective coding gain will be for this case?
I think it's the same
It's still zero, right?
Yeah, because it's saying that's 2-
Exactly.
All this is a 2-PAM constellation embedded in two dimensions.
It's 2-PAM along this line, right?
So I cannot hope to have a higher coding gain for this constellation over the original one.
I mean I told you the story that we can take a subset of points, and we can increase the minimum distance by throwing away some points.
But one has to be careful in that analysis.
There could be examples where you're strictly improving the minimum distance.
Note that the minimum distance here is going to 8 alpha squared now.
OK, so the minimum distance is strictly improved.
But what happens to be your energy per bit?
Well, the energy per symbol is 2 alpha squared.
But you only have one bit per symbol, so the energy per bit is 2 alpha squared.
So your nominal coding gain is d_min squared over 4 Eb, which follows from the relation we have here.
And d_min squared is 8 alpha squared.
4 Eb is also 8 alpha squared, so the nominal coding gain is 1, or 0 dB.
The average number of nearest neighbors, well, we only have one nearest neighbor, and we have one bit per symbol, so that's -- and this is A prime by the way -- is going to be 1.
And so the effective coding gain is still 0 d
[INAUDIBLE]
So probability of symbol error as a function of alpha has definitely decreased, right.
But the trade-off we care about is probability of bit error as a function of Eb N_0.
In other words, this curve is still here.
It's not changed.
I just shifted right by reducing my spectral efficiency.
OK?
So let's do one last example just to prove that it's not always the case that you get nothing by removing points.
OK, so the contellation that I have in mind is the one I introduced last time.
We'll start with a 3, 4 Cartesian product of 2-PAM.
OK, so in three dimensions it's a cube, and the 3,4 Cartesian product will have a point on each vertex of the cube.
And we only keep four points.
So the points we keep are these four points.
OK?
So I write B, the coordinates are alpha, alpha, alpha, minus alpha, minus alpha, alpha, minus alpha, alpha, minus alpha, and alpha, minus alpha, minus alpha.
So these are the vertices.
These are the coordinates of the four points here where we have the standard x, y and z-axis, which I'm not drawing because it will just look confusing.
So now we'll see that for this particular signal set, we do have a coding gain, OK?
Let's find the minimum distance.
The minimum distance here is what?
It's the distance along a diagonal.
And the diagonal is of length to -- each side of the cube is off-length to alpha.
So the diagonal is of length to alpha.
So d_min squared is 8 alpha squared.
What is the energy per bit going to be in this case?
Well, what's the energy per symbol now?
[INAUDIBLE]
3 alpha squared over 2 is the Eb, right?
We have 3 alpha squared units of energy per symbol.
We have four points, so we have two bits per symbol.
So the energy per bit is 3 alpha squared over 2.
So now if I look at my nominal coding gain, that's going to be d_min squared over 4 Eb.
d_min squared is 8 alpha squared.
4 dB is 6 alpha squared.
So that's 4/3.
On a linear scale and on a dB scale, this is going to be?
You guys should remember your dB tables
0.2?
12.
You have to multiply by ten.
So actually I think it's 1.3, more like 1.3 dB if you look at the last significant digit.
OK, so what's K_b of A going to be in this case?
Or K_b of B I should say.
I'm calling the signal set B here.
So what is K_b if B going to be?
[INAUDIBLE]
3 by 2, right?
You have three nearest neighbors per point, for each point has three nearest neighbors.
They're all equidistant.
And you have two bits per symbol.
OK?
So if you work out your effective coding again using this rule of thumb here, take gamma_c of A in dB and subtract 0.2 log2 you will see that the effective coding gain is approximately 1 dB.
Are there any questions?
OK, so it might seem that this is still not a very impressive number.
Yes, you had a question
What was it about this constellation that gave us coding gain?
Was is the face that we went to higher dimensions?
Right, so usually you do get a coding gain if you trade-off minimum distance with spectral efficiency.
So that's usually the case.
So that's why I presented you an example of that case.
It's just that this was a very special case where you end up with a 2-PAM constellation to start with.
OK, so the comment I was going to make is that 1.2 dB still might not be like a very impressive number.
I mean after all the hard work, you just get 1 dB effective coding gain.
So the question to ask at this point is are there any constellations that have much higher coding gains?
In particular, other constellations that come close to the Shannon limit at all.
And we'll look at one interesting class of signal sets, which are known as orthogonal signal sets.
And you probably saw these examples back in 6.450, but we're just reviewing it again here.
And these signal sets have a property that as you increase the number of dimensions, you come arbitrarily close to the Shannon limit.
So the definition of the signal sets is your A -- it's a collection of signals aj, where j goes from 1 to M such that the inner product between ai and aj is E A times delta_i,j, meaning that if i is not equal to j, the two signals are orthogonal.
And if i equals j, the energy is E(A).
Geometrically, we have two points.
The two points can be thought of as two points on the two axes.
So this is a case when M equals 2.
When M equals 3, the three points will lie on those three corresponding axes.
So this is the case when M equals 3 and so on.
So generally when you have M points, your signal spatializes in M dimensions, and each point can be thought of as a point on one of the axes.
So if you look at the distance between any two points, it's the norm of ai minus aj squared.
I could simplify this simply as the norm of ai squared plus the norm of aj squared because the end product is going to be zero.
OK, now ai squared is going to be E(A).
aj squared is going to be E(A).
So this is 2E(A).
So what we're saying here is that every point is equidistant from every other point, and the square of the distance is 2 E(A).
OK?
So in other words, the average number of nearest neighbors is always going to be a M minus 1.
So let us calculate the nominal coding gain for this constellation.
That's going to be d_min squared over 4 Eb.
d_min squared is 2E(A) over 4 Eb.
Well, the energy per bit is the energy per symbol, which is E(A) over the number of bits per symbol, which is log M to the base 2.
And this I'm going to write here is 1/2 log M to the base 2.
So as I increase my M, my number of dimensions, my nominal coding gain goes to infinity.
OK?
What do we expect for the effective coding gain?
Will it also go to infinity?
It approaches the Shannon limit
Well, we don't know whether it approaches the Shannon limit as of yet, but we know it cannot go to infinity.
It's upper bounded by the Shannon limit.
Right?
So the effective coding gain cannot really become unbounded, and what's really happening here?
[INAUDIBLE]
Right, but why does the effective coding gain stay bounded and the nominal coding gain blow up?
[INAUDIBLE]
Right, the number of nearest neighbors also increases with M, right?
Not just the nominal coding gain.
So let's write that down.
If I look at the number of nearest neighbors per information bit -- yes?
These don't have 0 mean, right?
Right, they don't have 0 mean.
That's a good point
Let's see, what happens to the mean as M goes to infinity?
If we look at what happens to the mean, it will be 1 over M. You find the mean of this guy, it's 1 over 2 times the distance here times distance here is 1 over 2 E(A), E(A).
OK?
If you look at three dimensions, it's 1 over 3 E(A), E(A), E(A).
So in M dimensions, it's going to be 1 over M times all these coordinates.
If we look at the mean squared, it goes to 0 as M goes to infinity.
So that's a good point.
So the mean does go to 0, and so we approach the Shannon limit.
So we have K_b of A over logM What's the number of nearest neighbors?
We saw it was M minus 1 over log M. And that goes to infinity as M goes to infinity.
OK, so the average number of nearest neighbors per information bit also go to infinity as M goes to infinity.
And if we look at the mean -- just to write down the comment that was made.
If we look at the mean of this constellation, that's going to be 1 over the number of points times root E(A) root E(A) and root E(A).
So there are M coordinates.
So that's going to be the mean if you believe this is the coordinate axis here.
And if you we look at the norm of M(A) squared, then it's going to be E(A) over M. And as M goes to infinity, this goes to 0.
And because the mean goes to 0, it's not unreasonable to expect that the performance does approach to the ultimate Shanon limit.
OK, so so far we have looked at K_b of A and gamma_c of A.
If you look at the effective coding gain, it is still bounded.
It's always bounded.
It is not clear at this point what really happens to it.
And in fact, it's not quite straightforward to actually show what happens to the effective coding gain.
If you have taken 6.450, then Professor Bob Gallager really goes into the details of proving how the effective coding gain for this signal set indeed does approach the Shannon limit.
But it can be shown that the effective coding gain does approach 11.2 dB as M goes to infinity.
The main trick here is that union bound is usually quite weak if the probability of error is small.
So we replace the probability of error just by 1, instead of the union bound, at some point.
And if you do that change, then things work out nicely.
But in this course, we will just assert that the orthogonal signal sets do achieve the ultimate Shannon limit.
And you can see 6.450 notes for the proof.
So that's the assertion that we are making right now.
So let's step back and take a look at what's happening so far.
We have a class of signal sets which are conceptually quite easy to describe.
They are just orthogonal signal sets.
They are not too hard to analyze, and they asymptotically approach the ultimate Shannon limit.
So this seems quite nice.
So why not we just implement them?
Now it turns that these are not very common in practice.
They are not implemented as often as you might think when you first look at them and because they have some drawbacks.
And there are a couple of drawbacks which make them very less appealing than what you might otherwise think.
And first of all, what's the spectral efficiency for these signal sets?
0, right?
It goes to 0.
And how does it go to 0?
It's 2log M to the base 2 over M. As M goes to infinity, this goes to 0.
And in fact, it goes to 0 quite sharply.
Now that's fine, but if you want to approach Shannon limit, our spectral efficiency should indeed go to 0.
But if you're looking at the system design problem, what do you really want?
You have a certain effective coding gain that you have in mind, and suppose you are presented with a list of several different signal sets that achieve this effective coding gain.
And your job is to pick one of these possible signal sets.
Which one will you pick?
Well, there are two things you will look for, right?
You will look for the spectral efficiency that these different signal sets require or achieve in order to get this effective coding gain.
And the higher the spectral efficiency the better.
And secondly, you will also want to look at the implementation complexity.
You do not want something which really takes a lot of time to decode.
OK?
So it turns out that because the spectral efficiency goes to 0 so sharply, it's not very surprising that there are other codes that have the same effective coding gain but higher spectral efficiency.
So there are binary codes that have a higher spectral efficiency for same effective coding gain.
And so they would be a natural choice over the orthogonal signal set.
Yes?
[INAUDIBLE] high-spectrum efficiency to mean bandwidth?
Right, because in practice, bandwidth is never really infinite, right?
So if you are achieving the same -- the first objective is to have a certain effective coding gain.
If you do have that, you want to pick something which has a higher spectral efficiency.
And indeed, the subject of chapters six and eight is to come up with binary codes that have a much better performance than the orthogonal signal set for a given effective coding gain.
Chapter six deals with the the block codes, whereas chapter eight deals with convolutional codes.
And chapter seven basically develops all the finite field theory that you require to study convolutional codes.
So the point is, in the subsequent lectures we will be focusing a lot on improving the performance over orthogonal signal sets.
The second point I mentioned was implementation complexity.
Now one particular way of implementing orthogonal codes, which you saw in the first homework, was using this idea of Hadamard transforms, right?
You have a Hardamard matrix, which is an M by M matrix.
The rows of the Hadamard matrix are your orthogonal signal sets, which are elements aj.
And at the receiver, what you would do is when you get the signal y, you simply multiply by the Hadamard matrix and look at the coordinate which is the largest.
So this is something you did in the first homework exercise.
And how many computations did you require for an orthogonal signal set of size M?
[INAUDIBLE]
Sorry?
OK, so I meant M being the size of the orthogonal signal set
Yeah, [INAUDIBLE]
Right
2 to the M by 2 to the
If it is 2 to the M by 2 to the M, it is M times 2 to the M. But there are M orthogonal signal sets, then it will be M log M, where M is the number of signal sets.
So that's the number of computations that you would require.
Now, the question to ask is, is this fast or is this slow?
Is this too many computations or is this too little computations?
And if you're looking in the power-limited regime, what we really want is to see the complexity per information bit because we normalize things per information bit.
Now, how many bits are sent when you have orthogonal signal set of size M?
We have log M bits, right?
So this quantity is actually exponential in the number of transmitted bits that we are sending, and so in fact this is quite slow.
Or in other words we are saying it is we require M times 2 to the M computations, where M is the number of information bits that we are sending
So if M is the size of your [INAUDIBLE] then you have log M bits?
Right
So M log M over log M is
Why are you adding by M?
You need M log M operations to decode a symbol
To decode a symbol
And each symbol gives you log M bits.
So you have M operations per bit
Well, when you send, say, log M bits, right, you can only send one of the M possible symbols.
And when you want to decode the symbol, you would end up recovering log M bits.
So you should not be dividing by M here
You're dividing by log M because you want to do it per bit
All right.
You're right.
So you're dividing by log M. You have M log M bits.
You're dividing by log M, so you still have --
You have M calculations per bit.
[INAUDIBLE]
So we are sending -- so I need this many computations to get log M bits.
So is this expression exponential in log M or not?
So that's all I was saying
I just put it another way that you could divide by log
M for information, right.
OK, are there any questions?
I mean we are both agreeing to the fact that this is too many computations than you would want.
And there exist other decoding algorithms for which the number of computations is much smaller than exponential.
OK, so let's next look at the bandwidth-limited regime.
Well, actually before that I wanted to mention a couple of other signal sets as well.
So there are two other important class of signal sets which are related to the orthogonal signal sets.
The first is this class of simplex signal sets.
And the other class is bi-orthogonal signal sets.
So the idea behind simplex signal sets goes back to the observation that was made that we have a non-zero mean for this signal set here.
So if we do, we can indeed subtract of the mean and get a performance which is better than the orthogonal signal set.
So in particular if I subtract of the mean here, I get a signal set, which is like this.
This is the simplex signal set when M equals 2.
When M equals 3, I'm going to subtract of the mean from this plane here.
The points lie on this particular plane.
And what I end up with is an equilateral triangle.
So this is the case when M equals 3.
What do you think M equals 4 would look like?
A tetrahedron right here.
That's a simplex signal set.
So basically, it's also not too hard to write an algebraic expression for these signal sets.
E of A prime is going to be E of A minus M of A.
You subtract off the mean from your orthogonal signal set.
If I want to write it in terms of inner products, the inner product between ai and aj is given by E(A) if i equals j.
And it's I believe minus of 1 over M minus 1 times E(A) if i is not equal to j.
So this is the inner product between ai and aj.
And I mean there are also many properties of this simplex signal set, but I believe that that's going to be in your next homework exercise so I'm not going into too many details.
What' the spectral efficiency in this case going to be?
I have a question.
So what is a simplex?
Is a simplex just a reference to anything with [INAUDIBLE]
Simplex signal set is derived from the orthogonal signal set by subtracting off its mean
Right, but in general simplex means anything [INAUDIBLE]?
I'm not sure about that.
What's the spectral efficiency going to be?
[INAUDIBLE]
Well, almost
You use one dimension
Right, you just use one dimension.
OK, and there are other properties that you'll be looking at it in the homework.
Now the idea behind this --
[INAUDIBLE]
This is 2 log M over [INAUDIBLE].
So the idea behind the bi-orthogonal signal set is you start with an orthogonal signal set, and for each point, you also put the negative signal of it.
So in addition to this, you will have a point here, and you will have a point here.
So the case in two dimensions, M equals 2, you will have four points like this.
So in this case you are increasing the number of signal points by a factor of two.
But this does come at a price, right?
Because if you're increasing the number of signal points, the number of nearest neighbors is also going to increase by a factor of two.
In this case, you have two nearest neighbors now.
So you have both of the effects going on.
I mean ultimately, all the signal sets, as M goes to infinity, are going to approach the Shannon limit.
So again, but they do suffer from the same kind of drawbacks that we saw for the orthogonal signal sets.
So these are more of theoretical interest as opposed to real, practical implementation points.
Any questions?
Minus [INAUDIBLE]
It's a bit involved to show.
I think you'll be showing it in the homework exercise or for an exam problem at one point.
You basically start with the orthogonal signal sets here, look at the inner product here, and use the relation between the simplex signal sets and the orthogonal signal sets to do a subtraction of the mean.
And then you can derive in a product expression.
OK, so let's now move on to the bandwidth-limited regime.
OK, so the main difference between -- yes?
[INAUDIBLE]
You mean here?
For each --
How far can you [INAUDIBLE]?
A factor of two.
I said a factor right?
So if you have M points in the orthogonal signal set, you have 2M points in the bi-orthogonal signal set because for each --
In the orthogonal sets you have 3
And in this you have 6.
So let's look at the bandwidth-limited regime.
The trade-off we care about is the probability of error for two dimensions as a function of SNR norm.
OK?
The baseline scheme here is your M-PAM scheme.
And we showed a couple of lectures ago that the probability of error for two dimensions is approximately 4 Q times root 3 SNR norm.
OK?
So let's plot the performance graph.
So on the x-axis I plot SNR norm.
On the y-axis I plot Ps of E again on a log-log scale.
This is ten to the negative six, ten to the negative five, ten to the negative four, ten to the negative three and so on.
The intercept is at 0 dB, this point here.
So this is your Shannon limit in the bandwidth-limited regime.
Now your performance curve is going to be something like this for the M-PAM system.
And we want to basically quantify now the effective coding gain in this regime.
So it's the same story as in the power-limited regime.
We will start off with the probability of error using the union bound estimate, which is K_min of A times Q of d_min over 2 sigma.
Now Ps of E is the probability of error for two dimensions.
Assuming we have N dimensions here, it is two times the probability of error over N. So this is probability of error per symbol.
We are dividing by the number of dimensions and multiplying by two because it is for two dimensions.
And this is going to be 2 times K_min of A over N, times Q of d_min over 2 sigma.
So now let's do the same trick that we did in the power-limited regime.
We will write Ps of E be approximately -- and instead of the right hand side, I will write it as Ks of A times Q of root 3 SNR norm.
But because I have a factor of four up there, I will also multiply and divide by 4.
That's not going to change anything
[INAUDIBLE] per two dimensions or by symbol
Per two dimensions.
And for bandwidth-limited regime, we normalize everything by two dimensions
But for N band we used it for symbols, right?
The calculations represent --
No, it was for two dimensions.
OK?, so now let's compare the two expressions.
We have Ks of A, which we'll define by two times K_min of A over N. And this is the number of nearest neighbors per two dimensions.
And I can write 3 times SNR norm is d_min squared over 4 sigma squared, OK?
Actually, I was missing this factor of gamma.
I knew I was missing something.
It's not just 3 SNR norm.
It's 3 times this nominal coding gain times SNR norm.
So 3 times SNR norm times gamma_c of A here.
So this is d_min squared over 2N_0.
So what do I have for the gamma_c c o A is d_min squared over 6 N_0 times SNR norm.
Remember SNR norm is SNR over 2 to the rho minus 1.
It's normalized signal-to-noise ratio.
So this is the d_min squared times 2 to the rho minus 1 over 6 times N_0 times SNR.
But N_0 times SNR is just Es.
So this is 6 times Es.
So this is the expression you have for the nominal coding gain.
So now again, given a signal set A, I can find those two parameters, the nominal coding gain and the number of nearest neighbors per two dimensions.
And I can use those to plot on the Ps of E versus SNR norm curve.
Again, the exact same story.
If I have a certain nominal coding gain, I will simply shift my curve around the x-axis by that factor as a pure translation.
And then because I have a factor of Ks A over 4, I'm going to plot -- how did that expression go?
This expression on the top.
Remember this curve here is 4 times root 3 SNR norm, right?
So the multiplicative factor I have is Ks(A) over 4.
So I will shift my curve up by that factor.
And I will get something which is like this.
And that's my union bound estimate for this new constellation A.
The distance here is gamma effective.
This distance here is gamma_c of A.
Are there any questions?
So now we also have a similar rule of thumb as in the power-limited regime.
I will write that rule of thumb here.
We have that gamma effective is going to be gamma_c in dB minus 0.2 times log2 times Ks of A over 4 in dB.
OK, so again, a factor of 2NKs will decrease your nominal coding gain by a factor of 0.2 in dB.
Are there any questions?
Well, so I mean this is the end of chapter five.
We still have like ten minutes remaining, so I thought I would give you a preview of the next chapter.
Professor Forney will be coming next class, and he will be starting chapter six all over I believe.
This chalk is much nicer to erase than that other one.
So in chapter six, we'll be looking at the binary block codes.
OK, and what is the main architecture of these codes?
Well, for an encoder, remember we have K bits coming in.
And we pass it now through a binary block code.
And what we get out is a binary sequence of length N. OK?
So x belongs to c, where c is a subset of binary sequences of length N. So we start with K bits, and we convert them to N bits, where N is going to be greater than or equal to K. Once we have this sequence of N bits, what we end up doing is we pass it through a binary signal constellation.
So I'm writing in as binary signaling.
And what we get out is S of x, which is a sequence of N symbols.
Each symbol can either take value minus alpha or alpha.
So this binary signalling is actually quite a trivial step.
Basically, the relation is S of 0 is going to be alpha, S of 1 is going to be minus alpha.
If I have a 0 coming in, I will produce an alpha.
if I have a 1 coming in, I will produce a minus alpha.
So this part here is trivial.
All our energy will be focused on will be to find a good code.
Given a sequence of input bits, we want to find a good binary code in order to optimize the minimum distance and so on.
So that's the architecture that we will be using for the binary linear codes.
Both chapters six and eight will be only focusing on this part.
And this binary signalling scheme, for the most part, will be implicit in our architecture.
So at this point, one might ask is there anything to lose in having imposed this type of architecture?
Remember, the nice thing about this architecture is we have coding, which is going on here, and we have modulation that is happening here.
And the modulation step is quite simple.
So we have a separation between coding and modulation.
OK?
And the question is, is this the right thing to do?
Now, it turns out that if we are going to live in this binary world where we are only constrained ourselves to sending binary signals over the channel, this is in fact an economical structure.
There isn't much improvement we can get, and that is quite obvious, right?
But it turns out that if you are going to go for non-binary signaling, particularly in the bandwidth-limited regime, there is a cost to be paid for this kind of separation.
In fact, in the 1980's there was this whole idea of turbo-coded modulation or trellis-coded modulation.
And the idea there was to combine coding and modulation in one step.
So this is not optimal for non-binary signalling, but's it's OK for binary.
So we will be focusing on this type of an architecture.
The other issue is OK, nobody told us that you can only send binary signals over the channel.
If you want to achieve the Shannon limit, when we derived the capacity expression -- or we just stated it but you can derive it.
We said that the Shannon limit is always less than log2 1 plus SNR.
And we never said that we can only send binary signals over the channel.
So it could be that this is some inherence of optimality in this type of architecture.
Why send binary signals in the first place?
Again, it turns out that the regime that we are interested in here is the power-limited regime because the spectral efficiency can never be more than two bits per two dimensions, right?
Remember, rho is 2K over N here, and K is going to be less than that.
So we are inherently in the power-limited regime.
And so a natural thing to do in order to understand how much fundamental loss we have in imposing this type of binary constraint over the signals is to not look at this expression, but to find the best possible performance we can have if we constrain ourselves to binary signals over the channel.
If we impose a certain signaling constraint, we can find another fundamental limit on the spectral efficiency.
So the point being, the best possible binary code can only achieve this upper bound here.
Now, this upper bound has to be less than log2 1 plus SNR, right?
Because the Shannon code assumes the binary signaling is a special case.
So it turns out that this expression is actually quite tedious to write out.
I'm not even sure if the closed-form expression exists, but you can calculate this numerically.
It's just some kind of a convex optimization problem.
And now, to understand how much loss we have, we can compare the two expressions in the power-limited regime.
So I'm going to plot the curves for the two cases.
I'm going to plot the spectral efficiency on the y-axis as a function of Eb N_0.
You can easily convert from SNR to Eb N_0 using the relations we discussed in chapter four.
Now we first plot the Shannon limit.
The ultimate Shannon limit is minus 1.59 dB.
This is going to be 0 dB here.
And if I'm plot the Shannon limit, it will be some code like this.
It increases with Eb N_0.
And minus 1.59 dB is 0.
This point here is 1.
Some point here is 2.
So actually I want to draw this horizontal asymptote at two when rho Shannon equals 2.
So this is my rho Shannon.
Now, I can now also plot rho binary from this expression here.
I can never exceed two bits per two dimensions.
So I had plotted this horizontal line.
So as Eb N_0 goes to infinity, the most I can get is two bits per two dimensions.
If Eb N_0 is smaller, I cannot do better so I will be doing worse and worse, and I will always be below this line.
Ultimately, it can be shown that I will have the same Shannon limit here.
So this is rho binary.
And the main observation here is if I'm in the power-limited regime, which is say, in this part of the curve -- like, say around one bit per two dimension -- the gap here is quite small.
In fact, this gap can be shown to be at most 0.2 dB.
So if I want to achieve a certain spectral efficiency, if I impose a binary signaling constraint, the additional Eb N_0 that I require when the spectral efficiency is 1 dB, one bit per two dimensions, is at most 0.2 dB.
So there is not much to lose by imposing a binary signaling constraint.
Said in different words words if you had the best possible binary linear code, followed by a binary signaling constellation, the most you can lose is 0.2 dB.
And so this type of an architecture does make sense.
There are a lot of details that the next powerpoint will be to kind of formalize the notion of a binary block code, then specialize it to the case of binary linear codes and so on.
Unfortunately, the first thing to do is to start with some finite field theory because there's a lot of finite field algebra involved in this algebraic block codes.
So how many of you have seen finite field theory before?
OK, if you haven't seen it, don't worry about it.
We'll be starting right from the basics.
But I think I will be stopping here for today because I don't want to go into those details today.
Chapter six, problem set three, problem set two solutions.
OK. Good morning.
Nice to see you all again on this mild day.
I want to start off by thanking Ashish greatly for giving the last three lectures.
I have very high confidence in Ashish, and I'm sure he give you excellent lectures and was able to answer all your questions, right?
But if you feel dissatisfied at any point, you'll have an opportunity to ask the question again today.
We will continue now in chapter six.
I understand Ashish got up to section six point one already, so that makes my life a little easier.
And we'll just continue to plow ahead through the notes.
You have as handouts today chapter six, the outgoing problem set, the solutions to the incoming problem set.
So does anyone have any comments or questions or suggestions as we go forward at this point?
Let's say I'll do a very quick review of everything so far, at least what we need to proceed, so you'll have a chance to ask questions along there if you like.
Anyone have anything to say?
Anyone want to say thanks to Ashish?
All right.
Great.
OK.
So a very quick review.
We first said we were going to work with a continuous time Additive White Gaussian noise channel, but we quickly reduced it to a discrete time additive white Gaussian noise model.
Vector model, Y equals X plus N, where the vectors may go on as long as you like.
And implicit in here is we have some kind of power limitation expressed by various -- I know I've probably introduced too many parameters in the early part of the course.
But you'll see them all in the literature, and I thought you might as well see the relation between them all at once.
The noise.
The main thing about this is it's an independent, identically distributed noise, also independent of X, and it's characterized simply by its variance per symbol, or average power.
Again, there are various notations that we can use for its power.
OK.
However you express the power of the signal of the noise, you're going to get some signal-to-noise ratio out of that, as long as you use consistent the same units for signal and noise.
And so the key parameters of this channel turn out to be just two.
One is the signal-to-noise ratio.
And the other is something which in the discrete time domain we might think of as a data rate, rho, the number of bits per two dimensions.
But we were at pains to show -- and the reason we measure it in bits per two dimensions is that it converts directly to the spectral efficiency, or we sometimes say, the nominal spectral efficiency, or the Nyquist spectral efficiency.
Basically, this is the spectral efficiency if you assume that you only use the nominal Nyquist bandwidth.
And both in 450 and here it was shown that you can really get as close to the nominal bandwidth as you like, its sharp or roll off as you like, by using filters.
Very sharp filters or very sharp digital filters that today we can really realistically get effectively to the Nyquist bandwidth.
So this is a good measure of nominal spectral efficiency, which is an extremely important parameter, the spectral efficiency in continuous time.
And then the Shannon limit is expressed in this very simple form that we can never do better in data rate or nominal spectral efficiency than log of 1 plus SNR.
And the units are bits per two dimensions.
So there's an awful lot about parameters and units in the first part of the course, and sometimes it's a little overwhelming and confusing.
But in my experience in engineering is, it's extremely important to get the units right.
It helps you in thinking clearly.
It helps you to focus on the right things.
And so in the first couple of chapters, there's a lot of emphasis on getting the units right.
For instance, bits per two dimensions.
I could simply assert to you that things are always to come out better if you do things per two dimensions.
This basically has to do with Gaussian distributions being simpler in two dimensions than they are in one dimension.
So you know you can get closed form integrals in two and four and so forth dimensions that you can't get in one and three and so forth dimensions.
So you know, the geometrical formulas for the volume of the sphere and so forth are much simpler in even dimensions than they are in odd dimensions.
This all, I think, has to do with the fact that really a one-dimensional complex Gaussian variable, or in general, complex Gaussian variables, are in some sense mathematically simpler than real Gaussian variables.
For instance, the Q function is a very ugly thing.
In two dimensions, you can closed-form integrals with probability of error.
So there's a lot of clues that we really want to think in terms of pairs of real dimensions.
All right.
And then we introduced a couple more parameters.
And I understand Ashish got the questions that I get every year.
Why do we introduce both of these things?
SNR norm, which is SNR normalized by 2 to the rho minus 1, that's motivated directly by the Shannon limit formula.
Or Eb over N_0, which is SNR divided by rho.
You know, these are both more or less equivalent to each other.
From this Eb over N_0 is just 2 to the rho minus 1 over rho times SNR norm, so why do we introduce both of them?
And there's not an intellectually very strong argument to introduce both of them.
Because one is easily translated into the other.
For fixed nominal spectral efficiency rho, there clearly is just a one-to-one translation.
We could use either one.
If you graph something versus Eb over N_0 or versus SNR norm, it's just a matter of shifting it by this factor.
They're going to be exactly the same graph with the 0 dB point in a different place, according to where 0 dB is.
Philosophically, of course, SNR norm, which is sometimes called the gap to capacity, is exactly a measure of how many dBs away are we from the Shannon limit.
Measuring things on a log scale in dB.
And so 0 dB is always the Shannon limit point with SNR norm, because this statement translates into SNR norm is greater than 1, which is 0 dB.
So here the focus is always, how far are you from capacity?
How far are you from the Shannon limit?
And that really is the modern view.
In the early days of coding, the view was, well, how much coding gain can we get over no coding?
And as we'll see, Eb over N_0 is often a very convenient parameter to use when we're focusing on coding gain.
In fact, for binary linear block codes, which is what we're talking about in chapter six, we get an extremely simple expression that Eb over N_0 it's just kd over N, where N, k, d are the basic parameters of a block code.
If you don't know those yet, you will by the end of this lecture.
And Eb over N_0 is what was put forward in the early days, when the principal coding application was coding for deep space communications.
It makes sense in the power-limited regime.
In the power-limited regime, we showed that this essentially goes -- this is rho log2 over rho, so up to a factor of natural logarithm of two, these are the same, almost independent of the spectral efficiency, as well.
And so this is a very natural thing to use, especially in the power-limited regime.
In the bandwidth-limited regime, as rho gets large, then these things become very different.
SNR norm always keeps us in the neighborhood of 0 dB.
This thing goes up to 10 dB, 20 dB, 30 dB, and so forth.
Nonetheless, you'll see in some literature, people continue to measure against Eb over N_0 maybe I would say, just because they don't know any better.
Anyway, since I started writing these notes about eight or nine years ago, I've been advocating SNR norm.
It's more and more widely used in our business, in the coding business.
Or equivalently, one always shows nowadays how far are you from capacity, and that's what SNR norm is about.
And you can always translate this into this and this into this by this simple formula.
So that's why we introduce them both.
Eb over N_0 is traditional.
SNR norm is more of the modern gap to capacity viewpoint.
Any questions about that?
Because I understood that Ashish got a number of questions about that.
Yes?
And the probability of error you mentioned [UNINTELLIGIBLE] is natural with SNR model
Well, this is a slightly different point, actually.
So we go on from this to talk about the power-limited regime, which we defined more or less arbitrarily as the regime where rho is less than or equal to two bits per two dimensions, and the bandwidth-limited regime, which is where rho is larger.
And at this point, I simply assert that it's better to do everything per two dimensions in the bandwidth-limited regime and to do everything per bit in the power-limited regime.
And the reason for this is basically long practice and intuition and experience that things do work out better, and this is the proper normalization.
But I think at this point in the course, with your background, this is only an assertion, all right?
So I simply say, this is the way we're going to do things in bandwidth-limited and power-limited.
For most of the rest of the course, we're going to be in the power-limited regime.
Then we'll come back to the bandwidth-limited very late in the course.
So you can now forget this assertion for a while.
Yes?
[INAUDIBLE]
Well, I'm trying to focus on that here.
That if two systems have different rhos, then you should take that into account in their comparison.
And when you see charts of a rate 7/8 system, or a rho 7/4 system versus one that's rate 1/8, those are very different regimes, and it probably isn't fair to compare two systems, just in terms of their probability of error versus Eb over N_0 at two different spectral efficiencies.
What is more fair is to compare them in terms of their gap to capacity.
You can get more powerful codes with more error-correcting capability the lower in rate that you go.
And so you should start from the point of view that two systems with different rho are incomparable, and then using this you can say, but, you know, if we can get within 1 dB of the Shannon limit with both of them, then they're both approximately equally powerful.
That would be the modern point of view.
But it's really apples and oranges if they have different rates, different rhos.
Yes?
[INAUDIBLE] If [UNINTELLIGIBLE] systems modulation [INAUDIBLE].
Then is it fair -- because if one of them has coding --
If none of them have coding, well --
No, just modulation
So what shall I say to that.
In the power-limited regime, an uncoded system is simply binary modulation or 4-QAM, and there aren't different systems to compare, really.
There's only one baseline system.
As you go up into the bandwidth-limited regime, then it is fair to compare systems of the same class, like M by M QAM.
That's a simple uncoded system.
Now there, rho keeps changing.
rho equals 2, rho equals 4, rho equals 6, rho equals 8.
Or you can easily find intermediate rates that are uncoded.
And there you find that this normalization makes all these systems comparable.
In fact, we saw that we got a universal baseline curve for the bandwidth-limited regime, which was independent of rho, for, say, the class of 4 by 4, by the class of M by M bandwidth-limited QAM systems.
I'm sorry.
I got off a plane at nine o'clock last night, so I may not be totally coherent today.
But all of these systems have exactly the same baseline curve up to the approximations we've made that their exactly four nearest neighbors and so forth.
But on a curve of probability of error per two dimensions, per QAM symbol, versus SNR norm, we get a universal curve.
So that indicates they really are comparable.
They form a class where the performance of the class is independent of rho.
And that's sort of typical of the bandwidth-limited regime.
Two systems that differ, that basically use the same code with smaller constellation or a larger-based uncoded constellation are going to be directly comparable, and are probably going to have the same gap to capacity.
Going to be just as far away from the Shannon limit.
But we don't have that phenomenon in the power-limited case.
Other questions, or shall I proceed?
OK. Again continuing what I hope will be a quick review, but I don't want to go any faster than you're comfortable with.
We get this baseline curve for 2-PAM or 2 by 2 QAM, same thing.
So this is the baseline of Pb of E on a log scale, where we typically go down to ten to the minus six, ten to the minus five.
Here's 0 dB.
This goes down through about 9.6 dB here or about 10.5 dB here.
And the ultimate Shannon limit is over here.
The ultimate Shannon limit for very low rho, as rho goes to 0, is about minus 1.6 dB.
And we get expressions.
0 dB is the Shannon limit at rho equals 1.
At rho equals 2, it's up about 1.8 dB.
Shannon limit for rho equals 2, and so forth.
And anyway, so we see as a function of rho, we can measure the gap to capacity at different probabilities of error and see how much coding gain is, in principle, possible.
And then for a coded system, we can put that on here.
The effective coding gain is the distance between here and here.
At some target probability of error.
It's going to differ according to probably of error.
You found a way of getting a good, rough-cut estimate, at least for not very complicated codes called the union bound estimate for any signal constellation in any number of dimensions.
So if we have a signal constellation A, which consists of M points in N dimensions, so forth, basically found that we could, in the power-limited regime, get an approximate expression from considering pairwise error probabilities that's very simple.
And I used this notation.
Q of the square root of some coding gain of the constellation times 2 Eb over N_0.
If it's just 2-PAM then the coding gain becomes 1.
This is the average number of nearest neighbors per transmitted bit.
And so the whole thing reduces to this expression in terms of couple of parameters.
the principal parameter is the nominal coding gain, which is the minimum squared distance of A over 4 times the energy per bit of this constellation A.
So we really only need to know this kind of normalized measure of goodness of the constellation.
And K_b of A Is the average number of nearest neighbors.
I forget what we call it.
What is the numerator here?
K_min of A, which itself is an average, over the number of bits that we're actually sending, which is log2 of the size of A.
So basically we only need to know a couple of parameters of this signal constellation.
Its minimum square distance is very important.
Energy parameter, which we choose to make the energy per bit, or so that we get this expression.
And the average number of nearest neighbors per bit that we transmit.
OK. And our best example so far is the tetrahedral constellation, where we basically pick every other point from the 4 simplex.
Maybe I should do that in a different color.
I don't think that's a different color.
Nope.
Anyway.
You know this quite well by now.
And if we do that, we find that normalizing everything, this is 4/3 or 1.25 dB.
And every point has three nearest neighbors, and we're sending 2 bits, so this is 3/2.
So then we get this handy dandy engineering rule to make a quick plot of the union bound estimate.
Given that we decided to put this on a log-log scale to start with, all we have to do to plot this expression -- if we're given, as we always are, the baseline curve, which is simply Q to the square root of 2 Eb over N_0.
How do you convert this curve to this curve on a log-log plot?
Well, you simply move it to the left by the coding gain.
And you move it up by the log of whatever K_b is.
So if the dominant effect is moving it left, in this case, by 1.25 dB -- I'm going to wind up getting about this curve, so I won't draw it again -- then we move it up by a factor of 3/2, we developed a rule of thumb that said a factor of two is basically going to cost you about 0.2 dB.
Around ten to the minus five.
This is just based on the slope of this baseline curve, as long as we're somewhat in that region.
So this will cost us about 1/10 of a dB, so we'll get an effective coding gain of about 1.15 dB.
So graphically, we take this curve bodily.
We move it over 1.25 dB, and we move it up by a factor of 3/2.
And what we'll find is the effective coding gain is thereby reduced to, I estimate, about 1.15 dB.
And this is all just engineering rules of thumb.
Nice template.
I sometimes say you should fill out a copy of this baseline curve, cut it out, put it in your wallet for the duration of this course, because you'll have the opportunity to make this kind of calculation again and again.
All right.
So that's union bound SNR.
For simple constellations, the union bound estimate is very accurate.
So this is a very good way to proceed, from an engineering point of view.
We want to write a paper?
You know, it's a little late to write a paper on the performance of the four simplex signal constellations, but if you wanted to, and you wanted to have one graph in that paper, you would have the graph of -- well, you would actually probably compute the exact error of probability, either by analysis or by Monte Carlo simulation, you would put that in there.
But in your first draft of the paper, you would put in union bound estimate and you would find that wasn't far off.
OK. Any questions on that?
This has basically got us up to chapter six.
So in chapter six now, which Ashish got into last time.
This is basically about binary linear block codes.
And well, we first just start talking about binary block codes.
That's what Ashish did last time.
We basically take 0 and 1 as our binary alphabet.
We take a blocks of length n. Sorry.
Are we using little n or big N?
Little n, where n is called the block plank.
So we take the set of all binary symbols of length n, and we're going to convert this to real n space.
How do we convert it?
Component-wise, by the standard 2-PAM map.
We map 0 into plus alpha, 1 into minus alpha.
So this maps into plus or minus alpha to the n. So the entire universe of possible code words that we have is this set of 2 to the n real n-tuples, which are simply the vertices of an n cube of size 2 alpha, obviously, right?
Just like this.
We have eight possible vertices in 3-space.
They obviously all have the same power.
They all lie on the surface of a sphere of squared radius n alpha squared.
So they're not only the vertices of a cube, they're vertices that are spread on an equal energy sphere.
And the whole idea is that our code is going to be some subset of this, and the code will map under the same map, which we call S, into some subset of the vertices of the n-cube.
OK?
And again, our favorite example is the tetrahedron.
For instance, if we take the code as being 0, 0, 0, 0, 1, 1, 1, 0, 1 1, 1, 0, those for binary three-tuples, this maps into T, the tetrahedron.
OK?
Well, we get this signal structure, which we've already found the coding gain.
It has a little bit of coding gain.
We actually accomplished something.
All right.
So that's the basic idea that we go through in 6.1.
That's our favorite example.
And Ashish also shows you that, you know, you might think this is an awfully restricted way of designing constellations.
But when we're talking about low spectral efficiencies, by going through the capacity calculation, we can assure ourselves that in principle, this is not going to be very sub-optimal for it.
Capacity is about the regime where n gets very large for very long block codes.
We don't lose much in potential coding gain, or in the Shannon limit, equivalently, by restricting the input output input alphabet to be just these two numbers, plus or minus alpha, I should say, rather than using the whole real line as an input, as long as the nominal spectral efficiency is less than or equal to one bit per two dimensions.
And the exact numbers -- at rho equals 1, you lose about 0.2 dB, in principle, using the Shannon limit as your guide.
And for lower rho, it just goes to 0.
And you did this last time.
OK.
So it's a very reasonable, and obviously attractive from an implementation point of view, way of designing signal constellations.
And again, this is basically all we're going to be talking about for the majority of the course, is signal constellations like this.
But of course as we get to codes that are thousands of bits long, or perhaps even infinitely long, as in the case of convolutional codes and trellis codes, sometimes you forget that we're really talking about just putting points in Euclidian signal space.
Because we're going to be very abstracted back into the binary domain.
But in this course, we're always thinking of codes as means of designing signal constellations.
Of course, codes are used far more widely than just for signaling over the additive white Gaussian noise channel.
I've sort of packaged this course as a search to get to capacity, we have added the additive white Gaussian noise channel.
First of all because this corresponds very closely to history.
Within this package, we can talk about all of the principal classes of codes that have been developed to date, and we can compare them in some performance terms.
How close do they get to capacity on the additive white Gaussian noise channel.
So you get most of what you would get if you took a communications-free view of coding theory.
You know, people are interested in codes for computer memories, for God knows what, lots of other things.
And for many of these other applications, you are going to be interested in the same class of codes.
Basically, you're going to want to maximize the distance between code words for a given rate, which has to do with the size of the code.
And so you're going to be interested in the same codes.
Putting it in the framework of getting to capacity on the additive white Gaussian noise channel gives a motivation, gives a very nice story, because over 50 years, we were able to get to capacity, and gives it a real communications flavor.
Some of you, I'm sure, are here without any interest in communications whatsoever.
You simply want to know about coding.
You will still get that story, but you'll get it in this nice package.
That's why I do it this way.
OK.
I'm about to get into new stuff.
End of review.
Any questions?
So you must have done a great job.
Everyone understands perfectly.
All right.
So now let's talk about, as I said, binary linear block codes.
When you see the word "linear," that's a signal that there's some algebra ahead.
And so this is the very first point at which we get into what's called algebraic coding theory.
The algebra will be extremely simple at this point, not to worry.
And what are we doing here?
First thing we do is to identify 0 and 1 with the binary field, which I'm always going to write as F2.
The older way of writing this is GF(2), for Galois field with two elements.
They mean exactly the same thing.
Nowadays most people write just F2.
Now we have [UNINTELLIGIBLE], we can write this blackboard F. And OK.
Step one.
We've identified our alphabet with a field.
So algebraically, this is a field.
Some of you know exactly what I mean by that.
Others don't.
We'll come back to this again.
Informally, I would say a field is simply something where you can add, subtract, multiply, or divide.
Our best examples of that before now have been the real and complex fields.
There is a more formal definition of that.
In this case, we have only two elements.
So let me just write down the tables, which you all know, regardless of your background.
What's the addition table for this field?
Well, 0 is the additive identity.
So we know that 0 added to anything gives itself, so that gives us three of the entries of this table.
What do we put down here?
There's really only one choice to satisfy one of the axioms of the field, which is that under addition, the field must form a group.
In particular, that means that all elements are invertible.
That means that each row or column has to be a permutation of the group elements.
And so the only possibility here is to put in a 0.
And well, we are forced, if we're going to make 0 and 1 into a field to find an addition table which is the table of mod2 addition.
You've had this stated as an axiom before.
You can derive it just from the fact that 0 needs to be the additive identity, and then we need to put a 0 in here in order to get invertibility.
Under multiplication.
What is the multiplication table?
Well, the additive identity is also a nullifier under multiplication in any field.
So 0 times anything is equal to 0.
1 times anything is equal to itself.
So that completely determines the multiplication table.
Again, just the table of mod2 multiplication.
So if I'd done this axiomatically, I would have given you the axioms of field.
Then I would show that under this binary addition operation, this multiplication operation, we satisfy the axioms.
I'll do that as we get into chapter seven.
OK.
So now we have 0 1 to the n, n-tuples, binary n-tuples, we will now regard as n-tuples of field elements, F2.
And these will be regarded as vectors in a vector space.
Well, I'll just say that F2 to the n is a vector space, which clearly has 2 to the n elements.
Now again, informally.
What's a vector space?
A vector space is always over a given field.
In this case, it's going to be over the binary field, of course.
F2.
The given field is called a scalar.
Just like vector space over the reals, the scalars are real numbers.
Here the scalars are elements of F2.
And what do we have to have to make something algebraically a vector space?
We have to have that the addition of two vectors is well-defined and gives another vector, and that the multiplication of a vector by a scalar is well-defined and gives another vector in our vector space.
All right?
OK.
So how are we going to define that to addition?
If we want to add two n-tuples -- again, you aren't going to see anything here that you haven't already seen in some other context.
What do we do?
We add them component-wise, using the component-wise rules of field addition.
OK.
So we just add two vector n-tuples, component by component.
We obviously get some result, which is itself a binary vector or an F2 vector.
No problem there.
Except that -- well, all right.
As long as we're talking about all possible n-tuples, the result is certainly in the vector space, right?
We add two binary n-tuples, we get a binary n-tuple.
So the result is in F2 to the n. This is going to be the key test when we get to subspaces of F2 to the n. OK. And multiplication is even easier.
How do we define multiplication by scalars?
Well, we only have two scalars -- 0 and 1.
So 0 times any vector is going to equal the all 0 vector.
Again, you can view that as just component-wise multiplication of everything by 0, and since 0 times anything equals 0, we're always going to get the 0 vector.
All right?
So is the 0 vector in the vector space?
Well, yes.
If we're talking about several n-tuples, it of course always is.
And one times anything, again, we can just do this component-wise, and it just gives itself.
Trivially.
So since this was a vector in the vector space, this is certainly a vector in the vector space.
OK.
This seems pretty trivial so far.
But what's a binary linear block code?
Again, focusing on the linear, is a -- now I'll give a formal definition.
it's a subspace of F2 to the n for some n called the block length.
All right.
What do I mean when I say a subspace?
I mean subsets of the elements of a vector space that itself forms a vector space.
So in this case when I say subset, I mean a set of binary n-tuples.
OK?
That itself forms a vector space.
OK. What are the components of that?
To check that it forms a vector space, let's see.
Multiplication by scalars is, again, easy to check.
If it's going to be a subspace, then the all 0 -- so it has to contain the all 0 vector in order that when I multiply by the scalar 0, I get another element of this subspace.
Multiplication by 1 is always trivially satisfied.
If I start off with a set, I multiply by 1, I'm going to get the same set.
So let's check whether the elements of a subspace are closed under vector addition.
What do I mean by that?
I mean if you add two elements of the of the subspace together, you get another element of the subspace.
That's the key property to check.
Key property -- we can write that as closure under vector addition.
Which is also called the group property.
It means that just under addition, under vector addition, the set of elements that you have forms a group.
You add any two elements, you get another element of the subset.
The example is our favorite example so far.
Let's take this little code, and I'm going to ask.
Is that a subspace of F2 to the three?
So does this equal subspace of the set of all binary three-tuples?
Anyone care to hazard a guess whether it is or isn't?
It is
It is?
Why?
It has the all 0 vector
It has the all 0 vector, first of all.
Good
[INAUDIBLE]
And it's closed under addition.
Now how might we see that?
You could, of course, just take all pair-wise -- you could form the addition table of these four elements, and you would find that you always get another one of these elements.
For instance, 0, 1, 1 plus 1, 0, 1, is equal to 1, 1, 0.
In fact, you easily see that you take any two of these, add them together, you get the third one.
If you call this a, b, and c, a plus b plus c equals 0, addition is the same as subtraction, because we're in a binary field.
So that means that a plus b equals c, a equals b plus c, c equals b plus a, whatever you like.
And of course, if you add 0 to anything, it's trivially closed under that.
All right.
So it is.
It satisfies -- it's all you've got to check.
A more abstract proof of this would be, this is the set of all even-weight three-tuples.
If I add even to even, I'm going to get even.
So of course my result is going to be another even-weight three-tuple, therefore in the set.
That's the more algebraic proof.
All right.
Suppose I just add 0, 0, 1 to all of these things.
I'll get the set of all odd-weight n-tuples.
Add any odd-weight n-tuple to this.
Let me take the C prime, which is the set of all odd-weight n-tuples.
Is that a vector space?
It doesn't have 0, and in fact, it's not even closed under vector addition.
All right.
If I take in C double prime is equal to 0, 0, 0 0, 1, 1, 1, 0, 1 and I stop there, is that a subspace?
No.
Because?
Not closed.
For instance, if I add these two together, I would get that, and that's missing.
OK.
So actually, everything is much simpler when we're talking about finite fields.
All the finite dimensional vector spaces consist of a finite number of elements.
It's easier than real and complex vector spaces.
There's no analysis involved.
So forth.
All right.
So a binary linear block code is in a subspace of F2 to the n. What are some of the key algebraic facts we know about vector spaces from our study of linear algebra, which I assume all of you have had in some form or another?
What's a key algebraic property of a vector space?
What's the very first property?
All vector spaces have a --
[INAUDIBLE]
Norm?
No.
Dimension.
Dimension, that's where I'm going.
What's the significance of dimension?
[INAUDIBLE]
The definition of dimension is the number of generators in any basis.
So we're talking about generators of the vector space, or a set of generators which form a basis.
If you think we have the same properties here in the case of vector spaces over finite fields, well, we probably do.
But let's see how it works out.
So we're talking about things like generators, basis, dimension.
These are all closely interlinked properties.
Again, the fact that everything is finite here gives us very elementary ways of addressing all of these concepts.
We don't have any geometry yet.
We don't have norms.
We don't have distances.
We don't have angles.
We'll talk about that in a minute.
Here we just have a set that basically has these two properties.
It contains the all-0 vector, and it's closed as the group property.
It's closed under vector addition.
All right.
So the first property is that the all-0 vector is always in the subspace, which I'll represent by C, meaning code.
So when I talk about a code now, I'm talking about a binary linear block code, which by definition is a vector space, a subspace of F2 to the n, where n is the code length.
All right.
Let me try to find a set of generators for the code.
All right?
If somebody gives me a code, they say, this is a binary linear block code.
Let me see if I can find a set of generators for it.
If I find three generators, then I'll know the dimension is three.
That's basically where I'm going.
I'll state this a little bit more formally as we go ahead.
So suppose I'm given a code.
I know it's a binary linear block code, so I know it has the all-0 element.
So how might I go about finding a set of generators?
Let's just take a greedy algorithm.
All right?
Suppose the code contains only the all-0 vector.
Is that a subspace?
What's its dimension?
0.
All right?
So that's a very trivial vector space, but it's a vector space.
Satisfies the axioms.
Suppose it is not the trivial code.
That means it has more than the all-0 vector.
So I take as my first generator any non-zero vector.
OK?
I can always do that.
Don't have the axiom of choice involved here, because everything is finite.
So I'm going to take g1 to be any non-zero vector.
Now I've got a generator.
And how many code words does it generate?
I want to take the set of all binary linear combinations of all the generators that I have so far.
At this point, the binary linear combinations are 0 times g1 and 1 times g1 And that just gives me this word and this word.
So now I have counted for two words with one generator.
Could it be that that's the whole code?
Sure.
The all-0 vector and any non-zero vector together form a one-dimensional subspace.
That's all you can get from one dimension.
So I could be finished here now.
But if I'm not finished, there's still more code words that I haven't accounted for.
Then I greedily pick a second generator.
So this is now, let's say, any vector not generated by g1.
So I have a branch here.
Either I've finished or I can pick another generator g2.
Now with g1 and g-two, how many vectors can I generate?
Let me take all binary linear combinations.
So a binary linear combination is any vector of the form alpha1 g1 plus alpha2 g2 where these are both scalars.
And therefore this can be 0 or 1, this could be 0 or 1.
So what have I got now?
I've got 0, g1 g2 and g1 plus g2 I've got four binary linear combinations of two generators.
Could that be the whole code?
Certainly.
At this point, again, consider our standing example.
I start out what I take as my first generator.
Let me take g1 equal 1 0, 1, g2 equals whatever, 0, 1, 1.
Then if I take all binary linear combinations of these two generators, I'm going to get the whole code, right?
These four code words.
They can all be expressed in this form.
Or I'm not done, and then I have to pick g3.
And how many binary linear combinations are there of g1, g2, and g3?
Eight.
Are they all necessarily in my subspace?
Yes, by the fact that the subspace is closed under scalar multiplication.
Alpha g1.
Alpha1 g1, alpha2 g2 alpha3 g3 are all in the subspace.
Any vector addition of any of these scalar multiples is in the subspace.
So I get now eight possible elements of the subspace, and I either may be done or not.
Continuing in this way, I get some number gk of generators, just by picking greedily the next one until I'm done.
All right?
When I'm done -- so I have to stop.
Why do I have to stop?
All right.
Let's look at the size.
At each point here, this accounts for two code words, this for four, this for eight, this for 2 to the k. How many binary n-tuples are there?
2 to the n. All right?
So I clearly can't find more than n generators.
More more than n independent generators, in the sense that the set of all the binary linear combinations are distinct.
All right.
So k is, at most, going to be n. So I will stop in a finite number of steps.
I'll stop at some number k. When I've stopped, that's because the code consists of all binary linear combinations of these k generators, and therefore has size 2 to the k. So the only possible size of a subspace is 2 to the k for k less than n. A power of 2 where the power is, at most, n. So any subspace besides the subspace -- I'm repeating myself.
OK.
So this means I found a basis.
Does this mean that all possible bases of any subspace have the same size?
Well yeah, it must.
I mean, I've proved now that any subspace has to have this size 2 to the k for some k. So obviously, if I go through this process, no matter how I choose my generators, I could choose any pair of these non-zero n-tuples as my generators.
So that would be a legitimate basis.
Take any two out of these three.
But it's always going to take exactly two of them, right?
Why?
Because the size of this subspace is four.
Two to the two.
So if I go through this process, I'm always going to come up with k generators, where k is the log to the base 2 of the size of this code that I was given, which I was told was a linear code, meaning it's a subspace.
So I'm somewhat free to choose the generators, but I'm always going to come up with k of them if the code has size 2 to the k. So a basis is a set of k linearly independent k-tuples, where linear independence has the same meaning here as you're accustomed to, meaning that all linear combinations of the k generators are distinct.
So I get 2 to the k distinct elements of the code.
And I say the dimension of the code is k. In this case, basically it's the size of the code is 2 to the k, then its dimension is k. Has to be.
And all basis have k generators in them.
And there are, in general, many ways to pick them.
All right?
So just by considering this greedy basis construction algorithm, I basically find the size of any subspace is a power of two, and the power is equal to the dimension.
And any basis is going to have that cardinality.
Are you with me?
This is a pretty simple proof.
And I call this an n, k binary linear block code.
n being the length.
That just means that every code word is an n-tuple over the binary field.
And k is the dimension.
So an n, k binary linear block code has size 2 to the k. That's 2 to the k distinct code words.
Main example is this guy.
Easy?
Any questions?
I think this is clear
Can n use the number of code word, I take like 2 to the k?
n is something I specify a priori as the length of every vector in the code.
In other words, it has size 2 to the k, and it's a subset of the set of all binary n-tuples, which I write like that.
In other words, the elements of the code are binary n-tuples.
If I write them out, each one has length n. OK?
We're good?
What are some other examples?
The simplest codes you can think of is, first of all, an n, 0 code.
That means a code with dimension zero, has size what?
1.
And what does it consist of?
[INAUDIBLE]
Yeah.
So this is the so-called trivial code, just containing the all-0 word.
Has to mention 0.
It is a binary linear block code, but it's not much use for communication.
So but nonetheless, we include that in this family.
Another trivial one is n, n. What's that?
[INAUDIBLE]
F2 to the n, right.
What's its size?
2 to the n. That means it has to contain all distinct binary n-tuples.
So this is called the trivial code.
This is called the universe code.
Contains the entire universe of binary n-tuples.
Let's get some slightly less trivial ones.
n, 1.
What would that be?
An n, 1 code.
What's its size?
2.
What does it consist of?
The 0 word, and?
Any other non-zero generator.
And that's correct.
This can be 0 and any generator, two words.
In communications, where we want to maximize the distance, in some sense, between the two code words, what is g always taken as?
0,1's, right.
So if it's in particular the all-0 and the all-1 word, which I might write as a vector of 0's and a vector of 1's, this is called the repetition code.
The binary repetition code of length n. It either gives me a 0 and I repeat it n times, or gives me a 1 and I repeat it n times.
So whenever you see n,1, you can pretty well assume it's the repetition code, though it might be any pair 0, g. And n, minus 1.
This is an interesting one.
Again, while this could be a lot of things, in communications, whenever you see this, this will always be the set of all even-weight n-tuples.
In other words, the set of all n-tuples with even parity such that if you sum up all of the components of any vector, mod2 equals 0.
OK?
So this I will call the single parity check, or more briefly, the SPC code, or the even-weight code.
That's equally good.
And here we maybe should do a little bit more work.
Say, is this in fact a subspace?
Does it include the all-zero code word?
Yes, all-zero has even weight.
The sum of any two even-weight code words, an even-weight code word, an even-weight n-tuple.
Yes.
As here.
This is an example.
This is the three, two SPC code.
OK. Why is this dimension n minus one?
[INAUDIBLE] Is every code word orthogonal to the one-vector?
OK. That's an excellent answer.
It's a little advanced for us right now.
I'm looking for an elementary argument
[INAUDIBLE]
OK.
So you're saying we have a set of generators that looks like this.
Is that what you're saying?
You are correct.
And how many such generators are there?
There are n minus 1 of them.
I always like to find the most elementary argument possible.
I think the most elementary argument here is that the number of even-weight n-tuples is equal to the number of odd-weight n-tuples, and together they form the set of all n-tuples.
So exactly half of the n-tuples are even weight.
That means there are 2 to the n minus 1 of them, 2 to the n over 2, and therefore, the dimension must be n minus 1.
But perhaps this is just as elementary a proof.
Well, however you do it, you'll find that there are 2 to the n minus 1 of them, or that here is clearly a set of generators.
It might take a few more lines to show that every even-weight code word is a linear combination of this particular set of generators, but it's certainly true.
All right.
So these four classes of codes, these two entirely trivial ones, these two which are actually somewhat more interesting for coding purposes -- we've already seen, we can get a coding game with this length three, dimension two code -- are basically the simplest codes we can think of.
The simplest binary linear block codes.
Now we'll see them again and again.
They turn up.
All right?
So the whole course is going to be about finding 1's in between.
More complicated ones.
There's clearly more room to play.
For instance, if k is equal to half of n, which means that rho is equal to one bit per two dimensions, there's a lot of possibilities.
And so we're going to explore those possibilities
[INAUDIBLE] [UNINTELLIGIBLE]
Sure.
Let's take the 6,5 code generated by these five generators.
Well, it contains some odd-weight code words, obviously.
But it's not as interesting from a coding point of view.
It's not the only one, but it's the only one you'll ever see here.
All right.
Let's see.
One thing I didn't point out in the notes but probably should have here is, what is rho for an n, k binary linear block code?
How many bits can we send?
Suppose we take the Euclidean image of this.
It is going to have 2 to the k points, 2 to the k vertices of the n-cube, and so what is rho?
What is the rate in bits per two dimensions?
[INAUDIBLE]
Right.
The rate is basically k over n over 2, if you like, or 2 k over n bits per two dimensions.
You can send k bits in n dimensions, or 2 k over n bits per two dimensions.
And since k can't be larger than n, this is going to be less than or equal to two bits per two dimensions.
So again, we see we're definitely in the power-limited regime, and that we can really get any nominal spectral efficiency between 0 and 2 by choosing k and n appropriately.
All right.
So n and k determine the rate, determine the nominal spectral efficiency.
All right.
Let's now talk about things that I think are also very old hat to you.
I mean weight and distance.
But we begin to get into areas that show us that these vector spaces are very different from the real and complex vector spaces that we're accustomed to.
So what are we doing now?
We're starting to get into the geometry of this vector space.
The geometry is not Euclidean geometry, but it's Hamming geometry.
We define the Hamming weight of a vector as simply the number of 1's in v. So the Hamming weight of the all-0 vector is 0, the Hamming weight of the all-1 vector is n, and in general, the Hamming weight is somewhere between 0 and n. Just the number of 1's.
Pretty simple.
And given two vectors x and y, what is their distance?
The Hamming distance between x and y is equal to the Hamming weight of x minus y.
This is the standard way of converting a weight metric into a distance metric.
Or because addition is the same as subtraction in a binary vector space, we might equally well write this is as the Hamming weight of x plus y.
And more informally, this is simply the number of places in which they differ.
OK.
So if x and y are identical, then x plus y is equal to 0 and the distance is 0.
If they are complementary, y is the complement of x, then they differ in every place.
The sum will then be the all-1 vector, and the weight, the Hamming distance, will be n. And so again, the Hamming distance is somewhere between 0 and n, measures how different they are.
Clearly going to be important for coding.
It's going to translate directly into Euclidean distance under this standard 2-PAM map.
OK. Again, let's check that it satisfies the distance axioms.
I don't know how many of you have seen this, but let's see.
What are the distance axioms?
Strict non-negativity.
In other words, the Hamming distance between x and y -- that's a single comma -- is greater than or equal to 0, and equality if and only if x equals y.
That's what strict means.
So if we find the Hamming distance is 0, we can assert that x equals y.
We have, of course, symmetry.
The Hamming distance between x and y is the same as the Hamming distance between y and x.
Order doesn't matter.
And finally we have the triangle inequality, that the Hamming distance between x and z certainly can't be more than the Hamming distance between x and y plus the Hamming distance between y and z.
If x differs from y in only n1 places, and y differs from z in only n2 places, then clearly z can't differ from x in more than n1 plus n2 places.
So check, check, check.
This is a legitimate metric for defining a geometry on the space, and this is the one that we use on the space of all n-tuples.
But notice it's not all like the Euclidean Distance.
Now when we have a linear code -- let's combine these things.
When we have a linear code, we have a group property which is, let me write it this way.
If we take any code word and add it to any other code word, that's in the code.
And furthermore, c plus c prime is not equal to c prime plus c single prime, c plus c single prime, because we can -- well, I'll finish it.
Unless c prime equals c double prime.
Why is that?
We can do subtraction, cancellation.
Cancel c out from each side.
So if we add c to c double prime, we're going to get a different result from adding c to c prime, if c prime and c double prime are different.
So this implies that c plus C -- I write that as an abbreviation for the set of all c plus c prime, as a c prime runs through the code C. So this is the 2 to the k sums of the code plus any code word in the code.
Sorry if I don't write down all steps.
What is that going to be?
C. How did we conclude that?
[INAUDIBLE] [UNINTELLIGIBLE]
Perfect.
Did you all hear that?
By the group property, each one of these things is in the code.
By this argument, no two of them are the same.
That means I get 2 to the k distinct elements all in the code, that's got to be the code, because the code only has 2 to the k elements.
All right.
So if I add a code word -- in other words, if I write down the code -- 0, 0, 0, 0, 1, 1, 1 0 1, 1 1 0- and I add any code word to it -- say I add 0 1 1 to the code.
So let me just do one column of the addition table.
I get 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1.
I'll get the code itself again.
This has a very important property.
The set of Hamming distances C and c prime, as c prime runs through c, from any given code word C, it is independent of C. So I can start from any code word, measure the Hamming distances, the 2 to the k Hamming distances to all other code words, including C itself -- let c prime run through the entire code, I'm going to get a set of 2 to the k distances called the distance profile of the code.
And I claim that it doesn't matter which code word I start from.
I'm going to get the same distance profile regardless of where I start.
In other words, the set of all distances from the all-0 word here, which is 0, 2, 2, 2, is the same as the set of all distances from the 0, 1, 1, code word, which is 2, 0, 2, 2.
And the proof is basically that this is simply the Hamming weight of C plus c prime as c prime runs through C. What is this equal to?
So the proof is that this is equal to the Hamming weight of c prime as c prime runs through C. So the distance profile from any code word is simply equal to the weight profile of the code itself.
The weight profile of this code is 0, 2, 2, 2, Start from any code word, measure the distances to other code words, I'm always going to get 0, 2, 2, 2, 0, to itself, and the others.
Sounds sort of like the tetrahedron, doesn't it?
It's zero distance from a code word to itself and equal distance to all the other code words, in that particular case.
OK.
So again, everything is very elementary here.
The distance profile is independent of C and equal to the weight profile.
So this has an extremely important corollary.
What's the minimum Hamming distance of the code?
You might expect this is going to be an important code parameter
[INAUDIBLE]
The minimum Hamming distance between any two code words is going to be equal to -- I think you said it
The 0 [UNINTELLIGIBLE]
Non-zero is important here.
If the distance profile is equal to the weight profile, one of the weights is always going to be zero.
And that corresponds to the distance between a code word and itself.
All right.
If I go through all the other 2 to the k minus 1 distances, they're going to be weights.
They're going to be the distances from a code to all other code words.
And the minimum distance is simply going to be the minimum non-zero weight of the code.
For example, in this code, the minimum Hamming distance between any two distinct code words is going to be equal to the minimum distance from the 0 code word -- that's another way of doing it.
Since it's independent of C, we may as well take the base code word C to be zero.
And then what's the minimum distance to 0?
To the 0-code word?
It's the minimum weight.
From the 0 code word, distance equals weight.
So the minimum distance is the minimum weight of any non-zero code word, which for this code is two.
Now here the weight profile is 0, 2, 2, 2, 0.
The distance profile from any code word to all the others is 0, 2, 2,2 2.
This is always the distance to itself.
So minimum distance to other code words is always going to be two.
Furthermore, the number of nearest neighbors -- to go back and use chapter five terminology -- the number of nearest neighbors is going to be the number of code words that have that minimum weight -- in this case, three.
Still sounding a lot like a tetrahedron, isn't it?
This easy map between Hamming distance and Euclidean distance for this case and in general for all of our cases.
So corollary.
The minimum Hamming distance, which implicitly means between two distinct code words of C, is equal to the minimum non-zero weight of C, and the number of minimum weight code words is independent -- I'm doing this backwards.
From any code word, the number of code words that distance, let's call this d, is equal to the number of weight d code words.
Sorry, you probably can't see that.
All right.
So we get this symmetry property for codes that follows from the group property of the code that if we stand on any code word and look out, we're always going to see the same thing.
We have this constant.
It's actually easiest to see this when we make the map from the code to the Euclidean image of the code.
So the Euclidean image S of C of the code word is going to be some set of 2 to the k vertices of an n-cube of side alpha.
Let's talk about the Euclidean image of these properties.
If the minimum Hamming distance of the code is d, what's the minimum squared Euclidean distance between elements of S of C going to be?
Well, let's do it coordinate by coordinates.
Let's take two code words, C and c prime, let's say.
And let's suppose we have some Hamming distance between C and c prime.
That means that c and c prime differ in the Hamming distance, number of places.
So if we map this into the corresponding pair of vertices of the n-cube in Euclidean space, S of C and S of c prime, how many coordinates are these going to differ in?
It's going to differ in same number of coordinates, D_h.
If they don't differ, what's Euclidean squared distance in those coordinates?
0.
If they do differ, the Euclidean squared distance is 4 alpha squared.
So the Euclidean distance D_e between S of C and S of c prime is simply going to be 4 alpha squared times the Hamming distance between C and c prime, yes?
So I should say this is the squared Euclidean distance.
Why do we always talk about the squared Euclidean distance?
Because it's additive, coordinate-wise.
And the Hamming distance is additive, coordinate-wise.
So there's a nice easy map here.
So what does this mean d_min squared is going to be?
d_min squared of, let's say, S of C. This constellation that we've formed by taking the Euclidean image of c. The minimum square distance between points in S of C is just going to be 4 alpha squared times d, where I don't think I ever -- d equals min Hamming distance.
And we're always going to talk about n, k, d as the three key parameters of a binary linear block code.
n is the code length, F2 to the n, k is the dimension, d is the minimum Hamming distance.
So by going into this Hamming geometry, we've got a third key property of the code.
And we see it's key, because we can get the minimum squared distance between this Euclidean image constellation, just 4 alpha squared d
nd makes a probability of [UNINTELLIGIBLE] that is dependent on [UNINTELLIGIBLE] then
Correct.
This is all we need to know to get the union bound estimate.
Well, a few more things.
We need to know what K_min average of S of C. And what is that going to be?
This is simply going to be the number of words in the code.
To get this minimum squared distance, we need a Hamming distance of d. So the number of words in the code of distance d, which is given by the parameter n sub d, is simply going to be the number of nearest neighbors.
Not just the average distance, but I want to emphasize this symmetry property.
If we stand on any point, on any vertex of this cube in n-space, which is the code vertex, and we look at all the other points in the constellation, no matter which point we stand on, we will always see the same profile of distances.
We'll see precisely nd code words at Euclidean distance 4 alpha squared d. We'll see nd plus 1 at Euclidean squared distance 4 alpha squared d plus 1, and so forth, right up the profile.
So there's complete symmetry in the constellation.
In that universe, you don't know which code point you're standing on just by looking out, because the world looks the same to you.
Is that clear?
OK.
So from a communications point of view, this is important, because it means it doesn't matter what code word we send.
The probability of error from any code word is going to be the same as the probability of error from any other code word, because the geometry is exactly the same.
The Voronoi regions are all the same shape.
So given the exact probability of error, not just the union-bound estimate, is going to be independent of which code word was sent.
This all follows from the fact that it's a linear code and therefore has the group property, which translates into this very strong geometrical uniformity property in Euclidean space.
Or actually in Hamming space too, but it's more striking in Euclidean space.
OK?
So we have everything we need to write down the union bound estimate.
Union bound estimate was just the probably of error per bit is well approximated by K_b of constellation, in this case, S of C, times Q of the square root of the coding gain of the constellation times 2 Eb over N_0.
I have our final exam schedule from the registrar.
It comes Tuesday morning of exam week.
It's on the web page.
You won't miss it.
I want to remind you that the midterm exam is on Wednesday, March 16, that it starts at 9 not at 9:30 so that you're less time-limited than you would be otherwise.
It's a two-hour exam.
And it will cover basically up through chapter eight, which is Reed-Solomon codes.
And you're responsible for anything that's been discussed in class.
If we haven't discussed it in class, then don't worry about it.
All right.
Any questions about any of those things?
It's 9 o'clock to 11 o'clock.
What's the underlined?
This line up here?
This says, "This class goes from 9:30 to 11.00."
Not 11:15.
That's for me, not for you, although I've tried to lay off some of the responsibility on you.
OK, let's continue.
I hope to finish up chapter six today, might not completely finish it.
There are really three topics left to go.
One is the orthogonality and inner product topic, which I skipped.
The second is Reed-Muller codes, which is our main objective in this chapter, a family of useful codes.
And the last topic is why making hard decisions is not a good idea.
And so I'll try to say as much as I can about those three things.
Just to remind you of where we are, we're in the power-limited regime.
We're trying to design good, small-signal constellations, or moderate-sized signal constellations now, with nominal spectral efficiency, less than two bits per two dimensions.
The technique we're using is we're going to start from a binary linear block code in Hamming space.
And we're going to take the Euclidean image of that and hope it's a good constellation in Euclidean space.
And as we were just talking about before class, the fact that a linear code is a subspace of F2 to the n means it's true if and only if, really, it has the group property, which in Euclidean space leads to a geometrical uniformity property.
We haven't proved that in its full scope.
But we have noticed that from every code word, every code word has the same distance profile to all other code words.
And in particular, the minimum distance is the minimum weight of any non-zero code word.
That's the main juice we've squeezed out of that at this point.
So we've now been talking a little bit about the algebra of binary linear block codes.
We've characterized them basically by three parameters, n, k, d, where n is the length of the code, k is the dimension of the code, d is the minimum Hamming distance of the code.
In the literature, this is what you'll mainly find, in the n, k, d code.
We have a subsidiary parameter, the number of words of minimum distance d, which we're going to need to get the error coefficient, or a union-bound expression.
This has less prominence in the literature.
Given just these numbers, we get a couple of key parameters of our constellation.
One is its nominal spectral efficiency, which is 2k over n bits per two dimension, upper bounded by 2.
Of course, often in the coding literature, you talk about in a more natural quantity, which is k over n, called the code rate.
Maybe I shouldn't call it cap-R, because R is used for the code rate in bits per second.
This means bit per symbol.
So just let me call this rate.
But when you're reading the coding literature, the rate of the code means how many information bits per how many transmitted bits.
And for n, k, d binary linear block code, it's k over m. And the nominal spectral efficiency is just twice that, since we measure it per two dimensions.
OK, and even more importantly, we get the union bound estimate in terms of a couple of simple parameters.
It's just an error coefficient, Kb, the number of nearest neighbors per bit times this Q function expression, which always has 2Eb over N_0 in it, multiplied by this multiplicative factor, which we call the coding gain, which is just kd over n. And this is equal to one for a 1,1,1 code.
Now you know coding.
So anything else?
Our basic effort is to get a larger coding gain by constructing more complicated codes.
This parameter here, which we need in order to actually plot the curve and estimate the effective coding gain -- the effective coding gain is derated from the nominal coding gain by a rule of thumb, which depends on Kb.
It's every factor of 2 in Kb costs you 0.2 dB, roughly in the right range, if it's not too big, all those qualifiers.
But it's a good engineering rule of thumb.
And that's just the number of nearest neighbors per code word divided by k. And so our object here is to see how well we can do.
And the Reed-Muller codes will be an infinite family of codes that give us a pretty good idea of what can be achieved for a variety of n, k, d that kind of cover the waterfront.
That's why I talk about them first.
They're all so very simple.
OK, but first, we forgot to talk about orthogonality and inner products and duality, both from a geometric point of view and from an algebraic point of view.
And so I want to go back and recover that.
The definition of an inner product between x and y, where x and y are both binary n-tuples, is, as you'd expect, a sort of dot product expression, a component-wise product, the sum over k of xk yk, where all the arithmetic here is in the binary field, F2.
And well, this clearly has the bilinearity properties that you expect of an inner product that's linear in x for a fixed y or vice versa.
But it doesn't turn out to have the geometric properties that you expect.
We can define orthogonality, this is another definition, in the usual way.
x and y are said to be orthogonal -- do that better.
x is said to be orthogonal to y if and only if their inner product is 0, which is the same definition you know from real and complex vector spaces.
OK.
But the overall moral I want you to get from this is while this inner product behaves absolutely as you expect in an algebraic sense, it behaves very different from what you expect in a geometric sense.
So the algebra's fine.
The geometry is screwy.
All right.
That's the catch word to keep in mind.
Why is that?
Where does this inner product live?
It's F2 valued, right?
I'm just doing this sum in binary space, and the result is either 0 or 1.
So when are two n-tuples orthogonal?
Simply if they have an even number of places in which they're both equal to 1.
Is there a question?
In particular, suppose I try to define a norm in the usual way, like that.
Is that going to have the properties that I'd want of a norm?
No.
Why?
Because one of the basic properties we want of a norm is strict positivity, that the norm of x is equal to 0 if and only if x is equal to 0.
That's clearly not true here.
What's the requirement for the inner product of x with itself to be equal to 0, in other words x to be orthogonal with itself?
It just simply has to have an even number of 1's.
If it has an even number of 1's, then this so-called norm is 0.
If there's an odd number, it's 1.
So it's perfectly possible for a vector to be orthogonal to itself, a non-zero vector to be orthogonal to itself.
And that's basically where all the trouble comes from in a geometric sense
So which of them would be [INAUDIBLE] here?
It's mod-2.
It's in F2.
All the arithmetic rules are from F2 which is mod-2 rules, correct.
So, good.
This means at the most fundamental level, we don't have a Hilbert space here.
We don't have a projection theorem.
Projection theorem is the basic tool we use in Euclidean spaces, more generally, Hilbert spaces.
Just say that every vector can be partitioned.
In a given space and its orthogonal space, we can express a vector as the sum of its projection onto the space and the projection onto the orthogonal space, which are two orthogonal vectors.
So it's an orthogonal decomposition.
So we have nothing like the projection theorem here.
Therefore we have nothing like, we don't necessarily have orthonormal or even orthogonal basis for subspaces
[INAUDIBLE]?
You do have a unique orthogonal complement.
I'll get to that in a second.
But for instance, we might have that a subspace can be orthogonal to itself.
That's what the problem is.
For instance, 0, 0, and 1, 1 is a nice, little, one-dimensional subspace.
And what's it's orthogonal subspace?
It's itself.
We may not have an orthogonal basis for a subspace.
And for an example of that, I'll give you our favorite 3, 2, 2 code, as I'll now call it.
It consists of these four code words.
A set of generators for this code consists of any two of the non-zero code words.
You can generate all of the code words as binary linear combinations of any two of these.
But no two of these are orthogonal.
So there's clearly no orthogonal basis for that code.
We shouldn't expect to find orthogonal basis, orthogonal decomposition, so all of these sorts of tools that we relied on heavily in 6.450 in Euclidean spaces.
OK, so this is just a great caution to the student.
Don't expect Hamming space to have the same geometric properties as Euclidean space.
Yes?
[INAUDIBLE]?
Nothing special.
It's just finite fields have a different geometry.
In F2, there's really only one geometry you would impose, which is the Hamming geometry.
In other finite fields, there actually could be more than one geometry.
But let's say the algebraic properties, however, you can still count on.
For instance, n, k, d code C has, as we've already shown, a basis.
It has k dimensions.
It has a basis g1 up to gk of k, linearly independent, though not necessarily orthogonal basis vectors.
So we can always write the code as the set of all u g such that u is a k-tuple of information bits, let's say.
In other words, this is a compressed form for the set of all binary linear combinations of these generators, where I've written what's called a generator matrix, g as a k by n matrix, whose rows are these generators.
And I've multiplied on the left with a row vector u, if I'm doing it in matrix terms.
You can do this more abstractly just as a linear transformation.
And side comment, notice that in coding theory, it's conventional to write vectors as row vectors, whereas in every other subject you take, it's conventional to write vectors as column vectors.
Why is this?
Is there some deep, philosophical reason?
No.
It's just the way people started to do it in coding theory back at the beginning, and then everybody has followed them.
I may have even had something to do with this myself.
And I'll tell you the reason I like to write vectors as row vectors is I don't have to write a little t up next to them.
That's the deep philosophical reason.
It's like driving on the right side or the left side.
Obviously, you could have chosen how to do it one way or another back in the beginning.
But once you've chosen it, you better stick with it.
There's actually some insight involved here.
This is sort of associated with a block diagram, where we take k bits, and we run it through this linear transformation.
And as a result, we get out, what shall I write, x n bits.
And in block diagrams, we tend to take the input bits from the left and proceed to the right, left to right kind of thing.
So this formula reflects this left to right picture, whereas I'd say the deeper reason why you usually see G u in system theory is that G is regarded as an operator that operates on u.
It's kind of a G of u.
And so that's why it's more natural to think of this as being a column vector, because then, we have a different picture.
G is the operator that somehow transforms u.
But these are all side comments, obviously.
They don't really matter for anything.
All right, just remember that vectors are row vectors.
OK. Let's define the dual code in the natural way, as the orthogonal code.
Say C dual.
The definition is that C dual is the set of all n-tuples y, such that x, the inner product between x and y is 0.
In other words, y is orthogonal to x for all the x and C. It's the set of all n-tuples that are orthogonal to all the words in C under our F2 definition of inner product orthogonality.
OK, so that's a natural definition.
For example, as I've already shown you, if C is 0, 0, 1, 1, then C dual is 0, 0, 1, 1.
If C were 0, 0, 0, 1, then C dual would be 0, 0, what?
1, 0.
Thank you.
Because 1, 1 is not orthogonal to this, 0, 1 and is not orthogonal to this.
So we simply go through and pick it out.
Now as I said, the algebraic properties of the dual code are OK.
I emphasize they're algebraic.
If C is an n, k code, in other words, has dimension k, what do you expect the parameters of C dual to be?
Its length is what?
It's n, of course.
And what's its dimension going to be?
If C has dimension k, what do you guess the dimension of C dual is going to be?
Just guess from Euclidean spaces, or any -- it's n minus k. The dual space has dimension n minus k. And that holds.
In the notes, I give two proofs for this.
There's the conventional coding theory textbook proof, which involves writing down a generator matrix for C k generators, reducing it to a canonical form, called the systematic form, where there's some k by k identity matrix and a n minus k by k parity check part.
Then, by inspection, you can write down a generator matrix for C dual.
And you find it has dimension n minus k. It's kind of a klutzy proof, in my opinion.
A second, more elegant proof, but one that you don't have the background for yet, is to use simply the fundamental theorem of homomorphisms.
This is in some sense an image.
This is the dual of an image, which is a kernel.
You work out the dimensions from that.
I am still in search of an elegant, elementary proof.
And anyone who can come up with a proof suitable for chapter six gets a gold star.
Believe me, the gold star will be very valuable to you.
OK, so exercise for any student so inclined, give me a nice proof of this.
It's surprisingly hard.
And I can't say I've spent great quantities of my life on it, but I've spent some time on it.
So anyway, the dimensions come out as I hope you would expect, based on your past experience.
And I won't give you a proof in class.
You get the fundamental duality relationship, that the dual of C dual, what would you expects that to be?
C. OK.
In words, C is the set of all n-uples that are orthogonal to all the n-tuples in C dual.
OK.
So this actually means that I can specify C. If I know C dual, I know C. Give me C dual, the set of all code words orthogonal to it tells me what C is.
So this implies that I can write C in the following form.
C is the set of all, just emulating this, y n F2 to the n such that x, y equals zero for all x in C dual.
I probably should have interchanged x and y for this.
x such that x, y equals 0 for all -- this is symmetrical.
The inner product of x and y is equal to the inner product of y and x.
So I don't care how I write it.
OK, so I can actually specify a code by a set of parity checks.
Now suppose I have a generator matrix, call it H, namely a set of generators, H1 through Hn minus k, for C dual.
I'm going to have a set of n minus k generators.
Then I hope it's obvious that I can test whether x is orthogonal to all of C dual by just checking whether it's orthogonal to all of these generators.
So I would now have C is the set of all x n-tuples x such that x, H, j equals zero, all j. OK.
I've just got to test orthogonality to each of these generators.
In other words, in each of these is what we call a parity check.
We take the inner product of x with a certain n-tuple, and we ask whether parity checks.
In other words, we ask whether the subset of positions in which H, j is equal to 1, in those positions, whether x has an even number of 1's.
Get very concrete about it.
So writing this out in matrix form, the test is C is the set of x in F2 to the n such that x h-transpose equals 0.
That's just a matrix form of what I've written up there.
So let me picture it like this.
Here the test is, I take x, which is n bits.
I put it into what's called a parity checker or syndrome reformer.
And I ask whether this is 0.
We call this, in general, the syndrome.
And we ask whether it's 0.
If we get a 0, we say x is in the code.
If it's not 0, then x is not in the code.
That's the test.
So when we summarize, we really have two dual ways of characterizing a code, which you will see in the literature.
We have a generator matrix, we might give a k by n generator matrix for the code.
And then the code is specified as C is the set of all U g such that U n F2 to the k. In other words, this is an image representation.
We take C as the image of a linear transformation.
We call G a linear transformation.
It goes from Fk to F2 to the k to F2 to the n. And then the code is simply the image of this transformation algebraically.
OK. Or we can specify it by means of a parity check matrix, H, which is the generator matrix of the dual code.
So this would be the parity check matrix for the dual code G. h is the generator matrix of the dual code.
And we ask whether -- now we specify it as I have up here, simply the x and F2 to the n such that x H_t equals 0.
And this is what's called a kernel representation, because it's the kernel of a linear transformation defined by H2 from F2 to the end, down to, this is a m by n minus k matrix.
So the syndrome is actually an n-minus-k-tuple.
The elements of the syndrome are the n minus k individual bits, parity check bits.
OK. And sometimes it's more convenient to characterize the code in one way, and sometimes it's more convenient to characterize it in the other way.
For instance, we were talking last time about the n, n minus 1, 2 single parity check code, or the even weight code, the set of all even weight n-tuples, which has dimension n minus 1.
And in general, for high-rate codes especially, it may be simpler to give the parity check representation.
What is the dual code?
Let's call this C. C dual is what?
C dual is what we call the n, 1, n repetition code.
In other words, it has dimension one.
It has two code words, the all-0 word and the all-1 word.
Clearly, the all-1 word is orthogonal to all of the even weight words.
And vice versa, a word is even weight if and only if it's orthogonal to all 1's.
So in this case, what is the generator matrix?
We had a generator matrix last time consisting of n minus 1 weight 2 code words all arranged in a kind of double diagonal pattern.
That's OK.
But that's an n minus 1 by n matrix.
Most people would say it's easier to say, OK, what's the parity check matrix?
The parity check matrix in this case, H, is simply a one by n matrix consisting of all one's.
And what's the characterization of the code?
The code consists of all the words that are orthogonal to this.
And that is why we call it single parity check code.
There's one parity check.
And if you pass the parity check, you're in the code.
And if you don't, you're not.
OK, so we see that these, first of all, here's an example of dual codes.
Are their dimensions correct?
They are.
Is each on characterized correctly as the dual of the other?
It is.
So we've passed that.
This is intended to make point C. So it's a good example.
One final thing is suppose I have a code with generator matrix G and another code with generator matrix H. Are they each other's dual codes are not?
And the answer is pretty obvious from all of this.
Yes, they are.
Let me a substitute in here, x is supposed to be equal to U g. So another requirement is that UGH_t equals zero for all u.
And without belaboring the point, the 5, 2 codes with generator matrix G and H, they are dual codes if and only if they have the right dimension, one is n, k, and the other is n, n minus k, and we satisfy G H_t equals zero.
Basically, this is the matrix of inner products of the generators of C with the generators of C dual.
And if we have k generators that are all orthogonal to these n minus k generators, then they must be the generators of dual codes.
That's is kind of intuitive and natural.
All right, so again, this is a concise form that you would actually, probably most commonly find in the literature.
It's not hard to get to.
OK, so in this course, we're probably going to talk quite a bit about orthogonality.
Duality is very powerful, but we're not going to be using it very much in this course, I believe.
I'll mention duality whenever there's a duality property to mention.
But in general, I'm not going to spend an awful lot of time on it.
But it's important you know about it, particularly if you were going to go on and do anything in this subject.
And at an elementary level, it's nice to know that we have two possible representations, and one is often going to be simpler than the other.
Use the simple one.
In general, use the representation for the low-rate code to determine the high-rate code.
OK. Any questions on this?
I've now finished up the preparatory.
Yeah?
So you're saying that every basis for [INAUDIBLE]?
That's necessary and sufficient, right.
OK, Reed-Muller codes.
Why do I talk about Reed-Muller codes?
First of all, they give us an infinite family of codes, so we can see what happens as n gets large, as k goes from 0 to n, whose parameters are sort of representative.
They aren't necessarily the best codes that we know of.
They're very simple, as I will show, to construct and to characterize their parameters, so we can do all the proofs in half an hour here.
And they're not so bad.
In terms of the parameters n, k, d, the Reed-Muller codes, up to length 32, are the best ones we know of, at least for their parameters.
There is no 32, 16 code that's better than a 32, 16, eight Reed-Muller code.
There's no 32k, eight code that has k greater than 16, all those sorts of things.
Actually, I'm not 100% sure of that.
So for short block lengths, they're as good as BCH codes, or any of the codes that were discovered subsequently.
For even longer lengths, up to 64, 128, 256, they are going to be slightly sub-optimal in terms of n, k, d, as we'll see in some cases.
But they still are very good codes to look at in terms of performance versus complexity.
The decoding algorithm that I'm going to talk about eventually for them is a trellis-based decoding algorithm, and a maximum likelihood decoding algorithm within.
They can be maximum likelihood decoded very simply by these trellis-based algorithms.
And that's not true of more elaborate classes of codes.
So from a performance versus complexity point of view, they're still good codes to look at for block lengths up to 100, 200.
As we get up to higher block lengths, then we'll be talking about much more random-like, non-algebraic codes in the final section of the course.
This is the way you actually get to pass it.
You don't worry about n, k, d. Right now, we're talking about moderate complexity, moderate performance.
OK, so they were invented in 1954 independently by Irving Reed and, I think it was, David Muller, D. Muller, in two separate papers, which shows they weren't too hard to find.
As you'll see, they're very easy to construct.
They're basically based on a length-doubling construction.
So we start off with codes of length or length 2.
And then from that, we build up codes of length 4, 8, 16, 32, and so forth.
So in general, their lengths are equal to a power of 2, and m is the parameter that denotes the power of 2.
So we only get certain block lengths, which are equal to powers of 2.
They have a second parameter, r, whose significance is that d is 2 to the m minus r for 0, less than or equal to r, less than or equal to m. Or some people, including me, are going to put the lower limit at minus 1, just to be able to include one more code in this family at each length.
But this is just a matter of taste.
You'll see this is a very special case, the minus 1.
So let's guess what some of these codes are going to be.
So we have two parameters, m, which can be any integer 0 or higher, so the lengths will be 1,2,4, so forth, and r, which goes basically from 0 to m, which means the distances will go from 2 to the m down to 1.
And what are some of the basic codes that we're always going to find?
We always write Rm of little rm.
I'm not sure I completely approve of how the notation goes for these codes, but that's the way it is.
So it's this notation.
That means the Reed-Muller code with the parameters m and r is written Rm of r, m. All right.
So Rm of m, m, this is going to be a code of length 2 to the m and distance 1.
What do you suppose that's going to be?
This is always going to be the 2 to the m 1.
Sorry, the distance is 1.
So it's going to be the 2 to the m, 1 universe code, in other words, simply the set of all 2 to the m-tuples, which has Hamming distance 1.
OK. RM of 0, m, what's that going to be?
This is a code now that has length 2 to the m and minimum distance 2 to the m. We know what that has to be.
It has to be the 2 to the m, 1, 2 to the m repetition code, a very low-rate code, dimension one.
OK. And then if we like, we can go one step further.
There is a code below this code.
This is the highest-rate code you can get.
This, however, is not the lowest-rate code you can get.
What's the lowest rate code of length 2 to the m?
Well, it's one that has dimension zero and minimum distance infinity.
And I don't think I ever defined this convention.
But for the trivial code that consists of simply the all-0 word, what's it's minimum distance?
Undefined, or infinity, if you like.
So this is the trivial code.
So if we want, we can include the trivial code in this family just by defining it like this.
And it works for some things.
It doesn't work for all things.
It doesn't work for d, for instance.
This definition holds only for r between 0 and m. It doesn't hold for r equals minus 1, because there the distance is infinite.
OK, so let's start out and get even more explicit.
We want to start with m equals 0.
In that case, we have only Rm of 0, 0, and Rm of minus 1, 0.
And this is going to be the one by either of these.
The universe code is equal to the repetition code.
It's the 1, 1, 1 code.
And this is the 1, 0, infinity code.
And that's really the only two codes that you can think of that have length 1, right?
This is the one that consists of 0 and 1, and this is the one that consists of 0.
I don't think there are any other binary linear block codes of length 1.
So that's a start, not very interesting.
Let's go up to length 2.
So m is going to be equal to 1.
Here, Rm of 1, 1 is going to be length 2.
And it's going to be the universe code, so it's only going to have distance 1.
Rm of 0, 1 is going to be length 2, but it's going to be the repetition code.
And Rm of minus 1, 1 is going to be 2, 0, infinity.
OK, so there are really the only three sensible codes of length 2.
This is the only one of dimension two.
This is the only one of dimension zero.
There are other ones of dimension one, but they don't have minimum distance 2, so they're not very good for coding purposes.
So this kind of lists all the good coding codes of length 2.
All right.
So now let me introduce the length-doubling construction.
Let me make the point, first of all, that all these codes are nested.
What does that mean?
That means that his code is a sub-code of this code, which is a sub-code of this code.
And that's going to be, in general, a property of Reed-Muller codes.
We're going to get a family.
And each lower one is going to be a sub-code of the next-higher one, which is going to be easy to prove recursively.
All right.
This is the key thing to know about Reed-Muller codes, how do you construct them.
Once you understand the construction, then you can derive all the properties.
It's called the u, u plus v construction for obvious reasons.
Apparently, Plotkin was the first person to show this.
I think Reed and Muller had two different constructions, and neither one of them was the u, u plus v construction.
Reed, in particular, talked about Boolean functions.
All right.
But this is the way I recommend you to think about constructing them.
And it's simply defined as follows.
We assume we've already constructed the Reed-Muller codes of length m minus 1.
It's a recursive parameter, m minus 1.
Now we're going to construct all the Reed-Muller codes of parameter m of length 2 to the m. How are we going to do it?
We want to construct Rm of r, m. And we'll say it's the set of all code words which consist of two halves, a pair of n-tuples of half the length.
so the two halves are going to be u and u plus v, where I choose u and u plus v as follows.
u is in Rm of r minus 1, m. And so what does that mean?
Sorry, it's got to be m minus 1.
So it's got to be half the length.
Is that right?
r m minus 1, v is in Rm of r minus 1, m minus 1.
Somebody who has the notes, like my valuable teaching assistant, might check whether I got it right or not.
Let's see.
What are going to be the parameters of these codes?
They both have n equals 2 to the m minus 1.
The distance here is 2 to the m minus 1 minus r minus 1, so the distance here is 2 to the m minus r, which is the distance that we want to achieve for this code.
And the distance here is 2 to the m minus r minus 1, which is half the distance we want to achieve for this code.
So we reach down, if we wanted to construct now a code of length four and distance two, we would construct it from these two codes, the one of half the length with distance 2, and the one of half the length with half the distance, distance 1.
So we would somehow use this to get a code of length 4 and distance 2.
And we don't know yet what k is.
OK.
So we're going to use these two codes to construct a larger code.
Is the construction clear?
All right.
So let's now derive some properties from this construction, and from the fact that we've started from a set of codes.
Let's say we start from length two codes.
Or we could start from length one.
You could satisfy yourself that this construction applied to the length one codes gives the length two codes.
I won't go through that exercise.
So we have some set of codes, m minus 1.
What's the first thing we notice?
Obviously, the length is what we want, because we've put together two 2-to-the-m minus 1-tuples, and 2 times 2 to the m minus 1 is 2 to the m. So we've constructed a set of 2-to-the-m-tuples.
The length of the resulting code is 2 to the m. This is Rm r, m, constructed in this way.
Second, it's linear.
It's a linear code.
All we have to check is the group property.
If we add two code words of this form, we're going to get another code word of that form, from the fact that these guys are linear, yes.
OK.
So it is a linear code, that's important.
Then we might ask, what's its dimension?
How many code words are there?
Well, do I get a unique code word for every combination of u and v. And however you want to convince yourself, you obviously do.
If I'm given this word here, I can deduce from it what was u, that's simply the first half of it.
Subtract u from the second half of it, and I find out what v is.
So there's a one-to-one map between all possible pairs, u, v, and all possible new code words.
And so what that means is, let me call it the dimension of the code with parameters r, m is simply the sum of the dimensions of the codes with parameters r, m minus 1 and r minus 1, m minus 1.
So for instance, in this hypothesized code here, if I want to know the dimension, well, I take all possible combinations of words here and words here.
How many are there?
There are eight.
It has dimension three.
So I'm going to get a 4, 3, 2, code, which it's not too hard to see is the single parity check code of length 4.
All right.
So that's how I do it.
I simply add up the dimensions of the two contributing codes.
Not a very nice formula.
In the homework, you do a combinatoric exercise that gives you a somewhat more closed form of the formula.
But I think this is actually the most useful one.
In any case, we eventually get a table that shows what all these things are anyway.
Now, just as a fine point, I want to assert that if I start out from a set of nested codes, then I come up with a set of nested codes, at the next highest level.
That, again, is sort of obvious.
If these guys were nested, then I get the appropriate -- if I take u, u plus v from sub-codes, I'm going to get a sub-code.
Look at the notes if you want more than that little bit of hand-waving.
OK, that's something I need for the next thing.
I want to find out what d is.
What's the minimum distance here?
Of course, my objective is to make it equal to 2 to the m minus r. Did I succeed?
What are the possibilities for u, u plus v?
The possibilities are that they're both 0, or that this one is 0 and this is not equal to 0, or this is not equal to 0 and this is 0, or that they're both not equal to 0.
And I'm going to consider those four cases.
Since every linear code include the all-0 word, it's certainly possible that this comes out at 0, 0.
The only possibility for this to come out 0, 0 is if I choose u equals 0.
Then I have to choose v equals 0.
So there's one-code word of weight 0 in my new code.
But that's OK.
If there were two code words with weight 0, well, then the dimension would be wrong.
This is in effect a proof that the kernel is just 0, 0.
And so the dimension is OK.
It's a one-to-one map.
All right.
So I don't really have to worry about that.
I'm going to get one all-0 word in my new code.
I can afford one all-0 word.
I'm always going to have to have it anyway.
It's linear.
All right, so these are the more interesting cases.
Suppose the first half is 0, but the second half is not 0.
That implies that u is 0.
That implies that the second half is just v, which is not 0.
So v must be a non-zero code word in this code, which has minimum distance 2 to the n minus r. So in this case, I prove that the distance is greater than or equal to 2 to the m minus r. Similarly, suppose this one is not 0, but this is 0.
OK, if that's 0, it can only be because I chose u equal to some v. and that means the first half, then, is that v. So again, v is in this code that has enough minimum distance.
So in this case, I proved that the code word has weight 2 to the m minus r. And finally, let's take the case where they're both non-zero.
In that case, u could be an arbitrary word in this code which only has distance 2 to the m r minus 1.
So the first half is going to have weight at least 2 to the m minus r minus 1, but that's all I can say about the first half.
But now the second half, what is this?
This is a higher-rate code than this.
This, by the nesting property, is a sub-code of this.
So if I add a word in a sub-code to a word in the code, I'm going to get another word in this code.
So u plus v is still in this Reed-Muller code, still has a minimum weight, if it's non-zero, of 2 to the m minus r minus 1.
So the distance is at least this in the first half, this in the second half.
And that's, of course, still good enough.
OK.
So by this construction.
I've assured that I'm going to get a d greater than or equal to 2 to the m minus r. And of course, you can easily find cases of equality, where it's only 2 to the m minus r. If this has a word of weight 2 to the m minus r, then you can clearly set up one like this that has weight 2 to the m minus r. Just pick one of the minimum-weight code words as v, and u as 0.
So the minimum distance is 2 to the m minus r. All righty.
So those are all the properties we need.
And then, I like to display these properties in a tableau which you have in the notes, which goes as follows.
Let's just start listing these codes.
Here are the length 1 ones.
We only found 2 of them, 1, 1, 1, and 1, 0, infinity.
So there are two codes of length 1.
Now it turns out that if you combine these according to the u, u plus v construction, you get 2, 1, 2, where the weight 2, 1 is just the -- you take the first half is 1, and the second half is 1.
So you can build this in the same way.
And similarly, we can say just by definition, we're always going to put a universe code at the top and a trivial code at the bottom.
So now I've listed all my Reed-Muller codes with length 2.
Now to construct the ones of length 4.
Again, I'll put a universe code at the top, a trivial code at the bottom.
I'll use my construction now to create a 4, 3, 2 code here.
I'm just using all these properties.
And down here, my construction, when you combine these two things, you always get a repetition code, again, 4, 1, 4.
And I guess I've hand-waved.
Exercise for the student, prove that combining a repetition code with a trivial code under the u, u plus v construction always gives a double length repetition code.
It's clear.
v is always the all-0 word.
u is either all-0 or all-1.
So we get two words, one of which is all-0, and one of which is all-1, double length.
All right.
So now I can just go on indefinitely, and without a great deal of effort.
Here I find that k is 7, just by adding up these two things.
The 8, this gives me an 8, 4.
4 code.
This gives me an 8.
1, 8 code, and similarly down here.
And this, I now just turn the crank.
16, 16, 1.
I always put that on top.
The next one is 16, 15, 2.
Next one is 16, 11, 4.
After a while, you don't know what you're going to get.
But you get something.
You've proved that all of these codes exist, that they're all linear.
They all have the n, k, d that we've specified, and that furthermore, they're nested.
And a final property, which you might suspect, looking at these tables, is that the dual of a Reed-Muller code is also a Reed-Muller code.
And they're paired up according to k and n minus k. Let's see.
15 and 11 is 26.
11 and five is 16.
5 and 1 is 6.
And 32, 1, 32, and so forth.
Did you see how I proved that all these codes exist?
And if I continued, I could get arbitrarily long codes, one of your simple homework problems is just to do this, continue this for 64 and 128, see what additional codes you get.
And so I now have this infinite family it of that kind of covers the space n, k, d in some sort of sparse way.
But it indicates how k and d go with n. And we come back to our original question, how well can we do with binary linear block codes.
Well, here's some binary linear block codes, pretty close to the best we can find, actually.
The ones I've listed up here are all as good as we can find.
And how well can we do?
Really, we don't need to know much to evaluate the performance of, say, 32, 6, 8 code, which is now getting to be a pretty substantial code, with 2 to the 16 code words, to 65,536 code words.
So we built a fairly sizable constellation here in a 32-dimensional Euclidean space.
And how good is it?
Let's take 32, 16, 8.
Can I graph its probability of error per bit, a good estimate of it?
Can I?
Is there any information I'm lacking?
OK, I've been talking too long, because when I go to the class, I'd like a little response.
So you were going to say something?
We don't have [INAUDIBLE]
OK, let me tell you that the n, d is approximately 600-something, so let's say 630.
It's probably not exactly correct.
In the notes I give a formula for n, d -- a formula that I don't derive, that is known for Reed-Muller codes.
And from that, you can compute n, d for any of these codes.
This parameter is m, r. All right.
So I'll give you n, d as well.
Now can I get the good estimate for the probability of error per bit?
How do I do that?
Union-bound estimate.
All right, so what's it going to look like?
What are the two subsidiary parameters I need?
One is the coding gain, right?
What is the nominal coding gain of this 32, 16, 8 code?
Come on, this is not a difficult computation.
OK, what's k over n?
1/2?
I think we can all do that one.
What's the nominal coding gain?
Take 1/2 times d, 8.
So now our coding gain is 4, which is what in dB?
6 dB.
Wow, gee whiz.
I've already got a nominal coding gain of 6 dB.
Remember, my whole gap was, depending on how I measured it, 9 or 10 or 11 dB.
This looks like already a sizable fraction of the gap, with just a simple construction.
Well, we better pay attention to this error coefficient as well, or the number of nearest neighbors.
Kb is, let's just take a rough estimate here, what's this going to be?
About 40.
That's good.
It's just this divided by 16.
So there are about 40 nearest neighbors per bit.
How much is that going to cost us?
Not so good.
Well, it's a little bit more than 5 factors of 2.
Maybe 5 and a 1/2 factors.
It's less than 5 and a 1/2.
So this very roughly, something will cost me about 1 dB.
And so I get an effective coding gain of 5 dB.
That's my first very gross estimate.
All right, well, it's not 6 dB.
It's only 5 dB.
That's still not bad.
Again, if I really wanted to know what this was, I would write this out as 40 or whatever it is times Q to the square root of four times 2Eb over N_0.
And can I plot that quantity?
Yes, if I have MATLAB.
Or actually, all I need is my baseline.
So if this is my baseline, which went through 9.6 dB at 10 to the minus 5, then we remember how to plot that.
I just take this whole curve bodily, and I move it 6 dB to the left.
So I get the same curve, this is not very good, going through 3.6 dB.
Sorry about that.
How's that?
Get it way down here.
But then I also have to raise it by a factor of 40.
So it really looks more like that.
That's really going to go more through about 4.6 dB or something like that.
That's what these calculations are, Ashish took you through, I believe.
And so while 6 dB was my nominal coding gain, my effective coding gain is 5 dB.
But still, hey, not bad.
I have very easily been able to construct a code that gives you about 5 dB of coding gain.
Is there any fly in this ointment?
Can I just go out and build it now?
I need a decoder.
Who said that?
Thank you.
Good point.
What's the decoding method assumed for this code?
Excuse me?
Just a table right now
Just a table, based on what?
I get some kind of received n-tuple, which is actually just a random, some kind of vector in 32-dimensional space, 32 numbers, real numbers.
And really, I'm assuming minimum distance decoding.
So in principle, I want to compute the distance to each of the 2 to the 1/6th, 65,000 code words.
And actually, nowadays, you might just do that.
That's not a formidable task.
Back when I got into this business, that would have been considered outrageous.
But nowadays, you could keep up a pretty good decoding rate, even doing 65,000 distance computations and just finding the best one.
That would do it.
That would give you this performance.
But of course, we are going to be looking for somewhat more efficient decoding schemes than that.
All right.
So at this point, that's the only fly in the ointment.
We have a way of getting this kind of error curve, provided that we're willing to do exhaustive maximum likelihood or minimum distance decoding.
And furthermore, we can continue.
It's also instructive to see what happens as we let n go to infinity.
What are we going to get with this construction?
We pretty well know.
These are all going to be universe codes, which are not very interesting to us.
They all have a nominal coding gain of one, and are just useless.
They're basically just send a bit 32 times, send 32 bits, I mean.
OK, what are these codes along here?
Let me start there.
These are all single parity check codes.
What's the nominal coding gain?
Well, as we get out here, what does the nominal coding gain approach?
The code rate approaches 1.
The code distance stays at 2.
So the nominal coding gain goes to 2 or 3 or dB, at a rate of 1 or a nominal spectral efficiency of 2.
OK. Well, these are totally simple codes, single parity check codes.
And even with that, it looks like we can get 3 dB of coding gain.
But what's the number of nearest neighbors here?
Number of nearest neighbors is just n, n minus 1 over 2, n choose 2.
So the number of even dividing by k, which is n minus 1, we still get a Kb that goes up linearly with n. So what's in fact going to happen to the effective coding gain?
The nominal coding gain will go up and reach an asymptote of 3 dB.
This is nominal coding gain.
But somewhere out here, as this has reached an asymptote, the effective coding gain is always going to be less, and it's going to have to bend over.
And according to our rule of thumb, it'll eventually go back through 0, because the cost just keeps going up linearly in terms of Kb.
So the effective coding gain is not as great.
It has a peak for some n. And so there's some maximum effective coding gain, which again I've left for you as a homework problem, that's less than 3 dB.
And I'll give you a hint that it's of the order of 2 dB.
It's pretty easy to work out in the five minutes before the next class, not a difficult problem to find out when this thing reaches its maximum.
But still, these are very simple codes.
We'll see in a second they have an extremely simple minimum distance decoding algorithm called Wagner decoding.
It's trivial, not hard to do minimum distance decoding for these codes.
And so, OK, not hard to get 2 dB of coding gain.
What happens if we go along this line here?
These are all codes of minimum distance 4.
Again, start here.
They're called extended Hamming codes.
A Hamming code has minimum distance three and is suitable for hard decisions, single error correction.
These codes all have minimum distance four.
We've seen that it's always worthwhile to add an overall parity check to get an even minimum distance, if we're looking at Euclidean space coding gain.
And so these are actually slightly better than Hamming codes.
They're called extended Hamming codes, because they're Hamming codes extended by a single parity check.
They have one more unit of minimum distance.
So again, asymptotically, these are called extended Hamming.
The nominal coding gain goes to what now?
This is, the rate is again going to 1.
The distance holds at four.
So the nominal coding gain goes to 4, or 6 dB, while again, the spectral efficiency goes to two bits per two dimensions, the nominal spectral efficiency rate to 1.
But again, you have this kind of phenomenon.
And I ask you to work that out also on the homework, where even though the nominal coding gain plateaus eventually at 6 dB, there is a maximum effective coding gain, which is something more in the range of 4 to 5 dB.
You figure out what it is.
And there's a limit to how much effective coding gain you can get with these codes.
Does this all makes sense?
Are you seeing how I'm arguing?
OK, here's another interesting sequence of codes.
These are all half-rate codes, or nominal spectral efficiency one bit per two dimensions.
I briefly mentioned that these things pair up in duals.
The 16, 5 code is the dual code of the 16, 11 codes.
If you see, there's a symmetry about rate 1 by 2, such that this is k, and this is n minus k. And so you would suspect that this guy is the dual of this guy, which it is.
This guy is the dual of this guy.
This guy is the dual of this guy.
And this guy is its own dual.
It's a self-dual code of rate 1 by 2 or spectral efficiency 1.
So these are self-dual codes.
And what does their nominal coding gain go to?
Well, the rate is always 1 by 2 times four, that's a nominal coding gain of 2.
This has a nominal coding gain of 4.
The next one in line would be a 128, 64, 16 code, nominal coding gain of 8.
So the nominal coding gain actually goes to infinity.
That's pretty good.
However, what is its true meaning?
What we really want to know is what's the effective coding gain.
And given that the nominal coding gain goes to infinity, is it possible the effective coding gain can get us all the way to the Shannon limit?
Can we completely close the gap to capacity?
As far as I know, this is an open question.
I strongly believe that you go along this sequence through maximum likelihood decoding, you will eventually get to the Shannon limit, that is the Shannon limit for this spectral efficiency, which is not the ultimate Shannon limit, but rather is 0, in terms of Eb over N_0 If you remember, for rate 1/2 for rho equals 1, the Shannon limit on Eb over N_0 was 0 dB.
You may or may not remember that.
So there's a question.
Does this take you all the way to the Shannon limit?
And that would be a nice question for somebody to answer someday.
I think it probably does, especially in view of the fact that if you take this set down here, again, this will be a homework problem, this turns out to be a set of Euclidean images of these codes are orthogonal signal sets.
For instance, 32, 6, 16, what is that?
That's a set of 64 constellation points in 32 dimensions.
And algebraically, it's not hard to figure out that every one of these code words is orthogonal in a Euclidean sense to one another, except for one that's complementary, which is just what you expect in a bi-orthogonal signal set.
So these give you bi-orthogonal, they're called bi-orthogonal codes, or first-order Reed-Muller codes, because the parameter r is 1 for all of these.
What's happening to the spectral efficiency here?
Spectral efficiency goes to 0.
So these become highly inefficient from a bandwidth point of view, as we already know about orthogonal, bi-orthogonal simplex signal sets.
They use up a lot of bandwidth.
But what else do we know about them?
In this case, we definitely know that the effective coding gain does go to the Shannon limit, and in this case, to the ultimate Shannon limit for rho equals 0 as rho approaches 0.
So here, there's a proof that these codes can get you to the Shannon limit.
But as we've already explained earlier, geometrically, it's at the cost of using much more bandwidth than you probably really want to use.
But here, at least, is one capacity-approaching set of codes, just among these rather simple Reed-Muller codes along this line.
And of course, we also see our repetition codes, our trivial codes.
So this is a nice representative family of codes.
And it really does tell you quite well what to expect.
Are you looking at your watch?
Thank you.
I appreciate the hint.
So we didn't quite finish chapter six today.
Next time, we'll start out with the penalties of making hard decisions, which at first brush seems like a not unreasonable compromise to make.
But it actually costs a serious penalty.
And that will finish chapter six.
I'll do that as briefly as I can.
And then we'll get into chapters seven and eight, which is finite fields and Reed-Solomon codes, which are the single great triumph of algebraic coding theory.
So OK, that's tomorrow.
Good moRning.
Again, we're left with a little stub of chapter six to finish.
This was our first chapter on binary linear block codes.
Very briefly, last time we developed the family of RM, Reed-Muller codes, parameterized by R and M. These are codes of length two to the M, and distance 2 to the M minus R for all reasonable combinations of M and R, integers.
And the reason I do these first is because it gives us a nice big -- in fact, infinite -- family of codes that kind of cover the waterfront.
Let us see what we can reasonably get in terms of the parameters n k d in codes.
And, of course, n k t d tell us most of what we know, want to know about the performance of moderate complexity codes in terms of coding gain.
And it's easy to see that among these codes, we have sequences in which the coding gain goes -- the nominal coding gain goes to infinity.
And we also know that there's at least one sequence, the bi-orthogonal sequence, for which the effective coding gain completely closes the gap to the Shannon limit.
And in all likelihood, there are many such sequences, though I'm not aware of the actual result there.
So it seems like we might be almost finished here, except I asked about what possible fly there could be in this ointment last time.
And one of you alertly reported that we still really hadn't worried about decoding.
And the implicit decoding algorithm here is minimum distance decoding.
Exhaustive minimum distance decoding.
We basically have to compute the minimum distance to each of the 2 to the k code words.
And that's going to get a little tedious as soon as k gets up to 16 or 107 or whatever.
So it was recognized pretty early in the development of coding theory that this was the essential problem.
The problem is not to construct codes whose performance under maximum likelihood decoding is near that of the Shannon limit -- you can use random codes for that.
But in constructing these more algebraically structured codes, what we're really trying to do is to facilitate the decoding problem, which is the central problem.
We need to find a decoding method that is actually feasible.
And, of course, Shannon didn't address that at all.
He just addressed the ultimate potential of these codes with the optimal decoding technique and didn't worry about its complexity.
All right.
So I'd say the fundamental problem here is decoding complexity.
Now, one of the -- an awful lot of the early work on coding was about binary codes, of course.
But it also considered them in the context of binary decoding algorithms.
In other words, what's assumed is what you send as a series of bits and what you receive as a series of bits.
That's not the picture we have here in this course, where what we receive is a series of real numbers.
Because we're going over a continuous additive white Gaussian noise channel.
Of course, there are channels which are inherently digital, at least by the time the coding engineer can get his hands on them.
What he's presented with is a string of bits.
So it makes sense to consider codes for bit error correction.
Classic codes of this type are, first of all, the repetition code.
If I send this -- if I use the 7 1 7 code, the repetition code of length 7, I'm going to be able to correct up to three errors.
Three bit errors.
Because four bits are still going to be correct.
In a majority vote, we'll do the decoding.
Most of you have probably seen Hamming codes.
These are codes with minimum Hamming distance 3.
That means if you make one error, you're still within the Hamming sphere of radius 1 around the original code word.
That's disjoined from all the other Hamming spheres, and so in principle you ought to be able to do single error correction with Hamming codes.
And there are very simple algorithms for doing that.
OK.
But that may be what you have to do.
The final point of chapter six is that in the additive white Gaussian noise context, that's not what you want to do at all.
And I just want to say a few words about that because this is one of the Achille's heels of a lot of classical coding theory, which was addressed to the binary in, binary out situation.
So let's talk about the penalty for making hard decisions.
That is, by hard decision, we mean a binary decision.
0, 1, plus or minus 1.
And I'll do this fairly quickly, but in a way that I hope will stick.
Let me draw our block diagram of our coding scheme.
We've got a code.
We've got an encoder, which basically puts out a code word.
An n-tuple in the code.
I'm going to consider the individual elements of this n-tuple.
They're just elements of the binary field F2 And we go through our standard 2-PAM map, which maps them to s of xk, which is going to be in plus or minus alpha in general.
Just plus or minus 1.
Doesn't matter up to scaling.
In order to send them, we're going to send this sequence of real numbers over the white Gaussian noise channel.
And as a result, we're going to get out s -- let's call it a received symbol -- rk, which is the transmitted symbol plus some Gaussian noise symbol, nk.
This is a real number.
And then -- then what we do?
Now we're into the decoder.
This is what we received, so it's free to us to choose what to do next.
I'm going to say observe, first of all, that without loss of generality, we can decompose rk into a sign and magnitude.
So this out here is going to be a sign of rk.
And this is going to be the magnitude of rk.
And let me just give this a name.
Let me call this yk, and let me call this beta k. OK.
So what is this?
This is in plus or minus 1, and this is in the positive reals, so the non-negative reals.
OK. And clearly, if I keep both of these, I haven't lost any information.
This is just a way of representing rk in sign and magnitude form.
Now, this is what's called the hard decision.
Actually, let me do this differently.
Let me pass this through the inverse to a 2-PAM map.
So out of this, I get a yk, which is in F2 OK.
Either of these -- the real sign or the binary hard decision -- could be considered to be the hard decision.
Now, what am I doing here?
I'm saying, well, if you force me to make a guess on whether this individual transmission was sending a 0 or 1, this is the most obvious guess.
If I send -- here's minus 1, here's our minus alpha, here's plus 1 -- if I sent a plus 1, then my maximum likelihood per bit decision would be to decide the plus 1 was sent.
If I get a positive sign in -- what I'm really mapping here is rk, and minus 1 if I get a negative side.
So that's what's called a hard decision.
I'm going to decide right here what I think it was, what it most likely was.
And this is the best way to do it.
Just take the sign.
This here is a real valued weight, often called the reliability, which is helpful to retain.
That's the main point of what I'm going to talk to you about.
If this is 0 -- that means what we received was right on this boundary between the positive and negative side -- then the reliability is 0.
In likelihood terms, that means it's equally likely that what was sent was a plus 1 or a minus 1.
So in some sense, we get no information when the reliability is 0.
Fails to discriminate between plus 1 and minus 1.
The larger this is, the further we get out here.
Or symmetrically, the further we get out here, the more certain we are that in that particular symbol, neither a plus or a minus was sent.
OK And it actually tuRns out, if you work out the Gaussian numbers, that beta k is the log likelihood ratio up to scale of the more likely versus the log of the likelihood, or the more likely versus the less likely symbol.
So it has a minimum of 0.
The bigger it is, the more reliable.
So it's natural to call it the reliability.
OK. An awful lot of traditional algebraic decoding neglects this channel down here.
It just says, OK, let's take these hard decisions and try to decode them.
If you've previously had a coding class, then that's probably what they did.
Or they assumed a context in which this was never available in the first place.
And my only point here, my main point, is that that's a very bad thing to do.
How can we evaluate that?
Suppose we just take this top channel and don't look at the reliability.
Then basically we have the channel model becomes a binary symmetric channel.
We have bits in, bits out, and we have probably one minus p that if we send a 0 we get a 0.
And since it's symmetric, the same probability -- if we send a 1, we get a 1 -- and a probability p of making an error.
So we get the traditional memoryless binary symmetric channel model.
We can compute the capacity of this model and compare it to the capacity of the additive white Gaussian channel -- this model.
And depending on the signal to noise ratio, from a capacity calculation, we find that there is of the order of 2 or 3 dB loss.
Well, as we go on in this course, we're going to find that 1 dB is a very worthwhile coding game.
So to throw away 2 or 3 dB just by throwing away this information, is a bad thing to do.
I can make this point in a different way by simply looking at what's the optimum decision rule.
For say, let me take very simple code, just the 2-1-2 repetition code.
That's just 0,0,1,1.
This is the code where you either send plus, plus, plus alpha, plus alpha, or minus alpha, minus alpha.
And what's the decision region?
It's obviously this 45 degree line.
And what is this squared distance to any decision region?
It's 2 alpha squared, right?
So basically, the probability of making an error is the probability that the noise variable has a magnitude in this dimension, and in this direction, greater than the square root of 2 alpha squared.
You've been through all those calculation several times now.
All right.
So this is -- if I keep reliability info -- in other words, if I keep the full received signal -- if I discard reliability, then basically what can I get out?
Then my y, as I say, is in 0, 0, 0, 1, 1, 0, 1, 1.
There are only four possible things I can see in two transmissions through this channel.
They're all binary two-tuples.
And what does that mean?
That means if my actual r is in this quadrant, then I'm going to make a hard decision of 0 and 0 both times.
So I'm going to say I'm in this quadrant.
And likewise, I'm basically going to decide which of these four quadrants I'm in, and that's all the information the decoder is going to have.
So here I am at this point here.
Now the decoder simply knows, via these two bits, which quadrant you're in.
Now, it has to decide which of these two code words were sent.
What's its maximum likelihood decision rule given just this information?
0, 0, [INAUDIBLE] 0, 0, [INAUDIBLE] 1, 1, 1, 1, 1.
And in the other case, any one
OK. That's correct.
So clearly, if you land in this quadrant, you decide 0, 0.
Now, if you land down here, you decide 1, 1 in this quadrant.
And what am I going to do here or here?
Actually, here the evidence is totally balanced.
I have nothing that tells me whether to go one way or another.
I could make an arbitrary decision in the hope of minimizing my error probability.
Flip a coin.
But whichever decision I make, what I'm going to find is that there's a probability of error that -- there's a certain noise, namely the noise that takes me from here to this decision boundary that is going to cause an error.
Or if I decide the other way, it would be the one that goes from here to here.
So I'm going to be stuck.
There are going to be certain noise variables of length, now, merely alpha, or square distance alpha squared, that are going to cause me to make a decision error, ultimately.
Regardless of how I set up this final block here.
Decoder.
Whatever rule I give to this decoder, it's only going to take, worst case, a noise of squared magnitude alpha squared to cause an error
[UNINTELLIGIBLE PHRASE]
That's right.
I shouldn't have thrown this away.
That's the elementary point I'm trying to make here.
All right
[UNINTELLIGIBLE PHRASE] the distance, d_min is different, essentially.
I mean, we can put it that way
That's where I'm going, is that the effective minimum squared distance here is 2 alpha squared.
And this is what shows up in our union-bound estimate in all of our -- we do this kind of minimum distance decoding.
But if I throw away this important information, then my effective minimum squared distance is only alpha squared.
That is the bottom line here.
In dB terms, how much of a cost is that?
3 dB, right?
So I've cost myself a factor of 2 in noise margin.
So 3 dB loss.
Because of lack of time, I won't go through the argument in the notes, which shows that, in fact, exactly the same thing occurs for any -- whenever the minimum Hamming distance is even here in this code that I started from, you lose precisely 3 dB.
It's not quite as clean an argument when the minimum Hamming distance is odd.
Then you lose up to 3 dB, and it goes to 3 dB as the distance increases.
But there's a very, pretty elementary geometric argument that hard decisions, again, cost you 3 dB loss, which is consistent with what the capacity calculation gives you.
All right.
Well, obviously the reason we did this is we were trying to simplify things.
What would be the first step to unsimplify things and get some of this loss back?
Let me suggest that what this amounts to is a two-level quantization of the received real number rk.
What's the next number higher than 2?
Integer
3
3.
All right.
How about a 3 level quantization here?
OK.
So what we're talking about is a highly quantized magnitude where instead of just making a decision boundary here, which is effectively what we did for 2 level quantization, let's make some null zone here between some threshold plus t and some threshold minus t. And we'll say, in this region, the received symbol is unreliable.
OK. That's called an erasure.
So we make a quantized magnitude where rk -- doesn't fit so neatly here.
Let me just say this quantized magnitude is going to give me out something which is either going to be minus 1, a 0, or plus 1.
It's going to be three levels, this is minus 1, 0, plus 1.
Or I don't want the minus 1 there.
The reliability is either a fixed reliability, or it's 0.
OK.
So now I've got a channel model that looks like this.
If you work it out, it's called the binary erasure channel with errors.
All transitions are possible.
I can send a 0,1.
I can receive, let's call it, a 0, a 1, or a question mark for -- this is called an erasure.
And this is 1 minus p minus q, and this is p and this is q symmetrically, for some p and q, which you can evaluate.
And let's choose this threshold t to optimize things and get the maximum capacity.
Now if you do the capacity calculation, you'll find there's only 1 to 1.5 dB loss.
Again, depending on the signal to noise ratio.
So in effect, you've already bought back, just with this very simple method, half of the loss that you inflicted on yourself by making hard decisions.
So making erasures is a good first step towards correcting this problem.
And let's think about it again for this simple code.
Suppose I establish these thresholds at plus t and minus t. Plus t and minus t. So now when I consider the problem that the decoder has, I have nine possible regions, all right, each with three possible decisions in two dimensions.
And so what's my decision rule going to be now?
For this 0, 0 decision, I'm going to include, of course, this region.
But now if I land in this region, that means one of the received symbols was erased, but the other one gave me a definite indication.
So the weight of evidence still goes up here.
And similarly, like so.
So my decision region for 0 -- definitely for 0, 0 is this region.
And my decision region definitely for 1, 1 is the symmetric region.
And, of course, I still have three regions where I really can't say anything.
This is two erasures.
This is 0, 1 -- two conflicting pieces of evidence.
This is 1, 0 -- two conflicting pieces of evidence.
So I still have to flip a coin out in here.
But how I improve things, that's measured by what's the minimum distance to -- what's the minimum size of error it takes to make a decision error.
And that's going to be either this length or this length.
You see that?
Just pretty clear, intuitively?
So the game here is we first of all choose t so that these two lengths are the same.
That's done by squeezing t. You see these go in opposite directions as t is raised or lowered.
So we find the t such that these two are the same.
Having equalized t, we find some value for the effective minimum squared distance, which is between alpha squared and 2 alpha squared.
As I remember, this somehow gains about 1.5 dB.
The moral is about the same.
You can get about half of your loss back by using erasures.
And again, this holds for any code which has an even minimum Hamming distance, if you can accept this method of analysis.
All right.
So a first step, even if you want to stay in the binary world, is to allow yourself to use erasures as well.
That will claw back half of the loss that you might have incurred by using hard decisions.
And, of course, even better would be to use a more highly quantized reliability and somehow have a decoding algorithm that can use soft decisions.
Soft decisions are decisions that have reliability metrics attached to them.
And in the early days of coding, people evaluated capacity.
They evaluated this sort of thing.
And it was pretty generally agreed that eight level quantization was going to be practically good enough.
16 levels, certainly good enough.
Nowadays you typically go up to 64 levels.
You might go up to much higher numbers if you have little other things to worry about like timing recovery and so forth.
But if you're purely in a synchronized, symbol by symbol transmission, then three or four bits of reliability information are going to be enough.
So from an engineering point of view, that's a good way to go.
Now, of course, you're going to need a decoding algorithm that can use soft decisions.
So as we go along, I'm going to talk about error correcting decoding algorithms that are not much, because correcting errors only is a very bad thing to do here.
Errors and erasure.
Correcting decoding algorithms, which are fairly easy in algebraic code context.
And then finally soft decision decoding algorithms, which is the kind we really want on this channel.
We really want to use the soft decisions.
We can't afford these huge losses.
We can't even afford 1 to 1 and a 1/2 dB loss.
OK, so there's more set on this in the last part of chapter six, but that's all the time I want to spend on it in class.
Does anyone have any questions?
yes?
[INAUDIBLE] transformation before we do the quantization, can we change the shape of the [UNINTELLIGIBLE] regions such that we can further improve the [INAUDIBLE PHRASE] improve the decoding?
Well, that's an interesting idea.
But think about it from an information theoretic point of view.
Can you actually get more information by introducing a one-to-one transformation, whether it's linear or nonlinear, whatever?
The point is, the original quantization is based on a straight line, [INAUDIBLE PHRASE].
[UNINTELLIGIBLE PHRASE]
You want to put a curved line on this plane?
[INAUDIBLE]
OK. That's going to imply decisions that are not symbol by symbol decisions.
That's going to imply that you need to save r1 and r 2 in order even to just decide where you're in this and then go through some kind of quantization on the plane.
And once you've got r1 and r2 and you're trying to draw crazy boundaries, I suggest it's going to be simpler to just compute the Euclidean distance to each of these points, which results in this decision region
[INAUDIBLE PHRASE]
OK.
I don't see the rationale for it yet, but there's been a million innovations in this business, and you might have a good idea.
Any other comments?
I really appreciate comments that go anywhere.
I like to address them now when they're ripe.
OK.
So let's go on to chapter seven and eight.
[UNINTELLIGIBLE] here.
Chapters seven and eight are closely related.
This is really my bow to algebraic coding theory.
I expect that if any of you have had a close course in coding before, that it was primarily on algebraic coding theory.
How many of you have had a course in coding theory before?
One, two, three.
Not so many.
Was it on algebraic coding theory?
Yes?
I [UNINTELLIGIBLE] two, one engineering and one algebraic
Excuse me?
Two -- one for -- actually, two algebraic and one out of kind of this stuff
OK. Well, you're a ringer then.
What was the one on this stuff?
It was out of Wicker's book
Out of?
Wicker's book
Wicker's book on turbo codes
No, the data storage one
Data storage.
OK.
I don't know that book.
But it talks about sophisticated soft decision type coding, and so forth.
What was the course you took?
Algebra
Algebraic.
OK. All right.
Well, let me just make a broad brush comment, which may or may not be supported by your previous exposure to coding theory.
Which is that for many, many years, when people said coding theory, they meant algebraic coding theory, coding theory of block codes that are constructed by algebraic techniques like Reed-Muller codes, like Reed-Solomon codes, BCH codes, cyclic codes.
And if you said, I want to leaRn some coding theory, asked your graduate student to go buy a textbook for you, it's probably going to be a textbook on algebraic coding theory.
Meanwhile, there was a bunch of us who were off actually trying to construct codes for real channels.
And conceRning with really how complex this is to implement, and what kind of performance can we really get.
And we hardly ever used algebraic codes.
Initially, we used convolutional codes with various kinds of decoding algorithms, threshold decoding, sequential decoding, the Viterbi algorithm.
And these get more and more elaborate, and then finally the big step, well, the big step was to capacity approaching codes.
Which now people are starting to call modeRn coding theory.
Long, random-like codes that have an iterative decoding algorithm.
And that's what we'll get to towards the end of this course.
So from an engineers point of view, to some extent, of all this work on n k d and algebraic decoding algorithms and so forth was a massive distraction.
It really missed the point.
Sometimes because it assumed hard decisions.
Or after assuming hard decisions, it assumed bounded distance decoding algorithms.
But it was all -- from Shannon's perspective, it was too deterministic, too constructed, too structured.
You want more random elements in your coding scheme.
However, two things: one, this theory is a beautiful theory, both the mathematical theory of finite fields and the coding theory, particularly of Reed-Solomon codes, which are uniquely the greatest accomplishment of algebraic coding theory, and which have proved to be very useful.
And B, Reed-Solomon codes are something that as engineers, ought to be part of your tool kit, and you will find very useful a variety of situations.
So the objective of this part of the course is, within some proportion within the overall scheme of the course, to give you exposure to this lovely theory.
You can leaRn all about it just by reading a book or two.
Probably less than one book.
And to give you some exposure to Reed-Solomon codes, which have been used in practice.
For more than 20 years, the deep space standard was a concatenated code.
That means a sequence of two codes, of which the inner code was a convolutional code decoded by the Viterbi algorithm, which we'll talk about after this.
And then the outer code, the code that cleans up errors made by the inner code, was a Reed-Solomon code of length 255 over the finite field with 256 elements.
And that's the dynamite combination that dominated the power-limited coding world for at least 20 years.
So you ought to know, as a minimum, about Reed-Solomon codes.
So that's my objective in going through these next two chapters.
However, because these have not basically been on the winning path to get to the Shannon limit, I'm trying to limit it to three weeks max.
And therefore, that means that the presentation is going to be a lot faster than what you're used to.
I'll probably do as much in these three weeks as most people do in a term in an entire algebraic coding theory course.
That's an exaggeration -- people do a lot more -- but I try to at least hit all the main points that you need to know.
So I apologize in advance if it seems that now we're on a very fast moving train.
But you ought to know this stuff.
You ought not to spend too much time on it.
I'm trying to reconcile those two points of view.
Any questions or comments?
All right.
Let's go.
Chapter seven.
We go through a series of algebraic objects.
First, the integers, then groups.
You certainly ought to know -- you do know about integers.
You ought to know about groups, at least a little bit.
Let's see.
What's next?
Then fields, particularly finite.
Then polynomials over fields, particularly finite.
And then finally, from this we actually get to construct finite fields.
And these are all things that you'll encounter again and again, have encountered.
I think you've probably encountered everything except possibly for groups and finite fields.
You certainly encountered polynomials over real and complex fields.
And so in some sense, this is a broadening of what you already know.
So I assume we start with the integers for two reasons.
One is you already know about the factorization properties of the integers.
We're going to -- that leads to a little bit of number theory, which we're going to use when we talk about groups, finite groups, cyclic groups.
So this is just to remind you, but it's also to put up a template.
And we're going to find the algebraic properties of polynomials are very, very similar to those of the integers.
This is because they're both rings.
They're both in fact principal ideal domains.
They're Euclidean domains.
They both have a Euclidean division algorithm as their key mathematical property.
And so pretty much any time you want to develop a result about the polynomials, you can start from the comparable results that you know about the integers and just change the notation.
And you'll have a sketch of the proof for polynomials.
So it's important to remind ourselves about the integers first.
Now, when I say the factorization properties of the integers, I'm not talking about anything more complicated than what you leaRned in grade school.
I just want to introduce a few terms.
We have the notion of divisors.
a divides b means that a times some quotient equals b. I could be talking about plus or minus, positive or negative integers here.
Everything divides 0.
OK. We have the idea of units, which are the invertible integers under multiplication.
And what are those?
Which integers have a multiplicative inverse?
[UNINTELLIGIBLE] any of them?
[INAUDIBLE PHRASE]
[INAUDIBLE].
All right.
That's a little exercise that shows you that, in general, the invertible elements of any set that has an operation like multiplication form a group.
[INAUDIBLE] group thing, these clearly form a group under multiplication that's isomorphic to the binary group.
Well, in fact, we've already been using it -- F2.
OK.
So we have units, and whenever we talk about the divisors, factorization, primes, so forth, the units are a nuisance.
Because we can always multiply things by units and it doesn't affect factorization properties.
This is [UNINTELLIGIBLE] a little bit more out there.
What I eventually want to get to is -- skipping a few things that are said in the notes -- is unique factorization.
Maybe before that I should define primes: are positive integers.
The reason we stay positive is, again -- of course, we could have a negative prime integer, but let's pick a unique representative of two units.
Positive integers -- I should've said in the notes greater than 1, we don't consider 1 to be a prime -- that have no non-trivial divisors.
And, of course, every integer has trivial divisors.
Namely, plus or minus 1 and plus or minus itself.
Let me give this a name.
i.
Not a very good name.
How about n?
So if we have any integer n, it's divisible by plus or minus one, plus or minus n -- that doesn't count.
So a prime is an integer that doesn't have any other divisors than these trivial ones.
And then I want to get to unique factorization, which I'll just state, not prove.
Which is that every integer is equal to a unique product of primes.
And we always have to add up to units.
We can add as many plus or minus 1's to the product as we want, as long as we have an even number of them.
And we'll have another factorization, so here, we want to exclude the units.
And then we have unique factorization.
OK.
This is all grade school stuff, high school stuff maybe.
Again, when I go through polynomials, you'll see there is a set of concept exactly like this.
Now, where we want to get to is mod-n arithmetic, which again, you all know about, I assume.
This is based on the fact -- we can think of it as being based on Euclidean division algorithm, which basically proves that given any two integers m and n, we can write m as some qn plus r where 0 is less than or equal to r, less than or equal to n minus 1, and is called the remainder of m, modulo n. And how is this proved?
Let's say these are both positive integers.
Say m is a big integer, n is a small integer.
We just keep subtracting integer multiples of n from m until we -- you know, the remainder will start decreasing until we get the remainder in this range, and it's obvious that we can always get a remainder that's in that range.
And further, that this decomposition is unique under this restriction on r. All right.
So we write the m is congruent or equivalent to r mod n. That means it differs from r by a multiple of n where r is unique and r is in the set 0, 1, up through n minus 1, which we say are the residue classes mod n. It has to be one of those n things.
OK?
No one is having the slightest trouble with this, right?
Boring.
You've seen it all before.
I hope.
OK.
So now you ask, suppose m equals r mod-n, and some other w equals s mod-n. What is m plus w mod-n, and you quickly prove that it's equal to r plus s mod-n. By transferring, by just checking the properties of ordinary arithmetic, you can use this kind of expression: m must equal something times n plus r. w must equal something times n plus s. So if we add m plus w, we're going to get something times n plus r plus s. And that is, by definition, going to be r plus s mod-n.
So we get a well-defined addition rule for these residues, for the elements of this set of n residues Rn, or remainders.
OK, so addition is well-defined.
And similarly, multiplication is well-defined.
mw mod-n, again, you can prove -- and this is done in the notes -- that if mw mod-n can be found by taking rs and reducing it, mod-n. Maybe I should have congruent signs here, but I won't worry about it.
So this is mod-n, addition and multiplication.
So what this says is that Rn, combined with this mod-n addition operation, which in the notes I write in that way, and this mod-n multiplication operation -- this is a well-defined set of n elements with a well-defined addition operation and a well-defined multiplication operation.
And I don't prove any properties about it here, but I come back to prove that this has almost all of the properties that we want of a field.
In fact, if n is a prime, it does have the properties that we want of a field.
So we'll later prove that the finite field with p elements is simply rp with mod-p addition and multiplication.
And, of course, for the particular case p equals 2, we already have a lot of experience with this.
That's how we get the binary field.
We just take the 0 and 1, considered as residues mod-2.
And then the field addition and multiplication operations give us something that satisfies the axioms of the field.
We'll find that works for every prime, and it doesn't work for non-primes.
Can anyone quickly see why this isn't going to work for non-primes?
Suppose n equals 6.
Anyone quickly see why this set under these two operations isn't going to be a field?
This, of course, assumes that you know what a field is, which we haven't actually said yet
[INAUDIBLE]
Excuse me?
[INAUDIBLE]
There's no inverse.
In a field, every non-zero element has to have an inverse, just like in the real and complex field.
But in the integers mod-6, 3 times 2 is equal to 6, which is equal to 0.
So there are two non-zero elements that multiplied together give 0.
And that means that neither 3 nor 2 can possibly have a multiplicative inverse.
So that's why you have to have a prime.
This doesn't happen, of course, if n were a prime.
OK.
So that's just a little warm up to remind you what mod-n arithmetic is like.
And in particular, this is going to be the key element of a lot of proofs -- the Euclidean division algorithm -- both for integers and for polynomials.
OK.
So that, I hope, was review.
Now we're going to talk about groups.
So let's do it.
How many of you feel you know what a group is?
Less than half the class.
OK. Well, we've been talking about groups so far here, but let's take a little bit more of an axiomatic approach.
In the notes, I actually give two possible sets of axioms for a group.
Let me start with a standard one.
You've heard of something called the group property in this course.
Well, first of all, what am I talking about when I'm talking about a group?
I'm talking about a set, g, and I'm also talking about some operation on that set, which I'll use the mod addition circle-plus as the group property.
Even if it's multiplication or matrix inversion or -- no, it couldn't be matrix inversion.
What this means is that if we take any two elements of the group, then a plus b is well-defined.
So you can think there's a table.
Here are the elements of the group.
a, b, c, d, e. a, b, c, d, e. And at this point, all we know is that we can fill out the table.
We get something that goes in all here.
All right?
So that defines the group
[INAUDIBLE PHRASE]
That means if I give a and b, then the quantity a plus b is -- I can tell you what c is.
There's a unique element in each of the cells of this addition table, or the group table.
OK. Group property, also called closure, is the obvious one.
What's an elementary condition of what all these sums have to be for this to be a group?
Have to be all in the group.
If I say I have a group that consists of 1,2,3,4,5, 1,2,3,4,5 and my group operation is ordinary addition, then I can certainly fill out the table.
And I get 2,3,4,5,6 -- whoops.
Yeah, that's the element that's not in the group.
So in order to have a group, I want a -- I mean, it's a plus b is in the group, all a and b.
So the group is closed under addition.
This was the property we used to define linearity for binary linear block codes.
The sum of any two n-tuples had to be another n-tuple in the code.
We have the associativity operation.
This is a requirement on the binary operation, really.
a plus b plus c has got to be equal to a plus b plus c. In other words, you can get rid of the parentheses.
And this, an expression like that, is meaningful.
It doesn't matter how you group, whether you add these two things first and then that, or add these two things first and then that.
Maybe now is a good time to talk about the special property, which is not an axiom.
So let me call it x abelian, or the commutative property.
This is true if a plus b is equal to b plus a, all a, b group.
Now this is, of course, true for ordinary addition and ordinary multiplication.
It's not true for matrix multiplication.
Say if I have a group of matrices, then in general, ab is not equal to ba.
So for some groups this holds, for some groups it doesn't hold.
In this course, almost without exception, every group we're going to talk about is an abelian group.
That means that in adding things up, the order doesn't matter.
So if that were true, then it in a plus b plus c, that means that's the same as c plus b plus a, or c plus a plus b, or whatever.
All the properties that you're accustomed to from ordinary addition.
But this is not one of the group axioms, and, of course, there are non-abelian groups.
The smallest non-abelian group has size 6, and is the permutation group on three elements.
So small groups are all -- size 5 or less are all abelian.
Because that's the only way we can do the group table.
All right.
So that's terminology at this point.
It's not part of the axiom.
But that's the reason that I use a symbol that looks like a plus, is I'm pretty much always thinking an abelian group, and I don't want to trouble you with thinking about non-abelian groups.
OK. Then we have the identity property.
There is an identity, which, in an abelian group, is always called -- ordinarily called 0.
In an additive group it's always called 0.
So I'll call it 0.
In more abstract treatments, it's called 1 or E for -- what does E stand for?
Eigen something.
There's a 0 in g such that 0 plus a equals -- or a plus 0 -- is always equal to a.
The 0 is the null operator on any element of the group.
If you add 0 to anything, you get what you started with.
Clearly, we have such a thing in ordinary addition.
Suppose I was talking in the real numbers under multiplication, is that a group, you guess?
It's actually not.
Suppose I talk about the non-zero real numbers under multiplication.
Is that a group?
Yes, and its identity is 1.
1 as the thing that if you multiply it by anything, it leaves what you started with.
There's the inverse, which we don't have in the reals under multiplication.
For all a and g, there exists something called minus a -- or a to the minus 1 in multiplicative notation -- in g such that a plus minus a equals 0.
That's an additive inverse.
So in the real numbers, the inverse of a is minus a.
In the integers, the inverse of a is minus a under addition.
In the non-zero real numbers under multiplication, every non-zero real number does have a multiplicative inverse, which is called 1 over a, or a minus 1.
That's why we have to exclude 0.
0 does not an inverse.
So a good example to think about is r-star, which is non-zero reals under ordinary multiplication, is a group.
Is it an abelian group?
Is ab equal to ba?
Yeah.
So this is an infinite abelian group.
All right.
OK.
I want to show that the axioms lead to a permutation property.
Because this is something that's going to come up again and again.
One way of saying this is if I have, if a plus b equals c -- suppose I write down that symbolic equation -- then I can solve for any one of the three given the other two.
How would I do that?
If I'm given a and b, obviously, that tells me c from the table.
If I'm given a and c, then a is equal to c minus b -- I'm sorry -- b is equal to c minus a. I should say here, from this inverse, we're going to write b plus minus a in a simpler form.
It's just b minus a.
So the fact that we have an inverse sort of defines another operation, which we call subtraction, which is not abelian.
Not commutative.
a minus b is not equal to b minus a.
But it's basically implicit in this that we subtract group elements.
We can cancel group elements.
If you have a plus c equal to b plus c, then a must be equal to b.
Because you can subtract c from both sides.
All right.
OK. What does this mean that this gives us a constraint on the addition table of a group?
Suppose I take a four element group, g, and one of the elements is going to be the zero element, and the other let's just call a, b, c. 0, a, b, c. And I have a well-defined addition table, which includes, in here, only a 0, a, b, c. From the property of the identity, I can fill in some of these things.
And now what freedom do I have to fill in the rest?
How many group tables are there of size four?
Well, the main point that I want to make with permutation property is that every row and every column has to be a permutation of the four elements 0, a, b, c. Why is that?
Suppose I had another a in this column here.
Then I would have a plus 0 is equal to a, and also a plus c is equal to a.
But that would imply that c equals 0, which it isn't.
All right.
So in order to be able to solve this equation, this has to be something different from a, and likewise.
Since I have, in this case, a finite group, I basically just have to write the group elements down here again in some order.
It tuRns out that for four elements, there are exactly two ways of doing that, generically.
One is sort of the cyclic group.
I'm sorry -- a, b, c, 0, a, c, 0, a, b.
That's one way of filling out the table such that each row is a permutation of 0, a, b, c. Each column is a permutation of 0, a, b, c. So that's a legitimate group table.
Let me just, for your interest, show you that that's not the only one.
We could also do 0, a, b, c, 0, a, b, c. If I write 0 here, then I've got to write c there and b there.
Got to write c here and b there.
And then down here, the only choice I have is this.
This is called the cyclic group of four elements.
This is called the Klein four group.
And this is all there is.
Both of them tuRn out to be abelian if you check it out.
And basically, this permutation property restricts these to be the possible groups of size four, abelian or non-abelian, just to satisfy the axiom.
I actually prove in the notes that if you replace axioms a and b with the permutation property, you get another equivalent set of axioms for a group.
So the permutation of properties is very basic.
All you need is -- for a set and a binary operation to form a group, all you need is identity, associativity, and the permutation property for the group table, the addition table.
Simple proof of that in the notes.
But this observation is going to be important as we go forward.
It's a fundamental thing we're going to use.
So you see a group can't just be anything.
It already has very definite restrictions on it.
All right.
Anything else I want to say right here?
OK, subtraction, cancellation.
OK, I guess we can go into the next property, which is cyclic groups.
And I'm only going to talk about finite cyclic groups, or finite groups in general.
Now, a cyclic group is probably the simplest kind of group.
Let me first put up the canonical cyclic group, which is the integers mod-n, which means the set of residues mod-n and the mod-n addition operation.
Which is written just as Zn -- there are various notations for it.
But this, of course, we'll just write it as Zn.
The integers mod-n.
Does this form a group?
Let's check the axioms.
Certainly, let's take the original axioms.
So the Rn is closed under addition mod-n.
If we add two elements, we're going to get another element.
The mod-n addition operation is associative.
In fact, it's abelian.
a plus b plus c is b plus c plus a -- it doesn't matter what order, from the same property for ordinary addition.
Is there an identity in here?
Yes.
The 0 is one of the elements here and acts as the identity.
0 plus anything mod-n is equal to the same thing mod-n. And is there an inverse?
OK. Let's explicitly write out Rn as 0, 1 up to n minus 1.
And then I claim that if I take any of these i, then n minus i is its inverse.
n minus i is in the right range if not zero.
The inverse of 0 is 0.
The inverse of i between 1 and n minus 1 is going to be n minus i because if I add i to n minus i -- is equal to n, take mod-n, that's equal to 0.
So every element has the additive inverse.
And so the group table has a very boring form, which does satisfy the permutation property.
If I take Z5, for instance.
0,1,2,3,4.
0 plus anything looks like that.
1 times anything is just 1,2,3,4 and 1 plus 4 is 0 mod 5.
2,3,4,0,1, and so forth.
3, 4, 0, 1, 2, 4, 0, 1, 2, 3.
Each row or column is just a cyclic shift to the previous row or column.
Just fill it out.
And does it satisfy the permutation property?
Yes, it does.
This one here, for instance, you can recognize as a cyclic group table.
And so, apart from relabeling, this is the group table of Z4.
Or we take a, b, c equal to 1, 2, 3.
If two groups have the same group table up to relabelling, they're said to be isomorphic.
OK?
That's an important concept in math.
And all it means is the two groups act the same, except that we've changed the names of the group elements.
OK.
So what I'm going to show is that all cyclic -- this group is sort of cyclic because, as you see in the notes, we can think of adding 1 as just being cycling around a circle.
In this case, with 5 points on it.
And adding 2 is going two steps around the circle.
And the elements on the circle are the group elements 0, 1, 2, 3, 4.
And then we get to 5.
We identify that with 0.
So we close the loop.
One we identify with 6.
This with 7.
This with 8.
Up to as many as you like.
So that's the intuition why these are called cyclic groups.
The addition operation cycles rather than being on a line, as it would be with the ordinary integers, where we go infinitely in either direction.
So the formal definition of a cyclic group is that g has a single generator.
Call it a little g. And -- am I running over?
Close to.
Is that really right?
I only have five minutes left.
We're not going to go at the speed I had hoped.
g is -- what does that mean?
If we know that g contains g, and this is never going to be equal to 0, then we know that g contains two elements at least: 0 and g. And by the group axioms, g contains g, g plus g, g plus g plus g -- we can keep adding them.
I'm going to call these the integer multiples of g. So this will be 1g, 2g, 3g.
That's what I mean when I write an integer in front of g -- I mean g plus itself, the integer number of times.
And all those things, by the group properties, have to be in the group.
All right?
Again, restrict this to finite.
It's a finite cyclic group.
Then at some point, I have to -- this is all the elements of the group.
So at some point, I have to come along and say that ng is 0.
If just by -- when I say generator, the set generated by g is just 1g, 2g, 3g.
And that's got to be all the elements of the group, so n times g has got to be 0.
And that means, without putting too fine a point on it, that if n were 5, for instance, the group has to consist of 0g, 1g, 2g, 3g, 4g, and then 5g is equal to 0.
So we call this 0g or 5g.
And furthermore, the addition table of the group, we can, you know -- however, if we add 2 g's to 3 g's, we're going to 5 g's, which is 0.
All right?
So 2g plus 3g has got to be 5 g's, which is 0g.
So this is what the addition table of the group has to look like.
It's exactly the same, except we just make a one-to-one correspondence like that.
So the conclusion is a finite cyclic group with n elements -- I'll write the size of g by this absolute value thing -- is isomorphic to the integers mod-n -- this canonical cyclic group that we started with -- must be isomorphic to Zn with the isomorphism being given by the one-to-one correspondence.
i_g in the group corresponds to simply i in Zn, or 0 is less than or equal to i is less than or equal to n minus 1.
The addition rule is exactly the same.
OK, so this is a pretty sweeping conclusion.
It says that really the only finite cyclic group up to relabeling is Zn.
OK, so if you understand Zn, you understand all finite cyclic groups.
Just in a general group, we're going to have some element which basically represents 1, called g, and its integer multiples, which represent 2, 3, 4, and so forth.
But we can operate with a group just as though it was Zn.
So it's easy to understand finite cyclic groups.
They all just look like this.
Question?
Yeah
Is the generator unique?
No, it's not.
No, it's not.
Next time we'll start to talk about subgroups of Zn, for instance.
In the set of Z5 -- well, let me take this.
All right.
In Z5 -- let me not even do that.
In Z5, 1 is the generator, but because 5 is a prime, 2 is a generator, too.
All right?
Another way of writing.
Let's take a generator equals 2 in Z5.
Then g equals 2, 2g equals 4.
3g equals 6, equals 1 mod-5.
4g equals 3.
And 5g, of course, equals 5, which equals zero.
So 2 is also a generator.
And, in fact, 3 and 4 are also generators.
There are four generators.
This is getting us right into the number theory.
OK.
So no, g does not have to be 1. g is just something that if you take all its integer multiples, you generate the group.
OK, I guess we stop there, and we'll resume next time.
So we're slightly into chapter seven, which is the algebra chapter.
We're talking about a number of algebraic objects, starting with integers and groups and fields.
Polynomials.
Our objective in this chapter is simply to get to finite fields so that you have some sense what they are, how they can be constructed, what their parameters are, how you can operate with them by addition, multiplication, so forth, as you would expect in a field.
Subtraction, division.
And that's really all we're aiming to do.
I'm trying to give you a short course in algebra, really, in two lectures or fewer.
And clearly I'm going to miss a lot of things.
In particular, I'm not going to cover even everything that's in chapter seven, which itself is a highly compressed introduction to finite fields.
I'm trying to do this while remaining faithful to the philosophy of this course and other courses at MIT, which is that you should really prove everything and show why things are true, and not simply make assertions.
So it's a little tough and it forces me to go a little fast, but I hope that you can keep up, and especially with the assistance of the notes or the many possible other things you could read on this subject, which are listed in the notes, you'll be able to keep up.
And some of you, of course, have seen this in other places in more extended form.
Now, this will get us in a position to start to talk about Reed-Solomon codes, which are the single major accomplishment of the field of algebraic coding theory.
Certainly for getting to capacity on the additive white Gaussian noise channel and for lots of other things, they're an extremely useful and widely implemented class of codes.
And we'll be able to maybe just get to the beginning of that by Wednesday.
And then I won't be here again next week, but fortunately we have an expert on campus who is far more expert than I in Reed-Solomon codes, their decoding algorithms, who has agreed to talk for two lectures, maybe one more focused on Reed-Solomon codes and one, I hope, on his whole philosophy of life.
Perhaps.
I don't know how it's going to come out, but I'll see it on TV when I get back.
Anyway, Ralf Koetter will be lecturer for Monday and Wednesday next week.
I think you'll enjoy the change of pace.
OK.
So where are we in chapter seven?
We're not very far.
We're talking about these various algebraic objects.
We've started with integers just to get you into the feel of it.
We mainly talked about integer factorization, the Euclidean division algorithm, things that you've known for a very long time, basically here because A, we're going to be using integers as we go along, and their factorization properties, B, it's a model for polynomials, which behave very much the same way as integers because they're both principal ideal domains.
In particular, we looked at the integers mod n with the rules of mod n arithmetic, which we're going to call Zn.
This is simply 0 through n minus 1 with the mod n arithmetic rules.
And then we went on to groups.
We first gave the standard axioms for groups, and then I gave you an alternative set of axioms which focused on this permutation property.
If you add, I'm calling the group operation addition, because essentially all the groups we talk about are going to be abelian -- if we add a group element to the group, what do we get back?
We get the whole group again.
It's permuted, it's the entire group, it's in a different order.
All right?
And with this and the identity, this plus the identity and the associativity axiom are also a sufficient set of axioms for the group.
And I think this is really the most useful thing to think about with a group.
We've also called it the group property when we talked about codes.
You know, if you add the code word to all the elements in the code, you get the code back.
And you saw how useful that was for seeing certain symmetry properties of minor linear block codes.
So we even talked about cyclic groups, specifically finite cyclic groups.
And we showed that all them basically are isomorphic to z mod n. A cyclic group is defined by a single generator.
If we identify that generator with one, G plus G is identified with 2, and so forth, then we get an addition table which is exactly the same addition table as Zn.
And that's what we mean when we say two groups are isomorphic.
So all finite cyclic groups look like Zn.
You can think of them as being images of Zn.
All right?
It's the only one you need to know about.
OK.
So that's where we are any questions on this material?
Pretty easy, I think.
Terribly easy if you've ever seen any of this before.
Probably takes a little absorbing if you haven't.
OK. Now the next natural subject to talk about is subgroups.
And what is a subgroup?
A subgroup simply a subset of elements in the group which, together with the group operation already specified in G, which we're calling circle plus, is itself a group.
What does that mean?
Well, associativity comes for free, because we already have that property for circle plus in G. Obviously H has to include the identity in order to satisfy the group axioms.
And finally, the third group axiom is this permutation or group property that if we add any element of the subgroup to itself, we have to stay within the subgroup and ultimately generate the whole subgroup.
That means that subtraction, cancellation hold in the subgroup.
All right?
So that's clear.
What's an example of a subgroup?
If we have -- we talked about Z10 as the group.
What would be a subgroup?
Anybody?
[INAUDIBLE]
0 to 4.
OK.
So you're proposing that H is the elements 0, 1, 2, 3, 4 out of G. Does that work?
This doesn't include 0.
But suppose I add 3 and 4?
I assume that you have modules for a reason
No.
In this group, the group operations modulo 10 addition.
Subgroup has to have the same operation as the group it came from.
What you've got here is you've already got, in essence, a quotient group.
Or at least that's where you're headed.
So this is not a subgroup.
Thank you for the suggestion.
Fails.
Anyone else?
What?
0,1?
0,1.
OK. Let's try that.
And it contains the identity 0 plus 1 is certainly in the group.
How about 1 plus 1?
It's not in the group.
0 and 5?
That sounds more promising.
Now, 0 plus -- what's the addition table of this?
0, 5, 0, 5, 0, 5.
What's 5 plus 5?
It's 10.
But mod-10 , that's 0.
So it seems we do have a group.
And in fact, this is a finite cyclic group generated by 5, and has two elements, so it's isomorphic to Z2.
In other words, the addition table looks just like the addition table of Z2 with a relabeling.
OK. Any other subgroups of Z10?
The even integers.
There.
Now we're really smoking.
H equals 0,2,4,6,8.
Those are all the even integers in Z10.
And again, evens plus evens equal evens.
So we get a group of five elements, satisfies is the group property, and it's isomorphic to what?
Z5 -- yeah.
This is obviously the same group is 0, 1, 2, 3, 4, just doubling everything.
Operates the same way.
So it's change of labels.
OK.
So there are some examples of subgroups.
Let's take another good example.
Let's just take the set of all integers.
That's an infinite group.
Mathematicians call it cyclic, even though it doesn't cycle.
What's a subgroup of that?
Even integers
All even integers!
Very good.
Check.
Does that include 0?
Yes.
Does it have the group property, even plus even is even?
Clearly subtraction holds, and so forth.
So this is the subgroup.
Interestingly, there's a one-to-one correspondence between Z and 2Z, so the subgroup is as big as the group itself.
You get into the whole issue of transfinite numbers.
But that's not where we're headed here.
OK. We'll just keep that in mind for now.
Obviously 3Z, 4Z, 5Z, and all the multiples of n are going to be a subgroup of the integers for any integer n. So more generally, we could take H equals nZ.
What's a coset?
Coset is also, in the abelian case, called a translate of a subgroup.
A coset, for instance, is in the form H plus G for some G in the group, not in the subgroup.
Now if G is in the subgroup, we get nothing.
The coset is just H again by the group property of H. So the interesting cases are where G is not in H. Let me give you some examples.
Let's take G equals Z10 and H equals 0,2,4,6,8.
You might call that 2Z10 It's the set of all elements which are twice the elements in Z10.
OK. What is a coset?
If I add any element in this group to H -- let's add one, for instance.
So H plus 1 consists of the elements 1,3,5,7,9.
Well, that's interesting.
That seems to exhaust all of the elements of Z10.
Let's take the Z equals the Z10, and H simply equal to 0 and 5, which we might call 5Z10.
And all right.
Now H plus 0, H plus 5 is just equal to itself.
H plus 1 is equal to 1,6.
H plus 2 is equal to 2,7.
H plus 3 is equal to 3, 8.
H plus 4 is equal to 4,9.
So we begin to see some properties of cosets here for which proofs are given in the notes.
First of all is that two cosets are either the same or disjoint.
H plus 2 is the same as H. H plus 1 is completely disjoint from H. Same over here.
If I had H plus 5, that would be the same as H. H plus 6 would be completely disjoint from H and would be the same as H plus 1.
In fact, I can take any of the elements of a coset as its representative, and I'm going to get the same coset, take any element outside the coset, and I'll get a completely distinct coset.
This follows just very easily from the cancellation property.
All right?
So the cosets, the distinct cosets, form a disjoint partition of G. We certainly have a coset that contains every element of G. Just take H plus that element G. That's going to contain G because H contains 0.
So there is a coset that contains every element of G. Any two cosets, distinct cosets, are disjoint, so that's what we mean by a disjoint partition.
We list all the elements of G in this way.
And in the finite case, that gives us a early famous theorem attributed to Lagrange.
What does that mean?
This means that H has to divide G. If G has size 1,10 and all the cosets have the same size, by the way, again by the cancellation property -- this means that G has to consist of an integer number of cosets, all of which have the size of H. OK?
And therefore some integer times H is equal to H, which is the same thing as H divides G. OK?
If G is a finite group, I mean by this kind of determinant notation, the size of H. The size of H divides the size of G. Or more elegantly, the cardinality of H divides the cardinality of G. But why say cardinality when you can say size?
I shouldn't have put it here.
Where H is any subgroup, any subgroup, of course, that's finite is itself a finite group.
So that's going to turn out to be quite a powerful theorem.
And it just follows this little exercise.
And we see it's satisfied, certainly, by these two examples.
In fact, it's pretty easy to see that the subgroups of Zm are going to correspond to the divisors of m in the same way.
We're going to get a subgroup for every divisor of m, and in the case of cyclic groups, this is the way it's going to come out.
Just pick any divisor.
You get a subgroup isomorphic to Zd.
Again, this is done with more care in the notes.
Suppose we have the infinite case here.
Suppose we have Z and 2Z.
So let me draw that case.
G equals Z. H equals 2Z.
And all right.
What's H?
H is the set dot dot dot minus 2,0,2,4, dot dot dot.
So this is H. And what's H plus 1, or in fact, plus any odd number?
It's going to be the odd integers.
H equals minus 1,1,3,5, and so forth.
So again, this works in the infinite case, that the distinct cosets form a disjoint partition of an infinite group G. But of course, we don't get Lagrange's theorem as a corollary, because I already said, there's a one-to-one correspondence between Z and 2Z, paradoxically.
So the cardinality of Z and 2Z are the same.
More elegant language.
Nonetheless, you see, this is a useful partition and standard partition into the even integers and the odd integers.
And we could also write this as 2Z and this is 2Z plus 1.
So we can divide the integers into even integers and odd integers.
All right.
Now we can actually add cosets, subtract cosets.
In these cases, we can even multiply cosets.
But let's just talk about staying within the group operation.
Given any abelian -- let's continue to say G is abelian -- how would you add two cosets?
Coset addition is defined as follows.
H plus G -- we want to have some addition operation, which I'll just indicate by plus -- H plus G prime, what's going to equal?
We define that to equal H plus G plus G prime.
And that makes sense.
I mean, if we really write all this out, we get H plus H plus G plus G prime.
H plus H is just H again.
So it's sort of a proof of that.
If you go through in detail, any element of this coset plus any element of this coset is going to be an element of this coset.
So this itself is a coset.
So we now have an addition table for cosets.
So in fact, it's easy to show that the cosets themselves form a group called a quotient group.
Start over here.
Cosets of H in G under coset addition -- that's going to be Z -- form a group.
We just defined coset addition.
And it's easy to check that they themselves form a group called the quotient group.
Usually written G slash H and pronounced G mod H. And we can can mod out anything.
We do the arithmetic in these quotient groups by modding out any elements of H. And let's take an example.
Here's a good one.
For example, the cosets of 2Z in Z, namely, 2Z and 2Z plus 1.
Under coset addition.
What is coset addition here?
If I add any even integer to any even integer, I get an even integer.
Any odd to even, I get an odd integer.
Odd to odd gives that.
I mean, odd to even gives that, and odd to odd gives me back evens again.
So that's the addition table.
The subgroup itself, x is the identity.
This is clearly isomorphic to Z2 with this addition table.
In fact, this is a very good way of constructing the cyclic group Z2, or more generally, Zn.
So this would be called Z mod 2Z.
And it's isomorphic to Z2, or in general, Z mod nZ is isomorphic to Zn.
A very good way of thinking of Zn is as residue classes or equivalence classes, modulo n. The cosets of nZ.
are nZ itself, nZ plus 1, nZ plus 2, up to nZ plus n minus 1.
And if you add them together, they follow the rules of mod n arithmetic.
If you just add the residues together and then reduce mod n. So we can think of Zn as being -- a coset is an equivalence class.
It's all the elements of the group that are equivalent, modular of the subgroup H. Or in the case of integers, it's all integers that are equivalent modulo the subgroup nZ.
They have the same remainder after division by n. They have the same residue.
These are all equivalence class notions.
And how do you add them?
You add them in the ordinary way, and then you take everything modulo m. In other words, you do mod-m arithmetic.
OK.
So I didn't quite go to quotient groups in the notes, but perhaps I should have.
Probably I should have, because this is maybe the most powerful idea in group theory, and certainly closely related to this little bit of number theory that we're doing in the integers mod n. And of course, it has vastly greater applications than just that.
OK. And you could do the same thing over here.
This is basically doing the same kind of thing.
But I won't take time to do that.
So here's another view of the integers mod-n that may be helpful as we go forward.
All right.
I think that's all I want to say about that.
Yeah.
Good.
All right.
Here would be a good place to start on fields.
OK. Fields.
Obviously very important in algebra.
Fields are like groups, only more so.
Groups are a set of elements with a single operation, which we've been calling addition.
A field is a set of elements with two operations, which we'll call addition and multiplication.
So what do we have?
We have a set of elements F. We're going to be particularly interested where the set is finite.
Those are called finite fields.
And we're going to have two operations, which I'll continue to write addition by simple plus and multiplication with an asterisk, just to be very explicit about everything.
And after a while, you can write these things as you would in ordinary arithmetic, with just ordinary plus and juxtaposition for multiplication.
And what are the axioms of a field?
They're presented in an elegant way in the notes, which obviously go back a long way, but I got from Bob Gallager, and I like.
All right.
Under addition -- so let's write it this way.
Now, just considering the addition operation is an abelian group.
Commutative group.
Which means it has an identity, and we will continue to call that identity 0.
Just as we do in the real field, let's say.
OK.
Think of the real field, if you like, as a model for all fields here.
All right.
So that's axiom one.
Axiom two.
If we take the non-zero elements of the field, which I write by F star, explicitly that's F not including 0 -- not a very good notation, but I'll use it -- and the multiplication operation, that, too is an abelian group.
So this, of course, is why we spent a little time on groups, abelian groups, so we'd eventually be able to deal with fields.
And its identity is called 1.
Meaning that under multiplication, 1 times anything is equal to itself.
And then we have something about how the operations distribute, the usual distributive law that A times B plus C is equal to A times B plus D times C, where I've written out all of these.
So this is how addition and multiplication interact again in a way that you're accustomed to, and after a while, you don't need to write all these parentheses.
OK.
So that's actually almost a simpler set of axioms than for groups, once we understand the group axioms.
And so let's check.
Is the real field, is the set of all real numbers under ordinary real addition and multiplication, is that a field?
What do we have to check?
We have to check that under addition, we're going to take the additive identity as being equal to 0.
Under our reduced set of group axioms, the main thing we have to check is if we add any real number to the reals, we get the reals again and the one-to-one correspondence is the permutation.
Is that correct?
Yes, it is.
And so this is OK. Now under multiplication, if we take the non-zero real numbers, here's the question.
If I have some alpha not equal to 0, is alpha times the reals, not including 0 -- the non-zero real numbers -- equal to R star?
And here I'm really implying a one-to-one correspondence.
So I might write it more that way.
I pose this question rather abstractly, but you can easily convince yourself that it's true.
Any non-zero number, if I multiply it by any non-zero number, I get a non-zero number.
Is the correspondence one to one?
Yes, because I can divide out this number and get alpha.
So alpha x on R star by multiplication to give R star again, and this is a one-to-one correspondence.
But it's obvious why I have to leave out 0, right?
0 times any real number is 0.
So at O, R star is simply equal to zero set.
So we always have to leave out 0 from multiplication.
0 doesn't have an inverse.
Everything else does have an inverse.
Under the standard group operations, that's what we have to check.
That would be the alternate question, does every non-zero real number have an inverse?
That's easier to see, the answer is yes.
Multiplicative inverse.
Inverse of alpha is 1 over alpha
[INAUDIBLE]
Yes.
I've used the alternative set of axioms, including the permutation property.
To check whether there's an abelian group, I've asked if alpha R star is the permutation of R star.
And without going through details, I claim it is.
Thank you.
All right.
So we checked that.
Of course, the distributive law holds.
So the real field is a field, which you probably were willing to accept on faith, anyway.
Similarly, you go through exactly the same arguments for the complex field.
What about the binary field?
We think we understand that by now.
Here the operations are mod-2 addition and mod-2 multiplication.
I've written down explicitly the addition and multiplication tables.
Under addition, we simply have Z2 again.
F2.
The additive group of F2 is simply Z2.
We forget about multiplication.
We've seen quite a few times now that that's a group.
Under multiplication, what are the non-zero elements of F2?
Just one element.
one.
This includes the identity?
Yes.
Is it a group?
Yeah.
It's a trivial group.
1 times 1 equals 1.
1 under multiplication is isomorphic to the trivial group 0 under addition.
Its group table is 1 times 1 is 1.
Sure enough.
That's the identity permutation.
Sometimes when things get too trivial, it's a little hard to check.
But yes.
And distributive is easy to check.
OK?
So that's all it takes to define a field.
Of course, by the inverse property, when we have addition, this also implies an inverse and a subtraction operation and a cancellation and additive identities.
We have a field element on both sides of a plus b equals a plus c, then b plus b equals c. That's what I mean by cancellation.
Similarly under multiplication, we get a multiplicative inverse.
1 over alpha, for any alpha in F. We, therefore, are able to define division.
And we have cancellation for multiplicative identity.
So we immediately get a lot from these group properties.
We get all the properties you expect of fields.
You can add, subtract, multiply, or divide, all in the usual way that we do over the real field.
OK. Let's stay over here.
I think my next topic is prime fields.
Yes.
So prime fields.
When we talked about the factorization properties of the integers, we talked about primes p. And now I'm going to talk about Fp is going to be a field with a finite number of elements where the number is a prime.
So what are the elements in this field going to be?
Are they simply going to be 0, 1 up through p minus 1 again, the same elements as were in the cyclic group with p elements where I'm restricting m now to be a prime p?
And for my addition operation and my multiplication operation, I'm going to just let these be mod-p addition and multiplication now.
And I claim that this is a field.
So actually, the proof follows very close to what I just did for F2.
F2 is a model for this.
But it's a little harder to check this.
Under a, under the addition operation, Fp really is just Zp again, so that's OK. That's an abelian group.
Zmod-p. Or the quotient group, Zmod-pZ.
We can also again think of this as Zmod-pZ, if we want.
So everything is going to become mod-Z.
That's a very useful way of thinking of it.
So we're really thinking of these as remainders or representatives of the residue classes of pZ in Z.
This is pZ, this is pZ plus 1 up to Z minus 1, up to pZ plus p minus 1.
This is the same as pZ minus 1.
OK.
The real question is, if we take the non-zero elements of Fp, is this closed?
And does every element have an inverse?
Or equivalently, when we multiply by a particular element, do we just get a permutation of this?
The reason that p has to be a prime -- let's suppose we take two of these things, a and b, and we multiply them.
What's the multiplicative rule?
a times b is just ab mod-p. That's what I defined multiplication as.
Now the question is, could that possibly be 0?
Which is the same as saying, could a times b be a multiple of p, where a and b are taken from the non-zero elements of the field?
And here, because p is a prime, it's clear that you can't multiply two non-zero numbers which are less than p and get a multiple of the prime p. If p were not a prime, then you could.
If we took n equals 10 again, let's say, and we multiplied 2 times 5 from the 10 elements of these residue classes, 2 times 5 is, in fact, equal to 0 mod-10, and therefore Fp star, or F10 star, would not be closed under multiplication.
We would get a 0.
But in this case, we easily prove, because it's a prime, that it's not equal to 0.
And therefore it's in Fp star, so it is closed under multiplication.
And the other thing we have to check is that it's one-to-one.
In other words, can a star b equal to a star c, and by the cancellation property, which holds in -- Sorry.
We've got to establish the cancellation property holds under mod-b arithmetic, but it does, and so we get the cancellation property, that this is true if and only if b equals c. So in other words, as we run through all of these multiples for any particular alpha, we're going to get a permutation.
We need to get the same set back.
Everything is finite.
We're going to get a bunch of distinct elements of the same size as the set itself.
Therefore, it has to be the set again.
I haven't said that very well again.
That's why we have notes.
It's written up correctly in the notes.
But we have basically checked everything that we need to check, showed that Fp star is an abelian group under multiplication when p is a prime, and clearly not when p is not a prime
[INAUDIBLE] the inverse, there is inverse of a?
Yes.
But basically, we have to prove that if I take any of these, if I take a particular one, say, alpha, and multiply times all of them in Fp star, that I'm just going to get Fp star again.
And to prove that, I have to prove that alpha times a is not equal to alpha times b if a not equal to b.
That's all I need to prove, right?
And that comes from the properties of mod-p arithmetic.
That is what is to be proved.
I need to use mod-p arithmetic to prove that
[INAUDIBLE]
Oh.
I have to check the identity is in here.
The identity is in here.
It has one.
I'm sorry.
I should have checked that, too.
But 1 is the identity for multiplication.
And then from this property, since we multiply alpha p star, we get Fp star again.
That includes one.
All right?
So it's got to be one of these guys which, times alpha, gives 1, and that shows the existence of an inverse.
So you can do it any way you want.
But the key to the proof is to prove this, and that's why I focused on the permutation property.
Permutation property is really what you prove to demonstrate this.
OK?
Good.
Everyone seems to be following closely here.
Any further questions?
This is important, because we've got our first finite field.
The integers mod-p are a finite field of size p for any prime state.
We've got F2, F3, F5, F7, and so forth.
OK. Further on this subject.
We have two closely related propositions.
One, every finite field with prime p elements is isomorphic to Fp.
So if you give me a finite field, you tell me it has p elements, I'll show you that it basically has the same addition and multiplication tables with relabeling.
And secondly, every finite field with an arbitrary number of elements, for every finite field, the integers of the field form a prime field for some P. You understand my abbreviations.
And the proofs of these are very closely related.
What do I mean by the integers of a field, of a finite field?
OK. Well, let's start from the very most basic thing.
What do we know?
We know that the field contains 0 and 1, and those are going to be two of the integers of the field.
So 0 and 1 are in F. Let's use the closure under addition.
Clearly 1 plus 1 is in F. We're going to call that 2.
1 plus 1 plus 1 is in F. We're going to call that 3.
And so forth.
And of course, since the field is finite, eventually this is going to have to repeat.
And from the fact it repeats, you're basically going to show that at some point, one of these is going to be equal to 0.
So there's going to be some n. The first repeat is going to be n equal to 0 in F. OK.
So that's what I mean by the integers.
The integers clearly form a subgroup of the additive group of F, to form a subgroup under addition.
And in fact, a cyclic subgroup.
I'm skipping over some of the details here, but that's a claim at this point that I haven't really demonstrated.
But just from a subgroup property, let's attack number one up here.
Suppose we have a field with p elements, and the additive group of the field has p elements.
It consists of the same elements.
And by Lagrange's theorem, what are the possible orders of that subgroup?
What are the possible number of elements in that subgroup, the sizes of the subgroup?
The order of a group is its size.
Well, it has to divide p. There aren't many things that divide a prime p. There's 1 and there's p. OK?
So the subgroup either has a single element or it's all of the group.
If there's a single element -- let's to keep an open mind here -- then what that means is that if I take G and I add it to itself, since it's a subgroup, it has to give an element of the group.
But there is only one element of the group.
Let's say G is the single element in this subgroup.
I guess it could only be 1.
Let's start out with one.
So suppose one is the only element of the subgroup.
Then I get the equation 1 plus 1 equals 1, which by cancellation implies that 1 equals 0.
OK, well, that can't be true.
In a field the multiplicative identity is not the additive identity.
So that can't be true.
That would only be true if we had a field with one element, and fields implicitly always have at least two elements, 0 and 1.
F2 is the smallest finite field.
I suppose we could set up a single element that sort of satisfies all these axioms, but then, what is the multiplicative group?
All right.
So this can't happen.
So that means this subgroup has to have p elements.
It has to consist of all the elements of the field.
So that means the integers are all the elements of the field.
But now the isomorphism, then, is that this is isomorphic Fp under the isomorphism.
This corresponds to 2, this corresponds to 3, and so forth, in Fp.
You can see, you know, 2 is 1 plus 1, 3 is 1 plus 1 plus 1, so 2 plus 3 is going to be 5 1's.
Mod size the field, whenever this cycles.
So this is going to have to be p, and basically, that shows --
[INAUDIBLE] to prove that it is isomorphical [UNINTELLIGIBLE] multiplying 1 plus 1 into 1 plus 1 plus 1.
But it is typical --
I really have only used the additive property here.
I don't think multiplication enters into it.
OK, here's where the multiplicative property adds in.
I have to prove not only that this is isomorphic to Fp as an additive group, but the multiplication tables are isomorphic under the same relabeling.
So for that, I have to show that 2 times 3, when I've defined 3 and 3 this way, gives me the same result as multiplying 2 and 3 in F p mod-p.
But again, I could do this just because sort of mod-p commutes with addition and multiplication.
If I multiply 1 plus 1, two 1's times three 1's, so I'm going to get six 1's, and that's exactly what I get in Fp, reducing everything mod-p.
So I have to check that also to prove this isomorphism.
And this is done carefully in the notes.
The distributive law holds because the distributive law holds for sums of n 1's.
1 plus 1 times 1 plus 1 plus 1 plus 1, it's going to be the same regardless of where you put it in, how you put the parentheses.
OK.
So with some sorry hand-waving here, we've basically given the idea of how to prove this.
It's basically Lagrange that the additive subgroup has to be of size 1 or p, and we prove quickly that actually p is the only case that works.
And then we extend all the arithmetic properties by just observing they'll hold for 1 plus 1 plus 1.
Yeah?
[INAUDIBLE] the line 1 plus 1 plus 1 like that?
Might we [UNINTELLIGIBLE PHRASE] 1 is equal to 0.
What is actually [UNINTELLIGIBLE PHRASE]?
Should we state that 1 has to be different than 0?
Yeah.
I guess I could simply get around that by stating that the multiplicative identity has to be different from the additive identity.
It clearly follows from this, and I think I put it as an exercise, 0 times any group element has got to be equal to 0.
So this is how 0 behaves under multiplication from these axioms.
But 1, as the multiplicative identity, has to satisfy that rule, so clearly, 0 cannot equal to 1.
Unless, in some trivial sense, there is only one element in the groups if there's any non-zero element.
So this implies that 0 is not equal to 1.
Just could have included that as an axiom.
Yeah?
Assume a and b [INAUDIBLE]
Excuse me?
Assume a and b, [UNINTELLIGIBLE] does not include 0?
Correct.
Yeah, you're right.
OK.
So it follows from this that 1 is not 0.
Yeah.
I'm sorry I don't personally have a lot of patience for these fine details.
For mathematicians, it's important to keep them all in mind.
But my effort is to make these propositions plausible enough so that you can believe them, and you can go back and read a real proof and see that the proof must be correct intuitively, without just mechanically checking it.
OK. Let me just again outline how this works.
It's very similar.
Again, given any finite field, if we define the integers of the field in this way, we show that eventually they form a cyclic group.
Their cyclic group is something that has a single generator.
The generator is 1.
So eventually it has to cycle for some number n. Now could n be a non-prime?
No, because this is a field, and if n were non-prime, then we would be able to find two integers that multiplied together gave 0.
And that's forbidden by the axioms of the field.
So the only possibility is that n is a prime, and in that case, we have found what's called a subfield, a subset of the elements of the field which itself is a field under the field axioms.
And the field has p elements, and we already know that every finite field with p elements is isomorphic to Fp.
So it can only be that the set of integers is a subfield which is isomorphic to Fp for some prime p. OK?
So within any finite field, we're always going to find, just by writing out the integers and seeing how they behave under the additive property that there are exactly p of them.
So every finite field has a prime called the characteristic.
The prime characteristic of the field.
This is defined as the characteristic.
The size of the integer subfield is the characteristic of the field.
And it has an interesting property, a very important property.
Suppose we take this p and we multiply it by any field element called data in the field.
By the distributive law, this is just equal to 1 plus 1 plus -- so what do I mean by this?
I mean beta -- whenever I write an integer times a field element, I mean beta plus beta plus so forth, p times.
But this is equal to 1 plus 1, so forth, by the distributive law, I guess, times theta, p times.
And what is this equal to?
This is equal to 0.
So this is equal to 0 times beta, which fortunately I just told you always must equal 0.
OK.
So the conclusion is that if we add any field element to itself p times, we're going to get 0 for all beta in the field where p is the characteristic of the field.
Now in digital communications, we're almost always dealing with a case where the characteristic of the field is going to be 2.
The prime subfield is just going to be the two elements 0 and 1.
1 plus 1 is going to be equal to 0.
So subtraction will be the same as addition.
And in that particular case, we will have that the sum beta plus beta of any 2 field elements in a field of characteristic two is going to be equal to 0.
Just as we had for code words in binary linear codes.
Binary linear codes are not fields, they're vector spaces, but it's a similar property here.
You add any element of field of characteristic 2 to itself, and you're going to get 0.
So this shows that addition is the same as subtraction.
Beta equals minus beta in a field of characteristic 0.
Which is a little bit more general.
OK.
So we have some fields now, and we find these fields are the only field of prime size, and that every finite field has an important subfield and a prime subfield.
And that has important properties, consequences for the field itself.
All right.
I think that's everything I want to say about prime fields.
Now we go on to the next important algebraic object, polynomials.
And again, it's hard to know just how detailed to be, because of course you've all seen polynomials, and you intuitively or formally know something about their algebraic properties, their factorization properties, and so forth.
So I'm going to go pretty quickly, and this will be in the nature of a review.
A polynomial -- maybe the simplest way -- how do you define a polynomial?
What does it look like?
It looks like this.
F0 plus F1 times x plus F2 times x squared, so forth, plus Fm times x to the m. That's what it looks like if it's a non-zero polynomial.
Or even if it's just -- you could consider all 0 coefficients to be the zero polynomial.
But in general, the convention is, we write f of x equal to that if x is non-zero.
What are these f's?
These are called the coefficients of the polynomial.
And where do they live?
We need the coefficients to be in some common field.
You've often seen these in the real or complex field.
Here they're going to be in finite fields.
In particular, very shortly, they're going to be prime fields.
But in general, we'll just say these f's have to be in some field.
So we're talking about a polynomial over F where F is some field.
So there's always some underlying field if there isn't a vector space.
Some similarities between this and vector spaces.
And we usually adopt the convention that Fm is not equal to 0.
All right?
So we only write the polynomial out to its last non-zero coefficient.
In general, this could go up to an arbitrary degree, but well, a polynomial, by definition has a finite degree, which means it has a finite m for which the polynomial can be written in this way.
And if the Fm is the highest non-zero coefficient, then we say the degree of f of x is m. So all polynomials have a finite degrees, except for 1.
There is the zero polynomial, which we have to account for somehow.
And here we'll just call it f of x equals 0.
Informally, it's a polynomial, all of those coefficients are 0.
But we'll just define it by its properties.
Zero polynomial plus any other polynomial is equal to the identity under addition for the polynomials?
What's the degree of the zero polynomial?
Anyone have a definition for the degree of the zero polynomial?
Is this well-defined?
Undefined?
OK. Well, I'll suggest to you that it should be defined as minus infinity.
This actually makes a lot of things come out nicely, but it is on the other hand, you don't have to do this.
If you like, you can define the degree of 0 to be minus infinity.
It's just a convention.
OK.
So the set of all polynomials over F0.
What's x here?
I've got this thing x.
What should I think of this as being?
Is this an element of a field, or is it something else?
In math, it's usually called an indeterminate.
It's just a placeholder.
It's something else we stick in order to define the polynomial.
It doesn't have a value, in principle.
A comment is made in the notes that with real and complex polynomials, you often think of x as being a real or complex number.
In other words, you evaluate the polynomial at some alpha in the real or the complex field by just substituting alpha for x.
And in fact, two polynomials are equal if they evaluate to the same value for all the alphas and they're unequal if not true.
When we get to finite fields, it's important this be an indeterminate.
Because consider x and x squared as polynomials over the binary field F2.
What are the values of these?
We'll call this F1 of x equals x, F2 of x equals x squared.
Then F1 of 0 is 0 and F1 of 1 is 1.
Right?
If I evaluate these at field elements, the two field elements, I get 0 and 1.
F2 of 0 is equal to 0, and F2 of 1 is equal to 1.
But these are not the same polynomial, all right?
So x is not to be considered as a field element.
It's to be considered just as a placeholder, a way of holding up these polynomials.
It's actually most important in multiplication.
But we gather common terms in x.
This is the multiplication rule.
But it's just something we introduce to define the polynomial.
All right.
So the set of all polynomials over F in x, or in x over F, is simply written as F square brackets of x.
That's the convention.
So that's what I will write when I mean that.
And it includes all sequences like this of finite degree, starting at 0, ending somewhere.
And also the zero polynomial.
How do you add polynomials?
Let's talk about the arithmetic properties of polynomials.
You know how to do this.
If you have F0 plus F1 plus F2 and so forth, you have some other polynomial doesn't have to be the same degree -- G of x is G0 plus G1 x, up there -- how do you add these together?
Component-wise.
Sum is F0 plus G0 plus F1 plus G1 x plus 2x squared and so forth.
That's an example.
So you basically insert dummy 0's out here above the highest degree term in G. You add them up component-wise.
The addition operation is where?
In this field, you have addition operation in that failed field, so you can do this.
And you get some result which is clearly itself a polynomial.
If all the coefficients are 0, you declare that the result is 0.
Otherwise the result has some degree.
If you add two polynomials with different degree, the degree of the resulting polynomial is going to be the higher degree.
If they have the same degree, you could get cancellation in the highest order term, and get a result which is of lower degree, all the way down to 0.
All right?
So addition is component-wise the degree of the result is less than or equal to the max degree of the components.
So we do addition.
How do we do multiplication?
You all know how to do polynomial multiplication.
Example.
F0 plus F1 of x times G0 plus G1 of x.
What do you do?
You just multiply it out term by term.
F0 G0 plus F1 G0 x plus F0 G1 x plus F1 G1 x squared.
You can combine these two together.
And that's your answer, which clearly is a polynomial.
So that's one way of doing it, is multiply out term by term, collect the terms.
The result of this is that what you get is a convolution for each of the coefficients in the new polynomial.
You convolve, just by the ordinary rules of polynomial addition, you can basically turn this around, you convolve it, and you'll get these coefficients.
This is written out in the notes.
So we know how to do polynomial multiplication.
What are some of its properties?
How is this defined again in F?
We see we're now going to need the multiplicative of properties of our field F. All of these products and ultimately convolutions are performed in F. That's why we did these coefficients to be in a field, so we can do all these things.
All right.
What are some of the properties?
What is the degree of the product of two polynomials?
It's going to be the sum of the degrees, right?
Provided that both the polynomials are not 0.
The highest non-zero term is clearly going to be a term of this kind, and it's going to be a coefficient of x to the sum of the degrees.
And since F1 and G1 are both non-zero, by the way we write polynomials, them this highest order term is going to be non-zero.
But we also have to basically have another rule that 0 times f of x is equal to 0.
So that's the way we multiply by 0.
And how does the degree formula work in this case?
Well, this is why I defined the degree of 0 to be minus infinity.
So the degree of the product 0 times f of x is -- I've defined this to be the degree of 0 plus the degree of f of x.
This is finite.
This is minus infinity.
So the sum is minus infinity, and so it holds.
OK?
That's why we defined the degree of 0 to be minus infinity.
We don't have to, but it's just so that the sum of the degrees formula continues to work, even if we're multiplying by 0.
Is there a multiplicative identity for polynomials?
Yeah.
What is the multiplicative identity?
1.
OK.
So one times f of x is equal to f of x.
So that's one of the properties of a field.
Gee.
The set of all polynomials over F in x, is this a field?
Let's go back and check our field axioms.
Under addition, does f of x form an abelian group?
Does it have the group property?
If we add two polynomials, do we get another polynomial?
Do we have cancellation, if F1 plus F2 equals F1 plus F3, is it necessarily true that F2 equals F3?
Yeah, it is.
In fact, this looks very much under addition.
These look like vectors.
If we confine ourselves to the set of all polynomials of degree m or less, we can add them.
And it looks very much like the vector space f to the m. Set of all polynomials of degree m or less corresponds to the vector space Fm, which does have the group property under addition.
So in fact, for f of x, yes.
It is an abelian group under addition with identity being the 0 polynomial.
And all of f of x is an infinite abelian group.
If we just take polynomials of degree m or less and restrict Fp to be a finite group, then there are only p to the m elements, and we would have a finite abelian group.
OK. Well.
Are the polynomials an abelian group under multiplication?
It has an identity.
It has all the arithmetic properties you might expect.
It's commutative.
f of x times g of x is equal to g of x times f of x.
So it's abelian.
It has cancellation.
f of x g of x is equal to f of x h of x.
That's true if and only if g of x is equal to h of x.
Is it missing anything?
Inverse, right?
Very good.
Just like the integers.
Not all the polynomials have inverses.
Which are the polynomials that do have inverses?
[INAUDIBLE]
No.
There are more than that.
So now we're getting into polynomial factorization.
And the particular topic is units, which are the invertible polynomials.
And what are they?
Does the 0 polynomial have an inverse?
We're a little unsure, are we?
What could it possibly be?
If it had an inverse, this would mean 0 times f of x is -- well, 0 times f of x would have to be a one-to-one map to all of f of x.
But it isn't.
It simply maps to 0.
Doesn't have an inverse.
What about the non-zero polynomials that have degree 0?
In other words, the degree 0 polynomial, is simply something that looks like this.
fx equals f0, where f0 is not equal to 0.
Is that invertible?
Yeah, because f0 is in the field, and it has an inverse.
So this has inverse 1 over f of x, if you like, equals just f0 minus 1.
1 over f0.
By the rules, you multiply these two things, and you get 1.
OK.
So the units in this -- well, I'm sorry.
Take a degree 1 or a higher polynomial -- does that have an inverse?
Let's suppose we take a degree 1 polynomial -- say, F0 plus F1 x -- and what I want to find is its inverse.
Let's call it g of x.
Is it possible to find a g of x such that the product of these two is 1?
Clearly not, because by our degree rule, what is the degree of this product going to be?
The degree is going to be the sum of this degree plus this degree, the degree of f plus the degree of g, which is going to have to be at least 1.
Provided that g of x is not 0, but clearly g of x is not the solution we're looking for here, either.
So this can't be true, and the invertible polynomials are the degree 0 polynomials, which means they're basically the non-zero elements of F. Yes.
Considered as polynomials
And how are the [UNINTELLIGIBLE PHRASE]?
Yes, indeed
[INAUDIBLE]
Ah, no.
We haven't introduced modulo polynomials and right.
The powers and the indices are the integers from 0 up to some finite number.
OK.
It's time to quit.
We'll finish this.
We'll, I believe, to be able to certainly finish chapter seven, maybe get a little bit into chapter eight, next time, on Wednesday.
And we'll see you then.
So if you want to hand in your problem sets, we have three handouts.
The old problem set solutions, the new problem set, and we're also handing out chapter eight today.
Of course, these are all on the web.
Reminders of previously announced events.
Next week I'm going to be away.
Ralf Koetter will be a much-improved substitute.
He'll be talking about Reed-Solomon codes and Reed-Solomon decoding.
He's one of the world experts on these things.
He's a great lecturer.
So you're really lucky, I believe.
I will, of course, look at the video afterwards and see how he did, but I'm sure he'll do it better than I would have done it.
And midterm is in two weeks.
That's just a reminder.
When I come back, we'll have a Monday class to go over anything that Ralf may have missed, or really anything in the whole course up through chapter eight.
You can bring questions in to that.
I don't know how much I feel I'll need to embellish and cover, so I don't know how much time there will be for questions.
But that can be a fairly free form lecture.
And then Ashish is going to run a review session on the Tuesday with exactly the same purpose.
So hopefully that'll put you in good shape for the midterm.
We have to make up the midterm.
No idea what's going to be on the midterm yet.
OK. Any questions about these things?
Homework processes going OK?
Office hours?
Solutions?
Everything seems to be all right so far?
Good.
We're in chapter seven.
I certainly plan to complete it today.
I'm not going to cover all of chapter seven, but the key elements that I want you to know.
You're responsible only for what's covered in class, by the way.
We've talked about a lot of algebraic objects.
Our main emphasis has been to get the finite field, and at this point, we have prime fields.
Fields with a prime number p of elements.
And we find that we can construct such a field by taking the integers mod p. You can write that as z mod p z.
You can equally well consider this as the equivalence classes of integers of the cosets of pz nz which are each identified by a remainder or residue r between 0 and p minus 1.
OK.
So basically, the elements of the field are these remainders or these cosets.
There are p of them.
We do arithmetic by simply doing mod p addition and multiplication.
So if you understand arithmetic mod p, you understand this field.
Yes?
Is Fp and Zp isomorphic?
As an additive group, it's isomorphic to Zp.
Zp we use that notation for the group.
We use this notation for the field.
And I write Zp is isomorphic to Z mod pZ as a quotient group.
Here I've added in the multiplication operation mod p to make this a field.
So this is somewhat more than just a quotient group.
Sorry.
The notation is supposed to be more suggestive than precise.
This is not a math class.
I hope it's helpful rather than otherwise.
OK.
So today, we're going to construct all the rest of the finite fields.
By the way, we showed that these are the only fields with a prime number of elements.
Today we're going to construct fields with a prime power number of elements in a very analogous way, and it will turn out -- although I'm not going to prove this -- that these are the only finite fields.
Well, these are generalization of this, so all finite fields have a prime power number of elements and are basically isomorphic to one of these fields.
How do we construct this field?
To give you a preview, very analogously to the way we constructed this field.
You'll see that the character of the arguments is very much the same.
And that's because there is a great algebraic similarity between the integers and the polynomials over a field.
And we'll talk about that in a second.
Basically, they're both countably infinite rings, or infinite rings.
Countably infinite if it's over a finite field.
And they both have analogous factorization properties.
They're both unique factorization domains, would be one characterization, algebraically.
All right.
How do we construct this field?
We're basically going to construct it by taking the set of all polynomials over Fp which is denoted by Fp square brackets x.
And we're going to take that mod the set of all polynomials that are divisible by G of x, or G of x times the set of all polynomials.
The set of all multiples of G of x.
Or more simply, just mod G of x.
Where we're going to take G of x to be a prime polynomial.
And I guess I have to add that the degree of G of x is going to be equal to m. So we basically need to find a prime polynomial degree m. Then we take the set of all polynomials modulo this prime polynomial.
They're going to be exactly p to the m residue classes, or equivalence classes, or remainders modulo g of x.
And we'll use the arithmetic operations from mod G of x arithmetic, addition mod G of x, multiplication mod G of x, and we'll find that the resulting object has satisfied the axioms of the field.
OK?
So that's where we're going.
You with me?
OK.
So let's talk about factorization.
And I think it makes it easy to understand polynomials if we understand the analogies to the integers.
In particular where I'm headed is unique factorization.
Both the integers and the polynomials have unique factorizations.
The integers into product of integers, and any polynomial is uniquely factorizable into a product of polynomials.
Now I wasn't quite precise when I said that.
We have to be a little bit more precise when we talk about factorization.
In particular, we have a certain kind of trivial factorization that involves units.
Units, if you remember, are the invertible elements.
We're in a ring, so not every element has an inverse, but some of them do.
And in the integers, what were the invertible integers under multiplication?
Everything here is about multiplication, you know?
A ring is something that satisfies all of the properties of the field, except that some of the elements don't have inverses, so that division is not necessarily well-defined, even for non-zero elements.
OK?
Sorry.
I think I said that before.
But we're not emphasizing that these are rings, although they are.
That's an informal definition.
So in the integers, of course 12 doesn't have a multiplicative inverse, but it's an integer.
But which integers do have multiplicative inverses?
Plus or minus 1.
So if an integer is divisible by n, it's also divisible by minus n. All right?
Trivially.
These are the only ones in the integers.
And last time we actually talked, if I remember correctly, about the units and the polynomials.
Which polynomials have inverses?
Excuse me.
The degree 0.
Thank you.
The degree 0 polynomials have inverses because they are basically the non-zero elements of the field.
OK?
It's slight abuse of notation.
We identify the field elements with the degree zero polynomials.
These are polynomials just of the form F of x equals F_0 a constant, where the constant is non-zero.
OK.
So similarly, we will have to have representatives of equivalence classes with respect to the units.
So if both plus or minus n divide something, what we take as the representative is we take the positive integers.
When we talk about divisibility, we talk only about divisibility by positive integers or factorization by positive integers.
It's trivial that if something is divisible by n, it's also divisible by minus n. Similarly for polynomials, the representatives are taken as mnemonic polynomials.
And this means that the highest order term, Fm is equal to 1.
So we can take any polynomial, and by multiplying it by one of these invertible elements, basically by a non-zero constant in the ground field, we can make the highest order term equal to 1.
Right?
So if a polynomial is divisible by F of x, it's also divisible by alpha F of x, where alpha is any non-zero field element, right?
And so we may as well fix the highest order coefficient equal to 1.
Actually, for some purposes in the literature, like you've probably seen this in filter design, or we always make the lowest order coefficient equal to one.
F0 equal to 1.
You could also adopt that convention, and say that that's going to be a mnemonic polynomial.
Here we'll focus on the high order coefficient, but you could do it either way.
And these both have the nice property that the product of positive integers is a positive integer.
The product of monic polynomials is a monic polynomial, right?
The highest order term of the product is going to have a highest order term equal to 1.
Or if you chose the lowest order one, that would work, too.
The lowest order term of the product would be equal to 1.
All right.
So having recognized this, when we talk about unique factorization, just as with the integers, what we really mean is factorization of a positive integer into a product of positive integers.
It's unique up to units.
We can always put units either on the integer that we're factoring or on any of the factors, and we can freely multiply any of these things by units, and it won't affect the factorization.
Similarly over here, when we talk about unique factorization, we mean unique up to units.
We're basically going to talk about the factorization of monic polynomials into a product of monic polynomials.
Not getting a real positive feeling that everybody's following me.
Would it help if I wrote down more things, or wrote some examples?
In F2 of x, for instance, we have -- well, this isn't a very good example.
Let's write R of x. OK?
We're going to talk about factorizations like this.
x squared minus 1 equals x minus 1 times x plus 1.
That's a factorization of a monic polynomial into a product of monic polynomials of a lower degree.
Happens in F2 of x.
Since there is only one non-zero field element, then all polynomials are monic, except for the 0 polynomial.
The only non-zero term we have play with is one.
So the highest order non-zero term is always 1.
All right.
OK.
So that's what we're going to mean by unique factorization.
Now, there's one other qualifier.
There's some trivial factors.
We took some care to use the standard mathematical terminology in the notes, so if it says trivial devisors, then that's what I mean.
What would the trivial divisors of integer n be?
1 and n are always going to divide in.
And we're really not interested in those when we talk about factorization.
Similarly, over here, the trivial divisor of a polynomial F of x are 1 and F of x.
We're not interested in those.
So when we talk about unique factorization, we mean up to units and nontrivial factors.
And for the integers, that means that we're going to talk about divisors d, let's say, such that d is between 1 and n. And for polynomials, what this means is we're not interested in degree 0 factors.
The only factor of degree the same as F of x is going to be F of x up to units.
We're interested in divisors d of x such that the degree that 0 is less than the degree of the divisor less than the degree of what it's dividing into.
Sorry.
I'm not defining everything, but I hope that's clear.
We're just interested in factors that have degree less than the polynomial that we're factoring, but we're not interested in degree 0 factors.
So that's what's meant by unique factorization.
It also shows you the analogy in general.
In the case of integers, the key thing in a divisor is that it have magnitude between 1 and n. The key thing in a polynomial is it have degree between 0 and the degree of F of x.
Basically, we want to factor something into smaller things here.
And when we say smaller, we talk about magnitude.
Here when we say smaller, we're talking about degrees.
In general, we go between these two things.
The concept of magnitude is replaced by the concept of degree, to say how big something is.
In both of these, the key to all proofs is the Euclidean division algorithm.
Suppose we want to see if n is a divisor of m. I've forgotten what I put in the notes.
Then we go through division, and we find that m is equal to q, some quotient times n, plus a remainder r. This is standard grade school division.
But it's really the key in the universe in talking about these two domains.
And this is what we've used, really, to prove everything about the factorization properties of integers.
And how would you actually prove that everything can be written this way?
There's an important caveat.
That the remainder we can always choose to have to be in the range from 0 less than r less than or equal to n minus 1.
And the remainder is what we call m mod n. We divide and we get a remainder that's one of these n things, and that's the main thing we get out of this division, is m mod n, which is equal to the remainder r. And there are precisely n remainders.
Now, how do you actually prove this?
You prove this, if you want, very easily, just by recursion.
You take m, and you ask, is it already in this range?
If it is, you're done.
m is the remainder and q is zero.
If not, then you subtract n from m, thereby reducing the magnitude of m. You can use the magnitude as an indicator of how far you've gotten in this process.
You reduce the magnitude, and then you ask, is the result in this range?
OK.
If it's in that range, fine, you finish this and q is 1.
Otherwise, you continue.
And in the recursion, you're continually reducing the magnitude, and it's easy to show that eventually, the magnitude has to fall into this range and be one of these n remainder numbers, and then you're done.
OK?
So it's a descending chain where the chain has a bottom at 0.
If you start with a positive integer, you can't go below 0.
And this is the only way it can come out.
Very easy to prove that.
All right.
Similarly in polynomials, we get an analogous expression.
If we want to take F of x and see if G of x is a divisor, we take F of x, and we can always write this as some quotient times G of x plus some remainder.
Where the important thing here about the remainder is that the degree of the remainder is less than the degree of G of x.
And here the remainder is called F of x mod G of x.
Just as the remainder here is called m mod n. And there's a unique remainder.
And again, how would you prove this?
You could just take any long division algorithm that you know for dividing G of x into F of x.
Basically long division amounts to taking F of x.
You can always choose some scalar multiple of G of x such that F of x minus alpha G of x has degree less than F of x.
So you pick the top term to reduce the degree, all right?
Let's take G of x to be monic.
We're only interested in monic polynomials.
If the top term of F of x is -- well, we're only going to divide it into monic polynomials.
But as we go along, we may get non-monic ones.
So you take the top term, whatever it is over here, f(m).
You multiply f(m) times g of x.
You subtract f(m) g of x from f of x.
You reduce the degree.
OK?
[INAUDIBLE PHRASE]
Correct.
Thank you very much.
You need also a term x to whatever the difference in degrees is here to move the degree up to the top.
When we're actually doing long division, we write f(m) f(m minus 1) down to f(0).
We divide g(n) down to g(1).
And the first term, g(n) is going to be equal to 1.
We take f(m) up here.
We implicitly move it over to get f(m) dot dot dot dot, down to f(m) alpha.
We subtract, and we're down to something that only has degree m minus 1.
That's what polynomial long division is shorthand for.
So you all know how to do this.
OK. Again, similar kind of proof, that you must be able to get a remainder in this range, and furthermore, the remainder is unique.
You basically can go through this process.
You reduce the degree by at least one every time.
Therefore, degree must eventually be reduced to where it's less than degree g of x.
At that point, you can't continue this process.
You're stuck.
And that's your remainder.
There's no way of taking something of lesser degree of g of x and then subtracting some multiple from g of x from it to still further reduce the degree.
All right.
So similar proof.
Uniqueness is pretty obvious.
So you get a unique remainder of lesser degree than g of x.
And so you can reduce any f of x to some remainder r of x, which is called f of x mod g of x.
And if we do arithmetic, the way we do arithmetic over here for addition is we just, if we want to add two remainders mod n, we take the sum of them, and then if necessary, reduce them mod n. Similarly with multiplication.
If we want to multiply them, we take the product of two remainders, and if necessary, reduce them again to a legitimate remainder which is in this range or to mod n. It's the same over here.
If we want to add two remainders, that's easy enough.
We can do that and we won't increase the degree, so we automatically get something when we add two remainders that satisfies this degree property.
We don't have to reduce mod g of x.
If we multiply, you do have to check that when you multiply two remainders, then all you need to do is reduce the mod g of x.
And basically, the assertion is that -- let's see.
r of x, s of x -- it's hard to write this without being tautological.
r of x, s of x.
Reduced mod n, I'm sorry, mod g of x. I'm sorry.
This is not worth writing.
r of x, s of x, mod g of x.
Something like that.
Bah.
It's a total tautology.
Forget it.
Said correctly in the notes.
I'd rather think of it as residue classes.
If we, say, we talk about this as a coset, f of x plus r of x, and we want to multiply any element.
This coset times any element of the coset f of x plus s of x.
Then the result is just multiplying out symbolically the polynomials times the polynomials are the polynomials.
Any polynomial times a polynomial is a polynomial.
Plus r of x f of x.
We could get some multiple of r of x, some polynomial multiple.
We could get some polynomial multiple of s of x.
Plus r of x s of x.
But this is a polynomial that's included in here.
We don't need to say that again.
Similarly here.
And so this is equal to f of x.
It's equal to the coset f of x plus r of x s of x.
But this is another polynomial that probably has higher degree.
This is equal to the coset f of x plus r of x s of x mod g of x. OK.
So this tells us that to multiply cosets, coset with representative r of x times the coset with representative s of x, we're going to get something in the coset whose representative is r of x, s of x mod g of x.
So this is a sketch of a proof that basically mod g of x commutes with multiplication.
To multiply r of x times s of x.
All right.
So --
[INAUDIBLE PHRASE]
Yeah.
You're quite correct.
What I mean is the cosets of g of x of f of x.
So I now understand all the blank looks.
Are we better off now?
We have a group consisting of the set of all multiples of g of x, which I write as g of x times all the polynomials.
The cosets of the group are precisely -- there's one remainder of degree less than g of x in every coset.
So this is the representative of the coset.
This is a sketch of a proof that multiplication basically just amounts to multiplying the remainders.
The representative of the product coset is the product of the representatives modulo g of x. I think I said it correctly for once.
Please read the notes if you are still confused, as I can see some of you are.
Maybe it would help to do an example.
Let's take g of x to be equal to x squared plus x plus 1 in f2 of x.
It's a binary polynomial.
Then r of x.
The remainders are equal to 0,1.
This was degree minus infinity, this is degree 0.
They're x or x plus one.
OK?
These are the possible remainders when I take any polynomial modulo g of x. Divide it by g of x, and I'm going to get a polynomial in f2 of x of degree less than or equal to 1, and those are all of the polynomials that are decisively for degree less than or equal to 1.
All right?
So what's the addition table?
Addition is pretty easy.
0 plus anything is itself.
1 plus 1 in F2 is simply 0.
1 plus x is 1 plus x, sorry, x plus 1 as I've written.
And 1 plus x plus 1 is x. x, x plus 1. x plus x is what?
Hello?
0.
Thank you.
x plus x plus 1?
Thank you.
x plus 1, x. x plus x plus 1, 1, and 0.
So that's what the addition table looks like.
Actually you could think of these as being just written as binary pairs, 0 0, 0 1, 1 0, 1 1, where this pair is basically F1 F0.
And then the addition table is precisely the same as the addition table for these binary 2-tuples.
You just add, component-wise, the lowest order coefficient, the F1 coefficient.
So addition is just like addition in F2 squared.
Or z2 squared, if you like.
OK. That's actually an additive group.
Check it out.
It's not z4 by the way.
The other abelian group, or the other group of size four, sometimes called the Klein four-group, but it's really just the addition table for the set of all binary [UNINTELLIGIBLE] two-tuples.
OK?
Multiplication.
One very nice thing about finite fields is you can simply -- sorry.
This was supposed to be addition, this was supposed to be multiplication.
You know can simply write out what all the rules are in a finite space.
OK.
So what's 0 times anything?
What's 1 times anything?
Itself.
1 is the multiplicative identity.
But you know.
You could formally do this by doing polynomial multiplication.
All right.
Here's an interesting one.
What's x times x?
Do x times x.
What is that going to be equal to?
x plus 1.
How did you do that?
We did the modulo.
First of all, we write that that's x squared.
But then we have to do x squared modulo g of x. x squared plus x plus 1.
So we have to go through a little long division process.
We have to subtract this out from that.
And that gives us x plus 1.
So that's the key rule.
Whenever we see something of degree two or higher, we can always reduce it by subtracting out some multiple of g of x down to something of lower degree.
Right?
So this is what I've been talking about.
So x times x is x plus 1.
What's x times x plus 1?
Equals what?
Good.
You can do that in your heads.
And what's x plus 1 times x plus 1?
Remember, we're doing mod-2 arithmetic in our base field here.
So this equals what?
x squared plus 1.
We reduce that, we get x. OK. Let me check right now.
Is this a field?
These four elements with these rules for addition and multiplication.
Does that form a field?
What do I have to check, apart from formalities like the distributive law?
Which follows from the distributive law for polynomials.
That's always going to hold through mod x arithmetic.
What do I have to check?
What are my field axioms?
Anybody?
Closure under multiplication.
That's getting towards a very crisp statement of the -- I'm looking for two group axioms.
One has to do with something we have to check for addition.
Something has to do with something we have to check for multiplication.
Inverses?
[INAUDIBLE PHRASE]
OK.
Between the two of you I heard the two answers that I want.
We have to check that this forms an abelian group under addition.
So we have to check that the addition table is the addition table of an abelian group.
And under multiplication, we have to check that the non-zero elements form a billion group.
So just this part of the table has to form an abelian group, and both these have to have an identity, of course.
But the identity in mod g of x arithmetic is always going to be 0 for addition and it's always going to be 1 for multiplication.
All right.
So I check this.
Is this a group table?
Basically, I just have to check whether every row and column is a permutation of the elements.
And it is.
And 0 acts as 0 should act.
Has the additive identity.
All right?
Here what I have to check is that the nonzero elements, these three, form an abelian group under multiplication.
Well, there really is only one group of size three.
It is isomorphic to Z3.
If I replace -- let's remember what Z3 looks like under addition.
This looks like 0 1 2, 0 1 2, 0 1 2, 0 1 2.
It's mod-3.
1 plus 1 is 2, 1 plus 2 is 3, which is 0, 1 plus 2 is 3, which is 0, 2 plus 2 equals 1.
So gee whiz.
This is isomorphic to that if I relabel 1 by 0, x by 1, and x plus 1 by 2.
That's the only thing it could be.
The only group table in which every row and column is a permutation of every other.
OK?
So we verified that we now have a finite field with four elements.
Prime power number of elements.
Right?
The elements of my field are these four remainders, or you can think of them as representatives for their cosets, modulo g of x.
The addition rule is addition modulo g of x, and the multiplication rule is multiplication modulo g of x.
And it satisfies the field axioms, therefore, it's a finite field.
All right?
I can add, subtract.
Addition and subtraction basically looks like addition and subtraction of binary two-tuples, just component-wise.
Multiplication is a little bit more mysterious right now, but it works.
Let me tell you where we're going to go on multiplication.
In this case, I can write x plus 1 in a different way.
I note that x plus 1 is equal to x squared.
All right?
So let me write a little log table over here for multiplication purposes.
I'm going to write x -- I'm going to call that alpha.
And x plus 1 is equal to x squared, or it's alpha squared.
What's alpha cubed?
Alpha cubed is x times this again.
Let me look in the table.
x times x plus 1 is equal to 1.
So 1 equals alpha cubed.
Or I could write that as alpha to 0.
If I multiply by x again, I just cycle.
So I'm going to get a cyclic group here.
And now I'm going to write the multiplication table as follows.
I'm going to write the elements of the group as 1, alpha, alpha squared, 0, 1, alpha, alpha squared.
Again, 0 times anything is 0.
We never have to worry about that.
1, alpha, alpha squared.
Alpha times alpha is alpha squared.
Alpha times alpha squared is alpha cubed.
But that's equal to 1.
Same here.
Alpha squared times alpha squared -- what's that?
Alpha to the fourth, but what does alpha fourth equal to if alpha cubed is equal to 1?
This has to be equal to alpha.
Point is, because of this relationship here, I can always reduce the exponents modulo 3.
I've basically got a little multiplicative cyclic group of order three that's, of course, isomorphic to the additive group, Z3.
So there are two ways I can do multiplication.
One is, I can do it by this mod g of x way.
I can represent things by these which basically stand for polynomials of degree one or less.
And I can multiply two of these by simply going through standard polynomial multiplication over the appropriate field.
And then I'll likely get some powers of x squared or higher, and I reduce those to modulo x squared plus x plus 1.
That's legitimate and that gives me this statement.
Well, the other thing I can do is for multiplication, I can have a different representation.
This is basically just a log table.
OK?
For each, I would have in my little computer a separate table where I'd write that this corresponds to 0, 0 1 corresponds to 1, 1 0 corresponds to alpha, and 1 1 corresponds to alpha squared.
Or of course, I would just represent these by their exponents that have some special value for 0.
And then all I have to do is add exponents modulo 3 for multiplication.
So log of x is 1, log of x plus 1 is 2, log of 1 is 0, and then I just use this for my multiplication operation, or equivalently this cyclic multiplicative group.
Then I go through some inverse log operation to get back to the other representation, if I wanted to.
And in fact, in finite field arithmetic, this is what's typically done.
There's just a little table lookup such that you can go back and forth between this representation, which we use for addition, and this representation, which we use for multiplication.
Yeah?
[INAUDIBLE PHRASE]
You have to represent it as being special in some way.
So if, in fact, literally, I made log x equal to 1, log x plus 1 equal to 2, log 1 equal to 0 -- one thing I suggest in the notes, you can make log 0 equal to minus infinity.
That will always work.
If you ever multiply by 0, you'll be adding minus infinity.
The result will be minus infinity, so the inverse log is 0.
So that's one way you can do it.
Or you could do what you do in ordinary real and complex arithmetic.
You could just, say, have some special routine for multiplication by 0 is always 0.
Division by 0 is illegal.
Put out some error message.
All right?
So that's how you actually build a little finite field computer for this finite field.
OK. Now, I chose this advisedly.
Suppose I have chosen g of x equal to x squared plus 1.
OK?
I can do mod g of x arithmetic for x squared plus 1.
Again, my four field elements are going to be the four binary polynomials of degree 1 or less.
The addition table is going to be exactly the same.
So let's just write the multiplication table over here.
0, 1, x, x plus 1, 0, 1, x, x plus 1.
0 times anything is 0 in ordinary polynomial multiplication, and therefore also in mod g of x for any g of x.
1 times anything is itself.
No problem there.
x, x plus 1.
So we again have to give a little bit care to x squared.
What's that going to be equal to?
1.
And x times x plus 1 is equal to what?
x squared plus x is equal to what?
Looks like there's a problem.
x plus 1 times x squared x plus 1 is equal to x squared plus 1, what's that equal?
0.
Yuck.
Didn't work.
What's the essential problem here?
Yeah.
This clearly is not a group.
It's not even closed under multiplication.
Because x plus 1 times x plus 1 is equal to 0.
The essential problem is that there are two polynomials of degree less than two whose product is x squared plus 1.
In other words, this is factorizable.
OK?
In F2 of x.
All right?
So whereas x squared plus x plus 1.
Does that have any factors of degree 1 or less?
Any nontrivial factors of degree 1 or less?
Well, basically we proved that it didn't, when we wrote this multiplication table.
We tried all products of degree 1 or less polynomials where they're both non-zero, and we never got 0.
So this will work if and only if g of x is irreducible, has no nontrivial factors, is a prime polynomial.
These are all equivalent in F2 of x.
There's this distinction in other fields that irreducible just means has no nontrivial factors.
Prime means there's a monic polynomial with no non-trivial factors.
So that's what I said back here.
That the way we're ultimately going to have to construct finite fields is take the polynomials in Fp, Fp of x -- we've been looking at F2 of x -- modulo a prime polynomial g of x.
So what we're going to need to find is prime polynomials.
Let's see.
Can I already prove that this is going to work for any prime polynomial?
So I'm going to force the g of x to be equal to a prime polynomial in Fp of x of degree m. All right For example, x squared plus x plus 1 is a prime polynomial.
And I'm going to ask if the remainders mod g of x form a field under mod g of x arithmetic.
OK. Let's flow this out a little bit.
Again, what are the remainders going to be of any polynomial of degree m?
So this is basically going to be the polynomials of degree less than m in Fp of x.
How many of them are there, by the way?
p to the m. So the size of this is p to the m. And one of the representations for these polynomials is just to write out F0, F1, up to m minus 1.
So this just basically looks like the polynomial m-tuples.
Set of F0, F1 up to F minus 1, for each of these an element of the field.
OK?
I can make a one-to-one correspondence between the polynomials of degree less than m and the set of all coefficient m-tuples over Fp.
These are just m-tuples over Fp.
So there are p to the m of them.
And so it's a finite set, size p to the m. Does it form a field under mod g of x arithmetic?
[INAUDIBLE PHRASE]
Correct.
And that's a homework problem.
So I'm not going to do it in class, but I will sketch here how it's going to be done.
But I'm glad that you instantly see that.
Because again, we just model everything we do on what we did for integers.
If you remember how we proved it for integers.
First of all, we really need to check two things.
Addition.
Just as we did for this specific example up here, we first have to check that the addition table is that of an abelian group, that these supposed field elements form an abelian group under addition.
And we've already observed several times that addition is just basically component-wise addition of these m-tuples.
So it's just like vector addition.
And vector addition, of course, has the group property.
So addition is basically just like vector addition.
Or I could write, perhaps more precisely, Zp over m. Distinction without a difference, really.
So it's just component-wise addition of the coefficients.
That's how we do polynomial addition.
And it will give us a remainder that has a degree of less than m, so we don't have to have ever reduce it.
Modulo g of x, we just simply add component-wise.
OK.
So that's easy to verify.
The addition table is always going to be OK.
So what do we have to prove now?
We have to -- what am I going to call these?
Rg of x.
The remainders mod g of x.
For multiplication, we have to prove that Rg of x star -- the non-zero polynomials form an abelian group, which, as I say, it's a homework problem.
Let me sketch the proof.
It's precisely analogous to the proof that we made for Zp star.
We basically have to check closure.
If we multiply two non-zero polynomials, do we get another non-zero polynomial?
Asking another way, is it possible to multiply two polynomials of degree less than m and get a result which is a multiple of g of x, which is either equal to g of x or a multiple of g of x?
And it's, I think, easy to convince yourself if this has no factors -- its only factors are itself and 1, no non-trivial factors -- then you can't multiply two lesser degree polynomials and get either g of x, of course, or any multiple of g of x.
And it's an exercise for the student to write that proof out.
OK?
But it's just exactly analogous to the proof that you can't multiply two integers less than a prime p and get a multiple of p. So use that as a model, if you want to, in your proof.
That of course depends on this being prime.
If this is factorizable, non-prime, then of course there are going to be two nontrivial factors, two remainders of degree less than m, whose product is equal to g of x itself, if it's factorizable.
So you can't possibly get closure.
So this is why non-prime polynomials don't work, just like non-prime integers don't work when we constructed Fp.
Exactly analogous reasons.
All right.
Second question is, when we go through this multiplication, suppose we take r of x times -- let's construct a multiplication table.
Let's take a particular row.
Let's take r of x times all the non-zero things.
Can we possibly get any repeats?
Can r of x times s of x equal r of x times t of x?
And again, it's easy to convince yourself that that's not possible.
If that were possible, then r of x times s of x minus t of x would be equal to 0.
And so we're back to where we were before -- mod g of x.
And this is impossible.
These are both degrees less than m. We can't multiply two things of lower degree together to get a multiple of g of x.
So it can't equal 0.
That means there can't be any repeats.
That means that each row is a permutation of every other row.
Similarly for columns.
If you want to do that, all you have to actually prove is one.
So every row or column is a permutations of every other one, if we just look at the non-zero polynomials, star.
And therefore, this forms an abelian group whose identity is always one.
Just as we proved up here.
So this depends on irreducibility.
Depends on no nontrivial factors of g of x. OK. And that's all we have to check.
So I claim that by this process, I can, given a prime polynomial in Fp of x of degree m, I can construct a finite field with p to the m elements, that the addition and multiplication rules can be taken as mod g of x arithmetic, and they will satisfy the field axioms.
So you can now construct a finite field for any prime power p to the m, right?
There's actually still a hole in this
[INAUDIBLE PHRASE]
Define prime polynomial?
The term, or --
Define the [UNINTELLIGIBLE] of degree m
Correct.
Is there going to be a prime polynomial of every degree?
I don't know.
Suppose you want to define an irreducible polynomial over, say, F2 of x or F3 of x of degree 4.
Could you do that?
[INAUDIBLE PHRASE]
Beautiful.
I mean, that's an excellent suggestion.
So the question is, does there exist a prime polynomial in Fp of x of every degree?
Or of a given degree?
And there are various ways of attacking this question.
First of all, I'll tell you the answer is yes.
For every p and every m, there does exist a prime polynomial.
Which is fortunate.
So from that, we conclude there is a finite field with p to the m elements for every prime p and every m greater than or equal to 1.
Now, how might we prove that?
One is, look it up on Google.
You can certainly formulate a question that will lead you to a webpage that will have a listing of all the prime polynomials of all degrees over any field that you're interested in.
So perhaps that will suffice for you.
Two.
What I'm going to talk about is the sieve method.
Three.
You could do a bound on the number of polynomials of each degree in d, and show that it's greater than or equal to 1, always.
And this is done in the notes.
Section 7.9 I think.
Or four, you can do what Mr. Agarwal has suggested.
You can actually do the closed form combinatoric formula, which -- I haven't done any number theory here.
There's a little bit of elementary number theory in the notes.
Euler numbers, this sort of thing.
We get formulas for the number of integers degree d that -- well, number of integers that have multiplicative orders d mod n, and so forth.
It's a lovely combinatoric field.
There is a lovely closed form for this that you get from the Mobius inversion formula.
And it can be found in combinatoric books.
I know it's in Berlekamp's Algebraic Coding Theory book.
And it's extremely pretty.
And then of course, given the formula, you have to prove to all of the -- again, n of d is greater than or equal to 1.
But there is a closed form expression for it that you get out of combinatorics.
We're not going to do that here.
I'll be satisfied -- well, this is the real engineering solution here.
This is the mathematical engineering solution.
And what do I mean by the sieve method?
Again, take the analogy with the integers.
One of the first mathematical accomplishments was Eratosthenes' sieve for finding prime numbers.
How does it work?
You start to write down all the -- well.
Imagine first writing down all the integers.
All right?
Cross off 1.
Start with 2.
All right.
So the first prime is 2.
Then you cross off all multiples of 2.
4,6,8, so forth.
OK?
So what's the next number that you haven't crossed off?
It's 3, so that's the next prime.
You cross off all the multiples of 3.
3,6,9.
So forth.
15.
And thereby you continue.
So the steps are, find the next integer on the list.
That's going to be a prime, because it won't have been crossed off by any previous steps.
It's not a multiple of any integer of lower degree.
Then cross off all of its multiples, up to however long your scribe has written this on the tablet.
And this way, you can find the primes up to any number you want.
Gets kind of tedious after a while, but you can certainly find all the primes up to 100 in a few minutes doing this, right?
Well, it's the same for integers, and it's the same for polynomials.
So let's, for instance, do a polynomial sieve.
Of course, what we're most interested in is the polynomials with binary coefficients, F2 of x.
And how do we do it?
Let's write down -- let's forget about 0 and 1.
Those are not considered to be prime.
Let's start with the degree 1 polynomials.
What are the degree one polynomials?
They're x, x plus 1.
Are these factorizable?
Obviously their only factors are one and themselves.
They have no nontrivial factors.
So these are primes.
OK.
So now let's write down all the polynomials of degree two.
I'm sort of doing this in an interleaved manner.
There are going to be two of degree 1, four of degree 2.
All I'm going to do is -- well, the only polynomials are monic in F2 of x, so I don't have to make that qualification.
All right.
So here are the four of degree 2, right?
Now I go through with my sieve and we take out all multiples of x.
Multiples of x are polynomials with 0 constant term, right?
A lowest order term.
So obviously this is a multiple of x, this is a multiple of x.
Multiples of x plus 1.
How do you recognize those over the binary field?
This is basically the polynomial whose root is 1.
That means the mod-2 sum of the coefficients is equal to 0.
So any polynomial that has an even number of non-zero coefficients is divisible by x plus 1.
Did you all get that?
If not, try that at home.
That's the easy way to recognize whether something is a multiple of x plus 1.
It has an even number of non-zero coefficients.
So this is a multiple of x plus 1.
We wrote it out explicitly.
It's x plus 1 squared in F2 of x, and this is not.
So there is only one prime polynomial over F2 of x that's degree 2.
So this is our only possible choice if we want to construct a finite field with four elements, due to the two elements.
OK.
So that might begin to get us scared.
Is it possible that there are no prime polynomials of degree three?
Well, we had two here, one here.
Is it going down?
Let's write down the polynomials of degree 3. x third, x third plus 1, x to the three plus x, to the three plus x plus 1, x to the three plus x squared, x to the three plus x squared plus 1, x to the three plus x squared plus x, x to the three plus x squared plus x plus 1.
Eight of them.
So that's working in our favor.
There's double the number of polynomials as we go up one in degree.
And again, let's go through the sieve.
Which are multiples of x, non-zero constant term?
Which are multiples of x plus 1, even number of non-zero coefficients?
If you doubt that, write it out.
Which are multiples of x squared plus x plus 1?
Well, they're going to have to be x squared plus x plus 1 times x plus 1 or times x, so we've already got them.
If we take this times itself, we're going to get a polynomial of degree 4.
Not going to be on this list.
So we have to multiply this by these.
We've already got those.
So whew.
We found two.
We could use either of these to construct a finite field with eight elements.
And so forth.
And it turns out as you go up to higher degrees, that the number now increases very nicely, and there's no problem finding a prime polynomial of each degree.
And in fact, you could be cute about it and try to find one with only three non-zero terms, or has some other nice property that makes it easy to calculate with.
And so forth.
So do you understand the sieve method?
If you do, then I believe you could find a -- suppose you want to construct a field with 64 elements.
What do you need?
64 is 2 to the sixth, so you're going to need a polynomial with p equals 2 and m equals 6.
You're going to need a binary polynomial of degree 6 that is prime, irreducible.
And again, in a few minutes by going through the sieve process, you can quickly find one, or you could look it up in Google, or in Peterson's book, or any algebraic coding theory book, or probably a lot of other places.
So this is a practical solution for the problem for any given p and m. Of course, it hardly proves that there's one of every degree, because they're kind of an infinite number of degrees.
Yeah?
So there's two order 3 polynomials that are prime.
Will it generate two separate fields, or are they going to be isomorphic to each other?
Great question.
All fields with p to the m elements are isomorphic to each other.
That's proved in the notes.
I'm not going to do it in class.
And the other thing that's proved in the notes is the analog to Zp.
There are no other finite fields with other than p to the m number of elements.
OK?
So this is it.
Now, if you explicitly write out these two, and write out their -- well, the addition tables are always look the same, because it's always just binary three-tuples, in this case.
But multiplication tables are going to look different.
But there is some isomorphism.
x and 1 may be equivalent to x squared plus 1 on the other one, or something.
But if you go through that isomorphism, you'll find that the field tables are the same.
Actually, I guess I'm going to prove that, because I'm going to prove that the multiplication table is always cyclic.
That the group to which it's isomorphic is z mod p to the m minus 1.
Just as it was to Z3 here.
And I guess that's sufficient with the addition table isomorphism.
I guess you have to prove they're equivalent.
You saw it in a different way in the notes.
But you're going to see that all the multiplication group is always a cyclic group, with p to the m minus 1 elements, and that goes a long way towards suggesting that these are always going to be isomorphic to each other.
Yeah?
Identify roots, right?
Identify roots?
If I understand what you're saying, that's basically the way it's done in the notes.
You first show there's always going to be some primitive element that generates the cyclic group.
Some single generator such that alpha alpha squared, so forth, is the entire non-zero set.
You're going to show that alpha has some minimal polynomial, and the set of all linear combinations of powers of alpha is basically equal to the whole field.
And so this allows you to establish the isomorphism.
and I think that's what you're suggesting.
So you're well equipped to read the notes, but I'm not going to do that in class.
Yeah.
I'm very interested in your questions.
Please ask more, as many questions as you like.
What I'm getting from it is, you know, you all come from different backgrounds.
Some of you have seen this in perhaps a math context, or some other context, or you've seen parts of it, or some of the words are familiar.
And of course, there are many different ways to present this and to make the proofs and so forth.
So I'm trying to pick a line that works for the particular results that I want to get to.
I don't think you would do anything much different if you wanted to develop finite fields.
But the class has a very different set of backgrounds, and I'm trying to reach all of you.
Don't be alarmed if you've never seen anything like this before.
You're not way behind everybody else, either.
I think it's pretty easy to understand.
Maybe it would take you a couple hours longer than somebody who has more background, but not more than that.
OK.
So that's how we construct finite fields.
You have an example of it.
Goodness.
Is it really 11 o'clock?
OK.
So I'm not even -- so I've simply asserted, but not proved.
You saw in this case that the multiplicative group was a cyclic group.
And we could always, for multiplication, represent grouped elements by this log table.
So I'd hoped to prove that in class.
I'm not going to be able to prove that in class.
And I'll simply ask you to read that, too.
This is important for working with finite fields, because it is the way, probably the preferred way, to implement multiplication.
So you ought to be thinking of how you would program up a finite field multiplier.
One way is polynomial multiplication.
The other way is just use the fact that the multiplicative group is cyclic.
And then it's easy.
Just add exponents and reduce modulo q minus 1.
OK.
I'm sorry not to have had a chance to go over that.
Next time, Ralf Koetter will start to get into chapter eight, Reed-Solomon codes.
Welcome to class again.
This time it's not Professor Forney it's me, so my name is Ralf Koetter.
You guys have some substantial chalk here at MIT.
And I'm visiting here from the University of Illinois, so Professor Forney thought I could teach this class here.
All right, let's see.
So I understand that last time, last Wednesday, you went through all the finite field stuff, meaning, so you know what that would mean, the finite field.
There's p elements, p to the m elements.
Whatever q you have here, is a power of a prime in order to be a field.
So this one, as a notation, is a ring of polynomials.
You've seen that too.
So I assume you know everything about finite fields that you will need to know here, at least, except for one more theorem which Professor Forney told me he did not cover.
And this is the fundamental theorem of algebra.
I have to write a little bit smaller with this thing here, otherwise I'll run out
[UNINTELLIGIBLE PHRASE]
Oh, I know, that's probably better.
Better.
With the algebra, at least that's what it's often called, and really, about 60 percent of all the proofs in algebra eventually boil down to this here.
And what it says is, polynomial of degree m, f beta equals zero, at most, m of beta.
At least, that's one way to formulate it.
Let me see.
So that's fine.
So what it says is a polynomial of degree m has at most m roots.
Once you all have seen that, probably one way or another, but because of its importance, I want to emphasize it once more.
Do we need a proof of this?
In true MIT spirit we do.
And the proof would go something like this.
You look at problem number one in your homework assignment, and from problem number one, I could prove that here, too, but since it's in the homework, I won't.
You can write the following given any beta.
Write f of x as f of x is equal to plus alpha.
Alphas are the field so that's by some sort of long division you get to that.
That's what I'm not going to prove.
Then f of beta is equal zero is the same as saying that alpha is equal to zero.
So if either is a root of the polynomial, zero, it follows that f of x has this thing here as a factor, this h of x, x minus beta, where because of the degree properties of polynomials, h of x is m minus 1.
And so the rest follows by induction.
So basically, then we can prove that this polynomial has, at most, m minus 1 roots, and so on.
And you can descend this route, and so the rest follows by induction.
In particular we can say if f of x has m distinct roots beta one, beta m, then it factors completely into the linear factors like this.
So I just wanted to quickly state the fundamental theorem of algebra, since we need it in a proof later on, and I think you didn't go through it.
OK.
So last time, you learned everything about fields, finite fields, extension fields, so chapter eight is pretty much what we have to cover now.
What is the whole idea of chapter eight?
It's linear codes, codes, MDS codes, and redundant codes.
Oh, by the way, do you have any questions about this here?
That in any way?
It's pretty straight, right?
OK, so I understand in chapter six or so, you had already linear codes over the binary fields.
So let's just define codes over a larger field, formally, a linear code C of length n. No subspace of Fn.
So whatever the field is.
So F could be any extension field, could be the binary field, so it really generalizes a definition of code, of what a linear code is.
OK, so it's a subspace.
What can be derived from that?
Since it's a subspace, it's a group.
And then we can derive minimum distance properties.
So let's first define it again, since it's slightly different than the definition for binary codes.
Between Fn, say Fqn.
So I denote the vectors with an underscore.
I think in the notes, it's boldface notation, so translate that online as I go here.
The distance between two words x and y, given as dx, the number of positions that x_i is unequal to y_i
What's the subscript?
There, a q. Oh, this is another thing I should warn you about.
My handwriting is bound to deteriorate during class.
So I usually start out reasonably okay, towards the end of the class it's -- I tell my students to throw little pieces of chalk at me when it gets too bad and I'm not facing them, so please just say something if it gets too bad.
So distance is defined as that, quickly.
So it doesn't really matter what the values are here.
The x_i and the y_i could assume different values.
It's a somewhat coarse measure for the real, the difference between code words, or difference between words.
Why do you think I say it's coarse?
In digital communications in particular?
Good question, right?
In the end, we want to map that into a modulation scheme.
In the end, we want to map our codes that we are deriving here into modulation schemes.
In the end, we want to embed them into some Euclidean space.
Now, different elements of our alphabet we will map to different elements in Euclidean space.
So basically, approximating their distance relation in Euclidean space, which we are really interested in with the Hamming distance here is pretty coarse, but we can do, so we do that.
It's an approximation, at least.
That clear?
All set?
All right
[UNINTELLIGIBLE PHRASE] the Hamming distance [UNINTELLIGIBLE] the same as the Euclidean distance?
Well, it depends on the modulation scheme.
It very much depends on the modulation scheme.
If you have a 8-PSK scheme, where you would label, put in the words, here, with three bit symbols, or with the symbol from F8, then it's definitely different.
It's definitely different.
So if you do anti-polar signaling, then it's directly reflected.
OK, I'm starting to digress already.
So just for completeness, minimum distance, minimum Hamming, of a code subset Fqn is d as a minimum code of dxy, and they have to be different, it's the same as before.
So now if I claim that -- so the minimum distance of a code is also given by the minimum between 0 and x in the code 0 and x, and this is minimum of the Hamming weight of x, and you could do 0 x in the code.
So that's all old stuff.
I just write it down so we get started here.
Is that clear, from the group property, why this would be true?
So if you just take this, we can add basically x to both x and y, just translating the whole relation to somewhere else.
So in particular, we translate it here, once we have it here, than the distance between 0 and x is just the weight.
OK.
So far, so good.
Now what is next?
Generate a matrix.
This is not really in the notes, but I think it's important.
So see, the code here is a subspace.
It's a linear space, so it has a generator, it has generators, k generators.
So let g1 be k, write this off the code.
So as a basis of the vector space, that this would be a basis of the vector space, any basis would be fine here.
Then C may be defined as all the x in Fqn such that x is sum over -- what do I call it -- fi gi, where fi is in Fq.
And the reason I introduce this, we can -- this is just the definition of a space, right?
That's clear.
So if you have these generators, you find a generator matrix, uh-oh, it already starts.
Let me -- matrix g which contains, as a m matrix containing the rows gi.
So the i-th row in the generator matrix is just gi.
Then you also can write as x is equal to f times g, f element Fqk, or just the same statement as this one, so nothing has happened.
So basically, the reason I did that, I wanted to introduce the term generator matrix, which is sort of important.
And one more property of this orthogonal complement of C, of the code.
So what does that mean?
So the orthogonal complement of the code you could write as Fqn such that sum of x_i y_i is equal to 0.
The sum is obviously over the field for all y in the code.
What's the dimension of this, of the orthogonal complement?
n minus k
n minus k, clearly, because we have ambient space is n dimensional, we impose k linear constraints on this, by the k generators, so the k dimensions of, take note, by the generators drop out.
So the dimension of the orthogonal complement is n minus k. So what else do we need to say about this?
C is called the dual code for this reason.
C is called dual code.
In particular, it's a code that's a linear space.
It's a subspace of Fqn again, it's a code.
So it's just as nice a code as C at this point in time at least.
So it's called a dual code.
To C, if it is a code, it has a generator matrix.
Let h be a generator matrix for C dual.
So in particular, we could define C dual now, for example, by the equivalent of this relation here.
But because it's a dual code, we now also can define the original code in an equivalent way such that x times h transpose is 0.
We could define our original code C either as the image of a matrix g, of a generator matrix g, or as a kernel of a parity-check matrix h. So h is a ...WRITING ON BOARD... for C. So that's all pretty much straight linear algebra, and I'm sure you've seen that in many different places.
Any questions about any of this?
So the addition of the dual [UNINTELLIGIBLE PHRASE] the summation [UNINTELLIGIBLE PHRASE] equals 0 for all [INAUDIBLE] other than x, right?
[UNINTELLIGIBLE PHRASE]
Oh no, no, no, it doesn't have to be different from x.
If y is in the code, if y is in C, then x has to be orthogonal to it.
They can be the same vector, in particular, if you have binary vectors, an even made binary vector is orthogonal to itself.
It's a little bit odd, but that's the magic of finite fields.
OK. Good.
So these are codes, now we could stop.
We have defined the object, and obviously it exists, because we could just write something down and it exists.
So once we have defined it, the next question is, what sort of codes do exist?
So that's what we're going to do next.
First, question one.
Codes do, what type of codes do exist?
So which codes do you know?
[INAUDIBLE]
You know Reed-Muller codes, you know probably sporadic binary codes that are out there.
These are all binary codes.
So what type of codes exist over larger fields?
Many, many classes.
There exists the equivalent of the Reed-Muller codes, there exist QRE Reed-Muller codes, and there exist generalized Reed-Muller codes, and, and, and, and, and.
But we are interested in a very special class today, which is MDS codes.
It stands for Maximum Distance Separable.
It's a strange name.
There's no particular reason for MDS.
But, let's see what we can do with that.
What type of codes do exist?
So we have parameters of codes -- oh, I think you write the curly bracket, right -- n, k and d. So that would mean a code of length n, dimension k, and distance d. And let me add something to it a q, if you want to emphasize that this is a query field.
So are all numbers possible here?
What do we have, a 20, 19, 17 code over, I don't know, over F8.
Is this possible?
What would you think?
No?
[INAUDIBLE]
It's not possible.
It doesn't seem likely.
What conflicts here, is the dimension and the distance.
If you get a large dimension, in particular, if we would make this 20, what would that mean?
It would mean we have to take the entire space.
If you take the entire space, then the minimum weight word is 1.
So this is possible.
You know this is possible.
If you drop this by 1, that seems very unlikely that we would get a 17 here.
But what do we get here?
2.
You get a 2 because that's what we can achieve with a single parity-check code.
The parity-check code doesn't have to be restrained to binary.
Why?
Why would it be restrained to binary?
You could just, the set of all vectors let's define the single parity-check codes s p c, q, as the set of all vectors such that sum of the x_i is equal to 0.
So could we have a word of weight 1 in here?
Obviously not, right?
If it has a weight 1, how would it add up to 0?
Because one position would never cancel with any other position.
So the minimum weight is 2 here, and we get a distance of 2.
So what's the next one?
It's tempting to say 3, right?
3, but this is very much a question, now.
Because this is not as easy to come by as a single parity-check.
And that's what we're going to do next.
We're going to define bounds on the maximum distance that a code can have altogether.
OK, so let's do the following.
Which parameter is possible?
OK.
So let's assume you have a code, an n,k,d code, and now we want to find a relation, a bound between n, k and d. How do we do this?
Any ideas?
Let a computer run for eternity and find all possible codes?
No, no, no, no.
We don't do this.
We wouldn't get far.
Let's assume we have an n,k,d code.
What does that mean?
Well, let's write the code words all down in a huge matrix, so each row in this matrix corresponds to one code word.
So this has a length n, this is q to the k, q is whatever the alphabet is of the code in question, and now we say it's an n,k,d code.
What that means, it means, among other things is, say we delete, just punch out, d minus 1 positions.
We punch out d minus 1 positions of all code words and we look at the code that remains.
You guys don't have colored chalk here, huh?
We look at the code that remains, it means we look at this part of the matrix.
Is that clear, what I'm doing here?
So if the code indeed had distance d, can there be any two rows equal in this part?
Remember, we punch out all d minus 1.
Can there be any rows in this part that are equal?
No, right?
Couldn't be.
They all have to be different.
What does that mean?
They all have to be different, but how many different tuples can we have in this part?
Well, we have at most q to the n minus d minus 1.
That's the length here.
This n minus d minus 1.
Different tuples.
So how can we patch that together into a relation on the parameters?
It basically says, q, this is q to the k. q to the k is upper bounded by this.
It's upper bounded by this.
And let me take the logarithm on here, and we get this relation.
That's a first incarnation of the tension that we get on code construction, on codes.
And bound on this, at least.
If you choose d large, the distance large, then k has to go.
If you choose k large, the distance cannot be very large.
So this is where we, for the first time, see this tension.
And it's also important, I'm sorry that I run around like this here, n has to be at least k plus d, k plus d minus 1.
So here, you see this 28 in 3, it would just satisfy this.
It would just satisfy this.
So do we know it exists?
No.
No, why would it?
So far, we only have looked at this here, and so, if it would exist, it would have to satisfy that.
But there's no reason to assume it exists.
At the moment, at least.
OK.
This is called the Singleton bound.
Any code over any field, phi n, this relationship on the parameters.
Good.
Any code satisfying and bound with equality is called MDS.
So we have an MDS code if and only if it satisfies the Singleton bound with equality.
That's the definition of MDS codes.
And now it makes maybe a little bit more sense to talk about Maximum Distance Separable codes, well, in a sense, they have the maximum distance among all codes.
You find all codes with the given n and k, if they're MDS, they have the maximum distance.
OK, let's think about this a little here
[INAUDIBLE] dependence on q [UNINTELLIGIBLE]?
Yeah, there's a very strong dependence on q.
The bound, not.
The bound has no dependence on q.
If the guys exist or not, that's very much dependent on q.
We'll get to that
[INAUDIBLE] when the q is large, we have more options to [UNINTELLIGIBLE]?
Absolutely.
Absolutely.
For binary, there's a very simple argument to show that there are no binary MDS codes except for the parity-check codes and the repetition codes and trivial code.
So say we have a binary code, a binary n,k,d code with a generator matrix.
So what could the generator matrix be?
There will be an identity part, and then there will be the rest of the generator matrix, and how could we possibly fill that in, in order to make it MDS?
Because this is n, this is k, and we see in order to make it MDS, every single row has to have all entries equal to 1.
Because if not all entries are equal to 1, here, then we immediately have exhibited a code word with a weight less than n minus k plus 1.
So OK, we know the first row has to have all 1's.
Because now, the weight of this row is exactly on the MDS [INAUDIBLE].
What about the next one?
The next one, same thing.
All entries have to be 1.
But now we see the problem, right?
Now we add those two guys, it should again be a code word, and we have a grade two code word.
So this is, in a nutshell, to prove that there are no binary MDS codes except the trivial ones.
So the trivial ones are n, n1, n, n minus 1, 2 and n1, n. These are the trivial ones.
The space itself, so it's in a parity-check code, and the repetition code.
These are the only binary MDS codes.
And the argument is roughly there.
OK, where was I?
Yeah, let's think about this a little bit more.
And we are getting to exactly your question about the [UNINTELLIGIBLE].
This here has to hold, this argument has to hold regardless of which d minus 1 positions we punch out.
This argument has always to hold.
Which means, think about it, it's an enormously strong combinatorial condition on the code.
So you have a code, that means you have a code, you write it in a matrix like this, all the code words.
You punch out an arbitrary collection of d minus 1one positions, and the rest, the remaining positions, have to make up the entire space here.
The entire space Fqn minus to that right exponent.
This is a very -- think about it, I mean, just writing down this is an enormously strong combinatorial condition.
So that will actually lead to the codes existing only for a very, very special, for a subset of field sizes.
In particular, like you said, we have to have enough freedom in the field size to fill up this matrix to satisfy this.
OK, before we get to that, before I say a word about the field size, let me formalize what I just said here, namely, that all of the other positions have to be exactly the q to the n minus d minus 1, different tuples.
And the definition, let the code with q to the k, code words over alphabet Fq.
Let subset of the positions in C, i is called an information set if C constrained to i runs exactly through all the q to the k, runs through all the q to the k. Fqk.
So what it means is, you have a code, and you have a subset of positions, maybe this one, this one, this one, this one.
This is a subset of positions if the code words.
So if the matrix that remains after you take out the punctured columns, runs through all the q to the k elements of Fqk, then this is an information search
[INAUDIBLE]
Constrained to i, because i has size k. i is just -- its just about enough to describe every code word, if the restraint of C to the set would indeed be giving a unique vector for each code word.
The reason to call it -- so this is the definition of information set.
The reason to call it an information set, it's pretty straight, right?
Why is it called an information set?
Because it's enough, right?
Because it's enough.
If you know exactly the value of a code word in these positions, then it is enough to recover the entire code word.
When some genie tells you, gives you a code world which was corrupted by noise or something, but tells you, these k positions are OK. That's enough, that's all you need.
That's an information set.
You can recover the information from them.
Actually, it is an application that pops up sometimes.
That somehow, you get side information about some positions in the code word indeed being correct, and others not.
And others you don't know about.
So that's the information set, with respect to our MDS code.
So with respect to our MDS code, a corollary of the thing that involved any k positions in an MDS code, an information set.
So any k positions on information set.
It's a really strong property.
Really strong combinatorial property.
OK, so far, so good.
This is so strong, this property, that we can say something about these codes even without even knowing if they exist.
So, so far, we have talked about these codes as if we knew they existed.
Well, it's not entirely trivial, since we know those guys here exist.
So it's not entirely empty, we're not out in cuckoo space, here.
But do any other one exist, except for those?
That's the question.
We don't know that yet.
We will show in a little while that they do, but we don't know that yet.
But the interesting part is that we can derive properties of those codes without even knowing they exist.
And how do we do that?
For example, we want to derive the following property, how many words of weight d exists in linear MDS code?
One could ask that, right?
If they exist, they're nice, and if they exist, we also want to know how many words do exist at minimum distance.
Because that translates, again, directly into union bound arguments later on, and probability of error.
So that's a good question.
How many words of weight d exist in a MDS code?
Let's call this n d, and we want to know how many.
So I'll let you think about this for a sec while I erase the board, and then somebody will tell me the answer.
So how can we think about this?
Let's try to do a similar argument as this one.
Let's look at a single word, and let's assume that d positions, we ask the questions does there exist a code word within the first d positions?
It is equivalent to the question, does there exist a code word that covers exactly all d positions?
Any set of d positions
0 everywhere else?
And 0 everywhere else.
Why is that nice?
If you could prove that, that there exists a word for all d positions, because then, we pretty much know what happens.
Then we know that, well, if this is true, then there are n choose d ways to choose those d positions.
And then within those d positions, and since it's a linear code, we can multiply with the q minus 1 on the repeated element.
So if we can choose our d positions arbitrarily, then this is the number over words at distance d. So let's look at a word, and let's, without loss of generality, assume it's a first d positions.
So the first d positions.
So in particular, these would be the first d minus 1 positions, which would mean that this have length k. So if we have an MDS code, this is an information set.
So if this is an information set, then we can fill up this thing with just about anything we want.
So we choose this information set to be equal to 1.
This is how we choose this information set, and by the property of MDS code, we are guaranteed that there exists a code word which is this part in the information set.
But we also are guaranteed it's a weight d word, right?
The minimum distance is d, that means all of these m entries here, they must all be non-zero, in this part.
Otherwise, it wouldn't have weight d. OK?
And there we have it.
That was all we needed to show.
Right?
Because now we have shown that there exists a word of weight d in the first d positions.
Is that clear?
Let's try again.
I will say the same words.
Maybe it becomes clearer by that.
Let's look at a code word.
This is a generic code word, at first, and we want to answer the question, does there exist a code word within, which has support only in the first d positions?
So does there exist a code word which is non-zero here, up to d, and which is zero everywhere else?
That's the question we want to answer.
OK. Now here's what we do.
We look at this road and say, you know what?
Let's look at the last k positions, which have an overlap of 1 with this word here, because it's an MDS property.
So we have this relation between n, k, and d. And since any k positions in the word are in information set, so we can choose whatever we want in this part, and this is what we choose.
By the property of MDS codes, this corollary, we are guaranteed there exists the code word which in the second half of the code word looks like this.
And in the first half, it looks different.
There's something else here, and I say, well it cannot have any 0 in here, because then it would have weighed less that d. So it has non-zeros here.
So indeed, we have shown the existence of a code word which has non-zeroes in the first d positions.
Very simple.
And that was without loss of generality.
You could make the same argument for any d positions.
What have we shown?
We have shown that indeed, we can choose any d positions in the code to support the minimum weight code word of weight d. This is how many ways we can choose this, then we have to multiply it with q minus 1.
All non-zero field elements.
The reason is, we might have chosen this, or we might have chosen omega or omega squared here, or just the multiples, the scalar multiples of it.
Interesting, right?
This property, that any k positions on information set is really strong enough to prove the -- actually, it's strong enough to prove the entire weight distribution of an MDS code
[INAUDIBLE] [UNINTELLIGIBLE]?
No, no, no, why no, no, no, no.
So then you would get too much.
If you write q minus 1 to the d, then you would want to multiply each position with a different value.
That would imply that there's more than one code word in the first d positions.
More than one code word so that they are not scalar multiples of each other.
If that would be true, then you could find a linear combination which is still 0 in this part, but has additional 0 here somewhere.
But if that is true, then we don't have enough distance anymore.
Then it's not an MDS code.
All right, so it's indeed q minus 1.
Within each d positions, we have one dimensional space.
It's just one dimension
[INAUDIBLE] far off minimum weight code words?
Yeah, yeah, definitely.
Any other code must have less, so it would have less.
But every other code would have a smaller minimum distance
[INAUDIBLE].
Suppose we let the last k minus 1 position zero, and the one before that, [UNINTELLIGIBLE PHRASE].
And you said that we can do it for any of [UNINTELLIGIBLE] total field?
Sure
Since it's a linear code, some of those code words should be in the linear code, right?
Sure
So because it's a field, also we are going to [INAUDIBLE] there exists an inverse [UNINTELLIGIBLE]?
Absolutely
So if we add those two code words, we should have all zero, [UNINTELLIGIBLE] k minus 1, and have inverse at the one before.
We get that code word which has a minimum weight, which is less that the one we have here?
Good question.
There is a trick out.
There's a way out of this.
Great argument.
But there's a trick out
There's gotta be an upper bound
No, no, there is a trick, there is a way out here.
Namely, so let's put it like this.
Right here we put in a 1, just for simplicity, let's assume all the other positions are also 1.
And then you say, this would be another code word, which has here an omega.
I say, you know what?
What's going to happen?
All the other positions are going to be omega 2.
There's no way to combine these two guys to get an additional 0, unless you get all 0's.
Unless you get to 0.
That's what I said, it's a one-dimensional space in these positions.
When it's a soft code
[UNINTELLIGIBLE PHRASE]
It tells you that if you write down the minimum weight code words in the q minus 1 times d matrix, that is, you have a Latin square, basically.
That's what it tells you.
There's in no position, if you have anywhere in here an element alpha and element omega, the same omega pops up nowhere else.
There's ramifications of MDS codes in combinatorics left and right, so this would be a Latin square.
You know, you can learn a lot a lot about MDS codes if you think a little bit about that, and about combinatorics altogether.
OK, where was I?
So we know that's fun.
And actually, in the homework, you going to do n d plus 1.
So the next one.
But once you do n d plus 1, do all of them.
In a sense it's just inclusion and exclusion from then on.
The first one is sort of the toughest one, the rest is inclusion exclusion.
And just for the heck of it, when you go home and do the homework, write them all out.
It's a pretty looking formula, in the end.
OK, so far, so good.
So we have still talked about MDS codes without knowing if they exist.
Except for the trivial ones here.
And the existence of MDS codes is actually not known for which parameters they exist.
So I give you a research problem.
The research problem is the main conjecture on MDS codes.
And it's always sort of tricky.
When a research problem has a name, then that signifies danger.
Then it means that it's not trivial.
The question is, for which k d and q, for which sets of parameters n k d q, do MDS codes exist?
And the conjecture this is that n k q, because in MDS code we can actually get rid of the d here.
e, the longest length of an MDS code.
The longest length of an MDS code, I mentioned k over an alphabet of size q.
The conjecture is that n, k, d is less than q plus 1 for k at least 2, unless -- I always have to look that up -- I think 2q.
And k plus 1 for k greater than q.
We talk about it in a second, except that n, there's a 3, 2 to the s. So if the alphabet is a power of 2, alphabet size an extension field of 2, basically.
q plus 2 and q minus 1 q to the s. q plus 2.
OK, so this is the main conjecture on MDS codes.
Basically, it says that the length can essentially be as large as the alphabet size, but not larger
[INAUDIBLE]
This q, yeah?
Oh, yeah, n k q, sorry.
It doesn't make sense otherwise.
So the lengths can be in the same order of magnitude as the alphabet size.
That gives enough room, enough choices, to fill up this matrix with the information set, with the MDS property on the information sets.
This is the parity-check code, this row is just taken out as a trivial code.
And then, the demon of mathematics conspired that this would also be true.
So if you have an extension field of 2, and you want to give it a dimension three, MDS code, they exist for q plus 2.
Right.
There are, of course, reasons for this, but they go pretty deep, why they exist for those parameters, and this is just mysterious.
One can give reasons, so on another hand, it's just so, right.
There are exceptionally enough that they have names.
The first one is the Hexacode, it's something with a generator matrix, and this goes over F4.
So that's an MDS code of length six, so this is a n6, 3, 4, MDS code over alphabet size 4.
That's the first one, in that sequence here.
Anyway.
Otherwise, we have this conjecture.
If you solve this, you are going to be rich and famous, you're going to live in Hollywood, and maybe, maybe not.
But you're going to be probably not rich, you're going to be famous about a couple of hundred people who know about this MDS conjecture, but very smart people have been looking for this for a long, long time.
OK. All right, 20 minutes left.
So it's better we define, we make sure those codes exist.
Do we have any question about this MDS conjecture?
OK, last 20 minutes, let's at least make sure those things exist.
Reed-Solomon codes.
So Reed-Solomon codes cover this case.
They are examples of codes which lie, which satisfy this equality.
OK, so how do we define Reed-Solomon codes?
Now, just in a true mathematician spirit, write down consider the following.
Consider the following code.
See?
Beta 0 beta q minus 1.
The beta i are the distinct field elements, the distinct elements in the finite field.
f is a polynomial.
f is a polynomial, and the degree is less than k. OK, good.
So we have defined a code.
So what that means is we start from polynomials.
The set of all polynomials of degree at most k. So what do we know about that set?
It's a vector space, right?
The set of all polynomials of degree at most k. We can add them to get a polynomial of degree at most k, we can multiply them with a scalar to get a polynomial with degree at most k. It's a vector space.
So we take this vector space and evaluate for any element in that vector space.
This element in all non-zero elements of the field and we get a code.
We get a set of vectors, so we get a set of vectors, and that --
[INAUDIBLE]
Yeah, I took all elements.
Why not?
Why not all elements?
Strictly speaking, I should have taken one more in order to get the one here.
We can talk about that in a sec.
But this one more element would be -- so it's a code.
First of all, it's a code.
Right?
We all see it's a code.
And once you see it's a code, we ask, what are the parameters?
The parameters.
So length, length is the easy one.
Well, it's q.
What is dimension?
Dimension of C. What's the dimension?
It's a little bit tricky, that question.
I actually, at Illinois, we have to take a class on teaching.
How to become an effective teacher.
And one of the things they told us is that if you ask a question, you have to wait for 12 seconds to get an answer.
So what's the dimension?
There you go.
This mapping, this mapping from a vector space to a vector space.
This mapping, also called evaluation map, is a linear map.
It's a linear map, meaning that, well, let's start differently.
Let's start differently.
Do any two polynomials map to the same code word?
That you know.
That you cannot.
Are there any two codes, two polynomials, so are there f of x, g of x, such that f of beta 0, so that they coincide in all positions?
No, then they would be the same, right?
And the reason is because if there would be something like that, then you could just look at h is f of x minus g of x, which is just another polynomial of degree k. And this would have to vanish in all positions.
If k is less than q, it could not possibly vanish in all positions, because then the polynomial of degree k would vanish in more than k positions.
Fundamental theorem of algebra.
The very beginning.
So the dimension of C is indeed k, the same as the dimension of this vector space.
The dimension of the vector space of polynomials of the degree k minus 1.
And the distance, if k is less than q, the distance is equal to q.
The distance, what is it?
Same argument, roughly the same argument.
I think that's a linear code, so if it's a linear code, the minimum distance of the code is the same as the minimum weight of a non-zero word.
What's the minimum weight of a non-zero word?
These are polynomials of degree k minus 1.
What's the minimum weight of a non-zero word?
Well, we start out with the weight 1, and whenever the polynomial evaluates to 0, one of the weights drops out.
So I claim the minimum distance as the minimum weight, weight of the non-zero word, and this is n minus, well, if any of these polynomials vanishes in all, it vanishes in at most, k minus 1 positions.
At most, k minus 1 of these vectors here, of these entries, is equal to 0.
So it drops by, at most, k minus 1.
Drops by at most, k minus 1.
And there we have it.
There we have it.
There we have, oh, this is q.
There we have it.
There we have that the minimum distance of the code satisfies this equation
[INAUDIBLE]
What?
The dimension?
The dimension.
So, it's the same argument, roughly.
So I say, the dimension, so let's just say the size of the code is q to the k. When is the size of the q to the k, if no two elements in the space evaluate to the same code word?
But if two of them would evaluate to the same code word, then we would less size than the vector space had.
But if two of them evaluate to the same code word, that means this is true for all four positions.
Then we could define a polynomial h of the degree k minus 1. which disappears in more than k minus 1 positions.
I mean, all positions.
Cannot be, hence the size of the code is q to the k, so this is a linear map, dimension is k. OK.
So cool.
So we have it, right?
We have our MDS codes.
They exist.
Here they are.
They are Reed-Solomon codes.
Not all MDS codes are Reed-Solomon codes, but the ones we are interested in, they are
[INAUDIBLE]
Well, the distance is at least this, but the MDS bounds is at most this, so it's equal to this.
But the MDS bounds, so the MDS bound has this is.
So with that.
So it's indeed, they lie exactly bang on to this.
There are MDS codes, Reed-Solomon codes.
So that is good.
So we know what they are.
So incidentally, where do you think this one more point is that you would evaluate our polynomials in?
You've heard about projective geometries?
There's one more point, it's infinity.
You have, basically, if you look at the numbers, in order to close it up, you want to add infinity to that, too.
In order to get this one more, this one addition in length, you want to evaluate this also at infinity.
You will have opportunity to do that in the homework.
I looked at the homework and I was pleased to see this problem there.
I hope you will be pleased, too.
OK, all right.
Any questions about this?
Let's see what else I wanted to say.
Because it just gives me a few minutes to talk about a few properties of Reed-Solomon codes, a few properties of Reed-Solomon codes.
And what did I want to say there?
On nested codes, so an RS code with parameters n k, maybe we define them [UNINTELLIGIBLE] like this.
q is properly contained, k minus 1, minus 1.
This is pretty straight from the definition of RS codes.
The set of polynomials of degree at most k minus 1 contains the set of polynomials of degree at most k minus 1.
So they are nested codes, property one.
You will see this is important, that they are nested codes, for various constructions where Reed-Solomon codes take part in later on.
A punctured RS code is again an MDS code.
Why is that so?
Why is that so?
Well, you see it?
Say if you puncture a Reed-Solomon code.
That means we just choose to not evaluate our code in this, this position.
And this field element.
Well, we just drop that coordinate.
Does anything change in the arguments we have made?
Well, the length is now 1 less, the dimension, well, the dimension is still the same, as long as k is not larger than the length of the code.
The distance, still the same as the length, the distance is at least the length minus the number of 0's.
So that equation still holds.
Well, but that's all we needed.
Still MDS code.
So there was really no -- it was not important.
It was not important if you took all field elements, or a subset of the field elements with MDS property.
That has nothing to do with it.
In particular, we often in the end, we often will drop the 0 element.
We often choose not to evaluate these polynomials in the 0 of the field.
A punctured Reed-Solomon code is an MDS code.
So what else did I want to say about this?
What else did I want to say about this?
A generator matrix.
How would a generator matrix look like?
Yeah, how would it look like?
Basically, we can come from here, right?
We can take the generators of that space.
So basically, we say that one -- generate the set of polynomials, that vector space of polynomials with -- so this is the basis of that vector space.
So if we map that basis, then we get a basis of the image of the mapping.
And the mapping of that basis would give this.
So we evaluate the function 1 in all field elements -- gives us 1.
We evaluate the function x in all field elements.
This gives us the next generator of the Reed-Solomon code.
Well, 0 gives 0, 1 gives, oh, let's write like this.
We evaluate it in all field elements.
These are all the field elements.
The next one, and this goes up to beta -- OK, so this would be a generator matrix.
That's fine.
So now, in order to make things a bit more interesting, do you have to stop five minutes early?
We just started five minutes late?
OK then, I think that's over.
I think it's over.
One more thing for you guys to think about until you reach home, then the rest you do next time.
So let beta 0 be equal to 0 beta 1, or beta i equal to omega i minus 1 where omega is primitive in the field.
Then we can write the matrix v of omega.
I tend to see that the first k columns, the first k rows of this matrix would be a generator matrix of a Reed-Solomon code.
Of course it's the same as [UNINTELLIGIBLE].
If we now delete the first position, we erase the first, we puncture the first position all out, and we look at the rest of the matrix.
This factor of the matrix.
Does this remind anybody of anything?
It's a DFT, it's a Fourier transform.
And that's what we start with next time.
So think about why this is a Fourier transform.
And maybe that's a nice analogy.
So we get the distance.
The distance is at least something, which means it's not impulsive.
It's not a single 1 somewhere.
The vector that we get is not impulsive.
Maybe it has something to do with the bandwidth constraint and the frequency domain.
That's what you have to think about on the way home, and that's it.
Thanks so much.
So OK, what did we do last time?
Thanks for coming back by the way.
So last time, we did MDS codes.
MDS codes, and we looked at this.
And we derived some generic properties of these things if they existed.
And then eventually, we moved to Reed-Solomon codes.
And so indeed, they exist.
We talked a little bit about existence altogether, the main conjection, MDS codes, I just wrote it down.
And like I said, you want to be rich and famous, solve this.
There's people doing geometry in math that would be deliriously happy if you solved it.
So if for nothing else.
So the last thing I wrote down last time was something about a matrix which somebody recognized as some Fourier transform matrix.
And that's where I want to pick up here, at least quickly make that connection.
So let me rewrite the definition of Reed-Solomon codes again.
And so a Reed-Solomon code, the way we defined it last time was all field elements, we would evaluate the polynomial in all field elements.
Now we do it slightly different.
Let me make it Solomon-Stein in order to denote that difference.
And let's say this is beta 0, beta n. So that's the old definition.
Only the, let's say, the beta 0, beta 1, up to beta n, they are the non-zero elements now.
And just for the heck of it, so we talked about punctured Reed-Solomon codes last time that it would still be MDS codes.
So nothing has changed there.
In particular, all the arguments would be the same.
In particular, just to humor me, let's say beta i is omega to the i, where omega is primitive in Fq.
So what that means, you remember that primitive means that the powers of omega cycle through all the non-zero elements.
Remember the non-zero elements are a multiplicative group.
And it's a cyclic group.
And it's the generator of the group.
Good.
So once we write it like this, we can actually write down the whole Reed-Solomon code in some sort of transfer domain description in the Fourier transform.
And the way this goes is the following.
So from now on, I will make implicit identification without making them.
So this is an identification which I just jump around between.
A vector -- so this one would be a Fqn.
And Fx would be maybe 0 to MDS 1.
Let's leave it at that.
So I make this --
[INAUDIBLE]
Yeah, let's make this identification free.
It's just a vector of length n and a polynomial of length n. And so whenever I want this to have a low degree, the polynomial, then I just write that expression.
So this identification we make freely now.
And then how can we write F?
Let F now be code word in the Reed-Solomon code.
And then I say Fi.
What is Fi?
The i is element.
Well, we know that this is F of omega to the i.
So this is, at the moment, counted from 0.
So this is -- you write it out -- Fj omega to the i to the j, which is omega i j.
And now, at this point in time, everybody recognizes this.
This would be the discrete Fourier transform of a vector.
This is an element of order n. Usually, there's e to the j and then an element of order n. Now it's in the finite field.
But what's the difference?
This would be the Fourier transform.
So a code word would be the Fourier transform of a vector F. Let's just make sure it also goes the other direction.
So I claim that Fl would be -- so what would be the Fourier transform if it, indeed, is the Fourier transform?
It is a minus, and then it goes n minus 1 Fj omega minus jl.
So the question is is that true?
Because that would be the inverse.
Is that true?
Well, let's do what we do with proofs of this type.
I need that space here
[INAUDIBLE]
Sorry?
[INAUDIBLE]
Yeah.
So let's do what we do with proofs of this type.
That means we write it out.
That's another beautiful thing about finite fields.
You never have to worry about convergence.
So for the F, we just plug that in here.
Sum, this would be -- what do we call it -- Fi omega to the ij.
So this would be Fj.
And then I want omega minus jl.
So is that true?
Is the identity now true?
Well, we see two sums.
What do we do with two sums?
If we're compelled to exchange the order, that's always what we do.
So Fi goes out.
And then you get omega to the -- what do we do -- i minus l times j.
We have this.
And so what is this?
How can we work this out?
Well, you know how to express the sum.
So this part of the sum is really just omega i minus l to the n minus 1 minus 1.
Well, that's a standard formula for summing these things up.
But omega is an element of order n. Omega was an element of order n. That's how we chose it
[INAUDIBLE]
It's what?
[INAUDIBLE]
You multiplied omega -- it's primitive in the field, so omega -- we have omega da-da-da-da-da i unequal to 1 for all i less than n and omega n is equal to 1.
So OK. That's fine.
So it's an element of order n. That means the n-th power.
We can interchange this.
So this is 1 to the some power remains 1.
So 1 minus 1 is 0.
The denominator -- well, actually, we get two results here.
If i is unequal to l, then this is 0.
If i is equal to l, then this was just the sum of n once.
So we get n here.
i equal to 0.
But n was q minus 1, the number of non-zero field elements because it was primitive in the field.
And this one is just minus 1 in the field.
Because q, we compute q is the power of the prime of p. So that would be 0.
If we just come out as minus 1, that explains this minus here.
And we can get rid of the question mark.
Look correct?
All right.
According to popular vote, it is.
So what do we really have here?
We have everything we want from a Fourier transform.
We have a Fourier transform pair.
We have a vector f, which we can think of as being in the frequency domain.
Then we have a vector F, which is a Fourier transform of that.
This one would be in the code if the degree of this polynomial here would be less than k. So now how can we understand Reed-Solomon codes now from an engineering point of view?
Classical -- what about band-limited functions?
What do we know about band-limited functions?
Because in a sense, the degree less than k, this means that Fi is 0 for all i greater than k. That's what it means.
So in a sense, it's nothing but a band-limited function if this is a frequency domain, if you consider this as a frequency domain.
So what do we know about the band-limited function?
Well, if a function is band limited, it cannot be impulsive.
We know that.
It's a part of the frequency, and the frequency domain is inversely relational to the support of the function.
So the whole idea of Reed-Solomon codes, in a sense, can actually be understood, at least intuitively, from Fourier transform and the duality between time and frequency.
Yeah
Was q a prime number?
q is not prime.
q is a power of a prime.
q to the n
[INAUDIBLE]
There always is.
Oh, sorry
[INAUDIBLE] or because for any q, there would be always a primitive?
Why there's always a primitive element in the field -- yeah, sorry.
I thought you had gone through that.
It's because the non-zero elements in a field always make up a multiplicative group.
And it's always a cyclic multiplicative group.
So it's a cyclic multiplicative group.
And it's just the generator of that group
[INAUDIBLE] it was a --
It's the same statement as saying, a cyclic group has a generator
If Fq, for example, was a z, zq, then q has to be prime
Yeah.
Yeah, in order for it to be a field, q has to be prime.
Right.
That's true.
That's true.
But we are not worried about -- There's also a beautiful theory [UNINTELLIGIBLE PHRASE] rings if this would not be a prime, very nice theory, becomes a little bit more technical.
You see, more technical, then more difficult.
I just wanted to give you this correspondence on the Fourier transform because it's, from an engineering point of view, a very nice insight.
One of the reasons one can understand that Reed-Solomon codes have a good minimum distance is because it has a transform of a band-limited function.
So that's what I wanted to say about the Fourier transforms here.
I do not want to spend too much time.
I have to get through decoding here.
Since we have now this Fourier transform correspondence, we can do another thing.
We can, namely, say, well, F is in the code if and only if the Fourier transform -- so that was the back transform here.
Also now, the code word, we can interpret the code word as a polynomial.
And the inverse transform is just evaluating it at an omega again.
F is in the code if and only the transform of a code word is 0 for all i greater than k up to n minus 1.
Yeah
[INAUDIBLE]
So it basically just means the support.
If this one is band limited, then this one cannot have arbitrarily small support.
That's what it means.
That's really all I wanted to say.
i greater than k and less than n. So we have that F is in the code if and only the polynomial evaluates to 0 at a bunch of points.
Now that leads to something interesting.
Because now we can say, well, so what if we've just construct our set of vectors F so that they evaluate to 0 somewhere where we want them to zero.
We construct them somewhat differently than here.
And in particular, let's define g as a product of i equal k up to n minus 1 of x minus omega minus i.
So it's a polynomial.
This is a polynomial which satisfies this.
So in particular, we have g of omega minus i is equal to 0 for all i less than n less than -- I'm sorry -- and greater than k. So this is a code word, right?
We are agreed this is a code word?
Good.
So we found a code word without going through this here, without going through the evaluation map.
We found it straight away.
But not only did we find this.
This has degree n minus k. This has degree n minus k. So maybe there's another interpretation.
Now we can give a whole new definition of the Reed-Solomon code, namely, as a set of polynomials, which are sort of implicitly identified with a set of vectors such that times this g of x and h of x is less then k again.
So what we have here, it's again, you take a vector space of functions the h. It's a k dimensional vector space.
And we multiplied with this g. It's again a linear map, this multiplication with a fixed polynomial.
A linear map of this vector space gives us, again, a vector space of dimension k. And all elements in this vector space evaluate to 0 for all omega to the minus i.
That's just the same again, just the Reed-Solomon code again.
So it's a very nice complementary description to the same Reed-Solomon code we could define up here, we have found again down here.
What is so nice about this?
Yeah, what do you think is so nice about this?
About this description?
So now you have to think as engineers
You can still evaluate it.
There, we have to evaluate it at end points of function
Yeah, exactly.
This is so easy to implement.
So now we think a little bit about encoder.
An encoder for a Reed-Solomon code.
How would we do that?
You look at this here.
Hey, let's do this.
Let's exactly do this.
Meaning in VLSI, we do something like -- and here, you would write -- and this is just a delay element.
This one would be g n minus k. This one -- so these would be the coefficients.
This polynomial, we can evaluate once and for all.
We do that once before we start our communication.
We evaluate that polynomial once and for all.
We know these coefficients.
So these are multipliers which multiply with those corresponding coefficients.
And we feed into here the coefficients of this polynomial.
This is our information symbols, which we simply feed into this circuitry.
And into this circuit, out comes F, out comes the coefficient.
Polynomial multiplication.
If you show that to a VLSI designer, they are deliriously happy.
They say, you know what?
I implement you this thing.
I implement you in 5,000 gates, which is close to nothing nowadays.
Maybe not.
Maybe 10,000.
And there you get to the second reason that Reed-Solomon codes are so extremely important in practice.
On the one hand, they are MDS codes, meaning they are about as good as it gets.
And on the other hand, they are algorithms.
They are circuitry for these things whose cost is close to nothing at least today.
When they were invented, that was quite different.
In the '60s, that would have not been implementable with transistors on a board or something like that.
But today, the cost is close to nothing.
So the whole algorithmic treatment of Reed-Solomon codes is very well developed.
The encoder you could do like this.
Yeah
Question about why do you still require that degree [INAUDIBLE]?
You don't need to do that.
But then the mapping is not one-to-one anymore.
Then the mapping is not one-to-one anymore.
Let's put it other ways.
You want code words of length n. If the degree is larger, you run over.
In order to still get a code word, then you have to take it modular x to the n minus 1.
And that would fold the coefficients back.
And still, it gives you a valid code word then, but this is not one-to-one anymore.
Anyway, the short answer is if you allow more, then they become longer than n. So what's the time?
OK.
So this is a nice encoder.
There even is a nicer encoder, which I just want to give the formula for.
The one reason that people still have a problem with this is it's not systematic.
People like to see the symbols in the code words themselves.
They want the systematic part.
How could we achieve that?
Well, we go from the same description here.
We say, well, let h of x be given.
Then compute x to the n minus k times h of x modular g of x.
So we divide this polynomial by this polynomial g -- it has a name.
It's called the generator polynomial.
We divide it by g. And out comes some polynomial r of x of some low degree, degree less than g. And degree r of x less than n minus k in particular.
So and then we can form a code word.
I claim F of x, no, F is -- and here we write r, and here we write h. The coefficient vector of h and the coefficient vector of r. Why is that a code word?
Why is that a code word?
Maybe minus here.
I'm always thinking characteristic 2 anyway.
So why is this a code word?
Anybody, it's clear?
What this is in terms of F of x is r of x plus h of x. Oh, minus r of x.
That's because we wrote it like this.
If we now take the F of x modular g of x, well, this part has degree low.
It's not affected by this modular operation.
This would be minus r of x plus this one, modular g of x, which is r of x.
So the whole thing is 0.
If this is 0, that means g of x is a factor of F. Hence, it's a code word.
So we have a code word here.
And in the code word, pop up our symbols right away, our encoded information symbols right away.
So we get some nice systematic encoding going.
This division circuit by g is pretty much the same size as this.
It's not larger at all.
So there, we get beautiful algorithms.
This is actually what's implemented.
If we go to any disc drive [UNINTELLIGIBLE], that is usually what is implemented in there, exactly this.
Algorithms.
Algorithmic treatment of Reed-Solomon codes.
Do you have any questions about any of this?
Yeah
I have a question about do any these Reed-Solomon codes map [INAUDIBLE]?
They are very costly map.
Usually, it depends a lot on the application.
If you think disc drives, [UNINTELLIGIBLE] as just being mapped, you take Reed-Solomon codes over a characteristic 2.
Then each field element is represented as a binary vector.
And that binary vector is mapped into on-off keying.
Straight off the bat.
Nothing more fancy.
That is for disc drives.
And as you do this, the satellite standard, where it's mapped onto the 256-QAM field elements [UNINTELLIGIBLE].
So it's many different ways.
Many different ways
But to prove some performance [UNINTELLIGIBLE] in the [UNINTELLIGIBLE], we need to have mappings, right?
In order to prove performance mode, we need to have mappings.
And the Hamming distance bounds that we get from here give you bounds on the minimum Euclidean distance.
If these are good bounds or not depends a lot on the modulation scheme.
And to be perfectly honest, they usually are not.
They usually are not very good, the bounds.
But it's a very difficult problem to design a code or to find a representation of the fields that maps nicely onto a modulation scheme.
Very difficult problem
How do we know how to think that this is a good code?
The code itself is excellent in terms of MDS, the MDS property
But why does MDS mean good codes?
In respect to modulation?
Yes
It doesn't.
It doesn't.
It's a bit like this.
It's really not easy to define codes in Euclidean space.
So all that we do is we find ways to do that and guarantee some performance, some sort of performance.
It's not easy to spread out -- I don't know -- 2 to the 1,000 points in 1,500 dimensions.
These would be typical numbers really.
It's not easy.
And since that problem is practically daunting, it's a daunting task, you have to develop all sort of crutches to do that.
And this is really one.
So how to code MDS codes playing a part in this mapping.
If you want to be more fancy about that, then you put a convolutional code or some other code also in there and do a combine scheme.
I think Professor Forney will talk about that more.
So here, it's just the coding theoretic groundwork of these things.
Anything else?
Yeah
[INAUDIBLE] of the [INAUDIBLE] n minus 1.
Close, very close [INAUDIBLE]
Yeah, thanks.
So the algorithmic treatment of Reed-Solomon codes is extremely elegant.
And that's the second main reason they are so much used.
It doesn't cost so much to implement them.
Well, at least we've seen that for the encoder.
That's a fairly small circuit.
So what about decoding?
Decoding.
How do we decode these things?
How could we possibly decode them?
And I give you --
Fourier?
Right, that's true.
We could do the Fourier transform.
But it doesn't help us so much because we receive something.
And we receive a vector, say, y.
From now on, let's say x is a code word.
rs, right.
So x, it's usually a code word from now on.
So this is a code word plus an error.
So in particular, if we take the Fourier transform, we take the Fourier transform of the code word, which is fine.
But then we get the Fourier transform of the error.
So that destroys all the fun.
What else could we do?
Here's typical parameters of a code.
255, 239, 256.
And you immediately see that, OK, any sort of group force is out.
How many code words do we have here?
We have 8 to the 239 code words.
Now you don't want to search that.
You definitely don't want to search that.
So how could we possibly decode these things?
Turns out, to decode them in some sort of optimal fashion, maximum likelihood [INAUDIBLE] actually, it's an NP-hard problem.
Maybe last year, actually, it has been shown that it's an NP-hard problem to decode Reed-Solomon codes.
It was known that decoding in general was NP-hard.
But this is now the constraint to Reed-Solomon codes is still hard.
So what do we do?
Yes, OK, what do we do?
Let's say x is a code word.
We know that.
And set rate e, let's just call it t. So what happens if you don't have an error?
Just thought experiment.
Thought experiment, if you don't have an error, then yi in all the positions is equal to F of xi for some F of x of degree.
In particular, since we know the x's, we know the positions -- sorry, oh, sorry.
That's not what I wanted to write.
This is definitely not what I wanted to write.
Let's keep that as c as a code word and position i in c is associated with xi in the fields.
So meaning ci would be F of xi.
That's really what I wanted to write.
It makes more sense.
So thought experiment.
No errors.
Then yi is F of xi for some F. If they ran no errors, then we could just solve this linear system of equations to find the coefficient of F. And the coefficient of F, say, were our information circuits.
Is it clear that this is a linear system of equations?
Yeah?
You could write it out as y0, y1, y2 equal to -- and here we have f0, f1, up to fk-1.
This is the linear system of equations.
We know the xi's.
This is a linear system of equation we have to solve.
If there are no errors, then life is easy.
That seems to be reasonable.
So what happens if we do have errors?
Somehow, we have to make sure that the errors that we get do not cause any problem for us.
And then we do something very ingenious.
We define something called an error locator which is a polynomial x such that x minus xi.
So it's a polynomial which is 0 in all error positions.
Well, you might say, we do not know the error positions.
Well, OK, that's true.
Basically, this is, in the end, what we want to find, this polynomial.
But nonetheless, this polynomial exists.
We can cast, actually, the coding problem -- this is form 1 s. Yes, what?
[INAUDIBLE PHRASE]
Yeah, it's an additive error model.
But you can cast pretty much anything in the [UNINTELLIGIBLE].
If a position is altered, you can always model that as if something was added to it.
Decoding problem one is find lambda of x of minimal degree such that lambda of xi, is 0 for all xi and degree f less than k. So I claim if you solve this problem, namely, this is a problem I give you.
I give you vector y. I give you vector y.
Here it is.
Here is vector y.
And I give you vector x which corresponds to the field elements where you evaluated that in order to get the code word.
And then I said, given y and x, find two polynomials lambda and f such that f has maximum degree k and lambda has minimum degree, the smaller degree possible so that this is true.
And I claim this solves the decoding problem.
This would solve the decoding problem because once we have found this, then we can take lambda to be the error locator.
And we can basically read off the information symbols from the f. So this decoding formulation now brings it, at least, into the algebraic ground, brings the whole decoding problem, makes it something algebraically.
But now the question becomes, is that easy?
Or can we do this?
This problem here.
Do you see any hope for solving this problem?
I guess the only answer that -- OK, anybody says no?
Anybody does not see any hope?
All right.
This is great.
You all see hope here.
You all see hope here, which seems to make you an opportunistic bunch.
Not opportunistic.
What's the word?
Optimistic.
Optimistic bunch.
Let's put it like this.
What is the problem in solving this?
It's not linear.
You get the coefficients of lambda.
Multiply the coefficients of f. That whole thing becomes a nonlinear problem, where we say in the end, find the solution to a set of polynomial equations in a field, which is a multivariate polynomial equations, where the coefficients of lambda and f are the variables.
You can do that.
You could use techniques like Grobner basis or so, and you could do that.
But this is very difficult.
This is computationally tedious.
So is that clear why this is nonlinear, and why this is hard to solve a nonlinear problem here?
If not, then you have to say something now, or forever hold your peace.
So what do we do with hard problems?
Once you take the optimization classes, there's almost like a reflex, there is a relax time.
We find the proper relaxation of the problem.
Now the proper relaxation of this problem is the following.
Decoding problem two.
Find lambda of x of minimal degree such that -- it's almost the same, almost the same -- lambda fi yi minus h of xi is 0, where the degree of h is less than k plus degree lambda.
So all that we did from this formulation, which would give us a clean solution to the whole thing, right now, we multiply in this lambda which gives here, it keeps the lambda times y.
And here, we get a new polynomial, lambda times f, which now has degree at most k plus degree lambda.
And then we say, let's instead solve this problem.
Let's solve this problem.
In particular, the question now becomes well, we do not require this anymore.
In this relaxed formulation, we do not require this.
And that makes all the difference.
It makes a world of difference because this one -- look at it -- it's a linear problem.
It's a linear program.
Why is it a linear problem?
Do you see it's a linear problem?
Could you write down the equation, the matrix equation?
It's pretty straight, right?
It's pretty straight.
You could, for example, write it like, here, a diagonal matrix yn.
Here, you would get something.
I think that seems to be all right.
And here, just the evaluation of the h. So up to hk plus degree lambda.
That's a linear system of equations.
We can certainly solve this.
Well, we do not know really what the lambda is, what degree the lambda has.
But we're just hypothesizing on all the possible degrees.
Well, we could say, OK, let's assume the degree lambda is 0.
It's a constant, which would be the same as saying there are no errors.
Then we can look at the system of equations.
Does it have a solution?
Well, yes, no.
If no, then we say, all right, let's assume it's 1.
Well, does it have a solution?
Yes, no.
And so on.
Then we can move on.
So we can solve this relaxed problem.
Does that help us?
It's nice to solve a relaxed problem.
And in the end, we get two vectors out of it, two polynomials, lambda and h. Does it really help us?
Well, yes.
Why?
Because -- let's put it like this.
We could easily check if this is true.
Once we have our h, we can easily check if this is true.
And if it is true, then we have solved this problem.
If it is true, we have solved this problem, which is the problem we wanted to solve.
Good.
So is that all we need to know about this?
We want to give guarantees.
We want to give guarantees that we correct up to so many errors, t errors, right?
So we have to guarantee that if there are not more than t errors, whatever t will turn out to be, we can guarantee that A, we will find a solution, and B, this will hold.
So two things to prove.
Is that clear?
That we have to prove those two things?
So the first one first.
When do we find a solution?
And are we guaranteed to find it?
Can we guarantee the existence of a solution?
Well, when can we do that?
Let's look at this.
These are n constraints.
This is a system of linear equations with n constraints.
If the total number of degrees of freedom exceeds n, then we are left with something non-trivial after we solve the system of equations.
So what's the total number of degrees of freedom?
The number of degrees of freedom -- so we get the lambda as a degree of freedom, so which is degree lambda plus 1.
And the other one is plus -- what is this one -- plus k plus degree lambda.
This is the total number of degrees of freedom here.
And the reason is that this one should be minus 1.
Sorry.
So this is total number of degrees of freedom.
If this is greater than n, then we can guarantee the existence of a solution, a nontrivial solution.
Then this thing will have a solution.
It's a homogeneous system of equations.
So the 0 is always a solution.
But that's not much good to us.
So what do we get here?
Degree lambda.
To a degree lambda greater than n minus k plus minus 1 or n minus k. So the degree lambda greater than n minus k over 2 -- does that remind you of anything here?
So this one is d minus 1 over 2.
So very interesting, right?
Once upon a time, you learned that if you make less than d/2 errors, you can correct that.
Very nice.
It pops up here.
It pops up here out of the blue.
The reason that it pops up here is, of course, the same statement, that as long as we stay within d/2 errors, we are guaranteed to be able to decode this.
So this is one.
So we know this one was the number of errors.
If t is greater or equal than this, then -- let's forget about the t. If you proceed in this algorithm hypothesizing the degrees of lambda, once we've reached this number in the degree, there will be a solution.
And we don't have to go further than that.
So that's the first one.
So the second thing we have to prove is that -- I shouldn't have done this -- that for some t -- Yeah?
[INAUDIBLE] standard degree of freedom or less constraints?
You want to guarantee a solution.
If you have more constraints than degrees of freedom, then, since it's homogeneous, you usually would be stuck with a zero solution.
Everything's 0, which is no good to us.
It solves this problem, but it doesn't give us any information.
Is that all right?
Why does it have to be the condition that degrees of freedom has to be greater than the constraints?
No, no, the degrees of freedom -- well, OK, it's true.
We could get lucky.
We could have redundancy in this system of equations.
And if we get lucky, that's just for the better.
And actually, you can show that you do get lucky if very few errors happen.
Then this would have low rank.
You can show that.
But short of knowing much, we want to guarantee -- I just want, at this point in time, to guarantee that there is a solution.
There is a non-zero solution to this system of equations, and the degree of lambda is not more than d minus 1 over 2.
There is a solution with a degree of lambda being upper bounded by this
So [INAUDIBLE] less constraints?
Yeah, no, no.
Once this is satisfied, I guarantee you there is a solution.
That means, the smallest solution is guaranteed not to be larger than that.
But there are, of course, many, many more larger solutions.
There are many solutions of larger degree here, which we are not interested in since we are interested in the minimal degree.
So this is an upper bound on the minimal degree lambda.
So now the other one.
So now we want to make the statement about this one here.
When can we guarantee that our relaxation didn't matter?
Despite our relaxation, our solution that we find that in the solution that we find lambda divides h. How can we guarantee this?
So how can we guarantee this?
How can we guarantee this?
Let's look at lambda xi yi minus h of xi.
And we know this is 0 for all xi.
We know that.
We know that this guy here can actually be written as ci plus ei minus h of xi.
This is just expanding this guy.
We also know that we can write lambda xi times ci alone minus f of xi lambda xi.
This is 0.
We know that this is true too
For some f
For some f. The text is to f, so that this is true too.
Then let's just subtract these two guys.
Let's just subtract these two things here.
And then we know that lambda xi times ei -- so the first two cancel -- minus h of xi minus So we know this is true too
[INAUDIBLE]
Yeah, sorry.
Just in time.
So what can we learn from this?
So what is the degree of this?
What is the degree of this?
Well, this guy here -- we'll write it somewhere else -- well, this guy had degree h was less than n minus k plus degree lambda less than k plus degree lambda.
That's because we set up that problem that way.
And we know that this guy degree lambda.
We know that.
So this whole thing, let's call it S. We know it's 0 for degree S less than degree lambda.
So why does that help us?
Well, let's look at the vector.
Now let's look at all the positions.
Let's look at the vector where we had, in the vector, we have lambda xi ei.
And now the fundamental question.
What is the rate of this vector?
At most?
What's the rate of this guy at most?
It's 12 seconds, [UNINTELLIGIBLE]?
Well, the rate of this vector -- it's definitely not more than the rate of the error.
Because every time the error is 0, the rate drops out.
That's at most t. What is the rate of this vector?
So here we have a vector on the other side, S of xi, a vector on the other side.
What's the rate of this guy?
That's just the polynomial of degree at most k plus degree lambda minus 1.
So we have that the rate of this vector is n minus k plus degree lambda minus 1 plus 1 because of the minus
Greater than or equal to?
It's greater than or equal, sorry.
Otherwise, I would have been in trouble in a second here.
Yeah, so what does that mean?
So actually, this one is nice to rewrite is equal to d n minus t plus 1 d -- I really should have done it like this in order not to -- it's d minus degree lambda.
So what can we learn from that?
So we know this is true for all the positions.
So if this is true for all positions, then -- so if d minus degree lambda -- actually, this is t because the rate was defined.
t was the number of errors.
And the error locator was just defined to have degree t. Right?
The error locator was just x minus xi over all positions where we had an error, so there are t of those positions.
The degree of lambda is equal to t. So now we take this.
If d minus t is greater than t, so if the rate of this vector is greater than the rate of this vector, then we have a problem because that cannot be.
Of course, by this identity, the rate of these two vectors is equal.
So what is the way out?
What's the way out?
[INAUDIBLE]
Yeah.
So if this is true, then S of xi I claim has to be 0 and lambda of xi times ei has to be 0 too for all i.
That's the only way out of this.
Because, obviously, this was allowed.
If we really did find what we wanted to find, namely, an error locator here, then this vector -- we said it has to be less than t. It can be less.
In particular, it can be 0 if this is, indeed, an error locator.
And if our relaxation didn't matter -- that means the h that we find factors as that -- if the relaxation didn't matter, then S would be 0 too.
So that is a fair solution.
That's a possibility.
And if d minus t is greater than t, then this proves this is the only solution that is feasible.
That's the only solution you have.
So what that means is if t less than t/2, then the relaxation doesn't matter and h of xi is equal to lambda of xi times f of xi.
So is that clear?
That's two very simple, yet nice ways to prove things.
Here, we guarantee the solution.
Here, we guarantee the solution.
Here, we guarantee that solution is correct, namely, if not too many errors happen, if less than t/2 errors happens, then relaxing the original problem here to this other form, which we could solve, does not change the solution.
That's what's proved there.
But about one-third looks puzzled.
About half looks puzzled, I would think.
Half looks puzzled.
Is there any way I can explain that better?
Think.
Let's just go through the steps quickly.
We had problem number one.
Problem number one, which is a nonlinear problem, but you see that if you could solve this, you could solve the decoding problem.
So you see that.
Anybody who doesn't see that?
All right.
You're smart guys.
If we can solve problem number one, we are home free.
We have solved the decoding problem.
Fine, we cannot solve it.
It's a nonlinear problem.
Well, we can solve it in exponential time, but that's little fun.
So we relax it.
We relax it into problem number two.
All that we do, we multiply things out.
And we do not require, once we solve the problem, this anymore.
Now we have solved this problem.
We have solved this linear system of equations.
And after doing so, we have found lambda and h. They are lying on the table and looking at us.
Now what?
Are they any good?
In particular, since a priori we cannot guarantee that this relaxation didn't completely destroy everything.
We have solved the system.
Now those two things are lying on the table, looking at us, and asking, what am I?
And in general, if there's an arbitrary amount of errors happening near the channel, actually, they do not mean much.
They do not mean much.
But now I claim that, well, if not too many errors happened, but if the number of errors is bounded to be this, less than half the minimum distance, then, actually, I claim it did not matter if we solved the relaxation or the original problem.
And the argument is roughly this.
It goes like this.
Here, we start.
We have solved this.
We have two polynomials lambda and h, so that this is true for all xi.
We know that we can write it like this.
That's just by definition of yi, just expanding the yi into this.
We know that this is true by the definition of the code.
The ci is the evaluation of f, of some polynomial f. So we can write this just by definition of the code.
So once we have these two guys here, we can subtract them [UNINTELLIGIBLE] here.
So we know that we have solved the following problem.
We have found lambda and h. So that there is guaranteed to exist the polynomial f, so that this whole thing, namely, S of xi, that S is a polynomial of degree at most -- what did we have -- k plus degree lambda minus 1.
So that's a semi-hairy step here.
Is that clear that we can guarantee the existence of a polynomial S of degree at most k plus degree lambda minus 1?
So that this equation holds.
Once we have solved this, we can guarantee the existence of this.
That's what we're saying.
We can guarantee the existence of this.
So now look at this.
If you write this out for all i's and put it in a vector, what's the rate of this vector?
Well, it's at most t because all the other e's are 0.
If lambda is an error locator, it is 0.
But we do not know that yet.
All we know, it's at most t. This is on this side, so that vector has a rate at most t. On this side, the rate of this vector is at least this.
Just by evaluating a polynomial of this degree, this is the maximum number of 0's we can get.
So now we have two equalities, this one and this one.
But obviously, the two vectors must be the same.
So how can these two vectors be the same and still satisfy these two inequalities?
Good question.
How can they be the same and still satisfy this inequality?
If t is less than d/2, then the rate of this vector would have to be larger than the rate of this.
If they are non-zero.
If they are non-zero, this would have to be larger than this because then d minus t would be greater than t. If this is true, this implies this.
If this is true, this implies this which implies that the rate of this vector would be larger than the rate of this vector if they are non-zero.
But how can that be?
Answer is cannot.
Cannot be, hence, they must be 0.
Both vectors must be 0 completely, which means this is an error locator.
Because otherwise, this wouldn't be 0, and this one -- where did I write it?
Somewhere I wrote it.
Where did I write it?
Oh, here.
So this S has to be 0 too.
If this S is 0, then h of xi is exactly this, means factors in the way you want it to.
So that's all I can say.
I could say it again, but it's exactly the same words.
And there's a limited benefit to even the repetition code.
So beautiful, right?
We have brought down the entire decoding problem for Reed-Solomon codes to solving a linear system of equations, namely, this one here or this one in short form, which, well, it's no problem at all in the grand scheme of things.
And that's the other thing that makes Reed-Solomon codes so beautiful.
They're access encoding, they're access decoding, they're everything that we want.
Let me do one more thing about the decoding just to bring it down a little bit, to talk a little bit about complexity.
Solving this linear system of equations has been subject to research for 30 years, 40 years.
So it started out with an algorithm which essentially solved that linear system directly.
It was formulated a bit different, but that's what it is.
It's called the Peterson-Gorenstein-Zierler algorithm.
They realized it was a polynomial time decoding algorithm, a nice decoding algorithm.
Then Berlekamp -- I should write down the name.
Berlekamp came up with a fast algorithm and square algorithm.
Massey had his own formulation of that algorithm, which was a bit more streamlined I think.
Then there was a later version, Berlekamp-Welch.
The complexity of these algorithms is all roughly the same, is all n squared roughly, which solved this here.
And then there's roughly nothing happening for 30 years.
And after 30 years, then Madhu Sudan made the next serious dent in the decoding history of Reed-Solomon codes.
So he found a way to solve this problem even beyond half the minimum distance.
And in hindsight, it's a very nice and very simple trick that he used.
So we started with the following.
We started that, well, if we have no errors, then the pairs of points xi, yi lie on this curve.
That's another way to formulate what we said that yi minus f of xi has to be 0.
We said, well, if there are no errors, all the points that we receive lie on this curve.
Because there are errors, we have to multiply this with an error locator and say this is 0 for all xi, yi.
So this was problem number one.
The relaxed form was lambda xy minus h of x 0 xi, yi.
That's the way we do it.
So what did Madhu do?
Very clever.
He said, all that we have to do is annihilate the effect of the error right in this.
The whole reason for that lambda was to annihilate -- so to take out the error influence out of this interpolation formula, to allow for a few errors to happen, which we can put in lambda.
Well, Madhu said, well, what about we take a polynomial in two variables?
A polynomial in two variables, then same thing.
If no errors happen, then all the points xi, yi will lie on that curve.
If a few errors happen, then let's find a lambda xy, two variables of some minimal degree, some very small degree so that this is still satisfied.
And the problem is entirely the same.
And this one, you cannot solve either.
You cannot solve either.
But again, you can solve the relaxation minus -- again, you can solve the relaxation.
This is, again, a linear system of equations.
And the coefficients of lambda and psi now.
Once you solve this linear system of equations -- actually, this one you can more handily write simply -- now there is no way to distinguish the y anymore since both sides anyway depend on y.
Find a polynomial in two variables such that this is 0 for all xi, yi.
This is his central problem.
This is his relaxation.
Find a polynomial in two variables, which he evaluates to 0.
And then he has, essentially, the same proofs we have here, a bit more technical, but not much.
I'm sure you can come up with this if you sit down at home.
Let's put it like this.
There's no heavy machinery in it.
There's no heavy math in it.
There's a lot of being clever in it, but there's no heavy math in it.
He can now guarantee that q of xy, if t is less than n minus square root of 2k -- is that right -- 2kn, then you can guarantee that y minus fx is a factor q of xy.
The same thing we wanted to guarantee.
And like I said, the proof is very clever, but no heavy machinery in it.
It's no heavy algebraic geometry or any of this stuff in that.
It's high school algebra.
Well, freshman college algebra.
So that's what happens here.
It took 30 years and a computer scientist to solve that problem.
That was not all I wanted to say, but that's all I have time for.
There's one minute left, so there's no point in starting something now.
Do you have any questions about any of this?
I should say that this 2 one can get rid of, but that takes a little bit more machinery.
The 2 one can get rid of, but that's a bit more heavy.
All right.
So thanks so much.
That's it.
That's it.
All right, I thought he would be.
It's always a hazard putting up somebody who you know is going to lecture better than you do.
But I hope that was a good change of pace.
And, of course, he knows more about the subject of Reed-Solomon decoding than I or more than practically anyone else does, so -- did he have much time to talk about decoding or was the most -- yeah.
Talked about the decoding algorithms, Sudan-type decoding algorithms.
Yeah?
OK. Good.
Well, in fact, he covered so much that I don't have much left to do today.
I remind you, Ashish put out an email.
There's a review session tomorrow for the midterm from 5 to 7 in a different room, 36-112.
The midterm is Wednesday in this room.
It starts at 9:05 to give you a little bit more time to work on it.
It's closed book except that you're allowed to make up three normal size sheets of notes.
You can write as small as you like.
This experience has shown this is the very best way of getting you to consolidate what you've learned up to this point, is writing your own three-page precis of the course to date.
So, yes?
[INAUDIBLE]?
What is the rule, both -- what?
Both sides
Both sides.
All right.
Six sheets.
If you can't write down the whole course in six sheets, then, well, you haven't been paying attention.
OK. And you can bring calculators, but we don't expect they'll be useful.
Please erase any erasable will memories if you do bring calculators.
These are things we tell undergraduates.
I guess they hardly seem necessary in a graduate course.
Any other questions about the logistics of the midterm?
What is the material going to be for the midterm?
Everything that's been covered up to today.
I won't hold you responsible for anything I've talked about today.
So it's basically chapters one through eight.
I really will only hold you responsible for what we covered in class, which chapters one, two, and three were so brief, you can't say we really covered them.
So really, chapters four through eight.
Any other questions?
All right.
You'll have more chance as we go along.
So as I say, Ralf really covered the main things about Reed-Solomon codes, which are the crowning achievement of algebraic coding theory.
They have a lot of favorable attributes.
They are optimum, from an n, k, d point of view, in the sense that they're maximum distance separable.
The parameters are as good as they possibly could be by this very basic Singleton bound.
They have efficient algebraic decoding algorithms.
And traditionally, this has meant hard decision bounded distance decoding algorithms like the Welch-Berlekamp algorithm or the Berlekamp-Massey match algorithm, which take hard decisions in, put hard decisions on symbols out.
But as Ralf suggested to you, now there are algorithms that can use soft decisions in, produce a list, at least, coming out, some indication of the reliability of each element of the list.
So they've been softened up.
Nonetheless, what are the negatives is, basically, that they work on bytes, or symbols, not bits.
So if we have a basically binary channel, as we very frequently do, then they need something else to make them adapted to the binary channel, or even on the additive white Gaussian noise channel.
I don't think we've gone through the calculation.
You could think of sending q or e symbols over the additive white Gaussian noise channel, but what would you need?
You would need a q or e signal set.
Now, if you pick that to be a q simplex signals set or a orthogonal or biorthogonal signal set, then that might work because the distance structure of such a signal set is like the Hamming distance structure in GF of q.
In other words, it's approximately equally likely to make an error to any other symbol.
The symbols are more or less equally spread apart in Euclidean space.
Just as when we count distance in Hamming space, we give the same weight to each possible kind of error or each possible kind of symbol difference.
So if you did that, then Reed-Solomon codes might work well.
That's one form of a concatenated scheme, which I will talk about in just a second.
But if you tried to apply Reed-Solomon codes to a binary input additive white Gaussian noise channel, just translate the, say, 8-bit bytes into bits and send them one bit at a time, then the distance structures don't correspond at all well.
It's much more likely to make a single bit error than an error that involves all eight bits or something.
And for that fundamental reason, you'll find that performance is not so great.
And certainly, this is not the way to get to capacity all by itself unless it's married with something else.
So that's something we always have to deal with in another applications, as I will talk about in a minute.
We naturally do want to correct bytes, or symbols, or blocks.
There are many bits.
And then Reed-Solomon codes are just exactly what we want.
They're optimal in any such situation.
And so what did I just say?
They're not the way to get to capacity, although they are sometimes part of schemes for getting to capacity.
And since getting to capacity in the additive white Gaussian noise channel is the theme of this course, then they play only a peripheral role in that theme.
Nonetheless, they're great codes.
They're in wide use.
They're part of every semiconductor manufacturer's core library.
You can buy (255,223) distance 33, 16 error correcting Reed-Solomon decoder in part of any chip package and it'll go at gigabits nowadays.
So it's kind of the all-purpose error correcting code, with the caveat that you need to somehow get what you correct to be bytes rather than bits, OK?
So today I thought I'd cover the last two topics in chapter eight and maybe, to the extent that time permits, go back to chapter seven.
But we'll see how we feel about that.
Applications of Reed-Solomon codes.
OK, basically, you can apply them wherever you want to correct blocks, rather than bits or packets or n-tuples or what have you.
So one obvious application is, when you send packets through the internet, if you use Reed-Solomon codes across the packet, then you can correct errors that occur in individual packets.
This is not the way the internet usually works.
The internet usually works by having an error detection code within each packet, a cyclic redundancy check (CRC).
You hash bits, whatever.
You provide some redundant bits within each packet if -- when you receive the packet, you check to see whether the parity check checks.
If there are parity check bits, then a very good rule of thumb is that the probability of not detecting an error for almost any type of error pattern is going to be of the order of 2 to the minus r. That's exactly what it would be if you had a completely random stream of bits.
Random stream of bits with r redundant bits in a block, the probability that all the check bits are random, the probability that they're all equal to 0, is 2 to the minus r. All right?
But it turns out that's very good approximation for just about any error mechanism.
So you pick r large enough.
It's often been picked to be something like r equals 32.
What kind of failure to detect does that give?
2 to the minus 32, which is what, about 2 the minus 9th, 2 to the minus 10th -- 10 to the minus 9th, 10 to the minus 10th.
Is that low enough?
It was certainly -- when I started in this business, that was considered just incredibly low, but now that we're sending mega packets per second, it doesn't seem quite so low anymore.
And so 32 is a little bit on the sloppy side.
There are going to be undetected errors that get through if you're spending 10 to the 9th, 10 to the 10th packets.
But 32 or, nowadays, 64 would be a better number.
Just like cryptography, the number of bits has to go up as technology goes up.
But 32 is often used.
OK, but here's an alternative scheme.
Suppose you coded across packets.
So here is a, say, a 1,000-bit packet.
And here's packet one, here's packet two, here's packet three, and so forth up to packet k, where we're going to use n, k, d equals n minus k plus 1, RS code over, it doesn't matter, GF of let's say 256, just for definite [INAUDIBLE].
Here I'm using older notation.
This is the same as F-256.
Or we sometimes write that as F 2 to the 8th.
OK, how are we going to do packet error correction?
We're going to code crosswise in columns, all right?
So these would be k. Let's make this eight-wide.
Just pick off a bite at a time here.
So each of these can be considered to be a symbol in GF of 256.
And then we'll code n minus k parity check packets down here.
So these are redundant check packets.
Encoded bytes at this point.
They're not packets yet.
But if we do that column-wise across here, we can fill in n minus k check packets and you can see the block length is quite huge here.
So you might have a delay or latency problem in executing this scheme.
Basically, you would have to send this whole thing before you could do error correction and then error correct the whole thing.
But how does error correction work?
You send this.
Perhaps some of the packets are erased or even received incorrectly.
You would then use some error correction or erasure and error correction scheme, which Ralf talked about, for correcting the Reed-Solomon code.
And you could correct up to d over 2 errors, or up to d minus 1 erasures, OK?
So this would be a very powerful packet error correction scheme.
What are its costs?
It's costs are quite a bit of redundancy.
It doesn't fit the internet architecture very well.
This is basically a point-to-point scheme.
So when you're sending from a to b, you'd have to send b an additional n minus k streams.
And if you're sending the same webpage, as you often do, to multiple people, you'd have to send this to everybody.
And it also doesn't have any feedback in it.
The great thing about ARQ is that you only have to send the information that's necessary.
You only have to restore the packets that are actually lost.
You send them again.
Whereas here, you're just kind of, a priori, deciding that you're going to send a great deal of redundant information down here to cover some kind of worst case, whatever you think the worst-case number of packets would be.
If there are more that worst-case number or packets, then you could retransmit again, but you're retransmitting this huge block, all right?
So for that reason, this is not actually used, as far as I'm aware, in the internet.
ARQ, wherever you have a feedback path, you should use it.
That's a general moral.
So ARQ is a wonderful scheme that is, on an erasure channel, that's a capacity-approaching scheme.
Just on a bitwise basis, if you have a binary erasure channel with erasure probability p, and then the capacity of the channel is 1 minus p, and you can reach that capacity if you simply retransmit each bit that was erased.
As it's erased -- it's erased, you send it again.
And a quick calculation shows you that the average number of retransmissions is p. And so the average reduction from 1 bit per second is -- 1 bit per transmission is p bits per transmission, you can achieve 1 minus p with no coding at all, just by retransmission.
So it's perfectly matched to the erasure channel.
And the internet, especially with parity checks, CRC, within blocks, long enough CRC is basically a packet erasure channel.
You really, in practice, hardly ever make a undetected -- let a packet with errors come through.
OK?
Any questions about this?
[INAUDIBLE]
The TCP/IP protocol just has a 32-bit CRC, I think, and if it fails to check, you ask for that packet again by packet number.
OK?
So Reed-Solomon codes, of course, are optimal if that's what you want to do.
You couldn't possibly have a better code for coding across packets.
But it's cumbersome in the packet environment.
OK. Let's talk about concatenated codes.
I guess I'll do it at the same place.
This is something -- I did in my thesis and last week I was at a tribute to Andy Viterbi, who invented the Viterbi algorithm, among other things.
Of course, everybody knows that the Viterbi algorithm, when Andy invented it, he had no idea it was actually going to be a practical algorithm.
It's become one of the super practical algorithms.
He was just trying to prove a certain result, exponential error bounds for convolutional codes.
And he thought the proof was easier to understand if you introduce this algorithm, which he describes perfectly.
It is the Viterbi algorithm.
But really, just a proof technique.
Didn't even realize it was an optimum decoder.
And then somebody, not him, not me, but in his company, fortunately, realized that gee, this could be very practical.
And that became a workhorse decoding algorithm.
Very similarly, concatenated codes are something that I did in my thesis in 1965.
And I was looking, basically, for a theoretical result.
How could you approach capacity with an exponentially decreasing error probability near capacity but with only polynomial increasing complexity?
And I looked into this very simple idea of basically taking two codes and using one to correct the errors of the other.
And I found that you could do that.
And I'd also never realized that this might actually be practical.
I remember going around giving talks when I was interviewing afterwards that I was looking for an industrial position.
I went down to places like Bell Labs, IBM labs, and did seminars talking about codes that were thousands of symbols long and everybody rolled their eyes and said, boy, this guy is not very practical.
And to my surprise and everyone else's surprise, this became and still is a very practical way of getting high-performance codes.
This, in conjunction -- well, I'll get into it, but this was, again, a workhorse coding technique and still is in wide use.
In fact, as you will see, this contains some of the ideas that go into modern capacity-approaching codes.
But the idea is very simple.
You can do it two ways, or probably many more than two.
Suppose we have a basically binary channel.
I'll just say a core physical channel there.
The channel is what you can't affect as an engineer.
So let's say we have a binary symmetric channel or a binary input additive white Gaussian noise channel or whatever you like.
And then we could put on it an encoder and decoder for an n, k, d binary linear block code.
OK?
Now, from the input and the output, what is this channel going to look like?
At the input, we're putting -- basically, per block, we put k bits in.
They're encoded into n bits.
They're sent over the channel, received some n-tuple of noisy received symbols.
The decoder tries to find the best code word.
The code word is identified by k bits.
Most of the time there'll be no error and these -- let's call this u in and u hat out.
u hat is going to equal u.
But, of course, let's imagine this is a relatively short code.
Let's imagine this is a maximum likelihood decoder or minimum distance decoder in the appropriate space.
Then the complexity is going to be of the order of 2 to the k. And so you can't let k get very big.
We'll find somewhat better ways to do maximum likelihood decoding.
But if you're really trying to do optimum decoding, the complexity is going to go up exponentially at this stage with the number of information bits k. So this is not the scheme that we're looking for that has only polynomial complexity.
We know that we can choose codes like this that with optimum decoding get to capacity.
That's Shannon's theorem.
But we don't know how to decode them with reasonable complexity.
But, all right, here's a start.
We could use a relatively modest length code that we can feasibly decode.
And this is what we get.
OK.
So we have blocks going in.
Block from an alphabet of size 2 to the k. Blocks coming out, alphabet size 2 to the k. Occasionally, we make errors.
What's a good idea to then do?
This is not very hard.
This is why you should have been in this field in the '60s rather than the year 2005.
There were lots of good ideas just laying around waiting to be picked up.
Class?
Anyone have a suggestion for how to make further improvement in this system?
[INAUDIBLE]
Add another code.
What should it be?
Let's put an encoder here for what?
[INAUDIBLE]
Everyone is extremely wide awake.
An n, k, d Reed-Solomon code over what field?
In principle, it should be over F2 to the k. Now, of course, if k is 64 or something here, you might not even go all the way to field of 2 to the 64 elements.
You might divide it up again as we did in the packet error correction.
But basically, the way this goes is you try to keep this as small as possible so that this can still work.
There's a tradeoff between the complexity of the two codes and therefore, the rates.
The decoder -- here we have an algebraic Reed-Solomon decoder, which, in the simplest case, could just do error correction.
Just take these k bits as a hard decision on u hat and try to correct any errors that occurred in this inner loop here, which we might think of this as being a 2 to the kre channel.
So this whole thing together is a 2 to the kre super channel.
Sorry if you can't read that, but you heard me say it.
So here's a channel that takes in 2 to the kre symbols, puts out 2 to the kre symbols and sometimes makes errors according to whatever the error probability you can achieve here is.
Or some of the discussion back when we were talking about hard decision with binary suggested that it would be better to pass along more reliability information here along with just the hard decision.
And the very simplest kind of reliability information is if this guy really isn't sure, if either the noise is too large so he can't make any decision or if the noise leaves him exactly halfway between two code words so it's a completely ambiguous decision, or whatever criteria you have, this might make erasures.
And I'm sure Ralf told you how to do error and erasure correction with Reed-Solomon codes.
Or this could be more likely that information just -- how likely was the code word you actually saw?
And there are various ways of devising an approximate likelihood metric.
You just do what makes sense that indicates that some words, you're absolutely sure of in this decoding.
And other words, you don't know -- if you have one or more good candidates or you have no good candidates.
So based on this reliability information, you can actually incorporate that.
Did Ralf talk about reliability-based decoding?
Soft decoding algorithms for Reed-Solomon codes?
No?
OK, well, that's a shame because he and Alex Vardy are the inventors of a very good class that's recently been improved.
So one of the knocks on Reed-Solomon codes is that, at one point, you might have said a minus can only work with hard decisions.
And then, fairly early, it was figured out how to correct to erasures.
But just in the last 10 years, we now have -- this is no longer true.
We can have soft, meaning reliability-labeled inputs.
And this is a plus.
That means we can do soft -- and we have corresponding algorithms that can do soft Reed-Solomon decoding.
They are, of course, more complicated than just straightforward error correction with hard decisions.
But there's still polynomial complexity.
They're maybe an order of slightly higher degree of n, but still polynomial in the block length or in k or d or whatever you're taking.
And so we now have some such algorithms that do make substantial gains.
And, of course, if he didn't tell you about them, I'm not going to tell you about them.
It's a rather specialized subject.
But it's something that's really happened the last 10 years.
And they've constantly been improved.
The first ones, like one I came up with called generalized minimum distance decoding, made a modest improvement.
In dB terms, 1 dB, where there's 3 dB to be got.
Now they're up to 2, 2 1/2 out of the 3 dB with these new soft Reed-Solomon decoding algorithms.
So anyway, if you pass some reliability information here, then you can improve the performance of this decoder.
This decoder knows which symbols are reliable and it won't change in the decoding in which are unreliable or completely erased.
Where there's no information about them, that actually helps you in the decoding.
If you knew that you had s erasures rather than s errors, this helps you quite a lot.
We can decode twice as many erasures as we can decode errors.
And similarly, soft information in between can improve the capabilities of your decoder.
All right.
So that's the set up.
You can see why it makes a certain amount of sense.
From a complexity point of view, what it does -- here, we're going to do maximum likelihood decoding, complexity order of 2 to the k. But we're not going to let k get very large, so this is feasible.
Then we're going to let the code get very long, as we know we have to do from Shannon's Theorem, using this outer code.
But now we know we have polynomial complexity Reed-Solomon decoding algorithms, even for the soft decision case.
So if we balance the complexity here, basically let this length be exponential and this length, as it is -- it's of the order of 2 to the k itself -- then we can get polynomial complexity overall, all right?
The overall complexity is going to be of the order of a small power of 2 to the k. The overall block length is going to be of the order -- is going to be linear in 2 to the k. And therefore, you're going to get polynomial decoding complexity.
The other question is performance.
And you can show that if you -- well, even if you don't use reliability information, you can get all the way to the capacity of this underlying channel here with exponentially-decreasing error rate for any rate below that capacity.
Not as good an exponent as if you just used random codes per Shannon and coded for the channel, but nonetheless, you basically get what you want.
So using two smaller, simpler codes to create one long code turned out to be a good idea.
First of all, for proving this theoretical result, you could get exponentially-decreasing error rate for polynomial increasing decoding complexity.
And within about 10 years, this is what people actually did.
And, in fact, what people actually did -- I've got a block code here, but what people actually did was they used a short constraint length convolutional code here, which we'll be talking about after the break.
Convolutional codes tend to have a better performance complexity tradeoff, particularly in soft decision situations than block codes do.
So in a practical system, it's almost always better to use a convolutional code than a block code.
We'll discuss the reasons why.
So this convolutional code here, there is a maximum likelihood sequence detector for the convolutional code down here, which is the Viterbi algorithm, which you've probably all heard about.
And this is very practical as long as the constraint length, k, now becomes the constraint length, nu, of the code doesn't get too large.
So for about 20 years, the standard for space communication was this was a (255, 223, 33) Reed-Solomon code over F-256.
This is a constraint length 6, rate 1/2 convolutional code.
This is a 64-state Viterbi algorithm.
So what's the -- again, if we have a look at the error channel, here is a stream of bits coming along.
After encoding and decoding, what we typically see is bursts of errors, all right?
So you get a burst that starts -- in convolutional codes, it can start at a random time, end at a random time.
You might get another discrete error event over here.
In other words, most of the time, the decoder is decoding properly.
What you're receiving is what you transmitted in a streamwise fashion.
The bits just are transmitted forever.
But every so often, the decoder gets off on the wrong track, puts out errors for a while, then gets resynchronized, and decodes correctly again.
Is my picture clear?
I'm using words, mainly, to tell you what it represents.
OK.
So now, basically, what you want to do is, just as we did in the packet communication case, you want to send multiple parallel streams, up to 256 of them, such that in this stream, you're going to get independent error blocks.
And this one, you're going to get errors occurring somewhere else.
They're going to have a typical length.
You can very well estimate what the typical length of error events is, but they occur randomly.
And now you use the Reed-Solomon code, coding across this, and that will basically give you this picture over here.
How would I get this?
One way is by using a block interleaver, which is described in the notes.
A block interleaver, you basically take a long block from the convolutional code and you just encode and transmit along here and ultimately decode.
And after a very long time, you come back and start transmitting the second row just as part of the same stream.
And then the third row and so forth.
And if the block length n, here, is long enough, the errors in different rows will be effectively independent, all right?
So again, you wind up with very long blocks in this scheme, but analytically, it works.
And in practice, this is roughly what's done, except there is such a thing as a convolutional interleaver, which again, operates streamwise and actually performs better in practice.
But I won't take the time to go into this because we're not going to really be getting into things like interleavers.
Do you see how this works?
OK.
So it works well theoretically.
It gives us the theoretical result we want.
Works well in practice.
As I say, this was the standard for space communications during the '70s and the '80s.
Around the end of the '80s, they upgraded this.
First of all, they replaced this by a nu equals 14, rate 1/6, lower rate convolutional code, so this now becomes a 2 to the 1fourth state Viterbi decoder.
They built this huge rack at JPL that had 8,192 parallel decoding units, add-compare-select units in the Viterbi algorithm, to decode this thing.
But, of course, they only need to do that in one place, in the space program.
Everything comes back to JPL.
So they built this BVD, Big Viterbi decoder.
And first, they used it with this code and then they souped up the Reed-Solomon code.
And by about 1990, 1991, they got this thing operating up within about 2 dB of the Shannon limit, OK?
Which was considered a heroic feat at the time, or even now.
So that kind of tells you, in the terms we've been using in this course, just what you can get out of this kind of approach.
At that point, that was considered coding is dead.
There's really nothing more to do.
We're within about 2 dB of the Shannon limit and it's inconceivable that we could actually get any closer than that.
Second half of the course will, of course, tell us how we can get within 0.0045 dB of the Shannon limit.
But that was the state of the art as of about 1990, 15 years ago.
Questions?
Comments?
OK. All right.
I'll just mention a third application is burst error correction, which really, we've already been talking about.
Suppose -- let's let this be a naked channel now.
Suppose you're transmitting with an interleave scheme like this.
And now, in the same way -- it's a radio channel or something and every so often, you have an outage just due to fading, or a thunderstorm, or somebody turns on his vacuum cleaner, or whatever.
Every so often, as you transmit, you're going to get bursts.
You don't have independent, identically-distributed errors.
The character of the channel is such that you see bursts of errors, which is precisely what you do in this super channel here.
Every so often, you see a bunch of errors if you have the original transmitted scheme to compare it to.
All right.
In exactly the same way, you can code -- with something like this block interleaver, you can code across channels and correct the bursts with Reed-Solomon codes.
And here, it's quite easy to show that, because Reed-Solomon codes have optimal parameters n, k, d, that Reed-Solomon codes do the best possible job of correcting burst errors that you possibly could do, given some specification for how long the bursts could possibly be and the minimum guard space between bursts.
This is classical stuff.
Reed-Solomon codes are optimum burst correctors if you describe the burst channel by max burst length in a minimum guard space.
Is that too brief?
Probably is.
Give you some impression of what we're talking about.
So Reed-Solomon codes are optimum for burst error correction, which, most real channels are bursty.
The only non-bursty channel that we really have is the additive white Gaussian noise channel, ultimately.
That is the channel in space communications, which is why you can apply a nice, textbook-type code like this to the space channel.
For any other channel, typically, you have to use some kind of interleaving to break up the bursts so that you can -- this simply is no information over a long time.
If you tried to encode crossways on this, you'd have to have a code that's long enough to see the average burst's behavior.
We'd make the burst have an ergodic model.
The way to do that in practice is to code across it so you get independent snippets of widely separated bursts which can be assumed to be independent.
Yeah
[INAUDIBLE]?
No different, really
[INAUDIBLE]
Yeah, in space channel -- all right.
Here, I'm talking about power limit.
That's a good question.
But if you think about the band-limited regime, let's suppose -- so this would be some [?
MRE ?]
channel.
You would use some multilevel signaling scheme like QAM with a large signal set or QPSK or something like that to get a basic transmission with a discrete signal set that doesn't lose too much over sending Gaussian over the channel.
Again, this will be later in the course.
We'll talk about this in detail.
Other than that -- well, all right, so now this is an MRE channel.
If M is large enough, then you could apply Reed-Solomon codes right here.
But typically, it's not.
M is like 16 or 64.
And so Reed-Solomon codes would still be too short.
You would not be able to get long enough codes over such a channel.
So a similar kind of thing can be done.
And the history of bandwidth-limited coding pretty much parallels that of power-limited coding.
What corresponds to block codes here is lattice codes for this channel.
What corresponds to convolutional codes is trellis-coded modulation.
And again, once you handle the basic physical channel with these codes that are well matched to that channel but whose complexity increases too rapidly if you make them long because of exponential decoding, then you can always clean up whatever errors you make.
You can think of this as a cleanup function out here.
Typically, the rates out here are very high.
You don't need much redundancy.
It doesn't cost too much in overall rate.
But even with very high rates, like 223 over 255, you can do a rather large amount of error correction.
You can correct up to 16 errors or 32 erasures with us channel.
So this is a cleanup function.
You basically work very hard in here, get the best error probability you can, which -- it doesn't have to be too low.
10 to the minus 2, 10 to the minus 3.
And then you use this to clean it up and that drives the error probability down to 10 to the minus 12 or 10 to the minus 15 or whatever you really want.
But the principles are the same in power limit and band limit.
But nice question.
Any other questions?
OK.
So as a result, Reed-Solomon codes are the general purpose code that has emerged from algebraic coding theory.
A good example of burst error correction is, you know on your CDs, there is an error correction scheme incorporated such that if you scratch the CD, it still works.
How does that work?
It could only be because there's some error correction in there.
What, actually, is the error correction scheme?
I'm not sure I know what the latest and greatest is, but generally, the error correction schemes for magnetic memory recording like that have used Reed-Solomon codes -- have used two Reed-Solomon codes in interleave fashion like this.
So the across code is Reed-Solomon.
The down code is Reed-Solomon.
As a result, if you make a scratch through the disc, you will cut all of these codes, but you will cut each one in a small enough place so that you can correct it.
So you can think of this as a actually a physical laying down of Reed-Solomon codes on the two-dimensional surface of the disc in such a way that -- well, even if your scratch came right across one whole channel here, then, of course, this decoder would be dead.
But you still have somebody decoding in this direction to pick up the errors.
And so that's why you can completely abuse these discs and they still work.
And when you think about what's actually going on, think of the transfer rates from a CD to your computer.
It's up in the hundreds of millions of bits per second.
Maybe its gigabits by now.
Perhaps it is.
And you've got this Reed-Solomon -- these are non-trivial Reed-Solomon decoders.
These are like this 16 error correcting decoder out here.
And so these are working away at that data rate, capable of doing decoding.
And they're cheap enough so that they're just buried in part of one of the integrated circuits in your disc player circuitry.
So that'd give you a physical sense for just how much error correction power and speed you can get out of this class of code decoder, OK?
So wherever you want an extremely powerful code and you're a little bit less concerned with getting to capacity, Reed-Solomon code is the way to go.
It's absolutely part of your toolbox, OK?
The last topic in chapter eight is BCH codes, which are binary codes.
So they're binary linear block codes of the class we've been discussing, comparable with Reed-Muller codes.
Reed-Muller codes were discovered in 1954 in two separate papers by Reed and by Muller in two separate points of view.
And there are a million points of view you can have for Reed-Muller codes.
We've used yet a third one.
BCH stands for Bose and Chaudhury and Hocquenghem.
Bose and Chaudhury were both professors in the US.
They wrote a paper about 1960 introducing Reed-Solomon codes, which, by the way, and introducing BC codes -- Bose-Chaudhury codes -- Reed-Solomon's paper was in 1960 also.
These were completely independent of each other.
And then this fellow named Hocquenghem in France, whose name I can't and few people can pronounce, turned out he had actually invented these codes in 1959 in a paper that unfortunately was in French, so few people read it.
But anyway, in honor of all three of these people, these are called BCH codes.
These fit right into the main theme of algebraic coding theory at that time.
Maybe stimulated it because these turned out to be cyclic codes, or at least as they were originally introduced.
And so when I was in school, which was in the early '60s, these were considered the greatest thing going.
People were interested in binary codes, algebraic codes because they are slightly better than Reed-Muller codes.
Now, I would say, there's sort of been a swing back.
Shu Lin gave a paper about 10 years ago that said Reed-Muller codes are not so bad.
In fact, in terms of performance versus complexity for certain kinds of decoding schemes, trellis-based decoding, Reed-Muller codes are better than BCH codes.
So from a more modern prospective, we're more interested in Reed-Muller than we are in BCH.
Nonetheless, you'll hear about these a lot if you read the literature and so you ought to know what they are.
OK.
Conceptually, what are they?
Since we already know about Reed-Solomon codes, the way that I'm going to characterize BCH codes is as subfield subcodes of the Reed-Solomon codes.
Now, if we have a Reed-Solomon code over F2 to the 8 or something, the field with 256 elements, or any power of 2, contains a subfield which is just 0 and 1, all right?
All fields contains 0 and 1.
If the field has characteristic 2, then 0 and 1 all by themselves are the prime subfield.
So it's possible that there are certain words in the Reed-Muller code -- sorry, in the Reed-Solomon code that are completely made up of 0's and 1's.
For instance, the all-0 word is certainly in the Reed-Solomon code, so we have, at least, that one that's made up of all zeros.
Almost always, we find that the all-1 word is a code word.
So we have a code with all 0's and all 1's.
And there might be more of them.
And that's actually very straightforward, to find out how many completely binary words there are in a Reed-Solomon code.
So it's the binary subfield.
We have F2 is a subfield of F2 to the M for all M. OK, so a subfield subcode, if we have an n, k, d equals n minus k plus 1 Reed-Solomon code, then the set of all BCH code is the set of all binary code words, code words that just use the 0 and 1 of the code alphabet.
Call this c in C. So we call this C prime.
OK, so what are its parameters going to be?
Its length is still going to be n, right?
All the code words here are n-tuples, so these have got to be n-tuples.
We don't know its dimension.
Let's just call it k prime.
That's how many such code words are there?
We're going to have to do some combinatorics to find that out.
And what's the minimum distance going to be?
Anybody?
[INAUDIBLE]
In general, well, it's certainly not going to be less than because, by definition of the minimum distance, the minimum Hamming distance between any two code words in this code is d, so we're just looking at a subset of the code words in this code.
And the minimum squared distance has to be at least d. There's an example given in the notes.
If we let c be the 4, 3 -- I think it's the 4, 2, 3, over F4, then C prime turns out to be the 4, 1, 4 code over F2.
In other words, the only two code words in this code that are all binary are the all-0 and all-1 sequences.
All right?
So we actually get a minimum distance of 4.
It has to be at least 3 in this particular case, so here's an example where it could be greater than 3.
Or if there are cases where the subcode just consists of the all-0 word, then that, by convention, has an infinite minimum distance.
So distance could be greater, but it never can be less than what you started with.
On the other hand, the dimension is clearly going to go down.
If it were still equal to what it was, then we'd have a binary MDS code and we know we don't get binary MDS codes.
So in general, k prime decreases quite a bit.
In this case, it just decreases by 1.
And we have to find out how it decreases.
OK.
I'm a little uncertain as to how much of the derivation -- so we need to find --
Is that [INAUDIBLE] at all points of the field, is it?
No.
One of the characterizations you have of Reed-Solomon codes, I assume, was that they're the set of all polynomials of length n that evaluate to 0 at some consecutive set of n minus k powers of alpha.
Did you see anything like that?
Yes
All right.
In other words, they're -- let me write that down because RS code is the set of all multiples f of x, g of x, of degree less than n where of g of x equals the product of x minus alpha to the i for i equals 0 to n minus k minus one or something like that.
This is called a generator polynomial and -- so what does this mean?
In other language, all polynomials of degree less than n that have roots at 1 alpha, alpha squared, or so forth.
It's better to make this i equals 1 to n minus k. You can also make it minus i.
There are lots of details in here.
But I'll just ask you broadly to recall that characterization.
This is a --
I have another question.
[INAUDIBLE] are not linear?
OK, let's ask that.
So we've got a set of binary n-tuples, OK, defined as the subset.
So what do we need to do to see whether it's linear or not?
[INAUDIBLE]
We just check the group property.
Is it closed under addition?
Suppose we take two of these code words we add them together, component-wise, vector form.
Seems to me we get A, that gives us another code word in the Reed-Solomon code because it's closed under addition, and B, by the addition rules, it's going to be another binary code word, all right?
So it's another binary code word in the RS code, therefore, it's in C prime.
So that's proof that, in fact, this code is linear
What about [INAUDIBLE]?
That's trivial.
For a binary code, we only have to consider two scalars, 0 and 1
[INAUDIBLE]?
But now I'm asking whether this binary code is linear.
There's the code over F2.
Yes?
You had a --
Same question.
The base field is F2 to the x, so if you add 1 and 1 there, it wouldn't be just 0.
But I guess if you restrict the field to F2, you have to check everything again, don't you?
No, because we said F2 is the subfield of F2 to the M. And what we actually showed was that if you just stay in the prime field, the integers of the field, then you get exactly the same addition and multiplication rules as you do in the prime field.
So in fact, in a field of characteristic 2, we always have 0 plus 1 is 1, 0 plus 1 plus 0 is 1, 1 plus 1 is 0.
That last one is the only one you really have to check.
So that accounts for the definition of characteristic, and that's the reason we can always use just plus signs, because subtraction is the same as addition
[INAUDIBLE]?
Yeah.
We showed -- well, very simple proof.
The order of a subgroup has to define the order of the group it's in.
The additive group of the prime field has order 2 to the M, so it has to be some divisor of 2 to the M, which is a prime.
I think we proved that another way, but that's a very simple proof.
OK, great questions, because it shows you're thinking about how this relates to what you learned back in chapter seven.
But at this point you've kind of got a little vague about chapter seven, what's really in there.
By Wednesday, you're really going to know chapter seven well.
Let me tell you.
OK. Good.
Thank you for the help.
Because these are all things that I should have just said up here, but they come out much better, actually, if they come out in response to questions.
OK.
So here's one characterization of a Reed-Solomon code which apparently you did see.
It's the set of all multiples of some generator polynomial.
The generator polynomial is described as this polynomial of degree n minus k whose roots are precisely alpha up through alpha to the n minus k. You did say that?
OK.
So similarly, the BCH code -- I'm just going to assert this just to show you how it goes.
It's the set of all multiples of g prime of x, which is the minimal degree binary polynomial with the same roots.
Now, how can a binary polynomial -- alpha to the n minus k. How can a binary polynomial have non-binary roots?
Well, it clearly can.
Let's consider g of prime of x.
It's an element of F2 to the x, the set of all polynomials over F2, which is a subset of the set of all polynomials over 2 to the M for the same reason.
F2 is the subfield of F2 to the M, so some of the polynomials over F2 to the M have purely binary coefficients, all right?
So that's the sense -- that's how we can get between these two things.
So some binary polynomials have roots in GF of 2 the M let me give you an example.
Let's consider x squared plus x plus 1 as a polynomial in F4 of x, which is certainly is.
It's a polynomial of degree 2 and F4 of x.
What are its roots in F4 of x?
Does it have any roots in F4 of x?
Does it have any roots in F2 -- sorry, I'm asking whether it has any roots in F4.
Now I step back and ask if it has any roots in F2.
So how do we do that?
We just ask if F of x is 0 for x equals 0.
We get 1.
So it's not a root of 0.
Substitute 1, we get 1 plus 1 plus 1.
1 is not a root F of x.
All right.
Well, we have two more candidates.
How about alpha?
For alpha, we get F of alpha equals alpha squared plus alpha plus 1.
And if you remember the addition table of F4, however it's constructed, this equals 0, OK?
0, 1, alpha, alpha, squared.
0, 1 alpha, alpha squared.
0, 1, alpha, alpha, squared.
This is plus.
1, alpha, alpha, squared.
0, 1 plus alpha is alpha squared.
1 plus alpha squared is alpha.
Any of these is telling you the same thing.
0, 1, 0, 1 alpha, squared alpha.
So we conclude from that that 1 plus 1 plus alpha, which is alpha squared, plus alpha squared is 0.
So alpha is, in fact, a root of this.
And what about F of alpha squared?
Alpha squared squared is alpha fourth, but we can reduce that mod 3 -- the exponent mod 3.
So this is alpha plus alpha squared plus 1 and that is equal to 0.
So alpha and alpha squared are roots of this polynomial.
There's a lovely fact proved in the notes, which is that every element of -- facts.
Every element of F2 to the M is a root of x 2 to the M plus x. OK, did you see something like this last time?
I see some shaking of heads.
OK. And how many roots could there possibly be of x 2 to the M plus x by the fundamental theorem of algebra?
2 to the
2 to the M. So that means that x 2 to the M plus x -- and for every root, it's got to have a factor of the form x minus, or in this case, plus beta.
So this must have a complete factorization of the following form.
x 2 to the M plus x is simply the product of x minus beta.
All right.
In fact, you can see that 0 a root, obviously.
1 is a root, obviously.
And we've checked up here.
Let's see, what are we interested in?
Here's another example.
x to the fourth plus 1, f of -- if we have g of x equals x to the four -- sorry -- plus x, then g of alpha is alpha to the fourth plus alpha.
But alpha to the fourth is equal to alpha, so that's 0. g to the alpha squared equals alpha to the 8th plus alpha squared.
Alpha to the 8th is alpha to the fifth is alpha squared, so that equals 0.
So, in fact, that's true for that case.
Every irreducible polynomial g of x and F2 of x whose degree g of x divides 2 to the M -- is that right?
This may be 2 to the M minus -- I think it's whose degree divides M -- is a factor of x to the 2M plus x in F2 to the x. OK.
So let's give you an example again.
Example M equals 2.
This means, what are the irreducible polynomials whose degrees divide 2?
The prime polynomials of degree 1 certainly divides to our x and x plus 1.
Do they divide x to the fourth plus 1?
Yeah, they do.
How many prime polynomials are there of degree 2?
There's only one, x squared plus x plus 1.
And if you divide it out into x to the fourth plus 1, it goes evenly.
It turns out that what this implies is that this is actually an if or only if.
A polynomial g of x divides x to 2 to the M plus x if and only if g of x has a degree which divides M and g of x is irreducible.
So this implies that x to the M plus x factors in F2 of x into the product of all irreducible polynomials whose degrees divide M. And I think you did a homework problem that used this, even though it was never proved in class.
So you at least read about it.
Again, for this example, that should mean that x to the fourth plus 1 equals x times x plus 1 times x squared plus x plus 1.
Is that true?
Pretty easy to see that it is.
x plus 1 times x squared plus x plus 1 is just x cubed plus 1.
Multiply by x -- I'm sorry, this x fourth plus x.
How many times have I done this?
Should be x to the fourth plus x.
Yes.
OK, it's kind of miracle when you first see it.
This works over any field for any PNM again, lest you get this kind of factorization.
OK, but now, comparing this to this, what does that mean?
That means this is one factorization.
This is in -- we can do a complete factorization in F2 to the M of x.
We can only do complete in the sense that we reduce it to a product of degree 1 polynomials.
In F2 of x, we can't do a complete factorization.
This is similar to, over the complex field you can always completely factor a polynomial into one-dimensional polynomials of this form, where beta is the set of roots of the polynomial.
Whereas over the real field, you might have to have some degree 2 polynomials in there.
This is exactly analogous.
You can't get complete factorization over this subfield.
But what does it mean?
Here we have a polynomial of degree 4, has 4 roots.
We factor like this.
So each of these factors must contain a number of roots equal to its degree.
So what are the roots?
This is the polynomial that has root 0.
This is the polynomial that has root 1.
As we just showed, this is the polynomial that has the roots alpha and alpha squared, OK?
So it's always going to turn out that way.
So you can group the field elements into subsets where each subset is the set of roots of some irreducible binary polynomial of some degree that divides M, OK?
These are called cyclotomic subsets.
Whence the name of Elwyn Berlekamp's company, Cyclotomics.
All right.
So this is where, I think, the theory gets just absolutely lovely and unexpected, at least when you start in.
And people who have a taste for these things, I encourage you to read a real book on the subject.
Berlekamp's is one of the most elegant.
But there are lots -- because it's an elegant subject, there are lots of elegant treatments of it.
All right.
So remember what we were trying to do in the first place.
We were trying to find the dimension of BCH codes.
So where are we now?
We've defined a BCH code as the set of all binary polynomials with roots alpha up to alpha to the n minus k where n and k are the parameters of the parent Reed-Solomon code.
And alpha is in the parent field.
So alpha in F2 to the M. OK. One final fact, I guess, we need to know here.
All right, if alpha is a root -- now I'm talking about any field -- then alpha squared is a root of a binary polynomial.
And again, this is due to a very lovely fact, which is that squaring is linear in fields of characteristic 2.
Why is squaring linear.
A little side calculation.
a plus b squared equals a squared plus 2ab plus b squared but characteristic 2.
We can wipe that out.
So in general, f of x quantity squared is f of x squared.
And that's why, if f of x evaluates to 0, then f of x squared must evaluate to 0 since 0 squared is always equal to 0.
All right.
So alpha is a root if and only if alpha squared is a root.
Is that true here?
Yes.
Alpha alpha squared.
Well, is it also true if we squared alpha squared?
What is the square of alpha squared?
It's back to alpha, all right?
So these sets now turn out to be cyclotomic cosets, they're called, which is the set of all elements and their squares.
We get alpha, alpha squared, alpha fourth, alpha eighth.
And then, at some point, it cycles.
Why?
Because what does alpha to the 2M equal?
[INAUDIBLE]
No.
Alpha.
Right.
Alpha to the 2M minus 1 is 1, so alpha to the 2M is alpha.
So it comes back to alpha again because of that.
So we get a finite number of roots which, of course, are equal to the degree here.
In some cases, it's fewer.
It's whatever the degree is.
It's d, which divides M. So from this, we simply start -- suppose we want alpha to be a root.
Let's start with high rate codes.
I think here in the example, in the text, I used F16.
F16 is going to have -- the elements of F16 are going to be the roots of all the degree 1, 2, and 4 polynomials because M is 4 and these are the degrees that divide 4.
So again, we get x and x plus 1, which have roots 0 and 1.
We get x squared plus x plus 1.
What are the roots of x squared plus x plus 1 and F16?
There are only going to be two of them and they've got to have the property that if we start with the first one, let's call it beta and beta squared.
Beta fourth has got to equal beta because we only have two of them.
That more or less forces beta to be alpha to the fifth or alpha to the tenth.
Let's see if that works.
We get alpha to the fifth, alpha to the tenth.
Square that, we get alpha to the 20th.
But we now take the exponents mod 15 and we get 5 again, all right?
So those are the two roots of x squared plus x plus 1 in F16.
Obviously, I'm not -- I guess that was a legitimate proof, but it was certainly quick.
OK, there are three polynomials of degree 4.
By the way, I think you can imagine how, given this result, we can now enumerate the polynomials of each degree.
We now have an expression.
There are two irreducible polynomials of degree 1.
So how many more of degree 2 is it going to take to make the overall degree 4?
One of them.
So we're looking for a single -- there must be a single irreducible polynomial degree 4.
Similarly over here, all right, we've accounted for four of the elements of F16.
The only other possible degree we could have is 4.
So how many irreducible polynomials are there of degree 4 over in F 2 of x?
Must be three of them.
And it always works out.
So, from this you can get a closed-form combinatorial formula again in terms of the Mobius inversion formula.
By doing a sieve or by some other method, you can find out that the three irreducible polynomials of degree 4 are those three and the corresponding roots for this one -- if this is the irreducible polynomial you use to define the field, then alpha is a root of it and you're going to get alpha, alpha squared, alpha fourth, alpha to the eightth.
Alpha to the eightth squared is alpha to the 16th, but that equals alpha, OK?
So these are the roots of that equation.
Here you get -- sorry, x fourth plus x third plus 1, since that's basically the reverse of this, turns out you get alpha to the minus 1, which is alpha to the 1fourth.
Square that and you get alpha to the minus 2, which is alpha to the 13th, or equivalently, alpha to the 28 -- reduces to thirteen.
This becomes alpha the 11th, alpha to the seventh.
and then alpha to the seventh gives you alpha to the 1fourth again
[INAUDIBLE] same thing about taking the square [INAUDIBLE]?
Correct
--from the [INAUDIBLE]?
Because you go around in a circle.
So it's a finite set.
So you can obviously divide by 2.
So there's one cyclotomic coset.
Here's the second one.
And then the third one must be whatever's left over.
What's left over?
We don't have alpha to the third yet.
Alpha to the third, alpha to the sixth, alpha to the 12th, and then alpha to the 2fourth, which is alpha to the ninth.
And alpha to the 18th is equal alpha to the third again.
OK, what's the moral of this?
We want to start off with -- let's just take a look at the Reed-Solomon, the very first one.
We just want n minus k to be 1 and we just want to have root alpha in here.
Well, if we get root alpha, we're going to get root alpha squared, alpha fourth and alpha eightth along with it.
If we want a binary polynomial with root alpha, it has to this one.
And it's going to have four roots.
All right.
So for root alpha, we're going to get x fourth plus x plus 1 as our generator polynomial.
And it's actually going to have roots alpha and alpha squared.
So it's going to be the same one as you would get if you wanted alpha and alpha squared.
That means n minus k is already up to 2, which means your distance is already up to 3.
Distance is n minus k plus 1.
So this leads to a code with n equals 16 -- is that right?
Yes.
BCH codes are always defined to be one shorter.
We're going to have four redundant bits, so k equals 12 and distance equals 3.
I think that's not quite right.
When we're doing BCH codes, they're always defined just on the non-zero elements of the field.
So we start with the shorter Read-Solomon code of length q minus 1.
15 in this case rather than 16.
And 0 never appears here as one of the possible roots.
So let me just do that.
I apologize I haven't carried through all the possible steps.
So we're getting all of the polynomial multiples of x fourth plus x plus 1 of degree less or equal to 14 rather than 15.
So it's n equals 15. n minus k is going to be 4.
So this is 11.
And the distance is going to be greater than equal to 3.
In fact, it is 3.
So it's a 15, 11, 3 code, which is, in fact, the Hamming code.
Yes?
[INAUDIBLE]
Oh, I'm sorry.
Wasn't this fun?
You're supposed to do that, Ashish
[INAUDIBLE]
OK, well, so you can see how it goes.
So this gets us both roots alpha and alpha squared.
If we want alpha third and alpha fourth, we already have alpha fourth from this polynomial.
So all we need is this one, which gets us another.
And that gives us an n equals 15, k equals 7, d greater than or equal to 5, which, in fact, is a 15, 7, 5 code.
So we get additional codes.
There's a table in there that shows you that most the time for short block lengths for distances which are equal to a prime power, we get the same parameters as Reed-Muller codes.
For lengths 64 and higher, sometimes we do a little bit better.
In addition, we get a lot more codes for every, basically, odd distance when you take the length odd or even distance if you take the length even.
And that's definitely an asset, that you have a larger number of codes to choose from.
So there's no doubt that in terms n, k, d, BCH is the larger and, for longer block length, slightly better class than Reed-Muller codes.
But as I said before, in terms of performance versus complexity, Reed-Muller might still be the ones you want to use.
OK. Good luck in the review and good luck in the exam.
I will see you Wednesday.
OK.
Welcome back.
I hope you had a nice break.
The midterms will be handed out at the end of the class.
Ashish has just gone to get them in my office.
With solutions.
I hope you look at the solutions.
There's more in the solutions than is strictly necessary.
The midterm is partly intended as a learning exercise, and I hope you learned something from it.
I didn't learn too much from it about the performance of the class, because some of the questions were easy enough that practically everybody got them, and some of the questions were too hard for anybody to get.
And so the distribution -- you know, what I would really like to get is a nice bell-shaped curve going from 0 to 100 with a median at 50 and the standard deviation about 25.
And I got a much more compressed curve than that.
So frankly, I didn't learn that much about relative performance.
And I gather from Ashish that you've all been doing the homeworks, and have been doing a fairly steady job on that.
So I think everybody is close to being in the same boat here.
If anybody has concerns about their performance, of course they're welcome to talk to me.
Drop date is a little while off.
If you're concerned about what your grade might be, I'll give you the best idea I can.
But really, it's going to come down to the final for most of you.
So it's not too much to say here.
I give the grade distribution and the midterm solutions, and from that, you could figure out where you are relatively.
And my grading philosophy is at MIT, in graduate classes, roughly they're half As and half Bs.
I have that as a guideline in mind.
I try to look for where the natural breaks are.
I decorate the As and Bs with pluses and minuses, but I don't think anybody pays attention to those.
And in other words, I generally try to follow the MIT grading philosophy as I understand it.
So are there any questions about the midterm or anything about the class as we go into the second half?
No?
I see some people aren't back yet.
It's a somewhat reduced group.
Maybe a couple people decided to drop on the basis of the midterm.
That sometimes happens.
I don't think it necessarily had to happen this year.
Or maybe people are just having an extended break.
OK. We're now going into conceptually a break, the second half of the course.
The first half of the course we were talking about trying to get to capacity, specifically on the additive white Gaussian noise channel.
So far all the schemes that we've talked about have been block coding schemes, which their Euclidean images wind up to be constellations in a finite dimensional space.
This is the kind of picture that Shannon had when he wrote his original paper on capacity and information, to start off information theory.
Basically he proved that you could get to capacity on any channel with a randomly chosen code.
In the setting where we are, that would be a randomly chosen binary code where you just pick all the bits of all the code words at random by flipping a coin.
OK?
Just construct a code book at random, and as the code gets long enough, that's going to be good enough to get to capacity.
It's a block code.
2 to the nr bits, and length n code words where r is the regular code.
Well, that was a brilliant achievement, to show that you could get to capacity by doing that.
If you can do it by choosing codes at random, then there must be some particular codes that are good.
Otherwise the average overall codes wouldn't be good.
So people started to go out to find good codes.
And there was a fairly quick realization that Shannon hadn't addressed some key parts of the problem.
Particularly the complexity of encoding and decoding.
The complexity of encoding was pretty quickly recognized to be not much of an issue, because Peter Elias and others proved that, at least for the kinds of channels we're talking about, which are symmetric, linear codes are just as good.
A random linear code will get to capacity.
In other words, say, a block code with a randomly chosen generator matrix.
You pick k generators of length n, again by flipping a coin, that's good enough to get to capacity.
So we know we can encode linear codes with polynomial complexity.
It's just the matrix multiplication to encode a linear block code.
All right?
So encoding isn't the problem.
But decoding.
What kind of decoding have we talked about so far?
I guess particularly with Reed Solomon codes, we've talked about algebraic methods.
But our generic method is to do maximum likelihood decoding, which basically involves computing a distance or a metric to each of the code words.
Exponential number of code words.
2 to the nr code words.
So we have exponential complexity if we do maximum likelihood or minimum distance decoding.
And that, of course, was the motivation for doing some of these algebraic decoding methods, which were the primary focus coding theory for several decades.
A great deal of very good effort went into finding good classes of codes that were algebraically decodable, like Reed-Solomon codes, BCH codes, Reed-Muller codes.
[UNINTELLIGIBLE] don't have such -- well, they do, we now realize.
But that was the focus.
And with the algebraic coding methods, there was polynomial complexity.
But it began to be realized that these were never going to get you to capacity.
Because of their difficulty in using reliability information, because most of them are bounded distance character, and in fact, that's still true.
You can't get to capacity with algebraic decoding methods.
So in parallel with this work on block codes, there was a smaller stream of work that has broadened, I would say, as the ancestor of the capacity-approaching codes of today, on codes with other kinds of structure.
And specifically dynamical structure, which is what we're going to talk about when we talk about convolutional codes.
And this was broadened into the field of codes on graphs.
And codes on graphs are nowadays the way we get to channel capacity.
All right?
So that's really where we're going in the second half of the course.
We're going to preserve linear structure.
Linear is always good for the symmetric channels we're talking about.
But in addition, we're going to look for other kinds of structure which are, in particular, suitable for lower-complexity decoding methods, are suitable for maximum likelihood or quasi-maximum likelihood decoding methods.
All right?
So that's our motivation.
OK. Convolutional codes.
Convolutional codes were actually invented by Peter Elias in an effort to find -- just as you can put a linear structure on codes without harming their performance, he was trying to find more and more structure but you could put on codes while still being able to achieve capacity with a random ensemble.
And he invented something called sliding parity-check codes, which, if you let the block link go to infinity, become convolutional codes.
And they say there was a trickle of effort basically motivated by the fact that a number of people did a practical comparison of performance versus complexity.
Convolutional codes always beat block codes.
And so, for instance, I was at a small company was trying to apply coding theory.
We quickly gravitated to convolutional codes, because they always gave us a better performance complexity straight off than block codes.
And I'm going to try to indicate why that's so.
All right.
So I think the easiest way to understand convolutional codes is by example, by putting down an encoder.
And the canonical example everybody always uses is this one, so why should I be any different?
Here is a rate 1/2 constraint length 2 convolutional encoder.
And it operates on a stream of bits coming in.
So what we have here is a time index stream of bits, uk.
And you can think of this stream as being infinite in length.
We'll talk about how to make this a little bit more precise as we go along.
And we have a shift register structure here.
The incoming bit uk is D means a memory element or a delay element.
It goes into a flip-flop.
It's delayed for 1 time unit.
There's some discrete time base going on here, which is indexed by k. So k is simply the set of all integers.
Comes from the index set.
Here is u k. Here's u k minus 1.
Here is u k minus 2.
All right?
And constraint length 2 means that we save -- we have the present information bit.
This is called the input or the information bit.
And the two past information bits are saved.
And from that, we generate two output bits.
y1 at time k is given by u k plus u k minus 2, where this is a mod 2 sum.
Everything is in f2.
y2k is the sum of u k plus u k minus 1 plus u k minus 2.
All right?
So this is where we get the redundancy.
What are we generally doing with coding?
We're adding redundant bits in order to get more distance between the sequences that we might send, and thereby hopefully to get a coding gain on the additive white Gaussian noise channel.
Here the redundancy comes from the fact that you get 2 bits out for every bit that you put in.
So there ought to be the possibility of getting some distance between possible code sequences.
The code sequences here are infinite or semi-infinite, but nonetheless, we can establish some minimum distance, or as it's called in this field, the free distance.
OK.
There are two kinds of structure here.
First of all, this is a linear time invariant system.
If we were talking about the real or complex field, we might call this a filter.
Or it's actually a single input, two output filter.
So each of the y sequence is a filtered version of the u sequence.
y2 is a filtered version of the u sequence.
And it's linear basically because it's made up of linear elements, lay elements, and mod 2 adders.
Over the binary field, it's time-invariant, because if I change the time index, it doesn't really change the equation.
It's very hand-waving rough here.
But so it's a linear time-invariant system, and we can analyze it the way we analyze a filter.
It's redundant, so it has more outputs than inputs, which we want in a coding system.
But other than that, we're going to find that the same kind of techniques that we use for analyzing discrete time linear filters can be used to analyze the system.
It's just over a different field, F2, rather than R or C. All right?
So that's one kind of structure.
And secondly, it's a finite state system.
Let's forget about all the algebra.
There's a discrete time system which has two memory elements in it, each capable of storing 1 bit.
So how many states are there in the system?
Four.
There are four possible states that reflect everything that went on in the past.
That's all you need to know about the future, is what the state is at time k, to see what's going to happen from time k onwards.
All right?
So this is a simple four-state system, in this case.
In general, if we build similar encoders out of shift registers, even if we put feedback in them, it's going to be a finite state system.
So we're going to restrict ourselves to finite state realizations, finite state encoders.
And that's going to ultimately be the basis for the decoding algorithm, which is going to be Viterbi algorithm, which is a very efficient decoder for any finite state system observed in memoryless noise, which is what we have here.
Hidden Markov model, as it's sometimes called.
All right.
So we're going to interplay these two types of structure.
And I feel it's really the fact that convolutional codes have these two kinds of structure that makes them better than block codes.
They have just the amount of algebraic structure that we want, namely, the linear structure.
They don't have too much algebraic structure beyond that.
We don't have elaborate roots of polynomials and so forth, as we do with Reed-Solomon codes.
We have a pretty modest algebraic structure here.
And it's sort of different in character than the algebraic structure Reed-Solomon codes.
And it's married to this finite state dynamical structure, which is what allows us to have low-complexity decoding.
So that's kind of the magic of convolutional codes.
Not that they're very magic.
All right.
This is a convolutional encoder.
What is a convolutional code?
The convolutional code is roughly the set of all possible output sequences.
And this is where we're going to measure the minimum distance of the code, minimum free distance.
What's the minimum Hamming distance between any two possible output sequences?
And in the notes, I go into considerable detail to explain how we choose the particular definition of all.
What do we mean?
Do we mean the set of all possible output sequences if we put in a finite input sequence?
An input sequences that starts at time 0, and this polynomial, let's say, that ends at some finite time?
Are we going to allow semi-infinite, bi-infinite input sequences?
So you see, there's some choices to be made here.
And actually, it's of some importance that we make the right choice to get proper insight into these codes
[INAUDIBLE] You have to choose the initial state, then?
Yeah, OK. You know from linear systems that the output isn't even well-defined unless you say what the initial state is.
And in general, in linear systems, we say, well, we'll assume it starts out in the all-zero state.
And that's basically the choice I'm going to make, except I'm going to do it in a couple steps.
But that's the right question.
How do we initialize the system?
That's a part of the specification of this encoder.
And of course we initialize it in the all-zero state.
But for that reason, in a general system, we can't allow bi-infinite inputs.
Because if we don't know what the starting time is, we don't know how to define the starting state.
Again, I'm speaking very roughly.
So let me speak a little bit more precisely.
We're going to define the set of all Laurent.
Which means semi-infinite sequences over the binary field.
And this is called F2 of D. And this means a sequence -- U, say -- which is only a finite number of non-zero terms before time 0.
In other words, it can start at any negative time, but it has to have a definite starting time.
I'm not talking about the set of all power series or the set of all sequences that start at time 0 or later.
The sequence can start at any time.
But it does have to have a definite starting time, and before that, it has to be all 0.
So that for instance, the sequence that -- I haven't introduced D transforms yet.
Maybe I should do that immediately.
But a sequence which, at minus 5, minus 4, minus 3, minus 2, minus 1, 0 -- these are time indices -- looks like 1 0 1 0 0 1 0, and goes on indefinitely in the future, but has all zeros back here -- that is a legitimate Laurent sequence.
Whereas a sequence that's dot dot dot 1 1 1 1 1 1 1 1 1 and so forth, all ones forever, is not, because it doesn't have a definite starting time.
And I sort of -- all right in this notation -- I need to introduce D transforms.
The D transform of a sequence U which has components Uk for k and z it is simply the sum of Uk D to the k. Let me write that as U of D. For all k and z.
This is simply a generating function.
It looks very much like the Z transform that you see in discrete time linear filter theory.
We would write for something up here, well -- So a sequence that is 1 at time 0, 0 at time 1, 1 at time 2 -- if U is equal to that, and 0 at all other times, we would just write U of D equals 1 plus D squared.
So it's a generating function.
The D just gives us a location of where the ones are.
It's easier to write this than this, frankly.
It's actually hard to write the sequence which is 1 at time k and 0 at all other times in this notation, but it's simply D to the k in this notation.
D here is algebraically simply an indeterminate, or a placeholder.
And this is the subtle difference from Z transforms.
The Z transforms, we would write U of Z minus one equals 1 plus Z minus 2 for something like this.
There's this Z to the minus 1 convention in Z transforms.
That's not the important difference.
We could clearly use D to the minus 1 rather than D. The important difference here is that Z is usually imagined to have values in the complex plane.
We look for it's poles and zeros in the complex plane and so forth.
And so the Z transform really is a transform.
It takes you into the frequency domain.
If you make z equals e to the j omega t or something, then you're in the frequency domain.
Here, this is still a time domain expression.
Just as a convenient, compact, generating function notation for a particular sequence.
So that's when I say the set of all semi-infinite sequences, now I can write -- a Laurent sequence always looks like this.
It starts off at some particular time.
Let's say the first term is Ud D to the d plus Ud plus 1 D to the d plus 1 plus so forth.
It always looks like that.
So it has a definite starting time d, which is called the delay.
The delay of u of D is equal to d, in this case.
And d can be any integer.
It can start at a negative time or at a positive time.
We don't require it to start at time 0 or later.
But nonetheless, the sequence has to look like this.
And we indicate the set of all such sequences here.
One of the immediate observations that you can make is that this is a time invariant set.
Which the set of all sequences that start at 0 or later is not.
Set of all sequences that start at 0 or later has a definite time in it, and a set is time-invariant if D times the set is equal to the set.
So what is D times F2 of D?
I claim it's just F2 of D all over again.
And in fact, D to the k for any k is equal to F2 of D. So the set of all Laurent sequences is time-invariant.
Whereas the set of all power series in D, that's sequences that have delay 0 or greater, is not time invariant.
It does not satisfy this.
Chew on that.
An important consequence of this is that the set of all Laurent sequences forms a field.
Let's say what we need to prove for a field.
Can we add Laurent sequences?
Yes, we simply add them in the usual way, component-wise.
Same way as you do in polynomial addition.
You just gather together compounds at time k. Can you multiply Laurent sequences?
You can certainly multiply polynomials.
1 plus D times one plus D is 1 plus D squared over F2.
Can you multiply Laurent sequences?
Well, yes.
We can define multiplication by convolution as we do with polynomials.
In other words, U of D times V of D is the sequence W of D, where Wk is the sum over k prime in Z of Uk prime Vk minus k prime.
Now what's the potential problem that can arise when you define multiplication by convolution?
If any of these terms are infinite, we have no notion of convergence over F2.
So we really have no way of defining an infinite sum.
An infinite sum of elements in F2 is, in general, not well-defined.
But from the fact that each of these starts at one time -- this is basically multiplying something by the time reversal.
We only have a finite number of elements in any such sum.
So that's the real reason for using the set of all Laurent sequences.
If we had the set of all bi-infinite sequences, then multiplication would not be well-defined.
But by insisting that they have a starting time, we ensure that convolution is always well-defined.
Therefore, multiplication of Laurent sequences is well-defined.
OK.
So we have addition, we have multiplication.
Maybe I'll go over here.
What else do we need for a field?
So far we just have a ring.
Inverses, yeah.
Let's directly check inverses.
So suppose I have some Laurent sequence U of D equals U(d), D to the d, plus so forth.
Does that always have an inverse?
In other words, we want to find something 1 over U of D equals what?
Well, let's just divide U of D into 1 by long division, and we can get an inverse.
Let me give you an example.
Suppose U of D is equal to 1 plus D. What's 1 over 1 plus D?
We take 1 plus D, we divide it into 1, we get 1.
1 plus D, D plus D, D plus D squared.
And so forth.
So we get a semi-infinite sequence.
Not a polynomial.
But nonetheless, it's a legitimate Laurent sequence.
It's actually a very simple periodic sequence of period 1.
Are you with me on this?
Could you find the inverse of any sequence that starts at time 0, let's say?
If you can do that, you can certainly find the inverse of any sequence that starts at time D. You just multiply by D to the minus d. So every non-zero Laurent sequence has an inverse.
1 over U of D, let's say.
Which is also a Laurent sequence.
Of course, as always, we can't divide by 0.
But that's the definition of a field.
That doesn't matter.
We have inverses.
We checked the associative, distributive, commutative laws.
They all work.
So this is actually a field.
Yes?
[INAUDIBLE] multiplication [INAUDIBLE]
Yeah.
The Laurent sequences certainly include polynomials, and this is consistent with polynomial multiplication.
So it's just extending polynomial multiplication really as far as we can to these semi-infinite sequences that are semi-infinite in the future, but not in the past.
They're infinite in the future, not in the past
I think that's [INAUDIBLE] infinite sequence of [INAUDIBLE].
[UNINTELLIGIBLE].
I mean suppose you -- To find the inverse we just have to keep solving the set of linear equations, right?
From the first equation, we get the first one, from the second one, by substitution we get the next one
Yeah.
It's long division, which is the same as the Euclidean division algorithm.
We just want to find a coefficient here that reduces the remainder to start at time 1 or later, to have delay 1.
We want to find the next coefficient to make the remainder have delay 2.
And we can always continue this, ad infinitum
[INAUDIBLE]
These are semi-infinite sequences.
Do you mean sequences that start at time 0 or later, which are called formal power series?
Again, there are some --
[INAUDIBLE]
Oh!
Sorry, yes.
Of course, we need to check that.
But you can see, it doesn't matter.
We could divide by 1 plus D plus D squared plus so forth, and the algorithm is the same.
In that case, of course, if we did that, we'd get 1 plus D plus D squared, plus so forth.
D plus D squared, plus so forth.
And what do you know?
D times that is that, and we get 0, and it terminates after 1.
So yeah.
Thank you.
Long division also works.
If we divide by any Laurent sequence, it's the same.
The same algorithm.
Excellent.
OK.
So every Laurent sequence has an inverse.
In general, what do the inverses of polynomial sequences look like?
Inverse of a polynomial sequence.
Can anyone guess what its special property is?
What is a polynomial sequence, first of all?
I say a sequence is finite if it only has a finite number of non-zero terms.
I'm going to say it's polynomial if it's finite and also causal.
Causal means it starts at time 0 or later.
Its delay is non-negative.
All right.
So that's the polynomials that we're already familiar with.
What does the inverse of a polynomial sequence look like?
[INAUDIBLE]
It's always going to be infinite, right?
Unless it's 1 or something.
So unless it's an invertible unit in the polynomials, it's going to be an infinite sequence.
But it's going to have a property.
Nobody can guess what the property's going to be?
I already mentioned it once.
Periodic!
Right.
And this is an if and only if statement.
I guess I have to elaborate a little bit, but roughly, that's correct.
This is going to be one of your homework problems.
Maybe I'm jumping ahead a little, but let me -- there are a couple of ways of seeing this.
One is to use some of the algebra we did back in chapter seven.
But since we didn't really complete that, I won't do that.
A second one is to see within this long division operation, again, if I'm dividing by a polynomial, there's only a finite set of possible remainders so I can get up to shifts by multiples of D. So if I see the same remainder again, if I see 1, D, and then D squared, of course the series is going to have to continue in the same way necessarily.
Since there are only a finite number of remainders, it's got to repeat at some point.
Well, a way that I can do it which is more in the spirit of this.
Suppose I consider the impulse response of a system with feedback, which is set up so the impulse response is equal to, in this case, say, 1 over D plus D squared.
You see that the impulse response of this is going to be 1 over 1 plus D plus D squared.
If I put in a 1, time 1, then next time, I'm going to get a 1 in here, going to get a 1 feeding back there.
Well anyway, I believe that's right.
This gives an impulse response of 1 over -- if it doesn't, then readjust it so it does.
Now, after time 0, the input is always 0.
So I just have an autonomous system which is producing the terms at time 1 and later of 1 over 1 plus D plus D squared.
And again, it's a finite state system.
Because for over F2, there are only 4 states in the system.
So what could its state transition diagram possibly look like?
First of all, the 0 state always goes around to the 0 state.
So the other states are interconnected in some way.
And at best, they're always going to cycle.
So this is a quick proof that 1 over 1 plus D plus D squared -- in fact, any polynomial inverse -- is going to give you a periodic response.
What I should really say is -- let me define a rational Laurent sequence.
I'll leave out the Laurent.
A rational sequence is something of the form n of D over d of D where these are both polynomial, or they're actually finite.
Let's say they're both polynomial.
You can reduce it to lowest terms, if you like.
Actually, I should make this finite up here, so that it can start at time before time 0.
And I have to have d of D not equal to 0.
And in fact, I generally require d0, the first term down here, to be 1.
So this is a polynomial with time 0 term equal to 1.
Like 1 plus D plus D squared.
So a rational sequence is one that looks like that.
We can generate a rational sequence by feeding n of D into a system that has response 1 over d of D. And the output -- I forget where you take the output off here.
Probably here.
This will be n of D over d of D. OK.
There's a linear system.
Single input, single output.
If it has impulse response, 1 over d of D, then if I put in the sequence n of D, I'm going to get out n of D over d of D. And so by the same finite memory argument, n of D over d of D is going to be eventually periodic.
So a sequence is rational if and only if it's eventually periodic.
And again, on the homework, I'm going to ask you to prove this more carefully.
You can use this finite memory argument if you like.
So this should remind you of something.
This should remind you of real numbers, integers, ratios of integers, which are rational real numbers, all that sort of thing.
What are the analogies here?
First of all, how big is it?
How many real numbers are there?
It's uncountably infinite, right?
How many Laurent sequences are there?
You can I think quickly convince yourself that it's uncountably infinite.
Now, we have a special subset of the real numbers called the integers.
Oh.
Let's think of this as a decimal expansion, by the way.
So we have a d, a d minus 1, and so forth, down to a0 decimal point a minus 1.
These are the coefficients of decimal expansion.
There's something interesting here.
Implicitly we always assume there are only a finite number of coefficients in the decimal expansion above the decimal point, right?
There can be an infinite number going this way.
So that's completely analogous to what we have here.
What are the integers?
The integers are the 1's that stop at a0.
That are all zero to the right side of the decimal point.
Over here we have something that looks like u d, D to the d plus u d plus 1, D to the d plus 1, and so forth.
And in here we have a special subclass called the polynomials in D. And what are these?
It's not precisely analogous, because I should really do it with Z minus 1, to make it analogous and expand it in the other direction.
But these are the ones that have only a finite number of coefficients to the right of the time 0 point, again.
We noticed before that there is a very close relationship between the factorization properties of Z and the factorization properties of the polynomials.
They're both principal ideal domains.
They're both unique factorization domains.
They're both rings of the very nicest type.
Then once we have in here the rational numbers -- that's certainly an important subset of the reals -- how many rationals are there?
The rationals, this is basically a ratio of integers.
And there's only a countably infinite number of rationals, right?
And what's the distinguishing characteristic of the rationals in terms of their decimal expansion?
It's eventually periodic.
So these correspond to the rational functions, which are denoted by this regular parentheses bracket.
So this is the polynomials.
This is the rational functions.
And because these are eventually periodic, or because all of these can be written as n of D over d of D, and both of these are polynomials, or these are clearly countably infinite sets, this is a countably infinite set.
The set of all rational Laurentian polynomials.
So again, this is just mnemonics.
Haven't proved anything.
But the behavior of these things very closely reminds you of the behavior of these things over here.
It's good to keep this in mind.
One other point.
Suppose I'd allow bi-infinite sequences.
Then 1 over 1 plus D doesn't have a definite expansion.
It has two expansions.
1 over 1 plus D, if we require that the expansion be Laurent, then we only have this possibility.
But if it could be bi-infinite in the other direction, then you can see that this is also equal to D minus 1 plus D minus 2 plus D minus 3 plus so forth, semi-infinitely.
So another reason we want to rule out non-Laurent sequences is to have a unique inverse.
This is just another way of saying that the Laurent sequences form a field.
That the bi-infinite sequences don't have the multiplicative property.
They're simply a group.
And this is all written up in the notes.
All right.
So where I want to get to eventually here -- come back over here.
Let's now analyze this convolutional encoder.
It's a linear time invariant circuit.
That means it's characterized by its impulse response.
Or responses, if you like, because it has two outputs.
What does y1 of D, if I write the D transform of y1, what is that going to equal to?
If I just put in a single impulse, single 1 at time 0 for u k, what am I going to get out up there?
I'm going to get out 1 0 1 and then all zeros.
So y1 of D, the impulse response to steps d1 of D, equals 1 plus D squared.
That's the impulse response of the first output.
And y1 of D is simply u of D times 1 plus D squared, bi-linearity and time invariance.
So if I put in 1 plus D squared, I'll get out 1 plus D fourth.
Is this too quick?
This is just linear system theory.
You've all seen this in Digital Signal Processing or something.
This is undergraduate stuff, except for over F2.
OK?
You've just got to check that it's linear, it's time-invariant.
Therefore it's characterized by its impulse response.
And so once you know the impulse response, you want to know the response to any sequence, you convolve that sequence with the impulse response.
This is that statement in D transform notation.
OK?
People happy with that?
I will spend time on it if it's mysterious, but I don't think it should be mysterious.
All right.
And let's complete the picture.
What's the impulse response for the second output in that picture?
1 plus D plus D squared, thank you.
So we have y2 of D is equal to u of D times 1 plus D plus D squared.
We can aggregate this by simply saying that a little 2 vector, y of D, is equal to a 1 vector, u of D, times a little 2 vector, g of D, where this means y1 of D, y2 of D, and this means g1 of D, g2 of D. OK?
So this is a very simple little matrix equation.
And now the convolutional code.
This is the output sequence in response to a particular input sequence u of D. So we're going to define the code as the set of all possible output sequences when the input sequences run through what?
Now, with all this elaborate set up -- so it's the set of all output sequences y of D equals u of D g of D as u of D runs through the set of all Laurent sequences.
Sequences that start at some definite time.
And now I've said much more precisely what I said back at the beginning.
Because all sequences have a definite starting time, we know what the starting state is for any of these sequences.
Always when the sequence starts at time D, the system is quiet, it's in the all-zero state, and it's driven from time D onwards.
So this convolution is well-defined.
If we had an undefined starting state, then this would not be well-defined.
OK.
So a long, longer than I attended lecture on Laurent sequences.
But it's turned out, in the theory of convolutional codes, it's very important to get this right.
People have used various definitions.
They've let this run through finite sequences, polynomial sequences, formal power series.
And trust me, this is the right way to do it.
And I don't think anyone is better qualified to make that statement.
So that is the way we define a convolutional code.
Now which direction shall I go from here?
I guess I ought to ask, what are we going to allow g of D to be?
In the picture up there, we have g1 of D and g2 of D polynomial, which means both causal and finite.
Causal means it starts at time 0 or later.
Finite means that it has only a finite number of non-zero terms.
OK.
I want to allow perhaps more general types of encoders.
I do want the encoder to be finite state.
So let me impose that I want the encoder to be finite state.
But I'm going to allow feedback.
Which I don't have there.
If I allow feedback, then I can get infinite responses, infinite impulse responses.
But by the same argument that I've already made, it's going to have to eventually be periodic, right?
If the encoder has finite state.
So we have to have g1 of D and g2 of D are general.
We're going to allow up to n of these for rate 1 over n encoders.
If we want this to be the impulse response of a finite state encoder -- they also have to be causal, don't they.
In order to be realizable, has to start at time 0 or later, the impulse response.
Then I'm going to have to require that these all be rational and causal.
And that's the only limitations I'm going to put on the g of D. So in order to get a finite state system, I need to have rational impulse responses.
In order to have a realizable system, I need causal responses.
So these are the requirements on the impulse responses.
Otherwise I'm not going to force them to be anything in particular.
Now let's see.
I go into realization theory a little bit in the notes.
Since I've spent so much time on Laurent sequences, I don't think I'm going to do that here.
Yeah?
Does this mean that the denominator is also polynomial?
Correct.
So in fact, let me always write these.
Let me take the least common multiple of all the denominators.
I'm going to write these then.
g of D is going to be some vector of polynomials up here.
I can always write the Least Common Multiple of all the denominators down here and have a single denominator.
OK?
So this is going to be my general form for a causal, rational, single input and output impulse response.
It's going to consist of something like this, where these are all polynomial.
This is a single polynomial with d0 equals 1.
Turns out that's necessary for realizability.
And I'll just tell you the fact, which you can read about in the notes.
In fact, I will leave this mostly as an exercise for the student, anyway.
Realizable with nu equals max of the degree of the n_i of D, or the degree of the denominator.
In other words, I take all these polynomials -- the maximum degree of any of them -- and I call that nu.
And I claim there's a realization with nu memory elements.
And of course, I'm going to assume here that I've reduced this to lowest terms in order to keep that down.
If I didn't have the denominator term, this would be clear.
If I just had g of D equal to a set of a vector of n polynomials of maximum degree nu, then it's clear that just by this kind of a shift register realization of here we call the constraint length equal to the number of memory elements, I can realize any n impulse responses which all have degree nu or less, right?
So the only trick is showing how to put in a feedback loop which basically implements this denominator polynomial d of D. If I start off by realizing 1 over d of D, then I can basically just realize this in the same way.
And nu is called the constraint length.
And I have 2 to the nu states.
So by constraining this to be of this form, I've now gone full circle.
The number of states is in fact 2 to the nu, and in particular, it's finite.
When I constrain the impulses responses to be like that, then I guarantee that I'm going to have a finite state realization.
So from now on, that's what my impulse response is going to look like.
All right.
Now let's talk about code equivalence, or encoder equivalence.
I've defined the code generated -- now I'm going to characterize an encoder by its impulse responses, g of D. The code generated by g of D is u of D g of D as u of D goes through the set of all Laurent sequences.
Two encoders are equivalent if they generate the same code.
Seems reasonable.
What we're really ultimately interested in in communications is the behavior of the code.
In particular, the minimum distance of the code, how far the sequence is separated.
We're not particularly interested in the encoder.
At the decoder, we're just simply going to try to tell which code sequence was sent.
So for most purposes, probability of decoding error, performance of the decoder and so forth, we're only interested in the code itself.
We're not interested in this little one-to-one map between information bits and the code sequences.
So this is a reasonable definition of encoder equivalence.
Now, for the case of rate 1/n codes, which is all I'm talking about here -- 1 input, n output, so the code generator looks like this -- it's very simple.
g of D and g prime of D are equivalent if and only if g of D and g prime of D differ by some multiple a of D, where I could let a of D be any Laurent sequence.
But then to keep everything in the same ballpark, I think I'd have to keep a of D rational and causal.
But even forgetting this -- in other words, if one is a multiple of the other -- then the two codes are equivalent.
That's clear because I can invert a of D. So this is just a single Laurent sequence, has an inverse.
This is the same thing as saying g of D times 1 over a of D -- which is another rational, causal sequence, is g prime of D. If this is true, then the sequence generated by u of D when the encoder is g of D is the same as the sequence generated by u of D a of D when the encoder is g prime of D. And this is another rational sequence.
And vice versa.
the code sequence generated by u of D, the encoder is g prime of D, is the sequence generated by u of D over a of D if the encoder is g of D. OK?
So that's really the proof.
So this is very, very simple theorem.
In other words, I can multiply any encoder n-tuple g of D by any rational causal sequence out front, in particular by a polynomial or 1 over a polynomial or something, and I'm going to generate the same code.
This is not going to matter.
Now, to see why we might want to take make use of this -- let's see.
Well, given this, we might want to pick some particularly nice encoder from this equivalence class of encoders, all of which generate the same code.
And basically, I'm going to suggest that the nicest encoder is the one we get if we multiply by d of D. And if there's any common factor of these n of D's, we divide it out.
That will give us the least degree polynomial encoder, which is equivalent to the given encoder.
If I start off with one like this, and I want to multiply through by the denominator to make it a polynomial, and if there's any common factor to the numerator, I want to divide it out.
And that's what I'm going to take as my canonical encoder.
Now let me give you a little motivation for that
[INAUDIBLE]
That's one motivation.
It will have the simplest realization.
It will have a feedback-free realization by getting rid of the denominator term.
And it will have the least constraint length of all equivalent encoders, and that's a theorem too.
But that's not the primary motivation.
Let me talk a little bit about distance at this point.
Let me take my example encoder again.
And let me ask, what's the known distance between code sequences?
This is clearly going to be an important performance metric of this code, right?
How would you go about answering that question, or at least two reasonable ways to go?
Do you think the known distance is greater than 1?
You think it's 1?
OK.
So how would you get two sequences that differ only in one place?
I mean, if you take the impulse response.
Or you take the --
Well, the impulse response, if I put in 1, u of D equals 1, then I get out g of D, which in this case is 1 plus D squared, 1 plus D plus D squared, I actually get a weight 5 sequence out.
You see that?
[INAUDIBLE] you get 1 and 1 0 1, and [INAUDIBLE] 1 and 1 and 1 --
So I get this out, in D transform notation.
Or I get 1 0 1 1 1 1 in terms of that times 0 1 1 and so forth.
y1 equals this, y2 equals that.
Right?
OK.
So that's a weight 5 sequence.
So if I put in the all-zero sequence, I get out the all-zero sequence.
So as always, the all-zero sequence is an element of the code.
This differs from the all-zero sequence in five places.
So all you've shown me so far is the minimum distance is not greater than five.
I've found two sequences that differ in five places.
Are there any that differ in fewer places than that?
The answer is no.
So in fact, it's called the free distance.
For this code, the so-called free distance is five.
Let's think of the two different kinds of structure for this code.
And they lead to different but complementary arguments that both get us to this conclusion.
First of all, the code is linear, right?
So C is linear.
Is it not?
If I have y of D equals u of D g of D, and y prime of D equal to, say, some other sequence.
u prime of D times g of D. Then let me check the group property, which is all I have to check for a binary code.
And I find that y of D the sequence, the sum of these two sequences, is the sequence that's generated by the sum of the two input sequences that led to these two sequences.
So the binary sum of any two sequences is another code word.
All right?
Ergo it has the group property.
Ergo C is a group.
And that's all we need to check.
It's actually linear over F2, as a vector space of F2.
0 is in the code, and 1 times any code word in the code.
So we've checked closure under vector addition and scalar multiplication.
So it's linear.
What is the main conclusion we get from this?
Therefore, the minimum distance between code sequences is equal to -- anybody?
The minimum non-zero weight of any y of D in the code.
Exactly the same argument.
We're talking infinite sequences here, but there's nothing that changes in the argument.
And so that really simplifies things.
We simply have to ask -- here's a non-zero sequence of Hamming weight 5.
You can add this to any code sequence and get another legitimate code sequence.
This is the code sequence.
Add it to any other, and you get a legitimate code sequence.
So from any other code sequence, there's going to be one of distance 5.
Are there any lower weight sequences in this code?
Well, what would you --
[INAUDIBLE]
Good.
So here's where we need to get into much more concrete arguments.
You can see that first of all, we're only interested in looking at finite sequences in the code.
And we might as well have them start at time 0, because if they start later, we could just shift them over by time and variance.
So we're only interested in finite polynomial sequences that start at time 0.
OK.
So just as you said, in the first place, they're always going to have bing bing!
Two 1's.
When the first 1 comes in, we're going to get two 1's out.
For sure.
Time 2.
This is what we get out if the second bit in, in the input sequence, is a 0.
What happens if the second bit in is a 1?
Then we add this to this, shift it over 1.
We have 1 1 in particular at this place.
We get a 1 0 out.
OK?
So we conclude at the second time, we're always going to get at least another unit of distance.
Now jump ahead.
However long this is, this is going to have to end at some time.
At the time when the last bit, the last non-zero bit, is shifting out of the shift register, we're also going to get 1 1 out.
So any finite code word has to end with a 1 1 at the last time out.
People don't seem to be totally comfortable with this, so I'll do it in a more elaborate way in just a second.
But this is the basis of the argument
[INAUDIBLE] this?
At the end?
No, I mean by [UNINTELLIGIBLE] at the same time
At the end of a finite code word?
Yes
All right.
What does a finite code word consist of?
I'm going to claim that we only get a finite code word out when we put a finite input sequence in.
OK?
There's no feedback here.
So there's going to be some last 1 in the input sequence as that shifts through here.
At the very last time, the last state, is going to be 0 1.
All right?
Just before that comes out, there's a 0 coming in, forever after.
So the last time we get a 1 at u k minus 2, and that forces these two bits out to be 1.
Or you can do it by polynomials.
You can always show that you multiply this by any finite polynomial, you're going to get highest degree terms, both equal to 1.
OK.
I'll do this by explicitly drawing out the state diagram.
So we can conclude here that we always have 2 here, 1 here, dot dot dot dot, and finally at the end, we have to have at least 2.
Therefore, every nonzero finite sequence has to have weight at least 5.
Since we've seen one of that has weight 5, that is the minimum distance.
OK?
Now let's do this by drawing the state diagram.
Which is where we're going
So I know that [INAUDIBLE]
No.
We had to construct a little argument here, and in effect, make a little search.
And this simple argument wouldn't work for more complicated cases.
So let me show you how to attack more complicated cases.
We want to draw a finite state machine.
How do you analyze finite state machines?
Well, a good way to start is to draw the state transition diagram.
And I don't know what the curriculum consists of nowadays, but I assume you've all done this.
We draw the four possible states.
0 0.
1 0, we can get to from 0 0.
0 0, if I get an input of 0, then I'm going to put out 0 0, then I'm going to stay in the 0 state.
All right?
Because this is linear, if the 0 comes in, that's 0 0, stay in the 0 state.
Or a 1 could come in.
And in that case, as we've noted, we'll put out two 1's as our output, and we'll transition to the state 1 0.
Now from 1 0, where can we go?
For 1 0, if a 0 comes in, then we'll put out 0 1 and we'll transition to 0 1.
Or we'll get another 1 in.
We would put out the complement of that, 1 0.
So we get another 1 1 in this linear thing.
Work it out.
I'll just assert that.
And we go to state 1 1.
From 0 1, if we get a 0 in, we return to the 0 0 state and we put out 1 1.
So here's our basic impulse response, right down here.
We put in a 1, we go through this little cycle, we come back to 0 0.
Or if we get a 1 in, we go back up here, and at that point, we actually only put out two 0's.
Is that right?
Yeah, because it has to be the complement of this.
And if we're in the 1 1 state and another 1 comes in, we stay in the 1 1 state.
And we put out what?
Let me do the 0 1 first.
0, and we're in 1 1 state, then what do we get out?
We get out a 0 down here and we get out a 1 up there.
We got 1 0.
Or if we get in a 1, we put out 0 1.
All right.
So that's the state transition diagram.
Now again, let me make the argument that I made before, now a little bit more concisely, because we're going to have to stop.
So every path through this state transition diagram corresponds to a code sequence, right?
Every finite or semi-infinite path always start in the 0 state.
For a finite code word, I have to come back to the 0 state.
Correct?
So I'm always going to get two 1's when I start out.
Then the next time, I'm either going to go on this transition or this transition, but either way, I'm going to get at least another unit of weight.
And then I can do whatever I want through here for a while, but eventually I'm going to have to come back out here.
And when I get back to the 0 state, I'm going to get two more 1's.
So that's maybe a lot easier way to say the minimum weight has to be 5.
Again, if it were more complicated, I'd have to make more of an argument.
All right.
Next time, when we come back, we'll talk about turning this into a trellis diagram.
We'll talk about catastrophicity, which is where I was going with this canonical generator.
It's just a minor algebraic point, but one you need to know about.
We'll talk about the Viterbi algorithm.
I think we can probably get through this chapter on Wednesday.
OK?
Ashish has your midterms and the midterm solutions.
--we'll start off with a little review, so you might chance it.
We're in the middle of the chapter nine on convolutional codes.
Today, I intend to hand out problem set six.
I believe Ashish will have it when he gets here.
And we'll hand it out at the end.
Last time, I introduced you to a lot of things about convolutional codes and their encoders.
I stressed that a convolutional encoder had two distinct characters.
One is it's a linear time-invariant system.
So we can use the linearity and time-invariance and algebraic analysis of convolutional codes.
Secondly, I stressed that's it's a finite state system, because it's a finite memory system where each memory element has a finite number of possible values, over F2 or indeed over any finite field.
And therefore, we can use that for different kinds of analysis.
We use that, for instance, in conjunction with linear property to find the minimum distance to a particular code.
And we'll see that that's the key to getting an efficient optimal maximum likelihood decoder -- is the fact that we only have a finite number of states in this system.
All right, so along the way I kind of went all over the map, as far as the introducing various algebraic quantities.
I focused particularly on formal Laurent series.
And I want to just give you a little table to make sure you have fixed in your mind what all these different kinds of sequences we're talking about are.
On the left side, I have Laurent sequences.
These are semi-infinite sequences that must start at some time.
Their delay is not equal to minus infinity, which would be if it were all 1's forever.
But their delay is some finite time.
It could be negative, could be positive.
That's their starting time.
And then they could go on, possibly, infinitely into the future.
And then, among those, we can look particularly at the ones that start at time 0 or later, whose delay is 0, or whose delay is non-negative.
These are called the formal power series, and you've probably encountered them earlier.
They're a lot more frequently encountered in mathematics algebra.
The rational sequences are simply those that can be written in a very finite way as a numerator polynomial over a denominator polynomial.
Because the formal Laurent series form a field, every element is invertible, and, in particular, every polynomial has an inverse, which, unless the polynomial is a monomial, is going to run on for an infinite length of time.
But nonetheless, there is a subset, a countable subset of these sequences that we can write in this way.
And we call these the rational sequences, and they're analogous to the rational numbers in the real field.
And their particular characteristic, as we showed, is that they're eventually periodic.
And it's easy to see that these, too, form a field.
Obviously, the inverse of N of d over D of d is D of d over N of d. Provided that n of d is not equal to 0, the denominator is always required to be non-zero for a rational sequence.
So that's a very nice subset.
It may or may not have operational meaning.
Certainly, when we're talking about realizations, it has meaning.
And then the finite sequences, by eventually periodic, I will from now on include the finite case.
If we have a finite sequence, then its end is all 0's, and that's certainly a periodic sequence.
So I would say a finite sequence is also eventually periodic.
But among the rational sequences we have, particularly the finite sequence, these do not form a field, because the non-monomial polynomials cannot be inverted.
So these are merely a ring.
However, D has an inverse.
Over here on this side, D minus 1 has an element of all these things.
D has an inverse.
Over here, on the causal side, these are the causal analogs to all of these.
In the case of causal rational, we don't have particularly good notation.
We just say these are the ones that are both causal, formal power series and rational.
But they are important, because we've seen that it's this kind of sequence that you can get as the impulse response of a time-invariant linear system that is realizable.
Realizable has two parts.
One is the impulse response must start at time 0 or later, by physical causality.
Two, the impulse response must be eventually periodic, therefore rational, if it's going to be realizable with a finite number of memory elements.
So we include that in our definition of realizability.
So these are sometimes called the realizable sequences, or the realizable D transforms.
And these, of course, the causal finite sequences are the polynomials.
These include the polynomials, which are trivially easy to realize with shift registers, as we saw in our example.
So that's just a review of all these quantities.
Does anyone feel the need for further elaboration or explanation?
Or can we go forward with this?
We tend to use all of them as we go along, so I want you to have them clearly in mind.
Yeah
Why was the [INAUDIBLE] uncountable?
What was the [INAUDIBLE]?
Why is that uncountable?
Try to count it.
I could put it in one-to-one correspondence with a set of all binary expansions of the real numbers, and that's an uncountable set.
Some of these side comments are just to trigger analogies in your mind, maybe make it more reasonable, what we're talking about.
OK, so next.
Just again, to continue a quick review, we talked about realizable linear time-invariant systems.
First, we talked about single input, single output.
I'm imposing an order that didn't necessarily exist.
In any case, these are characterized by impulse response, g of D, which, for realizability, that implies that g of D is causal rational.
The fact that it's rational means that we can write g of D as n of D over d of D. And the fact that it's causal, well, we would always reduce this to lowest terms in the first place, so there'd be no point carrying along common factors in the numerator and denominator.
By inserting enough powers of D, we can make sure that both of these are polynomials.
So if n of D, d of D are -- we can ensure that they're actually polynomials, not just finite sequences.
And if it's going to be causal, that means that we can always -- if we remove common factors of D, that means that the numerator might still have factors of D in it, but the denominator can't have factors of D in it.
You see that?
Because, once we shift all these over as close to 0 as we can, the denominator has to be closer to 0.
If D0 equals 1, that means 1 over d of D is something that starts at times 0.
We multiply times the numerator.
In order to have the whole thing be causal, the numerator has to be causal, because a point I didn't mention, when you multiply formal Laurent sequences, their delays add.
If a of D and b of D are Laurent, then the delay of a of D times b of D, defined as a convolution, is going to be the sum of the delays of each of the component sequence.
You just take the lower order term on both and that's what determines the lower order term on the convolution, just like in polynomials and analogous to degrees.
So from this, we find that we can always write a causal rational sequence in this form, if it isn't 0.
If it is 0, we just take n of D to be 0, and d of D to be 1.
And that satisfies this form, too.
I guess I didn't even have to make that comment.
We're already in that form.
You can convince yourself this is a unique way to write a causal rational sequence.
And every other way is just multiplying the numerator and the denominator by some common polynomial or finite sequence.
So now we had a little theorem that says that, if g of D is causal rational, and therefore equal to n of D over d of D in this form, then realizable with how many memory elements?
Does anyone remember?
nu equals the maximum of the degrees.
And I debated whether to actually do this in class, and I think it's worth taking a few minutes to actually do this in class.
It'll also appear as the first homework problem.
So this'll be a little head start on the homework.
So anybody have any ideas how we can do this realization?
What we want is a circuit with an input, u of D. Eventually, an output, y of D equals u of D times n of D over d of D. For these are both polynomials and D0 equals 1.
A lot of you have probably seen this in discrete-time linear filters.
It's going to be just the same technique.
But if we don't have volunteers, I don't want to waste time.
The key is to use -- we're of course going to need feedback in general.
To have a d of D, in general, will not be 1.
If it's not 1, then we're going to get an infinite impulse response.
To get an infinite impulse response, we need feedback.
So we're going to need some feedback term in here.
And here is the motivating idea of the construction.
We want to create something here, v of D, which is basically equal to u of D over d of D. If we can do that, we'll be done.
I'll show you why.
So we created a different sequence, which is u of D divided by d of D. And then we pass that sequence through a shift register.
And what do we get at the various stages of the shift register?
Here we would get v of D delayed by one time unit, v of D delayed by two time units, and so forth, up to -- let's make this nu.
I want to realize it in nu memory elements, where nu is the maximum degree of either of these defining polynomials.
So finally, out here, I get d nu of D, v of D. Now, what's the trick?
[INAUDIBLE]
Excuse me?
Because [UNINTELLIGIBLE PHRASE]
That is the idea.
What do we get if we take out these various lines, which are v of D, through d to the nu v of D, and we make a linear combination of them?
We're going to do this twice, actually, once to form the feedback, and once to form the output.
For the feedback, let me not take this into the linear combination.
Otherwise, I'd get a loop without a delay element in that, and that tends not to be well-defined.
So I want to take a linear combination.
In general, by taking a linear combination of these things, I can get the multiple of v of D times any polynomial degree, nu or less, just by taking the coefficients of that polynomial as my linear combination.
Do you see that?
I'm not seeing people's --
[INAUDIBLE]
Excuse me?
In the linear combination [UNINTELLIGIBLE PHRASE] you also take the one with zero derivative
You're getting ahead of me, which is fine, but I'll ask for that comment in just a second.
Let me first create the output.
Suppose I had v of D equals u of D over d of D. Then to get the output, I simply want a linear combination, which will give me n of D times u of D over d of D. And that would be the correct output.
And since n of D is a polynomial degree less than nu, I can do that by simply taking n nu times this, n nu minus 1 times that, n2 times that, n1 times that, and 0 times that.
And that will give me what I want as an output, if I'm successful in getting v of D of this form here.
So let me try a similar trick up here.
The trick is, I force d of D to start off with a 1, to have D0 equal 1.
So what I'm going to create up here is the linear combination, d of D minus 1 times v of D. First of all, let's verify that I can do that.
d of D minus 1 is what?
It's a polynomial.
Its degree is the same as the degree of d of D. Now, if d of D is equal to 1, if the denominator is just 1, this whole thing falls out.
I don't need anything coming in here.
d of D is 1. v of D is equal to u of D, so that's the polynomial case where I don't need feedback.
So let's assume d of D is not equal 1, therefore it's some polynomial of degree nu, degree of d of D, which is less than or equal to nu, and so is this.
But by subtracting out the 1, I have no constant term.
So it's going to be a multiple of D. That's another way of saying that.
It's going to have no constant terms, so I only need to form a linear combination of these terms.
So I can do it.
Now, if I do that, let's just solve this equation.
I create x of D. This is supposed to be a plus.
This is supposed to be a minus, to work over any field.
This trick works over real or complex, or what you'd like.
x of D is u of d minus d of D minus 1, times v of d, times x of D. Maybe I should just call this v of d. Now I'm going to solve for v of D. So I actually have a v of D on both sides of the equation.
This results in u of D equal v of D times d of D. And dividing both sides by d of D, I see that I succeeded in getting what I wanted.
So this is a typical trick of realizing a rational impulse response by including a feedback loop and negative feedback in your system.
This is done in the notes.
I'm surprised if you haven't seen this before.
Maybe it's that people in communications don't take circuits courses, or they don't take digital signal processing courses.
Most people look puzzled.
In any case, can you agree that, formally, I've shown that we get what we want here?
So I've given a way of realizing, given a causal rational function, where nu is defined is the maximum degree of these two polynomials.
There's a realization with nu memory elements.
A little bit more work, we could prove that this is the minimum possible number of memory elements for this system.
That gets into linear system theory
[INAUDIBLE]
I mean, what I want is down here, I want the sum of n_i times d to the i, v of D. So the linear coefficients are going to be these n_i.
If I drew it out, I have n0 here, n1 here.
So it's scalar linear combination, over F2.
And what is that?
v of D is common.
That's just n of D times v of D. Same trick up here.
Is that what was puzzling everybody?
Think about it.
What is a linear combination?
Something that looks like that.
The second part, let's now talk about a convolutional encoder.
And we're just going to talk about rate 1/n in this course.
So this is a single input.
n output linear time invariant system.
That's my definition, I guess, of what I mean by a convolutional encoder.
And when I say linear time-invariant, I also want it to be a realizable, in the two senses that its impulse response is causal rational.
So [INAUDIBLE] down.
Now it's characterized by n-tuple of impulse responses.
So it's going to be sum g of D equal to g1 of D, up to gn of D, where clearly all these have to be causal and rational in order for it to be realizable.
If any one of them was not causal or not rational, then that individual impulse response wouldn't be realizable and the whole thing wouldn't be realizable.
So where we have the gj of D are all causal rational.
And now, again, I'm going to put this into a standard form.
In general, this is going to be n1 of D over d1 of D, so forth, up to n_n of D over d_n of D. So I'm going to have different denominators for each of the numerators.
But I can always put it into the form, n1 prime of D over a common numerator, d to D, and up to n_n prime of D over d of D. Let d of D be the least common multiple of all these d_i's.
These are all polynomials.
Least common multiple is well-defined from our discussion of factorization.
So I can always put in the least common multiple here, and then whatever d1 lacks out of the least common multiple, I multiply top and bottom by both of that.
I have the same rational function, now with a common denominator.
So I'm always going to put it in that standard form.
And then I will say this is always realizable, with nu equal now the max of any of the degrees that appears in here.
So let me just abbreviate that by degree of the numerators, which I'll just write as vector n prime of D, or the degree of the denominator d, of the common denominator d of D. Now, that's very easy to see.
Why is that?
Can certainly realize the first one here just by doing that.
Yes?
If I want to just generate the first output, I build a circuit like this, and I don't need more of the new memory elements.
There might even be some redundancy in that.
All right, so how would I then realize the second?
Is it just me?
Nobody is volunteering anything.
I've realized this.
Now I'd like to realize a second output.
Now I want to realize this.
Make this n1 of D. I want to realize n2 of D over d of D, times u of
[INAUDIBLE]
Thank you so much
Just take out [UNINTELLIGIBLE]
All right.
So all I need is the second linear combination.
So let me just make this n linear combinations.
Then we're going to have n outputs, an n-tuple of outputs.
Then I can form each one as a linear combination of those up there.
I really like more of you to be speaking up.
Yes, thank you
[INAUDIBLE]
Excuse me?
There's a second combination -- I mean the inquiry should come [UNINTELLIGIBLE]
Yeah, but again, what I want to realize, I have all the delays of v of D up here.
What I want to realize is n2 of D times v of D, which is equal to that.
n2 of D is the polynomial of degree less than or equal to nu.
So I can do it
Yeah, but [UNINTELLIGIBLE PHRASE] parallel to that n1 D over d. The -- [INTERPOSING VOICES]
Yeah, OK.
I'm confusing you because I put all these here.
I actually wrote out the linear combination.
To say what I just said, I really need to go back to my original form, forget these multipliers, do the multiplications in here.
That's what I mean by the linear combinations.
Now I'm OK.
The inputs are just the shifts.
Make a linear combination of them, that's the output.
So easy proof.
What's significantly harder to prove in this case is you can't do any better than.
There aren't any realizations with fewer than nu, where nu is computed by first reducing to standard form and then evaluating this.
That's the best you can do.
But we aren't going to go into minimal system realizations, linear system realizations in this course.
I'll just insert it.
So this is how we can always build a convolutional encoder, whether it has feedback or not.
And so whatever we come up with as nu, this'll basically determine the state complexity of our decoder.
The state space is dimension nu, is finite dimension.
And because we're over a finite field, the actual number of states is only 2 to the nu.
Still can't get more than 2 to the nu states for that system up there.
So we're still in finite state world.
So in particular, it's finite state realization.
And again, this is if and only if.
If we have if and only if, we have an n-tuple of causal rational, or even a matrix of causal rationals.
We can make a realization like this.
So that's convolutional encoders.
At least write 1/n over F2.
And now the next step was what's a convolutional code.
And a convolutional code, we defined as, given a convolutional encoder, which is just the set of impulse response, given g of D, the corresponding convolutional code is just u of D, which is single sequence times this n-tuple of sequences, as u of D ranges through all the formal Laurent sequences.
Sequences that start at some time in the all-zero state and then continue perhaps forever.
It's very quick to show that C, its properties is it's linear.
Obviously, it's a vector space over F2.
Multiplication by scalars is trivial.
Addition is trivial, so it's linear.
And its time-invariant, meaning the shift of any code sequence is another code sequence.
If I have one particular code sequence, and I want to see if the shift of that is in the code, well, that code sequence must have been generated by some use, so if I just shift the u by the amount of time that I want to shift the output, y, then I'll get the shifted output.
So it's easy to show that D to the k of C is simply equal to C for any integer k. Actually, it just suffices to show that the single time unit shift is in the code, and that implies all the rest of this.
Yes?
[UNINTELLIGIBLE].
We would want at least one of the g_i of D's to start at 0.
Otherwise, they don't cancel out, do they?
Right.
So what you're saying is we don't want d to be a common factor of all the n of D's.
And that's true.
In fact, we don't want to have any common factors of all the n of D's
[INAUDIBLE]
The u we've defined, so it can start at any time
[INAUDIBLE]
So notice that these have separate algebraic characters.
This is a Laurent sequence.
These are called causal rational sequences.
So they have different restrictions on them, but they play together that way.
So where you're getting to is this idea of code equivalence.
I can just keep going.
So code equivalence, let's abbreviate it this way.
g of D and some other n-tuple, g prime of D, generate the same code.
We say that g of D generates C up here.
So two different n-tuples generate the same code, C, which is our notion of equivalence.
And we more or less proved last time it is true that g of D is some -- this is just a single sequence multiple of g prime of D. So we have to have a of D not equal to 0.
Otherwise, I think there could be any sequence there.
So what you were saying is that, if all of these n's had a common factor of d, then why not just shift them all over?
And that would be the same as multiplying -- suppose we had a g prime of D where they all had d's.
Suppose we just multiply them with d minus 1.
We're still going to get the same code, but we're going to reduce the degrees of all the numerators.
And therefore, we're likely to get a simpler realization.
And that's the track we're going down right now
[INAUDIBLE]
This would only shift the n's.
I'm assuming that all the n of D's are multiples of d
[INAUDIBLE] the root of d, right?
So if you can reduce the d of D you can't save anything
You might not save anything if you've got a denominator, d of D, that has a larger degree
And that's always the case
No, it's not always the case.
In the general case, either one of these things can dominate.
In that picture we had, if we have a low degree, d of D, then it's only picking off of these first elements up here
But that definition states [UNINTELLIGIBLE] is the maximum d and also the [UNINTELLIGIBLE]
Right.
So we could have a big one here making the outputs, and a small one there going back.
In fact, we could have d of D equal to 1.
Then we'd have no feedback whatsoever.
So it could be either way.
But then given this code equivalence concept, suppose we have a d of D that is very big.
Suppose it's dominated by d of D. What would be a good thing to do?
We have a certain code we want to keep, but the nu, the numbers for the dimension of the state space is dominated by the degree of d of D. So suppose we have g of D equal to n of D over d of D. And we have degree of d of D is greater than degree of n of D. What would be a good thing to do?
Why don't we just let g prime of D be equal to d of D times g of D, and that would be just n of D. So I'm going to convert from the code generated by these rational generators, which are infinite.
I can easily just multiply out the denominator, and now I have a code generated just by the numerator terms.
So it's feedback-free.
That may or may not be important to me.
Actually, it turns out it's very hard to make a case for not having feedback, but it's simpler, in any case.
We don't have this feedback term in the encoders.
So we get a feedback-free encoder.
And if this is true, we may reduce -- we can certainly never increase it.
But if this were true, then we would reduce it.
If on the other hand, this were less than or equal, then nu would remain the same.
So that seems like that would be a good thing to do.
So in fact, as I would advocate as a first step, that you clear the denominators, and just come up with a polynomial generator sequence which generates the same code.
We're still going to have the same minimum distance, the same code sequence, the problem from the decoder's point of view is going to be absolutely unchanged.
The decoder only cares what the consequences are.
It wants to find the most likely one that was transmitted.
So let's not make the encoder any more complicated than we have to
[UNINTELLIGIBLE PHRASE]
nu doesn't change in this condition.
nu does change in this condition
So we are always better off multiplying by g of
We can't be worse off.
[INTERPOSING VOICES]
--or the same
Exactly.
Got it.
All right, so let's now talk about whether there's anything we can do to n of D. We already suggested that if all the n of D had a factor of d in them, why don't just take it out?
Let's just write that down.
If all n of D have a factor of d, then let's just transform n of D to d minus 1 n of D. So that reduces nu also.
And let's, of course, do that as many times as we can.
So it's a good idea to take out common factors of d. We get a simpler encoder.
We're have a simpler state diagram.
This leads us to a topic called catastrophicity, which is a misleading title because you have no idea where I'm going right now.
So I'm actually going to try to fool you for a little while.
Yes?
Quick question.
In the definition of code equivalence, is there a restriction [UNINTELLIGIBLE] that shouldn't be rational, shouldn't be causal?
Should there be a restriction on this?
Yes, of course.
It has to be rational in order -- But that's a consequence.
If these are both causal rational, then what does a of D have to be?
It has to be rational.
It has to be non-zero.
And does it have to be causal?
No, in this case it's not causal
[INAUDIBLE]
That's right.
If there's a delay buried in g of D, if everything starts at time 1 or later, then a of D, in fact, can be non-causal.
So the key restrictions are that these two guys have to be causal rational.
And that will tell you restrictions on a.
Let me introduce this by example.
And I'll warn you that I'm going to try to fool you for a while.
Let's just take a rate 1/1 encoder.
This may seem a little peculiar to you, because it has no redundancy, one input, one output.
And I'm going to let g of D just be 1 plus d. So what do I mean?
I'm going to try to get some mileage out of this encoder.
Here's u of D. Here's y of D, equals 1 plus d times u of D. Now let's draw the state diagram for this encoder.
It's only got two states.
One is 0.
If we get a 0 in, we of course put a 0 out.
We get 0, 0.
Or we could get a 1 in.
In which case, we would get a 1 out.
Sorry, that was just a single output.
And go to 1.
If we're in the 1 state, and we get a 0 in, then we get a 1 out.
Whereas if we get a 1 in, then they're both 0, and we get the complement.
So with the 1, we get a 0 out.
So that's the state transition diagram of this encoder.
Now, let's go through a similar kind of argument to what we went through to find the minimum distance was 5.
This is a linear system, so we want to find the minimum non-zero weight of any code word.
And here's my argument.
There's a gold star to the first person who punches a hole in it.
Any code word, you have to start in the 0 state, so you're always going to get at least one unit of distance when you go to the 1 state.
You can stay out here as long as you like, but eventually you have to come back to the 0 state.
And at that point, you're going to get another unit of distance.
So I claim the minimum non-zero weight is 2.
Is that correct?
Let me try to create some doubt.
What is the code here?
The code is the set of all multiples of 1 plus d times u of D, where u of D is a formal Laurent sequence.
What is that?
I can write this briefly as just the set of all formal Laurent sequences times 1 plus d
[INAUDIBLE] 1 minus d minus d squared
Good.
That's correct.
That's what's wrong with my claim.
Let me detour around here.
Let me get an answer to this question.
What is the code in this case?
It's the set of all possible sequences you can get by multiplying 1 plus d times the formal Laurent sequence.
What is that?
[INAUDIBLE]
What?
Isn't it [UNINTELLIGIBLE]?
It's just the set of all formal Laurent sequences.
Certainly for every formal Laurent sequence, I can get such a sequence this way.
In particular, the code includes 1.
What do I need as an input to get the output sequence 1?
String of 1's
String of 1's, right.
So an example, if u of D is equal to 1/1 plus d, which is a string of 1's, then y of D is equal to 1 plus d times u of D, which is 1.
1 plus d has an inverse, and this is it.
So now we're coming back to Mr. Aggarwal's point.
Suppose we put in u of D equals this sequence, all 0's before times 0, all 1's after time 0.
Then where will we go?
We'll go 1, and then we'll just stay in this 0 loop forever.
Anything wrong with that?
Is that a code word?
That is a code word, the way we've defined the code.
If we had defined the code so that the u of D were only finite input sequences, then this claim would be correct.
For any finite input sequence, we eventually have to come back to the 0 state.
So this is true for finite inputs, not true for infinite inputs.
So this is wrong.
The fact is d3 equals 1.
The minimum weight of this code is 1.
But you see this is, first of all, it's a confusing situation for analytical purposes.
Let me further indicate why it might be bad to have an encoder like this in general.
The key catastrophicity, in general, will be defined as there are finite outputs generated by infinite inputs, as above.
Or equivalently, there are 0 loops in the state transition diagram.
So by choosing a proper input, you can just keep driving around a 0 loop forever
[INAUDIBLE]
Excuse me?
[INAUDIBLE] the output is becoming finite, so it's actually -- I'm just commenting on the whole catastrophicity
OK, well, I want to next tell you where catastrophicity comes from.
The opposite of this property is non-catastrophicity.
What we want is non-catastrophic encoders.
We want encoders where, to get a finite output, you get a finite output only from finite inputs.
That's better analytically and it's better in practice.
Here, let me draw you the practical argument.
We have, for this particular case, u of D comes in, gets multiplied by 1 plus d. That gives us y of D, which -- let me just abbreviate it.
We transmit that over a channel, and we get our decoder.
Actually, decoder doesn't have much to do here, because all possible sequences are allowed at the output.
So if this were a binary symmetric channel, the best the decoder could do would just be to put out whatever it sees as its guess, y-hat of D. So this is the decoder estimate.
So in any case, this can differ by code words from that.
All possible code words are possible.
And given y-hat of D, to get back to u of D, what do we have to do?
We have to divide by 1/1 plus d, in effect, to invert this equation.
For y of D is equal to 1 plus d times u of D, then the u of D that corresponds to a particular decoded code sequence, y-hat of D, we get by dividing by this.
But this is infinite sequence.
Suppose this makes just one error.
It's possible for this to transmit all 0's, and for this to make just one error.
That's a code word.
That's a legitimate code sequence.
So it's something the decoder might come up with.
If it does that, then what's this going to look like?
This is going to look like an infinite number of 1's, because this is the infinite input that causes this code sequence with a finite number of outputs.
So it's precisely this condition, whenever we make an error that corresponds to a finite output sequence, and we translate it back to the encoder input, that's an infinite encoder input.
There's a one-to-one correspondence between inputs and outputs.
So it's got to happen whenever we make that type of error.
And this is called catastrophic error propagation.
The decoder hasn't done anything bad.
It made one error, as it's going to do once in a while.
But the consequences are catastrophic.
We make an infinite number of errors in the bits that we actually deliver to the user.
Now, of course, what's really going to happen is that sometime later the decoder is going to make another little error.
And at that point, this will switch state again.
So that will terminate the error burst when the decoder makes another error.
But this is completely random.
It may take a long time for this to happen, and it's not good.
So for practical reasons, we don't want catastrophicity.
I was going to give you another example, but in the interest of time, I'll just write it down.
Here's another catastrophic example.
Let me just do a rate 1/2 encoder so it looks more reasonable.
And we'll make it look very much like the encoders we've had.
Our first example looks something like this.
1 plus d squared.
I'll say 1 plus d. So it's a feet-forward encoder.
It looks perfectly reasonable.
The top channel is impulse response, 1 plus d squared.
The second output has impulse response, 1 plus d. If you look at the minimum weight for all finite sequences, it's 4, not quite as good as the other one which had 5.
But is this catastrophic?
Yes?
Who said yes?
[INAUDIBLE]
Great.
So the same observation, if u of D is equal to 1/1 plus d, which is, again, this infinite sequence, then the output, y of D equals u of D, g of D, is equal to -- 1 plus d divides both of these, so we get 1 plus d1.
We get a finite output sequence.
You saw that immediately, because we know that polynomials are divisible by 1 plus d if and only if they have an even number of non-zero terms.
So that's a more elaborate example of where we can get catastrophic behavior.
So what should we do here?
Clearly, we should divide out the common factor from here.
We can do that.
If all of these polynomials have a common factor, we should divide it out.
And we know that the code generated by -- if this is g of D, we're going to say g prime of D is equal to 1/1 plus d times 1 plus d squared, 1 plus d. And that is -- we've already calculated -- 1 plus d1.
So by doing that, first of all, we got rid of the catastrophicity.
Second of all, again, we reduced nu.
So this is clearly a very good thing to do.
So more generally, going back here, if all n of D have any common factor -- in other words, let's call -- I've used d, so what'll I use?
b of D equals the greatest common divisor of all of the n of D, 1 plus --
[INAUDIBLE]
Who --
Do you think it could be 1?
You're, once again, 30 seconds ahead of me.
So yes, that's what I'm going to say.
If they all have a common factor, then they have a GCD that is not equal to 1.
That's if and only if.
So we're going to transform n of D to 1 over b of D. We're going to divide out the common factor, n of D. That will be our n prime.
And now what have we achieved?
We've achieved that there is no common factor of these n prime of D. So no common factor, which does turn out.
So that will mean it's non-catastrophic, and, furthermore, will reduce nu by whatever the degree of this common factor was, degree of b of D. Let me write degree of GCD.
We want them to have no common factor.
We want the greatest common divisor to be 1.
Another way of saying this is we want the ni of d to be relatively prime, the nj of d to be relatively prime.
And if that's true, then can we prove that it's non-catastrophic, that if we have a finite output, that it must've been generated by a finite input?
No, because u of D times n_i of D would have to be a polynomial to be catastrophic, and to get a polynomial.
And u of D is rational.
It has something in the denominator, because u of D is infinite.
That denominator should then get cancelled out to one of these
Yeah, you have the argument in your head.
Let me leave that as an exercise for the student, since at least a couple of you see it.
The only way that you can get this behavior, if you have an infinite input which has in its denominator a common factor of all the n's.
So if the n's have no common factor, then this can't happen.
That's the outline of the argument.
So what's our conclusion now?
Since we have multiple encoders that generate the same code, we'd like to get the nicest one, in some sense, as our canonical encoder for a given code.
So we have a given code.
We can specify it by any causal rational transfer function.
But then, having done that, we should clean this generator n-tuple up.
We should multiply out by the least common multiple of all the divisors to make it polynomial, feedback-free, possibly reduce nu.
And then we should check the numerators and see if they have any common factor, which could be d or it could be any polynomial.
And whatever it is, we should divide that out.
And that will certainly reduce nu.
So at the end of the day, we have a canonical encoder, which is equal to g of D times -- we want to multiply by the least common multiple of the denominators.
We want to divide by the greatest common divisor of the numerators.
And this will be some n prime of D, which will be polynomial.
It will be delay-free.
That means it doesn't have d as a common factor.
It will be non-catastrophic.
That means it doesn't have anything else as a common factor.
And it will have minimal nu, for any encoder that can generate that same code.
I haven't quite proved all those properties, but I think we're well along the way.
So this is really --
[INAUDIBLE].
There should be at least n1 and n2.
[UNINTELLIGIBLE]
Well, if we have a rate 1/1 code, we just have one of them, then what's the greatest common divisor?
Suppose it's n of D over d of D. The greatest common divisor is n of D. The least common multiple is d of D. We get 1.
So going through this exercise for any rate 1/1 code, we'll find that the canonical encoder is 1.
And that is clearly what we want for a rate 1/1 encoder, because a rate 1/1 is always going to have an out.
The code is just going to be the universe code of all the possible formal Laurent sequences.
So we get down to the right encoder for that code, not some stupid encoder.
And in a similar way, going through this process, we come up with the right encoder.
Is this process unique?
Is there only one encoder we could come up with?
Think about it a little, and you could convince yourself there's only encoder that you can get from doing this.
This is over F2.
Over a larger field, it's unique up to units again, up to a non-zero scalar multiple, which of course you can define to be 1 and somehow make this unique.
So it's basically a unique canonical encoder that has all of these properties.
You get it in this way.
So this is what we're always going to use.
For instance, for our original encoder, 1 plus -- what was it?
-- 1 plus d squared, 1 plus d plus d squared.
Is that already in canonical form?
This is irreducible.
This is divisible by 1 plus D, so these are relatively prime.
It's feed-forward.
It doesn't have any denominator terms, and it's delay-free.
So it satisfies all of these, and clearly you can't manipulate it.
The only ways you're allowed to manipulate it are by multiplying by a of D over b of D. And it's clear that if you have a non-trivial numerator or denominator, the numerator will make it catastrophic, the denominator will make it to have feedback.
Now, as I was saying, there's really nothing wrong with feedback.
And so some people prefer -- the one thing that this doesn't have, in general, is it's not systematic.
That means the input stream is not one of the output streams.
So given a canonical encoder like this, can we always make it systematic?
Yeah, we just divide by any of these polynomials.
So here's an equivalent systematic encoder.
For instance, if we divide both of these by 1 plus D squared, we get 1 plus D plus D squared, over 1 plus d squared.
This now is not polynomial.
It is delay-free.
It's non-catastrophic.
There's no 1 over something that we can put in to get a finite thing out.
It turns out it's still minimal nu.
We know how to realize this with two memory elements now.
And in general, if you're not going to introduce a denominator term that has a greater degree than the maximum degree of the numerator terms, so it's still realizable with minimal nu.
And so now it's systematic, but now it's not feedback-free.
So I consider this one certainly the nicest.
The canonical feedback-free encoder is the nicest one for analytical purposes, for instance, for trying to find minumum-weight finite code words.
Because of the finite property, if we're looking for the non-catastrophic property, if we're looking for minimum-weight code words, we only have to look at finite inputs.
We can now use the kind of analysis method that we use that says you always have to come back to the 0 state, because only finite inputs can produce finite outputs here.
This also has that property.
And people tended not to use these for a long time, but it was founded that in turbo codes, you want to have this, one of the generators be infinite.
And it was also nice to have one of them be systematic, so that led to a revival of interest in systematic convolutional encoders.
And this, again, it's a unique form.
If you say the first one has to be the systematic one, then there's a unique equivalent systematic encoder that generates any code.
So it's in alternative canonical form.
Take your pick.
Yeah?
What's systematic?
What's systematic?
Systematic means that the input that's appear unchanged as a subset of the output bits.
For convolutional codes, it means that one of the output sequences, if you have a rate 1/n encoder, one of the output sequences is simply equal to the input sequence.
In other words, one of the transfer functions is equal to 1
[INAUDIBLE]
Yes, very good.
Excellent comment.
Because now, if you have a finite output sequence, in particular you can just read off the input sequence, the information sequence, from this first term, and if the whole thing was finite, it's finite.
So systematic directly ensures non-catastrophicity.
Excellent, excellent comment.
Which you might try to prove by the more elaborate techniques that we had.
So that gets us pretty much through the algebra that I wanted to do.
Now I want to -- OK, good.
Now let's go more to the finite state picture.
And our objective here is mostly going to be to get to Viterbi decoding algorithm.
But along the way, we'll see that we can use something like the Viterbi algorithm to compute minimum distances, too, once we have a finite state picture to the code.
Again, take our example one, where g of D is 1 plus D squared, 1 plus D plus D squared.
We have a state transition diagram, which looks like this.
And I won't bother to label it.
And that's a complete description of the encoder operation, if I put labels of inputs and outputs on all these states.
So 0 slash 0,0, so forth, 1 slash 1, 1.
Just to draw the impulse response, 0, 0, 1, and 0 slash 1, 1.
There's the basic impulse response going around, like that.
Trellis diagram is a very redundant picture of the same thing.
We can start out in the 0 state that we like.
Let's say we start off in the 0 state at some time k, as we always are.
We just don't know what k is.
Then let's spread the state transition diagram out in time.
Let's draw all the states again at time k plus 1.
Actually, there are only two it can get to, at time k plus 1.
So we can either stay in this state or we can go down here to this state, at the next unit of time.
I'm just going to keep drawing all of the possible state trajectories or paths.
At time 2, I can reach all possible states.
0, 0, 1, 0, 0, 1, 1, 1.
Here again, I already know what these are.
From here, I can get 1, 0, 2, 0.
It goes with a 0, 1 out.
So 1 goes with a 1, 0.
And k plus 3, I now have all possible transitions.
It's going to look like this.
Where can I go from 0, 1?
I can go back to here, or I could go to here.
And I could label these.
I should label these.
But I won't.
Here, I can go to these two.
There are eight possible state transitions.
k plus 4.
It's completely repetitive, since it's a time-invariant system.
I just keep going like this.
And why have I gone to all this trouble?
This clearly contains the same information as this diagram, but it's just spread out in time.
The basic reason for doing it is that now there is a unique trellis path corresponding to every code word that can start at time k or later.
So for instance, if I just have an impulse at time k and nothing else, then the unique trellis path is this.
It starts at 0, goes through two non-zero states, comes back to 0, 0, and then stays in 0, 0 forever more
[INAUDIBLE]
I've just taken an arbitrary starting time, k. After a while, I'll just look at it -- well, it's entrained.
This is kind of the steady state version of the trellis.
I've included a little starting part here.
And if I terminated the trellis, then I would also have an ending time.
We'll talk about that.
So at this point, I just want you to see what the trellis diagram is.
It's basically just an unrolling of the state transition diagram out in time.
And it allows me to associate -- there's a one-to-one correspondence now between code words that start at time k or later, and trellis paths.
So every one corresponds to a unique, distinct path through the code.
That's what we're going to use in decoding.
Each of these paths represents one possible decision that the decoder could do about what was actually set.
The decoder is actually going to try to find the best of these paths, in some sense, the one that's closest to the received sequence, in the Hamming distance or Euclidean distance, or likelihood sense.
Let's pause for a second, and see how this could be used to compute the minimum distance of the code.
Let's assume that I have a non-catastrophic encoder.
Then I know the finite sequences are going to be those that start in the 0 state, go through some trajectory through the state transition diagram, and then ultimately come back to the 0 state.
If I have canonical encoder, I agreed what I'm going to use now.
So one way of computing the minimum distance would just be to perform a search through here, and try to find the minimum detour, the minimum weight detour that starts in the 0 state, eventually gets back to the 0 state.
It accumulates Hamming distance along the way.
And that is the Hamming distance of some valid code sequence.
So here, it's not very tough.
And I've already gone through an argument.
If you didn't know, you'd say, OK, here's the first one to look at.
This is the only one that gets back in three time units.
Let's look for the finite path that gets back to the 0 state in four time units.
Again, there's only one of them.
It looks like that, and it turns out it has weight 6.
So this is, again, it always ends up with a 1:1 when it merges back in here.
And this, I think, is 0, 1 or something like that.
The only 0, 0 path in here is this one.
But you can easily see the 0, 0 path doesn't form a loop.
You can't proceed down 0, 0 forever.
So I go through here, and I've already got four units of distance up to here.
In fact, I need to follow.
Next time I explore out here, 1, 1, 0, I've already got up to four units of distance.
Now I know I'm always going to have two at the end.
So any time I get to four or more, I can stop.
And by that argument, I'm already done when I get to this one.
Or if I didn't want to use that, I'd just explore -- I already know that there's one of weight 5, the impulse response, so I'd explore all the possibilities out until they accumulated at least weight 5, and assure myself that there was no way of getting back here with weight less than 5.
And that would be a very systematic way of evaluating the minimum distance to the code.
You can see you could do that for any generators.
You with me?
Just explore this little graph, picking up weight as you go along.
That's basically the way the Viterbi algorithm goes as well, except what we do is we have some received pair at time k, r1 k, r2 k, r1 k plus 1, r2 k plus 1.
We get two received symbols at every time, whatever channel we're going over.
Assuming that the channel is memory-less, I get a some kind of distance or weight or likelihood weight at each time, and maximum likelihood sequence decoding amounts to finding the sequence that's closest, the code sequence that's closest in Hamming distance or Euclidean distance or likelihood distance to the transmitted sequence.
So I can simply put a metric on each of these branches corresponding to what this was.
Now, if it was a binary symmetric channel and I received 0, 0, then I'd have 0 weight here, but weight 2 here, in terms of Hamming distance from the received sequence.
And similarly, as I go along, I'd have another set of metrics here.
If I received 1, 1, then this one would have weight 2, this one would be good, no distance from the received sequence.
Each of these would have distance 1 from the received sequence.
So I get some kind of cost for every path in here.
And then the decoding algorithm is simply a matter of finding the minimum cost path.
You could all do that.
We just had a celebration for Andy Viterbi out in USC, where he's given them $50 million, so they were very happy, and had a great celebration.
And the thing he's best known for is the Viterbi algorithm.
But what everybody knows is that, once you reduce the problem to this point, anybody could have come up with the Viterbi algorithm.
The Viterbi algorithm is just a systematic way for trying to find the lowest-cost path as you go through a trellis.
And if you know dynamic programming and it's obvious how to do it, or even if you don't, the next idea -- do we have a minute?
There's only one idea in the Viterbi algorithm, which is, when you get to this point, you can already make a decision about what the best path is to get to each of these states.
If the closest path over the first three intervals is this one, not this one, say, then you can forget about this path because it's never going to be the first part of the ultimate closest path, because we're simply adding up independent cost, independent increments.
So you can make a subdecision at each of these states, what the best path is up to that state, the closest path.
And you only need to remember that.
That's called a survivor.
And this was basically, Viterbi was the first one to put down this algorithm with the survivor.
You can make interim decisions.
You can only keep the survivor.
Then all you have to do is extend the survivor's one unit of time out here.
Again, you have to add the new metric.
You have to compare the metrics to see which one is better.
You select the survivor, and once again, you've only got four survivors going forward.
So you get an iterative recursive algorithm that just proceeds forward one unit of time at a time, and keeps finding the four best survivors, or in general, the 2 to the nu best survivors at time k plus 3, k plus 4, k plus 5, ad infinitum.
And that's the Viterbi algorithm.
So once you've drawn this trellis picture, it's very clear how you should decode, and you get a very sufficient decoding algorithm.
We'll come back next time.
I'll do that a little bit more carefully.
I'll talk about terminated codes.
I'll talk about how it works when the code isn't terminated, and I'll talk about performance analysis.
But we're actually pretty close to the end of chapter nine at this point.
I'm sorry.
I have to give this to you.
Basically OK, except you didn't understand it the first time.
OK. Good morning.
Happy spring.
Happy April.
We're running a little late.
I'll try to finish chapter nine today and start on chapter ten.
Ashish may have chapter ten here later.
If not, it'll be on the web.
So we've been talking about convolutional codes.
Specifically rate 1/n convolutional codes.
One input n outputs.
As you may imagine, there are also rate k/n convolutional codes.
Generalization to k inputs n outputs.
These have not been so much used in practice, because in general, we're using these binary codes in the power limited regime, and we want low rates.
So 1/n is a good rate.
1/2, 1/3, 1/4, or down to 1/6 are common kinds of rates.
So I haven't bothered to develop the more elaborate theory for rate k/n codes.
The rate 1/n codes, just to review.
The code is simply defined as the set of all possible output sequences of a convolutional encoder, whose N impulse responses are written as an n-tuple g of d as the inputs range over all Laurent sequences, bi-infinite sequences.
And what we showed last time is that without loss of generality or optimality, we could always take this g of d to be of a particular canonical form.
Namely, the g of d could be taken to be a set of n polynomials, g_j of d, that were relatively prime.
And for any code, you can find such a canonical encoder by its unique [UNINTELLIGIBLE] two units, and specifies the code, clearly, by this.
So that's a nice canonical encoder to take.
It furthermore has the property that there's an obvious realization of this encoder in the [UNINTELLIGIBLE] register form with nu memory units.
And therefore 2 to the nu states.
And we will prove in chapter ten that this is the minimal possible encoder for this code.
In other words, there are lots of different encoders that generate this code, but you can't possibly encode it with fewer than the state space of dimension less than nu.
That's the way it's going to sound in chapter ten.
So we haven't quite got there yet, but it's a minimal encoder, therefore, in that sense.
All right.
Today we're going to go now and exploit the finite state structure of this code, to get a very efficient maximum likelihood sequence decoding algorithm called the Viterbi algorithm, which I'm sure you've all heard of.
It's become very famous, not only in this field, but really in any place where you're trying to observe a finite state machine in memory-less noise.
A finite state hidden markov model, if you like.
And so now it's used, for instance, in the detection of genomes, where the exons end and the introns start and so forth, and the garbage is.
And people use it who have no idea that it was originally a digital communications algorithm.
But it's a very obvious algorithm.
It's come up in many different forms, and Viterbi, in a way, was just lucky to get his name on it.
Because it would come up very easily as soon as you pose the problem correctly.
Let's start out with terminated convolutional codes.
Which I forget whether I started on this last time.
I think I may have.
But we'll start from scratch again.
When I say terminated, I mean that we're going to take only a subset of the code words that start at a certain time -- say, time 0 -- continue for some finite time, and then end.
And in this set up, with this canonical encoder, it's polynomial, the easy way to specify that is to let u of d be a finite sequence that starts at time 0.
That's called a polynomial.
and let's restrict its degree -- degree u of d less than k. In other words, it looks like u0 plus u1 d plus so forth up to uk minus 1 d. Therefore I've chosen k, so I really have k information bits or input bits.
All right?
So I specify u0 through uk minus 1.
I use that as the input in my encoder.
What is then my code, my truncated code, let's call it ck -- that's not a very good notation, but we'll use it just for now -- will be the set of all u of d, g of d such that u of d is polynomial and the degree of u of d is less than k. OK.
So we ask what that code is going to be.
And let's take some simple examples.
Turns out when we terminate codes, we always choose polynomial encoders, so the code will naturally collapse back to the 0 state.
How long?
Nu time units after the input stops coming in.
The outputs will be all 0 at that point and forevermore.
The shift register will clear itself out nu time units later, and then they'll be all 0.
So the code really starts at time 0 and ends at time k plus nu.
So we aren't going to worry, in this case, whether the code is non-catastrophic or not.
Here one of the principal properties that we had was this property guaranteed non-catastrophic.
And in fact, is necessary for non-catastrophicity.
So we spent a lot of time talking about why that would be a bad thing, to be catastrophic.
OK.
But let's take a catastrophic rate 1/1 encoder, which simply we have g of d equals 1 plus d. You remember this encoder as nu equals 1.
The input looks like this.
uk, uk minus 1.
And we simply add these, yk.
So we get y of d equals 1 plus d u of d. OK. Now, if the input to this is a finite sequence, a polynomial, let's see what this code can possibly be.
What the code's going to look like.
The code's going to start in state 0, at time 0.
So here's the time.
At the first time, we can either get a 0 in our a 1 in, and accordingly, it will go to state 0 or 1.
At the second time, we can go to state 0 with another 0 or just state 1 with a 1.
If we get a 1 in and we're in state 1, then the output is going to be 0, and we'll stay in state 1.
If we get a 0 in, the output will be a 1, and we'll go back to state 0.
So this is what a typical trellis section looks like here.
We have all possible transitions between the two states.
And so now it goes on for a while like this.
Time invariantly.
There are 1, dot dot dot, and then finally, at some time -- say this is time k, or maybe it's time k minus 1 -- time index is a little fuzzy here -- we don't put any more 1s in.
We only put 0s on from that time forward.
So we could have one more transition back to state 0, and then after that it just stays in state 0 forevermore.
And this isn't very interesting.
It was in time 0 all the time before here.
It was in state 0 all the time before there and put out 0s.
It was in state 0 all the time after here and put out 0s.
And this is clearly not conveying any information.
Not an interesting part of the code.
So we're going to consider the code just to be defined over these k plus 1 units of time.
And so y of d we're going to consider to be a polynomial of degree k. Now we're going to assume that they're k bits in.
Then we're going to wait one more time to let the shift register clear out, which is nu.
And at that time, we're going to take whatever y is, and then we're going to terminate it.
So if we consider this now, we have a block code whose length is the number of non-zero coefficients of y of d. So we have n equals k plus 1, we have k by design information bits, and what's the minimum non-zero weight of this?
It's linear.
What's its minimum non-zero weight, or its minimum distance?
[INAUDIBLE]
1.
Show me a code word of weight 1
[INAUDIBLE]
I claim the minimum non-zero weight is 2 by typical argument.
I have the all 0 sequence.
If I ever leave the all 0 sequence, I accumulate 1 unit of Hamming weight.
And I need to ultimately come back to the all 0 sequence, because I've made everything finite here.
So in this case, I can make the argument that I always have to come back to the all 0 sequence.
Whenever I merge back into the all 0 sequence, I'll accumulate another unit of weight.
So this is a code with minimum distance 2.
Minimum non-zero weight 2.
And in fact, it's just the single parity-check code of length n equals k plus 1.
OK?
And if you ask yourself, can you generate any -- this is supposedly the even weight code.
It contains all even weight k plus 1-tuples.
And I think you can convince yourself that you can generate any of these k plus 1 even weight k-tuples.
In fact, the generator matrix for this code.
Look.
Here's a set of generators.
1 1 is in the code.
0 1 1 is in the code.
0 0 1 1 in the code.
So here's a set of generators.
0 1 1.
Generator matrix for the code looks like this.
We just go like that.
And with these k generators, you can generate any even weight code word.
So we get a kind of convolutional form to the generator matrix.
A sliding parity-check form to the generator matrix.
OK.
So I assert that this is the single parity-check code.
So here's a trellis representation for a block code.
Yeah?
[INAUDIBLE]
I'm sure I've got the time indices screwed up.
But I put in k bits.
I wait 1 more unit of time -- nu equals 1 -- for this to clear out.
So I get out k plus 1 bits.
Think of it in terms of this here.
I put in k, the last input you think of as necessarily being 0, and I take the outputs here for k plus 1 times.
OK.
So first of all, a couple points here
In general, the length will be k plus nu?
In general, length will be k plus nu.
So let's write that down.
So in general, by terminating a rate 1/n -- nu is called the constraint length -- constraint length nu, or 2 to the nu state code -- convolutional code after k inputs, we get a binary linear block code whose parameters are -- we have k information bits.
We have k plus nu non-trivial output times.
At each output time, I get n output bits.
So it's n times k plus nu is the total effective length of the code.
And the distance, we would have to say, is greater than or equal to the distance of the block code is greater than or equal to the distance of the convolutional code.
Why?
Because all of the words in this block code are actually sequences in the convolutional code.
I'm assuming here that the code is not catastrophic, so it's sufficient to look at all the finite sequences in the convolutional code.
All right.
And in general, the block code distance is almost always going to be equal to the convolutional code distance.
And this is just for finite code words.
So as another example, suppose we take our standard rate 1/2, nu equals 2, distance equals 5, convolutional code.
Our example 1 that we've been using all along.
Suppose I terminate with k equals 4, then I'm going to get what?
I'm going to get a 2 times 6, I'm going to get a 12, 4, 5 binary linear block code.
That is not so bad, actually.
There's a point that I'm not going to make very strongly here.
But terminated convolutional codes can be good block codes.
And in fact, asymptotically, by choosing the parameters correctly, you can show that you can get a random ensemble of terminated convolutional codes that is as good as the best possible random ensemble of block codes.
So terminated convolutional codes is one way of constructing optimal block codes in the asymptotic limit.
These parameters aren't bad.
Here's a rate 1/3 code with distance 5.
Not too long.
You know, you might compare it to what?
The BCH code?
There's a 15 7 5 BCH code, which is clearly better, because it's higher rate.
But if you shorten this code, you have to shorten this by 3, this by 3 to keep the distance.
You would get the same code.
So it's not optimum, but it's not too bad.
And furthermore, from this construction, you get a trellis representation, which in this case would look like this.
It looks like the trellis that we started to draw last time.
Sorry.
Doesn't look like that.
This guy goes down here, this guy comes here, this guy goes here.
This continues for a while.
And then finally, when we start to enforce all zeros, we only have one possible output from each of these.
And we get a trellis that looks like that.
All right?
Which is 1, 2, 3 -- I did it correctly for this code, actually.
Because each of these is going to have a 2-tuple output on it.
And that looked like 0,0 0,0 0,0 0,0 0,0.
Each of the information bits causes a two-way branch, no matter where you are in the trellis.
So here's 1, here's 2, here's 3, here's 4.
That's the end of them.
We wait for nu equal 2 to let the states converge back to 0.
So that's a trellis representation of this 12 4 5 block code.
OK.
I want to talk about terminated block codes, terminated trellis codes, convolutional codes as block codes, for two reasons.
One is to say we can get block codes that way.
But the other is to introduce the Viterbi algorithm for terminated convolutional codes.
I think it's easier first to look at how the algorithm works when you have a finite trellis like this, and then we'll go on and say, well, how would this work if we let the trellis become very long, or in principle, infinite?
Would the Viterbi algorithm still work?
And it becomes obvious that it does.
So this is going to be a maximum likelihood sequence detection or decoding algorithm.
We're going to assume that each time here, we get to see two things.
Corresponding to the two bits we transmit.
So we transmit yk according to which trellis branch we're on -- y0 at this point -- and we receive a 2-tuple r0.
If we were on a binary input additive white Gaussian channel, this would go into, actually, the 2-PAM map, two plus or minus alphas, and be received as two real numbers.
And we do the same thing here.
y1, we receive r1, and so forth, up to yk plus nu, or k plus nu minus 1, I think it is, rk plus nu minus 1.
OK.
So we have a transmit 2-tuple and a received 2-tuple every time.
And we're going to assume that the channel is memoryless so that the probabilities of receiving the various possible received values, the likelihoods, are independent from time unit to time unit.
That's the only way this works.
Then if I want to get the total probability of r given y, where now I'm talking about over the whole block, this factors into the probabilities of rk given yk, yes?
That's what I'm going to depend on.
This is what's called the memoryless condition, memoryless channel.
The transition probabilities do not depend on what's been sent or received in previous blocks.
Or it's more convenient to use the minus log likelihood equals now the sum of the minus log p of rk given yk.
So maximum likelihood is equivalent to minimize the neg-log likelihood.
OK.
I receive only one thing in each time unit.
And each trellis branch corresponds to transmitting a specific thing.
So I can label each trellis branch with the appropriate minus log p of rk given yk.
Do you see that?
In other words, for this trellis branch down here, which is associated with y2 equals 0 1, or s of y2 equals alpha minus alpha, suppose I receive r2 equals r1, r2, I would label this by, say, the Euclidean distance between what I transmitted for that branch and what I received.
For instance, on the additive white Gaussian channel, this would simply be rk minus s of yk.
The Euclidean distance squared.
Or equivalently, minus the inner product, the correlation between rk and s of yk.
You know, these are equivalent metrics.
Or on a binary symmetric channel, might be the Hamming distance between what I received and what I transmitted, both of which would be binary 2-tuples, in this case.
It's just some measure of distance -- Log likelihood distance, Euclidean distance, Hamming distance -- between the particular thing I would have transmitted if I'd been on this branch and the particular thing that I actually did receive.
So having received a whole block of data, I can now put a cost on each of these branches.
What's the minus log likelihood cost if I say that the code word goes through that branch?
What does it cost me?
All right.
Now what I'm basically depending on is note, there's a one-to-one map between code words y and c and trellis paths.
For this specific trellis up here, how many paths are there through the trellis?
Do you see that however I go through the trellis, I'm going to meet four two-way branches?
All right?
According to [UNINTELLIGIBLE], there are four yes-no questions.
It's a binary tree.
So there are 16 possible ways to get through this trellis from start to finish, if I view it as a maze problem.
And they correspond -- I haven't labeled all the branche -- but they do correspond to all 16 words in this block code.
So now what is maximum likely decoding?
I want to find the one of those 16 words that has the greatest likelihood, or the least negative log likelihood, over all of these 12 received symbols or 6 received times.
So what will that be?
Once I've labeled each of these branches with a log likelihood cost, I simply want to find the least cost path through this trellis.
And that will correspond to the most likely code word.
This is the essence of it.
Do you all see that?
[INAUDIBLE]
The independent transmission allows me to break it up into a symbol-by-symbol or time-by-time expression.
So I can simply cumulate over this sum right here.
And I'm just looking for the minimum sum over all the possible ways I could get through this trellis.
So now we translated this into finding the minimum cost path through a graph.
Through a graph with a very special structure, nice regular structure.
OK. Once you've got that, then I think it's very obvious how to come up with a nice recursive algorithm for doing that.
Here's my recursive algorithm.
I'm going to start computing weights.
I'm going to start here.
I'm going to assign weight zero here.
Right here I'm going to assign a weight which is equal to the cost of the best path to get to that node in the graph.
In this case, it's just the cost of that branch.
Similarly down here.
Similarly I proceed to these four.
There's only one way to get to each of these four, and I simply add up what the total weight is.
Now here's the first point where it gets interesting, where I have two possible ways to get to this node.
One way is via these three branches, and that has a certain cost.
Another way is via these three branches, and that has a certain cost.
Suppose the cost of these three branches is higher than the cost of these three branches.
Just pair-wise.
All right?
Claim that I can pick the minimum cost path to this node, throw away the other possibility, and I can never have thrown away something which itself is part of the minimum cost path through the whole trellis.
Clearly.
Suppose this is the best path from here to here.
This is worse.
X, X, X, X X.
And now I find the minimum cost path through the whole trellis goes through this node.
Say it's like that.
Could it have started with this?
No.
Because I can always replace this with a path that starts with that and get a better path.
So at this point, I can make a decision between all the paths that get to that node and pick just one.
The best one that gets to that node.
And that's called the survivor.
This is really the key concept that Viterbi introduced.
We only need to say one, the best up to that time.
We need to do it for all the 2 to the nu possibilities.
So we have 2 to the nu survivors at time k or time i, whatever your time index is.
Used k for something else.
OK.
So I can throw away half the possibilities.
Now each survivor, I remember what its past history is, and I remember what its cost is to that point.
Now to proceed, the recursion is to proceed one time unit ahead, to add the incremental cost -- say, to make a decision here, I need to add the incremental cost to the two survivors that were here and here.
So I add an incremental cost, according to what these branches are labeled with.
And now I find the best of these two possibilities to get to this node.
So the recursion is called an ACS operation for add, we add increments to each of these paths, we compare which is the best, and we select the best one.
We keep that.
We throw away the other.
We now have a one unit longer best path to this node and its cost for all 2 the nu survivors at time i plus 1.
And just proceeding in that way, we go through until we finally get to the terminating node.
And at this point, when we've found the best path to this node, we've found the best path through the whole trellis.
OK?
So that's all it is.
I believe it's just totally obvious once you reduce it to a search for a minimum cost path through the trellis.
It's very well suited to the application of convolutional codes, because we really are thinking of sending these bits as a stream in time.
And at the receiver, you can think of just proceeding forward one unit of time, 2 to the nu, add, compare, selects, and we've got a new set of survivors.
There are nice implications of this that involve, you can build a computer for every node or for every pair of nodes to do the ACS operation, make a very fast recursion through here.
So that's it.
That's the Viterbi algorithm for terminated convolutional codes.
Now let's ask about the VIterbi algorithm for unterminated convolutional codes.
Suppose this trellis becomes very long.
Across the whole page.
What are these survivors going to look like?
Suppose we start from a definite node here, and we've got a four state convolutional code, and we iterate and we iterate and we iterate with the Viterbi algorithm.
After a long time, somewhere out here, we're going to have four survivors and their histories.
And I've greatly simplified, but basically the histories are going to look schematically something -- I don't know -- could be anything.
Look like this.
The point I'm illustrating here is that the histories will be distinct right at this time.
They have to be, because they're going to four distinct stages.
But as you go backwards in time, you will find they merge, any two histories will merge, at a certain time back.
And this is a probablistic thing.
It depends on what's happening on the channel and so forth.
But with high probability, they will have merged not too far back in time.
So even if we never get to a final node, even if we just let this process continue forever, at this point we can say, we can make a definite decision at this time.
Right?
Because all survivors start with a common initial part, up to here.
So one way to operate the Viterbi algorithm would be just to, at this point, say, OK.
I'm going to put out everything before this time.
No matter how long I run the decoder, the first part of it is always going to be this part.
So these are definitely decided up to this time.
And then I still don't know about here.
I'm going to have to go a little bit further.
You proceed further, and after a while, you find more that's definitely done.
In practice, that would lead to a sporadic output rate.
That isn't really what you want.
So in practice what you do is you establish a decision delay, delta.
And what you put out is you hope that 99.999% of the time, that if you look back delta on all the survivors, they will all be a common path, delta back here.
So there's a very high probability you will have converged.
And so simply at this time, you put out what you decided on delta time units earlier.
Next time you put out the next one.
Next time you put out the next one.
So you get a nice, regular, synchronous stream of data coming out of here.
Every so often, it may happen that you get out to here and you still haven't made a decision.
Then you have to do something.
And it really doesn't matter terribly much what you do.
You might pick the guy who has the best metric at this time, or you simply might say, well, I'm always going to pick the one that goes to all zeros.
That wouldn't be symmetric.
You can make any decision you like.
And you could be wrong.
But you pick delta large enough so that the probability of this happening is very rare, and this just adds a little bit to your error probability, and as long as the probability of making an error because of this kind of operation is much lower than your probability of making an ordinary decoding error, then you're going to be OK. For convolutional codes way back at the beginning of time, people decided that a decision delay a 5 times nu, 5 times the constraint length, was the right rule of thumb.
And that's been the rule of thumb for rate 1/n codes forever after.
The point is, delta should be a lot more than nu.
You know, after one constraint length, you certainly won't have converged.
After five constraint lengths, you're highly likely to have converged.
And theoretically, the probability of not converging goes down exponentially with delta.
So big enough is going to work.
And a final point is that sometimes, you really care that what you put out to be a true code word.
In that case, if you get to this situation, you have to make a choice, you make a choice here, then you have to actually eliminate all the survivors that are not consistent with that choice.
And you can do that simply by putting an infinite metric on this guy here.
Then he'll get wiped out as soon as he's compared with anybody else.
And that will ensure that whatever you eventually put out, you keep the sequence being a legitimate code sequence.
So that's a very fine point.
OK.
So there really isn't any serious problem with letting the Viterbi algorithm run indefinitely in time once you've got it started.
How do you get it started?
Suppose you came online and you simply had a stream of outputs, transmitted from a convolutional code over a channel, and you didn't know what state to start in.
How do you use synchronize to a starting state?
Well, this is not hard either.
Basically you start, you're in one of four states, or 2 to the nu states.
You don't know which one.
Let's just give them all cost 0 and start decoding, using the four state trellis.
And we just start receiving 2-tuples from here on.
We get rk, r k plus 1, and so forth.
So how should we start?
Well, we'll just start like that.
And we get sort of the mirror image of this -- that after a time, these will all converge to a single path.
Or after a time, when we're way down here, this is what the situation will look like.
These things will each have a different route over here, but they will have converged in here.
There will be a long path over which they're all converged, and then towards the end, they'll be unrooted again.
OK. Well, that's fine.
What does that mean in practice?
That means we make errors during the synchronization.
But we say we're synchronized when we get this case where all the paths going to all the current survivors have a common central stage.
And how long does it take to synchronize?
Again, by analysis, it's exactly this same delta again.
The probability of not being synchronized after delta goes down exponentially with delta in exactly the same way.
It's just a mirror image from one end to the other.
So from a practical point of view, this is no problem.
And just starting the Viterbi decoder up, with arbitrary metrics here, and after five constraint lengths, if you like, it's highly likely to have gotten synchronized.
You'll make errors for five constraint lengths and after that, you'll be OK, as though you knew the starting state.
So the moral is, no problem.
You can just set up the Viterbi algorithm and let it run.
The costs will all, of course, increase without bound, which is [UNINTELLIGIBLE].
You can always renormalize them, subtract a common cost from all of them.
That won't change anything.
Keep them within running time.
So we don't need to terminate convolutional codes in order to run the Viterbi algorithm.
We just let it self-synchronize and we make decisions with some decision delay.
And the additional problems that we have are very, very small.
There are no additional problems.
Yeah?
[INAUDIBLE]
At this point?
Well, notice that we don't know that we've synchronized until we've continued further, and we've got -- you know, where we've really synchronized is when we see that every survivor path has the common root.
So at that point, there is really only one path here.
And we can say that the synchronized part is definitely decoded.
And we can't really say too much about this out here, because this depends on what's happened out in the past.
So you say this is erasures, if you like.
The stuff that we're pretty sure has a high probability of being wrong
[INAUDIBLE]
There is only one decoded path during this interval
But before that interval there are branches and so on right
Well here, we don't know anything.
Here we know the results of one computation.
What are you suggesting?
Just pick the best one at that point?
And finally after you reach the [INAUDIBLE]?
And finally?
I'm just not sure exactly what the logic is of your algorithm.
It's clear for this, it wouldn't make any difference if we just started off arbitrarily so, we're going to start in the zero state.
And we only allow things to start in the zero state.
Well, we'll eventually get to this path anyway.
So it really doesn't matter how you start.
You're going to have garbage for a while, and then you're going to be OK.
There's no point in doing anything more sophisticated than that.
I don't want to discuss what you're suggesting, because I think there's a flaw in it.
Try to figure out what time you're going to make this decision at.
OK. Do we all understand the Viterbi algorithm?
Yes?
Good.
We can easily program it up.
There will be an exercise on the homework.
But now you can all do the Viterbi algorithm.
All right.
So -- and oh.
What's the complexity of the Viterbi algorithm?
What is the complexity?
We always want to be talking about performance versus complexity.
So it's a recursive algorithm.
We do exactly the same operations every unit of time.
What do the operations consist of?
They consist of add, compare, select.
How many additions to we have to make?
Additions are basically equal to the number of branches in each unit of time in the trellis, which is 2 to the nu plus one.
Is that clear?
We have 2 to the nu states.
Two branches out of, two branches into each state, always for rate one.
So we have 2 to the nu plus 1 additions.
We get 2 the nu compares, one for each state.
Which is really, you can consider the select to be part of the compare.
Overall, you can say the complexity is of the order of 2 to the nu or 2 to the nu plus 1.
This is the number of states or the state complexity.
This is the number of branches.
I will argue a little bit later that the branch complexity is really more fundamental.
You've got to do at least one thing for each branch.
So in different set up, the branch complexity.
But these are practically the same thing, and so we say that the complexity of the Viterbi algorithm is basically like the number of states.
We have a four state encoder, the complexity's like four, it goes up exponentially with the constraint length.
This says, this is going to be nice, as long as we have short constraint length.
For longer constraint lengths, you know, a constraint length 20, it's going to be a pretty horrible algorithm.
So we can only use the Viterbi algorithm relatively short constraint length codes, relatively small numbers of states.
The biggest Viterbi algorithm that I'm aware has ever been built is a 2 the 14th state algorithm.
The so-called big Viterbi decoder out at JPL.
It was used for the Galileo space missions.
It's in a rack that big.
I'm sure nowadays you could practically get it on a chip, and you could maybe do 2 the 20th states.
But so this exponential complexity, in computer science terms, not really what we want.
But when we're talking about moderate complexity decoders, these really have proved to be the most effective, and become the standard moderate complexity decoder.
The advantage is that we can do true maximum likelihood decoding on a sequence basis, using soft decisions, using whatever reliability information the channel has, as long as it's memoryless.
So last topic is to talk about performance.
How are we going to evaluate the performance of convolutional codes?
You remember what we did on block codes?
We basically looked at the pairwise error probability between block code words, we then did the union bound, based on the pairwise error probabilities.
And we observed that the union bound was typically dominated by the minimum distance error events, and so we get the union bound estimate, which was purely based on the minimum distance possible errors.
And we can do exactly the same thing in the convolutional case.
Convolutional case, again, is a linear code.
That means it has the group property, has symmetry such that the distances from every possible code sequence to all other code sequences are going to be the same, since we're talking on a long or possibly infinite sequence basis.
And we need to be just a little bit more careful about what an error consists of.
We need to talk about error events.
And this is a simple concept.
Let us again draw a path corresponding to the transmitted code words.
A very long path, potentially infinite, but it is some definite sequence.
And let's run it through a memory list channel, use the Viterbi algorithm, and we're going to get some received code word, or decoded code word.
This is one place where you might want to insist that the Viterbi algorithm actually put out a code word.
What is that going to look like?
Well, if you're running normally, the received code word is going to equal the transmitted code word most the time.
Except it's going to make errors.
And what will the errors look like?
They'll look like a branch off through the trellis, and then eventually a reemerging into the same state.
And similarly, you go on longer, and you might have another error.
And any place where there's a difference -- should have done a different color here -- any place where there's difference, this is called an error event.
So what I'm illustrating is a case when we had two disjoint error events.
Is that concept clear?
We're going to draw the trellis paths corresponding the transmitted code word and the decoded code word.
Wherever they diverge over a period of time, we're going to call it an error event.
But eventually they will re-merge again.
So it could be a short time.
Can't be any shorter than the constraint length plus 1.
That would be the minimum length error event.
Could be a longer time.
Unbounded, actually.
OK. What is going to be the probability of an error event starting at some time?
And when I say an error event starting at time k, let's suppose I've been going along on the transmitted path, and the decoder has still got that there.
I'm asking, what is the probability that this code word is actually more likely on a maximum likelihood sequence detection basis than this one?
Well, simply the probability that the received sequence is closer to this one than this one.
We know how to analyze that for a finite differences.
What is the difference here?
The difference, call this y of d and this y hat of d. What is y of d minus y hat of d?
We'll call that e of d. This is a code word.
This is a decoded code word.
So the error event has to be a code word.
Right?
Decoder made a mistake.
The mistake has to be a code word.
So we're asking, what is the probability that y of d sent y hat of d decoded, where y hat of d equals y of d plus e of d?
We'll just ask for that particular event.
This is the probability that r of d closer to y hat of d than y of d. Now, making a big leap.
This is equal to -- we're just talking about two sequences in Euclidean space.
All that matters is the Euclidean distance between them.
What is the Euclidean distance between them?
It's 4 alpha squared times the weight of this error event, the Hamming weight of this error event.
And so if you remember how to calculate pairwise error probabilities, this is just alpha squared D over sigma squared, where D is the distance of e of d. The weight, the Hamming weight of e of d. Call it sigma squared.
Remember something that looked like that?
So once again, we go from the Hamming weight of a possible error event to a Euclidean weight, which is 4 alpha squared times the Hamming weight.
We actually take the square root of that and we only need to make an error, a noise of half of that length.
So that's where the 4 disappears.
And we just get q of d squared over sigma squared, where d is the distance to the decision boundary.
Compressing a lot of steps, but it's all something you felt you knew well a few chapters ago.
So the union bound would simply be the sum over all error events in the code such that the start -- so e of d is -- you want them to start at time 0, say.
Let's ask for the probably of an error event starting at time 0.
So we want it to be polynomial, have no non-zero negative coefficients, and have the coefficient at time 0 be 1, or not 0.
I'm doing this very roughly.
Of q to the square root of alpha squared times the weight Hamming of e of d over sigma squared.
Just as before.
So to get the union bound, we sum up over the weights of all sequences that start at time 0.
Let's do it for our favorite example.
Suppose g of d is 1 plus d squared, 1 plus d plus d squared, then what are the possible e of d's that I'm talking about?
I have this itself.
1 plus d squared, 1 plus d plus d squared.
This is weight equal to 5.
What's my next possible error event?
Would be 1 plus d times this.
So it's going to be a table.
We have g of d, 1 plus d times g of d, which is equal to 1 plus d plus d squared plus d cubed, 1 plus d cubed, that has weight equals 6.
What's my next longer one?
1 plus d squared times g of d equals 1 plus d fourth, 1 plus d plus d third plus d fourth, that has weight 6.
1 plus d plus d squared times g of d. 1 plus d plus -- this is 1 plus d plus -- plus d third, plus d forth.
I may be making a mistake here.
1 plus d squared plus d fourth.
It's hard to do this by hand after a while.
OK. Notice, we start tabulating all the error events, which I can do in order of the degree of u of d, always keeping the non-zero, the time 0 term equal to 1.
So this is the only one of length 1.
This is [UNINTELLIGIBLE] of degree 0.
This is the only one of degree 1.
There are two of them of degree 2.
There will be four of them of degree 3, and so forth.
So I can lay out what all the error events could be, and I look at what their weights are.
Here's my minimum weight error event.
Then I have two of weight six and so forth.
So the union bound, I'd be adding up this term.
I'd get one term where the weight is 5, two where the weight is 6, I don't know how many where the weight is 7.
I happen to know there were only two weight 6 error events.
And you simply have to find out what the weight profile is and put it in the union bound.
Or you can use the union bound estimate, which is simply -- let's just take k nd, the number of error events of minimum weight d, times q to the square root of alpha squared d, where d equals min weight over sigma squared.
OK.
In this case, nd equals 1, d equals 6.
Let me take this one step further.
nd times q to the square root of -- what is sigma squared equals n0 over 2?
And Eb equals n times alpha squared.
The energy per transmitted bit is alpha squared.
We're going to transmit n bits for every information bit.
So plugging those in, I get 2d over n times Eb over N0, which is again of the form nd q to the square root of 2 times the coding gain of the code times Eb over N0.
Bottom line is I get precisely the same performance analysis, or I get a nominal coding gain of -- in general, it's kd over n. If it were a rate k over n code, since we're only considering rate 1 over n codes, it's just d over n. And I get an error coefficient which equals the number of weight d code words, starting time 0.
Of course an error event could start at any time.
If there are nd starting at time 0, how many start at time 1?
Time invariant code.
So it's going to be the same numbers could possibly start at time 1.
So what I'm computing here is the probability of an error event starting at a particular time.
For this particular code, what do I have?
I have one event of weight 5.
So the nominal coding gain is 5 over n, which is 2.
Which is -- 5 is 7 dB, 2 is 3 dB, so this is 4 dB.
That's pretty good.
Nominal coding gain of 4 dB with only a four state code.
Obviously a very simple decoder for this code.
And what is kd equals 1?
That implies -- again, we have the same argument about whatever the error coefficient is.
You could plot this curve.
The larger the error coefficient is, the more the curve moves up, and therefore over.
You get an effective coding gain which is less.
But this means since it's 1, you don't have to do that.
The effective coding gain is the same as the nominal coding gain.
It's still 4 dB.
So it's a real 4 dB of coding gain for the simple little two state code.
This code compares very directly with the 8 4 4 Reed-Muller code, block code.
This code also has rate 1/2.
It has the same rate.
We'll see that it also has a four-state trellis diagram.
But it only has distance four, which means its nominal coding gain is only 2, 3 dB.
And furthermore, it has 14 minimum weight words, which even dividing by 4 to get the number per bit, there's still a factor of 3, still going to cost us another couple of tenths of a dB.
Its effective coding gain is only about 2.6 or 2.7 dB.
All right?
So this code has much better performance for about the same complexity as this code, at the same rate.
And this tends to be typical.
Convolutional codes just beat block codes when you compare them in this way.
Notice that we're assuming maximum likelihood decoding.
We don't yet have a maximum likelihood decoding algorithm for this code.
We'll find for this code, we can also decode it using the Viterbi algorithm with a four state decoder comparable to this one.
So it would be, I would say, for maximum likelihood decoding -- about same complexity.
But we simply get much better performance with the convolutional.
Yeah?
[INAUDIBLE]
Say again?
What happened to the k?
Why is k equal to 1?
Which k?
Over here?
Yeah.
kd [INAUDIBLE]
All right.
We're only considering rate 1/n codes.
If it were a rate k/n code, we would get k over n, because this would be n over k times alpha squared.
Just the same as before.
In fact, I just want to wave my hand, say, everything goes through as before.
As soon as you get the error event concept, you can reduce it to the calculation of pairwise error probabilities, and then the union bound estimate is as before
[INAUDIBLE]?
Well, I had my little trellis here.
And how long in real time does it take for two paths to merge?
An error event has got to take at least nu plus 1 time units for two paths to diverge and then merge again in the trellis.
Is that clear?
Or put another way, the lowest degree of any possible error event is nu, which means it actually takes place over nu plus 1 time units.
OK?
From this
[INAUDIBLE]
Why is the lowest --?
I'm taking g of d -- the definition of nu is what?
The maximum degree of g of d. OK?
So if that's so, then the shortest length error event is 1 times g of d, which takes nu plus 1 time units to run out.
So error events have to be at least this long, and then they can be any integer length longer than that.
You don't look totally happy
[INAUDIBLE]
You understand?
[INAUDIBLE]
How I see it from this diagram?
Where have I got a picture of a trellis?
Here I've got a picture of a trellis.
I've defined an error event by taking a transmitted code word.
It's a code sequence.
This is supposed to represent some path through the trellis.
There's one-to-one correspondence between code sequences and trellis paths.
Then I find another code sequence, which is the one I actually decided on.
Call that the decoded code sequence.
And I say, what's the minimum length of time they could be diverged from one another?
All right?
Let's take this particular trellis.
What's the minimum length of time any two paths -- say, here's the transmitted path.
Suppose I try to find another path that diverges from it?
Here's one.
Comes back to it.
Here's one.
Another one.
Comes back to it.
I say that the minimum length of time it could take is nu plus 1.
Why?
Because the difference between these two paths is itself a code sequence, is therefore a non-zero polynomial multiple of g of d. OK?
Same argument.
OK.
So I guess I'm not going to get into chapter 10, so I'll discourse a little bit more about convolutional codes versus block codes.
How do you construct convolutional codes, actually?
You see that really, what you want to do is to first of all, maximize the minimum distance for certain constraint length.
Subject to that, you want to minimize the number of minimum distance words.
You want to, in fact, get the best distance profile.
Is there any block codes?
We had nice, algebraic ways of doing this.
Roots of polynomials, Reed-Solomon codes.
We could develop an algebraic formula which told us what the minimum distance was.
Do we have any nice algebraic constructions like that convolutional code?
No.
Basically, you've just got to search all the possible polynomials where the maximum degree is nu.
Take pairs of polynomials.
If you want a rate 1/2 code of degree nu, there's not that many things you have to search.
All right?
You just take all possible pairs of binary polynomials of degree nu or less, making sure that you don't take any two which have a common divisor, an untrivial common divisor, so you can wipe those out.
You want to make sure that the constant term is 1.
There's no point in sliding one over so it has a larger constant term than 1.
But subject to those provisos, you simply try all pairs g1, g2, and you just do -- as soon as you've assured yourself they're not catastrophic, they don't have a common divisor, then you can just list, you know, the finite code words are going to be the ones that are generated by finite information sequences.
So you can just list all the code words, as I've started to do up here.
Or since the trellis is, in fact, a way of listing all the code words, there's a one-to-one correspondence between code words and trellis paths, you can just start searching through the trellis and you will quickly find all the minimum weight code words, and thereby establish the minimum distance, the weight profile as far out as you like.
And you choose the best.
You try all possibilities.
So Joseph Odenwalder did this as soon as people recognized that short convolutional codes could be practical, and he published the tables back in his PhD thesis in '69, so he got a PhD thesis out of this.
It wasn't that hard, it's done once and for all, and the results are in the notes.
The tables.
And you can see from the tables that in terms of performance versus moderate complexity, things go pretty well.
Here's this four state code.
It already gets you 4 dB of effective coding gain.
To get to 6 dB of effective coding gain, you need to go up to about 64 states.
First person to do this was Jerry Heller at Jet Propulsion Laboratory.
Again, about '68, immediately after Viterbi proposed his algorithm in '67, Heller was the first to go out and say, well, let's see how these perform.
So he found a good 64 state rate 1/2 code, and he did probability of error versus Eb over N0, and he said, wow.
I get a 6 dB coding gain.
And Viterbi had no idea that his algorithm would actually be useful in practice.
He was just using it to make a proof.
And he didn't even know it was optimum.
But he's always given the credit to Heller for realizing it could be practical.
And so that 64 state rate 1/2 code with Viterbi algorithm decoding became very popular in the '70s.
Heller and Viterbi and Jacobs went off to form a company called Linkabit.
And for the technology of the time, that seemed to be a very appropriate solution.
You see you get approximately the same thing as a rate 1/3, rate 1/4.
If you go lower in rate you can do marginally better.
If you are after tenths of a dB, it's worthwhile.
Later in the decade, is the best way to get more gain to go to more and more complicated convolutional codes?
No.
The best way is to use the concatenated idea.
Once you use these maximum likelihood decoders, the Viterbi decoders, to get your error rate down to 10 to the minus 3 or something, 1 in 1000, at that point you have very few errors, and you can apply then an outer code to clean up the error events that do occur.
You see, you're going to get bursts of errors in your decoded sequence.
It's a very natural idea to have an outer code, which is based on gf of 256, say, 8 bit bytes.
And so a Reed-Solomon code comes along and cleans up the errors that do occur, and drives the error probability down to 10 to the minus 12, to whatever you like it, with very little redundancy, very little additional cost.
So that became the standard approach for space communications in the '70s and indeed '80s.
I've already mentioned that around '90, they went up to this 2 to the fourteenth state Viterbi decoder.
They went to much more powerful outer codes, much cleverer.
And they were able to get to within about 2 or 3 dB of the Shannon limit.
And that was the state-of-the-art on the eve of the discovery of turbo codes, which is where we're going in all of this.
So from a practical point of view, in the moderate complexity regime, simple convolutional codes with moderate complexity Viterbi decoding are the best still that anybody knows how to do.
They have all these system advantages.
They work with nice synchronous streams of traffic, which is what you want for data transmission.
They use soft decisions.
They use any kind of reliability information you have.
They're not limited by hard decisions.
They're not limited by bounded systems as the algebraic schemes are.
So it just proved to be a better way to go on channels like the additive white Gaussian noise channel.
OK.
So we didn't get into chapter 10.
We'll start that next time.
[WRITING ON CHALKBOARD]
Good morning.
I think I'll start even though people are not quite all here, because it's time.
Sorry, I'd hoped to start Chapter 10 last time, but we'll start it this time.
When Ashish comes, we'll have handouts for Chapter 10, the new problem set and the old problem set solutions as usual.
This chapter is about trellis diagrams for block codes.
Binary linear block codes of the kind that we've seen before, it's actually a much more general notion that applies to -- well, linearity is important, although we don't see it very specifically in the trellis diagram.
But we'll see that for constructing a unique minimal trellis, linearity is an important aspect.
Or more generally, it's just the group property of codes that allows us to construct unique minimal trellis diagrams.
And I think it's best to motivate this chapter by an example.
Here's a nice example of a code that we know very well.
It has a nice trellis diagram.
It's the 8, 4, 4 Reed-Muller code.
Just to remind you of how we construct this code, or one of the methods.
We have many methods.
We take this kind of universal eight by eight -- well, it's a list of eight tuples.
And it's the tensor product of this little matrix three times.
And if we create this, we get a matrix that looks like that.
And we just pick the rows that have weight four or greater, and they are the generators of the 8, 4, 4 Reed-Muller code.
So we'll take those as our set of generators.
It's the 16 code words generated by these four generators here.
This goes back to the first part of the term, but I hope you recall that fairly quickly.
Now I'd like to draw a trellis diagram for this code.
I don't particularly know why yet.
One idea I might have in mind is to do a Viterbi algorithm decoding as a maximum likelihood decoding algorithm for this code.
So let's see if I can come up with an efficient trellis diagram.
That means one that first of all has really few states in it.
Relatively few branches would be efficient as a decoding map for the Viterbi algorithm.
And here's a very nice one.
This looks sort of like a rate 1/2 four-state trellis diagram.
Of course, it's not time-invariant, doesn't go on forever like for a convolutional code, for a block code.
Block code only exists over a finite time.
So we're going to get a block trellis that starts at a particular time, that ends at a particular time in a single state.
But we want the basic property that the -- the paths in this trellis are in one-to-one correspondence with the words in the code.
So, how many words are there in the code?
There are 16 in this code.
How many paths are there through this trellis?
Well, again it has a very regular structure.
There is a four-way branch here.
Wherever you get to, you have a two-way branch here.
You have a two-way branch here.
You have no choice here.
So there are 16 possible ways to get through this trellis, 16 trajectories if you like.
And next let's check if they correspond to all the possible code words.
Here is the all 0 code word up here.
What's next?
Let's check for the generators.
Here is one of the generators, 1, 1, 1, 1, 0, 0, 0, 0.
And it's a 1, 1, 0, 0.
This is 1, 1 also.
1, 1 -- no, I'm sorry.
1, 1, 0, 0, 1, 1, 0, 0.
That's right.
That's this one going through here.
And 1, 0, 1, 0, 1, 0, 1, 0.
That's here.
And what's the last one?
All ones.
Well, that's up here too.
The all one path.
Yeah
How did you know whether to make the [INAUDIBLE] procedure?
I had side information.
A genie told me.
I know that this is an efficient trellis.
Where we'll get to, and I hope today in this lecture, is a turn-the-crank method of constructing not only a trellis, but the minimal possible trellis.
So you'll be able to see how starting from a set of four generators like this, turn the crank, and I'll produce a trellis for you.
But of course you don't see that yet.
I just know that this is a trellis, and it's a nice trellis.
It has nice, regular structure.
You could decode this with a Viterbi algorithm, and you could see the complexity would be comparable to the complexity of decoding a rate 1/2 four-state convolutional code.
And in fact, you remember I compared our example rate 1/2 four-state convolutional code which had a coding gain of 4 dB with this code saying they roughly have the same complexity.
And when I said that, this is what I had in mind, that I could decode each of them with a Viterbi algorithm with approximately the same amount of complexity.
And unfortunately this only has a nominal coding gain of 3 dB, and an effective coding gain of slightly less than that.
So it's not quite as good as the convolutional code.
And at the end of the day, we're going to find out that that's kind of typical of a comparison between block and convolutional codes of the same trellis complexity.
But on the other hand, this is certainly a better way to decode this code, recursive by the Viterbi algorithm, than to do a full maximum likelihood decoding of all 16 code words, which involves just brute force computing the distance to 16 code words.
This is if you like a more organized way of performing the computation of how far is the receive sequence from each of the 16 code words.
So the reason we look for trellises of block codes is first of all, it's better than exhaustive maximum likelihood decoding.
It is a maximum likelihood decoding algorithm since we're going to find that we can construct a trellis for any linear block code, which at worst, it's never going to be more complicated than just enumerating all the code words and doing exhaustive maximum likelihood decoding.
And usually, as in this case, it's going to be a more efficient method of doing maximum likelihood decoding.
So we will have a general method of maximum likelihood decoding that is more efficient than the exhaustive method.
Secondly, there's an interesting link to system theory.
I won't make a great deal of this in this course.
But what are we really doing here?
We're representing a block code as a finite state system.
In fact, a linear block code is a finite state linear system, although again the linearity is not terribly transparent from this picture.
So we have a linear finite state system, just as we did for the convolutional code.
However, it's necessarily time varying.
It couldn't possibly be shift invariant because we only have a finite time axis, if you like.
Time axis only goes over four time units in this particular picture, where I've grouped two outputs at a time.
So that's interesting, and for our purposes an interesting aspect of this is that we now get a notion of how complex is a block code.
So far we've been focused on parameters like n, k, d, which are all algebraic parameters of the code itself.
But we really want to know about performance versus complexity.
So how complex is the 8 4 4 code?
Well, now we have a little bit of a handle on it.
We can say its complexity is that of a four-state machine.
So trellis complexity gives us a measure of the complexity of a block code so that we can, for instance, say the Reed-Muller codes tend to be less complex than PCH codes.
And I'll try to give some substance to that as we go along.
So we get additional parameters that have more to do with what we really care about, which is decoding complexity.
And finally, I introduce this subject because it's a stepping stone to where we're really going, is this notion of codes on graphs, which is the underlying concept for the capacity-approaching codes that is our goal point.
Our whole goal in this course is to get to capacity, specifically on the additive white Gaussian noise channel.
And the way that people have found they get to capacity turbo codes, low-density parity check codes, the underlying concept that I'm going to be framing that in is codes on graphs.
And this is kind of a code on a very elementary graph.
So it's a good way from here to there.
So that's why we're taking a little time to look at this subject.
Even though as I say, at the end of the day, even though there's a better way of decoding block codes than the ways we've had previously, and a better non-algebraic way, it still is not going to turn out to be as good as convolutional codes, which we already know about.
Any questions on what we're doing, motivation and so forth?
Yeah
[INAUDIBLE] convolutional codes is that each node doesn't have two things coming on to it?
That's right, it's a little bit more irregular, and this trellis section doesn't look the same.
We had a completely mixing trellis section for the convolutional code.
This kind of divides it into two halves which don't meet.
This basically expresses the code as a certain subcode, and its coset is down here.
So, it's not quite the same.
Looks different.
Maybe this will turn out to be interesting.
Good observation.
Anything else at this point?
You just said that this is not a convolutional code.
Or maybe we can have more than one states, and it is a convolutional code
Let's see, it's not a convolutional code.
It's not even a terminated convolutional code, because it doesn't have the simple shift register type of trellis diagram.
At least it's not the kind of convolutional code we know about.
But why should it be?
Let's open up our minds to more possibilities.
We don't really care.
The objective that we have in mind here is we're just going to try to come up with the most efficient trellis picture that we can for block code.
And we're going to use the linearity, and we're going to basically try to find as few states as possible at each time.
That's going to be our measure of efficiency.
That's not the only one you could think of, but it'll turn out that any notion of efficient representation comes down to the same thing.
So we'll focus on a minimal state representation of a block code, and you can think of this as an exercise in system theory and minimal realizations of a linear system.
This you can view.
A set of all 16 code words you can view is the set of possible trajectories of a linear system on a time axis of length 8, and we want to find a minimal realization, minimal state realization of that linear system.
So that'll make sense to some of you, and it won't make an awful lot of sense to the rest of you.
So we're after a minimal, in the state sense, minimal state complexity, let's say.
And let's continue to focus on this example, and let's focus on a particular time.
Let's focus on the halfway point here.
Where do the state times occur, by the way?
They occur between the symbol tags.
You can think of a state as being associated with a cut between a certain set of symbols which we call the past and another set of symbols which we call the future, again using just some theoretic temporal language.
So if we make a cut -- let's say I do it over here.
States are based on cuts between a past and a future, so the state actually occurs between the fourth and the fifth symbol here, not at either of them, if you want to draw where is the stated time.
And let's start with this example.
And let's ask if we could find any simpler trellis for this.
In other words, let's see if we can have fewer than four states at this midpoint.
Could we possibly have fewer than four states at the midpoint?
What's the key property of states?
The key property of states is that once you get to a state, that then becomes a summary of all the history that you know about the past, and any past sequence or partial sequence that gets you to this state has to have the same set of possible continuations over here in the future in order for this to be a valid trellis representation.
So maybe it will help if I draw this just focusing on the central time.
There are two ways to get to this central time.
One is to get to what we call the 0 state.
We can get there by 0, 0, 0, 0 or 1, 1, 1, 1.
There's a second state which we can get to again by two paths, which are 0, 0, 1, 1 or 1, 1, 0, 0.
There's a third one, which we can get to by 1, 0, 1, 0 or its complement, 0, 1, 0, 1.
We'll see that each of these involves a four-tuple and its complement.
And this is 1, 0, 0, 1 or 0, 1, 1, 0.
Those are the set of possible past sequences that can get to any of these four states.
There are two of them.
Now what this trellis says is that from this state, regardless of how we got here, either of these two things, there are two possible continuations, which happen to be the same thing.
And the property state has to be that either of these continuations is a legitimate continuation of either of the paths that it takes to get there.
So we now have four possible code words that go through this particular state, that pass through this state.
And down here, similarly, I think what we have is the same set of code words.
There's complete symmetry and so forth.
Let's ask, could we combine these two states, somehow smush them together into a single state?
And the answer is obviously no, because 0, 0, 1, 1 is not a continuation of the all 0 sequence, or of the all 1 sequence.
And 0, 0, 0, 0 is not a continuation of the 0, 0, 1, 1 sequence or the 1, 1, 0, 0 sequence.
The property of the state is the Markov property.
And there are many ways of phrasing this, but this is the defining property of states, is that if two past paths have a future continuation -- I hope you understand this language as I introduce it -- in common, then all their future continuations are in common.
So I have two past paths, let's say.
This is the definition of states.
If we can find such a situation, then we can say that these two past paths go through the same state at the cut time, because then we can smush them together.
You can think of this as our starting from, but suppose we drew all 16 code words, and we drew a -- here would be a 16 state trellis, and the first two elements of it might be 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1.
And we'd ask, can we smush these two states together?
And the answer is yes because of this property, because 0, 0, 0 can also be followed by 1, 1, 1, 1.
There are actually four states that we're smushing together here.
1, 1, 1, 1 and 0, 0, 0.
These are all legitimate code words, and because they're all legitimate, we can basically break this up into a Cartesian product of two paths with two futures, and represent all four of them by a single state.
So that's what states do.
If we want a minimal state diagram, we should do this as much as possible.
It's not very well-defined right now, but we should merge states wherever this property holds.
So this implies they go through a common state, and it's if and only if.
So you could see it very concretely from the trellis diagram.
This is the property that we need.
You can see that for this particular code, we do have the possibility of combining all 16 possible states here into 4, because we can combine them all pair-wise in this way.
This is going to come from the linear property of the code.
That's why everything is so symmetric.
But we can't go any further than that.
So we can conclude from this is that for the 8, 4, 4 code, the minimal state space at the center has four states, has size four.
For simple code like this, you can simply see what all the possibilities are, and this is the best we can do.
So do you get that?
This is fundamental system theory state realization theory.
I don't know where in the curriculum one gets this nowadays.
Probably somewhere over towards the control side, maybe in digital signal processing, but maybe not.
But we're doing system theory here.
We all get that?
Yeah
[INAUDIBLE] properties basically defined, I have two possible paths, and they have one common [INAUDIBLE] data, they have [INAUDIBLE]
Yeah, if they go through a common state, that implies this statement.
Conversely, if we have a set of past paths and future paths that have this property, then we can define a state.
In other words, states are always going to look something like this, with a set of past paths.
We have a set of future paths, and the state is simply a node that they go through such that you can combine any of these paths with any of these futures.
If it's symmetric, you could say the same thing.
If two futures have a past in common, then they have all their paths in common.
But whenever you have a state representation -- let's suppose we have three states there.
Then this is what, just looking at those states and the initial and final states, this is what the code is going to have to look like.
Because property of state is that however you got there, you have to be able to take any of these over here.
Otherwise the trellis is invalid.
If not all of these nine sequences are code words, then it's not a valid trellis representation, or equivalently, it's not a valid state representation.
So if, and only if, we can draw the thing in this way, we get it.
So for a small code, you can just see how much combining you can do.
In particular, you can ask, what can the 0, 0 state -- 0, 0 path sequence -- what can that be combined with?
Well, it can only be combined with sequences that have a 0 first part.
So let me introduce now the idea of subcodes.
We have a linear code C. Let's define an interval on the time axis: k, k prime, whatever.
The subcode C on that interval is the set of all code words.
Elegant way to say it is whose support is in k, k prime.
In other words, which are all 0 outside, which have all 0 symbols outside of this interval that we've identified.
So that's just notation.
So specifically, let's say the past code with respect to a certain time, like the midpoint there.
Well, we can define that as the set of all code words that are 0 outside the past, the first four symbols, and we define the future subcode as the set of all code words that are 0 outside the future.
For this code, what is that?
We list the 16 code words.
We make this the past.
We make this the future.
What is the past subcode here?
In this particular case, the past subcode is, of course, the all 0 sequence.
It was always in this code or this sequence, 1, 1, 1, 0, 0, 0.
It's a linear code of dimension one, and it consists of these two code words.
It is precisely the set of all code words that can be followed by the all 0 sequence.
So I can read it directly off of this trellis picture here.
So everybody with me?
The past subcode just consists of those two words.
The future subcode consists similarly.
Looking at the trellis, it's the 1's that are all 0 outside the future.
In other words, can follow a past which is all 0.
In other words, it's these two code words.
So here are two little subcodes with a code.
So by definition, this is subcode of C. If C is linear, you can quickly convince yourself this is a linear subcode, so it's going to have a certain dimension.
It's going to have size equal to a power of two.
It's even going to have a minimum distance that is at least as great as the minimum distance of C, because it consists of code words of C, so its minimum non-zero weight is going to be at least as great as that of C. So it has some properties immediately.
These past and future codes seem to have a lot to do with the structure of the trellis up here.
In fact, the zero state consists of precisely -- the sequences that go through the zero state here are not coincidentally the set of all past code words in Cp followed by the set of all future code words in Cf.
And why is that?
It's because the set of all ways of getting to here, getting to this state, have to be the set of all code sequences that can be followed by the all 0 sequence.
Similarly, the set of all continuations from this state have to be the set of all continuations of the all 0 sequence.
So the zero state is always going to be a little sub-trellis that is going to represent in effect -- you can draw it as a sum, or a product, of the past and future subcodes.
In other words, there's a two-dimensional code that has these two generators.
g1, g2 generates a little two-dimensional code.
Here's the fourth code word in it, the all 1 code sequence.
And every element in that two-dimensional code goes through the zero state
[INAUDIBLE] code word.
So then you know --
It's whatever it happens to be
Then you have a separate state just for -- you could combine them
It doesn't matter.
Suppose that Cp has dimension one.
Cf has dimension one.
That means Cp is going to be something like this.
Cf is going to be something like this.
Whatever, what way
[INAUDIBLE]
It just follows from linearity.
And clearly, since C itself was linear, we're allowed to add x, x, x, x all 0's to y, y, y, y.
That's a code word, because these were both code words in C. So the linearity allows you to fill in this fourth corner of the rectangle, if you like.
So Cp plus Cf is always going to look like this.
And we'll always call this the zero state, the state that you get to by the all 0 sequence.
And that can be followed by the all 0 sequence.
That's always going to be called the zero state.
By linearity, it's always going to look like that.
It might not be dimension one.
It could have any dimension here.
So we're really beginning to get somewhere now.
What do all these other states look like, grouped theoretically?
Cosets.
Somebody -- who said cosets?
Good, that's right.
So Cp plus Cf is itself a subcode of C, a two-dimensional subcode.
So there are four cosets of Cp plus Cf in C, and we'll see if they correspond to the four states.
So how do we do this algebraically?
Let's draw a generator matrix for C in a certain form.
Again, I'll draw the past, the future.
So that's the only division I'm concerned with right now.
And let me draw it in general form first.
I've defined this past subcode, Cp, at a certain dimension, so it has a certain number of generators.
So we're going to put up here a set of generators for Cp, g of Cp, however many there happen to be.
And what's their common characteristic?
They all are all 0 in the future.
So they're all going to look like that.
Then I'm going to take a set of generators of Cf, and similarly I'm going to use them up over there.
These are all code words, and they're clearly linear and independent of those.
So I'm on my way to constructing a generator matrix for C. But obviously I'm going to need some more.
We can call this the number of generators in the past, the number of generators in the future.
This is the dimension of Cp.
This is the dimension of Cf.
And now I need some more generators, which by definition are going to have to span both past and future.
Let's see.
Since I've already introduced the coset language, let me just mysteriously put that as the generators of a quotient group.
C mod Cp and Cf is what that means, and it's a quotient group, if you know or recall what a quotient group is.
But anyway, there are k minus kp minus kf of these.
And I don't particularly care what I put down in here.
Their property is that they have to span both past and future.
So for our particular example, here's the generator for the 8, 4, 4 code.
What we're talking about is a generator that looks like 1, 1, 1, 1.
That's k past is 1.
0, 0, 0, 1, 1, 1, 1. k future is 1.
They're both one-dimensional codes.
Now I need two other generators, which must span past and future.
So what'll I take?
Let me just take two more.
1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0.
That would be one possible choice.
And so this is k minus kp minus kf, which is 2.
What I'm going to claim is that I'm going to need a state for every possible linear combination of these generators.
So claim -- well, let me break it down into more logical sequence.
Every code word in C can be written as some linear combination of the generators.
What does that amount to?
It amounts to a certain code word in the past code plus a certain component in the future code, either all 0 or they're all one, plus -- let me call this the state code, S, as I do in the notes.
So in this case we have a two-dimensional state code.
Its elements are eight-tuples.
It's just a subcode of the code we started with, so it's not quite clear how it corresponds to states yet.
Well, let me just call that S for state sequence.
And I claim I can break this up because the proof is that for every c in C is equal to a sum of -- it's a linear combination of the generators.
So I just break that up into the three possible parts.
So I get one part that's in the past subcode, one part's in the future subcode, and one is in the state.
So this is the past, this is the future, this is the state.
So I just break it up according to this.
So the 16 code words can all be written in this way in our particular example.
So let's project this code word on the past.
What does projection mean?
It means you just don't look at the part that's in the future.
So we're looking at four-tuples now.
So C projected on to the past, this is the part that lives over here.
It is equal to -- well, if I project the past code word on the past, I basically get the past code word again.
Let me write it like that, but it's a little bit redundant.
Incidental comment: I can regard this past subcode as either an 8 1 4 code, but what is it really?
It doesn't live out here.
It's support is on these four.
It's really a 4 1 4 repetition code that lives on the past.
It's support is the past.
Similarly, the support here is the future, and it's effectively a 4 1 4 repetition code for our example.
But formally, if we project this on the past, it's a one-to-one projection.
What happens to the future part?
This disappears, because the projection of anything in the future code on the past is all 0 by definition.
Plus, it's the projection of the state sequence on the past.
Could the projection of the state sequence on the past be all 0?
This is actually an important point.
Could the the projection of any linear combination of these generators down here be all 0 in the past?
The answer is no by the definition of the future subcode.
If I found a linear combination of this that was all 0 in the past, I should add that to the generators of the future subcode.
So by defining this in this way, I've forced this to be nonzero, unless, of course, the state sequence itself was 0.
The all 0 sequence is in the state code.
So this is nonzero if S is nonzero.
And similarly, if I project a code word on the future, I get -- this becomes all 0.
I get the element of the future code projected on the future, plus the state on the future, the state projected on the future.
And now I claim further that this means I can draw a trellis as follows.
Start from here.
For the all 0 sequence, I'll have a bunch of parallel paths going to the zero state that together add up to the past subcode.
These are precise.
One of these is going to be all 0, and these are precisely the ones over here that can be followed by all 0, as I've already claimed.
So these are the ones that can be followed by 0, and similarly over here I'm going to put all the future subcode.
I will say these correspond to the state sequence 0.
These are the ones where if I put 0 coefficients down here, I just get linear combinations of the past subcode and the future subcode.
I'm going to write those as going through all one trellis state.
And I believe I've already made the argument that this state at least is legitimate, that every combination of something in the past subcode with something in the future subcode is a code word.
This is simply just Cp plus Cf, what I wrote before.
So I've got one state that represents all of Cp plus Cf.
And now let's take another typical state down here.
We'll say this state corresponds to the state sequence S, or S is a general state sequence.
Think of it as being nonzero.
And what am I going to put on that?
I'm going to put Cp plus this state sequence projected on the past.
And that way I'll get all past projections that are of this form for a specific S projected on the past, allowing this to vary through the past subcode.
And similarly over here, I will let the set of all these trellis branches be Cf plus the state projected on the future.
So all the things that go through this state here will be Cp plus Cf plus this particular state sequence.
So for each of the state sequences, I claim I can define a state.
So it's the set of all past continuations of anything that projects on the past as anything in the past subcode plus a past projection of the state sequence can be combined with anything in the future, Cf plus the future continuation of the state sequence.
One of these is simply the past projection of the state sequence plus the future projection of the state sequence.
And this is S, and by construction this is a code word in the code.
Now by linearity I can add any past code word to this past projection.
And that's a code word, so any of these elements plus Sf, continued by Sf, the future projection is a code word.
See, there we're taking something generated by Cp and the state code.
And similarly, anything by here comes out here.
So I think I've left one or two details undone, but are you convinced that I can define a state in this way such that all of these pasts can be followed by all these futures, and they're code words?
In fact, they correspond to this subset of elements of the code.
We have examples of it up here.
For instance, 0, 0, 1, 1.
Or let's see.
Let's take a past protection, 1, 1, 0, 0.
Here's a typical state sequence, 1, 1, 0, 0, 1, 1, 0, 0.
And I can add to this anything in the past, so I get 0, 0, 1, 1, 1, 1, 0, 0.
I can add to it any of these two, anything in the future.
So I get 1, 1, 0, 0, 0, 0, 1, 1 and 0, 0, 1, 1, 0, 0, 1, 1.
And I claim that all four of these have the state projected on the past equals 1, 1, 0, 0.
The state projected on the future is 1, 1, 0, 0.
And that any of these pasts can be followed by any of these futures.
So this is C past plus C future plus 1, 1, 0, 0, 1, 1, 0, 0.
That is the claim.
And furthermore, if I go through this, dot dot dot, I'm going to generate everything in C. So this is the partition of C -- into what?
S cosets of Cp plus Cf, subcode of C. So algebraically, that's what's going on.
But having done this partition, at least with respect to this state space, I can create something which has S states the size of S, which is -- what is this?
The dimension of S is the dimension of C minus the dimension of Cp minus the dimension of Cf.
So I've argued that I can get a state space with this dimension.
It's a linear state space, and it has a certain dimension, which is just the dimension of C minus the dimension of the past subcode minus the dimension of the future subcode.
It's the number of generators that I need for S. So in the example, I need two generators for S, so I get a state space of dimension two, or size four.
It's just a vector space over F2 in this case.
You all with me?
I think you are.
I don't see any great puzzles.
So I can get at least these few states.
Could I get any fewer?
Can I possibly merge any two of these states?
Could I draw a trellis in which, say, I've mushed the zero state together with one of these nonzero states?
And the answer to that is clearly no by the definition of the past and the future subcodes.
These are all of the future sequences that can follow the all 0 past sequence.
There can't be any more down here, so we can't possibly mush them together.
And similarly, just by going through this, if you can't take the past part of one state and combine it with a future part of another state sequence, because that clearly is not in the code.
And I don't have a slick proof of that in mind at the moment, but there is one in the notes.
The conclusion from this is what I call the state space theorem.
Given code C, any partition of the time axis, the total index set, into past and future, I can do this not just at the midpoint.
I can do this any point along the code that I want to, any partition into past and future.
The minimal -- we get a linear state space, in other words a vector space, S. And the dimension of S is equal to the dimension of C minus the dimension of this past subcode minus the dimension of the future subcode.
So we can simply calculate what the minimal dimension of a state space is at any point along here.
So for instance, at this point here, what's the minimal size of a state space?
What's the past subcode here?
What's the set of all code sequences that are all 0 in the future, as we have all 0's out here?
This point, Cp, is just 0, 0.
It just has dimension 0.
Yeah, 0 What's the future subcode?
This is the subcode which has support all out here.
This is anything that starts with 0, 0.
This is 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1.
Let me just write down generators.
1, 1, 1, 1, 0, 0.
So there is -- this is dimension Cp equals 0.
Dimension Cf equals 2.
And so the minimum size of the state space is 4 minus 2.
That's again 2
Does this mean that [INAUDIBLE] each state's the number of states is the same?
Certainly not true in general.
Let me ask about time three.
Can certainly do that.
Time three, what's the past subcode?
This is the code words that have support on the first three symbols.
Again, that has to have dimension 0, because the minimum weight of this code is 4.
So there can't be anything that just looks like 1,1,1 and then five 0's What is the future subcode for that?
Now the only generator of the future subcode is 0, 1, 1, 1, 1.
We have to go this far, and then it's what futures can we get from here?
There are only two words that are in the future.
So in that case we have dimension past is 0.
Dimension future is 1.
So if we drew a state space here, we would have to have eight states.
In fact, I can do that if I just draw a little state in the middle of each of these lines.
I get now a trellis picture which has an explicit state at this third time here, but it has eight states.
Oo ugly, so I've masked that.
I've suppressed that by this nice little four-state trellis.
But yeah, if you insisted on drawing a state space at time three, it would have eight states in it.
We'll get back to that when I get to this turn-the-crank procedure, if I do, of getting to minimal trellises.
So what have we done?
We've established that there is a uniquely defined minimal state space size for any partition between past and future.
In other words, if there's any time where you want to make a cut in the time axis between past and future, you can define what the minimal state space size is.
And now you might ask the question, can we simultaneously achieve minimal state spaces at all times?
In other words, can we draw a single trellis which gets the minimal state space at each time?
We haven't proved that there is.
It might be that if you push in the state space at one time, it forces the state space to balloon out at another time.
And in fact, for nonlinear codes, that is typically what happens.
But for linear codes, a minor miracle occurs.
And the answer is yes.
So how am I going to prove that?
I'm going to prove it via this very handy tool, which is also a construction tool which is called a trellis-oriented, or a little bit more formally, a minimum span generator matrix.
The idea here is, given a generator matrix, reduce to trellis-oriented form.
We're going to prove that this is unique, or effectively unique, and specifies minimal trellis at all times in a certain way.
Let's start again.
So loosely, what is a trellis-oriented generator matrix?
Well, if we have a generator, say 0, 0, 1, 1, 1, 1, 0, 0, its span is this interval from its starting time to its ending time.
It's this.
We say it starts at time 3, it ends at time 6, the span is the interval 3 to 6 in this case.
So we think of this as not being active before time 3.
At time 3 we get the first interesting thing happens.
A 1 comes up, but then goes through something.
These don't have to all be ones.
It could be 1, 1, 0, 1.
Then there's a last 1 at some point, and then it quiets down, and it's dead again
[INAUDIBLE]
Yes.
Except I'm not sure.
Would you say the support of this was 3, 4, and 6, or would you say it was the interval from 3 to 6?
I don't know.
What's the definition of support?
I don't know it
I'm not sure I know.
It's probably been defined both ways in different literatures.
So I'll call it a span, but it's roughly the support of the interesting part.
So that's the definition of span.
And what a trellis-oriented general matrix is, a minimal-span matrix, meaning all generators have a short a span as possible.
I'll just say, are as short as possible.
This is a lecture.
I can be loose.
Now it's not even clear that's well-defined yet.
But let me again give you an example, and show you how to find a trellis-oriented generating matrix just by example.
Let me take our example here, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1.
So there is a generator matrix that's not in trellis-oriented form.
In particular, I can see that by adding this to this, I can make this generator have shorter span.
I can replace it by one, which is four 0's and then four 1's, which we've already found useful in constructing a trellis.
But let me proceed through it more systematically.
How can I make -- this generator generates all words in the code.
So how can I find a set of linearly independent generators that has shorter span?
There's the sort of greedy way of proceeding.
Let's take the first two generators.
Can I combine these to find anything that has a shorter span, that I can replace one of the generators with?
Anybody?
No
No, I can't.
That's not the answer I expected to hear
[INAUDIBLE]
Let's identify what the spans of these are.
That has a span of four.
That has a span of six.
That has a span of seven.
This has a span of eight.
What I'm trying to do is to reduce that
[INAUDIBLE]
So if I add the first two, that seems like a good thing to do, because then I get 0, 0, 1, 1, and I've reduced the span of that generator.
Any way I can combine these two to reduce the span?
Obviously not, because the span of this is completely included in the span of this.
If I add these two together, then I'll get something which still starts here and ends here.
Why was I able to reduce the span here?
Because the spans had the same starting time, so I got a cancellation of the first bit.
I got a 0 up here.
That's what I wanted.
So that's the key to proceeding.
If I see any place, any two that have the same starting time, then I can add them together, and the result will be something that has a shorter span than at least one of them.
So let me do that.
I see two here that have the same starting time, and so I add that to that, and I get 0 1 0 1.
So I've got the span down to here.
I can do that down here.
And I get this, which is a very nice reduction in span.
Can I go any further?
I can clearly look for the same thing in ending times, if I could find two ending times that were the same.
I could do the same trick, add those two generators together, and the result would be something that was shorter than at least one of the component generators.
But here are my starting times, here, here, here, and here.
They're all different.
So I have no possibility for combining two and getting a better starting time.
Similarly, the ending times are here, here, and here.
They're all different.
So I'm done.
So, that's how you find a trellis-oriented generator matrix.
And there's a closely associated theorem, which is that a generator matrix g is trellis-oriented if, and only if, all starting times differ, all ending terms differ.
Yeah?
What's the definition of all generators as short as possible?
I'm now making it.
This is really just motivation.
So this can be my actual definition of a trellis-oriented generator matrix, which I've proved this theorem.
It's a matrix in which all the starting times are different, and all the ending times are different.
And what I've achieved is that all generators are as short as possible.
How would I go about proving the theorem?
Suppose I have a matrix that's not in this form.
And I've already proved that I can shorten the generators.
So if it's not in this form, it's not trellis-oriented.
The generators are not as short as possible.
So all I need to do is prove the other side.
Suppose it is in this form.
Then could I possibly find shorter generators?
And the proof of that is basically the following lemma, that the subcode C, take any interval kk prime.
So given a trellis-oriented generating matrix, namely one that satisfies these conditions, then the subcode consisting of all the code words who have support on this interval is generated by the gj in G that have support in kk prime.
And that's kind of intuitive and obvious.
On the one hand, it's clear that any linear combination of these generators is a code word in the subcode.
Obviously they all have support in this interval.
I can't generate anything that has support outside that interval by linear combinations.
So the only question is, can I possibly get anything that's in the subcode by combining with a generator that is outside this interval?
And again, it's clear that say I take kk prime to be here, if all the starting times are different, and all the ending times are different, and I start to add generators that are outside here, I can't possibly get cancellation of the starting time, or I can't possibly get cancellation of the ending time.
So I'm going to wind up with a code word which has support outside the interval.
So, it really is a simple lemma
[INAUDIBLE] have the all 0 code word?
Excuse me?
The all 0 word will be part of the subcode
The all 0 word is always part of the subcode
[INAUDIBLE] who has something outside this interval.
Then when you add that with the all 0 one, you would get something that does not belong to the subcode, [INAUDIBLE]
Right, but I have to ask if there's any code word.
So I have to consider all the code words, really.
But I'm saying if I have a code word C that is sum of the generators, and I have a nonzero coefficient here on any generator that has starting time outside this interval or ending time outside this interval, then I can't possibly get cancellation of that nonzero thing.
And so if it really requires this to be nonzero, and then I'm going to say that I get a code word that's not in this subcode.
So that's very quick, intuitive sketch of that proof.
Again, for writing it out, see the notes.
But this lemma then proves this theorem, because it now says I can't -- if all the starting times are different, and all the ending times are different, then I can't possibly get shorter support for any of the generators.
And I don't think so in a totally convincing way, but that basically is the point here.
Yeah?
[INAUDIBLE]
Well, now I'm defining this.
So, meaning that all the starting times are different, and all the ending times are different.
From now on that's going to be my definition of a trellis-oriented generator matrix.
So, saying that this property implies this property.
But now this lemma has an interesting consequence.
This implies for any past and future, Cp is generated by the generators in a trellis-oriented generating matrix with support in p. And similarly, Cf is generated by the generators with support in f. So for any p and f, I get a picture that looks like this.
I can draw now this boundary anywhere I want, and if the matrix was trellis-oriented, then I'm going to get a certain subset of generators that live in the past, and they'll be the generators in fact of Cp.
The dimension of them will simply be the number of generators that live in the past for that definition of the past.
There will be some other set that live entirely in the future.
Those will be the ones that generate the future.
And the ones that are neither wholly in the past or in the future will generate the state code.
So that says I can read off the dimensions of the state spaces just by looking at this single trellis-oriented generator matrix.
So now let me very quickly go through the calculation that I went through a little bit laboriously over here.
Suppose I make the whole thing the future.
In other words, I draw a cut before time 0.
Then the state space has dimension 0.
All four generators live on the future.
If I make the cut here, the state space has dimension 1, because three of the generators live on the future.
One is active at time 1.
So that means I'm going to get a two-state trellis at time 1.
If I make the cut here between past and future, I see two generators live on the future and are inactive, haven't started yet.
But two of the generators have already started.
At this time we have three active generators.
Only one is still completely on the future.
At this time in the middle I have two generators which -- past subcode, future subcode, and two that aren't.
So I get this form that we looked at over there.
And I get a state space here generated by these two generators of size two.
So just by looking at this I go right through the picture, and I find the size of the state space at each time.
So this is an algebraic way of finding the minimal state.
So now I can draw from this a trellis just by writing down these state spaces.
Let me leave these trellises here.
Let me draw now a full trellis, or an eight-section trellis.
I'm only going to put one bit on each trellis.
Here is the starting state, dimension 0.
One state.
From that I can go out to 0 or 1.
I won't label the state spaces.
Next time, I'm still just branching.
Actually, to make it look like that, I'm going to want to come down here, 0, 1, 1, 1, 1, 0.
I basically just have four states at time two corresponding to whatever the first two bits are.
They all go to different states, because all these have possible future continuations.
Then I have eight states at this time, still just branching.
Sorry, this is going to take some time to do correctly.
Add this time, then I come in here.
0, 1.
Anyway, and it's symmetrical on the other side.
So that's what a full trellis will look like.
And how did I do this trellis in principle?
I wrote down all 16 code words.
I wrote down what linear combinations they were up here, and I therefore found what states they went through at each time.
And then I just drew the graph that goes through that describes those trajectories.
So I wrote down all the code words.
I wrote down all these state codes at all these times.
I could have written them down in any order.
And then I just connected the dots according to which state -- I make this calculation as to which state they go through.
That's a good place to stop.
Let me summarize.
We now have a method, given a generator matrix for a binary linear block code, of reducing that generator matrix to trellis-oriented form.
Just by inspection of the trellis-oriented generator matrix, we can determine what the state space dimensions are at each instant of time, at each possible state cut.
And we can then draw a trellis which achieves that minimal state space dimension, or minimal state space size, for every moment of time simultaneously.
And I assert that this trellis is the minimal trellis in every respect, whether you're trying to minimize state complexity, or if you're using the Viterbi algorithm.
Really what you want to minimize is the number of branches, but there's a calculation about branch spaces in the notes that shows that this also achieves the minimal branch space size at every time, regardless of how you draw the trellis.
More elaborate calculations with a more refined notion of Viterbi algorithm complexities still come up with the same result.
The result is there is essentially a unique minimal trellis, where it's minimal in every way.
And this comes from the linear, or more broadly, the group property of the code.
So for any linear group code, there's a well-defined, essentially unique up to re-labeling, minimal trellis that achieves the minimum of whatever complexity quantity you want to define.
So in this sense, the trellis complexity of a group or linear code is very well-defined.
It's defined by this minimal trellis, or by its parameters, and we can use that as a measure of the complexity of the corresponding group code.
And from an engineering point of view, once we've got this minimal trellis, we can use the Viterbi algorithm to do maximum likelihood decoding.
Just in the same way we did for convolutional codes, sweep from the starting node to the ending node.
Digest that, and we'll come back and talk about it again.
Do a few more details on Monday, but that's the basic idea.
The topic of this chapter is finding trellis representations for block codes, in particular linear block codes.
We're looking only at binary block codes, except one of the homework problems looks at MDS codes over ground field of q.
The principles are the same for general Fq or in fact for group codes.
We're looking for minimal trellis representations.
And so just to review our line of thought, basically we proved a basic theorem called the state space theorem that tells us that there is a vector space over the ground field at any time which we can identify with the state space.
It has a certain dimension.
And we got a little formula in terms of subcodes for the dimension of the state space.
We set up a little generator matrix where we isolated the past part of the code with respect to a certain cut time future part of the code, generators for that.
We subtract those generators out.
What's left?
The number of generators is the number of generators it takes to generate the state space, in other words the dimension of the state space.
So we got a theorem on the dimension of the state space at any time.
And then we found a nifty little algorithm to find a -- to determine the dimensions of the state spaces at all times, the minimal state spaces.
And in fact we can use this algorithm to construct a minimal trellis realization.
The algorithm involves a trellis-oriented generator matrix.
And the idea here turns out to be very simple.
All you do is you find any generator matrix for your linear block code, and you reduce it to trellis-oriented form.
Trellis-oriented form means that the generators have as short a span as possible.
It's a minimum span generator matrix.
And we do that just in a greedy fashion by any time we have an opportunity to add two generators together such that either their starting position cancels, or their end position cancels, we do it.
And out of that we get a shorter generator, and we can replace one of the two generators in the combination by that.
We can keep shortening everything up.
The algorithm must terminate at the point where no further shortening is possible.
And that is explicitly the point at which all the starting times of the generators are different, and all the ending times of all the generators are different.
For example, here's a trellis-oriented generator matrix.
The starting times, the spans of the generators, are like this.
The active times, starting times -- there's the starting time of this one, and its ending time.
Here's the starting time of the next one, and its ending time.
The starting time of the next one, and its ending time is here.
And the starting time of this one, and its ending time is here.
We're not always going to find that the starting and ending times are disjoint.
This happens, it turns out, because this is a self-dual code.
In a self-dual code, you will always get this each time, either a starting time or an ending time.
But that's not a general property.
For instance, another example that we've used is, say, the 8 7 2 single parity check code.
What are its generators?
Its generators look like this, dot, dot, dot, down to 0, 0, 1, 1.
That's a set of trellis-oriented generators with starting time, ending time, the first one here.
Starting time, ending time of the second one there.
So in general for this one, every time is a starting and ending time up to here, where every intermediate time is both the starting and an ending time.
So it could happen that way, or another example would be the 8 1 8 repetition code.
What's a trellis-oriented generator matrix for that?
It only has one generator.
It looks like that.
It has a starting span as the whole thing.
The starting time is here.
The ending time is here.
So for general codes, the starting times and ending times could be anywhere.
There will always be k of them.
And otherwise, there's always one at the beginning.
If it's a nontrivial code, there's always an ending time at the end, et cetera.
But this particular behavior is for self-dual codes.
Anyway, having done that we can now read off the dimensions of the state spaces by simply looking for how many generators are active at any particular time.
For a state space, we're always looking at the times between the symbols, cut times.
We could in fact draw a trivial one there, but we're looking between here.
How many are active at this time?
One.
So dimension of the state space at state time k is 1, 2, 3.
This point, we come back to 2 again.
3, 2, 1, 0, 0 at the end.
It's always zero at the ends because at that point everything's in the future, or everything is in the past.
So these are trivial state spaces.
These are the nontrivial state spaces in the trellis.
And in fact, you remember we had a little picture.
We can draw this complete trellis out with the state spaces of size 1 2 4 8 4 8 4 2 1.
So this is called the state dimension profile, just this set, this ordered set.
Or the state space size profile is 2 to the dimension, of binary codes, whatever.
And similarly down here, if we look at state space sizes, we see the dimension is going to be one at every time.
So in this case, the dimension of the state space is 0, 1, 1, 1, 1, 1, 1, 1, 1, 0.
And similarly down here, we look at the cut time.
It cuts one active generator every time.
So it's 0, 1, 1, 1, 1, 1, 0.
So we get at least the state space dimensions just by inspection of the trellis-oriented generator matrix.
Question?
[INAUDIBLE]
We proved it in several steps.
First we have the state space theorem, which I gave to you in the following form: the dimension of the state space at time k is equal to the dimension of the code minus the past subcode.
Let me write that.
The dimension of the past subcode minus the dimension of the future subcode at time k relative to this cut time.
And we proved from the trellis-oriented generator matrix that the past subcode is generated by all the trellis-oriented generators that have support on the past, and the future by all the ones that have support on the future.
So, it's simply we take those out.
And what do we have left?
We have the ones that are active at time k, the ones that are neither wholly supported on the past nor on the future.
They're supported both on the past and future.
And those, that's what this difference is.
So explicitly we have -- the code is generated by all generators.
That's all k generators.
Cpk is generated by trellis-oriented generators supported on the past.
And Cfk is generated by the trellis-oriented generators supported on the future.
So the difference -- state code sk is generated by the trellis-oriented generators not supported on past or future, or active at time k. Probably good to write all that out at least once.
Yeah?
[INAUDIBLE] self-dual codes all have such properties that each time is starting and ending?
Yes
Why is that?
Why?
It's a duality theorem, and we're not doing many duality theorems in this course, so I'll simply tell you
[INAUDIBLE] has this property, it must be a self-dual
I don't think that's true.
It's possible that's true.
Duality theorems are very strong, but I don't think this is a sufficient property.
For instance, suppose I just changed one of the generators to look like that.
It still has disjoint starting and ending times, but now I don't think the code is self-dual.
So here's another code generated by these four generators, and I don't believe self-dual
[INAUDIBLE]
I'm guessing that it isn't.
To prove it I would compute the dual code, and I would find that it wasn't same code, didn't have the same code words.
But I'm guessing that if I just make random changes in the interior here, I'm not going to get a self-dual code, because that's a very strong structural property.
But I'm not going to take the time in class to do that.
But these are good questions.
The whole subject of duality is a wonderful and elegant topic that we just don't have time to do much in this course.
If I did it another time, I could spend many lectures on duality.
Some of the homework problems have suggested what you might be able to do in orthogonal codes, but -- well, one of the problems you'll do on the homework -- I was going to talk about it a little later.
There's a dual state space theorem that basically says the state space sizes, or dimensions, are the same for the dual code as they are for the primal code.
Here's one example.
Of course, dual is itself, so it's trivial that the state space size is the same.
But here's another less trivial example.
These two codes are orthogonal to each other.
And what is similar about their trellises?
Their trellises are both two-state trellises.
They don't look the same.
This trellis looks like this, because they're all zeroes up here, and all ones down here.
That's the repetition code trellis, whereas this trellis looks like this.
It has crosses every time and so forth, abstractly.
So trellises are not the same.
They have different branching properties, but the state space sizes are the same at all times.
This is a general property of dual codes which you will prove on the homework, if you haven't done so already.
Sorry, it's hard to know what to tell you about and what to suppress, because I'm also trying to cover more material this year.
There's a lot of material that's really nice that I have to skip over.
Let's continue from this.
Let's make a minimal realization.
So, a canonical minimal trellis.
I'm doing this in a little bit different order than it's done in the notes, but I think you will see this kind of argument very quickly.
I've got this trellis-oriented generator matrix, but suppose I didn't know what the trellis looked like.
I put it up the first time last time.
But all I have is this generator matrix, and I want to generate a trellis now for this code which has minimal state spaces.
It's also going to turn out to be minimal in every other way.
Well, let's start off by realizing the first generator here.
The first generator generates a little one-dimensional code.
G1, the code generated by the first generator, is simply the two code words 0, 0, 0, 0, 1, 1, 1, 1, 0, 0.
So let me realize that little one-dimensional code with a minimal trellis for that.
What would it look like?
If I go through this, I'll see that it has -- my trellis-oriented generator matrix for this code is simply 1, 1, 1, 1, 0, 0, 0.
I see from that that the dimensions of the state spaces is 0, 1, 1, 1, 0, 0, 0, 0, 0.
And you can easily see that what the trellis, the minimal trellis, a minimal trellis looks like is 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0.
So there is the minimal trellis realization for that one-dimensional code.
It has state spaces of dimension 0, 1, 1, 1, 0, 0, 0, 0, 0.
Therefore the state spaces are all minimal, as proved from this.
It's all sort of elementary.
Similarly, I can realize this one by another little machine that looks like this.
0, 0, 0, 0, 0, 0, 0.
This is just the state transition diagram of a linear time-varying finite state machine.
And it looks like that, and it too is minimal by the same argument for the one-dimensional code generated by this generator.
What is this saying here?
This is saying we really -- what we should imagine here is we have a little memory element that's only active at state times 1, 2, and 3.
So to realize this, this is a one-dimensional memory element.
Think of it as a little -- well, a memory for an element of f2.
It's active during the interval 1 through 3, during the time that this generator is active.
It's active at these three times, not otherwise.
So you can think of it as just disappearing.
It comes into existence at time 1.
It holds whatever the input bit is.
Think of -- we're trying to create the code as the set of all -- sum of ui gi.
And so what this memory element really does is it holds ui during the time that it's needed, which is at times 1, 2, and 3.
Whatever the coefficient is -- what could it be, 0 or 1?
Element of the base field, f2.
So the memory element holds, it contains u1 in this case.
Yeah?
What if we look at this matrix only for the second code
Yes
So that will give 0, 1, 0, 1, 1, 0, 1, 0
Correct
So some places it will suggest that the dimension is --
The fact that the values are -- there will always be 1 here at the starting time and the ending time.
Otherwise we wouldn't draw a branch there.
But they can be arbitrary in between.
So don't be thrown off by the fact that there are some 0 values in here.
What we're doing here is we're realizing u2 times g2, where u2 can be either 0 or 1.
And g2 can be arbitrary, except that it has a starting time and an ending time, so it's not arbitrary for this kind
What if we try to find the state space [INAUDIBLE]
For this, what I need is a one-dimensional memory element containing u2 which is active during a different interval, 2 through 6.
So think of these as like little stars that wink on or wink off.
This thing comes into existence for three consecutive times, and then it disappears.
This thing comes into existence for five consecutive times, and then it disappears.
Because those are the only times when it's needed to contribute to the output.
So altogether, we can think of there as sort of being four memory elements which contain u1, u2, u3, and u4.
And we form the outputs g1k times this, sort of like in the convolutional code.
g2k g3k and g4k.
The generators step along in time, and we form the outputs.
We sum this all together in a summer, and this provides yk, the output, as we go from k equals 1 through 8 as we go through the code word.
But we only need this one for -- it's only on from time 1, 2, and 3.
And this is on from time 2 through 6.
And this is on from time 3 through 5.
And this is on from time 4 through 7.
Because we don't need it any other time.
So you can think of an input coming in at time 1, at the starting time of the first generator, being saved for as long as it's needed, then it's discarded.
And in fact we don't need that memory element anymore.
How many people are getting this?
Raise your hand if you get this
[INAUDIBLE]
It's a time -- what I'm trying to do is draw a time-varying machine.
So how do I do that?
At time 1, it looks like this.
Time 1, it just has u1, and we multiply it by g11, which is 1.
And that gives me the output at time 1.
There's nothing else that contributes.
At time 2, we've got now another input, u1 and u2.
And we've got to sum the contributions from both of those, times their respective coefficients.
At time 3, we're up to three of them, and we need all of them.
We're going to form some linear combination according to the generator matrix
[INAUDIBLE]
It's time-varying, yeah.
So get used to it.
For a block code it obviously has to be time-varying.
Obviously if we regard these as impulse responses, this is the impulse response to u1.
It only lasts in terms of symbol times from time 0 1 2 3, for four time units of symbol time.
This is the impulse response to u2.
u2 you can think of as an input that comes in at time 2, and whose output rings for five more times.
So you need to save u2 for five more times, and then it disappears.
We don't need to save it anymore.
We can let it drop off the end of the shift register, if you like.
u3 comes in at time 3, and this is its impulse response.
And u4 comes in at time 5, and this is its impulse response.
Does that help?
How many people are not getting it now?
[INAUDIBLE]
We don't need it here, but we're going to need it again at time 4, so we have to save it.
What are you going to do with it at time 3?
You going to hide it somewhere?
So I still need to save it.
This, again, is linear system theory, except I guess you just don't get taught linear system theory anymore.
But this is what you have to do if you -- before, for convolutional codes, we did linear systems theory for linear time-invariant systems.
They have to be over a finite field, but who cares?
We did general linear system theory.
This is general linear system theory for realizing time-varying systems.
In fact, one comment that might be good to make at this point is that -- now go back and remember what we did with convolutional codes.
We started out with a rate 1 over n convolutional code c, and we found some generator for it.
Basically we can take any code word in the code, and all of its shifts will generate the code.
Any non-zero code word, all of its time shifts generate the convolutional code.
So take any code word g of d in a convolutional code, and g of d, dg of d, d squared g of d and so forth generate the code.
But then we converted it by doing a little polynomial or Laurent series algebra.
We took a rational one and we converted it to a canonical one, g prime of d. Which if you remember, this is the polynomial code word of minimum degree in the code.
We call that degree nu.
What were we doing there?
The shifts of g prime of d form a trellis-oriented generator matrix for this convolutional code.
For instance, for our canonical example, g prime of d looked like this.
Polynomial notation, it's 1 plus d squared 1 plus d plus d squared is the generator matrix.
Let me write that out as 1, 1 at time 0 or coefficient 0.
0, 1, 1, 1, and then all 0's before that, and all 0's after that.
Is somebody sleeping?
Wake him up please, before I turn around.
What?
Oh, all right.
I'm very glad to hear that.
So here is g prime of d written out as bits.
dg prime of d looks like what?
It looks like the same thing shifted over one.
d squared g prime of d looks like this and so forth.
Is this trellis-oriented?
In this case, each time interval contains two symbols.
Here are our divisions between symbols.
We've got a rate 1/2 code, so we chose to take them two symbols at a time.
Where are the starting and ending times?
This one starts here, ends here.
This one starts here, ends here.
Starts here, ends here.
So all the starting times and ending times are different.
Ergo, this is a trellis-oriented generator matrix.
Modulo some hand-waving about infinite sequences.
Ergo, we ought to be able to use the state space theorem to say what are the state space sizes.
Any time I have a cut here, what are the spans?
The spans are like there, there, there, so forth.
So the state space dimension at each time is going to be 2, 2, 2, 2, 2.
And if we go through this construction procedure, what are we going to get?
We're going to get something that at time 0, we need to have in memory, we need to remember u minus 1 and u minus 2.
At time 1, just doing what I did up here, we need to have in memory u minus 1 and u0.
At time 2, we need to have in memory u0 and u1.
And we achieve that just by implementing this with a shift register.
In the time-invariant case it's very easy, and you all understand it.
So exactly the same theory goes through for rate 1 over n convolutional codes.
I will tell you briefly that to treat rate k over n, it's basically just the same thing, except we need k generators starting at each time here.
But in order to get a canonical encoder, we just get a trellis-oriented generator matrix again.
And by the way, modulo this hand-waving about infinite sequences, I've now proved what I asserted, that this encoder is minimal among all encoders for this code, that it has the minimum number of memory elements.
Because from this argument, the minimal state space dimension at each time is 2 once I've achieved this canonical trellis-oriented generator matrix.
By the way, this theory is very general.
It's really all you need to know to do minimal realization theory of linear systems, whether they're time-varying or time-invariant over any field, or over groups.
So that was a parenthetical comment about convolutional codes just to tie it back to what we did in the previous chapter.
I haven't completed what I wanted to say about block codes.
How am I going to do this without erasing something?
Let's go back to block codes.
And I guess I'll move over.
The point that I was in the process of making was that I can realize each one of these generators individually by a little trellis diagram that's one-dimensional during the time that the generator is active, and zero-dimensional state space during the time the generator is inactive.
And think of this as just one component here.
To generate a code word, a particular code word is just the sum.
All I need to do is sum the outputs of all these generators independently.
How many different possibilities are there?
There are 2 to the k possible code words.
Each one of these generates one of two possible code words.
If I take the set of all possible sums of these four code words, I'm going to have 16 possible code words.
So that very quickly was what I was doing when I said, here's a little machine that generates the first code, c1.
Here's a little machine that generates the second code, c2.
Only has two possible code words according to the value of what's in storage.
This generates the third component.
This generates the fourth component.
Sum it all together, and you get -- sorry, I'm calling this y now.
y equals the sum of the ui gi.
You see that?
I hope I've beaten it into the ground enough now.
So it's a linear time-varying machine.
That is a realization of it, if you've understood my description of it now.
And now we can draw the trellis of it.
That what can happen at time 1, we can either have u1.
Now I can label my states here, at this time u1.
u1 can be either 0 or 1.
This is u1.
At time 2, I have both u1 and u2 in my state.
So this can either be 0, 0 or 0, 1, or 1, 0 or 1, 1.
And obviously if I've already determined u1 is 0 here, it's still going to be 0 at time 2.
I label these with their associated outputs.
This is 1.
This is also 1.
This can turn it back to 0.
And I just keep going.
At the third time, I have u1, u2, u3.
I have eight states.
At time 4, I come back.
I only have u2, u3 at time 4, and so these merge.
So I get something -- this doesn't merge like that, though.
It merges down here.
0, 0, 0, 0, 0, 1.
This merges to 0, 1 when I drop the first one.
It's probably worth doing this more carefully than I'm doing it.
0, 1 can go to 0, 1, 0, or 0, 1, 1.
0, 1, 0 can either go to -- u2 equals 1.
I guess these can't cross.
u2 --
[INAUDIBLE]
Excuse me?
[INAUDIBLE] u3 and u4
No, this is state time.
State time 1, 2, 3. u4 hasn't started yet.
So I'm sorry, this is 5 through 7.
Is that right?
It only lasts for three intervals.
So I only have u2, u3.
u1, u2 has to be 0, 0, 1, so this would then go to 0, 1.
0, 1.
These seem to cross.
Yes, they do cross.
So let's see if I can actually complete it.
This could go to 1, 0, 0, or 1, 0, 1.
And then this one goes up to either 0, 0 or 0, 1.
This one can go to 1, 1, 0 or 1, 1, 1.
And that can go like this.
So there's the trellis for the first half.
It's topologically the same, but looks a little different from what I put up before.
I interchanged these two with these two, and this one with this one.
But you can recover the trellis that I had before.
In any case, I claim when I've labeled this with what the appropriate output is at each time, I've got my minimal trellis.
And now we can make more comments about how the trellis works.
You see that whenever a generator starts, we have what's called a divergence in the trellis, a branching process, a branching out, whereas whenever a generator ends, we have a convergence in.
So a starting time -- if we have a generator starting, then we get a branch that looks like this, like here, and here, and here.
When we have an ending time, we have branches that look like this, like here.
So this is a starting time, starting time, starting time, and ending time.
So this tells you when things open and close in the trellis.
Now again, as I've said for these more general trellises, down here we can have starting and ending at the same time.
And in that case, we get a cross.
We get a starting and ending at the same time, we get something that looks like this.
And if we get neither starting nor ending, empty, then nothing happens.
We just get -- the trellis segment looks like that, like this one here.
In this case we have a starting time, an ending time in this trellis.
Here we have nothing happening.
Here we have nothing happening.
So you just get an extension, a parallel extension where nothing happens.
So again, the starting and ending times tell you what happens in the trellis.
There's one other thing that you can read from the trellis, which is the size of branch spaces.
And since as usual I'm running a little later than I intend to -- let me just state without proof that there is a similar theorem for branch spaces.
What is a branch?
When we look in a trellis, a branch is one of these things.
That's a branch.
What do we need to define a branch?
We have a initial state, the code output at time k. We're calling that y time k in the next state.
So it's that triple.
Again, this is linear system theory.
State, output, next state.
Those are the three things that define a branch.
This branch is state 1 0.
Output whatever it is, y k, and next state 1 0 0.
And that differentiates it from this down here.
If we define the branch as this triple, then we can add triples.
States form a vector space over the ground field.
These are simply symbols over the ground field.
These are vectors over the ground field.
So we can add them.
And we find that it's closed, and the set of all branches is a vector space.
It's fairly easy to show.
Look at the notes, but it's basically because these are all projected from code words in the first place, and code words form a vector space, so this is just a projection of the code words, a certain kind of projection onto the state spaces and symbol spaces and so forth.
A projection of a vector space is a vector space.
Roughly the proof.
And what is the dimension of the branch space?
Again, I'm just going to give you the answer.
Suppose we have a trellis-oriented generator matrix.
Let's suppose we want to find out the size of the branch space at this time.
What doesn't enter into the branch space?
What doesn't make a difference to the branches?
All the pasts that arrive at the same state at time k are equivalent.
So this generator doesn't enter in.
All the futures that depart from time k plus 1 don't enter in, so anything that's supported by the future at time k plus 1 doesn't enter in.
And what's left is all of the generators that are active at symbol time k. Notice the symbol times occur between the state times.
So certainly this is an easy -- this is one of those results that's easier to understand than to prove, and I'll refer you to the notes to prove.
The dimension of the branch space is simply the number of generators that are active at symbol time k. Because branches are synchronized with the symbol times, not with the state times.
So how does that come out over here?
There's the dimension of the branch space is 1 here.
It's 2 here.
It's 3 here.
It's 4 here.
I'm sorry, it's 3 again here.
This one hasn't started yet.
It's 3 again here.
It's 3 again here, 2 here, 1 here.
That means that the size of the branch space is 2, 4, 8, 8, 8, 8, 4, 2.
Is that right?
There are two branches here.
There are four branches here.
There are eight branches here.
There are still eight branches here.
And it's symmetric on the other side.
So it looks like we got the right answer there, and in fact it's a general rule.
So that's great.
That's another property of trellis-oriented generator matrix.
We can again, by inspection, get the dimensions of all the branch space sizes.
Are the branch sizes important?
Yes.
In fact, they're more important from a complexity point of view than the state space sizes.
95% of the literature has to do with state spaces.
State space is a little bit more elegant mathematically.
For instance, you don't have the same kind of dual branch space theorem as you have a dual state space theorem.
But actually, when you're asking what's going on in the Viterbi algorithm, what does the Viterbi algorithm have to do?
At a minimum, it has to do one thing for each branch.
It has to perform a metric extension, a metric addition computation at least once for each branch.
So the Viterbi algorithm complexity is better estimated by the size of the branch space than the size of the state space.
To progress at this point, a Viterbi algorithm has to do at least eight things.
To progress again, it has to do at least eight things.
So it's more relevant from the point of view of complexity.
Furthermore, you can play games with state space size.
And you've seen me play games with state space size.
For instance, if I draw this as a two-section trellis -- the first picture I put up was only a four-state trellis, and it looked like this.
I went immediately to time 2, and I aggregated these two things.
0, 0 and 1, 1 were up here, and 0, 1 and 1, 0 were down here.
And then I had a nice -- I just drew these four states over here.
And if I do it like that, then I get a much nicer picture.
It looks like this.
It looks only like a four-state trellis.
Now, is this still a legitimate trellis?
Yeah, it's still a legitimate trellis.
If I do a Viterbi algorithm decoding of this trellis, is it any different from this?
Well, in detail it is a little bit different.
You do the additions and the comparisons and the selection.
So you only do selections every this often.
And if you think about it, it's really legitimate to think of this -- if you think exactly how the Viterbi algorithm works, really what's it doing?
It's not making any selection here.
It's just adding an increment there, and then adding another increment, and it's not making any selection until it gets here.
So it's perfectly legitimate to think of this as basically being a four-state algorithm.
It really isn't doing any work at this stage.
All the work occurs at this stage if you think about the operation of the Viterbi algorithm.
So this is a quite legitimate representation.
And so what are we going to say?
Is the state complexity of this code, is it four states or is it eight states?
Well, to some degree it's just a matter of definition.
How do you choose to define it?
If you want something more fundamental, it's better to go to branch complexity.
What's the branch complexity here for the fully displayed trellis?
It's eight.
By branch complexity, I mean the maximum size of any branch space.
What's the branch complexity here?
It's eight.
We've still got eight.
Ultimately we've got to compare these eight things and reduce them to four, whether we do it here or here.
I gave you another trellis which was only a two-section trellis, where I just showed the four central states.
And we actually have two branches going there, two branches going there, two branches going there, and two branches going there.
What's the branch complexity of that trellis?
Still eight.
There are eight ways to get from the origin to here.
If you go back to how we do this theorem, if you believe this theorem, what happens now if we say arbitrarily, OK, we're going to define the state spaces as we do in a rate 1/2 convolutional code.
We're just going to define the state times to be every two times.
Then the dimension of the state spaces, just at those times we've defined, is 0, 2, 2, 2, 0.
They don't change.
We just have fewer of them by our choice.
What happens to the dimensions of the branch spaces?
Again, if the theorem is correct, we should just count the number of active generators at each now of these four bi-symbol times.
So it's 2, 3, 3, 2.
Is that right?
That is the branch complexity of this trellis down here.
It's 4 in the initial interval, and 8 over here.
So again it's correct.
Suppose we go to that so-called two-section trellis.
We just say there's a first half and a second half.
Now the states is going to be 0 2 0, and the branch complexity is going to be -- there are three active generators in the first half, and three active generators in the second half.
So by playing this kind of game, which is called sectionalization, we can make the state complexity sometimes appear simpler.
There are limits to how much simpler we can make it.
Well, there actually aren't.
Suppose we took the whole eight symbols as one time.
We'd have 0 dimensional state space at the beginning, and 0 dimensional at the end.
What does that trellis look like?
It looks like 16 parallel transitions from beginning to end.
That's a legitimate trellis, right?
We should have one state at the beginning and one state at the end, and 16 transitions.
Is that what we get if we went through this?
Yeah.
We have 0 0.
And what's the branch complexity?
It's 4.
The dimension of the branch space is 4 if we took the whole thing.
So we can mask state complexity by picking our state spaces at appropriate times, but you can't ever mask branch complexity.
And again, it's a very intuitive proof here.
Let's take this time here, where there are three active generators.
By expanding that time, can we ever get fewer than three active generators?
No, any interval that includes this particular time is going to have three active generators in it.
So branch complexity cannot be reduced by this kind of sectionalization game.
State complexity can be apparently reduced.
It isn't really.
But branch complexity cannot
[INAUDIBLE]
And if you go too far, it might increase.
So at the very end of the chapter, I talk about sectionalization.
It's not a very important issue.
There's a very easy heuristic for how you should sectionalize, which is basically you should make the sections as big as you can without increasing branch complexity.
So in this case, the heuristic says we know we're going to have to have branch complexity of 3.
But without increasing -- if I just make this one partition at the center, I just take these two sections, first half, second half.
Then I have an increased branch complexity.
So that's a good sectionalization.
This will probably lead to the simplest Viterbi algorithm and the simplest-looking trellis representation of the code.
So that's the character of the sectionalization algorithm.
You can read about it.
It usually winds up with a sensible time-varying sectionalization, and it's worth five minutes of discussion.
Yes
Is it like a monotonic kind of thing, like if you make sections bigger, then you'll only either increase or keep the same branch complexity?
Yeah, that was my basic argument, that if I have any section that includes this time, and I start making it bigger, the dimension can only increase.
So the algorithm is to keep increasing it until I get to some place where I might have to include another generator.
For instance here, if I'm taking this time, I don't want to push it out over here, because then I'll get the fourth generator in there.
So any section that includes time 3 should not be extended in this direction, because we'll get another generator.
But no problem with extending in this direction.
So that's the basic heuristic.
It's easy.
And it's not very productive.
It doesn't really change the complexity of the Viterbi algorithm as a representation of code.
So these are just a couple of arguments to say really we might -- the branch complexity is more fundamental.
We want to focus on branch complexity a little bit more.
But of course, if you have a fully displayed trellis, the maximum state space dimension is either going to be equal to the maximum branch space dimension or one less than it.
Because in these binary trellises, we get at most a binary branch at each time.
So any branch complexity is at most twice that of the adjacent state space.
So they're not going to differ by very much.
So state complexity is a good proxy for branch complexity in a fully displayed or unsectionalized trellis.
While I'm on this subject, there's another interesting direction we can go, which is average dimension bounds.
Again, think of our realization here.
What does it contribute?
What does a single generator contribute overall to state and branch space complexity?
A single generator Gj of span S, which of course has to be greater than or equal to the minimum distance to the code, right?
The span can't be less of a generator.
It can't be less than the minimum nonzero weight of any code word.
Span S contributes overall to all the state space dimensions.
It contributes S minus 1 state space dimensions, because it's active for S minus 1 state times.
And it contributes S branch dimensions because it's active for S symbols.
If I look at it spread out across time, this generator of span 4, it contributed three times to a state space dimension, and it contributed four times to a branch dimension.
So total state dimension has to be greater than or equal to k times d minus 1.
And the total branch dimension, just summing up across the trellis, is greater than or equal to k generators times d. Now how many different nontrivial state spaces are there?
There are n minus 1 nontrivial state times.
Excuse me.
For instance, in this code of length 8, there are 7 nontrivial state spaces.
So the average state dimension has to be greater than or equal to k d minus 1 over n minus 1.
And the average branch dimension has got to be greater than or equal to k d over the n symbol times.
So actually we might remember that quantity.
That's what we called the coding gain of the code, the nominal coding gain on the additive white Gaussian noise channel.
Well, if the average dimension is lower bounded by something, then certainly the maximum state space dimension is lower bounded by that.
Max has to be at least as great as the average.
So if these are what are called the average dimension bounds on the maximum state space size, and there are some interesting implications.
Again, this expression for branch dimension is nicer here, because it's this k d over n nominal coding gain thing.
It's in terms of the basic n k d parameters of the code.
And it says what?
It says if you want a nominal coding gain of 2, it implies that the max branch dimension is greater than or equal to 2.
Or the max size is greater than or equal to 2 to the 2, or you need a branch space of at least size 4, or dimension 2 to get a 3 dB nominal coding gain.
Similarly, if you want a 6 dB gain -- let's get right down to dB.
It says you're going to need a branch space size greater than or equal to 4, or at least a 16-state trellis.
So we know if we want a 6 dB coding gain, we're going to need at least a 16-state trellis, or a 16-branch trellis.
It might be an eight-state trellis.
Well, these are not tight, but they're quite indicative in fact of what we see in both the tables for block codes and for convolutional codes.
These same bounds hold for convolutional codes, by the way, just average over infinite time.
But they're still perfectly good.
So they tell us that in order to get -- if we want the nominal coding gain to keep going up to 8, to 16, to infinity, it says we're going to need infinite trellis complexity to get infinite coding gain.
Let's do some little asymptotics, be a little bit more refined about this.
A sequence of codes Cn of length n going to infinity is called good if both the rate k over n is bounded away from 0 as n goes to infinity, and the distance d over n is bounded away from 0.
The rate and the relative distance are both bounded away from 0 as n goes to infinity.
Is that clear?
In other words, we want the dimension and the minimum distance both to increase linearly with n in a good sequence of codes.
This is a classical algebraic definition, and it's easy to prove there exists good sequences of codes.
You just pick a code at random, rate 1/2 of binary linear code.
It on average will have a relative minimum distance about 11%.
This is just basic information theory.
So there certainly exists sequences of good codes, and this is what most of coding theory was occupied with constructing for a very long time.
What happens if you have a good sequence of codes?
We have this nominal coding gain, which is equal to kd over n, which is equal to n times k over n times d over n. If both of these are bounded away from zero, then the nominal coding gain has to increase linearly with n. That means that the average, or the maximum branch dimension, in a minimal trellis representation for your code has to increase at least linearly with n, which means that the actual branch complexity, the size of B, has to increase as -- it goes as 2 to the nominal coding gain, which is in other words exponentially with n. So basically what this says is that if your objective is to get a good sequence of codes, if this is the way you think you're going to get to channel capacity, then sure enough, your nominal coding gain will go up to infinity linearly with n. But how are you going to decode it?
Our first method was simply maximum likelihood decoding.
The complexity of that, computing the distance to every code word, that certainly goes up exponentially with n, exhaustive decoding, comparing with every code word.
But now we have a new method which is, we hope, simpler, which is trellis decoding.
But we find out, unfortunately, the complexity of trellis decoding is going to go up exponentially with n as well.
So the good news was that with the Viterbi algorithm and a minimal trellis, we can get a simpler optimum maximum likelihood decoding algorithm for block code, simpler than exhaustive decoding.
The bad news is that under this assumption, the complexity of that algorithm is still going to go up exponentially with n. Maybe just a lower exponent, that's all.
So we haven't really solved our basic problem, which is exponential decoding complexity, by going to trellis decoding.
We have, however, another way to go.
Do we really need the coding gain to go to infinity, the nominal coding gain to go to infinity?
If we go back and remember the name of the game, the playing field that we're on, at least in the additive white Gaussian noise channel, we found the distance between uncoded and the Shannon limit was some finite number, something like 9 dB at a 10 to the minus 5 error probability, larger at lower error probability.
So that's the maximum effective coding gain we can ever get.
So maybe we only need a finite nominal coding gain.
In practice, this means maybe we don't need a minimum distance that goes up linearly with n. Maybe we can have a very long code with a low minimum distance.
And that would be another way to crack the complexity problem.
In fact, some of the capacity-approaching codes that we're going to talk about, specifically turbo codes, tend to have bad minimum distances.
You have a turbo code that's thousands of bits long, and its minimum distance will be something like 20 or 30.
And yet it signals within 1 dB of the Shannon limit.
It gives you low error probabilities within 1 dB of the Shannon limit.
So maybe there's another way to go, at least if we're only interested in a certain error probability.
Maybe it's 10 to the minus 5.
Maybe it's 10 to the minus 10.
But it's still only a finite error probability.
Maybe we don't need the minimum distance to go to infinity.
Maybe we can get by sampling codes which are lousy in classical terms, which have poor minimum distance.
Maybe they'd be lousy in distance terms, or in nominal coding gain terms, but they'll be very good from the point of view of complexity.
Maybe there's some trade-off here.
We know the code is going to have to be long, but maybe it doesn't have to have large distance.
Yeah
[INAUDIBLE] That's what you are saying, isn't it?
We try to maintain the --
We certainly want to maintain a finite rate.
This was the problem with orthogonal and simplex codes and so forth, is that the rate went to zero, and therefore the spectral efficiency went to zero.
In practically all cases, that's unacceptable.
So we need to maintain some minimal rate or spectral efficiency on the additive white Gaussian noise.
So that we need, but this maybe we don't need.
So that's just a interesting little bit of philosophy we can do at this point, is how should you really design capacity-achieving codes?
And this seems to be a clue to the way to go that has in fact proved to be good.
However, I should add that not all capacity-approaching codes have this property.
A random low-density parity check code is going to have both low complexity and a very good minimum distance as well.
So this certainly isn't the full story.
There are just two more topics in chapter 10, and maybe I'll discuss them both briefly right now.
One is I skipped over projections.
And how shall I introduce this subject?
I talked about the subcode of, say, this code, which is pretty badly marked-up by now, but let me keep using it.
I talked about the subcode that's supported, say, on the first half.
In this case it's simply the subcode that's generated by the first generator.
Here's another code that we can look at that's kind of a first half code.
Suppose we just take the set of all possible first four-tuples in this code.
That's called the projection onto the first half.
In other words, what are the set of all possible four-tuples?
They're the set of code words that are generated by 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, and this doesn't contribute anything.
So it's a linear code.
It's length 4.
It has three generators, dimension 3.
And we can quickly convince ourselves that it's the even weight, or zero sum, or a single parity check code of length 4.
So if we project this on to the first half, we get a 4, 3, 2 code.
So that's what I mean by a projection.
If I project on to here, I get the 1 0 code.
I'm sorry, I get the universe code of length 1, all one-tuples.
On here I get all two-tuples.
On here I get all three-tuples.
But on here I don't get all four-tuples.
I get a single parity check code.
So those are my projections.
Some things can be done nicely in terms of projections.
Projections are kind of the dual to subcodes.
In fact, you prove this on the homework in the course of proving the dual state space theorem.
In other words, the dual of a subcode of a code C is the projection of the dual code of C, and vice versa.
They're always tongue twisters, the duality theorem.
But that's what you will prove along the way.
Let me give you the relevance of this to the state space theorem.
So I had the state space theorem right here.
So let me leave it up.
State space theorem is based on -- in general, we divide the generator matrix g prime into a part that is a generator for the past subcode 0.
0 generator for the future subcode.
And then some stuff down here, which is generators of the state space code.
By construction here, the projections of any nonzero vector in the state space code are nonzero.
If they were zero, any nonzero linear combination of these generators down here cannot be zero on here.
So the projection on to the first half here, the dimension of the projection -- I'm making a slight skip here -- is basically equal to the dimension of the past subspace code plus the dimension of the state code.
It's basically generated by these generators and these generators projected onto here.
We can forget about these generators.
So let me write that down.
The dimension of C projected onto the past is equal to the dimension of the past subcode plus the dimension of the state space code.
Which leads to another version of the state space theorem.
I could simply write state space theorem as dimension of the state space at time k is simply the difference between the dimension of the projection, C projected on the past, minus the dimension of the subcode.
It's all of these guys minus these guys.
And again, if we look back here, we see we have the 4, 3 code is C projected on p, 4, 3, 2, and C -- the subcode is 4, 1, 4.
In fact, for this, these are both Reed-Muller codes of half the length.
This is not an accident, as you will prove in the homework.
The state space size is the difference in the dimensions of these two codes.
This is a very handy thing to know, and you're going to need this.
This, for instance, will give you that in general for Reed-Muller codes, if you divide them in half -- we had this general picture of say the 8 4 code being made up by the new u plus v construction.
These two together make the 8, 4, 4 code.
If you have the u u plus v construction, there you are, first half, second half.
You will find that the projection on the first half is always this guy, which is the u code.
And the subcode is v code, which is the homework.
And so you can quickly read off from this what the dimension of the central state space is for Reed-Muller codes.
In this case it's 2.
But for instance if we have the 32, 16, 8 code, then what is that made up of?
That's made up from the 16, 11, 4 code and the 16, 5, 8 code.
And so we can see the dimension of the central state space here is going to be 6.
We're going to get a 64-state trellis for this code, at least just measuring the size of the central state space.
Now it turns out you can keep having -- if you go down and you make four cuts here, then the relevant codes are what you get one more space back.
At this space, we have the 2, 2, 1 as the projected code, and the 2, 0, infinity as the subcode.
And again, the difference in dimensions here is going to be the same as the different dimensions here.
Minor miracle at first.
It turns out to be just from the construction.
Here we go back to the 8, 7, 2 and the 8, 1, 8.
And again we see the difference here is 6.
So that means that Reed-Muller codes always have trellises, least if you put them in four sections.
They always look like this abstractly.
They look like what we've already seen for the 8, 4, 4 code.
And you'll do this on the homework using projections.
That's not very hard.
It doesn't mean that -- so we know, for instance, for the 32 16 8, we're going to do this, and there are going to be 64 states here, 64 states here, 64 states here.
We don't know what it's going to be in between, but you can figure that out too.
In fact, we get a minimal trellis for the whole thing.
That's what you need to do the homework.
There are actually two more topics that I want to do in Chapter 10.
One is the Muder bound.
And one -- I actually want to see how good are these block code trellises vis-a-vis convolutional code trellises.
So I'll do both of those at the beginning of next time.
For homework problem set, let's this week do only the first four problems.
So for this Wednesday, do problem set -- what are we up to?
Seven, is it?
One through four, and for Wednesday, 4/20, we'll do problem set seven, five and six, plus I'll probably have another one.
But we have a holiday next Monday also if you recall, Patriots' Day.
So we only get one more class on Wednesday.
So I probably won't have much more to add than this.
Is that clear?
Maybe Ashish you could put out a email to the class to do that.
So good.
We'll come back and clean up Chapter 10 quickly on Wednesday, and then move on to 11.
--wind up doing a substantial part of Chapter 11, although I do tend to be over-optimistic.
So we're handing out Chapter 11, which I expect to revise still further.
As we get towards these end chapters, these are things I've only taught from, at most, once before, and so I have more revisions to do than the earlier chapters which I've used for many years.
The Problem Set Seven, as we announced last time, you only needed to hand in the first four problems today, so we're only handing out the solutions for the first four today.
I haven't made up any additional problems, so Problem Set Eight is simply to do the last two problems in Problem Set Seven, due next Wednesday.
And a reminder that we have no class on Monday because of this peculiar local holiday we have called Patriot's Day, when we run the marathon.
And I hope you'll all be out there running the marathon.
So enjoy the long weekend.
So what have we been doing in Chapter 10?
We've basically been getting minimal trellis realizations, or state-state space realizations, in system theory terms, for linear block codes.
We've been focusing on binary linear block codes, but clearly everything we've been doing goes to block codes over any finite field or, in fact, over the real or complex field.
It's just basic realization theory, minimal realization theory.
So as long as we have a linear time-varying system, it works.
I had a question after class last time, well, doesn't it work just as well for nonlinear systems?
Can't we do the trellis in the same way?
In general, not.
If it's a nonlinear system with a group property, then you can't.
It's really the group property that's key to these constructions.
And we seem to have pretty well cracked the problem.
We have this tool called the trellis-oriented or minimal-span generator matrix, from which we can read all the parameters of the trellis, from which we can construct a trellis.
Really, we found that the trellis-oriented generator matrix contains all the information we need to construct a minimal trellis.
And it's just a matter of turning the crank once we have that.
The only degree of freedom that I've allowed so far in trellis constructions is sectionalization.
I've shown you that if we can choose where to put the state spaces wherever we like-- typically we put them in the center and other places, sometimes we put them everywhere we could put them, that's called an unsectionalized trellis-- and by so doing, we can sometimes reduce the apparent state-space complexity.
But we can never reduce the branch complexity.
So we're left with branch complexity as a fundamental complexity parameter for linear block codes, or so it seems.
Is there any other-- yes?
Does the complexity of the number of computations you need to go through the trellis depends only on the number of branches, right?
It depends primarily on the number branches.
It also depends on the edges.
If you go through a detailed operation count, I think I give the formula for if you count additions and comparisons as an addition and selection you don't count as a logical operation, then I think the total count for the Viterbi algorithm is twice the number of edges in the graph, which is the number of branches, minus the number of vertices, which you lose one for every--
What do you mean by vertices?
By vertices I mean states, now I'm talking about it as a graph.
Minus one or plus one or something.
So that's an exact count for the Viterbi algorithm.
But it's dominated by the total number of edges in the graph which, in turn, is well-estimated by the maximum branch complexity at any one time.
So there's a bit of a trade-off between an exact count and just getting a gross handle on the situation.
But I say it's certainly got to exceed the branch complexity.
So when I find the branch complexity goes up exponentially, even for one branch, then I know that the total Viterbi algorithm complexity has to go up exponentially with that.
All right?
So is that the whole story?
Do we now have a pretty definitive handle on the, quote, complexity of at least maximum likelihood decoding, trellis decoding of a block code.
And if you've read Chapter 11, you know there's one other degree of freedom that we haven't exploited yet in representing a linear block code.
Anyone read that far, or are you just going to wait for me to explain it to you?
Flip the coordinates, very good.
Could that make a difference?
Yes, it certainly could.
Let's take our standard example by now, the 8-4-4 code with trellis-oriented matrix that looks like this.
OK, and we know everything about this by now.
But suppose, say, we flip these two center coordinates.
Certainly it would be an equivalent code from all of its n, k, d and that sort of properties.
So I want to take these two coordinates and make them 1, 1, 1, 0, 0, 1, 1, 1.
Let's suppose that's a trellis-oriented generator matrix for the code.
I think it is.
It's got starting times here, here, here, and now here.
And it's got ending times now here, here, here, and here.
OK. Where I introduce the diverging and merging symbols that I used last time for starting and ending.
All right, I think that is a trellis-oriented generator matrix.
Well, it is.
We found a matrix that has distinct starting times and ending times, so it is.
Now what's its state branch complexity?
The state dimension is 0.
Now what are our spans now?
Two are the same, but two are larger.
OK, so now when we count, we get that the state complexity goes 0, 1, 2, 3, there are actually four that are now active in the center.
3, 2, 1, 0.
OK, and that's worse than we had before, at least in one place.
If we look at the dimensions of the branch spaces, which I'm arguing is more fundamental, then we get 1, 2, 3, again, 4, 4.
So the branch complexity is now 16 rather than 8.
It's going to be more difficult, definitely, to decode this code with the Viterbi algorithm.
This is a more complex representation of an effectively equivalent code.
Well, so is there-- so this is a bad permutation, it's something I didn't want to do.
Is it possible there's some good permutation that would reduce the complexity?
In general, when we're given a block code, traditionally at least, we think of it as the coordinates as just being not in any particular order.
We can write down a generator matrix, but the view is that any permutation of the symbol times is perfectly legitimate.
Certainly, if we send it over a memoryless channel, it's not going to affect the performance of the code, what order we send the symbols in.
So we should certainly regard all such codes as equivalent.
But we can affect the complexity, for better or for worse, so we should try to -- presumably, if we want a more fundamental complexity metric for the code, we want to find the complexity of the least complex trellis realization under all permutations of the coordinates.
All right, so that's a good issue, but very little is known about this issue.
It is known that finding the best permutation of an arbitrary linear code is an NP-hard problem.
So this is going to be hard.
We know for some specific classes of codes what the best permutation is.
For instance, the Reed-Muller codes are nice because it's been proved by Kasami and others that the standard coordinate ordering, the one that I've been using all along based on this length doubling construction, the u-u plus v construction, gives you the minimal trellis complexity.
All right, so we know what the best ordering is for the Reed-Muller codes.
And in a minute, I'm going to prove what the best possible we can get bounds-- in particular, the Muder bounds, which I'm going to discuss next-- which bound the best possible complexity that you could ever get.
And in some cases, we can find coordinate orderings that meet that bound, notably in the case of the 24-12-8 Golay code, we can find a coordinate ordering that's very closely related to the Reed-Muller ordering such that the bound is met.
And so we know that we've found the best complexity for that code.
It couldn't possibly be any better.
But apart from certain very nice cases like those, we don't know.
For instance, take an arbitrary BCH code.
What's the best coordinate ordering for that?
We don't know.
So this is the open question in this subject, and it's now been open long enough and enough smart people have looked at it, and we've got this NP-hardness result, so that it looks like it's going to stay open indefinitely.
All right, let me discuss, however, this Muder bound, because it's simple and sometimes definitive.
Yes?
[INAUDIBLE]?
Minimal branch complexity or any other measure that you like.
They all are quite fungible, it turns out.
OK, The Muder bound.
Let's look at a particular code and suppose we just know its parameters, n, k, d. Let's ask what the minimal trellis complexity could be for a particular coordinate ordering, say, that divides -- we have a certain number of past symbols, call it n_p, and a certain number of future symbols, n_f.
And I'm going to ask what the minimum state complexity, or I could also ask what's the minimum branch complexity for-- let me do now a state complexity, going back to where we started from, for a a certain division in the past and future.
To be concrete, let me take the 24-12-8 Golay code.
It turns out that it's been proved that all codes with these parameters are equivalent.
So we can just call this code the binary Golay code.
And let's look for any division into 16 future coordinates and 8 past coordinates.
So we don't know what the ordering is going to be, but we could say a few things.
Remember how we get the state complexity.
We look for what's the dimension of the past subcode.
So here's a generator for the past subcode here, the code whose support is the past.
We can ask what the dimension of the future subcode is, and then the dimension of the state space is the dimension of what's left.
So it's dimension of c minus the dimension of the past subcode minus the dimension of the future subcode.
State-space theorem.
OK, but we know a few things about this code.
We know what its length is and we know what its minimum distance is.
So for this particular case, we have that c past is an 8-something-8 code, because its minimum distance has got to be at least as great as this.
It's a subcode of this code.
So any non-zero code words in this past subcode have to have minimum weight of at least 8.
Correct?
All right.
And similarly, the future subcode, in this case, is a 16-k -- let's call this past -- k-future-8 code.
At least 8.
So that imposes some bounds on the dimension of this code.
How large could the dimension possibly be?
All right, anybody?
For the past subcode its clearly, we have that k_p less than or equal to 1.
We could have, at most, one generator here.
And if there were one, it would have to be the all-1 code word.
Or the all-1 symbols here, because it has to have weight 8.
So that's all we can get.
For the future code-- well, I guess you don't know this, but you can guess that the Reed-Muller code with length 16 and minimum distance 8 is the best there is, that distance 16, and that happens to be true.
So k-future can't possibly be better than 5, which is the dimension of the Reed-Muller code with those parameters.
For more general cases, you have to look up tables on bounds on the best codes that have ever been found.
But certainly for code lengths less than a 100, by this time we pretty much know all the best binary codes.
So we can look up in a table what's the best code of length 16 and distance 8?
Well, it has five information symbols, dimension five.
So we couldn't have more than one dimension here and we couldn't have more than five dimensions here, so the dimensional code itself is 12.
We have to say that the dimension state-space is now going to be greater than equal to-- this is k_p minus 1 minus 5-- so, that's 6.
So there have to be at least 6 generators down here that have support neither wholly on the past nor wholly on the future and therefore have to be included in the state-space.
All right, that's the idea of the Muder bound.
Now, we could do this for every single state-space.
We simply need to know what the dimension is of the best code of each length with minimum distance 8.
And we can do a similar thing for every possible partition in the past and future.
And that will give us a set of bounds, and we go through that exercise in the notes.
And I can write it on the board, maybe for memory.
And OK, that tells us what the best possible state profile is.
Let me guess state dimension profile is 0, 1, 2, 3, 4, 5, 6, 7.
Then it goes down to 6, 7, 8, 9, then it goes down to 8.
This is at the central state-space.
This is for dividing it in the way that we just did.
Let's do this here.
For the central state-space, what's the best k_p for a 12, k_p, 8 code?
Let's make the past and future both 12.
Anyone want to guess what the best dimension is?
Let's think about it.
Let's think of a code that's going to have to have minimum weight 8.
So let's start out with g1.
And we'll just make that look like that.
It's going to have length 12.
It's going to have to have at least eight 1s, so let's make them like that.
We want to add another 1 so that the total code has a minimum distance 8.
Subject to up to permutations, there's only one thing it can be.
We need another weight-8 generator, and it needs to differ from this in at least four places so that when we take the mod 2 sum of these two, it has distance 8.
And we want g1 plus g2.
This is really the only way it can be.
Are you with me?
We have four code words in this dimension 2 code.
And if they're going to have minimum distance 8, they're going to have to look like this, subject to up to permutation.
So from that little argument, we conclude that, in this case, this is less than or equal to 2.
This is less than or equal to 2, the Muder bound becomes that the dimension of the state-space has to be at least 8.
And so in the same way, and then it becomes symmetrical on the other side.
So I'm pretty sure I get it right.
And from memory, that's the best possible state dimension profile, dot dot dot.
And we actually know a coordinate ordering for the Golay code, a generator matrix for the Golay code, that gives this state dimension profile.
So from that, we can conclude that we know an optimum coordinate ordering for the Golay code.
I just want you to get the essence of the argument here.
You with me?
For branch spaces, the argument is much the same.
Remember in branch spaces, we basically -- let me say it in a slightly different way than we did before -- we take the past to be up to time k. And we have the symbol time, which is just one time in here for the actual symbol we're looking at.
And the future is one unit shorter than it was for the state-space.
So when we compute the branch dimension, the past goes up to k. So this is 0 and before k. And the future goes from k plus one up to n. So this is what the matrix looks like for this.
And the effect over here is that the future subcode now is one unit shorter.
So let's do the same calculation here.
If we look at a time where we have 12 in the past, this one time, the 13th symbol time that we're actually looking at and then 11 more in the future, the past subcode, again, could have dimension up to 2 because it has length 12 and minimum distance 8.
The future subcode now has a shorter length and by this little argument I went through here.
If we take out one of these coordinates, it's not going to work.
So if we only have length 11, the future subcode is the dimension -- still can only be 1.
We can't possibly add another second generator in there.
And so the dimension of the branch space has to be vastly greater than or equal to 9, at this point.
OK, so the Golay code has branch complexity of at least 512, we conclude.
And, in fact, this best possible branch dimension profile looks like this.
The branches are in the middle here.
They're 1, 2, 3, 4, 5, 6, 6.
Basically we have generators which look like this.
And then there's one that comes down, there are three more that goes up and one that comes down.
And from that, we get 1, 2, 3, 4, 6, 7, 7, 8 -- that can't be right.
Goes down 7, goes up, it's still 7, then it goes up to 8, 9, and 9, 9, 9.
Something like that.
That one's probably wrong, and symmetrically.
Where this 9 is the one we just calculated.
Similarly, if you come down to the neighbor of this 6, it's got a merge on one side, a diverge on the other side, and that's 7.
OK, so we conclude that the Golay code has branch dimension not less than 512 within 9.
Yeah?
Suppose we take past and future separated by 12, 12 in the past, 12 in the future.
Suppose we did it as they did in [UNINTELLIGIBLE] state?
OK, but that's not the way we do branches.
We've got to isolate the symbol time
Suppose we took 12-branch-12 and we calculate the state complexity, that's 8.
And then we took 13 and 11, we get the state complexity 9
All right, you're saying if we do this, we should get the state complexity 9.
And that's correct, we do
The maximum of those two numbers, is it the same as branch complexity?
It turns out to be in this case
Not always?
Not always.
But if we go to 14, 10, then it turns out we can add another one here because we can add another generator that looks like something like this.
1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1.
So if we have two more units, we can add another generator.
Sort of Reed-Muller style.
And so this goes up to 3 and this goes down to 8
So the branch complexity has to be at least the maximum of those two, right?
Obviously, right.
And really, together these give you the story of where the thing has to diverge and merge.
In this case, this is a self-dual code, so you only do one at each time.
So with this sort of thing, you can pretty well pin it down.
Again, I don't know how far to take this lecture.
I'm really just trying to get to you the idea of the Muder bound from this kind of argument, therefore we can get a bound on the best possible state dimension profile and the best possible branch complexity profile.
If you compute this bound and you're able to find an actual generator matrix that meets this bound, you know you've found the best one.
But this rarely happens, this is a very special code.
It's not the way you prove the optimality of the longer Reed-Muller codes, they're not good enough to -- this really assumes that you can get the best possible sub-codes at every point.
And you can imagine that's very hard to do.
It would be a very special case where you can optimize this for every single possible cut between past and future.
OK, so that's the Muder bound.
And you have a homework exercise on that as well.
Oh, there's another case where we can find what the best possible trellis complexity is.
It's MDS codes over GFq, and you're going to do that on the homework as well.
And there, we basically show that an MDS code, its n, k, d are such that it has to have the worst possible state complexity and branch complexity.
So because it's such a tightly-packed code, in Hamming distance sense, it forces you to have the worst possible complexity that you could have.
In other words, you have diverges immediately and as long as you can, and then they're all packed together on the left side and the merges are all packed together on the right side, and that's the only way it can be.
I'm just speaking very impressionistically.
But you'll see what I mean when you get to the homework, if you haven't done it already.
OK, so we also know MDS codes, we know their trellis complexity and it's terrible.
We can't get any effective reduction.
Really, over exhaustive decoding methods with MDS codes, we find that the trellis complexity is like either q to the k or q to the n minus k, depending on which is less.
So that's as bad as it could possibly be.
So if the trellis doesn't help for MDS codes, in general cyclic codes -- which were subject to a lot of study -- if we put the code into cyclic form, which if you remember Reed-Solomon codes and BCH codes can always be put into cyclic form, at least the shortened versions of them.
In that case, the generator matrix looks like this one up here.
It looks like x, x, x, x, x, x, x, 0, 0, 0, 0, 0, 0, 0, x, x, x, x, x, x, x, 0, 0, x, x, x, x, x, x, x.
In other words, the generators are just cyclic shifts of one another.
Down to x, x, x, x, x, x, x, 0, 0, 0, 0.
That is a generator matrix that is a trellis-oriented generator matrix because all the starting times are different and all the stopping times are different.
So this is the trellis-oriented generator matrix for cyclic code, and it's in the coordinate ordering in which it is cyclic.
And it's always going to have this worst case -- all the branch diverges are going to occur over here and you may get nothing for a while.
And then all the merges over here, which if it's a rate less than 1/2 code, this means that the trellis complexity is basically going to be 2 to the k. Or if it's a high-rate code, it means it's going to be 2 to the n minus k. And again, you get no improvement over some exhaustive minimum distance decoding, or maximum likelihood decoding.
So cyclic is very bad from a trellis point of view.
So I think that's about all the general comments I can make.
The very last thing that I want to mention is we now want to -- all right, so through this nice little theory, we've got a way of trellis decoding (maximum likelihood decoding), via the Viterbi algorithm, of block codes.
And now, did we achieve any improvements in terms of performance versus complexity versus the convolutional codes that we looked at previously?
And the basic answer is, no.
If you compare the tables, convolutional are always better.
I'll just say it.
You can imagine that there have been long debates in the fraternity, and that certainly some of the motivation for trellis decoding of block codes was, oh, gee, now we have a good way to decode block codes, just like we did for convolutional codes.
And maybe now we'll find that block codes are worthwhile.
But the answer is they're not.
I should put this comparison into Chapter 10.
You can actually get this comparison by taking the table -- for instance, for Reed-Muller codes.
In terms of performance versus complexity, Reed-Muller codes are very good.
I would say as a general statement, they're the best block codes that we know of, in terms of performance versus trellis decoding complexity, because they have this nice structure that gives them very low-complexity trellises for their n, k, d parameters.
So in this sense, it's a fair comparison.
Reed-Muller codes are the best general class we know of in terms of trellis complexity.
We compare them against binary convolutional codes and they're not nearly as good.
I've already talked about the comparison.
It seems like a very fair comparison to take our rate 1/2 four-state convolutional code example, which had distance 5, which had not only a nominal coding gain of 4 dB, but it also has an effective coding gain of 4 dB.
And for a block code, this 8, 4, 4 code is a very fair comparison.
It's also rate 1/2, so it has the same spectral efficiency.
It also has a four-state trellis that looks very much like a rate 1/2 convolutional code trellis.
But it turns out it only has minimum distance of 4 rather than 5, so its nominal coding gain is only 3 dB rather than 4 dB.
Its effective coding gain is less.
Its effective coding gain is 2.6 dN because it has 14 nearest neighbors, which is a 3 1/2 nearest neighbors per bit, and that costs you about 4/10 of a dN.
And so really, you compare the performance of the convolutional to the block code, that's very much better.
Another head-to-head comparison might be between the 32, 16, 8 Reed-Muller code.
That's a rate 1/2 code.
It's a very good one, it's known to be the best with those parameters, n, k, d. So what's its nominal coding gain?
Anyone?
It has rate 1/2, minimum distance 8, its nominal coding gain is?
Hello?
6 dB.
You take the rate times the minimum distance-- 1/2 times 8, you get 4.
So that's the nominal coding gain.
A factor of 4, which in dB terms is 6 dB.
But again, it has lots of minimum-weight code words.
It has 620 minimum-weight code words.
And therefore, the number of minimum-weight code words per bit is 39, which is between five and six factors of 3, so it costs you 5 and 1/2 times 0.2 dB in terms of effective coding gain, which brings you down to an effective coding gain of 4.9 dB.
I assume you're all very fluent in doing these, as I am.
And what's its complexity?
If you look its state complexity, let's take-- we're looking at the 32, 16, 8.
And all the Reed-Muller codes have four section trellises, you proved that this week.
If you just measure the state complexity of four places, equally spaced with four equal sections, then all the trellises wind up looking like this, for a general Reed-Muller trellis -- a four-section trellis.
And for the 32, 16, 8, it turns out you get eight parallel sections like that with eight of these each.
In any case, it's 64 states at each of these boundaries.
So you could say it's a 64-state trellis.
But if you're more honest, just exactly as with the Golay code calculation we went through, you find that the maximum branch complexity is 512.
So branch complexity is 512.
OK, and its nominal coding gain we said is 6 dB and its effective coding gain is 4.9 dB.
OK, so let's look for a convolutional code of rate 1/2 that has a coding gain of about 5 or 6 dB.
And how far do we have to go?
We actually only have to go up to a-- well, a 32-state code has an effective coding gain of 5.6 dB.
A 64-state code we can get up to an effective coding gain of about 6 dB.
In fact, the 64-state rate 1/2 convolutional code was a standard for a very long time, first in space communications and then in much more general applications.
This was the code that Linkabit Corporation -- Jacobs and Viterbi -- rode to fame and fortune, made their first pile before they went on to found Qualcomm, where they made a much bigger pile.
And so this was their-- they call it k equals 7.
But in my terms, nu equals 6, a 64-state rate 1/2 convolutional code.
So let's take that rate 1/2 64-state convolutional code, just with the standard trellis.
So its branch complexity is 128 at every time.
It has a more regular structure than this guy.
Its nominal coding gain is -- there are actually two of them, and I don't remember.
There's one of them that has a nominal coding gain of 7, the other one is 6.5.
So it's at least 6.5 dB.
And its effective coding gain is still 6.1 dB and it's clearly much better than this.
If you kind of try to plot a graph of this, you'll see that the effective coding gains of convolutional codes with the same state complexity is at least one dB better, as a general trend, than that of block codes.
And of course, if you do it in terms of branch complexity, it's an even better comparison.
So what happened was there was about a decade of effort on basically finding good trellis representations of block codes, and the net of it was that they couldn't get anywhere close to finding anything as good as convolutional codes.
Although, it's a nice little piece of theory.
And sometimes you want to block codes for other reasons.
Sometimes you want short blocks.
Sometimes you're limited to a block of less than 100.
And in that case, you're certainly not going to do better in terms of n, k, d than the best block code of a length 100.
Why are convolutional codes better?
First of all, they obviously naturally are well-adapted to the trellis idea.
They are linear time invariant machines that just have very nice, regular trellises.
So they're naturally adapted.
But also, they have exactly the right-- a rate 1/2 code is diverge, merge, diverge, merge, diverge, merge forever.
If you want to make it into a block code, you can terminate it or you can tail-bite it, which is something we're going to talk about shortly.
But whichever way you do it, to make it into a block code, you lose something.
The termination, you don't lose minimum distance, but you do lose rate.
You have to put in some dummy information symbols, so the rate becomes less than 1/2 if you terminate this code.
Nonetheless, as I believe I commented earlier, that's a good way to come up with good block codes, is to terminate a good convolutional code.
If you choose the parameters right, you can come up with pretty good codes.
But nonetheless, in terms of block code parameters, you're not going to do better than what 50 years of block code, algebraic coding theory has come up with.
So for short block codes with length less than 100, really the best we know of right now-- let's say 128-- is to pick a good Reed-Muller code or BCH code, find its best trellis and do maximum likelihood decoding.
It used to be that you thought of 64 states as a lot of states, but clearly, technology has progressed.
And now you wouldn't blink at 1,000 states or maybe even 8,000 or 16,000 states.
The biggest Viterbi decoder that was ever built had 2 to the 14 states for a space program out of JPL.
So anyway, it's doable now.
If you needed the best possible performance and you were constrained in block length, you'd pick the best block code at the rate that you wanted, and the best minimum distance at that rate, and the best effective coding gain, and you'd do maximum likelihood decoding with trellis decoding.
But if that's not usually the situation -- the situation in data communications, because usually more you have a stream of data or a packet length of at least 1,000, and then it would certainly be better to use convolutional codes.
And trellis decoding is -- as long as we're in this class of techniques.
Where we're finally going in this course is to the new codes that have been developed in the past decade, capacity-approaching codes, turbo codes, low-density parity-check codes.
These are simple enough to do so that as soon as you get past the small codes, length less than 100, you would switch over to that track.
You would use a capacity-approaching code.
But we don't know about them yet.
I guess that's the end of my lecture on this, so it's a very good place for questions about state-of-the-art, where we've been, where we're going.
The floor is always open.
But that was all I was going to say about Chapter 10.
Chapter 11, Codes on Graphs.
And now we begin our final stretch drive towards capacity-approaching codes, which is going to be the subject of the next three chapters.
Chapter 11 is sort of the beginnings of codes on graphs.
We're going to look at some elementary representations.
One of the ones we're going to look at is a trellis representation.
That's one of the reasons we've been going through this theory, is we're going to, again, encounter trellises as a natural, simple, cycle-free, graphical representation that, in some sense, is about as good as you can ever do, as long as you don't allow cycles in graphs.
But then we're going to go on to graphs with cycles, which is really the way to understand these capacity-approaching codes.
So in this chapter, we'll start the subject, we'll introduce graphical representations of codes in more general terms.
In Chapter 12, we'll talk about the generic decoding algorithms for codes on graphs called sum-product and min-sum decoding algorithms.
And then in Chapter 13, we'll talk about the actual classes of capacity-approaching codes that people have developed, what their graphs are, how you decode them -- I hope in enough detail so that you'll get a good sense for how all this works.
So that's where we're going.
What are we talking about when we're talking about codes on graphs?
There are many styles of representations of codes, and at this point, for linear codes, we have seen a number of ways we can characterize a code.
One of the ways is just by giving a set of generators for the code, k generators for an n, k code.
And that sort of characterizes the code.
Another way is to basically give the generator matrix for the dual code, the h matrix, which becomes the parity-check matrix for the code.
So we can also characterize the code by a dual-generator matrix.
That's a particularly efficient way to do it if you have a high-rate code like a single parity-check code, it's better to say that the dual code is the repetition code and it's the set of all code words that are orthogonal to the all-one code word.
That's the simplest characterization of the single parity-check code.
In other words, that the sum of all bits is equal to 0.
That's what orthogonal to the all-one code word means.
So that's a simpler representation than giving a generator matrix formed for the single parity-check code.
Or now we have this trellis representation, which certainly looks like a more graphical representation.
It certainly characterizes the code in a very direct way.
What does a trellis do?
It basically displays all the code words on a graph.
There's a 1:1 correspondence between the set of all paths, from the root to the n-node in the graph and the code.
So that's another way to characterize the code.
So now we're going to go to higher level graphical representations.
As a first step, we talk about behavioral realizations.
Again, a term from system theory, closely identified with Jan Willems, who promoted this way of representing linear systems.
In linear system theory terms, the basic idea in behavioral systems theory is you describe a system by the set of all its possible trajectories.
What are all the things it can do?
And now, how do you characterize the trajectories?
Often, you characterize them by a set of local constraints.
You can think of it as equations that the system has to satisfy.
Ordinarily, they're differential equations or partial differential equations or something.
And if it satisfies all of those constraints, then it's a trajectory that satisfies all these partial differential equations, is a valid trajectory.
So you can set up the whole system that way.
So it characterizes a system by its trajectories.
That's the fundamental thing.
And it characterizes trajectories -- legitimate, valid trajectories -- by local constraints.
Let me get a little bit more concrete.
In coding terms, by trajectories we just mean code words.
So this is a very compatible notion with the way we've been talking in coding.
What is a code?
It's simply a set of code words.
So if we simply say, what are all the possible code words?
We have to find all the trajectories of the code, and we have a very explicit example of that in trellises.
We've called the paths trajectories through a trellis.
What is a code?
It's simply the set of all valid code words.
And how do we characterize the valid code words?
Well, in each of these styles -- the generator, parity-check, trellis style -- the code words are the code words that satisfy certain equations or constraints.
For instance, the generator representation -- we've said a code is the set of all uG, such that u is in a field to the k. That's a generator matrix style of representing a code.
In other words, we can say it's the set of all y, such that y equals uG for some u in Fk.
Or we can say it's all y, such that y minus uG equals 0 for some u in Fk.
So it's the set of all y that, together with some auxiliary variables which are not part of the code word -- here they're input variables, if you like, or information bits, information symbols.
We'll come to think of these as state variables because they're hidden, they're not visible, they're just an auxiliary part of the description.
They're code words that, together with some auxiliary variables that, in this case, can be freely chosen, satisfy a certain set of linear homogeneous equations.
Something that's even better in the behavioral style is a parity-check representation.
Let me put that up.
In a parity-check representation, we say the code is the set of all y, such that y H transpose equals 0, where H is the generator matrix of the dual code.
This is sometimes called a kernel representation.
y is in the kernel of this linear transformation.
We see H transpose is a linear transformation from n-tuples to n minus k-tuples, and the ones whose image is 0 under this transformation, the ones that map to 0 are legitimate code words.
So this is a kernel representation, this is an image representation.
Here, we view the code as the image of the linear transformation, which is characterized by g, where the inputs are free.
But the parity-check representation is much more explicitly, we don't need any auxiliary variables here.
It's simply the set of n-tuples that satisfy a certain set of linear homogeneous equations.
So this maybe is a place where we start and, of course, it's the place where Bob Gallager started when he did low-density parity-check codes, which are a principal class of capacity-approaching codes.
Then in either case, we characterize the code by, in one case, directly satisfying some equations.
In the other case, here we say that the total behavior is the set of all y and u, such that y minus uG equals 0.
So we're going to take the set of all n-tuples and k-tuples, such that y minus uG equals 0.
And we say that's the total behavior of the code.
The set of all y's and u's that satisfy this equation is the total behavior.
And then we'll say the code word is just the projection of the behavior on the y's.
In other words, it's the set of all y's, such that this is satisfied first some u.
And that's an equivalent description.
So the general setup, going back and forth, is that we have the observed symbols.
And in the coding case, this will always just be the n symbols in the code word.
These are the ones that we really care about.
But we're also going to have some hidden or auxiliary symbols or state variables, hidden variables, which, in this case, for instance, are the u's.
And we can introduce these at our convenience, just to make a good realization.
So we have two types of symbols: the observed, or external variables-- sometimes these are called external variables, internal variables.
Again, in system theory, you're consistently seeing this.
There are some that we can actually observe, interact with.
These are the observed symbols, and then there are internal variables to the system, very often called state variables, which we can't directly see but which are part of the description of the system, as in this case.
The whole system, we find the set of all internal and external variables that satisfy a set of constraints.
But we only care about the patterns of observed variables that are consistent with some set of internal variables.
So the final element here in this general setup is constraints, which in this case are simply linear homogeneous equations on subsets of variables.
So let me go through similar kinds of examples, as I do in the notes.
But let me maybe build up from the simplest one first, parity-check representation of our favorite 8, 4, 4 code.
In this case, the parity-check matrix is the same, we can take it to be the same as the generator matrix.
Since, by now, we like trellis-oriented generator matrices, we'll take that one.
So to get very explicit, so the code is the set of all y, such that y H transpose equals 0, and y is 8-tuple.
So it's the set of all code words that satisfy these parity-check constraints.
The first one is y1 plus y2 plus y3 plus y4 equals 0.
That's orthogonality to this first code word.
The second one is y2 plus y4 plus y5 plus y7 equals 0.
The third equation is y3 plus y4 plus y5 plus y6 equals 0.
And the fourth is y5 plus y6 plus y7 plus y8 equals 0.
If I have any 8-tuple y, such that the elements of the 8-tuples satisfy these four equations, then it's a code word, if and only if.
A very explicit description of the code.
Now let's go onto a graph of behavioral realization.
So it's very natural to draw the following kind of graph of this realization.
We let the y's -- y1, y2, y3, y4 -- sorry, I've changed the index set, haven't I?
I get 1 through 8 now.
I'm just going to make these into vertices of my graph.
So here we have symbols, and I'm going to make a bipartite graph, and over here I'm going to have equations, or constraints, or checks.
And I have four checks, which I'm going to draw like this.
The first check is that y1 plus y2 plus y3 plus y4 equals 0, so I'll just draw that like this.
That means the same thing as this.
The second one is y2 plus y4 plus y5 plus y7 equals 0.
And the third one is these four -- two, three, four.
And the last one is these four.
This, this, this, this.
It's done more nicely in the notes.
This is called a Tanner graph, after an extremely good paper by Michael Tanner in 1981, which was about the only paper of any consequence in this subject between Gallager's thesis in 1961 and the rediscovery of low-density parity-check codes around 1995.
That's a graphical picture of these equations.
And what do we think of?
You can think of testing an 8-tuple.
Take an arbitrary, binary 8-tuple, think of these as memory elements, put the bits of that 8-tuple in here, and then ask if all these checks are satisfied.
Since we're talking mod 2 arithmetic, we're basically asking if there are an even number of 1s attached to each of these checks.
If and only if that's true, we've got a code 8-tuple.
From the set of all 256 8-tuples, we find 16 that actually satisfy these checks, and that's the code.
That's one way of describing the code.
So that's how the graph is associated with local constraints.
Well call them local, each of these constraints, because each of them only involves four of the symbols.
In a much larger graph, a low-density parity-check code, the idea is that there are not very many symbols that are checked by each check.
The graph is sparse, in that sense.
But we're not quite there yet.
So that's pretty simple.
Let me do a generator representation.
This is going to be a little bit more complicated because we have these auxiliary input variables.
But that doesn't really complicate it very much.
I keep using this 8, 4, 4 code just to emphasize that a single code can be described in many different ways.
We're going to wind up with half a dozen different graphical representations of this code, and they can be grouped according to style.
So again, we could use our favorite generator matrix for our favorite code.
It looks like that.
Now what do we mean?
What are our constraints now?
Well, now we have some hidden variables, ui, and we have y1 is equal to u1.
We're going to basically multiply this by u1.
I've got some u1, u2, u3, u4 multiplying this.
So y1 is u1, y2 is u1 plus u2, y3 is equal to u1 plus u3, y4 is equal to u1 plus u2 plus u3, so forth.
y5 is u2 plus u3 plus u4, y6 is u2 plus u4.
I jumped one here.
This is u3 plus u4, y7 is u2 plus u4, and y8 equals u8.
And you will agree that the code is described, if I can find any pair, (y,u), such that this is satisfied, then y is an element of the code.
I can choose u freely, that is the input.
If I find a y at any u, such that these equations are satisfied, then I've found a code word.
And that's an if and only if.
So that's another characterization of the code.
And its Tanner graph looks like this.
Here in this case, we have hidden variables.
So let's put the inputs-- or, we'll see they can also be thought of as states, but they're hidden variables that we don't actually see in the code word.
u1, u2, u3, u4.
You can think of these as free-driving variables.
And to get y1, here's what we do.
We're going to take y1 and we're going to create a mod 2 sum of some of these inputs.
In this case, it's just u1, so I can draw it as a straight through.
But in general, I'm going to have to make a combination to get all of these.
So I have eight symbols over here -- code symbols -- and again, some constraints that have to be satisfied.
Equations, mod 2 sums, y8.
And I just, again, draw a picture.
y2 is u1 plus u2, so u1 plus u2.
these sum to give y2.
And so, again, if I put down some u's, put down some y's, test whether all these sums are correct, then I will have tested whether I found a valid behavior or trajectory.
Next one is u1 plus u3.
The next one is u1, u2, u3.
The next one is u2, u3, u4.
The next one is u3, u4.
The next one is u2, u4.
And the last one is just that one.
OK, so that's another graph whose constraints describe the code word.
Here it's helpful to start to introduce the following convention.
We fill in the circle of the variables that we actually want to see, and we leave these open, the free variables over here.
Some people would call it a generalized Tanner graph.
In the notes, I just call this Tanner graph 2.
Tanner didn't actually consider state variables.
This was introduced into graphical models by a guy named Wiberg in his PhD thesis in Sweden in about 1995.
And that was a huge contribution, to have hidden variables as well as observed variables, because otherwise you couldn't really draw a generator representation as a graph, or a trellis representation.
You need hidden state variables.
So those are two graphical styles, and the idea here is that every set of 12 variables, such that these eight equations are satisfied, gives you a legitimate code word.
Here you can easily think of this as -- you notice that we've described these as passive constraints.
It's not an input-output, block diagram type of representation.
It causes cause-effects.
In this case, it implicitly is.
How would you actually implement this generator representation?
You implement it by picking four input bits and then seeing what code words they generate.
In other words, there's a very definite input-output relation here, where we could make this into a directed graph by drawing arrows here.
In general, in our graphical models, we're not going to do this.
The behavioral style is very much not in this spirit.
But in this case, we clearly can.
So if you actually wanted to generate all the code words -- that's why it's called a generator representation -- you simply jump through all 16 possibilities here and you generate all 16 code words.
Or if you wanted to do a simulation, generate code words, this is the way you would do it.
This has easy cause and effect style.
The kernel representation is much more implicit.
What is the cause and the effect here?
In this case, it's actually possible to find a cause and effect representation.
It turns out that these four bits can be taken as information bits, or these four places form an information set.
So suppose we choose these four bits, and it turns out that for this graph, we can trace through as follows.
These three bits going into this zero-sum, these all go in.
That determines this output, right?
d minus 1, if d is the degree of the zero-sum node, any d minus 1 inputs determine the last output, because the parity-check has to check.
So that determines this one, which then determines y4.
Now, knowing that, we know all but one input here, because I also have this one.
And that determines this, and I think it probably also determines this.
I now have three of the four inputs here, so I can get this.
And anyway, by tracing through it I can create a directed graph that gives me a cause and effect relationship from here.
But it's much more implicit and it can't always be done.
Sometimes you can find an information set that works with these particular set of equations and sometimes you can't, sometimes you get stuck.
We can come back to this when we talk about binary erasure channel and stopping sets.
In this case, given these four, you can determine the remaining four.
If you think of it in equation terms, if you're given y1, y2, y3, you can basically go through a Gaussian elimination of these equations and find y4, y6, y7 and y8.
So you can regard y1, y2, y3, and y5 as inputs.
This is much more the behavioral style, where it's just simply any n-tuple that satisfies the constraints is a legitimate n-tuple.
This is more cause and effect style over here.
Since we're at the end, let me just introduce one more thing.
This is a bipartite graph.
We basically have a graph with two types of nodes, one representing variables, one representing constraints.
And the edges always go between a variable and a constraint.
So that's what we mean by a bipartite graph.
There are two types of vertices, and all the edges connect one type of vertex to another type of vertex.
So is this over here.
I've drawn it in a way where it's not so obvious.
In a generalized Tanner graph, you'd put all the variables over on one side and the constraints over on the other side.
In this case, these observed variables go naturally with the constraints, so I put them over here.
But I could have put them over here and you would see this is also a bipartite graph.
So this is the general character.
What do we think of the edges as doing in this graph?
We think of the edges as conducting the variables that they're associated with.
So all of these edges, in fact, convey u1 to this constraint.
We could, if we wanted to, label all of the edges with their associated variables.
That's what they mean.
They mean here that u1 equals y1, or that u1 plus u2 equals y2.
So the edges just serve for communication in this graph.
There's another style of graph, which I introduced, and therefore, people -- not me -- but other people have called it a Forney graph.
I call it a normal graph.
The difference is that basically the vertices all equal constraints, and the edges represent variables.
And it has certain advantages that have led to it being used more and more.
Let me just indicate how we go from a graph of this style to a graph of this style.
We basically just make all the variables into equality constraints.
So in this case, I draw y1, y2, and so forth.
I put a little dongle on them to indicate that they communicate with the outside world.
They are observed, external variables.
And then I put a little equality constraint that says all of the variables attached to this constraint have to be the same, which is really doing the same thing.
Otherwise, the topology of the graph looks the same.
These are, again, zero-sum constraints, or parity-check constraints, and the connections are done just as before.
I forget.
I'm not going to do them all out.
The only change, in this case, is to replace external variables by an equality constraint in this.
So the equality constraint basically says this edge still equals y1, this edge still equals y2.
That's what the equality constraint means.
Everything tied to that equality constraint has to be equal.
So we can think of these as just replicas of this original variable out here.
So far, you don't see any reason to prefer this.
If I do it in this graph, then, again, I get equality constraints here.
I get zero-sum constraints, again, over here.
Eight of them.
And in this case, I can just represent the external variables directly by these little dongles.
And again, I have the same graph here.
You still probably don't see that there's very much difference and, in fact, I'd agree.
There's hardly any difference between these two styles.
So forth.
This is the normal graph that's equivalent to this Tanner graph, it's obtained in the same way.
If I wanted to do it more painstakingly, I would put equality constraints for each of these, I'd put little external variables, and then I'd see I don't really need the equality constraints.
So I can just compress it into y1 through y8.
The advantages, I will say, of the normal graph will appear as we go along.
The main advantage is that you can prove a very nice duality theorem for normal graphs -- that basically shows the duality between generator and parity-check constraints.
But to say anything about that right now would just hand-waving, so we'll come back to this next week.
See you in a week, have a nice weekend.
We're into chapter 11.
I've asked Ashish to hand out today a revision of the previous version of chapter 11, plus chapter 12 on the sum-product algorithm, in case we get to it.
I'm giving you a complete copy of the problem seven solutions for both weeks, and to get back on track, we're handing out problem set eight.
There probably will only be one more problem set that you hand in.
All right.
We've been getting to the centerpiece of the second half of the course which is codes on graphs, which is the way we're going to get to capacity-achieving codes by building them on graphs of linear complexity and decoding them on the graphs with iterative decoding.
I frame this in the language of behavioral realizations.
I say a graph is a realization of a code via a behavior, which is the set of all the possible trajectories on the graph.
So this is, again, putting it in system theory language.
I want to distinguish between two types of variables.
The external ones, which communicate with the outside world-- in our case, these will be the symbols in a code word.
These are the ones that are actually involved in the code.
So these are more or less pre-specified for us when we're trying to realize a code.
But we can adjoin auxiliary variables, internal variables, we often call them state variables, hidden variables, latent variables, which are really there for our convenience.
They're to make the realization simpler, or more elegant in some way, or to have properties, or whatever.
But these are free for us to have because they're inside the box and the world never sees them.
Now, we describe the behavior by a set of constraints on the variables, which in our case, these are local constraints that only involves a few of the variables, each one.
The idea is to express a global behavior via local constraints on variables.
And in our case, the constraints will be expressed by equations, as we've seen, or in a little bit more generality by little codes, which will go in to make up this big code.
So in some sense, you can think of codes on graphs as being making big codes out of little codes, making global codes out of local codes.
The behavior, then, is simply defined as the set of all combinations of variables, both external and internal, that satisfy all the constraints.
If we can find a set of values for all the variables such that all these local constraints are satisfied, then that's called a valid trajectory, or whatever.
And that's the entire behavior.
And the code is simply the set of all external variable n-tuples that occur as parts of valid behaviors-- in other words, the projection of the behavior just onto the external variables.
All right, so it's an elegant and rather implicit way of describing the code, but we'll see it's very, very powerful.
And last time, we talked about various styles of realizations.
I gave you examples of generator representations and parity-check representations.
This is really where it all started, with parity-check representations, where this style of graphical representation is very natural.
You've got the symbols in the code, those are the only symbols you need, and you've got parity checks, and the parity checks constrain the symbols.
If you've got n minus k parity checks, you get n minus k constraints.
And it's a kernel representation in the language of linear algebra.
It's simply the set of all code n-tuples that satisfy all the parity checks.
That's it, we don't need any hidden variables.
In a generator representation, this is a more natural representation, when we're trying to generate or simulate a code.
We want to run through a set of inputs that in effect create all the outputs, all the code words.
In this case, we don't actually see the inputs as part of the code words, necessarily.
We might, we might not.
But we distinguish them in this representation and we regard the inputs as hidden variables, or state variables, if you like.
They're part of the realization, but they don't actually occur in the code words, so they fit into the hidden category.
OK, and we talked about two styles of graphs.
Tanner graphs, which are older and more established.
Tanner really wrote the foundation paper in this subject in 1981.
I regard him as the founder of the subject of codes on graphs.
However, like a lot of good papers, his was completely ignored-- that's almost true-- for 15 years.
And then it was rediscovered around 1995, when people began to wake up to this subject again.
And Tanner graphs really come out of this idea.
They were closely associated with that.
We can generalize them to be more like this.
But it's this simple idea.
We have a bipartite graph, we make one class of vertices into the variables, one class into the constraints, and we simply tie a variable to a constraint when it's involved in the constraint.
And we can generalize it by making internal and external variables, which we have to distinguish in the picture in some way.
And the constraints, initially, they were just zero-sum constraints and parity-check codes.
One of the things Tanner did was he said, well, they could be any codes.
A single parity check is like a single parity-check code.
And we can make any codes the constraints.
And the edges don't work very hard in the Tanner graph.
They implicitly carry the variables to the constraints so we could label them with the variables.
And we showed how a Tanner graph could always be transformed to this normal graph, which I think is certainly in more complicated situations, a cleaner way of explaining things, and cleaner in a variety of senses.
Here is our taxonomy.
Vertices indicate constraints, half edges indicate external variables.
A half edge is something you can tie to another half edge to form an I/O pad.
Edges tie together two constraints.
They're internal, and so these represent the internal variables.
And this is cleaner both analytically-- there's this nice duality theorem about normal graphs that I won't be presenting to you, and a couple other things.
They're cleaner when you come to implement the sum-product algorithm.
I think they're just generally cleaner, but I may be biased.
I want you to see both because these still predominate the literature, although more and more people seem to be using this style of graph, and that's what I will use in the course.
All right?
So that's a review of where we are.
OK. Today we're going to go on to other important styles of realizations, and particularly the next one we'll talk about is trellis realizations.
So this will tie back to all the work we did in chapter 10.
And what did we discover in chapter 10?
The key to a trellis realization was a trellis-oriented generator matrix.
And again, I will put up our favorite matrix on the board, the generator of the 8, 4, 4 Reed-Muller code.
And in the form we've come to know and love, it looks like this.
And what makes it trellis-oriented is that all the starting times are different and all the ending times are different.
Therefore these generators are as short as possible.
The main feature of this generator that we want to look at is the spans, the active parts of the generators, which we've made as short as possible.
And let me explicitly write out what they are.
The first generator is active from symbol time 0 to symbol time 3.
This one is active from 1 to 6.
This one is active from 2 to 5.
And this one is active from 4 to 7.
OK, so those are the active spans.
And we know from that, we can find the dimensions of minimal state spaces and of minimal branch spaces.
OK, we just count the number of active generators at either of the state times, which are between the symbols, or the symbol times themselves for the branches.
OK, again, quick review, especially because we have a few guests here who we haven't seen previously in the course.
So that's where we've been.
All right.
Once you have any generator matrix, or in particular a trellis-oriented generator matrix, the code is simply described as the set of all linear combinations of the matrix elements.
And in particular, the code symbol at every time is a linear combination of the matrix elements in a particular column.
All right, so let's use that to get a trellis realization.
I've done this in various ways.
I'm still not satisfied, though the way that I do it in the notes is quite notation heavy and mathematical.
I hope this picture of what we're going to do is going to give you a totally transparent idea of what's going on.
OK, we're going to have a trellis realization.
Let's write down the symbol times.
And I'll write them all the way across the board.
0, 1, 2, 3, 4, 5, 6, 7.
And let me save the coefficients, ui, here.
And I'll write them as equality constraints, but you can also think of them as a little binary shift register.
I'm going to need u1 at four different times.
I'm going to need u1 during its span at time 0, 1, 2, and 3.
So I'll set up a set of constraints here to propagate u1 between these three times.
And u1 is going to be an internal variable.
Strictly, there are three replicas of this internal variable here.
And I can pull it out at each time, too.
So this is where I'm going to keep u1, in effect.
Similarly, u2 I'm going to keep around for all the times that I need it, which is these six times.
OK, so is that six?
That's only five.
These are all the times that u2 is actually involved in the output symbols, so I'm going to have it when I need it.
u3 I need for four times.
I've been inconsistent in whether I write the variables or the values of the variables.
In this case, I'm writing values.
And finally, u4.
I see I'm going to need more than one board here.
So now I've got everything I need at the time that I need it.
Now, this is going to be tricky.
I'm simply going to have a linear combination-- call it g0, g1, and so forth-- which is going to create my output at time 0, or time 1, or time 2, and so forth.
And you'll agree from this generator matrix that the output at time 1 is some linear combination of u1.
And the output at time 2 is some linear combination of these two guys.
OK, and the output at time 3 is some linear combination of these three guys, and so forth.
You see what I'm doing?
So I could describe these as simply a linear constraint.
What are the possible combinations of inputs and outputs in this constraint?
And similarly-- and don't make me draw the lines in.
OK, so I claim this is a realization directly from this trellis-oriented generator matrix.
Yes?
In this example in the generator, g has a 0 [UNINTELLIGIBLE]
Right
So before, the component had to be 0 in the linear combination?
Or are you--
I can do it that way.
Clearly, I don't need this input at this time.
So I could erase that.
Would that make you feel better?
Actually, that gives an even better picture.
It's not going to make any fundamental difference, but I'm doing this for a generic set of generators that cover these spans.
All right?
For the particular ones we have here, yes, I can make that further change and eliminate those two inputs.
But I'm shortly going to aggregate this and put it in a more aggregated form.
And what's happening internally here then won't matter to us very much.
OK. And notice that I can regard this as the branch space at this time.
And it has the right dimension for the branch space at each of these times.
And let me take it a little more slowly.
First, I claim it's a realization.
If I built that thing with the appropriate linear combinations here, then the behavior of this is basically the set of all u's and y's that satisfy all these constraints.
And if I just look at the outputs, that's the set of all code words.
So I've constructed a behavioral realization in a certain way.
OK, and now I'm going to start to aggregate.
Aggregate just means drawing lines around sub-graphs.
Or sometimes in the notes, I call this agglomeration.
So I'm going to regard all this as one constraint on the variables that come out of this.
The constraints that are affected by this agglomeration-- the variables that are affected are y0 and u1.
OK, so this I can draw-- let me draw it this way-- as a little constraint code which operates at time 0.
And it's a linear 2, 1 code that basically ties together y0 and u1, whatever the relationship between them is.
It probably is the y0 equals u1.
The first generator starts at this time.
That's about the only thing it could be.
Similarly, I'm going to aggregate all this part of the realization here and consider it a big block, or a big constraint.
At time 1, there's some constraint that affects y1 and u1 and u2.
So this is a little 4, 2 code on the four incident variables.
One of the constraints is that u1 on this side equals u1 on this side.
These are really two replicas of the same thing.
So there's an equality constraint propagating through there.
Then I have some function of y1 as a function of u1, which I'm going to regard as the state at this time.
And u1 and u2, let me call that state space at time 1.
Let me call this at time 0, let me call this the state space at time 1.
And I did go through a development where I showed that these u's could be regarded as the components of the state spaces.
This is a more constructive way of seeing the same thing.
And notice they have the right dimensions.
So the states are what carry the information from time 0 to time 1, just as we would want.
They kind of embody the Markov property.
Yes?
Why is c0 length 2?
Why is c0--
Length 2
Length 2.
Because it affects two bits
Thank you
It affects these two bits.
Let's see, how did I know that there's only one possible constraint here?
Because that's the dimension of the branch space.
It's really all determined by u1.
Over here, however, it's determined by u1 and u2.
So the dimension is 2. u1 and u2 are free, And the others are fixed once we know those.
So I was a little ahead of myself there.
And similarly, let's keep drawing in this fashion, here's a constraint code at time 2 which relates these possible variables, u1, u2, u3, which I'm going to call the state space at time 2.
So my times are not what I expect.
But it's always really a problem to keep the indexes consistent.
We want state time 1 to occur after the first symbol.
So this should be state time 2, this should be state time 3 to be consistent with what's in the notes.
I'm sorry.
And so forth.
What is the length of this code?
It's simply the number of bits that it controls, which is 6.
1, 2, 3, 4, 5, 6.
What's the dimension?
It's 3, because u1, u2, u3 are free if I only look at this constraint independent of everything else.
OK, and continuing this way, at this point, you notice I only have u2 and u3.
So my state space at time 4 has gone down in dimension.
And here I have c3, which is, again, a 6, 3 code, and so forth, as I go ahead.
Let me just draw it.
This is also a 6, 3.
This is time 5.
I have the state space at time 6.
This is back down to 1, 4.
It's symmetrical on this side, as you know.
So that's my trellis realization.
And I claim this, too, is a realization, in the sense that any combination of u's and y's that satisfy all of these constraints is a legitimate trajectory.
And the set of all y's that are part of those legitimate trajectories form the code.
In fact, it's pretty explicit here that the dimension of the code is 4, corresponding to u1, u2, u3.
So I have 8 outputs, I have 16 possible behaviors.
I pick off the combinations of u's and y's that could make that, which is basically determined by this matrix.
And I just pick off the y's and that's my 8, 4 code.
OK, so that's what trellis looks like.
The constraint code really embodies the branches.
This encapsulates everything that can happen at time 0.
This encapsulates everything that could happen at time 1.
The u's are really my state spaces.
This constrains state, output, next state, in nice linear system theories style.
This tells me what combinations of state, output, next state, I can have at time 2, and so forth across the board.
So the constraints are local constraints-- local in time, in this sense-- that constrain what can happen as you get to the next state.
Once you get to the next state, this state has the Markov property.
This is all that the memory you need of the past in order to determine the future, or the whole set of future possibilities.
Each of these is a linear vector space over the ground field.
In fact, it's just the space of 1-tuples, 2-tuples, 3-tuples, 2-tuples, and so forth.
And it has certain dimension, in this case, equal to its size.
And if you calculate the dimensions of-- call this the state space, call this the branch constraint code if you like, or the branch space, it's isomorphic, the dimensions are minimal here by construction from the trellis-oriented generator matrix.
If we want to know the minimal size of the state space at time 4 here in the center, we just calculate the number of generators that are active at time 4, and that's precisely what we're going to get over there, too, by our construction.
If we want to compute the dimension of the branch space at time 3, it's the number of active generators at symbol time 3.
There are 3 active ones, and we've just forced this to have that structure.
So this is a normal graph of a minimal trellis realization of the 8, 4 code.
OK?
So we can clearly do that for any code, right?
For any code, we can find a trellis-oriented generator matrix.
We can go through these steps, and we'll always come up with something that looks like this down here.
OK, now, what are the properties of this graph?
Does it have cycles?
Does this graph have cycles?
Anybody?
"No," is one answer I get.
What's the other possible answer?
How many people say no, and how many people say yes?
I want to see a show of hands.
How many people think this graph is cycle free?
OK, and how many people think that it has cycles?
OK, the majority is wrong.
This graph clearly has cycles.
Here are two edges here, and here's a cycle.
OK, so as a graph, it has cycles because it has multiple edges going between these constraints.
All right, so a good thing to do is to make this a cycle-free graph, and that's just a matter of regarding this as instead of two binary variables, we regard it as one quaternary variable.
All right, so we're going to regard this as a single binary variable of dimension one.
This is a quaternary state space.
There are 4 possible states here.
So the dimension of the state space at this time is 2.
So when I draw abbreviations like this, you can think of them as being decomposable into binary variables, if you like.
But the advantage of drawing them as larger variables, higher-valued variables, is that now does this graph have a cycle?
No, this graph is cycle free.
I've just gotten rid of all the possible cycles.
How can you tell if a graph is cycle free?
Various ways.
I think the most elegant one is to say if every edge is by itself a cut-set, then the graph is cycle free.
If I remove any edge, then I decompose the graph into two parts.
And that's clearly true of this graph.
We don't really have to test the half edges.
In fact, this is kind of the defining concept of a state space.
A state space is kind of a cut between the past and the future.
When we asked about the minimal state space, we asked what's the minimal dimension of the information, the state, that we need to pass from the past to the future?
And so states correspond to cuts.
If we make each state space, if we consider it to be a single variable, then we get just a chain graph on a sequential time axis, as is conventional in system theory, as your conventional integer-time axis for discrete time systems.
In this case, it only has a finite number of times where anything happens.
But you can think of this as part of the infinite set of integers.
And so this is everything that happens.
And a trellis is always going to be very boring.
It's always going to look like this.
But the advantage of now we have this cycle-free graph realization, we'll find out that we can do exact decoding of cycle-free realizations using the sum-product algorithm, or the min-sum algorithm, which in this case kind of reduces to the Viterbi algorithm with an asterisk on it.
And the cost of doing that is that now have to instead of considering two binary variables here, I have to consider a single variable that has four possible states.
So basically, I'm going to be carrying messages which are vectors indexed by the state space across here.
And in this case, the vector is going to have to have four elements, which are going to be likelihoods or metrics or weights of some kind.
It's basically telling me what's the weight of each of the four possible survivors, if you can think of the trellis that goes with this.
Or here, I'm going to need to carry a vector with 8 elements.
So it takes more to specify a vector with 8 elements than 3 vectors each with two elements.
So by aggregating these, I've created a sort of exponential complexity situation as these state spaces get very big.
But on the other hand, I get cycle freedom.
All right?
Yeah
There's no real difference between this and what you had before, right?
As President Clinton might have said, it depends what the definition of "real" is.
What do you mean by real difference?
I mean, sure.
We think of something u1, u2, you can think of that as two binary variables or a single quaternary variable.
Is there any real difference between that?
Well, when we actually go to the implement decoding algorithms, we'll find there is a real difference in which attitude you take.
But mathematically, it's a distinction with hardly any difference
So then there's no fundamental difference between the cycle freeness and the cycle--
No, you'll see this real difference-- there is another real, significant difference.
If I drew this as two binary variables, I have a graph with cycles.
And we'll find that I can't do exact decoding with a sum-product algorithm on a graph with cycles.
So if I tried to apply that algorithm to this, I'd sort of iterate around this little cycle.
Whereas if I agglomerate them into a single variable, I get rid of that behavior.
And I can summarize everything in four values.
So in that sense, it's a huge difference
Can I always transform this whole graph into cycle-free graph using this kind of technique?
I didn't get the first part of your question
Can I always transform the graph with cycles to a cycle-free graph using this-- combining those--
Yeah, OK.
So this agglomeration technique of drawing lines around sub-graphs and then considering everything inside there to be a constraint, and all the variables coming out-- well, it depends on their topology, but I can group the variables coming out however I want-- yes, I can always do that
So we can always do that for things [UNINTELLIGIBLE] transform the graph into cycle free
Right.
But we may find that to make it cycle free, I then have to aggregate all the edges into a single edge between any two parts of the graph.
And that may radically increase the complexity.
And I'll give you several examples of that today.
So these little fine points all of a sudden loom large when we actually come to build something.
Let me give you another example of that.
A very fine example would be to say, what does sectionalization consist of?
We talked about sectionalization.
Suppose we want to get a trellis graph for a 4-section trellis, where we take pairs of variables.
Well, the graph realization of that is simply obtained by agglomerating pairs of these blocks, like that.
OK, so let me do that, and then let me see what I've got.
I've now got here, a constraint that affects two variables.
Well, first of all, I've now got only two visible bits in each of the state variables.
Here I have u1, u2, here I have u2, u3, here i have u3, u4.
So we get rid of some of this state complexity.
We did this trick in another way before, by sectionalization.
So we get rid of a lot of state spaces, including ones that are big.
Let's see, what do we have here?
This is now a code which has got two bits coming in, two bits coming out.
It's a 4, 2 code.
It's basically controlled by u1 and u2.
Probably y0 and y1 are some simple linear function of u1 and u2.
So this is a 4, 2 constraint code.
It controls these two bits and these two state bits, these two symbol bits, and these two state bits.
What is this over here now?
We've got 2, 2, 2.
So this is going to be a code of length 6.
And what dimension?
This is, remember, u2, u3.
So this whole behavior here is affected by u1, u2, and u3, it has dimension 3.
Or there's a shortcut I can do here, because this is a self-dual code, all of these little codes are going to be self-dual.
Well, they're not going to be self-dual, but they're going to be rate 1/2.
Half as many bits here as they do here.
But that's another duality theorem that we won't prove.
OK, and symmetrically, we get another code here, which is a 6, 3 code, and another one here.
So we can do this.
The reason I call this styles of realization is there's obviously a lot of freedom in how we want to depict the code.
And depending on how we depict it, when we get to decoding algorithms, it may affect the complexity of the algorithms.
So we want to find a nice way of depicting it
I don't understand why you-- so you want the ... freedom ... is [INAUDIBLE], because you would only have two [UNINTELLIGIBLE] to be used [UNINTELLIGIBLE].
And then you have two [UNINTELLIGIBLE] u1 and u2.
[UNINTELLIGIBLE] u2 and 3
There are two bits here, two bits here, two bits here
But that's 6.
That's 6.
[INAUDIBLE]
Weight 6, all right
[INAUDIBLE] freedom that you talk about
How many possibilities are there for these 6 bits?
I've got to consider all possible combinations of u1, u2, and u3 to drive what's happening in these two times.
If I go back here and look at those two times, I see that there are three generators that I've got to consider, these three.
All right, so for all eight possible linear combinations of those three generators, I'll get different combinations of these six bits here.
And I can go further.
Actually, at the end of the day, I concluded here, we said the idea of sectionalization was to sectionalize as far as possible without increasing the branch complexity, which we've now translated into this constraint code complexity.
And so it's even better to just keep aggregating.
Consider the first half.
And this is 1, 2, 3, 4, two bits there is still only a 6, 3 code.
And the second half is still only a 6, 3 code.
When we get to the sum-product algorithm, that means we have to compute eight things when we get to this node.
Maybe it's a slightly more complicated thing, but as long as we keep the dimension down to 3, we're only going to have to compute eight things.
So we haven't really increased the decoding complexity at all by doing this.
So that we consider to be our best sectionalization.
And if we tried to aggregate these two, what would we get?
We'd simply get the eight bits here, we get a constraint that says it's an 8, 4 constraint, it says these eight bits have got to be in the code.
So we only go that far, because that would increase the complexity to 16.
All right, so that's our minimal trellis.
We call it a two-section trellis where the symbol bits have been grouped into 4-tuples.
And for decoding, that's the simplest one.
So you see what kind of games we can play here.
Yeah?
At the beginning of this process, you started at the [UNINTELLIGIBLE]
Well, you remember in more detail how we concluded this was the optimal sectionalization.
We looked at time 3, in particular.
And we said there are three generators that are active at time 3.
So I'm going to expand that as far as I can without bringing any more generators into the picture.
So I can't expand it over here, because I'll hit u4.
But I can expand it as far as I want over here.
And that's the way we sectionalize
So do you always start with [UNINTELLIGIBLE]?
Well, there's some art in this.
Here it's kind of obvious where to do it.
And I give two heuristic algorithms in the notes on sectionalization, which we'll come up with basically the same thing.
But for simple codes, you can just eyeball it.
OK, so let's see.
That's another general and very useful style of realization.
It's a trellis realization, or a sectionalized trellis realization.
Is there anything else I wanted to say about that?
We can do it so that we get minimal branch and state spaces, or constraint complexities.
It's cycle free if we aggregate the states into a single state space.
Sectionalization, we talked about that.
OK, so let me leave that up, because we're of course not done with that.
Now let me talk about some general properties of graph realizations.
And the most important thing I'm going to talk about here is the cut-set bound.
I want to get across the idea that a graph really captures dependency relationships.
If variables are incident on the same node, they're obviously all dependent.
Or if you think of larger sets of variables aggregated in this way, then they're somehow dependent.
Variables that are very far away on the graph are less dependent on each other than ones that are close to each other on the graph.
This is all kind of vague and woolly statements.
Let's see if we can make them more concrete.
First of all, there's a very simple but important observation that -- I'll put it this way-- disconnected if and only if independent.
What does this mean?
If I have a disconnected graph, let's suppose I have one graph over here with certain external variables.
Let me aggregate them all into y1 and some constraint 1 over here, and a completely separate graph, some different code over here, c2, constraining some separate set of variables, y2.
OK, that's a disconnected graph realization.
It has to look like that, right?
So we're aggregating the two disconnected halves.
All right, what can I say?
What code does this graph realize?
This graph, the code that it realizes is simply the Cartesian product of c1 and c2.
In other words, this means the set of all pairs, c1, c2, or I should say y1, y2-- y1, y2 such that y1 is in code 1 and y2 is in code 2.
That is the set of all y1, y2 that satisfy all these constraints, right?
So it's a behavioral realization of a Cartesian product code.
Now this is really a notion of independence, right?
This independently realizes a code word from c1 in this part and a code word from c2 in this part.
What would the generator matrix look like?
The generator matrix from this would look like a generator for c1, 0, this is a 0, and a generator for c2.
So there's a generator matrix notion of independence.
In other words, these are just two independent codes that for some reason, we choose to regard as one code.
And the same thing is true that if we have a Cartesian product code, simply a code made up of two independent components, then we can always realize it in this way, right?
We just realize c1, realize c2.
So that's elementary, but it begins to get across the idea that graph properties have to do with dependence properties.
And here's the most radical and simple form of that.
If we have a disconnected graph, then it really realizes two independent codes.
Now, more important than that is the cut-set bound.
What is a cut-set?
It's a set of edges.
Actually, in graph theory, it's defined in various ways.
It could be vertices, it could be edges.
Here, we're going to say it's a set of edges whose removal disconnects the graph.
Probably there are people in this room who know more graph theory than I do?
Does anyone want to quibble or refine that in any way?
That's my idea of what a cut-set is.
Any elaboration?
No?
All right.
All right, so we have some large graph realization.
It's got vertices which I always draw as blocks, its constraint codes, they have various interrelationships.
I'm just showing one.
We have various external variables that we bring out of that, and that's the general graphical realization of the code.
What are some cut-sets in here?
I've drawn this, I'm thinking of this as a cut set.
If I take these two edges and remove them, then I've disconnected the graph.
So they form a cut-set.
I've already mentioned the very close connection between cut-sets and cycle-free graphs, which is the graph is cycle-free if and only if by removing any single edge, I disconnect the graph.
So every single edge is itself a cut-set.
That's a very elegant characterization.
All right, so I'm thinking of a cut-set.
I'm going to write it as chi for scissors.
And it's a set of edges.
So in our cases, what are the edges?
It's a set of state spaces for some index set.
OK, so I'm going to select some minimal set of state spaces that disconnects the graph.
All my edges are internal variables.
We don't really need to worry about these half edges out here, as I've said before.
They're always going to trivially separate their variables from the rest of the graph, and it's just tedious to try to keep them in the explanation.
So we're only talking about state-edges here.
All right, once I've done that, let me now do my agglomeration trick.
Sorry, I'm trying to draw a dotted line around a part of the graph that I'm going to leave outside the external variables that I still want to be external.
I'm going to leave the state variables, which I'm going to remember were once connected.
And let me reconnect them now.
So I'm taking my original graph and I'm dividing it according to this cut-set into two parts.
And I'm arbitrarily going to call this the past, p-- arbitrarily but suggestively-- and the future, f. And it doesn't matter which one I call p and which one f. And I'm going to redraw this, so this is just a re-drawing of that in a nicer form.
I'm going to aggregate all of these variables over here.
And I'm going to call that the set of past external variables, y projected on the past.
And similarly over here, I'll get the set of all external variables projected on the future.
It was just the ones that occur when I make this kind of cut through state edges.
Some of the external variables wind up in the past side, and some wind up connected to the future side.
But there are obviously none connected to both because the definition of a cut-set.
We've disconnected the graph.
All right, so there's the past.
There's the future.
And here are the state variables.
Now I'm going to define these all together as a super-state space as I did in the trellis graph over there.
So let's see, here I labeled the edges by a set of states.
But what I mean here is that the super-state space is just the product of the component state spaces.
It's elements s chi are just the set of sj, j, and sigma-j, is that clear?
In other words, I have a vector of states here.
And I consider the super-state space to be just the Cartesian-- this, again, is the Cartesian product of all the state spaces.
OK, and then I have constraints.
So we'll call this the aggregate past constraint, and the aggregate future constraint.
This constrains these variables in this state.
This constrains these variables in that state.
OK, so that's where figure 5 comes from.
Every year, people say, how did you get figure 5?
That's how I get figure 5.
Is there any confusion about that?
Today I've been talking about agglomeration quite a bit, so maybe this year it'll come through better than in past years
[INAUDIBLE] that we see projected on it.
Or is it just--
It's just the aggregate of all these constraints.
What does this say?
It's the set of all the possible values of these variables, and these variables that can actually occur.
That forms a linear space which we call little code.
Anything consistent with that satisfies this constraint.
Anything else doesn't
[INAUDIBLE] for the c's projection of the code on the past?
No.
The projection of the code on the past would be you realize the code and you just project onto these variables here.
So no, this is just an index.
This is a constraint whose index is the past.
OK, well, all right.
Oh, but this looks like a trellis, like a two-section trellis.
It looks very much like this thing right here.
It has the same general form.
And we can think of this super-state variable here, it sort of has the Markov property.
It, again, tells everything about the past that is needed to specify what possible futures there could be.
We're really interested in these up here, eventually.
But this is the whole set of constraints that determine the futures along with-- there may be some more free variables over here.
But these are the constraints we get from this half of the code on this half of the code and vice versa.
So again, a cut-set is related to a Markov property.
Let me try to state this property more explicitly.
The code by the behavioral realization is simply the set of all combinations of -- let me say, the behavior is the set of all past states and futures that satisfy the constraints.
OK, so this gives me a certain set of pasts that are consistent with each possible value of the state variable.
Let's call that y projected on the past that's consistent with a particular value of the state variable, the super-state, the vector of states here.
And these are the ones in the future that are consistent with that state variable.
We define that so the set of all possible past and future y's is simply this Cartesian product.
And we say that in a more graph theoretic way.
If I specify the state here as some particular value, sx, and I fix that, I've really disconnected the graph.
For a fixed sx, I can just put that over here, put that over here.
And this realizes, this connected graph that realizes the Cartesian product of whatever -- yp of sx.
This just comes through and affects that with yf of sx.
So for any particular value of the state, I have a certain Cartesian product of pasts and futures they can occur.
Any past that's connected with that state can be connected to any future that's connected with this state.
All the possible future continuations of any particular past in this equivalence class are the same.
This should sound very much like the kinds of things we were talking about when we did the state space theorem.
And we get a sort of state space theorem out of this.
Just to draw this out, we kind of get a two section trellis realization where we have the various states in the state space here, s0, s1, and so forth.
We have the pasts that are connected with s0, s1, and so forth.
And we have the futures that are connected to s0, s1.
So we could get from this, this picture of the two sectioned trellis.
This really represents a set of parallel transitions here.
These are all the possible pasts that are consistent with that state.
These are all the futures that are consistent with that state.
We can tie them together, and that realizes the code.
I should say explicitly, the code now is the union over the states in the state space, the super-state space of the pasts there times the futures.
They're consistent with the state.
So it's a union of Cartesian products.
And that's expressed by this diagram.
This is one of them.
This is another one.
These are all of them.
How many are there?
The size of the super state variable, which by the way is the product of the sizes of the state spaces of each of these individual state spaces that I had coming across here.
OK. What's the cut-set bound then?
For linear codes, it's basically the same as the state space theorem.
The state space theorem said that the dimension of the state space at any time -- well, let me consider now this state space, the super-state variable.
The dimension of this super-state variable has got to be greater than or equal to the dimension of the code that we're realizing minus the dimension of the -- this is going to be bad notation now.
Now I mean the sub-code, so I should have written something else for the constraint here.
Minus the dimension of the sub-code whose support is entirely on the past, minus the dimension of the sub-code whose support is entirely on the future.
And the state space theorem still applies for this partition.
So that gives me a lower bound on the total dimension of the states in any cut-set.
So for example, in the 8, 4 code, suppose we have any realization, so some graph here, and any cut-set such that the size of -- I should say the dimension.
I really mean that there are four external variables in the past and in the future.
This is dimension, dimension.
In other words, we have four variables sticking out here, and four variables sticking out here, and we have a cut set somehow going through there.
So our picture now is going to look something like this.
What's the minimum possible dimension of the total dimension of the edges that cross this cut-set, the state spaces that are in this cut-set.
We know this code very well now.
Let's do the Muder bound.
What's the maximum dimension?
The dimension of the code is 4.
What's the maximum dimension of any code that lives on the past?
It's 1, because such a code has to have minimum distance 4.
So it can't have dimension greater than the repetition code of length 4.
So 4 minus 1, same argument for the future, equals 2.
So it says that however you do it, you're going to have to have at least two binary edges or one quaternary edge crossing a cut-set that divides the external variables into two equal sized parts.
So let's go further.
Let's suppose we want a cycle-free graph.
OK, if we have a cycle-free graph, then all of the cut-sets are single edges.
All right, so in a cycle-free graph, when we make a cut, it's through a single edge.
And we're saying the state space if there's a way you can make a cut that divides this into two equal parts, the state space has to have dimension 2, it has to have size 4 for any cycle-free graph.
Similarly, if we have the 24, 12, 8 code, we've proved that the minimum dimension of any central cut has to be at least 6.
No, in the center, it has to be at least 8, right?
Yeah, because the dimension of the code is 12, and a sub-code that has support on 12 can't have dimension more than 2.
This can't have dimension more than 2.
So that would be 8.
So for the 24, 12, 8 code, the set of edges that cross the cut have to have total dimension at least 8.
And if we want it to be cycle free, then there has to be a single edge that has dimension 8.
So the general conclusions -- we first talked about trellis realizations.
We can generalize that any cycle-free realization-- and we're going to see that we have strong motivation to keep our realization cycle-free, because then we can do exact maximum likelihood decoding.
So we'd like to have cycle-free realizations.
But what have we just seen?
We've just seen that the state complexity cannot be less than that of a trellis realization with a comparable cut.
In other words, one that divides the past and future in the same way, divides the external variables into the same size.
OK, so we really can't beat the trellis bounds, the Muder bound, in particular.
Now again, there's some games we can play.
Let's ask about the 24, 12, 8 cut.
We've said if we want to realize it with the cycle-free realization, this says, well, if we have a cut that divides the external variables into two parts of size 8, then we're going to be stuck with a state space of dimension 8 and a size 256.
No way around it.
Here's a slight way around it.
Let me hypothesize a realization that looks like this.
This is a 14, 7 code.
So is this.
So is this.
We're going to divide the external variables into three parts, each of size 8.
And we're going to have three internal state spaces, each of dimension 6.
And we're going to constrain those by an 18, 9 code.
And again, these all have rate 1/2 because this code is dual.
Now, is there anything in what I've done that prevents this kind of realization?
Let's test the cut-sets.
Where are the cut-sets in-- well, A, is this cycle-free realization?
Yes, OK, good.
Let's test the cut-sets.
It's kind of a three-way symmetrical thing.
The internal cut-sets are here, here, and here.
Each of these cut-sets divides the coordinates into one set of size 8 and one set of size 16.
In that case, what does the Muder bound give us?
In that case, 8, 16, the dimension of the code is still 12.
The minimum dimension of a code of length 16 and minimum distance 8 is 5.
And one of length 8 and minimum distance 8 is 1.
So as we did on the homework, we find that we are permitted to have a state space-- the minimal state space here could have dimension 6.
So this possibly could occur.
Again, we're kind of camouflaging some of the state space.
You see there is no central state space in this.
There is no cut-set that partitions it into 12 and 12.
So by being a little bit clever, we've got it down to something where instead of a 256 central state space, we've got three 64 state spaces.
And this is what we saw on the trellis realization too, that if we sectionalize into three sections, then we get a 64 state and a 64 state at the boundaries between the three sections, but we still got the same 512 branch complexity in the middle.
And what do you know?
We still have a constraint code with complexity 512 here, too.
So if you consider that the more valid measure of complexity, the minimum dimension of any constraint code or branch space, then we haven't improved.
So subject to these qualifiers, this is a pretty strong argument that by going beyond trellis realizations to general cycle-free, graphical realizations, we can't gain very much.
So the moral is to significantly reduce complexity, we're going to need to go to graphs with cycles.
We must go to graphs with cycles.
OK, so you don't get much for free in this life.
So when we get to our sum-product decoding algorithm, we're going to have to apply it to graphs with cycles where it's iterative, where it's not exact, where it's just a lot less nice.
So what do we do when we go to graphs with cycles?
Where is the potential gain?
The potential gain is that now this super-state variable could be made up of a number of simpler variables.
We've got a certain minimum dimension that we're going to need any cut-set, like 8 for the central state space of the Golay code.
But now we can spread it around over two or more sub-state variables.
So for instance, maybe we can find something where we now have a graph with cycles where we have two state spaces here.
Here's the cycle again.
And the two state spaces each have size only 16.
That would still satisfy this cut-set bound.
So we greatly reduce the size of the state spaces, which is exponential in their dimension, but at the cost of introducing a cycle.
I insist that's a cycle.
Let's go over to this trellis here.
The next thing I might talk about is tail-biting trellis realizations.
This is very simple.
What's the first realization you would think of on a graph with a cycle?
Well, here's a trellis realization.
It's always going to look like this.
And then let's just loop this around.
In other words, we make the last state space equal to the first state space.
We don't require they used to be trivial state spaces of size 1.
We allow them to have some dimension.
And here's a potential realization on a graph with a cycle.
Cut-sets.
Where are the cut-sets in this graph?
Well, the cut-sets now look like that.
All the cut-sets involve at least two edges, two state spaces.
OK, so we potentially might get this kind of benefit.
Here's an example.
Suppose instead, go back to our very favorite example, suppose we let u2 go across here, and we bring out u3 out of here, and we bring it all the way back to here.
Everything still good?
It's obviously another realization.
Now I've made it clear that when I draw u2 and u3 separately here, I've really got a cycle.
I've made a big cycle.
And now what's the number of states in this two-section realization of the 8, 4 code?
This is a two-state tail-biting trellis, because each of these is just a little binary variable.
So I get some kind of two-state trellis.
If I were actually to draw out the trellis, I would have two states at time 0, two states at time 4, some kind of -- what have I got-- 6, 3.
So I've got 8 branches that go back and forth, a couple of parallel branches.
It's going to look like that.
Then same thing out to time 8.
I don't have a time 8.
This is really time 0 again, where I identify these two state spaces.
This is really the same state as this, and this is the same one as that.
So that's if I really drew out the trellis in the style I originally drew trellises, this would be a picture of all the-- there's a 1-to-1 map between all the paths through this trellis that start and end in the same state, because that's what I mean when I identify the beginning and end states.
So if I start in this state, there's a four-way branch here, another four-way branch there.
Is that right?
No, there's only a two-way branch such that I can get back to the initial state.
So there are 8 possible paths that start here and get back to the same state, 8 possible paths that start out from here and get back to the same state.
And they together correspond 1-to-1 to all the code words.
OK, well, that's not a very big deal.
I was able to reduce a four-state trellis to a two-state trellis.
But suppose I do the same thing with the Golay code.
In the Golay code, there's a very beautiful tail-biting trellis realization.
It looks like this.
It has 12 sections, so maybe I won't draw all the sections.
It groups each of the output variables into pairs and comes around like that.
And each of these state spaces has dimension 4, or size 16.
And I give the generator matrix for this in the notes.
So this is for the 24, 12, 8 code, and now you test me.
Is this a possible-- use the cut-set bound.
See if this violates the cut-set bound anywhere.
Does it for this code?
No, it doesn't.
Because every cut-set, no matter how I draw it, we're going to get two edges, each with dimension 4, adding up to a super-state of dimension 8.
And the state spaces at all the even times, notice I've sectionalized here, so I only need to look at even times.
The state spaces at all even times in this code could have dimensions as small as 8.
Remember, at the odd times, they go up to 9.
And these are all little-- 4, 4-- these are all little 10, 5 codes.
So the branch complexity is only 32.
So this is very much simpler than any of the conventional trellises that I've-- than the minimal conventional trellis that we were able to draw for this code.
Or if this funny little pinwheel with three arms on it, which is also a cycle-free realization, this code.
OK, so they're going to tail-biting.
That's a significant advance.
An aside note-- if we just break this, it turns out we have a generator matrix that has period 4, and let this go on infinitely, this is a realization of a rate 1/2, 16 state, branch complexity 32, as we expect-- perfectly conventional, except it's periodically time varying-- linear convolutional code with absolutely phenomenal performance.
Like the Golay code, it has d equals 8.
Therefore it has a nominal coding gain of 4, 1/2 times 8, or 6 dB.
This is with only a 16 state, rate 1/2 code.
Look at the ones on the table.
This is significantly better than that.
And it's effective coding gain, I forget.
But it's still very good, excellent, for the complexity of this code.
So this is called the Golay convolutional code.
And it's an interesting example of the interplay between this block coding stuff and the convolutional coding stuff, in this case.
Actually this code was first discovered in computer searches, but nobody realized how wonderful it was.
And it was only when its connection with this Golay code was recognized that people realized this was really a special code, both from a performance versus complexity point of view, and also algebraically.
OK, well that brings us to the end.
There's actually one more style of graph realization that I want to mention to you, for the Reed-Muller codes.
We know that they're basically based on Hadamard transforms.
It's a very nice kind of Fourier transform like graph realization of Hadamard transforms.
So that gives us another style of realization of Reed-Muller codes that again, we can aggregate to lead to a lot of nice structures.
Again, at the end of the day, unless we allow cycles, we don't really gain anything.
So I'll do that the first part of next time and then do the sum-product algorithm.
I hope I can do it in one lecture.
--solutions for problem set 8.
I found some problems as I went through them.
These are all new because we've never been in this territory before this early where we could actually offer problems.
So it's a little bit of a shakedown.
8.2, simply, I don't think you have enough background to actually do it.
If you want to try it, be my guest.
I'll give you a sketch of a solution, anyway.
But really, for the same reason that I didn't do exercise 3 in Chapter 11, we really shouldn't have done exercise 2, either.
For 8.3, which is a decoding exercise using the BCJR algorithm.
I don't know if we'll get to the BCJR algorithm today.
If we do, then please do it.
If we don't, then we can defer it.
And you also need to know that the noise variance is 1.
Or, pick something, but I suggest you just pick sigma squared equals 1, even though that's a large value.
OK, any questions?
Has anyone tried to do the homework yet?
Not yet.
Good strategy.
OK, Chapter 11.
We have one topic to go in Chapter 11, which is just the last little bit on Hadamard transform realizations, which I do want to cover.
We've really been talking about styles of realizations on codes on graphs, graphical realizations.
We had the generator.
And we had the parity check and we had the trellis-style.
And we even had a tail-biting trellis.
And we also developed the cut-set bound which doesn't exactly prove, but strongly suggests that we're going to need cycles in our graphical representations.
And one reason I want to talk about the Hadamard transform realizations, the Reed-Muller codes, is that they really demonstrate this point I think very nicely.
If we allow cycles, we get very simple representations.
If we demand cycle-free, then we get still pretty nice realizations, but much more complicated.
They meet the cut-set bound everywhere.
Reed-Muller codes in general are pretty good anyway for trellis realizations, but they're certainly much more complicated than the realizations with cycles.
So that's what we're going to do today is-- at least the first part is Hadamard transform realizations of Reed-Muller codes.
OK, what's the Hadamard transform over the binary field F2?
It's related to but a little different than the Hadamard transform over the real field.
So people seeing the latter, you'll see similarities, but you may see differences as well.
So the Hadamard transform.
I think you remember when we defined Reed-Muller codes, one of the ways we defined them was in terms of a universal matrix Um, which I defined as the m-fold tensor product, very briefly written like that, of a 2 by 2 matrix, u1.
What does u1 look like?
u1 looks like-- over the binary field, it just looks like this.
It's the set of generators for the Reed-Muller codes of length 2.
What you would expect, it's a lower triangular matrix.
And if we want to take-- I don't know if you've run into tensor products before.
If we want to take the 2 by 2 tensor product like that, we simply replace each one by the matrix itself and each 0 by a matrix of 0's.
So we get a 4 by 4 matrix which looks like this 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0.
And what do you know?
That's what we called our universal Reed-Muller generator matrix for m equals 2.
In other words, for Reed-Muller codes of length 4.
And you see how all the Reed-Muller codes are generated.
We can find generators for all the Reed-Muller codes from this set.
And similarly, u3 was this matrix that we've now seen many times.
So all these matrices have common properties.
They're all lower triangular.
They have many properties.
One of them is that all of them are self-inverse.
And that's very easy to prove from the fact that u1 is clearly self-inverse.
u1 times u1 is the identity matrix.
And because these are all tensor products, that makes all of these their own inverses.
So what's the 2 to the m by 2 to the m Hadamard transform is if we have a vector u, then u times u m, a vector y is called the Hadamard transform of u.
What's the inverse Hadamard transform if you like?
Well, since u m is its own inverse, if we just multiply both sides by u, we get u equals y u m. So to get u back, we just transform y again by u m. So the inverse Hadamard transform is the Hadamard transform again in this case.
So we get the typical kind of transform relationship.
You can think of this time domain, frequency domain.
Anyway, this is the set of variables from which we can get this set of variables.
They're somehow in dual domains.
OK. Well, now recall one definition of the Reed-Muller codes.
The Reed-Muller codes are-- Rm of mr is-- sorry, it's always rm, which I've always found curious notation.
Since m is more important than r, it would seem that you would say m first, but maybe it was for this, I don't know.
This is generated by-- so let's say that.
The rows of u m of weight 2 to the m minus r [?
for ?]
greater.
So for instance, the 0 order Reed-Muller code of length 8 is simply the 8, 1, 8 repetition code and it's generated by this last row.
It's the only one of weight 8.
If we want the Reed-Muller code of length 8 and minimum distance 4, we take this, this, this, and this as our four generator rows and we get the 8, 4, 4 code that we've been mostly using for examples.
If we want the single parity check code of length 8, the 8, 7, 2 code, we take all these 7 generators, everyone except for the unique weight 1 generator up here.
And as a hint for doing the homework, it's helpful to notice that if we write the indices of these rows as 0, 1, 2, 3, 4, 5, 6, 7, then notice there's a relationship between the Hamming weight of the m bit binary expansion of these indices and the weights of these rows.
Is that clear instantly?
I won't prove it, but if the Hamming weight is 3, then the row has weight 8.
If the Hamming weight is 2, then the row has weight 4.
If the hamming weight is 1, then the row has weight 2.
And if the hamming weight is 0, then the row has weight 1.
And that's the easiest way I've found to do problem 8.1.
So these-- there are infinity of relations, nice relations, you could prove for these things.
They're very important combinatorially and so forth.
OK, so from this observation, we can therefore define a Reed-Muller code as the set of all u times u m where we simply let some of the ui free for some rows.
In other words, it's just a binary variable.
And 0 for other rows as I just described in words over here.
So basically, where we're going is we're going to define some kind of graph which realizes u m. We're going to put the 8-- do it either way.
I'm going to put y0 y7 over here.
If we're doing a code of length 8, I'll put u3 in here.
We're going to have 8 other things sticking out over here.
I shouldn't draw these because they're really going to be internal, implicit variables.
So this will be u0 through u7 in some order.
It turns out to be necessary to play with the order to get a nice-looking graph.
And we're going to fix some of these to 0.
For instance, for the 8, 4, 4 code, we're going to fix those to 0.
And we're going to allow these coefficients corresponding to rows of weight 4 or more be free arbitrary binary variables.
So the set of all code words will be generated as the set of all Hadamard transforms of 8 tuples, which are fixed to 0 in these 4 positions, and which are free in these 4 positions.
Is that clearly stated?
I think I'm reasonably proud of that.
So that's what I mean by this notation here.
And it's clear that the dimension of the code is the number of free coefficients here.
Basically, because the u m matrix is invertible, so we're going to get an isomorphism between 4 tuples and code words.
So that's a way of constructing Reed-Muller codes.
Now, how do I get a nice graphical representation of u m?
Let's work on that for the next little bit.
Well, as always with Reed-Muller codes, we should do things recursively.
We should start with u1.
That's a pretty simple relationship.
How do we draw a graph for the relationship y equals u1 u?
OK, that means y0 equals u0.
Let's see.
Is that right?
I've been putting u on the left here to imitate what we do with generator matrices.
So that equation is y0, y1 equals u0, u1.
Sometimes I have problems with basic things like this
[INAUDIBLE]
Yeah, I want row vectors.
Sorry.
OK, so that's the equation, which we can see that says y0 equals u0 and y1 equals u0 plus u1.
Do I agree with that?
y1 equals-- no, still not right
y0 is [INAUDIBLE]
Plus u1.
And y2 is-- y1 is u1.
All right.
Sorry, now we're in business.
OK, so let's draw a little graph of that.
We have y1 equals u1.
And if I make an equals down here, I'll get a little u1 in here.
So I can put u0 in here and get out y0 equals u0 plus u1.
OK, so that's a graphical realization of this Hadamard transform.
If I put 2 bits in here for u, then I'll get the Hadamard transform out here for y.
But by the way, notice there aren't any arrows here.
This is a behavioral realization.
And I could equally well put a 2 tuple here on the y's and I'd get out the Hadamard transform at the u's.
It goes either way.
And this I find somewhat interesting is called a controlled NOT gate.
The idea here is that this bottom variable is the control and it determines whether this variable is inverted or not as it goes across here.
And this is a basic gate in quantum mechanics.
Maybe the basic gate in quantum mechanical realizations.
[INAUDIBLE] at least [INAUDIBLE] cubits.
But anyway, that's actually neither here nor there.
All right, so that's how we realize u1.
All right, now recursively there must be some way of putting together little blocks like this to make bigger blocks, say, for u2.
To realize u2 or u3.
OK, so for u2, let me just try to steam ahead and hope this will work.
We build another gate to do a set of transforms to an intermediate variable here.
Intermediate variables, and then we sort of do a butterfly.
We're going to build this out of 4 little blocks like this.
We're going to write like this.
And how are we going to tie them together?
The outputs here are going to go to the equals signs.
The outputs here are going to go to the plus signs.
And we get something that looks like that for the 2 by 2 Hadamard transform.
And y0, y1, y2, y3.
Now, probably this should be u2 and this should be u1 just from the way these works.
And if we do that, we get what?
This is u3.
This is u1 plus u3.
This is u2.
This u0 plus u2.
So we get that-- doing it backwards, y3 equals u3, y2 equals u2 plus u3, y1 equals u1 plus u3, and y0 equals u0 plus u2 plus u1 plus u3.
And that's probably correct, but it's something like that.
Done in the notes.
And those of you who have ever looked at the fast Fourier transform will see some resemblance here to the butterflies that occur in a fast Fourier transform.
And it's not surprising because these are all based on groups of-- in this case, z2 squared.
And so they have the same algebraic structure underlying them.
But that's, again, a side comment that I won't take time to justify.
And similarly for 8, if we want to do the 8-- there's a picture in the notes-- we build 8 of these gates.
Maybe I actually have to do it because I want to reduce it also.
So 1, 2, 3, 4.
Sorry, this takes a little time.
This is where making up slides ahead of time is a good idea.
All right, so we have three levels of 2 by 2 Hadamard transforms as you might expect.
At each level, we do four 2 by 2 Hadamard transforms.
And we connect them together.
At the first level, we stay within the half.
We connect them together in the same way.
And at the next level, we have to spread out over the whole thing, so we get-- this goes up to this equals.
This goes up to this equals.
This goes up to that equals.
This comes down here.
This comes down here.
This comes down here and this goes across.
That gives y0, y1, y2, and so forth.
y3, y4, y5, y6, y7.
And again, we have to jigger with the order here.
Probably something like this.
OK, in any case, it's done correctly in the notes.
All right, so this executes an 8 by 8 Hadamard transform.
You put y's on here and you get the Hadamard transform out on the u's, or vice versa.
You with me?
OK. Notice that the components here are all very simple.
I've got a graphical realization of the Hadamard transform relationship that involves only binary variables, first of all.
All the internal variables here are just bits.
They're all binary.
And all of the constraint nodes are these simple little degree 3 constraint nodes.
The only two nontrivial ones you will probably ever see.
Namely, the zero sum type of node and the equality type of node.
3, 2, 2 and 3, 1, 1 if we consider them as codes.
So it's made up of extremely simple elements.
And in that sense, we can say it's a simple randomization.
But of course, this graph is full of cycles, isn't that right?
Yes.
These little things cause cycles.
So we get cycles as we go around like that, for instance.
Nonetheless, it's a perfectly-- you put something over here and you get something deterministic out over here.
All right.
So to realize the 8, 4, 4 code-- are you all with me on this?
I should check again.
Is there anybody who doesn't understand this?
That's always the way to put the question.
Yes?
You count the output when you're counting degree?
Do I count the output when I'm counting degree?
In this case, yes.
I reserve my options on that.
Sometimes I do, sometimes I don't.
It's whatever is convenient.
My choice.
Sorry, I think I just whacked the microphone.
OK.
So now, let me go to my Reed-Muller code realization, the 8, 4, 4 code.
How are we going to realize that?
We've got these internal variables here, some of which we're going to hold to 0.
These four we're going to hold to 0.
And the rest we're going to let go free.
And so I claim that already now I have a realization of the 8, 4, 4 code, if I regard these as internal variables, these as my external variables over here.
OK, does everyone find that plausible at least?
OK.
But having done that-- now, watch this part closely because this part you're going to have to imitate on the homework.
We can do some reductions.
All right.
If I have a 2 by 2 Hadamard transform of 0, 0, what's the result?
0, 0.
OK.
I can do that in several steps, but that's a good way to do it.
So I don't really need these gates here.
I can just put 0's over here.
Similarly, down here if I have a 2 by 2 Hadamard transform of 2 free variables, internal variables, what do I get?
I get two free variables.
One of them happens to be u3 plus u7 and the other one is just u7.
I could call this u3 prime, let's say.
And since it's an internal variable that I'm just using to generate the code, I can delete that, too.
So what can I do about these here?
When I have one free variable and one zero, what's going to happen?
This free variable is going to come through here and here.
The 0 doesn't count here, so a plus just becomes a repetition in effect.
It's got to have the same parity as this input, so it is that input.
So by another major sweep, I'll just-- this is an internal variable.
I'll just call this u6.
OK, I don't need to actually draw a gate to generate it.
It has to be the same here and here, so I have to tie it together.
Similarly, down here I can just call this u5.
And now, continuing in here-- I hope this is going to come out all right.
It doesn't seem quite right so far, but let's see how it comes out.
What do we have coming in here?
We have two different free variables.
So again, let me just call this u3 prime again.
And this is maybe u5 prime.
But in any case, I can just draw this.
In this case, I'm coming around this way.
And u3 prime was dangling anyway, so I can leave that over here.
And similarly down here, I get two free ones, so let me just call them like this.
And this is equal to y7.
Well, it's a little strange.
I did something wrong there.
I did something wrong.
Well, maybe not.
Maybe not.
This is not going the way it's gone in the past, so maybe I drew the graph differently in the first place.
This is u6 again.
This is still u6 up here.
But this is a 0.
So this is u6 also coming out here.
But I need this little-- I do need to replicate u6 three times.
And maybe that's what I needed down here too, was to replicate u5 three times
[INAUDIBLE]
Excuse me?
The bottom line
Say again, I just didn't hear it
The bottom line.
You remove that [INAUDIBLE] shortcut
I removed-- this equaled a shortcut
Yeah.
But the [INAUDIBLE]
There was an [INAUDIBLE] on top of it, right.
And--
That cannot be replaced
That cannot be replaced by a shortcut.
OK.
Thank you.
So it should look like that?
I don't know.
I think I've botched it, so you're going to do it right on the homework.
Let me just-- yeah, good.
This is what it should look like.
You do need the equals there.
It's because I drew the graph backwards in the first place.
Anyway, presto change-o.
Going through that kind of manipulation, what we eventually come up with is something that looks like this.
And I apologize for botching it.
And coming around, this is tied to this and this is tied to that.
All right.
And I asked you to imagine that if I-- what I did was since I can draw the Hadamard transform-- either way, I drew it the wrong way.
And therefore, I wasn't going to get what I intended to get, which is this.
But it's done correctly in Figure 10.
And the types of reductions that I explained to you are the types of reduction that's necessarily to get from Figure 10A to 10B.
Please do try this at home.
Now, so I claim that this is a realization of the 8, 4, 4 code.
And you can verify that either by putting-- you'll find there are four free internal variables here and you can either set them to 1 or you can try this trick that I discussed before, I think.
That we can regard these as an information set.
And by fixing y0, y1, y2, and y4, you can trace your way through the graph and find out that y3, y5, y6, and y7 are determined.
Again, do try this at home.
So you can convince yourself that this is a graph for 6-- that has 16 possible solutions for all the internal and external variables.
16 possible trajectories that are valid that satisfy all the internal constraints.
And that the external 8 tuples that are parts of these trajectories are the 8 tuples of the 8, 4, 4 code.
I just assert that now.
Please verify it.
All right, so now let's step back and take a look at this realization.
I believe it's the simplest representation of the 8, 4, 4 codes if you account the complexity of the constraints and of the variables.
Notice that all the internal variables, the states if you like, they're all binary.
So it's a two-state representation if we want to use that language, that all the constraints are simple, either 3, 2, 2 or 3, 1, 3 constraints.
We count that as the branch complexity in effect.
This isn't a trellis, but this is what's analogous to the branch complexity as we'll see when we get to the sum product algorithm.
The complexity is proportional to the number of code words in each of these constraints.
So there are either 2 or 4 code words.
It's a very simple realization.
And there are only 12 of these constraints.
On the other hand, it does have cycles, like this cycle here.
When we get to the sum product algorithm, we'll see that there's a fundamental difference between decoding on a graph with cycles and a graph without cycles.
Without cycles, we get an exact decoding algorithm.
It does maximum likelihood decoding, like the Viterbi algorithm.
Or actually, a posteriori probability decoding.
And in any case, it gives us the optimum, in some sense.
When we decode on a graph with cycles, we get some degradation from the optimum.
Sae-Young Chung tried decoding with this.
And as I remember, it was a few tenths of [?
db ?]
sub-optimum.
It wasn't bad.
But it wasn't optimum either.
So that's the trade-off we make.
Suppose we want to make this into a cycle-free graph.
We can probably do that by agglomeration.
Let me suggest the following agglomeration.
Let's make all of this-- let's suppress that cycle into one big block.
Let's suppress this cycle into one big block.
So we're going to consider this one overall constraint.
What is that constraint?
We have two outputs.
Or I'm sorry, two variables over here.
We have four variables over here.
So it's going to be some code of length 6.
And from the fact that you have three inputs coming in here, you might guess it's going to be a 6, 3 code, which it is.
So if we do that, we simply get 6, 3.
And similarly, a 6, 3 constraint down here.
There are 8 possible valid behaviors just for this constraint.
And again, since everything is self-dual here, it's going to be 6, 3.
So let me, again, assert that.
Does anybody recognize what this realization is?
Would it help if I drew it like this?
This is four variables, four variables.
Anybody?
Come on, you've seen this before
[INAUDIBLE]
No.
This is a realization of a different style.
This is now a cycle-free realization, as you can easily see either from this or from-- well, it's cycle-free now.
Sorry, I should do one more thing.
I should make this into a quaternary variable.
So this will be 2.
It's basically u5, u6 in this notation.
Without doing that, there's still a cycle.
This little cycle going back and forth there as we said before.
So somebody I heard say it.
Say it again
Trellis
Trellis.
It's our two-section trellis realization of the 8, 4, 4 code where we divide a code word into two sections of length 4.
And simply, this is the trellis at the first half.
Eight possible lines, each one containing two parallel transitions.
Sorry.
Four lines each containing two parallel transitions going to one of eight states.
And that's what the actual trellis looks like.
So this is, indeed, cycle-free.
What's the difference between this realization and the other one?
Well, first of all, we've got four states in this cycle-free realization and we've got-- certainly, a much larger constraint code here, which will turn into complexity, computational complexity, when we go to decoding.
Now, we've got a branch complexity of 8.
Whereas, we never had anything larger than 4 in the zero sum constraints of the previous diagram.
So in that sense, this is more complicated.
Its state complexity, its branch complexity is higher than the cycle-free one that we did.
Than the one with cycles, but it's cycle-free.
You remember also we did this one.
We have both of these going down here and we consider them each to be a binary variable, then we get our tail-biting realization, which is only two states.
But it now has a cycle.
And if we're going to go to cycles, I would go all the way to that other one.
But maybe not.
If you decode this tail-biting realization, maybe its performance is a little bit better because it more faithfully represents what's going on inside the circuit than the other one.
How many different ways have we had of realizing the 8, 4, 4 code now?
We've had at least half a dozen, maybe more than that.
And this is all to illustrate the various styles that we have.
We've now had at least one of each style.
And now we have one of Hadamard transform.
Or, as I'm going to call it, we have a reduced Hadamard transform, the one with only the 12 simple constraints in it.
Or, we also have the agglomerated Hadamard transform, which gets us to cycle-free.
This has cycles.
But this is simple and this is, at least, more complex.
So there's a trade.
What do you want?
OK. Just one more side comment on the subject.
For exercise 2, which I didn't assign, shows that in general an agglomerated Hadamard transform realization of a Reed-Muller code, you can always get one that looks like this.
Let's see.
Which is sort of a four-section realization where each of these has length 2 to the m minus 2.
This is like the four-section realization of the Reed-Muller code.
Each of these state variables has dimension which meets the cut-set bound.
And the cut-set bound for a cut that divides the trellis into one quarter and three quarters or one half and one half.
You remember all of these dimensions were the same as simply the state dimension of a four-section trellis.
And I hope you remember that the state dimensions were the same at all four boundaries.
So we always get the same dimension for each of these, the same as in a four-section trellis.
And we get the minimal trellis complexity here.
So this is going to turn out to be length equals 3s and k equals t, where t is the branch complexity, the minimal branch complexity parameter for a trellis that we had in a table back in Chapter 6.
So this actually turns out to be a nice-- the nicest I know-- cycle-free realization of a Reed-Muller code.
For instance, for a 32, 16, 8 code these dimensions are 18, 9, 18, 9.
These are all 6.
It's a 64-state realization with 8 symbols at each of these points.
And each of these is a 14, 7 code.
Is this fundamentally simpler than a four-section trellis realization of the 32, 16, 8 code?
Not really.
The branch complexity is the same as it is in the central sections of this code.
This is very close to being a-- it is a four-section realization if we draw the four sections like this.
Except we have to draw a little additional thing up here, which is slightly helpful.
So it's sort of a expanded trellis that looks like this.
OK, so maybe think of that as a quote, "explanation" for this.
And I give explicit expressions for s and particularly for t, which perhaps you can derive or perhaps you don't care.
You won't be held responsible for it, anyway.
You might even remember that the Golay code, the 24, 12, 8 realization, which looked very much like this but contained only three sections, each of length 8 and a single 18, 9 constraint in the middle.
These were all 6.
Maybe you remember that, maybe you don't.
But that's a realization of the Golay code.
And it shows that these two codes are very close cousins.
There are many ways of showing that they're very close cousins.
They are.
OK, so now I'm just going off into side comments.
You obviously won't be held responsible for any of the side comments.
But I hope they give you something of a flavor for how the subject can be developed.
OK, that's the end of Chapter 11.
There is an appendix in Chapter 11 that talks about other flavors of graphs.
Factor graphs is a more embracing philosophically, conceptually notion where each of the constraints, instead of representing codes represents the factors, local factors, some global function factors into a product of local functions.
The local functions are represented by constraints.
It's a nice way of looking at things.
It's not strictly necessary for this course, so I haven't gone into it.
But most of the literature now is in terms of factor graphs.
In other fields, we have related topics, like Markov graphs or Markov random fields.
And slight differences in choices have been made in these areas.
Here, you see we're putting all the variables on the edges and the constraints as nodes in a graph.
And that's the normal graph style.
In Markov graphs, they make exactly the opposite convention.
They make the variables into nodes, which is probably what most people would do if they sat down to start doing a graphical model of something.
And they make the constraints into edges.
But there are several problems with that.
One is that if you have a constraint of larger degree than 2, then you really need a hyper-edge, which is represented by a clique.
And cliques sometimes introduce artifacts that you don't really want.
So I think it's actually an inferior style, but it's a very popular style in some fields.
Physics, particularly, for indicating dependencies.
All of these graphical models are methods of indicating dependencies and what's related to what.
There's another style.
It's very popular in statistical inference, which is called Bayesian networks.
And this style is similar to ours except the graphs are directed.
Everything is a cause and effect relationship.
So you have one variable that causes another variable.
We've generally stayed away from cause and effect here.
We've used the behavioral style, undirected graph.
But Bayesian networks have been hugely popular in statistical inference.
And so the appendix is just to explain the relationship between these various styles.
And obviously, you can work with any of them.
It's a matter of preference.
OK, any other questions about Chapter 11?
This was the basic introduction to codes on graphs.
Chapter 11 is, what are codes on graphs?
Chapter 12 is, how do we decode codes on graphs?
Which is the sum product algorithm.
So we'll move into that now.
And then Chapter 13, we'll actually talk about classes of capacity-approaching codes.
So these three chapters certainly form a closely related set that should all be read together.
OK, what are the basic facts about sum product algorithm?
I'm going to describe it as a method of doing A Posteriori Probability decoding, APP decoding, initially on cycle-free graphs.
I'll define-- what do I mean by a posteriori probability decoding?
Then, we'll develop the algorithm for cycle-free graphs where the theory is kind of complete.
There are various theorems that you can easily prove about the performance of the algorithm here.
The algorithm completes in a finite amount of time.
Basically, equal the amount of time it takes to get from one side of the graph to the other, which in graph theory language is called the diameter of the graph.
And it does exact a posteriori probability decoding.
So if that's actually what you want to do, this is a good way of doing it.
If you do this on a trellis for instance, it's the, by now, well-known BCJR algorithm, for Bahl, Cocke, Jelinek, and Raviv, who published this algorithm back in 1973.
Probably known in other fields before that.
Actually, you can sort of find it in Gallagher's thesis.
You can certainly find the sum product algorithm there.
Anyway, the BCJR algorithm is now widely used as a component of these decoding algorithms for turbo codes, in particular, and capacity-approaching codes in general because it's an exact way of decoding a trellis.
And probably fast.
Although, we'll find it's more complex than the Viterbi algorithm.
So if you are just interested in decoding a trellis, you would probably use the Viterbi algorithm.
But you use the BCJR algorithm when you actually want to compute the a posteriori probabilities.
And that's what you want when it's part of a larger algorithm.
So we'll do all the development on cycle-free graph.
Where we really, eventually, want to use it is on graphs with cycles.
And there are hardly any theorems.
In this case, you find yourself going around the cycles.
So there are all kinds of new questions.
How do you start?
When do you stop?
What are its convergence properties?
It becomes an iterative and approximate algorithm.
For instance, when I said that decoding of that simple, little reduced Hadamard transform realization for the 8, 4, 4 code, which had cycles in it, I said it was a couple of tenths of [?
db ?]
sub-optimum.
That was obtained by running this algorithm on that little graph with cycles until it seemed to have converged to something and plotting its performance.
And it's not quite as good as the performance would be on a cycle-free graph like this one.
All right, so you give up something in performance.
And you give up-- it becomes an algorithm that runs indefinitely.
And you'd have to figure out some stopping criterion and so forth.
All together, it's much more, what are we doing here?
We can't really say with any precision what we're doing here in most cases.
Although, in Chapter 13, I'll give some cases where we can say quite precisely what it's doing.
But it becomes much harder to analyze.
But in coding, this is actually what you want to do.
This is what works.
Just try it on a graph with cycles and hope for the best.
And because we design our own graphs in coding, we can design them so that the algorithm works very well.
Basically, we want to design them to have cycles, but extremely large cycles, large girth.
And then the algorithm tends not to get confused and to be near optimal.
Because it takes a long time for information to propagate around the cycle and it's pretty attenuated when it comes back in.
Whereas, with the short cycles that you saw here, that's not good.
And in many of these other fields, like physics for instance, image processing, you're dealing with graphs that look like grids.
Something like that.
You have lots and lots of short cycles.
And then just applying the sum product algorithm to a graph that looks like that won't work well at all because you have these little, short cycles.
So if you can design your own graph, this is a good thing to do.
OK, so let's get into it.
First of all, let me describe what I mean by a posteriori probability decoding.
This means simply that for every variable, both internal variables and external variables, we want to compute the probability of that variable.
Let's call it probability that some external variable y, k takes on some particular value y, k given the entire received sequence.
OK, so you can think in terms of the 8, 4, 4 code.
We're going to take this code.
We're going to transmit it over some kind of a noisy channel.
We're going to get a received sequence.
And from that, we want to figure out the probability that any particular input bit is equal to a 0 or a 1.
That might be more precisely what we're going to do.
So we really want to develop a vector consisting of the values of this a posteriori probability for all possible values of the variable.
This is sometimes called a message.
But this is what we're going to try to compute.
If we're actually decoding a code and we get the a posteriori probabilities for all these variables, then at the end of the day, what do we want to do?
We want to make hard decisions on each of these and decide what was sent.
It's quite easy to show.
Actually, I'm not sure I do this in the notes.
That a posteriori probability decoding is what you want to do to minimize the probability of bit error.
What's the probability of bit error?
It's the probability that y is not what you guessed it was.
In other words, it's 1 minus the max of all of these.
You would choose the max.
The probability of it actually being the max is given by that a posteriori probability.
And 1 minus that is the probability that it's anything else.
And that's the bit error probability.
So doing maximum a posteriori probability decoding on a bit-wise basis is the way of minimizing the bit error probability.
Now, that isn't actually what we've been doing in the Viterbi algorithm, for instance, or in general in our decoding.
We've said, we want to minimize the probability of decoding any code word on a sequence basis.
We want to minimize the probability of error in decoding the whole sequence.
So these are not exactly the same thing.
In fact, if you do max APP decoding, you may not even get a code word.
You'll make individual decisions on each of these bits, and that may not actually be a code word.
There's nothing that requires it to be.
Usually, it is.
Usually, there's no difference.
These algorithms tend to-- at least in relatively good signal noise, reach ratio situations.
They tend to both give the same answer.
And the probability of bit error or word error is not markedly different whichever one you use as a practical matter.
But it's certainly possible in principle that since maximum likelihood sequence decoding gives you the lowest word error probability, maximum APP bit-wise decoding has got to give you a higher word error probability.
So in any particular situation, you've got to decide which is more important to you, minimizing the bit error probability or minimizing the probability of any error in the whole block.
Actually, generally, the latter is what you prefer.
That governs your minimum time between decoding errors and things like that.
So that's a general discussion of what APP decoding is.
Why do we want it here?
Because when we get to capacity-approaching codes, we're going to talk about big codes that are made up of smaller codes.
The great thing about this graphical realization-- this is realization of linear systems.
You realize a big system by putting together blocks representing little systems.
That's a good way of designing things for all kinds of reasons.
It's going to be the way we design capacity-approaching codes.
And we're going to want a decoding algorithm for a part of a system.
Say, this 6, 3 part, or say something else.
Say this whole thing is a part of the system.
This whole thing is just 8, 4.
That yields soft probabilistic information in the output that we can then feed into decoding algorithms for other parts of the system.
So we won't want to make hard decisions.
We'll want to feed the whole message, the whole vector of a posteriori probabilities from one-- a partially decoded part of the system.
We'll feed that into some other part and then go decode that.
For instance, in turbo codes we'll decode-- we'll make it up out of several convolutional codes.
There will be several trellises.
We'll do BCJR decoding of one trellis.
That will give us a bunch of a posteriori probabilities about the bits that are involved in that trellis, then we'll pass them back to some other trellis decoder that's decoding-- apparently, a different trellis, but involving some of the same bits.
So maybe I shouldn't go on too long with these generalities, but that's why APP decoding is going to be a good thing for us to do.
And it's not because ultimately we want to do APP decoding of a single code.
It's because we want the soft decisions to feed into other decoding.
All right.
How do we compute this?
Probability that yk equals yk given r is going to be a basic probability theory.
It's just the sum over all, let's say, code words in which yk equals yk of the probability of those code words.
So sum over all y. I'm just making up notation here.
In our code, we're going to have some code words in the code that have yk equals 0 and some code words that have yk equals 1.
So we'll divide the code words into two sets, the ones that are compatible with yk equals 0 and the ones that are compatible with yk equals 1.
The a posteriori probability of yk equals yk is simply the sum of the a posteriori probabilities that we get of any of those code words.
I don't have to make proportional sign there.
But by Bayes' law, each of these-- Bayes' p of y given r is proportional to.
Here's a sign that a lot of people use.
I'm not completely comfortable with it yet, but that means proportional to.
Prop 2 in [INAUDIBLE].
Probability of r given y.
In other words, the a posteriori probability of a code word is proportional to the probability of r given y.
Why?
Because by Bayes', it's actually equal to that times probability of y over probability of r. But this is the same for all of them.
If the code words are all equiprobable, these are the same for all of them.
So it's just proportional to the likelihood of getting r given that you transmitted y. I'm sure you've seen this many times before, so I won't do in detail.
So this is proportional to the same sum.
y in this code word, where y-- a portion of the code where yk equals y. p of r given y.
But that's easy.
I'm assuming that I have a memoryless channel here.
That's been implicit throughout this course, like the Additive White Gaussian Noise channel or the Binary Symmetric channel.
So this just breaks up into a component-wise product.
This is just the product of p of-- over the k. p of rk given yk.
And because yk has the same-- I should say k prime here.
Sorry, that's very confusing.
So this is over all k prime.
But I can break this up.
One of these terms is going to be k. For that, yk is fixed at little yk, at this particular value.
So I can write this in turn as this for k prime not equal to k. And then I get a common term which is p of rk given yk.
OK, so this is the expression that I want to compute.
Two comments.
When I say "sum product," it's really because this is just a sum of products here.
We see we want to compute this actually rather nasty sum of products.
We're looking for an efficient way of computing the sum of products.
And we're going to show how this could be computed by tracing through the graph.
That's one comment.
The second comment is that in the literature, particularly in the turbo code literature, this has a name.
This part, the direct likelihood of the symbol given what you received corresponding to that symbol, is called the intrinsic information about y, k. And this part, which has to do with, basically, the likelihood of yk given all the other yk prime, for k prime not equal to k, is called the extrinsic information.
We basically take these two parts, which we can compute separately, and we multiply them together.
And we get the overall probability.
So the probability consists of a part due to rk and a part due to all those other rk prime, which we're going to compute by going through the graph.
And I just introduced this language because if you read papers, you're going to see this throughout the literature.
OK. Let me start moving back.
I never used this.
Well, let me introduce one more thing.
So we want to compute all these messages.
In particular, we want to compute the probability of yk equals yk given r. This part here is really the r on k prime not equal to k. Terrible notation.
The other r's.
So that'll be a message.
We also want to compute similar things for all the internal variables, which we have sometimes called state variables.
And it breaks up in the same way.
And I won't take the trouble to write out the expression.
But again, now we really want to go over all configurations which have a particular state variable in them.
So we're really going over all behaviors.
And we can compute probabilities this way.
I won't belabor that right now.
So we want messages, really, indicating the likelihood of each of the external variables and each of the internal variables.
The whole set of possible likelihoods, the APP vector, the message, for both the symbol variables and the state variables.
And that's what the sum product algorithm is going to do for us.
OK.
So the sum product algorithm is based on-- I discuss it three parts.
One is the past future decomposition of the code.
Another is the actual sum product update rule, which is based on another decomposition.
And finally, we need an overall schedule.
OK, so let me discuss this first.
Let's take a particular state variable.
I only mean internal variable.
And here it is.
Somewhere in our big graph, it's an edge.
Our big graph is cycle-free.
So what's the special property of edges in cycle-free graphs?
Every edge is a cut-set.
So if I take this edge, it divides the whole graph-- we've done this before-- into two parts, which we can call past and future.
And if we were to take out this edge, it would disconnect the graph from these two parts.
So there's no other connections between the past and the future because the graph is cycle-free.
And there's some set of past observed variables, rp, and there's some set of future observe variables, rf.
So before, I called this yp and yf.
These are the received observations corresponding to all the past variables, the received observations corresponding to all the future variables.
And the past future decomposition is basically like this here.
Let me write it out.
It basically says that the probability of-- that state variable has a particular value given r is proportional to the probability that the state variable has a particular value given r-- that part-- times the probability that it has a particular value based on the future part.
And that we can simply-- when we take the vector, we can simply multiply the two components that both have a common value sk and we get the overall a posteriori probability that this state variable has a particular value.
Now, I'm sure there is a-- OK.
In the notes, I go through a little development to show this.
The development is, first of all, based on the fact that if I ask-- now in the corresponding expression here, I want to have the sum over all y and s such that the behaviot-- I'm going to just lose myself in notation here.
We only have a few minutes left.
This is basically going to be over a code that's consistent with the state having this particular variable.
But when I cut this code up like this, the overall code of all configurations that are consistent with a certain state variable sk is going to divide up into a Cartesian product of the past part of the code that's consistent with sk across the future part of the code that's consistent with sk.
And this was precisely the argument we went through when we developed the cut-set bound.
That given sk, we can take any past that's consistent with sk and any future that's consistent with sk and map them together.
And in effect, fixing the state does disconnect the graph.
And this is the basic Markov property that we used to develop the cut-set bound.
Really, just depending on that again.
Plus the basic Cartesian product, lemma, which is that if I have a sum over a Cartesian product, x cross of f of x g of y, what's a fast way of computing that?
[INAUDIBLE]
Right.
So this is trivial.
But in fact, that's what we do in many fast algorithms.
It's just the sum over the x of the f part, the sum over y of the g part.
And you multiply those two together.
Just think of x and y being on some rectangular array.
Here, we compute all the products corresponding to every combination of x and y individually.
Here, we first take the sum of all those, the sum of all those, the sum all those.
We multiply them times the sum of-- in this way.
And we will get exactly the same terms if we multiply them out.
And so this is just a fast way of computing this.
And that's really what we're doing when we divide this up in this way.
I think I'm incapable of explaining that any more clearly right now And we're at the end of our time, anyway.
So we do want to not do problem 8.3.
You have only one problem due on Wednesday, which is 8.1.
It will probably take you a while, anyway.
We didn't get to the BCJR algorithm.
We'll take this up again and complete the sum product algorithm next time.
See you Wednesday.
We are still in chapter 12 on the sum-product algorithm.
I hope to get through that today, fairly expeditiously, and to start with chapter 13.
The handouts for today will be chapter 13.
And, as usual, problem set 9 with problem 8.3 has been moved up into problem set 9.
Problem set 8 solutions for 8.1 and 8.2.
So I'm going to start off on a different tack of presenting the sum-product algorithm.
Where is everybody?
1, 2, 3, 4, 5, 6, 7, 8, 9.
Is there any competitive event that I should know about?
I know the traffic was bad and it's a rainy, late morning.
Problem getting here myself.
Well, this is improving.
The write up of the sum-product in the notes is in terms of equations.
And I've never been terribly satisfied with this write up.
It's concise, but we have to invent some new notation, which I don't consider to be completely transparent.
I've never found it particularly easy to present in class, and it's been a little frustrating to me.
So I'd like to try a different approach.
I'd like in class just to go over through a very explicit example, our favorite 844 code.
What we're trying to achieve with the sum-product algorithm and how the sum-product algorithm helps us to achieve it in an efficient way.
So it's proof by example.
It won't be very satisfying to those of you who like to see real proofs, but that you can get by going back and re-reading the notes.
So I'm going to try doing it this way and you will let me know by your looks of gladness or of frustration whether you like it or not.
So let's start at the beginning again.
What are we trying to do?
We're trying to compute the a posteriori probability of every possible value of every symbol, internal and external, that appears in a graphical realization for a code.
And we will consider these to be APP vectors.
In other words, if a symbol is eight valued, there will be eight APP values in the vector.
And we're going to assume that we have a cycle-free graph realization, so that we can proceed by the cut-set idea to solve independent parts of the graph.
And at the very end, we'll relax that and see whether we can still do OK. All right, one concept that seems very simple in theory but that some people don't get till they really try to do it in practice is that an APP vector is really a likelihood weight vector normalized so that the sums of all the probabilities are equal to 1.
For instance, if we send plus 1 or minus 1 through an additive white Gaussian Noise channel with sigma squared equal to 1 or whatever, then the probability of-- we call this y and this r, the probability of r given y equals whatever.
This is probability of r given y, is 1 over square root of 2 pi sigma squared e to the minus y minus r-squared over 2 sigma-squared.
And so we can compute that for y equals 1 and y equals minus 1.
And we'll get a vector of two things.
The APP vector would then consist of this evaluated for y equals 1.
Let me just take this part of it, a to the minus r minus 1 squared over 2 sigma squared and e to the minus r plus 1 squared over 2 sigma-squared.
Something proportional to these two vectors normalized and that's why I continue to use this proportional to symbol, which you may not have seen before.
Or maybe you have.
So the APP vector in this case would be proportional to-- well, actually 1 over square root of 2 pi sigma-squared times each of these two things.
But this would basically tell you how much weight to give to plus 1 and minus 1 as possible transmitted symbols.
And all this matters -- there's one extra degree of freedom in here.
You can scale this, the whole vector, by any scale factor you like and it gives you the same information.
Because you can always normalize the two terms to be equal to 1.
So if this, for instance, was 0.12 and this was 0.03, that's equivalent to 0.8 and 0.2.
Two probabilities.
Actual a-posteriori probabilities that sum to 1, all you've got to do is keep the ratio right.
This is four times as large as that, so what that really means is 0.8 and 0.2.
And in implementation, so the sum-product algorithm, we never really worry about constant scaling factors.
As long as they appear in every term we can just forget about scaling.
I mean we don't want things to get too large or too small, so we can apply an overall scaling factor.
But as long as we've got something, the relative weights, then we have what we need to know.
Let's go to the 844 code since we know that very well and ask ourselves what it is we're trying to compute.
We're given APP vectors for each of the eight received bits, let's say.
We transmit y, we receive some r, perhaps over an additive white Gaussian noise channel.
So we get a little 2 term APP vector for each of these y's, telling us the relative probability that a 0 was sent or a 1 was sent.
That's called the intrinsic information.
Let me just call that.
So for each one we get p0 and-- let me make it p0 and q0, the two terms for y0; p1 and q1 for y1 and so forth.
So we have that information for each of the received symbols.
What's the likelihood of each of these code words then given these individual bitwise likelihoods, the probability of the all zero code word.
It's likelihood, let's say, is then proportional to p0, p1, p2, p3, p4, p5, p6, p7.
Whereas the probability at 1, 1, 1, 1, 0, 0, 0, 0 is q0, q1, q2, q3, p4, p5, p6, p7 and so forth.
Each one is a product of the eight corresponding terms.
Yes?
You're assuming that the bits are independent
Yes, it's a memoryless channel.
So the noise in every symbol transmission is independent.
And that's important too
So basically once again, the second bit is independent of the first bit in transmission?
I'm sorry, I was writing and I missed what you said
Well, I mean, if the first bit is 0, maybe that is a code where if the first bit is 0, the second bit has to be 0
Yeah, there's certainly dependencies among the code words.
That's what we're trying to take into account.
This is how to take it into account.
So I'm assuming a memoryless channel.
But I'm of course, assuming dependencies within the code.
How do I do that?
One way is maximum likelihood decoding.
What do we do when we do maximum likelihood decoding in principle?
We exhaustively computed each of these 16 products, each of these 16 likelihoods for each of the 16 code words.
We'd simply pick the one that's greatest and that's the code word that is most likely to have been sent.
So that's what exhaustive maximum likelihood decoding consists of, finding the maximum likelihood word.
And that clearly takes into account the dependencies.
We're only considering valid code sequences when we do that.
In APP decoding we're doing something else, but it can be characterized quite simply.
What's the a-posteriori probability that say, the first bit, is equal to a 0 or is equal to a 1?
We can get the APP by summing up the likelihoods of all the code words that are associated with the value of the bit that we want to see.
So we take the eight code words that have a 0 in this place and we'd sum up these likelihoods for those eight code words, and that would be the weight, the likelihood weight of y0 being 0.
And the likelihood weight of y0 being 1 would be the sum of the other eight.
And again you see that scale factors aren't going to matter here.
All we want is the relative weights of these things.
So that's what we're trying to do in APP decoding, but we're trying to do it now for every single variable.
At this point, I've only shown you the symbol variables, the external variables.
So a brute force way of doing that would be to fill out this table, all 16 likelihoods, do these sums and then we'd have the APP vector for each of these.
Now we're looking for a more efficient way of doing this.
The efficient way of doing this is going to be based on the fact we have a cycle-free graph realization.
Which in this case, is a trellis realization.
I'm going to use this two section trellis where each section is a four-tuple.
We've drawn it two different ways.
This is a very explicit way showing the two parallel transitions that go from the initial node to the central state, and then two more down here.
The set of code words is the set of all possible paths through-- corresponds to the set of all paths through this trellis.
So for instance, includes the all zero word, (0,0,1,1,1), (1,1,1,0,0,0), (1,1,1,1,1,1,1), and so forth.
And these I happened to have listed as the first four here.
Or we now have this more abstract way of writing this same thing.
This says there are four external variables and two state variables or a vector space of dimension 2 here and a vector space with dimension 4 here.
That the dimension of this constraint is 3.
That means there are eight possible values for these combinations of 4 and 2.
They're shown explicitly up here.
And they form a linear vector space.
Either way, this is our trellis realization.
Now, we know that to do maximum likelihood decoding, it's useful to have this trellis realization, because, for instance, we can do Viterbi decoding.
And that's a more efficient way of finding what the maximum likelihood sequence is.
Basically, because we go through and at this point we can make a decision between 0, 0, 0, and 1, 1, 1, 1.
We only have to do that based on this four-tuple and then we can live with that decision regardless of what else we see in the other half.
And likewise in the other direction.
So now we'd like to use this same structure to simplify the APP decoding calculation, which looks like a more complicated calculation.
So how do we do that?
We introduce two state bits here.
All of these have state bit 0, 0, 0, 0.
Sorry, this point, we've called this state 0, 1.
Just to keep it visually straight, 0, 1, 0, 1, 0, 1, and then there are eight more corresponding to the other two possible values of the state bits over here.
This is a quaternary variable.
We want it to assume that it's a four-valued variable and to make the realization cycle-free, I don't want to consider it to be two independent bits.
So there are four possible values for this state variable.
So maybe I just ought to call this my state space -- which is a four-valued state space.
Now, what's the a-posteriori probability of the state being say, 0, 0?
I compute that in the same way.
I compute that simply by summing up the four likelihoods of the code for the external variables, the code words that are associated with state variable 0, 0 -- the first possible values of this quaternary variable.
So it'd be the sum of these four things.
Yes?
[UNINTELLIGIBLE PHRASE]
I've only listed half the code words.
This is only 8 of the 16 code words.
You want me to write the other 8?
And by the way, there's at least an implicit assumption here that the code sequence determines the state sequence.
As we found when we did minimal trellis realizations, the code sequence always does determine the state sequence.
So if I know the entire code word that also tells me what the values of all the state variables are.
There's a 1:1 correspondence between code words and trajectories in a minimal trellis.
And I believe I'm sometimes implicitly using that assumption.
But anyway, it holds whenever we have a cycle-free graph.
In a cycle-free graph, we have a well-defined minimal realization.
And the minimal realization does have this 1:1 property.
So the state variables are determined by the symbol variables.
Suppose I want to compute the values of these state variables here.
Let me write this out.
This will be p0, p1, p2, p3, q1-- sorry --- q4, q5, q6, q7.
And then there's one that's all q's.
q0, q1, q2, q3, q4, q5, q6, q7.
So basically, what I want to do is take the sum of these four things, and that's just a sum of products and I could just do it.
But the Cartesian product lemma gives us a simpler way of doing it.
We notice that I can have either of these two four-tuples as the first half and either of these two four-tuples as a second half.
In fact, I see over here, explicitly, these four code words are the Cartesian product of two four-tuple codes, the repetition code on 4.
So I can use the Cartesian product lemma to write this as -- again, let me write it very explicitly.
p0, p1, p2, p3, which you recognize as the probability of this four-tuple plus q0, q1, q2, q3.
The probability of this four-tuple times p4, p5, p6, p7 plus q4, q5, q6, q7.
Now do you agree that the product of those two terms is equal to this, the sum of these four terms?
Yes, there are four terms in this sum and they're what we want.
And it's because we have this Cartesian product structure, which we always have in the state space.
This is the basic Markov property of a state variable.
So that's a simpler calculation.
This requires four multiplications, each one is seven-fold multiplication.
This only requires one multip -- well, it requires one, two, three, four-fold multiplications, plus one overall multiplication.
And it's certainly a simpler way to compute that.
And I can similarly do that for each of these down here.
What this leads me to is what I called last time the past future decomposition.
We have one APP vector over here that corresponds to the-- let's call it the APP vector of the state given the past received symbols.
We had something like that.
In other words, given what was received for y0, y1, y2, y3, what's the probability, the partial a-posteriori probability for each of the four possible state variables given this past?
And similarly, we have another vector which I will consider to be a message coming in this direction, a message consisting of a vector of four values.
Which is what's the probability the state vector given these four received simple, which is computed in the same way?
That's this here.
And the past future rule is that just the overall probability for any of these internal state variables, the overall APP vector, is just obtained by a component-wise multiplication.
This would be the APP of the state vector being equal to 0, 0 given the past.
This would be APP of the state vector given 0, 0 given the future.
We'd have the same thing for 0, 1, 1, 0, 1, 1.
And to get the overall APP vector given all of r, the rule is just multiply them component-wise.
We take the likelihood weight for 0, 0 given the past and multiply it by likelihood weight of 0, 0 given the future.
And that gives us the total likelihood weight up to a scale factor.
The notes derive this in equation terms.
You see it basically comes from this Cartesian product decomposition, which we always get for any internal state variable.
The possible code words consistent with that state consist of a certain set of possible past code words consistent with that state.
Cartesian product with a set of possible future code words consistent with that state.
And as a result of that Cartesian product decomposition, we always get this APP vector decomposition.
That's what the proof says in the notes.
Do you like this way of arguing?
A couple people are nodding at least.
It's an experiment.
All right, so that tells us what we're going to want to do, at least, for the internal variables.
And actually, the same is true up here.
The variables going in are easy.
That's just p0, p1, p2, p3, or whatever.
The ones coming out are-- this is called the intrinsic variable, the one that we get from direct observation, the intrinsic APP vector, excuse me.
The extrinsic APP vector is the one that we get from all the rest of the inputs and symbols and the other parts of the graph.
And we combine these component-wise to get the overall APP vector.
All right, but we now have to do this for every part of the graph.
Let me go through another calculation, which will-- how do we, for instance, find the APP vector for y7, the extrinsic APP vector for y7, given that we have a trellis like this?
In general, for y7, we're going to be adding up all the zeros, these guys.
And four more down here, and we'll make that the extrinsic APP vector of 0.
So we want to add up this and this and this and this and so forth, eight of those.
We want to do that in an efficient way.
This defines a code word with eight possible components.
Let me move this up now.
So let's talk about this 6, 3 code.
What does it look like?
With state vectors 0, 0, I can have y4, y5, y6, y7 be 0, 0, 0, 0.
Or I can have it be 1, 1, 1, 1.
The state vector is 0, 1.
I can have 0, 1, 0, 1 or I could have 1, 0, 1, 0.
With state vectors variable 1, 0, I can have 0, 0, 1, 1 or 1, 1, 0, 0.
And with 1, 1, I can have 0, 1, 1, 0 or 1, 1, 1, 0, 0, 1.
All right, so that's precisely the 6, 3 code I'm talking about.
Now suppose I have the APP vector for each of these.
The APP vector for each of these, 0, 0 we've already calculated.
It'd be p0, p1, p2, p3 plus q0, q1, q2, q3.
That's the likelihood vector for 0, 0 coming from the past.
So that's one part of this message here.
For either of these I get p0, q1, p2, q2, q3 plus q0, p1, q2, p3.
Reflecting the two possible ways I can get to this value.
I'm going to get four terms that I've computed for the past message.
I'm going to have four likelihood weights corresponding to each of the four values of this state variable.
So I've got that vector.
I already computed it as part of the computation to do the APP for here.
It's the past part of it.
So now, I want to compute the probability that y7 is a 0.
The rule here is to take every combination in this code and again, I'm going to get a-- well, let me write this part of it.
This is going to be p4, p5, p6, p7.
That's the probability of this four-tuple.
This is q4, q5, q6, q7.
This is p4, q5, p6, q7, q4, p5, q6, p7 and so forth.
That's the weights I want to assign to each of these.
So now I get a weight for each of these six-tuples.
I multiply this by this to get the weight, the likelihood weight, for this code word.
This by this to get the likelihood weight for this code word and so forth.
And I add these to buckets for-- I simply go through this and I'll add this one to the bucket for y7 equals 0.
And I'll add this to the bucket for y7 equals 1.
And this to the bucket for y7 equals 1.
And this to the bucket for y7 equals 0 and so forth.
It's clear that in this bucket I'm always going to get the same value for p-- I'm always going to get a p7 for 0 and a q7 for 1.
So just looking at y7, I don't have to compute the seventh value here.
So if I did this separately as a little y7 variable, then the incoming message, the intrinsic information, would just be this p7, q7 likelihood weight vector.
And what I really want to compute is the outgoing part, which has to do with y4 through y6.
So let me draw it that way.
This I don't think is very good exposition.
Bear with me.
So now I'm going to put this times this in my bucket for 0.
This times this in my bucket for 1, and so forth.
The question here is, does this method give me the same answer as my exhaustive method up here?
And again, because of the Cartesian product character of the states, you can convince yourself that it does.
In other words, I can use this as a summary of everything in the past that got to the state value equal to 0, 0, 0.
Any time I got to 0, 0, 0, I got through one of these two things.
And so for any possible continuation over here, I could have got to it in one of these two ways.
And I'm going to find terms, e.g., these two terms.
Let's see.
What do I want?
I want two ways of getting to 0, 0, 0.
I'm sorry, these two.
I claim that this times these is equal to the sum of these two.
And it is.
I can get to 0, 0, 0, 0, 0 either through this or this.
And I get to 1, 1, 1, 1, which is this one, either through this or this.
And again, it's a fundamental property of the state that whenever I get one of these, I'm going to get both of them in equal combination.
So I can summarize things in this way, which leads to an efficiency of computation.
So what I'm basically explaining to you now is the sum-product update rule, which is how to propagate messages.
Propagating messages, which are APP vectors.
In general, my situation is going to be like this.
I'm going to have some constraint code, (n,k), I'm going to have some incoming messages over here which tell me about everything that's happened in this past, p prime, and in this past, p prime.
Give me a summary of all the weights corresponding to that.
I'm going to have some output symbol over here and I want to compute now what the APP vector is for the possible values of this output symbol, which is going to be some aggregate of this down here.
What I do is I go through all 2 to the k code words, call this constraint code.
Sorry, I'm using k again.
That's 2 to the k code words.
And compute the product of the inputs.
Of input APP vectors according to the possible combinations of the input variables and the output variables that are allowed by this constraint.
That's explicitly what I'm doing here.
Here are all the possible combinations of state variables and symbol variables that are allowed by this 6, 3 constraint code.
There are eight of them.
For each one, I'm going to compute the relevant product of APP vectors and I'm going to add it to a bin.
So add to a bin.
Bins which correspond to-- draw it like that-- output.
I'm calling this the output variable values.
OK, and that'll give me the summary APP vector of the outputs for now the past, which consists of everything that could have happened in this side of the graph.
Again, I'm using the cycle-free assumption in several ways.
I'm saying that everything that happens up here is independent of everything that happens here.
Everything that happens completely in this past is just the sum of what happens here.
What happens here are subject to this constraint and has nothing to do with what happens over here in the future.
Eventually I'm going to do the same thing in the future.
I'm going to compute a future component that's based on all this stuff.
I'm going to get messages going in each direction and I'm going to sum them up.
OK, now I'm seeing a fair amount of puzzlement on people's faces.
Do you get abstractly what I'm trying to do here?
Does anyone want to ask a clarifying question?
So now--
[UNINTELLIGIBLE PHRASE] the bottom right, output variable?
Output variable values, I'm sorry.
I think there's a little bit of confusion here because sometimes I've considered this output to be a 16-valued output.
Here, in fact, well, it has eight actually valid values.
And if I was considering it that way, then I would just have one bin for each of these likelihoods.
But now if I consider these as four binary variables, then I would aggregate these things according to the values of each of the binary values.
And I'd get four vectors of length 2 for the individual APP.
Which is ultimately what I want here, so I'm sorry I've gone back and forth between subtly different realizations.
I do find this very hard to explain.
Perhaps there's someone else who can do a better job.
You have a homework problem which asks you to do this, and I think having done that you'll then get the idea.
But it's a matter of going back and forth between the gross structure and the actual equations and just working it out.
All right, so the key things in the sum-product algorithm.
I've already explained some level to them.
We have this sum-product update.
That basically explains given messages coming into a node, how do we compute the message going out of the node?
We take contributions, products corresponding to what the code allows.
We dump them into the appropriate bins in the output vector and that's the sum-product update rule.
We have the past future decomposition or combination rule.
And that says at the end of the day, you're finally going to get messages going in both directions.
And you just combine them component-wise as we started out with.
And finally, we need to talk about the overall schedule of the algorithm.
So let me draw an arbitrary cycle-free graph.
OK, suppose I have an arbitrary cycle-free graph.
It consists of external variables out here, internal variables and constraint nodes.
How am I going to schedule these computations in order to do APP decoding of this graph?
Well, what do I have at the beginning?
At the beginning, I measure the received variables corresponding to each symbol, and that gives me the intrinsic information, which is incoming message you can consider from every symbol variable.
Basically, what did I see on the channel?
That's what that likelihood weight vector corresponds to.
All right.
What do I need in order to do the sum-product update rule?
I need the inputs.
Think of this now as a directed graph.
I need the incoming messages on all of the incident edges on this node except for one.
If I have all but one, I can compute the last one as an outgoing message.
So that's the structure of that update.
So at this point, can I compute anything here?
No, I would need the inputs.
But I can compute this outgoing message.
I can compute this outgoing message.
I can compute this outgoing message.
I haven't drawn these of very high degree.
Normally we have degree higher than two.
If I had another input here, however, I could still get that output.
All right, so that's time one.
Think of there being a clock and at time one I can propagate messages into the graph to depth one if you like.
There's going to be a systematic way that we can assign depths.
All right, now at time two what can I compute?
I can compute this guy because I have this input.
Now I have both inputs coming in here, I can compute this guy.
Anything else?
So maybe 0, 1, 2.
I should put times on each of these, so I don't get confused.
0, 0, 1, 0.
So this is 2 at the output.
Now I'm at time three, what can I compute?
At time three, I now have two inputs coming in here.
I can compute this output.
Actually, from these two inputs, I can compute this output at time three.
And from these two, I can compute this outgoing message at time three.
Are there other things I can compute?
Can I compute this?
No, I don't have this yet.
That's probably it.
Does anyone else see anything else I can compute at time three based on the time zero, one, and two messages?
No.
All right, but I'm making progress.
And should be clear that I'm always going to be able to make some progress.
At time four I can compute this message in that direction.
Let's see, I now have everything I need to compute this message in that direction.
And I'm almost done.
And finally, I can compute this message in this direction at time four, the extrinsic information for those variables.
I guess I can do this at time four.
I had that at time three.
And now at time five, I can get everything else that I need.
Time five I get that out.
I get this out based on this input and this input.
And I get this out based on this input and this input.
So in a finite amount of time, in this case, five computational cycles, I can compute all of the messages in both directions for every variable in the graph.
Now as a final cleanup step, I simply component-wise multiply the vectors in the two directions, and that gives me the APPs for all the variables.
This is a kind of parallel computation of all the APPs for all variables in the graph.
And I claim two things.
It's finite and it's exact.
And the exactness is proved from the equations and the cycle-free assumptions.
So we can always decompose things as Cartesian products.
Independent components.
The finiteness, well, I assumed this is a finite graph.
What is this number 5?
It's really the maximum length of time it takes me to get from any of these external symbol variables to any other external-- from leaf to leaf in graph theory terms.
And that's called the diameter of the graph.
And again, some people count the last little symbol node and some people don't count the last little symbol node.
But it's clear that a finite graph is going to have finite diameter and that the propagation time is basically going to be equal to the diameter.
That's how long it takes to get across the graph.
So each of these messages in each direction has a certain depth.
That's the distance in the graph to the most distant leaf node.
And the maximum sum of these is always going to be 5 in this case.
Notice that it is.
A tiny little graph theory theorem
[UNINTELLIGIBLE PHRASE] you see why if you sum these messages flying through those past and future of the single edge, you would get the overall APP because you're summing the contribution from both the set of outputs and those [UNINTELLIGIBLE PHRASE].
How would you prove that that should be overall the APP of the symbols?
Recall the state space theorem.
Let's take any edge in here.
Suppose we think about cutting it.
A cut-set through that edge.
Suppose we consider agglomerating everything back here and calling that the past.
Agglomerating everything out here and calling that the future.
This is past and future.
The idea of the state space theorem was that this is exactly the same situation as a two section trellis.
Because everything is linear you get the state space theorem basically, which says that you get a Cartesian product of a certain past, a certain future.
You get a minimal state space, which is kind of the total code.
The modulo, the part of the code that lives purely on the past, and the part that lives purely on the future.
That quotient space is really what it is, corresponds to the state space at this time.
Because every edge in a cycle-free graph is by itself a cut-set, you can do this for every edge in any cycle-free graph.
You can make this same argument.
So you get a well-defined minimal state space.
You get a Cartesian product decomposition, which has a number of terms equal to the size of the state space, due to the dimension of the state space over binary field.
And so everything goes through just as it did in the trellis case.
That was the argument.
So mushing all this together, you get the sum-product algorithm.
This is the fundamental reason why I took the time to go through minimal trellis realizations of block codes.
Not because that's actually the way we decode block codes all by themselves, or even use block codes.
We use convolutional codes.
It's so that we could prove these very fundamental theorems about graph decoding.
And we aren't even going to decode cycle-free graphs.
But that's the intellectual line here.
Thank you for that question, it was very well timed
[UNINTELLIGIBLE PHRASE]
Let's take a look at the structure of this.
Where do the computations occur?
Every node there's a computation.
What's the complexity of that computation?
It's somehow approximately equal to the size of the constraint code.
We do one thing for every possible combination of legitimate combination of variables, which is what we mean by the size of the constraint code.
So if this is a 6, 3 code, we need to do eight things here.
Basically, eight multiplications.
So this, in more general cycle-free graphs, that's what corresponds to branch complexity and trellis graphs.
We found that the branch space corresponded to a constraint code in trellises.
So the computation is basically determined by the constraint code complexity and really, overall what we'd like to do is minimize the constraint code, the maximum size of any constraint code because that's going to dominate computation.
So we try to keep the k in these things as small as possible.
Yes?
[UNINTELLIGIBLE PHRASE] and you get larger dimension constraint for [UNINTELLIGIBLE PHRASE]
Yeah, we're actually not too concerned about diameter.
Because things go up exponentially with constraint code dimension, just that single parameter tends to be the thing we want to focus on.
But we remember that's hard to minimize in a trellis.
We showed that basically our only trick there after ordering the coordinates was sectionalization and that we could not reduce the constraint code complexity, the branch complexity, by sectionalization.
And then again, we build on that and we went through this argument that said, gee, this corresponds to some trellis.
We could draw a trellis where this, literally was the past and this was the future.
And the minimal size state space in this cycle-free graph would be the same as in that trellis.
What I guess I didn't show is you can't reduce the constraint code complexity by significantly -- actually, I'm not sure there's a crisp theorem.
But the fact that you now are constrained to size of state space, this is certainly going to be at least as large as state space sizes.
Tends to indicate that you're not going to be able to do a lot about constraint code complexity by going to more elaborate cycle-free graphs than just the chain trellis graph either.
That's, I think, a missing theorem that would be nice to have.
I had a paper about a year or two ago, I'm not sure it ever quite gets to that theorem.
But that's the theorem you'd like to see.
You really can't improve this.
So ultimately, we're bound by what you can do in a trellis.
And in a trellis you can't do too much about this branch complexity parameter once you've ordered the coordinates.
A bit of hand-waving here, but we're going to need to go away from cycle-free graphs.
One reason I like normal graphs is that you get this very nice separation of function.
You get the idea that nodes correspond to little computers.
You could actually realize this with little computers.
Put a computer in here for each node.
Its job is to do the sum-product update equation.
And then, well, what are the edges?
They're for communication.
These are the wires that run around on your chip.
And so when we talk about state space complexity, we're really talking about something like bandwidth.
How wide do you have to make these wires?
Do you have 6 bits, or 9 bits, or whatever?
So state space really has to do with communication complexity.
Constraint or branch has to do with the computational complexity, which sometimes one's more important than the other.
Computational tends to be the thing we focus on.
But we know more and more as we get to more complicated chips, communications complexity tends to become dominant.
And this, of course, is the I/O.
These are your paths which go to the outside world, your I/O paths.
So it's really a good way to think about this.
And in fact, some people like Andy [UNINTELLIGIBLE] and his students have experimented with building analog sum-product decoders.
The general idea here is that you can compute an output as soon as you have the two corresponding inputs.
Well, imagine another schedule where this thing just computes all the time based on whatever inputs it has.
In a cycle-free graph, say you're in the interior.
Initially, what you have on your inputs is garbage.
But eventually, you're going to get the right things on all your inputs.
And therefore, you're going to put the right things on all your outputs.
So again, up to some propagation times, just putting in non-clocked little computers here, whose function is to generate the right outputs given the inputs in each direction, is eventually going to be right in a cycle-free graph.
And in fact, there are very nice little analog implementations of this where it turns out the sum-product update rule is extremely amenable to transistor implementation nonlinear rule.
So you just put these in, little computing nodes, and you put whatever you received on here as the received symbols, and you let it go.
And it computes incredibly quickly for a cycle-free graph.
Where we're going to go is we're going to say, well, gee, we got this nice little local rule that we can implement with local computers for the sum-product algorithm.
Maybe it'll work if we do it on a graph with cycles.
And on a graph with cycles this kind of parallel or flooding schedule, where every little computer is computing something at every instant of time, is probably the most popular schedule.
And again, you build an analog implementation of that and it goes extremely fast and it converges to something.
It just relaxes to something, there's a local equilibrium in all these constraint nodes.
And therefore, some global APP vector that satisfies the whole thing.
I think this is a nice attribute of this kind of graphical realization.
All right, a particular example of how this schedule works.
The cycle-free realization that we care most about is the trellis.
The BCJR algorithm is simply the implementation of the sum-product algorithm on a trellis.
SP on a trellis.
So generically, what does a trellis look like?
I guess I should allow for some nontrivial code here.
So all trellises have this same boring graph.
What is this schedule?
It's a cycle-free graph, so we could operate according to this nice controlled schedule, compute everything just once.
And how does that operate?
We get intrinsic information that's measured at each of these symbols here.
What did we receive on the channel?
We put that through a constraint code and we get here, let's give this a name.
This is called intrinsic 0.
Here is alpha 1, which is just a function of the intrinsic information at time zero.
And we combine that with the intrinsic information at time one and we get alpha 2, and so forth.
So we get a forward-going path down the trellis.
If any of you knows what a Kalman filter or a smoother does, this is really exactly the same thing.
This is finding the conditional APP probabilities of this state vector given everything in the past.
In Kalman filtering or smoothing, it's done for Gaussian vectors.
Here we're talking about discrete vectors.
But in principle, it's doing exactly the same thing.
It's computing conditional probabilities given everything observed in the past.
And so the diameter here is just the length of the trellis basically.
So this is i n minus 1, i n. And here we have alpha n. When we have alpha n finally in here, we can compute the extrinsic information based on alpha n as just extrinsic n. We combine these two and we're done for that guy.
Meanwhile, we have to compute the backward-going information on each of these branches.
So we compute first b n based on i n. Then we get b n minus 1 based on i n minus 1 and b n, and so forth.
We get a backward-going propagation of conditional information.
Each of these betas represents the a-posteriori probability vector given the future from it.
And finally, when we get down to here we can compute extrinsic 0 from data 1.
And by this time, when we have both alpha 1 and beta 2 coming into here, we can, of course, get extrinsic 1.
When we get both the alpha and beta coming in here, we can get extrinsic 2, and so forth.
Of course, you can write this down as equations.
If you want to read the equations, you look in the original Bahl, Cocke, Jelinek, and Raviv article in 1973.
This was again, a kind of lost paper because just for decoding a trellis, the Viterbi algorithm is much less complex.
And gives you approximately the same thing.
I mean, I talked last time how probably what you really want to do in decoding a block code is maximum likelihood.
That's minimum probability of error in a block-wise basis.
What APP decoding gives you minimum probability of error.
If you then make a decision based on each of these extrinsic information, or on the combination of these two to get the overall APP vector, that's called a map algorithm, maximum a-posteriori probability.
If you make a decision based on that, that will give you the minimal bit error probability, but it may not necessarily give you a code word, and it will not minimize the probability of making any error, which is generally what you want to minimize.
So for trellis decoding of block codes this was not favored.
However, a block code as a component of a larger code, you want the APP vector to feed off to something else.
And so with the advent of much larger codes, of which block codes are components or convolutional codes, BCJR is the way you want to go.
And some people say it's about three times as complicated as the Viterbi algorithm.
All right, so you'll have an exercise with doing that in the homework.
There is a closely related algorithm called the min-sum algorithm, or the max-product algorithm.
And the idea of that is that you can really carry through the same logical computations except when you get to one of these combining operations, rather than doing a sum of products, you can do a max of products.
And that the Cartesian product law still holds, the same kind of decompositions that we have still hold.
For instance, in this example, to get the state 0, 0, 0 before what we said we're going to get an APP vector where this is the past APP for the 0, 0 value of the middle state vector.
We simply combine the likelihood weights of the two possible ways of getting to that state and we sum them.
Instead of doing sum, let's just take max.
The maximum probability of getting-- in other words, what's the best way of getting here?
Same question as we ask in the Viterbi algorithm.
We'll let that be the likelihood weight for the state vector.
Otherwise, the logic is exactly the same.
Similarly, coming this way, what's the max of these two?
OK, what's the best way of getting to the state vector in terms of likelihood from the future?
And what do you know, if we combine these things at this point, we now have a past vector and a future vector.
Let's take the max of this times the max of that.
That gives us the maximum likelihood way of going through the 0, 0 value of the state.
If we do exactly the same logic for decoding this trellis, we get a two-way algorithm.
It's like this.
It's like the BCJR algorithm, where at every point we're computing the maximum likelihood.
It's not hard to show that what this gives you is that each of these symbols out here, it gives you the symbol that belongs to the maximum likelihood code word.
How does it differ from the Viterbi algorithm?
It gives you the same answer.
It gives you the symbols, one by one, of the maximum likelihood code word.
It doesn't have to remember anything.
It doesn't store survivors.
That's its advantage.
It's disadvantage is it's a two-way algorithm.
You have to start from both ends and propagate your information in the same way.
And for block codes, maybe that's OK. For convolutional codes that's certainly not something you want to do because the code word may go on indefinitely.
So the Viterbi algorithm gets rid of the backward step by always remembering the survivor.
It remembers the history of how it got to where it is.
So that once you get the max, you don't have to-- you simply read it out rather than having-- this backward part amounts to a trace back operation.
That's probably much too quick to be absorbed.
But I just wanted to mention there is a max product version, which if you take log likelihoods becomes a max-sum.
Or if you take negative log likelihoods it becomes min-sum.
So this is called the min-sum algorithm.
And it does maximum likelihood decoding rather than APP decoding.
And sometimes it's used as an approximation or very intermediate approximations between these two
[UNINTELLIGIBLE PHRASE]
No, if you use the min-sum, basically everybody will converge on the same maximum likelihood code word.
So all these constraints will be consistent with-- you'd be taking a single max at each point.
But at each point you will solve for the maximum likelihood code word.
And what you'll see up here is the symbol that belongs to the maximum likelihood code word.
This is maybe a bit of a miracle when you first see it, but you'll independently get each of the bits in the maximum likelihood code word.
Check it out at home
Why not always do this?
Why not always do that?
It turns out that we want softer decisions when this is part of a big graph.
We actually want the APPs, the relative probabilities of each of the values.
The max, in effect, is making a hard decision
[UNINTELLIGIBLE PHRASE]
Yeah, all right.
It does give reliability information.
I'm not sure I can give you a conclusive answer to that question except APP works better.
That's a more empirical answer.
Yes?
[UNINTELLIGIBLE PHRASE] Where does that flexibility of the input symbols being [UNINTELLIGIBLE PHRASE].
We should be able to take that into account [UNINTELLIGIBLE PHRASE]
Yeah.
OK, well, if the input symbols are independently not equiprobable, you can feed that in as-- you know, once they came out of the channel, once we see the channel information, the two symbols are not equiprobable anymore.
So if somehow they a priori independently were not equally probable, you could just feed that in as part of this intrinsic information vector.
That's almost never the case in coding.
The intrinsic information is sometimes regarded as a-priori and this has a-posteriori, or the combination of the two of them is then a-posteriori when this goes off somewhere else.
But it doesn't really mean that the symbol itself was-- it means that the evidence is biased one way or the other.
And when this appears somewhere else in the graph, this will now appear somewhere down in some other graph.
It's called the a-priori information, but what it is is the a-posteriori information given all of the received symbols in this part of the graph
[UNINTELLIGIBLE PHRASE] Does the i0 of that other graph contain this table sharing information added to it?
Yeah, the i0 for another graph is just the e0 of this graph.
I mean, if you consider it all as one big graph, that's what you want.
You want the message that goes in this direction, which basically summarizes all of the inputs from this graph.
Generally, you'd have a little equals node here.
You'd have the actual intrinsic information coming from the outside world there.
You would compute the e0 from this graph.
And at this point, you would combine e0 and i0, and that would go around here.
So the very last page of chapter 12 just says, OK, we've got a nice, clean, finite exact algorithm that will compute all these APPs on a cycle-free graph.
Now, suppose we're faced with a graph that has cycles.
And here's the situation.
Suppose the graph has cycles in it.
Then first thing that might occur to you, at least after this development is, well, we now have a local rule for updating at each of these computational nodes, each of these constraint nodes, in this representation.
Why don't we just let it rip?
See what happens.
We're going to put in some intrinsic information at the outputs as before.
We can compute some output message here based on-- well, eventually we're going to get some input message from here.
And we can compute some output message here based on all of these and we'll pass that over.
Let's assume what's called a parallel schedule or a flooding schedule, where in each cycle every one of these little guys does its thing.
It takes all the inputs that it has available at that time and it generates outputs.
Well, hope for the best.
It will compute something.
You can sort of intuitively see that it'll settle.
It'll converge, perhaps, into some kind of equilibrium.
Or hopefully it will converge into some kind of equilibrium.
There is a basic fallacy here, which is that the information that comes in here is used to compute part of this message.
Which is used to compute part of this message.
Which ultimately comes back and is used to compute part of this message.
And then part of this message again, so that information goes around in cycles and it tends to be used again and again and again.
Of course, this tends to reinforce previously computed messages.
It tends to make you overconfident.
You heard some rumor and then the rumor goes all around the class and it comes back and you hear it again.
And that tends to confirm the rumor.
So it's that kind of telephone situation, and it's not really independent information.
Intuitively, makes sense that if all of the cycles are very large-- in other words, the rumor has to go to China and back before you get it.
That it probably is highly attenuated by the time you get.
It's mixed up with all kinds of other information, so maybe it doesn't hurt you too much if the cycles are very large.
And in fact, that's the way these capacity approaching codes are designed.
Whereas, as I mentioned last time, if you're in a physical situation where basically your graph looks something like this, you don't have the freedom to make your cycles large, then this is probably a bad idea.
And this isn't why in fields, like vision, for instance, people try to use this-- the sum-product algorithm is the belief propagation algorithm, if you've ever heard of that.
Widely used in inference on Bayesian networks.
And the religion in belief propagation used to be that you simply can't use belief propagation on graphs that have cycles.
Why?
Because most of the graphs they dealt with look like this.
And in fact, it works terribly on that.
It was a great shock on that field when the coding people came along and said, well, we use belief propagation on graphs with cycles and it works great.
And this, as I understand it, really had an enormous impact on the belief propagation community.
But it was realized that gee, coding people have it nice.
You make your own graphs.
And you make them so this is going to be a very low order, low impact event.
And that's right.
But coding we do have this freedom to design our own graphs.
We realize from the basic arguments that I put up before that we're really going to need to have cycles.
As long as we have cycle-free graphs, we're going to get this kind of exponentially increasing complexity, which is characteristic of trellis graphs.
So we're going to need cycles, but as long as the cycles are very long and attenuated, as long as the girth of the graph is great, maybe we can get away with it.
And that, in fact, is the way the story has developed.
And Gallager basically saw this back in 1961, but it took a long time for it to catch.
So I didn't even get into chapter 13.
Chapter 13 we're going to first just introduce several classes, the most popular classes of capacity approaching codes.
And then I'm going to give you some real performance analysis, how you would design, simulate, in these codes.
And it's what people actually use.
OK, see you next time.
This time we're actually going to be able to do a little work that I hope will give you a good idea of why it is that at least low-density parity-check codes work and even broader classes of capacity-achieving codes.
But I think we're going to limit ourselves to a week, so that we can say something next week about everything we've been talking about as binary codes for the power limited additive white Gaussian noise channel.
We've completely left aside codes for the bandwidth limited channel, where we're going to have to talk about non-binary codes of some kind.
So next week I'll give you a very rushed overview of that kind of code.
And it's my intention, by the way, that last week, you'll not be held responsible for on the exams, so that we kind have a nice, pleasant week without worrying about whether we're responsible for it or not.
And the last homework set, I will prepare a homework set on the bandwidth limited type codes.
On Wednesday, I'll hand it out, but I won't expect you to give it back.
It's good to do it.
I'll hand out solutions on the final day of the exam.
OK, so any questions about where we are, what we're doing?
All right, let's get into it.
We've been building up to this for a while.
First we did trellis representations of block codes.
As a warm-up for codes on graphs, we found that some of the things we discovered about trellises were important particularly in the context of the cut-set bound.
The cut-set bound says, where you have a cut-set, the complexity at that point must be at least as great as that of a trellis diagram which divides the two parts of the code into two halves, and the past and future in the same way.
And from that we concluded that we're going to need cycles.
Because we'd already proved back in chapter 10 that trellis complexity needed to go up exponentially with block length, at least for codes that maintained a nonzero relative minimum distance as well as a nonzero relative rate.
So we're going to need cycles in our graph we think -- to get to capacity.
Chapter 12 was a fairly short chapter introducing the sum-product algorithm.
I believe that you don't really understand the sum-product algorithm till you do it once, so that's on this week's homework.
You will do it once for a very simple case, and then I think you'll understand it.
We developed the sum-product algorithm for cycle-free graphs, where it's a theorem that the sum-product algorithm does APP decoding for every variable in any cycle-free graph in a very systematic and efficient way.
It's a finite and exact algorithm in that case.
It takes a number of steps approximately equal to diameter of the graph.
And when it's finished you have the exact APPs everywhere.
OK, but we really want to have a decoding algorithm for graphs with cycles.
Having developed the sum-product decoding algorithm, we see that it has a nice, simple update rule at each computation node.
It's a local kind of algorithm.
Why don't we just let her rip, start it off and see how it does?
And, in our case, we can make that work by designing graphs that have, in particular, very large girths such that the graph looks-- the term-of-art is locally tree-like.
In the neighborhood of any node, if you go out some fixed diameter from that node, you won't run into any repetition.
So locally the graph looks like a tree.
And we're going to, ultimately, be looking at asymptotically long codes, so we'll be able to make that locally tree-like assumption hold for as far out as we need to.
Other disciplines are not so fortunate as to be able to do that.
OK, so in this chapter we're finally going to get capacity approaching codes.
I'm first just going to mention a couple of classes of codes, the most famous and important ones, show you what their graphs look like.
From that, you will know everything up to schedule what there sum-product decoding algorithm should be.
I'll tell you what the conventional schedules for decoding these graphs with cycles are.
And that'll be an introduction.
And then we'll get into actual analysis, in particular, for low-density parity-check codes which are, really, the simplest of these codes and therefore the most amenable to analysis.
And we'll see that, under the locally tree-like assumption, we can do an exact analysis first, on the binary erasure channel.
Then I'll at least discuss what's involved on more general channels, where we can do a fairly precise analysis.
So that's where we're going in chapter 13.
Questions, comments?
OK, so let's talk about low-density parity-check codes.
These codes have a fascinating history.
They were invented by Professor Gallager in his doctoral thesis in 1961.
He was a student of Peter Elias.
What people were doing at MIT then was trying to figure out ways to actually build codes that get to capacity.
At MIT the emphasis was on so called probabilistic codes.
Namely ones that had sort of the random-like elements that Shannon basically prescribed.
People here were not so interested in the algebraic codes.
And Gallager basically had the idea that if you take a code -- you can describe it by a generator matrix or parity-check matrix.
If you make a sparse generator matrix, that's not going to be any good, a random, sparse generator matrix, because a sparse generator matrix will naturally have some low-weight code words in it.
If the rows are sparse, then it will have low, minimum distance.
But maybe if we make the parity-check matrix sparse, very few ones in the parity-check matrix, we can still get a code that's quasi-random.
And therefore we can hope that the codes, as they get along, will come close to doing what Shannon said random codes should easily be able to do.
The amazing thing about Shannon's proof is he showed that you pick any code out of a hat, and it's highly likely to be good.
So, basically, Gallager was trying to replicate that.
And he came up the idea of sparse or low-density parity-check matrices and was able to prove quite a few things about them.
That they couldn't quite get to capacity the way he formulated them, but that they could get very close.
They had a threshold a little bit below capacity.
He came up with the APP decoding algorithm for them which is probably the first instance of the sum-product algorithm in any literature, certainly the first that I'm aware of.
And he simulated them.
At that time, what MIT had was-- over in Building 26 on the first floor, where I think there's a library now.
It sticks out into the adjacent parking lot.
That room was full of IBM equipment, 70, 90 computer.
And there were people in white coats who attended all that equipment.
And by running that computer for hours, he was able to simulate low-density parity-check codes down to an error probability of about ten to the minus fourth.
And he showed that he could get down to 10 to the minus fourth, pretty close to capacity.
I don't remember exactly how close, but it was enough to be convincing.
Actually, in 1962 there was a little company form called Codex Corporation.
It was what we would now call a start-up.
Back then, there was no tradition of start-ups.
And it was basically founded to exploit Gallager's patents, which he took out on low-density parity-check codes, and Jim Massey's patents on threshold decoding, which is an extremely simple decoding technique that we haven't even talked about, very low complexity.
And I joined that company in 1965 when I finish my thesis.
And I was director of research and advanced product planning.
And I can assure you that in the 20 years that those patents lived, we never considered once using Gallager's low-density parity-check codes, because they were obviously far too complicated for the technology of the '60s and '70s.
We didn't have rooms full of computers to decode them.
And more generally, in the community, they were pretty completely forgotten about.
There's a key paper by Michael Tanner in 1981 where he kind of-- that's the founding paper for the field of codes on graphs.
he puts it all into a codes-on-graphs context, develops all the basic results.
It's a great paper.
It too, was forgotten about.
And the whole thing was pretty much forgotten about by us until other people, outside the community, rediscovered it right after turbo codes were invented.
Turbo codes were first presented in 1993 at the ICC in Geneva.
They had actually been invented some four or five years before that by Claude Berrou in France.
And nobody could believe how good the performance was.
This guy is not even a communications guy.
He must have made a three dB mistake, didn't divide by two somewhere.
He showed you could get within about a dB of capacity with two simple codes.
And we'll see that in a second.
But, sure enough, people went and verified that these codes work the way he said they did.
And, all of a sudden, there's an explosion of interest.
And low-density parity-check codes were rediscovered by a couple of people, notably David MacKay who also was a physicist like Berrou, in England.
But he had an interest in information theory, at least he was trying to solve the right problem.
And he had similar ideas to what Gallager had had.
And he simulated them-- by now, you can simulate these codes on a desktop computer-- and found that they got within a dB of capacity and so forth.
And so then we were off to the races, and the whole paradigm of coding changed.
But it certainly is an embarrassment to those of us who were faithfully in the field for 30, 35 years.
Why didn't it occur to us that by the '90s the technology was far enough advanced that we could go back and look at what Gallager did and realize that these were actually pretty good codes and now implementable?
So I think that's a nice story.
It's kind of humbling to those of us in the field.
On the other hand, we can say that, jeez, Gallager did it back in 1960.
What's the big deal about turbo codes?
Depends what perspective you take.
I never know whether I should tell a lot of stories or not so many stories in this class.
If you like more stories, just ask.
Anyone have any questions at this point?
All right, low-density parity-check codes, Gallager in 1961, forgotten until 1995, rediscovered by David MacKay, Niclas Wiberg, and others.
And this was what they look like.
The basic idea is we have a parity-check representation of the code, a long code.
So the code is the set of all y such that yht equals 0.
And that's standard.
And the idea now is that this parity-check matrix should be low density or sparse.
In fact, the number of ones in it-- It's a matrix, so that you would think a random matrix is 0s and ones.
m minus k by n matrix would have a number of ones that goes up as n squared.
As n becomes large, let's make sure that the number of ones only goes up linearly with n, as the block length n becomes large.
So sparse h means number of ones linear with n. And we'll see that implies that the complexity of the graph is linear with n. The ones are going to correspond to the edges in the graph.
And we'll have a linear number of edges, also a linear number of nodes or vertices.
OK, in general, what does a code like this look like?
We have y 0, y one through-- what does its graph look like, I should say-- yn minus one.
And then, so we're going to have n symbols, which in normal graphs we write as equals nodes.
Then over here we're going to have n minus k, a lesser number of constraints or checks, which in our notation we've been writing like this.
And we're going to have an edge for each one in here.
You can easily see that every one in the parity-check matrix ensures that it says that one of the symbols participates in one of the checks.
And therefore, the number of edges is equal to the number of ones.
So this is equal number of edges in graph.
Not counting these little half edges out over here, again.
So there's some random number of edges.
We won't have them got to the same place.
But this also is only linear with n, here.
And, in fact, let's choose these edges.
These edges are like a telephone switchboard.
It's just a giant plug board where everything that goes out here, you can consider going into a socket.
So let's draw, for instance, for a regular code, let's draw the same number of sockets going out for each of the symbols.
Each symbol is going to participate in the same number of checks.
Each one is going to have the same degree, so that's why, in graph terms, it's regular.
And each one of these checks, we're also going to say has the same degree.
So this was basically the way Gallager constructed codes, regular.
We'll later see if there's an advantage to making these somewhat irregular, these degree distributions.
But this is certainly the simplest case.
All right, how do we make a random parity-check code?
So that we're going to specify n, n minus k. We're going to specify these degrees.
So that's not random.
But we'll connect the edges through a big switch board which is denoted by pi, which stands for permutations.
This is basically just a permutation.
We've got to have the same number of sockets on this side as on this side.
And this is a just a reordering of these sockets.
That's what a telephone switchboard does.
Or it's very often in the literature called an interleaver, but, probably, permutation would've been a better word.
All right, so one way of doing it is just to take this edge here, this socket here and, at random, we have-- let's say, e, for number of edges-- e sockets over here and just pick one of those sockets at random.
Take the next one and take the e minus one that are left and connect that to one of the the e minus one at random.
So this would give you one of the e factorial permutations of e objects.
And it'll give you one of them equiprobably.
So we're basically talking about an equiprobable distribution over all possible permutations of e elements.
All right, so we consider all these are equally likely when we come to our analysis.
This is why these codes are called probabilistic.
We set up some elements of their structure, but then let others be chosen randomly.
Clearly it's not going to matter for the complexity of the decoding algorithm how this is done.
It might have something to do with the performance of the decoding algorithm, but any decoding algorithm is going to work the same way.
It's just a matter of when you send a message in here, where does it come out here or vice versa.
So it doesn't really affect complexity.
Let's see, there has to be some relation between the left degree d lambda and the right degree d rho here in order that we have the same number of edges on each side.
We have e equals n times d lambda, if we look at the left side.
But it's also equal to n minus k times the degree on the right side.
Again, I'm only counting the internal degree here and leaving aside the external guy.
And this gives us the following relationship that n minus k over n is equal d lambda over d rho.
Or, equivalently, with this is one minus the rate of the code, or we can write rate equals one minus d lambda over d rho.
So that by deciding what left degree and what right degree to have, we're basically enforcing this relationship which comes out as the rate of the code.
For instance, the choice that we're going to talk about throughout is we're going to take a left degree equal to three, right degree equal to six, as I drew it.
If you do that, then you're going to get a code of nominal rate 1/2.
It's only a nominal rate because what could happen here?
I'm basically choosing a random parity-check matrix, and there's nothing that ensures that all the rows of this parity-check matrix are going to be independent.
So I could have fewer than n minus k independent rows in the matrix, which would actually mean the rate would be higher than my design rate, my nominal rate.
In fact, you can ignore this possibility.
It's very low probability even if you choose it at random.
If it does happen, people are going to forget about the dependent row and pick another independent row.
So I won't make a big deal.
We'll just call this the rate.
So if this were three here but only four here, then we would have a rate 1/4 code.
Well, we'd have to have 3/4s as many checks as we have symbols in order for everything to add up.
So that's really, basically, it.
Let's just specify the degrees on the left, the degree on the right, pick our permutation at random, pick n or e. We've designed a low-density parity-check code.
And now, how do we do we decode it?
Well, we use the sum-product algorithm.
We've got a code on a graph.
In fact, it's a very simple graph, notice it's characteristic.
It's, again, just got equality nodes and zero-sum nodes or check nodes.
These are really the simplest codes.
We could, more generally, have more elaborate codes here.
That's one of the things Tanner did in is 1981 paper.
But this is very simple.
All of our internal variables are binary.
I, for one, thought that it would be necessary to go beyond binary internal variables in order to get to capacity but that proved not to be the case.
Really, this structure is all you need to get to capacity it turns out.
Except for one thing, you need to make these degrees irregular.
And for the most part, you only need to make the left degrees irregular.
So we'll get far enough to see why that is going to do the trick.
But right now, I talked about the decoding algorithm.
So the decoding algorithm is pretty much what you would think, I would hope.
Now that you've seen some product decoding.
You receive something on each of these lines, depending what the channel is.
If it's an additive white Gaussian noise channel, you get a real number, received vector that gives you some intrinsic information about the relative likelihoods of 0 and one on each of these lines.
So each of these points, you get an in going message based on what you receive.
At a repetition node that message is simply propagated, if you look at the equations of the sum-product algorithm.
Repetition nodes just propagate what they receive at one terminal on all the output terminals.
All the d minus one terminals are determined by the by one terminal.
So we get the same thing going as and output message.
This, very often, is drawn as a Tanner graph.
In that case, all we have here is symbols.
That's natural to just take the likelihoods of the symbols and send them out.
So we haven't done anything more than that.
These messages all arrive here as incoming messages from various far and distant lands.
To find the outgoing message say, on this line here, what's our rule?
Our rule is to take the incoming messages on all d minus one of these nodes and combine their likelihoods according to which of these is consistent with a 0 output here, and which of these is consistent with the one going out here.
We get a bunch of weights for 0, and a bunch of weights for one.
We add them all up.
And that gives us now, a re-computed likelihood.
A message consisting of probability of 0 given what we've seen so far, and probability of one given what we've seen so far.
But now it goes out.
Hopefully that's got some improved information in it now coming from all different parts of the graph, extrinsic.
So this, eventually, will come back somewhere say, here and similarly on all the other lines.
And now this will give you additional information.
You combine these, and now what you have coming out here is a combination of what you had coming in here, which was originally a state of complete ignorance, but now you combine these you get more information going out and so forth.
So you iterate.
You basically do a left side iteration, then a right side iteration, a left side iteration, a right side iteration.
Hopefully the quality of your APP vectors is continually increasing during this.
You're somehow incorporating more and more information.
You're hopefully converging.
And so, you just keep going back and forth.
And low-density parity-check codes you can do this typically 100 times, 200 times.
It's a very simple calculation.
You can do it a lot.
So you can do that.
When do you stop?
A very popular stopping rule is to stop as soon as you've got-- If you made hard decisions on the basis of all these messages, that would give you binary values.
And if all those binary values check at all these places, then that's a code word.
Or that's a trajectory that's consistent with the code word.
Basically, everything that's over here then has to be a code word.
So whenever all the checks, check just based on hard decisions, you stop.
So if there's very little noise to start with, this can happen very quickly and terminate the calculation well short.
Or you have some maximum number of iterations you're going to allow yourself.
If you go 200 iterations, and these things still don't check, probably you would say this, I've detected a decoding error.
One characteristic of low-density parity-check codes, is they tend to have good minimum distance.
They tend to have a minimum distance that's sort of what you would get in a random-like code, which goes up linearly with the block length.
So for a rate 1/2 of code of length a 1,000, a random code would have a minimum distance of about 110.
And a low-density parity-check code would typically have a large, maybe not 110, but a large minimum distance, unless you were just unlucky.
So the probability of your actually converging to an incorrect code word is tiny.
You don't actually make decoding errors.
All of your errors are detectable.
Which is a nice systems advantage some of the time for low-density parity-check codes.
Yes?
[INAUDIBLE] possible that if you add one variation [INAUDIBLE]
Possible in principal but extraordinarily improbable.
And once you've converged to a code word, basically, you've got all of all the messages lined up in the same way.
It's like a piece of iron ore where you've got all of magnets going in the same direction.
What is there in that that's going to cause them to flip?
There's no impetus for any of these to-- They'll tend to get more and more convinced that what they're doing is right.
They'll reinforce each other.
So, as a practical matter, I don't think you would ever-- The kind of landscape of these codes is that there's large areas of the parameters where it doesn't tell you to go in any particular direction.
And then there are deep wells around code words.
So, typically, the decoding algorithm is kind of wandering around the desert here, but then once it gets into a well, it goes wrong boom, down to the bottom of that well.
And once it's down here, it's extremely unlikely that it'll spontaneously work itself out come back over here again.
Powerful basins of attraction here.
Which is another reason why when we design the code, we're basically designing this landscape.
And we can design it to-- when it starts to converge, it really converges fast.
That's why some classes of these codes are called tornado codes.
They wander for a while, and then when they get the idea, then [WOOM], everything checks very quickly.
Yes?
[INAUDIBLE] so you would compute the-- at each [UNINTELLIGIBLE] calculate one of the so do you calculate all of the messages that go back from each [UNINTELLIGIBLE] first and then propagate then back--
Yes, correct.
I'm sorry.
We nominally calculate this one, but at the same time, we calculate every other one.
All e of them.
Each one of these is based on the d rho minus one-- other inputs.
So it ignores the message that's coming in on that particular edge.
It uses all the other messages to give and extrinsic message going out.
Which you could combine back here or here, there the same thing, to give you your best overall APP right now from your right going in your left going messages.
But you don't.
You now use this new information in your next round over here.
And the same thing on this side.
You always compute one on the basis of all of the others.
And the initialization is that initially you have no information coming this way.
You can represent that by an APP vector that's 1/2, 1/2.
Probability of one is 1/2, probability of 0 is 1/2.
Any other questions?
OK, so that's how it works.
And it works great.
And, as we'll see, even with regular codes you can get within-- additive white Gaussian noise channel terms, you can get within about a dB, a dB and a half of capacity.
And merely by making these degree distributions irregular, this is how, Sae-Young Chung got the results that I showed, I think, in chapter one, where you get within 0.04 dB of white Gaussian noise capacity and that's it.
So we knew how to get to capacity since 1961 almost, we just didn't realize it.
Yeah?
With the LDPC regular [UNINTELLIGIBLE]
I'm sorry, I missed--
With regular LDPC you can still have a certain gap or?
With regular LDPC there's a threshold below which the algorithm works, above which the algorithm doesn't work, as we'll see shortly.
And the threshold is not capacity.
It's somewhat below capacity.
By going to irregular you can make that threshold as close to capacity as you like.
You can never make it equal to capacity.
All right, so let's talk about turbo codes.
None of these ideas is very difficult, which is, again, humbling to those of us in the community who failed to think of them for 35 years.
And I really hope some historian of science someday does a nice job on culture, and why some people were blind to this and some people saw it and so forth.
OK, so this is Berrou, Glavieux, and Thitimajshima-- which I always have problems spelling-- in 1993.
And their idea was pretty simple too.
Their idea is based on two, rate 1/2, convolutional codes.
Let me use notation.
We have a certain u of d, which is a set of information bits.
And to make a rate 1/2 convolutional code, all I have to do is pass that through a certain g of d. So I'm saying my generator matrix for the convolutional code is one and g of d. That'll make a rate 1/2 code.
And this, I'm going to call that a parity bit sequence, occurring at the same rate as the information bit sequence.
Easy enough, now, let me take this same information bit sequence, and let me put it through a large permutation.
Which, again, I'll represent by pi.
And I want you to think of this as taking a 1,000 bit block and mixing it all up or something like that.
Actually, it's better to use a so-called convolutional permutation which is kind of stream based, continuous, the same way that convolutional codes are.
All right, but it's basically the same sequence permuted.
And let's also put that through another g prime of d, which, very often, is g of d. So I'll just call that g of d again.
And this is second parity bit sequence.
And this all together, I will call a rate 1/3 code.
For every one information bit in, there are three information bits out.
Same sense as convolutional.
All right, so the second code, g prime of d is one g prime of d. And the interesting thing is, these codes don't need to be very complicated.
One thing they realized is they do want this to be a recursive code.
In other words, g of d should look like n of d over d of d. And one particular purpose of that is that so a single information bit will ring forever.
The impulse response to a single information bit is infinite.
It takes at least two.
And you can prove to yourself there's always some sequence of length two in order to get a finite number of parity bits, a finite number of nonzero parity bits out of here.
And when those two bits are permuted, they're going to come up in totally different places, so they are going to look like two individual information bits down here, and therefore they're going to ring for very long time.
The basic idea is, you don't want an information sequence which gives you a short code word out of here and a short code word out of here.
And making it recursive, have feedback is what that means, is what you need to do to prevent that.
It also gives you better codes, but I don't think that's really why-- If you have g of d over n of d, this is going to be equivalent to the code we're multiplying to the non-recursive feedback free code, which is d of d, n of d, right?
We worked all that out.
It's equivalent to the code where you multiply through by the denominator.
And so you get the same code with d of d, n of d, but you don't want to do that, because you are concerned here about the input output relationship between input bits and the encoded bits.
And you want the information bit sequence to come through by itself, because it's going to be reused.
You want the information bit sequence to be in common between these two.
That's going to be the length when we decode these codes.
Basically, the messages concerning the information bit sequences is what's going to go back and forth between the two codes.
All right, do you get the set up here?
I gave you a very different perspective on it by drawing the graph of this code.
I mean this is a graph of the code, but it's not one we want.
Let's draw the graph of this code.
So turbo code graph, I'll draw a normal graph, but again, if you like to do it as a Tanner style, be my guest.
All right, this first code-- Let me do this in two steps.
First of all, a rate 1/2 code graph.
So for one of these code, what does it look like?
It's a trellis graph.
Which in the normal graph picture is a very simple chain graph.
It looks like this.
Here's the information sequence.
Here are the states.
So these might be simple codes.
These are typically four to 16 states, so two to four bits of state information, not very complicated.
So this is what it looks like.
We get one of these for each.
This is the branch constraint for each unit of time.
This would be y0, y1, y2, y3, you get two bits out for each unit of time.
And this case, one of these is actually the input at that time, u0, u2.
Or we could adopt some other indexing scheme.
It's systematic so, we think of these as the inputs sort of driving an output.
So maybe a better way to do this is-- an alternative way of doing it is like this.
But it's all the same thing, right?
It's what you've already seen.
So I just want you to get used to this way of writing the graph of a trellis of a rate 1/2 systematic convolutional code.
Here are these systematic bits, which are both inputs and, ultimately, outputs on to the channel.
And these are the parity bits which are only output onto the channel.
All right, and these are the branch constraints for each time.
And these are the state's basis for each time.
So that I can always draw that, that way.
All right, now up here, I really have two of these codes, possibly identical.
And what's the connection between them?
The connection between them is that they have the same information, but in some random order, some pseudo-random interleaved order.
So I draw a graph of that, again, focus on the interleaver.
Make it nice and big now.
There's always going to be a big interleaver in the center of all our charts.
This is always the random-like element.
Now, let me draw my first convolutional code over here.
I'm just going to tilt this on its side.
I'm going to put the parity bits on this side, the first parity sequence.
So now this is running in time either up to down or down to up.
I don't care.
So forth, down to here, et cetera, dot, dot, dot, dot, dot, dot-- And I'll put the information bit over here.
And I'm going to do it in this way.
The information bits are, first of all, used in this code.
So I have a little equality.
So these are going to be in my information bits.
But then they're going to go through this permutation and also be used in another code over on the right side.
Is this clear?
So this is everything up to here.
I'm about to go into the interleaver.
Stop me with questions if anything is mysterious about this.
OK, after some huge permutation, they come out over here.
And now these are information bits that go into another identical trellis-- or could be identical.
So we have a trellis over on this side that's connected to one parity bit and one information bit.
It's another rate 1/2 trellis.
This is parity bits, these are, again, info bits.
Same information bit sequence but drastically permuted.
Is that clear?
So again this is the same idea of a switchboard with sockets we have the same number sockets on the left side and on the right side.
And we make a huge permutation that ties one to the other.
It's only edges in here.
Notice there's no computation that ever takes place in the interleaver.
This is , again, it's purely edges, purely sending messages through here.
So the analogy of a switchboard is a very good one.
Even though we don't have switchboards anymore.
Maybe some of you don't even know what a telephone switchboard is?
It's a quaint but apt analogy.
All right, so this is the normal graph of a turbo code.
I've drawn it to emphasize its commonality with low-density parity-check code.
The biggest element is this big interleaver in here, and that's key to the thing working well.
All right, so how would we apply the sum-product algorithm to this code?
What would our decoding schedule be?
In this case, when we observe the channel, we get messages in all these places.
So we get intrinsic information coming in for all the external symbols.
First step might be to
[INAUDIBLE]
That's first step, we can-- In the same way, this is just a distribution node when originally all else is ignorance.
So these same messages just propagate out here, and if we like, we can get them out over here.
All these are available too.
OK, somebody else.
What's the next step?
Somebody was just about to say it.
Be bold, somebody
[INAUDIBLE]
Excuse me?
The trellis decoding--
Trellis decoding, right.
If we forget about that side and just look at what the sum-product algorithm does over here, we're doing the sum-product algorithm on a trellis, a BCJR algorithm.
So given all these messages, we do our backward forward algorithm, alphas and betas, eventually epsilons, extrinsic information coming out on all nodes there, there, there, and so forth.
And that requires going up and down the whole trellis.
If this is a 1,000 bit block, as it often is, or larger it means going from top to bottom through the whole block and doing one sweep of a BCJR algorithm through that block, which will generate a bunch of extrinsic information over here.
Now, we have more over here.
We still got the same intrinsic information coming in here, but now we can take this new extrinsic information and combine it with this intrinsic information and, hopefully, get an improved, higher quality information.
Because we've incorporated all the information from all these receive symbols on this side, and we send that over here.
And if you think of it in signal to noise ratio terms, as some people do, you hopefully have improved the signal to noise ratio in your information that's coming out.
It's as though you received these bits over a higher quality channel.
They tell you more about the information bits then you got just from looking at the raw data from the channel.
That's basically where turbo comes from.
You then do BCJR over here using this improved intrinsic information, of course, the parity information hasn't improved.
You might think of ways to improve this too.
But this was the basic scheme.
It works just fine.
You come over here.
You then do a complete sweep through a 1,000 bits or whatever, and you develop outgoing messages here.
Which again, let's hope we've improved the signal-noise ratio a little bit more, that these are even more informative about the input bits.
And they come back, and now, with this further improved information, we can sweep through here again.
That's where the turbo comes from.
You keep recycling the information left to right and right to left.
And, hopefully, you build up power.
You lower the signal-noise ratio each time.
Yes?
[INAUDIBLE] and then you [UNINTELLIGIBLE] the other side is kind of [?
encouraging.
?]
Now, why can't you, because on this side, actually, the equilibrium is just the [UNINTELLIGIBLE] version of the other side.
Why would you do it [UNINTELLIGIBLE] carry out, and do it again?
So a good question.
You certainly could.
So after you've done it twice, you basically processed all the same things as this thing does in one iteration, you spent twice as much effort.
The question is, is now the quality of your information any better?
And I haven't done this myself, but I suspect people in the business did this, and they found they didn't get enough improvement to make up for the additional computation.
But that's an alternative that you could reasonably try
The reason I say that is because if you do it that way, that means for the second decoding process of the left side, the information is very fresh.
Because it's sort of [?
right ?]
information.
If you do this and this and this, it looks like there is enough propogation of the same provision to itself
There certainly is.
There are certainly cycles in this graph
[INAUDIBLE]
OK, well, these are great questions.
What I hope that you will get from this class is that you will be able to go back and say, well why don't we try doing it this way?
Whereas everybody else has just been following what they read in the literature, and it says, to do it first here, and then there, and then there.
Try it.
Maybe it's good.
You're suggesting that suppose this guy it actually has poor information.
He could actually make things worse over here
Yeah, and also [INAUDIBLE]
So then if this guy just operated on the fresh information.
It's a probabilistic thing whether this would work better on the average.
I don't know.
I suspect that, once upon a time, all these things have been tried.
But certainly, if you're in any kind of new situation, new channel, new code, you might think to try it again.
It's a very reasonable suggestion.
Because computation is not free, there is a marked difference in turbo decoding in that you're spending a lot more time decoding each side.
This PCJR algorithm is a lot more complicated than just executing-- well, it's gone now-- but the sum-product update rule for equals or 0 sum node.
That's a very simple operation.
On the other hand, it doesn't get that much done.
You can think of this as doing a lot of processing on this information over here before passing it back.
Then this does a lot of processing before passing it back again.
And so for turbo codes, typically, they only do 10 or 20 iterations.
Whereas for low-density parity-check codes, one does hundreds of iterations, of much simpler iterations.
But, in either case, you've got to deal with the fact that as you get up in the number of iterations, you are getting to the region where cycles begin to have an effect, where you are re-processing old information as though it was new, independent information.
This tends to make you overconfident.
Once you get in the vicinity of a decision or of a fixed point, whether it's a decision or not, it tends to lock you into that fixed point.
Your processing, after a while, just reinforces itself.
Now, another thing, turbo codes in contrast to low-density parity-check codes, generally have terrible minimum distance.
If you can work through the algebra, and there's always some weight 2 code word that produces maybe eight things out here and 10 out here.
So the total minimum distance is only 20, no matter how long the turbo code gets.
All right, so when I say terrible, I don't mean two, but I mean 20 or 30 could be a number for minimum distance.
Certainly, this is called parallel concatenation, where you have these two codes in parallel, and just pass information bits back and forth between them.
It's typical that you get poor minimum distance which eventually shows up as an error floor.
In all these codes, when you draw the performance curve, as we always draw it, you tend to get what's called a waterfall region, where things behave in a very steep Gaussian like measure.
When you measure distance from the capacity, you might be-- Oh, let's do this first, assess an r norm.
So capacity is always at 0dB.
And this is probability of error.
So this is your distance from capacity, and this might be 1dB, or 3/10 dB, or something very good.
But then, as you get down below, some cases it could be 10 to the minus three or four, some cases it could be 10 to the minus seven.
So where it starts is important.
Then this tends to flatten out like that.
And this is simply what you get from the union-bound estimate for poor distance.
So, in this region, other things hurt you worse than the distance did.
If you are purely limited by distance, then you would have a performance like this.
There a very, tiny number of minimum distance code words.
There may be only a few in 1,000 length block, so the error coefficient is very small.
That's why this is way down here.
The interleaver tends to give you a 1/n effect on the error coefficient.
So this is way down here, and it doesn't bother you in the waterfall region, but then at some point it becomes dominant.
You can think of these as being two types of curves.
This is the iterative decoding curve.
This is simple maximum likelihood of decoding when you have minimum distance of d with a small error coefficient.
And so where this error floor is an important issue for turbo codes.
In some communications applications really all you're interested in is 10 to the minus four, 10 to the minus five.
And then you really want to put all of your emphasis on the waterfall region and just keep the error floor from being not too bad.
So that would be a very good application for turbo code.
But if, as some people think they want, error probabilities like 10 to the minus 12, or 10 to the minus 15, probably turbo code is not going to be your best candidate.
And it's not that you couldn't create a code with better minimum distance, but it's going to get harder.
I don't know that much effort has gone into this, because we have low-density parity-check codes for this other application.
Any questions on turbo code?
That's about all I'm going to say.
Yeah?
Why not expurgate the bad code words?
Expurgate the bad code words.
Well, how exactly are you going to do that?
Any malady report.
We disallow certain using [?
passes.
?]
So you're not going to mess with the g of d, but you're going to go through, you going to find all the u of d's that lead to low weight sequences, and then you're going to expurgate them.
Of course, it's a linear code, so if there's a low weight sequence in the neighborhood of 0, then there is going to be a corresponding sequence in the neighborhood of any of the code words.
So you're going to have to expurgate, somehow, around all the code words.
I don't know how to do it
You would sacrifice rate
You will sacrifice rate?
Oh, one other thing I should mention.
Very commonly in turbo codes, they don't send all of these bits.
They send fewer bits, which.
in effect, raises the rate.
But then, of course, makes decoding harder, because you don't get any received information from the bits that you don't send.
But there are various philosophies of what's called puncturing-- that I'm certainly no expert on-- and it's possible that by being very clever about puncturing you could do something about minimum distance.
Shortening is where you hold some of these bits to 0 and don't send them.
Puncturing mean to let them go free and don't send them.
So I don't know.
But I don't think this is a very profitable way to go.
But it's certainly a reasonable question to ask.
Any time that you get into a minimum distance limited situation, you should think about, is there some way I can just expurgate that small fraction of code words in the neighborhood of every code word that have that minimum distance.
This, as I said, is called parallel concatenation.
Sorry, I didn't mean to cut off questions about turbo codes.
I did say, I'm not going to talk about them again.
They, of course, gotten a lot of publicity, are very popular initially.
This was a revolution in coding.
Turbo codes got into all the new standards and so forth.
Every communications company had a small group simulating turbo codes.
And so they got a tremendous amount of momentum.
Low-density parity-check codes really didn't get looked at till a couple years later.
As you'll see, there's a lot more analytical work you can do a low-density parity-check codes, so they're popular in the academic community for that reason.
But, in fact, it turned out that the analysis allowed you to do fine tuning of the design of low-density parity-check codes.
There are more knobs that you can do, such that you really can get closer to capacity with low-density parity-check codes than with turbo codes.
They have other differences in characteristics, but, for both of these reasons, there is certainly a move now.
The low-density parity-check codes seem to be the favored ones in any new standards.
But we still have a lot of existing standards, and people who've already developed turbo code chips and so forth who have a vested interest against moving.
Perhaps analog devices is one like that.
So, although you can see the trend is toward low-density parity-check codes, there are many industrial and competitive factors and inertia factors.
They'll both be around for a long time
[INAUDIBLE] turbo code they have a patent you have to pay a royalty to the company.
And with LDPC you don't have to--
Yes, since this is a theoretical class, but, of course, issues like patents are very important.
Berrou, Glavieux, and all were working at ENST Bretagne up in Brittany.
They were funded by France Telecom.
France Telecom went out and owns all the turbo code patents and has decided to enforce them.
Whereas Bob Gallager's patents all-- We owned the ball.
We never made a dime.
They all duly expired around 1980.
And no one needs to worry about at least basic patents on low-density parity-check codes.
Of course, refinements can always be patented.
So I don't know what the situation is there.
And that often makes a big difference as to what people want to see standardized and used in practice.
So, excellent point.
All right, well, we can come back to this at any point.
Serial concatenation-- Here the idea is you, basically, go through one code, then you interleave, and then you go through another code.
This is the kind of setup that I talked about back when we talked about-- This could be a small block code or convolutional code that you do maximum likelihood decoding on, need to do a Viterbi algorithm.
And this could be a Reed-Solomon code, and the interleaver might be in there for some kind of burst protection or to decorrelate any memory in the channel out here.
So that's the context in which concatenated codes were originally invented.
That's the way they're used on space channel for instance, additive white Gaussian noise channel.
But here we're talking about the same block diagram, but we're talking about much simpler codes.
And again, the idea is to build a big code using very simple component codes, which is what we did in low-density parity-check codes.
That's a big code built up, basically, out of 0 sum and repetition codes.
Turbo code is a code built up out of two simple convolutional codes.
Here, I'm going to talk about repeat-accumulate codes just because they are so simple.
These were proposed by Divsalar and McEliece really for the purpose of trying to find a simple enough turbo code so you could prove some theorems about it.
There are hardly any theorems about turbo code.
It's all empirical.
We've got now some nice analytical tools.
In particular, the exit chart, which was a very nice, empirical engineering chart like Bode plots and Nyquist plots, things like that.
It has a real engineering flavor, but it works very well.
The repeat accumulate code is basically this.
We take input bits here.
We repeat n times.
So this, if you like, is an n one n repetition code.
That's pretty simple.
We have a stream and a long block, we would then permute everything within the block.
And then this is a rate one or one over one convolutional code called an accumulator.
This is the code with transfer function one over one plus d, that means that y k plus one is equal to y k plus x k plus one.
So we have x coming in.
Here is one delay element.
Here's y k coming out.
And there is what that looks like.
Xk plus one.
yk plus one.
So it's a very simple rate one convolutional encoder with feedback where I guess we take this as the output.
That's called an accumulator because, if you look at it, what is yk doing?
You can show that yk plus one is simply the sum up to k plus one of all the xk prime.
k prime equals whatever it starts at through k plus one of all the xk prime.
It's nothing more than saying than saying, g of d equals one over one plus d, equals one plus d plus e squared plus d. The impulse response to this is infinite.
And every past xk shows up in every y.
This is just the sum of all the xks.
But, of course, in the binary field it's the mod 2 sum, so it's only going to be 0 and one.
Well, it's a little two-state, rate one convolutional code.
About as simple as you can get and obviously useless as a standalone coded.
Rate one coders have no redundancy.
They can't protect against anything.
So that's the structure which prior to turbo codes or capacity approaching codes, you would have said, what?
What are we trying to achieve with this?
And the reason I talk about them it that the codes actually does pretty well.
All right, what's its graph look like?
Over here we have input bits.
The input bits aren't actually part of the output in this case, so I won't draw the dongle.
We just have an equals sign with n call n equals three, repetition from each.
Oh, what's the overall rate of this code?
Here we have a rate one over n code.
We don't do anything to the rate here, so the overall rate is one over n. So let's take a rate 1/3.
So the graph, dot, dot, dot, looks something like this.
And then, as always, this element.
We take these bits in.
We permute them in some sort of random permutation.
And then over here, what do we have?
Well, we have a trellis again.
In this case, I can draw the trellis very explicitly.
If we consider this to be the information bit, and this to be the previous state bit, so this is yk.
And here's-- We're considering the y's to be the output, so this is xk.
I think it works either way.
Let's call this x k plus one plus y k equals y k plus one.
So this enforces the constraint.
This is really the branch.
This is just to propagate the y k and present them as outputs.
So this little two state code, here's the state.
The state is also the output.
The state is what's in here.
And this is what it's trellis looks like.
That's the way this looks.
Of course, again, we need an equal number of sockets on this side and on this side.
So how would you decode this code now?
What do we see initially?
We initially get received data, measured intrinsic information, for each of the bits that we actually send over the channel.
And what's our next step?
Trellis?
Yeah.
We do the sum-product algorithm on this trellis.
So we just do BCJR ignoring the fact that it's rate one.
That doesn't matter.
And so we will get some information for each of these guys that we compute from the forward backward algorithm.
We'll ultimately get messages going in this direction.
Which come over here when we get them all.
And again, from these two messages, we can compute an output message here, from these two, compute one there, from these two, compute one there.
So we combine and re-propagate all these messages.
And that gives us new information over here.
And now, based on that, we can do BCJR again.
So again, it's this left right sort of alternation, one side, other side which, eventually, spreads out the data through the whole block.
We can use all the received data in a block to decode all the data in the block.
Now does this work as well as either of the two previous ones I put up, low-density parity-check or turbo?
Not quite, but it works amazingly well.
This is the moral here.
That even this incredibly simple code, where nothing is going on-- Again, all we have is binary state variables.
We have equals and 0 sum nodes, that's it.
It's made out of the simplest possible elements.
Even this code can get within a dB or two of channel capacity.
And I'm old enough to know that in 1993, we would have thought that was a miracle.
So if this had been the first code proposed at the ICC in Geneva in 1993, and people had said, we can get a dB and a half from capacity, everyone would have said, totally unbelievable until they went home and simulated it.
So I put this up just to illustrate the power of these ideas, and where they come from.
Of course, this idea of using the permutation as your pseudo-random element so that you, from very simple components, you get the effect of a large pseudo-random code, could be a block length, a 1,000, 10,000, what have you, this is absolutely key.
And this, basically, is what Gallager had when he said, if I just use my parity-check matrix at random, even if it is sparse, it's going to give me a random-like, which means a pretty good, code, because that's the intuition I get from Shannon and Elias.
That's the important part of the code construction, but then the decoding part of the power is the sum-product algorithm used in an iterative way.
The idea that you can simply let this algorithm run, and, if the code is properly designed, it will eventually integrate all the information over the block and converge to some fixed point.
Although, even for such a simple code as this, for this code McEliece and his students have a few theorems and can say something about the convergence behavior, and what it's actually going to do.
Even for this code, they can't get anything very definitive, I would say.
So to me, the more important thing about-- these are called RA codes-- is that incredibly simple codes can do better than concatenation of a 1,000 byte Reed-Solomon code over g f of two to the 10 with a two to the 14 state Viterbi decoder.
That doesn't do as well as this crummy little thing does.
So that's good to know.
Any questions?
Well, it's a good stopping place.
So next time, now, if you kind of have an impressionistic idea of how we do these codes.
Next time, I'm going to analyze low-density parity-check codes over the binary erasure channel, where you can do exact analysis, at least in the asymptotic case.
And that will show very clearly how it is that these codes actually work.
It's a good model for all the codes in general.
I guess that means that you can't do a lot of the homework again.
So for Wednesday, what should we do?
[INAUDIBLE] on Friday then?
Friday or Monday?
Monday seems to be popular.
So for Wednesday, just do-- There's one problem left over from problem set eight.
And that's kind of a grungy problem but worthwhile doing.
Try to do the sum-product algorithm at least once.
So just do that for Wednesday, and then for next Monday, a week from today, problem set nine other than that.
I'll see you Wednesday.
So the handouts will be just the problem set 8 solutions, of which you already have the first two.
Remind you problem set 9 is due on Friday, but we will accept it on Monday if that's when you want to hand it in to Ashish.
And problem set 10 I will hand out next week, but you won't be responsible for it.
You could try it if you're so moved.
OK. We're in the middle of chapter 13.
We've been talking about capacity approaching codes.
We've talked about a number of classes of them, low density parity check, turbo, repeat accumulate, and I've given you a general idea of how the sum product decoding algorithm is applied to decode these codes.
These are all defined on graphs with cycles, in the middle of which is a large pseudo random interleaver.
The sum product algorithm is therefore done iteratively.
In general, the initial observed information comes in on one side, the left side or the right side, and the iterative schedule amounts to doing first the left side, then the right side, then the left side, then the right side, until you converge, you hope.
That was the original turbo idea, and that continues to be the right way to do it.
OK. Today, we're actually going to try to do some analysis.
To do the analysis, we're going to focus on low density party check codes, which are certainly far easier than turbo codes to analyze, because they have such simple elements.
I guess the repeat accumulate codes are equally easy to analyze, but maybe not as good in performance.
Maybe they're as good, I don't know.
No one has driven that as far as low density parity check codes.
Also, we're going to take a very simple channel.
It's actually the channel for which most of the analysis has been done, which is the Binary Erasure Channel, where everything reduces to a one-dimensional problem, and therefore we can do things quite precisely.
But this will allow me to introduce density evolution, which is the generalization of this for more general channels like the binary input added white Gaussian noise channel, if I manage to go fast enough.
I apologize today, I do feel in a hurry.
Nonetheless, please ask questions whenever you want to slow me down or just get some more understanding.
So, the Binary Erasure Channel is one of the elementary channels, if you've ever taken information theory, that you look at.
It has two inputs and three outputs.
The two inputs are 0 and 1, the outputs are 0 and 1 or an erasure, an ambiguous output.
If you send a 0, you can either get the 0 correctly, or you could get an erasure.
Might be a deletion, you just don't get anything.
Similarly, if you send a 1, you either get a 1 or an erasure.
There's no possibility of getting something incorrectly.
That's the key thing about this channel.
The probability of an erasure is p, regardless of whether you send 0 or 1.
So there's a single parameter that governs this channel.
Now admittedly, this is not a very realistic channel.
It's a toy channel in the binary case.
However, actually some of the impetus for this development was the people who were considering packet transmission on the internet.
And in the case of packet transmission on the internet, of course, you have a long packet, a very non-binary symbol if you like, but if you consider these to be packets, then on the internet, you either receive the packet correctly or you fail to receive it.
You don't receive it at all, and you know it, because there is an internal party check in each packet.
So the q-ary erasure channel is in fact a realistic model, and in fact, there's a company, Digital Fountain, that has been founded and is still going strong as far as I know, which is specifically devoted to solutions for this q-ary erasure channel for particular kinds of scenarios on the internet where you want to do forward error correction rather than repeat transmission.
And a lot of the early work on the analysis of these guys-- Luby, Shokrollahi, other people-- they were some of the people who focused on low density party check codes immediately, following work of Spielman and Sipser here at MIT, and said, OK, suppose we try this on our q-ary erasure channel.
And they were able to get very close to the capacity of the q-ary erasure channel, which is also 1 minus p. This is the information theoretic capacity of the binary channel.
It's kind of obvious that it should be 1 minus p, because on the average, you get 1 minus p good bits out for every bit that you send in.
So the maximum rate you could expect to send over this channel is 1 minus p. OK. Let's first think about maximum likelihood decoding on this channel.
Suppose we take a word from a code, from a binary code, and send it through this channel, and we get some erasure pattern at the output.
So we have a subset of the bits that are good, and a subset that are erased at the output.
Now what does maximum likelihood decoding amount to on this channel?
Well, the code word that we sent is going to match up with all the good bits received, right?
So we know that there's going to be at least one word in the code that agrees with the received sequence in the good places.
If that's the only word in the code that agrees with the received word in those places, then we can declare it the winner, right?
And maximum likelihood decoding succeeds.
We know what the channel is, so we know that all the good bits have to match up with the code word.
But suppose there are 2 words in the code that match up in all the good places?
There's no way to decide between them, right?
So basically, that's what maximum likelihood decoding amounts to.
You simply check how many code words match the received good bits.
If there's only one, you decode.
If there's more than one, you could flip a coin, but we'll consider that to be a decoding failure.
You just don't know, so you throw up your hands, you have a detected decoding failure.
So in the case of a linear code, what are we doing here?
In the case of a linear code, consider the parity check equations.
We basically have n minus k parity check equations, and we're trying to find how many code sequences that solve those parity check equations.
So we have n minus k equations, n unknowns, and we're basically just trying to solve linear equations.
So that would be one decoding method for maximum likelihood decoding.
Solve the equations.
If you get a unique solution, you're finished.
If you get a space of solutions, so dimension one or more, you lose.
OK?
So we know lots of ways of solving linear equations like Gaussian elimination, back propagation/back substitution (I'm not sure exactly what it's called).
That's actually what we will be doing with low density parity check codes.
And so, decoding for the binary ratio channel you can think of as just trying to solve linear equations.
If you get a unique solution, you win, otherwise, you fail.
Another way of looking at it in a linear code is what do the good bits have to form?
The erased bits have to be a function of the good bits, all right?
In linear code, that's just a function of where the good bits are.
We've run into this concept before.
We called it an information set.
An information set is a subset of the coordinates that basically determines the rest the coordinates.
If you know the bits in that subset, then you know the code word.
You can fill out the rest of the code word through some linear equation.
So basically, we're going to succeed if the good bits cover an information set, and we're going to fail otherwise.
So how many bits do we need to cover an information set?
We're certainly going to need at least k. Now today, we're going to be considering very long codes.
So suppose I have a long (n,k) code.
I have an (n,k) code, and I transmit it over this channel.
About how many bits are going to be erased?
About pn bits are going to be erased, or (1 minus p) times n. We're going to get approximately-- law of large numbers-- (1 minus p) times n unerased bits, and this has to be clearly greater than k. OK, so with very high probability, if we get more than that, we'll be able to solve the equations, find a unique code word.
If we get fewer than that, there's no possible way we could solve the equations.
We don't have enough left.
So what does this say?
This says that k over n, which is the code rate, has to be less than 1 minus p in order for this maximum likelihood decoding to work with a linear code over the binary erasure channel, and that is consistent with this.
If the rate is less than 1 minus p, then with very high probability you're going to be successful.
If it's greater than 1 minus p, no chance, as n becomes 1.
OK?
You with me?
[UNINTELLIGIBLE]?
Well, in general, they're not, and the first exercise on the homework says take the 844 code.
There's certain places where if you erase 4 bits, you lose, and there are other places where if you erase 4 bits, you win.
And that exercise also points out that the low density parity check decoding that we're going to do, the graphical decoding, may fail in a case where maximum likelihood decoding might work.
But maximum likelihood decoding is certainly the best we can do, so it's clear.
You can't signal at a rate greater than 1 minus p. You just don't get more than 1 minus p bits of information, or n times (1 minus p) bits of information in a code word of length n, so you can't possibly communicate more than n times (1 minus p) bits in a block.
OK.
So what are we going to try to do to signal over this channel?
We're going to try using a low density parity check code.
Actually, I guess I did want this first.
Let me talk about both of these back and forth.
Sorry, Mr. TV guy.
So we're going to use a low density parity check code, and initially, we're going to assume a regular code with, say, left degree is 3 over here, right degree is 6.
And we're going to try to decode by using the iterative sum product algorithm with a left right schedule.
OK.
I can work either here or up here.
What are the rules for sum product update on a binary erasure channel?
Let's just start out, and walk through it a little bit, and then step back and develop some more general rules.
What is the message coming in here that we receive from the channel?
We're going to convert it into an APP vector.
What could the APP vector be?
It's either, say, 0, 1 or 1, 0, if the bit is unerased.
So this would be-- if we get this, we know a posteriori probability of a 0 is a 1, and of a 1 is a 0.
No question, we have certainty.
Similarly down here, it's a 1.
And in here, it's 1/2, 1/2.
Complete uncertainty.
No information.
So, we can get-- those are our three possibilities off the channel.
(0,1), (1,0), (1/2, 1/2).
Now, if we get a certain bit coming in, what are the messages going out on each of these lines here?
We actually only need to know this one.
Initially, everything inside here is complete ignorance.
Half, 1/2 everywhere.
You can consider everything to be erased.
All right.
Well, if we got a known bit coming in, a 0 or a 1, the repetition node simply says propagate that through all here.
So if you worked out the sum product update rule for here, it would basically say, in this message, if any of these lines is known, then this line is known and we have a certain bit going out.
All right?
So if 0, 1 comes in, we'll get 0, 1 out.
It's certainly a 1.
Only in the case where all of these other lines are erased-- all these other incoming messages are erasures-- then we don't know anything, then the output has to be an erasure.
All right?
So that's the sum product update rule at a equals node.
All right?
If any of these d minus 1 incoming messages is known, then the output is known.
If they're all unknown, then the output is unknown.
You're going to find, in general, these are the only kinds of messages we're ever going to have to deal with.
Either, we're basically going to take known bits and propagate them through the graph-- so initially, everything is erased, and after awhile, we start learning things.
More and more things become known, and we succeed if everything becomes known inside the graph.
All right?
So it's just the propagation of unerased variables through this graph
[UNINTELLIGIBLE]
No.
So they're not only known, but they're correct.
And like everything else, you can prove that by induction.
The bits that we receive from the channel certainly have to be consistent with the correct code word.
All these internal constraints are the constraints of the code, so we can never generate an incorrect message.
That's basically the hand waving proof of that.
OK.
So we're going to propagate either known bits or erasures in the first iteration.
And what's the fraction of these lines that's going to be erased in a very long code?
[UNINTELLIGIBLE]
Its going to be p. All right?
So initially, we have fraction p erased and fraction 1 minus p which are good.
OK. And then, this, we'll take this to be a perfectly random interleaver.
So perfectly randomly, this comes out there.
OK?
All right, so now we have various messages coming in over here.
Some are erased, some are known and correct, and that's the only things they can be.
All right, what can we do on the right side now?
On the right side, we have to execute the sum product algorithm for a zero sum node of this type.
What is the rule here?
Clearly, if we get good data on all these input bits, we know what the output bit is.
So if we get five good ones over here, we can tell what the sixth one has to be.
However, if any of these is erased, then what's the probability this is a 0 or a 1?
It's 1/2, 1/2.
So any erasure here means we get no information out of this node.
We get an erasure coming out.
All right, so we come in here, and if p is some large number, the rate of this code is 1/2.
So I'm going to do a simulation for like p equals a little less-- small enough so that this code could succeed, 0.4-- so the capacity is 0.6 bits per bit.
But if this is 0.4, what's the probability that any 5 of these are all going to be unerased?
It's pretty small.
So you won't be surprised to learn that the probability of an erasure of coming back-- call that q-- equals 0.9 or more, greater than 0.9.
But it's not 1.
So for some small fraction of these over here, we're going to get some information, some additional information, that we didn't have before.
And this is going to propagate randomly back, and it may allow us to now know some of these bits that were initially erased on the channel.
So that's the idea.
So to understand the performance of this, we simply track-- let me call this, in general, the erasure probability going from left to right, and this, in general, we'll call the erasure probability going from right to left.
And we can actually compute what these probabilities are for each iteration under the assumption that the code is very long and random so that every time we make a computation, we're dealing with completely fresh and independent information.
And that's what we're going to do.
Yes?
[UNINTELLIGIBLE]
When they come from the right side, they're either erased or they're consistent.
I argued before, waving my hands, that these messages could never be incorrect.
So if you get 2 known messages, they can't conflict with each other.
Is that your concern?
Yeah.
Because you're randomly connecting [UNINTELLIGIBLE], so it might be that one of the plus signs gave you an [UNINTELLIGIBLE], whereas another plus sign gave you a proper message.
And they both run back to the same equation
Well, OK.
So this is pseudo random, but is chosen for once and for all.
It determines the code.
I don't re-choose it every time, but when I analyze it, I'll assume that it's random enough so that the bits that enter into any one calculation are bits that I've never seen before, and therefore can be taken to be entirely random.
But of course, in actual practice, you've got a fixed interleaver here, and you have to, in order to decode the code.
But the other concern here is if we actually had the possibility of errors, the pure binary erasure channel never allows errors.
If this actually allowed a 0 to go to a 1 or a 1 to go to 0, then we'd have an altogether different situation over here, and we'd have to simply honestly compute the sum product algorithm and what is the APP if we have some probability of error.
And they could conflict, and we'd have to weigh the evidence, and take the dominating evidence, or mix it all up into the single parameter that we call the APP.
All right.
So let me now do a little analysis.
Actually, I've done this a couple places.
Suppose the probability of erasure here-- this is the q right to left parameter.
Suppose the probability of q right to left is 0.9, or whatever, and this is the original received message from the channel, which had an erasure probability of p. What's the q left to right?
What's the erasure probability for the outgoing message?
Well, the outgoing message is erased only if all of these incoming messages are erased.
All right, so this is simply p times q right to left, d minus 1.
OK?
[UNINTELLIGIBLE]
Assuming it's a long random code, so everything here is independent.
I'll say something else about this in just a second.
But let's naively make that assumption right now, and then see how best we can justify it.
What's the rule over here?
Here, we're over on the right side if we want to compute the right to left.
If these are all erased with probability q left to right, what is the probability that this one going out is erased?
Well, it's easier to compute here the probability of not being erased.
This is not erased only if all of these are not erased.
So we get q right to left.
One minus q right to left is equal to 1 minus q left to right, to the d minus 1.
And let's see, this is d right, and this is d left.
I'm doing it for the specific context.
OK, so under the independence assumption, we can compute exactly what these evolving erasure probabilities are as we go through this left right iteration.
This is what's so neat about this whole thing.
Now, here's the best argument for why these are all independent.
Let's look at the messages that enter into, say, a particular-- this is computing q left to right down here.
All right, we've got something coming in, one bit here.
We've got more bits coming in up here, and here, which originally came from bits coming in up here.
We have a tree of computation.
If we went back through this pseudo random but fixed interleaver, we could actually draw this tree for every instance of every computation, and this would be q left to right at the nth iteration, this is-- I'm sorry.
Yeah, this is q left to right at the nth iteration, this is q right to left at the n minus first iteration, this is q left to right at the n minus first iteration, and so forth.
Now, the argument is that if I go back-- let's fix the number of iterations I go back here-- m, let's say, and I want to do an analysis of the first m iterations.
I claim that as this code becomes long, n goes to infinity with fixed d lambda, d rho, that the probability you're ever going to run into repeated bit or message up here goes to 0.
All right?
So I fix the number of iterations I'm going to look at.
I let the length of the code go to infinity.
I let everything be chosen pseudo randomly over here.
Then the probability of seeing the same message or bit twice in this tree goes to 0.
And therefore, in that limit, the independence assumption become valid.
That is basically the argument, all right?
So I can analyze any fixed number of iterations in this way
[UNINTELLIGIBLE]
OK, yes.
Good.
So this is saying the girth is probabilistically-- so limit in probability going to infinity, or it's also referred to as the locally tree-like assumption.
OK, graph in the neighborhood of any node-- this is kind of a map of the neighborhood back for a distance of m-- we're not ever going to run into any cycles.
Good, thank you.
OK, so under that assumption, now we can do an exact analysis.
This is what's amazing.
And how do we do it?
Here's a good way of doing it.
We just draw the curves of these 2 equations, and we go back and forth between them.
And this was actually a technique invented earlier for turbo codes, but it works very nicely for low density parity check code analysis.
It's called the exit chart.
I've drawn it in a somewhat peculiar way, but it's so that it will look like the exit charts you might see an in the literature.
So I'm just drawing q right to left on this axis, and q left to right on this axis.
I want to sort of start in the lower left and work my way up to the upper right, which is the way exit charts always work.
So to do that, I basically invert the axis and take it from 1 down to 0.
Initially, both of these-- the probability is one that everything is erased internally on every edge, and if things work out, we'll get up to the point where nothing is erased with high probability.
OK, these are our 2 equations just copied from over there for the specific case of left degree equals 3 and right degree equals 6.
And so I just plot the curves of these 2 equations.
This is done in the notes, and the important thing is that the curves don't cross, for a value of p equal 0.4.
One of these curves depends on p, the other one doesn't.
So this is just a simple little quadratic curve here, and this is a fifth order curve, and they look something like this.
What does this mean?
Initially, the q right to left is 1.
If I go through one iteration, using the fact that I get this external information-- extrinsic information-- then q left to right becomes 0.4, so we do to the outer.
Now, I have q left to right propagating to the right side, and at this point, I get something like 0.922, I think is the first one.
So the q right to left has gone from 1 down to 0.9 something.
OK, but that's better.
Now, with that value of q, of course I get a much more favorable situation on the left.
I go over to the left side, and now I get some p equal to-- this is all done in the notes-- 0.34.
So I've reduced my erasure probability going from left to right, which in turn, helps me out as I go over here, 0.875, and so forth.
Are you with me?
Does everyone see what I'm doing?
Any questions?
Again, I'm claiming this is an exact calculation-- or I would call it a simulation-- of what the algorithm does in each iteration.
First iteration, first full, left, right, right left, you get to here.
Second one, you get to here, and so forth.
And I claim as n goes to infinity, and everything is random, this is the way the erasure probabilities will evolve.
And it's clear visually that if the curves don't cross, we get to the upper right corner, which means decoding succeeds.
There are no erasures anywhere at the end of the day.
And furthermore, you go and you take a very long code, like 10 to the seventh bits, and you simulate it on this channel, and it will behave exactly like this.
OK, so this is really a good piece of analysis.
So this reduces it to very simple terms.
We have 2 equations, and of course they meet here at the (0,0) point.
Substitute 0 in here, you get 0 there.
Substance 0 here, you get 0 there.
But if they don't meet anywhere else, if there's no fixed point to this iterative convergence, then decoding is going to succeed.
So this is the whole question: can we design 2 curves that don't cross?
OK.
So what do we expect now to happen?
Suppose we increase p. Suppose we increase p to 0.45, which is another case that's considered in the notes, what's going to happen?
This curve is just a simple quadratic, it's going to be dragged down a little bit.
We're going to get some different curve, which is just this curve scaled by 0.45 over 0.4.
It's going to start here, and it's going to be this scaled curve.
And unfortunately, those 2 curves cross.
So that's the way it's going to look, and now, again, we can simulate iterative decoding for this case.
Again, initially, we'll start out.
We'll go from 1, 0.45 will be our right going erasure probability.
We'll go over here, make some progress, but what's going to happen?
We're going to get stuck right there.
So we find the fixed point.
In fact, this simulation is a very efficient way of calculating what the fixed point of these 2 curves are.
Probably some of you are analytical whizzes and can do it analytically, but it's not that easy for a quintic equation.
In any case, as far as decoding is concerned-- all right, this code doesn't work on an erasure channel which has an erasure probability of 0.45.
It does work on one that has an erasure probability of 0.4.
That should suggest to you-- yeah?
[UNINTELLIGIBLE]
Yes, so this code doesn't get to capacity.
Too bad.
So I'm not claiming that a regular d left equals 3, d right equals 6 LDPC code can achieve capacity.
There's some threshold for p, below which it'll work, and above which it won't work.
That threshold is somewhere between 0.4 and 0.45.
In fact, it's 0.429 something or other.
So this design approach will succeed it's near capacity, but I certainly don't claim this is a capacity approaching code.
I might mention now something called the area theorem, because it's easy to do now and it will be harder to do later.
What is this area here?
I'm saying the area above this curve here.
Well, you can do that simply by integrating this.
It's integral of p times q-squared dq from 0 to 1, and it turns out to be p over 3.
Believe me?
Which happens to be p over the left degree.
Not fortuitously, because this is the left degree minus 1.
So you're always going to get p over the left degree.
And what's the area under here?
Well, I can compute-- basically change variables to 1 minus q, q prime, and 1 minus q is q prime over here, and so I'll get the same kind of calculation, 0 to 1, this time q to the fifth over pq, which is 1/6, which not fortuitously is 1 over d right.
So the area here is p over 3, and the area here is-- under this side of the curve is-- that must be 5/6.
Sorry, so the area under this side is 1/6 so it's 1 minus this.
It's clearly the big part, so this is 5/6.
All right.
I've claimed my criterion for successful decoding is that these curves not cross.
All right, so for successful decoding, clearly the sum of these 2 areas has to be less than 1, right?
So successful decoding: a necessary condition is that p over d_lambda -- let me just extend this to any regular code-- plus 1 minus 1 over d_rho has to be less than 1.
OK, what does this sum out to?
This says that p has to be less than d_lambda over d_rho, which happens to 1 minus r, right?
Or equivalently, r less than 1 minus p, which is capacity.
So what did I just prove very quickly?
I proved that for a regular low density parity check code, just considering the areas under these 2 curves and the requirement that the 2 curves must not cross, I find that regular codes can't possibly work for a rate any greater than 1 minus p, which is capacity.
In fact, the rate has to be less than 1 minus p, strictly less, in order for there to-- unless we were lucky enough just to get 2 curves that were right on top of each other.
I don't know whether that would work or not.
I guess it doesn't work.
But we'd need them to be just a scooch apart.
OK, so I can make an inequality sign here.
OK, well that's rather gratifying.
What do we do to improve the situation?
OK, one-- it's probably the first thing you would think of investigating maybe at this point, why don't we look at an irregular LDPC code?
And I'm going to characterize such a code by-- there's going to be some distribution on the left side, which I might write by lambda_d.
This is going to be the fraction of left nodes of degree d. All right, I'll simply let that be some distribution.
Some might have degree 2, some might have degree 3.
Some might have degree 500.
And similarly, rho_d is the fraction of right nodes, et cetera.
And there's some average degree here, and some average degree here.
So this is the average degree, or the typical degree.
This is average left degree, this is average right degree.
If I do that, then the calculations are done in the notes.
I won't take the time to do them here, but basically you find the rate of the code is 1 minus the average left degree over the average right degree.
OK, so it reduces to the previous case and the regular case.
Regular case, this is 1 for one particular degree and 0 for everything else.
It works out.
If I do that and go through exactly the same analysis with my computation tree, now I simply have a distribution of degrees at each level of the computation tree, and you will not be surprised to hear what I get out as my left to right equations, is I get out some average of this.
In fact, what I get out now is that q left to right is the sum over d of-- this is going to be lambda_d times p times q right to left to the d minus 1.
Which again reduces to the previous thing, if only one of these is 1 and the rest are 0.
So I just get the-- this is just an expectation.
This is the fraction of erasures.
I just count the number of times I go through a node of degree d, and for that fraction of time, I'm going to get this relationship, and so I just average over them.
That's very quick.
Look at the notes for a detailed derivation, but I hope it's intuitively plausible.
And similarly, 1 minus q right to the left is the sum over d of rho_d, 1 minus q left to right to the d minus 1.
OK, this is elegantly done if we define generating functions.
We do that over here.
I've lost it now so I'll do it over here.
So what you'll see in the literature is generating functions to find is lambda_x equals sum over d of lambda_d x to the d minus 1.
And rho of x equals sum over d, rho_d, x to the d minus 1.
And then these equations are simply written as-- this is p times lambda of q right to left, and this is equal to rho of 1 minus q left to right.
OK, so we get nice, elegant generating function representations.
But from the point of view of the curves, we're basically just going to average these curves.
So we now replace these equations up here by the average equations.
This becomes p times lambda of q right to left, and this becomes rho of 1 minus q left to right.
OK, but again, I'm going to reduce all of this 2 curves, which again I can use for a simulation.
And now I have lots of degrees of freedom.
I could change all these lambdas and all these rhos, and I can explore the space, and that's what's Sae-Young Chung did in his thesis, not so much for this channel.
He did do it for this channel, but also for additive white Gaussian noise channels.
And so the idea is you try to make these 2 curves just as close together as you can.
Something like that.
Or, of course, you can do other tricks.
You can have some of these-- you can have some bits over here that go to the outside world.
You can suppress some of these bits here.
You can play around with the graph.
No limit on invention.
But you don't really have to do any of that.
So it becomes a curve fitting exercise, and you can imagine doing this in your thesis, except you were not born soon enough.
The interesting point here is that this now becomes-- the area becomes p over d_lambda-bar, again, proof in the notes.
This becomes 1 minus 1 over d_rho-bar.
And so again, the area theorem-- in order for these curves not to cross, we've got to have p over d_lambda-bar plus 1 minus 1 over d_rho-bar, less than the area of the whole exit chart, which is 1.
We again find that-- let me put it this way, 1 minus d_lambda-bar over d_rho-bar is less than 1 minus p, which is equivalent to the rate must be less than the capacity of the channel.
So this is a very nice, elegant result.
The area theorem says that no matter how you play with these degree distributions in an irregular low-density parity check code, you of course can never get above capacity.
But, it certainly suggests that you might be able to play around with these curves such that they get as close as you might like.
And the converse of this is that if you can make these arbitrarily close to each other, then you can achieve rates arbitrarily close to capacity.
And that, in fact, is true.
So simply by going to irregular low-density parity check codes, we can get as close as we like, arbitrarily close, to the capacity of the binary erasure channel with this kind of iterative decoding.
And you can see the kind of trade you're going to have to make.
Obviously, you're going to have more iterations as these get very close.
What is the decoding process going to look like?
It's going to look like very fine grained steps here, lots of iterations, but-- all right.
So it's a 100 iterations.
So it's 200 irritations.
These are not crazy numbers.
These are quite feasible numbers.
And so if you're willing to do a lot of computation-- which is what you expect, as you get close to capacity, right-- you can get as close to capacity as you like, at least on this channel.
OK, isn't that great?
It's an easy channel, I grant you, but everything here is pretty simple.
All these sum product updates-- for here, it's just a matter of basically snow point propagating erasures.
You just take the known variables.
You keep computing as many as you can of them.
Basically, every time an edge becomes known, you only have to visit each edge once, actually.
The first time it becomes known is the only time you have to visit it.
After that, you can just leave it fixed.
All right, so if this has a linear number of edges, as it does, by construction, for either the regular or irregular case, the complexity is now going to be linear, right?
We only have to visit each edge once.
There are only a number of edges proportional to n. So the complexity of this whole decoding algorithm-- all you do is, you fix as many edges as you can, then you go over here and you try to fix as many more edges as you can.
You come back here, try to fix as many more as you can.
It will behave exactly as this simulation shows it will behave, and after going back and forth maybe 100 times-- in more reasonable cases, it's only 10 or 20 times, it's a very finite number of times-- you'll be done.
Another qualitative aspect of this that you already see in the regular code case-- in fact, you see it very nicely there-- is that typically, very typically, you have an initial period here where you make a rapid progress because the curves are pretty far apart, then you have some narrow little tunnel that you have to get through, and then the curves widen up again.
I've exaggerated it here.
So OK, you're making great progress, you're filling in, lots of edges become known, and then for a while it seems like you're making no progress at all, making very tiny progress on each iteration.
But then, you get through this tunnel, and boom!
Things go very fast.
And for this code, it has a zero-- the regular code has a zero slope at this point, whereas this has a non-zero slope.
So these things will go boom, boom, boom, boom, boom as you go in there.
So these guys at Digital Fountain, they called their second class of codes, [UNINTELLIGIBLE], tornado codes, because they had this effect.
You have to struggle and struggle, but then when you finally get it, there's a tornado, a blizzard, of known edges, and all of a sudden, all the edges become known.
Oh by the way, this could be done for packets.
There's nothing-- you know, this is a repetition for a packet, and this is a bit-wise parity check for a packet.
So the same diagram works perfectly well for packet transmission.
That's the way they use it.
Yeah?
[UNINTELLIGIBLE]
Yeah.
Right.
So this chart makes it very clear.
If you're going to get this tornado effect, it's because you have some gap in here.
The bigger the gap, the further away you are from capacity, quite quantitatively.
So I just-- this is the first year I've been able to get this far in the course, and I think this is very much worth presenting because-- look at what's happened here.
At least for one channel, after 50 years of work in trying to get to Shannon's channel capacity, around 1995 or so, people finally figured out a way of constructing a code and a decoding algorithm that in fact has linear complexity, and can get as close to channel capacity as you like in a very feasible way, at least for this channel.
So that's really where we want to end the story in this class, because the whole class has been about getting to channel capacity.
Well, what about other channels?
What about channels with errors here?
So let's go to the symmetric input binary channel, which I-- symmetric, sorry-- symmetric binary input channel.
This is not standardized.
The problem is, what you really want to say is the binary symmetric channel, except that term is already taken, so you've got to say something else.
I say symmetric binary input channel.
You'll see other things in the literature.
This channel has 2 inputs: 0 and 1, and it has as many outputs as you like.
It might have an erasure output.
And the key thing about the erasure output is that the probability of getting there from either 0 or 1 is the same, call it p again.
And so the a posteriori probability, let's write the APPs by each of these.
The erasure output is always going to be a state of complete ignorance, you don't know.
So there might be one output like that, and then there will be other outputs here that occur in pairs.
And the pairs are always going to have the character that their APP is going to be 1 minus-- I've used p excessively here.
Let me take it off of here and use it here-- for a typical other pair, you're going to have 1 minus pp, or p 1 minus p. In other words, just looking at these 2 outputs, it's a binary symmetric channel.
The probability of p of cross over and 1 minus p of being correct.
And we may have pairs that are pretty unreliable where p is close to 1/2, and we may have pairs that are extremely reliable.
So this 1 minus p prime, p prime, where p prime might be very close to 0.
But the point is, the outputs always occur in these pairs.
The output space can be partitioned into pairs such that, for each pair, you have a binary symmetric channel, or you might have this singleton, which is an erasure.
And this is, of course, what we have for the binary input additive white Gaussian noise channel.
We have 2 inputs, and now we have an output which is the complete real line, which has a distribution like this.
But in this case, 0 is the erasure.
If we get a 0, then the probability of the APP message is (1/2,1/2).
And the pairs are plus or minus y.
If we get to see y, then the probability of y given 0, or given one, that's the same pair as the probability of minus y given-- this is, of course, minus 1, plus 1 for my 2 possible transmissions here.
Point is, binary input added white Gaussian noise channel is in this class.
It has a continuous output rather than a discrete output.
But there's a key symmetry property here.
Basically, if you exchange 0 for 1, nothing changes.
All right, so the symmetry between 0 and 1.
That's why it's called a symmetric channel.
That means you can easily prove that for the capacity achieving input distribution is always (1/2,1/2), for any such channel.
If you've taken information theory, you've seen this demonstrated.
And this has the important implication that you can use linear codes on any symmetric binary input channel without loss of channel capacity.
Linear codes achieve capacity.
OK, whereas, of course, if this weren't (1/2,1/2), then linear codes couldn't possibly achieve capacity.
Suppose you have such a channel.
What's the sum product updates?
The sum product updates become more complicated.
They're really not hard for the equality sign.
You remember for a repetition node, the sum product update is just the product of basically the APPs coming in or the APPs going out.
So all we've got to do is take the product.
It'll turn out the messages in this case are always of the form p, 1 minus p-- of course, because they're binary, and so has to be like this-- so we really just need a single parameter p. We multiply all the p's, and that normalize correctly, and that'll be the output.
For the update here, I'm sorry I don't have time to talk about it in class, but there's a clever little procedure which basically says take the Hadamard Transform of p, 1 minus p. The Hadamard Transform in general says, convert this to the pair of a plus b, a minus b.
So in this case, we convert it to a plus b is always 1, and a minus b is, in this case, 2p minus 1.
Works out better, turns out this is actually a likelihood ratio.
Take the Hadamard Transform, then you can use the same product update rule as you used up here.
So do the repetition node updates, which is easy-- so it says just multiply all the inputs component-wise in this vector, and then take the Hadamard Transform again to get your time domain or primal domain, rather than dual domain.
So you work in the dual domain, rather than the primal domain.
Again, I'm sorry.
You got a homework problem on it, after you've done the homework problem, you'll understand this.
And this turns out to involve hyperbolic tangents to do these.
These Hadamard Transforms turn out to be taking hyperbolic tangents, and this is called the hyperbolic tangent rule, the tanh rule.
So there's a simple way to do updates in general for any of these channels.
Now, you can do the same kind of analysis, but what's different?
For the erasure channel, we only had 2 types of messages, known or erased, and all we really had to do is keep track of what's the probability of the erasure type of message, or 1 minus this probability, it doesn't matter.
So that's why I said it was one-dimensional.
For the symmetric binary input channel, in general, you can have any APP vector here.
This is a single parameter vector.
It's parameterized by p, or by the likelihood ratio, or by the log likelihood ratio.
There are various ways to parameterize it.
But in any case, a single number tells you what the APP message is.
And so at this point-- or I guess, better looking at it in the competition tree-- at each point, instead of having a single number, we have a probability distribution on p. So we get some probability distribution on p, pp of p, that characterizes where you are at this time.
Coming off the channel, initially, the probability distribution on p might be equal to y, I think it is, actually, or p to the minus y, and you get some probability distribution on what p is.
By the way, again because of symmetry, you can always assume that the all-zero vector was sent in your code.
It doesn't matter which of your code words is sent, since everything is symmetrical.
So you can do all your analysis assuming the all-zero code word was sent.
This simplifies things a lot, too.
p then becomes the probability which-- well, I guess I've got it backwards.
Should be 1 minus pp, because p then becomes the probability.
If the assumed probability of the input is a 1, in other words, the probability that your current guess would be wrong-- I'm not saying that well.
Anyway, you get some distribution of p. Let me just draw it like that.
So here's pp of p. There's probability distribution.
And again, we'll draw it going from 1 to 0.
So that doesn't go out here.
OK, with more effort, you can again see what the effect of the update rule is going to be.
For each iteration, you have a certain input distribution on all these lines.
Again, under the independence assumption, you get independently-- you get a distribution for the APP parameter p on each of these lines.
That leads-- you can then calculate what the distribution-- or simulate what it is on the output line, just by seeing what's the effect of applying the sum product rule.
It's a much more elaborate calculation, but you can do it, or you can do it up to some degree of precision.
This you can't do exactly, but you can do it to fourteen bits of precision if you like.
And so again, you can work through something that amounts to plotting the progress of the iteration through here, up to any degree of precision you want.
So again, you can determine whether it succeeds or fails, again, for regular or irregular low-density parity check codes.
In general, it's better to make it irregular.
You could make it as irregular as you like.
And so you can see that this could involve a lot of computer time to optimize everything, but at the end of the day, it's basically a similar kind of hill climbing, curve fitting exercise, where ultimately on any of these binary input symmetric channels, you can get as close as you want to capacity.
In the very first lecture, I showed you what Sae-Young Chung achieved in his thesis.
He got the binary input additive white Gaussian noise channel.
He got under the assumption of asymptotically long random codes.
He got within 0.0045 dB of channel capacity.
And then for a more reasonable number, like a block length of 10 to the seventh, he got within 0.040 dB of channel capacity.
Now, that's still a longer code than anybody's going to use.
It's a little bit of a stunt, but I think his work convinced everybody that we finally had gotten to channel capacity.
OK, the Eta Kappa Nu person is here.
Please help her out, and we'll see you Monday.
Today we finish up chapter 13.
I think we are finished, but I'll take questions.
We get into chapter 14, which starts to get into coding for bandwidth-limited channels.
Here we're coding directly in terms of Euclidean space rather than in terms of Hamming space.
And there's this wonderful quote from Neil Sloane.
"Euclidean space coding is to Hamming space coding as classical music is to rock'n'roll."
Meaning that, in the Euclidean space, things are continuous, whereas we were discrete back in Hamming space.
Nonetheless, there are strong connections between the two, which in two lectures I'll basically only have a chance to hint at.
You were supposed to hand in problem set nine last Friday, or today if you haven't already.
Today, we'll hand out the solutions for nine, and then chapter 14 and its problems, which are problem set 10.
These are not due.
As always, I recommend you do them.
The solutions will be handed out Wednesday.
However, on the exam, the exam will not cover chapter 14.
So everything we do this week is gravy.
The exam is a week from tomorrow in the morning.
Ashish will administer it because I won't be here.
For this exam, it's closed book, except you're allowed to make five pages of notes, as you did three pages on the midterm.
And I do recommend you bring a calculator, because for one of the problems the calculator will be helpful.
And I was told that erasable memory should be cleared on the calculator, so point of honor if you do that.
Any questions about the exam or chapter 13?
Yes?
Are you including the midterm [UNINTELLIGIBLE]?
Yeah, oh, everything in the course up to chapter 13.
The rule is everything that we covered in class.
If we didn't cover it in class, then you're not responsible for it.
Other questions?
Any last questions on chapter 13?
This is kind of the finale for binary codes.
We found that, at least on the binary erasure channel, we had techniques that would allow us to design LDPC codes that would get us as close to capacity as we liked, or at least I made that assertion, directed you to where you could find that done in the literature.
And I further asserted that, at least for binary symmetric input channels, like the binary additive white Gaussian noise channels, you could do much the same thing using techniques like density evolution or EXIT charts.
Again, the issue is to design code, such as this iterative decoding, after many, many, many iterations, will, with probability one, converge to the correct solution (or with probably one minus epsilon).
So that's the great conclusion to the story for binary codes, channels in which we want to have binary inputs.
And way back in the beginning, we said that that was perfectly sufficient, really, for the general additive white Gaussian channel.
When we were talking about binary code rates less than 1/2, or nominal spectral efficiencies less than one bit per second per Hertz, which are equivalent statements.
So what happens if we want to go over channels where we want to send at higher spectral efficiencies?
We want to send four bits per second per Hertz, as we certainly do over, say, telephone line channels, which are 3,000, or 4,000-Hertz channels, over which we want to send tens of thousands of bits per second.
Or many radio channels will support high numbers of bits per second per Hertz, and so forth.
Well, obviously, you can't do that with binary codes.
So, today and Wednesday in this chapter, we very quickly, of course, go through the series of techniques that have been developed to go over bandwidth-limited channels such as the bandwidth-limited additive white Gaussian noise channel.
Clearly, you're going to need a non-binary constellation, and somehow code on that constellation.
But what you'll see is that both the techniques and much of the development are very reminiscent of the binary case.
Historically, the binary case was always done first, and then the generalizations to non-binary followed.
And so we'll see -- you can start off with very simple, hand-designed constellations, as we were doing back in the early chapters, four and five.
We played around with little m equals eight constellations and tried to optimize them from the point of view of, say, minimizing power for a fixed minimum distance between points in a certain number of dimensions in Euclidean space.
Now, we want to go up to considerably longer constellations.
So that was kind of at the level of Hamming codes and single parity-check codes and very elementary binary coding techniques like that.
So we want more complicated codes, let's say moderate complexity codes.
We will find things that are analogous to block codes, linear block codes, namely lattices.
These are structures in Euclidean space that have the group property, as linear block codes do.
And then we'll find trellis codes, which are the Euclidean space analog of convolutional codes.
And basically, our goal in the next two lectures is to get to the level of a union-bound estimate analysis of the performance of these kinds of coding techniques, which, like their binary analogs, sort of sit in a moderate complexity, pretty good performance, get up to 6 dB coding gain, can't get to capacity.
So that's as far as we'll be able to go in chapter 14.
In the last page or two of chapter 14, I briefly mention higher performance techniques.
And it's generally accepted that you can get as close to capacity on these bandwidth-limited channels, additive white Gaussian noise channels as you can on the binary channels, in some cases just by adapting binary techniques in a kind of force-fit way, and others that are somewhat more principled, I might say.
But in any case, by adapting the binary techniques, you can get turbo codes, low-density parity-check codes, capacity approaching codes of various types for the bandwidth-limited channel.
But I just hint how that can be done in one page, and I can't include it in the course.
So that's where we're going.
So basically, we're trying to design a constellation set of signal points.
Constellation c or signal set or alphabet, script c, if you like.
And we're doing the kind of things we were doing back in chapter four and five.
What are some of the things that the parameters we have-- we're going to do this in n-dimensional space.
We're going to be concerned with the size of the constellation, the log of the size, the number of log 2 of the size, the number of bits we can send.
We call this m at one point.
That will determine the number of bits we can send per n dimensions, or we can normalize it per two dimensions.
Typically, I said, in bandwidth-limited regime, we'll normalize everything per two dimensions.
We're going to have an average power of the constellation, which I'm just going to write like that.
We're going to have a minimum squared distance between points in the constellation, which I'm going to write like that.
And the idea is to optimize the trade-off between this and this.
Clearly, if I take a given constellation and I scale it up, I'm going to get better minimum squared distance without the cost of increasing power.
So I either fix the minimum squared distance between points and try to minimize the power, the average energy over the centroids of a bunch of spheres, or vice versa.
So when we do this, this is kind of the game, we get into a classical problem called sphere-packing.
If we say, hold the minimum squared distance constant, that means we want to have a certain minimum distance.
Here I'll make it d_min over 2.
That means that there's going to be a sphere of radius d_min over 2 around each point in this space, which must not be violated by the sphere of any other point.
So basically, we're trying to pack spheres as closely together as we can.
And we're maybe given the dimension.
Here I have two dimensions to play with.
We may be given the number, m. In general now, we're going to be thinking of m becoming very large.
We want to go to arbitrarily large number of bits per second per Hertz.
So think of schemes that will continue to work as we add more and more points.
And if I do that in two dimensions, we get penny-packing.
Take a bunch of seven pennies out of your pocket, and try to squeeze them together as tightly as you can.
That's effectively what we're trying to do.
They maintain their fixed radius, and you get something that quickly starts to look like what this is called, a hexagonal sphere-packing, because the centers line up on a hexagonal grid.
And in fact, I forget exactly what we did.
We did some of these back in the early chapters.
And let me draw a 16-point constellation, which I remember we did draw.
Here's a constellation consisting of 16 pennies.
And subject to my drawing ability, it's packed as closely as possible for a fixed minimum distance between the points.
And as far as anyone knows, this constellation, consisting a set of points on a hexagonal grid, is the most dense packing of 16 points in two dimensions, as measured by the average power of the centers.
And it's not very symmetrical, but you can see it's based on a symmetrical packing.
This hexagonal, this is basically a subset of 16 points from the hexagonal lattice.
If we extend these points in all directions-- again, I haven't done it very well-- but the hexagonal lattice is simply the underlying lattice here.
Let me just draw a set of points that look like this.
I'm going to fail again.
Sorry, I'm better at rock'n'roll than I am at classical music.
Anyway, this is the lattice, and it's called hexagonal because, if we draw the decision regions for this lattice, if we consider this as a set of points in a constellation and draw the decision regions, it looks like that, this little hexagon, defined by the six nearest neighbors.
They define six sides of a hexagon.
And so each lattice point has a little region of space that belongs to it, you can think of as its decision region.
Now, what makes this a lattice?
What's the definition of a lattice?
We'll say an n-dimensional lattice.
Very simple and elegant definition for a lattice.
It's a discrete subset.
That means a set of discrete points of Rn, sitting in Euclidean n-space, that has the group property.
And you now all remember what that means.
It means the group property, under ordinary addition of vectors in Euclidean n-space, addition or subtraction, of course.
What are some of the implications of having a group property?
That means that the sum of any two vectors is another vector in the lattice.
Does the lattice have to include the 0, the origin point?
Anybody?
Oh, good.
I'm going to ask this on the final.
Come on.
Does it or doesn't it?
It does
It does.
Why?
[INAUDIBLE]
OK, that's one good answer.
It has to be closed under addition and subtraction.
Subtraction of the point from itself is going to give you 0.
A more mathematically trained person would probably, say, well, it obviously has to have an identity.
Every group has an identity element.
And that means, if lambda is in the lattice, does minus lambda have to be in the lattice?
Yeah, every element has to have an additive inverse, and so forth.
Well, so if I have a lattice like this, that means that one of these points has to be the origin, say this one.
Now, of course, you can have a translate of a lattice, and geometrically it has all the same properties as the lattice itself, at any fixed t to all the lattice points.
In other words, make the origin somewhere else.
But that's not itself a group.
That's the co-set of a group.
It would translate as a co-set operation.
So lambda must be a group, and lambda plus t, any lambda translate is a co-set of lambda.
So most of the things you say about lambda are also true for lambda plus t, but in order to fix a group, we need the origin to be part of the points set.
OK, if it's a group, what does that mean?
That means, if we take any particular point, say this one, regard it as a vector, then call this lambda_1 -- or let me it g1, more suggestively.
Does g1 plus g1 have to be in the group?
Yes.
So that forces this point to be in there, 2g1, 3g1, so forth.
All of these have to be in there, as well as minus g1, minus 2g1, and so forth.
So basically, once we found a generator, all integers times that generator have to be in the lattice.
So that means there's a cross-section along the axis defined by g1, such that these points are simply the integers, scaled by whatever the norm of g1 is.
And of course, that holds for any of these neighbors here.
You want to look for nearest neighbors of 0's.
Take g2 and then, of course, all of its integer multiples have to be in the lattice.
So we've got a set here that, in some sense, spans n-space.
We have now two vectors, g1, g2, which are not linearly dependent.
They're sitting in 2-space.
So if we take all real linear combinations of these two vectors, we're going to get all of 2-space.
If we take all integer linear combinations of these two vectors, they all must be in the lattice, right?
By the group property?
We've already proved that all the integer multiples of either of them is in the lattice, so any thing plus or minus those two.
So certainly, the lattice includes the set of all integer multiples of a_i times gi, where i equals 1 to 2, where a1, a2 is in the two-dimensional integer, lattice Z-squared.
Could there be any other points?
There actually could, if you made a mistake and picked your initial vector to be 2g1, because then you wouldn't have taken account of g1.
So you've got to take your two initial vectors to be two of the minimum norm points in the lattice.
But if you do that, then you can easily prove that this is all of the lattice points.
So in fact, a two-dimensional lattice is equal to a set of all integer linear combinations of two generators, which we can write as a little two-by-two generator matrix, G. Or very briefly, we could write lambda as G times Z-squared, and we can view it as just some linear transformation of Z-squared.
So in this hexagonal lattice, we can view it as starting out with a square grid.
Of course, Z-squared itself is a lattice.
Z-squared is a group.
This is just the square lattice in two dimensions.
We have Z_n as a lattice in n dimensions, in general.
So I take Z-squared, Z_n, more generally, and I distort it.
I just run it through some g, which is a linear transformation, and slants some of the generator points.
And that's a general way of getting a lattice.
That's not the way we usually visualize the hexagonal lattice.
We usually visualize it in such a way as to recognize that it has six-fold symmetry.
But that's certainly a way to get it.
And this has one interesting consequence.
One of the measures we want for a lattice is the volume per lattice point.
In here, the volume, you can divide this up into volumes that are all of equal size, and we call that the volume of A2.
This lattice -- hexagonal lattice, is called A2, and it's the volume of that hexagon.
1 over the volume of that hexagon is the density of points in 2-space, if you like.
So density is 1/volume.
I mean, think of every point as having a unique volume, and if we basically consider the cells around each lattice point, they fill up 2-space, or the perfect tessellation tiling of 2-space.
These hexagons will.
One easy consequence of the group property is that all the decision regions have to be congruent, in fact, just translates of one another by lattice points.
The decision region around this point is the same as the decision region about 0, translated by a lattice point, namely the center of that decision region.
I won't prove that but it's true and easy to prove.
So that's one way of finding what's the volume.
The volume will mean the volume of 2-space per lattice point.
But what's another way to do that?
Another way is just to take this little parallelotope, whose volume is easier to calculate.
And we can see that we can equally well tile 2-space with these little parallelograms, just anchor -- in this case, this is the one that's anchored in the lower left corner by the origin.
Then we take another one that's anchored by this point.
We take another one that's anchored by this point, another one that's anchored by this point.
And it's clear that's another tiling of 2-space by regions, one for each lattice point that completely fills 2-space with overlaps only on the boundary, which don't count.
So the volume of A2 is also the volume of that, what's called a fundamental parallelotope.
And what's the volume of that fundamental parallelotope?
Well, one way of calculating it is to take the volume of Z-squared.
We can view this as the distortion of Z-squared, which looks like this.
So if this is Z-squared, what's the fundamental volume of Z-squared?
The volume of Z-squared per lattice point.
This is (0,1,-1,1, so forth,1,1).
What's the volume of Z-squared per--
1
--1.
Clearly.
And we have this decision region.
This is little square side 1, or this parallelotope is also little square side 1.
It's volume is 1.
General geometric principle, if we take this and distort it by G, then what's the volume of, say, this parallelotope going to be?
This parallelotope goes by G into this one, and its volume is the Jacobian, which is the determinant of G. So the volume of a lattice is the determinant of any generator matrix for that lattice.
Just a nice little side fact, but it's an example of the kind of things you get in Euclidean space that you don't get in Hamming space.
Of course, there are various generator matrices that you could use to generate this lattice.
But whichever one you choose, this is going to be true.
So this is an invariant among all the generating matrices.
So that's a lattice.
I haven't mentioned what's the most important thing for us about a lattice.
A lattice is a group.
What's the fundamental property of a group, which I guess was embodied in the alternate set of axioms for the group?
If I take the group plus any element of the group, what do I get?
[INAUDIBLE]
The group again.
Let's interpret that geometrically.
One of the nice things about this subject is you're constantly going back and forth between algebra and geometry.
Geometrically, what does that mean?
That means, if we take this lattice, say the origin is here, and we shift it, say, so that the origin is up here, then nothing changes.
Everything is invariant to this shift.
We can do this for any lattice point.
That shows you that all the decision regions have to be congruent.
It shows you that the number of nearest neighbors, the minimum distance from any point to its nearest neighbors is always the same for any lattice point.
Basically -- it's a lattice is geometrically uniform.
That means you can stand on any of the points and the world around you looks the same, as if you stood on any of the other points.
Think of these as stars in the universe, and somebody blindfolds you and drops you on one of these points.
Can you tell which one of those points you've been dropped on if nobody's written anything on these points?
No, you can't tell.
The world looks the same from whichever one you drop on.
So that means the minimum square distance from any point is the same as every other.
So the number of nearest neighbors at minimum square distance is the same.
It means the total distance profile, starting from any point, is the same.
It means if we were to take one of these points and send it over a Gaussian channel from a very large set, and we take one of the points from the interior and send it over an additive white Gaussian noise channel, the probability of error is going to be the same, regardless of which point we send it from, because the probability of error is the probability of falling outside your decision region.
And since the decision region always has the same shape, it's invariant to which point you send.
So this geometrical uniformity property, which that's the general name for it and it has all these specific consequences that I just listed, is the principal geometrical property of a lattice.
This just follows from its group property.
You don't have to be a lattice to have this property.
For instance, the translate of a lattice is not a group, technically, but it obviously has the same geometrical uniformity property.
And there are more elaborate examples of things that look less like lattices that are nonetheless geometrically uniform.
On the other hand, lattices definitely are geometrically uniform.
So this seems to be a good thing to do, because what's our objective?
We're trying to have d_min-squared be the same for every point in our constellation.
We want to tack these pennies as close together as we can.
That means we want to hold d_min-squared constant for every point that we're going to use.
And so lattices have that property.
So that seems like a good way to do sphere-packing.
And in fact, sphere-packing, in n-dimensions, is a very old mathematical subject.
How many spheres can you pack around a three sphere, is it 12 or 13?
That was debated in the 18th and 19th centuries, and I think it's only finally just been resolved a few years ago.
This turns out to be not an easy problem.
The densest sphere-packing in four dimensions was found in the mid-19th century.
In eight dimensions, I think around the turn of the 20th century.
In 16 dimensions, it's the Barnes-Wall lattice, which we'll see a little later, which was found in '54, about the same time as Reed-Muller codes, or maybe it was '59, maybe a little later than Reed-Muller codes.
The Leech lattice was certainly found a little earlier than that, but around the same time as the Golay code, to which it's a very close cousin.
And so forth.
So you might think that these questions had all been settled, but in fact they're still open for most specific dimensions, n. Conway and Sloane have the authoritative book on this, which basically lists what we know about the densest sphere-packings in all dimensions.
So here's our idea.
Our idea is we're going to go consult the mathematicians and find out what the densest sphere-packings that they found are in various dimensions.
In two dimensions, it's the hexagonal lattice.
In one dimension, it's the integer, and basically the integers are the only packing with regularity properties in one dimension.
And two dimensions, you could consider Z-squared, or you can consider A2.
These are the two obvious candidates, the square and the hexagonal packings.
In three dimensions, you get face-centered cubic packing.
And what's the BCC?
The base-centered cubic packing, I guess
Body-centered
Say again
Body center
Body-centered, thank you.
I'm sure you're right.
So anyway, there's some things that are known about this from classical mathematics and current mathematics.
So our idea is we're going to try to find the densest possible lattice in n dimensions.
But for digital communications, for coding, we only want a finite number of points from that constellation.
So what we're going to do is we're going to create our constellation based on a lattice and on a region of n-space.
And the idea is simple.
We're just going to intersect the lattice.
Or maybe let me allow a translate of a lattice with the region.
And make this a finite, compact region.
And this will give us a certain finite subset of the points in the lattice.
Now, when I draw a constellation like this one, of a size 16, that's what I've done.
I've basically taken A2.
Or it's actually a translate of A2.
I think the origin is in here somewhere.
And I draw a sphere around the origin.
And I take the 16 points that happen to fall within that sphere.
And I call that my constellation.
So that's the idea in general.
Think of the region, r, as a cookie cutter.
We're going to take the cookie cutter and chop out a finite number of points from this nice, regular, infinite grid.
And that will give us a constellation.
So especially for large constellations, we'll be able to make some approximations that will allow us to analyze that kind of constellation very easily.
And actually, the best constellations that we know are of this type.
We've taken this baseline, m-PAM.
Well, what's m-PAM?
We take the lattice plus t, let's say to be Z plus 1/2.
So that means we don't take the origin, but we take the points 1/2 minus 1/2, 3/2.
They're equally spaced points, but we've shifted them so as to be symmetric about the origin.
5/2 minus minus, so forth.
So this is Z plus 1/2 is our lattice.
And then if we want 4-PAM, we take a cookie cutter consisting of this region.
So this is the region going from minus 2 to 2.
It's not unreasonable to expect that, if we take a region of length 4, we're going to get 4 points, if, again, it's symmetric around the corner, because each of these points takes up about one unit of space over here.
So if we made it from minus 4 to plus 4, we'd have a region of length 8, and we'd get 8 points.
And so forth.
So that's how we do m-PAM, and then scale it if we want to.
But this is the basic idea.
Similarly, for m by m QAM, like everything else, it's just doing the same thing independently in two dimensions.
Here we take our lattice to be Z plus 1/2 squared, which is just the offset Z-squared, offset so as to be four-way, to be roughly symmetrical to four-way rotations, 90-degree rotations.
So here is Z-squared plus (1/2, 1/2).
And then, again, here to get 16-QAM, for instance, we might take the region to be what?
(-2,2)-squared which, again, has area 16.
You might think of this as just taking the little square around each of these.
Each of these squares has area 1.
So when we take the union of all those squares, we get the region.
That's another way of characterizing this region.
It's just the union of all the decision regions of the points that we want.
If you had started from the points, you could certainly invent a region that is the union of those decision regions.
It would have an area equal to the number of points times the volume of 2-space per each point.
That's the kind of approximation we're going to use.
The bounding region is clearly going to have something like the number of points in the constellation, times the fundamental volume as its volume.
So this is our idea.
This is what we're going to call a lattice constellation.
And we're going to investigate the properties of lattice constellations, not the most general thing we could think of but good enough.
Seem to have the right sort of property.
And we're going to take it at least as far as the union-bound estimate, so we can get a good estimate of probability of error if we take one of these constellations, like m-by-m QAM, and use it to transmit over a Gaussian noise channel.
And furthermore, for large constellations, meaning as we get well up into the bandwidth-limited regions of large nominal spectral densities, we're going to be able to separate the analysis into lattice properties and region properties.
And we'll basically be able to just add these up to get the overall analysis, by making some judicious approximations that become exact in the large constellation limit.
So let's now restrict ourselves to lattices, or translates of lattices, and start to analyze them.
What are their key parameters for our application, which is digital communications?
Well, one is the minimum squared distance between lattice points.
We've seen that doesn't depend on the particular point.
So this is the minimum squared distance from any lattice point.
The number of nearest neighbors.
The volume per lattice point.
And is there anything else?
It seems to me there's a fourth one.
No.
I guess that's all we need to know.
So I want to combine these into a key figure of merit for coding.
Again, we did this back in chapter four for a complete constellation.
We found the figure of merit by holding the average power fixed for a fixed number of points, m, on a fixed dimension, n, and maximizing the minimum square distance, or, conversely, holding the minimum squared distance fixed and minimizing the average power.
So we get some figure of merit that was kind of normalized, and we could just optimize that one thing.
The key figure of merit was called by 19th-century mathematician, Hermite's parameter, but we are going to call it the nominal coding gain because that's what it's going to turn out to be.
And it's just defined as follows.
The nominal coding gain of a lattice is just this minimum squared distance, so it goes up as the minimum squared distance.
But then we want to normalize this.
And the obvious thing to normalize it by, we want something that's going to scale as a two-dimensional quantity.
If the lattice was scaled by alpha, then this would go up as alpha squared.
So we want something that'll scale in the same way.
And what we choose to normalize it by is the volume of lambda per two dimensions, if you like.
So you can see immediately, this is invariant to scaling, that the nominal coding gain of lambda is the same as the nominal coding gain of alpha lambda.
Is alpha lambda a lattice, by the way?
If lambda's a group, and we have any scale factor alpha greater than 0, then alpha times lambda is also a group.
Everything's just multiplied by alpha.
So alpha lambda is a lattice.
And this is actually one of the homework problems.
If you scale everything by alpha, d_min-squared goes up by alpha squared.
The volume goes up by alpha to the n, for an n-dimensional lattice.
But when we take 2 over n, we're going to get back to alpha squared.
So alpha c of lambda is alpha c of alpha lambda.
So this is what you want.
You want a kind of scale-free version of a lattice, because, obviously, when we use this, we're going to feel free to amplify it or compress it any way we want, so we do with QAM.
All right, so this is normalized in that way.
We prove in the homework it's also invariant to other things.
If you took, say, an orthogonal transformation of this lattice, a rotation, reflection, multiply it by a matrix that doesn't change the scale, think of it geometrically as an orthogonal transformation, is that going to be a lattice?
It is going to be a lattice.
Everything's just going to have some h out here, times G times Z-squared, and that's still of the form of a lattice.
Or call it U for orthogonal, unitary.
So we multiply it by some orthogonal matrix, U.
It's still a lattice.
If we look in here, orthogonal transformations don't change squared distances.
We're not going to change this -- it doesn't matter.
Actually, I might.
No.
It's a rigid rotation, I'm sorry.
Everything here stays the same.
Hexagonal lattice still is hexagonal lattice, because it's a rigid rotation or reflection geometrically.
And the volume clearly stays the same, so none of these things changes.
And orthogonal transformation isn't going to change this quantity either.
So we can rotate this or do whatever.
And then, finally, there's an even more interesting one, which is, if we take the lattice, lambda to the m, this is simply the Cartesian product.
It's the set of all lambda_1, lambda_2, lambda_m, such that each of these lambda_k is in Lambda.
So it's the set of all n-tuples of elements of Lambda.
This clearly lives in dimension mn, so it's a much higher-dimensional lattice.
The simplest case is Z to the m - it lives in m dimensions, whereas Z is down in one dimension.
So it lives in dimension mn.
It's a way of constructing high-dimensional lattices from low-dimensional lattices.
It's something that, in communications, we consider to be almost a non-event.
Sending a sequence of elements of lambda is the same as sending a sequence of elements of lambda_m.
In fact, if we just send a sequence of things from here, and a sequence of things from lambda, there's no difference from a receiver's point of view.
So we regard these two lattices as equivalent, from a communications point of view.
Well, what happens to d_min-squared?
d_min-squared doesn't change, because we can always-- d_min-squared is, by the way, the weight of the smallest-- should have said that.
You can always do things relative to 0. d_min-squared, therefore, is the squared norm or lowest squared energy of any non-zero lattice point, same as we did with Hamming codes, just by the group property.
K_min is the number of such lowest Euclidean weight points.
OK, here, what's the lowest weight point?
It would look something like this, where this is one of the lowest weight points in lambda.
So d_min-squared doesn't change.
This changes, but we're not considering that.
The volume, what is the volume?
It turns out that it's not hard to show that V(lambda to the m) is V(lambda) to the m. And therefore, you can show that this is figure of merit doesn't change, even if you take Cartesian products.
So for instance, I chose that the nominal coding gain of to the m is equal to the nominal coding gain of Z.
Which is what, by the way?
I should have done this as the first example.
What's the minimum squared distance of Z?
1.
What's the volume of Z?
1.
So this is 1, or 0 dB.
We're going to measure nominal coding gain in dB, which, well, we're getting us into communications.
What's the nominal coding gain of the hexagonal lattice?
By asking this question, I'm basically asking, is the hexagonal lattice denser than Z-squared in two dimensions than the square lattice in two dimensions?
And if so, quantitatively, how much denser is it?
[INAUDIBLE]
OK, well, there's a root 3 in it, but that isn't exactly the answer, I don't think.
There are various ways that you can do it, but one of the ways is to take, as a generator matrix, (1, 0; 1/2, root 3 over 2).
You like that as a generator matrix for the-- That's taking this and this as the two generators, and recognizing this as 1/2, and this as root 3 over 2.
That makes the length of this equal to 1/4 plus 3/4, equals 1 square root, and it's 1.
So these are my two generators of length 1 in a hexagonal lattice of minimum squared distance 1.
So based on that, I can say that the volume of A2 is what?
[INAUDIBLE]
Root 3 over 2.
Not hard to take the determinant of that, right?
And what's the minimum squared distance in this scaling?
What's the minimum non-zero weight -- squared norm, it's 1.
So this is equal to 1 over root 3 over 2, which is equal to 2 over root 3.
Which is equal to what?
This is 3 dB.
This is 4.8 dB, roughly.
Square root is 2.4 dB.
So this is about 0.6 dB.
That's where it pays to be facile with dB's.
This area, by the way, if you tried to compute the area of this little hexagon here, what would it be?
If the area of this parallelotope is the square root of 3/2, what's the area of this hexagon?
The same, right?
They're both space filling when centered on lattice points.
So they have to have the same area.
So that's an easy way of computing the area of a hexagon, or of this particular hexagon anyway.
But you would get the same result in either way.
So comparing this with Z-squared, I now have a quantitative measure of how much denser A2 is than Z-squared.
Put in one way, if I fix the minimum squared distance of each of them to be 1, if I'm doing penny-packing and I say I want pennies of a certain radius to be packed into two-dimensional space, then this is 1 over the volume, which is basically the density.
The density of A2 is 2 over root 3, times [UNINTELLIGIBLE] as the density of points in z squared.
I get more points per unit area with the same distance between them using A2 than I can with Z-squared.
So that's the significance of this parameter.
So it's an obvious thing for Hermite to have come up with.
It's going to turn out to be equally fundamental for us.
So that may be all we want to do on lattices for the time being.
Yeah, seems to be.
So we're talking about lattice constellations.
The other thing we have to develop is the properties of regions.
So let me talk about regions, R -- I'm thinking of a compact region in an n-dimensional space.
So again, let me fix the dimension to n. And we're going to have a certain volume.
That's obviously an important characteristic of a region.
And I'm going to assume that this is finite.
And what's the other thing that we basically want?
Well, when I pick a constellation like this, by intersecting a lattice with a region, eventually I'm going to let the points in this constellation be equiprobable, as I always do in digital communications.
16 points.
Say they all have probably 1/16, the uniform probability over the discrete points.
One of the approximations I am going to make -- so what's the average power of that constellation?
We could figure it out by taking 1/16 times the energy.
I should say the average energy, I'm sorry.
We're not talking power here.
We're talking energy, but I read it as P. What's the average energy?
It's 1/16 times the L2 norm of each of these points, if you like that.
It's simply the Euclidean squared norm.
And we get some average.
But here's the key approximation principle.
I'm going to say the average power of this constellation, especially as the constellations get large, is simply going to be approximated as the average energy of a uniform distribution over this bounding region.
Suppose I simply put a uniform continuous distribution over this circle, this sphere, two sphere that I've used to be my cookie cutter.
I'm going to say that's going to be a good approximation to the average energy of the 16 discrete points, which are, by construction, they're spread uniformly over this region.
The approximation is, I'm kind of approximating a unit impulse of weight there by distributed weight across its-- well, actually across its whole little region.
If we have a hexagon around each one, uniform distribution over the hexagon.
And if that's a reasonable approximation, then it's reasonable to say that the average power of the whole constellation is just the average power of a uniform distribution over this region.
It's good exercise to try it for some moderate size cases.
And you'll find it is a very good approximation.
In any case, it's the one we're going to use.
So the other key parameter is going to be P(R), which is, in words, the average energy of uniform distribution over R. Sorry, script R. Now, when we write this out, this actually gets a little formidable.
So I think it's better to remember what we're actually trying to compute.
Oh, and one last thing, we're going to normalize it per dimension.
So what do I mean?
What's a uniform distribution over R?
This is P(x equals 1) over the volume of R for x in R and 0 otherwise.
Uniform within, 0 outside.
And what does it have to equal?
It has to be equal to 1 over V(R), because its integral has to be equal to 1 if it's a probability distribution.
So to compute P(R), we basically take the integral from over R, of this probability distribution, 1 over V(R), times what?
The energy per dimension.
The total energy of x is x-squared, the L2 norm of x. Normalize per dimension -- we're going to put an n over there -- dx.
So that's an equation for what it is I wanted to compute.
I think it's much harder to remember the terms in this than to remember this verbal description of what we want.
But you may be more literate than I.
Now, let's define -- we want to normalize quantity comparable to Hermite or coding gain over here.
The normalized quantity we're going to use -- we want to take P(R) -- and what should we normalize it by to make it scale-invariant, let's say again?
Again, if I take the region, I scale the region by alpha and get alpha_R.
What's going to happen to the average energy of a uniform distribution over R?
[INAUDIBLE]
It's going to go up as alpha squared.
Energy scarce, goes as the square of the scale factor.
So again, I want to normalize it by V(R) over 2 to the n, because this is also something that will go up as alpha squared if I scale R by alpha.
So this is what's known as the dimension-less or normalized-- it has both names-- second moment.
Another name for this is the second moment per dimension of a uniform distribution over R. This is a normalized or dimension-less second moment.
I think I call it normalized in the notes, but it's always called G in the literature.
And let me just introduce here-- well, what's the normalized second moment of an interval?
Say, a symmetric interval around the origin.
G to the minus 1 to the 1, which is like the interval that we used to pull out the m-PAM signal structure.
So let's take n to be equal to 1, R just to be the interval from -1 to +1.
And V(R) equals what?
It's the length of R, which is 2.
What's P(R)?
The integral from -1 to 1, x squared, dx, which is 2/3.
I think so
[INAUDIBLE]
Right, 1/2.
Thank you.
That's 1/3.
All right, so what is G(R) going to be, or G going to be?
This is going to be P(R) over V(R)-squared.
And so this is going to be 1/3 over 4, or 1/12.
OK, that's not as nice as this over here, but it is what it is.
There's a number that shows up in quantization, for instance, uniform scalar quantizer, you'll find a 1/12 in there.
Why?
It's because of this equation.
If we ask again about the invariances of this-- this is the last homework problem for this week-- it's invariant not only to scaling, but, again, it's invariant to orthogonal transformations, because orthogonal transformations won't affect either of these things.
A rigid rotation, about the origin, won't affect V(R) or P(R).
And furthermore, it's invariant to Cartesian products, again.
If we just have a series of regions, a region made up basically as the Cartesian product, like an n-cube is a Cartesian product.
We just pick each of the dimensions independently from some set R. Then that's not going to affect the normalized second moment either.
Exercise for the student.
So from that result, I can say that G([-1 1]) to the m. Which is what?
This is an m-cube of side 2.
If I say each of the coordinates, x1, x2, x3, up to xm, has got to be xk n times [-1 1], then what I'm going to get, the region defined by that is an m-cubed, centered on the origin of side 2.
So the normalized second moment of that is going to be 1/12.
This is where the shortcut begins to help you.
I guess I'm trying to put too much on one board.
But the last thing I want to put up on here is the shaping gain.
The shape gain of a region is defined as the shaping gain of the region is equal to 1/12.
So we're kind of taking as a baseline, the m-cube, in any number of dimensions n, over G(the region).
In other words, how much less is the dimension-less second moment, the normalized second moment, than what you would get for an m-cube?
In every dimension except for one, a sphere is going to be a better region, from a second moment point of view, than an m-cube.
An m-cube has corners.
Really, it's pretty obvious that, if you want to minimize the normalized second moment in any number of dimensions, you would choose an m-sphere, because if you have any wrinkle that comes out of an m-sphere, anything that's not an m-sphere, you should try to squash it back in it.
And that'll make it an m-sphere.
You'll take higher energy and make it lower energy for the same little unit of Play-Doh or mass stuff.
So a sphere will give you the best normalized second moment in any number of dimensions n, so you're always going to get a G(R) that's less than 1/12.
That's good.
And this measures the amount of gain.
Again, you can measure this in dB.
That's going to translate directly to dB.
I suppose I could do an example for the sphere in two dimensions, but I won't.
How are we doing on time?
Let me see if I can do the union-bound estimate.
That would be a trick, but I think it's doable, because this is where we're going.
So the union-bound estimate, which is something we did way back in chapter five or something, is basically, given a constellation, which we're going to have a lattice constellation based on some lattice lambda and some region.
So this is like our m-by-m QAM, or it might be this guy, which is better than 16-QAM, in some sense.
In two senses, actually.
The union-bound estimate, what is it?
The probability of error is approximately equal to-- let me do it per block.
So I'll have the number of nearest neighbors per block of our constellation times Q to the square root of d_min-squared of our constellation, over what is it?
4 sigma squared?
4 sigma squared.
And if you remember how we got this, we took the union-bound, we took all the pairwise error probabilities, and from that, we get contributions from kd at every kd contributions at distance d. We added them all up in the union-bound.
We said, well, for moderate complexity and performance, at least this is going to be dominated by the minimum distance.
So this is the minimum distance terms.
This is the number of minimum distance terms.
And this is error probability -- the pairwise error probability we get for each minimum distance term.
Now, to compute this, I'm going to use three approximations, but two important ones, this continuous approximation that I've already referred to.
First, I'm going to say that the average power of my constellation is approximately equal to just the second moment of R. The average energy per dimension of R. And then I'm going to say, furthermore, how many points are there in the constellation?
This will affect what rate I can go at.
The average number of points in the constellation -- also mention this-- is going to be approximately equal to simply the volume of my bounding region, V(R), over the volume taken up by each point in my lattice.
That makes sense, doesn't it?
What I want to do here to get 16 points is I take a sphere whose area is roughly 16 times the area of one of these little hexagons or one of these little parallelograms.
And I'll get about 16 points, because that's the density of it.
And there's a final little approximation, which is that I'm going to assume that we're somewhere in the interior of the lattice, so that the average number of nearest neighbors in the constellation is simply equal to the number of nearest neighbors in the lattice.
So that's a reasonable approximation, and that's a very obvious one.
If I take any constellation based on the hexagonal lattice, for a large enough constellation, the average number of nearest neighbors is going to be 6.
If I take it on the square lattice, the average number of nearest neighbors is going to be 4.
But we know we're not terribly interested in that either.
Let me do a little manipulation here.
I'm going to write this as Q of d_min-squared of c, which clearly I'm going to also assume that d_min-squared of c is equal to d_min-squared of the lattice.
So I'll make that d_min-squared of the lattice.
And I want to normalize that by v of lambda, to the 2/n, whatever dimension n I'm in, to get something that we're familiar with.
So I need a V(n) over 2 to the n. And anticipating a little, I'm going to put a V(R) over 2 to the n. And then over here, I'm going to put V(R) over-- Is this going to come out right?-- over 2 to the n, over-- this seems upside-down.
Ah, no, it isn't upside-down.
Because I'm using this expression here, I do want 1 over G(R).
So this is 1 over G(R).
If I multiply this by 1/12, now I get 1 over G(R), so I need to put a 1/12 here.
And then I have a P(R) over 4 sigma squared.
So does everyone agree I can write it that way?
This'll be good.
Go out with a bang.
So what are all these things?
This is the coding gain of lambda.
This, according to this, is the size of the constellation to the 2 over n, approximately.
The rate of the constellation is log 2 times the size of lambda, R. So the rate is equal to just log 2 of this quantity.
So I can say 2 to the R. This is 2 to the R equals this.
So this would be 2 to the 2 over n, 2R over n. Is that right?
Again, I'm unsure if that's what I want.
This is the shaping gain of the region, these three things together, times 3, times-- what's P(R) over sigma squared?
That's the SNR.
That's the average power per dimension.
And this is the average noise power per dimension.
I've used up these in that.
So I've got a 1 over 1/12.
That's 12 times that.
So this is 3 times SNR.
And let's see.
This is 2R over n is just 2 to the--
Isn't that just 2R?
Because [UNINTELLIGIBLE] normalized by the dimension
Yeah.
And furthermore, it's minus, because it's upside-down.
So it's 2 to the - 2R.
Actually, what I want is the spectral efficiency.
And the spectral efficiency is the rate per two dimensions.
So this is just 2 to the minus rho.
So with a little help for my friends, what's SNR over 2 to the rho?
[INAUDIBLE]
It's approximately SNR norm.
Certainly true as row becomes large.
It's actually SNR over 2 to the rho minus 1.
So this is SNR norm.
OK, so with a little hand waving and a few high SNR approximations, we get that, in summary, for the UBE, the probability of error is approximately K_min(lambda) times Q to the square root of the coding gain of the lattice, times the shaping gain of the region, times 3SNR norm.
And that's a wonderful approximation.
Let me just call out some of its properties to you, and then we'll talk about them next time.
But first of all, there's nothing here-- all these terms involve either lambda or R. So we have completely separated, in this analysis, the effects of the region from the effects of the lattice.
We can choose the lattice and the region independently.
And they independently affect how good a performance we get.
For the baseline, which was either m-PAM or m-by-m QAM, we have, for the baseline PAM or QAM, we've set it up so that both the coding gain of the lattice and the shaping gain of the region are equal to 1.
So for the baseline, this gives us what we already developed back in the early stages.
I won't worry about this -- Q-hat, but it does give you the right coefficient out here, too, depending on whether you normalize it per two dimensions or what.
It just gives you Q to the square root of 3SNR norm, which I hope you remember from chapter five.
So this is our baseline curve for m-PAM or m-by-m QAM.
We just plotted versus SNR norm, the probability of error per two dimensions, which is what I recommended.
And we got some curve.
So this is the baseline.
And now we've got everything separated out in a nice way, so that if somebody gives us a lattice with a certain coding gain and a region with a certain shaping gain, those both just add up as gains.
If that's all we had, forget the coefficient, we would just move to the right here, and it would give us effective reductions and the required SNR norm by the coding gain of the lattice and the shaping gain of the region.
And we'd get the same curve moved out over here.
Again, as in the binary case, typically we're going to get a larger number of near neighbors, leading to a larger coefficient out here.
We want to normalize this per two dimensions.
Per two dimensions, it's only 4 for the baseline.
But in general, we're going to get a KS of lambda, the number of nearest neighbors per two dimensions, which is going to raise this a little.
And that will lower the effective coding gain down here.
But we've basically, by these large constellation, high SNR approximations, we've really reduced the analysis of lattice constellations to a very simple task, just to analyze what's the coding gain, what's the shaping gain, number of nearest neighbors, plot the performance curve, end of story.
So we'll start there next time.
OK, well I'm going to start now.
As I say, I know that the wireless course has its final at 11:00, so we might not see the people who are taking it.
Is anyone here in the wireless course?
OK. Q.E.D.
Ipso facto ergo sum.
OK, just to remind you, I think we have Problem Set 10 solutions today.
In any case, they'll be available on the web.
The final is next Tuesday.
Do bring calculators if you have them.
They won't be absolutely essential, but they might be useful.
You may bring five sheets of notes.
It's a good exercise to make these up.
And otherwise, it's closed-book.
Ashish will hold a review session tomorrow, 4:00 to 6:00, 26-328, which I'm sure will be excellent as usual.
All right, just to remind you, we're in Chapter 14.
Most of the course we talked about coding for power-limited channels, where basically you could get away with binary codes with very little loss of optimality.
For bandwidth-limited, additive white Gaussian noise channels, we've got to consider non-binary, multi-level codes, we've really got to go out into Euclidian space and use Euclidian distance as our metric rather than Hamming distance.
So we started out with a simple idea, build a lattice constellation.
A lattice is an infinite set of points in end space that has a group property.
That's it, a discrete set of points.
We take a finite set of them by chopping out a region R, a compact region, which chops out a finite number of points in this lattice or in a translate of the lattice.
And we went through this exercise of computing the union-bound estimate, which I remind you is a reasonable estimate for codes that are not too complex.
We're not going to use this for capacity-approaching codes.
But for codes, where..., basically dominated by minimum distance properties, how many nearest neighbors are there, the likeliest error events are all to the nearest neighbors, we can use the union-bound estimate, which basically just tallies the probability of error to the nearest neighbor events by the union-bound, which itself is inaccurate if you get too many near neighbor events.
And we've got this nice approximate expression that the probability of error per n dimensions, if we're signaling in n dimensions, is the number of nearest neighbors in the lattice times the Q to the square root of a function times an expression, which involves a coding gain of the lattice, and separably a shaping gain of the region times 3 times SNR norm.
So this embodies all kinds of simplifications.
It separates the effects of the lattice and the region.
Basically, we can optimize each of these separately to optimize this expression.
And also, by the way, by talking about SNR norm, which is SNR normalized by 2 to the spectral efficiency, it makes this whole expression independent of rate or equivalently independent of spectral efficiency.
We saw this back when we talked about m by m QAM, we got a universal expression.
The probability of error was simply this expression with a 3SNR norm in here - regardless of what m was (regardless of how big the QAM constellation was), we got the same approximate expression.
So this is also independent of the size of the constellations in this large constellation, high SNR approximation.
Once you get to enough points, everything sort of washes out.
So it's very useful.
And I remind you that the coding gain basically depends on the minimum square distance normalized by the density of the lattice, if you like, normalized to two dimensions.
The shaping gain is basically the shaping gain of an n-cube divided by-- this is the normalized second moment of an n-cube, this is the normalized second moment of your region, which better be better than that of an n-cube, meaning smaller.
And writing it out, it's equal to this, where this is the second moment of your region.
And this is the volume, again, normalized to two dimensions.
So the baseline constellations, we showed these were invariant to Cartesian products.
In any number of dimensions, n, we can take the integer lattice or any translate of the integer lattice and we can take a region, which is the n-cube -- which will chop out basically a square or cubic constellation, a regular constellation in n dimensions-- and we'll get that the coding gain is the shaping gain is 1.
So we'll get just our baseline expression for any integer lattice shaped by an n-cube centered on the origin.
And this is what it looks like.
This is back from Chapter Four or whatever.
We get this universal curve, the baseline curve, which holds for m-PAM, m by m QAM, or you can Cartesian product these up into as high a dimension as you like.
The constellation we're talking about is really m-PAM Cartesian product to the nth.
For any of those constellations, if we normalize the probability of error to two dimensions-- which is what I'm recommending to you, normalize everything per two dimensions in the high SNR regime-- then we basically get the number of near neighbors in Z-squared, the QAM lattice, which is 4 times Q to the square root of 3SNR norm.
And it looks like this.
We plot probability of error per two dimensions over SNR norm.
I think at 10 to the minus 5, it's 8.4 dB.
Does anybody remember?
But it's somewhere around 9 dB.
And the Shannon limit, as we recall, is for SNR norm, it's at 0 dB.
So this is where we can play.
Any code will fall somewhere between these two curves, and the name of the game is to get a code the gets as close to the Shannon limit as possible.
And at least in this moderate complexity regime, the way we play the game is we try to optimize the shaping region, we try to optimize the lattice, we're going to get a certain gain from shaping, and a certain gain from coding, de-rated by the number of nearest neighbors, which is going to turn out to be an important effect for some well-known lattices because they have huge numbers of near neighbors.
And that will give us some curve somewhere in here, and we plot that and we're done.
We've written a paper for that particular lattice constellation, for its performance.
Today I'm going to talk quickly about shaping, because that's a very easy story, and obvious, and it won't take long.
Then I'm going to talk about lattices, which is really a subject in mathematics, what's known about lattices.
I guess the decoding part of that story is not mathematics.
That's engineering.
And then I'm going to talk about trellis codes, which are the analog of convolutional codes, and which, similarly to the way convolutional codes beat block codes for the bandwidth for the power-limited regime, trellis codes tend to beat lattice codes in terms of performance versus complexity for the bandwidth-limited regime.
So, shaping.
Shaping is really an almost trivial story.
This really wasn't recognized as a separable issue until perhaps sometime in the '80s, when trellis-coded modulation was invented.
In '82 was Ungerboeck's paper.
That was really the breakthrough in bandwidth-limited coding, that trellis codes were the first bandwidth-limited codes that were widely implemented.
And after people analyzed them for a while, they realized that you could separate out the shaping effect from the coding effect, which is what I've shown by this expression.
So anyone care to guess what the optimum region is in n dimensions?
I guess I've already mentioned it last time.
The optimum region, what do we want to do?
We want to minimize the normalized second moment-- well, a second moment with appropriate normalization, what would you do?
Well, the optimum region for just about anything is an n-sphere, and it certainly is here.
So we take Rn equals an n-sphere.
If we have a two-dimensional sphere, what are its parameters?
We certainly know that the volume of this region, if this is R, it's pi times R squared.
And the second moment of this region, how do we do that?
We integrate ... go to polar coordinates, 0 to 2pi, d pi, 0 to R, rdr.
And then what are we measuring?
The second moment is r squared, but we normalize it per dimension, which is 2.
Oh, and we've got to take the uniform distribution over v of R, and that's one over pi R squared.
So have I got it now?
This is the second moment of a uniform distribution, a probability distribution over the 2-sphere.
And so this will be 1 over 2pi R squared times R 4th over 4, right?
Is that all correct?
And so p of R over v of R is already not normalized to two dimensions, so we don't have to re-normalize it.
I didn't get the 2pi in here.
I need some help, class.
When it comes down to integration, I'm not necessarily the world's best.
So R squared over 4... R squared over 4pi R squared, so we get 1 over 4pi.
Is that right?
That looks about right because it's fairly close to 1 over 12.
So what is the shaping gain of a 2-sphere?
And I can just say 2-sphere, because I know it's invariant to scaling.
Rotation isn't going to do much to it.
It's 1/12 over this, the normalized second moment, 1 over 4pi, which is pi over 3, which is about 2/10 of a dB.
OK, so I've improved the second moment by 2/10 of a dB, reduced it to 2/10 of a dB.
This is not very impressive.
So of course, a lot of the early work was on shaping two-dimensional constellations.
And this basically says if you get a large constellation, you don't even see this effect to, say, 64-QAM.
If you have a 64-QAM constellation, 8 by 8-- Somebody at Paradyne noticed that you could improve it slightly by taking the four corner points out and putting them up here, without reducing the minimum distance.
OK, so there is an improved, more spherical 64-point constellation.
It's improved by about 0.1 dB, as I remember.
And what this calculation says is if you go up to 256, 1024, and you simply take all the points in the QAM grid that are surrounded by a sphere of appropriate size to pick out 2 to the n points, for high, large constellations you get a gain of about 0.2 dB.
So this is nothing to write home about.
What happens as n goes to infinity?
We look up in a math book what the expressions are for the normalized second moment-- actually, you can do it just by this kind of calculation.
It's easy enough.
And just by scaling, you'll find that the shaping gain of the n-sphere goes to pi times e over 6, which is 1.53 dB.
A famous number not only now in shaping, but also in quantization, where you're also interested in minimizing the second moment of a quantization cell.
Or it's something like 0.229 bits per two dimensions, is another way of expressing it.
All right, so what is this?
This is the maximum shaping gain you could get by the optimum region in the optimum number of dimensions, which is as large as you can go.
So this says this is the ultimate shaping gain.
You can never get a shaping gain of more than 1 and 1/2 dB.
So it says when we play this game, we've got about 9 dB down here that we want to make up at 10 to the minus 5, 10 to the minus 6, we can get only 1 and a 1/2 dB of it by doing good shaping in large dimensions.
That's the best we can ever do.
So that's a little bit discouraging.
It says shaping is never going to give you that much.
On the other hand, you can't get this 1 and a 1/2 dB in any other way.
If you have n-cubed constellations, you will always be limited to be 1 and a 1/2 dB away from the Shannon limit.
So if you want to approach the Shannon limit, you do have to address the shaping problem.
You can't ignore it.
1 and a 1/2 dB is enough to be noticed in most applications, so you should do some moderate shaping.
Now, of course we can plot the curve.
It's done in the notes for any n whatsoever, and you'll see that it goes up fairly steeply at first.
It gets to about 1 dB at 16 dimensions, so if you shape in a 16 dimensional sphere, you can get 1 dB out of this maximum of 1 and a 1/2 dBs you can ever get.
If you go to 32, it's 1.2 dB or something like that, it begins to slope out.
So that gives you a good idea of what you can actually attain.
In the V.34 modem, for instance, a rather simple shaping scheme is used that gets up to about 0.8 dB of shaping gain.
It's called shaping on regions.
I just mentioned some techniques for shaping.
As with a lot of things, once shaping was realized to be a separate subject from coding, there was a flurry of activity on it.
People basically proposed a number of different schemes, particularly in the context of the V.34 modem standards development.
Of which trellis shaping, I would say, is the most elegant.
This is kind of a dual to trellis-coded modulation where we use trellis codes for shaping rather than coding.
So that's elegant.
But a cruder and quite effective, just fine technique, shell mapping, was used in the V.34 modem.
The basic idea here is that you take a two-dimensional constellation, you just divide it into shells like this.
It turns out in two dimensions, if you take a constant delta here, that the average energy goes up, I think, linearly with delta or some way nicely with delta.
You take a constellation bigger than you really need, so you have the possibility of some redundancy.
So you identify each point in the constellation by which region it lands in.
You then go up to 16 dimensions or something-- I think it is 16 dimensions in the V.34 modem-- and you assign each series of points a score depending on which shells its points land in.
And you basically have a big table that chooses the lowest energy combinations from all the possible series of eight shells, which can be done fairly elegantly using generating function techniques.
I realize this is just hand-waving and you can't get it, but the effect is that you use this shell with very high probability.
You sort of get a Gaussian-like probability on the shells.
Let me just draw that like this.
You don't use the points out in this outermost shell very much.
You use this a lot, and they sort of have an overall Gaussian distribution.
And I should mention that this is an equivalent way of getting shaping gain.
If we're shaping uniformly over an n-sphere in a high number n of dimensions and we basically have constellations which consist of two-dimensional QAM points, what is the projection of a uniform distribution over an n-sphere down onto two dimensions?
If we look at each two-dimensional component of these n-dimensional vectors, what distribution will it have if we impose a uniform spherical distribution on the high dimensional constellation?
It will turn out that it becomes Gaussian in two dimensions.
Think of the projection of the 2-sphere down on one dimension.
The projection of a 2-sphere on one dimension is something like this.
It's not Gaussian, but there's more probability that one of the coordinates will be near 0 than it will be out at one of these maxima here.
So this is just the same thing raised up to n dimensions.
So an equivalent way of looking at this problem is we want somehow to get a Gaussian-like distribution on a one- or two-dimensional constellation, and this is one way of doing it.
And you can get the same pi times e over 6 factor by just looking at it from an entropy point of view, the entropy of a two-dimensional Gaussian distribution relative to the entropy of a uniform distribution over a square.
That will give you the same factor if you do everything right.
OK so basically, shaping is easy.
These techniques are not difficult to implement.
You can get one dB of shaping gain quite easily through a variety of techniques.
It's worth getting that one dB, you're not going to get it by any other means.
But having done that, that's all you're ever going to get, and so end of story on shaping.
So as I said, it was a flurry of activity in the '90s, and no one has looked at shaping since then.
So that's part of the story.
Let's continue now with lattices, an old mathematical subject.
So what we're interested in is, what's the densest lattice packing in n dimensions?
This appeared in mathematics, long before it appeared in engineering, as Hermite's parameter.
And Hermite said this was the right way to measure the density of a lattice, normalized in all the appropriate ways.
So any version of a lattice is equivalent.
A version is something that you get by scaling or rotating or even by Cartesian products, which I'm not sure how widespread that idea is.
So what's the densest lattice in two dimensions again?
We want to find the densest lattice, in terms of coding gain or Hermite's parameter in n dimensions.
Anyone care to guess what the densest lattice in two dimensions is?
It's the hexagonal lattice.
So in two dimensions, hexagonal, which reasons of mathematical history is denoted A_2.
And its coding gain, I believe we computed it last time, and it was 0.6 dB.
So what's the most we could ever do by fooling around with two-dimensional constellations?
Suppose I want the best possible 256 point QAM constellation?
Well, according to these large constellation approximations, the best I could do would be to take the 256 lowest energy points in some translate of the hexagonal lattice.
So I might fool around with the translation vector a little bit to try to find the absolute lowest energy point, but it really wouldn't matter very much regardless of what I did.
I could expect 0.6 dB of coding gain and 0.2 dB of shaping gain, or 0.8 dB better than the 8 by 8 - sorry the 16 by 16 QAM constellation with the same number of points and the same minimum distance.
This basically tells you what your options are.
2D, there's not much to do.
You remember way back in Chapter One, I gave you this problem about the 16 point, 2D constellation.
That is precisely 16 points from a certain translate of the hexagonal lattice that fall within a circle that's just big enough to enclose 16 points.
And it has, I forget, something like a 0.7 dB gain over the 4 by 4 square constellation.
We computed that in the very first homework.
So again, staying to two dimensions, things aren't very exciting.
Turns out, it's most interesting to go in powers of two.
In four dimensions, the densest lattice is something called D4, Schlafly's lattice, 1850-something.
And its coding gain is precisely the square root of 2, which is 1.5 dB.
8 dimensions, it's E8, Gosset's lattice, about 1900.
The D4 and E8 come, again, from the mathematical literature and have something to do with their relationship to certain groups.
Lattices and finite groups are very closely connected.
And this has a coding gain of 2 or 3 dB.
16 dimensions, it's something just called lambda 16.
It's a Barnes-Wall lattice, as all of these are, actually.
And it's 2 to the 3/2, or 4.5 dB.
You see a certain pattern here?
In 32 dimensions, we have two lattices that I'll mention.
There's the 32-dimensional Barnes-Wall lattice, which has a coding gain of 4.6 dB.
Or there's actually Quebbeman in the '60s, I think, came up with a denser lattice.
So this is still a subject of active investigation, particularly in dimensions like 19, where it's not so easy to find the densest lattice.
And this, I forget, is maybe 6.2 dB.
It's definitely denser.
But it's something like that.
This basically goes to infinity.
You can get denser and denser lattices just by following this chain here which is a chain of Barnes-Wall lattices.
The nominal coding gain can be taken to infinity, just as we saw with kd over n. That goes to infinity for rate 1/2 Reed-Muller codes.
But of course, Shannon says we can't ever get more than 9 dB of effective coding gain here.
Or actually, we can't get more than 7 and a 1/2 dB if we take into account that 1 and a 1/2 is going to be given by shaping gain.
So what must be happening here?
We must be having a huge number of near neighbors.
The number of nearest neighbors in the hexagonal lattice is six.
In D4 it's 24, in E8 it's 240, in lambda 16 I think it's 4320-- the correct numbers are in the notes.
By the time we get up here, we get something like a 146,680 or something like that.
So these lattices are designed to have a very high degree of symmetry.
That means if you stand on any point, you look around you, they've pushed the neighboring points as far away from you as they can.
But now there are a great many of them, so you get a uniform sky just filled with points around you, a number of dimensions.
So that's what the problem is and that means that the effective coding gain-- I again do the calculation.
I know here we lose at least two dB.
This is something like 3.8 dB.
OK, so the effective coding gain actually sags and begins to top out as you get out here.
Maybe nowadays you could think of implementing a larger lattice, but I think this is the largest anyone has ever thought about implementing.
And what do we get?
A coding gain of less than 4 dB -- an effective coding gain.
So if you use that, you'd be, say, somewhere around 5 db here.
We go over 6 dB, but then we go up by this normalized per two dimensions, so we divide by 16.
That's still 10,000, which is still 13 factors of 2, which is 2.6 dB.
You can see where the problem is.
So anyway, we get something that's really this moved way up a lot more, and as a result we get a much steeper curve.
This method doesn't work up here, but this might be lambda 16 or lambda 32.
And then we can get another 1 and a 1/2 dB of shaping gain.
So maybe 4 dB, optimistically.
OK, not really.
So that's a little disappointing.
Just by looking up in the math textbooks, we don't do very striking things.
I mention in particular the Barnes-Wall lattices, and if I had more time I would develop them because they're very much analogous to the Reed-Muller codes.
They are constructed by a u, u plus v construction in the same way as Reed-Muller codes.
They are length-doubling constructions that also double the distance, just as the Reed-Muller codes do now Euclidian distance.
But let me just give you the first couple of sentences in a discussion of the Barnes-Wall lattices, which were only discovered in '59, about the same time as the Reed-Muller codes.
Suppose we take the QAM grid like this.
An important observation is that if we take every other point in sort-of checkerboard fashion, what do we get?
Imagine it's going to infinity in both directions.
So this is a subset of half the points, basically.
Half the density, twice the volume of points that, itself is a rotated and scaled version of the integer lattice.
Take the centers on a checkerboard and they form a 45 degree rotated and square root of 2 scaled version of the integer lattice.
If this had minimum squared distance 1 between points, then this has minimum squared distance 2 between points.
So we could label the points red, black, red, black, red, black.
If we did that, the red points would be a co-set of this sub-lattice and the black points would be a co-set-- meaning a translate-- of this sub-lattice.
So let me write that as follows.
If we take Z-squared, it has a sub-lattice which we call RZ squared, r for rotation.
Rotation by the Hadamard matrix, 1, 1, 1 minus 1.
If I write d min squared, this has d min squared at 1, this has twice the d min squared.
So I partition my lattice, or any translate of this lattice, into two co-sets of the sub-lattice with twice the minimum squared distance.
This is the first steps in what Ungerboeck called set partitioning.
So it's an obvious way to partition any QAM-type signal set, red and black points.
Having done that, I can do it again.
Alright, do it again, what do I get?
So now I'm going to take every other one of these points, which will leave me, say, these 4 points.
Let me call these A, A, A, A.
These points might be B, B, B, B, the other ones that I cut out.
I could call these C, C, C, C and D, D, D, D. Now I've partitioned each of these subsets into two subsets.
I've partitioned one into A, B and the other into C, D, such that - what does A look like?
A is another translate of a lattice.
This happens to be Z-squared scaled by 2.
It's not rotated, it's just Z-squared scaled by 2.
So we get down to 2 times Z-squared.
And what's its minimum squared distance?
4
4.
So I've doubled the distance again.
This is an elementary but important observation.
I can continually go through these, partitioning into two subsets.
Every time I do, I double the distance.
I get another version of the integer lattice at a higher scale and, perhaps, rotated.
So if I did it again, I would get eight subsets with minimum squared distance 8 between points in each subset.
So it's just a repeated divide-by-2 scheme.
All right, now to get the Barnes-Wall lattices, you remember how we got the Reed-Muller codes.
We took, say, two codes of length 2 and we put them together with the u, u plus v construction and we got codes of length 4 that had the minimum square distance of the higher distance code, if we chose them properly.
That's the same thing we do here.
If we put together these two lattices, if we choose them-- well, just use the u, u plus v construction where u is from Z-squared and v is from RZ-squared.
So in words, that's what you do.
If you're interested in looking it up, look it up in the notes.
What you get is a lattice, D4, that has normalized density halfway between that of Z-squared and RZ-squared, it turns out-- sort of analogous to the code-- and its minimum square distance is 2.
Or if you put these two together, you get something called RD4, whose minimum square distance is 4.
So you get something whose normalized volume is halfway between that of these two normalized volumes, but it has the same minimum distance as this guy, and that's where you get the coding gain from.
This is denser but still has the same minimum distance.
If we do this again, go out to eight dimensions, we get, it turns out, E8.
And this has minimum square distance 4.
But it has the same normalized volume normalized back to two dimensions as this guy, which only had minimum squared distance 2.
So that's precisely why this has a nominal coding gain of 2.
We've doubled the minimum square distance without changing its density, normalized back to two dimensions.
All right, so you get all the Barnes-Wall lattices just by doing this.
This turns out to be a sub-lattice of this guy.
Not surprisingly, just as these lattices are nested, just as the Reed-Muller codes are nested, so you get a similar tableau.
You get a similar construction that you can take out to 16 dimensions, 32 dimensions, however many dimensions you want.
And every time you do, you increase the coding gain by a factor of square root of 2, so the coding gain eventually gets driven to infinity, and that's the story on Barnes-Wall lattices.
So similarly to Reed-Muller codes, these are lattices that are the densest known in certain small dimensions like 4, 8, and 16.
We don't know any denser lattices.
You start to get up to 32, 64 dimensions, then we do know of denser lattices.
But they are not nearly as nicely structured, and maybe these are still the best from a complexity point of view.
You can play similar games on building trellis representations of these lattices, as we did for Reed-Muller codes.
They have similar kinds of trellises because they're based on the same construction.
So that's the story of Barnes-Wall lattices.
The most famous lattice is the Leech lattice, which is a super-dense lattice in 24 dimensions.
It has a coding gain of 6 dB.
And unfortunately, it has 195,560 near neighbors or something like that.
So you can see, it turns out it's a very close cousin of this 32-dimensional Barnes-Wall lattice.
It's related to the 24-12-8 Golay code in the same way as the Barnes-Wall lattices are to the Reed-Muller codes.
So the fact that the Golay code is so exceptional can be seen just as a projection of the fact that the Leech lattice is so exceptional.
This somehow has the Golay code buried in it, if you can find it in here.
And the Leech lattice is particularly notable not just because it has remarkably high coding gain for its dimension-- it's remarkably dense for its dimension, exceptional-- but also because it has remarkable symmetries, so it's connected to some of the largest exceptional groups, the monster exceptional groups.
And it was a key step in the classification of all finite groups to find the symmetry group of this lattice.
Just a little liberal arts for you.
So that's what we can get with lattices.
By going out to the Leech lattice or the Barnes-Wall lattice-- this could equally well be the Leech lattice-- again, effective coding gain is only about 4 dB or maybe even less because of the exceptional number of near neighbors here.
To one digit of significance, that's approximately its coding gain.
So how can we go beyond this?
As I said already, the next great breakthrough was trellis codes for which Gottfried Ungerboeck became famous.
Trellis codes, as I say in the notes, are basically-- first of all, Ungerboeck thought about the problem from a Euclidian distance point of view.
So what you had to do to break out of Hamming distance thinking.
He had the idea that in the bandwidth-limited regime, what you want to do is expand the constellation rather than expand the bandwidth.
So to send four bits per second per Hertz.
You could do that using uncoded 16-QAM constellation.
Or Ungerboeck saw, well, let's make it into a 32-point constellation.
Now we have some redundancy, now we can do some coding on this constellation.
He used the set partitioning idea, which I've just explained to you.
Initially, he just looked at a QAM constellation and he said, let's divide it up into A, B, A, B, C, D, C, D, A, B, A, B, C, D, C, D. So let's partition it from A, B, C, D into A and D as I've done it here, and B and C are these two subsets.
And we'll break this up into A and D and B and C. And then some theoretical people came along and said, well, this is just Z-squared, this is RZ-squared, and this is 2Z-squared.
Let's, first of all, partition our enlarged constellation.
Now here's the setup we're going to use for coding.
We are going to bring in our ...If we're trying to send-- I forget what notation I used in the notes-- b bits per two dimensions.
Let's take one bit per two dimensions and put it into a rate 1/2 convolutional code.
So we're going to get out two bits.
Everything here is going to be per two dimensions because we're going to use a two-dimensional signal set.
And we'll bring in our other b minus 1 bits down here.
These are called uncoded bits.
For these two bits, we're going to go into a subset selector.
Two bits are going to select among-- this going to be something from A, B, C, or D. It's going to tell us which subset to use.
All right, then we'll use that in combination with these b minus 1 bits to select a point from a 2 to the b plus 1 bit QAM constellation.
Two dimensions.
So this is actually our signal point y, our two-dimensional signal point.
Do you see how the numbers work out?
We want to send b bits per QAM point, per two dimensions.
We're going to use a 2 to the b plus 1 point constellation, so 32 instead of 16, say.
Take one bit in here, generate two streams, one redundant.
This will tell us A, B, C, D, so that's two of the bits you need to select one of these points.
Say, in a 16-point constellation, we'll provide a bit label that looks like this.
For A, it will be 0, 0. b is 0, 1, c is 1, 0, D is 1, 1.
That's always the low-order bits in any selection for A, B, C, D. And then we'll take two high-order bits, which then tell us which of these four groups it's in.
So we'll have another 0, 0, 0, 1, 1, 0, 1, 1.
Now I'm talking about b equals 3.
We have two bits that come in here and select one of these blocks, and then two bits that come out of here and select A, B, C, D within the block.
There's two parts to the label.
One of them determines A, B, C, D, the other one just determines some larger group.
These bits are called uncoded because there's no protection.
If you sent point A over here and received point A over here, that would look like a perfectly legitimate code word.
If we make an error of that far, which is d min squared equals 4, say.
We make an error of size d min equal 2 all the way over here, then there's no further protection from this code because this code only determines the sequence of As, Bs, Cs, and Ds.
This is called a within-subset error.
There's no protection on within subset errors.
What the code does is it protects against smaller errors that are between subset errors like A to B, which has squared distance of 1 or A to D has a squared distance of two, and so forth.
Are you with me?
Nobody's asking questions.
Maybe I haven't permitted it.
Do you follow what the overall setup is?
I've done everything now, except choose an exact constellation.
For the exact constellation, I'll just choose one of the best QAM signal sets.
If it's large enough, I'll shape it like a circle.
Or I might even try to put Gaussian shaping on it by some completely extraneous thing that works only on these clumps.
So shaping takes place on the higher order bits.
You can see that, given a large enough constellation just by choosing these clumps, I can make the constellation sort of have the shape of a circle.
The subsets of A, B, C, and D points have to all be the same size in order for this scheme to work.
But I can do that within a sort of circular constellation if the constellation is large enough.
Or I can put some Gaussian distribution on these clumps.
So I do shaping down here.
Initially, of course, Ungerboeck didn't worry about shaping.
But if I try to do shaping, it's down on this index, which is considered to be the higher order bits that determine the growth structure of the constellation.
These are the lower order bits, these two bits, and they determine the fine structure which is determined by this code.
So the last thing I have to select is the code, and it turns out that the same code we used in the example is a good code to use here.
Suppose we use the four-state rate 1/2 example code that I've used before that is everybody's favorite example because it has such excellent distance properties.
So it looks like this.
So that's my rate 1/2 encoder.
You remember as a binary code, it had minimum Hamming distance 5.
And the only minimum distance code word was the impulse response.
Let me write the impulse response as this way: 11, 01, 11.
This is what we get out at time 0, time 1, and time 2.
Let me assign the following labels to these subsets.
We'll make that 00, this one 11, this one 01, and that one 10.
In other words, this is the complement of that.
This is the complement of that.
What is the nearest neighbor point in Euclidian space for this sequence?
If I complement the first two bits coming out, then I'm going to go from one of these subsets to another, but within one of these RZ-squared subsets, which has minimum squared distance 2.
So let's remind ourselves that the minimum squared distance within A, B, C, D is 1, within A, D or B, C is 2, within 2Z-squared is 4.
So I'm claiming-- there's actually a problem on this on Problem Set 10 if you tried to do it-- that any error of this type, 11, is going to cause me a Euclidian error of minimum squared distance 2, at least.
It can only cause me to go from A to D, which is always at least squared distance 2, or from B to C, or vice versa.
So in terms of Euclidian distance, d min squared, I'm always going to get a squared distance of 2 at this point.
And actually, any place you diverged in the trellis, because of the structure of the code, this is going to be true.
I'm always going to get a squared distance of 2 when two trellis paths come together in the trellis.
I always have a binary difference of 11, so by the same argument I'm always going to get a Euclidian squared distance of 2 when two trellis paths come together.
And furthermore, I'm going to have at least one difference somewhere in between, which is going to, again, in terms of Euclidian distance, lead to a Euclidian distance of at least 1.
So the conclusion is that between any two sequences of constellation points that are labeled by two distinct code words here, I'm going to have a cumulative Euclidian squared distance of at least 5, by the same argument as I had for the binary code.
But now it turns out that I'm still in a regime where binary distance, Hamming distance translates directly to Euclidian distance.
This is what you saw.
Basically, what we're using here is this is Z-squared, you can think of as the image of the (2, 2) binary code.
RZ-squared is the image of the (2, 1) binary code, and this 2Z-squared is the image of the (2, 0) binary code.
You get the same partition structure for these codes and their co-sets.
This is the set of all four two-tuples.
We divided it into two subsets, each of minimum distance 2 here, and here we divide it into four subsets, each of minimum distance infinity.
I'm not going to take the time to make this precise, but think of how that relation goes.
So I've assured myself of a Euclidian squared distance at least 5 between signal point sequences that correspond to two different code words.
That's excellent.
Is the minimum squared distance of the code then equal to 5, as it was in the binary case of this trellis code?
No, it's not.
And the problem is I could make this kind of error, which is completely invisible to the code.
I can make a within subset error, and this only has minimum square distance of 4.
So the conclusion of this analysis is that this is what's called a four-state two-dimensional trellis code.
This four-state two-dimensional trellis code has a minimum squared distance of 4, and it has a number of nearest neighbors per two dimension of 4.
That's the number of nearest neighbors within 2Z-squared, which is one of these subsets.
So the conclusion is that already, this has a legitimate 3 dB coding gain.
Its coefficient is still just the QAM coefficient, and this very simple code has this performance curve moved over.
So it's 3 dB all the way up the line.
So that's pretty good.
This is a trivial code that could be decoding-- by the way, it was, again, done by the Viterbi algorithm of a four-state trellis.
You just take the sequences and it's exactly the same.
You measure distance in terms of minimum squared distance.
The only difference is when you decode, you first find what the closest point is within the A, B, C and D subsets.
That is what you're going to take as your representative for that subset.
So you first make basically an uncoded decision, which is the closest point in A?
Which is the closest point in B, which is the closest point in C, which is the closest point in D?
You're going to get a metric for each of those.
How far is each of those from the receive point?
So then in trellis decoding, what the trellis is going to look like is something like this.
A, D, A, D, B, C, and so forth.
The trellis is going to look exactly the same, but here you get A, D, D, A, B, C, C, B.
So what metric are we going to use for each of these branches?
First, we find the closest point within each subset, and then we use the metric for that point.
Clearly, if we're going to choose A for this time interval, then we should use the closest point in A as our receive decision.
We can get that survivor without any memory whatsoever when we make that decision, and then we take the sequences of representatives and find the best of those representatives as our best trellis-coded sequence.
Any sequence of points that has labels that correspond to some path through this trellis is a valid point and vice versa.
If and only if the sequence of labels is according to this trellis is it a legitimate trellis-coded point.
So it's a nice amalgamation.
Great.
With a simple four-state code, basically we've doubled the size the constellation.
The constellation expansion is 2, the constellation expansion factor with twice as large a constellation, we've been able to achieve a coding gain of 3 dB.
Not bad.
So how do we extend this idea?
First, extension would be to-- all right, let's go up to an 8 state, 16 state, whatever code.
But is that going to improve our minimum distance using this block diagram?
It's not, because we're still stuck with this within subset distance.
So we can never get past d min squared equals 4 with a four-way partition.
What we have to do is make one more level of partition and we need to make an eight way partition, at which point we're going to get a within subset distance of 8.
I left out one step, I'm sorry.
I'll come back to it.
We can get a code minimum squared distance up to 8, if we extend the minimum squared distance up to 8.
The thing I didn't use, I didn't tell you before, was in computing the coding gain of this code-- let's just call it c-- what should it be?
It should be the minimum squared distance of the code over the volume of the code per two dimensions.
In this case, n is 2 so I'm just going to do it over the volume of the code.
Now what is the volume of a convolutional code?
This is actually an interesting problem, because the Voronoi region of a trellis code is actually an infinite dimensional object.
A trellis code is really an infinite dimensional lattice.
It's a lattice in infinite dimensional space.
And if you look at the Voronoi region, it extends out into all these dimensions.
It might not for such a simple code, but in principle it does.
However, you can find other fundamental regions.
The way to think about this is instead of putting the volume here-- well, what have we cost ourselves in volume?
We've doubled the size of the constellation.
And in two dimensions, that means we've doubled the volume.
So we've quadrupled the minimum distance at the cost of doubling the volume.
We've covered twice as much space now because we have twice as many points.
And so, that's where we get the-- this is equal to, for this code, that's 4.
We have a factor of 2 loss in volume, and so that's where we get what I said was a coding gain of 2 or 3 dB.
Just to get you to the bottom line, we should replace the volume by two times-- I call it eta of the code-- where eta of the code is its redundancy per two dimensions.
In this case, we introduce one redundant bit per two dimensions, and that's why the constellation has to expand by a factor of 2.
You see, we get the right answer.
The redundancy has increased by one bit, so this has to be 2 to the b plus 1, so the constellation has to be twice as large, so the volume is twice as large.
I'll just hand-wave and tell you that's the factor you want down here representing volume -- 2 to the redundancy.
And you can prove this by finding a region which you can show is a fundamental region of the trellis code and has this volume.
But I won't go into that.
OK, so that's the calculation which I should've mentioned before, I skipped right over it.
Back to what I was saying.
To improve trellis codes, the first thing you can do is to, say, go to an eight-way partition.
So now we're going to have eight subsets.
So we're going to need three bits here.
So let's use two bits here and use a rate 2/3 trellis code.
We're going to have three bits coming out here to select subsets, and we're going to have b minus 2 bits down here to select a point still a 2b plus 1 point QAM constellation.
So our redundancy per two dimensions is still one bit per two dimensions.
We're still going to lose a factor of 2, but now we can go up to d min squared equals 8 and we can go up to a coding gain of 6 dB.
So in his original paper, Ungerboeck did this.
He showed codes going up to 256 states in two dimensions that gets you up to a nominal 6 dB coding gain.
And because they have low coefficients here, they get very close to 6 dB of effective coding gain.
Already in Ungerboeck, we have codes-- I mean, eventually you can get back to just the within-subset coefficient again, which is always 4.
You can get up to, say, 6 dB with maybe 512 states of effective coding gain, which is considerably better than the densest lattice we would think of using, and also quite reasonable to implement.
Now we're going to need a 256, 512-state Viterbi algorithm decoder.
This was a stunning breakthrough.
Immediately in 1993, everybody started building turbo codes.
Back in 1982, everybody started building trellis codes.
It happened to be the time of the V.32 modem, and everybody said boy, this is just what we need to get an additional 3 or 4 dB coding gain in the V.32 modem.
What was actually done there is this was simply an eight-state convolutional code, so it was rate 2/3.
And there was tweaking of this.
It was made rotationally invariant, made nonlinear.
And a lot of activity, which was very nice, stimulated by the standards effort.
But that was basically the state of the art in the '80s.
The one other improvement that was made was the following idea.
We really don't like doubling the size of the constellation, particularly for a large one, mostly for implementation reasons.
There were some feelings against nonlinearities a larger constellation would cost you.
You wanted to minimize the constellation expansion ratio.
So how were we ever going to do that?
You can't do it this way in two dimensions.
The way to do that is to go up to four dimensions.
So let's do subset partitioning in four dimensions.
So now think of four dimensional constellations.
In practice, these are just built from the Cartesian product of two two-dimensional constellations.
We start with Z4.
The next one down is D4, which has a minimum square distance of 2.
It's sometimes called a checkerboard constellation in four dimensions.
It's just the red-black idea again.
Then below that, we have a four-way partition into RZ4, which still has minimum distance 2.
And one more level of partitioning here into something into RD4, which has minimum squared distance of 4.
So this is an eight-way partition of a four-dimensional constellation.
This idea was introduced by Lee-Fang Wei.
Again, this is an interesting column over here.
You can think of this as being the (4, 4, 1) code.
This is the Euclidian image of the (4, 3, 2) code.
It kind of expanded in the lattice.
We know we can't do any better than (4, 2, 2).
Down here, this is the (4, 1, 4) code.
So again, this is the same partition as we get for binary codes with the distances-- again, for binary codes and this level of partitioning, Hamming distance translates to Euclidian distance.
It's really just the image of these codes as you go up to the lattices.
Now in this case, what's the best we can ever do?
Coding gain.
We still can't go up to a squared distance of greater than 4 because our within-subset distance is going to be within RD4.
So this is still limited to 4.
But now, what's our redundancy per two dimensions?
Here, say, we're going to use still a rate 2/3 convolutional code.
We'll now have an eight-subset selector in 4D, so everything is per 4D here.
So this is going to be 2b minus two bits and this is going to be, it turns out, a constellation that is twice as large in four dimensions.
We still have one redundant bit per four dimensions, so we need twice the size of the constellation in four dimensions.
But that's only 1/2 a bit of redundancy per two dimensions.
So this denominator becomes 2 to the 1/2.
Waving hands furiously here.
And so this could go up to 2 to the 3/2, which is 4.5 dB.
So there's actually a particularly nice eight-state code here that Wei came up with which, really, in terms of performance versus complexity is the only code that anyone has ever come up with that is distinctly better than any of Ungerboeck's original codes.
All of Ungerboeck's original codes were in one dimension, two dimensions, or PSK codes.
You could also do this on phase shift key constellations.
So with the Wei code, it gives you a very simple way to get 4 1/2 dB.
Actually, you have to go up to 16 states to get 4 1/2 dB.
The Wei code is only 4.2 dB.
And this is basically what was adopted for standards in the early '90s.
The V.34 modem standard, also the ADSL standard has the same code in it.
If you have DSL at home, that's what it's using in it.
Again, with a very simple code, the eight-state Viterbi algorithm, you can get 4.2 dB of coding gain.
The error coefficient in this case, this has 24 nearest neighbors in D4, or 12 per two dimensions.
So you lose 0.15 dB or something.
I'm sorry, 0.3 dB.
Anyway, this is a very nice code.
There are 32 and 64-state codes in V.34.
They don't gain you very much.
They each double the complexity and only gain you about 0.2 dB for each one.
All these things tend to saturate.
You can't get more than about 5 to 6 dB of coding gain with a reasonable size trellis code.
So where are we now?
We can get 5 or 6 dB of coding gain with, say, a 256-state trellis code.
We can get another 1 to 1 and a 1/2 dB of shaping gain.
So all together, we're up to 6 to 7.5 dB.
From here, we're up somewhere into the 2 to 3 dB range.
In other words, combining trellis code with, say, 256 states with some shaping scheme with maybe 1 dB of shaping gain, this gets you maybe 5 dB or 5.5 dB.
We get 6 dB total effective gain, and we're still 3 dB or maybe 2 or 3 dB from Shannon limit.
So, close but not quite there yet.
Five minutes to go.
So in the notes, there's one page on higher performance schemes.
There are higher performance schemes.
There are turbo codes for the bandwidth-limited regime that get you to within 1 dB of the Shannon limit.
Sequential decoding gets you to within about 1 and a 1/2 dB of the Shannon limit.
The idea of multi-level codes -- you can regard each of these two-way partitions as being specified by one bit.
So with three bits of labels, you can specify an eight-way partition.
Another idea is to code on each of these levels separately.
Use a binary code on each of these levels separately.
Each of these corresponds to an effective channel that has different characteristics.
You've basically got to take the Gaussian distribution and alias it around in accord with whatever this partition actually is.
But you can come up with an effective binary channel at each of these levels.
It turns out to be a binary symmetric input channel, always in these cases.
So you could think of taking a binary-- now, each one of these channels is going to have its own capacity.
It turns out that information theoretically, the capacity of multi-level coding on each of these levels independently is the same as the aggregate capacity of coding it on the whole channel.
It's a lovely separability result.
So in principle, if you could code at capacity on each of these binary channels, then you could get to the capacity of the aggregate channel, with a large enough constellation.
So that's a good idea.
We now know a lot of binary capacity-approaching code, so let's use a turbo code or a low-density parity-check code at each level, an appropriate code.
It turns out, as you might expect, that the first level is the really hard level.
In this case, you might have a rate 1/3 or 1/2 code and really work hard with your code.
At the next level, the effective error probability is much better.
Making hard decisions, you might have a probability of 1/10 or lower of making an error.
So these are really much easier problems.
It might be a better idea at these levels just to use algebraic coding or something that works very well.
The capacity is nearly one per bit at each level, so you want to code with very little redundancy at these next levels, but it's still capable of approaching capacity.
Well, it turns out this is a good scenario for Reed-Solomon codes or some kind of algebraic code.
So the moral is you only really need your capacity-approaching code at maybe the first or first and second levels.
Down here, just use a very high rate-- 1,023, 1,003 Reed-Solomon code over gf of 2 to the 10th-- very little redundancy, and drive the error rate down as low as you like at each of these levels.
So multi-level coding is a good way in principle.
I'm not sure if it's been done in practice.
In fact, I don't know of any capacity-approaching code that has been done in practice, except there's a very crude technique, I would say, called Bit-Interleaved Coded Modulation, BICM, which is the following simple idea.
Let's just take a single low-density parity-check code or some capacity-approaching code.
It's going to have to be designed for a BICM channel.
Let's totally interleave the bits and then let's use them as the bits that control each of these levels here.
At the output of the channel, we know which level the bits came out of.
So at the output of the channel, we know what the appropriate a posteriori probabilities are for each bit.
They're going to be wildly different.
A bit sent through here is going to be sent through a very highly reliable channel, a bit sent through here is going to be sent through a very low reliability channel.
But with all this interleaving and the independence approximation in the analysis, we're going to be able to design a code for this mixture channel, a mixture of high reliability and low reliability binary channels, that's going to get to the capacity of this mixture channel, which as I've already told you is equal to the capacity of the aggregate channel.
I guess there's some elegant features of it.
But that's, in practice, what people use.
They use it more because they've already developed a chip for binary channels, and they want to reuse that chip for some large constellation channel, some bandwidth-limited channel.
So this was originally advocated by Qualcomm as a, quote, pragmatic approach.
OK, we've got a chip that works for binary channels, so let's just interleave outputs and use it over this multi-level channel and it'll probably work well enough.
And, in fact, that's true.
And the information support for it, again, is the fact that you don't lose anything by coding bit by bit from a capacity point of view.
I think that's the only one that's been used in practice.
And on wireless cellular channels for instance, that tends to be what's used.
And of course, I know that perhaps a couple of you have an exam at 11:00, so I'll make a point of letting you out of here.
I've really enjoyed teaching the course this term.
It was the first time I was able to get to the analysis of capacity-approaching codes, so I enjoyed doing that.
At this point, I think the subject of coding for the additive white Gaussian noise channel, the traditional channel, is pretty well wrapped up.
I don't think we're going to see any great improvements.
There's still some very interesting theoretical questions about the convergence of these iterative decoding algorithms.
What do they converge to?
How fast do they converge?
There are interesting connections to statistical physics and inference and so forth that people are still exploring.
They're interested in graph theoretical properties.
There's attempts to algebraically construct these capacity-approaching codes, so we can say a lot more about their parameters and their performance, particularly for lower probabilities like 10 to the minus 15.
So you go to the information theory symposium, you'll still hear a lot of papers about coding effectively for additive white Gaussian noise or memoryless binary symmetric input channels.
But people ask me, is coding dead now?
Are Fourier transforms dead?
No, they're not dead.
But they've become sort of part of the standard core of what you should know.
I think this is a subject that anyone in communication should certainly learn, but is it an area where people are going to make further advances that are important from a practical point of view?
Obviously not very much, in view of the Shannon limit.
We are very close to the Shannon limit for these channels, and have been now for a few years.
So it's important to know about these things.
It's important to know when you're in the wireless course, for instance.
You basically calculate mutual information and assume that you can go at the mutual information.
This is how you do it.
So you should know how it's done, and that's why I think this course continues to be worth teaching.
But from a point of view of research, it's become a rather mature area, and the action has gone off to other kinds of channels.
Multi this, multi that, so forth.
Anything to do with networks or multi-user is where the action in coding is.
Data compression, of course, is a subject that we haven't talked about at all, which is also coding.
So anyway, I think it's good for you to have all this under your belts.
I hope that you find it useful in your future lives, and I wish you all well on the final exam.
Thank you.
OK.
So the thing that we're going to talk today about are macromolecules.
And often these are polymers.
Now, all macromolecules comprise of multiple carbons joined together in some way.
But the real definition of a polymer is that there is some kind of monomer that is joined together in a repeated way to make some kind of multimer that forms the polymer.
OK. Now, integral to understanding all of this is the notion of chemical bonds.
And you're going to, in this week's Section you're going to be addressing, remembering, understanding what chemical bonds are.
And here are some concepts that are essential that you understand for chemical bonds.
Let me list them on the board.
And then we'll go through them in some more detail.
And then I'm going to move on.
And you are going to deal with this more in Section, but this is really integral to understanding the rest of the course.
So one of the terms you need to know is electronegativity.
Anyone want to give me an electronegativity definition?
I may be able to find another chocolate fish for an electronegativity definition.
Where?
A little hand.
Yeah?
Say it again.
OK. Good.
Yeah.
How strongly an atom attracts electrons.
This is good.
You see, you get to practice catching and I get to practice throwing.
So the relative affinity of a particular atom for electrons.
OK?
So that's one think you need to know.
Valance.
Super important.
Valance.
Yeah?
How many electron slots available for connections.
Try to rephrase.
Yes, but rephrase it a bit.
We're talking about chemical bonds.
So?
OK, how many bonds.
I did not give that away.
OK.  How many bonds a molecule can, an atom can make.
OK?
OK.  And then we get into the bonds per se.
And I'm going to list them here.
I'm going to list them in order from strong to weak.
Covalent bonds.
Ionic.
I'll go through some slides in a moment.
Hydrogen and something I'll call hydrophobic.
OK. And covalent bonds are the strongest and these so-called hydrophobic bonds are the weakest bonds.
Ones that we are particularly interested in, actually, we're interested in all of them, but I would say covalent, hydrogen and hydrophobic bonds are particularly important.
And you will see when we talk about DNA, hydrogen bonds are very important.
So let's go through some pictures.
This is a table from your book, I'm not going to dwell on too much, but you really need to be familiar with this, you need to understand what is in this.
And it will indicate for you what the nature of the different bonds are.
OK. Covalent bonds, really, the two atoms become very closely joined together.
And, again, you're going to deal with this.
Let me just move onto, in your recitation.
Let me deal with two things that I want to dwell on a little bit.
The first is the notion of polar and nonpolar molecules.
OK?
So polar molecules have got differential charge distribution.
All right?
So we can add that actually to our list.
If you want to go back and add something to your list, its part and parcel of the notion of electronegativity, but let's also write polar versus nonpolar molecules.
In a polar molecule, the charges in the molecule are asymmetrically distributed.
Whereas, in the nonpolar molecule the charges are uniformly distributed and there is no net asymmetry of charge in that particular molecule.
And that will become important in a little bit.
Here is one that, again, you will dwell on in Section but I will introduce now.
Hydrogen bonds are between a hydrogen and an oxygen, and the hydrogen or something else, something that is more electronegative, the hydrogen, it's the hydrogen, this is not a covalent bond.
And the hydrogen interaction is integral to the structure of DNA, as we will discuss in this lecture and when we talk about molecular biology.
OK.
So that is a very cursory introduction to something that you will be dwelling on more in Section this week.
OK.
So I want to talk now about polymers.
And I want to talk how they form biologically.
And I'm going to tell you about two types of reactions that you should understand.
Condensation reaction which forms a polymer and a hydrolysis reaction which breaks a polymer.
So let's look at a diagram here that I've taken from your book.
Here are two monomers.
And you can see there is a hydroxyl group and a hydrogen group and a hydrogen.
And those two things react with one another and, with the use of energy, water is released, and the two monomers are joined to one another.
OK?
Again, you're going to have plenty of practice at this.
Don't try to draw this diagram here.
OK?
You can, we'll deal with it in a little bit.
Just look at it and understand what I'm saying at the time and it really is sufficient.
So, here, you've got the exclusion of water, the use of energy to get two monomers joined up, and then the process continues and you get a polymer being built up of more and more monomers.
Now, the converse of that is hydrolysis where you add water and also use energy to break up a polymer and release a monomer.
So either hydrolysis reaction or a condensation reaction are energy requiring reactions.
And this energy has to come from somewhere.
And we will spend a lecture talking about where the energy comes from.
So I want to draw the analogy, and it may seem hokey to you, but it's actually quite useful, a useful way for me to think about this, so I want to share it with you.
I want to draw the analogy to the cell as a factory.
And I want to begin by talking about the materials that the factory uses to synthesize whatever it is it's going to synthesize.
And then as we move through the biochemistry section I'll talk about the machinery within the cell that makes the synthesis take place.
So we're going to talk about the materials that are present in the cell.
And particularly we're going to talk about four groups of macromolecules, the big four from which everything else is built and that really are pivotal for life.
These are proteins.
And, actually, let me not list them in that order.
Let me list them in the order in which I'm going to talk about them.
I'm going to talk about lipids, carbohydrates, nucleic acids.
And I'll talk about those three today.
And then next time I'll talk about proteins.
So if you look at the dry mass of a cell it's very interesting.
Lipids comprise about 5% of the dry mass, carbohydrates about 25%, nucleic acids 10%, and proteins about 55%.
OK.
So that's what we're going to be talking about, these four groups of macromolecules that make up the components of the cell.
Now, two things that I want to point out as we talk about these macromolecules is that very often in biology the polymers have two properties that are very important.
One, they have polarity.
That means one end is different from the other end.
OK?
And the other thing that they very often have, not always but some classes in particular, is a linear order.
OK?
So that's, both of those properties, if you give this a moment's thought, you will see are very similar to sentences.
There's a beginning and an end of a sentence, and within a sentence the words come in a particular order.
And that gives you an information content.
That gives you a way to carry out or to store information.
So I'll point this out.
This is particularly true for the proteins and the nucleic acids.
It's also somewhat true for carbohydrates and less so for lipids.
So these two properties of biological polymers are fascinating and they are crucial for life to proceed.
All right.
So let's move onto one of the big four.
Lipids.
Lipids.
So lipids are a somewhat large and almost amorphous group of molecules in that I cannot give you a chemical formula, really, to say this is definitely a lipid and this is not a lipid.
They have one property that is crucial for defining them as lipids, and this is that they are hydrophobic -- -- or nonpolar.
And this property of being hydrophobic, so lipids in foods.
What are the lipids just, you know, in your food pyramid?
It's just to make sure we're all talking about the same stuff here.
Fats and oils, right.
Yes.
And you can come and get a fish later.
OK.
So fats and oils.
They're used for a couple of things.
One of the things about fats and oils is that they store a lot of energy.
So they serve as a long-term energy source for the cell.
And the other thing they do really has to do with their insulation properties.
And their insulation properties fall into multiple categories.
The most important one we touched on last time when we talked about the components of a cell.
We talked about the fact that the cell is a cell, and the organelles in the cell are organelles because they are membrane bound.
And I threw out at you the term lipid bilayer, and I'll throw it out at you again in a moment.
This is something made of lipids that insulates the cell from its environment or subcellular compartments from one another.
So part of the insulation property are membranes.
And, again, this term, I'll write on the board this time, lipid bilayer.
And the other aspect of insulation or the more classical things you think about, you don't get cold because you've got a layer of fat under your skin, and that stops yourself from losing heat.
So heat loss.
And, for example, birds use lipids to waterproof their feathers so that they can stand in the rain and not get wet.
OK.  Another aspect that I'll throw out at you also here is that lipids also make things called hormones, which are substances that control various aspects of body function.
I'll have a little more to say about them in a couple of minutes, and then will have quite a bit more to say about them when we talk about reproduction.
So let me see what is next here.
OK.
So let's talk about some typical lipids and some examples of lipids.
This is a diagram from your book, so you can go and look at it at your leisure, of a triglyceride.
Now, what I want you to take from this is that, so triglycerides, tri meaning three.
And the glyce- part is from glycerol, which is this molecule up here, onto which three long carbon chains are added.
And these long carbon chains are called fatty acids.
And that's important to know.
So a triglycerides is glycerol onto which three fatty acid chains have been added.
And it's the fatty acid chains that are hydrophobic.
So triglycerides are a very common type of lipid.
Now, after I just told you, ah, it's fixed.
After I just told you that lipids were all hydrophobic.
I'm now going to tell you that some lipids are amphipathic.
And they're amphipathic, so amphipathic means that they have both polar plus nonpolar components.
And one of the most important amphipathic molecules in the lipid field is a phospholipid.
So this is a molecule that's got these fatty acid long carbon chains on one end and on the other end has got a very polar phosphate group.
OK?
And here's phosphate with something called choline added onto it.
So this is phosphatidylcholine.
And phospholipids, and I spelt amphipathic incorrectly.
Phospholipids are essential for formation of the lipid bilayer membranes.
And the reason they are is because of this dual charged and noncharged portion.
So here is a diagram that I showed you last time that was not as well labeled.
This is a diagram of a lipid bilayer, so one of the membranes that surrounds your cells and your organelles.
Facing out onto the water of the cell are the hydrophilic heads of this phospholipids, the charged heads.
And then facing one another are two rows, OK, of these hydrophobic fatty acid tails.
So the bilayer is because you have one layer of these molecules with the hydrophilic heads facing the water, the hydrophobic tails pointing in.
And those interact with another layer where the hydrophilic heads are pointing to the water and the hydrophobic tails are pointing inwards.
The hydrophobic tails interact with one another and you get this very stable lipid bilayer.
OK.
It's very cool.
That's how things work, and they work very easily.
And you can get them to be synthesized in a test-tube very easily.
Here is another one.
Saturated versus unsaturated fats.
Unsat, OK, as an unsaturated.
In saturated fats, the fatty acid side chain contains no double bounds.
OK?
So it saturated four hydrogens.
In unsaturated fats, there are some double bonds that would allow one to add more hydrogens if they were broken.
OK?
So saturated versus unsaturated fats.
So saturated fats are the things you find in butter and in hard fats.
And they're supposed to be bad for you.
OK?
So saturated fats have been implicated in raising something called low density lipoproteins which are involved in various aspects of cell function that are believed to lead to heart disease.
But it turns out that there's more to it than that.
So if you've ever been really bored and you've read the ingredients on the box of cookies that you've eaten, you may see something that says partially hydrogenated vegetable oil.
Yeah?
OK.  Now, your vegetable oil is rather unsaturated.
OK?
It's got a lot of double bonds.
And one of the reasons that it is liquid is because these double bonds are there and you don't get the chains of the lipid packing very tightly and it's a liquid.
But you can take your vegetable oil with lots of unsaturated bonds and you can partially saturate it.
And you can get out of that two kinds of partially saturated fat, partially hydrogenated fat.
You can get this stuff called transunsaturated fat.
Now, if you look at this diagram, you will see these fatty acid side chains are in nice order here.
OK?
And they are because the hydrogens that are across from the double bonds are intrans from one another.
They're kind of diagonally opposite from one another.
And what that does is to allow these side chains to pack very tightly, and you get a very hard fat that way.
Somehow this partially hydrogenated stuff is very bad for you.
It raises your LDL, it reduces this stuff called high density lipoprotein, which is good for you, and it increases your heart disease risk.
Interestingly, you do not find these types of molecules in nature.
You only find them when you hydrogenate your vegetable oil and make your margarine or make the stuff that McDonald's cooks its French fries in, OK, and its chicken nuggets.
So a couple of years ago there was so much issue about this that McDonald's pledged to find a different vegetable oil in which to fry its fries.
And it's still searching, apparently, because there's some issue with taste being compromised as you go to healthier fats.
OK.  Now, in contrast to the transunsaturated fats there's this sysunsaturated fat also.
And if you look here, you can see that these side chains are kinked.
And the reason they're kinked is that the hydrogens across the double bond are on the same side of the molecule as one another.
That has the effect of not allowing the side chains to pack tightly.
So you get side chains floating all over the place, and that changes the energy of the fat so that it is a liquid rather than a solid.
And these are much better for you, these sysunsaturated fats.
And you find them in lots of vegetable oils and vegetable fats.
OK.
Here's another one.
Here's our joke for the day.
OK.  Steroids.
Steroids are a lipid.
They are derived from, OK, so let me just say for this unsaturated.
I mentioned sys and transunsaturated fats.
I'm going to mention steroid hormones.
There are a few of them.
Steroids are not all hormones, although cholesterol, we don't think of cholesterol as a hormone, although, actually, I think it really is.
Vitamin D is a vitamin that works together with other molecules.
Cortisol and testosterone are classically thought of as hormones.
Very small amounts of them will change body function in profound ways.
And we'll have more to say about testosterone later on in the course.
The androgens that lead to people who have large green muscles are a class that are related to testosterone, androsterone and so on, but they're all the structures with these four rings in this characteristic way are all characteristic of a steroid.
All right.
So that is all I have to say about lipids.
Let's move onto either number two here, carbohydrates.
So carbohydrates have as their -- OK.  Carbohydrates.
Also classically known as sugars.
And because of that there are a couple of functions of carbohydrates.
They are the major source of short-term energy.
If you need short-term energy you use carbohydrates.
They're also a carbon source, so we'll talk extensively about building up molecules.
And you need carbons to do that.
Carbohydrates serve as a carbon source for building other molecules.
And, as I'll mention briefly, they also can convey information.
Carbohydrates have, as their chemical principle, so we can find a chemical principle that unifies them, they all contain carbons that are joined to a hydrogen and to a hydroxyl, and obviously to other things as well.
So you can loosely give carbohydrates the formula of CH2O to the N.  Now, the monomer, now we can actually, with lipids it's very hard to talk about a monomer and a polymer.
In carbohydrates one can.
And one can talk about the monomer, which is called a monosaccharide which polymerizes to form a polysaccharide.
And they do so.
One can also get, let me just write it here, one can also get disaccharides where disaccharides are two and polysaccharides are really more than two.
They're usually more than ten or so, but for our purposes more than two is fine.
OK.
So in order to form a polymer of carbohydrates, one forms a glycosidic linkage or glycosidic bond.
So it's a covalent bond where one has -- Two carbohydrates or two monosaccharides interacting with one another through a condensation reaction, classical condensation reaction with the elimination of water to give you, excuse me, yeah, no.
I wrote this down incorrectly in my notes.
Apologies.
OK.  To give you a glycosidic linkage.
OK?
So this is a classical condensation reaction.
Now, polysaccharides or carbohydrates are interesting.
They can either be rings or they can be linear.
And they can also be modified.
So like the lipids where you could get phospholipids, carbohydrates can have various extra things added onto them that changes their properties.
Let's take a look at a few of them.
Here are some monosaccharides.
And the ones that are super important for you to know are these two down here.
OK?
And you should know the structures of these.
This is ribose and deoxyribose.
And the reason for this will become clear when I move onto the next class of macromolecules, DNA and RNA, because those sugars are part of DNA and RNA.
OK?
So these ribose and deoxyribose are five carbon rings.
You can see they're not quite a five, they're not a five carbon ring.
They are five carbon sugars.
And this is an isomer of one of the forms that this five carbon sugar can take.
OK.
So some of you may be lactose intolerant and might not be able, in fact, to enjoy digging in and enjoying a bowl of creamy delicious ice cream.
So, in fact, the reason, I didn't make that up, though the reason you cannot is because you need a specific protein, a specific enzyme that we'll talk about, not the specific enzyme, but we'll talk about the notion of enzymes in a couple of lectures that breaks a bond between the disaccharide that is lactose.
So lactose is a disaccharide that is a galactose joined together with a glucose.
OK.
So these are six carbon sugars.
And there's a bond between the two of them that you have to break in order to digest the lactose from milk.
And if you cannot then this lactose causes you digestive issues.
Here on the top is sucrose which is a dimer, a disaccharide, a glucose joined to a fructose.
OK.
So here's an interesting question.
I don't know when you were little, but when I was little I used to try and eat grass.
And, clearly, that doesn't work very well.
Humans cannot eat grass.
Whereas, cows and many animals can survive just very well on a diet of just plants.
And that is because we cannot deal with this carbohydrate in plants called cellulose.
Now, interestingly, we can digest other plant carbohydrates, particularly starch or amylose which is what most of the carbohydrates that we eat come from.
The reason for this is interesting, and it has to do with these glycosidic linkages.
OK.
So here we have a linkage between glucose, two glucoses that give something called maltose.
And maltose is part of the starch carbohydrate, or the amylose carbohydrates.
And what you should know here about these chains, these circles, and you should remember from old chemistry as if there is a bold part in the front of this ring then it is coming out of the board towards you.
OK.
So here you have these two glucoses linked such that a bond joining them is coming out of the board towards you.
Now, conversely, and this is called an alpha 1, 4 bond where the bonds coming out of the board towards you in cellobiose, which is a precursor for cellulose, the grass carbohydrate, the bond goes back, OK, back away from the rings.
And that's called a beta 1, 4 linkage.
And we can break this alpha 1, 4 linkage but not this beta 1, 4 linkage.
And I'll come back to this when we talk about enzymes.
We do not have the correct enzymes to do so.
And finally with respect to carbohydrates, I mentioned that they were sources of information.
One of the interesting sources of information from carbohydrates is in your blood cells.
So your blood group is as a result of adding specific chains of specific carbohydrates, specific monosaccharides, and a whole bunch of them, onto particular red blood cell surface proteins.
So if you're an A blood group, you have a series of specific carbohydrates added on in order to one of these cell surface proteins.
If you're a B blood group, you also have this carbohydrate chain but the specific carbohydrates, the specific monosaccharides are different.
All right.
So let's move onto the reason that you're all really sitting here.
So, really, the reason that you're all sitting here is because of this next group of macromolecules.
And I don't mean that that's the reason, you know, you're alive, that you exist in life.
I mean that's the reason you're talking 7.013.
Because nucleic acids are the thing that have, and the understanding of nucleic acids is what has revolutionized biology so that -- So that everyone in every discipline at MIT has some association, has some way of using the information from nucleic acids to their advantage and in their careers.
OK.
So nucleic acids, as I'll discuss, do serve as an energy source or as a way of storing energy, but most important is their function, and that's a super important function.
But the reason you're talking 7.
13 is not because of that.
The reason you are is because of the information content that nucleic acids are able to carry and to perpetuate from one generation to another.
So, really, what they are involved in is information storage and transfer.
All right.
So, again, as for carbohydrates, we can talk about monomers and we can talk about polymers.
The monomers for nucleic acids is a nucleotide.
Oh, OK.
Enough air play.
Watson and Crick get a lot of air play, so that's enough.
You can go and look at their picture at your leisure.
OK.
So I've taken this from your book and I'm going to draw some stuff on the board.
I want to point out that in the new edition of your book, you will find there is a mistake in the structure of your nucleotides.
That was true in the last edition.
I wrote to the senior author.
I said there is a problem with the sugar structure in the nucleic acids.
Would you please change it for the next edition?
And he said absolutely we will.
And the new edition came out and there we are.
OK?
So we'll have to, so we're going to deal with that, but I think we'll get you all free copies, or your successors free copies of the next edition of the book or something to make up for this.
But, actually, it's a good exercise for you because you can learn the proper structure of the sugar.
So what is a nucleotide?
OK. A nucleotide, as you gather, has a sugar associated with it.
It has, the sugar it has associated it with is either ribose or deoxyribose.
This sugar is joined to a phosphate group.
And the sugar is also joined to something called a base.
Now, the sugar is a five carbon ring.
I'll come back to this slide in a moment and we'll talk about the bottom part of it.
The sugar is a five carbon ring.
OK?
Here it is.
It's actually a four carbon ring with an oxygen and there's another carbon sticking out.
And what they've done in your book is to forget about this carbon up here and turn this oxygen here into a carbon.
So it's kind of weird.
Later in the book the structure is correct, but you should bear this in mind.
I've pointed this out in your PowerPoint handouts.
OK.
So what's really important, there are two things about nucleotides that are really important.
One is this phosphate and sugar because that is the way nucleotides polymerize with one another.
And the other is the base which is the part of the molecule that gives rise to the information.
Actually, let me just stop this and write the bases here.
So let me tell you the names of the bases.
And you need to know these.
There are four of them.
Five really.
There is adenine, guanine, cytosine and thymine.
And then there's also something that's an alternate to thymine called uracil.
And they are abbreviated by their first letters.
OK?
So A, G, C and T and U.
And you need to know these letters.
And you need to know what they stand for.
All right.
So let me, in the last two minutes, tell you about some properties.
Actually, let me tell you about the bond that forms the nucleic acid polymer.
And then I'll finish this off next lecture.
The bond that forms the polymer is called a phosphodiester bond.
And it forms by the reaction of the phosphate and the sugar with one another, or groups on the phosphate and the sugar with one another.
The base has got nothing to do with this polymerization.
So we have here a phosphate with some oxygens that's joined onto the sugar.
So I'm abbreviating the phosphate P, I'm abbreviating the sugar S, and I'm abbreviating the base B.  OK?
And the sugar is attached to a base.
And the sugar has, in one part of it, a reactive hydroxyl group.
And the phosphate has some reactive oxygen groups.
And let's not worry too much about bonds and double bonds and things for now.
OK.  And I'm not worrying about charges on oxygens and things either.
What happens, basically, is that there is an interaction between this hydroxyl group and the phosphate group, OK, or one of the oxygens on the phosphate group.
And you get out of this a linkage that goes -- OK.  Where you get between the sugar and the phosphate this phosphodiester linkage.
OK?
An OPO linkage that joins the two sugars to one another through a phosphate molecule.
This is your phosphodiester bond.
And I want to point out that the bases, which I will tell you next time, are the part of the molecule that carries the information, have got nothing to do with the polymerization of nucleic acids.
So I'm going to continue with this next time and briefly finish nucleic acids and then move onto the proteins.
So we're going to continue our discussion about genetics and patterns of inheritance.
We're going go relatively quickly compared to last time.
We've got a lot to get through today so I'm going to go relatively quickly.
And I printed a handout for you because I'm going to rely on the PowerPoint to a greater degree than I have before.
So I want you to have them in case you want to take notes on them.
OK?
And we will do that occasionally for particular lectures.
Now, importantly, we teach you about inheritance, not so much because we think you need to know about peas but rather the rules of inheritance which were derived from such studies but applied to many other studies.
Including in agricultural settings, in research settings in the laboratory involving fruit flies and mice, but more relevant to you to humans.
The rules that Mendel laid down apply to you and allow us to predict inheritance patterns based on visible traits but also disease traits.
And you'll see in next lecture that the information that we've given you helps us understand inheritance of various forms of genetic disease and predict, for example, frequencies within such pedigrees.
OK.
So, firstly, I wanted to review some terms which you need to know.
And I'll use a lot today.
The genotype which is the alleles present for a given gene in an individual.
OK?
The genes.
The alleles.
The term homozygous which means having two of the same alleles.
Heterozygous, having two different alleles for a given gene.
The phenotype which refers to the trait.
That's what the researcher or the individual observes.
The manifestation of the activity of those genes, of those alleles, the observed trait.
The first filial, or F1, generation which is the product of a cross of individuals who are homozygous for different alleles of a given trait.
I think this should say gene, not given, given.
Such that the F1 individual is heterozygous for that gene.
That is, has two different alleles.
If you cross two F1s together you generate the F2 generation.
The products of an F1 intercross.
And based on Mendel's First Laws, which I told you about last time and I'll reiterate in a second, the ratio of the genotypes that you see in the F2 generation is 1:2:1.
Homozygous for one allele, heterozygous and homozygous for the other allele; 1:2:1.
And the ratio of phenotypes is 3:1.
That assumes that there is a simple dominant recessive relationship between the two alleles.
We closed with that last time.
The dominant is the allele that determines the phenotype in a heterozygous individual.
Big S, little S you're smooth.
Big S is dominant over little S. Recessive allele is the opposite.
In a heterozygote it's not seen.
And, in fact, the only time you see the phenotype associated with the recessive allele is when the organism is homozygous for that recessive allele.
And finally there's a term codominance, which I'm going to mention now.
They don't really need to know, Claudette?
We bring it up in Section, so I'm going to mention it here.
And I refer you to the relevant parts of the book, but I'm not going to cover it in lecture but I'll show you where to go in the book to review that.
And that's relevant because Mendel's Rules apply as simple dominant recessive relationships, but they actually don't apply in terms of scoring them when you have a codominant situation.
So this is Mendel's First Law laid out for you.
When any individual produces gametes, germ cells, sperm, egg, the alleles, for a particular gene which control particular traits separates, so that each gamete receives only one of a pair of alleles present in the original individual.
And, as I said just a second ago, these experimental tests work well when there's a simple dominant recessive relationship between those two alleles.
There are exceptions to this in terms of codominance and incomplete dominance.
And I refer you to Figures 10.
3 and 10.14 in your book which show you examples from fruit flies as well as, sorry, from humans as well as from pea plants that illustrate both codominance and incomplete dominance.
Now, to emphasize the simplicity, really, of Mendel's First Law, I want to show you an example which also derives from your book using coins.
OK?
And it's just to emphasize the point about probability.
So what this says is that you have two, if you have two alleles of a given gene, big A, little A, big S, little S, you're going to hand off to your germ cells one or the other with equal probability.
And your mate will do the same such that the genotype of the offspring is the product of the probabilities of the genotype of the germ cells.
OK?
So I have two coins here, two pennies which have a head and a tail.
All right?
Head.
Tail.
So if I flipped this coin, what's the likelihood that this is a head?
1:2.
If I flip this coin, what's the likelihood that it's a head?
1:2.
If I put these together, like in fertilization, what's the likelihood that it's both heads?
1:4.
Very good.
One in two times one in two is one in four.
That's really the essence of Mendel's First Law that the alleles segregate randomly into the germ cells, and you can then figure out Punnett Squares and the phenotype based on the probabilities of the different genotypes that derive from that simple calculation.
OK.  We'll come back to a slightly more complicated version momentarily.
OK. Now, last time we talked about smooth and wrinkled peas.
I don't like this yellow chalk.
Is there white chalk here?
We talked about smooth and yellow peas, smooth and wrinkled peas.
We talked about genotypes, big S, big S, big S, little S, little S, little S.  Remember that?
We talked about phenotypes.
What's the phenotype of this pea?
Smooth.
What's the phenotype of this pea?
Smooth, because you know that big S is dominant over little S. And what's the phenotype of this pea?
Wrinkled.
Good.
What if I gave you, I handed to you on your dinner plate a smooth pea?
What's its genotype?
What's its genotype?
Who wants to venture out into the unknown here?
What's its genotype?
Come on.
Yes?
Thank you.
You don't know what its genotype is because it could either be big S, big S or big S, little S. It could either be big S, big S or big S, little S.  How are you going to tell?
How can you tell?
You can test it.
You could do what we call a Test Cross with a tester strain.
And a tester strain is homozygous for the recessive allele.
OK?
And based on this test we can figure out whether we're dealing with a big S, big S or big S, little S pea.
So we're going to do this using Punnett Squares to emphasize the use of Punnett Squares.
We have two possibilities.
Either our pea is big S, big S or our pea is big S, little S.  Either way we're going to cross it to a pea, really a pea plant, but a pea that's little S, little S.  OK?
Now, when I generate a Punnett Square what gametes do I put over here?
What alleles do I put over here?
Little S, little S. This plant only gives the little S allele to its gametes because that's all it has.
What alleles do I put over here?
Big S on both because this plant only gives big S to its gametes.
And so therefore all of the offspring will be heterozygous, big S, little S, big S, little S. And what will the phenotype be of all of the offspring?
Smooth.
100% smooth.
OK.  And how about over here?
Well, this side of the Punnett Square is the same.
This individual only gives little S to its gametes.
And how about over here?
Flip a coin, half of the time it's big S, half of the time it's little S. If this gamete meets up with this one then that individual is going to be big S, little S. The same thing over here.
With this gives little S, little S, this is the same thing.
What are the phenotypes here?
Half and half.
50% smooth, 50% wrinkled.
OK.  That's great.
Now -- What I want to now consider is a second trait.
And after we do this we're going to consider the inheritance of the two traits together, but let's just introduce the second trait now.
The gene is the Y gene.
There are, again, two alleles, big Y and little Y.
And the phenotypes associated with these genotypes are yellow and green.
OK?
Mendel does a lot of crossing, etc., creates pure breading strains, the parental strains.
And I'm going to tell you that their genotypes are homozygous big Y, and these produce yellow peas.
He also produces pure breeding strains that only ever produced green peas, and their genotype is little Y, little Y.
You cross these together to generate the F1.
What's the genotype with respect to Y in the F1?
Big Y, little Y.
They're all big Y, little Y.
And what's the phenotype of the F1?
You don't know.
That's very good, because I haven't told you which is dominant over which.
By convention, you might have guessed that big Y was dominant over little Y.
And if that were true, and it is, then what would be the phenotype?
Yellow.
And from that you can conclude that Y is dominant over little Y.  OK?
I always run out of space.
Do these boards get smaller every year?
I mean it's unbelievable to me.
Now, if we take these F1s and do an intercross to generate F2s, we're just reinforcing things we just learned here, in the F2 generation we're going to generate individuals who are Y, Y, big Y, big Y, big Y, little Y and little Y, little Y.
What will be the frequency of generating such individuals in this cross?
One quarter will be like this, a half will be like this, a quarter will be like this, and that would give a ration of 1:2:1.
The phenotypes associated with that will be yellow, yellow and green.
So the ratio of the phenotypes will be 3:1.
OK?
We just redid Mendel's First Law for you.
Now, what happens when we consider the two of these traits together, smoothness and color in the same cross?
Well, this led Mendel to propose his Second Law which is illustrated here, which is that alleles of different genes, the S gene and the Y gene, assort independently of one another in producing gametes.
What happens for one does not effect what happens to the other.
And this is true for genes that lie on separate chromosomes.
Although it's not necessarily true for genes that lie on the same chromosome.
I make that point here, and I'll make it again later because I don't want you to come to the conclusion that two genes always assort from one another independently.
They do, and Mendel's Second Law is based on that, in some instances, but they don't always.
So to illustrate that point we turn to the coins again.
I always also lose the coins.
OK.
So now we have our pennies once again, heads and tails.
And dimes, heads and tails, yeah?
OK.
So I'm going to put a penny and a dime in each pocket.
What's the frequency of generating a penny in the head configuration and a dime in the head configuration together?
Well, let's do it differently.
What's the frequency of generating a head, a penny in the head configuration here?
1:2.
What's the frequency of getting a penny in the head and dime in the head?
One over four.
It's one in two times one in two.
And what's the frequency of getting penny head, dime head in this hand?
One over four.
So what's the frequency of getting penny head, penny head, dime head, dime head in the zygote?
1:16.
Again, simple probabilities.
OK?
And for Mendel's Second Law in its simplest form, it determines what you would expect to see in such crosses.
And, again, we'll come back to a twist on that in a moment.
Now, in biological terms it looks like this.
And this derives from your book, so you can go and look at it.
It's adapted from Purves 10. .
And there's a nice little animation that goes along with that.
But we're going to go through this because there's some subtly here that I want you to understand.
So here is the parent.
It's a diploid.
It has the two chromosomes that are relevant that carry the S gene and the Y gene.
And the parent is heterozygous for both.
It's a double heterozygote.
Big S, little S, big Y, little Y. OK?
Now, when this individual undergoes meiosis, it can generate four possible haploid gametes.
And those have the genotypes of all four combinations.
It can be big S, little Y, big S, big Y, little S, little Y, little S, big Y.
All four possibilities.
And based on the simple laws of probability that we just went through with the coins, those are generated in equal amounts amongst the gametes.
OK?
They're generated in equal amounts amongst the gametes.
Why are they?
Well, let me just show you another slide that illustrates it in slightly more detail.
When these guys undergo the first, undergo duplication to generate the two chromatids for each chromosomes, enter prophase of meiosis one, the homologs pair, as you recall.
The two chromosomes don't have anything to do with one another.
The homologs pair but the two chromosomes don't have anything to do with one another.
They're independent of each other.
Now, if we consider this chromosome, the one that carries the S gene, it will line up on the metaphase plate in an orientation which is random.
In this particular example, the big S allele is pointing down and the little S allele is pointing up.
OK?
Based on that, let's fix that.
Based on that there is one of two possible alignments for the chromosome that carries the other gene.
In our example it's the Y gene.
One orientation would have the big Y carrying chromosome pointing up, but it's just as likely that that chromosome could end up the other way around such that the big Y carrying chromosome is pointing down.
And it's based on the fact that the two chromosomes are independent of one another when they choose which side of the metaphase plate to go to that gives us this random assortment in meiosis and the development of Mendel's Second Law.
So if you carry this through then you generate these gametes in a 2:2:2:2 or 1:1:1:1 ratio.
OK?
That's the most important thing.
If you understand that then you can work it out as to which genotypes you get and which phenotypes you get using Punnett Squares.
OK?
But it's important that you understand why it is that you get random assortment of the alleles in such a cross.
Is that clear to everybody?
It's probably not clear to the guy reading the newspaper.
Oh, the woman reading the newspaper.
Usually it's a guy reading the newspaper but not today.
Is this clear?
Are there any questions?
OK. As I said, you can work through this with Punnett Squares.
I decided not to do it on the board because it takes a lot of time, but the book shows you, in this specific example, what you should expect.
Here are the two parentals.
They're homozygous for each of the alleles, big S, big S, big Y, big Y, little S, little S, little Y, little Y.
In the F1 generation you create the double heterozygote, big S, little S, big Y, little Y, and then that generates the four gametes in equal proportion, as we just discussed.
If you cross two of these F1s together then you can create a Punnett Square which now has 16 squares instead of just four which allows us to consider the products of all four of these gametes crossed to all four of these gametes.
So we get various combinations of genotypes and we get various combinations of phenotypes.
And you can calculate what those are based on looking at this.
There's actually another way of doing it, which I find to be simpler, which is illustrated here.
Again, because the traits and the alleles are segregating from one another independently you can think about this as two Punnett Squares separately and figure out the consequences of those two traits by simply multiplying between then.
So what I mean by that is if you consider the S allele in isolation then there are the familiar 1:2:1 ratio of the three different genotypes.
And that's true also of the Y genotypes, 1:2:1 of the three possible genotypes.
In total there are nine possible genotypes, but when you separate it from S and Y it's 3:3.
Likewise with respect to the phenotypes, three-quarters of these guys will be smooth, one-quarter will be wrinkled.
And over here three-quarters will be yellow, one-quarter will be green.
If I want to figure out what the frequency is of any compound genotype, I just have to multiply across.
So if I want to know the frequency of big S, big S, big Y, big Y, and it's just the product of a quarter times a quarter or a sixteenth, just like the coins example by the way.
If I want to know the frequency of being big S, little S, big Y, little Y, then it's a half times a half or four-sixteenths, sorry.
A half times a half or a quarter which equals four sixteenths.
And the other genotypes comprise the remaining sixteenths that we see, another 11.
And likewise with respect to phenotypes.
If you want to know the frequency of smooth yellow offspring you just multiple across, three-quarters times three-quarters gives you nine-sixteenth.
Of wrinkled green ones it's a quarter times a quarter or one-sixteenth.
So, again, when you're asked to do Punnett Squares for two traits that segregate independently, you might think about doing it using separate Punnett Squares.
OK.
So we've talked about Mendel's Laws.
We're wrapping up Mendel's Laws here.
The first law is independent segregation of the two alleles.
The second law with respect to different genes, they segregate independently from each other.
OK?
Now, as I said, this is true for genes that are on different chromosomes or actually genes that are far away on the same chromosome, but it's not always true.
And to emphasize that point we're going to return to our coin friends.
So let me just remind you, let me just remind you what we saw.
So this doesn't apply to genes that are linked together on the same chromosome.
Let's remind you what we determined for genes that are unlinked, the dime and the penny.
We said that the frequency of getting in my left hand head, head and in my right hand head, head, and together was 1:4 and 1:4 and together was 1:16.
That was the situation when they were independent of one another.
But let's consider the situation when I've taped them together.
Now I have a penny and a quarter which are taped together.
I have fixed the orientation of these two coins with respect to one another.
So you can see the heads are on the same side, the heads are on the same side, yeah?
OK.  What's the likelihood that this is going to have the penny in the head configuration?
1:2.
And if it is what's the likelihood that the quarter is going to be in the head configuration?
One.
So what's the likelihood that the coin is going to have head, head when I turned it up?
1:2.
Not like this example.
1:2.
And so what's the likelihood that it's going to be head, head in this configuration, in this hand?
1:2.
So what's the likelihood that it's going to be head, head, head, head?
1:4.
So the frequencies don't work out, according to Mendel's Rules, if the two genes are linked, the two genes are linked together.
So thank you for your help.
It pays to sit in the front row every once in a while.
So this is what we just talked about.
In my left hand the frequency of head, head was one-half, in my right hand frequency of head, head was one-half, so the product was a half times a half or a quarter.
OK?
Now, if we want to think about that is sort of genetic terms, let's think about chromosomes.
And genes can be linked on chromosomes like those two coins can be linked together by tape.
So here are two genes, the penny gene and the quarter gene, two alleles, the head allele and the tail allele, and they're present on two chromosomes in a heterozygous individual.
And they're in this configuration such that the head allele of the quarter is next to the head allele of the penny.
The tail allele of the quarter is next to the tail allele of the penny.
When this goes through meiosis the alleles stick together.
The head alleles stick together in the gametes.
The tail alleles stick together in the gametes.
It's just like Mary Had a Little Lamb.
Remember that one?
I'm getting curious looks.
Mary had a little lamb.
Its fleece was white as snow.
Everywhere that Mary went the lamb was sure to go.
A long way to go for that one, but anyway.
Everywhere that Mary went the lamb was sure to go.
They stay linked.
OK?
And importantly you don't find, or you almost never find this situation where the quarter in the head, sorry, the penny in the head configuration went with the quarter in the tail configuration.
You don't find those, or you almost never, and this is the point of the final phase of the lecture.
OK.
Very good.
So let's look at this with respect to peas.
Why not?
We're going to talk about now linked genes in peas.
We're going to talk about two genes.
We're going to have our familiar Y gene which has its two alleles.
And now we're going to talk about a new gene, the D gene, which likewise has two alleles and two phenotypes which are dense peas and light peas.
OK?
Dense peas and light peas.
And it turns out that these two genes are on the same chromosome and very close to one another.
So if you think about the chromosomes in a heterozygous individual, double heterozygous individual they would look like that.
If we think about the chromosomes in a tester strain, which is homozygous for the recessive alleles for both.
What's going to happen to the offspring?
What?
What?
I'm sorry.
First of all, this phenotype is yellow and dense.
Big D is dominant over little D. This phenotype is green and light.
OK?
What do the gametes look like in this Punnett Square?
And why am I drawing four boxes, only four boxes anyway?
Didn't I tell you a little while ago that if you have two alleles, two genes you have to draw a Punnett Square with 16 boxes?
Why am I drawing only four?
Because they go together.
They're linked.
So what do the alleles look like?
Well, this parent is going to give either big Y, big D or little Y, little D.  And this parent is always only ever going to give little Y, little D.  OK?
So therefore the genotype follows, sorry.
And the phenotype.
What am I going to get when I deconvolute this?
What am I going to get?
The phenotype in that cross, 50% will be yellow dense.
And 50% will be what?
Green and light.
The very same two phenotypes that I got in the parents.
These are the parental phenotypes.
And in a test cross like this, of two genes that are very tightly linked, you would always regenerate the parental phenotypes.
They would never separate from one another.
You'd always only ever generate the parental --
OK.  We are wrapping up our segment on genetics today, Genetics 4.
We're going to talk about human disease genetics, modes of inheritance for human diseased genes.
And then we'll transition next time into molecular biology, which is the next blank square here.
And then recombinant DNA cell biology and beyond.
I wanted to start today by clearing up a couple of things.
Firstly, the term F1, in the last lecture I gave you a list of terms, and I used this definition of the first filial generation, the F1 generation as the product of crosses between two homozygous individuals, homozygous for different alleles such that the offspring were always heterozygous, big A, little A at that particular gene.
Well, it turns out that some people use the term F1 to refer to the offspring of any cross.
So you have the parents and you have the F1s regardless of the genotype of the parents.
So that's a looser definition of the term F1.
And it's been used in Section and so on, so I didn't want to confuse you.
The strict definition is the one I told you, but don't get hung up on that.
We're always going to give you the relevant genotypes so you'll be able to figure out what the genotypes of the offspring are.
Also, near the end of the last lecture, I talked about crosses involving linked genes.
If you remember the Y and D gene controlling color and density, I think, of peas.
And I told you that they were linked, and we carried out a test cross.
And we then worked out what the percentages of the different phenotypes and genotypes would be.
Well, in that example, I assumed that the Y allele and the D allele were together on the same chromosome.
But importantly if you're just given the genotype as it's written here, you actually cannot know that.
The big Y might be on the opposite chromosome from the big D. So to avoid confusion about that sort of thing, in future examples and in problems, problems, that is test questions, we'll always show you the chromosomes so you'll know that the genes are together on the same chromosome or on opposite.
Yeah.
And in the absence of this, if the question is to tell us what the alleles must look like and they give you the phenotype, you have to conclude the genotype.
If the question is tell us what the alleles must look like then we're not going to give you the answer, but in other situations where there's ambiguity about what the alignment might be, we'll tell you what that is.
OK?
We also had a question from a student, actually an emailed question, those are also welcome, email question from a student.
It was good question because in our discussion of dominance and recessiveness, we really haven't dealt with the molecular nature of that.
And that was this individual's interest.
Why is an allele dominant over another one?
Why is a trait dominant over another one?
And I gave an example to him, which I will give to you, which I think covers most of the examples that we've been discussing in class.
And it also is an opportunity for me to reinforce the notions related to chromosomes, genes, DNA and proteins, because apparently there are some of you who are still a little fuzzy on those relationships.
So let's imagine a gene which is present on a chromosome.
Chromosome shown here.
Gene shown here.
And it's the S gene that we've talked about before that controls smooth versus wrinkled pea texture.
So here's the S gene.
It's made up of DNA.
And based on the sequence of that DNA within the gene, therein lies the information to produce a protein.
And we're going to call this protein, in our example, this is a hypothetical example, the starch synthase protein.
It's an enzyme that controls a reaction, that catalyzes a reaction from some sugar substrate into some starch.
And if you make enough of that starch product then you have a smooth shape.
OK?
This enzyme controls shape because it produces this product which is involved in the shape of a pea.
OK?
So this is the normal situation.
The big S allele produces a functional starch synthase enzyme which produces enough product to give the pea its smooth shape.
In this example, the little S allele, which is the recessive one, has a mutation within the coding sequence of the gene.
We won't talk about the nature of that mutation just yet, what it is, why it causes what it does because you're going to get that in the next segment.
Just suffice to say that it's a mutation, an alteration of the DNA such that the protein that's produced from this allele is nonfunctional.
You might be able to see I've inserted a little X there.
It's actually quite little and invisible on your handout, but you might want to just circle it if you look carefully.
There's a little X there which is the result of this mutation.
And that X causes the protein to be nonfunctional.
The enzyme now does not catalyze the production of this starch product, so no starch product is produced.
If you don't make the starch product you don't have a smooth shape, you have a wrinkled shape.
OK?
If the genotype of the pea is little S, little S then none of this enzyme is produced such that none of the starch product is produced such that the pea is wrinkled.
OK?
Now what happens if you're big S, little S, the heterozygote?
Well, for many biochemical reactions, having just one copy of the gene that encodes the functional enzyme is enough.
For many biochemical reactions that's true.
And so in a situation where big S is dominant over little S, we assume that having one copy of big S makes enough of the starch synthase protein to make enough of that starch product to give the pea its smooth shape.
OK?
So that's a simple example of why big S is dominant over little S, why you only see the phenotype associated with the little S allele, the wrinkled phenotype when you have two copies of the little S allele.
OK?
So I hope that helps clarify the situation for you and actually is useful as we talk about human disease genes as well.
So this is a slide from your book which allows us to transition from concepts of inheritance to real-life stuff.
Genes that regulate how your body functions normally or in response to various environmental stresses.
We're now in an era where we can relatively easily figure out whether a disease has a genetic component by looking at families that might have that disease, and based on that information and using mapping techniques like I described to you last time, we can isolate where that gene might lie on all of your chromosomes.
And then using molecular techniques, which we'll talk about in future lectures, we can actually isolate that gene, determine its sequence, and based on that produce lots of various valuable things like better ways to diagnose the disease, better ways to understand how the disease process takes place such that we can then perhaps prevent the disease from occurring in the first place or treating it more effectively by replacing the gene with a new copy or producing a drug that can replace the gene in other ways.
So this is what we're after.
So we need to understand diseased genes and how they behave in such affected families.
So there are a number of diseases which have a genetic component.
And these diseases have various modes of inheritance.
Some of them are autosomal dominant.
The diseased gene is dominant over the wild type gene, and I'll give you examples of that.
And the term autosomal means that it's not sex linked.
That is the disease gene is carried on chromosomes 1 through 22, one of the those chromosomes, not on the X or the Y.
It doesn't matter if your father, whether the disease gene is coming from a male or a female, passed on to a daughter or a son.
There's no sex linkage for these autosomal dominant diseases.
There another class of genes, disease genes which follow autosomal recessive inheritance patterns.
Here the disease gene is recessive to wild type.
You only see the disease phenotype when you're homozygous for the mutant allele.
And, again, autosomal because it's not sex linked.
There are X linked diseases which are dominant.
In this case, the disease gene is on the X chromosome.
There are also X linked diseases that are recessive.
There are very few, but for the sake of completeness, Y linked diseases.
But there are so few that we're actually not going to talk about them at all in this class.
And, finally, there's a class of diseases that we're also not going to talk about but get inherited not from the genes that are in the nucleus of the cell along your chromosomes but rather get inherited from the mitochondria.
And since we haven't talked about mitochondria with you really at all we're not going to expect you to know about those diseases, but just have in the back of your minds that those are relatively important.
Autosomal dominant diseases are not terribly common.
There are about 200 known.
Autosomal recessive diseases are actually much more common, though not very frequent in the population.
There are about 2, 00 of these known.
And together I would estimate there are about 25 sex linked diseases.
So we've used the term autosomal dominant, autosomal recessive, sex linked.
What does this really look like in terms of the genes and the alleles?
Well, just as in the case of peas, a dominant disease allele will cause disease regardless of the nature of the other allele.
It's dominant over the normal common version of the gene.
So if we call this allele the disease allele of some gene -- -- and this allele the commonly found on in the population, and we refer to these often as the wild type -- -- in an individual who has this genotype for a dominant disease gene, will they develop disease?
Yes, because the disease allele is dominant over the normal copy of the gene.
So these individuals will develop disease.
For recessive disease alleles you need to have both copies, both alleles be mutant in order to manifest the disease.
So here's another gene, which we'll call the H gene, which has a disease allele.
This individual is homozygous for the disease allele.
Will this individual develop disease?
There might be other people in the population who are heterozygous for that allele and have a wild type allele on the other chromosome.
Will they develop disease?
No.
These individuals are called heterozygous carriers.
They carry the disease allele, but because they also carry a wild type allele they don't develop the disease.
They're normal.
They're normal in the sense of their phenotype.
They have an abnormal genotype in the sense that they have a disease allele, but they're normal in the sense of their genotype, phenotype.
Now, for X linked genes, sorry, before I do that let me -- Let me review for you sex determination.
We talked about this briefly.
In order to understand the inheritance pattern for X linked genes you need to remember this.
That males of course have one X chromosome and one Y chromosome.
Females have two X chromosomes.
If you think of a Punnett Square related to the inheritance of chromosomes males will pass along an X or a Y at 50% each, females will pass along one of their two Xs.
Will produce an equal number of males and females from such crosses.
But it's important to think about where the X chromosomes come from, specifically males and the Y chromosomes.
Males transmit their only Y, their only Y to all of their sons.
If you're the son then you've gotten your father's only Y.
Males transmit their only X chromosome to all of their daughters.
If you're the daughter, when you're the daughter of a male, you have inherited his X chromosome.
OK?
Females transmit either X to each daughter or son.
In the case of the female X, you can be a male who got this X chromosome or a male who got this X chromosome.
You can be a female who got this X chromosome or a female who got this X chromosome.
And that's important for understanding the disease genetics.
So let's imagine a dominant disease involving the X chromosome.
A gene, we'll call it Q.
Males only have one X chromosome.
They only have one X chromosome, therefore if they carry the disease gene they're going to get disease.
Females have two X chromosomes.
They might carry the disease allele on one but a normal copy on the other.
In this scenario, will they develop disease?
Yes, because it's a dominant disease allele.
For recessive X linked disease, once again, males only have one X.
Are they going to get disease?
Yes.
Females have two X chromosomes.
If they carry a disease allele on one they are likely to carry a wild type copy on the other.
Are they going to get disease?
No.
They're going to be heterozygous carriers.
And you'll see how this plays out towards the end of the lecture.
OK.
So let's look at some examples.
We're going to start with recessive diseases first, recessive autosomal diseases first.
And one you've already seen before, and that's PKU, phenylketonuria.
If you recall, this is a disease which is associated with failure to make an enzyme that converts phenylalanine to tyrosine.
It's an enzyme called phenylalanine hydroxylase.
If you don't have that enzyme then you produce too much of this byproduct, phenylpyruvic acid which is toxic and leads to brain damage and retardation.
This can be avoided, the disease can be avoided by limiting your amount of dietary phenylalanine.
And that's why such patients are not supposed to have Equal, as we discussed before.
The disease, PKU, is autosomal recessive.
The incidence is rare, about one in 10,000 to 15,000 individuals have PKU.
As I talked to you about last time, the mutant allele has a single amino acid chain change in the active site of this enzyme which changes an arginine residue to a tryptophan residue.
And that makes the enzyme inactive.
This is an autosomal recessive disease.
So if you have one mutant allele and one normal copy of the phenylalanine hydroxylase gene, do you have the disease?
No.
You would be a carrier but you would not have the disease symptoms.
Because again, as in the other hypothetical example, most of the time, if you have a single normal copy of an enzyme that carries out some biochemical reaction that's enough and your phenotype is normal.
Here's another common example, cystic fibrosis.
This is actually a more common disease.
It affects about one in 2, 00 to 3,000 individuals.
That's not an insignificant number.
So there are a lot of people with cystic fibrosis in this country.
Again, it's autosomal recessive.
Patients present with the disease at birth.
They have problems both in their intestinal system as well as in their lungs.
It tends to be the lung symptoms that ultimately kills the patient due to inability to breathe properly, but actually more importantly persistent infections which ultimately are so severe that they lead to death.
And these patients tend to die within the first two to three decades of life.
The gene encoded by the disease, the gene responsible for this disease is a chloride channel.
When it functions properly it allows chloride ions to move through, and this achieves a proper water balance inside the cells.
When the disease allele is present in both copies, this channel is not formed properly, and therefore the water balance in such cells is not correct and it leads to the build up of this kind of mucousy sticky substance both in the lungs and in the intestines.
These individuals, as you can see, have a very hard time breathing, and they have to get this mucous cleared from their lungs periodically.
And they also often have to have respirators to allow them to breath.
OK.
So that's some examples of autosomal recessive diseases.
Let's think about the genetics.
And we'll talk about CF specifically.
Let's imagine the disease allele of CF.
We'll call it CFD.
There are actually many different disease alleles for this disease, but we'll just lump them together.
Here are two individuals with this genotype.
Do these individuals manifest the disease?
Yes or no?
Do they manifest the disease?
No, because this is an autosomal recessive disease and they have a wild type copy.
But if we think about the offspring that they could produce, this individual will produce gametes that carry the D allele and the wild type allele, likewise this individual, the D allele and the wild type allele, such that the offspring will either be DD, wild type D, wild type, wild type or wild type, D.  Right?
So you'll have one wild type, wild type individual produced out of such a cross.
And this individual now has no disease allele present.
This individual is both genotypically and phenotypically normal.
No more issues about cystic fibrosis for this person or his or her descendants.
There will be two individuals who are heterozygous, and these individuals are carriers.
They don't have the disease but they do have a mutant allele.
So depending on whom they marry, I shouldn't generalize, depending on whom they have children with they may or may not have to worry about the fact that they have a mutant CF allele.
And one, on average, out of four will have two disease alleles and develop CF.
OK?
So we're going to now think about these diseases in inheritance looking at pedigrees, which I think are probably familiar to most of you.
They're described in your book as well.
But I just want to make sure you understand what the symbols mean before we get into it.
Females are represented as circles.
Males are represented as squares.
Seems appropriate.
Lines between them represent a mating, they have mated.
And the offspring are represented below.
If the symbol is left open then the individual is normal, genotypically and phenotypically.
If the individual has a shaded symbol they are affected, they show disease symptoms.
And if the symbol is partially shaded, I usually fill in half the symbol, the book will sometimes put a little circle inside the circle then they are carriers.
They're heterozygous carriers.
OK?
So let's look and think about this example up here.
We talked about two individuals who were both heterozygous for the disease allele, so they were carriers.
Their genotype was CFD, CF wild type.
They mated.
They had four children.
There were four possible genotypes, rather three possible genotypes.
And, in fact, we observe all three genotypes in this generation.
We have one individual over here who doesn't have her symbol filled.
We have another whose symbol is fully filled.
And then we have two whose symbols are partially filled.
What's the genotype of this individual?
Wild type, wild type.
This one?
D, D.  This one?
D, wild type.
And this one?
Same thing.
OK?
So that's what pedigrees look like.
Let me show you a larger pedigree and give you some of the rules that apply to autosomal recessive diseases.
So here's a large pedigree involving an autosomal recessive gene, disease gene.
Again, the two parents are heterozygous.
They had many, many offspring all here.
Roughly a quarter of them will carry both copies of the disease gene and develop disease.
Among the rest there will be unaffected individuals but of two classes, ones that don't have the disease gene at all and ones who are heterozygous carriers.
Two-thirds of those unaffected offspring are themselves carriers, two-thirds.
Two-thirds of the unaffected offspring are themselves carriers.
Among those half of the offspring of a carrier are themselves carriers, as shown here.
So if this individual came to you at a genetics clinic and wanted to know what is my likelihood of carrying the D mutant allele, you'd be able to say, without knowing the genotype of his mother you would be able to say that it's half of a half or a quarter.
OK?
Now, every once in a while two affected individuals, two homozygotes have children, as shown here.
And you can see in that scenario all of the offspring have disease because the only alleles that are present are the mutant alleles.
That has to be true because these are both affected, and therefore all of the offspring will also only inherit mutant alleles and develop the disease themselves.
OK?
And importantly because the disease gene is coming in on chromosomes 1 to 22, one of those and not the X or the Y, there is no gender associations here.
Mothers can pass the disease to their daughters and their sons.
Fathers can pass the disease to their daughters and their sons.
There are no gender associations with this scenario.
OK.  Disease alleles are present in different frequencies in the population.
Some of them are extremely rare, but some of them are actually rather common.
So we're talking about the allele frequency, the percentage of alleles among all of our chromosomes that are the disease type.
For PKU it depends a lot on where you're from.
For example, in Turkey it's actually rather common, one in 206,000 alleles of this gene are mutant in this population, but in Japan it's much less common.
One in 220,000 Japanese carry this allele.
Exactly why this is we're not entirely sure, although I'll give you some speculation in a little while about what controls the frequency of these actually rather deleterious alleles.
CF is fairly frequent.
One in 25 alleles are CF.
One in 25.
There are as maybe 200 of you in this room, so that's some number of mutant alleles among you.
It's a fairly high number.
You're sitting there carrying a mutant copy of the CF allele.
This is actually relevant to European descendancy.
I'm not entirely sure whether it applies to all descendencies but let's just say that it's roughly that number.
And another disease that you might have heard of Tay-Sachs disease in the Ashkenazi Jewish population -- -- is also quite common.
One in 25.
It's so common, in fact, that in certain parts of the world individuals undergo genetic testing before they decide whom to date because they want to avoid the risk that they're going to date somebody else who carries such a mutation.
This is actually a very horrible disease.
I shouldn't joke about it.
So they want to avoid ever having to face the problem of having a child who has Tay-Sachs disease.
So they undergo, in a sense, pre-marriage counseling to figure out their genotype to decide whether or not to date.
And you can also imagine with similar tests we could figure out whether or not individuals carry mutations in these genes to let them know what their risks of developing disease are or whether or not to maintain a pregnancy of a child who may or may not be affected and so on.
So they're very serious societal implications for these kinds of disease alleles.
Now, some of the alleles are very rare.
Here's one example in the Japanese population of PKU.
The likelihood that two individuals would randomly get together who had mutant alleles of a phenylalanine hydroxylase gene and have a child is exceedingly small.
And so in these situations, where the allele frequency is extremely rare, when you find an affected individual it's almost always a sure sign of inbreeding.
Consanguinity is the term used in genetics where cousins or other relatives marry and have children.
And since they are related and have a higher allele frequency within their families of a particular allele then the likelihood that they'll have an offspring who has two copies of the allele is much higher.
And so when you see a pattern such as this, it's a fairly clear indication that you're dealing with an autosomal recessive disease involving consanguineous mating.
Here are two cousins who are producing offspring both of whom are affected.
And when you see a pattern like this you can actually figure out, or at least figure out pretty well who were the carriers in this scenario, who were the carriers in this family.
Since both, since the children have two mutant copies then both of their parents must be heterozygous carriers.
OK?
That goes without saying.
They're heterozygous.
Since this individual is heterozygous and the allele was passed on from their grandparents then his mother must also carry the allele inherited from one of the two grandparents.
Likewise this woman's father must carry the mutant allele.
And in this generation we actually don't know whether it's the male or the female who carries the mutation.
It could be either.
And it's been passed along to both sides of this family tree and reduced to homozygocity in this generation.
OK?
You can also begin to figure out what the likelihood of other members of the family tree is being heterozygous.
So this individual here has a one in two chance.
One of these two parents is definitely a heterozygote, and so the likelihood that he's a heterozygote is one in two.
And among his children you could say they have a one in four chance.
If he has a one in two chance then there's a one in two chance that he'll pass it onto them, so overall there's a one in four chance that they will be themselves carriers.
And this, again, this kind of genetic testing is done frequently to figure out what your relative risk of developing a particular disease are.
Now, as I said, for other disease alleles like CF and Tay-Sachs, the allele frequency is actually strikingly high.
And yet if you're homozygous for these mutations you're dead.
Not necessarily right away but not for very long.
It clearly reduces your reproductive fitness.
And so why would these alleles be present in our population at all?
Why wouldn't they be removed through natural selection?
We're not going to get into this in great detail, but there are theories out there and some evidence to support them that there might be actually an advantage in certain circumstances for being heterozygous wild type over mutant.
And there are theories both related to Tay-Sachs disease, sickle cell disease and cystic fibrosis such that if you are heterozygous you actually survive better in the face of certain pathogenic exposure than do people who are wild type for both alleles.
And that's the argument for why these alleles actually built up, at least in the past, over the evolution of our species.
OK.  Let's transition now to autosomal dominant diseases.
This is a famous example, Huntington's disease, otherwise called Huntington's chorea.
Relatively rare.
About one in 10, 00 to 25,000 individuals affected.
It exhibits an autosomal dominant pattern of inheritance, as you'll see in a moment.
The age of onset is about 35 to 40 years of age.
These individuals actually are totally normal for the first three to four decades.
You wouldn't know that they had a disease.
But starting at that time they begin to develop symptoms which involve both effects on their personalities but more importantly effects on their movements.
That's what you first begin to see.
That's what chorea means.
It's sort of this dance-like movement.
And then that gets much, much more severe and stereotypical.
And eventually, in addition to that, there's death of cells in the brain.
And the combination of these affects leads to the death of the individual by about the fourth or fifth decade of life.
This is actually a movie of an individual who has Huntington's disease.
He's being told to hold his arms straight out, but because of this neurogenerative process that's taking place within his brain and also other parts of his nervous system he's unable to do so.
And this is sort of the stereotypical presentation of Huntington's chorea.
They have this wave-like movement and also their limbs get stuck in particular postures, as you can see this individual here.
At the molecular level the problem in these individuals is that their brain cells are dying, particular ones, actually, within a particular region of the brain surrounding this ventricle here.
And this is a normal space in a normal brain.
And there are various sets of neurons on either side of those ventricles.
And in HD patients those cells are progressively lost over time, and when those cells are lost you lose motor control and you also develop rather severe dementia.
And the reason that those cells are being lost is that the mutant protein, the mutant form of this protein called Huntington, the mutant form builds up inside of those cells and aggregates and causes the cells to die.
OK?
And this is why this is an autosomal dominant disease.
If you have the disease allele then you will get the aggregation of the protein and you will develop the disease.
So let's take another example of a cross between an individual who is normal, two normal copies of HD and an individual who has a disease allele and a wild type allele who is heterozygous.
Is this individual diseased?
He either is already or he will be in the case of HD.
It's an autosomal dominant disease.
If you have the disease allele you will develop disease.
If we look at a Punnett Square, this individual will always transmit the wild type allele, this individual will transmit the mutant allele half the time the wild type allele of the other.
These individuals will be wild type D, wild type D, wild type, wild type, wild type, wild type.
So you'll get half diseased, half normal.
OK?
A rather different picture than we saw previously.
And importantly, as I mentioned, the presence, the mere presence of the D allele leads to the production of this toxic protein.
And ultimately brain damage.
And that's why it's dominant.
OK.
So let's look at a pedigree of an autosomal dominant disorder.
Here's an affected individual, a female who marries, who has children with an unaffected male.
They have four children.
On average half of them will inherit the defective allele and therefore develop disease.
Half of the offspring of an affected parent will be affected.
Importantly, the unaffected offspring of an affected parent have unaffected offspring.
If you are normal, you do not inherit the disease allele, you're scot-free.
Your children will no longer have to worry about this disease.
But in this case, in this individual, again roughly half, in this example two-thirds of the offspring do develop the disease.
This is another really important case of genetic testing.
This guy might have wanted to know that he was going to develop Huntington's disease in order to decide whether to have children in the first place.
This individual here likewise might want to know before the disease actually manifested itself what would happen in order to make lifestyle decisions.
Am I going to quit work and have a good time for the next ten years?
Because pretty soon I'm actually not going to be able to.
So genetic testing actually can make an extremely important set of decisions for individuals affected in this way.
And there's actually a subtlety here, too.
Sometimes people don't want to know because there's actually nothing to be done for them.
In the case of Huntington's disease that's true.
We don't have a cure for it.
So Arlo Guthrie who is the son of Woody Guthrie, who died of Huntington's disease, apparently doesn't want to know because if he learns that he's going to get it he's just going to be depressed.
If he learns that he didn't get it he might be relieved, but he doesn't want to take that chance so he's just leading life in hopes that he doesn't have the disease allele.
Now, I actually don't know in his case whether he has children or not.
He has children, so he actually made that decision almost for his children as well.
So important implications for these kinds of genetic diseases.
Now, here's an interesting pattern that I want to share with you.
And this relates to a phenomenon called penetrance.
Penetrance.
Penetrance is a number which reflects the percentage of individuals who have the disease genotype who end up getting the disease.
I've been telling you about examples where that's 100%.
If you have the disease genotype you get the disease.
But that's not always true.
Sometimes you can have the disease genotype, but because of other factors like environmental factors, what you eat, what you get exposed to you actually don't develop the disease.
Or maybe you just got lucky because some of these diseases are stochastic in nature.
And you might have been one of the lucky ones.
That would be an example of lack of penetrance.
You have the disease genotype but you don't have the disease itself.
And this can lead to the development of individuals in pedigrees, such as the one I'm going to show you, who are obligate carriers, obligate carriers.
We call them obligate carriers because they have a child who is affected but they themselves were not affected.
And they have a parent who likewise was affected.
So the simplest explanation for a pedigree such as this is that this mother passed along the disease allele to her daughter but she did not manifest the disease, an example of lack of penetrance, but she still had the disease allele which she passed onto her daughter who developed disease.
OK?
And we call these individuals obligate carriers.
Even though they don't manifest the disease they must be carriers.
And in that sense this circle should be shaded.
We haven't shaded it because actually there are other explanations for how you can get this pattern.
Sometimes, for example, the disease is such that there are non-familial forms of the disease which complicate the analysis.
Heart disease is a good example.
There are familial forms of heart disease and there is sporadic heart disease.
Maybe this woman doesn't have the predisposing mutation and her daughter just developed heart disease.
That can happen.
Another example is that maybe she picked up a new mutation.
Her mother is actually clean but she picked up her own mutation in the development of the sperm or egg that gave rise to her, and then she has the disease.
That would be rare but not unprecedented.
And the final example, which is the most interesting, is so-called non-paternity or even non-maternity.
So we're making an assumption here based on what the family has provided us in terms of this pedigree that this girl is the daughter of this mating, this woman and this man.
But about 10% of kids born in this country actually have a father who isn't their father.
It's a shocking statistic, I know, but it's true.
This is called non-paternity.
So it might be the case that this woman actually doesn't have the mutation.
It's just that her father, her mate, the father of this girl [LAUGHTER] is not that guy.
And this is sick but true, most often in situations like this it's Uncle Bob or brother Steve.
I know it's disgusting but this is often an example, the explanation for examples such as this.
And another interesting example is non-maternity.
We're assuming that this is the mother of this child but sometimes families don't want to admit, for example, that this girl had a baby.
And so they give it to Aunt Sue.
And now Aunt Sue's child gets the disease, but it's not because of Aunt Sue's genes.
It's because of her cousin.
OK?
So here are some examples.
Now, I have one more minute, and I need to riffle through the slides because I'm going to leave you for a few days.
And I want to just remind you that there are X linked diseases that affect, that are both dominant and recessive.
Most of them are recessive.
And here's a classic example, an X linked recessive disease involving hemophilia in the royal families of Europe.
Importantly, when you look at X linked disease pedigrees, and here's Duchenne muscular dystrophy, another familiar disease, again X linked recessive, there are certain rules that dictate the development of the disease over the generations.
They're summarized here.
And you should look in your book for more information.
And, finally, there are rare examples of X linked dominant diseases, and their rules of inheritance are somewhat different.
So you should familiarize yourself with X linked modes of inheritance.
OK.  And here we are in the molecular biology section.
And the goal of this section, as Professor Jacks started to tell you during the Genetics module and Professor Baker told you at the beginning of last lecture is try to link together in molecular terms the question of genotype and the question of phenotype.
And we presented to you this notion that goes by the ponderous name of the central dogma that the link between genotype and phenotype is related to DNA as a genetic material that then proceeds to transmit its information to a final outcome, which is very often a protein, through an RNA intermediate.
And the point of these molecular biology lectures is to tell you about the molecular biology, and then at the end to try to bring together this genotype and phenotype in molecular terms.
Now, last lecture you talked about DNA replication, DNA as the genetic material required to be replicated faithfully and accurately so it can transmit its information to the next generation.
Professor Baker I know stressed the requirement for accurate replication, but she did not do one part of this lecture that I want to spend a couple of minutes now discussing with you.
And that is the question of DNA repair.
So there are two types of DNA repair.
Actually, three types that I want to talk to you about.
And the first pertains to the accuracy of the DNA polymerase that replicates the DNA.
So DNA polymerase -- -- three, or the DNA polymerase that replicates the DNA makes mistakes.
It puts in the wrong nucleotide.
It puts in the wrong base.
And it does so about one in ten to the fifth bases.
OK?
So one in a hundred thousand bases is wrong.
Now, if you think about the fact that there are more than ten to the ninth bases in a human genome, every cell cycle that translates to ten thousand or so mistakes, that's a lot of changes in the DNA.
That's not a very faithful kind of DNA replication.
So this has been selected against evolutionarily.
And there is a mechanism that's called proofreading -- That allows this high error rate to be corrected.
And it's actually very cleaver.
So this DNA polymerase has what is called an exonuclease activity.
Exo meaning out.
Nuclease meaning to break down nucleic acids.
And this exonuclease proceeds from the 3 prime to the 5 prime direction, the 3 prime nucleotide being the one that was added last as you should now know.
And so what DNA polymerase does as it is replicating is it kind of feels whether or not the double helix has reformed in a smooth way.
And if it feels that there is a bubble there, a bubble where the two bases, actually look at me rather than the diagram.
I think it's easier.
I can do it better with my hands.
Where you've got a nice smooth helix, if there is a mismatched nucleotide, the bases do not pair, there will be a bubble.
OK?
There will be a bubble in the helix or a space in the helix.
The two strands will not be joined together.
And the DNA polymerase can sense this and it can go back and it excises the wrong nucleotide and puts in the correct one.
OK?
This is called proofreading.
And it's extremely necessary and it's actually very good.
And what it does is to decrease the error rate to one in ten to the ninth bases.
OK?
So you get four orders of magnitude improvement in the accuracy of DNA replication.
Now, there is another set of things that can go wrong.
And these actually fall under the heading of mutagens.
Mutagens, as Professor Jacks mentioned to you, being agents which change the base sequence of the DNA once the DNA is there.
And these can either be chemical or these can be ionizing radiation.
And in those cases also the helix gets changed because the wrong base gets put in.
No, not because the wrong base gets put in.
But because there is a chemical reaction which might modify a base, which might, for example, covalently link two bases.
thymine for example.
If two thymines are sitting next to one another in the helix, ultraviolet light is very good at cross linking those.
And you now have something called a thymine dimer.
And that is very bad because that is not a normal base sequence.
And when replication time comes along that DNA helix is abnormal and the replication machinery doesn't know what to do about it, and that can lead to all sorts of problems and to mutations.
So there are mechanisms that can get rid of abnormal bases.
So mutagens can chemically, actually, maybe not say chemically.
Let me just say change bases.
They change base structure either to something that looks like another normal new base or to something that looks abnormal.
And there are two mechanisms to get rid of this.
One is called excision repair and the other is called mismatch repair.
I have them written in the reverse order than is on this diagram from your book.
In mismatch repair there is one nucleotide that looks normal, but it's different.
It doesn't match the, usually it looks normal.
It doesn't match the one opposite to it.
And in that case the repair machinery can go in and remove the abnormal or the mismatched nucleotide, and there's another enzyme that will go and correct it.
In excision repair, one very often, excision repair occurs when, for example, two nucleotides have become covalently linked to one another, and the one strand of the helix is just a mess.
And there is an enzyme, or enzyme complex that will go in and actually excise a little chunk of the helix.
And then another enzyme will come in and fill in the gap so that you get the helix repaired.
Now, the challenge in this, and you may be asking yourselves this, is how does this repair machinery know which the correct strand was?
In the case of proofreading it's very interesting because initially after replication the newly synthesized DNA strand is not modified.
It's just a normal nucleotide polymer.
However, the template strand, the template strands, the parental strands over time become chemically modified.
The bases actually get, especially adenine gets some methyl groups added to it.
And this  is different than the newly synthesized with doesn't have these methyl groups.
And so the polymerase knows which strand is the old strand and the correct one and which is the new strand and the incorrect one.
In the case of excision and mismatch repair, that's sometimes not clear.
Where you've got these thymine dimmers, these Ts that are joined together then that's clearly the wrong, that's clearly wrong.
OK?
The enzymatic machinery can take that out and copy the other strand.
Sometimes, though, if you just have a chemical conversion of one base to another, the repair machinery does not know which strand is the correct and which isn't.
And that's when you'll get mutations fixed in the DNA because at replication you really may get the changing, you may get the incorrect, you may not get the correct base repairs.
And then that incorrect base will be passed on through the next generation.
OK.
So this is a very rapid zip through DNA repair that I wanted you to be able to think about.
I want to move onto the next step in the transmission of information from gene to final product today.
And I want to talk to you about the generation of RNA.
And so let us begin with a quiz.
And I have for you a new incentive to pay attention, a new prize that you can use to think about the conversion of potential to kinetic energy, and also you can use to amuse yourself when you're downloading very poor, when you're downloading things from the Internet and have nothing better to do.
I can usually get this right across the room.
There you go.
You can also use it to think about the nature of amphibians, they're nice flying frogs.
OK.
So let us pose the question here, what is RNA?
And you've had some of this on a problem set, but you really need to know what I'm talking about.
This is a ribonucleotide.
How do I know that this is a ribonucleotide?
Think about it.
You can put your hands up, but I want everyone to think about it.
OK.  And you need to identify the precise chemical group, please, that tells me.
I saw you two first, so yes.
What does the lower right mean?
Give me a name.
It's the?
There's a number there.
The?
Ah, we have a discrepancy of opinion here.
Someone says it's a 3 prime hydroxyl on the ribose and someone says it's the 2 prime hydroxyl on the ribose.
Let's take a vote.
Who thinks that this is identified as a ribose because of this 2 prime hydroxyl?
Thank you.
And who believes it's the 3 prime hydroxyl that identified riboses?
OK.  We have a smaller but firm contingent.
In fact, it's the 2 prime hydroxyl that identifies this is ribose.
You remember, and you really need to remember that this three prime hydroxyl is the reactive group that allows the sugar phosphate backbone to polymerize.
This 2 prime hydroxyl is a reactive group.
It identifies this as ribose rather than deoxyribose, and it also is an additional reactive group.
And the fact that it is a reactive group makes RNA rather labile.
OK?
So let's write a couple of important things here.
So this is RNA as the nucleic acid polymer.
You should really know this.
Ribose has both a 3 prime hydroxyl and this 2 prime hydroxyl.
And this is a reactive group.
And because of this RNA is a much less stable polymer than DNA.
Here's another one.
What type of polynucleotide is this and how do you know?
Yes.
You.
OK.
It's RNA.
And it's RNA we know because of these uracil groups.
OK?
So uracil is an alternate base to thymine that's found only in RNA.
Here are the Us.
It tells you it's RNA.
OK?
So you need to know those facts about RNA.
Good.
So let me pose a question to you.
In this litany that you've had several times now where the flow of information moves from DNA to RNA to protein, why is the RNA there?
This is a rhetorical question.
I'm going to try to answer it for you.
Why is the RNA there?
Why is there an RNA intermediate?
You could imagine that the DNA double helix could open up and that nucleic acid could be directly translated or could be directly converted or the code could be changed to form a protein without any RNA intermediate.
But, in fact, universally throughout biology, throughout our world anyway, throughout our earth, RNA is there as an intermediate.
Why?
Well, I think the answer actually lies in evolution.
RNA is probably the most ancient of the information polymers.
That is widely believed now.
So RNA is ancient.
It was the first, strongly believed now that it was the first information carrying polymer.
RNAs themselves were catalytic.
They became able to replicate.
And they also probably became able to be translated into protein before DNA was invented.
OK?
So DNA's chemical structure is different and it's a derivative of ribonucleic acid, and undoubtedly came second.
There was an advantage of having DNA because it's so much more stable, and it made the hereditary material much more stable and much more faithfully transmitted from generation to generation.
So RNA was ancient.
And the relationship between RNA and protein is probably a very old one, and we'll talk about this relationship next lecture.
And I believe that that relationship has persisted, and then DNA was kind of an add-on.
And the DNA to RNA to protein does not necessarily reflect the only way or the best way to do things.
Evolution is a capitalization of various changes.
And RNA to DNA, DNA to RNA to protein is how things work now.
But this, I think, is a consequence of the evolutionary past.
Now, however, in our modern world RNA serves two main purposes.
One of the things it does is to allow one to use just a subset of the genes to make proteins.
So, as you've been told several times, you and I have about 30, 00 genes in our genomes.
Not all of those genes, and we will discuss this in great depth as the course goes on.
Not all of those genes are used at any one time.
We use just a subset of the genes.
And having them converted into an RNA intermediate is one of the ways that you can allow just a subset of the genes to be used.
So I'm going to write here subset.
Subset of gene usage.
OK?
Because you can turn just some of those genes, or you can convert some of those genes into RNA, the information in some of those genes into RNA.
And the other thing it lets you do is to amplify the signal from each gene.
So there are two copies of each gene in a diploid cell.
When it comes to RNA there can be up to 10,000 copies of RNA per cell of a particular RNA.
OK?
So you can get an amplification of the signal -- -- from each gene.
RNA copy number per cell ranges from about one copy to about 10, 00, that's rare, copies per cell.
All right.
So here we are.
Why RNA?
We've dealt with that.
So I want to talk to you about two things.
I want to talk to you about synthesizing the RNA, and then I'm going to talk to you about modifying the RNA a bit.
And the first thing I want to cover is something called transcription.
Which is also known as RNA synthesis.
And you all should have this handout.
So I'm not going to draw it but I will write some salient features on the board for you.
And we're not quite ready to use that.
I'm going to leave this up here, but I'm going to work on the board for a little bit.
The basic idea behind transcription, RNA synthesis, is that one copies a DNA template into a complementary RNA strand, complementary RNA.
And one does this, as I've alluded to, only from the genes.
And this is an interesting point because although you have 30, 00 genes in your genome, in fact, those 30,000 genes only take up about 5% of the total amount of DNA in each of your cells.
So 5% of your total DNA of your genome comprises the genes, the information carrying entities in your DNA.
And the rest is other stuff.
So the 95% is not genes.
It consists of various repeats, repetitive DNA that can be there at just a few copies per genome or at 10,000 copies per genome.
They can be real little, 10 base pairs, six base pair repeats, or they can be a few kilo bases repeated many times.
Oris, Origins of Replication that you talked about last time are not genes.
Those are there, too.
Centromeres, the middles of chromosomes.
Telomeres, the ends of chromosomes.
All of these things are not genes, and they comprise the bulk of your DNA.
Now, this isn't true in all organisms.
OK?
Some organisms have got very little of this repetitive extra DNA.
We happen to have a great deal of it.
OK.
So let's pursue this a bit more.
And let's think a bit more about these genes.
And in particular let's think about the kinds of RNAs that those genes make.
So I'm going to talk about gene classes or classes.
And this is with respect to the RNA and the functional RNA that comes from those sets of genes.
And I want to distinguish two major classes of genes.
The first are the protein encoding genes.
And protein encoding genes move through a type of RNA that is called messenger RNA, abbreviated mRNA.
Messenger RNAs comprise about 1% of the total amount of RNA in a cell.
And they can range in size from let's say 100 base pairs to 10, 00 base pairs.
OK?
So there's a very wide size range.
No, not base pairs.
Yell at me.
Why not base pairs?
Why was I wrong saying base pairs?
Tell me about RNA.
Raise your hand.
This is worth a frog.
I caught myself, but if you can catch me, too.
Yes.
You.
Good.
OK.  Generally RNA is single, woops.
RNA is single-stranded.
It does not form, it can form a double helix, OK?
It's not as stable as the DNA double helix, and many RNAs, probably most RNAs have some double-strandedness to them, but that is an intromolecular double strand in this.
There are some RNAs that form intermolecular double strands, but in generally I'm going to assume that RNAs are single-stranded.
So we talk about 100 bases rather than 100 base pairs.
OK?
Second class of genes are the ones that do not code for protein, and in this case the RNA is the final product.
And this litany of DNA to RNA to protein doesn't hold.
You just stop at the RNA.
And the RNA is the functional thing.
So here RNA is the final product.
And we can break these into a bunch of different classes.
Ribosomal RNAs, abbreviated rRNA are a very abundant class of RNA that comprise about, I've moved over here, let me move here, 98% of total RNA.
And there are a few thousand bases in length that say 2, 00 to 4,000 bases in length.
OK?
So this is 98% ribosomal RNA.
This is fascinating.
I'll tell you next time.
This is the RNA that comprises a very large proportion of the ribosome that is the factory that makes the proteins.
OK?
And so I will tell you more about these next time.
Some other ones, tRNA, the T for transfer RNA.
tRNA comprise about 1% of all RNA and are about 100 base pairs, 100 bases long.
OK?
And then an interesting one that MIT has had a huge role in discovering and studying, these things called micro RNAs, abbreviated miRNAs, which are, they're at relatively low abundance.
Less than 1% of total RNAs.
And these are small.
In their mature form they're about 22 bases in length.
OK.
So now, and I believe I cannot do anything with these boards.
Ah, I can do something with this one, but that one is stuck.
All right.
So I'm going to do something with this one.
And then I'm afraid it's going to disappear, but it's not going to matter because you have the handout in front of you.
So now I'm ready to move on with you to the basic idea of transcription.
And I'm going to write some facts on the board, and we're going to look at these cartoons that I drew for you together because I think your book is kind of difficult.
So I decided to draw some cartoons to help you with the basic idea.
Transcription or RNA synthesis takes place in the nucleus.
Anyone else need a handout?
Why don't you come on down.
Actually, one of the TAs, could you be an emissary and just hand out to those people with raised hands?
Thanks.
Transcription takes place in the nucleus.
And the idea is really analogous to DNA replication with a difference.
The analogy is the synthesis of a complementary strand of nucleic acid on a template strand.
So this is an enormously important principle that you need to have.
Super important that you get the principle.
The basic idea involves synthesis of a complementary strand of nucleic acid from a template strand.
The template, actually, let me start even earlier than that.
We start with a gene that generally comprises double-stranded DNA.
There are exceptions to almost everything that I will tell you, or that Professor Jacks will tell you.
You should understand that there are exceptions.
Some organisms, particularly viruses have genomes that are RNA that can be single-stranded or double-stranded RNA.
Some have genomes that are single-stranded DNA.
But in general most genomes are double-stranded DNA.
And the deal is this.
The double-stranded DNA separates its strands, and one of the strands, and this is the difference between DNA replication and transcription, one of the strands becomes the template strand.
And this template is copied to form a complementary strand.
And it's copied by an enzyme called RNA polymerase.
So RNA polymerase synthesizes the complementary strand to the template strand.
complementary strand.
And it does so, of course, as RNA, because we're talking about RNA synthesis and this is RNA polymerase.
It does not, unlike DNA polymerization, require a primer.
So this does not require a primer.
OK.  You should know, and it should be getting deep within your neural circuitry that polymerization occurs by adding nucleotides to the 3 prime end of the growing polymer.
Yes.
If that didn't make, you know, if you didn't say "yeah" to that, go back and think about it, go back and look at problem sets and you'll get more practice in this.
But you really need to know that the growing chain adds onto the 3 prime end.
OK.
So after the polymer, after the RNA polymer is made the RNA is released from the template strand.
As its being transcribed it forms this complementary strand.
And, as you know, complementary strands can base pair.
After it's made it is released from the template.
And it usually then goes into the cytoplasm where it does its thing.
So if you look at the diagram I gave you, that's what's up here, here's your double-stranded DNA, your gene.
The strands separate.
One strand is transcribed into RNA.
The RNA is release.
Obviously, your double-stranded template, or what was your double-stranded template will reform its double strand.
So perhaps that's not so obvious, but the double-stranded, originally double-stranded template will reform its double strands, thus released RNA, then goes into the cytoplasm where it is translated into protein, or where the RNA is the final product.
So let's look at that in a bit more detail.
I've got here a template strand shown in red.
This is, again, the second picture in the handout in front of you.
And I've got three features added here.
I have got a precise start site of transcription.
I've indicated elongation where the polymer is elongating.
And I have a precise termination site where transcription ends.
OK?
Now, let me see what I have here.
I have a movie here.
Watch the movie.
I'll show it to you once, and then you can go and watch it at your leisure.
This is meant to be RNA polymerase.
There's the helix opening up locally.
Here are ribonucleotide triphosphates coming in, and RNA polymerase is catalyzing their synthesis.
OK?
So the template strand is the bottom and here is the RNA being released.
There's RNA polymerase moving along the helix.
And the depiction is that the helix is opening locally and then closing again behind the RNA polymerase.
At transcription termination, the helix, the gene helix zips up again and the transcript is released.
So this is a very much simplified story.
But is the basic principle of transcription.
And you should know it.
And in particular I have put onto this second diagram, and because you have him in front of you I'm not going to write this on the board, I'm going to use this as something to tell you, I have put the directionality of the strands of the double-helix on this diagram.
This should be something you can deal with.
5 prime to 3 prime on one strand.
The other strand is anti-parallel.
RNA, any nucleic acid is synthesized by adding onto the 3 prime end.
And that newly synthesizing nucleic acid polymer is anti-parallel to the template.
This is also something that you should be familiar with.
And you will have, will have, have not yet, will have practice on doing this kind of polymerization, but it should be something you really, really should be familiar with, this anti-parallel requirement.
So, in fact, you can tell the direction of transcription because of the directionality of the template strand.
OK.
So this is very important for you to go and think about after class the directionality of the template and of the newly synthesized polymer.
These are some diagrams from your book, and you can go and look at them.
I'm not going to dwell on them.
They indicate the different between, or the steps in transcription initiation, elongation and termination.
And I've put them up there just to tell you there are these diagrams in your book and you can go and take a look at them and read the accompanying text.
OK.
So I see three problems with transcription that are very interesting problems.
One is how to find the genes.
I'll write them on the board and then we'll go through them.
5% of the genome is genes.
That's most of it that is not genes.
How does the transcription enzyme, how does the RNA polymerase know which is a gene and which is not a gene?
How does it know, even if it finds the gene, which strand is the template strand and which is not the template strand?
I could have drawn your previous diagram where the top strand was the template, and that what would have happened would be that the RNA synthesis went in the other direction.
So which strand is the template?
And that also gives you the direction, of course, of transcription.
And the third one I'm going to write, and then I'll tell you about this in a moment.
I'm going to write how to unwrap chromatin.
OK.
So in each of your cells, you have to look at me for this.
In each of your cells there is this length of DNA.
One meter.
This is a little over, but one meter of DNA.
How big is the average cell in diameter?
Give it to me in micrometers.
Worth a frog.
On average.
Well, that's actually a really big cell.
It's about ten times less than that.
But whoever that was, who was it?
No way.
These are very bad to throw.
Very bad to throw.
You can have it because you caught it.
See me afterwards.
I'll give you one.
OK. [LAUGHTER]  OK.
So how do you pack a meter of DNA into a cell that is about ten microns in diameter?
OK.
So, OK, Jamie, you want to hazard an answer here?
Your hand was up.
OK.  Good.
You can wind it up.
OK.
The other thing you have to do, of course, is to make it really thin.
It has to be a lot thinner than my piece of rope.
But once you've made it really thin you can then wind it up.
OK.
It's logical and this is how it's done.
And you can wind it up and then it will fit into your ten micron cell.
Now, in actual fact, there's a whole process to do that.
And I'm going to go through them as we go through these problems here.
So here is problem one exemplified.
I've got red little dots for each of the genes.
How does RNA polymerase find these genes in this vast amount of DNA that is not genes?
Here's the other one.
Which strand is the template?
Oh.
And here is a nice problem that in the interest of time I am not going to do here in class with you, but I want you guys to go and do this as an exercise.
I will tell you that the answer is not on your handout on the Web.
I took it off.
Sneaky, ha?
So that you can go and think about this.
I want you to go and understand that the products of synthesis from either strand of a DNA double-stranded helix are not the same.
OK?
And I'm going to zip through this because I want to move on here.
OK.  And I want to move to problem three which is this thing I called chromatin.
DNA is wound up around proteins.
These are called histones, and we'll have more to say about them later in the course.
And wound up and wound up and wound up.
And there is a very set number and type of proteins that the DNA is wound around.
And once the DNA has been wound around once, those DNA protein complexes are wound up some more, and then wound up some more.
And eventually you get them wound up and wrapped up so much you get the characteristic rather large chromosomes which are very much packed DNA.
Now, this is a great way to fit DNA into a cell.
However, this wrapping up of the chromatin into, the wrapping up of the DNA into this chromatin structure inhibits transcription.
And in order to allow transcription to proceed, you have to remove these proteins from the DNA and allow it to unwind locally.
And that takes a whole series of enzymatic steps, again that we'll explore more later in the course.
But the problem I throw out at you now is hw do you unwrap the chromatin where transcription is needed?
And the answer to all of these things lies in a specific, no, stop.
Stop.
Down.
Up.
OK.
The answer to all of these questions lies in a specific DNA sequence or a series of specific DNA sequences that are collectively called -- -- the promoter.
Here's another one.
What is a promoter?
And I need to make the distinction now between transcribed DNA of a gene and untranscribed DNA of a gene.
The promoter is part of a gene but it is not transcribed.
It usually depends on the gene and the type of gene.
It usually lies 5 prime to the transcriptional start site.
And it is a DNA sequence that says this is a gene, and it also says transcription should proceed in this direction.
OK. And the way it does these things, I'm going to each of the answers to each of the problems now, is that it binds proteins that specifically recognize the sequence of the promoter.
So you talked about the DNA replication origin and proteins that specifically recognize the nucleotide sequence of the origin.
This is analogous.
There are proteins that recognize promoter sequences which are similar but not identical from gene to gene.
So it binds proteins.
And these are called transcription factors.
And these transcription factors bind in a DNA sequence specific way.
OK.
It also binds RNA polymerase which I'm going to abbreviate RNA polymerase, RNA pol.
OK?
And the answers to the three questions are that firstly the protein-DNA interaction is sequence specific, sequence specific, and so this allows you to actually find the genes.
Secondly, and this is cool and I'll show you a picture of this in a moment, the proteins interact with a promoter DNA differently on different strands of the helix so they bind asymmetrically.
They may bind more to one strand than to the other strand.
And this gives directionality to the transcription so the protein binding is asymmetric or strand specific.
Not for all of these proteins but for a significant number.
And that helps you decide which strand you're going to use as the template.
And thirdly these proteins have got associated with them activities that will unwrap the chromatin, that will unwrap the DNA from its protein complexes and allow it to be accessible to the transcription machinery.
OK.  Let's zip so I can show you this.
You can look at this on your slides.
Here are some pictures from your book.
Really important in this, don't move.
Really important in this is a protein called TF2D which recognizes a sequence called, that goes T-A-T-A-A-A.
This is called the TATA binding protein.
And it's really important.
And it's the one thing that, the major thing that gives asymmetry to this transcription, set of transcription factors on the promoter.
Once TF2D has bound to the promoter, other proteins come along, including these various other things called BFHG and so on.
And here's RNA polymerase.
And you can see this complex positioned asymmetrically on the DNA.
And this complex you should know the name of, I'm going to put it in its own box here, is the initiation complex.
And I want to show you a crystallographic rendition of the TATA binding protein called TBP, also sometimes called TF2D.
But TATA binding protein here shown in purple.
And if you look here, you're looking head on at the double helix.
OK?
Here's the helix.
You're looking down the helix.
And you can see that this protein is positioned on just one side of the helix, so that gives you asymmetry.
Here's another transcription factor bound to DNA.
This is a protein called GAL-4.
It binds as a dimer.
And you can see that GAL4 is the blue.
And here it is contacting just one side, one strand of this red double helix.
And I'm going to stop there and finish off the last little bit on Monday.
OK.  Parts of a gene.
We have our promoter, which is part of the untranscribed region of a gene, usually in the 5 prime end.
Not always but for the genes we're talking about at the 5 prime end, the so-called 5 prime end of the gene, or so-called upstream of this transcribed region.
And downstream of that there is more untranscribed region that interestingly can also contribute to the promoter, even though it's far away from this more upstream part of the promoter.
But I'm going to call it just for now untranscribed, two flanking regions of untranscribed DNA sequence and one region of transcribed sequence.
Now, I want to discuss with you very briefly a phenomenon called splicing.
And this is a phenomenon that occurs within the RNA that is transcribed from a gene and, therefore, pertains to the transcribed region of the gene.
It turns out that in this transcribed region there are two kinds of sequences.
There are things called exons and there are regions called introns.
The exons code for something, code for the final function of the RNA or for eventually a protein.
So these are coding.
The introns are noncoding.
Both of them are transcribed.
You'll see this definition is a little loose as we move on in today's lecture, but it's good enough.
In the transcript that initially is made from the gene in this transcribed region, both introns and exons are present.
So these are present in what's called the primary transcript or primary RNA.
And primary refers to the first RNA that is transcribed from the gene.
And subsequent to that, still in the nucleus, those introns and exons are subject to a process called splicing whereby the introns are removed -- -- or spliced out is the term, such that in your mature RNA only the exons are present.
This process is likely a consequence.
I'm going to put up a diagram that you had on your last time's handout.
You can watch it now.
And you can refer back to a previous lecture if you don't have it with you.
This notion of introns and exons is probably a consequence of evolution whereby different parts of genes were combined and shuffled to give new kinds of genes and, therefore, new kinds of proteins.
Here on my diagram I have exons in black and introns in blue, and they're all just DNA sequence, but when the RNA is transcribed in the first place is primary RNA.
It's a copy of the gene.
It has both exons and introns.
And then a very complex enzymatic machinery comes on and it loops out and excises these introns.
OK?
So this is very interesting such that in your mature mRNA there are no introns.
And the introns have been looped out and they form these little structures that are called lariats.
And at this point your mRNA is mature -- -- and it moves to the cytoplasm.
Now, this process what discovered by Professor Phillip Sharp here at MIT and he got the Nobel Prize for it in 1993.
It's a very important process because it's absolutely required for maturation of RNAs.
And also, and I'll come to this in a few lecture's time, it allows different proteins to be made from the same mRNA.
So here's a rule.
In this RNA there are what are called splice donor cites that I've put as a circle and splice acceptor sites that I've put as a square.
Just watch this.
Just watch this for now because we will come back to it.
So watch what I'm saying rather than trying to madly write down.
Any spliced donor can join to any splice acceptor and remove the stuff between them.
So in this top example I've got each introns being neatly removed because splice donors and inceptors interact.
But look at the example below.
I've got this splice donor next to exon one interacting with a spliced acceptor next to exon three.
And when that happens you remove the hull of exon two.
So you actually are going to make a different protein.
Whereas, in the first case you'll have exons one, two and three and four.
In the second case you'll have exons one, three and four.
OK?
So this process is very important for allowing different kinds of proteins to be made from the same gene.
I want to make you aware of this now, and I will come back to it in the formation module when we talk about how different kinds of cells are generated.
All right.
So let's move onto the major topic of today's lecture -- -- which takes us back to the central dogma.
And I want to introduce to you a term that is very important that you know and you understand.
And this is the term gene expression.
And really what we've been talking about is gene expression.
Gene expression simply refers to the generation of the final product of a gene from the gene.
So we're talking about the formation of a protein as directed by a particular gene.
OK?
So gene expression is, if you like, the readout.
Here's another way of putting it.
The readout, the final readout of a gene, or the generation of the final product of a gene.
I'm going to come back to this term over and over again, and I will ask you to define it in your own way, but it's a term I want to throw out at you now because you do need to know it.
It's very pervasive.
Today I want to talk about the step in gene expression or translation whereby RNA is converted or is used to direct synthesis of a protein.
So let's define translation because it is, I think, one of the most interesting questions in molecular biology.
Certainly from a historical perspective that was true.
And the notion in translation is that the base sequence of a mRNA somehow leads to the synthesis of a protein with a defined amino acid sequence.
Now, if you think about DNA replication, transcription and translation, the relationship between them, there is a nice analogy that one can make.
DNA uses the base code, four bases.
Transcription RNA uses those same four bases as a code, but it's slightly different from DNA.
So the synthesis of RNA using a DNA template is kind of like changing fonts in a document that you have.
It's kind of like going from Times New Roman to Helvetica.
You haven't really changed much.
It just looks a bit different.
Translation is very different.
The use of mRNA to direct the synthesis of a protein is much more analogous to changing language where you've taken English and translated it into Chinese or Russian and translated it into French.
OK?
So this is a really different process.
And it was clear from the outset, historically, that one had to think in a slightly different way about how this process was directed.
And I want to talk about four things with respect to translation.
Firstly, I want to talk about the genetic code that allows RNA to direct protein synthesis.
I want to talk about something called the interpreter of that code.
I'm going to talk about the factory in which the synthesis takes place.
And then I'm going to get to a discussion of the molecule bases for genotype and phenotype.
So let's think about the code.
And thinking about this starts from a very simple logical place.
And the place is this.
One starts with four bases, A, G, C and T or A, G, C and U, depending if you're talking about DNA and RNA.
And somehow those four bases have to be used in some kind of code to give you an outcome of 20 amino acids.
And I am going to use the abbreviation AA for amino acids.
So you can look at this and immediately understand there has to be some kind of combinatorial code in order to specify those 20 amino acids.
So you can do combinations and you can say, OK, if two bases were used and you could have combinations of doublets, how many combinations can you get to and would that be enough to specify those 20 amino acids?
Well, no, because two base combinations would only give you 16 possible amino acid combinations, or the ability to specify 16 amino acids.
OK?
Four squared.
How about three base combinations?
Well, that's better.
What you can get out of that is 64 different combinations.
OK?
And that is plenty to specify your 20 amino acids with some left over.
And, in fact, this is what is used.
Combinations of three bases.
And these combinations of three bases are termed the triplet code.
The discovery of the triplet code is really fascinating.
I don't have time to go into it in this lecture, but your book is not too bad on the discovery.
And I will post on your website, for those of you who really want to get into it, a reference to a very interesting historical account of the discovery of the triplet code and indeed of much of molecular biology.
But it's a fascinating story.
But I'm going to tell you the code is a triplet code.
OK.
So what does that mean?
It means that three bases correspond to a particular amino acid.
OK?
So one triplet of bases correspond, I'm writing this out because it's really important that you know this, correspond to one amino acid.
And this base triplet gets a special name.
It's called a codon.
And the thing that you will have noticed is that what I've told you is there are 64 possible combinations of triplets and only 20 amino acids.
And so that leaves some over.
What happens?
Well, they're all used.
And what happens is that although the code is universal, as far as we know it arose just once, all living organisms on our planet use this code, it is a redundant code.
So I will write down it is redundant but not ambiguous, and tell you what that means.
So what that means is that an amino acid can be specified by more than one triplet, and I'll show you that in a moment, but that any triplet of bases only corresponds to one amino acid.
Let's look at some diagrams to show you what I mean.
This is a table of your amino acid code.
These letters in columns represent the bases.
And next to them are written the amino acids that correspond to this particular code.
Let's start with an easy one.
This is methionine encoded by AUG. And that's one you should actually remember.
OK?
And for methionine there is only one possible codon.
It is AUG and always AUG.
But let's keep going here.
And let's look at the amino acid lucine.
lucine is encoded by six possible triplets, six possible codons, UAA, UAG, CUU, CUC, CUA and CUG.
Any one of those in a mRNA can encode lucine.
However, CUU only encodes lucine.
It never encodes another amino acid.
OK?
And that's what I mean by redundant.
More than one triplet can encode one amino acid, but any given triplet only corresponds to one particular amino acid.
OK. You will have practice on this kind of thing as you go along.
So let's get some basics down here.
The template in the whole translation process is your mRNA.
OK?
It's the code.
It contains the code.
It is read to give a protein readout from 5 prime to 3 prime.
And the readout of the protein, as I mentioned to you way back when, reads out from the amino to the carboxyl end.
New amino acids are added onto the carboxyl end -- -- and the free amino group corresponds to the first amino acid polymerized.
So it is read 5 prime to 3 prime, and that corresponds to the amino to the carboxy growth of the protein.
All mRNAs start with the same amino acid, and that is methionine.
And the start or initiation codon in all proteins is methionine, oops, is AUG which encodes methionine.
Now, not all final proteins have got methionine at their amino ends because it can be cleaved off.
OK?
So you don't have to land up with a protein that has a methionine end, its amino end, but it starts off with methionine there.
And then there are no gaps in the message.
It is read without any punctuation marks, except for the fact that the codons are next to one another in a non-overlapping way.
OK?
So there are no gaps.
And the only punctuation is the start codon and a series of stop codons which do not encode any amino acids.
These are UAA, UAG and UGA.
And you can remember them if you want, but we're not going to test that you do.
OK?
You can use your amino acid tables.
OK.
So your punctuation is the start and the end of the message.
All right.
So let's go on and talk about the interpreter and what I mean by the interpreter.
In this diagram here I have got, look up here for a moment.
This is quite a nice diagram not from your book.
I've got your DNA strand, which is your template strand.
Your corresponding RNA, your mRNA, and the readout of the RNA to the protein.
And here are the codons, UGG, this is in the middle of the protein so that's why there's no methionine, UGG corresponding to tryptophan, UUU corresponding to phenylalanine.
You can see how the codons are right next to each other, OK, but do not overlap.
In fact, I'm going to write that on the board.
So no gaps and no codon overlap.
Very important that you understand that.
So when people looked to this and figured out what the codons corresponded to in terms of amino acids there was the question of, well, how do you actually get those amino acids corresponding to those codons?
And there was a sense that you needed some kind of adapter or interpreter molecule that both recognized the codon and recognized the amino acid.
And that's the next thing that I'm going to tell you.
And -- -- stop.
Well, I apologize on behalf of our illustrious institute for the boards in this room.
OK.
So all right.
So let's talk about interpreter.
And I'll tell you that this is the class of RNA someone brought up earlier called tRNAs.
So tRNA, as you may recall, are these very small RNAs.
There are about 100 base pairs, 100 bases in length, and there are a lot of them.
And there is a tRNA that corresponds to every codon.
So tRNAs recognize both the amino acid and the specific codon.
And they recognize, let's talk about the codon first.
They recognize the codon by DNA complement, by RNA complementarity, by base pairing to a region on the tRNA called the anti-codon.
So let's talk about methionine for a moment.
The codon for methionine is AUG.  That's the codon.
Woops.
Hold on one second here.
5 prime AUG, that's your codon.
And what will be complementary to that on the tRNA from the 3 prime end is UAC.
OK?
So this anti-codon is on the tRNA.
Anti-codons can either be written from the 3 prime end or you can switch them around and talk about 5 prime CAU.
It's the same thing.
OK?
So that's one thing.
I'll show you a picture in a moment.
The other thing that a tRNA has to recognize is the amino acid.
And that's more complicated.
For different amino acids there are different parts of the tRNA molecule that recognizes specific amino acids.
And it hasn't actually been figured out completely which part of which tRNA recognizes a particular amino acid, but the recognition is also on the tRNA -- -- and not really on the anti-codon.
Or certainly not the anti-codon alone is probably fair to say.
So let me show you a picture of a tRNA.
tRNAs are single-stranded RNAs that fold up on themselves in a complex way.
OK?
Here's the representation of the three-dimensional structure of a tRNA.
And these cross things are hydrogen bonds.
So there's a lot of base-pairing within the tRNA.
Represented more simply, the tRNA forms this kind of cloverleaf structure, and the anti-codon is at one end of the tRNA.
OK?
So this is the thing that's base pairing to the codon and the mRNA.
The amino acid attaches to the very 3 prime end of the tRNA at this site which is a CCA.
OK?
And there is a covalent attachment of the tRNA to the amino acid at this CCA region.
All right.
But the part that recognizes the amino acid can be somewhere in the rest of the tRNA molecule.
It's very complex.
OK.
So let's move on now.
Actually, let me tell you one more thing, though I'll tell it to you in a moment.
OK.
So let's move on now to the question of the factory.
And by factory I mean the place where protein synthesis or translation takes place.
And the factory here is the ribosome.
We mentioned ribosomes right at the beginning of the course in the second lecture and haven't said a whole bunch about them since.
Ribosomes are very large structures.
They are not membrane bound, but they are very large.
This is a representation of a ribosome from bacteria that has a small subunit and a large subunit.
And, interestingly, ribosomes are an obligatory complex between the so-called rRNA, or ribosomal RNA, plus proteins.
There is a small subunit, this is really bad.
Let's try this one.
Small subunit which consists of one ribosomal RNA of a particular kind and 33 proteins.
And there is a large subunit.
And I tell you this not because you need to remember this, but you need to appreciate that this is a very complex structure.
It's a very cool and complex structure.
The large subunit comprises of three RNAs and 45 proteins.
You can represent the structure of the ribosome much more beautifully in this diagram, or in this representation, where the RNA is shown in gold, or the two RNAs are shown in gold, or the multiple RNAs are shown in gold, and some of the proteins are shown as these other structures and you can see the alpha helices of the proteins.
OK?
And what you should be able to see on this diagram, let me point to this one for a change, is this tunnel, this hole through the structure.
And this is the tunnel through which the mRNAs thread as it is translated.
So this is truly a factory.
tRNAs come into this, the mRNA threads through, and as that takes place so the mRNA directs the synthesis of the protein.
OK.
This is a representation from your book.
I don't like most of the diagrams from your book so I redrew most of them for you, but I left this one.
This is a representation of translation.
The mRNA is shown in green and the large subunit and small subunit of the ribosome come together, form the complete ribosome, and then the mRNA actually is thread through the ribosome and the protein, well, here they've called it a polypeptide chain is thread through.
So let's explore this in a big more detail.
And in order to do so, I've got to conserve boards here because we are one board short.
In order to do so I need to introduce you to the various parts of a mRNA.
And this is on one of the diagrams that I handed out today.
OK?
So you don't need to redraw it.
Just look at the diagram.
In the mRNA, and this is crucial for translation, there are three parts that are really important.
Two of them, excuse me.
Two of them are actually added to the mRNA after it is transcribed.
The thing at the very 5 prime end called the cap and something at the very 3 prime end, which is a long string of up to a couple of hundred A residues contiguous, which is called the poly A tail.
And these parts of the mRNA are crucial for the first part of translation which is initiation.
As in replication and transcription, you can divide up these synthetic processes into different steps.
And initiation is the first step.
And in order for initiation to occur one needs the parts of the RNA that are added on post-transcriptionally.
You need the cap, this poly A tail -- -- and also a region that is just upstream or 5 prime of this AUG initiated codon in a region called the UTR, the 5 prime UTR which stands for untranslated region.
And you also need the AUG codon.
OK.  And what happens is that the ribosome and various initiation proteins bind to the 5 prime cap and simultaneously to the poly A tail.
So this is really cool.
The mRNA is translated as a circle where this poly A tail, the very 3 prime end is brought all the way around to the 5 prime end, and you get a whole mess of proteins sitting on that part of the RNA and starting translation.
So you get initiation proteins, which are called initiation factors, and you get ribosome assembly where the small subunit and the large subunit come together, and you get a tRNA carrying a methionine amino acid coming and sitting on the AUG. OK.  Let me show you more.
So here we have a cartoon, you have this in front of you but I'm going to show it to you in a step-wise fashion, of this ribosome recognition sequence.
Actually, I'm not going to show you now but in your handout there are pictures of the circular RNAs being translated.
OK?
That's something new and it's something very interesting.
I'm not going to dwell on it now.
OK?
Where the poly A tail comes all the way around to that 5 prime so-called cap region.
I should just point out, again, I'm not going to dwell on it, the so-called 5 prime cap region is a modified guanine.
OK?
MEG stands for methyl guanine.
You can call it the cap.
It designated the very 5 prime end of the message.
OK.
So let us look at the sequence of translation.
And what I'm going to tell you, before I go through the cartoon, is that in the elongation process sequential tRNAs carrying their particular amino acids are going to come in.
And they're going to sit on these various codons.
And peptide bonds are going to form between adjacent amino acids so you get the polypeptide chain growing.
OK?
So let's start off with the initiator.
There's your tRNA that is joined to methionine.
And I need to introduce you to a term now which is a charged, I didn't have space before.
The term charged tRNA refers to the tRNA covalently linked to its amino acid.
And then correspondingly the uncharged tRNA has no amino acid.
The amino acid has fallen off or been used.
OK.
So there is a tRNA sitting on the first codon, the AUG, and that's the start of the sentence.
That positions the beginning of the protein.
Now, watch what happens.
Here comes another tRNA that corresponds to lucine, and you're getting base pairing here.
That first tRNA is base paired to the AUG codon through its anti-codon.
The second tRNA is base paired to the second codon through its anti-codon.
And now you've got a methionine tRNA sitting next to a lucine tRNA.
OK.  Everyone with me here?
And what happens now is that a peptide bond forms between the methionine and the lucine.
In particular, this methionine is going to move over to that lucine over there and lead to uncharging of that particular tRNA.
Take a look.
OK, so I've shown you that methionine is going to form a peptide bond with the lucine.
Now, watch what happens next.
Here's the methionine tRNA.
It's lost its amino acid, OK, so it falls off the message.
It's done its thing.
It's no longer needed.
Along comes, no, sitting there is this lucine tRNA which is now covalently attached to its peptide bond to the methionine.
And there's a free amino end here which designates the first amino acid synthesized in a polypeptide chain.
And here comes in the next tRNA that corresponds to a serine tRNA based paired by its codon, base paired by its anti-codon to the codon on the mRNA.
OK?
And the same thing is going to happen again.
The lucine and the methionine is going to be transferred over and make a peptide bond with the serine, and so you get elongation of the polypeptide chain.
So what I'm going to write under elongation is that adjacent amino acids join.
Uncharged tRNAs leave, are released, and sequentially new tRNAs corresponding to codons come in.
OK.  All right.
So this whole process goes on until the mRNA, until the ribosome and all these tRNAs reach a place in the mRNA where there is a codon that doesn't correspond to an amino acid.
A so-called stop codon.
And at this point there is a process called termination where there is a stop codon that does not code for any amino acid and doesn't have a corresponding tRNA therefore.
And at this point the protein polypeptide chain falls off the message.
All right.
You guys OK with that?
OK.
I'm going to refer you, I'm not going to go and watch this movie.
Go and watch this movie.
Go and watch the movie by yourselves.
OK?
I don't want to take the time to watch it now.
It's an animation of what I've just told you.
There are some diagrams in your book.
You can look at them.
They talk about things called A sites and P sites in the ribosome.
To me that is less important than you understand the actual interactions between the tRNAs and the mRNAs.
Here is a circular RNA with that poly A tail and the 5 prime cap of binding proteins to initiate translation as a circular RNA.
All right.
So, finally, let's move to this complicated, I think fantastic bringing together of mutation from genotype to phenotype.
You've had a genetics module where you talked about mutations, you talked about the genotype, you talked about the phenotype.
We've been throwing at you genotype has got something to do with the DNA base sequence.
Phenotype has got something to do with the final product, particularly the protein sequence.
Let's explore that in a bit more detail now and ask, what is the molecular basis for changes in genotype and how do these correspond to changes in phenotype?
OK.
So genotype to phenotype.
And I want to emphasize again that phenotype is an outcome of a change in function of the final product of a gene.
It isn't necessarily the same as the final product of a gene.
OK?
So, for example, a phenotype is giantism.
Someone who is very tall.
The molecular basis for that could be multiple things.
It could be production of too much of a hormone, a protein called growth hormone so that someone grows too tall or taller than normal.
OK?
So that is the phenotype is connected to the production of a particular protein that's not the same as.
So here's another diagram, something for you to think about.
Mutations, almost anywhere in a gene, can have an affect on the protein produced.
And there are two ways the protein produced can be affected.
One is in the amount of protein and the other is in the sequence of the protein produced.
Now, if one gets a mutation in this promoter region or often in the introns, but particularly the promoter I've focused on, one can change the amount of RNA that is being transcribed from a particular gene.
And that change in the amount of RNA will lead to a change in the amount of protein.
And you may get a phenotype because you're making too little or too much protein.
Conversely, changes in exons can lead to changes in the actual sequence, the amino acid sequence of the protein and, therefore, to its function.
So those are two important distinctions to make.
OK?
So mutations can change the amount or the sequence of a protein.
I'm going to go through some examples of mutations, and you will go through more in Section, and you are expected to know these changes and you are expected to know how the change in DNA sequence may lead or not to the change in protein sequence.
So look carefully.
I'm not going to get through all my examples today.
You can go and you can do the examples that are posted on your website.
You'll get more practice.
And you really need to know this.
OK, so here's a wild type gene.
The top two strands are the DNA.
The bottom of the strands is the template strand.
This DNA is transcribed into a mRNA and that is translated into the protein indicated here.
OK?
Let's look at an example of what happens when there is a change in the DNA.
So here I've got a change in the DNA, OK, such that this particular base pair has been changed.
The mRNA, oh, this is the wild type again.
Here's your wild type sequence, wild type mRNA, wild type protein.
This is a class of change in the DNA that is called a nonsense mutation.
Watch carefully.
So at this position that I've underlined, watch this.
Don't try to write anything down.
OK?
You'll have plenty of practice.
This is all posted.
Just watch.
At this particular underlying position, instead of a GC base pair there is now an AT base pair.
And that changes this codon UGG into UAG.
And UAG happens to be a stop codon.
So here's your gene, your mutant gene, here's your mRNA that comes from the mutant gene, and here's the protein.
It starts OK with a methionine.
But, look, the next codon is a stop.
OK?
So the protein is truncated.
Now, there are a number of classes of mutation.
I am going to write these on the board.
I'm going to ask you to go and read your handout carefully.
And you will cover these in section.
Again, you need to know them so let me list the types of mutation.
In the interest of time, I'm not going to go through them, but you will be able to work through these examples both in Section and on your own.
So, to end off, the mutations in exons that you should know are silent mutations that don't change the sequence of the protein, nonsense mutations that I've just covered, something called missense mutations which change the amino acid sequence, and something called frameshift mutations which also are likely to change the sequence of the amino acid.
OK?
As I say, this will be covered.
If you want to come and see me personally in office hours tomorrow or the next day, please do, and I'll go through these examples with you.
OK.
I think we'll get started.
We're moving into another phase of the course today talking about cell biology.
There are going to be three lectures on cell biology covering a variety of topics.
So we've been talking about cells in various contexts over the course of several lectures, actually, but more or less we've talked about them as static objects that carry out replication and transcription and translation and so on and so forth.
But, in fact, as I'm sure you are aware, cells are highly dynamic structures which do many things at all times.
And this is a movie from which shows the movement of cells.
Cells will move randomly when you place them in a dish, but they'll also move in a directed fashion in response to factors in their environment that attract them or they'll move away from factors that repel them.
So cell movement is an example of a dynamic process.
Cells in the immune system, for example, are cells that do a lot of very critical movement to get to the site of a wound, to get to the site of an infection, and then to elicit the proper response to that infection.
It's a late arriving crowd today.
Another aspect of the dynamic nature of cells is the fact that they divide.
And I've actually shown you this movie previously.
From early embryogenesis through the formation of the fully formed organism and then throughout adult life, cells are receiving signals to divide at the appropriate times and to stop dividing when it's not longer necessary.
This is a process of signal transduction, receipt of a signal which tells the cell it's time to do something, and then to process that signal in a way to leads to the proper cellular response.
Maybe one of the most well understood or familiar examples of this process of signal transduction is the neuron depicted here in its actual state filled with a dye so you can see its various processes and diagramed here.
Neurons, of course, are part of the information transfer in the nervous system where signals are received from these structures here called dendrites.
That signal is then propagated through the body of the neuron through these long structures called axons and then released.
The signal finally acts upon the structures at the very end of these nerve terminals leading to the release, for example, of neurotransmitters.
These are impressive cells for a variety of reasons.
And Professor Sive will actually give you some lectures which deal more specifically about signaling in the nervous system.
One of the ways that they're impressive is their sheer length.
You have neurons in your body that are more than a meter in length, and whales have neurons in their body that are tens of meters in length.
So these are remarkable cells which carry out very specific functions.
So basically what we want to talk about over the course of today's lecture, we'll get there eventually, we have to do some other stuff first, is the process by which cells respond to a stimulus.
And then, much as a computer would do, to process that signal in very sophisticated ways, which I'll introduce to you this time and talk about in greater detail next time as well, to lead to a particular cellular response.
And I've given you three examples of cellular responses in the movies and images this morning.
For example, the cells might respond by moving in a particular directed fashion.
They might divide because it's time for them to divide.
Or they might, in response to this signal, release a neurotransmitter -- In the peripheral nervous system or in the brain.
So what we want to understand is what does this really mean?
What are the mechanisms that cells use to process signals?
And you'll actually see analogies to integrated circuits as we go through some examples.
Before we get there, we need to understand a little bit about how cells are built.
Largely, but not entirely, the structure of a cell is related to its function.
So neurons are an example.
Their particular structures allow them to do the things that they do.
Erythrocytes are another example.
Small red blood cells, which are oxygen-carrying cells, are built the way they're built to do what they need to do.
And largely, but not entirely, a cell's structure is determined by what proteins are expressed.
And also where those proteins are found inside the cell.
Where those proteins are localized inside the cell.
So I want to spend a little bit of time talking about protein localization.
So I want to spend a little bit of time talking about protein localization, how is it that proteins within your cells get to where they're going, in a sense know where they're supposed to go.
And a critical tool for understanding this process of protein localization -- -- is fluorescence imaging.
And so I think you're all aware of standard light microscopy where you pass white light through a sample, let's say a cell or a collection of cells, and based on the absorbance of the light through the different structures within those cells you're actually able to visualize what the cells look like.
With fluorescence microscopy, what you do is shine UV light onto the cell, which brings energy to the cell and allows molecules that will fluoresce in response to that wavelength of UV to be visualized.
So if you have a fluorescent signal within the cell, you can see where that signal is coming from using fluorescence microscopy or fluorescence imaging.
So how is it that we get fluorescence from the particular structures, proteins or other things that we want to see inside the cell?
Well, there are two general ways of doing that.
The first is fluorescent antibodies.
So let's imagine we have a protein of interest.
We want to know where it is inside the cell.
And we're able to make that protein, say, in bacteria.
We can inject that protein into a bunny, that's a bunny, or into a mouse, that's a mouse.
OK.
These are the common organisms used for production of antibodies.
We'll actually talk about how antibodies are made and how they recognize the things they recognize in subsequent lectures.
But I think you all know that antibodies are made by the body to recognize foreign stuff.
In this case, this protein that we've produced is foreign to the bunny or to the mouse.
And so in response to that it makes an antibody, or a series of antibodies.
These are proteins which have a particular structure that we often diagram like this.
They have four chains, two big ones and two little ones.
So these are antibodies.
And importantly the antibodies are specific.
That is they will bind to this protein but they won't bind to other proteins.
So they specifically recognize this protein which is called the antigen in the context of antibodies.
We can purify these antibodies and then chemically modify them to add a fluorochrome.
A chemical moiety that will fluoresce in response to a particular UV wavelength of light.
We can then apply those fluorescently labeled antibodies to cells that we've put onto a glass slide and actually poked holes in to allow the antibody to access the inside of the cell.
And then we can visualize where the fluorescence comes from within the cell using fluorescence imaging microscopy.
So, for example, if the protein of interest is in the nucleus, and we have labeled our antibody with a fluorochrome that fluoresces in the green wavelength, now when we visualize this cell by fluorescence microscopy the nucleus, if the protein is nuclear, will look green.
OK?
And this has been used in cell biology for a very long time, and it's still used today.
It's a very powerful method for determining protein localization.
This method is called immunofluorescence.
Immuno for the antibody and fluorescence.
And you can actually use multiple antibodies labeled with different colored fluorochromes and find multiple proteins in relation to one another.
So that's one method.
To an extent, this method has been replaced by a new method which relies on the fact that there are naturally fluorescent molecules.
Naturally fluorescent proteins, I should say.
You've actually all seen these.
If you go to the beach, you might have encountered jelly fish which fluoresce.
And they fluoresce because they make a protein which itself fluoresces when activated by the proper UV wavelength.
These proteins have been isolated from these species, cloned into plasmids, and then used in recombinant DNA techniques, like I told you about before, to visualize where other proteins go inside the cell.
And I'll give you some examples of that in a minute.
But here's the jelly fish naturally.
Here are these structures.
The jelly fish actually makes a light which then causes these protein bundles to fluoresce green.
The gene was cloned, as I said.
It was given the name GFP for green fluorescence protein.
And a cDNA copy of the mRNA is used in molecular biology all over the place to do an experiment like this.
And I'll take you through an example of another one like this in a moment where you can actually fuse your gene of interest, gene X, cDNA corresponding to that gene with the GFP cDNA to make a fusion protein that has your gene of interest, plus GFP as one long polypeptide.
And now this protein will fluoresce.
And you'll be able to visualize it.
And, again, I'll give you an example of how that's done in a moment.
We've also used this, we, the field has used this to make organisms, other organisms which fluoresce.
In my lab, for example, we've mad green mice which express the GFP protein in the cells of the skin of the mouse.
So if you shine UV light on the mouse it turns green.
It's kind of cool.
And others have made pets that fluoresce.
Surprising, but true, you can actually buy a fluorescent rabbit.
I'm not sure who sells them, but they are available commercially.
And I decided to show you a picture of one, of a rabbit called Alba the fluorescent bunny, which should be showing here.
But Alba is actually a little bit shy so she doesn't always come.
Alba the fluorescent bunny.
Oh, here she comes.
So this is actually a fluorescent rabbit which was made by introducing this GFP gene into the cells of the skin, such that it expressed in the cells of the skin of the rabbit.
So it's a very powerful technique, but let me give you a few more examples of how it might be useful.
I know.
Everybody now wants one.
So if you have a cDNA for GFP, and actually there are clever molecular engineers who have modified this gene GFP to produce fluorescent proteins that fluoresce in the red wavelength.
So they're RFP, red fluorescent proteins.
There are yellow fluorescent proteins.
There are blue called scion CFP fluorescent proteins.
So there's a whole kit, a whole coloring box, if you will, of different color fluorescent proteins that you can play with.
You can then add that, in techniques that you are familiar with, to a cDNA of your gene of interest.
So this is a cDNA of your gene of interest.
If you now combine those using recombinant DNA techniques that we told you about in previous lectures, you could make a plasmid which has GFP fused directly in frame, to make a long open reading frame with your gene of interest.
OK?
And in that plasmid you would also include a promoter to allow the expression of this fusion.
And it's also necessary to have, at the other end, a polyadenylation sequence, again, to insure proper expression of this fusion protein in the cells.
And this would then be housed inside of a plasmid vector.
OK?
We could then transfect, introduce, using methods similar to transformation, but now not into bacteria.
Instead into mammalian cells.
We could introduce this plasmid into cells in culture so that in the nucleus of these cells that I've modified, I now have this new gene present which can express a green fusion protein with my gene of interest.
Now, importantly, the goal of this experiment is to understand where the protein that's encoded by my gene of interest is localized.
And the way that I'm going to do that is to follow where the green color goes.
If it's a cytoplasmic protein the green will be in the cytoplasm.
If it's a nuclear protein the green will be in the nucleus and so on.
So if I now look at this cell by fluorescence microscopy, I will see the green signal in the nucleus.
OK?
It's a very, very powerful technique that allows us to follow the products of genes of interest wherever they might go inside of cells.
And I just made a collection of images from the literature.
Here you can see, actually, two different proteins being visualized simultaneously.
The blue is actually a DNA stain.
That shows you were the nuclei are.
The green is one particular protein.
And you can see that it's largely cytoplasmic.
And the red is another protein rimming the plasma membrane of these cells.
There are a whole bunch of cells here.
You can see their shape by where the red color is.
And the green is then marking the other protein which is largely cytoplasmic.
In this example, similar to what I drew on the board, the fusion protein is in the nucleus.
So the fusion protein is a nuclear protein.
It drags the GFP with it into the nucleus.
This is a structure called the endoplasmic reticulum, the ER.
We'll come back to it later in the lecture.
It's involved in the sorting of proteins to different parts of the cell.
And here's an ER protein that's been fussed to GFP.
So now the signal is coming from this structure inside the cell, the endoplasmic reticulum.
And here is a protein which resides in the mitochondria, that energy-producing organelle inside the cell.
You can drag GFP into the mitochondria to highlight those structures.
Here's a red example, RFP fused to a cytoskeletal protein, so you now see these long filaments inside the cell now fluorescing red.
And here's another kind of neat example.
This is a protein that moves.
It's normally present in the cytoplasm.
As you can see here it's filling the cytoplasm of the cell and it's not in the nucleus which is labeled green.
However, in response to a signal the protein moves.
It goes from the cytoplasm into the nucleus.
It might be, for example, a transcription factor, which is kept in the cytoplasm so that it won't turn on the expression of its target genes.
But in response to the signal it now moves from the cytoplasm into the nucleus such that it can access the DNA and begin to turn on the expression of those genes.
OK?
So proteins can be followed dynamically.
And you can actually do movies of these things over time.
OK.
So proteins can be moved from place to place.
How do they get there?
What is the mechanism by which proteins move from one compartment of the cell to another?
How do they know where to go?
There are two answers to this question.
One is signals in the polypeptide.
Signals within the polypeptide tell the protein where to go.
These signals can be short, a few amino acids, four, five amino acids, they can be a little longer, 15 or 20 amino acids, but they mostly function as a molecular zip code.
They can be read by the cell's protein sorting machinery.
Like a post office reads a zip code, it knows to send certain letters to certain places.
This machinery reads these signals and sends proteins to different places within the cell.
There's also a separate and sometimes related mechanism -- -- known as post-translational modification.
Here it's not a sequence within the protein amino acid sequence.
It's not a signal within the amino acid sequence, but rather it's something that's added onto the protein after it's been made.
That's why it's called post-translational.
And, again, this can influence where the protein ends up inside the cell.
So let me give you some examples of what I'm talking about here.
With respect to signals that are contained within the polypeptide, proteins that are synthesized in the cytoplasm on ribosomes, this is a ribosome, this is a mRNA, 5 prime end, 3 prime end, will give rise, through the process of translation, to proteins.
And when this process is completed, those proteins will be released into the cytoplasm of the cell.
OK?
Most translation occurs freely inside the cytoplasm of the cell.
There are exceptions to that, as I'll show you in a moment.
Most of it takes place freely in the cytoplasm of the cell, so when the process is done protein winds up in the cytoplasm.
These proteins have two fates generally speaking.
One is that they remain cytoplasmic.
And I gave you some examples of cytoplasmic proteins earlier.
Lots of the enzymes that are present inside your cells just hang out inside the cytoplasm and do their job.
Other proteins are synthesized first in the cytoplasm will end up going to the nucleus, and these proteins are distinguishable by virtue of the fact that the ones that end up in the nucleus have a little tag on them.
And this little tag has a name called the nuclear localization sequence.
This is one of the short tags.
It can be just four or five amino acids, usually positively charged.
And inside the cytoplasm of this cell there are proteins which will recognize that little tag.
Attached to it these proteins belong to a class known as importins.
And what the importins do is drag the protein that has an NLS to the nucleus where they interact with a structure within the membrane of the nucleus known as the nuclear pore complex.
And importin then drags the protein into the nucleus.
It imports it into the nucleus.
So nuclear proteins get into the nucleus by virtue of the fact that they have a nuclear localization sequence on them.
OK?
And you might imagine, and probably you will think hard about how you could visualize this process of nuclear importation, for example, in a problem set.
So that's how cytoplasmic resident proteins are distinguished from nuclear proteins.
What about the proteins that fully leave the cell?
Many proteins that you make actually get secreted.
They leave the cell entirely.
Still other proteins wind up on the membrane itself.
We're going to talk later in the lecture and next time about receptors that sit on the outside of your cell waiting for signals to be sent.
Those proteins have to get there, and there's a special mechanism that insures that.
So this applies to both secreted, as well as what we refer to as transmembrane proteins.
And here the situation is a little bit different.
In this case, when the ribosomes synthesize the very first few amino acids, the very first polypeptide chain, at the end, the end terminal end of these polypeptides, as they're emerging from the ribosome there is a longer, more like 15 or 20 amino acid sequence known as the signal sequence.
If a protein has a signal sequence on its end you know it's going to end up as a secreted protein or a transmembrane protein.
And when it emerges from the bottom of the ribosome, as depicted here, there's a specialized protein that will bind to it, it's actually a complex of proteins called the signal recognition particle.
SRP has a very important function, actually two important functions.
Firstly, when it binds to the ribosome, when it binds to the signal sequence like this it leads to translational arrest.
It stops translation.
It blocks the process of translation right at this point.
OK?
The next thing that happens is that it drags this complex of the ribosome with its little nascent polypeptide to a structure known as the endoplasmic reticulum.
In the membrane of the endoplasmic reticulum.
That was really loud.
Is it that phone?
I got to do this the other day in a class.
Hello.
Oh, they hung up.
Last time I was teaching a class and the computer guy left his cell phone, and it rang.
And the name of the person calling actually showed up on the screen, so I picked it up and I said Herb.
And we actually talked for about five minutes and I told him I had to go and keep teaching a class.
Anyway, inside the endoplasmic reticulum there's another receptor, not unlike the nuclear pore complex.
This is called the SRP receptor.
It binds to this thing SRP, signal recognition particle.
And this then allows SRP to go away and allows translation to continue allowing the protein to be synthesized through this membrane and end up inside the endoplasmic reticulum.
So the end result of this process is that the protein will now reside within this structure, this membrane-bound structure.
Subsequently, the protein leaves this structure in a vesicle.
It buds off the membrane, gets carried like cargo in a little pod coming off the mother ship here.
It goes to another sorting structure called the Golgi apparatus.
And it eventually goes to the plasma membrane where it fuses to the plasma membrane allowing this secreted protein to leave the cell.
I realize that this is a little bit messy, but there are comparable pictures in your book.
The point being that these proteins have a signal which drags them to the ER.
There's then a sorting process that takes them out of the cell, takes them to the plasma membrane and then out of the cell.
So that's how you deal with secreted proteins.
How do you deal with proteins that end up in the membrane?
If they don't get secreted they wind up resident in the plasma membrane.
This happens by virtue of another signal.
And I'm going to pick up this process, the beginning of the process is exactly the same.
I'm going to pick up this process at a slightly later point where we have the SRP receptor bound by a ribosome.
It's actually partway through the process of producing a polypeptide that has wound up inside the ER.
For proteins that wind up ultimately in the membrane there's another sequence called a stop transfer sequence.
And when a stop transfer sequence is hit and recognized by this complex it causes the ribosome to disassociate from the mRNA, or rather allows the synthesis to occur without continuous transfer through this complex.
So the protein just gets built on this side of the membrane.
And the result of that then is a protein that ends up halfway in and halfway out of the endoplasmic reticulum.
If you then take this through the sort of sorting steps that I just outlined for secreted proteins, this protein will end up with part of it on the outside and part of it on the inside of the cell.
OK?
So that's how we deal with proteins that are transmembrane.
And you'll actually see proteins that go through the membrane multiple times.
That's because they have signal sequences and stop transfer sequence, signal sequences and stop transfer sequences that allow them to transition through the membrane more than once.
Many of the proteins just once but other proteins multiple times.
OK.
So that's the way it's done in terms of signal recognition.
I also mentioned that there is this class of posttranslational modification.
Proteins get modified after they're made by translation.
And the examples include proteolysis.
The protein might actually get cleaved, cut.
Half of it goes one place.
The other half goes somewhere else.
Phosphorylation.
Addition of a phosphate can determine the localization of a protein.
Glycosylation.
Addition of sugar groups, carbohydrates onto the protein can determine where the protein goes.
And finally lipid addition.
Addition of a lipid molecule can determine where the protein goes.
And I'll give you one example of that last one.
A protein that we work on in my lab, the protein called RAS, which will actually come up in the next lecture.
It's a signaling protein, which actually turns out to be very important in cancer.
In its unmodified state the protein is cytoplasmic.
However, it gets modified by an enzyme called farnesyltransferase.
Which modifies the very end of the protein, the C terminal end of the protein, and it sticks onto it a lipid group, about a 20 carbon lipid group, which is very much like the fatty acid side chains in the plasma membrane.
And, in fact, this is a very hydrophobic structure.
And it ends up dragging the protein to the plasma membrane.
Here's the plasma membrane.
I'm drawing it now as a lipid bilayer, whereas I drew it as a single line previously.
And that lipid tail, which has been stuck on the RAS protein, inserts into the lipid bilayer and localizes RAS then to the plasma membrane.
OK?
So here are examples of post-translational modifications that effect where the protein ends up going.
So we can determine where the protein goes, that's very important.
I also want to make the point before we move onto the next topic that protein-protein interactions are very important in determining function.
These proteins that we've been talking about almost never, probably never do their functions by themselves.
They interact with other proteins.
And these are determined, again, by sequences that bind to one another between the two proteins.
So this is actually a protein structure from a colleague here at MIT, Thomas Schwartz, solved by him.
What you're going to look at hopefully, this is a ribbon diagram of two proteins, the one in blue and the one in pink.
And you can see that these two proteins bind to each other based on amino acid residues at this interface between the two proteins.
This ribbon diagram will be replaced by a space-filling model.
This is what the protein really would like inside the cell.
And here's that interface more clearly visualized with residues in light blue on one side, pink on the other that allows the two proteins to interact.
So protein-protein interaction is extremely important, allowing proteins to interact either for a long time because they carry out some essential function together or very transiently.
Protein-protein interactions can be long-lasting or very transient.
And to just give you a visual reminder of that, the transient interaction, this I took a couple of years ago, Johnny Damon getting run into by Damian Jackson.
A very transient but actually quite important interaction between these two baseball players.
And then there can be longer lasting interactions.
Here's Brad Pitt, well, it's not that long-lasting, I guess, but the point is that proteins do interact either very transiently or in a long-lasting way.
Jennifer actually did the cutting out herself of this picture that she sent me.
OK.
Finally, proteins can change their shape.
Proteins can change their shape in response to binding of other proteins or binding of cofactors that they interact with.
And here's another movie from Thomas Schwartz.
This is a signaling protein, much like the RAS protein actually.
And you'll see that upon binding to a nucleotide here, the protein dramatically changes its shape.
It actually unwinds a little bit and exposes a new structure for binding.
And this is one of the critical ways that proteins participate in this signal transduction, signal processing function.
They receive signals, change their shape, and this allows them to bind to other proteins or carry out other functions.
OK.
So I want to move now to the signaling introduction for the last 12 minutes or so.
I started the day with this slide of cells moving around.
They're receiving signals from one another or from the medium and they're responding to it.
I also gave you examples of cells dividing or not dividing, cells differentiating, turning into one type of end-stage cell or another, changing their function.
We'll talk later about cells that receive signals and kill other cells, cells that receive signals and commit suicide in a couple of more lectures.
So cells will do very complicated things in receipt of various signals.
And this general process is known as signal transduction.
And basically it's the ability of cells to respond to things in their extracellular environment, stimuli in their extracellular environment.
These could be physical stimuli, light, odorants, or they could be biological stimuli, peptides, proteins, hormones, which then get sent first from the outside of the cell and then to the inside of the cell leading to either short-term responses -- I'll give you an example of glucose secretion in response to a hormone epinephrine or adrenalin.
This happens exclusively inside the cytoplasm of the cell.
The signal is received, processed, and the result is the secretion of glucose.
Other signals will make their way into the nucleus of the cell, and this might lead to changes in gene expression.
New genes are turned.
Others might be turned off.
Changes in gene expression.
And this might lead to a longer term response.
And we'll give you an example of this in the next lecture.
So how does this work?
Well, it's different in every way.
Each example is slightly different.
I show you this slide to give you sort of a visual picture of what it's like, we think.
Like one of these, what do they call them, Kineto-something or rather sculptures that you see at Logan Airport, for example, where a small perturbation at one end is propagated through a series of devices and leads to sounds and whistles and movements taking place within the structure.
It's not unlike dominos.
When you flip a domino at one end it can spread, increasing its effect because of the involvement of additional dominoes as it goes along a complex structure.
We actually found, as we were looking at the Web for pictures of this movie of Mr. Domino, does anybody know what Mr. Domino is?
I had no idea.
It's a computer game, and they have a cool little animation which I thought I would show you.
Here's Mr. Domino.
And it's actually kind of relevant because he'll go and his signal is being transmitted along a linear path involving other types of molecules spreading out.
Again, other types of molecules being involved, and so on and so forth.
It's not a great example but it is kind of fun.
OK.
So before I give you a specific example, I want to give you some principles about signal transduction.
And, again, the principles that are relevant here in biology are not dissimilar to signal propagation in electrical engineering, for example.
Signal transduction in its simplest form means that something which takes some form, like a ligand binding to a receptor on the surface of the cell, changes its form inside the cell.
It causes a change at the membrane to produce a different type of response inside the cell.
Importantly, for almost all of these signal transduction pathways, what happens at the membrane is usually inadequate to change whatever process is to be changed inside the cell such that it is necessary for signal amplification to take place.
And we'll give you examples of how signal amplification occurs in biology.
But rather than making one square, in fact, you might make multiple squares.
There's also the opportunity, and it's often the case for signal diversification, so that in response to the same signal, in fact, processed slightly different ways temporally or spatially with inside the cell you might make, sorry, let's make this a circle.
You might make two different responses that coordinate the overall cellular response.
There's the opportunity for what we call signal integration.
So with respect to signal integration, you might imagine one signal which produces a particular response.
A separate signal which produces a distinct response.
But if you add the two together, that is if both signals are received simultaneously, you get some third response.
So signal integration, again, not unlike electrical engineering.
And finally signal modulation.
So let's imagine again that we have, in some circumstances, a particular signal giving a particular response.
If, in a different context with a different signal being received by the cell at the same time, instead of producing this response, there's an inhibitory effect so that nothing is produced.
This signal inhibits the processing of this signal, so this is inhibitory.
And you could imagine, and we know exists, the opposite such that in the presence of this signal plus now a distinct signal there's an augmentation of the response so that now you make much more of whatever you were making.
So this is augmentation.
So all of these things happen in the process of cellular signal transduction, cellular signal processing.
At the beginning of this process, for at least the examples we're going to talk about, things happen at the plasma membrane.
So here is the plasma membrane again.
You can think of the plasma membrane as sort of the outside of your house with the antennas sticking up into the air waiting for signals to be received.
Well, in biological circumstances, those antennas are polypeptide receptors.
These are in this category.
Or proteins that have transmembrane domains, get stuck in the membrane, and end up on the plasma membrane.
These polypeptide receptors are sitting out there.
There are often thousands of a given polypeptide receptor type on the outside of a cell.
And these bind to ligands that fit inside them.
There's complementarity between the shape of the ligand and the receptor such that you get specific binding of the ligand to the receptor.
The consequences of binding of ligand to receptor we generally think of now as changing the shape of the receptor.
Something that happens on the outside through binding of the ligand to the receptor is propagated through this polypeptide chain to a change in the structure at the inside.
OK?
It's as though I took this guy's head and I twisted it.
Eventually his feet are going to move.
The same thing here.
If I make a change out here, I can propagate that change through the polypeptide chain and change the inside structure.
So if you imagine a ligand which has a particular structure, I'm sorry a receptor that has a particular structure, when you add the ligand you now change the structure.
And take a protein that might be inactive, this might have some enzymatic activity, from inactive to one that's active.
Or another example, if I take this receptor and I bind the ligand, change its structure, I might now allow, based on this change in its structure, an interaction between this protein and some other signaling molecule, some other signaling protein.
And, again, we know examples of all of those things.
Usually, maybe always, the change that takes place at the plasma membrane, like the first domino that you flip, doesn't have enough energy associated with it to change the biological response.
It's necessary to propagate that signal and to amplify that signal through various stages.
This process of signal amplification is very important.
And there are two general classes of signal amplification.
One is the production of secondary signaling molecules called second messengers.
And the example that we'll give you next time is the second messenger cyclic AMP -- -- which can be produced in large amounts following the binding of a ligand to a receptor, thereby amplifying the signal.
And the other, which we'll give you as two examples next time, is enzymatic, very often kinase cascades.
Enzymatic cascades where you turn on the activity of one enzyme, which turns on the activity of more other enzymes, which turn on the activity of still more other enzymes and so on.
So, we'll talk about specifics related to these next time.
We're going to continue our discussion today about cell biology, and specifically about the subject of signalling which we talked about in general terms last time.
I gave you some of the general principles about signaling, signal propagation and signal processing in biological systems.
And today we're going to talk about two specific examples.
First a short-range response which results in the production of second messenger cyclic AMP and the results of that in terms of downstream events.
And the second will be an example of a long-range response which requires new gene expression, specifically in the stimulation of cells to divide.
And the example I'll give you in the context of wound healing.
So just to finish up what I started at the end of last lecture.
As I mentioned, the energy associated with the binding of a single ligand to a single receptor molecule at the cell's surface is insufficient to trigger any of the downstream events that are necessary either for the short-term or long-term responses.
And so it's necessary that the signal be amplified.
And there are various ways that this happens.
The first, which I mentioned last time, is the production of molecules which we refer to as second messengers.
Second because they are not the primary thing.
The primary thing is the ligand binding to the receptor.
They come after that messenger because they transmit the message related to the binding of the ligand to the receptor inside the cell.
And the example that I mentioned and that we'll talk about in more detail today is a second messenger called cyclic AMP or cAMP.
I also mentioned that there are examples of enzymatic cascades.
And, again, the purpose here is to amplify the signal.
One enzyme turns on another enzyme which turns on many more enzymes, et cetera.
It's like a pyramid effect where something little grows to something big.
And the example I gave you last time was the analogy to dominos.
Pushing one over basically spreads the signal down the chain.
So signal amplification with respect to second messengers, the principle is quite simple.
Imagine an enzyme which is in an inactive state.
The binding of ligand to receptor makes that enzyme go to an active state.
And in its active state it can convert a substrate into a product.
Now, if that product can stimulate a second enzyme, imagine enzyme two, which is in an inactive state now in the presence of this product, is converted to an active state, you've amplified the signal.
You've turned on a lot of enzymes which make even more of this product which now stimulate yet another enzyme.
So this process then results in a form of signal amplification.
The same is true for enzyme cascades.
If you imagine an enzyme which is in an inactive state.
The binding of a ligand to a receptor converts that enzyme to an active state.
If that enzyme now acts on another enzyme, E2 goes from inactive state to an active state, and that enzyme then acts on another enzyme, E3 goes from an inactive state to an active state, again, you've gone from a relatively small amount of stimulus propagated through the stimulation of this enzymatic activity, which acts on a number of these to make even more of these active, which acts on even more of these to make even more of these active, again, one develops signal amplification.
OK?
Now, these changes that take place, both in the generation of second messenger, as well as in the turning on of enzymes, often result from changes in protein states.
I tried to give you indications last time that proteins are not static things.
They can move, they can interact, and they can actually change shape.
And changes in protein states are often reflected in changes in protein structure, subtle changes in protein structure, but nevertheless changes in protein structure.
And these can come in noncovalent forms.
The protein is not changed by any new covalent bonds but by binding to something in a noncovalent interaction.
And the classic example of this are G-binding proteins.
These proteins bind guanine nucleotides.
That's why they're called G-binding proteins.
And they're often shortened and referred to as G proteins.
We'll meet two G proteins later in today's lecture.
G proteins change their structure depending on whether or not they're bound to GTP or GDP.
Typically, in their inactive state, they're bound to GDP.
However, in response to a signal transduction pathway, there's an exchange event whereby GTP binds to the molecule and GDP is released.
This reaction is usually catalyzed by another protein known as a nucleotide exchange factor.
We're exchanging this nucleotide for this nucleotide, and in so doing we change the shape of the molecule.
When it's bound to GTP now it has a different shape.
And by virtue of this different shape, it might now be able to interact preferentially with another molecule.
So imagine that we have a signaling molecule which has a shape that looks something like this.
A signaling protein.
Given the lack of complementarity between this shape and this shape, these proteins will not interact.
However, upon nucleotide exchange such that the protein now binds GTP and now changes its shape, now we can get a productive interaction between the G protein and the signaling molecule.
And this can lead to a form of signal amplification.
OK?
So that's an example of a noncovalent change, and a reversible one.
The GTP can be hydrolyzed -- -- back to GDP, thereby shutting the signal off.
OK?
Another example of a structural change in a protein but now a covalent one -- -- is in the form of phosphorylation.
So imagine that we have a protein here, an enzyme which is in its inactive state.
And this protein has among its amino acids, in some critical position, an amino acid with a hydroxyl residue.
Does anybody remember back in the Dark Ages when you were learning your amino acid structures which amino acids have hydroxyl residues?
I'm tempted to offer money to anybody who would remember all three.
Anybody.
Does anybody remember even one?
Serine it is.
Yes, that's one.
Serine.
Anything else?
Tyrosine is another.
Very good.
And the last one?
Threonine.
So these three amino acids have hydroxyl residues.
And by virtue of that they are subject to, or can be subject to protein phosphorylation at the hands of enzymes, which I've referred to in previous lectures, called protein kinases.
These proteins can be modified such that they have a phosphate group on them.
And this can change the structure of the protein and make it go from an inactive to an active state.
There are actually examples where it goes the other way, phosphorylation takes an active protein and makes it inactive, but you see the point.
Phosphorylation can change the activity.
And, once again, this is reversible.
There are protein phosphatases which will remove the phosphate and return the protein to its baseline state.
OK?
So just to give you an example of how this might be useful.
Imagine that I have an enzyme here which has an active site, but this active site, by virtue of the confirmation of the protein, is closed.
That is the substrates cannot get in there in order to be modified.
At the hands of a specific kinase that might recognize a hydroxyl residue right here, the shape of this protein might change.
So now the active site becomes open and the enzyme becomes active.
OK?
And likewise this can be reversed through the action of a phosphatase.
OK?
So protein structures change, activities of proteins, enzymes change.
And that's one way in which we can propagate signals.
OK.
So let's now go to a specific example.
And the example that I want to give you is one that hopefully not many of you are familiar with.
If you're camping and you run across this fellow here, what is your immediate reaction?
You run.
You run.
You never try to feed the bear a marshmallow with your mouth.
That's the first thing I learned about camping, you run.
This is the famous fight or flight response.
And when you're dealing with a bear, you want to take the flight option.
It's stimulated by fear, some perception of fear.
And upon perception of fear -- -- through a complex physiology, a small molecule is released from your adrenal glands.
Glands that sit on top of your kidneys.
This molecule is called epinephrine, otherwise known as adrenaline.
So this is released from your adrenal glands and various things happen.
One, that we're going to spend a little time talking about, is that your blood glucose levels go up.
Why?
Why do you want your blood glucose levels to go up when you see this fellow here?
So you can run.
Remember, that's the first thing you want to do when you see a bear, run.
In addition, your heart rate goes up so you can pump more blood to your big muscles, your breathing rate goes up so you can oxygenate those muscles, and your blood pressure goes up.
And there are other things that happen as well, but these are the most important with respect to the fight and flight response.
This happens because this hormone, epinephrine, triggers all of these responses, actually in different cells within your body.
Because it's acting at distance sites within your body, from the adrenal glands to the muscles to the liver to the heart to your brain, this is referred to as an endocrine response.
An endocrine response is basically defined as hormones acting at a distance.
OK?
Produced somewhere.
Acting somewhere else in your body.
We differentiate that with the paracrine response in which there is a local release of growth factors and other ligands.
So here there's a local issue.
Something happens locally, factors are released right in the environment, and they act upon cells right in their neighborhood.
And finally a related turn called autocrine which is cells -- -- which release the factor and bind it themselves.
Autocrine.
So they stimulate themselves.
And very often you'll start a process one of these ways.
And then the cell will take over.
Basically amplifying the signal for itself.
OK?
Now, in the example that I want to talk about in the first portion of the lecture.
We're going to deal with this one here, blood glucose, the increase levels of blood glucose.
As you probably know, your body stores energy different ways.
One way is in a polymer of glucose called glycogen, which is stored in your liver.
Another way is through the storage of fat.
For this response we want to liberate the energy that's stored in the form of glycogen and turn it into glucose which is much more consumable.
It can be used directly by your muscles to make ATP which drives various reactions like contractions and so on.
So the analogy here is money in your bank account versus money in your wallet.
When you're in a situation like that, you want to transfer money from your bank account to your wallet.
That's the direction we're heading in this example.
In others situations, where you've just had a huge meal and you're not running from a bear, you go the other way.
You take glucose and you turn it into glycogen, which I said is a polymer of glucose.
Now, this balance between glycogen and glucose is controlled by enzymes which are sort of in the center of today's example.
In this direction the enzyme is glycogen phosphorylase.
Glycogen phosphorylase is the last enzyme that breaks down glycogen into a form of glucose that can be then further metabolized.
Glycogen phosphorylase.
And the other, in the other direction, is an enzyme called glycogen synthase.
OK?
So the goal of this reaction, or set of signals, will be to stimulate this process and actually inhibit this process.
And you'll see how that's done in a second.
This is epinephrine.
Again, it's a small chemical produced by your adrenal glands.
As I said earlier, it's released during the flight or fight response.
It binds to a class of receptors which I'll take you through.
They're called beta-adrenergic receptors because they bind to a factor produced in the adrenal glands, beta-adrenergic receptors.
These are receptors, as you'll see, that have a complex tortuous root through the membrane.
They go through the membrane seven times.
They're actually sometimes referred to as serpentine receptors for that reason.
And they bind to and activate GTP binding proteins, G proteins, as I mentioned down here.
And this is a particular form of G protein called trimeric G proteins.
And you'll see why in a second.
As I mentioned, epinephrine binds to cells in different parts of the body resulting in various physiological effects.
The ones I listed here and another one which I didn't list here which I find paradoxical.
Actually, I know what, I know what the evolutionary advantage to all of these things is, but I don't know what the evolutionary advantage is to the relaxation of the smooth muscles around the colon during the fight or flight response.
Some of you may have felt this before when you're extremely fearful.
I'm not sure why it is our bodies do that.
But anyway.
OK.
So how does this work?
So we're going to consider just the mobilization of glucose.
And the site of action is the liver.
That's where glycogen is stored.
And on the cells in the liver, the hepatocytes, on the plasma membrane of those cells, there are these receptors, these serpentine receptors.
They look like that.
They look like snakes, as I mentioned.
Beta-adrenergic receptors.
There are also alpha-adrenergic receptors, but we're going to focus on beta-adrenergic receptors.
And these bind to epinephrine directly.
So here's our molecule of epinephrine.
It physically binds to a portion of the polypeptide chain that's sticking out from the cell's surface.
Now, bound on the inside to this receptor is this trimeric G protein.
It's called trimeric because there are three subunits.
There's an alpha subunit, a beta subunit and a gamma subunit.
And in this configuration, prior to the binding of epinephrine, the alpha subunit is bound to GDP.
And, as I mentioned before, that's usually associated with inactivity.
So the alpha subunit is inactive.
The binding of epinephrine to the surface has two effects.
The first is an exchange wherein the alpha subunit now binds GTP.
And the GDP that was present there comes off.
The other effect is that the G protein, the alpha subunit releases from the beta and gamma subunits and they float away.
In this, now GTP-bound configuration, the alpha subunit is now active.
And it can move from this position to another position within the membrane and bind to another protein.
In this case an enzyme called adenylate cyclase.
The GTP bound version of the alpha subunit associates with adenylate cyclase, thereby converting it to an active form.
When bound to this it becomes active.
And the consequence of that is that the adenylate cyclase can now convert ATP to the second messenger cyclic AMP.
OK?
And this process that we've gone through from binding of a single molecule to the production of a lot of a cyclic AMP is an example.
OK?
We've made a lot of second messenger from the binding of a single molecule because we've turned on a lot of these, which can go on and turn on a lot of these, which can go on and make a lot of this.
Now, there's another step in signal amplification here in this pathway.
There's an enzyme known as protein kinase A which becomes activated in the presence of cyclic AMP.
So protein kinase A in the presence of cyclic AMP goes from an inactive state to an active state.
And when it becomes active it converts another enzyme, glycogen phosphorylase, which I mentioned to you a moment ago, from an inactive state to an active state.
OK?
So this is a phosphorylation reaction, similar to the one that I drew up here.
PKA phosphorylates glycogen phosphorylase, making it go from an inactive to an active state.
Another step of signal amplification.
We can make some of this active which makes a lot of this active.
OK?
And this is the enzyme we needed to turn on in order to break down glycogen to glucose.
Now, there's another kind of cool thing that happens in the control of the reverse reaction, because it turns out that the enzyme responsible for making glycogen, glycogen synthase is inactivated in the presence of cyclic AMP.
So it goes from an active state in the presence of cyclic AMP to an inactive state.
So at once we turn that off, turn this on, and we can generate a lot of glucose.
OK.
So this is the reaction.
We focused exclusively on the mobilization of glucose here.
Just to remind you, this same pathway controls heart rate, breathing rate, blood flow.
And, actually, some of you might have heard of beta-blockers.
And beta-blockers are used to control this exact reaction.
Beta-blockers bind to beta-adrenergic receptors so that your body won't respond to adrenaline or epinephrine when you don't want it to.
So people who get really nervous talking in front of audiences or taking exams in 7.
1 will sometimes take beta-blockers so that they don't suffer, you know, fast heart rate or this feeling of anxiety and so on.
OK.
So we've turned on this signal in response to binding of epinephrine to the cell's surface.
We've started all these happening.
How do we turn it off?
How do we deal with signal termination?
How do we turn the signal off?
Anybody have any ideas?
Well, one might be not so obvious.
And that is -- -- a process which I introduced to you last time called pathway modulation in which the activation of a second pathway can actually inhibit the activation of the primary pathway.
So, for example, I've been telling you about the beta-adrenergic receptors, which through the binding of this kicks off a series of events which results in the activation of this pathway.
There are many examples in biology where the binding of a second ligand to a distinct receptor results in the propagation of a signal which actually inhibits the first.
So, for example, I mentioned here that there's a kinase that's activated, protein kinase A.
This pathway might activate the relevant phosphatase, thereby turning this signal down.
So modulation of the pathway, very important in biology.
We're only really beginning to appreciate how important it is the more we learn.
Another obvious example is ligand diffusion.
You make a certain amount of ligand from your adrenal glands.
And it just diffuses eventually or breaks down, thereby turning the signal off.
There's also a phenomenon known as receptor internalization.
Very often when a receptor binds to its ligand, it doesn't just stay out there on the surface but actually gets internalized inside the cell.
So I'll draw a different class of receptors here.
Here's the receptor.
Here it's bound to its ligand.
I told you last time that proteins can get to the cell's surface through a complex sorting pathway that involves actually vesicles encapsulating the proteins and then bringing them to the cell surface.
Well, the reverse can happen as well where the proteins can be removed from the cell's surface into vesicles.
Removing them from the cell's surface then takes them away from the possibility of being activated, and thereby turning down the signal.
Another possibility would be to degrade the second messenger.
OK?
Cyclic AMP is an example.
You need to make it in order to stimulate these various downstream events, so get rid of it.
And, in fact, there's a clear pathway for that process.
As I told you, ATP can be converted through the actions of adenylate cyclase to make cyclic AMP.
But likewise there's an enzyme, actually, a whole class of enzymes known as phosphodiesterases which converts cyclic AMP to AMP, thereby inactivating the signal.
These enzymes are actually very important in pharmacology.
There are lots of inhibitors of phosphodiesterases which stimulate the production or the maintenance of high levels of cyclic AMP and other cyclic nucleotides.
So caffeine for example, one of the reasons you get a buzz from caffeine is that it blocks a class of these enzymes leading to more cyclic AMP which gives you that kind of nervous energy.
OK?
Another related process involving another cyclic nucleotide, this time cyclic GTP.
This is broken down by a different phosphodiesterase.
In Viagra and like chemicals are inhibitors of this class of phosphodiesterases, thereby leading to an over stimulation of that pathway.
And, finally, you can reverse the structural changes that I mentioned previously.
So I said earlier that enzymes can be activated through kinases.
And they can be inactivated by phosphatases.
I'm going to have to move to a different board here.
And G proteins which get activated in the beginning of these processes, from a GDP bound form, through the action of exchange factors to the GTP bound form, can be inactivated by GTP hydrolysis.
And this is often through proteins called GAPs, which are called GTPase activating proteins.
They activate the GTP hydrolysis activity of the G protein, thereby returning the G protein to its inactive state.
OK.
So that's the beta-adrenergic signaling pathway.
It's a classic.
You should know it in principle and in some detail.
It's covered extensively in your book.
Here's a figure from your book.
I won't go through the details of this because we just did them on the board, but just so you know it's in there.
Here's the production of the second messenger, cyclic AMP.
And then it triggers this kinase cascade that I drew up for you before, activation of PKA, activation of glycogen phosphorylase, and the production ultimately of glucose.
And here's the second messenger.
I'm pleased to see that I put this in the book, in my slide set last year because the book had an incorrect structure for cyclic AMP in the previous edition.
I'm pleased to see they know have corrected the structure.
This is the actual proper structure of cyclic AMP.
OK.
So now want to change signaling pathways and consider not what you do when you first encounter the bear, but what do you do, or what does your body do if you fail to successfully run from the bear?
So it's funny what you can find on the Web, boy.
You can find anything you want on the Web.
In fact, when I was looking up, I was pretty sure that Viagra blocked phosphodiesterases.
I was pretty sure, but I wasn't certain so I did a Web search.
This is true.
I did a Web search.
And, God, I cannot believe what I came across.
It's unbelievable.
But you can find some pretty nasty bear wounds on the Web as well, and here's one of them.
So this guy didn't run fast enough.
So what's going to happen?
What is his body going to do following an injury such as that?
Well, obviously he needs to heal the wound.
Your body is remarkable in its ability to detect damage and fix it.
So we're going to heal the wound.
And this is an example of one of these long-term effects of a signaling pathway.
Whereas, this was short-term, everything was happening in the cytoplasm, generation of glucose to be secreted.
This is a long-term effect.
The effect has to take place ultimately in the nucleus because we need to turn on gene expression to convince cells to divide and heal the wound.
This may be a troubling picture.
Maybe I should go back to the bear.
There.
So if we imagine the skin, which is a tight complex of cells forming a protective layer against the elements and so on keeping stuff out that should be out, in that should be in.
In a wound you breach that structure, you get a scratch.
And one of the things that happens, as you bleed into that wound, is that small packets, former bits of cells called platelets bind to the edges of that wound.
And the platelets then release a factor imaginatively called platelet-derived growth factor.
And platelet-derived growth factor, otherwise known as PDGF stimulates other cells to divide.
And specifically it stimulates cells called fibroblasts which sit underneath these cells normally as part of the connective tissue underneath the epithelial cells.
In the presence of PDGF, the fibroblasts proliferate.
And this results in the formation of a scar.
OK?
And what we want to understand now is how is it that PDGF stimulates fibroblasts to divide?
So we can model this process in cell culture.
We can extract fibroblasts from your skin and put them in a tissue culture dish, let's say one times ten to the fifth cells.
So this is cell number on this axis and this is days in culture on this axis, one, two, three, four days.
If we plate 100,000 fibroblasts in a tissue culture dish, in the absence of growth factors they won't divide.
They'll just sit there.
But if we add PDGF or other growth factors, the cells will divide, and therefore we'll see an increase in cell number.
So, again, the question is how does that happen?
What is the mechanism that allows that to happen?
Oops.
What did I do?
Well, here we're talking about another class of receptors.
These are called growth factor receptors.
And these are different from seven transmembrane receptors in that they have, on their cytoplasmic side, enzymatic activity, specifically kinase domains.
These receptors are also enzymes.
And their enzymatic activity is to add phosphates to other proteins.
In this configuration, however, they're inactive.
What the growth factor does is causes those receptors to dimerize with one another.
To become in close proximity to one another.
So the growth factor serves to link them.
Here's our growth factor.
And in this now increased proximity, these kinase domains become active.
And the consequence of that is we get phosphorylation of the neighboring protein.
We call this transphosphorylation.
So ligand induced activation, transphosphorylation.
And now what we've done, on the inside of the cell, is to create a structure.
And it's the formation of that structure that then allows the rest of the process to work.
These phosphorylated residues act as the binding sites for other proteins.
The first one is known as an adaptor protein.
An adaptor protein which binds to a class of proteins to that I referred to previously, a guanine nucleotide exchange factor.
And that then allows the exchange factor to stimulate the exchange of a small GTP binding protein, which is generally bound to GDP and inactive, to stimulate the exchange of GTP for GDP.
This small GTP binding protein is a protein called RAS, a very important signaling molecule.
And, actually, a very important cancer molecule as well.
Mutationally activated in a high percentage of cancers.
When you now have RAS in its GTP bound state, it can activate a kinase cascade.
Which I'll go through with you on slides in a moment.
It first activates a protein called RAF which then activates a protein called MEK which then activates a protein called ERK.
ERK is a map kinase.
It's a kinase itself.
Its enzymatic activity is to phosphorylate other proteins.
MEK phosphorylates ERK so it's called a map kinase kinase.
And RAF is a kinase itself which phosphorylates this kinase so it's referred to as a map kinase kinase kinase.
I'm not making it up.
OK?
So it stimulates a kinase cascade, similar to what I introduced you to before.
And then, as you'll see, ERK, which is a cytoplasmic protein, moves from the cytoplasm into the nucleus.
I'll show this to you on slides, and it's in your book as well.
And in the nucleus this kinase phosphorylates transcription factors which then bind to target genes turning them on stimulating the cell to divide.
I would have drawn that for you but we're running out of time.
So I will show it to you instead on the overheads.
There's our wound.
So here's the pathway.
It's just straight out of your book.
Here are the growth factor receptors, again, not dimerized, not active.
The growth factor causes the two proteins to come together in close apposition.
This causes the kinase activities to get stimulated resulting in the phosphorylation of the cytoplasmic tails.
The adaptor protein then can bind to the phosphorylated tail.
This then stimulates through an exchange factor not shown here, the exchange of GDP for GTP, making an active RAS molecule.
This then can activate RAF making it now an active kinase which activates MEK making it an active kinase which activates ERK called map K here for map kinase.
This now goes into the nucleus of the cell from the cytoplasm, and in the cytoplasm it can encounter transcription factors.
It then phosphorylates those transcription factors which bind to their target genes turning on gene expression and leading to a change in the cellular phenotype.
And the change in the cellular phenotype that we're talking about here is cell division, the details of which we'll cover next time.
So this is sort of a static picture.
But to give you a more dynamic picture of what's happening here, I borrowed from the Biocreations Website an animation, which I find very helpful and, actually, is kind of funny as well.
So we're going to start this process much the same way that we've just talked about using the terms that I used on the board.
Oop.
We need sound.
Let's do that again so you can hear it in all its glory.
OK.
So the first thing is the receptors get dimerized by the growth factor leading to the phosphorylation of these cytoplasmic tails.
And then this protein, which is the adaptor, it's called grab-2 grabs onto one of those phosphorylation sites.
It has attached to it an exchange factor called SAS here.
OK?
So that's the first thing that happens.
You recruit this signaling complex to the membrane.
Once you have it there it's now in close proximity to RAS which is, if you remember, a membrane associated protein.
It's now in its inactive stage, its GDP bound state.
Oops.
Sorry.
We'll just have to watch that again.
Oh, no.
Wait.
Wait.
Wait.
Go back.
Forget you saw that.
That's the end.
You never want to see the ending.
Oh, wait a minute.
OK.
Here we go.
OK.
So next we're going to activate RAS.
It gets a little kick in the butt.
GDP comes off.
GTP comes on.
Now RAS is active.
It's now capable of interacting productively with RAF, the first of these kinases, the map kinase kinase kinase.
And it's going to cause it to become active.
Once it's active it's now capable of activating the next guy in line, which is MEK.
Now MEK is active, and it's going to phosphorylate ERK, and now ERK is phosphorylated and active.
Now, there have been a couple of other steps, which you can look at, at home, but the RAS went from a GTP bound stage to a GDP bound state.
I told you that you can turn these signals off up here.
That happens in this pathway.
Another GAP comes along and turns RAS off.
And here's a protein phosphatase, which is going to clip off the phosphate, thereby turning MEK off.
So each of these steps is reversible.
But, importantly, we're down here at this level where ERK has been phosphorylated.
And now it's capable, dammit.
All right.
We'll have to watch it again.
Wait a minute.
Come on.
OK.
Here we go.
I know.
It's like a video game.
OK.
So ERK is active.
And now it's going to go in the nucleus.
Phosphorylate Jun, one of these transcription factors.
And upon doing so the transcription factors dimerize, bind to the promoter of a target gene.
They recruit RNA polymerase.
And this leads to the production of new gene expression.
mRNAs that go into the cytoplasm get translated into proteins that convince the cell now it's time to divide.
OK?
So I encourage you to look at this to get the details.
Now, if you just wait one more second.
In the last 30 seconds I want to point out the fact that these pathways, while we're teaching to you as simple linear pathways, are actually highly complex.
Pay attention to this clock here.
It's 11:54.
Highly complex, highly interactive, not linear, and therefore we frequently analogize what's happening inside a living cell to an integrated circuit.
And increasingly these days in biology, we're thinking about these things in analogy with computers, and actually relying extensively on computers to help us understand how that complexity actually works using methods from chemistry and physics and mathematics and other disciplines in a new and emerging area called systems biology, which you might be interested in.
And the new Biological Engineering major will have a strong focus in this area.
OK.
So today's lecture talks about the cell cycle, the control of the cell cycle, and also cell death.
So obviously, cell division is extremely important in multi-cellular organisms.
We've talked a fair bit about control of the cell cycle, in terms of mitosis and meiosis, in earlier lectures.
Obviously, cell division is necessary in a variety of circumstances, probably the most obvious is development.
You go from one cell to ten to the thirteenth to ten to the fourteenth cells.
That's accomplished by a tremendous amount of cell division that goes on over your gestational period.
I've also pointed out, in fact we talked about it last time, that cell, that wound healing is, in part, a process of cell division.
Fibroblasts for example, and other epithelial cells, get recruited to divide, in order to repair damage to a tissue.
And even without damage, there's a lot of cell turnover, for many tissues.
You may not realize this, but your blood, for example, turns over about every month, so you have to replace all your red blood cells, and all your white blood cells, and so on, periodically.
Cells in other tissues, in the skin for example, your skin cells are born, they migrate up to the superficial layers of your skin, and then they get sloughed off, and of course, they have to be replaced.
There's a lot of cell division that's going on naturally, in the process of what we call homeostasis, and to give you an example of that, if you consider your intestines in a human, your intestines undergo ten to the eleventh cell divisions per day.
That's a remarkable number.
Ten to the eleventh cells dividing in your intestines per day.
If you imagine that a cell in your intestine is about ten microns in diameter, if you lined up all the cells next to each other that were born in a given period of time, in 38 days you would have enough cells lined up end-to-end to go around the earth, and in a year you'd have enough cells to make it from the earth to the moon.
So you produce a tremendous number of cells just naturally, in the process of organ function and homeostasis.
Of course we can have too much cell division, and this is pathological condition, which we typically associate with cancer.
And indeed, many of the factors that I'm going to talk to you about today, that relate to the normal control of cell division, and also cell death, are perturbed in the development of cancer.
Cancer, as you almost certainly know, is a disease of too many cells, and a major factor in that is excessive cell division.
[UNINTELLIGIBLE PHRASE].
So this happens to be a human cancer cell line growing in tissue culture, and one of the hallmark features of cancer cells is that they have, in a sense, unlimited and unregulated cell division.
So if you were to watch this movie, which actually comes from your book, supplementary materials from your book, you could watch these cells dividing, and they would do so in an unnatural fashion.
Without response to proper growth factors that would stimulate cell division, they would divide on top of one another, which normal cells don't do, and other places and times when other cells are kept in check, cancer cells are not.
So the basic process, which again is well familiar to you, is to take a single cell and turn it into two.
And we discussed the very last stages of this in earlier lectures on mitosis, how the chromosomes get divided from the mother to the two daughter cells.
We also have briefly alluded to the fact that there's a second, critical event, and you talked about the mechanisms of DNA replication.
The DNA needs to get duplicated so that it can be properly divided, and this, as you know, occurs during a particular phase of the cell cycle, known as S-phase, for synthesis phase.
And then there's chromosome segregation, and this occurs in a distinct phase of the cell cycle called M-phase, or mitosis phase.
And the development of cells over the course of, say development, or in wound healing, can be thought of as a successive iteration of DNA duplication, DNA replication, and chromosome segregation.
So in a sense, you go from mitosis to S-phase to mitosis to S-phase.
These segments are separated by periods of time when cells are accumulating the biomaterials that they need, basically to function, and also to carry out the next phases of the cell cycle, and these are refereed to as gap phases, G1 for the one that separates mitosis from S-phase, and G2, the one that separates the S-phase from mitosis.
Now, this is a cyclic process, one cell gives rise to two, that then undergoes this process again, and so we think of these things as cycles from M-to-S, connected by these gap phases, G1 and G2.
Now there's another phase that we haven't told you about, which many of your cells are actually in right now, and that's a phase called G0.
This is a resting phase, and this can be either permanent or temporary.
Many of your cells, when they're born, will stop dividing forever.
Cells in your brain, for example, or most cells in your brain, cells of your cardiac muscle, and there are many other examples.
Once they get born, they undergo mitosis, they go into this resting phase, and they'll never come back out again.
And that's why it's difficult, for example, to deal with brain injuries, or spinal cord injuries, because those cells cannot be recruited back into the cell cycle, they can't make more of them.
But there are other cells in which the resting phase is temporary, and these cells can then reenter the cell cycle, going back from G0 to G1 and through the process again.
And a lot of the progenitor cells that I referred to up there, in terms of the skin and the blood, and the intestines, are in that situation.
They're resting, and then they be recruited back into the cell cycle, in order to make more derivative cells.
It's also useful to consider what the chromosomes are doing in these different phases of the cell cycle.
So in a G1 cell, if we think about a single chromosome, and we're representing it by a single-line, although this is of course, a double-helix.
In the G1 phase there's a single, double-helical chromosome, in S-phase that gets duplicated in the process of DNA replication, initiation shown here, and it would be completed so that the single-chromosome would be ultimately duplicated into two.
And then in M-phase, those two chromosomes get separated from one another, so these are now joined, separated from one another, eventually into two daughter cells, which can then go through this process again.
OK?
And much of what we think about, when we think about how the cell cycle is controlled is to deal with the initiation of DNA replication, and then the initiation of mitosis.
OK, so this is a cyclical process, a highly ordered process, this is a representation of the cell cycle, as shown in your book, just so you know it's in there.
Exactly as I described to you, mitosis and S-phase, where the action is, these gap phases, G1 and G2, and this little arrow represents G0, and I've also used this term before, interphase, that represents all the phases where the DNA is not evident under the light microscope.
It's only evident in mitosis because of DNA condensation, so the rest of the cycle, G1, S, and G2, are called interphase.
So this is a highly ordered process, each step preceded by a particular other step, leading to a particular outcome, events leading to a particular outcome, and in a topical sense, you can think about it as the NCAA pools, starting tomorrow there'll be individual events, games, which will lead to next events in the next round, and next events in the next round, to an ultimate conclusion, in this case, the NCAA champion, which can only occur if the proper events have happened, prior to it.
And you'll see another specific example of how the order is assessed in the development of the stages of the cell cycle.
OK, so we know that the cell cycle is important.
The question is, how is it controlled?
How do you accomplish this orderly process of cell cycle progression?
What are the genes -- -- that regulate this process?
What do the genes encode, what do the proteins, and what do the proteins do?
What are the pathways that they regulate to ensure the stages of the cell cycle?
This was a huge question for decades, and there was rather little progress in trying to understand how it happened, especially in us.
Our cells are sufficiently complex, and there's relatively imprecise, or particularly was in the past, imprecise methods for dissecting complex processes in mammalian cells.
And so it took experiments performed in yeast, a single-cell, eukaryotic organism, budding yeast and also fission yeast, to allow us to understand what the details of the cell cycle are.
Budding yeast, as shown here in a picture from your book, is a single-cell organism.
It's also haploid, or it can be at least, haploid, which means that it has a single set of chromosomes, it doesn't have two sets, single set of genes, which makes it amenable to genetic analysis.
You can make mutants more easily, if you only have one copy of each gene to worry about.
So that's another reason that yeast was attractive.
And another reason, which is kind of seen here, not really obvious here, but when yeast cells divide, they go through a very characteristic, morphological change.
They start off as spheres, like this one, and as they go through the cell cycle, they develop a small bud.
That bud then gets bigger, and eventually the two, the joint between the mother cell and the bud, get severed to form two cells.
And you can actually figure out where you are in the cell cycle, by examining exactly what the morphology of the yeast cell is, and I'll show you that on the next slide.
This is the yeast cell cycle, with overlay of the diagram of what the cells look like in the different phases, so a yeast cell that's in G0, let's say, looks like this, fairly round, nondescript, as it enters G1 it creates a small bud, that bud then gets bigger as the cells are now duplicating their DNA, the bud actually gets bigger.
It gets bigger still, during G2 phase, and then at M-phase you can actually see a little junction between the two cells, and the mother and the daughter cell are more-or-less the same size, and then they separate from one another, and they can go through the cycle again.
And this is useful because you can tell if you have a mutant, as you'll see in a moment, if you have a mutant in a gene that affects one of these processes, you can actually figure out where the mutant gene acts within the cell cycle, based on what the yeast cell looks like.
And these individuals here, won the Nobel prize for their efforts to understand the cell cycle, they won it about five years ago, or so.
This is Lee Hartwell, he's an American, happens to run a cancer center out at the University of Washington, very suave and sophisticated guy.
These two goofballs are Brits, very nice guys actually, Paul Nurse, who's now the president of Rockefeller University in New York, and Tim Hunt, actually Sir Tim Hunt, because he's knighted after this Nobel Prize.
And I'm going to explain the experiments that all three of these guys did today, which won them the Nobel prize, and gave us tremendous insight into the control of the cell cycle in yeast, which turned out to tell us how the cell cycle is controlled, also, in us.
OK, so the first thing we need to do is to create mutants.
That's a general theme in biology.
If you want to understand a process, find a mutant that can't do it, and understand the genes that get mutated in that process, and so that's what happened in the hands of Lee Hartwell, the guy in the upper-right, he took this same yeast, the budding yeast, which is also Brewer's yeast, the yeast that makes beer.
He took a population of these yeast cells, and he mutagenized them.
I told you about this before, you can add chemical mutagens that soak into the cells, bind to the DNA, cause mutations.
Each cell might have one, or a few nucleotides changed, and at some frequency those mutations will affect genes, and at some frequency the genes affected will influence the process that you're trying to study.
So he was trying to study growth control, cell division, so he asked about the ability of the cells to grow, and particularly, he did so at two different temperatures.
He tested the ability of the cells to grow at their normal temperature, which is about 25 degrees centigrade, and he found that many of these yeast cells would grow, following mutagenesis, at 25 degrees.
Probably some of them wouldn't grow because they would've inactivated some critical gene, and even at 25 degrees, they couldn't grow.
However, he then took these plates, and he used a technique, which I referred to in a previous lecture, called replica plating.
He took a stamp, basically, of these colonies, transferred them onto a fresh plate, and examined how the cells grew at 30 degrees centigrade.
And what he found was that, whereas many of the cells that were able to grow at 25 degrees continued to grow at 30 degrees, so they produced colonies in exactly the pattern that he had seen previously.
Some of the cells didn't.
This colony here, while it was successful, the cells were successful in growing at 25 degrees, failed to grow at 30 degrees.
And this type of mutant is referred to as a temperature-sensitive mutant, -- [PAUSE} -- abbreviated TS.
We imagine that this mutation affects the amino acid sequence of the protein, at low temperatures the protein is still able to function, but at higher temperatures, maybe it becomes a little unstable, and now it can't function, and that's why you see no growth here.
So we had a TS-mutant that affected the growth of the cells, OK?
So my question to you is, is that TS-mutant interesting?
Is it interesting?
Well we can't actually know whether it's interesting yet, because there are two general classes of mutations that would give you this same phenotype.
One of them is interesting, does that cement always show, or do we have a problem here?
It's always there?
Never noticed it, in all these years.
You can distinguish between these two classes of boring and interesting mutants, based on the morphology of the yeast cells when you shift the temperature.
If you start with a population of yeast cells growing just randomly at 25 degrees centigrade, including these mutants, if you were to look under the microscope, you would find mutants, cells that were at various stages of the cell cycle.
OK?
Now, let's imagine that our mutation affects a general enzyme that's required for cell viability, DNA polymerase or maybe, better still, some ribosomal protein that all cells need all the time, in order to function.
If I were to take these cells which were growing at 25 degrees centigrade, and I were to shift them to 30 degrees centigrade, and look under the microscope, what would I see?
Well, if it's a ribosomal protein, and now translation just stops, these cells are going nowhere.
Regardless of where they were in the cell cycle, they don't go any further because you need protein synthesis to do anything, so if you look under the microscope at these cells, following a temperature shift, you would still find a distribution of cells in the different phases of the cell cycle.
Lee Hartwell was not interested in this class of mutants.
They could be anything, he didn't care.
However, he reasoned that if he were dealing with a mutant that affected specifically a stage in the cell cycle, he might find that the cells, upon temperature shift from 25 degrees to 30 degrees, got to a particular point in the cell cycle and couldn't go any further, that this gene affected one of these important transitions, and so for example -- -- if he were to look under the microscope, all of the cells would have arrested with a small bud, would have arrested in G1, or maybe they would've arrested looking like this, just prior to mitosis, or like this, somewhere in G2.
But importantly, they would arrest with a specific morphology that would indicate that they had a specific cell cycle block.
And so he did this, and he found many, many such mutants.
He called them cell division cycle mutants, or CDC mutants.
And it was his methodology, which really broke this field open, and won him the Nobel prize.
Now one example of a CDC mutant that he discovered, called CDC2, was particularly important.
It's been renamed CDK1, or cyclin-dependent kinase one, for reasons that I'll come to.
Cyclin, sorry, cyclin-dependent, kinase one.
And this one acts at a particular phase in the cell cycle, early on in the transition from G1 to S.  So if you imagine cells in G0, going to G1, and then progressing from G1 into S-phase, into G2, and finally, in mitosis.
This particular mutant blocked right here, and so it said that this gene, CDC2 or CDK1, was required for this transition.
And just to emphasize the TS-temperature shift and building up of the cells in the cell cycle concept, imagine a cell that looked like this, before the temperature shift.
Ok?
If you now do the temperature shift, this cell will progress to this point, but it won't go any further because CDC2 is required.
Maybe I'll draw this over here, to make it even more obvious.
CDC2 is required to go beyond this point, so that cell will arrest right here.
If there were a cell that looked like this, in the population of 25 degrees, and now you shifted the temperature, it would complete this phase because CDC2 is not required.
It would make it all the way to here, it would keep going, and then it would get stuck here again.
And that's why you get cells building up with a particular morphology.
OK?
So this was successful, and as I said, he isolated a large number of such CDC mutants, which have taught us not about just that transition, but many other transitions in the cell.
Now what Paul Nurse did, this guy here, he actually was performing very similar experiments in a related yeast species, but the critical experiment that he did was to clone the gene that is responsible for CDC2 function.
So he took CDC2 mutant cells, which at 30 degrees -- -- will not grow.
And he added, through a cDNA library, CDC2 cDNA, of yeast origin.
So he made a cDNA library from yeast, he introduced it into these mutant cells, and then he asked whether the cells could now grow at 30 degrees -- -- and the answer was yes.
That is, if the cells now carried an extra copy of CDC2, in the form of this cDNA, they now could grow at 30 degrees.
He complemented the mutation through the addition of a normal copy of the CDC2 cDNA.
Quite surprisingly, he did the same experiment -- -- using not a yeast gene, but a human gene.
Frankly, there was great skepticism in the field about whether what these guys were doing in yeast had anything to do with cell cycle control in humans.
Most people actually assumed that it probably had nothing to do with it, and so therefore, when Paul Nurse proposed to try to complement his yeast mutation with a human gene, people were skeptical.
But in fact, it worked.
And this told him, and the field, that the machinery that controls the cell cycle in this simple, single-celled eukaryote, is highly conserved, all the way through to humans, and that we could therefore understand cell cycle control in us, by doing experiments like this in yeast.
So this really broke the field open, and that's why Nurse won the Nobel prize at that same time.
However, there was a problem.
They did in fact clone the gene, they were able to sequence the gene, and they found that the sequence of the gene, which I'm going to refer to now by the other name, CDK1, looked like a kinase.
It had amino acid sequence, which made it resemble known kinases, so it was assumed to be a protein kinase.
However, it didn't have any kinase activity.
If you mixed it with various substrate molecules in the presence of ATP, those substrates were not changed, they were not phosphorylated.
So the purified CDK enzyme had no kinase activity, and therefore, it wasn't entirely clear what its function really was, and the field sort of got stuck there for a while, trying to understand the biochemical function of CDK1, and other cell cycle regulators.
And that then led to the experiments done by Tim Hunt, this guy here.
And Tim Hunt, actually doing experiments at Woods Hole, down at the Cape, in a summer course, with summer students, did a famous experiment using sea urchins.
Sea urchins, when they're fertilized, undergo very rapid, and very synchronous, cell divisions.
In the first few hours after you fertilize sea urchin eggs, they will divide and divide again.
And importantly, they will do so in a very synchronous manner, so all the cells will produce two cells at around the same time, and those will all produce four cells at around the same time.
And this is important if one wants to do biochemical experiments, to understand what is changing within these cells as they're going through these various cell cycles.
And so, what Tim and his students did was to label the proteins in these dividing sea urchin cells, with a radioactive amino acid, S35 methionine.
And then they simply made extracts of these cells at different time points thereafter, and asked whether anything was changing in an interesting pattern, at different times that correspond to different stages of the cell cycle.
So they ran protein gels, they separated the proteins and then exposed the gels to x-ray film, to visual what the protein concentrations were at different time points.
They took cells at time zero, they took cells after 30 minutes, after 60 minutes, after 90 minutes, 120 minutes, 150 minutes, 180 minutes.
And they knew already, from their earlier analysis, that this corresponded to the first cell cycle, and this corresponded to the second cell cycle.
OK?
Now some proteins, when they visualize them that way, didn't change.
At the different time points they saw roughly equal concentrations of that protein throughout, but interestingly, other proteins changed in abundance, and did so according to a pattern.
They seemed to oscillate, they seemed to cycle, in a pattern that corresponded with the cell cycle.
And so he called these cyclins, and he suggested that they might have something to do with the control of the cell division cycle.
Well, meanwhile, Nurse and Hartwell were doing their thing on CDKs, and so they came up with the idea that maybe these two things have something to do with one another.
And particularly, maybe the failure of CDK to function as a kinase was due to the fact that it didn't have an accessory protein that it needed, mainly the cyclin.
And so, whereas CDK2, sorry, CDK1, was an inactive protein kinase.
Now in a test tube, in a biochemical experiment, if they mixed CDK1 with one of these cyclins, they observed kinase activity.
And thus the name, cyclin-dependent kinase.
It's not a kinase, it's not an active kinase in the absence of cyclin, it only becomes active in the presence of cyclin.
So let me give you two quick examples of proteins that are then phosphorylated by this active kinase, to give you a sense of how this kinase regulates cell cycle transitions.
There's a class of proteins that are involved in the regulation of replication initiation.
We'll just call them replication initiation factors.
In the absence of phosphorylation, they're inactive, and that's one of the reasons that replication is not initiated at those times in the cell cycle.
However, in the presence of CDK cyclin, now the protein becomes phosphorylated, and becomes active.
OK?
So one of the ways you trigger the transition from G1 into S-phase, is to turn on this enzyme which phosphorylated this target protein, and stimulates S-phase entry.
A second example is a protein called pRB.
In its un-phosphorylated state, it is active, different from here where, in its un-phosphorylated state, it was inactive, and the function of the RB protein, in its active state, is to block again, the transition from S-phase to G1.
This is actually an important cancer gene, it's mutated in a large number of cancers, and so we'll talk about its function, exactly how it blocks the transition, in later lectures, but suffice it to say, it does that.
And when it is phosphorylated by CDK cyclins, it becomes inactive, thereby allowing cells to progress from G1 into S-phase.
OK?
So those are two examples of how CDK cyclins operate.
Now, importantly, cyclin kinase activity is determined by the level of the cyclin.
As I told you, cyclin levels oscillate, where this is cycle one, cycle two, cycle three.
So these are cyclin concentrations inside the cell.
CDK levels do not oscillate.
Concentration of the CDKs inside the cells is rather constant.
However, at this point in the cell cycle, when there's not enough cyclin, you don't have kinase activity.
Only when you go past the threshold, do you get kinase activity.
When it drops again, you lose kinase activity.
But in the next cell cycle, the cyclin levels increase again, and you get kinase activity, and so on.
So the oscillation of cyclins determines the oscillation of kinase activity, which determines the periodicity of the cell cycle.
Now, for the truth in advertising, the situation is actually more complex, there are actually multiple CDKs and multiple cyclins.
So in our cells, for example, if we imagine G1, S, M, G2, G1, S, M, G2, there's a cyclin called cyclin-D, which comes up in G1, goes down, comes up in the next G1, goes down.
There's another cyclin called cyclin-E, which comes up a little bit later, in S-phase goes down, stays down, comes up in the next S-phase.
And there's finally another one called cyclin-B, which comes up in G2, and comes down, goes up in the next G2.
Anyway, the point is that there are different cyclins that get induced in different phases of the cell cycle, and actually control, through binding to different CDKs, different transitions in the different phases of the cell cycle.
OK.  Let's see.
So, another concept: the transitions from the cell cycle don't occur, from one cell cycle position to the next don't occur, unless the previous cell cycle event has been completed.
And that makes sense because you don't want to, for example, try to divide your DNA in mitosis if you haven't fully replicated your DNA in S-phase.
So there are processes that are overlaid on top of cell cycle control, which ensure the completion of one phase before the next phase is initiated.
And I draw, as an analogy, your washing machine, which likewise, has checkpoints which will determine whether or not the previous phase of the wash cycle has been completed.
For example, you don't spin your wash until all the water has been rinsed out.
There's a sensor that will determine whether that's not true, and if that sensor is tripped, it blocks the wash cycle at that point.
Your cells have very similar checkpoints that will monitor and regulate cell cycle transitions, and they're called checkpoints.
There are checkpoints, actually, that operate at different phases of the cell cycle.
I'll only give you one example, it's actually the one that's best known.
It occurs at the transition in mitosis, from metaphase -- -- where, if you'll recall, the duplicated chromosomes line up on the metaphase plate.
In the process of anaphase, where the chromosomes separate from one another, the chromatids I should say, separate from one another.
There's a cell cycle checkpoint that makes sure that this happens before that can happen.
And in particular, if you have a chromosome which is only attached to one side of the mitotic spindle, so if you recall, these are centered in the middle through attachment to microtubules that are emanating from the two poles.
If you have a chromosome that is only attached to one of the two spindles, it's called monopolar attachment, whereas perhaps other chromosomes, maybe all of the other chromosomes, are properly attached at the metaphase plate.
This one, unattached chromosome will literally send a signal, a biochemical signal, which is the equivalent of "wait for me".
And that signal will inhibit cell cycle progression.
It'll specifically inhibit the transition from metaphase to anaphase.
While that signal is being sent, the cells will just sit there, waiting for another microtubule to bind to this end of the chromosome, thereby extinguishing the signal, relieving this inhibition, and allowing anaphase to progress.
OK, so these checkpoints are critical in insuring that these things happen in a timely and ordered fashion.
OK.  That's it for the cell cycle, so for the final ten minutes, and I always give short shrift to the next topic, which is cell death, which is another fascinating topic, fortunately there's not a lot of board work to show you.
Cell death.
So cells are born, divide, as we've just been talking about, but remarkable numbers of cells in your body die.
They will die, too, because of cellular injury, but they'll also die because they're programmed to die, or they'll decide to commit suicide, or they'll be murdered by other cells.
This happens in development, for example, your brains, as developing fetuses, have ten times the number of cells than you end up with as a young infant, because 90%, for some of you more than 90% of the cells will actually be killed through this process of programmed cell death.
Other cells, in your immune system, for example, are eliminated by this process, so as to avoid them attacking your own body.
And there's many other examples of relevant and important programmed cell death.
So we want to understand this process, partly because it's critical in normal development, and also because deregulation of this process is critical for many diseases.
Too little cell death can give you proliferate diseases like cancer or autoimmune disease, too much cell death can give you diseases like neurodegenerative diseases, too many cells in your brain dying.
So the regulation of cell death is quite important.
Here are two other famous examples of programmed cell death.
This is the loss of the tadpole's tail that occurs through an orderly, cellular suicide program, in which all of these cells die, giving rise to a frog with no tail.
And likewise, in development, your hands are formed in such a way that the digits are actually connected by other cells, but through this process of programmed cell death, the cells in the middle, the interdigital cells, are eliminated, thereby sculpting the formation of your fingers.
And this can be regulated, and has been regulated, in evolution, in some species, like ducks, the loss of the interdigital cells doesn't take place, so they have webbing.
In other species of birds, like in us, the interdigital cells are removed, so they have sculpted toes.
This is what programmed cell death looks like.
It is a very rapid, and as I said, a very orderly process.
Cells get signals to die, they can get signals because they get damaged, or they can get signals from their neighbors.
When they get those signals, they undergo a series of biochemical changes.
Their nucleus gets condensed, the DNA within the nucleus breaks up, and then the cells themselves break up into small fragments called apoptotic bodies, and then interestingly, the cells that surround those cells, eat the damage.
It's like disposing of the body after a crime, the cells are eliminated through a process of phagocytosis, this is a very clean process then, there's very little junk, there's very little cellular debris that remains when this process of programmed cell death is completed.
This is a normal looking lymphocyte, and this is a lymphocyte undergoing cell death.
It's like, you know, this is your brain, this is your brain on drugs, this is a cell, this is a cell undergoing apoptosis.
It's a fairly violent death, at least visually.
This movie shows you an example of that.
Here are cells undergoing apoptosis, I hope.
Maybe not.
Oh well.
It comes from your book, so you can see it yourselves, in fear that that would happen, I'll show you another set of stills.
Here are normal cells growing in the tissue culture dish, you add some agent which induces apoptosis, the cells begin to round up as you can see here, and their cell surfaces actually become this horrible mess of blebbing membrane, and they will eventually break up, as you can see starting to happen here.
So apoptosis is critical and actually very, very interesting.
As I said, too much cell death can give rise to various diseases, neurodegeneration stroke is complicated because of the effects of apoptosis, too much apoptosis, even in AIDS, there's a lot of cell death that takes place in apoptosis, and too little cell death, as I said, is important in cancer, autoimmune disease, and certain viral infections.
Importantly, we know about the genes that are regulated in cell death, that regulate cell death, through genetic experiments.
And here, the genetic experiments were performed, in large part, at MIT, by a professor in the biology department named Bob Horvitz.
He took comfort in the fact that the process in C. elegans, a simple worm, has very many similarities to the process that I've outlined to you, and therefore, he hoped that the genes that regulate programmed cell death in C. elegans, were similar to the ones that do so in mammals.
And so he then undertook a study to ask what genes regulate the cell death process in this simple organism called C. elegans, which as an adult has only 1,090 cells.
And the beauty of this organism is that you can actually see through it during development, and you can therefore follow the fate of individual cells over the course of development.
And he and his students and fellows did that, he observed that in the development of certain cell lineages of the developing worm, individual cells died.
In fact, about 130 cells underwent this process of apoptosis.
And then he carried out a genetic experiment.
He asked, if I mutagenize worms, can I find ones in which this process doesn't happen, or perhaps, in which this process happens too much.
What are the genes that regulate apoptosis?
And he found several such genes that turn viable cells into cells that have undergone programmed cell death, or apoptosis.
In particular, there were two genes that positively regulated this process.
They were called Ced-3 and Ced-4.
They were required for apoptosis to occur.
Mutants in Ced-3 and Ced-4 didn't have this cell death process in those developing worms.
And another gene called Ced-9 was required to prevent apoptosis.
In a Ced-9 mutant, there was too much apoptosis going on, OK?
They then cloned these genes, and it turned out they all have homologs, versions, in our cells, and those genes in our cells likewise regulate apoptosis in us.
And we now know that at least members of this family of genes are important in diseases, including cancer.
I'll just mention the function of one gene, right now.
Ced-3, it is a protease, which cleaves target proteins -- -- into fragments.
So, one of the key things that happens in apoptosis is, you turn on this enzyme, this protease, and it goes around cutting up various target proteins, which eventually lead to the death of the cell.
There are many target proteins that get cleaved when these caspases, these proteases are called caspases, when these proteases get activated, and I just gave you one example of one such target.
There's an enzyme that is normally involved in cleaving your DNA, this is a DNA gel, of viable cells and cells that have undergone apoptosis after a certain number of hours.
The DNA of the cells is normally intact, and large, because this enzyme is kept in check.
However, when its inhibitor is cleaved by the caspsase, it now goes about cleaving the DNA, and this is one of the reasons why apoptotic cells die, because their DNA gets cut up into very small fragments.
There are a number of other targets, you can read a little bit more about this in your book, I apologize that we had to rush through apoptosis, it is important, and we will in fact come back to it in a discussion of cancer.
What we talked about last time was formation of different types of cells in the embryo, different positions, different shapes.
Am I on?
And we talked about one of the overriding principles of all of this being the control of gene expression.
I'll explore this a little more with a couple of slides in a moment.
Let me just write some points.
So we talked about the control of gene expression as being an overriding principle of how cells types are formed.
We talked about the fact that cell types were formed in a stepwise fashion.
And different parts of the embryo indeed in a stepwise fashion.
Not all at once.
And we talked about a combinatorial code -- -- of regulators that, together, specify different types of cells.
So here we are to remind you talking about cell type in the formation module stepping through our board of life.
And here are a couple of slides to remind you what we talked about in last lecture to bring you up to speed.
You should look at the PowerPoint on the Web if you have not already.
Cell type is defined by the proteins a particular cell makes.
For example, red blood cells carry oxygen around the body because they make globin which is able to carry oxygen.
The set of proteins that are made are defined by which genes in a cell are active, or expressed is the term you should know.
And, therefore, which genes are active, also called the control of gene expression, controls which cell type forms.
This triad is essential for you to understand what we're going to be talking about now and really for the rest of the course.
So you should really get it.
If you don't see me, go back and look at the previous lecture.
And this lecture is going to also reinforce this triad.
I pointed out last time that gene expression can be controlled at many levels.
In fact, at any point from transcription, even from replication actually of the DNA through transcription, splicing, translation, export of the RNA to the cytoplasm, I haven't written up here, messenger RNA stability, protein modification, protein export.
Any of these steps is along the way to getting the final gene product.
And any of these steps can be controlled to allow the readout of the final gene product or not.
I told you that along the way to becoming the final type of cell that a cell is going to become, it goes through a couple of steps.
Or it can go through many steps, and we'll talk about that extensively today.
Cells begin as uncommitted or naive.
They don't know what they're going to become.
They then become committed or determined to a particular fate, a position or a structure, but they may not look very different from that first set of uncommitted cells.
And then they go on eventually to form so-called differentiated cells or the final cell type which has the function of that particular cell type.
And so the point is if there is a stepwise formation of the final cell type.
And that final cell type is directed through the action of regulatory factors that we termed inducers or determinants.
There is nothing magic about these.
Inducers referred to factors that were involved in cell-cell signaling.
Determinants referred to regulatory factors that acted within a cell and were inherited by it.
You shouldn't be trying to write all this down now actually because this is really review.
And if you weren't at the last lecture, try to get some of it to wash over you and then go back and make sure you really know this stuff.
And then as the differentiated cells form there is activation of a set of genes called differentiation genes which are the things that carry out the final cell function, the globin protein in red blood cells, the neurofilaments and neurotransmitters in neurons and so on.
And we'll talk about this again as we go through today.
OK.
So this is not unrelated to what we are going to talk about today.
Because what I want to do today is to explore with you how you get a particular kind of cell.
And I want to use that exploration to underline some of the principles that I haven't yet covered as to how cell type is determined in the embryo.
And the cell type I'm going to talk to you about is skeletal muscle.
No, this is not the [UNINTELLIGIBLE].
OK.
So the point today is to look at a particular cell type and how it forms.
The type we've chosen is skeletal muscle.
Skeletal muscle is one of three kinds of muscle in your body.
The other two being cardiac in your heart and smooth muscle around your internal organs.
Skeletal muscle acts in a voluntary fashion to allow movement.
It's an extremely complex structure.
The final form of skeletal muscle includes 200 proteins which have the extraordinary property of contractility in response to nerve input.
I'm not going to talk about the contractile process, although it's fascinating.
And you're not going to be tested on this for the physiology of contractility either.
But if you're interested in this, I would suggest you look at Chapter 47 in your book which talks about this quite nicely.
Skeletal muscles are arranged in large groups of fibers.
And each muscle fiber is a multinucleate cell, is a giant cell that consists of many myofibrils or myofibrils that are together in a sheath and are surrounded by many nuclei.
So skeletal muscle forms from the fusion of many mononucleate cells to give a multinucleate syncytium.
And during the process of skeletal muscle formation there are a number of steps that one can delineate.
One can talk about a kind of cell called a myoblast which is a single mononucleate cell -- -- that then fuses to form, through many steps, something called a myotube which is multinucleate.
And these various myotubes come together to eventually form a muscle fiber.
Muscle is obviously an extremely important cell type for movement.
All of the meat that you eat, if you're a meat eater, is muscle, essentially a skeletal muscle.
And there are many diseases that are associated with abnormal skeletal muscle.
Muscular dystrophy, for example, is a deficit of one particular protein in the muscle, affects one in about 3,000 boys.
OK.
So the myofibril is a very beautiful construction of many proteins organized in a very ordered structure.
And the deal is that these proteins slide relative to one another during contractility.
And all of these bands of proteins slide in an ordered fashion relative to one another as the muscle contracts.
It's an extraordinary story in physiology itself.
And I regret I don't have time to go through it with you.
But I want to use this rather as an example of a cell type and to explore with you how the cell type forms during development.
So the story I'm going to tell you has to do with a particular transcription factor called MyoD.
MyoD, shown in green here, no, not shown in green here.
That's the DNA double helix.
Shown in red here binds to a particular sequence on DNA, its target site.
It binds as a dimer.
And it acts to activate the transcription of genes required for muscle formation.
OK.
So MyoD is a transcription factor.
And, as I'll tell you a moment, it gives rise to the determination of skeletal muscle.
Now, I'm going to tell you a few things about this that are general principles of how this transcription factor works and then I'll go through some slides with you.
MyoD is both necessary, although I'm going to clarify that in a moment, and sufficient for muscle determination.
This is a dyad of words that is very important, necessary: required for, sufficient: enough to.
And whenever you're thinking about gene function, this is the dyad you want to think about.
It may not be a gene, and then I'll tell you in a moment.
I'm going to put a caveat here with respect to the necessary, OK?
But necessary and sufficient is one of these litanies that you want to ask yourself when you're thinking about regulatory factors.
The other thing that I want to tell you is that MyoD is a member of a gene family.
What's a gene family?
A gene family is a group of genes with similar structure and often similar function.
This gene family that MyoD belongs to is called the MRF gene family, which stands for muscle regulatory family.
And there are four members.
And these members share some function with one another.
And that leads us to another term which is the term redundant.
MyoD is an extremely important gene.
And if you make knockouts in mice there is some phenotype, but they have muscle.
And it's not until you make a gene mutation in another member of the MRF family that you see mice that don't have any skeletal muscle.
So redundant in this case refers to, or in all cases refers to shared function.
It can also refer to, and does in this case, it can also refer to alternate pathways.
So the notion that we have is that we're starting here as an uncommitted cell and we're going to end up over there as a differentiated muscle cell.
And we can get there by walking straight this way, but we could also get there by walking up and out the back and all the way around and to that same spot in front.
And both of those would be bona fide pathways that would get you from point A to point B.
So sometimes there can be alternate pathways that will get you along the path of forming particular cell types or even carrying out some kind of biochemical function.
OK?
This is a general principle of biology.
So let's look at this in a bit more detail with some slides.
The notion is that MyoD binds to the promoters of many different genes expressed in skeletal muscle, together with other transcription factors, some of which may be common to all of these genes and some of which may be different.
And it activates their transcription.
And you can see I haven't drawn the genes in a really accurate way.
These are just representations.
There's no double-stranded DNA meant to be implied or not implied there.
MyoD is sufficient to activate the skeletal muscle program.
And this is a very exciting finding about 15 years ago when it was shown that one could take the MyoD gene, and you actually put it in a virus, and then you can infect a group of cells that normally wouldn't become muscle with this virus.
And lo and behold the virus that is expressing the MyoD protein will turn these cells into skeletal muscle.
And this was the first example of a regulatory protein that could take one cell type and turn it into another cell type.
And MyoD and the other MRFs are very good at doing this.
They can, for example, take brain cells and turn those into muscle.
They can really take most cell types and turn them into muscle cells.
So MyoD is sufficient to convert non-muscle to muscle cells.
And this is actually particularly extraordinary because, as I showed you in a one line schematic on this board, forming skeletal muscle is a multi-step program.
You start off with these cells called, they call them myotome cells, and then you get these things called myoblasts which are dividing.
The myoblasts start to align in arrays.
And then the cells fuse with one another, which is really unusual for cells.
I mean it's a really kind of weird cell type.
And you get these long tubes that have got thousands of nuclei in them.
And then the tubes start to make proteins that align themselves in these very ordered ways that allow them to slide relative to one another, and you get this multinucleate myotube that has contractile function.
And what's really cool about MyoD is that it can start this whole process which then follows one step after another.
So it's been termed a master regulatory switch because it can activate a whole cell type specific program.
Now, in contrast, and this is what I was alluding to on the board, knockout or removal, a genetic mutation of MyoD does not remove skeletal muscle from mice.
So this is a micrograph of muscle.
You can see the ordered myotubes from a wild type mouse.
This is a MyoD null mouse.
And there's perfectly fine skeletal muscle there.
It's a little, there's a little less of it, but it's still there and the mouse is viable.
However, if you go in and you make a double mutation in the MyoD gene and another member of the family called Mif5, there is no muscle at all in any region, or no skeletal muscle at all in any region of the body.
The other muscle, the heart muscle and the muscle around the intestine, the smooth muscle is fine, but the skeletal muscle is completely absent.
So we can say that the MRFs are necessary for muscle formation but that MyoD is a member of a redundant gene family.
And that seems to be the rule in biology, that genes come in families, and very often one can take over the function of another when one of them is mutated.
And that serves as a kind of fail-safe for making sure that development works properly.
OK.
So here comes something to think about that I alluded to before spring break.
I remember this.
Don't know if you do.
MyoD is, in the entire embryo the MyoD gene is only expressed in the future skeletal muscle.
This is a picture of a frog embryo.
And you can see these stripes along the back of the frog.
These are regions of the embryo called the myotones.
They're part of the somites that I'll talk about.
And this is what's going to become the future skeletal muscle.
And this is where MyoD is expressed.
And, brilliant, only there.
And now this is a chicken embryo and you see the same thing.
MyoD is expressed in these stripes along the back in the future myotones of the somites.
It's also expressed in the future limbs as the muscle of the limbs is starting to form.
And so you're going to ask me, or you should, how does MyoD get activated just in the future skeletal muscle?
Because in a way this is a copout.
I've told you MyoD activates the skeletal muscle program, and it's only expressed in cells that are future skeletal muscle.
Well, how does it know to be expressed just in this one region of the embryo?
And to answer that question we have to actually go back to the dawn of time to fertilization.
And I'm going to take you through some of the steps that will get MyoD expression into this particular pattern that will allow the skeletal muscle to grow out in just the right places of the embryo to give just the right amount and the right position of skeletal muscle.
OK.
So.
And to do that we are going to pose some questions.
And I'm going to start by telling you something and then pose a question.
If you look at an embryo, you could look back at that chick one if you wanted to do or you can look at this fish.
Here is your fish fillet, your skeletal muscle, and it is on the so-called dorsal or backside of the embryo.
So last time we talked about dorsal and ventral as two of the so-called axis or positional coordinates in the embryo.
And the muscle arises from a region that is termed dorsal which is the back.
OK?
So you might ask me, how do you know this?
And I'll tell you how I know this.
You should ask me how do I know this.
OK, so I'm going to.
Having put questions into your mouths, I'm going to tell you how you know during the development of an organism where a particular cell type is going to arise from.
And the technique I'm going to tell you about is something called fate mapping.
So let's begin with the sentence muscle, and I'm talking about skeletal muscle when I write muscle, arises from the dorsal or the back of the embryo.
OK?
And so how do you know this?
How is this known?
And it's known through use of a technique called fate mapping.
What is fate mapping?
So this is very interesting.
The idea is to take a very early embryo, I've shown you here a fish embryo, and to inject one or a few of its cells that you can distinguish by some kind of morphological, obvious indicator from the rest of the embryo.
And to inject those cells with a dye.
So here we've used a fluorescent dye.
And we've put that dye near the site of the embryo that's got this bump.
OK?
And as the embryo develops the cells will inherit that dye because it's a non-diffusible dye.
And that dye will be inherited.
And in the later fish, we're looking down now onto its brain, you can see that the whole front of the brain is outlined with these green fluorescent cells.
And what this tells you is that the cell you injected initially has gone on to give rise to this whole front region of the brain.
OK?
So fate mapping tells you what cells will become.
All right.
I know it's on the slide, but it's important that you understand this because it is distinguished from something I'm going to tell you later.
So you can do this for skeletal muscle.
And I'm going to talk about frog development because so much is known.
So here is a frog at the 32 cell stage.
And one can see, because of the size of the cells and because of pigmentation differences I'll show you in a moment, which side of the embryo is what.
And I've injected a cell here that's called the C3 cell, all these cells have got different names, with dye.
And later on you can see, in this thought experiment, that the somites are labeled with the blue dye.
So the C3 cell, and actually the one on the other side of embryo, are going to give rise to skeletal muscle.
OK.
So the reason that I wrote on the board what was on the slide already is that it's really important to distinguish what cells are going to become from when they've made the decision to become it.
So the next thing we want to know is when cells decide what they're going to become.
And I can rephrase that by saying when is the fate decision made?
And fate mapping doesn't tell you that.
All it tells you is what a cell is going to become, not when the cell has decided to become that thing.
For this you have to do some other assay, and you do assays that are generally called determination assays.
And I'll show you an example of one such assay.
So here's our 32 cell stage frog embryo again with the C3 cell labeled.
And one can go into the embryo and dissect out that cell.
Frog embryos are about a millimeter in diameter.
And you can use very fine glass needles to tease out that cell from the embryo.
And you can grow it in a very simple culture method with just some saline solution and the cell will grow fine and divide.
And you can ask whether that cell goes on to make skeletal muscle.
OK?
Does it know that it's going to be skeletal muscle?
And the answer at this stage of development is no, it doesn't.
So if you remove the cell, grow it in culture, it goes on to become these things called fibroblasts which are cells that spread out in the dish.
They don't fuse.
They don't make all these muscle proteins.
And so the cell, although it's destined to become muscle, has not yet made the decision to take that leap of fate.
However, if you wait a few hours, and we're actually on your handouts now.
If you wait a few hours and you go and you remove the cells that have arisen from C3, which are divided now.
They're still labeled and you can distinguish them because it's a very powerful dye and doesn't diffuse.
So you can remove that same group of cells a bit later.
And you can then grow those cells in a culture, in a simple saline solution.
And lo and behold they go on, after some hours, and form multinucleate myotubes.
They go on to form muscle.
So somewhere between the 32 cell stage and the stage of a much older embryo of many hundreds of cells, the cells have made the decision to go onto their particular fate.
So something has changed in terms of the regulatory genes that they're expressing, presumably, that allow them to go and become muscle.
OK.
So let's break this down a bit more.
And let's ask the question, as to what inputs are required to turning on MyoD activation in a more defined sense?
And the first thing I'm going to talk about is how the embryo knows where dorsal is.
OK?
So dorsal determination.
And what I'm going to tell you is a very interesting tale of determinants, of cell autonomous regulatory factors that are specifically required to tell the embryo where the future dorsal or backside of the embryo is going to be.
And I'll tell you that these determinants act to stabilize a particular protein that is a transcription factor, and this protein is called beta-catenin.
And beta-catenin, once it's stabilized, so it is a transcription factor, goes on to activate a set of genes that are dorsal specific genes, but not MyoD.
We're not there yet.
Gene.
G.  I cannot spell, can I?
OK.
So let's discuss this using your handouts.
This is a movie that I was showing you at the beginning.
Woops.
Let's show it to you again because I want to point something out.
All right.
This is a four cell embryo.
Maybe it's going to be an eight cell embryo.
Do you see how these embryos have got some, two cells that are kind of darkly pigmented and two cells that are lightly pigmented?
Yeah?
The lightly pigmented cells are the ones that are going to be the future dorsal site of the embryo.
OK?
That's where the muscle is going to come from.
And these cells on the other side, on the ventral side of the embryo, that's where the belly is going to come from.
Now, how does this happen?
Well, it happens, and I've augmented the diagram on one of your handouts a bit so you'll have to add a circle.
It happens because of movement of these purple dorsal determinants.
And these purple dorsal determinants, as I'll show you, move within the newly fertilized embryo.
In fact, within about 16 minutes after fertilization these dorsal determinants have moved in the embryo and have told the embryo where the future dorsal side is going to be.
And the way they move is that, in fact, the early egg and the zygote consist of two kinds of cytoplasm, an outer sphere of cytoplasm that's kind of gelatinous and an inner sphere of cytoplasm that is more liquid.
And in this outer gelatinous cytoplasm sit these purple determinants.
And when fertilization takes place these move by a good angle, at least 30 degrees relative to where they were, and relative to this inner liquid cytoplasm, which is why I've got this doted equator there to indicate that there is a relative movement of these determinants.
OK?
It's not the whole embryo that's moving.
It's just part of the embryo that's moving relative to the other part.
And this movement of determinants defines the dorsal and the ventral side of the embryo.
OK.
Here's a movie to show you that.
OK.
So this is a movie that is about the first 20 minutes of fertilization.
And you can see here pigment moving from one region to another region.
Now, the pigment is not the determinants.
It just moves with the determinants.
OK?
So this is an indication that there is stuff moving, actually, from the vegetal pole region up towards this so-called animal pole region.
So the terms animal and vegetal pole indicate two different sides of the radially symmetric egg.
The animal pole is pigmented.
The vegetal pole is not.
And they're just useful landmarks.
And, again, this is a frog embryo.
Things may be similar in humans, but we understand much more about frogs.
So that's why I'm going to tell you how things work in frogs.
OK.
So there is a manifestation of these dorsal determinants moving.
These dorsal determinants move by a really fantastic process.
They move on microtubules.
Remember microtubules?
What part of the embryo, what part of the cell were microtubules part of?
Anyone remember?
Dredge back in your memories.
I heard something.
Yes?
Thank you.
Cytoskeleton.
Yes.
See me for a frog afterward.
So the microtubules are part of the so-called cytoskeleton.
They are large rods of proteins, polymers of proteins.
And when the sperm enters the egg it brings with it a centriole that is a microtubular organization center.
And what it does, in this series of three pictures, is to take this mass of red microtubules that are oriented in every which way, and it organizes them so they form these beautiful parallel tracks of microtubules in the outer cytoplasm or between the outer and inner cytoplasm of the egg or of the zygote, of the fertilized egg.
So the sperm nucleus or the sperm, excuse me, the sperm centriole is organizing this microtubular array.
And it's on these tracks of microtubules that these determinants move, kind of like train tracks.
And I'll tell you in a moment, it really is that way.
So, oh, it didn't quite work.
But OK.
So here is a zygote and here is this protein I've told you about, beta-catenin.
Now, in the zygote beta-catenin is phosphorylated.
And that renders it unstable for various reasons.
And it's also cytoplasmic.
OK?
With the action of the determinants I've been telling you about, by about the two to four cell stage, so very soon after fertilization the beta-catenin on one side of the embryo, the side where these determinants are has been dephosphorylated.
So there is a post-translational modification.
Dephosphorylated.
And that renders it stable for various reasons.
It's no longer subject to protein degradation.
It also allows it to enter the nucleus.
It allows it to act as a transcription factor.
So these determinants are changing the stability and the protein structure of this beta-catenin transcription factor.
Beta-catenin is really an important protein.
This is a wild type frog embryo.
Two wild type frog embryos.
The head, tail, here's the eye forming.
In embryos that have been depleted of beta-catenin you get these kinds of blobs that have no dorsal structures at all.
They've got no muscle.
They've got no nervous system.
Nothing that would normally come from the dorsal side of the embryo.
So what is the identity of these dorsal determinants?
And, again, this you have partly on your handouts.
And the identity of the dorsal determinants ergo is something that stabilizes beta-catenin and dephosphorylates it.
And what these are believed to be presently are two proteins called GBP and dsh.
OK?
You don't need to know the names.
It's the principle that's important here.
But what's very interesting is that GBP and dsh proteins bind to another protein called kinesin, and kinesin attaches or is part of the microtubule system.
So you get these determinants attaching to the microtubules, and as these microtubules polymerize and have a rotation associated with them, it's not quite clear they rotate at this point, this GBP and dsh are associated with the microtubules.
And then these two proteins, GBP and dsh, also called disheveled, inhibit another protein called GSK3 which is a kinase that phosphorylates beta-catenin and renders it unstable.
So you're getting here dorsal determination.
You can go think about this more clearly.
Dorsal determination because you are inhibiting an inhibitor.
All right?
Complicated?
Yes.
Beautifully understood?
More or less.
This is really the most beautiful example that we know of how you get determination of a particular region of the embryo.
OK.
So one step.
We probably have about 20 steps to go.
And clearly we're not going to cover them all so I'm giving you a little meander through the number of steps that are involved.
So another step informing muscle, that I want to tell you about, is forming a particular kind of cell from which the muscle will arise.
So not only does muscle arise from cells that are dorsally located.
Muscle also arises from a cell type called the mesoderm.
The mesoderm is part of a system of cells that are the earliest known cell types in the embryo.
And these are the germ layers.
And the germ layers consist of mesoderm, and also of two other cells types called ectoderm and endoderm.
OK. And these cell types were discovered a lot time ago.
They're not differentiated cell types but they do have different properties.
They are stepwise along the way to becoming different things.
The mesoderm, shown in red here, gives rise to the muscle, blood, kidney, heart and various other things.
OK?
The ectoderm, for example, gives rise to all of the nervous system, and the endoderm to the gut and the lung.
And they get their names from their position in the early embryo.
The endoderm is on the inside, the ectoderm on the outside, and the mesoderm between the two.
OK.  How does the mesoderm form?
The mesoderm forms through another complicated system.
It forms through the action of a transcription factor called VegT.
And this is a maternally expressed transcription factor.
So we can distinguish maternal and zygotic effects, genetically and in development.
VegT is maternally expressed.
That means it is present in the egg.
VegT in term activates in just one region of the embryo another gene called nodal.
It's actually a set of genes.
Nodal is a secreted factor.
It is actually a ligand and, therefore, an inducer.
And it is zygotically expressed.
So another principle, "zygotically X" for expressed, and "maternally X" for expressed.
OK.
This is one your handouts.
So we're talking now about the same time of development that we've been talking about but a different system that is working in parallel.
Here is an embryo.
And I've shown you between the 200 and the 500 cell stage that has got this VegT transcription factor present in the nucleus.
It's a transcription factor.
And you can find a concentration gradient of this thing.
In one part of the embryo at this equatorial region there are low amounts of it.
Towards the vegetal pole there are high amounts.
And where you have this low amount of VegT you get a band of cells that's going to become mesoderm.
This mesoderm then goes on to express this nodal gene which is a secreted ligand.
And, for those of you who are familiar with this, it's a member of something called the TGF-beta family.
So this gives you another set of cells to deal with.
And what I'm going to tell you now is that to think about where the skeletal muscle comes from we have to superimpose the two things I've told you about becoming dorsal and becoming mesodermal.
So the next thing we need to do is to talk about the dorsal mesoderm.
And we can write an equation here where beta-catenin plus nodal makes dorsal mesoderm.
And I'll tell you that this dorsal mesoderm has a very special property and a special name.
It's called the organizer.
Is the dorsal mesoderm the future skeletal muscle?
I'm sorry to tell you it's not really.
There's another step involved.
And I will tell you in a moment that the organizer is going to induce the precursors of the skeletal muscle.
So let me write this down just to give you a sense of where we are.
OK.
So let's move on here.
And let's define dorsal mesoderm by an equation.
And, in fact, you can define this whole process by an equation.
But, of course, the genes that have to be expressed and so on have to be expressed in a specific temporal order.
So it's not a simple linear equation.
It's got a time component, too.
OK.
So here we have beta-catenin on the dorsal side.
And we have mesoderm defined by nodal signaling in a band across the equator.
And if we superimpose them we get a region where both of these factors are in the same group of cells.
And where both of these factors are in the same group of cells we get a special type of mesoderm formed called the dorsal mesoderm or the organizer.
The organizer is a very famous piece of mesoderm.
And it is famous for this reason.
This is a very old experiment that was done in the 1920s where one could take a piece of donor tissue from a donor embryo, that is the future organizer, and move it to a host embryo, and put it on the side of the embryo where the organizer wasn't.
OK?
So to put it on the ventral side of the host embryo.
And when this was done, to the surprise of the investigators, because they really didn't do the experiment for this reason, they found that they got these conjoined twins.
Conjoined twin embryos.
Here's your host embryo.
And this is a second embryo that's formed.
And you could look at see where that piece of transplanted tissue went.
And when you did you found it was in a strip along the back of the embryo, but most of the second embryo, the conjoined embryo came from tissue that was the host embryo tissue.
In other words, this piece of transplanted tissue had organized or induced a second embryo, including all the skeletal muscle.
OK?
And including a head and a brain and so on.
This is a very famous experiment, very famous piece of tissue.
And for this experiment the Nobel Prize was awarded to Hans Spemann in 1935.
Here are conjoined frog embryos made by transplanting organizers.
You can see they have two heads and they're joined at their tail region.
What about humans?
Well, the same thing is true in human embryos.
Usually identical twin embryos will split somewhere between the two cell stage and a very early stage where there is something called the inner cell mass.
And you get two identical embryos made and you get completely separate embryos made.
Sometimes, very rarely, you get incomplete splitting of the embryo.
And the organizer is split in two in a human embryo, and you land up with human conjoined twins.
This is very rare.
The survival of conjoined twins is very rare.
There are some instances, and I've put on your handout a diagram of, these are actual girls who had been diagramed that were just like the frogs, with shared heads, with a shared trunk and separate heads.
So the processes in frogs, as far as we know, are very similar to those in humans.
Now, what I'm going to do because I have very interesting stuff I'd like to finish this off, finish off here, but I see time is creeping on, is to end off with a teaser.
And to tell you that the final step in getting MyoD on is to activate these things called somites.
And I'm going to take five minutes at the beginning of next lecture to complete this and then we will move onto our next topic.
So thank you.
What I'd like to do today, is to give you an epilogue, the very end of the last lecture on cell type, that I did not want to cram into the very last one minute, so I'm going to take five minutes now to talk about that, and then I want to move on to a topic that is very related, because everything, of course, is very much related by the control of gene expression, by cell-cell interactions, by cell fate, and so on, but I want to talk to you about a particularly important set of interactions between cells.
Here we are, on your game board of life, we're moving up through the formation module, talked about cell type, and we're going to build on that conversation in talking about fertilization today.
So let me go back and remind you where we were.
At the very end of last lecture we were talking about cell type, and what I wanted to do with you, was to trace for you the ontogeny, the development of a particular type of cell, and indicate to you how many different interactions there were between cells -- -- how many different sets of genes had to be activated, and had to interact with one another, to get you from a fertilized egg, all the way to a differentiated cell type, and I used the paradigm of skeletal muscle.
And really, the point that I think you got, was that this is very complicated, and if you go on in your careers and think about, as engineers for example, as biologists, engineering a particular kind of cell from another kind of cell, you really have to know these steps along the way, and ask whether or not in order to get from point a to point b, you have to go through the whole set of 30 steps -- -- or whether you can circumvent those steps, and short-circuit, and go on a much shorter route.
And we'll talk more about this in subsequent lectures, but this is very relevant for thinking about how to engineer particular cell types.
What we were talking about was an extraordinary experiment that was done in the 1920's, that demonstrated a piece of embryonic tissue called the organizer of the dorsal mesoderm, was able to activate the formation of a second -- [PAUSE[ -- axis, or a second conjoined embryo, where most of the second embryo was derived from the host tissue, into which this piece of organizer, or dorsal mesoderm, had been transplanted.
And this organizer was able to induce a second embryo, including skeletal muscle, must be made from the host cells.
OK, who does not have their handouts here from the last lecture?
I'm not trying to nail you, I just want to know if I need to write something on the board.
I need to write something on the board, OK.
So, the notion that we were left with was that, until now, we had gotten to the point where genetic information that had defined the dorsal part of the embryo, synergized with genetic information that formed the dorsal mesoderm that formed the mesoderm.
That gave rise to this piece of tissue called dorsal mesoderm, abbreviated meso, which is also known as the organizer.
And this piece of dorsal mesoderm, or organizer, itself, as I said, a very powerful inducing center, and it induces formation of a group of tissue blocks called somites.
These somites, as I'll show you in a moment, are segments, or segmental units, that are present along the whole axis of the body, the whole trunk region of the body of vertebras.
And they actually hark back to our ancestry, derived from organisms that were fully segmented, worms for example, earthworm type animals, that had multiple segments.
OK.
The somites, in turn, become subdivided, and I put this in just to indicate, again, the complexity of the process, and part of the somite, termed the myotome, activates the expression of the transcription factor myoD.
And it was pointed out to me by Christine that I was abbreviating transcription factor, TXN that is my abbreviation, many of you may remember from way back, for transcription.
So, if you have in your previous notes, a TXN, and you're not sure what I meant, I meant transcription.
OK, so these somites get subdivided, they make the myotome that expresses myoD, and as we talked about last time, we come in full circle to where we wanted to be, which is the activation of the skeletal muscle program.
Let me show you this on this PowerPoint slide.
OK, so here are the organizers in diagram, activating the formation, or inducing the formation, of these somites.
The somites get divided, myoD is expressed, myoD and other family members, remember I made the point this was a member of a redundant transcription factor, a redundant gene family, myoD maintains its own expression through a positive alter regulatory route.
That means it binds to its own promoter, and perpetuates, or activates, its own transcription, so once myoD is expressed, it maintains its own expression, a very popular way of stabilizing gene expression in particular cell types.
OK, here are some pictures of early human embryos.
Now, you should realize, the reason in this previous slide that I went from a round circle representing a round ball of cells as an embryo, to no circle, is that everything I've told you until now, took place in a round ball of cells, OK?
Up to about the 4, 00th cell stage, or about the 10, 00th cell stage, the frog embryo is just a round ball of cells, and most embryos are not very distinguished looking.
But the process of making somites and dividing them, goes on until the embryo is a million cells or more, and during that time, the embryo reorganizes its body plan, so it's not longer a ball of cells, it starts to take on its characteristic form.
And some of the characteristic form it takes on, is shown in this human embryo, about 19 days after fertilization, this region here is going to be the nervous system -- -- and on either side of the nervous system, I've got in red here, boxed these little segmented blocks, and you can see these segmented blocks in older embryos.
Those are the somites from which most of the skeletal muscle is going to form.
This is a movie to show you the segmentation of the somites.
So here they go, here are the blocks of cells being divided one from another, it's a very beautiful process, and you can see it's very ordered.
Down the middle here is the neural tube, which is starting to give rise to the central nervous system, and on either side of the neural tube is this future muscle tissue, or the future somites that include the skeletal muscle, and you can see them being divided symmetrically on either side of this midline here, in to these chunks of tissue.
And it's actually a very interesting story in the regulation of gene expression, as to how you time things in such a coordinated fashion.
In all animals, the somites become segmented first in the head region, and then the segmentation moves down towards the tail, and there is an intrinsic timer in the embryo, that I don't have time to describe to you, that again, converges on the regulation of gene expression, that allows this extraordinarily coordinated segmentation of the body plan.
OK, so what I've done is to try and trace this ontology of the cell type.
What you'll realize if you think about this, is that I haven't actually told you what transcription factors actually activate the myoD promoter in the somites.
This is an RNA expression pattern, and the whole myoD regulatory loop, is controlled in a transcriptional way.
And we don't know what exactly activates myoD expression in the somites, but we know a lot about what happens before.
And with that, I am going to move on to what I want to spend most of today talking to you about, and that is the question of reproduction.
And I'm going to talk to you about the molecular biology of reproduction, and particularly we're going to talk about sexual reproduction -- -- as one of the most important examples of cell-cell interactions.
And in fact, the one without which none of us would be here, although that, in a sense, is a silly thing to say because without most of the interactions, the cell interactions during development, none of us would be here anyway.
But, this is a very interesting process, a very important process, and it exemplifies some points that I want to bring up here.
So the notion here is that two haploid cells, the sperm, and N is my designation for haploid, I presume you're all with me when I use N and 2N, as haploid and diploid, OK, fine.
Sperm and egg go through the process of fertilization -- -- to form a single cell, a zygote that is diploid, and the zygote will then go on to perform a multi-cellular embryo, etc.
Now, there are some really extraordinary things about this.
One of the extraordinary things is that the sperm and the egg are fully differentiated cells.
I have been telling you, indeed really trying to hammer into you, the notion that cells progress from less specialized states to more specialized states.
And this is the one instance during development, where the flip is true, where the opposite is true.
So the sperm and the egg are fully differentiated cells, they have reached this final cell type, but when they fuse, and they form the zygote, they now make the most fully undifferentiated cell.
So the zygote is uncommitted, that may or may not be spelled correctly.
So the zygote is uncommitted, and I want to introduce you to a new term, which is that of potency, where potency refers to the number of possible fates that a cell can assume.
We're going to talk a lot about this when we talk about stem cells.
The number of possible fates that a cell can assume, and the sperm and the egg are uni-potent, they have one possible fate, which is themselves.
And the opposite is true of the zygote.
It is totipotent, it can become everything, all possible cell types.
And that is the antithesis of what happens during development in essentially, I would say, in every single cell type of the body.
And it's extraordinary, and we don't understand, we don't understand, how you get this complete reversal of fates, although I'll try to go through this with you in a future lecture.
All right, why go to all this trouble?
This is a lot of work for an organism to go through, and the bottom line is that it leads to genetic re-assortment, and it leads to genetic diversity.
So the notion of sexual reproduction, I'm going to write it up here, is that it engenders genetic diversity, and it does so because genetic materials becomes re-assorted during meiosis, and during recombination, or during the joining together, of the sperm and the egg.
OK, I divided this lecture up into a few parts.
The first is gametal genesis -- -- where the gametes are the egg and the sperm, just a different name for them, and gametogenesis is one of those developmental biology terms that has the suffix "-genesis", the creation of the gametes.
Both the sperm and the egg start off from diploid, precursor cells, and this is a very interesting story, but I'm not going to tell you.
But I'll remind you, well, I'm going to allude to it.
I'll remind you that long ago, I showed you these beautiful series of pictures from worms.
When we talked about determinants, cell autonomous factors that control cell fate, and I showed you how these determinants were segregated into a single cell, of the 32 cell worm embryo, and that this cell was going to give rise to the gametes, the egg and the sperm.
In fact, the exact same thing is probably true in all animals, including ourselves.
And certainly in frogs, and in fish, we know that there are determinants that are segregated, and we know what these are, they turn out to be proteins that bind RNA and control translation of presumably specific target genes.
So, somewhere back in the dawn of time, during development, the germ cells become determined -- -- but I am not going to talk about this, I am going to talk about them when they actually already know that they're germ cells, and when they start thinking about assuming their final differentiated fate.
And I want to make a couple of contrasting points, and OK, well let me do my board work and then we can look at that.
OK.
The precursor of the spermatozoa is the spermatagonium.
This is a diploid cell that undergoes meiosis, to give rise to a haploid spermatazoan.
The spermatagonium is a moderate sized cell, it's just a regular sized cell, and when it undergoes its conversion, its differentiation, to a  spermatazoan, or a sperm, it becomes very small, it jettisons most of its cytoplasm, and it becomes motile.
So you go from a medium sized cell to a tiny cell that has the very important property of being motile.
And this is true, really, universally, some, and I'll talk about this in a moment.
The way that spermatozoa are motile is slightly different in different organisms, but they pretty much, all spermatozoa are motile.
In contrast, the egg arises from a diploid oogonium and the oogonium also starts off as a medium sized cell, but it does the opposite.
It gets very big, OK?
And the oogonium moves to change into something called an oocyte, or I prefer the term, egg, and this was a haploid cell.
OK.
It's large, by necessity sessile, and it's large because it stores a bunch of stuff, and the stuff that it stores is food, to nourish the early embryo before the embryo can fend for itself, or before it can be nourished by the mother.
And it also contains components that regulate early development.
So it's full of things like determinants.
It's also full of machinery that can make ribosomes and so on, and the size of the egg varies from organism to organism.
In frogs, for example, the egg is about a millimeter in diameter, in humans the egg is about 50 micrometers in diameter, because the mother nourishes the egg sooner than, or the embryo, sooner in humans than in frogs.
All right, so let's look at a couple of these diagrams that I gave you.
This is to remind you about meiosis and what meiosis is, and I want to make a couple of points that are up here on the top, some of which are more important than others.
Spermatozoa, and spermatogenesis, is featured by continuing mitosis throughout life of the male.
All the meiotic products become sperm, but this continuing mitosis is what's really important.
The spermatogonium, the precursor of the spermatozoa, persists throughout life, and so you get a lot of sperm produced.
So we can add to our list here, that you get many sperm.
In humans there are about five times ten to the seventh sperm per ejaculate.
If you go about five-fold lower than that, you're in the range of infertility.
We can talk about why that is, and so one can do calculations, and it's been calculated that males can make up to ten to the 13th sperm in a lifetime.
If you do the calculation, you will understand that not all of those sperm are used, and some are them are resorbed, so you can do that rather interesting calculation another time.
OK.  Oogenesis, in contrast to the plentiful and cheap sperm, and we'll come back to this point in a moment.
In females, mitosis of the oogonia, ends before birth, and in fact, by the time a child is born, it is a female, it has about a million or so, primary oocytes, which are still diploid, they're along this pathway towards formation of the egg.
And by the time the child is about five, there are only about 500, 00 primary oocytes sites.
OK?
And those cells have withdrawn from the cell cycle, they are not making anymore of them.
So that is your pool from which to make eggs, and in fact, most of those, 90% of those, are going to die, and you're going to get a greater than 90% are going to die, and only about 500 eggs mature during the lifetime of a human female.
OK?
And it's actually not quite clear why, but it's certainly harder to make a good quality egg, probably, than to make lots of smaller sperm, for reasons of energy expenditure of the animal.
All right, let's look through these slides a little more.
The testes is an extraordinary organ that comprises many, many, many tubules, all coiled together.
Within the seminiferous tubules there is an array of sperm production that is architecturally very interesting.
Ignore this red up here for the moment, and look at this architecture of the wall of one of the The developing spermatozoa are embedded in cells called Sertoli cells.
Sertoli cells supports, and nourish, and induce the formation of the mature spermatozoa.
So the spermatozoa that are most immature, are lying away from the luminal, the cavity, of the seminiferous tubule, and as these cells mature and undergo meiosis, they move closer and closer to the lumen, until you get a bunch of spermatozoa, with these tails pointing out into the lumen of the tube, ready to be shed upon ejaculation.
So it's a very beautifully, architecturally very beautiful process.
Now what does the spermatazoan actually look like?
It's got three parts.
It's got this head that's got a nucleus that is highly condensed.
I've told you previously that it's a real feat to pack the meter of DNA that's in every cell, into a little, tiny nucleus.
You have to really pack it tightly.
Well, it's even worse for a sperm because the nucleus of a sperm is an order of magnitude smaller, in many cases, than a somatic cell.
So you have to pack the DNA even more tightly, and there are special proteins that overcome the charges on the DNA, to overcome the charge repulsion on the deoxyribonucleic acid, OK?
So these are basic proteins that really pack the DNA tightly, to fit into this compact nucleus.
The other thing we'll talk about, in a bit, is this structure called the acrosome.
It derives from the Golgi, which you remember is involved in protein trafficking.
Two other regions of the sperm that are almost universal, and very important, are this thing called the mid-piece, which is packed with mitochondria that are going to provide energy for the sperm as it moves.
And finally, this tail region, which is a flagellum, now what's a flagellum?
So, a flagellum is one of these biologically fantastic structures that uses polymers of proteins to make something that works in a concerted fashion.
The proteins that it uses makes up the microtubules, so they are the tubulin proteins, we'll have more to say about them on Friday.
We mentioned microtubules in the last lecture, in this case, microtubules pack together in a very ordered way, you can read this in your book.
Microtubules pack together in a very ordered way, to make this very long structure, that when supplied with energy, will beat in a regular, often a sinusoidal fashion, and propel the cell forward, or wherever it is going.
All right, female reproductive system.
The ovary is the sight of egg production.
In mammals, the uterus is the site of embryo growth, fertilization usually occurs somewhere between the ovary and the uterus, in the tube, the oviduct, or the fallopian tube, between the ovary and the uterus.
All right, this is a diagram from your book again.
In the developing, in the ovary, one can find eggs of developing, or oocytes sites of different stages, as they undergo development.
They grow larger, and lie in a fluid-filled cavity, and eventually, I'll talk a bit more about this in a moment, this fluid filled cavity gets very large, and it bursts, and releases the egg.
It's very interesting, in mammals, the egg is actually released almost into the body cavity, and it's gathered up by so called fimbrae, or folds of waving tissue that lie at the opening of the oviduct, or fallopian tube, and these kind of make a current that draws the released egg into the oviduct, so it can get on its way to the uterus.
Subsequent  to the egg being released, what's left of where the egg developed in the ovary, forms this thing called the corpus luteum, which secretes substances, hormones, that I'll talk about in a moment, that are very important for maintaining pregnancy, if this occurs.
All right, you can see what I'm dwelling on here, not dwelling on.
I'll dwell on things that I feel are more important, by writing things on the board, OK?
So, stay with me.
Let's go now to a system that I do want to dwell on more, which is the hormonal control of gametogenesis.
Professor Jacks talked to you previously about hormones.
Hormones are made in most animals, and their characteristic is that they work at very long range.
So I want to characterize these as long-ranged, inducers, and they can be very long-range inducers.
The hormones that control gametogenesis are made in the brain, in the hypothalamus, and the pituitary and they travel way through the bloodstream to get to the ovaries and the testes.
This looks like a hermaphrodite here, [LAUGHTER], OK and this, I just realized this, OK, whatever.
To get to the ovaries or the testes, or both, whatever, {LAUGHTER], OK, anyway, that point I want to make is that these hormones work at a very long distance.
And I want to contrast that with the kinds of inducers that wee have been talking about in the preceding couple of lectures.
The kind of classical, embryonic inducers, which act at very short range.
So, for example, we talked about nodal signaling -- -- last lecture.
For example, nodal, which is a ligand I mentioned, pivotal in making the mesoderm, nodal can only work over one, or a few, cell diameters.
So if you were cells, you could have a conversation with one another, or maybe, the signal could get all the way to here, but it sure wouldn't get to the back of the room.
Whereas, if you were a hormone, your signal would get all the way back to the back of the room, and probably all the way down the infinite corridor.
OK, so there's a real order of magnitude different in the way hormones and other inducers can act.
So, one to a few cells.
All right, there are a bunch of different kinds of hormones that are involved in control of gametogenesis.
I want to distinguish the steroid hormones, which are derivatives of cholesterol, and we mentioned them long ago in the lipid section.
Steroids that include testosterone, estrogen, and progesterone, and act through a series of receptors that actually lies within the cell, they're not membrane-bound receptors, they are so-called nuclear receptors, so they act through nuclear receptors.
I'm actually not going to write that on the board.
And the other hormones I want to indicate are peptide hormones, that as their name indicates, are small polypeptide chains, and these include, I'm going to put abbreviations up, and you can get the full names later.
FSH, LH, and GNRH, follicle-stimulating hormone, luteinizing hormone, and gonadotropin-releasing hormone.
One of the things that is similar about the hormonal control of gametogenesis, to many other things we've been talking about, is the notion of feedback control.
So, there is an enormous amount of feedback, both positive and negative in the circuitry that gives rise to the formation of egg and sperm.
This is an example of the circuitry leading to stomatogenesis.
Here is GNRH, released from the hypothalamus that stimulates the anterior pituitary, to produce lutenizing hormone, or follicle stimulating hormone.
These act on cells within the testes, which act to, one set of cells acts to activate testosterone, the major male steroid hormone, and that acts back on the Sertoli cells in the seminiferous tubules, and together, these orchestrate the production of spermatozoa, and other various responses to these hormones.
Now, the green indicates positive loops, and the red indicate negative feedback, so too much testosterone can have the effect of inhibiting the anterior pituitary's production of lutenizing hormone, or follicle-stimulating hormone.
And there's a very careful balance of these hormones in the normal animal.
Now, in females, things look similar.
Again, GNRG, LH, and FSH, are produced to stimulate, release, and maturation of oocytes, and release from the ovary.
The ovary, instead of producing testosterone, although it does produce some of that, produces estrogen and progesterone, and those are both required for normal maturation of the egg, and also for subsequent pregnancy.
And also, as you will see, there is a feedback loop of both estrogen and progesterone, onto the hypothalamus and the anterior pituitary.
And as this feedback loop that is exploited in the birth control pill, where if one gives high levels of estrogen and progesterone, you suppress the production of both GNRH from the hypothalamus, and LH and FSH from the anterior pituitary and therefore suppress the production of eggs from the ovary.
Now actually looking, it's interesting that there's a female birth control pill, because really, there are certainly targets in the male for birth control pills, but these have not yet been exploited.
But the notion is that you exploit this feedback loop, and its well-enough understood that you can.
OK, so let me move on to talking about the process of fertilization, per say.
The process of fertilization involves the egg and the sperm coming together, and recognizing one another as being from the same species.
This is very important -- -- very important that you get species-specific recognition, in order that you get correct transmission of genetic material.
So the first thing that we mention is sperm-egg recognition -- -- and this has many levels.
In some animals, the egg, or the uterus, or the female sends out various chemo attractants to attractants, to encourage the sperm to get to the egg.
Some of these have been isolated, and there are small peptides, for example, sea urchins, which are the organism that have been used to study fertilization.
Most have very small peptides, or the chemo attractants, in humans there is some data that there may be chemo attractants released by the oviduct that causes the sperm to swim in a directed fashion, but it's very interesting that they can't be that powerful, those chemo attractants, because one needs so many sperm to be released in order to fertilize the egg.
And it's very interesting that although there are five times ten to the seventh, or more, spermatozoa in a human ejaculate, if you actually look in the region of the oviduct, where fertilization would take place, you only find a few hundred sperm there.
So most of them disappear on the way, and so if there are chemo attractants, I say they're not very good ones.
OK, another process that is very important in mammals, is something called capacitation.
The sperm, as I told you, have this capacity to be motile, that they contain a lot of energy stores, or they contain mitochondria that can convert energy to movement.
But they save that energy for a time when they think they might be in the vicinity of an egg, and when sperm are exposed to fluid from a mammalian oviduct, they undergo an increase in motility, so capacitation leads to an increase in sperm motility.
This process is mediated by some kind of signal transduction mechanism, that involves the second messenger, cyclic AMP that you discussed with Professor Jacks, and this is very important.
If capacitation doesn't occur, fertilization doesn't occur either.
OK, so once the sperm are in the vicinity of the egg, they then have the challenge of fusing with it.
This is an animation I showed you previously, of a sperm moving towards an egg.
Now this is a mammalian egg, and it's surrounded by a bunch of stuff.
It's surrounded by these circly things, which are cells that released from the ovary, along with the egg, and it's surrounded by a number of so-called membranes.
The most important one of which is called the zona pellucida, and as the sperm goes towards the egg, and approaches the egg, it has to burrow its way through a bunch of membranes to get there, this animation doesn't finish fertilization clearly.
OK, so if you look at a mammalian egg at fertilization, there is the egg per-say, that is a cell surrounded by a plasma membrane, and then there's this thick, yellow layer, that's called the zona pellucida.
Around that, is this layer of cells called the cumulus, and the sperm have to borrow through both the cumulus layer, and particularly the zona pellucida, before they get to the plasma membrane, and before they can then try to fuse with the plasma membrane, and enter the egg.
So, the second thing that we need to think about is contact, and there's two steps to this.
The first contact and they're actually very interesting steps because there's a bunch of reciprocal interaction, and reciprocal receptor ligand interaction.
The first interaction is between the sperm, which provides a receptor, and the egg, and this layer called the zona pellucida around the egg, and the zona provides the ligands for the interaction.
And it turns out that the zona pellucida, although it's called a membrane very often, is not a lipid by-layer, it's a glycol-protein layer, OK?
And the proteins in the zona are called zp1, zp2, and zp3, and they form a very complex meshwork of glycol-proteins.
The sperm receptor in mammals recognizes a protein called zp3 in the zona, and then subsequently, there's a shift in carbohydrate arrangement of the zona proteins, and it then recognizes something called zp2.
Now, this receptor-ligand interaction has a very interesting consequence.
It leads to activation of second messengers, it goes through a system of proteins called G proteins, but what was most important is that it leads to the release of calcium within the sperm.
And this calcium, let me move the calcium here, so this leads to the release of calcium with the sperm, and this causes something called the acrosome reaction.
Now the acrosome, as I said, derives from the Golgi, and it is filled with Golgi-derived proteins, particularly proteases.
Upon the acrosome reaction, there is a massive exocytosis of the acrosomal contents, to release proteases, and these proteases digest the zona.
All right, some diagrams.
This is a diagram you have on your handout.
Here is the sperm moving through the cumulus layer, getting through the zona, at contact there is a reaction, such that the acrosome releases its contents, and the next step can happen.
This is a movie of the acrosome reaction from the horseshoe crab, limulus, it has an extraordinary acrosome.
The acrosome in limulus compromises acrosomal protein, the proteases, but a lot of unpolymerized, and upon contact with the limulus egg, the, let me stop this so you can watch it while I'm telling you about it.
The actin that is in the acrosome polymerizes, this is a matter of seconds, and sends out this incredibly long process, here it comes, look at this.
See this thing shooting out?
That is a polymer of actin that is coated with these acrosomal proteases.
So this is the sperm head, and so it's shooting out this process that is many, many times longer than the sperm head, and it's allowing it to get into the limulus egg.
Mammalian sperm do not have such a spectacular acrosome reaction, but they do have one that serves the same purpose.
OK, all right, so the next thing that is going to happen is that there is a second reciprocal interaction, and this time, the egg is going to provide the receptor, and the sperm is going to provide the ligand.
So, we now have the sperm, whose acrosomal membrane that had been the Golgi membrane, is now exposed, and is covered with ligands, so the acrosome, the acrosome membrane, is covered with ligand.
This interacts with receptors on the egg plasma membrane, and there is, as a result of this receptor-ligand interaction, the activation of a signal-transduction system, and the signal transduction system does two things, so there is this interaction leads within the egg, to activation of signal transduction, that has the effect, eventually, of releasing calcium with the egg.
And this calcium released within the egg does two things.
The first thing that it does is to prevent any additional sperm from entering the egg, and I'll tell you about this in a second.
So it blocks, or activates the block, to polyspermy, and in particular it activates the so-called slow block to polyspermy, and it also activates, a little while later, the zygote, to begin dividing, to enter the cell cycle, and to begin various other processes required in the early embryo.
So cell cycle entry of the zygote.
All right, so let's look at a movie of the sperm entering the egg.
OK, here is, I'll show you this again.
Here is the sperm, its tale is sticking out, there, it's going right into the egg.
The cell membrane reciprocates, and buckles, and in some organisms, it engulfs the sperm as it's going in, but the actual sense in most organisms, the membranes either fuse, or somehow you get the whole sperm being pushed into the egg.
In most animals the sperm membrane doesn't actually enter the egg, but in some animals it does, and in some cases it doesn't seem to matter.
OK, what is this block to polyspermy?
This is something that happens in the first 30 seconds after fertilization, and it involves exocytosis of a set of granules called the cortical granules, that contain enzymes, that do two things.
These enzymes firstly, when they are released, break apart junctions between the outer membranes, in this case, this is shown in frogs.
The outer membrane in frogs is called the vitelline envelope, its equivalent to the zona pellucida in mammals.
And before fertilization, this outer envelope is tethered by proteins, to the plasma membrane, so they're very closely opposed.
At the release of the cortical granules, these protein contacts are broken, and the fertilization envelope lifts off the surface of the egg.
A number of other things happen, as well.
In mammals the zona pellucida is completely remodeled, so it no longer binds with the sperm, the carbohydrates are stripped off the zona proteins so that they no longer bind sperm.
This is a movie of the block to polyspermy, or the consequence of the block to polyspermy in the frog.
Here is the frog egg.
At fertilization, watch carefully, you can see there is a big halo that arises above the frog egg, there it is, and this is a consequence of cortical granule exocytosis, and the blocked polyspermy.
OK.  OK.  Before you go, what I did not cover in lecture today, don't go for one second.
What I want to talk to you today about is the creation of 3-dimensional structures.
And I want to emphasize their connection to the function of the organisms and the organ.
Let's look, for example, at the kidney.
This is a fantastic example of structure- function relationships.
The kidney comprises thousand of tubules that are involved in filtering the blood and collecting urine for excretion.
Now, not only do these tubules function to transport the urine as it is being processed, they also are connected very closely to various cell types that are involved in this filtration process.
So the 3-dimensional structure of the kidney is integral, is required for its normal function.
And this is true in essentially every organ.
So it's very important to think about the relationship of different cells to one another in a particular organ.
And if you're thinking about, for example, using tissue engineering to develop an artificial kidney and making some kind of pastiche of cells and polymers and putting that together in some functional way -- -- it's important to understand these structure-function relationships in the normal organ before you try to engineer something that is a surrogate.
Individual cells also have particular shapes, and this is integral to their function.
We'll talk a bunch about neurons, cells that have very long processes that transmit nervous impulses or transmit electrical impulses and communicate in that way.
The shape of the neuron, the structure, the 3-dimensional structure is absolutely required for its function.
Tom, I think we could have a little more light at the back there.
It looks really dark if you guys are trying to take notes.
Thank you.
OK.  We have, you've seen this movie previously.
OK.
This is the zebra fish.
The zebra fish embryo.
Isn't it dark at the back?
Yeah.
Tom?
Oh, it takes a while.
It takes a while.
OK.
I want to show you this movie again that you've seen a while.
You have it on your website.
And I want to show it to you now in a different way.
We're actually going to first show that during the early development of the fish, these two cells on top initially give rise to a ball of cells sitting on top of this yoke cell that is just a ball of cells.
And then suddenly these cells start to move during the gastrula and then the neurula phases of development.
And it's this cell movement that builds structure.
And I want to try to address with you how this cell movement occurs and how it builds structure.
So let's look at this movie again.
Here we go.
Cell division, building up the raw materials to make the embryo.
A group of cells sitting on top of the yolk.
And now watch here.
It starts to move.
Those cells have made the decision to move, and they going to move towards this one dorsal side of the embryo.
And they are going to move and move some more to build the eye, the brain and the various somites along the length of the axis.
OK.
So that process, the generation of 3-dimensional structure is a very large part of the process of building the embryo.
And it's interdigitated with making the cell types.
A muscle cell is not a muscle cell and it's not functional unless it is a fused myotube that has the appropriate 3-dimensional structure.
So cell type and 3-dimensional structure are very closely related.
OK.
So let's talk about a toolkit that is involved in this process.
And the first thing that we can discuss is what is in this toolkit.
Well, actually, you don't really have much in the toolkit to build an organism.
I can think of one thing.
What's in your toolkit to build your 3-dimensional organism?
Yes.
Did I hear cells?
Yeah.
I hope I heard cells.
Well, there are cells.
And then if you want to look for something else to build the organism with there are cells and then there are cells.
That's what you've got.
In your toolkit you've got cells.
And the challenge of the organism is to use that singular building material to build the various and the huge array of structures that there are.
So what about these cells?
And the first thing about these cells is that they come in two forms.
They come as so-called epithelia or epithelial sheets.
And they come as single cells which are called mesenchyme.
And these two types of cells interconvert with one another.
And it's from these two groups of cells, that can be different cell types, but they're either sheets or they're single cells, that one builds the organism.
So the epithelia are sheets, the mesenchyme are single cells.
And let me look at the next diagram that I drew for you.
And you have it in front of you.
If you don't have the handout, does anyone not have the handout?
Could I have, Lesley, could you [UNINTELLIGIBLE] please?
Thanks a lot.
OK.
If you look at the handout, you have this except for one thing I added this morning.
Here is a sheet of cells.
So I'm going to do minimal board work today because you have a lot of the information right in front of you.
You have a sheet of cells here.
And this sheet of cells called an epithelium is a sheet of cells because it is joined together very tightly.
And the cells are joined together by various junctions that I've shown here in yellow or that I've shown in blue.
And this sheet of cells, as Professor Jacks told you, has two part, has two sides, an apical side and a basal side.
OK?
Remember from cell biology?
And it touches something called the basement membrane or the basement lamina, which is part of the extracellular matrix that I'll mention again later.
Now, this epithelium, this sheet of cells can transform into single cells.
And these single cells have the property of being non-adherent.
And they don't attach very tightly to the extracellular matrix, although they are in contact with it.
And in order to go from an epithelium to a mesenchymal state there are changes in gene expression that take place, we know, at the transcriptional level.
And these result in changes in cell adhesion and changes in the organization of the cytoskeleton.
This is a reversible process, and mesenchyme can go and turn back into epithelium.
Now, the epithelial sheet is a very important part of the body, both as a building block for various structures, but also as a barrier.
So your skin is a barrier because the cells in it are very tightly joined together and they form an impermeable barrier.
But that's true of essentially every organ you have.
Every organ you have is surrounded by an epithelium, or one or more epithelia, and these surface barriers so that stuff in the organ doesn't get out, stuff outside the organ doesn't get in.
And if there is a lesion, a break in this epithelium it is a big deal.
And that is wound.
That's what a wound is.
It's a break in the epithelium.
And there is a very rapid and profound system of wound healing to repair these epithelial sheets.
Now, during development there are many transitions from epithelium to mesenchyme.
So the term that you really need to know is abbreviated EMT.
Not to be confused with other EMTs.
This refers to the epithelial mesenchymal transition, which I did not write on your handout so I will write it here.
The epithelial mesenchymal transition.
And it's always said that way, even if it's actually a mesenchymal epithelial transition.
One of the most profound examples of epithelial mesenchymal transitions is during the formation of the nervous system.
Your central nervous system, as we'll discuss in a few lectures, forms from a tube that rolls up during development.
And this tube is an epithelial sheet.
As the tube is rolling up, a group of cells, shown here in yellow, moves away, breaks away from the tube and undergoes an epithelial to mesenchymal transition.
This group of cells is called the neural crest cell population, and these neural crest cells then migrate away from the neural tube and go and set up the entire peripheral nervous system.
Not eripheral nerves, peripheral nerves.
They also make all the pigment cells in the body and the adrenal medulla.
This epithelial mesenchymal transition is not only crucial during development.
It's also believed now to be crucial for metastasis of tumors during progression of cancer.
And Professor Jacks will address that later in the course.
This is a movie demonstrating the migration of the neural crest cells out from the neural tube which is in the middle here, this white tube.
And these little dots migrating out are shown over a period of about 12 hours.
The single cells migrating out from the neural tube as they have undergone that epithelial mesenchymal transition.
All right.
So I wasn't wrong in facetiously saying what's in your toolkit is cells.
That's what's in your toolkit.
But clearly the cells are slightly different from one another or profoundly different from one another in their disposition.
And we'll talk about what mesenchyme and what epithelial can do in a moment.
The other thing that's different or the other thing that's important that makes cells actually very good for building is that they are plastic.
So let's go through a few things that they have going for them.
I want to talk about adhesion.
I'm going to mention junctions, I'm going to mention cell sorting, and I'm going to mention the extracellular matrix.
And some of this stuff you've had before so I'm going to go through it quickly.
This is a diagram that's on your handout, it's from your book, to indicate that there are many ways epithelial sheets are stuck together that involve very tight apposition of the cell membranes, or in the case of tight junctions or slightly less tight apposition in the case of these things called desmosomes.
You can go back and remind yourselves.
We mentioned these previously.
The most important thing is you understand that cells are joined together.
In this micrograph, to demonstrate how tightly cells are joined together, this is a sheet of cells where the nuclei are stained green and the red is staining for a particular specific protein that is found in a kind of junction called a tight junction.
And these tight junctions outline the cells.
In other words, the cells are glued together very tightly.
Here's another one.
Cell sorting.
There is another kind of cell adhesion interaction that's very important for cell adhesion.
I've indicated here something called cadherins.
Cadherins are interesting.
They're calcium-dependent adhesion molecules.
And if you have sharp eyes you'll see up here something that's got a beta and then there's a word here catenin.
Remember beta-catenin from dorsal-ventral axis formation?
This is the same beta-catenin.
Not only is it a transcription factor, it is also involved in cell adhesion.
There's an interesting complication of biology for you, but I don't want to dwell on that.
Cadherins are essential for sticking cells together.
These are pictures of frog embryos.
This is a normal embryo that's been cut open and the cells remain a tight mass.
However, if you take an embryo and you inject it with inhibitors of cadherin function the cells become completely loose from one another and you can actually see the outlines of the cells because they are no longer stuck together.
Now, this is fascinating and very important for the animal because it turns out there are lots of different adhesion molecules.
And different cells types sort out according to the adhesion molecules they are expressing on their cell surfaces.
So this is a rendition of a fantastic old experiment that was done in the 1950s to demonstrate how cells sort out.
What was done, these are two frog embryos, and a piece of the future skin or epidermis was removed from one, and then a piece of the future neural plate, the nervous system was removed from another.
And these had different colors so you could tell which cells were which.
Now, if you take those cells and you put them in a medium that does not contain calcium all the cadherins and other adhesion molecules cannot work.
And these cells fall into a pile of single cells.
And you can take your pipette and mix them up in your little dish, and you get this salt and pepper arrangement of the two kinds of cells.
And then you can add a little bit of calcium back to the medium, and the cells will form this big ball of cells.
And it's salt and pepper again.
The two neural plate cells and the epidermal cells are mixed up.
But if you go away and have dinner or have a good night's sleep and come back the next day and look at your ball of cells, you see that amazingly the cells have sorted themselves out.
The epidermal cells have gone back together and the neural cells have gone back together with one another.
And you can do this with cells from almost any organ.
You can mix cells from two organs particularly when they're embryonic organs.
You can mix them together, but also in the adults to some extent.
And these cells from different organs will sort out.
And they sort out because of specific adhesive molecules that they have.
And the term that one uses for this is homotypic binding where cells will interact with one another through membrane-bound receptors.
Again, I'm assuming you remember this is a lipid bilayer with a protein sticking through.
Two receptors, proteins sticking through the lipid bilayer interacting with one another in calcium-dependent or independent ways.
And there may be more than one receptor that mediates cell-specific interactions.
But these interactions keep different cells separate from one another and facilitate development and building structure.
OK.
Here's another one.
The extracellular matrix.
What is the extracellular matrix?
We mentioned this at the beginning of the course.
Professor Jacks threw it at you in cell biology.
The extracellular matrix refers to the stuff on which the cells sit, so cells in your body are not sitting on nothing.
An epithelium is not just floating free or a tube is not floating free in liquid.
It is actually surrounded by a bunch of proteins and carbohydrates that are secreted by other cells and form the extracellular matrix.
Or, in the case of epithelia, the basement membrane.
OK?
That's what the extracellular matrix is called.
It's highly organized in the case of the epithelium.
And it consists of proteins and various things called proteoglycans which are sugars bound to proteins.
Now, one of the proteins in the extracellular matrix is collagen.
And collagen is the most abundant protein in the Animal Kingdom.
OK?
It comprises a very high percentage of your body mass.
And if collagen, if there are mutations in collagen many things can go wrong.
For example, there is a disorder called osteogenesis imperfecta where your bones don't form properly.
That's a mutation in one of the collagen genes.
There are a lot of collagen genes, and the protein mass of collagen is enormous.
Now, the cells are sitting on the extracellular matrix.
But there is also a fantastic, and then that's good, that's helpful for the cells in a support sense.
It gives them some support.
But the extracellular matrix does a lot more than that.
It actually communicates to the cells.
And it does so by means of adapter proteins.
You have this as a handout, if you didn't realize.
This is one of your handout diagrams.
OK?
These adaptor proteins have the property of being transmembrane or at least integral membrane proteins that are attached to the cells.
But they also attach to proteins in the extracellular matrix.
And this allows them to sense what the extracellular matrix contains and to transmit that information to the cells.
Because the thing, of course, about living organisms and the 3-dimensional structures in them is that the process of both building the structure and maintaining the structure is a dynamic one.
It's not equivalent to building Building 10 or Stata.
It's not equivalent to putting components together and getting a structure.
You get the structure, and the structure is maintained because the structure senses how it's doing, whether it's intact or not.
If one of the walls in this building fell down, it would fall down until someone repaired it.
If one of the tubes in your body gets a hole in it, your body will sense it and try to repair it.
That same thing is true when the epithelia, and I'll tell you in a moment, when mesenchymal cells are actually doing their building process, they are sensing what is around them.
And they do it through these adapter proteins.
The adaptor proteins are receptors, by definition, they're binding to something in the ECM.
And a large number of them comprise of class of proteins called integrins, as an example of a name.
All right.
So let's move on here and let's mention -- -- the cytoskeleton as being required for shape and movement.
We've mentioned the cytoskeleton before.
I'll talk about it more in a moment.
This is a diagram from your book that you had before.
The cell is not a floppy bag of liquid.
It contains many filaments that keep it rigid, that allow it to have particular shapes and that allow it to change its shape.
We're going to be talking most about microfilaments which comprise polymers of actin, the protein actin.
There are also intermediate filaments and microtubules which we mentioned last lecture when we talked about the sperm flagellum.
This is a micrograph of tube stained, of cells stained for the microtubule network.
The nucleus is yellow.
Stained for one of the major proteins in the microtubules, which is tubulin.
And you can see this very extensive meshwork throughout these cells.
And the same is true for actin.
The same is true for intermediate filaments.
So cells are very well supported by these filaments.
OK. And, finally, let me mention cell division and cell death as being part of the toolkit that allows one to build structure.
And I'll show you one experiment which involves the inhibition of cell death in the embryo.
This is a mouse, normal mouse embryo, and this is a mouse embryo in which a protein was removed.
This protein is called caspase-9.
And caspases are involved in killing cells during development, as Professor Jacks mentioned to you.
In this mutant animal, one of the things you should notice is that the brain is hugely overgrown.
And this is an indication of the amount of cell death that has to happen during normal brain development.
OK?
And the flip side I'm not going to dwell on.
You need to get not only the normal amount of cell proliferation, but it needs to be in the right place.
All right.
So let's talk about the behavior of cells and how this works together to give 3-dimensional structures.
And I want to talk very briefly about, actually, not so briefly, about the behavior of single cells -- -- and the property that is most important to them, which is the ability to move.
So the point about an epithelial mesenchymal transition is that what comes out of it are single cells.
And this is important in metastasis and in normal development.
The difference between this epithelium and these single cells is that the single cells are free to move because they are not attached to a sheet.
And that is how cells get from one place to another in the body.
OK?
So let's dwell on this in more detail.
So how do cells know where they're going?
Well, this is a very interesting question.
They know where they're going because they're told to go somewhere.
So I told you the neural crest migrating out of the neural tube.
This is a group of cells from an organism called dictyostelium.
And these are single cells.
They've been fluorescently labeled.
And up here in the corner someone has put invisibly a drop of the second messenger cyclic AMP.
And you can see these cells that had been milling around randomly start to realize this and move as a consorted group towards the source of cyclic AMP.
OK?
And this is one of the ways a dictyostelium organizes itself.
The cells chemotax, they follow a particular chemical stimulus.
How do they move?
How do cells move?
Well, they move because the reorganize their cytoskeleton, particularly their actin cytoskeleton in a concerted way.
So I've drawn for you a kind of animated cartoon.
You have the whole thing in front of you.
And I'll animate it and see how it goes.
We start off here with our green cell.
This is a mesenchymal cell.
And it is sitting on some kind of extracellular matrix, which I've shown as kind of organized, but actually it may not be.
But that doesn't matter.
It is loosely attached to this extracellular matrix.
It's got these receptors sticking out of its membrane that can sense extracellular matrix, but only at the rear of the cell is it actually attached by these red triangles to the ECM.
OK?
That's what my red triangles indicate.
Now, the cell suddenly as it's sitting there or moving randomly along comes across these black circles -- -- which are some kind of molecule that can interact with these receptors and tell it that it ought to go in a particular direction.
So how does it do this?
Well, the first thing it does it to elicit this interaction with a substrate or with these black molecules in the substrate.
This is a receptor ligand interaction.
It's exactly the same kind of interaction that you talked about in cell biology, that we talked about in the formation, earlier formation lectures, cell receptor ligand interaction.
And it does the same thing that other receptor ligand interactions do.
It activates some kind of signal transduction process, but this signal transduction process has the property of telling the actin filaments in this local region of the cell that they should polymerize.
OK?
So this is cell signaling in a very local region of the cell.
And I'll show you the movie in a moment.
You'll see it's extraordinary.
So here are the actin filaments forming in this region of the cell.
And I've called this the front of the cell.
OK?
It's an arbitrary term.
But the front of the cell is arbitrary but it's going to refer to the direction that the cell is moving.
So there's actin polymerization of the front.
And the cell also sends out some protrusions called filopodia or lamellipodia.
And it sends out these protrusions to try to sense where it should move next.
Now, once it's done that, once it's made these polymerized actin filaments it contracts.
However, it doesn't go anywhere because it's still attached at the rear.
So it's kind of like pushing down on your accelerator when your hand break is still engaged.
OK?
You don't really go very far.
So the next thing it has to do is to lose the adhesion at the rear.
I couldn't get this to work but it will work in a moment.
OK.  And it also depolymerizes the actin at the rear.
So there.
There we go.
There's loss of the adhesion and there's loss of the actin at the rear.
And now it's in a position, it's disengaged the hand break and it can move forward, so it goes forward.
OK.
This is my cartoon.
This is the real thing.
This is a cell where actin has been labeled fluorescently.
And I want you to watch two things.
It's on a repeat loop so we'll watch it a number of times.
At the part of the cell that is sticking out, you should be able to see these bright cables or these bright lines.
These are cables of polymerized actin.
OK?
And the cell is sticking out these protrusions and polymerizing actin.
And it's moving down in this plane of the board, or the screen as it's making these polymers.
OK?
So this is the front of the cell.
And then if you look at the back of the cell, let's wait for the loop again to start, and you will see, here's the back of the cell with initially polymerized actin.
And if you watch individual lines you will see them go away as the actin is depolymerized and the cell loses its adhesiveness at the back.
OK?
So this process involves selective adhesion, loss of adhesion mediated by actin polymerization through the cytoskeleton.
There's a signal transduction pathway involved.
It involves things called GTPases that you've talked about previously.
And it's really an extraordinary process.
All right.
But let's move on and let's talk about cell sheets.
So cell migration is crucial to get from one place to another, but you cannot really build much with single cells because they're single cells.
They don't really give you any kind of integral structure, any kind of cohesive structure.
And so you have to turn these single cells back into sheets if you're going to use them then for building.
So what do cell sheets do?
Ah, OK.
So let me go through a couple of things that cell sheets do.
Cell sheets can get longer.
They can change the number of layers that they have.
They can change the number of length, the number of layers that they have, and they can bend.
Here are some cartoons that I drew for you.
Here's a sheet of cells.
And you can lengthen it in two ways.
You can stretch it out so that the surface area to volume ratio changes.
And you get a longer, thinner sheet of cells.
You can also take this initially bilayered sheet of cells and fit the cells in between one another by a process called intercalation, or the term that's usually used is convergent extension.
And that lengthens a sheet of cells.
And the reason that a small embryo becomes longer is because of this process of intercalation where the embryonic cells interdigitate with one another and make the whole embryo stretch out.
You can also change the number of cell layers to get a more complex tissue, a more complex structure.
You can break two cell layers into two individual layers.
That's called delamination.
Or you can turn two cell layers into one cell layer by another process of intercalation so that you get a single sheet from what was initially a two cell thick sheet.
Again, this is in front of you and I don't want you to dwell on it.
Let's talk about bending cell sheets.
This is really interesting because one of the things that comes out of bending cell sheets are tubes.
Now, tubes are pervasive.
There is not an organ in your body that does not have some kind of tubular structure.
And if you think about it, if I were to give you a pile of cells and say, here, build me a tube.
You could probably come up with several ways that you build a tube.
And, in fact, the organism has come up with several ways.
One of the ways is to roll an epithelial sheet into a tube.
So here is the sheet.
And you roll it up to form a tube.
This is how your neural tube is made.
And also part of this is that the cells change shape as they are rolling up.
So if you have a sheet of cells that is either cuboidal or columnar, to get that sheet of cells to bend you have to make wedge-shaped cells out of them.
You have to give them a constriction, a so-called apical constriction.
So you go from a sheet of columnar cells to a bench sheet of cells because you have apically constricted those particular cells.
And you can do that through an actin process or through our old friend beta-catenin here acting as a cell adhesion molecule.
All right.
So here's the process of rolling up the neural tube in frogs.
It's rolling up by apical constriction and by the epithelial sheets bending.
OK?
You've seen this movie previously, and it is on your website so you can look at it again.
Here's another way.
You can balloon out an epithelium.
You can imagine an epithelium, and you make a little, you blow it and you get a little balloon.
And then you blow it some more, you pull it some more and you can turn an indentation or a very small vesicle into a long tube.
OK?
And I'll show you.
This is not the best diagram, I've got to work on this one, but the idea is that you pull this.
Dr. Gardel is agreeing with me, I've got to work on this diagram.
OK.
But you pull out this sheet of cells into a long tube.
I'll talk more about this when we talk about lung formation in a moment.
Here's another way.
You can take your single mesenchymal cells that have migrated to wherever they're going and you can get them to condense into an epithelial sheet.
And this epithelial sheet can then form a tube.
This is exactly what happens in formation of the blood vessels.
Your blood vessels have got lots of layers of epithelia.
The first one to form is the endothelium.
It's the inner most layer of the blood vessel.
And it forms by condensation of mesenchymal cells into an epithelial sheet.
OK.
Excellent.
So I want to end by giving you a specific example and talking about the genes involved in a particular aspect of 3-dimensional structure generation.
And the example I've chosen is formation of the lung and the tubules in the lung.
So your lungs are made up of multiple tubules that have got little sacks on the end of them, alveoli that are the places where oxygen is exchanged between the air you breath in and between the blood vessels that feed into or that surround all the cells of the lung.
Formation of the lung tubules is very interesting because mathematically, if you look at it, you can really define it by a Mandelbrot set where you have some kind of reiterative branching process.
And if you look at the lung tubules and count them and look at the number of branching, you can figure out that you have to go through 20 branching events to get to the set of lung tubules that is in an adult human lung.
OK?
If you look at the structure, the structures during the process start with the trachea and this thing called the bronchial bud -- -- which divides into two to give two primary bronchi.
Then each of those divide more, branch more and so on.
So these are multi-cellular, these are epithelial sheets that are branching reiteratively during lung formation.
Here's a movie of the process.
You can get this to take place in a Petri dish with the appropriate signals added.
And so here is the lung tubule branching reiteratively over a period of about a day or two in tissue culture.
OK?
So this is a 3-dimensional tissue engineering, or at least you can get this aspect of 3-dimensional tissue structure to form in tissue culture.
All right.
So what genes control this?
As you know, as I've told you over and over and over, everything is controlled by genes and the interaction of genes.
And what do we know about lung formation?
Well, we know most about lung formation from insects.
In insects the evolutionary relic or the evolutionary precursor or the evolutionary equivalent is the system of trachea.
These are a system of tubes that branch out from the spiracles, which are holes that lead to the outside.
And then tubes come off these spiracles and branch in the insect and actually carry air directly to the tissues.
There's no circulatory system equivalent as there is in our cells.
But the way these tubules branch and form is very equivalent to ourselves.
And we know that this involves a receptor signaling system that you are familiar with, which is a receptor tyrosine kinase signaling system.
And the particular one I'll tell you about is the fibroblast growth factor signaling system.
You remember that growth factors bind to receptors, and these growth factors can be in the extracellular matrix.
They usually are.
That's one of the places where growth factors are.
They bind to a receptor.
And then there is a cascade of signal transduction that, in this case, involves kinases.
And eventually the final kinase moves to the nucleus where it does stuff to transcription factors and you change the transcriptional activity of a cell and you get stuff happening.
OK.
So this is a diagram of what happens in the fruit fly where we know most about the process.
The epithelium that's going to give rise to the trachea undergoes cell division, and at the same time it balloons out to form this thing called the primary tubule.
You have this diagram in front of you.
It forms the primary tubule.
After a bit, this primary tubules branches to give secondary tubules.
And later on it branches again to give these tertiary tubules.
There are a number of mutants in drosophila that affect each of these processes, the primary tubule formation secondary or tertiary tubule formation.
And these are members of the fibroblast growth factor signaling system.
In the primary tubule the FGF receptor is expressed, and the cells receive FGF which tells them to divide and to divide in a particular orientation to give you this primary tubule as it grows out.
OK?
And there's a ligand sitting around in the extracellular matrix that tells the cells to do this.
Now, what's really cool is that at the point where the secondary tubule, I'm going to take 20 seconds, so I'd appreciate it if you'd listen.
At the tip of the tubule an FGF inhibitor called Sprouty is made.
This inhibits cell division of these cells right at the tip, but the cells on either side continue to divide.
And so you get the branching, the secondary branching taking place.
And I'm going to stop there.
And those of you, don't forget, come along to Stata this afternoon if you need a review.
Differentiation.
For someone who didn't see the board.
Yeah?
Yeah.
So what a cell will become eventually.
And differentiation is the process by which it becomes that.
OK.  Good.
So these terms are essential for you to know for today's lecture.
And if they are a little murky for you then you should really make sure they become clear.
OK.
I'm going to talk to you today about stem cells.
This is the how-to module, our second how-to module.
We have two lectures, stem cells and cloning.
And what I want to do today is to go beyond the media, beyond the hype and tell you about stem cells, what we really know and why there is such a fuss about them.
And to put it in perspective of where you are in the course, in our game board of life we're moving right along past foundations up through formation and now into our second how-to module.
So let's start with a question.
And you have a handout.
If you do not have a handout, Shamsah, could you be mail person?
And on the other side it looks like we need another mail person.
You will realize that I typically do give you a handout of the most important slides so that you don't have to grapple with things and that you don't, in fact, have to print out the PowerPoint handout before you come.
So you should always just check.
OK.
So what is a stem cell?
Well, a stem cell is something that has two important capacities.
Firstly, it can self-renew.
That's a very popular term.
It can make more of itself.
And, secondly, it can give rise to a number of different, one or more different cell fates.
So it can give rise -- -- to one or more differentiated cell types.
Schematically, I've represented it for you in this way.
So here is something that's called an uncommitted stem cell.
And you should be familiar with the term uncommitted, naive and so on.
And this uncommitted stem cell can undergo cell division to make more of itself.
And it can also go on to become something that's termed in the field a "progenitor cell".
It's just a term.
It means a cell that is going to give rise to something else.
And I've depicted it here as a determine progenitor.
It knows what it's going to become.
And this term uncommitted stem cell, the uncommitted part is a little misleading.
I'll talk about that more in a moment.
This determined progenitor then goes on to give rise to a whole bunch of one or more, actually, differentiated cell types.
And the number of cell types that a stem cell can go on to give rise to is a measurement of its potency.
That is the definition of the term potency.
So this is the notion of the stem cell concept.
You have this.
It's the first figure on your handout.
And you'll find this in many newspaper articles and books.
So what's so special about this and why are there newspaper articles and newspaper articles and newspaper articles and television programs and covers of Time Magazine about stem cells?
What's the hype?
Well, the deal is that it is believed by some that stem cells are going to be some kind of universal repair kit.
That somehow because stem cells can give rise often to differentiated cell types and make more of themselves that they are in a position to repair parts of the body when the body needs repair.
You have a heart attack, you throw in some stem cells, repair the heart, you have a brain lesion, you throw in some stem cells and you repair the damage.
And if you go through the newspapers, as I did very, very briefly, it's very easy to find many, many different headlines.
One of the big ones last week, I'll talk about this at the end, is that the Massachusetts House passed a law expanding the use of stem cells in research.
Hopkins is beginning human trials with donor adult stem cells to repair muscle damage from heart attack.
This is an enormous amount of interest in these cells because of this notion of universal repair kit-ness.
There are also advertisements.
The Stem Cell Bank and the Cord Blood Registry would love you to pay them to save your child's umbilical cord stem cells, or the blood cells from a newborn's umbilical cord in the possibility that you'll use these later on to save the life of this child or somebody else.
And the other reason that stem cells have got so much hype is that, as we'll discuss towards the end of the lecture, they involve the use, or are likely to involve the use of human embryos.
And that's a real fire point.
But let's step back and let's talk about stem cells and biologically what are these things and what do you need them for?
And the thing we have to consider is the question of organ maintenance.
Now, I've told you several times, oh, you see, it popped up.
There you go.
We have to give these frogs a moment to pop up.
OK.  That must have been somebody's frog that popped up.
I've told you several times that our bodies consist of about ten to the twelfth cells.
That's a lot of cells.
But it's actually much worse than that because it is not the same ten to the twelfth cells that you started with, once you got there, that you keep throughout your life.
The cells in your body are constantly dying and are constantly being replenished.
And, in fact, if you look at your ten to the twelfth cells, over the course of your lifetime you will make many times ten to the twelfth cells.
You're not going to replace every cell in your body because some of them life forever, but for many populations of cells in your body you will replace those organs completely.
OK?
So we're talking about making lots and lots of cells.
And the issue really is one of organ and tissue maintenance.
And if one considers how organs and tissues are maintained, they fall into three categories loosely.
Those that have a high turnover rate where cells do not live for very long and need to be replenished rather frequently to maintain the size and function of the organ.
Those organs that have a low turnover rate and those that are said to be static.
Organs with a high turnover rate include, as I'll tell you a moment, the blood, some of your hair.
Organs with a low turnover rate include things like your liver and pancreases.
And organs that were thought to be static, but in fact that may not be true, was the nervous system.
And now we know that that's not true.
So let's talk about how we get to these terms.
High turnover.
Low turnover.
Let me just write it out.
Turnover.
And static natures of organ maintenance.
And the way we do this, or the way this has been done is by an assay that's called a pulse-chase assay.
And a pulse-chase assay, and this is the second slide on your handout, goes like this.
In a cell population one can add a nucleotide analog bromodeoxyuridine.
It's uridine but it's incorporated into DNA because it's got that deoxy there.
And bromodeoxyuridine is a nucleotide that is incorporated and can then be detected using various stains.
So you can tell which cells have incorporated BrdU by staining them appropriately.
So you can tell which cells are dividing, which cells have gone through S phase by whether they are labeled or not.
And if you give a cell population a short pulse, if you give BrdU for just a short time, this is called a pulse, it gets incorporated into the DNA, and then the BrdU is gone.
And so you label your cell population.
And then you follow the cells over a long time without any added label.
And so the only cells you look at are the ones that were labeled during this pulse period.
And if you do that, you see that your initially labeled cell population lives for a while, and then the cells start to disappear because they die.
And you can monitor the half-life a population by counting how long cells live for.
OK?
So this is a pulse-chase analysis.
And this will give you the cell turnover time of a particular organ.
And this is very useful in trying to figure out whether cells have a high turnover rate or a low turnover rate.
Now, you can do something else with this.
You can look at your labeled cell population, and you can actually look not just at the numbers of cells but you can look at what those cells become.
And sometimes your initially labeled cell population will be the final form, the differentiated state of the cells, but sometimes it won't.
And then it's very informative to follow that labeled cell population and see what they do.
This is your third handout, your third slide on your handout.
And if you follow them you might see that this labeled cell population, during the chase, differentiates into something.
And that's very interesting.
And then later on those cells will die and you'll be able to measure a half-life as previously, but this is very interesting because it tells you that this labeled cell population was not the same as the final fate of the cells.
And it implies that there is some kind of precursor cell, some kind of progenitor in your labeled cell population that's giving rise to the differentiated cells.
And it implies that these initially labeled cells either are stem cells or are progenitors derived from stem cells that are responsible for repopulating a tissue as the tissue or organ needs to be maintained.
So these are two assays that are very useful and are used to, have been used to define high turnover organs.
So what are some of these organs that have, or tissues that have high turnover?
In fact, they've been used, these assays have been used to define many tissues, those with high and those with low turnover.
But it's the high turnover ones I want to talk about.
The red blood cells in your body have a turnover time, half-time of about 120 days.
And if you do the calculation, because you have a lot of red blood cells, you're making more than a billion cells a day.
OK?
As you are sitting here, you are probably, during the course of this lecture, making several million new red blood cells.
Your intestine, some of the cells in your intestine that are responsible for food absorption turnover every three to five days.
Your skin cells, the epidermis of your skin turns over with a half-life of about 14 days.
And your hair, depending on where it comes from, turns over with a half-life of about 14 days to about four years.
And the sense in many of these organs, and I'll go through this a bit more in a moment, is that this high turnover and the replenishment of these cells as they disappear is driven by a stem cell population.
And I'm going to abbreviate stem cells SC.
And I'm going to put forth the hypothesis that stem cells replenish these organs.
Now, in some cases you can actually look at the organ and you can do your pulse-chase, just like I told you, and you can see that this must be so.
And an example that we've talked about previously, not in this context but previously is the testes, the spermatagonia are the cells sitting right at the edge of the seminiferous tubule, which we'd label with BrdU.
And as you do your chase you would see those cells as labeled cells moving towards the lumen and differentiating into the stem cells.
OK?
And so you can actually see those cells giving rise to the differentiated cells.
Another tissue where this is very beautiful is in the epidermis of the skin.
So the epidermis of the skin comprises multiple layers stacked up on top of each other.
And as you get towards the outside of the epidermis, the cells in the outer layers are dead and they form the layer that we feel as skin.
But, in fact, there are a lot of cells that are living underneath the outer layer.
In the very deepest layers of the epidermis reside a dividing population of cells that are, that include the stem cells, and I'll show you an assay for some of this in a little bit.
These dividing cells, if you follow them, give rise to the cells in the layers above.
So they either are the stem cells or they are progenitor cells that are replenishing the epidermis.
Now, in the blood, I told you that you can do a pulse-chase assay.
And you can very accurately measure the half-life of red blood cells.
And it's clear that red blood cells must come from some precursor.
In ourselves, in most vertebrates, in many vertebrates red blood cells do not have nuclei so they cannot divide themselves.
And so there must be some kind of progenitor.
But the progenitors of red blood cells and, in fact, many other cells in the blood lie deep within the bone marrow, the cavity of the bones.
And so you really cannot observe them well to come to the conclusions that I've just given you about the testes and the epidermis of the skin.
So you have to do some kind of other assay to figure out where these cells are coming from.
And over the years a huge body of data has given rise to this notion.
In the bone marrow of the long bones, and some of the shorter bones, too, there is some kind of pluripotential or pluripotent hematopoietic stem cell, that I've abbreviated HSC.
And this HSC gives rise, through a series of progenitors which then go on to differentiate, into all of the cell types that are found in the blood.
The red blood cells.
The so-called white blood cells.
The immune cells.
A huge spectrum of differentiated cell types.
And this hematopoietic stem cell has been very, very useful in helping people who have got blood associated illnesses in that it can, after removal of diseased blood cells, it can repopulate the tissue and help people get better.
And it's done so, and I'll go through how this is done both in an assay sense and point out that this is exactly what's done in humans, it's done so in a bone marrow transplant.
So in assays, let's just write down a couple of things here.
I've told you one assay already for stem cells.
OK?
So assays for stem cells ask how do you know if you've got a stem cell?
Well, you can do that.
You can ask this using the pulse-chase by direct observation with your pulse-chase.
You can also do it by a repopulation assay.
And the notion in a repopulation assay is that you try to restore a particular cell type by putting in what you think is the precursor of that cell type.
Now, in order to do these assays you usually have to get rid of the normal cells that were there so that you can assay your repopulation.
OK?
Otherwise, the normal cells can out-compete the cells you're putting in.
OK?
But that's an aside.
So what is a bone marrow transplant?
The notion here is that one takes an animal, and this is done in humans, and you irradiate the animal so that you destroy all its bone marrow.
This will eventually kill the animal, OK, because you cannot live without a hematopoietic stem cell and without all the cells that derive from that.
And so, although this irradiated animal would die, you can rescue it by injecting normal bone marrow.
You inject it into a mouse in the tail vein, into people it goes into a central line that goes into your blood system.
And then those bone marrow cells, it's extraordinary, know where to go.
They home to the bone marrow and they repopulate it and they make again the hematopoietic stem cell and all the lineages that come from that.
Or they are the hematopoietic stem cell, they maintain themselves and they give rise to all the other lineages.
One of the criteria that I told you about stem cells is that they need to be self-renewing.
You can ask whether or not these stem cells are self-renewing in an assay like this, where you take a mouse that you've rescued by bone marrow transplant, and you can ask whether it can serve as a source of marrow of hematopoietic stem cells that will rescue another irradiated mouse.
In other words, you could have rescued that first mouse just by giving it a whole spectrum of almost differentiated red blood cells, white blood cells, immune cells and so on.
OK?
In order to test whether you actually rescued it by giving it a stem cell, you can ask whether or not you can take this rescued mouse and use it to rescue another mouse.
And, in fact, you can.
And this is consistent with, not unequivocal, but it's consistent with a sense that you have generated more stem cells in the rescued mouse that can go on to rescue another mouse.
Another set of experiments that has been really pivotal in understanding stem cells is to try to purify the stem cells and to assay what are these pure stem cells.
And part of this is asking how many cells do you need to rescue in a repopulation assay?
OK.  And so let me make a note here.
That when it comes to stem cell isolation the overriding conclusion from many assays done by many labs is that stem cells are rare and they are hard to find.
In this experiment, which is number five on your handout, you can assay for how many stem cells you need to rescue a mouse by removing bone marrow from one mouse and staining that bone marrow, if you can, for a stem cell marker.
Now, what is a stem cell marker?
OK?
If I have a stem cell marker I would be able to go and pull out the stem cells.
Well, there are no markers.
A marker could be a protein or an RNA that is expressed in a particular group of cells that you suspect are the stem cells.
OK?
This is very much an experimental-driven field.
There are markers that always appear, proteins that are always there, antigens on the surface stained with antibodies that are always there where you have a population of cells that is able to rescue a mouse.
And so you can use these to hypothesize that this stem cell marker is, in fact, marking the stem cells.
And you can go and purify cells that are expressing a particular marker.
And the way you do this is by using a really fantastic machine called the Fluorescence Activated Cell Sorter (FACS).
I'll go through it very briefly in a moment.
But what it is able to do is to purify cells that have got particular fluorescences.
And they have these particular fluorescence spectra because of the way you have stained them for particular stem cell markers.
And what you get out of your FACS machine is a population that is much more enriched for what I've called stem cells, but they're actually cells that are expressing this punitive stem cell marker.
And then you can take these cells and say, well, have I really enriched for the stem cell, by asking how many cells you need to introduce into an irradiated mouse to rescue that mouse.
And you can do a dilution assay where you put in one cell, ten cells, a hundred cells.
Obviously, when you're down in the one cell range, you do your dilution assay, you get into a Poisson distribution, and you may not have exactly one cell.
But you can do this statistically and you can figure out how many cells from your population it takes to rescue a mouse.
OK?
And the data to date tells us that one cell of the correctly purified population is sufficient to rescue an irradiated mouse.
You have to do some tricks.
You have to add some extra cells to this one cell.
You cannot just put one cell into the mouse and rescue.
You have to mix this one cell with some other cells that help it do its work.
OK?
But this is really an extraordinary assay that tells us we're very close to purifying a hematopoietic stem cell.
What is a FACS machine?
Very briefly.
You can look at this later.
Cells that are labeled with fluorescent antibodies are broken up into a stream of tiny droplets each of which will contain one cell.
And the cells then pass in a stream through a laser which activates the fluorescence.
There's a fluorescence detector that you can program to respond to a particular wavelength.
And what happens is that when the detector finds that it has responded to a cell passing through it because of the fluorescence emitted, a charge is given to that cell, and that cell with a particular fluorescence moves down the stream through some deflecting plates where it will be deflected, according to its charge, into a collecting bin.
And that way you can really get some very highly purified cell populations.
You can purify about 300,000 cells an hour using the FACS machine.
And we have a lot of them here at MIT and use them.
Many laboratories use them for various things.
A really cool machine.
OK.
But we need to move on.
So we talked about some assays for stem cells.
Talked about the potency of hematopoietic stem cells.
I want to tell you something now about the control of stem cells and how a stem cell decides what to do.
So if you think about organ maintenance, there is an implicit understanding that the organ knows how much it needs to be maintained.
It needs to know how many cells are being lost, it needs to know when to make more cells, and it also needs to know if there has been some catastrophe.
If there's been a liver, some kind of a resection of the liver, or if there has been skin wounding, OK, the organ needs to be able to respond.
And the stem cells in an organ need to be able to respond to this catastrophe.
And this is where this term niche comes in.
Niche is one of these fields in the term, terms in the field.
It's like the term progenitor.
And niche is nothing that you haven't heard about before.
What it refers to are the cells, as written up there, that surround the stem cell.
And what I'm going to tell you is that the niche cells, the surrounding cells control, by induction, stem cell fate.
And I'm also going to point out that there can be some input of the environment.
So this is a cartoon that I drew for you.
It's number six on your handout.
It's six across on your handout.
And I have here some stem cells that I've called quiescent.
They are not dividing.
Or they may be dividing just a little bit just to maintain themselves but not to give rise to any progenitors that will go on to differentiate.
And these quiescent cells are maintained in this quiescent state, we believe, because they are surrounded by a group of cells that is maintaining them quiescent.
What happens then is that these quiescent cells sometimes get a stimulus, some kind of environmental input, be it wounding, be it some kind of hormonal input from another part of the body that influences the surrounding cells and changes the surrounding cells such that they now change their patterns of gene expression and start secreting stuff, most likely, that changes the activity of the stem cells that they are surrounding.
I've shown you arrows there indicating induction from the stem cells, from the surrounding cells to the stem cells.
And what this inductive process does, just like in the early embryo is to activate the stem cells to divide and to form progenitor cells which will go on to do their thing and differentiate into various fates.
And also, of course, the stem cells are renewed.
So this notion of the surrounding cells telling the stem cells what to do is kind of new in the stem cell field.
If one is a developmental biologist, it's one of these, OK, kind of observations because we know that cells signal to one another.
And it's not surprising that stem cells are told what to do by the cell surrounding them.
But there has been not very much information on this.
And I want to give you one system where it's been very  beautifully looked at and where the contribution of the surrounding cells has been looked at.
And this system is the hair follicle.
So your hairs come from a sheath of cells that starts off in the epidermis and moves deep down, extends deep down into the deeper layers of the skin called the dermis.
And there is a shaft of cells of which these matrix cells down here are important because these matrix cells divide and they give rise to cells that will form the center of the shaft and secrete, or not secrete but synthesis proteins called keratins.
The keratins are this stuff.
And eventually the cell gets so full of keratin that it dies.
And at the same time more cells are coming in the bottom here and this whole shaft, this whole hair shaft is pushed out of the hair follicle.
OK?
And so that is why your hair grows.
It has been pushed out from the bottom.
And all the stuff that we are so fond of is dead cells.
It's actually protein.
It's a very interesting polymer of various keratins.
Now, what's very interesting here is that over the years researchers have shown that the hair, well, let me back track for a moment.
You know your hair falls out, right?
You brush your hair and your hair falls out.
OK?
Ergo, it must be replenished somehow.
And over the years it has been shown that these bulge cells, or cells in a little bulge, which was a rather insignificant little bulge but it's there, sitting on the side of the hair shaft is actually the source of the stem cells for the hair.
And another region, I'm going to tell you about in a moment, is down here.
It's labeled DP.
Bear it in mind as I tell you something and then we'll come back to it.
So let me first show you some data that tells us that the bulge cells are the cells that contain or that include the stem cells for making hair.
One can do an experiment where one takes these bulge cells and does essentially a repopulation assay, just as in the case of the bone marrow transplant.
And one can do this into a strain of mice called nude.
So nude mice have got multiple issues.
But one of their issues is that they have no fur.
OK?
And one can take bulge cells from a normal mouse and transplant them into a nude mouse.
Here's the control transplant, and you can see there's no fur.
And here is a transplant of bulge cells from a normal mouse.
And here you see this tuft of fur growing out on the back of the nude mouse.
And, in fact, some very cool assays have been done where these transplanted cells are labeled green.
And you can look at these tufts of fur and see them growing fluorescently green.
So you know that the new fur came from the transplanted cells.
OK.
So what about niche?
Well, you know that your hair falls out.
But, actually, there's a lot more to it than that.
And it turns out there is a whole hair cycle of when the hair grows and when it doesn't grow.
And it goes like this.
There's a time called anagen which is a period of growth when these matrix cells down here are dividing and they are pushing out this hair and it's growing.
After a while the cells down here stop dividing and the hair follicle doesn't grow anymore, or the hair shaft doesn't grow anymore.
This is called catagen.
And it's a time that's termed regression.
It is when there is no more growth.
And if the hair is going to fall out, it might fall out then, or it might stick around and the next step might take place.
And this is a step called telogen which is a resting state.
And then after that there is a new state, a new anagen where the hair will start growing again.
And it may either be a new hair or it may be the one that's there that just grows longer.
It depends on the part of the body.
OK.  Now, what I'm going to tell you is that a region called the dermal papilla induces stem cell activation at certain times during the hair cycle.
So look at this.
At anagen, during growth, the bulge and this dermal papilla, OK, this DP, it doesn't matter what the name is but just look at the red, are far apart from each other.
At catagen they're still far apart, but look at what happens at telogen in this resting stage.
The dermal papilla and the bulge are touching one another.
And this is the phase, although it's called a resting phase, this is the phase when the hair follicle is actually getting ready for the next big spurt of cell division and differentiation that is going to make the hair longer or make a new one grow.
And then, subsequently, the dermal papilla and the bulge stay close together, but eventually they will start pulling apart again.
Well, it's been shown very elegantly that this dermal papilla signals to these bulge cells when they're touching one another and tells the bulge stem cells to activate and start making more hair or a new hair follicle.
And the signaling is done through a pathway called the wnt signaling pathway.
And I point this out because it uses a protein that should be familiar to you, which is our old friend beta-catenin that we talked about way back when as being important for dorsal-ventral patenting in the early embryo.
And I point it out to you to make the point that proteins are used over and over again.
Regulatory proteins can be used over and over again in the body.
OK.
So this is a very lovely example of what happens in a stem cell niche.
So let's move on and let's ask how plastic adult stem cells are.
OK.
So here we have some issues.
Well, the goal of the medical community is to get a stem cell that can repair every tissue.
And it may be that every adult tissue has a stem cell population in it, but these can be difficult to isolate.
It's very difficult to isolate these hair stem cells.
You can get enough to get tufts of hair on another mouse, but it's hard to get enough of them to repair someone's bald head for example.
OK?
So the Holy Grail of stem cell-ness is to try to find a cell that is a multi-potent adult progenitor.
OK?
And this has been the real goal of the adult stem cell field.
And the question that has been posed is, is there a multi-potent, a highly plastic, multi-potent also termed plastic, adult stem cell?
And if there was then in theory one could use it to repair all organs.
So here again is our stem cell concept where some kind of stem cell goes on to give rise to a set of differentiated cells.
Now, the set of differentiated cells forms what's called a lineage where all the cell types that come out of a single stem cell are related to one another because they come from a common heritage, from a common progenitor, from a common stem cell.
And you can, for different lineages, make different trees of stem cells progenitors and differentiated progeny.
And it has been hypothesized that there is some kind of interconvertibility between different stem cells in the adult, and that one stem cell for one lineage can be under the right circumstances converted to a stem cell for a different lineage.
And that maybe there's even some kind of master stem cell floating around in the adult that can give rise to all other stem cells.
Maybe there is some totipotent or pluripotent very, very highly potent adult stem cell that can give rise to all these other stem cell lineages and can, therefore, be used as this universal repair kit.
And this has been assayed.
And the prediction from this is that adult stem cells from one lineage, for example the hematopoietic lineage, can contribute to another lineage.
For example, the muscle cell lineage.
And this prediction has been tested in the following way.
It's particularly been tested with hematopoietic stem cells.
And the question has been posed can these hematopoietic stem cells contribute to other lineages?
Here's a mouse expressing green hematopoietic stem cells, or making green hematopoietic stem cells because of expression of the protein GFP under an appropriate tissue-specific promoter.
You can isolate bone marrow cells from the GFP expressing mouse.
Transplant them into an irradiated mouse.
You get rescue, as we've discussed, but that's not the point of the experiment.
The point of the experiment, and this is the second to last slide on your handout.
The point of the experiment is to ask not about the bone marrow cells in the rescued mouse, but to ask whether any of the other tissues in the mouse contain these green cells that would have arisen from hematopoietic stem cells.
OK.
So we're asking whether these hematopoietic stem cells can contribute to lineages other than the blood and the immune system.
And this has been an enormous focus of research.
And the answer initially appeared to be yes.
And this was a cause of tremendous excitement.
Here's one piece of data.
This is a picture of some cells from a mouse that was rescued with green hematopoietic stem cells.
And this is a picture of the muscle from the mice.
The blue are nuclei and the green is the GFP that would have come from these transplanted hematopoietic stem cells.
And you can see green cells also in the nervous system, nerve cells that appear to come from these hematopoietic stem cells.
Well, it turns out alas that this is an artifact of the experiment.
And, in fact, what happens is that these green hematopoietic stem cells rescue the mouse in the correct way.
But then they also fuse together with other cells.
They fuse with muscle cells and with nerve cells.
The two cells actually join together.
Their cell membranes mingle and you get a kind of hybrid cell that is a mixture of the hematopoietic cell and the regular muscle cell.
And this whole notion of the adult hematopoietic stem cell contributing to muscle, to nerve, to various other lineages has completely fallen to disfavor in most fields, in most people's opinion.
Although, there is still data, there is still much experimentation to do to really address this, both in the hematopoietic stem cell and other lineages.
But no data currently supports that adult stem cells can interconvert one to another.
And that leads us to the difficult position of asking, so where is this universal repair kit going to come from?
And this is where we move into the field of, I'm going to just leave that now, of embryonic stem cells.
So you know now that embryos have got many cells that are multipotent, pluripotent.
And indeed the zygote, as we discussed, is totipotent.
These cells can give rise to many, many different tissues because during embryogenesis that's the deal.
That's what has to happen.
And so it came to the attention of people long ago that embryonic stem cells might be a great source of stem cells that can be used as this universal repair kit.
Now, I want to tell you, I'm not going to go through this, you can go and look on the PowerPoint, you can go and look on the PDF file on your website.
I'm going to tell you about a type of cell, in the last few minutes, called the embryonic stem cell.
And what you really need to know about this is it's almost totipotent.
And the first slides, I'm not going to go through, show you that this embryonic stem cell type is almost totipotent.
Now, what about these embryonic stem cells?
They come from, in taking a very early embryo, at a stage called the inner cell mass stage -- -- when there is a little mass of cells in the developing embryo, that's going to give rise to the whole embryo.
And taking this cell, removing these inner cell mass cells and putting them into a Petri dish with various culture media, what happens is that the cells from this early embryo divide and they make various clumps.
And the clumps are descended from one particular cell.
And each of these clumps of cells is called an embryonic stem cell line.
Now, what's a cell line?
A cell line is a population of cells that can grow continuously in culture.
In other words, it self-renews.
What's a stem cell line?
A cell line, this is on your PDF file so you might want to sit and listen to this and then go and get this afterwards.
OK?
What is a stem cell line?
A stem cell line is a cell line that has the potential to go on and differentiate into various different lineages.
OK?
And so from this embryo that has been put in a Petri dish and the cells allowed to grow you get out a number of stem cell lines.
And the notion is that each of these stem cell lines, if treated with the correct factor that's called here a differentiation factor, can go on to give rise to different tissues.
Heart muscle, pancreas, cartilage and so on.
And the notion is that different ES cell lines will have different potencies.
So some will be really good at making hearts, some will be really good at making muscle and so on.
And it's not quite clear how one gets these different ES cell lines, but they exist from the mouse.
And some of them exist from the human.
And this is where we get into the other major issue of stem cell biology and the reason that there is so much legislature.
Because in order to get these human stem cell lines you have to use a human embryo.
To get human embryos you use methods of assisted reproductive technology.
I sent you some notes on this and you can read this in the book.
And the notion is that you remove eggs from a female, you make embryos in vitro, in a test-tube.
And then when the embryos are balls of cells you harvest them, you kill them and you turn them into ES line.
And the thing that is sticking people right now in the legislature and is something that is really worth thinking about is whether ethically it is OK, guys, it's a 11:54, so stay.
Stay.
OK. And the thing that is killing people, is really sticking people ethically is whether it's OK to take these human embryos and turn them into cell lines, because this does involve killing an embryo and this is an ethical issue.
And I'm going to leave this here, and we'll come back to these ethical issues when we talk about cloning.
We're going to discuss today the definition that you two gave which is making a genetically identical copy in terms of organisms.
And I want to begin by distinguishing with you the terms reproductive and therapeutic cloning.
Reproductive cloning refers to making a whole organism, doing cloning with the intent, and here we are in our board of life, same place we were last time, stem cells cloning, let's not dwell there.
Reproductive cloning, the stuff that covers' of Time Magazine are made of.
The outcomes of reproductive cloning, the desired outcome is that you get the whole organism.
And I want to distinguish that from therapeutic cloning where the desired outcome is that you get some kind of cells and specifically stem cells.
So reproductive cloning is the stuff of movies, it's the stuff of science fiction and of people having fun with covers of the New York Times Magazine.
It's also the subject of scams.
And I'll tell you at the outset, there are no human clones now.
Human cloning has not yet worked, but I am sure someone somewhere is trying.
And we'll come back to that later one in the lecture.
The other type of cloning, therapeutic cloning is something that extends directly from our lecture last time.
When you think about stem cells and you think about using those in repair, one of the issues that comes up, and this comes up in any kind of repair mechanism, repair therapy, transplantation of any kind is the question of autology or self or matching, whether or not the donated tissue genetically matches your own If it doesn't your immune system is going to try to reject it, and if it does you have a lot of problems.
The sense of therapeutic cloning is that one may be able to take cells from an adult person and turn them into an embryo and then use the embryo cells, I'll tell you how in a moment, and then use the resulting embryo and the inner cell mass of the embryo to make various stem cell lines, as we discussed in last lecture.
And the promise of therapeutic cloning is this autologous nature to it, that you would get something that was an exact tissue match.
So let's discuss a bit more, why one might want to do this various types of cloning.
In reproductive cloning there are some senses of replacing people who have been lost, some way of improving or overcoming infertility.
Spare parts, this is the stuff that some good books have been made of.
House of the Scorpion from Nancy Farmer.
If any of you have read that it's directly relevant to this.
If you haven't, House of the Scorpion it's called.
It's a very interesting book that thinks about a time in our society where we can clone exact copies of ourselves.
And these copies are made brain damages.
And they are kept in hospitals and they are used for spare parts as the real person ages.
OK?
Sounds weird, sounds peculiar, sounds unethical, but it is possible.
And I suspect there are people who would be quite keen on having this as something that could be done.
Useful producer animals, there's been a sense in the agricultural industry that if one can make identical copies of particular animals it would be very useful in getting whole herds of goats, for example, that are making very high quantities of a protein that is pharmaceutically useful, some kind of drug for example.
And there are in different ways.
That kind of approach has already taken.
Therapeutic cloning, as I said, autologous stem cells and the use, which I may or may not have time to touch on, of these cells in correcting genes which are mutant and which are bad.
But let's step back for a moment because the question of cloning has a long history.
And its history really started with a question that we have dwelt on for a number of lectures, and this is the question of how cell types are determined.
And the question of when a cell type is determined, whether or not that is accompanied by some irrevocable change in the DNA.
So this cell type determination, or let's say differentiation because that's the end point, as you know, is cell type differentiation caused by DNA change?
Or let me actually say DNA change or DNA loss, which is the big one.
And this question, and you could imagine, as you form a retinal cell or as you form a skin cell, you might get these different cell types because you throw out, you actually delete from the genome whole batteries of genes.
OK?
Or you make lots of copies of another gene.
So you may actually change the DNA.
You kind of know the answer to this question because I've already told you that all cells have pretty much got the same amount of DNA, the same type of DNA.
And what's important is whether or not different genes are active or not.
But how did I tell you that?
How was I able to tell you that?
So let's spend a few minutes exploring that.
So the first question that was asked in terms of the cell type differentiation question was the question of cellular potency.
Where cellular potency, and we're on number two of your handout, where cellular potency asked in an early embryo, when can an early embryonic cell or set of cells support growth of the entire organism?
In other words, at what time during development is there totipotency, is there complete information in the cell?
And this is to exemplify the kind of experiment that was done, as exemplified here in the sea urchin where you can take an eight cell stage sea urchin embryo and transect it in one plane or another plane.
And if you do it in one plane you get an abnormal embryo and a normal embryo.
They say abnormal but it's sort of normal-ish.
And in the other plane you get out two normal, but small, larvae.
OK?
And what this said was that the cells of this eight cell embryo are not equivalent to one another.
And, indeed, even four cells put together, if you bisect in the wrong plane, was not sufficient to confer totipotency to those cells.
So at the eight cell stage in the sea urchin, the cells are not totipotent.
And you can do similar types of experiments in many organisms.
And what you find is that in the very early embryo, you can often split the embryo into its component cells and you can get out a number of genetically identical organisms.
This clearly happens in humans in the case of identical twins.
It happens in armadillos routinely where armadillos, for their own reasons, split into eight cells at the eight cell stage, and you get identical octuplets out of most armadillo pregnancies.
OK?
So there's totipotency but it's transient.
And this transient nature of this cellular potency, so the embryonic potency is transient.
And the question was then raised why?
Was it something that happened to the cytoplasm or was it something that happened to the nucleus?
And this is where we get into the questions of nuclear potency and the beginnings of cloning as we know it.
And the technique that you need to know is called somatic cell nuclear transfer.
Abbreviated SCNT.
All right.
So this is what SCNT looks like.
This is number three on your handouts.
And it goes like this.
One takes an egg, early embryonic cell that you know is able to support total embryonic development with its own nucleus, and you go in and you remove the egg chromosomes.
You can do this because most eggs are arrested at meiotic metaphase two.
The chromosomes are condensed and they are gelatinous, and you can actually remove them.
I'll show you a movie of this in a moment.
You can remove these egg chromosomes, and you can then get a nucleus from some adult or other cell type.
And you can put it in a glass pipette, and you can pipette it into the enucleated egg.
The egg heals up.
It's now got a substituted nucleus.
And you can go and you can ask what this egg which its somatic nucleus can do.
And the somatic cell nucleus is diploid so you don't need fertilization because you've already got the right complement of chromosomes.
And if you do that, as was done in frogs in the early 1960s, you get results like this.
The host egg came from a frog which was brown and the donor nucleus came from a frog, or cells of a frog which was albino.
And that was important because when you looked at all the cloned frogs they were all albinos.
OK.  And so you could see that they came from the donor nuclei.
Great question.
See me afterwards.
I actually don't know the answer.
That's a great question.
OK.
Here are the cloned frogs.
OK?
And you can get buckets full of these things, but.
I've seen buckets full of these things.
OK.
But.
OK.
But before the but.
So let's do a couple of things here.
So want I can tell you is that in a 500 cell frog embryo, one can very clearly distinguish cellular potency and nuclear potency differences.
So if you take any cells from a 500 cell frog embryo, you isolate the single cells and culture them as single cells, none of them will form an embryo.
However, if you isolate nuclei from each of those cells and transplant those nuclei each into an enucleated egg, 100% of them will form normal embryos.
So there is a massive difference between cellular and nuclear potency at that stage.
It tells you that the nuclei are totipotent and the problem was with the cytoplasm of the cell and that the egg cytoplasm has got something in it that is special to support development.
So that was a very positive result, OK, that you could get these cloned embryos, these cloned frogs from nuclei that by cellular experiments were not totipotent anymore, but -- But, and this is really important, if you did that experiment with nuclei derived from older and older embryo, here's blastula, gastrula, neurula tail bud and so on all the way up into adult cells, the frequency of embryos that arose from, or frogs that arose from the somatic cell nuclear transfer plummeted.
So if you did the experiment at blastula stages you got 80%, even 100% success rate with the cloning with your somatic cell nuclear transfer.
If you did it at gastrula it would drop to 50%.
By late gastrula, which is just in frogs, about five hours after late blastula, you were down at around 15%.
And by the time a day later had gone by and you were a tail bud embryo, you were close to zero percent success.
So although nuclei were totipotent at some stage, there was a tremendous decrease in the potency of the nuclei as development went on.
And so that raised issues, which we'll come back to in a little bit.
Now, after the success with frogs, people started looking to mammals immediately to see if they could clone mammals.
And there is a real checkered history here and some shady science and some claims particularly that in mice one could clone mouse embryos and could do it with high efficiency.
And it turned out that all of that data was faked, and that really put a damper on the field for several decades.
And it wasn't until the folks in Edinburgh in 1997 tried again and did it with an open mind that they were able to clone this sheep, Dolly, who never knew how famous she was, who was the first mammal who was cloned by nuclear transfer.
So how does nuclear transfer work in mammalian embryos?
The principle is identical to what I've shown you in the frogs, but I'm going to show you a few movies to give you a sense of how it's really done.
So this is a microscope that's set up for somatic cell nuclear transfer.
And these things here are micromanipulators, including injection and suction devices that I'll show you in a minute.
I'm going to show you a series of three movies that were made by a very talent MIT graduate student, Kevin Eggan who is now a fellow over at Harvard, in Professor Rudolf Jaenisch's lab, one of my colleagues over at the Whitehead Institute.
And the first movie is going to be the isolation of nuclei from somatic cells.
So here is a glass pipette, and Kevin is drawing up a cell into the pipette.
And he is going to spit out, he has, you'll see it again.
He has spit out the contents of the cell other than the nucleus.
So there's this little thing in there that's a nucleus.
So here's another cell.
He's pipetting it up and down to break it open and is retaining the nucleus because it fits well in the pipette and doesn't come out as readily as the other cell contents.
There goes the cell into the pipette, and here comes the cell debris, and in the pipette is retained the nucleus.
OK.
Step two, removing the chromosomes from an unfertilized egg.
This thing is a suction device that sucks and holds the mouse egg.
Here is a microinjection needle.
And it's not pointed because it's actually being helped by an electric pulse to enter the egg.
And Kevin is now pulling out the metaphase plate from an egg.
Those are the chromosomes from the egg.
How does he know where they are?
Well, he's a smart guy.
And if you've looked at enough mouse eggs, you can actually see them under the right microscopy conditions.
OK.
So there goes the pipette again and here comes some more chromosomes.
And there you go.
OK.  Easy when you know how.
There.
OK.
So once you've got your bucket load of eggs that are enucleated and you've got your pipette full of somatic cell nuclei, you can put the two together.
So here's your enucleated egg held on by suction and here's your pipette that has got those nuclei that came from the somatic cells.
Watch the barrel of the pipette or the microinjection pipette and you will see it.
There is goes.
There's a nucleus going down into the cell.
And if you watch carefully we will see when the pipette comes out.
You really have got to get it out of there.
That nucleus is gone.
OK.
Here's another one.
There's the nucleus.
You've got to watch.
There it goes.
You can actually see it going it out if you look carefully.
He's really making sure he pushes it out.
And there, another nucleus somatically transferred.
OK.  And the third one here is the nucleus.
A little pulse of electric current to get the pipette in, and then you push the nucleus out by pressure, and there you are.
And the last one, here's a lost nucleus.
There is goes.
All right.
OK.
So this works.
It does work.
These mice are living proof that somatic cell nuclear transfer gives clones and so is Dolly, but there are issues.
There are many, many problems.
Cloning by somatic cell nuclear transfer is very inefficient.
It depends somewhat on the species one has looked at, but it is generally around or less than 1%.
So fewer than one in a hundred of those mouse eggs actually make it on to give you a mouse.
The ones that do, although they might look normal, turn out all to have something wrong with them.
So in mice 100% of the animals that you get out have got something wrong with them.
And this is probably true in all clones that have been made.
It was true of Dolly who died an early death due to arthritis and various other issues.
And it's true in essentially every animal, in all animals that have been carefully examined.
So what are the problems?
This is to remind you again that cloning efficiency from adult nuclei is very low.
One of the problems, and these animals would not be viable, is something called large offspring syndrome.
This is a normal newborn mouse and this is a cloned mouse.
And you can see that it's at least double the size of the normal mouse.
This is the placenta from the normal mouse and from the cloned mouse also enormous.
So there is an issue here with the size of embryos or the size of the animals.
There's also an issue with lifespan.
This is a survival curve for mice.
And mice cloned animals are pretty much all dead by about 750 days after birth.
Whereas, for most wild type or even animals made by artificial reproductive technologies, your survival is way, way longer than at.
At least double that time.
So there is an issue with survival in general.
And I'll show you a little bit, I pulled out a couple of panels of some primary research data to show you that the reason these animals are abnormal is because their gene expression is abnormal.
OK?
So this is a PCR-based assay looking at various RNAs from various different genes.
OK?
These are, each of these names up here is a gene name.
It doesn't matter what they are.
And each of these white bands here is the demonstration that an RNA is present.
In a normal embryo there is a band under each of these gene names indicating that the RNA is present.
In a cloned embryo, and this is true for all cloned embryos, you see that at least three bands are missing.
And here's one I didn't diagram, didn't put an arrow on.
Four bands are missing.
And in different cloned embryos different RNAs are missing.
So you can show that in cloned embryos where you look gene expression is abnormal.
Why is gene expression abnormal?
Well, there are a couple of reasons.
One is that in fact we're wrong and that the DNA content of cells really is changing as they get older.
But that's not supported by projects like the Human Genome Project which suggests that the genes, in fact, are stable in all cells.
A much more plausible and accepted explanation of the data has to do with the control of gene expression.
And an aspect of the control of gene expression that we have mentioned, although not by this particular term.
And so I want to introduce you to a new term which is epigenetics.
Epigenetics means outside genetics.
And this refers to the control of gene expression that does not directly bear on the DNA sequence.
We can have an argument about this later, but that's how I'm going to define it now.
This refers to the regulation of gene expression -- -- that is not base sequence directed.
That is not directed by the DNA base sequence.
All right.
And I wrote it for you.
Let's look at some examples.
This is a cloned cat.
This is a kitten and this is its mom.
And you can see they're very cute but they are different from one another.
OK?
They're different but they've got exactly the same DNA content.
They're different.
They're expressing different genes.
OK?
This is where we've been.
Let's move somewhere else.
I would think we might have someone who is one of an identical twin pair in the class.
Anyone an identical twin?
Oh, yeah.
Well, we're statistically borderline.
They're one in a thousand.
So we don't have it.
Do we have fraternal twins?
Are you a fraternal twin?
One.
OK.  All right.
A few.
OK.
So there we're OK. All right.
OK.
Identical twins.
Identical twins are generally not absolutely identical.
They might look a bit different, they might have different behaviors, and they may have different more quantitative traits.
The term concordance refers to whether or not twins share a trait.
And that's particularly useful in looking to see whether diseases are causes by DNA based mutations.
There's something called scleroderma which is a skin disorder that has 98% concordance.
If one twin has it the other twin has it.
And it is directed by specific changes in the DNA sequence.
On the other hand, asthma only has a 54% concordance.
And that suggests that asthma is directed, this is a high concordance, it's higher than fraternal twins have, but it certainly doesn't reach the heights of scleroderma.
And what the data suggests is that scleroderma is caused only due to changes in DNA base sequence and asthma is caused due to changes in the DNA base sequence as well as something else, and the something else is believed to be changes in gene expression that are regulated above the level, at a different level than the base sequence per se.
Now, if we think back to molecular biology and the regulation of transcription, I pulled this slide from one of your previous handouts, your gene, your promoter and your promoter activity leading to production of RNA.
That is the level of gene expression that seems to go wrong in cloned embryos.
We talked about regulation of transcription by binding of transcription factors to the promoter.
And you've just had an exam on this previously.
And also by something called chromatin modification.
And epigenetics has to do with chromatin.
Again, this is an old slide.
You've had it previously.
And I pointed out to you that DNA is packaged into tightly wound coils called chromatin by winding around particular proteins.
And the particular proteins it winds around specifically are called histones, and these histones and the DNA wound around them are inhibitory to transcription and they must be loosened up or removed to allow transcription to take place.
So bear that in mind while I tell you something else that can happen to DNA.
DNA can be covalently modified, and this is on your handout.
This is number six on your handout.
DNA can be covalently modified.
In particular, cytosine, the base cytosine can have a methyl group added to its 5 position, and it is now called 5-methylcytosine.
And this does not change the base sequence of the DNA, but it does have a profound affect on whether or not the DNA is transcriptionally active.
So let me go through a cartoon with you to indicate how this is, and then I'll come back to the relevance of this for cloned animals and their abnormalities.
So this is number seven on your handout.
The DNA that I've shown in black, and I've shown double-stranded DNA as a single line here so don't get confused, is wound around purple barrels of histone proteins.
And this chromatin configuration where the DNA is wound around histone proteins has to be removed or has to be modified to activate transcription.
When the histones are removed the transcription factors can do their thing and you can get transcription taking place.
And it turns out that when the DNA is methylated that does not happen.
Methylation prevents the removal of the histones from the DNA template, from the gene, and thereby prevents transcription from taking place from a gene.
OK?
So on the handout that I've posted on the Web, I have put an explanation of why methylation prevents histone removal.
I'm not going to go through it in class, but you can go and look at it for your interest on the Web posted.
OK?
But what I want you to have in your minds now is that methylation prevents DNA removal.
Now, methylation therefore, because it can repress transcription, is used by this body to use by the cell as a mechanism to regulate which genes are active and which cell type.
So in cell type one where there's a gene that I have depicted as so, double-stranded and there's a promoter, no methylation, it's expressed.
That same gene in cell type two might be methylated and therefore not expressed.
This is number nine on your handout.
Reciprocally gene two might be methylated in cell type one where it's not expressed, not methylated in cell type two where it is expressed and so on.
And what you can get from this is that in a particular cell type a set of genes will have a characteristic methylation pattern.
And you can get a kind of methylation profile of the entire genome that dictates which genes are going to be active and which are not active, or is one level of regulation that dictates which genes are active and expressed and which are not.
There is another level of methylation that has to do with what's in the egg and the sperm and what methylation patterns are there.
And it turns out that not only are there cell type specific methylation differences but there are gender specific differences.
And the methylation patterns of female genes, of genes in the female are different than some of the methylation patterns of genes in the male.
This particular kind of methylation difference is called imprinting.
Here's an example.
Here's a gene.
Gene one in the egg has a particular methylation pattern.
That same gene two, gene one in the sperm has a different methylation pattern.
OK?
And this difference makes it a so-called imprinted gene.
Here's gene two.
Also differences between the male and the female copies of the genes.
And here's gene three which has the same methylation pattern in both egg and sperm and is therefore not imprinted.
So there are these two kinds of methylation that take place, the cell type specific and the gender specific.
And these all get put together into a complicated arrangement of changing methylation patterns during development.
And I'm going to go through this.
This is number ten on your handout.
And I'm going to go through this as an animated cartoon.
So here is, represented by the colors, the methylation pattern in the egg and in the sperm.
They are imprinted with their own methylation patterns.
During fertilization they join to form a zygote which goes on to make an embryo.
Now, early on during embryogenesis some cells are set aside.
And we looked at this with determinants in Caenorhabditis.
Some cells are set aside to make the future germ cells.
And very interestingly as that happens there is a demethylation of all the imprinting of methyl groups.
And as those germ cells then go onto become egg or sperm new methylation takes place and you get the imprinted patterns put back on egg and sperm appropriately.
It's a complicated process.
And if you think about it a little you'll see there are some real issues there, but I'm throwing this out as a cartoon.
Also in the embryo new methylation patterns arise that I've called embryonic methylation patterns.
So there are enzymes that are adding these methyl groups onto cytosine, and there are a bunch of different enzymes.
And in the embryo there are new methylation patterns that arise.
And as the embryo goes through its thing and makes its different cell types, additional new methylation patterns arise.
I've called these somatic methylation patterns.
And they are different in the different cell types, as I've just shown you.
So there is this complicated way of changing methylation patterns during development, and this has implications for the chromatin structure and it regulates the expression of genes in different cell types.
It is one important level of gene regulation.
And for cloning purposes you should know that adult methylation patterns never normally revert to embryonic patterns.
The germ cells are set aside very early during embryogenesis, and once that has happened you go on and make all these new methylation patterns and you never turn them back.
OK.
So what does this have to do with cloning?
Well, the problem with clones and the problem with gene expression in clones, it is believed, is that there is a problem with changing the methylation patterns and the chromatin structure back to the embryonic structure.
So in normal developments, I've summarized the cartoon that I just showed you in complicated form, you move from some kind of egg-sperm methylation state that moves on, that allows you to move on to activate the early embryonic genes.
And because you have put in the appropriate embryonic methylation patterns.
And then in a stepwise way you go on to act two, form the adult methylation patterns and activate the adult genes.
OK?
So there's a connection between the methylation patterns and correct gene expressions.
This is a summary of the big cartoon I just showed you.
After somatic cell nuclear transfer there are three possible outcomes with regard to methylation patterns and with the prognosis for the health of the resulting embryo.
One outcome is that this adult donor nucleus that has its adult methylation patterns, when injected into enucleated egg, will undergo some kind of reprogramming and will land up with a normal embryonic set of methylation patterns.
If it does that it will be able to support normal development.
However, it is believed that this very, very rarely, probably never happens.
What is much more likely is that this adult donor nucleus, when injected into an enucleated egg, goes on and is partially reprogrammed by the enzymes that are present in the egg.
And it goes on to set up some kind of abnormal set of methylation patterns, some of which look like the embryonic patterns, some of which look like the adult patterns.
And it goes on correspondeningly to activate gene expression abnormally.
And, therefore, to make an abnormal embryo.
And then a third outcome is that you just don't reprogram this nucleus at all, and it's an adult nucleus sitting in the egg.
And in that case you get no embryo resulting from the somatic cell nuclear transfer at all.
So it is, and this is number 12 on your slide, on your handout.
So the notion now is that abnormal clones are abnormal because of an inability to reprogram the DNA, they methylation patterns and the chromatin structure to turn you into a normal embryo.
So let's move on to the final thing I want to discuss with you today which are issues.
Issues with somatic cell nuclear transfer and the whole notion of cloning.
And I have a list of them here.
They fall into two categories.
One are the ethical issues and the other are the methodological issues.
We raised last time the ethics of using human embryos to make human stem cell lines.
And this ethical issue is still there.
Is it acceptable to take a human embryo at the blastocystic stage, when it's a ball of cells and to harvest it, to kill it, to take the cells and to make stem cell lines?
Someone asked me whether you couldn't take one cell out of that embryo and make a cell line out of that embryo and let the rest of the embryo grow.
And in theory you actually could do that.
It would be a pain in the neck, but you actually in theory could do that.
OK?
But that's not a possibility that's really being seriously entertained.
This whole use of human embryos to support actually making a whole organism has its real issues.
What if you do that and you land up with as you do in the mice and the cows and the sheep and the everything else?
You land up with a whole bunch of abnormal human beings, what are you going to do about this?
What's the fate of them, what's your commitment to these abnormal babies and what is the commitment of society here?
These are very big issues.
Another issue, which is a very real one, is where are the eggs that you're going to do all this somatic cell nuclear transfer going to come from?
So there was a recent report from a Korean group who had actually made a human stem cell line after somatic cell nuclear transfer.
They had used 650 human eggs.
And they had used 16 women to donate these 650 eggs.
At the present rate of making clones and making stem cell lines, you would probably need at least 500 to 1,000 human eggs per attempt to get autologous stem cells.
Now, I told you previously, when we talked about assisted reproductive technology and you retrieve oocytes from the ovary and do in vitro fertilization, you usually get about 20 oocytes every time you try to get eggs from a donor.
That's a lot of people who have to donate these eggs.
And if you're actually going to try and grow these into babies that's a lot of surrogate mothers.
So this is an issue that has not been resolved or even discussed really very well.
But these are the ethical issues that make these scientific issues move into the realm of legislature.
There is currently a federal ban on using federal money to make human stem cell lines.
So you cannot use money from the National Institutes of Health to do this.
There is no actual ban on trying to clone humans per se.
There is a gridlock because it's not clear whether to separate therapeutic and reproductive cloning from one another, but these are issues that are very loaded.
And, as I mentioned last time, in the Massachusetts House we just approved the use of private funding to make human stem cell lines in Massachusetts, but not the use of federal funding.
We cannot do that.
OK.  Methods.
To date no primate has been cloned.
There is something difficult about cloning primates.
No human, no monkey has been cloned yet.
Why?
What's the problem?
Not clear at this point, but I suspect those problems will be overcome in the near future.
OK.
Some other thoughts.
What about these eggs?
I told you the source of eggs for somatic cell nuclear transfer in humans is a big issue.
What if you could turn ES cells into eggs?
Could you treat then with something and turn them into eggs?
And that would overcome the need for donors.
Well, in fact, an experiment was published a little while ago showing that you could take ES cells and you could activate them in various ways.
And activate expression of these proteins that you should recognize, ZP1, ZP2 and ZP3.
Remember those?
Yes?
OK, the zona pellucida proteins.
So that you could take these cells and at least start turning them into eggs.
That's the sense that you might be able to do.
OK. And then finally the question of how can you reprogram these adult nuclei to make the whole process more efficient?
And I'll end with telling you two thoughts that are being entertained by people in the field.
One of them is to take an adult donor nucleus and to try to reprogram it into an embryonic state sequentially.
So if you take the adult donor nucleus -- Thank you.
If you take the adult donor nucleus, you inject it into an enucleated egg, you let it partially reprogram, and then you take the nucleus out of that partial embryo and put it into a new egg, that nucleus does much, much better.
And that's true in frogs as well.
And you can then sequentially reactivate the genetic program.
And that's actually one of the best arguments that the genetic material really does stay the same.
And it's a problem with the chromatin structure of the DNA.
And the other dream that people are pursuing is to take these adult donor nuclei and incubate them in some kind of reprogramming extract, some mix of proteins that will magically change the chromatin structure of the nucleus and allow it to revert to a normal embryonic nucleus so it an go on and do its thing.
And I am going to stop there.
And thank you very much.
So systems refer to, as in any case, something working together.
In the case of biology, systems working together for a specific function.
The tissues and the cells working together to a specific function.
We're going to begin with a discussion of the nervous system.
Let me write this on the board because you really need to know this definition.
Many organs or tissues collectively giving one function.
And we're going to begin with a discussion of the nervous system.
The function of the nervous system is one of communication.
But that single term communication really belies the complexity and the enormity of the nervous system.
Could I please have a couple of letter carriers?
Why don't you [do the other side?
.
So you can do each side.
OK?
All right.
OK.
So we're going to talk about the nervous system and its role in communicating.
Communicating things coming from the outside in, communicating things that happen within your body.
Within the nervous system there is one type of cell that is the most important cell with regard to communication.
And the term, the name of that cell type is a, wait.
I've got a new system here.
A neuron.
Very good.
Excellent.
Right.
It's sort of your moment of fame.
But you have to be careful, I might hand the microphone over for the rest of the lesson.
OK.
So the cell type that you need to know is the neuron.
And the neuron is really a curious cell.
In my opinion, it's one of the cells that is really one of the ancestral cell types.
In fact, I think it might have been one of the first cell types because the thing that all cells really have in common, whether they live as single-celled organisms or as multicellular organisms, is a way of sensing what's around them.
Even single-celled animals sense whether or not there are particular chemicals around them.
I showed you in the morphogenesis section the dictyostelium, single-cells moving towards a cyclic AMP source.
Those are single-cells sensing their environment.
And I think ancestrally this kind of sensing of one's environment by a cell might have been the first function that really was encoded about the basic functions that allowed a cell to replicate in the first place.
So if you look at all multicellular animals, this is taken from your book, all multicellular animals, you can find networks of neurons that serve as the wires for this communication network within an animal.
The nervous system begins to develop very early during human development.
By 19 days after fertilization you can pick out a region that is going to become the future brain.
In studies in my own laboratory we've shown, using frogs and fish as models, that you can find cells that know, that are determined as nervous system cells when the embryo is still a ball of cells.
So in human equivalent terms, when it's really less than two week's old, you can find cells that are expressing genes that are characteristic of the nervous system.
So the nervous system starts to develop very early.
Now, the cell type characteristic of the communication part of the nervous system is the neuron.
And it has two interesting features.
Its interesting features are called dendrites and axons.
The dendrites and axons are both processes that project out from a cell body.
Now, the cell body is where the nucleus lies, and these dendrites and axons are processes, some of which are extremely long.
So this is a single cell.
And some axons can be a meter in length.
So there are axons that arise at the base of your spine, where they exit your spine, and they extend all the way down most of your legs.
So they can be a meter or more long.
That is a single-celled process.
So that's very thin, OK?
Not as thin as DNA but it's very thin.
And it must be fairly robust.
Usually axons are bundled together and tied together by other tissue to make them more robust.
But they are very long.
And neurons communicate by means of these very long processes connected one to another.
And we'll talk more about that in a moment.
Now, the way the nervous system is organized is actually rather peculiar and it isn't intuitively obvious that it would be organized in this way.
Neurons, let me set this up, communicate, do a lot of their communication intracellularly, that is within one cell.
And, actually, let me hold off on that until the next board.
Within one cell, as opposed to having multiple little cells all next to one another that have conversations with one another.
OK?
So you'll have a neuron that extends an axon that could be as long as these tables.
And that neuron will then communicate with another one next door in a kind of wiring diagram.
You could imagine an alternate way of communication where you had a little cell sitting next door to another cell, another cell, another cell, another cell and so on.
And you would get the same pathway, you'd get the same circuitry, but it would be an intercellular means of communication.
And it's interesting to consider why that hasn't happened.
It probably hasn't happened because it's faster, for reasons I'll explain in a bit, to communicate within the cell than between cells.
So the neuron exploits for its communication its intracellular communication.
Something that's present in all cells.
And this is a potential difference across the plasma membrane.
So all cells, as far as we know, have a potential difference across the membrane.
And this is a number you probably ought to know.
It's about 60 millivolts.
Now, that doesn't sound that much, but if you extrapolate that into terms that we think about and think about the width of the plasma membrane which is just a few nanometers, that's actually a very large voltage difference.
And if you do the calculations, you actually come to the conclusion that the potential difference across a cell membrane is about 100 fold higher than it is in high voltage power lines, OK, but it's very little.
The actual numbers are very little, and so you don't realize that.
But it's a very substantial potential difference.
And what I'm going to tell you is that the neuron exploits this potential difference for intracellular communication.
Now, all cells have got this potential difference.
And in most cells, and there must be some reason all cells do, otherwise it wouldn't be maintained.
But I must say for most cells it's not really understood why you have this potential difference across the membrane.
OK. Moving on.
There are lots of different shaped neurons.
You can look at your book for a diagram.
These are some actual pictures that are taken of silver stained neurons.
Neurons take up silver dyes very effectively and show up as black.
And you can see here is one with lots of processes.
Here are some with much fewer processes.
Here's one that is much, here are some that are much more tangled and so on.
So there are many different shapes and forms of neurons.
There are about 200 different recognizable neuronal cell types, and they're found in different parts of the nervous system and do different things.
And as we move through these lectures, what I'm going to do is talk about today intracellular communication, next time I'll talk about communication between cells, and in the third lecture of the nervous system module I'll talk about how the wiring circuitry is set up in the organism.
But today the focus is going to be communication within the neuron.
All right.
So I'm going to summarize that a little more because I want you to be with me.
So some kind of input.
The notion is that some kind of input impinges on the dendrites, the dendritic processes of a neuron.
And these dendrites communicate this information often through the cell body, not always, to the axon of the same cell.
And this axon then communicates to a dendrite of another adjacent cell.
And that dendrite communicates with the axon and so on.
So one can distinguish two different kinds of communication here.
The first is the one that I have told you already, intracellular communication.
And this is an electrical ionic movement.
Well, this is an electrical communication.
And it involves the movement of ions.
And the second kind of communication is between one cell and another.
This is intercellular across something called a synapse, or synapse, depending on your preference.
And this is usually a chemical kind of communication, although in some instances it can be electrical.
OK?
And so, again, today's lecture we're going to talk about intracellular communication and on Wednesday we'll talk about this intercellular communication or synaptic communication.
All right.
So let me go through briefly on the board the notion of how a neuron sends a signal from its point of origin along the length of this very long axon, or usually fairly long axon or sometimes long axons.
Axons can be very different lengths, but they generally are much longer than your regular somatic cell which is just about 10 microns in diameter.
How is this intracellular communication affected?
So what I'm going to do is to draw you part of a cell.
We'll get to your diagrams in a moment, but I'm going to draw you part of a cell.
So this is the axon, part of an axon.
PM is the plasma membrane.
And let me take just one piece of that plasma membrane and magnify it over here.
So we can have out and we can have in.
And the deal is this.
The cell is set up, and we'll talk about how in a moment, such that it is more positively charged in the extracellular environment than it is in the intracellular environment.
And that is stable.
And it's stable at a particular potential that I've given you already, the 60 millivolts, which is called the resting potential.
Now, along comes some kind of input, a sound, a touch, some food that you've eaten.
And the neuron is told that there's an input.
And what this does is in one portion of this axonal cell membrane, plasma membrane, it reverses the polarity transiently and very profoundly such that -- -- in this region, I'll just circle it.
I think it's easier.
In this region of the plasma membrane you have switched the polarity.
This switching of polarity gives a change in potential difference.
It almost reverses the potential difference completely.
So whereas it's minus 60 millivolts in your resting potential inside to outside, it almost completely reverses to about plus 50 millivolts.
And this reversal of polarity -- -- is given the name a depolarization or is a depolarization.
And its neurobiology term, which you must know, is an action potential.
All right.
So this is very interesting, but what's even more interesting is what happens next.
And so you should be asking at this point, well, how does that happen?
And I'll tell you that in a bit.
But what even more interesting is what happens next because that depolarization, that action potential then moves along the neuron.
So if we draw a couple more, we actually only need to draw one more view of our cell membrane, after some period of time, and this is a millisecond timeframe here.
After the millisecond timeframe, that initial depolarization reverses, and next door the membrane depolarizes.
OK.
So the signal then propagates along the neuron, along the axon in a particular direction.
And you're going to ask now, well, how does it know which direction to propagate in?
And I'll tell you that in a moment, too.
Interestingly, the action potential is all or none.
You either reverse the polarity completely or you don't.
You don't get a little action potential or a big action potential.
You get an action potential.
OK?
And that's important to know.
So that is enough to start.
And if you look at the handout I gave you today, if you don't have look at the one next door or come and zip down here and get one, that would be fine.
I have in my first cartoon the outside of the cell and the inside of the cell.
Now, I've shown these as completely positive and completely negative.
That is a lie.
They are not.
OK?
There is a potential difference.
And so there's a charge distribution that's unequal, but it is certainly not completely positive and completely negative.
That is for cartoon purposes only.
So here's your resting potential and here's input from the outside of the cell.
And what happens, to initiate this action potential, is that there is a small change in the membrane potential.
It just changes a little bit such that it's now at minus 50 millivolts or so instead of minus 60 millivolts.
And that is enough of a trigger to tell that membrane, and the membrane right next door, to depolarize and to give a full-blown action potential.
So now that piece of the membrane has reversed its polarity.
Positive charges have rushed into the cell.
I'll talk about this more in the mechanism in the moment.
Positive charges have rushed into the cell, or positive ions have rushed into the cell, and you've got your action potential.
So the action potential is this reversal in membrane polarization, this depolarization.
The action potential gives you an unequal charge distribution within the cell, so you've got these positive ions around.
And they actually start defusing a little bit.
Now, the axon is not a very good current conductor.
Charges leak out of it.
And very rather quickly and in an active way this charge in equality will be redistributed.
But it does leak a little bit.
It moves a little bit by diffusion.
And, as it does so, it depolarizes the membrane next door just a tad to this threshold potential.
And that will make this membrane next door amenable to an action potential and it will depolarize, and so will get an action potential next door.
Now, I drew up here on the board that you get, in the first place where there was an action potential, the redistribution of charges again so that you get rid of the positive charges on the inside.
And I've depicted this here.
OK?
So here are the charges from, this is all on your handout, guys, OK?
These are the first, I believe, three figures on your handouts.
So look at them and make notes, but you certainly don't need to draw these.
OK.
So here I've depicted the axis positive charges, the axis positive ions being moved back out of the neuron.
And I've put these circles with a line through the middle which conventionally means no something or other.
And what it means in this case is that this membrane that had just been part of an action potential is now no longer competent to elicit another action potential.
There's some kind of inhibition that's been put on it so it cannot give another action potential.
So it re-polarizes, but it cannot re-depolarize.
And, again, there is about a millisecond timeframe window before it can depolarize again.
And you can do the same thing.
You can get the action potential moving up along the neuron.
Here's the direction of propagation.
And the direction of propagation is a result of diffusion, slight diffusion of the positive ions that are coming in during the depolarization that change the threshold, that change the membrane potential right next door to a threshold potential which allows the action potential to be triggered.
And the propagation direction is unidirectional because the membrane that has just depolarized, that previously depolarized is no longer able to re-depolarize for some lag period.
Yes?
Maybe?
Whatever?
All right.
We'll go with whatever category.
OK.
I always have to work at this.
OK?
I study this stuff and it takes me a while.
So think about it.
It's actually a very elegant strategy by the cell.
You'll deal with this more on problem sets.
All right.
So let me move on.
And let me move onto the question, actually, I'm not going to move onto this question.
Yes.
I'm going to move onto, all right.
I will move onto this question.
I'm going to tell you more next time that the action potential initiates at one particular place in the cell which is right at the place where the axon starts.
And that is called the axon hillock.
And we'll talk more about this next time when we talk about synapses.
I do want to very briefly address the speed of propagation of the action potential.
So it's been calculated that action potentials move at a speed such that you, if you ran the length of a football in one second, that is how fast the action potential is propagating down the neuron.
OK?
So you can do the calculation of how many meters per second it is.
It's very, very rapid.
OK?
So you could get a nerve impulse in milliseconds going from one part of your body to another.
It's actually interesting, though, it's not that rapid.
And you've probably all had the sensation of touching a hot stove or something hot.
And it takes you a moment before you actually realize that you've touched the hot stove.
And then realize and you remove your finger.
That lag, that moment of realization is a measure of your action potentials getting to your brain and then your brain saying, wow, or not even quite in your brain saying, ooh, I better move my finger.
So that's actually a measure.
There is a finite time that propagation takes.
OK. Now, propagation of action potentials can be increased by increasing the diameter of a neuron.
The fatter the diameter the greater the, the lower the surface area to volume ratio the more that current can move within the cell before being dissipated in various ways.
And things like invertebrates, so the squid for example, has got axons which are a millimeter in diameter.
They're called squid giant axons.
They've very popular preparations to be used in neurobiological studies because they're so big.
And they are very, they transmit an action potential very rapidly, or they propagate it very rapidly because they've got this high volume to surface area ratio.
We do not do that to increase the speed of propagation.
What we do instead is to wrap our axons in insulation.
And the insulation we wrap our axons in is called myelin.
It's secreted by cells called Schwann cells that are part of a system of support cells in the nervous system called the glia.
And these myelin sheaths that wrap around the neuron do two things.
Firstly, they prevent leakage of current as it moves along inside the axon.
And so you get rapid propagation of current inside the axon.
And, secondly, you get depolarization only at specific places where there is a break in this insulating myelin sheath.
From your book, this is front your book.
And from your book also I have taken this diagram where, it's actually hard to see here, but this is a break between the myelin sheath.
It's called a Node of Ranvier.
And here is where you get depolarization.
The ions that cause the depolarization rapidly move along inside the axon to the next place where depolarization can take place.
And that really increases the speed of propagation of a signal in our own neurons.
All right.
But, you might be asking, how is this potential difference set up and how is propagation occurring at the mechanistic level?
I've thrown out charge movement, ion movement in a somewhat undirected way, but let's try to be more directed now and talk about what is actually happening.
And the thing that's really important to know about is a set of proteins, actually, two sets of proteins.
One are called ion channels and the other are called pumps.
So the membrane is a wonderful thing.
They insulate cells from one another.
It allows the cells to function as a unit.
And, indeed, it allows intracellular organelles to function as a unit.
But there is an issue with the membrane in that it's insulating.
And very little can get across it unless it's a hydrophobic something.
So anything hydrophilic, including water, cannot cross the plasma membrane.
And so you have to now devise, you've got this great invention of the membrane, but now you've got to devise a way to actually get things in and out of the cell.
And this is where channels and pumps come in.
So an ion channel is a protein, or usually a set of proteins that work together, that make a channel, a pore or a channel in the plasma membrane.
There are a couple of different kinds of channels you should know about.
All of them are selective.
Selective means that they let some things through and don't let other things through.
Some of them are open all the time.
They also may be called leaky.
I don't really like that term.
I think open is a better term.
And some of them have regulated opening.
They're closed but they can be opened.
And these are called gated channels.
Another class of proteins that you need to know about are the pumps.
Pumps are also proteins that sit in the plasma membrane.
And they transport things across the plasma membrane, too, but unlike channels which are literally pores through the membrane and things can move through by diffusion, pumps use ATP to transport things across the cell membrane.
And you can have things going into the cell, you can have things going out of the cell, and you can have things going both ways at kind of the same time.
And ion channels are really key here in terms of setting up the membrane potential and propagating an action potential.
This is a reconstruction of an x-ray crystalographic analysis of an ion channel.
So what you can see is that there are nine, eight different proteins that are arrayed together to form this circle.
And in the middle is a hole, literally, and that is the pore.
And that is the pore through which the ion will move.
And it's very interesting that the selectivity of an ion channel, well, how do you get a ion channel selected?
Well, it hasn't been clear until this kind of analysis, which is fascinating.
So here is a potassium channel down here.
So this is something that lets potassium through, but it will not let sodium through, even though sodium is smaller than potassium, that sodium ions are smaller than potassium ions.
So how does that work?
Well, the way it seems to work is that the ions get through the pore if the pore doesn't know they're there or if the ion doesn't know it's going through a pore.
So if the ion cannot distinguish whether it's interacting with water in solution or with the ion channel, it will be able to diffuse through the pore.
If it can distinguish it thermodynamically it will not.
So here's an example.
Here is potassium interacting with water.
Notice the spacing of the oxygens and the potassium.
And here's sodium interacting with water.
And you can see that since sodium is smaller the oxygen, the spacing of the oxygens and the sodium is closer, tighter.
When potassium goes through the pore it turns out that there are a number of oxygen sticking out into the pore, and they interact with potassium in exactly the same disposition with exactly the same spacing as those water molecules in water.
So this potassium ion cannot tell whether it's interacting with water or whether it's traveling through the pore.
On the other hand, if the sodium ion were to get near the pore its charge interactions would not be the same as they are in water.
It would only be able to interact with two of the oxygens and not the other two.
And this is thermodynamically unfavorable, and it would not be able to go through the pore.
So this is a very recent and very beautiful explanation of how pores can be selective.
The ions go through one at a time in single file, and they can go through very rapidly because this is a diffusion driven process.
OK.  Now, what about these gated channels?
I've shown here, from your book, a picture of an open channel and a gated channel that is open sometimes and closes at other times.
The voltage gated sodium channel, we'll talk more about in a moment, is a particularly important channel.
And this is a diagram that is, you know, it's a diagram but it will illustrate two points that I want to make.
Firstly, you can find a state of a gated channel where it's closed and it won't let any ions go through.
Upon a stimulus that channel will open and it will let the ions through.
And then what seems to happen is that there's some kind of lash back where the channel says uh-oh, and there's some feedback that goes and closes the channel.
But it's not in the same way that it was closed in the first place.
And so I've called that an inhibition or closing or being refractory.
And if you think about what I told you about propagation of information along a neuron, you'll be able to see why I am starting to tell you this.
And then later on the system resets itself and is amenable to being used again.
OK.
I should point out this is an ion channel.
This is a gated ion channel.
The Nobel Prize a couple of years ago was given to Rod MacKinnon and a colleague whose name escapes me for getting the structure of these gated ion channels.
And the notion is, I'll be an ion channel now.
The notion is that in the membrane you're normally sitting something like this.
OK?
As an ion channel you get some kind of stimulus.
The confirmation, and in the case of the neuron it's going to be an electrical stimulus, that slight threshold depolarization.
And that is going to change the charge distribution on the proteins.
Those proteins will undergo some kind of conformational something and they'll move like this and they'll open up the channel.
OK?
And we know a bit more than that, but the notion is exactly that.
That changing the charge on the proteins of an ion channel, of a gated ion channel changes the confirmation of the protein and the channel opens up.
All right.
So I want to talk now, and I'll use the diagrams that I've given you, about how you generate the resting potential and how you generate the action potential in terms of the specific ions and the specific channels that are being used.
All right.
Channels that generate the resting potential, and actually I don't need to write this on the board because this is, these are going to be the last three diagrams, diagrams four and five and six on the handout that I gave you today.
So how do you get this potential difference across a plasma membrane?
Well, in all cells, not just neurons, in all cells it seems to be a function of something called a sodium-potassium ATPase.
And I will write this.
The sodium-potassium ATPase is a pump.
OK?
And its name kind of tells you that.
The ATPase.
It pumps sodium out of the cell and it pumps potassium into the cell.
And you can look in your book for a bit more information about this.
There is a fairly detailed understanding of the mechanism by which it does this.
It's a very complex mechanism, but this gives you a gradient of sodium, or gives you unequal distribution of sodium and potassium ions across a plasma membrane.
But, of course, that doesn't necessarily give you a membrane potential.
That just gives you sodium on one side and potassium on the other side.
So how do you get the membrane potential?
Well, this is very interesting because what you do is use some channels that are open all the time.
Now, what you've done with this sodium-potassium ATPase is to pump potassium in and sodium out.
So you've got high potassium on the inside of the cell.
If you open up the potassium channels, the potassium is going to diffuse down its concentration gradient and get out of the cell until there is some kind of charge pull and you get an equilibrium because it's being pulled back and pulled out and diffusing out with equal force.
The sodium channels are closed, so sodium is trapped outside the cell.
And these three things, the sodium being pumped out, the potassium pumped in, the open potassium channels and the fact that there are no open sodium channels gives you this unequal charge distribution across the membrane.
Now, obviously there has to be some balance between this sodium-potassium ATPase action and the channels.
Otherwise, you would not get a membrane potential.
But there is.
And the sodium-potassium ATPase pumps just enough that you maintain this membrane potential.
OK.
So that's your resting potential.
How about your action potential?
So your action potential involves another set of channels.
And the channels it involves are these gates -- -- sodium channels.
OK.
So in the region, this is not animated, I'm going to tell you this.
In the region of the membrane where there is an action potential, these gated sodium channels that are closed elsewhere open up and allow sodium to move down its concentration gradient into the cell.
And they do it on a very mini scale, OK, over micron or submicron domains of the cell membrane.
So you're getting this very little region of the membrane where these sodium channels are opening and they're voltage-dependent opening through this conformational change I belatedly demonstrated to you.
At the same time there is a set of channels, which are gated potassium channels, and those are closed.
There is a phase of repolarization that follows the action potential.
And during this the gated potassium channels open up, potassium leaves the cell, and the gated sodium channels close up.
And, as I mentioned, the ones in the vicinity of the action potential, or the ones that have just opened are now refractory for opening for another millisecond or so.
So in the repolarization you use potassium channels, but a different set of potassium channels, a set of gated potassium channels.
And you're also using the sodium-potassium ATPase during this time, because the sodium has to get out of the cell eventually.
OK?
And it will do so through the sodium-potassium ATPase exchange.
All right.
Here is a movie that will illustrate these different points.
This is a diagram of an axon.
I've got in a loop so we'll look at it a number of times.
The membrane is shown in pink, and these boxes are the ion channels that are initially closed.
And as the action potential moves along the axon they open up.
So here it comes again.
They're closed.
And here they are opening sequentially to allow the action potential to propagate.
Behind the action potential, take a look.
You'll see that they remain closed for a while as it propagates.
And then they eventually open up and reset themselves.
And so this is a very beautiful example of an intracellular feedback loop, both a negative and positive feedback loop.
And so here you're regulating.
If you'd like to think of it in terms of regulating gene expression, you're kind of doing it at a very mini level.
You're regulating gene activity, certainly, or the outcome, the products of gene activity.
This is a diagram above.
I haven't dwelled on this.
You'll see this more in section of the depolarization, the action potential moving along.
And you can depict that, as it's often done, by showing, by plotting change in potential against time.
All right.
Again, this is taken from your book.
Here's a diagram of the threshold potential, the action potential and the repolarization.
And I'm going to end off with a teaser for next lecture.
OK?
So that's what I want to say to you about the action potential.
I'm going to end off with a teaser for the next lecture.
These are two neurons.
Each of the red dots, see the red dots?
You probably cannot see them because there are so many.
They kind of look like a red blur.
But, in fact, they're each red dots.
Each of those red dots is a connection between one of these two neurons with another neuron.
It is a synapse.
So we've talked about moving the signal through the neuron, but now we are at the point where the neuron, that axon needs to pass its signal onto the next neuron.
And it's going to do it through the synapse, but it's not necessarily going to do it through one synapse per cell or one synapse per axon.
It's going to do it through up to 1, 00 or even more perhaps connections from each axon to the next one.
And what I'm going to talk to you about next time is how this is set up.
We have a couple of minutes left so I will take you coming down here and asking me questions, and I'll finish the class now.
Thank you.
The following content is provided by MIT OpenCourseWare under a creative commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu The end of last lecture I got an interesting question about apoptosis which I covered very briefly towards the end of that lecture.
And the question was this: Do healthy cells tell sick cells to die?
Or in turn, do sick cells tell healthy cells to die?
So, is there some kind of communication between cells as to who is healthy and who is not healthy?
For those of you who are just coming in, there is a handout outside, and up front, it's the same thing.
And that's actually a very interesting question because they really does seem, in the body, to be some sense of monitoring whether cells are healthy or not.
Now, a lot of the time, cells that are not healthy intrinsically activate their own death program, or they have the ability, they gain the ability to respond to some kind of extrinsic signals.
And in fact, signals could be sent both ways, we believe, where sick cells will tell healthy cells to die under conditions that are obviously not favorable to the organism, but more important, that healthy cells can tell sick cells not to survive.
So, the balance of apoptosis and cell survival is a very delicate one and regulated by many, many things.
So, it's a good question.
All right, we move on.
We move on to a new module.
So, you have, in fact, covered the foundations of modern biology.
You've covered them in a very superficial way.
And I want to emphasize that.
You should, by now, know how to take a piece of DNA, conceptually turn it into RNA, and conceptually turn that into a protein.
You should know what to do if the DNA sequence is changed.
You should have the expectation that the RNA and protein that are made from it will be changed, too, and you should have various other nuggets of information that are covered by foundations.
These foundations are not going to go away.
We're going to use them throughout the rest of these uncovered black boxes.
And I'm going to assume that you remember a bunch of stuff as I go through the material in the next several lectures.
But we are going to move in to the formation module.
And today, we are going to talk about things in a kind of overview way.
And I'm going to tell you five major things that are important.
And I'm going to write them on the board, and some of the things they need to know about them.
And we will use the slides as well.
So, what is formation?
What is this module all about?
Well, it's about something called "development," where development is the process by which one cell, the fertilized egg or the zygote goes on to make a multi-cellular and complex organism.
This is not about squishy little embryos.
OK, this is about life as it began, as it continues in your own bodies.
And it has immense applications for multiple biomedical and bioengineering processes that I'll talk about as we go through.
The first thing you need to know is what's written up here is that the development occurs over time and space.
And that's one of the things that makes it an incredibly complex set of processes to think about.
The overtime can be a long time.
And what's important to understand is that developmental processes occur throughout life.
Initially, in the formation of the organism from a single cell, the fertilized egg, the zygote, and later on, early in the formation of the embryo, and later, in the renewal of the adult.
And this is where the stem cells come in.
And we'll have a lecture on stem cells later on.
But development is something that occurs throughout life.
And in fact, I teach an upper-class course on development to graduate students.
And the first thing I tell them when I start to teach them is really there is no such subject as developmental biology because it covers, it includes, it encompasses biochemistry, genetics, molecular biology, protein structure, and many different disciplines.
And it occurs throughout life.
So, it really covers everything.
But we have put these things together in the formation module, and I think you will find them interesting.
OK, so here's a schematic of how things work courtesy of Picasso.
And really, this whole process is quite remarkable.
It starts with two dying cells.
The egg and the sperm, haploid cells that have a half-life that have a life of about 12 hours up to which they are dead.
But when they fuse, there's magic that happens, and you get a viable cell, the zygote, that has the capacity to go on and divide many, many times to form an embryo and ultimately an adult organism.
And as we've mentioned many times, it is estimated that there are about ten to the 14th cells in the human.
In the adults, and even in young adults, there is much replacement of cells throughout life.
And this whole process obviously takes place over time.
But that's important because the time component means that things change over time.
And in order to understand this entire slide, you have to factor in this fourth dimension.
So, what are some of the things that we're going to cover in this module, and why are we bothering to talk to you about the subject?
Well, one of the things that's of interest is the effect of chemicals on the formation of a normal body.
So, there are these things called teratogens, which are chemicals that affect formation of early steps in development of the embryo.
This is a famous one, the effects of a famous one called thalidomide that was given as an anti-nausea medication during pregnancy some decades ago.
It has no effect on rodents on which it was tested, but it was not tested on primates before you give it to people, and it has the devastating effect of preventing limb formation, and a number of people were born in the late 1940s, early 1950s, who lacked limbs because of the effect of thalidomide.
And I have to tell you, this is very interesting for those of you who are interested in chemistry.
Thalidomide has a pretty simple chemical structure.
It has been studied for many, many decades.
It's a very interesting drug because it turns out it prevents cachexia, which is the wasting that occurs with many disorders including HIV-AIDS infection.
So, thalidomide is a very, very useful drug.
It has a simple chemical structure.
It's been studied for decades, and we do not know its mechanism of action.
OK, here's another one that you might be familiar with, the drug, not the syndrome, I hope.
So, fetal alcohol syndrome, so, a normal brain, this is a brain taken from a fetus.
That is the late stage gestation human whose mother drink excessive amounts of alcohol during pregnancy.
So, during the early stages of pregnancy the first few months, alcohol is devastating to formation of the brain and also to the bones of the face.
And even a reasonable number of drinks, apparently, just a few can significantly lower the IQ points of a baby.
And so, it's a good reason not to drink if you're pregnant.
This was a very devastating case.
You can see the brain is very smooth.
It doesn't have those nice valleys and hills, the sulci and the gyri, that the normal brain has, and this fetus would not have been able to survive.
Here's one.
This is the dream of all biologists, developmental or otherwise.
It's a dream of bioengineers.
It's probably the dream of everybody.
Someone cuts off their arm in an accident: can we grow a new one?
Well, newts can.
So, in this example, newts can.
So, in this example, the newt limb, this poor animal was amputated just above the elbow.
But over a period of a few months, another limb grew back, smaller than the original but perfectly functional.
We can't do that.
How do newts do it, and how could we use that information to help people grow new limbs, or new hearts, or new eyes?
Fish, for example, the fisheye work on zebrafish can grow a new heart.
You can cut the heart into two.
Take away half of it.
It'll regenerate a new heart.
There are animals were you can remove half of the retina or all of the retina, and it will regenerate a new retina.
We do some regeneration.
You can take away two thirds of the liver, and new liver will grow back.
And that's one of the principles behind liver transplants.
But we can't do things like regenerate eyes, and hearts, and limbs.
It's been the focus of study for a long time to try to figure out how newts do it, and to ask whether we can capitalize on that knowledge.
And I have to tell you, it's been a very tough area of biology, still wide open.
And that's where the great god of stem cells comes in.
Because it's been so difficult to get limb regeneration, for example, and heart regeneration, there is a sense that perhaps we could get groups of cells to repair damaged organs.
And stem cells are the things that hold this promise.
And we'll have a whole lecture on this later.
And part and parcel of stem cells is the very newsworthy issue of human cloning, making identical replicas of things.
And we'll talk about this also in a separate lecture.
OK, all right, so let's move on to the next set of things that you need to know.
And that's the notion that there are multiple processes that are involved in setting up development formation of any multi-cellular organism.
So, multiple processes are involved.
And these are cell division, right, a single cell, so, ten to the 14th cells.
You figure out how many rounds of cell division at us.
Actually, that's a tough question.
You could go and do that, OK, and over spring break if you're lying on the beach or hanging out on a mountain, if you can find any snow you can go and figure out how many cell divisions it takes to get ten to the 14th cells.
But it's not so easy because cells don't just divide from one to ten to the 14th.
As they are dividing, there's a balance of cell division, and as we talked about last lecture, of cell death.
OK, so cell division versus cell death, and so I can't tell you how many divisions that takes because there are a lot of cells that die along the way to those ten to the 14th cells.
Here's another one: cell type.
What's cell type?
Different kinds of cells in the body, we mentioned this right at the beginning of the course.
Skin cells, cells that produce the hair, nerve cells, red blood cells, and so on.
There are probably 500 different cell types, and these each have specific functions.
And the definition of a cell type really is a group of cells with a particular function.
Something else that's interesting about turning that single cell into a multi-cellular organism is the notion of position.
So, if you look at yourself in the mirror with all your imperfections, you likely have your arms coming out of your shoulders, and your head coming up above your shoulders also.
Something has made the decision to put those parts of your body in their correct place.
And that system is a set of positional information.
It's kind of like a map.
And as you're going from a single cell to a multi-cellular organism, there is a set of molecular information that really puts the coordinates on a map just as it you had a blank map of the world and you put on the Cartesian coordinates.
So you would do that to the developing embryo.
So, there's something called positional information, and the final thing is three-dimensional structure.
And we'll have a whole lecture on 3-D structure.
But suffice to say for now, cells do not work as single entities.
The blood subsystem, the hematopoietic system, is the only real example in the body of cells working as single systems.
And even there, they don't really.
In all cases, cells group together to form tissues, and is tissues grouped together to form organs.
And the precise three-dimensional structure by which they form is really important.
So, here are some examples.
I'm going to show you a movie of some stages of human development.
This is taken from material collected a long time ago.
And what I want you to see is that over a period of a few weeks how the sides of the embryo changes due to cell division.
OK, so -- So these are all taken at the same magnification.
And you can see that as time progresses, the size of the embryo changes, and this was all due to cell division.
You can see that shape changes as well, and this is due to various processes including modeling through cell death.
One of the interesting things about this, and this will be on your website.
You can look at it.
One of the interesting things about this is that early human embryos have a tail.
So, you had a tail, a really pretty good tail until you were about 56 days old.
And then, it disappeared.
And in these early embryos, if you go back and look at this again, you will see the embryo has a tail.
So, this is a manifestation over the tremendous change in size of a human embryo caused by changes in the number of cells.
This cell death, duck feet, chicken feet, chicken feet are not web.
Duck feet are webbed.
They start off looking almost identical.
And the difference is that between the digits, between the fingers or the toes, the cells die in the case of the chicken, and they do not die in the case of the duck.
And if you go back and look at this, you will see that there are little dots between the digits of the chick, and not the digits of the duck, and those are the cells dying.
So, there is a controlled process of cell death that's very important.
Tissues, cell type, this is really one of the most extraordinary examples in the body of different cell types, and different cell types working together to a common function.
This is a diagram of the retina that I took from your book, and it exemplifies two things: one, a bunch of different cell types.
These are all nerve cells that are in the retina, and they are nerve cells that in various ways, either sense light or transmit the signal of the light once the light has been sensed to other nerve cells.
You can see, firstly, there are these different cell types, and you can see secondly, represented by colors, you can see that they're organized in layers.
There is a very precise and very important layering of cell types in the retina.
If you disrupt that layering, the retina doesn't work.
And in many cases of retinal degeneration, the retina doesn't work because the cells have, the layering has been disorganized, and the cells can't make the proper contacts with one another.
Position: we talk about axes in the animal, and in the adult as well.
We talk about an anterior, posterior axis, which is the set of organs from the head to the tail.
We talk about a dorsal, ventral axis from the back of the animal to the belly.
And we talk about a left-right axis from the left to the right.
So, if you look at yourself in a mirror again, you'll look pretty symmetric.
If you were to peel back your body wall and look at your organs inside, you will see that you are not symmetric at all.
You have an asymmetry along your left-right axis.
OK, so you will need to know these terms: anterior, posterior, dorsal, ventral, and left-right is easy.
And finally, here's three dimensional structure.
Here is the heart.
The heart is a muscle.
It's actually got several different cell types.
But it is mostly muscle where the muscle is arrayed in various, precise, organization.
And it's this precise organization of the cells that allows the heart to pump.
How do you get this organization?
We'll talk about that later on in the course.
Can you regenerate this organization either artificially, or in some kind of tissue culture system?
Very tough to do, and as you may know, there is no good artificial heart out there right now.
This is a great thing for some of you to think about for your future careers.
It's wide open.
Are there circumstances where we could regenerate a human heart in a test tube or in a large test tube on a Petri plate, for example?
It would have to be a very large test tube, right, a box, a test box?
I don't think so.
I think that is really going to be very, very tough to do, and I think we have to think more carefully about how we are going to repair damaged hearts.
Again, we'll talk more about this.
So, what I'm going to do now is to show you a movie of the development of an early fish embryo.
The kind of fish is called a zebra fish.
I work on these in my laboratory.
I have about 10, 00 of them there, and I'm going to use them.
Yes, I have 10,000 fish.
If you want to come and visit my laboratory and see my fish, you can.
It's a fantastic model system.
And I'm going to use this movie and this embryo to show you something about the sequence of events that take place during development.
I want you to look for cell division, and I want you to look for structural changes as you watch the movie.
Let's look at the movie again without the music.
And let me show you what you're looking at.
So, to zebra fish, like the chicken, and actually almost like the human develops as a disk of cells, or from a little disk of cells.
I'll show this to you again when it starts again, that sits on top of a yolk cell.
Chickens have got a yolk cell.
That's the yolk of egg.
Humans don't.
But otherwise things are very similar.
Let me try to start that.
OK, good.
So, we're going to look at this again.
Here is the yolk cell.
This big round ball here is the so-called yolk cell, and that's going to be the food of the embryo.
These two bumps on top are two cells.
This is a two cell stage embryo.
And as you watch the movie for the last time, and I'll post it on your website so you can watch it as much as you want, you'll see these two cells divide into four cells, and eight cells, and 16 cells, and so on, until they've made a little dome of cells sitting on top of the yolk cell.
And then, at that point, suddenly at some point, and this is very interesting.
That group of cells realizes that there's enough of them, and they start to move.
And they spread out to cover the whole yolk cell.
And as they do this, a bunch of them also move to the right hand side of the board.
And if you watch, you could see stuff happening.
And at the end of the movie, you could see something that might have been somewhat recognizable to you as a developing little animal in that it had an eye, and I'll point this out, and it had a tail.
So, let's watch it again and I'll stop it at various points, and I will point out stuff.
Where did my mouse go?
There it is.
OK, so here's the cell division.
Cells are dividing.
They are dividing, they're dividing, they're dividing, and making this little dome on top of the embryo on top of the yolk.
So, this little group of cells is going to give rise to the whole fish.
And although humans don't have a yolk cell like this, human embryos look very similar.
And now, this group of cells is going to spread out.
You will see a kind of haze spreading out over this yolk cell.
That's the cells moving.
There they go.
They're moving, moving, moving, moving, moving down.
And at the same time, a bunch of them are moving to the side of the embryo.
And as you watch, take a look.
Here is the high.
This oval is the eye and the brain is going to develop up here.
And watch these little shiver and shaky things on the side as well.
Those are the developing muscles.
OK, and so, there it goes, a lot of shape forming.
And we are going to move on from there.
OK, so go and take a look at this at your leisure.
We are going to talk about some of those processes again.
So, let's move on to the next point that we need to deal with, and that is a point that I've called here cell fate requires differential gene expression.
But I'm going to state it more simply on the boards, which are now not responsive.
OK, here's an easy way to state this.
Genes control development.
OK, well, that doesn't sound so surprising.
Genes control everything as far as you've been told, but let's talk about that in a bit more detail.
And there are three things you should know about this.
Firstly, the function of a cell depends on the specific proteins present.
So, cell function depends on specific proteins.
Second thing: all cells contain the same genes or the same set of genes -- And the third thing is that only some of those genes are used in each cell type.
And I'm going to use the term expressed, which we've encountered before.
Only some genes are expressed in each cell type.
And I'm also going to introduce to you a term called fate where the term is similarly used to the English term fate, but not quite, where fate and development refers to the final form and function of a cell.
All right, so let's see what we have.
Here's something we've seen before, this kind of tiresome diagram that's useful and that should be really familiar to you, the passage of information from DNA through an RNA intermediate to a protein product.
And the formation of the final product from the gene is termed gene expression.
Cell types are different because they make different proteins.
We've talked about these three cell types previously.
Erythrocytes are so because they're making globin, which carries oxygen around the body.
Neurons are making proteins, which send out the filaments, and chemicals that allow nerves to communicate with each other and so on.
Number 15 on your hand out, the first slide on your handout, you need to know this really, really clearly.
This is important.
And part of the deal here is something I'm going to write on the board, firstly, that in any process of figuring out what a cell is going to become, there are multiple steps to a final fate.
It doesn't just happen at once.
And secondly, I want to make the distinction between regulatory genes and differentiation genes -- -- where regulatory genes, we can rephrase this, control cell fate, and differentiation genes affect cell fate.
They carry out cell fate.
And here is a litany that I've written out for you that is important that you know.
In the life of a cell, as it's deciding what to become, it goes through multiple stages.
It starts off not knowing what it's going to become, and we call those uncommitted cells.
Kind of like you when you came to MIT, you were uncommitted and perhaps still are as to what you're going to do next.
But over time, through the passage of time, you get various inputs and you decide what you are going to become either as a cell or as an MIT student.
And at that point, you are called committed or determined.
That's the jargon.
You need to know it.
It's very important.
Committed or determined cells have made the decision what to be.
But they haven't gone on and become the final thing that they're going to be.
So, maybe you are premed, and by now you have committed to being premed.
So, you are different than when you came in here, but you certainly are not a qualified physician at this point.
In order to do that, you're going to have to go through another bunch of steps and find your final form.
You're going to have to differentiate into a physician or into a cell type.
So, this uncommitted-committed differentiated litany triad is something we're going to talk about over and over.
It's going to come up throughout the course probably in most of the lectures that both I and Professor Jacks will give you.
During this passage, and you've got the slide, this is the slide you actually have, is two sets of genes are activated, regulatory genes and differentiation genes.
As the uncommitted cells decide what they're going to be, regulatory genes are activated.
As the committed cells go on to read out the final thing, their final function, the activation of differentiation genes takes place.
So, how does it work?
How does a cell decide what to become, and how do different cells decide to become different things?
Well, this is the way I think about this.
This isn't the way your book thinks about it, but this is the way I think about it.
I think that there is a combinatorial regulatory code that controls cell type.
And so, let's have an example of the three cell types I've talked about before, your erythrocyte neuron and sperm.
And let's look at the regulatory genes that each expresses.
You can look on the screen.
You have this in front of you as a handout.
So, I've arbitrarily said that an erythrocyte is expressing regulatory genes, R, F, and K, the neurons expressing A, I, and K, and the sperm is expressing A, F, and K. OK, now let's look at those.
Those three groups of three letters are different from one another.
And so, for each of these cell types, there is a unique set of letters or a unique regulatory code.
What's this code made up of?
Well, it's made up of cell type specific factors, or cell type specific regulatory genes, regulatory gene products.
RNI are expressed in just the erythrocyte, or just the neuron.
They are cell type specific regulators.
General factors, K is expressed in all of the cells, and restricted factors, F and A, are expressed in just two of the three cells.
And you can do that for any cell type.
You can come up with some kind of combinatorial code of gene expression of regulatory genes that control cell fate.
Now, these regulatory genes will go on to activate the expression of a whole bunch of differentiation genes.
And usually a small set of regulatory genes will activate a large set of differentiation genes that carry out the final function.
So, here I've given you an example for erythrocytes.
It's, again, on your handouts.
The regulatory code for erythrocytes, RFK, a small regulatory code, actually it's probably about 20 genes, will go on and activate at least a hundred genes which will be the gene products that actually carry out the function of red blood cells.
OK, so regulatory genes activate differentiation genes.
What's all this regulatory stuff?
What are these regulatory genes?
So, here you have to think back to previous lectures.
Remember this diagram?
You have it.
You've had it several times.
This is the hierarchy of things that happened from the gene in the nucleus to the final, modified, localized protein products.
You can control, the cell can control gene expression at any point along this hierarchy.
It can control transcription, initiation, or termination, RNA splicing, stability or exports, translation, the initiation or elongation.
It can control export of proteins to different parts of the cell through protein trafficking, modification and control of protein stability.
And there are more.
OK, this is some of the list of steps at which gene expression can be regulated.
And those hypothetical regulatory factors I told you about in the last couple of slides can be anything that affect any of these steps from the gene to the final product.
OK, all right, so here's another one that you need to know.
Complexity increases with developmental age.
And what I'm going to tell you about is how as development proceeds, as you get different cell types forming, you have to make groups of cells with specific regulatory codes.
And these cells -- And these regulatory codes will lead to a specific fate.
So, here's a nice example just by looking at development of an organ, for example, and this is important if you're going to try to engineer an organ through tissue engineering.
You have to understand there are normally many, many steps involved.
These are the beginning steps in formation of the eye.
And it doesn't matter what each of these steps are, but if you look at the progression of cells, each of these lines are groups of cells.
You can see that they organized in different ways.
They fold.
Bits break off, and you'll end up with a structure that's a lot more complex just in a pictorial way than it was in the beginning.
And that is true in terms of every aspect of this organ.
It is more complex at the end than at the beginning.
And somewhere through here, you have to increase the complexity of this developing tissue.
So, how do you do it?
This is the way I like to think about this, and this is number 20 of your handout.
Here's an egg or a zygote, if you like, a one cell embryo.
It's one cell.
There's only one kind of cell there.
Or, I like to think in terms of territories.
There's only one territory there.
Now, I've also put a red dot in the cell.
And this red dot can be anything.
But it really is in terms of our conversation a group of regulatory gene products, or one regulatory gene product.
Watch what happens when in my hypothetical example the cell divides.
The red dot goes to one cell, and not the other cell.
And so, now I've got two different kinds of cells.
The regulators are in one cell and not in the other cell.
And then, I've gone through this and done it again.
When the cells go to four cells, suddenly I've magically put a blue dot in one of the cells, and there's a red dot in another cell, and there's some cells with no dots.
And those give me three different kinds of cells, or if you like, three different kinds of territories.
And for territories, you can read regulatory code, set of regulatory gene products.
So, what have we done?
What we have done in this example is to take a zygote or an egg, if you like, that is symmetric at least along one axis, and to divide it so that it's now got two daughters cells that are different from one another.
So, there is a breaking of symmetry somewhere in this process.
Now, in actual fact, if you look at the egg, it's really only symmetric in one axis.
And so, it's not uniformly symmetric.
But one of the things we think about in development is that every time you make a new cell type, you have to break symmetry.
You get two different kinds of cell from one kind of cell.
All right -- So, I'll come back to that diagram in a moment, but I want to, the fifth point I want to give you is that regulatory factors act within cells and between cells.
Really important, that's why it's up on the screen and it's on the board.
I'm going to tell you about three things.
I'm going to tell you about regulatory factors that are given the term, determinants that work in a cell autonomous way.
That means with inside the cell that carries them.
I want to tell you about regulators called inducers that act between cells or in cell-cell signaling.
And I'm going to tell you about a subset of inducers called morphogens, which work in a concentration dependent way.
So, this is a kind of magic, and it isn't a kind of magic.
The molecules I'll tell you about our molecules that you've heard about previously, OK?
The phrasing, the jargon is a little different and you need to learn it because this is jargon that's used throughout biology.
Let's first of all talk about these things called determinants.
And the notion here is that cells become different because of what they inherit.
And I've drawn for you here, and this is number 21 on your handout, I've drawn for you here a cell, this blue thing with squares in it.
The squares are determinants, and determinants are some kind of regulatory factor.
They may be one regulatory factor, maybe more than one regulatory factor.
They may be transcription factors.
They may be splicing factors.
They may be micro-RNA's.
They may be all the things we've talked about in previous lectures, but I've drawn them as squares.
And here's a cell with those little squares.
And I've drawn it so that all the squares are on one side and not the other.
And of course, you're sitting there asking, how did the squares get on one side but not the other?
And that is a really good question.
And I'm not going to tell you how to get on one side but not the other.
But there is a whole system in the cell of pulleys using things like microtubules where specific squares, specific molecules, can actually be pulled to one side of the cell or the other.
And then, you're going to ask me, well, how do they know which side of the cell to move to?
And that's another great question that I'm not going to answer.
But suffice to say there is a whole molecular machinery that can move regulatory molecules to different places in the cell.
But let's go back to our cell that's got squares on one side and not on the other.
When it divides, it gives rise to a cell that's got lots of squares, lots of determinants, and another cell that doesn't have any.
The cell with the determinants goes on to make cell type one because it has a specific set of regulatory factors.
The cell without them goes on to make another cell type.
It's not that it doesn't have any regulators; it just doesn't have the ones in the squares.
So, cells are different because of what they inherit.
Here is a fantastic example.
These are early worm embryos, early embryos of [scientific name].
We mentioned this last time that Professor Horvitz in the biology department got the Nobel Prize for it several years ago.
And this is an example of determinants moving to different cells during development.
So, the top row are cells that have been stained for their nuclei.
And you can see one cell, two cell, and 32 cell stage embryo.
These bottom pictures are florescent pictures of things called pea granules.
These are determinants.
And you can see even at the one cell stage, there are all located on one side of the cell.
At the two cell stage, they all go to one of the two cells.
And at the 32 cell stage, they are all in the cell over here.
And those cells or that cell is going to give rise to the egg and sperm of [scientific name] , and those pea granules are determinants for the germ cells.
Here's the other big thing.
Cell-cell signaling: cells may secrete an inducer.
This is a ligand.
Remember signal transduction that you talked about in cell biology II?
Those same ligands that are secreted by cells bind to receptors on target cells.
Receptor ligand interaction leads to activation of signal transduction pathways, again, from cell biology II.
And that over time can change the genes that a cell is expressing and changed the fate of the cell.
So, we call these things inducers because of the specific assays involved.
But really, they're ligands, often proteins, sometimes lipids, that bind receptors, activate signal transduction, and change cell fate.
This is number 23 on your handout.
So, induction: a process by which cells become different because their neighbors tell them to do so, and a variation of induction, oh, here's an example of induction, sea urchin embryo, the induction is everywhere.
But I've picked this one example.
Sea urchin embryo goes on through time to make this little thing called a pluteus larva.
If you remove the bottom half of the embryo, it goes on.
The rest goes on to make a kind of a round thing with lots of cilia sticking out.
And you can take four little cells that were right on the bottom of the normal embryo called micro-mirrors and stick them back on this top half of the embryo that didn't make a normal one.
And it will restore a pretty normal embryo.
And it's not that the red cells have made all the parts of the embryo that we're missing, or the parts of the larva that were missing.
It's that those red cells have sent out a signal, and that signal has told other cells what to become.
And in the case of the sea urchin, part of that signal is something called the delta protein.
This is a micrograph of the a sea urchin embryo at a very early stage, and this brown-purple group of cells here are those cells that contain the delta protein.
These are the micro-mirrors, and these are involved in inducing the rest of the embryo to become what it does.
Now, inducers, ligands, can be tricky.
They can act in different ways at different concentrations.
And this is one of the big questions of biology.
How do they do this?
So, for example, if you have a lot of an inducer, it may tell cells to become cell type one.
If you have a little bit of an inducer, it may tell cells to become cell type two.
And we'll talk about in a subsequent lecture the molecular basis for this.
So, an inducer that can induce different fates at different concentrations is terms a morphogen --
We're going to talk about the nervous, and continue to talk about the nervous system.
And in the game board of life and our journey through life we are well into the course.
We've talked about the foundations, where things come from.
And now we're talking about how different organs work together in the system module.
We'll talk about the nervous system.
Professor Jacks will move on into the immune system, probably not immediately after, but later into the immune system.
And you're going to get a broader perspective on how things work in a biology sense.
So we talked last time about the notion of the nervous system and how the nervous system is really a wiring diagram, a circuit.
And it's actually, the circuitry in the nervous system, if you were to take wires and connectors and connect it up is very simple.
What's not simple is that the circuit is incredibly complex.
Enormous, of enormous complexity.
And what's also not simple is that unlike many electrical circuits, it's a very flexible circuit.
And, unlike most electrical circuits, it learns.
It's plastic.
It learns from its mistakes and it learns from its triumphs.
And that's where the difference comes in.
We talked last time about neurons as the wires of the nervous system.
And today I want to talk about synapses, or you might hear them pronounced synapses, but this is an unusual pronunciation.
Synapses, which are the connectors between the different wires.
And then next time we'll talk about how these things are all put together into circuits.
And that will be in our lecture on Friday.
Actually, Claudette, did you want me to make an announcement about what you have on the board there?
Can you guys see this?
I can read it.
It says RO6 Lesley's 10:00 AM, R16 Kyle's 11:00 AM.
Go to 8119 tomorrow and next Thursday.
OK?
So if you are in Kyle's or Lesley's section, please note what is on the board.
OK.
So what about the number of connections?
So this is a staggering figure.
So the figure of the number of cells in our body ranges between ten to the twelfth and ten to the thirteenth.
The number of neurons that are estimated to be in the brain is about ten to the tenth.
Each neuron can make about ten to the third, a thousand synapses.
But in some neurons there can be even more than that.
There can be about, there can be up to about ten to the fifth connections between neurons.
And if you start doing some simple math you can get to a realization that the complexity of the circuitry that you can get between neurons, just within the brain, is really staggering.
OK?
And it's way beyond our understanding right now to figure out how you get, at least these ten to the third connections, ten to the thirteenth connections.
But wait.
It may be a couple of orders of magnitude more than that, how you get those set up and how you get them maintained.
So let's try to simplify the problem a bit and talk about the connections between neurons because that is a much simpler question than how you get this enormous number of circuits set up as you wire the nervous system.
This is a simple circuit that I took from your book.
A diagram of a simple circuit that I took from your book, motor neurons that are in the spinal cord innervate the leg muscle.
Sensory neurons that are in the leg muscle innervate the spinal cord, and there is connectivity between the muscle, this sensory neuron and the motor neurons in the spinal cord that give you the classic reflex when the doctor taps your knee.
I don't even know.
I guess they do that when you're little, not when you're older, unless you look like you're in bad shape.
But that is a simple reflex arch that has a number of connections between neurons but accountable number of connections, just a few kinds of synapses.
So let's talk about the synapse.
And I'm going to do some board work here and give you a sense of what the wiring in the nervous system looks like and where these connections take place.
And one needs to consider some kind of input which can be from a neuron or it can be from some kind of external stimulus like light or the food you eat.
And this input interfaces with sensory neurons, hence the name sensing.
Sensory neurons generally connect to things called interneurons which are wires that connect one part of the nervous system to another.
These connect to motor neurons.
And motor neurons are the things that direct what the output is going to be, muscle contraction, swallowing, etc.
Wherever there is a connection between neurons there is a synapse.
And often, especially in the case of muscle, there can be a synapse between motor neurons and their output.
There are two kinds of connections between neurons, two kinds of synapses.
There are those that are electrical where the electrical input from the axon of one neuron is transferred to the next neuron, and you keep the signal an electric one.
Electrical synapses are very rapid.
They involve things called GAP junctions, which we've talked about briefly, which are direct connections between cells.
And you get direct movement of ions between cells.
And electrical synapses are used at some points in the Animal Kingdom.
And they are, in fact, used in our nervous system, particularly when it's developing.
This is new work.
But the type of synapse that is much more prevalent is the chemical synapse.
Chemical synapses are relatively slow.
So electrical synapses are extremely rapid and they are almost instantaneous.
You get the action potential moving down an axon, moves across the synapse, and there is really no break in the sequence or in the timing of the movement of the signal.
Chemical synapses are slower.
And there is a lag of milliseconds to minutes, in some cases, from the receipt of the signal at the one side of the synapse to its transference to the other side of the synapse.
So mini-seconds.
That should say seconds, S-E-C, to minutes.
And the thing about them, about chemical synapses that makes them so attractive is that you can regulate them in various ways.
So I'll say regulatable.
And I'm going to spend this lecture talking about chemical synapses and not about electrical synapses.
If you're interested in knowing more about electrical synapses, email me and I'll direct you to some literature that will be of interest to you.
So let me draw on the board the essence of a synapse.
And let's draw, and you should draw it at the same time.
And so let's draw the axon of one neuron.
So this is neuron one and this is its axon.
And here's another neuron.
And this is going to be a part of the neuron termed [the 41?
.
I have a new item.
I have a new item as an incentive.
I have, since it's getting to be summer, I have a clownfish, a clownfish squirt.
You will need a bottle of water to use this item.
OK.  What part of the next neuron is the axon going to connect to?
Isn't it the dendrites?
It is indeed the dendrite.
Yes.
Thank you.
OK. OK.
So here is neuron two and the dendrite.
And what is going to happen is stuff that is really not specific to the nervous system, except in the details.
It is stuff that you've heard about in cell biology, that we've talked about in the formation module, and it involves receptor ligand interactions and signal transduction but in a specialized way.
Here comes the action potential, hereafter abbreviated AP, coming down neuron one.
And that action potential, as you remember, is driven by transient depolarization and inward sodium current.
At the very terminal of this axon, which is called the presynaptic terminal or the presynaptic cell, there is a system of vesicles.
And these vesicles contain stuff called neurotransmitter, which we'll talk more about in a moment.
And on the opposite cell, which has the name of the postsynaptic cell, on its membrane are things that we've talked about many times, receptors.
And this is the deal.
The action potential moves along the axon of the presynaptic cell.
As it gets towards the terminus of the presynaptic cell, you can call that the presynaptic terminus also, it activates another set of channels.
These are calcium channels that give an inward calcium flow, an inward calcium ion flux that causes these vesicles to fuse with the presynaptic membrane in the process that you learned about in cell biology called exocytosis.
And these presynaptic vesicles disgorge their contents, their neurotransmitter into the space between the presynaptic cell and the postsynaptic cell.
This space is called the synaptic cleft or the presynaptic cleft.
It doesn't matter.
Well, it does matter.
You should know that, but it's just a space.
And it's a space that is usually about ten to twenty nanometers.
So it's a fairly substantial space.
And that neurotransmitter diffuses across the space, binds to receptors and signal transduction happens.
And we'll talk about what the signal transduction is and what the outcome may be.
And the outcome may be an action potential, but it is not guaranteed to be an action potential.
And the thing about chemical synapses that makes them slow is that there is a diffusion driven process.
The stuff has to, these neurotransmitter molecules have to diffuse across the synaptic cleft, and that takes time.
And that is why chemical synapses is slow.
You also have to get exocytosis of those vesicles, and that is slow as well.
All right.
So synaptic density.
Here's a slide to show you synaptic density.
Increase in synaptic density during development is really extraordinary.
And the number of synapses and the type of synapses changes in childhood but also throughout life.
This is a movie to illustrate what I've told you.
You can look at it again.
And graphically here's your electrical message being transferred into a chemical message, and then recreating an electrical message.
So there's change in the transduction mechanism of this particular, oh, it's supposed to be a loop but I guess it's not, in this particular mechanism of transmission.
Here are the presynaptic vesicles disgorging their neurotransmitter.
Look in this case, the neurotransmitter is going back into the vesicles.
And we'll talk about this more in a moment.
OK.
Item one on your handout today is a diagram from your book, which shows you the synapse, the presynaptic cell, the postsynaptic cell.
You'll get all the information that I have put on the board in a more complicated way.
And this is done for a cell that has got something called acetyl-CoA in it.
OK.
So you can bear that in mind.
I want to address a number of issues now, a number of issues now.
And I want to point out a diagram, actually, let me go back to this one, number one on your handout.
One thing I would like you to look at and to think about and to compare with your cell biology module is the exocytosis of these vesicles.
So you talked about transport of molecules around the cell, particularly proteins, and you talked some about exocytosis and endocytosis.
The exocytosis of these neurotransmitter vesicles is a classic example of exocytosis.
The proteins involved in this are the same proteins involved in exocytosis in many different cell types.
In addition, there's a process I'll mention later on of endocytosis where the neurotransmitter may be taken back up into the cell and used again.
On your handout that's on the Web is this diagram.
You can look at it in more detail.
I'm not going to dwell on it here.
You can look at it in more detail.
It will indicate where calcium comes into the triggered exocytosis of the neuro transmitter.
But moving right on, I want to talk to you now about neurotransmitters.
Neurotransmitters, the ligands that transfer a signal from one neuron to another.
Here are some facts.
Any kind of neuron can make more than one neurotransmitter.
And different neurotransmitters are usually in different vesicles.
And they can be used at different times and different places in the neuron where it synapses onto another neuron.
Neurotransmitters are typically three things, and you'll be surprised at what some of them are.
It's very interesting.
They can be nucleosides, they can be amino acids, and they can be peptides.
Let that on the table here.
All right.
So here are some neurotransmitters.
Most of these, you can go and look at these later.
Most of these come from amino acids.
Acetylcholine which is one of the largest most important neurotransmitters does not, but the other ones do.
They come from glycine from glutamate, tyrosine, tryptophan, and so on.
And that monosodium glutamate that you have often on food, on Asian food that makes the food taste better, the reason it makes the food taste better is that it's a neurotransmitter and it increases synaptic transmission across some of the neurons that are involved and taste perception.
So, yeah.
All right, so neurotransmitters.
Peptides, amino acids and nucleosides.
It's very interesting that these ligands are very small.
Amino acids are very small.
Nucleosides are very small.
And it's interesting that these are used as neurotransmitters.
And I think there are two reasons.
One of the reasons is that because they're small they diffuse relatively rapidly, and so they minimize the time of getting the synapse in the postsynaptic cell activated.
And also because nerve cells are evolutionarily very ancient.
These were the molecules that were around at the time, and perhaps they were around at a time where it was easiest to use things that were preexisting.
So in some ancestral cell, which we don't really understand, perhaps before there was a very large protein cohort and cells wanted to communicate with one another, a very ancestral type cell.
These were the types of things that would have been around.
And these, perhaps there's an evolutionary reason that these things are used as neurotransmitters.
There are a vast number of neurotransmitters.
There are about 25 known neurotransmitters, and they have different functions.
So the two big classes you should know about are the ones that are excitatory and the ones that are inhibitory.
And we'll talk about this in detail in a moment, but excitatory neurotransmitters are ones which will promote an action potential in the postsynaptic cell.
And, in fact, the major one in the central nervous system is glutamate.
And the major one that works between your nerve cells and your muscles, at the neuromuscular junction, is acetylcholine.
And then there are inhibitory neurotransmitters, and these are the ones which will tend to inhibit an action potential from taking place in the postsynaptic cell.
And one of the major ones there is gamma-aminobutyric acid, abbreviated GABA.
And another one is glycine.
And those are both found in the central nervous system.
However, I'm going to also add to this inhibitory list acetylcholine, which will indicate an important principle you should understand.
And you'll see it gets more and more complicated, that neurotransmitters can either be excitatory or they can be inhibitory depending on the receptors to which they bind.
This is a table from your book.
You should look at it.
You do not need to memorize what each of these neurotransmitters does, but you should become familiar with their names.
And you will be interested to read what some of them do.
And I'll come back to a couple of them later.
I'll come back to acetylcholine, already mentioned.
Serotonin which is involved in promoting good mood.
And I'll also talk about adenosine as something that you affect every morning when you drink a cup of coffee.
But before we get there, let's talk about receptors.
And let me introduce you to two classes of receptors that you should be familiar with.
So you could imagine, as we've been talking about, that the receptors for these neurotransmitters could directly influence an action potential in the next cell by allowing ions in or out of the cell.
And, in fact, I'm going to add it to your first diagram.
A lot of the receptors have eventually the effect of allowing sodium to flow in.
But these receptors also may allow the influx of calcium or they may allow the efflux, the removal of chloride.
And we'll talk more about this and I've got diagrams for you in a moment.
So the receptors in general have got something to do with ion channels.
And they can either be ion channels themselves directly.
And in this case they get the name ionotropic receptors or they can be indirect and they can influence the activity of ion channels in an indirect way.
And in this case they are called metabotropic.
Metabotropic.
OK.
So the end result is that ion channel activity is changed.
Ionotropic receptors, because they are the ion channels respond to neurotransmitters very rapidly.
Within the millisecond timeframe, metabotropic receptors are much slower and they respond much more slowly within the second to minute range.
And the deal with metabotropic receptors, I'll talk about both of them with pictures in a moment.
The deal with metabotropic receptors is that between the receptor ligand interaction and activation of the ion channel, so the ligand will bind a metabotropic receptor.
And the metabotropic receptor will then very often activate the system of G-proteins that you've talked about previously.
And those G-proteins will then activate downstream ion channels.
Here's an example of the acetylcholine receptor, which was the first receptor, neurotransmitter receptor to be purified.
This is an ionotropic receptor.
It's a complex of multiple subunits.
And it works in a way that we talked about long ago when we talked about enzymes.
And we talked about allosteric activation of enzymes.
Remember that way back when?
Something binds to a protein and changes its confirmation and changes the activity of the protein.
That's exactly what happens here.
Acetylcholine binds to the receptor in defined places, it changes the confirmation of various subunits, and going from a closed channel it now opens this channel.
So this is a gated channel.
OK?
It's a gated channel in the same way that the sodium channel we talked about was gated, but it's gated not by voltage but by neurotransmitter binding.
All right?
That is the definition of gated.
It can be gated by anything.
So this is the acetylcholine receptor.
It's also called the nicotinic receptor because it is the thing that binds nicotine, and can be activated by binding of nicotine.
Metabotropic receptors activate ion channels.
Here is something from your book.
And I have these on your handout.
So these are number two and three on your handout.
Here is the neurotransmitter binding a receptor, interacting with various G-proteins, hydrolysis of GTP.
One of the subunits goes off, and through a series of events opens up another ion channel that is not linked physically to the receptor.
Metabotropic receptors.
Now, interestingly, and here is another note you can make, a neurotransmitter can either activate an ionotropic or a metabotropic or both types of receptors.
Acetylcholine is ionotropic.
It can also be metabotropic.
In hot muscle there are these things called muscarinic acetylcholine receptors, and they are responsible for slowing down or speeding up the contraction of the heart muscle.
So there is a lesson there.
Receptors can be either or both ionotropic or metabotropic.
So up on the top board I've mentioned the terms excitatory and inhibitory.
Let's go back to them and let's talk about excitatory versus inhibitory synapses.
So the deal is this.
You've got your signal coming along, you've got your next cell, or you've got your signal coming along, you've got your next cell here, you're passing the information to the next cell.
And the question that that next cell is trying to answer is should I fire an action potential?
That is the only question it is capable of answering.
That is the only output that has real relevance for a cell, OK, is whether or not it's going to fire an action potential and transmit the nervous information along the pathway of the nervous system.
And in order to decide that there are two things that can happen.
Neurotransmitter receptor interaction can promote, can facilitate, can make easier an action potential to happen in the postsynaptic cell -- -- or can inhibit an action potential from occurring in the postsynaptic cell.
So let's go back to our discussion from last week where we talked about resting potential, threshold potential and so on.
And let me give you some information.
The threshold potential is constant.
For one fish, could somebody please define for me the threshold potential somewhere up here?
Threshold potential.
Yeah.
Yes, you're correct.
The threshold potential is around negative 50 millivolts where outside is more positive than inside.
What is this threshold potential?
What happens when you cross it?
Yeah?
And action potential is created.
Right.
Good.
OK.
So one first for you.
And see me for another fish later on.
You came up here with one.
OK.
The threshold potential, the point of no return at which the action potential is going to take place is constant from cell to cell.
It doesn't change.
What can change is the resting potential.
And excitatory and inhibitory synapses act upon the resting potential and bring the resting potential closer or further away from the threshold potential.
So excitatory synapses and the interaction of neurotransmitters with the receptors inhibit, OK, excitatory synapses will bring the resting potential closer to the threshold potential.
They'll make it easier for an action potential to take place.
So they increase the resting potential.
RP for resting potential.
Inhibitory synapses decrease the resting potential and make it harder to get to the threshold potential.
OK?
So you increase the resting potential and you still get closer to TP or the threshold potential.
And you here you get further from the threshold potential.
And the way you do this is by acting on specific channels directly or indirectly.
If you're going to increase the resting potential so that you get closer to the threshold potential the channels involved are sodium and calcium.
And those two ions will flow into the neuron.
If you are going to decrease the resting potential, make it harder to get to that threshold potential, you want to do the opposite.
So you could take sodium or calcium out, but that's not how the cell usually does it.
The way the cell does it is to open up chloride channels.
There's a lot of chloride.
I'll show you some diagrams in a moment.
There's a lot of chloride on the outside of the axon.
It opens chloride channels and the chloride will flow into the cell.
All right.
Let's look at some pictures and some of the things I've drawn for you.
So this is a diagram from last lecture which I drew for you.
It shows you the action potential and it shows you the transient depolarization and reversal of charge distribution across the membrane.
As a couple of you pointed out to me this is not accurate.
Of course it's not accurate.
It's a cartoon.
And it's not accurate specifically because things aren't all or none.
It's not entirely positive outside and entirely negative inside.
And there are a number of different ions involved.
So I have had a go at making it slightly more accurate.
And this is number four on your handout.
And now I've drawn it in terms of synapses.
It's a complicated diagram but we can work through it.
Outside the cell or in the synaptic cleft.
And here's your postsynaptic cell.
Again, this line as previously is the membrane.
So here is your ion distribution.
Outside the cell there is high sodium.
Inside the cell low sodium shown in red.
Inside the cell high potassium.
Outside relatively low potassium.
And you remember that that potassium-sodium gradient is constantly being generated by the sodium-potassium pump, and then potassium is flowing outwards according to its gradient through open potassium channels.
That is going on all the time.
I haven't indicated that, but that's going on all the time.
In addition, outside the cell are high levels of calcium in brown and high levels of chloride ion shown in gray.
Inside the cell there are negative charges.
A lot of them are on proteins rather than free ions.
OK.  And I've shown you your resting potential is around minus 60 millivolts and your threshold potential around minus 50.
So let's look what happens at an excitatory synapse.
And the thing that you want to look for are channels that will hypopolarize, that will make the potential difference lower so that you get closer to that threshold potential.
And the way you do that is to open up a series of gated sodium channels.
Those same voltage gated sodium channels which we previously talked about.
But I should tell you that there are dozens of different channels that are gated.
And different cells have different types of gated channels but the idea is the same.
These are voltage-gated sodium channels, and they are going to allow an inward flow of positive ions.
In addition, there are gated calcium channels which will allow an inward flow of calcium.
So this is an excitatory synapse.
And you will get here a hypopolarization.
Conversely, on your next handout and number six, at an inhibitory synapse channels that hypopolarize, that make you more negative will open.
And these particularly are chloride channels that will give you an inward flow of chloride ion and make the inside of the postsynaptic cell even more negative and move it even further away from threshold potential.
This is a diagram from your book I'm not going to dwell on.
And I want to point out here again that I mentioned up here acetylcholine can either be excitatory or inhibitory.
At the neuromuscular junction acetylcholine is an excitatory neurotransmitter.
Here it is increasing or making less negative the membrane potential.
So it is pushing you closer towards threshold.
And here in the heart muscle it is acting as an inhibitory neurotransmitter making the membrane potential, the resting potential much lower.
OK.
So I want to move on now to something called summation.
So we're building up the sense of whether a neuron, once it makes the synapse is going to fire an action potential.
And that depends on this excitatory versus inhibitory stuff.
But there's anther wrinkle, and the other wrinkle is the one that I alluded to and I've told you on the first board neurons make lots of synapses.
They're not just making one synapse.
They're making lots of synapses.
And the input from many synapses needs to be added up, needs to be summated in order that the neuron decides whether it's going to fire or not.
And that summation can be two-fold.
It can be spatial in that a number of different neurons, a number of different synapses are made with the same neuron.
And the sum of all the changes in the membrane potential are added up.
And if you get above the threshold potential then you fire an action potential.
Or they can be temporal where over a period of millisecond all the inputs into a neuron are added up.
And if at the end of that time period you are above threshold potential then an action potential fires.
This is number seven on your handout.
Here is an axon.
Actually, here is a neuron.
Here are a bunch of synapses.
They've said that they're excitatory synapses coming into the axon.
The synapses can actually occur at many different places.
In an axon they've shown them here coming into a part of the cell body.
And there's this thing called the axon hillock which has got a particular configuration of channels.
And that's the place where the cell often makes the decision as to whether it is going to fire an action potential or not.
But it is misleading to see these synapses coming in all together because they can come in at multiple different places on the neuron.
OK.
So here are synapses coming in.
And you sum up the input from these.
This is the next diagram on your handout.
So here in a graphic way are a number of small changes in membrane potential from a number of different synapses indicated by the numbers here.
And these are summated at the axon hillock.
And then the decision as to whether or not you're above threshold and whether you make an action potential is made.
You can also do this over a period of time, again, on the millisecond time scale, and that will tell you whether an action potential fires or does not fire.
OK.
So the thing you need to remember is that the action potential is the output, and the decision is whether you make an action potential or not.
So the decision to make an action potential.
And the other thing that I should note is your action potential is your action potential.
It has the same magnitude, and it's propagated once it starts.
What you can change is the frequency with which you make that action potential.
So you can have action potentials arising at slow rate or you can have them arising very quickly.
So the decision is to make the action potential, it's all or none.
I mentioned this last time.
But you can change the frequency at which action potentials fire.
All right.
Controlling neurotransmitter activity.
Neurotransmitters, here's acetylcholine and here is atropine, which is an inhibitor of acetylcholine receptor activity.
Acetylcholine is an example of a neurotransmitter that is made and then destroyed.
There it is being made, being used for receptor binding and for doing whatever it's going to do in the postsynaptic cell.
And here is an enzyme called acetylcholinesterase that cleaves the acetylcholine and prevents it from being used again.
Acetylcholinesterase is essential to acetylcholine function.
Nerve gases, very typically, inhibit acetylcholinesterase.
That is a very popular way that nerve gases work.
And when they do so, what happens is that you have an excess of acetylcholine around and you get occupation of the acetylcholine receptors on the postsynaptic membrane that makes that postsynaptic cell fire repeatedly or fire without stopping.
And that will inhibit your ability to breathe and not breathe and breathe and not breathe.
And you will not be able to get contraction and relaxation and contraction and relaxation of the muscles that allow you to breathe.
And nerve gases are generally toxic for that reason.
They paralyze the muscles.
Our troops in Iraq and elsewhere have in their pockets atropine syringes.
Syringes filled with a substance called atropine.
Atropine is a nerve gas antidote.
And the way it works is in the following way.
It acts by blocking.
It's a competitive inhibitor of the acetylcholine receptor.
And so even in the presence of nerve gas, the acetylcholine that is free and floating around in the synaptic cleft will not be able to do its thing because atropine is preventing it from doing so.
Now, it's interesting.
I've told you acetylcholine does many things through both ionotropic and metabotropic receptors.
And atropine will affect all of these things.
OK?
It will affect the ionotropic receptors particularly, but what is has the side effect of doing is making you very sleepy.
Nonetheless, it will save your life.
OK.  Serotonin, another neurotransmitter involved in mood.
It makes you feel good about things.
Here are two examples of ways that serotonin activity is regulated.
Prozac is a class of molecules called a specific serotonin reuptake inhibitor.
Serotonin, when it is made, is released and then is retaken up into the vesicles and used again.
Prozac slows down this reuptake and allows the serotonin to act for a more prolonged period than it normally would have.
And here is another one that you're very familiar with, caffeine.
Caffeine is a competitive inhibitor of the neurotransmitter adenosine.
Here is adenosine and here is caffeine, and you can see that they both have this purine ring.
Caffeine will bind to the adenosine receptor.
Adenosine normally promotes signal transduction pathways that make you sleepy.
And adenosine can be counteracted by caffeine.
You can go and look later at the pathway that caffeine and adenosine both act upon.
And I will talk more about this next time.
All right.
So let me move onto the last lecture in this module.
Not in the module.
In the last lecture in the nervous system part of the systems module.
And I want to start by talking briefly about learning and memory, very briefly about learning and memory.
We talked last time about setting up synapses, we talked about making connections between the wires, and we talked about how changes in membrane potential could either determine whether a synapse fired or whether it did not.
However, there's more to it than that.
There is significant data that indicates that the more often you use a synapse the strong that synapse will become.
What do I mean by stronger?
Well, I'm actually not sure what I mean by stronger, except in a couple of cases that I'll tell you about.
So the term I want to, so synaptic connections, this is going to be long.
Synaptic connections strengthen with use.
And the phenomenon I want to tell you about is long-term potentiation or long-term depression.
And this is a phenomenon that was discovered in neurons that were studied in isolation.
So it's very difficult, it's impossible to study neurons within the brain of any complex animal, especially mammals.
And so what's done is to use so-called slice cultures.
Where you take a slice out of the brain, a very thin slice that's just got a few layers of cells, and you look and see what these neurons do in some kind of plastic tissue culture dish with media.
And they actually do quite a bit.
The neurons fire and they make synapses.
And a lot has been learned with them, with these kind of slice cultures.
So here's the deal.
If one has a cell, if you look at this picture here, if one has a cell and one stimulates the synapse you see that there is a response to the first stimulus.
And you're actually measuring change in potential in the cell.
Let's not worry about how the experiment was done because we're not going to get through the material I want to cover today.
You can come see me in office hours and I'll explain to you.
What I want to point out is that after a first stimulus there is a change in the potential of the responding cell.
If you continue to give that stimulus, you get a much stronger response later on.
So the strength of the response, the change in membrane potential in a responding cell increases with stimulation of that presynaptic cell, of that synapse.
And that is what long-term potentiation is.
It increases the chances that the postsynaptic cell will fire.
And it is believed that long-term potentiation has got a profound role in memory and in learning.
So how does that actually work?
Well, it's not clear in most cases, but this is an example from your book that is the shining example of how learning and memory can work at the molecular level.
And it has to do with glutamate.
You do not have this on today's handout.
It was on your last lecture's in-class handout.
But you can look up here and then go back to your last lecture's handout if you don't have it with you.
It has to do with glutamate.
So glutamate is the major excitatory neurotransmitter in the central nervous system.
And glutamate activates two different ionotropic receptors.
One of them is called the AMPA or the non-NMDA receptor and the other is called the NMDA receptor.
With low level of stimulation or with single stimulation of a presynaptic neuron the AMPA receptors are activated.
And these are sodium channels, so these are ionotropic channels.
And glutamate is released, binds to the AMPA ion channel, changes its confirmation, and you get an influx of sodium, and therefore get closer to threshold potential.
However, on the same synapse, like right next door are these other receptors called the NMDA receptors.
And they are not used if the presynaptic cell is stimulated at a low level or infrequently.
But if you keep stimulating that presynaptic cell, eventually these NMDA receptors will open.
And they will open and they will allow an influx of both calcium and sodium.
And what happens is that you get a much stronger change in potential because you've got now two sets of receptors open.
And this postsynaptic cell is much more likely to fire and is actually much more stable in the synapses that it has made.
So this NMDA channel only open after repeated stimulation.
So that's an example of what repeated stimulation can do to changing channels.
And there is some molecular data, although it's really not complete, as to why you switch this channel usage.
One thing, however, that's very interesting about long-term potentiation, and indeed about learning and memory anyway, is that you need -- -- protein synthesis for learning especially.
For classical learning, the kind of learning that you do before a test, the kind of learning that a mouse does to find its way through a maze repeatedly requires protein synthesis.
And this is very interesting because what I've been telling you about in this module is stuff that happens in the neuron that is actually independent of transcription or protein synthesis.
It involves changes in gene activity.
It can involve changes in gene regulation, as I'll talk about in a moment, but it does not actually require any transcription or protein synthesis per se.
Learning and memory of the classical kind that we think about does.
And it's one of the reasons that you need to get a few hours sleep before you take a test.
It's one of the reasons if you pull an all-nighter, and there are many, you do not do that well the next day because you need this round of protein synthesis that does stuff that probably synthesizes new receptors that allows the synaptic connections you've made through your learning to be strengthened.
And that's still a mystery, really, as to what the protein synthesis is required for.
But if you're interested, this work from [Ed Plezia?]
that I mentioned at the beginning of class has shed some really fascinating light on it.
So come see me if you want to.
All right.
So I want to move on now to circuitry.
All right.
I'll show you this slide because it's up there.
This is a piece of data that I pulled down that's kind of interesting.
This looks at the AMPA NMDA ratio, receptor ratio with cocaine.
So in a control there are about the same number of AMPA and NMDA receptors on a particular synapse.
It's very interesting in some studies with cocaine use, you see the amount of the AMPA receptor.
That is the receptor that is the short-term receptor increases relative to the NMDA ratio, an NMDA receptor level, indicating that with cocaine you might not be as good at learning as without cocaine.
It is not clear.
There are a lot of studies like this and the interpretations of them are complicated, but this is a solid piece of data that you do change the ratio of the non-NMDA and the NMDA receptors.
All right.
So let's move on.
So let's talk about circuitry.
We've talked about the wires, the passage of current along the wires, we've talked about the connections between the wires, and now I want to talk about the circuits.
And I want to start by asking you a question.
In thinking about your brain and how much of your brain you used in circuitry, how many of you were taught that you use one-tenth of your brain?
Raise your hands.
I want to see what you've been taught at schools.
It's no reflection on your teachers.
It's OK.
Thank you.
It's a substantial portion.
I feel that it is my duty as your professor to debunk that myth once and for all forever.
It's not true.
And you might want to actually email your high school teachers and tell them it's not true.
OK. And you perhaps even learned about the boy who used, you know, one-fifth of his brain and was an unbelievable genius.
There is, none of this, I was taught that as well, and it is absolutely untrue.
And the data for it is very solid.
For example, any neurosurgeon who goes into your brain knows that no matter what part of the brain he or she goes through to remove your brain tumor or do whatever it is, is going to damage some aspect of your brain function.
Anyone who has any kind of stroke, any kind of brain hemorrhage, any kind of leakage of blood into the brain is going to have some kind of deficit.
It might be a small one or not.
But there is ample evidence that you use all your brain.
And, indeed, why would one go to the trouble of making a brain that is so large if one was not using it?
So there we go.
Let me talk a little bit about circuitry.
To indicate the complexity of circuitry in your brain let me point out what parts of your brain it takes to put together language.
If you're looking at words, it takes this area right at the back of your brain, the base of your brain.
Listening to words takes a different region.
Speaking words takes a different region.
Generating words take different regions.
And these are very, these are PET scans.
We can talk about what they are.
They measure the energy usage in your brain.
And these are very, very large parts of your brain.
OK?
These are millions, close to billions of neurons involved in each part of working with language.
OK?
And you can go to your brain and you can actually, people have mapped these quite finely.
So the circuitry in your brain is enormous, and the effort to both set up and maintain the circuitry is enormous.
Here is another interesting piece of information.
About 20% to 25% of all the energy that you use is used by your brain, which is one of the reasons you probably need to sleep.
It's not clear why we need to sleep.
But it is a tremendous, there takes a tremendous amount of effort to get the wiring set up and to keep the wiring set up and to keep the circuitry running.
Here's another example of circuitry that I'm going to come back to.
It's called a retinotopic map.
And I'm going to start talking about it by addressing the question of how the circuitry in your nervous system is set up in the first place.
And I'm going to use this as an example throughout the rest of the class.
So when people realized way back when that there were circuits set up in the nervous system, the question was how did these circuits get set up?
How did neurons know which connections to make?
And there were two competing theories.
One I'm going to call the random/survival theory.
They've got lots of names in the literature, but I call it the random/survival theory.
And the notion here was that neurons grow all over the place, and if they make connections that work, if they're productive connections, those connections will be maintained, the neurons will survive, and there you go.
You have your circuit.
The other theory is the guidance theory, that there's actually something telling the neurons where to go.
And I'll show you a piece of data that suggested that this guidance theory was really correct.
But today, in fact, both of these theories seem to be correct when put together in a way I'll try to build up for you.
So your eye, your eye and your visual system are very complex.
Your eye starts off sort of like a camera, but very rapidly that analogy breaks down.
And light falls onto your retina and the photoreceptors in your retina, and from your retina a series of axons are bundled together to make the optic nerve, which goes to your brain.
And, in fact, I should point out, I'm not going to talk more about this, but the image that your retina sees we know quite clearly, a Nobel Prize was given for it, is not the image that you see when you look at me or you look around the room.
It's a series of shadows and bright points and so on.
It's really not a direct camera image as the camera does it for us.
But that is no matter now.
What is important is that some set of stimuli go from your retina through the optic nerve back to your brain.
And they go to two places in your brain.
So here are some diagrams.
Let's look at the very left most one.
This isn't on your handout but look at it up here.
I've got an equivalent one later on.
So in your eye, from your eye the axons from the eye are sent back, project back to a region in your brain called the tectum.
It's got a lot of different names, lateral geniculate nucleus or the thalamus or the tectum.
OK. We'll call it the, I think I'm calling it the tectum in this lecture.
Let me see.
Let me make sure I've got it correct.
Yeah, I'm going to call it the tectum.
OK?
So we'll talk about the tectum.
And from the tectum connect synapses are made, another set of neurons go back to another region of your brain called the visual cortex.
And it's the summation of all these different connections that somehow allows you to interpret what you see and see it.
It's extremely complicated.
What I want to concentrate on are the connections between the retina and the tectum.
And what's interesting in ourselves and any animals that see binocularly, that can see 3-dimensionally, is that the reason you can see in 3D is that some of the neurons that come from your right eye go to the right side of the brain and some of them go to the left side of the brain and vise versa.
Some of the neurons on the left side of your brain go to the left side, yeah, and some of them go to the right side.
And so if you look in some, just look at the diagram so you're with me.
If you look in some, you'll see that some of the neurons go to the same side and some cross over and go to the other side.
OK.
So you can look at this in a different way.
This very beautifully labeling experiment where you can actually label neurons in different parts of the retina, and you can follow where they go.
The label is maintained in their axons.
And as the axons, well, I'll talk about in a moment.
As the axons send out their processes back into, oh, I've called it the thalamus here, back into the thalamus then you can see those same axons.
So let's look here.
This is the left eye and the right eye.
What?
The left eye and the right eye.
OK. And let's look here at the images that you're going to get here on the left side, the connections you're going to get on the left side of your brain that will give you binocular vision in your right visual field.
Here are a set of neurons.
They're called the nasal neurons because they're on the side of your retina that's next to your nose.
And these red neurons are going to send their axons back into the left side of the brain.
Here they are.
And the same time the neurons from the left eye, from the side of your eye nearest the outside called the temporal, are going to send their axons back to the left side or to a different side of this thalamus or this tectum.
OK?
So what I want you to see is red can go to one side.
Excuse me.
The red here cross over when they go to that side and the green cross over and go to that side.
So let me diagram this more clearly so that we can actually discuss this with the diagram in front of you.
This is number one on your handout.
OK?
So let's look at this and indicate it more clearly.
These N normally in setting up binocular vision, this circle here is meant to be one eye, and this back here is meant to be one side of the tectum.
Let me write on the board that in retinal tactile mapping the tectum can also be called the thalamus.
All right.
So what's important here is that these N neurons go back to this region of the tectum called the C. OK?
And I see I have not wrote, I have not indicated tectum here or retina.
I'll go back and do it.
But you should write here, the circle here is the retina and this oval here is the tectum.
And then we will be together.
So these N neurons from the retina send out axons that get to this C or caudal region of the tectum.
And these T neurons or temporal neurons from the retina send out axons that get to this R or rostral region of the tectum.
So this is a long preamble to tell you two things.
Firstly, these connections are what you always see in every animal with either binocular or monocular vision.
Now, the way this is set up is actually quite confusing.
And I want to, let's not worry about the details too much, but let's look at, as I see it I realize it's quite confusing.
I took it from a very old paper, which I thought would be of interest, but as I see it here there's one point of confusion.
But what I want to point out is an experiment that was done a long time ago by a guy called Roger Sperry which addressed this question of whether axons were guided in a particular way or whether or not they randomly found their way.
And then by some kind of survival cue the connections in the nervous system were set up.
So what Sperry did was to take this tectum and he rotated it 180 degrees.
So the C is now 180 degrees reversed.
And if he did that he had to break the axons that connected the normal retina and the tectum.
OK?
But what happened was that those axons grew back.
This was done in frogs and they grew back.
And there are two possible outcomes to this experiment.
The first is that these nasal, don't you love it?
Thank you so much.
OK.
The first is that these nasal neurons will grow back to this R region, which they would not normally do, but it's far away from them so maybe that's how they work.
The second is that these nasal neurons will grow back to the C region, N to C in the same way that they would in the normal case, even though they're closer together.
And the outcome of the experiment was unequivocal and it was fantastic and it was one of the reasons he got a Nobel Prize, which was that you always get these nasal neurons growing back to the C region and these temporal neurons growing back to the R region.
All right?
So this is a very long preamble for a very complex axon writing system, but the bottom line here that I want you to have is that these retinal neurons knew which part of the brain they had to project back to.
And that was very strong evidence that there was guidance of these neurons to the correct region of the brain.
All right.
So let's move on to how this guidance takes place.
And I'm going to come back at the end to talk about survival because that's also an important part of this.
And the part of the cell that you need to know is the growth cone.
Now, some lectures ago I gave you a lecture about 3-dimensional structure.
And we talked in some detail about cell migration and how cells move in a mechanistic sense.
What we're going to talk about now is very related to that, but instead of talking about a whole cell moving we're going to talk about just part of a cell moving.
We're going to talk about this growth cone.
So what is the growth cone?
The growth cone is the axon tip.
It has both sensory function because it's got various receptors of the type we've talked about, more in a moment, and it also has motor function because it has all those actin-based microfilaments that allow it to move in exactly the same way that we saw the whole cell moving.
But you remember, of course, that axons are long and they're far away from the cell body, so we're just talking about the axon growing now.
This is number two on your handout, and this is a diagram of the end of an axon in the growth cone.
The axon's structure, the length of the axon is stabilized by microtubules.
And subsequent to the ending of where the microtubules are, the cell sends out a series of processes that are called filopodia or lamellipodia.
I've given you the singular here.
Filopodia are thin.
Lamellipodia are fatter.
It doesn't matter.
And it's in these filopodia and lamellipodia that there are microfilaments that, you should know, are comprised of actin polymers.
And I've shown you also receptors on this growth cone that can bind two molecules in the extracellular matrix or can bind two molecules on another cell.
OK.
This is what a growth cone really looks like.
This is a time-lapse of minutes.
And you can see that this is a very dynamic structure.
The cell is sending out processes and retracting them, sending them out and retracting them.
And the sense that one has when one watches axons sending out their processes is that they really are feeling around their environment and trying to decide where they should go.
I want to remind you this is a diagram from one of your previous lectures from Formation 4.
The processes by which the axon is moving and sending out its processes are exactly the same as the processes that change during mesenchymal cell migration, but we're just talking about sending out part of the membrane and not the whole cell moving.
All right.
So what is important in this process?
Well, what seems to be important are several things.
OK. And one set of things that are important are called short-range guidance cues.
Short-range refers to how far they can work.
Whether these are diffusible molecules or not.
And I'll talk about long-range ones in a moment.
The distinction between them is a little semantic.
But short-range guidance cues include things in the extracellular matrix and include things that are irrevocably stuck onto cells so that you can really only get interactions directly between two adjacent cells or between a cell and something that it is directly just opposed to.
And I want to talk about two classes of cues.
Oh, and them I'm going to indicate later on long-range guidance cues.
So let me first talk about cell adhesion molecules.
We've talked about the extracellular matrix a bunch.
Laminin, in particular, is a very important molecule for guiding axons through their paths to where they are going.
This is a movie showing an axon.
This ought to be a movie showing an axon trying to decide what substrate it wants to migrate on.
Here is an axon.
There it's sending out its growth cone.
And this axon has been put onto a dish that's got two different kinds of coatings.
On the top half it's got laminin and on the bottom half it's got polylysine.
And if you look at this you can see that it's sending out processes that are touching the polylysine, but as it goes on it decides it's really not interested in interacting with the polylysine and it's interested in interacting with the laminin.
And that's where it lands up.
Other things that you should be aware of that are important in axon guidance in terms of cell adhesion are the molecules cadherins that we talked about previously.
Now, some of these are attractive.
In other words, axons will like to grow on a particular substrate with particular cell adhesive, with particular cell adhesive or cell interactions.
So attractive cues will promote axon outgrowth.
And one can counter this with repulsive cues or repulsive signals which will inhibit axon outgrowth.
And this is going to be true both for the cell adhesion molecules I'll talk to you about and for other molecules later on.
So let's move onto diffusible molecules or the longer range molecules.
OK. And I want to give you, to illustrate this, an example of a very beautiful axon guidance phenomenon.
And so we're on number three on your handout.
And this is the example I want to discuss with you.
It has to do with two sets of neurons in the spinal cord and the brain that are called commissural neurons and trochlear neurons.
This diagram is a section, it's a cartoon of a section through a spinal cord.
Spinal cord transverse section.
If you lay on the ground and someone cuts you in half, that way that is a transverse section.
OK?
So here's the hole in the middle of the spinal cord, this is the roof plate of the spinal cord nearest the skin, and something called the floor plate which is deepest in your body.
The commissural axons grow from this dorsal region down towards the floor plate.
In contrast, these trochlear axons that I've shown in blue here grow from the floor plate away towards the roof plate.
And this data is shown very beautifully by staining these cells with appropriate markers here.
And I apologize if you're red-green color blind.
I know this is an issue, but this is how the stains are actually done.
These are the real colors of the stains.
So up here in the top of the spinal cord, these red dots are the cell bodies of neurons.
And the green are the axons that are sending out their processes.
And these are the commissural axons and they are growing down, down, down towards the floor plate.
And actually eventually will cross to the other side of the embryo.
So how do these axons know where to go?
Well, one model, one model suggests that these axons are being told where to go by this little region at the bottom called the floor plate.
And this was tested in the following way.
One can go into a developing spinal cord, and one can cut out different pieces of it.
So you can cut out a piece of the dorsal spinal cord and you can stick it close to a piece of the floor plate or you can stick it close to a piece of the roof plate.
And then you can look and see what happens.
And what happens with time is that neurons will develop in these pieces of dorsal spinal cord.
In the case of a set of tissues that's got the dorsal spinal cord and the floor plate, you can see very clearly that axons will grow out towards the floor plate.
However, when you have the roof plate there, the axons do not grow out from the spinal cord.
There's no outgrowth.
And this type of experiment indicated that the floor plate was attracting these commissural axons to it, and they were growing out because of this attraction.
And here's what a real experiment looks like.
Here's a piece of dorsal spinal cord, here's a piece of floor plate, and here in the control experiment there was dorsal spinal cord.
And this is just a chunk of tissue culture cells instead of the roof plate.
And what you see here are these streaks coming out from the dorsal spinal cord explants towards the floor plate.
And these streaks are the axons growing out.
So this is actually a very robust assay.
And so using this assay, some years ago Marc Tessier-Lavigne's lab tried to find what the floor plate was secreting to attract these commissural axons.
And it was a long, hard road that involved dissecting 35, 000 chick brains in order to do biochemistry to get the proteins that were doing this attraction of these commissural axons.
But they got something in the end.
And what they got is a protein called netrin-1.
Panel B is a section through the spinal cord, and you cannot see it very well, except for this black glob at the bottom.
But what this is the floor plate.
And it's been stained for netrin-1 RNA.
And you can see this netrin-1 RNA is only in the floor plate region.
And if you look at the protein, it's also in the floor plate region.
Although, it diffuses a little bit out of that region.
Is netrin-1 really important for making these axons grow in the right place?
Well, yes.
Because if you take a mouse and knock out, you genetically mutate the netrin-1 gene, the commissural axons don't grow properly.
So here's a wild type mouse spinal cord, here are these axons growing down to the floor plate, and here is a mutant mouse that's lacking netrin-1.
And these commissural axons grow but they never get to the floor plate.
They don't really know where to go.
Now, if you look back at handout three -- -- there was something else about this floor plate region in that these trochlear axons were growing away from the floor plate.
And so one could construct the ultimate hypothesis that the floor plate was actually repelling these trochlear axons from growing towards it.
And it turns out that is exactly what's happened, and netrin-1 can act both as a chemo attractant and a chemo repellant.
And it does so, and this is number five on your handouts.
It does so by using two different receptors, one of which makes it attractive to neurons and the other which makes it repulsive.
So here's netrin indicated as a diffusible molecule.
It binds to a receptor called DCC which was initially implicated in colon cancer called deleted in colon cancer.
It's not clear if it really is still involved in cancer or not.
But here's netrin binding to its receptor.
And subsequent there are a bunch of signal transduction that happens that changes whether or not these axons move to the floor plate and whether or not they, in fact, grow out at all.
In the case of repulsion, it turns out that there's another receptor called unc-5.
And that makes a connection with the DCC receptor.
There's an interaction between the unc-5 and the DCC receptor.
Netrin still binds to the DCC receptor, but the net result of this is a different signal transduction pathway activated such that you now get repulsion of these neurons by activation of this duet of receptors.
So for this diffusible process, I've given you the example of netrin that can act both as an attractant and a repellant.
And many, I have no idea how to spell repellant.
Repellant, I think it's got an E. OK. Repellant.
And there are now several receptors known that are able to be, there are several guidance cues known that can either be attractants or repellants.
What happens when a growth cone sees a repulsive signal?
This movie will show you.
That is the growth cone collapsing.
Let me show it to you again.
So you start off with a pretty robust growth cone, and it collapses when you put a little drop of some guidance cue that it doesn't like up in this corner of the slide.
And what happens is that that actin cytoskeleton that is there just collapses, depolymerizes, and that axon tries to get as far away from the repulsive cue as it can.
So we've talked about short-range and long-range guidance cues.
We've talked about attractive and repulsive guidance cues.
We've talked about how one single guidance cue can be both an attractant and a repellant.
And now I want to get back to this very old hypothesis that the way that you actually got synaptic connections and the way you've got circuitry set up was by some kind of random way that involved survival of neurons that made the appropriate connections.
In fact, that is true once you get neurons to where they're finally going.
So you're setting up neurons, they're going along their path set up by some guidance cues.
And of course you should be asking the question, well, how do the guidance cues get there in the first place, right?
OK?
So that's a question you can go and sit and think about.
But in the end what happens is that you stabilize connections.
And you do it in two ways.
Both of which, or one of which seems to be activity dependent, or another of which is activity independent.
By activity dependent, I mean that the nerve has to fire.
There has to be synaptic activity.
And that will maintain the connection.
If there is not synaptic activity you will not.
And there are some cases where that is not the case.
This is number six on your handout.
When neurons grow out to the neuromuscular junction they form a synapse, or when the neurons grow out to the muscle they form a synapse with the muscle.
The synapse is initially a rather weak one, and over time there are proteins that are activated.
Acetylcholine receptors, for example, that are activated that set up the synapse and stabilize it and maintain it.
So that is one example of stabilizing connections.
Another very famous example is growth factors that are survival signals.
One of the most famous is NGF which stands for nerve growth factor.
And this is a protein that is involved in making certain sets of neurons survive when they get to their targets.
There seems to be a limiting amount of nerve growth factor around in specific regions of the animal, and only those neurons that can capture enough of that nerve growth factor on their receptors will survive.
So there are neuronal survival signals that may or may not require a neuronal activity for their effect.
OK.
So two final things.
One is this retinotectal map.
We talked about the outcome of the Sperry experiments and how you got the neurons going back to where they are.
What's the molecular basis for that?
Well, you can show very beautifully that the neurons prefer to grow on a particular part of membrane that you give them.
So, for example, the retinal neurons will choose to grow on the anterior rostral and not on the more posterior caudal neurons.
And you can show that the reason for that is due to a class of molecules called the ephrins.
Ephrins are secreted molecules, but they are short range.
They work just on the cell next door.
And you can show, I'm flashing this up, I know you can go and look at it later on, that wherever there is ephrin you do not see axonal outgrowth.
So ephrins are inhibitory to axonal outgrowth.
And I'm going to refer you to, I'm not going to go through it.
I'm going to refer you to number eight on your handout.
I'm not going to go through this.
You can look at it.
But I will just tell you that there are different combinations of ephrins and Eph receptors which guide retinal axons to the correct part of the tectum.
The last thing that I want to spend the last minute on is a particularly fascinating point.
And that is the question of nerve regeneration, and nerve regeneration in the central nervous system.
So we believe right now that the same, in fact, we know that the cues that tell axons where to go in the first place in an embryo are maintained in the adult nervous system.
And we also know that if you cut one of the nerves in your peripheral nervous system in your arm it will grow back fine.
But we also know if you sever your spinal cord it will not grow back.
And there has been intense research to try to figure out what it is that is making the axons in your spinal cord not grow back to where they were, although, in fact they should.
Here's a diagram of an intact CNS axon.
If it's severed of course you're going to get a fragment that's not connected to a cell body, but the other part of it is still connected to the cell body.
And what happens is that that dies, that cell dies.
It undergoes apoptosis and dies.
And it's really not clear why it doesn't regenerate.
One hypothesis is that there it is a poor growth environment in the extracellular matrix around the axon in the CNS.
Another class of proteins of great interest are the Nogos and related ligands which are believed to be inhibitory to axon regrowth.
And there's a lot of interest in this kind of thing.
And I'll stop there.
Thanks.
So up until this point in the class you've learned basics about cell molecular biology, genetics and development, how things normally work, how cells normally function, how tissues are formed, how organisms are formed and function.
And hopefully you've had an ample founding in understanding normal biology.
We're going to take a turn today and for the next three lectures talk about abnormalities that lead to disease.
And, whereas, I have tried to keep things light for you in my lectures and try to be entertaining if possible, the subject of the next three lectures is not a light one.
It's a very serious one.
And it is cancer.
It is a disease that I know a great deal about and have worked on in my lab for the last 17 years.
It's a very important and very devastating disease.
And we're going to try to give you an introduction to it.
And you'll see that many of the things that you've learned over the course of this class, up until this point, really prepare you to understand the details of cancer, cancer development, and hopefully also cancer treatment.
And I'll say right up front that I'm actually quite optimistic that the progress that we've made over the last several years in understanding the molecular basis of cancer has put us into position to treat the disease differently than we've done in the past and more effectively.
The number of drugs that have been approved by the Food and Drug Administration for the treatment of cancer today that are directed and in this more effective class is not great, but there are some.
And that number will increase.
So I think there's reason for optimism utilizing the information, the kinds of information, the strategies that we'll cover in the next three lectures.
So cancer, I think in some respects, needs no introduction.
It's a disease that probably has affected some of you already in your lives personally.
Certainly everybody knows a family member or friend who has developed the disease.
It is a remarkably common disease.
In this country alone -- -- more than a million people are diagnosed every year with cancer.
And this does not include another million who are diagnosed with the less serious forms of skin cancer, basal cell carcinoma and squamous cell carcinoma.
So, in that respect, more than two million people get the diagnosis of cancer every year in this country.
Moreover, each year -- -- more than half a million people die in this country from cancer.
That's more than 1,500 people per day.
So in two days more people die of cancer than died in the World Trade Center disaster.
It's a devastating disease.
Half of the men in this room and a third of the females -- -- will be diagnosed with cancer in their lifetimes.
And overall more than a quarter of all deaths are attributable to cancer.
A major problem.
A major burden.
Clearly something we need to do something about.
So I'm going to show you some slides that introduce you to the disease.
Cancer, as you know, is a generic term that covers a class of diseases that affect many, actually, almost all of your organs.
You can get cancers in different places.
They're all defined by one common property which is too many cells.
Too many cells in a tissue causing a lump.
Too many cells in the blood.
Too many cells in lymph organs.
So that's the common theme that relates to all cancers, but there are very important differences in cancers in different sites.
This is a radiographic image of lung cancer.
What you're looking at here is a standard chest x-ray.
And this individual, you can see, has an opacity right here, a fairly large opacity that's probably about the size of your fist in this lobe of the lung.
And that's fairly advanced lung cancer.
This is a more precise diagnostic test called computed tomography, which is essentially a series of x-ray slices that get reconstructed.
It gives better resolution.
And you can see, again, a very large mass in the lung of this individual.
So that's a solid tumor imaged by radiography.
This is leukemia.
Leukemia is also too many cells but here it's in the blood.
This is a normal blood smear.
These pale cells without nuclei are your red blood cells.
These nucleated cells are white blood cells of different types.
And you can see in this abnormal blood smear of a leukemia patient there is a vast increase in the number of white blood cells.
So this is a diagnostic feature of leukemia, too many cells.
And it's diagnosed by looking at blood counts.
And the blood counts in leukemia patients of white cell counts can be elevated by thousands, if not hundreds of thousands of fold.
You can also use other imaging techniques to detect and diagnose cancer.
This is endoscopic examination, colonoscopy.
It's now recommended that everyone undergoes this exam every year when they reach age 50 to detect tumors at an early stage so that they can be removed surgically which is, in fact, the most effective treatment for colon cancer.
And, actually, for almost all cancers.
The most effective treatment for cancer is the removal of the tumor.
If you can get to the tumor at an early stage through some of these imaging techniques and eradicate it, remove it, you can greatly limit the mortality associated with the disease.
So this is endoscopy.
This is what a normal colon looks like, sort of a smooth structure.
This is an early stage colon tumor.
It's called a polyp at this stage.
And I'll give you a little bit more nomenclature in a minute.
This is actually not cancer.
A polyp is not cancer.
It's a benign tumor.
This may or may not progress to a cancer, but if the doctor were to find a polyp like that they might follow it for one or two years.
But if were to get bigger they would remove it.
And they can actually do it endoscopically.
They can remove it using a surgical endoscope.
They don't actually have to do surgery to remove polyps anymore.
At some frequency these polyps do progress to colon cancer.
And this is a true cancer, a carcinoma which has changed its shape and importantly changed its relationship to the tissue.
Whereas, the polyp is sort of sitting up on top of the tissue, having grown out of the tissue, the colon cancer has invaded into the tissue.
It has become much more destructive in that sense.
And also, importantly, has the capacity to move outside of the colon through that process of invasion to other parts of the body, and thereby cause problems elsewhere.
Usually, when these are diagnosed by endoscopy or other methods, they are removed.
This is done by surgery where a portion of the colon is actually removed and then the two resected ends joined.
And you can see the colon cancer right here.
If the cancer is detected at an early enough stage, surgery is curative for colon cancer.
Unfortunately, it's often too late when it's diagnosed.
It looks like that.
Because it's already moved outside of the colon and tumor cells have found their way to other parts of the body.
So surgical removal of the primary tumor, while helpful, may actually not be curative.
So I want to go through in diagrammatic form some of the points that I've just made.
Cancer develops in states.
It doesn't happen all at once.
As I showed you in the normal colon polyp to colon cancer series, things happen in stages.
It's actually best worked out in the colon.
But we believe the same phenomena hold true for other cancers in other places.
It doesn't happen all at once.
It happens in stages.
And these slides illustrate the basic principles of that.
This illustrates a normal tissue.
It could be anywhere, but let's say that it's in the colon.
And there are certain important features to pay attention to.
One is that the cells have regular shape and a regular relationship to their neighbors.
As you know, tissues are formed for normal function.
They have particular genes expressed.
They have particular structures inside them.
They interact with their neighbors in order to perform their function.
So the cells of the colonic epithelium line up like this touching each other, interacting with each other and interacting with the cells underneath them, as well as other material underneath them.
Here is a material called the basement membrane, otherwise known as the extracellular matrix.
And this helps support the cells and provides the cells with certain nutrients and growth factors.
There are also cells below these cells in the colon and in other tissues which replenish these cells when they get lost.
These stem or progenitor cells exist probably in all adult tissues.
They get recruited when cells turnover.
And we now think, and it's not illustrated on these slides, but we now think that these cells are probably very important in tumor initiation.
We think that these are cells, these stem cells of the tissue are likely to be the place where the tumors initiate and then produce cells, like this brown cell here, which is already abnormal.
Cells that have acquired a mutation which makes them different from their neighbors.
Now, the first consequence of those early changes is that you get too many cells.
And this is a process known as hyperplasia.
Hyperplasia simply means hyper, too many, plasia, division, cell division.
So there are too many cells.
But the cells that are present there look pretty normal.
They don't histologically, by looking at the tissue, appear different from their normal neighbors.
There are just too many of them.
This process continues until you can actually see a discernable lump, a mass, and that is a formation of a benign tumor.
In the colon that tumor, that benign tumor is called an adenoma.
It's called other things in other places but let's say, just for the sake of simplicity, it's a benign tumor.
And, again, this is an increase in the number of cells but the cells themselves don't look very different from their normal neighbors.
And these tumors are not life-threatening yet.
They sometimes will never be life-threatening but in this form they rarely cause problems.
They're just a lump.
Like a wart is a benign tumor, a lump that doesn't cause problems.
At some frequency these tumors can progress further.
And now, as you can see from this different shading, the cells take on different characteristics.
They look different from their normal neighbors.
Their cyto architecture, the cell shape looks different.
The nucleus often looks different from the normal cells.
And so they've become fundamentally changed in their appearance.
Moreover, they have begun to recruit a blood supply.
The presence of these blood vessels which were not present up here is another indication that things have progressed.
And tumors require a blood supply of their own to feed the tissue, this abnormal growing tissue.
As soon as they get to a certain size they need to recruit a new blood supply.
This tumor at this stage has changed enough that we now call it cancer.
This is based on these histological changes, the appearance of the cells, as well as the cells' relationship to one another.
And this is now considered dangerous.
If you're diagnosed with a cancer, even if it's present in the tissue, the initial tissue alone, it would be recommended for removal.
However, it rarely stays there.
Tumors have the capacity to move from their original site through a process of invasion and then metastasis.
This illustrates invasion, local invasion where the tumor cells are moving from the mass through the basement membrane and ultimately accessing the blood supply.
Literally moving into blood vessels, traveling through the blood, and then moving out of blood vessels and beginning to proliferate, expand in other tissues.
And this is the process of metastasis.
And the tumor cells can literally go anywhere.
Certain tumors have preferences to go to certain tissues than others.
And when they get there they begin to grow again and turn, become actually more advanced, more dangerous, cause local tissue damage, organ failure.
And it's usually this that causes the death of the cancer patient.
It's the metastatic spread of the disease as opposed to the primary tumor that tends to kill cancer patients.
And sadly it's this phase of the disease, metastatic spread that we know the littlest about.
We actually know rather little about what allows cells to move from their primary site through the blood, establish themselves elsewhere and then grow there.
And since it is the most deadly phase of the disease, it represents a clear need, clear opportunity to learn more.
So let me just illustrate, or rather emphasize some of these bits of terminology for you.
Hyperplasia is simply too many cells.
You have hyperplastic lesions, we call them, throughout your body.
Most of them no problem at all.
They're not going to do anything.
It's just there are too few many cells in that particular place.
These can progress to benign tumors which are non-aggressive.
That is the cells are relatively well-behaved.
They're not dividing that much.
They're not moving around.
They have normal relationships with their neighbors.
And with respect to the tissue that they reside in, they're nondestructive.
They haven't breached the underlying basement membrane.
They haven't changed their relationship with their normal neighbors.
And they're also defined through a lack of spread.
They haven't moved out into the local lymph nodes or to surrounding tissues.
These can progress to malignant tumors.
And these are true cancers.
We actually only use the term cancer to refer to malignant tumors, not to benign tumors.
These are true cancers.
And these are aggressive.
The cells have changed shape.
They have much greater proliferative capacity.
They move around to a greater degree.
They are destructive with respect to the tissue in which they reside.
And they have the potential to spread.
Not all diagnosed cancers have already spread, but the view of the pathologist is that they have the potential to spread and therefore they are typically removed.
Now cancers in different places have different names.
As I said, you can get cancer in virtually all tissues of your body.
There are probably 250 or 300 different types of cancer clinically defined, and those are defined in part based on where they are and in what types of cells they occur.
So there are tumors of epithelial cells, cells of the skin, lining of the intestine, mammary epithelial cells, epithelial cells of your brain.
And these tumors are called carcinomas.
These cancers are called carcinomas.
And you might have heard, for example, a subset of carcinomas called adenocarcinomas.
Adenocarcinoma is what I showed you with respect to the colon cancers.
Adenocarcinoma of the breast.
Adenocarcinoma of the lung.
These are carcinomas that have a glandular appearance.
And there are other kinds of carcinomas as well.
Those are the cancers.
And the benign tumors of these tissues, especially of the class that gives rise to adenocarcinomas are called adenomas.
There are also tumors of your connective tissues like you cartilage or your muscles or your tendons.
These are called sarcomas.
And you've probably heard of myosarcomas.
Those are tumors of the muscle.
And there are different types of myosarcomas.
And then there are tumors of the blood system.
These are called leukemias if the tumors are present, tumor cells are present within the blood itself.
And I showed you an example of that.
Or lymphomas.
And here it's defined as tumors which stay within lymph organs like lymph nodes or spleen or thymus.
They haven't made their way out into the blood, but they are still tumors of the blood cells.
OK.
So we know a lot about what these tumors look like based on visual inspection.
We also know what they look like based on histological analysis.
This is a progression series actually taken from a mouse model of cancer generated in my lab.
It's a tumor of the lung.
You might be able to see the lacey appearance of the lung here.
And this circle you can see hyperplasia, too many cells.
Cells, if you were to see them up close, look pretty much like their neighbors, but there are just too many of them.
These progress into lumps, masses called adenomas in this case.
These are benign tumors.
They don't have the features of the more aggressive malignant cells.
And you can see them here.
They all look pretty much the same.
And if you saw them next to a normal cell they would look pretty much like those.
But these can progress into adenocarcinomas, more advanced tumors.
These are true cancers.
And you can tell that based on the fact that the nuclei, if you compare one cell to another, looks very different, and the cell shape looks very different from one to the other.
So there's heterogeneity.
And what you cannot see here is that the tumors are also much more proliferative.
They are dividing.
The cells are dividing much more frequently.
So, again, histologically we can characterize what these tumors look like.
What I'm going to tell you about from this point really to the end of the class is that we understand, at least to a degree, what these arrows represent, what happens to a normal cell on the way to hyperplasia or to an adenoma or to an adenocarcinoma.
And the answer is defects to genes.
Defects to many genes underlie these transitions, allow a normal cell to develop into a cancer cell.
And so in that sense we consider cancer as a genetic disease.
Now, cancer actually is also a more traditional genetic disease in some instances.
There are people who are genetically predisposed to developing cancer.
And I'm actually going to tell you about them next time.
But even for people who are not genetically predisposed to cancer, the development of cancer in them is a genetic disease.
How do we know that?
Why do we think that?
Well, we think that for a number of reasons.
And one of them, which we've appreciated for more than a hundred years is that the chromosomes of cancer cells are screwed up.
They look different than normal cells.
I've actually shown you this slide before.
This is a karyotype of a normal cell, 46 chromosomes, 23 pairs.
These are highlighted by stains that allow us to distinguish one chromosome from another.
So chromosome one stains yellow, chromosome two stains red, and so on and so forth.
And this is the karyotype of a cancer cell.
It's different in many ways.
Name me two.
I didn't hear anybody.
I heard there was mumbling but I didn't hear anybody.
Say again.
The number and shape of chromosomes.
Good.
So clearly there are too many chromosomes.
I didn't count them up, but there is more than double the proper number of chromosomes in this cancer cell.
So chromosome number defects are quite common in cancer cells.
But also structure, and I've boxed two of them here, the structure of chromosomes in cancer cells are different.
This chromosome has undergone what we call a translocation in which a piece of one chromosome has been joined to a piece of a different chromosome.
And we now know that when that happens it's reflective of a mutation that either is inactivating or activating a gene that's present at the point of translocation.
And this chromosome has undergone a truncation, a deletion.
Genetic material has been loss.
And this, again, indicates that there was a gene present there that the tumor cell wanted to get rid of in the development of the cancer.
So chromosomal abnormalities are quite common, which was a first clue that cancer is a genetic disease.
The second indication, which came along as early as the 1920s, is that many carcinogens, and what I mean by that is agents that cause cancer.
Many carcinogens are mutagens, agents that cause what?
Yeah?
Not all.
But there is a very high correlation between things that will cause cancer and things that will cause mutation in simple mutation assays.
And you can do assays to determine the carcinogenicity or mutagenicity of an agent of interest.
Carcinogenicity assays are done typically in experimental animals, mice or rats.
And one takes the animal and exposes it to agent X.
It might be injected into the animal or it might be painted on the skin of the animal.
And then one waits a period of time and asks the question did the animal develop a tumor?
And if so the agent is a carcinogen.
And you can actually determine how strong a carcinogen is, how strong a carcinogen it is by determine the dose, the amount of the agent needed to produce a tumor.
There are also mutagenicity tests.
And this is typically done in bacteria.
So one takes a bacterial strain and asks whether the agent can cause mutations in that strain.
And usually to do this one takes a bacterial strain that already has a mutation, a bacteria that is histidine deficient, which means that it's not able to synthesize its own histidine and amino acid.
That means that if you take that bacteria strain and plate it onto a Petri dish that lacks histidine, you haven't added histidine to the media, will it grow?
No.
It's histidine minus, it cannot make its own, if you don't provide it, cells cannot grow, so you get no colonies.
However, if I now add agent X, which I think might be a mutagen, to these bacteria, if it is a mutagen it might mutate the defective histidine gene and turn it into a normally functioning histidine gene such that if I now plate these bacteria that have been treated with the mutagen I may get some colonies.
And you're still using a his-minus plate here.
And the number of colonies that I get is an indication of the strength of the mutagen.
Lots of colonies, strong mutagen.
Fewer, not so strong.
And what I'm saying is that there's a strong correlation between things that pass this assay and things that pass that assay, another indication that cancer might be caused by mutations to DNA.
Now some things that we know for sure are carcinogens.
They pass this carcinogenicity test very, very strongly.
Fail in this simple mutagenicity test.
Why?
How can that be?
They clearly are cancer-causing.
But if you mix them with bacteria in isolation here you don't get any increased mutation frequency.
What's different?
What's different about this situation versus this situation?
Yes?
Excellent.
Precisely correct.
There are certain agents which in their native form are not mutagenic.
We call them promutagens.
However, when acted on by the body in the process of metabolism they become mutagens.
The body senses that these are actually dangerous compounds and will try to detoxify them, typically by adding hydroxyl groups to them to make them more water-soluble so that they can be secreted or excreted by the body.
But in the process of making them more water-soluble they go through an intermediate form which turns out to be highly mutagenic.
And that's illustrated here for a very powerful carcinogen known as benzo(a)pyrene.
This is an agent found in cigarette smoke, for example.
This agent is not highly mutagenic on its own, but in this type of test it is highly carcinogenic.
And what happens is that the body, in an attempt to add hydroxyl groups to make it more water-soluble, will actually introduce epoxides in different ring positions of this molecule, as an intermediate towards making it more hydroxylated.
And these epoxides are very interactive with DNA and therefore mutagenic.
So the body turns something that actually wasn't mutagenic into something that is mutagenic.
And therefore this test, this mutagenicity test has been modified, principally based on the work of a bacteriologist at Berkeley named Bruce Ames.
And so we now call it the Ames Test.
And the Ames Test takes the compound, the agent that you're trying to test the mutagenicity of, and rather than adding it directly back to bacteria it's mixed first with an extract of the liver.
Many of the enzymes that do this detoxification are present in your liver.
Cytochrome P450 enzymes, for example.
In which case the compound becomes modified, or might become modified.
And through that modification becomes mutagenic.
And then the agent gets tested in the standard bacterial mutagenicity test.
OK.
So we can now expand our list of things that are mutagenic through exposing them to some of the body's own enzymes.
In this sense, the body is actually doing you a disservice in thinking it's actually trying to help you.
There are still other agents which are not mutagens, even in the Ames Test, but are clearly carcinogens.
Asbestos, for example.
That stuff in packing material that's now been banned exposure to which gives people, or can give people mesotheliomia, a tumor of the lining of the lungs.
It's not a mutagen.
However, you test it, it's not a mutagen.
Alcohol, which is clearly associated with liver cancer, is not a mutagen.
These agents we think act as irritants.
They case damage to tissue which causes the cells to have to increase their proliferation to repair the tissue, and that is inherently a mutagenic process.
And so these things can act as carcinogens in that very indirect way.
OK.  Now there are many things in our environment that are carcinogens -- -- that we get exposed to either passively or deliberately.
Sunlight, for example, is a carcinogen and a mutagen.
It causes damage to DNA, causes mutation, increases your risk of skin cancer.
Other things that we do to ourselves.
Certain dietary carcinogens we impose on ourselves.
But what's the biggest carcinogen that we impose on ourselves?
Smoke.
Tobacco smoke.
And obviously tobacco smoke is associated with one particular type of cancer, although it's not limited to lung cancer.
Any idea how many people die of lung cancer in this country per year?
Anybody want to hazard a guess?
10, 00?
100,000?
175, 00.
175,000 people per year die of lung cancer each year.
And more than 150,000 of them it's because of smoking.
Lung cancer is the number one cancer killer.
It kills more people in this country than breast cancer, colon cancer and prostate cancer combined.
And it's completely preventable, or almost completely preventable through a change of habit, failure to smoke.
So let me just emphasize the importance of this.
These are curves that show the smoking frequency in this country among men and women and the incidence of cancer, lung cancer in this country among men and women.
You can see that early on, in the early part of the century smoking was  uncommon and lung cancer was uncommon.
But as people began to smoke this frequency increased and then years later the incidence of lung cancer increased dramatically.
It takes a little while.
You've got to expose yourself to these carcinogens for a period of time for this process to have its effect, but eventually that smoking exposure leads to the development of lung cancer.
You can see that women started smoking a little bit later, but they quickly caught up.
And now the lung cancer incidence in women is also extremely high.
Lung cancer among women is the leading cause of cancer deaths.
It's now surpassed breast cancer as the leading cancer killer of women.
The statistics about smoking in this country, despite these facts, are remarkable.
47 million adults smoke.
Roughly one in four men, and almost as many women smoke.
Even more amazing to me than that, in 2002, I don't have more recent statistics than that, but in 2002 the percent of high school students who responded to a survey about whether they smoked was what?
What do you think?
High school students.
28%.
So this is not just people who have been smoking a long time, who started when they didn't know any better.
Kids who get exposed to the message about smoking and lung cancer from an early age are not paying attention.
So the problem is not going away.
These folks obviously have a dramatically increased risk of dying from lung cancer.
And it's not just lung cancer.
Smoking increases your risk of heart disease, of stroke, of emphysema.
I read a startling statistic when I was preparing this.
Of all the people who are alive today on this planet, 500 million of them will die early due to tobacco usage.
500 million people who are alive today would live longer if not for exposure to tobacco products.
So if you learn nothing else from this class this year, if you now smoke, stop.
If you don't smoke, don't start.
OK.
So carcinogens in our environment and carcinogens that we exposure ourselves to are important.
No doubt about it, they can be a cause of cancer.
I call these exogenous or environmental mutagens.
And, as I mentioned, sunlight and other forms of radiation, certain dietary things that we expose ourselves to fall into this category.
But they're not the only thing that causes cancer.
And it's very important that you know that.
In fact, more of the mutations that happen in cancer are due to indigenous processes, things that happen inside your bodies regardless of what you get exposed to, like DNA replication mistakes.
DNA polymerases that duplicate your DNA are pretty good, they have proofreading functions, but they still make mistakes.
And these mistakes can be then in critical genes which can ultimately lead to tumor development.
Your DNA can sometimes get broken.
It gets moved around inside the cell during various processes.
Sometimes it gets broken, and sometimes those breaks are not properly repaired.
And this can lead to some of the defects that I've shown you on this slide like the translocations where different pieces get rearranged with the wrong other piece.
Defects in chromosome segregation.
We talked about mitosis early in the class and how the cells have an intricate ability to properly segregate their chromosomes, that each daughter cell gets the right number.
Sometimes that process doesn't work properly so that one cell gets too many chromosomes or one cell gets too few.
These chromosomal imbalances can also contribute to tumor development.
Defects in DNA repair.
Your cells have all sorts of enzymes that will try to find damaged DNA and fix it, but these enzymes themselves can be mutationally inactivated in the development of cancer.
So now you've debilitated the cell's ability to fix the damage leading to an increased frequency of damage, increased frequency of mutation, increased cancer risk.
And importantly -- -- production of indigenous mutagens in contrast to exogenous mutagens.
Your body actually produces things that are mutagenic.
And the biggest class of these are oxygen radicals such as superoxide -- -- or hydrogen peroxide.
And these radicals are highly reactive to DNA, can cause DNA breaks and base changes, and cause mutation.
And your body produces these naturally in the process of metabolism.
You have enzymes that will kind of keep the concentrations of these down but, nevertheless, they're present and cause damage at a certain frequency in all cells.
If that damage occurs in the wrong cell at the wrong time it can be cancer-causing.
So there are lots of ways that the DNA of your cells can get mutated, which then overall contributes to the development of cancer.
Now, this doesn't happen all at once, as I've tried to indicate to you.
It doesn't happen from normal cell to cancer cell in a single step.
We think that this process happens over time.
And in humans over decades of time.
Cancers often initiate maybe in mid-life, but they don't present themselves as a clinically advanced tumor until late in life.
and that's because the process requires lots of steps that require lots of time to accumulate.
And so we now consider tumor development from a normal cell to a more advanced cancer cell as a clonal evolution process.
A clonal evolution process whereby abnormal cells arise in a population.
These now have a selected advantage compared to their normal cells and will grow, for example, better than their normal cells, divide faster than their normal cells, make more of themselves.
The mutation that took place is now present in all the daughter cells of that abnormal cell.
And within that now-expanded clone of abnormal cells a second mutation takes place which increases the capacity of that cell to grow and divide, survive, move around compared to its neighbors.
So now this cell and of its descendents have two mutations.
And within that expanded clone a third mutation can take place and so and so forth.
We now think that cancers, advanced cancers in human probably have five, ten, maybe twenty distinct mutations in genes controlling important processes.
What are those processes?
There are many.
We're going to emphasis, in this class, just proliferation and cell death.
I'll say a bit more about that in a second.
But it's also important for you to know that the cancer cells also regulate factors that increase the blood supply.
Cancer cells turn on growth factors that recruit blood vessels.
If they didn't do so they would never be able to progress.
You cannot actually make a solid tumor bigger than two millimeters in diameter without recruiting a new blood supply.
And so the tumor cells turn on these factors to deal with angiogenesis.
They also turn on factors that allow them to move around to a greater degree or separate themselves from their neighbors to invade into the basement tissues, the basement membranes.
To turn on proteases, for example, that chew up the extracellular matrix to allow the cells to move around, and other factors.
Factors, for example, that allow them to move into the blood vessels or move back out of the blood vessels in the process of metastasis.
So we're going to emphasis proliferation and cell death in the next lecture, but I want you to know that many other aspects of cell biology are important in this process as well.
So proliferation, it's kind of obvious that in cancers there might be changes that lead to more cell division.
And, indeed, the original cancer-associated genes did this.
And many of the important cancer-associated genes that we know about are involved in promoting increased cell division.
But we also know that cell death or apoptosis, which I briefly introduced you to in an earlier lecture, is important in tumor development.
And do you find more apoptosis or less apoptosis during tumor development?
Less.
There's decreased amounts of cell death in tumor cells compared to normal cells.
And this is due to mutations in genes that regulate apoptosis, positive mutations that block apoptosis, loss of function mutations in genes that are normally required for apoptosis.
And we think, therefore, that the balance, if you think of this as a scale where proliferation is on one end and cell death is on the other.
Whereas, this process is well balanced in normal tissues, what we call normal homeostasis, you produce as many cells as you need, you kill off as many cells as you need in order to give rise to a normal, average cell number.
In cancer this process is deranged.
And this can happen in one of two ways, and typically in both.
There can be an increase in proliferation and a reduction in cell death.
And we'll talk about some of the factors that control those specifically next time.
But the final way that we know that cancer is a genetic disease is by looking in the DNA of a tumor, by looking at the genes themselves.
You know about genes.
You know about gene sequences.
You know how to figure out what the sequence of a gene is.
When this is done in a cancer cell compared to a normal cell one finds mutations.
And this is actually the very first cancer gene found in the context of human cancer.
It was found here at MIT by Bob Weinberg's lab.
And when its DNA was sequenced compared to the normal sequence shown above it incurred a mutation, a mutation which blocked the ability of this protein to be properly regulated leading to increased proliferation.
And I'll tell you that story next time.
And he took what was left, which much be very, very small, smaller than the size of a cell because the cells were removed by filtration, and he injected it into another bird.
Now, a healthy bird.
And strikingly that bird developed a tumor.
So he was able to transfer whatever agent it was that was associated with tumor development from this bird to this bird.
He concluded that it was something smaller than a cell, and he concluded that it was a virus.
And indeed he was right.
Now, the tumor that these birds developed is called a sarcoma.
It's a tumor of the muscle in this case.
And the virus that was responsible for this tumor was named after Rous, and it goes by the name of Rous sarcoma virus or RSV.
Now, there was great skepticism about the connections between viruses and cancer at that time, and actually for many decades thereafter.
And so Rous' work was not fully appreciated for some time.
But it was eventually appreciated.
And Rous actually won the Nobel Prize 50 years after he made this discovery, when it was confirmed that indeed the stuff that he was studying was highly relevant to the situation in humans.
Not analogous, necessarily, but highly relevant.
Over the years, since that discovery and discoveries made by others, demonstrated that the Rous sarcoma virus viral genome carried a set of genes that were familiar.
These are the same sorts of genes carried by many viruses of this class.
Rous sarcoma virus happens to be a retrovirus.
I'll teach you more about those in a future lecture.
It's the same general class that includes HIV.
And there were some familiar genes that were responsible for the replication functions of the virus.
But then there was a new gene present in this strain of the virus, specific to Rous sarcoma virus, which was named Src and was later considered to be the relevant oncogene, the gene that was responsible for the tumor forming capabilities of this Rous sarcoma virus.
So there was then great interest in what was this Src gene?
What was this special gene that could cause normal cells to become cancer cells?
And research then went on for, again, a number of years studying Src, studying its biochemical functions, but the real question that was of great interest over the years was where did Src come from?
Most viruses in this class don't carry Src, and it wasn't at all clear what the origins of this cancer-causing gene were.
Was it a rare viral gene or perhaps did it come from the host cells themselves?
And in experiments that were done, actually by my PhD advisor, Harold Varmus and his colleague Mike Bishop, it was shown that the Src gene actually does indeed reside in the cells of the host and gets picked up by a recombination process by the virus.
And I'll just illustrate that for you.
It turns out that there's a related virus fairly common in chickens called avian leukosis virus.
And this is the predecessor of Rous sarcoma virus.
It's a virus that has in its genome just the replication genes.
And I'll just draw a viral capsid around this virus.
So this is the avian leukosis virus with its genome and replication genes.
Now, what they showed was that the avian leukosis virus, when it infects cells, here's a normal chicken cell.
Most of the time it just makes more copies of itself like viruses will do.
But occasionally, through a recombination mechanism, the avian leukosis virus will actually pick up a gene which is present in the normal genome of the chicken, a gene which looks very much like the Src gene present in Rous sarcoma virus.
And through this recombination mechanism, the details of which I won't go through, very rarely you'll get a recombinant virus produced which has in its genome the replication genes, but now also Src gene.
And this virus, which modifies the Src gene slightly in the course of the development of the virus, now, when introduced into birds, will very efficiently cause tumors.
And what this showed, what this actually Nobel Prize winning experiment showed was that the oncogene, powerful cancer-causing gene resides in the DNA of the chicken.
And actually there were similar genes, very analogous genes residing in the DNA of all vertebrate species, including humans.
You are sitting there with two copies of the Src gene inside of you.
So a question is, if that's true, if we all have these oncogenes in our cells why don't we get cancer?
Why aren't we all sitting there as one large sarcoma?
Anybody?
Well, there are two answers.
Yeah?
Somebody.
Claudette is pointing to somebody.
Am I blind?
Oh, hi.
Excellent.
Yes.
So what's different about the situation in the chicken cell and the situation in the viral genome is how the gene is expressed.
Here it might be expressed properly under what we would call physiological control, express where it's supposed to be when it's supposed to be.
And here it's been removed from that regulatory network and is expressed perhaps at two high levels at the wrong time.
And under those conditions it can push cells to inappropriately divide.
So the virus has hijacked this gene and changes its regulation.
That's one explanation.
There's potentially another explanation for why we're not all sitting around with tumors, and I'll come to that in a moment.
Now, as I said, Varmus and Bishop won the Nobel Prize for this work in 1989.
And the day that they won the Nobel Prize Harold Varmus' sister-in-law was standing in a cafeteria line at Berkeley and she overheard two guys talking behind her.
One guy said to the other, what are you going to have for lunch?
And the other guy said, I don't know but it ain't going to be the chicken because two guys just won the Nobel Prize for showing that chicken causes cancer.
[LAUGHTER]  Which is not exactly true.
But nevertheless.
Now we know since that work that there are a large number of these viruses which are called collectively acutely transforming viruses.
There are a large number of acutely transforming viruses that carry in their genomes and oncogene which they have cooperated from the host cell.
These viruses are not viruses of human beings.
They're viruses of experimental animals and usually generated in an experimental setting.
Mice, rats, chickens, turkeys, other species have been used to generate such viruses.
And about 50 or so oncogenes have been discovered through this context, including one that's hopefully familiar to you already and will come again, members of the Ras gene family which I've told you about as an important signaling molecule in mitogenic signaling pathways, and another important oncogene that I'll mention briefly later called mic, and about 50 more.
And it turns out that they were a very useful source to identify cancer-associated genes in humans.
Many of the genes that we now know are important in human cancer were initially discovered through that process.
Now, as I said, most of the time human cancer has nothing to do with viruses so don't be confused.
There are a few human cancers that are virally associated.
The major one is cervical cancer.
About 50% of cervical cancers, particularly in the Developing World, are associated with the virus called human papillomavirus or HPV, and specifically the high-risk types.
Not all papillomaviruses will cause cancer.
Papillomavirus is the same virus that causes warts, for example.
And there are many papillomaviruses that are not associated with true malignancies, but human papillomaviruses of the high-risk type are.
And they can give rise to cervical cancer.
Fortunately, companies are now developing vaccines against HPVs which are greatly affecting the risk of cervical cancer around the world.
So this is a major step forward in controlling this particular cancer type.
So because most human cancers are not virally associated, there was still some skepticism about the importance of this discovery for human cancer.
Maybe it was true of the viruses, maybe it was true of these experimental animals, but is it true of human beings?
Do these oncogenes have anything to do with human cancer?
And this debate went on for a little while longer until Bob Weinberg here at MIT in about 1980 did the following experiment.
He took DNA not from a bird but from a human being who carried a tumor in his bladder.
So this individual had bladder cancer.
This cancer was put into cell culture and turned into a cancer cell line, and it was actually this material that Weinberg's lab worked with.
And they asked the question, are there genes in the cancer cell line that are cancer-causing?
Are there oncogenes that I can discover within those cancer cells?
So the experiment that they did was to take DNA, they isolated DNA from the cancer cells, and they introduced it into mouse cells in the laboratory that were "normal".
And what I mean by that is they looked pretty normal compared to the cancer cells which had a deranged sort of architecture, as I mentioned last time.
They grew flat and didn't grow on top of each other like cancer cells will do.
And, importantly, if you were to inject these cells into an immunocompromised mouse they wouldn't form a tumor.
They were nontumorigenic.
Whereas, these cells, if injected into a mouse, would form tumors.
OK?
So they isolated DNA from the cancer cells and put it on the mouse cells.
The mouse cells will, at some frequency, take up the DNA, incorporate the human DNA into their own genomes and begin to express the genes from the human DNA.
And they found at low frequency that within this population of otherwise normal-looking mouse cells abnormal colonies of "transformed" cells could be found.
And these transformed cells looked a lot like the cancer cells.
Their shape was different, they grew on top of one another, and they had other properties that also made them similar to cancer cells in the sense that if they injected these cells into an immunocompromised animal, whereas the normal cells would not form a tumor, these transformed cells would.
So they were able to convert normal cells into cancer cells through the addition of DNA from a cancer cell line.
So what's going on here?
What did they do in this experiment?
Yeah.
They eventually did.
To get to that point they were able to isolate from the full genome of the cancer cell an individual gene that had been transferred into these mouse cells.
They isolated that human gene away from the mouse genes, and then they preformed DNA sequencing.
And they discovered the first human oncogene that was isolated from a cancer as opposed to a virus.
And, as I alluded to last time, the gene that they isolated in this fashion was a familiar one.
It was the Ras gene, a gene that had been discovered in the context of acutely transforming retroviruses.
It was already known to be cancer-associated, but now it's in the context of real human cancer.
And this was a major advance.
Moreover, they sequenced the Ras gene from the cancer and compared it to the sequence of the Ras gene from normal cells and they found a difference.
They found that it had been mutated, just as the genetic theory of cancer would predict.
Compared to the normal sequence of the Ras gene, which is shown above, the sequence of the Ras gene isolated from the human bladder cancer sample carried a single mutation which converted a glycine amino acid to a valine amino acid.
And the consequence of that mutation changed the Ras protein from being a signaling molecule that could be regulated from an on state to an off state to a signaling molecule that could not be regulated.
Once it go turned on it couldn't' get turned off.
So it was a constitutively active signaling molecule, which makes sense for a cancer cell that is continually proliferating.
Now, there's an issue here that I would like you to ponder.
And that is that this is a single point mutation, a single base change.
In all of your cells, during cell division, there's a particular mutation rate.
It's not known exactly, at least for all cells in your body what that rate is.
But let's estimate that it's about ten to the minus ninth per base pair per cell division.
Now, if you calculate how many cell divisions are going on in your body at any given time, you can estimate that you have about ten to the third to ten to the fifth Ras mutant cells inside you as you sit there today.
Maybe as many 100, 00.
Maybe more than a million cells that carry that very mutation in your bodies.
So I ask you again, why is it that you don't all have tumors?
This time it's not the explanation that we heard before about regulation because this is the native gene which is being changed.
It's being regulated the same way.
So why is it that we don't all have bladder cancer, for example, or some other type of tumor?
Well, it relates to what I told you last time, that cancer is rarely a single-step event.
Cancers rarely are associated with a single mutation.
A single mutation may be involved, may be involved in initiation, but it's not sufficient to give you all the steps you need for a full cancer.
As I mentioned last time, you might need five, ten or twenty distinct mutations to get a true cancer.
So you might have mutant cells in you, they might be totally normal, but if layered on top of those mutations are other mutations they could progress.
OK?
Now, that's particularly important because the mutations that we're talking about are, at least by some assays, dominant.
And the old genetics terminology of dominant and recessive, these mutations are dominant.
Note, in this assay we're transferring a single gene into an otherwise normal cell and seeing a phenotype.
That's an example of dominance.
And these genes can, in the right context, function in a dominant fashion on top of a normal copy or normal copies of the gene.
And you'll see why a distinction between dominant and recessive is important in a moment.
Now, as I hope you remember from earlier lectures about signal transduction, the Ras proteins fall within a signal transduction pathway that's very well worked out.
They link transmembrane growth factor receptors through intermediary proteins, adaptors and exchange factors, to downstream components involved in, for example, phosphorylation kinases and transcription factors that turn on the expression of target genes.
This is the pathway that I've illustrated to you before.
And we now know that many of the components of this pathway, this so-called mitogenic signaling pathway that drives cells to divide are mutated in cancers.
Not just Ras, which is actually mutated at pretty high frequency in cancers, but many of these components are mutated as well.
And they're mutated through different mechanisms.
Subtle mutations, like I've just told you about for Ras, where single nucleotides can be changed and a single amino acid can be changed to convert a normally regulated protein into an abnormally regulated protein like Ras.
Some of the growth factor receptors are likewise activated by that mechanism.
But there are other mechanisms, other activation mechanisms for cancer-associated genes, including gene amplification and translocation which I'll illustrate on the board.
I hope you noticed on the right here, Review Sessions for Monday night for the exam on Wednesday, as well as Tutoring Sessions and indications of where you're supposed to go.
So gene amplification is important in cancer.
There are quite a few genes that are amplified in tumors compared to normal cells.
And I'll give you one example which is relevant to breast cancer and ovarian cancer.
In normal cells, in the DNA of normal cells there's a gene called HER2 which is one of these growth factor receptors like number two up there.
And of course the normal cells are diploid so they have two copies of the HER2 gene.
And through normal transcription and translation that's going to give rise to a particular concentration of the growth factor receptor on the surface of the cells.
And that's going to give rise to a certain signal emanating from that growth factor receptor which will cause the cells to divide at the appropriate time and place.
In about 30% of breast cancers and a similar percentage of ovarian cancers, the number of copies of the HER2 gene has been increased dramatically, sometimes as much as a hundred fold.
And these are due to errors in DNA replication.
Instead of having DNA copied once in this region, it gets copied again and again and again and again.
And now, in these cancer cells, you don't have that same concentration of the growth factor receptor.
You have a much higher concentration of the receptor.
And because there's a much higher concentration of the receptor, you might get as much as ten or a hundred times the level of signaling.
And the consequences of this are that a cell that shouldn't divide doesn't have actually enough concentration of growth factors where it should normally divide will now divide anyway.
Inappropriate proliferation, the hallmark of cancer.
Now, the good news here is that there are antibodies against this HER2 protein.
And they are actually effective in the treatment of breast cancer.
They can prolong the life of women with breast cancer, specifically those who have amplifications of this gene.
This drug actually does nothing for women who don't have amplifications, but it prolongs the life of women who do.
So that's an important link between basic biology and treatment.
We'll talk more about those next time.
Another mechanism of activation of oncogenes is translocation.
I showed you last time karyotypes of cancer cells in which one piece of one chromosome gets linked to a piece of a different chromosome.
The reason that happens and is selected for is that new regulatory networks or new genes can be produced by those kinds of break and joining reactions.
And one famous one involves this gene called myc which is a transcription factor like the bottom of this pathway, like number eight there, transcription factor which is important in driving cells into the cell cycle.
myc is normally expressed from a "weak promoter".
Which means that not a whole lot of myc mRNA is produced, which means that not that much myc protein is produced at any given time in a cell.
And this gives rise to regulated cell division.
However, occasionally in cancers, particularly in certain types of leukemia and lymphoma, translocation events take place where a new segment of DNA is produced which joins the normal myc gene to a very strong promoter.
And now, just like this gentlemen mentioned before about the RSV, now a gene is not regulated properly.
It's now regulated, in this case, too strongly.
So a lot of myc mRNA is produced which gives rise to a lot of myc protein which gives rise to unregulated cell growth.
Again, the hallmark of cancer.
So the DNA of tumor cells changes by point mutations, by gene amplifications, by translocations, as well as by gene deletions, which I haven't shown you but also occur.
And together these mutations are found in virtually all human cancers.
Sometimes RAS, sometimes in growth factor receptor, sometimes in transcription factor, but one place or another you would typically find a mutation in this pathway.
So signaling proliferation is key to tumor development.
And that's not too surprising given how we think about how cancers arise.
But these oncogenes, these positive regulators of tumorogenesis are not the only genes that are important.
Normal cells do, in fact, use these signals.
And I've equated them with like the gas peddle on your car, go signals, which cause normal cells to divide when it's appropriate to divide during development or injury repair or cell replenishment.
But there's another class of genes that turns on when it's appropriate for cells to stop dividing.
So these genes are the equivalent of the stop signals which would impose themselves when cell division should cease.
Now, oncogenes promote cell division.
Too much oncogene product, a mutant oncogene product will cause normal cells to divide too often.
But in addition to those we find mutations in the stop signals, the signals that would normally halt proliferation.
And this allows an even greater accumulation of cells.
The go signals are the equivalent of oncogene mutations and the stop mutations are the equivalent of another class of genes known as tumor suppressor genes.
Tumor suppressor genes have been known for about 20 years now.
And the first one comes from this tumor here.
This is a child with a tumor called retinoblastoma.
It's a tumor of the retina.
It's actually really rare.
It occurs in about one in 40, 00 births in the general population, but there are individuals who are familialy predisposed to retinoblastoma.
And in these kids 90% of the time they will develop retinoblastoma actually affecting both eyes.
So can be actually a fairly common and severe disease in that sense.
This gene, the gene responsible for retinoblastoma was cloned, again in Bob Weinberg's lab several years ago.
And we now know that it's an important cell cycle regulator, as is indicated here.
So the retinoblastoma gene encodes a retinoblastoma protein.
The retinoblastoma protein is called pRB.
And, as is illustrated on this slide, the RB protein is a negative regulator of the cell cycle.
As you recall, cells normally cycle from M phase to G1/S phase through again.
And the RB protein acts normally to restrain cell division.
It blocks cells in the G1 phase of the cell cycle when it's active.
When cells receive signals from growth factor pathways they inactivate the RB gene through phosphorylation, the RB protein through phosphorylation.
And this occurs by cyclin-CDK complexes, which I told you about in the cell cycle lectures.
There are actually two different cyclin-CDK proteins which phosphorylate RB and inactivate it.
So when you need to divide you inactive this brake on the cell cycle and the cells can divide.
When you need to stop dividing you activate certain growth inhibitory pathways that inhibit the kinases that block the phosphorylation of RB so RB stays in its active state.
And now the cells cannot divide anymore.
So RB is a molecular brake on the cell cycle.
RB mutations, and mutations in other components of this pathway occur in about 90% of human tumors.
It's a very, very commonly affected pathway in tumor development.
What kind of mutations would you expect to see in the RB gene in human cancer?
Would you expect to see gene amplifications, translocations that cause too much of the RB protein to be produced?
Does that sound right?
Remember, it's a brake.
So what kinds of mutations do you expect to find?
Nonsense mutations.
Or more generally inactivating mutations.
Nonsense mutations.
Maybe deletions which would remove the gene all together.
Now, are these mutations dominant or recessive?
Inactivating mutations are recessive.
So what's going on here?
We all have two copies of the RB gene, and yet we're saying that the mutations that take place in this gene are loss of function mutations, inactivating mutations.
So how is it that you ever find RB mutations in cancer?
Well, the answer is that one mutation is not enough.
One mutation of one copy of the RB gene is not enough.
For tumor suppressor genes were the mutations are recessive, you need two hits.
Two mutational events are required to now fully deprive the cell of the brake.
One mutation is not enough.
And this is now known in the field as the two-hit model of tumor development, at least for tumor suppressor genes.
And it's illustrated here.
Because we carry two copies of all of our genes, two mutational events are required to eliminate the function of a tumor suppressor gene.
And this is what it looks like.
If this is a normal cell which has two normal copies of the RB gene, the first thing to happen is that one copy of the RB gene incurs a mutation.
It might be a nonsense mutation or a deletion, but whatever it is it eliminates the function of the RB gene.
But this cell itself is normal because it still has one normal copy of the RB gene and this is a recessive mutation.
So this cell here is actually normal, but it's one step away from lacking the RB gene all together.
And there are a number of ways that that second copy of the gene can be lost.
And they're illustrated by these different arrows.
The simplest thing to think about is that that second copy of the RB gene picks up its own mutation.
We call that a de novo mutation.
And now you have a cell which has two independently mutated copies of RB, and that's a cell that now has lost control of proliferation.
But there are other mechanisms, which are illustrated here, where the second copy of the RB gene is actually lost all together because the chromosome that carries that gene is itself lost giving rise to a cell that only has one copy of the chromosome.
And it's the one that has the mutant RB gene.
Or there's a recombination event which replaces the good copy of the RB gene with the bad copy of the RB gene giving rise to a cell that is lacking functional RB once again.
And this term here is called loss of heterozygosity or LOH.
And it's a hallmark of tumor suppressor gene mutations.
It's an indication that there's a tumor suppressor gene mutation in the region.
It's usually detected not by a change in the tumor suppressor gene itself but rather by some linked polymorphic marker like a SNP which is close by to the RB gene or whatever tumor suppressor gene it is.
This individual has a big A, little A, and therefore is heterozygous.
He or she is heterozygous, big A, little A.
But if you notice in the tumors, if the tumors arise by chromosome loss or by mitotic recombination, the individual has lost the little A allele.
Either there's only big A or there are two copies of big A, in which case we've seen loss of heterozygosity, LOH.
And that's a common feature of tumor suppressor gene mutations.
And I suspect you'll hear about it in problem sets and beyond.
So LOH is an important mechanism which allows us to go from one hit to two hits.
OK. Now, RB is not the only tumor suppressor gene that's mutated in cancer.
And proliferation is not the only process that affected in tumorigenesis.
As I told you, at the end of the day in cancer we care about how many cells we have.
And you can have too many cells by increasing proliferation.
And the RAS mutations and RB loss are examples of how you can have too much proliferation.
But you can also have too little cell death.
And we actually have examples of both oncogenes, which are activated to prevent cell death, and tumor suppressor genes that are lost to prevent cell death.
So let me tell you a little bit about those.
The first gene that was associated with apoptosis and cancer was this gene called Bcl2.
It's related to one of the first apoptosis genes discovered in the Horvitz lab here at MIT, and it's involved in regulating caspases.
You've probably forgotten by now but caspases are proteases that are kept normally inside your cells in an inactive state.
And they get activated by cell death signals.
Signals that would trigger the cells to commit apoptosis which causes the caspases to become active.
And this then leads to the program of cell death.
The Bcl2 gene functions to block, downstream of the death signals, the activation of caspases.
Is Bcl2 an oncogene or a tumor suppressor gene?
Does it get activated during cancer or inactivated during cancer?
Activated.
You want to suppress cell death in tumorigenesis.
And Bcl2 does that, so Bcl2 gets activated.
It's an oncogene.
And it's typically activated by translocation, in the cases where it is involved.
Translocation, like with myc, too much Bcl2, inappropriate cell survival.
Another very important gene in tumorigenesis that regulates apoptosis is p53.
p53 is actually the most commonly affected gene in all human cancer.
50% of all tumors -- -- carry p53 mutations.
That's a remarkable number.
And p53, we now know, is an important regulator of apoptosis.
It doesn't *only* regulate apoptosis but it is an important regulator of apoptosis.
p53 is like a molecular policeman inside your cells.
It's present at very low levels normally, and it's responsive to various stress signals.
When cells get stressed, for example when their DNA gets damaged or they recognize that they're proliferating abnormally or they're starved of oxygen, the levels of p53 protein go way up.
And when that happens the cells either arrest to try to repair the damage or they will commit suicide killing themselves so they cannot go on to become life-threatening tumors.
So is p53 an oncogene or a tumor suppressor gene?
Is it activated in cancer or inactivated in cancer?
Inactivated.
It's a tumor suppressor gene.
Cancer cells don't want to die so they get rid of p53 to prevent the death.
So p53 is a tumor suppressor gene and a very frequently mutated one in human cancers.
You can study p53 in a variety of ways.
My lab has been studying it for a dozen years using mice that carry mutations in p53 that were generated by gene targeting technology.
You don't need to know the details of this, but suffice to say it's possible to create mice which are either heterozygous for a p53 mutation or homozygous for a p53 mutation.
And the question we asked in those experiments is would these heterozygous mutant mice or homozygous mutant mice be cancer prone?
Should they be?
Would you expect an animal that's lacking p53, one copy or both, to be cancer prone?
It's a tumor suppressor gene so, yeah, absolutely.
And sure enough it is.
If you look at a population of mice and plot their percent survival as a function of time, one year, two years, three years, mice live about three years, normal mice with two copies of p53 will live a normal lifespan.
Mice that have one functional copy of p53 die sooner.
They live about a year and a half.
And if you look in the tumors of these mice, the normal copy of p53 is missing, which is the prediction of the two-hit model of tumorigenesis.
You need to get rid of both copies.
And p53 minus-minus mice only make it about four to six months and die from cancer.
So p53 is an incredibly important tumor suppressor gene.
If you're mutated in it, your risk of getting cancer goes way, way up.
OK.  Now, in the last five minutes I just want to mention the fact that cancer, while typically a sporadic disease -- -- by which I mean there's no family history -- -- about 5% of the time there's familial predisposition.
Cancer gets passed through the family, through the generations.
And you can see that in this pedigree here.
This is actually a pedigree of familial retinoblastoma.
The individuals who inherit the mutant copy of the gene most of the time will develop the tumor, and they'll pass along that predisposition to their children.
If you were to look at this pedigree for a while, you would see that it looks like an autosomal dominant pedigree, if you remember back to that portion of the course.
If you inherit the mutation, you have a high likelihood of developing the tumor.
And it doesn't matter whether the mutation is coming from your mother or father or whether you're a boy or a girl.
What's surprising, despite that fact, is that most familial cancer syndromes like that one are caused by tumor suppressor gene mutations.
So what's being passed through the generations here is a mutant copy of a tumor suppressor gene, the RB gene, or the breast cancer susceptibility gene, or colon cancer susceptibility gene.
A loss of function copy of that gene such that these individuals who are developing cancer and passing the mutation onto their kids are heterozygotes.
And, as you think about it, you have to wonder why are they cancer prone?
Most of their cells should be normal because most of their cells are heterozygous for the mutation.
But, importantly, those cells are one step away from mutating the other copy of the gene.
In contrast to the general population, it carries two normal copies of the tumor suppressor gene.
And in whose cells two mutational events have to happen in succession to mutate the first copy and then the second copy.
In these people all of their cells carry the first mutation.
They're only one mutational event away.
And the likelihood that that will happen in the retinas of their eyes or in the developing breast or colon is sufficiently high that there is a near certainty that they will develop cancer.
OK?
So what event must happen during tumor development?
A mutation in the second copy of the gene, one of these LOH events or a second de novo mutation.
I also wonder what might explain the fact that this gentleman here, who must have the mutation because he passed it onto his sons, did not himself develop the tumor.
Why is that?
What's going on in that guy?
Excellent.
That's exactly the way I phrase it, he was lucky.
In his developing retina by chance no cell underwent the second hit, and so his cells were just like normal and he developed normally, but he still passed the mutation onto his children and they weren't so lucky.
And, finally, what would happen if two individuals who were heterozygous produced a homozygous mutant offspring?
What do you think would happen in those individuals?
Well, if they survive they should be mighty cancer prone because no mutational event is required to eliminate the function in them.
They might look like these p53 minus-minus mice.
In fact, most tumor suppressor genes are required for normal development.
So you cannot do that experiment.
The embryo would never survive because the gene is important for proliferation control all over the place.
So the embryo doesn't make it.
So most of the time you cannot do that experiment.
And, finally, I'd mentioned last time and I just want to emphasize that the genes that we've focused on for you in this class are genes that involve proliferation control and cell death.
And we'd mentioned briefly DNA damage control.
But there are many other processes that are important in tumorigenesis that are responsive to mutations in other kinds of genes like the process of invasion and angiogenesis and the all important process of metastasis.
So I'll stop there.
The following content is provided by MIT OpenCourseWare under a creative commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu In particular, we have a class quiz up there.
Take a moment if you haven't.
You've heard a lot of these terms before.
One you haven't.
I have a particularly wonderful item to make up for the tough exam.
It is a flashing, isn't that so cool?
It's a flashing jellyfish ball.
OK, so let's see what we can do.
Yes, I know, this is a post-test item.
OK, settle down.
Please, we have a lot to cover.
You've heard most of these terms before.
This lecture is all about these terms phrased in a different way.
So I want to make sure we are together before we start.
[Student speaks] Yes, ma'am?
OK, so the final function of a cell, OK. Potency in the pink top.
I saw your hand first, yeah?
Yeah, OK, potency.
This is a very big one in stem cells, the number of possible fates that a cell can assume, commitment, decision to make a cell type, differentiation, someone on the side of the room must have some thoughts.
Yeah?
OK, I'll give that to you, to the final transition here.
Do you want one that's orange rather than pink?
God knows I don't want to be genderist here, but there we go.
The differentiation, the process by which a cell assumes its final fate.
And what we are going to discuss today is the kind of gradations of commitments in differentiation, commitment particularly, some of the steps of commitment.
The last one we have not discussed in class, but if anyone wants to have a go, I will, yeah?
Lineage, the term lineage in biology.
Ish, ish.
Let's try another one.
Good, OK, a group of cells or a group of cell types that descends from a common precursor, so, this is a funny term, lineage, because it's used a lot in stem cell biology.
Let's think about it for a moment.
The definition I have is the set of cell types that normally arise from a precursor cell.
Now, of course, if you think hard, all cells arise from a fertilized egg from a zygote.
So, really, all cell types are of a common lineage.
And this gets to the question of semantics, and where you put your definition boxes.
OK, in stem cell biology, and a lot of biology, the term lineage refers to a subset of cells arising from a common precursor that is somewhat arbitrarily defined.
All right, so this diagram which you've seen over several lectures is really important.
That's just what we've been talking about, the progression from uncommitted cells through committed cells through differentiated cells.
And so, we are in the how-to two module, talking about stem cells.
Stem cells are in the news all the time, and I want to, today, take you beyond the newspapers, beyond the magazines, and tell you what I think stem cells are all about.
So, if you look at our game board of life, we are, I would say, halfway plus through the semester, and we've gone through all of these things, sticking out here into stem cells.
So, where are we?
OK, so what's a stem cell?
Let me go back for one moment, and let's talk about stem cells.
For those of you who don't have a handout, you really should get it.
It's two pages up front here.
So, let's do some board stuff, stem cell, which I will heretofore abbreviate SC is a cell type that I am going to refer to as somewhat committed.
So that's deliberately vague.
This is not a committed cell type, but it's somewhat committed.
It knows more or less what it's going to become.
It can be either unipotent or pluripotent or totipotent.
And that stem cell, by definition, will go on and make an asymmetric cell division.
So, it will give rise to itself, another, not itself, but another stem cell.
And it will also give rise to a committed, and here's jargon you should know, progenitor.
And so, this is an asymmetric with one S, asymmetric cell division.
So, you'll end up with two daughters that are different from one another, a committed cell, and a stem cell.
And so, the stem cell in a sense is self renewing, or is self renewing.
And then, this is one of the big deals about stem cells.
That committed progenitor will go on to divide, and it eventually gives rise to one or more differentiated cell types.
OK, in those differentiated cells will come from a specific lineage.
So, the stem cell is undifferentiated.
The committed progenitor is undifferentiated, and eventually you'll end up with a differentiated cell.
You can look at your first handout.
Here it is; I've drawn it for you out.
You need to know this, so I've done it in two places.
There is the asymmetric division stem cell plus progenitor, and the progenitor goes on to give these differentiated cell types.
There is no magic about this.
This is the same process we've been talking about all along except for the fact of the self renewing thing.
So, in the normal embryo, cells go on down the pathway of commitment towards differentiation.
But they generally do not renew themselves.
So, a zygote, for example, although it's totipotent is not considered a stem cell because it doesn't self renew.
And that's true of most embryonic cells.
They do not self renew and so they are not considered stem cells.
So, what's the deal with stem cells, and why, when I Googled stem cells in Google News last night did I come up with several thousand entries for the previous week?
This is the deal.
One of the great goals of biologists is to repair organs, to repair damaged tissues.
And this has been very difficult to do.
We touched on this a few lectures ago., stem cells have some kind of promise a some kind of universal repair kit.
And the idea is this, that one could isolate some kind of magic stem cell, or some kind of stem cell, and it would be autologous, which means it would be matching your own cells.
OK, so you isolate some kind of autologous, self matching stem cell.
That stem cell under ideal conditions is pluripotent, or that set of stem cells, and can be coaxed to form many different cell types depending on how you treat it.
So, you can imagine adding some kind of factor.
And if you like, you can call that an inducer as we have been talking about secreted factors in developmental biology.
And that inducer would turn those stem cells into progenitors.
And those progenitors would have a specific -- -- future fate by virtue of which factor you treated the stem cells with.
You would then take those progenitors, inject them into someone whose body needed repair -- And the notion is that these progenitors would go on to differentiate and repair whatever needed repairing.
OK, so this is the dream.
And this is why you can find headlines like this everywhere.
These are some of the ones I pulled out last night: Stem Cells May Help Repair Stroke Damage.
Stem Cells May Repair Broken Bones.
Note the use of the term may.
OK, there's a lot of hype about stem cells and not much that's been proven.
There's a lot of money involved in stem cells, trying to isolate stem cells to repair people, and there is also, there are also advertisements where you pay people to take your stem cells, and to store them.
So, poured blood refers to the blood of the umbilical cord of a newborn, which is believed to be, it was known to be full of stem cells of a certain kinds.
And you can pay someone a couple of thousand dollars to freeze that group of cells and keep it in case the child needs some kind of stem cell therapy in the future.
So, what's the deal?
Do stem cells actually exist?
Well, yes, OK. Do stem cells exist or are they just hype?
They do exist, and the idea is that sometime in embryogenesis, as cells are normally going on and making the different cell types, some cells are put aside, that the body is going to use later on for repairing itself.
And there are some examples, and I will tell you a couple which are really fantastic illustrations of this.
So, do stem cells exist?
Yes.
Where?
Likely in the older embryo, and the adult, and the particular organs that contains stem cells is not clear.
There are several very good examples, but it's not clear whether or not some, yes, all, not clear.
It's not clear whether all organs contain stem cells.
So, let's look at this a bit more, and let's go back to talking about how stem cells were discovered.
Why are we having this conversation?
And I'm going to be referring to number two on your handout.
So, let's talk about the discovery of stem cells.
The discovery of stem cells was an accident, and it came about when people started looking to see how long cells lived in a particular organ.
And the way you do this is by using a protocol called a pulse chase experiment.
And a pulse chase experiment gets to the question of turnover rate, or if you want, half-life of cells in an organ.
OK, this is the way it goes.
You've got this as number two on your handouts.
You take a cell population, or you just take the whole organism if you like, you feed it something called bromodeoxyuridine.
It doesn't have to be this, but this is a good one.
Bromodeoxyuridine is a nucleotide.
Deoxyuridine, so, you know uracil is normally an RNA, but if you make the deoxy form, it gets incorporated into DNA.
The bromo part allows it, later on, to be detected by various colorimetric assays, and you can add BRDU to an organism for a short time.
This is called a pulse.
The BRDU is incorporated into DNA of those cells that are undergoing DNA synthesis, and then by various means you wash out the BRDU.
And what you get is the labeled cell population that had undergone DNA replication during this pulse period.
And then, if you follow this group of cells over a period of hours or days or weeks or months or years, you can look and see what happens to those labeled cells.
So, in my example, I started off with for labeled cells over some period of time, the number of labeled cells per total unit number of other cells decreases by half.
And that gives you the half-life of the population.
And if you do this for many organs, you find that adult organs do not just sit there with a cohort of cells that doesn't divide.
There is an enormous amount of cell division in adult organs.
And we talked briefly about this previously.
So, for example, red blood cells have a half-life of about 120 days.
And that actually means that there are more than ten to the 7th new cells produced per day.
The intestine is something we touched on previously.
Cells in the intestine, some of the cells in the intestine had a half-life of three to five days, and you're producing about ten to the tenth new cells per day.
Obviously, the number that you're producing per day depends on the total size of the population.
Skin has a half-life of about 14 days, hair on your head, a half-life of about four years, and your eyebrows and eyelashes, half life of about 30 days.
One of the mysterious half-lives are the neural cells, your nerve cells.
It was believed for a long time that nerve cells never divided, and once you've got all your nerve cells by about age two, you never made any more.
In fact, that doesn't seem to be true.
And there are certainly populations of neurons that divide.
But we don't really know the half-lives for those cells.
So, this was very interesting data, and it said that there had to be some way that the organism was using to replenish these cells that were dying, and to repopulate the organs so that things functioned properly.
And in fact, you can go further than this because you can not only count the labeled cells, you can ask what those labeled cells become.
So, you can look at your labeled cell population and assay the fate of those labeled cells.
And if the cells want to differentiate from an initially undifferentiated population, you know that these differentiated cells must have been derived from stem cells or from progenitors by definition.
OK, so what organs do we know have got stem cells?
Let's talk a bit about this.
The test is a great example, and the spermatogonia, the diploid precursors of the spermatozoa are dividing cells that divide throughout life, and go on to give rise to themselves so they can replenish themselves.
And they go on to give to these primary spermatocytes, which are the first step in the cascade or in the lineage of cells that are going to differentiate as a spermatozoa.
OK, and you know you can do these.
You can look very clearly, and mathematically look and see the number of cells, do this pulse chase analysis, and know that the spermatigonea must be stem cells.
This is a very important stem cell lineage.
It's the hematopoietic lineage.
In the bone marrow, there is some kind of pluripotential hematopoietic cell that gives rise to all of myeloid progenitors, so all the red blood cells, and the various other cells in the blood stream as well as to all the immune cells.
And those come from a single progenitor, a single pluripotent cell.
This is called the hematopoietic lineage.
And we will talk more about that in a moment.
This is an example that I mentioned to you many lectures ago.
Newt limbs, if they are amputated, will regrow.
The reason that they regrow is that there seem to be a population of cells in the lab which are called blastema cells, and those are the cells which can go on and to form the entire limb again.
And that's something obviously we can't do, but people are very interested in.
That's been a tough system to look at.
This might be a better system to try and understand the details of regeneration.
So, this is a planarian.
This is a flatworm.
These are little guys.
They can grow up to a couple of centimeters, or a few centimeters.
They are very pervasive animals in the animal kingdom.
And they have this extraordinary property, as many simple animals do, that you can cut them into pieces, and they will regenerate the whole animal.
So, you can take out the head.
And over time, it will regenerate the tail.
You can take off the middle; it will regenerate a whole animal, and so on.
And this bottom picture is a planarian that's stained for BRDU incorporation, or for another marker as well.
And each of these dots is a cell called a neoblast.
The neoblasts are the stem cells of planaria that can regenerate the whole animal.
And if you look right up front here, it's not very distinct, but you will see a region where there are no neoblasts.
And that is the one region of the animal that cannot regenerate.
So, if you cut off the very tip of the sort of nose equivalent region.
It will not regenerate a new animal because there are no neoblasts.
And the system is being studied here at MIT by Professor Reddien of the Whitehead Institute, who is a new faculty member, and who some of you might went to Europe with sometime.
OK, so let's talk about isolating stem cells, and how you do this.
If one is going to use stem cells for repair, you've got to be able to isolate these things.
And the challenge of isolating stem cells is that they are rare.
For example, in the bone marrow, the hematopoietic stem cells, which I'm abbreviating HSC, comprise about 0.01% of the bone marrow.
And I would say it's fair to say that no one has really seen a cell and said, oh, this is a stem cell.
It's hard to pinpoint exactly what a stem cell really looks like.
It's just a cell.
But it's got particular properties, and you need the appropriate way to look at the cells and see these properties.
One way that's been used to isolate stem cells is something called FACS, or fluorescence activated cell sorting.
I'll talk about it in a moment.
And there are two ways that FACS is being used to isolate stem cells.
One is by getting a group of cells called SP cells where the SP stands for side population.
We'll talk about that more in a moment.
And the other is through the use of cell surface proteins that are characteristic, or enriched in stem cells.
And this has been many decades of work by many people to come up with a set of criteria by which one can enrich for stem cells in a particular population.
So, here's the way.
Actually, before I go through that, let me go through the assays, and then we will go through all the slides together.
How do you assay for stem cells?
Well, one of the ways you know you have a stem cell is if you have something that can act as a stem cell.
And there are two assays that you should be aware of.
One of them is a repopulation assay usually done by transplant, and very often you remove some endogenous group of cells, and then try to replace that group of cells by using stem cells.
So, you remove some set of differentiated cells.
And then, you try to replace with transplanted stem cells.
And the other way you can do this is in some kind of in vitro culture assay which I will talk about later on.
All right, so with this in mind, let's go through some of the slides.
You've all heard of bone marrow transplants, which people who have leukemia and other associated disorders undergo to repair themselves to get rid of the leukemia cells to get rid of the cancer.
This is how it works in a mouse.
And this is a repopulation assay.
I'm going to use bone marrow transplants as an example.
You take your mouse, or your person if you are undergoing bone marrow transplants.
And you irradiate to destroy the bone marrow.
The reason you do that is to make space for new cells to come in to expand and grow.
If you just put your new cells into an animal with an intact bone marrow, they kind of disappear amidst the masses.
So, you have to give the cells your assaying a chance.
And then, that irradiated mouse would die.
That irradiated person would die.
But you get them, now, an injection of normal bone marrow.
And if things go well, the mouse or the person lives.
And there was a Nobel Prize given out some years ago for developing bone marrow transplantation.
One of the gold standards in the stem cell field is asking whether or not this rescued mouse has regenerated stem cells because you can imagine that what you're doing in this case is giving cells that are the progenitors.
They are one step down from the stem cells.
Or they might even be partly differentiated.
So, you can imagine that you are restoring this mouse is life by giving cells that are not self-renewing.
Excuse me, one of the gold standards in the stem cell field is to take this rescued mouse, isolate more stem cells or more putative stem cells, and take those and then tried to rescue another mouse.
And in the hematopoietic system, you can do this over and over again.
So, how many stem cells do you need to repopulate a mouse?
Actually, you need one, and I will tell you how this is done.
So, the idea of the first thing when you are trying to do assays to figure out how many stem cells you need for rescue.
You take your bone marrow; you stain it for some stem cell marker.
You have this in front of you.
OK, this is number nine of the first page of your handout.
You stain somehow for a stem cell marker.
I'll tell you in a moment how you sought the stained cells through this fluorescence activated cell sorter.
And you isolate from it a pure-ish population of stem cells, an enriched population of stem cells.
And then, you do a dilution assay where you inject different numbers of cells into a recipient irradiated mouse.
Now, you don't actually injects one, ten, and 100 cells.
You inject one cell, and millions of carrier cells to help those cells along, OK?
The one cell would just disappear in your syringe.
So, you've got to give it some companions.
But the bottom line is you really only need one stem cell to rescue the life of that mouse.
It can go and repopulate the entire hematopoietic system, all the blood cells, all the immune cells.
What is this fluorescence activated cell sorter?
Fantastic machine.
It looks like this.
Again, you have this in front of you, so look up here.
The idea is that you take a reservoir of cells that you have labeled with particular antibodies.
And these are living cells.
OK, you label them in certain ways, and they can be labeled with fluorescent antibodies or fluorescent dyes.
I'll tell you a dye example in a moment.
And you put them in a reservoir, and you trip them out so that one drop of liquid contains one cell on average, or zero cells.
And you let those cells drip through a laser and a fluorescence detector.
The laser activates the cells.
They fluoresce, and you set the detector to activate a charging collar at a specific wavelength of fluorescence.
And when the charging collar is activated, it will activate some deflecting plates which will give charge to cells of particular colors, and move them into particular tubes so that they are sorted on the basis of their color.
OK, this is done one cell at a time but it's really quick.
And you can purify millions of cells to do these kinds of assays.
This kind of thing is not done for human bone marrow transplants.
There, you take a much bigger population of cells from the bone marrow, and you generally don't purify them very much.
But if you want to do specific stem cell assays, and many others, the fax machine is really fantastic.
So, what do you get out of this?
Well, you can plot what the cells look like.
So, here in this example, I've got cells that are labeled in red and green.
And you can label them according to this plot as to whether they have no label, red label, green label, or both red and green.
This is a real example of an experiment that was done here at MIT a decade ago in Richard Mulligan's lab.
And this is what a real FACS plot looks like.
OK, it's a mess.
Every little is a cell.
But for reasons known only to the Mulligan lab, Peggy Goodell who is now a professor in her own laboratory, they assayed this little region of cells in the bottom left-hand corner for their stem cell properties.
They called these SP cells or side population cells, and they found to their surprise that these SP cells were highly enriched for hematopoietic stem cells 1,000 fold or more.
And in fact, if you take the very bottom left-hand corner where there are almost no cells, you get a 10,000 fold enrichment.
So, the way they sorted these was with a dye called Hest 3342.
This is a vital dye.
It stains the DNA but it doesn't tell the cells.
And somehow, these SP exclude the dye, or efflux it, remove it from the cell.
And it's really not clear what that has to do with stem cellness, but this is still one of the very best ways to isolate stem cells from almost every organ.
These SP cells, these things that don't stain with these DNA dyes seem to be the ones that are stem cell-like.
OK, so let's move on to the question to several questions that I want to discuss with you that fall under the umbrella of regulation and control of stem cell fate.
And there's several questions I like to pose to you.
Firstly, what makes a stem cell self renewing?
What's the molecular basis for that?
What makes a stem cell decide whether it's going to make progenitors or not?
And the big one for the stem cell field, what controls the potency of a stem cell?
Well, this is where we step back into the developmental biology that we've been talking about because it all controls at some level by gene expression.
And in particular, there are both intrinsic and extrinsic factors that seem to control certainly the first two points on my list, the self renewing and the progenitor aspect, and perhaps also the potency also.
Intrinsic factors, cell autonomous factors, determinants -- -- and extrinsic factors, non autonomous factors, secreted ligands, inducers, and these and extrinsic factors have been given a special name in the stem cell field just for argument's sake.
They are called the niche where the niche contains all the cells that influence stem cell activity, cells that influence stem cell activity.
OK, this is just a term.
You should know it because if you read it you will know it.
So, here's how it works.
The surrounding cells in the niche are cells that seem to maintain stem cells usually in acquiescent state.
So, it's believed that in most organs, stem cells are sitting there quietly.
They're not dividing very much, but they can be stimulated to divide, and this is on the second page of your handout.
They can be stimulated to divide by some kind of environmental input, and this changes the surrounding cells.
And the environmental input could also be a secreted ligand for example.
And the surrounding cells then induce the stem cells to be activated.
And they go on to make progenitors, and of course also to self renew.
This is a fancy way of saying that cell fate is controlled by induction.
OK, so you should be nodding.
The should not be anything new for you at this point.
It's phrased a little differently, but that's all.
This is a very interesting example of control of cell fate by surrounding cells.
This is from my colleague, Professor Fuchs at Rockefeller who studies the hair follicle, and who has shown that this region called the bulge is the source of stem cells.
So, this brown thing is the hair that sits in a shaft of cells that got a bunch of interesting cells around it.
In particular, there's a rather inconspicuous group of cells on one side called the bulge.
And some years ago, there is another group of cells.
I'm also going to refer to two at the bottom, which is called the dermal propeller.
But some years ago, Professor Fuchs took the cells of the bulge and she transplanted them into a mouse that didn't have any hair.
It's called a nude mouse.
It has lots of problems including no hair.
But when she did that, here's the control and here's a mouse into which these stem cells have been transplanted.
And you can see all of a sudden this poor little nude mouse has got tufts of hair.
OK, and in fact, these hairs actually glow-in-the-dark.
They've been labeled with green.
It's really cool, OK?
The cells that were transplanted, this is something you know, as well.
They were lineage labeled, and so, you could prove that these hairs came from the transplanted cells.
All right, so let's look.
So, during the hair cycle, so you're hairs grow cyclically.
And there are a whole bunch of processes including growth, regression, induction of growth, and new growth.
And during that period of time, the bulge and the dermal papillae are in relatively different positions.
So, during a process of growth, they are far away from each other, and then as the hair and growth regresses, they come closer until they're actually touching each other during the process of induction of new growth.
And it's at that time that new growth in the hair is stimulated.
And it's clear that the dermal papillae is signaling to the bulge cells.
And here's what it's using to signal.
It's using something called the wind pathway, and in particular, a molecule called beta-catenin.
If you think back several lectures, we talked about beta-catenin is one of the things that told the embryo to make its back rather than its belly.
So, here's a different use of the same molecule in stimulating hairs to grow.
And so this is a particularly cool example of cells coming together at particular points to stimulate stem cells to go on and to make progenitors.
All right, let's move on to something important called embryonic stem cells And all right -- -- also called ES sells.
So, although I told you that most adult organs are likely to have stem cells, and this has been very clearly shown for many, there are many organs where it's not clear whether they have stem cells, or it's very difficult to isolate them.
Stem cells are rare in all organs, and things like neural stem cells in the nervous system seemed to be a exceedingly rare and difficult to isolate.
So, the push has been to try to find a source of stem cells that would be more plentiful and more useful for repairing lots of different organs.
And that's where the embryo comes in this particular kind of stem cell called an embryonic stem cell.
And the idea is if one takes an embryo or part of it, puts it into culture, after many steps you get out cells that are called ES cells.
And these ES cells are pluripotent.
They are not totipotent.
But they are very, if I can be forgiven, they are very pluripotent bouquet, and they are pluripotent, and you can control their cell fate.
So, let me going to the slides, and we will talk more about this through the slides.
So, in mammalian development, and we're going to talk about mammals here specifically, mammalian development, the blastula forms.
And at a certain point in development, a group of cells called the inner cell mass segregates from the rest of the cells which form a shell around the embryo.
We mentioned this previously.
This yellow stuff is fluid, and this little group of cells called the ICM, or inner cell mass, is the thing that's going to give rise to the embryo proper.
The cells surrounding are going to give rise to the placenta and the other extra embryonic components.
So, the idea in trying to get these ES cells is to take the inner cell mass of an early embryo, take it out of the embryo, and put it in a Petri dish that's got nutrients and various factors to disperse the cells such that you've got single cells dispersed in the plate, and give them nutrients.
And over time, those cells will grow, and they will form clumps, each of which is derived from a single embryonic cell.
OK, now, normal embryonic cells do not do this.
OK, they will normally go on and differentiate, and stop dividing, but there's something that happens during this culture process that is abnormal.
And it turns the cells into groups of cells that can self renew.
OK, so you've turned these cells into self renewing cells.
And each of these groups of cells or colonies that you get may be able to grow into a stem cell line.
And I'll talk about cell lines in a moment.
I'll talk about cell lines now.
So, you might not be familiar with the term cell line.
What is the cell line?
A cell line is a cell population, a homogeneous cell population that could grow continuously in culture.
So, it's self renewing.
OK, so all cell lines are self renewing.
A stem cell line is a cell line that has the capacity to go on and differentiate into specific normal cell types.
So, there are many cell lines that will grow continuously in culture, but they will never go on and differentiate as anything.
They are very abnormal cells.
They are useful for many studies, but they are not stem cells.
The stem cells not only can grow continuously, but they can go on and differentiate.
But you are dealing with an abnormal cells here.
This is not a normal embryonic cell.
OK, how do you test the potency of these ES cells done in the following way?
The ES cells from a mouse, a black mouse, are taken, and they are injected into an early embryo of an embryo derived from white parents.
And if you do that, the ES cells incorporate into the embryo.
You then take the embryos, and put them into a surrogate mother.
And when the mice are born, when the babies are born, you can see that they often are not just pure white.
They often got black stripes, and you can look at the various organs and show that these ES cells have incorporated into various organs.
So, these ES cells are highly potent.
And the idea is that you can take these ES cells and add to them various factors.
So, you can add a particular red factor that will turn an ES line into heart muscle cells or pancreas cells or cartilage cells.
And you can use those cells in your stem cell repair assays.
OK, and this is true.
You can take ES cells, and you can do exactly this to them.
And then, they will become these different kinds of differentiated derivatives.
So, this is really cool.
And the push has been to try to get this to work not for mice but for humans.
And this is where the huge controversy in the stem cell field comes from.
So, the controversy comes from the fact that you need embryos to get these stem cells.
You get the embryos from in vitro fertilization that we touched on a few lectures ago.
Eggs are isolated by ovarian stimulation.
They are fertilized in vitro, and they are allowed to grow in vitro for a week or so.
And at that point, there are harvested or killed to make the ES line.
And this is the very controversial point, whether or not it is ethically OK to harvest these embryos, and turn them in to stem cell lines or not.
Presently, there is no federal funding that is allowed to be used to make human ES cell lines.
There are some that exist, and President Bush has told scientists that they need to use the ones that exist.
However, they are not very good cell lines, and so scientists have used private funding to make new human ES lines that hopefully will be more useful.
I don't think there's any right or wrong answer whether this is OK or not.
My opinion is that this is an OK thing to do, with all due respect to the embryo.
I think one can save people's lives, well, and the embryo at this point is not a differentiated entity.
But it is an embryo.
And so, this is an ethical issue.
There's opinion here, and it's good for you to think about what your opinion about this type of research is.
OK, so let's move on.
And I'm going to, in the last minute, just touch on something called stem cell plasticity.
One of the things that has come out of the human ES work or all the ES work, and that has come out of the quest not to use embryos for stem cells is to ask whether or not one can turn one stem cell line into another.
So, it's easy, or relatively easy, to get hematopoietic stem cells.
And wouldn't it be wonderful if you take those hematopoietic stem cells and turn them into brain stem cells, and fix people who have Parkinson's or Alzheimer's?
OK, wouldn't it be wonderful to fix people who have muscular dystrophy by turning hematopoietic stem cells that you can get lots of into muscle cells?
So, the idea is maybe you could turn something from one lineage, a hematopoietic stem cell into a stem cell from another lineage and get it to do something else?
And the hypothesis, then, is that if you do appropriate experiments, you may be able to figure out where the stem cells from one lineage can contribute to another.
This is the second to last slide on your handout.
So, this is the way the experiments were done.
It's the last thing I'm going to tell you.
Bear with me.
You can take a mouse that's been dyed green, and its hematopoietic stem cells are expressing GFP, which is a green fluorescent protein.
You use that mouse as a daughter for bone marrow transplants.
You put in the great cells.
Not only do you rescue the bone marrow, you also ask whether other organs have got green cells in them, indicating that the hematopoietic cells can contribute to other lineages.
And when you do that, initially people got very excited because you saw results like this.
And this is data from my colleague, Dr. Camargo over at the Whitehead Institute.
When this experiment was done, he could show that the liver of such an animal had lots of green cells, suggesting that the hematopoietic stem cells could also be liver stem cells.
But when he looked more closely, this is a complete artifact.
And what had happened was that the hematopoietic stem cells had actually fused with the liver cells, making the liver cells green.
And in fact, presently, there is no data to suggest that you can interconvert stem cell lineages.
And I'll stop there.
Thank you.
While she's writing that up, I also wanted to clarify a point that was raised by a TA last time at the end of last lecture.
Some of you might have thought about this fact, and it's important to clarify at least as best we can.
I told you last time that people who inherit a defective copy of the RB, retinoblastoma tumor susceptibility gene are highly predisposed to the development of retinoblastoma, a tumor of the eye.
Actually, these patients end up getting bilateral retinoblastoma, affecting both eyes, and typically have about a dozen tumors, independent tumors.
And, if you recall, those tumors arise through the loss of the normal copy of the RB gene in the cells that give rise to these tumors.
And I also told you that the RB gene is a critical regulator of cell cycle progression.
And so you might have wondered why don't these people get all sorts of tumors?
Why are they predisposed only to retinoblastomas?
Why not breast cancer, lung cancer, pancreatic cancer and so on?
We don't actually know in complete detail why that is, but we suspect that there's a fundamental difference between retinal cells and other cells with respect to their requirement for RB gene function.
So I told you previously that RB is a regulator of the cell cycle.
Specifically it regulates the entry of cells from the G1 phase of the cell cycle into S phase.
It blocks.
And it has to, itself, be inactivated for tumor development.
Or rather for normal cell cycle progression.
An we think that in retinal cells this is the key regulator of S phase progression, of S cell cycle entry, so that if you get rid of it you now have deregulated cell cycle control and tumor development.
And we suspect that in other cells where RB is almost certainly important -- -- there are probably other factors, let's call them X, which can also regulate cell cycle progression.
So that even if the cell were to lose RB, there are other factors that can, in a sense, back it up.
And in these cells, in most of your cells, although RB loss might contribute to tumor formation, it's not sufficient.
In these cells other events must be necessary to inactivate, one way or another, these X functions.
OK?
So hopefully that helps clarify.
Now, I told you last time, the last two times that we now think of cancer as a clonal progression from normal cells to tumor cells.
The acquisition of mutations in cellular genes, oncogenes, tumor suppressor genes, and collectively these give rise to cells that are malignant and potentially life-threatening.
This is interesting information from a scientific point of view, but is it useful?
Why do we want to understand these cancer-associated genes?
Why do we want to understand these mutations?
Well, there are a variety of reasons why.
We're going to focus today on therapy which is directed against the mutations that arise in cancers.
But there are other purposes that I just want to mention to you.
Early detection.
Cancer is most easily treated when caught early.
If we know somebody has cancer, before the cancer has spread, they have a much better chance of curing that individual.
And so it's desirable to have tests for early detection.
And increasingly there are PCR-based tests looking for cancer cells in bodily fluids.
Sometimes the blood, urine or other tissues.
PCR-based tests looking for mutations in the cancer-associated genes, looking for cells that have a Ras mutation or have a p53 mutation or have an RB gene mutation and so on.
So this is not commonplace, but there are now tests, commercially available tests that are based on PCR looking for such mutations.
There are also blood tests for what we call cancer markers.
You've probably heard of the PSA test for prostate cancer.
There are certain other tests for other types of cancer.
These are blood tests that detect inappropriate levels of something, often something produced by the cancer.
And, again, increasingly, as we understand what happens in cancer cells, we'll have more and more precise cancer makers that will be detectable in the blood in a very simple screening test so that you can go to the doctor every year, go through one of these tests and know whether or not you have an early form of one or another type of cancer.
That's not, again, happening today, at least in a widespread way, but it will happen in the years to come.
And when it does, we will be in a position to do what we call cancer prevention.
Rather than waiting until somebody has a full-blown tumor and trying to treat it, which is difficult, we will hopefully detect those tumors at a very early stage and then prevent their progression.
So this is not treating cancer really, but treating the hyperplasias I told you about.
Or early lesions, benign lesions before they progress to true cancer.
And we think that this will be easier to do because those cancer cells will have acquired fewer mutations.
And, therefore, it will be easier to design very specific agents that will effectively limit their proliferation or possibly even kill them.
OK.  Today we're going to focus on the use of this information for better therapies, ways to design more effective, more specific anti-cancer agents.
And I'll come towards the end to using this information and related information to do better diagnosis to try to distinguish two people who have clinically similar tumors.
But those tumors might actually be quite different at the molecular level, and we'd like to understand that.
OK.  Before we get into sort of the New Age cancer treatments, I thought I should at least mention to you conventional therapies.
Right now, if you have to have cancer treatment, you might get one of the drugs that I'm going to tell you about later in the lecture, but more likely you're going to get what we call a conventional anti-cancer treatment.
And these anti-cancer treatments have actually been around for quite some time, and they do work.
They do work, but they don't work as well as we need them to work.
Radiation is a very common anti-cancer agent, as you probably are aware.
And there are a variety of drugs that we list along with radiation like Adriamycin, Cisplatin.
And there are a variety of other chemical agents, which together are grouped because they cause DNA damage.
These are DNA damaging agents, and they're also effective anti-cancer agents.
There's another category of anti-cancer agents which is exemplified by a drug called Taxol, and there is a series of Taxol-related compounds.
And these are microtubule inhibitors.
Microtubule inhibitors.
And these therefore, microtubules are important in mitosis, if you'll remember the mitotic spindle.
So these are anti-mitotic drugs.
And these drugs do work.
And we think that they work in part because cancer cells are rapidly dividing cells compared to most normal cells in your body.
And, therefore, if you damage their DNA or you block their ability to divide you'll more effectively block cancer growth compared to normal cell growth.
Now, overall, in the context of cancer, what we're looking for, and actually in the context of other diseases as well is something called a therapeutic window.
A therapeutic window is defined as a difference in the concentration of a drug necessary to kill the cell of interest versus normal cells in the body.
These drugs, anti-mitotics and DNA damaging agents will kill normal cells.
So if you look at a graph of percent killing versus drug concentration, normal cells will eventually die.
The hope is that cancer cells will die sooner.
And this difference is defined as the therapeutic window.
And that does exist for many of these drugs for many different types of cancer.
And so these agents will, in fact, give initial responses.
Unfortunately, they tend not to be, tend not be durable.
That is patients tend to relapse.
Not always but tend to relapse in response to these agents.
This slide just makes the same point that many of the drugs that we know about affect the cell cycle either in S phase during DNA synthesis or in M phase during mitosis.
And this also points out that many of these agents cause the death of cancer cells by inducing apoptosis, inducing the death of cells.
In contrast to most, but not all, most normal cells in the body, which in response to that same concentration of drug, will not die.
And instead those cells will arrest at some point in the cell cycle and repair the damage.
And the difference, what makes up the therapeutic window in many cases, is that the cancer cells are dying at a given concentration, the normal cells are staying alive and simply arresting.
However, there are other cells in the body that in response to the same drug at the same concentration will undergo apoptosis.
And you actually know what those cells are if you've thought about cancer chemotherapy before.
It's the cells that support the hair follicles.
Those cells die in response to these drugs.
And that's why cancer patients lose their hair.
It is cells in the blood, in the bone marrow which will die in response to these concentrations.
And that's why cancer patients get anemic.
And in cells of the lining of the stomach and intestine which will die in response to these drugs.
And that's why cancer patients feel sick, feel nauseous.
So there are side effects in response to these drugs.
And that's because many cells, some cells in your body will also die by apoptosis.
Now, we've learned, actually my lab has participated in this process, that the p53 tumor suppressor gene that I've told you about is actually quite important in guiding the responsive cells to these drugs.
Many normal cells turn on p53 in response to this damage and arrest.
Those other cells that I just told you about will turn on p53 and die.
And cancer cells, likewise, if they have a functional p53 gene will turn it on.
And this will induce apoptosis.
And it's the difference between cancer cells turning on p53 and dying compared to normal cells turning on p53 and resting, it gives the therapeutic window.
Unfortunately, as I've mentioned to you, about 50% of human cancers carry p53 mutations.
And given that p53 is important in this response, if you don't have p53 then you won't die, or won't die as effectively, and that limits the therapeutic window.
And this is one of the reasons why cancer therapy is not as good as it should be and why cancer cells will sometimes come back, because they're now no longer responsive to the drug, at least especially responsive to the drug.
So we'd like to do better, and we think we can do better by taking advantage of the information that we've gained over the last 30 years about cancer-associated mutations.
And I'm going to review for you in detail the first three of these new agents, all three FDA approved in the last five years or so for the treatment of one or another type of cancer.
And I'll also mention anti-Ras therapies, although we don't have an FDA approved drug for those.
If there's time I'll mention inhibitors of an enzyme called telomerase, as well as anti-angiogenesis.
There are other therapies, not drug-based therapies but other therapies that are under consideration, and in some places in use.
Gene therapy, replacing cancer mutation genes.
Immunotherapy, trying to convince your immune system to attack your cancer.
And also cancer prevention strategies, which I mentioned actually last time, trying to make vaccines against viruses that are associated with certain types of cancer including human papillomavirus and cervical cancer.
So Ras is the first one that I'd like to mention to you.
And it's an example of where we haven't done enough.
We don't know enough.
Even though we know that Ras is mutated in 30% of human tumors, 30%, 90% of pancreatic cancers carry Ras mutations.
Pancreatic cancer is one of the worst killers in the cancer category.
If you get pancreatic cancer, of a particular type at least, it's a very, very, very serious disease.
We know that these tumors carry mutations in the Ras gene but we cannot do anything about it at the moment.
So, as I've told you, Ras proteins are involved in signaling proliferation.
And this takes place through kinase cascades, phosphorylating enzymes that phosphorylate other enzymes in a cascade.
When you have a mutation in Ras, it turns the protein on in a constitutive fashion leading to increased signaling down these pathways and increased proliferation.
We know some of these enzymes.
I've told you about Raf and MEK and MAP kinase.
And so many drug companies are now trying to find inhibitors that might block those enzymes, small molecule inhibitors, drugs.
And because these are kinases, the approach is to try to find ATP analogs.
Drugs that look like ATP, can get into the active site of the enzyme and compete for ATP, and thereby block enzyme function.
Some of these drugs work, at least in cells in culture.
There's a bit of a fear that these pathways are so commonly used in normal cells that the drugs might be highly toxic and therefore not tolerated.
And, importantly, we don't understand what these arrows mean well enough.
We have some basic ideas, but we don't have enough detail to know exactly which kinase to inhibit in exactly which type of tumor.
So this is in progress but it's not quite there yet.
I'll come back to another couple of stories related to ATP analogs that do work and are now in use in cancer treatment.
Before I do, I want to mention another class of inhibitors, and these are antibodies.
Antibody-directed therapy.
Cancer cells often up-regulate proteins on their surface.
I mentioned one last time in the context of breast cancer.
It's a protein called HER2.
I mentioned the fact that 30% of breast cancers have an amplification of the HER2 gene and, therefore, make more of this HER2 receptor on their cell surface.
So, in contrast to normal cells which will have a certain concentration of this receptor on their surface, cancer cells, breast cancer cells that carry this amplification will have a much higher density.
Maybe ten times or a hundred times the level of this receptor on their surface.
And they are using that increased level of receptor to increase the signal downstream of that receptor to promote proliferation.
Now, the receptor is responding to ligands as it would normally do.
And therapy is based on the fact that the ligand has to bind to the receptor in order to activate it.
And so what was done by a company called Genentech out in California was to make antibodies that block to the receptor, that bind to the receptor and block the binding of the ligand to the receptor.
So these are anti-HER2 antibodies.
And this drug, which is now approved by the FDA, is called Herceptin.
And it works.
For those breast cancer patients who have amplification, too much of this receptor on their surface, Herceptin works and can give them months, sometimes years of symptom-free survival.
It's not curative, unfortunately, but it does extend life.
And it's therefore an extremely important drug.
This is just a blocking antibody.
There's nothing attached to the antibody.
It's just blocking the binding of the receptor to its ligand and thereby blocking the function of the receptor.
But antibodies can also be linked to toxins or radionuclides, and thereby deliver bad stuff to the tumor cell, either a toxin or something that will irradiate this cell.
And these are being tested currently.
There are no FDA approved versions of this, but I suspect that will change in the years to come.
So Herceptin is an effective antibody-based therapy.
There are a couple more now, but it was the first.
And this is actually from the Genentech website which gives you a little bit of information about Herceptin and shows you a bottle of Herceptin as you would see in the pharmacy.
And this diagram is just a reiteration of what I've told you already.
Normal cells have low levels of the receptor on their surface, cancer cells have higher concentrations of the receptor on their surface, and the antibody binds to the receptor thereby blocking its function.
OK?
So this is a clear example.
We learned that Herceptin was over-expressed in cancer, breast cancer and ovarian cancer.
The company made an antibody and it works.
Another story, my favorite story relates to a disease called chronic myelogenous leukemia -- -- or CML.
CML is a disease that affects young adults, adults and children.
Child patient shown here.
It is leukemia so it's a disease of the blood.
It affects both the blood, as well as the bone marrow.
And we've learned a lot about this disease over the years.
It's not a very common disease.
It only affects about 4,000 or 5, 00 people in this country per year.
And it falls in stages.
Initially the person is diagnosed with CML based on relatively low concentrations of, low levels of white blood cells in their circulation.
And then they progress with that phase in what's called the chronic phase where there are still relatively low levels of white blood cells, higher than normal but lower than are dangerous.
However, this can progress over time through an accelerated phase where there's even higher levels of white blood cells in the blood to the final phase which is called blast crisis where the levels of white blood cells really shoot up.
And this is lethal.
And these patients invariably progress through these stages and eventually died.
So what have we done?
This is a picture of what the blood cells look like in a normal individual.
This is a white blood cell you could see in a CML patient.
There are higher levels, and they can be even higher than this.
This disease has been studied for a very long time.
And we now know that there's a signature mutation, a mutation that takes place in almost all CML cases.
It's a translocation that rearranges two genes called BCR and ABL and places them together on a translocated chromosome.
There's a swapping of genetic information from chromosomes 9 and 22 such that there's production of a new gene called BCR-ABL that results in a new protein, a fusion protein that has a little bit of this BCR protein and a little bit of this ABL protein.
And you can see in the karyotypes of these individuals that they have an abnormal chromosome 9 which is a little shorter, sorry, a little longer than it should be, and an abnormal chromosome 22 which is a little shorter than it should be.
And when you look at cancer cells of CML patients you always find that translocation.
It's called the Philadelphia translocation because it was discovered by researchers in Philadelphia.
And it's sometimes referred to as the Philadelphia chromosome.
And, again, it's a translocation involving chromosome 9 which has a gene called ABL which is a tyrosine kinase.
And so it's a signaling protein.
And chromosome 22 which has a separate gene called BCR.
And, in the development of CML, breaks take place on these two chromosomes leading to a translocation and the formation of a new chromosome that has a fusion gene composed of both BCR and ABL.
And this gives rise to a fusion protein with a piece of BCR and the kinase domain of ABL.
And this leads to increased proliferation, as well as increased survival of the cells that carry that translocation, more cells in the blood and eventually leukemia.
And the hope is, the hope was, as this was being worked out, actually important experiments done at MIT in the early 1980s here.
As this was being worked out that maybe, because it's such a common mutation in this disease, if you could find an inhibitor -- -- maybe you could block the proliferation of these cells or perhaps induce their death.
This just gives you a little a bit, a sort of cartoon version of BCR-ABL signaling.
I don't want you to literally pay great attention to this.
Suffice it to say BCR-ABL as a signaling protein stimulates many of the pathways that you've learned about already in this class and causes cells to proliferate, as well as to survive better.
So, again, can you find an inhibitor that blocks the activity of this enzyme and thereby blocks the proliferation of these cancer cells?
This was undertaken by probably many drug companies in the world, but a drug company now called Novartis, which has its research headquarters here in Cambridge, succeeded.
They generated this drug which goes by the name Gleevec.
It has a trade name, the name of which I can never remember, but everybody called it Gleevec when it was being developed.
It was also called STI571 but Gleevec is the common name.
They found this drug through a screen looking for small molecules that look a little bit like ATP, although it doesn't look much like ATP anymore, that can specifically bind to and block the kinase activity of this particular kinase.
And this drug is successful.
It does bind to the kinase and blocks its kinase activity.
And importantly in cell lines, as well as in mouse models, it was found to be effective in killing CML cells.
It was then used in treatment of CML patients and found to be effective there, too.
So the number of white blood cells in these patients dropped dramatically and the number of Philadelphia chromosome positive cells likewise.
So if you were to plot the number of white blood cells in a normal patient it would be low.
In a CML patient, in the early phase of the disease it would be down here, and then it would go up in the accelerated phase and then it would go up still further in blast crisis.
And, as I said, this could take years, several years to progress.
And at this stage, late-stage disease, this person might have hundreds of thousands of white blood cells per mill.
But when treated with Gleevec the white blood cell counts dropped to mere normal.
And amazingly the drug is extremely well-tolerated.
So even though all of your cells have this same ABL kinase, not fused to BCR but the same ABL kinase unfused, and it's probably doing stuff in your cells, those cells don't need it.
But the cancer cells, in the context of this BCR-ABL fusion, are totally dependent on it.
And if you inhibit it now the cells will not proliferate anymore.
And, indeed, as you can see how precipitous this fall is, the cells will actually die, undergo apoptosis.
So the drug is extremely effective.
As I said, clinical tests were done.
Sorry.
This just illustrates a cartoon version of what we've been talking about.
Here's the BCR-ABL protein.
Here it's in its normal state binding to ATP and transferring a phosphate to some substrate protein in the context of signaling.
And what Gleevec does is binds to the ATP pocket and blocks the access of ATP to the enzyme and, therefore, blocks the kinase activity.
And this is actual clinical data provided by Novartis in this case.
And what you're looking at here is the number of Philadelphia chromosome positive cells in the blood.
And what percentage reduction you're seeing, either somewhat or completely, looking at the accepted therapy before Gleevec came along, which was not very effective, only 12% of patients showed any response, or rather a major response, and only 3% showed a complete response.
That is when you looked in their blood by PCR you could find no more Philadelphia chromosome positive cells.
But now with Gleevec, 75% of patients showed a major response.
And 54% of patients showed a complete response, you could not find Philadelphia chromosome positive cells by PCR in the blood of these patients.
So it really worked extremely well.
It gave what we call clinical remissions.
Clinical remissions.
And these patients survived, had, you know, dramatically extended lifetimes.
This would go on for in some cases as little as a half a year, in other cases up to ten years increased survival, especially if the patients were treated early in the disease.
But unfortunately for all the patients the numbers went back up.
Clinical relapse.
An all too familiar problem in cancer therapy.
You might see initial treatments, they might even last a while, but too many patients undergo relapses where their disease comes back.
So what's going on here?
These patients are continuing to receive Gleevec throughout this course, and yet the tumors are returning.
Why?
What might be going on?
Remember that cancer cells acquire mutations?
Cancer cells are always acquiring mutations.
So what kind of mutation might be taking place in these cells that would lead them to be Gleevec resistant?
Well, maybe they're acquiring mutations within the BCR-ABL gene, that fusion gene, which blocks their ability to bind the drug.
So Charles Sawyers, investigator at UCLA, took the cancer cells from these relapsed patients, PCR-ed up the BCR-ABL gene, sequenced it, and lo and behold he found a bunch of mutations.
Individual tumors had one or another of these point mutations within the ABL kinase.
And the consequence of those mutations was that the drug Gleevec could no longer bind.
And that's illustrated here.
So this is, again, a cutaway view of the BCR-ABL kinase.
Here's Gleevec where it normally sits, but that red dot is a mutation that sticks an amino acid side chain right in the way of where Gleevec binds.
So now it cannot bind anymore.
The drug cannot bind, it cannot be effective, cancer cells come back.
OK?
So that's a problem.
What are you going to do about it?
What can be done?
What would you do?
Maybe we could find a drug that will bind even if there is a mutation there.
And that actually works and that's why this Gleevec fits nicely into that pocket and blocks the activity.
So this enzyme is inhibited.
The problem is that apparently invariably mutant BCR-ABLs arise which are Gleevec resistant.
So you can have Gleevec around, but it cannot get in there, the enzyme still functions, it still causes proliferation.
But working with a different drug company, Charles Sawyers screened new drugs.
And he found a drug which has a similar but slightly different shape than Gleevec, and it's able to bind to the mutant BCR-ABLs.
This drug is called BMS-354825 produced by Bristol-Myers Squibb.
And just in December of this past year Charles Sawyers reported the first clinical trial with this drug in patients who had failed Gleevec therapy who had relapsed.
And 31 out of 36 responded.
And to my knowledge they are still in remission.
And the five who didn't respond had a particular kind of mutation that actually also blocked the binding of this drug.
But the majority of mutations, even though there are several mutations that will affect Gleevec resistance, the majority of them are still sensitive to this new drug.
So this is smart and smart again.
Smart, understanding how cells can be sensitive.
Then smart again, finding out how they become resistance.
And then smart for a third time, finding new drugs that will bind even in the presence of those resistance mutations.
And this, I suspect, is the future of cancer treatments.
Understanding the molecular signature mutations, finding specific drugs, and then being prepared to find second-generation drugs that will still work even if resistance arises.
A very similar story, which I'll have to tell you quickly, comes from the world of lung cancer.
In this case, several drug companies were trying to find drugs that would block a different tyrosine kinase.
This time a receptor tyrosine kinase by the name of epidermal growth factor receptor, EGF receptor.
They were motivated to do so because this receptor is over-expressed, there is too much of it in many types of cancer, including lung cancer.
And so different companies made different drugs.
One of them is called Iressa which goes by the trade name Gefitinib.
Another made by Genentech is called Tarceva.
And these do function as EGF inhibitors, anti-EGF receptor inhibitors.
They work.
They work in the test-tube.
However, when tested in clinical trials they were a spectacular failure.
Even for patients who had high levels of EGF receptor on the surface of their cancer cells, the drug didn't do anything.
And they were almost not going to be FDA approved for that purpose, except that a very small number of lung cancer patients responded extremely well to the drug.
about 10% of lung cancer patients showed responses like the one I'm showing you here, where this, and outlined in red, is a lung tumor where the tumor is basically filling the entire lobe of the lung.
Six weeks after treatment with this drug Iressa, you can see massive resolution of the tumor.
The tumor is almost all gone, and that white stuff is probably just fibrotic tissue.
The tumor cells are practically gone.
A dramatic response.
What's going on?
Why do those 10% of patients respond so well?
Well, it turns out that those 10% of patients carry a mutation in the EGF receptor gene.
In those 10% of patients, one of the ways the cancer cells are growing and surviving is that they have a mutation that activates this gene making those tumor cells highly dependent on that particular protein in the same way that these cancers are highly dependent on BCR-ABL.
And if you deprive those cancer cells of that activity by using these drugs, the cells will die.
So this is a good example of what will come in the future of individualized medicine.
If you're a lung cancer patient, you shouldn't just indiscriminately take Iressa because 95% of the time it won't do anything for you.
But if you're one of those 10% who has a mutation in this gene, you would benefit dramatically from having it.
And that will happen more and more in cancer and other diseases.
Your tumor will be molecularly typed to find out exactly what mutations it has to find out which of a collection of targeted therapies you should be taking.
These patients, just so you know, tend to be women, non-smokers, and tend to be Asians for reasons that we don't understand.
Any of those three we don't understand, but the percentage of EGFR mutant lung cancer patients are higher in those categories of people.
And so they have a greater likelihood of being responsive.
But, in fact, nowadays if you have lung cancer you get your EGFR gene sequenced.
And if it has a mutation you take this drug.
And it will work.
It will resolve your tumor.
Unfortunately, your tumor will come back.
The same story but faster.
These patients tend only to get three months, six months, maybe a year, two years, three years extra survival, and then their tumors come back.
Same story, the receptors that are now insensitive to the drug carry a new mutation that blocks access of the drug to the receptor.
Fortunately, just last month there was a paper that described a new drug that will still work even if the receptor carries such a resistance mutation.
So there's hope that we'll see a story similar to this one emerging for lung cancer.
Now, I've told you three of, just a handful of molecularly targeted agents for therapy in cancer.
There are more to come.
Telomerase is an enzyme that cancer cells need, that normal cells or at least most normal cells don't need.
We won't go into the details of that, but suffice to say this is a promising area for therapy as well.
Angiogenesis, I've mentioned to you before.
Tumors, solid tumors need a new blood supply.
If you can block the ability of the tumor to recruit a blood supply, you might be able to block the development of the tumor.
And there has recently, Genentech once again, been an FDA approved anti-angiogenesis drug that blocks, that prevents progression of colon cancer, and also they just reported breast cancer.
So anti-angiogenesis is a viable therapy strategy as well.
Gene therapy, putting lost genes back.
Cancer cells have mutations in tumor suppressor genes, p53, RB I've told you about, and others.
Perhaps you could just put the gene back in, the gene that got lost.
Make a virus that expresses that gene and put it back.
This is being tried.
I'm not super enthusiastic about whether it will work because I don't know that we can get the virus carrying the good gene into all the cancer cells.
But, nevertheless, it's something to consider.
Immunotherapy is also a promising area.
It's possible that we can convince your immune system to detect the abnormal proteins that cancer cells make by virtue of the mutations that they carry.
You can make antibodies or T cells that eliminate those, and that's underway.
And I've mentioned already the cancer prevention approaches with vaccines.
I want to draw your attention to the fact that the Cancer Center here at MIT will have a symposium in June.
Many of you will have gone home for the summer, but some of you may not have.
June 24th.
This is free to MIT students, and actually features many alums of MIT.
Two of these people were undergraduates here at MIT, including Dan Heber, the guy who did the EGF lung cancer story that I just showed you.
This is a very great group of people who will be telling you the latest and greatest about the new science of cancer therapy, an extension of what I've told you about today.
All right.
Before we finish I want to just briefly mention that in addition to tracking mutations in cancer-associated genes, we can also track the expression patterns, the levels of expression of all the genes in a cancer cell.
And this is done using a technology called array technology where all of the genes of a cell, the 30,000 genes of a cell can be assessed based on how much RNA is being produced in those cells at any given time or at any given sample.
This is called a GeneChip or a gene expression array.
And increasingly it's being used to diagnose cancers.
Cancer is typically diagnosed by histopathology.
You look at the tumor, or the pathologist looks at the tumor in a histological section and says it's a this or it's a that.
The problem is that many cancers look very similar to the histologist or the pathologist, but in the underlying molecular level they might be quite different.
Some of them might be fairly benign.
Others might be really dangerous.
And maybe you cannot tell that apart by looking at the cells, but looking at the activity of the different genes inside those cells you may be able to get to that.
So this is done by comparing, on a glass slide, the levels of expression of all the genes from the cancer cell compared to some reference RNA, let's say the normal cell of that tissue.
And then the signal that you read out by looking at labeled RNAs, the cancer cell being labeled red, for example, the normal RNA being labeled green, the signal that you read out from each of those spots using a laser and a CCD camera to detect the signal coming off of the chip, the signal that you see can be quantified.
And a red signal would mean there's more RNA in sample one for that particular gene, a green signal more RNA for sample two, and a yellow signal roughly equal.
And the pattern that you get from these chips can then give you information about the state of those cells, and that information might be extremely important clinically.
And in the last two minutes I'll just tell you a brief story about how it is being used.
This is a collection of 75 breast cancer specimens, early stage breast cancers, node negative, lymph node negative breast cancers.
These patients, before this technology, all would undergo removal of the tumor and then chemotherapy.
But it turns out that only some of them will actually go on to progress.
These patients were followed for ten years after that sample was taken.
And it was known that some of them progressed, and those fall over here, progressed in the sense that they developed metastatic tumors.
And others didn't progress.
And so what was done was is to take RNA from these samples at this early stage, RNA from these samples and do one of these GeneChips and ask, is the pattern of expression of genes correlative to the outcome, the eventual outcome?
It might take some years for it to happen.
And what they found was that indeed -- And you can probably see it from where you're sitting, that this collection of genes in red over here is highly expressed in the guys who actually do pretty well and is relatively lower expressed in the guys that don't do well.
And this collection of genes is highly expressed in the ones who don't do well compared to the ones who do.
And so now there's a clinical test.
You can have your early-stage breast cancer typed by this analysis and it will tell you with some degree of certainty, not complete, whether your tumor will eventually, maybe five year down the road progress into a metastatic tumor.
If you get the signal, if you get the answer yes, then you have it removed and you undergo therapy.
If you get the answer no, you have a choice.
Perhaps you get the tumor removed, but you don't undergo what is in fact quite difficult and sometimes damaging therapy.
So this is an example of, again, stuff to come, molecule medicine, specific patient-oriented medicine.
So we're going to start by talking about this story.
This is a famous moment in the treatment of infectious diseases and the prevention of infectious diseases.
This is the first human vaccination, at least the first purposeful human vaccination where the famous British doctor, Edward Jenner, is vaccinating a child for the prevention of smallpox, a devastating disease at that time and for centuries before that time which would wipe out literally millions of people due to its very aggressive nature.
Not everybody who died or was infected by smallpox would die from it, but a lot of people did.
And it was an important infectious agent throughout the world, actually brought, for example, by colonizing troops in Mexico and wiped out Native peoples there, used purposely as a biological agent in warfare.
Smallpox was used by the British to suppress Native Americans in warfare.
In fact, the town of Amherst, Massachusetts was named after Jeffrey Amherst, I think, who was an officer in the British Army whom ordered that smallpox infected blankets, or blankets from smallpox infected individuals be sent as a gift to Indian tribes in the region so as to spread the disease to them thereby limiting their ability to fight in an oncoming war.
So this is an important agent, and this is an important moment in the treatment and prevention of this particular agent.
So I want to tell you this story because it illustrates some important points about immunology.
Edward Jenner, who as you see, was doing this work in the late 1700s, made two important observations.
One was that milkmaids, people who got milk from cows, milkmaids had smooth skin.
And that was unusual in the time, because most people had been infected by this agent smallpox, developed the smallpox disease which led to a pocking of their skin if they survived.
So most people did not have smooth skin, but milkmaids had surprisingly smooth skin.
One wonders what was behind this observation by Jenner, but it doesn't really matter.
He also noticed that milkmaids had previously been exposed, in many instances, to a related disease of cows called cowpox.
And this was manifested on their hands, they developed sores on their hands, but it didn't progress into a disease that looked like smallpox.
And so Jenner made a hypothesis that prior exposure to cowpox, whatever the agent was that caused cowpox, prior exposure to cowpox protected against smallpox.
And this was the suggestion of a phenomenon known as immunological memory.
And it was well known by then that you only got a disease once.
You only got exposed, if you got exposed to a disease and developed symptoms and recovered, you were protected against getting that disease in the future.
And this is a phenomenon of immunological memory, which we now understand in some detail, and I'll tell you how it all works.
But that was known already by then.
And so what Jenner said was that maybe you develop immunological memory to your cowpox exposure which then protects you against smallpox.
Well, so that was the hypothesis.
He needed to do an experiment.
And the experiment is illustrated up there.
He started with a milkmaid by the name of Sarah Nelms who had a cowpox sore on her hand.
He took that cowpox sore from her hand and injected it into James Phipps -- -- who was a clueless neighbor's boy.
He just happened to live in the neighborhood wandering by one day and where Jenner invited him in for some cocoa or something and said, by the way, since you're here, let me just inject you with some of this stuff.
[LAUGHTER]  And that's exactly what's depicted here.
Sarah Nelms is holding the boy and Jenner is injecting him with some cowpox stuff.
He then waited three weeks, and he injected him with smallpox.
A known lethal agent.
He then waited six weeks and injected him again.
[LAUGHTER] Then he waited and waited and waited.
And the kid was disease-free.
He had, in fact, protected the boy against the development of smallpox.
The first purposeful vaccination.
It's called vaccination, by the way, because cowpox, which is the agent used to vaccinate the boy, is caused by a virus known as vaccinia.
So in honor of that, the whole field, the whole approach is called vaccination.
So this is an example of introducing an agent which the body then responds to.
And based on the body's response to that agent, the body is protected against further disease.
Now, this works because the agent that was used to vaccinate the boy, vaccinia is very similar to the agent that causes smallpox.
It's not identical but it's sufficiently similar that the body's response to that agent is compatible with a response to the dangerous agent.
This is called a heterologous vaccine.
And I'll briefly mention other vaccine strategies towards the end.
Jenner succeeded in doing this with a number of patients directly after that.
And he actually published his paper, or he tried to publish a paper on this finding in the early 1800s.
He was actually blocked from doing so.
People didn't want to believe it.
People were extremely nervous about this approach.
It lead to editorials in local magazines such as shown here.
This is a cartoon of Jenner with young James Phipps and a bunch of people getting vaccinated with this cow thing.
And the consequence, as you might be able to see, is that these people are sprouting cows out of their arms, out of their faces, out of their butts.
And there was general nervousness about tinkering with the natural species in the world in this deliberate way.
But it was generally accepted, used extensively in Europe and actually in the United States.
And now it's sufficiently successful, has been sufficiently successful over the last couple of centuries that smallpox is no longer a problem.
It's been eradicated through proper vaccination using this approach.
So there is no longer smallpox out there, although there are vials of smallpox sitting in freezers which have been a concern to governments in the sense that bioterrorists might get their hands on it.
And there have actually been efforts to bring back smallpox vaccination as a protective against the potential use of the smallpox agent for some sort of bioterrorism.
Do you have a question?
OK.  Now, as you know, vaccinations against viral agents and other pathogens are commonplace.
You've all been vaccinated against lots of things.
They've changed the course of human history in a dramatic way.
This was not very long ago, 1952, a bunch of children who were infected with polio virus.
It led to deaths of many kids and paralysis of many more.
And this is a picture in a hospital ward of children in iron lungs, which is how they were kept alive because of their paralysis.
Many of them survived this early phase but then went on to develop paralysis in their extremities.
And it was a very devastating disease.
Fortunately, both Drs.
Sabin and Salk in the early 1950s developed vaccines using the principles laid out by Jenner 150 years before.
And these were successful.
And so kids were treated with polio vaccines for the next 30 or so years.
And polio itself was wiped out.
So there are no, or very, very few examples of active polio outbreaks these days.
And, in fact, kids are not vaccinated with polio vaccines in this country because it's not a threat.
It's not that it's not a threat worldwide.
In fact, last year when I was teaching this course we read that there were some new outbreaks of polio.
So it's not that the virus is gone.
It hasn't been fully eradicated.
But it's very, very uncommon thanks to these kinds of efforts.
And we think about this also again to protect against deliberate use of pathogens.
Here, Anthrax, I've shown you this slide before.
So there are now efforts to develop vaccines against Anthrax.
In case somebody were to use it as an agent, you could protect people from getting exposed to Anthrax.
And pathogens are important out there.
Many of the diseases that have been scourges of humanity over the millennia are due to pathogens.
It's been a constant battle between humans and pathogens, viruses, bacteria, other single-cell organisms.
And our greatest hope against controlling these infections is to protect them through vaccination.
Where that's worked it's been extremely successful.
Where it hasn't worked, like in the case of HIV, it's been much more problematic.
So vaccination in general, immunological response, immunological memory are extremely important.
OK.
So to understand what's happening with respect to vaccination and immunological memory, you have to think about what happens in an infection and how your body responds to it.
So if you plot a time course of infection where this might be the very earliest stages.
When you get exposed to a pathogen, virus, bacterium, it enters your body and it begins to reproduce itself.
It will build up.
And it might peak in its concentrations at about a week or so, maybe two weeks.
And it's at this stage where you're developing symptoms.
And those symptoms might be sufficiently severe that it can cause death.
But if you live then you observe typically that the pathogen concentrations drop and can be eliminated all together within two or three weeks.
The reason that they drop is because your body is making defense mechanisms against it in two forms, antibodies and specialized cells called T cells that are built to eradicate the agent.
And so if you plot the concentration of antibodies and T cells that are active against this agent, you find that initially there's a delay.
And then the concentrations of these antibodies and T cells rises.
And as they do they are acting on either the agent itself or the cells that the agent has infected.
And a combined activity then leads to the elimination of those agents.
Concentrations stay up and then they fall, but they don't go to zero.
They stay around.
You have low levels of these specific antibody-producing cells or T cells that are directed against this agent that stay in your blood forever such that if you get a second infection, even years later, these cells are already there set aside such that the response to that agent is very rapid and the concentrations of the agent never rise very high.
So you're able to control it before the amounts of the agents build up and cause symptoms or death.
And that's why you don't develop secondary infection.
This process is called immunological memory.
And, as I said, it's the setting aside of cells that are specific to the thing that's causing the disease.
And we'll talk about how that happens in a moment.
So this phase, I should have said, is called recovery.
OK. Now, I mentioned two cell types that are important in this process.
There are B cells and T cells, and they are the ones that I'm going to focus on.
But it's important for you to know that they are not the only cells of your immune system.
The body produces a whole series of cells shown here in addition to B cells and T cells and a specialized form of B cells called plasma cells.
And these cells also contribute to your immune response.
They're part of what's called the innate immune response.
And they act in a way that is nonspecific.
So they recognize classes of agents, viruses as a class or bacteria as a class.
They recognize things that are common to all viruses or most or all bacteria or most.
And they're very important in the early phases of an infection.
And, in fact, in this first phase where you're ramping up the antibody producing cells in the T cells, it's those cells that are acting to help suppress the proliferation of the agent.
And so this is where the innate immune response is taking place.
And Claudette will review for you some of those cell types in the next lecture.
We're going to focus on B cells and T cells.
And they fall into what's called the adaptive or specific immune response.
And that can be broken down, as I said, into these two cell types.
There are T cells which are T lymphocytes.
And the T lymphocytes constitute what's called cell-based immunity.
It's the cells themselves that are participating in the eradication of the agent.
And then there are B lymphocytes, and they participate in what's called humoral which is also liquid phase immunity.
And what that means is that they're producing something that gets secreted into the bodily fluids, blood or whatever.
And it's that which is participating in the eradication of the agent.
In particular, what that is, is antibodies, as we'll talk about.
These cells have these names, B cells and T cells, because where they mature.
So both B lymphocytes and T lymphocytes start off in the bone marrow, or I should say the precursors of these cells start off in the bone marrow.
The precursors to B lymphocytes make their way either to the spleen or they stay in the bone marrow, and that's why they're called B cells.
And they go on to make B lymphocytes, including these cells called plasma cells, which produce antibodies.
A separate precursor called the T cell precursor makes its way to the thymus which is a lymphoid organ in your chest, and there the cells mature into two types of T cells, cytotoxic T cells, otherwise known as CTLs, or TC cells, and helper T cells or TH cells.
OK?
And because they go to the thymus they're called T cells.
Now, these cells are distinguishable based on the types of protective molecules that they produce.
This slide just shows you an overview of the lymphoid system in your body emphasizing the points I just made.
The bone marrow is important based on the origin of the cells, as well as it's where B cells mature.
B cells also mature in the spleen.
And here's the thymus where T cells mature.
And in green here you see the lymphatic system.
These are the vessels that carry your lymph fluids where many of these cells move to get to the sites of infection.
And what's not so obvious to see are lymph nodes, which is again where many of these cells mature and turn into fully blown antibody producing cells or matured T cells.
I also want to point out that the handout that you have has the wrong numbers.
I was looking at least year's book when I took these numbers down.
So the figures are what I'm showing you here.
And on the Web, the version that's on the Web has been corrected, so you might want to pay attention to that.
Now, what the antibodies that are produced by B cells and the molecules on the surface of T cells are recognizing are things called antigens.
Antigens are one of a number of different types of molecules, lipids, carbohydrates, proteins, that are specific to the pathogen.
And your body makes antibodies produced by T cells, produced by B cells, as well as proteins on the surface of T cells that will specifically recognize these antigens.
And that's why it's called a specific immune response, because the protein that the lymphocytes are producing specifically recognizes the antigen, whether it be a lipid antigen, a carbohydrate antigen or a protein antigen.
And that's illustrated on this side where you have a virus particle here or a secreted protein that might be floating around in the blood.
And you can see that there are bound to it these colored structures.
These are antibodies.
And the different colors are recognizing different antigens.
So this orange colored antigen is being recognized by that orange colored antibody.
The purple colored antigen is being recognized by the purple antibody.
Your body has an amazing ability to generate tremendous diversity in the structures, these antibodies or T cell receptors that recognize particular antigens.
B cells, B lymphocytes start off by producing on their surface antibody molecules.
These antibody molecules are heterotetromers.
They have two heavy chains and two light chains.
And we'll talk about how those produced in a moment.
B cells mature and they turn into plasma cells.
And plasma cells secrete the antibody into the bodily fluids.
OK?
T cells have on their surface a protein called the T cell receptor.
And likewise it is very specific.
Just like antibodies, as depicted up here, have a particular sequence at the end, which is the antigen binding site.
And no two antibodies produced by different B cells would be the same with respect to their antigen binding site.
Likewise, this region of the T cell receptor is unique.
And, again, that's what gives specificity to the immune system.
There are B cells that produce particular antibodies that recognize particular antigens.
And T cells that have on their surface T cell receptors that are specific to particular antigens compared to other T cells.
This is a detail from your book which shows again an antibody molecule.
You can see that it's a heterotetromer, two heavy chains, two light chains, and at the very tips, both of these arms are these antigen binding sites.
And, again, this is where the diversity comes from.
This antibody, with its particular structure, will bind to this particular antigen because that antigen fits into the pocket formed by that antigen binding site specifically.
Likewise, on the surface of T cells there are T cell receptor proteins.
They're composed of two chains, an alpha chain and a beta chain.
And the coming together of the alpha chain and the beta chain produces, again, an antigen binding site that is unique to that particular T cell receptor.
OK.
So this is interesting.
And it raises an important question, an important problem.
There are lots of pathogens.
There are lots and lots and lots of things out there that can get into your body and cause you harm.
And, therefore, in order to effectively fight those things you need many different antibodies -- -- and many different T cell receptors.
It's actually estimated that there are ten to the seventh distinct B cells and T cells in your body at any time.
So you've got ten million different B cells and T cells that make different antibodies or different T cell receptors on the surface.
So where does that diversity come from?
How do you get ten million different B cells or T cells in your body?
Well, one possibility would be that there are ten million different genes.
That there are ten million different antibody genes or ten million different alpha genes and beta genes in the T cell receptor.
Is that the likely explanation?
How do you know it's not true?
Because I've already told you that there are only 30, 00 genes total throughout your genome.
So there sure as hell aren't ten to the seventh different antibody genes.
So that answer is wrong.
It could have been from alternative splicing.
Maybe there are genes with many, many exons.
And depending on how those exons get joined together, through a process of alternative splicing, you could produce diverse antibodies or T cell receptors.
That could be true.
And we actually think a lot of diversity in biology does come from alternative splicing of complex genes, but that's not the answer here.
Instead, the answer has to do with a process known as DNA rearrangement.
This is a process that was discovered, in the context of the immune system, by Susumu Tonegawa who was a professor in the Cancer Center at MIT.
And, actually, he won the Nobel Prize for that discovery some years ago.
We now know that the generation of all of these different antibodies and all of these different T cells is due to complex rearrangements of a very small number of complex genes.
And I want to go through that with you now.
And it is a little complicated.
And so I warn you that I'm going to show you slides which come directly out of your book.
And we'll walk through them.
But then I advise you to read your book which I think does a good job explaining how this rearrangement process takes place.
And hopefully together that will solidify the concepts for you.
So, again, immunological diversity means that each B cell produces particular antibodies, as I've said.
And these are the product of uniquely rearranged heavy chain genes and a uniquely rearranged light chain gene.
And, likewise, each T cell, each distinct T cell has a specific T cell receptor on its surface which is the product of a uniquely rearranged T cell alpha genes and a uniquely rearranged T cell beta gene.
And you can kind of think about the process that we're going to talk about like a roulette wheel that in roulette, as you know, one wheel spins and a particular object shows up and the next wheel spins and a different object shows up or the same one, and likewise the third way.
And you end up with a unique combination of objects.
The same is true is here.
And there are lots of different combinations that can come up.
Actually, last year when I was talking about this, I realized that an even better analogy is Mr.
Potato Head.
And I thought to bring this morning the Mr.
Potato Head thing from my three-year-old daughter, but she threw herself across the door.
So I actually was unable to bring Mr.
Potato Head with me, but hopefully you remember Mr.
Potato Head.
It's a bland head, and you can put a different mouth or a different nose or a different pair of eyes or hair.
And based on the combination that you choose you get a very different looking face.
The same thing is true in the generation of antibodies and T cell receptors.
It's a choice at different positions, and based on the choice that's made you make a unique looking antibody or T cell receptor.
So this is illustrated here.
What we're looking at is a piece of DNA that constitutes the un-rearranged heavy chain gene of antibodies.
And you can see that there are different regions colored differently.
There's a region called the V region which has several segments, several bits, which are similar but a little bit different.
There might be a hundred or so of these V segment exons clustered together here.
Next door is another set of segments called the D segments, and there are about 30 of those.
And then next to them are the J segments, and there are about six of those.
Next door is another set of segments called the constant region exons.
These actually get used in a slightly different way.
They don't come together by DNA rearrangement but rather by splicing, but this also adds to the diversity.
When this rearrangement process is done you can choose a different exon down here by alternative splicing.
So you can maybe get a sense, if we're going to pick one of these and then randomly one of these and then randomly one of these we can generate a lot of diversity.
This happens, as I said, through an ordered process of DNA rearrangement.
Our goal is to take an immature B cell, B cell precursor and then sequentially rearrange the heavy chain gene and the light chain gene.
And then finally, when that's all done, that mature B cell is going to be sticking on its surface a specific antibody molecule which will have this structure here.
And the region of the antigen binding site will be the unique product of the coming together of those V segments, D segments and J segments.
So the first step that happens is, with respect to the heavy chain gene, rearrangements take place.
There are enzymes that get turned on in the immature B cell which specifically recognizes sequences next door to these segments.
The first segments that recombine together involve the D region and the J region.
So randomly one of the D region segments and one of the J region segments gets chosen, and the enzyme comes along and clips the DNA right next door to that segment and right upstream of that segment looping out the stuff in the middle, getting rid of it and joining together that particular pair of segments.
So now you have a unique new piece of DNA which joins one of the D segments with one of the J segments.
Next, one of the V segments is chosen which then gets clipped, and this region up here also gets clipped such that the middle piece gets taken away and the two pieces get joined together.
And the product of that is a new piece of DNA that has a unique V region joined to a D region joined to a J region.
OK?
So there's a series of cut and paste reactions that produces a unique combination of V, D and J.
Once that process is done then an RNA is produced from that locus.
The RNA gets spliced, as I said, to allow the VDJ segment to get joined to one of the constant region segments.
That mRNA product then gets translated into the heavy chain.
Once you make the heavy chain you more or less go ahead and do the exact same thing with the light chain.
You do a VDJ recombination.
Join it up with a constant region.
Now that cell is able to produce both a heavy chain and a light chain which is unique.
Those come together to form a unique specific antibody molecule.
So, again, that's complicated stuff.
I don't expect that those of you who haven't heard it before will understand it from what I've just said, but all of what I told you is in the book.
And there's an animation, which is where this comes from.
So I advise you to read your textbook.
OK?
The important point is that recombination of these individual segments, random joining together of these distinct segments gives tremendous diversity.
It allows you to produce ten to the seventh different T cells, sorry, B cells.
The exact same process happens in the rearrangement of the alpha genes and the beta genes in the T cell receptor.
So you get, again, tremendous diversity in that way.
Now, I'm just going to mention this but I don't expect you to know it for life.
Well, you might know it for life but you don't need to know it for a test.
There are still other mechanisms that the body uses to add even more diversity.
It turns out that this process of joining is actually intentionally messy so that the joints that occur between these segments are not all the same from one cell to another.
Even if the cells were to rearrange the same two or three segments, they wouldn't necessarily produce the exact same antigen binding site because the process is inherently sloppy in order to even make more diversity.
OK?
But you don't really need to know about that specific aspect of it.
OK.
So through this process of rearrangement, through this process of rearrangement we are able to make, in the case of B cells, lots and lots and lots of distinct unique B cells that are circulating throughout your body.
They're different from one another because they have on their surface, initially in the case of B cells they have on their surface these antibody molecules which are different from one another in these regions.
So these antigen binding sites -- -- are unique.
This one is different from this one.
And I should have mentioned I wanted to emphasize that because of this rearrangement process the DNA of this cell is different from the DNA of that cell.
I point that out because we've emphasized in the past that the genome that you get when you're first fertilized is stable and exactly the same in all of your cells.
That's actually clearly not true in the case of B cells and T cells because they purposely rearranged the genome in the production of antibodies in T cell receptor genes.
So the genome actually is a little bit different, at least in the case of the lymphoid cells.
So these cells then are produced and they're quietly circulating in your blood.
So how do you mount an immune response?
You're now infected by some pathogen.
How do you build up the concentrations of one of these particular B cells or T cells that can recognize that pathogen?
How do you mount an immune response?
Well, the way you do it through a process of clonal selection.
And, again, this comes right out of your book.
These cells which are floating around in the blood and not doing very much in terms of making more of themselves, they're not proliferating, can respond through the exposure to the antigen.
So if floating around the blood along with these cells is a particular antigen, and it's able to bind to that antibody molecule which is sitting on the surface, that sends a signal to that cell it's time to divide.
There's something in the environment that we like so that we need to make more of ourselves in order to fight off whatever that thing is.
This induces a rapid and impressive proliferation.
So you go from one or a small number of these cells to many, many of these cells, particularly these cells.
These cells don't proliferate because they're not binding to the antigen.
These cells do proliferate.
They make many, many more of themselves, and when they get to a certain point they differentiate.
They begin to make the material they need to secrete really well.
They actually change their shape dramatically.
They become very efficient secretory cells.
And so these B cells that have the antibody on their surface then become secretory cells, these plasma cells.
And the plasma cells then secrete the antibody into the bodily fluids allowing the antibody to then go off and bind to the antigen whether it is itself circulating or it's on the surface of the pathogen like a virus or a bacterium.
So this process of clonal selection then allows for the cells to build up and, in this case, to differentiate into antibody-secreting cells.
Now, these are the cells and the antibodies that are going to fight the infection.
These are the cells that are building up right here producing, in this case, the antibody.
The same exact thing happens with respect to T cells which you'll learn about in detail next time.
However, after the infection is cleared most of those cells go away.
They're no longer being stimulated and they actually die off.
But importantly not all of them go away.
Some of them are set aside as these memory cells.
And if these memory cells that just kind of hang around in higher concentrations than they were at the very beginning of this process but it's still relatively low concentrations, and it's the presence of those memory cells that when you're infected again they kick into action quickly.
They've already matured to a very great extent, as you can see here.
They're on the edge.
They're poised to make a lot of antibody or be an effective T cell.
They build up very quickly and they suppress the immune response, they suppress the infection.
So this process of clonal expansion is the early phase.
The setting aside of the memory cells is the late phase.
And it allows us to effectively respond in a second infection.
And it's also the reason that you can respond once vaccinated.
Vaccination is the exact same process except you're exposed to something that is not inherently dangerous.
In the case of an active infection, you're infected with the pathogen, the active pathogen, and it builds up and then you respond to it.
But there's this dangerous phase where you might actually die.
In the case of vaccination, you get exposed to something that is like the antigen, like the pathogen, but it's not otherwise dangerous.
But, still, the same stuff happens.
You build up antibody producing cells.
You build up T cells.
They then get set aside in the process of immunological memory.
And when you're exposed to the real thing, if you're exposed to the real thing later on, like James Phipps was exposed to smallpox, you already have those cells set aside and you can respond effectively to the agent.
So I want to now just mention in closing the various vaccine strategies that are used.
And they all rely on this same phenomenon of exposing you to a related agent or antigen allowing you to make T cells or B cells and then fight them effectively.
So the first one that I mentioned is cowpox for smallpox and I used the term heterologous vaccine.
A heterologous vaccine is an organism which is very similar to the organism that causes the disease cowpox virus versus smallpox virus.
And it's so similar that the antigens that it produces direct the development of antibodies or T cells that will also work against the dangerous pathogen.
That's a great one if you can have it, but there aren't very many of them.
There are very few examples of heterologous vaccines.
It's just coincidence or luck that it worked in the case of cowpox and smallpox.
A more common one is the attenuated vaccine.
In this case, you take the active agent like polio virus and the Sabin polio vaccine is.
You take that active agent and you grow it in the laboratory for a long time under conditions in which it changes.
It adapts to the laboratory conditions and it's no longer dangerous to a person.
If infected with this you will not get polio, but it's sufficiently similar to the original virulent pathogen but it has the same antigens so you mount a proper immune response.
This is an effective strategy.
It's used all the time.
It's a little bit dangerous because if the attenuation process isn't good enough and there's still a little bit of active pathogen in there it could cause disease.
And this happens every once in a while with attenuated vaccines.
Another way is to just take the agent, whatever it is, and kill it.
Mix it with a chemical that will crosslink its genome or heat it up really high so its DNA will be destroyed.
That's an agent which cannot reproduce itself in your body.
Its genome has been destroyed, but it still has in it the antigens.
It still has on its surface the antigens.
Your body will still recognize it, make antibodies and, therefore, if you were ever exposed to the live thing you will be protected.
This is also very effective.
This is the Salk polio vaccine.
But every once in a while the killing process isn't perfectly effective.
And so there's a little bit of live virus or whatever in there and people get disease.
Another strategy is component vaccines.
Nowadays, we can purify from the pathogen virus or bacteria a piece of it.
You can purify some protein from the virus or from the bacterium and just use that as the antigen.
That's not dangerous because it cannot replicate on its own inside you, but your body makes antibodies against it.
And, therefore, you'll be protected at some later time against the real thing.
And, increasingly, we're using recombinant vaccines.
Using molecular biology, genetic engineering, we can actually build new agents that carry the genes of dangerous pathogens.
And, actually, a commonly used one is vaccinia itself.
So if you take vaccinia, the cowpox virus, take some of its genes out and put in the genes of a dangerous virus like polio virus, now that cowpox virus will get into you.
It will replicate a little bit and it will start making those antigens.
Your body will recognize that with antibodies and T cells.
You'll clear up that infection because it's not a dangerous agent.
But you will have made antibodies and T cells that can recognize those other antigens such that if you were infected at a later time you would be protected.
So these are examples of vaccines, again extremely effective.
They rely entirely on immunological memory, the generation of immunological diversity.
Now, importantly, and this is something that you'll pay attention to next time, not all of these vaccines are created equal when it comes to producing a B cell or a T cell response.
Some will produce both.
Some will only produce a B cell response.
And you'll see why that is in Monday's lecture.
OK.
I think we'll get started.
So we're going to continue our discussion today about viruses, finish talking about influenza, influenza A virus and the flu, and then talk a bit about HIV and AIDS.
Do you guys know more about the tutorial session and so on and so forth for next week?
Has that been decided yet?
OK.
So you'll get more information Monday and Wednesday.
And I'll also have an office hour, probably a couple of office hours at the end of next week.
Most likely, if it doesn't conflict with this tutorial session that's being planned, it will be during this hour from 11:00 probably until 1:00 in this room because it's free since we're usually here.
But that may conflict with you guys so we'll have to see.
I'll let you know on Monday.
So, again, the situation with viruses is an ongoing one.
And, actually, one of you brought to my attention an outbreak of polio that's happening now both in Africa and Indonesia.
This is actually from today's CNN.
om where there's a fear that polio is starting to spread.
In this case in Jakarta, in Indonesia.
And there's a very extensive reaction to this.
There's a great fear that polio will spread.
This is an area that is not well protected currently by polio vaccination.
And so if you look at this you'll see that they are raising lots of money in order to try to vaccinate with standard polio vaccines, which work extremely well, 5.2 million children, to try to stop the spread of this potential epidemic of polio.
It hasn't reached anything near those levels to be that concerned, but in order to protect against that possibility they're going to vaccinate.
And, obviously, if you vaccinate, individuals cannot be successfully infected.
If they cannot be successfully infected, the virus cannot propagate so they cannot pass it on.
So you just close off the infectious cycle in a very small circle of infected individuals.
Also, in today's Globe, there was a discussion of a local company that makes vaccines against smallpox.
And I've mentioned to you that smallpox is largely eliminated, if not fully eliminated in the world, but there are stocks of smallpox in freezers around the world.
And there is a concern that an individual could access smallpox virus or make one because the relevant sequences of the genome of the smallpox virus is known.
And so, in theory, you could synthesize your own smallpox virus genome, and thereby create your own smallpox virus and expose now unprotected individuals because we don't get vaccinated currently against smallpox.
And so there are companies like this one that are making currently and distributing smallpox vaccines.
They distributed 182 million doses in this country alone just in case somebody tried to deliberately release some smallpox.
So that's what we can do when we know what we're dealing with.
We can create effective vaccines.
In this case they're effective, but sometimes we get exposed to stuff that we cannot effectively deal with.
And I introduced this to you last time.
This was the major flu pandemic from 1918.
Some people worried that there might be another pandemic this year because in the off season between 1918 and 1919 when this happened, and 20 to 40 million people died, that's when the Red Sox last won the World Series.
[LAUGHTER]  So the Armageddon folks thought maybe this was the year, but so far so good.
So, as I mentioned last time, this was a major outbreak and a major problem worldwide.
But you also should know that influenza is an annual problem.
And I, for one, didn't appreciate this.
Where there are about 25 thousand deaths per year in the United States.
And, of course, to deal with that we make available, especially to the elderly and infirmed immunodeficient flu shots.
And flu shots are simply flu vaccines.
And I'll tell you a little bit about how they're made towards the end of this section, but they're generally effective in combating against the flu that is going to hit the population that year.
And they change every year because, as you'll see, flu changes all the time as well.
So it's an annual problem.
There are occasional epidemics which you could think of as -- -- population-based infection where many individuals within a particular population are infected.
And the virus, in this case, is spreading throughout that population.
And there are even more occasional pandemics.
And that's a worldwide infection.
Where, because of the virulence of the virus and its ability to access different populations and the absence of immunity throughout the worldwide population, lots and lots of people get infected.
And that's what happened in 1918.
So how do we explain this?
How do we explain the occasional epidemics of a virus that we're so familiar with?
Flu, as I said, we get every year, lots of people get it every year.
And, yet, we seem to be susceptible year after year.
So why is that?
What causes the resistance to this immunity in small measure and in large measure?
And that's what I want to review for you today.
So this is the responsible agent.
It's a virus, of course.
More specifically, it's an enveloped virus, which means it has its own lipid bilayer that it picked up from the host cell.
And you'll notice on the outside are things sticking out of the bilayer.
These are proteins encoded by the virus which are going to be responsible for binding the virus to the host cell.
And also allowing the viral membrane to fuse with the host cell membrane.
And you'll see that that's done in the case of flu virus in a slightly different way than for some other viruses.
And then inside you have the capsid.
And wrapped up in this protein, this helical protein structure are the nucleic acids of the virus.
And the nucleic acids of this virus are multiple.
That is it's not just one.
Oops.
It's not just one.
It's actually several.
So, again, it's called influenza A virus.
It's an example of an enveloped virus.
And I said you should note the envelope proteins, which are largely encoded by the virus itself.
And we call these envelope glycoproteins.
Glycoproteins because when they emerge through the sorting pathway of the cell they pick up sugars.
And so when they're on the outside of the cell, it's not just protein.
There is some sugar there, so glycoproteins.
And sometimes the antibodies that we make are directed against the sugar moieties alone or in combination with peptides.
So the envelope glycoproteins are important.
As I said, it's a segmented RNA genome, which means that there are multiple distinct nucleic acids in the viral particle.
And in the case of flu there are eight.
The RNA genome is single-stranded.
And all of the genome segments are in the minus configuration.
So their sequence is the complement to the mRNA.
OK?
And we talked about the significance of RNA viruses that have a minus strand polarity in the sense that they need to have their own RNA polymerase coming in with the virus in order to help, in order to initiate the replication cycle.
So, hopefully, that's familiar from last time.
This is a diagram of the virus, and it just makes the same points I just made.
So, again, it has a lipid bilayer.
It has envelope glycoproteins sticking out from the surface.
You'll see that they're important for different things in a moment.
Inside that is a nucleocapsid which surrounds the nucleic acids.
And you can see that there's a series of nucleic acids that are enclosed within this.
And, again, there are eight of them.
And they're all minus strand.
And, importantly, contained within here is a protein, an RNA dependent RNA polymerase, the yellow dot.
And it has to get incorporated with the virus so that when the virus infects it goes in there, too.
And it can then act on the RNA genome and convert it into a plus strand, which will then serve both as the RNA for translation, as well as the template for the production of more of the minus strand RNA.
OK?
So this comes from your book, and it is a summary of the infectious cycle of this virus.
I've modified it a little bit because some of the details which I think are important were missing.
So you might want to pay attention to this figure as you're reading the section, the relevant section in the book.
But this is a typical viral lifecycle.
The virus attaches.
Remember, the terms that I used last time?
The virus attaches.
Here one of the viral glycoproteins binds to a protein on the surface of the target cell.
That then initiates an internalization process, a penetration process in which the virus gets brought in through an endosome.
So it gets brought in through a separate vesicle, an endosome.
That vesicle, the endosome then fuses with a lysosome, not shown here.
And you may recall that the lysosomes have very low pH.
And it's in the context of the low pH that one of the viral glycoproteins changes its shape.
And when it changes its shape it allows the viral membrane to fuse with the membrane of the lysosome.
So this is a pH-dependant protein conformational change which turns an inner protein into a protein that facilitates fusion.
And when that happens, the viral capsid can then slip out into the cytoplasm.
It gets unpackaged.
The RNA, which is the minus strand RNA gets released.
And then it gets acted on by that RNA dependent RNA polymerase that came in and gets converted from minus strand to plus strand.
And the plus strand can then do two things.
It can get translated into viral proteins, both the glycoproteins that go through the sorting pathway and make it to the membrane, as well as the structural proteins that form the capsid.
The structural proteins then join up with the viral RNA segments, now the minus strand RNA segments.
They meet at the membrane and then bud off to form a new virus.
OK?
So this is a standard infectious cycle for this class of viruses.
And because it's going to be relevant later, I just want to emphasize a couple of points.
Through interactions between the cytoplasmic tail of the envelope glycoprotiens, the capsid proteins coalesce underneath the plasma membrane.
And there's an interaction between these proteins and the tails of these proteins which causes the virus to start to bud off the surface of the cell.
Now, there are separate interactions with the proteins on the inside of the capsid which bring with them the different viral RNA.
So remember there are eight.
And there's a rather amazing process by which the eight distinct RNA segments get brought together into the capsid, and then the capsid goes to the membrane and gets budded.
So there is a sorting process, which we don't fully understand, that ensures that viruses get at least one copy of each of the genomic segments.
And that's necessary because if a virus doesn't have all eight, when it gets into the host cell that it might infect, it's not going to be able to replicate.
These RNA segments encode one, or at most two of the viral proteins that are necessary for the infectious cycle that's drawn up there.
And you'll see why that's important in a moment.
OK.
So we get infected, we develop antibodies against the proteins of the virus, and we typically develop them against the glycoproteins that are sitting on the outside.
We develop T cells that can recognize virus infected cells and kill them.
And yet we keep getting flu.
So why is that?
How is it that we can, how is it that the virus can overcome this immune resistance?
And the answer is that the virus changes.
And this is a major problem with all viruses, but it's a particular problem with RNA viruses and a particular problem with flu.
So here's a depiction of the virus again.
It's got its genomic segments in here.
And on the surface it has these glycoproteins.
And I'm going to draw a little bit more detail in these glycoproteins now.
There are actually two distinct glycoproteins on the surface of the cell.
There's one that's called HA and there's another that's called NA.
And they carry out different functions.
When you get infected, your immune system sees these epitopes sitting on the surface of the virus and you make antibodies against them.
And there are three particular places on these proteins where good antibodies can be made inside your body.
One of them is here, another one is here, and a third one is here.
And different people make different of these antibodies.
And these antibodies that are made have a name which is neutralizing or blocking antibodies.
They're called that because if you have one of these antibodies, and it will bind to the surface of the viral particle, this virus can now not infect the cell.
There's an interference between the, what the hell?
Between the glycoproteins on the surface and the receptors to which they bind by the antibody.
So they block binding.
And if you cannot bind you cannot infect.
And this is very efficient.
This works.
So that's how you can overcome the infection to that particular virus.
The problem is that viruses change.
During their replication there are mutations.
And sometimes these mutations affect the structure of the proteins that are sitting on the surface.
So you might imagine the strain here gives rise to variant here which has an HA protein which looks more or less the same, but it has an NA protein which has changed.
So now this epitope, which was recognized by antibody three, cannot be recognized by antibody three because it's got a difference sequence.
OK?
If you were a person who made predominantly antibody three in response to this infection, you would not be susceptible to this virus.
So if this one came along next year you'd get the flu.
If you were a person who made antibody one or antibody two as your major protective antibody, you'd still be resistant to this.
You'd be one of the lucky ones.
And because there's heterogeneity within our population, even if these new variants arise, they don't spread all that much because there are enough people in the population who are pretty well protected.
So those are low-level spread or low-level epidemics.
OK?
Now, this process, one day I'm going to figure out how this works.
This process of generating variant viruses through this mutational mechanism is called antigenic drift.
It's a slow drift of the antigens on the virus.
And it happens because the virus is highly mutagenic.
And it's true of RNA viruses in general.
And there are two reasons why RNA viruses create variants at such high levels.
One is that RNA is less stable than DNA.
Remember, there's a difference in the structure of RNA versus DNA.
There's an extra hydroxyl in the structure of RNA which can cause increased mutation, increased breaks and base substitutions in our RNA molecules compared to DNA molecules.
So RNA is inherently less stable than DNA.
And also RNA polymerases lack proofreading functions.
Which means they have an inherently higher mutation rate.
And hopefully, again, this is familiar from our discussions about DNA replication.
All polymerases make mistakes but your DNA polymerases have what's called a proofreading function which can recognize the mistakes and correct them.
RNA polymerases lack that, and so they make the mistakes and they stay as mistakes.
So the mutation rates for RNA polymerases are at least ten fold higher than for DNA polymerases, and in some places even higher still.
OK.
So that's what's called antigenic shift, sorry, antigenic drift.
And it happens all the time and is responsible for why we keep getting mild cases of the flu.
But that's not what's responsible for the Spanish flu of 1918.
Instead of antigenic drift -- -- these are due to a phenomenon called antigenic shift.
Rather than generating subtly different variants, like I've shown up here, this is the process of generating essentially entirely new strains.
And it happens in the case of flu all the time.
And one of the reasons that it happens is that there are lots and lots of types of flu viruses which are kind of similar that infect other species.
There are duck flus, swine flus, horse flus, seal flus that are all caused by versions of this influenza A virus.
Similar.
Not identical but similar.
And sometimes those viruses cross over into the human population.
And this is an example of a zoonotic infection -- -- which is an infection from another species into the human population.
And actually happens all the time.
Many of the new pathogens that arise in the human population arise through zoonotic infection.
Now, most viruses that are evolved to grow in the cells of one given species, at the temperature of a given species, will not replicate very well inside human cells.
So even if they did infect, they wouldn't reproduce themselves very well.
And that would be true of these viruses, too, swine flu virus, bird flu viruses.
The problem is, in the case of flu, the human virus is so common and the mechanism of replication of this virus is so complex and amenable to generation of recombinant viruses that we can generate new viral forms.
And that's what I want to talk to you about now.
So here's an example from not too long ago.
This is the collection and culling of a lot of birds somewhere in Asia.
I'm not really sure where this was.
But there was a concern that a new virus was developing in a bird population and was crossing over into humans.
And it caused some deaths.
And so to avoid that further they just wiped out some millions of these birds in order to prevent further cross infection.
But, again, it's not the presence of the virus, the bird virus or swine virus itself that's the problem.
It's the fact that the two viruses, the human virus and the bird virus can recombine to form a dangerous new virus which is rather similar to the human virus but carries some segments of the bird virus.
And that's illustrated here, and I'll show you on the board in a second.
But basically the idea is that a human virus carrying its eight segments, including the segment for HA and NA infects an animal which also gets infected by a bird virus.
So that in the same cell there are the segments for the human virus, as well as the segments for the bird virus.
And during this process of the generation of a new viral particle, you can get mixing of the segments so that most of the segments come from the human virus and, therefore, will replicate well in the context of the human cell.
But two new segments come in from the bird that encode its HA and NA, a version that's never been seen in the human population.
Nobody has antibodies against it so the virus can spread like wildfire.
And that's what we believe happened then and we fear continues to happen over time.
So, again, imagine a cell which has been infected by two different viruses, the avian virus, in this case it was a duck, as well as the human virus in that given year.
In that cell there are going to be lots of different segments of virus RNA, the human ones H1, 2, 3, 4, 5, 6, 7, 8 and so on, as well as the duck ones, duck 1, duck 2 and so on and so forth, duck 7 and duck 8.
When the viral capsids coalesce underneath the viral glycoproteins to produce a new virus, they cannot necessarily distinguish between the human and the duck subunits.
So there might be human 1, human 2, human 3, duck 4, human 5, duck 6, human 7 and human 8.
OK?
This then makes a virus which is perfectly able to replicate in human cells because it's got mostly the genes that are optimized for human cells, but it's got new HA and NA forms that have never been seen by the human population.
So there is nobody who has antibodies against it.
So the virus will spread like crazy.
OK?
So this is an example of antigenic shift, and it's responsible for these worldwide pandemics.
OK.  Any questions about flu?
One thing I failed to mention, sort of trivial but you probably know, is it spread very well because it can spread in water droplets.
If you sneeze on your friends they'll also get the flu.
It's a very efficient virus so you don't need very many viral particles to go inside of your respiratory system in order to initiate the infection.
And this is another good example.
When you have a very active infection inside of you, you start producing lots of the cytokines, that I mentioned last time, like interferons.
And, again, it's largely those that are causing the symptoms of the flu, and in large enough quantities can cause death.
You can also get secondary infections with bacteria when you're that sick, which can be another reason why folks die.
OK.
So if there are no other questions about flu, we'll now turn our attention to HIV.
And I think HIV is probably fairly familiar as a topic.
It's, of course, a major, major worldwide health problem.
The initiation of this epidemic goes back a very long way.
The first cases were reported in 1975, so about 30 years ago.
And so that's considered to be the initiation of this epidemic or pandemic.
The first case in the United States is considered to have occurred in 1980.
And here's another example.
We think that this probably occurred by one of the zoonotic infections.
But it happened a long time ago.
It happened at least before 1950 because there are samples of individuals who died of unknown causes that have now been tested and can be shown to have HIV sequences by PCR.
So whenever this happened, and from whatever species it happened, it happened quite a while ago.
Why it then initiated in this epidemic form much later is not so clear.
The etiological agent -- -- is a virus called HIV, human immunodeficiency virus.
It is a retro virus.
And we'll briefly review how it replicated, but I told you more or less how that happens last time.
And this was another good example.
The syndrome was first discovered in 1975, gained a lot of attention in this country starting in the early 1980s, and the virus was first purified and characterized in 1983.
So it didn't take too long to identify the agent and then ultimately sequence its genome.
And also to develop tests for the presence of actually not the virus itself.
The AIDS test is a test for the presence of antibodies in you that recognize the viral glycoproteins.
And this was very important, obviously, in screening populations, screening blood banks and so on for contaminated samples.
Now, since this time, in the early 1980s, the situation has, of course, gotten much, much worse.
In this country, there are currently -- -- a million people infected with HIV.
That's between, you know, one and 300 individuals total.
And, amazingly, about 20% of those people don't know it.
Worldwide, anybody have any idea how many people are infected with HIV in the world?
40 million.
And total about 60 million people have been infected because since the early 1980s, 20 million people have died from this disease.
More than 20 million.
And the prevalence is remarkable in certain places in the world.
In Southern Africa, for example, not the country of South Africa but in the Southern Region of Africa, Saharan Africa, there are 25 million people infected.
And in certain countries that's one in four sexually active individuals.
Remarkable infection rates.
And the effect of this disease and the death that it causes has also had dramatic effects on the population.
So that in Zimbabwe, for example, where the life expectancy in 1990 was 52 years old, not a huge number but 52 years old, in 2003, because of AIDS, it's less then 35 years.
So it has dramatically affected the lives of people throughout the world.
OK.  And, as you know, I suspect you know that transmission for this virus occurs through direct contact of contaminated fluids.
Sexual contact, which is both homosexual and heterosexual.
As well as blood transfusion, direct blood contact, through transfusion, and also needle sharing.
And sometimes occasionally organ donation, but that's very, very rare.
OK.
So people die from AIDS because they develop infections.
And we'll talk a bit more about that in a moment.
But here's one patient who has developed an infection with a virus, a particular type of herpes virus now called Kaposi's sarcoma virus.
That virus then is able to replicate in certain cells of the blood vasculature and cause tumors.
These are actually tumors.
And this individual will die from a disease called Kaposi's sarcoma, a type of cancer, another example of a virus-associated cancer of humans.
And in this country AIDS, as you probably know, has taken over as the leading killer of young male adults.
It was not known in the early 1980s, and by 1992, just ten years later had become the major killer of young male adults in this country.
And, of course, likewise around the world.
So here's the virus.
As I said, it's a retrovirus.
Retroviruses also are enveloped.
They have glycoproteins on the outside, envelope glycoproteins.
They have a capsid like all viruses do.
They have a genome.
The genome is made of RNA.
And, as you know, through the retroviral lifecycle, the RNA is converted into a DNA form.
We'll go over that next time.
And that's accomplished by an enzyme called reverse transcriptase which gets prepackaged with the virus.
Again, it has to be there because your cells don't have an RNA dependent RNA polymerase.
So this gets prepackaged.
The virus then infects cells.
And this, again, is a somewhat oversimplification.
I'll show you more details in a second.
But the virus attaches via glycoproteins on its surface, membrane proteins on the surface of the host cell.
In this case it's a familiar protein to you called CD4.
That leads to a fusion event.
So now at the plasma membrane, which is different from what we saw with flu virus which happened in an internal membrane, at the plasma membrane there's a fusion event.
So now the viral membrane fuses with the host cell membrane and the viral capsid can lead into the cytoplasm.
There's a little bit of unpackaging that goes on.
And then reverse transcriptase, that enzyme that got prepackaged there acts on the viral genome, which is made of RNA and converts it to a DNA form, a double-stranded DNA form which is called a provirus.
And that provirus then gets integrated into the host cell genome in a random process, random integration, the host cell genome.
This is also accomplished by a pre-packed enzyme called integrase.
Once in the genome it gets treated like any old piece of DNA.
It gets transcribed into actually multiple different RNAs.
They get translated into viral proteins which come together at the plasma membrane and the glycoproteins, the capsid proteins which incorporate the genome, plus the reverse transcriptase, and you make a new particle.
OK?
So this is more or less the lifecycle.
There is some detail that I want to share with you in a moment because it's interesting.
This is what it looks like in real life.
These are HIV particles budding off the surface of an infected cell.
They have a very characteristic electron dense structure.
You can actually tell it is HIV from just the way it looks in the electron microscope.
And this is an HIV infected T cell.
Now, CD4 should have rung a bell that it's one of the proteins on the surface of helper T cells, CD4 positive T cells.
That is a major cell of infection for HIV.
And here is one such cell infected.
And all these little blebs budding off the surface of this cell are viruses getting out.
This process actually doesn't itself, the budding process doesn't itself kill the T cell but, nevertheless, T cells, this class of T cells dies.
And, actually, that's a critical event in the development of HIV-AIDS, the disease.
We'll come to that in a second, but let me first tell you one of the interesting details.
The situation is a little more complicated than the figure that I gave you on that slide.
There are two glycoproteins on the surface of HIV.
One of them is called GP120 for glycoprotein 120.
It's linked by a disulfide linkage to another protein called GP41.
And we now know that both of these proteins participate in binding and fusion.
And on the surface of the infected cell is CD4, as was indicated on that slide.
And that is the major viral receptor that is present on helper T cells, CD4 positive T cells.
It's also present on macrophages.
As well as certain cells in the brain called glial cells.
So there are CD4 positive macrophages and glial cells.
AIDS-associated dementia, which you may know about, is caused by the destruction of the glial cells in the brain.
Now, there's another protein which participates, not so much in the binding but in the fusion event that allows the virus to get in.
And this is actually a class of proteins called chemokine receptors.
There are actually multiple chemokine receptors on your cells.
And, in fact, different HIV forms bind to different of these.
But just to simplify, let's say that there is one of these which binds to the GP41 portion.
And it's that binding that allows the virus to fuse.
If you don't have this you cannot fuse.
And, therefore, the cells are not infected.
What's interesting about that is two-fold.
One, it could represent a therapeutic target.
That interaction could be a therapeutic target.
But it came to light in an interesting way which was there are people who have been followed a long time who lived in high-risk populations, drug-users or homosexual men with multiple contacts with known HIV infected people who have not themselves gotten HIV.
They seem to be naturally resistant to the development of HIV, despite documented exposure to the virus.
And they were studied.
And when that observation was made it was found that these individuals lack the chemokine receptor.
And if you lack the chemokine receptor, you may have CD4 on the surface of your cells but you don't have that critical co-receptor.
And so there can be binding but no fusion.
So your cells are resistant.
So these individuals are naturally resistant to HIV.
OK.
So HIV infects.
It infects a lot of T cells, helper T cells and other cells in the body.
Why do you get sick?
Well, as I mentioned, people get sick because of the loss, the absence -- During infection, the CD4 positive T cells get eliminated.
How exactly they get eliminated, why exactly they get eliminated we don't yet know, but we know that the levels of these cells goes down inside the infected individual.
And, as you probably remember, CD4 positive T cells -- -- are required for the development of these cells.
So if you don't have CD4 positive T cells you cannot make antibody producing cells.
You cannot make functional antibodies.
And, to some extent, they're also required for the development of cytotoxic T cells.
So that part of the immune system is also compromised.
So the virus wipes out these cells and, therefore, the immune system crashes.
And that's why it's called AIDS for acquired immunodeficiency.
Acquired because it comes through an infectious agent.
Immunodeficiency because your immune system crashes.
And when your immune system crashes you cannot fight infections.
And AIDS patients largely die because they get one or another type of infection.
So this is a graph which shows the time course of HIV infection.
And it kind of makes the point that T cells go away.
Shortly after you are infected, the level of HIV in your blood goes way, way up, as you can see here.
It spikes in the first several months after your infection.
Some people actually manifest that aspect of the disease.
Many don't so they don't know that they have such a high concentration of virus.
And, importantly, your body deals with it so you make antibodies, in black.
You make anti-HIV antibodies.
And, remember, it's these that are detected by the AIDS test.
If you've been exposed then you make antibodies.
And they stick around so that you know that you've been infected.
Likewise, T cells, cytotoxic T cells, CD8 positive T cells go up.
And that controls the level of the virus.
And it stays low for a while, but after a while, because of the affects on the CD4 positive T cells, the other cells in the immune system are no longer sustained so the B cells start to drop off and the T cells start to drop off.
And now you're starting to lose the battle against the HIV, so the levels of HIV go up.
And as they go up, you lose more CD4 positive T cells and the situation gets worse and worse.
So after a while, usually in the several years out time point, your immune system is now no longer functional and you're very susceptible to opportunistic infections.
So AIDS patients will develop infections, and the infections will progress in a way that won't happen in a healthy person.
Certain bacterial infections that you would clearly very easily take hold and can cause major problems.
Kaposi's sarcoma is caused by infection of this particular virus.
AIDS patients are very susceptible to tuberculosis, this particular fungal infection, certain other viral infections.
And in aggregate, the exposure to these various infectious agents and the damage that they're causing leads to the death of the patient.
OK.
So that's bad news.
And obviously it's have major and devastating consequences around the world.
So what can we do about it?
Well, there is some good news here.
There are antiviral therapies.
Remember that viruses largely use your enzymes.
And you cannot make good drugs against your own enzymes for treatment of a viral disease, but I've told you about one viral protein to which you might be able to make drugs.
And indeed we have.
HIV has an RNA genome that gets converted to a DNA form by an enzyme reverse transcriptase.
There have been drugs produced that block that enzyme.
And you've probably heard of the drug AZT.
There are other drugs like dideoxyinosine and dideoxycytosine that likewise bind to and block the activity of this enzyme.
They actually cause chain termination in the same way that DNA sequencing works.
And they work because this enzyme is sensitive to these drugs at concentrations where your polymerases are not, so that gives you some selected effect in the treatment of the virus.
And these agents, when used singly, work for a while.
And then the virus becomes resistant and comes roaring back.
How does it become resistant?
It becomes resistant through mutation, exactly the way cancer cells become resistant to targeted therapies.
Variant forms of HIV arise which have slightly altered reverse transcriptases that now won't bind to AZT or DDI or DDC.
And the problem is very, very serious because viruses, RNA viruses in particular are highly mutagenic.
They create variant forms at high frequency.
An HIV infected person makes ten to the tenth viral particles a day.
At a mutation rate of ten to the minus fifth, that would mean ten to the fifth variants of every nucleotide.
OK?
So variants come up frequently that are resistant to this therapy.
Oops.
Fortunately, there are alternatives.
The viral RNA gets translated into a large polyprotein -- -- which is processed by a viral protease encoded by the virus into different mature proteins.
Reverse transcriptase is one of them.
The integrase that I mentioned is another.
And there are others.
This is a viral protein, a viral enzyme.
And so, in theory, it's possible to make inhibitors against the protease.
And that's been successful.
Several companies have made inhibitors that will specifically inhibit the viral protease that work well.
They work for a while and then resistant forms arise for exactly the same reason I said.
You can create versions of protease which still work but won't bind the drug.
OK?
So what do you do?
Does anybody know what you do to overcome this problem?
You put the drugs together.
So currently HIV patients, in this country anyway, and hopefully soon around the world will get triple therapy when they're diagnosed.
And triple therapy includes two reverse transcriptase inhibitors and one protease inhibitor.
Since the development of resistance against each of these agents is independent of the other, to get resistance to all three is the product of the individual frequency.
So I said that the frequency of developing resistance to any one is ten to the minus fifth.
To be resistant to all three would occur at ten to the minus fifteenth, which is highly, highly unlikely.
And so people tend not to develop resistance to all three and, therefore, triple therapy tends to keep the virus in-check for sustained periods of time.
As you probably know, this works.
The fear is that it is starting not to work.
And there are starting to be some resistant forms developing here and they are starting to make their way into the population, so we're going to have to develop even better and more different inhibitors to now combat these variant forms.
Before I let you go, the last thing I wanted to mention, and I'll just throw it out there and you can think about it, is that what we'd really like to return to for HIV is what we can do for polio.
We'd like to be able to treat people routinely in this country or out in the field in more distant regions with an effective vaccine, which isn't possible today.
We don't have one for HIV.
You might think about what you would do to make one and why it has been so hard.
And I'll touch on that next time.
The following content is provided by MIT OpenCourseWare under a creative commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.
IT.edu.
Cancer is a disease that really needs no introduction.
It's a very familiar disease.
It's a very common disease.
The American Cancer Society predicts that there will be 1. million new cases of cancer in this country, and that excludes the common forms of skin cancer, squamous cell cancer, and basal cell cancer.
There's about another million of those.
So, 1. non-cutaneous skin cancers in the US this year.
Both wide numbers are probably 10 to 15 times as many.
In 2006, more than 550,000 people in the US -- More than 550, 00 people in the US will die from this disease --= -- more than 1, 00 each day.
So, in today's, cancer takes as many lives as died in the World Trade Center tragedy.
It's a very common disease.
It's estimated that in their lifetimes, one half of all males -- -- and a third of females will be diagnosed with cancer of one sort or another.
So the likelihood is that the person sitting next to you, or you yourself will be faced with a diagnosis of cancer in your lifetime.
And in this country, a quarter of all deaths are due to cancer.
Soon, probably this year, cancer will be the leading cause of death in the United States.
It will surpass heart disease as the number one killer.
So, it's a major disease.
And that sounds depressing.
But by the end of this section on cancer, my hope is that you will actually feel optimistic about our prospects for dealing with the disease more effectively in the future, and I don't mean in the distant future.
I actually think that within the next several years, we'll have new methods, new agents to control cancer much, much more effectively.
We are already starting to see the first examples of these.
And I'll tell you about them.
And before long, we will have many more.
So hopefully by the time you guys are at the age where cancer is the greatest risk, these numbers will actually change dramatically.
OK, so although cancer is familiar based on its prevalence, it may not be so familiar.
And I hope that is not so familiar in detail.
So, I thought I would show you a couple of slides which really give you a picture of the disease.
Lung cancer is a common type of cancer.
We'll talk about it today.
And it's diagnosed like many cancers are diagnosed through radiological methods, through imaging, in this case a chest x-ray or computed tomography, which is a serial chest x-ray basically.
And you can see this mass in the lung.
This is lung cancer at a fairly advanced stage.
Cancer is a disease of too many cells.
They often grow together in a mass, a lump, and so this is an example of cancer diagnosis, a solid tumor.
You can also have cancers of the blood, leukemias, as well as blood cell lymphomas.
And here, you can, again, see an overabundance of cells.
Here's a normal blood smear with red blood cells, these smaller guys which have no nuclei, and some white blood cells which are larger and do have nuclei.
And here, you can see a blood smear of a leukemia patient where there are way too many white blood cells, nucleated cells, and they also happen to look a little immature.
They have been differentiated into their fully developed state.
Too many cells, in this case, not in a lump but dispersed throughout the blood.
And leukemias, and some lymphomas, are diagnosed by doing a blood test of this sort.
Other cancers are detected through other imaging techniques like colonoscopy as shown here.
Colonoscopies are now fairly common.
They are recommended for all people beyond the age of 50 because colon tumors are actually rather common.
And if they are caught early enough, there relatively easily treated.
And here's the normal colon viewed from the inside.
And here's an early colon tumor called a polyp.
And I'll give you a little more nomenclature in a minute.
But this is an early stage tumor, probably not a malignant tumor.
If you saw this tumor, you might not actually remove it because it might not progress to a full-scale cancer.
But these days, because they're pretty easy to remove using very similar methods as this gentleman here with the colonoscopy, one would clip them out, and not risk the possibility that they could progress.
If they do progress, they get larger.
And more importantly, they begin to invade through the wall of the colon.
And this is very dangerous because now the cancer cells can spread throughout the body, taking up residence elsewhere and causing even greater problems for the patient.
So, this would require surgery.
A section of the colon would be removed as is shown here.
This is a section of colon where the colon cancer is highlighted by this circle.
And this is a life sparing surgery.
Individuals can live just fine.
So if colon cancer is caught even at this stage, an advanced stage, but before it has spread to other parts of the body, it's relatively treatable, even curable by surgical methods.
Now, this set of pictures gives you a sense that cancers progress.
They progress from early-stage, relatively normal looking tissue through much more advanced cancers, including to the point of metastasis.
And we are beginning to understand this process in some detail both in terms of what the cells look like, the tissues look like, but also what the genes look like that are driving these processes forward.
And I'm now going to give you some examples, or introduce the concept, that cancer progression from normalcy to malignancy is associated with changes in genes.
That's really going to be a major theme of today and in the following lectures.
But histologically, the process looks like this in cartoon form.
One starts with a normal looking tissue.
All of your cells are organized in one way or another in tissues, where they have important interactions with their neighboring cells: so here, a series of, say, epithelial cells.
Maybe these are the cells that line your gut.
They are in normal juxtaposition with their neighbors.
They are sitting atop what's called a basement membrane, proteins that are secreted by those cells and other cells to give the cell something literally to sit on it, attach to.
And then there are other cells in their vicinity including the cells that give rise to these cells, the stem cells or precursor cells, of the tissue that will be called on to differentiate into these end stage cells when these cells are lost, sloughed off.
At some frequency in all of us, alterations occur in these cells such that an abnormal cell arises.
This is a cell which has acquired a change.
I'll tell you now that it's a genetic change which allows that cell to do things that its neighbors can't do.
And in particular, it allows those cells to grow, to divide inappropriately.
Your tissues are very carefully orchestrated systems with the appropriate numbers of cells.
And cancer is fundamentally a disease of too many cells.
And so one of the first things to go wrong in the cancer development process is this orderly cell division is disrupted such that now, too many cells are produced.
And this results in a phenomena known as hyperplasia, hyperplasia meaning too many cells.
But importantly these cells look pretty normal.
So to the pathologist, and I'll show you some pathology slides in a second, to the pathologist, they would say the abnormal cells in number have a normal morphology.
Their shape looks right.
Their internal structures look right.
Their nuclei look right.
This process can continue, and probably often does continue until there is a discernible mass at least when it comes to solid tumors, a discernible mass.
And here we refer to this as an adenoma.
And I'll show you these terms again in a minute on the board, in this case, an adenoma, a benign tumor.
And you heard that were before, benign, as opposed to malignant.
And a benign tumor is a tumor in the sense that there are too many cells.
So it's a mass.
But the cells look pretty normal.
In a benign tumor, it's not considered to be life-threatening.
The cells have not undergone sufficient changes that we know for sure that they're going to, for example, invade and spread throughout the body.
And oftentimes, benign tumors are left alone because they don't have the potential to spread or to become more aggressive.
But some of them do.
And through additional changes within the cells, additional alterations take place such that now the cells really do look different from the neighbors.
Their structures look different.
Their overall shape looks different.
Their nuclei look different.
Their interactions with the neighboring cells look different.
And the structure of the tissue changes as well in the sense that now, new blood vessels have been recruited into this mass, which was important actually in feeding this growing tumor.
This is true cancer.
It's called in C2 cancer because it's present at the original site where the process started in C2 cancer.
And it's called cancer based on these various characteristics: how much the cells are growing, dividing, and also what the cells look like, their shape and their nuclear shape.
That's a problem.
But it's a bigger problem because it facilitates and usually leads to the development of these tumors into an invasive state.
So now, the cells are no longer staying where they started.
They are starting to move out into the surrounding tissue.
They break down, degrade the extracellular matrix, and actually start to migrate away from the primary tumor into the surrounding tissue, into the surrounding musculature, and ultimately into the blood vessels.
It can move into the blood vessels.
And once there, they can move throughout the body, and escape the blood vessels, and take up residence elsewhere in a process known as metastasis.
This process happens at some frequency once started.
If it reaches this point, it's a very bad situation.
Most cancers that kill people, kill people because of this process of metastasis.
90% of cancer deaths are attributable to metastasis.
And although we know many of the things that happened over the course of these earlier stages, we actually know this process relatively less well.
And obviously, we need to understand it better if we are going to control mortalities due to cancer.
This shows you some histology.
These are slices through tumors and other tissue that a pathologist would look at, and kind of emphasizes the points that I was making on an earlier slide in cartoon form.
So, here you have a lung.
And in it is some abnormal cells in the red circle here.
There are too many cells compared to the surrounding tissue.
You can see them in this kind of dark blue depiction.
And you'll see it easier on your computer screens later.
But again, these cells look relatively normal.
There are just too many of them.
So we call this hyperplasia.
These can progress into small and benign tumors called, in this case, adenomas.
And you can see on the histology that they are rather homogeneous in their structure.
The cytoplasm looks pretty much the same.
Cell to cell, the nuclei look pretty much the same cell to cell.
And they actually look pretty similar to the normal cells that surround them.
So this is not true cancer.
This is a benign tumor.
But it can progress to a true cancer, adenocarcinomas.
And you can see from the high magnification view that this looks very different from this.
The cells don't look the same one to the other, and their nuclei don't look the same either.
And what you can't appreciate is that these cells are also dividing at a much greater rate compared to these cells.
And so, this is true cancer.
And this has the potential, then, to spread to metastatic disease.
OK, so I've given you, there, all bunch of terminology.
Let me tell you a bit more about it.
Hyperplasia: too many cells.
That's all it means.
Hyper: too many.
Plasia: growth, too much growth.
Hyperplasias can result in benign tumors, which are non-aggressive -- -- nondestructive -- And they, at this moment at least, have not spread to nearby tissue, non-aggressive meaning that the cells look pretty normal.
They don't look different from their normal neighbors.
These can give rise to malignant tumors.
And the word cancer is actually reserved for malignant tumors.
If you have a tumor that's benign, you don't have cancer.
Cancer refers to malignant tumors.
And these are different in the sense that they are aggressive.
The cells are much more active.
They've changed their shape.
They've changed their structures.
They are destructive, that is, they secrete factors into the environment, which break down, for example, extracellular matrix.
And they have the potential to spread.
They've begun to invade into the local environment, and can't access the blood supply.
OK, there are cancers in many of your organs in many different cell types within those organs.
In the cancers in different places in your body are referred to by different terms.
For epithelial tissues, skin, intestines, breast, brain, lung, epithelial tissues, we call the tumors carcinomas.
Carcinomas, and depending on the particular cells that are affected, whether they are glandular cells or secretory cells, we can give them additional names, for example, an adenocarcinoma.
And that's a cancer that looks like the glands that give rise to the tumor in adenocarcinoma.
And there could be adenocarcinomas in the different places that I mentioned previously.
And there are benign versions of this tumors.
And for adenocarcinomas, the benign version the adenoma.
You can get tumors of connective tissues, and these are referred to as sarcomas.
And there could be different sarcomas depending on where they are found, for example, various types of myosarcomas would be found in what type of tissue?
These are muscle tumors, rhabdomyosarcomas, leiomyosarcomas.
Another example are chondrosarcomas -- -- which are tumors of cartilage.
In this class of tumors, which are called soft tissue tumors, were in the news just today as a matter of fact, front page of the Boston Globe if you happen to see it.
There is a site, a former dye factory that was dumping pollutants into ponds and Ashland.
And they've just determined that the exposure to those dyes greatly increased the risk of children to later develop soft tissue sarcomas.
Several children then turning into 20, 30 year olds developed tumors of this type which fits to the theme of the lecture in terms of exposures leading to cancers.
Tumors of the blood system are referred to by different names, leukemias -- -- in which the tumor cells are in the blood, B cell leukemias, T cell leukemias, erythroleukemias, myeloid leukemias where the cells are in the bloodstream like the one I showed you before, and lymphomas where the tumors are in the lymph organs like the spleen or the thymus or the lymph nodes.
So, that gives you some terminology.
And I'll be referring to some of this terminology as we go through the remaining lectures.
And it's just useful to know as you learn about and hear about cancers.
Now, cancer research, over the last several decades has been dedicated to try to understand this process of normalcy to malignancy.
How does that happen?
And the conclusion from all of this work, which I will review for you is that cancer is a genetic disease.
What do I mean by that?
What do I mean by cancer is a genetic disease?
Does anybody want to hazard a guess?
What do we normally think of when we talk about genetic diseases?
Heritable diseases, predispositions to disease, we talked about several of them previously in the class like cystic fibrosis and many others.
Cancer can be a genetic disease in that sense.
There can be inherited predispositions to certain cancers.
And I'll mention some in later lectures.
But in fact, all cancers are genetic diseases.
All cancers arise through alterations in your genes.
Those alterations in most cases are not inherited from one of your parents but are acquired through your lifetimes through mistakes or exposures to things that cause such changes.
Why do we believe this so strongly?
Well, firstly, and this has been known for some time, cancer cells don't look normal chromosomally.
If you look at the chromosomes of normal cells, they're all the same.
All have 46 chromosomes.
The chromosomes have proper structures but when you look at cancer cells, that's not true.
For most cancers, in contrast to the normal karyotype, the normal chromosomal number in architecture, and this is an image which highlights the different chromosomes by different colors.
This technique allows us to distinguish one chromosome from the other based on the color that it stains with these dyes that are used here.
Anyway, this is a normal cell, 46 chromosomes.
And here's a cancer cell.
And it's different in many ways.
There are way too many chromosomes.
So it's a defect in chromosome number.
But it's also a defect in chromosome structure.
And you can see in the boxes a couple of alterations here.
There is a chromosome 7, which is lacking a little bit of its end.
So it's got a truncation.
I think that's a seven.
Actually, maybe that's not.
Many that's chromosome 24.
Anyway, it doesn't matter.
And here, you can see, it looks like an ice cream cone: a little bit of paint on the bottom, a little bit of green on the top.
This is a new kind of chromosome that has been fused by two different chromosomes coming together.
So this is a translocation.
And here's another translocation here.
So, chromosomes and cancer cells are often abnormal in their structure.
They can have translocations.
They can have deletions.
They can have other kinds of rearrangements.
And so, based on this, we know that cancer has defects in its genomes, and ultimately we know in its genes.
We also know that agents that cause cancer -- -- and we refer to these as carcinogens -- -- cancer-causing agents, carcinogens are most often mutagens -- -- agents that cause mutations.
Carcinogens are mostly mutagens.
And therefore, it's likely that the reason they are carcinogens is because they lead to mutations in cellular genes.
We believe that very strongly, and it's based on work from many groups over long periods of time that have established this kind of correlation.
To sort of prove the point, many, many agents, many, many things that have been suspected to be cancer-causing have been tested to see whether they are mutation causing, to see whether carcinogens are mutagens.
And they are tested using assays -- -- like carcinogenicity assays where the compound in question or the agent, might be a physical agent like radiation, is tested to see whether it will promote tumor formation.
And this is typically done in laboratory animals.
So, you expose, say, a mouse or a rate to the agents in question, and you ask the question, over time, does that promote the formation of a discernible tumor?
If yes, then the agent is a carcinogen.
OK, and then you ask the question, is the agent a mutagen -- -- using mutagenicity assays.
And these are typically done in bacteria because they can be done very, very rapidly.
We can ask whether the agent is able to cause mutations in bacterial cells.
And the standard assay that has been developed, and is still in use, is to take this agent, drop it into a broth carrying bacteria, and not just any bacteria, but bacteria that have mutation in a gene required for histidine synthesis.
They can't make their own histidine, the amino acid histidine.
And so, for them to grow, they require histidine in the medium here, or in the medium on the plate, the auger plate that you grow them.
If you were to take some of these histidine minus bacteria, and plate them on a tissue culture plate, or rather a bacterial plate, that lacked histidine would any of them grow?
No, because they can make their own.
You have been given them any.
So they can't grow into colonies.
You would get zero colonies if you did that.
This is due to a mutation in a single gene.
If you can create another mutation at that site and revert the abnormal gene into a normal gene, now the cells can make their own histidine.
So, if you can mutate these cells back to becoming competent to make histidine, then if you plate these cells on a histidine minus plate, you might get colonies.
You would get some colonies.
For every bacterial cell that acquired a mutation at the right site, fixing the histidine synthesis gene, now that cell would be able to grow into a colony.
And actually, the potency of the mutagen determines how many colonies you get.
So, you can have weakly acting mutagens or strongly acting mutagens.
And as I mentioned, most agents that pass this test will also pass this test, OK?
However -- -- some known carcinogens fail that test.
They clearly are carcinogens.
You paint them on the back of a mouse or feed it to an animal: the animal will develop cancer.
And yet, when you do the bacterial reversion test, they don't show up as being mutagens.
Why is that so?
Anybody?
Yes?
They could be so mutagenic that they block the development of the bacteria at all.
That's actually a very interesting idea, not what I was thinking of.
But it's a good idea.
They are super mutagens.
So they fail the test for that reason.
Anything else?
What's different between this test of this mutagenicity, and what actually happens in the body?
Anybody?
Yes?
That's good.
And I'll actually come back to that.
There is another good suggestion, but I'll come back to that answer in a second.
The difference is metabolism.
Your body metabolizes things that you get exposed to that you eat, or that you inhale, or that you inject yourselves with.
It metabolizes it oftentimes to detoxify it or to make it more water-soluble so that you can excrete it.
There is a lot of metabolism breaking down or changing the things that you get exposed to.
And so, it's been observed that there are certain compounds which are not themselves mutagens but are pro-mutagens.
They have the capacity to turn into mutagens.
And in the process of metabolic enzymes, they become mutagens.
And there are very important carcinogens that fall into this category.
And one of them is shown here.
This is a polycyclic hydrocarbon called benzoate pyrene.
It's a very important ingredient in cigarette smoke as well as in barbecued beef.
It's a very potent carcinogen.
It passes this test with flying colors.
But if you do the bacterial reversion test, you don't see anything.
And the reason is that in the body, many of these double bonds had to be broken and replaced ultimately by hydroxyl groups to make this compound more water-soluble so that it can be excreted.
And an intermediate step in the detoxification, the hydroxylation, is the formation of epoxide.
And these epoxides are actually highly reactive to DNA.
And these molecules, these intermediates, are highly mutagenic.
And so, when you make these in your body, thinking you're doing a good thing, you're actually doing a bad thing by creating a mutagen.
And it's this that's the mutagenic agent.
And so, now the simple bacterial reversion test has been replaced by something called the Ames test named after Bruce Ames from Berkeley in which the compound in question is not just given directly to the bacteria.
Its first passed through an extract of the liver, usually rat liver, and it's in the liver where these detoxifying enzymes do their job, these cytochrome P450's that are referred to here are present in abundance in your liver.
And so if mutagens are going to be formed, they should be formed in this in vitro extract.
And then, one takes the product of that and performs the bacterial mutation or reversion test.
OK, and now the things that were carcinogens but not seen to be mutagens can be seen to be mutagens again.
OK, but -- -- still -- There are still some truly non-mutagenic carcinogens.
And two examples are alcohol and asbestos.
These do not pass any mutagenicity tests that I'm aware of.
But both of them are clearly linked to cancer development: alcohol in the case of liver, cancer, and head and neck cancer, and asbestos in the case of mesothelioma, the lining of the lung.
So these are not mutagens.
So how did they promote cancer formation?
Based on the mechanism that was suggested over here.
These agents cause irritation in tissues -- -- which results in tissue destruction, loss of cells -- -- and their replacement of those cells, cell replenishment.
So, it recruits a lot of cells to begin to divide when they normally wouldn't.
So it results in increased proliferation, and the increased potential for mutation because - when cells divide.
When they duplicate their DNA, there is an inherent risk of making mistakes.
It's not an error-free process.
So the more cell division there is, the more opportunity there is for error, and therefore, the increased risk of acquiring mutations and becoming ultimately a cancer.
OK, I want to take a couple of minutes talking about one really important environmental carcinogen, something that we get exposed to.
And that is cigarette smoke.
I mentioned that benzoate pyrene is one of the important ingredients in cigarette smoke, but it's actually not the only mutagen.
There are actually hundreds of mutagens in cigarette smoke.
And this is the most important environmental carcinogen that we have in our environments.
And it results in a large number of deaths per year.
Tobacco smoke is responsible for a high percentage of the 175, 00 people who die from lung cancer in this country each year.
About 150 were either smokers or former smokers.
And the association between lung cancer and cigarette smoking is really striking.
If you look here, the green line represents cigarette consumption in males from the beginning of the century, the 1900s, through 1960.
And you can see it rapidly increased through availability and social acceptance and social trends.
Smoking was really rampant in the society.
And shortly thereafter, with about a 20 year lag, you could see that lung cancer risk increased dramatically as well, which is directly attributable to the cigarette smoke exposure.
It took time because cancers don't develop right away.
They develop over years, and so you need to get exposed to stuff over a long period of time.
But eventually the lung cancer started to rise.
And you could see the shape of the curves are virtually identical.
You can also see that the smoking rate has dropped off a little bit in men, and actually has dropped off a little bit starting about 15 to 20 years ago with respect to lung cancers as well.
So smoking cessation has an effect.
If you reduce the exposure, you can reduce your risk.
That's true of men.
Women have not caught up yet.
Women started smoking later than men, about 20 years later, and their lung cancer rates followed, then, their increases in cigarette smoking with about a 20 year lag, and you can see now that lung cancer among women is still rising.
And it recently passed breast cancer as the most common type of cancer deaths among women.
So lung cancer is a major disease and is directly related to smoking and smoking history.
Surprisingly, given that fact -- -- still 47 million Americans smoke, in recent polls something like 26% of men and 22% of women, a remarkably high number.
And if you go to other countries, the numbers are worse.
So the message clearly has not been adequately delivered.
But what is even more surprising to me was the results of a survey done in 2002 where high school students are asked whether they smoked.
What do you think?
What percentage of American high school students smoke?
Anybody?
It was 28%; more than one in four smoke.
And I've heard recently that the number is increasing since 2002.
So, this very, very detrimental agent which one controls oneself is clearly not going away and will lead to, unfortunately, the deaths of many of these individuals because of lung cancer.
And by the way, it's not just lung cancer that smoking is a problem for: emphysema, heart disease, and other diseases as well.
I read a statistic which was really shocking to me.
Of that however many billions of people are present on the planet today, the 500 million of them will die early because of complications of cigarette smoking.
500 million people will live less than their full lifespans because of cigarette smoke.
So, if you learn nothing else from this class, if you currently smoke, stop.
And if you don't smoke, don't start.
OK, that was my little sermon.
Hopefully it had some effect.
And it provides a segue into -- -- the sources of the mutations that occur in genes during cancer progression.
We've just been discussing exogenous, or environmental mutagens, cigarette smoke being one, some lead exposure being another, dietary carcinogens I mentioned, environmental pollutants like the dyes in the ponds and ash land.
They are common.
They do certainly have an effect, but there are probably somewhat overblown except for some light and tobacco smoke, and then these rare instances of high exposure to very, very toxic things.
A more common problem are replication errors, mistakes that your cells make in duplicating their DNA.
They put in the wrong base or they skip a base, or other errors take place during DNA duplication.
Your DNA can break inside of your cells as it gets moved around from place to place or because of an exposure to certain things from the outside.
And then this can lead to translocations or deletions or other rearrangements.
And I'll tell you later about how those can lead to cancer-causing mutations.
Defects in chromosome segregation, dividing the number of chromosomes properly between cells, is another problem.
And I shown you an example of that.
Here you can see that the number of chromosomes is also wrong in addition to their structure.
So, defects in chromosome segregation are a problem in cancer.
Defects in DNA repair processes can then facilitate further mutations.
And various metabolic processes can produce mutagens insider cells like super oxide molecules which are very reactive and very mutagenic.
Your cells have ways of detoxifying, but they're not always perfect.
You can also take antioxidants to try to reduce your risk of getting this kind of exposure.
But nevertheless, these endogenously produced mutagens are another common source of mutations in cancer.
So, we get exposed to things.
We produce things, we make mistakes, and in our cells, alterations occur in cellular genes which drive this process forward.
And overall, therefore, we observe a clonal evolution of more and more abnormal cells in the development of cancer.
And that's captured here.
Clonal evolution from a normal cell, a cell that acquires a single mutation, this cell now has increased capacity to divide.
So it divides more.
More of its progeny are produced.
We are maybe in the stage of hyperplasia here.
And within this increased mass of cells, another mutation occurs.
Now the cell has even greater capacity to divide or begin to act abnormally relative to its neighbors.
It begins to grow.
And within this increased collection of cells, yet another mutation occurs.
There is a clonal evolution of more and more abnormal cells.
And we know that cancers, in fact, have acquired many, many changes in their DNA as evidenced by the chromosomes, but also as evidenced by our knowledge of the sequence of key genes that are affected in cancer.
Now, as we'll discuss, many of these genes affect proliferation -- -- and cell death.
Again, cancer is a disease of cell number, too many cells.
And this process is normally very carefully balanced inside of your bodies inside of your tissues.
There is an equal balance between proliferation -- -- and cell death so that in normal homeostasis -- -- you have the appropriate members of cells produced as die off, and so that ultimately you are in balance.
But in cancer, the balance is shifted.
It's shifted in the direction of proliferation, and away from cell death.
And the consequences of that are that you've now produced too many cells for the tissue, too many cells in the gut, too many cells in the blood.
And these cells, then, accumulate and acquire additional changes that imbue them with those qualities that advanced malignancies have.
Increased proliferation is one thing, but importantly it's not the only thing.
Increased proliferation and cell death are important qualities for sure.
But also as I mentioned, cancer cells develop in other ways that are important in the development of malignancies.
Today, for example, a recruit a blood supply.
They secrete factors that cause the blood vessels to grow into them, to feed the tumor.
They increased their motility.
Cancer cells move around.
They move into tissue.
They move into the bloodstream.
They move back out again.
And that's what this intravasation term means, moving into the blood supply, and other characteristics as well, how they remodel the extracellular matrix, or how they avoid the immune system, all sorts of things that cancer cells require in order to survive to get bigger, to thrive, are acquired through alterations in cellular genes.
Now, the final thing, and I'll close with this and pick it up again next time, the final thing that convinces us that cancer is a disease of your genes is that we've now sequenced the genes of cancer cells and have found alterations.
Like in this gene here that you've been exposed to, the RAS gene, in which in normal cells we see one sequence and in cancer cells we see a different sequence, and we can understand that as a change in the activity of this important signaling protein that causes cells to divide inappropriately.
And I'll stop there.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu We're going to continue our discussion about cancer and cancer genetics today.
One small point of clarification: apparently some people were confused about my out of balance scales here related to proliferation and cell death, pointing out that if there were more proliferation and less cell death, then the scale should shift this way instead of this way.
That's an astute observation.
It doesn't really matter which way the scales point really.
The point was that they are out of balance, which is critical for tumorigenesis.
But anyway, in review, last time I told you that cancer cells arise from normal cells over a series of events that relate to mutations in cellular genes.
And I gave you lots of evidence that things that cause cancer affect our genes, and that our genes are disrupted in cancer cells.
And I briefly mentioned that we have evidence for mutations in cellular genes in cancer cells.
And I'll show you that again today.
But all of that brings us to the point of needing to know, what are the genes?
What are the genes that are affected in cancer?
And this is important to help us understand how cancers arise, so from a basic biomedical research point of view, but also as we'll see next time, this information is actually very useful in teaching us how to diagnose cancer more effectively or treat it more effectively.
And I'll just show this slide, which I showed at the end again as a summary.
The cancer cells arise from normal cells through the acquisition of mutations in cellular genes through this process which we call clonal evolution, more and more abnormal cells growing out from this accumulating mass of cells.
And as we'll discuss today, these genes affect proliferation and cell death, this out of balance scheme that I alluded to before, but not only these processes.
And I emphasize this because these other important processes get a little bit of short shrift in our discussions.
But angiogenesis, this process of recruiting a new blood supply in cancer is very important, and also an important therapeutic target.
Cell motility, movement of cells, as I mentioned, it's important to get cells away from the initial mass in the process of metastasis, and invasion likewise.
And several other processes are also affected in the course of tumorigenesis.
So what are the genes?
How do we know which genes are affected in cancer?
Well, this set of experiments, really, has occurred over the course of arguably the last century.
I'm trying to find the genes that are mutated in cancer cells.
And many people point to landmark experiments done in 1910 by an investigator at the Rockefeller University by the name of Payton Rauss.
Payton Rauss was a virologist, studied viruses at Rockefeller University.
And one day, a farmer from Long Island brought a prize-winning hen to this famous doctor at Rockefeller University in New York because the hen was sick.
Specifically, the hen had a big tumor in its breast muscle.
And if you remember, that would mean the tumor was a sarcoma.
And he wanted the doctor to cure his prize-winning hen.
Payton Rauss said, thank you, I'll do what I can, and then prominently killed the bird in order to try to study what was giving rise to this tumor.
And he did a famous experiment.
He took the tumor, ground it up, filtered it so that there were no cells present in the filtrate, and also used a small enough pour filter so that bacteria, which were also known at that time, were removed.
So, he had a filtrate that contained no cells, no cancer cells, and no bacteria.
And he took this and injected it into a non-tumor bearing bird.
And he observed over time that this bird developed a tumor.
So he was able to demonstrate the transmissibility of something which could cause cancer in an otherwise unaffected animal.
Because it wasn't cells and it wasn't bacteria, he suggested that it was a virus.
And it turned out to be, over the course of the next 50 years or so, the virus which was responsible for this disease was identified and given the name Rauss sarcoma virus, named after Payton Rauss.
And the structure of this virus was also determined.
It was found to be a retrovirus.
Retroviruses are in the class of HIV.
Their genomes are made of RNA, and they convert it to a DNA form.
We'll learn a little bit more about retroviruses in a few lectures.
It was a retrovirus, and the structure of its genome was determined eventually.
And it was found to contain in its genome the genes that the virus required for replication.
And all viruses of this class has a very similar set of replication genes.
They need to produce more virus.
But at the end of the viral genome, at the extreme right side of the viral genome, there was another gene which was given the name sarc for sarcoma.
And it was termed an oncogene, onco for mass, oncology, cancer associated gene, sarc-oncogene.
So, the virus it was found can cause cancer in birds by virtue of transferring into those cells a potent cancer associated gene called sarc.
And Payton Rauss went onto win the Nobel Prize, actually about 50 years later in the early 1960s for this work on the discovery of Rauss sarcoma virus.
And it was extremely important.
But it was still a mystery as to where this cancer associated gene came from.
Where did the sarc gene originate?
Was it a viral gene which the virus used in some fashion to cause cells to proliferate, perhaps for its own replication?
Or did it come from some other source, some other origin?
And here in the story entered two researchers, Mike Bishop and Harold Varmus.
Mike Bishop is now chancellor at UCSF.
Harold Varmus is president at Memorial Sloan-Kettering.
They did this work at UCSF, and I got quite familiar with it because I ended up working for Harold Varmus as a Ph.D student.
This work was done before I got there in about 1975.
And they set out to ask this question, what is the origin of this sarc-oncogene?
Where did it come from?
And what they were able to show was that this Rauss sarcoma virus derived from a virus that was related to it called avian leucosis virus.
And avian leucosis virus had only those replication genes, the genes that the virus uses to produce more virus what they were able to show was that this avian leucosis virus, when it infects cells, and here's a chicken cell, and here's the nucleus of that cell, inside that cell are normal copies of the sarc gene.
They were able to show that the virus actually acquires this sarc-oncogene from the cells that it infects.
Now, it doesn't do that every time.
In fact, most of the time, the virus just goes in and replicates and produces more virus.
But rarely, and really it's very rarely, the virus actually acquires a piece of the sarc gene and sticks it into its own genome, thereby creating -- -- Rauss sarcoma virus, which has the structure that I showed you above with the replication genes and the sarc oncogene.
And this is done through a process called transduction, which is a form of recombination.
The main point of this observation, and the really important point of this observation was this potent oncogene is present in chicken cells.
The sarc is a cellular gene.
It's not just a viral gene.
It's derived from host cells of chickens, and it's not only in chickens.
It's in all vertebrates including humans.
So, you all have a sarc gene in your cells, too.
And this was a really momentous observation.
And it led Bishop and Varmus to win the Nobel Prize back in 1989.
So, what I've just told you is that your cells carry a potent oncogene, sarc.
So, why don't you all have sarcomas?
Anybody?
Remember, we are still being filmed.
Anybody?
Yes, somebody's got their hand up, but I don't see them.
Yes?
Almost right.
The gene is not expressed properly, OK?
The virus has co-opted this gene and placed it into its own genome.
And now this gene is being regulated improperly.
Maybe it's expressed too well or at the wrong time, or in the wrong cells.
And some combination of those things allows the gene to cause cancer in this setting, whereas when the gene is properly regulated, and the cells of chickens or in you, it's not cancer associated.
It doesn't cause cancer in its state in the cellular genome.
So, this was extremely important, and as I said, led to the Nobel Prize for these guys.
RSV -- -- is what's called an acutely transforming retrovirus, acutely transforming meaning that it can transform cells, turn them from normal to cancerous quickly.
And as I said, it's a retrovirus.
We now know of many acutely transforming viruses of animals: mice, rats, chickens, turkeys, other things.
And from them, we have learned the identity of about 50 or so oncogenes, all of which were derived from the host cell.
So, we know from this type of experiment of many potential cancer associated genes.
Importantly, there are no known acutely transforming viruses of humans.
Importantly, the vast majority of human cancers are not associated with viral agents.
Occasionally, there are such cases, but they are quite rare.
Most major tumor types are not caused by viral infection.
But there is one big exception to this -- -- human papilloma virus.
Human papilloma virus, or HPV, and specifically high-risk types of HPV called type 16 and type 18 are associated with cervical cancer in the US, but mostly in other parts of the world.
And that's important, because if one can control exposure to HPV, or eradication of HPV, one could do something about cervical cancer, and that's actually happening now.
And I'll mention that again next lecture.
But I want you to remember that most cancers, human cancers, are not virus associated.
But the viruses have taught us a great deal about cancer genetics.
OK, so if most human cancers don't arise from a viral infection, the infection with an oncogene carrying virus, then where do these mutations come from?
What are the mutations in human cancer?
And here, just after these experiments were done, around 1980, comes another one of our local heroes, Bob Weinberg, who has a laboratory at the Whitehead Institute here at MIT.
He hasn't yet won the Nobel Prize, but I think he's going to, based on this work and a lot of other work probably in the next several years.
And what Weinberg's lab did back in this early 1980s.
Was to look in human cancers, so not cancers of animals, but in human cancers.
And specifically, they took bladder cancers and isolated the cells, and grew them in the laboratory.
They assumed that inside of the cells were altered genes, and to become altered through the life of this individual, giving rise to this cancer.
And they wanted to know what those genes were.
And so, they isolated the DNA from these bladder cancer cells, and they transfected it into cells from mice.
They sheared it up, and they introduced it.
That's what transfected means, into the cells of mice.
And these cells that they put them into were fairly normal -- -- mouse cells, by which I mean they grew a single layer on the bottom of the dish.
There are structures look like normal cells as opposed to cancer cells.
They were relatively normal cells.
And what they observed was that a small percentage of these cells that had picked up some bits and pieces of the DNA derived from this human cancer began to behave abnormally -- -- and produced what were referred to as transformed cells, cells that looked more like cancer cells.
So they had converted otherwise normal mouse cells to what looked like cancer cells through the introduction of one or more human genes.
And they further demonstrated that these were cancer associated cells, cancer-causing cells, by introducing them into immunocompromised mice, and observing that over time these cells could produce tumors.
So these were cancer cells indeed.
So that's very interesting because it says that there are altered genes in these cancer cells that can convert normal mouse cells into cancer cells.
And then to obvious question was, what are those genes?
What is the gene that's responsible for this process?
So, they -- -- isolated the human DNA from the transformed mouse cells.
And I won't tell you how they did that, but suffice it to say, it's possible to isolate the human DNA away from the mouse DNA in these transformed mouse cells.
They cloned the human DNA into bacteria using recombinant DNA methods that we've talked to you about previously.
And they sequenced the cancer-causing gene.
It turns out it was just a single gene that was providing this property.
And it turned out that this gene was H-RAS, RAS being a gene that we've talked to you about before, and important signaling protein in cells.
Moreover, they sequenced the copy of RAS that was present in the cancer cells, and compared it to the sequence in normal cells in the other cells of this individual.
And they found a single alteration.
OK, so what you are looking at here now is the sequence of the H-RAS gene in normal people or in the normal cells of this cancer patient has a particular sequence.
It encodes a protein, H-RAS, which if you recall is an important signaling protein that shuttles between a GTP-bound form and a GDP bound form.
In its GTP bound form, it's active and signals, and then it undergoes hydrolysis reaction.
So, the GTP is converted to GDP, and becomes inactive.
The mutant copy found in the cancer cells and in these transformed mouse cells has been alteration which blocks the ability of the proteins undergo GTP hydrolysis.
Therefore, the protein gets locked into this active signaling state, and stimulates proliferation continuously.
And that's why it's a cancer associated gene.
OK, so here we have for the first time evidence that a normal, cellular gene gets mutated, presumably by an error or by exposure to some mutagen that is responsible at least far an aspect of the transformed state.
OK, so that's very, very interesting.
It also raises an important question.
If you note that the change between the normal gene and the abnormal gene is signal-base change that converts this alanine residue, sorry, this glycemic residue to a valine residue: single base change.
Such single base changes occur in your cells all the time.
It's estimated that the mutation rate inside your cells is about ten to the minus ninth per cell division.
And if you calculate how many cells you have in your body at any given time based on how many cell divisions have arisen, we can estimate that there are about ten to the third to ten to the fifth -- -- rasmutin cells in all of U as you sit there now.
You are all carrying 1, 00-100,000 cells that have this very mutation.
So, why don't all of you have cancer?
Why don't all of you have bladder cancer or some other form of cancer?
Anybody?
The reason is that cancer is not a single step process.
Cancer cells arise from normal cells to the acquisition of many mutations that occur over the lifetime of the individual.
So a single mutation in RAS is not enough to give you a cancer.
It might be enough to cause the cells to behave somewhat abnormally.
But that in and of itself does not produce even hyperplasia necessarily, let alone an adenoma, an adenocarcinoma and invasive cancer.
It's necessary to acquire multiple mutations.
So you may have cells that are on their way.
But they are not fully there.
And hopefully they'll never get there.
Now, I want to remind you that this pathway, this signaling pathway that involves RAS is something we taught you about previously.
This should look familiar based on the cell signaling lectures that we had before.
RAS sits right here, this important signaling molecule that links events that take place at the cell membrane.
Sound at all familiar?
Yes?
Transmembrane receptors?
Growth factor receptors that bind to ligands and cause a series of events that for example convert RAS to its GTP bound form, and then initiate a kinase cascade of protein kinases, map kinase, kinase, kinase, kinase, kinase, kinase, remember the little movie?
Moves into the nucleus, phosphorylates transcription factors that initiate the expression of target genes that, for example cause cells to proliferate.
And when RAS gets stuck in its on state as it does here, these signals are sent constitutively so that the cell is being told to divide all the time instead of in a regulated fashion.
Now, I point out all the different components which I've told you about in previous lectures because, in fact, it's not just RAS, this one signaling protein that gets mutated in cancers.
Many different of these signaling proteins get mutated in cancers.
It is a very common pathway that's affected in not necessarily all but a very high percentage of cancers of different types.
The RAS gene is mutated at about 30% of cancers.
30% of all cancers have mutations in one of the RAS genes including, for example, 90% of pancreatic cancer, 50% of colon cancers, 30% of lung cancers carry mutations in this gene.
But other components get mutated as well.
For example, the growth factor or growth factor receptor genes get altered in cancers.
They become amplified.
And I'll mention what I mean by that in a second period.
We observe translocations, these chromosomal alterations in different genes, including the transcription factor genes that lie at the bottom of this pathway.
There can be deletions that affect the function of these proteins, make the function abnormally, and subtle mutations like the one in RAS itself.
And there are other examples, subtle mutations that lock the proteins, for example, in an active signaling state.
So, DNA amplification is one such mechanism to alter the function of a cellular gene in cancer.
And an example of that is in a case of a gene called HER2.
It encodes a growth factor receptor like item number two up here.
It encodes a growth factor receptor.
And when it's present in single copy or two copies per cell, it produces a certain concentration of this growth factor receptor protein.
And it signals at normal levels.
And that gives rise to a normal amount of proliferation.
And this is particularly important in cells of the mammary gland.
Mammary epithelial cells depend on this signaling pathway to be produced in the right amounts.
Unfortunately at some frequency in cancer cells, and specifically cancer cells of the breast, and sometimes ovary, an amplification event takes place so that now instead of having just one copy of this HER2 gene on the chromosome, there are many copies.
An error takes place in the replication process so that instead of just copying this gene one time, it gets copied multiple times.
So, now we have too many copies of this gene, and therefore in these cells, the concentration of this receptor is significantly higher than in normal cells such that in these cells, there might be ten times or a hundred times the amount of signaling, or ten to a hundred times the amount of proliferation.
So, the dysregulation of this gene's function by making too much of the protein, it's an otherwise normal protein.
There's just too much of it, causes the cells to get overstimulated and divide too often.
That's bad news for cancer.
The good news is we now have a therapy, and I'll put you about that next time, directed against this protein.
I'll mention another mechanism of activation, and that's translocation.
I showed you slides of cancer cells that have broken chromosomes or rearranged chromosomes.
Those rearrangements often produce abnormal genes, sometimes fusions between one gene and another.
Other times, rearrangements of the promoter of the gene such that the normal promoter of the gene, which is responsible for its level of expression is replaced by a different promoter, which might give too much expression.
And that's true for an important oncogene called MIC.
The MIC oncogene is a transcription factor.
It's like number eight down there.
It's a transcription factor that binds to DNA and regulates the expression of other genes.
And it's normally expressed from a, let's call it, weak promoter.
You don't want to have too much of this protein produced.
It needs to be produced at normal levels.
So, we get a certain concentration of the protein, which gives -- -- regulated cell division.
You get as much television as you need based on the appropriate signals in the environment of those cells.
In certain types of leukemia, one sees a translocation.
The DNA is broken and rejoined so that the MIC oncogene doesn't have its own promoter anymore.
Instead, it's next to a very strong promoter.
And now, again, the concentration of this protein is increased.
And this gives rise to deregulated cell division, same idea.
Change the proper regulation of this signaling network, and now the signaling network is broken.
And as I said, there are many, many such genes that are known.
We know about 50 or so oncogenes from these acutely transforming viruses.
We probably know of 50 more oncogenes because of their changes in the DNA of cancer cells.
And by the way, there's a lot of overlap between these two sets.
Many of the genes that were found in the context of viruses were later found to be mutant in human cells having nothing to do with viruses.
And RAS is one such example.
MIC, actually, is another example, and a relative of this HER2 gene is another example.
OK. OK, so the slide is meant to reinforce this notion that cell division, normal production of cells, is tightly regulated through these signaling networks, and that oncogenes function normally, their normal cellular function is to control this process.
That's what they do in normal development.
That's what they do in normal individuals.
But they can be subverted in cancer cells by mutation.
It makes them act in a deregulated or uncontrolled fashion.
There is another set of genes that act, in a sense, in opposition to the oncogenes.
These genes function to inhibit this process of cell division.
And they go by another name: tumor suppressor genes.
Tumor suppressor genes function to block the normal cell division process or to inhibit the production of cells.
And they, too, are important in cancer.
You can think of these genes as the on switch and the off switch of an electrical circuit.
In cancer cells, oncogenes become hyperactive.
When the lights switch gets stuck in the on state.
OK, it gets taped open.
So now, you're getting signaling through this circuit continuously.
Tumor suppressor genes normally inhibit cell division.
And so, in cancer, these genes get inactivated.
It's like the light switch getting turned off.
And the process is normally regulated that way, and these get stuck off.
OK, so want to tell you a little bit about tumor suppressor genes now which is the other major category of cancer associated genes that we'll cover here.
Briefly, just to reiterate that point, in addition to the light switch analogy, people often use the gas pedal and breaks analogy, or the go signal and stop signal analogy.
So, I'll use that as well.
In normal cells, when they receive the signals to divide, they are stimulated to go, that is, to divide to make more such cells.
This happens in development when you need to make more cells.
It happens in wound healing.
It happens in normal homeostasis.
And then, when the process is completed, there are stop signals sent.
The go signals represent oncogenes, normally regulated, stop signals tumor suppressor genes normally regulated.
In cancer, oncogenes become deregulated.
And pushed too strong they send constitutive signals.
And as a consequence, too many cells are produced.
And likewise, the stop signs, the stop signals are lost so that one produces even more cells.
OK, so hopefully these are useful analogies to think about these two opposing sets of oncogenes and tumor suppressor genes.
OK, so to get into tumor suppressor genes, sorry, this just makes the point, oncogene mutations, tumor suppressor gene mutations.
To get into tumor suppressor genes, the second class, an important class of human cancer genes, let me tell you a little bit about this disease which is called retinoblastoma.
Retinoblastoma is a childhood tumor of the eye, of the retina.
It's not very common.
Only about one in 40,000 children develop this tumor.
But it's very important what it's taught us about cancer genes.
And you can see in the normal eye a normal looking retina, and in the cancer containing eye, these blobs of tumors.
We now know the gene that's responsible for this disease, the gene that's mutated in the formation of this disease.
And that gene is the gene called RB, named after retinoblastoma.
And this gene functions as an important regulator of cell cycle progression.
If you remember cell cycle progression, mitosis, S phase mitosis, the G1 phase, the G2 phase, again, dim memories from earlier in the class.
The RB protein, which goes by the name of PRB, this is the protein encoded by this tumor suppressor gene, RB, acts to inhibit cell cycle progression.
That's its function.
It blocks cell cycle progression literally like the brakes on a car.
Now, you might ask, if it's there and functioning, then how do you ever get cell cycle progression?
And the answer is that this active inhibitor can be inactivated through phosphorylation.
When this protein gets phosphorylated, it becomes inactive.
And now, cell cycle progression can occur.
It becomes inactive when the cell receives growth stimulatory signals like through that signaling network that I mentioned previously.
These signals, then, act on cycline CDK complexes, cell cycle associated kinases that you were taught about in the cell cycle lecture, and these go about inactivating the RB protein, allowing the cells to cycle.
In addition, when the cells shouldn't be cycling in the presence of growth inhibitory signals, inhibitory signals are sent to the cycline dependent kinase proteins, and they don't function, thereby blocking the inactivation of RB.
RB stays in its active state and blocks cell cycle progression.
OK, so that's how this really important cell cycle regulator functions more or less.
And it's a very important human tumor suppressor gene, which is itself mutated and probably 30 or 40% of human cancers.
And the pathway that regulates this gene, this pathway, is mutated in probably 95% of human cancers very, very commonly affected.
OK, so given what I've told you so far, would you expect the mutations that we find in the RB gene in human cancer to be activating mutations or inactivating mutations?
Inactivating.
It's the brakes.
Cancer cells want to get rid of the brakes.
And so we find inactivating mutations in the RB gene in human cancers like deletions or nonsense mutations, that is, stop codon mutations.
OK, that's the kind of mutations we find in this gene.
For the oncogenes, which we focused on at the beginning, are the mutations in oncogenes dominant or recessive?
Are the mutations in the oncogenes dominant or recessive?
The oncogene mutations are dominant.
When you acquire mutation in an oncogene, whether it's a subtle mutation or an amplification, it doesn't matter what's happening to the other allele.
This thing has new function, gain of function.
It will transform or start the process of transformation, regardless of what is happening to the other allele.
And that's evident, by the way, in this transfection experiment.
This would only work if it were a dominant mutation.
So, oncogene mutations are dominant mutations.
Are tumor suppressor gene mutations dominant or recessive?
These are recessive mutations.
Here, it matters, the state of the other allele of the RB gene.
As long as you have one functional copy of the brakes, you can do this.
You can control cell cycle regulation.
In order to make this dysregulated, you need to get rid of both copies.
You need to get rid of the brakes entirely.
So, these mutations are recessive.
And this class of mutations are recessive.
Tumor suppressor gene mutations are recessive.
So this raises an interesting question.
Tumor suppressor gene mutations are recessive.
I've told you that tumor suppressor gene mutations occur commonly in human cancer, not just this RB gene but others too, and yet you are all diploid.
You all carry two copies of the tumor suppressor genes.
So, how do we ever find mutations in these genes in human cancer?
Is it enough to mutate just one copy?
No, because they are recessive mutations.
So, to find mutations in tumor suppressor genes in human cancer, it's not enough to mutate just one copy.
Both copies need to be mutated.
Recessive mutations require there be two mutations.
And this is what typically happens in the development of a tumor with a mutation in the RB gene, or any other tumor suppressor gene.
Again, there are probably 20 to 50 different tumor suppressor genes.
Initially, a cell, which has a normal copy of RB on both copies of chromosome 13, that's where this gene sits, and here we are looking at a linked polymorphic marker, big A little a, if you remember polymorphic markers, big A little A.
First, one copy of RB gets mutated.
In this cell, there is still a normal copy of RB.
And therefore, this cell that has acquired this mutation is normal.
It's functionally normal.
However, it now has just one copy of the RB gene.
And so, it's predisposed to becoming a cancer cell if that normal copy gets lost.
And what this shows you are various mechanisms by which the normal copy of the RB gene can be lost.
The other copy can acquire its own mutation.
Or, the chromosome that carries the normal copy of the RB gene can get lost in the developing cancer cell.
Or, it can be a recombination of that, that replaces the normal copy of the gene with the abnormal copy of the gene.
And all three of these things happen in the development of cancer cells.
And depending on which mechanism is used -- -- one can observe something referred to as loss -- -- of heterozygosity, loss of heterozygosity within the cancer cell.
You will notice that this cancer cell, sorry, this normal cell, is heterozygous for this A allele, big A little A.
And you will notice that here and here, the cell has become, the cell carries only one version of that A allele, big A.
So, there was heterozygosity.
And in the cancer cell, there's now loss of heterozygosity.
Loss of heterozygosity is a hallmark for tumor suppressor genes, the involvement of tumor suppressor genes in the development of cancer.
OK, now I told you several times that proliferation is important and cell death is important, too.
Cell death control in cancer: I don't have a lot of time to tell you more about this in this class, but I want to just emphasize that the pathways that lead from cell death signals, the signals that tell cells to commit suicide to ultimately undergo this process called apoptosis are also dysregulated in cancer.
For example, there's a gene called BCL2, which becomes mutated in certain types of cancer.
BCL2 normally blocks this process of cell death.
And in this type of cancer, it becomes hyperactive.
So there's not as much cell death.
BCL2 is an oncogene.
Another gene, very important gene in human cancer is P53.
It stimulates cell death.
P53 gets lost in a high percentage of cancers.
It's a tumor suppressor gene, and therefore gets inactivated.
BCL2 is an oncogene.
It gets hyperactivated in cancers.
I'll talk more about this pathway next time, but just to put in your minds, P53 is a very important signaling protein responsive to many signals upstream, giving rise to signals that lead to death as well as arrest.
Hang on, please.
Hang on.
Two minutes left.
And this slide is just a summary of what we've been talking about so far.
Control of proliferation, control of cell death, the normal balance, perturbations in these pathways, oncogene mutations that stimulate proliferation, tumor suppressor gene mutations that stimulate proliferation.
Alterations in apoptosis, oncogene mutations that block apoptosis, tumor suppressor gene mutations that block apoptosis.
All of these are important in cancer.
Next time I'm going to tell you that some cancers are caused by inherited mutations.
And this is a pedigree of such an example, and lastly to remind you, mutations affect proliferation in cell death.
But they also affect many other things: invasion, angiogenesis, as well as metastasis.
And I'll stop there.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
Handout for today's lecture.
And you'll get a sheet that asks you about your preferences for recitation time.
For the moment, that's the critical piece of paper.
What you want to do is, if you are -- well whoever you are, get your name on their, your ID number, I don't remember, it probably asks for your email address.
It does not ask for your year.
Please put that, and also put down if you're a BCS Course Nine major.
We will attempt to lottery into the class as many of you as we can.
And we will do that, basically, today.
And you'll get an email from us telling you whether you are in the class or not.
I will take care of all those add-drop cards then next Tuesday when we meet again.
How many people here are here without being lotteried in, just to give me a feeling for?
Well, all right how many people are lotteried in?
How many people don't think they know what that means?
OK, good.
Well, we'll see what we can do.
We will try to squeeze as many of you in as we can.
Those of you who are not lotteried in, do not check the little box that says, I was lotteried in.
Because we can check and if you a nasty lying person, we don't want you in this class and we will do bad things to you or something.
Let me see.
What, yes --
[UNINTELLIGIBLE]
OK. Let's do it that way.
Who still needs a syllabus?
OK. Oh, I see we have a new improvement this year.
Last year they didn't have a cool digital clock there.
OK. Who still needs a writing assignment?
I'm just trying to see which, who still needs a writing assignment?
It's the one that says writing assignments on it.
All right.
So they're scattered around randomly, too.
I don't understand how this is working.
But I'm sure it's working somehow.
I need a Handout 1 by the way, when somebody has a Handout 1.
No, I've read the syllabus.
[UNINTELLIGIBLE] Well, here, you just got deputized.
Who still needs the cool TA signup, recitation signup sheet?
This lovely person will help you.
Up there somewhere.
If you're just coming in, come down here and collect handouts.
Somebody please deliver me enough handouts so we can give people handouts.
There's the handout for today's lecture.
There are four things altogether, sit around until the other two show up.
All right, these are the other two.
All right let's try this game again.
Who's still looking for a syllabus.
A couple of syllabi, out in the cheap seats back there.
And a few over there.
OK, who still needs the writing assignments?
We need some writing assignments down front.
And who still needs the recitation signup sheet?
That's the critical one.
Recitation signup sheet, who's got those?
OK. Once you've got stuff, you can go.
Leave me all the extras.
At this point, if you don't have one of the handouts, pick it up later.
Unless you're still missing the recitation signup sheet.
None of the other stuff [UNINTELLIGIBLE] OK. Is there anybody who still needs the recitation sheet?
No, OK, good.
So, name, ID, email, year.
Whether or not you're a BCS major.
Pick your top four choices.
Number them one, two, three, and four, because we'll try to put you in your first choice.
And tell us whether you're a winner in the Haas lottery.
There will be handouts, typically, in each lecture.
Typically they will be there and there when you come in.
And, remember the there part.
Because when 300 people all try to get it there, it's kind of a pain.
So either side in the front.
How many people need more time with this exciting recitation sheet?
We want it now.
We want it instantly, because we're going to solve the problem of recitations.
Actually, I suppose we don't have to solve it before the end of class this way, do we?
But we might as well, while we've got everybody here.
So do that now.
And while people are playing with that, I will entertain any other entertaining questions about the course that are on people's minds.
Yes?
[UNINTELLIGIBLE]
Is the fifth edition book OK?
You can use the fifth edition.
We're on to the sixth edition.
You can use the fifth edition as long as you don't come and complain if you get the question wrong on the final because the info happens to have been in the sixth edition.
Basically, you're not going to flunk the course using the fifth edition.
It's not going to be a big issue.
But it's a dealer's choice.
You do it at your own risk.
The risk is smallish.
You can decide if it trades off sufficiently with the money involved.
I like the first edition best, myself.
I think it's all been downhill since then.
But I wouldn't use the first edition at this point.
Anything else that I should be answering about the course while people are playing with forms?
Yes
[UNINTELLIGIBLE]
It's an hour.
Once a week.
At a time that you are currently determining for yourself with luck.
OK. Time for another round of polling here.
How many people still would like more time to play with this exciting form?
I will take the opportunity, at least I will try to take the opportunity, depends if I can find the opportunity This is a Hass-D course.
Your writing papers.
Some of you know that that's hard work for you.
Some of you will discover that you and your TA disagree about your talents in writing.
In either case, you may want to avail yourself of the fact that we have three writing tutors attached to this course.
They're good, they do great stuff.
At least your predecessors have told me so.
And it is my belief that two of them were going to be here today.
Oh yes, look at that.
There they are.
I couldn't tell, they looked so young like undergraduates.
Why don't you introduce yourselves?
[UNINTELLIGIBLE]
When you read the syllabus, you will discover useful things like their names, again, and their email contacts.
And also useful thoughts like, 3 am the day before it's due is not the time you want to send a panicked note saying, I can't get started on my paper.
Think about it ahead of time and they will be useful.
Also useful to you will be, we will have a series of workshops in the library for how to do research for these sorts of papers.
And details about that will emerge later on in the term.
But I would strongly encourage people to go to one of those sessions.
Because doing research for college-level papers in a course like this is different than what you did in high school, probably, and it's certainly different than feeding the keywords for whatever topic you're writing about into Google and calling it a day.
You laugh, but some people did that last year.
It was not a pleasant sight.
OK.
I think that people are done with this exciting form, don't you?
She wants them passed to the center aisle.
It's very bad form to rip up the paper of the guy next to you.
OK, are there any of those forms still out there?
All right.
Now that we've done the really exciting part, let's try saying something about what this course is about.
And why you bothered to try to get yourself lotteried in.
Many, many, many many -- any other forms?
Last chance.
Many many, many many MIT courses are of the form, so when you go to the first class you get the syllabus.
You look at the list of topics.
You say, I don't know anything about these topics but I sure hope that by the end of the course I know something about these topics.
And Intro Psych class is different from that.
Because if you take a look at the list of topics in the course, which you can find by the way on the back page of the syllabus but you don't need to go looking for it right now, you will discover memory, learning, cognition, emotion, personality, intelligence, a whole set of topics that you already have some notion about.
They're terms that you use in everyday speech.
That you can converse about perfectly happily.
The job of this course is to dig behind the, what you would know in what's known as folk psychology.
Folk psychology is a term coined in the 19th century to refer to the psychology that folk know.
As opposed to the psychology that one would learn in the academy, like you are here.
What we used to say, actually, when the term was defined for me in graduate school, I think somebody said, folk psychology, that's the psychology that your grandmother knows.
That doesn't work so well.
Because your grandmother may well have a Ph.D in psychology these days.
Folk psychology is what the psychology that you would know without bothering to take a course like this.
The psychology you just pick up on the street.
This course is designed to go beyond that.
Well, let me dig up an example.
Let me close the door.
And so, as an example, I could assert oh, let's see, I need a person.
Well, this is why one sits in the front.
You're a person.
That's very good.
What's your name?
[UNINTELLIGIBLE]
You're Mark.
OK. Who is that woman sitting next to you?
This is Laura.
OK.
I could say that Mark loves Laura.
It may or may not be true.
For all I know, Mark, it is true.
Well, this is going to be a more interesting example then sometimes.
But, all right.
If I assert that Mark loves Laura without much knowledge of either Mark or Laura here, you had -- that's not a sentence that's hard for you to understand.
But what is it that we're actually talking about?
So, all right, what's love?
I think, I didn't check, I was going to check this morning but I didn't check.
I think that probably the line you would get in a dictionary would assert that love is an emotion, and then it would go on to describe what kind of an emotion it is.
But it's an odd emotion if it is an emotion.
If you think about other emotions that are pretty straightforward.
Let's say, sadness or happiness to give two straightforward examples.
You know what sadness and happiness feel like.
They have a distinctive feeling to them There is a distinctive feeling to being in love.
There's no doubt that there's a feeling.
It's actually an interesting question why we use the same word feeling to talk about feeling in love, and feeling the table.
What's the commonality there?
But the feeling of being in love is not simple in the same way that the feeling of sadness of the feeling of happiness is.
For instance, you can be sad or happy in love.
You can be in love and the experience of that could be either sad or happy.
The experience of being sad cannot be either sad or -- sad is sad.
It's, in some sense, an atomic sensation in a way that love does not appear to be.
You can wake up in the morning feeling sad.
Disembodied sort of sadness.
Or disembodied sort of happiness.
You can't really, it's hard to imagine what it would be to wake up and say I feel love.
It's not clear that that makes sort of a quasi-religious sense that you could get to it.
I feel love for the whole universe, or something like that.
It's lovely, but, you see the distinction.
So maybe it's not an emotion in the simple sense that sadness or happiness is.
Maybe it's a way of thinking about your current state.
You know, I feel happy and my heart is pounding and I'm sitting next to, oh God I've forgotten her name already.
Laura.
Lara.
No U. L-A-R-A Lara.
And Mark.
They both have two letters in common.
That's the basis for this relationship.
Anyway, where was I?
You're not going to wake up feeling a disembodied sense of love.
Maybe maybe what it is is a thought the state that you are currently in.
So maybe it's a cognition rather than an emotion.
But if it's a cognition it's a different kind of combination from what you might think of as, sort of, an atomic cognition.
Because it has this aspect of feeling to it.
That that cognitions don't, what's the, capital of Equatorial Guinea.
I have no idea.
But it's a thought that, I can think about that fact.
It doesn't carry with it any deep feeling unless it turns out to be a question on the final exam.
But you can have thoughts that aren't colored in this emotional kind of a way.
So it's not a simple cognition.
It might be more useful to call it a motivation than to call it an emotion.
Motivations are like emotions in the sense that having this affective, affect is the sort of jargony term for feeling, this affective component to it.
But the sorts of things that get talked about as motivations are things like thirst.
So, it being warm, I'm thirsty.
That's a faintly unpleasant state and a motivation is something that motivates me to do something to change my state.
And hunger is another good example.
So what kind of motivation would love be?
Love might be a motivation that it's directed to an object of some sort.
And, I probably shouldn't have caught that cold from the third-grader in my house.
That's better.
It's a motivation that, if love is a motivation, what it's motivating you to do is to perhaps get closer, to possess the object of desire.
Now Mark and Lara are busy sitting there saying, oh boy, that doesn't sound so good.
But love, of course, can be used not just in the Mark and Lara sense.
But you can love your laptop or something like that.
And have, we won't go there, Mark, it's OK.
But those are two clearly different senses of love.
Or, at least your chortling suggests that you think of them in different senses.
You might ask what love is for.
And the motive, if thirst is a motivation, it's because you need liquid in order to be alive.
If you don't drink stuff you're going to be dead.
What's love for?
Well, the usual answers to that, these days in psychology, would come out of the chunk of psychology that's called evolutionary psychology.
That sees these core motivations, if you like, as things that evolved over the history of the species and of life in general, to serve useful purposes for the organism.
In this case, love is presumably in service of that great evolutionary good which is to get your genes into the next generation.
That your goal, if you're a thoroughgoing evolutionary psychologist, your goal in life basically is to get genes into the next generation to perpetuate your genes.
And the sex and reproduction thing is a good way to do that.
It's not the only way to do that, it's important to note.
How many of you are only children?
Well, actually, let's go at it the other way.
How many of you have siblings?
OK. One way for you to gain this sort of evolutionary immortality of getting your genes into the next generation is to be a really good aunt or uncle.
You protect and preserve that little niece or nephew of yours, and they're carrying a bundle of your genes, too.
So it's not that having and raising children is the only evolutionary route to the future.
But it is a good one.
And the love of business would seem to be related to that.
But it's not necessary.
If you think about it, well, all right spider.
Let's think of spiders.
Spiders, the reproduction thing works fine with spiders.
It's not clear how much love and romance there is involved here.
It's not clear for a couple of reasons.
One of them is, it's very unclear what's going on in the mind of an animal in general.
You just don't have any access.
In fact, or at least, you have only the barest of influential access.
In fact, I don't have any access to what's going on inside your mind.
It is an assumption on my part that you guys have mental lives like mine.
You could be what the philosophers, what is, in philosopher jargon, you could all be zombies.
Which, if you're doing philosophy means, you're things that look human, behave like they're human, but aren't human because there's nothing happening in there.
There's no, you could be cunningly designed machines.
I don't know that, but it's a useful assumption that your mental life and my mental are the similar.
It is not a terribly useful assumption that my mental life and a spider's mental life are similar.
And in any case, if you look at spider behavior, it doesn't look much like romantic love.
I mean, the guy spends most of his time trying to avoid being killed by the female spider, who is typically much bigger.
He does a variety of courtships, we'll talk about this later in the term, but I cannot resist, there are variety of spider species where the spider does engage in a lot of courtship behavior.
The male spider, he brings her presents.
Like, he brings her a dead fly.
The reason he brings her a dead fly is that she's busy chowing down on the dead fly, she doesn't bite his little head off.
And it's not desperately romantic.
But, in any case, it does work perfectly well for reproduction.
It suggests that there might be, even when you're talking about love between potential mates, that there might be a distinction to be made between love and, well, lust or some sort of sex drive or something like that.
There is evidence at a neurochemical level that these are distinct.
The chemical pathways in the brain that are involved in romantic love seem to be those that are also involved in parent-child bonds of attachment.
They involve chemicals like, well, your brain generates opioid chemicals, which are like opium, which is why opium is a powerful drug when taken externally, taken from the from the outside.
But in any case, the bonds of love involve those chemicals.
The bonds of lust involve things like testosterone and other androgens and estrogens that they may have talked about in endless length in high school biology or something of that sort.
But there's evidence that these are separate.
How did they come together?
Well, evolutionary psychologists suggested the link might be that if you bring love to bear, this romantic bonding, this attachment to bear, then you've got pairs of people who stick around.
Stick to each other in ways that are good for the begetting and raising of children.
And that maybe this bonding between parent and child got co-opted into being romantic love.
Between potential mates.
Another way of putting it would be to say, well, maybe romantic love exists because straight-out, just regular old lust, is not compatible with civilization.
If people are just simply acting on their evolutionarily based desire to mate, it's hard to run a civilization.
It's hard to run a university, for instance, because people are busy.
So what you've got to do, what you've got to do, is, if you're going to run a civilization you have to find some way to channel that sex drive into other activities so that the rest of civilization can keep going.
Now, if you start talking in those terms, you're talking in terms that you will see are very much like what Freud said.
Freud argued that things like romantic love are necessary in order to have a civilization.
In the absence of, if you're a spider you don't need romantic love.
But you're also not going to have, spider civilization is somewhat more limited.
Now, how does Lara know that Mark loves her?
Well, he said so.
Right?
Well, that's cool.
I can say so, too.
Right?
I love you.
I might even sort of, kind of, mean it in this sort of generic, I love all of humanity kind of sense.
But it's obviously different.
Now, there are various other ways that he can be indicating the fact that he is in love with her.
But she's got to figure this out.
Is he really in love with me, or is he just kind of, actually he's got to figure it out too.
Am I really in love with her?
Her active perception is, it would be perfectly possible for her to be wrong.
She looks at him, she thinks, he's in love with me.
And he's thinking, I'm not in love with her, I'm just faking.
Could be.
Now, could it go the other way?
Could he be thinking, I'm in love with her, and it not be true?
Is that logically possible?
Now, and who can tell you that it wasn't true?
Your mother?
You don't really love her.
Well, you probably wouldn't buy that any more.
But you would have once.
If you plot, let's see, how are we going to do this?
All right.
Well, this is age.
I know this axis is age.
So if you plot the percentage of people who say, you ask kids, who knows your inner emotions the best?
You or, give an open-ended question but the real categories are you or Mom and Dad.
It turns out, so, if this is you and this is Mom and Dad, that function crosses the 50% point at the surprisingly late age of early adolescence.
Before that, the majority of kids will assert that Mom and Dad know their emotions better than they themselves know it.
How could that be the case?
Well, it's hard to get yourself back into that mental state at this point.
It gets easier again once you have kids of your own, because you're looking at your kid.
And you're, like, your four year old kid.
And your kids doing, and you say, gotta go to the bathroom.
No, I don't gotta go to the bathroom.
You gotta go to the bathroom.
Nuh-uh, I don't have to go to the bathroom.
You sure you don't have to go to the bathroom?
No, no, OK, I guess you don't have to go to the bathroom, let's all get in the car.
I've got to go to the bathroom.
And so, if the kid is a little self-reflective he thinks, hey, wait a minute, first of all not only do I have to go to the bathroom, but you know Mom and Dad, they, like, knew that before I did.
That's cool.
That's also a little scary because I thought a whole bunch of other things.
I wonder if they know those things too.
Eventually, you figure out, no, they didn't know that stuff.
Which is just as well for all concerned.
But returning to the love example the point is that you need to know.
You need to perform an active perception to decide somebody else loves you.
And you need to do an active sort of self-perception to decide that you love somebody else.
You might also ask yourself at that point, well, what is it that you love?
Physical attributes?
That's considered sort of crass.
And even worse would be to say, I love her for her money and possessions.
That sounds terrible.
But if you ask yourself, what are you watching on TV, there's a continuous pairing of love, sex, and cars, soda, anything, right?
Well, I can't think of any particular ads at the moment because I don't watch enough TV.
I'll ask my kids if there are any good sex and soda ads on at the moment.
But there certainly have been.
In any case, typically, we say things like, I love her for her mind.
Or something like that.
Now, you could love your laptop for its mind, too.
It doesn't seem to be quite the same thing.
Now, all right.
So, the question of what it is that you might love.
Where is that love, if in you?
Does that question make any sort of sense?
Well, there's a traditional answer to that.
If you asked where love lives, the sort of conventional, out-there in several hundred years ago, probably would be simpler.
Kind of answer would be, heart?
Well, yeah.
Liver is good too but that's only if you're, the cool white virgin snows upon my heart have abated the ardor of my liver, says the prince guy in Shakespeare's Tempest.
There's this notion, very much a folk psychological notion, that emotions are resident in the viscera.
And that makes a degree of sense.
Because, when you see the object of your heart's desire, you know it's your heart's desire because your heart goes thump, thump, thump.
Right?
Now, you think that if love is localized, if there is some chunk of you that is your love for whoever this significant other might be, where's that likely to be?
Brain, somebody muttered up there.
Yeah, it's probably in your brain.
You can imagine why that was a hard notion to come by.
Because you look at the object of your desire and you don't suddenly say, aah, my head.
And you probably won't be sending brain-shaped chocolates to anybody at Valentine's Day either.
But it is an interesting question to ask, whether or not, all right brain maybe.
Is it localized enough that there's some little piece of brain that we could go to and say, Mark's love for Lara is there.
And if we went in and pulled that board out of his computer, that he'd know who Lara was, he knew everything else, they just wouldn't be in love any more.
Is that level of localization plausible?
Is the love normal?
Is his love, Mark loves Lara for her mind.
Is that normal?
Suppose he loved her for her foot?
Would that be normal?
What's the difference?
Why would one be -- and if it wasn't normal, who gets to decide that it's normal?
Do we, as a community, get to decide?
Does the family get to decide?
Is there some objective way to decide whether something is or is not normal?
And if it's not normal, what do we get to do about it?
Do we get to, if it's not normal, would drugs fix it?
Would surgery fix it?
Would some sort of therapy fix it?
You can see what's going on here.
From a simple sort of statement like, you know, Mark loves Lara, we can get to, essentially, any of the topics that will show up in a course like this.
And you could have done, we wouldn't have had to use love as the particular example.
Behind any of these things that we use perfectly comfortably in everyday life is a large set of very interesting questions, only a very few of which, of course, I can manage to address in an introductory-level course.
But the goal here would be to give you some sort of overview of the sorts of topics and the sorts of answers that modern academic psychology has come up with.
The way that we do that is through lectures.
Through recitations, which should not be treated as optional.
We expect you to be there.
They're not just like reviews for the exam, or there are no problems in an introductory psych class the way there would be in chemistry or something.
you're expected to be there.
They're an integral part of the course.
And there's the textbook.
It's a good idea to look at it.
I won't go into a lot more detail about the mechanics of the course.
The syllabus will tell you most of what you want to know.
And it's a really good idea to actually take a look at the syllabus and to take a look at the writing assignments.
Because they have due dates attached to them, and things like that.
I will answer any blazingly mechanical questions about the course, if they have occurred to anybody at the moment.
Otherwise I will plunge into the brain in a moment.
After satisfying my first motivation yet more.
OK, let me say that the sort of ideological grounding of this course is that the course, like academic psychology generally, is materialist in its ideology.
What does that mean?
What that means is that the mind is what the brain does.
And that if you have thoughts, if you have feelings.
If you have memories, that these things arise out of the brain.
This is not to say they're necessarily, if we wanted to teach a straight neuroscience course we'd just teach you about the brain.
But it is to say that it is in distinction to another old philosophical position.
Which is dualism.
Which says that the mind is somehow an immaterial something that is separate from the brain.
It interacts with you, it interacts with your body, but it is separate.
The reason for talking about the brain for this lecture and a chunk of the next lecture is this notion that the mind is what the brain does.
And, well, if we want to find out what the brain does, let's see.
We could make a little bit of a list here.
You go up.
Gotta get rid of a little more of whatever that was.
So, do I have a copy of the handout here somewhere?
Yes.
OK.
So I can ask because it just has for broad classes of methods on the handout, without describing them.
How might we go about finding out what the brain does?
Oh, this reminds me to say, anytime during the course of the term, if you've got a question, comment, or whatever, feel free to raise your hand.
And I will attempt to answer it.
If I think we're being hopelessly you diverted from where I think we need to get in the lecture, I may have to cut off discussion.
But I'm more than happy to have the opportunity.
So don't, just because it's a great big lecture, don't feel that you've just got to sit there and not do anything interactive like raise your hand.
Of course, if you raise your hand you might end up exposing the details of your romantic engagements with the person sitting next to you, but that's a risk you have to take.
Anyway.
If you wanted to find out how the brain worked, what the brain did, how could we go about doing that?
Anybody got -- Yes?
[UNINTELLIGIBLE]
You could cut the brain open and look at it.
OK, that's Option 0 on my list of 1-4.
Not because it's trivial in any sense, but I'm looking for techniques that get more at the function than at the structure.
But there is a certain amount that you do learn from just looking at the structure.
In, fact, Descartes, one of the founders of the dualist view, looked at the anatomy of the brain, thought, if the immaterial mind and soul are interacting with the brain, well, the brain is a double structure, right?
It's got these two, sort of, symmetrical hemispheres.
That interaction between the immaterial soul and the body must be at one point.
He looked at the anatomy, saw the pineal gland, which is a piece of brain lying on the mid-line and thought, because there's only one of those, that must be the point.
So, you can make inferences from structure alone.
That particular inference is wrong, but you could do it.
Yes, the person in front there.
Yes, you
[UNINTELLIGIBLE]
OK, MRI.
We will make that into the larger category of imaging techniques.
For psych purposes, the most dramatic version of this is so-called FMRI, for Functional Magnetic Resonance Imaging.
This is the new entry on the list, really.
Remarkable breakthrough in the time that I've been teaching this course is the ability to now look at the structure and some aspects of the function of the brain in intact, you know, a wide awake human beings.
You simply couldn't do that any more.
And that's a remarkable -- well, it's a remarkable boom for psychology and a huge advance for medicine.
So, for instance, if you had a brain tumor.
There are symptoms of having a brain tumor.
But the only way to find out if you actually had a tumor, because you want to take this tumor out if you could, the only way to find out, a generation ago was to do exploratory surgery.
And open up the brain and see where the thing was.
Now you'd be in an MR scanner by the end of the day.
And we'd know.
So, yes, very useful technique.
You're going to offer me another one?
[UNINTELLIGIBLE]
OK.
So we'll put this under the category of lesions.
Brain damage.
If this is the newest, this is probably the oldest of the basic techniques.
If you damage brains in the same way, you end up getting symptoms that are similar across people.
And even across species, in many cases.
And that is a tipoff that, first of all, the brain has something to do with mental life.
And that specific bits of brain have specific things to do with specific aspects of mental life.
The real advances in this sort of research, and this sort of understanding of the brain, came, I guess, mid- to late nineteenth century, with advances both in medicine and in, I suppose, what you could call military technology.
In earlier times, if you got a penetrating wound to the brain, odds were that you were simply going to be dead.
But by the mid- to late nineteenth century, people were surviving with penetrating brain injuries.
And it became clear -- and bullets could be leaving comparatively small lesions.
And so it became increasingly clear that different bits of brain damage did different things.
Let me find myself a place to draw another picture here.
I need a brain.
So, here we go.
One quick, this is your brain.
Just to orient things here.
OK, so.
Front, back, bottom, top.
If I'm facing that way, this would be looking at the outside surface of the left hemisphere.
You've got two cerebral hemispheres, about this size, stuck inside your head.
The brain's got lots of wrinkles.
There are a couple of large ones that I'm drawing here for my purposes.
Because the cerebral hemispheres are conventionally divided up into four lobes.
The terms for which are written on the handouts, so I will just give you the initials rather than writing out the whole thing.
Conveniently, the frontal lobe is at the front.
After that, you just simply have to learn them.
Temporal lobe down at the bottom.
Occipital lobe at the back.
And the parietal lobe on the other side of this big central sulcus that divides the frontal from the parietal lobe.
What was found, I think originally in the Franco-Prussian War, if memory serves.
1870s, was that if you got lesions to the back of the brain here, to the cortex, the wrinkled outside surface is known as cortex.
That's from the Latin word for cork.
Which is what anatomists thought it looked like.
If you get lesions back here, you get problems with your vision.
Very specifically.
And, if you got a great big lesion that took out the entire occipital cortex on both sides, you'd be functionally blind.
But if you got a small lesion, let's say in the left hemisphere, you would have a small region of blindness.
And it would always be on the right side of the visual world.
And that's relative to where you're actually looking.
Fixating at the moment.
So, if I fixate on this guy there, who I'm now staring at, then if I had a lesion in my left hemisphere some region out here, I would be blind.
If the lesion was in my right hemisphere the blind region would be on the opposite side, on the left.
That's the sort of thing that you could discover from lesion studies.
And so, lesions happen when you get brain injury from a bullet.
Lesions happen from strokes.
One of the great advantages to imaging technologies is that in the 1870s, or even until quite recently, the only way you knew about where the lesion exactly was, was if the patient died and came to autopsy.
You could look at the brain, and look at what was damaged.
Now you can look at that brain in an intact individual and see where the damage might be.
So, damage to the brain can tell you something about.
It's got the drawback that you're trying to infer normal function from an abnormal brain at that point.
And that's not perfectly straightforward.
But it does provide, and is perhaps the oldest technique for providing lots of information.
Yes, you were about to give me another method?
[UNINTELLIGIBLE]
OK, behavior is item, that's 0, than behavior is going to be 5.
And we'll spend an awful lot of the course talking about behavioral measures.
But in behavior, without going in and doing something with the tissue of the brain, you're essentially treating the brain as a black box.
At that point.
It may tell you something about the brain in connection with the rest of this.
But we're looking for techniques that are specifically about studying brain itself.
Yes?
[UNINTELLIGIBLE]
[?
Wake ?]
surgery?
[UNINTELLIGIBLE]
This just sounds cool.
Keep going on.
What did you have in --
[UNINTELLIGIBLE]
What's the surgery part?
[UNINTELLIGIBLE]
I'm just trying to see if there's a 6 here that I'm just not thinking of.
That would be cool.
Otherwise I can sort of --
[UNINTELLIGIBLE]
Oh yeah, actually.
That's one I didn't put in, typically.
Which is that you can go and start infusing stuff into, chemicals into, the brain.
And changing behavior.
Do I want to say anything more about that at the moment?
Good, I'll use that one.
Thank you.
We'll call this stimulation in general.
Stimulation.
You can go in and stimulate brain.
So, I was going to do that and segue from the chemical stimulation to electrical stimulation, but you got there for me.
You can go in and stimulate the brain electrically in rats.
But you can also do this in humans.
In fact, so back in the 1950s, Wilder Penfield, psychologist who's working with some neurosurgeons up in Montreal.
And what they were doing was, they were doing neurosurgery on patients with intractable epilepsy.
Epilepsy is an electrical storm in the brain, basically.
And often is started by a piece of abnormal or damaged tissue that acts as a generator.
And then what happens is a sort of an abnormal electrical activity spreads from that point of generation across the brain or across some chunk of the brain.
And causes a seizure or lapse of awareness.
The sort of symptoms that you get in epilepsy.
Often, this is controllable by drugs.
But in some cases it's not.
And one of the treatments that can be tried is to go in and try to lesion this little chunk of brain.
If you take out the generator you can often reduce the severity or eliminate the seizures.
But if you're going to take brain tissue out, you gotta be really careful.
Because, well, actually where I put this is a pretty good example.
We knew from lesion studies going back, again, into the nineteenth century, that areas around here that go by names like Broca and Wernicke's area, which are the names of 19th century neurologists, actually.
Vitally important in the production and understanding of language.
If you are right-handed, those areas exist in the left hemisphere of the brain.
So this is a left hemisphere the brain.
That'll work fine.
And if they're damaged by stroke, you will have trouble with understanding language or producing language, or both.
Depends on the exact nature of the.
Stroke But very bad lesions to have.
So, if you were to discover that the epileptogenic focus, the piece of bad tissue, was sitting in there, you would say, look, I'm sorry.
We don't want to do this surgery.
Because while I could perhaps reduce the frequency of your seizures, it's going to mean that you've got real problems with language.
And that tradeoff is not worth it.
On the other hand, if the generator was here, I can reduce the frequency of your seizures.
And you'll have a relatively small area of blindness out in the right visual field somewhere.
Say, look, I'm not thrilled about being blind in some part of my visual field.
But that tradeoff is worth it.
So you need to know where you are in the brain in order to do this sort of surgery.
And what Penfield did was to stimulate the brain of awake individuals.
In effect, to ask, what's this bit of brain doing.
Now, how can you do that?
Doesn't that kind of hurt a lot?
The answer is that they're under local anesthetic to get through the skull and the outer surface, the membranes across the surface of the brain.
Because, in case you hadn't noticed, digging holes with a sharp stick in your skull really hurts.
But once you get to brain, there are no pain receptors in the brain itself.
There's nothing.
Once you've got the brain exposed, you can do anything you want to the brain.
And the brain won't particularly complain.
I don't suggest you try this.
But you could.
Now, let's step back.
Why have pain receptors at all?
What's pain good for?
Yeah
[UNINTELLIGIBLE]
Pain's there to keep you from hurting yourself.
If you don't have pain, and your hand's on the hot stove, you say, oh, I'm cooking my hand that's no big deal.
In fact, there rare cases of people who are congenitally insensitive to pain.
And it's bad news.
They end up injuring themselves.
That being the case, why not have pain receptors in your Brain Yeah
[UNINTELLIGIBLE]
Yeah by the time -- if you're thinking in evolutionary terms, by the time the bear is chewing on your brain, you might as well just kind of go with the experience.
Because it's not going to matter much.
So, the advantage here is that you can go in and stimulate brain tissue.
And it's a somewhat, from the sound of it, somewhat uncanny experience.
But it's not an unpleasant experience.
And so, all right, so what Penfield did, stick a little electrode, say, here.
And put a little electrical current there.
And what the patient would report, if it was here, this is to the parietal side of this big central sulcus.
What the patient would report is is, it feels like you're touching my foot.
OK. Move the electrode a little bit.
OK, now it feels like you're sort of touching me in the middle of my back.
Now it feels like you're touching my hand.
It's the end of my nose.
And what Penfield found was that the surface of the body was laid out across the surface of the brain.
And the left hemisphere would actually be the surface of the right side of the body.
I don't know what that is And it is feet up, head down, as I recall.
But that chunk of brain which is called somatosensory cortex, the term's on the handout somewhere, represents the sensations coming from the skin.
Now, I haven't drawn a terribly realistic picture here.
But the actual map is hugely distorted.
yes?
Is something -- OK.
I'll do that.
Pay no attention to the man behind the curtain.
Anyway.
So this map, known as the homunculus, for the little man in the head.
The sensory homunculus, specifically.
It's massively distorted.
Some chunks of your skin are heavily over-represented in the brain.
Some are heavily underrepresented.
So what's over-represented?
Hands.
And?
Face.
And on the face.
Lips.
Lips and tongue and things like that.
So, somewhere in Gleitman, a chapter you'll read tonight with luck, there's a picture.
People love making little models of the homunculus.
So you project it back out into the world, and what would it look like if this was a 1:1 mapping.
And you get this guy with you huge lips and huge hands, and a little tiny back.
And the way to get a feeling for this is to ask yourself, if you were trying to figure out what something felt like, how would you go about doing that?
Well, you would go and explore it with your hands.
A little kid, less restrained than you, would also probably put it in its mouth and feel it that way.
What would you wouldn't do is rub it against your back, or something like that.
Because it's the hands and the lips and stuff that have the very dense sensory skin receptors.
And it's sparser on your back.
Now, you will also, you can test the prudery already of your introductory psych text by checking out whether the homunculus is anatomically lee.
Correct The genitalia are well represented in the sensory homunculus also.
My recollection is that they are well represented in Gleitman there are texts where it's kind of -- Actually, there's one great version where the homunculus is wearing, like, a little towel.
I've seen that once.
That was cute.
So stimulate here and the person feels like you're touching them on their skin.
If you go around to the right hemisphere, they'd feel the stimulation down the left side of the body.
Go to the other side of this, into the frontal lobe, and instead of saying, oh, I feel like you're touching my leg, the leg would twitch.
Left hemisphere, right leg.
You move a little further the arm would twitch.
Or just one muscle in the arm, or something like that.
Here you've got a motor homunculus, laid out across the motor cortex.
And this chunk of the brain is important in the generation of voluntary movements.
Again, it's a distorted map.
Hands, very over-represented.
If you're a monkey, feet are very over-represented.
But you, who cannot hang upside down from a tree by your feet or your tail or whatever, your feet are comparatively impoverished in their representation.
Tongue and vocal apparatus, heavily over-represented.
So, same sort of story.
Sensory and motor.
Now, these days, you don't even need to go and cut open the skull to do this sort of thing.
Transcranial magnetic stimulation, or TMS, for short, involves generating a large magnetic field close to the skull.
As your physics background will tell you, that generates an electrical field.
If you shape the magnetic field right, you can generate an electrical field in the brain tissue.
And, basically, produce the same sort of brain stimulation that Penfield was producing in an open skull.
With the same results.
Nancy Kanwisher, who's a professor in the department here, used to be up at Harvard.
Got on the front of the Harvard Gazette for teaching her intro class with a TMS simulator in hand and say, want to see where my motor cortex is?
Pow.
Pow.
No, I'm not going to do that, sorry.
The reason I'm not going to do that, by the way, is I'm a chicken.
TMS is almost undoubtedly safe.
Lots of people do it.
But it sounds too much like things that I learned about when I was taking intro psych, that were used to generate epileptic foci in rats.
Let's stimulate, pow, pow, pow, pow, and now the rat's got epilepsy.
It doesn't seem to happen in TMS, but I'm a chicken.
When it comes to my brain.
Yes?
[UNINTELLIGIBLE]
Well, we'll find out if I can spell homunculus.
Paul, is it a U or an O?
U?
Ho-munc-u-lus.
One of the reasons for handouts, by the way, is so that when people ask me can I please spell it, it's on the handout somewhere.
Because the answer's no I can't.
So the homunculus, probably derived from the Latin for a little man in the head, and I apologize to the women.
I was trying to figure out today what the little woman in the head would be.
But my Latin has waned since I took it in college.
Maybe it's the femunculus, I'm not sure.
But, in any case, that's the homunculus.
So, that's stimulation techniques.
I'm shopping for one more useful technique here.
Anybody care to offer me another technique for figuring out what's going on in the brain?
[UNINTELLIGIBLE]
Case studies are typically these lesion studies.
Somebody has something wrong with them.
And we try to figure out what's wrong with them.
That goes way back, by the way.
The early efforts to figure out what bits of brain did come from the phrenologists, early 19th century.
Phrenology.
If you know anything about phrenology, you learned it from the Cartoon Network.
Because you watched Bugs Bunny beat Elmer Fudd over the head.
And then the bumps showed up and you felt the bumps.
Looking at me like, what's he talking about.
Or you know it because there's a wide, any of these umpteen cartoons or sculptures where you have a head that's got little functions all over it.
Good, somebody remembers it.
I'm not just making up weird stuff.
The phrenologists believed that the shape of the skull reflected the underlying shape of the brain, that bigger meant more of that particular function.
Which is rather like this.
I mean, it's not a completely wacko assumption.
And they then used a case study system to try to figure out what different parts of the brain did.
So if you wanted to know where criminality was in the brain, you go and find yourself some criminals.
And check where they got big chunks of their brain.
And that's where those sort of maps came from.
It's a rather ad hoc sounding kind of process, now.
So, amativeness, lust, is typically located back here.
The reason it's located back here is because Spurzheim, one of the founders of phrenology, had in his care a, how did he describe it, I think she's described as a passionate widow.
And every time Spurzheim put her his hand on the back of her neck, it became red and inflamed.
And he therefore concluded that that's where lust lived.
Erm.
It turns out, though, that if you read old phrenology texts, the original phrenology texts, they read not unlike introductory psych texts, or introductory brain science texts, with a few unfortunate assumptions.
Like the notion that they skull shape reflects underlying brain shape, which it does not very well.
There was -- yes, pink person
[UNINTELLIGIBLE]
OK well that's -- so, we'll put the, looking at the cells part in under the anatomy.
My wife would hate this.
My wife is a neuroanatomist.
That's what she does for a living.
She would want many categories for that.
All by itself.
But, yeah, so you can go look at, certainly as, well you can use this in a number of contexts.
If you're going to go stimulate, these days, if you're going to go stimulate brain cells, you might be doing it on a slice of brain in a dish.
To see how it's connected to the next one.
And you would be looking at the individual cells under a microscope while you were doing that.
Yes
[UNINTELLIGIBLE]
Oh yeah, that's a, genetics is another candidate for 5.
But I will, actually if you take a look at, that allows me to make a somewhat different point.
On the handouts, the writing assignments that you get in this course this year will have as their starting point readings that live on the stellar website for the course.
Or they particularly will if I ever give Mara the disk to put them up there.
I'll have to remember that.
The one that's on for this lecture, if one wanted to write based starting from this lecture, is actually a genetic study.
Turns out that there are two flavors of vole out there.
A little rodent-like animal.
Some voles are promiscuous.
Some voles are not promiscuous.
They just hang out with one, I guess lady voles, that's a basketball team, isn't it?
Anyway, the, I guess they're [?
valls, ?]
not voles.
Anyway, the claim of a new paper this year is that manipulation of a single gene is able to turn promiscuous voles into faithful voles.
So in that sense, yes, absolutely genetics are a new tool for getting to this.
It's not what I'm pushing for.
I'll take one more bit of fishing expedition and then I'll just -- all right, that'll be, that's fine, too
[UNINTELLIGIBLE]
In vole land yes?
It's, the male voles who are promiscuous.
And they're manipulating the sexual behavior of the male voles.
The female voles don't have the big asymmetry, the two species of voles don't have the big asymmetry in their behavior to start with, apparently.
So yeah, I think it's a guy thing.
More on that later in the term.
Yes
[UNINTELLIGIBLE]
Reading brain signals.
We'll call that recording.
In some fashion or other.
If you can stick an electrode into the brain and stimulate, you can also stick an electrode into the brain and record from brain tissue.
And ask what a particular cell or bunch of cells are doing.
You can do this at a large scale level in walking around human beings or animals, by putting electrodes on the skull.
You can read massed activities of large numbers of neurons off the surface of the skull.
And that's so-called electoral encephalogram, EEG.
But that's very useful, for instance, in telling the difference between different wake and sleep states.
If you are deeply asleep, the sort of sleep that when the alarm clock goes off you're uh, where am I.
If you look at the EEG, you find big, slow waves.
All the cells are active.
And then all the cells are quiet.
And they're all firing together.
When you're looking at an awake individual, the waveform is much smaller amplitude and higher frequency.
Because the calculus piece of your brain is currently napping.
And the psychology piece is wide awake.
And various bits are doing things out of sync with each other.
So you can see that from massed recordings.
But if you now go stick an electrode.
Let's suppose we go and stick an electrode into, well, let's take the same cell that I stimulate here.
And if the patient said, I feel that in my toe, if instead of stimulating, you record, you find out that that cell doesn't change its behavior at all over a huge range of possible things that the organism could be doing.
But if you poke the toe, that cell says, here I am, I'm interested in that and exactly that.
You go poke the armpit or something, the cell doesn't care.
You poke the other, the next to over, the cell doesn't care.
You poke one particular toe, let us say, that cell will care.
Take a cell out here somewhere, and that cell will care only if you put visual stimuli in a particular little spot in the right side of the visual space, in a very particular part of visual space.
Move over a little bit, and it'll want stimulation in a different piece of visual space.
Move elsewhere in the brain, and now rather than liking, say, just something simple like a line moving around, a cell down here, this is typically done in things like monkeys and so, that cell might be very interested if you show it the hand of a monkey.
The cell's thrilled.
Show it a hunk of chalk, I don't care.
A nearby one might like the face of a monkey.
And so on.
So you can find out what individual cells are interested in, by recording their activity.
There are some significant drawbacks to this.
One of them is that you can only sample from a tiny fraction of the cells, even in an area that you are interested in.
How many cells do you think there are, how many neurons do you think there are, in the brain?
So, the answer's -- yes, a lot is a good answer.
But the answer's going to be expressed in terms of 10 to the X.
What's X, would you guess?
[UNINTELLIGIBLE]
The 24's are a little high and the 9's are a little low.
When I asked this question when I was in graduate school, I got the answer that there are approximately, nobody knows the answer for sure, because who's going to go and count.
That there are 10 to the 12 neurons in the brain.
And of these, I didn't even tell you on this picture, but we can draw a tail on this thing.
Hanging off the back of the brain, sort of underneath the back, is the cerebellum.
It means little brain.
It's important in learning and motor control sorts of things.
We're not going to say much about it here, sadly.
But what my professor said was, there are, 10 to the 12 neurons in the brain.
Of these, 10 to the 13th are in the cerebellum.
The quick on the uptake crowd picked up that there's a problem here.
What he was trying to tell me was not that he didn't understand math.
But that, look, we don't even really know to an order of magnitude how many cells there are in the brain.
But, lots.
On this order.
How many cells can you record from in a study of some function in the brain?
Heroic single-unit, single-cell recording studies might have on the order of 10 to the 2nd, 100 cells recorded from.
Moreover, if you're going to stick an electrode in the brain, the cells that it encounters are going to be biased by things like size.
You're more likely to get big cells than little cells.
And big cells do things that are different than little cells do in the brain.
So, you can imagine that it would be difficult to figure out what your laptop did by recording from single components on the motherboard.
That wouldn't be particularly easy.
It gets significantly worse when you try to do that in the brain as a whole.
The things we know best about in the brain are things where we can bring all of these techniques to bear, and get what would be called converging evidence, converging data, on it.
Evidence from multiple different sources, and then corroborate that by looking at behavioral measures.
Oh, let me say, all right.
This is the moments in the lecture where I really feel my wife looking over my shoulder.
In two minutes I will polish off the rest of the brain.
So that's just the surface of the brain.
The brain is a 3-D structure.
One way to very grossly think about the overall structure of the brain is that if this cortex, lying underneath cortex is what, are a bunch of structures that often get called limbic structures.
This is really an outdated term.
It comes from the word for ring, and it was the notion that there was a circuit of structures lying under the cortex.
But there's a collection of structures who will show up later in the term, with names like the amygdala, from the Latin for almond.
Hippocampus, Latin for almond or Greek for almond, I'm not sure at the moment.
Hippocampus, from the word for seahorse.
Not because you have almonds or seahorses in your brain, but because the shapes of these things are faintly reminiscent of such things out in the real world.
And other structures that turn out to be vitally important in your emotional life and in memory.
Hippocampus will show up later as critical in memory.
Lying underneath there, and heading on down into the spinal cord, oh I don't know, let's call it the core today, or something, are a set of structures that, well, we tend to spend a lot of time in psychology talking about the cool stuff happening up in the cortex and stuff like that.
But the stuff you really don't want to lose, the stuff we don't know about from lesion studies, is down here.
Because something in your brain has to say to your heart, beat.
Beat.
Beat.
That piece of brain goes, you've got really big trouble.
There's another piece down there that says, time to breathe.
Time to breathe again.
I mean, it's not glamorous work.
But, yeah, somebody, so there's a whole set of stuff here running vegetative processes.
And also important, not just for pacemaker kinds of things.
But also for general issues of arousal, and sleep and wakefulness, those sort of things.
So that is an embarrassingly brief tour of, you can spend your life on neuroanatomy.
Trust me, my wife does.
We'll come back and talk about single cells on Tuesday.
OK what I want to do for the first couple of minutes of today's lecture, well, I guess looking at the hand-out what I want to do is play fill- in- the- blank with this great picture of a neuron.
I got to talk about the gross structure of the brain last time, but I didn't get to say anything about a neuron.
Neurons are the, are brain cells.
Not all are brain cells, right?
This is going to be a neuron.
At least half the cells in the brain would be in the category of glia, that's a word that comes from the word for glue.
And all the years I was learning about this stuff, glia kind of got shoved into the category of like support staff, right?
These were the guys who cleaned up after the neurons who were doing all the cool stuff.
But it's becoming increasingly clear that glia have many functions to play in brain function beyond the merely vegetative, and if you go into neuro-science, odds are that you'll end up learning a lot more about glia than I did when I was taking the equivalent of course.
But the main units, if you like computational units for the brain are neurons and so here's a quick tour of a sort of a canonical typical neuron.
A typical neuron would collect information either from other neurons on from sense organs through its dendrites, the word comes from tree-like, because its branching structure.
So all these little branches making connections with lots of other neurons or with sense organs in say the skin or something like that, and the information that the dendrites collect you can think of as being in the form of little pulses of excitation or inhibition.
They can be either positive or negative in nature.
They flow down towards the cell body here.
The neuron is a cell like other cells.
It does all those good cellular sorts of things, but for the purposes, for our purposes, its big job is to act as a sort of a summator and to ask, how excited am I?
Am I, you know, if I sum up all these pluses and minuses, am I excited enough to want to tell other neurons about my level of excitement here?
If the excitement passes a threshold, the neuron sends an action potential, a signal, an action potential -- no, that doesn't look right, does it?
Well, potential, there we go.
This is why there's a handout, right, because I can't spell under good circumstances, and I can't spell online at all.
Anyway, that should say potential.
It sends a signal down the axon, the long wire of the neuron is the axon.
That action potential has a couple of important properties.
Property one is that it's of a single polarity.
There aren't pluses and minuses, there's just a single polarity.
And it's a fixed size.
A more excited neuron does not produce bigger action potential, so how's a neuron going to tell another neuron that it's more excited?
Would you guess?
It's going to say make more action potential.
So its frequency coating it's level of excitement.
But action potentials are fixed size, and fixed sign electro chemical signals powering their way down this axon.
These bumps wrapping around the axon are actually another cell type.
They're a form of glia known as myelin.
They act as, it acts as insulation, keeping the signal in one neuron separate from the signal in others, and it also acts to speed transmission.
There are a number of neurological diseases whose cause is not a problem with the neuron, but a problem with the myelin.
De-myelinating diseases, like multiple sclerosis, for example, have their affect because the insulation is breaking down, not because the nerve cells themselves are breaking down.
That signal reaches the arborization of the axon.
It then goes off and talks to a bunch of other neurons.
Or if this was a motor neuron, it would be talking to muscle.
Typically it does not make a direct electrical contact.
If you look at connections between stuff in your computer, those are direct electrical contacts.
Not the way things work in the nervous system.
In the nervous system, there's a small gap, called a synapse, and to signal from one neuron to another, an electrical signal coming down the axon causes chemicals to be released into the gap.
They diffuse across the gap and bind to receptors on the other side, and that binding either produces a little plus or a little minus, depending on the nature of the chemical, the nature of the receptor, and then the whole process starts again.
So you go from an essentially electrical signal to a chemical signal then back to an essentially electrical signal.
These neuro trans, so called neurotransmitter chemicals are the targets of lots of the drugs that are used to modulate mental states, like antidepressant drugs, for example.
And not just drugs over- the- counter, or not just prescription drugs.
Let's see, what are you drinking there?
Oh that's just tea, how boring.
Well, I wasn't actually thinking she was probably putting away a margarita, but that drug would be working on a synapse.
The caffeine that you are pouring into your brain is working on neurotransmitter and synaptic properties.
That where you get your chance to affect large numbers of neurons at one time.
We'll talk more about that later.
Now if I look at my cute little diagram, yes you should have been able by now to fill in all the -- I wonder what that interesting thing is at the top, actually, the middle one at the top.
What do you figure?
Cell?
It's not axon.
It's not myelin.
Those are the two at the bottom right and center.
The one on the top right is pointing to synapse.
The one at the top left is pointing to dendrites, and the one at the bottom left is pointing to the cell body.
So --
[UNINTELLIGIBLE]
Well, the one on the bottom left, I figure, is pointing to the soma, or cell body.
I think the one at the top middle is pointing to the whole thing
Neuron
Neuron, yeah
[UNINTELLIGIBLE]
Oh maybe, ah, yes, thank you.
Oh, somebody's been doing their reading.
Yes, OK, she, you are correct.
So the top middle is almost undoubtedly pointing to the axon, that whole long thing.
And the bottom, what she's pointing out is that the bottom middle one is pointing to the valleys amongst the bumps, and those are called the Nodes of Ranvier.
Undoubtedly named after, you know, Joe Ranvier, who I don't know anything about.
But the way that myelin speeds up transmission is it makes it possible for the action potential, rather than just going down the axon in a nice smooth fashion, it jumps from node to node.
And so that's why the nodes are interesting.
OK, I'm glad we've solved that.
You would have thought I would have thought about that before putting the thing on the handout.
OK, any other questions about neurons or things, yes --
So, soma, is that boring cell stuff happening all over the place?
I mean, well, there's boring cell stuff happening all over the place.
We shouldn't, of course call it boring cell stuff, because then guys in like, course seven come in and, well, you know, put a hit on us and things.
But yeah, neurons are cells like other cells and they have to do all the, you know, respiratory things that other cells have to do.
But, you know, our interest in them, of course, is as little communication and computational units.
Yes?
What is the myelin, exactly?
It's another cell type that wraps around the axon and acts as insulation and speed bumps, I guess.
You think of is these as the speed bumps, except they speed things up rather than slowing things down.
Boy I bet that just confused matters.
OK that said, what I want to today is to talk about why we do anything.
It's the general topic of motivation.
But, you might, well, you might ask a question like, why do you eat?
And you say, well, you got to eat because you need -- if you don't eat, you die
Prove it
Prove it, right.
That's not a stupid point.
He meant it as a stupid point, but it's not a stupid point.
Does a rat -- why does a rat eat?
Does it know it's going to die?
Is a rat sort of sitting there, you know, with existential concerns about it's -- I don't think so, particularly.
And even if it were the case, now, if you don't you you're going to die.
Well, so what?
YOu've got to answer -- I mean is it why you do anything is a question that requires some sort of an answer.
And I will give a three- part answer.
Part one of the answer will be that you are a slave to, you do things as a slave to the environment.
Part two will be that you do things, as a slave, in fact, to your own brain, and part three will be to say well, parts one and two seemed a little simple- minded, we'd better try and get to a somewhat richer explanation that might explain why you're sitting here in a classroom doing this rather complicated activity, since what we'll be talking about is -- oh, one of the things will be talking about is the reason that -- is the obvious fact that I'm a psych professor and not an artist.
So you will have to bear with the fact that, for present purposes, that is a cat, and that is a rat.
You are neither, are the third part of the lecture will be devoted to trying to talk up about the richer set of motivations that you might have, and that even animals might have.
Well, a good starting place, even if it's a bad bit of art, would be -- this is an unusually pathetic cat, even for me.
Back in the -- shortly before the turn of the last century, Thorndike, Edward Thorndike up at Harvard was doing experiments of the following form: he would take cats, who were hungry cats, and he would throw them in a box that looked sort of like a packing crate kind of thing, you know, slats all the place, and outside of the box is fish, well, bit of fish, but if I drew a bit of fish, you'd never know what it was.
This at least looks sort of like a fish.
So there's a bit of fish.
You got a hungry cat.
You got the fish.
Cat wants fish.
What's the cat going to do?
Well, the cat is of bouncing off the walls being annoyed and agitated and the way this box is set up is, well, they're called puzzle boxes, because in effect, er there's a trick to getting out.
It might be that there's a lever back here that's attached to some string and pulley arrangement that lifts a gate over here, and if the cat happens to bounce into this lever, the gate goes up, the cat goes out, gets the fish.
Great, well it's a little piece of fish, and he's a hungry cat.
And what happens to the cat immediately thereafter, is Thorndike grabs the cat again, throws him back in the box.
What's the cat do?
Well, whatever the cat is doing, what Thorndike is doing is measuring the amount of time that it takes for the cat to get out of the box.
As a function of the number of times that the cat has been chucked back in the box.
So you've got time as a function of -- we can call them trials, the number of times the cat has been in the box.
It initially takes the cat some amount of time.
The next time maybe it's a little quicker, and you got some sort of a function that eventually gets to some sort of asymptote where you chuck the cat in the box.
The cat looks at you in a disgusted kind of way.
Goes over to level, says "plahtttt", eats the fish, says, OK, we gotta do this again, fine.
This is a learning curve.
In fact, this is the original use of the term learning curve, when you talk about going up the learning curve and stuff like that, you are invoking Thorndike.
What is the cat learning?
In the 19th century, and even today if you go to, you know, the animal section of Borders or something like that, there are lots of books that will tell you about the, you know, the rich mental life of your cat, and so on.
Thorndike wasn't buying it.
What Thorndike said was that there are good things in the world.
We'll call those reinforcers.
They come in two forms.
There are positive reinforcers.
Well here, as an example, if you tell a joke and people laugh, that's a positive reinforcer.
you did something.
Something good happened.
There are also negative reinforcers.
Those are also good things.
The jargon IS unfortunate, but a negative reinforcer is a good thing.
Focus on the reinforcement part not the negative part.
A negative reinforcer is the car alarm is going off, right?
You do something that makes that noise stop.
Right, that's a good thing.
The sensation of something unpleasant, like hunger, for example, is a negative reinforcer.
It's also a good thing.
A bad thing in this jargon is punishment, is called punishment, right?
You tell a joke.
The person you're telling the joke to take offense and smacks you.
That's punishment.
What Thorndike said, what Thorndike formulated, which is undoubtedly on the handout somewhere, is th law of effect.
Yes, top of the second page.
He proposed what he called the law of effect.
And he said that something that's followed by a reinforcer, a behavior that's followed by a reinforcer, is more likely to recur.
Take the job example.
If you tell a joke and somebody, and people laugh, the next time the opportunity presents itself, you are more likely to tell that joke.
The negative law of effect is or is not on the handout somewhere.
Well, yes, there it is, a little further down under where it says punishment, the negative law of effect says that you do something, and something bad happens, you're less likely to do it again.
You tell a joke.
Somebody smacks you, the next time the opportunity presents itself, unless you're a teenage male, odds are that you won't tell that joke again.
Well, it is a separate problem.
This is the burping problem, right?
When you were like 12 maybe, you discovered you could burp the alphabet or something, and your twelve- year- old guy friends thought it was really great.
So the behavior got reinforced, and unfortunately it takes a lot of smacks before you unlearn that, but that's actually the next lecture.
We'll talk about that next week.
I'm looking at the handout and realizing that I failed to give you the great bunny example.
And you're going to look at your notes later and wonder what this is.
But I can use that to illustrate this same point.
I have a rabbit.
Anybody want a rabbit?
We have a rabbit.
It's an accident.
You can have him.
But anyway, this rabbit, when I go down into the kitchen in the morning, goes berserk.
He's thumping up and down in his cage.
Why is he?
Well, what's he doing?
Well basically, he's signaling that it's time for breakfast.
Not just any breakfast.
If you give him rabbit chow, he keeps thumping.
He's addicted to Reese's pieces, not Reese's pieces, the Reese's cereal.
It's very sad.
He's a sugar cereal junkie.
And in these terms, what's happened, the way, where his behavior is coming from, what's driving his behavior, is he was making some sort of arbitrary noise at some point when we fed him, and perhaps when we fed him Reese's cereal the first time.
It was really good.
It was a positive reinforcer.
It made more likely the behavior that he had just been doing.
So the next time the opportunity presented himself you went thump, thump, thump, being the right time of day anyway he got more Reese's stuff.
And now, by this law of effect kind of training, he has learned that if he goes thump, thump, thump, we'll remember that it's time for the sweet cereal.
And he gets very disappointed when it turns out that, you know, that the sweet cereal supply has run out and we're trying to feed him like some health food stuff.
He's not interested.
Carrots are OK, but only barely.
So why is it that animals -- well, how many of you have cats and dogs?
Cats or dogs, one or the other, you don't have to have both.
Almost all of you can probably tell me something about the brilliant, intelligent behavior of this dog that doesn't sound, or cat, that doesn't sound like this sort of accidental learning of an association between a behavior and it's consequences.
This sort of association learning, by the way, is what I'll talk about, the rules of it I'll talk about extensively next week.
But why is it that we're so convinced that animals have a much richer intellectual life?
Well, one possibility is that they do have a richer intellectual life than this.
The other, the one that Thorndike thought he was seeing, was what really boils down to selective reporting.
Did I put the whole quote on the handout?
No.
What Thorndike said is dogs get lost hundreds of times, and nobody ever notices it or sends an account to a scientific magazine, but let one dog find his way from Brooklyn to Yonkers, and that fact immediately becomes a circulating anecdote.
Thousands of cats, he says, sit on thousands of occasions helplessly yowling, and nobody ever writes to his friend, the professor, about it.
But let one cat claw at the knob, seemingly in, supposedly as a signal to be let out, and straight away, that cat becomes the representative of the cat mind.
What Thorndike thought was, that you are, well at least the cat, and by extension, you are, a slave to your environment.
The contingencies in your environment shape what you do.
If the environment rewards you for pushing this lever, you'll push this lever.
And the cleverness, and the content of that, the seeming content of that, is imposed from the outside.
It's not there.
It's not necessary.
You can explain it all, you can explain why you do things purely in terms of these, of this law of effect.
Suitably elaborated.
And as I say, the suitable elaboration on that will come, will come next week.
The elaboration, by the way, is known as behaviorism and the school of scientific, of psychological thought, that says that the explanation for your behavior can be found in the contingencies that relate your actions to their consequences in the world and that relates, the relationships between stimulae in the world.
That gets you some distance, but what is it that actually makes something reinforcing?
Why should something be reinforcing at all?
What is it about say food that, you know, why does the cat care that he gets out and satisfies his hunger?
For that, you want to switch to the second form of slavery, which is this notion that you are slave to your own brain in a sense.
And the emblematic experiment for that is done some 50 years after Thorndike is working with his puzzle box.
And that comes from an experiment done by [?
Olton ?]
[?
Milner ?].
It actually started as a mistake done by [?
Olton ?]
[?
Milner ?].
Here's what they were doing.
They were sticking electrodes it's into the brain of rats, into the brains of rats, down into the brain stem, because what they wanted to do was to study arousal.
Centers down in the brain stem, I mentioned last time, are intimately involved in things like sleep and wake, so a general level of arousal.
How do you know if a rat is aroused or not?
Well, one of the ways to measure arousal in a rat is just to look at the amount of activity that that rat is emitting.
A rat that's just sort of sitting there is not very aroused, and a rat that's running around is presumably aroused.
Great, so they're going to stick an electrode into a rat's brain and look at general levels of activity.
There's a problem that requires a control experiment first.
What other than sort of generalized arousal might get a rat to run around, would you think?
[UNINTELLIGIBLE]
Sticking an electrode into their brain.
Yeah, well, why might sticking an elecrode in their brain make them run around?
Pain
Pain, it might hurt, right?
You know, so suppose all that happens when you turn on the electricity, you know, to stimulate the brain of the rat is it hurts.
The rat is going to run around, but you wouldn't want to argue that this is a center for generalized arousal.
Plus, you might not really want to be, you know, hurting your poor rat.
So anyway, here's the experiment that they did.
What they did was they watched the electrodes in the rat, and they're looking at the rat, and they only turn the electricity on when the rat is over in this part of the cage.
So, turn on the electricity.
If it hurts, what does the rat do?
Runs around
Runs around, but
[UNINTELLIGIBLE]
To the other, but hey, ooh, it stopped.
Now wander back over here, oh, no, OK, right?
Everybody knows rats learn this.
They knew that nobody had done this with stimulation of the brain.
This is so-called aversive learning.
One of the typical ways of doing it is you electrify the floor of the cage, right?
So, the rat goes over here, discovers that when he puts his little rat toes on the cage floor over here, there's electric current running through it and phew, he's over here.
OK, so that's what they did, and so the rat goes over here, and on comes the juice for a little bit, and the rat doesn't go anywhere, and then the juice goes off, and the rat is wandering around and then the rat comes back over here, juice comes on for a little bit.
Rat doesn't go anywhere.
In fact, the rat comes back over here.
The rat is sort of hanging around over here, yeah, let's do that again.
Turns out that the rat was behaving as though the stimulation was reinforcing.
We could say as though he wanted to be stimulated, but it's a little dicey to talk about what a rat wants or doesn't want.
So you can use this more neutral language of reinforcement.
The rat was behaving as though stimulation of the brain was reinforcing.
This was a surprise to [?
Olton ?]
[?
Milner.
?]
The next surprise came when they actually looked at where the electrode had gone in this particular rat's brain.
That's the mistake part.
It turns out the electrode had bent, and instead of being where they wanted it in the brain stem, it was in the chunk of brain known as the lateral hypothalamus.
At least we think it was in the lateral hypothalamus.
Don't actually know, because not only did the electrode bend, but then they lost the brain.
Subsequently, they did very well for all the mistakes they made here.
Hypothalamus.
So, what they did that was to deliberately stick electrodes into the hypothalamus and to set the experiment up in a more formal fashion, beautifully illustrated here by my rat, who's got an electrode in his brain, and that's a battery up there.
And that's a lever.
Now the rat can go off and self- stimulate.
He can go and choose, if he pushes on this lever, he gets a little pulse of electricity to his brain.
And what does he do?
Indeed, rats with electrodes implanted in the right bits of the lateral hypothalamus will sit there and work for brain stimulation, the way another rat might work for rat pellets water if it's thirsty or something like that.
And they'll work hard.
The most dramatic version I can recall reading is a rat that will work bar pressing 7,000 times an hour, I think, for to keep the brain stimulated there.
So this is apparently good stuff.
Not everything there is good stuff.
There are chunks of brain where the rat will work actively to turn off the stimulation.
So again, so turning on the stimulation would be a positive reinforcer.
Turning off an aversive stimulation, whether it's electricity to your feet or electricity to the wrong part of your brain, that's a negative reinforcer.
So you know, there's current.
If you push the button the current stops.
That's good from the rat's point of view.
In various surveys of rat brain, it turns out there are more positive locations than negative locations.
I don't know.
You know, we don't know if that's true of humans, and we don't know if that means anything -- actually the study, the study that I noticed this morning that I was thinking of here says that they're 60 percent, in this one study, 60 percent neutral locations, 30 percent rewarding locations, and 5 percent aversive locations.
Now the mathematically sophisticated among you will notice that adds up to 95 percent.
And neutral rewarding and aversive would kind of seem to cover all the possibilities here, so I have no idea what the other 5 percent of neurons we're doing.
That seems a lot for rounding error or something.
Anyway more pleasant locations then unpleasant locations.
Now, around this time, somebody who is a science fiction buff should say, I've heard this before, it shows up in one of several classic science fiction stories.
The world is full of science fiction stories written by people just like you, which is to say people who heard an introductory psych lecture and thought I can make a story out of this.
Memento, is perhaps the most recent movie example of that.
We'll get to that lecture later on in the term.
Anybody familiar with the sci fi story that's related to the rats here?
Any of the various sci fi stories?
Mrs. Frisbee?
Is
Oh, no.
Mrs. Frisbee and the rats of NIMH.
What's -- no, apart from the fact that NIMH is short for the National Institute of Mental Health and they don't much care for that.
But what are the rats of NIMH doing?
Oh no, they're busy taking drugs and getting smart, right?
Na, Na.
they're not sitting there stimulating their little selves, yeah
Terminal Man?
Yeah, Terminal Man, that sounds right.
What's Terminal Man's issue?
He has epileptic fits, [UNINTELLIGIBLE]
And it does give him pleasure, yes.
OK that's -- yeah
Flowers for Algernon?
Flowers for Algernon?
No, what's he doing?
Is he being stimulated?
[UNINTELLIGIBLE]
Oh yeah, but he just becomes a super smart rat like the rats of NIMH one.
He's got a different -- we don't know how to make super smart rats, we just know how to make rats who will work hard on pushing levers.
The Terminal Man one is one of the sci fi examples that I know about.
Anyway, look, the critical issue that gets people writing science fiction stories is, suppose we did this to a human, would that human, could you killed somebody by hooking them up to self- stimulate their pleasure center, and they would kill themselves.
How would they kill themselves?
By, you know, sitting there, oh, oh, oh, and forgetting to eat, right?
Well nobody's ever done this in humans of course.
It's been done in rats.
And if you put a lever a lever that gives food next to a lever the gives stimulation in a good spot in the rat's brain, the rat will sit there and work away on the self- stimulation lever.
And at least in some of these studies, well, every now and then, sort of, say oh, get a little food, OK, back over here to the video game stuff.
This is really good.
This is what your parents thought you were doing, right?
Yeah, come on down and eat dinner.
Oh, Mom, I can't put it on pause, yeah.
Anyway, the -- would an animal actually kill itself?
Gleitman says that hungry rats will opt for self- stimulation even though it literally brings starvation.
And then he sites an article that I went back and read, and the rats in that study didn't starve themselves to death, because the researchers wouldn't do it, presumably because it seemed unethical to actually let a rat starve itself to death.
But they will certainly reduce, you know, they'll certainly get themselves pretty hungry while amusing themselves with the, by stimulating their brain.
There was a hand up, oh there still is a hand up there, look at that
[UNINTELLIGIBLE]
I don't know that the truth or falsity -- so the story is a guy played video games for, what was it, 74 straight hours, 74 straight hours in Korea, and then he died.
Actually the Korea part is good here, because I have no idea whether this is true one way or the other.
It has all the Hallmarks of a so- called urban legend.
It has a grain of plausability, and is just sort of one step removed from checkability.
It's like the Kentucky Fried breaded rat, right?
Everybody knows somebody else who knows that their cousin got a breaded rat once.
There are whole books of these things, they're great.
I don't know, maybe somebody managed to play video games until they died.
People do all sorts of stupid things.
I don't recommend trying this yourself.
In any case, rats will work very hard in order to stimulate different parts of their brain.
What is it that they're working for?
The evidence suggests that they're not just working for some sort of disembodied pleasure, but that they're working for the reward associated with one or the other of sort of basic hard- wired, innate reward systems, that you come with, but that the rat comes with too.
The reason that in this aspect of the slavery story, that you'll work to get the fish if you're the cat is that you came predisposed to think that hunger is bad and food is good.
This, yeah, look, it doesn't take very sophisticated evolutionary psychological theory to think this might be a good idea.
Organisms that went into the world thinking that hunger is good and food is bad had many fewer progeny than the ones who thought that hunger was bad and food was good.
So it's not that surprising that you've got circuitry that says if you're hungry you want to go and eat something.
And the centers that are being stimulated are in chunks of brain that if you mess with them, mess with the rat's ability to regulate its eating.
So there's a wonderful picture in Gleitman of a rat with a hypothalamic lesion, a lesions in this sort of chunk of brain.
The specific lesion in this particular rat is in a chunk of brains that seems to be important in telling you when you're full.
So this rat never knows that he's full.
With the result, he just keeps eating.
And so there's this picture in chapter three of a rat on a -- some of you may already have seen it, of a rat on one of those diets scales.
You know, and he looks like a fury football.
He's just sort of overflowing.
that's And that's because he's got a specific lesion in this little specific chunk of brain.
How do we know that's what's going on here?
Well, you got a rat who's working a way to be stimulated in this little particular chunk of brain.
And if he, so let's take this little chunk of brain, if he's a full rat, if you feed him up, he doesn't work as hard.
It's just not as interesting.
You know this, in your own -- so, you like M&Ms let's say.
You're willing to put, you know, you're willing to push the lever to get the M&Ms, right?
You're willing to put the quarters in the machine.
Would you do that forever?
No, because eventually you get full and the M&Ms kind of look disgusting.
So the same thing happens here.
So the pleasure that the rat is getting is modulated by satisfying, you know, the thing about direct electrical stimulation is it gives the sensation of the satisfaction without actually satisfying anything.
So, if you're getting the oh, the M&M is good thing without getting the M&M, you can sit there, oh the M&M is good forever and ever, and keep working on it.
Okay, so that's this little chunk of brain.
You go over to this other chunk of brain where the rat will also work hard, and that rat doesn't care if he's full, that rat cares whether he's mated recently.
So you know, this one is part, you know, this little chunk of brain is presumably related to the reward systems having to do with sex and mating.
Again, it doesn't take an evolutionary psych genius to figure out that this would be sensible stuff for a brain to come with.
Organisms that didn't find mating rewarding didn't leave many offspring.
Yeah, just didn't work out that well for them.
And so, you know, there's another reward system.
Similarly, for thirst there are separate reward systems.
OK, so, in some sense you end up doing stuff in order to light up little pieces of brain or to turn off little pieces of brain.
That's the sense in which you are a slave to your brain.
Now, you, know that only gets you so far, because that's fairly simple.
You know, eat food, little hunk of brain says you're happy.
Your behavior is somewhat more complicated than that.
How do we get to that more complicated behavior?
Well, let's stick with the M&M example.
How did you get to the M&Ms?
Well, you got to go down to the vending machine and put the money in.
And how do you learn to do that?
Well, maybe you learn to do that by a chain of associations back to the M&Ms.
M&Ms, oh, they're good.
Getting M&Ms becomes good in its own right.
Somebody gives me M&Ms, that becomes rewarding by association, with the fact that I will now get to eat the M&Ms.
This machine will give me M&Ms, but only if I give it quarters.
So, I learn to give it quarters.
That's an interesting ring.
It sounded like a regular old phone.
Put them on vibrate.
That's what I do, because let's see, well, yeah, it's just about the right time.
I should be vibrating any time, woops, not that one.
My tenth grader tends to phone in after school which is great, the only thing is that after school two days a week turns out to be here, and it's not a good time to take a call.
Anyway so ah well, so the tenth grader, he likes M&Ms.
He needs quarters.
So, he's got to emit behavior that I like so I'll give him quarters right?
And so you can see how you could build back from a primary sort of reinforcer, like food is good to something much more remote like cleaning my room is good.
Why is cleaning my room -- if I clean my room, daddy gives me quarters.
If daddy give me quarters, me I take quarters.
I put them in machine.
Machine gives me M&Ms and all is good.
That's OK, but it probably doesn't do quite well.
The brain suggests to us that that isn't quite enough, for organizing even the feeding behavior of a complex organism like us.
Suppose that you are doing -- that what gets you to do your complicated sort of behavior around feeding is the fact that you're hungry -- oh, chapter three will tell you all about how you know that you're hungry.
It's lovely sort of engineering work.
You've got little detectors, receptor is in the blood.
They're doing things like keeping track of blood sugar levels.
When those levels drop, chunks of brain become active.
Activity there is apparently aversive, probably corresponding to what you and I think is hunger.
When you get sugar in, activity there drops.
Lovely, beautiful sort of control systems.
Stuff worked out in great detail and discussed in considerabe detail in chapter three, but suppose you're not living in your food, right?
Suppose you're, well suppose you're like out on the plains of Africa somewhere, and the M&Ms aren't just sitting over there in the vending machine.
They're running around, and you're going to have to hunt them down, right?
If you are sitting around until the blood sugar level gets low enough for unpleasant things to happen in these hard-wired chunks of brain, to make you think, OK, all right, I'll remember this, M&Ms or wildebeest s, whichever it was, they're good.
I can eat them.
To get to wildebeest , I got to go clean my room -- anyway, there's some complicated thing that you gotta do.
But if you wait long enough before starting, you may not get it, the job done in time, right?
It's going to take you a while to hunt down the wildebeest .
If you wait till the gas tank is almost empty, yeah you may find yourself flat out on the plains with, you know, no food.
So it would be useful if something got you up and mobilized for activity before you were actually hungry, before you actually were in some sort of desperate need.
And the distinction that gets made, and I trust is bolded at the, yes, it's down there at the bottom of page 2, is the distinction between consummatory and appetitive behaviors, or consummatory and appetitive pet motivations.
consummatory ones, are the ones we've been talking about so far, eating M&Ms, that's good.
Appetitive ones are ones where the act of looking, if you'd like, or getting yourself in a position to consume is itself rewarding.
In that case of food, that might be sort of the thrill of the hunt kind of thing.
Think about cat, right, your cat that likes to jump on anything that moves.
Why is that?
It's because the cat is hard wired to like hunting stuff.
It's just fun to grab stuff and kill it.
Now that's a useful thing to do if you are a cat-like animal because then there's going to be something around to eat when those blood sugar levels drop and you need to, you know, chew on a wildebeest for a while.
And it's pretty clear that we have similar, actually, it's pretty clear that in yesterday's political news that we have similar sorts of circuitry in ourselves.
Yesterday the assault rifle, the assault weapon ban expired.
Bush spent a lot of time not talking about it.
Kerry spent a lot of times saying I'm a hunter.
I love to hunt.
I love to jump on moving things and stuff like that, but I've never done it with an assault weapon.
You don't need to use an -- he was on the -- but he was very big on pointing out that he loves to hunt, because apparently there's lots of people who love to hunt and go out and you know, shoot things and sometimes eat things.
This is presumably related to, this is presumably a form of appetitive behavior in humans.
If you don't happen to love to hunt it doesn't mean that you're missing a piece of your brain.
Another realm where appetitive behavior is important is in the sex and romance department, right?
If it were not the case that the act of looking up for a mate is motivating in its own right, well, what are you going to do, you know, wait around until all of a sudden, I don't know, whatever levels it is drops low enough, ugh, it's time to mate?
It's just not the way we organize our, it's not the way we organize our behavior.
Our social lives suggest that we find a degree of pleasure in the appetitive aspects of seeking a mate, not merely in the consummatory parts of it.
So you come with a set of motivations that get you motivation, in the sense of getting you to move, that seem to come with the systems.
These can be quite specific -- by the way, that's the notion of preparedness that I see, what is related to this notion of preparedness that I see is actually above where it says consummatory and appetitive behaviors on the handout.
you are surprisingly fine tuned by, presumably by the history of the species.
So for instance, you are built to get away from scary things that are going to hurt you and that seems reasonable enough.
And that can be specific enough to the point that it's easier to get you to be scared of spiders and snakes than it is to get you to be scared of bunnies and chickens.
That sounds kind of obvious, but there's no obvious reason why, in fact, one of the famous experiments in the sort of behaviorists tradition has a little baby known to history as Little Albert -- there's a great film from the 20's of this Little Albert is sitting there, he's playing with a little white bunny.
John Watson, founder of American behaviors, then sneaks up behind Little Albert with a gong and pow, rings the gong.
Little Albert goes, if it was a cartoon, would go straight up in the air, and is you know, scared out of his Little Albert gourd and is screaming and crying and stuff like that, and you know, it's kind of harsh for Little Albert, but then you bring Albert back and show him the bunny again.
Does he want to play with the bunny?
No.
Does he want to play with anything that's white and furry, including Santa Claus, apparently, I seem to recall.
Guys with bushy white beards -- he recovered by the way.
He didn't grow up to be a psycho or anything, but you can train, so you can you can train somebody to be scared of rabbits, but it's easier to train them to be scared of snakes, at if they were prepared by the history of the species to be scared of things like snakes and spiders.
OK, so you're a slave to the environment because your brain responds in rather specific ways to the environment but you are a lot more complicated than that, or at least you feel like you're a lot more complicated than something that's just a bundle of sort of engineering responses to glucose level low, must go find wildebeest.
And I want to spend the remainder of the lecture talking about some of those complexities, but first I want to give everybody a chance at least to stand up and stretch a little bit and fan themselves because it's warm in here.
I Oh, yes, yes you want your little form right now?
Sure
Ok send this form back, and I trust that means that you can be here at like five of two --
[UNINTELLIGIBLE]
Yes, well, it's not useful after class.
OK this is all the information I have.
If there's a problem let me know.
And your name is?
Jean
Jean, nice to meet you, and I trust that you'll see happily erasing my boards.
OK, let us reconvene here.
Here's the deal.
I will typically try to be good and remember to give you a chance to sort of stretch your limbs a little bit, and, but it'll be a short stretch.
So, you know, but why am I telling you this?
It's the guys who thought they were off for ten minutes who are now halfway down the hall to get the M&Ms who had the problem, but anyway.
Or the ones who are out looking for wildebeest are really in trouble.
You might wonder what the role of the emotion is in all of this.
Clearly, motivation and emotion are closely related.
They both talk about feeling a lot, you know, that there's this feeling of hunger, but the emotion part is well, one way to think of it would be as a sort of a language, a symbol system, that's a short hand that helps you mobilize you, a way of talking to yourself about what resources you might want to mobilize.
Oh, I feel fear.
That means I should do the following sorts of things.
And it helps you generalize from event to event.
I feel fear now.
I felt fear before.
You know, maybe there's some sort of relationship that I ought to be noticing.
And perhaps, one of its most important uses is as a form of communication between people or between animals more generally.
If I am afraid, let's go let's go back to the plains of Africa and switch sides.
If I'm a wildebeest, and I see the leopard over there, stalking through the under brush, I should be afraid.
If you're not seeing the leopard, but you're seeing me, I look afraid, and you're the rest of the wildebeests, you should be afraid too.
It's a good idea if you pick up on the way that I feel.
Now we tend to devalue that a bit because we tend to use verbal communication a lot to convey information back and forth, and for some things it's obviously a much more useful technique.
I mean trying to teach psychology by a set of emotional expressions would not -- I can't even conceive of how I would go about doing that.
However, we probably over value the straight, this sort of straight verbal nature of communications, even between humans.
Obviously, non-verbal animals are going to have to use some other form of communication, but even among humans, we underrate the value of this current of emotion information transmission.
Perhaps the clearest example of this is email, where, how many of you at some point have gotten into trouble on email, because email has the characteristics of conversation without the tone of voice.
You said something that you thought, that you meant sarcastically, or you thought was funny, and somebody else read it as flat prose, and they were not amused at all.
And then you got -- actually that's the root of all these little smiley faces and all these symbols that I don't know from nothing.
You all know this.
There's some very elaborate vocabulary of basically emotional tokens for instant messaging or for email because it lacks that emotional tone of voice, and without that, the quality of the communication goes down.
So you've got emotions at sort of a super ordinate language that allows you to talk about motivational states, even in the absence of any sort of verbal language.
I think you could probably make a convincing argument that basic motivational states exist in very, very simple organisms.
I mean any organism has to figure out, has to have in some sense the motivation to feed itself and to reproduce.
It's only when you get to more elaborate organisms do you get something that looks like the sort of emotional life that you and I might have, where it's being used, as a sense, a form of communication.
But look, so far, we're still in pretty simple terms here.
You know, we've got, sex and reproduction is good.
Getting eaten by the leopard, that's bad, and so on.
Where we want to get, and where the chapter on motivation, for instance, never gets, is you know, sex and reproduction, good, eternal damnation, bad.
Or something like that.
Or how do you get from, you know, hunger drives and sex drives to thou shall not murder and you know, I want to grow up to be a chemical engineer, or something like that.
It's those rather more complicated motivations that require more than just feedback loops and the lateral hypothalamus to make a good story.
Now, one step to a richer account of why we do anything is to broaden the set of motivations.
I have pointed already in the direction of one act of broadening that set, and that is this notion that if I'm a frightened wildebeest, you should be a frightened wildebeest.
This is the notion that emotions are catching across individuals.
We talk about, on the handout I put it as empathy and described it as a social emotion, and it's a social emotion in the sense that it helps to organize social behavior, interactions between people.
So, for example, well let's start with a bit of evidence that this comes with the system, that it's not something, I mean you spend a fair amount of time trying to teach people to be more empathic, but the core of it is there from the start.
So, how do you know it's there from the start?
You take a baby, here's a baby, here's a bunch of other babies.
Poke this baby.
What's this baby do?
Cry
Cry.
What do these babies do?
They also cry.
Why?
You haven't hurt those babies, but those babies have, in a sense, caught the emotional state of the other baby.
Not only do babies catch their emotional, each other's emotional states, but you are all here today because of the same sort of empathic reaction.
When you were a little pre-verbal slug, and you were hungry, what did you do?
Cried
Yeah, right, you cried, right, is all you could do.
And how did that make your parents feel?
Well, some of you are here today because you were lucky because apparently it made your parents feel angry, and then -- which it probably did, particularly at weird hours -- Mara is looking at me.
How old is the kid?
Four-and-a-half months
Four-and-a-half months.
We can check out -- if you haven't met Mara, she's the head TA, so she knows everything about the course and what recitation you're in, and we can also use her all-term as the official parent person, or at least parent of young child type person.
She will confirm that when you're baby cries, you want to make that baby stop crying.
Fortunately, most parents, the ones who want their genes to make it into the next m make the baby stop crying by doing things like feeding it and changing it and stuff like that.
If it didn't make you feel bad, if the crying baby wasn't the car alarm of the developmental world.
Hey, the baby is crying.
Oh, that's cool, you know?
Didn't know the baby could make that noise.
Wonder what happens in a little -- the baby cries, you feel bad, and you doing something about it.
And, you know, that's how you end up getting there, getting to the point where you can take care of yourself, because somebody else was willing to take care of you at that point.
And obviously, again, in non-verbal animals, like the rest of them, it's really useful if, when your infant animal is feeling uncomfortable, you recognize the need in some fashion and you and you do something about it.
So in a sense, emphathy is sort of a prerequisite for successful parenting.
And this empathic catching of fear is a useful way to keep yourself alive.
If you don't see the danger, but somebody else does, that's -- oh, and it's a good, it's good for keeping you alive in other ways too.
I won't do it, but if I were to suddenly decide to make myself throw-up, a bunch of you would feel really lousy, right?
You know this.
By the way, this is one that goes away when you become a parent.
You can modulate these things over -- if you felt incapacitatedly nauseous every time your child threw-up, your child would never make it, because -- you just got to deal and clean up.
But at this point, if I throw up, you're going to feel sick.
Some of you will throw up.
I shouldn't keep talking about it because some of you will start feeling gross just thinking about it.
That's also useful.
Why?
Because if I ate the wildebeest, and it was a bad wildebeest, you know, and we're all vultures on the same wildebeest here, if I'm looking kind of sick, you might as well get rid of wildebeest now, because it's just not going to be doing you any good.
So this catching businesses has its utility.
Now it's not uniformly a good thing, because you can catch anti- social emotions just the way you can catch pro- social emotions.
So when you see seemingly incomprehensible events, like how in the world can you explain something like gang rape or a massacre in a war zone?
Part of what's going on there is the same sort of emotional contagion.
If one person is in a murderous rage, people around them, around that person, can catch the murderous rage, just the way they can catch the fear or catch the nausea.
It's not just sort of a pro social motivation, but empathy does serve the function of organizing larger scale sorts of behavior.
Oh, now, how does this contagion actually work?
How does emotional contagion actually work?
Interestingly, part of, the sort of reflexive mechanism of it seems to be that when you read my emotion, you also mimic my emotion.
It is most clearly seen with facial expression.
If I smile at you, you will smile.
You may not smile much, but I'd be able to pick up, if I was recording from your facial muscles, I would be able to pick up a transient smile flitting across your face.
Why is that important?
Well how do you know if you are happy?
Well, it turns out -- and this is, one of the senses in which feelings, emotions our feelings is that you go out and you feel yourself and ask yourself, how do I feel?
Part of that is to feel your face, in effect.
You know, how does my face feel?
Well, I seem to be smiling.
Why would I be smiling?
I must be smiling because I feel good because only dumb people smile when they're miserable, right.
You know, well, there are various social circumstances when you might do that, but anyway, it's useful information.
You can actually try to demonstrate this -- here's an exercise for the reader if you like, I've done it as one of the writing assignments for the course in various years.
It's sort of fun.
Take say ten events of about the same duration, say an hour, because that works for a lecture, for example, now a lecture, recitation, dinner, a few events, study break, events that are going to take about an hour.
Take ten of them.
Shuffle them around.
Put five of them in one pile randomly, put five in the other pile.
Then deliberately either try to smile or frown through, you know, smile through pile A and frown through pile B.
And ask yourself, you can ask yourself a variety of questions, the most obvious being, how do I feel, and what's the effect on people around me?
If you frown like an idiot, the effect may be to make people smile, but if you, but you know what you, you've done the experiment.
You were doing it like when you were early adolescences is one of the best times to do this particular experiment.
You show up at the dinner table.
Everybody's in a pretty good mood, but you're not.
By the end of the meal, you know that nobody's in a good mood anymore, right?
Well, you can still do this.
I mean I'm not sure how ethical it is as an experiment to try out on your friends at dinner tonight, but it you go to dinner feeling like junk and acting that way, you can make everybody else miserable too, and isn't that great?
But more to the point, for the present point, you can make your -- all that's sappy garbage about a smile is your umbrella turn out to be sort of true.
And it doesn't even have to be a real smile.
If you take, no that's somebody else's pen.
I'm not going to do it with somebody else's pen.
If you take a pen, this is an experiment that was published a few years ago, take a pen and hold it like this, that forces your face into sort of an idiot smile, whereas holding a pen like this does not.
It forces, it actually forces into a bit of a frown.
If you do this, and don't -- I'm already spoiling the control aspect of the experiment.
You don't tell people why they're doing this.
You just make them do it for awhile.
The people who do this feel better than the people who do this, because the chunks of your brain that are saying, how do I feel, are rolled up and encapsulated enough that they don't know, he's smiling, he must feel good.
The fact that he's smiling because he's got a pen in his mouth turns out -- not that it matters, but it doesn't matter that much.
So if you're feeling down, bit on a pen, you'll feel much better about it.
OK, well look, so you can get a certain amount of work done by broadening the set of available emotions, available motivations that you want to talk about.
Why isn't the world a better place?
You've got this empathy thing going on.
You've got this beautifully wired up brain that you're a slave to that's designed to make you eat and mate and do all those sort of good things.
Why isn't the world a better place?
Why doesn't it work better?
I point, look, I can't solve that in the remaining 12 minutes or so.
I pointed to part of the answer just because you catch emotions doesn't necessarily mean you catch good ones.
It is also the case that not all of your motivations may be entirely benign.
So, for example, what's it got on the handout?
It doesn't have on the handout, but aggression seems to be, there seems to have deep biological roots to it.
It is a not unreasonable reaction to a world of limited resources.
It's not necessarily a nice reaction to a world of limited resources, but, you know, if you are, if there's a limited number of mates out there, and you can punch out all the other guys and grab all the women, well, then you get the have more progney than the guys you punched out.
Fortunately human society is not organized particularly that way.
But there are plenty of animal societies that seem to be, elephant seals, you've seen these guys on, you know, your favorite cable channel, right?
They come up in thousands on the beaches near Antarctica and stuff, and, you know, these huge male guy elephant seals have like a thousand women -- what is a woman elephant seal?
I don't know.
Anyway, a thousand female elephant seals in their harem, and they have these huge fights over who gets to have a -- you know, they can't share.
They just never went to preschool and learned how to share properly.
But part of, there is certainly enough aggression in human behavior to make it clear that that sort of a motivational system is going to lead to a certain amount of conflict in the world, and it's not all going to work out as sweetness and light.
People like Freud proposed that there was actually a death wish motivation out there.
Thanatos Freud's term.
I might talk about that more later in the term.
So that's, the possibility that there are motivations that themselves are not particularly nice, if you like, is one possible reason that the world is not a perfect place.
But perhaps the most useful way of understanding why the world doesn't work out so well, necessarily, given all this marvelous motivational hardware if you like that you've got is that, look, most of the time when we study these things, we study a motivation in isolation.
You want to study this cat's ability to learn about the puzzle box and the relation to the motivation of hunger, you take a hungry cat.
You put it in a box.
The only thing of any interest here is how to get out of the box and get to the fish.
You know, the only thing of interest here is, you know, there's a rat who is maybe a hungry rat.
In this case, he's got an electrode now in the feeding centers of his brain.
And all the other things that a rat might do are just not relevant here.
You don't operate in the world like that.
You operate in a much more complicated space.
I mean we can cartoon that here.
Here's you.
You know, if we got in the lab, maybe we can just do an experiment on when you do or do not eat, but that's not what you're dealing with out in -- well, alright, here is a way to get to this question.
How many of you have ever eaten something when you weren't hungry.
Well what's the matter with you, right?
You know, I just told you, you've got this elaborate stuff I've already erased the lateral hypothalamus.
You've got a brain full of circuitry designed to make you eat when you're hungry, and not to eat when you're not hungry.
And you don't look like that fury football with a lesion in your brain.
So what's your problem that you're eating when you're not hungry?
Well all right, let think about this is sort of metaphorical terms.
Let's imagine a situation where there are lots of different possibilities pulling on you.
Suppose you're at some sort of a social gathering of some sort, there are lots of possible things, lots of motivational systems at work at any one time.
I mean one of the things you could probably do is eat.
Maybe there's the possibility for fighting, mating, sleep, study.
You know, you could do all sorts of, you know, run.
You could do all sorts of stuff.
And all of these things, each of which may have it's own beautiful hard- wired core and it's an elaboration of a lifetime of learning, all of these things are pulling on you at once.
And, well, you know, all right, so maybe the sleep motivation and the study motivation are pretty weak at any given moment, but you know, let's suppose that there's pretty strong drives in this direction.
OK, so if you're sitting in this exciting vector field, where the resultant?
Right?
Right
[Laughter]
Now, I'm not proposing this as a serious model of why you might eat when, well I am proposing it as -- I'm not proposing it as a serious mathematical model of why you might eat, but I'm supposing it has a serious metaphor, that you might engage, and it's going to be much more complicated than this, right, I mean part of the reason, it's not just that there's, you know, want to eat?
Want to fight?
Oh think I'll go eat instead.
But, want to mate, you know, here?
Now?
Can't do that, but they're, you know, there those brain stem chunks of general arousal, oh yeah, well, I've sort of got a lot of activity here that I gotta do something.
So you end up with sort of displacement activity.
The problem is that the world is a complicated place, and if you were ever going to reduce this to map, it's going to have a lot of variables.
It's going to end up with a lot of variables in the equation.
Most of us manage to do OK. We eat when we don't want, but that's all right.
Later on in the course, you end up talking about, you know, perhaps the people where you find that they're fighting too much or something like that.
No lecture Thursday, just to reiterate, I will sign all those lovely add drop cards.
I'll see you next week.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.MIT.edu
Anyway.
If you're sitting there saying, what are they talking about weird thing in the syllabus.
This suggests you didn't read the syllabus, which is why we put the weird line in the syllabus about -- what is it -- what a warrior your mother would have been.
That sound faintly familiar?
No
Well it should sound familiar from reading the syllabus right?
Nobody's got a clue where it's from right?
No
Except for Rachel.
I know Rachel knows, because I told her today.
And obviously I cleverly arranged for it not to be something that could be Googled.
Because several years ago when Google appeared, I had a line in there to make sure people were reading it and, you know, within 10 seconds of handing out the syllabus, half the class said, oh yeah that's from like Richard the third or something.
So this one couldn't be Googled.
It is in fact from a book by John Steinbeck.
You probably all read some Steinbeck in high school.
Those of you who are fans of Arthurian literature might want to read his rewrites of the King Arthur or chunks of the King Arthur story.
And when you do, you will discover that that line about thinking about what kind of a warrior your mother would have been is in there.
Great book.
Any other questions of similar moment that I should be answering at the present time?
No.
OK. Well what I'm going to talk about today is more of the detail, if you like.
Last time I had up on the board this business about you're a slave to the environment, you're a slave to your brain.
Well this is the details of the way in which you are a slave to your environment.
Now typically that is the heading of the title of the lecture is something like learning the chapter is learning.
You're here thinking that what you came to do today is to learn.
And that's not what we're talking about.
The sort of learning that you are doing here is material that really gets covered in memory.
I present material to you, you store it in memories somewhere.
At the appropriate time you retrieve it from memory.
We'll talk about that in a few sessions down the line.
What I'm talking about today is a particular form of learning, which you can think of as association learning.
A very reflective form of learning.
A form of learning that is phylogenetically very old.
It shows up in essentially any beast you care to try it out in.
Those of you who are budding neuroscience sorts, if you were to go to Eric Kandel's website and look at his Nobel Prize lecture -- because he won the Nobel Prize.
You don't usually poster your Nobel Prize lecture until you win it.
Because little arrogant otherwise.
He won the Nobel Prize for working out the details simple association learning.
Initially, his work was done in a beastie known as Aplysia or a plysia, which is a sea slug.
It looks like a dill pickle and has just about as much personality and brain.
But it is capable of this form of learning.
It's a gorgeous lecture.
Nice art on the website.
if you're going to give a Nobel Prize lecture, you kind of don't want to waste it.
So I heard him give a version of this it a different meeting in Australia, and it's a good lecture.
All right what kind of learning is this that we're talking about.
How many people here like strawberries?
Some people like strawberries some people don't like strawberries.
Now you did not come into the world knowing that fact.
You now know that.
And I do not think that most of you learned that fact, you know, like in the circle time at preschool.
Hello children today we're going to decide if you like strawberries.
You like strawberries.
You don't like strawberries.
They make you, you know, break out in a rash.
Whereas no right?
You learned about your taste for strawberries in some different way.
You learn that you should, you know, dive under the bed when you see a flash of lightning.
Because you're scared of the thunder.
You learned about that not because you went to thunder school, but because you learned about the association.
You picked it up from the environment.
And it's that sort of association learning that I'm going to talk about today.
I'm going to talk about two different versions.
One of them is learning associations between stimuli in the world, traditionally known as classical conditioning or Pavlovian conditioning in honor of Ivan Pavlov.
The other is the learning the associations between what you do and the consequences of what you do, often called operant conditioning.
Sometimes called Skinnerian conditioning in honor of B.F. Skinner, one of it great practitioners.
Now let's start with this stimulus form of learning.
The basic classical conditioning experiment is one of those experiments that people know about before they come into a psych class typically.
That's Pavlov and his salivating dog.
What Pavlov was doing was studying digestion.
And dogs were his animal of choice.
In fact he's also a Nobel Prize winner, but his Nobel Prize lecture -- well it probably is posted on the web somewhere.
But anyway he won the Nobel Prize for work on the digestive enzymes in I think the stomach.
He then decided he was interested in digestive enzymes in saliva.
And he needed a supply of saliva.
Where you're going to get saliva from?
Well what he did was he put a little cannula, a little hole, in the cheek of a dog, and a little collection tube.
And then he took his dog, put him in a sort of harness and sprayed powdered meat into the dog's mouth.
If you spray powdered meat into a dog's mouth, the dog does what?
The dog salivates.
And since the dog's got a little hole in here some of the saliva drools out.
And you can go off and study it.
What Pavlov discovered that was of interest to him -- it turned out to be of interest to him -- was that he'd get up in the morning, grab the dog, put the dog in the harness, and the dog would start salivating before any meat powder showed up.
Pavlov's thinking, hmm I can save on meat powder
[LAUGHTER]
But the difference between the rest of us schlumps and Nobel Prize winning guys is when they see something interesting, they know that it's interesting.
And what Pavlov figured out is the dog is anticipating the meat powder.
The dog has learned that this situation means meat powder in some fashion.
Gee that's actually more interesting said Pavlov than dogs drool.
I'm going to study that for a awhile.
And so what he did was he set up the more familiar form of this where -- let's introduce some terminology.
He had food, the meat powder.
We will call that an unconditional stimulus.
The characteristic of an unconditional stimulus is that it produces an unconditioned response, saliva in this case, without you having to do anything.
You know, get your off the shelf dog.
That dog's going to drool for you if you give it food.
Then what he did was to pair that unconditioned stimulus with a conditioned stimulus, like a bell or a tone or something like that.
So now he was going bell, food and the dog obligingly drooled.
Bell, food, drool.
Boom, boom, boom, over and over again.
Then the critical thing to do is you do just the bell by itself and skip the food.
What do you discover?
You discover that the bell by itself now produces the saliva.
And when the conditioned stimulus alone is producing the response, we call that the conditioned response.
The animal has learned an association between bell and food.
And you're seeing that learning by measuring this conditioned response.
That kind of learning, which is what Kandel was studying in Aplysia, is presumably reflexive, automatic outside of the realm of consciousness even if I fall into sort of sloppy kind of language later.
Understands that the dog is not presumed to be sitting there thinking hmm bell, what does that bell mean?
That bell symbolizes for me the appearance of.
Nah.
Reflexive happening automatically.
Doesn't matter what the, you know, what the dog does or does not think.
Indeed you can fall victim to this sort of thing.
Quite apart from your conscious, wishes, desires, whatever, there's a vision researcher sort of a half of generation older than me who still mad at me.
Because 30 years ago I couldn't escape being conditioned in this fashion.
Back in those days the apparatus when you were putting up a visual stimulus made a noise.
I was supposed to push one button as quickly as I could.
One button if I saw something, and another button if I didn't see something.
But every time he put up a stimulus, the apparatus went click.
And I just would hit the stupid button.
I had learned the association between click and the stimulus.
And, you know, my little sea slug brain was refusing to be overridden by this, you know, great big conscious apparatus of this guy wants me to look for some really boring thing.
And he's still mad at me.
It's very, very sad.
OK.
This is a very ruled governed behavior.
In fact it probably says that on the handout.
No it says what's being learned here.
It's a very ruled governed behavior.
Pavlov himself worked out many of the rules, and then a large body of research afterwards continued that effort.
And let me tell you about some of the constraints on this form of learning, and then explain a bit about how it is that this might actually have anything to do with learning more interesting things.
What the sea slug learns is that gee when I sense shrimp juices I recall in the water, that means somebody's going to poke me.
So I should retract my gill.
That's the response that the sea slug makes, to retract its gill.
The response the dog makes is to drool.
I mean that's nice, but how does that relate to anything like human behavior?
So let me tell you a few of the constraints on it.
First a constraint that's really more like an advertisement for next time is to say that we talk about these conditioned stimuli out there.
In this case the bell.
Even something like deciding what the stimulus is, is not an entirely trivial business.
So suppose I was to come in here with a rabbit let's say.
Come here with a rabbit.
Every time I walk in with the rabbit, I activate the electric grid that's in your sea.
And you all get a little zips.
And, you know, you jump in the air.
Eventually I bring the rabbit in and what happens?
You jump without me having to bother running electric current through your posterior.
What's the stimulus there?
Well we sort of naturally would say it's, you know, it's the rabbit.
And it could be the rabbit's ear.
It could be rabbit ear combined with blackboard or something like that.
It turns out that we have natural, and animals have natural, ways of carving up the world into stimuli, into objects perhaps.
And when you're trying to figure out what's governing the behavior in animal, it's not even immediately trivially obvious what that stimulus might be.
That's what we'll end up talking about for the next few lectures after this.
But sticking with the realm of the constraints on the learning itself, one important constraint is that you have to notice the relationship between the conditioned stimulus and the unconditioned stimulus.
One version of that is completely trivial.
If I use a very quiet bell that you can't hear you don't learn anything about it.
You know big deal.
The more interesting case is illustrated by a phenomenon known as overshadowing.
Suppose what I do is I have to stimuli that appear at the same time.
This bell and a light.
A nice little, you know, little LED d or something like that.
So bell and light followed by food.
Bell and light followed by food.
Bell and light followed.
There's a loud bell and a little light.
Got it?
OK what happens in this situation is that when you play the bell by itself, you get salivation.
When you play the lights by itself, even though the light was always followed by the food, you don't get salivation.
Because the light has been overshadowed -- that's where the jargon comes from -- the light has been overshadowed by the presence of the bell.
What's the critical control experiment in an overshadowing paradigm?
What do you need to know here for this to be interesting?
Yeah?
[INAUDIBLE]
You have to do a training session with some other dogs or rat or something with just the light.
You have to go light, food, light, food, light, food, and show that this relatively wimpy light -- if presented in isolation -- will produce perfectly fine learning.
In the interesting version of the experiment, that's in fact the case.
But when you present, you know, bell and lights together, the stronger bell captures the learning and the weaker light loses.
So, you know, is a silly example where I was bringing in a rabbit.
You know, if the rabbit was, oh I don't know, wearing a little silver ring or something like that, you'd learn the association between rabbit and shock, but, not the association between ring and shock perhaps because it was too small.
It was overshadowed as a stimulus.
It is also critical that the CS predicts the US.
Did I get that right?
Yes.
The conditioned stimulus has to predict the unconditioned stimulus.
It is not adequate simply for the CS to just show up every time the US shows up.
So a silly example, everybody who bombs the midterm in this class, almost everybody maybe it's not perfect but it's a pretty good association, almost everybody who bombs the midterm in this class drank something shortly before doing that right.
So drink, bomb.
And it works every year.
So there's a lot to pairing.
Drink, bomb.
Look, I wouldn't on the basis of that give up drinking.
There's no predictive value there.
Because you drink a bunch of times when nothing has anything to do with the midterm.
There's no relationships.
What's important is a contingent predictive relationship.
This is a brain mechanisms that's there to learn what in the world predicts what other thing in the world.
In the standard Pavlovian set up here, the prediction is perfect.
Every time you ring the bell, you get the food.
If you cut that down so that the prediction is imperfect, sort of correlational, you'll still learn but you'll learn less and you'll learn more slowly.
So if, you know, half the time the bell rang you got food, and the bell never rang any other time, the animal would learn that bell more or less means food.
But it would be slower and less strong.
That's not what I wanted to move.
You know, been speaking of learning, I've been teaching in this room for years, and I have still never managed to reliably use these boards.
Always push the wrong button.
All right.
So suppose if you've got a good strong, you know, bell, food, bell, food, bell, food kind of thing, what you're going to do is over time you build up a rate of responding to some sort of asintotic level.
What happens if you change the contingency?
Most particularly what happens if you stopped feeding the dog after the bell?
The response will go away
Oh the responsible will go away, not the dog.
The dog doesn't have the option in this particular case.
Yeah.
The response will go away.
That's known in the conditioning literature as extinction.
The response will extinguish.
I mean that makes sense right?
You know, if the contingency no longer applies, why should I continue to respond on the basis of a contingency that's no longer valid.
Did you forget, you the dog, did you forget that contingency or unlearned that contingency?
Is it gone?
The answer is no.
It still seems to be there in some sense.
It's just you're no longer making the response.
How do we know that?
Well for instance, if you were to give the dog a little break, send him home, then put him, back in the apparatus.
You would discover that the response, even if you never presented anymore food, the response would come back and then extinguish again.
This is known as spontaneous recovery.
And, you know, to use the language that we shouldn't be using, it's like the dog saying, all right, you know, all right bell predicts food.
Oh bell doesn't predict food anymore.
I mean it's not like I forgot that the bell used to predict the food.
I'm not a dumb dog, I remember that.
But it doesn't anymore, so why should I drool.
OK. Now so I wonder what situation we're in here.
Are we back here or are we here?
Well we'll drool a little bit just to see.
Again the dog is not presumed to be doing anything like that kind of thinking.
But it's not that far off from more complicated situations that you could imagine in the real world.
So you develop a relationship with another person that produces some rate of conditioned response or something like that.
Because, you know, while you're responding in any case.
Then he decides not to have anything further to do with you.
Response disappears.
Then comes, oh I don't know, summer vacation.
And you see her again after vacation.
Do you remit a response?
Well are we here, or maybe we're back here
[LAUGHTER]
So you know, maybe a little response.
So that's spontaneous recovery.
The original notion was sort of that you were general purpose association learn, you can learn anything.
The evidence from preparedness, which I talked about before, suggests that you are better prepared to learn some things than others.
So you're better prepared to learn that snakes are scary then that bunnies are scary, for instance.
It's fairly general purpose or at least there are aspects of association learning that seem to be general purpose, but not all of it is completely general purpose.
One thing that you're certainly set up to do is to notice associations only over a limited time window.
Oh I lost underneath there.
Right.
Tone followed by food.
Tone followed by food that's the CS followed by the US, right?
So let's take this asintotic points here and plot that over here.
Now suppose that we systematically vary the relationship in time between the conditioned stimulus and the unconditioned stimulus.
Well this is on the order say of one or two seconds.
If I ring the bell today and feed you tomorrow, how much learning you figure we get?
[INAUDIBLE]
Not a lot.
First of all it's a very long boring experiment, since we need lots of pairing.
But basically you don't have to go that far.
But it's going to fade off in this direction.
That you can't have too long a period between the conditioned stimulus and the unconditioned stimulus.
Zero doesn't produce much conditioning and negative relationships.
So I give you food and then I ring a bell.
That says, you know, food that was food.
Food that was food.
You know OK.
It doesn't hit zero at zero, but there's really basically very little happening down that way.
So there's a narrow time window within which this chunk of you is looking for associations out there in the world.
The the world example here might be bump signs on the highway.
When is a bump sign on the highway useful?
If the bump sign shows up at exactly the same place as the bump, thank you.
You want it to show up a little bit before the bump.
If it shows up after the bump, you know, that was a bump
[LAUGHTER]
That's not that useful.
So it's limited constrained in time.
Now how can we go and get from this nice rule governed behavior to more complicated behaviors?
One way to see how that might work is a paradigm in the conditioning literature known as century -- did it again didn't I look at that, got it got it wrong on both dimensions -- called sensory preconditioning.
I'll just restart this over here.
It's easier.
Here's how a sensory preconditioning experiment might work if you were a rat or dog or something like that.
So step one.
I'm going to show you a light.
And then I'm going to ring a tone.
I'm going to do that a bunch of times.
Light, tone, light, tone, light, tone.
Then I'm going to play the light alone, or I play the tone alone.
Do you salivate?
No, why should you salivate?
It's got nothing to do with validation.
Step two.
Tone, food.
That's just the classical conditioning paradigm.
Tone, food, tone, food, tone, food.
Play the tone alone.
Well tone food whoops.
Saliva.
So now I'm going to play the tone alone.
And low and behold, we know I'll get the conditioned responses of salivation here.
The critical test is now I play.
I show the light in isolation.
What happens?
Well what happens is that you get the conditioned response.
The animal will now salivate in response to the light alone.
Why is this interesting?
Well what's the animal doing?
Well again we can sort of think in sort of conscious terms that aren't appropriate.
The animal is saying, light.
I know about light.
Light means tone.
Ah ha tone, I know about tone, tone means food.
So if tone means food, I should be like salivating.
And if light means tone, I should be salivating there.
So I think I'll just start, you know, drooling all over the place now.
What's important about this is actually you can see sort of graphically here.
I'm managing to get more remote from the response.
I'm working my way backward.
Suppose it was the case that at the end of every one of my lectures around 3:23, I was to say in summary.
And that your habit was to immediately run out of class thereafter and get yourself a snack.
You might discover that as I said in summary that somebody walked in
LAUGHTER]
You might discover that as I said in summary that you start to feel hungry, because of a learned association.
It might never occurred to you until you examined this, why you feel hungry when I said in summary.
But if we look at this timing thing, it's not the case that I say in summary, and one second later I blow M&Ms into your mouth or something like that.
You've got of somehow explain how you work backwards from the eventual M&Ms here back to something that I said.
And this is one way to do it.
You imagine that what you've got is an automatic association learner.
Its job is to look for sort of if a then b situations in the world.
And that it is capable of chaining those together.
That you can go if a then b, if b then c, if c then, you know.
Oh hey look if this then this thing way down the line somewhere.
And if you can start chaining these simple associations together, all of a sudden you've got the possibility of having a much more complicated kind of tool that could potentially explain much more complicated kinds of behavior.
So that's classical conditioning.
Good for explaining or good for talking about relationships between stimuli out in the world.
Another thing that you would want to learn about is the relationship between what you do and its consequences.
That is what we talked about a bit last time when I did the Thorndike puzzle box example.
You know Thorndike's cat was not so interested in learning about relations of stimuli in the world.
What it was interested in learning is if I do this then I get that fish.
And that's the core of operant conditioning.
Now I took this course once upon a time, but I took it at Princeton where I was an undergrad.
And that allows me to encapsulate the basic difference between a Princeton undergraduate education -- look at that, there's a guy wearing a Princeton sweatshirt right now.
Don't cover it up.
You know unless she's no longer talking to you.
That's extinction curve over there.
Anyway where were we.
Oh yes.
The difference between a Princeton education and a MIT education is that at Princeton this course satisfied the science requirement
[LAUGHTER]
You laugh.
But if I was to give this lecture at Princeton and say at MIT, this course satisfies a sort of, you know, humanities literature kind of requirement, they'd all laughs.
So the main difference between the courses was not in what, you know, the guy standing up front did in the way of lecturing, but here of course to make it a Hass course you do a lot of writing.
And at Princeton it was a lab course.
And it's a great pity in some way.
It's excellent that you are off writing, but it's a great pity not to have a lab component to this.
Because this operant conditioning thing is a lot cooler if you got a pigeon of your own.
Which is what we had.
You don't end up with a salivating dog of your own.
But pigeons are OK.
So the version of an operant conditioning experiment that people tend to know something about before they come into a courses, is a so called Skinner box.
Which is, you know, a pigeon
[LAUGHTER] He can be a rat too if you want.
It's just a matter of where you put the whiskers.
Anyway the idea of a Skinner boxes was you'd have an environment that could really control the options available to the pigeon or the rat.
And what you'd have in a pigeon Skinner box is a box that had nothing much in it except for a key here that is hooked up to a little micro switch or something.
And that the animal could peck at.
And we could record the pecks.
And a little bin down here where the bird seed could show up.
And stick a pigeon in there, and this is a hungry pigeon because you haven't fed it.
And so yes, a bummed out pigeon.
But if he pecks there in the right ways we'll feed him.
So this is simply the somewhat higher tech version of Thorndike's puzzle box you'll recognize.
The pigeon has to figure out to peck a the key.
If you just stick a pigeon in a Skinner box, even a hungry pigeon, the pigeon just sits there.
I mean, you know, does stuff.
But the pigeon doesn't immediately say, hey I've taken intro psych I know about this, I'm going to go and peck that thing.
You have to do what's known as shaping its behavior.
The first thing you have to do that's important, you have to get your pigeon from the basement to the fourth floor, where the lab was.
And should you every need to do that, the important thing to know is that plastic juice containers are really good.
You take the pigeon sticking it head first in the plastic juice container.
Not with the juice in it
[LAUGHTER]
And then the pigeon is actually quite calm under those circumstances.
If you decide how tough can a pigeon BE, and you don't need to do that because you can't find your juice container anyway, and you grab your pigeon because you're bigger than the pigeon.
Try to carry it upstairs, what you end up with is lose pigeons unlike the second and third floor.
So things that you're going to miss out on just because you're writing papers instead.
What you do once you stick that pigeon in the Skinner box?
If you just sit there, it's going to be a long day.
What you have to do is to shape the pigeon's behavior.
Probably says shaping somewhere on the handout.
Yeah use the law of effect to shape the animal it says there.
That doesn't mean like grr.
What it means is that the pigeon's just sitting there moving around doing pigeony like things, feeling hungry.
And the pigeon turns towards the wall that's got the key on.
And you push a little button that gives him a little bird seed.
He goes and eats the bird seed, and goes back to doing pigeony things.
But now the law of effect's working.
That bird seed is a positive reinforcer right.
So the chunk of the bird's brain that is doing this association learning between action and its consequences saying, birds seed that was good bird seed.
I like the bird seed.
What do I do to get more bird seed.
Well it's not saying that explicitly, but it's saying we'll do whatever we were just doing.
OK. Well now you don't want him to just be looking at the wall.
So now after a couple of rounds of that, you say, OK bird you only get the bird seed if you move a little closer to the wall.
The bird moves closer to the wall all right.
All right now you only get it if you're looking at the key.
Now you're only getting it if you're right up there.
And eventually you get the bird pecking at the key, and then you can go off and do other cool experiments from there.
But you have to shape the behavior in the way you want.
And the key business.
I mean there's a key.
There's the bird food.
That's somewhat arbitrary.
It's just, you know, what Skinner wanted to use to control the birds, you know, to make it possible to measure the bird's behavior.
We had a very clever pigeon when I did this lab.
And we got through the stuff that was in the lab book like really quickly.
Probably because the pigeon had been the pigeon for a half a dozen other, you know.
Oh yeah, yeah I'm supposed to fake it here and claim I don't know what's going on here.
All right yeah.
Oh good we're shape now give me the food
[LAUGHTER]
But anyway.
We still had some time.
We thought well we'll mess with this pigeon.
And so we started reinforcing the pigeon for turns.
So, you know, no bird seed unless you make a quarter turn, OK. We eventually had the bird making three full turns for each.
And that sort of staggering over to the bird see
[LAUGHTER]
But we had this great ballet dancing pigeons.
And that's in fact the basis of trained animal acts that you may have seen at various places.
And I should advertised that it's not confined to pigeons and things like that.
You can perfectly well condition human beings.
I already think I've used in the last lecture the example of jokes as a case of behavior getting shaped.
Jokes that get rewarded with laughs get repeated.
Jokes that get rewarded with smacks don't get repeated.
But again going back to my intro psych class, I should tell you that it is possible to condition the professor.
What you need to do is figure out well what reinforces professors.
The answer, at least in this sort of a setting, is students who look kind of interested, and smile and write notes.
At least that's what my a professor, who was in fact an important learning theorist in his own right, that's what he told us.
He said it's reinforcing if you're looking like you're interested and you're writing notes.
And so you should, he told us, be able to pick a behavior that the professor is doing, and reinforce it by smiling and writing notes at the appropriate time.
So, you know, I was going to be a psych major.
I did what I was told I was sitting about there.
My friend and I every time, the next psych lecture, every time Professor [?
Cayman ?]
moved to the left, we smiled and took notes.
And by the end of lecture he was most of the way out the door
[LAUGHTER]
It was great.
Now as evidence for the notion that this is quite unconscious and reflexive kind of learning, we told him about it afterwards.
And he absolutely believed that it was possible that it happened, and absolutely reported no awareness that it had happened.
So you're welcome to try this.
Your even welcome to try it here.
Let me tell you that I have made this offer in years past, and nobody has reported a great success.
But the great failure was when one recitation section got together and decided their strength in numbers.
But it turns out that if everybody in a recitation sits -- I still remember they were all sitting there -- if you all sit in one place, and I don't remember what I was supposed to do.
But when I did it, they all went [INAUDIBLE].
I kind of cracked up, but I don't believe I learned anything particular
[LAUGHTER]
But it can sit out there as a challenge for you.
If anybody gets it to work on, you know, the math guy or something.
Is Arthur Mattuck still teaching?
No
18, O whatever?
[UNINTELLIGIBLE]
1803 in the spring.
Oh that's too long to wait.
I've never been sure Mattuck will notice anyway.
But that's a separate issue
[LAUGHTER]
Anyway if anybody succeeds in conditioning there professor do let me know.
The points here is that this is a form of learning that does apply in the human world as well as in the pigeon world.
Once you've got that animal shaped to do what you want, you can then start studying the rules that govern that behavior.
Let me say a word about schedules of reinforcements, since I see that it's there on the handout.
Actually it's doesn't say schedules, oh it does.
Fix and variable ratio and fixed and variable interval schedules reinforcement.
What that means is if you're reinforcing the pigeon every time he pecks, he's on a fixed ratio 1 schedule.
You shape up the behavior, and at some point the pigeon is trained.
And if you're reinforcing him every time he makes a peck, he will emit pecks.
So let's call this the peck rate.
He'll emit pecks at something some asintotic rate, not unlike the salivation thing over there.
Actually I want to save myself a little space here.
So let's only go this far.
So an asintotic rate of responding.
Now suppose that you change the rule.
Suppose you say, well you can say it to the pigeon because the pigeon won't understand you, say to the pigeon, OK now you get reinforced only every 10th peck.
What's the pigeon going to do, do you think?
How's the behavior going to change?
[INAUDIBLE]
It's going to go squiggly, apparently.
No, every 10th peck, not every 10 seconds.
I'm interpreting your squiggly as the answer to a later question.
So save the squiggle, it'll be good
Peck faster
Peck faster.
Whoever said that, that sounds good.
Well if you've gotta work harder to get the same reward, you're going to end up pecking faster.
Right?
So if you sort of think about it in terms of your once upon a time, you could get an A in whatever.
You know writing.
For writing like my eight year old wrote this morning, seven sentences of at least six words each.
Took him all of breakfast to do that, because he'd lost his book.
An anyways it's a long story.
But, you know, so he emitted his writing behavior to relatively low rate.
If you emit your writing behavior at that rate, you guys are going to be in serious trouble, right?
To get the same reward, you now have to produce more work.
And as a result you'll crank up the level.
So that would be a fix ratio.
What I was describing every 10 pecks, that would be a fixed ratio of 10.
A variable ratio is on average every 10 pecks you get reinforced.
And that actually produces even higher levels of responding.
Now what the squiggle was the answer to is what happens if instead of a ratio schedule, you have an interval schedule?
Suppose you reinforced the behavior only for the first peck after every minute?
Make a different graph over here.
So a bird can peck it all they want.
But only when they peck after this interval, marker goes by or they're going to get reinforced.
Well I mean a pigeon with a wrist watch is going to look at his watch and say, hmm.
Pigeon doesn't have a wrist watch of course.
But they do have an ability to estimate time.
And so what they do, you know, immediately after getting rewarded, they just sort of sit there.
So this is again going to be rates of response.
They just sit there, and then they say, it's like a minute gone yet nothing happened.
Got to be now.
No, no, it's still not.
Oh it's coming.
No, no no.
Boom
[LAUGHTER]
And you get this very scallopy behavior.
Now if you don't think that has anything to do with human behavior.
Well first of all you can ask yourself about your writing output.
So for instance there's a paper do on a sort of a regular interval
[LAUGHTER]
A smart pigeon would be emitting words at about that rate right?
Well actually I did meet a student who claimed he was doing that.
More power to him.
But I think most of view, you know, it's do friday?
I think this point here is like Friday 4:00 pm or something.
Anyway.
Or you come to my house where if you're an eight year old you get your allowance on the weekend, right.
But only if you do your chores.
So Monday is my kid asking what chores he should do?
No.
Saturday morning?
You bet.
What can I do?
What can I do?
You know make my bed.
I made my bed.
Can I lick the floor now, you know
[LAUGHTER]
Oh and the downside of this is one of the reasons for paying people on a Friday.
Is that if you pay them on a Thursday, they don't show up on Friday.
Even though at some level they're much more likely to be sick on the Friday if you pay them on the Thursday.
So you want to pay them on a Friday and then they can be sick all weekend on their own time.
So that's what we mean by schedules of reinforcement.
This is also, operant conditioning, is also a very rule governed behavior.
It is governed by rules that are in many ways similar to the rules for classical conditioning.
Let's just do this one as an example.
It's probably later on in the handout.
So I reward little all old every time he pecks the key.
Now I stop rewarding and what happens to the behavior?
I take that boom to mean, yeah, it extinguishes.
If I give him a break, I'll get some spontaneous recovery and it'll extinguished again.
Just like you'll get in classical conditioning.
What happens in these?
How does the extinction compare in a schedule like this to a schedule like this?
How about a hand?
I'm hearing good mother mutterings but nobody has a hand.
So slower.
I heard some more slower mutterings.
It's slower.
If that's not intuitively obvious, ask yourself about the difference between a slot machine and a Coke machine, right?
You put a buck in the coke machine, you get a coke right?
You put a buck in the coke machine, you don't get a coke.
How many more bucks do you put in?
[LAUGHTER]
Well all right.
So you're not too dumb.
That's good.
OK. Now you go to Las Vegas you put a buck in the slot machine and you don't get anything out.
What do you do?
Put another buck in.
Actually Las Vegas is a beautiful example of applied conditioning theory.
We can send everybody on a class trip.
Actually we did that didn't we, sort of.
There's the best seller -- that was more applied math than applied psychology.
The applied psychology thing is when a slot machine pays off, traditionally what happens is the coins come out right.
Now often they're electronic, but they simulated anyway.
The coins come out.
What di the coins hit when they come out?
Metal
Metal.
Why do they hit metal?
To make a sound
To make a racket.
Why do you want to make a racket?
Well that racket itself becomes positively reinforcing.
You want to hear that sound right.
You want to put that buck in and hear crunch crunch.
Oh yeah this, is very good stuff.
Now slot machines.
I mean you don't go out and play slot machines a lot, but you see the movies.
I hope.
Anyway.
Slot machines are typically located in small sound proof booth?
No.
Where are they located?
[UNINTELLIGIBLE]
On a giant floor with thousands of slot machines.
What does that mean?
That means when somebody five blocks over gets a pay off, what do you hear?
You hear the sound.
You've been conditioned to find that reinforcing.
And you are so dumb
[LAUGHTER]
They know that you are willing to being reinforced by somebody else getting paid off.
And you say oh man there's money happening here let me put some more into this thing.
Anyway.
Lovely applied psychology in, you know, the worst sense of the word.
Well maybe not the worst.
I can think of a few others.
But while I'm thinking of a few other worse ones, take a minute to stretch.
Reinforce your neighbor.
Wake up your neighbor if that's necessary.
[SIDE___CONVERSA TION___WITH___STUDENT]
OK.
There is a sense in which you are a general purpose association learner.
And have mechanisms in your brain that are basically saying, you know, teach me about the associations between stimuli or the association between my acts and their consequences.
And I can be fairly broad minded about what those stimuli are, or what those acts might be.
And then there are special purpose versions of these things designed for very particular tasks.
Let me fish around for what's perhaps the best example of this.
Is there anybody here who can think of in their life of food that they used to love, but they now hate or won't touch, and they know exactly why?
Yeah out there in the cheap seats?
I ate a chocolate frosted cupcake when I was little.
And I went home and I got the flu, and I don't eat it anymore
Perfect Gee.
First try no weird examples this year.
Now you've got the flu.
let's get a little graphic here.
You've got the flu and what happened?
Vomit a lot
Thank you that's what we needed to know
[LAUGHTER]
And not only that, we also need to know do you believe that the chocolate cupcake gave you the flu?
Not anymore.
But I use to definitely think that.
[LAUGHTER]
Yeah?
Alright, when I was in kindergarten --
He doesn't care that I already have a good example.
He wants to give me another cool example.
All right.
All right.
All right.
All right.
It's not as good?
When I was in kindergarten the first day of school or the first week I ate lunch in the cafeteria and didn't eat a school lunch again five, six years
This is why he's slim to this day
[LAUGHTER]
Actually we'll come back to that example, it's sort of a useful function to.
So she ate the cupcake, she got sick.
She knows at some level it wasn't the cupcake that made her sick.
But there is a chunk of brain that says, I got sick.
I ate cupcake.
Shortly thereafter got sick.
Don't want to eat cupcake anymore.
This is a chunk of your brain that is quite immune, or is actually very hard for you to talk to and say, wait a second we used to love cupcakes.
Cupcakes are really good.
It doesn't want to hear about this.
It says, you ate it, you got sick.
And this is a form of association learning.
Well the original version of this was discovered by some rat who ate a cupcake and threw up.
But in the learning literature, it was studied by a guy named Garcia.
So it's sometimes goes by the name of the Garcia effect, what he did was he had rats, in a sort of a Skinner box situation.
And he gave them a new flavor of water like spearmint or something.
And then he made the poor rats sick.
Strapped it to the turntable of a record player
[LAUGHTER]
An experiment you can't do m because you can't fit the rat into the CD drive
[LAUGHTER]
And then the next day you give the rat a choice.
You want spearmint water or you want this other new weird flavor.
And the rat goes to the other flavor.
Says forget it, you know, I'm not going for the spearmint water.
But this is a very specific form of learning.
It sometimes called taster version, which is not quite right.
If you happen to be a sensation and perception person.
Taste, as the sensation that your tongue does for you, is really restricted to sweet, sour, salty and bitter.
What you become averse to is the flavor, most of which is smell.
Or perhaps in human cases, a larger context like school lunch or something like that.
But there are strong restrictions on what it is that you will develop in a version two, particular if you're a rat.
So the other version of the experiment Garcia did was he gave rats a new bottled of water.
And this one was flashing lights right.
Water never did that before.
Drink the water.
Get sick.
The next day you got a choice between the flashing light one, and the one that's making clicking noises or something like that.
Rat doesn't care.
This little chunk of brain was not built to figure out that flashing water is a problem.
This is a chunk of brain that is there to tell you that it knows some things about food.
Foods have flavors, and if you get sick some chunk of time after eating, you should avoid that flavor the next time, and things associated with that flavor.
Also the timing.
Where did my timing parameter go?
The timing is different.
If you eat something and get sick instantly, nothing happens.
You don't learn anything.
Because this little box knows it takes a while to get sick.
I think your story was ate the cupcake, went home, got the flu.
Hours later she's losing it.
And this little junk of brain is working back in time.
It's got its own time window.
It's working back in time saying, what did we eat a few hours ago?
Cupcakes.
Don't want to eat the cupcake.
Very useful thing right.
If you were grazing, you know, an omnivore kind of animal, very useful thing to have a device that says don't eat it because, you know, last time you ate it made you sick.
Oh the other thing when we were doing this tone, food, tone, food pairing over and over again.
How many cupcakes did you eat?
One cup cake.
And you repeated this lots of time?
No.
Right.
Because this is a little association mechanism designed to keep you alive.
And if, you know, eat the cupcake almost die.
You know, you don't want a bunch more pairings of eat the cupcake, almost died, eat the cupcake.
Oh man that time dead
[LAUGHTER]
So one try of learning, which by the way when Garcia discovered it, it was thought to be impossible.
But it's a special case of an association learning that works in a single trial.
It has its pitfalls.
And in fact it's sort of cognitive impenetrability.
The fact that it doesn't care what you know is a problem.
So except for the seniors here, of course alcohol has not past your lips.
And even the seniors have not drunk to excess.
So what may be news to you is that if you drink too much alcohol you can get sick
[LAUGHTER]
They knew that.
OK.
Anyway drink too much alcohol you get sick right.
The question is what do you develop the aversion to?
A smart system, rather like this, you know, the writing example, the smart system would say I just drank, you know, 12 screwdrivers in a row or something like that.
No more vodka for me.
The problem is that alcohol has very minimal flavor to it, and what happens is you drink a screwdriver -- orange juice plus vodka or something like that -- you get violently sick.
I'm not going to touch that orange juice anymore
[LAUGHTER]
You know.
Pile of beers and a pile pretzels.
You're violently ill, and the pretzels just look miserable to you.
so it is a disadvantage.
It's called clever that this thing is out there to save your life.
Unfortunately in civilization, we have figured out clever ways to misuse it I'm afraid.
Well since it says example two of superstitious behavior, let me say a quick word about superstitious behavior.
This is the best time to see what Skinner called superstitious behavior, because all you have to do is watch baseball.
The Law of Effect said if something good happens, you're going to do what you were doing just beforehand.
Think about baseball.
What something good that can happen?
You get up to the plate.
You hit the ball.
It goes over the fence.
And you get to run around and stuff.
Well what were you doing just before?
Well, you know, maybe you wiped off your shoes or something like that.
So next time you come up to the plate, you wipe off your shoes.
Don't think anything about it.
Pretty soon you're doing this all the time.
Some sports writer ask you, you know, how come you wipe off the shoes every time?
The plate's slippery or some story like that.
You got no idea why it is.
But it's probably been shaped into place by the contingencies of reinforcement.
The second best place to see this in my experience is at exams.
Look around at exams perhaps.
Look at yourself at exams and say, look at those three neatly lined up pencils.
Points exactly lined up there.
The coke has to be exactly to the left of the test paper, and the snickers bar has to be right there yeah.
You know, or I'm going to flunk
[LAUGHTER]
Again, if you're asked about it, the answers oh I don't know it just makes me feel comfortable or something like that.
But it's again sort of superstitious behavior perhaps shaped into place by this sort of this Law of Effect kind of work.
Let me say a bit more about how this applies out in the real world.
Schedules of reinforcement are the sorts of things that we do to ourselves all the time.
For example in child rearing.
Is that actually the next example on the handout maybe?
Yeah it says something about parents and children.
Suppose that you were a little kid, which you once upon a time you were.
and you want a cookie.
So you say, can I have a cookie?
And your mom gives you a cookie.
And that's good.
The next time you say, can I have a cookie, and it's just before dinner , SO SHE says no.
So what do you?
Well maybe you extinguish the behavior immediately.
And you're a perfect model child, and you end up at, you know, like MIT or something like.
Can I have a cookie?
Please can I have a cookie?
Pretty please can I have a cookie?
Eventually your mom will say, oh here have a cookie right.
All right so what have you done?
We now moved you from an FR1 to an FR something else, or actually it's probably a VR schedule of some sort.
So now you're going to be emitting cookie behavior at a much more rapid right.
Can I have a cookie?
Can I have a cookie?
Please can I have a cookie?
I want a cookie.
I want a cookie right now
[LAUGHTER]
Eventually, you know, your parents get tired of this.
They decide they're going to cut you off here.
But you're now up here somewhere.
How's that extinction curve look?
Oh man, it goes out to about, you know, age 23 or something like that
[LAUGHTER]
Plus parents are not good at this.
So what you do is can I have a cookie?
Can I have a cookie?
Can I have a cookie?
Can I have a cookie?
I really want a cookie.
OK have a cookie.
OK.
So now we're on VR 2064 kind of schedule.
And everybody's going nuts.
The other place you see a similar sort of example is sleeping through the night.
The kid cries right.
You go and comfort the kid.
Put the kid back down go to sleep.
Eventually you decide this little monster needs to sleep through the night, because like I need to sleep through the night.
So I'm going to not get up when he cries.
He's crying, I can't sleep.
Crying for like an hour
[LAUGHTER]
Oh maybe just this time.
All right.
Now you've rewarded him for crying for an hour, rather than for crying for a couple of seconds.
And again you end up with a similar sort of problem.
My eldest actually had this to an absolute art when he was little.
Of course he didn't know it again.
Because all nice unconscious and reflexive stuff.
He came into the world built so that if he cried for more than a few minutes, he also threw up
[LAUGHTER]
All right.
So it's, you know, besides it's time for him to sleep through the night.
He's crying you're saying, oh man if he doesn't go back to sleep in a second not only am I going to go nuts, but I'm going to have to do the laundry again.
But if I reward him then.
I don't think they've sleep through the night yet.
Is this encouraging yet?
Mara is getting so depressed there.
All right that you're not babies anymore.
Let's give an example of the relevance of this that might have something to do closer to adult behavior.
One of the interesting realms where these schedules of reinforcement may have an unintentional, well none of it has intention to it, an unfortunate negative consequence is a phenomenon known in at least in the research on sexual behavior as getting to yes.
What is this about?
Typically in heterosexual relationships the request, verbal or otherwise, for some sort of positive reinforcement of a sexual nature comes from the guy and is delivered to the woman in some fashion.
Typically that request, verbal or otherwise, is met with a no initially.
If the relationship develops over time of course, eventually odds are that there will be a yes.
Well what does that sound like?
That some sort of a variable schedule of reinforcement.
Oh why might that be the case?
We'll talk about this more later in the term.
But there is interesting evidence that guys are not some, you know, unbridal bag of hormones
[LAUGHTER]
But that they actually fall in love faster than women do.
So, you know, it's the Romeo and Juliet thing right.
You know, he's allegedly in love with Rosalind.
He goes and see Juliet, and two seconds later it's oh she doth teaches the torches to burn bright and all that sort of stuff.
Woo, he's gone
[LAUGHTER]
And when he walks out of the party later, and meets his friend the psychologist who does a survey kind of thing.
He will report he feels like he's in love.
And I don't know what yet, Juliet may be a different example.
I mean they all end up dead and things like that.
But typically sheet will be slower to report that she feels like she's in love.
So he's sitting there saying, you know, I'm in love and I'm going to be with her forever.
And if I'm going to be with her forever, there's certain things that we might be interested in.
And she's saying, I just met him five minutes ago, I mean what is this about?
[LAUGHTER]
Anyway.
The result is some sort of a variable ratio schedule of reinforcement.
And in most cases this works itself out just fine.
But suppose the relationship doesn't go particularly well.
Suppose for example -- well where's my extinction curve?
Here's a good extension curve.
So you're up here somewhere on this variable variable ratio schedule of reinforcement.
And let's say she decides that this is not a relationship that she's interested in anymore.
It takes him now an extremely long time potentially to get the hint and stop, you know, pressing that key in the Skinner box.
And we'll talk later about the ambiguous nature of communications in this regard.
If he's not sure whether she is saying yes or no or whatever, you can end up in trouble.
And this an example, you know, where these sort of rules of association learning suddenly become relevant in behavior that's much more complicated than whether or not you're pecking a response key in a Skinner box.
It's that kind of application, the possibility of going from these very simple behaviors to much more complicated, much more richer behaviors that drove what was the leading school of psychology in the first half the 20th century in the US, which was known as behaviorism.
Which was the doctrine that basically said, look classical conditioning, operant conditioning, a couple other bits, those are the atoms of behavior.
And we can build up the molecules and the rest of the organism out of those atoms.
That's all we need.
So I put this quote from John Watson who's the founder of American behaviorism on there, where he says, you know, we can write a psychology.
Define it as the science of behavior.
That was a move by itself right.
No science of behavior and mental life, or no signs of human mental life or something.
The science of behavior of observable behavior, never go back on our definition.
Never use terms like consciousness, mental states, mind, content and so on.
It can be done in terms of habit formation, habit integration, which are terms for this sort of association learning.
He is basically saying, that look if you're taking chemistry, and you go to the chemistry prof and say, you know, these elements you've got here, they're nice elements.
But I think I want another 16 kind of constructs to explain what's going on here.
The chemistry professor's going to look at you and say, no sorry these are the elements.
Everything can be built out this stuff.
You don't like it you got a problem.
And the behaviorists were saying essentially the same thing about psychology.
They thought that a very small set of atoms in this case, these Laws of Association, were the elementary properties, and you could build up everything out of that.
That carried with it a couple of an interesting bits of ideology that will recur later in the course.
One of them was the idea that you were basically an association engine.
You were there to learn associations.
And that everything that was worth knowing in psychology, everything that was worth studying in psychology, was something that was learnable.
That it was not interesting beyond, you know, for their purposes kind of trivial things.
Like you came with eyes and not radar dishes.
You know beyond that sort of thing, it was uninteresting what nature had provided to you.
Psychology was the effect of what the interaction of the environment and these atoms of learning.
That Everybody was essentially identical until this learning started.
And what got you to MIT was the contingencies of reinforcement over the prior 18 years of your life.
This resonated with a certain democratic current in American political thought.
In sort of, you know, grade school civics lessons form.
This was the doctrine that anybody could grow up to be president.
You didn't actually have to be a Bush to be president.
Anybody could be president.
What was important was how the environment shaped you.
That's a doctrine in its strongest form known as empiricism.
That look about right?
OK.
The opposite pole is nativism, the notion that your genetic innate endowment is determinitive.
And that you're here, because you were born with, you know, you've got the gene for calculus in there somewhere.
Some of you may have now decided you lost it somewhere.
But you had once upon a time.
So the notion here is what the behaviorist thought was they were strongly weighted towards this empiricist, environmentally driven side of things.
In a couple of minutes.
Why am I using the past tense?
Why aren't we all behaviorists now?
There are a number of reasons.
I suppose the broadest reason is a course like this is full of, you know, on the one hand position on the other hand, you know.
Empiricism and nativism.
And I'll give you a hint for the exam.
The answer always turns out to be both right.
The extreme positions, the extreme theoretical positions in psychology never seem to quite work out.
But sort of less trivially.
The basics of association learning theory are absolutely worth a lecture in this course.
And absolutely worth chapter 4.
And you'll find on the handouts, you know, guide for reading in chapter 4.
But they're not all of psychology.
Watson's statement says, you know, we can build a science of behavior without ever talking about, you know, consciousness for example.
But, you know, that's an interesting topic.
And if I want to know about it, I don't want to be told by the field's idealogues that it's not a legitimate area of inquiry.
I don't want to be told that the feeling aspect of emotion is not a legitimate topic of psychology.
And that the only thing we can do is, you know, observe how many tears are coming out, something like that.
It proved to be an incomplete psychology.
In.
The same way I should say, that the occupants of the nativist poll at the moment are the most dogmatic of the evolutionary psychologists.
Evolution has wonderful explanatory power in psychology.
But evolution is not itself a psychology.
It's not a complete psychology by itself.
It doesn't give you the richness of the field as a whole.
Another important reason we're not all behaviorist is it turns out to be somewhere between uninteresting and wrong to argue that everything that was of interest was learnable.
Language, we'll see later in the course, is an example.
Course you learn the language that you speak.
But you learned it because you had a brain that was built to learn a language.
And that innate endowment, that language learning endowment, is worth study in its own right.
And that wasn't something that fit into the behaviorist program.
So that's why you're not a behaviorist now.
But even if you're not a behaviorist, you still want to read chapter 4.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
What I'm going to do for the next three lectures is to talk about how we see.
What I'm going to do today is talk about how we take information in.
Next Tuesday, I'll talk about the fact that we take in too much information -- more information than we can use and we have to select amongst that information.
On Thursday of next week, what I'll talk about is the fact that perception is never just a direct, somehow registration of the outside world.
It's always an active interpretation of the world, and I'll talk about how we go about interpreting the information that we've gotten from the outside.
I will be barely able to scratch the surface of the topic.
I will abandon all discussion of the other senses, for instance, which is a great pity.
Now, it's true that I can barely scratch the surface of almost any topic in this course, but here the danger is greater because this is actually what I do for a living.
So, on any little bit of this, I can talk forever.
So, exactly how much of the surface I scratch will be determined by how far afield I end up going.
This slide is up there to illustrate, in some sense, those three bits.
Let's see, if I take the stage, chalkboard light off -- there we go, you want to go off?
Then you can see a little bit better.
You can sort of see stuff.
That's this Bruegel's painting of the battle between the good angels and the bad angels.
For me, the purpose is to say look, it's clear that you see something, it's clear that what you are seeing is an active interpretation.
In fact, you may not have particularly known that it was a battle between good angels and bad angels before you look at it.
It's also clear that even though you, in some sense, see the whole thing, you're not processing it fully, at least not all at once.
So, for instance, you may not have particularly noticed -- I don't know what he is, he's a really cool beast -- I'm not sure if that's a bad angel falling or something.
But you probably hadn't particularly noticed this fish guy.
Actually, in this picture you're probably still not noticing the fish guy -- he just looks like a blob.
It's a great picture, look at it sometime.
And this guy's got very bad stomach problems, whatever he's doing.
OK. Rather than continuing to try to look at a murky picture, let's go to a less murky picture and let's start by talking about how you actually get information in from the visual system, which obviously involves going through the eye.
On the handout, which -- do I have a copy of it here, yes, OK. A lot of terminology.
Are you responsible for it?
Yes, of course.
If you miss some little bit of terminology, that's not going to sink you in the course, either on this topic or on any other topic.
You want to put your effort into thinking about the broader topic, but it's useful in being able to talk intelligently about it to get a grip on some of this terminology, and hence, the tour of the visual system that I will now embark on.
From the outside-in, the purpose of the eye, or at least the front half of the eye is to form an image on the back half of the eye.
So all of this, sometimes called the anterior segment of the eye, is there to form an image, like a camera would, on the back surface of the eye.
Most of the optical power of the eye is at the very front surface of it.
That front surface is called the cornea.
It has, apart from its virtues as a lens focusing light, it is also one of the two surfaces in the body most sensitive to pain.
You may try this out if you want at home.
I don't recommend it, but if you scratch your cornea you will know it and you will be happy.
What's the other one?
What's the other surface, do you think?
Anybody care to venture a guess?
Thumbs?
No, that's a novel thought.
No, look -- he's thinking like the cartoons, right, you hit the thumb with the hammer and ow, it hurts.
Let me tell you, if you hit your cornea with the hammer it hurts a lot more.
Yes?
Some place inside the mouth?
Some place inside the mouth.
No.
Again, you think about that inside the mouth is a place where pain happens, but that's because you do stupid things with your mouth, like hey, that pizza looks good.
Ow.
You would never stick the pizza on your eyeball.
So, again, nicely innervated for pain but not particularly.
Not the other great location.
I'll take one more here
[INAUDIBLE] at the bottom of your foot?
Yeah, yeah, same sort of thing, right.
You step on nails or something.
The bottom of your foot's a good place.
There's two people here, I've just gotta take their answers because they're so eager
Yeah, that's what I was going to say, somewhere inside your ear
Somewhere inside your ear is the point, not the chunk that so many people are thrilled with.
Oh, let's put a hole in this -- oh, that was fun, let's do it again.
That obviously hurts, I guess.
I've never gone in for this myself.
One of the reasons your mother said don't put anything inside your ear that's bigger than your elbow is because if you put something small and pointy in there, it eventually reaches your eardrum and that's the other surface.
When you had an ear infection as a kid and the pressure from your middle ear, the fluid in your ear, stretched that, it hurt, and that hurts a lot.
So, don't put the hot pizza in your ear either.
OK.
The cornea is just a relatively thin layer.
It's held in shape by fluid, a watery-like fluid, which is known technically as the aqueous humor -- humor in this case has nothing to do with being funny.
It's an old word for a fluid, It forms a space that separates the cornea from the iris.
The iris is the bit that gives your eye its color, but it's basically a sphincter of muscle with a hole in it, and that hold is the pupil of the eye, which is the aperture of the camera.
That's where light's getting in from the outside into the inner portions of the eye.
OK.
The pupil changes size, because you've got this sphincter of muscle that either relaxes or contracts.
What makes it change size?
Light
Light.
We know that.
More light makes it do what?
Get smaller
Get smaller.
Why?
[INAUDIBLE]
Yeah, so there's less light.
Sure, it's going to be less light, but why do you have this?
What did they teach you in 3rd grade?
[INAUDIBLE]
Never stick your elbow in your eye.
Yes, I know that.
No, I need a hand here.
There's too many mutterings.
There's a hand
Because you don't want to go blind
Because you don't want to go blind.
Why would you go blind?
There's too much light
There's too much light.
See, that's what they teach you in 3rd grade.
It's bogus.
It sounds good and it seems to make sense, but ask yourself what level of protection that pupil can give you.
Right, so how much light is it going to cut out?
The answer is it cuts out a bit more than a factor of 10.
From the largest it can be to the smallest it'll be if you stare at -- you know, if you're out in really bright light.
So it can take it can change light levels by about a factor of 10.
If you go from in here right now -- actually, it's fairly dim in here -- but if you go from here to just go outside into the courtyard, the change in light level is going to be on the order of a factor of 1,000 right there.
The change in light level that your visual system can deal with from, say, a moonlit night to bright sun on a snow field is on the order of 10 to the twelfth or 10 to the thirteenth.
So there's no way that the need to protect yourself is going to be taken care of by the pupil of the eye.
There's just not enough range there.
The pupillary constriction, what it's probably doing for you, those of you who are photographers, is it's increasing the depth of field on your camera.
A pinhole camera will focus many different distances at once.
A larger aperature will only focus one plane at one time.
And so if you've got lots of light, you crank down the pupil in order to be able to focus -- in order to get better focus, basically.
But the protection thing just doesn't -- it's not a stupid answer because it's what everybody believes, but it doesn't turn out to be a correct answer.
Oh, now there's several other things that will change the size of your pupil, but one other of interest, what else changes the size of your pupil?
Anybody know anything else that they can change the size--?
Yup?
Looking all the way down there at you
Looking all the way down there at me.
Well, yes, but for reasons that are--.
Why?
Well, no, well you're changing -- sorry, that's the next bullet here.
What you're changing when you change focus of the eye is you're changing the shape of the lens -- at least you are.
I'm not because at my age the lens has basically become a rigid lump, and so when I want to look from there to here, I use an external lens.
Yes, back to the pupil
When you lie
When you lie.
Yes, that will work a little bit maybe, not reliably enough to -- you know, the CIA or something every now and then gets the idea that that really works.
It's on the right general track.
Yeah?
Some drugs do that?
Yup.
Some drugs will certainly modulate it.
All right, [UNINTELLIGIBLE]
When you look at the object of your affection
There you go.
That's particularly the one I was fishing for.
Your level of arousal will change the size of your pupil.
Notably, if you look at something that you like or someone that you like, your pupils get bigger.
Obviously, not everybody here knew that explicitly, but you sort of knew that implicitly, because the result of that is that you think that things with big pupils are cuter than things a little pinhole pupils, right.
That's why sappy cartoons or cards have big pupils.
It is also why in years gone by, women would put a drug known as atropine or known as a cosmetic as belladonna for pretty woman, pretty girl.
In their eyes what it did was it paralyzed and relaxed the muscles of the irish and made a great big pupil.
It didn't do anything good for their vision because it made things kind of blurry.
It also paralyzing the lens.
But it gave them great big pupils, and so they're looking at the guy who may or may not be the object of their desire -- they're looking at some guy, they can't see him so well, which who knows, that may have helped.
But he's looking at her thinking implicitly her pupils are really big, I know what that means.
Well, it turns out, following up on whoever it was over there, it actually means she's on drugs.
In any case, pupil size is modulated by emotional state and it's one of these things that we sort of knew implicitly, people know implicitly, and think big pupils are attractive for that reason.
It's only when you show up here you find out these things explicitly.
All right, I've already said something about the lens.
The most notable thing about the lens is it gets harder with age and -- you could try this out.
In fact, it's a good time in your life to start this experiment.
Take something like the handout and ask yourself how close you can get and have the letters still be in sharp focus.
This is so sad it like makes me weep.
But I know you're getting there, too.
Now what you can do -- you could actually, if you're compulsive about these sort of things, write it down somewhere and just check every year.
This is one of those things where the signs of aging start showing up now.
You will be able to see, even during the course of your twenties that that so-called near point is moving further out.
For me without my glasses we're out here somewhere.
And eventually you're locked at one plane of focus, and depending on where that is, determines which particular set of glasses you need -- do you need glasses to read, do you need bifocals, whatever.
It's because you've lost the ability to fine-tune that focus.
Now, you need something to hold the rest of the eye in shape.
What you've got in there is it's really a jelly called the vitreous humor for the glass-like substance.
For our purposes it's not desperately interesting, it's there to hold your eye out so that the front end of the eye can make an image on this back surface.
The back surface of the eye where the transduction from photons, the light energy, into signals in the nervous system where that takes place, that tissue is known as the retina -- the word comes from the word for a net.
The fovea is the point -- fovea comes the word for pit -- it's the point on the retina where you have the highest visual aquity, the greatest ability to resolve small details.
You don't particularly realize in the course of your normal day-to-day existence just how bad vision gets away from the fovea.
You have this notion of a highly detailed world that's in focus all over the place.
But that, to jump ahead to the third lecture in this series, is a construction.
It's not reality.
And that you can prove to yourself.
Take the handout again, hold it at some comfortable reading distance, fixate on the l on that second line there where it says Lecture 4.
If you're fixating on the l, the l is forming an image on your fovea.
Ask yourself while you're fixated on the l, how many letters off to the right you can actually read.
And the answer is going to be not many.
If you think you can read to the end of line it's because you're not fixating -- you're moving your eyes around.
Again, like the pupil thing, this is something you knew implicitly.
Nobody sits there and says I'm going to read this handout.
I can see it, it's there, but I can't read it because the outside of the fovea, I simply don't have the resolving ability to do it.
Why not?
Well, I can give you at least one reason.
You have about a hundred million photoreceptors in your retina.
Photoreceptors are the cells that convert light energy into a signal in the nervous system, they're the transduction elements.
You have only about a quarter of a million to a million axons, nerve fibers running from the eye to the brain.
So that hundred million has got to get boiled down to the much smaller number that you've got available to you.
So that means you can't treat each photoreceptor as a pixel with a private line to the brain.
You do have something approximating this in the fovea where you have fine detailed resolution.
But in the periphery, away from the fovea, what you have is many, many, many, many photoreceptors that are all ganged together to send their signal off to the central nervous system.
So you lose resolution out there in the periphery.
Let's say a little more about the details of that retina.
Oh, the optic nerve is the bundle of fibers going off from the eye to the brain.
I'll say a bit more about that in a minute or so.
A quick tour of the retina itself.
Up at the top there you've got the photoreceptors, these transduction elements.
There are two flavors of them called rods and cones -- named rods and cones because under the microscope rods are rod-shaped and cones are cone-shaped.
They are different in their functions and capabilities.
Cones operate in bright light at daylight levels.
They do mediate color vision and they are concentrated at the fovea.
They are most densely represented at the fovea.
Rods work in dim light levels.
They are not able to mediate color vision, which is why on a moonlit night you don't see colors -- it's not because color's somehow drained out of the world.
It's because the visual system that's looking at that world can no longer analyze the input for color information, but only for grey scaled information.
Rod photoreceptors are concentrated away from the retina -- I'm sorry, away from the fovea.
They are, in fact, absent from the central fovea.
The best place to see this actually is if your room gets dim enough at night that you can't see color, you're working in the rod realm, and then you will discover that if you fixate on a dim spot of light you can't see it.
That there might be something out here -- you notice it and then you move your eye to look at it and it'll disappear, because the image, if it's small enough, will now fall into the fovea and there are no rods there, you can't see it.
This is why if you're a star gazer and you're looking at a dim star, you want to look at that star out of the so-called corner of your eye.
The corner of your eye is about 20 degrees away from straight ahead because that's where the density of rod photoreceptors turns out to be greatest.
I was going to say something else about rod photoreceptors, but that has disappeared from my mind.
So I will say that the rest of this is a collection of nerve cells, of neurons.
It's really a little chunk of brain.
Developmentally the retina is a chunk of brain that's pushed out into the eye, and this is the start of the visual nervous system.
For present purposes it's worth thinking about two aspects of this organization.
One is a through path that runs from the photoreceptors through so-called bipolar cells to ganglion cells.
There's sort of a vertical path in this picture heading from the photoreceptors out to the brain.
Then there are cells whose processes are sitting here and here making horizontal connections so that photoreceptors don't behave like isolated pixels so that they can talk to each other, and so you can start doing some sort calculations, if you like.
Those are cells like horizontal cells that have their processes up here, and amacrine cells that have their processes down here.
So those are things making horizontal connections across the retina.
Photoreceptors, bipolar cells, ganglion cells are forming this through pathway going from eye to brain.
Now, if you were to look at this and I was to say where is light coming from in this picture, the reasonable answer would be?
[INAUDIBLE]
It would seem like a reasonable answer would be like it's coming from the top hit in the photoreceptors and heading off to the brain and out the bottom.
But in fact, light's coming from there.
The retina is put on backwards in an interesting kind of a way, or intuitively backwards at least, so that light shines through all of this stuff before actually getting to the photoreceptors.
Why is that?
Well, there are a variety of possible answers.
One of the answers is -- let's see, the remaining term on the list that I haven't mentioned in that retina list, which is actually at the top ofo the list, it says pigment epithelium.
Up here there would be a layer of what boils down to black gunk coating the back surface of the eye, and the photoreceptors are stuck into that.
That's the pigment epithelium.
What it's there for is -- well, let's suppose that all of you guys are photoreceptors and here I am, I'm a photon and I'm part of this image of the world on the outside and I'm coming flying in, and with luck I'll be absorbed by Mr. Photoreceptor here.
But suppose I don't?
Suppose I hit over here.
Well, photons don't have much conscious life as far as we know.
You, the owner of this visual system, don't want that photon bouncing around anywhere else.
You want it either to be absorbed by a photographers or to be gone.
Because if it bounces and hits up there somewhere it's going to degrade the image, right.
It's going to be like haze across the image if it's not landing where it should to make a nice starp image.
So this black gunk is there to soak up extra photoreceptors.
A very good idea if you are a diurnal animal, an animal who works out in bright light.
Not such a good idea if you're an animal who is nocturnal or running around in dim light.
There what you want to do is catch every photon that you can possibly catch.
So if you're a cat, for example, you do this differently.
Instead of a pigment epithelium at the back of your eye, you have a structure known as the tapetum -- it's actually very beautiful, it looks like Mother of Pearl, but you only get to see that if you take it out of the cat, which makes the cat less beautiful.
But it's a reflective surface.
This is why if you get a cat in your flashlight beam, its eyes seem to glow at you, because the light comes in -- if it doesn't get absorbed by, say, the photoreceptors, it can bounce and an amount of it will bounce back out.
Some amount of light will bounce back out of your eyes.
That's what you see as red eye in an unlucky photograph, for example.
Is that a hand?
Wouldn't that distort the image?
To the cat
Yeah
Yeah, absolutely.
That's why you almost never see cats reading The New York Times.
No, cat visual aquity is pretty lousy.
A cat has different visual desires than you and I do.
The cat's not big on sweating the visual aquity thing.
The cat wants to know, did it move?
Can I jump on it?
Can I bite it?
Is it another cat?
Can we do stuff?
All this sort of stuff.
This sort of thing doesn't require a lot of visual aquity and the cat wants to be able to do it in dim light.
So, we have this notion that cats can see in the dark, which, of course, is not really true -- no light, no vision.
People think cats can see in the dark because cats can see stuff in dim light that you and I can't, and part of the reason for that is that they are willing to let stray photons go where they will.
Because, in fact, I gave a sort of a radical example.
If my hypothetical photon missed my hypothetical photoreceptor here, odds are it's not going to go out to outer Mongolia, it's going to get absorbed someplace nearby, and so the effect will be a blurring, not a complete degradation of the image.
I think the cat can cope.
The cat doesn't mind too much.
By the way, if you're interested in design, taking a look at the design of eyes across the animal kingdom is a beautiful--.
The number of different ways that evolution or the hand of God, you can take your pick, has chosen to solve the problem of collecting light, an amazing array of possibilities.
Once you start -- cats are one thing, but you go off into insects and other beasties and it's great.
Wonderful stuff.
If you want to be an engineer, probably lots of good industrial design there.
All right, let me take a quick trip up into the rest of the visual system here.
This picture that's up on the screen now is mimicking the picture on the upper left of the second page of the handout.
It's showing the connections -- oh, I just remembered.
I want to do a quick demo here before I do that.
Let's go back and do that.
If the retina is on backwards, you got this interesting problem.
So, there are those nerve fibers to the optic nerve.
How are they going to get out of the eye if light is coming from there?
What they do is they run across the surface of the retina until they reach the optic nerve and then they go out up to the brain.
Where are the photoreceptors at the optic nerve?
The answer is there aren't any photo receptors at the optic nerve.
If there aren't any photoreceptors at the optic nerve, what do you see there?
The answer isn't nothing because you're not aware of the fact that there's some big hole in your visual field.
But there is.
You will automatically fill it in because if it's green here and green here and there's no information there, it's a good bet that it's probably green all the way across and your visual system can figure this out.
But it's worth it to prove it to yourself that there is a blind spot there.
Let's see, on the handout -- well, flip to the back of the handout where you got a little extra white space, and put an x on handout here like this, and a little ways away like about like that, put a little black dot or a little dot -- I don't care if it's black.
So the dot is to the right of the x.
Close the left eye, cover your left eye.
Look at the x, hold it so that the dot is straight out to the right and then move the paper back and forth and you will find that there's a spot where the dot completely disappears.
What you've done at that point is move the dot into the optic nerve head -- the place where the optic nerve is leaving the eye.
If you're saying I can't do this, I don't see a blind spot, it's not because you're special and don't have one, it's because you're doing it wrong.
Well, there is an alternative.
I'm given to understand that the octopus retina is not put on backwards in this way and so perhaps you're an octopus.
OK, let's go back to this wiring diagram here.
You can play with this later.
It's cool.
Do things like -- well, if you flip it around you can find that you got one in the other eye, too.
You can also do things like draw a line with a hole in it.
If you draw a line with a gap in it and put the gap in the hole, the line will appear to complete.
If you get really adept at this you can do what King Charles the Second of England was reputed to have liked to do, which is you can look at your friends and put their heads in blind spots and watch their heads disappear.
Charles the Second didn't get to chop off as many heads as his predecessors had, so this was the best he could do I guess.
All right.
How do the axons from the optic nerve feed the visual centers of the brain?
You will recall, from the first lecture perhaps, that everything from the right side of my body on the skin senses gets represented in the sensory homunculus on the left, and everything from the left in the sensory homunculus on the right.
The way to not do this individual system would be to have let's have everything from the right, that's my right eye, my right eye go to my left hemisphere, and everything from the left eye go to the right hemisphere.
That wouldn't be any good.
Isaac Newton figured out without bothering to do any anatomy, Isaac Newton figured out that that wouldn't be any good.
Because I want to look out at the world with two eyes and I want to see one world.
So I'm going to have to do something that brings the information from the two eyes together.
What you actually do is the following.
Everything from the left side of each retina ends up in the left hemisphere, and everything from the right side of each retina ends up in the right hemisphere.
The image on eye is flipped -- simple lenses do that, right.
So I've got a simple lens in that anterior portion of my eye.
That means that the image of you guys landing on my retain has your feet up, your head down and left and right reversed.
So what that means here is if I stare at the person wearing yellow here -- if I stare at the person wearing yellow she'll look embarrassed.
It always works but that's a different phenomenon.
All right, but I'm staring at you but I'm not actually paying any attention to you.
What I'm actually doing is looking at -- my aquity's so bad, it's a black blob right about there.
I assume she's a person -- yeah, she's a person, OK.
Anyway, acquity's very bad.
If you're on my fovea, the woman to the right is in the right half of my visual field landing on the left side of my retina.
She lands on the left side in both eyes, and as a result, she's landing in my left hemisphere.
If I'm looking at you, some arbitrary blob -- there's a blob that just moved over there.
Oh, she's a person, too.
This person in the left visual field ends up on the right retain in each eye and in the right hemisphere.
So there's only one of her, in a sense, up in my brain, and only one of her up in my brain and they're getting together in that--.
And the result is that half the fibers from each retina have to cross to the other side.
That's what this cute little map on page 2 is telling you.
But the important thing not to get confused about is don't tell us on some exam that your left eye goes to your right hemisphere or something.
It's left half of each retina goes to left hemisphere, right half of each retina goes to the right hemisphere.
What it does once it gets there is -- this is a monkey brain, and what you're looking at is the cortical surface here.
That must be the outside surface and this is the inner surface of the hemisphere, if there are two hemispheres.
The one on the bottom is looking at the inside surface.
The colored areas are ones that are important in vision.
I talked at the beginning about the fact that primary visual cortex is right here at the back of the brain, but it's amazing how much of the brain is devoted to vision.
It's very hard to look at in a picture -- well, it's very hard look at, period.
What vision researchers like to do is to make maps like this where you take the cortical surface and flatten it -- it's all this wrinkley involuted structure -- use a variety of techniques to flatten it out like a map.
Then you see that this is the piece that's at the back of the head, the one primary visual cortex.
Then you've got all these gazillion other regions that are important in visual processing, some of them for rather precise sort of things.
Where did mt go?
I think right there is mt, for example, which is important in motion processing.
Lots of work being done trying to figure out what different bits of it do.
Look, let me give you a broad organizing scheme for how to think about this.
It's not perfect but it ain't bad.
From primary visual cortex where very basic information about spots and lines and primitive bits of motion gets pulled out, more advanced processing can be thought of as in two large streams.
One large stream going up towards the parietal lobe is telling you things about where stuff is in the world.
How is the world laid out?
So, my notion that this space is laid out as a big slanted plane in front of me is probably a product of this sort of a pathway.
A second pathway going down into the temporal lobe is very concerned with what stimuli it might be.
So there, if you're a monkey and I spear individual cells, it might be where I would find a cell that was particularly interested in a hand, a monkey hand, or a hairy leg or something like that.
It turns out that the hairy part can be important.
Some of the work on things like monkey hand detectors was being done by one of my professors at Princeton, and actually my TA in intro psych, when I was an undergrad, was a grad student who was working on this, and he reported to us that he was recording -- you go spear a cell.
It's one thing if you're just trying to figure out if it likes spots of light, but how do you figure out if the cell likes something like a hand.
Well, they had in those days drawers full of stuff and you waved all this stuff in front of the monkey and see what makes the cell go.
These experiments in the good old days tended to run for very long time, because this was an anesthetized monkey and the monkey wasn't going to wake up from this particular operation.
You ran the experiment for days and days and days and you stayed up the whole time.
So, Bob Desimone, now a famous visual neuroscientist is up late at night with this stupid monkey with his stupid cell and the cell isn't doing anything.
He's waving everything at it.
So finally, he took off his pants, wrapped a towel around his waist and did a little dance in front of the monkey.
The cell went nuts.
Being a good scientist, he then narrowed this down to the notion that this was a cell that actually seem to be interested in hairy legs, and the hairy part turned out to be important.
I feel free to tell this story about Bob Desimone because Bob Desimone is running around the country with a talk at the moment that has a picture of me in it kissing a wombat.
Don't ask.
It all seems fair to me.
All right, so if you go up into, way down here in the temporal lobe, you might find cells that respond only to something like a hairy leg or a monkey hand or something like that.
If you're hanging out back here in primary visual cortex -- well, actually let's go back and hang out in the retina even.
Look at that, we'll hang out right there.
If you're hanging out in the retina, if you like much simpler needs than this hairy leg detector somewhere up in the temporal lobe, I mentioned before that you might be bringing together a whole bunch of photoreceptors through, say, a bipolar cell to a ganglion cell, to bring together a whole bunch of them to send a signal off to the brain.
But what I didn't mention is that not all the photoreceptors that are hooked up to stimulated ganglion cell are going to excite that cell.
Let's draw some pictures over here.
What actually happens, and I'm just going to mimic this over here, is that you'd have some photoreceptors who are set up to excite a ganglion cell.
Then some other guys around them who would be set up to inhibit it.
The result is that what this cell -- that's a one-dimensional slice and this would be a two-dimensional picture -- the retina's after all a two-dimensional surface.
So what you'd really get is some region of photoreceptors that excite the cell, and some surrounding region where light inhibits the subsequent ganglion cell.
Then light out here does nothing.
So this is the receptive field of this cell -- this is the only part of the retina that this cell over here cares about.
It's going to send its signal off to the brain.
What does this cell do?
Well, among the things that this cell can do is it can tell you I am most excited by a spot that's exacty this size.
If the spot gets any bigger it's going to encroach on this inhibitory stuff and reduce my excitation.
If the spot's any smaller, I'm not going to get all of my excitatory guys excited, and the response will get smaller.
So I'm most excited by a spot of exactly this size.
If I build receptors with different size centers, this guy's only going to be excited by a little spot and I could make one that would be excited by a bigger spot.
So, even at the level of the retina, you can start to get cells that are telling you more than just light on, light off, they're starting to tell you something about the size of things in the visual field.
When you get up to the cortex, you find that now cells come to be interested in things like lines -- not just a spot but a line.
And this cell will way, I like lines in this orientation.
I don't like lines in this orientation.
In fact, I like lines in this orientation only if they move this direction.
So you can imagine that what you start to do early in the visual system is to pull out a whole bunch of little features, if you like, that you can then use subsequently to try to figure out is there a monkey hand here.
Is there a laptop computer or something like that.
Early in the system you've got basic features being pulled out of the image.
Now, this presents a problem.
Suppose that what you want to do -- Let's start with orientation.
Suppose you want to decide that a line is a particular orientation.
How good are you at that?
The answer turns out to be that you are good enough to be able to tell roughly 1 degree increments.
How are you going to do that?
Well you could have a cell specifically designed to look for zero degrees vertical, 1 degree, 2 degree, 3 degree, 4 degree, and you need one of those everywhere in the visual field, and at all different sizes -- a little line that's vertical, a big line that's vertical.
You're going to need an awful lot of cells then.
That's not what we do.
What we seem to do all over the place in the visual system, and in sensory systems in general, is to make comparisons.
That comparisons are very powerful in this game.
So how do you decide that something's vertical?
To a first approximation what you do is you look at whether or not the lift-tilted guys and the right-tilted guys, the cells that like stuff -- the cells that look for something that's tilted a little to the right, cells that are interested in stuff that are tilted to the left.
Are they roughly equally excited?
If they're roughly equally excited in balance one with the other, then odds are that you're looking at a vertical stimulus.
OK?
If the right-tilted stuff is stronger than the left-tilted stuff, so that balance gets tiled, you're going to say that the stimulus is tilted.
It turns out that if you do something to make -- let's say you go off and you make the left-tilted stuff weak by tiring it out -- how do you tire it out, you show it a bunch of left, show it a bunch of left lines, that makes this weaker.
Then something that's vertical's going to turn out to look tilted to the -- you're not going to believe that.
Let me show this to you.
I'll show it to you in -- I'm going to jump to here.
Well, we'll jump here first.
OK.
Here are a very boring picture.
It will be even more boring if I turn out a little more of the lights here.
OK. How you know that something's white or grey or achromatic?
Again, in this case you've got color mechanisms that say how red or green is this?
If the answer is well, it's balanced between red and green, and you also ask how blue or yellow is this?
If it's not blue and it's not yellow and it's not red and it's not green, well, you know what it's probably white.
OK, well, so here let's take this red/green thing and let's make the green side weak by wearing it out.
We'll show you some green.
Now I'm going to show you something white -- the green is weak, the red is strong, the balance goes this way, and you think that something that was once white now looks kind of green.
There, see, isn't that amazing?
They're not convinced.
The ones who are convinced are confused.
Stare at the black dot at the center here.
You just want to stare at the black dot and think about, say, that red dot up at 12 o'clock.
What you're doing is you're tiring out the red side of this red/green equation in this particular case, and just keep staring there, and then I'm going to go boink.
So, the people who went "oh," presumably saw colors.
Let's do that again, here.
What you want to notice is that the color is this opposite color, it's going the other direction.
So the red one will turn green, the yellow one will turn bluish, the green one will turn reddish, right.
So stare at the center, boink.
It works.
That's what's known as a negative after image.
You can just toggle back and forth.
If you look at the black dot on the right and then move your eyes to the left, you'll see the colors show up in the left hand circles, because that after image is essentially cooked onto your retina -- it's there in retinal coordinates and moves with your eyes.
Alright, now let me illustrate to you that this is a ubiquitous phenomenon that shows up for basic phenomena all over the place by turning this back on.
Is that going to be enough light for this?
So, I won't show you the orientation one, even though I managed to get a PhD studying it, but I'll show you the motion equivalent of it.
Here's what you want to do.
You'll still see it over there, except for the guy who's asleep behind the pillar, that's his tough luck.
So, look at this, and we're doing the same game.
We're just taking the game and transferring from the color realm, or the orientation realm, down to the motion realm.
So this is contracting, right?
Stare at the center, that's important.
So stare at the center.
Again, think about 12 o'clock.
All the contours are moving down.
You know that something's stationary because it's not moving up, down, left or right.
But at 12 o'clock I'm wearing out your down detectors, and at 6 o'clock I'm wearing out your up detectors and so on.
The consequences of this evil act of mine will be seen vividly if, when I take it away, what you do is look at my nose.
Of course, what really worries me here is several people's pupils got much larger, which means they think I look better that way.
Now, a word of warning.
Two words of warning.
This is the best demo in the whole class.
It's all downhill from here.
The other word of warning is it's a very salient demo, but it's there for a purpose.
It's really lame on the mid-term or the final when some motion -- this is called a motion after effect -- when a motion after effect question shows up to say oh, that was cool, man, his head was shrinking and stuff.
Because you don't get much points for that.
What you want to know is why this is actually interesting, which I will reiterate by going the other direction since it seems a pity not to, right?
OK, so again, the way you figure out that something is, in this case, stationary is by making a set of comparisons, and Iam using this clever little tool to systematically distort the comparisons that you're making.
By the way, I'm also causing you to make one of these inferences that we'll talk about next week.
By seeing all these various bits of odd motion, you are making the conclusion that my head is shrinking or growing or something like that.
So this is expanding, right, so this is the amazing pinhead version.
Ready?
The original version of this, by the way, is known in the literature as sometimes it's the waterfall illusion, because it gets described multiple times in the literature, but one of them is by a Scotsman named Adams who was staring at a waterfall and then noticed that the rocks on the side of the waterfall seem to be drifting up.
This is a known waterfall in a known spot in Scotland and there's a group of vision researchers who meet there every year and the entry requirement for joining this particular elite group is a bottle of scotch of a variety that they have not tried before, which I think helps make the rocks move in all sorts of interesting ways.
They've never invited me so how would I know.
OK, this is what I'm going to do.
Let me follow up.
So you've seen a negative color after image.
You've seen a negative motion after effect.
I've asserted that these exist in orientation.
You can show this often -- you show somebody big stuff and then medium stuff looks small.
It's ubiquitous.
It shows up all over the place.
I want to show you one more version of this kind of a effect, and that's going to take a certain amount of build-up here.
So the first thing to do is say the vertical and horizontal stripes, do they look different to you?
If they look different somebody raise their hand and tell me what the difference is.
What's the difference?
The horizontal ones are horizontal
Yeah, alright.
In their color or shading in some fashion.
Yeah?
The vertical ones look blurry
The vertical ones look a little -- they do look a little blurry.
Yeah?
The vertical ones -- the black lines look like they're thicker than the ones--
Well, we're not getting a lot of vivid color reports here
The horizontal lines look brighter
Brighter?
Nobody knows what the word color means.
Last try
The horizontal lines are greyer and the vertical lines are black
OK, look, this makes the point just fine.
Never mind guys.
There is no in general consensus here.
I'm about to produce a general consensus.
That's the motivational part of today's lecture.
No, actually what I have to do now is for the next few minutes what you want to do is keep your head upright.
You don't have to stare at any one spot.
You don't even have to your head upright, just keep it in the same orientation.
If you flop all over the place and fall asleep nothing good happens.
So you're going to look at these green vertical lines and these red horizontal lines.
All I need to do here is toggle back and forth between them.
Now you can see the negative after image here.
If you stare at this green -- oh, I should have done that quickly before.
So stare at this green thing for a while.
You get an nice purply red thing?
OK.
So that's just the negative after image.
But we're not interested in the negative after image anymore.
Actually, we are.
There's an aspect of the negative after image that's really cool here, which you may start to notice shortly.
During the blank period, shout out the orientation that you see
[INAUDIBLE PHRASE]
That's a little weird.
Wouldn't you think?
Now that's a little weird because if you stare at these horizontal stripes -- in fact, I should be able to get you back to seeing horizontal.
So, stare at these -- fixate, don't move your eyes around and we'll just stare at that for a minute or so.
Has it been long enough?
Yeah.
Does it look horizontal?
Still looks vertical, oh boy.
Well, here's the get rich quick part of this lecture, assuming that you're unscrupulous, you can use this to make your fortune claiming that you can teach people to be psychic, right.
What's the next orientation I'm going to show?
What's the next orientation I'm going to show?
If you can make your fortune out of that, just send me 10%.
But what's really going on here, what this is actually illustrating, this aspect of it is illustrating, is that you're seeing with your whole visual system, not with your retina all by itself or something like that.
So, each time you look, say, the red horizontal, you're building up a green horizontal negative after image on your retina, right?
And this is building up a red vertical kind of after image on your retina.
If you go back and forth, you build up sort of a plaid.
So think about your retina as having a plaid on it.
Now you don't see it here, but the plaid is -- when you're looking at a blank screen, it's just that plaid that's getting shipped up to your brain, basically.
OK, well what else is happening?
Well, when you're staring at this green vertical thing, up in your brain the cells that are interested in vertical are saying we're getting tired.
Go to the blank part, tired vertical guys and more awake horizontal guys are looking at this weak plaid and they're only seeing the horizontal part.
Now you look at the horizontal, the horizontal guys get tired out -- oh, we can only see the vertical, and so on.
You may ask isn't it about time we saw something interesting here?
The answer is undoubtedly so, but this particular effect takes a few minutes to build up, and so you gotta keep looking at this.
Oh, you've got to ask yourself what the effects ought to be.
What I'm building up here is an effect called the McCullough Effect, named after Celeste McCullough, who unlike most people who discover visual effects, was actually looking for this.
She knew what she was looking for and she knew what she could find.
Now I already showed you that the pattern you're going to look at again, right, it was that black and white pattern of vertical and horizontal lines.
What do you think the vertical lines should look like when we're done?
They should look kind of reddish maybe.
The horizontal should look?
Green
Green.
Let's see, well we might have done this long enough.
This is one of these things where if I do it long enough I can hit absolutely everybody, and if I quit too soon not everybody will see the effect.
All right, here we go.
So, close your eyes, because that will let me get to the right slide, but it will also let you perhaps see that checkerboard floating there in the dark?
Murmur if you can see a checkerboard floating in the dark.
OK, good.
That's that's the negative after image thing.
OK, now take a look at this test pattern again.
How many people think that the vertical and horizontal look different now in terms of their color?
How many people think it might look a little sort of maybe kind of different, but it's not exactly biting me on the posterier?
Alright, that's a better test.
Now, the way to find out whether you've actually got the right effect is without knocking out your neighbor, tilt your head 90 degrees.
So that's cooler.
I'll say a word more about this, but I won't say a word more about this until you've had a chance to get up and stretch for a second.
OK. How many people when they look at these dots still see reasonably compelling colors?
Not many.
How many people when they look at my head still see it shrinking?
Yeah, except for the weird people.
That's because your visual system is continuously re-generating itself.
You go stare at the red thing, you wear out, in a sense, the red detecting chunk of your visual system, and then if you flip your eyes over you find out that you see a white thing is green.
But then you recover from it, right?
Good.
How many people still see this effect?
Tilt your head just to make sure if you think it's there.
A little weak today.
We probably should have gone a little bit longer.
Those of you who are seeing it, and perhaps even those of you who are not -- did I put a test pattern on the handout, I forget?
No, OK.
I'll just bring the Powerpoint back next week.
Not only are you still seeing this some minutes afterwards, but particularly if I don't show you the test pattern a lot you may see it for days and weeks.
The longest report in the literature for the McCullough effect is three months.
The only reason it's only three months is because at that point the undergraduates, who, of course, were the subjects, went away on vacation and the experiment ended.
The evidence suggests that if we locked you in the dark that the McCullough effect would last forever.
We won't lock you in the dark.
I teach this up at Harvard where there are many more pre-law students, and the immediate reaction to this news is half of them consider filing suit.
This will not, in fact, ruin your life in any fashion.
In fact, it'll only be noticeable if you happen to end up looking at a test pattern like this or perhaps serving time in the near future.
But this was something that McCullough was not expecting.
McCullough thought that what she had done here was, OK, you can fatigue color, you can fatigue orientation, well I'll fatigue the cells in the brain and there are cells in the brain that are interested in red vertical lines.
They'll get tired, I'll see white vertical as green, they'll recover, I'll see white vertical as white.
But what happened was you looked at red vertical lines for a while, white vertical looked green and white vertical looked green for like a really, really long time thereafter.
What seems to be going on here is something like this.
Let's suppose that the way you decide that something is white vertical is by looking at the balance of -- well here, let's draw it this way.
You've got red vertical input and you've got green vertical input, and they're being like let's put a little dial here.
If they're roughly equal you decide something looks white.
So I show you red vertical for a while and this thing gets weak and cruddy, and now something that should be producing equal input gets more from there and it ends up looking a little green.
Alright, that's fine.
What happens if this cell gets sick -- it's not going to be one cell, but suppose that this process gets weak in some fashion, not because you were looking at some stimulus, but because you went out and ate the mushroom that you shouldn't have eaten or whatever and this cell just took a hit.
And so now this is kind of a cruddy cell.
Well, there are a couple of possibilities.
One of them is that for the rest of time you're going to end up seeing white things as looking green.
That's not going to be good right.
You don't want that happen.
So if you decided that this was the situation, what you would want to do is change the gain on here.
So if this was previously just equal gain, let's crank this up a little bit so that we can drag that back up to being equal.
That sounds OK. That seems like a clever enough thing to do.
But how are you gonna do that?
How are you going to figure out that you're broken.
How would you know that your visual system was not reporting the truth to you.
Well, I mean one way would be you're saying hey, that looks white to me, and somebody else says that looks green and they laugh at you or something.
Nah, it doesn't work.
No evidence that that works.
What you do is that you seem to have some sort of notion of the statistics of the world, the way the world ought to be.
If the world is systematically different from that, you draw the conclusion that you've got trouble here of some variety or other.
Well not necessary, you don't necessarily draw the -- let me give you a non-troubled version of this.
How many people have been to one of these omni theater kind of movies?
The main reason for going to an omni theater movie is what?
You want to feel like you're moving, right, when you're not moving.
Because all the movies that they ever show you at the science museum are things where you go driving around in race cars, stuff like that.
Why does this work?
You're sitting at this huge screen, and everything in your visual field is moving in the same way.
You know something about the world.
The world on average is stationary.
If the whole world is moving there's a good explanation for that typically.
The answer is not the world, it's you.
What is it that causes the whole world to slide across your retina?
This is what causes the whole world to slide across the retina.
Sit in the omni and the world is sliding across your retina.
Your brain says uh-huh, the world isn't moving, I'm moving, and I think I'll throw up now for a while or something like that.
That's a separate interesting question.
What is it here that could tell you that you're out of whack?
Well, what is the normal relationship between color and orientation in the world?
Typically in the world if you see something that's vertical, what do you know about its color?
That's right, nothing.
So what's going on here?
It's vertical.
It's green.
If it's horizontal it's red.
If it's vertical, it's green.
If it's horizontal, it's red.
You do this for long enough and a little chunk of your brain, rather like the little chunk of your brain that's saying I ate the cupcakes, I threw up, I'd better do something about this.
This little chunk of your brain is saying, oh, there's a correlation between color and orientation.
The only way that happens is if I'm broken, and the only thing to do about that is to adjust the gain on chunks of my visual system.
So, everything looks green.
All the vertical things look green.
Oh my goodness, what am I going to do?
Well I better crank down the gain on the green side, it's obviously too high, and I'm want to crank up the gain on the red side.
Unfortunately, that was a goof.
And it was wrong, so then when you look at a white thing it now looks red because you cranked the gain up too much over here.
How do you recover from this?
Well, if I now show you the opposite one -- what were we doing before?
What was it, green vertical?
If I show you red verticals and green horizontals for a while, that'll push it back the other way.
But that doesn't typically happen in the world.
Simply being exposed to a world where there's no correlation will slowly push you back.
But the reason the effect lasts so long is that the evidence from the real world is kind a weak compared to this artificial situation that I cooked up.
The way I'm thinking about this is like -- here's another way of thinking about this.
Suppose you're measuring average weight of people.
I'm just going to keep weighing people and what's the average weight of people.
It'll bounce around a little bit.
Alright.
This is how much people on average weigh.
Now what happens is the amazing 1,000 pound guy comes into my experiment.
What happens to the average?
Now I start weighing normal people again.
Does it just go -- no.
Weighing regular people it'll take a comparatively long time for me to get back to normal.
So that's what's going on here.
The McCullough effect adapting stimulus is the 1,000 pound guy that's causing you to think that your brain is broken.
You go and re-adjust your brain and now slowly it will get better.
How's it look there?
Not better yet.
How many people have a nice after effect?
Still, nice McCullough effect?
Celeste McCullough would be so proud.
You will notice that I have not said a word about signal detection theory.
I noticed that, too.
There is a chunk in [?
Gleichman ?]
about it.
I might decide to talk about it on Thursday.
Oh, you may also notice that you have lost your cell phone.
If that is true come and talk to me about it because we found it.
One last thing.
How many people know what that is?
I didn't ask you what it was, I just wanted to know if you knew what it was.
How many people recognize this?
Only a few.
OK, now you can shout out
It's a dog
It's a dog.
How many people does that help?
It's head is -- well, wait a second.
There's its head, foot, back paw, it's back, it's sort of sniffing at a ground plane that's like here.
You can store that one away till next week and we'll talk about it.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license, and MIT OpenCourseWare in general, is available at ocw.mit.edu
Good afternoon
Good afternoon
So there I was in my car this morning as the pouring rain started, thinking if I make a dash for it -- I've got to take the computer.
I need the coffee, because otherwise I'm going to fall asleep.
I don't need anything in that bag, do I?
So I took off without the bag.
And it was true that I didn't need most of what was in the bag.
But the lecture notes for today's lecture would have been a useful thing to take with me.
On the other hand, if there was ever going to be a day where I forgot the lecture notes, this is probably the one to do it.
Because I'm going to talk about attention today.
And attention research is what I do for a living.
If there's anything that I should be able to just stand up and lecture about, this is it.
Now of course, that means that you should find this to be the most gripping topic in the entire course, and that you should decide that you want to do this for a living.
Well, when you decide you want to do it for a sort of a living, like a $10 an hour living, you can come and be a subject in attention research in my lab.
I would again advocate that you sign up -- I saw there are still these notes around about signing up to be a subject generally in BCS.
My lab is separate from the BCS business because I'm technically Brigham and Women's Hospital.
But you can sign up with us, too, and we'll pay you $10 an hour to do visual attention research.
What could be better than that?
Is Kristen here?
I don't see Kristen.
Kristen, one of the TAs is also in my lab, and I was going to point her out.
Anyway, send me an email, we'll sign you up.
Talk to me.
We'd love to have you.
And you can do Where's Waldo experiments for $10 an hour.
You think I'm joking.
Let me try to explain why it is that I'm putting an attention lecture in between a sensation lecture and a perception lecture.
It's not terribly typical.
More typically, if people talk about attention, they go off and do it later, after doing sensation and perception.
But why am I putting it in there?
The core reason is that you simply cannot process all of the information that you take in from the world.
You're taking in a vast amount of sensory information.
Your perceptual capabilities -- for instance, those that allow you to recognize specific objects -- are limited.
You cannot recognize all of the objects in the world that you are looking at, all at the same time.
It simply doesn't work.
And so, roughly speaking, there's the situation.
You've got a lot of stuff coming in from the outside.
And you've a box here that does, let's say, let's call this one a recognition box.
And only one thing at a time gets to go in and come out of that box, basically.
So this is like the basic MIT metaphor about drinking from the firehose.
Well, if you're really going to drink from the firehose, it's a very useful idea to restrict the flow in some fashion, and let some of that water just [SPLAT] and get you wet, or whatever.
And so there's a severe constriction, sometimes called a bottleneck -- I think I've got some slides that call it a bottleneck later -- that takes all of this and only lets some of it through.
And that bottleneck is governed -- it's not just random what gets through -- it's governed by mechanisms of selective attention that allow some things to get through and leave other things on the on the floor.
And so if you think of this as sort of sensation and perception -- which is a little bald, but -- then that's why you put attention in the middle there.
Now to motivate this a bit further, let me do a demonstration.
Actually, this is the demonstration of why reading the Tech while listening to my lecture may not be a brilliant idea.
Well, it may be a brilliant idea.
It just depends on your particular goals in life.
I need a couple of volunteer type people who wish to read here.
All right, there's a volunteer person, and there's a pink volunteer person.
Yes, you, MIT person.
But you have to come up here.
So kick a few people on the way by and stuff like that.
You have to come up here and do a dramatic reading.
What I'm going to do is have these people both read to you at the same time.
You're going to read from here, from where it says "Catherine."
And you're going to read from here.
And you're both going to read nice and loudly and steadily.
At the same time, yes.
That's the interesting part.
And what you're going to do is you're going to listen to her -- for, actually, to "her" specifically.
Listen for the third instance of the word "her."
When you hear "her," say "her."
For the third time, raise your hand.
OK?
Yeah, we got this?
Yeah, all right.
This her, not that her
I could tell you my name
That would help.
You are?
Nina
That's Nina.
This is?
Zaina.
[LAUGHTER]
Right.
OK. Zaina or Zena?
Zaina
OK. At least it's not just one letter
My surname's [?
Navarre, ?]
if that helps
No, this is not going to help at all.
Her.
When Nina says "her" for the third time, raise your hand.
OK?
You got it?
You got it?
On your mark, get set, read.
[OVERLAPPING VOICES]
OK, thank you.
All right, that was excellent.
All right, so what was she talking about?
Yeah, something.
No, no, somebody raise their hand.
Hand, hand.
What was --?
A letter, thank you.
That sounds good.
She'd gotten a letter.
Was it a nice letter?
Who knows?
It didn't sound too good.
Her countenance wasn't doing good things.
What was she talking about?
Zaina?
What?
Oh, uh.
She was talking about uh.
OK. Was she talking?
Yes
So what's your problem?
What you were doing was -- so how many people -- well, obviously the hands suggested that everybody could manage to do the task.
What could you pick up from -- Zena?
Zaina
Zaina.
I'm not going to -- otherwise it's going to turn into warrior queen and stuff like that.
What could you pick up about Zaina's speech?
Anything?
No content.
How many people knew she was talking?
All right, so you can pick up something.
What else did you know about her?
The tone she was speaking in
The tone she was speaking in.
If it'd been a male voice, if she had switched to a male voice, you would've noticed.
Anything else, you think?
Was she reading from Heart of Darkness?
Was reading from Heart of Darkness?
No, actually what she was reading from was Lucretius, On the Nature Of the Universe.
A wonderful book.
De Rerum Natura in Latin.
He's a Roman author.
This is sort of the first intro psych book.
It's also the first intro physics book, intro everything.
In those days, you could write a book called On the Nature Of the Universe, in verse.
This is a prose translation.
She was actually reading Lucretius' theory of vision.
And even she may not have noticed that, because it's all about thin films and cool stuff like that
[INAUDIBLE] video
A video.
Yeah, well it's an ancient Roman video.
But only a very limited amount of stuff got in.
So there was a certain amount of stuff that was getting in.
But at some point your auditory system gave up on processing that stream.
And in terms of extracting meaning, understanding the words, it went with Nina, because that was the job.
You can't do both of them.
We'd better let them go sit down.
Thank you for being -- What would have made the task easier?
What would make it easier to pay attention to one and not the other, do you think?
Amplifying one of them
Amplifying one of them.
Yes, if the warrior queen would have just been quiet, it would've been no problem at all
If they read the same thing
If they read the same thing.
No, that's -- that's probably true, but not a deeply interesting true.
Well, for instance if she was male, it would be easier to segregate the two voices.
If we moved them apart further it would be easier to segregate the two voices
If one was singing
If one of them was singing it would've actually probably been easier to segregate them.
So if you change the sort of low level sensory information, it would be easier for you to decide which one to pay attention to.
This is something that happens.
Oh, so if you're sitting there reading the newspaper while you're trying to listen to this lecture, odds are you are missing one of the two messages.
It's sort of dealer's choice there.
But it's also not desperately polite, in case anybody was wondering.
If you want to read the paper, you might as well go somewhere else.
But this happens in the real world all the time.
There's a version of it known as the cocktail party effect.
You go to a party and you're talking to someone, and you hear, typically, what?
Like, your name, over there.
So you do this selective attention thing.
You listen to that conversation.
You seem to be paying attention to this guy who's talking to you, but you're actually listening over there.
The problem is eventually, this guy stops talking.
And you realize, oh yeah, I'm supposed to say something now, right?
I wonder what we're talking about.
It can lead to a certain amount of embarrassment.
Now this happens ubiquitously in sensory systems and across sensory systems.
So for example right now, until I mention it, you are not particularly aware of the pressure of your posterior on the seat.
If I direct your attention to that, you say, oh, yeah, there it is.
It was presumably there all along; I wasn't floating a moment ago.
But until I direct your attention to it, it doesn't rise to the level of current conscious awareness.
And it shows up in vision, because the visual world is far too rich for you to process everywhere at once.
And that's what makes these sort of Where's Waldo problems interesting and fun.
If there was not a bottleneck like this, Waldo man would not have gotten rich.
Right?
Yeah, where's Waldo?
There he is.
Big deal.
Have you found him?
No
Oh look, I have a little laser today.
Isn't that nice?
So does it work?
That's Waldo up there.
So now you say, oh, that's really stupid, because I can't even see him now -- Oh, and we decided to exploit the technology by having it on three screens.
There's no added information there, it's just it was too cute not to do it.
But if I say, where is the elephant spraying a car?
You can find it.
You might have noticed it before if you had been scrutinizing it.
It was certainly visible all along, right?
It wasn't that there was a black hole here before.
It's just that only when you had the desire to go in search for it did you manage to direct your attention to it in a way that allowed you to recognize these couple of objects.
And it's that ability to constrict your processing that's really the focus, at least of the first part of today's lecture.
Let me show you the equivalent of the talking example, but now switch to reading.
What you want to do here is to look at the little asterisks.
And I'll put up two streams of text, columns, one on the left, one on the right.
Nice and big so that you can read them.
But what you should notice is -- keep your eyes moving down from asterisk to asterisk.
What you should notice is you can read one or the other; you just can't read both at the same time, even though they're nice and big.
Right?
It just doesn't work.
It's not a visual restriction.
It's a central -- it's a capacity limitation later on in the system.
So this is by way of an answer to question one on the handout: what's the problem that attention is solving?
Attention is solving this problem of having too much going on.
Oh, and attention is a grab bag term.
I'm going to be talking about visual selective attention.
Attention isn't one thing, like my laser pointer here.
There are attentional mechanisms, selective mechanisms, all over the place in the nervous system.
So when you are attending to the pressure of your posterior on the seat, you are selecting, probably using a different set of neural circuitry than when you're selecting one of these words.
It's the same basic idea, but it's not like there's a single attention box in your brain somewhere.
OK.
Some things, as we saw in that auditory demo, the reading demo, some things escape the bottleneck.
Some things can be appreciated everywhere, all at the same time.
Well, question two is, what is that set of things?
And the answer is not babies.
The answer is that there is a limited set of basic features that can be processed across the entire visual field at one time.
Or, you could do it in auditory space.
There'd be a set of basic features in auditory space, too, that could be processed at the same time.
But I'm going to stick with vision.
So all these babies look alike.
It doesn't take much to figure out that now there is -- da da da, where'd Mara go?
Oh, there's Mara.
If the baby turns green, you do something about it.
Right?
It's a highly salient stimulus.
Or if the baby's head gets squashed, you know.
So they're a collection of simple, basic features, like color, size, orientation, that are not bottleneck limited in the same kind of way.
You can find that if there's a single red thing in the field, you can find it anywhere without having to go hunting around.
Other things that you might think would be pretty obvious are not anywhere near so obvious.
So as you look around here, you may notice that most of these baby heads are upside down and two of them are right way up.
But it's not like the green baby head.
You have to go hunting for upright versus upside down.
Even though that's a very salient thing in the real world, whether or not you're upright, or whether your baby's upright or upside down.
So there are about, by last count -- last count was done by me, as it turns out -- 12 to 18 of these things that seem to escape the bottleneck.
And that's probably about it.
And they are a bunch of simple things -- well, seemingly simple things -- like color, orientation, and size.
Things that you could imagine, for instance, the earliest stages of visual cortical processing doing.
And then there are some other, more elaborate things that also escape this bottleneck.
And they're things like -- well, if you believe my friend Chen from China, this would this would be an example of the importance of topology.
He thinks that the distinction here is that this has a hole and this doesn't have a hole.
The other possibility is that this has line terminations and that this doesn't.
These are the sort of things you can fight about in this field.
But anyway, it's easy to find that among that.
Curvy things among straight things are easy.
Orientation in the third dimension works.
So that cube is pointing up this direction; these cubes are pointing down over here.
That turns out to be easy.
Other examples would include motion.
Though actually, motion makes an interesting point.
It's easy to detect the presence of something, but not so easy to detect its absence.
So imagine the following.
I didn't make a demo of this; I could have.
Imagine you're looking at the ground and there's one little ant moving around.
He's pretty easy to find, right?
Because motion is one of these features that you don't have to go hunting for; it's just sort of there.
On the other hand, imagine you're looking at an ant's nest, and there's one dead ant.
How easy is it to find one ant who's not moving?
Not easy.
So the absence of a feature can be hard to detect.
The presence of a feature, one of these 12 to 18 basic features, can be easy to detect.
Now, how do you actually go about establishing that something is easy to find or hard to find?
I've been doing this in very qualitative terms.
But now let me explain how you actually go about studying this.
What we would pay you $10 an hour for if you show up in the lab.
What we would do is show you a computer screen full of stuff, and ask you a question.
A simple-minded question like, on the next one, is there a tilted line?
And what you would be doing is sitting there with a couple of computer keys.
Bang one key if the answer is no, bang another key if the answer is yes.
Do it as fast and accurately as you can, and we're going to measure your reaction time.
The amount of time from the onset of the stimulus to the onset of your response.
How fast can you do it?
Well, I don't have keys for everybody here, so let's just do it verbally.
Say yes or no as fast as you can in response to these guys.
Tell me, is there a tilted line present?
Ready?
Yes
Ready?
No
Ready?
Yes
Ready?
No
OK, that's pretty straightforward.
What's the next thing?
OK. What you should have heard is that your answers were given crisply, in unison, and it didn't make any real difference whether there were lots of vertical lines on the screen or a few vertical lines on the screen.
So if we were to collect real data and to plot the reaction time in milliseconds -- thousandths of a second -- as a function of the set size -- the number of items on the screen -- what you would get for any of these 12 to 18 items, if you did the experiment right, is an essentially flat line here.
This would be the line for saying yes, it always turns out to take a little longer, or typically turns out to take a little longer to say no, but it's not dependent on the number of items on the screen.
So is there an L, is there a green thing, is there an X among these pluses?
All those things would produce similar looking results where the slope of this reaction time by set size function would be, essentially, 0.
Not all tasks behave that way.
So let's do a different one.
In this case you're looking for the letter T. It can be rotated by 90 degrees left or right, or -- maybe it can also be upside down; I don't remember what I put in.
But it may not be an upright T. But it'll be a T. The distractor items are all L's.
And I just want you to say as fast as you can, is there a T present?
Ready?
No
Ready?
Yes.
Yes.
[INTERPOSING VOICES]
OK, ready?
Yes
Ready?
Yes
You also heard the speed - accuracy tradeoff there.
This is a known phenomenon in reaction time studies, which is, one can respond very quickly if you don't sweat the accuracy things.
And people do that routinely.
When people do that a lot in our studies, we call them bad subjects.
And we don't invite them back.
But what you should have heard there, and should have felt yourself, is that the responses were faster when there were fewer items present.
And that the responses of the group, particularly for these larger set sizes, were spread out.
Why were they spread out?
Well, some people got lucky.
This thing came up and their attention happened to be around here.
Oh look, there's a T. Some people were unlucky -- oh dee do dee dee, oh yeah, there's a T. And some people were trying to psych out the professor and said, there was a yes, there was a no, there was another yes.
I know about this: there's going to be a no.
And they said no without doing anything so boring as to actually look at the display.
So what you get for data in an experiment like this would look much more like this.
As you increase the set size, now the reaction time increases in a fairly linear kind of a way.
The slope on these is quite fast.
I mean, this is 20 to 30 milliseconds, thousandths of a second, for each additional item to say yes, and about twice that amount to say no.
Depending on how one exactly models this, this suggests that you're running through 20 to 40 of these letters a second.
So you're going through it quickly, but you're having to search now.
It's not simply obvious that there's a T there; you've got to go and hunt for it.
Over here you can look for the 5, is another typical sort of task that would produce results like that.
I wanted to say one other thing about that, but now I don't remember what it was.
Oh yes.
What I wanted to say was that the speed of this tells you that you're not looking at the rate of fixation on each letter.
If you're doing this in the lab, you make sure that your stimuli are big enough that you don't have to move your eyes to look at each one.
If you have to move your eyes, your eyes only move at a rate of about 4 per second.
And so if you have to fixate each one of the items before you can tell if it's a T or an L -- so if you used little teeny letters -- this slope would be more like 250 milliseconds per item, not 40 or 50 or something like that.
Attention can move much more quickly than the eyes.
One of the things that tells you is that you can attend where you're not looking.
Something that basketball players know very well.
When you hear that a basketball player has great peripheral vision, what that really means is that he can be looking here and he can be paying attention to his teammate over there, and throw the ball and fake out the opposition.
Because the usual assumption is that you're attending where you're looking.
Most of the time that's true.
But OK, so now I'm looking at this guy wearing red up there.
And he thinks that I'm actually paying attention to him.
But I'm not, actually.
Because of acuity limitations, I have no idea what I'm paying attention to here, but I think it's a woman person, and I think she just moved.
Oh yeah, look, it is a woman person.
I can move my attention away from the point of fixation.
And I can move my attention much more rapidly than I can move my eyes.
Now, the find the red thing among green things is a case where the property of the target is one of these basic features and immediately gets your attention.
The find the 2 among 5's, or the T among L's is a case where everything in the relevant display is essentially the same as far as the early visual system is concerned.
T's among L's, it's a vertical and a horizontal line among other vertical and horizontal lines.
There's nothing in this early processing the tells those apart, it turns out.
Most real world searches are not like that.
In most real world searches, oh let's see, what do I feel like looking for?
I'll look for glasses.
If I'm looking for eyeglasses -- there are some right there, and there's some more.
There's no process early in my visual system, you know, some huge chunk of cortex devoted to eyeglass detection.
it just doesn't happen.
At the same time, I don't search around randomly.
No glasses there, no glasses there, no glasses there, no glasses there.
I'm searching in an intelligent fashion.
Here's how you do that.
Let's do one more basic search.
What you're looking for here is a red horizontal line.
Tell me as fast as you can whether it's present
Yes
Now how you do that is not by having a chunk of your brain devoted specifically to read horizontals.
Oh, remind me later; I've got to check whether you still have a [?
McCullough ?]
effect, speaking of red horizontals.
We'll check that out later.
The way you do that is, you use those 12 to 18 basic features to guide your attention around in an intelligent fashion.
So if you're looking for red horizontals, you've got something that can do red.
You know, give me all the red things.
You've got something that can do vertical.
Was I looking for red horizontals or red verticals?
Well, anyway.
This is a red vertical.
You've got something that can do vertical.
So I've got the red things, I've got the vertical things.
I can do that early on in the system.
All I need is something that will do something like an intersection operation.
And if I were to guide to my attention to the intersection of the set of all red things and the set of all vertical things, that'd be a really good place to look for red vertical things.
Oh look, there it is.
So what you've got is a front end that collects information that can be used to control this bottleneck to guide your attention around, to feed sensible things to the back end of the system.
I think that's sort of pictured there.
And the result is that a search for something like a red vertical line, it's not as easy as finding a red thing among green things, but it's pretty easy.
It's easier than finding a 2 among 5's or a T among L's, or anything like that.
Now this sort of guidance comes in two different forms.
Or you can think of it as coming in two different forms.
There's a bottom-up form that's sort of stimulus driven.
And then there's a top-down form that's user driven by your desires.
Let me illustrate that with a couple more searches for a T. Tell me as fast as you can whether or not there's a T in the next display.
Ready?
Yes
That was pretty crisp.
How did you do it?
Muhmuh.
That's what I thought.
Most people probably found their attention sort of automatically grabbed by this one oddball, which conveniently enough turned out to be the T. And so rather than having to search around, your attention was grabbed bottom-up to this item.
Top-down is based on what you know, or what you've been told, or instructions that you've somehow given to yourself.
So I'm going to tell you, if there's a T in the next display, it's red.
What happened out there?
Oh, that was another -- that was also grabbing attention.
It works in the auditory domain, too.
If we set off an explosion, unsurprisingly, you would notice.
All right, you ready?
Is there a T in this next display?
Yes
Whoever said no was another speed-accuracy tradeoff, try to smoke out the professor who had a yes on the last one and therefore must have a no on this one.
Look at the display!
Anyway, that's not as easy as the previous one.
But if you searched around, you probably noticed, or you may have noticed, that you were searching through the red items.
You're not going to bother searching through the black items if you know the T is going to be red.
Right?
So let us suppose we did an experiment where the T could be either black or red.
And I show you a bunch of displays like this, and I vary the set size, the number of items on the screen.
Measure your reaction time.
Let's suppose that the slope of that function was 30 milliseconds an item.
If that's the case, and half the items are red in this display -- or on average half the items are red -- what's the slope going to look like if I tell you that the T is always red if it's present?
[INAUDIBLE]
Less steep.
Yeah.
Specifically how less steep?
[INAUDIBLE]
Very less steep.
That's not specific.
I want a number
15
15.
Good number.
Right?
If you can eliminate half the items, the effective rate of search is going to be twice as great.
So the slope will drop in half.
And that's exactly what you get in experiments like this.
They work very nicely.
If you have only half the items on the screen relevant, subjects behave as though they are only looking through half of the items.
So by now I have answered question two: what escapes the bottleneck of attention?
Well, there are these 12 to 18 basic properties or features of the world that seem to escape the bottleneck.
We can study this by measuring reaction time.
There are other methods, too, of course, but I was telling you about the reaction time methods.
Oh, I see I put Anne Treisman and feature integration theory on there.
Don't worry about the feature integration part.
That's simply to allow me to a give honor to Anne Treisman who really founded the modern study of visual attention, after having pioneered an awful lot of the auditory things.
The auditory demo at the beginning was a classroom version of what's called dichotic listening.
Typically what you do is put on a pair of headphones, and you would have one stream of speech in one ear and one stream of speech in the other ear.
And you ask questions about, if you're attending through this ear, what can you still pick up through this ear?
Anne was doing those things in the late '50s, early '60s.
Went on in the '70s and '80s to really invent this field of the study of visual search, and is still doing great stuff, now at Princeton.
She was not at Princeton when I was an undergraduate there, but she's there now.
All right, so I answered question three.
And question four I answered by saying -- oh, conjunction search.
That search for a red vertical thing is a conjunction of two basic features.
It's not adequate to know that it's red; it's not adequate to know that it's vertical.
The conjunction of those two sources of information is adequate, is what defines the target.
And you can you use this basic feature information, the basic attributes of the stimulus, to guide your attention around in an intelligent fashion.
So that guidance comes in two forms.
It can be bottom-up stimulus driven or top-down user driven.
All right, so what is that attention actually doing?
Why is it that you need to have this -- what is attention making possible here that wasn't possible before?
Oh look, it says that right there.
Or what were those features doing before attention shows up?
Well, here is an answer to that.
The answer is that you've got all those features.
And in fact, early processes in the visual system seem to cut the scene up into what you might consider to be proto-objects.
But those features are just sort of bundled together with an object.
So before your attention arrives, something like this would be red and green and vertical and horizontal, and it's got points on it, or something.
What attention does is to bind those features together in a way that makes it possible for you to know that the greenness goes with the verticalness here, and the redness goes with the horizontalness.
And those points are arranged, the whole thing's arranged into a plus.
The argument is that, OK, I need attention in order to recognize any given individual.
Before attention arrives on that individual, that person isn't, you know, a black hole in space.
That person is a loose bundle of features.
That attention allows me to bind those features together in a way that allows me to understand how they interact, and what that recognizable feature might be.
So oh, there's Kristen.
Hey, Kristen, stand up and wave.
No really, I was plugging you before.
So if you want to do this for $10 an hour, go find Kristen.
So all right up, now we can make fun of Kristen.
So before Kristen arrived -- no, before Kristen arrived, she was not visible.
Before I attended to Kristen, there was presumably a proto-Kristen object out there that was a bundle of Kristen bits.
Only when I got my attention to her -- even though she'd been visible all along, and I've looked over there a bunch of times -- even though she'd been visible all along, only when I got my attention to her could I bind those features together and make her into a recognizable Kristen.
Let me see if I can illustrate that to you with another demo here.
And the way that's going to work is -- OK, so what you want to do in the next slide is to look for red verticals again.
You ready?
So tell me if you find a red vertical
Yes
Yeah.
In fact, you might have noticed there are two of them.
Very easy.
What's the point?
Well, this is a standard guided search kind of thing.
Give me all the red things; give me all the vertical things; look at the intersection of those two sets, and oh lookie, there's two red verticals up there.
Now what I'm going to do is to simply take the horizontal here and jump it up to the middle of the vertical bit That's why this is in this sort of odd arrangement.
I'm going to jump it up here.
So I'm going to make a plus, like those pluses that we just saw.
The reason for doing this is, I'm going to keep all the same pixels on the screen.
Right?
I'm just going to rearrange where the reds and greens are.
And of course I'm going to change the location of the red vertical, because it's really boring if I keep it in the same place.
But you're looking for red vertical again.
Ready?
Yes
Who said no?
I said woah
Oh, woah.
OK. Woah's good, woah is good.
Particularly by the time it says find the two red vertical lines.
Anyway, you should have found both of them.
Let's let's check intuition here.
How many people vote that it was easier to find the red verticals when they were in pluses?
How many vote that it was easier when they were ripped apart?
That is the correct intuition.
Actually, I think I put the data -- I think I realized earlier than I put half the data on a slide.
This is the data for looking for the pluses.
Quite steep slopes of about 50 milliseconds an item here and about 140 here.
Just looking for the red verticals when they were in the disassociated pluses would have been down here, with a slope of about 10.
But I somehow left it off the slide.
Why is this?
Why are the pluses so much more difficult?
The answer is that before attention arrives on the object, these two pluses are essentially the same thing.
They are red and green and vertical and horizontal.
And without attention, you just don't know the difference between them.
This thing, this square has red and green and vertical and horizontal in it, but it's in two objects.
And so since you direct your attention to objects -- to things that are objects; I've got too many "to"s in there -- this is not a problem in the way that these guys are a problem.
In fact anything that you do -- I don't think I brought the demo, but anything that you do to make this less like a single object makes the task easier.
So if I was to put a little shadow on here, so that it would look like this thing, the vertical piece was sticking out in front of the horizontal piece, it would get easier.
Because now you could direct your attention separately to different planes in depth.
So attention is directed to objects, and objects are available ahead of time as just sort of these loose configurations, constellations of features.
Once attention gets there, they get glued together into recognizable objects.
All right.
So what happens when you move away from an attended object?
That's not a unreasonable question in this framework.
So let's see.
I need -- Rachel.
There's Rachel.
I thought I recognized her.
All right, I have now recognized Rachel.
Limited number of people who I actually recognize by name in here.
And they come to regret it.
But anyway, all right.
So she was here all along.
I happen to have attended to her and bound Rachel into a recognizable Rachel object.
I now, without moving my eyes in fact, I'm attending elsewhere.
And somebody's up there, again, my peripheral vision's lousy, but I can see that somebody was moving up there.
They waved a piece of white paper a moment ago.
The question is, when I moved my attention elsewhere, what happened to Rachel?
Did she remain bound, or did she collapse into Rachel bits again?
[INAUDIBLE]
What?
She collapsed into Rachel bits.
How could you tell?
[INAUDIBLE]
That's why I was deliberately still looking at her, to avoid the issues of blur.
But the way to do this is not to continue picking on Rachel, but the switch to dancing chickens here.
There we have -- you can tell we're back in the realm of my artwork.
Oh, I like this, with the chickens on three screens.
This is so good.
Anyway, I like those a lot.
Now, so you know you know what you're looking at here.
You're looking at a bunch of chickens, right?
And they're doing this little leggy thing.
You would think that, having recognized that there's a bunch of chickens there who are doing this little dance, that if one of those chickens fell apart into chicken bits, that you would notice, right?
Seems reasonable.
How many of you noticed?
Ooh, ooh, very slow group here.
It should be -- how many chickens are there here, about 20?
It should be about one in 20 of you happen to be -- you have all seen that already.
So one of these chickens fell apart.
Well, if you think, quite apart from the fact that the artwork is a little lame, the implications are non-lame.
The implication is, all right, I'm looking at you guys.
I think I'm looking at a bunch of humanoid life forms.
They're moving a little bit, stuff like that.
And you would think that if one of you just went to pieces here, that I would notice.
The data strongly suggests that that's not the case.
That I would eventually notice, as my attention roves around the room, if it turned out that, oh my god, not only has that person not dozed off, but her head fell off, I would notice that and react with according shock and amusement.
The way this experiment is actually done is not with the cute little dancing bits.
You'd be looking at a screen like this, and you'd hear, beep.
And the question would be, is there a destroyed chicken?
Yeah, it's there, right?
Beep.
Beep
Yes
Beep
No
Beep
Yes
And so on.
You can do it.
It's not a difficult task at all, particularly with a few big chickens.
But you have to search.
You have to search through the chickens each time.
And you're no better with a display that's got the same fixed number of chickens up there all the time, compared to a display which has de novo chickens popping up out of nothingness each time.
Oh, the feet are moving around.
for the demo, why are the feet doing this little chicken dance?
Remember I said that motion is one of these things you can pick up automatically?
If you don't have something like the little moving feet, then when you have a chicken fall apart - boink - the movement of the contour, compared to all the ones that aren't moving at all tips you off that there's something there.
And that tells you that motion's important, but it doesn't tell you the interesting fact that you're not aware when an otherwise coherent object falls to bits.
By the way, it turns out you're also not aware when previously incoherent material coheres into a chicken.
We did the classic chicken soup experiment.
We had a screen full of chicken bits like this, and you heard beep, and you had to figure out whether or not there was now a chicken present.
And you had to search for that, too.
So chickens emerging from the chicken soup, which you might think would be striking, don't turn out to be striking either.
All right.
Well the chickens are kind of ugly and complicated.
How bad is this problem?
So let's get basic here.
No more trying to fool you.
Well, of course I'm trying to fool you.
No more dancing around chickens, and then oh, did you see -- after the fact I ask you whether you saw something that fell apart.
These are what?
Red and green dots.
If you weren't sure about that, it says so at the top.
All I'm going to do is, I'm going to cue one dot -- I don't care about any of the other dots.
All I want to know is, did that one dot change color?
Say yes or no.
Whoops.
Where'd it go?
[INAUDIBLE]
Well, the answer turns out to be no.
This is such a great exercise in applied statistics, right?
How many -- he can't really be -- he said the last one, so no
Yes
Oh yeah, but he can't possibly be doing three in a row, right?
No
That does turn out to be a no.
Look, you can hear people going both ways.
People are terrible at this.
They're just barely above chance.
And the barely above chance is consistent with them sort of sitting on two or three dots.
Because you're not just doing a couple of these, you're doing hundreds of these, for $10 an hour.
So you can sit on a couple of them and say, if I get really lucky and he cues the one I'm looking at, I'm going to get this right.
And if he doesn't, I'm clueless.
I mean, it's red and green.
It doesn't get more basic than that.
Yup?
I have a question.
Do people's reaction times change?
Because red and green, they have the same after color, or afterimage
They'd better not have the same after -- they have the opposite.
Yes
No, no.
But the opposite of red is green, and the opposite of green is red.
So if you do yellow and blue or something else --?
Well, yellow and blue are also opposite in the same sense.
But it doesn't matter.
The color does not matter.
In fact, we can do another one with different colors.
Look at this new.
More cool colors.
But maybe I was just being nasty to you.
Because there were a lot of dots up there for you to choose among.
So I'll tell you the relevant dots.
What I'm going to do here is I'll ask you about the color of specific dots.
I won't change them.
I'll just put them up there and ask you about particular dots.
And what I want you to do is tell me the color.
So if I say, what color is that dot, the answer is --
Purple
Good.
If I happen to cover it up with a black blob, tell me what color it was before I covered it up.
OK?
Ready?
All right, here we go.
You'll see how this works.
Where'd it go?
There we go
Red.
Yellow.
Blue Green.
Green
Good.
See, you're not -- I put this in because at this point you might be sitting there saying, I'm so hopeless!
And I wanted to prove to you that you're not.
Well, you are, but not that hopeless.
All right, ready?
Purple.
Red.
Blue.
Yellow.
Red.
Green.
Yellow
Ooh, a few people actually got it.
A bunch of people did the, urp.
But yes indeed, that was yellow.
It was cued before, so we know you paid attention to it.
But it was cued about five items back.
And so you'd paid attention to it.
It didn't take much binding to say, that's yellow.
You'd already done all the work on it.
Five blobs later, by the time your attention is somewhere else -- it wasn't invisible during that time, right?
You don't really know what it is.
All right, try this
Red.
Green.
Red.
[MURMURING]
A couple of people caught on.
He changed it.
This is what happened here.
Whoops, not that way.
Go back.
OK.
So this makes a useful and important point.
So, red.
While your attention was diverted, I changed the color.
Why is that important?
What that tells you, with a very basic sort of stimulus, is that the following ought to be true: I attend to Rachel, I attend away.
While I've attended away, Rachel is replaced by a kangaroo.
I am now asked, what was there?
I say, you know, it was Rachel.
The fact that, even though, you know, still visible in the visual field and everything, until I attend back, I would simply not know that something had changed there.
So in fact, if you're worried that -- the trick here, obviously, since there are 300 of you or so, you want to convince me that you're paying attention in this class, you draw my attention early in the class, and then you subtly sneak out.
And presumably I think you're here attending the whole time.
Because how often do I get back to each individual person?
Well, actually, it's not that good.
Because at 30 to 40 people per second, I can get back to you pretty quickly.
So forget it.
But don't forget the basic point here, which is that you're only aware, you're only updating your knowledge about the world, through this narrow bottleneck of attention, for the current object of attention.
Everything else, you're basically working on your hypothesis based on the last time you checked up on it.
So here is actually what the data for an experiment like this look like.
So if you didn't pay attention to the colored dot, right?
If I never asked about it at all.
Here's chance.
50% in this particular experiment.
Because this is a two color version of it.
Is it red or is it green?
You've got about a 50-50 chance of getting it.
You do a little bit better than that.
If it was recently cued -- if I just asked you whether it was red or green -- you do pretty well.
But as soon as it's four items ago, or eight or 12 ago, you're back to being pretty pathetic.
So you don't keep a good record of this.
You're only updating in the current object of attention.
This suggests that your memory is pretty small here.
We'll talk about memory more extensively later.
But let me illustrate that your memory is actually fairly small.
Here what we're going to do is, I want you to remember these guys.
Got them?
OK, take them away.
Are these the same?
No
OK, well your memory isn't that small.
That's good.
How about these guys?
No
No, no, no.
This is a new set.
[LAUGHTER] Ready?
Boink
Yes
Whoops.
Sadly, I can't remember.
Remember these
[INAUDIBLE] They look the same, don't they?
Yes
OK.
So, well.
How about these?
No.
No, this is a new set
Yes
Yes, something changed.
So this time I transposed the red and the yellow.
That's a little more difficult, because I didn't introduce a new color.
How about this?
Yes.
Yes
People aren't quite sure.
The answer is that the capacity of this sort of memory is about four.
And some of you will have gotten the fact that there was another transposition, right?
Of the yellow and the green?
Whoops.
The yellows and the greens.
Yeah.
The yellow and green guys are -- whoops!
-- switching there.
Some of you will have gotten and some of you will have not gotten it, because some of you were sitting on the right four and some of you were sitting on the wrong four.
But it's only about four.
Four what?
It turns out to be four objects.
Look at this.
Tell me if anything changes.
So here we have at least color, shape, and orientation going on
Yes
Yeah, most people will know here that the red thing flipped from pointing up to pointing down.
That would seem to suggest that you can keep track of 12 things, because there are four colors, four shapes, and four orientations.
But if I spread those out across 12 objects, you'd be very bad.
It's that you can keep track of about four objects.
You can keep track of multiple features of each of those objects, but it's only about four objects that you can keep track of.
Now let's see.
How are we doing in question land?
OK, so the answer to question six, at least to the first part about it, is that the objects don't seem to stay bound.
That you need to continuously update the visual world in order to have some idea of what its current state is, and that you're only updating the current object of attention.
After a brief break, we will establish what the Sistine Chapel has to tell us about that fact.
But those of you who wish may study this image for the next couple of minutes or so.
And everybody else can just sort of stretch.
And then we'll come back.
While I apologize to Rachel for picking on her.
You're not traumatized for life or anything?
OK, good.
[?
[CROWD NOISES] ?]
Have you seen this video they have where it's a bunch of people bouncing balls to each other?
Yeah.
That's now gotten to be so common that I'm not using it.
[PRIVATE CONVERSATION]
Do you know who did that?
Yes, Dan Simons.
Then at Harvard, now at University of Illinois.
I will describe a different Dan Simons experiment in a minute.
OK, let's get back together here.
All right, to briefly review.
the story I have been developing thus far is that even though you are looking at this scene from the Sistine Chapel, and this is the expulsion from Eden, there's Adam and Eve, and this very cool snake.
And there's Adam and Eve getting chucked out, with the angel poking them in the head and stuff like that.
Even though you are looking at this, you know what you're looking at, that at any given moment the only thing that's really coming through from the world to recognition is whatever is currently being fed through the bottleneck, the current object of attention.
And that maybe three or four objects, the recent status of three or four objects is currently held in this visual short term memory.
The implication here is that I could change this scene and you wouldn't notice.
So let's find out.
What did I change?
[INAUDIBLE]
I need a hand or two here.
Yeah, sure, what?
[INAUDIBLE]
Oh, the fig leaf.
The fig leaf, yes.
The originator of change blindness, which is what this phenomenon is known as, is Ron Rensink, now at the University of British Columbia.
And he refers to what he calls "areas of interest."
If you change something that people are paying attention to, they notice that.
But of course I knew that.
And so how many people picked up the other three changes?
[INAUDIBLE]
Oh, some.
We have a few people picked -- what did you get?
I can't hear you
[INAUDIBLE]
The stick thing.
And what?
Sorry?
[INAUDIBLE] Something showed up at the top that's funny.
The stick thing moved, and something showed up at the top that's funny.
So now with that information, we can go -- whoops
Right there
You got the stick.
See, the reason for the blank is the same as the moving chicken legs, which is that you don't want to have motion transience giving stuff away.
But if you have motion transience -- do do do do do -- you would think that if you were in the Garden of Eden and the branches were moving from tree to tree, or for that matter Eve's foot was moving to Adam's body, you would notice.
But if you're not attending to it, you don't notice.
So this is part of a large set of phenomena that come under the general heading of change blindness.
At the break, somebody was reminding me of one that you may have seen because it's made it onto Nova and things like that.
Done by Dan Simons, where you're watching people apparently play a weird game of basketball in front of the elevators, it turns out in the psych department at Harvard.
And while you're doing that, a guy in a gorilla suit -- actually, Stan reminded me, a woman in a gorilla suit.
It's hard to tell; she's in a gorilla suit -- walks in, walks into the middle of the game, waves, walks out.
And then afterwards you ask -- oh, and you're doing a demanding task.
You're supposed to count how passes there are, or something like that.
And you're asked, did you notice the person in the gorilla suit?
Well, first you're asked, did you notice anything weird?
Eh, no, very boring.
Notice the person in the gorilla suit?
Yeah, right.
What person in a gorilla suit?
Show them the video again.
Oh my -- Another great Dan Simons experiment was done when he was at Cornell, actually.
You're on the street in Ithaca, New York, and some guy walks up to you and asks you for directions.
Actually it's Dan Simons walks up to you and asks you for directions.
And so, since you are a nice person, you start giving Dan directions.
Now you're standing there on the street and, who knows why, but these two guys with a door are carry a door down the street.
And they walk between you and Dan.
Which is kind of rude.
And then they're off down the street somewhere.
And the question is, do you continue to give directions once you see Dan again?
Of course, the real question is, did you notice that when the door went by, Dan Simons ducked down and left with the door, and his then-student Dan Levin popped up in his place?
And it's a different guy.
50% of the subjects in this study kept talking.
A surprisingly large number of these, on being debriefed later, claimed to have noticed a change.
Which is a little strange, right?
I'm talking to this guy and the door, and now I'm talking -- there's another guy here, but what the heck?
He probably wants the answer to the same question.
I don't know what that's about.
But the important finding there is that 50% of the people behaved as though they hadn't noticed the change from one person to another, who they were talking to.
What's going on here?
Now people aren't completely stupid.
The experiment has not been done, but we kind of absolutely know that if I'm talking to Dan Simons, short white guy, and now the door goes through, and a tall black woman is standing there -- hm, you know?
Probably that's, again, the sort of front-end stuff that people tend to pick up on.
But if what you're doing is, I don't know this guy, but I've got a sort of a model of this guy.
I'm talking to kind of a short, white guy person.
And da da da, I'm still talking to a short, white guy person.
It's not the same one, apparently, but that turns out not to be a problem.
This has given rise to a notion that perception is what Kevin O'Regan has called a grand illusion.
That the only thing that you actually see is the current object of attention.
That I think I'm seeing all of you, but all I'm really doing at the moment is paying attention to the guy with the grey stripe on up there.
Yeah, there he is.
And now that he's riveted my attention by waving at me, the rest of you are just not there.
You are just some sort of grand illusion floating around in my head.
Now in some sense, that's correct.
That what you are seeing is a creation -- the burden of the lecture next time will be to say that you're always seeing a theory about the world.
You're not seeing the world directly.
You're always making an interpretation, your best guess about what the stimulus means.
And all the evidence I've been showing you for the past hour or so suggests that you're only updating that theory through this very narrow bottleneck.
So in some sense, you are only seeing this creation of your mind, and the only object that you are currently updating is the one that you are currently attending to.
But to call the whole thing an illusion, it seems to me, misses an important aspect of the experience.
Let's take a very old example.
The French philosopher of the, I'm thinking early 18th century, whose name I will now proceed to misspell.
Does that look -- any good philosopher sorts?
That about right?
Condillac, I believe is how you pronounce it properly.
But anyway, Condillac wrote a number of very interesting things about sensation and perception.
He's most famous for his statue.
His statue that he proposed as an entity with no senses at all.
And he asked what would the mental life of this statue be?
And argued that, in the absence of any sensory input, there would be no mental life.
And now, he said, let's imagine opening up, I think he opens up the statue's nostrils.
And argues that the entire mental life of this statue is now the smell.
Whatever, I think he waves a rose under it or something like that.
But a little further on he has a different example where he says imagine, you're in a dark -- a dark chateau, I believe.
And it's completely pitch black, because of these heavy curtains.
And it's morning, and you throw open the curtains.
If it were the case -- this is not what he's saying, but if it were the case that all of vision was nothing but a grand illusion, you only saw the spotlight of attention, this one thing that you're attending to at any one moment, your experience of this brand new scene ought to be like sort of a weird paint brush.
Initially, I don't see nothin'.
Because I haven't attended to anything.
Now I attend to an object.
And now this person, object, is the only thing in the scene.
And boom, boom, boom.
And I slowly fill you in.
That's not the impression you get ever when you see a new scene.
You may not know what you're looking at, but you see something everywhere instantly.
And the grand illusion thing misses the fact that you're somehow sensing something about the entire visual field all at once.
Let me offer a way of understanding that that will then tie back to the visual physiology that I was talking about in the last lecture.
Here's the idea.
Early in your visual system, you've got the processes that, sort of a big river of information that tells you about those 12 to 18 features or attributes that you can get out -- these are eyes.
This is my drawing again.
So from your eyes, you've got this big flow of information up into your brain.
And at some point, it hits this bottleneck that's taken care of by attention.
Object recognition, the ability to tell that that's a branch, that that's a snake, and so on, only one object at a time can go in and come out and rise to the level of some sort of perceptual awareness, populating your visual experience.
And that bottleneck is guided by these collection of basic features that you've got.
If you know you're looking for red stuff, you set these settings for red.
And maybe vertical, and big and moving and so on.
And so you can regulate what gets through here.
And only the one thing at any one time is getting up into there.
And so the current object of attention gets to rise to awareness, and you know what you're looking at.
That's the story that I've told you to this point.
That's the story that gives rise to the notion that everything else in the visual field is some sort of an illusion.
But look, when I was doing that red and green dot thing, it wasn't that you didn't see the other red and green dots.
They were there.
You just somehow had a very impoverished ability to tell me anything about them.
And a way to think about that is to propose that there's another pathway, another big fat river of information about, say, these 12 to 18 attributes, that isn't limited by the bottleneck.
But that it doesn't let you -- it's not a cheat.
This doesn't now let you go and recognize objects everywhere all at once.
It can only do a few things.
It can sort of give you the statistics of the world.
You know, I'm looking out at you guys and I'm seeing a sort of texture of people amongst purple.
And that sort of impression of purpleness, of a tilted plane, is the sort of thing that you might get out of this big, broad, unrestricted, nonselective, as it's labeled on there, pathway.
There's evidence that you can get a little bit of semantic information.
Semantic means the meaning, when you're talking about language, it's the meaning of the utterance, let's say.
When you're talking about vision, it's the meaning of the stimulus.
So I might get the notion that I'm in an enclosed space.
This pathway by itself is not going to tell me what enclosed space I'm in.
But I'm in a space.
There's a tilted surface there.
And so on.
But this is going to give me, that broad pathway is going to give me the feeling that there's something happening everywhere.
And this pathway is going to tell me what's happening specifically here, now.
And between the two of them, I can build up an idea in my head of, oh, I'm in 10-250.
I'm talking to this bunch of people, some of whom I know by name, some of whom I recognize because they've been here before, and so on.
And I can keep updating that 20, 30 times a second through this pathway.
And I can keep experiencing something, that sort of wallpaper of the world effect, through this other pathway.
Now that ties back, it might tie back to things that we talked about before.
If you remember the idea that you can broadly cut visual processing, visual cortical processing, into two big pathways, a what and a where pathway.
A what pathway going down into the temporal lobe, and a where pathway going up into the parietal lobe.
This selective pathway, this thing that only does one object at a time, would then be mapped onto the what pathway.
What am I looking at, what am I attending to right now?
If you were to lesion that, if you were to lesion it, or you were to have damage to the temporal lobe of your brain, you might well end up with an agnosia.
That's not a term that ended up on the handout, so you want to write that one down.
An agnosia is a failure to know, if you like.
To know what something is.
So an agnosic, if you have a person with a fairly global agnosia, visual agnosia, they would be able to say, yeah, I'm looking at a bunch of objects here, but I don't know what they are.
Here's this object.
It's sort of orange.
It's got orange and brown and white blobs on it.
And it's got this very long part, and there are these four pointy things coming off the bottom of it.
I've got no idea what that is; maybe it's furniture of some sort.
You'd look at it and say, that's a giraffe.
An agnosic would be able to tell you about it, but not know that it was a giraffe.
Smaller lesions produce rather specific agnosias.
There are reports in the literature of agnosias specific to, say, fruits and vegetables.
More common is a form of agnosia called prosopagnosia, which is a specific inability to recognize faces.
You know that it's a face, it's got two eyes, it's got a nose and mouth.
You don't know who it is.
Small lesions down in that pathway can produce that sort of damage.
That would suggest, then, that the other pathway ought to be mapped onto the where pathway.
And if you get bilateral damage, for instance, to the parietal lobe, you can end up with a disorder known as Balint's syndrome -- might as well write the word down here.
Named after Balint -- that has as one of its properties what's called a simultagnosia.
This is a situation where you can recognize an object if you can get your attention on it.
But that's the only thing you can respond to, in some sense.
It is as if the grand illusion theory was really right, that you can only see the current object of attention.
So you do something like this with a simultagnosic, say, what's that?
Draw his attention to it.
That's a book.
OK, what else have we got here?
OK, what's that?
That's a cell phone.
What's that?
That's a cell phone.
Anything else?
No.
What's that?
That's a book.
What's that?
That's a book.
Anything else?
No.
So one object at a time.
As if the where of the world had disappeared.
If you get damage -- we'll talk about this more later in the term -- but if you get damage to the parietal lobe on one side, particularly on the right side, what you can end up with is a disorder known as neglect.
It comes in a variety of flavors, again depending on the particular lesion.
But the characteristic is, you ignore the contralateral, the other side.
Now that can be the other side of space, so that if I'm a patient with a right hemisphere parietal lesion and I'm looking at MIT volleyball here, everything in the left visual field, I would simply ignore.
I would behave as though it did not exist.
If I took away everything else and put a stimulus in my left visual field, I could show that the patient could still see it.
But with a full visual field, he behaves as though there's nothing there at all.
Patients with neglect will do weird things, like -- they're in the hospital, typically, because they've had a stroke.
You give them their dinner.
They eat everything on the right side of the plate and leave everything on the left side of the plate.
Why?
Because they didn't like the mashed potatoes?
No.
If you rotate the plate, they'll eat the stuff on the other side of the plate.
It's as if it just didn't exist, in some fashion.
Now, you'll remember the parietal lobe is also where you get the representation of the body surface, and stuff like that.
So neglect patients can also be patients who neglect one half of their body, and deny that part of their body is theirs.
This is a little easier to understand if you figure that the stroke might well have also knocked out the ability to control that side of your body.
So a stroke on the right might leave you paralyzed on the left.
But you can end up with situations like one described, I think, by Oliver Sacks in one of his books, where a patient is saying, "This is a cheap hospital.
This is a really cheap, lousy hospital."
How do you know it's a cheap, lousy hospital?
"Because they're doubling up on beds."
What you mean they're doubling up on beds?
He says, "Look at that leg.
That's not my leg."
So you can get, this is somebody looking at their own leg and denying that that leg belongs to them.
That's another aspect of neglect.
OK, what I'm going to do next time is to talk about the way in which you make hypotheses about the world.
The following content is provided by MIT open courseware under a creative commons license.
Additional information about our license, and MIT open courseware, in general, is available at ocw.mit.edu
Good afternoon.
In response to public demand here-- let's see how many people do you-- again, the colors may have faded by now, but check it when you tilt your head.
The effect changes.
How many people still have some sort of an effect?
It's really nice to know that this course has some lasting impact on people.
Maybe we should try posting the Powerpoint on the web, because then if you want to build up your own McCullough effect you can do this and entertain yourselves.
If you have been following the story thus far, you should basically have the idea that at the front of the visual system, or of any sensory system, you've got all sorts of information coming in.
That the job early processes in the visual systems do.
The sorts of things that the early parts of visual cortex do and say, oh look, there's a little line at this point in the visual field.
Oh look, there's a dot there, and a line here, and it's moving like that.
Little bits of information all over the place.
Too much of it for you to handle.
So, last time I talked about this bottleneck of attention that allows only some of it through to processes that would do things like, say, recognition.
And, then, I'm going to run out of places to draw.
We'll take a little detour here.
Somewhere up here, you're going to get perception.
And the job of today's lecture is to convince you that that percept, your current understanding of the world, is always the result of many, many inferences.
Many, many guesses of what the nature of the world might be.
Because, not only is there too much information coming in, there's also too little information coming to specify exactly what's going on out there in the world.
Consider just a couple of the problems -- I think are they on the hand out?
Couple of them are-- well, one of the obvious ones.
It's the world is 3D.
Your job is to figure out what's going on out in the world.
The input is inherently 2D.
That retina that you've got is a 2D surface.
So, if you are seeing 3D, you are recovering that 3D information from essentially 2D input.
You're collecting information about light intensities.
You don't care about light intensity.
You care about surface properties in the world.
So, everything patch that you're seeing -- if you're looking at this spot right here, what you're seeing is the product of the surface, the properties of the surface, and the properties of the illuminant, the properties of what's lighting it up.
You don't care about the properties of the illuminate.
You want to recover just the properties of the surface.
So how do you successfully ignore the properties of illumination?
We'll say a little bit about that.
The world you're looking is an essentially stable world.
I mean things move around in it, but the whole world doesn't jump around.
But, you're looking at it from an inherently unstable vantage point.
You're moving around.
And more to the point, even if you're salk still, the way you're looking at the world is you're moving your eyes around.
Try this for a moment.
Look at the lower left hand corner of the screen.
Well, actually that might be a little large, so look at the lower left hand corner this McCollough test pattern.
Look at the lower right hand corner of the McCollough test pattern.
Did the McCollough effect test pattern jump when you did that?
No.
It didn't.
Why didn't it jump?
Because when you're looking here-- do I have a laser pointer today?
No.
Oh well -- when you're looking here, the bulk of that square is to the right of fixation.
When you're looking here, the bulk that image is to the left the fixation.
So it's in two different spots on the retina.
If I put something here, and something here, on your retina, it'll look like it's moving.
Why didn't that look like it's moving?
The reason is, that when you tell your eyes to move, you send a copy of that command, in effect, to visual centers of your brain saying, look, I just told my eyes to move, kindly ignore the resulting smear.
In fact, I want you to do two things.
I want you-- you, your visual system-- I want you to shut down during the course of the eye movement, and I want you to compensate for the fact that everything is been displaced.
You can see what would happen if that was not the case, by taking your finger, and poking your eye.
On your eyelid, you should try this, because it's more interesting if you actually try this, it if you wiggle your eyeball.
You can do it slowly, or you can do it quickly.
Look at me, and poke your eye ball around.
You will notice that all your friends look funny.
But, you'll notice that things are jumping around.
Why is that?
Well look.
Millions of years of evolution did not provide you with a mechanism that said, I am now going to poke my eye.
Please send a copy of that signal to the visual centers of the brain, saying to cancel that out.
There's no cancellation signal here.
And so, you see the image moving around.
You don't see the image moving around when you move your eyes normally, because you're compensating for it.
All right, so you're collecting all this information.
You're doing your best to register it in a stable kind of a way.
Oh look at that.
It also says I'm going to demonstrate the vestibular ocular reflex.
Let me do that, just because you might as well get to use your fingers some more.
You also want the world not to jump around too much when you're moving your head.
One of the things you do very reflexively is, if you rotate you take your head one way, your eyes counter rotate the other.
That's a very quick reflex.
If you want to see how quick it is, try this.
Hold your finger out in front of you.
Look at your finger, and move your head back and forth, and just keep your eyes on the finger.
No problem, right?
You know you can do that.
Now, at the same speed, move your fingers, and try to keep your eye on the finger.
Can't do it.
It doesn't work, because that tracking movement -- there are more neurons involved.
It's a slower process.
The vestibular ocular reflect is a very short latency kind of a reflex designed to keep the input relatively stable.
So, you take all that lovely input in, you got all these little bits information all over the place, and then you've gotta make your best guess about what it is that you're looking at.
The reason that this is an interesting picture, it's in the book, by the way, so when you can't see here, you can go and study it in the book, until you can see the dalmatian dog that really is there.
How many people can see it now?
Oh, we got most of them.
His head is right above my finger, front paws, back paws.
He's on a road sloping from lower left to upper right.
The point of a picture like this, is that it's slows down the process of inference enough, that you can sort of feel it happened.
Normally, when I look out at you, for instance, my visual system kicks up one, and only one, interpretation of what I'm looking at, so rapidly, that I never notice all the work that's involved.
The purpose of this picture, and really the purpose of this lecture, is to show some of the work that's involved.
So what you've got here is a bunch of isolated little black and white regions.
In order to figure out what's going on here, you've got to decide who goes together.
How are we going to decide who goes together?
Well lets think of this in the context of a bunch of little line segments.
You can, sort of, decide that some of these guys might go with some of the other ones but what I'm going to do, is rotate all of them by 90 degrees, if I recall.
No.
Maybe some other orientation.
But anyway, now they are the same lines on the screen, but now some of them hang together, right?
Got that sort of potato shape thing there.
Why?
What makes these guys go along with each other now in a way that they didn't before?
Well, if you're a little chunk of brain, whose job it is to figure out where contours are out in the world, what you're getting from earlier in the visual system, is word that there's a little bit of contour here, a little bit of a contour here, a little bit of a contour here.
I wonder if those should go together?
Now how do we decide that the little bits of contour might go together?
Well, you might do something like this.
If you've got a line here, and you're asking what's the best bet about if this is really a piece of a continuing contour?
Where's this likely to go next?
Well, it might make a hair pin turn and go off that way.
Doesn't seem really likely.
More likely, it's going to go off in something like the direction that it's pointing.
And that turns out to be the potato shaped ones.
This is a very rule governed behavior.
If you've got a bunch a little line segments, as long as the deviation here isn't more than about, as I recall, 30 degrees or so, from co-linear, you're willing to string those together pretty happily.
If it starts to be more than, that your unlikely to string them together.
The beginning of an effort to tie little pieces of information together into larger structures.
All right, what do you see here?
Two lines crossing each other.
A reasonable interpretation.
Though, it's not the only possible interpretation of this.
I mean, it could be something like this.
Two birds kissing each other, or something like that.
Why do you see and, in fact, this is something like that.
And you see that as an x with two lines crossing each other, but if I provide enough other details here, it's a sea lion?
I don't know what it is.
The point is, he's still a lousy artist.
It hasn't gotten any better.
The point is more or less the same.
You've got this little process that's worrying about that little piece of line segment, gets to this junction, and is busy doing the three roads diversion of yellow wood kind of thing, trying to decide which way to go.
And, it's guessing that all else being equal, I should probably go with that.
It sometimes goes by the name of good continuation.
It's one of a variety of so- called grouping rules that were developed first by the Gestalt psychologists starting in the early part of the twentieth century, and they're really rules for figuring out how bits of the scene might hang together.
You can see a similar sort of process going on here.
You can see that as three isolated line segments, but you probably don't.
You see that as a curvy lines that occluded, right?
If I suddenly reveal this, you don't go, ooh.
Amazing!
Kind of what I thought was there.
That's also highly rule governed.
If you've got a line segment and you've got another line segment, you're perfectly happy to see this one and this one is connected.
Well how about this.
This will do.
If I put one up here.
All the more if I erase the stuff in between.
That, you're less likely to see as connected.
Now why are you less likely to see that as connected?
The rule turns out to be, that if I can connect two lines with a smooth curve, I'm in business.
I'll be willing to see those as connected.
But, if I have to put an inflection in it to make it work, then it doesn't look as convincing.
I mean it's not that I deny the possibility this could ever connect with that, but if I give people a bunch of stimuli like this, and ask good connection or a bad connection?
The good connections are the ones that can be done can with a single smooth curve.
That can't quite.
I think you probably have to get an inflection in there somewhere.
And if you have to inflict the curve, it fails.
People report it as looking less convincingly continuous.
All right, it's voting times here.
Here we've got a whole bunch isolated guys, but they do seem to have something to do with each other.
If you had to pick, this being organized by columns or rows, how many vote for columns?
How many vote for rows?
Ain't much to chose, right?
But, if I do this, OK, how many vote for columns?
How many vote for rows?
So, we've now skewed it very heavily in the direction of columns.
And, all that I've done is change the proximity of elements.
Once the distance from one item to the next item in a vertical direction is closer than to the items in the horizontal direction, it's another one just these gestalt grouping rules of proximity takes over, and says, well, all else being equal, if I had to guess who goes with who, the guys that are close to each other.
They probably go with each other.
Multiple rules operate at the same time, so I'll keep the proximity rule working here.
Now, if you have to vote, how many vote for columns?
How many votes for rows?
So now, I've skewed it very heavily in the direction of rows, even though the proximity rule, is still going for columns.
Those things are closer to each other in a vertical direction that horizontal.
In this case, the similarity is trumping that.
You can balance these off against each other.
How similar to does it need to be to compensate for a two to one difference in distance for instance or something like that, but the important point here is, that what you're trying to do.
These are, you know, demonstration versions of presumably what you're doing all the time.
I'm looking out there, and I'm seeing regions of redness.
I see sort of disconnected regions of redness, but I'm guessing, they're all part of her top.
It's your top there.
It's you.
No, not her.
That's pink, you're wearing.
The woman behind you.
Well, the woman next to you is also red.
But, in this case, you can see the role of proximity here.
So, the similarity thing is telling me, all those red things are tied together, and I'm making it into sort of one piece of clothing.
The proximity thing is saying, well I don't think her red top, and her red top are the same red top.
That would be a very weird assumption.
On the other hand, the guy she's sitting next to is also wearing red.
So maybe they're just wearing one garment.
No.
I'm probably not going to come up with that assumption either.
But what my visual system is doing, is continuously trying to cut the world up into meaningful chunks, that are going to be worth subsequent analysis.
I don't want to go off and analyze every little pixel in this scene.
I don't have the brain power to do that.
I want to have meaningful chunks that are worth analyzing, so here I might decide to the meaningful chunks were the rows.
They are a something or other.
OK.
There's a couple of reasons why you need to do this grouping business over little elements in the world.
One of them-- I was sort of cartooning over here-- which is that early in the visual system, the chunks of the brain that are looking at it bits of the world are only looking at very teeny bits, and you're going to have to tie those together.
The other reason, is that out in the world, contours don't behave well.
They tend to do awkward things like disappear on you.
And you don't want to get the idea that these are-- if I've got a contour like that, if for some reason bits of it are deleted, maybe because there's an occluder, or maybe because something just bad happened in the image, you don't want to lose this whole structure, because bits of it have been deleted.
So you have a lot clever mechanisms designed to help you find where the edges are of things out there in the scene.
And, putting the bits of edges together into a long coherent one.
If you play with Photoshop, you can go and find the edges I think.
Isn't there like a find edges filter, or something?
So, try that sometime.
Do find edges on an image of, say, a person, and you'll see that it comes up with a lot of edges that you recognize as being related to this person, but its fragmented all over the place.
And, in fact, getting your computer to figure out which bits go with each other is a tricky piece of work.
It's tricky for you too.
You just don't know that it's tricky, because, it works all the time.
All right.
So, this beautifully boring stimulus is there, because you see this beautiful vertical contour, right?
Now, all I'm going to do, I'm going to leave it there, but I'm going to put a new background on it.
Isn't that lovely?
Why is that interesting?
Well, among the reasons it is interesting, is because it's the same gray stuff that was there before.
So, the top of that bar, and the bottom of that bar are still identical.
All four of those little rectangles are all identical.
Even though they no longer look identical.
Because what the system is doing, is-- actually can I do this?
I forget what I programmed.
Oh, there we go.
little key's going to turn into that one.
Isn't that fun?
Get back there.
There we go.
-- but my real point -- so what that is, is a simultaneous contrast affect, by the way.
This square looks bright, because it's surrounded by darker stuff.
That square looks dark, because it's surrounded by brighter stuff.
And even though this bar is continuous with it's gray level from bottom to top, it picks up it's apparent brightness from the immediately surrounding contours.
The interesting aspect of this from the point of view of understanding were edges are, is that if the bar is brighter than the background here, and darker than the background up there, there must be a place in the middle where it's gone.
Where there is no contour.
But you don't see that.
I was too busy making this little thing move, and I forgot to do that.
Oh well.
It's not a small region.
There's a fairly sizable region of that middle there where there is no physical contour.
But you fill it in.
You know, in some fashion, that contour is there.
That's called a subjective contour, where you're completing a contour that doesn't have any real support in the image.
That's the piece that your Photoshop filter will have a hard time doing, by the way.
Oh, we can also do a second order affect here, that's kind of -- well, no, we'll do that later.
Oh.
Come on.
Go away.
You've moved often enough.
OK. Let's continue the same point here.
All right, everybody sees this rectangle, right?
And what are those black things?
Three quarter circles.
Yes.
The literalists figured out that these are Pacmen or three quarter circles, or something like that, but you didn't really see that when it came up.
You said, oh that's a rectangle sitting on top of four circles of some variety, and, in fact, you still see it as a rectangle sitting on top of four circles.
In fact, you're probably reasonably convinced that you can see the contour.
That's weird.
I'm reasonably convinced that there's an interesting artifact on the screen that's creating contours.
I don't know what that's about.
It looks better over there.
It looks less bogus over there.
You can probably see the whole rectangle.
The white on white borders are not there.
There's simply is no physical contour there.
There may be here, because the project's doing something mutant.
But, there's certainly no contour there, even though the rectangle at the center looks somewhat brighter.
Again, what you're doing is, making a guess about what is it that actually created that image that's landing on my retina now?
It could be a little conference of pac-people.
Four little three quarter circles that got together to talk to each other.
But that doesn't seem the most likely possibility.
What seems more likely here, is that it's a white rectangle sitting on top of four black circles.
And you end up completing that contour.
Or this circle for that matter.
What are you doing here?
You don't need to have fancy computer graphics to do this.
One of the advantages of the material in this particular lecture is that it provides great material for doodling in other lectures.
So, if you like subjective contours, you can make your own.
And, so how's that look?
Got a subjective contour there?
It's not perfectly circular.
That's OK.
But what seems to happen is that you generate a hypothesis that says, the whole problem I got here, is contours don't just end in the world, they tend to continue.
Well, if this guy's continuing, well, what happened here?
Well.
Maybe it ran into another edge, it's being hidden.
All else being equal, it probably ran into an edge that's orthogonal to the direction it's going, let's guess that.
And, if we guess a whole bunch of little orthogonal edges, we're back to that earlier demonstration with a bunch of little line segments.
I can tie those little bits together.
They make a kind of a circle thing.
So I end up seeing that imaginary circle.
And, in fact, I'm going to start filling in the contour all the way around.
That suggests, by the way, that if I was to tilt all these lines a little bit off of straight radial, so that the virtual line segment was not forming a nice, neat circle, that the impression of a subjective circle would get weaker.
And you can decide whether or not that's true here.
So, see this circle?
Now, the question is, we'll give you three choices here.
Stronger, weaker or about the same?
How vote that this one is stronger than the previous one?
How many vote for just about the same?
How many vote for weaker?
Just about the same.
That's a boring demo.
I'll have to change that next year.
Because, it didn't work well enough.
OK. Another example.
If your following along on the notes, we have now gotten to the so-called Craik-O'Brien-Cornsweet illusion, named after Craik, O'Brien and Cornsweet.
suites.
We're continuing the edge business, but, now, what I want to do is tie that into that topic that I mentioned early in the lecture, about how it is that, what you're interested in is things the surface properties in the world, and you're not interested in lighting.
Lighting is very boring.
So, what do you see here?
Some bold soul described this complicated image, boy, slow group, gray thing.
Two gray things.
One is darker.
Oh boy, I'm going to bring my pliers next time.
This is like pulling teeth.
Yes
It's like the top of the pyramid, and there's shading on the other side
Oh.
Yeah.
Ok, so the light's coming from the left, or something.
OK. Yeah.
That's a complicated inference about these.
OK.
But what you don't particularly see, is this.
If I take the edge out, if I take that edge away from the middle here, what you discover is the whole thing is the same gray.
That's not obvious.
If you were to draw the luminance profile, drag a photo detector across this thing, what you would get is something like.
Which side is bright?
OK. That side bright.
So it rises, then drops across the edge, and then rises back like that.
So, that if you take out the actual edge, it's equal on the two sides.
So, that's kind of weird.
Why does it look like, now, they didn't believe me.
They thought I was doing something evil here.
Here we go.
Look.
That's why I had this thing slowly sneak up, so you could, well, semi slowly, fast-ly sneak up.
What's going on here is that the visual system knows something about edges and about lighting.
Edges in the world tend to be fairly abrupt.
Lightning changes, so it's bright here, and dimmer over here, lighting changes tend to be fairly gradual.
So, if what I'm interested in is seeing what's on the surface, as opposed to seeing the product of boring light and shade variations.
What I might want to do is to look for abrupt changes, and to, in effect, suppress gradual changes.
And that's what's going on here.
We might as well do an entertaining second order effect here.
Remember that negative after image thing?
You know, look at red, you see green, and stuff like that?
What I should be able to do here, is produce a negative version of this affect, where you'll end up seeing this side is dark and this side is light, even though I'm not going to change anything over here, over here.
Stare at that center line, right.
Stare rigorously at the center line, and keep staring.
And then when I do that, so, this is two illusions concentrates on top of that.
Do it again, was that?
All right.
For the people incapable of following instructions the first time, try it again.
So stare at the center and hold your fixation there.
Actually, people who got it the first time, keep staring at the center, but the people who got it the first time, can try something tricky-er which is move your fixation, a little bit, and you'll change the proportion that looks light or dark.
Isn't that fun?
You should understand why that works.
If you don't, write yourself a little note on your paper saying I don't understand why that works and figure it out.
All right.
So the important thing is that what you're trying to do here, is you're trying to get rid of information about the light.
You don't care about the light levels.
What you care about is what's going on in the world.
Now Ted Adelson in the brain and cogs department here has exploited this fact brilliantly in a variety of gorgeous demos.
One of which is this.
It is extremely difficult, even if you've seen this before, to convince yourself that the gray levels of A and B are identical.
Which they are.
In fact, it's so difficult that even if I stick a bar across it that's clearly the same thing, it's still kind of hard to see that as identical.
Your brain wants to do all sorts of things to deny that possibility.
What's going on here?
What's going on is that virtual cylinder is casting a virtual shadow, that you are busy discounting.
You're saying, I don't care about the shadow.
There's a checker board.
I know about checker boards.
Checker boards to go dark square, light square.
I can figure this out.
That means B is a light square, because it's surrounded by dark squares, and A is dark square because it's surrounded by light squares.
And, it's a particularly lovely example of, among other things, this ability to get rid of information about the illumination.
This isn't to say that what you do is somehow just run some sort of, throw away the light source information, and don't do anything with it.
Get back there.
What's it say?
Cow.
There's only white and black on that screen.
Look at the C and ask where that contour is coming from.
That contour, particularly the outside of that C, has extremely little support in the image.
What you are doing is making an inference.
This time, you're using the shadow information.
You don't want to see the shadow.
How many people see it not only as cow, also as an embossed word cow, sticking out a little bit?
The reason you're seeing the cow at all, is you're assuming that those black things are shadows.
If those black things are shadows, it follows the light is coming from the upper left.
If the light is coming from the upper left, well, we can go and figure out what shape object must have been producing those shadows, and the answer spells cow.
But, if you were to take a look at the C, is about the clearest.
Well, I don't know, the O is pretty good too, and so the W, for that matter.
Anyway, looking any of those guys, and look at the shapes of the black bits, which is the only thing that stands out from the background, of course.
There's nothing else there, except the black bits on a white background.
None of those bits say C or O or W. It's a construction based on the assumption that the black bits are shadows.
You use that sort of information all the time to do things like see faces.
These are so called Mooney faces, named after a guy named Mooney.
You can tell me a lot about the curvature of these faces, even though, again, there's nothing on the screen except for black regions and white regions.
None of which are, themselves, particularly faced shape.
I mean look at the eye.
Are any of those eyes actually shaped -- look at the guy on the right-- I mean his eye, he's only got one apparently, that eye looks like, I don't know, a mutant bunny or something.
If I just presented the eye piece on the on the guy on the right, that black glob that's defining his eye, if I just presented that in isolation, nobody would say, oh yeah, sure.
That's an eye.
You'd all be saying dalmatian dog before, mutant bunny this time, or something.
And, it relies on these assumptions about shadow, is what is what you're doing here.
It survives inversion reasonably well.
But, those don't look like faces very much.
What went wrong?
You might think, I know that shadows aren't red and blue.
But that's actually not the problem.
Here they look pretty good, right.
Those faces look ok, but these faces look lousy.
Why do they look lousy?
Somebody raise a hand, or something, yeah there's a theory
The shadows are lighte than the [INAUDIBLE]
Yes, if you make the shadow regions lighter than the lit regions, the brain says, that's not a shadow.
I don't care about shadows.
I don't want to see shadows as entities in their own right particularly, most of the time.
But I'll tell you one thing I know about shadows.
Shadows are darker than the other stuff.
If the shadows are lighter than the other stuff, it's not shadow.
It's something else, and something weird.
So this doesn't work.
But that works.
Oh, I suppose the fact that it said on this one, shadow must be darker, might have tipped some people off.
OK. Let's see here.
I think what I will do -- this makes it very natural break point -- where it says Mooney face and we'll go on to this question about making the best guess you can make in the context of going from 2D to 3D.
D But before we go on to that, let's us lets take our brief, stretch your limbs kind of break, here.
OK. Let us gather back together here.
By the way, it looks like it's getting to be that time in the term, where people are abusing their natural sleep mechanism that I will talk about later on in the term.
But looking around at this crowd, I would say that you're not getting the seven to eight you need.
Or, if you are, maybe you really need ten and you're catching the extra to 2 here.
I've been talking about sort of little, almost like atomic small scale examples, of these sort of inferences that you make.
And, now what I want to do, is sort of head for the larger picture of how you make an inference about the whole scene.
I'm not going to get all the way there, and there are many realm I could talk about this in.
So I'm going to talk about it in one restricted area, which is this question of going from a 2D image to 3D inferences about the world.
Something that you do automatically, all the time.
I want to explain a bit about how you do it.
You will see, on your hand out, this is very useless, blank region, that says one two, three, four, five.
I'm going to go through a series of depth cues.
Probably more than five of them.
And, that's what's supposed to go in there.
They're all lots of different sources of information that you use to go from the 2D world to the 3D world, many of them seen here in this lovely piece of renaissance art that we will come up back to.
But, here's a much more boring piece of art.
What do you see?
Circle, square and a diamond.
And, then there's the clever person, who's trying to figure out, I've describe them as pac-man before, but these aren't.
But, you see a circle, square, and a diamond.
You don't have any serious difficulty inferring that -- triangle, sorry.
You don't see this.
For present purposes, the important point here is, you can also tell me their depth order.
It's in some sense so obvious that you never think about it, but it's a very important source of information about depth order, that you get simply because you know that solid object occlude each other.
So you firmly believe that I am standing in front the screen.
I could had, well no, I couldn't have, it'd be theoretically possible that I had suddenly cut a cunningly wolf shaped hole in the screen and I'm A, very large, and B, standing over towards east campus somewhere.
And, you're looking at me through this set of holes.
No.
You automatically leap to the assumption that if A looks like it's occluding B, A in front of B. Ah, my bunnies.
I don't know what happened to them.
They got kind of pixellated.
But, all right, which bunnies are closer to you?
The big bunnies are closer to you.
Right.
Why do you think the big bunnies are closer to you?
Because, you're making an inference that bunnies are roughly bunny sized.
And, in the same way, I'm currently making the assumption that you guys are all more or less people sized.
If I did not make that assumption, I would come to some very odd inferences about the current view that I'm looking at.
So, the people in the front row -- there's a person in the front row -- her head is about two degrees of visual angle.
Remember 360 degrees around my head each degree it's about my thumb, so her head takes up about two degrees.
And, let's see, there's this guy in the cheap seats back there, his head is only about half a degree.
I could make the assumption that he's a pinhead.
A guy with a real small had sitting out the deep back out there.
Well actually, I wouldn't make the assumption that he was sitting in the back.
He's a pin head sitting at the same distance as large head woman here in the front.
But that's dumb.
Right?
Your visual system knows that's dumb.
Your visual system knows people are roughly people's sized.
Not exactly people sized, but roughly people sized, and if I see a bunch of small things there and a bunch of big things here, odds are that this is closer than that.
And that's part of what's giving me my current inference that I'm looking at a tilted plane of people in purple seats is this information about size.
If I organize the bunnies the way you guys are organized, I got a much clearer sensation of depth from this texture radiant.
So, now you should be able to see sort of tilted rabbit plane, right?
Even though you know objectively, it's just sitting flat on the screen, it looks tilted.
Is that a hand up there?
That was a hand.
Sorry I missed that.
My previous image.
We can do that
[INAUDIBLE]
Yeah.
That's another possibility.
It could would be that.
And, in fact, it is, just a flat image.
And you actually sound like you're getting a sort of a hybrid of a tilted plane with bunnies of a range of sizes.
Actually, we can see that combination here.
So, we got a whole bunch of big bunnies, and two little bunnies.
Which is the smallest bunny in this display?
The bottom right bunny, right?
These guys are identical in size.
It's a very minimal display.
There's a much more vivid version of this illusion in the book as I recall.
It's a very minimal version of the illusion.
Because you assume, if you are -- All right.
So I still can't draw.
-- If I'm looking at two bunnies, if I'm looking at a ground plane, closer is also lower in the visual field.
So, you make an automatic assumption that was what was giving this woman that notion of a fairly tilted plane in the first bunny example.
You make the assumption that the bottom of the image is closer to you then the top of the image.
Well, if the bottom of the image -- let's go a back here-- if the bottom of the image is closer then the top, and these two bunny images are the same size, if this guy's closer, it must be really small.
Suppose I took the guy from the back row here.
His whole upper body fills sort of the top joint of my thumb in visual angle terms.
If I moved that image to the front, and sat him in a seat here, he would be about the size of this woman's upper arms.
And, I would think, that's a really small guy.
She could wear him on her --instead of wearing your heart on your sleeve, you could wear the whole guy on your sleeve.
So, that's what's going on here.
You're making the assumption that small bunny one is closer than small bunny two.
They're the same image size.
You therefore infer that out in the world, this must be a really small bunny, and that one is just a reasonably small bunny.
Now the bunny part turns out to be not that critical.
Here you can also see a nice plane going off into the distance with objects that are clearly not meaningful.
Right.
Big stuff, front and low.
Small stuff high in the image and, you simply infer this is close, and stuff up there is far, and you get a nice impression of a tilted plane.
This is one of the cues that is interestingly variable depending on where you're from.
The atmosphere scatters light, particularly water in the atmosphere scatters light, that's why the sky is blue.
The result is that objects that are far away tend to be both hazier and bluer.
Something that you can see in all sorts of works of art.
Go to the museum, you can see artists take advantage of this right, left and center.
And, you can probably get even in my pathetically reduced version of it.
You probably get a sensation of depth here that you don't get here.
The geographic aspect of it, -- anybody here from Arizona?
Doesn't work well in Arizona.
I know this because I went to a meeting in Tucson and it was boring, so I went out for a walk, and I saw this hill, and I said to the guy who was at the street corner with me, how long would it take me to walk to that hill?
And he said, three days, four days?
It's like 50 miles away and it's like 6000 feet high.
But it was extremely crisp.
And, there's no water in Arizona.
I don't know why people live there.
It's like hot all the time.
And there's no water.
But anyway, aerial perspective cues don't work, so not only are you thirsty, put your short one depth cue.
Anyway, this works much better in a humid setting that in a non humid setting, but the point is that again you know about-- this is your visual system making use of the physics of the situation in order to infer something about the depth of the situation.
You also know about the geometry of the world.
So, this is an extremely limited picture.
If I say, this is a highway going off to infinity somewhere in Arizona, or something like that, that's not a great picture.
But you can believe that.
Because you know implicitly that parallel lines in the world, if they're in depth, will look like they are converging towards a vanishing point somewhere.
Now it is sometimes claimed that this is known as linear perspective, that linear perspective was discovered by artists during the renaissance.
That's only sort of true.
What happened in the renaissance, was that they made this knowledge explicit, and became able to use it for instance to make their art works.
But, your cat and your lizard and stuff know about linear perspective.
They just know it implicitly.
The same way they knew about arial perspective and size clues and occlusion cues, and things like that.
It wasn't that we woke up one day in Renaissance, Italy and suddenly we could use this depth cue.
What we figured out was how to paint with this depth cue.
And you could do all sorts of amusing things with the depth cue.
So, for instance.
Lines look more or less the same size, right.
Let me see if I can change that here.
All right.
Even though we're using crude materials, let's do a forced choice vote here.
If you know they're at the same size, so it's boring to ask if they're the same size.
But if you had to vote bigger or smaller, how many people would vote that the bottom line now looks bigger?
How many would vote that the bottom line now look smaller.
Well, I guess that worked, cheap chalk and all.
This is an illusion known as the Ponzo illusion.
There are a number of ways to account for it, but one of the intuitively appealing ones, at least, is to say, what this is doing, even though you're not particularly seeing it as a depth cue, is it's telling the chunks of your brain that are trying to figure out 3D, I see these two converging lines.
If they're parallel lines in the world, they must be going off into depth.
If they're going off into depth, then this thing is further away than this one.
Well, if this one is further away, and they are the same size on my retina, this must be bigger.
If that's not intuitively obvious to you, think about this as train tracks.
So here are train tracks going off into the distance, and ask yourself, which maiden here, tied of to the tracks is in more distress?
It's obvious that this must be the bigger person if we interpret this as train tracks going off into the distance.
Even though we know that they're essentially the same size.
You're getting the results of the inference are then influencing what you see.
They are influencing other inferences that you make about the image.
Now you can exploit these rules of linear perspective in much more elaborate fashion then that.
And the great master of that game is Escher.
Here is one of Escher's pictures.
What you want to do again, I think the image looks rather sharper up on these sides guys, but ask yourself where the vanishing point is.
So, look at the first floor of that structure.
And, it's pretty clear that those railings are converging to a vanishing point off to the right somewhere.
Well, now look at the top floor.
That's converting to vanishing point off to the left somewhere.
And, then when you tried it put the whole thing together, you've got a structure the doesn't quite hang together quite right.
This tells you a couple of things.
Thing one it tells you Escher was a very clever draftsman.
Thing two that it tells you is that you do a lot of these calculations about perspective very locally.
What you do is you say, your attending, let's say, to the lower floor there, and you say, yeah, this all adds up.
It makes sense.
And, then you direct your attention to the upper floor.
And it all adds up.
It makes sense.
It's only when you try to combine all of that across whole image, that you realize that the whole image somehow doesn't quite make sense.
And, that's how Escher can do things like, have staircases that always go up, and water falls that fall apparently in an infinite loop and things like that.
Grab yourself your favorite Escher website and or your favorite Escher book, and you can watch him manipulating these depth cues endlessly.
It's great entertainment.
Now what I want to do is to bring together the themes of the lectures to this point in single demo.
At the moment, that doesn't look like much of nothing, except that, well, I don't know, what does it look like?
Cubes.
All right.
That's interesting, because I don't see no cubes!
Where's that inference coming from?
What you're really have is a bunch of Y junctions.
And you know about Y junctions.
Those are probably corners.
And, they look like they might be kind of cube-y corners.
But, what I'm going to do is rotate each of those.
Now, this is cool stimulus for a variety of reasons.
First of all, now you're really seeing a cube.
Right?
No problem.
Second of all, there are two cubes.
There's the cube, with its face pointing.
Let's try this.
There's that face, pointing down and to the right.
There's that face pointing up and to the left.
So, you've got two cubes.
This is an ambigious by stable figure.
It's known as a Necker cube.
We can quickly draw one of those.
Endless fun for doodling again.
You can make yourself ambiguous figures instantly.
That figure by itself is known as the Necker cube after a guy named Necker.
So, you're inferring two cubes.
You're inferring one of two cubes.
The cube isn't particularly there.
The black stuff -- all you're seeing is the verticies of this cube-- but you're still managing to infer the rest of the cube You can probably see the lines of the cube in the black region, right.
In fact, you can see the intersection, if you straight up from here, you can see the intersection of two lines that aren't there.
I can make those lines go away.
Now take a look at that cube, and imagine that what you're looking at is a cube -- sort of a wire frame cube-- that's behind a sheet of sort of black swiss cheese.
You're looking at it through holes.
Can you get it to go back there?
If you get it to go back there, you can hold it back there, you probably noticed that the subjective contours pretty much disappear on you.
Why is that?
Well, if it's behind, there's no reason that you should be seeing those contour.
They would be invisible.
And so the invisible contours become invisible.
Now if you bring the cube back in front in your perception, you'll see, oh yeah, now if that cube's floating in front.
I ought to be able to see the whole cube.
And now, I can see the subjective contours.
So you can make the subjective contours contingent on which particular interpretation you care to give to the image.
So, I think this illustrates very nicely the notion that what you are seeing is your current hypothesis about what might be generating the image on the screen.
Another thing that it points out, is that you are only willing to entertain at limited set of hypothesis.
It is extremely hard to look at this and see it as however many it is.
Eight little disks with chicken feed in them.
With little Y's in them of some sort.
It's very, very hard to see that.
Even though that is perfectly consistent with the image hypothesis.
You are out there trying to make a guess about the world.
And, you're not willing to entertain all of them.
At least most of them are immediately relegated to the realm of the very unlikely.
And, normally out in the world, what happens is that a single hypothesis immediately pops to the four, and you accept it.
In weird situations like this, you can entertain a few of them.
You immediately narrow down the realm of possibilities to a few, not to an infinity.
Even though there's an infinite number of possible ways to generate this thing.
Shadows I've already shown are a depth cue.
The reason for putting this nice piece of renaissance art up there, is to point out that, while shadows are a depth cue, you're not actually terribly picky about the physics of the situation.
At least not the global physics of the situation.
So where's the sun here?
This is an outdoor scene.
Where's the sun coming from?
Well, if you look at the people in the lower left -- the people on the ground plane there -- it's pretty clear that the sun must be down and to the left somewhere.
Right?
Well, look at the shadow underneath that portico.
Well, the sun must be up and to the right there somewhere.
There's no consistent source of illumination in this image, but your perfectly willing to use the shadow information to give you depth information.
It's giving it to you locally.
The fact that it doesn't add up globally, doesn't bother you.
And it doesn't even bother you to the extent that it bothers you in the Escher picture where the building was actually impossible.
Here, you don't recognize the impossibility at all, unless it's pointed out to you directly.
And, finally I should add to the list, three more than are rather hard to demonstrate.
That are hard to put up just as Powerpoint slides.
When people think about depth perception, if they think about depth perception, it's stereopsis binocular vision that they typically think of.
Your two eyes are in two different places in your head for most of us.
If you blink from eye to eye, or just cover your one eye after the other, it's actually better if you hold one finger out in front of you, and blink from eye to eye.
You'll see the image in the two eyes is not the same.
The difference in those two images is highly geometrically regular.
And you make use of that regularity as a depth cue.
It's called binocular disparity.
Very useful depth cue.
It's what's giving you the magic eye demos that you get.
Those posters that if you cross your eyes just right, they jump out in depth.
Those still around?
It's a very useful depth cue.
It's a little on the overrated side, because it's a lot of fun to study.
People who think that binocular vision is the be all and end all depth cue, should cover one eye, and ask if the world suddenly looks very flat.
It looks a little flatter, but I can still perfectly well tell who's in front of who.
On the other hand, if you want to see what's stereo is doing for you, on a beautiful day like today, go outside, lie under a tree, close one eye, and look up into the branches, and try to figure out which twigs are in front of other twigs.
You'll have very hard time doing it.
Open the eye, and the whole thing will jump out at you in depth.
That's the sort of information that stereo is giving you.
You don't have to do it with stereo.
Motion parralax to jump to the bottom of that list, is a similar sort of geometric clue.
If I'm here, and then I'm here, the image changes in a geometrically regular way.
You guys are sliding around on my retina in such a way that these guys are moving more than you guys out in the back on my retina.
And, I know that, again implicitly, like I know linear praralax and I can use that to inferred depth.
Try this under the tree, and you can see the same thing.
Close one eye, look up into the branches.
The branches look relatively flat.
Now rather than opening this eye, just move your head back and forth, and the tree will jump out at you in depth.
Actually quite striking, you should try this sometime.
I don't know if anybody ever does take me up on this suggestion.
So if you actually try it, let me know so that I know that somebody actually tried it.
Oh and vergence is another one of these geometrical cues.
Hold your finger out in front of you.
Look at your finger.
Now move the finger towards you, holding it as a single finger as long as you can.
So I can watch you go cross eyed.
What you are doing is converging your eyes.
And, if you could move your finger further out, you would be diverging your eyes.
Well, what you can think of that as doing, is taking like a pair of sticks and pointing them at the object.
It's your visual axis in the sense, you're pointing at the object there.
And the angle formed by those sticks is narrower if you're looking far away, then it is if you're looking close up.
And you have a fairly impoverished ability to use that as a depth cue too.
If you were a chameleon you'd be much better at this.
Chameleons have eyes the move independently and are very sensitive to the angle that their eyes are pointing.
And, in fact, you've seen the Animal Planet kind of videos where the chameleon's tongue goes out the length of its body, and it grabs a fly or something like that.
How does that it know where the fly is?
It knows by measuring the angle of its eyes.
How do we know that?
Well, we know that because, somebody went and put glasses on chameleon that diverged the eye.
So, in order to point it's eyes at the fly, it had the angle wrong, basically.
So, you put a fly on a popsicle stick you put the glasses on the chameleon and then your film the chameleon's tongue, and the chameleon keeps missing.
If I put those prisms on your eyes, you will adapt, and you will eventually be able to catch the fly again.
Should you be so inclined.
The chameleon turns out to be a less adaptable creature than you, and will not adapt.
You can do the same game with chickens.
Put a pair of prisms on your eyes to divert everything off say fifteen degrees to the left, and then if I tell you pick this up, you'll reach fifteen degrees in the wrong direction.
But eventually you'll learn.
Put the prisms on a chicken.
Put some grain down.
Here are the grains here.
The chicken sees it over there.
Chickens pecking over there.
Chickens not getting any grain.
Chicken will do that forever and apparently not learn to get it right.
Oh, good.
I left myself with enough time to talk about inferences in a more global sense of combining information from across the senses.
So, you've got this job to try to figure out what's going on, I've been talking about doing that specifically with the visual system.
But, you're collecting information from multiple sources, and taking whatever the best information is, so if I show you a movie, you'll get captured by the visual information, and you're perfectly happy to hear the words coming out of the mouth of the guy on the screen, even though it's coming out of some speaker on the side.
And it doesn't matter if they got fancy dolby stereo or something like that.
Use a cheap, simple speaker sitting off to the side and you'll still hear it as coming out of the guy's mouth if you're watching it.
So you're combining information from multiple senses.
The particular example I thought I would discuss with you, is the ever pleasant example of motion sickness.
So, when I first came to graduate school, my lab was doing research for NASA on the effects of looking at large fields that were rotating.
Sort of Omni theater stuff.
If you look at a whole field that's rotating counter clock wise around your line of sight, you feel like you're rotating clock wise.
Meantime guys up at Brandeis who we were collaborating with, were doing the same sort of thing, but their depended measure was how long it took you to throw up.
Fortunately, that was not my introduction to graduate school.
But, it will make you sick.
I might as well collect some data here.
How many people have ever been motion sick here?
OK. What made you sick?
Motion.
Yes.
Thank you.
Could were get a little more specific, while keeping within the realm of good taste here?
Oh.
Reading on a bumpy bus.
That's one good example.
Anybody got another good one?
We'll take one more here.
Oh, spinning around in a circle.
Did you get sick while you're spinning around in circle, or after you stopped?
After you stopped.
And, then when you stopped, you felt like you were spinning around in the other direction.
So when you're spinning yourself around in the circle, -- you have, in your inner ears, these tubes filled with fluid, -- and quick drawing here, quick bit of vestibular physiology --need a nice fat piece of chalk -- oh, it's yellow -- you've got these tubes inside your ears that are filled with a fluid, and inside a little space in there, are these hairs.
If you bend the hairs, it sends a signal off to your brain, that's the transducer to the equivalent of the photo receptors from the eyes.
You can imagine taking a bucket and starting to move, the fluid stays behind for a little while, and sloshes around.
That's why its hard to carry buckets of liquid around if they are too full.
So if you rotate your head, the fluid tends to stay put.
And, it moves over the hairs bending them and telling you that you move your head.
That's what's telling you about this sort of motion.
Well, you spin around for awhile, and the fluid in here eventually catches up with you and starts moving.
Then you stop.
And the fluid keeps going, and you say, oh no.
The other way you can do this, by the way-- so it's very carefully calibrated system, turns out that alcohol is lighter then this fluid.
If you drink, the reason you get dizzy, it's not because you pickled your brain, which is also true, but because you've dilute it this fluid with alcohol and this stuff is uncalibrated.
And, now, you move your head a little, and the brain, says, oh, man.
We just moved a lot.
Don't move that head.
What she was pointing out, is that what turns out to be the great stimulus for making yourself motion sick, is a mismatch between the information that in particular this vesticular system, this balance system, and your eyes are giving you.
So the example given here of reading on the bus is a marvelous example.
Because what you're doing, in order to read, you're holding this thing so it's a relatively stable visual stimulus.
Nothing's happening here.
But your vestibular system is saying [?___bumbidingdi bingdaeda--___?]
lots of stuff is happening.
And your digestive system is busy saying [?
Blech.
?]
Same thing happens on a plane, right?
The problem on a plane is that when the plane bounces up and down, I mean short of catastrophic bouncing up and down, when a plane hits turbulence and it's bouncing up and down, what do you see?
Nothing.
Right.
There's nothing happening visually.
What do you feel?
You feel [?
bompitdumitbomp.
?]
and you know, [?
bloop.
?]
And, the interesting question is why'd you get sick?
It's fine to say that the mismatch of visual and vestibular information turns out to be nauseating.
But, why should it make you sick?
The answer is, it's another inference.
So, the next question then is what's the inference?
What are you guessing?
You're guessing that the airline food was lousy.
Yeah.
Yeah, you're guessing that you were poisoned.
Why you guessing that you were poisoned?
It's the mismatch that's critical.
If you just make your vision strange, I could show you weird stuff you hadn't seen before, and you wouldn't throw up.
And, I can also bounce you around, and until I do fairly dramatic stuff, you won't throw up.
But, if I mismatch what your visual system is telling you about your body, and what your vestibular system is telling you, what this is,is a protection of sorts against neuro-toxins.
How do poisons work?
Well some of them work by attacking your nervous system.
Lot of work by attack your nervous system.
How are you, the owner of the nervous system, going to know, you're going to think, I can't integrate anymore.
I can no longer remember that I love my mother.
The sorts of things that immediately present themselves to you as you are being poisoned, are, your senses are coming unglued from each other, and so if your eyes are saying, bong-de-bong bong, and your vestibular system is saying that your body infers that you have been poisoned, and a useful idea, if you've been poisoned, is to get rid of whatever you just ate.
So, why does this happen on airplanes and stuff like that?
You weren't built to be flying around at 30,000 feet bouncing around in the clouds.
It just wasn't what's nature set you up to do, and so you get the unfortunate situation where even though you haven't been poisoned, you get sick.
OK.
Enough of that cheery topic.
The following content is presented by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at OCW.MIT.edu
Good afternoon here is the plan for today's lecture.
There is out there somewhere, unless you're philosopher, a world, it's huge.
A lot of stuff out there in the world.
Their is inside use somewhere a long term memory, which is also huge.
And and hopefully getting huger all the time.
You cannot transcribe all of that world from the outside into your long term memory, as you may have already discovered if you've had a test this term.
And in a move that ought to looks familiar to you by now.
We can described that as a bottleneck of some sort.
Not the same bottleneck that I was talking about in the context of attention, but a bottleneck nevertheless, that is involved in providing and putting a severe limitation on the ability to encode the world into long term memory.
The first part of today's lecture is going to be about this problem of encoding and about the nature of this bottleneck.
The second part is it that OK some stuff, a mere trickle of the world, managed to get through this bottleneck.
It needs to stay there.
It needs to stay there for a long time.
You've got things in here, in this long term memory of yours, that are 15, 16 years old, maybe more at this point.
How they stay here, how they are made firm, is the problem of consolidation.
Which it would say underneath there if it wasn't that I put a screen in front.
And finally, it is of absolutely no use to you have -- as you also may have discovered on some exam this term already -- to have a long term memory that is absolutely chock full of marvelous material if you can't get it back out.
Unlike the story that we were telling before about perception, where there is a world of bottleneck and then some construct that was your perception of the world.
In this case you've got a two way street, that you cannot appreciates the full contents of your long term memory at any given moment.
You need to retrieve -- that's what it says on the far board -- you need to be able to retrieve material from long term memory, and bring it back as we will see.
In effect to bring it back into the bottleneck.
Got to be able to get back out into this limited capacity bottleneck where you can do things like write down the answer on a test, or tell me who's running for president this year.
Nobody
Bush
Yeah that's one of them.
Bush yeah
Kerry
Kerry OK. And a few other characters too.
Right.
OK.
Presumably except for the most politically obsessed among you, that was not an active thought in your mind until I asked it.
But Kerry and Bush live in your long term memory, isn't that a scary thought?
[LAUGHTER]
And on demand, you could retrieve that back out into what we will call and your working memory.
If we were just thinking about this and encoding terms, we might just call this short term memory -- stuff comes in from the outside, it's maintained for a little while somehow in a short term memory, and then somehow makes it into long term memory in ways that we'll talk about.
But if we think about this in a two way, in a bi-directional kind of way, it's better perhaps to think of this as your working memory.
Think of it as your computer desktop.
It's got access to all the stuff that's on your hard drive.
It's got access to the external world.
You can tell it's got access to the external -- oh no we can't see it there.
Well, you know, it's wireless and so it's got access to the web.
And so that's the external world.
But only a limited amount of it can be on the desktop any one time.
Similarly, only a limited amount of either the world or the contents of your long term memory can be on your mental desktop at one time.
That's working memory.
That's an important junk of this bottleneck.
Well what does it mean to say that a limited capacity bottleneck of some sort?
Let me illustrate the limit to that capacity.
We saw a particular example of that in what's known in the trade as visual short term memory last week, when I was putting up, you know, four little colored blobs and saying, OK now what changed.
And there's a limit of about four objects that you can keep track of.
But in trying to ask what's the limit of this bottleneck from the outside world into a more long term varieties of memory, one of the typical measures is what's called digit span.
If you end up in the hospital because somebody bopped you over the head, one of the things you neurologist would do is read you a list of numbers, and ask you to repeat them back.
And how many you can do is an index of whether you are OK.
So let's see if you're OK.
Except I won't do this with numbers, I will do it with color names.
And we can come back to that in a minute.
Where are my color names?
Here are my color names.
So what I'm going to do is read you a list of color names.
I'll then say repeat, and you in glorious unison repeat this back to me.
OK.
So if I say, red, blue, purple, repeat.
You say?
Red, blue, purple
It never fails
[LAUGHTER]
Right, [?
Mara, ?]
last year too?
Always you get the repeat back too.
And I'm not sure what it is about this that demands, that at least from some people, the repeats of repeat.
But in any case good, good.
You're not deeply brain damaged yet.
Ready?
We'll do some more of these.
Green, pink, yellow, white yellow, repeat
Green, pink, yellow, white, yellow
OK. That was fine.
All right, here we go.
Oh, I forgot to mention.
It turns out to be really easy to repeat back in essentially infinitely long list of these, if you write them down while I'm saying them
[LAUGHTER]
There are a variety of demos that I'm going to do today that are of that form, and it's really boring.
So, you know, OK we know you can do that, so don't do that, because that's boring.
Be a nice honest person.
OK ready?
Where was I. I just did 5, right?
Let's try 7.
Ready?
Oh and I told you when I'm going to stop.
Oh what the heck.
Pink, red, yellow, green, purple, pink, orange, repeat
Pink, red, yellow, green, purple, pink, orange
It's getting a little weak there at the end.
But you will see, if you look at the handout, you will see that the answer to this question is given in the title of George Miller's classic paper on the subject.
And is answer is 7 plus or minus 2, and some of you were on the minus side.
Oh I should also say -- I should've said this at the beginning of the course -- people in intro psych classes, particularly by the time you get around starting to talk about cognitive things, like memory, take demos very personally and think 7 plus or minus 2, I got minus.
I'm doomed
[LAUGHTER]
You're probably fine.
I mean if you, if it was the red, yellow, purple, you're saying what was the one after red.
You know I admit there might be an issue there.
The advantage of a 300 person intro class is we're looking at the average, and what you're doing on any given single isolated demo trial is not actually deeply relevant to whether or not you're going to pass physics for instance.
OK, one more.
Ready?
Blue, green, purple, red, pink, yellow, green, blue, white, orange, repeat
Blue, green, purple, red, green, yellow [LAUGHTER]
Yeah, yeah, yeah.
You can hear.
That was 10.
And you could here it in the group, and you could probably feel it in yourself, that, you know, up to 6, 7, 8, you're yeah, we're good here.
Oh man if I start taking in this next one, that I'm losing them here.
Something bad is happening.
So there's this capacity limit on what you can get into -- well let's used this jargon for the time being -- into some sort of short term memory, and then spit out immediately to me.
Now 7 plus or minus to what?
Let's try another one.
You ready?
Same game.
Red, red, red, red, blue, blue, blue, blue, green, green, green, green, repeat
Red -- [LAUGHTER]
You know you can do this.
That's 12 items.
How come I all of a sudden manage to boost your memories so effectively?
Yeah classification, or what Miller originally, and the field had stuck with the notion of chunking.
If you can break things into meaningful chunks , it's 7 plus or minus 2 chunks.
And what's important is how you managed to cut the world into chunks.
This by the way explains why I don't do this demo in the traditional way of giving you digit strings.
Because years ago I discovered that if you give somebody a 10 digit, you know, give 300 MIT undergraduates a 10 digit string, you know, a bunch of them crud out on you, but then there's a handful of people who just raddle it right off.
And you say oh my goodness, these people have this amazing memory.
And you ask them how'd you do it?
And they say oh well that was obvious.
It was the natural log of 16
[LAUGHTER]
MIT students tend to junk numbers rather more readily than the rest of the population.
They don't tend to junk color names particularly adeptly I've found, and so you get the right 7 plus or minus 2 answer.
Now this sort of chunking, you should think of this more like a rate right.
It's not 7 plus or minus 2 things ever, ever, ever.
It's 7 plus or minus 2 things in some unit of time that you can manage to deal with.
Otherwise as soon as I had presented 7 plus or minus 2 units of information in this lecture, you'd be like dead for the rest of the day.
Or maybe, you know, the first lecture shot your brain and that was it for the term.
That's obviously not the case.
What the question is how big a chunk can you manage to cut things into as it's coming in.
One of the places you can see that is in language as you're trying to keep track for instance, what I'm saying.
So if I ask you to repeat this sentence, memory is a fascinating topic that would be on the final exam, you would say?
Memory is a fascinating topic that would be on the final exam
But if I lettuce, quadrangle forms only only column bug de la forms aftunic reply quintilian.
Kelly You say?
[LAUGHTER]
Say it roughly the same number of syllables.
But ouughh.
You know, you can't repeat that back, because you can't chunk it.
Well you can, but the chunks that you were making were too small to survive the rate at which they were being presented.
So everybody here, if forced to, would've produce some little bit of it.
But people would not be able to produce the whole thing, because you can't chunk it into meaningful chunks that work.
This is also the route presumably of the near universal experience that when you go to some other country where they speak another language, that you nominally speak because you took four years of it in high school.
You discover that those Parisians or those Spaniards or the Japanese or whoever talk really, really, really fast.
What's the chance that everybody else in the world talks really, really, really fast.
You know, it's not true of course.
And those of you who are non-native English speakers, and have now arrived at MIT, are thinking no, you know, back in, you know, Kazakhstan they all spoke really slowly or whatever.
But these English speakers are wackos.
The problem is you are in your native tongue, or in any town but you are really fluent in.
You are very good at cutting into meaningful chunks rapidly.
If you're busy trying to say, what was that word.
I know.
I damn it that was the week I was asleep in high school or something
[LAUGHER]
No that was the year I was asleep in high school.
Anyway.
The chunks go by too fast like in the nonsense that I was producing for you.
You're set up to get chunks through in some domains much better than in others.
And in fact, let me illustrate that here.
By where did I hide?
That looks like it's probably it.
All right.
Got to memorize some more stuff here.
Come on.
You want to go.
And there we go.
Too much light.
Let's kill the stage lights.
Good.
OK.
I want you to memorize all these pictures.
You ready?
There's a picture.
And there's a picture.
And that's a picture.
And there's the back end of a horse or two.
And there's a Christmas tree or something.
And I don't know I must've rated a housing site.
But, you know, these are definitely pictures.
And that's a picture.
And that looks like a picture.
And oh look there's tomatoes.
OK. Got them all?
Yes
OK now what I want you to do is, I'm going to show you some pictures that are either old or new, and you just say yes if you've seen it before, no if you havent
No.
No.
Yes.
Yes.
No.
Yes Yes, No.
No.
Yes.
No
OK.
The important point is that you're very good at this.
The interesting side point is that this particular one is flipped left, right, reverse.
So if you were sitting there saying why are these people next to me changing their mind.
It's because some people noticed that it was left, right, reverse.
But if I had lots of time, I could have done this with a lot of pictures.
The largest report in the literature is 10,000 pictures shown to the usual collection of college undergraduates.
And some days later, the students still remembered I think it's about 80% accuracy on that.
If I read you 10,000 color names -- well there aren't 10,000 color names -- if I read you 10,000 words or almost any other material that I would care to present to you, you would do nowhere near as well.
We're not entirely clear on how you do this.
This is quite a remarkable ability.
But it does speak to the ability of picture information.
Familiarity with a picture.
Getting through this bottleneck into some sort of long term memory with remarkable efficiency, that you are somehow specialized for doing that.
There's also a little bit of a cheat here.
It's only a little bit of a cheat, but it's an interesting cheat.
This was a recognition test.
The color one, the previous demos were recall test.
I was asking you what, you know, pull out of your memory what was there.
Here I was saying was this there.
And you know this.
But, from your test taking experience you know that, OK if I give you a choice between fill in the blank and multiple choice, which do you want to do?
Multiple choice
Multiple choice, because that's a recognition test.
Fill in the blank is recall test.
The distinction is OK here's your long term memory.
Here's the target that you're trying to find in long term memory somewhere.
If I'm doing a recall test, I'm saying, you know, go find this thing in all the vast halls of your long term memory, and you send some little probe out into long term memory that may or may not find it.
If I say is this in there.
That's an obviously simpler matching kind of task.
And you are typically much better at doing recognition than you are at doing recall.
So you have presumably special purpose mechanisms in your system that are really good at doing some tasks.
What do you do in the case of those unfortunate tasks.
Actually I don't need that anymore at all.
In those case of those unfortunate tasks which you were not well designed to do.
For instance the memorization of neuroanatomical terms for a midterm in intro psych, which is only the beginning.
How many people over here are pre-meds or think they might be pre-meds?
You guys are the ones who are going to devote years of your life to the memorization of things that you were not built to memorize.
Like where all the bones are, and stuff like that.
So how do you do that?
Let's try memorizing some nonsense.
Go to sleep.
Why don't you want to go to sleep
[LAUGHTER]
It's like your children.
Mara, what do you do?
I don't want to know OK. You mean I'm supposed to hit it?
[LAUGHTER]
Oh OK. All right.
It went to sleep.
It happens from time to time.
Anyway where were we.
Oh yes nonsense.
This is a line of work that starts really with a guy named Ebbinghaus in Germany in the late 19th century, who wanted to understand memory processes.
He's in isolation from these issues about meaning.
And so what he did was he got himself actually for starters to memorize nonsense syllables that allegedly had no meaning.
It's extremely hard to find material that has no meaning.
But I will attempt to read you some now.
Grab a hunk of paper.
I don't want your writing them down while I read them to you, but I do want you to be in a position to write all the words down when I tell you it's time to write.
Oh while you're doing that, let me make my annual disclaimer.
MIT is a hugely multicultural institution.
You speak a vast range of languages.
To my knowledge what I am saying is meaningless.
If it turns out that it is meaningful, particularly if it turns out that it's like deeply profane, do let me know later and I will change my nonsense yet again.
I have over the years said a number of absolutely remarkable things.
Including something in Mali so good that the young women who told me I couldn't say it, wouldn't tell me what it was I had said.
But --
[LAUGHTER]
-- they looked at my list and they just crossed these things out.
Anyway.
So it's mostly gone.
So all right.
You're ready to write?
I'm going to read you a bunch of nonsense syllables, and you're going to write them all down.
Ready OK. Fip.
Dut.
Moche.
Yill.
Saz.
Tirt.
Varl.
Bince.
Jucks.
Gouf.
Zauce.
Rab.
OK. Write down everything you can remember
[UNINTELLIGIBLE]
Now don't copy off your neighbor.
On your hand out on the second page I believe, you will find the axes for what's called the serial position curves.
Which is the way we want to represent the data from this experiment.
We want to represent percentage recall, which in this case will be in sort of number of hands, as a function of position in the list.
Where in the list that I read did the word show up.
And I will tell you the answer at ahead of time, and then we'll check if it worked.
The answer is that the data will look like this.
There will be what's known as a primacy effect, where you'll remember the first words in the list quite well.
And a recency affect, with some preservation of the last words in the list.
That at least is my assertion.
Let us actually check the data here.
How many people got fib?
Alright.
Data point turns out to be about there I think.
How many people got dut?
Few, fewer.
How many people got -- well you see your falling a little below the line here.
I am worried about you guys.
How many got tirt?
Ouhh how interesting.
How many people got varl?
Ouhh that's pretty pathetic.
How many people got zauce.
Oh coming back.
Oh that's pretty good actually.
That's above the line here.
And how many people got rab?
That's sort of thereish.
OK. What's with tirt?
[LAUGHTER] You guys did bet it.
Well you know, teaching concourse this morning, I discovered that one of my words has currently acquired a slang meaning that I'd never heard of before.
So tirt you just got lucky on, or tirt suddenly means --
[UNINTELLIGIBLE]
What's a tirt?
What?
[UNINTELLIGIBLE]
Tert.
Oh oh tert like as in tertiary or something?
Yes
Oh, see I'm spelling it t-i-r-t, and it never occurred to me.
All right, all right.
We're going to have to do something about tirt.
Anyway.
OK.
The question here is going to be how did you do it?
Part of the answer is clearly any word that happened to make some sort of meaningful association to you is going to have a preferential chance of getting into long term memory.
Where did my beautiful little map of the world go?
It must be underneath there.
You go up.
One way to think about this is sort of by analogy to immigration.
You know here's America, and here are all of our various forebearances and stuff trying to get into the country.
And there's this choke point at immigration.
You know, if you got your Uncle Charlie already in the country, he can pull you in ways that if you have no relations here it's harder to get in.
So if tirt reminded you of tertiary, that's Uncle Tertiary in long term memory.
Sort of pulling you through.
But the question is what do you do if you don't have that?
So the things at the beginning.
Anybody got any intuition about what they were doing with fip and dut and stuff like that?
Yeah, Yeah,
I was writing down this fip and dut
Oh OK. Apart from the fact that you were -- no, no, you presumably were being a good person and not writing them down when I read them first though right?
OK.
So what were you doing while you were waiting to dump them back out?
Yeah
I mean I was going over them in my head.
Rehearsing them
Yeah.
OK. You were sitting there rehearsing.
How many people had the experience that they were rehearsing, and discovering that oh man there's more stuff here than I can rehearse?
That's the capacity limit again.
But there's a rehearsal loop for auditorium material sometimes called a phonological loop.
Does it say phonological loop on the handout anywhere?
No.
Well.
You know, phono like phonograph, phonological.
Where you repeat this stuff to hold it in this working memory.
Keep it alive in working memory.
And in that way sort of keep it alive.
Anybody got a different experience for zauce and rab at the end of there?
I'm sorry I heard something that sounded promising.
Yeah?
They were just there
They were just there is actually the right intuition.
It was almost like you could still hear them.
They didn't need to be preserved yet.
They were still fresh and hadn't rotted.
The claim though is that the process that gives you the recency effect is different than the process that gives you the primacy effect.
Let's give that a try.
What I'll do is I'll read you another list of nonsense like the first one.
I'm going to ask you to write them down again, but this time at the end instead of saying write I'll say count.
When I say count, I want you to count backwards from 431 by 3's out loud.
You know all in beautiful unison.
Right got that?
That would make a lot of noise, so when I do this, keep an eye on me.
When I do this, write everything down.
OK?
OK.
They kind of got the instructions.
All right, where's the rest of my nonsense here?
[UNINTELLIGIBLE]
OK. OK ready?
Sip.
Forth.
Lig.
Vop.
Hearn.
Mope.
Jick.
Tinned.
Mez.
Wamp.
Flob.
Gone.
OK. Count 431
431.
[UNINTELLIGIBLE]
OK. Write them all down.
The giggles would probably do just fine
[LAUGHTER]
OK.
The claim here is that what that should do, did you feel yourself rehearsing the first ones again?
Yeah
The claim is that the primacy affect, which is due to some sort of rehearsal into long term memory.
That that primacy effect should still be there.
But the recency effect due to some sort of it's just thereness, should have been disrupted by the counting and the chordals and stuff like that.
OK.
So let's check here, where's my list gone to?
Here list.
I should remember this by now.
OK. How many people got sip?
And look.
That's almost exactly the same number.
How many people got forth?
Ooh.
A lot of forths.
How many people got jick?
Ouhh deeply pathetic
[LAUGHTER]
How many people got mez?
Few more.
How many people got flob?
Ouhh that's pretty pathetic.
How many people got gone?
No gone?
Gone was pretty well gone.
These were supposed to be circles, So they'd be a different data set.
So anyway, nice primacy effect.
Recency effect gone
[UNINTELLIGIBLE]
Yeah.
There are other words in there.
This is the phenomenon of MIT students really wanting to do well on these sort of things right?
[LAUGHTER]
Man, he didn't ask me.
It's like, you know, you have the same experience for keeps on the exam.
You sit there and you study chapter 3 until you're blue.
And oh man they only asked about chapter 2.
So yeah, yeah, yeah.
I'm glad you got whatever it was wamp or --
[LAUGHTER]
The worst thing about this you realize is that the stuff that you rehearsed into long term memory may stay there for a surprisingly long time.
I had a guy come up to me on the subway some years back.
Say to me, you don't know me right?
I said that's right I don't know you.
He said fip, dut
[LAUGHTER]
I said you are a very strange person.
But I now know where I know you from, or don't you from or something like that.
You may have now warped your brain on ongoing basis.
No.
I don't want you to go up.
Get back here.
Stop.
Come back down.
You go up.
All right.
So back on the first page of the handout, you have what was sometimes called a standard model for getting from the outside world -- whoops my bottleneck disappeared again -- into long term memory.
Which was stuff comes into a short term memory.
It needs to be preserved in short term memory by something like rehearsal.
If it lasts long enough in short term memory, it gets into long term memory where it manages to stay.
There are certain problems with this kind of a model, which you ought to be able to anticipate now.
But we can illustrate easily enough.
How many people here had breakfast?
OK.
Person with the MIT shirt on there.
What did you have for breakfast?
Bagel
Bagels.
Anything else?
Bagel and cream cheese
Bagels and cream cheese.
Oh OK. That's good.
We want a little bit of a memory load here.
Because I need to be able to explain that the reason that she knows that she had bagels and cream cheese for breakfast is that on the way out of her living arrangements, she was going bagels and cream cheese, bagels and cream cheese, Bagel and cream cheese
[LAUGHTER]
Right?
OK. That's good.
No.
She wasn't doing that.
You weren't doing that, but you remember what you had for breakfast too.
Obviously this rehearsal thing while it's important for getting nonsense from the world into long term memory, is not the only way to get through this bottleneck.
Again things with meaning manage to get there more readily.
And we can see that that fact tells us something perhaps about the way that our long term memory is put together.
Now here what we're talking about is what's know as explicit long term memories as distinct from implicit.
Explicit long term memory are the memories that you can get to if I ask you about them.
That you can recover.
So if I say, you know, who's running for President, you have an explicit memory that the answer is Kerry/Bush and whoever else you can name down the line there.
If I asked you how do you say the word president, what are the tongue motions that are required.
If you say president, you can now monitor it and figure out what they are.
But you have no notion ahead of time of how you do that.
You certainly remember how to do it in the sense that you can produce it on demand.
That would be an implicit memory.
And how you ride a bicycle.
There are a whole slew of things like motor memories that are implicit.
But here what we're talking about is getting into an explicit long term memory.
It's useful to divide explicit long term memory into episodic memories for the episodes of your life, and semantic memories, your body of knowledge and facts and things of that sort.
There not like walled off from each other.
If I say, what's the capital of the United States, you dig out of your semantic memory the answer Washington DC.
If you've been to Washington, you might then start digging out associated episodic memories of being there and so on.
But within semantic memory, the notion here -- remember the Uncle Charlie or whoever he was notion, that things can reach out to the world and help pull things into long term memory -- is related to the way that we think that your long term up memory might be organized.
And one of the popular ways of thinking about this is sort of a giant network of associations called a semantic network often.
If I say cat, the first word that comes to your mind is?
Meow
Meow
Dog
Dog
Mouse
OK.
So there's a note in your long term memory that is cat in some sense in this story.
It's close neighbors are things like dog and meow and -- what did we have -- mouse.
And now it's important to realize this is not some sort of Linnaean taxonomy of, you know, species and geneus and all that sort of good stuff.
You know, even though it's going to be connected with animal up here somewhere, animal and fur and stuff.
But you might also have connections that go like meow, mow
[LAUGHTER]
Actually that goes back here.
I actually have a cat whose name is Chairman Mao by roughly this association
[LAUGHTER]
You know, then, you know, China
[LAUGHTER]
Plates
[LAUGHTER]
So, you know, nobody has the notion that you could somehow map this out for any given individual.
It's going to be a vastly complex and continuously changing set of associations.
But there is evidence for this notion of proximity by association.
By meaning in long term memory.
And one of the ways you might find that out is by doing what are called priming experiments, which I might as well mention because I see that's -- is that on the handout properly.
Well if it isn't, it should be.
It probably says priming somewhere.
Priming experiment.
Evidence from priming.
Look at that.
On my computer screen here, I'll put up a string of letters, and you tell me whether or not it's a word.
Right?
So I put up.
And you say?
[UNINTELLIGIBLE]
No.
Even though it's now in your long term memory right.
But if I put up.
Yes.
OK now.
If the next word was something like truck, unless you have a dog in your truck or something like that.
Well all right, let's compare this.
The next word might be truck, or the next word might be cat.
You'll be faster to say cat.
You'll be faster to confirm that cat is a word than truck is a word.
Why is that?
Well seemingly when you go in to your semantic network to discover that dog is a word, you light up the dog node, and the activities spreads -- it's called spreading activation -- away from that node to the neighboring nodes.
As a result, when you get the word cat, cat's already a little activated.
The bell's been rung a little bit already.
And your quicker to confirm that cat is a word than truck.
Which is you know, down here in, you know, in the next county somewhere.
In principle I suppose you could map out somebody's semantic network with a technique like this.
But it would be a lifetime's endeavor ever, and not clear what the point is.
But you could you can use this sort of a technique to show that some things live closer to each other than other things.
Oh and by the way, when you go and reach in to retrieve -- this is sort of jumping to the retrieval part -- but when you go reach in to retrieve something from one node, you might goof and pull something from the neighboring node.
And you know this, or you have experienced this if you are not an only child perhaps, even if you are.
How many do you have siblings?
Keep your hands up.
How many of those have ever had your parents call you by the sibling's name?
Almost everybody.
Including some people who didn't have their hands up before
[LAUGHTER]
Who didn't know that they had siblings until that happened.
Or worse, you know, you get called by the name of the dog or something like that.
This is not because whatever you may believe, you know, that your parents are unusually dim people.
It's because all those terms presumably live very close to each in this semantic memory.
And when particularly under some pressure you reach into grab kid number -- all my kids have the same name.
Their name is Ben Phillip Simon, whatever your name is.
And it works for all of them.
Now there are certain drawbacks to this.
It turns out to be decidedly unfortunate if in a moment of passion you murmur the name of the last girlfriend
[LAUGHTER]
No matter how close they live in your semantic network.
We can explain it, but we can't excuse it if you know what I.
So big world, small desktop, small working memory, big long term memory.
What's going on when the stuff in long term memory gets turned into something long term?
It's really there for a long, long time.
If you get a chance, David Pillemer's article that's in the website under the memory category about very long term memories, is fun to read as he goes and probes people's oldest memories from way, way back.
How does stuff get consolidated.
Well we know some things about this.
We know quite a lot about this.
But I want to tell you one bit, a couple of bits, that are important on a sort of a physiological front.
So here's the brain again.
Underneath the temporal lobe, not in the cortex of the temporal lobe, but underneath the temporal lobe struc the hippocampus, and to a lesser extent the amygdala.
I'll just mention.
But really we're talking about hippocampus here.
Parts of the so called Limbic System that are vitally important in this process of consolidation.
And we know that in the first instance from perhaps the most famous patient in the neuropsychological literature.
He's a patient known in the literature as HM.
He's the inspiration for the film Memento if you saw Memento.
When he was a young man he had a bike accident that was probably the cause of his epilepsy.
So he was an epileptic.
You may recall I said earlier in the course that epilepsy is a sort of an electrical storm often started from a damaged piece of tissue in the brain.
If it's not responding to drugs, which this was not, one of the treatment is to go and try to excise, to take out, the generator.
The evidence in HM's case pointed to structures deep in the temporal lobe.
Particularly the hippocampus and amygdala.
And what was done in the late 50's, in the late 50's or early 60's, anybody remember?
Anyway a long time ago, at this point, up in Montreal was that these structures were largely destroyed on both sides of his brain.
That's important.
This is a bilateral lesion.
And this did have the effect of largely controlling his seizures.
The seizure problem went away.
However, he's a one of a kind patient, because the side effects, the unintentional consequences of this lesion, was so devastating that nobody could ethically ever do this again.
At least not to a human patient.
So he's still alive.
And in fact, he's in a nursing facility near Boston.
Because he was brought to Boston essentially as a psychological subject.
Been unable to live on his own since the surgery.
What's his problem?
Well, he really has two.
But the most relevant one for the present purposes is that he is simply unable to form new explicit long term memories.
Take his interaction with a once upon a time TA in this course name John Gabrieli now a Professor at Stanford soon to be a Professor back here at MIT.
But who did, I think he did his doctral work studying HM.
But he certainly studied HM as a graduate student.
He'd come into the lab to meet HM.
Say hi, you know, have we've met before.
HM said no.
I don't think so.
He's perfectly intact short term memory, which is fine for conducting a conversation.
Also decently intact memories, this long term memory from before the operation.
So it's not that we've somehow wiped out his storehouse of memories.
It's the ability to put new stuff in here that's gone.
So in comes Gabrieli.
Have we ever met before?
No.
Hi I'm John Gabrieli, you know, it's nice to meet you.
I'm HM, well I suppose he didn't call himself HM, but anyway.
You use initials to protect the privacy of the patient.
Not necessarily the initials of the patient.
Something to label them.
So anyway, fine have a nice conversation.
Gabrieli goes out.
Comes back in.
Says hi, have we ever met before.
No I don't think so.
Well hi.
I'm John Gabrieli nice to meet.
Have the same conversation.
You can do this over and over again.
In a more control kind of way, do a digit span kind of experiment.
Give him a few digits to remember.
No problem.
He's got the same 7 plus or minus 2 that the rest of us have pretty much.
Distract him momentarily, and it's just gone.
You've had this experience yourself if you try to get from the phone book to the phone with a phone number.
And somebody says, didd you do x, y and z?
Ahh.
There goes the phone number
[LAUGHTER]
That's his continuous experience, and has been for 50 years at this point.
A few things have gotten in.
He happens to know that men have landed on the moon for instance.
Perhaps because they're a little bits of the structure that are maintained perhaps from another route.
We're not quite sure.
But basically no new episodic memories.
What this is telling us is that the hippocampus in particular, other research indicates is critical for the act of consolidation.
It is not the memory itself.
It is the mechanism that allows you to make those memories solid in some fashion.
Yes?
But, does he know he has this disorder?
He knows he's got a memory problem.
And this actually is a good point.
He ties into his other problem, which is in some sense an off setting benefits.
The Limbic System, certainly the amygdala, very important in the mediation of emotion.
His experience of emotion is greatly flattened.
Flattened affect in the jargon of the trade.
So.
Which is good.
Because he does know that there's something wrong, but it doesn't bother him that much.
You know what bothers him?
What bothers him is looking in the mirror.
Now why should that be?
Well imagine this.
His lesion happened when he's in his 20s.
He's still got the long term memories that he had in his 20s.
Basically your age.
Now you go back to your room, you look in the mirror, and what looks out at you is a 70 year old guy or women as the case may be.
I suppose it doesn't particularly matter.
It's going to be a disaster under either circumstance right?
You're going to think I'm in a sci-fi movie, and I don't have the part.
You know the good guy with the laser gun part that I really wanted.
You know I'm in big trouble here.
And this is disturbing to him.
Not as disturbing as it would be to you again, because his emotional responses has been very severely blunted.
So much so that he needs to be asked whether or not he's sick.
You know do you have a stomach ache?
No that you mentioned it, yes I do.
It's not that he couldn't feel the pain, it's just without the emotional overlay so, you know.
I see blue.
I have pain what's the big deal?
[LAUGHTER]
Yes?
[UNINTELLIGIBLE] motor memory
Yes.
Nice point.
So for example, I don't know how many of you have hung out at -- you know get rid of the brain here.
Who needs the brain?
So any you ever done a mirror maze?
Some people may have actually built one of these once upon a time.
So imagine you have like a star.
Oops.
Imagine you could actually draw a star
[LAUGHTER]
All right.
Well.
So you make this out of like aluminum foil, and you go and trace it with a metal stylus.
And if you go off the aluminum foil it makes a nasty noise.
Because you're breaking a circuit or something like that.
Anybody ever build one of those?
Oh what's the world coming to.
It's easy right?
Anybody can do this.
What people can't do regular readily is do this looking at the image in a mirror.
Right?
Because then think they got [UNINTELLIGIBLE]
[LAUGHTER]
But you learn to do this.
And HM has been trained in one set of experiments to do this.
In multiple sessions he got better.
If you asked him have you ever done this before on the nth time that he did it.
The answer is no.
I don't think so.
And he goes [UNINTELLIGIBLE].
Hey, you know, that was really good.
Most people asked how did you do that?
Yeah.
I've always been good at that kind of thing.
Or, you know, just lucky.
No episodic memory, no explicit memory of ever having done it.
But yes a motor memory that allows them to do it.
Similarly, your sense of familiarity runs on different pathways.
So if I bumped into you wandering around the halls, I may well have the sense that I've seen you somewhere before.
And I might guess that that's, you know, I've seen you in intro psych, because that's where I see a lot of people whose names I don't really quite know and stuff like that.
So that would work.
And HM gets that sense of familiarity too.
So after a while actually with Gabrieli, Gabrieli comes in, have we ever met before?
Yeah.
You looks familiar.
Do you know my name?
Know you're name?
Forgotten your name.
Who do you think I am?
I think you're probably somebody from my high school class.
Now why would that be.
Well Gabrieli at the time was a young 20 something year old guy.
This guy's got a 20 something year old long term memory.
A reasonable guess of somebody who looks more or less familiar, but isn't in my current collection of people I know well.
It's yeah maybe from high school, or something like that.
We're about the same age as the 70 year old HM to the 20 something year old Gabrieli.
So those sort of things he does learn.
It's that new explicit long term memories that require the hippocampus to be consolidated.
How long does this take?
Let me say a quick word about that, and then we'll take a quick break.
You can measure the time course of consolidation by doing an experiment of the following sort.
You can draw bad rats.
OK there's a rat.
He's on a little pedestal.
If you were a rat on a small pedestal not too far above the floor, what are you going to do?
Jump off
Jump off, right.
I'm a rat.
Man I got to go look around at stuff.
So I jump off.
Well what they didn't tell me back at home was that this particular floor is electrified.
So when I jump off my little toes get an unpleasant, not damaging, but an unpleasant shock.
So now the guy puts me back on here.
What do I do?
Jump off
I don't jump off.
I'm not a stupid rat.
I'm just a rat.
And so I stay there.
OK, well.
I'm a rat who's in an experiment.
And the question is what percentage of the time do I jump off?
And the variable here is after I jump off, and after I get my toes shocked, I'm also going to get a dose of what's called electroconvulsive shock.
Which is basically electrical current run through the brain.
You know you can kill people or animals that way.
Obviously that's not the point here.
This is to disrupt the on going electrical activity of the brain.
Not to scramble the structure.
Not to cook proteins.
But just to disrupt any ongoing electrical activity.
So this is the delay to ECS.
Rat steps down when does he get his ECS?
If he gets the ECS immediately, put him back on the platform.
Basically a 100% of the time the rat will step down the next time to.
Why?
Not because the ECS is like fun or something like that.
But because the ECS has wiped out the memory.
The memory is not consolidated and the rat simply doesn't know that his little feet got fried.
And he steps down again.
As the delay gets longer, you get an essentially exponential looking curve with a time course in this case of seconds that represents the time of consolidation of that memory.
At least to the point of supporting the behavior of not stepping down.
but it's probably better to think it's not the case that all memories consolidate within seconds.
There's evidence from human studies.
Now why would you do human electroconvulsive shock studies?
The reason is that electroconvulsive shock in humans is electroconvulsive therapy
[LAUGHTER]
It sounds very odd.
It sounds very odd.
And those of you who've seen the now quite old movie to kill, Not To Kill A Mockingbird, One Flew Over The Cuckoo's Nest.
One of those bird movies.
One Flew Over The Cuckoo's Nest probably have a dim view of electroconvulsive therapy.
But that's badly earned.
Int he case of intractable both depression.
Depression that's not being broken up successfully by medication or by psychotherapy.
Severe depression of that sort carries with it a very serious risk of mortality from suicide.
So this is not a, you know, casual disorder.
It turns out that electroconvulsive therapy is often very effective in breaking up such depressions.
And is actually is useful part of the spectrum of treatments that are available in this case.
Like most medical treatment, it's not entirely cost free.
It is the case that there are memory consequences of electroconvulsive therapy.
It's part of what actually gives it a somewhat bad reputation.
And you can actually see those memory consequences.
You can see traces of them going back years.
This is in studies for instance of memory for TV shows seen only once.
You know sort of thing that you might dimly remember under the best of circumstances.
These rather limited kind of memories can be damaged by a electroconvulsive therapy years later.
As if this exponential has a very long tail.
And when you want to call it consolidated is perhaps a function of may be something like how important the memory is.
If I do this again, I'm going to be hurt right now.
That memory can be made solid enough relatively quickly that you don't go and do it again.
On the other hand, if the memory is, was it Bridgette Loves Bernie, or Benji.
All right that kind of good.
You know no stakes kind of memory is apparently still vulnerable.
At least still partially vulnerable going back for a very long time.
The hippocampus seems to be behaving like something like scaffolding.
It holds the bits of the memory in place until it's firm enough to stand on its own.
And then you can really call it in the sense of sort of a permanent memory.
Now that doesn't solve the issue of getting stuff out of memory.
And I will say a word or two about that momentarily.
But first you can wipe out your short term memory by stretching for a second.
[SIDE___CONVERSA TION___WITH___STUDENT] OK.
Looking at the handout, let me just make sure that I explained the jargon here.
So a loss of memory as everybody knows is an amnesia.
A loss of memory for events prior to the point of injury would be a retrograde amnesia.
That's not what HM has.
HM has an anterograde amnesia.
A problem with his memory subsequent to the lesion, subsequent to the injury.
We won't do this is a demo, but if I were to take a baseball bat and pop her on the head.
Well apart from the fact that I probably wouldn't be back on a Thursday
[LAUGHTER]
When she came to, she might not remember having actually been hit, because the set of memories from immediately before that whop would have been wiped out in a retrograde amnesia.
It is also possible that she wouldn't remember things for a period of time after she had regained consciousness, if I managed to knock her out completely.
There might be a period of time after she regained consciousness before her memory would -- she might have been talking and wandering around and say, I'm fine, I'm fine.
But have no recollection of that either.
That would be an anterograde amnesia.
Sorry I didn't really mean to hit you on the head.
You'll be fine.
So let me say a few words in the remaining time about retrieval from memory.
Like any of these topics, there's a great deal to be said.
But probably the most important thing to understand about retrieval from memory is that retrieval particularly if any rich sort of memory is not the recovery, the replay, of the tape for some true guaranteed image of what you experienced or what you learned or something like that.
This sort of image here of a scaffolding holding together that memory is also an image of a notion that any memory is probably stored multipley and in different bits of brain.
Different aspects of different bits of brain.
And if you want to recall it, that what you're doing is reconstructing it.
You can probably get some intuition about this if you asked yourself about famous stories about you in your childhood that are famous in your family that you remember.
Not just the stories they tell about you from when you hit with the bat and stuff like that.
But stories that you remember is sort of shaping incidents in your childhood.
But that also get retold every time the family gets together, right.
So in the Jewish tradition is a holiday called Passover.
The tradition is that you drink four cups of wine.
It's not really a brilliant tradition if you happen to be about four years old.
And I vividly remember my sister getting into the nice sweet wine that you drink and conching out at the table.
And I know whose house we were at.
This is a tale that has been retold to my sister's no doubt the light every year for the intervening 40 plus years.
And it is absolutely unclear to me given what I subsequently learned about memory, whether what I am remembering is the original event at this point or reconstruction based on all the family stories.
I think I can still remember the original thing, but there's no real way of knowing.
What you are remembering is a reconstruction.
Before I talk more about that, let me load you up with some more words.
These aren't going to be nonsense unless I can't find them.
There we go.
There's some nice words.
Which words do I want to use.
Oh these are good words.
OK.
I'm going to read you a list of words, and then we'll do a recognition test.
Don't have to write anything down this time.
I'm going to read you a list of words, and you're just going to tell me the word was on the list or the word was not on the list.
Ready?
So try to remember all of these.
Fury.
Rage.
Enrage.
Carpet.
Club.
Emotion.
Ire.
Mountain.
Fight.
Fear.
Mean.
Mad.
Place.
Hate.
And road.
OK. You got all those nicely stored in memory?
OK.
So let's see, where's my test list here?
Looks like my test list.
OK. Did you hear the word fight?
Yes
Science?
No
Rage?
Yes
Fear?
Yes
Emotion?
Yes
Wrath?
No
Club?
Yes
Anger?
[UNINTELLIGIBLE]
Well.
Road?
Yes
Ire?
Yes
Hatred?
No
OK. How about enrage?
Yes
OK. so there were a couple in there where you could hear people hesitating, or you could hear a diversion of opinion.
And those are the critical elements there.
The most critical element in this case is the word anger, which a fair number of people asserted it was on the list.
It was not in fact on the list.
And even those people who were asserting that it was not on the list, there was a measurable what's going on there.
This is in effect known as the Deese-Roediger-McDermott effect or on the handout the Deese-Roediger-McDermott demo.
Roediger and Mcdermott discovered it.
And then like all good phenomena in psychology discovered that Deese had published it 10 years prior.
And so it now goes by all of their names.
In any case, what they did, they were deliberately looking for this, what they did was they -- where did my semantic network go?
Oh here's my semantic network.
What they did was they took a word and they asked people to give the associates to it.
So let's take the word anger.
What words come to mind when you think of the word anger?
Well fight, rage, enrage, emotion, ire.
This collection of words.
You then use a training list of words that has the associates, but not the target word.
And what you're in effect doing is saying, animal, meow, mouse, dog, fur.
And each time you do that, the spreading activation gets all sorts of stuff, but all of these are pointing towards cat.
And then later I say well was cats on the list?
And you go and look at that note and lo and behold it's glowing softly in your mind.
And you say, yeah, yeah.
That was there.
And if you take one of other, it didn't particularly work in this case, ire sometimes works for instance.
You take a word like ire people are no more sure that ire was on the list than they are that anger was on the list.
People are thoroughly in these experiments confused about whether or not the target word was on the list.
Because you don't have access to the truth about your memore.
You have the access to what you can manage to reconstruct about it.
Now that's fine.
That sounds benign enough as a lab demo, but let's suppose that you're not just sitting in some nice intro psych class, but you're sitting in the witness box.
And somebody is saying, where were you on the night of May 5th?
And expecting you to tell the truth.
Well of course you're going to try to tell the truth, but the truth in this case is going to get shaped by the nature of your particular semantic network, among other things.
Let's talk about that part a little bit first.
Famous experiment done first in the 40's replicated many times since in many different ways, but let me describe the original version.
You, and in this case you are a white male, are shown a depiction of an altercation on a subway.
Two guys get into a fight.
In the 40's it was done as a hand drawing cartoon.
It's been replicated with, you know, snazzy video.
It doesn't matter.
Two guys get into a fight, or get into a sort of shouting kind of argument.
At some point one guy pulls a knife, and then sometimes subsequent we'll ask you about this, OK. Well the question you're going to be asked is who pulled the knife.
And the variable of interest is the race of the participants.
So you got a nice little 2 by 2 here.
They can both be caucasian, they can both be African American, or it can be crossed.
The interesting and disturbing finding is the number of times when the knife is pulled by the white guy, it ends up in memory in the hand of the black guy.
Now as I say, this has been replicated all sorts of different ways.
This is not a, you know, white guys are all biggits experiment.
This is we all have biases, and the interesting and disturbing thing is that they can actually interfere with what we think of as, you know, our nice clear memory.
These people weren't sitting there saying, you know, oh yeah I'm being paid, you know, 1940 $0.10 an hour to be in this psych experiment.
Why don't I see if I can stick it to an imaginary black guy.
No.
They were presumably being as honest as they could be.
But the structure of their memory.
Which included things like who do I think might pull a knife, influenced what they recalled, what they pulled out of memory.
And we can impose this structure from the outside as well.
So suppose we do the following experiment.
This is an experiment also done many different ways at this point.
Elizabeth Loftus is the name associated with this line of work.
You're going to see another film.
This time you're going to watch a car crash.
The red car is going to get into an accident with the blue car.
All right.
OK. You see this.
You're going to be asked about it.
We'll show all of you guys the same film.
And now we'll ask the question three different ways.
At what speed did the red car bump into the blue car?
At what speed did the red car hit the blue car?
At what speed did the red car smashed into the blue car?
That's the only manipulation here.
You all saw the same video.
So what do you figure the difference is?
What's the result look like?
So who gives the higher speed responses?
The smashed guy
The smashed guys Lawyers know this.
This is why courts attempts to avoid leading question.
But it's very hard to avoid.
But, you know, that's a pretty subtle kind of lead.
But it's not a subtle kind of effect.
It's about as I recall a 15 MPH kind of effect, which is a sort of thing that has an impact, you should pardon the expression, on what a jury is going to think about whether or not you, the driver of that red car, was at fault or not.
If you were screaming through the intersection at 45 MPH that's worse than if you're screaming through the intersection at 30 MPH or something like that.
So your memory can be influenced by the way that you are asked about that memory.
This shows up not just on the stand.
Well actually a version of this shows up beautifully in the experience of most people at some point who come to MIT.
Most people who get to MIT are smart people, and they would do well to remember that.
Because most people sometime, like within the first few weeks of being at MIT, receive a data point of some sort that suggests that this might not be true.
Right?
Like the first calculus test or something like that.
You've never gotten a score below 99.8 on a math test in your life.
You get back the first test in 18 or whatever it was, and you know, 10 cool.
Oh man it's not 10 out of 10 is it?
[LAUGHTER]
You know this is really bad.
And so that's a depressing data point.
But a surprising number of people come to the conclusion from that, that they not only are stupid but always were stupid.
And probably unlovely and generally, you know, bad, horrible individuals.
And you should remember this is one of these sort of context effects.
That may be a little over stating it.
But look you know -- and this has actually been tested with clinical populations of depressive people who go from being depressed to being undepressed.
Suppose you ask somebody who's depressed clinically or otherwise, you wake up in the morning it's pouring cold rain out there.
The problems that did you get done because you fell asleep in there, and there are now drool marks on it.
And not only is there drook mark, there's a little heart shaped note next to it saying, I'm leaving you for your roommate
[LAUGHTER]
Right?
So if I asked you at that point how was your childhood?
Not how do you feel, but what kind of childhood did you have?
You had not maybe a miserable childhood.
But to exaggerate, you know, you grew up in a closet.
Later on if you're feeling better I ask you about the same childhood.
The childhood has magically improved.
All right.
We'll see you on Thursday if I remember correctly.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license, and MIT OpenCourseWare in general, is available at ocw.mit.edu
Good afternoon.
Before I forget, today's topic is cognition.
And in part, a good chunk of this lecture is going to be based on work done by Kahneman and Tversky, who won the Nobel Prize for a bunch of this work some years back.
A couple of years back, now.
The reason that is of particular interest is that Danny Kahneman is here tomorrow speaking.
E-51, 4 o'clock, right?
So if you happen to be a, interested in the material and b, at leisure at 4 o'clock tomorrow, I'd go hear him.
He's a very good speaker and there aren't that many folks who I get to talk about in psychology who win Nobel Prizes.
So, you should go hear him.
That's one characteristic of today's lecture.
I realize that the other characteristic of today's lecture is that I get to tell you a bunch of things that are applicable to the forthcoming election.
And to politics in general.
This particular swath I'm cutting through, the large area of cognition, includes a bunch of topics that if you've watched the debates thus far, or if you go and watch the debate that's Friday, right, the next one's Friday evening, you'll be able to see some of the stuff I'm talking about today in action.
There are two large parts to what I want to talk about.
And this is described, at least in rough detail, in this abstract at the top of the handout.
We can make the distinction between the way, using the computer as a model for how the brain works is sort of a common game, rather like using the camera as a model for the eye.
There are important differences.
One of the important differences that I'll talk about, that I started talking about last time, is that if you call up a document off your laptop, you are expecting that thing pulled out of memory to be exactly what you put into memory.
As we saw, as we saw towards the end of the last lecture, that's not true about human memory.
I'll elaborate on that a bit.
The other thing you expect is that your computer will do mathematical and logical operations correctly and quickly.
And that also turns out to be not particularly characteristic of human cognitive behavior.
For interesting reasons that I will describe in the second part of the lecture.
So let's start with this issue, this point, that says that recall is not just taking stuff out of storage.
Last time, I talked about a couple of examples of that.
And those are the ones labeled as old on the handout.
There's the context of remembering.
If you are depressed, your memories of childhood are more gloomy than if you are in a good mood.
There is retriever's bias, that was the example I was giving last time, with these experiments where the knife in a simulated altercation that you see, the knife might move from one person's hand to another person's hand in your memory, depending on the particular biases and associations in your memory.
Now, let me tell you about a few more ways in which memory is not just like pulling a document off the hard drive somewhere.
One of these is what I've labeled on the handout as failures of source monitoring.
Source monitoring is what we're busy making sure you do correctly on your papers.
Telling us where the information comes from.
And evaluating the information on the basis of where it comes from.
It turns out that that's not as easy to do as one might think it ought to be.
Indeed, I got this quote from Euripedes on the handout that says man's most valuable trait is a judicious sense of what not to believe.
If you hear something that's clearly false, is it possible for you to not be influenced by that.
This turns out to have been a question that philosophers have worried about.
Descartes thought that the answer to the question was yes.
If you hear a statement that's clearly false, you can label it as clearly false and, in a sense, put it aside.
Spinoza, in contrast, thought that it would be very nice if that were true.
But he doubted that it really was.
This has subsequently been tested experimentally.
And I don't have enough colored chalk around, but I'll simulate it.
Here's what Dan Gilbert and his colleagues did a few years back.
They did an experiment where you read little biographies of people.
And the material you're reading is color-coded.
And so you're reading along.
And then some of it, here, we'll use straight lines for a different color.
Some of it is red.
If it's red it's a lie.
Don't believe it.
But if it's green, it's true.
And if it's red it's a lie, and true, and so on.
Read a paragraph of this stuff.
We're going to make sure that you've read it because we're going to test you afterwards and make sure that you actually know what it said.
Because it's very boring to discover that you can ignore lies if you don't ever hear the lies.
But we know you've heard these various lies.
And then what we'll do is ask you about this person.
The metric they were using was a scale between college student and criminal, I believe.
And you were asked how criminal you thought these particular people.
So you're reading along.
And so-and-so is taking 1803.
And steals calculators.
And does the problem sets reasonably reliably.
And then fences the calculators.
And so on, right?
The result is that even though you know that the statements are false, your opinion of this hypothetical person is lowered by negative lies.
And, reciprocally, raised by positive lies.
You can't completely discount the information that you're hearing, even if you know it's a lie.
Now, if it is not immediately obvious to you that this has direct relevance to the current political campaign, you can ask yourself about the behavior you can see in this campaign, or in any other campaign that you ever care to subject yourself to.
Which is, you get charges that are made, typically by surrogates of the candidate.
So a clear example, this here would be the charges about Kerry's Vietnam behavior, brought by the Swift Boat people, not directly by the Bush campaign.
And I should be able to think of a flip side.
Oh, the National Guard stuff for Bush.
Regardless of the truth, we're not going to get into the truth assertion of this.
But, in both cases the really nasty attacks are raised by surrogates.
If it is then proved to be a lie, then the candidate can have the great and noble opportunity to run around the country saying, look, people should not be saying that John Kerry molests small animals.
It's a lie that he molests small animals.
Anybody who says that should given a dope slap.
And I would never say anything like, John Kerry molests small animals.
And I will tell all of my people not to say that John Kerry molests small animals.
And you can go home with this righteous feeling around you saying that I've done the right thing of condemning this smear.
Not only that, I've spread it around a bit, and the dumb suckers are going to believe it.
It's not going to swing the election one way the other, but it's going to push things.
And political campaigns do this absolutely routinely.
Perhaps not out of a completely Machiavellian understanding of the psychology, but because they know it works in some fashion.
I don't know that they're sitting around reading Dan Gilbert's papers to figure this sort of thing out.
This has applications elsewhere in your life, too.
It's very important to understand -- I'm going to take zinc because it's going to keep me from getting colds.
Now, why do I think that?
Is that because my aunt June, who's the medical wacko of the family thinks that's true?
Or is that because I read it in the Journal of the American Medical Association?
It makes a difference where the information comes from.
So, some sort of an ability to evaluate the source is important.
And the fact that we're not very good at it is problematic.
A particular subset of the problems with source monitoring is what's known as [?
crypt ?]
amnesia.
And that's the problem where you don't remember where your own ideas come from.
The canonical form of this, in a setting like MIT -- actually, this is the part of the lecture for the TAs.
The canonical version of this is you go to your doctoral adviser with a bright idea.
Yeah, I got this cool idea for new experiment.
Your doctoral adviser looks at you, says, that's stupid.
And you go away.
You know, you're kind of devastated.
Next week, in lab meeting.
Your doctoral adviser has this really bright new idea.
It's your idea.
He thinks it's his idea.
He does not, he's being a -- probably not, being a slimeball to say I'm going to steal their ideas.
It's, you actually forget where it comes from.
And in the absence of a clear memory of where the idea came from, if I've got an idea in my head about my research, gee, I wonder where that came from.
It came from -- I think they come from my shower head.
Because all the good ideas I have always come in the shower.
But, most people tend to think it comes from them.
I just cite the shower.
But this is actually a very good argument, by the way, for taking copious notes in a lab book when you work in a.
Lab Because if it ever gets to the point of an argument, you want to be able to say, look, in this dated entry from October 5th, 10,000 and -- 2004, it might take you a long time to get your doctorate.
here's the idea.
I wrote down this idea, and then I wrote down this tear-stained comment where you told me I was a doofus.
But, anyway, I really do think this was my idea.
And I ought to be an author on the paper that you just submitted about it.
Anyway, it's a serious problem.
Another place where that shows up is there are recurring scandals in the sort of academic book-publishing world where some well-respected author is proven to have taken a chunk out of somebody else's book.
Or something very close to a chunk out of somebody else book, and publishing it.
Some people are doing that because they're sloppy, slimy people.
Other people are doing it because you read a lot of stuff.
Maybe you took good notes, maybe you didn't.
And then when you start writing, this clever phrase comes to you -- or maybe just a pedestrian phrase comes to mind.
But a phrase comes to mind.
And you simply have forgotten the source of that phrase.
Which is that you got it from somebody else's writing, and then -- it's a strong argument for making sure that you keep very clear track of where you're getting your material from.
And it's part of why we beat so hard on the notion of referencing in papers in a course like this.
That's what [?
crypt ?]
amnesia is, in any case.
So, those are failures of source monitoring.
It is also possible, to move on to the next one, to have memories that feel like memories but just are not true.
Again, Liz Loftus has done some of the most interesting research on this.
One of her experiments, another one of these car crash experiments, remember the experiment of, how fast did the car smash, or bump, or tap, or whatever it was, into the other car?
Well, here's another one.
You watch another one of these videos of a car getting into an accident.
And that part of the problem is that the car went straight through a yield sign.
Later, you are given information that suggests that it might have been a stop sign.
Can do this in a variety of ways, including ways that tell you that that is bad information.
Another source monitoring problem.
But, given later information that it was a stop sign, not a yield sign, people will incorrectly remember it -- not everybody, but you will get a percentage of people -- who will now misremember the original scene as having had a stop sign in it.
If you ask what color was the sign, they'll say red, even though a yield sign would be yellow, and was yellow in the in the actual video that you saw.
So you've recreated, you've now put together the true memory with some false later information.
And it's corrupted that true memory.
Now, that's of obviously some interest in terms of getting testimony in court and things like that.
The place where this has become extremely controversial and a real problem, is the issue, is it possible to recover memories that were suppressed?
Suppose something really bad happens to you when you're a kid.
Is it possible to suppress that memory and not have it available to you until some later point.
At which point you now remember this memory.
Is that possible?
The evidence suggests that it may indeed be possible to not think about something for a very long time.
Have it bubble up into your memory.
Sadly, the evidence is also strong that it is possible for stuff to bubble up into your memory that isn't true.
Under the right circumstances.
If what's bubbling up into memory is memories of, for instance, childhood sexual abuse, for which there were a large number of criminal cases about a decade ago, now, it's a really important question to know whether these are real memories.
And it is deeply unclear that there's any nice, neat way to know the answer.
Now, look, obviously you can't, one of the reasons it's very unclear, is you can't do the real experiment.
You can't take a little kid, abuse the little kid and say, let's come back 20 years and see if they recover the memory.
I mean, it doesn't take a genius to figure out that that's not going to work.
But, Liz Loftus has a very nice experiment that points in the direction of this sort of problem.
Here's what she did: you go and interview families about, let's take one specific example.
Getting lost in a mall.
Did so-and-so, the designated subject.
You are --
Olga
Olga.
Did you ever get lost in the mall?
[UNINTELLIGIBLE]
OK, good.
We won't ask her that.
But we'll ask her family whether Olga ever got lost in the mall.
And they'll say, no, no, no, it never happened.
OK, good.
Got a brother or sister?
Yeah
Good.
What have you got?
Brother
Brother.
So now we call her brother aside here, and say, we're going to do a little experiment here.
Go ask Olga if she remembers the time that she got lost in the mall.
And you do this with a lot of, Olga might look at her brother and say, no.
And that's the end of that particular experiment.
But some percentage of people, given brother who seems to think that this happened at some point, will, yeah, I remember that.
Well, tell me about it.
Well, you know how this story goes.
Even if you were never lost in the mall, we could ask Olga.
She could generate the story pretty successfully.
It'll be something like, well, you know, I was there with my parents, right?
And I was looking at these toys, like, in the window.
And then I followed these, I was little, I was really little.
So you don't get lost in the mall when you're 18 or something.
So, I was little.
I'm following these legs around.
And it turns out to be the wrong legs.
I thought it was my mom and it was somebody else.
And I started to cry, and then this big man came and he took me to the security guys.
And they announced it over the PA system, that I was lost.
And my mother came back -- people fill in really good, detailed stories of this.
That are apparently invented out of -- well, not out of really whole cloth, I suppose.
They're invented out of the understanding that you have of the possibility of being lost in the mall.
And the childhood fear of being lost in the mall.
But it's not a real memory.
Feels like a real memory.
Subjects in Loftus's experiments, confronted later with the reality that this was all a setup, were often quite surprised -- Sure I was never lost in the mall?
No, says Olga's brother.
Liz Loftus just told me to tell you that.
And, oh, ah, hm, that's a little weird.
But, the point is it is possible to generate things that feel like memories that are not memories.
Yes
What about remembering dreams, supposing something happened in a dream that could happen, and you remember that as a [INAUDIBLE]
If you get routinely confused about what happens in your dreams and what happens in reality, that's known in the trade as a failure of reality testing.
And is not considered a good psychiatric sign.
But, sure, those sorts of things are possible.
Particularly when you end up having desperately realistic kinds of dreams, the sorts of things where these, sort of, reality testing things do.
Some dirt is, how many people, I'll probably ask this again when we talk about sleep and dreams.
But, never mind.
How many people have had the dream of their alarm clock going off?
Dreamed that they turned off their alarm clock and got up.
Only to then discover, oh, man, I'm in bed.
And I'm supposed to be in my psych recitation or something like that.
Or, at least tried to give that a story to their TA.
So, yeah, there are possible sources of confusion like that.
The last item on this list of sources of possible confusion I think, says something like becoming plastic again.
This is new stuff which I will come to momentarily
[UNINTELLIGIBLE]
That's
Is there a way to distinguish between these false memories or if they've actually occurred?
[UNINTELLIGIBLE] If you don't know them [UNINTELLIGIBLE] comes and says, I think I was sexually abused as a child, is there a way to determine, is this, like --
There is, at the present, time no ironclad way to know that.
I mean, sometimes you can go and get evidence -- there's no ironclad way to know simply from the memory testimony.
There may be ways to find out by talking to other witnesses or something like that.
There have been various efforts to look, for instance, at brain activity and see whether or not there's a difference in the pattern.
So, you do this at the lab.
Create a false memory and a real memory, and see if you can see the difference somehow in a FMRI scan or something of that sort.
There are hints that maybe you can get there.
It's like lie detectors.
A fascinating topic for a whole lecture that I'm not giving.
There is no ironclad way of telling a lie from the truth by wiring anybody up.
Regardless of what the FBI tells you.
Actually, I shouldn't be telling you this, because the only reason that these things work is because people think they work.
If you think that the lie detector's really going to work, then if you're lying, you tend to get kind of more nervous than if you don't think, than if you're telling the truth.
And that's what they can monitor.
But a really good pathological liar, or a well-trained agent is apparently pretty good at getting past your average lie detector.
Lots of people, you can get serious defense money by having a brilliant new idea about how to tell truth from lie and real memory from false memory.
There's a lot of interest in doing that.
But, no ironclad way of doing it at the present.
Makes for great sci-fi novels, though, right what's going to happen when we finally figure out that we can actually stick this thing on your head and tell what your memories, whether your memories are true or not.
But at the moment it's still sci-fi.
Not quite sci-fi, but still very new, is the indication.
Remember, I talked about consolidation last time.
This process of making a fragile memory into something that will last for years and years.
There is evidence at least from rats in a fairly restricted range of studies, that when you recall a memory, when you bring it back out of, say, long-term memory into that working memory, that it becomes plastic again.
And that perhaps some of the distortions in memory can arise at that point.
Because the memory is again plastic.
It needs to be reconsolidated in order to be, in effect, restored.
So you bring it out.
It's now vulnerable again in ways that it wasn't before you remembered it.
My guess is that if you come back to the course ten years from now, that there'll be much more to be said about that.
That's new and interesting.
But the general point should be clear enough.
Which is that recall out of memory, retrieval out of memory, is an active reconstruction.
It has multiple pitfalls in it that might cause what you are pulling out as a memory to be distorted in some fashion or other.
Now, the second part, the larger part, of what I want to talk about today is the sense in which we are not simple logic engines in the way that our computers, most of the time, are.
Now, one way to think about that is to think about a distinction between what's called narrative thought and propositional thought.
Propositional thought, and this runs parallel.
Narrative thought and episodic memory.
Propositional thought and semantic memory.
They travel together.
So, propositional thought is the sort of thought that your computer would be good at.
Moving symbols around, is x bigger than y, and problem set kind of thinking is mostly propositional thought.
Narrative thought is thought that has a story-like aspect to it.
Perhaps an episodic memory type aspect to it.
So, an example of narrative thought might be do you want to go to the party that is happening on Saturday night at Alpha Beta Gamma house, or something like that.
How do you figure that out?
Well you probably don't do some symbolic cost-benefit analysis.
You probably imagine yourself there.
Would I be having fun?
Who's going to be there.
You'd run a scenario.
And that's the sort of narrative thought.
Turns out that narrative thought has a power in our own decision-making that is capable of running across, running over, the simple facts of the situation.
Even when we know the facts correctly.
Risk assessment is one example of this.
So, forced choice.
You gotta pick one of the other.
Don't just sit on your hands here.
Which makes you more nervous: taking a ride in the car or flying in an airplane?
How many vote for riding in the car?
How many vote for flying an airplane.
As usual half the people can't figure out that forced choice, pick one or the other, means you have to raise your hand for one of these two options.
At least if you're moderately alert.
But anyway, plane wins big-time.
How many people, let's just ask the propositional thought, or the semantic memory nugget.
Which is more dangerous?
Flying in a car, that's only -- that's the Harry Potter version.
Flying a plane or driving a car.
How many vote for plane?
How many vote for car?
Everybody knows the proposition here.
It's riskier to, you can measure this in all sorts of different ways.
Per mile, per time.
It's it's riskier traveling in a car than traveling in a plane.
But people are much more nervous about planes.
Why is that?
There are multiple reasons.
But perhaps one of the driving ones is that the narrative surrounding planes, and particularly surrounding plane crashes is sufficiently vivid that it overrides your knowledge of the mere facts of the matter.
If and when a plane sadly goes down, that's big news.
A lot of people tend to get killed.
It makes big news in the papers and on TV.
Highly salient.
The fact that there's a steady drumbeat of car crashes, unless it happens to be very local to you, doesn't have any particular impact on you.
And the raw statistic doesn't much matter.
In the same way, this turns out to be true in course catalog land.
So that the MIT course evaluation guide, when it comes out, has vast piles of cool statistics.
What people thought of this particular course.
And then it tends -- I haven't seen the most recent one.
But typically it has a paragraph of comments.
One of my favorites over the years, so-and-so is to the teaching of physics as Einstein is to the National Football League.
That's a very salient bit of narrative.
And it turns out that those sort of comments have much more impact than all the statistics.
In spite of the fact that the nice rational you knows that one person's brilliant catty comment, or brilliantly positive comment, is presumably much less valuable than the statistical average across 300 students.
But the narrative, again, overwhelms.
You can watch this on Friday, when you watch the debate.
You can be guaranteed that somebody is going to, it'll probably be either Kerry or Bush, actually.
That Kerry or Bush will take some boring statistics and personalize it.
They do this all the time.
It's become a sort of it a trope, a standard item in the State of the Union message where the President always put a couple of heroes.
They probably are, but, you know, heroes up in the balcony there so that he can point to them and say, look at this guy.
Look at that guy.
And in the debate it'll be, there'll be some discussion of employment statistics.
And presumably this would be Kerry.
Because he'll be beating on Bush about it.
I don't care about what you say about the economy recovering or whatever.
What I care about is poor little Olga.
She got lost in the mall, and then her father lost her job as a mall security guy because of the decisions that your government, your administration made.
And what am I going to say to poor little Olga.
Why?
It's very sad about poor little Olga, who's now been lost in the mall and she's never going to sit in the front again.
But there are, like, 260 million people, whatever it is in the country at the moment.
It's very sad about poor little whoever it is.
What's really important are the broad economic statistics.
Broad economic statistics are wonderful propositional reasoning kinds of stuff.
But are really boring.
Unless you're an economist.
But poor little Olga is really riveting.
And poor little Olga is what you'll hear about in the debates.
We should do a debriefing about this on Monday.
If anybody listens to the debate and hears any of these little tidbits, send me email.
In case I miss it.
And then we can we about this.
Now, this can get studied in the lab.
Here, take a look.
On the handout, there's a thing called the framing demo.
Says that there's an unusual flu coming.
It's expected to kill 600 people in the U.S., and there are two things you can do.
There's Program A and Program B.
Take a quick look, decide on Program A or Program B, and then we'll collect a little data and talk about it here.
Don't cheat off your neighbor's test.
OK. Everybody manage to come up with an answer here?
OK.
So, let's see for starters.
How many people went for A?
How many people went for B?
How many people don't know what they're going for?
OK, well it looks about evenly divided.
Boy, that's really boring.
Well, it would be really boring, except that there are two versions of the handout out there.
So, Version One of the handout says that if Program A is adopted, uh-oh, I don't know which Version One says.
Well, it doesn't matter which I call Version One, does it?
Version One says, 400 people will die.
Version Two says 200 people will be saved.
Did I get the language right there?
What you will notice it, if the net is 600 here, these are identical, right?
There's no difference except in the way that it's phrased.
Or the way the question is framed.
There's a similar change to the second option, B in the die -- in this person, it's described as a 1/3 chance that no-one will die and a 2/3 chance that 600 hundred people will die.
Do you know about the notion of expected value?
You can calculate the expected value of that option, which will be 1/3 chance that no-one will die and a 2/3 chance that 600 people will die.
And then that's going to be 400 people dying, again, here.
So all the options are mathematically identical.
Now, how many people have Version One, the 400 hundred people will die thing?
OK, how many people have the 200 people will die thing?
OK, good.
So, it's roughly evenly divided.
Now, what I want to do is, I want to get the A and B for these two different versions.
If you have the 400 people will die version, how many people opted for Option A?
How many people opted for Option B?
OK, so B beats A, and actually it beat A big time here.
If you have the 200 people will be saved version, how many people went for A?
How many people went for B?
OK, in that case A beats B not quite as massively.
But, clearly this manipulation has flipped the answer.
It ain't math that's doing it, because the math is the same.
What's changing here is what Kahneman and Tversky called the framing of the problem.
But really it's the narrative around the problem.
What happens is, you read Option A.
400 people will die.
That's terrible.
Well, the second possibility in this case, the B is, woah, there's something we could do.
That probabilistically might save some of those people.
That sounds OK.
I'll opt for this.
As opposed to this one, 200 people will be saved.
Well, that's better than nothing.
And the alternative, B, here.
You read the part that says, everybody might die.
Hm, doesn't sound too good.
And so you flip it.
So what you're doing here is the way that you ask the question changes the answer you get.
Well, we can go back to the politics game here just fine.
Pollsters do this all the time.
The neutral pollsters desperately try not to do it.
But campaign polls, when they're trying to get the correct answer out, who would you rather have for President, a strong leader who has ruled the free world with a firm hand for four years, or a flip-flopping senator from a liberal East Coast state?
Or you could ask the question, who would you like to have as your President for the next four years, a guy who barely made it out of Yale and has driven the country into a ditch, or a guy who actually got decent grades at Yale and is a national war hero and has really good hair?
You can move the question around, by the way, and not just in polling.
But the way you frame issues.
I mean, talk about framing issues.
If you frame the war in Iraq as part of the War on Terror, and an integral part of the War on Terror asking whether you want to keep doing that, is different than taking the same thing and calling it a distraction from the War on Terror, for example.
The facts remain sort of similar.
It's harder to do the experiment in a nice, clean, controlled -- here, we can make all the math come out right.
In a political debate, there's an argument about the actual facts, too, of course.
But the way that you present the facts, the way you frame the facts, is designed to get the answer that you want out of those facts.
There's a beautiful round of this in today's paper, because there's the report of the whatever they're called, Iraq Commission that was looking for weapons and didn't find any weapons.
And if you read the Kerry camp statements about what that means, and what the Bush camp says it means.
The Bush camp is very big on intention this morning.
He really, the report that Saddam wanted weapons.
And so the Bush camp is busy saying, he wanted weapons and he would have gotten them as soon as he could.
And the Kerry camp is looking at the same facts and saying, we all want stuff.
Man, he didn't have anything.
So.
Anyway, you can have lots of fun Friday.
You can watch them do this one, too.
All right, so.
The narrative that you put around a story.
The narrative that you put around the story does not influence your computer.
You can tell your computer about 1/3 times 600 plus 2/3 times minus 600, any way you like.
You can use big font, you can use little font, you can use you want, it's going to come out with the same answer.
Not true of the way that you think about these things.
You're going to get different answers depending on how you frame the question.
Let me do a different example.
That's not my handout, I got to see what's actually on the handout here.
So, right underneath that, underneath the bit about framing, there's another demo.
I will, in advance, promise you that this is the same on all the handouts.
Because otherwise you're going to sit there trying to figure out what's the weird thing.
So, this is the same on all the handouts.
I'm also aware that I have six slots between most probable and least probable.
And only five options to go in there.
Deal with it.
Here's what you've got to do.
You've got this guy Henry.
Henry is a short, slim man.
He likes to read poetry.
He's been active in environmental and feminist causes.
And Henry is a what?
And so there are these five options here.
What you want to do is rank order them from most likely to least likely.
And then we'll talk about that a bit.
Let's see how that -- No, you can't copy off your neighbor.
You know Henry.
I should have brought Henry in.
Just grabbed some character and said, this is Henry.
All right.
How many people need more time for this?
OK, a couple of people still working on it.
All right.
Well, we'll just go up.
Let's look at a couple of particular comparisons on here.
How many people put down, let's look at A and B. Whoops.
That looks propositional thought.
All gone.
OK, let's look at A and B, for example.
How many people said A was more probable than B?
How many people said B was more probable than A?
All right.
B wins.
And if I've got the right one here, so ivy beats truck big-time, right?
All right.
How many truck drivers are there in the country?
A lot
A lot.
10 to the -- I don't know the answer to this, but?
5?
4 is only a thousand
5
5 is probably-- it may be 6, but let's go with 5.
OK. How many Ivy League classics professors are there in the country?
How many Ivy League schools are there?
Eight
Eight to ten, something like that.
Maybe, if you're lucky, there's ten classics professors at each.
So that's going to get you to 10 to the 2nd.
Is it a thousand times more likely that some random guy that you meet, who happens to be -- well, let's phrase this a different way.
If you took all, whatever this is, a hundred thousand truck drivers in the country.
I'm sure there are more than that.
But let's just suppose there are only a hundred thousand truck drivers in the country.
Are there fewer than a hundred of them who might be, for instance, short, poetry-loving feminists?
And that assumes that every one of the Ivy League professors is a short -- there's no Ivy Leagues professors who are big, burly guys with tattoos and stuff.
Or women, for that matter.
Probably half of them are women, so this is already -- What you've done here is you've ignored what's known as the base rate.
All else being equal, it's more likely that, if I just say, here's Henry, is it more likely that Henry is a truck driver or an Ivy League classics professor?
It's way, way, way more likely that he's a truck driver.
In fact, it's so much more likely that he's a truck driver that it's probably more likely.
It's almost undoubtedly more likely.
Even for this description of Henry the amazing feminist, whatever he was.
You see, this is what's known as the base rate fallacy.
People do not tend to take into account the base rate in the population when they're thinking about this.
I'm asking how likely it is, and you're answering a different question.
You're answering the question, how typical is Henry of your image, of your stereotype, of a truck driver versus of an Ivy League professor?
And realize that even if -- let's suppose your stereotype is right.
What's the chance that Henry could be a truck driver?
Maybe it's only 1 in 100.
All right, if it's 1 in 100, there's still 10 to the 3rd of them.
And we're still swamping the population here.
So, failing to take into account the base rate is going to lead you to the wrong answer there's.
Let's look at another piece of these data.
Let's look at -- Is it more likely, A and E. A, E. How many people said that A was more likely than E?
How many people said that E was more likely than A?
Yeah, well this is this is Kahneman got the Nobel Prize, you see.
Because he said that, too.
Well, he didn't say that.
He said, that's what you would say.
Or he found, anyway.
So E is much more likely.
Remember, like, third grade set theory stuff?
Let's make the little picture here.
All right.
The set of all truck drivers, right?
The set of, I can't even remember what E is.
E's like Mensa Ivy League truck drivers or something.
It's some small little set, it's a subset of A.
So, for starters it's a little unlikely that the chance of being in here could be greater than the chance of being in here.
The best it's going to be is equal.
In some fashion.
Here's Henry.
I'm meeting some random Henry.
What's the chance that he's in E?
It's just not -- Again, you're answering a question that says something about typical rather than probable.
And you're ignoring what you would perfectly well know, of course, if we drew it out in set theory terms.
You can't be more likely that something is in here, if the larger set includes that.
That doesn't work really well.
So, people are very bad at base rates.
Is this a bad -- is the purpose of this lecture to say we're all morons here?
No, not really.
The faults we have, the problems that we have in the way that we reason about problems like this, reflect -- What Kahneman and Tversky talked about were, if I'm spelling it right, heuristics.
Sort of, mental shortcuts that do a certain amount of work for us.
Our job isn't actually out there in the world to figure out set theory problems.
Our problem is to get home safely at night.
So, you're walking down the street at night.
And it's a dark kind of street.
And you see this big guy with a stick.
And he's walking down the same side of the street as you.
Do you cross the street?
Well, if you're in a Kahneman and Tversky kind of experiment you say, I know what I'm supposed to do here.
I'm supposed to think hard about the base rates.
Is it more likely, how many people are there in the world who are bad, evil, people with sticks that are going to, like, hit me.
Versus, how many people are there who are, like, guys coming home from a softball game with their bat, or something like that?
And they're nice people, mostly.
Or at least even if they're bad, evil people, they're not really going to hit me.
The answer is, there's a very small population of people out there with sticks wanting to hit you.
But, in the occasion that you get that wrong, it's a really bad mistake to make.
And so, a mental shortcut that says not, what's the base rate here, but is this typical of a situation that could be dangerous, sort of, flipping it around.
Could this be a bad thing.
If I make a mistake and he's a really a nice person, so you might feel a little hurt that I crossed to the other side of the street.
What's the big deal.
If he's a guy with a bat and a nail in it and he's going to poke me in the head, and terrible things are going to happen.
I'm not going to worry about the base rates here.
And your cognitive hardware seems to have been set up under those sort of constraints more than it was set up under what might be optimal public policy solving constraints.
So, this is why you can then run a political campaign where one side or the other spends a lot of time conjuring up images of bad guys with sticks, of some variety or other.
Because you hear about the bad guy with the stick often enough.
And if this guy promises he's going to save you from the bad guy with the stick, well, you'd better go with that.
At least, that's an appealing thought.
And it works for them.
It works as a political technique.
All right, this is all -- who knows about Henry the, some weird truck driver that I invented or something like that?
How about a realm where things really ought to be logical and mathematical.
Which would be money.
I mean, if there was ever going to be something that ought to just work out in terms of the math, you would think it would be money.
And this is, of course, the reason why there is no Nobel Prize in psychology.
But there is a Nobel Prize in economics.
And that's what Kahneman, in fact, won.
Kaheman and Tversky would have won it, but Tversky, unfortunately, died young.
And you don't win the Nobel Prize posthumously.
So, Kahneman and Tversky also looked at the way that people reason about money.
This is something that goes back long before Kahneman and Tversky.
And, I can perhaps illustrate one of the older bits with an example.
Let's suppose that you're going to buy a toy car for your cousin.
You like your cousin, just in case you were wondering here.
So you're buying this toy car for your cousin.
And there are two toy cars out there at the moment.
They are, for present purposes, functionally equivalent on all dimensions of kid loveliness or something.
There's the midnight blue, I don't know, Model PT roadster or something from Toys R Us.
And the midnight green Model PT roadster from Kids Are Toys, or something.
So, the difference is the color.
The green color is cooler.
You know your cousin actually likes the green one better.
It's a bit cooler.
Anyway, the midnight blue one costs $12 and the midnight green one costs $40.
How many people are going to buy one of them.
You have to pick one, again.
How many pick the blue one?
How many people pick the green one?
OK. You guys have nice cousins, that's good.
All right, the blue one wins big-time.
All right.
You now have magically graduated from MIT.
Congratulations.
You got the job.
You're now going to buy the car.
And so, you're going to buy -- you're going to buy a real PT roadster.
and And they come in two different colors.
Amazingly, for today's purposes, they come in midnight blue and midnight green.
The midnight blue one costs $30,012.
The midnight green one costs $30,040.
You like the green one better.
How many people are going to buy the green one?
OK, so you graduated from MIT with this brain.
It's the same.
So, obviously it's now going very much in the other direction.
It's the same $28.
What's your problem here?
[UNINTELLIGIBLE]
You were going to --
You're already paying over $30,000 so the $20 just seems very trivial
Yeah.
It's the same $20.
This is absolutely right, of course.
But yeah
[INAUDIBLE]
The toy car is probably going to last you least as long as the roadster.
I keep trying to fine-tune this to make this, you're not going to buy 15 cars here, and -- We're assuming a one-time purchase here, a one-time purchase here.
And they're each going to last three years, or 100,000 miles, or whichever comes first, or something.
Yeah
[UNINTELLIGIBLE]
That much more money?
All right, all right.
Would your answer have changed radically if I say, congratulations you have just graduated.
It's also your cousin's birthday.
You'd all of a sudden be -- Boy, there must me sort of a U-shaped function.
Because by the time you get to a parent or something, ask your parents, who graduated a while ago.
Hey, Mommy, can I have the midnight green car, it only costs three times what the other one cost.
Oh, actually I just sort of gave away the answer.
It's the three times issue.
And you were alluding to this already.
That money is perceived, even though it's obviously the same $28, that people think about money in ratio terms and not in absolute terms.
This was actually picked up by Bernoulli a couple of hundred years ago.
I don't know which Bernoulli.
It turns out that there were buckets of Bernoullis.
Five Bernoullis, and they were all geniuses.
So, but one of the Bernoullis basically asked, what is the psychological value of money?
Actually, in Kahneman and Tversky language, that's often called the utility of money as a function of actual money.
And what Bernoulli asserted was that the utility of money, the psychological value of the money, basically went up with log money.
So it's basically preserving ratios and not the absolute.
It's not preserving the absolute values.
Which, by the way, when you go out to buy that first car when you have all this money that you're apparently going to have when you graduate.
The guy selling you the car knows all about this.
And he knows that, do you want the plain vanilla one, or do you want the PT roadster with the really cool racing stripe on it.
And this is part of our cool new graduate package, of stuff that cost us $3.95 and we'll crank the price up by $500 here, because on $30,000, who can tell, right?
And he knows.
He knows that it's the same $28 here and here.
And his job is to get as many of those $28 out of you as possible.
And will take advantage of exactly this fact.
If somebody said, I'll put a little stripe on your toy car for an extra $28, well it's a smaller car.
I'll put two stripes on it because it's a little car.
You'll say, get away from me.
But just listen to yourself go for the options when you go off and buy that car eventually.
All right.
let us take a momentary break.
And then I will say a word more about the oddities of money here.
[RANDOM NOISE] OK. Let us contemplate a little more of the way in which the -- now, the reason this is important.
The reason that this is the sort of stuff that eventually gets -- that where working this out is worth the attention of the Nobel committee in economics is that economists, for a long time, had this notion, had a psychological theory, in effect, of what people were doing economically.
And it was the notion of a rational consumer, a rational person who's making decisions that were in a sense propositional.
That they were doing the math to the best of their ability, and working things out on that basis.
And this the Kahneman and Tversky program of research tells you is, the very interesting constraints on that rationality.
The ways in which it sort of systematically deviates from what your rational computer might think about the same kind of question.
So, consider the following.
Let's think about doing some gambling games.
Where this is a one-shot game.
You only get a chance to play this game once.
I got a coin.
It's a fair coin.
If I flip it and it comes up heads, I give you a penny.
If it comes up tails, you give me a penny.
How many people are willing to play that game?
Few people
[UNINTELLIGIBLE]
No, no.
One shot.
One time.
The whole game is this one penny.
How many people are willing to play one round of this game?
Give me the quarter instead
Oh, she wants to play for real money.
OK. That's fine.
Let's do that instead.
So.
Most people are willing to play that, though.
By this point in the lecture a variety of people are starting to wonder what is that I'm revealing about myself if I say I'll do this.
So, the way you write that, you could write that as, you've got a 0.5 chance of winning a penny and a 0.5 chance of losing a penny.
And the expected value of this is 0.5 -- you can figure that out, it's 0.
It's a fair game.
OK, let's play it again.
We'll flip, we'll play for her.
Flip the coin.
Heads, I give you $100.
Tails, you give me $100.
How many people want to play?
Your computer doesn't care much about this, because the expected value remains 0, right?
What's the problem?
[UNINTELLIGIBLE]
People are risk averse.
Yeah, that's certainly one.
That's what you find here.
But why don't you once want to play?
[UNINTELLIGIBLE]
No, the expected value is just a thing defined in math land or statistics land.
It's zero on this.
But the two, you could either win or lose, it's true.
You're not going to come out at 0.
But if we played it for everybody in the classroom, if I had a lot of $100 bills.
Gee, I wouldn't want to play that game.
Yeah
The expected utility of playing is less than the expected utility of not playing
Hmm.
That sounded cool.
But I'm not being quick enough to figure this out.
What did you --
I'm saying that winning $100 is less good than losing $100
Thank you.
That's exactly right.
It turns out that this curve is not symmetrical around the origin.
That negative money, the utility of negative money, tends to be somewhat steeper and more linear than positive money.
With the result that, exactly what the gentleman said.
That losing, that the absolute value of losing $100 on this scale is greater than the absolute value of gaining $100.
I mean, one way to intuit this is, there's plenty of room for you to gain $100.
You can do that all the time.
Do you have $100 handy that you can afford to lose?
Maybe not.
It would just hurt a lot more.
And if we decide -- And now we can -- There are some versions of this -- it's not that you'll never play a game for this kind of stakes.
I mean, if I say I'll flip a coin.
Heads, you give me a penny, tails, I give you $100, how many people want to play?
And the other ones didn't figure out what I said, presumably.
So, somewhere between that extreme and the even game is the point at which we would measure how risk averse you are.
It'd be different for different people.
I mean, would you play if heads, one time play, heads you give me $50, tails I give you $100?
Now, some people would be willing to play.
He's good, he's got $50 handy.
Or do you have a question?
No, you were just --
[UNINTELLIGIBLE] 198
198, OK.
He's got -- talk to him later.
He's got more money than he knows what to do with.
But, in any case, there will be a balance point at which your -- where the loss and the gain balance out.
And then you'd be presumably willing to pay.
Strange things happen at the extremes.
And that's the basis for a willingness to play lotteries.
So, if you have a wager that says, heads you give me $1, tails, I give you $10 million, but the probability of coming up tails equal 10 to the minus, I don't know, 80 or something like that.
I'm not getting the right numbers, particularly, for a state lottery.
But it's not -- it's that kind of thing.
The reason people run state lotteries is because they want your dollars.
Not because they're interested in giving you a pile of money.
So, even so by the time you get these very extreme kinds of gambles, as long as the cost is sort of minimal down here and the potential payoff is huge, for instance, you're willing to pay radically unfair games that you would never play in the range of, in the middle kind of range.
If I tell you, well all right, we can make this just a tenfold difference.
So if we play a game that -- you can figure this out.
At the extremes you're willing to risk $1 to make $100 million.
Even if the odds are against you.
In a more moderate game, you would not be willing to do that.
And it's that sort of reasoning, or lack thereof, that allows lotteries to make their money.
Even -- it's probably also confounded there by the fact it it's not clear that the population as a whole really understands that this is a sucker's game.
That the state is actually in the business of taking your money from you, not just redistributing the wealth.
But even people who perfectly well understand that their chances are less than even of winning, even if they were to buy 10 to the 80th tickets, continue to buy lottery tickets and continue to.
It's a strange way to raise money for your state, but never mind.
So, money is like other aspects of reasoning.
Is subject to the sorts of narrative stories that say, it hurts more to lose $100 than it feels good to gain $100.
The sort of narrative that is beyond just the simple math of the situation.
Let me -- OK, I have time to do one more example.
Which is an interestingly illustrative one.
And involves these cards.
Which I believe I put on the handout.
This is known as the Wason selection task, because good old Wason developed it.
And, oh look, the rules are properly described on the handout.
You've got four cards here.
Each card has a letter on one side and a number on the other side.
You know that.
What you want to know is, and here's the rule.
The rule is, if the card has a vowel on one side, then it has an odd number on the other side.
That's the rule that we want to check.
Which cards, what's the minimum and full set of cards that you need to flip to check whether that rule pertains for this set of cards.
That's the question.
And so we'll vote on each card as we go along.
How many people want to flip E?
All right.
Lots and lots of people want to flip E. How many people want to flip the S?
Hm.
Couple of people want to flip the S. How many people want to flip the 7?
A bunch of people want to flip the 7.
How many people want to flip the 4?
About the same bunch of people want to flip the 4.
And some of the same -- I probably, well, it's too complicated to collect all the details of how many cards people think you need to flip.
The answer is, you need to flip two and only two cards here.
And before I tell you which ones it is, I should say that like many of the demos in this lecture, this is the sort of thing that drives MIT students nuts.
Because they -- it's a cognition lecture, it's a logic lecture.
It's got, like, mathy things in it.
And it's logic.
And that's why I'm here.
And I'm getting them all weird and wrong, and, and, and, and, and I'm going to flunk all my courses.
And I'm going to die, or something.
I mean.
I should note that one of the characteristics of the Wason selection task is that everybody gets it wrong.
I don't remember if it's Wason in originally who did it.
But he tried this out on logicians and they blow it.
And if you're feeling, like, depressed after this, you take this to the calculus TA next hour, and see if she can do it.
So, as people correctly figure, you've got to pull this guy.
Because if it's got an even number on the other side you're dead.
And you don't have to do anything because we don't know -- there's no assertion about what non-vowels are.
You don't have to flip him.
Because it doesn't say, it says -- here, let's do this in good logic terms.
Which for mysterious reasons always talks about P. The assertion is, if P, then Q.
Then odd.
If P then Q.
If you know Q, who cares.
It doesn't say, if Q then P. So this could be an S without violating the rule.
This guy, on the other hand, is not Q.
This guy's not P. Everybody figured out we don't care about not-P. Not-Q is the other one that you do need to check.
Because if this has an E on the other side, the statement is false.
The rule is false.
So you've got to check for an E on the other side of this.
So, there are now three people here who say, I've got it, I've got it.
And good for you.
That's nice.
People routinely do very badly on this.
And the interesting question is, is why -- well, actually, I'm not sure that's an interesting question.
Why you do that.
The interesting fact is that people do very well on other versions of exactly this problem.
So let's set up another version of this problem.
We've got somebody with a -- a beer.
We've got somebody with a soda.
We've got an 18-year-old.
And we've got a 22-year-old.
The rule is, you have to be 22 to drink.
Well, 21.
You have to be over 21 to drink.
Which of these cards do we need to check?
So do we need to check this one?
Yeah, sure, right.
If this sucker's an 18-year-old, he's going down.
Do we need to check this one?
We don't care about that.
Do we need to check this one?
Yeah.
We need to check this one?
No, we don't care about that.
People are perfect at this, essentially.
Sometimes if I phrase it right you can manage to get yourself a little confused.
So if you got it wrong don't sit there and say, I'll never drink again.
So, people are perfectly good at this sort of real world example.
It turns out that they're not good at all real world examples.
I could, I won't bother trying to generate another real world example here.
Leda Cosmides has argued that what people are good about reasoning about is cheating.
Is somebody getting goodies that they're not supposed to Get Is this 18-year-old getting a drink when I can't get a drink yet?
I'm going to keep good track of him.
And the guy drinking this beer looks kind of young to me.
Whereas, that four -- yeah, nobody cares about that.
And if you reframe the problem, there are plenty of ways to reframe this, to put things like, oh, there's a version about, if you're in Seattle it must be raining.
Examples like that.
People bomb that all over the place.
There's nothing at stake there.
Cosmides and a variety of the evolutionary psych people argue that your cognitive capabilities have been shaped by the problems that you need to solve out there in the world.
Is somebody getting goodies they shouldn't be getting?
Is the guy with the stick going to hit me with the stick?
That kind of thing.
It would be lovely if that was true.
It might be true.
The problem is that the research subsequent to this has made the issue more complicated.
It is not -- there are cheating examples that people have cooked up that bomb.
And there are non-cheating examples that people successfully manage to solve.
But what does seem to be pretty clear is that we are not built to solve abstract problems.
That's why you have to show up at MIT to learn that kind of stuff.
Nobody goes to MIT to take courses in carding guys at bars.
You come with that.
You can figure that out for yourself.
It's when you have -- unfortunately.
The problem of running the country turns out to be probably a lot more like this than like this.
But the decision is made like this.
So, everybody go watch the debate and tell me what you find.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license, and MIT OpenCourseWare in general is available at ocw.mit.edu
Last time I was talking about cognition and I'm framing the argument in part in saying that we're not like computers, even though computer's a nice metaphor for various secure aspects of cognitive function.
We could continue that today.
When you get an off-the-shelf computer, you expect that its basic way of thinking is going to be -- it's there.
You might have to put some programs on it to teach it to do specific things, but you don't have to educate its motherboard or something of that sort.
With kids it's not clear that that's the case.
That if you take an off-the-shelf baby, it's not clear that that baby is thinking the way that an adult is thinking.
That's really what I want to talk about today.
Let me put up a picture that I can then return to multiple times.
If this is age, and we're interested in development, what we were talking about last time is the adult state -- something about the adult state.
What we want to know in studying cognitive development or any sort of development is what's the starting point?
If you have a newborn baby, is that starting at zero or does that baby come with some sort of innate endowment that gets them on the way towards the adults state.
Then what does the transition from the kid state to the adult state look like?
Does it look like some sort of gradual improvement?
Does it look like some sort of step function?
It might be useful to describe from the realm of vision, a couple of examples just to show you what this means.
If you want to know what a baby sees, get yourself a newborn off-the-shelf baby and give them -- well, you can't give them an eye chart to read because they can't read, so you give them the illiterate E chart, as it's called in the trade, and you ask them which way do the letters point, the kid doesn't tell you anything useful, because the kid can't answer that question.
However, conveniently enough if you take even a newborn baby, young as you can test them, and put them in front of a couple of screens -- little peephole here.
You guys can all be babies and the experiment, if you're looking out at you through this peephole, if you put black and white bars on one screen and you put on the other screen a grey of the same average luminance, so if this is zero and that's 100, this is 50 all over the place.
If the baby can see the bars, conveniently enough that baby will look at the bars.
If you are looking at the baby, you can figure out, you move the bars from side-to-side from child-to-child, you can figure out where that baby is looking and you can figure out where the bars are based on where the baby is looking.
That's known as a preferential looking method, very useful in infant research.
What you discover, if you do experiments like this studying infant visual aquity, is that babies don't start at zero.
A newborn baby isn't blind.
Newborn baby can see something.
If normal adult vision's around 20/20, newborn baby vision is around 20/400 -- the big E at the top of the eye chart roughly.
The progression from the infant state to the adult state is a gradual, relatively steady increase over about the first couple of years of life.
Every month you get roughly the same kind of improvement until you hit the adult level.
So, not a zero starting point and a gradual improvement to adult level.
Now let's take a different example from the same realm.
Remember you've got two eyes, the differences in those two eyes are a depth cue -- known as binocular stereopsis.
Well the cue is binocular disparity, the ability is stereoscopic depth perception.
If you test a baby, so baby's it turns out also like things that stand out in depth better than they like flat stuff, so if you show a baby bars standing out in depth, a newborn baby gives you squat.
To all appearances there is no binocular stereovision in a newborn baby.
For about the first three to five months, nothing happens.
Then over a very short period of time, performance rises to essentially the adult level, as if this was waiting for something to get wired up in the brain.
When it got wired up in the brain, bingo, the kid is fine.
Oh, by the way, if you are a woman person you got there first.
It turns out to be a reliable sex difference that females get binocular stereovision a month or so before males.
Does this make any difference to anything?
No, but we found it years and years ago when I was a grad student so I still remember it.
So these are two of the various possibilities for a function like this for a developmental course.
We're going to talk about three specific realms of cognitive development in which there are classic demonstrations of children doing things differently than adults do them.
I will try to take each of those apart and to give you some idea of what's the starting point, what's the ending point and how do we get between those.
The three, which are outlined on the handout have to do with so-called childhood, animism and egocentric behavior.
Let me describe each of these problems to you in turn and then we'll take each of them apart.
These are all demonstrations that originally come from the work of Jean Piaget, the great early mid-20th century developmental psychologist, who among other things, bequeathed to us a great collection of phenomena that shows striking differences between the way kids behaving and the way adults behave and that cry out for explanation.
So here would be one of them.
These are M&M's, let us say.
This is another one of those forced choice kind of things, so think hard about this.
Are there more M&M's here or here?
How many vote up?
That's good.
However, if you were 3, 4, 5, maybe even 6, you'd say here.
You would be perfectly happy.
So the first obvious thought is well, the kid's just not quite understanding the question.
It a languagy kind of thing.
We're saying where's more and they're saying which is longer.
So that's why it's important that you use M&M's.
You say OK, kid, which pile would you like?
Kid says I want this pile.
Why do you want this pile?
Well there's more there.
In fact, this is why you shouldn't be born to a psychologist because your psychologist dad is always tricking you out of the candy by playing this game on you.
It works.
You teach this stuff long enough and you say is this really true?
And then you've got yourself like a three-year old kid and you mess with his brain -- yeah, it's true.
This is good stuff.
The question here, of course, is then what's going on?
This is part of a general problem.
So let's take a fat beaker.
Here's a big fat beaker with this much fluid in it.
Now I pour that into a narrow beaker.
The fluid level is?
Higher
Higher.
Is there more fluid now than there was when it was here?
No.
Does a little kid think there is?
Yes.
If you let the little kid pour it back into the big bowl, now he thinks there's less now.
Take a ball of clay, roll it out into a snake, there's more clay says the little kid.
Squish it back together, now there's less.
These are so-called conservation problems.
We're particularly interested in the version that has to do with number -- how come they think, in this case, that five is bigger than six.
Second problem is what's called childhood animism.
The classic Piagethian version would involve something exactly like his so-called three mountain apparatus where e had sort of a table top -- he's from Switzerland, so he's into the Alps.
It's a table top version of the Alps, on the Alps are things like, that's a river and that's a hut.
The point is that you're sitting here looking at this thing.
You're like a three-year old or something looking at this, and mom's over there on the other side.
You can see mom.
The question that you're asked is can your mother see the river?
And you say, yeah sure.
As if the fact that you could see the river makes you think that she can see the river.
As if you couldn't take her point of view.
At least Piaget thought that the problem here is that you can't quite take somebody else's point of view.
My favorite anecdotal version of this is a story of a little kid out to eat with his brother at some burger place.
Little kid wants more French fries, and so he does this, on the theory, presumably, if I can't see what I'm doing, you can't see what I'm doing.
Most people don't do that by the time they reach adulthood.
So there's something to explain there.
That's not childhood animism at all.
Actually, that's egocentric behavior.
Sorry, I misspoke.
That's the third example that we'll get to.
So, egocentric makes more sense.
Ecocentric is the inability to see things from somebody else's vantage point.
So I've just described to you the problem of childhood egocentrism.
Childhood animism is a tendency to declare things to be alive that you would not declare to be alive.
Is the cat alive?
Yeah, fine.
Is the sun alive?
Yes.
Are the clouds alive?
Yes.
Little kids will assert that and you typically will not.
That's animism.
Piaget understood that as a problem with mapping concepts onto a discrete set of entities out in the world, and so we'll take a look at that, too.
So, number, animism, you egocentric behavior, those are the things that I want to talk about today.
Let's start with number, and let's start by taking apart the idea of number a bit.
Because you an I tend to think of number as there's five of those things and six of those things.
The concept of number, the idea of number can be broken down into three different bits at least.
Let's see if I can do this right.
Oh, well they went by.
How many warthogs were there?
[INAUDIBLE]
Let's try this again
7.
8
9, 8, 7.
Some number like that.
It wasn't 150.
But how about this?
2
OK, that's good.
How about this?
3
How about this?
This is data.
If you do this task, what you find is that people are essentially perfect for 1, 2 and 3, maybe 4, and then it falls off.
I think the answer was 8 before, but sometimes you say 7, sometimes you say 9, and so your percent correct, being absolutely correct will systematically fall off.
If I put 1,500 wart hogs up there you'll just not get it right at all.
Similarly, if I was to measure not percent correct but reaction time, the reaction time for 1, 2, 3, 4 would be essentially flat, and then start to roughly linearly increase.
That linear increase could be, for a number like 8, could either be due to some sort of counting operation.
Somebody who thinks they saw 8 of them, tell me how they figured that out?
Yeah?
You saw a line of 5 and then 2 and 1
A line of 5 and 2 and 1.
OK. Yeah?
I couldn't get a count because they went away
OK, yes.
If I was doing this as a proper experiment I would have masked it so you couldn't count like the after-image or the iconic memory.
Yup?
Just recall the picture
OK, so you were also dumping it out of iconic memory in some fashion.
The answer I was fishing for is something like the 5, 2 and 1 thing.
That you can count those displays, and in fact, the people who got them back later might have done this, too -- by more typically saying oh, there's one group of 4, there's another group of 4.
This ability to handle small numbers is called subitizing, and it works for small numbers like 1, 2, 3, 4.
You have some immediate appreciation of those small numbers that's different than your appreciation of 8 or 17 or 150.
That's one wart hog demonstration.
How about which one has more wart hogs?
How many vote that there were more here?
How many vote that there were more here?
OK, you got that.
How many wart hogs were there here?
A lot, yes.
And how many wart hogs were here?
A lot but not as many.
You could count these, obviously.
But you didn't count them and you didn't subitize them.
What you did was you got some sort of general impression of their numerosity.
This is if you go back to perception chapter, this is another Weber's law kind of thing, or the money example from last time.
You can also decide something about how many things there are there by a sort of a ratio comparison.
There's more here than here.
If there were 750 here and 745 here you wouldn't know that.
There has to be a critical proportional difference between the two.
But given a proportion difference, that this is something like 2:1, like that, it doesn't matter how many things there are, you simply know that there's more here and less here.
That's sort of a crude sense of numerosity.
What we typically think of when we think about number is counting things, and that's the third property.
So here, you can count them.
Somebody can go and figure out exactly how many wart hogs there are there.
You just have to go through and use your counting ability to figure that out.
You have that ability.
It seems to be critically piggybacked on linguistic ability.
Recent interesting evidence for that comes from -- I don't know how to pronounce it properly, but something like the Bari Rahua tribe in the Amazon.
Their number terms consist of 1, 2 and more than that.
The result is that they bomb tasks that require counting.
They can do subitizing tasks, they can do numerosity tasks, but they don't do counting tasks.
They fail versions of the M&M task in effect.
You say which of these boxes do you want -- the got fish in them or something like that, and one has a little label with 4 fish on the top and another has 5 fish on the top and as far as these guys are concerned, it's all just got lots of fish.
What you see here is a task where this was the experimenter's marks, and the Bari Rahua person was asked to just copy it.
So, one mark, make one mark, two, I can make two marks, three, three.
That's just a bunch, I'll make a bunch of marks.
Once you got out of the subitizing range they simply didn't do the task.
I think I'd put these guys on this article from which I stole -- this picture may live on the website.
I seem to remember that that was one of the things that I posted there.
So, what to kids have and how does that tie into the M&M problem.
Well, turns out that little kids have babies, have this numerosity -- Weber's law, comparison kind of numerosity ability, and here's an experiment done by [?
Shew ?]
and Liz Spelke up at Harvard that illustrates that.
You're a baby, you're looking at this, and every now and then the display changes.
OK, there's another bunch of dots.
And another bunch of dots.
What I'm doing is I, the experimenter am measuring how long you look at this.
I'm changing the shape and the distribution of these guys, but I'm not changing the number.
You are slowly getting bored, or to be more technical about it, you're habituating.
You may recall that from the learning chapter.
I don't think we talked about it in the chapter.
Well, we can talk about it right now.
A bunch of people oriented to that.
Now those people are having no problem at all veeing back out again.
That's habituation.
You do it, babies do it, slugs do it, everybody does it.
Great.
Useful technique, because the baby's sitting here saying this is all kind of more or less the same and it's just not that interesting.
Wait a second, that's interesting, says the baby.
And the baby says it by looking more.
What happens here is that looking time goes down, down, down, down as the number on each trial stays the same.
Then when you jump from 8 to 16, the baby says hey, something's happened, I gotta take a look at this.
Do 16 for awhile, blah, blah, blah, blah, blah, blah, blah, blah.
What this tells you is that the baby could tell the difference between 8 and 16.
The reason for changing the shape and the distribution and the density and stuff like that is to make sure that that's not what you're picking up on.
[?
Shew ?]
and Spelke were careful about that.
So what you're picking on here is numerosity.
Babies can do that, but if you go from 8 to 9, nothing happens here.
In fact, if you go from 8 to 12, a 50% increase in dots, nothing happens here with, how old are these babies -- six month old babies.
So, 8 to 16, a 2:1 change -- that babies pick up on.
So, one thing we know that if we're talking about number, the starting spot isn't zero for number sense in infants.
The numerosity piece, some chunk of the numerosity piece is there for starters.
Perhaps more dramatically, the subitizing piece is there, and the subitizing piece is there along with what seems to be an innate mathematical ability.
Little kids can add and subtract.
Babies can add and subtract.
This was a surprise.
This is work done by karen Wynn, and it is worth making the historical note that she was a TA for this course once upon a time, now a professor at Yale.
But in any case, she figured out that babies can add and subtract.
Look, if you ask a baby what's one plus one, you get much the same answer as if you asked the baby which way is the E pointing.
You gotta get clever about this sort of thing.
Here's how Karen got clever about it.
So, you're looking at a screen.
You're a little baby looking at a screen like this and you see a hand come in with an object.
And you see a hand come in with another one.
You got a pretty good idea what's going on here, right.
So, if I lift the screen, what do you expect to see?
Now, we're going to do this again.
The speed would be the same, of course, I just didn't want to keep sliding them in slowly.
The baby will respond to this by looking longer.
The baby will say -- who knows what the baby will say, but the baby is apparently in effect saying put one thing in this, two things in here, two stuff, two wart hogs back there.
Now there's only one wart hog and this violates the rules of the universe and I said better look at this for a while to figure out what's going on.
You can use this technique to show that babies can do -- do I have a subtraction one here, I forget.
It works for monkeys, too.
We'll talk about that in a minute.
But you can do addition, you can do subtraction.
So, subtraction, you have the two things here, you put down the screen, the hand comes in and pulls one out.
When the screen comes up the baby expects to see one.
If the babiy sees zero or the baby sees two, the baby says something weird's going on here, I need to think about this for a while.
Now, if you do this with big numbers outside of the subitizing range, you don't get the result.
So, you stick 15 wart hogs back there and you lift up the screen and there's 13 wart hogs and the kid says I don't care.
Once you get above about 3 with babies, it doesn't work anymore.
So, what you get is evidence for subitizing being present and some sort of addition and subtraction kind of operation for, essentially as early as you want to test it.
You can also get these abilities in animals -- that's what this slide is intended to remind me to tell you.
Animals, certainly Macaque monkeys will show numerosity effects, and they can also do this sort of Karen Wynn style math.
They prefer eggplants -- at least Marc Hauser's monkeys like eggplant, so that's why there are eggplants here.
And you worry about doing this in the lab, because maybe your lab monkeys have led this elaborately studied life and are kind of smart.
Hauser went down to this island in the Caribbean where he's got a free-ranging troop of monkeys.
Then you kind of have to coax them into playing your game.
There's this great video that Marc has of monkeys -- he's hanging out with his eggplants, then he's starting to do this eggplant puppet show with the Macaque's looking longer at the weird ones.
The guy put two eggplants there, there's only one there now.
Hope he's going to feed me some of these eggplants eventually.
But in any case, monkeys can do the task.
Monkeys will look longer at things that violate their expectation.
However, monkeys don't count.
At least they don't count the way that you and I count.
This goes for great apes.
Fine, let's go find the brightest beast other than humans that we can.
There's at least one chimp who's been trained for 20 years -- oh, I misspoke when I talked about this in Concourse.
I'd said he'd been trained for nine years and never gotten past 20.
It's worse than that.
He's been trained for 20 years you never gotten past nine counting.
He can count to nine, do pretty good stuff up to nine.
But he's doing this by basically association learning.
You can train the monkey to realize that this sign or this graphic or something means nine things.
But this is very different from what you and I can do.
It didn't take you 20 years and you got well beyond nine.
Counting seems to be heavily dependent on two things.
One is language, and the other is an act of inductive reasoning.
You can sort of see counting happening, the development of counting in a series of steps.
Kids, as they are learning language, learn that there is this kind of game about saying 1, 2, 3, four and so on, that parents really like to play.
If you start counting they'll give you the cookies and stuff.
Good things happen.
They don't have any particular clue initially that this maps on to anything in particular.
So, what you get with like two and a half year old kids is a two and a half year old kid, if you ask for one cookie will give you one cookie.
But if you ask for anything else -- give me two cookies, give me four cookies, give me whatever, they'll give you some random -- here, have some cookies.
It's one and other stuff.
Eventually they learn to map the counting onto the subitizing range and then start blowing it later.
If you have little sibs or cousins or stuff that you've hung around with, you may have seen them counting things -- 1, 2, 3, 4, 5, 7, 12.
They got this notion that the numbers keep going in some fashion, but once you get out of the small numbers it's not terribly clear what's going on there.
But even in this stage you can start to see the development of something more sophisticated.
Six -- they may not know how many six's, they know six is a lot of stuff.
If you ask for six cookies you'll get some arbitrary number of cookies.
But if you ask a little kid if I take a lot of cookies and I add a few, do I still have a lot of cookies?
Kid says yeah, it works.
So, alright, so we know that.
Now, if I have six cookies, can you give me six cookies?
Some random number of cookies.
If I have six cookies and I add some more cookies, do I still have six cookies?
No.
You got a lot of cookies but it's not the same a lotta.
So, even when they don't really know how many cookies you've got, they have the notion that the number names are naming something specific.
They don't quite get what it is, but it's something specific in a way that a term like a lot is not.
Eventually, you make the breakthrough that you can count, basically, and if you add one to each of them you can just keep doing this forever.
Again, I don't know how much you hang around with little kids or remember your own experience, because this is getting old enough that you might actually remember this state yourself, of asking what the biggest number in the world is.
Anybody remember asking what the biggest number in the world is?
That's the problem with MIT students, right, they figured all this stuff out when they were like two.
So anyway, little kids want to know what the biggest number in the world is, and so you say to the little kid, well what do you think the biggest number in the world is?
Well, it's a million.
OK. What happens if we had a million plus one, is there a number for that?
I guess a million and one's the biggest number in the world.
Well, [UNINTELLIGIBLE] add a million and two.
You can keep doing this for a while.
Eventually, around the time when you get in this conversation, the kid catches on to the notion that you can always get a bigger one.
This is really the very boring car rides sometimes.
Do you want to see how high I can count?
That's a kind of cute game with a three-year old.
Aw, look, he can get all the way to 20.
By the time you get a 4 or 5 year old, million and one, million and two, million and three.
OK, we've made it to California kid, shutup.
What you get is you've got this basic numeric competence or core knowledge, as Liz Spelke would call it.
Then you're waiting for the development of language.
When you get language you can then get this counting piece to work.
When you got the counting piece to work, you can combine it with your ability to make the inductive leap that numbers go on forever.
Then you're in a position to go off to school and learn some math.
There's, in a sense, not an adult level of numerical competence.
You guys know more about numbers than I know, I would imagine.
But the ability to go off and learn that kind of math is based on this core knowledge, then waiting on language before you can make the leap to an ability to do 180 whatever.
So, it's an example, if you like, of the synergistic combination of information from a couple of different realms.
Let's switch to the problem of childhood animism that, if you're not completely confused from my earlier mis-speaking, animism is the tendency to say is the sun alive?
Yes.
Are clouds alive?
Yes.
That seems quite different from what you and I do.
One might wonder exactly how different it really is.
Does it really reflect a fundamental problem with the concept alive?
It's not that's obvious how you map the concept like that, even as an adult.
This is perhaps easier to think about in connection with dead.
Mapping dead onto a discrete set of things is not that trivial.
Is a rock dead?
Probably not because it was never alive, right.
Is George Washington dead?
Seems about right.
Is he dead in the same way that something that you can, like a cat that just recently died -- the here, dead, now.
Is that's the same?
How about a dinosaur?
Or the dinosaurs, are the dinosaur's dead?
Well yes, they're dead but they're also extinction, which is somehow kind of different.
Or to use an example from driving around, now that my kids have gotten out of the counting to a million stage -- I don't know how we got on the subject of what is alive.
Maybe because I was thinking about this lecture.
But anyway, my 10th grader asked my 3rd grader is fire alive?
Well, I don't know about the 3rd grader, but it took dad in the front seat a surprisingly long time to figure -- dad knew that the answer is no, but the principled reason why it's no, took me a couple of beats to get to.
Because fire breathes, right, it grows, it reproduces, it can die, it can do all those good things.
Turns out that in 10th grade biology they teach you that if you're going to be alive you've gotta be made of cells, and fire's not made of cells, except that then you get into 11th grade English and you read some sci fi novel where this thing is alive but it's just a forcefield in deep space or something like that.
I don't know what to do about that.
Concepts like alive are pretty complex stuff, and the fact that a kid doesn't map it exactly the same way you map it may not be that definitive.
Does the kid have a concept at all that they can use?
That's what we really want to know.
What's really going on there?
So maybe we ought to ask the question a little bit differently.
Kid had some notion of what alive things do.
So suppose we ask a kid does a cat breathe?
Yeah, a cat breathes.
How about a cow?
Yeah, a cow breathes.
How about the sun?
I don't think so.
The moon?
No.
The clouds?
No.
And you can do the same thing with babies.
Not asking the babies, but talking about can a cat have babies?
Yeah, sure.
Can the sun have babies?
No.
You get the feeling that there's a concept there that they're just not using the word alive the way that you and I are using it.
Well, maybe all you're looking at here is the fact that they don't actually have a concept.
What they have is a long list of learned instances.
They are hang out at these fancy upper middle class preschools where they get taught kitty has babies, cow has babies, and they never gets taught sun has babies, because the sun doesn't have babies.
And so they learn all this stuff.
Maybe it's not a concept, maybe it's just a list of instances.
How can we figure this out?
Well, Sue Carey did an experiment where what she did was she taught kids something new.
She said see this cat?
This cat has an omentum.
And you have an omentum.
And the cow has an omentum.
Now, how about the sheep?
Think it's got omentum?
Probaby got no mentum.
And the dog?
Yeah, the dog's probably got one.
How about a fish?
Yeah, probably, could be.
Worm?
I don't know.
Rock?
No, rock doesn't an omentum.
How about sun?
No, sun doesn't have an omentum.
You don't have one either it turns out.
She wanted a real biological term, but I think it's some weird membrane that an insect has or something like that.
But the point was that it wasn't something that you would have ever learned.
It's a new piece of knowledge, and if you say that things in this category, things in the category that are encompassed by cats, cows and you all have this, who else has it?
And little kids, three, four-year old kids, map this in a plausible kind of a way.
It might map something like mammal, it might map something like animal.
I don't remember if any of her kids mapped something like alive in which case they would assert that a tree had it too or something.
But it maps a concept that has a biological ring to it.
What's the difference between the childhood use of these concepts and the adult use of these concepts?
It seems to be, in this case, education.
You go to school to learn the right concepts.
In this sense, there is no firm adult state.
My job here is to teach you a collection of facts and the right concepts in psychology.
But I can teach you some of them and I keep learning new ones, and the adult par keeps sort of drifting there.
You sort of go off to learn these sort of concepts.
But you don't learn to have a concept per se.
What the kid is doing -- well, actually it's rather similar to what we're doing here.
The kid is mustering the data that is available to him, and working with the concept--.
So what do you think the concept of alive is for a little kid?
What do you think goes into it that causes little kids to give the wrong answer when asked by Piaget what's alive and what's not?
What's the critical difference between your notion of alive that you picked up in 10th grade biology and a four-year old's notion of alive?
Anybody?
Yup?
Something alive is something that moves and you didn't see how [INAUDIBLE]
Yeah.
So moving on its own seems to be an important--.
So cars aren't alive because you've got to make them move in some fashion, but self-produced motion seems to be an important part of the definition, and the sun and the clouds and the moon seem to scootch around in the sky, particularly if you're driving in that car, in ways that suggest that it might be alive.
It gets a little more complicated because kids have also learned that trees are alive, and outside of Lord of the Rings, trees don't do much in the way of wandering around, but growing on your own is good, too.
But that would be part of the biological definition.
It's the self-produced motion thing that kids overuse in their definition, seemingly, that you don't use.
You can see if you get transcripts of little kids using language, you get a feeling for this little scientist aspect of childhood concept.
Well, concept formation, concept verification.
So, here from the what is alive literature is a kid who is asked are rocks alive, and the answer is yes, rocks are alive.
But they're only alive until you step on them and then they die, which is kind of reasonable, right.
The kid's got presumably some experience with other things like that.
This ant -- the ant's alive, I step on it, it's not alive anymore.
And this kid produces some evidence in favor of this theory, which is if you look on a path, how big are the rocks on average?
You could take a walk in the woods down a rocky kind of path, there are rocks on it, they're kind of small rocks.
You look off across at the mountains, how big are the rocks?
They're huge.
What's going on here?
Well, on a path people are walking all the time so the rocks die real young and they never get to grow big.
Whereas the rocks off the path out in the mountains, they got to grow to be really big rocks.
It's not a correct theory, but you can see the kid using the data he's got.
right, a certain amount of data there.
If I step on it, it dies.
Maybe that's what happens to rocks and people step more here than they step there, so big rocks, little rocks.
It all hangs together.
I got a great theory here.
Another one, are rocks alive?
Yes.
Can they have babies?
Yes.
Are all rocks alive?
No, some rocks are dead.
How can you tell if a rock is dead?
Well, the dead ones just lie there.
This is capturing the notion that motion is important in little kids' definitions of the notion of aliveness.
I don't know how many rocks you saw moving under their own power.
In any case, you can see the little scientist aspect of this.
OK, so what's the starting point here?
The concepts that you're building seem to have some foundations that can be picked up very early and are probably innate.
I'll just talk about two of them here.
So concepts like about things like aliveness.
If you're going to decide that a thing is alive, you need for starters to know that it is a thing.
So you need to have some notion of an object.
The other thing you need is some notion that objects, classes of objects, have essences -- I think it's on the handout somewhere as -- towards the bottom of page 2 says essence and there's a reference of Susan Gelman's work on psychological essentialism -- the notion of an essence is a property that's not visible or at least not currently visible.
In order to have these sort of category things, you've got to have the notion that objects continue to have properties even if you can't see that.
So, for example -- well, I won't embarrass anybody -- but there are people here presently who are not moving, who are not, to all appearances, growing.
I can't tell from this distance whether they're made of cells and they don't appear to be having babies or anything like that.
Nevertheless, I assume that they are alive.
That's part of their essence.
That this aliveness is something that I attribute to that object without being able to see it.
Now what's the evidence that little kids have that?
Well, I'll tell you that and then we'll have a break and then we'll go on to the problem of egocentric behavior.
Let's start with evidence that babies know something about objects.
Here's a tube.
If I was doing this right, I'd show you that there's nothing in the tube.
And then I would take a stuffed animal -- he's a duck.
we take the duck and he's a squishy duck, so I'm going to stuff him in the tube.
So the baby's watching all this cool stuff happening here.
There's little babies watching this stuff.
Then I take Nemo or whatever, my little plush Nemo, and I stuff him in the tube.
You with me so far babies?
Now, I reach into the tube and I pull out the fish.
And you, being good babies, stare at this.
You say this is weird stuff because I saw the duck go in first -- this is not quite Spelke's experiment.
I realize I'm getting the methods a little screwed up but the principle is right.
I saw the duck go in, then the fish went in, and I know about objects.
Objects do not interpenetrate.
Objects are solid things.
If the fish is out here, where'd the duck go?
Spelke's done a series of experiments like this that suggest that kids know about object -- babies, young babies, as early as you can successfully test them, so getting very close to being at the starting point -- know about objects, and know a few things about those objects.
Notably things like that objects don't interpenetrate.
They don't know everything about objects, so they don't, for instance, if you set up some fish and ducky kind of thing that violates the law of gravity, kid doesn't care.
They don't seem to be particularly sensitive to gravity.
Yet a two-year old in his high chair -- two-year old in the high chair thinks gravity's great.
Here's the hot dog.
OK, so dad picks up the hot dog and puts it back on the plate and the kid drops it again, because he's a little scientist, right, you gotta check that gravity is still -- you'll get there.
Marrah knows about gravity but her baby doesn't know about it yet.
Her baby looks at gravity demonstrations and says this is really boring.
But her baby knows that the fish cannot go through the duck.
It just would be bad news for the duck.
Babies don't care about inertia either.
Things like object in motion continue in motion unless acted on by--.
Tell that to a baby, the baby says yeah, right.
Don't care about that.
But I know that that fish does not go through the duck.
So that's this notion of objects.
What about essence?
Well essence is harder to get to in early childhood.
It's harder to ask the question, but by the time you get to -- how old are we in this experiment -- you're like two or three years old, but you're doing something like this.
You take the duck and you say see this duck, what we're going to do is we're going to do an operation on the duck.
I promise it's not going to hurt the duck.
But we're going to take off his skin and we're going to put raccoon skin on him.
He's going to look just like a raccoon.
OK, is this a duck or a raccoon?
As soon as the kid can meaningfully answer the question for you, to kid will say that's a duck.
I mean you dressed him up like a raccoon but he's a duck.
Inside he's got duck essence -- he doesn't say anything about duck essence, but he knows that there's a core of duckness that is not dependent on looking exactly like a duck.
How many of you are familiar with the classic Pepe Le Pew cartoons?
Yeah, it's Cartoon Network stuff.
The basic plot, there's a whole bunch of these, but the basic plot is always the same.
Pepe Le Pew is a French skunk, a French lover skunk.
And this cat, female cat, gets a white stripe down her back by some paint or something like that and Pepe spends the next five minutes of the cartoon chasing her around trying to get her to fall in love and she doesn't want to fall in love because she knows he's a skunk and she knows that she's not a skunk.
What's funny about this, the reason this is funny for little kids, is that little kids know that the fact that she's got a white stripe down her doesn't make her into a skunk.
She's still a cat who happens to look like a skunk and isn't that funny because the skunk is chasing around after her and getting hit in the head with bats and stuff like that, and all the good things that happen in cartoons.
So, not only do kids know that things have an essence like that, they are also, to be a little recursive here, they're innate nativists it appears.
They believe that these properties came with the system and were not learned.
So, Susan Gelman did a clever experiment where what she did was she said, see this kangaroo?
Take this kangaroo -- this is a baby kangaroo, it's a newborn kangaroo, and you know what we're going to do, we're going to take care her and we're going to give her to the goats and she's going to get raised by the goats.
Now she's all grown up.
Let me ask you a few questions about her.
Does she have a pouch?
Well, yeah sure, that's pretty straightforward.
That's not the interesting question.
The interesting question is is she going to be hopper or a climber?
And little kids -- two, three year old kids will say it's going to be a hopper.
It was a kangaroo at birth, kangaroo's hop.
It's part of their essence.
The fact that she lived with the goats isn't going to change that.
This is the amusement value of the Disney Tarzan cartoon, right.
Tarzan gets raised by the gorillas.
Have you seen that?
Oh man, when did that come out?
I got to calibrate my Disneyness here.
Which Disney movies came out when you were of an age to watch them?
Alladin
Alladin.
Oh, you were too old when Tarzan came out.
You're the Alladin cohort.
Jungle Book came out when I was a kid.
Well, no not quite.
The ones that came out when I was a kid were things like Sword and the Stone, I thought that was great.
That was good stuff.
And don't claim that I was a kid when Snow White came out -- that was my father.
So, Jungle Book works, right, and you get raised by the wild animals, but you're essentially a human and that eventually catches up with you.
Actually, kids are incorrect nativists because they believe things that turn out -- and it's true that the kangaroo will probably hop even if it gets raised by a goat.
But suppose we have here a Chinese baby and we take this Chinese baby and we bring it to the United States and it's raised in California.
When that kid grows up, will she be able to speak Chinese?
It depends on the--.
What?
[INAUDIBLE]
It depends on the neighborhood.
But kids aren't quite clear on these demographic issues in the same way.
But a little kid will figure incorrectly that language is one of these essential properties.
That if you're Chinese, that means you can speak -- you might be able to speak English, too.
You could learn that, they can figure that out.
But you'd have a privileged ability to speak Chinese because after all, you were a Chinese kid and that ought to work.
If you're a Chinese kids who grew up in an Anglo linguistic environment, you may have discovered that there were plenty of your relatives who also thought you should be able to speak Chinese, but that's a different issue.
In any case, little kids start certainly with the notion of objectness and probably quite early with the notion that objects have essences, and then they bild on that in this little scientist kind of a way to get closer and closer to what we consider to be the adult concept of whatever, alive or kangaroo or any of those sorts of things.
Here's the case where the adult state is not particularly fixed, but we're nevertheless, what you're doing is you're elaborating on juvenile concepts, not suddenly discovering the notion that there is such a thing as a concept.
OK, I want to do egocentrism still, but I will start doing that when that thing says 3:10.
BREAK IN LECTURE Let us talk about egocentrism.
So, you will recall that the basic problem here was that the kid seemed unable to take the point of view of another, in this case, the mother on the other side of this table.
The kid asked about things like the river and the hut, claimed that the mom could see them, even though mom couldn't see them, only the kid could see them.
You can get kids to perform quite well on this task if you come from like Nebraska instead of coming from the Alps.
So, in one version of this experiment -- so I think the answer is that four year olds failed this.
You can get three-year olds to be successful in an equivalent task, if what you do is just simplify the situation.
So there's a version of this that may have a picture in the book still, I don't remember, of sort of a city block with a car in it and a Sesame Street character.
You drive Grover around the block or something and ask questions about that much simpler situation.
Then the kid can successfully answer questions from the vantage point of the mother, as if part of the issue here was that the task was just too complicated.
Oh, I never mentioned the M&M's are under there somewhere.
I never mentioned well why is that the kids bombed the M&M task.
That seems to be an issue where you're asking kids who haven't gotten to the successful counting stage to do this and they can't do it either with the numerosity or the subitizing piece, so they fail the task.
If you make the M&M task, if you put that into either the numerosity range or the subitizing range, little kids will succeed in doing the task.
So if you say would you rather have this pile with two M&Ms or three M&Ms, little kids don't bomb that.
They bomb five versus six or six versus eight, things like that.
And a similar sort of thing seems to be going on here that the task is in some fashion a little too complicated for them.
How far can we push this back?
Well, if you take two year olds, suppose you take two year olds and you give them a pair of sunglasses.
Put the sunglasses on the kid, kid looks around for a while.
Now we take the sunglasses, they may have been mirrored shades even.
Put the sunglasses on mommy and you ask can mommy see you?
Well, you can't see mommy's eyes anymore because of the glasses, but the kid, even a two year old will be able to say yes, indeed mommy can see me, even though I cannot see mommy's eyes.
Normally, if mommy is doing this, it's a pretty good bet that mommy can't see you.
That's the root of a lot of entertaining games.
But they can manage to solve the problem, if it's simplified to the point of sunglasses on or off.
This if I can see you, can you see me thing, you can see, as you get more and more complicated, that it takes you longer and longer to solve the more and more complicated versions of it.
You don't remember what it was like to be thinking when you were a two year old, but you may remember a stage later in your life when it was not clear to you how mirrors worked.
That if a can see b, b can see a in a mirror.
This is the sort of thing that amuses early school age kids often.
I can see around the corner.
I can see dad around the corner there.
I bet he can't see me, and it comes as a shock that dad can see you, because the mirror kind of works both ways, and then there are more complicated arrangements with keyholes and stuff like that where it takes even longer to be able to take the point of the person on the other side.
So, some aspect of this is simply a matter of complexity.
In fact, Paul Bloom and his student -- actually, I should mention Paul Bloom is married to Karen Wynn.
They were both TAs for this course and are now both professors at Yale.
Anyway, Paul works on similar sorts of problems and talks about the curse of knowledge that shows up in adult behavior, too.
That it's very difficult, it's often very difficult to remember that somebody else doesn't know what you know.
So, it's a problem for me teaching.
The things that are hardest to teach are the things that I don't know anything about.
That's not surprising.
The second hardest category are things that I know too much about, because it's hard to remember how little you know about vision.
How can it be that you haven't actually spent the last 18 years of your life bathed in visual phenomenology.
You can try using that as an excuse on the exam but forget it.
We'll still take the points off.
That turns out to be an example of the curse of knowledge.
Or another example would be if you've ever been giving somebody directions while you're sitting next to them in the car, you might discover that you go past the street where they were supposed to turn because you knew you were supposed to turn there, and you just kind of forgot that they wouldn't know that and that's why you were sitting there supposedly giving them directions.
It's hard, even for adults necessarily.
It's hard for adults sometimes to take the vantage point of another person.
Now one of the great examples of this in kid behavior, a paradigm that's yielded an awful lot of research.
This gives you some insight into this, if you like this curse of knowledge problem, is what's known as the false beliefs task.
The first false belief you need is that this is Big Bird from Sesame Street.
So here's how a typical false belief task experiment might work.
You're watching this.
Big Bird comes in to the scene and he has candy.
He puts the candy in the refrigerator and he goes out.
Now, I didn't draw Grover or whatever, but Grover comes in, takes the candy out of the refrigerator and puts in the drawer.
Now, Big Bird comes back in -- he didn't see this abut you saw this.
Big Bird comes back in and you are asked where will Big Bird look for the candy?
And your answer, if you're like a four year old, is that Big Bird will look in the drawer.
You cannot somehow access the fact that Big Bird thinks in his brain that the candy's in the fridge.
When you updated your knowledge about the candy, you updated his knowledge about the candy, too, in some fashion.
Now there are endless versions of this experiment.
This is a task that younger than four will fail, older than four roughly will tend to get.
There are endless tasks that look at this and try to break it apart, see what's going on here in developing this ability to take into account what is in the mind of another person or bird, as the case may be.
This is sometimes talked about as mind reading and is sometimes described as the major social problem in autism.
That maybe one of the reasons that autistic kids and adults have a real problem relating to other people is an inability to understand that other people have mental states, which you have that ability, but here, you as a four year old, have failed to manage to keep track of Big Bird's mental state.
Let's do a variation on this.
So, here's a variation where you don't see that initial scene.
Now you are asked to guess -- the candy is either here or here.
Where do you think that the candy might be?
Well, you say 50/50.
Oh, I think the candy's in the drawer.
OK, now Big Bird comes in and you're told well, you know, Big Bird thinks it's in the fridge.
Where's Big Bird going to look?
The answer now is, even if you're as young as three, that you say Big Bird's going to look in the fridge.
Your guess isn't strong enough to overwrite your understanding of what's going on in Big Bird's head.
This tells you something important.
This tells you immediately that you're capable of understanding something about the contents of Big Bird's head.
It's not a complete blank to you.
Even younger than that, little kids can see somebody playing with a banana pretending it's a phone, right, and you ask does he know that it's not a phone?
Yeah, yeah.
The two year old will tell you, yeah, he's just pretending.
And that you have the notion that somebody else could pretend to pretend something.
So again, you can push this notion of being able to read minds back earlier if it gets simpler.
And maybe this isn't really about the contents of Big Bird's mind it all.
Here's a cute version of the experiment done by Debbie [?
Zicheck ?]
once upon a time.
Big Bird comes in, it's the original false belief game.
big Bird comes in, puts the candy in the fridge, but this time Big Bird takes out his camera and he photographs it.
He's got a picture of it.
There's the candy in the fridge.
Now, Big Bird goes out and Grover comes in, moves the candy to the drawer, you now know that the candy is in the drawer.
Big Bird comes in, you are asked where's Big Bird going to look for the candy?
Even though Big Bird has taken a picture of it, even though there's now a physical instantiation of the contents of Big Bird's mind, you still say it's in the drawer.
You as a four year old are still unable to keep track of this.
This suggests that maybe the problem is less with your ability to see into somebody else's mind, and more with something like the capacity of your working memory.
So, if you think about it this is very parallel to [?
Lofta's ?]
experiment I talked about either last time or the time before.
You see somebody go through a yield sign.
You later hear that it was a stop sign, and you believe, you come to have a false memory or a false belief, if you like, that it really was a stop sign, because we've managed to somehow overwrite your memory here.
Maybe in this case what you're dealing with is not a qualitative change where you're seeing a change in the ability to see into somebody else's mind, but a big enough working memory that you can keep track of the task.
So, another case where the development here waits on the development of another ability, a separate ability in this case, working memory.
OK, that's as good a place to stop as any.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu
In the last lecture I was talking about the fact that kids go through roughly the stages of language development.
They go through the same set of language development stages.
A babbling sort of free verbal stage, one word, two word, and then more than two word stages.
The question we're left with at the moment is well, what is it that they're actually learning when they're learning language.
There's lots of stuff that's got to be learned.
One of the things that, it's on the handout somewhere there as I recall, is they gotta do a remarkable job of learning words and word meaning.
Between about a year and a half and six years of age, in order to account for the vocabulary that a six year old has, you have to assume that children are learning five to 10 words a day, every day.
So, that's a rate sort of approaching a word an hour or so.
They're not doing it in school, right.
It's not that they're going to school and every day they're being handed long vocabulary lists.
They're just picking this stuff up.
It's an amazing and not trivial feat for figuring out how you do that.
Somebody says oh, look, there's the chalk, and the kid not only has to learn this new word, chalk, but has to figure out that it somehow refers to this box of stuff, and not to the color yellow or to the letter c or to the small boxes when Bush is president, or any of a vast number of other possibilities.
It's a very daunting problem.
I'm going to focus rother than on the word meaning problem, on a different problem, which is the problem of learning grammar.
There, there are sort of two large scale classes of theory that I have cartooned -- I cartooned them on the board and described them on the handout as the East coast view and the West coast view.
The East coast view could really be called the MIT view originally.
It really starts with people like Chomsky here and is nicely wrapped up in Steve Pinker's book called The Language Instinct, which if you turn out to be interested in the topic of language and language learning, I'd strongly recommend you read it.
Among other things, Pinker is one of the better writers in our field and it's a good read.
The West coast view was a view you held by people like Mark Seidenberg at UCLA.
Well, Mark Seidenberg's now at Minnesota, so this geographical thing doesn't work so well anymore.
Here the distinction.
Let's start with the specific problem.
Specific problem might be something like, as it says on the handout, how do you make the past tense in English?
You'd obviously have to learn to make the past tense in some other language if you were going to learn yourself some other language.
But I walked to the store.
Yesterday I did what?
[INAUDIBLE]
I walked to the store.
Today I plan to go to the airport, actually, right after this lecture.
Yesterday I did what?
[INAUDIBLE]
All right.
So, how do we go and make the past tense in English?
e-d
Yeah, stick an e-d on things most of the time, except today I'm going to sing yesterday I--.
[UNINTELLIGIBLE] I didn't singed.
Today we ring the bell, yesterday--
Rang
We rang the bell, or we rung the but -- no we didn't rung the bell yesterday.
The bell was rung yesterday, but we rang it--.
Anyway, it gets complicated.
Today I bring you all this information about language, and yesterday I--
Brought
Brought -- it's not even brang.
So the rule -- there's sort of a sub sing-sang kind of rule, there's a bring-brang.
No, brang doesn't work.
Brought.
So, kids behave as thought they have learned a rule and a set of exceptions.
The East coast view of this is that they really have learned a rule.
That somewhere deep in their little brain, you could hypothetically go in and find the rule, that it is written in there.
One of the lines of evidence for this is that at a certain point during development as they're learning this, they start to overuse the rule.
A child will say things like I goed to the store yesterday, or I bringed the ball home from school yesterday, or something like that.
They'll start to overgeneralize.
They don't do it all the time.
But they start to behave as though they are saying ah-ha, got this verb here, I know I need to make it into the past tense.
What's the rule for making the past tense?
I gotta stick an e-d on the end of it.
Even though a few months earlier they might have successfully said went.
It is as if they learn a rule, and they learn a set of exceptions.
When you go in and look for what you're trying to say, past tense of go.
If the past tense of go, you know, I went, doesn't get there fast enough, you default to the rule.
You say I goed.
This assumes a certain amount of stuff in the head ahead of time, that's what this cartoon is about.
It doesn't need to be exactly this kind of stuff, but this is the sort of stuff that you might have in the head.
Let's think, actually, going back to the lectures I was giving on cognitive development last week.
Remember, there's pretty good evidence that little kids have in their head an idea about objects.
They know there are such things about objects.
So -- oops, I didn't put objects in here.
Let's assume that there are such things as objects.
Also, we saw that little kids know something about number, like they can do that Karen Wynn style baby math.
They know that there's 1 and more than 1, certainly.
Now let's assume that they've got some innate idea, some innate capability to learn language.
Well, if I'm going to learn a language of any use, it's going to have to be able to talk about these object things.
So I'm going to have some sort of innate idea that there will be nouns.
I don't know what the nouns are going to look like in this language, but I come into the world in this view knowing that there have to be noun-like things.
Not only that, I know that there can be one thing and there can be more than one thing.
It therefore follows that I need to be able to talk about that, too.
And so my job as a little language learner is to figure out what do the nouns look like, and how do I make those into plural in my language.
I said there's an infinite number of ways in principle that I could designate a plural.
But I will learn that, for the most part, in English it's going to be add s, right?
Oh, look, there's the cat.
Oh, look, there are the cats and so on.
I'm going to have to do the same thing as I was talking about in the past tense.
I'm going to have to learn there are exceptions.
Yeah, cat, cats, rat, rats, chair, chairs, mouse -- uh, mice.
Fish -- for some reason fish is fish.
One fish, two fish, red fish, blue fish -- all that kind of stuff.
So again, I'll learn a rule, I'll overgeneralize that rule when I get to be three or four years old, and I'll learn a set of exceptions.
I will find when I become a literature professor later on and study the history of English literature, that things shift over time -- things that were exceptions or things that were ungrammatical once upon a time might become grammatical.
It's a moving target language.
You have to figure out what the current state of your language might be.
The alternative is the West coast view, which is really sort of a neural net parallel distributed processing kind of view is to say that we don't have to propose that there's all this stuff, these structures in the head in quite the same way.
What we can say is look, there's this very powerful association learner in here, a device out there looking for statistical regularity.
There's evidence -- I think I mentioned last time that little kids long before they can speak a language are already sensitive to the statistical regularities of what syllables can follow what other syllables in the noises that they are hearing.
So you've got this very powerful statistical learning devise in your head.
It ends up behaving as if it had a rule, but the rule is not written out anywhere in brainware.
It's just the way the network behaves.
In the same way that this hunk of chalk -- where's somebody who looks reasonably alert.
Do you look reasonably alert?
No, now she's looking elsewhere.
All right, how about if I just throw it out.
There we go, look at that.
Very good.
Send it back.
That's a smart hunk of chalk.
It knew to fly on a parabolic path, right, knew a whole bunch of 801 style physics.
It didn't know it in any real sense, of course.
It behaved as if, because it's operating under simple physical forces in the world and ends up behaving as if it knows physics.
It's a somewhat stretched analogy, but the same sort of thing is going on here.
You learn about the linguistic world you are in and then you behave as if you knew these rules, rather than having some sort of mechanism by which you actually, in a sense, at birth knew that there should be such rules and merely needed to figure out the particular form that the rule took in your language.
These are, in a sense, modern and not hugely different views of initial positions that are radically different.
So, if this is a continuum here, way over here somewhere would be radical nativist views of language.
Way over here somewhere would be radical imperacist views of language.
The most radical nativist views nobody particularly held, except I think I mentioned that little children are subject to this as an erroneous thought.
That's the view that you are not only born to have language, but you're born to have a specific language.
So that if you are a Saudi kid, that you are born to speak Arabic and that's all else being equa -- well, even all else not being equal.
You're a Saudi kid.
You get transplanted to Sweden, and at the right moment Arabic appears.
That's not true.
Nobody other than a small kid I think ever particularly believed this, because it was clearly falsifiable.
The earliest experiment I know in this area, and I don't know if the experiment was ever done, is something reported by Herodotus in his book about traveling in Egypt.
He says that King Psammetichos of Egypt 2,500 years ago believed that there was an original language of humankind where King Psammetichos had grown up in the Judeo Christian tradition, he might have called it the language of the Garden of Eden.
What language did Adam and Eve speak in the garden?
Well, the original language of humankind.
Psammetichos figured he would go and figure out what that language was.
He actually had a hypothesis about what it was.
He was King of Egypt.
It stood to reason that the original language of humanity was Egyptian.
So what he is alleged to have done is to take two children, give them to a shepherd, tell the shepherd to raise them with sheep and not talk to them, and to report to the King what the first word was that these children spoke.
The report is that when the children first spoke, their first word was "beccau," which sounds suspiciously like they were imitating the sheep I think.
But anyway, beccau turned out to mean bread, and that seemed like a pretty encouraging thing, except that it meant bread in Phrygian, the language of Phrygia, which is in modern day Turkey.
It wasn't word for bread in Egyptian at all.
So, Psammetichos is reputed to have been disappointed by the results of the experiment.
Look, we don't know that the experiment was ever done.
We do know or we're pretty sure we know how the experiment would have come out had it actually been done.
Which is that they wouldn't have spoken a language at all.
That in the absence of a linguistic environment, even if you come into the world with some native endowment to learn a language, which does seem to be the case, you do not speak a normal human language unless you are raised in a linguistic environment at some point.
We think we know this from an isolated set of cases over the course of historical record where kids are somehow deprived of linguistic input.
Now that happens typically in odd and disturbing kinds of circumstances, and so it's nothing like a real experiment because of course, you'd never do the real experiment, it wouldn't be even close to ethical.
This class of children are known as wolf children, a term that has always disturbed me personally.
But the illusion is not to wolfe with an e, but to the legendary story of the founding of Rome where the twins Romulus and Remus are abandoned -- I can't remember why -- but anyway they get suckled by a she-wolf.
So if you look at Roman coins and things you'll see this picture of a great big wolf and two little babies suckling underneath.
Presumably, the wolf wasn't speaking good Latin, but presumably Romulus and Remus managed to come up with good Latin somehow out of all this.
Occasionally it happens that some kid gets abandons in the woods and somehow survives.
I think there's the case of an Indian child who's described in [?
Gleitman ?].
There was a very interesting but disturbing book called Genie, written a few years ago, about a child of deeply disturbed parents who had taken her basically locked her in a closet for all of her childhood.
The bottom line is that this is not going to be good for any aspect of your development, but you don't develop a normal language.
If you don't get exposed to language during a so-called critical period, which for language is certainly over by early adolescence and is getting pretty much over, six, seven, eight years old you'd better get your language experience in there before that.
You're just never going to speak a normal grammatic language.
The door closes on it.
So, Genie, this poor girl who was raised in a closet, could be taught a certain amount of vocabulary and things like that, but could never somehow get a real grip on the grammar of a language.
It also turns out to be easier to learn a second language during the critical period, but not impossible to learn one afterwards, as those of you have just started learning French or something will be happy to know.
The claim is that it's very difficult to become a truly fluent speaker of a second language later in life.
But it's not impossible in the way it appears to be impossible to learn a first language after the end of the critical period.
The alternative, the radical alternative, to King Psammetichos on one side, if we're sticking with classical references.
On this side would be somebody like Saint Augustine who writes that he knows how he learned language.
He learned language by being explicitly taught.
The same way you'd learn anything else.
That people said that's chalk, that's a chair, that's your TA and you learned this sort of stuff.
That it was an act of learning not unlike other acts of learning.
This gets developed -- Saint Augustine was not particularly working on learning theory.
But by the time you get to the mid-20th century and the heyday of behaviorism and all that good Skinnarian pigeons in the box kind of thing, you get a series of language learning that say well, look, language learning is just a fancy variety of any other kind of learning.
You want your kid to speak English and so you reward him for speaking English.
You shape the behavior.
The same time you're trying to get the kid to do some sort of complicated thing like being toilet trained, you're also trying to get him to do things like saying I go to the store.
Yesterday I went to the store.
It's just the same thing more or less.
One of the great efforts to make that into a single coherent story is B.F. Skinner's book called Verbal Behavior -- good name from a good behaviorist.
It's just another kind of behavior.
The start of the East coast story and the start of modern cognitive science in some ways is Noam Chomsky, then a young professor at MIT, writing a scathing review of that book, which still bears reading.
It's still fun to read that, or you can go read the summary of it or the recap of it in Pinker's book.
But you have to imagine this as sort of a cultural event.
Skinner is the leading American psychologist of the time, and this young Noam Chomsky guy is going after him with a bat and both barrels blazing and any other weaponry you can think of.
It is not a temperate review.
It is insulting in many ways.
It's an insulting piece written by a young guy about a very senior, established, important guy, and it starts a revolution.
Anyway, I'm not going to go through the whole argument.
But the sort of thing that Chomsky points out is that -- you know, Skinner's talking about schedules of reinforcement.
The patterns of reinforcement for language just aren't there.
This has been extensively developed since Chomsky's time.
But you can imagine this as just an example.
Imagine yourself back home as the little kid and you go up to your mother and you say, "Mommy, I loved you so much."
What's the chance that she's going to say, "Bad child, bad child, it's I love you."
By the same token, what's the chance that if you go up and say, as a precocious three year old, "Mother dear, I'm sorry to have to say this but it turns out that I actually loathe you and despise the ground on which you walk."
Your mother is not going to say "That's a very sophisticated utterance for a three year old.
Good job."
You're not going to get -- look, that's a cartoon version.
But when you go and look at substantial bodies of child language and their interaction with the grown-ups in their life, you don't find good evidence for a systematic correction of language or a systematic reinforcement for grammatical structure.
You don't find any great sensitivity on the part of the kid to that direction when it's given.
The sorts of things that every parent -- they show up in the child language literature -- but any parent can tell you that the kid comes up you and says, "Mommy, I loved you," and you say, "Oh, that's sweet, honey but it's really Mommy I love you."
The kids says, "Yeah, that's right, Mommy, I loved you."
They don't show an awful lot of sensitivity to this sort of negative support for what they're saying.
How they actually figure out eventually that it's wrong is an interesting and important problem.
But what is pretty clearly the case is that a story that will explain how it is that I persuaded my pigeon in my intro psych class to do ballet -- I think I told that story here.
But anyway, I shaped my pigeon to do ballet.
That was great.
That such a story does not cash out well as a story for how you're trying to explain language.
It does not seem to be possible to explain language in those terms.
As noted, if you want to know more about this particular argument, I really would recommend Pinker's book.
It's good stuff.
So, what do we got?
We got the notion that specific languages are learned.
You didn't come into the world pre-wired for English, Arabic, Chinese or whatever.
Language in the abstract, it's not exactly clear what it is that's innate, but something is innate that allows you to learn language.
That something could be as specific as having, in a sense, the grammar written out in some sort of abstract form in your head and you're filling in the blanks.
It could be something as non-specific as having a device that's designed to look for regularities in linguistic input.
But something is in there that pre-disposes you and any linguistically normal human to learn language.
In fact, it's not just something that's pre-disposing you to learn a spoken language.
One of the most interesting bits of work in recent years in this area is the development of what really boils down to a brand new language growing up, as much as anything, out of whatever this innate endowment is.
This is the story of Nicaraguan sign language before the Sandinista Revolution in the '70s in Nicaragua.
I gather that deaf kids were typically raised at home and treated as though they were not just deaf, but not bright.
They weren't going to learn much.
I don't know if this is universally true, but there was a population of such kids.
When the Revolution happened, one of the things that happened was a school for the deaf was set up in Managua, in the capital, and a bunch of deaf kids were brought together for the first time.
What happened was that -- deaf kids, even in the absence of other members of the deaf community, will produce signs, gestures called home signs that indicate certain things.
It's sort of a basic vocabulary, not a language, but sort of a core vocabulary that you can use to communicate with your hearing parents, for instance.
What happened was that all these deaf kids got together and I don't know what was going on in school, but I think what was going on at night up in the dormitory was they were working something out so they could communicate with each other.
They put together sort of a proto language.
Then the next generation, the next cohort of kids come, and now there's a crude linguistic environment.
What has happened over the next few cohorts of kids is that the next round of language learning kids have systematically developed a language that now has grammatical structures in it that look like the grammatical structures of other languages.
American Sign Language, which is the common language used in the deaf community, the signing deaf community here, is a language like French, English, Chinese or whatever.
It's not English translated into hand gesture.
Nicaraguan Sign Language is not American Sign Language, it's a new sign language with its own set of vocabulary and its own grammatical structure.
But that grammatical structure seems to reflect an underlying tendency for all languages to have certain structures to them.
Let me give you one example because it's brand new -- this year's discovery in this realm.
Suppose you're telling the Jack and Jill story, right.
Jack and Jill went up the hill and then stuff happens and they fall down the hill, right.
In English, the fall down part and the hill part are separate.
They are obligatorily separate.
There's the verb part and there's the object of that verb.
You fall down and you fell down the hill.
There's no reason that would need to be the case in a sign language, right.
You could perfectly have a gesture that was blah that designates the act of falling down a hill.
And indeed, apparently, the first generations of signers in this Nicaraguan school, have signs like that.
But the new kids, the kids who came in young and are learning, developing this new grammatical version of sign language, have now separated those sings out and have a falling sign and a hill sign.
I don't know that the signs look like that at all.
The important piece of it though -- that's another misconception about signs is that the signs necessarily need to be sort of pictographic.
There's no requirement that the hill sign look like a hill.
But those of us who are complete illiterates in anything like a sign language it might as well be because it gives you the idea.
So, they have developed instead of falling down the hill sign, a thing falling hill grammar that corresponds to the same sort of constraints that show up in other languages.
This whole project has been interesting evidence for the powerful in which the human brain is there to learn, and even if necessary to create language.
Well, that raises the interesting question of well, what about non-human brains.
Do animals have language?
This has been a question for a long, long time.
It's been a scientific question for a reasonably long time.
We're going to talk about it's important to distinguish between two different sorts of language.
One would be any natural language that the animal might have on their own -- do animals talk to each other in the absence of humans intervening.
The other question is can we teach them our language, can we teach them how to talk like us in some fashion.
Within this natural and trained language dichotomy, then it's important to break the question of language down into smaller pieces because otherwise you end up in an endless sterile argument about well, what exactly do you mean by language.
If I write a question on the final exam, because it won't be on the mid-term because today's topic is not on the mid-term.
If I write a question on the final exam that basically says do animals have language, it would be perfectly possible to write a coherent answer for either the yes or no position.
But that would depend on what it is that you mean by language.
So let's break this down a little bit.
For starters, do animals have something that we could call signals.
Do they have sensory stimuli that are communicative in their intent?
Well, the answer to that is clearly yes.
There's no problem with figuring that out at all.
A peacock's tail is a signal to the peahen.
Tropical fish -- do tropical fish have all these beautiful colors all over them?
Well, the idea is that I'm a tropical fish with, oh, my goodness with stripes on, and I gotta look for another tropical fish with stripes on, because if I mate with one with spots on, nothing's going to happen.
But if I mate with one with stripes on, we get more little stripey fish.
The peacock tail story is supposed to be slightly more complicated.
The standard evolutionary story for peacock tails is that peacock tails are really stupid.
They don't help you fly, they make you more likely to get chomped on by a predator and stuff.
But a huge investment.
What this id is the male peacock saying I am so fit, my genes are so good that I grew this whole thing just for you, honey.
You really ought to come and check this out.
So it's a signal.
It's not just an awful lot of animal signals seem to be about sex and mating issues.
Not all of them.
So, Vervet monkeys, for instance, have three distinct alarm calls.
How do you know they're distinct alarm calls?
Well, you go off and record them, and then you get your -- not your iPod because they don't have the little ear -- you get your boom box or something and you play them to a bunch of Vervet monkeys.
If you play call number one, all the Vervet monkeys jump up a tree.
If you play call number two, all the Vervet monkeys jumps down a hole.
If you play number three, they all stand up and look all around.
Apparently, what this is about is if you're a Vervet monkey and you see a leopard, you give one scream and everybody jumps up the tree because the leopard can't go up the tree.
If you see a hawk or whatever the big predator bird is who eats Vervet monkeys, you give a different screen, you should jump down the hole because the bird ain't going down the hole, and makes you want to just keep an eye out for them.
So if somebody thinks they see a snake, everybody else's is just going to look around.
There's sort of a three word vocabulary there.
It also gets onto the second issue on the handout.
Yes, it should say meaning.
Do these signals have meaning?
Well, sure.
There's no point to having a signal of this sort if it doesn't have meaning.
Well, you could have a signal without meaning.
Look, I got a rock here.
This crystal could be thought of as signaling its safireness or something like that.
But no other safire gives a hoot about it.
There's no meaning there from the point of view of the rock, it just is.
Whereas, an animal's signals of the sort that we've been talking about clearly have communicative intent.
There is a purpose here.
It's not even confined just to animals it turns out.
There are trees who respond to predator attacks by releasing a chemical.
It's all quite a passive kind of thing.
The bug chews on the leaves, that ruptures the cells which causes the chemical to be released, which can then be picked up by the next tree over.
The next tree over in reponse now mobilizes a defense against that predator.
So, in a sense, the trees are talking to each other.
There's a signal there and it's being received.
So, no doubt that the animal kingdom has signals.
The equivalent in human language would be phoneme.
No doubt that these signals have meaning, at least in some cases, and that could be considered to be sort of the equivalent of morpheme.
What about human language?
What about training an animal to use human language?
Could be the case that the only reason that a chimp, who is a close relation of ours, hasn't talked to us is because nobody talked to him.
So back in the '30s, the Hayes's, a couple in Salt Lake City as I recall, tried this out.
What they did was they had a kid, [?
Maracatridas.
?]
They had a kid, they got a chimp.
They said we'll raise the two of them identically.
I don't know, the chimp might sleep through the night sooner.
They'll talk to the kid, they'll talk to the chimp.
After how long?
It doesn't say how long they were at it.
You don't want to do this for too long because after awhile, raising your chimp just hanging around in the play room with your a kid, is not going to be good for the kid, because the chimp may not talk a lot, but the chimp grows big and strong and will play with your kid in ways that your kid will not enjoy.
But the chimp didn't talk.
This is important, because any neurologically normal child raised in a linguistic environment will talk, will learn to talk, or if deaf will communicate, you and given appropriate input will communicate in other ways.
The chimp is reputed to have said, I think mamma, papa and cup, and that was it.
Now, one of the problems here that was quickly identified is that the chimp's vocal apparatus is not like yours and mine and is poorly designed to make the range of human phoneme that are important in human language.
So, it occured to subsequent researchers to try using sign language with chimps because chimps are very good with their hands -- there's no problem making the sort of gestures, there's no physical limitation on the soft to gestures that the chimps could make or gorillas.
Limited work with orangutans who turn out to be big and lazy, I understand.
But chimps, Bonobo chimps and gorillas all make good signers.
So most of the effort to teach human language to primates has involved sign language or computer keyboard arrangements where you're sort of teaching the animal to type, in a sense, rather than to sign.
Now it's true that chimps don't have the vocal apparatus to talk the way you and I talk.
But that couldn't be the whole story, because if you have a child born with something congenital malformation that makes it impossible for them to talk properly, that child will talk badly but will talk and will learn a language.
A chimp in the same situation will simply not talk.
There is a fundamental difference in the way that the chimp and the child are responding to linguistic input.
So, can you teach it to sign?
Yes.
Actually, you can teach them to sign.
They've got a whole string of chimps who do really quite well with vocabulary on the order of hundreds up signs that seem to have discreet meaning.
It's not limited even to chimps.
Irene Pepperberg who was here for a while, is now a Fellow at Harvard this year, has made a career out of studying African Grey parrots.
It's a not terribly dramatic, nice looking little parrot with a nice little bird brain.
The nice thing, of course, about parrots is that they can form the sounds -- they're pretty good with the phoneme, Polly want a cracker and all that sort of stuff.
But they're also remarkably bright.
Alex, her great African Grey has a vocabulary on the order of 50 to 100 words, I think, that he can use in various and reasonable ways.
So if Irene says to Alex, pick up the blue thing or the blue one, Alex will toddle over and grab a blue thing.
If you say get the truck, it'll go get the truck and stuff like that.
It clearly has the grasp of vocabulary and ability to use it in some fashion or other.
Mostly trained.
Almost all of the examples of animal use of a human language, any aspect of a human language, are heavily trained.
Again, that's different from a little kid where a little kid will pick up a language just by being around speakers.
There are a couple of instances now in the literature where children, the offspring of signing apes have picked up signing on their own.
So it may, in fact, be the case that you don't need explicit training.
But what does seem to be the case is that chimps and parrots can use what you'd probably consider to be morpheme -- signs with meaning of some sort.
There are issues about the data quality.
So if you point to something, if you point to this banana and ask one of signing apes what it is, you'll probably get banana but you might get something out like banana, banana, me tickle, banana, you tickle me, banana, banana or something.
You might wonder whether or not you should code that as banana.
I saw one description that said that chimps tend to run on at the hands -- you know, the expression that somebody runs on at the mouth.
Well, chimps sign a lot, these signing chimps, perhaps because they're playing the same guessing game that we're all playing with language.
What does this guy want from me?
What's he trying to -- pointing at that thing, oh yes.
If I make these banana signs, maybe a few other signs in there, good things will happen, he'll be happy with me and maybe we'll get good stuff.
So, in any case, we've got evidence for signals, we've got evidence that they have meaning.
How about grammar?
Is there any evidence for grammar in animals?
Things get thinner there.
Recall that you can use grammatical information to get yourself into the meaning of an utterance, even if you don't really--.
Did I put it on the handout?
"Colorless green ideas sleep furiously," which I think is a Chomsky-ism originally designed to point out that purely syntactic information tells you something, even without much in the way of, you know how that sentence put together.
Actually, this goes back before that.
How many of you are great Alice In Wonderland fans from your youth at some point?
Oh, very limited.
So, most people cannot sit here and quote back to me "'Twas Brillig and the slithy toves, did gyre and gimble in the wabe, all mimsy were the borogoves and the mome raths outgrabe."
You have no idea what that's about
[INAUDIBLE]
It's the Jabberwocky poem in Through The Looking Glass?
I think it's the Looking Glass.
Anyway, it's in one of the two Alice books.
But the interesting thing about it is even though you have no idea "'Twas Billig and the slithy toves, did gyre and gimble in the wabe."
You, what?
Well, but you know that now that there are some things cold toves.
They have the attribute of slithiness.
They can do things that are apparently gyring and gimbling.
You can say things like gyring and gimbling -- i-n-g'ing these things, even though I got no idea what this verb means, you just know it's a verb.
So that's the sort of power of the grammar of the syntax of the language.
To what extent do animals have a grip on that?
The answer seems to be a pretty limited grip.
There is evidence for word order sensitivity in animal signing, for instance.
So you can have the chimp equivalent of John hit Mary being different from Mary hit John.
Those are the same word, different order, different meaning.
Chimps will show a sensitivity to that.
In fact, when this was a hot issue and people were arguing about it vociferously, Epstein up at Harvard who was one of Skinner's last students, used to have a great time trying to do these fancy chimp language demonstrations with pigeons.
He showed that sure enough, you could get a pigeon to understand that you got a bunch of colored keys in your Skinner box, that if I pick red, green, blue I get food, but if I do blue, green, red I get water, and the pigeon would respond appropriately to that.
So order information is there in chimp signing and it's in there in other behaviors too.
There's order information, structural information in natural animal language, too.
It's not just ah means snake or something like that.
So, for example, bees.
How many of you have seen the waggle dance of bees?
It's pretty thin stuff here.
You get in free to the science museum here, I believe, as MIT students.
Sometime, like Friday when you're done with the mid-term or something, you want to go over to the science museum, you go up to the third floor, fourth floor where the library is.
They got a beehive there.
If you look at this beehive -- it's a big glass plane so you can look in on it -- what you will see, not so much at the time of year because they're all going to bed at this point.
But anyway, what you'll see is a bee who will be marching up the hive with a characteristic waggle of her backside, and then looping back down and repeating this while a whole bunch of other bees are gathered around -- it's easy to find because typically there's sort of a circle of bees watching this.
This is a highly structured bit of communication about where to go to look for food.
I didn't go off and review my waggle dance information, but I'm pretty sure that memory serves correctly to say that the length of the vector here gives information about how far you've got to go.
That the direction of the vector is giving you orientation relative to the sun about which direction to fly.
I think amplitude of the waggle is telling you about the magnitude of the food source -- how good is this stuff.
Got any waggle dance experts?
[INAUDIBLE PHRASE]
An altitude
[INAUDIBLE PHRASE]
Signaling how high you should fly?
[INAUDIBLE PHRASE]
That's cool.
I didn't know that.
Why would they care?
[INAUDIBLE PHRASE]
Oh, of course.
Not all the flowers are just lying on the ground.
Particularly in Boston, it could be a roof garden.
In any case, that's cool.
But you don't remember what the signal is?
[INAUDIBLE]
The looping back piece contains information too, but I don't remember what that is.
Very clever research that people do one this.
It's a lovely body of literature.
But in any case, very structured and full of a sort of native grammar.
But it's pretty limited stuff.
The newest evidence for what the limitation might be that makes animal grammar fundamentally different from your grammar and my grammar comes from an experiment by -- that I think I put on the handout.
Yeah, new paper by Fitch and Hauser where they basically were teaching grammatical structures to primates, to non-human the primates.
They were pretty good.
They were doing fine on the John hit Mary is different from the Mary hit John part.
But it was nested construction that they simply never managed to get.
So, Mary who was engaged to Sam hit John.
That's perfectly fine for you, sticking one clause inside another.
That was stuff that they could just never manage to get their primates to pick up on.
Now this is not an argument -- Hauser in particular is not somebody who spends a lot of time arguing that apes are stupid or something like that.
The great apes are remarkably clever beasts.
In studying them you find cognitive abilities not unlike a lot of our cognitive abilities.
I don't think it's a Hauser experiment, but one of the ones that comes to mind, that I could have talked about in the context of egocentric behavior before, is that important landmark in your mental development was when you learned how to lie.
It's another one of these ones that didn't necessarily get a lot of positive reinforcement from your parents when you demonstrated it to them.
But it is an important cognitive landmark because it says I know that what's in here is not what's in there, in your head.
I could tell you something that I know is bogus and you'll believe it.
Monkeys can do that too.
How do you know this?
Well, I can't remember who did the experiment, but he set up this clever experiment with a good person and a bad person.
The monkey knows both of these people.
Both of these people have access to these boxes of stuff.
The monkey sometimes has access to the box of stuff.
When the monkey gets access, he can look under the box and he finds the banana.
Oh, cool.
Great.
Or the grapes or there's nothing under there, that's boring.
Now the monkey's on the other side of the cage and here's the stuff.
If the good guy comes in, he looks under one of these things, finds the banana and he shares it with the monkey.
If the bad guy comes in, looks, finds the banana, he eats it and doesn't give the monkey any.
So now, do this a few times -- monkey's got this bit down.
Now a third party comes in and baits one of these two wells -- puts the banana under one of them.
Now if the good guy comes in what does the monkey do?
The monkey basically says look there.
You only get the look there, that's where you want to look.
The bad guy comes in, what's the monkey do?
Let's look there.
Monkeys will also do this to each other, apparently, in the wild.
There's a version where you stash stuff around a field and -- now how does it work that the monkey lies to his friends?
You arrange for monkey one to discover the good stash.
Then this monkey in some fashion goes off and tells his buddies, go look over there -- the good stuff's over here.
The monkey's, you guys go over there.
And he goes and eats all the bananas.
So, they're clever animals, but there do seem to be fundamental differences that you don't get around.
At least nobody's figured out how to get around them yet.
Nobody has figured a way how to get monkeys to anything like the richness of the grammar that human language has.
The result of this is that even if you want to concede -- this is why you can answer both ways to the do animals have language.
Suppose you decide that all you want is some evidence for phonemes, morphemes and an ability to be sensitive to structure.
Well, yeah, there's some evidence that chimps can do that, for instance.
But the hurdle of developing a productive language, a language where you can say new stuff, that does not seem to be a hurdle that any non-human species gets over in either natural language or in trained language.
So, bees -- great sophisticated stuff, but no evidence that the bee ever learns to talk about anything new with this.
The bee doesn't get to -- they don't get to sit around and apparently muse about, remember last year?
Remember that patch of flowers that was over there?
At least nobody to my knowledge has ever found the past tense in waggle or anything.
It's their up to a limit, but not further.
Now, what I will do is take a momentary break here, say a word about why does this matter, and then what I'll do is entertain mid-term questions for a few minutes before the end of the hour.
So, take a quick stretch and then let's come back and finish things up here.
Why does it matter?
This has been a very contentious topic, moreso than many -- people argue about visual attention, but people don't get really agitated about it like it's a moral issue.
People get agitated about animal language like it is a moral issue, because in some fashion it is -- it is not itself a moral issue, but it impinges on issues that have more to do with morality than with science.
Well, you could couch it in theological terms, if you like.
In Psalms, the psalmist asks rhetorically at some point of God, What is man that you are mindful of him?
What is it that makes us so special that we're different from all these animal things out there.
That's the question that becomes difficult if there is not at nice clean, qualitative difference between what an animal is and what us is.
This was one of the reasons why the advent of the theory of evolution was so problematic because it suggested that species were not completely distinct and that we were somehow a completely separate sort of creation, but that there was a continuity between all forms of animal life up to and including us.
If that's the case, who is it that gets the sort of rights and privileges that we would arrogate to ourselves, and who is it that we get to treat like animals?
The search for a nice clear -- I'm not going to give you the answer to this because there is, at least in my view, there is no nice sharp, dividing line that gives you a nice clean answer to this.
But to search for a nice clean answer could include, well, who can talk to us?
Maybe the ability to speak or communicate in a linguistic way is the critical division.
Because I mean look, think about it in terms of what you would eat.
If it were the case that you'll walk out into the field and the cow can talk to you and carry on a conversation, even sort of a simple conversation, a child-like conversation.
Good grass, like grass, nice grass.
What's the chance that you're going to eat him?
It doesn't feel quite right anymore.
It's not just language.
We don't like to eat things that we get to know.
Language is a good way to get to know something.
Third grade at the moment reading Charlotte's Web.
How many of you read Charlotte's Web at some point?
Oh, good, it's still a classic out there.
It's an issue about language and who you're going to get to eat.
There's a great one in the little Jewish kid repertoire called The Carp in the Bathtub where a big -- a Carp is basically an overgrown Goldfish -- thing comes home and is living in the bathtub and the kids love it but this thing is intended to be food, and sad things happen to the Carp.
Things that are close to you are not things that you want to kill, not things that you necessarily want to hurt.
So, how you draw that line is important.
If you could argue that it is absolutely the case that there's human language ability and animals just don't got it at all--.
Another realm where this was tried was humans are tool users and animals just aren't.
There's a sharp division there.
We can see it.
It makes these sorts of discussions somewhat easier.
It doesn't make them a lot easier necessarily.
Because look, if you have a human child who has a congenital neurological problem that means that that child will never speak -- the kid's mentally retarded in some fashion, is never going to speak.
That doesn't mean you can eat him or anything of the sort.
That's obvious sick nonsense.
So it's not a simple division.
But the efforts to figure out what makes us distinctively human, and as a result, who do human rights extend to is a serious problem, one that I'm not going to attempt to solve here, but one that is certainly worth your ongoing consideration.
What I will attempt to solve here for the remaining 10, 15 minutes or so, is I will dutifully attempt to answer any questions about the mid-term that you might have at the present moment.
A couple of people asked me, is it a hard mid-term?
The answer is that the first two TAs who read it, said boy, this is easy.
The second two TAs who read it, said boy, this is hard.
Sadly, the distinction between the first two TAs and the second two TAs in this case was the first two TAs had TA'ed the course last year and had been through this once before and the other ones hadn't.
So, you may use that tidbit of information in any way you see fit.
What other tidbits of information can I provide for you?
[INAUDIBLE PHRASE]
The closed class and open class words, it's on today's handout.
I think I mentioned this last time, but let me reiterate that closed class, so it was on last time so it's fair game for the mid-term.
Closed class words are words that serve grammaticals and tactical roles and do not -- the, and, uh -- words that don't have a definition in any straightforward sense, except to explain that the means noun is a-comin'.
And means I'm joining a couple of things together.
Open class words are the other ones.
Closed class, it's a much smaller set.
The closed class words are a very small set and the open class set is the set of all other words that fill up the dictionary and everything else.
So I think that's what you need to know about that.
Oh, the other thing to know about it is the closed class words are what shows up when you get beyond the two word stage in language development.
Little kids, two word stage kids don't say "the ball."
They say "Go ball" or something like that.
But when they start to say "I see the cat," longer utterances, that's when you start getting these closed class things coming in.
OK, what else you need to know?
Everybody knows everything, this is great.
What?
[INAUDIBLE PHRASE]
If you look at the exam from years past, the mix roughly comparable.
I didn't actually check.
There's sort of an interesting push-pull here.
The longer answers are worth 10 points, so if I write long answer questions, I can get done writing the exam faster.
I don't have to write as many questions.
The short answer questions, the multiple choice questions are a lot easier to grade.
So if I write a lot of those it's an easier test to grade, and so the mix of those two pressures leads me to sort of the usual mix, I think.
It didn't look weird, did it?
No.
Kind of the usual, some of each
[INAUDIBLE PHRASE]?
The ones of you who come here will.
The ones who sleep late -- at least it's taught in the afternoon here.
Most of the people make it awake.
Is the final in the morning or in the afternoon?
[INAUDIBLE]
Morning?
Afternoon.
Oh good.
Whenever it's in the morning there's all these sad people who traipse in an hour and a half late with some great story about I woke up when my alarm clock woke me up but then I went back to sleep and I had a dream that I was awake taking the exam.
Can't I get credit for the answers I was writing?
Yeah, it's here.
It's here.
Everybody's taking it here.
Look, do me a favor, do your best to put a seat between each of you, and if you're not putting a seat between each of you because we've run out -- because there are 300 of you and not 600 seats.
It's considered good form not to take the exam like this, particularly since the guy sitting next you is probably just as clueless as you are.
It's really, really embarrassing when your wrong answer is exactly the same as the wrong answer sitting next to you.
I mean it's OK on a multiple choice thing, but this happened -- actually, this happened to me once upon a time in grade school.
I was accused of plagiarizing a report and they showed it to me.
Here's this kid's report.
Here's your report.
These are identical.
Which was true.
And arrogant, smart kid that I was, I said, do you think I'd plagiarize from him?
It didn't take much of a brain to figure out who'd plagiarize from whom because this was a kid who was still reading the books with the print that you could read across the room.
I wasn't going to plagiarize his -- I was sort of flattered that he took my paper I guess.
Does anybody actually have a question I should answer here?
Yeah, all right
[INAUDIBLE PHRASE]?
Can I go over who?
[INAUDIBLE PHRASE]
The logic of perception.
Well, the logic of perception is Irvin Rock's jargon for what I was really talking about in lecture, for instance, as inference.
I think maybe in the book also is Helmoltz's idea of unconscious inference.
It's this idea that you're continuously making the best guess you can from an impoverished stimulus.
That you've got a set of rules built into that help you to try to figure out what it is that you're looking at.
There's more to Rock's argument than that, but not much more in the book, as I recall.
Something about like that.
There was another hand over there a moment ago.
Well, now there's one here
[INAUDIBLE PHRASE]?
Lectures or books.
This is the triage question.
I haven't read the book, I wasn't at the lectures, now what do I do?
The answer is that the intersection of book and lecture is the best place to devote your energy.
So things that were covered in both are probably important enough that -- well, they're important enough that Gleitman thought it was important and I thought it was important.
That sounds important.
Beyond that, I used to say that because I am as egocentric as the next person, that things that are in my lectures but not the book are more likely to show up than things that are in the book but not in the lecture.
But I'm not really sure that's true anymore.
So I would say that after that, if I was trying to figure out how to ration my precious time, I would look at the set of kind of bullet questions that I put on most of the handouts, until I ran out of time, for the chapter readings.
Because what you've got there is what I think is important in the chapter, and that's a pretty good bet on things that I might ask about.
I got no particular stake in being arbitrarily evil and saying can I find some really teeny little piece of the book that I can quiz you about and eh-ha, nobody got it.
The amusement of that wears off after the first 20 years of teaching the course it turns out.
So, another way of looking at it is I tend to ask questions -- when I'm going for stuff that's just in the book, I tend to go for stuff that's covered in more than one sentence in the book.
So if something is worth a page to Gleitman, you might as well pay more attention to that than a passing figure caption.
All right, we'll take one more back there
Is the exam an hour or an hour and a half?
Hour and a half.
You should know that it's no longer than it used to be when it was an hour long.
So you'll be fine
[INAUDIBLE PHRASE]?
What genre of book is it based on?
Printed.
Good luck.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license at MIT OpenCourseWare in general is available at ocw.mit.edu
The midterm marks is sort of an interesting dividing point in the subject material of the course.
It's not an absolute division by any stretch of the imagination, but it's a division.
Up to this point we've been talking about things that we could measure of various varieties like your memory span.
You can remember 7 plus or minus 2, color names, you can see these colors, you can do this, you can do that-- in these cases the you has been a sort of a plural you and the data point of interest has been the average data point, the mean data point, what's normal, what's standard.
We measure people's ability to memorize color names, the average of that we've decided is going to be 7.
They'll be some variation around that, but it's the 7 that's been interesting.
That turns out not to be the case in the same way when you're talking about something like intelligence.
Particularly, intelligence testing-- announcing that on average people have average intelligence is kind of boring.
What's interesting about intelligence and what is interesting about things like personality is the variation around the mean and the explanation of that variation.
And that's what we need to start talking about now.
Oh by the way, while it is of course, by definition the case that on average people have average intelligence, they don't believe it.
If you ask people, do you think that you are more intelligent, less intelligent or about of average intelligence compared to the rest of the population you'll get some distribution around that, too.
But you'll find out that the average perception of intelligence is that we're all above average.
This goes for a variety of other questions like, how good looking are you?
Above average, below average, or average?
Well, we're all a little above average there, too.
It turns out that there is one group that gets the answer correct, that if you ask this group of people on average-- you brighter or dumber than average?-- they'll come out in the middle.
You ask them, are you cuter or uglier than average-- they'll come out average.
Anybody know who that group is?
[UNINTELLIGIBLE] Hand, we need a hand.
No hands, nobody who cares to speculate-- yeah, yeah.
Yes, you with the computer there
Kids
No, I don't think so.
I don't know what the answer is.
Certainly it's not true for parents of kids-- all of whom know their children are above average
Depressed people?
Yes, it's the depressed.
Depressed people have an accurate assessment of their own intelligence and good looks.
In fact, it has been seriously argued that part of what keeps us undepressed is an unrealistic assessment of the world.
We're smart, we're good looking, we're going places.
If you knew the truth we'd all be depressed.
But that's the topic for another day.
Like when you get the midterm back.
Oh, I shouldn't have said that.
But the midterm doesn't depress us actually, it provides a certain amount of lighthearted merriment for us you'll be happy to know because people do come up with some great answers to stuff.
So, as I say, we've been interested mostly in the mean value of measurements to this point.
Now we're interested in the variation, the distribution of those points for a wide range of measures.
If you measure a whole population of people and you count the number of people-- whoops-- who fall into each bin, you know, have a bunch of bins here-- different scores on a test, let's say.
You'll get one of these bell shaped or normal curved distributions.
The questions that we're interested in here are questions about, where does that variability come from?
The variance of a distribution is the sum of the squares of the difference between the mean and the data points.
So, got the mean, got a data point-- take that distance, square it, sum all those up and divide by the number of observations.
That's the variance and the square root of that is one standard deviation away from the mean, so this would be-- and those units of standard deviation are sort of the yard stick for how far away you are from the mean of a distribution.
So if you take something like the SAT, for example, the SAT is scaled in such a way that the mean is intended to be at about 500.
At least that's where they started and each hundred points is one standard deviation away.
What that means, for example, is that if you've got 700 or above that the number of people getting 700 or above on a properly normed SAT type test is about 2.5% of the population-- actually, that line would be at 1.96 as it shows on the handout if you want the 2.5% point.
But roughly speaking two standard deviations above the mean gives you about 2.5% of the population.
Three standard deviations above the mean, so that 800 score gives you-- I can't remember-- it's a very small percentage of a normal population above the mean.
That's at least the ideal for something like an SAT test.
Works well for the SAT 1 Math and Verbal kinds of things.
They are more or less normally distributed.
The place where it's a disaster is things like the SAT 2.
How many of you took the SAT 2?
Their are two versions of it, right?
There's the hard version, the easy version.
How many of you took the hard version of the SAT 2?
INTERPOSING VOICES]
What?
Oh, sorry, the math.
There are two versions of the math.
I got my jargon wrong because they used to be called achievement tests, they're now SAT 2s, right?
Is it still AB and BC or something?
[INTERPOSING VOICES]
Well, whatever it is-- I don't care what it is.
This is-- whoa, I'm losing my glasses now.
Getting too excited here.
The problem is I took the easy version.
I took the easy version of it because I looked at this distribution and I realized that the whatever it is-- the fancy version of the SAT 2 actually has a distribution that looks like this, which is to say everybody who takes it gets an 800 on it and those of us, mathematical incompetents who were going to score-- I don't know, 760 or something, we were going to come out in the fifth percentile and not from the good side of the distribution.
So I took what was then the achievement test because I knew that-- the math 1 achievement test-- because I knew that I could score in the upper end of the distribution.
The other people taking that were people who couldn't add and subtract.
So anyway, it worked for me.
Where does variability on tests like this come from?
There are lots and lots of potential sources of variability.
One important point to make at the moment is to note that things that cause variability across groups are not necessarily the same things the cause variation within groups.
Let's give a silly example, if you grow a bunch of tomato plants and you vary the amount of fertilizer you put on them-- more fertilizer, bigger plants, right?
So you can have something that shows you that you can account for some of the variability in the size of the plants from the fertilizer.
Now if you grow some tomato plants and some redwood trees you get very different heights too, but the source of variability between tomatoes and redwoods is different than the source of variability within tomato plants.
This becomes relevant in more subtle and interesting ways when you start talking about variation within population groups and across population groups on a measure like IQ or intelligence.
So IQ is just a number on a test, but it's supposed to be a measure of intelligence.
What is it actually measuring?
Well there's a certain circularity that's popular in the field, which is to say that intelligence is what IQ tests measure and what do IQ tests-- well, they measure intelligence, but it'd be nice to be a little more interesting than that.
A little more interesting than that is to note that you can divide up this sort of intuitive idea of intelligence in various ways.
One of the interesting ways to divide it up is into so-called fluid and crystallized intelligence.
With IQ tests being a punitive measure of fluid intelligence, fluid intelligence is the sort of set of reasoning abilities that let you deal with something like abstract relations.
It is said to have reached its mature state, its adults state by about age 16 or so.
So you guys have pretty much leveled out on that.
Crystallized intelligence is more the application of knowledge to particular tasks and can continue growing throughout the life span.
A particular example would be something like vocabulary.
Your vocabulary is not fixed and with luck will continue to grow as you age and the idea that the IQ is picking up intelligence.
Tests that are IQ like are more tests of fluid intelligence than they are of this crystallized intelligence, at least that's the intent.
What is it that is this fluid intelligence?
One possibility is that when you talk about somebody being mentally quick that that's literally what you're talking about.
That what differs between people who score high on these sort of tests and people who don't is simple speed of response.
And it is in fact the case that simple reaction time is related to measures of IQ.
People who bang a response key more quickly in a reaction time experiment are also people who score higher on IQ tests.
Not a perfect relationship, not even a hugely strong relationship.
And in fact, we probably want to look for something a little more subtle than that in understanding what intelligence might be.
More interesting are claims that it has something to do with the constellation of operations we were talking about in the context of working memory earlier in the course.
These sort of executive function-- the desktop of the computer of your mind kind of functions.
How much stuff can you have up there and how effectively can you manipulate it?
So one possible correlate would be something as simple as digit span or in this class, color span.
How many color names can you name?
I go red, green, blue.
You say, red, green, blue, that sort of thing.
Again, related.
It tracks along with intelligence, but not all that well.
People like Randy Engle in Georgia have worked on creating tasks that they think capture this working memory executive function aspect of it better and that's-- I think, I put on the handout this notion of active span tasks.
These are tasks that are like the color name task, but a little more complicated.
So the sort of thing that you might do if you were in Randy Engle's lab would be something like this-- what I'm going to get you to do is I'm going to read you a list in words and you're going to spit them back to me in order.
And the measure that's going to relate to something like an IQ test score is going to be how many you can get back in order.
But I'm not just going to read you the names.
What I'm going to do is give you a pair-- an equation and a word on each presentation.
Either on a computer screen or I can do this orally.
So I might ask you to verify, is it true that 2 plus 3 minus 1 equals 4?
And at the same time you see the word uncle.
And so your job at the time is store uncle and verify the equation.
OK, next one would be 4 minus 3 plus 5 equals 6 and the word would be fish.
So-- no, that one doesn't sound right.
OK, now what were the two words?
Uncle and fish.
That's good.
But if I was to do this without the long song and dance and go up to 4, 5, 6 of these, you would discover that you started losing them.
And the number that you can hold while doing these calculations at the same time turns out to be a more powerful predictor of things like IQ scores.
Again, not perfect, but getting closer, perhaps, to what the underlying substrate of what we mean by intelligence might be.
That it might have something to do with how well you move things around in that mental space that we were calling working memory.
Another possibility, not necessarily unrelated, but another possibility is that it has to do with the degree to which your brain is plastic-- not in the plastic kind of sense, but plastic in the modifiable sense.
That maybe the ability to change and modify the structure of your brain is the neural substrate of intelligence.
In any case, whatever it is, it is a useful predictor of a variety of things.
It's a usable predictor of performance in school, which is in fact where it started.
Binet in France-- that's B-I-N-E-T, but it's French, so it's Binet, in France, started intelligence testing as a way of seeing which kids might need help in school.
It is a predictor of a variety of a things.
So more IQ points, higher lifetime salary.
More IQ points less of a chance of a criminal conviction, less of a chance of teen pregnancy.
So it's related to stuff that you'd like to know something about and you'd like to know as a result where it is that the variability comes from.
Part of the class of statistical tests for where variances come from, the effort to parcel it out you will have seen in a variety of the papers that you read, probably-- statistical tests called ANOVA.
It stands for Analysis of Variance.
It's part of the statistical armamentarium that allows you to take the variance apart and say, some of due to this and some of it's due to that.
Now this is made simpler by the fact that variances are additive.
So if you have two sources of variability that they add in a nice direct way.
So if we've got the total variation and we are in a psychology course one way to divide up the variance, at least theoretically, is into variance that's due to genetic factors-- our nice nativist component and variance that's due to environmental factors.
In principle, if this is a good way to think about the variance we should be able to go in and do experiments that allow us to see the genetic variance and the environmental variance and see how they add up to make the total variance.
What you will often see in papers on intelligence and actually, widely in the popular literature on the genetic component of pick your favorite function.
Chance [UNINTELLIGIBLE], what is the genetic component to male fidelity or something like that.
You'll often see discussions of so-called heritability usually written with a big H. Heritability is simply the-- whoops-- the genetic variance over the total variance, which in this story would be the environmental variance plus the genetic variance.
There are difficulties with thinking about-- we're withdrawing conclusions from that that we will come to shortly, but the first thing I want to do before going on to that is to talk about this sort of data that are brought to bear to understand variability.
Because a lot of this is so-called correlational data and it is important to turn you into educated consumers of correlational data, so that's why you have this lovely second page of the handout.
Correlation is simply a way of describing the relationship between two variables measured on the same subject.
So let's take the silly example in the upper left there.
If I took everybody here and measured your height in inches and measured your height in centimeters and for each person-- now each person generates a data point-- those data points had better lie on a straight line, right?
Those are two measures that are really seriously correlated; one with the other.
If I know inches, I know centimeters.
Correlation coefficients are calculated, so you calculate the line that fits the data.
And then you calculate how strong your correlation is by looking at the distances of data points away from that line.
In this case, all the data points lie on the line and that leads to a regression value, regression coefficient-- usually written as little r of 1.
A correlation coefficient of 1 means if I know this variable then I know this variable perfectly.
Now those are really boring data.
Nobody spends a lot of time working out correlation coefficients for that.
You work out correlation coefficients for data where there's some variability.
That's the whole point here.
So the second one is height and weight-- actually, taken from a subset of a 900 class some years ago.
If you measure height and weight you now get data points that are clustered in a broken hunk of chalk around some line.
If you calculate the regression coefficient now it's going to be something less than 1, but still positive.
So greater than zero, less than 1.
What is it actually on the handout?
Like 0.78 or something?
The relationship between height and weight is pretty strong.
So if I know that your 6 foot 5 I don't know your weight exactly, but I can make a nonrandom guess about that weight.
On the other hand, if I know your height-- let's plot, OK, we still have height there.
So let's plot last two digits of social security number against height.
It is my firm belief-- I don't know this to be true not having collected the data, but it is my firm belief that those data are just a random cloud of spots, right?
I don't think there's anything about social security number that's related to your height, I hope not.
So, if I know your height I know squat about your social security number.
That is a correlation of zero.
Correlation can go below zero, but below zero is not worse than zero, it just tells you the direction of the correlation.
So let's go back to the inches-- OK, we'll stick with height.
So this time I'm going to measure-- get everybody here-- I'm going to measure their height and I'm going to measure the distance from the top of their head to the ceiling, very exciting data collection.
I'm going to get orderly looking data, but this time it's going to look like this right?
I've just changed the slope.
That's going to give me a correlation in this silly example of minus 1.
It's a perfect correlation, but the direction of relationship is the other way.
If I know distance from the floor I absolutely know distance from the ceiling.
It's just that as one goes up the other one goes down.
Again, that's a really boring example.
A more interesting example is on the handout, which is if you come into my lab and we do a reaction time experiment and I plot your average reaction time in whatever the task is and I plot your error rate, what I will find across individuals is data that are noisy, but look like this.
The faster you go the more errors you make.
It's known as a speed accuracy trade-off in dull literature, which you will notice has an interesting set of initials, so SAT gets written about in my trade all the time, but has nothing to do with standardized tests, it's a speed accuracy trade-off.
But it will produce a negative correlation.
I think I put one of those on the handout, too.
What's the correlation there?
Negative 0.52
OK, so negative 0.52 meaning a pretty good relationship, but going in this negative direction.
You will often see in papers r squared rather than r. r squared of course is always positive.
So r squared here would be what, about 0.26 or something?
The reason people use r squared is it turns out that that gives you the percentage of the variance that you're explaining.
So if you've got a correlation of 0.5 you can explain a quarter of the variability.
If you know one you know about where a quarter of the variability would come from in the other variable in this case.
Now it should say on the handout, oh actually I think I put the answer on the handout this time too.
I'm not sure it's really the most important thing to know about correlation, but the thing that gets lost in discussions in psychology all the time, certainly in discussions of intelligence all the time about correlation is that correlation does not tell you about causality.
It is extremely tempting, it's not extremely tempting in these examples, right?
Does inches tells you about centimeter?
Yeah, but not because inches cause centimeters-- that's stupid.
But it's really tempting to infer directly from nothing but correlational data, to infer causality.
I put fertilizer on the field, the tomato plants grow bigger, so it I plot amount of fertilizer in yellow now-- cool-- against height of tomato plants I presumably get data that look like this, some nice positive correlation.
I may have a notion, I may even have a correct notion that the fertilizer is the cause of this change in the height of the tomato plants, but just these data don't tell me that.
I would need more.
I'd get exactly the same data if I flipped the axes, right?
Height of tomato plants, fertilizer.
Get exactly the same data.
The height of the tomato plant causes how much fertilizer I put on it.
No, that would be stupid.
You have to impose a theory on your correlational data.
The correlation's just math.
It's not by itself a theory.
It can be used to support causal theories, but it is not by itself a causal theory.
I'll try to point out later where this runs into trouble.
OK, so we can get correlational data.
Correlational data are very important data in the study of variability in intelligence, so you might get data like the following: let's plot parent's IQ against child's IQ.
What you'll get is another one of these clouds-- I think it actually says on the handout that the r value is about 0.5 for that.
So it is the case that if you know the parents IQ you know something about the child's IQ.
And the goal is to figure out why, where's that relationship coming from?
The obvious temptation of course is to think that the answer is, well, the parents have good genes or bad genes-- they have some amount of intelligence coded into their genes, they pass it onto their kids.
To understand why it's not completely trivial recognize that parental wealth is correlated with child's wealth not because there's money coded into the genes.
It is true that the richer your parents are the richer you're likely to be, but the causes-- you do inherit that, but in a different kind of non-genetic sort of way.
As I say, the simple story has been to try to partition the variance into a genetic component and a environmental component.
The simple version of that leaves out an important piece of it, which is that if you do an analysis of variance-- everybody should do an analysis of variance sometime because the computer will do it for you on your data now.
Everybody should really do one by hand-- back when we were young we had to do them by hand, which took a long time.
But anyway, when you do that, suppose you've got two variables, like a genetic component, an environmental component.
An analysis of variance will tell you-- this statistical test, think this much of the variance is due to this and this much is due to this.
But it'll also give you an interaction term to say that the genetics and the environment might interact in some fashion.
Interestingly that term never gets introduced into these calculations-- doesn't quite get introduced in these calculations of heritability, but let me try to give you a feeling for why that's important.
Let's move off of intelligence and ask about a sort of a personality variable.
How anxious do tests make you?
You've now taken a bunch of MIT tests, how anxious do you get?
Well, let's develop a little variance partitioning theory here.
Let's not develop that theory, let's develop this theory.
Whoops-- oh this looks like a great theory.
Get rid of that theory, that's got too much fancy stuff in it.
We will have a simpler theory.
So the simple theory might be that the total variance, the total anxiety is a function of the variance due to you-- to your personality-- are you an anxious person or an unanxious person?
And the variance due to the class.
You know, if you're taking introduction to clay ashtrays-- make you that anxious and if you're taking advanced thermonuclear chemical integral thermodynamical bio-something or other and you skipped all the prereqs- yeah, it makes you a little anxious.
And you could try partitioning your variance into these two components.
But it turns out that's not where the action is.
The action in anxiety about tests is all in the interaction term, which we can call you crossed with class interaction term.
Because how anxious you are depends on how you are doing in this class.
You-- know I'm sure it matters if you're basically a nervous person or not, but you're going to be nervous in intro psych if the midterm's coming up and you never cracked the book.
You might be less nervous in calculus because you're good.
Your neighbor might have memorized Gleitman and forgotten to show up in calculus and have the flip anxiety, the anxiety is dependant on the interaction.
And those interaction term, I mean, you will almost always get a nice nod to the notion that of course we know in something like intelligent that the environment and genetics interact, but we will typically not get much more than a nod at that.
You then get simple minded statements about just how heritable something is.
Let me give you an example from the intelligence literature from a new and I gather somewhat controversial study-- controversial in the sense that there are other people who claim that the methodology is flawed and that they don't believe the results.
But this is new enough that we don't know how it shakes out, but it makes the point about the importance of interaction terms here.
Suppose you calculate how heritable intelligence is, but we're going to do it separately as a function of, oh here, let's introduce a little more jargon.
SES stands for socioeconomic status.
It's the fancy way of asking how much money and other resources you've got.
If you look at high SES kids and ask, how much of a genetic component is there to IQ?
You get estimates, as I recall, it's something like 0.6, 0.7 something like that.
A relatively high number for that H, that heritability number.
If you look at low SES kids, in this particular study it was dramatically lower, about 0.1.
What's going on there?
Could it possibly be the case that the genetics are operating differently at the low end of the economic scale than at the high end of the economic scale?
I mean, this implies that if you plot something like parent's IQ against kid's IQ-- one of the ways of looking for a genetic contribution-- that at the low end of the economic scale that the data look like a cloud and at the high end they look closer to the line kind of data.
That's very odd.
What really seems to be going on is an interaction between environment and genetics influencing this heritability thing.
What's going on?
Well, here look at this equation and ask yourself what happens if you drive the environmental variance to zero?
Well, you're going to drive this up towards 1.
It's just going to be the genetic variance over the genetic variance.
It may be that for the point of view of intelligence or what's measured on IQ tests, that by the time you get into the middle class the environment is largely homogeneous.
Everybody gets fed, everybody goes to school, everybody has books, and everybody has access to medical care.
Most people come from family situations that are at least not vastly problematic.
At the lowest end of the socioeconomic scale that's not true.
It's not true that everybody has the same access to resources and this environmental factor makes it much larger with the result that this measure of the apparent heritability, the genetic component if you like, seems to get bigger.
So you see how the interaction can end up being important.
All right, let's go back to this example of parents and kids across the population as a whole that relationship gives you an r squared value of about 0.5.
What does that mean?
Well, we don't know really because while you share some genetic material with your parents, assuming you're not an adopted child-- we'll come back to that in a minute-- if you share genetic material with your biological parents, if you were raised with your biological parents you also share a lot of environment with them.
The two factors co-vary.
In order to sort of separate this out what you want to do is you want to get the two factors to vary at least somewhat independently.
Now the little table on page 3 of the handout gives you one effort to do that.
Let's look at sibling pairs who vary systematically in the amount of genetic material they share.
So identical twins are genetically identical and they have very high correlations between their IQs, a correlation of around 0.9.
Fraternal twins-- same sex fraternal twins I should've added-- have a correlation that drops to about 0.6, which tells you that environment, well, they are only as related as standard siblings.
And so you reduce that genetic component, you reduce the correlation.
Siblings, age different siblings- standard brother sister, brother brother, kind of pairs have a correlation of 0.5 and if you have unrelated children raised in the same family that correlation drops to about 0.2.
So that's certainly indicates a genetic contribution to IQ, but it's not entirely clear what's going on because again, environment and genetics are co-varying; one with the other.
Identical twins get treated more similarly, they have a more similar environment than fraternal twins.
Fraternal twins by virtue of being the same age are treated more similarly than age separated siblings.
And typically, adopted children are treated somewhat differently.
Children who have been adopted into a family have experiences that kids born within the same family don't have and vice versa.
So again, there's more of a difference there.
So environment and genetics are co-varying, so it's not a clean experiment.
One way to do the clean experiment is to take a jumbo jet full of identical twins at birth, put little parachutes on them, fly around the world and push them out at random.
Come back 18 years later after fluid intelligence has reached its asymptotic level, collect all your twins and see what the correlation is.
This is, for a variety of reasons, a difficult experiment to actually do and so we don't have the data on that.
But there is a literature on identical twins reared apart.
It's not common, but it does happen.
It happens when either both twins or one of a pair of twins gets put up for adoption.
The one of a pair of twins thing may sound a little strange, but this happens for instance, typically at the lower end of the socioeconomic scale when you're thinking, oh my goodness.
I think we can just barely manage this kid when he or she is born and there's two of them.
And so one of them gets put up for adoption.
It's rare, these things are rare.
But it does happen, and a variety of efforts have been made over the years to collect data on such.
Some of this has less to do with IQ than with the sort of general personality variables.
There's a whole lot of amusing, though god knows what to make of it, literature on identical twins who only meet their twin-- didn't realize they had a twin until they're adults and then they meet and oh my god, they both married women named Gladys and they both have dogs named Gerald Ford or something like that.
That's a little strange.
It's a little hard to believe that [?
Gladys-ness ?]
was wired into the genes, but weird coincidental things happen.
The identical twins reared apart stuff is again, not perfect data.
Because for instance, twins that are separated at birth are sometimes separated by going off to live with aunt so and so, who lives in the next valley or something like that.
Is that really separated?
Hard hard to know.
Oh, the place where you do these studies-- well, there's one thing on the handout about the Minnesota twins study because those guys are at Minnesota, but the place you want to do these studies is Scandinavia.
Not because they take your twins apart a lot in Scandinavia, but because boy, do those guys keep records.
You get yourself some idea that you want to know where all the twins who were separated at birth are in Denmark and there's somebody in the Danish bureaucracy who could get that answer for you.
I mean, in the U.S. bureaucracy you're lucky if they can figure out-- oh, I don't know-- where your 300 tons of the Iraqi explosives are or something like that, but in Denmark they'll know where your stuff is.
The previous remark should not be construed as being paid for by any particular political campaign.
It's just what bubbled into my fevered brain here.
But in any case, when you do the identical twin reared apart study, people yell and holler about these, but the best of these data look like a correlation of around 0.7 or so.
Significantly lower than the 0.9 for identical twins reared in the same family, but clearly indicating some sort of a genetic component to what is being measured by IQ.
So I think it's been a very hot topic for a variety of reasons, many of them not well formed to ask how much of a genetic contribution there is to IQ.
And it's been an answer where the answer that you're looking for is heavily driven by your politics as much as by your science.
In part, well look on the handout.
There are a bunch of pitfalls to this idea of heritability.
Jumps down to number 3.
Number 3 is perhaps the reason that it's been most politically loaded.
There are people and there are groups whose IQ, on average, are lower than other groups or other people.
If you believe that there's a strong genetic component to IQ and you believe that this item 3 here, that things that have strong heritable components are largely unmodifiable then what you're saying is that if you've got low IQ that's sort of your tough luck.
There's nothing much that we as a society could do about it.
This was the thesis of-- actually, it's been a thesis of a repeated series of books.
The most famous recent one is a book called The Bell Curve by Herrnstein and Murray and the basic argument ran like this, there's clear evidence for heritability of IQ.
IQ is not very modifiable.
IQ is correlated positively with good stuff and correlated negatively with bad stuff.
And that this country is an IQ stratified country.
Another way of putting it is a meritocracy.
It's not that you get born the Duke of Cambridge or something like that, you get to rise to the top because you've got this great IQ that gets you to MIT, that gets you the good job and then you become President of the United States or something like that.
Interesting.
Or you get this great IQ and you come to MIT and you hang around in the infinite corridor making [?
boingy ?]
noises.
That's different.
Anyway, if you take-- I don't know, that 4 points or something-- you take those 4 points and you add them up, what you get to-- somebody over there take a little walk down the hall and ask [?
boingy ?]
to cool it.
Thank you.
What you get to is the notion that there are some people who are going to be at the bottom of the stack in America.
And that's just the way it's going to be, you know?
They're going to be the criminals, they're going to get pregnant, they're not going to make any money and that's just tough.
You know, there's nothing much to be done about it.
I'm oversimplifying the book.
It ought to be fairly obvious that I disagree strongly with that thesis.
But I should say, it's not a stupid book.
If you're interested in these issues it's worth reading the book in the way that you know, if you're on the left politically it's worth listening to right-wing talk radio to kind of sharpen your brain and if you're on the right politically it's worth listening to left-wing talk radio to sharpen your brain as long as it-- well actually talk radio's probably a wrong example because it really is pretty stupid; most of it.
Herrnstein and Murray aren't stupid.
I think they're wrong, but they're not stupid.
In any case, let me give a non-intelligence example for why this pitfall of thinking that these things are fixed and unalterable just because they're inherited.
Did it work?
Oh, you weren't the person looking for-- it's not the same guy.
Oh, this is a lovely change blindness thing, right?
It's perfect.
A guy went out, the [?
boingy ?]
stopped, the guy came back.
Unfortunately I guess, the [?
boingy ?]
guy has killed the guy who went out.
This is really sad.
We could send another one out, but where was I?
OK, so things can be inherited, things can be inherited related to intelligence and nevertheless quite changeable.
PKU is a disorder where you are unable to metabolize one of the basic amino acids and the result is it produces zero toxins.
It munches up your brain-- kids with this disorder, it's a genetic birth defect type of disorder-- kids with this disorder before it was understood were condemned to severe mental retardation.
Once it was figured out what the problem was you could prevent the severe mental retardation by controlling diet.
Basically, by controlling an environmental factor.
You take the precursor out of the diet, you don't produce the neurotoxins, you don't produce the mental retardation.
It doesn't mean that the disorder was any less heritable.
It's a birth defect.
Are you the guy?
Is he the guy?
Thank you.
What was it?
It was just someone walking through the hall making noise.
He left
OK, he's not going to need medical care or nothing, right?
He's
OK, good.
No, I was worried that you might have worked him over or something.
Thank you for taking care of that in any case.
So you can have something that's clearly genetic in origin, where an environmental change might make a difference.
PKU is simply a dramatic example of that.
Let me just take a look.
OK, so I've already hit the pitfall 2, the notion that the interaction term might be important.
The evidence for that might be this low socioeconomic status, high socioeconomic status influence on the apparent heritability of it.
And I really also already hit that first pitfall there.
High heritability might just mean that there wasn't much variation in the environment in your particular experiment.
If you grow all the tomato plants in exactly the same field with the same water, the same sun and the same fertilizer, there will still be some variability of course.
You'll get a heritability score that will be near 1, but that's because you've driven this term down to zero or near zero.
And that doesn't prove that the environment is unimportant.
So the fact that there is a significant genetic component doesn't mean there is not a significant environmental component.
OK, take a quick stretch, make [?
boingy ?]
noises
[UNINTELLIGIBLE]
That's good.
Thank you
Hi, we actually decided that we were going to go over the midterms in class
Yes, of course.
Go ahead
OK, but my class is right after this
No, go ahead.
Absolutely.
No, this is in case there are all those society for neuroscience TAs.
No, by all means.
I want to cover for them, but not to discourage you.
Looking at the time and looking at my notes I realize I'd better get cooking here.
So let me cook here.
What I want to do is to tell you a little bit, well, let me frame this.
Since I've already introduced the Herrnstein and Murray book, the most controversial piece of the Herrnstein and Murray kind of story is when you start getting to group differences and you say, look, in America at the present time the average African American IQ score is about 10 points lower than the average white American score.
That's data.
And what you care to make of those data is very important from a policy point of view.
Herrnstein and Murray were making the argument that since genetics has this big role and these are things that are unalterable that the fact that blacks in the U.S. also, on average, make less money and so on.
You know, hey, man, that's just science and genetics.
And the fact that we're a meritocracy and good things like that, live with it.
What I want to do first is to describe a couple of what are really sort of amusing historical anecdotes about the history of intelligence testing drawn from Stephen Gould's book called The Mismeasure of Man, I will answer the question on the handout already.
What's the point of these amusing stories?
The point is not to make fun of folks operating 100 or 150 years ago.
The point is to make us take with caution our understanding of similar answers that we're getting today.
So for example, who has a higher average
men or women?
How many vote that it's men?
How many vote that it's women?
How many vote that it's equal?
How many vote that I ain't touching this because I smell a political question when I can-- how many vote, I don't vote?
That's the wrong answer.
At least, next Tuesday.
This ad paid for you by the league of women voters.
Just stepping aside here, it's sort of a pain often to vote if you're an undergraduate, right?
Because your way away from whatever district you supposed to vote in, but go and vote.
If for no other reason than nobody polled you and you can screw up all those polls that claimed that they knew the answer-- which they don't know-- to the election one way or the other by voting because when they called your parents you weren't home.
So anyway, but you should vote.
Oh, yeah, so men and women-- the answer is that men and women have the same average IQ.
But the interesting question is why that's the case?
Why is it the case?
Well, back at the beginning of the 20th century when IQ tests came from France to America, how do you build an IQ test?
What you do is you make up some questions that you think might be reasonable measures of intelligence, you give them to a bunch of kids because this was originally a school testing kind of thing, and then you sort of ask, are these sensible questions?
Like, do the kids who the teacher thinks is smart, do well on them?
And you work from that.
And so they're working on standardizing the original tests and on the original drafts of the test women, girls were scoring about 10 points higher than men.
The guys literally, who made up the test knew that this was wrong.
Now it is to their credit in the early 20th century that they knew a priori that the correct answer was that men and women were equal.
It would have been no big surprise at that point to have them figure out that men should have been scoring higher, but they knew that the answer was that men and women were equal.
And here's what they did, you do what's known as an item analysis on your test.
You look at all the questions, you say, oh look, guys did much better on this question than women did.
Oh look, women did much better on this question than guys did.
You know what we're going to do with this question?
We're going to throw it off the exam.
You know, we're making up the exam as we go along and so they tooled the exam to get rid of that 10 point difference and subsequent tests are standardized against the older tests.
That's why men and women have the same IQ.
Yes?
How do you factor motivation into this?
Aren't little girls more focused in general--
Oh, there's a whole lot of-- you know, I don't know when it is that guys finally get focused.
And I don't have much direct experience with this having nearly had 3 unfocused guys myself in my family.
I always wanted one of those focused girls who just-- anyway.
There's all sorts that stuff like that.
That's a whole other course.
But what they did was they made the difference go away by manipulating the test.
Now these tests were hardly the first efforts to study intelligence.
Actually, in an interesting circular movement, these days there's great interest again in looking at the brain and saying, is there something about smart brains that's different from less smart brains?
We can do that with the renewed interest comes from the fact that you can now go and look at brains in walking, talking people.
The previous boom in looking at brains was in the 19th century.
The reason Binet developed his test was that looking at brains was only good at autopsy.
You're not going to figure out if your kid needs help in school by cutting his head open and looking at his brains any good.
In the 19th century it's just not a really practical solution, but people like Paul Broca of Broca's area-- famous French neurologist, spent a lot of time looking at the brains of their deceased colleagues and others to see what made people talented.
Broca's doctrine, translated from the French, but quoting is that all other things being equal, there's a remarkable relationship between the development of intelligence and the volume of the brain.
He was interested in the size of the brain.
To a first approximation he's got to be right.
One of the reasons there are no chickens enrolled as MIT undergraduates is because little chicken brains just don't cut it when it comes to surviving at MIT.
So having quantity of brain really does make some difference.
So what he did was to look at the sizes of brains.
And his conclusion was that in general, the brain is larger in mature adults than in the elderly, in man than in women, in eminent men than in men of mediocre talent, and in superior races than in inferior races.
This reflects very much the biases of 19th century European, but in fact that's sort of the point.
It's not to say ooh, Broca was an evil, nasty man, but you should be a little suspicious when your science places you at the pinnacle of creation.
That should be a little warning bell that goes, oh, the inferior races by the way were for Broca, were the Chinese, the Hindus, that's subcontinental India, and blacks.
I can't remember if Jews made it onto his list or not.
But in any case, it was sort of the 19th century collection of not white males.
But how did Broca get to this conclusion?
Well, he weighed a lot of brains and what he discovered was in fact, it's true.
That female brains are on average a little lighter than male brain, on average.
All right, so?
Chicken brains are lighter than your brain, chickens don't go to MIT; female brains are lighter than male brains, but he also found that French brains were lighter than German brains.
Broca was French.
So he wrote, "Germans ingest a quantity of solid food and drink far greater than that which satisfies us.
This, joined with his consumption of beer makes the German so much more fleshy than the Frenchman.
So much more so that the relation of brain size to total mass far from being superior to ours seems to me, on the contrary, to be inferior."
Well, what he's doing is not stupid.
He's correcting for body size and there's a very interesting graph that you may have seen someplace or other.
If you plot body size weight against brain weight-- I can't remember if it works just for mammals or for sort of everybody, but anyway-- let's claim it's mammals, you get a very tight correlation.
All the way up from mouse to whale.
Except that humans are up here somewhere.
They're off the curve.
So the argument is people always-- a lot of people think-- whales, they must be these philosophers of the deep, they got a huge brain.
But nobody quite understands what they're doing with that great big brain, but they do lie on this function.
Humans have a very big brain relative to their body size.
So this is the point at which some-- typically, woman-- in the class is supposed to raise their hand and say?
Females are smaller
Yeah.
Females are smaller than males, so if we correct for body weight-- well, Broca wasn't stupid, he knew that.
And he wrote, "we might ask, if the small size of the female brain depends exclusively on the small size of her body.
We must not forget, says Broca, that women are on average, a little less intelligent than men.
We are therefore permitted to suppose that the relatively small size of the female brain depends in part on her physical inferiority and in part, on her intellectual inferiority."
Now those 2 quotes from Broca are taken from completely different publications.
Even Broca might have noticed if you put them right next to each other that there's a certain conflict there.
Again, the point isn't to say, Broca was a mean, nasty, stupid man.
There's no evidence for any of that, but there's clear evidence that what he thought he knew ahead of time was influencing how he was reading his data.
The brain weighing endeavor sort of fell apart because it didn't work all that well.
You know, people like the scientist who gives his name to that unit of magnetic whatever it is and Gaussian curves and stuff like that turned out to have a small brain.
Had a lot of wrinkles in it though.
Lenin, the Communists were always big on carving up the brains of their ex-leaders.
Lenin was reputed to have a cortex-- we all have a cortex, its got 6 cell layers in it-- Lenin's allegedly had 7.
Again, the point of these stories is not to say that we're smart and they were all stupid and silly people, the point is to say that people tend to impose their ideas on their data as well as imposing their data on their ideas and you've got to keep an eye open for that.
In the remaining time let's ask a bit about this question about, suppose we really do think that more IQ is better.
Is there any evidence that we can get more?
You know, is there a way to get more IQ?
The answer appears to be yes, though we're not-- some of it's kind of mysterious.
The mysterious bit is on the handout on the back page I see.
Well, on page 4 at least as the Flynn effect.
Flynn is a political scientist, James Flynn is a political scientist sitting down at the University of Otago at the bottom of New Zealand, which nobody had ever heard of until the "Lord of the Rings" movies, but you don't get further away unless you go to Antarctica.
But what he did was he took a look just at raw IQ statistics, initially in the Pacific Rim countries in the period shortly before and since World War 2-- couple of generations worth.
And what he found-- well, it's summed up in the title of his paper-- massive IQ gains in these countries.
So for example, back a decade or two ago when the Japanese economy was booming, people liked to point out-- people given to these group IQ arguments liked to point out that the average Japanese IQ was about 10 points higher than the average American IQ and the reasons they're eating our lunch is because they've been breeding with each other for years, you know?
And they're just smarter people than we are and we're all doomed to serve the Japanese forever.
Well, that argument has kind of gone away since they went into a 10-year recession or something.
But the more interesting point, from our point of view, is that apparently they got smart really fast because before World War 2 the average Japanese IQ was about 10 points lower than the U.S. IQ.
There is no genetic story that makes that work.
And this happens in Pacific Rim country after Pacific Rim country.
That after World War 2 IQs just go way up and then subsequently, it turns out what's happening all over the world in fact, it had been mapped in the U.S. because IQ tests keep getting renormed.
The average IQ in the U.S. keeps drifting higher and then because average IQ is defined as 100 you renorm the test so that the average is 100 again.
Flynn wrote a wonderful paper, which I didn't cite on here, but if anybody wants send me an e-mail, where he was having fun with these statistics proving that if you used these statistics literally you would conclude that our founding fathers were all congenital idiots and were probably too dumb to walk across the street if you just extrapolate back.
Something very odd is going on.
It's not clear what it is.
One of the thoughts had been that it was simply nutrition.
Pacific Rim nutrition got a lot better after World War 2 and maybe better food makes better brains.
That seems to be true, but there's counter evidence.
For example, there's no dip in the Dutch average IQ for the cohort that was in the vulnerable part of early childhood during World War 2 when the Germans systematically starved the Netherlands.
There's no dip in the IQ, so it's not clear that the food story works.
There are other stories that say that what's going on is it's something about a change in the culture.
That the world culture has become this much more information intensive culture that supports and nourishes the sorts of things that are measured by IQ.
That the sorts of talents that you needed when you were a subsistence farmer, you needed some smarts to make yourself survive, but they weren't the sort of thing that were getting picked up on IQ tests and that now your life builds IQ points and something-- it's not clear what's going on.
What is clear is it's not genetic.
It's just too fast to be a genetic change.
Now that's at the level of groups, can you change individual IQs?
And the evidence there is yeah, you can do that, too.
And the classic experiments come from the 60s.
Oh, what is he today?
He's a rat.
All right, so we're going to take rats.
Sort of a homogeneous population of rats and we're going to randomized them into 3 populations.
Population 1 goes into-- well, let's stick with socioeconomic status because that's what they were trying to model.
This is the low SES group, which meant in rat case that they lived-- looking like ducks or something-- they lived all alone in an isolated cage with nothing to play with.
The medium group lived in what was sort of the standard lab cage of a couple of cockroaches.
A couple of rats, not much action.
And then there was the high SES rat group or the enriched group, who lived in a sort of a big group rat daycare center.
You know, with lots of cool toys to play with and the Habitrail thingy and stuff.
You know, a lot of good, cool stuff there.
Let them grow up in these environments, test them on little rat IQ tests, cognitive tests for rats.
These guys do not do as well as these guys.
Look at them at the end of the experiment.
I mean, the real end of the experiment when the rat is now dead and you discover these guys have brains that on all sorts of measures look better than these brains.
Thicker cortex, more synapses, bigger brain.
Clearly, the enriched environment was having some sort of affect.
These data, back in the 60s, were part of what motivated Head Start.
The notion that kids in low socioeconomic environments had lower IQs than kids in high socioeconomic environments, they could be brought along to the same higher level by putting them in school earlier.
Putting them in a preschool enrichment situation.
And it worked.
That if you went-- [UNINTELLIGIBLE] I need a spot-- if you took low SES kids who plateaued out at this level and you took them and put them in Head Start they went up to this higher level that was the same level as the high SES kids.
Well, what are Herrnstein and Murray talking about then when they say you can't change it?
Well, Herrnstein and Murray said, yeah, that was very nice.
Look what happened when these kids hit middle school and high school.
And the answer is [UNINTELLIGIBLE], they went right back down while the high SES kids stayed up.
Herrnstein and Murray concluded that what this is, it's like a rubber band.
Yeah, you can stretch it a bit, but when you let go it just snaps back to where it was.
The alternative view is yeah, it's exactly like a rubber band, but why did you let go?
When you let go you drop these kids back into a lousy inner city school rather than this enriched state and they fell back to where they were.
The effects were transient.
You want to-- sort of a silly example-- think about diabetes.
So Diabetes has a similar sort of graph except it's a little more stark, no insulin you're dead by your 20s.
With insulin you're not dead.
OK, so here we're in the not dead state.
Let's say, OK, well you're done with that insulin stuff now, we stopped giving you insulin.
Oh, look, they're dead now.
There wasn't any point to giving insulin.
Well, that's a stupid conclusion.
In Diabetes obviously, isn't that the conclusion is you better keep giving insulin.
And you could argue that the same thing is going on here.
That we know that if you want more IQ points, putting people in better environments works, but you gotta keep them there.
I will tell you one more factoid and leave it at that.
In adoption studies, if you adopt kids into typically, when adoption moves you from a lower to a higher socioeconomic status because that's just the typical reason why kids would be put up for adoption, it doesn't go the other direction, typically.
So most kids are moving from low to high when they're adopted.
Who does their IQ correlate with?
The answer is it correlates with their biological relatives because there is a genetic component to IQ.
What is their average IQ, however?
Their average IQ is the IQ of the environment that they are now in.
So their IQ is, on average, indistinguishable from the biological kids in the same family.
How can that be?
That doesn't sound like it ought to work.
But let me just draw you a quick picture and then you can go off and think about it.
So here are these pairs of siblings.
And their IQs-- let's take pairs of siblings-- their IQs are correlated with each other.
Now we're going to take 1 of each of those pairs and that kid's going to get adopted out.
And that kids going to move, as a result, to a higher socioeconomic status.
That doesn't change the kid's genetics any.
And so, the kid still has a IQ that's related to his brother or sister, but it shifts all of-- so this is the kid who's in a low socioeconomic state and stays there.
This is a kid going for low to high.
What happens is that the low to high move pushes everybody up.
So the cloud of spots just moves up.
The correlation is the same.
The average IQ here is higher because going from a low to a high socioeconomic status gives you IQ points.
If you really thought that what you wanted to do as a society, if you really believed more IQ points mean more good stuff, there's a clear enough way to do it.
It becomes a social policy issue about how you end up doing this, but it seems perfectly clear that it is possible to do that.
I'm covered in chalk.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about out license at MIT OpenCourseWare in general is available at ocw.mit.edu
The topic for lecture for the next couple of days is going to be romantic love or mating behavior or the interaction thereof or something like that.
And I just learned the Shakespeare Ensemble is doing Taming of the Shrew.
When?
The nights of Friday, Saturday and [UNINTELLIGIBLE PHRASE]
Tonight, tomorrow night, and Saturday at?
8 o'clock
8 o'clock.
Where?
Little Kresge Theater
Little Kresge Theater.
We should probably assign it like as homework or something like that.
But you can go after today's lecture.
You can go and try to apply evolutionary psychological theory to Taming of the Shrew if you like.
And you can tell Bianca how you liked it.
All right, enough Shakespeare.
Shakespeare shows up again maybe later today.
Shhh.
Let me tell you about Chris and Terry here who are having an argument.
I realize this is sort of a working memory exercise, too.
So you have to pay attention to the particular details here.
So Chris and Terry are having an argument.
They've been going together for about a year.
They don't live together, they do sleep together.
Chris accuses Terry-- so Chris accuses Terry-- of seeing someone else on the side.
Terry doesn't want to talk about it.
Chris says that Terry never wants to talk about anything.
Terry shouts that well, maybe there is somebody else on the side and says, well it's not like there's been much activity in this relationship lately anyway.
And Chris snorts that there's more to a relationship than just sex.
All right, you got that?
And you will have noticed, even Kristen-- who stayed up all night cheering the Red Sox-- will have noticed that Chris and Terry are deliberately ambiguous as to the sex of who is who.
They could of course be the same sex.
And that reminds me to say ahead a time, I'm going to talk almost exclusively for the next two days, next two lectures about heterosexual relationships.
Lots of fascinating stuff to say about same sex relationships, like so many topics in introductory psych, it's just one I'm not going to get to.
So if we assume that Chris and Terry are male and female, one of them is male and one of them is female.
And I don't remember which side I was gestering too, so I don't remember-- was this Chris or Terry?
Terry
That was Terry.
This was Chris.
OK, so how many people vote that Terry is the male person?
How many vote that Chris is the male person?
How many vote that I got lost in the details?
I can't remember who's who anymore?
All right, well quick review.
Chris accuses Terry of seeing someone on the side.
Terry doesn't want to talk about it.
Chris says Terry never wants to talk.
Terry shouts that there may be somebody on the side and there hasn't been much activity and Chris says there's more to a relationship than sex.
How many vote Terry is male?
How many vote Chris is male?
Gee, there seems to be a fairly strong asymmetry there, which is capturing something of the cliche that man are just interested in sex and women are interested in commitment or something like that.
This is sort of a cartoon version recognizing that-- actually, we sort of backed off what I was saying last time about talking about individual differences and now we're talking about on average.
Again, recognizing that there's vast variations in relationships between men and women.
Is there any truth to this sort of asymmetry?
If there is, how in the world would you explain it?
That's the job for today's lecture.
Well, I mean, there's lots of anecdotal evidence.
How about a little bit of data evidence from one of my favorite experiments in the annals of experiments that you can't see how they ever got it passed-- oh, does it say IRB on the handout?
IRB stands for Institutional Review Board.
You don't have to worry about that bit of jargon, but those are the people who look at experimental protocols and decide whether it's permissible to do this.
So if you volunteered to be a subject in our lab-- please do, we need subjects, talk to Kristen-- anyway, if you do we'll have you sign a 7 page consent form because they're really worried about the damage we can do to you looking at little red and green bars on a computer screen.
So you have to imagine how this experiment ever got passed the IRB, though it does seem like a good one for reality TV perhaps.
Anyway, so here you are, you're wandering around on campus, and an attractive member of the opposite sex approaches you and asks you one of these 3 questions.
I should note that I was explaining this to my wife this morning and she got confused and thought this was a multiple choice test.
It's not a multiple choice test, which would be even weirder.
So the percentages don't need to add up to 100.
That's important.
But this person comes up to you and asks you either-- says something to the effect of, I've seen you around campus and you're really an attractive person.
Would you go on a date with me?
That's question 1.
Would you come back to my apartment with me or would you have sex with me?
What I can't find out is what happens after the subject gives an answer, right?
This is a quite famous experiment, but the original experiment is published in a very obscure journal and I don't actually have a copy of the article.
So we've got these 3 questions.
There's the date question, the apartment question, and the sex question.
And so we'll plot percent yes here.
OK, if you are a woman person, what do you think, what percentage of women said, yes, I'll go on a date with you would you think?
50%
What?
50%
50%.
That sounded terribly authoritative.
Is that because it's written down in the book or something?
I read it
You read it.
It's not in [?
Blightman ?]
is it?
No, it's on the website
It's on the website.
Oh, the articles on the website?
Yeah
Wait a second.
I thought I didn't find the original article
[UNINTELLIGIBLE]
What?
An article [UNINTELLIGIBLE]
Oh, OK. Well 50%.
Thank you.
This-- whoops 50% doesn't go all the way up.
OK, 50%.
This kind of takes the guess work out of it a bit
I won't answer anymore
I thought we could find out if you memorized all the percentages.
So all right, if you're not a good person who went and read everything on the website-- all right, how many women do you think were willing to go back to the guy's apartment?
[INTERPOSING VOICES]
2, 9 sounds great.
The average is about 6.
So 50, 6.
Number or percentage of women who said sure, I will have sex with you?
1
Zero
Zero would be a good, kshht.
Now the IRB or sort of the sequel issue here that you'd love to know the answer to is, how many times the guy got like punched or reported to the campus police or something.
But now, look.
Do I have any color chalk?
No, we'll just have to go for hatched bars today.
So that's the female data.
Let's start at the other end here.
What percentage of guys approached by a woman who said, I think you're attractive.
Would you like to have sex, said yes?
75
She said she wasn't going to be answering anymore.
Yeah, OK.
It's 75%.
It's also interesting to see what happens with the rest of the questions.
If you ask, would you go back to the apartment with me that drops to about, I think, 67%.
And if you ask, would you go on a date with me it drops to 50.
So, now look the world is full of interesting asymmetries in data.
Even sex differences in say mathematical ability or something like that.
Sex differences in most stuff look like this.
There's a data point with an error bar and there's another data point with an error bar and you say, oh yeah, look.
There's a significant sex difference if we run 8.5 million subjects.
You don't get data like this most of the time.
This is like a huge asymmetry.
So one interesting question is, why is this huge?
And another interesting question is this apparently counter intuitive slope to the male data where you're more likely if you're a guy to say, oh sure.
I'll have sex with you right now, but I don't actually want to go on a date with you.
Which is on the face of it, perhaps a little odd.
But we'll come back to try to explain that.
There are multiple ways.
So this is not some weird isolated data point where people just did a very odd-- it's an odd experiment.
Here, this is the one that you really wonder about what the next sentence was, you know, hi.
You're very attractive, would you have sex with me?
Oh yes, sure.
Just kidding.
It's a very odd experiment, but a very interesting one.
Yes, Mara knows the answer to the rest of--
We have a question.
Did they control for the attractiveness of the questioner?
I mean, what if--
Well, the description-- my expert witness can probably tell me better because she's read this more recently apparently, the description I think is simply that the experimenter -- this experimenter's confederate is described as being attractive
Well, I'm just saying-- p
But look.
Do the intuition.
You can only do the real intuition for the sex that you happen to belong to.
But ask yourself, really attractive guy comes up and says, would you like to have sex with me right now, do you think the point moves a lot off zero?
And short of-- I don't know-- nonhuman woman, it's not clear how much that point moves either.
I don't know.
And it's not one of these things.
Also, by the way, I do not recommend you know when you go off into-- course 15 seems to be the place where they like to do survey experiments and stuff like that.
I don't recommend this one as one to try because I really do think that it's hard to imagine how to do this without getting into serious trouble.
Fascinating as the experiment is.
Other efforts to get it the same point have been done in sort of more indirect fashion, so you can be your subject in this one, which was one where you were asked to rate a variety of events as being positive or negative-- effectively positive or negative-- if this happened.
And the most salient one seems to me to be one where the description was you're on a crowded subway car, an attractive member of the sex to which you are attracted begins to surreptitiuosly touch you on the subway car.
Is this a positive or a negative event?
The answer in the data is women overwhelmingly say eww, that's kind of creepy and negative.
This is not good.
And guys fairly overwhelmingly say, yeah.
What's the problem here?
Again, you get a hugely asymmetrical result and what we want to do is try to account for that asymmetry.
Now assuming that you have not been entirely comatose for the first half of the course you will recognize that the broad classes of explanation are going to be nature nurture kinds of things and of course, it's always a combination.
In particular, today what I'm going to talk about is a biological account and that's the account coming out of evolutionary psychology.
Evolutionary psychology, the application of evolutionary theory to the topics in psychology is very prevalent, very important these days.
It gets applied to a huge range of topics, some of which it probably doesn't have much to say about.
But if there's anything that it's got a lot to say about it's probably human mating behavior.
And so that's why I'll talk about evolutionary psychology in this particular context.
This is not to say it's the only way of accounting for mating behavior, but it's a particularly-- I think, it's very hard to argue that these sort of biological forces don't have some shaping influence on the way that we select mates and I want to try cashing out that argument today.
If you want to read more about it there are several-- I put reference to several books on the handout.
All of which I put on there because they talk about the topic, but they're all good reads.
My current favorite actually is the [?
Bowmeister ?]
and [?
Tyce ?]
one.
It's less of one of the ones that's sort of been on the bestseller list, but it's a beautifully written book on this topic if you're interested.
All right, so let's run down the basic evolutionary psych argument and then attempts to see how it might account for the sort of data that we see in this area.
If you are operating within this sort of evolutionary framework your goal as an organism is to do what?
Pass on the genes
Pass on your genes.
You've got to propagate your genes in some fashion.
You want to get that genetic material into future generations.
And how you going to do that?
Weren't you paying attention in high school?
There are several ways, but sexual reproduction is considered perhaps the most direct of these.
It's not the only one, by the way.
It's important to point out that if you think that again, by the way, all of this stuff is presumably operating at a very unconscious implicit kind of level.
Nobody assums that people are wandering around the dating scene saying, got to pass on my genes.
How am I going to do that?
But the forces, the forces that are shaping that behavior are within this viewpoint shaped by this drive to pass on genetic material into the next generation.
You can also propagate your genes by taking care of your relations.
So if you've got a brother or sister with kids, those kids are genetically related to you and if you nurture them, if you pay for their college education or something, they're not as genetically related to you as your own children, but they're genetically related to you.
And so you can do the propagation thing indirectly rather than through just regular old-fashioned sexual reproduction.
This, by the way, is one of the arguments offered by evolutionary psychologists for the persistence of same sex relationships.
Same sex relationships are a problem from this evolutionary point of view because they don't lead to offspring, right?
Whatever else is going on, they don't lead to offspring.
I don't know if there's any data for it, perhaps because I haven't looked, but one of the arguments that has been put forward from time to time are things like oh, gay men make great uncles.
They're not going to have kids of their own, but they take care of their sister's kids and that's why it's not selected out-- it's not selected against by evolution.
In any case, sticking with this sort of mainline tail, you want to propagate your genes.
Sexual reproduction is a good way to do it.
Now the thing about sex is you get pregnant.
Actually only 1/2 of you get pregnant.
It was sort of an ambiguous sentence.
This is 1/2 of you, plural, get pregnant.
Not 1/2 of you as an individual get pregnant.
You never know what people learn in high school.
In any case, it is that fact, that's the key fact this story that produces the asymmetry.
Women get pregnant, men don't get pregnant.
Why is that important?
Well, that's important because being pregnant ties you up for 9 months, being pregnant for 9 months at a time greatly is a profound rate-limiting step on how many kids you can have.
And then if you're a human and not a spider or something like that.
Once the kid is born or hatched or whatever you still got a problem here.
You want this little bundle of genes to do something for you in evolutionary terms, you got to keep it alive.
So it's not just the 9 months.
There's a long time, long term commitment to this little evolutionary project that you've gotten into.
The same constraints do not necessarily apply to men.
Certainly the rate-limiting step does not apply because in principle guys can go off and have a gazillion babies-- that's a technical term-- as long as they can find the requisite gazillion women.
The rate-limiting step for man is just how many women can you successfully impregnate?
We're not talking in what is right in social, civilized world or something like that, right?
But we're just talking about the biology of the situation here.
The men don't get pregnant.
If you get woman 1 pregnant today you can get woman 2 pregnant tomorrow in principle.
That's not true for the woman.
If you are a woman and you get pregnant today you don't get to get pregnant again tomorrow.
That doesn't work.
So fundamental difference, the evolutionary psych argument is that this should produce fundamental differences in behavior.
In particular, if your goal is to get as many successful bundles of genes into the next generation as possible guys should be inclined to have sex early and often with as many partners as possible because that's going to produce lots and lots of babies.
Women should be looking for something more like a commitment because it's a lot easier to take care of this little bundle of joy if somebody else's helping out.
Either helping take care of the kid or at the very least, providing resources for this.
It's an expensive business raising a kid, not just an expensive business if your a nice middle class kid and you're expecting to go to MIT or something like that.
It's an expensive business period.
You gotta feed the kid, you gotta keep it warm, and so on.
Resources are required.
And it is in your interest as the woman to have guys around, at least a guy around who's going to provide resources.
Now, you can make up other stories.
You could make up a story that says it would be in the woman's interest to have multiple mates who are kept in some doubts about who the father is so that they all provide resource.
That sounds funny, but there's no reason why that story and there are species where that seems to-- you can almost always find an animal where it works out.
If humans were a polyandrous species, a species where females had multiple mates at the same time that would require explanation and you could try explaining it the same terms.
As it turns out across cultures the pattern is either one male, one female or one male, several females.
That there are much rarer instances of one female, many males in human populations.
So the mainline story is that the constraints of biology are going to incline the population to a population of males who are interested in frequent sex and whenever.
And women who are going to be looking for a mate who's willing to make some sort of commitment.
Now is there any evidence for evolutionary forces, biological constraints on mate selection in humans?
Quite apart from these questions about who wants sex and when.
Is there any evidence for this at all?
The answer is yes.
If I give some examples it's sort of surprising that this was ever controversial.
Well, let's give an example.
For example, if you're looking for evidence that something has these sort of biological, evolutionary roots what you're looking for is something that is ubiquitous in human societies.
You don't want something where it's deemed attractive in one population or desirable in one population and not in another population.
That can be very culturally determined.
So I don't know of much in the way of great evolutionary psych arguments in favor of which bits of your body should be pierced.
That seems to be the sort of thing that some groups think this is attractive and some groups think that is attractive and yeah-- but across human cultures there is a consistent preference for mates who are young and healthy as opposed to mates who are old and sick.
There is no place where the ideal of attractiveness is an old person with a disease.
It just isn't the case.
Now it almost sounds funny, but-- I got a house full of books.
I like to collect books.
It turns out that the books I like best are pretty old and diseased and that's not weird.
That doesn't strike people as odd.
You do need to explain, it doesn't take much work to explain why there's a preference for young, healthy mates over old diseased mates.
If you think that this has something to do with a drive towards reproduction you're much more likely to reproduce successfully with a young, healthy make.
The reason that these topics have been controversial has less to do with the science of the situation or it has had less to do with the science of the situation than with the politics of the situation.
It is possible, as in the IQ case to go from uncontroversial or fairly straightforward bits of science to undesirable or at least controversial conclusions in a more political realm.
So the sorts of things if you agree, if you are forced by the data to say look, there's a whopping asymmetry between males and females in this area.
It is perhaps-- the concern is-- that it's a slippery slope from here to saying well, it's OK to pay women less than men.
The actual causal chain between this particular factoid in paying women less than men is not immediately obvious to me as I'm saying it right here.
But the reason that this is politically controversial is because anything that acknowledges a difference, some sort of biological difference between say men and women can in principle be used as a defense for social inequalities that you may not find desirable.
But that's really a separate question.
The scientific question or the psychological question is are there differences?
There certainly seem to be data differences and where did they come from?
There are other sort of universal preferences that seem to reflect this desire for a mate who is likely to be a good partner in reproduction.
What is it that makes a face attractive?
Well, one of the things that is attractive is facial symmetry.
That all else being equal it's considered to be better-- you're consider to be more attractive-- a face is rated as more attractive if it's symmetrical than if it's asymmetrical.
Why is that?
Most of the things that are likely to produce noticeable asymmetries in the face are things that suggest either a genetic deformity of some sort or an accident.
Neither of them are going to be great advertisements for reproductive success.
Imagine somebody with half a face, you know intuitively that either you're watching a sci-fi movie or there's something fundamentally wrong here.
Oh, what is an attractive face in general is an interesting-- I didn't put all those references on the handout.
There's an interesting literature on what makes an attractive face that is worth a little side note.
You could have two theories.
One theory would be attractive is one end of a scale that runs from ugly to gorgeous.
That all those hollywood actors and actresses are pegged out here on the gorgeous end of some scale.
The alternative theory is that it is average faces-- that an average face is what's normal, is what's deemed attractive.
The original evidence for this comes from a photographic technique where what you did was you took a whole bunch of photographs and superimposed them to make a composite face and you found that those were rated as more attractive than individual faces as a whole.
And this was being done by clever techniques back-- oh, I think in early part of the 20th century.
And it's gotten a lot easier now that we've got computers to do these sort of things.
The difficulty, one of the things that made this research a little tricky is that one thing that averaging faces does is it does wonders for your complexion.
Everybody's nose is in more or less the same place, but everybody's zit is not.
And so if you average 100 faces you get people with gorgeous complexions and there's some part of that effect that just comes from the fact that they look like they had really healthy skin.
It's a debate that jumps back and forth, if you turn out to be deeply interested in what makes a face attractive send me a note, I'll send you the reference since I see I didn't put them on the handout.
We can ask for a couple of other things that suggests these universals across human populations.
What's more desirable in a mate?
Richer or poorer?
Richer
Richer, that's easy enough.
Is that preference the same for men and women?
Who's more interested in finding-- I'll tell you the answer-- it's not the same, how many vote that it's men who are more interested in finding a wealthy mate?
How many vote that it's women?
You got the right intuition and the evolutionary psych story there is the resource story.
That what you're looking for is evidence of resource.
Is this guy going to help me feed all those hundreds of babies that he thinks we're going to have?
And this is also presumed to be-- why is it that driving, cruising around town in the fancy car, you know what's that about?
That's a peacock tail effect.
I think I already described the peacock tail effect earlier in the course.
The idea is I'm showing off my resources, I've got so much money that I can buy this stupid car.
My hubcaps cost more than your house, which is why you want to mate with me.
This is presumed to be the way that they-- OK, how about age of spouse?
Or age of mate?
On average, if you are a male, you want a-- and this is very importantly on average, because it's not to say, well, let's get the data point first.
If you're a male, on average how many people vote that you were looking for a mate who is-- well, this is a 3 alternative forced choice, you got to vote for one of them: little older, exactly the same age, little younger.
How many vote for little older?
How many vote for just the same age?
How many vote for a little younger?
OK, got that intuition right.
So males on average, in culture after culture seek mates who are a little younger.
The amount varies culturally.
In some cultures it's significantly younger.
In some cultures it only a couple of years, but it is across cultures younger.
OK, so the point I was going to make before is you're sitting there saying, my girlfriend is like 2 years older than me.
This does not mean that you are some sort of genetic misfit.
It means that we're back to talking about the mean value and all of these things have a distribution around them.
It would be considered odd if your choice in mates was, I only date women who are 60 years older than me.
That would be considered unusual, but that should be way, way out on one tail of it.
But typically, the mean of that distribution lies a little younger.
OK, why is that the case?
If you're following this evolutionary psych story, why do males seek females who are slightly younger?
They can have more babies
More babies, right.
The males may not be limited in principle by the same rate-limiting step, but if you're in a culture that's going to limit you to 1 woman you better get 1 woman who's going to produced the maximum number of babies.
So you better start young.
So, if you are a woman, same multiple choice.
If you are a women, the average-- across cultures women prefer-- how many vote that women prefer males who are slightly older?
Exactly the same age?
Slightly younger?
You got the right intuition.
Again, it's very convenient that this is the way it works out.
It's really bad news if everybody wants younger mates.
That doesn't work well.
But actually, we'll come back to that in the next lecture.
I'll just put that out there as a problem to solve.
Everybody does want to max out on the attractive, rich gorgeous thing, but most of us aren't going to get that.
It's one male to one female roughly speaking.
We can't all have that person, it just isn't going to work well.
So we'll have to solve that next week.
OK, why do females want males who are slightly older?
Maturity
Maturity-- good luck.
The evolutionary psych argument is the resource argument.
I'm sure maturity doesn't hurt either, but it's the resource argument.
A guy has had a chance to accumulate more resource if he's a little older.
And so you've got these endless-- sometimes, literally endless-- big, thick 19th century romance novels that you read in English lit at some point where he loves her but they can't get together for 800 pages because he's got to go out and collect resource in some fashion until he can come back and show daddy that he can support the daughter.
The alternative version of the 19th century romance is the interview where daddy asks, how much money your daddy left you.
Oh, yes, I have a thousand pounds.
They're all English novels.
A thousand pounds a year in the funds or something like that.
That's resource, that's good.
Oh, there's one exception.
There is a population of males who seeks slightly older women.
Who is that population?
[?___
__ _Teenagers.___?]
Exactly.
There's this cruel sick chortle that we just got from-- all the answers given to that were by women.
All of whom were snickering about it, which is harsh.
But it's teenage males, particularly young teenage males who develop crushes on slightly older or maybe quite a bit older women.
Why?
What's that about?
[UNINTELLIGIBLE]
They're fertile
They're fertile, exactly.
Now, I mean, the kid doesn't know this.
Oh, I think I'd better fall in love with somebody who's fertile.
That's presumably not what's operating at any sort of conscious level here, but if evolutionary forces drive you to look for a mate who's going to be fertile and you're an early adolescent male, 9-year-old girls are not the population of choice.
And you may notice that even the example gives you a sort of a eww kind of feeling.
We have a sort of a gut, built in reaction that there are some relationships that are OK and some relationships that are odd.
And so relationships and in particular, relationships-- guys can go and pursue fertile females.
That's OK. That can be shown in the airline movie.
Again, we don't have the airline movie, the headline comes up the search for the fertile woman, but ask yourself who it is-- who romantic comedies are about?
You wouldn't put a movie on the airlines for family consumption about some guys unrequited passion for a 9-year-old girl.
It would seem icky.
That there's something wrong there.
Why?
Presumably this has something to do with the same sort of forces.
OK, so let's move on a bit to this, elaborate on this issue a bit, who should be faithful in relations?
Who should cheat?
So let's start with guys.
Now recognize here, by the way, that we are talking again, within the context of this evolutionary psych viewpoint.
We are not saying who should cheat in your relationship right now with the person sitting next to you or something like that.
Or who does society say it's OK to cheat with.
But in terms of this theory about evolutionary forces working on mate selection, who should have an evolutionary drive to cheat?
Should guys?
Yeah
Yeah, right.
That's a pretty straightforward argument.
That if you can get away with it, if you can have a few no cost babies somewhere-- great.
You've done your evolutionary work.
Look if we switched to a different context of cultural, civilized world kind of you may be a jerk, but you're an evolutionary successful jerk.
OK, how about women?
Should women be unfaithful and play around?
Yes
OK, yes argument.
What's the yes argument?
Again, from an evolutionary point of view
Like a stronger mate
Yeah, you might get a stronger mate.
You're hedging your genetic bets.
If you are going to link up with this guy, maybe he turns out to be a genetic dud.
If you just have your 18 babies with him you know, how good is that?
But if you spread it around a bit, your genes are the same in all cases, who cares?
You're always get your half of the package.
And so you might as well mix it up a bit and so that's the yes argument.
Any other yes arguments?
Yep?
You could have sex with a guy with better genes, but remain more faithful to the guy with more resource
Yes, OK.
This is a particular version of the more general argument.
There's a resource argument here.
He's giving a particular version, which is go have sex with the guy with the good genes and snooker the guy with the good resources and convince him that it's his kid.
So he devotes the resources to raise his genes.
More generally, you might just play around, just shop around because you get more resources in general.
Guys will give you stuff in some fashion or other.
So there might be sort of a quasi economic-- go out and get the goodies kind of bargain.
OK, those are good yes arguments.
How about no arguments?
Against the notion of-- yeah?
You could lose the resources
You could lose the resources.
Why would you lose the resources?
What's the guy's problem?
Not his kid
It's not his kid.
And this is a crucial issue.
Again, another part of this fundamental asymmetry.
Because it's the woman who gets pres-- president right.
The woman who gets pregnant, she always knows who the mother is.
Actually, he knows who the mother is, too.
There's no doubt about who the mother is.
The maternal genes are not an issue here, paternal genes are an issue.
And doubts about paternity have a variety of consequences.
Male sexual jealousy is a real and potentially dangerous phenomenon.
In fact, I don't think there are any states where it's still the case-- there were states where it was the case that killing an unfaithful wife was not murder.
I don't think it was ever the other way around.
You couldn't kill your unfaithful husband, but killing the unfaithful wife wasn't necessarily a laudable act, but it wasn't murder in the way that killing somebody else was.
I don't believe that-- anybody happen to know that they come from a state where you can still do that with impunity?
I think Louisiana was the last.
Oh, and there are interesting wrinkles on this.
When a baby is born the first question anybody asks is, is it a boy or girl?
Which is an interesting factoid in its own right, but sort of the second topic of conversation is, who does the kid look like?
And what's the answer?
The answer is a bias towards saying that the kid looks like the father.
Now why should that be the case?
And it turns out that the bias is strong-- it depends who's giving the answer.
The answer by the wife's family-- by the mother's family-- is more skewed towards saying it looks like the father.
And there are at least two reasons why this is the case.
One of them seems to be an effort-- again, quite unconscious to prove to the dad you don't have to say it looks like the mother, we know who the mother is.
Saying it looks like the father is reassuring to the father who might otherwise deny resource to this kid or go out and kill the mother-- something inconvenient like that.
So it's important to believe that this is your bundle of genes.
The guy is in theoretical doubt, the woman is not in doubt.
The other reason why it may be true is there is some evidence that babies actually look more like their fathers when they're young and that this wears off with age.
If this is true it would be an interesting argument for this evolutionary pressure because babies who looked clearly like dad probably had a little edge because dad looked at this little bundle of joy and said, it's mine.
And if the kid looked like the milkman or something, the kid may not have-- actually, can you even use that as an example anymore?
Are there still milkmen out there?
It used to be.
It's a sort of an anachronism, like talking about dialing the phone.
This is a very mid 50s kind of anachronistic sort of cliche about the woman who would be at home, the milkman, who would be the man who delivered the milk since you didn't go to the supermarket to buy milk-- it came to your house and then various other things happened and anyway.
So much for cultural enlightenment here.
So in any case, there's a pressure for women to be faithful because of basically, in some fashion because of the danger of alienating the male in some way or other.
There's a problem here, sort of a problem.
There's a potential problem with this.
If you think about it we've just come up with a theory that explains why women should be faithful and shouldn't be faithful.
And in fact, we could elaborate on this and say that the evolutionary pressures should cause-- you could argue that the evolutionary pressures could push women to have less of a sex drive therefore, making them more likely to remain faithful or you could have it explain that they have just the same sex drive, but they're supposed to deny the existence.
You can have a lot of different explanations.
This is a problem with the theory, a theory that explains all possible outcomes isn't a great theory.
On the other hand, in human mating behavior all the possible outcomes are out there.
So you don't want to have a theory that just explains one thing because that wouldn't be an adequate theory here either.
I think the fairer thing to say about evolutionary theory is that it hasn't reached a degree of sophistication where it can explain in detail when a woman should or should not be faithful.
There are examples elsewhere of these sort of trade-off situations that evolutionary sorts of theories explain rather well.
So an example, sort of in the same ballpark, comes from cuckoos.
You know about cuckoos?
Cuckoos are nasty birds.
Cuckoos like to lay their eggs in somebody else's nest because this business of raising kids is resource intensive and if you can get somebody else to do it, great.
So cuckoos lay their eggs in somebody else's nest and it gets better from there because the cuckoo egg hatches early and the cuckoo chick has this lovely reflex where what it does is it arches its little cuckoo back and kicks the other eggs out.
So it kills off its adopted siblings and then it sits there making I'm hungry noises and gets fed by genetically unrelated birds.
There's an obvious problem here.
It's in the cuckoos interest to do this, but you can't have too many cuckoos.
If cuckoos do this to every nest then there's no next generation because there's nobody left dumb enough to raise your kids in the next generation.
So there's an equilibrium, you can write down equations for this that say a healthy population can tolerate this many cuckoos; this many cheaters.
You might ask by the way, why the stupid bird-- the stupid sparrow or whatever who's got a cuckoo in her nest doesn't say, get out of here.
And the problem seems to be that the cuckoo behaves as a sort of a supernormal baby stimulus.
You as a mommy sparrow or daddy sparrow-- you are built to say, oh, look at that.
It's cute, it's a baby, I want to feed it.
And your success is whether this thing is growing.
And here's this honking, huge cuckoo baby.
Wow, I hit the jackpot.
I got the greatest baby in the world.
Doesn't happen to be genetically related to me, but it's a great baby.
And so I stuff it full of worms.
Last year there was an article about a species of I don't remember what, but some other non cuckoo bird that has learned now how to recognize or has evolved the ability to recognize the cuckoos and you can't lay your cuckoo eggs in those nests because that species will get rid of you and preserve their own.
But anyway, there's an equilibrium state here.
You have a balance that tolerates a certain amount of cheating.
The same thing happens in human relationships though the details are much more complicated to work out.
But do people lie?
Yes
Yeah, sure.
Do people lie all the time?
No, you can't do that.
You can't have an arbitrary relationship between truth and udderances cannot have arbitrary truth value because then there wouldn't be any point to communication.
So on average people as a whole have to tell the truth.
Does everybody have to tell the same amount of truth?
No, the population can tolerate some amount of real liars, right?
Pathological, lying through their teeth all the time.
Oh yes, I love you baby.
I will devote all my resources to you.
Liars, or whatever realm it is, but again, there's going to be an equilibrium equation there somewhere.
Not as well worked out as with cuckoos that says, we can only tolerate so many liars.
So you could ask the question, is there motivation for guys to lie in relationships?
Are you seeing anybody else?
Oh no, not me.
If everybody lies about that the game falls apart.
If some people lie about it they get away with it.
So the population as a whole will tolerate some subset of cheaters.
One can have the hope-- the theoretical hope that another few generations of work in the area and you'd be able to write the equation.
You'll be able to say, look, a population like an undergraduate population can tolerate 10% cheaters.
If we've got 11% cheaters we have to invoke the dean's office or something and throw them out because the population will crash otherwise or something.
Anyway, we're nowhere near that point.
At this point we can't write the equations.
But that seems to be the way you get to the solution to the problem.
That the theory at the moment predicts kind of everything.
The theory also doesn't predict the full range of behavior.
I see I've got these problems listed on the handout.
In particular, it's explaining group behavior, explaining the behavior of individuals is something that I'll get to in the next lecture.
Actually, I'll get to some of it later in the present lecture.
And the third problem, let me say a bit about that before we take a break.
Is that civilization may have changed the rules.
One of the most striking-- there's several examples of this.
One of them is if you were operating in a sort of a purely Darwinian kind of a world that would be very difficult probably, but very different than operating in a world where you are being told explicitly what is and is not appropriate sexual behavior.
You're being told explicitly that your genes, your evolutionary history if you're a male may be saying go off and mate with everybody under the sun.
But there are societal strictures that say, not really.
That's just not quite really right.
So that complicates things, but a more interesting complication arises more recently in history with the advent of artificial contraception.
If the fundamental route to these asymmetries is that women get pregnant-- that sex makes you pregnant and it's only women that get pregnant, what happens if you decouple sex from pregnancy?
Does it change the game?
Well, potentially yes.
It has not made these asymmetries disappear.
That's a topic that I will actually return to in a later lecture.
It's not made these asymmetries disappear and that suggests that forces that shaped our brains, the evolutionary forces that shaped our brains over long periods of time still operate even though the rules of the game have changed somewhat.
You can get an intuitive feeling for this-- let's return to the question of female fidelity.
Should a woman cheat?
Well, part of the answer to why a female might be inclined not to cheat in a relationship was that the male might be jealous and that would have a variety of possible bad consequences; withdrawal of resources or potentially violence in some fashion, something bad might happen.
OK, how much do you think that those bad consequences are ameliorated.
If when she's confronted, you know, did you cheat on me?
Yes, I cheated on you, but it's OK because we took precautions.
Do you think that makes a difference in the argument?
No
No, intuitively it doesn't seem likely that it makes a difference.
Suggesting that the forces that are driving these questions of who should cheat and who shouldn't cheat are walled off from knowledge about modern contraception.
In sort of the way that the forces that are driving taste aversions are walled off of what you know about what it is that made you sick.
Remember from earlier in the course, you got sick because you got the flu, but you ate a tuna fish sandwich the day before you got the flu and now you can't eat tuna fish anymore because this little chunk of brain said, tuna fish poisoned us, we never eat it again.
Same sort of story.
Somewhere in the brain is a chunk of brain that is saying-- chunk of guy brain that is saying, need to know woman is faithful.
Why?
Well, because I need to know whose genes they are.
Woman isn't pregnant.
I don't care.
I'm a very simple little chunk of brain, I just need to know woman is faithful or I have unhappy things happen in me.
So the persistence of these sorts of asymmetries and these sorts of behaviors is another argument that can be used to argue that these are deep seated behaviors with evolution.
Oh, I should say, by the way, that these are explanations not excuses in any way shape or form.
Coming home and flipping to the other side of this right?
You're the guy this time.
Did your cheat?
Yes, but that's OK because I have deep seated evolutionary forces within me that make it necessary for me to cheat.
That's an interesting explanation, but it's not an excuse in a relationship any more than it's an excuse.
Yes, I hit my girlfriend because I was jealous.
You know, great, lovely, we have these great evolutionary arguments for it.
That's not an excuse.
That's where you get into again, why these things are politically controversial, is the possibility of turning explanation into excuse.
You can decide well, to use an example that I will probably use again, it's a little like saying, look gravity is a force.
You can't thwart gravity, so I'm just going to lie here forever.
Oops, lost the glasses.
Where'd the glasses go?
That's a really stupid argument and it's similarly stupid to say biology is a powerful force, therefore I had to do whatever.
There are other powerful forces out there in civilization: morality, culture, whatever are powerful forces and if they say don't do it, it may be hard work not to do it, but that doesn't mean you shouldn't do it.
Or you should do-- whatever, you got the basic idea.
Well on that sentence with way too many double negatives in it, let us take a brief break and then I'll turn to the question of individual behavior.
[INTERPOSING VOICES]
Oh, thank you
Should we do it before-- I mean, after they come back from the break maybe?
Or would you prefer at the end?
We [UNINTELLIGIBLE]
Basically what it is we're doing the comedy passes, whoever wants one we just have to pass them out
Oh no, if you're going to pass stuff out, be here at the end.
[INTERPOSING VOICES]
Yep, I'll do my best.
Will it be funny?
Quick, funny announcement.
All right, so be here in about 20 minutes
Yeah, no problem
So, let me elaborate.
I realized looking at the handout that I didn't elaborate particularly well on this problem 2 part about these evolutionary theories not explaining the full range of behavior well.
So it explains perhaps-- the theory as elaborated thus far explains for instance, why you might be attracted to one person or another, but it doesn't explain what you do in any particular way.
So you see an attractive, potential mate, if you're birds you might do some sort of weird mating dance or something like that.
If you're a human you might do some weird mating dance or something like that.
You can ask yourself about what do you do if there's an attractive mate possibility?
You could dance, you could hang around, you could act pathetic, you could act non-pathetic.
You could get person A to talk to person B and who knows C, who's the uncle of D or something like that.
Just knowing that there was evolutionary pressure here isn't really a full story about human mating behavior by any stretch of the imagination.
What's going to elaborate on it, you can probably get a good piece of mileage from asking about law of effect kinds of things.
What you do in mating situations or courtship situations is going to be shaped, at least in part, by the schedule of reinforcement.
You do a mating dance and the potential mate responds, you're going to be inclined to do that dance more-- it's not why birds do it.
Birds are wired typically to do the dance, but if you go off and do some weird dance and he or she says, hey, this is good.
You'll do it again.
The whole ritual, I don't know what you want to call-- who touches who when, where, and how is also heavily shaped by law of effect sorts of things.
If it's rewarded it happens again.
If it's not rewarded it doesn't happen again.
More on this later, but I did want to just say that there's nothing in these sort of evolutionary psych theory that really talks to that.
Now let me go onto try to say what does talk about individual behavior in romantic relationships.
You know, to get beyond this notion of what average populations do in a world full of potential mates, why are you attracted to this one now?
What can we say about that?
There certainly seems to be something to explain.
There are a variety of candidate sorts of theories out there.
One of them is nicely encapsulated in shakes-- oh I realized I didn't put the most clear cut piece of citation on that quote at the bottom of page 3-- That's Midsummer Night's Dream.
Shakespeare's Midsummer Night's Dream and Shakespeare has Theseus, Duke of Athens say, that lovers and madmen have such seething brains, such shaping fantasies that apprehend more than cool reason ever comprehends.
The lunatic, the lover, and the poet are of imagination all compact.
This is a theory of romantic behavior that says you're attracted to whom you're attracted to at the moment and you're doing what you're doing in response to that now because you're nuts.
I mean, basically it's a form of insanity, and not uncommon theory that shows up in literary circles.
The place to get some intuitive feeling for that is to remember what it felt like if you were ever completely infatuated with somebody.
Preferably somebody who didn't know you were infatuated with them.
Some random other person across the crowded room kind of thing.
That has the feeling of being more or less, out of your mind.
But you don't realize at the time and reams of bad high school poetry are written unders those conditions.
And if you're very lucky nobody else gets to see that stuff, mostly.
In my experience, back when I was in high school-- how many people have ever seen a book called Jonathan Livingston Seagull?
Oh, it's still out there.
It's very sappy.
And it was this sort of thing you gave to somebody who you were deeply in love with, preferably in high school.
And it turns out that both my wife and I have copies of this book given to us by people other than each other because we didn't know each other in high school-- both with inscriptions so florid that you just can't read them.
One of these I mean, why haven't we destroyed them?
Well, there's a certain nostalgia there of some sort and you gotta save these things for your children.
One day one of my kids is going to be up in the attic and find this thing and read it and fall down dead laughing.
All right, the relevant question here is that's not a bad theory, why isn't this the abnormal psych part of the course?
Why aren't we talking about love in the context of psychopathology?
What's the answer?
Nobody knows what the answer is.
Yep?
Psychopathology generally deals with things that are [UNINTELLIGIBLE]
Yeah, there's a reason it's called abnormal psychology as a [UNINTELLIGIBLE].
Now look, I don't know quite what would happen if bad people found a way to put something in the water that made us floridly schizophrenic.
If it became the majority state it would still be psychopathological probably, but by and large what we talk about in abnormal psychology is abnormal and being in love may be a sort of insanity, but it's a very-- well look, Shakespeare knew this too.
So if we borrow from As You Like It, we have the explanation of this given in this case by Rosalind in As You Like It saying that love is merely madness, merely a madness and I tell you, deserves a dark house and a whip as madman do.
Which tells you something about Elizabethan psychotherapy.
The reason they are not so punished and cured, which also tells you something about Elizabethan psychotherapy; the notion was not that you were just being nasty to crazy people, but that being nasty was curative.
The reason they are not so punished and cured is that the lunacy is so ordinary that the whippers are in love too.
It doesn't make any real sense, it's not a useful theory to say that being in love is a form of insanity because it's essentially a universal form of insanity.
You might think that there's nothing much to say in an intro psych class about this at all and that the whole topic is kind of best left to Shakespeare and other literary sources.
That's actually not a bad argument, but I think there is something useful to be said in the context of psychology.
We can switch back to Shakespeare and ask-- wait, first we have to check on how high school education is going these days.
How many people have to read Romeo and Juliet in high school somewhere?
OK, good.
Just checking.
Romeo is looking across a crowded dancehall here and he sees Juliet and he's supposed to be in love with?
Rosaline
Rosaline.
Good, you got that question right on the test.
He's been mooning around about this woman Rosaline.
He walks in, he sees Juliet and it's oh, she doth make the [UNINTELLIGIBLE PHRASE].
You can see why I'm not a Shakespearian actor.
She doth teach the torches to burn bright and all that good stuff.
Why?
What can you say about it?
There are a variety of things.
Well, we can develop what we can call the chemical theory of love here.
We passed through the insanity theory, let's work on the chemical theory a little bit.
Chemical theory gets you a certain distance.
Oh, I didn't put this on the handout yet because I wasn't convinced I was going to get here.
I'll put a few of these terms on there next time, but you could write on the back of the handout.
So chemical aspects.
One thing that's related to chemical aspects is the proximity factor.
You want to get two things to interact and make heat, you gotta bring them together.
It sounds trivial, but the scary version of this is that a not insignificant number of you will meet your eventual mate here.
Not here like in 10-250 right now, but during your undergraduate years at MIT.
So you know, somebody in this room may be that person or something like that.
One of the reasons is it's a great deal easier to fall in love with somebody you actually interact with.
There are lovely people at Cal Tech I'm told, but mostly-- no?
Well, there are bitter, jealous, disappointed people at Cal Tech I'm told.
All of whom actually wanted to go to MIT, but anyway, you see the point.
So proximity helps.
Now another chemical aspect of this romantic love business is that if you want a chemical reaction to speed up one of the things that's useful to do is to add heat, right?
That works, too.
And it works in not just like turning up the thermostat, but you are more likely to fall in love with somebody if your arousal level is up.
That seems pretty obvious.
The non-obvious piece of it is that you're pretty stupid about this and you don't care too much where the arousal comes from.
The Romeo and Juliet thing is not a bad example.
Romeo is which family?
Montague
Montague.
That sounds right.
OK, Romeo is at who's party?
[UNINTELLIGIBLE]
Do they want him at this party?
What are they going to do if they find him?
Kill him
So how does that make him feel?
Aroused
His little heart is going pitter pat, right?
There is some sense in which that contributes to-- that modulates how attractive you think somebody else is.
The clearest pop culture example of this is ask yourself about the standard adventure movie paradigm.
It's got a guy, it's got a girl, right?
Do they like each other at the beginning of the movie?
No
What do they do for the duration of the movie?
They find themselves in disasterous situations after disasterous situation.
Do they like each other during?
No, no, they spend the whole movie sniping each other and double crosssing each other and stuff like that.
The end of the movie they're both hanging by one fingernail from the helicopter over the volcano or something like that and what do they realize?
We're in love.
This is great.
It sounds silly, but the fact that we have been willing to tolerate that plot line forever suggests that there's something to it.
OK, I got enough time to tell you about what is arguably one of the great experiments in experimental psychology.
Did I already tell you about the shaky bridge experiment?
No
No, good.
This is an experiment so good that I did a pilgrimage to the place where it was done, which turns out to be in Vancouver.
Vancouver has a park with a deep gorge and a Raiders of the Lost Ark style suspension bridge going across it.
Or Shrek, you know with the donkey going across-- you know the bridge.
No volcanoes or dragons and anything, but it's a shaky bridge.
Here's what they did.
You're a guy for this experiment.
You're a guy and you are walking across this shaky bridge and in the middle of the bridge you meet a woman.
And she says, I'm doing a study for psychology, would you fill out my little questionnaire?
Well, OK. As part of process says, there's [UNINTELLIGIBLE] if you want to know more about this experiment once we've collected the data, you can phone me at this number at the lab.
My name is Sarah.
OK, so that's group 1.
Group 2 turns out round the corner from this shaky bridge, there's another bridge over the same river, it's a big concrete slab.
It don't shake at all.
You're a different white guy.
White guy?
I don't remember if you're a white guy.
You're a guy.
You're walking across the bridge and you meet a woman and same story.
If you're interested in the results of this experiment phone this number and ask for Rebecca.
Needless to say, Rebecca and Sarah are the same person.
The only thing that differs is whether you met her on the shaky bridge on the nonshaky bridge.
That data are, who calls?
So your call up the number, some guy answers, you say is Rebecca there?
And he says, no, no I'll take a message and hangs up the phone.
Another one for Rebecca.
It turns out that more guys who met Sarah on the shaky bridge phone in than who met Rebecca on the solid bridge.
Why is that?
The argument-- this arousal heat argument is, it's known in the trade as misattribution of arousal.
You walk across the bridge, you meet a woman person, you ask yourself, how do I feel?
My heart is beating.
My palms are sweaty.
It must be love.
And it's enough of an effect to change whether or not you'll phone up to get the results of the experiment.
Look, you shouldn't overstress the result.
Nobody's done this particular experiment, but if you meet a chicken in the middle of the bridge you don't say, my heart is beating.
My palm's are sweaty.
I love the chicken.
But it does modulate things.
The following content is provided by MIT open courseware under a creative commons license.
Additional information about our license and MIT open courseware, in general, is available at ocw.mit.edu
When last we met, I think, you have to remind me of this, I put the bits on the handout that I didn't have from last time, but we talked about --if you look at the hand out-- we talked about what I'm calling the psychopathelogical theory of love, and the chemical theory of attraction.
I seem to recall that we finished up last time talking about the shaky bridge.
So, that allows me to start today talking about social exchange theory.
I put the other stuff on there because I felt bad about not having it on there last time.
But we'll start in the top middle of the page two.
And, the question here continues to be, how do we explain why a particular relationship happens?
The evolutionary theory is good for saying something about broad forces, but now we want to know something about why specific relationships happen and do not happen.
And, I can think of no better place to begin then with me as a high school freshman.
The reason that I'm explaining this as me as a high school freshman, I made up this example a few years ago to illustrate the point that I wanted to make.
I ran through this elaborate scenario about high school freshman stuff like that.
And then I gave your predecessors a choice.
Did he do A, or did he do B?
Somebody raised their hand.
He said B. I said no, that's the wrong answer.
And, at that point, I realized I've been giving autobiography.
So I thought I might as well just go with straight autobiography.
All right, you've got to imagine this high school freshman, who you can imagine was kind of short, and nerdy, but good in English.
I'm just noticing, as the gentleman in the white shirt over there moves things around, Google has these, how to use your brain things.
What's the actual title on it?
How to care for your big, wonderful, high performing brain
Yes.
How to care for your big wonderful high performing brain.
If you find one of those around campus, over half the people have them, I realize you all want to go work for Google, but an entertaining act might be to see how many of the little factoids and stuff you think are directly contradicted or directly contradict the contents of this course.
There are a variety of oddities on that.
But, it's not bad, and you should send your resume to Google and become rich and famous, rather than just being a nerdy high school freshman, who was good in English.
So I was in Sophomore Honors English.
Great.
In Sophomore Honors English was, at least through the fog of memory, a very beautiful, very brilliant cheerleader and woman person.
And, if we return to the psychopathology theory of love, I was pretty infatuated with her.
The pick A or B question is A, I let her know that this infatuated guy was sort of sitting next to her in class whenever possible.
And, we had a perfectly nice high school relationship.
Or, B, she managed to graduate from high school presumably completely unaware that I had been deeply infatuated with her.
How many people vote for A.
How many people vote for B?
Why isn't John Kerry voting?
You would think of all people, he ought to be voting, wouldn't you?
[APPLAUSE AND LAUGHTER]
He does have excellent hair.
All right.
Look John, stop grinning at me, we can't all marry billionaires.
[APPLAUSE AND LAUGHTER]
All right, so the intuition of the vast, cruel bulk of you is correct.
I suspect, in her life, I kept sort of turning up like a bad penny.
I was continuously around.
But, there was no relationship there.
Why not?
What was my problem?
We won't ask for vast details.
All right, we'll ask for some details
Fear?
Fear works.
But the last example we had Romeo there.
He was presumably scared out of his little wits more or less.
Yeah
[INAUDIBLE] look at her from afar
I like that.
That sounds good.
Self- confidence.
I think I won't pursue this further, because I may learn way more than I want to know about exactly why I should have been fearful.
Low self- confidence and otherwise.
But, the fact is I didn't do nothing.
The other fact that occurs to me, is that, it would be entirely possible by this point, that her son or daughter could be a student here.
Yeah.
I heard good deep intake of breath there.
There's a scary thought.
The intuition is clear.
Let's flip the example around, and ask suppose that this gorgeous and brilliant high school sophomore was attracted to this nerdy, little high school freshman or something like that.
Would she do anything about that, one way or the other?
Let's do the A or B vote.
How many vote that she would say, let's go out together, and that his hair would curl or something.
How many vote, yes, she would do that.
How many vote no?
OK. Well little more divided.
There's more chance that maybe she would say something, but the majority goes with the no, or the, I'm not voting on this.
I voted once today already.
Maybe I voted twice.
But, anyway.
The why would be ok.
But, why not?
What's wrong here?
Economics.
Either we're sitting here reading the hand out, or he's on to something.
Where's economics, Matt
[INAUDIBLE] her kinds of values would go down
Oh.
We don't have to make this quite so specific, do we?
[LAUGHTER]
I should add, this would be deeply pathetic if since high school, I've been a broken and pathetic person who never formed a meaningful relationship with another person, and I was still waiting for her.
But, when I did finally get engaged to somebody, my dear sister's first reaction, was sort of like that.
Which was, how did you ever get somebody so attractive to go with you?
And, my mother's first reaction, -- we were going to be engaged for two years because we've been reading up this was a long time ago, before evolutionary psych was big, but we were ahead of the curve, and we knew that she had to be a couple of years younger than me, because she needed to be evolutionarily fit.
Anyway, she was two years younger than me.
I was already up here in grad school.
She was still at Princeton as an undergrad, so we would get married two years later when she graduated.
My mother's first declaration on this subject was that if anything happens to the relationship in the intervening two years, she was keeping Julie, my wife, and getting rid of me.
So, you may be on to something there.
Yes
In high school, women generally didn't ask guys out --
OK I could have flipped this around and change the genders here.
But there's a possibility of a social norm that the women don't ask men.
But, I will assert that there would have been a certain amount of discomfort if it'd been a gorgeous, brilliant male cheerleader, Hey, don't knock it, my sister- in- law married one of those.
University of Michigan, you know, gorgeous male cheerleaders are right there.
I don't know.
How is the MIT male cheerleading squad a big thing?
No.
OK.
Anyway, I think there would be a certain ambivalence about this relationship even if we switched the genders on it.
Any other comments on this?
Well, let me assert that this sort of relationship feels uncomfortable because we have a very deeply seated desire for relationships to be reciprocal.
If you want to account for my failure to ask this young woman out, in economic terms, you could imagine that what I did was, I looked in my romance wallet.
And, I said, I don't you enough to afford this relationship.
Not in strictly economic terms.
But I don't have enough status for this.
I'm a freshman.
She's a sophomore.
Made a big difference at that point.
You know, she's this, I'm this.
She's this, I'm this.
This isn't going to work.
But, it also doesn't work like this.
If you've got too much in the bank, in a sense, there's a discomfort that has to do with unbalanced relationships.
And there seems to be a very deeply seated notion that relationships need to be appropriately reciprocal.
Now this isn't just about romance.
If I open the door for you, you'd say, thank you.
Now that's a very simple sort of thing but, it suggests that you feel a need to reciprocate for whatever act might come your way.
In the sub culture in which I live, if I invite somebody over to my house for dinner, their reflective response is to say, what can I bring?
Not because they have some notion that the larder is bare at home, and if they don't bring something, there's no dinner.
But there's is a notion that needs to be reciprocal.
Quite typically, my response would be, oh you don't need to be to bring anything.
And, quite typically nevertheless, whoever's coming over would bring something anyway.
Now you get into interesting problems.
The exact nature of what is and is not reciprocately appropriate, seems to have a strong cultural overlay.
This was brought home to me by a tale my parents told me, which was, a few years ago, new neighbors moved in.
And, so my parents, being nice people, brought them over a welcome to the neighborhood gift.
I don't know what it was.
But they brought over a little gift.
That's nice.
But the people who moved in were Japanese.
The Japanese cultural expectation was, if you get a gift, you have to give a gift.
So, they gave my parents a gift the next day.
But, my parents' expectation wasn't that they'd get a gift, so if they got a gift, there's now a reciprocal demand here.
So, they gave the neighbors another gift.
At least by the time you get my mother's rendition of this, it sounded like the end of this was sort of medieval warfare.
You know, lobbing gifts over the fence.
Stop it already!
[LAUGHTER]
All of these examples point out that there's a deep- seated notion, that we don't like one- sided relationships.
We like relationships that are reciprocal in some fashion.
And the reciprocation needs to match the original act.
If I invite you to dinner and you throw yourself on your face, and swear eternal fealty to me, that's reciprocal, but that's odd.
That's out of proportion, and wouldn't be right.
The efforts to make these intuitions, into something more than intuition, into a systematic theory, one of the efforts is known as social exchange theory.
It is an effort to talk about social relationships in terms that borrow from common sense economic terms.
Where, one of the goals is to maximize your profits.
There are benefits.
There's the income side.
There's the cost side, and your profits are going to be something like the subtraction of the costs from the benefits.
Now, costs and benefits in social exchange are not going to be in strictly monetary terms by any stretch of the imagination.
Benefits in social exchange can be tangible like dinner.
They can be intangible like a compliment of some variety.
Oh that's a lovely, yellow t- shirt with some cool description on it.
Right.
I just added to the plus side of his ledger.
Of course, I've also added to the minus side, because I've directed attention to him, and now, he's embarrassed.
Everybody's looking at him
You're not getting anything back!
I'm not getting anything back.
No reciprocal here, at all.
I just got something back.
Well, anyway.
And, they can be internal.
The benefits could be internal, like a feeling of self-worth.
Some boost in morale, something like that.
All these would be things on the plus side.
And, quite different than just economic transaction.
And, the same on the minus side.
You could have a physical minus if someone smacks you, or something like that.
An intangible minus; I won't pick somebody out, because that would be rude, but did you get enough sleep last night, or something like that?
It sort of suggests that you don't look so good, and you think, I did get a lot of sleep last night.
Maybe I just don't look so good.
And, it could be internal.
You know, a feeling of diminished self- worth, or something like that.
And then you've got your profits that are the difference between those.
There is a necessary, at least at this stage of any sort of model development, there is a necessary imprecision here.
Dinner minus two insults equals what?
It's clear that at least a decent dinner is on the plus sign.
The insults on the minus side.
But you can't do the calculation the way you would do a profit loss statement in economics.
So suffice it to say that in its present state, the model is qualitative, more then quantitative.
But, rather like evolutionary psych, the promise is, or the hope would be, that you could move in a more quantitative direction.
Do people actually do any sort of math of this sort?
Well you can get some intuition about that perhaps by asking yourself about some sort of a scenario.
Like, you go to the polls today, you're in line, and you see this guy who you think you saw voting earlier in the day, when you voted the first time.
So, what do you do?
Now, you could challenge him or not challenge him or something like that.
And ask yourself, what would determine that decision?
You'd sort of run down a mental checklist.
If I challenge him I might get it sort of a tangible benefit of preserving the democratic process in some fashion.
I get an intangible set of compliments from people all around me about my bold stand for ballot integrity, and I'd get this great feeling of self-worth.
On the other hand, I might be wrong.
He might punch me.
And, my neighbors in line, might simply derive me for being a stupid busybody, and it doesn't really matter what your answer is.
The point is that this sort of cost- benefit analysis has the feeling of the thing that we do, not explicitly, not on a piece of paper with a check list, but the thing that we do quite automatically.
The notion that you want to maximize your profits is only one of the core tenants of social exchange theory.
You know that because the handout, I think, lists three of them.
One of them is to maximize your profits.
Oh, there it is.
It says, "Three Tenants Of Social Change Theory.
Maximize Your Profits."
The second one is the notion that the profits for both sides in an exchange should be roughly equal.
That's the embodiment of this notion of reciprocity.
We don't like relationships where you maximize your profits by exploiting the other party to the relationship.
And the third one, that we haven't talked about yet, is that for a relationship to last, it's gotta be better than the perceived alternatives.
The plus or minus of that checklist is not itself adequate.
If you think about this in boring economic terms.
You've got a pile of money in this account.
It's making 1%, so you're making money.
But, you see that there's this other opportunity over here, to make 5%.
Well, what are you going to do with your money.
You're going to move it over.
Look at that.
Not only did I get a sweatshirt out the deal, I got a nickel.
And, a cute little pink thing.
He left the coin he flipped.
This is a great day.
So, the same logic applies in social exchange theory to relationships.
So you're in a relationship that's getting you a net profit, with this perfectly nice person.
You see this other person, and you say, I could get 5% on my relationship over here.
What do you do?
Now we'll come back later in the lecture to the possible problematical nature of that tenant of social exchange theory.
But, the important point for now is, it's not just that you want to be running positive, not negative, you want to be running as positively as possible, and you're looking at your alternatives to see how you can do that.
Now, this sounds, particularly by the time we start talking about moving your investment from one person to the next person, terribly crass.
Is there any evidence that people use is this sort of quasi- economic calculation in romance?
In making romantic attachments with other people?
There have been a number of experiments that have endeavored to look for evidence for social exchange theory principles in the settings of romantic type relationships.
And I want tell you about a few of them.
That's why I've got all these cute little boxes on the handout that allow you to plot out the results, the predictions and the results.
So, let me tell you about one of the earlier great efforts.
Down on the handout is the great computer match dance.
Here's the way it works.
In the early days of computers, signs go up on campus for a big dance but the thing is you're not supposed to invite anybody.
You're not supposed to take your date to this dance.
What you're going to do, is you're going to come fill out this big form, collecting buckets of information on you, your GPA, your height, your weight, family income, all sorts of stuff.
And, the people on the other side of the table, when you're doing this, are looking at you, and rating your attractiveness, on a scale from like 1 to 10 or something like that.
And, then all this goes into a good 1960's style computer, so it's big computer with lots of lights that flash on it, and out the other end is going to come out the person you are matched with to go to the dance.
OK. Got the basic set up here?
So we've got the male.
He can be rated on some scale.
You've got the female.
She can be rated on a similar scale.
And, the story here, is that we've matched you up with the perfect person for you in some fashion or other.
Actually, I don't think they told you that this was the way it's going to be done.
It is some complicated algorithm.
The fact is the complicated algorithm was to flip coins.
The pairings were completely random.
So, the data on any dimension that you looked for, were designed that male and female ratings would be uncorrelated.
OK. Now you have the dance.
And, the question is, this being the 60's, the issue of males asking females rather than the other way around, is a much more straightforward societal norm.
The question they asked was which males asked which females out again for another date?
What's the social exchange prediction here?
That's a hand?
[UNINTELLIGIBLE]
Yes.
People should sort themselves in some fashion to produce essentially reciprocal balanced relationships.
So, this quadrant is the me not asking her out in high school neighborhood, and this is the reverse.
Her not asking me out hypothetical relationship.
But these are the matched relationships that ought to work.
In fact, this experiment is a bomb.
It doesn't work out the way social exchange theory predicted.
So, you look at the correlational data for all sorts of variables, and it turns out that the only variable that makes any difference is ratings of female attractiveness, and the data look like this.
Basically females who were rated as more attractive got asked out.
What's going on here?
So the answer is there's a fatal methodological flaw here.
There're two possibilities.
One is that social exchange theory is wrong.
But If I thought that were the case, then why would I bother to lecture about it, at least in this sort of detail.
The other possibility is the experiment had a methodological flaw.
So what's the problem?
Why didn't this work out?
Oh, Out in the cheap seats, next John Kerry there
People thought that they were well matched
That's good.
Usually I get half a dozen bizarre ideas did before somebody hits on the right idea.
But good for you, you got the right idea first.
The problem is with the cover story.
It's not people specifically who are the problem.
It's guys.
The guys thought, so you gotta imagine a guy here.
So if this is the scale of attractiveness, so here's ugly guy, ugly guy has thought all along that he was, I've got a warm heart, but I'm ugly, and then see, this high tech computer, spits out your perfect match, and she's gorgeous.
It's the economics of romance equivalent to looking into the wallet and discovering you've got a whole bunch of 20's you didn't know were there.
And so everybody decided that they were well matched, and this had it's biggest effect on the guys who had previously thought that they didn't have that much in the bank, and they thought well, I guess I can afford this.
They all asked out the-- what we don't know, by the way, which of these phone calls actually ended up in a date, and or relationship.
The social change prediction would have to be that only these would have survived.
But we can come back to that minute.
Anyway, the experiment's a bust, because the cover story gives people the wrong idea.
This makes an interesting point about these valuations.
They're based on your perception in relationship space, they're based on your perception of the other, and your perception of yourself.
We have met males, for instance, who consider themselves to be God's gift to women, against the evidence.
If you have a systematic misperception of your own value, you're going to end up in the non- social exchange part of this space.
And, then, the question becomes whether or not its a self- correcting system.
If you think that you are a 10 on every scale, and you keep asking people who are by your assessment also 10's on every scale, and they keep saying no to you, does that eventually cause you to come up with a more realistic assessment of who you might actually be in this marketplace.
Well.
We haven't got any evidence if this works at all yet.
Let's go find ourselves an experiment that doesn't have this fatal flaw in it.
Did I give this a clever name of something sort?
No.
It just says, let's try again.
So, here's this Kizler and Barrel experiment, that works as follows.
You go into a psych lab to do what you think is some sort of a cognitive experiment.
The task don't matter.
You're doing this task, broken into two halves.
You're a guy.
And, at the end of the first half, you're going to take a break, and you're giving some feedback about how you're doing.
The feedback is either of the form, wow.
You did really well on that.
Most people don't get scores that are that good.
You're really good at this sort of thing.
So that's a good, self- esteem boosting kind a bit of feedback.
Or, you're given feedback of the form, did you get enough sleep last night?
Most people get a higher score.
Well, never mind.
It's OK. Let's take a break.
And so you've gotten the bad information.
Now, needless to say, this is a set up.
What information you get is uncorrelated with your score.
So, you may have done brilliantly.
You may have done badly.
You have no idea how people do on this task in general.
And, and the experimenter has flipped a coin and arbitrarily assigned you to the high self- esteem or low self- esteem group.
That's the manipulation of the male.
This is the guy.
Now, we're going to go have a snack.
You and the experimenter go off to the cafeteria to have a snack.
And, you get some food, sit down at the table, and a female friend of the experimenter comes and says, Hi, can I sit down with you?
And, the experimenter says Hi, sure.
A female person comes in two flavors.
As you might guess from this two by two thing.
There's only one female here, but the manipulation here is she's either looking good, or she's looking bad.
I don't think, I've ever seen pictures of this, so I don't quite know how they did this, but the idea is that she's arranged herself to look attractive, or comparatively unattractive.
Same person, so you know no personality variables changing, and a fairly tight script.
And, at this point, the experimenter says, I need to go rinse a few things out.
I'll be right back.
You two talk.
And, so this woman, also an experimenter is sitting there.
And the data for this experiment are any indications of romantic interest on the part of the guy.
Does he ask for her phone number would be a sort of a transparent kind of thing.
But there's a long checklist of things that he might do or say that would indicate some sort of interest, And, you're just making little check marks presumably relatively subtly.
The trivial social exchange prediction is he's feeling good, she's looking good.
It must be love, kind of thing.
And this the cell that gets the highest number of indications of interest.
The question is where's the second highest set of marks in this experiment.
And the answer is down here.
She's looking bad, he's feeling bad.
It must be a match.
And, these two lag behind.
I don't know which one is three and which one is four, actually.
But the important point is that there are more indications of interest here, has would be predicted by a social exchange theory that's looking for matching.
This is the case where you looked in your wallet and you discovered that you actually had a couple less twenties then you though, but you still want to buy something.
And, well I guess.
Doesn't this sound terrible?
It not only sounds terrible, it sounds a little on the artificial side.
Is there any evidence that this happens in a more realistic setting?
And the effort to figure that out was done with the realistic setting of large introductory psychology classes.
So, large intro psych class, I think maybe Minnesota, or Ohio State someplace where they had a huge intro class.
And, what they did was, they a largely freshman class, they looked for who formed relationships.
So, let's go just like the computer match dance thing, but we'll take the relationships first, and work backwards from that.
So they got like 213 I think, couples, out of intro psych.
And, they did the same business.
Rated them on attractiveness.
Rated them on everything under the sun.
SAT scores.
Religion and all sorts of stuff.
And, scatter plot the data.
What you find is that these freshman pairing were not highly correlated.
They weren't random, like if you randomized in the computer match thing.
There are factors that did correlate.
But, they weren't particularly striking.
But now, let's ask, so we've got some cloud of data points again, with perhaps some positive correlation to it.
Let's come back two years later, and ask, who is still together?
And, what you find, two years later, I don't remember how many were still together, but of the ones who are still together, now essentially all of the variables correlated.
So, you could pick anything you liked, SAT scores, religion, looks, socio-economic status, and now it's not that it all lay on the line of unit slope, it's not that somehow you went through State University of Minnesota or something like that, and you found the one person who had exactly the same SAT scores as you.
And, their parents made exactly the same amount of money.
And you were exactly the same height.
No.
What you've got is it's still a big cloud of data points, but now, all the variables are positively correlated.
Well, what's the cliche here?
I'll give you a hint.
Thank you.
Birds of a feather flock together would be the sort of punch line of this experiment.
There may have been a degree of noise in the initial assortment, but the relationships that lasted where the relationships where the birds of a feather were flocking together.
And that hand is going to say, but, Thank you.
Next thing on the handouts says the other cliche is opposites attract.
Now the thing about cliches is that cliches don't get to be cliches unless there is an element of truth in there somewhere.
In the element of social exchange theory, truth in opposites attract is that opposites attract when they increase each other's profits.
So, this sense in which opposites attract is she talks, he listens.
From the outside, it may look like, my goodness, he talks all the time.
She talks all the time.
And, she never says a word.
How can they how can they live together?
Well, the answer is its great deal easier probably, to live together that way, then if they both talk all the time.
Those two birds of a feather are going to drive each other sort of nuts.
And, he's a great cook.
She's a great eater.
Or, at least an appreciative audience or something.
That is the sense in which opposites attract.
You can ask yourself about roommate issues, where there's a certain amount of random assortment of people.
I gather here, you pick your roommate on the basis of sketchy information when you arrive.
Is that the current scheme?
So, it's a sort of a courtship on the basis of limited information.
Anyway, when you discover that even though you were correlated on a bunch of variables, it turns out that there are few variables that really matter.
Like person A is a slob.
Person B is a neatness freak.
That ain't an opposites attract kind of situation.
There's a couple of head shaking around here, saying I know that.
Or, in our lab, she wants it to be 80 degrees.
I wanted to be about 50 degrees.
That ain't opposites attract.
But there we have it.
This is also not reciprocal, asymmetrical, relationship, where I'm the boss, and I get to play with the thermostat first, unless Kristen gets in first, in which case, it's tropical season in the lab.
Anyway, the problem is she got numbers on her side.
The other people in the lab, seem to like it hot.
I don't understand them.
Anyway opposites attract, in these romantic relationships when they increase each other's profits.
That's a useful, complication to keep in mind.
It's not just a matter of we all rate each other on all these scales, and then pick the closest thing to exactly us.
That's not what's going on.
That's going to end up being one factor, but there's a more general factor of what you're looking for are relationships that are profitable and roughly equally profitable for both sides in the relationship.
Let's see.
I think I couched the next bits as the problem set.
The social exchange problems set.
Let's let's turn to that after taking a brief break here.
Let me say a quick word.
I got a couple of questions about the syllabus.
Let me say quick word about this.
The chapters that are assigned at the moment for instance are the chapters on social psychology.
Like any topic here, huge area.
I'm taking a love and romance cut through it.
So I think, for instance, you'll find stuff about social exchange theory in the book, but not particularly necessarily in this sort of straight love and romance track.
I'm assigning chapters that's are, it'll be less obvious that the book and I are marching in lock step then it might have been for the memory chapter, or something.
Somebody pointed out that the sleep and dreams lecture has associated with that a chapter that seems to be about mental illness.
Well that's because sleep and dreams going to be connected with Freudian interpretation of dreams.
It's all part of talking about Freud and psychoanalysis, so that's the most relevant chapter at the moment.
It'll be less true for this half the course then for the first half of course, that there's an absolute clear match between the set of topics in lecture one day and the set of topics in the accompanying chapter.
They will talk to each other, but they will not match up quite as tightly as they might have earlier in the course.
Let's do a few social exchange sort of problems.
One of these is the problem of bad relationships.
One of the mysteries on the face of it is why would somebody stay in a relationship that's running a deficit.
Perhaps the clearest example of this is why would a woman stay in an abusive relationship?
I pick a woman because while there are males who are abused in heterosexual relations, and other males who are abused in homosexual relations, the bulk of abused partners in relationships are women.
But, why would anybody stay in an abusive relationship?
Why would a woman stay in an abusive relationship?
That's where this third tenant of social exchange theory comes in.
The social exchange answer to the question is you would stay in a bad relationship if you couldn't see a better alternative.
And, so, if you imagine a hypothetical situation of somebody in an abusive relationship, maybe she's got a couple of kids, and while she was raising these kids, she's not working.
So, what's she going to do?
You know, if she leaves, what's she going to do with the kids?
How's she going to feed herself.
Et cetera.
Part of the logic of things like shelters for abused women is a logic of giving an alternative that looks better then the state you're in now.
There's nobody saying that living in a shelter for abused women is a relationship that's running a great big positive score.
But if it's running less of a negative then the relationship that you in, you might get out of the abusive relationship.
That's part of what drives this notion that it's important, not just what the sign of your profit- loss statement is, but what the alternatives that you perceive are now.
And the you perceive part is important here too, because we all know people who are in relationships where you from the outside look at this relationship and you think you know that's not a good relationship.
Why is he in that?
Why is she in that?
But, if they see it as the best current possibility, that in social exchange theory, will be relatively stable.
Now, these next two are sort of related to each other.
Who has power in a relationship, and who works harder in a relationship.
In a sense, they are sort of reciprocal of each other.
Who works harder?
The person with less power.
Let's do this through a cartoon.
It's another sort of cliched relationship out there.
Young, beautiful woman.
And old, less- than- beautiful guy.
What's the other part of the description of the guy.
Rich.
Why?
We need a hand here.
Nobody's got a clue
That gives him a way in which he doesn't have to work all the time --
Well this perception that she's bringing more to the relationship than he is, right?
She, in a sense, has power.
If she looks around and what are my alternatives?
Here I am.
Beautiful.
I'm hanging out here.
Gee, I ought to be able, if I look at my alternatives, to move here.
So I have a certain power to move if I'm this young beautiful female person, hypothetically.
And, what he's got to do, the reason he's gotta work hard is to prevent that movement.
Well, how do you prevent that movement?
One way to do it is to increase -- well actually, I suppose not to prevent that movement -- it's to make that movement within the relationship that you're in now.
If she's a 10, and he's a 1 or 2 or something, he's gotta make himself into a 10.
How's he make himself into a ten?
He can't make himself young, he can't make himself beautiful.
What he's gotta do is in the cliche version, shower her with diamonds and furs, or something like that, increasing the benefits that she's getting in this relationship, so that other relationships don't look more attractive.
So, the basic notion is that if you're bringing less apparent stuff to the relationship, your partner is sitting there saying, I could do better, at least hypothetically.
And you're sitting there saying, if I don't want him or her to go elsewhere, I've gotta to be more loving, or I've gotta cook better, or I've gotta do something here.
So I'm the one who's got to work hard.
And, the other is the one with the power there.
Now, are romantic relationships really just this crass economic kind of thing?
Sounds pretty grim.
Well, there are important differences between economic exchange and social exchange.
One of them we've already pointed to, which is the fact that the economic costs and benefits are typically much more calculate-able than the social ones.
That's the dinner minus two insults problem.
The other one that I point out here, is that the rules of negotiation, what's permissible as negotiation in economic exchange, is considered desperately gauche, not done in social relations.
You just wouldn't have a conversation that says you can come up to my at room later if you take me to dinner.
OK. What kind to dinner do you want?
Well, you gotta take me to a four star restaurant.
No way.
We'll go to Burger King.
No.
How about three stars, three stars.
All right, how about the two star Thai place down the street?
Oh.
OK.
It's a deal but you can only come up for 20 minutes.
That sort of haggling in strictly economic terms is not considered mainstream romance mostly.
It's also not really mainstream US economics haggling these days, either.
If you go to Target and want to argue about the price of a toaster oven or something, they're not actually going to be that interested.
But, in those places in the economic sphere, where haggling is still permissible, a talented haggler, a talented negotiator will work hard to be your friend.
Why?
Because the clearest example that I know of in American economic life is car dealerships.
Go to a car dealership.
You will do this one of these days.
Go to a car dealership.
The guy who comes out -- typically a guy-- who comes out to sell you a car is going to be your best buddy almost instantaneously.
Very nice, friendly guy.
Why?
Well, maybe he's a very nice, friendly guy.
We shouldn't disparaged that possibility.
But, the other reason is, that if you behave to him as though you are in a social exchange, while he knows perfectly well, that what you are doing is an economic exchange, he's going to make money.
Because you are disabled.
You cannot argue with your friend about money in the way you can argue with somebody in a strictly economic market place.
So he's going to be your friend.
You're going to come to a nice friendly agreement, and then, it turns out that after you have made this deal, that's not enough.
You shake hands on that, that's not quite enough because then he's gotta go talk to his manager about it.
He's such a good friend of yours, that he may have gone a little too far, he worried.
So, he's gotta go talk to the manager.
And the manager is not your friend.
So, he's gotta go and talk to the -- but he's your friend, so he'll go and fight it out with the manager -- you're never invited to this discussion, by the way.
He goes back to talk to the bad manager.
And I think what they actually do is, they chortle a little bit about their kids, and the Red Sox, and stuff like that.
And then your friend comes back, with a sad face, and says, my mean, nasty, evil manager, not quite those terms, won't go for it.
We can't just can't do this.
You and I, we could do this, with just an extra say, $500 of your money.
And, because we're friends, you put up the $500.
I think, but, I don't know this, but I think, the car dealers are explicitly -- they know about social psychology.
They are instructed that this is a useful way to negotiate.
By being your friend, by moving the discussion from an economic exchange to a social exchange, because of the differences in those sorts of interactions, that can be exploited in a way that makes money the for the car dealer.
The most beautiful example of this, in my own personal experience, was not at a car dealership, but was in the city of Marrakesh in Morocco where everything is negotiable.
A cup of water; the price is negotiable.
But anyway, a lot of haggling there.
And, so I was there for a conference, and we were taken off as a group, for a tour of the old market, which is great.
You should all go there sometime.
We went to a rug merchant.
This was early in my career.
I didn't have no money.
So, I'm not buying any rugs.
Because to buy rugs, it's good to have money.
So, I'm sitting there drinking buckets of mint tea, which is what you get in Morocco while other people are negotiating for rugs.
It's kind of fun to watch.
But at some point, some guy decides he's going to sell me a rug.
And, he's going to show me the special rugs out back.
So, we're going out back, we're up the back stairs, and he asked me, are you Jewish?
My mother brought me up to be honest.
Yes, I'm Jewish.
The next thing you know, we're hugging and kissing, because he's Jewish too.
Last week, I think he was a Methodist.
But, in any case, so now we're friends, of course.
Not friends, I mean, we're practically blood relations.
He shows me only rugs woven by Jewish virgins.
How do you know this is the case?
Well, they've got this six pointed star on them, as he points out.
Well, it turns out that every rug in Morocco seems to have six pointed stars on it.
But never mind.
Plus, the guy has also taking his intro psych class, and he knows about forced choice psycho physics, rather than saying do you like this rug, what he's doing is holding up two rugs, and saying which one do you like better?
So, you gotta give an answer, right?
I like that one better.
OK.
Goes in the stack.
By the time we're playing this game a little while, not only does he have the theory that I'm buying a rug, I'm buying a stack of rugs.
But I still don't have any money.
And, so I remember what it said in guide book.
In the guide book, it said go to the state store, because at the state store, they only mark up, by fifty percent.
And, so you the price you're looking for is to negotiate down by about fifty percent, and then everybody will be happy.
So, I remember this, and we're in the state rug store, I know what I'm going to do.
I'll offer this guy 10% of his asking price.
He'll be so offended, that he throws me out.
All right.
So I offer him 10%.
And, he accepts.
Now, you want to violate both the borders of social and economic exchange, backing off your own offer, is really, really rude.
And, at that point, he threw me out.
Which is too bad.
I probably should just maxed out the credit card.
I think I probably had a pretty good deal at that point.
But, in any case, it was another beautiful example of trying to use the rules of social exchange, not to mention half a dozen other bits of applied psychology, to close and economic deal.
I think all my friends who came back saying I got him down to only 50% of the price probably made a bunch of people really, really happy.
The last thing I want to mention here, is one last problem in social exchange.
That's the problem of vastly unequal and inevitably unequal relationships.
There are relationships out there that are simply not going to be reciprocal.
One of the clear examples, are parent- child relationships.
Like even if you're thinking strict economic terms, how much money have your parents put into you?
How much money have you put into your parents?
And it doesn't get any better by the time you're my age.
It's inevitably a relationship that is going to be unequal, and that produces, as you may have noticed at various stages in your life, a sense of discomfort about that inequality.
How do you deal with it?
There are lots of ways to deal with it.
And social exchange theory is not capable of predicting which one to chose at this point.
But you know, for example, a very typical early adolescence one is the assertion to one's parents, you've never did nothing for me.
No rational 13 year old in the bits of some fight, you never did nothing for me.
OK. Where did the food come from?
Well, yeah, you did that, but.
It's not a logical statement, but it's a statement that attempts to balance the accounts in some fashion.
It's a way of dealing with this feeling that the relationship is unequal.
There are the parental -- how many how many of you have heard, at some point, the line from your parents that one day, you should have a child like you?
Or words to that effect?
That's another sense in which this can be balanced.
The parental notion that the accounts will be balanced when your children do unto you as you have done.
And, guess what?
Your parents are right.
And, can you remember thinking, at some point, that you're never-- should you ever have children, --you're never going to say anything as stupid, to your own children.
Oh one perfectly lovely concrete example, why do I have to do this?
The parental answer to which is, because I said so.
Have you heard the little voice in your head promising that you will never say anything that lame to your own kids.
Guess what?
It's very grim.
And, you can just sit there listening to yourself as you say it, and say, oh god, I said I'd never do that.
All right.
Beyond that realm, what do you do about these unequal relationships?
Last thought, there's an interesting thought that comes out of Roger Brown's book, Social Psychology, a beautifully written textbook on social psychology.
He talks about the central exchanged in social exchange.
The argument is that not all of the interactions needs to be between you and the person with whom you were having a relationship at this particular moment.
There can be a bank.
Here's the experimental evidence.
A bit of experimental evidence; last little two by two on there.
And, in my new free nickel will do just fine.
It's becoming at slightly archaic example, because most people don't use phone booths anymore, but, you still know what a phone booth is, right?
You go into a phone booth, the first thing you do, before dialing the number or anything is to check what?
Not if you have a quarter.
Is there a quarter in the change slot.
Right?
Everybody goes and check that little change slot.
And, oh, I just changed her life.
Check next time.
Sometimes there's a quarter there.
It's really good.
Anyway, we can use this two by two here.
What this experiment did was to spike phone booths.
They put quarters in the phone booth.
And, so you go into the phone booth, and some people got a coin.
And some people didn't get a coin.
This is not a big change in your life, but it is an unreciprecated good.
You're getting this good, but you can't say, I gotta go do something to the guy who gave me a quarter.
Can't work.
So, here's what happens next.
You come out of the phone booth, having made your phone call.
And, as you're walking out a woman comes by and she trips.
And her stuff goes all over the place.
The question is do help her?
And, the data are, from this particular experiment, so it's help and no help.
Of the people who got a coin in this particular study, 14 helped.
1 didn't.
Of the people who didn't get a coin, so this is the bulk of the population as a whole, 2 people helped.
And 24 didn't.
So, now that's very interesting.
That suggests that just getting that coin produced what Brown would see as a an effort to reciprocate to the world more generally.
Now, that's a big effect.
Does this mean that we could make the world a better place by mailing everybody a quarter?
Doesn't sound right.
Now the experimental evidence that it's not right comes from another version of this experiment.
You're at home.
Somebody rings the doorbell and says, I'm new in town, I'm setting up a new company.
We're going to sell paper.
I wanted you to have this stationary.
Nice stationary.
Thanks.
Bye.
Shortly thereafter, you get a phone call.
Somebody on the phone says, is so and so there?
No, you got the wrong number.
Oh no!
I'm in a phone booth.
I don't have any more money.
Could you please call Fred and tell them I'll be home late or something.
I don't remember what the exact story is, but it's a request to do a favor.
If you just got the phone call, no paper involved, only 12% of people were willing to do this favor.
If you got the paper, and you got a phone call five minutes later, 80% of people complied.
But, if you've got the paper, and, were called 20 minutes later, it dropped back down to 12.
So, it produced a little bump of good feeling.
Now, what we don't know, because you can't really do the experiment, is whether this means that the central exchange idea is trivial, and is a very short lived teeny phenomenon, or if you live a life where much bigger good things happen to you, or for that matter, bad things.
Do you end up with an account at the central exchange.
If life is good in general, do you feel a central exchange obligation to be good to others?
If life is bad, in general, do you feel a need to trip people and watch the folders fly?
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare is available at ocw.MIT.edu
Good afternoon.
For the past couple of lectures, what I've been doing is using love and romance as the way to talk about broad issues like evolutionary psychology that could be talked about with a wide range of other examples.
Love and romance just happen to provide a particularly good set of examples for that particular topic.
I'm going to do the same thing now with attitude formation and the links between attitudes and behavior.
But I'm going to switch from the love in relationships topic, to the topic of racist or prejudicial behavior and attitudes.
Again, not because that's the only set of attitudes that are interesting or important, but because it makes for an interesting path through this material.
So, when you read the book, get the topics discussed and rather more general terms.
And, here, I'll discuss them in the specific terms of this particular problem.
I should note at the outset what it says about halfway down, I see the first page of the handout, which is, I'm going to do my best to give an explanation about why prejudiced attitudes are easy to come by and, are readily comprehensible in terms of psychological processes that we actually know something about.
But explaining these things is not the same thing as excusing them.
You can have a society that says, look, all else being equal, racial prejudice and particularly behaviors based on prejudices, based on gender race, national origin, religion, that those sort of biases are bad.
And when they lead to behavior, we want to change that behavior.
That's quite separate, not unrelated to, but it's separate from the question of how you would explain it psychologically.
It's important to remember that explanation is not the same thing as excuse.
Because I don't want people going out of the lecture saying, my psych professor explained that it's really, really easy to develop prejudicial attitudes, and that's OK.
It's the that's OK part -- I also put on the handout a pair of quotes from the early days of the civil rights movement; one from Eisenhower saying that you can't legislate morality, and a response from Martin Luther King saying, well maybe not, but you can legislate the moral behavior.
That's really the social policy point.
If you decide you don't like something that biology, psychology or whatever is pushing people towards, you simply have to do something to make it harder for them to go where that tendency might push them.
Well, in any case, what I'm going to do, is work a story about the development of prejudicial attitudes that's got four factors here listed at the top.
And, I work through each of them.
I hope you can see how they're tied together to make prejudice a very available option to us.
Why this sort of thing happens to us with some regularity.
The first factor that I list there is ethinocentrism.
That's the tendency to think that your group is the best group.
We're number one kind of thing.
If you want to come up with, say, an evolutionary psych argument for why this might happen, it's kind of trivial.
If you've got some notion that you want to get your genes into the next generation, then you might as well favor the people who are more closely related to you.
So you should be more favorable to humans than to mice.
You should be likely to be more favorable to people within your group, and even more so to people within your family, out of this sort of fairly straightforward application of evolutionary theory for example.
The more interesting aspect psychologically is how easy it is to get ethinocentric effects.
So to get the we're number one effect.
Interesting evidence for this comes from these minimal attachment experiments that called, minimal group affiliation experiments.
There's a lot of them.
Let me describe a couple to you.
So here's an experiment.
You come into the lab, and we're going to do an assessment of your taste in abstract art.
And, we're going to show you a bunch of abstract pictures.
And, you're going to say how much you like it on a scale of one to seven or something.
And, then the feedback you're going to get from this at the end is that you like the work of Paul Clay, one abstract artist, better than you like the work of Wassily Kandinsky, another abstract artist.
Or it might be the other way around.
I don't even remember, by the way, whether they were actually using real Clay and Kandinsky pictures.
But you're going to get told that you're in the Clay group or Kandinsky group.
This is not a group assignment where there is a lot at stake.
Let us suppose that you're in the Clay group.
You've been assigned to the Clay group.
In the second part of the experiment, you're playing some sort of a game.
I don't quite remember what the story was.
And, it ends up with you being able to operate under one of two pay- off rules.
In one rule, you're going to give the other group one buck, and your going to get two bucks.
That's one possibility.
The other possibility is a rule that gets you three bucks and give them four bucks.
So, this is you.
This is them.
You've got a choice between option A, and option B.
Clearly, the rational choice, from your vantage point is option B, because you get three bucks, and not two bucks.
But, in fact, there's a bias towards picking this option.
Why is that?
Well, if you get two bucks, you getting more then them.
Here, I'm going to get more than I would have got there, but these guys are going to get a whole bunch more.
Why should I give those Kandinsky lovers a bunch of stuff?
Maybe it turns out that after you figure that these Kandinsky sorts really are a different sort of scummy kind of person, who you really ought to stick it to.
Well, in fact, there's no difference between these groups.
The group assignment is completely random.
So any such distinction that you had made in your own mind is meaningless but maybe you didn't know that.
So we can re-run this whole experiment by saying get rid of this silly cover story.
We're going to flip a coin.
You're in group B.
B.
A.
A.
B.
B.
B.
B.
Look guys, it's random.
There is nothing differentiating your group from the other group.
Guess what?
You get the same result.
I think it does get a little bit weaker, but not much.
So, just the act of being in a group, causes you to be inclined to favor that group.
You are also inclined to think that group membership is diagnostic.
Let me describe a different experiment actually, as the easiest way to describe this.
Suppose I put up a bunch of dots, and we're above the subitizing range here, obviously.
OK. Quickly.
How many dots are they there?
I don't know.
You could guess.
You get lucky, and be right on.
But you'd either be above or below, so you're going to do this for awhile.
Guess how many dots there are.
And, we are going to declare that you are an overestimater or an underestimater.
So, that's the group assignment in this case.
Now the interesting thing in this experiment is that you've done it with another person.
And, the possibilities are that it could be two guys, both of them are overestimaters.
It could be two guys, one of them an overestimater, one of them an underestimater.
Similarly for females right?
They could both be underestimaters and the critical conditions are.
It could be a male, and a, female who, are both overestimaters both labelled as overestimaters or both labelled as underestimaters.
And the critical condition is male overestimater, female underestimater or the other way around.
But the important point here is that the male is identified as one, the female is identified as the other.
So you've got people in all these various groups.
I don't know.
Somebody can figure out how many groups there must be for the full design here.
Got people in all those groups.
And, now you ask a question, at the end of the whole experiment, do you think there's a sex difference, a systematic sex difference between men and women on this task?
What do you think the sex difference is, if there's a sex difference?
All these groups, all the same sex groups, on average, I don't think there's a difference.
Some people say, I think women are better.
Some people think men are overestimaters or whatever.
But, there's no systematic bias here.
Nothing systematic happens here.
Here something systematic happens, in this particular version of it.
You would declare that males as a group are overestimaters and females has a group are underestimaters.
What's the evidence for that?
One of each.
Clearly not meaningful.
There's no statistical reliability that you could glean from this.
It could be that brown haired people or blond haired people or something are different.
But you know that there's a group difference here, because males and females are definitely different groups.
As soon as you've got evidence that there's a difference across this group, you are willing to start making assumptions that difference applies to the group as a whole.
You see how that works?
More or less?
Somebody's nodding their head.
That's encouraging.
Ok.
So that's factor one-- that you're inclined to see your group as number one.
And this is a point we'll come back to, which is that you're inclined to see groups as having properties that pertain to that group.
And, you'll jump to that quickly.
Now the tendency to think that group identification tells you something about properties of the group, that's known as stereotyping.
If I know that you are part of this group, I believe I know something about you.
Period.
When we talk about stereotyping, we tend to think of it in negative terms.
That is a fairly self evident consequence of factor one mixed with factor two.
If you are inclined to think that your group is number one, and you're inclined to think that group identity tells you something, it follows that being a member of another group, means you're of a group that is, at best number two.
Ain't number one, that's my group.
So you in this group that I've identified, are in the some other lesser group on this scale.
That is fairly obvious.
What's a little less obvious, and is, at least worth mentioning, is that stereotypes are not just descriptions of things that are common that to that population.
Here's the silly example.
Let us consider the Asian women stereotype.
Is it part of the Asian woman, is bipedality the Asian woman stereotype?
No.
Nobody sits around and says all those Asian women have two feet.
That's stupid.
Nobody's going to say anything like that, because everybody's got two feet.
What's important in a stereotype is, stereotypes are different scores in the sense.
It's not necessarily accurate ones you understand.
They can be completely bogus.
But what defines a stereotype, is what somebody thinks differentiates one group from the population as a whole.
So I put some data down here from one big study of stereotypes.
Please note that these are not facts.
I mean they are facts in the sense that they are data.
They're not true facts about, in this case, the German population.
What they are is what this particular group of subjects reported believing about different groups.
In this case, the Germans.
I excerpted it from a huge study.
The factors, efficient, extremely nationalistic, scientific- minded, and pleasure- loving.
You will note that the largest single category, the highest value for the German population, in this collection of data points, is pleasure- loving, but that's not part of what would be considered the stereotypes here, because it's not higher then the population as a whole.
This particular group of subjects asserted that 82% of people in the world would be pleasure- loving, and a mere 72% of the Germans would be.
Therefore, pleasure- lovingness would not be considered part of that stereotype because it it's not something that distinguishes this view of Germans from the view of people as a whole.
And, so, efficient, yes.
Extremely nationalistic yes.
And interesting is something like scientifically- minded would be considered part of the stereotype, even though it's not even held to be a majority.
This perception doesn't say that a majority of Germans are scientifically- minded, but more Germans are scientifically- minded according to this view then the population as a whole.
Again, I have no idea what the true data would be for the German population.
I don't know if they think about science at all.
But, the perception is that the stereotype would include efficient, nationalistic, and scientific, and not pleasure- minded, because of this issue of a differential relationship to the perceived baseline.
To the population as a whole.
One of the factors contributing to the power of stereotyping is, what's known as the out group homogeneaity effect.
So, the in group is you.
You're in some group.
The out group are other people.
The out group homogeneaity effect is the tendency to think that all of them are kind of alike.
You don't think that about you in the same way.
From the last lecture, the nerdy high school freshman group.
If that was my in group, I would know that they aren't all alike, because I'm one of them, and I know that I'm different from all those other nerdy high school freshman.
But, the jocks, man, they're all alike.
It's an effect, an ignorance effect.
Now, how does this play into bias?
How does this ignorance factor play into not just thinking that all those people are alike, but the thinking less of those people?
If you ask about what you know about groups that you don't interact with much, where do you got your information about them?
You get your information from the news.
What gets you on to the news?
The fact that you're a good student, and you love your mother?
No.
But, if you decide to go off and commit an armed robbery, or something like that, that might get you on the news.
And, if you are a member of a distinctive group, of some sort, that's not my group, I'm going to say, hey, look, I've got a data point about, -- we can keep picking on the Asian women -- I've got a data point about Asian women.
That one committed an armed robbery.
I know about Asian women now.
They all commit armed robbery.
Well, you don't do anything quite that bald and stupid.
But that's the sense in which you are willing to color the entire group on the basis of whatever information you have about one member of it.
That's another version of this effect.
Is likely to lead to negative assessments of the out group, because the information you get about groups that you don't interact with is skewed towards the negative.
The stuff that's going to make the news about a group is going to be typically the negative effect.
So where are the strongest stereotypes in the American population -- well actually this is a somewhat old study at this point -- but when you assess, you can use various questionnaire assessments to assess how strongly a population holds stereotypic views of another population.
The heavily stereotyped views held by an American population -- this is now about 15 years ago-- were held of Turks, Arabs to a lesser degree of the Japanese, groups that were not heavily represented in the general Americans mix.
Less heavy stereotypes for groups with large immigrant populations in this country, because you were more likely to know some people in that group, and that makes the stereotypes less firm, less strong.
One of the reasons that these stereotypes matter is because along with being willing to build them quickly and easily, we are also inclined to think that the attributes that we put on other groups are causal.
At least in others.
This is what's known as the fundamental attribution error.
Let me explain a little.
But I sent a note to a couple of my social psych colleagues yesterday, as I was thinking about this lecture, asking who was it who named the fundamental attribution error, because that's a great thing to be able to do.
To be able to call what you work on fundamental.
And, it turns out to be Nisbett, N I S B E T T, from the University of Michigan if you want to track that down.
But, in any case, let me explain what this is.
There are two broad ways of thinking about personality.
We all have some notion that we've got a personality.
And, thinking about the attributes that make up that personality can be divided into two broad categories that map onto the usual nature/ nurture kind of arguments in the field.
There are trait theories that are typically on the more nature side of it.
There are fundamental attributes of personality maybe coming from a genetic origin.
You are who you are because you have these traits.
The alternative from the more nurtured side of things, the environmental side of it is, is a situationalist account that says you are who you are because of where you are.
Trait theory, you are here now because you where born smart, and hard working, and studious, or you basically have consistent over time, hardworking, studious attributes to you.
The situationlist account says you're sitting here right now emitting student behavior because you're in a student kind of environment.
If we put you on a farm, you would not be sitting there in the middle of the field with a notebook taking notes about the cow.
That's not what you'd be doing.
In that situation, you'd be doing farm kind of stuff.
Like all such debates in the field, the truth is going to lie somewhere in between.
There's going to be bits of both.
You're not going to get any mileage out of arguing strictly one or strictly the other.
What you think about this, what you think about the balance here, is important for the policy purposes, for instance.
Why did this guy commit a crime?
We know he committed a crime, because we just convicted him of committing this crime.
Why did he commit a crime?
Is it because he's a criminal sort of person?
Fundamentally, dishonest, nasty, kind of person.
Or, is it because he was in a situation that promoted criminal behavior?
Why does that matter?
Well if you're in this mode, you may send him off to prison in both cases.
In this case, you're likely to think of prison as a place where you put bad people to keep them out of the way for a length of time that's appropriate to whatever bad thing they did.
But, it's basically as a punishment.
This is the personality theory, that would lead you to call your prison a correctional institution.
Because you think that you could correct this person.
If you think he's fundamentally a bad person, the trick is to make sure he can't do that anymore.
If you think it's because of the situation, that somehow forced to him into or pushed him into criminal behavior, you want to fix that.
Modern prison philosophy is neither all the way here, or all the way there.
But, the balance is really a personality theory question.
The fundamental attribution error is a tendency to hold to a more trait theoretic position when you're talking about other people than when you're talking about yourself.
Why is it in error?
Well it logically can't be the case that you are the product largely of the situation and that they are a product of their invariant parts.
That over the population as a whole isn't going to isn't going to hold up.
Why did this guy rob the bank?
He robbed the bank, because he's a criminal.
Why did I rob the bank?
I robbed the bank, because I was hungry, and the door was unlocked, and I didn't really rob the bank, I just kind of picked up the money that was lying on the floor, and anyway, it was other guy.
Much more likely to give us situational account.
Why did you get a bad grade on the midterms.
Suppose you got a bad grade on the midterm.
Why did you get a bad grade on the midterm.
Well, the story was really lame, and it distracted me.
And I didn't get enough sleep.
And the course wasn't a big priority for me.
That's why I got it bad grade.
Your TA says-- well your TA is a good person, and doesn't say this-- but your TA looks at the exam and says, why did they get a bad grade on the exam?
They're stupid!
That would be the fundamental attribution error.
Why did your TA get a bad grade?
I didn't get enough sleep and stuff like that.
We're more inclined to give situationalist accounts of our own behavior and more inclined to get trait accounts of other people's behavior.
All right, so let's see where this has gotten us to.
We're inclined to put people into groups.
We're inclined to assign attributes to those groups.
We're likely to assign more negative attributes to groups than we ought to because our information about groups that we don't know is being skewed in that direction, and we're likely to attribute behavior to what we now perceive correctly or incorrectly as the traits of the group.
And, so you can see how you're going to end up with a story that is a pretty negative account of some out group or other.
Oh by the way, there's a very interesting wrinkle on the fundamental attribution error, or at least there's certainly used to be, I do not know whether this is still the case in the population as a whole, and I certainly don't know, though I would be very interested to know, whether it's true at MIT.
Let's try this as an experiment, and see how your intuition goes -- OK we can do the trait versus situational thing, and we're going to have a math test.
You're interpreting your own score on a math test.
That score can be good or bad, as you may know.
There are explanations of why the test for was good.
A trait theory would be, I'm a genius.
A situationalist theory would be, I'm lucky.
And, if the test is bad, you can have an assessment that it says I'm dumb, or you going to have an assessment that said, that the test is unfair.
The interesting bit is that one gender is more likely to say this.
And the other gender is more likely to say this.
One gender is more likely to say this.
And the other gender's more likely to say that.
So each of these cells can be associated with a gender.
So I got a good grade on the test.
I'm a genius.
Who are we talking about?
Men
All right.
And, it follows that this must be the female cell.
I got a bad grade on the test.
I'm dumb
Female
And, that is the historical finding.
These are studies that came out in the early days of the women's movement.
And, I don't know about whether or not it's still pertains.
But the disturbing finding at the time was that males were inclined to give trait theoretic answers for the good stuff, I'm good, I'm bright, I'm gorgeous.
And, females were likely to say I'm lucky and it's all makeup, or something like that.
And on the other side, on the bad side, the females were likely to give trait theoretic answers.
I'm dumb and ugly and depressed and its terrible.
And the guys were likely to say I'm brilliant, I'm gorgeous, et cetera, and the test was really kind unfair and my teacher hated me and stuff.
It would be interesting to know whether that was still true.
We might as well take a poll.
How many people think that if we actually collected data, we'd find that something like that was still true?
How many think that we would find it has gone away?
I have no idea if there's new data on that.
The basic point is that with that modulation possibly we tend to see situational explanations of our own behavior, and trait explanations of other people's behavior.
Now let's take a look at this last factor, which I'm calling the role of the ignorance in person perception.
And see how that can lead to what looks like a biased outcome, perhaps even if you didn't have these other factors.
This ignorance factor's now going to interact with these other factors to make biased outcomes.
It's quite easy to come by.
So, let's do a version of the classic physics joke, about assume the horse is a sphere.
We're going to over simplify the situation.
The issue here is who are you going to be friends with?
Well, first of all, we have to go back to the good, ernest high school discussion about making friends with people.
And, does it matter if they're wearing the latest designer whatever?
And, the answer, of course, is no, because you shouldn't judge a book by its cover.
Good.
Nice cliche.
And, it is, of course, true.
In the first assume the horse is a sphere over simplification, the problem, let us assume that the set of people with whom you might be friends in the world, is the set of let's just make it all MIT undergraduates.
And we know we don't want to judge books by covers, so, therefore, what you're going to do is, set up in depth interviews with everybody.
And decide who's going to be your friend on the basis of that.
No.
That's not going to work.
Well, all right, the other alternative is, don't want to judge books by their cover, therefore I won't talk to anybody ever again.
That's not going to work either.
So it is self- evident again, various bits of this are self- evident, you've gotta make snap decisions on the basis of imperfect information.
It doesn't mean that you have to make your decisions on the basis of whether or not they're wearing designer whatevers, of course.
But you're necessarily going to have to make a first cut through the population on the basis of essentially superficial information.
Well, what's that going to do?
Let us assume that in the world, in an act of massive, further over simplification, there are bad people and are good people.
And, your job is to divide the world into those two categories.
So you're going to make an assessment.
You're going to perform an act of person perception, and divide the world into bad people, and good people.
We've got a nice simple two buy two design here.
Ideally you want everybody to be in those two populations, and in those two cells.
Even if you could do in depth interviews with everybody, it's not clear you'd never make a mistake, but clearly, if you're just going to be basing your decisions on relatively superficial information, there are going to be errors.
I labeled these Type 1 and Type 2, which is actually jargon from signal detection land, but don't worry about that.
It just, in this case, give us a chance to ask the question.
Which of these type of errors is worse?
If you had a choice about which error to make, how many people, given the choice between, let's be clear about this, you've got a good person, and you declare that good person to be bad, or you've got a bad person, and you declare him to be good.
You got a choice between which of those errors you're going to make.
How many people vote for Type 1?
How many people vote for Type 2?
OK.
So a Type 2 person.
Why do you you prefer the Type 2 error?
Any Type 2 person?
[INAUDIBLE]
You're missing out on good people.
That's the nice person answer.
You don't want to inadvertently tar some nice, good person with the label of being bad.
How about a Type 1 person?
[INAUDIBLE]
There's a lot of people who can be your friends.
You don't need all of them.
And you want to get to those bad people.
How come?
Anybody else?
It could be harmful
Yeah.
it could be dangerous.
If we dichotomise this into good and really bad, nasty, dangerous people, then that intuition becomes a little clearer that you whatever you else you do, you want to keep these people away.
This is applied signal detection theory.
Usually you do signal detection theory in visual perception land or something.
But this is what the next page of the handout has on it.
But here's where this is coming from.
Let us suppose, again, for the sake of vast over- simplification, that there are on a scale of goodness, that the only two types of people in the world.
There are good people and bad people.
And, you want to pick the good people and reject the bad people.
But the difficulty is that you can't successfully see this, because your information is lousy.
The effect of your information being lousy is that what you see is a distribution of goodness and badness, something like that.
By the way, if you were doing this in vision land, this would be one light and another light.
Can you tell the difference between a dim light and a bright light or something like that.
By the time it goes through your nervous system, rather than being always looking exactly like this, or always looking exactly like this, sometimes the bright light looks a little dim, sometimes the dim light looks a little bright.
And how do you decide which one you've seen?
So, you got a person in front of you.
How can you decide whether they're good or bad?
Well, the best you can do, is put a criterion in there somewhere.
So let's just divide it.
If I do that, that's OK. That means I'm going to declare everybody on this side to be good, and everybody on this side to be bad.
And, OK, so I'm declaring all of these people, who are, in fact, good to be good.
The difficulty, the sad thing, is the good people who I'm declaring to be bad, so the Type 1 errors are here.
These are good people who I declared to be bad.
That's too bad.
OK.
Here, on this side are all the bad people who I declared to be bad.
That's exactly what I wanted to do.
But these guys, these are the Type 2 errors.
These are bad people who beat my criterion level, and I said they're good.
Now, you should be able to figure out, looking at a picture like this, there's no way to eliminate error.
If this is the situation of the stimuli I have to deal with, there's no way to eliminate error.
All I can do is apportion error.
So, if I decide that the Type 2 errors are the dangerous errors that I need to avoid, then I'm going to move my criterion over.
This is the second picture on the handout.
If I move my criterion over so that I reduce my Type 2 errors to just these few, let's say, now the result is that I've massively increased my Type 1 errors.
I'm now declaring all these lovely people to be people I don't want to make friends with.
It's sad, but that's the way it go.
Because, as the gentleman back there said, yeah.
There are a lot of people here, so I've got plenty of people to be friends with.
And these guys will just have to deal with the fact that they're not my friend.
That's just the way it is.
But, I'm not letting any of these mean, nasty, rotten people in, except for these guys.
Most of these guys, I'm going to avoid.
OK. Now, look what happens when you deal with an out group.
When you deal with a group other than your own.
If the argument is that part of what makes an out group the out group is the fact that you know less about them.
The way to express that in these sort of signal detection terms, is as an increase in the noise.
An increase in the spread of these distributions.
So what that's going to end up looking like is you still have the good people and the bad people.
But now your perception is less accurate.
OK. That'll do.
So, they now just overlap more, because we just don't know as much about these people.
You're not as good at -- you want a trivial example of this?
Let's take wolves, there's an out group you know, the ones with the big sharp, you know, grandma what big teeth you've got, kind of wolves.
Maybe there's a good wolf out there somewhere.
A nice wolf.
You know the kind of wolf that we're supposed to have adopted back in antiquity to make into dogs eventually.
But you meet a wolf on the street, and you don't know much about him.
Where should you draw your threshold before bringing him home to play with your six year old?
You're going to draw your threshold out here somewhere, right?
I don't care if I reject the one nice wolf.
It's just really risky to bring wolves home.
And, that's because you're just really, really ignorant about wolves, you just don't know much about them.
Maybe if you knew wolves better, you'd know who the nice ones were.
All right.
With human populations it's obviously much less dramatic than that.
But you don't know much about these other people, so the distributions theoretically overlap more.
You still only want to make very few errors where you let bad people in next you.
So that's going to cause you to move that threshold still further over in this direction.
Not because you don't like these people.
Understand that there's no explicit bias going on here.
You're just being cautious in this in this story.
You can get explicit bias out of those the first three factors.
But here, this factor has no explicit bias in it at all.
Just ignorance.
So now you say oh good I'm only letting this percentage of really bad people and now obviously there's a little problem here.
Your Type 1 where you reject good people.
Now you have rejected almost the entire population of this other group.
You know that you're not biased in your heart of hearts, because you can still say, as it says on the handouts, this little tale of the distribution some of my best friends are X.
Whatever that out group is.
Some of my best friends are white, black, Christian, Jewish, whatever the other out group is you're dealing with.
This signal detection story will get you there with some people in the group who are fine, because they beat your threshold, and the vast bulk of the rest of it, who disappear because you're applying the same caution to an out group, that you were applying to the group that you knew something about.
So, that's how ignorance can end up being a factor in producing what looks like biased behavior.
If you compare these two, you'd have to say I'm biased against this group.
Because 100 of these people, I'm only letting five of them be my friend.
100 of these people I'm letting 60 of them be my friend.
That's a biased outcome from no explicit bias.
Now this question of explicit versus either no bias or implicit bias is an interesting one.
You don't necessarily have a clear idea of the biases that you may have.
One of the more interesting and more disturbing bits -- there's a thing called an implicit attitude test, the IAT.
If you want to try this out on yourself, go to www.prejudice.com I think is the right site.
That is one site.
But if that fails go and find your way to the website mosuronbanashi at Harvard.
She is one of the leading practitioners of this, and her website will link you to a place where you can try this out on yourself.
As the website will tell you, before forewarned.
You may find the results of this experiment to be disturbing to you.
But, it's well worth trying out yourself.
Now what is this experiment about?
This experiment is, in effect, a version of a Stroop interference test.
The classic stroop interference experiment is an experiment where what you do is you see a collection of words and your job is, whatever, the word says, it's just to tell me what color the ink is that the word is written in.
So if I write cat in red ink, you say red.
And, if I write dog, in blue ink, you say, dog.
No.
You say blue.
That's a different interference.
The problem is, that if I write red in blue ink, some people will simply make the mistake of saying red, and everybody on average, will be substantially slowed down, because of an inability to suppress that response.
And if I do red in red ink they'll be speeded up.
So, if the two sources conflict with each other, you're slowed.
If the two sources agree with each other you're speeded.
OK.
So, here's what you do in an IAT experiment.
What you do, is you tell people, I'm going to show you some words.
And, if they're good words, they're nice words, you push this button.
And, if they're nasty words, you push this button.
OK. No problem.
So nice boink.
Evil.
Boink.
Pain.
Boink.
And so on.
Not very tough.
OK. Second task.
I'm going to show you some pictures of people, if it's an old person, I want you to push one button, if it's a young person, I want you to push another button.
So we put me up there, boink.
Put you up there, boink.
Now what we do, is we do mixed blocks.
Where you're going to see words and pictures together.
And, I'm going to tell you, OK, now if you see a nice word, or an old face, push this button, if you see a nasty word, or young face, push this button.
It's just the tasks on top of each other.
Whatever we do in this regard, you'll be slower, because now you have to keep two rules in mind at once, but what's striking, is it if you do nice and old, you're significantly slower than if you do nice and young.
Its as if nice and old maps the same response.
It causes a conflict that doesn't work for you.
And nice and young does.
As if you've got a biased in favor of young over old.
If we ask you to why is this an implicit attitude test if we give you an explicit attitude test that says what's your attitude towards young and old people.
You may perfectly well report I love old people.
I love young people.
I love everybody.
But, you'll still come up with this result.
I deliberately did this one, because it doesn't tend to carry an awful lot emotional loading for people.
But the reason this may be a disturbing test for people to take -- you should still go off and do it-- is that if I do nice and black, African American pictures and nasty and white, regardless of your explicit report, of what you consider your bias to be, the white population in this country, will, on average, have slower reaction time to nice, black pairings then to a pairing of nice and white.
Actually, one of the more interesting and depressing findings, is that doesn't even completely reverse with an African American population of subjects.
In the African American population, the last time I checked on the data, the pairings were roughly equal.
So, the African American population has presumably some ethnocentric biased in its own favor, an implicit bias in its own favor, but that's counter balanced, by some incorporation of overall bias against, and so they come out is roughly equal.
The debates and the literature about this line of work is this talking about implicit attitudes which is a notion that regardless of what we think, we're all racists or something like that.
Or, is this just say that we have somehow incorporated that we know, at some level, the biases of the culture, as a whole, even though, we, ourselves, may not be biased.
It is an interesting question beyond the scope of what I can talk about today, whether there is a serious difference between those two.
But, the disturbing finding is, -- and you can do this -- it's not black, white, old, young, by no means is the limit on this game.
Once you've got the methodology, you can do it on anything.
So, shortly after September 11, they started doing these tests on opinions about nice -- and you don't need to do with faces either, so if you want to do Arab versus non- Arab, you can just do it with names.
So, you do Abdul, and Mohammed and so on, and then you do Chris, and Jane.
And, you find that in the American population, at the moment, nice and Mohammed is slower then nice and Robert.
It's not surprising, but, it is, nevertheless, disturbing how easy it is to show these effects.
They're robust.
They show up across populations, and they show up fairly independent of what people report.
It doesn't matter if you explicitly, and let's assume that people are reporting honestly.
It doesn't matter if you explicitly have the bias.
You can show something that looks like a bias with a test of this sort.
Anyway, give it a try.
It's interesting and disturbing.
And the interesting and disturbing department, I will continue in a minute or two talking about the link between attitudes and actual behavior.
So, this is a good place for a short break.
So, look.
Bias.
We can presumably agree that bias isn't a good thing, but if it stays in the realm of private opinion, or if it stays in the realm of implicit opinion that you don't even know what you have yourself, it's not exactly front page news, but we also all know from reading the front page, that there are regrettably frequent occcasions where one group is willing to slaughter another group based on very little more, if anything more, than group identity.
So, an absolutely critical question, is how can people be moved from attitude to action?
And, the disturbing aspect of what we know from experimental psych about this, is that it is surprisingly easy to have your behavior controlled by outside forces.
The experimental work, of course, cannot get people to go out and slaughter each other.
Nothing like that would be even faintly moral, but rather like the Clay and Kandinsky experiments.
You can do experiments that show that it's surprisingly easy to manipulate the situation in ways that changes behavior in ways, that changes behavior in ways that look at least a little bit disturbing.
One of the classics, back in the '50's, that gets this literature going, was done by Ash at Columbia at the time.
I think that the picture is still in the book.
A marvelous collection of male, Columbian nerds from the '50's.
Anybody read the chapter yet, and happen to know if it's there?
Here's the basic experiment, here you come into this experiment, and you're doing an experiment on visual perception.
Ash shows you a card with three lines on it.
Your job is to say which line is longer.
Now, the odd thing about this, well, you're not an experimentalist, so why should you care?
But it's a little odd, as an experiment to be doing this in a group.
But it turns out, you're doing this in a group.
And, so Ash holds up the card, and you say B, and you say B, and you say B, and everybody said B.
Next card.
I'm not going to bother changing my cards, but you get the basic idea.
On the critical trial, up comes this card.
He says, C. He says C. He says C. She says C. C. C. Now, we're up to -- I stuck with her because, she's got glasses.
Because the nerdy Columbia guy has glasses on too.
What does she do?
Well, why did all these guys say C?
Are some kind of morons?
They're all confederates of the experimenter.
The only real subject is this person.
And, the question is, does she say C?
The answer is about a third of the time in the original Ash experiment, the answer's yes, she says C. It is completely clear that even when she doesn't say C, she's uncomfortable.
This is an experiment on peer pressure.
And, it's perfectly clear that when everybody else is saying C, she's busy taking off her glasses, and checking them, and stuff like that, to see what's going on here.
There's something wrong.
What do you think manipulates -- the standard result is you got about a third of the people complying with the pressure.
What reduces that compliance?
How [INAUDIBLE]
Yeah, sure.
Presumably it's hard to get the result.
But, OK if we keep the physical stimuli the same.
Yes
[INAUDIBLE]
If somebody picks A, and it just looks noisy.
I don't know if they ever did that particular manipulation.
That's an interesting question.
That might change things
If they have six or seven people, [INAUDIBLE]
It doesn't take any support.
You probably know this from arguments with groups of people or something.
It's hard to be the first person to voice the minority view.
It's much easier to be the second person.
I think I actually have the data from that.
Yeah.
One supporter.
One supporter in the group drops compliance from one third to 1/12.
So, big drop in the amount of compliance.
The more people who say C, the more likely you are to comply.
The smaller the group, the less likely you are to comply.
But, the point is, that even in a matter as seemingly straight forward as which line is longer, you can feel that pressure from others.
The most famous experiment in this canon is an experiment done by Stanley Milgram.
And, in that experiment, here's the set up.
You come into the lab for a study on learning.
The effects of punishment on learning.
And, there are two of you.
And, one of you is going to be the learner and one of you is going to be the teacher.
OK. And we're going to decide this randomly.
Flip a coin.
You're going to be the teacher today.
Now, in fact, this is not random.
The subject is always the teacher.
The learner is always a stooge of the experiementer and here's what happens.
You're told you going to do some sort of a task, and your job, as the teacher, is to give the learner a shock every time they make a mistake.
This was done back in the '60's with this great big hunk of electrical equipment, with a gazillion switches on there, running from 15 volts to, I think 450 volts, in 15 volt increments, with instructive little labels like mild, and then, up here somewhere is severe, and by the time, you get up here, it's labeled something like XXX.
The rule is every time you make a mistake, you increase the voltage.
Now, we will give you a 45 volt shock.
You, the teacher, gets a 45 volt shock, just to see what it's like.
And, a 45 volt shock from this apparatus is mildly unpleasant.
It's nothing you'd want to sign up for.
It's not going to kill ya, though the suggestion is that this might.
And, that's the rule of the game.
Now, before doing the experiment Milgram went and asked everybody under the sun, what the result would be.
Well, the answer is that everybody under the sun will bail out pretty early here.
Nobody's going to get them very massive shocks.
He asked theologians, he asked psychologists, he ask people off the street, and everybody agreed this is not going to lead to much in the way of shocks.
And, this made the result of the first experimental a little surprising.
Everybody went all the way through to 450 volts.
The entire population of subjects in the first study went through to 450 volts.
Now, were they with a thrilled about this?
No.
It was also absolutely clear that the subjects were very uncomfortable about this.
And, that they questioned whether they should do this.
And, Milgram had an absolutely stereotypical, he'd prepared, in advance, the response.
Please continue, the experiment must go on.
And, that was it.
You were free to leave.
Though he didn't explain this to you in great detail.
But if you said, should I do this?
He said, please continue.
The experiment must go on.
And people did.
Now, he was a little surprised.
In the original version, the alleged learner had been taken out and put in a different room.
And Milgram figured, well, look.
Maybe these people just didn't believe the set up story here.
So, they moved the alleged learner to a position where he was visible and making noises about this.
He's vigorously protesting as the voltage gets larger.
At some point, up here, he says he's not going to respond anymore.
So, the experiment's rigged, so that he keeps making mistakes as a result.
No response is considered an error.
And Milgram is there saying please continue, the experiment must go on.
Oh.
He also makes useful comments like, I have a heart condition, and stuff like that.
It's pretty vivid stuff.
So, what happens in that case?
Well.
Great.
No longer do 100% of the subjects go all the way through to the end.
Only 2/3 of them go all the way through with a subject who has stopped responding, and has announced for all you know you're killing this guy, perhaps.
This is pretty disturbing kind of stuff.
It was very disturbing to the the subjects.
Actually, this is one of the experiments that produces the need for informed consent in experimental psychology.
You didn't just go and hijack people off the street and say, you have to be in my experiment.
These people were volunteers.
But, there was no sort of consent process the way we would have today.
And, the level of distress produced in these subjects was part of what drove the field to require informed consent.
Why was the experiment done at all?
This is an experiment done in the '60's, less than a generation after World War II.
And, a question that had obsessed social psychology, since World War II, was how could the Nazi atrocities have happened?
Who were the people who went and killed millions of other people?
Not the soldiers, but the people who killed six million Jews, and I don't remember, a million and a half gypsies, and some large number of gays and so on.
Who were these people?
There was a theory out there that encapsulated in a book called The Authoritarian Personality which among other things, fed into stereotypes about the Germans which was that there was a certain type of person, who was willing -- the Nuremberg trials, the war crime trials after the war, had produced over and over again, the line, "I was just following orders".
And, the notion was, well there are some people who are just good it that.
They just follow orders.
And, the rest of us were not like that.
Milgram suspected that was not the case.
Milgram suspected that the answer was that under the right situation, many people could be pushed into acts, that they would objectively think were impermissible.
This is his effort to get at that question.
If this sounds like current events to you, every social psychologist with half a credential was on the news after the Abu Ghraib Prison Scandals earlier in the year, because it just sounded so much like this sort of problem again.
The people who were charged, many of them responded with the response, that they were simply if not following orders, following the implicit instructions that they felt around them.
You got to endless articles in the papers, saying, Oh, you know, so and so was just like everybody else back home.
I don't understand how he, I don't understand how she, could end up in these pictures doing things that any reasonable person would say are unacceptable in a in a military prison situation.
It's the same kind of question, different scale of magnitude, of course, to the Nazi atrocities.
But it's the same question.
How do people end up doing things like this?
Before attempting to answer, which I can see is going to run into my next lecture, let me tell you about a different experiment designed to get at the same question.
This is an experiment where you think you're in a study, sort of a consumer relations kind of study, that you might run into it the shopping mall.
They've set up the experiment at a shopping mall.
And, the cover story is this.
We're doing some research on community values.
Because we're basically doing investigations for a legal case.
Here's the situation.
This guy was living with a woman to whom he's not married.
His employer found out about it, and fired him.
This has gone to court, and the court case hinges on community standards.
If community standards are that living in sin, out of wedlock, is a bad, bad thing, then it's OK to fire him.
Otherwise not.
So we've gotta find out what the community standards are.
Let's all have a discussion.
So, I've told you the story.
I've got the cameras rolling here.
I, the experimenter, I'm going to step out and you guys discuss.
OK. You guys discuss.
That's great.
Now, I come back and there's a group of ten of you or something discussing this.
I come back in, and I say, that was great.
Thank you very much.
I know what you believe, because I've been watching this.
But, I want you, you and you, please to argue from the point of view, that the guy should be fired.
I know it's not what you really believe.
But just argue for that.
OK. Comes back in.
OK. Now I want you, you, and you to join that argument arguing that the guy should be fired.
Fine.
And then in the penultimate step, is I want everybody to have a chance to look into the camera, and say why the guy should be fired.
I know you don't believe.
But, just give me a little speech as if you believed it.
OK.
Cool.
OK. Last step.
Here's this statement that says that I can use my videotape in any fashion I wish including submitting it as evidence in court.
Will you please sign.
I'll be back in a minute.
I gotta go rinse a few things out.
But you guys decide.
The question is do people sign?
I mean this is obviously basically asking you to perjure yourself if that wasn't what you believed.
Now this was a huge experimental design originally.
They were crossing a number of people with gender and they originally had a plan for, well I can't remember, a gazillion groups.
They called off the experiment early, because like the Ash experiment and like the Milgram experiment, this experiment produced extremely strong feelings in their subjects.
They had gotten a form of informed consent, which I can't go into now, but even with that, it felt unethical to them to keep going.
So, they had 33 groups.
It was a busted design.
But they had 33 groups.
Of those 33 groups, of those 33 groups, how many did the Milgram thing, went all the way and everybody sign?
Do you think?
The answer is one.
I think.
One group got total obedience.
16 of the groups unanimous refusal to sign.
Nine groups got majority refusal to sign.
So, in this experiment, compliance was much, much lower.
And, the question that I'll take up next time, for the start of the next lecture, is what's the difference between the Milgram experiment -- let me say one last thing about the Milgram experiment.
This is not an isolated result -- Milgram's at Yale.
It's not just an isolated experiment that happens in New Haven in 1960.
Replicated all over the place.
Doesn't matter in America.
Doesn't matter age, sex of the subjects.
It replicates beautifully.
Why does this work, and the other experiment didn't?
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license at MIT OpenCourseWare in general is available at ocw.mit.edu
Good afternoon.
I decided to go for a little amplification today because I'm feeling a little underpowered and I think that if I do my usual unamplified version that sometime around 3 o'clock I'll fall over and somebody will have to haul me away.
Actually, a lot of the experiments I'm going to talk about today suggest that you wouldn't do it.
You'd just leave me in a little heap there and slowly file out and I could try that as like a demo.
But that would be sad.
So I don't think I'll do that, but I will amplify myself in the effort not to run out of steam here.
When last we met I had setup the Asch experiment-- that's the 3 line thing, the Milgram experiment where it turned out that people would shock another apparently innocent individual to a pretty severe degree without what looked like an awful lot of provocation.
And then the manufacturers, whatever it was, consultation-- whatever it says on the handout, that experiment suggested that under other circumstances people wouldn't be so willing to comply with a suggestion that would run against what they would normally do.
And I left it there to say that we'll come back today and talk about why that must be.
Why that might be the case.
And the context in which I want to talk about that is the social impact theory.
The source I put on the handout out is-- the guy's got a great name-- his name is Bibb Latane.
It looks like Latane, but you pronounce it Latane and the B stands for Bibb.
I don't know who names their kid Bibb, but maybe that's how you grow up to be a famous social psychologist.
So when you're studying for the final you can remind yourself that social impact is a function of SIN, but that's just the acronym.
I don't know that Latane every particularly noticed this, but that's the acronym for his 3 big factors in this theory of strength, immediacy, and number.
And what I want to do is try to explain those in the context, particularly at the Milgram experiment.
See how those speak to the question of when it is that you will give into the social pressures from the outside.
So strength in this case refers to-- well, in the Milgram case really, it would be the authority of the guy giving you the instructions.
Remember that all the subjects in Milgram's experiment were upset by the notion that they should be shocking somebody.
And shocking them apparently, severely.
And Milgram had trained himself to say, please continue.
The experiment must go on.
And not more than that.
He wasn't going to get into an argument with them, he was just going to say that.
If the guy's saying, please continue.
The experiment must go on is Stanley Milgram, white-jacketed scientist professor from Yale, you got 63% of people going through all the way to 450 volts in that experiment.
But if you made a small change in the experiment and the person running it was just some guy.
You know, Kristen [UNINTELLIGIBLE] well-intentioned research assistant in the lab or something like that, compliance dropped to 20%.
I don't know.
I'm sure actually if Kristen was doing it, compliance would be like 100%.
Because she'd smack them around and stuff like that, but if it was not an authority figure, if the person giving you the instruction was a person with less authority compliance dropped way down.
That is the sort of thing that the military knows, right?
Who gives you the order makes a difference.
Immediacy refers to the vividness of the factors.
Sort of the salience of the factors here.
So I already talked about one immediacy manipulation.
Remember, in the original version of the experiment 100% of the people went all the way through to 450 volts.
Milgram figured, oh my goodness.
That's because they didn't buy the story at all and he brought the learner slash victim into sight and into the range of hearing and that change in the immediacy-- increasing the vividness of the impact you were having on this other person decreased compliance from 100 to about 2/3 of the population.
If you manipulate the vividness of the pressure on the subject, compliance can change.
So if Milgram is right there saying, please continue.
The experiment must go on.
You went on.
But if you said, look, I got a rinse a few things out.
I'll be down in my office, if you got any questions about the experiment call me.
So you know, I have a heart attack.
You know, the learner is busy making all of its bad noises.
The subject is getting antsy.
He picks up the phone.
Please continue.
The experiment must go on.
Yeah, right.
Compliance dropped again to 20%.
So big drop in compliance if the source of the authority was not right there to help you out.
And if you make the consequences even more vivid, you can again drop compliance in this experiment.
So in one particularly vivid version of Milgram's experiment, Milgram didn't have the learner off in another booth.
He had the learner right next to you making his noises.
And at a critical time in the experiment the electrode is slipping off the guy's hand and Milgram says, hold the electrode on him.
And he's still doing the same thing, right?
[UNINTELLIGIBLE] I'm not going to respond anymore.
I quit.
[UNINTELLIGIBLE] Compliance drops, you'll be glad to know.
What you may be a little distressed to know is it drops only to 30%.
30% of the people were still willing to deliver these punitive shocks all the way through to the end even when they're holding the guy's hands down on the electrode.
It's this immediacy factor that makes it psychologically easier to shoot a missile over the horizon and kill somebody than to actually look somebody in the eye and kill them.
They're dead the same way in either case, you may know objectively that they're dead.
This is a real issue for the military.
The military knows that it is in the unfortunate business of killing other people and it also knows that in a way, much more profound of course than the Milgram experiment that this is very upsetting to its own soldiers.
To say nothing of the people who get killed, but people don't like killing other people, typically.
It takes work to get somebody to kill other people and it's psychologically less damaging if it's over the horizon than if it's face-to-face.
Though, all you need to do is watch the latest video of some charming person slitting somebody's throat-- some hostages throat in Iraq-- to realize that it is also perfectly possible to kill somebody right there.
But there is an immediacy effect, a real world immediacy effect.
Well, that's perhaps a gruesome one then and you might think this is only stuff that works in the realm of the truly horrible.
That's not true either.
Suppose I get curious this evening and I send a note around saying, what do you think of the Milgram experiment?
I might get an e-mail back.
I wouldn't bet on it.
Maybe a couple.
I know who would be sending them too by this point in the term.
If I ask right now, well, what do you think about the Milgram experiment?
Is that a disturbing experiment?
Few people will nod their heads.
The eastern bloc might raise its hand.
But you know, most of you will successfully sit on your hands and not do anything.
But you know, if I was to wander over here somewhere-- she's getting very nervous here.
You can see the experiment really works.
And if I was to ask, what do you think of the Milgram experiment?
I won't actually make her respond, but I could, right?
There's no doubt.
The only person capable of not responding under those circumstances would be my 15 year-old son, right?
You have perfected this within the realm of your own family, why did you get that grade?
But under other circumstances the social impact, the social pressure there is so strong by the time it's all directed at you that you got to respond.
You can feel that social pressure.
Actually, it was a great experiment.
This was a great experiment because several people thought I might be headed their way.
And I caught these great fleeting looks of relief from that side of the aisle when they realized, he's going that way.
That's immediacy.
That's also actually, an illustration of the effect of this third factor, which is number.
In the Milgram experiment this was not manipulated.
Milgram experiment is a one-on-one experiment.
There's Milgram or his surrogate exerting pressure on you, the subject.
In that manufacturers consultants' study or in the Asch experiment, it is manipulated and you find that the more people that the authority is trying to spread his impact over, the less the impact on any given individual.
Similarly here, if I ask you, 300 of you, what do you think?
Each of you gets-- I don't know if this is actually a linear operation-- but roughly, 1/300 of my impact.
But if I go and find her she gets all the impact and she has to say something.
Actually, I think, that we could probably work out the equations here because there's also probably a distance equation, right?
You know there are people, you out there in the cheap seats know there are people who sit out in the back because the possibilities of social impact are less out there.
I can do what I want out there in the cheap seats.
He can't see anything anyway.
Look at all those computers up there, they're all surfing the web anyway.
But I don't know that anybody's ever looked at the distance effects in any sort of a direct way.
There's a very nice study done at Ohio State some years ago during Greek Week.
What the experimenters did was they hung out backstage because at Greek Week at Ohio State what you had to do was your fraternity had to get up in front of all these people, like a big stadium full of people and do something.
Typically, something kind of embarrassing to show how cool you were and why everybody should join your fraternity.
And what the experimenters noticed was that the number of guys in the group varied.
You know, you sent out your one guy or you sent out this troop of 10 guys to do something.
And so what they did was they simply asked people before they got on stage, how nervous are you?
And the answer was, it was monotonically related to the number of people.
If it was just you, you felt much more nervous than if there were 10 guys going out there.
Now there's no real rational basis for that.
You're not going to get hurt out there, but if you imagine that everybody in that stadium had a tomato you can kind of understand the social impact version of this.
How many tomatoes are going to hit you if there's just one of you verses if there's 10 of you.
It's going to diffuse in this case, very literal sort of impact.
Social impact theory has one of its roots actually in a lurid and media- circus murder from the 60's.
In fact, a murder that I remember because it happened in New York City when I was growing up in the New Jersey suburbs and so it got a lot of coverage.
This was the murder of a woman named Kitty Genovese, who was murdered in the courtyard of her fairly large apartment block.
And it wasn't a quick business.
The guy stabbed her and she sort of crawled off into the bushes and she didn't die and he came back and he finished.
It was gross.
You know, just disgusting murder.
And the reason gross disgusting murders I mean, there are whole New York radio stations devoted to that.
The reason this was a particularly salient one was it was clear afterwards that there were at least 40 or so people who had heard this and nobody had called the police, let alone intervene directly.
Nobody had called and this generated a whole lot of sort of media brow beating and stuff about, was it the case that New Yorkers were just cold, heartless individuals?
Or what?
And rather like the Milgram experiment being motivated by the question of, were the Nazis special in some evil sense?
Some of this work on social impact theory was based on the hypothesis that no, that this didn't illustrate the cruel, heartless nature of New Yorkers.
What it illustrated was what happens when you're on the losing end up of what boils down to a social impact experiment.
The problem was that she was diffusing her impact across this whole apartment building and when you interviewed people, people said over and over again, I thought somebody else would have called.
I wasn't quite sure what was happening.
Somebody else would have had a better look.
They figured that somebody else would have taken care of it.
The notion is that had this been taking place in the front yard of your isolated farmstead on the Great Plains or something like that, you would've done something.
Because you would have known nobody else could have.
Is there any evidence for this?
Well, yes actually.
John Darley, who was one of my professors at Princeton, and Latane, who worked with him, did a series of beautiful experiments on this.
Obviously they didn't go around murdering people in courtyards, but what they did was you'd come into the lab and lab had a whole bunch of isolation booths in it.
Because except if you're in a weird experiment like that Asch three line experiment you want to have people doing the experiment alone.
So a lot of labs are setup this way and so, you, the subject, would go into this little booth to do the experiment.
And then something would happen.
For instance, you'd hear crash and help.
And the question was, who cares what you're doing in the booth?
The only interesting question in this experiment is, how fast do you come out of the booth to help?
The answer is it depends on how many other booths you think are occupied.
And of course, Darley and Latane made very sure that you knew exactly how many booths were occupied.
So if it was the case that you knew that you were all alone you were out there fast because the impact was on you.
If there was going to be any help around it was going to be you.
If the booth was full-- if you knew that there were six other people in the experiment you were slower to come out because well, I'm not the biggest, strongest guy.
Somebody else will help and I didn't take that Red Cross course or whatever.
The responsibility gets diffused across multiple people.
It's not just for helping.
In another one of their entertaining versions, smoke comes into the lab.
How fast do you get out of there?
If you're alone, you're moving.
If you're not alone, you're thinking, he's not moving.
He must know something I don't know.
And if he's not moving it's going to be embarrassing, right?
If I go, fire!
Fire!
And I go running around.
Everybody's going to think I'm a complete doofus.
Now the downside, this is a beautiful experiment showing beautiful social impact effect-- the downside of this was this was done at Princeton where I eventually showed up as an undergraduate to do my nice boring research on visual perception.
None of my subjects ever believed that I was doing a visual perception experiment.
It didn't help that the room I had was one of these rooms with a half silvered mirror in it.
So that I could look in and they couldn't look out.
I didn't care I was planning to look in, it was left over from the social psych guys.
But nobody ever believed I was there to do a regular just straight up and down-- I just want to know what color it looks like experiment.
They all figured, you're watching me, right?
Something weird's going to happen here.
Oh, along those lines, didn't happen to me, but a word to the wise should you ever find yourself in an experiment in one of these rooms that looks like one wall is mirrored.
If you're in a psych lab it's probably a half silvered mirror.
The implication of a half silvered mirror is that that's not a mirror that you want to use to readjust large parts of your anatomy.
It was a great story.
The poor grad student was embarrassed for life.
He was a very, kind of straight up and down squarish kind of guy and anyway, it was very embarrassing.
The grad student was the observer here and it was embarrassing.
So don't do that.
You don't want to embarrass people.
Oh, one last example, closer to a real world example of this, the great I think, it's another Darley and Latane, liquor store experiment.
You go into a liquor store.
You're hanging out in the liquor store.
These two guys come in, they ask the guy in the liquor store, do you have any [UNINTELLIGIBLE] weird beer, some weird beer.
The liquor store guy says, I don't have it up here.
I might have some in the basement and he takes off for the basement.
As soon as he's gone the two guys look at the case of Bud, say, haha, he'll never miss this.
Pick up the case of Bud and leave.
The guy comes up from the basement and asks, where are those guys?
And the question is, everybody's a stooge in this experiment of course except you.
The question is, what do you say?
The answer is, if you are all alone 2/3 of you say, the guys-- they like stole the beer.
But if you're with even one other person that drops to 1/3.
Again, because of this diffusion of responsibility, you say to yourself, unconsciously, implicitly, or whatever-- if he's not saying anything, why should I say something?
And so you're less likely to say something.
Now the one good side of this, this is all sort of depressing, right?
It sort of suggests that if we're going to hang out in large social groups, one of the downsides of this is that we're not going to look after each other terribly well.
There's a useful inoculation effect here and I've just inoculated you.
The fact that you know about this, there are experiments suggesting that the fact that you know about this makes you more likely, the next time the occasion arrives to discount the number of other people there.
So now it is probably the case that if I do keel over somebody will rescue me, even though the responsibility for this has been diffused across.
Of course, the reason is because somebody's going to think it's a fake.
And they think, I'm going to win the prize if I collect him or something.
But I'll try not to do the experiment at all.
But in any case, there is evidence that once you know about these things you're less vulnerable to those effects and less likely to find yourself complying with an instruction that's wrong.
I mean, look, you probably have a good intuitive feeling and I hope you're right that all over Iraq now, if you're holding prisoners you've now been inoculated against doing a whole round of stupid things.
When the Abu Ghraib broke every social psychologist in the country was on some talking heads show on national public radio saying, we knew about this.
We can explain this because that's exactly the sort of thing that's understandable in the context of these sorts of experiments.
People doing things that they would never-- you know, what's the chance that you were back home before going off to Iraq saying, I'm going to go to Iraq.
I'm going to get me some prisoners, put them in compromising sexual positions and get photos sent back home.
No, nobody thinks about doing that, but the social situation that you find yourself in with perhaps an implicit command from somebody that loosens things up a bit produced the effect.
The fact that this was a disaster has presumably now inoculated the system at least in the short-term against it happening again, at least we may hope so.
The practical sort of questions, like what do you do about prisoner abuse are one interesting line that grows from this sort of work.
Another question, returning perhaps to this sort of more basic, psychological side of things is what I think I labeled on that handout as-- oh yes, the fundamental question.
I decided if you could have the fundamental attribution error that I would invent for today's purposes, the fundamental question.
But understand that that's my invented jargon and I won't put a question on the exam saying, what's the fundamental question?
The question here is, who is this-- I keep talking about you-- you know, you can be pushed around by these laws of social impact and stuff like that.
Who is this you, or I that can be so readily manipulated?
There's a lot of folk psychology about it.
Well, look, we understand for instance, it is possible to control a car in such a way that the car kills somebody.
It is also possible to train a dog in such a way that the dog kills somebody, and history tells us, it is possible to train human beings in such a way that a human being kills another human being.
There's something fundamentally different, folk psychology understands that there's something fundamentally different about the responsibility involved here on the part of the car, the dog, and the human.
And that seems to have to do with this sense of self.
The sense of you as a self-aware, psychologically continuous overtime kind of entity.
Is there anything useful that we can say about that?
About that sense of you as a psychological self?
All right, let's back into this a little bit.
If you look at me, you know who I am, right?
Pretty much, those of you who don't, it's getting a little late in the term.
But you know, you recognize me.
That's fine.
Could I be a cunningly disguised actor who's impersonating Wolfe today?
Yeah, that's possible.
Be mildly interesting.
OK fine.
Switch.
All right, let's suppose this is your mother or somebody else who you feel strongly about.
You look at that person and the recognition has two pieces to it.
More than presumably looking at just some person you know.
There's a fairly strong affective component.
Let's take the best case scenario here.
You see your mother you recognize, that's my mother and I love my mother.
OK, good.
Let's suppose that something goes wrong with the affect piece of that.
You know, there's some hunk of brain that's doing that for you.
Something goes wrong with that little hunk of brain and you look at your mother and you say, I recognize my mother and I hate that.
Or I don't feel anything would be another possibility.
That doesn't make sense.
This again, happening-- you're not sitting there with a little checklist saying, does this makes sense?
You're toting up your current experience and asking implicitly, does this make sense?
The answer is no.
How could that be?
Well, one possibility-- it sounds a little odd-- but one possibility is that's not really my mother.
It looks like my mother, but actually she's been cunningly replaced by a fake.
Why would that be?
Well, you know, if it was my real mother, I love my mother.
I don't love this, therefore it's a fake.
Now that sounds weird, but it's a real neurological condition.
It's called Capgras syndrome.
I put a definition of it, I think, on the handout.
I always think of it as crab grass syndrome, but the Capgras is correct and it's the name of somebody.
The disorder seems to be an effort to reconcile an unreconcilable pair of thoughts.
It sometimes travels with schizophrenia as a diagnosis.
It may be the result of a brain injury.
Typically, when there's a brain injury involved it looks like it's frontal lobe in some fashion.
It's relevant to the current discussion because you can have the same phenomenon for yourself.
You look in the mirror, you say, that's not me.
And you have a denial that you are you.
It points out that even something as self- evident as-- is it a Dr. Seuss book-- I am I, where's that from?
I can almost hear it.
You haven't been reading your kid lit recently enough
Sam I Am
No, Sam I Am, but I don't think Sam has Capgras issues.
I am I, and I can't remember what comes next.
Oh well, doesn't matter.
In any case, one might ask whether there is some little piece of neural tissue that is you in this sense.
That if we were to go in and take out this one little bit of tissue that what we would discover is that you weren't you anymore.
That you'd be-- yeah, you'd still have a memory, you'd still have this, you'd still have that, but you just wouldn't have this sense of being you.
The short answer to that is there doesn't seem to be any specific lesion that does that.
That used to be sort of the end of this part of the lecture.
However, particularly with the advent of fMRI it's becoming increasingly clear that aspects of this sense of the self do have homes in the brain that can be localized to some extent, damaged in various odd neurological disorders, and give us some idea about how it is that this sense of self is assembled.
So what I've done is I've actually modified a list from a recent article in Trends in Cognitive Science to come up with six aspects of the self.
To give you some notion about how you could take this problem apart the first of these-- I must have just listed them on the handout, right?
The first of these is representation.
Nobody, when asked about themselves, what kind of person are you?
Nobody says, Oh, I am just a collection of reflexes responding to the current situation in the environment.
Everybody has a list of you know, I'm an honest, loving individual.
You may have some sort of trait or situational theory about how you came to be honest and loving and all that, but you've got a set of traits that you believe to be fundamental to you, at least at the present time.
And that's your representation of yourself.
That's your theory of you.
You also, as part of that theory you have this notion of agency.
That you are responsible and in fact, you are the cause of at least a reasonable number of your actions.
You feel that you own those actions.
That's part of what makes the Milgram experiment disturbing.
It's not disturbing that a machine can produce 450 volt shocks of some schnook.
It's disturbing that you can because you figured that you own that action and that if you did it that says something about you.
That's this question of agency.
Now not only do you own actions, you own your body.
That's the aspect of ownership that I'm talking about in number 3 there.
You own your body.
Look, you also own your laptop.
But you own your body in ways that are more profound than the way that you own your laptop.
There's something fundamentally different about that.
You're able to use this information that you've got about what kind of you you are.
You're able to use that to perform new calculations, if you like.
Like, do I like-- I don't know-- I need a new fruit.
Durian.
Do I like durian?
Well, it's a fruit.
I like fruits.
It's supposed to smell like fermented garlic.
Sounds [UNINTELLIGIBLE].
I can ask myself questions about my self based on my knowledge of myself.
That's this evaluation aspect.
Monitoring is sort of a sub-- well, I don't know if I would consider it to be a subset of that.
Monitoring is keeping track of what you are doing and asking yourself how it's going.
All right, so I'm self- monitoring here saying, am I really going to fall over today?
No, I don't think so.
I'm feeling pretty good just at the moment.
That's monitoring.
You're asking yourself, am I still awake?
If you're answering yes, the answer is probably yes.
If you answer no, that's a little weird.
Or it's lucid dreaming or something.
That's nice, too.
And finally there's this sense of integration where the whole thing is tied together into a single self.
There's been plenty of evidence that we've been accumulating during the course of the term that you are in some sense a committee.
That you've got lots of semi-autonomous bits and pieces doing their thing and giving rise to you, but an important sense of you is that it doesn't feel that way to you.
You don't think of yourself as a committee or a collective or what's that thing-- is Star Wars where the colonial organism-- is that the Borg?
The Borg, right?
You don't think you're the Borg.
You've got this sense that you are a unitary thing.
Now each of these bits, while it's not the case that we can go in and somehow pull out the thing that is you, at least nobody's figured out what piece of brain that might be.
Each of these aspects of you is subject to attack and to some degree at this point of localization in the brain.
A clear example would be the question of ownership.
I think I actually talked about some of this earlier when we were talking about perception.
If you get a lesion in your right parietal lobe you will typically show symptoms of what's known as contralateral neglect.
Contralateral meaning on the other side and what I think I talked about when I was talking about perception is the fact that you will ignore the left side of visual space and I think at that point I also mentioned that people ignore the left side of their body.
You can have all sorts of varieties of neglect, but it can include ignoring the left side of your body to the point of denying-- I mean, you can see it, but you might deny that it's you.
That's neglect.
Interestingly, if you lesion the left parietal lobe you do not typically get neglect of the right side of the body.
At least not in right-handed individuals.
The best understanding I've ever come up with for that is that it may be that in right-handed individuals the right side of the body is represented more robustly or redundantly than the left side of the body and so a typical lesion that leaves you alive, at least, doesn't take out the entire sense that you own that side of the body.
Whereas, a lesion of the less dominant, right hemisphere, left side of the body takes out this sense-- leaves you with the sense that you don't own the left side of your body at all.
Now the flip side of this is our phantom limb phenomena.
So suppose you lose the limb rather than the piece of brain that represents the limb.
So if you lose a limb very typically-- well, the hunk of brain is still there.
And the brain is a kind of a noisy place.
What's going to happen if you've got activity that goes above some sort of threshold in the chunk of brain that used to represent this arm?
The answer is, you're going to think you've still got an arm.
And you may know at some nice cognitive or depressing cognitive level that you don't have that arm anymore, but it's going to feel like it's there.
And it can be really annoying.
Because phantom limbs do things like they get stuck in a position.
Why's that annoying?
Well, that's annoying because somebody with a phantom limb stuck like this walking out the door does things like this.
Which is exactly what you would do of course if your arm was like in a sling out there because you don't want to bash into the door.
But you know, you feel a bit of a fool if you're doing that with a limb that you know perfectly well isn't there.
What do you do if you've got an itch in your phantom limb?
Or actually pain in phantom limbs is an even more serious problem.
Perfectly possible to have pain in your phantom limb.
It's not particularly easy to treat that.
Now it turns that the brain, the brain, like nature, abhors a vacuum.
And if you lose a limb the chunks of the brain surrounding the arm representation start to say, there's open territory here.
We could take that.
And over time the brain reorganizes and you get very curious reports about the limb changing.
So the limb may shrivel up.
There are reports where the guy comes back to the doctor and says, how you doing?
I'm doing much better.
Now instead of this stupid limb out here I just have fingers hanging off my shoulder.
Which it's kind of weird, but since they're invisible it's not that weird.
What seems to happen is that neighboring stuff takes over what's near to the arm and the hand in the sensory homunculus, for example, are things like the face.
And I put a reference to Ramachandran's entertaining, as I said, a book on the handout.
Blakesley is a science writer, Ramachandran or Rama as he is usually called is-- well, I think of him as a vision researcher though, mostly these days he's gone off more into the sort of neurological work.
But what he did was he was testing phantom limb patients.
He would touch them on their face and ask, what do you feel?
And the patient-- he's got interesting video actually of the patient saying, well, look of course, I can feel you touching my face, but the interesting thing is it also feels like you're touching my phantom thumb.
You move over a little bit and now it feels like you're touching this phantom finger and so on.
And you could actually map out for hand and arm across the surface of this guy's face.
He loved it, by the way.
Why did he think this was good?
Yeah?
He could itch it?
Yeah, he knew where to scratch if it itched.
And it worked.
You get an itch on your phantom arm, he's now learned that he can scratch there and you get a degree of relief.
So the sense of ownership of the body can be localized in the brain to the things like these parietal structures.
The degree to which the brain is plastic even in adulthood is a growing story where you find that the chunks of-- oh there was an fMRI study showing for instance, that if you look at violinists, the representation of the fingering hand in the brain is larger than the representation of the bowing hand.
The fingering hand is learning a whole lot of stuff that the bowing hand doesn't need to and it ends up using more brain, apparently.
All right, lets go back to this notion of the self as a bundle of personality attributes.
And of your understanding of the implications of that knowledge.
So these are the ideas of representation and evaluation from that earlier list.
Does this have a home?
Maybe is a reasonable answer.
There is evidence from neuropathology that suggests disorders that specifically attack that aspect of the self.
The disorder I'm thinking of is known as Pick's disease, which I probably labeled on the handout as a fronto-temporal dementia.
Oh, it says that on the handout also.
The way to think about it is sort of like an Alzheimer's of the personality.
It's an Alzheimer's like disorder, it's a terrible diagnosis by the way because it typically is a very progressive-- quite rapidly progressive dementia that leaves you witless fairly quickly and dead fairly shortly thereafter.
Very bad news.
But it is of psychological interest because while the initial manifestations of Alzheimer's are often this sort of forgetfulness, the manifestations-- what gets your family to bring you to the doctor if you've got Pick's is a change in personality.
I'll tell you a couple of anecdotes here.
Here's a patient, 54 year-old woman described as a charming, dynamic real estate agent who goes from wearing expensive designer apparel to choosing cheap clothing and gaudy beads and asking strangers how much their clothes cost.
Once a lover of French cuisine, she adopted a love of fast food, particularly, Taco Bell.
Now look, if you like Taco Bell that doesn't mean you're going to die of dementia next week.
The point is that what people notice, what your family typically notices is an unexplained change in that sort of list of attributes that makes you you.
Or a patient, 63 year-old woman described as a well- dressed-- I don't know why they're all well- dressed-- well- dressed, lifelong political conservative who became an animal rights activist who hated conservatives, dressed in T-shirts and baggy pants and liked to say Republicans should be taken off the earth.
So again, you can imagine somebody whose political opinions evolve, but you could also imagine that if your rock- ribbed Republican grandma was suddenly doing a kind of a late hippie thing and people might worry.
The lesions here seem to be typically at this stage in the right frontal lobe when you get these people into a scanner.
That was a mistake.
Shouldn't try lecturing while you're drinking.
But no, you don't have to rescue me.
I think I'll be able to rescue myself.
Anyway, so the loci that seem to be particularly important, but if I've been eating you should come up and then Heimlich me and stuff like that.
I don't think the Heimlich maneuver works well for water.
Anyway, the particular chunks of brain that seem, at least one recent study suggests are important here, are chunks of the frontal lobe lying on the midline, so your brain's like this.
If you open it up that's the medial surface of the cerebral hemispheres.
And there's evidence that locations sort of in there are what's important.
What may be very important for putting together this sense of the self.
I think I did-- oh yes, I put on the handout the notion that orbital medial prefrontal cortex might be a possible home for representation.
I put that there just because it's got to be so much fun to go off and be able to talk about the OMPFC and to distinguish it from the DMPFC.
I promise not to ask exactly that-- is representation localized in the OMPFC or DMPFC?
This is for the subset of you who are dying to go off to med school and stuff like that and want practice using jargon that nobody else will understand.
Don't get me wrong, this is useful stuff.
If you're interested in the topic the reference on the handout to-- did I put the reference on the handout?
Yes, this comes from this Northoff and Bermpohl article way up at the top of page 2.
By all means, go check it out.
I just realized when I was thinking about lecturing about this that I knew that a whole bunch of people are going to go get deeply obsessive about worrying about whether they've got the OM versus the DM straight and I don't want people to spend a lot of time worrying about that.
What I want you to come away with is the notion that there are a set of localizable bits of particularly frontal structures in the brain where you can make a case for each little bit being responsible for a little bit of this overall sense of the self.
Well, one of the ones actually where the little chunk of brain-- again, I probably won't ask a question, but where little chunk of brain is quite clearly doing something specific for you is the anterior cingulate.
Again, and the midline of the brain or the medial surface of the cerebral hemispheres are wrapped around the corpus callosum, that big bundle of fibers that connects the two halves of the brain.
The cool thing about that is that it really looks like it's serving a monitoring role.
What it does is it's the thing that's siting there checking, am I making a mistake here?
If you're in our lab I can feel my anterior cingulate light up because if you're in the lab and you're doing a whole bunch of visual search experiments and you hit the no key just as you see the target-- that "Oh dear" phenomenon.
Where you can just feel yourself saying a word you shouldn't say, that's your anterior cingulate error monitoring.
How do we know that?
Because if you do these experiments in a scanner when people make errors that lights up.
It's really quite lovely.
It's also, in the practical advice side of the course, it turns out at least from a couple of things that I read that the anterior cingulate is particularly vulnerable to alcohol.
What's that mean?
Well, it means have a few drinks and the chunk of your brain that's saying, "Is this a mistake?"
takes a vacation.
So have a few drinks, go down to Kenmore Square to celebrate the Red Sox-- look, there's a car.
Could we flip that over?
Yeah, we could do that.
Would that be a mistake?
I don't know, the guys asleep so don't wake him up.
Or you know, go and drink a few things and should I sleep with that person?
I don't know, we'll ask in the morning.
Well, I guess the answer was no, I shouldn't have.
But anyways, next time you are inclined to imbibe something stronger than Poland Spring Water just think of your little anterior cingulate that's bathing in alcohol and then you can go drive down the highway and say, "Would it be a mistake to pass this semi on a wet road tonight?
Well, I don't know, let's find out."
So you want to keep that piece of your brain and you want to keep it more or less functional.
The conscious you, this notion of integration seems to have something to do with the notion.
What's it mean that it's all one thing?
In some way this is all tied into the fact that you feel like a single, self-reflective, conscious entity.
Well, one interesting question is, are you conscious?
Well, sure.
Now that I asked you.
Were you conscious before I asked you?
Oh maybe.
This is what's known in the philosophical literature as the refrigerator light problem.
Is the refrigerator light on when the refrigerator is closed?
No
Well you know that the answer is really no and look-- even most philosophers know that.
But that's because you have some sort of a deeper understanding of refrigerators.
But if you're just faced with this thing and the question is, is the light on when the door is closed?
How would you know?
Open the door, light's on.
It's always on.
Every time I open it up the light is on, it's cool.
People seriously entertain the possibility that consciousness is like that.
It feels continuous because every time you ask yourself in any fashion about whether or not you're conscious, not counting the guys who were out cold and well, they can't ask themselves anyway, you feel conscious, right?
The closest you can get to a sort of a feeling that there might be an issue here is there's a phenomenon known to long haul drivers where you're driving down the interstate, you get from point A to point B and you have no recollection of having done that.
The good thing is if you're doing that while you're asleep.
Asleep is a different problem.
It's a very serious problem, but the greatest number of car accidents per mile is not like in downtown Boston where you would think it ought to be.
But it's like way out West because 600 miles, the road doesn't even turn.
The problem is the 601 mile it makes a 10 degrees slant off to one side and you're semi doesn't.
And you're straight into the sage brush or something like that because you've gone to sleep.
But the less dire version of this is you can get into a sort of a "nnnnn" kind of state driving down the interstate where you really seem to lose a chunk of time and you ask yourself, was I conscious during that?
I must have been awake because I didn't crash the car, but there's nothing there.
Whether or not you were conscious or not I can't speak to because of this refrigerator light problem.
If we asked you while you were in the midst of the state, are you conscious?
You'd say, "Of course I'm conscious.
That's a stupid question."
If you're interested, one of the projects-- one of the more interesting neuroscience projects out there is to decide which bits of brain are important for consciousness.
Christof Koch at Caltech has done a lot of work on this, wrote an interesting new book, which I would've put on the handout if I could have quickly found the reference, but if you go to my website you can find my one page review of the book-- which tells you what the books about without you actually having to read it, but it would also give you the reference to the book.
But Christof refers to the neural correlate of consciousness.
The question is, which bits of brain do you need to be conscious?
And then you can go on from there to ask, what that consciousness might actually be for?
It's very clear that we can do an awful lot of stuff without being conscious.
This is an important wrinkle on this issue of agency.
There's an interesting body of research that suggests that when you feel like you are making a decision-- which one of these things do you want-- the brain, we can pick up a lecture of physiological indications that you have made the decision before you, the owner of the brain, know that you've made the decision.
One can wrap oneself in philosophical knots trying to ask yourself what that means about the agency of your acts, are you-- the conscious you-- committing those acts or are you merely reporting on them afterwards?
But there is this interesting data that suggests that you have made the decision before you know that you've made the decision.
In any case, what we can't find is clear evidence of a single, little piece of neural tissue that is you.
No single lesion seems to leave everything else intact, but take you out.
So what I want to do now is to switch from neuropathology to psychopathology.
But I will do that after you take a brief stretch here and then we'll talk about that a bit.
[INTERPOSING VOICES]
I had a question about the syndrome that you talked about where you can't recognize who you are.
Is it visual?
Like so if you look in the mirror--
Oh, no.
You know perfectly well looking in the mirror that that looks exactly like me.
The weird thing is it just isn't me.
If I had a photograph of you, you would know that was you and you would also know that it wasn't you.
I mean, it's not you, you're this person right here.
Now if I had a photograph of your mother you would know that this was a representation of your mother, but wasn't necessarily your mother, per se.
The weird thing about Capgras is if I had your mother here you would have the same sort of sense of it being it looks like mother, but it just isn't.
It's one of those things where it'd be cool if you could have this reversibly just to know what it felt like.
It'd be very hard to wrap you brain around what that particular delusion must feel like.
There are visual disorders where you don't recognize faces, that's prosopagnosia.
I think I talked about earlier.
Where I look at you, I say, two eyes, nose, mouth, it's a face.
Whose is it?
I don't know.
This is in some sense the opposite, a complement of that.
I know exactly who it is, I just don't believe it in some sense
Well I mean, if I have that disorder and [UNINTELLIGIBLE PHRASE]
Your hand.
That's interesting.
I'm not sure I've ever read anything about whether people deny their own-- in neglect you would deny it was your own hand, but in neglect typically the hand would be paralyzed.
You wouldn't be doing anything with it because of the nature of the stroke.
I don't know.
I thought you were going to ask whether I could look at that water bottle and deny it was my water bottle and it doesn't seem to extend to objects.
It seems to be a very person specific kind of delusion.
That it's about people and about people with whom you have strong affective relations.
Very sad kind of thing because you read these anecdotes about how would you feel because typically somebody is in tough shape for one reason or another when they've got this disorder.
So you're visiting your kid in hospital said he's busy declaring you're an evil agent who's disguised as Daddy.
You know, that must be heartbreaking.
And I remember reading one account where Daddy, in an effort to cure the kid, comes back and declares.
I did it, I killed him.
And I think the story was it provided some temporary relief and then it didn't stick.
But very weird stuff and very, small, it's rare stuff; small literature.
I just realized I suppose it didn't matter, but I just realized that if I'm going to mike myself having discussions during the break could be a little risky.
I got to make sure that I'm just talking about the details of Capgras syndrome, which we were as opposed to having a discussion about-- I hate my TA.
Oh yeah, what's the name of your TA?
That wouldn't work.
Or I love my TA, and nevermind.
Let's get onto psychopathology-- my own and others.
The first thing to do is to say something about the distinction between a neuropathology and a psychopathology, which might seem a little weird in a course that began by asserting the mind is what the brain does.
I mean, in what sense can you talk about neuropathology on one side and a psychopathology on the other?
I decided today and I'll see if I still think tomorrow that it's a useful distinction to think of it sort of like a hardware bugs versus software bugs.
Same computer not working in some interesting fashion or other, but you can have a problem because some piece of the equipment is broken.
And you can have a problem because the program running over that equipment is not working properly.
Now I realize as I'm saying that that there is one problem with that analogy, which is that if I have a hardware bug I'm going to be inclined to fix the hardware and if I have a software bug I'm going to be inclined to fix the software.
That distinction is by no means as clear in psychopathology versus neuropathology.
So you can have a brain lesion where the treatment is essentially behavioral because you want to do some sort of retraining of the brain that you've got left and you can have a psychopathology, a problem if you like with the way that the software is running where the intervention is to give somebody a drug that's doing its work essentially on the hardware.
So the metaphor is not perfect, but it gets at something.
Psychopathologies are... it's worth thinking of as problems with the way that the program is using the brain.
Are there psychopathologies that specifically attack the self?
The answer is yes.
Those are what's known in the trade as dissociative disorders.
Much beloved of various and sundry literary outlets, so the cleanest one if you like is psychogenic amnesia or depersonalization.
Occasionally resorted to by soap opera writers who have backed themselves into a corner.
That's the get a knock on the head and I don't know who I am, I don't know where I am.
And that allows you to do a certain amount of plot twiddling as needed.
Oh, let me say a word about-- now that we're talking about psychopathology-- I need to say something about psychiatric hypochondria, which is psychogenic amnesia.
Very rare, it's not going to happen to you.
It's not even going to happen to you after I tell you we don't really know what the cause is, but the best we know about it is that it's a response to severe stress.
The problem is, you say that and people say "huhh!".
I'm under more stress than I've ever been all my life.
I'm going to wake up tomorrow and not going to know who I am.
Not going to happen, very rare stuff.
But this is a close kin, this is the intro psych kin, to med student hypochondria which is a very real known phenomenon.
Med students are always coming down with spectacularly rare diseases, at least in their own minds.
You hear about some new set of symptoms of some weird tropical disorder and you can kind of-- is that like a tape worm right about there?
They worry about these things a lot.
And in the same way that they probably don't have that tape worm because you just don't pick them up at Harvard Med, mostly.
You shouldn't worry about whether tomorrow you're going to wake up without knowing who you are.
Even though you may be under a certain degree of stress.
That said, so let me tell you one case.
A woman wakes up-- a woman is found, actually in a bad state under the bushes in Florida somewhere back in 1980.
Doesn't know who she is.
Can't remember anything about her past.
And eventually, in an effort to figure out who she is and they call her Jane Doe of course, and then take her on morning TV and ask basically, does anybody know who I am?
And immediately got a string of phone calls that suggested that they knew exactly who she was.
She had apparently left Ohio a few years earlier and moved to Florida and then disappeared.
Her parents hadn't heard from her for several years.
And unclear what happened in this particular case, atypically she did not recover her memory.
Typically, these global amnesias are transient.
In her case, at least at the time of the report I was reading, she had not recovered her memory.
That's the most global, if you like, of these dissociative disorders.
Fugue states, from the Latin word "to fly" are related.
Again, it's presumed to be a response to severe stress.
There's a very nice case described by William James in his great Principles of Psychology-- the best intro text ever written in 1890, so we can't still use it.
But he talks about the Reverend Ansel Bourne who took some money out of the bank in Providence, Rhode Island and vanished.
He reappeared-- well, he woke up one morning in, I think, Norristown, Pennsylvania and discovered that for the past two months he had been running a store in the name of A.J.
Brown.
He had no recollection of the preceding two months.
Hypnosis turns out to be a useful, if somewhat mysterious tool in such cases.
And James, who was very interested in hypnosis found that under hypnosis he could sort of call up either Brown or Bourne.
They didn't seem to know each other, but what seems to have happened was-- that the characteristic of a fugue state is that you disappear to another location and pick up another identity.
Now one of the interesting things about this and it's actually a theme that will recur through a lot of discussions of psychopathology.
Is that a fugue state is in a sense an abnormal version of something that we all understand as a normal story.
In fact, most of us in our family histories somewhere have somebody back in Germany or Poland or China or Russia or Korea or somewhere who was under a lot of stress in the Old Country, picked up, and developed a whole new identity in the U.S.
The major difference of course, is that grandpa remembered the old country.
Grandpa hadn't suddenly just gone poof and hey, now I'm running a store in Brooklyn.
Nothing like that.
But the notion that a response to an intolerable situation here might be to move here is sort of like the American story.
The pathological aspect is if you don't remember what had happened before.
Now by the time you've got a couple of different characters, particularly, who can be called in and out of awareness under hypnosis, you are very close to everybody's favorite dissociative disorder, long known as multiple personality disorder.
Now more officially known as dissociative identity disorder, but I will continue to refer to it as multiple personality disorder because if I think multiple personality disorder I know what I mean immediately.
If I say dissociative identity disorder I have to think about it.
They are essentially the same thing.
This is the situation in which two or more personalities appear to occupy the same individual.
Popular with popular supermarket psych books.
The great movie of my youth was a movie called Three Faces of Eve.
Has anybody ever seen it?
Oh good, must still be playing in reruns on some old-- scared the wits out of me when I was little.
Can't remember anything about it except that I mean, she didn't have really three faces, she did have three personalities.
Let me tell you instead about Billy Milligan, a convicted rapist.
Not a nice person.
But one who attempted to get off his conviction by claiming a lack of responsibility due to a multiple personality disorder.
Depending on which of the multiple accounts you read of this he had anywhere between 10 and 24 personalities including Arthur, a sort of master personality who knew all the others rattling around inside him.
Arthur is reputed to know Arabic and to be self- taught in physics and chemistry.
Regen was aggressive, fluent in Serbo-Croatian, spoke with a thick accent and was reported to be the initiator of the sexual assaults.
Adalana is a shy, introverted lesbian with a spontaneous nystagmus, reported to be the actual rapist.
Now there's a whole collection of weird stuff there.
One point is that there's no guarantee that all the personalities are the same gender.
A perhaps more salient point here is this spontaneous nystagmus thing.
And nystagmus is a beating movement of the eyes.
It's hard to fake.
There's an argument about whether multiple personality disorder is for real.
And even if it is for real there's always an argument in a court case for whether or not somebody's just trying to get off the hook by faking it.
This spontaneous nystagmus is important because it's hard to fake.
If you sit there and try to vibrate your eyes continuously that's not easy.
So as Allen he's a con man, Tommy knows electronics and plays the sax, Samuel is an Orthodox Jew and a woodcarver and so on and so on and so on.
So it's very odd business.
On the face of it it's a fragmentation of the self.
If you think of the self as a kingdom this is like a bunch of the provinces have rebelled and they have complicated diplomatic relationships with each other.
Who knows who?
Who's talking to who?
Who likes who?
Very complicated construct here.
Oh, one important way to distinguish a psychopathology, which if it's for real-- it clearly is-- from a neuropathology is nobody would expect to put this guy in a scanner and call up-- I don't know-- Regen, the Serbo-Croatian rapist and find, there he is.
That piece of brain and now the other guy's over here and this guy's over here.
Look, they're all making use of the same brain.
They're all forming memories, for instance, and using presumably the same hippocampus to do it.
This is a case of very odd software running over the hardware.
So it's a troubling pathology.
Yep, there's a question?
What's the deal with the eye beating and why is that hard to fake?
Why is it hard to fake?
Well, just sit yourself in front-- well, you can't look at your own eyes moving in the mirror, so you can't try it that way.
Pick a friend, look at a friend, try to keep your eyes beating rhythmically for five minutes by an act of will.
Make sure you have aspirin handy.
It's just very hard to do.
I can do all sorts of really weird things with my eyes actually, but that's not a nystagmus.
That's some sort of weird, spastic movement.
Even if you could produce a movement any competent opthamologist can look at a nystagmus or you can record it and know whether this is a plausible neurological sign or something weird that the guy's doing.
Who knows?
You know, maybe he's a genius at this sort of thing, but it would be a hard thing to fake.
I don't think there's much more to be said about it than that.
But this issue of whether it's fake or not-- one of the other arguments in favor of the notion that there is a real disorder is imagine you've got 24 personalities or imagine your parents tell you-- how many parents when you were little made up some sort of long-running story that they told you?
No?
Oh good.
The rest of you was deprived.
They were just reading out of the books.
Anyway, I used to have a nice long-running story about bunnies.
Very complicated stuff.
The problem is once there were like 24 bunnies or whatever, I don't remember how many there were-- the kids would keep calling me on errors.
No, no, he lives over there.
It's very difficult to keep all this stuff straight.
If you have a story with 24 characters in it that you're making up as a fake in your own head you've got to keep that lie.
Well, you don't have to do stories.
Think of a good lie.
How many times when you were a kid did you get stuck because the lie fell apart because you told it wrong?
You changed the story.
If it's a truth it's easy enough because you just go back and refer to the Truth and it's just kind of there.
But the lie you have to remember the details of and there's an argument that the same thing is going on here.
If you really had twenty-four different personalities it'd be just very difficult to keep them straight unless you were just reporting on-- so here's Chuck, the personality, who do you know?
I know boom, boom and boom.
If you're faking it-- you-- the owner of this whole story have to think, Chuck, Chuck-- oh God.
I don't remember who Chuck knows and so on.
Now a more troubling fact perhaps, is that multiple personality disorder was practically not on the radar screen twenty years ago.
When I first started talking about multiple personality disorder I could like read the whole literature on this-- handful of cases from a handful-- a handful of patients from a handful of therapists.
In twenty years it's skyrocketed.
It became sort of flavor of the month, everybody had a multiple personality disorder for a while.
That's faded off a bit since then, but that's odd.
Why should that be?
One theory is that is that it's a well- intentioned delusion, cooked up between a therapist and a patient.
Patient's got some real problem, the therapist believes in multiple personality disorder, and through the use of hypnosis, which is a common tool for therapists who treat multiple personality disorder you come to the conclusion-- you and the therapist come to the conclusion that you have more than one character rattling around in there.
And that it wasn't real before you came into the therapist's office.
The counter argument to that is that what brings people in often is that they're missing chunks of their lives.
You come to the therapist's office.
You say look, I have no idea what I did Tuesday afternoon, but there's a new dress in my closet and there's a charge on my Visa card and this is disturbing.
If things were happening to you, you just didn't remember it might eventually get you into therapy
Well, couldn't multiple personality disorder popularity skyrocket after that movie came out and everyone saw it?
Oh yes.
To anticipate what I will actually end up getting to do in next Tuesday's-- since there's no class on Thursday-- there is a sense in which you get to choose how to go insane.
The culture around you enables certain forms of certain patterns of mental disorder and not others.
And sure, there's a lot of this sort of copycat cracking up, if you'd like.
Let me say one last thing about this issue of what gets people into therapy.
It's one thing to find a dress in your closet.
The sort of thing that drives you to the therapist really quickly is you wake up in the morning, there's a guy in bed with you, it's pretty clear that he's been there for awhile and the suggestion is that you've been doing stuff of some sort and you don't know who he is.
There's a lovely-- well, lovely-- scary anecdote by one patient where sort of a case where bad girl/good girl personalities.
The good girl didn't know the bad girl personality, but the bad girl knew the good girl and the bad girl in therapy said, oh, this is great.
She's such a prig.
So I take over and we go out to a bar and pick up a guy.
I bring him home and you know, we do stuff, and then I just withdraw back into her and watch the next morning.
It was great, man, she'd wake up and she'd freak every time.
Well, you've got to imagine the guy's point of view, too.
You know, guy comes home with somebody, the next morning she's ranting and raving that she's never seen him before and kicking him out the door.
It must be very strange for all concerned, but is certainly the sort of thing that would land you wanting to talk to somebody in the mental health field.
All right, we'll pick up next week.
The following content is provided by MIT OpenCourseWare under a Creatve Commons license.
Additional information at MIT OpenCourseWare in general is avaialable at ocw.mit.edu
I'm not actually going to use because I've discovered that-- and I need the CD player later, but I did promise Simon, age eight, that I would tell you all that he loaned me this boom box and that you should know that this was his.
And that I should bring it back.
We were talking in the last lecture about multiple personality disorder and my recollection is that I left off discussing the question of whether or not it's a real thing.
And I'd say that the burden of evidence is that there really is a real problem there of some sort.
You know, people showing up at their therapist's office saying, I don't remember this big chunk of time in my life and then weird stuff happened and so on.
Those sorts of things point to a real phenomenon.
But there's this troubling business that the disorder is new, relatively new.
That there were very few such cases in the 1970s.
They start showing up in the 80s.
They're extremely popular in the 90s.
I'm not sure if they're still as popular, but that there's a big upsurge.
Now look, if there's a big upsurge in flu this year we sort of think we understand it.
Some chicken in Thailand got a new version of the flu and then we all get sick.
But when that happens with a psychiatric disorder there's no sense in which we think that there's a pathogen out there doing it.
What's the explanation?
So I put the bones-- the bare bones of one account on the handout there.
The first of them is that while multiple personality disorder might be a new entity.
Or newly popular entity, dissociation is not.
That dissociatve disorders have a long history.
They have a long history in psychiatry and they have an even longer history if you start poking around in literature looking for evidence of it in the past.
It's just that Point 2 as I recall on the handout.
You get some choice in the way that you manifest your mental illness.
The current way to manifest or one of the leading current ways to manifest, whatever the underlying problem is in dissociatve disorders is multiple personality disorder.
In prior eras or in other places in the world the same underlying problem seems to have shown up in different ways.
Multiple personality disorder-- one of the ways that it comes to our attention is when somebody says look, I know you caught me doing this, but I'm not guilty because it wasn't me.
It was this other piece that somehow is not me.
And therefore, I don't have personal responsibility for that act.
That's an interesting question in the interface between the law and psychology.
How should we understand that?
Does it make sense to say, you don't need to go to jail for this crime because it was done by something that looks like you, but isn't you in some sense.
That's an interesting topic for recitation perhaps.
Things of a similar sort occur throughout history.
So for instance, think back into medieval culture or into various parts of the world today where the idea of demonic posession let's say is taken as a real option.
Why did you do X?
I did this because I was possessed or the cliche, which I can't remember which TV show it was where that the tagline was "the devil made me do it".
That's sort of a modern pop version of a seriously considered hypothesis that's been popular in western civilization and elsewhere in the world.
I don't have anything particular to say about whether or not there is such a thing as the real entity of demonic possession, but you got to see the possibility that something external takes control of you and makes you do things that you wouldn't otherwise do as having a very similar shape to what we now call multiple personality disorder.
Or if you go into the 19th century there are lots of literary cases, you may have read some of these in novels where a dead relative typically takes over a living person's body.
You know, somebody's soul is unquiet and their spirit is wandering the world and they take over somebody else and you do stuff that you hadn't particularly intended to do.
Again, this notion of an external something taking over what would normally be the volitional, conscious part of your mind and making you do stuff.
So it's possible that multiple personality disorder is merely the current version of, if you like, trick of the mind that has existed for a long time.
Now why did it disappear?
Why did the dissociative disorders as a whole disappear?
It disappeared in American and European psychology early mid 20th century.
Perhaps the best way to say it would be that Freud made a mistake.
Freud was seeing patients with similar sorts of problems, but he saw something else he thought.
To explain that it'll help to say a bit first of all, about what causes multiple personality disorder and then to say a lot more about what Freud thought.
Multiple personality disorder, like the other dissociative disorders is associated with stress.
It's a disorder of severe stress, but one of the things you see in patient account after patient account is a particular kind of stress, specifically childhood abuse and very frequently within that set, childhood sexual abuse.
One way of describing multiple personality disorder-- I'm roughly quoting I think, from a therapist named Ross, is that multiple personality disorder is a little girl saying that the abuse is happening to somebody else.
Why a little girl?
Multiple personality disorder is a diagnosis that has about a 9:1 female to male ratio.
It's a very heavily female diagnosis.
This is not to say-- there are a bunch of disorders where there's great asymmetry in which sex manifests the disorder.
This isn't to say that women are somehow the fragile sex whose self is going to crack up while us strong guys, you know, are doing all right.
If you were looking for a male disorder, what might that be?
Alcoholism?
Alcoholism, I'm not sure is actually.
I don't know, but I don't think that it's particularly heavily divided between male and female.
Yes?
Autism?
Autism.
I think that's right, but these days we tend to think of that less as a psychiatric then as a neuropsych disorder.
Any autism experts?
Autism is heavily male?
Yeah, that sounds right.
Yes?
Serial killers
Serial killers.
How many female serial killer-- well, you don't even have to be serial about it.
Just regular old killer will do, right?
Now it's interesting that we don't tend to typically think of these primarily as a psychiatric diagnosis, but if you ask who gets in trouble for aggression that's very heavily males.
So sexual assault, murder, various other forms of assault-- very heavily populated by males.
Dissociative disorders in the present era, at least, heavily populated by females.
So why might this history-- well, I already gave you a hint about why this history of child and sexual abuse might be causative.
One way of thinking about multiple personality disorder is as a maladaptive, adaptive response.
What the world's that mean?
Suppose something really terrible is happening to you and certainly childhood sexual assault would count.
And there's nothing you can do about it.
Well one way to deal with that intolerable situation would be to somehow make it something that's happening not to you, to some other entity that is not you.
Now this isn't obviously a conscious choice.
You don't sit there as an abused child and say, oh look, I think I'll create a sequestered piece of my psyche that I can put all this pain on and if I'm clever not even remember that it happened to me.
This is something that happens rather than something that is chosen.
And it has a certain adaptive value in protecting, if you like, some core of the self.
But it's also obviously, not an ideal way of dealing with stress, problems, pain and becomes even less ideal if it becomes a sort of a trope, a trick that you use repeatedly.
All right, well, this worked for this disaster of my childhood.
Now if I'm having relationships, they're kind of stressful.
So maybe I'll wall off relationship person here.
You know, calculus, that's pretty tough and so on.
You could imagine that you develop this trick of saying, you know, we all talk about people who compartmentalize.
Well, this is a case of compartmentalizing where you're cracking up, literally.
You're taking an otherwise unitary self and breaking it into little pieces.
The fundamental mistake Freud made was that he also, his clientele, his primary clientele probably were young women.
And they also were telling him tales of childhood sexual abuse.
He concluded they were wrong.
He concluded that this was a delusion.
Why?
There are a number of possible reasons why he got it wrong.
Not the least of which, is that these were the daughters of men of his own social class-- his neighbors, his friends, people he knew in many cases, and he simply and perhaps, in some sense, lacked the imagination.
Sort of the horror movie imagination to imagine that these women had been abused by the men in their lives who he in fact knew or knew people like them.
He just didn't think that that was plausible.
Plus, he had a theoretical framework that gave him a different understanding of what was going on and we'll come back to that in-- well, no we won't come back to that.
We'll do that right now.
Look, one of the things that Frued believed fundamentally was that the mind was unified.
He didn't see disassociation because he didn't think that disassociation was something that really happened.
He thought that the mind was-- well, here, if this is the dissociation model here's a sort of a Freudian model.
It's not actually his metaphor, but it'll work.
Freud thought that the mind was a unitary whole.
That there was a conscious part of this whole-- that's this little king or queen on the top of the castle.
Ain't it beautiful?
And then this conscious guy got access to a lot of stuff, so here's the pre-conscious and it's sort of like long-term memory or the big semantic network that you've got up in your head or something like that.
It's the stuff that you got access to.
So at this event this Thursday they're going to serve what?
Ice cream
Ice cream, right.
OK, you weren't sitting there the whole time that I've been talking unless you're a pretty weird, obsessive sort going ice cream, ice cream.
Ice cream's in my conscious.
No, it was there.
You could get to it.
It's in your pre-conscious.
Freud acknowledged certainly that there was stuff that you didn't have access to and that was what was living in the Freudian unconscious, which I will put down-- unconsciousness-- which I will put down here sort of in the dungeon.
That's the stairs down.
Somewhat different, both similar and different to the unconscious processes that we've talked about so far.
We've already talked about, in many aspects of the course, the notion that there are processes that go on outside of your awareness.
Freud's unconscious will turn out to be somewhat different.
Before I launch into explaining that though, I ought to explain more generally why I'm going to talk about Freud as much as I am all together.
The reason I need to explain that is that if you were taking this course at most other-- probably a majority of other fine institutions of higher learning around the country, you'd probably get less Freud than you're getting here.
I'm out on one end of the distribution on that.
Many people in academic psychology subscribe to the line attributed to some professor of this course at a big midwestern state university who said, look, there's only two things you need to know about Freud.
He's wrong and he's dead.
Now I wouldn't try that as the answer on the final.
Though both of those statements are in indisputably true.
So all right, why talk about him?
There are a couple of reasons.
One reason is that the influence of Freud is so widespread in the culture that it would be a disservice to people taking an introductory level course not to talk about it at some length because we all use the vocabulary.
We all sit around yacking about our friends and neighbors saying, my he's got a big ego and she is so anal and both of them are in denial and-- oh, I could keep going.
There's a whole vocabulary that is borrowed from Freud.
It's often used in ways that would cause Freud to cringe, no doubt if he heard us using the way we do today, but that's where the language comes from.
The influence is so widespread that you need to know about it.
The other reason is I think that while Freud is demonstrably wrong on point after point after point he is right or at least interesting on a number of large scale issues that deserve consideration.
He's wrong in the way that Darwin is wrong.
Or the way-- pick your favorite 18th, 19th century giant of science-- they're all wrong.
They're also all dead.
But the way we play the science game of course is that we build on-- yeah, yeah, yeah, all right.
So Darwin didn't get this bit right, but I can modify Darwin's theory and I can be a neo-Darwinian or something and build on it.
The problem with Freud was that his disciples and the word is chosen deliberately, treated his writings-- particularly after he was no longer around to revise them-- more closely as holy writ than as scientific theory.
With the result that they're developed in psychology-- particularly in the sort of clinical side of psychology, this sort of you're either with us or against us notion.
That if you questioned one of the tenants of the master you were not just another scientist attempting to revise Darwin or Newton or whoever you want to be revising.
You were an anti-Freudian.
So there ended up being this split between the people who thought that the canon was fixed in some sense and the people who didn't and once you get passed as you're the opposition well, you just sit down and check off why he's dead and he's wrong and we don't need to think about it anymore.
So there's a sort of a historical-- it's not an accident-- but it's sort of an accidental reason why he's fallen out of favor.
My hunch is that were Freud still alive that he would be more than happy to say, oh yeah.
Got it wrong big time on this, this, and this.
Let's see what we can salvage of the theory and move bits around.
I put one Drew Westen quote on the handout.
The other good Drew Westen quote from the same article is that "since Freud's death in 1939 he has been slow to revise his theory."
And that's true.
And that's perhaps a pity.
What Freud was trying to do-- well, look.
It says what Freud was trying to do.
He was trying to understand why we do things and why we think things that we don't seem to want to do or think.
Well look, so was he on to something?
Has anybody here ever done something that they didn't want to do?
Yeah, I know people who weren't putting up their hands are the ones who are still reading The Tech there or something.
Right?
Of course, everybody does that.
Well you can need to do that because somebody's going to hit you if you-- how many people?
So OK, saw lots of hands there.
How many people have ever done something they didn't want to do even without any obvious overt coercion?
Or failed to do something that they did want to do?
You didn't talk to her or you did talk to her.
How many people have ever just done something they didn't want to do and there wasn't anybody with a gun or anything.
So, Freud's onto something there, right?
I mean, this is a serious question why people-- and I should note that this doesn't mean that you're nuts.
It doesn't mean that your self has fractured into little pieces and that the reason that you did whatever it is-- you ate the M&M's or you drank the two espressos at midnight when you thought you were going to bed or something like that.
You know, whatever it is, you didn't do it because Irving-- the other piece of you-- suddenly took over.
It's just the way we seem to be.
For all that we think that we're king of the castle or queen of the castle we don't seem to have the degree of control over ourselves that we might want.
Why is that?
That's the fundamental scientific problem that Freud was trying to get at.
And he was particularly interested in the cases where this went sufficiently off the rails that people were doing things that they didn't want to do that we're getting them into real or at least, psychological trouble.
They were in psychological pain of some variety and needed to deal with this.
Most of us are just doing things we don't want to do and surviving well enough with that.
Thank you.
In the case of multiple personality disorder or of these patients who came to Freud saying that they'd been sexually abused in many cases, Freud concluded that this was a delusional thought on the part of his patients.
What he thought was that these were women saying that I-- or not saying, I desire my father or some other male in an inappropriate kind of a way.
That that would be an unacceptable desire.
And what they had done to defend themselves against this unacceptable thought was to engage in what Freud called projection; an act of the mind where you flip the actors, the subject and the object in that sentence around.
So he was thinking he was hearing daddy wants me and he was concluding that what the woman was saying really was, I want daddy.
He was probably wrong about that, but was he completely wrong about the notion that there's something like projection.
How wacko is that?
Well, all right here, think of the following: How many people here can think back into typically sort of like elementary school, though it could be college, it'll work-- where you had a teacher who hated you.
How many people at least remember thinking that their teacher hated them?
All right, now look.
Odds are that most of you have always been pretty good students, by the way, right?
You didn't get here by being the sort of students who teachers routinely hate even if teachers do hate students, which I don't think they do all that much.
Otherwise nobody in their right mind would teach middle school.
If you were inclined to hate students you'd run screaming from that assignment after the first week.
But anyway, so if you think about that teacher who hated you, you know, odds are they didn't.
But where does this idea come?
You must have been in some conflict with this teacher, at least in your own mind.
It is possible that you conjured up in your mind at some place in your mind the thought, I don't like Mrs. McDonald, my third grade teacher.
But then some other little chunk of my mind says, that's not a good thought for me to have.
I'm not the kind of person who hates my teacher.
So Mrs. McDonald hates me.
That's why she gives me bad marks in penmanship.
I was so glad when penmanship went off the report card.
Which, it was there all the way through elementary school and it was always the black mark on my-- because nobody could read what I was writing.
But anyway, so projection, even if Freud got it wrong about the notion that even if he ended up systematically denying the possibility that his patients were actually abused, the notion that you could have a trick of mind that would do this, you know, subject object reversal that's not as unlikely as it might otherwise sound.
So he systematically didn't believe in dissociation.
It wasn't that dissociation wasn't known.
If we didn't know about Freud, if Freud had not existed I would probably be giving a sort of working historical lecture about Pierre Janet who was the leading-- in France he was a leading psychologist.
I think, actually wrote a book with a title like Dissociation.
Dissociation was something that 19th century psychology believed in and Freud killed off for awhile.
What Freud did was to-- so if we stick with this sort of metaphor land, dissociation proposed that there was rebellion in the kingdom of the self and that province's seceded.
What Freud put up in place of that was a dungeon in the castle of the self where stuff was being imprisioned.
It wasn't different from you, but it was kept away from the conscious self.
Now why did he come up with this idea?
To understand that you've got to know a little bit about what Freud thought he was-- what Freud was out trying to cure.
His patients, a lot of his patients came to him with hysterical symptoms.
Hysteria, which we these days sort use to talk about hysterical-- something that's hysterical is very, very funny.
That's a change in the meaning.
A hysterical symptom was a physical symptoms with no good organic cause.
The classic Freudian or 19th century hysterical symptom is something like a glove anesthesia.
A patient who comes to you saying, Doc I can't feel anything in my hand.
It's like I've got a glove on to here, let's say.
Everything above this line is numb.
Well, there is no lesion that will do that.
That if you cut nerves you get very different patterns of numbness.
You don't get this sort of glove anesthesia.
So in the absence of an organic cause these are the things that got labeled hysterical symptoms.
They could be other things, a cough with no good explanation.
Even headaches with no good expiration.
Symptoms with no good organic explanation were deemed to be hysterical.
Anybody know where the word comes from?
The root of hysterical?
The wandering uterus
Oh, we know lots about this.
Yeah, the root comes from the word for uterus.
And it was in fact a serious discussion in the 19th century about whether men could be hysterics because in case you were sleeping that week in high school health men don't got uteruses right?
Or uteri or whatever the plural might be.
This is a very old idea.
The old idea going back to Egyptian writings was that the uterus could come loose and wander around the body and produce weird symptoms, which you can sort of imagine it would if this actually happened.
To my knowledge it doesn't actually happen a lot.
But there are a lot of terms in the language that we still maintain by the way, that echo the various ancient theories about psychology.
I think I may have mentioned the doctorine of the 4 humors.
So somebody melancholic was melancholy because he had too much black bile.
You were sanguine because you had excess blood.
If you were phlegmatic it's because you had too much phlegm.
And if you were choleric it was because you had too much yellow bile and the balance between these-- it's rather like the big five in personality theory-- that the balance of these big four humors was mainstream European personality theory.
I don't know, probably from Rome up through 16th, 17th century or so.
And those words, to be choleric, to be melancholy are still in the language and reflect this older psychology.
So Freud was there trying to treat hysterical symptoms.
He was by no means the first person to try to do this because hysteria didn't start with Freud.
I mean, there were lots of people running around.
It was very popular in the 19th century, these disorders and even before.
It's worth spending a little time on at least one earlier effort.
Actually, we will now pause for the musical portion of today's lecture if the CD player wants to work.
And let's see-- push the button, play with the volume.
I don't hear nothing.
I hear very little.
Play you silly machine.
You don't want to play, let's try this one.
You play.
That worked.
[MUSIC PLAYING]
Here comes the doctor, ecco il medico.
That's what you're listening for.
[MUSIC PLAYING]
That's the doctor.
Those are the girlfriends.
[MUSIC PLAYING]
Same girlfriends.
This is diagnosis.
For those of you with fluent Italian-- [MUSIC PLAYING]
I'm waiting for the important line here.
[MUSIC PLAYING]
All right, treatment time.
Which in one production I saw involved a giant horseshoe magnet, which turns out to be important.
And this is the critical line.
[MUSIC PLAYING]
She said is, I'm going to cure them with Mesmer's magic stone.
Which in a modern production might be this giant horseshoe magnet.
The girlfriend's are singing.
The boyfriend's aren't singing because they're lying on the ground here and that music is nee-ee-ee-ee-ee.
Now they're singing, oh they're twitching.
They're writhing.
All sorts of stuff is happening.
And the doctor says, stand back.
[MUSIC PLAYING]
Well, that's good.
That worked, as you could tell of course.
And they're very excited.
They're saying, this doctor is worth more than all the gold in Peru.
Oh, now where's the stop?
Stop.
No, stop.
Die.
All right, I think I got it.
There we go.
That's a lovely opera
What opera?
Well, of course that's the question.
I gave away part of the answer here.
You know, I've played this before and I asked, what was that?
And somebody says, opera.
I know that.
OK, so anybody know wrote it?
Yeah?
Mozart
Oh, good.
All right, you don't turn out to be like a person one year who sang the role?
No
OK.
I thought that was pretty cool.
This is definitely Mozart.
OK, now for the $200 question, do you know which opera?
[UNINTELLIGIBLE PHRASE]
She's good.
Giver her a round of applause.
With an e, right?
Not an i?
And of course, the relevant question here is, why?
And for that I got to tell you a little bit of the plot.
So what's going on here, we'll do this in good MIT terms.
So we need some equations I think.
Or at least symbolic representations of the plot.
What we have is two guys and two women and the X factor here.
So they're together.
They're in love, they sing beautiful stuff.
And their buddy comes along and tells the guys, women are all fickle.
The cosi fan tutte means something like they're all like that.
And you can't trust them and the guys sing for another few hours.
No, no our girlfriends are good and true and stuff.
So here's what we'll do.
You guys pretend you've been called up to go to war and then you sneak back into town disguised as Albanian soldiers and we'll try the cross product or something like that, right?
We'll see if we can make this work.
And so a pile more singing ensues and the girls are being very faithful.
And so the guys decide they're going to resort to desperate measures.
Well, not terribly desperate, semi-desperate.
They'll fake taking poison and because there's so much of [UNINTELLIGIBLE] take [UNINTELLIGIBLE] they're dead.
And so the women are very upset by this.
And so they need to cure these guys.
Now there's a maid whose name is Despina, so she can be D. That's who you heard singing as the doctor.
She pretends to be a doctor.
It's a comic role rather than a terribly tragic-- we've taken poison thing.
And what she's doing is attempting, she's claiming to cure them with Mesmer's magic stone, which is basically a magnet.
Now you've heard the term mesmerize-- to mesmerize somebody.
That actually comes from one Franz Anton Mesmer who was working in Vienna and then later in Paris at the same time as Mozart.
In fact, he was a friend of the Mozart families.
And this was a time of a lot of scientific advances in the study of magnetism.
And Mesmer concluded that there were magnetic forces everywhere.
And the planets in particular, had important magnetic forces and that these could be channeled if you were an appropriately receptive individual.
You've heard of animal magnetism, right?
Well animal magnetism, a term which has also lost its original meaning-- animal magnetism, which today is sort of like I'm awash in musk, or something like that.
You know, I've got great animal magnetism or whatever.
It originally is Mesmer's term for the notion that you could channel or specifically that he could channel these magnetic forces of the heavens and channel them for curative purposes.
And he set himself up with a clinic in Vienna and then in Paris to do just that.
It sounds like it must have been quite a scene.
If you went to Mesmer's-- Mesmer would have liked this room actually because he was in to purple, apparently from what I read.
But in any case, you'd come into Mesmer's sort of waiting room and there would be a string band playing quiet music on magnetized instruments.
And a tub often, in some of his establishments.
A big tub filled with magnetized water and people could sit in the tub or there were sort of like metal rods coming out of the tub that you would poke into the part of your body that was afflicted.
But this was all sort of warm up.
The critical piece was when Mesmer himself would sweep in-- from all the descriptions-- like something out of Harry Potter in flowing purple robes with a purple hat on.
Extremely charismatic guy to all appearances and then what he would do is he'd come in and lay-- in fear you'll forgive me-- lay hands on you and you having bought into this general notion and rather like this notion that you would know what the options are for going insane you would also know that your job here was to have a seizure and to lose consciousness.
So you don't need to do that, it's OK. You'd then be carried off to another room where you would recover and with luck, your symptoms would be ameliorated.
Now what Mesmer had discovered or rediscovered since the various forms of it over history was hypnosis.
To mesmerize and to hypnotize have all a similar kind of meaning to them, particularly in the earlier literature and it probably didn't do much for the people who came to him with cancer and various other things that 18th Century science couldn't do anything about.
But the people who came to him with hysterical symptoms apparently got some degree of relief.
Now hypnosis fell into disfavor.
In part because he was a bit of an act and the Paris scientific establishment was very dubious of him.
They put together a panel to investigate him.
The panel is an interesting one in its own right.
Its members included Benjamin Franklin then U.S.
Ambassador to Paris.
The chemist Levousier and one Dr.
Guillotine, later famous for a different invention.
But they concluded that he was a quack and he died in disgrace.
And hypnosis has come in and out of fashion.
As we talked about in the context of multiple personality disorder it's considered an important tool for some therapists today.
In the late 19th Century it was revived, again, in Paris.
Particularly, by a psychiatrist named Charcot.
Charcot was at the leading French Parisian mental institution.
Had some very disturbed patients and was working to treat them with hypnosis by inducing symptoms in them and then suggesting-- well, basically putting them under hypnosis and telling them how to be.
It was essentially manipulating a passive kind of a patient.
Now Freud was in Vienna at the time.
Came to Paris to study with Charcot because Freud wasn't sure what to do with his patients.
Let me tell you about one of his patients.
Known in the literature as Anna O.
And she's known in the literature as Anna O. because when Freud wrote about her he attempted to preserve her privacy by giving her a pseudonym.
Her real name is Bertha Pappenheim.
I'm not actually violating patient confidentiality here, like almost every important patient Freud ever had, she wrote a memoir.
You can spend your life reading the memoirs of Freud's patients.
Anyway, she was Bertha Pappenheim, but known in the literature as Anna O.
She was a grab bag of hysterical symptoms precipitated most immediately by the crisis of attending her father through his final illness.
And she was actually being treated originally not by Freud but by Freud's colleague, Josef Breuer whose name I think I put on the handout.
Yes, there we go.
And they took to using hypnosis, but they took to using hypnosis in the new way.
Rather than using it to manipulate the patient or to cause a seizure or something like that they used it in the context of what Anna herself called a talking cure.
They talked to her.
Under hypnosis she seemed to be able to recover memories that she could not reach otherwise.
That these memories were often accompanied by violent emotional reactions.
And that this emotional eruption had an effect of purging the symptom.
Borrowing a term from Greek tragedy, Freud called this catharsis.
Borrowing a term from the neighbors or something, Anna O. called this chimney sweeping.
The difficulty was that it seemed in her case to be rather temporary business.
In fact, Freud abandoned hypnosis eventually because of this problem.
That hypnosis seemed to be useful for treating symptoms, but somehow not getting at whatever the underlying cause was because she'd go away, her symptom cured and then something else would pop up and she'd be back for another round.
Now the reason we know about Freud and not Breuer is that Breuer ended up running away from this and Freud ended up jumping into it.
What Breuer ran away from was Anna coming back with a new symptom and this new symptom was a hysterical pregnancy.
Which is to say, she thought she was pregnant.
She was convinced she was pregnant and she was convinced Breuer was the father.
Now there's no particular evidence that Breuer was sexually involved with her at all, but you've got to imagine being a psychiatrist with a practice with young women, you know what's it going to do to your reputation if one of these young women even mistakenly thinks she's pregnant?
Doesn't matter that she's not pregnant.
Matters that she thinks she knows how she got pregnant and that's just trouble.
And he said, I can't see you anymore.
But Freud on the other hand, I believe actually continued to treat her, but more to the point, what Freud became interested in were the more fundamental questions like look, if there really are these memories here somewhere that we're only getting to by this hypnotic technique in this particular case, where are those memories?
And why do they only come out when we're doing this psychotherapy with her?
Why are they hidden?
And what's their connection with the symptoms?
Those were the questions that he really wanted to know the answer to.
He wanted to be able to treat patients too, but this was the sort of underlying interesting, scientific questions that he wanted to get to.
And he came to the conclusion that there was something that was actively interfering with these memories.
It wasn't just like you can't remember the answer to some multiple choice question on the final because you're just having a retrieval problem.
He thought that in these cases there was something that was actively fighting against you remembering anything, and he wanted to know what that might be.
The heart of his depth psychology was the idea that these thoughts that they were dredging up were thoughts that had been locked away in the Freudian unconscious.
Different, if you like, from the cognitive unconscious from earlier in the term.
These things had been deliberately locked away down here and kept away from consciousness.
He didn't invent the idea of the unconscious.
It existed as sort of a literary metaphor in 19th century European writings, for instance.
But he gave it a place in a theory of the mind.
And he had an explanation for why it was populated with these thoughts that were somehow hidden from the user.
I'll tell you about that at 3 o'clock
If, like you said, if most of the psychology functions on Freud was wrong, then why are psychologists and scientists still so obsessed with proving that he was wrong?
I think most would answer, they're not.
That at this point it's a done deal.
But to the extent that they are-- no, to the extent that they are it's because Freud is still, you go out on the street and ask people, tell me the first ten things that come to your mind if I say psychology.
One of them is going to be Freud and two others will be terms that have something to do with Freud.
And if you think that that's all hooey it's going to drive you nuts and if this is your own particular area you're going to work to kill it off, I suppose
Because when I was writing my paper and I was reading all these books about sex and sexual orientation and how it comes to be developed-- like half the books were like well, Freud is wrong
That'll depend a little on how old the books were that you were-- but Freud was actually, if you're talking about sexual orientation, Freud was actually more interesting than his followers.
Freud regarded homosexuality as not as a biological issue, but as a developmental issue.
But did not think that it was abnormal.
He thought it was a variance.
There's a famous letter written to the mother of a gay man where he says basically, it's not the mainline path, but don't worry about it.
It's just one of the ways that things work out.
The American psychological establishment then declared it to be a disorder and only in the '70s declared that it was not.
But Freud in that sense was-- well, assuming that we are right at this point, that it is biological in its roots he was wrong, but he would have been cool about that, I think
I have a question about hypnosis.
Is it true that they can't make you do something that you don't want to do?
I would lecture about that a bit, but I'd run out of-- my hunch is that it's not really true.
I think that it's a little like this Mesmer thing.
That we have a set of things that we believe about what hypnosis does.
It can make me behave like a child, it can make me behave like a chicken, it can't make me murder you.
I don't know of any clear evidence, one way or the other on that.
I mean, it's not complete sucession of the will in some sense, so there is some chunk of you saying, I'm not going do that.
But could you make somebody do something modestly wrong?
I bet you could.
But anyway, very interesting topic.
Freud's understanding about what was going on or what was stuffed away down here in the dungeon of the unconscious starts with Freud's understanding of what it means to be a baby.
He thought that babies came into the world as amoral creatures.
Certainly his contemporaries at the time tended to misunderstand this to mean that he meant that babies were immoral.
That wasn't what he meant.
What he was really doing, had he taken my version of intro psych he probably would have drawn one of those little developmental pictures that I was drawing when we were doing development and said, well look, here's units of moralness or something.
Here's time.
Here's the adult state and I, Freud, am asserting that babies-- that the starting place is here.
And that there's some function that gets you up to the adult state and isn't that interesting.
The naive, probably Victorian Era notion was probably something more like babies start here.
They are pure, they have little wings and only when they grow up do they become sinful, nasty things like you and me.
And so this notion of babies as amoral agents was not something that his contemporaries were really thrilled with.
Freud was probably wrong about this, right?
We've already seen that babies seem to have some rudiments of something like empathy, so all right, let's start them there.
But the larger point-- that babies do not come into the world with anything like adult moral reasoning is an interesting one because you've got to imagine as Freud did, what's going on in the little kid's brain.
Sorry, little kid's mind.
So here you are, your little amoral baby, let's give that to Freud for the time being and I don't know, a year later there's a new baby.
New baby wants stuff.
Wants stuff that you used to get all the time, like Mom, for instance.
So you're amoral, there's a baby, what do you want to do?
Get rid of the baby
Get rid of the baby.
Well, how we going to do that?
[UNINTELLIGIBLE]
Eat it, that might be good.
We'll kill it in some fashion, the details aren't terribly important if you're a one year-old.
Let's just kill it.
The nice thing is it's not easy for you to do that so that your little brother grows up anyway.
You know, there's lots of things like that in your life where not a late little baby brother, but like Dad.
Dad's a problem.
Man, he comes home and I've spent all day with Mom and Mom's been nice and everything and then Dad comes home and Mom's ignoring me.
Let's kill Dad.
And let's take Mom all for ourselves, stuff like that.
So what's driving the baby?
Freud proposed that what's driving infants and young children is what he called the pleasure principle.
Doesn't take an awful lot to decode that.
Freud thought, baby wants pleasure, baby wants it now, right?
But look, you're going to work your way up, if you end up as an adult in that state you are a psychologically deformed adult.
An amoral adult is a real problem.
An adult who's governed entirely by something like the pleasure principle is a real problem.
It's not going to work.
So you're working your way up there and you've got to imagine someday you wake up and you've moved to the next moral level in some fashion.
Now how do you know who you are?
Well, you ask yourself, what kind of a person am I?
And you go and examine this stuff and you say, I'm a good, honest, loving person except for that time.
All right, so you wake up, you look in your head and ask, what kind of a person am I?
And you go rummaging around and you say, kill little brother.
Kill Daddy.
Hurt Daddy, grab Mommy.
Hurt mommy because Mommy didn't like to be gra-- wait a second.
I don't like this.
This is not a me that I can live with.
A me who is-- look, or imagine this.
Suppose you woke up tomorrow with your head full of thoughts that said, I'm going to kill my roommate.
This is how I'm going to do it.
I've been thinking about this through.
If I'm describing you, by the way well, we should talk.
But you would be deeply disturbed if you had what felt to you like an obsessive set of thoughts about going off and doing harm to somebody.
Or sexual thoughts.
Every time your mind is completely filled with thoughts about doing something with somebody and it's for some reason either inappropriate, unlikely, or whatever.
If you got a brain full of thoughts like that, it's going to be disturbing and Freud figured that that's basically what happened as kids got older.
They found that they had a mind full of these infantile thoughts that were simply unacceptable and they stuffed them down here.
That's Freudian repression.
They repressed these thoughts.
Now that seems stupid.
Why do that?
Why don't we just get rid of them?
Well, you have only rather limited abilities to actually deliberately forget stuff-- as you've probably discovered at various emotional times in your life.
You know, you break up with somebody and you think, I'm just going to wash that man right out of my hair or something like that.
It turns out to not be quite that easy.
And there are probably good reasons for that like waking up-- well, not waking -- going to class, getting back the calculus test with the score on it in single digits and thinking, oh man, I hate this.
I wish I could just forget calculus completely.
Oh man.
No, it wouldn't be good if that worked.
That's why your computer asks you, do you want to erase your disk drive?
And your version of this is to simply not have that ability in any very comprehensive fashion.
So you don't have the option of just getting rid of these thoughts.
They're there, so you stuff them down here.
Now an important piece of this is that what Freud is describing is his view of normal.
Again, Freudian jargon that we tend to use out on the street, you say somebody-- oh, he's so repressed.
Sounding like that's wrong.
Freud thought that this sort of repression was absolutely required for adult psychological human life.
You know, maybe not if you're a chimp or something like that, or a gorilla.
But if you were going to be a human, you were going to need to repress this stuff because-- the patients he was seeing in his vew were patients where this had gone bad in some fashion, where it wasn't working for them.
But all of us, in Freud's view are built like this.
And all of us have an unconscious that's filled with these unacceptable thoughts and all of those unacceptable thoughts, there is no capital punishment down here.
You may be able to forget everything that I taught you in psychology, but it turns out -- according to Freud-- that you're not forgetting any of this stuff.
It's there and it's like a collection of prisoners who are trying to make a break for it.
They always want out.
So if you, the king, got this jail full of characters who want to get out and trash the castle, you'd better have guards and the defense mechanisms like projection that I talked about before are the guards who are there to keep the repressed material repressed or to channel its energies in ways that are harmless to you.
To avoid exposing to you the stuff that would be unacceptable.
So this stuff, the sitting here protesting, how does it make a break for freedom?
Typically, by association with something in the outside conscious kind of world.
So you know, you wanted to kill off your little brother and now you're playing football and-- I'm making this up-- and there's a little guy playing football and some chunk of all of this says, association time.
Association time.
Reminds me of little brother.
Let's kill it.
Let's kill it.
A little bit of that channeled -- when you channel it into sport it's called sublimation in Freudian terms.
But you know, that's OK. Good, hard tackle or something.
If you actually then pound him into a pulp that's less good and you've got mechanisms there to keep you from doing that and to keep you from ever recognizing any sort of association with any of these sort of infantile thoughts.
So I already mentioned projection as one of these devices.
You are angry at your brother and you conclude that your brother is angry at you.
There are a bunch of them.
They're talked about in the book.
A bunch of them are rather straightforward in the sense that the term defines itself.
Denial, simply saying, no, I don't think that.
I don't hate him, not me.
Or that rationalization.
The great rationalization in my family is a canon of stories.
I have twin sisters and one of them reported to my mother that she had pushed the other one down the stairs on purpose by mistake.
The nice thing about little kids is sometimes-- you can sort of understand where Freud was coming from if you watch little kids because sometimes the mechanisms that Freud's talking about are more transparently visible.
So there she is giving some sort of an account, a rationalization of this act that she doesn't want-- not that a little kid is going to say look, I really pushed my sister down the stairs because I have these deep infantile, hostile, intents towards her.
I'm really tired of having a twin sister and I was trying to kill her.
That wouldn't be something that fortunately, my sister would say.
Another example where you can see this in action in little kids, you can sort of see where Freud was coming from, is the defense mechanism that he called-- maybe it's his daughter who actually named it-- but it's reaction formation.
Reaction formation, it's easy to confuse with projection.
Projection is you know, I hate you is the truth and you hate me is the result of the projection defense mechanism.
I decide that you hate me.
In reaction formation the flipping is a little different.
I hate you, but I decide I really love you.
You flip the emotional around to an acceptable emotion.
And the place to see this best-- how many of you have younger siblings?
Can any of you remember an occasion where you were told to stop hugging your little siblings so hard?
No, it doesn't ring any bells.
You see this all the time.
I just saw it with sisters-- three year-old and a one year-old, and it's an absolute classic if you're in a Freudian frame of mind.
An absolutely classic reaction formation-- I love you so much that I'm going to hug you till you turn blue.
And you know, what's that about?
The kid doesn't do that.
The observation is that it is unusual to see a kid take their teddy bear, for instance, and say, I love you so much.
Squish, squish, squish.
Uh-uh.
It's a characteristic behavior of older sibs to younger sibs.
So it kind of looks like Freud might've been onto something.
Again, all normal stuff that you've got to have.
It's not that reaction formation is sick, weird stuff or that denial means that you are an emotional cripple or something like that.
Freud says, the only way that you can function is by having a set of mechanisms to defend yourself against this stuff.
And what he thought he was seeing when he was seeing patients is cases where this didn't work so well.
Let's continue with my football example.
There's this guy who somehow reminded you of your little brother, you've tackled him, all right already, but now you're reaching for the axe that's conveniently located on the sidelines or something like that and these defense mechanism are sitting there working overtime saying, what are you going to do here?
Can we make him love him?
No.
Man, we don't have enough time.
Let's just repress the whole thing.
We'll repress everything.
Forget it.
Forget all this business and forget the whole arm too.
We got to act fast to prevent a complete disaster from happening.
And so you end up in Dr. Freud's office saying, Doc, I can't move my arm.
I have no idea why.
And the job of therapy, what Freud decided once he abandoned hypnosis was that what was therapeutic was to get that hidden material out in a safe atmosphere where it's energy could be dissipated and that that would be curative.
And his method of treatment was designed to do that.
The idea of lying on a couch, the great image that everybody has of Freudian psychoanalysis is of a patient lying on the couch with Freud behind him-- with the analyst behind him not looking at him because there's something different and in a sense protected about conversations where you're not making eye contact with the person.
The closest that most of us get to this is conversations had in the car.
I don't know if you've had this experience, but where both of you are sitting in the car looking out the front window, you can have conversations that you just don't have if you're looking at each other over the dinner table or something like that.
I certainly know that I've had conversations with both my parents and my children of that form where it really helps.
One of them is the passenger, but the driver is not doing this because you're going to all die.
It makes a real difference.
All right, what do you do then once you're lying on that couch?
Well, what you don't do is lie down and say Doc, my hand is paralyzed and he says, well do you hate your brother?
That's not the way it works.
The classic form of a psychoanalytic session would involve you lying there and just talking, doing what Freud called free association; saying whatever came into your mind.
So here's your mind and in here somewhere is this nugget that we're trying to find.
And Freud just sets you loose.
And you're wandering around in semantic network space, if you like, or something like that.
Oh you know, here I am.
I'm lecturing in intro psych and that reminds me of other forms of combat and oh yeah, reminds me of playing football, which I don't do.
Remember that time I was playing the guy and he reminded me of my brother and I thought I'd kill him-- no.
That's not what happens.
It's not that you somehow get sucked into this.
You're wandering around and by chance at some point you bump up against this, and what Freud was looking for was not some flash of realization that-- oh my goodness, I just found it!
But you'd be going oh, [UNINTELLIGIBLE] and football.
You know, I have a really interesting math problem I'd like to tell you about.
Yeah, OK?
Tell me.
[UNINTELLIGIBLE PHRASE] And all of a sudden you're going that way and now you've reversed completely.
What Freud was doing was listening for places where you ran into a barrier of some sort.
Could be a jump in the logic of the talk.
It could be sort of a reversal in direction, if you like.
It could just be a pause.
That all of a sudden you couldn't think of anything.
Why aren't you talking?
I got nothing to say.
Yeah, right.
Or it could be a speech error.
Another place where I think Freud was wrong, I don't remember if he actually ever said what's attributed to him in this case, but Freud certainly believed that speech errors were informative.
He may have said that all speech errors were Freudian slips.
I mean, he wouldn't have called them Freudian slips, but were the result of a defense mechanism.
That's not right.
When your mother calls you by the name of your sibling, it's probably not that she loves the sibling more than you, or something like that.
It's probably just that they're stored nearby and in the effort to get speech out you grab the wrong thing.
There are lots of speech errors like that, but there's no denying that there are speech errors that seem fraught with meaning.
So years ago there was a faculty minicourse here given by the course 21 folks where faculty could go and we could sit and read Shakespeare together.
Lots of fun.
The then head of the literature section of 21 was lecturing about Measure For Measure.
Measure For Measure has a duke in it.
He wanted to talk about the evil duke and out of his mouth came a sentence about the evil dean.
This was a man who everyone in the audience knew was fighting tooth and nail with the then dean of humanities and you listen to that and you just say Freud was on to something.
But at the same time, almost exactly the same era, I'm walking down the street with my wife, I look off to one side, I say to my wife, look at that beautiful bunch of lenses.
And my wife looks at me like, oh there he goes again, but then looks over there and realizes that what I had seen was a collection of irises, the flower.
If you are a vision researcher-- lenses-- you got it.
Look it is unlikely that what that was doing was reaching some deep unconscious desire of mine to poke out my wife's eyes or something unacceptable like that.
It with just a speech error.
And I suspect Freud knew that at some level too, whatever he may have written.
But what his job was as a therapist was to slowly, painstakingly listen to his patient and see if he could find the lump.
It's like those things you do at Halloween haunted houses.
You know, you stick your hand in some place where you can't see, it's always spaghetti or something.
No, it's not spaghetti?
It really is brains?
Well, anyway, so you don't know what it is.
You're feeling around, that it's sort of like that.
Oh, and last bit, since I know the terms are on the handout.
Two other clues that Freud believed were useful were what he called transference and countertransference.
In the protected environment of the analytic session, analysts train themselves-- Freud trained himself to be as emotionally neutral as possible.
And have the setting as neutral as possible.
And nevertheless he would hear from time to time, with actually some regularity, accusations from his patient like I'm having real trouble dealing with the hostility that I'm getting from you today or something like that.
And he came to the conclusion that what this was was a transference of emotions from the outside, from the real problem onto the protected setting of the analytic session.
Anna O.
's hysterical pregnancy could be seen as a sort of transference here.
That a love or a romantic attachment that might have been appropriate out there somewhere was being brought into the analytic session.
Part of the idea of keeping yourself emotionally neutral was that you might catch this feeling yourself.
Not hear it expressed directly by the patient, but you might find yourself feeling somehow inexplicably hostile to your patient.
That's countertransference.
You might think of it as a sort of a form of empathy.
But it's sort of an odd form of empathy where you're feeling emotions that you don't think are really yours as the analyst.
It's not that you love your patient or that you hate your patient or that you are actually anxious.
It's that you're picking up this as though you were an antenna for your patient.
Now of course the difficulty is, do you really think it's possible to keep completely neutral about somebody you're working with, maybe several times a week?
How do you know when an emotion that you're feeling is really a countertransference, you the analyst, or when it's really just a lapse in your own behavior?
These are the sorts of problems that worried analysts in their practice, but the broader goal here is it's all detective work.
You're looking for that lump so you can bring it into the open, diffuse it, and release it so that it doesn't trash the castle.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license at MIT OpenCourseWare in general is available at ocw.mit.edu
Good afternoon.
I want to do two things today and they involve two sets of Freudian jargon.
I want to talk about Freud's so called structural psychology that involves the terms id, ego, and superego and will give me a brief chance to talk about Freud's theory of civilization as a whole.
And then I want to talk about Freud's developmental psychology which is where we get these notions of an oral, anal, phallic, and then onto latency in adult stages of development.
And that'll give me a chance to talk about fairy tales or more broadly about the practice of using Freudian ideas to interpret literary texts.
The second part may well run into next Tuesday as I think about it.
Freud thought that babies came into the world thinking that they were co-extensive with the universe.
That they were everything.
Not that they were looking out in space saying, look, the stars-- they're me.
But that what the initial experience of being a conscious entity was thinking that you're it, you're everything.
It's not as though that's new with Freud.
I won't attempt to do a reading of the piece of Tennyson that I put on the handout because I discover I'm a bad reader of Tennyson.
But if you read that you will see Tennyson saying very much the same thing.
That the job of an infant is to learn the use of I and me and finds, I am not what I see other than the things I touched.
You've got to figure out what's you and what's not you.
Freud thought this little bundle of-- that this universe of a baby wanted one thing, which talked about last time.
The kid wants pleasure and the kid wants pleasure now, anyway he/she can get it.
That collection of unbridled desires is what Freud called the id.
Now this all looks like very fancy or not very fancy-- looks like jargon, but in Freud's original writing it's not particularly jargony.
It is just a Latinization of the word for it, the animal piece of you.
The it that wants stuff and wants it now.
The id is essentially unconscious.
And well, actually to combine the two bits of the -- where does the id get its pleasures?
Freud's series of developmental stages were driven by a notion of where the sources of pleasure were in the kid's life and so initially for instance, it's all oral.
The fact that this all has a sexual overrun to it is in fact, probably more problem for Freud than useful for Freud.
All right, so the kids get some-- oral stuff-- that what's giving the kid pleasure.
That's sort of interesting.
What really is important in this early stage of development though is well, one of the things is figuring out who I am.
How much of this universe belongs to me?
And is this universe a safe place?
The real force behind Freud's stages of development are these stages where particular issues emerge really for the first time.
Issues that will then turn out to be important for the rest of people's lives.
So the initial issue that comes up in this oral stage of development is safety.
Am I going to be fed when I need to be fed?
Am I going to be warm enough?
Am I going to be taken care of?
That's newborn stuff, but it's not stuff that goes away.
You know that concerns about, is the world a safe place is the sort of things that can occupy your mind now too.
How you understand the world, how you understand those problems says Freud -- not unreasonably -- are likely to be shaped by these early experiences.
If your early experiences is of a safe, comfortable world you're going to treat later challenges differently than if your early experiences-- who knows if mom's ever going to get around to feeding me again.
That's going to be a different kind of experience.
The anal stage-- all right, why is it anal?
Well, it's anal because Freud had some notion that there's certain pleasures in the elimination process and things like that.
But the real issue here, the life-long issue that emerges is who's in control?
And the reason that's an anal stage issue is because this is the point at which if you're an infant it's not just anything goes anymore.
Up to this point the first great crisis of control in a little child's life is very often toilet training.
Up to this point you could do whatever you wanted, wherever you want to do it.
Now all of a sudden somebody's saying, do it there, now.
Don't play with it.
Don't do any of the various-- it wasn't disgusting before.
Back when I was very little they looked at it-- oh, great [UNINTELLIGIBLE].
Now it's disgusting and I gotta put it over there.
What's this about?
And you got to figure out how much control do you have over yourself?
And how much are you in the control of somebody else?
Here the issue is identity.
Who am I?
Identity.
Who am I?
And more specifically, what does it mean to the male or female?
I'll come back to the details of that a little later on because it gets me a little ahead of the story.
Presently we're still back here with little Mister Id.
This unbridled collection desires who wants everything and wants it now.
At some point the id runs up against what Freud called the reality principle, which is you can't get what you want all the time, exactly when you want it.
If you haven't noticed this at this point that's probably a little grim.
The ego, which is just the Latinization of the word for I or the self.
The ego grows out of the collision of this little Miss Id with reality.
So the ego has to check the desires of the id.
And the ego is sort of the embodiment of this effort to figure out how much of the universe really is me.
So the sort of thing you can imagine a kid discovering is all right, I'm an oral stage baby.
I like sucking on stuff.
There are some things that I can suck on anytime I want.
You know there's this thumb thing-- anytime I want it it's there.
There's other things, like the bottle or the breast or whatever, they're different.
I like them, but they're fundamentally different.
They're not me.
At the most basic level you have to figure out how much of the world is you.
Now this again, ramifies through the rest of your life.
If you grow up considering that you are co-extensive with the universe and that you make the planets turn in their orbits, you're going to be an odd adult, right?
You've got to shrink it down from that.
If you shrink down too far-- I'm not in control of anything.
This didn't work out well for me.
I'm not in control of everything.
I'm just a little worm pushed around by force-- you know, that's not going to be very healthy either.
So the job of this emerging ego is to come up with some reasonable estimate of what your powers are and what it is that you might be in some sort of control over.
So now the ego's busy there fighting with the id.
So that the id is saying I want to kill my little brother.
The ego says, you can't do that.
The id says, how come?
The ego says, well because mom would whack us.
And the id says, I don't care.
I still want to kill him.
And the ego at this point starts to say things like forget it.
We're going to repress that idea.
This is where you get the beginnings of ideas of repression that I was talking about last time.
All of these Freudian bits of theories attempt to interlock in some fashion.
So the ego is busy taking the more unacceptable bits of the id and stuffing them away in that unconscious reservoir of the repressed.
Now you'll see that that's gotten you from an id who just was ruled by pleasure.
This really essentially amoral id.
You've now got the beginnings of a sense of morality here.
It's not very sophisticated, but it's on the one hand, this would be fun.
On the other hand, we can't do it because we might get punished.
So it's sort of a reward and punishment.
Why not rob the bank?
Because you'll get arrested and thrown in jail.
It's that sort of crime and punishment kind of morality.
Morality gets more complicated when you reach this oedipal stage of development.
To explain that I need to say a little more about that stage, which I can see that on the handout I promised I would.
I'll do the male version of this.
If this is about identity and particularly, sexual identity-- there's going to be a male story and a female story.
Let's do the male story and we'll come back to the female story.
The large scale notion that makes some degree of sense is that you've got to figure out what does it mean to be a male?
That's an important thing to figure out and it's likely that Dad or other adult males in the immediate vicinity are going to be sort of a model for what that's going to mean.
The difficulty for Freud or the start of the conflict in an oedipal conflict is that the child starts very much attached to Mom.
It's Mom who provides nourishment.
It's Mom who gave birth to you and so on.
How are you going to get an attachment or identification with Dad?
Well, what Freud proposed was that little kids, little boys-- stick with boys here-- little boys initially see themselves in competition with Dad.
They like Mom.
Mom is great.
They want Mom.
All right, it's called oedipal because Freud saw a parallel with the Greek Myth of Oedipus.
Oedipus, who kills his father and marries his mother.
So you get this weird notion that somehow Freud thought little boys wanted to sexually possess their mothers and that sounds both weird and icky.
Freud didn't do himself any help here, he was talking about infantile sexuality.
Everybody heard the sexuality bit and sort of left off the infantile bit.
Freud is not thinking in any sort of adult sexual terms here.
He's saying look, the little kid likes Mom.
Mom's a good thing.
He wants Mom all to himself.
Now it turns out that there's Dad.
Dad mysteriously seems to have some claim on Mom, too.
And so the little kid in this infantile kind of way figures, I'm going to have it out with Dad.
We're going to have a fight here man because there's only one Mom and I want her.
I've still got a lot of id going on here, I want Mom and-- all right, that's Stage One.
That's the conflict stage.
I can't remember what buzz word I used for the second stage.
Oh, the second stage is capitulation.
Well, there's a problem with this conflict.
OK, I'm going to have a big fight with Dad.
Dad.
Oh man, like Dad's really big.
If Dad ever figures out that I want to have a fight with him Dad is going to kill me.
Maybe he's going to castrate me, I'm not sure.
But it's something really, really bad is going to happen here and so I better deny that I was ever interested in this.
In fact, what I'm going to do is I'm going to reject, suppress my desire for Mom and I'm going to idealize Dad.
I'm going to come to take Dad, in response to this perceived threat, perhaps to my very life I'm going to idealize Dad.
And in some sense incorporate Dad into me.
And that act of incorporation is the beginnings of development of what Freud called the superego.
Another Latinization, this time of a term that just means sort of over I.
You know, the thing that is above the self.
It's not quite your conscience.
It's not unrelated to your conscience, but it initially for Freud starts out as the voice of the father for a little boy.
Well, let's just say voice of parent.
And will eventually become the voice of society as a whole telling you how to live.
Not necessarily consciously, not necessarily explicitly, but telling you what the rules are.
Now again, you might think that this whole business of a conflict with Daddy over Mommy and the notion that you have some internalized voice of the parent that is going to be sort of the roots of your morality, your more sophisticated adult morality.
That that all sounds a little strange.
You can get a feeling for where such thoughts might come from again, if you hang around with kids.
Remember last time we were talking about you know, hugging your little brother till he turns blue.
Well look, I've now raised three sons through the oedipal stage of development.
I think every last one of them at some point in the ages of around three, four the appropriate Freudian age has hopped into bed some Sunday morning or something like that-- into our great, big, huge, king-sized bed and asserted this bed isn't big enough for the three of us.
Why doesn't Daddy get out?
And mostly what you're sitting there thinking is, no kid should never be born to a psychologist.
What's he doing, reading the books on the side here?
And then I thinking, what am I supposed to do now?
You know, am I suppose to like threaten to kill them or something because that develops morality?
So you can really see these sorts of seemingly oedipal comments in kids of that age and you can also sometimes here this voice of the parent as it gets internalized, like a little kid who's been told, the cake's over here.
It's for dinner, don't mess with it.
Don't go sticking your finger in the chocolate icing again.
So no, no, no, you hear the kid say it.
The nice thing about little kids is that the superego will verbalize itself for you-- no, no.
So the idea is that the stress or the trauma of this oedipal conflict is what drives the superego down into the psyche almost like a spike from the outside.
Well, I would be remiss if I didn't say something about Freud's theory about how this works out for women.
Now I might be remiss, but you wouldn't be missing anything in terms of-- people have all sorts of problems with this oedipal theory business.
Because for instance, what happens if you get raised in a single-parent family with just Mom?
Is there any evidence that your development comes out differently?
No, in fact.
So you don't need Dad there.
You do need to figure out what it means to be male.
Let me say a quick word about homosexuality.
Freud regarded homosexuality seemingly as a variant of an outcome to this.
He did not seem to regard it as a pathology.
So he's got a very interesting letter written to the mother of a gay man.
The mother was worried about this and he says basically, there's a bunch of ways this plays out.
The mainline one is in the words of the old song you grow up to, I want a girl just like the girl who married dear old Dad.
That's the sort of ultimate goal of this.
In sort of the mainstream version you reject your desire for Mom, you go into this latent period.
You go and you do math for awhile and then you wake up as a sexually mature adult, not looking for your mother, but looking for an appropriate object of your desires.
Freud thought that there were other possible outcomes.
That they were just other possible outcome.
Subsequent generations of American psychology declared homosexuality to be a pathology and it was only in the '70s that American psychology returned in a sense to this view that it's a variant, not a problem.
So women-- how did we get to women.
What's the problem with women?
The problem is you need the same fight thought Freud.
He's got himself sold on this oedipal thing.
And you need the same fight, but now you need a fight between the daughter and Mom for Dad.
And the problem is that the initial bonding of the daughter is also to Mom.
So that's no good because you can't fight with the person you're bonded too, so he's got to get the bond to flip.
And so he came up with one of his less credible constructs-- the one that even good Freudians don't really believe in, which is penis envy.
What is that?
Well, he figured that little girls at some point figured out that they did not have something that little boys had, and that this was a disaster.
This was very bad and they figured out that Mom didn't have one either.
And therefore, Mom was defective and we ought to admire Dad instead.
And then we can go and fight with Mom.
This was traumatic, this discovery.
But it wasn't so traumatic, so Freud also theorized that since it was that trauma that was creating the superego and as a result, adult morality that it followed that women were simply less moral than men.
A conclusion born out anytime you open the newspaper, right?
It's not one of the more successful pieces of Freud, but it is the source of my favorite, really bad experiment in this realm.
Which was an experiment where somebody gave a paper and pencil test to a group like you.
One of these sort of SAT things where you're filling in the little things with the golf pencils and stuff.
And then you hand the thing back and the experimenters don't care a bit about the test at all.
All they care about is who returned the little pencils.
And more guys returned the pencils than women.
Anyway, it's really lame.
So look, the broad issue of what does it mean to be a boy or a girl, a man or a woman-- who is going to turn out to be an appropriate object for my sexual desire and so on?
Those are interesting questions.
There is considerable question about whether this actual oedipal conflict-- actually there's very little doubt that the strict form of it is not required.
The single parent family thing is one good bit of evidence against that.
And the woman story-- it's just kind of whack.
In any case, around this time you do get this development of a more sophisticated morality.
And in Freud's view what you had was this poor ego here that was on the one side, assailed by the desires of the id that wanted to do all sorts of unspeakable and stupid things that you couldn't do in the face of reality and then there's the demands on the other side of the superego mostly, that doesn't want you to do stuff too.
But anyway, it's got its own demand.
And the ego has to somehow thread a path through here.
Now how does the superego let you know what you're supposed to do?
Well, it says on the handout, does it?
Where'd I put it on the handout?
Oh, no, I put a blank.
Look at that.
The superego has a weapon that lets you know when you've done what it does not approve of.
To figure out what that weapon is you can invoke your own introspection.
What you need to do is imagine some activity that your parents would disapprove of.
It can be society as a whole if you like, but parents will do.
So think of something that your parents would deeply disapprove of if you phoned up and said, guess what I did.
And ask yourself, how would you feel the next day?
Guilty
Guilty, thank you.
Guilt is the weapon of the superego in this view.
So when you feel guilty a good Freudian would say, you're feeling guilty because you've done something that transgressed the boundaries set by the superego.
Now look, I went out and I committed murder and I feel really bad about that.
Yeah, all right that's not too terribly interesting.
The more interesting case is where patients who said, I feel guilty.
I don't really know why.
I just feel this sense of guilt.
And a Freudian would say, well look.
This isn't all conscious stuff.
You've apparently transgressed some boundary that you don't quite recognize yourself consciously, but it's there.
If you want to get over this guilt we need to figure out what that problem is.
Now, how many people here enjoy feeling guilty?
All right, so it follows that the ego, which is not eager to sit around feeling guilty is going to have some defense against that.
So introspection oughta work here, too.
Let's go back to that, whatever it was that you were going to do that would be really, really wrong.
Now let's suppose you haven't done it yet.
But you're going to do it.
It's right over there.
You can just go in and do it.
How will you feel?
Anxious
Anxious.
I heard an anxious over there somewhere.
Thank you for the anxious, wherever anxious is.
Anxiety is considered to be the defense mechanism of the ego against the predations of the superego.
So when you feel anxious in this context the answer is well, you're pushing up against something that the superego doesn't want you to do.
Now let's go back to the case where I feel anxious, but I don't know why.
Classic Freudian view would have been there's some chunk of your superego that you're getting close to there, it's rules and it's not happy with you.
Any reasonable more modern view would include the possibility that you are just having an anxiety attack that we might consider to be more biological than psychological.
That there are chunks of the brain that if they are overactive-- the experience is being anxious.
If they're overactive for essentially neurochemical reasons you may feel a sort of a disembodied anxiety that's got nothing to do with whether or not you're about to do something that Mother doesn't approve of, but has everything to do with the balance of your chemicals.
The current position in the pendulum that swings back and forth on how to handle essentially psychiatric issues would be to treat free-floating anxiety as essentially biological problem that we might want to medicate.
And we tend to undervalue the possibility that you're feeling anxious for underlying psychological rather than for underlying neurochemical reasons.
And of course, you can feel anxious for things that don't have anything particular to do with the superego or the imbalance of your neurotransmitters.
If I stand on the lectern on one foot I might feel a certain amount of anxiety for this because there's a straightforward threat to my personal safety.
So there are multiple roots into anxiety, but within this context anxiety is the warning sign that you are embarking on a course that mother would not approve of or maybe dad or maybe the broader society as a whole.
So the result here for Freud is that with this collection of psychological structures-- again, psychological structures, nobody thinks you're going to go into it an MRI machine and find the locus of the superego.
But the result of this is that we don't end up raping and pillaging our way across the landscapes.
We sublimate our aggressive urges into things like sports or work.
The unbridled sexual desires of the id get redirected into courtship or into literature or into some other appropriate kind of realm.
From this Freud ends up generalizing a theory of civilization as a whole.
He wrote a very interesting book late in his life called Civilization and Its Discontents.
Why its discontents?
Freud believed that in order to be civilized you were going to have to fight against the desires of your id.
Your id was going to be unhappy with you and you were going to have to fight against the whacko strictures of the superego, which were going to be too strict for any normal person to hold onto.
You were necessarily going to be repressed, dissatisfied, and so on.
That was the price you paid for being civilized.
Or one way to think about this, one line from the book is that "the first man to hurl a spear-- sorry, to hurl an insult rather than a spear is the founder of civilization."
The first man who can redirect that sort of id like desire into something that's consistent with maintaining a civilized world, that's where you start to get the possibility of civilization.
Now I recommend book to you in part because some of it is extremely entertaining, for all the wrong reasons.
As you may have gathered by now, not all of Freud's detailed ideas have stood the test of time.
So I cannot resist the urge to give you Freud's account of the domestication of fire.
Here's the problem as Freud saw it.
Fire happens out in the world.
What did primitive man do when he saw fire?
says Freud
Burned himself
Burned himself.
Well, that would be one possibility.
Well, we can follow with this line, if you weren't going to burn yourself, what might you want to do to that fire?
Put it out
Put it out.
If you're a guy, how you going to put it out?
Smack it
Smack it?
No, that's if you're a dumb guy.
No, Freud proposed that you would urinate on it.
And that this was an essentially sexual act-- don't ask me why.
But that the first man to renounce that desire, that apparently id like desire to go and urinate on fire-- he could bring the fire home and that again, would be a building block of civilization.
Not only that, said Freud, this explains why in many cultures women are the keeper of the hearth.
Why is that?
Well, it's a great deal more difficult if you're a woman to urinate on the fire.
It's just not going to work out well for you.
No, you don't have to believe that.
It's on page 37 apparently of my copy of Civilization and its Discontents.
I mean, the book is a beautiful example of exactly why I think it's still worth teaching Freud.
There's a lot of whacko stuff in there about peeing on the fire, but the broader, large scale thought that repression of infantile urges is a price that we pay for civilization is, it seems to me, a worthwhile thought to rattle around.
That as you become more civilized in a sense, you end up becoming more repressed.
That the id's going to sit there trying to escape and you fill more and more walls around it and you live with more anxiety and more guilt as the price for being in some sense more civilized.
Now that's sort of a dark view.
It's interesting that this dark view crops up-- later in his life he had to flee Austria from the Nazis.
He was a sick and dying man when he wrote Civilization and its Discontents.
It's a rather dark view of civilization.
Suppose this is all true, in some fashion.
Well, as it says on the handout, how you going to talk to the children about this?
If these are important issues and your kid is hopping into bed with you and saying, Daddy get out of bed.
I want Mommy and stuff like that.
How do you talk to-- I'll tell you how you don't talk to your kids.
You don't say, I recognize that at this stage in your life that you are thinking that you're in conflict with me, but what you need to realize is that I'm much bigger than you and if you pursue this much further-- see the scissors?
You're in very big trouble here.
That's not a conversation you want to have with a four year-old kid.
Even if you're a psychologist.
This is not a good idea.
But you could imagine that these issues, which not explicitly understood by parents or children, but that these issues about, is the world safe?
Who runs the show?
What am I?
You know, these sort of issues are issues that you're going to want to talk about.
And if there's this sort of psycho-sexual development thing running underneath it those are the issues you're going to want to talk about in those terms.
Bruno Bettelheim, following up on ideas in Freud's own writings wrote a very interesting book called The Uses of Enchantment in which he argued that among other things, fairy tales or more precisely, folk tales serve the role of a hidden language or a way to talk into your children in hidden language about these issues.
Fairy tales is an English term.
It's a little misleading because fairy tales don't necessarily have fairies in them, but folk tales is the more accurate term.
So something like the Tales of the Brothers Grimm, the brothers did not write those stories, they collected them.
They were early 19th century anthropologists wandering around northern Germany getting Grandma to tell them stories.
And a folk tale is a story that's not explicitly written down for literary purposes, it's a story that has a grownup organically from parents telling children and children saying, tell me the one about the witch with the candy house and stuff like that.
And these stories-- the good ones-- get passed on and Bettelheim says the good ones get passed on because they do this kind of work.
They allow you to talk about these kinds of issues.
I see that I said, what are the characteristics of fairy tales?
So I better say a word about that.
Characters in fairy tales are typically good or bad.
You don't get-- in the woods there was a witch.
She wasn't really, like really a horrible person, she was kind of misunderstood and she'd been abused as a child.
No, she's a bad witch.
And there was a boy.
Very schematic chara-- oh, that's the second point actually.
Very schematic characters often with names like the boy, the girl.
Or just a simple descriptor like Snow White, which is just telling you something about how she looks.
She's good, right?
It's not that she's got issues or something, she's good.
And they're figures that a kid could identify with typically.
This is in contrast for instance, to myth.
Oedipus is not a folk tale in this sense.
You don't, at age four say, let me tell you about this cool story where this guy goes off and kills his dad and then marries his mom and he blinds himself in the end, it's really neat.
But you say, ooh, that's really gross.
But ask yourself-- we're going to do Hansel and Gretel in a minute here.
How many people know the story of Hansel and Gretel?
Think about Hansel and Gretel in those terms.
Or think about the headlines, poverty parents ditch kids in wood.
Girl, five, cooks old lady in oven.
I mean, this stuff is just as gruesome in it's own way as king of Thebes finds out he's married to Mom, blinds himself and wanders off into three more plays by Sophocles.
It's not immediately obvious that it's just grossness that somehow differentiates myth and folk tale.
But the characters are typically something that a kid could identify with.
They have optimistic endings, that clearly distinguishes you from Greek tragedy or something like that.
At the end of a classic fairy tale good things happen to good people and bad things happen to bad people.
Witches get killed.
Good little girls and boys go home and live happily ever after and stuff like that.
But at the same time, while they're optimistic in their ending they do not typically have overt morals on them like Aesop's Fables or something like that.
You don't get fairy tales that say, they came home after having cooked the witch and they said, we'll never go into the woods again or eat any candy, ever, ever, ever.
It's good things happen to good people, bad things happen to bad people and [UNINTELLIGIBLE].
No overt punchline at the end.
An important point is that even if Bettelheim is correct, that these are somehow talking in this sort of coded Freudian language, parents don't know the language and kids don't know the language or don't know that they're doing this all again, sort of implicitly rather than explicitly.
Nobody, except somebody who took too many notes in Intro Psych says, let's see-- the kid's three and he was making these cracks about the bed and Dad and stuff like that.
Let's see, I need an oedipal fairy tale.
Jack and the Beanstalk that'll work tonight.
But the notion is that the kid will be requesting stories that address issues that he wants to hear about and that the parents may find themselves choosing stories to read or to tell that serve the issues that are arising in their minds at the time.
Now a lot of this has been somewhat diluted in our era because we tend to read fairy tales or watch them on videos where they have been turned into literary constructs in ways that they weren't in northern Germany in the 19th century or something like that.
We'll talk a little later about some of the ways that Disney has modified some of the great fairy tales.
They're wonderful things, but Bettelheim is very mad about the ones where the revisions that make nice at the end-- Cinderella stories where you do something nice to the step daughters, stepsisters-- you're not supposed to do anything nice to the stepsisters.
They're supposed to get their eyes pecked out by crows because they're bad and bad stuff happens to bad people.
Cinderella isn't supposed to arrange for them to mary some duke or something like that.
That makes Bettelheim very agitated.
All right, let's talk fairy tales here.
So Hansel and Gretel is in this way of thinking about things an oral stage fairy tale.
Oh, let me say something about that-- oral stage, that's like year one.
And you're sitting there saying, I like Hansel and Gretel, does that mean I never got out of the oral stage?
No.
Remember the idea here is that the issues that arise in the oral stage about, is the world a safe place, for instance, are issues that will persist for the rest of your life.
So you're a seven year-old kid in this view or something having a certain amount of concern about whether the world is safe and if you walked down the street to school you're going to get hit by a car or something like that.
And maybe Hansel and Gretel would appeal on those terms.
Another example would be, Beauty and the Beast.
A female oedipal story, we will see, if and when I get to it.
You're saying, I'm a guy.
I'm a guy.
I like Beauty and the Beast.
Is this a problem?
No.
Part of this is not only do you have to figure out what it means to be a guy, you also have to figure out what it means to be a woman.
You don't happen to be a woman, but it would be really interesting to understand what it might mean.
And so again, you might be interested in that particular story not because some strange path is going on [UNINTELLIGIBLE].
These are interesting questions.
Hansel and Gretel.
Let's step back.
What's the family situation here?
Who we got at the characters in Hansel and Gretel?
Well, there's Hansel and Gretel.
Who else do we have?
Father
Father and?
The mother
The mother?
Stepmother
The stepmother.
She a good stepmother?
No
Or a bad stepmother?
There are an awful lot of bad stepmothers in fairy tales.
Now that's a little mysterious.
There are a couple of ways to understand this.
That one of them is to say that in the days when childbirth was a much riskier proposition many more people had stepmothers because Mom had died in childbirth.
As indeed happens in any number of fairy stories.
And in fact, you can come up with an evolutionary psych argument that says that the stepmother that systematically favors her own genetic children over the step- children, but that's not the Freudian account here.
The Freudian account is look, the initial mother who you encounter when you're a baby is a really good mommy.
She does everything for you and she says yes all the time.
You cry, she jumps.
It's great.
Eventually Mom gets tired of this and Mom starts saying no and Mom starts saying things like it's time to go poop in the bucket and not in your pants and all this other stuff.
And that, in this view is the transition between good Mommy and bad Mommy and there are so many fairy tales where you get born to a good mommy and then the bad mommy shows up.
Way outside of the range of what would be probable in the population.
That you can imagine that it's a stand-in for something else.
Maybe for a stand-in that you're coming out of this oral stage of development in this story.
All right, so what's the problem that's faced by our lovely little family unit?
What's the issue here?
Poor
Poor.
We're out of money, so what's the specific result of being out of money?
[UNINTELLIGIBLE]
A few hands would be handy rather than the general purpose muttering.
No?
We've scared them off.
What have we run out of here?
Food.
We don't have any food.
The natural thing to do when you've run out of food is do what?
Gather some
Gather some.
Well, that didn't work apparently or at least that was a really boring story.
You know, once upon a time there was a mother, a father, and a little boy and girl and they got hungry and they went in a field and they gathered mushrooms and unfortunately they were poisonous and they all died.
That was one of those stores that didn't get repeated.
You know, tell me the story about the poison-- no.
So what happens in Hansel and Gretel?
Get rid of the kids
You get rid of the kids.
You stick the kids out in the woods.
But Hansel hears the plan, mom and dad are busy discussing this at night.
Hansel hears about it, he does what?
Runs away
No, he doesn't run away.
That's the Russian version
Kills the parents
Kills the parents, no.
Wait a second guys, I thought you said you knew this story.
Let's get this straight.
He goes out into the garden, he collects a bunch of white stones.
He leaves the white stones behind as a trail and--
Bread
Bread crumbs
No, that's the second time.
You guys are all going to flunk the final.
This is very bad.
So first time it's white rocks and when the moon rises so you can see the white rocks, they come back.
So you've got an oral stage problem.
There ain't no food.
And the initial crisis where the kids get left in the woods is met with a nonsolution.
You just returned back to the same situation.
Second time, same thing.
They get abandoned out in the woods, but this time this nasty old Step-mom has been cagier about it and locked Hansel in and all he's got is his bread.
And so now he tries what you can think of as an oral stage solution to an oral problem.
He leaves this collection of bread crumbs behind.
Does an oral solution solve this problem?
Well, no.
What happens to the bread crumbs?
The birds eat them
The birds eat the bread crumbs, so there's no bread crumbs.
And they wander around and they fall asleep in the woods and there's a beautiful chorus in the Engelbert Humperdinck opera which you should all hear sometime-- gorgeous opera-- and Engelbert Humperdinck really was his name.
Very sad, but the opera's lovely.
Anyway, so the next day they wake up.
They're hopelessly lost.
They wander around.
They're hungry.
They're not terribly happy and they find?
[INTERPOSING VOICES]
The house, right.
The house made of gingerbread and candy and stuff like that.
And so like all good little kids what they do is engage in a little petty vandalism.
Gretel eats a window and Hansel eats a hunk of the roof and then they hear this voice saying, "Nibbling, nibbling like a mouse, who's that nibbling at my house?"
And they respond, "The wind, the wind doth blow from the heavens to the earth below," which makes no sense.
But anyway, so out comes the?
Witch
How do you know she's a witch?
Because she's [UNINTELLIGIBLE]
Now the way you know she's a witch it says is because her eyes glow red.
This is a tip-off in case-- this is practical advice for you.
If you're vandalizing somebody's house and the owner comes out and her eyes are glowing red, worry about it.
But that this means that witches don't see very well, which turns out to be handy.
They have keen senses of smell we're told.
Anyway, this witch-- now the interesting thing about these fairy tales is that you have a problem at home and then you are thrust out into-- well, in the north German versions of the Grimm fairy tales are into the woods to solve it.
Somebody will have to tell me what you do if you're reading Saudi Arabian fairy tales whether you get thrust out into the dunes or something like that.
But the important thing is you go out away from-- you don't solve your problem where it is in these stories typically.
You out and away and what you find out there in story after story is a charicature-- an extreme version of the problem at home.
So here we've got an oral problem.
There isn't enough food or perhaps, if we're thinking in infantile terms, where's the food come from if you're an infant?
Well, it comes from Mom.
So not enough food means you've kind of eaten all that you can eat off of Mom and you might worry, what's going to happen if Mom wants this back?
Well, what you find in the woods is this orality gone nuts because what this witch does is eats little kids.
That's her stock and trade.
And she's going to eat Hansel, but it turns out he's too skinny for her.
So there's this fattening up period.
Eventually, it's cooking day and the witch says to Gretel, climb into the oven and see if it's warm because we're baking bread-- haha.
Climb into the oven and Gretel who's onto this says, I don't think I can do that.
Can you show me how?
And the witch says something like, you silly goose.
Anybody can do that.
And she climbs up to the oven and Gretel slams the door on her and she perishes horribly, making loud screams.
Which, not much better than blinding yourself and things like that.
But is deeply satisfying in this story.
So now they've killed off this symbol of orality and there are two important things that happen thereafter.
First of all, what they now discover is that the house is full of gold and jewels.
And they stuff their-- there's a little more looting going on here-- but they stuff their pockets not with candy, not with more oral solution kind of stuff, but now with something that allows them to go home and have moved to a different stage.
A stage, where in principle they can be contributors, not just consumers.
This isn't supposed to be an accurate model of what happens-- one year-old kid moving out of the oral stage.
Oral stage kid doesn't suddenly say at the end of a year OK look, like I'm weaned and now I'm going to go do chores.
You've got to understand this in a little more symbolic way.
Rather than an oral solution we've got a different kind of solution.
The other thing that's very emblematic here is what do the children find when they get home?
Anybody remember?
The stepmother is dead
The stepmother is dead.
Presumably, the stepmother is dead at the instant that the witch is killed because they're in a sense, one and the same character.
They are the problem.
They are the emblem of this oral stage problem.
It gets killed off and everybody's good.
Well, I think before the break, I'm going to skip the anal stage for the present.
I promise to come back with an anal stage fairy tale.
But what I'm going to do before the break is I'm going to skip to-- and there are lots of oedipal stage fairy tales and I am going to use Jack and the Beanstalk as an example of an early oedipal stage fairy tale.
Jack and the Beanstalk.
All right, let's do Jack and the Beanstalk quickly.
Jack and the Beanstalk, what's the crisis at the beginning?
What's Jack sent off to do?
Sell the cow
Sell the cow.
Why are you selling the cow?
[UNINTELLIGIBLE]
It's not giving milk anymore.
If you want a nice clear image that you're out of the oral stage, the cow having run dry on you is pretty good.
So Jack's supposed to go off-- he's supposed to do the Hansel and Gretel thing, right?
He's supposed to go from the dried up cow thing and get some money so we can go do the food thing the other way.
But what's he do?
He trades it for...?
Magic beans
Magic beans.
All right, he comes home.
Is Mom happy about this?
No
No.
So what happens to Jack?
He gets sent to bed without any supper.
Another sort of end of oral stage kind of image.
And Mom does what with the beans?
Throws them out
Throws them out the window or sows his wild oats or something like that.
In any case, something grows really big at night in this story.
You don't have to be a vastly Freudian imagination to think that this looks a little phallic here maybe.
Anyway, you wake up in the morning and there's this giant beanstalk.
Climbs up the beanstalk.
You climb up the beanstalk.
What do you find at the top of the beanstalk?
[UNINTELLIGIBLE]
You find another-- just like in Hansel and Gretel you find a cartoon version of what was at home.
So you find the castle, which is just the house and who lives there?
Giant
Well, who does he find when he gets there first?
The giant's wife.
Or the ogre's wife who's the stand-in for Mom.
Is she bad?
No, she's really very nice.
They have a very nice relationship.
They're playing all day.
The problem is "Fee, fie, fo, fum, I smell the blood of an Englishman.
Be he alive or be he dead, I'll grind his bones to make my bread."
Otherwise known as Dad has come home.
One of the things that's sort of interesting here is that this models a classic sort of nuclear family with a stay- at- home mom that-- and you can sort of imagine in the context of this oedipal conflict.
So here's the son.
All day long he gets to play with Mom.
You know Mom's cool.
Mom's great.
And then this ogre comes home and he wants Mom and he wants dinner.
And he wants to grind up little kids and stuff like that.
A little strange, but it is an image of the situation that a young child might find himself in.
Now you might wonder whether in a few more generations our fairy tales will have to change in response to the-- in my family both of us were out of the house working.
So the nice giantess at home model isn't going to quite work.
You sort of wonder whether or not in a few years, a few generations or something, Jack will be there with the nice daycare provider or something like that and the mom and dad giants will come home at the end of the day.
Fee, fie, fo, fum, let's microwave the little bum or something like that.
But in this version you got Dad.
He comes home and-- the dad's stand-in and so what does Jack do?
Well, first of all, the mother hides him in the oven, which is a sort of womb-like symbol, which is rather nice.
But he goes out and steals stuff from Dad-- from the giant.
He spends all of time ripping Dad off here and basically being in conflict with Dad.
Eventually he tries to steal Dad's golden harp, which is bad because the thing makes a racket.
He got the goose out of there without the problem.
Goose that lays that golden eggs-- good thing to steal.
Kept the goose quite, but the harp made a noise.
Anyway, the giant catches onto this and what's he do?
What's Jack do?
[UNINTELLIGIBLE]
Didn't understand a word of that.
Where's Jack go?
Down
Back down the beanstalk.
What's he do with the beanstalk?
Chops it down
He chops it down.
It depends on the version -- either Dad crashes or the giant crashed or not.
But in any case, if this was a full blast oedipal story you'd be going [UNINTELLIGIBLE] and get to the point where you go off and for instance, marry the princess or something like that.
This is a partway oedipal-- not all the way through the story-- thing.
You get that beanstalk and you get rid of it.
Either it's sort of a reversion back or it's going into a latent period.
None of this stuff that grows in the night-- nothing man, we're done with that for the time being.
Nothing good happens here.
Large, big guys chase me around when I do anything of that and forget it, so it's a story that doesn't get all the way through to the end of this whole psycho-sexual story that Freud's weaving.
So let's see here.
All right, I've already given you a hint that classic male oedipal stories are those stories where you do end up marrying the princess.
I'll say a bit more about that in a minute.
But let's take a short break because otherwise I'll never get to the end of this
Thanks for the final.
Can you put up some links for me because I really don't remember some of these stories
Oh, don't worry.
No answer on the final would require you to know the details of the story.
And I hope I'm telling enough of the story that you can get the feeling for how it maps back and forth to the bits of Freud.
But if you happen to be fond of fairy tale literature the Bettelheim book is a lot of fun to read.
I mean, it's now-- how old is it?
God, it's twenty-five years old.
I read that sucker when it came out the first time.
Anyway, I keep looking for new stuff to read on it and there's nothing as good.
It's fun to read.
In the full blast male version oedipal fairy tales there and endless ones of these and they are of the form-- they're very typically of the form-- the lead character is typically somewhat older.
Often described as a prince rather now than as a boy.
Rather than being like abandoned in the woods, he is now thrust out of doors by his father for some crime or other.
He has to go out into the world.
He's at home in this oedipal conflict.
They reach the crisis in the conflict and Dad literally says, I'm going to kill you.
The kings says, I'm going to kill you or something and he's got to go out into the world.
And he goes out in the world and has adventures.
What he does out in the world in these stories is very typically to kill a stand-in for the father.
In Greek tragedy, you may kill your father.
In a fairy tale you kill the ogre the dragon, the giant-- something that stands in for the father and the reason you're doing this is because the ogre, the giant, or the dragon has in his possession, what?
Princess
A princess.
Almost always a princess who thinks that the prince is really quite nice and all that.
And not Mother.
You don't go out and rescue your mother from the dragon.
It just doesn't happen.
What you're going to go out in one of these stories and do is find an appropriate mate.
Not the inappropriate oedipal conflict stage mate.
You're going to go out and find the princess, kill the dragon.
You're going to then bring her home and everything is going to be good.
And in fact, at this point typically in one of these stories the King will not die, but will retire.
And Prince Charles in England been waiting for this for years, but it's never worked out for him.
You come home with the right princess and the King retires and you get to rule the roost.
That's getting through all the way to the end of the story.
Now female oedipal stories are different in interesting ways because of the difference in the traditional family structure.
If in a male structure you've got the giant in Jack and the Beanstalk who comes in only intermittently because the primary care-giver is the mother.
The female's got the conflict with the primary care-giver, so you have in these sort of stories the girls who are captives of nasty, older women in some fashion or other.
So let's look, for instance, at Snow White.
Snow White, another one of these names where the name is just a generic term.
She's white as snow and got hair as black as coal or something.
I don't remember.
Anyway, lovely mother-- lovely mother dies.
The husband, the King remarries another lovely woman, but a very jealous one.
The jealous stepmother has what cool device?
Mirror
The mirror made famous most recently in Shrek endlessly.
And the cool thing about the mirror in Snow White is it speaks not literally, but figuratively with the voice of the little girl.
Initially, when the stepmother asks, "Mirror, mirror on the wall who's the fairest of us all?"
The mirror says, you.
It's you.
The voice of the little girl saying, look, I idealize you.
You're my mother.
You're the greatest thing since sliced bread.
Little later, "Mirror, mirror on the wall who's the fairest of us all?"
Well look, Queenie, you look pretty good for a woman of your age, but have you seen Snow White?
You know, she is a blossoming, young woman here and hmmm-- well anyway, this drives the nasty old stepmother nuts.
And in the best tradition of oedipal stories she decides to do what with Snow White?
Kill her.
Another characteristic of female oedipal stories is that the dad tends to either be absent or useless.
The King does nothing here.
The other dad stand-in who's the huntsman, who the Queen gives Snow White to to have her killed.
The huntsman fails both the Queen and Snow White.
Disney's actually quite close to the original story in this one.
The huntsman takes her out into the woods, doesn't have the heart to kill her, but also doesn't have the guts to save her so loses her out in the woods basically.
He goes off and kills what?
A deer
A deer.
Does what?
Cuts out its heart
Cuts out its heart.
Brings it home.
The Queen does what?
She eats it.
Nice woman.
Anyway, so she goes off into the woods.
What does she find out in the woods?
[UNINTELLIGIBLE]
Hi ho, hi ho-- she finds a lovely little encapsulated anal stage fairy tale is what she finds.
The women will have some intuition about this.
If you told your mother, Mom, I found this really great housing arrangement at MIT.
I'm going to go and live with seven hairy guys.
Would mom be thrilled?
No, she would not be.
If however, they're the dwarves from Snow White even though the ones in Grimm aren't Sleepy and Dopey and Whiffy and Waffy and whatever they are in Disney-- you wouldn't worry about it because these guys, they're little asexual guys.
There is no sense of threat here at all.
And what do they like to do?
They like to dig stuff out of the dirt and put stuff back in the dirt.
And dig stuff out of the dirt and put stuff back.
And they're devoted to Snow White, they think she's great.
But they're a little anal stage fairy tale tucked in there.
And so the problem is that the mirror rats out Snow White, right?
"Mirror, mirror on the wall who's the fairest of us all?"
Yeah, yeah Queen you look great.
Guess who's blossoming down in the woods with a bunch of little hairy asexual guys?
So she disguises herself, the Queen disguises herself and goes off in an effort to kill Snow White.
She tries three different times and they are interestingly emblematic of an effort to keep a girl from turning into a woman.
The clearest one of these is the corset that she sells-- she goes as a peddler woman and sells Snow White a corset that squeezes her body back into a little girl body so hard that she can't breathe and falls over dead.
Or semi-dead.
And always at the last minute-- Hi ho, hi ho and the dwarves come back.
She's dim.
You know, it's one of these sort of fool me once shame on me things, but three times she goes and falls for the old lady thing.
And the last time it's with this Adam and Eve kind of apple.
She eats the apple.
The dwarves come home and she's dead and they can't do anything about it.
Now what would the natural thing be to do under those circumstances?
Bury her, right?
Or something.
She's dead.
What do they do?
They put her in a glass coffin.
It says they can't bear to bury her.
They put in a glass coffin and put her in a clearing in the woods.
And it says in the story that she continues to grow into a beautiful young woman, which is a lovely image of a latency stage, right.
Nothing's happening, but she's turning into a sexually mature woman is what's going on here.
You would think that the dwarves might catch on.
There's something weird here guys.
That normally the dead body thing doesn't work like this.
But anyway, the dwarves don't-- she's getting beautiful and through the woods comes the...?
Prince
Prince.
The prince does what?
[UNINTELLIGIBLE]
Nah, the prince doesn't kiss her.
None of that stuff.
What the prince does-- I can't remember if he does this in the Disney version, but in the traditional version the prince orders the dwarves to bring the coffin to his castle.
What is he thinking?
You know, this would be a really cool coffee table?
It's very weird stuff.
But what happens is the dwarves who have been extremely careful all along-- the dwarves trip, the coffin shatters and the apple pops out of her throat and she's alive, she's fine.
This is a story that doesn't quite get all the way through to adult sexuality in the sense that the story ends without her saying that she loves the prince or anything like that.
It says, he was nice and so she went with him.
It doesn't quite get you all the way through.
The classic stories that get you all the way through to the end of the story on the female side are the so-called "animal groom" stories.
I don't have time to do Little Red Riding Hood really.
But Little Red Riding Hood is a story where all men are beasts and they stay beasts.
The classic versions of female oedipal stories that get you all the way through to the end of the story are the ones where the beast is redeemed by the love of a good woman.
You know, one class of these is kiss the frog and it turns into a prince.
But let's do Beauty and the Beast.
Now Beauty and the Beast in the Disney version is a wonderful movie about female empowerment or something like that, but it's a long, long way from the original folk tale version of it, unlike the Disney Snow White.
So let me tell you a little more about the version that is in Grimm's fairy tales-- if you aren't familiar with it.
So in Grimm's fairy tales or in the classic Beauty and the Beast story for starters, there's no reason why the Beast is a beast.
In these stories typically, the Beast in the Disney version is a beast because he's got no love and he was nasty to whatever she was who showed up at the door and stuff like that.
None of that in the traditional version.
He's a beast because some old woman said, you're a beast.
In the all men are beasts school of beasthood.
So when we start out with Beauty and the Beast, Beauty is at home with her two sisters.
No Mom onsite at all in this one, but there's Dad and Dad's going on off on a trip.
Not because he's got some whacko cool invention, he's just going on a trip and he says, what do you want me to bring back and one greedy, old sister says, oh bring me a Mercedes and the other one says, bring me a fur coat and Beauty who's a bit of a sap says, just bring me one rose.
So anyway, he does well on the trip and he gets the Mercedes and gets the fur coat.
He forgets the rose, right?
So he's coming home and he passes this ruinous castle and there's a rosebush of course, and he says, nobody's going to miss one rose, right?
So he goes and picks the rose.
So bad move because out comes [UNINTELLIGIBLE].
And you picked my rose, now I'm going to kill you which seems a little disproportionate, but you know, he's a beast after all.
And so the daddy explains way more than he should be under the circumstances.
Oh, I was just picking one for my daughter-- and the beast says, OK. Fine, I won't kill you.
Send me the daughter.
Dad says, OK.
There are two ways of looking at this.
One of them is that dads very traditionally, in animal groom stories are the person who, in a sense, walks the daughter down the aisle here.
The other way of thinking about this more charitably about Dad is that Dad's thinking, I'm just getting out of here man.
I'm not going to send my daughter back.
But he goes home.
He tells the story and very much like in the Disney version Beauty says, hey, you gave your word.
I gotta go.
And so she goes off to live with the Beast.
Now the Beast-- well, he's a beast, but he's kind of like a gentleman.
He's not really a bad beast as these things go.
They live a perfectly reasonable life there in this grungy castle.
Every night, after dinner he says, will you kiss me and she says, oh yuck.
You're a beast.
And he doesn't press the issue, right?
And they just go on like this for a long, long time.
And eventually she gets a postcard or something-- no, gets the wedding invitation.
She gets an invitation that says, your sister's getting married.
Why don't you come home for the wedding?
And the Beast is reluctant to let her go, but says, you can go as long as you come back within a month or I'll die.
So she goes home and big, long party, I guess because she forgets all about it.
The magic mirror in some versions, but in any case, on the thirty-first day she either looks in the mirror or realizes in a dream or something that the Beast is dying.
The castle is in ruins.
The Beast is dying.
And she wishes herself back there.
She's magically transported back there.
Nobody gets to get killed on the roof in the rain or anything like that.
She realizes that she does love this beast and kisses him and kaboof!
Now he has been redeemed by the love of a good woman and he's no longer a beast.
You've gotten through all the way to the end of the story.
So Little Red Riding Hood there's no sense that she's going to go marry the Beast.
In Little Red Riding Hood what you've got is the situation where Little Red Riding Hood says to-- when confronted by the Beast on the road in effect she says, I'm much too young for you.
I'm an inappropriate sexual object.
Why don't you go see Grandma?
She's a much more experienced woman.
Go read your Little Red Riding Hood again.
Little Red Riding Hood gives way too much-- if you go and meet a stranger, a bad, scary stranger and he asks a bunch of questions you're not supposed to say, I live or I'm going to Grandma.
She lives at 400 Shady Brook Lane and I think she leaves the back door unlocked.
That's more or less what Red Riding Hood does.
So what the Wolf does is it goes there and gobbles up Grandma.
Gets into bed and then gobbles up Little Red Riding Hood.
Who is later delivered by cesarean section, more or less and describes the whole experience as yucky.
Not scary.
It was all just sort gross.
And when the Wolf is captured by the huntsman, Daddy, the Wolf is described as you old sinner.
What's that about?
Anyway, the Little Red Riding Hood ends with Little Red Riding Hood saying, I will never go off the straight and narrow again.
Because the Wolf's been saying go smell the flowers.
Go off the track and stuff like that.
So that's a story where you don't get through the whole thing.
If you wanted to write a Little Red Riding Hood story-- you wanted to morph it into a full blast story-- I suppose it'd be a little crass if Little Red Riding Hood had to go and marry the guy who killed Grandma.
That's a little gross, but she'd end up kissing the Wolf at the end and the Wolf would turn into something nice.
There's a great MIT fairy tale-- a tool and die fairy tale, but you'll have to ask me about it some other time.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
This is the lecture on sleep and dreams, and you want to know about this because it's -- well, at MIT, sleep and dreams is a topic that falls under the heading of abnormal psychology.
All right, let us ask to begin, how many people here got some sleep last night?
OK. Of those people, how many people went to sleep, even though they knew they still had stuff they wanted to get done?
All right.
So, what's your problem?
Well, the typical MIT answer to that is sort of a moral argument.
I'm weak.
I'm a miserable human being.
If I was a good, strong human being, I would not have gotten sleep, I would have done that problem set and so on.
If you were not at MIT -- MIT is merely at one extreme here.
Western civilization in general increasingly, since the invention of the lightbulb, has been an increasingly sleep-abusing, mechanism-abusing culture.
But if you were left to your own devices or to what nature had intended, you'd be sleeping about eight hours a night, and you would be doing this across your life span with the result that you would spend about a third of your life asleep in this altered state of consciousness or unconsciousness.
That, if you're teaching a course on human behavior and human mental life, that requires some explanation.
Why do people sleep?
What is the nature of that behavior left to its normal state?
What happens when you abuse it?
Why do people dream, which is psychologically, perhaps, the most interesting piece of sleep.
Is it possible to get any -- do those dreams, do those weird stories have any meaning?
Those are the questions that I'll talk about.
I don't know if I'll get to end of it today, otherwise I'll pick up after Thanksgiving.
Let's talk initially about why and how you sleep.
Well, look, I mean one level of answer would be you sleep because every so often nuclei, collections of nerve cells deep down in your brain stem send volleys of signals up to the rest of your brain that basically make like a hammer and knock you out.
But that's not a deeply satisfying answer.
It does point to the fact that it's a very basic drive, and it's a very powerful drive.
Look, if you wanted to you could, for example, as a statement of extreme political conviction, sit right here and starve yourself to death.
Hunger is an important drive.
There are several people illustrating that at this very moment.
If you wanted, you could override that and not eat, even to the point of killing yourself.
That's not true, for example, about your breathing.
If you wanted to sit here and hold your breath until you died, it wouldn't work.
You may have tried this when you were little.
One of my two sisters used to be a fan of this.
If we were teasing her and chasing her she would just hold her breath until she keeled over, which was very dramatic, but not, in fact, life threatening, except in indirect sense that my mother was not thrilled with us making my little sister pass out.
You could ask her, she'll be here tomorrow, but you'll all be leaving.
Anyway, this drive to sleep is more like the need to breath than the need to eat in that sense.
If you were to sit here and by an act of will declare I will not fall asleep -- and look, as you well know, there's a substantial number of you who can't manage to do that for an hour and a half.
That has to do with how you're abusing the normal mechanism on -- when I first was teaching, I used to look out at 10250 full of intro psych students in they're ahh.
I'm thinking I'm not being lively enough here.
I realize at this point, I could be setting off large explosions on a regular basis and it wouldn't matter for some people.
We'll get to that in a minute.
But even those of you who are well-rested, you could not sit here and not sleep.
Probably you couldn't make it for twenty-four hours.
Absolutely you could not make it for forty-eight hours without external help, either from drugs, like caffeine, or more particularly from somebody smacking you around to keep you awake.
If you're in a sleep lab and you want to keep somebody awake for thirty-six hours or something to see what happens, you sure don't say, I'll give you ten bucks if you can stay awake all night.
You need somebody there to slap them around and keep them moving or they'll be out.
There's just no way to suppress that.
It's a very, very powerful drive.
So, how come?
What's doing this?
Your need to sleep is driven by two primary factors.
One of them is a so-called circadian rhythm.
The term comes from -- circadian is about a day, because it's a rhythm with a periodicity of about twenty-four hours.
If you manage to measure it in isolation, and you were to measure something like alertness, it would have a roughly sinusoidal shape to it, this particular drive.
When you wake up, it's actually at a relative low point.
Yeah, right.
At six a.m. on this chart, is it a relative low point?
It rises your alertness driven by the circadian clock, rises systematically during the course of the day till early evening, and then declines to a nadir around four or five in the morning.
When you're trying to stay up all night, that time in the wee hours of the morning when it's just really, really tough and you're getting the shakes and all, is when this circadian clock is crashing, is hitting its minimum.
How do we measure this?
How do we measure circadian cycles?
Well, if you're a mouse it's actually relatively easy.
Stick a mouse in a cage with one of those little running wheels and put a sensor on the running wheel, and you'll get out a graph that looks like the graph that I put on the handout.
Each of the lines on that graph is one day, so you've got a day here, and the mouse is doing nothing much.
Then at a certain point, the mouse jumps on the wheel and does stuff.
Now, the convention of the circadian trade is to double-plot their data.
So the line is the same data plotted twice.
It's just makes it easier to see trends over the day.
So this is inactivity, activity, same day.
This is twenty-four hours.
You can see that the mouse does this every day, for day after day, like clockwork, as it were, because the mouse has this little clock that's telling it.
Now, the point at which the mouse is firing things up, right about here, is the mouse was on a twelve hour light/dark schedule, and when the lights go out, the mouse turns on.
It's a nocturnal beastie.
When the lights go out he jumps on the wheel and goes berserk.
Now, what happens halfway down that graph where there's an arrow pointing at the side of the data, at that point you leave the lights on continuously.
What you discover is the mouse still behaves in a very systematic fashion.
I'm not going to bother double-plotting my data anymore here.
But now the period is starting to drift.
The mouse is now doing what's known as free-running, and this is a way that you can take a look at the circadian cycle in this beastie because he's now no longer got the light signal telling him what time it is.
What you discover is that it's advancing by about thirty minutes a day, because the mouse clock has a period, a circadian period of about twenty-three and a half hours -- not quite twenty-four hours.
If you were in the same experiment and you were inclined to run around in a wheel, the day would actually go this way a little bit, because the human clock has a period of about twenty-four and a quarter to twenty-four and a half hours, some variation.
It's again, around a day.
It gets synchronized to the real day by light most strongly.
So you know when it's morning because that's when the sun comes up.
If you never get outside and you live in constant light, your clock may be drifting too in something fashion.
But given a normal exposure to light/dark cycles, that sets the clock.
How many people are flying across time zones to the next few days?
So you guys are all going to be jet lagged in some fashion, because your clock is set for East coast, U.S., you'll go someplace else.
Get outside because it's exposure to the sun that will reset you.
Exposure to these sort of lights will reset you too, but the sun is sufficiently brighter that it makes a bigger difference.
This clock, by the way, has a home.
It's located in the suprachiasmatic nucleus.
Looks more complicated than it is.
Supra for above.
Chiasmatic for the optic chiasm, that x where the optic nerves cross on the way from the eyes to the brain.
This sits right on top of that and gets direct input from the retina.
It's got its own visual system in a sense.
How do we know that this is where the clock lives?
There are a number of ways.
But the coolest way that we know that this is where the clock lives is that there are clock mutants out there, probably some human, but certainly mouse mutants.
Suppose you got a mouse mutant who, let's say, has a clock running at, say, twenty-two hours or something like that.
You can go in, take out the suprachiasmatic nucleus from that mouse, put it into another mouse -- and this is a case where a brain transplant will actually work.
The recipient mouse will now start running with the clock timing of the donor mouse.
If you want a new clock, you can actually go and get -- I don't know if you go and get one from a mouse, and probably your neighbor doesn't want to donate.
There are also other clocks, by the way, that live in places like your liver, but the main clock running your behavior is sitting up there in your brain and reset by light, and in fact, can be transplanted, apparently.
Now, humans don't tend to run in little mouse wheels very much.
If you want to measure this in a human, if you stick a human in continuous light, and cut them off from the external world, which you can get yourself paid to do if you want.
One of the leading sleep labs in the world, actually, is across the river at Brigham and Women's.
My lab actually does some collaborating with them.
So if you're interested talk to me.
People do this as a summer job.
You get locked in the lab for the summer.
People think oh, I'm going to write the great American novel or something like that while I stay up all night or something.
But you get paid a decent rate for this.
Now we're going to want to keep track of what your circadian clock is doing.
The sorts of things that vary with your clock are, for example, body temperature.
Your body temperature has about a one degree plus and minus swing with the circadian clock.
So, if you've got a fever, for example, you'd probably remember from when you were a little kid that your fever would peak in the early evening.
If you stay up all night it's four in the morning where you suddenly feel chilled.
That's because your body temperature is actually varying as a function of the circadian cycle.
There are lots of circulating hormones, like melatonin and others, in the blood that also track the circadian clock.
So, if you're in the lab, somebody is going to be monitoring these various aspects of your physiology in an effort to take a look at where your circadian clock is.
This is one of the things that sort of discourages people from immediately signing up, because well, for instance, if you're going to be taking continuous measures of body temperature, the thermometer is not under your tongue.
It's a slow group.
Anyway, so you got the circadian clock that's driving your alertness.
The other piece that's driving you're alertness, or inversely this powerful drive to sleep, is sometimes called the sleep homeostatic.
It's simply a homeostatic mechanism that says the longer you're awake, the more tired you get.
This is not a huge surprise.
So we can graph that here.
So this again, alertness.
Let's say this is 6:00 a.m. again, now let's say that you wake up at something like 8:00, and basically it's a roughly linear function of during the day you get sleepier and sleepier and sleepier and sleepier, and sleepier, and sleepier.
This thing builds up pressure to sleep.
If you now go to sleep at let's say 11:00, that recovers.
It recovers at least initially rather steeply, and then probably has a sort of an exponential tail.
It's not absolutely clear.
But if you combine these two, well, if you were to combine these two in the way that nature intended, what you get is a sustained period of -- let's look at the course of the day.
Suppose you wake up in the relatively early morning, your circadian clock is relatively depressed, so it's still feeling kind of sleepy.
But you're homeostat is fully refreshed, so you're up here, let's say, in alertness land.
Now as the homeostat is going down, the circadian chunk is going up.
So for a sustained period of the day, say till early evening, you have roughly level alertness.
Believe it or not, that's what really happens to people who are well-rested.
They are awake during the day.
Some time in early evening when the circadian clock turns downward, things start going downhill for you and your alertness starts steadily dropping.
At some point it drops enough that you -- well, that people who are not abusing the system -- go to sleep.
So now, you're asleep so you're not terribly alert.
So what's happening now is, so you're down here in asleep land somewhere, now your homeostat is recovering, right?
But in the early portions of your sleep, or during the course of the night, this circadian thing is still crashing down.
So even though you may have climbed up here already on the homeostat, you're crashing down to here and the circadian thing is saying stay asleep, stay asleep, it's good for you.
You should be asleep for this sustained eight hours.
Then it turns around in the early morning hours and 4:00, 5:00, 6:00 a.m. you wake with the chickens and you're bright and perky and alert, like you are every morning.
You're back.
So what you get out of this system, if it's being allowed to work properly, is a sustained period of alertness during the day, and the sustained bout of sleep during the night.
If you crash the circadian system, for instance, you end up with broken sleep.
So, you blow up a mouse's circadian clock, and what you get is a mouse who runs and naps, and runs and naps, and runs and naps -- doesn't have this organized pattern to it.
Now, the great tragedy, of course, is that you guys who are in the peak of your ability to do this aren't doing it.
You're abusing the pants off it.
The structure of sleep changes over the course of the lifespan.
Babies sleep for a lot longer than adults do.
After early childhood, people have a fairly constant requirement for about eight hours.
The range is really about seven to nine hours.
People who say they require only four hours of sleep or something like that are basically lying to you.
Thomas Edison, right, he's always trotted out as the example here.
I don't remember what he needed, two hours a night, four hours a night.
Well, there was a reason the man had a cot in is office.
The people who don't sleep eight hours at night are catching naps when they can and they're building up a sleep debt the rest of the time.
The sleep debt is -- if you are Dement, William Dement, Stanford sleep researcher coined the term sleep debt, I think, argues that it may be the biggest health problem in the U.S. Why is that?
Well because all sorts of stuff from traffic accidents to industrial accidents and so on, occur basically in the wee hours of the morning when the circadian clock is busy crashing, because sleepiness is not good for behavior, as you may have noticed in various and sundry ways.
When you get older, it turns out to get harder -- for reasons we don't really understand -- to maintain this consolidated period of sleep.
So, it is a sad fact that I, for instance, do not typically manage to sleep eight hours solidly, anything like eight hours solidly, at a stretch anymore, even though I have long ago abandoned my various efforts to abuse the system.
I'd love to sleep a solid twelve, seven would be happy enough.
But as you get older it's harder to maintain it.
You can do it now and you're not doing it.
It's so sad.
So, what are you doing and why are you so deluded?
Well, in part you're being deluded by this circadian clock, because what happens?
You stay up all night, right.
So, where should we plot this?
What we don't have any colors.
Let's assume that you were once well-rested like before you arrived at MIT or something, and you stay up all night.
So you were doing fine till here and now you're going to stay up.
I'm getting sleepier here.
Well, there's the coffee and the Jolt cola.
Who knows what else smacking you around.
Stuff happens.
But you know, you're fighting a losing battle here.
At 4:00 in the morning and you're barely holding on, and let me tell you that psych paper that you're writing at that point is not your best work.
So there goes 4:00 in the morning.
But 6:00, 7:00 in the morning, you've been up all night.
What happens?
The circadian clock turns around.
I did it, I did it, I don't need your sleep, I'm so good.
I'm a real MIT student!
This is great, but it's all fake.
You think you're getting back up here, forget it.
You're in here somewhere.
Klunk, like that.
So you're ambling around in this business.
If you think that this is great evidence that I'm superman and I don't need to sleep, try it the second night.
I don't know how many of you have ever tried staying awake two nights.
The way to stay awake two nights in a row is to have somebody shooting at you.
Leading funders of sleep research are Army and Airforce, and they have a very particular interest.
Their interest is -- well, they have sort of two interests.
One of the army interests is we're going to take over this country in a hundred hours straight and our boys ain't going to sleep until it's done, and we don't want any ill effects of this.
Got some good drugs for us, we need drugs, we want good drugs.
That's for the Army research program.
The Airforce says, we can fly at night, this is cool, we've got really cool toys, we can fly at night.
It's really bad when the helicopter guy falls asleep, so we need him to stay awake at bad times.
So they're big time funders of this kind of stuff.
It is the fact that the sort of thing -- your general level of arousal will keep you up, and so the panicked sense that you're going to flunk psych if you don't finish the paper will do some good.
The panicked sense that if you fall asleep somebody's going to shoot you does a lot of good.
You can stay awake for a long time under that basis.
But it's not that you miraculously if you just can make it past here you get back up here.
This thing, if you don't keep recovering, it just keeps going.
Eventually you're going to fall asleep.
It's just going to happen to you.
So that's the broad sort of twenty-four hour scale of sleep.
Within that, there's a structure on top of that.
This is sort of a ninety minute ripple sitting on top of this circadian clock that runs all day and while you're asleep.
You know this during the day by the variations -- what would you may have noticed were reliable fluctuations in your own alertness during the day.
In fact, you can probably do this -- I should collect data on this some year.
I'll have to make up a little questionnaire.
The clock is about ninety minutes, this lecture's about ninety minutes.
So sometime during this lecture, you'll hit the nadir of that ninety minute cycle.
Actually, I can at least ask a general question.
How many people think they could identify within plus or minus fifteen minutes the point during an average intro psych lecture where they're going, if they're going to lose is, where are they going to lose it?
Many of us know that there are just chunks of time that are bad.
You might notice, if you check out, if you were to systematically survey those during the course of the day, of course, it interacts with things like when you eat your meals, when you get up, whether you ever got to sleep last night and things like that, but you may be able to peg a series of these spaced out at ninety minutes or maybe 180 minutes during the course of the day.
You see these at night, too.
To talk about that let's fill out this cute little chart on the handout.
If I just stick a bunch of electrodes on your head and record the electroencephalogram the mass potentials off of billions of neurons in your brain, what I will see grossly is high frequency, low amplitude stuff while you are awake.
So this is EEG.
Depending on where I put my electrode in the brain -- on the scalp, rather, I can get a lot of other interesting information out of it.
But for present purposes the important thing is that the mass activity of the brain is relatively high frequency, low amplitude stuff.
If I now take a look at you when you are deeply asleep, what I will find is low frequency, high amplitude waves, roughly corresponding to -- what's going on is in deep sleep, large bodies of neurons are firing together -- everybody's on then everybody's off.
If you're woken up out of that state, that's the I don't know where I am kind of state.
You're deeply asleep here.
The interesting thing, discovered now about fifty years ago, is that every ninety minutes or so you'll go into what we will call Rapid Eye Movement sleep for reasons that will be apparently in the next line of this table, where all of a sudden the brain goes back to this high frequency, low amplitude kind of signal.
Now, research over the course of the fifty years makes it clear this is not exactly the same as this, but it certainly looks awake-like.
So the person is still clearly asleep.
The interesting thing is if you wake somebody up out of this state, they will reliably report that they are dreaming.
Not all the time.
If you wake people up out of this state, they will reliably report that they are not dreaming.
Again, not all the time.
It's not a perfect divide.
But lots of narrative dreaming here.
Here you tend to get if you wake people up and say what was going on, you might get an isolated thought or some idea that was rattling around, but you won't get that I was flying naked through the infinite corridor with a panda on my back or something.
We'll talk about that later.
This is the nighttime manifestation of that ninety minute cycle that you also see running during the course of the day.
You can pick that up in the movements of the eyes, too.
That's the electro-oculogram.
While you are awake, your eyes move ballistically to the left and to the right.
Your eyes are sitting in one place then jumping someplace else and moving around like so.
The two eyes are moving together, so-called consensual eye movement.
In deep sleep, those ballistic movements, called saccades, disappear and the consensual movements are damped down to so the eyes are kind of rolling around in your head and they're not necessarily rolling in that same place.
When you go into Rapid Eye Movement sleep, the reason it was called Rapid Eye Movement sleep, it's the eyes look like they're awake again.
So again, ballistically jumping around and jumping together.
Is the pattern of eye movements related to what's going on in your dream?
Well, there's anecdotal evidence for it, but if you think about it it's very difficult to do the experiment in any terribly meaningful -- actually there's some interesting data in animals at this point.
In any case, the eyes look like they're awake here.
The eyes, they're doing the same thing that they were doing when they were awake.
You can see this in your roommate, particularly if your roommate has relatively thin lids.
You can often see the eyes moving around under the lids, and if you watch your sleeping roommate, your sleeping roommate may think you're a weirdo when he or she wakes up.
But you can actually see the eyes jerking around under the lids.
If you have an unusual roommate who sleeps with the eyes open -- this actually happens.
The reason for sleeping with your eyes closed has less to do with -- it presumably has something to do with cutting the light out, but it had mostly to do with keeping the cornea hydrated, keeping it moist while you're asleep.
If you sleep with your eyes open, you wake up with nasty, scratchy, itchy eyeballs.
Because if you think about it, you don't sleep with your ears closed.
It's perfectly possible to use your brain in some fashion to shut off enough of the ambient outside stimulus to go to sleep to get rid of the auditory stimulus.
So, in principle, you can do that with your eyes too.
But you close your eyes to keep them nice and moist.
So some people that doesn't work terribly -- I traveled across France in a train with a guy who fell asleep but his eyes were open, and it's like traveling across France with the undead.
Because at some point he went into REM sleep and his eyes were going all over the place, but he ain't looking at nothing that's there.
It was a little strange.
So, so far, awake and dreaming sleep look the same.
If you think you're having trouble telling whether your roommate's actually asleep, here's how you can tell.
You measure muscle tone, electromyogram.
When you are awake, your muscles are on, right.
You have good muscle tone.
That's why your head doesn't fall off of your shoulders and I can stand upright and stuff like that.
When you are in deep sleep, actually your muscles remain on.
The muscle tone remains high.
It's when you go into REM sleep that you lose muscle tone.
You can see this in your friendly neighborhood cat, because all this stuff shows up widely in mammalian species.
So, your cat who's sleeping like this, that's a deeply sleeping cat whose muscle tone is doing just fine, thank you.
Your cat who's sleeping like that is a cat who is in REM sleep and has lost muscle tone.
Now, why would you turn the muscles off in REM sleep?
[INAUDIBLE PHRASE]
Yeah.
If you're busy being Superman and leaping tall buildings in a single bound in your dreams, it's a really good idea if you don't actually try that out.
Not only are you protecting yourself, you're protecting anybody else you happen to be sleeping with.
A known sleep disorder in older men is that this blockade of voluntary muscles fades off, and the result -- and this is a known disorder because older women bring the older men to the doctor, because their husband is thrashing around in the middle of the night and whacking her.
This is a real problem, people get hurt.
So, you want the muscles to go off.
There are a number of corollaries here.
It turns out that sleep walking -- Shakespeare has it wrong.
Lady Macbeth is clearly dreaming about having murdered Duncan and all that when she's sleep walking.
Typically, sleep walking is not acting out your dreams.
It is a version of non-REM sleep, and it's much more typical in children than in adults.
How many of you know that used to walk around at night when you were--?
Little kids wandering around, their eyes are open, but nobody's home here.
I don't actually know that much about it.
My theory is that part of what's doing this is it's your bladder telling you to get up and the rest of your brain's saying we want to stay asleep here.
My theory is based on one of the great moments of child-rearing in my household, which was one of my children.
We were awakened in the middle of the night, my wife and I, by one of my son's yelling "No, not here!"
to the other son who was about to use a trash can for the wrong purpose.
Anyway, it all worked out in the end.
But this is a deep sleep phenomenon.
Talking in your sleep is often, apparently, talking out of -- you're not going to hurt yourself too much if you don't blockade your jaw, right?
It's the main body muscles that are the real threat.
So sometimes people will end up talking in their sleep, often to the amusement of other people in the room, or to the inconvenience of other people, or of your own relationships, if what you're talking about is somebody else other than the person who's currently present with you.
Anyway, it's not clear -- when we get to meaning of dreams we can talk about this.
Anyway, so this is the sort of within the circadian cycle, this is the sort of structure you get in sleep.
Oh, now there's much more to this than what I'm telling you.
Deep sleep can be subdivided into multiple stages that differ in the activity of different parts of your brain and so on.
So now between that twenty-four hour cycle and this ninety minute cycle, there is yet another level of structure to a night's sleep where there's more non-REM, slow wave, deep sleep early in the night.
If you have a consolidated bout of eight hours of sleep, a normal night's sleep, what you will see is more of this stuff early in the night and more of this stuff later in the night.
This is accompanied by systematic changes in the neurochemistry of the sleeping brain and so on.
There's an intermediate level of structure to sleep also.
The impression one gets is of a beautifully structured system that is designed to do something.
So, I mean apart from the obvious, it's designed to keep you from feeling sleepy.
One wants to know, well, what is sleep for?
You'd be surprised if a third of your life is spent in this state for no good reason.
Oh, look at that, it says Part Two: Why do we sleep?
It suggests that there are several possible reasons here.
Anybody care to offer any possible series of why we sleep?
Nobody's got a clue.
Yes?
[INAUDIBLE]
So there's a sort of a basic rest and regeneration notion.
That you use your muscles all day, it builds up all those metabolic byproducts in muscles that you learned about once upon a time.
Sleep/rest will give you a chance to clear that out.
So certainly, one possibility is it's an enforced rest period.
You could enforce rest some other way, presumably.
Well, it could be good for you to rest, period -- just lie around and do nothing.
But sleep kind of enforces that.
Yes, there was a hand over there somewhere
[INAUDIBLE]
So, there's a whole class of theories, and these are actually the ones where there's, I suppose, the most current interest that has to do with learning and memory.
That somehow what sleep is about is tuning up your memories and you're consolidating memories, improving learning or something like that.
Let's talk more about that in a minute.
But if you're sitting there saying, I'm a real MIT student, I don't need to sleep.
You might worry about theories that say sleep is vital to the consolidation of memory.
Yeah?
It's a lot easier to rest during the day so we don't need to waste all the energy [UNINTELLIGIBLE]
So another possibility is energy conservation.
You could be awake all of the time, but if you can't hunt when it's dark, you might as well go and power down the disk drive or something like that and not use as much energy.
Your brain's pretty active while you're asleep, as you may gather from this, but the rest of you, of course, is getting some rest.
Anything else?
Any other good possibilities?
Along with that energy saving one -- we can motivate this one related to energy saving by asking where do you sleep when it gets dark?
[INAUDIBLE]
In bed.
That's seems like a good place.
Where did you put your bed?
[INAUDIBLE]
Anybody big on sleeping on, like putting their bed out on the sidewalk?
Those of you who are hunter/gatherers, when you're tired you just lie down in the middle of the field?
No.
Because it's not really very safe.
So another possibility is that you want to sleep for purposes of protecting yourself at times that are-- as a corollary of this you can't hunt all day thing.
There are other things out there that have adapted themselves to the niche that you are not in.
Like take hunt at night, so if you're not going to hunt at night, not only do you want to be calm and quiet, you want to be out of the way, because otherwise oh, look, he's asleep, let's eat him.
So there's a sort of a safety aspect to it.
The other part of the safety aspect is that not only don't you want to hunt at night, if you just go down wandering around aimlessly at night, you're going to falli in a hole and hurt yourself.
So, you've got to get in the hole first and go to sleep.
Do I have any other brilliant theories here that I wanted to mention?
That's pretty good.
You guys managed to do a quick survey of the leading theories.
As I say, a lot of the cool, current data centers around this idea of sleep having a role in learning.
Let me tell you a little bit about that.
The picture that's on the handout is an illustration of what looks like a really trivial task.
The question is, is there a little vertical region in that display somewhere?
The answer is well, yeah.
Now I can make that task arbitrarily difficult by flashing the stimulus very briefly and following it with a mask of some sort.
So I can make it hard.
I can make it as hard as I like.
If I do that, what I can do is measure, let's call it a threshold time.
So maybe in order for you to get this task correct 70% of the time, I initially need to have the stimulus on for 200 milliseconds.
If you practice it for a while, you'll get better and that'll level off at some point.
What a number of people have -- well, is it Carney and Sagi I think?
Doug Sagi is the guy I know.
What a number of people found, starting I think with Carney and Sagi, is that if you let people sleep, in particular if they are allowed to have REM sleep, that performance gets -- well, here's let's have him sleep for eight hours here.
That performance gets better, as if something happened while they were asleep.
If they don't sleep during this period they don't get better.
So it's a sleep dependent improvement in behavior.
This is a very specific little bit of learning.
This isn't just a general overall improvement in behavior.
If you trained people with that little vertical task, that you're doing a task, is that region vertical or horizontal, right.
That the three thing is vertical or the three thing is horizontal right It's always I'm looking here and it's always down and to my left.
I learn to be faster here.
I don't learn to be faster here.
The improvement in learning is specific to the little piece of retina that I have trained.
So it's very specific sort of learning that seems to be dependent on getting a decent night's sleep.
Now, interestingly, there is evidence -- how many people here are nappers?
How many people are nappers just because it happens?
We can do it this way.
How many people here think that if they take a nap in the middle of the day, they wake up feeling better for the experience?
How many people think if they have a nap they wake up feeling cruddy for the rest of the day?
There are people who find naps to be refreshing and useful, and there are people who find it being disruptive.
It was certainly a big improvement in my professional life when I reached the level where I had an office of my own.
The biggest part of that was it made napping a great deal more graceful.
After lunch I could take the latest issue of Vision Research and prop it up and stare at the same two sentences for like fifteen minutes and I feel much better thereafter.
There is evidence that a properly timed nap is just about as good as a night's sleep for this sort of perceptual learning.
This is work done by Sara Mednick originally, up at Harvard.
This is new stuff.
We don't really quite understand fully how far we can take this.
It's the sort of thing that your average MIT student is tempted to take and run with, right.
Oh man, I can get the whole benefit of eight hours of sleep if I just go for -- I think it's ninety minutes between like 2:00 or 3:00 in the afternoon.
That works.
Great.
Now I can stay up all night.
Anyway, we don't know that yet.
But Mednick's study is an interesting suggestion that sleep during-- napping sleep can serve as a same sort of perceptual learning kind of role.
But look, one might actually wonder whether anybody other than vision researchers cares about whether or not you can find little briefly flashed lines in one corner of your visual field.
That's not what you want to know about learning and memory.
What you want to know is is anything really useful happening while you're asleep.
In particular, one of the things that people want to know is, can you do serious thinking in your sleep?
One thing you can't do is study in your sleep.
There have been endless efforts because people smelled money at the far end of this to produce like tapes that you could stick under the pillow.
Record Gleitman and why waste precious waking hours reading Gleitman?
I'll just have somebody else read it to me while I'm asleep.
Sorry, the overwhelming evidence is that nothing happens.
The mild effects that are sporadically shown here come from periodic awakenings during the night.
That also goes for sleeping through the lecture.
If you're sound asleep during the lecture it's lovely that you were here, but you'll have to ask somebody else what actually happened.
So that doesn't work.
But can you have insights, or can you do your problem sets during your sleep in some sense?
How many people have had the experience of being sort of dead-ended on a problem, it's time to just give it up, you go to sleep and you wake up and somehow the answer kind of seems to be there?
It's a not uncommon experienc and there are lots of anecdotes in the literature about this.
One of the famous ones is the guy who first described the ring structure of benzene, claims that he had a dream of a snake biting his tail and he woke up knowing that benzene was a ring or something.
It doesn't always work, by the way.
My father spent his career as a physicist and had the same experience.
He's working on a problem, not getting anywhere.
Goes to sleep, wakes up in the middle of the night, he knows the answer, and he knows you don't remember stuff from the middle of the night, so he writes it down fast and goes back to sleep.
Gets up the next morning as predicted, has no recollection of what the great answer was, but he doesn't remember writing it down, which is good.
So he looks at it and he sees that the answer is, "All the world is fused with the smell of cinnamon," which I'm sure was an answer, but was it necessarily the answer he was looking for.
So this notion that you could actually be doing something productive in that sense during sleep was really in the realm of anecdote until this year.
I put the reference on the paper by this group in Germany led by -- I suppose since he's in Germany, he must be Wagner et al.
Beautiful experiment.
You know these number games where you get a string of numbers and you have to predict the next number in the series?
Well, I can't remember the particular one here, which is just as well because at MIT all that would happen is if I presented it, you'd be sitting there for the rest of the lecture trying to work out what's the rule here.
Anyway, it's a clever -- go look at the paper.
It's a beautiful paper.
But they picked one of these things where there were two ways to solve it.
There is a sort of a laborious multi-step thing.
You add the last two numbers and then divide by the third fac number.
It's some weird thing.
You can do it, but it takes a long time.
So, if you measure the amount of time for -- so this is going to be time again.
Measure the amount of time to get the answer.
People are initially very long and then they get better and to get to some sort of stable.
But there's a trick.
If you have this flash of insight, you realize I don't remember what, but there's some trivial way of knowing what the number is, and if you get it, it's trivial to see in the data because your time drops like a rock and it stays there.
So now what you do is you take people, you train them on this.
They're at this the high asymptote here, and you send them away.
You either send them away for eight hours of daytime stuff -- come back tonight we'll test you again.
You send them away -- I don't remember, do they actually keep them in the lab?
You can keep them in the lab and keep them awake.
Anyway, you send them away at night because you want to make sure that it's not a day/night thing, but don't let them sleep.
Or you have people who go to sleep and have eight hours worth of sleep.
The finding is that the next day -- eight hours later rather -- some people are here, some people are here or rapidly get here, and 50% more of the people who had a night's sleep got here.
It's not that everybody went to sleep and had this kaboom of insight.
But seemingly something about that night sleep produced this rapid burst of insight in many more people in the sleep group than in the two other groups.
It's I think the first really good evidence, experimental evidence, for an ability to get something more like higher level cognitive thought happening while you're asleep, Not just these relatively low level perceptual learning kinds of things.
So, the general evidence suggests that sleep has an important role to play in managing your cognitive light in some fashion.
It probably serves a role in consolidating memory.
It probably serves a role in problem solving.
It certainly is the sort of thing that one might think about as one is saying I'm going to stay up all night reading Gleitman because tomorrow's the final exam.
Well, if you haven't read Gleitman before the final exam it's probably true that reading Gleitman all night is better than hoping that eight hours of sleep will produce a flash of insight about the imagined contents of Gleitman.
All right, let us discuss -- I'm supposed to tell you what's going to happen when you're sleep deprived I see, but let's do that in a minute after you wake up the person next to you who's dozed off.
Because they want to hear this piece.
[INTERMISSION IN LECTURE] All right.
So what happens to you, you were clearly built to sleep.
What happens if you abuse the privilege and you're running a sleep debt?
And the answer is most people here, in fact, most people in America are running some sort of a sleep debt.
What is the cost?
Well, you can measure cost in a variety of different ways.
You can certainly measure cost in terms of things like industrial accidents, car crashes and stuff like that.
The classic sleep deprivation car crash is a crash on a clear straight road on a moonlit night -- you're a nice long distance trucker who's abusing the rules about how long you're supposed to be able to drive at once.
You're out, there's no car in the vicinity, the road is straight, it's not bright light, but you can see everything.
The road turns a few degrees and you go straight out into the cactus or something like that.
Because when you get really sleepy, if you take a look at people's brain waves, you can watch them flipping into these little micro sleeps that don't necessarily last very long, but at sixty miles an hour, eighty miles an hour, ninety miles an hour, they don't need to last very long for you to end up in sage brush.
There are plenty of ways to measure it in the lab too.
If you sit somebody in the lab and tell them all I want you to do is push a button when I turn on this little light.
The only thing you need to do.
What you discover is that people are about 200 milliseconds slower at adverse circadian phase, at the bottom of that circadian trough than they are at the top of it.
And on top that just general slowing.
There are also a whole bunch of times when they'll just miss the light altogether.
The light will come on and the subject will be just ah and not do anything about it.
Now, 200 milliseconds, what's the big deal there?
It's only a fifth of a second.
Well, again, at ninety miles an hour it makes a difference and if you're flying your supersonic jet around the skies of Iraq or something it makes even more of a difference.
So there's a generalized slowing.
There is a tendency to trade-off speed and accuracy more than usual in favor of speed.
So people tend, when they're sleepy -- we've actually done this in our lab.
So if you're doing one of these visual search kinds of things where you're looking for something exciting like where's the red vertical line, people, if you go too fast you make mistakes.
People when they're sleepy tend to be willing to tolerate higher error rates.
Again, that's the sort of thing that is brought out into the real world if your error rate goes up by two or three times, that could be a real problem in doing real world tasks.
It is harder to prove but pretty clear, perhaps introspectively, that being sleepy makes you stupid.
You just don't think as well.
For a long time it was kind of hard to show any effects of sleep deprivation because people tended to do things like ask you to add up columns of numbers, and as long as I smack you around enough to keep you awake during the task, you can do that.
But if I ask you to do, I don't know, your problem set when you're really sleepy, it turns out that the throughput on more complicated cognitive tasks goes downhill.
You're just not as good at it.
Will it kill you?
How bad can it be?
The answer is probably yes.
Not obviously at the level that people inflict on themselves, but prolonged sleep deprivation will kill you if you are a rat, at least.
What kills you is not that you just get so sleepy that you fall over dead, but that staying awake, or more to the point being kept awake, which is what's required if you're going to go any extended period of sleep deprivation, is highly stressful, and stress will kill you.
The rats who die in sleep deprivation studies die of things like bleeding ulcers and stuff like that.
Symptoms that look for all the world like the effects of severe stress.
You can also ask whether or not deprivation of particular kinds of sleep is particularly damaging.
So, for example, it's relatively easy to deprive somebody of REM sleep.
You can just monitor their eye movements or their brain waves and wake them up every time they go into REM sleep.
They will sleep for most of the night that way, but they'll have little or no REM.
You do get effects of that on these perceptual learning tasks -- REM seems to be important in that.
But you can be REM deprived for an extended period of time without collapsing in a heap.
There are two reasons we know that.
One of them is that people have done experimental studies of this.
The other is that one class of antidepressants known as Monoamine Oxidase Inhibitors, MAOIs, systematically suppress REM sleep, and these patients report that they don't dream.
But they don't have obvious evil side effects.
We don't quite understand how that relates to these theories.
People on these things don't say oh my goodness, I can't learn anymore.
So there's something we don't understand about the relationship between REM and learning there, because otherwise these antidepressants would have a powerful side effect, cognitive side effect that we don't see.
The other way to do this in animal studies is to keep an animal REM deprived for a month at a time and see if the animal show these sort of stress symptoms, and the answer seems to be, well, no, not really.
If you get sleep, even without REM sleep, the stress part can not be there.
Oh, my favorite technique for REM depriving a cat in this case, this is a low REM -- if you want to REM deprive your cat, here's how you do it.
You put the cat -- take a flower pot, turn the flower pot upside-down, put the cat on the flower pot in the middle of a little artificial lake.
Eventually the cat goes to sleep, right.
Then the cat falls into REM sleep.
Falls off into the water.
Cat doesn't like to fall into the water.
Cat wakes up.
And the cats learns the not go into REM sleep.
In some sense, trains itself not to fall into REM sleep, with the result that you can REM deprive the cat for an extended period of time.
This is a very weird study.
I don't where they got their measures.
The only thing I can remember that the only thing they found that was unusual about this cat is that a normal cat, you got the cat food, if you put a July the 4th kind of sparkler in the cat food, a normal cat will not eat the the cat food, but a REM deprived cat will.
Why?
I don't know.
I don't know nothing about this.
I only know what I read in the literature.
Which reminds me, however -- actually, this should have reminded me.
That the most practical piece of advice, apart from the fact you should get eight hours of sleep at night.
Since you won't get eight hours of sleep a night, I notice, I've been lecturing to this course for twenty plus years, and in spite of my exortations that your physical and cognitive well-being really relies on you going to sleep at night, none of you are going to pay the slightest bit of attention to that.
So the more practical piece of advice I can give you is how to sleep in class.
The trick is -- I mean obviously what you want to do is you want to sleep in a way that kind of masks that fact.
The difficulty is this REM sleep business.
The problem is the muscle tone thing and it's the head bob portion of the-- So I can look out there and watch the whiplash and that's not going to work.
So, what people figure out on their own is that what you want to do is you want to give the illusion that what I or somebody else is saying is so deep that you had to close your eyes to think about it.
Now, there are a couple of important principles here.
One of them is that it's less convincing that you're thinking deeply about this if your mouth falls open.
If you're drooling it's really bad.
This is good.
Support for the chin is good.
The other thing that you learned in like 801 was that three-legged stools stand better than two-legged ones, right?
So, I favor these.
I'm really thinking very hard about what you're saying.
That can actually be quite -- in fact, you're not fooling anybody.
You know perfectly well you're sound asleep.
What?
[INAUDIBLE]
Oh, keeping the airway open is good.
That was the winner of the most embarrassing moment in Intro Psych Student Experience.
This was actually the first year I was teaching the course.
I was co-teaching it with another guy in course nine at the time and he was lecturing.
I've long ago stopped figuring that this was an editorial comment when people fall asleep.
But anyway, somebody sitting right about over there fell sound asleep.
But nobody was sitting next to them.
They fell sound asleep and they started to snore.
Eventually this got like loud enough that my colleague had to say somebody's going.
If you want the definition of embarrassed -- the definition of embarrassed is you're sitting in lecture, everybody's looking at you and you have no idea why.
So, sitting next to somebody's another good strategy if you're actually trying to stay more or less alert.
My old doctoral advisor and I had this as a pact when I was in graduate school during the Friday afternoon, terrible time, departmental colloquia.
We would sit next to each other and if it was actually talk that we cared about, we had an agreement, if I go out, poke me, because I kind of want to know what this guy is talking about.
He was department chair.
Turns out to be very important, because one of the rules of you're department chair is you have to be able to ask an intelligent question at the end of the talk.
Really good department chairs can do that after sleeping through the talk.
My advisor, who was otherwise an exemplary department chair, was caught up short on this at one talk where he asked a very good question, but one that he wouldn't have asked had he been awake for those twenty minutes of the talk.
Because the guy had discussed this matter extensively.
So if we would poke each other to stay awake.
The guy joined our lab and he sat two seats away from me.
He went out cold.
That's OK.
But he started shouting.
In Japanese.
Actually, it was probably a bonus that it was in Japanese because whatever he was shouting was incomprehensible to anybody else.
I made this sort of dramatic lunge to get to him before he could manage to get out too much.
So now if you're worried about that, you kind of want to be sitting with your friends here so that when you start talking in your sleep, somebody can get to you before it becomes really desperately obvious.
All right.
I suppose the bottom line is that sleep deprivation will not kill you, but it won't do anything good for you, except perhaps getting you through the your work.
Now, let's see do I want to -- you know this looks like a pretty good place to entertain questions and we'll go on to dreaming after Thanksgiving.
Any sleep questions while we're here?
We got a question, then you can run for it
[INAUDIBLE]
Well, the piece of that is right -- so the question is do you want to sleeps in cycles of ninety minutes?
Ideally, you want to wake up on your own.
Typically out of the end of a REM stage is what would normally happen.
Yes, you will tend to feel more refreshed there than if you wake up out of this.
But most of us wake up when the alarm clock goes off and sorry.
But if you can manage to adjust that to wake you up -- even if you have to move it a little earlier, you may feel better if you're waking yourself up out of a REM state.
Yup?
[INAUDIBLE]
Oh, that's the neck floppy thing.
When your muscles are going, when you're losing muscle tone and drifting off into a REM state, bits of you are kind of going where gravity wants to go and that can feel like a falling sensation, which inconveniently can sometimes lurch you back awake, or conveniently if you happen to be going out during class.
But I think that's typically an effect of the losing muscle tone part.
Last chance, here we go
[INAUDIBLE]
Oh, if you really live in really, really northern -- sleep disorders go up as latitude goes up or way down.
But as long as you're living someplace where the sun comes up every day, that's not an issue.
But yes, absolutely, if you go and live someplace where it's dark for six months of the year, people have disruptions of their circadian clock and they start free-running.
All right.
Drive carefully.
Have a good vacation.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu
Good afternoon.
Now that all you turkeys are back from Thanksgiving.
I shouldn't have said that.
How was Thanksgiving?
Good.
As I'm wrapping up the sleep and dreams thing, I should look around and see whether people look like they're awake.
How many people discharged a substantial amount of their sleep debt over the last few days?
Good.
The rest of you are still rolling around.
I am going -- I'm going, but I'm coming back.
I think I'm going to depart a little from the organization of the lecture that's on the lecture notes, and I will trust on your talents to figure out where -- I might even point out where I've gone to -- but I'm going to re-organize things a bit, because it occurred to me that I can organize at least the first part of the lecture in the form of a sort of a summary statement about Freud, who I'm going to abandon fairly shortly here.
And I can work two examples -- one of them about dreams and one of them about abnormal psych, where Freud or Freud's followers are wrong in both cases.
But in one case, we can do this business that I've been doing for the last few lectures of saying look, we can salvage something important out of Freud's ideas, and in the other case, which will turn out to be the abnormal psych case, we'll say that, in this case, Freud's followers are simply wrong.
And have been superceded by later work.
So let me start by picking up in sleep and dreamland talking about dreams.
Well, I won't ask if anybody had any good dreams over the course of their catching up on sleep over Thanksgiving.
I know what I'll ask.
I will ask how many people did sleep and dream stuff in recitation?
Not many.
OK. How many people can remember?
Not many.
Oh, good, good, that's encouraging.
All right, that allows me to ask one of the things that tends to surprise people sometimes is that there's a fair degree of commonality in at least some of the dreams that people have.
They seem like such idiosyncratic personalized stories, that it's surprising to discover that some of the plot lines go across individuals.
It's a weird state, right.
I mean it's clear there are weird aspects of the sleeping state.
The normal laws of physics seem to be suspendable -- you can fly sometimes, do things like that.
You can fall for hours and hours and never hit the ground.
The normal laws of reality testing or the awake state of reality testing seems to be up, because you can fly and it doesn't strike you as weird.
When you wake up you say that was weird, but if you think about it, while you were flying around or doing whatever other video game kind of thing you were doing in your dreams, yeah, that was just the way things were.
If you have the same problem with reality testing, when you're awake you end up in the part of the course on abnormal psych that we'll get to in a minute.
It's diagnostic of having a variety of forms of mental illness if you cannot tell the difference between dream state and awake state, that's not healthy.
When you're dreaming you can't tell, but you're not clinically insane.
There was one other sense in which it was different from the awake state that I wanted to make sure that I mentioned, but now I don't remember what it is.
Oh, yes, just to point out that the transfer from experience to episodic memory does not work well for dreaming.
This is why you forget your dreams so readily is that you know when you wake up that dream is just sort of there sitting around in some sort of short-term buffer, and if you don't do something like write it down or repeat it yourself you discover that those normal mechanisms that take your experiences and transfer them into some sort of long-term memory, they're just not working.
So, that said, let's see if we can conjure up--yup?
Especially when people say they know they're dreaming
All the stuff about different states of sleep and awakeness should be seen as not absolutely categorical.
So there are these borderline states -- the sort of jargony term is liminal states where you're in between being asleep and awake.
In the course of a normal night's sleep, you'd probably wake up out of REM sleep and different bits of you are coming online at different times, and you may be aware, still asleep, but I'm not quite asleep.
How many people have had the experience of saying I'm waking up here and I want to know how the story comes out?
Yeah, yeah.
It's always cause you're waking up at the good parts.
Or how many of you have the other experience, which I think maybe what you're getting to which is I'm asleep here and this dream is not a good dream and it's time to wake up out of this dream.
I'm going to wake myself up.
Fewer.
Now, are there people here who are experts at what's known as lucid dreaming?
Lucid dreaming is a sort of a halfway state where people claim to be able to control the plot line of their dreams.
There are books in the bookstore that will both claim that they can train you to do this, and maybe you can, I don't know.
It sounds good.
I mean all those people who said, oh man, I woke up and I wanted to hear out the thing came out.
If you'd go and train yourself to be a lucid dreamer maybe you can go and keep it going.
I have these extremely frustrating dreams that I'm sure a good analyst would have a field day with, but I choose to think that they are basically nice, simple, transparent dreams.
I like to ski, I dream about skiing, but in my dreams I spend a great deal of time getting up the mountain and I never get to ski.
I get to the top and the snow melts or something like that.
No, I don't want to hear your interpretation of what this is all really about or anything -- I can do that, too, and make up all those good stories.
But in any case, I sometimes think I'm going to get one of these lucid dreaming books because dammit, before I die I'm going to get to ski down that mountain.
The conditions are always a great, look a lot better than they do in the northeast when you actually go skiing.
So, let us consider a couple of plot lines.
Some of the things, there's a sort of element that show up in people's dreams.
How many people have had falling dreams?
How how many people have ever hit the ground?
Usually you don't hit the ground, right -- you wake up just before you hit the ground.
Oh, there's a whole raft of sort of pseudo-scientific nonsense about--.
How many people have ever been killed in their dreams?
A fair number.
Well, that's presumably a counter examples to the theory that if you die you in your dreams you die for real.
It's one of those great assertions.
How would you know that was true?
Right.
What were your dreaming about?
I don't know, I'm dead.
But typically, typically people report that you wake up just before you get killed by the whatever it is that's going to kill you.
But the plot lines get more elaborate than that.
How many people have had dreams -- this is typically a dream of grade school, the grade school era where you show up in school inappropriately dressed in some fashion.
All right, anybody care to describe one of those?
Sorry, I heard a--.
Somebody wave a hand if you want to actually tell us the -- you don't have to tell us your dream.
It's fine.
Nobody wants to-- How inappropriate was it?
I mean it doesn't get much more inappropriate than I was buck naked, I suppose, which is the stand-- My version of it happened to be that -- I still vividly remember what must have been a kindergarten dream because I ended up in my kindergarten classroom of driving down the street from my house to my school in my crib, which presumably, by the time I was in kindergarten I was long out, and I ended up in school in my crib in my pj's.
All right, so you don't have to tell us what you weren't wearing.
There is a characteristic of this dream which is that once you get there, once you end up in class, what happens?
Anybody got any intuition about the common plot line here?
People point and laugh
Oh, people point and laugh.
Do you believe it?
Where's the inappropriately dressed crowd?
Raise your hand if you were in the inappropriately dressed class.
It's dropped, the numbers are dropping.
OK, not inappropriately dressed right now, this is in your dreams.
Inappropriately dressed.
How many people agreed that what happened was they pointed and laughed?
No, it's only in Russia.
The characteristic form of this dream is that nobody notices.
I mean that's not to say that you're weird for having the variant -- either that or you're making it up, I don't know.
But that you would think, I would think if I showed up, if I drove my crib in here, everybody's going to point and they're going to laugh.
But they don't.
This is assumed to be some sort of a school anxiety dream.
School anxiety dreams of some sort are quite common.
They also morph as you get older.
So, how many people have had a dream of the form where -- well, all right, let's rephrase this.
We'll get the form out of you.
Has anybody had what they would consider to be an exam anxiety dream?
Anybody willing to describe that?
I don't think they mostly involve being grossly inappropriately dressed.
Yeah,
Before the psych mid-term I started [INAUDIBLE] psychological theories that don't actually exist
Psychological -- that's a novel form of the dream.
Any good ones?
Well, I won't ask you that
[INAUDIBLE]
OK, so one version is the I overslept and you missed the exam.
Variance thereon?
Yeah
I slept until about an hour before the exam.
Woke up and didn't study [INAUDIBLE]
OK, I haven't studied enough for the exam, and perhaps because -- something is going to make it uh-oh.
Yeah.
They all pointed at you and laughed at you, right
The exam was in some other language
Yeah, well that happens.
Yeah, the exam was in another language.
How many people have had the version where you realize in your dream that you have an exam in this course and you've never been to the course?
Medieval French -- I've got the Medieval French exam, I've never taken Medieval French.
Or there are also versions of this where you get to the exam and you realize that there's something about the exam -- the different language thing is a good example of this where you look at the exam and you realize there's no way I'm passing this exam.
Not because you didn't make up the right psych theory, but because I can't see the page in some weird way, or sometimes -- oh, somebody was telling me this morning of a great version where I realize I'm going to be late to the exam and it's down the other end of the infinite corridor, and the infinite corridor really is infinite.
You're just going down it forever.
Anyway, if you haven't had that dream don't worry, you will.
When I was an undergrad at Princeton, somebody did a study of this.
25% of the freshman class, 50% of the sophomore class, 75% of the junior class, and you can see where this goes.
It's an extremely common college era type dream.
And it doesn't get better after that.
All that happens is that, if you stay in academia, is that the form moves.
Any number of my colleagues have reported dreams where the -- this is the grad school version coming up here -- the dream where your thesis committee comes to you and says it was a big mistake, we're taking the PhD back, or that you have to take your oral exams over again -- the thing you crammed for months and then forgot all of it the next week, or something like that.
And then if you end up in my position actually teaching, it just flips around the exam anxiety dream where you have dreams where you get up in front of the class and you realize Medieval French, I don't know anything about Medieval French, what do you mean I'm supposed to talk about Medieval French for the next hour?
Now you may think that having anxiety dreams about not knowing what you're talking about is not that far from reality in the case of some of us who -- I won't say more about that anymore.
But there are these sort of common themes that run through dreams.
People have known this forever, and people have attempted to interpret dreams forever.
So there's dream interpretation in the Bible.
I'm sure in whatever culture you or your ancestors came from there's a canon of dream interpretation.
Have I mentioned Lucretius's great book De Rerum Natura, On the Nature of Things?
Yeah.
Maybe.
Some people think so, some people don't think so.
Oh well, what you gonna do?
Roman author, and it really is about the nature of things.
It's everything, you know, it's 1801 and 801 and 900, and it's all rolled in one great big book.
But he's got a nice dream theory in there in which he says that lawyers dream of court cases and sailors are wrestling with the winds and things of that sort suggesting, as Freud would later suggest that every dream reflects something about the preceding day.
Oh, let me say something about that.
I think I'm jumping ahead in my handout again.
Oh well.
This was always assumed anecdotally to be true.
You may have had this experience.
You do something all day and you dream about it all night.
There wasn't much data on this.
Bob Stickgold at Harvard Med has now gone off and collected some beautiful data on this.
What he did was he got people to play Tetris, a lot Tetris.
Then he sent them to bed with little wires on their head and when they went into REM sleep he kicked them -- pow -- what are you dreaming about?
Oh man, bricks, they're falling, man.
Nice clear evidence -- of course, there's a non-Tetris control group.
You kick them, oh I'm flying down the infinite corridor.
You know, something else.
The Tetris group dreamed about Tetris.
The coolest thing about Stickgold's experiment is that he tested a population of amnesics -- patients with the same basic problem as HM.
Remember HM, no new long-term memories.
Learn how to play Tetris?
Grr.
What are you doing?
I'm playing Tetris.
Five minutes later.
What were you doing?
I don't know.
Right, doesn't remember any of this.
So he takes these amnesics, they play Tetris for a while, they're having a great time.
You send them to bed, you kick them in the middle of the night.
What are you dreaming about?
Oh, bricks, bricks, the bricks are falling.
They were dreaming about Tetris even though they didn't remember ever having played Tetris.
That was pretty cool.
Anyway, there's now experimental evidence for this notion that your dreams incorporate material from the preceding days.
Freud asserted that all dreams incorporated something from the preceding day.
Is that absolutely true?
Well, that's one of those assertions like all snowflakes are different, that's kind of hard to prove.
But certainly material from the preceding day gets incorporated.
Is that a hand?
That is a hand
I was just wondering, so there an experiment done where they had these animals running through the [INAUDIBLE]
Oh, yes, thank you for reminding me
I was just wondering that [INAUDIBLE], when they were in, when the [UNINTELLIGIBLE], their brains, I mean is a fMRI where it's showing activity--
Answer's pretty good, she's going to give the rest of the story here.
None of you guys are from that lab, right?
It's done here.
So, if you look in the brain of, in the hippocampus of a rat, what you discover is the hippocampus not only is there encoding the rat's episodic memories in some sense maybe but, it's also serving as a spatial map.
So when a rat learns a maze, it develops a map of that maze, in effect, in its hippocampus, and it develops cells that are so-called place -- it's got place cells and the place cells develop responses that are specific to a specific place.
So, the rat's running around in a maze, and this cell goes off right here -- nananana -- go over here, different cell goes off -- nananana -- go over here, different cell goes off.
And as the rat learns the maze, you, the experimenter, can watch the rat learning this because you can watch cells in the hippocampus lighting up in order, in effect.
So now the rat does this all day -- nananana.
OK, the rat goes to sleep, the rat goes into REM sleep.
All mammalian species, except for the spiny anteater -- don't ask me why I know that -- all mammalian species have REM sleep.
So you kick the rat, you ask the rat what he was dreaming about.
Rat doesn't say nothing.
But, if you're recording from his hippocampus while he's asleep, what you see is the same cells lighting up in order again.
Who knows what the rat's actually experiencing, but what the rat's hippocampus is doing is running the maze.
This is, first of all, part of the argument for sleep having something to do with consolidation of memory and learning, and it's also an argument for the possibility at least that the rats are dreaming away there, too.
Well now, arguably Freud's greatest work is Interpretation of Dreams -- that's the name of the book -- big, fat book.
Still kind of fun to read but it's kind of long -- from 1900.
Remember that Freud was writing before we knew any of this stuff about REM sleep or anything of that sort.
And what Freud thought was happening was that pressure was building up in your unconscious.
So let's switch from sort of it dungeon model to sort of a plumbing model.
Well, we've got the boiler down in the dungeon and it's building up pressure -- this repressed material that we've been talking about.
It's building up pressure and it's going to blow the roof off the whole building unless we vent some of the pressure.
And so you've got to be able to -- whoosh -- get some of that out of there.
One of the ways to do it is dreams -- to release this while you're asleep.
Now what Freud said was that dreams start when some hunk of the preceding day, that he called the day residue, some idea, some activity, some something made an association with something down in your repressed -- in your unconscious.
And that allowed this unconscious prisoner to try and make a break for freedom.
The form of that break would be an effort to fulfill this repressed wish, if you like -- that's why it was called wish fulfillment -- meant that you were going to try to fulfill one of these repressed wishes while you were asleep.
There's a problem with that, said Freud, which is if you take the materials out -- the repressed material to be this stuff that's repressed because it's unacceptable to you, to the conscious you, to the ego, what's going to happen if you start dreaming about it?
Well, we've already gotten part of the answer here.
Many people here have had dreams so awful that they've deliberately woken themselves up out of them.
Now most of those dreams, I would imagine, were dreams where the monster's about to put the bite on you or something like that.
What Freud said is that look, if you were going to start dreaming about, oh, let's dismember my little brother -- oh, look at his little guts.
Stuff like that, that would be really gross and really disturbing and you would say to yourself, I don't want to be having this dream.
And you would wake up.
So what you need to do is to disguise the material in the dream.
So you've got, said Freud, the latent content of the dream that you don't know about.
The latent content is the repressed material that's trying to be vented out.
What you see is the manifest content, which is the storyline of the dream.
It's hidden material -- it is hiding the latent content from you so that you can sleep.
Freud argued that the primary role of the sort of sleep mechanism was to protect the sleeper, to keep you asleep and allow you to sleep, and allow this sort of venting of repressed steam to take place.
Were you to show up in analysis the reason your dreams would be, what Freud called the royal road to the unconscious, was if you could work back from the manifest content to the latent content, which was a job of analysis, not a job of picking up some supermarket guide to your dreams, but a job of working with you and an analyst.
But if you could go back from the manifest content to the latent content, you'd know something about what it was that was bugging you.
Now the fact that, like repression as a whole, the fact that you have these disguised dreams all the time doesn't mean that you're somehow sick and diseased.
It means that's part, again, of what it means to be human for Freud.
You're going to need repression, you're going to need to be able to bubble off some of this repressed energy, and dreams are a nice safe way to do it.
That's an interesting theory, but it runs into some major problems, which, of course, Freud didn't know about because he died in 1939 and REM wasn't discovered for another couple of decades or so.
When REM is discovered it becomes a lot harder to believe that you are boiling up this little poof of repressed energy every ninety minutes like clockwork.
That's a bit surprising.
Well, that's a bit surprising.
More surprising is if you look at babies, a one month old is REMing a great deal more than you're REMing.
How much repressed stuff does a one month old baby have to deal with?
Not much.
How about a lion?
Lions, they REM all the time.
Apparently predators show more REM than prey, as I recall.
So yeah, all the mammals are sitting there doing this REM thing, right, you know lions, except for like Scar in the Lion King and stuff like that.
There's not a lot of repressed work that a lion presumably has to deal with.
Or a cow -- cows don't REM that much, but you know, ah, I really wanted to hurt the grass.
I feel bad about the -- I don't know.
So presumably, now what's going on when you dream and when you go into REM is that there are centers down in your brain stem that spray of vast amount of activity up into cortex and light it up as though you were awake -- that's what you end up seeing in REM.
One of the senses in which Freud was right is the sense in which dreaming does protect the sleeper.
If you light up -- you're trying to go to sleep, right, and I light up cortex by coming into your room, turning on all the lights and turning the stereo way up, you have trouble sleeping.
Well your brain's doing the same kind of thing when you go into REM sleep, but you've got to do something with all that cortical activity or you're just going to wake up.
So you turn it into this storyline for who knows who to watch on the big wide screen TV in your brain.
In that sense you do end up protecting your sleep by dreaming.
In fact, many -- Freud wasn't thinking about alarm clocks, but an example of this would be how many of you have incorporated your alarm clock into a dream and managed to successfully stay asleep?
Well that's sort of a non-Freudian sense in which the dream engine is protecting your sleep.
I want to stay asleep here, there's a loud noise, well I don't know where this loud noise is coming from, my brain stem's making these loud noises all the time, so let's just pretend it's from the inside and not the outside and we can dream that we're getting up.
Right, dream that we're getting up.
We'll dream that we're going to class and taking the exam and then we wake up and it really is halfway through -- don't do this at the final, please.
It's so lame.
Particularly since the exam's at -- the 900 final's 1:30, right?
Got plenty of time to wake up by 1:30.
But every year somebody wanders in at like 2:30, 3:00 o'clock with the big bags under the eyes, right.
So you do sort of believe them -- I just woke up.
Well, good, sit down, see if you can see the exam.
Anyway, so there's that sense in which Freud had a point that sleep was protecting the dreamer.
but is there any meaning to it?
Is there any sense in which there there's meaning in your dreams.
I mean after all you are dreaming because your brain stem decided to throw all the switches in your visual cortex or something and you're busy trying to stay asleep.
I think there is a sense in which you might be able to argue that you could get something meaningful at a dream -- you're not just looking at static on the brain.
Actually, I thought of a way to demo this, but I didn't bring any of the props because I only saw it when I saw this thing.
So, a piece of paper -- I've probably got a piece of paper.
Who's got a coin on them?
Somebody have a coin?
Let's see if I can make this work.
That looks pretty so far.
And I also need a pencil.
Somebody have a pencil -- it works better with a pencil than with a pen.
Oh my goodness, it's coming all the way from the cheap seats.
Visual search.
Oh, you wanted to do this because -- he's got a fancy coin.
This is very cool here.
All right, pencil, got a pencil?
Here we go.
OK, this is my brain.
Thank you.
Didn't you have something else you needed to do today?
Well actually, it's just my cortex -- it can just be my cortex.
Well that's my cortex under there.
This is my brain stem.
So now I will spray activity across my cortex here, and let's see if this sort of works.
It looks like squat, doesn't it?
Well you know what?
That's a really boring dorkey demo.
It's a fidelity issue.
I can see what-- Use a different pencil?
A wooden pencil.
No, no, no, no.
It looks just fine from here.
Here, we'll try the other side up, see if that works any better but I don't think it will.
Do it at more of an angle?
Hold the pencil up more of an angle.
All right, we'll try this one more time.
Oh look, maybe we're getting -- I don't know, what are we getting?
We're getting nothing.
We're getting nothing that you're going to see.
Oh yeah, no, you can sort of see something but it's really boring.
Well that was lovely.
Here we go.
Here's what I was doing that didn't work but makes a brilliant model anyway.
You can try this for yourself.
If you stick a coin under a piece of paper and rub a pencil over it, you can actually see the shape of the pattern on the coin.
Anybody here ever done brass rubbing or greystone rubbing or any of that sort of--?
No.
A few people.
All right.
Look, the point is that by spraying random noise across a structured object, in effect, I can see something of the structure.
Boy, this took more minutes than it should have.
By spraying random noise from your brain stem, across your particular brain, the particular dreams that you get are going to say something about that underlying structure.
So, Attila the Hun, when he was dreaming was not dreaming about trains going into tunnels or stuff like that, because Attila the Hun didn't know anything about trains.
You dream about trains, they may or may not -- trains going into tunnels, you know it's sort of a classic supermarket dream analysis image of a sexual dream or something like that.
It may or may not be a sexual dream, but the content of that dream, that set of imagery, is a function of your particular brain.
If your dream about Tetris after playing Tetris all day, it's because you were playing the Tetris, not because somebody else was playing Tetris.
So it could well be if you thought you had a process for going back from the storyline that's there to protect you from waking up with all that noise in your head to the underlying structure of your mind, if you had a process that you thought worked, then it follows that you might actually get some sort of interesting meaning out of your dreams.
Now it's quite a leap to decide that you actually know how to do this.
Analysts think they do.
People who write guides to dream interpretation think they do.
I think I'll just leave it at that as an exercise for the reader to go and decide whether or not they've got deep meaning in their dreams.
So that's an example where Freud didn't know the subsequent hundred years of sleep research, so if you actually read Interpretation of Dreams, a lot of it would sound kind of odd at this point.
But the core ideas that sleep needs to protect you, keep you asleep and that it's sort of a re-packaging of this activity of your mind, those seem to be core ideas that can still do a certain amount of work for us.
Now a place where the effort to apply Freudian ideas did not pan out is in the treatment of schizophrenia.
So I'm going to jump to talking about that, and I may or may not manage to jump back to the other things that I claimed I was going to jump back to.
But I might.
So, in the middle part of 20th century, mental hospitals in the U.S. were filled with chronic schizophrenics.
Schizophrenia is an extremely disabling disease when it's in its florid state.
So, schizophrenia, the word comes from split mind.
As it says on the handout, for five points free on the final exam probably, not the same thing as a multiple personality disorder.
The split in the mind is a split that 19th century psychiatry saw between emotional life and cognitive life.
It is, in fact, sort of an unfortunate term because it does make people think that it's the same thing as multiple personality disorder.
It's not.
Multiple personality patients can actually function at least somewhat in the world reasonably well.
Floridly schizophrenic patients do not work well out there in the world, that's why they end up hospitalized.
The sorts of symptoms that you get are hallucinations, hearing voices, paranoid delusions often of vast conspiracies to control your mind.
You can see this as sort of an adaptive response, right.
If your mind, if some chunk of you, in a sense, knows that your mind is out of control, it's not an unreasonable irrational thought to believe that somebody else is doing this to you.
So it may, in some sense, be reasonable to think, to imagine how people could come to believe that their mind is being controlled by somebody else.
The sorts of things that led to the notion of a dissocation -- I shouldn't say dissociation -- of a split between cognition and emotion, another characteristic of schizophrenia is so-called inappropriate affect.
That's just jargon for showing the wrong emotions at the wrong time.
The sort of classic symptom is laughing at the funeral.
You know, if you're just yucking it up at graveside or something like that, people will think that that is not normal, and it is, in fact, one of the characteristics of schizophrenia.
And that schizophrenia's sort of a messy diagnosis -- not every schizophrenic has every one of these symptoms.
But you can imagine that by the time you're gripped by fantastical thoughts and conspiracy theories and you're hearing voices and seeing things that aren't there, that this is pretty disabling stuff.
It's not so disabling that you can't be out there in the world, by the way.
When I was a graduate student, the MIT Department was called psychology.
One of the reasons, though perhaps not one of the more serious reasons, for changing the name was that my doctoral advisor who was chair of the department got tired of getting phone calls from people who wanted to know about stuff like abnormal psychology, which nobody at the MIT Department ever did anything with.
So he would always get the phone call or his secretary would get the phone call and she was instructed to say, don't call us, they do that stuff at Harvard.
I remember one time when I was a grad student that rather than phoning up, one of these people came to the lab and he wandered into the lab and said that he knew that we were controlling his mind with microwaves beamed from this location and he wanted us to stop right now.
I'm there sort of uh-oh.
But my advisor with great presence of mind I thought, walked over to the biggest rack of equipment in the lab, threw the biggest switch he could find, said there, does that feel better?
The guy said yes and left.
I'm not sure I recommend this as a treatment for delusional patients, but it worked in this -- and then we changed the name of the department.
In any case, it can be very disabling and it often can lead to hospitalization.
What would Freud have said about it?
What Freud said about it was these are not patients that I want to deal with, basically.
He doesn't write extensively about schizophrenia and says in various places that look, if you're going to do psychoanalysis, the first thing you need is a patient that you can at least talk to.
Anna O.
With her collection of hysterical symptoms may have been in psychiatric trouble some variety, but you could talk to her about it.
You could engage in this dialogue.
If you've got a patient who is delusional and hallucinating there's not much Freud thought you could do with them.
However, his followers thought you could and developed a theory growing out of Freudian psychodynamic ideas of where schizophrenia came from.
So in the mid-20th century, psychiatric hospitals were filled with schizophrenics and filled with good Freudianly trained analysts attempting to treat them.
What they thought the cause was the so-called schizophrenogenic parent, typically Mother.
Schizophrenogenic is a great word.
It's somebody who generates schizophrenia.
Anyway, a schizophrenogenic mother, what was the problem here, the problem was what the therapists perceived as a double-bind situation that the mother was somehow putting the child in.
The mother was saying come here, come here, come here, go away, go away, go away, I love you, I love you, I love you, get away, get away, get away.
Somehow both saying come near me and get away from me at one and the same time, and that this was literally driving the child mad.
Now why would they have come up with such a theory?
One of the reasons was that onset of schizophrenia is quite typically late adolescence, early adulthood.
Oh, that reminds me to reiterate or iterate for the first time, if I haven't mentioned it yet -- no, I think I did mention it.
Remember what I was saying about psychiatric hypochondria -- the notion that if I sit around and lecture about psychiatric disorders, you shouldn't go and be feeling your brain for the rest of the term saying oh man, I'm going to wake up with a dissociative disorder tomorrow and that's going to progress to schizophrenia and dimentia and I'm going to be dead.
The fact that schizophrenia has a typical onset in a population of about your age doesn't mean that tomorrow we're going to have 300 schizophrenic students in this class.
It does happen.
What convinced me -- a small aside.
When you're discussing abnormal behavior, which we are now, there are two senses of that.
One sense of that is all behaviors that you care to measure have some variability in them.
You can simply assert that everybody above two standard deviations or something is abnormal.
We'll see as we go along that there are times when that is not an unreasonable thing to do.
There are other times when it's a whacky thing to do.
All right, for a wacky example, virtually everybody here is abnormal in that sense, but then the axis is SAT score, right.
Most of you are up here in the upper tail of the distribution.
Sure it's abnormal but we don't consider it as pathological, right?
At least not in this community.
So the other possibility is that sure, the normal population is distributed like this, but there is a disease state of some sort that looks like this that's essentially discontinuous with the normal state.
Arguments have been made, notably by a psychiatrist in the '60s whose name is probably pronounced something like Shosh -- a Hungarian name.
Anybody happen to know the right answer?
Anyway, Thomas Szasz will do or Shosh or something.
In any case, he made the argument that in a book entitled Being Sane in an Insane World, that schizophrenia was simply a way to adapt to the fact that the Russians had lots of bombs and we had lots of bombs and we were all going to blow each other up, and that it was just one position on this normal curve.
It's an interesting argument.
But if you actually see somebody in the midst of a florid schizophrenic break from reality, it's hard to believe.
I had, when I was an undergraduate, the roommate of a close friend of mine had such a break.
It's unmistakable -- it's at once fascinating and deeply frightening.
This was a friend of ours who became delusional -- she was hallucinating, she absolutely couldn't function, and was scaring the heck out of her roommate and she eventually -- actually, she eventually did fine.
We caught up with her like twenty years later, and people do recover from schizophrenia and she was doing quite well, thank you.
But in any case, she was the data point, that at least for me, argued strongly for the notion that this is a different state, not just a position on a continuum.
In any case, the psychiatrists were busy trying to treat this with sor of Freudian psychotherapeutic techniques.
It wasn't working.
This wasn't because they were bad people -- these were well-intentioned people doing the best by their lights that they could, it just wasn't working.
It was a case where the theory was just wrong.
Oh, one of the engines that moved theory away from this, by the way, were parent advocacy groups.
You've got to imagine that having a seriously ill child, whether that's a physical illness or a mental illness, is very traumatic for parents, period.
But imagine how you would feel if the leading theory was that this was your fault, that you had done this to your child.
Not intentionally, of course, but it was you who had done it.
Parents simply didn't believe it.
And as science started to emerge suggesting that in schizophrenia's case it was much more like a brain disease than like the product of bad mothering, that groups, funding groups -- you know, the guys who send you lots of mail at this time of year saying send us your check and we'll support good research on this disease that disease or whatever, a number of these groups powered by parents of schizophrenics were helping to move the research away from the notion of a schizophrenogenic mother.
But what really did it was it just didn't work.
Nobody could make this story work.
The psychiatric hospitals are no longer filled with schizophrenic patients, and that's not because of a great breakthrough in Freudian psychotherapy, it's because of drugs.
In the '50s drugs like Thorazine, Chlorpromazine is the technical name, came on the market.
In this particular case these are drugs that block dopamine receptors -- dopamine's a neuro transmitter, the receptor on the other side of the synapse.
Thorazine decreased the uptake of dopamine, and that served to ameliorate the florid symptoms of schizophrenia, in particular.
You put somebody on Thorazine and they didn't hallucinate anymore, and their paranoid dilusions slipped away.
Oh, let me tell you another anecdote about schizophrenics.
Schizophrenics I have known.
We had a very interesting woman who was the librarian in the department here some years ago -- PhD Classicist and also, she was schizophrenic and successfully medicated and functioned well as the psych department librarian when on medication.
Anytime I tell you the root of something like schizophrenia, the Latin and Greek roots of these words, it's because I'm remembering what she taught me.
But she would slip off of her medication.
You could tell when she was slipping off her medication because her handwriting, her handwritten notes would start getting smaller and smaller and longer and longer.
She developed this micro-writing and very extensive writing, and then the content would start to slip, too.
And she would also start -- if it got dim in the library you knew you were in for trouble because she became photophobic.
One of the interesting problems here and one of the problems in treatment was that as she fell off her medicine she became paranoid.
What was the nature of her paranoia?
The nature of her paranoia was that people wanted to control her brain using chemicals.
Well that was exactly true.
But if we would tell her, you gotta start taking your medicine, that was just evidence for her that we were trying to control her brain, which was true.
So it wasn't even really -- it's very tricky stuff.
I mean it's funny and sad at the same time.
Eventually, she was no longer able to control this and was no longer able to hold the job, which was sad.
But it's also a real issue, a real medical ethics issue.
When the hospitals were emptied by these drugs, the promise was that supports would be placed in the community that would it make it possible for medicated schizophrenics to function.
It's not a cure because the medicines are imperfect.
For instance, it's not the case that dopamine -- the problem in schizophrenia isn't that dopamine is bad stuff and if you have lots of it you're in trouble.
It is clear that in schizophrenia, bits of the brain are awash in too much dopamine, but if you wipe out -- if you just blanket, push down dopamine, other bits of the brain which really like dopamine are starved for it.
The sorts of symptoms that you get when dopamine is removed from these chunks of the brain are the symptoms of Parkinson's disease -- muscle tremors, an inability to initiate voluntary activity.
So a patient overmedicated on Thorazine isn't hallucinating anymore, but is also sort of inert.
So it's not trivial to just say take this pill, great you're cured.
It was clear that management was going to be needed.
It was clear in many cases that people were going to need housing that was like sort of halfway housing, not complete independence, but you could be out in the world and you wouldn't need to be hospitalized anymore but you'd need some sort of continuing care.
The problem is that we've systematically not given enough, basically money and resources to that.
The result is there are numbers of imperfectly -- perfect medication isn't out there yet -- but inadequately medicated, unsuccessfully medicated patients who ought to be cared for, but who are not being cared for for by the system.
Schizophrenics are over-represented in the homeless population because if you are not successfully medicated it's hard to hold a job, it's hard to keep an apartment or something like that and you end up on the street.
That loops back to this issue about your paranoid delusion about somebody trying to control your mind.
When is it OK then, now that you're on the street, now that you're floridly schizophrenic again, when is it OK for us as a society to come in and say guess what, we don't care that you think we're trying to control your mind, we're going to medicate you whether you want to or not because it's for your own good.
You can argue yeah, it's for your own good, but there's lots of things that -- you know, I might decide it's for your own good that you should sleep eight hours a night.
Do I get to enforce that?
No, I don't.
If I decide that it's good for you to take a drug that is going to, in fact, control your mind, when do I get to do that, and this ends up in court regularly, and it ends up on talk radio all the time, right, because it's a sort of thing that yakking head talk radio loves to get excited about.
You know some judge declares that some clearly schizophrenic individual has a constitutional right to live under a bridge yelling obscenities at you, and talk radio goes nuts.
It's an interesting question.
It's not a question with an easy answer.
When do you get to decide that you're allowed to take control over somebody else's mental life?
At the present, the view of schizophrenia is, as I say, much more that it's a distinct state of brain disease of some sort.
What might cause it?
Well, the fact that a pill that reduces the effects of dopamine does something tends to suggest to people that there's a biological cause there.
In the case of schizophrenia that well maybe true, but it's important to note that that doesn't need to be true.
Things that are cured or symptoms that are cured by pills aren't necessarily symptoms that are caused by bad genes or bacteria or some biological story like that.
A perfectly nice example is motion sickness.
You go on an airplane, you get bounced around, you throw up.
What are you going to do about this?
Well, you can take scopolamine or meclizine that's Bonine -- Bonine is meclizine and dramamine is scopolamine -- two perfectly nice drugs that affect your vestibular system, and they will keep you from getting motion sick.
That's not because you've got bad motion sickness genes or you got the motion sickness bacteria.
It's a biological treatment, if you like, for an environmentally caused disorder.
I point that out only to make the point that what cures or treats a disorder doesn't necessarily tell you what the cause is.
There's not one-to-one mapping there -- that's important to keep in mind.
But the drugs certainly do help with the symptoms.
What psychopharmacologists work on a lot now is getting drugs that work better.
All right, we want to reduce dopamine here but not here.
Well it turns out that there's a bunch of different dopamine receptors -- let's make a drug that only acts at this location and not at this location.
That's a sort of project that psychopharmacology works on at the present time.
But I want to go back and talk about the possible causes of mental illness, but I'll do that after we take a brief break here.
OK. What I want to do with the remaining portion, probably all of the remaining portion, of today's lecture is to make two points, which I have already alluded to.
One of them is that there are multiple roots into mental illness, not just any sort of single root, and you're going to need to work this out on a case-by-case basis.
The other related point is that the border between normal and abnormal is not necessarily a sharp clearly marked one.
Oh, by the way, if you're interested in this latter point, there's an article in today's New York Times science section about really exactly this point in the context of eating disorders -- a topic that I hope to get to before the end of the course -- where it's very important to be able to give a label to psychiatric conditions.
Why is that important?
In part it's important because third party payers will pay if you have a disorder, but they won't pay if you're just feeling a little screwed up, right.
So you gotta have a labeled disorder.
So, there's been a big movement within the American Psychiatric establishment to come up with clear diagnostic criteria.
The nice thing about, you know you go into your pediatrician and getting a throat culture for strep is that I can grow it in the lab, and if you got strep you got strep, and I'd give you something for strep.
Mental illness is almost never like that.
You got some collection of symptoms, are they adequate to get you the definition of this disease?
The Times article is pointing out that therapists are having a real problems with people who any idiot can see has, let's say, an eating disorder, but the official criteria for anorexia nervosa, the eating disorder of self-starvation, are you have to have boom, boom, boom, boom and boom.
Well, if you're missing that boom, you can't make the diagnosis.
What do you call them now?
Anyway, so today's New York Times science section, go check it out.
In the meantime, let us return to 1900 to talk a bit about possible causes of mental illness.
In 1900 half the patients in American mental hospitals had a disorder known as dementia paralytica.
This is the bad stuff.
It's a disorder that starts with mania and grandiosity as its symptoms.
If you think that you are Jesus and that you can turn water into wine and stuff like that, except in the rare case where that turns out to be true or something, it's a psychiatric symptom and it would be a manic symptom of some sort.
If you think you're the reincarnation of Napoleon and this time you're going to get the invasion of Russia right or something like that, you know, that's the sort of thing that gets you into psychiatric care.
In the case of dimentia paralytica, it then proceeded to the other two bits of the name.
Dementia, people became demented, they then became paralyzed, lost control their muscles and then they died.
This is not a good disorder.
It has disappeared essentially.
Why?
Does anybody happen to know the answer?
Yes.
Which person?
What are you looking behind you for?
How many people are wearing witchy hats?
How long have you thought you were a witch?
Anyway, you were saying
This is caused by bacteria I think, or something that they like [INAUDIBLE]
Well, all right.
The problem is people know too much around here.
You can fish around for all sorts of interesting answers, like one of the possible reasons for a diagnosis to disappear is you fractionate the disorder into a bunch of other stuff and we've now renamed it and this is now called Alzheimers and a few other things.
But no, in fact, this is an outside pathogen.
In this case, dementia paralytica was the result of tertiary third stage syphilis.
The reason you don't see it anymore is that nobody gets to third stage syphilis anymore.
If you show up with symptoms of syphilis somebody treats you.
This was the result, by the time the organism had gotten to your brain and chewed up your brain.
So it disappeared with treatment.
But it points out is that one of the roots to mental illness is an assault, a biological assault from the outside.
So there are theories, unproven theories that a rise in autism, that's been seen in this country over the last couple of generations, is due to some sort of external insult to the nervous system, not clear what it is, but there are proposals out there that something that we're doing to the environment is causing more autism, for example.
But this is an example of a clearly biologically rooted mental illness.
Perhaps the extreme other end of the scale would be -- well, it's more like a triangle.
We could have biology, we could have an environmental disorder, and we could have what you might consider to be a societally defined disorder.
The distinction here is this is a case where the environment somehow really does make you insane, and this is a case where we take the normal distribution and declare the upper tail of the distribution to be a psychopathology of some sort.
So let's look for examples of both of these.
A perfectly nice example -- not nice -- an example of an environmental disorder would be what was called shell-shock in World War I and would now probably be more generally lumped under the category of traumatic stress disorder.
Exposure to extreme stress and being shot at can produce a range of disabling psychiatric symptoms.
So, in World War I when people were fighting this insane warfare of having vast numbers of people in entrenched positions shelling each other and just charging at each other, some fraction of the population of soldiers cracked up.
They just couldn't function.
They would cry hysterically, they would just lie there and not do anything.
The first reaction of the military to such things is to decide that this is insubordination and to shoot you or something of that sort for failing to obey orders.
But it became clear to medical folks during World War I that this was not a voluntary act, this was a psychiatric problem.
They went and pulled people back behind the lines, worked very hard to treat them so that they could return them to the war in a sort of a sick irony.
But this is a disorder that is clearly environmental.
Now there's probably some -- not everybody who's placed under stress produces a post traumatic stress disorder, so it's clear that there's some underlying variation of unknown nature that makes some people vulnerable to it and some people not.
But the precipitating cause here is environmental.
Now this is something that's a topic of a lot of current interest in psychiatry and in clinical psychology.
Because as it's been appreciated that this sort of trauma does produce psychiatric symptoms, efforts are made to prevent this when it's clear it's going to happen.
So, first responders, the police and firemen who are going to be the first guys that crash scenes or into burning buildings, are going to see really terrible stuff, and it's known that that's going to produce a collection of psychiatric symptoms.
So, an interesting question is well if you know that, what could you do perhaps to ameliorate the situation?
One very popular idea has been that after a crisis -- I mean you hear this all the time when something really bad happens like a school shooting or something, one of the things you do is you immediately pour in a bunch of therapists, and the idea is is you get people together to talk about it.
One particular modality of therapy is you basically get all your firefighters who were at this scene into a room together to talk it through, and that this has been proposed as a mechanism for preventing these stress disorders.
Unfortunately, a big study came out last year suggesting sounds lovely, doesn't work.
Not that it does anything particularly bad or anything, it just doesn't work.
What it points out is that the state of the art is a little like the problem I was talking about with Freudian attempts to explain schizophrenia.
These are difficult problems where good well-meaning clinicians try stuff out that doesn't necessarily work.
Very hard to get a clear feeling from this, from reading the popular press where you get one miracle cure after the other and then the miracle cure fades away, but it fades away in the literature somewhere -- you don't get to see this very clearly if you just sit there reading the science section of Time magazine or something of that sort.
Anyway, so third stage syphilis munches up your mind, trench warfare munches up your mind.
Are there diseases that are defined purely by society and what does that mean?
Are those legitimate?
Well, let's talk about one that's not terribly legitimate.
I don't think anybody here is deeply familiar with the disorder called drapetomania.
Any drapetomania experts?
No.
OK. Any Greek scholars?
No.
So we're not going to get anybody to tell us what the root is.
So, it's not a manic desire to put up drapes.
It's drapeto comes from a Greek root having to do with wandering off, and the mania's the mania part.
So it's an insane desire to wander away.
All right, that sounds like it could be insane.
This is a disorder discovered by a doctor named Cartwright in Louisiana in the 1840s.
He writes about it in a book endearingly titled, Peculiarities of the Negro Race.
So you need to think -- all right, time to review your AP American history.
Right, 1840s, Louisiana.
If you are a member of the negro race you are what?
A slave.
If you are a slave you might do what?
Well you might wander off or run away.
It wasn't that Cartwright discovered this.
People knew that slaves ran away.
But rather like earlier attitudes towards people cracking up on the battlefield, the standard attitude towards the slave that ran away was that's a bad slave.
Right, he's just bad, and what we should do is catch him and beat him and put him back to work.
Cartwright was an odd sort of liberal from the sound of it in the sense that he said, no, no, no, no, no -- they're not bad, they're sick.
Because after all, as he understood it, the proper place for a slave for an African American was to be a slave, right?
So if you wandered off, you weren't being a healthy, normal slave.
You'd gone up here somewhere and you were being abnormal.
He also, by the way, had a lovely biological theory about the cause of this, which was decarbonized blood.
I don't know, it sounds like it would do something bad to me.
And he had a treatment regimen.
Remember that the standard treatment for a slave who ran away was catch him, beat him and put him back to work.
Cartwright said, when you catch him, what you need to do is slap some oil in with a broad leather strap and put him to work.
All right, so the treatment didn't sound all that different.
In any case, it's a pretty clear case of a disorder defined entirely by society.
There's not a -- I don't know, decarbonized blood not withstanding.
There's not an organism there.
There's not -- well, you can think of the environment as causing the slave to run away, but not in the sense of causing a disorder.
We would consider this to be essentially a normal -- we'd be with Thomas Szasz on that, that's a sane reaction to an insane world, perhaps, to be a slave and to run away.
Now, you might argue that we don't do that anymore, but if you were to, say, OK, what this is, is weight.
Let's make this normal distribution weight.
The Surgeon General is perfectly happy and all sorts of outfits are perfectly happy to declare that if you're above this level whatever it is, you're abnormal, you're overweight.
And there are plenty of theories, folks psychological and otherwise, that basically give a psychiatric spin to what's going on there, right.
If you've heard theories of obesity being -- there are pop theories that oh, they're overcompensating -- that they're eating to substitute for the lack of love in their life, or it's a lack of self-control, or it's because they have gene for fatness.
All of this might be true at some level.
I'm not casting any doubt on the particular theory, that's a separate topic.
But you gotta realize that what you're doing is taking an essentially normal distribution with the exception of some very few -- there are a small number of clear pathologies of weight regulation.
Mostly what you're doing is taking one tail of the distribution, declaring it to be abnormal, and putting an explanation on it.
How you understand that is culturally dependent.
How we talk about obesity is in part defined by how we talk about beauty.
In cultures where obesity is considered beautiful, you don't get theories that say it's an abnormality, right?
So in some fashion we do this all the time.
In the remaining time, what I want to do, and then I'll loop back to this point, is look on your handout on page three.
Let's do a little bit of armchair diagnosis here.
We don't have much time.
Quickly read these three vignettes and quickly decide where they lie on the scale from fine to very weird.
I'm deliberately using very casual language there in order to suggest that you not think deeply about whether or not you're giving them a psychiatric diagnosis here, I just want to know whether they're nuts.
We'll collect a little data.
How are we going to collect this data?
Don't cheat off your neighbors, please, and [?
wavel ?]
in when you're done.
How many people are done?
Most people are done.
OK, good enough.
In the interest of getting some data relatively quickly, let's group between -- let's do aluminum hat boy, Jim.
How many people gave him a sort of 1-3, he's fine?
4-7 he's a little weird.
I take it we got a lot of population-- All right.
So Jim's here.
How about Jack there at the bottom -- we'll come back to Sam.
How many people thought Jack was in the 1-3 he's fine category?
A bunch of people.
How many people thought he's in the 4-7 category?
A bunch more people.
Let's give him some hash marks here so we can tell the difference.
So this is -- what is his name?
Jack.
How many people thought he was very weird?
So only a very few.
So, you can, in fact -- so Jim is sort of the guy who came to my lab all those years ago, and Jack can be found in Harvard Square any time selling some far left newspaper and yelling at you about the fact that we're all controlled by something, and we just consider them to be sort of weird but kind of -- this is in the realm of free speech.
All right, let's take a look at Sam.
How many people thought Sam's just fine?
OK. How many thought Sam was just in the middle?
OK. How many thought that Sam was pretty weird?
All right, so that looks-- I need some space here.
Well actually, lots of fines, a few less in the middles, and almost as many weird.
Pretty wide distribution, right.
The other two are very heavily skewed.
Could somebody in the fine department explain why he's fine?
Yes, find person
I think he's weird, I just don't think he's disabled
I need somebody who thinks he's fine, fine.
Yes.
Yes, you pink person
He's just a stamp collector -- he sounds fine
A stamp collector.
All right.
The trick here is there is a two-word variation between what half of you have and what the other half of you have.
Let's see if that makes a difference.
The people who don't have the stamp collector one know the difference, which is that one half of you have his major passion is collecting stamps, and the other have his major passion is collecting women's undergarments.
Right.
That is the only difference here.
Yes?
[INAUDIBLE]
You know, actually that's lovely because I could probably expand the description.
If I said he used to write all of his relatives and asked them to send him used stamps, nobody would think that was weird.
If I said he used to write to all his relatives and asked them to send used underwear, that's really weird, right?
If I changed it a little more to say -- what is his name?
Sam was a shy, withdrawn man, the president of the Haines underwear company and at the corporate headquarters he had the big underwear exhibit or something, then he becomes less weird.
Given the time, let me just reiterate the point that what this is intended to illustrate is that the border between what makes you weird and what makes you just a little different is not clear.
It's not something that we can define nice and sharply.
See you Thursday.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.mit.edu
Good afternoon.
When we parted last time, I was -- you just did that demo with Sam, who was either collecting stamps or lady's undergarments, depending.
And the point there was to describe the fact that any behavior is going to be distributed across the population in some fashion.
It may well be that the extreme of the population is abnormal in some meaningful sense, maybe even pathological in some sense, but that the border between what is normal and what is abnormal is going to be necessarily ambiguous.
There's not going to be some nice, neat line that can be drawn in all cases.
I was arguing that something like schizophrenia is a bit more like that, where the abnormal state really does look quite different from the normal state and less like that it's on a continuum.
You will notice on your handout that it says Lecture 20.
And those of you who are particularly observant will say, last one was called Lecture 20.
And that's true.
I just liked 20.
No, this is really 21.
And with a bit of luck, I'll label the next one 22.
Otherwise it'll just turn out that there's 20A and 20B or something like that.
Anyway, it really is the right handout.
You'll discover that it is closely related to the later pages of the last handout that we didn't get to last time.
Yeah, it's kind of the same is what people are noting.
But I didn't have any faith that people would remember to bring the real Lecture 20 handout.
Anyway, what I want to talk about, at least initially here, is to talk a bit about how you would end up in this abnormal bin of schizophrenia.
And one reasonable question to ask is, well, it's on the handout.
It says, is there evidence for a genetic component?
And the answer is yes.
And the way you get one of the important sources of that information is similar to the data that we were talking about in intelligence testing.
Take a look at people who vary in the amount that they are genetically related to each other.
Notably here, MZ stands for monozygotic or identical single-egg twins.
And DZ stands for dizygotic two-egg twins who are no more similar than siblings.
Concordance means what the chance is of Twin Two having the disease if Twin One has it.
And so what you can see is that for monozygotic twins, if one of a pair of identical twins becomes schizophrenic, there's a 50% chance that the other one will, too.
That tells you two things.
Well, it only tells you two things if you look at the dizygotic.
The dizygotic twins have a much lower concordance rate of about 15%.
So that tells you first of all that there's a probable genetic component there, and second of all, that it's not absolute.
If you have the genetic mutation that makes you color-blind or color anomalous, you'll be color anomalous.
There's no big mystery at this point about that.
If you have whatever the genetic predisposition is for schizophrenia, something else has to happen before you would end up showing the disorder.
Now, these twin studies run into the same problem that the intelligence testing twin studies did, which is that monozygotic twins are simply more similar on all sorts of things.
You know, they're treated more similarly and so on.
One way to look at whether or not there is a, you know, to try parceling this out, is this third line here.
This was a population of dizygotic twins who thought they were identical.
You know, they looked alike.
Presumably they're the same sex and stuff like that.
Thought they were identical.
You'll see that their concordance rate was less than the 50% that the true monozygotic twins had.
So there does seem to be something to being really genetically identical.
You can also do twins reared apart things.
But look, if you have a set of -- twins reared apart -- not common.
But all those twins have IQs.
You can always measure their IQs.
If you take twins reared apart and then take the cases where one of them has schizophrenia and ask what the concordance rate is, the answer is it's quite elevated.
But, you know, it's a really small population of people.
You know, twin reared apart and schizophrenic, it's not a big crowd.
Now, when you go look for the genetic marker, since the late '80s, there have been a slew of papers on genetic markers for schizophrenia.
The way you look for this -- one of the typical ways to look for this is that schizophrenia runs in families.
And if you go and analyze the genetics of families, you can find specific loci that seem to be abnormal in the schizophrenic members of the family.
The difficulty or the interesting aspect of this genetic story is that the story that works for this family doesn't necessarily work for this family.
In the early days of this, there'd be some big article in Science or Nature or something saying gene locus on chromosome eight for schizophrenia or whatever.
I'm making it up.
And then, you know, either in the same issue or in the, you know, next week, there'd be another thing saying, but not in this family.
Clearly schizophrenic but not clearly having the same defect.
It suggests that there are multiple roots, genetic predispositions, to schizophrenia.
It's also possible that you need more than one genetic insult.
That's jargon -- you don't say, you got an ugly gene.
You need to have perhaps more than one defect in order to have the susceptibility to schizophrenia.
One of the curious little wrinkles that I dug up in my reading on this -- and I've only read this in a secondary source.
I have been unsuccessful in tracking down the primary source.
But I'll tell you anyway -- is a study -- it turns out that there are two types of identical twins.
There are, sort of, plain vanilla identical twins.
You know, Twin One is right-handed.
Twin Two is right-handed and so on.
And then there are so-called mirror twins.
They're genetically identical, but the phenotype, the actual organism, is mirrored rather than identical or something like that.
So if one's right-handed, the other is left-handed.
You know, swirl on the top of your head, if it swirls clockwise for one twin, it swirls the other way for the other twin and so on.
And the curious finding reported in this one relatively small study, I think, is that the concordance rates for identical identical twins, the right hand, right hand identical twins, very high in this study, like over 90%.
If one twin was schizophrenic, the other was schizophrenic.
The mirrored twins, the left, right ones, were concordance rate down at about 25%.
Both of these are genetically identical pairs of twins.
It's abundantly unclear what it is that makes these two groups different, if indeed that turns out to be a reliable result.
I went hunting in the databases, and I think I found the guy who did it.
So I sent him an email, but he didn't get back to me in time.
Oh, that reminds me.
It's a little late in the day to tell you this, but if you discover that something that you're looking for in primary source land, you know, you're looking for the Journal of Hoozy Phoozy Results or whatever, and oh my god, MIT doesn't carry it so you don't have access to it, it's often very useful to go and see if you can find the website for the lab where the work was done.
Because many, many of us post all of our publications, or as many as we have PDFs for, on the website to be downloaded.
And so you can dig things up.
You can dig things up that way.
So anyway, there's pretty strong evidence for a genetic component.
And then there's pretty strong evidence that that by itself is not enough.
Unlike a variety of other neurological disorders, there are plenty of neurological disorders where if you've got this problem with your genome, it's going to munch up your brain in the following fashion, you're going to have this problem.
And fortunately most of these are very rare.
But they're straightforward genetic stories.
In schizophrenia, something about the environment seems to be important to trip it off.
The leading notion is stress.
So, actually, does anybody remember?
The book used to go on at vast lengths about the stress diathesis model of schizophrenia.
Does this sound familiar to anybody?
Has anybody looked at the relevant parts of the book yet?
Nobody knows.
All right, well anyway, one of the leading notions about what it is that actually produces the florid disease, is that you need to have some genetic predisposition and then you need something like stress to drop-kick you into pathology.
Now, this is an interesting -- this raises interesting problems for public policy.
I think we talked last time about, you know, when is it OK to force somebody to be medicated, right?
This is a similar sort of problem.
Suppose you are applying to MIT this year.
Should I be able to -- should I go and suggest to the head of admissions that we really should get little tubes of blood from everybody?
Because if we got little tubes blood from everybody, we could genetically screen them.
And we know where -- we don't know exactly which loci are critical, but we know a bunch of loci that seem to be related to family histories of schizophrenia.
And anyway, there's all sorts of other interesting genetic markers out there at this point.
We could screen the incoming population or the population of applicants and systematically decide not to admit people who had the genetic marker for schizophrenia or for some other -- well, schizophrenia is a good example.
Why in the world would we do this?
Well, is there anybody currently present who believes that MIT is a at least modestly stressful environment?
All right.
Short of Guantanamo Bay or something, you know, it's right up there.
All right.
The state of the art is that we know that stress is a contributing factor to a variety of mental illnesses, notably in this example schizophrenia.
If we admit people to an environment that we know is highly stressful, we are asking for a certain number of psychiatric problems.
Isn't it our responsibility to prevent that from happening?
Well, now, typically the -- I won't take a poll, but the typical answer is, no, that doesn't sound right.
You know, that somehow that sounds a little too paternalistic, too invasive for me to be screening your genes.
And, you know, who knows?
Down the line, we take a look at this hunk of the genome and we say, you know, sorry, you're just never passing 18.03.
You know, what would we ever do with information like that?
But the disease case, it doesn't feel right typically to people.
But on the other hand, we spend a lot of time worrying, believe it or not, about your mental health.
The dean's office has substantial branches devoted to mental health issues as does the medical department.
We know that university environments kick up a certain amount of psychiatric problems.
And we know that that can be catastrophically bad for the person and for people around them.
Wouldn't it be our responsibility to do something about that?
Well, look, I'm not going to give you an answer to that, and I promise not to ask on the exam, you know, should we get blood samples from everybody and, you know, the choices are yes, no, all of the above or something like that.
I raise that as an issue to think about.
When you take a look at the brains of schizophrenics, by the way, they don't look like the brains of healthy, normal individuals.
So you get things like shrinkage of grey matter.
You get enlargement of the ventricles.
The ventricles are those nice, empty fluid-filled spaces in your brain.
Great answer for a -- if you're looking at the multiple-choice question on the final, and you're thinking, I'm going to go for, you know, the third ventricle as the answer, you're probably getting suckered here, right?
Because the ventricles are empty fluid-filled spaces.
It's not where any of the more important mental functions are thought to be localized, at least not since the Middle Ages, where people did think that memories were stored in the ventricles, because after all they looked like store rooms.
Anyway, the ventricles get bigger.
That's presumably not a good thing because it's reflecting loss of brain tissue.
What's not absolutely clear is whether or not abnormalities in brain structure precede disease.
You know, are they there somehow from birth?
Is part of what the gene is doing somehow changing the structure of the brain in such a way that it makes it vulnerable, or is it that having the disease itself is bad for the brain?
In any case, the point to take away here is there are lots of different routes into the kingdom of schizophrenia.
It's not a nice, clean simple diagnosis where you say, you get this bacterium, you get this disease, we give you this antibiotic, and you're OK.
Multiple routes in and treatment options are, you know, getting better but remain difficult.
Let's see here.
Oh yes.
What I think I will do is switch gears and talk a bit about depression.
But in order to talk about depression, what I want you to do is to decide how you want to treat my three invented patients on the handout.
The third one is on the back of the page.
So play with that for a second.
Kristen, can you do me a favor?
Go find me a bottle of water.
I'm going to expire today otherwise
I have some.
Do you want some?
I don't know.
Do you got any interesting diseases?
No
I don't want to steal yours.
Find me something somewhere.
Thank you.
I'm recovering from a cold still, and I can feel that I'm going to run out of moisture.
And my tongue is going to get all big and like a parrot or something.
I won't be able to talk.
All right, have you managed to do this?
Still working on it.
All right.
Don't cheat off your neighbors.
But I promise you this time, none of these guys are dealing in lady's undergarments.
Here.
Let's collect some.
All right.
Let's collect us some data.
OK. Everybody pretty much done here?
Make assenting noises or something.
OK, that's good.
That sounded very assenting.
OK, so Sarah had a bad day.
How many people would, let's see, so that -- slow numbers are the "no way, we're not going to medicate this depression."
How many people -- oh, that was quick.
That was good.
Thank you.
How many people are going to give her no meds?
Oh, OK.
So the answer is everybody, more or less.
Don't worry if you said, I'll give her a little bit of med.
That's OK, too.
But I won't bother asking about it.
These really are all the same, by the way.
There's no fifty-fifty split.
This is just straight up, normal.
I'm not tricking you this time.
OK. And by contrast if we look at Sam, Sam's depressed, no particular reason, just happens.
And he oscillates between manic and depressed states.
How many people think no treatment or unlikely to medicate?
How many moderately like, you know, somewhere in the four to seven department?
Well, a few people in there.
And you have all embodied the ethos of the yeah, yeah, OK, good.
I mean, I don't know if it's good or not, but it's clearly where medical science, I suppose, is these days, which is that he would be a candidate for an antidepressant in a way that she wouldn't.
Why not?
Why wouldn't you give her an antidepressant?
She's depressed
It's normal for her to be depressed.
Like, anyone could be depressed in that situation, but if you're depressed for no reason, then that [UNINTELLIGIBLE] problem in the brain that could be solved by medication, whereas with her thing, all that needs to happen is the problem's [?
cured.
?]
There's a couple of things that are worth picking up on here.
One is, I mentioned last time that the exact mapping between cause and treatment shouldn't be seen as one to one.
So even if we take as correct for present purposes that Sam's got something biochemical wrong with him, it wouldn't necessarily be the case that the only way to deal with this would be by a biochemical kind of treatment.
There isn't that necessary one-to-one mapping.
But it's not an unreasonable thought.
And the other part of that thought was, you know, she's depressed for a reason.
And we don't want to shield her necessarily from that depression.
Though this gets raised as an issue too now.
I talked last time about treatments for post-traumatic stress disorder.
One of the notions is that you could give people medications that would in a sense prevent them from feeling as bad about something horrible as they normally would.
I mean, this is a real something horrible now, you understand.
But we're going to get you -- something horrible has happened -- we're going to give you a pill that is going to decrease, we hope, the chance of you having a post-traumatic stress disorder, but it's going to do that by blunting your emotional response and your memory for that event.
Is that something that we want to get into?
Now, Sam I described as having this -- if this is now, oh, I suppose, mood -- Sam I described as having one of these oscillatory or bipolar conditions where he bounces back and forth between -- well, let's see, which end is this?
This can be the manic end and the depressed end.
You sometimes think that it might kind of be worth it, right, because you have to put up with the depression, but the manic energy thing would be good.
Well, the manic energy thing is kind of good here, maybe.
You know, you get all your problem sets done in ten minutes or something.
But by the time you get out here, you are just as abnormal, if you like, or just as crazy if we want to be colloquial about this, out here as you are down here.
And you can be pretty hard on your friends and neighbors and yourself, in fact.
I don't need -- I mean, this is very MIT, right -- I don't need to sleep anymore.
I can talk all night.
You don't want to listen to me?
Well, I'll talk anyway.
I remember reading a memoir by the child of a manic depressive mother, who as I recall comes home -- the child comes home from school one day, discovers mom has decided we're going to have a great old-fashioned Christmas this year.
So she went out and bought like thirty-five Christmas trees.
And you walk into the living room and it's just, you know, forest in there.
She's stuffed the Christmas trees all over the -- and, you know, this is, I'm sorry, I'm glad you're feeling really up, but this is nuts too.
It's not working.
It's an interesting observation.
There are interesting data suggesting that this sort of bipolar disorder, this oscillation between deep depression and this wild mania, that that's overrepresented in at least some creative populations.
The notable one is poets.
It's notable because a woman named -- what is her full name?
-- well, anyway, Jamison is her last name.
I think it's spelled like this.
Anyway, she wrote a book called Touched by Fire where what she did was she went and looked at the biographies of, in particular, sort of the run of famous English poets.
And over and over in their biographies you get these descriptions that -- you know, they didn't have the diagnosis in 18th century England, but it sure sounds like a manic depressive disorder.
You know, Shelley or Keats or somebody writes eight million poems and then lies in bed for a month.
It's that sort of thing.
Now, great, you know.
I can be a famous poet.
All I have to do is put up with this.
The problem -- well, there are a number of problems with this.
First of all, you become a little intolerable to your immediate surroundings.
The other thing is that a very serious consequence of deep depression is the threat of suicide.
Interestingly, you would think that this was a continuum thing, right, that you go way, way down here and if you get this far down into depression, that's when people commit suicide.
Turns out not to be the case.
When people describe their really deep -- there's a huge genre here.
It's a very popular genre at the moment.
If you're into depression, your, you know, Christmas reading list can be all set out because there's all sorts of people who are happy to describe -- happy to describe their depression?
-- anyway, more than willing to describe their depression in volumes thin or thick.
Jamison actually has a volume of her -- she got interested in this because she was a manic depressive and trained in psychiatry.
So she's coming at it from every which way.
Anyway, she has a memoir -- I don't remember the title of it -- about her own depression that's actually nice and slim.
It's not a bad read.
But what the descriptions sound like is that when you're really, really deeply depressed, you just don't do anything.
You might in some sense want to be dead, but that's just too much effort to do anything about that.
The danger for suicide in bipolar disorders turns out to be -- well, let's get rid of this -- turns out to be when you're on your way back out.
Still depressed, but now you're activated enough that you can do stuff, but you're not convinced you're ever really going to get better.
I mean, look, most of us have not been really clinically depressed, but many of us can tap into this, into this aspect in the sense of have you had the flu or some, you know, relatively disgusting disorder of some variety where you're sitting there thinking, "I know rationally that I'm going to get better, but am I really going to get better, or am I just going to be barfing forever?"
So you're coming on your way out, and that's where the suicide risk is high.
Now, that phenomenon may be related to something that hit the news a lot this year, which was concerns about giving antidepressants to children and adolescents.
Because there are data out there that say that the suicide risk in depressed adolescents taking antidepressants is elevated.
Exactly what you don't want to have happen, right?
What you really want to not have happen is people committing suicide when they're depressed.
What is the cause of this?
Well, we don't actually know, I don't think.
At least I haven't seen anything definitive.
But there are a number -- one theory is that antidepressants work differently in young brains than in older brains.
That's one possibility.
But the other possibility is that what the antidepressant did was it moved the patient along to a point where they weren't so paralyzed by the depression that suicide became an option.
It sounds weird and disturbing, but that is one possibility for where this elevated rate of suicide in antidepressant treated young patients came from.
Let's see, what else do I want to say about Sam and Emma here?
Oh, Emma's the one on the back.
All right, so how about Emma?
We need to collect some data on Emma.
Did I describe Emma as dysphoric?
Yes.
Emma is a dysphoric.
That's the opposite of euphoric.
So it's not depressed.
It would be sort of lying around here somewhere.
Just kind of flat.
All right.
So how many people voted that we'll leave her be?
OK. How many people voted maybe?
And how many people voted sure?
Well, OK, we got a normal curve out of that.
Well, I could have gone -- it looked sort of like this to me.
So people are sort of unsure about -- oops.
Well, I guess it's going over there now.
People are sort of unsure about it.
So what are the issues here?
Why wouldn't you give it to her?
Yeah?
If she doesn't want it
Oh, yes, yes.
Well, that's a separate issue.
Did I say anything about it?
No, no, no, this is just -- well, actually, we'll come back to that.
That's an interesting problem.
But yes, I was not suggesting that we jam it down her throat, which is by the way not the problem with Prozac.
Let's imagine that the antidepressant is Prozac here, or one of its newer friends.
Prozac gets a lot of play in the public sphere.
Whoops.
Well, I'll put a P there now.
Uh oh.
C or K?
C. Yes.
Thank you.
OK. Prozac is a serotonin -- that's a neurotransmitter -- selective -- thank you, where was that coming --
It's first
Oh, it's a selective serotonin, not serotonin selective.
It's SS.
All right.
All right.
It's a serotonin selective -- reverse -- uptake, reuptake -- can I do reuptake?
[INAUDIBLE]
Yeah.
Yeah.
OK. Reuptake inhibitor.
Or an SSRI, which makes it understandable how I would have thought it might have been an SSRI rather than an SSRI.
But let me explain what this is doing actually.
So remember that in schizophrenia, the first big psychopharmacological breakthrough was the discovery there's too much dopamine in bits of the brain.
If we block some dopamine receptors, we reduce the symptoms.
In the case of at least some forms of depression, the problem seems to be too little serotonin -- or a problem.
Seems to be too little serotonin in some parts of the brain.
And so you've got a synapse.
And serotonin is being released here and binding to the other side.
But now, in order to have this synapse be reusable, once you've released some serotonin, you've got to get rid of it.
You've got to get it out of the synapse so you can do it again and again and again.
One of the techniques is to take the molecules and suck them back into the originating neuron.
That's called reuptake.
Now, if you don't have enough serotonin here to do the job, one thing to do is to slow down this reuptake so that the serotonin hangs around here longer.
And that way you get more bang for your molecular buck.
And so that's what Prozac is doing.
The reason I put Emma on here is that Prozac seemed to do good things for dysphoric patients, patients who did not meet a clinical criterion for depression but who found -- I think in part because the drug manufacturer gave the docs piles of free Prozac.
You know, patient comes through and says, "I feel kind of down."
So, "Oh yeah.
Try this.
This is new."
You know, "All right.
Hey!
I feel better.
I feel a bunch better.
I'm not clinically depressed, but I really like this stuff.
And not only that, if I go off it, I go back to my dysphoric state.
I go back to -- " The example says something to the effect of, that's just the way she is.
If she goes off Prozac, there are lots of reports even in the popular literature of this, of people who said, you know, if I'm off Prozac, I'm just, ughh.
But if I'm on Prozac, I'm just a better me.
Well, do you give them the drug?
Do you give them the drug?
Do you allow them to continue taking the drug for the rest of their natural born days, particularly since we have no idea since it's a new drug -- well, now it's over a decade old.
But, you know, we don't know what happens if you take this for twenty, thirty, forty years.
Who should pay for it?
You know, if you think that this drug makes you feel better, that's lovely.
Should your insurance company pay for it on a continuing basis?
So those are the Emma issues here.
The issues about whether or not it's OK to make somebody better than they, in some sense, better than they were.
Are you the same person?
Is this, you know, is this the real Emma, or is this some medicated Emma?
And, well, all right, now let's look back to the comment about forcing them.
Well, you know what?
It works on kids, too.
And you know that there's a fairly substantial industry in feeding psychiatric drugs to kids for various and sundry problems.
Suppose, well, we know that shy -- how did I describe old Emma here?
Is she shy and withdrawn?
No, that was what's his name collecting the underwear, wasn't it?
Oh well.
I could have described Emma as shy and quiet.
We know that the kid who's shy and quiet in first grade doesn't get the same amount of attention that the bouncy, active kid gets.
And in fact, there's some evidence that if you feed something like Prozac to this kid, the kid will move from here in the -- well, we've got to change the axis again.
You know, I don't know, what's the axis here?
Bounciness?
Technical term.
You know.
So the kid's going to move from being unbouncy to, you know, Tigger is up here somewhere.
As are several -- couple of people out there halfway up the -- so there's Tigger, I guess.
But, you know, move them up here, let's say.
Get them above average and the teacher's going to pay more attention to them.
Is it OK to medicate your kid?
No
If it's -- all right, it's not OK to medicate your kid.
I'm not going to medicate my kid.
Uh oh.
The neighbors.
They're medicating their kids.
So what they've done -- my kid was right here.
He was average, right.
All right, now what the neighbors and the population as a whole has done is they moved the whole distribution over.
Because they're all handing out Prozac in the lunch box.
Now is it fair to my kid -- this is a hypothetical my kid.
This is not a real issue on my street in case you're wondering.
But is it fair to my kid to not medicate him?
He's going to fall behind.
Is this systematically different, you know, is this sort of morally or ethically different than the Kaplan or Princeton Review approach to SAT tests?
What Princeton Review and stuff does is it takes the population and it shoves it on -- everybody gets to gain those hundred points or whatever.
So is it OK for you to say, "My kid doesn't need to do that.
I didn't do that when I was a kid, and no kid of mine's going to go off and get tutored.
He'll do it on his own brains.
It's an aptitude test after all."
Right.
You know, is there a difference between everybody being signed up for Princeton Review versus everybody being signed up, you know, going down to the nurse's office for the daily Prozac dose?
Or is there a difference between just sitting there saying, "I'm not going to go and make myself into a better me by popping Prozac all the time.
I'll stick with --" What is that, Mountain Dew?
You know, twice the caffeine of whatever, right?
Late night last night
Yeah.
OK, well, that's good.
You're vertical.
That's encouraging.
Give some to the guy next to you there.
He's gone.
[LAUGHTER FROM AUDIENCE] I shouldn't have said that.
Anyway, we do this all the time, right?
We self-medicate.
We self-medicate, particularly with caffeine.
But perhaps with a variety of other things.
The use of alcohol as an antidepressant is probably a mistake, but is not uncommon.
So when is this a sort of an unwarranted medicalization of your life, and when is it, you know, a lifestyle choice?
And again, there's not going to be a question on the final that says, "It's OK to drink Mountain Dew: yes, no, only if it was a late night."
But you can see that there are issues to wrestle with.
And the doctors ended up with or have ended up with a serious problem, which is what do you do with these patients who you cannot in good conscience give a clinical diagnosis of clinical depression to, but who say, you know, if you don't give me my Prozac, I'm just going to be so depressed, who really need that or who think that they really need that.
Oh, by the way, OK, what's this?
Want to make sure that I do what it says.
Oh.
Well, OK. Let's say something about how antidepressants work.
Here's a mystery for you.
Take Prozac.
It blocks the reuptake pathway within hours.
It doesn't take much time at all.
OK.
So you are down here, clinically depressed.
You come into the doctor's office.
He says, here's your Prozac.
You take the Prozac.
A few hours later, more serotonin at all those synapses.
How do you feel?
Well, you feel depressed, actually.
It takes typically at least days, more like several weeks, before an antidepressant of this sort has an effect, beyond perhaps placebo effects.
There are placebo effects where you can take any old damn thing and you feel a little better if you're convinced that it's going to work.
But the main effect of antidepressants seems to take a long time.
That's a mystery.
At least as far as I know, there's no neat, clean answer to why it should be the case that the pharmacological effect is there instantly almost and the psychological effect takes a vastly longer time.
The reference that I put on the handout is a theory.
It's not proven.
But it's an interesting theory about what might be going on.
As I think I mentioned earlier in the course, one of the interesting new findings in neuroscience is that we grow new neurons.
We didn't used to think we grew new neurons.
But now it's pretty clear that you grow new neurons.
And there's a pretty strong suggestion that this neurogenesis, as it's called, is depressed in depression.
You're not making those new neurons.
And one theory is that what an antidepressant is doing is jump-starting the neurogenesis, that this is the beginning of a cascade of events that leads eventually to the birth of new cells, new neurons.
And only when you manage to sort of grow yourself a new bit of brain, in a sense, does it help you to break out of the depression.
The data on how well these things work is a little depressing in its own right.
Antidepressants, like a lot of these -- antidepressants don't work as well once they go off patent.
It's a mysterious thing.
You want to worry about this.
Over and over again you get some cool new drug like Prozac.
It's got no after effects.
It cures the disease like that.
And everybody is better, and it's a miracle drug.
Amazingly, by the time, you know, Eli Lilly or whoever's patent has worn off, it now has side effects, doesn't work all that well, but you want to try my new amazing wonder drug.
So the last few reviews I've read of the effectiveness of antidepressants have been, well, they've been sort of depressing, actually, that they're not magic bullets.
Now, there are several ways to treat depression.
And how you treat it depends -- there are also several flavors of depression.
Not all depressions -- well, certainly not all are bipolar.
And there's evidence that they differ both in their symptoms and in their neurochemistry.
So for instance, say, I think it says on the handout -- yeah, typical versus atypical depression.
I'm sure if I rummaged around I'd remember which was which.
But in some forms of depression, people sleep too much and eat too much.
In other forms of depression, they sleep too little and eat too little.
They're both depressions in terms of the mood, but they are clinically distinct and psychopharmacologically distinct.
Actually, bipolar disorders, the treatment of choice for long before Prozac came on the scene was of all things lithium chloride.
It's just the salt of lithium.
You know, just move right up from sodium chloride.
Jamison credits lithium with keeping her alive, that if she had not been put on lithium as a treatment for her bipolar disorder, that she thinks she probably would have ended up committing suicide.
Anybody an expert on how lithium works?
No.
I think I don't know the answer to this.
I'm not sure it's known.
The last time I checked on it, we didn't know how it worked.
It was -- how was it discovered?
It was discovered in some bizarro fashion.
You know, where do you find lithium?
Like a salt -- I'm remembering some odd story about, you know --
Batteries?
Fine.
Batteries?
That people -- that's --
7 Up
7 Up.
OK, if you dissolve your battery in 7 Up, I don't know what happens but I don't imagine it's good because the mercury and the rest of it probably isn't really good for you.
The other treatment for -- now, sadly not all depressions are amenable to pharmacology.
Oh, first of all, the big mistake, which I think the insurance people are growing out of, was the notion, OK, now that we've got these cool pills, we'll pay for the pills, but that's it.
So if it's going to take you several weeks for the medicine to work, it turns out to be very helpful if you're actually talking to somebody during this time, if there's some psychotherapy of some sort going on.
Because if you are down here in the depths of the depression and some genius hands you a pill and says, "Go away.
Don't bother me no more."
You know, if it's true that as you're slowly coming out of the depression, that's when the suicides take place, you know, this is a recipe for disaster.
And if it takes a long time to work -- you know, if you take a medicine and nothing happens for weeks, what do you do?
You stop taking the medicine.
And so what you really need typically as part of a sensible treatment for a depression is not only somebody who will give you the medicine, but somebody who will talk to you about it too.
But even so there are depressions that are not amenable -- or don't seem to respond well to any of the medicines that we have at the moment.
And under the heading of even more unlikely treatments that really turn out to work, more unlikely than feeding them lithium salts, is what's known as ECT for electroconvulsive therapy.
I talked about this a bit when we were talking about memory.
Remember this is little rats.
They step down.
You run electrical current through their heads.
You shock their little feet.
You run electrical current through their brains.
They don't remember that you shocked their feet, so they step down again, you know.
Sound familiar?
OK, good.
Well, this is a treatment for depression.
Not abundantly clear why it works, but in my circle of friends, I have a friend who actually went back to -- graduated college, went off, did a whole bunch of stuff, then went back to med school, then became desperately clinically depressed and had to be hospitalized.
And what got her through it was electroconvulsive therapy.
Not an entirely benign procedure.
It does have memory side effects.
But it used to be that nobody in an intro psych class thought this was a good idea at all because everybody had seen One Flew Over the Cuckoo's Nest.
How many people have seen that movie?
Oh, so still a fair number people have seen it.
That's a movie that does nothing good for the reputation of electroconvulsive therapy.
Electroconvulsive therapy these days is, you know, you give somebody a sedative.
They don't thrash around.
They don't hurt themselves.
It still has memory consequences, consolidation of memory consequences, but it really does seem to mysteriously break up depressions.
Now the latest wrinkle on this, you know, what boils down to smack-them-up-th e-side-of-the-head treatments of depression is the use of, very new, is the use of transcranial magnetic stimulation, which I think I also talked about earlier.
But basically what you do is you put a giant electromagnet or strong electromagnet next to the skull.
You fire a big pulse of -- a big magnetic field that induces an electrical field locally and knocks out neural activity locally.
And there's some evidence that properly applied, that this is a sort of a kinder, gentler version of ECT, that it acts to break up depressions.
I don't -- anybody know if there's a brilliant theory for how that works?
Uh, who knows?
But it's very new.
But I expect that in a few years, I'll be lecturing more about that.
Because it, at least initially, sounds rather promising.
All right.
I think this would be a good time to take a break.
Let's take -- unless you're too depressed to get up, take a break
I read Jamison's book
Uh huh.
Which one?
Prozac Nation
No.
Prozac Nation is somebody else's book
Not Jamison?
No, that's somebody else.
I can't remember who wrote it, but yes, that's a cool book, too
Yes.
That's what I thought you were talking about
Nope.
Nope.
But you'll come back and tell me who wrote it.
Because we're both forgetting who wrote it
Have they ever done, what is it, using sugar pill and Prozac and hand them out to patients and just seeing if the sugar pill --
It's a big problem in mental illness in general and certainly in depression, which is, particularly if you've got a cycling depression, who's going to get better?
Right?
And, yeah, the difference between placebos and Prozac is, at least in some of the studies that I've read, a lot less dramatic than you would think.
And the problem with a population like us is that we're too well informed.
Placebos are great.
The side effects are really limited.
In the good old days, your doctor would come around, have no clue, you know, how to treat you, but give you, you know, Dr. Hoozy's magic elixir, and you'd feel better.
And it's a great pity that now if I fed you Dr. Hoozy's elixir, the first thing you'd do is Google it and discover that it's snake oil, and feel worse and sue me.
So, yeah, if you turn out to be particularly interested, send me an email and I can send you a couple of the references of things that depressed me when I read them.
Because they weren't that much good.
On the other hand, my mother absolutely swears by this stuff.
She's had her own issues with depression and thinks that Prozac is wonderful.
[PRIVATE CONVERSATION]
I want to talk about a couple of other disease, you know, mental illness states, in part as a way of illustrating -- I guess what I really want to illustrate in the last bit of this lecture is the, sort of, mind and brain links.
We tend to like to make up theories that are one or the other.
And the current popular trend is towards brain explanations.
Oh, you're depressed because you don't have enough serotonin happening for you.
We'll boost up your serotonin.
You'll be fine.
It's a very brain kind of thing.
Freudian theories were very mind kinds of things.
But there's a number of cases that really illustrate, I think, well, what you sort of know intuitively, right?
The brain isn't here and the mind is here.
It's, you know, waves and particles.
They're two manifestations of the same thing.
And brain stuff and mind stuff bounce back and forth here all the time.
So let's think, for example, of panic attacks.
Which I didn't put on the handout at all, I see, but I suddenly developed a need to talk about because I was feeling really anxious and -- no.
So panic attack is this feeling of anxiety out of the blue.
People report that they just suddenly feel very, very, very fearful and anxious.
And it's scary and it's unpleasant and so on.
You may recall that Freud and his followers had a perfectly nice notion about this, which was that anxiety was warning you that you were about to tread on the super ego.
And it was the warning sign of the ego that was saying, don't go and do that.
But that hasn't worked that well.
More modern notions tend to focus on the idea that it's essentially neurochemical.
You get too much of some juice, some neurotransmitter.
It fires off the bits of your brain that are responsible for the experience of anxiety.
And you feel this anxiety.
And this is a lovely case of looping back to a prior mode of understanding.
The notion that you feel too anxious because you have too much of this fluid, it's an awful lot like the notion that you feel melancholic because you have too much black bile.
The difference of course is that we think we've got a better scientific grounding than we had.
But the form of the argument is rather similar.
So is it the case that panic attacks, the clinical condition of a panic attack, is just too much of some juice.
It's possible to think about it as being somewhat more complicated than that.
And, well, I didn't put that reference on the handout, because I didn't.
But I'm stealing a story from -- who am I stealing it from?
Oh, Dave Barlow, who's over at BU.
This is a three-step route into panic attack.
And the first part of it is that you feel the visible symptoms of anxiety.
Your heart starts -- remember the shaky bridge experiment all about misattribution of arousal?
You're feeling "ooh-ooh-ooh" and you see her, and you just think you're feeling "aah."
Right?
So that can work in all sorts of ways.
Suppose that your heart is going through -- there are a variety of things that cause your heart to go pitter-pat.
Suppose that you get these sort of symptoms and you don't interpret it as love, you interpret it as "I feel nervous about something.
I can't quite tell exactly what it is.
But I don't --" People don't have panic attacks, by the way, typically, when they're exercising vigorously.
Because they've got a good account of where that symptom is coming from.
Your heart's pounding, you're sweating, and stuff like that.
Yeah, all right, fine.
If you're walking down the street and your heart is pounding and you're sweating, you need a different interpretation.
Now, this is unpleasant.
So you develop, flipping back to an earlier piece of the course, you develop a conditioned fear of these symptoms.
So you start avoiding situations that would produce the symptoms, or that you think might produce these symptoms.
And this can be quite unconscious, right?
This is good classical association learning or something of that sort.
And that in turn -- that's sort of step two, is sort of a learning story that is going to cause you to start -- you're going to act to avoid situations that make you feel nervous.
But, all right, remember Pavlov?
Bell, tone.
Bell, tone.
Bell, tone.
Now I ring the bell by itself.
No.
Bell, tone.
That's stupid.
I'm not supposed to be the one who notices that first, guys.
You're supposed to -- "What's he talking about?"
Anyway, bell, food.
Bell, food.
Bell, food.
Eventually the bell by itself produces salivation.
So you make this association in your own mind.
So when did this happen to me?
I was outside.
OK.
So now I'm outside, it doesn't happen to me.
But I feel -- I get the conditioned response in effect.
So now something else acts as a trigger for the panic.
The initial feeling of panic might have been, I don't know, too many bottles of Mountain Dew, right.
You drink enough caffeine, and you'll get b-b-b-b.
Forget that you drank -- your brain doesn't remember the caffeine part of the story.
And you're going b-b-b-b.
And you say, "Oh!
I'm having a panic attack."
Then you don't want that -- this is an aversive event.
It's just like the business of the rat stepping down, right?
The rat steps down -- oh, all right.
Rat's up here, rat steps down, gets his little feet shocked.
That's unpleasant.
Rat's put back up here.
Does the rat step down again?
No.
Why?
Because he's got this anticipatory fear.
You can actually measure the fear physiologically.
All right, so you caffeine overdose or whatever.
You have this unpleasant experience.
Now you're in that environment again sometime later and you have the experience again because it's a conditioned response now, no caffeine involved.
You come to the reasonable, if perhaps unconscious, conclusion that what makes me nervous is being outside in public or being on shaky bridges or whatever.
Then you get step three, which is again learning theory kind of stuff like Thorndike's Law of Effect.
You start doing things to avoid this punishment, right.
You're going to avoid the aversive stimulus.
So you can get secondary symptoms to the panic, like a fear of open spaces, let's say, because you've now, perhaps completely unconsciously, concluded, "If I go out in public, the sky is so big.
I feel so small.
I get panicky.
And so now I'm not going to go outside."
And so what you end up with is the possibility that the clinical thing that you see, what you get in your office if you're the clinician, is somebody who says, "I have, you know, I have uncontrollable panic when I'm outdoors.
This is crippling my life because I don't go out anymore."
And what this may be is a cascade of events coming from some sort of an initial physiological issue that you then learned about, if you like, using the laws of animal learning, in ways that that don't work well.
Well, what are you going to do about it?
Well, one thing you might do about it if you believed that there was a significant learning component here is to use learning theory kinds of things to treat it.
Now if it turns out that there really is a chemical imbalance and every time the chemicals go out of whack, you feel panicked, you might want to do something about the chemicals.
But if you've got one of these situations where the person has basically scared themselves into maladaptive behavior, you've now got to un-scare them.
So what are you going to do?
One of the standard techniques for these sort of disorders would be desensitization methods.
Is it Barlow who was doing this?
So one of the versions -- you can imagine -- I did a version where you became scared of the outside.
Another version is you have a panic attack in an enclosed space and you become very nervous about enclosed spaces.
You won't go into elevators and stuff like that.
I can't remember if it was Barlow -- I got a feeling it was -- whose technique for -- there are a variety of ways of using learning theory to get around this.
One is the kinder, gentler one.
So you come to me.
You say, "I'm really afraid of being in enclosed spaces."
You say, "OK" -- the first cure out in this field that you've come to, that I'm using for my doctor's office at the moment -- "Imagine that you're in a space.
Can you deal with that?"
"Oh, it's making me nervous, but, yeah, OK, I can deal with that."
"OK, now we'll go to 10-250.
It's a big enclosed space.
And then we'll go to a smaller room and a smaller room.
And eventually we'll put you in a closet for a while.
And you'll discover -- what you'll do is un-learn the response, in effect.
You'll learn nothing bad happened to me."
And this behaviorist sort of therapy does work to break up these sorts of behaviors in many cases.
I think it's Barlow who has been advocating a much less kinder, gentler version, which is to shut people in car trunks.
So none of this business of doing it gradually.
This is known in the trade as flooding.
You just take the -- the guy says, "You know, I think if I'm in an enclosed space, I'm going to die."
"All right.
Boom.
You dead yet?
No.
OK. You're cured.
Good.
Out."
Well, it's not quite that simple, but the basic idea is that if you have a patient who can actually deal with this, that you can basically speed up the therapy rather dramatically by not moving gradually to -- this is used for all sorts of phobias.
So, scared of snakes, you know, you're pathologically scared of snakes.
OK, what we'll do is first we'll show you -- so the two versions are desensitization and flooding.
The desensitization story would be: "We'll show you pictures of snakes."
"Ooh, I hate it.
I hate it."
"Are you dead?"
"No."
"Oh, OK." So now we'll go to the zoo.
"And see that snake cage way over there?
Are you dead yet?"
"No."
"OK, let's get closer.
OK, that's a snake, right.
OK, now you're going to get in with him.
Yeah."
"Aah."
All right.
"But I'm not dead."
And the flooding technique is "You're afraid of snakes?
Boom.
See those boas?
Are you dead?"
"Yes, actually."
[LAUGHTER FROM AUDIENCE] So, anyway, the point from the panic attack example is that it's possible that what you're seeing as the clinical entity is a dance between physiology and learning, in effect, and that what you learn -- and that a mode of treatment would be to unlearn.
Now, let's take another example which I did put on the handout, which is obsessive compulsive disorders.
Obsessive compulsive disorders are, well, there's the obsessive part and there's the compulsive part.
The obsessive part refers to obsessive thoughts.
What gets you into treatment is not some sort of obsessive thought that "I love him.
I really love him.
Oh, I love him a lot.
Oh yeah, yeah, yeah.
I can't help but think about him."
Yeah, it's a little obsessive, but tough.
It's "Every time I look at him, I want to rip his guts out.
I'm imagining the knife."
You know, or something.
It's disturbing thoughts that somehow won't get out of your brain.
That's the sort of obsession that people seek treatment for.
And compulsive behaviors are behaviors where -- everybody has some degree of this.
How many people here have left their room, gotten halfway down the hall, said "Did I really lock it?
", and gone back to check?
Everybody does that sometime.
Or did I turn the stove off or something like that.
That's normal on compulsion land.
And almost always you have locked it, right?
Because the time you didn't lock it is the time you don't go back to check, and somebody steals all your stuff.
Very annoying.
I donated a laptop to the criminal elements doing exactly that.
Very annoying.
But anyway, if you have to go and, you know, get down the hall, "Did I lock it?
Yep, it's locked.
Did I lock it?"
You've got to do that forty or fifty times and you can't get on with your life, literally.
You can't get to the next event in your life.
That's a compulsion that is likely to lead you to want to do something about it.
It's interesting that these are compulsions -- rather like there are similarities in people's narrative dreams -- that compulsions seem to be drawn from a limited vocabulary of candidate compulsions.
I mean, I'm sure there's one of everything out there, but there's lots of hand washers, right.
Actually, I can see my hands are covered in chalk at the moment.
I should go wash them.
But, you know, why do you wash your hands?
Because your mother said so, right, and because there's germs on them.
Can you see the germs?
No.
So you got to do the act and convince yourself that you've done it.
OK, hands are clean.
Are they really clean?
I can't tell because there might be germs there.
I'm going to get sick.
I'll wash them again.
If you have to do this so often that your hands are getting red and raw, that's a compulsion of some sort.
Now, you can treat OCD, as it's known in the trade, in a number of different ways.
It turns out, for example, that Prozac is good against obsessive compulsive disorders.
It breaks up the obsession, you know, the obsessive quality here of this sort of thought that doesn't quite work for you.
You can imagine that you've got some little circuit in the brain somewhere that's a little checker.
You know, did I do what I was supposed to do?
And if that goes a little wacko on you, that that's got too much serotonin or too little serotonin, I suppose, or whatever, you know, you might end up in a sort of an infinite loop where, "Got to check.
Got to check.
Got to check.
Got to check."
And something chemical that gives it a kick will help.
OK. You can treat it that way.
You can also treat it using the same sort of behavioral techniques that I was just talking about for phobias.
So let's do it for hand washing.
All right, Patient So and So, why do you have to wash?
"Well, I'm just really concerned that my hands are dirty.
When I get dirt on my hands, I get all excited and [NERVOUS SOUND]."
Right.
All right.
So we can do the densitization business or we can do the flooding business.
In the interests of time, we'll do flooding.
"All right.
See this?
This is dirt.
Put your hands in it."
Right.
"Are you dead yet?"
"No."
"All right.
How long can you leave them there?"
And you can basically desensitize this feeling of deep anxiety about not cleaning your hands.
All right.
So that's mildly interesting.
What's interesting, the reason for raising this question or this example, comes from this nice article, now more than a decade old, by Baxter.
You can see obsessive compulsive states in brain imaging.
There are little bits of brain specifically in the caudate.
It doesn't particularly matter for present purposes where that might be.
But it's overactive.
You look at the brain of an obsessive compulsive sort, and this little chunk of brain is busy, you know, just going nuts.
If you give the patients Prozac, that chunk of brain now doesn't look weird anymore.
It comes into the normal range.
If you'd go and do behavioral therapy on a patient with the same story, the same piece of brain comes to look normal.
The point here is that you can reach this chunk of the brain, if you like, either through the brain sort of neurochemical route, or through the psych behavioral route.
They're not necessarily either mutually exclusive or, you know, one works and the other doesn't or something like that.
This is a case where you can get to the source of the problem, at least in a neurolocalization kind of a way.
You can get to the source of the problem through either of these routes.
And I suppose the bottom line here is that in disorder after disorder, what you find is that there isn't just one way of treating -- one problem is that in most mental illness situations, there isn't one magic bullet treatment that automatically works for you.
And then in many of these cases, there are both brain and mind routes to therapy.
So when you're asking -- when you have like these patients we were cooking up, when you have a particular patient, there's a spectrum of choices that might potentially be of some use.
It says on the handout, "What happens when drugs make us better than we ever were?"
That was the example I was talking about before of Emma, who feels better on Prozac even though she wasn't technically -- didn't have a reasonable diagnosis of something being wrong with her.
So we've already talked about that.
So I think I'll quit.
See you next week.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
I have what I think of as a couple of fascinating questions that I will use to occupy us for the next -- for the last couple of lectures of the course.
In the context of talking about sleeping and dreams, I mentioned that the hunger drive is strong, but it's not so strong that if you wanted to, you couldn't starve yourself to death.
But an interesting question would be why an otherwise healthy adolescent girl would do that.
And why in doing that she would argue vociferously that there was nothing wrong with her and that she wasn't hungry even while starving herself potentially to death.
That's one question.
That will occupy probably today.
You'll see that I did handouts for both this lecture and the next lecture rolled together in case I get to the next one today.
The second question is -- well, you might have found the evolutionary psych argument about asymmetries between the sexes in terms of what they want in relationships to be reasonably compelling.
That argument about guys wanting to -- everybody wanting to propagate their genes, but because women get pregnant and are thus tied up with the baby for an extended period of time, that you end up with an asymmetry that would cause males to want more sex, more often, with more partners, than women.
You might have found that appealing, at least as an intellectual construct.
But how would we explain, would that explain, how might it explain why at least some males end up engaging in coercive sexual behavior, sexual behavior that does not appear to be consensual between the partners?
And that's what's going to occupy us for the last lecture of the course.
Now these are very different problems, the problem of eating disorders on the one hand and coercive sexual behavior, or more colloquially, date rape on the other side.
But they do have some interesting parallels, one with the other.
These are outlined a bit, I guess, in the abstract part of the handout.
They're both problems that are gender specific.
The overwhelming number of eating disorder patients are female, and the overwhelming number of people who get into trouble for, you know, brought up on charges in coercive sexual behavior cases, for instance, are male.
So they're gender specific.
That's of some interest.
They both serve a useful function at the end of the course, because they're both useful for bringing back theme after theme that we've seen throughout the course, themes like revisiting is it nature, is it nurture?
Is it biology or is it sociology?
And the answer always turns out to be, or often turns out to be, that these are complicated interactions.
And the problems of eating disorders and coercive sex turn out to be particularly rich interactions between a number of factors.
And by the same token, neither of them is well dealt with in any simpleminded, oh, give them a pill, for instance, kind of way.
And they're both what could be described as being successful disasters.
I'll elaborate on that later.
But both of them are in a perverse sense successful behaviors and in a more obvious sense disasters, unsuccessful.
Let me start by sketching a portrait of what would be a typical patient with the diagnosis of anorexia nervosa.
The term simply means -- anorexia means a lack of appetite.
Nervosa is just what you stick on when it's of nervous origins.
You know, you could have called it psychogenic or something like that.
It just means a loss of appetite that you're not going to explain by some organic cause.
A typical patient would be an adolescent girl, say fourteen, fifteen, sixteen, who gets brought typically by her family to the doctor.
Very skinny.
Part of the characteristics you can see on the clinical criteria for the diagnosis -- part of the characteristic is a refusal to maintain what's considered a normal body weight.
So maybe, you know, at her height and age, she should weigh about a hundred pounds.
She might come in weighing seventy-five pounds looking to all the world like she is emaciated -- looking to all the world but one like she's emaciated.
One of the characteristics is a -- well, I think it's on the handout -- distortion of body image, a denial that you're actually skinny.
And it's not just, you know, some sort of a cranky, you know, what did you get in biology this term?
Did you flunk biology this term?
No, I didn't flunk biology this term When you can see on the report card that it says F or something like that.
It's not that kind of denial.
It's almost like a perceptual disorder, where a young girl who looks to everybody else like she is a famine victim looks at herself in the mirror and says, I look fat.
As though she was seeing something that literally wasn't there.
This is combined with an intense fear of being fat, of being overweight.
And one of the official diagnostic criterion is a cessation of menstrual cycles for more than three months.
There was an interesting article -- Bo, you sent it around, right?
Did you send around the New York Times thing about EDNOS?
An interesting article in the New York Times science section last week pointing out one of the problems in psychiatric diagnosis.
Suppose you got a patient.
She's fifteen years old, skinny as a rail, intense fear of being fat.
Says she looks fat.
And her monthly cycles are irregular, but they've never disappeared for a full three months.
Under the official rules of the game, you can't give her a diagnosis of anorexia because she doesn't meet all the diagnostic criteria.
Clearly has an eating disorder.
What you end up labeling her as is eating disorder not otherwise specified.
EDNOS is the official category for this.
It becomes an issue in the economics of health care because what you can get reimbursed for, what you can treat for, are real diseases.
I mean, reasonably enough, you can't go to your doctor and say, you know, I want a pile of drugs and therapy that I want somebody else to pay for because I hate my calculus TA pathologically.
Yeah, OK. You can't just decide that -- well, you can just decide that you're ill in something.
Or you can't just walk in and say, you know, I want you to treat me.
I desperately need you to change the shape of my nose, and I'm not thrilled with my rear end and, you know, I kind of want you to reshape my whole body because, you know, my body, it's just not right.
You have to get a diagnosis of, you know, body dysmorphic disorder or something like that before you can persuade the third party payers to do it.
Anyway, in psychiatric land this is a big problem because it's very difficult to come up with -- you know, there's no blood test, right?
If you've got polio, there's a polio virus somewhere to look for.
If you've got an eating disorder there's not.
And so exactly how to label people is a substantial problem.
But I'm not going to say more about it.
In any case, so here's a patient.
This disorder is interesting as an interplay of mind and body in many, many ways.
Part of the reason that it's interesting is because there are all sorts of psychiatric consequences that just arise from the fact that this girl -- I'll stick with it being a female patient because anorexia is about eight to one, nine to one female -- because she's starving herself.
For instance, starving yourself is depressing.
If you are really hungry, you also tend to have more symptoms of clinical depression.
Is depression part of the eating disorder pattern?
Is it part of the pathology or is it a side effect of the pathology?
In any case, it is not an uncommon disorder.
Point prevalence is jargon for saying if I go and look at a population at this point in time, how many people have the disorder I'm looking for?
And if you look at a population of young women, the point prevalence of anorexia is about half of a percent.
If you broaden it to this EDNOS, eating disorder not otherwise specified category, that can go up to anywhere between 2% and 5% depending on what you read.
If you look at prevalence rates, as I say, they're about eight to one female to male.
So it's a very heavily female disorder.
And it's a dangerous disorder.
For patients with full-blown anorexia diagnosis, the mortality rate can be as high as about 5%.
That's very high for a disorder of a population that's otherwise, you know, young healthy women.
What kills you, by the way, is not starving to death, but things like heart attack.
Your electrolytes get to be so out of balance that your heart fails to work and things of that sort.
It is also a disorder, in a sense like multiple personality disorder, that has changed greatly in its frequency in the population in historical time, like the last forty years.
It's different from multiple personality disorder in that this change may be a real change in incidence.
Remember the argument was that in multiple personality disorder, dissociative disorder had been there all along, and that multiple personality was merely the way that it manifest itself, that the dissociative disorder manifested itself in modern Western culture.
In the case of eating disorders, it really may be that the disorder is itself more frequent than it's been in the past.
It is certainly a -- depends on what society you are in.
It was characteristically, even as recently as probably twenty years ago, it was absolutely characteristically a disease of middle-class white girls.
It has since managed to embrace diversity and reach out to other groups in Western culture.
For instance, virtually unknown in Japan until fairly recently, but now is a disorder showing up in Japan.
I think I just read something that even more recently, it's now starting to show up in mainland China, where it was unknown.
One of the requirements for a good eating disorder population, by the way, is a culture where there's lots of food.
You don't have people with eating disorders of this sort in famine.
You can be seventy-five pounds when you're supposed to be a hundred pounds because there isn't enough food, but it's not because you're refusing to maintain your body weight.
So it is a disorder of a society that is feeding everybody, that would be able to feed everybody successfully.
So where does this come from?
What's it's etiology?
What are the risk factors that contribute here?
I can tell it must be time to flip pages on the handout, right?
Look at that.
What's the causes?
And it reminds me to caution, to step back for a couple of cautions.
The primary caution here is that this is not intended to be talking about you specifically.
Because, well, all right, here, let's jump ahead.
We can say stuff about, again typically, the girl who is a patient.
There are characteristics that -- not every -- you know, it's a big, wide distribution as usual.
But we can say something about where the sort of typical patient might lie.
Typically she is adolescent -- sometime after the onset of puberty is the, sort of, risk period for the onset of anorexia typically -- described as a good girl, something of a perfectionist, the one who never gave any trouble.
And now you should be immediately able to see why I put this caution in that I'm not talking about you.
Who in the world gets to MIT by being a bad, evil, non-perfectionist?
Well, you know, there's a couple of you here maybe who, you know, just got through because you're brilliant, bad, evil, non-perfectionists or something like that.
But the rest of you spent those adolescent years being -- those of you who were women -- the rest of you didn't spend your time being women.
But, you know, you spent your time being good and doing your homework and not raising a lot of trouble and stuff like that.
Oh, great.
Now let's talk about the family a bit.
It's an interesting disorder because it turns out to be important to talk about the family.
If you are dealing with a lot of other health issues, your family background is of some interest, particularly for sort of genetic reasons, but the structure of your family is typically not desperately important.
Here it may really be important.
The family -- again, wide range possible -- but the typical family would be one that's described as -- what have I got here?
-- overprotective and also perfectionist, achievement oriented.
And you're sitting there saying, oh my god, not only was I good, but my family, they're overprotective, and they're achievement oriented.
And when I said that what I really wanted to do was go to the community college, they locked me in my room for three days until I finished the MIT application and stuff.
You know, this is our history.
These are our people.
We know them.
The other characteristic, to introduce a little piece of jargon here, the families are described as enmeshed.
This is a piece of jargon that I understand well, because I come from a family that is beautifully described, you know, beautifully fits the enmeshed idea.
I don't understand how my sisters got through their, you know, youth without an eating disorder.
Because every year when I talk about this, if I talk about the risk factors, you know, it just sounds like my family, our nice achievement-oriented enmeshed family.
Enmeshed means a family where people -- imagine a family, like yours maybe, where people finish each other's sentences and stuff like that.
They're sort of all over each other psychologically.
You know what they want.
They know what you want.
The place to see it in the Wolfe family, as my wife who of course married into this found out, is go out to dinner with them.
You don't want to do this, because it's pathological.
First you've got to decide where to go for dinner, right?
So everybody gets together.
You decide where to go to dinner.
You've almost got a consensus.
You're going to go to Legal Sea Foods or something.
At which point one of us, it might well be me, points out my mother doesn't really like fish.
Is my mother fussing about this?
No, my mother's perfectly happy to go along with what everybody else wants.
But, you know, I'll say, you know, but Mom doesn't like fish.
Well, actually, these days I would say, but Gaga doesn't like fish.
My eldest child when he was a toddler named my mother Gaga.
And that's been her name ever since, which is, you know, she deals with it.
Anyway, so all right, we've got to cancel that one.
So now the whole conversation starts again.
And you can do this for hours.
Julie, my spouse, learned the best thing to do, particularly if we're out somewhere, you're wandering around trying to decide where to go to a restaurant, is just drop about five, six paces back.
You just don't want to be in this conversation.
But eventually we'll end up at a restaurant, at which point everybody's ordering for everybody else and is ordering stuff that they don't necessarily want to eat, but they think somebody else might want to eat.
My mother is the queen of this.
My mother will order things she doesn't even like because she thinks one of her children wants to try it or something like that.
And this is still true now that her children are in their late forties.
It's OK, Ma.
I'm not starving anymore.
Anyway, that's enmeshed.
These families are also characteristically intolerant of expressions of conflict and anger.
It's not that there aren't conflicts in these settings, but the family will act as a unit to smooth them out.
This is not a, you know, dramatic family where people throw crockery at each other and then, you know, in act five they all hug or something like that.
This is a family where if there's an issue, it's dealt with in some quiet, perhaps even subterranean, kind of a way.
So that's sort of the patient and the family.
And then you've got to ask, given a disorder that has been a disorder of Western middle-class culture, you've got to ask what it is about the culture that might be contributory here.
Again, there are other situations in which the broader environment is important, like is it putting stress on you in some sort of global kind of a way.
But here the specific demands of the culture turn out to be of particular interest.
And the particular demand that people focus on is the degree to which American culture, more broadly Western culture, is simultaneously obsessed with both food and thinness.
This is a difficult combination, right?
So how do we know that this is a culture obsessed with thinness?
One of the most telling ways is to take a look at the ideals of female beauty.
What is it that society as a whole declares to be beautiful in women?
And if you do something -- I should check if there's a cool website for this.
If you just line up the pictures of Miss America, for instance, over the last fifty, sixty years -- if you take a look at Miss America from the 1940s, you would agree that she's a perfectly attractive-looking woman, but you'd think she looks a little chunky.
There's a lot more on her than would be the case for a modern Miss America or a modern -- open up a fashion magazine at random and look at the models.
Much skinner than the ideal would have been a couple of generations ago.
Or take a look at one that I learned about when my children became big fans of the James Bond movies.
So in every James Bond movie, there's the girl, right.
She's been getting a lot skinnier over the years.
If you go back to the early ones -- I don't know how many of you are great James Bond aficionados.
But if you go back and look at a movie like Goldfinger -- I will not recite all the names of the characters because they're all bad lewd puns I'm realizing as I'm thinking about this -- but if you go back and look at the girl in Goldfinger or in Dr. No or something like that, you will find again -- that's from the early '60s, mid '60s -- you'll find that she looks, you know, perfectly attractive woman.
But again, there's more of her than there would be in -- who's the tart in the most recent --
Halle Berry
Halle Berry.
Yes.
Nothing there, right.
She's pretty skinny.
And another move in the James Bond thing, which is an interesting thing in its own right, which is that they still have to end up together by the end of the movie, but they're getting a lot crankier about the whole business, right.
It's like they get into this movie.
They realize that by the end of the movie, I've got to, you know be in one of those silly scenes with whoever's playing Bond this week, but he's a jerk really.
But anyway, that's a separate issue.
Anyway they've been getting skinnier and skinnier.
Barbie has been skinny forever, but gets blamed -- is held up literally as an icon for this problem, you know.
Women do not look like Barbie.
If you scale Barbie up to human size, there are serious biomechanical problems that ensue, apparently.
Somebody wrote a marvelous -- there's a great engineering piece on that somewhere that I saw once upon a time.
There are cantilevering problems and all sorts of very bad problems with Barbie if you scale her up.
But the notion is that the ideal of what is beautiful has been getting skinnier and skinnier.
At the same time, there has been this surge in both the availability and the diversity of food available.
So you're expected to be skinny while plunked down at this spectacular all you can eat buffet.
And that's, you know, you can imagine that there are a certain amount of problems there.
There are specific populations, and these are unsurprising specific populations, where you see higher than average rates of eating disorders.
So for instance, gymnasts, eating disorders are overrepresented in that population.
If you're looking for male eating disorder patients, a great place to look is in the wrestling population.
Why is that?
Well, if you're wrestling in a weight class and you have to be exactly this weight, particularly since there's a pressure to wrestle in the lightest weight class you can sort of get away with, there's going to be a lot of pressure of the sort that's similar to this pressure that you might imagine being applied by an ideal of female beauty that's very skinny.
So there are these pressures -- yes, yes, yes?
In wrestling they also have [UNINTELLIGIBLE] they measure you a set number of hours before the actual match
Yes.
There are all sorts of reasons why this is not inclined to produce wholesome eating behavior as I understand it.
My own personal wrestling career -- [LAUGHTER] Why are they all laughing?
You know, it's only mildly -- well, I suppose maybe it's completely hilarious.
It was clear that I wasn't going to wrestle heavyweight, right.
But, you know, the world is full of these little tough guys as opposed to just merely little guys.
[LAUGHTER] Kristin, that letter I was writing for you -- it's really bad when the people in your lab are the ones who are becoming hysterical.
Anyway, I might as well complete this embarrassing story.
High school wrestling, the gym teachers were always weird people.
And so high school wrestling you had to wrestle by weight class, right.
That made sense.
But our guy thought it was like a circle.
So you wrestled the guy next to you in weight.
Well, if you were the lightest guy, it's going to be a hoot.
Let's get him to wrestle the heaviest guy.
Which was actually OK because the heaviest guy was just about as athletically talented as I was.
The major danger was that he would fall on me, right, because then it was just going to be all over.
So I'm -- athletics was not my strong suit.
So there was no danger that I was going to do him any serious damage.
And there was no serious danger that, as long as he didn't fall on me, I was going to get hurt.
So I figured out a way to get on his back and sort of rode him around for a while, and did sort of manly things that looked like I might be trying to flip him, but that wasn't ever going to happen.
It was a great moment, made greater of course by the fact that -- never mind.
No, I should continue.
This was also -- he was -- this was full of -- this guy, deeply homophobic gym teacher who also was fond of explaining that how hard you wrestled showed whether you were a real guy.
And anyway, it goes downhill from there.
So in any case, the -- where were we?
Oh yes.
This explains why my wrestling career did not produce any eating disorders in my case.
So there are these societal or specific cultural factors that might be pushing one towards these sort of disorders.
And then there's the question of whether or not there's a biological factor there.
Is there some sort of a genetic predisposition towards this?
The sorts of things that people point to are data, for instance, that there are family histories of depression, more depression showing up in the families of eating disorder patients than in the general population.
There's also more depression in eating disorder patients themselves.
But as I've said, that's hard to disentangle from the effects of starvation, that that itself produces problems.
In any case, well actually there is one thing that is potentially a biological underpinning.
Has nothing to do with the brain at all, but is simply -- in diet land, these are known as set point issues.
There's a notion that people come with some sort of weight that is sort of the weight that they are set for.
The little thermostat in the hypothalamus is to say, you know, you're going to be 180 pounds or whatever.
You can push this around, but this is where we want to be.
It's related to the fact that there are certainly genetic factors that are making you one body shape versus another body shape.
And if you are one shape and you're feeling strongly pressured to be another shape that is not the shape that you were sort of built to be, that's considered to possibly be a biological factor pushing people towards eating disorder.
That's not like the sort of factors we're thinking about as a genetic predisposition to something like schizophrenia.
That's really an interaction of body type with these sort of cultural pressures.
All right, so you've got characteristics of family, patient, the culture as a whole, possibly biological underpinnings.
How might all of this contribute to producing an actual disorder?
One way to think about this is to think about it as a trap that some subset of girls fall into.
Not consciously.
Nobody wakes up in the morning and says, "Gee, I think I'll have an eating disorder."
But the unconscious process might run something like the following.
You hit puberty.
So, all right, so you've been this sort of perfectionist good girl all along.
You hit puberty and sort of a time that stretches or strains perfection altogether.
Being perfect requires a certain amount of self-control, and you may recall the sort of wash of hormones that accompanies the onset of puberty is not particularly good for self-control.
So the sense of control is not what it used to be.
Your body is changing in ways that you don't control.
And you're in this family where the set of ways that you can act out is very limited.
There are lots of ways to act out in adolescence, but you're in a family where most of them are sort of not sanctioned.
So one way to sort of re-exert a sense of control, a very culturally endorsed one, is well, let's go on a diet, right.
The percentage of women in this country on diets at any time is huge and that goes down to school-girl age.
And if you ask -- you go and talk to middle school boys about diet.
Ehh.
They sort of grunt at you or something.
You go and talk to middle school girls about diet and they can talk your ear off.
Whether or not they're on some sort of a diet, they know all about it.
This and the that.
It's there.
In fact, you might go on a diet endorsed by mom.
Mom might say, you're looking a little chunky.
And, you know, we can do it together.
It's a mother daughter kind of thing.
Well, you know, particularly since, you know, we're really enmeshed here anyway.
All right.
Well, you can sort of imagine that a perfectionist on a diet, under the right circumstances, could be a recipe for disaster, you know.
It's sort of like MIT and sleeping.
People out there say, all right, I need to do a little extra work so I'll stay up an extra hour.
MIT student says, I don't need to sleep.
You know.
Most people out in the world say, diet, oh, OK, we'll cut out the double dip, you know, double-stuffed Oreos and, OK, we won't deep fry the pizza.
And this hypothetical perfectionist says, water.
Lettuce and water.
That'll work.
So anyway, you can imagine that the diet could -- that what happens when this is spiraling downwards, this diet can become increasingly severe.
And at some point it becomes a real issue.
It becomes a source of conflict in the family because it becomes clearly unhealthy.
It's clear to anybody except for the patient that this is unhealthy.
But at that point, if you regard this as some sort of an assertion of autonomy, this is the last place where this girl has managed to find a place where she can carve out someplace where she is in control.
Now the family is saying, you can't do that either.
And she draws the line.
This is where I'm going to make my stand.
And it can become a clearly pathological state.
People in full blast anorexia deny that they are hungry in spite of being strongly malnourished.
The rituals around eating, the rituals that get built up around eating can be extremely elaborate and extremely time consuming.
Along with, for instance, very vigorous exercise regimens that -- basically it takes over your life.
Plus being starved not only tends to make you depressed, but it also can make you delusional in a variety of ways.
So it's clearly not a good place to be.
So that's the sense in which it's a disaster.
It's also in a sense -- the sense in which it's a success, this notion of it being a successful disaster, the sense in which it's a success is if you regard this as an issue about control, in some sense, the patient, the daughter, has managed to take control over the family.
That wasn't her goal, but there's nothing like a life-threatening condition to galvanize the attention of a family.
And here in this clearly pathological way, the daughter has become the center of the family's attention.
This quiet, never caused any trouble, you know, bury the problems elsewhere daughter has now successfully taken over the family.
The problem with that success is it's clearly desperately maladaptive and potentially dangerous.
Well, what do you do?
As you can imagine, virtually every major psychotherapeutic regimen has been tried in some fashion.
The punchline, the bottom line, is that my understanding of best practice is that nothing by itself works terribly well and that what you do is some of everything.
Now, what is everything?
Well, one bit of everything is you've got to get the patient to eat.
Often by the time you've got an eating disorder patient who has reached the point of being in medical care, this is a potentially life-threatening situation.
And the patients often get hospitalized.
Under hospital settings, one of the things that works really well, and one of the things that was once upon a time touted as a magic silver bullet cure, comes straight out of Skinner boxes and behaviorist theory, you know, learning theory.
And that is to set up the ward, well basically as a giant Skinner box.
OK, there's stuff you want, right?
You want to be able to call your friends?
Good.
Here's what you've got to eat.
You want to be able to watch TV?
Good.
Here's the weight standard you have to reach.
And so you set up a clearly defined set of rewards in response for eating behaviors.
And it works very nicely in many cases.
You can get people back up to weights that are appropriate for, you know, age and height.
Works just fine.
Why isn't it a magic bullet?
Well, what happens when you go out of the hospital?
You go out of the hospital, you're back in the environment that produced the disorder in some sense.
And it tends to fall apart.
The problem was that relapse rates were very great.
So one of the things you're going to want to do is not just treat -- people aren't pigeons.
You don't want to treat them just as pigeons.
You can train your pigeon to do whatever with a nice schedule of reinforcement and probably that doesn't work too well when you release the pigeon into the wild either.
With a human, it's nice to talk to them.
So psychotherapeutic approaches have been -- you know, various forms of talk therapy have been tried.
Freudian therapy centered around the idea that -- the core issue here was a fear of growing up in the sort of typical Freudian tendency to sexualize everything under the sun, this gets described as a fear of pregnancy, that, well, you know, if you manage to suppress menstrual cycles and things like that, you're not going to become pregnant.
And by starving yourself, your body will be more like that of a little girl.
It's a little like the sort of metaphoric senses of trying -- of Snow White's evil stepmother trying to keep Snow White as a little girl.
But anyway the Freudian notion was that there is a certain terror about getting older, you know, growing up, that the patients are dealing with.
Just talking to him about it didn't turn out to be a huge success, in part because patients who are that undernourished are a little like schizophrenic patients.
They're not good candidates for psychoanalysis or for much other in the way of talking therapy.
Delusional people are not great at insight.
So combining some sort of conversation, some sort of therapeutic conversation, with something like the behavioral modification technique, that helps.
There is evidence that pharmacology helps.
There is an obsessive quality to the thought processes of anorexic patients.
They're deeply obsessed with food.
In fact, you get these fascinating case histories where the anorexic is doing all the cooking for the family.
She loves to cook for everybody.
She just doesn't eat any of this.
You know, absolutely obsessed with food, swearing they're not hungry and engaging in elaborate rituals around eating.
You know, that's an obsessive kind of behavior that turns out to be broken up by Prozac and similar drugs quite well.
So one thing to do is to feed them an antidepressant like Prozac, in part because they may be depressed and in part because it acts against the obsessive thoughts.
Just feeding them an antidepressant by itself has no great track record.
But, all right, let's combine a little antidepressant medication with a little behavior modification, a little talk therapy, this is all working -- this all might have some meat to it.
One of the things that seems to be an interesting part of treatment regimens in eating disorders, more so perhaps than elsewhere, is it's often useful in anorexia particularly to talk with the family also.
Because that family, they're actually -- there are clinicians who argue that anorexia is a disorder of the family where it just happens to be that the girl is the designated patient, that the family as a system is sick and that the daughter is showing the symptom.
And that if you want to have this work out, that what you need to do is to treat the family and get the family out of each other's faces a little more, and more willing to express feelings to each other perhaps and things of that sort.
The prognosis is a little reminiscent of Anna O., if you remember that story from the history of Freud's development.
Anna O. had symptoms that Freud and Breuer could treat, but when those were treated, something new popped up.
There's a flavor of that when you read about case histories of anorexia, that you can -- most patients recover.
They become non-anorexic.
But the population sees more depression going forward than a typical population.
The population sees more other eating disorders than a typical population, as if there's something that was not dealt with.
And exactly what that is, we clearly don't really know, but that anorexia was a particular crisis and that there are issues that may need to be dealt with on a long-term basis.
Now, one of the disorders that you can, sort of, progress to, if you like, not infrequently, or you can manage to generate it all by itself, is bulimia, or more technically, bulimia nervosa.
Though anorexia nervosa gets called anorexia nervosa, bulimia often just gets called bulimia.
Oh, did we just switch?
Oh, look at that.
It's so much fun to watch everybody go [SWISHING SOUND].
It's glad to know you're following along.
So bulimia is -- one of the reasons -- a follow-on disorder -- well, I don't know if it's a reason.
It's not surprising it's a follow-on disorder.
It's characteristic of an older population.
Typical onset in college age or twenties and a different kind of, sort of a different story.
So one important characteristic is these are patients typically who are outside of the family situation.
They're not embedded in their family.
They've gone off to college or they've gone off to work or something of that sort.
A bit insecure -- let's go back to this we're not necessarily talking about you thing.
A bit insecure -- right.
Who isn't?
The ones who aren't a little bit insecure are the people, you know, near and dear to us who are insufferable.
So a little bit insecure, maybe.
Maybe a bit more insecure than usual.
A bit impulsive.
What's impulsive?
When you get the family history, sorry, the patient's history, you get anecdotes about, you know, dumb impulsive things.
Oh, on a whim I shoplifted once.
It's not like I'm a chronic shoplifter.
You get sort of dumb single incident kind of things in these reports.
And self-esteem not great, and wants to fit in, which of course qualifies for most of us also, at least the wants to fit in part.
So the dilemma faced by this woman now living independently is that the fitting in piece involves socializing.
And the socializing involves a lot of eating and drinking.
Fitting in also involves, rather literally, fitting into those jeans or whatever that were designed for some half-starved model in the fashion magazines somewhere.
And so you've got this conflict between food and thinness made quite concrete.
And these are our patients who, in a sense, stumble on a trick.
And the trick is that if you eat and then you get rid of the food, those calories don't end up on your hips, right?
So if you eat and then take a laxative or throw up, then you don't get the calories, right?
Now, that's a solution, but it's not a really great solution.
It's not a great solution for any number of reasons.
One of them is that, interestingly, it feels shameful.
That's interesting because you could go onto a whole thing about why -- no -- I suppose no is probably too strong.
A typical bulimic is not going to have a buddy bulimic who, you know, let's go off and go to this party and eat and go throw up together.
I mean, it even sounds weird -- oh, by the way.
This is not unknown culturally, though I realized talking to my concourse class this morning that it's apparently much less known than I thought.
Because my eight year old can tell you all about this.
But in Roman culture, sort of high Roman Empire culture, there were, I don't know how widespread this was, but certainly in the orgy class, there was a custom of having these, you know, very elaborate parties where you'd eat and eat and eat and then you couldn't eat anymore.
But you wanted to keep eating so you went to the next room, you threw up, and you came back and you ate some more.
And so there are even villas with rooms that are designated as the vomitorium.
And so how many people knew about this?
All right, so it is a minority.
It's absolutely fascinating to a certain round of eight year-old boys, who think this is, you know, really very interesting.
Not something they want to try, thank you, but you know, that's really -- well, they like other gross stuff.
Anyway, the problem is that first of all it's something that feels shameful and needs to be done in private.
It's clearly sort of an odd solution.
So maybe you try a more mainstream solution.
Let's go on a diet.
And maybe it becomes a fairly severe diet.
But unlike anorexics, who deny their hunger and really don't seem to feel that hunger, bulimics feel hungry.
They feel really hungry.
And so the diet doesn't work.
And what you end up with is that the diet breaks down at some point.
You go off the diet, perhaps in some spectacular bingy kind of a way.
Well, that feels lousy too.
And so, well, you figured out this trick once.
You go and purge again in some fashion.
And that gives you some transient release from the shame of having broken the diet.
But again, the purging thing isn't great either.
But this can become -- and in full-blast bulimia it becomes a very ritualized behavior in its own right.
Actually, somebody was telling me that some bulimics plan things very carefully so that they eat the nutritive foods first in a sort of a binging session and then finish up with the half gallon of ice cream and the jar of mayonnaise.
I mean, the things you read people -- seriously, the things you read the people eating on bulimic binges are really quite amazing.
Imagining sitting down and eating a thing of mayonnaise is just very odd, but this is the sort of thing that gets, that apparently -- anyway.
But you stagger it in such a way so that you get at least enough of the nutrients that you need before you go and purge that you're not starving yourself.
And bulimia, unlike anorexia, can be a very long-term disorder.
People can maintain body weight for a long time doing this.
It's clearly not adaptive.
I mean, it's a success in, again, in a limited sense.
Let's you eat and lets you maintain this skinny weight.
But in many other ways it's a disaster.
First of all, repeated purging is very bad for, you know, do a variety of bits of permanent damage to you physiologically.
It carries with it its own risk of mortality.
Again, typically from heart attack from getting electrolytes sufficiently imbalanced to stop the heart.
And it does really bad things to that hunger drive circuitry in places like your hypothalamus.
Because what happens is the body is smart.
It says, we ate a half gallon of ice cream and a thing of mayonnaise.
And we got this much caloric bounce out of that.
You know, that's odd.
I don't know what the problem here is, but I know how to adapt to that.
If it is the case that huge amounts of food are not providing me with the calories I need, me up in the hypothalamus here, well, the answer is something's wrong in the gut.
But the answer is I better eat more.
I've got to make this sucker hungrier, ramp up the hunger.
Bulimia, the word comes from to have an appetite like an ox, like a bull.
Because what happens is that the body learns that food isn't doing it for you anymore and you just keep eating it.
It's a vicious cycle.
You've got to eat more and more.
Well, this other -- it's two chunks of your brain fighting.
This hard-wired chunk of hunger drive is saying, need this many calories.
If it takes, you know, this many truck loads of food to get it, well, go out and eat that truck load.
And you've got this other chunk of the brain saying, we're getting in those jeans, man, so go and throw up again.
And these two are fighting with each other in a way that's clearly at this point maladaptive.
Now, we have this -- the regular population away from the eating disorder population runs into the same problem, by the way, with diet foods.
Diet foods don't give you the caloric punch of regular foods.
That's the whole idea.
So these little chunks of the brain say, hey, eat twice as many.
See the label?
It says half as much fat.
Brain says, eat twice as many.
And the other thing it says is, you know, what do you like?
Well, you're built to like -- brain says, we like double-stuffed Cheerios -- Cheerios.
Oreos.
There's probably double-stuffed Cheerios now too.
Anyway, we like double-stuffed Oreos because they've got lots of calories and this little piece of my brain is still, you know, foraging around on the savannah.
And when I found double-stuffed Oreos on the savannah, man, I could live for a week off of one of those suckers.
It's great stuff.
Now, you're eating the low-fat reduced everything version of this, and the brain is saying, lousy stuff, man.
You're eating bark and twigs here.
So it doesn't taste good to me no more.
Even though it tasted perfectly -- this is one of the reasons why diet foods cycle through the supermarket much more rapidly than high-fat foods, which is that any diet food tends to have a fall off in its appeal to the population fairly rapidly.
The Oreos stay there forever because people love them.
The reduced fat all-twig Oreos have to be replaced next week by the, you know, low carb, reduced fat, high cardboard Oreos or something, in a brand-new package so that you'll go out and try it.
Because, well, you know, those low fat ones, they were OK for a while, but my hypothalamus is now telling me they weren't that good.
So you have to keep churning the product line in order to keep you happy.
Treatment turns out to be much the same story as with anorexia.
Nothing by itself works.
There are a couple of big differences.
One is, as I said before, you can go for a very long time without seeking treatment because it's not as acute a disorder as full blast anorexia is.
It's not good for you, but you can maintain this.
Plus it's a very private disorder and so you can tend to maintain it all by your little lonesome for a fairly long period of time.
The other thing is that it no longer turns out to be desperately interesting to talk to the family about this.
You're outside the family.
But in place of that what turns out to be an interesting component of treatment is group therapy.
This has been a very isolating disorder, and when you finally come to treatment, you say to yourself in some fashion, I am so weird.
There's nobody else like me in the world.
And that doesn't help you.
That's just depressing and off-putting in its own right.
If you're with a group of other people with a similar disorder, you're saying, look, hey, it turns out there are other people like this.
We'll work through this thing together.
And that turns out to have some therapeutic value.
Again like anorexia, people tend to get over this and then tend to have, to need to sort of be vigilant about food issues and vigilant about other such issues ongoing past that as if there were some underlying issue that hasn't quite been addressed.
So, well, I think what I will do unless there's some -- oh, no, I got to all the various words on the handout.
Look at that.
Let's take a break, and then what I'll do is at least set up the problem of coercive sexual behavior and we'll go on from there.
[PRIVATE CONVERSATION]
Is there any good evolutionary reason that girls get anorexia or is it just that we idealize the girls --
The mainline argument I think is the pressures are exerted on girls much more than boys, that if the ideal guy had -- there are sort of muscle building-ish disorders in guys that might be comparable
Is it formally diagnosed?
Is it formally diagnosed?
I don't know.
[INTERPOSING VOICES]
[INAUDIBLE]
I would be amazed if it wasn't because everything is formally diagnosed somewhere.
But, you know, what really shows up there, I think, is substance abuse, right?
In this culture it's much less that you get somebody who's pathologically working out in the gym, though I'm sure that happens.
They go and pop enough steroids to get themselves into trouble
What about the cases where people make themselves throw up but not particularly related to food?
Well, OK, there's a whole -- and I don't actually know much about the whole set of self-harming disorders.
Actually I should probably sometime teach myself something about it.
Because there's another pathology that is new, I think.
Or if it's not new it was deeply hidden before, the notion of people cutting themselves and things like that.
So it may be that there's a -- I don't know anything about it, but there may well be a make yourself throw up in the self-harming kind of category
Have you ever heard of the anorexia Web sites?
Like not the --
The anorexia --
Like, Web sites --
Oh, yes, yes, yes, yes.
I read an article about it at one point, where people are busy encouraging each other in their eating disorders.
Not wholesome, I wouldn't say
Yes, clearly.
But did they change the [INAUDIBLE] at all, or the way anorexia --
I don't know.
I don't know.
And I don't know if it's widespread enough to be -- I mean, I've just seen a couple of, you know, I think sort of popular press things about it.
I just don't know how widespread that is.
I know who to ask at MIT.
There are a couple of people who are good experts on that at MIT.
Anyway, the dangers of relatively complete handouts or of comparatively complete handouts is that if I don't then hit all the lines on the handout, there's a danger that somebody's going to want to know what I meant by magic cures for imperfections and the superwoman myth.
So let's jump back to societal pressures pushing people towards anorexia.
The magical cure thing is the notion that we're a culture that firmly believes that you should be able to cure your problems right now and without too much real hard work.
So that if you are not sexy enough, here is the mouthwash, the clothes, and the six amazing acts that you can perform in the privacy of your own room that will do it all for you.
And you can read these at the supermarket checkout, right?
One of my favorite -- I mean, you get stuck at the supermarket checkout -- I love bouncing down just the headlines of what are described as the women's magazines, Cosmo being the best of these.
Because Cosmo is always willing to make you completely sexy by next month.
And amazingly they need to do it all again the next month.
But, you know.
This move will drive your man wild, once a month on the -- anyway, it's great stuff.
At least I suppose it's great stuff.
I don't get to read the journal much.
It's not one of the technical journals in my field.
All right, so there's the magic cure piece.
And then the superwoman myth, which many of the superwomen here may relate to, is a -- I can't remember who coined the term -- but it's the idea of the woman, or the guy for that matter, who can do it all, right.
She's got the high-powered career based on her great MIT degree.
And she also is the mother of fifteen and making it to all the school plays and making the food and of course knitting the clothes by hand and stuff like that.
There's typically not a similar superman myth because even in these more egalitarian days, it still remains the case that the burden of child rearing falls more heavily on the female than on the male typically across this culture.
So if anybody is going to be massively stressed out by trying to balance the home and career thing, it's more likely to be the woman.
At least, that's the argument that's being made here.
All right, let me jump then to this other topic of coercive sexual behavior.
And let me advocate that you should take advantage of your time in Cambridge sometime to go to the American Repertory Theater just off the Harvard campus, across the street from Radcliffe.
Because I think -- my recollection is -- actually I haven't checked lately because I haven't been a student for a long time.
Students used to get in amazingly cheaply.
And so you should do this.
They put on a range plays from modern to classic, but almost always with some kind of out there attitude about it.
So I remember this great production of a Handel opera, which they had decided to set -- act one was set in a trailer park in a Florida swamp, and act two was set on Mars, neither of which I believe were described as the settings in the original libretto.
Somebody -- that kind of thing.
Anyway, some years ago, I saw a production of Shakespeare's Midsummer Night's Dream there.
And at the beginning of the play, what they had done was they staged a practice fight.
It was clear that these were two knights practicing rather than fighting in an actual battle, sparring with each other.
And they're whacking away at -- the lights come up and they're whacking away at each other.
And finally one of them flattens the other and pulls a sword and it's at the neck of the guy on the ground.
And at that point, the lines you hear are, "Hippolyta, I wooed thee with my sword; and won thy love doing thee injuries.
But I will wed thee in another key, with pomp, with triumph, and with reveling."
What the A.R.T.
had done was at the moment that the fight was over, the guy on the floor is reciting those lines.
And the other person takes off her helmet.
And of course to make it dramatic, her big hair fluffs out all over the place.
The beginning of Midsummer Night's Dream is the set up -- the set up is that there's going to be a wedding between Theseus, Duke of Athens, and Hippolyta, the Queen of the Amazons, who he has conquered and is now going to marry.
But the notion of "I wooed thee with my sword and won thy love doing thee injuries" does not sit well in, at least not in a Cambridge academic kind of setting.
And so the A.R.T.
played this against type, right, with Theseus on the floor saying, "I wooed you with my -- oh, get that away from my neck."
But it's an interesting, curious thought that you might woo somebody with your sword and win their love doing them injuries.
I suspect there aren't very many women present who would subscribe to the notion that that would be a marvelous form of courtship, and maybe not even an awful lot of guys.
But forms of relations that become coercive are by no means unheard of.
And the place we typically end up hearing about them is when they end up in court.
How do they end up in court?
Well, let me tell you a court case.
This happened about ten years ago now.
This is a court case where a woman is suing a collection of -- she's suing them?
No, I think this was actually a rape case in the criminal court.
They're charged with rape and she's the plaintiff.
What happened?
Well, considerable disagreement about what happened when you end up in court.
But what basically seems to have happened is that the woman and one of the guys met at, I think it was rifle practice, some sort of athletic practice.
He invited her back to his residence.
Alcohol followed and she ended up having some variety of sexual relations with like six different guys and subsequently charged them with rape.
She argued that she had been passing in and out of consciousness and that they had basically abused her.
They argued that this was consensual.
I think what happened in this particular case, by the way, is that there was no conviction for rape but several of the guys ended up being expelled from school.
There are lots of things one could talk about here.
I'm not in the business for a psych course of giving a sort of an RO week lecture about, you know, good behavior or something like that.
But the interesting issue -- well, there are lots of interesting psych issues.
The interesting psych issue that I want to focus on is the question of how this could come to pass given that probably nobody wanted it to come to pass.
It is extremely unlikely that she went to whatever it was, rifle practice, saying, oh gee, I think I'll go home with this guy, get smashed out of my mind, and have sex with all of his roommates.
Doesn't seem likely.
Nor does it seem very likely that he said to himself, oh I think I'll take this friend of mine home and we'll all have too much to drink and she'll have sex with everybody under the sun.
So it's pretty clear that this is not what anybody particularly had in mind.
How does it come to pass?
Oh, look at that.
I must have said that on the handout.
What am I talking about next?
Lots of questions, most of which we cannot address.
All right.
So let's address some of the ones we can address.
As it says there, very gender-specific problem.
There are certainly instances of sexual coercion going the other -- female- on- male sexual coercion.
And within homosexual relationships there are certainly instances of sexual coercion.
But the great bulk of these cases are male- on- female coercion.
In fact, the female- on- male coercion incidents is vastly higher in the erotic literature than it is in reality apparently.
Well, remember the example from earlier on about imagine that you're on the subway and some member of the sex that you find attractive begins to touch you surreptitiously on a crowded subway car.
Is this a good thing or a bad thing?
Women uniformly say this is not good, and guys, well, not quite uniformly, say mmm, OK. You can get similar data with coercive relations.
You know, the leather-bound woman who shows up and says, "You're having sex with me right now" is a fantasy figure.
The leather-bound guy who comes up to a woman and says, "You're having sex with me right now" is not typically a fantasy figure.
So there's this odd asymmetry there.
So there are lots of ways to understand how this could come to pass.
One of the reasons, as I said earlier, for making this an end of the term kind of topic is it does sort of neatly recapitulate an awful lot of the themes about causality that have shown up in the course of the term.
And then what I'll probably, oh, very clearly do next time is weave a story that combines many different threads into one.
Let me just say a word about incidence today, and then we'll go on to etiology when we pick up next time.
How common is this as a problem?
Well, you've got a real problem here, which I have mentioned before, which is the data on sexual behavior are lousy.
Because people don't -- well, first of all we don't collect that much of it.
And second of all people lie.
Recall that if you ask males how many sexual partners have you had and you ask females how many sexual partners have you had, you discover that there's a third sex out there somewhere.
Because the math doesn't add up.
You see that too in -- well, maybe you see that in the data on -- that's gotten basically from survey data -- on coercive sex.
If you ask females, "have you ever been coerced into sexual intercourse?
", in at least one study the answer is 15% of women say yes.
If you ask men, "have you coerced anybody into having sexual intercourse?
", only 7% of males say yes.
Now there are a number of possible explanations, like serial coercers or things of that sort, but you can see that there may be reporting issues that are difficult there.
One last point on this, on the incidence point, if you ask have you ever -- 25% of women in one study back from the mid '80s reported intercourse because quote "they were overwhelmed by a man's continual arguments and pressure."
That's an interesting statistic in a couple of ways.
First of all it's interesting because it's quite high.
Second of all it's interesting because it points out the very distinct status of sexual relationships.
If I am coerced into buying a used car by the man's continual arguments and pressure, that's too bad, right.
If I decide that I'm going to strangle -- who am I going to strangle?
Oh, I don't know.
I won't strangle anybody particular.
Oh, maybe I'll strangle Kristin because she was laughing at me earlier.
I'll strangle Kristin because I was pressured by Anna's continuous verbal demands or something like that to do this.
You know, I can make the case that it was her demands that made me do this, but I'm still going to jail.
It doesn't explain away my behavior.
Sexual relations are interestingly different from other relations in that way.
And, OK, we'll pick up on, at our last meeting, on how this all comes to pass.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
There's no new handout today.
We're working off the old handout.
If you don't have the old handout you'll probably survive without it.
The last thing to say before I say something is that between this morning and this afternoon I changed the end of the lecture.
I decided I had a new idea and we will discover whether it turns out to be a new idea when I hear it coming out of my mouth.
But if it slightly confuses the handout I'm sorry, it's the product of actual thought.
All right, it is my belief, somebody remind me if I'm wrong that we had gotten to the incidence part of this story about coercive sexual behavior and we had not gotten to the etyology part.
Does that sound about right to people?
Yep, OK. Let me see then.
What I want to do is to talk about some risk factors then and rather like the eating disorders story, risk factors are risk factors.
They're not by themselves causative.
And again, most of this is coming off of survey data.
So for example, one of the things that turns out to be a risk factor for some sort of coercive relations taking place is who pays for dinner.
And this is not because guys who pay for dinner are rapists, it's because some small fraction of folks who pay for dinner somehow think they paid for something else perhaps.
It's not clear what the actual causality is, but if you go and take some giant body of data and you ask, what's correlated with something that looks like date rape as the end product?
One of the correlated factors turns out to be things like, who pays?
A similarly, not terribly powerful correlation is found with whose car it is or whose room it is or things like-- these are issues of control.
That it's more likely that again, as sort of a weak correlative bit of data, his car is more dangerous than her car from that point.
Does it mean that everybody with a car is a rapist?
No of coure not, doesn't mean anything of the sort.
But there's a correlation there.
These are smallish effects.
If you want a big effect go for alcohol.
If you read this literature-- this somewhat depressing literature-- but if you read this literature you find that at least a majority of cases reported seemed to involve consumption of alcohol.
Generally, fairly large amounts of alcohol.
In several of the studies it's anywhere between 50% and 75% of cases involve consumption of alcohol.
This is alarming if it's taken along with the rise in binge drinking on college campuses.
That's something that's risen over the last twenty years and if you assume that a big risk factor for coercive sexual relations is alcohol it's not going to be a good thing if there's a lot of binge drinking around.
Some of this appears to be, in a sense, deliberate where some guys pressure some women to drink too much in the interests of promoting sexual activity of some variety.
But mostly, this is a case of what Floyd Alexander-- is it Floyd or Lloyd?
I can't remember.
It says on the handout, right?
Floyd
Floyd.
Where'd he go?
Franz.
Well, it's neither Floyd nor Lloyd.
No, Lloyd Alexander-- that's a lovely blend.
Lloyd Alexander is an author of young adult fantasy novels, which some of you may have read.
Franz Alexander is the American Freudian.
So if you put Franz and Lloyd together apparently you get Floyd Alexander, who's nobody.
But Franz Alexander, this early American Freudian, had the felicitous line that the superego is soluble in alcohol.
We might be more inclined today to look at the neuroimaging data that says "Oh look, here's this chunk of the frontal lobe.
It seems to monitor errors.
Oh look, here's this chunk of the frontal lobe on alcohol.
Wow, it's really suppressed."
Alcohol has an inhibitory effect on the nervous system.
It has a particularly strong inhibitory effect on the chunk of the brain that, to the best of our knowledge, is there for doing things like error monitoring.
If your error monitoring device is off all sorts of stupid things happen.
If you're error monitoring is off in the context of sexual activity all sorts of stupid sexual activity sorts of things happen.
That looks to be the most likely path for the effects of alcohol.
It's simply makes people do stupid things.
Both the man and the woman.
It's an equal opportunity stupidifier.
While it is an equal opportunity stupidifier, suppose that she subsequently thinks or-- well, let's suppose this ends up in court.
She charges him with date rape in some fashion or other.
If everybody's drunk, what's the chance that there's a conviction?
Grr, That's about right.
You don't end up getting a conviction typically if there's a lot of alcohol involved and if everybody's drunk, blame tends to migrate to the woman in these sorts of cases.
I'm not arguing that that is the right thing or anything, it's just an interesting factoid about responsibility.
It turns out, at least in legal settings, to be very difficult to get a conviction for any of this if everybody is apparently drunk.
These days in a dubious sort of progress we can improve on alcohol with so called date rape drugs.
I put the name of the date rape drug, I mean, one gets sort of labeled that on the handout.
But there's a whole class of these out there.
The one on the handout in particular is part of a family of tranquilizers that are like Valium, basically.
Their effects are to produce disinhibition, relaxation of voluntary muscles, and great amnesia.
They produce an anterograde amnesia.
Review time.
Anterograde amnesia-- that's amnesia from after the point of the insult.
So in this case, getting hit over the head is taking the drug so you have an amnesia for events subsequent to taking the drug.
So you're disinhibited, stuff happens, and you don't remember.
Here there are repeated reports of cases of drinks being spiked with drugs and things of that sort.
So there is more of a sense that these may be being used in a deliberate sort of a sense.
The alcohol by the way, potentially aids the effects of these sort of tranquilizers.
So a mixture of alcohol and one of these drugs is a really potent way to end up doing things that you don't have any intention of doing and would not to do were all of your neurons functioning in a halfway reasonable fashion.
Let's see.
Other risk factors that I put on the handout.
What do you do with time a is a risk factor for time b it says on the handout.
That just says what your nervous aunt used to say or something like that.
If you're holding hands with somebody the next thing you know you're hugging them.
And if you're hugging them the next thing you know-- so each progressive act increases the likelihood of the next act.
Again, that's just the sort of thing that you're going to pull out of correlative data.
The other biggie in risk factors is miscommunication.
Did anybody talk to anybody in the process of this interaction?
Just as you get in report after report you get reports of alcohol use, you also hear a recurring litany-- she said, he said sorts of reports.
She thought she said this, he thought he heard that.
So there's some variety of miscommunication there.
This raises the issue of consent.
That the notion is-- the foundational notion is that you want all sexual relations to be consenual in some fashion.
Most people will sign onto that notion in a pretty straightforward kind of manner.
But exactly what consent means is rather difficult to make concrete.
If you're in a laboratory situation consent is spectacularly concrete.
You have to fill out forms.
But nobody fills out forms in sexual relations.
Every now and then you get some well- meaning-- Antioch College had a behavior code that boiled down to a consent form for interpersonal relations.
But nobody really runs their life that way.
Hi, I think our relationship has gotten to a certain point, would you please sign the following sixteen page form?
It doesn't quite ring true.
Though in fact, it might be a marvel-- most of these efforts sound a little like softcore porn for the academic class.
You know, long, written out forms that describe in detail, who's going to do what to whose body?
It might be a kind of an interesting exercise, but suffice it to say that communication in intimate relationships doesn't look like that.
And the possibilities are ripe for miscommunication and when you ask in survey data about relationships that became coercive in some fashion, what you find out is that there often seems to be a core chunk of miscommunication.
So this gives us some sort of an epidemiological look at the problem.
It doesn't have any explanatory power in itself.
It just says that look, there are these things that are risk factors.
It's more likely that if you're drunk in the back of his car that something's going to happen.
How can we understand this from a more theoretical point of view?
Well, that's why it says theories on the handout, I see.
Again, one of the reasons we're talking about this is that gives me a lovely chance to go back and review the entire course.
The theories run the usual run that you should be extremely familiar with at this point.
Well, since we're in the chunk of course on abnormal psychology there are psychopathological theories that say that the guys who end up accused of date rape are sick.
That they are mentally ill in some fashion.
That is not a bad theory for what's known as blitz rape.
For the stranger in the bushes kind of rape.
Those guys, and again, it's overwhelmingly males who commit stranger-- well, blitz rape is the term for some stranger grabs a person and rapes them.
Those guys, those rapists when they are studied don't look psychologically normal, typically, as a population.
They look deviant.
However, there's nothing in these date rape population that looks like that.
The standard description in a court case, I think I mentioned is, oh, just a regular guy.
Just the guy next door and that really seems to be the case.
There's some evidence for instance, that they might drink more than the population as a whole, but that's not telling you very much.
So if it's not an individual pathology there are a class of theories out there that suggest that it's a gender pathology.
That it's a problem of guys in general.
The name, at least that I most closely associate with this is, Thornhill or Thornhill and Thornhill since there appear to be two of them from the reference on the handout.
I think only one of them wrote the book, two of them wrote the article.
There's an MIT Press book that made quite a bit of a stir in the media-- oh, must be five years ago now or so-- that addressed the argument, the evolutionary psych argument that rape is adaptive.
Again, to explain is not to condone.
This is not that Thornhill was busy saying it's adaptive, so it's good.
They're saying that it's got an evolutionary, adaptive function to it and it doesn't take a genius to figure this out if you were following the earlier arguments.
If a male's goal is well, let's go propagate our genes-- and because he doesn't get pregnant and she does there's this asymmetry.
She wants somebody who's going to have commitment to the whole project.
He just wants to impregnate women and get those genes into the next generation.
Taken to the extreme, somebody who goes out and just forcibly goes and impregnates people has a chance of getting more of their genes into the next generation and maybe that's adaptive.
One can argue that there is no specific adaptation toward sexual violence, but that it is the combination of a general-- males are on average, more aggressive than females.
There's lots of data for this.
At the start of the women's movement there was a fairly strenuous effort to argue that the differences in aggression between males and females were all just driven by culture.
That little boys were trained to be soldiers and little girls were trained-- they were playing with the dolls and the guys were playing with the guns and so in the 70s, 80s that thing you did if you were a parent of feminist sort of inclinations is you made sure that your little girl had guns and your little boy had dolls and the experience over the course of, I suppose, the generation or so now is that if you're a little boy you can make a gun out of Barbie too, if you hold her just right she looks like a submachine gun.
To recapitulate a large literature in a line or so there is plenty of evidence at this point that human males are bigger and more aggressive than human females.
Not true across the entire animal kingdom.
I don't think I've told you about the octopus.
The greatest case of sexual asymmetry in the animal kingdom as far as I know is some seagoing octopus that I think was actually only discovered as a species within the last decade or so.
But this is an octopus where the male is small enough that he could fit into her eye.
So this is an asymmetrical species.
It's not clear-- and his goal in life-- he's a little octopus harpoon.
And his goal in life is to impregnate one of these gigantic [UNINTELLIGIBLE] octopuses swimming around.
It's abundantly unclear that she knows what's happened at all, right?
[UNINTELLIGIBLE PHRASE] Oh look, we've got babies or something.
I don't know.
Do octopus bear live young or not?
I don't remember.
Anyway, very dramatic asymmetry running the other direction.
There are plenty of species where the female is the more aggressive species-- did I tell you about spiders?
Yeah, I told you about spiders.
Praying mantises are good too.
Praying mantis sexual relations apparently involve her biting off his head, which apparently doesn't impair his functions as a mate at all.
One of the great advantages of a distributed nervous system, the head is overrated in praying mantises apparently.
Everything is out there, but in humans it's clear that males are more aggressive.
Between being more aggressive and being inclined to have more sexual partners again, probably because of evolutionary forces.
You could imagine that rather than having sexual aggression as a specific adaptation it ends up being sort of a byproduct of other facts that occur.
The notion that rape is sort of a male pathology, a gender wide pathology doesn't only show up in biological theories it also shows up in very sociological theories of sexual behavior.
Notably, in feminist theories.
One version of these would be the theory that-- this I'm borrowing from a book by [?
Sandais ?]
that rape is the means by which men are programmed for violence.
A way to induct younger men into masculine roles.
That it's somehow a training ground.
She argues that in cultures where everybody sort of lives at harmony with nature and the mother child bond is sacred and things like that, that this doesn't happen.
That rape happens in cultures where you need guys to go out and deal with the dangerous world.
They gotta go out and do in the bears and the lions or something and that rape is in some sense, a training ground for that.
It's an odd theory on two grounds actually, this one.
One reason is it's desperately unclear that there are any of these lovely pastoral societies out there.
We used to think people like Margaret Mead had told us that down in the South Seas there were-- in Samoa-- there were people without sexual hang-ups, everybody got along marvelously and it was all lovely.
The data don't look so good when you go back and look.
Subsequently, she saw a little bit of what she wanted to see and if you go and look at-- when you go and look at these nice simple cultures-- you get these reports from time to time of hunter-gatherer cultures that live in harmony with nature and stuff like that.
When you actually go look at it it turns out they murder and rape each other at exactly the sort of rates that we murder and rape each other.
There is no Garden of Eden out there, at least not anymore where everybody's living in beautiful harmony.
Another problem with the theory that it's got a general role is that the culture as a whole, at least explicitly, says that rape is illegal and says that rape is bad.
That makes it hard to argue that the entire male gender is in on some plan to use this as a training ground for going out and killing the bears or something of that sort.
A much more interesting, it seems to me, feminist account of what's going on is that rape is a giant protection racket run by males.
If you read feminist tracts on this I think they write as though there's some group of guys sitting around somehow deciding explicitly that this is the way things are.
But the theory makes more sense if you think of it implicitly.
Rather like if you think about it in terms of the cuckoo story.
Remember cuckoos?
Cuckoos are going and cheating and laying eggs in other bird's nests.
But you can't have too many cuckoos.
The argument runs-- because if you have too many cuckoos everybody dies, right?
The argument runs in a similar kind of way in this argument.
You can't have all males being rapists.
That just isn't a recipe for civilization.
But if the society condones or spits up or creates a cadre of rapists then the rest of the guys can take on the role of protecting women from those bad guys.
The argument is that rape is condoned in the society, perhaps implicitly rather than explicitly.
I've got a useful, normal curve here.
You know, we'll have a few guys out here who are bad and we, the rest of us, us good guys can now protect women from those bad guys.
Well, what's that going to mean?
Well, it might mean gee, you shouldn't go out much, for instance.
You know, you should stay at home.
Or you should not be seen by men, you should not wear this, you should not do this.
In any case, feminist argument argues that it can be a tool of control over women even if you only have a very small minority of males engaging in the activity.
That strikes me as actually a fairly clever argument.
I don't know if it's true, but it's an interesting argument.
The feminist arguments often point towards pornography as a culprit here, as a training ground in the sense that if you took the earlier notion that the mind gets to pick its abnormality-- gets to pick how to go insane if you like, from the menu that's provided by the broader culture.
So currently, if you want to have a dissociative disorder the way to have it is a multiple personality disorder.
Couple hundred years ago it might have been to be possessed by a demon.
These are the scripts if you like provided by the culture.
Feminist writers argue that pornography and notably, violent pornography provides the script for coercive sex.
A number of feminist legal scholars have worked hard-- Catherine MacKinnon is the most famous name here-- have worked hard to make at least some classes of pornography illegal on grounds that they are actually dangerous to women.
That it's not a free speech issue, that it's really a health and safety sort of issue.
So you've got levels of explanation that occur at psychological, biological, sociological levels.
Well, let me give you a version that's perhaps a version that would fit the general ethos of this course, which is that the answer is never-- it's all biology or it's all sociology or something like that.
It's some curious interaction that's taking place.
Well, what's going on here?
What occurred to me between this morning's version of this lecture and concourse and this afternoon's is that the job may be less to explain coercive sexual relations, which you can see as-- like everything else, sexual relations are going to be distributed along some normal curve.
It's going to have multiple dimensions but this one could be on a scale from extremely consenual to extremely not consenual.
I suppose the extremely consenual abnormality one is the one where people really do sign forms or something like that and talk to each other in great detail about what's going to happen next.
But the job may be not just to explain or to explain this the extreme point on what's after all going to be a normal function, but to explain how the overall behavior gets put in place all together.
It's one of those behaviors where high school health class not withstanding, nobody gives you much in the way of lessons and the lessons that you get are stupid lessons that you get from listening to your peers lie their heads off in the locker room.
Where is this behavior coming from?
People are learning this behavior in some fashion, we're not after all, praying mantises where the whole business is just wired into some ganglion in your spinal cord and if you take off the head everything works just fine anyway.
So where is this behavior coming from?
The pieces, fortunately, for the handout-- I'm not going to depart wildly from the handout-- the pieces strike me as similar to the pieces I was going to use anyway.
One important piece and it's back to this idea that you get to choose your disorder or just your behavior in some fashion from the set of behaviors that your culture presents to you.
Part of that is what we can consider to be the power of narrative thought in these sorts of behaviors.
What's narrative thought?
To review, I think.
So propositional thought is moving symbols around and the sorts of things you might do in problem set land.
Narrative thought is storytelling.
You got the stuff?
She's got the stuff.
It's good.
This is the evaluation forms for later.
So narrative thought is about telling stories to yourself.
And it's about telling yourself who you are.
If you ask, why did that happen?
You know, you don't generally sit down and start writing equations about it.
You run through stories in your head.
If you ask, what's going to happen next?
You very much start running stories in your head.
If I ask her out, what's going to happen?
That's sort of narrative thought.
Well, where did the narratives come from?
In the case of relations between the sexes-- well, let me give you a cartoon version of an answer to that.
But I think it is illustrative.
If you go to the romance aisle, I mean, if you want to understand the narrative about romance you might as well go to the aisle in Borders that says romance on it.
I mean, if you go to the aisle that says engineering you can find something different.
So you go to the romance aisle and there's rack after rack of books.
Conveniently enough I don't know what's in these books, by and large, because it's not a genre I read extensively, but I look at the covers and the covers seem pretty well illustrative as it were.
What's on the cover of your typical romance novel?
All right, we'll start with the basics.
How many people are there on the cover of--
Twp
Two.
That's a good number.
OK.
I'm sure somewhere that there is a genre of same sex romance novels, but they're not typically what's on the shelf at Borders or Barnes & Noble or something.
There what you've got is you got two people.
One of them is a male and one of them is a female.
I will assert that in case after case it's the same picture more or less.
Drawn in some gaudy kind of fashion, but the picture looks like this.
Right?
Yeah?
OK, so we got two people.
We got this picture.
Do we know which one is the guy?
Yes
So one person-- there's this guy.
He's the guy-- the pirate, the sailor, sometimes the doctor or something like that.
You know, I don't think I've ever seen one where it was the college professor.
Doesn't seem to be on the list at all.
Then there's the woman who's having clothing issues typically.
And she's in this sort of supine position.
This is telling you something.
By the way, I was doing research on this-- I kid you not-- I thought I'd better check whether it's still true.
You know, it could that in this latter day, I mean, if you ask for instance-- actually, I should ask again because these things keep shifting.
The standard model when I was, well, when I was in high school whatever the standard model was I was ignoring it and was completely clueless, but in any case, the standard model appeared to be that the male person asked the female person out.
Sometime that sort of disappeared, near as I can tell and nobody asked anybody out and people just sort of-- it was kind of like a chemical thing.
Numbers of women and men were in the same place and by some mysterious technique they bonded.
Let's see.
Since all the data here are survey data, let's get some survey data.
Is it typical for males to ask out females?
How many think that that's typical?
How may think it's typical for females to ask out males?
Oh, isn't that interesting?
See that's interesting because that's not the answer that I would have gotten from-- I haven't asked that question reliably every year, but it's not the answer I would have gotten ten years ago.
Ten years ago the local custom had changed so that everybody was asking everybody or nobody was asking nobody, but there wasn't this sort of notion that males specifically were taking the lead and asking females.
And so in a sense, it sounds like it's reverted to the pattern that existed when-- well, I won't say how many years ago, but it was a few years ago.
Anyway, so I got to check whether or not the romance novels still look like this.
So I went to Borders and went down to the romance aisle and I discovered this cool new thing.
The cool new thing, at least it's new to me, is that the romance literature, these sort of series books have become explicitly stratified.
That there are now multiple series of these things.
They're color coded.
It looks like homeland security, it's great because it runs from I think sort of a blue/green line, which has a title something like family values and then it runs over to this red line here-- I can't remember what the title of this is.
This is all under the sort of general heading of Harlequin romance or one of those big romance publishers, but it's got some word like torrid in it or something like that.
But interestingly, the only difference in covers that I could detect on a brief survey is in the family values thing both parties are still more or less upright, but he's sort of leaning in.
By the time you get to the torrid one she's, boom.
Anyway, the principle is the same.
It is not the case that-- there just are no books out there, there's probably one out there somewhere.
The central tendency of the distribution does not have a lot of female pirates overwhelming mild- mannered accountants or something like that.
It's just not there.
I'm only using this as an example of a narrative that I think is very widely out there.
You can pick it up in movies, you can pick it up on TV and so on.
Importantly, it is not a narrative with anything like rape written at the top of it.
It's a narrative with romance or seduction or something written at the top of it.
And the promise in the vast bulk of cases that the ending is going to come out positive.
I'd love to know what they just found on the web over there
We're looking up gay romance novels
You're looking up gay romance novels, OK.
Thank you for being diligent in your research there.
Well, you can tell me who's-- anyway, nevermind
There's no picture
They don't have pictures.
Meantime there's several people trying to figure out on their cellphones what's there.
Anyway, so the script out there, the sort of script that people are operating from is a script where-- well, it's, urk.
That sort of script.
Again, importantly not with the idea that it's not going to be consenual or whatever, but the notion here is that it's going to be in effect, the male who's doing the asking and the female who is giving or not giving consent in some fashion.
This is presumably tied into evolutionary psych roots that we've talked about already.
I think I mentioned earlier that there's evidence that males experience the falling in love part of falling in love more rapidly in a relationship than females.
They certainly want more sexual partners and sex more often by all sorts of measures.
These sort of pressures are likely to produce a situation in which it is the guy who's asking and the woman who is either giving or not giving consent.
Now if it is the case that the male is falling in love more rapidly or falling in lust more rapidly-- it kind of doesn't matter-- it follows that the first answer's going to be no.
Typically, there are going to be some no's.
If this is this script that the conversation is going to involve a bunch of no's.
Now we should switch gears a little and do a little reviewing of learning theory sorts of things.
I think it says something on the handout like the power of conditioning.
But let's think a little bit about learning theory.
Well, let's think about law of effect kind of stuff for starters.
The drive to reproduce is a strong one.
There are strong positive reinforcers associated with sex.
Nobody spends a lot of time-- I mean, look, if you think about the amount of-- I don't need to explain that.
You knew that already.
What Thorndike's law of effect tells you is that any of these sort of yes activities here, at any level in the relationship are going to be positively reinforcing.
Are going to tend to reinforce whatever happened beforehand.
And so one of the things that people are doing and one of the things that they don't appreciate that they're doing even if they're signing fifteen page consent forms, which they're not-- is that people are shaping each other.
The activities that couples are engaging in with each other are-- it's your own private Skinner box.
And you're busy shaping that pigeon who is the other half of this relationship.
The great difference of course is you're both the experimenter and the experimental animal in your study here.
But let's just think from the male's point of view.
So the male is hearing a lot of no's.
If the relationship is eventually going to progress to let's have fifteen children then eventually there will be some variety of yes in here.
That sounds like, what kind of schedule of reinforcement?
[UNINTELLIGIBLE]
Broomph.
No, somebody needs to raise their hand and nobody's going to raise their hand.
They've all gotten shy.
You scratch your head you're going to end up getting called on there.
So the answer is this is a partial reinforcement schedule or more like a variable ratio schedule.
It's not a variable interval schedule unless you have a very weird relationship.
No.
No.
No.
Monday, yes.
No.
No.
No.
Monday, yes.
That's a weird relationship.
But more likely it's no.
No.
No.
Yes.
Maybe.
No.
Yes.
Yes.
No.
No.
No.
No.
No.
Anyway, what do we know about those sorts of schedules?
What do they produce in the way of behavior?
High rates-- if this all sounds completely new to you guys you've got work to do between now and next Thursday, let me tell you.
But these sort of schedules of reinforcement if it's in a rat trying to get fed produced very high rates of bar pressing.
If it's in a guy, it presumably produces very high rates of bar pressing, too.
So he's going to emit a lot of behavior there.
And it's also very hard to extinguish.
So if you decide at some point to say, no forever it may be a long time before the guy gets the hint.
At least the pigeon aspect of the guy.
One other thing tying into this-- we ought to say one more thing based on the sort of romance novel example, which is that the dialog doesn't tend to be great in those things.
Guys doing manly things, pounding on their chests and females swooning or something like that.
But there's just not a lot of conversation.
So in the absence you're this giant committee-- another theme from the whole course-- your brain, your mind is this committee of semi-independent operators.
Well, there's one operator there that's doing the verbal consent thing and thinking deep thoughts, but it's in a script that does not have much in the way of good dialog in it.
That leaves this other chunk of your brain.
The chunk that has an awful lot in common with rats pressing bars for food in Skinner boxes.
A lot of realm to govern behavior.
If you're not going to govern behavior with what you might want to call higher cognitive powers, well, these nice associative learning mechanisms will do a perfectly fine job for you.
Hmm, got reinforced.
Let's do that again or something of that sort.
So you end up with a situation where you've got multiple forces acting on you.
And you've systematically disabled some of the more intellectual parts of you.
It reminds me, how many people have been to the Science Museum?
Another place you should go sometimes if you haven't been.
I think you get in free, right?
Do you get in free?
Yep
Yep.
Get in free, go to the science museum.
What you'll discover if you go to the-- I don't think, I don't know if you'll still discover it-- remember my spiral?
Somebody from 900 went and was a volunteer at the Science Museum and said, oh you got to build this cool-- they still have a great motion after effect spiral, which I believe is patterned on having seen it in this class years ago.
And then when they would do the little show they were using my lines.
I didn't even get residuals.
But one of the demos they have there in the math section is this wonderful thing with an endless collection of pins arranged into sort of triangular way-- one, two, three-- is that Pascal's triangle?
Yeah
Yeah, yeah, yeah.
Anyway, it's a version of that.
Anyway, the cool version of the demo is that you drop a ball at the top and it bounces from pin to pin and in effect at each pin it's got a fifty-fifty crack of going one way or the other.
And if you have enough layers of pins what you end up with is essentially-- well, is exactly in the limit, a nice normal distribution.
From a ball that just completely neutral, dropped at the top, most of the time it's going to end up here, some time it's going to end up at one extreme or the other.
I suspect that this is not a bad model for a wide range of behaviors.
I'll argue in closing that it's a good model-- that it's at least, a model worth thinking about for something like coercive sexual behavior.
That we know or at least we think we know that the population of guys engaging in that behavior are not sitting out here in some separate distribution.
They seem to be the distribution of regular guys that just ended up in an odd part of the distribution.
And it may well be that this collection-- if you want you can think of it in terms of shaping.
That the collection of positive reinforcers that they got just balanced randomly off into this direction and produced what boils down to a very maladaptive behavior where the same guy starting from the same point might well have ended up in-- would most guys starting up from the same point would've ended up in the same, in this vast bulk of the normal end of the distribution or the middle of the distribution.
That you don't need to posit that there's anything specifically wrong and that you might have a very difficult time, the perhaps disturbing piece of this if you were trying to set public policy is that you might have a very difficult time trying to say, gee, what can we do that would prevent this tale from happening?
Because it might really be the result of a succession of random events.
One thing that you might guess is that the degree to which you let cognition back into the game and let it run less on the rat pushing the bar kind of thought, the less chance that you have of ending up in some maladaptive corner of this particular distribution, but that's not desperately clear.
Can I prove this?
No.
And so does that mean that the right answer on the final will be where does coercive sex come from?
Apparently it comes from bouncing down the pin.
No.
This is clearly marked as Wolf's theory and not only that it's Wolf theory since this morning.
I may decide by this evening that it was really dumb and feel deeply humiliated that I ever got up and said it in public, but at the moment I don't.
But what I do feel is that what I want to do-- well, I want to do a couple of things.
Thing one is to say, thank you for listening all term.
It's kind of a privilege to be able to yack for a whole term and have people listen.
It doesn't happen at home much.
The second, there are two different forms floating around.
Well actually they're not--
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's go ahead and get started.
I hope everybody saw the correction to a typo in homework 1 that was posted on Stellar last night and sent out to you.
That's going to happen from time to time.
We have four course staff that review all the problems.
We try to look through it for any issues or ambiguities.
But from time to time, we'll miss something and try to make a correction.
The TAs gave a hint that would have let you solve the problem as written.
But that's more difficult than what we had intended for you guys.
We don't want to give you a homework assignment that's punishing.
We want to give you an assignment that'll help you learn.
Some people in this class that are very good at programming have apparently already completed that problem with the hint.
But it's easier, as originally intended.
And the correction resets that.
So maybe you'll see the distinction between those things and understand why one version of the problem is much easier than another.
But we try to respond as quickly as possible when we notice a typo like that so that we can set you guys on the right course.
So we've got two lectures left discussing linear algebra before we move on to other topics.
We're still going to talk about transformations of matrices.
We looked at one type of transformation we could utilize for solving systems of equations.
Today, we'll look at another one, the eigenvalue decomposition.
And on Monday, we'll look at another one called the singular value decomposition.
Before jumping right in, I want to take a minute and see if there are any questions that I can answer, anything that's been unclear so far that I can try to reemphasize or focus on for you.
I was told the office hours are really well-attended.
So hopefully, you're getting an opportunity to ask any pressing questions during the office hours or you're meeting with the instructors after class to ask anything that was unclear.
We want to make sure that we're answering those questions in a timely fashion.
This course moves at a pretty quick pace.
We don't want anyone to get left behind.
Speaking of getting left behind, we ran out of time a little bit at the end of lecture on Wednesday.
That's OK.
There were a lot of good questions that came up during class.
And one topic that we didn't get to discuss is formal systems for doing reordering in systems of equations.
We saw that reordering is important.
In fact, it's essential for solving certain problems via Gaussian elimination.
You won't be able to solve them.
Either you'll incur a large numerical error because you didn't do pivoting-- you'd like to do pivoting in order to minimize the numerical error-- or you need to reorder in order to minimize fill-in.
As an example, I've solved a research problem where there was something like 40 million equations and unknowns, a system of partial differential equations.
And if you reorder those equations, then you can solve via Gaussian elimination pretty readily.
But if you don't, well-- my PC had-- I don't know-- like, 192 gigabytes of RAM.
The elimination on that matrix will fill the memory of that PC up in 20 minutes.
And you'll be stuck.
It won't proceed after that.
So it's the difference between getting a solution and writing a publication about the research problem you're interested in and not.
So how do you do reordering?
Well, we use a process called permutation.
There's a certain class of matrix called a permutation matrix that can-- its action, multiplying another matrix, can swap rows or columns.
And here's an example of a permutation matrix whose intention is to swap row 1 and 2 of a matrix.
So here, it looks like identity, except rather than having 1, 1 on the first two elements of the diagonal, I have 0, 1 and 1, 0.
Here's an example where I take that sort of a matrix, which should swap rows 1 and 2, and I multiply it by a vector.
If you do this matrix vector multiplication, you'll see initially, the vector was x1, x2, x3.
But the product will be x2, x1, x3.
It swapped two rows in that vector.
Of course, a vector is just a matrix, right?
It's an N by 1 matrix.
So P times A is the same as a matrix whose columns are P times each of the columns of A.
That's what this notation indicates here.
And we know that P times a vector, which is the column from A, will swap two rows in A, right?
So the product here will be all the rows of A, the different rows of AA superscript R, with row 1 and 2 swapped with each other.
So permutation, multiplication by the special type of matrix, a permutation matrix, does reordering of rows.
If I want to swap columns, I multiply my matrix from the right, IP transpose.
So if I want to swap column 1 and 2, I multiply A from the right by P transpose.
How can I show that that swaps columns?
Well, A times P transpose is the same as P times A transpose transpose.
P swaps rows.
So it's swapping rows of A transpose, which is like swapping columns of A.
So we had some identities associated with matrix-matrix multiplication and their transposes.
And you can use that to work out how this permutation matrix will swap columns instead of rows if I multiply from the right instead of the left.
Here's an important concept to know.
Permutation matrices are-- would refer to as unitary matrices.
They're transposed.
It's also they're inverse.
So P times P transpose is identity.
If I swap the rows and then I swap them back, I get back what I had before.
So there are lots of matrices that have this property that they're unitary.
We'll see some today.
But permutation matrices are one class, maybe the simplest class, of unitary matrices.
They're just doing row or column swaps, right?
That's their job.
And so if I have some reordering of the equations or rows of my system of equations that I want, that's going to be indicated by a permutation matrix-- say, P1.
And I would multiply my entire system of-- both sides of my system of equations by P1.
That would reorder the rows.
If I have some reordering of the columns or the unknowns in my problem, I would use a similar permutation matrix, P2.
Of course, P2 transpose times P2 is identity.
So this product here does nothing to the system of equations.
It just swaps the unknown.
So there's a formal system for doing this sort of swapping.
There are a couple other slides that are in your notes from last time that you can look at and I'm happy to answer questions on.
We don't have time to go into detail.
It discusses the actual methodology, the simplest possible methodology, for doing this kind of reordering or swapping.
So this is a form of preconditioning.
If it's preconditioning for pivoting, it's designed to minimize numerical error.
If it's preconditioning in order to minimize fill-in instead, that's meant to make the problem solvable on your computer.
But it's a form of preconditioning a system of equations.
And we discussed preconditioning before.
So now we know how to solve systems of equations.
It's always done via Gaussian elimination if we want an exact solution.
There are lots of variants on Gaussian elimination that we can utilize.
You're studying one of them in your homework assignment now, where you know the matrix is banded with some bandwidth.
So you don't do elimination on an entire full matrix.
You do it on a sparse matrix whose structure you understand.
We discussed sparse matrices and a little bit about reordering and now permutation.
I feel like my diffusion example last time wasn't especially clear.
So let me give you a different example of diffusion.
You guys know Plinko?
Have you seen The Price Is Right?
This is a game where you drop a chip into a board with pegs in it.
It's a model of diffusion.
The Plinko chip falls from level to level.
It hits a peg.
And it can go left or it can go right with equal probability.
So the Plinko chip diffuses as it falls down.
This guy's excited.
[LAUGHTER] He just won $10,000.
[LAUGHTER] There's a sparse matrix that describes how the probability of finding the Plinko chip in a certain cell evolves from level to level.
It works the same way the cellular automata model I showed you last time works.
If the chip is in a particular cell, then at the next level, there's a 50/50 chance that I'll go to the left or I'll go to the right.
It looks like this, right?
If the chip is here, there's a 50/50 chance I'll go here or I'll go there.
So if the probability was 1 that I was in this cell, then at the next level, it'll be half and a half.
And at the next level, those halves will split again.
So the probability that I'm in a particular cell at level i is this Pi.
And the probability that I'm in a particular cell level i plus 1 is this Pi plus one.
And there's some sparse matrix A which spreads that probability out.
It splits it into my neighbors 50/50.
Here's a simulation of Plinko.
So I started with the probability 1 in the center cell.
And as I go through different levels, I get split 50/50.
And you see a binomial or almost Gaussian distribution spread as I go through more and more levels until it's equally probable that I could wind up in any one of the cells.
You can think about it this way, right?
The probability at level i plus 1 that the chip is in cell N is inherited 50/50 from its two neighbors, right?
There's some probability that was in these two neighbors.
I would inherit half of that probability.
It would be split by these pegs.
The sparse matrix that represents this operation has two diagonals.
And on each of those diagonals is a half.
And you can build that matrix using the spdiags command.
It says that there's going to be two diagonal components which are equal to a half.
And their position is going to be one on either side of the central diagonal.
That's going to indicate that I pass this probability, 50/50, to each of my neighbors.
And then successive multiplications by A will split this probability.
And we'll see the simulation that tells us how probable it is to find the Plinko chip in a particular column.
Yes?
[INAUDIBLE]
Yeah.
So in diffusion in general?
Well, in this instance in particular because [INAUDIBLE]
Well, OK. That's fair enough.
This is one particular model of the Plinko board, which sort of imagines alternating cells that I'm falling through.
We could construct an alternative model, if we wanted to, that didn't have that part of the picture, OK?
So that's a matrix that looks like this, right?
The central diagonal is 0.
Most of the off-diagonal components here are 0 and 1 above and 1 below.
I get a half and a half.
And if I'm careful-- somebody mentioned I need boundary conditions.
When the Plinko chip gets to the edge, it doesn't fall out of the game.
It gets reflected back in.
So maybe we have to choose some special values for a couple of elements of this matrix.
But this is a sparse matrix.
It has a sparse structure.
It models a diffusion problem, just like we saw before.
Most of physics is local, like this, right?
I just need to know what's going on with my neighbors.
And I spread the probability out.
I get this nice diffusion problem.
So it looks like this.
Here's something to notice.
After many levels or cycles, I multiply by A many, many times.
This probability distribution always seems to flatten out.
It becomes uniform.
It turns out there are even special distributions for which A times A times that distribution is equal to that distribution.
You can see it at the end here.
This is one of those special distributions where the probability is equal in every other cell, right?
And at the next level, it all gets passed down.
That's one multiplication by-- it all gets spread by 50%.
And the next multiplication, everything gets spread by 50% again.
And I recover the same distribution that I had before, this uniform distribution.
That's a special distribution for which A times A times P is equal to P. And this distribution is one of the eigenvectors of this matrix A times A.
It's a particular vector that when I multiply it by this matrix AA, I get that vector back.
It happens to be unstretched.
So this vector points in some direction.
I transform it by the matrix.
And I get back something that points in the same direction.
That's the definition of this thing called an eigenvector.
And this will be the subject that we focus on today.
So eigenvectors of a matrix-- they're special vectors that are stretched on multiplication by the matrix.
So they're transformed.
But they're only transformed into a stretched form of whatever they were before.
They point in a direction.
You transform them by the matrix.
And you get something that points in the same direction, but is stretched.
Before, we saw the amount of stretch.
The previous example, we saw the amount of stretch was 1.
It wasn't stretched at all.
You just get back the same vector you had before.
But in principle, it could come back with any length.
For a real N-by-N matrix, there will be eigenvectors and eigenvalues, which are the amount of stretch, which are complex numbers.
And finding eigenvector-eigenvalue pairs involves solving N equations.
We'd like to know what these eigenvectors and eigenvalues are.
They're non-linear because they depend on both the value and the vector, the product of the two, for N plus 1 unknowns.
We don't know how to solve non-linear equations yet.
So we're kind of-- might seem like we're in a rough spot.
But I'll show you that we're not.
But because there's N equations for N plus 1 unknowns, that means eigenvectors are not unique.
If W is an eigenvector, than any other vector that points in that same direction is also an eigenvector, right?
It also gets stretched by this factor lambda.
So we can never say what an eigenvector is uniquely.
We can only prescribe its direction.
Whatever its magnitude is, we don't care.
We just care about its direction.
The amount of stretch, however, is unique.
It's associated with that direction.
So you have an amount of stretch.
And you have a direction.
And that describes the eigenvector-eigenvalue pair.
Is this clear?
You've heard of eigenvalues and eigenvectors before?
Good.
So how do you find eigenvalues?
They seem like special sorts of solutions associated with a matrix.
And if we understood them, then we can do a transformation.
So I'll explain that in a minute.
But how do you actually find these things, these eigenvalues?
Well, I've got to solve an equation A times w equals lambda times w, which can be transformed into A minus lambda identity times w equals 0.
And so the solution set to this equation is either w is equal to 0.
That's one possible solution to this problem or the eigenvector w belongs to the null space of this matrix.
It's one of those special vectors that when it multiplies this matrix gives back 0, right?
It gets projected out on transformation by this matrix.
Well, this solution doesn't seem very useful to us, right?
It's trivial.
So let's go with this idea that w belongs to the null space of A minus lambda I.
That means A minus lambda I must be a singular matrix, whatever it is, right?
And if it's singular, then the determinant of a minus lambda I must be equal to 0.
So if this is true, and it should be true if we don't want a trivial solution, then the determinant of A minus lambda I is equal to 0.
So if we can compute that determinant and solve for lambda, then we'll know the eigenvalue.
Well, it turns out that the determinant of a matrix like A minus lambda I is a polynomial in terms of lambda.
It's a polynomial of degree N called the characteristic polynomial.
And the N roots of this characteristic polynomial are called the eigenvalues of the matrix.
So there are N possible lambdas for which A minus lambda I become singular.
It has a null space.
And associated with those values are eigenvectors, vectors that live in that null space.
So this polynomial-- we could compute it for any matrix.
We could compute this thing in principle, right?
And we might even be able to factor it into this form.
And then lambda 1, lambda 2, lambda N in this factorized form are all the possible eigenvalues associated with our matrix A, right?
There are all the possible amounts of stretch that can be imparted to particular eigenvectors.
We don't know those vectors yet, right?
We'll find them in a second.
But we know the amounts of stretch that can be imparted by this matrix.
OK?
Any questions so far?
No.
Let's do an example.
Here's a matrix, minus 2, 1, 3.
And it's 0's everywhere else.
And we'd like to find the eigenvalues of this matrix.
So we need to know A minus lambda I and its determinant.
So here's A minus lambda I.
We just subtract lambda from each of the diagonals.
And the determinant-- well, here, it's just the product of the diagonal elements.
So that's the determinant of a diagonal matrix like this, the product of the diagonal elements.
So it's minus 2 minus lambda times 1 minus lambda times 3 minus lambda.
And the determent of this has to be equal to 0.
So the amounts of stretch, the eigenvalues imparted by this matrix, are minus 2, 1, and 3.
And we found the eigenvalues.
Here's another matrix.
Can you work out the eigenvalues of this matrix?
Let's take 90 seconds.
You can work with your neighbors.
See if you can figure out the eigenvalues of that matrix.
Nobody's collaborating today.
I'm going to do it myself
[INAUDIBLE]
It's OK. OK. What are you finding?
Anyone want to guess what are the eigenvalues?
[INAUDIBLE]
Good.
OK.
So we need to compute the determinant of A minus lambda I.
That'll be minus 2 minus lambda times minus 2 minus lambda minus 1.
You can solve this to find that lambda equals minus 3 or minus 1.
These little checks are useful.
If you couldn't do this, that's OK.
But you should try to practice this on your own to make sure you can.
Here are some more examples.
So the elements of a diagonal matrix are always the eigenvalues because the determinant of a diagonal matrix is the product of the diagonal elements.
So these diagonal values here are the roots of the secular characteristic polynomial.
They are the eigenvalues.
It turns out the diagonal elements of a triangular matrix are eigenvalues, too.
This should seem familiar to you.
We talked about easy-to-solve systems of equations, right?
Diagonal systems of equations are easy to solve, right?
Triangular systems of equations are easy to solve.
It's also easy to find their eigenvalues.
So the diagonal elements here are the eigenvalues of the triangular matrix.
And eigenvalues have certain properties that can be inferred from the properties of polynomials, right?
Since they are the roots to a polynomial, if we know certain things that should be true of those polynomial of roots, that has to be true of the eigenvalues themselves.
So if we have a matrix which is real-valued, then we know that we're going to have this polynomial of degree N which is also real-valued, OK?
It can have no more than N roots, right?
And so A can have no more than N distinct eigenvalues.
The eigenvalues, like the factors of the polynomial, don't have to be distinct, though?
You could have multiplicity in the roots of the polynomial.
So it's possible that lambda 1 here is an eigenvalue twice.
That's referred to as algebraic multiplicity.
We'll come back to that idea in a second.
Because the polynomial is real-valued, it means that the eigenvalues could be real or complex, just like the roots of a real-valued polynomial.
But complex eigenvalues always appear as conjugate pairs.
If there is a complex eigenvalue, then necessarily its complex conjugate is also an eigenvalue.
And here's a couple other properties.
So the determinant of a matrix is the product of the eigenvalues.
We talked once about the trace of a matrix, which is the sum of its diagonal elements.
The trace of a matrix is also the sum of the eigenvalues.
These can sometimes come in handy-- not often, but sometimes.
Here's an example I talked about before-- so a series of chemical reactions.
So we have a batch, a batch reactor.
We load some material in.
And we want to know how the concentrations of A, B, C, and D vary as a function of time.
And so A transforms into B.
B and C are in equilibrium.
C and D are in equilibrium.
And our conservation equation for material is here.
This is a rate matrix.
We'd like to understand what the characteristic polynomial of that is.
The eigenvalues of that matrix are going to tell us something about how different rate processes evolve in time.
You can imagine just using units.
On this side, we have concentration over time.
On this side, we have concentration.
And the rate matrix has units of rate, or 1 over time.
So those eigenvalues also have units of rate.
And they tell us the rate at which different transformations between these materials occur.
And so if we want to find the characteristic polynomial of this matrix and we need to compute the determinant of this matrix minus lambda I-- so subtract lambda from each of the diagonals-- even though this is a four-by-four matrix, its determinant is easy to compute because it's full of zeros.
I'm not going to compute it for you here.
It'll turn out that the characteristic polynomial looks like this.
You should actually try to do this determinant and show that the polynomial works out to be this.
But knowing that this is the characteristic polynomial, what are the eigenvalues of the rate matrix?
If that's the characteristic polynomial, what are the eigenvalues, or tell me some of the eigenvalues of the rate matrix?
0
0.
0's an eigenvalue.
Lambda equals 0 is a solution.
Minus k1 is another solution.
What is this eigenvalue 0 correspond to?
What's that?
[INAUDIBLE]
OK. Physically, it's a rate process with 0 rate, steady state.
So the 0 eigenvalue's going to correspond to the steady state.
The eigenvector associated with that eigenvalue should correspond to the steady state solution.
How about this eigenvalue minus k1?
This is a rate process with rate k1.
What physical process does that represent?
It's something evolving in time now, right?
So that's the transformation of A into B.
And the eigenvector should reflect that transformation.
We'll see what those eigenvectors are in a minute.
But these eigenvalues can be interpreted in terms of physical processes.
This quadratic solution here has some eigenvalue.
I don't know what it is.
You use the quadratic formula and you can find it.
But it involves k2, k3, k4.
And this is a typo.
It should be k5.
And so that says something about the interconversion between B, C, and D, and the rate processes that occur as we convert from B to C to D. Is that too fast?
Do you want to write some more on this slide before I go on, or are you OK?
Are there any questions about this?
No.
Given an eigenvalue, a particular eigenvalue, what's the corresponding eigenvector?
We know the eigenvector isn't uniquely specified.
It belongs to the null space of this matrix A minus lambda I times identity.
Even though it's not unique, we might still try to find it using Gaussian elimination, right?
So we may try to take-- we may try to solve the equation A minus lambda I times identity multiplied by w equals 0 using Gaussian elimination.
But because it's not unique, at some point, we'll run out of rows to eliminate, right?
There's a null space to this matrix, right?
We won't be able to eliminate everything.
We'd say it's rank deficient, right?
So we'll be able to eliminate up to some R, the rank of this matrix.
And then all the components below are essentially free or arbitrarily specified.
There are no equations to say what those components of the eigenvector are.
The number of all 0 rows-- it's called the geometric multiplicity of the eigenvalue.
Sorry.
Geometric is missing here.
It's the number of components of the eigenvector that can be freely specified.
The geometric multiplicity might be 1.
That's like saying that the eigenvectors are all pointing in the same direction, but can have arbitrary magnitude, right?
It might have geometric multiplicity 2, which means the eigenvectors associated with this eigenvalue live in some plane.
And any vector from that plane is a corresponding eigenvector.
It might have a higher geometric multiplicity associated with it.
So let's try something here.
Let's try to find the eigenvectors of this matrix.
I told you what the eigenvalues were.
They were the diagonal values here.
So they're minus 2, 1, and 3.
Let's look for the eigenvector corresponding to this eigenvalue.
So I want to solve this equation A minus this particular lambda, which is minus 2, times identity equals 0.
So I got to do Gaussian elimination on this matrix.
It's already eliminated for me, right?
I have one row which is all 0's, which says the first component of my eigenvector can be freely specified.
The other two components have to be 0.
3 times the second component of my eigenvector is 0.
5 times the third component is 0.
So the other two components have to be 0.
But the first component is freely specified.
So the eigenvector associated with this eigenvalue is 1, 0, 0.
If I take a vector which points in the x-direction in R3 and I multiply it by this matrix, it gets stretched by minus 2.
So I point in the other direction.
And I stretch out by a factor of 2.
You can guess then what the other eigenvectors are.
What's the eigenvector associated with this eigenvalue here?
0, 1, 0, or anything proportional to that.
What's the eigenvector associated with this eigenvalue?
0, 0, 1, or anything proportional to it.
All these eigenvectors have a geometric multiplicity of 1, right?
I can just specify some scalar variant on them.
And they'll transform into themselves.
Here's a problem you can try.
Here's our series of chemical reactions again.
And we want to know the eigenvector of the rate matrix having eigenvalue 0.
This should correspond to the steady state solution of our ordinary differential equation here.
So you've got to do elimination on this matrix.
Can you do that?
Can you find this eigenvector?
Try it out with your neighbor.
See if you can do it.
And then we'll compare results.
This will just be a quick test of understanding.
Are you guys able to do this?
Sort of, maybe?
Here's the answer, or an answer, for the eigenvector.
It's not unique, right?
It's got some constant out in front of it.
So you do Gaussian elimination here.
So subtract or add the first row to the second row.
You'll eliminate this 0, right?
And then add the second row to the third row.
You'll eliminate this k2.
You have to do a little bit more work to do elimination of k4 here.
But that's not a big deal.
Again, you'll add the third row to the fourth row and eliminate that.
And you'll also wind up eliminating this k5.
So the last row here will be all 0's.
And that means the last component of our eigenvector's freely specifiable.
It can be anything we want.
So I said it is 1.
And then I did back substitution to determine all the other components, right?
That's the way to do this.
And here's what the eigenvector looks like when you're done.
The steady state solution has no A in it.
Of course, A is just eliminated by a forward reaction.
So if we let this run out to infinity, there should be no A.
And that's what happens.
But there's equilibria between B, C, and D. And the steady state solution reflects that equilibria.
We have to pick what this constant out in front is.
And we discussed this before, actually, right?
You would pick that based on how much material was initially in the reactor.
We've got to have an overall mass balance.
And that's missing from this system of equations, right?
Mass conservation is what gave the null space for this rate matrix in the first place.
Make sense?
Try this example out.
See if you can work through the details of it.
I think it's useful to be able to do these sorts of things quickly.
Here are some simpler problems.
So here's a matrix.
It's not a very good matrix.
Matrices can't be good or bad.
It's not particularly interesting.
But it's all 0's.
So what are its eigenvalues?
It's just 0, right?
The diagonal elements are the eigenvalues.
And they're 0.
That eigenvalue has algebraic multiplicity 2.
It's a double root of the secular characteristic polynomial.
Can you give me the eigenvectors?
Can you give me eigenvectors of this matrix?
Can you give me linearly independent-- yeah?
[INAUDIBLE]

[INAUDIBLE]
OK. Good.
So this is a very ambiguous sort of problem or question, right?
Any vector I multiply by A here is going to be stretched by 0 because A by its very nature is all 0's.
All those vectors live in a plane.
So any vector from that plane is going to be transformed in this way.
The eigenvector corresponding to eigenvalue 0 has geometric multiplicity 2 because I can freely specify two of its components.
Oh my goodness.
I went so fast.
We'll just do it this way.
Algebraic multiplicity 2, geometric multiplicity 2-- I can pick two vectors.
They can be any two I want in principle, right?
It has geometric multiplicity 2.
Here's another matrix.
It's a little more interesting than the last one.
I stuck a 1 in there instead.
Again, the eigenvalues are 0.
It's a double root.
So it has algebraic multiplicity 2.
But you can convince yourself that there's only one direction that transforms that squeeze down to 0, right?
There's only one vector direction that lives in the null space of A minus lambda I-- lives in the null space of A.
And that's vectors parallel to 1, 0.
So the eigenvector associated with that eigenvalue 0 has geometric multiplicity 1 instead of geometric multiplicity 2.
Now, here's an example for you to do.
Can you find the eigenvalues and some linearly independent eigenvectors of this matrix, which looks like the one we just looked at.
But now it's three-by-three instead of two-by-two.
And if you find those eigenvalues and eigenvectors, what are the algebraic and geometric multiplicity?
Well, you guys must had a rough week.
You're usually much more talkative and energetic than this.
[LAUGHTER] Well, what are the eigenvalues here?
0
Yeah.
They all turn out to be 0.
So that's an algebraic multiplicity of 3.
It'll turn out there are two vectors, two vector directions, that I can specify that will both be squeezed down to 0.
In fact, any vector from the x-y plane will also be squeezed down to 0.
So this has algebraic multiplicity 3 and geometric multiplicity 2.
I'm going to explain why this is important in a second.
But understanding that this can happen is going to be useful for you.
So if an eigenvalue is distinct, then it has algebraic multiplicity 1.
It's the only eigenvalue with that value.
It's the only time that amount of stretch is imparted.
And there will be only one corresponding eigenvector.
There will be a direction and an amount of stretch.
If an eigenvalue has a algebraic multiplicity M, well, you just saw that the geometric multiplicity, which is the dimension of the null space of A minus lambda I-- it's the dimension of the space spanned by no vectors of A minus lambda I-- it's going to be bigger than 1 or equal to 1.
And it's going to be smaller or equal to M. And we saw different variants on values that sit in this range.
So there could be as many as M linearly independent eigenvectors.
And there may be fewer.
So geometric multiplicity-- it's the number of linearly independent eigenvectors associated with an eigenvalue.
It's the dimension of the null space of this matrix.
Problems for which the geometric and algebraic multiplicity are the same for all the eigenvalues and eigenvectors, all those pairs, are nice because the matrix then is said to have a complete set of eigenvectors.
There's enough eigenvectors in the problem that they describe the span of our vector space RN that our matrix is doing transformations between.
If we have geometric multiplicity that's smaller than the algebraic multiplicity, then some of these stretched-- we can't stretch in all possible directions in RN.
There's going to be a direction that might be left out.
We want to be able to do a type of transformation called an eigendecomposition.
I'm going to show you that in a second.
It's useful for solving systems of equations or for transforming systems of ordinary differential equations, linear ordinary differential equations.
But we're only going to be able to do that when we have this complete set of eigenvectors.
When we don't have that complete set, we're going to have to do other sorts of transformations.
You have a problem in your homework now, I think, that has this sort of a hang-up associated with it.
It's the second problem in your homework set.
That's something to think about.
For a matrix with the complete set of eigenvectors, we can write the following.
A times a matrix W is equal to W times the matrix lambda.
Let me tell you what W and lambda are.
So W's a matrix whose columns are made up of this-- all of these eigenvectors.
And lambda's a matrix whose diagonal values are each of the corresponding eigenvalues associated with those eigenvectors.
This is nothing more than a restatement of the original eigenvalue problem.
AW is lambda W. But now each eigenvalue has a corresponding particular eigenvector.
And we've stacked those equations up to make this statement about matrix-matrix multiplication.
So we've taken each of these W's over here.
And we've just made them the columns of a particular matrix.
But it's nothing more than a restatement of the fundamental eigenvalue problem we posed at the beginning here.
But what's nice is if I have this complete set of eigenvectors, then W has an inverse that I can write down.
So another way to state this same equation is that lambda-- the eigenvalues can be found from this matrix product, W inverse times A times W. And under these circumstances, we say the matrix can be diagonalized.
There's a transformation from A to a diagonal form.
That's good for us, right?
We know diagonal systems of equations are easy to solve, right?
So if I knew what the eigenvectors were, then I can transform my equation to this diagonal form.
I could solve systems of equations really easily.
Of course, we just saw that knowing what those eigenvectors are requires solving systems of equations, anyway.
So the problem of finding the eigenvectors is as hard as the problem of solving a system of equations.
But in principle, I can do this sort of transformation.
Equivalently, the matrix A can be written as W times lambda times W inverse.
These are all equivalent ways of writing this fundamental relationship up here when the inverse of W exists.
So this means that if I know the eigenvalues and eigenvectors, I can easily reconstruct my equation, right?
If I know the eigenvectors in A, then I can easily diagonalize my system of equations, right?
So this is a useful sort of transformation to do.
We haven't talked about how it's done in the computer.
We've talked about how you would do it by hand.
These are ways you could do it by hand.
The computer won't do Gaussian elimination for each of those eigenvectors independently, right?
Each elimination procedure is order N cubed, right?
And you got to do that for N eigenvectors.
So that's N to the fourth operations.
That's pretty slow.
There's an alternative way of doing it that's beyond the scope of this class called-- it's called the Lanczos algorithm.
And it's what's referred to as a Krylov subspace method, that sort of iterative method where you take products of your matrix with certain vectors and from those products, infer what the eigenvectors and eigenvalues are.
So that's the way a computer's going to do it.
That's going to be an order N cubed sort of calculation to find all the eigenvalues and eigenvectors [INAUDIBLE] solving a system of equations.
But sometimes you want these things.
Here's an example of how this eigendecomposition can be useful to you if you did it.
So we know the matrix A can be represented as W lambda W inverse times x equals b.
This is our transformed system of equations here.
We've just substituted for A.
If I multiply both sides of this equation by W inverse, then I've got lambda times the quantity W inverse x is equal to W inverse b.
And if I call this quantity in parentheses y, then I have an easy-to-solve system of equations for y. y is equal to lambda inverse times c. But lambda inverse is just 1 over each of the diagonal components of lambda.
Lambda's a diagonal matrix.
Then all I need to do-- ooh, typo.
There's an equal sign missing here.
Sorry for that.
Now all I need to do is substitute for what I called y and what I called c. So y was W inverse times x.
That's equal to lambda inverse times W inverse times b.
And so I multiply both sides of this equation by W. And I get x is W lambda inverse W inverse b.
So if I knew the eigenvalues and eigenvectors, I can really easily solve the system of equations.
If I did this decomposition, I could solve many systems of equations, right?
They're simple to solve with just matrix-matrix multiplication.
Now, how is W inverse computed?
Well, W inverse transpose are actually the eigenvectors of A transpose.
You may have to compute this matrix explicitly.
But there are times when we deal with so-called symmetric matrices, ones for which they are equal to their transpose.
And if that's the case, and if you take all of your eigenvectors and you normalize them so they're of length 1-- the Euclidean norm is 1-- then it'll turn out that W inverse is precisely equal to W transpose, right?
And so the eigenvalue matrix will be unitary.
It'll have this property where its transposes is its inverse, right?
So this becomes trivial to do then, this process of W inverse.
It's not always true that this is the case, right?
It is true when we deal with problems that have symmetric matrices associated with them.
That pops up in a lot of cases.
You can prove-- I might ask you to show this some time-- that the eigenvectors of a symmetric matrix are orthogonal, that they satisfy this property that-- I take the dot product between two different eigenvectors and it'll be equal to 0 unless those are the same eigenvector.
That's a property associated with symmetric matrices.
They're also useful when analyzing systems of ordinary differential equations.
So here, I've got a differential equation, a vector x dot.
So the time derivative of x is equal to A times x.
And if I substitute my eigendecomposition-- so W lambda W inverse-- and I define a new unknown y instead of x, then I can diagonalize that system of equations.
So you see y dot is equal to lambda times y where each component of y is decoupled from all of the others.
Each of them satisfies their own ordinary differential equation that's not coupled to any of the others, right?
And it has a simple first-order rate constant, which is the eigenvalue associated with that particular eigendirection.
So this system of ODEs is decoupled.
And it's easy to solve.
You know the solution, right?
It's an exponential.
And that can be quite handy when we're looking at different sorts of chemical rate processes that correspond to linear differential equations.
We'll talk about nonlinear, systems of nonlinear, differential equations later in this term.
And you'll find out that this same sort of analysis can be quite useful there.
So we'll linearize those equations.
And we'll ask is their linear-- in their linearized form, what are these different rate constants?
How big are they?
They might determine what we need to do in order to integrate those equations numerically because there are many times when there's not a complete set of eigenvectors.
That happens.
And then the matrix can't be diagonalized in this way.
There are some components that can't be decoupled from each other.
That's what this diagonalization does, right?
It splits up these different stretching directions from each other.
But there's some directions that can't be decoupled from each other anymore.
And then there are other transformations one can do.
So there's an almost diagonal form that you can transform into called the Jordan normal form.
There are other transformations that one can do, like called, for example, Schur decomposition, which is a transformation into an upper triangular form for this matrix.
We'll talk next time about the singular value decomposition, which is another sort of transformation one can do when we don't have these complete sets of eigenvectors.
But this concludes our discussion of eigenvalues and eigenvectors.
You'll get a chance to practice these things on your next two homework assignments, actually.
So it'll come up in a couple of different circumstances.
I would really encourage you to try to solve some of these example problems that were in here.
Solving by hand can be useful.
Make sure you can work through the steps and understand where these different concepts come into play in terms of determining what the eigenvalues and eigenvectors are.
All right.
Have a great weekend.
See you on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this is going to be our last lecture on linear algebra.
The first three lectures covered basics.
The next three lectures, we talked about different sorts of transformations of matrices.
This final lecture is the last of those three.
We're going to talk about in another sort of transformation called the singular value decomposition.
OK, before we jump in, I'd like to do the usual recap business.
I think it's always hopeful to recap or look at things from a different perspective.
Early on, I told you that the infinite dimensional equivalent of vectors would be something like a function, which is a map, a unique map maybe from a point to x to some value f of x.
And there is an equivalent representation of the eigenvalue eigenvector problem in function space.
We call these eigenvalues and eigenfunctions.
Here's a classic one where the function is y of x, OK?
This is the equivalent of the vector, and equivalent of the transformation or the matrix that's this differential operator this time, the second derivative.
So I take the second derivative of this particular function, and the function is stretched.
It's multiplied by some fixed value at all points.
And it becomes lambda times y.
And that operator has to be closed with some boundary conditions as well.
We have to say what the value of y is at the edges of some boundary.
So there's a one-to-one correspondence between these things.
What is the eigenfunction here, or what are the eigenfunctions?
And what are the eigenvalues associated with this transformation or this operator?
Can you work those out really quickly?
You learned this at some point, right?
Somebody taught you differential equations and you calculated these things.
Take about 90 seconds.
Work with the people around you.
See if you can come to a conclusion about what the eigenfunction and eigenvalues are.
That's enough time.
You can work on this on your own later if you've run out of time.
Don't worry about it.
Does somebody want to volunteer a guess for what the eigenfunctions are in this case?
What are they?
Yeah?
[INAUDIBLE]
OK, so you chose exponentials.
That's an interesting choice.
That's one possible choice you can make.
OK, so we could say-- this is sort of a classical one that you think about when you first learn differential equation.
They say, an equation of this sort has solutions that look like exponentials, and that's true.
There's another representation for this, which is as trigonometric functions instead, right?
Either of those is acceptable.
[INAUDIBLE] the trigonometric functions, that representation is a little more useful for us here.
We know that the boundary conditions tell us that y of 0 is supposed to be 0.
That means that the C1 has to be 0, because cosine of 0 is 1.
So C1 has 0 in this case.
So that fixes one of these coefficients.
And now we're left with a problem, right?
Our solutions, our eigenfunctions, cannot be unique.
So we don't get to specify C2, right?
Any function that's a multiple of this sine should also be an eigenfunction.
So instead the other boundary condition, this y of l equals 0, needs to be used to pin down with the eigenvalue is.
So the second equation, y of l equals 0, which implies that the square root of minus lambda has to be equal to 2 pi over l, it has to be all the nodes of the sine where the sine is equal to 0.
That's the equivalent of our secular characteristic polynomial that prescribes with the eigenvalues are associated with each of the eigenfunctions.
So now we know what the eigenvalues are.
The eigenvalues are the set of numbers minus 2 pi n over l squared.
There's an infinite number of eigenvalues.
It's an infinite dimensional space that we're in, so it's not a big surprise that it works out that way.
And the eigenvectors then are different scalar multiples of sine of the eigenvalues, square root of the eigenvalues, minus x.
There's a one-to-one correspondence between all the linear algebra we've done and linear differential equations or linear partial differential equations.
You can think about these things in exactly the same way.
I'm sure in 1050, you started to talk about orthogonal functions to represent solutions of differential equations.
Or if you haven't, you're going to very soon.
This is a part of the course you get to look at the analytical side of some of these things as opposed to the numerical side.
But there's a one-to-one relationship between those things.
So if you understand one, you understand the other, and you can come at them from either perspective.
This sort of stuff is useful.
Actually, the classical chemical engineering example comes from quantum mechanics where you think about wave functions and different energy levels corresponding to eigenvalues.
That's cool.
Sometimes, I like to think about a mechanical analog to that, which is the buckling of an elastic column.
So you should do this at home.
You should go get a piece of spaghetti and push on the ends of the piece of the spaghetti.
And the spaghetti will buckle.
Eventually it'll break, but it'll buckle first.
It'll bend.
And how does it bend?
Well, a balance of linear momentum on this bar would tell you that the deflection in the bar at different points x along the bar multiplied by the pressure has to balance the bending moment in the bar itself.
So this e is some elastic constant.
I has a moment of inertia.
And D squared y dx squared is something like the curvature of the bar.
So it's the bending moments of the bar that balances the pressure that's being exerted on the bar.
And sure enough, this bar will buckle when the pressure applied exceeds the first eigenvalue associated with this differential equation.
We just worked that eigenvalue out.
We said that that eigenvalue had to be the square root of 2 pi over l squared.
And so when the pressure exceeds square root of 2 pi over l squared times the elastic modulus, this column will bend and deform continuously until it eventually breaks, right?
It will undergo this linear elastic deformation, then plastic deformation later, and it will break.
The Eiffel Tower, actually, is one of the first structures in the world to utilize this principle, right?
It's got very narrow beams in it.
The beams are engineered so that their elastic modulus is strong enough that they won't buckle.
Gustave Eiffel is one of the first applied physicists, somebody who took the physics of elastic bars and applied them to building structures that weren't big and blocky, but used a minimal amount of material.
Cool, right?
OK, so that's recap.
Any questions about that?
You've seen these things before.
You understood them well before too maybe?
Give some thought to this, OK?
We talked about eigendecomposition last time that, associated with the square matrix, was a particular eigenvalue or particular set of eigenvalues, stretches and corresponding eigenvectors directions.
These were special solutions to the system of linear equations based on a matrix.
It was a square matrix.
And you might ask, well, what happens if the matrix isn't square?
What if A is in the space of real matrices that are n by m, where n and m maybe aren't the same?
Maybe they are the same, but maybe they're not.
And there is an equivalent decomposition.
It's called the singular value decomposition.
It's like an eigendecomposition for non-square matrices.
So rather than writing our matrix as some w lambda w inverse, we're going to write it as some product U times sigma times V with this dagger.
The dagger here is conjugate transpose.
Transpose the matrix, and take the complex conjugate of all the elements, OK?
I mentioned last time that eigenvalues and eigenvectors could be complex, potentially, right?
So whenever we have that case where things can be complex, usually the transposition operation is replaced with the conjugate transpose.
What are these different matrices.
Well, let me tell you.
U is a complex matrix.
It maps from the space N to R N to R N, so it's an n by n square matrix.
Sigma is a real valued matrix, and it lives in the space of n by n matrices.
V is a square matrix again, but it has dimensions m by m. Remember, A maps from R M to R N, so that's what the sequence of products says.
B maps from m to m. Sigma maps from m to n. U maps from n to n. So this match from m to n as well.
Sigma is like lambda from before.
It's a diagonal matrix.
It only has diagonal elements.
It's just not square, but it only has diagonal elements, all of which will be positive.
And then U and V are called the left and right singular vectors.
And they have special properties associated with them, which I'll show you right now.
Any questions about how this decomposition is composed or made up?
It looks just like the eigendecomposition, but it can be applied to any matrix.
Yes?
Quick question
Sure
Do all matrices have this thing, or is it like the eigenvalues where some do and some don't
This is a great question.
So all matrices are going to have a singular value decomposition.
We saw with the eigenvalue decomposition that there could be a case where the eigenvectors are degenerate, and we can't write that full decomposition.
All matrices are going to have this decomposition.
So for some properties of this decomposition, U and V are what we call unitary matrices.
I talked about these before.
Unitary matrices are ones for whom, if they're real valued, their transpose is also their inverse.
If they're complex valued, and they're conjugate transpose is the equivalent of their inverse.
So U times U conjugate transpose will be identity.
V times V conjugate transpose will be identity.
Unitary matrices also have the property that they impart no stretch to a matrix-- or to vectors.
So their maps don't stretch.
They're kind of like rotational matrices, right?
They change directions, but they don't stretch things out.
If I were to take A conjugate transpose and multiply it by A, that would be the same as taking U sigma V conjugate transpose, and multiplying it by U sigma V. If I use the properties of matrix multiplications and complex conjugate transposes, and work out what this expression is, I'll find out that it's equivalent to V sigma conjugate transpose sigma V conjugate transpose.
Well this has exactly the same form as an eigendecomposition.
An eigendecomposition of A times A instead of an eigendecomposition of A.
So V is the set of eigenvectors of A conjugate transpose A, and sigma squared are the eigenvalues of A conjugate transpose times A.
And if I reverse the order of this multiplication-- so I do A times A conjugate transpose-- and work it out, that would be U sigma sigma U.
And so U are the eigenvectors of A A conjugate transpose, and sigma squared are still the eigenvalues of A A conjugate transpose.
So what are these things U and V?
They relate to the eigenvectors of the product of A with itself, this particular product of A with itself, or this particular product of A with itself.
Sigma are the singular values.
And all matrices possess this sort of a decomposition.
They all have a set of singular values and singular vectors.
These sigmas are called the singular values of the A.
They have a particular name.
I'm going to show you how you can use this decomposition to do something you already know how to do, but how to do it formally.
What are some properties of the singular value decomposition?
So if we take a matrix A and we compute it's singular value decomposition, this is how you do it in Matlab.
We'll find out, for this matrix, U is identity.
Sigma is identity with an extra column pasted on it.
And B is also identity.
I mean, this is the simplest possible four by three matrix I can write down.
You don't have to know how to compute the singular value decomposition, you just need to know that it can be computed in this way.
You might be able to guess how to compute it based on what we did with eigenvalues earlier and eigenvectors.
It'll turn out some of the columns of sigma will be non-zero right?
There are three non-zero columns of sigma.
And the columns of V, they correspond to those columns of sigma, spanned the null space of the matrix A.
So the first three columns here are non-zero, the first three columns of V. I'm sorry, the first three columns here are non-zero.
The last column is 0.
The columns of sigma which are 0 correspond to a particular column in V, this last column here, which lives in the null space of A.
So you can see, if I take A and I multiply it by any vector that's proportional to 0, 0, 0, 1, I'll get back 0.
So the null space of A is spanned by all these vectors corresponding to the 0 columns of sigma.
Some of the columns of sigma are non-zero.
These first three columns.
And the rows of U corresponding to those three columns span the range of A.
So if I do the singular value decomposition of a matrix, and I look at U, V, and sigma and what they're composed of-- where sigma is 0 and non-zero, and the corresponding columns or rows of U and V-- then I can figure out what vectors span the range and null space of the matrix A.
Here's another example.
So here I have A.
Now instead of being three rows by four columns, it's four rows by three columns.
And here's the singular value decomposition that comes out of Matlab.
There are no vectors that live in the null space of A, and there are no 0 columns in sigma.
There's no corresponding columns in V. There are no vectors in the null space of A.
The range of A is spanned by the rows corresponding to the non-zero-- the rows of U corresponding to the non-zero columns of sigma.
So it's these three columns in the first three rows.
And these first three rows, clearly they span-- they describe the same range as the three columns in A.
So the singular value decomposition gives us direct access to the null space and the range of a matrix.
That's handy.
And it can be used in various ways.
So here's one example where it can be used.
Here I have a fingerprint.
It's a bitmap.
It's a square bit of data, like a matrix, and each of the elements of the matrix takes on a value describing how dark or light that pixel.
Let's say it's grayscale, and it's value's between 0 and 255.
That's pretty typical.
So I have this matrix, and each element to the matrix corresponds to a pixel.
And I do a singular value decomposition.
Some of the singular values, the values of sigma, are bigger than others.
They're all positive, but some are bigger than others.
The ones that are biggest in magnitude carry the most information content about the matrix.
So we can do data compression by neglecting singular values that are smaller than some threshold, and also neglecting the corresponding singular vectors.
And that's what I've done here.
So here's the original bitmap of the fingerprint.
I did the singular value decomposition, and then I retained only the 50 biggest singular values and I left all the other singular values out.
This bitmap was something like, I don't know, 300 pixels by 300 pixels, so there's like 300 singular values, but I got rid of 5/6 of the information content.
I dropped 5/6 of the singular vectors, and then I reconstructed the matrix from the singular values and those singular vectors, and you get a faithful representation of the original fingerprint.
So the singular value decomposition says something about the information content in the transformation that is the matrix, right?
There are some transformations that are of lower power or importance than others.
And the magnitude of these singular values tell you what they are.
Does that makes sense?
How else can it be used?
Well, one way it can be used is finding the least square solution to the equation Ax equals b, where A is no longer a square matrix, OK?
You've done this in other contexts before where the equations are overspecified.
We have more equations than unknowns, like data fitting.
You form the normal equations, you multiply both sides of Ax equals b by A transpose, and then invert A transpose A.
You might not be too surprised, then, to think that singular value decomposition could be useful here too.
Since we already saw the data in a singular value decomposition corresponds to eigenvectors and eigenvalues of this A transpose A, right?
But there's a way to use this sort of decomposition formally to solve problems that are both overspecified and underspecified.
Least squares means find the vector of solutions x that minimizes this function phi.
Phi is the length of the vector given by the difference between Ax and b.
It's one measure of how far an error our solution x is.
So let's define the value x which is least in error.
This is one definition of least squares.
And I know the singular value decomposition of A.
So A is U sigma times V. So I have U sigma V times x. I can factor out U, and I've got a factor of U transpose, or U conjugate transpose multiplying by b.
So Ax minus b is the same as U times the quantity sigma V conjugate transpose x minus U conjugate transpose b.
We want to know the x that minimizes this phi.
It's an optimization problem.
We'll talk in great detail about these sorts of problems later.
This one is so easy to do, we can just work it out in a couple lines of text.
We'll define a new set of unknowns, y, which is V transpose times x, and a new right-hand side for a system of equations p, which is U transpose times b.
And then we can rewrite our function phi that we're trying to minimize.
So phi then becomes U sigma y minus p. U is a unitary vector.
It imparts no stretch in the two norms, so this sigma y minus p doesn't get elongated by multiplication with U.
So it's length, the length of this, is the same as the length of sigma y minus p. You can prove this.
It's not very difficult to show at all.
You use the definition of the two norm to prove it.
So phi is minimized by y's, which makes this norm smallest, make it closest to 0.
Let r be the number of non-zero singular values, the number of those sigmas which are not equal to 0.
That's also the rank of A.
Then I can rewrite phi as the sum from i equals 1 to r of sigma i i time y i minus p i squared.
That's parts of this length, this Euclidean length, for which sigma is non-zero.
Plus the sum from r plus 1 to n, the sum over the rest of the values of p, for which the corresponding sigmas are 0.
I want to minimize phi, and the only thing that I can change to minimize it is what?
What am I free to pick in this equation in order to make phi as small as possible?
Yeah?
y
y, so I need to choose the y's that make this phi as small as possible.
What value should I choose for the y's?
What do you think?
[INAUDIBLE]
Perfect, right?
Choose y equals p i over sigma i i.
Right, y i is p i over sigma i i.
Then all of these terms is 0.
I can't make this sum any smaller than that.
That fixes the value of y i up to r. I can't do anything about this left over bit here.
There's no choice of y that's going to make this part and the smaller.
It's just left over.
It's some remainder that we can't make any smaller or minimize an smaller.
There isn't an exact solution to this problem, in many cases.
But one way this could be 0 is if r is equal to n. Then there are left over unspecified terms, and then this y i equals p i over sigma i is the exact solution to the problem.
So this is what you told me.
Choose y i is p i over sigma i i for i bigger than 1 and smaller than r. There are going to be values of y i that go between r plus 1 and m, because A was a vector that mapped from m to n, right?
So I have extra values of y that could be specified potentially.
If that's true, if r plus 1 is smaller than m, then there's some components of y that I don't get to-- I can't specify, right?
My system of equations is somehow underdetermined.
I need some external information to show me what values to pick for those y i. I don't know.
I can't use them.
Sometimes people just set y i equal to 0.
That's sort of silly, but that's what's done.
It's called the minimum norm least square solution.
y has minimum length, when you set all these other components to 0.
But the truth is, we can't specify those components, right?
We need some external information in order to specify them.
Once we know y, we can find x going back to our definition of what y is.
So I multiply this equation by V on both sides, and I'll get V y equals x.
So I can find my least square solution to the problem from the singular value decomposition.
So I can find the least square solution to both overdetermined and underdetermined problems using singular value decomposition.
It inherits all the properties you know of solving the normal equations, multiplying by A transpose the entire equation, and solving for a least square solution that way.
But that's only good for overdetermined systems of equations.
This can work for underdetermined equations as well.
And maybe we do have extraneous information that lets us specify these other components somehow.
Maybe we do a separate optimization that chooses from all possible solutions where these y i's are free, and picks the best one subject to some other constraint.
Does it makes sense?
OK, that's the last decomposition we're going to talk about.
It's as expensive to compute the singular value decomposition as it is to solve a system of equations.
You might have guessed that it's got an order N cubed flavor to it.
It's kind of inescapable that we run up against those computational difficulties, order N cubed computational complexity.
And there are many problems of practical interest, particularly solutions of PDEs, for which that's not going to cut it.
Where you couldn't solve the problem with that sort of scaling in time.
You couldn't compute the Gaussian elimination, or the singular value decomposition, or an eigenvalue decomposition.
It won't work.
And in those cases, we appeal to not exact solution methods, but approximate solution methods.
So instead of trying to get an exact solution, we'll try to formulate one that's good enough.
We already know the computer introduces numerical error anyways.
Maybe we don't need machine precision in our solution or something close to machine precision in our solution.
Maybe we're solving engineering problem, and we're willing to accept relative errors on the order of 10 to the minus 3 or 10 to the minus 5, some specified tolerance that we apply to the problem.
And in those circumstances, we use iterative methods to solve systems of equations instead of exact methods, elimination methods, or metrics decomposition methods.
These algorithms are all based on iterative refinement of an initial guess.
So if we have some system of equations we're trying to solve, Ax equals b, we'll formulate some linear map, right?
xi plus 1 will be some matrix C times x i plus some little vector c where x i is my last best guess for the solution to this problem, and x i plus 1 is my next best guess for the solution to this problem.
And I'm hoping, as I apply this map more and more times, I'm creeping closer to the exact solution to the original system of equations.
The map will converge when x i plus 1 approaches x i, when the map isn't making any changes to the vector anymore.
And the converged value will be a solution when x i-- which is equal to i minus c inverse times c, if I replace x i was 1 with x i appear, so I say that my map has converged-- when this value is equivalent to A inverse times B, when it's a solution to the original problem, right?
So my map may converge.
It may not converge to a solution of the problem I like, but if it satisfies this condition, then has converged to be a solution of the problem that I like as well.
And so it's all about using this C here and this little c here so that this map converges to solution of the problem I'm after.
And there are lots of schemes for doing this.
Some of them are kind of ad hoc.
I'm going to show you one right now.
And then when we do optimization, we'll talk about a more formal way of doing this for which you can guarantee very rapid convergence to a solution.
So here's a system of equations I'd like to solve.
It's not a very big one.
It doesn't really make sense to solve this one iteratively, but it's a nice illustration.
One way to go about formulating this map is to split this matrix into two parts.
So I'll split it into a diagonal part and an off diagonal part.
So I haven't changed the problem at all by doing that.
And then I'm going to rename this x x i plus 1, and I'm going to rename this x x i.
And then move this matrix vector product to the other side of the equation.
And here's my map.
Of course, this matrix multiplied doesn't make any-- it's not useful to write it out explicitly.
This is just identity.
So I can drop this entirely.
This is just x i plus one.
So here's my map.
Take an initial guess, multiply it by this matrix, add the vector 1, 0, and repeat over and over and over again.
Hopefully-- we don't really know-- but hopefully, it's going to converge to a solution of the original linear equations.
I didn't make up that method.
That's a method called Jacobi Iteration.
And the strategy is to split the matrix A into two parts-- a sum of its diagonal elements, and it's off diagonal elements-- and rewrite the original equations as an iterative map.
So D times x i plus 1 is equal to minus r times x i plus b.
Or x i plus 1 is D inverse times minus r x i plus b.
If the equations converge, then D plus r times x i has to be equal to b, we will have found a solution.
If it converges, right?
If these iterations approach a steady value.
If they don't change from iteration to iteration.
Is The nice thing about the Jacobi method is it turns the hard problem, the order N cubed problem of computing A inverse B, into a succession of easy problems, D inverse times some vector C. How many calculations does it take to compute that D inverse?
N, that's right, order N. It's just a diagonal matrix.
I invert each of its diagonal elements, and I'm done.
So I went from order N cubed, which was going to be hard, into a succession of order N problems.
So as long as it doesn't take me order N squared iterations to get to the solution that I want, I'm going to be OK.
This is going to be a viable way to solve this problem faster than finding the exact solution.
How do you know that it converges?
That's the question.
Is this thing actually going to converge or not, or are these iterations just going to run on and on forever?
Well, one way to check whether it will converge or not is to go back up to this equation here, and substitute b equals Ax, where x is the exact solution to the problem.
And you can transform, then, this equation into one that looks like x i plus 1 minus x equal to minus D inverse times r x i minus x.
And if I take the norm of both sides and I apply our normal equality-- where the norm of a matrix vector product is smaller than the product of the norms of the matrices of the vectors-- then I can get a ratio like this.
That the absolute error in iteration I plus 1 divided by the absolute error in iteration i is smaller than the norm of this matrix.
So if I'm converging, then what I expect is this ratio should be smaller than 1.
The error in my next approximation should be smaller than the error in my current approximation.
That makes sense?
So that means that I would hope that the norm of this matrix is also smaller than 1.
If it is, then I'm going to be guaranteed to converge.
So for a particular coefficient matrix, for a system of linear equations I'm trying to solve, I may be able to find-- I may find that this is true.
And then I can apply this method, and I'll converge to a solution.
We call this sort of convergence linear.
Whatever this number is, it tells me the fraction by which the error is reduced from iteration to iteration.
So suppose this is 1/10.
Then the absolute error is going to be reduced by a factor of 10 in each iteration.
It's not going to be 1/10 usually.
It's going to be something that's a little bit bigger than that typically, but that's the idea.
You can show-- I would encourage you to try to work this out on your own-- but you can show that the infinity norm of this product-- infinity norm of this product is equal to this.
And if I ask that the infinity norm of this product be smaller than 1, that's guaranteed when the diagonal values of the matrix and absolute value are bigger than the sum of the off diagonal values in a particular row or a particular column.
And that kind of matrix we call diagonally dominant.
The diagonal values are bigger than the sum and absolute value of the off diagonal pieces.
So diagonally dominant matrices, which come up quite often, can be-- those linear equations based on those matrices can be solved reasonable efficiency using the Jacobi method.
There are better methods to choose.
I'll show you one in a second.
But you can guarantee that this is going to converge to a solution, and that the solution will be the right solution to the linear equations you were trying to solve.
So if the goal is just to turn hard problems into easier to solve problems, then there are other natural ways to want to split a matrix.
So maybe you want to split into A lower triangular part which contains the diagonal elements of A, and an upper triangular part which has no diagonal elements of A.
We just split this thing apart.
And then we could rewrite our system of equations is an iterative map like this, L times x i plus 1 is minus U times x i plus b.
All I have to do is invert l to find my next iteration.
And how expensive computationally is it to solve a system of equations which is triangular?
This is a process we call back substitution.
Its order--
N squared
--N squared.
So we still beat N cubed.
One would hope that it doesn't require too many iterations to do this.
But in principle, we can do this order N squared operations many times.
And it'll turn out that this sort of a map converges to the solution that we're after.
It converges when matrices are either diagonally dominant as before, or they're symmetric and they're positive definite.
Positive definite means all the eigenvalues of the matrix are bigger than 0.
So try the iterative method solving some equations and see how we convert.
Yes?
How do you justify ignoring the diagonal elements in that method?
So the question was, how do you justify ignoring the diagonal elements in this method.
Maybe I was going too fast or I misspoke.
So I'm going to split A into a lower triangular matrix that has all the diagonal elements, and U is the upper parts with none of those diagonal elements on it.
Does that make sense?
Yeah
Thank you for asking that question.
I hope that's clear.
l holds onto the diagonal pieces and U takes those away.
So let's try it.
On a matrix like this, the exact solution to this system of equations is 3/4, 1/2, and 1/4.
All right, we'll try Jacobi, we'll have to give it some initial guess for the solution, right?
We're talking about places where you can derive those initial guesses from later on in the course, but we have to start the iterative process with some guess at the solutions.
So here's an initial guess.
We'll apply this map.
Here's Gauss-Seidel with the same initial guess, and we'll apply this map.
They're both linearly convergent, so the relative error will go down by a fixed factor after each iteration.
Iteration one, the relative error in Jacobi will be 38%.
In Gauss-Seidel, it'll be 40%.
If we apply this all the way down to 10 iterations, the relative error Jacobi will be 1.7%, and the relative error in Gauss-Seidel 0.08%.
And we can go on and on with these iterations if we want until we get sufficiently converged, we get to a point where the relative error is small enough that we're happy to accept this answer as a solution to our system of equations.
So we traded the burden of doing all these calculations to do elimination for a faster, less computationally complex methodology.
But the trade off was we don't get an exact solution anymore.
We're going to have finite precision in the result, and we have to specify the tolerance that we want to converge to.
We're going to see now-- this is the hook into the next part of that class-- we're going to talk about solutions of nonlinear equations next for which there are almost no non-linear equations that we can solve exactly.
They all have to be solved using these iterative methods.
You can use these iterative methods for linear equations.
It's very common to do it this way.
In my group, we solve lots of systems of linear equations associated with hydrodynamic problems.
These come up when you're talking about, say, low Reynolds number flows, which are linear sorts of fluid flow problems.
They're big.
It's really hard to do Gaussian elimination, so you apply different iterative methods.
You can do Gauss-Seidel.
You can do Jacobi.
We'll learn about more advanced ones like PCG, which you're applying on your homework now, and you should be seeing that it converges relatively quickly in cases where exact elimination doesn't work.
We'll learn, actually, how to do that method.
That's one that we apply in my own group.
It's pretty common to use out there.
Yes?
One question, is that that Gauss, [INAUDIBLE]
Order N squared
Yeah, that's what I meant.
So now we've got an [INAUDIBLE].
So we basically have [INAUDIBLE] iterations, right?
This is a wonderful question.
So this is a pathological problem in the sense that it requires a lot of calculations to get an iterative solution here.
We haven't gotten to an end that's big enough that the computational complexities crossover.
So for small Ns, probably the factor in front of N-- whatever number that is-- and maybe even the smaller factors, order N squared factors on that order N cubed, play a big role in how long it takes to actually complete this thing.
But modern problems are so big that we almost always are running out to Ns that are large enough that we see a crossover.
You'll see this in your homework this week.
You won't see this crossover at N equals 3.
You're going to see it out at N equals 500 or 1,200, big problems.
Then we're going to encounter this crossover.
That's a wonderful question.
So first small system sizes, iterative methods maybe don't buy you much.
I suppose it depends on the application though, right?
If you're doing something that involves solving problems on embedded hardware, in some sort of sensor or control valve, there may be very limited memory or computational capacity available to you.
And you may actually apply an iterative method like this to a problem that that controller needs to solve, for example.
It just may not have the capability of storing and inverting what we would consider, today, a relatively small matrix because the hardware doesn't have that sort of capability.
So there could be cases where you might choose something that's slower but feasible, versus something that's faster and exact, because there are other constraints.
They do exist, but modern computers are pretty efficient.
Your cell phone is faster than the fastest computers in the world 20 years ago.
We're doing OK.
So we've got to get out to big system sizes, big problem sizes, before this starts to pay off.
But it does for many practical problems.
OK I'll close with this, because this is the hook into solving nonlinear equations.
So I showed you these two iterative methods, and they kind of had stringent requirements for when they were actually going to converge, right?
I had to have a diagonally dominant system of equations for Jacobi to converge.
I had to have diagonal dominance or symmetric positive definite matrices.
These things exist and they come up in lots of physical problems, but I had to have it in order for Gauss-Seidel to converge.
What if I have a system of equations that doesn't work that way?
Or what if I have an iterative map that I like for some reason, but it doesn't appear to converge?
Maybe it converges under some circumstances, but not others.
Well, there's a way to modify these iterative maps, called successive over-relaxation, which can help promote convergence.
So suppose we have an iterative map like this, x i plus 1 is some function of the previous iteration value.
Doesn't matter what it is.
It could be linear, could be non-linear.
We don't actually care.
The sought after solution is found when x i plus 1 is equal to x i.
So this map is one the convergence to the exact solution of the problem that we want.
We've somehow guaranteed that that's the case, but it has to converge.
One way to modify that map is to say x i plus 1 is 1 minus some scalar value omega times x i plus omega times f. You can confirm that if you substitute x i plus 1 equals x i into this equation, you'll come up with the same fixed points of this iterative map x i is equal to f of x i.
So you haven't changed what value will converge here, but you've affected the rate at which it converges.
Here you're saying x i plus 1 is some fraction of my previous solution plus some fraction of this f. And I get to control how big those different fractions.
So if things aren't converging well for a map like this, then I could try successive over-relaxation, and I could adjust this relaxation parameter to be some fraction, some number between 0 and 1, until I start to observe convergence.
And there are some rules one can use to try to promote convergence with this kind of successive over-relaxation.
This is a very generic technique that one can apply.
If you have any iterative map you're trying to apply, it should go to the solution you want but it doesn't converge for some reason, then you can use this relaxation technique to promote convergence to the solution.
You may slow the convergence way down.
It may be very slow to converge, but it will converge.
And after all, an answer is better than no answer, no matter how long it takes to get it.
So sometimes you've got to get these things by hook or by crook.
So for example, you can apply this to Jacobi.
This was the original Jacobi map.
And we just take that.
We add 1 minus omega times x i plus omega times this factor over here.
And now we can choose omega so that this solution converges.
We always make omega small enough so that the diagonal values of our matrix appear big enough that the matrix looks like it's diagonally dominated.
You could go back to that same convergence analysis that I showed you before and try to apply it to this over-relaxation form of Jacobi and see that, while there's always going to be some value of omega that's small enough, that this thing will converge.
It will look effectively diagonally dominant, because omega inverse times D will be big enough, or omega times D inverse will be small enough.
Does that make sense?
You can apply the same sort of damping method to Gauss-Seidel as well.
It's very common to do this.
The relaxation parameter acts like an effective increase in the eigenvalues of the matrix.
So you can think about L. That's a lower triangular matrix.
It's diagonal values are its eigenvalues.
The diagonal values of L inverse-- well, 1 over those diagonal values are the eigenvalues of L inverse.
And so if we make omega very small, then we make the eigenvalues of L inverse very small, or the eigenvalues or L very big.
And again, the matrix starts to look diagonally dominated.
And you can promote convergence in this way.
So even though this may be slow, you can use it to guarantee convergence of some iterative procedures, not just for linear equations, but for non-linear equations as well.
And we'll see, there are good ways of choosing omega for certain classes of non-linear equations.
We'll apply Newton-Raphson method, and then will damp it using exactly the sort of procedure.
And I'll show you how you can choose a nearly optimal value for omega to promote convergence to the solution.
Any questions?
No, let me address one more thing before you go.
We've scheduled times for the quizzes.
They are going to be in the evenings on the dates that are specified on the syllabus.
We wanted to do them during the daytime.
It was really difficult to schedule a room that was big enough for this class, so they have to be from 7:00 to 9:00 in the gymnasium.
I apologize for that.
We spent several days looking around trying to find a place where we could put everybody so you would all get the same experience in the quiz.
I know that the November quiz comes back to back with the thermodynamics exam as well.
That's frustrating.
Thermodynamics is the next day.
That week is tricky.
That's AICHE, so most of the faculty have to travel.
We won't be able to teach, but you won't have classes one of those days so you have extra time to study.
And Columbus Day also falls in that week, so there's no way to put three exams in four days without having them come right back to back.
Believe me, we thought about this and tried to get things scheduled as efficiently as we could for you, but sometimes there are constraints that are outside of our control.
But the quiz times are set.
There's going to be done in October and one in November.
They'll be in the evening, and they'll be in the gymnasium.
I'll give you directions to it before the exam, just say you know exactly where to go, OK?
Thank you, guys.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, we're moving on.
No more linear algebra.
We're going to try solving some more difficult problems-- of course, all those problems will just be turned into linear algebra as we move on, so your expertise now with different techniques from linear algebra is going to come in handy.
The next section of this course, we're talking about systems of nonlinear equations, and we'll transition into problems in optimization-- which, turns out, look a lot like systems of nonlinear equations, as well.
And we'll try to leverage what we learn in the next few lectures to solve different optimization problems.
Before going on, I just want to recap.
All right, last time we talked about singular value decomposition-- which is like an eigenvalue decomposition for any matrix.
And associated with that matrix are singular vectors, left and right singular vectors.
And the singular values of that matrix.
Your TA, Kristen, reminded me that you can actually define a condition number for any matrix, as well.
The condition we gave originally was associated with solving square systems of equations that actually have a solution.
But there is a condition number associated with any matrix, and it's defined in terms of its singular values.
So if you go back and look at the definition of the two norm, you try to think about the condition number associated with the two norm, you'll see that the conditions of any matrix maybe can be defined as the square root of the ratio of the biggest and the smallest singular values of that matrix.
So there's a condition number associated with any matrix.
The condition number as an entity makes most sense when we're thinking about how we amplify errors-- numerical errors, solving systems of equation zones.
It's most easily applied to square systems that actually have unique solutions, but you can apply it to any system of equations you want.
And the singular value decomposition is one way to tap into that.
OK, the last thing we did discussing linear algebra was to talk about iterative solutions, the systems of linear equations.
And that's actually our hook into solutions to systems of nonlinear equations.
It's going to turn out that exact solutions are hard to come by for anything but linear systems of equations.
And so we're always going to have these iterative algorithms, where we refine initial guesses for solutions until we converge to something that's a solution to the problem that we wanted before.
And one question you should ask, when you do these sorts of iterations, is when do I stop?
I don't know the exact solution to this problem.
I can't say I'm close enough-- what does close enough even mean?
So how do I decide to stop?
Do you have any ideas or suggestions for how you might do that?
You've done some of this already in a homework assignment.
But what do you think?
How do you decide to stop?
Yeah?
[INAUDIBLE]
OK, so this is one suggestion-- look at my current iteration, and my next iteration, ask how far apart are these two numbers?
If they're sufficiently far apart, seems like I've got some more steps to make before I converge to my solution.
And if they're sufficiently close together, the steps I'm taking are small enough that I might be happy accepting this solution as a good approximation.
That's called the step norm criteria-- I'll give you a formalization of that later-- it's called the step norm criteria.
How big are the steps that I'm taking?
And are they sufficiently small that I don't care about any future steps?
Another suggestion?
I've got a question
Yeah?
[INAUDIBLE] absolute [INAUDIBLE] when we did that for homework, I tried to do it [INAUDIBLE],, and [INAUDIBLE]
Yes.
I will show you a definition of the step norm criteria that integrates both relative and absolute error into the definition.
And we'll see why, OK?
One problem may be-- what if the solution I'm trying to converge to is 0?
How do you define the relative error with respect to the number 0?
There isn't one-- there is only absolute error when you're trying to converge to 0.
So you may want to have some measure of both absolute and relative step size in order to determine whether you're converged.
Is that the only way to do it, though?
Have any ideas, alternative proposals for deciding convergence?
[INAUDIBLE] the residual
The residual.
OK, what's the residual?
[INAUDIBLE]
Good, OK, so we asked, how good a solution-- we can ask how good a solution is this value that we've converged to by putting it back into the original system of equations, and asking, how far out of balance are we?
I take my best guess for solution x, and multiply it by and A, and I subtract b-- we call that the residual.
And is the residual sufficiently converged, or not?
If it's small enough in magnitude, then we would say, OK, maybe we're sufficiently close to the solution.
If it's too big, then we say, OK, let's iterate some more until we get there.
That is called the function norm criterion.
We're going to talk about these in detail, as applied to systems of nonlinear equations.
But these same criteria apply to all iterative processes.
Neither is preferred over the other.
You don't know the exact solution-- you have no way of measuring how close or far away you are from the exact solution.
So usually you use as many tools as possible to try to judge how good your approximation is.
But you don't know for certain.
What do you do, when that's the case?
We haven't really talked about this in this class.
You have a problem, you program it into a computer, you get a solution.
Is at the end of the story?
We just stop?
You get a number back out, and that's the answer?
How do you know you're right?
How do you know?
We talked about numerical error-- every calculation has a numerical error-- how do you know you got the right answer?
What do you think?
[INAUDIBLE]
OK, yeah, this is true-- so you plug your solution back into the equation, you ask, how good a job does it do satisfying that?
But maybe this equation is relatively insensitive to the solution you provide.
Maybe many solutions nearby look like they also satisfy the equation, but those solutions are sufficiently far apart.
So how do you--
[INAUDIBLE]
OK, so that sort of physical reasoning is a good one.
In your transfer class, you'll talk about doing asymptotic expansions, or asymptotic solutions to problems.
Solve this complicated problem in a limit where it has some analytical solution, and figure how the solution scales, with respect to different parameters.
So you can have an analytical solution that you compare against your numerical solution in certain limits.
You have experiments that you've done.
Experiment-- that's the reality.
The computer is a fiction that's trying to model reality, so you can compare your solution to experiments.
You could also solve the problem a bunch of different ways, and see if all these answers converge in the same place.
So we're going to talk about solving nonlinear equations-- we're going to need different initial guesses for our iterative methods.
We might try several different initial guesses, and see what solutions we come up with.
Maybe we converge all to the same solution, or maybe this problem has some weird sensitivity in it, and we get lots of different solutions that aren't coordinated with each other.
One of the duties of someone who's using numerical methods to solve problems is to try to validate their result-- by solving it multiple times or multiple ways, or comparing against experiment, or against known analytical results, and certain limits where the answer should be exact.
But you can't just accept what the computer tells you-- you have to validate it against some sort of external solution that you can compare with.
Sometimes it's hard to come up with that solution, but it's immensely important.
We know every calculation can be an error.
And as we go on to more complicated problems, it's even more important to validate things.
So, systems of nonlinear equations-- so these are problems of a type f of x equals 0, where x is some vector of unknowns, and has dimension N. And f is a function that takes as input vectors of dimension N, and gives an output a vector of dimension N. It's a map from R N to R N. But it's no longer necessarily a linear map-- it can be some nonlinear function of all the elements of this x.
And the solution to this equation, the particular solution of this equation, x-- for which f of x equals 0-- are called the roots of this vector-valued function.
The linear equations, then, are just represented in this form as A x minus b, A x minus b equals 0-- it's the same as the linear equations we were solving before.
So we're searching for the roots of these functions.
Common chemical engineering examples include equations of state, often nonlinear, in terms of the variables we're interested in.
Energy balances have lots of non-linearities introduced into them.
Mass balances with nonlinear reactions.
Or reactions that are non-isothermal, so their kinetics are sensitive to temperature, and temperatures are variable, we want to know.
These sorts of nonlinear equations crop up all over the place, and you want to be able to solve them reliably.
Here's a simple, simple example.
The Van der Waals equation of state-- here I've written it in terms of reduced pressure, temperature, and molar volume.
Nonetheless, this is the Van der Waals equation of state.
And somebody told you once that if I plot pressure versus molar volume for different temperatures, I may see that, at a given pressure, there could be just one root-- one possible molar volume that satisfies the equation of state.
Or there can be one, two, three potential roots.
It turns out we don't know, with nonlinear equations, how many possible solutions there are.
We knew, for linear equations, I either had zero, one, or an infinite number of solutions.
But with nonlinear equations, in general, there's no way to predict them.
This problem can be transformed into a polynomial, and polynomials are one of the few nonlinear equations where we know how to bound the possible number of solutions.
So we can transform this nonlinear equation to the form I showed you before-- we just move 8/3 T to the other side of this equation.
We want to find the roots of this equation.
So possibly, given pressure and temperature-- pressure and temperature-- find all the molar volumes that satisfy the equation of state.
This is actually overly simplified for a particular physical problem, of looking at vapor liquid coexistence of the Van der Waals fluid.
You can't specify pressure and temperature independently-- the saturation pressure, the coexistence pressure depends on the temperature as the Gibbs phase rule.
So actually, phase equilibria is made up of three parts-- there's thermal equilibrium.
I'm going to add two phases, a gas and a liquid, and they have to have the same temperature, otherwise they're not in equilibrium.
There's got to be mechanical equilibrium of two phases, the gas and the liquid.
They better have the same pressure, otherwise one is going to be pushing harder on the other one.
They'll be in motion-- that's not in equilibrium.
They've got to have the same chemical potential-- they have to be in chemical equilibrium.
So there can't be any net mass flux from one phase to another, otherwise, one phase is going to grow and the other is going to shrink.
They're not in equilibrium with each other.
So actually, the problem of determining vapor liquid coexistence, in this Van der Waals fluid, involves satisfying a number of different equations, some of which are nonlinear, and are constrained by the equation of state.
Given the temperature, there are three unknowns-- the pressure, and the more volumes of the gas and the liquid.
And there are three nonlinear equations we have to solve.
Two of those are the equation of state, in the gas and the liquid-- I'll show them to you in a second-- and the other is this Maxwell equal area construction, which essentially says that the chemical potential in the two phases is equal.
This is one way of representing that.
So we have to solve this system of nonlinear equations for the saturation pressure, the molar volume of the gas or vapor, and the molar volume of the liquid.
And these are those equations.
Here's the equation of state and the gas, here's the equation of state in the liquid, and here's the Maxwell equal area construction at defined values of P sat, V G and V L that satisfy all three equations.
There's not going to be an analytical way to do this-- it has to be done numerically.
Here's a simplification that I can make, though.
So I can take that equal area construction, and I can solve for P sat in terms of V G and V L. And so that reduces the dimensionality of these equations from three to two.
And when it's two dimensional, I can plot these things, so that's helpful.
So let's plot f 1-- the equation of state in the gas is a function of V G and V L. Where that's equal to 0, that's this black curve here.
Let's plot f 2-- the equation of state in the liquid as a function of V G and V L, where that's equal to 0, that's this blue curve here.
And the solutions are where these curves intersect.
So we're seeking out the specific points graphically where these curves intersect.
First, this solution, and that solution aren't the solutions we're interested in at all.
That would say that the molar volume of the gas and the liquid is the same-- that's not really phase separation.
We want these heterogeneous solutions out here.
So we need some methodology that can reliably take us to those solutions.
We'll see that that methodology-- the most reliable methodology, and one of the ones that converges fastest to the solutions-- is called the Newton-Raphson method.
But even before we do that, let's talk more about the structure of systems of nonlinear equations, and what sort of solutions we can expect.
Does this example makes sense to everyone?
Have you thought about this before, maybe?
Yeah.
So given a function, which is a map from R N to R N, find the special solution, x star, such that f of x star equals 0.
That's our task.
And there could be no solutions.
There can be one to an infinite number of locally unique solutions, and there can be an infinite number of solutions.
A solution is to be locally unique if I can wrap that solution by some ball of points, in which there are no other solutions.
That ball can be very, very small, but as long as I can wrap that solution in some ball of points, which are not solutions, we term that locally unique.
So consider the simple function, one which depends on two variables-- x1 and x2, and f 2, which depends on x1 and x2, equals 0.
And I'm going to plot, in the x1, x2 plane, were f 1 and f 2 are equal to 0, so that these curves here.
And here, we have a locally unique solution-- we see the curves cross at exactly one point.
Here, you can see these two curves are tangent with each other.
They could be coincident with each other, over some finite distance.
In which case there's a lot of solutions that live on some locally tangent area.
Or they could just touch in one point.
So they may be tangent, and the solutions there are not locally unique, or they may touch at one point, and the solutions are-- there's one solution, and it's locally unique there.
The reason why we talk about locally unique solutions is it's going to be hard for a numerical method to find anything that's not locally unique in a reliable way.
Locally unique solutions, numerical methods can find very reliably.
But if they're not locally unique?
My iterative method could converge to any one of these solutions that lives on this line, any of these tangent points.
And I'm going to have a hard time predicting which one it's going to go to.
That's a problem if you're trying to solve something reliably over and over again-- if I converge to one of these solutions, or another solution, or another solution, the data that comes out of that process isn't going to be easy to interpret.
There's something called the inverse function theorem, which says if f of x star is equal to 0, and the determinant of this matrix, J, which we call the Jacobian, evaluated at x star is not equal to 0, then x star necessarily is a locally unique solution.
So it's the inverse function theorem.
The Jacobian is a matrix of the partial derivatives of f, if an elements of f, with respect to different elements of x.
So the first row of the Jacobian is the derivatives of the first elements of f, with respect to all the elements of x, and the other rows proceed accordingly.
If the determinant of this matrix, evaluated at the solution, is not equal to 0, then this solution is necessarily locally unique.
That's the inverse function theorem.
The Jacobian describes the rate of change of this vector-valued function with respect to all of its independent variables.
And you may find that, for some solutions, the determinant in the Jacobian is equal to 0.
We can't really say what's going on there-- the solution may be locally unique, it may not be locally unique.
I'm going to provide you some examples in a second.
And most numerical methods are only going to find one of these local unique solutions at a time.
If we have some non-local solutions, that'll cause us problems.
So we tend to want to work with functions that have locally unique solutions to begin with.
OK, here's an example.
Oh, you have your notes, so you know the formula for the Jacobian.
Compute the Jacobian of this function.
So this function has a root at x1 equals 0, x2 equals 0.
If you think graphically about what each of these little functions represents, you would agree that that route is locally unique-- it's just one point where both elements of this vector-valued function are equal to 0.
Here's what the Jacobian of this function should be.
If you take the derivative of the first element with respect to x1, and then x2, take the derivative of the second element with respect to x1 and then x2-- at the solution, at the root of this function, where x1 is 0, and x2 is 0, the Jacobian is a matrix of zeros.
Its determinant is 0.
But the solution is locally unique.
The inverse function theorem only tells us about what happens when the determinant's not equal to 0.
If the determinant's not 0, then we have a locally unique solution.
Solution may be locally unique, its determinant may be equal to 0.
Does that makes sense?
You see how that plays out?
OK.
There's a physical way to think about-- or a geometric way to think about this inverse function theorem.
So think about the linear equation, f of x is A x minus b.
You can show-- and you should actually work through this-- that the Jacobian of this function is just the matrix A.
It says how the function changes, with respect to small changes in x.
Well, that's just A-- this is a linear function.
So the equation, f of x equals 0, has a locally unique solution when the determinant of the Jacobian-- which is the determinant of A-- is not equal to 0.
But you already knew that, right?
We already talked through linear algebra.
And so you know when this matrix A is singular, then we can't invert this system of equations and find a unique solution in the first place.
So the inverse function theorem is nothing more than an extension of what we learned about when functions are and aren't invertible.
Because a locally unique solution when A is invertible.
In the neighborhood of f of x, in the neighborhood of a root of f of x, we can often approximate the function as being linear.
We can treat it as though it's a system of linear equations, very close to that root.
And then the things that we learned from linear algebra are inherited by these linearized solutions.
So here's this set of curves that I showed you before.
Near this root, let's zoom in-- let's zoom in.
These lines look mostly straight.
It's like the place where two planes intersect-- intersect this x1, x2 plane-- they each intersect at a line, and the crossing of those lines is the solution.
And it's locally unique.
Because these two planes span different subspaces.
Here's the case where we may have non-locally unique [INAUDIBLE].
Zoom in on this root, and very close to this root, well, it's hard to tell.
Maybe these two planes are coincident with each other, and they intersect and form the same line-- in which case they may not be locally unique.
Maybe if I zoom in close enough, I see, no, actually, they have a slightly different orientation with respect to each other, and there is a locally unique solution there.
It's difficult to tell here.
So these cases where the curves cross are easy to determine.
These are the ones that the inverse function theorem tells us about.
These ones are a little harder to work with.
So I mentioned that you can zoom in, and look close to a root, and approximate the function is linear.
This is a process called linearization.
You are in this for 1-D functions-- f of x, at a point x plus delta x, is f of x plus its derivative times delta x.
And this'll typically be valid for reasonably well-behaved functions-- this sort of a linearization is going to be valid as delta x goes to 0.
So as long as I haven't moved too far away from the point to x, I can approximate my function in the neighborhood of x using this linearization.
You know, turns out the same thing is true for nonlinear functions.
So f of x plus delta x is f of x plus-- well, I need the derivatives of my function with respect to x, and those derivatives are partial derivatives now, because we're in higher dimensional spaces.
That's the Jacobian multiplied by this vector of displacements, delta x.
And this will typically be valid as long as the length of this delta x is not too big-- as long as I haven't moved too far away from the point I'm interested in, f of x, this will be a reasonably good approximation.
As long as our functions are well-behaved.
There's an error that's incurred in making these approximations.
And for a general function that's well-behaved, that error is going to be ordered delta x squared-- and they're in the 1-D, or the multi-dimensional case.
And this sort of an expansion, it's just part of a Taylor expansion for each component of f of x.
So we take element I of f. I want to know its value at x plus delta x.
That's its value at x.
Plus the sum of partial derivatives of that element, with respect to each of the elements of x, times delta x-- each of those delta x's.
Plus, there are some high order term in this Taylor expansion, which is quadratic in delta x instead.
There's a cubic term, and so on.
These quadratic terms are what give rise to this order, delta x squared error.
In higher dimensions, we typically don't worry about these quadratic terms.
We're pretty satisfied with linearization of our system of equations.
Sometimes for 1-D nonlinear functions, you can take advantage of knowing what these quadratic terms are to do some funny things.
But in many dimensions, you usually don't use that.
Usually you just think about linearizing the solution.
So if I know where the solution is, if I know it's close, if I can figure out points that are close to the solution, then I can linearize the function in that neighborhood-- I can find the solution to the linearized equation, instead.
That's going to be suitably close to the exact solution.
Does that makes sense?
Nonlinear equations, like I said, are solved iteratively.
Which means we make a map in our algorithmic map which takes some value xy and generates some new value x plus 1, which is a better approximation for the solution we're after.
And we designed the map so that the root, x star, is what's called a fixed point of the map.
So if I put x star in on this side, I get x star in on the other side.
By design, the root is a fixed point of the map.
The map may converge, or it may not converge, but the root is a fixed point.
And we'll stop iterating when the map is sufficiently converged.
You guys came up with two different criteria for stopping.
One is called the function norm criteria.
I look at how big my function is-- I'm trying to find the place where the function is equal to 0.
So I look at how big, in the norm space, my function is for my current best solution, and ask if it's smaller than some tolerance epsilon.
If it is, then I say, well, my function is sufficiently close to 0-- I'm happy with this solution.
The solution is close enough to satisfying the original equation that I'll accept it, and I stop.
The other criteria, it's called the step norm criteria.
I look at two successive approximations for the solution.
I take their difference, and ask, is the norm of that difference smaller than either some absolute tolerance, or some relative tolerance, multiplied by the norm of my current solution?
So suppose x is a large number, that spacing between these x's may be quite big, but the relative spacing may actually be quite small.
And if the relative spacing is small enough, you might say, well, this is sufficiently converged.
And so that's where this relative error, relative error tolerance, comes into play in the step norm criteria.
Suppose x is a small number, close to 0 instead.
These steps may be very tiny-- these steps may be quite tiny.
They may satisfy this relative criteria quite well, but you may want to put some absolute tolerance on how far these steps are before you stop instead.
Because these x's may be small in and of themselves.
And so this one is easy to satisfy, but this one becomes harder.
So you usually use both of these.
Sometimes you have solutions that are converging toward small numbers, and then the absolute error tolerance becomes important.
Sometimes you have solutions that are converging towards large numbers, and so the relative error tolerance becomes important.
Does that make sense?
Of course, you can't just use this one, or just use that one.
You typically like to use all of these.
Because they can fail.
And if the function norm criterion fail-- here's an example where I'm taking some iterations, some approximate solutions that are headed towards the actual root of this function.
And at some point, I find that this solution is within epsilon of 0.
And so I'd like to accept this solution, but graphically it looks like it's very far away from the root.
So this is a case where the function has a very shallow slope.
It's a very shallow slope, and the functional criteria, not so good, really.
I call this a solution, but it's quite a ways away from star.
So sometimes it's going to work, but sometimes it's not going to work.
Here's the step norm criteria-- here, I have a function nowhere near a root-- I have no idea where I am on this function, I don't know what value this function has, but my steps suddenly got small enough that they're smaller than this absolute tolerance, or they're smaller than the relative error tolerance.
I might say, OK, let's stop.
I'm not taking very large steps anymore, this seems like a good place to quit.
But actually, my function just after this didn't go to 0 at all, it curved up and went the other way.
There's not even a solution nearby.
So both of these things can fail, and we try to use both of them instead to evaluate whether we have a reasonable solution to our nonlinear equation or not.
Make sense?
Are there any questions about that before you I go on?
No.
OK. We also talk oftentimes about the rate of convergence of the iterative process that we're using.
We might say it converges linearly, or quadratically.
And the rate of convergence is always assessed by looking at the difference between successive-- well, we look at the ratio of differences for successive approximation.
So here's the difference between my best approximation, step i plus 1 minus the exact solution, normed, divided by my best approximation at step i, minus the exact solution normed, and raised to some power, q.
And as I go to very large numbers of iterations, i-- this limit should be i, I apologize.
I'll fix that in the notes online, but this limit should be i-- is i, the number of steps gets very large.
This ratio should converge to some constant.
And the ratio will converge to a constant when I choose the right power of q here.
So when this limit exists, and it doesn't go to 0, we can identify what sort of convergence we get.
So if q equals 1, and C is smaller than when we say that convergence is linear, what is that saying, really?
This top step here is the absolute error, an approximation i plus 1.
This bottom step here is the absolute error in step i-- remember two is one for linear convergence.
So the ratio of absolute errors, as long as that's less than 1-- I'm converging, I'm moving my way towards the solution.
And we say that rate is linear.
If C is 10 to the minus 1, then each iteration will be one digit more accurate than the previous one.
The absolute error will be 10 times smaller in the next iteration versus the previous one.
That would be great-- usually C isn't that small.
If this power, for which this limit exists, q, is bigger than 1, we say the convergence is super linear.
If q is 2, which we'll see is something that results from the Newton-Raphson method, then we say convergence is quadratic.
What that means is the number of accurate digits in my solution will actually double with each iteration.
Linear, which equals 10 to the minus 1, I get one digit per iteration.
Quadratic, I double the number of digits per iteration-- I have one digit on one iteration, I get two the next one, and four the next one, and eight the next one.
So quadratic convergence is marvelous.
Linear convergence, that's OK. That's about the minimum you'd be willing to accept.
Quadratic convergence is great, so we aim for methods that try to have these higher order convergences.
So you really quickly get highly accurate solutions.
You can go back and look at your notes and see that the Jacobian method, and the Gauss-Seidel method both show linear convergence rates.
They're linear methods.
Does this make sense?
OK.
So I meant it mentioned Newton-Raphson.
Hopefully, somebody at some point told you about the Newton-Raphson method for solving at least one-dimensional, nonlinear equations.
It goes like this, though.
You say, I guess my solution is close to this green point here.
Let me linearize my function at that point, and find where that linear approximation has a root-- which is this next green point.
And then repeat that process here.
I find the linearization of my function, this pink arrow.
I look for where that linear function has a root, and that's my next best approximation.
And I repeat this process over and over, and it will reliably-- under certain circumstances-- converge to the root.
What does that look like, in terms of the equations?
So I linearized my function, so I want to approximate f of x at i plus 1, in terms of f of x at i-- so it's f of x at i, plus the derivative, multiplied by the difference between x i plus 1 and x i.
And I say, find the place where this approximation is equal to 0, and determine what the next point that I'm going to use to approximate my solution is.
So I solve for x i plus 1, in terms of x i.
How big of a step do I take from x i to x i plus 1?
It's this big, so the ratio of the function to its derivative at x i.
And the derivative does the job of telling me which direction I should step in.
Derivative gives me directionality, and this ratio here tells me the magnitude of the step.
The magnitudes, you know, they're not very good oftentimes, because these functions that we're trying to solve aren't very linear.
Usually they're highly nonlinear.
What's really helpful is getting the direction right.
You could go right, you could go left-- only one of those ways is getting you to the root.
Newton-Raphson has this advantage-- it always points you in the right direction.
OK?
Of course, you can do this in any number of dimensions, not just one dimension.
So you can approximate your function as linear-- f of x i plus 1 is approximately 0.
And then let's take our linearized version of the function, and let's find where it's equal to 0.
Sometimes what's done is to replace this difference, x i plus 1, minus x i, with an unknown vector, d i-- which is the step size.
How big a step am I going to take from x i to x i plus 1?
And so we solve this equation for the displacement, d i.
Move f to the other side, so you have Jacobian times d i is minus f. And solve-- d i is Jacobian inverse times f. It's just a system of linear equations.
Now we know our step size.
So x i plus 1 is x i plus b, or x i plus 1 is x i minus Jacobian inverse times f. The inverse of the Jacobian plays the same role is 1 over the derivative.
It's telling us what direction to step in, in this multi-dimensional space.
And this solution to the system of equations is giving us a magnitude of the step that's good, not great, but is taking us closer and closer to the root.
So this is the Newton-Raphson method applied to the system of nonlinear equations.
This is really the way you want to solve these sorts of problems.
It doesn't always work-- things can go wrong.
What sorts of things go wrong here?
Can you guess?
Yeah?
[INAUDIBLE]
OK, this is good.
So in the 1-D problem, sometimes the Newton-Raphson method can get stuck.
So it won't have good necessarily global convergence properties.
If you have a bad initial guess, it might get stuck someplace, and the iterates will converge.
That can be true.
What else can go wrong?
[INAUDIBLE]
Good, so if you're derivative is 0, that's going to be problematic.
What's the multi-dimensional equivalent of the derivative being 0?
[INAUDIBLE]
What's that?
Singular Jacobian, right?
So if this is J, the Jacobian, has some null space associated with it, how am I supposed to figure out which direction to step in?
There's some arbitrariness associated with the solution of this system of equations.
So the derivative is 0 in the 1-D example, that's a problem.
That problem gets a little fuzzier, but it's still a big problem when we try to solve for the step size, or the step-- the Newton-Raphson step.
They may not be able to do this.
You don't run into this very often, but you can, from time to time.
One place where this is going to happen is if we have a non-locally unique solution.
When we have one of those, we know the determinant of the Jacobian at that point is going to be 0.
If we're close to those solutions, well the determinant of the Jacobian is going to be close to 0-- you might expect that the system of equations you have to solve becomes ill-conditioned.
So even though there may be an exact solution for all the steps leading up to that point, the equations may become ill-conditioned.
You may not be able to reliably find those solutions with your computer, either.
So then these steps you take, well, who knows where they're going at that point.
It's going to be crazy.
There are ways of fixing all these problems.
Let's do an example.
This is a geometry example, but you can write it is a system of nonlinear equations, as well.
So we have two circles-- circle f 1, circle f 2 in the x1, x2 plane.
They satisfy-- these are the locus of points, that satisfy the equation f 1 of x 1 and x 2 equals 0-- this is the equation for one circle, and this is the equation for the other circle.
And we want the solution, we want the roots of this vector-valued function, f, for vector-valued x.
And those are the intercepts between these two circles.
You can do it using Newton-Raphson, so you're going to need to know the Jacobian.
So compute the Jacobian of this function.
This is practice-- maybe most of you know how to compute a Jacobian, but some people haven't done it before.
So it's always good to make sure you remember that the first row of the Jacobian is the derivatives of the first element of f. And later rows are later elements.
You don't want the transpose of the Jacobian.
Then it won't work.
OK, so should look something like this.
There's your Jacobian.
The Newton-Raphson process tells us how to take steps from one approximation to the next.
The step is equal to minus the Jacobian inverse evaluated, and my best guess for the solution multiplied by the function-- evaluated at my best guess of the solution.
You're never going to compute Jacobian inverse explicitly-- that's just code for solve this system of linear equations.
So use the slash operator in MatLab, for example.
And here you go-- I had an initial guess for the solution, iterate 0 at minus 1 and 3.
This is somewhere outside the circles-- it's pretty far away from the solutions.
But I do my Newton-Raphson steps, I iterate on and on.
And after four steps, you can see that the step size in absolute value is order 10 to the minus 3.
The function norm is order 10 to the minus 3, as well-- and maybe order 10 to the minus 2, but it's getting down there.
These things are decreasing pretty quickly.
And I move to a point that you'll see is pretty close to the solution.
Here's some things you need to know about Newton-Raphson, you'll want to think carefully about as we go forward.
So it possesses a local convergence property.
I'm going to illustrate that graphically for you.
So here, I didn't solve the problem once, I solved it-- I don't know, 10,000 times.
And I chose different initial points to start iterating with.
Here, minus 1, 3, that was one point.
But I chose a whole bunch of them.
And I asked, how many iterations-- how many steps did my Newton-Raphson method have to take before I got sufficiently close to either this root here, or this root there?
I don't remember what that convergence criterion was-- it doesn't really matter, but there was some 10 to the minus 3, or 10 to the minus 5 or 10 to the minus 8, the convergence criterion that I made sure the Newton-Raphson method hit, in both the function norm and step norm cases.
And then, if the color on this map is blue-- the solution converged to this star in the blue zone-- if the color's orange, the solution converged to the star in the orange zone.
And if the color is light, it didn't take so many iterations to converge.
And if the color gets darker, it takes more and more iterations to converge.
So that's the picture-- that's this map.
I solved it a bunch of times, and then I mapped out how many iterations did it take me to converge to different solutions?
So you can see if I start close to the solution, the color is really light.
It doesn't take very many iterations to get there.
And the further way I move in this direction-- still doesn't seem like it take so many iterations to get there, either.
I need a good initial guess-- I want to be close to where I think the solution is.
Because once I'm over here somewhere, I do pretty well.
And the same is true on the other side, because this problem is symmetric.
There's a line down the middle here, and along this line, the determinant of the Jacobian is equal to 0.
So we talked about these points with the Newton-Raphson method.
And if I pick initial guesses sufficiently close to this line, you can see the color gets darker and darker.
The number of iterations required to converge the solution goes way up.
Now, the Newton-Raphson method possesses a local convergence property.
Which means, if a locally unique solution, there's always going to be some neighborhood around that solution for which the determinant of the Jacobian is not equal to 0.
And in that neighborhood, I can guarantee that this iterative process will eventually reach the solution.
That's pretty good.
That's handy.
These iterates can go anywhere-- how do you know you're getting to the solution?
Are you going to waste your time, or are you going to get there?
So that's this local convergence property associated with it, which is nice.
But it'll break down as I get to places where the determent of the Jacobian is equal to 0. so there could be a zone-- it's not in this one-- there could be a zone, for example, like a ring on which the determinant of the Jacobian sequence is 0.
And if I take a guess inside that ring, who knows where that solution is going to go to.
Could be something like Sam said, where the iterative method just bounces around inside that ring and never converges.
But when I have roots that are locally unique, and I start with good guesses close to those roots, I can guarantee the Newton-Raphson method will converge.
I'll show you next time that not only does it converge, but it also converges quadratically.
So you start sufficiently close to the solution, you get to double the number of accurate digits in each iteration.
You can see that happening here.
OK, so I have one accurate digit, now I have two.
The next iteration I'll have four, and so on.
The number of accurate digits is going to double at each iteration.
So that's going to conclude for today.
Next time, we'll talk about how to fix these problems with the Newton-Raphson method.
So there are going to be cases where the convergence isn't ideal, where we can improve things.
There are going to be cases where we don't want to compute the Jacobian or the Jacobian inverse.
And we can improve the method.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Should we begin?
Let me remind you, we switched topics.
We transitioned from linear algebra, solving systems of linear equations, to solving systems of nonlinear equations.
And it turns out, linear algebra is at the core of the way that we're going to solve these equations.
We need iterative approaches.
These problems are complicated.
We don't know how many solutions there could be.
We have no idea where those solutions could be located.
We have no exact ways of finding them.
We use iterative methods to transform non-linear equations into simpler problems, right?
Iterates of systems of linear equations.
And the key to that was the Newton-Raphson method.
So I'm going to pick up where we left off with the Newton-Raphson method, and we're going find out ways of being less Newton-Raphson-y in order to overcome some difficulties with the method, shortcomings of the method.
There are a number of them that have to be overcome in various ways.
And you sort of choose these so-called quasi- Newton-Raphson methods as you need them.
OK, so you'll find out.
You try to solve a problem.
And the Newton-Raphson method presents some difficulty, you might resort to a quasi Newton-Raphson method instead.
Built into MATLAB is non-linear equations solver.
fsolve.
OK, it's going to happily solve systems of nonlinear equations for you, and it's going to use this methodology to do it.
It's going to use various aspects of these quasi- Newton-Raphson methods to do it.
I'll sort of point out places where fsolve will take from our lecture and implement them for you.
It will even use some more complicated methods that we'll talk about later on in the context of optimization .
Somebody asked an interesting question, which is how many of these nonlinear equations am I going to want to solve at once?
Right?
Like I have a system of these equations.
What does a big system of nonlinear equations look like?
And just like with linear equations, it's as big as you can imagine.
So one case you could think about is trying to solve, for example, the steady Navia-Stokes equations.
That's a nonlinear partial differential equation for the velocity field and the pressure in a fluid.
And a at Reynolds number, that non-linearity is going to present itself in terms of inertial terms that may even dominate the flow characteristics in many places.
We'll learn ways of discretizing partial differential equations like that.
And so then, at each point in the fluid we're interested in, we're going to have a non-linear equation that we have to solve.
So there's going to be a system of these non-linear equations that are coupled together.
How many points are there going to be?
That's up to you, OK?
And so you're going to need methods like this to solve that.
It sounds very complicated.
So a lot of times in fluid mechanics, we have better ways of going about doing it.
But in principle, we've have any number of nonlinear equations that we want to solve.
We discussed last time, the new Newton-Raphson method, which was based around the idea of linearization.
We have these nonlinear equations.
We don't know what to do with them.
So let's linearize them, right?
If we have some guess for the solution, which isn't perfect, but it's our best possible guess.
Let's look at the function and find a linearized form of the function and see where that linearized form has an intercept.
And we just have an Ansatz.
We guess that this is a better solution than the one we had before.
And we iterate.
It turns out you can prove that this sort of a strategy-- this Newton-Raphson strategy is locally convergent.
If I start with a guess sufficiently close to the root, you can prove mathematically that this procedure will terminate with a solution at the root, right?
It's going to approach after an infinite number of iterates, the root.
That's wonderful.
It's locally convergent, not globally convergent.
So this is one of those problems that we discussed.
Take a second here, right?
Here's your Newton-Raphson formula.
You've got it on your slides.
Take a second here and-- this is sort of interesting.
Derive the Babylonian method, right?
Turns out the Babylonians didn't know anything about Newton-Raphson but they had some good guesses for how to find square roots, right?
Find the roots of an equation like this.
See that you understand the new Newton-Raphson method by deriving the Babylonian method, right?
The iterative method for finding the square root of s as the root of this equation.
Can you do it?
[SIDE CONVERSATION]
Yes, you know how to do this, right?
So calculate the derivative.
The derivative is 2x.
Here's our formula for the iterative method.
Right?
So it's f of x over f prime of x.
That sets the magnitude of the stop.
The direction is minus this magnitude.
It's in one d, so we either go left or we go right.
Minus sets the direction.
We add that to our previous guess.
And we have our new iterate, right?
You substitute f and f prime, and you can simplify this down to the Babylonian method, which said take the average of x and s over x.
If I'm at the root, both of these should be square root of s. And this quantity should be zero exactly, right?
And you'll get your solution.
So that's the Babylonian method, right?
It's just an extension of the Newton-Raphson method.
It was pretty good back in the day, right?
Quadratic convergence to the square root of a number.
I mentioned early on computers got really good in computing square roots at one point, because somebody did something kind of magic.
They came up with a scheme for getting good initial guesses for the square root.
This iterative method has to start with some initial guess.
If it starts far away, it'll take more iterations to get there.
It'll get there, but it's going to take more iterations to get there.
That's undesirable if you're trying to do fast calculations.
So somebody came up with some magic scheme, using floating point mathematics, right?
They masked some of the bits in the digits of these numbers.
A special number to mask those bits.
They found that using optimization, it turns out.
And they got really good initial guesses, and then it would take one or two iterations with the Newton-Raphson method to get 16 digits of accuracy.
That's pretty good.
But good initial guesses are important.
We'll talk about that next week on Wednesday.
Where do those good initial guesses come from?
But sometimes we don't have those available to us.
So what are some other ways that we can improve the Newton-Raphson method?
That will be the topic of today's lecture.
What's the Newton-Raphson method look like graphically in many dimensions.
We talked about this Jacobian.
Right, when we're trying to find the roots of a non-linear equation where our function has more than one dimension-- let's say it has two dimensions.
So we have an f 1 and an f 2.
And our unknowns our x 1 and x 2, they live in the x1 x2 plane, right?
f1 might be this say bowl-shaped function.
I've sketched out in red, right?
It's three dimensional.
It's some surface here.
Right?
We have some initial guess for the solution.
We go up to the function, and we find a linearization of it, which is not a line but a plane.
And that plane intersects the x 1, x 2 plane at a line.
And our next best guess is going to live somewhere on this line.
Where on this line depends on the linearization of f 2.
Right?
So we got to draw the same picture for f 2, but I'm not going to do that for you.
So let's say, this is where the equivalent line from f 2 intersects the line from f 1, right?
So the two linearizations intersect here.
That's our next best guess.
We go back up to the curve.
We find the plane that's tangent to the curve.
We figure out where it intersects.
The x 1 x 2 plane.
That's a line.
We find the point on the line that's our next best guess, and continue.
Finding that intersection in the plane is the act of computing Jacobian inverse times f. OK?
If we project down to just the x 1 x 2 plane, and we draw the curves where f 1 equals 0, and f 2 equals zero, right?
Then each of these iterates, we start with an initial guess.
We find the planes that are tangent to these curves, or to these surfaces.
And where they intersect the x 1 x 2 plane.
Those give us these lines.
And the intersection of the lines give us our next approximation.
And so our function steps along in the x1 and x2 plane.
It takes some path through that plane.
And eventually it will approach this locally unique solution.
So that's what this iterative method is doing, right?
It's navigating this multidimensional space, right?
It moves where it has to to satisfy these linearized equations, right?
Producing ever better approximations for a root.
Start close.
It'll converge fast.
How fast?
Quadratically.
And you can prove this.
I'll prove it in 1D.
You might think about the multidimensional case, but I'll show you in one dimension.
So the Newton-Raphson method said, xi plus 1 is equal to xi minus f of xi over f prime of xi.
I'm going to subtract the root, the exact root from both sides of this equation.
So this is the absolute error in the i plus 1 approximation.
It's equal to this.
And we're going to do a little trick, OK?
The value of the function at the root is exactly equal to zero, and I'm going to expand this as a Taylor series, about the point xy.
So f of xi plus f prime of xi times x star minus xi, plus this second order term as well.
Plus cubic terms in this Taylor expansion, right?
All of those need to sum up and be equal to 0, Because f of x star by definition is zero.
x star is the root.
And buried in this expression here is a quantity which can be related to xi minus f of xi over f prime minus x star.
It's right here, right?
xi minus x star, xi minus x star.
I've got to divide through by f prime.
Divide through by f prime, and I get f over f prime.
That's this guy here.
Those things are equal in magnitude then, to this second order term here.
So they are equal in magnitude to 1/2, the second derivative of f, divided by f prime, times xi minus x star squared.
And then these cubic terms, well, they're still around.
But they're going to be small as I get close to the actual root.
So they're negligible, right?
Compared to these second order terms, they can be neglected.
And you should convince yourself that I can apply some of the norm properties that we used before, OK?
To the absolute value.
The absolute values is the norm of a scalar.
So these norm properties tell me that this quantity has to be less than or equal to, right?
This ratio of derivatives multiplied by the absolute error in step i squared.
And I'll divide by that absolute error in step i squared.
So taking the limit is i goes to infinity, this ratio here is bound by a constant.
This is a definition for the rate of convergence.
It says I take the absolute error in step i plus 1.
I divide it by the absolute error in step i squared.
And it will always be smaller than some constant, as i goes to infinity.
So it converges quadratically, right?
If the relative error in step i was order 10 to the minus 1, then the relative error in step i plus 1 will be order 10 to the minus 2.
Because they got to be bound by this constant.
If the relative error in step i was 10 to the minus 2, the relative error in step i plus 1, has got to be order 10 to the minus 4, or smaller, right?
Because I square the quantity down here.
I get to double the number of accurate digits with each iteration.
And this will hold so long as the derivative evaluated at the root is not equal to zero.
If the derivative evaluated at the root is equal to zero, this analysis wasn't really valid.
You can't divide by zero in various places, OK?
It turns out the same thing is true if we do the multidimensional case.
I'll leave it to you to investigate that case.
I think it's interesting for you to try and explore that.
It follows the 1D model I showed you before.
But the absolute error in iterate i plus 1, divided by the absolute error in iterate i-- here's a small typo here.
Cross out that plus 1, right?
The absolute error in iterate i squared is going to be bound by a constant.
And this will be true so long as the determinant at the Jacobian at the root is not equal to zero.
We know the determinant of the Jacobian plays the role of the derivative in the 1D case.
When the Jacobian is singular, you can show that linear convergence is going to occur instead.
So it will still converge.
It's not necessarily a problem that the Jacobian becomes singular at the root.
But you're going to lose your rate of quadratic convergence.
And this rate of convergence is only guaranteed if we start sufficiently close to the root.
So good initial guesses, that's important.
We have a locally convergent method.
Bad initial guesses?
Well, who knows where this iterative method is going to go.
There's nothing to guarantee that it's going to converge even.
Right It may run away someplace.
Here are a few examples of where things can go wrong.
So if I have a local minima or maxima, I might have an iterate where I evaluate the linearization, and it tells me my next best approximation is on the other side of this minima or maxima.
And then I go up, and I get the linearization here.
And it tells me, oh, my next best approximation is on the other side.
And this method could bounce back and forth in here for as long as we sit and wait.
It's locally convergent, not globally convergent.
It can get hung up in situations like this.
Asymptotes are a problem.
I have an asymptote, which presumably has an effective root somewhere out here at infinity.
Well, my solution would like to follow the linearization, the successive linearizations all the way out along this asymptote, right?
So my iterates may blow up in an uncontrolled fashion.
You can also end up with funny cases where our Newton-Raphson steps continually overshoot the roots.
So they can be functions who have a power loss scaling right near the root, such that the derivative doesn't exist.
OK?
So here the derivative of this thing, if s is smaller than 1, and x equals zero, it won't exist, right?
There isn't a derivative that's defined there.
And in those cases, you can often wind up with overshoot.
So I'll take a linearization, and I'll shoot over the root.
And I'll go up and I'll take my next linearization, I'll shoot back on the other side of the root.
And depending on the power of s associated with this function, it may diverge, right?
I may get further and further away from the root, or it may slowly converge towards that root.
But it can be problematic.
Here's another problem that crops up.
Sometimes people talk about basins of attraction.
So here's a two-dimensional, non-linear equation I want to find the roots for.
It's cubic in nature, so it's got three roots, which are indicated by the stars in the x1 x 2 plane.
And I've taken a number of different initial guesses from all over the plane and I've asked-- given that initial guess, using the Newton-Raphson method, which root do I find?
So if you see a dark blue color like this, that means initial guesses there found this root.
If you see a medium blue color, that means they found this root.
See a light blue color, that means they found this root.
And this is a relatively simple function, relatively low dimension, but the plane here is tilled by-- it's not tiled.
It's filled with a fractal.
These basins of attraction are fractal in nature.
Which means that I could think that I'm starting with a solution rate here that should converge to this green root because it's close.
But it actually goes over here.
And if I change that initial guess by a little bit, it actually pops up to this root over here instead.
It's quite difficult to predict which solution you're going to converge to.
Yes?
And in this case, you knew how many roots there are
Yes
Often you wouldn't know.
So you find one, and you're happy.
Right?
You're happy because [INAUDIBLE] physical.
Might be the wrong one
So this the problem.
I think this is about the minimum level of complexity you need.
Which is not very complex at all in a function to get these sorts of basins of attraction.
Polynomial equations are ones that really suffer from this especially, but it's a problem in general.
You often don't know.
I'll show you quasi Newton-Raphson methods that help fix some of these problems.
How about other problems?
It's good to know where the weaknesses are.
Newton-Raphson sounds great, but where are the weaknesses?
Let's see.
The Jacobian-- might not be easy to calculate analytically, right?
So far we've written down analytical forms for the Jacobian.
We've had simple functions.
But maybe it's not easy to calculate analytically.
You should think about what are the sources for this function, f of x, that we're trying to find the roots for.
Also we got to invert the Jacobian, and we know that's a matrix.
And matrices which have a lot of dimensions in them are complicated to invert.
There's a huge amount of complexity, computational complexity, in doing those inversions.
It can take a long time to do them.
It may undesirable to have to constantly be solving a system of linear equations.
So might think about some options for mitigating this.
Sometimes it won't converge at all?
Or to the nearest root.
This is this overshoot, or basin of attraction problem.
And we'll talk about these modifications to correct these issues.
They come with a penalty though.
OK?
So Newton-Raphson was based around the idea of linearization.
If we modify that linearization, we're going to lose some of these great benefits of the Newton-Raphson method, namely that it's quadratically convergent, right?
We're going to make some changes to the method, and it's not going to converge quadratically anymore.
It's going to slow down, but maybe we'll be able to rein in the method and make it converge either to the roots we want it to converge to or converge more reliably than it would before.
Maybe we'll be able to actually do the calculation faster, even though it may require more iterations.
Maybe we can make each iteration much faster using some of these methods.
OK so here are the three things that we're going to talk about.
We're going to talk about approximating the Jacobian with finite differences.
We're talking about Broyden's method for approximating the inverse of the Jacobian.
And we're going to talk about something called damped Newton-Raphson methods.
Those will be the three topics of the day.
So here's what I said before.
Analytical calculations of Jacobian requires analytical formulas for f. And for functions of a few dimensions, right?
These calculations are not too tough.
For functions of many dimensions, this is tedious at best.
Error prone, at worst.
Think about even something like 10 equations for 10 unknowns.
If your error rate is 1%, well, you're shot.
There's a pretty good chance that you missed one element of the Jacobian.
You made a mistake somewhere in there.
And now you're not doing Newton-Raphson You're doing some other iterative method that isn't the one that you intended.
There are a lot of times where you-- maybe you have an analytical formula for some of these f's, but not all of them.
So where can these functionalities come from?
We've seen some cases, where you have physical models.
Thermodynamic models that you can write down by hand.
But where are other places that these functions come from?
Ideas?
[INAUDIBLE]
Oh, good
[INAUDIBLE]
Beautiful.
So this is going to be the most common case, right?
Maybe you want to use some sort of simulation code, right?
To model something.
It's somebody else's simulation code.
They're an expert at doing finite element modeling.
But the output is this f that you're interested, and the input to the simulation are these x's.
And you want to find the roots associated with this problem that you're solving via the simulation code, right?
This is pretty important being able to connect different pieces of software together.
Well, there's no analytical formula for f there.
OK?
You're shot.
So it may come from results of simulations.
This is extremely common.
It could come from interpretation of data.
So you may have a bunch of data that's being generated by some physical measurement or a process, either continuously or you just have a data set that's available to you.
But these function values are often not, they're not things that you know analytically.
It may also be the case that, oh, man, even Aspen, you're going to wind up solving systems of nonlinear equations.
It's going to use the Newton-Raphson method.
Aspen's going to have lots of these formulas in it for functions.
Whose going in by hand and computing the derivatives of all these functions for aspen?
MATLAB has a nonlinear equation solver in it.
You give it the function, and it'll find the root of the equation, given a guess.
It's going to use the Newton-Raphson method.
Whose computing the Jacobian for MATLAB?
You can.
You can compute it by hand, and give it an input.
Sometimes that's a really good thing to do.
But sometimes, we don't have that available to us.
So we need alternative ways of computing the Jacobian.
The simplest one is a finite difference approximation.
So you recall the definition of the derivative.
It's the limit of this difference, f of x plus epsilon minus f of x divided by epsilon, as epsilon goes to zero.
There's an error in this approximation for the derivative with a finite value for epsilon, which is proportional to epsilon.
So choose a small value of epsilon.
You'll get a good approximation for the derivative.
It turns out the accuracy depends on epsilon, but kind of in a non-intuitive way.
And here's a simple example.
So let's compute the derivative of f of x equals e the x, which is e to the x.
Let's evaluate it at x equals 1.
So F prime of 1 is e the 1, which should approximately be e to the 1 plus epsilon minus e to the 1 over epsilon.
And here I've done this calculation.
And I've asked, what's the absolute error in this calculation by taking the difference between this and this, for different values of epsilon.
You can see initially, as epsilon gets smaller, the absolute error goes down in proportion to epsilon.
10 to the minus 3, 10 to the minus 3.
10 to the minus 4, 10 to the minus 4.
10 to the minus 8, 10 to the minus 8.
10 to the minus 9.
10 to the minus 7.
10 to the minus 10.
And 10 to the minus 6.
So it went down, and it came back up.
But that's not what this formula told us should happen, right?
Yes?
So just to be sure.
That term in that column on the right?
Yes?
It says exponential 1, but it represents the approximation?
Exponential 1 is exponential 1. f prime of 1 is our approximation here
Oh,
Sorry that that's unclear.
Yes, so this is the absolute error in this approximation.
So it goes down, and then it goes up.
Is that clear now?
Good.
OK, why does it go down?
It goes down because our definition of the derivative says it should go down.
At some point, I've actually got to do these calculations with high enough accuracy to be able to perceive the difference between e to the 1 plus, 10 to the minus 9, and e to the 1.
So there is a truncation error in the calculation of this difference that reduces my accuracy at a certain level.
There's a heuristic you can use here, OK?
You want to set this epsilon when you do this finite difference approximation, to be the square root of the machine precision times the magnitude of x, the point at which you're trying to calculate this derivative.
So that's, usually we're double precision.
So this is something like 10 to the minus 8 times the magnitude of x.
That's pretty good.
That holds true here.
OK?
You can test it out on some other functions.
If x is 0, or very small.
We don't want a relative tolerance.
We've got to choose an absolute tolerance instead.
Just like we talked about with the step norm criteria.
So one has to be a little bit careful in how you implement this.
But this is a good guide, OK?
A good way to think about how the error is going to go down, and where it's going to start to come back up.
Make sense?
Good.
OK, so how do you compute elements of the Jacobian then?
Well, those are all just partial derivatives of the function with respect to one of the unknown variables.
So partial f i with respect to x j is just f i at x plus some epsilon deviation of x in its j-th component only.
So this is like a unit vector in the j direction, or associated with the j-th element of this vector.
Minus f i of x divided by this epsilon.
Equivalently, you'd have to do this for f i.
You can compute all the columns of the Jacobian very quickly by calling f of x plus epsilon minus f of x over epsilon.
Just evaluate your vector-valued function at these different x's.
Take the difference, and that will give you column j of your Jacobian.
So how many function evaluations does it take to calculate the Jacobian at a single point?
How many times do I have to evaluate my function?
Yeah?
2 n
2n, right.
So if I have n, if I have n elements to x, I've got to make two function calls per column of j.
There's going to be n columns in j.
So 2n function evaluations to compute the Jacobian at a single point.
Is that really true though?
Not quite.
f of x is f of x. I don't have to compute it every time.
I just compute f of x once.
So it's really like n plus 1 that I have to do, right?
N plus a function evaluations to compute this thing.
I actually got to compute them though.
Function evaluations may be really expensive.
Suppose you're doing some sort of complicated simulation, like a finite element simulation.
Maybe it takes minutes to generate a function evaluation.
So it can be expensive to compute the Jacobian in this way.
Just be expensive to compute the Jacobian.
How is approximation of Jacobian going to affect the convergence?
What's going to happen to the rate of convergence of our method?
It's going to go down, right?
It's probably not going to be linear.
It's not going to be quadratic.
It's going to be some super linear factor.
It's going to depend on how accurate the Jacobian is.
How sensitive the function is near the root.
But it's going to reduce the accuracy of the method, or the convergence rate of the method by a little bit.
That's OK.
So this is what MATLAB does.
It uses a finite difference approximation for your Jacobian.
When you give it a function, and you don't tell it the Jacobian explicitly.
Here's an example of how to implement this yourself.
So I've got to have some function that does whatever this function is supposed to do.
It takes as input x and it gives an output f. And then the Jacobian, right?
It's a matrix.
So we initialize this matrix.
We loop over each of the columns.
We compute the displacement right?
The deviation from x for each of these.
And then we compute this difference and divide it by epsilon.
I haven't done everything perfect here, right?
Here's an extra function evaluation.
I could just calculate the value of the function at x before doing the loop.
I've also only used a relative tolerance here.
I'm going to be in trouble if xi is 0.
It's going to be a problem with this algorithm.
These are the little details one has to pay attention to.
But it's a simple enough calculation to do.
Loop over the columns, right?
Compute these differences.
Divide by epsilon.
You have your approximation for the Jacobian.
I've got to do that at every iteration, right?
Every time x is updated, I've got to recompute my Jacobian.
That's it though.
All right, that's one way of approximating a Jacobian.
There s a method that's used in one dimension called the Secant method.
It's a special case of the Newton-Raphson method and uses a coarser approximation for the derivative.
It says, I was taking these steps from xi minus 1 to xi.
And I knew the function values there.
Maybe I should just compute the slope of the line that goes through those points, and say, that's my approximation for the derivative.
Why not?
I have the data available to me.
It seems like a sensible thing to do.
So we replace f prime at x1 with f of xi minus f of x i minus 1.
Down here we put xi minus xi minus 1 up here.
That's our approximation for the derivative, or the inverse of the derivative.
This can work, it can work just fine.
Can it be extended to many dimensions?
That's an interesting question, though?
This is simple.
In many dimensions, not so obvious right?
If I know xi, xi minus 1. f of xi, f of si minus 1.
Can I approximate the Jacobian?
What do you think?
Does it strike you as though there might be some fundamental difficulty to doing that?
Yeah?
Could you approximate the gradient?
[INAUDIBLE] gradient of f at x

But I'm sure if you whether you can go backwards from the gradient in the Jacobian
OK.
So, let's-- go ahead
Perhaps the difficulty is, I mean when they're just single values--
Yeah
You can think of [INAUDIBLE] derivative, right?
Yeah
[INAUDIBLE] get really big, you get a vector of a function at xi, a vector of a function of xi minus 1 or whatever.
Vectors of these x's.
And so if you're [INAUDIBLE]
Yeah, so how do I divide these things?
That's a good question.
The Jacobian-- how much information content is in the Jacobian?
Or how many independent quantities are built into the Jacobian?
[INAUDIBLE]
And squared.
And how much data do I have to work with here?
You know, order n data.
To figure out order n squared quantities.
This is the division problem you're describing, right?
So it seems like this is an underdetermined sort of problem.
And it is.
OK?
So there isn't a direct analog to the Secant method in dimensions.
We can write down something that makes sense.
So this is the 1D Secant approximation.
That the value of the derivative multiplied by the step between i minus 1 and i is approximated by the difference in the values of the function.
The equivalent is the value of the Jacobian multiplied by the step between i minus 1 and i is equal to the difference between the values of the functions.
But now this is an equation for n squared elements of the Jacobian, in terms of n elements of the function, right?
So it's massively, massively underdetermined.
OK?
Here we have an equation for-- we have one equation for one unknown.
The derivative, right?
Think about how it was moving through space before, right?
The difference here, xi minus 1, that's some sort of linear path that I'm moving along through space.
How am I supposed to figure out what the tangent curves to all these functions are from this linear path through multidimensional space, right?
That's not going to work.
So there's underdetermined problems.
It's not so-- that's not so bad, actually.
Right?
Doesn't mean there's no solution.
In fact, it means there's all a lot of solutions.
So we can pick whichever one we think is suitable.
And Broyden's method is a method for picking one of these potential solutions to this underdetermined problem.
We don't have enough information to calculate the Jacobian exactly.
But maybe we can construct a suitable approximation for it.
And here's what's done.
So here's the Secant approximation.
It says the Jacobian times the step size, or the Newton-Raphson step, should be the difference in the functions.
And Newton's method for x i, said xi minus xi minus 1 was equal-- times the Jacobian, was equal to minus f of xi minus 1.
This is just Newton's method.
Invert the Jacobian, and put it on the other side of the equation.
Broyden's method said, i-- there's a trick here.
Take the difference between these things.
I get the same left-hand side on both of these equations.
So take the difference, and I can figure out how the Jacobian should change from one step to the next.
So maybe I have a good approximation to the Jacobian at xi minus i, I might be able to use this still underdetermined problem to figure out how to update that Jacobian, right?
So Broyden's method is what's referred to as the rank one update.
You should convince yourself that letting the Jacobian at xi minus the Jacobian at xi minus 1 be equal to this is one possible solution of this underdetermined equation.
There are others.
This is one possible solution.
It turns out to be a good one to choose.
So there's an iterative approximation now for the Jacobian.
Does this strategy make sense?
It's a little weird, right?
There's something tricky here.
You got to know to do this.
Right, so somebody has to have in mind already that they're looking for differences in the Jacobian that they're going to update over time.
So this tells me the Jacobian, how the Jacobian is updated.
Really we need the Jacobian inverse, and the reason for choosing this rank one update approximation is it's possible to write the inverse of j of xi in terms of the inverse of j at xi minus 1 when this update formula is true.
So it's something called the Sherman Morrison Formula, which says the inverse of a matrix plus the dyadic product of two vectors can be written in this form.
We don't need to derive this, but this is true.
This matrix plus dyadic product is exactly this.
We have dyadic product between f and the step from xi minus 1 to x.
And so we can apply that Sherman Morrison Formula to the rank one update.
And not only can we update the Jacobian iteratively, but we can update the Jacobian inverse.
So if I know j inverse at some previous time, I know j inverse at some later time too.
I don't have to compute these things.
I don't have to solve these systems of equations, right?
I just update this matrix.
Update this matrix, and I can very rapidly do these computations.
So not only do we have an iterative formula for the steps, right?
From x 0 to x1 to x 2, all the way up to our converged solution, but we can have a formula for the inverse of the Jacobian.
We give up accuracy.
But that's paid for in terms of the amount of time we have to spend doing these calculations.
Does it pay off?
It depends on the problem, right?
We try to solve problems in different ways.
This is a pretty common way to approximate the Jacobian.
Questions about this?
No.
OK. Broyden's method.
All right, here's the last one.
The Damped Newton-Raphson method.
We'll do this in one dimension.
So the Newton-Raphson method, Newton and Raphson told us, take a step from xi to xi plus 1 that is this big.
xi to xi plus 1, it's this big.
Sometimes you'll take that step, and you'll find that the value of the function at xi plus 1 is even bigger than the value of the function at xi.
There was nothing about the Newton-Raphson method that told us the function value was always going to be decreasing.
But actually, our goal is to make the function value go to 0 in absolute value.
So it seems like this step, not a very good one, right?
What are Newton and Raphson thinking here.
This is not a good idea.
The function value went up.
Far from a root, OK?
The Newton-Raphson method is going to give these sorts of erratic responses.
Who knows what direction it's going to go?
And it's only locally convergent.
It tells us a direction to move in, but it doesn't always give the right sort of magnitude associated with that step.
And so you take these steps and you can find out the value of your function, the normed value of your functions.
It's bigger than where you started.
It seems like you're getting further away from the root.
Our ultimate goal is to drive this norm to 0.
So steps like that you might even call unacceptable.
Right?
Why would I ever take a step in that direction?
Maybe I should use a different method.
When I take a step that's so big my function value grows in norm value.
So what one does, oftentimes, is introduce a damping factor, right?
We said that this ratio, or equivalently, the Jacobian inverse times the value of the function, gives us the right direction to step in.
But how big a step should we take?
It's clear a step like this is a good one.
It reduced the value of the function.
And it's better than the one we took before, which was given by the linear approximation.
So if I draw the tangent line, it intercepts here.
If I take a step in this direction, but I reduce the slope by having some damping factor that's smaller than 1, I get closer to the root.
Ideally we'd like to choose that damping factor to be the one that minimizes the value of the function at xi plus 1.
So it's the argument that minimizes the value of the function at xi plus 1 or at xi minus alpha, f over f prime.
Solving that optimization problem, what's hard as finding the root itself.
So ideally this is true.
But practically you're not going to be able to do it.
So we have to come up with some approximate methods of solving this optimization problem.
Actually we don't even care about getting it exact.
We know Newton-Raphson does a pretty good job.
We want some sort of guess that's respectable for this alpha so that we get close to this root.
Once we get close, we'll probably choose alpha equal to 1.
We'll just take the Newton-Raphson steps all the way down to the root.
So here it is in many dimensions.
Modify the Newton-Raphson step by some value alpha, choose alpha to be the argument that minimizes the norm value of the function at xi plus 1.
Here's one way of doing this.
So this is called the Armijo line search.
See?
Line search.
Start by letting alpha equal to 1.
Take the full Newton-Raphson step, and check.
Was the value of my function smaller than where I started?
If it is, let's take the step.
It's getting us-- we're accomplishing our goal.
We re reducing the value of the function in norm.
Maybe we're headed towards z.
That's good.
Accept it.
If no, let's replace alpha with alpha over 2.
Let's take a shorter step.
We take a shorter step, and we repeat.
Right?
Take the shorter step.
Check whether the value of the function with the shorter step is acceptable.
If yes, let's take it, and let's move on.
And if no, replace alpha with alpha over 2, and continue.
So we have our step size every time.
We don't just have to have it.
We could choose different factors to reduce it by.
But we try to take shorter and shorter steps until we accomplish our goal of having a function which is smaller in norm at our next iterate than where we were before.
It's got-- the function value will be reduced.
The Newton-Raphson method picks a direction that wants to bring the function value closer to 0.
We linearize the function, and we found the direction we needed to go to make that linearization go to 0.
So there is a step size for which the function value will be reduced.
And because of that, this Armijo line search of the Damped Newton-Raphson method is actually globally convergent, right?
The iterative method will terminate.
You can guarantee it.
Here's what it looks like graphically.
I take my big step, my alpha equals 1 step.
I check the value of the function.
It's bigger in absolute value than where I started.
So I go back.
I take half that step size.
OK?
I look at the value of the function.
It's still bigger.
Let's reject it, and go back.
I take half that step size again.
The value of the function here is now smaller in absolute value.
So I accept it.
And I put myself pretty close to the root.
So it's convergent, globally convergent.
That's nice.
It's not globally convergent to roots, which is a pain.
But it's globally convergent.
It will terminate eventually.
You'll get to a point where you won't be able to advance your steps any further.
It may converge to minima or maxima of a function.
Or it may converge to roots.
But it will converge.
I showed you this example before with basins of attraction.
So here we have different basins of attraction.
They're all colored in.
They show you which roots you approach.
Here I've applied the Damped Newton-Raphson method to the same system of equations.
And you can see the basins of attraction are shrunk because of the damping.
What happens when you're very close to places where the determinant of the Jacobian is singular, you take all sorts of wild steps.
You go to places where the value of the function is bigger than where you started.
And then you've got to step down from there to try to find the root.
Who knows where those locations are?
It's a very complicated, geometrically complicated space that you're moving through.
And the Damped Newton-Raphson method is forcing the steps to always reduce the value of the function, so they reduce the size of these basins of attraction.
So this is often a nice way to supplement the Newton-Raphson method when your guesses aren't very good to begin with.
When you start to get close to root you're, always just going to accept alpha equals 1.
The first step will be the best step, and then you'll converge very rapidly to the solution.
Do we have to do any extra work actually to do this Damped Newton-Raphson method.
Does it require extra calculations?
What do you think?
A lot of extra-- a lot of extra calculation?
How many extra calculations does it require?
Of course it requires extra.
How many?
[INAUDIBLE]
What do you think?
[INAUDIBLE]
It's-- that much is true.
So let's talk about taking one step.
How many more-- how many more calculations do I have to pay to do this sort of a step?
Or even write the multidimensional step?
For each of these times around this loop, do I have to recompute?
Do I have to solve the system of equations?
No.
Right?
You precompute this, right?
This is the basic Newton-Raphson step.
You compute that first.
You've got to do it once.
And then it's pretty cheap after that.
I've got to do some extra function evaluations, but I don't actually have to solve the system of equations.
Remember this is order n cubed.
If we solve it exactly, maybe order n squared or order n if we do it iteratively.
And the Jacobian is sparse somehow, and we know about it sparsity pattern.
This is expensive.
Function evaluations, those are order n to compute.
Relatively cheap by comparison.
So you compute your initial step.
That's expensive.
But all of this down here is pretty cheap.
Yeah?
You're also assuming that your function evaluations are reasonably true
This is true
[INAUDIBLE]
It's true.
Well, Jacobian is also very expensive to compute then too.
So, if--
[INAUDIBLE]
Sure, sure, sure.
No, I don't disagree.
I think one has to pick the method you're going to use to suit the problem.
But turns out this doesn't involve much extra calculation.
So by default, for example, fsolve in MATLAB is going to do this for you.
Or some version of this.
it's going to try to take steps that aren't too big.
It will limit the step size for you, so that it keeps the value of the function reducing in magnitude.
It's a pretty good general strategy.
Yes?
[INAUDIBLE] so why do we just pick one value for [INAUDIBLE]
I see.
So why-- ask that one more time.
This is a good question.
Can you say it a little louder so everyone can hear?
So why, instead of having just one value of alpha and not having several values of alpha [INAUDIBLE]
I see.
So the question is, yeah, we used a scalar alpha here, right?
If we wanted to, we could reduce the step size and also change direction.
We would use a matrix to do that, instead, right?
It would transform the step and change its direction.
And maybe we would choose different alphas along different directions, for example.
So diagonal matrix with different alphas.
We could potentially do that.
we're probably going to need some extra information to decide how to set the scaling in different directions.
One thing we know for sure is that the Newton-Raphson step will reduce the value of the function.
If we take a small enough step size, it will bring the value of the function down.
We know that because we did the Taylor Expansion of the function to determine that step size.
And that Taylor expansion was going to be-- that Taylor expansion is nearly exact in the limit of very, very small step sizes.
So there will always be some small step in this direction, which will reduce the value of the function.
In other directions, we may reduce the value of the function faster.
We don't know which directions to choose, OK?
Actually I shouldn't say that.
When we take very small step sizes in this direction, it's reducing the value of the function fastest.
There isn't a faster direction to go in.
When we take impossibly small, vanishingly small step sizes.
But in principle, if I had some extra information on the problem, I might be able to choose step sizes along different directions.
I may know that one of these directions is more ill-behaved than the other ones.
And choose a different damping factor for it.
That's a possibility.
But we actually have to know something about the details of the problem we're trying to solve if we're going to do-- it's a wonderful question.
I mean, you could think about ways of making this more, potentially more robust.
I'll show you an alternative way of doing this when we talk about optimization.
In optimization we'll do-- we'll solve systems of nonlinear equations to solve these optimization problems.
There's another way of doing the same sort of strategy that's more along what you're describing.
Maybe there's a different direction to choose instead that could be preferable.
This is something called the dogleg method.
Great question.
Anything else?
No.
So globally convergent, right?
Converges to roots, local minima or maxima.
There are other modifications that are possible.
We'll talk about them in optimization.
There's always a penalty to doing this.
The penalty is in the rate of convergence.
So it will converge more slowly.
But maybe you speed the calculations along anyways, right?
Maybe it requires fewer iterations overall to get there because you tame the locally convergent properties of the Newton-Raphson method.
Or you shortcut some of the expensive calculations, like getting your Jacobian or calculating your Jacobian inverse.
All right?
So Monday we're going to review sort of topics up til now.
Professor Green will run the lecture on Monday.
And then after that, we'll pick up with optimization, which will follow right on from what we've done so far.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, let's go ahead and get started again.
So we had a nice review on Monday of some of the different concepts that we've talked about up to this point in the course.
We've covered linear algebra, solutions of systems of nonlinear equations.
There's been a lot of fundamentals involved in that.
Things are going to get more applied as the sorts of problems we're trying to solve get more complicated.
Complicated problems require very sophisticated methods to solve them.
It's not going to be our expectation that you're necessarily able to implement all of these sophisticated methods.
So for example, you implemented Newton's method on your last homework assignment.
That'll be the last time we ask you to do that by hand.
You'll utilize tools built into Matlab to solve other systems of nonlinear equations going forward.
You understand how it works and what the limitations are.
You understand how to build different tools that will be inputs to these functions like the function you're trying to find the root for, or an explicit expression for the Jacobian.
You even understand how to set certain parameters associated with these methods, and what those particular parameter values mean.
Like setting the function norm tolerance, or setting the step norm tolerance associated with these methods.
Setting the number of iterations you're going to let the method go.
You've implemented it, so you understand what these things mean, what the consequences of choosing them are, how they might increase the time associated with computations that you're going to conduct.
So we've done a lot of fundamentals.
Maybe that was difficult for some of you who haven't seen this before.
Now things are becoming more about the practice of numerical methods.
How do we actually solve things reliably, and that's what today's lecture will talk about.
We discussed solving systems of nonlinear equations and needing good initial guesses for our methods.
Bad initial guesses can lead to unpredictable results.
Good initial guesses, we know the Newton-Raphson method is going to double the number of accurate digits each iteration, and we should converge very rapidly to a solution.
So it seems like having good initial guesses is really ideal.
And this topic, homotopy and bifurcation, will cover precisely those issues, and the practice of getting these good initial guesses.
I think is one of the most useful things you'll learn in this class, actually.
It's a really nice way of going about solving complicated problems.
Before we begin, I want to address a question I got via email last night.
That's always good if these questions go on [?
Piaza ?]
so everyone can read them, but I thought it was a very nice question about how do you find-- the question was how do you find multiple roots using the Newton-Raphson method.
So if I have a polynomial, where the function doesn't cross through the origin, but it just touches the origin, and goes back up.
That's a root with multiplicity in the polynomial.
And can the Newton-Raphson method find that?
Because the derivative of the function is 0 at the root.
So who says the Newton-Raphson method will be able to find those roots?
What do you think?
Yes?
And who says it won't be able to find those roots?
Or who doesn't know?
You don't know.
That's OK. That's a good question, right.
Do you know or don't you know?
So we know the Newton-Raphson method relies on knowing the derivative of the function at different points in order to take our next Newton-Raphson step towards that solution.
It may be the case, the derivative is 0 at the root, but nearby the root, the derivative may not be 0.
And then the Newton-Raphson steps are going to be fine.
They're are always going to keep stepping down towards the root.
If you go through an analysis of how the algorithm converges, you'll find out that you only get linear convergence in that case, instead of quadratic convergence.
So there will be some penalty to pay for the fact that the derivative is 0 at the root itself, but it's not going to be so bad.
The method can still converge to those sorts of solutions.
Not all is lost there, but the rate of convergence may be not as good as you expected it to be.
And the same is true in the multi-dimensional case as well.
It's a little hard to imagine what a multiple root is going to look like.
It's something like tangent curves in the solution of plane.
So if I have f1 of x1 and x2 equal 0, that's a curve in the x1, x2 plane.
And I have f2 in that same plane equal to 0.
The place where they touch might be one of these multiple roots.
That's a place where the determinant of the Jacobian may be equal to 0.
You'll have the same sort of circumstance.
Nearby, the Jacobian can be non singular, and you can take steps towards the root, but there's a worst case scenario where you're coming in on a direction that's parallel that belongs to the null space of the Jacobian at the root, and then everything's going to slow down.
You can still converge, but it may be very slow as a consequence.
I thought that was a really nice question.
It's a detailed question about how Newton-Raphson works.
It's not necessarily going to fail under these weird circumstances.
They don't come up too often.
They're going to come up in our discussion today, actually.
It's not going to fail, but it may converge very slowly to the solution.
OK?
Good.
I'm going to remind you, the last thing we did with Newton-Raphson methods, we talked about one way of fixing them.
In lieu of having good initial guesses, we may not want to take the full Newton-Raphson step.
We may want to do these quasi Newton-Raphson methods instead.
One of those was this damp Newton-Raphson where we introduced some damping parameter.
We don't take the full step, we take some shorter step, and we try to make the absolute value over function become as small as possible.
That's an optimization problem.
It's as hard as finding the root of the system of equations that we're looking at.
So we don't solve that optimization problem exactly.
That's crazy.
Instead we use this backtracking line search method where we change our damping parameter in some systematic way until we get to a point where we're satisfied.
Where we say OK, in the process of changing this, we've actually found a smaller value of our function, and we accept that as our best guess for where the next iterate is supposed to go.
And this sort of approach can give you global convergence for the algorithm, which is great, but it's not globally convergent to roots.
It's globally convergent to minima, asymptotes, and roots as well.
But it's globally convergent.
So it'll terminate.
The algorithm will stop, and when it stops, you can ask is this a root or not?
Do I think this is sufficiently close to a root or not?
If it's not sufficiently close to a root, then you know you terminated somewhere like a minima, or you rushed out towards an asymptote, and maybe you need to start with a new initial guess for your solution.
These quasi Newton-Raphson methods are very useful.
This is what Matlab uses in order to-- it uses a more sophisticated version of this called the Dogleg method, but it's the same principle.
It's trying to make sure you're not taking steps that are too big.
Trying to make sure you're taking steps in the right direction so that you're approaching the roots of your function.
Or at least reducing the absolute value of your function as close to zero as you can get it.
Are there any questions about this?
Anything we need to follow up on in the previous lecture?
No.
OK, good initial guesses.
This is the key, really, for solving systems of nonlinear equations.
It's the key for doing optimization problems too.
We need to have some idea of what our solution looks like, and we want guesses that are close to those solutions so that maybe we are in this region of local convergence for the Newton-Raphson method, and everything converges very quickly.
So where do they come from?
This is an important question, and I'll show you how you can get them.
There's another issue too, which is related to this, which is the non-linear equations, they can have multiple roots.
If we do optimization problems-- that's our next topic-- they can have multiple minima or maxima that we're trying to seek out.
Depending on our initial guess, our numerical method may find one of these roots or another root.
It may find one of these minima or another minima.
Maybe we're looking for the global minimum, so we need to look at them all.
Are there methods that we can use to find them all?
So we'll talk about continuation in homotopy, and we'll talk about bifurcation.
And it turns out there are methods one can use to find all the roots for a system of non-linear equations.
In fact, the homework assignment you have this week will ask you to do ternary phase equilibrium problem.
Vapor liquid phase equilibrium with three components.
There are five parts to this problem.
Four of them, I think, are relatively straightforward.
They're about finding the pure component roots or randomly guessing some initial conditions for your nonlinear solver in order to find all of the different azeotropes in the system.
There's a fifth part that asks you to actually implement this homotopy method, and try to find all the roots in one go with your algorithm.
Like the other homeworks you've had, that part's a little more tricky.
Usually these homeworks have one thing that's a little bit more tricky than everything else.
That part is a little bit more tricky, but it's really instructive to try to do it.
We'll learn how to do it today.
You can find all of the coexisting compositions of this ternary phase diagram, simultaneously, with one go through this algorithm.
So let's talk about continuation first.
This is an interesting idea, and it relates to trying to find roots of these equations, having good initial guesses.
Here's a simple example.
We have a third degree polynomial, and it has three roots.
We can look, graphically, because this is one dimension, and see that one of these roots is near minus 1 and 1/2, another root is near a 1/2, and another root is near 1.
And so I can go to my non-linear equation solver, and give it initial guesses, minus 1/2, 1/2, and 1, and I'm probably going to find these roots pretty reliably.
But in many dimensions, we can't do these sorts of graphical approaches.
You can't look at this function.
It may be difficult to actually get enough data to even look at it if it's a two-dimensional set of functions.
It's hard to see where these roots are, and so continuation is a method of transforming the problem into an easier one to solve.
So let's take this problem, this third order polynomial, and let's transform it.
Let's replace the 2 with a 2 times the lambda, and let's try to find the roots of this polynomials as lambda grows from 0 to 1.
This is sort of a weird idea, but we actually know when lambda is equal to 0, this goes away, and there's only one root to this polynomial-- one real root-- which is minus 1.
We actually know the answer when lambda is equal to 0.
If I make lambda a little bit bigger than 0, there will be a root which is close to minus 1.
In fact, minus 1 is a good initial guess for that root, and my non-linear equation solver ought to converge very quickly to that value.
If I choose lambda to be small, then it should be in that region of local convergence of my Newton-Raphson method, and I very quickly should find the next root.
So I can step lambda along.
I use the lambda equals 0 root as an initial guess for the next root, and I keep increasing lambda from 0 all the way up to 1, and when lambda is equal to 1, I'm going to find the root to the initial problem I was looking at.
Does that makes sense?
So I'm continuing from one set of solutions to the next.
I vary this parameter continuously, and for previous values of that parameter, they serve as initial guesses for the next solution.
Here's what a code that does that would look like.
I define my lambdas.
We don't know good ways to define the lambda.
I've got to figure that out for myself, but here lambda is a vector that goes from 0 to 1, in 1/100 increments.
Maybe that's too fine, but that's OK. And my initial guess for the solution when lambda equals 0 will be minus 1, and I loop over all the values of lambda, and I use f0, which is the one-dimensional non-linear equation solver in Matlab, to find the roots of this polynomial function-- x cubed minus 2 lambda x plus 1.
Here's the solution and that becomes my initial guess for my next solution.
So this loop is going to go round and round until I get lambda equals 1, and the solution that results there should be the solution to the initial problem I was interested in.
Here's what that looks like graphically.
I started with lambda equals 0.
The root there serves as an initial guess for lambda equals 1/100, and so on, and these roots converge eventually, to something that's close to minus 1 and 1/2.
So the actual value is a little bit bigger or a little bit smaller than minus 1.6.
Here's one of the roots.
So rather than having to have a precise initial guess, I transformed my problem from an easy to solve one to a more difficult to solve one, and I used the path along that transformation to give me rapid convergence to better and better initial guesses to the problem that I was interested in.
Make sense?
There's another way of doing this too.
You don't have to go from 0 to 1.
Let's go from lambda equals something big back down to 1 instead.
So if lambda's big, this term dominates the polynomial equation.
This is the biggest term in the polynomial equation, and it can either balance x cubed, or it can balance 1.
It can't balance against both of them, it turns out.
That So if we have this term large, and it balances against 1, then when lambda's large, an approximation for the root will be 1 over 2 lambda.
If x equals 1 over 2 lambda, and these are the two largest terms in the equation, then these two will cancel, and f will be very close to 0.
So that's one root when lambda is very large.
If I balance these other two terms against each other, I'll see there are two other roots, which are plus or minus the square root of 2 lambda.
So let's start with a large lambda instead, and these as initial guesses for the roots-- and let's trace them back to lambda equals 1.
When lambda equals 1, we recover the problem we were interested in before.
So here, I start with three different initial guesses, and I trace them back, and I'm actually able to find all three roots of the initial equation.
So there's something close to minus 1 and 1/2, something close to 1/2, and something close to 1.
So in one go, I found all three roots using continuation.
Is this clear so far?
Sam
Is that one go or is it three separate [INAUDIBLE]
That's an excellent question.
So I had to initiate this process with three separate guesses, but it turned out in this case, all three guesses converged to three distinct roots in the end.
it's one go in the sense that there's one trajectory that I had to follow, one passive lambda as I had to go through, but I had to solve three times to follow each of these three paths.
There's actually no reason to stop at lambda equals 1.
I stopped there because that's the solution to the problem.
I'm interested in, but there's actually something richer to see here.
If I keep decreasing lambda, eventually these two branches combine with each other, and then they jump down, and they follow the original branch that I had that went between minus 1 at lambda equals 0, and this root about minus 1.6 at lambda equals 1.
So the paths that these trajectories follow can be very, very complicated, actually.
A little bit unwieldy or unpredictable.
But oftentimes, they can result in predictions of all three roots or multiple roots associated with the system of equations.
Yeah
Going back to the first [INAUDIBLE] solve from lambda equals [INAUDIBLE]
Zero to one?
Zero to one.
So then would you find only one solution there?
Yes.
So going from 0 to 1, I found one solution.
Going from infinity-- 10 is big enough.
10 to 1, I was able to find three.
I didn't know beforehand that I'm going to be able to do that.
It happened to be the case that we could with this by construction.
This is a particular problem where it works out that way.
Other questions?
So this is neat.
Like we've done before with iterative methods, we turn hard to solve problems into a sequence of easier to solve problems.
Good initial guesses converge fast with Newton-Raphson.
We have a sequence of better and better initial guesses, which all converge very quickly to eventually, the solution that we're looking for evaluated at lambda equals 1.
So this is one way of getting good initial guesses.
You have a sequence of good initial guesses.
Make sense?
So we're just here.
We had this polynomial problem and we injected lambda in a particular place.
We injected it there because I knew it was going to work.
I knew the problem could be transformed in this way in order to work out and distinguish this.
It's not always obvious how to transform a problem to make it look this way, but oftentimes we have physical problems in which there's a parameter that we can do continuation with that is obvious.
So think about fluid mechanics problems.
Those are nonlinear partial differential equations.
So we're trying to solve say, the Navier-Stokes equations.
Rather than jump in and try to solve that right away, we might do continuation.
We might find a solution at very small Reynolds numbers, where the equations are almost linear, exact solutions are known in that case, and we might progressively step the Reynolds number up, and use our solution set small Reynolds numbers as initial guesses for increasingly large Reynolds numbers until we get to the one that we're interested in.
You can imagine doing that in mass transfer problems where you vary the Peclet number.
In multicomponent phase equilibria problems, this might happen by varying temperature or pressure until we get to the temperature or pressure that we're interested in.
If we have reaction equilibria problems, we might vary the different reaction rates until we get to the right balance of reaction rates to determine the equilibrium compositions.
So they're often natural parameters in the problem that we're free to adjust.
We can transform from easy to solve problems to hard to solve problems in a continuous fashion.
It's not always going to work out the way that I showed.
It's not always going to be so nice and neat.
It may be quite a bit more complicated, and sometimes there are problems for which there isn't a natural parameter available to vary.
In those cases, there's another strategy one can employ.
It's called homotopy.
This is the transformation from one problem into another problem in a continuous fashion.
So oftentimes, we're seeking a set of roots.
x star, which are a function of a parameter lambda, and they're the roots of an equation that looks like this.
So they're the roots of this function, h, which is a linear superposition of a function f and a function g. When lambda equals 0, h is equal to g, and so the roots of h are the roots of g. And when lambda equals 1, h is equal to f, and so the roots of h are the roots of f. So we might transform-- we make very lambda continuously between 0 and 1 in a construction like this.
So we might transform the roots from the roots of g into the roots of f. Maybe the roots of g are easy to find, but the roots of f are hard for us to find.
There's a smooth transformation though, from g to f. When we have those sorts of smooth transformations, there's a chance things will be well-behaved, or at least where we encounter bad behavior-- and I'll show you what I mean by bad behavior in a minute-- there's a reliable way to interpret what's going on there.
So we vary lambda in small increments from 0 to 1, and the solution x star for some lambda i is used as the initial guess to find the roots at some lambda, which is a little bit bigger than lambda i-- our next iterate in lambda.
Does this idea make sense?
Where do f and g come from?
So this is a big question-- f is the problem you want to solve, typically, and g is some auxiliary problem.
It can be arbitrarily chosen.
There's nothing to tell you why one g should be better than another one, but oftentimes, we choose them in a physical way.
So there will be a physical connection between the f problem and the g problem, and I'll show you that right now.
So here's an example.
We looked at this example before.
We want to find the roots of the van der Waals equation of state.
It's written in terms of the reduced pressure, the reduced molar volume, and the reduced temperature, and let's find those roots for a given pressure and temperature.
So this is a function f of the molar volume equals 0, and we're looking for these three roots here.
There's a root down here, which is the molar volume is low.
Not a lot of volume per number of moles, so maybe this is a liquid like root.
Out here, the molar volume is high.
We have a lot of volume, the number of moles and materials.
This is a vapor like root, and then there's an unstable root in between them.
So three roots to find.
We'd like to be able to find them in one go, or we'd like to have some reliable mechanisms for finding them.
Homotopy is one way to do this.
So let's construct a new function h of the molar volume, which is lambda times f. We want to find the roots of f plus 1 minus lambda times g. And as g, let's use something physical.
So let's let g be a function that represents the ideal gas equation of state.
So PV is nRT, but I've written everything in terms of the reduced pressure, molar volume, and temperature of the van der Waals equation of states.
So you pick up this factor of 8/3 there.
But this is the ideal gas equation of state.
Its roots will be the roots of the ideal gas equation of states.
I'm going to transform from something that's easy to solve.
I know how to find the molar volume here, there's just one, to something that's hard for us to solve.
It has three roots-- the roots of them van der Waals equation of state.
So when lambda equals 0, I'll get the roots associated with the ideal gas, and then lambda equals 1, I'll get the roots associated with the van der Waals equation.
Yes?
If you're starting with lambda as 0 and you get the roots of g, you only get one root, but you're trying to find two roots [INAUDIBLE]..
So how does that work?
I'll show you.
Here's what the process might look like.
I've got to start with a reduced temperature and pressure.
Here's my initial guess for the molar volume.
It's the root of our ideal gas equation.
Here's the definition of f, and the definition of g, and the definition of h. l times f plus 1 minus l times g, and I'm going to loop over all of my lambdas, solve the equation subject to some initial guess, and update my initial guess every time I update lambda.
That's fine.
That's the code you.
Can implement the code if you want or play around with it.
So what happens if I try to carry this out?
So I'm going to vary lambda between 0 and 1.
I start with my initial ideal gas guess for the root, and as lambda grows to 1, the molar volume shrinks until I get to lambda equals 1, and I've found one root of my equation.
One root, good.
There's not necessarily any reason to stop at lambda equals 1.
That's the root I wanted, but there's no reason I have to stop there.
So I could continue.
If I keep pushing lambda up higher and higher, eventually I'll get to a point where all of a sudden the molar volume jumps down to a different place.
This is the result of the algorithm.
I just told lambda to go bigger.
But there will be a discontinuity or a cusp here that I jump off of, and I find myself on a different solution branch, which is sort of crazy.
So now I can do something clever.
I found myself on a different solution branch.
Why not try decreasing lambda along that solution branch instead?
Maybe I'll find a different solution, and it turns out that's what happens.
All right, so i jump down to this other solution branch, and now I try decreasing lambda from 2 back down to 1, and sure enough, I found the vapor molar volume, and now I found the liquid molar volume.
There's no reason to stop and lambda equals 1.
So if I keep decreasing lambda, eventually I'll hit another cusp, and I'll jump back up to the original solution branch.
So I do my homotopy.
I vary lambda from 0 to 1, and I find one root, but if I keep going, I might identify a cusp, and jump off that cross.
I might reverse my direction with the homotopy, and I could find another root.
I might hit another cusp as well.
Now, when I hit a cusp, maybe it's best not to jump off the cusp, but instead to reverse direction at the cusp, and try to trace it out the other direction.
And if I do that, if I get right to this last point, and I try to reverse direction again, I can trace out one more solution path, and I can find the third root to the equation.
So in one go, if I'm paying attention to how my roots are changing-- if they change by too much, I might switch the direction with which I change lambda, and I could find all of these roots at once.
So three roots in one go.
Yes
Is this like trial and error?
How do you know when to change?
This is a wonderful question.
So in this case, you can do it by trial and error.
You can detect these jumps.
That's even better.
You look for jumps in the value of the solution that are of sufficient magnitude, and when you detect them, you can reverse the direction.
There's an even more systematic way to do this, which is called arclength continuation.
Give me one second, and I'll give you slide on arclength continuation, which is a systematic way of doing exactly this process of tracing out this path in the solution plane.
Roots versus homotopy parameter.
These cusps here, are sometimes called turning points, and they have a special property, which is the determinant of the Jacobian of the homotopy-- the homotopy function-- is going to be equal to 0.
The Jacobian is going to be singular right at these cusps.
So you can detect them by tracking what the Jacobian is doing right.
The Jacobian is going to be non-singular at most of these other points, but when you hit one of these turning points, the Jacobian will be singular.
There isn't a well-defined direction for the homotopy to proceed along.
It hits this cusp where the slope is infinite in this plane.
There isn't a well-defined direction to step along here.
The question was-- OK, it seems a little ad-hoc.
You've got this 2D plane, and it's easy to guess when you jump from one solution to another, and trace the path back.
That's fine.
There's a systematic way of doing this, which is called arclength continuation, which says I know that there's some curve in this solution plane.
There is some curve here.
I've shown it to you graphically.
Let's try to parametrize that curve in terms of a measure of the arclength.
Call it s, the distance along the curve.
So the roots are a function of lambda, which can be a function of the distance along the curve, and lambda is a function of the distance along the curve, and there is a constraint which says s has to satisfy an arclength formula.
This is the formula for the arclength in this plane of roots versus lambda.
The derivative of this function with respect to s squared plus the derivative of lambda with respect to s squared equals 1.
That defines s as a measure of length along this curve.
You learned arclength formulas in the context of integration at one point.
This is the differential version of that same formula.
So I've got a differential equation.
Well, it's actually a differential algebraic equation.
This is an algebraic constraint that needs to be satisfied for all values of s, and it depends on how the roots change with s. We'll learn how to solve differential algebraic equations in the two sections from now.
So we don't quite know how to solve something like this yet, but if we know how to solve those sorts of equations, then it's clear that we can trace out this entire curve in the solution plane, and then go back and try to find the places where the curve intersects lambda equals 1.
So if we follow this path by solving this equation from s equals 0 out to some very large s, then we should be able to find all of the roots that are connected to the path.
Yes
If there's a turning point that [INAUDIBLE] then does that imply multiplicity?
Yes.
So it's a real problem.
That turning point singularity is actually built into this arclength formula as well.
This is the derivative of x with respect to s squared, which takes the direction you're moving in out of the problem.
This is a length, not a direction.
So when we get to these turning points, the solution methods for these equations are going to have to know that I was headed in a downward direction first.
I can get to this turning point, and then the question is which direction do I move in from the turning point?
Do I go up or do I go down?
Which way does the solution path change?
The slope isn't defined there, so it's quite challenging to figure out exactly how to proceed.
But it is what you say.
It's like the solution has multiplicity at that point.
Curves are tangent in the solution plane.
It's a great question.
OK, so that's the notion of arclength continuation.
We may talk about this later when we talk about differential algebraic equations, and we'll talk about some equivalent sets of equations that we know how to solve, but they have to satisfy this constraint that s is a measure of arclength along the curve.
But there you go.
You can find all the solutions, at least all of the solutions connected to this path defined by the homotopy equation.
You can find all of them in one go using some clever methodologies.
Oftentimes, we're not just worried about these turning points or cusps, we encounter bifurcations in the solution.
So we may have a path that has one root along it, and then that path may split, and there may be more than one root.
Here's a simple example.
Find the real roots of x cubed minus rx.
If r is negative, there will always be one real root, and it'll be x equals 0.
If r is equal to 0, there will still be one real root, it's 0.
That root will have multiplicity 3 in that case.
And if r becomes positive, I'll suddenly have three real roots instead of one.
So as I vary the parameter r from a negative number to a positive number, I'll go from one real root to three.
In the solution plane, that will look like this.
Here are the roots as a function of r. r is less than 0.
All the roots are equal to 0.
When I hit r equals 0, I have multiplicity associated with the root, and the solution bifurcates, and there will be three possible solutions now.
If I was trying to do a homotopy problem where I very r continuously from some negative number to some positive number, I'll hit this point, and I'll want to follow all three paths to find solutions out here at positive r. But I can detect those points.
Just like I was said before, at these weird turning points or bifurcations where we have multiplicity in the roots, the Jacobian in is going to be singular.
The Jacobian of the homotopy is going to be singular.
Its determinant will be equal to 0.
So I can detect these points by checking the determinant of the Jacobian.
And when I find one, I want to do something smart to try to follow these paths.
Your homework this week has a problem like that.
You're trying to find all the azeotropes of this ternary vapor liquid equilibrium system, and you'll have a homotopy parameter that you change, and it'll bifurcate.
OK, so as you change the parameter, you'll go from one path to two, and then those two paths will split again.
You'll be splitting from a single component solution into a two component azeotrope, into a three component azeotrope.
And you'll want to detect where these bifurcations occur, and try to follow solution branches off of those bifurcations, and there's some good hints in the problem for how to do that.
So occasionally, we'll encounter problems that switch from having one solution to having many solutions.
You've seen how this happens in the discontinuous sense with these turning points.
Here, for a given value of lambda, all of a sudden it's clear that not only is there one solution here, but there's another solution down there.
My path has to choose.
Do I want this solution or the one below?
I jump off this turning point.
That's a sort of bifurcation.
These are continuous bifurcations.
They're often easier to take care of.
So the bifurcations in a homotopy let us find multiple solutions.
Every time we detect a bifurcation, if we just track those solution branches, we split our algorithm up to track each of the branches separately, they'll terminate at different roots to the homotopy equation.
And when we get to the value of lambda equals 1 in our homotopy, we're there.
We found multiple roots to the original equation.
These are often of great physical interest.
Do you know what sort of things these bifurcations represent?
One occurs in the van der Waals equation.
So if I start at very high temperatures and a given pressure, the van der Waals equation will have how many real roots?
One-- a gas.
And then as I turn the temperature down, there is a propensity for liquefaction of that gas phase.
The formation of a liquid in coexistence with the vapor.
So I'll go from having one root to having three roots all of a sudden, and there will be this bifurcation point where that transition happens in a continuous fashion
Can I ask you a question?
Yes
[INAUDIBLE]
Yeah
So as you early put it out that the turning point, the determinant of the Jacobian [INAUDIBLE] including lambda is singular.
But here you're writing j of x.
Is that [INAUDIBLE]
Excuse me.
This is a typo.
So there should be a little h here.
This is the determinant of the Jacobian of the homotopy equation at that root.
Where there's a bifurcation, the Jacobian will be singular.
I apologize.
That's a little bit--
And that lambda
And that lambda
Just to make it clear, you have n state variables.
Like x is how many unknowns you have, and then you're adding another unknown to lambda, another parameter.
This Jacobian has n plus 1 dimension.
This is also [INAUDIBLE] Is this correct?
Well, no.
Actually, the Jacobian with respect to x only is also singular at these turning and bifurcation points
[INAUDIBLE] as well
Yes.
It's also true what you said, that if I have the Jacobian, and I take the derivatives with respect to lambda, that will also be singular, but it will be singular because the subspace, which is the derivatives with respect to x, gives you a singular component.
The condition is the same as here.
So it's the Jacobian of h, taking the derivatives with respect to x-- all the different x's of h. The determinant of that matrix at a particular lambda will be equal to 0, and the value of lambda will be the lambda at which you have this turning point, or which you have a bifurcation.
Here's what it looks like in a two-dimensional sense.
So I'll have my homotopy equation where I'm trying to find the roots of h. Say it's a two-dimensional-- x is dimensions.
So it has x1 and x2, and these curves represent h1 equals 0 and h2 equals 0, and they intersect at one point for a given value of lambda.
There's the root.
Now I increase the value of lambda.
The homotopy equations change.
So these curves change shape, and all of a sudden I go from crossing curves to tangent curves, and we know tangent curves in the plane will have a Jacobian which is singular.
And this is the bifurcation point.
These two curves become tangent, and as I go to a bigger value of lambda, they'll change shape again, and I'll now have multiple roots.
I'll bifurcate from one solution to two, or one solution to three, but the bifurcation will occur through this transition where I have tangent curves, and we know there, the determinant of the Jacobian is 0.
There's a singularity under those conditions.
Does that makes sense?
OK, so a couple of things about the practice, and I'll give you a simple example you can try out, which I think is an interesting one, if pathological.
So in practice, it's hard to hit this bifurcation point exactly.
And I'm telling you that you should be detecting these branches.
You should follow these branches, but clearly, I've got to know where this point is in order to follow the branches.
That's a problem.
If the determinant of the Jacobian is 0 at the bifurcation point, then it's going to change from positive to negative as I cross the bifurcation point with lambda.
Actually, I don't necessarily have to find the point where the determinant of the Jacobian is equal to 0.
What I really should do is try to track the sign of the determinant of the Jacobian, and see when it goes from positive signed to negative signed.
And when it does that, it will have crossed through one of these bifurcation points or one of these cusps.
That's the point at which I should stop, and say, OK, there's possibly other solution branches to follow.
So as I step from my smaller lambda to my bigger lambda, the sign changes.
I want to step in some different directions than the direction I was heading before, and those different directions will trace out the other solution branches.
It turns out those different directions belong to the null space of the Jacobian.
So if you find vectors that are very close to the null space of the Jacobian, those directions are the directions you should go to find these other solution branches.
So it's very interesting.
Yeah
Can you have a Jacobian that goes from positive down to 0 and [INAUDIBLE]?
That may be possible.
You won't encounter that in the problem that you're doing.
That would be a very difficult problem to do branch detection on.
Let's Suppose that was the case and you want to find exactly where that bifurcation point is.
These bifurcation points are of physical significance.
In the van der Waals equation of state, that bifurcation point is the critical point.
It's the critical temperature at which that phase separates.
Maybe I want to know that exactly.
Well, all I need to do is solve an augmented system of equations.
So let's solve a system of equations, which is our original hematology equation equal to 0, and determinant of the Jacobian of this homotopy equation equal to 0.
This is if h had dimension n. This is now n plus 1 equations, and let's solve it for n plus 1 unknowns.
Let's solve it for x and for lambda.
x has dimension n, and we add one more unknown-- the value of the homotopy parameter at which this bifurcation occurred.
So we can find that point exactly.
So there may be cases where this happens where it doesn't continuously transition from positive to negative.
It may hit 0 and go back up, but that's a really challenging case to handle, but you can handle it.
You can find precisely the point at which this bifurcation occurs by solving these augmented equations.
It's just another non-linear equation to solve.
Yes
I'm not sure about the previous slide.
So in the previous slide, why is the [INAUDIBLE]
Let me go back to here.
So let's let the blue line here, be the place where the first element of h is equal to 0.
So this is all the values of x at which the first element of our homotopy top equation is equal to 0.
And the red curve here, let's let that be all the values of x at which the second element of h is equal to 0.
And so h itself, is only equal in 0 where both of these curves are equal to 0, and that's their intersection.
So they intersect here, and this star is the solution.
It is the root of this non-linear equation up here.
Now, I increase lambda.
I make lambda a little bit bigger.
That changes h. h is a function of lambda.
So h changes, and these curves change shape in the plane.
And at that bifurcation point, these two curves will become tangent to each other.
There will be one root, but in this solution plane, the two curves will be tangent.
And we saw before that when we get these sort of tangency conditions or the equivalent in higher dimensions, the Jacobian of our function we're trying to find the roots of will be singular.
It's like a multiple root.
And when we increase lambda further, curves change shape again.
I'm not changing the red one because that's complicated.
I just changed the blue one for you as a graphical illustration.
So the blue one changes shape, and now I've gone from having one solution to having three.
So in order to go from one solution to three, in a continuous fashion, I have to go through this tangency state, and that defines the bifurcation point.
Does that make sense?
Here's a simple example you should try out.
So I'll present it to you, and then you should try to solve it in Matlab.
You actually know the solution, geometrically, so it's not too hard.
So here's a function.
The first element to that function, it's a function of two unknowns-- x1 and x2.
The first element to that function is an equation for a circle of radius r with center at minus 3 minus 1, and the second element is an equation for a circle of radius r with center at 2, 2.
So there's two circles.
We want to find the radius where the circles just touch, and that point is a bifurcation point.
If r is perfect, these two just touch.
If r is a little too big, the two circles overlap, and there are two solutions to this problem.
And if r is a little too small, there are no solutions to this problem.
So the perfect r for touching is a bifurcation point.
It has all the properties of a bifurcation point, just like we discussed.
It's a multidimensional equation and it bifurcates.
You know the solution to this problem, geometrically.
You could work it out in a couple of minutes, but you could also solve it using these augmented equations.
So we know that point at which the circles just touch, the value of r at which those circles just touch, will be the place where f of x is equal to 0 at which these equations define a circle, and at which the Jacobian of these equations-- the derivatives of f with respect to x, it's determinant is also equal to 0.
So we want to solve this system of three equations for three unknowns.
The values of x1 and x2, where the circles just touch, and the value of r, the radius of these circles at which they just touch.
That would find the bifurcation point.
That will find the touching of the circles.
Can you see that, geometrically?
All these bifurcation problems work exactly that way.
There's some homotopy parameter.
In this case, that's r. But there's some parameter we're varying as we pass through that bifurcation.
We'd like to know the values of our unknowns and the value of this parameter at that bifurcation point or at that critical point.
So you find this point by solving the augmented equations.
We have a non-linear equation-- three nonlinear equations for three unknowns, and you can do that with a Newton-Raphson iteration.
It's a non-linear equation.
We know Newton-Raphson is the way to solve these things.
To do Newton-Raphson, that means we need to compute the Jacobian-- this is confusing.
We've got to compute the Jacobian of these augmented equations, the derivatives of these augmented equations with respect to our augmented unknowns.
So that will be the derivatives of f with respect to x, the derivatives of f with respect to r, the derivatives of the determinant of the Jacobian with respect to x.
That's this gradient here.
The derivatives of the determinant of the Jacobian with respect to r. This is our augmented Jacobian.
The augmented Jacobian times the Newton-Raphson step.
That's equal to minus the function we're trying to solve evaluated at the previous iteration.
Solve the system of equations, you'll get your Newton-Raphson step, and you can find the solution to the augmented equations.
It'll converge quadratically.
You should commit yourself, you can figure this out.
Don't try to compute all of these derivatives.
That's complicated.
You're going to make a mistake.
I'm going to make a mistake if I try to do that.
What do we do instead to compute the Jacobian?
How do we compute these derivatives?
What's that?
Finite difference, right.
Don't play games with it.
Try to use finite difference to compute the elements of this instead.
You know the exact solution too, so you can use that as an initial guess, or something close to the exact solution as an initial guess to see if you can find this bifurcation point.
So homotopy bifurcation, there are ways to get good initial guesses by continuously changing our problem.
We get a succession of problems that will converge faster because we have good initial guesses, and that will give us a reliable convergence, hopefully, to one of the roots of the equation we're after.
If we're able to pick up solution branches, we can track these cusps, or find these bifurcation points, we might be able to find multiple solutions.
There are problems for which depending on your choice of g in this homotopy function, maybe some of the solution branches are disconnected from each other.
Boy, that's crazy and complicated.
We're not going to worry about those sorts of cases, but they can pop up and happen.
But the point is there are a robust strategies you can employ to find multiple roots of these equations or multiple minima in optimization problems using precisely this technique.
You'll have a chance to practice this on this week's homework assignment, and I think you should take a look at this problem.
It's simple.
It looks complicated , but if you can understand this from, then you'll understand everything you need to know about bifurcation and homotopy.
Any questions?
Yes
At a bifurcated point, we go from having one root to three roots, per se.
And you said that when that occurs, we need to follow each of these solutions, right?
Yes
And you said that we need to step in a different direction.
So what does that exactly mean?
In a 2D sense, we have this stepping forward or stepping backwards
This is a wonderful question.
So let me go backwards here.
Here's the bifurcation point, and when we go through that bifurcation point, there are now going to be three solutions.
And I'm going to need some initial conditions for my solver to find each of these three solutions.
I can't give them all the same initial condition.
I'm going to try to track all three of these branches with different initial conditions.
If I give them all the same initial condition, and my algorithm meets the minimum threshold for what an algorithm should be, they're all going to converge to the same root.
So that's useless.
So I need to give them different initial conditions.
So what are those supposed to be?
In your problem, I'll give you explicit directions for how to find those.
It turns out those initial guesses should be perturbed in directions that belong to the null space of the Jacobian.
So I want to perturb from that one solution, which was here, in a direction that's parallel to the tangency of those curves.
Those two curves were tangent with each other.
In the other solutions, they're going to sit off in the direction that's tangent to those curves.
So I've got to find the vector that's in the null space, or very, very close to the null space, and that will tell me how to get better initial guesses for picking up these other solution branches.
It's not trivial.
It's actually quite challenging.
You'll have a chance to practice.
It's part five of five parts on this problem.
The first four, they're relatively straightforward and easy.
Part five is a little bit tricky.
You get explicit directions, and the TAs know that that's the most difficult part of that problem.
So we'll help you through it.
It's something interesting to think about trying to do.
You may find you're only able to pick up some of these solution branches, but not all of them.
That's OK.
There are ways to pick them all up if your algorithm is designed well enough.
You may just pick up some of them, and that may be the result you turn in.
You can look at the solution to figure out how the other ones are picked up as well.
But you're looking at the direction along these tangent curves.
That's a vector in the null space of the Jacobian.
And if you perturb in those directions, then you can trace out these other roots so you can find the bifurcation.
It's a wonderful question.
You'll get a chance to practice on it.
OK, last question
[INAUDIBLE]
Yes
Is this supposed to be jrx?
That's the Jacobian of f. That's the derivatives of f with respect to x
Not including r?
Yes.
Not including derivatives with respect to r. Just derivatives with respect to x.
So this is the Jacobian of f. The derivatives of f with respect to x.
The determinant of that will be 0 at the bifurcation point.
It'll turn out that-- this will be the last thing and then I have to get off the stage here.
It'll turn out that this matrix is also singular at the bifurcation point.
It will be singular because the subspace j is singular.
So it'll turn out to be singular as well.
Don't worry about it.
The condition for the bifurcation point is that the determine of the Jacobian is equal to 0, and that point is a root of the equations you're looking at.
OK, thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Let's go ahead and get started.
We saw a lot of good conversation on Piazza this weekend.
So that's good.
Seems like you guys are making your way through these two problems on the latest assignment.
I would try to focus less on the chemical engineering science and problems that involve those.
Usually the topic of interest, the thing that's useful to you educationally is going to be the numerics, right.
So if you get hung up on the definition of a particular quantity, yield was one that came up.
Rather than let that prevent you from solving the problem, pick a definition and see what happens.
You can always ask yourself if the results seem physically reasonable to you not based on your definition.
And as long as you explain what you did in your write up, you're going to get full points.
We want to solve the problems numerically.
If there's some hang up in the science, don't sweat it.
Don't let that stop you from moving ahead with it.
Don't let it make it seem like the problem can't be solved or there isn't a path to a solution.
Pick a definition and go with it and see what happens, right.
The root of the second problem is trying to nest together two different numerical methods.
One of those is optimization, and the other one is solutions of nonlinear equation, putting those two techniques together, using them in combination.
The engineering science problem gives us a solvable problem to work with in that context, but it's not the key element of it.
OK, good.
So we're continuing optimization, right.
Move this just a little bit.
We're continuing with optimization.
Last time we posed lots of optimization problems.
We talked about constrained optimization, unconstrained optimization.
We heard a little bit about linear programs.
We started approaching unconstrained optimization problems from the perspective of steepest descent.
OK, so that's where I want to pick up as we get started.
So you'll recall the idea behind steepest descent was all the unconstrained optimization problems we're interested in are based around trying to find minima, OK. And so we should think about these problems as though we're standing on top of a mountain.
And we're looking for directions that allow us to descend.
And as long as we're heading in descending directions, right, there's a good chance we're going to bottom out someplace and stop.
And when we've bottomed out, we've found one of those local minima.
That bottom is going to be a place where the gradient of the function we're trying to find the minimum of is zero, OK. And the idea behind steepest descent was well, don't just pick any direction that's down hill.
Pick the steepest direction, right.
Go in the direction of the gradient.
That's the steepest descent idea.
And then we did something a little sophisticated last time.
We said well OK, I know the direction.
I'm standing on top the mountain.
I point myself in the steepest descent direction.
How big a step do I take?
I can take any size step.
And some steps may be good and some steps may be bad.
It turns out there are some good estimates for step size that we can get by taking a Taylor expansion.
So we take our function, right, and we write it at the next iterate is a Taylor expansion.
About the current iterate, that expansion looks like this.
And it will be quadratic with respect to the step size alpha.
If we want to minimize the value of the function here, we want the next iterate to be a minimum of this quadratic function.
Then there's an obvious choice of alpha, right.
We find the vertex of this quadratic functional.
That gives us the optimal step size.
It's optimal if our function actually is quadratic.
It's an approximate, right.
It's an estimation of the right sort of step size if it's not quadratic.
And so I showed you here was a function where the contours are very closely spaced.
So it's a very steep function.
And the minima is in the middle.
If we try to solve this with the steepest descent and we pick different steps sizes, uniform step sizes, so we try 0.1 and 1 and 10 step sizes, we'll never find an appropriate choice to converge to the solution, OK. We're going to have to pick impossibly small step sizes, which will require tons of steps in order to get there.
But with this quadratic estimate, you can get a reasonably smooth convergence to the root.
So that's nice.
And here's a task for you to test whether you understand steepest descent or not.
In your notes, I've drawn some contours.
For function, we'd like to minimize using the method of steepest descent.
And I want you to try to draw steepest descent paths on top of these contours starting from initial conditions where these stars are located.
So if I'm following steepest descent, the rules of steepest descent here, and I start from these stars, what sort of paths do I follow?
You're going to need to pick a step size.
I would suggest thinking about the small step size limit.
What is the steepest descent path in the small step size limit?
Can you work that out, you and your neighbors?
You don't have to do all of them by yourself.
You can do one, your neighbor could do another.
And we'll take a look at them together.
OK, the roar has turned into a rumble and then a murmur, so I think you guys are making some progress.
What do you think?
How about let's do an easy one.
How about this one here.
What sort of path does it take?
Yeah, it sort of curls right down into the center here, right.
Remember, steepest descent paths run perpendicular to the contours.
So jumps perpendicular to the contour, almost a straight line to the center.
How about this one over here?
Same thing, right?
It runs the other way.
It's going downhill 1, 0, minus 1, minus 2.
So it runs downhill and curls into the center.
What about this one up here?
What's it do?
Yeah, it just runs to the left, right.
The contour lines had normals that just keep it running all the way to the left.
So this actually doesn't run into this minimum, right.
It finds a cliff and steps right off of it, keeps on going.
Steepest descent, that's what it does.
How about this one here?
Same thing, right, just to the left.
So these are what these paths look like.
You can draw them yourself.
If I showed you paths and asked you what sort of method made them, you should be able to identify that actually, right?
You should be able to detect what sort of methodology generated those kinds of paths.
We're not always so fortunate to have this graphical view of the landscape that our method is navigating.
But it's good to have these 2D depictions.
Because they really help us understand when a method doesn't converge what might be going wrong, right.
So steepest descent, it always heads downhill.
But if there is no bottom, it's just going to keep going down, right.
It's never going to find it.
Oh, OK.
Here's a-- this is a story now that you understand optimization.
So let's see, so mechanical systems, conservation of momentum, that's also, in a certain sense, an optimization problem, right.
So conservation of momentum says that the acceleration on a body is equal to the sum of the forces on it.
And some of those forces are what we call conservative forces.
They're proportional to gradients of some energy landscape.
Some of those forces are non-conservative, like this one here.
It's a little damping force, a little bit of friction proportional to the velocity with which the object moves instead.
And if we start some system like this, we give it some initial inertia and let it go, right, eventually it's going to want to come to rest at a place where the gradient in the potential is 0 and the velocity is 0 on the acceleration is 0.
We call that mechanical equilibrium.
We get to mechanical equilibrium and we stop, right.
So physical systems many times are seeking out minimum of an objective function.
The objective function is the potential energy.
I saw last year at my house we had a pipe underground that leaked in the front yard.
And they needed to find the pipe, right.
It was like under the asphalt.
So they got to dig up asphalt, and they need to know where is the pipe.
They know it's leaking, but where does the pipe sit?
So the city came out and the guy from the city brought this.
Do you know what this is?
What is it?
Do you know?
Yeah, yeah, yeah.
It's a dowsing rod.
OK, this kind of crazy story right, a dowsing rod.
OK, a dowsing rod.
How does it work?
The way it's supposed to work is I hold it out and it should turn and rotate in point in a direction that's parallel to the flow of the water in the pipe.
That's the theory that this is supposed to work on.
I'm a scientist.
So I expect that somehow the water is exerting a force on the tip of the dowsing rod, OK.
So the dowsing rod is moving around as this guy walks around.
And it's going to stop when it finds a point of mechanical equilibrium.
So the dowsing rod is seeking out a minimum of some potential energy, let's say.
That's what the physics says has to be true.
I don't know that flowing water exerts a force on the tip of the dowsing rod.
The guy who had this believed that was true, OK.
It turns out, this is not such a good idea, though, OK. Like in terms of a method for seeking out the minimum of a potential, it's not such a great way to do it.
Because he's way up here, and the water's way underground.
So there's a huge distance between these things.
It's not exerting a strong force, OK.
The gradient isn't very big here.
It's a relatively weak force.
So this instrument is incredibly sensitive to all sorts of external fluctuations.
The gradient is small.
The potential energy landscape is very, very flat.
And we know already from applying things like steepest descent methods or Newton-Raphson that those circumstances are disastrous for any method seeking out minima of potential energies, right.
Those landscapes are the hardest ones to detect it in.
Because every point looks like it's close to being a minima, right.
It's really difficult to see the differences between these.
Nonetheless, he figured out where the pipe was.
I don't think it was because of this though.
How did he know where the pipe was?
What's that?
Where the ground was squishy?
Where the ground was squishy.
Well yeah, had some good guesses because it was leaking up a little bit.
No, I looked carefully afterwards.
And I think it turned out the city had come by and actually painted some white lines on either side of the street to indicate where it was.
But he was out there with his dowsing rod making sure the city had gotten it right.
It turns out, there's something called the ideomotor effect where your hand has very little, you know, very sensitive little tremors in it.
And can guide something like this, a little weight at the end of a rod to go wherever you want it to go when you want it to.
It's like a Ouija board, right.
It works exactly the same way.
Anyway, it's not a good way to find the minimum of potential energy surfaces, OK. We have the same problem with numerical methods.
It's really difficult when these potential energy landscapes are flat to find where the minimum is, OK.
So fun and games are over.
Now we got to do some math.
So we talked about steepest descent.
And steepest descent is an interesting way to approach these kinds of optimization problems.
It turns out, it turns out that linear equations like Ax equals b can also be cast as optimization problems, right.
So the solution to this equation Ax equals b is also a minima of this quadratic function up here.
How do you know?
You take the gradient of this function, which is Ax minus b, and the gradient to 0 to minima.
So Ax minus b is 0, or Ax equals b.
So we can do optimization on these sorts of quadratic functionals, and we would find the solution of systems of linear equations.
This is an alternative approach.
Sometimes this is called the variational approach to solving these systems of linear equations.
There are a couple of things that have to be true.
The linear operator, right, the matrix here, it has to be symmetric.
OK, it has to be symmetric, because it's multiplied by x from both sides.
It doesn't know that it's transpose is different from itself in the form of this functional.
If A wasn't symmetric, the functional would symmetrize it automatically, OK.
So a functional like this only corresponds to this linear equation when A is symmetric.
And this sort of thing only has a minimum, right, when the matrix A is positive and definite.
It has to have all positive eigenvalues, right.
The Hessian right, of this functional, is just the matrix A.
And we already said that Hessian needs all positive eigenvalues to confirm we have a minima.
OK?
If one of the eigenvalues is zero, then the problem is indeterminate.
The linear problem is indeterminate.
And there isn't a single local minimum, right.
There's going to be a line of minima or a plane of minima instead.
OK?
OK, so you can solve systems of linear equations as optimization problems.
And people have tried to apply things like steepest descent to these problems.
And it turns out steepest descent is kind of challenging to apply.
So what winds up happening is let's suppose we don't take our quadratic approximation for the descent direction first.
Let's just say we take some fixed step size, right.
When you take that fixed step size, it'll always be, let's say good for one particular direction.
OK, so I'll step in a particular direction.
It'll be good.
It'll be a nice step into a local minimum.
But when I try to step in the next gradient direction, it may be too big or too small.
And that will depend on the eigenvalues associated with the direction that I am trying to step in, OK. How steep is this convex function?
Right?
How strongly curved is that convex function?
That's what the eigenvalues are describing.
And so fixed value of alpha will lead to cases where we wind up stepping too far or not far enough.
And there'll be a lot of oscillating around on this path that converges to a solution.
I showed you how to pick an optimal step size.
It said look in a particular direction and treat your function as though it were quadratic along that direction.
That's going to be true for all directions associated with this functional, right.
It's always quadratic no matter which direction I point in.
Right?
So I pick a direction and I step and I'll be stepping to the minimal point along that direction.
It'll be exact, OK. And then I've got to turn and I've got to go in another gradient direction and take a step there.
And I'll turn and go in another gradient direction and take a step there.
And in each direction I go, I'll be minimizing every time.
Because this step size is the ideal step size.
But it turns out you can do even better than that.
So we can step in some direction, which is a descent direction, but not necessarily the steepest descent.
And it's going to give us some extra control over how we're minimizing this function.
I'll explain on the next slide, OK.
The first thing you got to do though is given some descent direction, what is the optimal step size?
Well, we'll work that out the same way, right.
We can write f at the next iterate in terms of f at the current iterate plus all the perturbations, right.
So our step method is Xi plus 1 is Xi plus alpha pi, right.
So we do a Taylor expansion, and we'll get a quadratic function again.
And we'll minimize this quadratic function with respect to alpha i when alpha takes on this value.
So this is the value of the vertex of this function.
So we'll minimize this quadratic function in one direction, the direction p. But is there an optimal choice of direction?
Is it really best to step in the descent direction?
Or are there better directions that I could go in?
We thought going downhill fastest might be best, but maybe that's not true.
Because if I point to a direction and I apply my quadratic approximation, I minimize the function in this direction.
Now I'm going to turn, and I'm going to go in a different direction.
And I'll minimize it here, but I'll lose some of the minimization that I got previously, right?
I minimized in this direction.
Then I turned, I went some other way, right.
And I minimized in this direction.
So this will still be a process that will sort of weave back and forth potentially.
And so the idea instead is to try to preserve minimization along one particular direction.
So how do we choose an optimal direction?
So f, right, at the current iterate, it's already minimized along p, right.
Moving in p forward and backwards, this is going to make f and e smaller.
That's as small as it can be.
So why not choose p so that it's normal to the gradient at the next iterate?
OK, so choose this direction p so it's normal to the gradient at the next iterate.
And then see if that holds for one iterate more after that.
So I move in a direction.
I step up to a contour.
And I want my p to be orthogonal to the gradient at that next contour.
So I've minimized this way, right.
I've minimized everything that I could in directions that aren't in the gradient direction associated with the next iterate.
And then let's see if I can even do that for the next iteration too.
So can it make it true that the gradient at the next iterate is also orthogonal to p?
By doing this, I get to preserve all the minimization from the previous steps.
So I minimize in this direction.
And now I'm going to take a step in a different direction.
But I'm going to make sure that as I take that step in another direction, right, I don't have to step completely in the gradient.
I don't have to go in the steepest descent direction.
I can project out everything that I've stepped in already, right.
I can project out all the minimization I've already accomplished along this p direction.
So it turns out you can solve, right, you can calculate what this gradient is.
The gradient in this function is Ax minus b.
So you can substitute exactly what that gradient is.
A, this is Xi plus 2 minus b dotted with p, right.
This has to be equal to 0.
And you can show that means that p transpose A times p has to be equal to 0 as well.
You don't need to be able to work through these details.
You just need to know that this gives a relationship between the directions on two consecutive iterates, OK.
So it says if I picked a direction p on the previous iteration, take how it's transposed by A, and make sure that my next iteration is orthogonal to that vector, OK. Yeah?
So does that mean that your p's are all independent of each other, or just that adjacent p is?
K, k plus 1 p's are?
This is a great question.
So the goal with this method, the ideal way to do this would be to have these directions actually be the directions of the eigenvectors of A.
And those eigenvectors for symmetric matrix are all orthogonal to each other.
OK?
And so you'll be stepping along these orthogonal directions.
And they would be all independent of each other.
OK?
But that's a hard problem, finding all the eigenvectors associated with a matrix.
Instead, OK, we pick an initial direction p to go in.
And then we try to ensure that all of the other directions satisfy this conjugacy condition, right.
That the transformation of p by A is orthogonal with the next direction that I choose.
So they're not independent of each other.
But they are what we call conjugate to each other.
It turns out that by doing this, these sets of directions p will belong to-- they can be expressed in terms of many products of A with the initial direction p. That'll give you all these different directions.
It starts to look something like the power iteration method for finding the largest eigenvector of a matrix.
OK?
So you create a certain set of vectors that span the entire subspace of A.
And you step specifically along those directions.
And that lets you preserve some of the minimization as you step each way.
So what's said here is that the direction p plus 1 is conjugate to the direction p. And by choosing the directions in this way, you're ensuring that p is orthogonal to the gradient at i plus 1 and the gradient i plus 2.
So you're not stepping in the steepest descent directions that you'll pick up later on in the iterative process.
OK?
So when you know which direction you're stepping in, then you've got to satisfy this conjugacy condition.
But actually, this is a vector in n space, right.
This is also a vector n space.
And we have one equation to describe all and components.
So it's an under-determined problem.
So then one has to pick which particular one of these conjugate vectors do I want to step along.
And one particular choice is this one, which says, step along the gradient direction, OK, do steepest descent, but project out the component of the gradient along pi.
We already minimized along pi.
We don't not have to go in the pi direction anymore, right.
So do steepest descent, but remove the pi component.
So here is a quadratic objective function.
It corresponds to a linear equation with coefficient matrix 1 00 10, a diagonal coefficient matrix.
And b equals 0.
So the solution of the system of linear equations is 00.
We start with an initial guess up here, OK. And we try steepest descent with some small step size, right.
You'll follow this blue path here.
And you can see what happened.
That step size was reasonable as we moved along the steepest ascent direction where the contours were pretty narrowly spaced.
But as we got down to the flatter section, OK, as we got down to the flatter section of our objective function, those steps are really small.
Right?
We're headed in the right direction, we're just taking very, very small steps.
If you apply this conjugate gradient methodology, well, the first step you take, that's prescribed.
You've got to step in some direction.
The second step you take though minimizes completely along this direction.
So the first step was the same for both of these.
But the second step was chosen to minimize completely along this direction.
So it's totally minimized.
And the third step here also steps all the way to the center.
So it shows a conjugate direction that stepped from here to there.
And it didn't lose any of the minimization in the original direction that it proceeded along.
So that's conjugate gradient.
It's used to solve linear equations with order n iterations, right.
So A has at most n independent eigenvectors, independent directions that I can step along and do this minimization.
The conjugate gradient method is doing precisely that.
Doesn't know what the eigendirections are, but it's something along these conjugate directions as a proxy for the eigendirections.
So it can do minimization with just n steps for a system of n equations for n unknowns.
It requires only the ability to compute the product of your matrix A with some vector, right.
All the calculations there are only depended on the product of A with a vector.
So don't have to store A, we just have to know what A is.
We have some procedure for generating A.
Maybe A is a linear operator that comes from a solution of some differential equations instead, right.
And we don't have an explicit expression for A, but we have some simulator that produces, take some data, and projects A to give some answer, right.
So we just need this product.
We don't have to store A exactly.
It's only good for symmetric positive definite matrices, right.
This sort of free energy functional that we wrote or objective function we wrote only admits symmetric matrices which are positive definite.
That's the only way it will have a minimum.
And so the only way a steepest descent or descent type procedure is going to get to the optimum.
But there are more sophisticated methods that exist for arbitrary matrices.
So if we don't want symmetry or we don't care about whether it's positive definite, there are equivalent sorts of methods that are based around the same principle.
And it turns out, this is really the state of the art.
So if you want to solve complicated large systems of equations, you know Gaussian elimination, that will get you an exact solution.
But that's often infeasible for the sorts of problems that we're really interested in.
So instead, you use these sorts of iterative methods.
Things like Jacobi and Gauss-Seidel, they're sort of the classics in the field.
And they work, and you can show that they converge on are lots of circumstances.
But these sorts of iterative methods, like conjugate gradient and its brethren other Krylov subspace methods they're called, are really the state of the art, and the ones that you reach to.
You already did conjugate gradients in one homework, right.
You used this PCG iterative method in Matlab to solve a system of linear equations.
It was doing this, right.
This is how it works.
OK?
OK. OK, so that's country ingredients.
You could apply it also to objective functions that aren't quadratic in nature.
And the formulation changes a little bit.
Everywhere where the matrix A appeared there needs to be replaced with the Hessian at a certain iterate.
But the same idea persists.
It says well, we think in our best approximation for the function that we've minimized as much as we can in one direction.
So let's choose a conjugate direction to go in, and try not to ruin the minimizations we did in the direction we were headed before.
Of course, these are all linearly convergent sorts of methods.
And we know that there are better ways to find roots of non-linear equations like this one, grad f equals zero, namely the Newton-Raphson method, which is quadratically convergent.
So if we're really close to a critical point, and hopefully that critical point is a minima in f, right, then we should rapidly converge to the solution of this system of nonlinear equations just by applying the Newton-Raphson method.
It's locally convergent, right.
So we're going to get close.
And we get quadratic improvement.
What is the Newton-Raphson iteration, though?
Can you write that down?
What is the Newton-Raphson iteration that's the iterative math for this system of non-linear equations, grad f equals 0?
Can you work that out?
What's that look like?
Have we got this?
What's the Newton-Raphson iterative map look like for this system of non-linear equations?
Want to volunteer an answer?
Nobody knows or nobody is sharing.
OK, that's fine.
Right, so we're trying to solve an equation g of x equals 0.
So the iterative map is Xi plus 1 is Xi minus Jacobi inverse times g. And what's the Jacobian of g?
What's the Jacobian of g?
The Hessian, right.
So the Jacobian of g is the gradient of g, which is two gradients of f, which is the definition of the Hessian.
So really, the Newton-Raphson iteration is Xi plus 1 is Xi minus Hessian inverse times g. So the Hessian plays the role of the Jacobian, the sort of solution procedure.
And so everything you know about Newton-Raphson is going to apply here.
Everything you know about quasi-Newton-Raphson methods is going to apply here.
You're going to substitute for your nonlinear.
The nonlinear function you're finding the root for, you're going to substitute the gradient.
And for the Jacobian, you're going to substitute the Hessian.
Places where the Hessian is, the determent of the Hessian is 0, right it's going to be a problem.
Places where the Hessian is singular is going to be a problem.
Same as with the Jacobian.
But Newton-Raphson has the great property that if our function is quadratic, like this one is, it will converge in exactly one step.
So here's steepest descent with a fixed value of alpha, Newton-Raphson, one step for a quadratic function.
And why is it one step?
[INAUDIBLE]
Good.
So when we take a Taylor expansion of our f, in order to derive the Newton-Raphson step, we're expanding it out to quadratic order, its function is quadratic.
The Taylor expansion is exact.
And the solution of that equation, right, gradient f equals 0 or g equals 0, that's the solution of a linear equation, right.
So it gives exactly the right step size here to move from an initial guess to the exact solution or the minima of this equation.
So for quadratic equations, Newton-Raphson is exact.
It doesn't go in the steepest ascent direction, right.
It goes in a different direction.
It would like to go in the steepest descent direction if the Jacobian were identity.
But the Jacobian is a measure of how curved f is.
The Hessian, let's say, is a measure of how curved f is.
Right?
And so there's a projection of the gradient through the Hessian that changes the direction we go in.
That change in direction is meant to find the minimum of the quadratic function that we approximate at this point.
So as long as we have a good quadratic approximation, Newton-Raphson is going to give us good convergence to a minima or whatever nearby critical point there is.
If we have a bad approximation for a quadratic, then it's going to be so good, right.
So here's this very steep function.
Log of f is quadratic, but f is exponential in x here.
So you got all these tightly spaced contours converging towards a minima at 00.
And here I've got to use the steepest descent step size, the optimal steepest descent step size, which is a quadratic approximation for the function, but in the steepest descent direction only.
And here's the path that it follows.
And if I applied Newton-Raphson to this function, here is the path that it follows instead.
The function isn't quadratic.
So these quadratic approximations aren't-- they're not great, right.
But the function is convex, right.
So Newton-Raphson is going to proceed downhill until it converges towards a solution anyways.
Because the Hessian has positive eigenvalues all the time.
Questions about this?
Make sense?
OK?
So you get two different types of methods that you can play with.
One of which, right, is always going to direct you down hill.
Steepest descent will always carry you downhill, right, towards a minima.
And the other one, Newton-Raphson, converges very quickly when it's close to the root.
OK, so they each have a virtue.
And they're different.
They're fundamentally different, right.
They take steps in completely different directions.
When is Newton-Raphson not going to step down hill?
[INAUDIBLE] What's that?
[INAUDIBLE]
OK, that's more generic an answer than I'm looking for.
So there may be circumstances where I have two local minima.
That means there must be maybe a saddle point that sits between them.
Newton-Raphson doesn't care which critical point it's going after.
So it may try to approach the saddle point instead.
That's true.
That's true.
Yeah?
When Hessian [INAUDIBLE]
Good, yeah.
With the Hessian doesn't have all positive eigenvalues, right.
So if all the eigenvalues of the Hessian are positive, then the transformation h times g or h inverse times g, it'll never switch the direction I'm going.
I'll always be headed in a downhill direction.
Right?
In a direction that's anti-parallel to the gradient.
OK?
But if the eigenvalues of the Hessian are negative, if some of them are negative and the gradient has me pointing along that eigenvector in a significant amount, then this product will switch me around and will have me go uphill instead.
It'll have me chasing down a maxima or a saddle point instead.
That's what the quadratic approximation of our objective function will look like.
It looks like there's a maximum or a saddle instead.
And the function will run uphill.
OK?
So there lots of strengths to Newton-Raphson.
Convergence is one of them, right.
The rate of convergence is good.
It's a locally convergent, that's good.
It's got lots of weaknesses, though.
Right?
It's going to be a pain when the Hessian is singular at various places.
You've got to solve systems of linear equations to figure out what these steps are.
That's expensive computationally.
It's not designed to seek out minima, but to seek out critical points of our objective function.
Steepest descent has lots of strengths, right.
Always heads downhill, that's good.
If we put a little quadratic approximation on it, we can even stabilize it and get good control over the descent.
Its weaknesses are it's got the property that it's linearly convergent instead of quadratically convergent when it converges.
So it's slower, right.
It might be harder to find a minima.
You've seen several examples where the path sort of peters out with lots of little iterations, tiny steps towards the solution.
That's a weakness of steepest descent.
We know that if we go over the edge of a cliff on our potential energy landscape, steepest descent it just going to run away, right.
As long as there's one of these edges, it'll just keep running downhill for as long as they can.
So what's done is to try to combine these methods.
Why choose one, right?
We're trying to step our way towards a solution.
What if we could craft a heuristic procedure that mixed these two?
And when steepest descent would be best, use that.
When Newton-Raphson would be best, use that.
Yes?
Just a quick question on Newton-Raphson
Yes?
Would it run downhill also if you started it over there?
Or since it seeks critical points, could you go back up to the [INAUDIBLE]
That's a good question.
So if there's an asymptote in f, it will perceive the asymptote as a critical point and chase it.
OK?
And so if there's an asymptote in f, if can perceive that and chase it.
It can also run away as it gets very far away.
This is true.
OK?
The contour example that I gave you at the start of class had sort of bowl shape functions superimposed with a linear function, sort of planar function instead.
For that one, right, the Hessian is ill-defined, right.
There is no curvature to the function.
But you can imagine adding a small bit of curvature to that, right.
And depending on the direction of the curvature, Newton-Raphson may run downhill or it may run back up hill, right?
We can't guarantee which direction it's going to go.
Depends on the details of the function.
Does that answer your question?
Yeah?
Good
Sir, can you just go back to that one slide?
Yeah.
I'm just pointing out, if the eigenvalues of h is further negative, then the formula there for alpha could have trouble too
That's true
Similar to how the Newton-Raphson had trouble
This is true.
This is true, yeah.
So we chose a quadratic approximation here, right, for our function.
We sought a critical point of this quadratic approximation.
We didn't mandate that it had to be a minima.
So that's absolutely right.
So if h has negative eigenvalues and the gradient points enough in the direction of the eigenvectors associated with those negative eigenvalues, then we may have a case where alpha isn't positive.
We required early on that alpha should be positive for steepest descent.
So we can't have a case where alpha is not positive.
That's true.
OK.
So they're both interesting methods, and they can be mixed together.
And the way you mix those is with what's called trust-region ideas, OK. Because it could be that we've had an iteration Xi and we do a quadratic approximation to our functional, which is this blue curve.
Our quadratic approximation is this red one.
And we find the minima of this red curve and use that as our next best guess for the solution to the problem.
And this seems to be working us closer and closer towards the actual minimum in the function.
So since quadratic approximation seems good, if the quadratic approximation is good, which method should we choose?
Newton-Raphson
Newton-Raphson, right.
Could also be the case though that we make this quadratic approximation from our current iteration, and we find a minimum that somehow oversteps the minimum here.
In fact, if we look at the value of our objective function at this next step, it's higher than the value of the objective function where we started.
So it seems like a quadratic approximation is not so good, right.
That's a clear indication that this quadratic approximation isn't right.
Because it suggested that we should have had an minima here, right.
But our function got bigger instead.
And so in this case, it doesn't seem like you'd want to choose Newton-Raphson to take your steps.
The quadratic approximation is not so good.
Maybe just simple steepest descent is a better choice.
OK, so it's done.
So if you're at a point, you might draw a circle around that point with some prescribed radius.
Call that Ri.
This is our iterate Xi.
This is our trust-region radius Ri.
And we might ask, where does our Newton-Raphson step go?
And where does our steepest descent step take us?
And then based on whether these steps carry us outside of our trust-region, we might decide to take one or the other.
So if I set a particular size Ri, particular trust-region size Ri and the Newton-Raphson step goes outside of that, we might say well, I don't actually trust my quadratic approximation this far away from the starred point.
So let's not take a step in that direction.
Instead, let's move in a steepest descent direction.
If my Newton-Raphson step is inside the trust-region, maybe I'll choose to take it, right.
I trust the quadratic approximation within a distance Ri of my current iteration.
Does that strategy makes sense?
So we're trying to pick between two different methods in order to give us more reliable convergence to a local minima.
So here's our Newton-Raphson step.
It's minus the Hessian inverse times the gradient.
Here's our steepest descent step.
It's minus alpha times the gradient.
And if the Newton-Raphson step is smaller than the trust-region radius, and the value of the objective function at Xi, plus the Newton-Raphson step is smaller than the current objective function, it seems like the quadratic approximation is a good one, right.
I'm within the region in which I trust this approximation, and I've reduced the value of the function.
So why not go that way, right?
So take the Newton-Raphson step.
Else, let's try taking a step in the steepest direction instead.
So again, if the steepest ascent direction is smaller than Ri and the value of the function in the steepest descent direction, the optimal steepest descent direction or the optimal step in the steepest ascent direction is smaller than the value of the function at the current point, seems like we should take that step.
Right?
The Newton-Raphson step was no good.
We've already discarded it.
But our optimized steepest descent step seems like an OK one.
It reduces the value of the function.
And its within the trust-region where we think quadratic approximations are valid.
If that's not true, if the steepest descent step takes us outside of our trust-region or we don't reduce the value of the function when we take that step, then the next best strategy is to just take a steepest ascent step to the edge of the trust-region boundary.
Yeah?
Is there a reason here that Newton-Raphson is the default?
Oh, good question.
So eventually we're going to get close enough to the solution, all right, that all these steps are going to live inside the trust-region ring.
Its going to require very small steps to converge to the solution.
And which of these two methods is going to converge faster?
Newton-Raphson
Newton-Raphson.
So we prioritize Newton-Raphson over steepest descent.
That's a great question.
Its the faster converging one, but its a little unwieldy, right.
So let's take it when it seems valid.
But when it requires steps that are too big or steps that don't minimize f, let's take some different steps instead.
Lets use steepest descent as the strategy.
So this is heuristic.
So you got to have some rules to go with this heuristic, right.
We have a set of conditions under which we're going to choose different steps.
We've got to set this trust-region size.
This Ri has to be set.
How big is it going to be?
I don't know.
You don't know, right, from the start you can't guess how big Ri is going to be.
So you got to pick some initial guess.
And then we've got to modify the size of the trust-region too, right.
The size of the trust-region is not going to be appropriate.
One fixed size is not going to be appropriate all the time.
Instead, we want a strategy for changing its size.
So it should grow or shrink depending on which steps we choose, right.
Like if we take the Newton-Raphson step and we find that our quadratic approximation is a little bit bigger than the actual function value that we predicted, we might want to grow the trust-region.
We might be more likely to believe that these Newton-Raphson steps are getting us to smaller and smaller function values, right.
The step was even better than we expected it to be.
Here's the quadratic approximation in the Newton-Raphson direction.
And it was actually bigger than the actual value of the function.
So we got more, you know, we got more than we expected out of a step in that direction.
So why not loosen up, accept more Newton-Raphson steps?
OK, that's a strategy we can take.
Otherwise, we might think about shrinking instead, right.
So there could be the circumstance where our quadratic approximation predicted a smaller value for the function than we actually found.
It's not quite as reliable for getting us to the minimum.
These two circumstances are actually these.
So this one, the quadratic approximation predicted a slightly bigger value than we found.
Say grow the trust-region, right.
Try some more Newton-Raphson steps.
Seems like the Newton-Raphson steps are pretty reliable here.
Here the value of the function in the quadratic approximation is smaller than the value of the function after we took the step.
Seems like our trust-region is probably too big if we have a circumstance like that.
Should shrink it a little bit, right?
We took the Newton-Raphson step, but it actually did worse than we expected it to do with the quadratic approximation.
So maybe we ought to shrink the trust-regional a little bit.
And you need a good initial value for the trust-region radius.
What does Matlab use?
It uses 1.
OK.
It doesn't know.
It has no clue.
It's just a heuristic.
It starts with 1 and it changes it as need be.
So this is how fsolve solves systems of nonlinear equations.
This is how all of the minimizers in Matlab, this is the strategy they use to try to find minima.
They use these sorts of trust-region methods.
It uses a slight improvement, which is also heuristic, called a dogleg trust-region method.
So you can take a Newton-Raphson step or you can take a steepest descent step.
And if you found the steepest descent step didn't quite get you to the boundary of your trust-region, you could then step in the Newton-Raphson direction.
Why do you do that?
I don't know, people have found that it's useful, right.
There's actually no good reason to take these sorts of dogleg steps.
People found that for general, right, general objective functions that you might want to find minima of, this is a reliable strategy for getting there.
There's no guarantee that this is the best strategy.
These are general non-convex functions.
These are just hard problems that one encounters.
So when you make a software package like Matlab, this is what you do.
You come up with heuristics that work most of the time.
I'll just provide you with an example here, OK.
So you've seen this function now several times.
Let's see, so in red covered up back here is the Newton-Raphson path.
In blue is the optimal steepest descent path.
And in purple is the trust-region method that Matlab uses to find the minima.
They all start from the same place.
And you can see the purple path is a little different from these two.
If I zoom in right up here, what you'll see is initially Matlab chose to follow the steepest descent path.
And then at a certain point it decided, because of the value of the trust-region that Newton-Raphson steps were to be preferred.
And so it changed direction and it started stepping along the Newton-Raphson direction instead.
It has some built in logic that tells it when to make that choice for switching based on the size of the trust-region.
And the idea is just to choose the best sorts of steps possible.
Your best guess at what the right steps are.
And this is all based around how trustworthy we think this quadratic approximation for objective function is.
Yeah, Dan?
So for the trust-region on the graph what Matlab is doing is at each R trust-region length it's reevaluating which way it should go?
Yes.
Yes.
It's computing both sets of steps, and it's deciding which one it should take, right.
It doesn't know.
It's trying to choose between them
Why don't you do the Newton-Raphson step through the [?
negative R?
?]
You can do that as well, actually, right.
But if you're doing that, now you have to choose between that strategy and taking a steepest descent step up to R as well, right.
And I think one has to decide which would you prefer.
It's possible the Newton-Raphson step also doesn't actually reduce f. In which case, you should discard it entirely, right.
But you could craft a strategy that does that, right.
It's still going to converge, likely.
OK?
OK.
I've going to let you guys go, there's another class coming in.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Well, everyone's quieted down, so that means we have to get started.
So let me say something here.
This will be our last conversation about optimization.
So we've discussed unconstrained optimization.
And now we're going to discuss a slightly more complicated problem-- but you're going to see it's really not that much more complicated-- constrained optimization.
These are the things we discussed before.
I don't want to spend much time recapping because I want to take a minute and talk about the midterm exam.
So we have a quiz.
It's next Wednesday.
Here's where it's going to be located.
Here's 66.
Head down Ames.
You're looking for Walker Memorial on the third floor.
Unfortunately, the time for the quiz is 7:00 to 9:00 PM.
We really did try hard to get the scheduling office to give us something better, but the only way to get a room that would fit everybody in was to do it at this time in Walker.
I really don't understand, because I actually requested locations for the quizzes back in April.
And somehow I was too early, maybe, and got buried under a pile.
Maybe not important enough, I don't know.
But it's got to be from seven to nine next Wednesday.
Third floor.
There's not going to be any class next Wednesday because you have a quiz instead.
So you get a little extra time to relax or study, prepare, calm yourself before you go into the exam.
There's no homework this week.
So you can just use this time to focus on the material we've discussed.
There's a practice exam from last year posted on the Steller site, which you can utilize and study from.
I'll tell you this.
That practice exam is skewed a little more towards some chemical engineering problems that motivate the numerics.
I've found in the past that when problems like that are given on the exam, sometimes there's a lot of reading that goes into understanding the engineering problem.
And that tends to set back the problem-solving.
So I'll tell you that the quiz that you'll take on Wednesday will have less of the engineering associated with it, and focus more on the numerical or computational science.
The underlying sorts of questions, the way the questions are asked, the kinds of responses you're expected to give I'd say are very similar.
But we've tried to tune the exam so that it'll be less of a burden to understand the structure of the problem before describing how you'd solve it.
So I think that's good.
It's comprehensive up to today.
So linear algebra, systems of nonlinear equations and optimization are the quiz topics.
We're going to switch on Friday to ordinary differential equations and initial value problems.
So you have two lectures on that, but you won't have done any homework.
You probably don't know enough or aren't practiced enough to answer any questions intelligently on the quiz.
So don't expect that material to be on there.
It's not.
It's going to be these three topics.
Are there any questions about this that I can answer?
Kristin has a question
[INAUDIBLE]
OK.
So yeah, come prepared.
It might be cold.
It might be hot.
It leaks when it rains a little bit.
Yeah, it's not the greatest spot.
So come prepared.
That's true.
Other questions?
Things you want to know?
What can we take to the exam?
Ooh, good question.
So you can bring the book recommended for the course.
You can bring your notes.
You can bring a calculator.
You need to bring some pencils.
We'll provide blue books for you to write your solutions to the exam in.
So those are the materials.
Good.
What else?
Same question.
OK. Other questions?
No?
OK.
So then let's jump into the topic of the day, which is constrained optimization.
So these are problems of the sort.
Minimize an objective function f of x subject to the constraint that x belongs to some set D, or find the argument x that minimizes this function.
These are equivalent sorts of problem.
Sometimes, we want to know one or the other or both.
That's not a problem.
And graphically, it looks like this.
Here's f, our objective function.
It's a nice convex bowl-shaped function here.
And we want to know the values of x1 and x2, let's say, that minimize this function subject to some constraint.
That constraint could be that x1 and x2 live inside this little blue circle.
It could be D. It could be that x1 and x2 live on the surface of this circle, right, on the circumference of this circle.
That could be the constraint.
So these are the sorts of problems we want to solve.
D is called the feasible set, and can be described in terms of really two types of constraints.
One is what we call equality constraints.
So D can be the set of values x such that some nonlinear function c of x is equal to zero.
So it's the set of points that satisfy this nonlinear equation.
And among those points, we want to know which one produces the minimum in the objective function.
Or it could be an inequality constraint.
So D could be the set of points such that some nonlinear function h of x is, by convention, positive.
So h of x could represent, for example, the interior of a circle, and c of x could represent the circumference of a circle.
And we would have nonlinear equations that reflect those values of x that satisfy those sorts of geometries.
So equality constrained, points that lie on this circle, inequality constrained, points that lie within this circle.
The shape of the feasible set is constrained by the problem that you're actually interested in.
So it's easy for me to draw circles in the plane because that's a shape you're familiar with.
But actually, it'll come from some sort of physical constraint on the engineering problem you're looking at, like mole fractions need to be bigger than zero and smaller than one, and temperatures in absolute value have to be bigger than zero and smaller than some value because that's a safety factor on the process.
So these set up the constraints on various sorts of optimization problems that we're interested in.
It could also be true that we're interested in, say, optimization in the domain outside of this circle, too.
It could be on the inside, could be on the outside.
That's also an inequality constrained sort of problem.
You know some of these already.
They're familiar to you.
So here's a classic one from mechanics.
Here's the total energy in a system for, say, a pendulum.
So x is like the position of the tip of this pendulum and v is the velocity that it moves with.
This is the kinetic energy.
This is the potential energy.
And we know the pendulum will come to rest in a place where the energy is minimized.
Well, the energy can only be minimized when the velocity here is zero, because any non-zero velocity will always push the energy content up.
So it comes to rest.
It doesn't move.
And then there's some value of x at which the energy is minimized.
If there is no constraint that says that the pendulum is attached to some central axis, then I can always make the energy smaller by making x more and more negative.
It just keeps falling.
There is no stopping point.
But there's a constraint.
The distance between the tip of the pendulum and this central point is some fixed distance out.
So this is an equality constrained sort of problem, and we have to choose from the set of v and x the values subject to this constraint that minimize the total energy.
And that's this configuration of the pendulum here.
So you know these sorts of problems already.
We talked about this one, linear sorts of programs.
These are optimization problems where the objective function is linear in the design variables.
So it's just the dot product between x and some vector c that weights the different design options against each other.
So we talked about ice cream.
Yes, this is all premium ice cream because it comes in the small containers, subject to different constraints.
So those constraints can be things like, oh, x has to be positive because we can't make negative amounts of ice cream.
And maybe we've done market research that tells us that the market can only tolerate certain ratios of different types of ice cream.
And that may be some set of linear equations that describe that market research that sort of bound the upper values of how much ice cream we can put out on the market.
And then we try to choose the optimal blend of pina colada and strawberry to sell.
So those are linear programs.
This is an inequality constrained optimization.
In general, we might write these problems like this.
We might say minimize f of x subject to the constraint that c of x is 0 and h of x is positive.
So minimize it over the values of x that satisfy these two constraints.
There's an old approach that's discussed in the literature.
And it's not used.
I'm going to describe it to you, and then I want you to try to figure out why it's not used.
And it's called the penalty method.
And the penalty method works this way.
It says define a new objective function, which is our old objective function plus some penalty for violating the constraints.
How does that penalty work?
So we know that we want values of x for which c of x is equal to 0.
So if we add to our objective function the norm of c of x-- this is a positive quantity-- this is a positive quantity-- whenever x doesn't satisfy the constraint, this positive quantity will give us a bigger value for this objective function f than if c of x was equal to 0.
So we penalize points which don't satisfy the constraint.
And in the limit that this penalty factor mu here goes to zero, the penalties get large, so large that our solution will have to prefer satisfying the constraints.
There's another penalty factor over here, which is identical to this one but for the inequality constraint.
It says take a heaviside step function for which is equal to 1 when the value of its argument is positive, and it's equal to zero when the value of its argument is negative.
So whenever I violate each of my inequality constraints, Hi of x, turn on this heaviside step function, make it equal to 1, and then multiply it by the value of the constraint squared, a positive number.
So this is the inequality constraint penalty, and this is the equality constraint penalty.
People don't use this, though.
It makes sense.
I take the limit that mu goes to zero.
I'm going to have to prefer solutions that satisfy these constraints.
Otherwise, if I don't satisfy these constraints, I could always move closer to a solution that satisfies the constraint, and I'll bring down the value of the objective function.
I'll make it lower.
So I'll always prefer these lower value solutions.
But can you guys take a second and sort of talk to each other?
See if you can figure out why one doesn't use this method.
Why is this method a problem?
OK, I heard the volume go up at some point, which means either you switched topics and felt more comfortable talking about that than this, or maybe you guys were coming to some conclusions, or had some ideas about why this might be a bad idea.
Do you want to volunteer some of what you were talking about?
Yeah, Hersh
Could it be that [INAUDIBLE]??
Well, that's an interesting idea.
So yeah, if we have a non-convex optimization problem, there could be some issues with f of x, and maybe f of x runs away so fast that I can never make the penalty big enough to enforce the constraint.
That's actually a really interesting idea.
And I like the idea of comparing the magnitude of these two terms.
I think that's on the right track.
Were there some other ideas about why you might not do this?
Different ideas?
Yeah
[INAUDIBLE]
Well, you know, that that's an interesting idea, but actually the two terms in the parentheses here are both positive.
So they're only going to be minimized when I satisfy the constraints.
So the local minima of the terms in parentheses sit on or within the boundaries of the feasible set that we're looking at.
So by construction, actually, we're going to be able to satisfy them because the local minima of these points sits on these boundaries.
These terms are minimized by satisfying the constraints.
Other ideas?
Yeah
Do your iterates have to be feasible?
What's that?
Your iterates don't have to be feasible?
Ooh, this is a good point.
The iterates-- this is an unconstrained optimization problem.
I'm just going to minimize this objective function.
It's like what Hersh said, I can go anywhere I want in the domain.
I'm going to minimize this objective function, and then I'm going to try to take the limit as mu goes to zero.
The iterates don't have to be feasible.
Maybe I can't even evaluate f of x if the iterates aren't feasible.
That's an excellent point.
That could be an issue.
Anything else?
Are there some other ideas?
Sure
[INAUDIBLE]
I think that's a good point
--boundary from outside without knowing what's inside
Short.
So you'll see, actually, the right way to do this is to use what's called interior point methods, which live inside the domain.
This is an excellent point.
There's another issue with this that's I think actually less subtle than some of these ideas, which they're all correct, actually.
These can be problems with this sort of penalty method.
As I take the limit that mu goes to zero, the penalty function becomes large for all points outside the domain.
They can become larger than f for those points.
And so there are some practical issues about comparing these two terms against each other.
I may not have sufficient accuracy, sufficient number of digits to accurately add these two terms together.
So I may prefer to find some point that lives on the boundary of the domain as mu goes to zero.
But I can't guarantee that it was a minima of f on that domain, or within that feasible set.
So a lot of practical issues that suggest this is a bad idea.
This is an old idea.
People knew this was bad for a long time.
It seems natural, though.
It seems like a good way to transform from these constrained optimization problems to something we know how to solve, an unconstrained optimization.
But actually, it turns out not to be such a great way to do it.
So let's talk about separating out these two different methods from each other, or these two different problems.
Let's talk first about equality constraints, and then we'll talk about inequality constraints.
So equality constrained optimization problems look like this.
Minimize f of x subject to c of x equals zero.
And let's make it even easier.
Rather than having some vector of equality constraints, let's just have a single equation that we have to satisfy for that equality constraint, like the equation for a circle.
Solutions have to sit on the circumference of a circle.
So one equation that we have to satisfy.
You might ask again, what are the necessary conditions for defining a minimum?
That's what we used when we had equality-- or when we had unconstrained optimization.
First we had to define what a minimum was, and we found that minima were critical points, places where the gradient of the objective function was zero.
That doesn't have to be true anymore.
Now, the minima has to live on this boundary of some domain.
It has to live in this set of points c of x equals zero.
And the gradient of f is not necessarily zero at that minimal point.
But you might guess that Taylor expansions are the way to figure out what the appropriate conditions for a minima are.
So let's take f of x, and let's expand it, do a Taylor expansion in some direction, d. So we'll take a step away from x, which is small, in some direction, d. So f of x plus d is f of x plus g dot d, the dot product between the gradient of f and d. And at a minimum, either the gradient is zero or the gradient is perpendicular to this direction we moved in, d. We know that because this term is going to increase-- well, will change the value of f of x.
It will either make it bigger or smaller depending on whether it's positive or negative.
In either case, it will say that this point x can't be a minimum unless this term is exactly equal to zero in the limit that d becomes small.
So either the gradient is zero or the gradient is orthogonal to this direction d we stepped in.
And d was arbitrary.
We just said take a step in a direction, d. Lets take our equality constraint and do the same sort of Taylor expansion, because we know if we're searching for a minima along this curve c of x better be equal to zero.
It better satisfy the constraint.
And also, c of x plus d, that little step in the direction d, should also satisfy the constraint.
We want to study only the feasible set of values.
So actually, d wasn't arbitrary.
d had to satisfy this constraint that, when I took this little step, c of x plus d had to be equal to zero.
So again, we'll take now a Taylor expansion of c of x plus d, which is c of x plus grad of c of x dotted with d. And that implies that d must be perpendicular to the gradient of c of x, because c of x plus d has to be zero and c of x has to be zero.
So the gradient of c of x dot d-- it's a leading order has also got to be equal to zero.
So d and the gradient in c are perpendicular, and d and the gradient in g have to be perpendicular at a minimum.
That's going to define the minimum on this equality constrained set.
Does that make sense?
c satisfies the constraint, c plus d satisfies the constraint.
If this is true, d has to be perpendicular to the gradient of c, g has to be perpendicular to the gradient of d. d is, in some sense, arbitrary still.
d has to satisfy condition that it's perpendicular to the gradient of c, but who knows, there could be lots of vectors that are perpendicular to the gradient of c. So the only generic relationship between these two we can formulate is g must be parallel to the gradient of c. g is perpendicular to d, gradient of c is perpendicular to d. In the most generic way, g and gradient of c should be parallel to each other, because d I can select arbitrarily from all the vectors of the same dimension as x.
If g is parallel to the gradient of c, then I can write that g minus some scalar multiplied by the gradient of c is equal to zero.
That's an equivalent statement, that g is parallel to the gradient of c. So that's a condition associated with points x that solve this equality constrained problem.
The other condition is that point x still has to satisfy the equality constraint.
But I introduced a new unknown, this lambda, which is called the Lagrange multiplier.
So now I have one extra unknown, but I have one extra equation.
Let me give you a graphical depiction of this, and then I'll write down the formal equations again.
So let's suppose we want to minimize this parabolic function subject to the constraint that the solution lives on the line.
So here's the contours of the function, and the solution has to live on this line.
So I get to stand on this line, and I get to walk and walk and walk until I can't walk downhill anymore.
and I've got to turn and walk uphill again.
And you can see the point where I can't walk downhill anymore is the place where this constraint is parallel to the contour, or where the gradient of the objective function is parallel to the gradient of the constraint.
So you can actually find this point by imagining yourself moving along this landscape.
After I get to this point, I start going uphill again.
So that's the method of Lagrange multipliers.
Minimize f of x subject to this constraint.
The solution is given by the point x at which the gradient is parallel to the gradient of c, and at which c is equal to zero.
And you solve this system of nonlinear equations for two unknowns.
One is x, and the other is this unknown lambda.
How far stretched is the gradient in f relative to the gradient in c?
So again, we've turned the minimization problem into a system of nonlinear equations.
In order to satisfy the equality constraint, we've had to introduce another unknown, the Lagrange multiplier.
It turns out this solution set, x and lambda, is a critical point of something called the Lagrangian.
It's a function f of x minus lambda times c. It's a critical point in x and lambda of this nonlinear function called the Lagrangian.
It's not a minimum of this function, unfortunately.
It's a saddle point of the Lagrangian, it turns out.
So we're trying to find a saddle point of the Lagrangian.
Does this make sense?
Yes?
OK. We've got to be careful, of course.
Just like with unconstrained optimization, we actually have to check that our solution is a minimum.
We can't take for granted, we can't suppose that our nonlinear solver found a minimum when it solved this equation.
Other critical points can satisfy this equation, too.
So we've got to go back and try to check robustly whether it's actually a minimum.
But this is the method.
Introduce an additional unknown, the Lagrange multiplier, because you can show geometrically that the gradient of the objective function should be parallel to the gradient of the constraint at the minimum.
Does that make sense?
Does this picture make sense?
OK.
So you know how to solve systems of nonlinear equations, you know how to solve constrained optimization problems.
So here's f. Here's c. We can actually write out what these equations are.
So you can show that the gradient of x minus lambda gradient of c, that's a vector, 2x1 minus lambda and 20x2 plus lambda.
And c is the equation for this line down here, so x1 minus x2 minus 3.
And that's all got to be equal to zero.
In this case, this is just a system of linear equations.
So you can actually solve directly for x1, x2, and lambda.
And it's not too difficult to find the solution for all three of these things by hand.
But in general, these constraints can be nonlinear.
The objective function doesn't have to be quadratic.
Those are the easiest cases to look at.
And the same methodology applies.
And so you should check that you're able to do this.
This is the simplest possible equality constraint problem.
You could do it by hand.
You should check that you're actually able to do it, that you understand the steps that go into writing out these equations.
Let's just take one step forward and look at a less generic case, one in which we have a vector valued function that gives the equality constraints instead.
So rather than one equation we have to satisfy, there may be many.
It's possible that the feasible set doesn't have any solutions in it.
It's possible that there is no x that satisfies all of these constraints simultaneously.
That's a bad problem to have.
You wouldn't like to have that problem very much.
But it's possible that that's the case.
But let's assume that there are solutions for the time being.
So there are x's that satisfy the equality constraint.
Let's see if we can figure out again what the necessary conditions for defining a minima are.
So same as before, let's Taylor expand f of x going in some direction, d. And let's make d a nice small step so we can just treat the f of x plus d as a linearized function.
So we can see again that g has to be perpendicular to this direction, d, if we're going to have a minima.
Otherwise, I could step in some direction, d, and I'll find either a smaller value of f of x plus d or a bigger value of f of x plus d. So g has to be perpendicular to d. And for the equality constraints, again, they all have to satisfy this equality constraint up there.
So c of x has to be equal to zero, and c of x plus d also has to be equal to zero.
And so if we take a Taylor expansion of c of x plus d, about x, you'll get c of x plus d plus the Jacobian of c, all the partial derivatives of c with respect to x, multiplied by d. We know that c of x plus d is zero, and c of x is zero, so the directions, d, belong to what set of vectors?
The null space.
So these directions have to live in the null space of the Jacobian of c. So I can't step in any direction, I have to step in directions that are in the null space of c. g is perpendicular to d, as well.
And d belongs to the null space of c. In fact, you know that d is perpendicular to each of the rows of the Jacobian.
Right?
You know that?
I just do the matrix vector product, right?
And so each element of this matrix vector product is the dot product of d with a different row of the Jacobian.
So those rows are a set of vectors.
Those rows describe the range of J transpose, or the row space of J.
Remember we talked about the four fundamental subspaces, and I said we almost never use those other ones, but this is one time when we will.
So those rows belong to the range of J transpose, or they belong to the left null space of J. I need to find a g, a gradient, which is always perpendicular to d. And I know d is always perpendicular to the rows of J.
So I can write g as a linear superposition of the rows of J.
As long as g is a linear superposition of the rows, it'll always be perpendicular to d. Vectors from the null space of a matrix are orthogonal to vectors from the row space of that matrix, it turns out.
And they're orthogonal for this reason.
So it tells us, if Jd is zero, then d belongs to the null space.
g is perpendicular to d. That means I could write g as a linear superposition of the rows of J.
So g belongs to the range of J transpose, or it belongs to the row space of J.
Those are equivalent statements.
And therefore, I should be able to write g as a linear superposition of the rows of J.
And one way to say that is I should be able to write g as J transpose times some other vector lambda.
That's an equivalent way of saying that g is a linear superposition of the rows of J. I don't know the values of lambda.
So I introduced a new set of unknowns, a set of Lagrange multipliers.
My minima is going to be found when I satisfy this equation, just like before, and when I'm able to satisfy all of the equality constraints.
How many Lagrange multipliers do I have here?
Can you figure that out?
You can talk with your neighbors if you want.
Take a couple minutes.
Tell me how many Lagrange multipliers, how many elements are in this vector lambda.
How many elements are in lambda?
Can you tell me?
Sam
Same as the number of equality constraints
Yes.
It's the same as the number of equality constraints.
J came from the gradient of c. It's the Jacobian of c. So it has a number of columns equal to the number of elements in x, because I'm taking partial derivatives with respect to each element of x, and has a number of rows equal to the number of elements of c. So J transpose, I just transpose those dimensions.
And lambda must have the same number of elements as c does in order to make this product make sense.
So I introduce a new number of unknowns.
It's equal to exactly the number of equality constraints that I had, which is good, because I'm going to make a system of equations that says g of x minus J transpose lambda equals 0 and c of x equals 0.
And the number of equations here is the number of elements in x for this gradient, and the number of elements in c for c. And the number of unknowns is the number of elements in x, and the number of elements in c associated with the Lagrange multiplier.
So I have enough equations and unknowns to determine all of these things.
So whether I have one equality constraint or a million equality constraints, the problem is identical.
We use the method of Lagrange multipliers.
We have to solve an augmented system of equations for x and this projection on the row space of J, which tells us how the gradient is stretched or made up, composed of elements of the row space of J.
These are the conditions associated with a minima in our objective function on this boundary dictated by the equality constraint.
And of course, the solution set is a critical point of a Lagrangian, which is f of x minus c dot lambda.
And it's not a minimum of it, it's a critical point.
It's a saddle point, it turns out, of this Lagrangian.
So we've got to check, did we find a saddle point or not when we find a solution to this equation here.
But it's just a system of nonlinear equations.
If we have some good initial guess, what do we apply?
Newton-Raphson, converge rate towards the solution.
If we don't have a good initial guess, we've discussed lots of methods we could employ, like homotopy or continuation to try to develop good initial guesses for what the solution should be.
Are there any questions about this?
Good.
OK.
So you go to Matlab and you call fmincon, do a minimization problem, and you give it some constraints.
Linear constraints, nonlinear constraints, it doesn't matter actually.
The problem is the same for both of them.
It's just a little bit easier if I have linear constraints.
If this constraining function is a linear function, then the Jacobian I know.
It's the coefficient matrix of this linear problem.
Now I only have to solve linear equations down here.
So the problem is a little bit simpler to solve.
So Matlab sort of breaks these apart so it can use different techniques depending on which sort of problem is posed.
But the solution method is the same.
It does the method of Lagrange multipliers to find the solution.
OK?
Inequality constraints.
So interior point methods were mentioned.
And it turns out this is really the best way to go about solving generic inequality constrained problems.
So the problems of the sort minimize f of x subject to h of x is positive, or at least not negative.
This is some nonlinear inequality that describes some domain and its boundary in which the solution has to live.
And what's done is to rewrite as an unconstrained optimization problem with a barrier that's incorporated.
This looks a lot like the penalty method, but it's very different.
And I'll explain how.
So instead, we want to minimize this f of x minus mu times the sum of log of h, each of these constraints.
If h is negative, we'll take the log of the negative argument.
That's a problem computationally.
So the best we could do is approach the boundary where h is equal to zero.
And as h goes to zero, the log goes to minus infinity.
So this term tends to blow up because I've got a minus sign in front of it.
So this is sort of like a penalty, but it's a little different because the factor in front I'm actually going to take the limit as mu goes to zero.
I'm going to take the limit as this factor gets small, rather than gets big.
The log will always get big as I approach the boundary of the domain.
It'll blow up.
So that's not a problem.
But I can take the limit that mu gets smaller and smaller.
And this quantity here will have less and less of an impact on the shape of this new objective function and mu gets smaller and smaller.
The impact will only be nearest the boundary.
Does that make sense?
So you take the limit that mu approaches zero.
It's got to approach it from the positive side, not the negative side, so everything behaves well.
And this is called an interior point method.
So we have to determine the minimum of this new objective function for progressively weaker barriers.
So we might start with some value of mu, and we might reduce mu progressively until we get mu down small enough that we think we've converged to a solution.
So how do you do that reliably?
What's the procedure one uses to solve a problem successively for different parameter values?
[INAUDIBLE]
Yeah, it's a homotopy, right?
You're just going to change the value of this barrier parameter.
And you're going to find a minima.
And if you make a small change in the barrier parameter, that's going to serve as an excellent initial guess for the next value.
And so you're just going to take these small steps.
And the optimization routine is going to carry you towards the minimum in the limit that mu goes to zero.
So you do this with homotopy.
Here's an example of this sort of interior point method, a trivial example.
Minimize x subject to x being positive.
So we know the solution lives where x equals zero.
But let's write this as unconstrained optimization using a barrier.
So minimize x minus mu times log x.
Here's x minus mu times log x.
So out here, where x is big, x wins over log x, so everything starts to look linear.
But as x become smaller and smaller, log x gets very negative, so minus log x gets very positive.
And here's the log creeping back up.
And as I decrease mu smaller and smaller, you can see the minima of this function is moving closer and closer and closer to zero.
So if I take the limit that mu decreases from some positive number towards zero, eventually this minimum is going to converge to the minimum of the constrained inequality, constrained optimization problem.
Make sense?
OK. OK.
So we want to do this.
You can use any barrier function you want.
Any thoughts on why a logarithmic barrier is used?
No.
OK, that's OK.
So minus log is going to be convex.
Log isn't convex, but minus log is going to be convex.
So that's good.
If this function's convex, then their combination's going to be convex, and we'll be OK.
But the gradient of the log is easy to compute.
Grad log h is 1 over h grad h. So if I know h, I know grad h, it's easy for me to compute the gradient of log h. We know we're going to solve this unconstrained optimization problem where we need to take grad of this objective function equal zero.
So the calculations are easy.
The log makes it easy like that.
The log is also like the most weakly singular function available to us.
Out of all the tool box of all problems we can reach to, the log has the mildest sort of singularities.
Singularities at both ends, which is sort of funny, but the mildest sort of singularities you have to cope with.
So we want to find the minimum of these unconstrained optimization problems where the gradient of f minus mu sum 1 over h, grad h, is equal to zero.
And we just do that for progressively smaller values of mu, and we'll converge to a solution.
That's the interior point method.
You use homotopy to study a sequence of barrier parameters, or continuation to study a sequence of barrier parameters.
You stop the homotopy or continuation when what?
How are you going to stop?
I've got to make mu small, right?
I want to go towards the limit mu equals zero.
I can't actually get to mu equals zero, I've just got to approach it.
So how small do I need to make mu before I quit?
It's an interesting question.
What do you think?
I'll take this answer first
So it doesn't affect the limitation
Good.
So we might look at the solution and see is the solution becoming less and less sensitive to the choice of mu.
Did you have another suggestion?
[INAUDIBLE]
Set the tolerance.
Right,
[INAUDIBLE]
Mhm.
Right, right, right, right.
So you--
[INAUDIBLE]
Good.
So there were two suggestions here.
One is along the lines of a step-norm criteria, like I check my solution as I change mu, and I ask when does my solution seem relatively insensitive to mu.
When the changes in these steps relative to mu get sufficiently small, I might be willing to accept these solutions as reasonable solutions for the constrained optimization.
I can also go back and I can check sort of function norm criteria.
I can take the value of x I found as the minimum, and I can ask how good a job does it do satisfying the original equations.
How far away am I from satisfying the inequality constraint?
How close am I to actually minimizing the function within that domain?
OK.
So we're running out of time here.
Let me provide you with an example.
So let's minimize again-- I always pick this function because it's easy to visualize, a nice parabolic function that opens upwards.
And let's minimize it subject to the constraint that h of x1 and x2 is equal to 1 minus-- well, the equation for a circle of radius 1, essentially.
The interior of that circle.
So here's the contours of the function, and this red domain is the constraint.
And we want to know the smallest value of f that lives in this domain.
So here's a Matlab code.
You can try it out.
And make a function, the objective function, f, it's x squared plus 10x-- x1 squared plus 10x2 squared.
Here's the gradient.
Here's the Hessian.
Here, I calculate h. Here's the gradient in h. Here's the Hessian in h. I've got to define a new objective function, phi, which is f minus mu log h. This is the gradient in phi and this is the Hessian of phi.
Oh, man, what a mess.
But actually, not such a mess, because the log makes it really easy to take these derivatives.
So it's just a lot of differential sort of calculus involved in working this out, but this is the Hessian of phi.
And then I need some initial guess.
So I pick the center of my circle as an initial guess for the solution.
And I'm going to loop over values of mu that get progressively smaller.
I'll just go down to 10 to the minus 2 and stop for illustration purposes here.
But really, we should be checking the solution as we go and deciding what values we want to stop with.
And then this loop here, what's this do?
What's it do?
Can you tell?
Is it Newton?
What's that?
Newton?
Yeah, it's Newton-Raphson, right?
x is x minus Hessian inverse times grad phi, right?
So I just do Newton-Raphson.
I take my initial guess and I loop around with Newton-Raphson, and when this loop finishes, I reduce mu, and it'll just use my previous guess as the initial guess for the next value of the loop, until mu is sufficiently small.
OK?
Interior point method.
Here's what that solution path looks like.
So mu started at 1, and the barrier was here.
It was close to the edge of the circle, but not quite on it.
But as I reduced mu further and further and further, you can see the path, the solution path, that was followed works its way closer to the boundary of the circle.
And the minimum is found right here.
So it turns out the minimum of this function doesn't live in the domain, it lives on the boundary of the domain.
Recall that this point should be a point where the boundary of the domain is parallel to the contours of the function, since actually we didn't need the inequality constraint here.
We could have used the equality constraint.
The equality constrained problem has the same solution as the inequality constrained problem.
And look, that actually happened.
Here's the contours of the function.
The contour of the function runs right along here, and you can see it looks like it's going to be tangent to the circle at this point.
So the interpoint method actually solved an equality constrained problem in addition to an inequality constrained problem, which is-- that's sort of cool that you can do it that way.
How about if I want to do a combination of equality and inequality constraints?
Then what do I do?
Yeah
[INAUDIBLE]
Perfect.
Convert the equality constraint into unknowns, Lagrange multipliers, instead.
And then do the interior point method on the Lagrange multiplier problem.
Now you've got a combination of equality and inequality constrained.
This is exactly what Matlab does.
So it converts equality constraints into Lagrange multipliers.
Inequality constraints it actually solves using interior point methods.
Buried in that interior point method is some form of Newton-Raphson and steepest descent combined together, like dog leg we talked about for unconstrained problems.
And it's going to do a continuation.
As it reduces the values of mu, it'll have some heuristic for how it does that.
It's going to use its previous solutions as initial guesses for the next iteration.
So these are very complicated problems, but if you understand how to solve systems of nonlinear equations, and you think carefully about how to control numerical error in your algorithm, you come to a conclusion like this, that you can do these sorts of Lagrange multiplier interior point methods to solve a wide variety of problems with reasonable reliability.
OK?
Any more questions?
No?
Good.
Well, thank you, and we'll see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so we're going to start a new topic today about ordinary differential equations initial value problems.
But before I start to talk about that, I want to remind you next week's schedule is weird.
So there's no classes on Monday, but instead we'll have Monday class on Tuesday.
So I'll see you next on Tuesday morning.
And then we're not going to have a class on Wednesday, but we have the quiz on Wednesday night.
And then we'll get back to normal on Friday.
And I would expect that possibly the TAs might be interested in giving a help session or review on Wednesday the class period
We'll be here at 3:00
They'll be here.
So if you want to come at class period, and just ask questions and stuff
And I'll have your PSAT graded back, if you like
All right, got that?
PSAT be ready as well.
OK, so I'll talk about today ODE-IVPs
Yeah, we'll still have office hours Monday [INAUDIBLE]
Can you guys hear this?
Yes, office hours like normal Monday.
And for those of you who want to get your homework started early, we might even post homework.
And I'm going to focus exclusively on ODE-IVPs you can write this way.
All right, and I would just comment, what do these things mean?
ODE means ordinary differential equation.
It means that the only differentials in the problem are with respect to one variable, so I'm going to call that t. It can be z or x or whatever you want, but I'm going to call it t here.
And so this is not partial differential equations where you have differentials respect to many variables.
Initial value problems mean that all the initial conditions are given in at a single t point where t is that the variable that appears in the differential.
All right so this is the form we're going to do.
This is first order a way of writing the ODEs.
If you recall from homework zero problem number one, we showed you there how you can write higher order differential equations-- convert them into the form of first order.
And I guess that's worthwhile to point down.
I would also comment that I going to just talk about f of y, however the ODE solvers in Matlab expect f of the t comma y as their from.
All right, so let's just explain all the details about this.
So suppose I had a differential equation like this.
All right, so this is like a differential equation you'd have for a body move moving-- a particle moving with some friction force on it and some time varying force.
Yeah?
Are you going to be posting your notes online?
No, I don't have any notes.
This is it.
All right, so this is a differential equation.
What you can do is to convert it into the form I have above, is to write a new variable, say v, that's defined to be dx dt.
That's the velocity.
And has a time dependence, but I told you I don't like time dependence, so I'm not going to write it that way.
So we can also define another variable.
So we can define our new y vector to be x, v, and t. So this is y-- the first component of y, this is the second component of y, this is the third component of y.
And then I can write down what f of y is.
So the equation for dx dt is y2.
It's the second component of y, dx dt is equal to v. The equation for the dv dt is given by this equation.
So that's f of y3 over m minus gamma y2.
And the equation dt dt is just one.
So this is my f of y.
So this is f1, f2, f3.
Is that all right?
So this is just how you can convert-- if somebody writes you an equation this way, and you want to get it into the standard form that I'm going to show you here-- dy dt equals f of y-- this is how you convert.
And a skill we haven't really emphasized so far in the course, but I think a very important skill, is to be able to get like a chemical engineering problem and rearrange it so it becomes a standard form that maps up to one of the solvers you know how to use.
So you've got a problem has some differential in it.
Right away you actually, can I somehow rewrite it so it looks like this standard form-- or actually for Matlab, this form.
So Matlab will say f of t, y.
If you can do that, then you can go ahead and use those Matlab solvers to solve it.
But your variables may not have a letter y appearing anywhere, right?
You have to figure out how to write it into the standard form.
And then once you can do that, you can use the solvers.
And the same thing with the non-linear equations solvers, with the optimization software.
If you can rewrite it into the form that matches the standard form, then once you're done, you're done.
Then you can just go and call the canned programs that work well for those forms.
All right, now what do we want as the output?
So this is the problem.
This is the problem as proposed.
What do we really want out of this?
What we're going to get out of it is a bunch of points y-- actually a matrix, right?
It would be a vector of y-values.
Typically y is a vector.
And we want to know it a bunch of time points.
And we would a lot of time points so we're getting pretty plots of how the components of y vary with time.
That's what we usually want.
Sometimes we just want the final value, y time value.
We integrate to some point and then stop, and just run through the final value.
But a lot of times, we actually want to make pretty plots, so we really want a lot of points here and making plots.
Now how many points do you need to make a nice plot?
Maybe, I don't know, 100 points, 50 points, you can make a nice plot.
So that's the kind numbers you'd like.
And so the output of these programs is going to be something like a vector of the time points, and then a matrix of the y-values corresponding to each time point.
And the way it does in that lab, it's reasonable.
Each row of y are the y-values that correspond to the same time point, the first time point.
So the first row of y corresponds to the first number in the time vector.
The second run is the second.
Last one will be at time final.
And then you can plot them.
OK so far?
And ideally, we would like these y-values that we compute, these y's we get out, ideally one that really close to what the true solution of the true differential equation is.
But we're doing numerical stuff, so it's never going to be exactly the same.
And so a big part of the effort is trying to figure out, how can we get the thing to compute y calculated at each time point to be as close as possible to the truth, of what the true solution of the equations are?
And that turns out to be difficult, so that's a very important thing to worry about.
What else can we say about this?
We specify this from t naught, and you typically have to input your t final, how far you want to go away from your t naught as another input.
Normally, people write it where t naught is less than t final, and so you're like integrating from left to right.
But you could actually write them, put a negative sign equation which corresponds to a negative t, change your variable to of t prime that was equal to t final minus t if you wanted to, and integrate the other direction.
And in fact in homework zero problem one, we did that too.
So you can integrate frontwards.
You can integrate backwards.
If somebody tells you the initial condition or condition in the center of the range, you integrate forward for part of the range, and backwards the other way, so anything like this.
So what's important for this to be what's called an initial value problem is just that all of the specifications of y are given at the same value of t. It doesn't matter exactly what value of t it is.
Later, we'll talk about what happens if you don't know all the specifications at one value of time.
All right, now how would you approach this?
Probably, a bunch of you've done this already as undergraduates, or maybe even high school.
You can write like a Taylor expansion, y of t plus delta t-- is y of t plus delta t-- times dy dt.
You can write that.
And then I say, wow, I know dy dt.
That's actually f, so this is actually equal to-- right?
And then if I truncate the Taylor expansion, now I have an update formula.
So if I just forget this last bit, I can put in my current value of y, like the initial condition.
The initial condition here, evaluate f, multiply by delta t, add them together, and I'll get a new value of y.
And then I can repeat over and over again.
And in fact, this is a very famous method.
It's called the Forward Euler method.
Euler was a pretty smart guy, so it's not completely ridiculous, but as I'll show you, it has some problems.
So the Forward Euler, another way to write it is y new is equal to y old plus delta t times f of y old.
It's called Forward Euler.
And we can write a little implementation of this pretty easily.
So you can write y is equal to y naught, t is equal to t naught for i equals 1 to the number of steps, y is equal to y plus delta t times f of y, t is equal to t plus delta t. Somewhere up here, I need to write delta t. OK, so there's your code for doing Forward Euler.
How many of you have done this before?
OK, so I can skip over this very fast.
How many did this, and it didn't work for a problem?
OK, so you're not doing hard enough problems.
Well, we'll correct that.
All right, so I noticed that this closely resembles the rectangle rule.
So the rectangle rule would look a lot the same.
However, if you're integrating a function-- so I have I is equal to the integral of f of t dt, then I can notice that the derivative dI dt is equal to f of t. And so if I was going to do the rectangle rule, it would look just the same as this, except this would be a t. Maybe I would pick y0 to be 0.
And the final value of y I would get to would be my integral, my i value that I want.
So how many of you have done rectangle rule?
Yes?
All right, so you did this already.
Now you remember in high school, when they took the rectangle rule, they probably mentioned that it's not very accurate.
And so then they also taught you some other ones.
You guys, how many of you learned the trapezoid rule?
How about the midpoint rule?
How about Simpson's rule?
All right, so at least a high school math teachers thought that this is inadequate for doing real work, so they tell you all these other things instead.
This was like your first baby thing.
So let's talk about why was that.
So let's do the Taylor expansion a little bit further.
So let's do it for the numeric [INAUDIBLE] case.
So y t plus delta t is equal to y of t plus delta t times f of t plus 1/2 delta t squared times the second derivative.
And so when we made the approximation to do the rectangle rule, we just threw this term away.
So that's a leading term that we threw away.
So we made an error order or delta t squared.
Because normally the second derivative is not exactly zero.
So we have an error in each step that's the order of delta t squared.
But how many steps do we make?
The number of steps is equal to t final minus t naught over delta t. So therefore, we're making one over delta t order steps.
So the total error is going to be approximately the number of steps times how much error you making each step.
So it's going to be something the second power in delta t divided by something to the first power of delta t. So the total error in the integral I is going to be older of delta t if you do the rectangle rule.
And so order delta t is not very good.
So you want to increase your accuracy-- if you cut your step size in half, that means you double your CPU time, because you have to do twice as many function evaluations.
But you only gain-- you reduce your error by a factor of two.
So if you want to, say, gain three significant figures of accuracy in your number, then you have to do 1,000 times much work as you did before.
So I have some amount of some precision with some number of steps.
I want to increase, get three more significant figures.
I'll have to do 1,000 times as much work.
That's kind of bad scaling.
If I want to get six more significant figures, I've got to do a million times more work.
So at some point, this going to be kind of expensive.
So if you contrast it, you can do things like trapezoid rule.
And as you're probably-- well let's talk about trapezoid rule for a bit.
So the idea of this is you have your function, you have some point t, and t plus delta t. You're trying to integrate between them.
Let's say t naught, t naught plus delta t. Try to integrate.
Here's the value of the function at t. Here is the value of the function that t plus delta t. If you do the rectangle rule, what you say is I just draw a line here and integrate that rectangle.
But say the real function really looks like this.
Then I missing that area there, so that's the error.
That's why the error's so big.
If I do that trapezoid rule, then the error-- I actually overestimate the value here, but the integral of the difference between this dotted line and the solid line is less than the integral between the solid line and that dotted line, so its more accurate.
So that's why people like the trapezoid rule better than the rectangle rule.
If you did the midpoint rule, you'd evaluate this function halfway between these two points and then draw a dotted line here and a dotted line there, and integrate that guy.
And it would overestimate for part, and underestimate for part, and they would partially cancel each other out.
And so that also would be more accurate than doing the rectangle rule.
So you did it before.
And so what we're going to try to do is, instead of using this formula we did before, we're going to replace this with something else that I'm going to call g. And g is something that's supposed to be the average value of f over the step.
Instead of just taking the value f at the starting point, I want to get sort of the average.
Right, because an integral is related to an average over the delta t. So g is going to be my way to estimate the average.
And there's going to be a zillion update formulas that different mathematicians have proposed over the years.
They give you different g's that tell you how to do the update.
And the key is try to find a g that's really accurate.
You want to have one that's the most accurate you can, because then your errors will be less.
And if your errors are less, than your delta t can be bigger, which means your number of steps will be less, which means your CPU time will be less.
So you might be able to get more accuracy and use less CPU time both if you can find a really good formula for g, for the update.
So what's the g for the trapezoid rule?
It's f of t plus f of t plus delta t over 2.
That was the g you used when you did the trapezoid rule when you were kids.
What is the g for the midpoint rule?
It's f of t plus delta t over 2.
And so the different g's are called different update formulas.
And these particular ones reduce the error from delta t squared for [INAUDIBLE] step to, I think, delta t to the cubed power.
And then when you multiply by this 1 every delta t, it turns out you go from being order of delta t to order of delta t squared, and that's a really big change.
So now if I cut the step size by a factor of 10, I gain a factor of 100 in the accuracy.
If I want to get six more significant figures, I do 1,000 times more work, not a million times more work.
So it's a pretty significant improvement.
And then people have pushed this on further and further.
And so actually very common integrators that you might use nowadays would go out to fourth and fifth order methods.
And so they have complicated update formulas that are carefully designed to cancel out all the low order delta t errors and just leave very high order ones.
And that's kind of the-- typical ones are like that.
And then some fancy ones might go even up to eighth order, I've seen, if you're really pushing it.
You have really complicated g formulas then.
Now, this is all for the integration, but really our problem was that the ODEs [INAUDIBLE]..
So we have to do f of y not f of t. Let's see if I can show you that.
Yeah, so we did the rectangle rule, we had f of t. But the real problem is we have to do f of y.
Now the problem is, we don't know what y is, y is our unknown.
So we generally have a problem here.
In Forward Euler, we can get away with it because we just put the y old value in.
Like for example, if we wanted to do midpoint here, we'd have to know f at a different time point that we haven't computed yet, because it's forward in time.
Same thing with trapezoidal rule.
We have to know f at some future timepoint.
So it's not so easy, actually, to go directly from these formulas to some explicit formula like this with a g in here, where the things inside g, the y's inside g, are all y-values we actually know.
So this is a serious problem.
But this didn't stop people from doing it.
So one idea is you could go back to the Taylor expansion, and now do the Taylor expansion but in terms of f of y.
So this is f of y of t. And then over here, this is d squared y dt squared, but dy dt is f, so this is really df dt.
But f is actually not a function of t, it's a function of y, right?
But we can do chain rule.
So df dt is equal to the sum df dyn dyn dt.
But dyn dt is f. And this is what we call the Jacobian [INAUDIBLE]..
So this is really saying that this is equal to J times f. So that would be one possibility, is you could go and write the Taylor expansion out to the next higher order explicitly, putting in J times f here, and then just being in the cubic terms.
Now, people don't do this usually.
Any idea why this is not a popular thing to do?
Yeah?
It's computationally expensive to [INAUDIBLE]
That's right.
That's right.
So how many function evaluations does it take to get the Jacobian?
Right, so each function is n values, right, because it's a vector.
And you have to do at least, say, forward differences would be n function evaluations.
So instead of just doing one function evaluation like we were doing with the Forward Euler, now we have to do n plus 1, or something like that, function evaluations.
And actually, you probably want to use center differences, so you need 2 times n plus 1.
And if you do them analytically, unless it's sparse, it's still going to be really expensive to compute it.
So it's a lot more effort to do it.
So you need to think, well, is it really worth it?
And so people have found other formulas that basically get rid of the delta t squared term that don't require so much effort, and so that's what people do instead.
But this is an option.
You could do it.
All right, so instead, what people do is they've decided to generalize the midpoint rule and try to figure out, how can we get an estimate of the midpoint value that's cheap to evaluate?
And so what they say is, well, this is g midpoint for numerical integration.
Let's make an new g midpoint that works for when f is a function of y.
So let's make it function of y evaluated at t plus delta t over 2.
So that's what the formula looks like.
But then you say, well, I don't know y is at t plus delta t over 2, because I only know it at y of t. So then they say, well, let's do Forward Euler for that.
So then they say, well, let's say it's approximately f of-- what's the Forward Euler formula?
It's y of t plus delta t over 2 times f of y of t. How many parenthesis there?
So this is the formula that's actually used in practice a lot.
And this one is called Runge-Kutta two, second order Runge-Kutta formula.
And it's just a-- it's the midpoint rule where we estimate the value of y of t plus delta t using Forward Euler.
And the advantage of this is it's very cheap, right?
Just evaluate-- I just have to evaluate one function here, one function here, so two function calls.
[INAUDIBLE] and that way I can get an update formula.
And this one, turns out that it has the nice delta t scaling that you might like.
So that's the kind of thing people do.
And the Runge-Kutta guys went crazy, and so one of the most popular solvers is called Runge-Kutta four five, and that's the ODE four five program in Matlab.
And what that does is the fourth order extension of this, and the fifth order extension of this.
And it uses them both, and it compares them, and uses the difference between an estimate the error in the calculation.
And then uses that to decide what size delta t you need, depending on what tolerances you demand.
And the formulas for all those-- [INAUDIBLE] formulas are given in the textbook.
Now, this starts to lead the problem.
So we weren't able to actually evaluate the true g midpoint, which we'd like to be here.
Instead, we have to use some approximation in order to extrapolate forwards to the y.
So this is hinting at the whole problem.
This whole problem is actually we're just extrapolating.
We know the true value of y only at t naught, and all the rest we're doing is extrapolating forwards.
And we're do it over and over again, so we're going to do it for n steps.
Maybe we'll do 1,000 steps.
So you're extrapolating.
And then based on that extrapolation, you going to extrapolate again.
And then based on extrapolation, you going to extrapolate again.
And you can see that this is not really the most robust thing to do, right?
Because if you have any experience, you're not that comfortable extrapolating at all.
And then you have to extrapolate 1,000 times.
You should be a little bit worried about what can happen.
So let's think about how the error will propagate.
So we're going to have some errors while we do these extrapolations.
So we have here a t naught, t naught, we're happy.
We know y naught.
This is exact.
We're like woohoo.
We know one value of y, really good.
All right, so now the question is the first step.
Now we're going to get to value y1 that we compute with one of the formulas, and it's going to be good to some accuracy depending on how good or g is, our update formula.
And so it's going to have an error.
I should comment that this y naught, although we treat it as exact, it could have an error too.
But it will probably be more like a machine precision kind of error because we don't really know the initial conditions that well.
Or might be an experimental error about how well we know what the initial conditions are.
But I'm not going to worry about that one for now.
But be aware, this is true, we don't really know initial conditions perfectly either.
But certainly, even if we treat this as being known perfectly, we're going to have some error by the time we extrapolate to y1.
So now what happens at y2?
So now we've added some more, so now we're at t2.
And we see started from y1 true plus some error.
This is y-- y naught was true.
Now we're starting from y1 true with some error in it, because of a mistake we made the first step.
And now we see how that step propagates further.
So let's see what that's going to say.
Well say y2 is going to equal y1 plus delta t times some update formula which would depend on y1.
But y1 was not true anymore, so this is actually equal to what should have been the true value of y1 if we only knew what it was, but we don't know.
And we have our error that we don't know exactly how big it is that we added on.
And then we have another term from this guy, so it's going to be delta t times g of y1 true plus delta 1.
And then we know our update formula's not right anyway, so there's actually another error, delta 2, just because of the error in the update formula.
Now if we had a great update formula, these deltas are kind of small.
And if somehow, by luck, our delta 1 was 0, and we had a great update formula, then we think that this error in y2 is going to be pretty small.
But it's not true.
So really the errors are pretty big.
So let's see if we can figure out-- we're going to do a Taylor expansion here, so this'll be g of y1 true plus d dgy-- the Jacobian of g with respect to y-- times the delta 1.
That's the first order Taylor extension.
All right, so now let's group the terms.
So y2 is equal to y1 true plus the update formula of y1 true plus the error in our update formula if we had started at the true value.
So this is what we-- if somebody told us what the true value of y1, this is the size error we'd expect.
It's to be that little error we calculated before.
But then on top of it, we have these other terms.
We have a delta 1, because y1 was not the true value.
Sorry, I lost a delta t here.
And then we have delta t times dg dy times delta 1.
And I'll emphasize these are all vectors.
I think that's it if we only keep to first order.
So what is this thing?
This is a matrix, right, dg dy.
I could write it this way.
It's the identity matrix plus delta t times the Jacobian dg dy times the vector d1.
And you can see, at every iteration when I do this, I'm going to get a similar factor like this.
And it's going to again multiply delta 1.
I keep on multiplying the-- errors from my previous step keep getting multiplied by this kind of factor.
So it's going to be really important to us about what the norm of this matrix is.
What's going to happen if the normal of this matrix is big?
What's going to happen?
[INAUDIBLE]
That's right, the error's going to get multiplied by some factor, say bigger than 1, every iteration.
And after we go 1,000 iterations, how big is the error going to be?
Really big, right?
Even if that thing is quite close to 1, 1 plus something to the 1,000 power it's gigantic.
So what was originally a pretty small error-- because we chose a good g method-- is going to get multiplied by this huge amplification factor.
So the key thing for a method to be what is called numerically stable is that the norm of this matrix has got to be less than 1.
Or maybe even equal, equals 1 might be OK.
If this is much bigger than 1, you're doomed.
Your numeric order is just going to blow up from step to step to step.
And what's bad about it, is actually sometimes you might not even be aware of this.
So you'll get some solution, because it still will calculate some numbers for the y's at each type-- y1, y2, y3, y4-- it's just running through the calculation.
But the numbers may become increasingly meaningless further away from the y true as time goes on, as the steps go
[INAUDIBLE]
Yes, it can.
Yeah, so that's what we call a stable numerical method is one that, if you make an error in an earlier step, its impact diminishes as the steps go on.
That's what you want.
You can't always achieve this.
But if you can, that's really great.
Then that's you would call a really stable, robust kind of method.
Even though I give you some stupidity early on, my stupidity evaporates away, and it goes to the true solution.
That's what you want.
OK, so let's think of cases where it's going to be problematic for sure.
So if dg dy-- say if all it's eigenvalues are positive and you add it to identity matrix, the thing is still going to have eigenvalues bigger than 1.
Can you see that?
And actually, what do we say when dg dy is positive, has a positive eigenvalue?
You guys have a word for this already you learn in your differential equations class.
What do you call these kind of differential equations?
Maybe not dg dy.
Back up, how about df dy?
If df dy, if that Jacobian has a positive eigenvalue, what do you say?
You guys remember this?
So this is what we call unstable differential equations.
So if it has any positive eigenvalues, then two initial conditions that differ by a little differential exponentially separate as time goes on as long as the difference as any projection in the direction of the bad eigenvector.
So those are called numerically unstable.
A lot of those actually exist in reality.
So explosions, that's because you had some positive eigenvalue somewhere for example.
Amplifiers, like you buy an amplifier to crank up the music, you get positive feedback when-- remember Professor Swan walked to some strange place, and for some reason, he got the feedback from the speakers?
So he had a positive eigenvalue going there.
A little tiny noise and his microphone, and it amplified to a big squeak.
So there are real systems like this that amplify.
And sometimes we want to model them, like explosions for examples.
That's a pretty important for chemical engineers.
But it's going to be really tough, because if it's got positive eigenvalues, then this is probably going to have a norm bigger than 1.
And then whatever little errors we may get any step in the whole calculation, we're going to amplify and amplify and amplify.
And who knows if we're going to have any reality left in our solution by the time we get to the end.
All right, so that's one kind of problem.
So how about we have a problem that's intrinsically stable.
That means that all the eigenvalues of the Jacobian of the original problem are negative.
So the thing should be perfectly stable.
From whatever initial condition they started from, the whole thing should sort of come down to like an equilibrium point.
So that's a pretty common situation in chemical engineering too.
You start from some state, and it relaxes down to like an equilibrium state or a steady state.
Well-behaved systems act like that.
So what I'm going to show you is that, despite the fact that the df dy is well-behaved, it's possible that this sum matrix might still be poorly behaved and still have a norm bigger that one.
And so that means you started for a problem that was pretty well-behaved, and you broke it by your choice in numerical method.
So this is a very bad one.
This is the kind that can make you really embarrassed if you're working in a company.
Your boss gives you a problem.
He says, please tell me the time is going to take our reactor to relax after a start up.
the system is perfectly well-behaved.
He gives these beautiful equations that your previous coworker worked out.
Everything's fine.
You put it in there.
You use your stupid numerical solver, and you get a wild oscillation, and the thing blows up.
OK, and that's going to be because of a poor choice of g so that it makes this norm of this matrix bigger than one.
It's pretty easy for that to happen.
So you don't need a very complicated case to show it.
So let's just do a scalar case.
Say dy dt is equal to negative 2y.
So this is a well-behaved differential equation.
What's the solution?
So with a y naught y at t equals 0 is equal to 1.
What's the solution?
[INAUDIBLE]
Yeah, OK, so it's perfectly well-behaved function.
And the plot when you plot it, here's one.
It goes [PLANE DIVING NOISE].
So now let's solve this with the Forward Euler method.
So I'm going to say that y new is equal to y plus delta t times f. So in this case, this is f, right there, negative 2y.
So for Forward Euler, this is just saying that this is negative 2y.
All right, so let's choose a delta t, say delta t equals 1 to start with.
So we're going to compute.
We start at t equals 0.
We're computing out to t equals 1. so we're out here.
So we put 1 in here.
Our initial condition was y was 1, so that's 1.
So as 1 times negative 2 times 1 is negative 2.
This is a 1.
1 minus 2 is negative 1.
So at the first step, it goes from here down to there.
This is not looking good.
This physical variable is supposed to be gradually relaxing towards 0.
Going negative is not what you want.
And then I think we could go the next step, if you want.
So we can put in suppose y is negative 1.
And we're still doing a delta t of one step.
So negative 1 times negative 2 is positive 2.
Plus negative 1, so this goes back to the starting point.
We'll get the sawtooth, is what you get.
And it turns out, actually, if you make delta t any larger than 1, then this sawtooth amplifies, so you have a oscillating solution that explodes.
Which is a little bit different than the physical solution, which gently relaxes.
And so this is just a scalar equation that you guys could have done in high school.
If you high school math teacher had been brave, he would have showed it you.
But he don't want to ruin your faith in the Forward Euler method, so he just showed you a simple one that worked out.
Now you can make this work better by making the delta t smaller.
If you made the delta t small enough-- in this case, I think it's delta t is less than half-- then this actually behaves somewhat reasonably.
And so what's going on there is that the identity matrix is positive.
My Jacobian is negative.
And if I make delta t small enough, this big positive number is bigger than this smaller negative number, and the whole thing stays positive but less than 1, and so then the normal is good.
But if I make delta t too large, I make this second term much larger.
If it gets much larger than 1, it overwhelms the first term.
The thing's negative, but I what do the norm it's positive.
And so it has an expansion.
Then the negative is why we get these oscillations.
So if you ever do a numerical differential equation solution and you start seeing this kind of stuff, it's almost certainly related to this numerical instability of the method.
and not really a physical thing.
I mean, there are physical things that do this, but not too often in chemical engineering, fortunately.
All right, questions?
No questions, OK.
So in the textbook, they have a big discussion about how to evaluate different g methods in order to figure out if they're guaranteed to be stable, or under what conditions they'll be stable.
And so that's worth a look to see.
And they have detailed derivations for several of them.
Questions?
All right, so for the homework, homework four, you're going to have to write your own ODE solver.
So you'll do update formulas and compare them.
When we come back on Tuesday, I'll show you how this is usually dealt with by switching to implicit solvers and talk about that.
All right, have a good weekend.
Enjoy the three days off.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, I think we're gathered.
So let's get going.
We were going to DAEs today, but we decided to postpone for one day and do a little catch up on different things that you should know.
And I thought I'd first back up a minute and recall what you've learned so far.
So we started out talking a lot about linear algebra and, particularly, the problems of this type where you're given a matrix and you're given a vector, and you want to figure out what the x is that makes this equation true.
And you've become extremely familiar with the convenient Matlab backslash function.
You've probably used this a lot of times.
And if you remember, this is one of the only problems where we often have a unique solution.
We know a solution exists.
And we can do a finite number of steps in order to get the solution.
And actually, the method we have is really good.
So it almost always gives us the solution and to machine accuracy.
So this is a brilliant method.
And so everything is based on this in our whole field, actually.
So we learned this first.
We learned it takes n cubed operations in order to get the solution, where n is the size of the vectors.
And when we do the solution, the x we get for backslash, invariably, it either is a solution, solves this equation to machine accuracy, in the sense that you multiply m times x, you get b to as many digits as you can read.
However, we talked about how m could be poorly conditioned so that there might be multiple x's that would solve this equation to machine accuracy.
And so that's the problem with ill conditioning.
So it's not that it doesn't solve the equation.
It's that we're not really sure that's the right x.
You could get a bunch of different x's that, basically, solve it as well as we can tell.
All right, so do you guys remember this?
All right, and so that also implies that if you make a very small change in b, you might get a gigantic change in x.
Because even for one b, you have a quite a range of x's that could work, all right.
Then we try to solve problems like this.
Right, systems of equations, not just linear ones, but nonlinear ones as well.
And what we decided to do as one of our best methods is Newton-Raphson, where we keep on doing j backslash f [INAUDIBLE] negative f. Right, so all we did in this method, a lot of times, is we just kept on calling this method over and over again.
All right, so this took order of n cubes.
This thing is going to take order of n, I don't know what you call it, steps or something.
Maybe if you guys have a word for it.
What do you call the iterations?
Iterations.
So it takes the number of iterations times n cubed.
Now, if you really can make Newton-Raphson work, and you have a really great initial guess, the number of iterations is really tiny.
Because Newton-Raphson is such a fantastically great convergence.
However, we're usually not such good guessers.
And so it might take a lot of iterations.
And in fact, we use the dogleg method to solve this as we've discussed about where you mix up different methods.
And you have a trust region and all this baloney.
And so it might actually be quite a lot of iterations to get there.
And then we went and tried to solve this problem.
And what we decided to do for both of these was almost do the same thing.
So I said, well, we can really solve this one by instead solving this.
Right, that's really what we ended up doing.
That's what happens inside fsolve is it converts the problem of finding f of x equals 0 into a minimization problem to try to make the norm of f as small as possible.
Because the answer will be when the norm is zero.
And if f of x equals 0 has a solution, then the global minimum here is a solution, as long as your initial guess is good enough that you're in the valley that leads to the true minimum.
Then this will work.
And that's actually what fsolve does.
So these two guys get kind of coupled together, the minimization problem and the root-finding problem, or the problem of solving systems of non-linear equations.
And in both of them, we're going to call the first method a million times.
OK, so that's the way we're working so far.
And actually, when you do fmincon, which was the next level, then you actually end up looking for saddle points, which, again, we look for as minimizing the norm of a gradient squared, which gets it back into this kind of form.
And so they're all the same thing.
And you can, again, work out estimates of how much effort it takes.
And it's something times n cubed, Typically, if most your work is this.
Now, I should make a comment that, in practice, that's not always true.
So in practice, sometimes, although the highest order term is n cubed, the linear algebra programs that do this are so great that the prefactor is tiny.
So in order to get the limit where the n cubed term was actually dominate the CPU time, you had to have a gigantic n. And in fact, the limit, it's often the effort to construct the matrix j.
And so it really goes as n squared.
So although this nominally is like n cubed, in practice, a lot of times, it's n squared.
And I say it for these other methods.
And then a lot of them get into how good the initial guesses are.
Because the number of iterations can get crazy.
And in fact, maybe, it never converges if your initial guess is bad.
So that can be a really critical thing.
[INAUDIBLE] is better.
Because you can't change n I mean, how many unknowns you've got is how many unknowns you've got.
But you can change the number iterations by, say, more clever initial guessing.
And that was the whole concept about continuation and homotopy and all this kind of stuff was trying to help reduce this number.
Last time, we talked about implicitly solving ODEs.
And so we had an update formula.
So we were solving an ODE.
The initial value problem, y of-- Right, and we said that, first, we went through the explicit methods.
And they were perfectly straightforward to do.
And you don't have to solve any systems of nonlinear equations to do them.
And so that was great.
But then we pointed out that, a lot of times, they're numerically unstable.
And so then you have to instead use implicit methods.
But the implicit methods generally work this way.
They have your new value of y.
What you're trying to compute is your current value of y plus delta t sums g. It depends on the size of delta t and y new.
Maybe y old also.
And these g's are things like the Crank-Nicolson, which you're going to work on in the homework, and we show it in class too.
And so there's different update formulas of people suggesting how to do this.
The trick of this is that the y new is inside the function.
So therefore, this is really another problem like we had before.
You can really rewrite this that way by just putting this on the one side.
Put them all on one side of zero.
And now, I'm trying to figure out what y new is.
And I'm going to solve that using this method.
And I'm going to solve that by using this method.
All right, so now, how is the scaling going to go?
At each time step, I have to solve one of these guys.
How much effort did that take?
We did that, right.
Here it is, n [?
after ?]
times n squared.
And then how many times do I have to do it?
It's h times 7 times [INAUDIBLE]
Right it depends on how many times steps I have [INAUDIBLE] my delta t. So it's something like t final minus t 0 over delta t. That's the number of times steps.
But I have constant delta t. Times the number of iterations that it takes to do the nonlinear solve times n squared, which is a cost of forming the Jacobian matrix.
Now, this can be pretty large.
So as we talked about, this might be as big 10 to the eighth in a bad case.
Certainly be less than 10 to the eighth because otherwise, you'll have numerical problems.
Maybe, it's really going to be 10 to the sixth for a real problem.
So you have a million time steps.
At each times step, you have to solve the number of iter-- the solve costs this much.
Number of iterations might be what?
How many iterations do you guys take to solve nonlinear equations?
You solve a lot of them now.
How many iterations does it take?
10
10, does that 10 sound right?
Yeah, [INAUDIBLE]
You've got an initial guess 10.
If you have a medium initial guess, maybe 100.
It it's more than 100, you're not going to converge, right?
Yeah
Yeah, OK.
So this is order of 10 maybe.
So the million 10 and then n squared just depends on how big your problem is.
So in my group, the problems we typically do, n is approximately 200.
So this is like 200 squared.
And then you can see that the number of iterations, and this is maybe a little bit overestimated.
Maybe I can get away with 10 to the fifth, something like that.
But you can see it starts to get pretty big.
But for writing it, as you guys are writing software, it's not that big a deal.
So I can assign it as part 1 of a three-part homework problem.
It's the right ODE solver.
It solves implicit equations.
And you can write it.
And the reason you can is because you've already written methods that solve this.
Actually, Matlab did it for you.
But you write it yourself.
I assigned to the [?
10:10 ?]
students.
So you write the [?
backsub ?]
solution to solve the equations.
And we we've already wrestled this.
And I think you guys wrote one of these yourself.
And also, we can use fsolve, which is just another little bit better implementation of the same thing.
And then now, you write your next level.
So you can see you're doing a composite process, where you're doing something at the top level.
It's calling something at a lower level.
That's calling something at a lower level.
And then what you really have to be concerned about if you're worried about CPU time or whether the thing is going to solve before the homework is due, is how big does this get.
How many operations are you really asking the computer to do?
Now, remember the computer is fast.
Right, so it can do a gigahertz or something.
So you can do like 10 to the eighth somethings per second.
And what should I say about this?
I have not included, but I should that it's-- well, I'm assuming here that the function evaluation is sort of order n of how many variables they have.
OK, sometimes, function evaluations are much more difficult than that, primarily because, sometimes, they're actually including further nested stuff inside.
So your f is actually coming from some really complicated calculation itself.
And then you have another big giant factor of how much it costs to do f. But on this right here, I'm just assuming it goes as n. So actually, let's just multiply this out.
So this is what, 10 to the fourth, 10 to the 10th.
So this is 10 to the 10th.
10 to the 10th kind of flops.
But your computer does 4 times 10 to the ninth per second.
Is that right?
Is that [?
what they ?]
told us, four gigahertz?
Yeah, so is this really not bad with just a few seconds on your computer.
So that's why you can do these homework problems, and you still get them done.
And if you have even [?
porous ?]
limitations, it takes 1,000 times longer.
Still, 1,000 [INAUDIBLE] take 20 minutes on your computer.
So you're OK, but you can see we're starting to get up there.
And it's not that hard to really make it a big problem if you do something wrong.
And just a notation thing, a lot of these things we're doing are writing software that takes as input not just numbers or vectors.
So those are things that we call functions.
Right, things that take a number of vectors and input and give you, say, a function, a number of vectors and output.
But oftentimes, we actually want to take the names of functions as inputs.
So for example, the root-finding problem, it takes as input what is f. Right, you have to tell it the name of a function for fsolve to be able to call it.
So fsolve is an object that takes as input the name of a function.
And things that do that are called functionals So for example, this one is x is equal to this root-finding functional, fsolve.
And I'm going to write with square brackets.
It depends on f. That's it, right?
[INAUDIBLE] f. So you give it the function f. And if fsolve's good, it should be able to tell you what the x is.
Now, in reality, we help it out.
So we give it x 0 [?
2 ?]
just give it a better chance.
I write the square brackets just indicate that one of the things inside is a function, not just numbers.
We do this a lot.
So you guys have done this since you were kids.
So derivatives Yeah, ddx of f. So we have, I don't know, for example, grad of f. That's a functional, the grad.
The symbol means the operator, the functional, it operates on functions.
You could write a general gradient function that takes any scalar valued function and computes the gradients with respect to the inputs.
This week, we give you one that computes a Jacobian.
Give it any f of x.
It gives you the Jacobian of f with respect to x.
So that's a functional.
So anyway, we do a lot with functionals.
You've been using them all the time.
I'm just trying get you to think about abstractly.
And then there's a whole math about functionals.
And there's a whole thing called calculus of variations.
It's all about, if the objects you're worrying about are functionals, not functions, then you have another whole bunch of theorems and stuff you can do with that, which you'll probably end up doing before you get out of this place.
But I'm not going to do that right now.
I would comment that one kind of functional that's particularly important to a lot of people in your research is the fact that the energy of any molecule is a functional of the electron density.
And this is the basis of what's called density functional theory.
And so this is why a lot of people, you and many of you included, are going to end up using computer programs that are trying to find what the electron density shape is, things like molecular orbitals and stuff.
You've probably heard these before.
And then there's the functional that gives the energy of the molecule.
And what you care about, mostly, is energy, because that connects to all your thermo.
You're doing [?
1040, ?]
it's all about the energies.
And once you know all the energies, you can actually compute the entropies and all kinds of stuff too.
And so this is one application that's really important.
It was a little surprising this was true and able to be proven to be true.
And so the guy who did that got the Nobel Prize-- his name was Cohen-- a few years ago.
Actually, in liquid phase stuff, you work with Professor Blankschtein.
They also have a version of this for properties of solution.
And then where it's not the electron density, but it's actually the density of the material in the solution phase.
Maybe Professor [?
Swan ?]
too.
You guys do that?
Yeah, so if you work with Professor [?
Swan, ?]
you might learn about that too.
So that's one where the actual functional is like a big focus of the entire project.
And the key is that this is a function.
The density is a function of the position.
And then the functional converts that into a number.
That's all page one here.
Now, I'm going to sort of change topics, but it's related.
It's about the connections between numerical integration, interpolation, and basis functions.
These things are all intimately tied up together.
So I'll try to show you.
So we try to solve a numerical integral.
And then let's look at what we're really doing.
When we're doing this, we typically only do a few function evaluations.
So we have a few points where we know the numbers f of tn.
We've picked a few t's.
We evaluated f. And from those few points, that small number of function evaluations, we want to get a good estimate of this whole integral.
Now, it's a little goofy, actually, if you think about what we're trying to do.
So suppose you're trying to do an integral from negative 1 to 1.
We have some function that has a point here, a point here, a point here.
And just for concreteness, let's say this is 1 over 26.
And this is 1.
And this is 1 over 26.
And I have a particular function in mind that has those three points.
Now, the first thing to keep in mind is there's an infinite number of functions that go through those three points.
Now, what we've been doing is we've been fitting some function to these points and then integrating that function that we fitted.
So when you're in high school, you did the rectangle rule and what the function was used was this and this.
And you knew the integral of this function and the integral of this function.
And you added them up.
And you decided from the rectangle rule that the integral was equal to 1 and 1/26.
But then some other people in high school are a little smarter.
And they said it's not so smart [INAUDIBLE]..
This thing was symmetrical to begin with.
And now, I'm fitting it with a function that's wildly unsymmetrical.
So let's instead not evaluate those points.
Let's instead evaluate the midpoints.
And maybe we find out that the midpoint values are here and here.
And so I would assume that the function is just a flat line.
That looks not so good either, because it doesn't get through these points.
But anyway, for the two points I knew, if I only used those two points, that would be the midpoint method.
And I would get the I midpoint method.
And I'd get something.
And I did it for this particular function I have in mind.
And it was 8 over 29.
And then some other people say, well, let's use the trapezoid rule.
So let's approximate the function is this over the interval, which looks kind of sensible.
And if you do that, it turns out you get the same numbers as you do with the rectangle rule.
So I trapezoid.
Now, let me just as a warning, if you just do the rectangle rule and then the trapezoid rule, and you got the same number both times, you might be tempted to conclude you have converged.
And you have the true solution, because you did two methods of really different orders, and you got exactly the same number.
And you're super happy, and you're ready to publish your paper.
But of course, the function might not look like this.
So the particular function that I always had in mind was f of t is equal to 1 over 1 plus 25 t squared.
OK, that was just the function I had in mind.
And the way that function looks like is like that.
And the real integral of that function, which you guys if you remember how do it from high school, you can actually evaluate that integral.
It's nothing to do with either 8/29 or 1 in 126.
It's something completely different.
Now, those of you who went to really advanced high schools, you also learned Simpson's rule too.
And Simpson's rule says let's fit it to a parabola.
And so that would give you something like this.
And if you do Simpson's rule, I ended up getting this if I did the arithmetic right.
1 and 1/3 plus 2/78.
Who knew?
OK, which also has little relation to the true value of the integral.
Now, if you're smart, and you did trapezoid and then I it to Simpson's, then you say, oh my goodness, these are not really that similar.
And so I'm not converged, and I'd better do something else.
So I just want you to be aware when we do these rules, what we're actually doing is we're fitting our points to some continuous function that we know how to integrate.
And then we do the analytical integral of that function, which is a fine thing to do.
But just be aware that it's like extrapolation.
It's like we're imagining what the function is doing between these points, when we actually haven't tested it.
Now, we could.
We could just call more points.
We could calculate.
But that would cost us more function evals.
And we're trying to save our computer time.
So we want to finish off before the TG so we can go buy some beer.
And so we have to take some choices about what to do.
And we can't do an infinite number of points for sure.
All right, so in all these methods, what we're doing is we're approximating our true f with some approximate f, which is equal to a sum of some basis functions like this.
So we pick some basis functions.
We know how to integrate.
We fit, somehow determine what these c's are.
And then we decide that the integral is equal to the integral of f tilde, not the actual original f. And then that's actually going to be the sum of ck times the integral.
And we carefully chose basis functions that we don't how to do this integral; analytically.
So we just plug that in, and we just have to figure out that the c's are to get this to work.
OK, so that's what you really did when you're doing this trapezoid rule and Simpson's rule, and all that kind of stuff.
And it's just different choices of what the basis functions were that you used.
Now, there's a lot of choices about how to do that kind of approximation to make f tildes that are sort of like f's with some limited amount of information.
And it generally has to do with how many k's we have.
So if k is less than n, then it's really an approximation.
Because there's no way that we would expect that with just a few parameters, we could get a lot f of tn.
So we would expect that f tilde of tn is not equal to f of tn, at least not at all the points.
Because if n is-- if we have a lot of points and we only have a few parameters, we don't expect to be able to fit it perfectly.
So we know this is what we expect.
It might happen if, by some luck, we chose a basis function which was actually the true function f, then it would actually fit perfectly.
But most of the time, not.
By Murphy's Law, in fact, it's going to be completely wrong.
But this is not such a bad idea.
We may come back to this later.
Another possibility is to choose k equals to n. And then usually, we can force f tilde of tn to equal f of tn.
So at the points that we have, if we have enough parameters equal to the number of points, then often, we can get an exact fit.
We can force the function to go through all the points.
And when we do this, this is called interpolation.
Now, when should you do the one or the other?
When should I make k as big as n?
Or when should I use k less than n?
A really crucial thing is whether you really believe your points.
If you really believe you know f of tn to a very, very high number of significant figures, then it make sense.
You know that's true, so you should force your fitting function to actually match that.
However, if you get the f of tn, for example, from an experiment or from a stochastic simulation that has some noise in it, then you might not really want to do this.
Because you'll be fitting the noise in the experiment as well.
And then you might want to use a smaller number of parameters to make the fits.
OK, so you can do either one.
You're in charge.
Right, you get the problem.
If somebody tells you, please give me this integral, they don't care how you do it.
They just want you to come back and give them the true value of the integral to some significant figures.
So it's up to you to decide what to do.
So you can decide what to do.
What a lot of people do in the math world is this one, interpolation, where you're assuming that the f of tn's are known exactly.
And so your function evaluating function is great.
And in that case, you can write down this nice linear equation.
So we're trying to make f of tn with a true function to be equal to ck phi n of t. Sorry, phi k of t. Which n in [INAUDIBLE] k is equal to n. And I can rewrite this this way.
This is like a set of numbers.
So I can write as a vector.
I'll call it y.
And this thing has two indices on it, so it looks like a matrix.
So I'll call that m. And c is my vector of unknown parameters.
And wow, I know how to solve this one.
So c is equal to m backslash y.
Now, for this to work and have a unique solution, I need m not to be singular.
And what that means is that the columns of m have to be linearly independent.
So what are the columns of m?
They are phi k of t1, phi k of t2, phi k of tn.
And I want that this column is not a sum of the other columns.
So this is kind of requirement for j not equal [?
to k ?]
This is kind of a requirement if I'm going to do this kind of procedure is I can't choose a set of basis functions that's going to have one of them be a linear combination of the other ones.
Because then I'm going to get a singular matrix.
My whole process is going to have a big problem right from the beginning.
So I have to ensure that this is not true.
Yeah, that this is true.
These are not equal to some of them.
Another way to write that is to choose the the columns that phi to be orthogonal to each other.
And so I can write it this way.
OK, so I want them to be orthogonal to each other.
And then you can see how this kind of generalizes.
If I had a lot of t's, this would sort of look like the integral of phi k of t, phi j of t is equal to 0.
And so this gives the idea of orthogonal functions.
So this is orthogonal functions, and my discrete point's tn.
And this is the general concept of orthogonal functions.
And then when I define it this way, I need to actually decide my own a's and b's that I want to use.
And if you use different ones, you actually have different rules about what's orthogonal to each other.
And also, sometimes, people are fishy, and they [?
spit ?]
a little w of t in here just to do, because they have some special problem where this w keeps showing up or something, and they want to do that.
But that's up to them.
All right, we won't specify that.
But this is the general idea of making basis functions orthogonal to each other.
And it's very analogous to making vectors orthogonal to each other, which is what we did here.
These are actually vectors.
Right, it's just as the vector gets longer and longer, it gets more and more like the continuous function, and we have more t points.
And eventually, it looks like that integral as you go to infinity.
So anyway, so a lot of times, people will choose functions which are designed to be orthogonal.
And one way to do it is just to go with a continuous one to start.
Now, we can choose any basis functions we want.
Over here, we already chose a whole bunch.
You guys have done this already.
I don't know if you knew you were doing it.
But you chose a lot of different basis functions completely different from each other.
You can choose the different ones if you want.
And one popular choice is polynomials.
But it's not the only choice.
And in fact, a little bit later, we're going to, in the homework problem six, I think we're going to have a problem where you have to use a different kind of basis functions.
You can use any basis functions you want.
And there's kind of ones that match naturally to the problem.
However, people know a lot about polynomials.
And so they've decided to use polynomials a lot.
And so that's the most common choice for numerical integration is polynomials.
One thing is polynomials are super easy to integrate, because that was the first thing you learned, right, how to do the intervals of polynomials.
And also, they're easy to evaluate.
It only takes n operations to evaluate a polynomial of nth order.
And there's a ton of theorems, and things are known about polynomials.
We have all kinds of special properties.
And we have all kinds of convenient tools for handling them.
So let's go with polynomials for now.
Now, I want to warn you, though, that how you order polynomials are really not very good fitting functions or interpolating functions for a lot of problems.
And in particular, for this one I showed here where this is the function.
Here, that function, you can try to fit that with higher and higher order polynomials.
So I can have my function.
The function value at 0 is 1.
So the function looks kind of reasonable.
Like that, symmetrical, better than I drew it.
And you can put some number of points in here.
And then you can do an interpolating polynomial that goes through all the points.
And if you do it for this one with five points, you'll get something that looks like this.
It goes negative, comes up like this, goes negative again.
So that's one you get if you use five points across this interval.
If you use 10 points, then you get one that looks like this where the peak value here is two where as the max of the original function's just one.
And the original function's always positive, but they both go negative.
So even within the range of the interpolation, the polynomials can have wild swings.
And so this first one was for a fifth order polynomial, or maybe fourth order, five points.
And this one is for the next 10th order or 9th order polynomial.
I'm not sure which one.
Yes
What points are these polynomials trying to fit?
Or does it seem like [INAUDIBLE]??
So I didn't draw very well.
When you guys go home, do this.
Right, on Matlab it will take you a minute to solve this equation where you put in the function values here.
And put in the polynomial values that [INAUDIBLE] here for the monomials, t to the first power, to the second power, to the third power, like that.
And just do it, and you can get the [?
pot ?]
yourself.
And just to convince yourself about the problem, and this is very important to keep in mind.
Because we're so used to using polynomials that think they're the solution to everything.
And Excel does them great.
But it's not this solution to everything.
Nonetheless, we're going to plow on and keep using them for a few minutes.
So one possibility when I'm choosing this basis set is I could choose what are called monomials, which is phi k is equal to x to the k minus 1.
And this one is super simple.
And so a lot of times it satisfies this orthogonality thing, which I showed you before.
But it's a really bad choice usually.
And what it is is that the matrix you have to solve for this interpolation, if you have evenly-spaced points and you have x to the k, it turns out that this matrix usually has a condition number that grows exponentially with the number of terms in your basis, the number of how many monomials you include.
So condition numbers that grow exponentially is really bad idea.
So this is not what people use.
So you might be tempted to do this, and maybe you did it once in your life.
I won't blame you if you did.
But it's not a really smart idea.
So instead, a lot of the mathematicians of the 1700s and 1800s spent a lot of time worrying about this.
And they'd end up deciding to use other polynomials that have people's names on them.
So a guy named Lagrange, the same guy who did the multiplier.
He suggested some polynomials.
They're kind of complicated.
They're given in the book.
But what they really are is I'm taking the monomial basis set, and I'm making a new basis set.
So I want phi l to be equal to the sum dlk of phi k, where this phi k it the monomials.
So really, you can choose any d you want.
You're just taking linear combinations of the original monomial basis set to make up polynomials.
And if you cleverly choose the d, you can choose it to have special properties.
So Lagrange gave a certain choice of the d that is brilliant in the sense that, when you solve the problem m times c is equal to y to try to do your interpolation, it turns out that the c's you want, ck, is equal to f of tk.
So you don't have to do any work to solve what the c's are.
It's just the f values directly.
No chance for condition numbers, no problems, that's it.
So that's the Lagrange polynomials.
Now, Newton had a different idea.
So this is for Lagrange polynomials.
If you choose a Lagrangian [?
polynomial ?]
basis, then this is it.
And you do a square system.
That's it.
If you choose the Newton, even he got into this business too.
And he made some polynomials.
And he made ones that have a special property that if you solve it-- so he chose a different d. You still have to do some solve.
You get the solution.
And then you decide that I don't like that interpretation.
I want to put some more points in.
I'm going to do a few more function evaluations.
I'm going to add some more.
So from the first solve, I have the values of c1, c2, c3, up to c the number of points I had initially.
And now, I want to add one more data point, or add one more f of tn, one more function evaluation.
I want to figure out with c of n plus 1 is, what the next one is.
It turns out that it's his polynomials.
The values these c's don't change when I make this matrix a little bit bigger by adding one more y.
And I add one more parameter.
And I make one more row in the matrix, one more column in the matrix.
When I do that, it's a unique property that none of these other c's change.
The solution is going to be the same, except for the n plus one [?
level.
?]
So then I can write a neat, cute little equation just for the n plus 1 one.
I don't have to mess with the previous ones.
And so I'm just adding a basis function that sort of polishes up my original fit.
So that's a nice property too.
Whereas with the Lagrange ones, you have to recompute everything.
And all the c's will change.
Actually, the c's won't change.
I take that back.
You add points.
The c's won't change, but the actual polynomials will change.
The d will change.
Because the d depends on the choice of the time points in a complicated way in Lagrange.
So now, you start to think, oh my goodness, I have all these choices, I can use Lagrange.
I could use Newton.
I could use monomials.
Maybe, I should go back to bed.
And you think, well, could someone just tell me what the best one is?
And I'll just do that one.
I don't need to learn about all this historical stuff about development of polynomial mathematics.
I'd just like you to just tell me what the answer is.
So Gauss, typically, figured out what the best on is.
And he said, you know, if we're going to do integrals, we'd really like to just write our integrals this way.
So our approximate value to be equal to some weight functions times the function evaluations at some times.
Right, that's easy to evaluate.
That would be good.
If I pre-computed the weight functions [?
and ?]
the times, then I put any function f in here, I could just do this like zip.
It's one dot product of the f of tn's I'll have the solution.
So I want to do that.
So what's the best possible choices of the w's and the t's to make this really be close?
Well, the problem is that there's an infinite number of functions that somebody might want to integrate.
I can't do them all.
So he said, let's just find the one that works for as many monomials as possible to the highest order, as far out as I can go.
And it turns out I have nw's.
I have nt's.
So I have two n parameters that I can fiddle around with.
And so I should get something like two n's in my polynomials, maybe 2n minus 1.
So what I want is that I want to choose this so that for the monomial 1, and the monomial t, and the monomial t squared, that this i is actually going to be exact.
So what I want is for the monomial 1, I want the summation of wn.
My f for the monomial 1 is always 1.
So it's times 1.
I want this to equal the integral from a to b of 1dt, which is b minus a. OK, so that gives me one equation for the w's.
Then for the next one, I want the summation wn times tn to equal the integral of a to b of t. dt, which is equal to b squared over a number 2 minus a squared over 2 if I did my calculus correctly.
And so there's another question.
I have another question for these guys.
And I keep going until I get enough equations that determine all the w's and c's.
All right, so I do it for t squared.
So I can wn tn squared is equal to [INAUDIBLE] b t squared dt, and so on, OK. And so if you do this process, you get the weights and the points to use that will give you what's called the Gaussian quadrature rule.
Now, the brilliant thing about this is that by this setup, it's going to be exact for integrating any polynomial up to order 2n minus 1.
So if I choose three, here I'll be able to determine-- if I choose three points, I'll have six parameters.
So I'll be able determine up to the 11-- 11th order polynomial would integrate exactly.
No, that's wrong.
n's only 3.
I have six parameters.
I can't do this.
2 times 3 minus 1, what's that?
5, there we go.
To the fifth order polynomial integrate exactly, I'll have an [?
error ?]
for sixth order polynomial.
So this is a rule.
So before, if I gave you three points like here, most of you probably would use Simpson's rule.
Fit this to a parabola.
It has three parameters, a, ax squared plus bx plus c, right, for a second order polynomial.
I would fit it.
Then I'd integrate that.
That would be the normal thing that a lot of people would do.
I don't know if you would do that.
But a lot of people would do that.
So that would be that what you would get.
And what [?
error ?]
would you get?
You'd have error that would be the scale of the width of the intervals to the fourth power.
Right, because Simpson's rule exact up to the cubic.
You guys remember this from high school?
I don't know if you remember delta t to the fourth.
Yeah, OK.
So Simpson's rule has an order of delta t to the fourth.
But Gauss's method has order of delta t to the sixth.
So this is for three points.
In Gauss's method, you can extend it to as many points as you want.
All right, so that's kind of nice.
You only do three points, you get [?
error ?]
of the sixth order.
So that seems pretty efficient.
So people kept going.
And so the very, very popular one use, actually, seven points for Gauss.
So you'd be able to get to the n minus 1.
So you'd be exact for 13.
So if you have delta t to the 14th power as your error, that's pretty good.
And then what people often do is they'll add eight more points.
And they make another method that uses all these points.
It's called [?
Conrad's ?]
method.
He made another method that gives you the best possible solution if you had the 15 points, given that 7 of them are already fixed by the Gauss procedure.
And so you can get the integral using Gauss's method and using [?
Conrad's ?]
method.
So you can compute the i using Gauss's method, the seven points, the i with the [?
Conrad ?]
method.
I don't know how to spell his name, Actually I think it's like this.
15 points.
And then you can take the difference between these two.
And if these two are really similar, then you might think your integral's pretty good.
Because they're pretty different methods, and if they get the same number, you're good.
And people have done a lot of numerical studies in this particular case, because it's used a lot in actual practice.
And it turns out that the [?
error ?]
is almost always less than this quantity to the 3 Gauss power.
So you have an error estimate.
So you do 15 function evaluations.
You get a pretty accurate value for your integral, maybe, if you can be fit well with whatever order polynomial, 13th order polynomial.
13th order polynomial, yeah.
And so any function that can be fit pretty well with the 13th order polynomial, you should get pretty good value in the integral.
And you can check by comparing to the [?
Conrad ?]
estimate, which uses a lot more points.
And if two of these guys give the same number or pretty close, then you're pretty confident that you're good.
And there's even a math proof about this particular one.
So this is like really popular, and it's a lot better than Simpson's rule, but it's kind of complicated.
And part of it you have to recompute these tn's and wn's for the kind of problem you have.
Yeah
If you're using 15 points already, why not just split your step size smaller?
And if you did that, how would the error [INAUDIBLE] the simplest?
Use a trapezoid
Trapezoid or Simpson's rule of something like that.
Yes
How would the error with those 15 steps using trapezoid compare to [?
Conrad ?]
and Gauss?
So we should probably do the math.
So say we did Simpson's rule, and we have delta t, and we divide it by 15 or something.
Then we can figure out what this is, but it's still going to have the scaling to the fourth power.
So the prefactor will be really tiny, because it will be 1/15 to the fourth power.
But the scaling is bad.
So that if I decide that my integral's not good enough, and I need to do more points as I double the number of points to get better accuracy, my thing won't improve that much.
Because it's still only going to use the fourth power, whereas the Gauss' one say it's going use the sixth power if I only use the three points.
Where we use 15 points, is going to get scale to some 16th power or 15th power or some really high power.
So for doing sequences, the Gauss' one is a lot better.
However, what people do is they usually pre-solve the system of equations.
Because this system of equation does not depend.
When you do it with the monomials, it doesn't depend what f is.
You can figure out the optimal t's and w's ahead of time.
And so they [INAUDIBLE] tables of them depending on where you want to go.
People recomputed them, and then you can get them that way.
Time is running out.
I just want to jump up.
Now, for single integrals, I didn't have to tell you and of this.
Because you could have just solved it with your ODE solver.
So you can just go to ode15s or ode45, and you'll get good enough.
It won't be as efficient as Gaussian.
But who cares.
You still get a good answer.
You can report to your boss.
It'll take you three minutes to call it e45 to integrate the one-dimensional integral.
The real challenge comes with the multidimensional integrals.
So in a lot of problems, we have more than one variable we want to integrate over.
And so if you have an integral, say, of a to b from the lower bound of x to upper bound of x.
So this is dx, dy, f of xy.
This is the kind of integral that you might need to do sometimes.
OK, because you might have two variables that you want to integrate over.
And so how we do this, the best way to think about it, I think, is that f of x is equal to the integral of l of x u of x dy f of xy.
And then, if I knew what that was, I would evaluate that say with a Gaussian quadrature or something.
So I'd say that this i 2d is approximately equal to the sum of some weights-- I'll label them x just to remind you where they came from-- times f of xn.
And I would use, say, Gaussian quadrature to figure out what the best value of the x's and the y's are to use.
And then I would actually evaluate this guy with a quadrature two.
So I'd say f of xk or xn is equal to the integral from l of xn to u of xn dy f of xn y.
And so this itself, I would give it another quadrature.
This is going to be equal some other weights.
nj f of xn yj.
Is that all right?
And so this is nested inside this.
So now, I have a double loop.
So if I took me, I don't know, 15 points, 15 function evaluations to get a good number for my integral in one dimension, then it might take me 15 squared function evaluations to get a pretty good integral for two.
15 might be a little optimistic, so maybe 100 is more like if you really want a good number.
So I need 101 dimension, 100 in the second dimension squared.
Because I subdivided the integral into six equal pieces, say.
And then now, so this is OK.
So you can do this.
It's only 10,000 function evaluations.
No problem, you've got a good gigahertz computer.
Now, in reality, in a lot of them, we do have a lot more dimensions than that.
So in my life, I do a lot of electronic structure integrals.
We have integrals that are dx 1d y1 d z1 for the first electron interacting with a second.
And then something that I'm trying to integrate.
It depends on, say, the distance between electron one and electron two.
OK, so this is actually six integrals.
And so instead of being a 2 in the exponent, this is going to be like a 6 in the exponent.
So now, this is 10 to the 12th.
That means that just evaluating one integral this way might take me 100 seconds if I can evaluate the functions like that.
But in fact, the functions I want to evaluate here, these guys will take multiple operations to evaluate the function.
And so this turns out I can't do.
I can't even do one integral this way with its six integrals deep.
And so this is the general problem of multidimensional integration.
It's called the curse of dimensionality.
Each time you add one more integral, you're nested there.
All of a sudden, the effort goes up tremendously and more, a couple of orders of magnitude more.
And so we'll come back later in the class about how to deal with cases like this when you have high dimensionality cases to try to get better scaling.
Instead of having it being exponential, it was really 100 to the n right now, 100 to the n dimensions.
[?
Can we ?]
figure out some better way to do it.
All right, thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
WILLIAM GREEN,
Shall we get started?
Yes.
Apologies for the notes being posted so late.
I had trouble with the new two factor authentication system, which I need to get into Stellar and put things up there.
My phone wasn't working so I had to find my iPad someplace and use that to get credentials to log in.
So if you don't have them don't worry about it.
The full notes are posted online, no missing pieces.
So you can utilize those afterwards if something is unclear.
All right?
So your quizzes are graded.
The TAs have them.
I think overall, we were really pleased with the outcome of this quiz.
It is very conceptual in nature and we thought people really understood a lot of the material quite well.
And you can see that from the distribution of scores.
So the mean on the quiz was about 77, a standard deviation of 12.
I think everyone did really quite well.
And on a number of the problems, perfect solutions were posted.
So I'll take a second here.
If there are any questions about the material that was on the quiz, or the format of it that you'd like to have addressed, I'm happy to answer them.
No?
Does it seem like it took too much time to complete all three questions in the two hours allotted?
Or was it about the right length of material for the time slot?
So we vetted it with the two TAs.
And it took each of them less than an hour to complete it.
So that was a good sign that we thought you guys should be able to get through most of the material in two hours.
So let's recap.
We're going to move on today to a topic called differential algebraic equations.
But before doing that, there are a couple of concepts I want to review from numerical integration, or actually concepts that weren't covered when we talked about numerical integration on Friday that I think are important.
And then I want to review briefly implicit methods for ODE-IVPs because those are going to be important for differential algebraic equations.
So for numerical integration we talked about quadrature of various integrals, how to develop quadrature formulas.
But there's actually one type of integral we didn't talk about there, which is sort of improper integrals, where the limits of integration are unbounded.
And these come up all the time in engineering problems.
We'd like to know what the, say, integrated response of some function is over all possible times.
And we may have to evaluate that numerically.
So how do you do those sorts of integrals?
And it turns out the strategy that's most often used is to divide this domain up into two pieces.
One piece, which is proper integral over a finite domain, and another piece, which is an improper integral over an infinite domain.
This first integral, you can handle it with an ODE-IVP method or some higher order polynomial interpolation.
But the second one we have to do differently.
And there are couple of ways to do this.
One is to transform variables.
So you map this domain onto a finite domain.
And the other option is to substitute an asymptotic approximation.
So we say, when, say, tau is large, f of tau has a characteristic asymptotic approximation.
Maybe it's exponentially decaying.
And we take the integral of that asymptotic approximation.
And the more accurate that approximation is the better our approximation for this whole integral will be.
There's another type of interval that we have to do, where the same idea can be applied.
That's the integral over integrable singularities.
So here's an example.
So say we want to integrate the function cosine of tau over square root of tau from 0 to some finite positive time point.
As tau goes to 0, cosine goes to 1 and square root of tau diverges.
But this integral actually has a finite value.
This 1 over square root of tau, that's an integrable singularity.
So how do you do this?
Well, you want to split the domain again into two parts.
One part that contains the integrable singularity and the other part, which excludes it.
And in the part that contains the singularity we might do an asymptotic expansion of the function, and integrate each of those asymptotically accurate terms separately.
So the integral of 1 over square root of tau is going to give us 2 tau 0 to the 1/2.
The integral of this ratio here is going to give us minus 1/2 t0 to the 5/2.
And then we have an integral over a finite domain, which doesn't include the singularity.
And the accuracy of this method can be dictated by two things now.
One is how accurately can we do this integral.
And the other is how accurate is our asymptotic expansion here.
So if we want higher accuracy we need more terms.
We might need to be selective about how we choose the end point for this domain in order to minimize the error.
There are lots of methods to do this.
Sam
Will you talk a little more about what typical integral [INAUDIBLE] is?
WILLIAM GREEN,
Yeah.
So you know these things.
If I try to integrate 1 over x, as x goes to 0 this thing will always diverge.
You know the antiderivative of 1 over x is going to be the log.
And the log diverges.
So power is weaker than 1 over x, like 1 over x to the 1/2.
Don't diverge like the log.
They actually go to a finite value.
The log is like the most weakly singular function there is.
And integrals over weaker powers of 1 over x actually don't produce a singularity anymore.
They're what we call integrable singularities.
The function itself diverges but its integral does not.
So you usually have this asymptotic power law type behavior.
OK.
So they come up all the time.
We see these things in places like, say, squeezing a thin film of fluid between two plates.
How long does it take for these two plates to come together?
You might say, well, as the plates get closer and closer together the pressure starts to diverge in the gap and they'll never come together.
But maybe depending on the geometry of the plates there can be a sort of finite time at which they'll come together.
Depends on their geometry.
So that's one topic.
I think that's useful because these things come up all the time.
And you can't actually handle those cases with quadrature by itself.
The function diverges.
There's no polynomial interpolation that's suitable to match it.
So the quadrature can't handle it.
And if you're trying to integrate over an infinite domain, well, good luck.
You're never going to be able to fit enough points in to know that you've accurately integrated out to infinity.
The other thing I want to recap here, and this is a problem for you to try.
So here we have a simple first order ordinary differential equation.
We want to use the implicit Euler method to solve it.
So can you give a closed form solution for one step of the implicit Euler method, or for n steps of the implicit Euler method?
What result does that produce?
Can you compute this?
Do you guys talk about backward Euler methods for for ODE-IVPs?
Backwards difference methods in general and the backwards Euler method specifically?
No.
OK. Well, clearly I should have been here.
So what's the strategy?
We have to approximate the derivative.
Yes?
So the approximation for the derivative we use is a finite difference approximation.
So we write dx dt as x at a point k plus 1 minus x at the point k divided by delta t. Backwards difference.
We evaluate the right-hand side of the ODE-IVP at k plus 1, xk plus 1.
And then we solve for xk plus 1.
We just substitute in here.
Backwards difference approximation for the differential.
And xk plus 1 for the right-hand side.
And we find that xk plus 1 will be 1 over 1 minus delta t times this lambda times xk.
And if we iterate this from k equals 0 up to some finite k, it's like multiplying by powers of 1 over 1 minus delta t times lambda.
Yes?
Does this makes sense?
So our backwards Euler solution to this fundamental problem is going to look like this.
And a lot of times we ask about the stability of these sorts of solutions.
So if for example, the solution to this equation was supposed to decay to zero, we would expect our numerical method to also yield results which decay to zero.
We would call that stable.
So we all know under what conditions does xk, for example, go to 0 as I change the product delta t times lambda.
Lambda could be a real number or it can be a complex number too.
Doesn't really matter.
So what needs to be true for xk to the k to 0 as k gets bigger?
Well what needs to be true is this quantity in parentheses needs to be smaller than 1 because I'm taking lots of products of things that are smaller than one.
That will continue to shrink xk from x 0 until it goes to zero.
If this is smaller than 1, then this quantity down here needs to be bigger than 1 in absolute value.
And then I'll get solutions that slowly decay to zero.
So if I ask, for what values of delta t times lambda is the absolute value of this thing bigger than 1, and I allow lambda to be a complex number, for example?
I'll find out that's true when 1 minus delta t times the real part of lambda squared plus delta t times the imaginary part of lambda squared is bigger than 1.
And if I plot the real part of delta t times lambda versus the imaginary part of delta times lambda, this inequality is represented by this pink zone on the outside of the circle.
So any values of delta t times lambda that live in this pink area will give stable integration.
And the solution xk, well, the k to zero as k goes to infinity.
So it makes sense?
This is a little funny though.
What does the solution to this equation do when lambda is bigger than 1?
When it's bigger than 0, excuse me.
When lambda is bigger than 0, what does the solution to this ODE-IVP do?
Does it decay?
[INAUDIBLE] WILLIAM GREEN,
It blows up.
It grows exponentially.
But the numerical method here, when I have lambda is bigger than zero, can be stable and produce decaying solutions.
So this is sort of peculiar.
So we're often concerned when we solve ODE-IVPs and we try to use these sorts of backwards difference methods, these implicit methods, to solve them.
We're often concerned with getting stability.
We want to get solutions that decay when they're supposed to decay.
Sometimes we get solutions that decay even when the solution is supposed to blow up.
So that's sort of funny.
So the exact solution should look like this.
But there are going to be circumstances where lambda is bigger than zero.
In fact, lambda is bigger than-- lambda delta t is bigger than 2 along the real axis, where the solution will actually blow up rather than decay away.
So stability and accuracy don't always correlate with each other.
We really have to understand the underlying dynamics of the problem we're trying to solve before we choose a numerical method to apply to it.
And this is going to be true of differential algebraic equations as well.
Are there any questions about this?
There's a discussion of these things in your textbook, these sorts of stability regions associated with different backwards difference methods.
The implicit Euler method is one backwards difference formula.
It's the simplest one.
It's the least accurate one that you can derive but it's an easy one to use.
But now we want to move on to a different sort of problem.
And it's one that we come across in engineering all the time.
These problems are called differential algebraic equations.
And they're problems of the sort a function of some state variables, and the derivatives of those state variables and time is equal to zero.
And there are some initial conditions.
The state variables at that time 0 is equal to some x0.
So this is a vector valued function of a vector valued state and it's time derivatives.
And we want to understand the dynamics of this system.
How does x vary with time in a way that's consistent with this governing equation?
Usually we call these well-posed problems when the dimension of the state variable and the function are the same.
They both live in the vector space rn.
So we have n different states.
I have n different functions that I have to satisfy.
And I want to understand how x varies with time.
Here's an example.
A stirred tank.
I call this stirred tank, example one.
So into the stirred tank comes some concentration of solute.
Out of the stirred tank comes some different concentration of the same solute.
And we want to model the dynamics as a function of time.
So we know that a material balance of the solute on the stirred tank, if it's well mixed, will tell us that d c2 dt is related to the volumetric flow rate into the tank divided by the volume of the tank multiplied by the difference between c1 and c2, the amount carried in and the amount carried out of the tank.
And let's put a constraint in here too.
Let's suppose we're trying to control this tank in some way.
So we say that c1 of time has got to be equal to some control signal, gamma of t. And then we want to understand the dynamics of this system.
At time 0, c2 is some c0, the initial concentration of the tank.
And c1 is whatever the initial value of this control signal is.
And the solution is c1 equals gamma of t. And c2 is, well, the initial concentration's got to decay out exponentially.
And then I've got to integrate over how the input is changing in time.
Those also produce exponential decay.
So if the input is a little delta function spike then I'll get an exponential decay leaving the tank.
If the input is changing dynamically in time, I need this convolution of the input with this integral.
So you just to solve this system, this ordinary differential equation by substituting c1 in there.
But this system of equations here is not just a differential equation.
It's a differential equation and an algebraic equation.
There's something peculiar about that that's different than just solving systems of ordinary differential equations.
Oftentimes in models, we write down all the governing equations associated with the model.
Some of them may be differential, some of them may be algebraic.
There are times when we can look at those equations and see clever substitutions that we can make.
But if you have a hundred equations, or a thousand equations, or a million equations, there's no way to do that reliably.
We just kind of wind up with a system of ordinary differential equations and algebraic equations mixed together.
And we have to solve these reliably.
Here's another example.
Oh, excuse me.
So let me write this in this form.
So we said there should be a vector valued function of the states and the derivatives of the states.
The states are the concentration c1 and c2.
And the derivative here is dc 2 dt.
So here's my vector valued function.
The first element is this, and it's equal to 0.
The second element is this, and it's equal to 0.
Here's my initial condition over the states.
So I can just transform that list of equations to the sort of canonical form for a differential algebraic equation.
Here's a separate example.
Suppose instead, I'm trying to either control or make a measurement of the outlet concentrations c2.
And we call that gamma of t. And I want to know now what is c2 and what is c1?
So I've got to solve this system of equations for c2 and c1.
The solution for c2 is easy.
I know that.
c2 is gamma, by definition.
I got to substitute this into the first equation and solve for c1.
And I see, c1 is gamma plus v over q gamma dot.
And I have to have initial conditions to go along with this.
There's something funny here.
We can see the initial condition for c2 has to be gamma 0.
The initial condition for c1, what does that have to be?
Well, it needs to be consistent with the solution here.
There were no free variables when I solved this equation for c1.
It isn't like solving a differential equation, having a free coefficient to specify.
So it better be the case that this c0 here is the same as gamma 0 plus v over q gamma dot 0.
Somehow I have to know this input signal to prescribe the initial conditions for c1.
So that's peculiar and different from just ODE-IVPs.
You see this picture?
You see how this goes together?
The solution is funny too.
Suppose we are trying to use this system of equations to do the following.
We are trying to measure c2 and use the measurement of c2 to predict c1.
So gamma is the measurement, and we're trying to solve this system of differential algebraic equations to predict what c1 is.
All measurements incur numerical error.
So even though our signal for gamma that we measure may be continuous, it's bouncing around wildly.
It's not a constant signal.
It's going to move around a great deal.
And c1, our predicted value for c1, depends on the derivatives of gamma, which means c1 is going to be incredibly sensitive to how that measurement is made.
So it's peculiar.
It means that there's going to be a lot of problems with stably solving these equations because the solutions can admit cases where they're not integrals of the input but they're actually related to derivatives of the input.
Look at the solution back here again.
Oop, sorry.
The solution here was related to an integral of the input.
Suppose I was making a measurement of c1 in trying to predict c2 instead?
My prediction for c2 would be related to the integral of that measured signal.
And we know integrals smooth signals out.
So it's not going to be so sensitive to the measurement.
So one way of doing this seems really sensible.
And the other way of doing it seems a little bizarre.
Well, we can construct these equations however we want.
So there's something about how we formulate models that are going to make them either easily solvable, as DAEs, or quite difficult to solve.
So these pop up all the time in engineering problems.
They pop up in dynamic simulations, where we have some sort of underlying conservation principle, or constraints, or equilibria.
So if we have to conserve something, like total energy, total mass, total momentum, total particle number, total atomic species, total charge, there will always be an algebraic constraint that tells us the total mass has to stay this, or the total number of atoms of a certain type has to say this.
Yeah, Ian
Just to be clear, were you saying, on the stirred tank example, that those were the same problem with different solution approaches?
WILLIAM GREEN,
So good question.
Let me go back.
So they are fundamentally different problems.
So in one case I specified the value of c2 to be this gamma.
I tried to make a measurement of c2 and then predict c1.
In the other case, I specified c1.
I tried to measure c1 and predict c2.
So one case I measured the input and tried to predict the output to the tank.
And in the other case I measured the output and tried to use it to predict the input.
Yeah?
So physically the same system with a choice by the operator.
WILLIAM GREEN,
Yes.
That's right.
That's right.
So I formulated the model differently.
The same underlying system but I formulated the model differently.
And it led to completely different behavior in the underlying solution to the differential algebraic equations.
So you've already solved problems that involve conservation.
You've done things like ODE-IVPs on batch reactors, where the total amount of material in the reactor had to remain constant.
Instead, you probably solved for the dynamics of each of the components undergoing the reaction.
You might have checked to see if at each point in time the total mass in the reactor stayed the same.
Because you solved all these differential equations, they were interconnected with each other but they all incurred some numerical error.
It's possible that numerical error accrues in a way that actually has you losing mass from your system.
So there may be benefits to actually trying to solve these sorts of systems of equations with, say, one less differential equation for one of the components and instead an algebraic constraint governing the total mass or total number of moles in the system.
So these pop up all the time.
You see them in models of reaction networks, where we try to use a pseudo steady state approximation.
We say some reactions go so fast that they equilibrate.
So they're not governed by differential equations but by algebraic equations for equilibria.
They can pop up in models of control, where we neglect the controller dynamics.
So you say I'm going to try to control this process and I get to instantaneously turn on or off this valve in response to a measurement.
Actually, controllers have inherent dynamics in them.
But if I don't put those dynamics in then I may get an algebraic equation instead of a differential equation for how the control process occurs.
There are different types of DAEs that we talk about.
So there's one type that's called semi-explicit DAEs.
And in these types of differential algebraic equations we can write them in the form M dx dt is equal to f of x and t, some initial condition.
You say that this looks like an ODE-IVP.
M is called the mass matrix.
And it may or may not be the case that M is invertible.
M may or may not be a full rank matrix.
So let's look at that stirred tank example one.
This was the underlying equation, f of x and dx dt and t. Can write it in semi-explicit form.
If c has two components, c1 and c2, then dc 2 dt-- oops, this is a typo here.
I really apologize.
This should be a 0, and this element should be a 1.
I'll correct that online.
I apologize for that.
So this should be this matrix multiplied with this differential should give me dc 2 dt, this term here.
And a 0 for the second equation.
And then I've got q over v multiplying c1 and minus c2.
So that gives me the first line here.
I've got minus c1 coming on the second equation here.
And those balance with a 0 and a gamma.
So the lower line reads like this equation here and the upper line-- fix this typo-- replace 0 with one, reads like the upper equation here.
This is the semi-explicit form of the differential equation.
You can see this mass matrix is singular and has a row that's all zeros.
There's no way to invert this matrix and convert it into a system of ODE-IVPs, which you already know how to solve.
If the mass matrix is full rank, which can happen, you can formulate your model in such a way that it takes on this semi-explicit form.
But if the mass matrix is full rank and invertible, then you just invert the mass matrix.
And now you can apply all your ODE-IVP techniques to solve this thing.
The mass matrix doesn't need to be constant.
Could depend on the concentration.
It can depend on the states as well in this form.
I just chose an example where that isn't the case.
But in general, it can depend on the states.
If it depends on the states and we invert it, then we need to be sure that the mass matrix is invertible for all values of the states, which may or may not be easy to ensure.
So here's what I'd like you to try to do.
So here's that stirred tank example two.
The only difference between this and the previous one is that c2 now is set equal to gamma instead of c1.
I want you to try to write it in semi-explicit form.
A sort of test of understanding.
Can you define the mass matrix?
Can you write out the right-hand side of the equation in semi-explicit form?
Take a second, work with your neighbor.
See if you can figure out how to do that
Is this clear?
OK to do this?
WILLIAM GREEN,
You can look back on the previous slide.
Actually, the semi-exclusive form is going to be the same.
This is a 0, that's a 1.
The only difference is going to be c2 is now balanced with gamma, the input.
So this minus 1 is got to shift over.
So I switched the 0 for the minus 1, you got the second, the semi-explicit form.
There's another way that these equations can pop up.
So we have this mass matrix.
And it might be the case that M is diagonal over many of the states and then 0 for all the other states.
So we might be able to write the equation in this form.
This comes up all the time.
So we might be able to write dx dt is a function of x, y, and t. And 0 is equal to another function of x, y, and t. And we have some initial conditions prescribed for the x's, of which we're taking the differential in one of these equations.
And we've also got to satisfy those initial conditions for these variables y associated with the second nonlinear equation.
The x's are called the differential states because they're involved in equations where they're difference with respect to time, or the differential with respect to time is taken.
y are called the algebraic states because they only appear as themselves and not as their time differential in any of these equations.
So we might want to solve for both the differential and the algebraic states simultaneously.
We have to know these algebraic states to determine the differential states.
We have to know the differential states to determine the algebraic states
Professor?
WILLIAM GREEN,
Yes
[INAUDIBLE] WILLIAM GREEN,
That's right.
So here the x that I have is not all of the states.
It's not all of the unknowns.
It's only the set of the unknowns of which I have to take a differential.
That's right, that's right.
So we've sort of tooled down the kinds of equations we might want to look at.
We've gone from the generic differential algebraic equation to a semi-explicit form to the splitting, this differential algebraic splitting.
Many problems can be formulated in this fashion.
This one's the easiest one to think about so that's the one that we'll discuss for most of the lecture.
So let's look at some examples.
I think examples are interesting to think about.
So here's one.
Think about a mass spring system.
There's an algebraic equation that governs a conserved quantity, the total energy of this mass spring system.
So the kinetic energy, 1/2 m times the velocity of the mass squared.
Plus the energy stored in the spring, 1/2 kx squared has got to be equal to the total energy.
And the total energy can change in time.
So we have a differential algebraic equation.
We have a differential state because we're taking the time derivative of x.
And we'd like to be able to determine x as a function of time.
So f of x, x dot and t. That's this equation, 1/2 mv squared plus one half kx squared minus E is equal to 0.
We'd like to solve this differential algebraic equation.
It's well-posed.
We have one equation for one unknown, one state.
The equation has a solution, which is a times cosine omega t. It's not the only possible solution.
It's a solution to this equation.
We can substitute that solution into the equation and determine, for example what this oscillation frequency omega has to be.
It's square root of k over m. Or what the amplitude has to be.
It's the square root of E over k. So you've seen this from physics before.
We've already solved differential algebraic equations.
The thing you saw before though, was actually conservation of momentum.
You converted this entirely to a second order differential equation in x and solved it.
But in principle, we could have tried to find various solutions to this equation instead.
We would have said, well, we've observed masses and springs and we know they oscillate in time.
So let's propose some oscillatory solutions and see what values of the amplitude and omega we need to satisfy this equation.
It would have been a perfectly acceptable way of solving this problem.
The thing is, this mass spring problem, the differential algebraic equation contains nonlinearities with respect to the states.
So it's nonlinear in the velocity and it's nonlinear in the position.
Nonlinear equations in general are hard to solve.
When you converted that to a momentum balance you got linear equations.
You got the second derivative of position was equal to stiffness over mass times position.
It was a linear differential equation to solve, which is easy to solve.
So you've got higher order differentials, but you linearize the equation so it made it easier to solve, in a certain sense.
But we'd like to sometimes understand what we can do with these equations.
There are certain limiting cases where we can do interesting things with these equations.
I'm going to show you one now.
So in the case where the Jacobian of this function f with respect to the differential of the state variable.
So take the derivative of f with respect to x dot and hold time in the state constant.
So this is a matrix, one that's full rank.
Then the DA be represented as an equivalent to ODE.
Here's how you can show that.
The total differential of f is this Jacobian with respect to x dot times the change x dot plus the Jacobian with respect to x times the change in x.
Plus the partial derivative of f with respect to time, times the change in time.
And this total differential has to be equal to 0 because f is equal to 0 for all states in time.
So let's divide by dt.
We'd like to know how f changes with time.
So total differential of f with time.
We divide each of these little differentials by time.
And then we solve for dx dot dt.
That's going to involve moving this term to the other side of the equation and inverting this Jacobian.
So if I can invert the Jacobian then I can convert my DAE system into a system of ODEs.
This equals 0 shouldn't be here.
I apologize.
That's another typo.
I move this term to the other side of the equation to where the 0 is and I solve.
So I went from a first order equation, a first order equation in x to a second order equation in x.
Two differentials of x with time instead of one differential of x with time.
But I could convert it to an equivalent system of ordinary differential equations.
Does that make sense?
Do you see how that works?
Any questions?
Let's go back and look at an example.
The mass spring system.
So here's our ODE that we're trying to solve.
I'm sorry, our DAE that we're trying to solve.
Can you convert this to an equivalent ordinary differential equation using the approach I just described?
Think you guys can figure that out?
Try it.
You can work together.
[SIDE CONVERSATIONS] WILLIAM GREEN,
Guys are either tired or friendships have been destroyed over the mid-term season.
Maybe both.
So let's compute the things we need to know.
We needed to know the partials of this f with respect to the states and with respect to time.
So partial f respect to x dot, holding all the other variables constant, is mx dot.
Partial f with respect to x is kx.
Partial f with respect to t is t. And I told you before that you could write this says dx dot dt is equal to df dx dot inverse multiplied by-- is equal to minus df dx dot inverse multiplied by df dx.
This was on the previous slide, which just gives you conservation of momentum.
This gives you the acceleration is equal to the force minus kx divided by the mass m. So we can solve either, the DAE or the ODE.
Here's a more complicated example, but it works, well, it's more complicated.
So a lot of times we do molecular dynamics simulations.
We try to model molecules moving around.
There we also want to conserve energy.
There's usually a conserved quantity, something we're trying to hold constant.
So suppose we try to do that.
The total energy in this system of molecules, maybe it's 1/2 mass times the velocity squared.
Now x is the position of all these molecules.
Ad x dot is the velocity of all these molecules plus the potential energy, which is a function of f, a function of their positions.
So the DAE representing this constraint is this.
It's the same as for the mass spring system, where we've replaced the potential energy with a generic one.
This actually isn't a well-posed DEA.
We have one equation for, let's see, if we have n molecules in here we have three n states.
We can move around in three dimensions.
So this is a really ill-posed equation.
If we try to apply the same procedure, then we say, well, this quantity still should be conserved.
Over time it needs to be equal to zero.
Then we can try to compute all these differentials.
But we'll find out that partial f, partial x dot, the Jacobian of f with respect to x dot, that's just the momentum.
That's not a matrix.
That's not invertible.
So we can't convert this equation to an equivalent system of ordinary differential equations.
So it's just something pathological for a one-dimensional system.
Instead, what we find is zero is equal to the velocity dotted with the mass times the acceleration plus the gradient in the potential.
So minus the forces acting on the molecules.
We know that if we were to integrate the equations of motion for the molecules exactly, the momentum balances for those molecules exactly, then this term here would always be equal to zero.
And we'd satisfy conservation of energy.
Of course, all solutions of ordinary differential equations require approximations.
So as you move the molecules around, their velocities and positions will tend to drift away from the solution where this is exactly in balance and equal to zero.
If you desire to do that in the molecular dynamic simulation then you better have a method that somehow respects this geometric property instead.
That the velocities are orthogonal to m x dot plus grad v. And there is a set of ODE-IVP methods that do that.
They're called symplectic integrators.
And they integrate the equations of motion while exerting some control over the error in the total energy.
So if you were just the take a ODE45 and try to solve this system of equations, over long times you would find that the energy drifts away from the place where it started.
Even though your positions are being integrated with reasonable accuracy, over time the energy will drift away from where you started.
The system will heat up effectively or cool down.
That's undesirable if you're trying to do modeling.
So instead, one uses the symplectic integrators, which are designed to control the error and the energy in addition to integrate the equations of motion.
So you can look those up and read about them.
I think they're quite interesting.
If you're going to do molecular modeling you'd like to understand better how they work.
But actually, conservation of energy here doesn't give us a well-posed DAE.
We actually need the momentum equations to formulate or to determine the trajectories of the molecules.
Let's talk about their numerical simulation.
So let's think about these semi-explicit DAEs, the ones that can be split between differential and algebraic states.
It's kind of useful to look at this problem in particular.
And you might say, well, let's do the simplest possible thing.
Let's do the forward Euler approximation.
So let's take our derivative here and represent it as the state evaluated at x plus delta t minus the state at t divided by delta t. Let's make the right-hand side of our equation be evaluated at time t. And let's try to step in time, do time marching forward to determine the trajectory of x and t. How do the states change?
We see the first equation is an easy one to solve because presumably I know x of t and y of t. Well, do I know y of t?
y of t has to satisfy this second equation.
So first, if I know x of t I need to solve this nonlinear equation determine y of t. Then I know x of t and y of t and I can step forward in time to determine x of t plus delta t. So every iteration here with the simplest possible approximation for the differential requires solution of a system of nonlinear equations.
We already saw before that a forward Euler approximation is not a great one to apply to solutions of IDE-IVPs because it has very limited stability properties.
You need small time steps in order to stably integrate in time.
So even though this seemed like it was easy to get a solution for the differential equation, we've always got to satisfy this nonlinear equation here.
So inherently, simulation of DAEs are implicit.
They require solution of nonlinear equations in order to advance in time.
There's actually no point in using this weakly stable method for any very small time steps.
It makes more sense to choose a naturally implicit method for the differential equation, where I can take big time steps and still have reasonable stability of the integration.
Does that make sense?
Consider now the fully implicit DAE.
So f of x, x dot, and t is equal to 0 and have some set of initial conditions.
x of 0 is equal to x 0.
Here there is no way in general to avoid solving systems of nonlinear equations.
And so one substitutes often backwards difference approximations for x dot.
And then you solve for the next point in time.
So here's an example with the backward Euler, implicit Euler approximations.
So I replaced the time derivative with the difference between the state at point tk and the state of point tk minus 1 divided by the difference between tk and tk minus 1.
The other state variables are evaluated at tk.
So x at tk.
And I evaluate time at tk.
And I try to satisfy this equation and use it to determine x of tk.
So I solve this system of nonlinear equations for x at tk.
And then I go to the next point in time, tk plus 1.
And I solve the same equation, but replacing tk with tk plus 1 and tk minus 1 with tk.
So at each iteration I solve a system of nonlinear equations.
No more painful than doing the forward Euler approximation I showed you before.
So that's good.
But still a lot of work.
We might wonder how do we control the error in these sorts of approximations.
And that's what we're going to talk about next.
So here's the equivalent time marching formula applied to a semi-explicit DAE.
So I've got to satisfy the equation 0 is equal to the derivative of x with respect to time, which I approximate with the backward Euler approximation, minus f evaluated at tk.
And the equation g of x at tk, y at tk for this x of tk.
And actually y of tk as well.
I'm sorry, I left that off.
y of tk as well.
Got to solve for the differential and the algebraic states simultaneously.
And I use a backwards difference formula to try to do this in a stable way.
One way to think about in particular these sorts of DAEs is as exceptionally stiff ordinary differential equations.
Did you guys discuss stiffness?
So imagine if I replace the 0 with, say, dy dt.
And I introduced a constant on the right-hand side, a multiplicative constant, say, 1 over lambda here.
Let's say lambda, actually.
Say I introduce a multiplicative constant lambda on the right-hand side here.
So as lambda becomes very large, the system becomes increasingly stiff.
The dynamics of this equation take place over very, very short time scales.
So short in fact, that eventually, if I let lambda diverge to infinity, I've just got to satisfy this algebraic equation at each point in time.
So these are exceptionally stiff equations.
And the methods for solving these equations share a lot in common with stiff ODE-IVP solvers.
So I've got a couple of minutes left.
I'm going to work through some examples.
So consider the stirred tank example one.
dc 2 dt.
It's related to q over v, flow rate over volume.
The difference in the concentrations coming in and out of the tank.
And c1 is equal to this input signal gamma.
Maybe that's a measurement and I'm trying to predict c2.
So can you apply the backward Euler method to these equations to determine c1 and c2?
What would one step of the backward Euler method look like, say, going from time 0 to time delta t, or time t to t plus delta t, or tk to tk plus 1, however you want to write it?
So do you get something like this?
So this equation is easy.
c1 to tk is gamma of tk.
I got a substitute up here.
c2 of tk minus c2 of tk minus 1 over delta t. T Just tk minus tk minus 1.
And then solve for c2 of dk.
And you get a formula like this.
This formula was based on an approximation for the derivative, which had a leading order error that was proportional to tk minus tk minus 1.
When I go through and I solve for c2 of tk, the leading order error in c2, the local truncation error is going to be order tk minus tk minus 1 squared.
So as I shrink the spacing between each of my two time points, the local accuracy, the error induced in one advance of this time stepping algorithm is going to scale like the step size squared.
So I have the error-- I have the spacing.
The error will go down by a factor of four.
I reduce the spacing by an order of magnitude.
The error will go down by two orders of magnitude.
This is a really desirable sort of error control and a method.
And we can do better.
You've seen us do better with other methods.
Let's look at this equation now.
This is stirred tank example two.
I just replace c1 with c2.
You don't have to write this one down.
I'll just show it to you.
The notes are online so you're not missing anything.
All these things will pop up in the notes online.
So c2 at tk is equal to gamma of tk.
I got to come up to my first equation now and I have to substitute an approximation for the derivative.
It's this, the backwards difference formula.
So if find c1 of tk is c2 of tk plus v over q. c2 of tk minus c2 of tk minus 1 divided by the time step.
Remember, this approximation for the derivative has a leading order error that goes like tk minus tk minus 1.
So my approximation for c1, the local approximation for c1, is not second order accurate.
It's only first order accurate.
I applied the same approximation method to both equations.
One equation I got something that was second order accurate.
The next equation I got something that was first order accurate.
They look almost identical.
It might be hard to see from the start why it should work out that way.
That's how it works out.
Let's do one more example.
Here's a system of DAEs.
c2 dot is equal to c1.
c3 dot is equal to c2.
And 0 is equal to c3 minus gamma.
So gamma, again, is some measurement.
We solve this equation to determine c3.
We substitute c3 up here to determine c2.
We substitute c2 up here to determine c1.
But we want to find the solution with the backward Euler method.
And the solution there is going to be c3 of tk is gamma of tk.
c2 is related to the derivative of c3.
So I use my backwards difference formula for the derivative of c3.
c1 is related to the derivative of c2.
So I use my backward difference formula for c2.
But how accurate are each of these approximations?
So here I have to determine c2 from an approximation for the derivative of c3.
We know this backwards difference approximation has a leading order error proportional to delta t, the time step.
Now I got to approximate c1 with the finite difference backwards difference approximation in c2.
We know this should incur an error proportional to the time step delta t, except I don't know c2 at tk exactly.
Actually, c2 tk, the exact value of c2 at tk is this plus something proportional to the time step.
Which means the exact value of c1 is this plus something that's order one because I carry over the error from my solution to the previous equation.
And I divide it by the time step.
So I went from having something-- well, let's see.
The first equation had order delta t squared local truncation error.
As I shrink the spacing, my solution gets more and more locally accurate.
And it seems to do it in a fairly robust fashion.
The next system of equations we solved had a local truncation error proportional to delta t. Again, I can shrink delta t and I can make my solution more and more accurate.
That seems like what you want in a numerical algorithm.
This one, there's no hope.
Change delta t, who knows what you're going to get?
And this looks fairly innocuous.
You just look at it and you say, well, OK. Let's solve it and see what happens.
Well actually, you can't solve this problem accurately with the backward Euler method.
There's something fundamentally different about this problem from the previous one.
There's something fundamentally different about stirred tank example two from stirred tank example one.
Right you might have already perceived that.
You can see what's going on here.
c3 is equal to gamma.
That's the exact solution.
c2 is equal to gamma dot.
And c1 is equal to two derivatives of gamma, the exact solution to this equation.
So c1 is incredibly sensitive to changes in gamma.
And that's peculiar.
And it has important numerical consequences.
So I'm going to show you, our next lecture on Wednesday, how to formalize an understanding of the differences between these problems.
And we'll talk about some more subtleties associated with differential algebraic equations.
You guys can pick up your quizzes.
We should do it outside rather than inside because the next class has to start.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
You get your quizzes back on Monday.
Maybe you've had some time to think about them and reflect on your answers and the problems that were posed, maybe how they compared to the quiz that-- sample quiz that was put online.
I think it was pretty fair exam overall.
It looked very similar, actually, to the exam that we've given in the past, but we tried to-- tried not to put any sort of chemical engineering problems on top of the numerics, which tends to lead to confusion basically.
Spent a lot of time reading and trying to understand what the underlying physical problem is instead of just testing the numerical methods.
Are there any questions about the quiz now that you've gotten it back?
You've seen your answers, things that were unclear that were asked on there?
Now is a good time for reflection if there are.
No.
I heard this last homework assignment was really long, so sorry about that.
I didn't write this one.
[LAUGHS] I'm not going to Take credit for it.
It's hard to judge sometimes how long these things take.
A lot of it has to do with how facile you are with coding up some of the problems.
So conceptually they can be simple, but the coding can be quite difficult, so it's hard to estimate.
I think this week's, which should be posted and is on the topic of DAE, should be easier.
It's only got two problems instead of three.
You're likely going to be able to leverage some of the code from your solution last week for this week.
So I don't think it should be quite so challenging.
There is a last part to the first problem that seems to always give first-year graduate students trouble.
Somehow you guys don't know how to write down energy balances on reactors.
So you might ask pointed questions about this at your office hours, so you don't spend time trying to solve the wrong equations.
This happens.
I think many of you maybe were taught incorrectly or just forgot at some point how to write these things down.
So the office hour is a good point to bring that up, and that way you don't spend a lot of time troubling yourself with the wrong system of equations.
Any questions?
Yes
[INAUDIBLE]
What's that?
I'd-- well, to be fair, almost all the lectures for the rest of the class are going to be taught by Professor Green.
I'll step in and do some review for two sessions, review before the second quiz and review before the final.
Since he's leading the lectures, he'll be responsible for largely generating the homework assignments.
I'll try to help out with that as best I can, but this will be the last formal lecture you get from me this term.
Other questions?
Well, that's very, very sweet of you.
Thank you so much.
Least there wasn't any hushed yes!
Today's lecture is our last one on DAE, so we're only going to do two.
You're going to see today that differential algebraic equations are pretty complicated actually.
And when they get sufficiently complicated, you really need to reach out to existing codes that are designed to solve particular classes of differential algebraic equations.
So for you, it's more important to be able to identify in the models you formulate when these complications arise and what are the essential ingredients of the model that can be put into one of these solvers that you get a result that isn't nonsense.
And that's what we're going to do today.
There's going to be lots of examples to work through.
So make sure you're sitting next to somebody you like because I'm going to ask you to try to think about these things and discuss things as we go through.
Let me review where these complications come from.
So last time we talked briefly about semi-explicit and fully-implicit differential algebraic equations.
I told you in principle, you could simulate these things with backwards difference formulas in solving nonlinear equations at each time step and just marching forward.
Did this in your homework for ODE IVP So you can do the same thing for differential algebraic equations.
But there was a catch to that, and I illustrated that at the very end.
So maybe your brain was fried at the end, and you missed this.
It's good to recap.
So I showed you an example of a stirred tank where you had some-- you had transport of some solute into the stirred tank, and then you're pulling it out at a different concentration.
And we're trying to track the dynamics of the system, the concentration in and the concentration out, and we imagined a problem in which we measured and the concentration in and we tried to predict the concentration out.
So we had a system of differential and algebraic equations to solve.
And I showed you that if I used the backward Euler method, the lowest order backwards difference formula that I can-- of the canonical class of these things that I can craft, where I approximate the derivative of an unknown with a relative error or an error proportional to the time step delta t, that carrying out one time step with this backward differentiation formula would determine c1 in principle exactly-- if I knew gamma exactly, I would know c1 exactly.
And it would determine c2 to within order delta t-squared.
It's the local truncation error here was order delta t-squared.
So you just substitute this formula in, and you ask does this error term change at all.
At some point, I wind up multiplying by delta t, and so I go from order delta t to order delta t-squared It's the numerical error that gets carried around in this calculation.
I just switch the model a little bit.
It seems like a irrelevant change, but it turns out to be incredibly significant.
So now I imagine a different problem in which I'm measuring c2 the outlet, and I want to predict c1 the inlet.
I still have a system of DAEs that I have to solve, and if I applied the backwards Euler method, well, c2 is automatically determined by this algebraic equation.
So I know that exactly.
I got to go up to this first equation and solve it for c1, and when I do, I have to have an approximation for the derivative.
And the derivative is proportional to delta t-- carries an error around with it that's proportional to delta t. So I know c1 to within order delta t, not order delta t-squared like in the previous model.
So there's something fundamentally different about these two circumstances.
And all I did was go from c1 to c2 in this algebraic equation.
So that's peculiar, and that should make you really suspicious about your ability to solve these problems accurately.
And here was the third example I gave you.
So I said imagine this system of DAEs instead.
So here's the differential equation.
Here's an algebraic equation.
Apply the backward Euler method.
Well, c3 is determined automatically by this algebraic equation.
I know it exactly.
C2 is related to the derivative of c3, so I need to approximate it with my backward Euler derivative.
And that picks up an error, order delta t. C1 is equal to the derivative of c2, so I need an approximation for c2.
The derivative of c2 that's the backwards Euler approximation.
That has an error that's proportional to order delta t. But c2 itself also has an error proportional to order delta t. And order delta t divided by delta t is order 1.
So I get c3.
I get c2.
I even know what c1 is.
I solve this problem with my backwards difference formula, and I have no resolution of c1, no clue what that value is.
So this is some complicated control scheme that I've set up, and I need to know the value of c1 to figure out how to operate this process.
I'm lost.
This is never going to work.
So these three problems have fundamental differences between them.
And I'm going to show you how to predict when these differences are going to occur, how to name them.
So the stirred tank example 1, it carried a local truncation error, one time step order delta-t squared.
Stirred tank example 2 carried a local truncation error.
It's order delta-t.
This third DAE example had a local truncation error that's order 1.
No change in delta-t will improve my solution to that problem.
It's independent of how I choose to do my time setting, which, of course, is ridiculous.
So here's another problem.
And I'd like you to try to do this.
We'll see how far you can get through it.
You don't have to get all the way through it, but see how far you can get through this problem.
So can you apply the backward Euler method to this system of DAEs and try to predict how the air propagates.
[INTERPOSING VOICES] OK, I don't want to break up the conversation too much, but tell me what you're finding as you try to do this.
Yes
We found that c3 has to be 0
OK. How did you find that c3 has to be 0?
Stick in this [INAUDIBLE] that's what the difference is of the [INAUDIBLE]
Yes
[INAUDIBLE]
Right
[INAUDIBLE]
Right
[INAUDIBLE]
OK, so this is very clever, and this has to do with how the model itself is formulated.
So you did manipulations with the fundamental equations that I wrote down to determine that c3 had to be 0.
That's good.
We can do that.
We can look at the equations.
We can figure how to eliminate variables and find out-- c3 in this case always has to be 0.
That may or may not be obvious, but one way to do it is take this constraint equation and take its derivative.
This has to hold at every point in time, so its derivative must hold at every point in time.
Substitute these other two equations in, c1 dot plus c2 dot eliminates c1 and c2, and all that will be left is c3 has to be 0.
So if I manipulate these equations, I know c3 has to be 0.
Let's suppose I don't manipulate the equations, though.
Let's suppose I just put in the backward Euler approximation over here for the derivative, and I evaluate c1, c2 and c3 at the current time and ask what is c1, c2, and c3 at the current time in terms of what it was at the previous time And you get an answer that looks like this.
So there's a simple system of equations that one has to solve.
It's a three-by-three system, and it's easy to eliminate.
I didn't really expect you-- actually expected this answer to come up because you guys are all very clever, so you try to make things easy beforehand.
But it's-- there's a problem here.
If I try to just do the backwards difference formula with the equations as they're written, I'll find out that when I solve this thing that c1 will be c1 minus c2 at the previous time step over 2 plus an order delta t-squared error.
C2 will take on the opposite sign plus an order delta t-squared error.
C3 will be minus c1 plus c2 over 2 delta t plus an order delta t error.
And actually if I were to apply successive approximations, different-- if I step-- take many steps in a row with this backward Euler method, there's nothing to guarantee that these constraints are satisfied exactly.
I can have numerical error in my solution of this algebraic equation.
So it isn't necessarily true that in my numerical solution c1 plus c2 is equal to 0.
I can show you right here if I add c1 to c2.
Well, these two are the same.
They'll cancel, but am I guaranteed that these errors will cancel, too?
Or will there be a small numerical error that propagates?
So model formulation is key.
This problem is like stirred tank problem 2.
One of our solutions lost in order of accuracy in the local truncation error.
It's not delta t-squared.
It's order delta t. And what you want it to do, which is not necessarily a bad thing, was to try to use this constraint to somehow eliminate equations and simplify things.
That can be helpful, but it can also be true that if I use this constraint to simplify equations and eliminate things, my numerical solutions may no longer satisfy the constraint at all.
They may drift away from that algebraic equation.
Does that makes sense?
I'm not controlling the error with respect to that equation anymore.
So how do you give a name to this kind of behavior?
And the thing one talks about is the differential index of a DAE system.
So let's look at the stirred tank example 1, and let's ask this question.
How many time derivatives are needed to convert to a system of independent ODEs having differentials of all the unknowns.
So there are two unknowns in this system.
One is c1, and the other c2.
Actually have one equation, which contains a differential of c2.
How many derivatives of these equations with time do I need to take in order to get an equivalent ODE system?
And it's just one.
If I take the derivative of equation 2, I'll get a differential equation for c1.
Now, I have a differential equation for c1 and a differential equation for c2.
DAEs of this type, where I only have to take the time derivative once of the algebraic constraints, are called index 1 DAEs.
We saw that when we applied the backwards difference formula to stirred tank example 1, the local truncation error was the same as what we would get in an ODE IVP system.
So index 1-- DAEs are easy to solve.
They behave like ODE IVPs.
You determine whether it's index 1 or not by asking how many derivatives do I have to take in order to get a system of independent ODEs.
So I put this together with this, I have two independent ODEs.
I could solve these, subject to some set of initial conditions, and the solution might be the same as the solution to the DAE.
Let's do stirred tank example 2 now.
So here we go.
They look very similar, but now c2 appears in the algebraic equation instead of c1.
So let's take a derivative of the algebraic equation, and I get a differential equation for c2.
But I already had a differential equation for c2.
I want a differential equation for c1.
So what do I do?
While I know dc2 dt from up here.
So let's substitute that in and solve for c1.
So a substitute equation 1 in here.
I solve for c1.
Now, let's take another derivative.
Call this equation 3.
Derivative of equation 3 now gives me a differential equation for c1.
It's also in terms of dc2 dt.
If I want-- I don't have to-- but if I want to, I can substitute for that again to just get dc1 dt in terms of c1 and c2.
But it took two derivatives to generate a system of independent ODEs from the DAEs.
And this is called index 2.
So it's a different character from index 1.
We saw what happens.
We tend to lose an order of accuracy at index 2.
Somehow index 2 problems are more sensitive than index 1 problems.
Here's another example.
So DAE sample 3 is going to proceed the same way.
So first we need a differential equation.
Which one's missing here?
We don't have differential equation for c1, and that's what we're hunting for.
So let's take a derivative of the third equation, the algebraic equation.
We get c3 dot is gamma dot.
And we already have an equation for c3 dot.
That was 2.
So substitute 2 in.
So now we got c2 is gamma dot, but we're really looking for something in terms of c1, so let's take the derivative again.
We've got c2 [?
dot ?]
is gamma dot dot.
Doesn't help us much, but c2 dot we know is equal to c1.
So we substitute one more time and take a third derivative.
Now, we have a differential equation for c1.
And we have differential equations for c2 and c3.
We can take this is the differential equation for c3.
This is the differential equation for c2.
This is the differential equation for c1.
Some subset of these can be chosen, and we can replace this algebraic equation with a differential equation.
This is an index 3 DAE.
It's not always a good idea to replace the DAE system with these derivatives.
I can write down-- I can have a particular function that's equal to 0 and higher-order derivatives of that function are also equal to 0.
But the solution to those differential equations are not necessarily equal to that function.
There are lots of functions that might have this same property that its derivative-- certain number of derivatives of it are equal to 0.
So there have to be extra initial conditions or constraints on the solution that confine me to the manifold of solutions that reflects the DAEs.
We'll talk about consistent initialization later on.
And that's going to fix this problem.
So it's not always a good idea to do this, but it's one way of understanding, given this model, what sort of sensitivity is it can exhibit.
We try to calculate its differential index.
Yes?
So [INAUDIBLE]
Yes.
Yes
[INAUDIBLE]
Well, we have to be careful.
We need to choose a set that are independent of the others.
OK.
So like these three are clearly going to be independent, and it's going to be OK.
It's going to be a problem if I choose this one and these two.
They're not going to be independent of each other.
So we have to select independent ones from the set.
It may not be obvious what independent means in general
[INAUDIBLE]
1, 2, 6 could also work, yes.
Yes.
So in general-- not in general.
Let's talk about the differential index of a semi-explicit DAE system.
Remember semi-explicit meant that the differential variables can be written as x dot or dx dt is equal to some function of x and y. Y are the algebraic states, and they satisfy a separate equation, g of x and y and t is equal to 0.
Semi-explicit form.
So with a semi-explicit DAE, the differential index is defined as the minimum number of differentiations required to convert the DAE to a system of independent ODEs.
What does that mean?
Means let's take the algebraic equations and let's take a time derivative of them.
That will give us a new function.
Call it g1.
It's going to be a function of the differential state, the algebraic state, and the time derivative of the algebraic state.
It could also be a function of the time derivative of the differential state, but we know that in terms of f, which is a function of x and y.
So let's not try putting x dot in there for convenience.
Let's just write out like this.
In principle, we can do this.
Let's take two derivatives of it.
It will be the same way.
It'll give us some function g2, which is a function of the differential state, the algebraic state, and the time derivative of the algebraic state.
Let's take as many of these as we need.
It will give us a system of equations that we can eventually solve for the time derivatives of all the algebraic states and convert this DAE to a system of ODEs.
And the question is how many of these derivatives do I have to take?
Index 1, I need to take one.
Index 2, I need both sets of equations in order to get something that I can actually solve for dy dt.
Index 3, I get to take three derivatives and higher.
OK, so here's an example.
Let's see if we can work through this.
So we have a DAE system.
Can you calculate the differential index of this system?
Go ahead.
Not 0.
Not 0.
ODE IVP is-- are DAEs of index 0.
They require no derivatives to generate a system of ordinary differential equations.
What do you think?
[INAUDIBLE]
Index 2 you say.
How do you come to index 2?
Why is it index 2?
How'd you do it?
[INAUDIBLE]
OK, differentiate the algebraic equation
Stick it in [INAUDIBLE] 4.
You get c3 for 0.
Then you have to differentiate [INAUDIBLE]
Good so differentiate this equation.
Substitute for c1 dot and c2 dot.
C1 minus c1 will be 0 c2 minus c2 will be 0.
So you have 2c3 equals 0.
That's still an algebraic equation.
We need one more derivative to get a differential equation for the only algebraic state we have, c3.
So one more derivative will tell us dc3 dt is 0.
So two derivatives to get differential equations.
It's an 2 to DAE.
Sam?
What if you can get c3 equals 0 and just eliminate it then wouldn't you write an equivalent some simple equations, but that would be [INAUDIBLE]?
You could.
So we could write an equivalent system of equations that says instead of this equation, report c3 equals 0.
That's model formulation.
Here we've given-- we're given a model, and we're asked what index it is.
We could formulate a different model, and the model will have a different index.
If I were to replace this equation with c3 equals 0, what would the index of the DAE system be instead?
Index 1 instead.
I already told you index 2 is harder to solve than index 1, so if you can formulate an index 1 DAE, you should probably do it.
But it may not be obvious whether you have or haven't.
So it's an issue of model formulation.
It's a great question.
Does that distinction-- is that clear?
Or is it a little-- am I not making it clear how this works?
Unresponsive.
OK.
It's OK. Let's-- there's a generic index 1 example that we can talk about actually.
So let's take a look at a semi-explicit DAE system.
So we have x dot is f of xy and t and 0 is g of xy and t. Let's take the time derivative of the algebraic equation.
So the total derivative of g with respect to t is the Jacobian of g with respect to x times dx dt plus the Jacobian of g with respect to y times dy dt plus the derivative of g-- partial derivative of g with respect to t. And now let's solve.
Let's push the dy dt term to the other side of the equation, and let's substitute for dx dt.
We know dx dt is f, so you get this equation here.
If the Jacobian is full rank-- if dg dy is full rank, then I can invert this matrix, and I automatically get my system of ODEs for the algebraic states.
What's problematic about this equation when dg dy is not full rank?
That should be a partial.
That's sloppy.
I was doing this at 1:00 last night, so that's why that's there.
What's the-- what's problematic about this root finding problem here if dg dy or the Jacobian of g with respect to y is not full rank?
You recall?
If--
[INAUDIBLE]
Yes, you won't be able to compute the inverse to get your dy dt's.
That's true.
Remember this Jacobian-- if it's not full rank, it's determinant is 0.
There is no inverse of this thing.
We talked once about the systems of nonlinear equations and locally unique solutions, the so-called inverse function theorem.
We can only guarantee there is a locally unique solution when I can invert this thing.
That if I got in really close to the solution, it looked like a linear system of equations.
That linear system of equations had a unique solution associated within.
And everything was good.
We were happy.
So index 1 DAEs have that property.
If I wanted to solve this equation for y, there will be a locally unique solution for y.
Higher index DAEs don't have that property.
We know that's a sort of unhappy generic circumstance to be in if we have to solve this equation.
It can be hard to find those roots using, say, Newton-Raphson because the Jacobian [?
e ?]
would need to become singular during the root finding procedure.
This is connected intimately to what we did for systems of nonlinear equations.
So for an index 1 DAE, you can show its index 1 because this Jacobian is invertible.
Let's do some more examples on differential index.
This seems to be the most important thing.
If I have a model, what's its index because its index is going to tell me how sensitive it is to small perturbations.
So here's a model.
It's the same as the model you saw before, but now it's two mixing tanks.
So here comes an inlet flow q1 carrying concentration c1.
Out comes the same flow q1 carrying concentration c2.
The mixer has volume v1.
And then I blend this with some more of whatever the solute is at a flow rate of q2 and a concentration c3.
And those both go into this tank, and they come out concentration c4, flow rate q1 plus q2, and this has volume v2.
Does that look good?
Well-posed model?
OK, so here's your material balances on the mixers.
And I'm going to give you different algebraic constraints.
So this is a problem now where we say measure c1 and c3.
And we want to try to predict c2 and c4.
Can you figure out the differential index of this DAE system?
Feel free to talk to each other.
It's OK. OK, the differential index is 1.
This one's the easy one right?
Just take one derivative of the algebraic equations.
Now, I've got dc2 dt, dc4 dt, dc1 dt, dc3 dt.
Differential index of 1 is easy to solve as an ODE IVP.
This is the natural problem, too.
It's useful to think about the inputs c1, c3 and asking about what the outputs are.
Physically, this problem seems easier in nature.
OK let's change it now.
Same problem.
But let's change the algebraic constraints.
Now, the algebraic constraint is I measure c3 and c4, And I want to predict c1 and c2.
What's the differential index here?
Yes
[INAUDIBLE]
Sure
Why was this a differential index of 1 if you have to take a derivative for c1 and this equation?
Oh, good question.
I'm going to push the slide back, and I apologize for this.
And I'll go back for it.
So the question was why was this differential index one if I had to take a derivative of both this equation and that equation.
So the way to think about this conceptually is I took a derivative of the algebraic equations, the set of algebraic equations.
It was one time derivative of a vector valued function.
I can see where the ambiguity is there.
So it's important to be clear.
I took one time derivative over the equations that prescribe the algebraic states.
That's the distinction.
So this is index 1 because I needed one time derivative of this type, dg dt of the entire set.
Here we go.
We're on this one.
Prescribe c3.
Prescribe c4.
What is the differential index now?
What do you think?
We got an answer?
[INTERPOSING VOICES] I hear 2.
I hear 3.
I saw 3 like this.
Almost mistook it for a 2, but that's 3.
Yes.
It's 3.
Let's see if we can work through why it's 3.
So I take a derivative of the algebraic equations, one derivative.
And after taking that derivative, I'll get a differential equation for one of the algebraic states.
So c3 is taking care of.
Now, I'm in the hunt for c1.
So I have-- after that first derivative, I have an equation dc4 dt is gamma 2.
And I know dc4 dt, so I drop that in, and I get an algebraic equation relating the derivative of gamma 2 to c2, c3, and c4.
This new algebraic equation.
This is like the g1 that I prescribed in the generic scene.
I take a derivative of it again because I'm in the hunt for c1.
So I take the derivative of this new equation, and I'll get a derivative of c2, a derivative of c3, and a derivative of c4.
And I know all those.
I know the derivative of c2.
I know the derivative of c4.
And I know the derivative of c3 from the previous level in the hierarchy.
I figured that out already.
So it's two derivatives.
Still hasn't gotten me a time derivative of c1.
But when I make those substitutions, I'll get an algebraic equation in terms of c1.
And I can take one more derivative, and that will give me a dc1 dt.
And I'll have an od for c1, c2, c3, and c4.
So its differential index 3.
It makes sense sort of.
I'm taking a measurement of an output way down the line here.
And I'm trying to predict what the input was in the first place.
It's easy to imagine that there's a huge amount of sensitivity in that calculation.
Here's another one.
What's the differential index here?
I'm now prescribing c1 and c2, and I want you to tell me c3 and c4.
[INTERPOSING VOICES] Oh, good.
Somebody noticed early on.
So you say it's not possible.
Right.
Why isn't it possible?
You take derivatives of both of those--
Yes
[INAUDIBLE] you can't isolate c3--
Right.
We'll never be able to isolate a derivative of c3 here.
There's actually there's something wrong physically with this problem.
Yes, somehow I'm supposed to measure c1 and c2 and use that to predict c3 and c4.
I don't know c4, and c3 is an input.
I don't know it either.
How do I figure out an input when I don't know the output?
It's impossible.
So this model is flawed.
We can formulate any model we want, pick any set of these variables to prescribe algebraically, but not all of them are going to admit solutions or make sense.
There's no number of derivatives that's going to give us a closed system of ODEs.
This again comes back to the point that really DAEs is all about model formulation.
There are lots of good numerical methods.
They work like the numerical methods for ODE IVPs.
What's important is getting consistent models down, models that, for example, have solutions.
You might feed this to a DAE solver and get nonsense because I can almost freely prescribe what c3 is, and I'll get some answer for c4.
Any c3 can give me an answer here.
So who knows whether this numerical solver is sensitive or not to this particular pathology in the problem we've written down.
We got to speed up just a little bit, and I'm sorry for that.
So we looked at index 1.
That was stirred tank 1.
Index 2, that was stirred tank 2.
Index 3, that was stirred tank 3.
And one thing we saw when we looked at them was this index 1 solution.
It had pieces that were proportional to this forcing function gamma, this prescribed function in our system of equations and proportional to its derivative.
So if gamma is jumping around, well, c1 will jump around.
C2 is going to be smoothed out version of gamma because it depends on its integral.
It's not very sensitive, kind of like an ODE IVP.
Doesn't really show any more sensitivity than an ODE IVP does.
The dynamics are different for index 2, though.
When we look at the solution of the index 2 DAE, we found c2 goes like gamma, but c1 has to go like gamma dot.
So that's pretty sensitive if you're making a measurement, and there's noise in the measurement.
How do you even know what this derivative is?
So c1 is wildly bouncing around.
Our prediction of what c1 is is-- it's not going to be very good.
Our solution of DAE example 3 when it told us c1 was related to the second derivative of the forcing function, c2 to the first and c3 was proportional to the forcing function.
So this is hugely sensitive to changes in the forcing function.
So the higher the index goes, the greater the sensitivity to perturbations in the system.
Here's another simple example.
You have all that data, actually, in your paper, so I won't ask you to do this one given our time constraints.
But here's-- mechanical systems that have constraints are often indexed 3 DAEs it turns out.
You can show this one is an index 3 DAE.
This is the case of a pendulum swinging back and forth.
So it's a-- change in position is its velocity.
Its acceleration balances gravity, and there's some arm that holds the pendulum in place.
We can imagine that that arm has some tension in it.
It acts like a spring with a spring constant that changes in time in order to hold the pendulum at a fixed distance from its center.
So two differential equations, one algebraic equation.
This is a differential algebraic equation.
You can imagine lots of mechanical systems work in exactly this way.
They can only move along prescribed paths.
They're constrained in how far they can stretch or go.
I don't know.
If you're trying to design a robot or something, DAEs are pretty important, and they're all of index 3 type it turns out.
So differential variables here are x and v, and the algebraic variable is the spring constant, k, which has got to adjust dynamically in time in order to-- got to get away from the speaker-- adjust dynamically in time in order to provide just the right stiffness to keep this going around on a circular orbit.
So when-- let's suppose I start with my pendulum down, and it's not moving, then this has to be just stiff enough to balance gravity.
The pendulum is 90 degrees, and it's not moving.
I don't need any stiffness.
K and just be 0.
There's no forces to balance.
The pendulum is swinging around, and I have to be stiff enough to balance gravity where I am and also counteract any sort of centripetal [?
acceleration.
?]
So this k has got to be wildly fluctuating.
If I was trying to control that k somehow to give this system these particular dynamics, you might imagine it's very difficult to do.
So there's your differential variables.
Here's your algebraic variable.
You take a derivative of this equation, you'll get a constraint that tells you the velocity is orthogonal to the position.
Of course it is.
I'm on a circular trajectory.
You take a derivative of this equation now, this algebraic equation.
You'll get another constraint that gives you some relationship between k, x, and v but not a differential equation for k. Take another derivative of this algebraic equation, and you'll get a differential equation for k. So in principle, I could formulate a system of ODEs with dk dt, dx dt, dv dt will be equivalent to this.
It took three derivatives to do this, though.
So it's an incredibly sensitive system.
It's index 3 in nature.
So if I transform to this equivalent set of ODEs, the problem-- we discussed this earlier-- is that the solutions may drift away from the initial set of constraints.
The solutions also need the right sort of initial conditions.
Here I know that at time 0, c3 better be gamma 0.
I know that in the actual solution to this problem, c3 dot was gamma dot.
So at time 0, c2 better be gamma dot of 0.
And at time 0, c1 better be gamma dot dot of 0.
And if it's not, then there's going to be some step jump or some strange behavior in the solution to this ODE system.
So I have to choose the right sort of initial conditions.
If those initial conditions aren't chosen correctly, then I'm not going to get the right solution.
I won't be constrained to the manifold of solutions that's given by the original DAE.
So here's some things you need to know.
Index 1, semi-explicit DAEs can be safely handled in MATLAB.
So ode15s, ode23t, they have the ability to take an input of mass matrix.
We discussed mass matrix last time.
And a right hand side to the system of equations in f of x and t and solve it just like all the other solvers, all the other ODE IVP solvers in MATLAB.
If it's not index 1, MATLAB can catch it.
So it'll try to look at the Jacobian of f with respect to the variables and the mass matrix and to infer from that what the differential index is.
And it will tell you often times-- I think depends which package you're using.
But if you're using certain packages, it'll tell you.
It's not index 1oe-- DAE.
We can't handle this.
The methods built into it aren't suitable.
For generic DAEs, there are specific DAE solvers.
So something like SUNDIALS or DAEPACK-- this is Professor Barton's software actually-- that are designed to handle DAEs up to some certain index instead.
They're built to reliably solve these problems.
So we have robots with constraints on them.
We solve all sorts of problems for chemical process systems that are of higher index.
And the way we do it is using specific numerical methods.
They're based on the same sorts of schemes, backwards difference formulas, but careful analysis of the equations, which are in general unstructured, the software doesn't know beforehand what those equations look like, but a careful analysis of them in order to figure out how to minimize numerical errors and stay on the correct solution manifold.
As input to these, though, we have to give initial conditions.
So that's going to be the last thing that we talk about here.
And those initial conditions have to be prescribed what we call consistently, or we can get numerical errors, or the software can just quit and throw an error, or it can run off on some other solution manifold that doesn't look anything like the solution we're after.
The pendulum is an interesting example to think about.
So you might say, well, what do I want to know about this pendulum.
Maybe I want to know where it started initially and what its velocity initially was.
Can I prescribe the initial position of the pendulum arbitrarily?
What do you think yes or no?
No
No.
Why not?
Why no?
[INAUDIBLE]
Good, yes.
We know it's got a fixed arm length.
Can't pick any position.
That's not going to work.
It's got to satisfy that algebraic constraint.
Can I specify its velocity arbitrarily?
What you think?
Can I give it any initial velocity that mass at the end of the pendulum?
No
No.
Why not?
[INAUDIBLE]
What's that?
[INAUDIBLE]
Well, explain that to me, though, because I had equations for the velocity and for that acceleration and then an equation that told me that the length of this moment arm had a certain length associated with it.
So why does that constrain the velocity?
You're right.
It does.
I'm not saying you're wrong.
You're right, but why?
Those initial three equations don't tell me anything about the initial velocity was supposed to be.
We know something about the trajectory this is supposed to sweep out, though.
It's supposed to sweep out a circular trajectory.
And the velocity in that circular trajectory is always going to be orthogonal to the moment time.
And that equation actually-- it appeared.
It appeared when we took a derivative of the constraint.
So this was a hidden constraint.
It's secret.
You didn't know it was there until you took a derivative, and you tried to go and figure out what index this was.
Then you discovered velocity and position or orthogonal to each other.
My initial condition should respect that.
How can they not?
The initial conditions have to be a solution to the equations as well.
So now can the initial stiffness be specified arbitrarily?
No.
There was also an algebraic equation that popped up at some point right here when I took the second derivative, which tells me how the stiffness has to be related to position and velocities.
It's also hidden inside the structure of this DAE.
So this-- again goes to model formulation, what are the initial conditions that I have to prescribe?
If I prescribe the right ones, I'll get a solution that matches the dynamics of the problem I was actually interested in.
If I prescribe them incorrectly, who knows what's going to result?
We can't really predict.
Depends on the solver we're working with.
Here's a formal way to think of it.
So usually in an initial value problem, usually we want to know what's-- for these first order difference-- differential equations, what's the initial value of the state, and what's the initial value of its derivative.
That completely specifies what's going on at the beginning.
One way to think about these problems is I've got this equation that I want to solve.
I could think in some sense that x and x dot are independent of each other.
I could always write this is something like an equation for f of x and z.
And then I could add some extra information that tells me actually x dot is the same as z.
So initially, I would like to know the values of x and z, that z is x dot.
So I'd like to know initially the values of x dot and x.
And tell me where I start, and these need to be consistent with the initial value problem that I specified.
If I'm doing an index 0, an ODE IVP, at least be consistent with this.
So if you tell me x naught, that I should put x into this equation, then that should tell me what x dot is.
If you tell me x dot, then I should solve my governing equation for x naught.
You could tell me some equation that relates x naught and x dot initially, and now I got to solve this equation in conjunction with my governing equation to determine these-- this set of values.
I'd like to know both of these things initially.
The fully-implicit DAE, I really have two n unknowns here.
I don't know x, and I don't know x dot.
And I've only got n equations for those.
So apparently there's n degrees of freedom to specify here.
I need n more equations to say what those initial conditions are.
And these hidden constraints I point out actually reduce the number of degrees of freedom.
The fact that these are not index 0 problems but have a finite index will introduce extra constraints that reduce the number of degrees of freedom or the number of other equations I can use to specify those initial conditions.
Does that make sense?
Let me give you some examples, and then I'll let you go.
So if I have a problem that has separate differential states and algebraic states, this is like f of x dot x and y and t equal 0, the set of things that I need to specify-- I need the initial derivative of the differential state, the initial value of the differential state, and the initial value of the algebraic state.
I'd actually need to know the initial value of the derivative of the algebraic state here because I don't need that to satisfy this initial equation.
Let's look at a couple of examples.
So here's stirred tank example 1.
I converted to a system of ODEs by taking the derivative of equation 2 here.
So here's my two ODEs.
I got to look back at my initial equations and see how do they constrain the initial conditions.
So equation 2 tells me that c1 initially better be gamma initially.
Otherwise, my initial condition doesn't satisfy the governing equations of the model that I wrote down.
Equation 1 tells me that the derivative of c2 initially better be related to the derivative of c1 and the derivative of n-- I'm sorry to the value of c1 and the value of c2 initially.
This third equation that I came up with, it doesn't constrain.
It's a derivative of one of the original algebraic variables, so it doesn't constrain that original set of-- I need to know the initial condition on the differential variables, the derivative of the differential variables, and the algebraic variables.
This only tells me something about the derivative of this algebraic variables.
This equation doesn't give me any other constraints.
So I just got a free variable that I can specify.
One more equation.
It can be whatever I want.
So maybe I say c2 of 0c0, or maybe I say there's some relationship between c1 of 0 and c2 0.
Or maybe I say that c2 of 0 and c2 prime of 0 are related in some way.
Whatever I specify, I have a system of three equations for three unknowns, c1 of 0, c2 dot of 0, and c2 of 0.
I need to have a unique solution to that.
Otherwise, there's going to be a problem solving for it in time, the system of equations.
So I have to choose this one consistent with these other ones, but beyond that I'm OK.
Here's stirred tank example 2.
So I take a derivative of equation 2, and that gives me a differential equation for dc2 dt.
Actually, I already had one of those, so I substitute for dc2 dt, and I take another derivative to get a differential equation for c1.
And let's ask about the initial conditions.
So the initial algebraic equation, it constrained dc2-- it constrained c2.
It said c2 of 0 has to be gamma 0.
The initial differential equation told me that c1 of 0 had to be equal to c2 of 0 plus v over q, c2 dot of 0.
Remember, I need three equations to specify c2 of 0, c1 of 0, c2 dot of 0.
I actually can't specify c2 dot of 0 freely because in the process of deriving this differential equation here so that I had dc1 dt and dc2 dt, I introduced a constraint on the derivatives of c2.
And this tells me that c2 dot of 0 has to be gamma dot of 0.
There are no free variables to specify.
There's three equations.
Two of them come from the initial system of DAEs, and the third one is related to this extra equation that popped up as we tried to define the index.
It's an implicit constraint on the problem.
Does that make sense?
I can't specify these things freely.
They're specified by the signal that I measured gamma.
There are two examples here that you should be able to work through.
One is an index 2 example, and one is an index 1 example.
We're not going to have time to go over them.
Sorry for that.
But they work the same way.
So convert to a system of ODEs, ask what initial conditions will satisfy the initial constraints and were there any other-- were there any other constraining equations that popped up?
So if you work through this example, you'll find out there was a hidden strength that either the initial derivatives of c1 and c2 sum together have to be 0 or equivalently the initial value of c3 had to be equal to 0.
You can take this constraint in either place.
Maybe you want to take both of them to make sure that it's always satisfied, that there is no numerical error that drives you off of this constraint.
It's OK to have these things over constrained.
It's just not OK to have them under constrained.
And you'll find out that while there were five things I could pick, the derivatives, and the values of c1 and c2, and then the algebraic variable c3 initially.
So there are only four constraints-- for equations here for the initial conditions.
I get to pick one more.
I can get it however I want.
Just has to be consistent with these.
There's one more example.
Again, should work through these on your own.
But you do the same thing.
Convert to a system of ODEs.
You'll find out that when you do this, you'll get a differential equation for c3.
It didn't introduce any new constraints on c1 dot, c2 dot, or c1, c2, and c3.
So you just need to satisfy-- your initial conditions have to satisfy the initial equations, and you can put in two more conditions, whatever you want them to be.
Add five unknowns and only three equations for the initial conditions.
And that's it.
So thank you very much.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
So today we're going to talk about boundary value problems.
But I want to try to start back with a motivating example.
So unfortunately, like most technologies, the biggest driver for differential equation solutions was from military technology.
And the military problem-- I'll tell you in a second.
Let me first remind you.
So we have ODE-IVPs.
And they are-- right.
So that's the ODE-IVP is we have n differential equations, and we have n initial conditions.
And you guys are really good at these now.
Had a lot of practice.
But you can have problems where some of the conditions are not specified at the t0.
They are specified somewhere else.
So if all of them are specified at t0, we call it an IVP.
And we call it a BVP is if the conditions are like this.
This is not so good.
And not all the tm's are the same.
If all the tm's are the same, we call it t0, and we're back to IVP.
So IVP is like a special case of the ODE-BVP problem.
And here I'm ranking them as if everything is explicit.
So we know an explicit equation for dy dt.
We have an explicit equation for the boundary conditions.
In fact, often, we actually only have implicit equations.
So I'll show you an example in a minute.
But that's it.
And we still need to have n equations.
Well, I guess I could do this.
Take this back.
Basic goals, and just be just be careful.
These ends are-- it's the same number of them.
It's the same number of conditions.
This is OK to write it this way, but just be careful.
Is that OK?
Just to make it confusing.
All right.
I guess I can call this 0 now.
See, but now it's not really 0, right?
It's really maybe-- call it star at the boundary.
There's some special value-- some special place where you know the value of y.
All right.
So let's do the motivating example, which comes from military technology.
And this is quite an old problem.
So, because, in the old days, people didn't have motor vehicles, and, basically, the roads were no good, and there were a lot of bandits out on the roads, people liked to ship everything by water transportation.
And so all the towns grew up on rivers, or on ports on the ocean.
And so, if you lived in a nice little town, you know, up in the hills there, and then, at the base of the hills, you have your cute little town.
You have your house.
You have some warehouses where you keep all your nice goods.
You have a dock that goes out onto the water.
Here's the land.
Here, these nice ships come and park at the dock.
They unload nice goods.
You trade your stuff to them.
You make a lot of money.
Everyone's happy and prosperous and having a great time living.
This is a port city.
But then you look out on the horizon in the water, and you see a ship is coming.
And sometimes that a ship is coming is really good news for you.
But sometimes the ship coming is not good because on the back, it has a flag, and on the flag, you see skull and crossbones.
And you realize it's a pirate ship coming.
And now you're in serious trouble because you really don't want the pirates to come to your nice little town because who knows what they'll do there.
But it won't be good.
All right, so the people who live in this town, after this has happened to you a few times, you're really alarmed, and you're really happy, and you don't want to rebuild your town again because it just got burned down the last time and the pirate came and pillaged it and took everything.
And you guys all had to run away into the hills.
So instead, they've invented cannons or catapults, put one of the cannons, say, at the top of the hill above your town.
And then you say, the next time the pirates come, I'm ready for them.
So drawing the cannon.
And so the question is, you want to shoot the cannonball so it hit the pirate ship at the waterline, when the pirate is distance L from the shore-- hopefully, pretty far, so they're not going to shoot at you first.
And you're shooting from up at the top of the hill, so you can shoot further than they can.
So you're going to shoot distance L. And you're shooting from an elevation H. And you want to make sure your cannonball hits them, ideally, right at the waterline.
Boom, makes a nice hole in their ship, and it sinks.
So that's the plan.
Now the problem is, getting the cannonball to go from here to there is not so easy.
It's a big ocean.
There's a lot of places your cannonballs can fall into the ocean, and it might not be where the pirate ship is.
And reloading canon is pretty slow.
And you guys live in this prosperous port.
You're not military guys, and so you don't have that much practice doing it anyway.
And you're cheap because you're merchants.
And you don't want to spend your money buying lots of cannonballs anyway.
So you only get a few of cannonballs up here.
And there's a little pile of cannonballs next to the cannon.
And then, when you see the pirate ship coming, you guys run up the hill quick, and you get to take a couple of shots, and that's it.
And if you hit them, then your town is safe, and, otherwise, everything you love and know in your world is going to be destroyed by the pirates.
OK?
So this was the motivating sample for that element of differential equation solvers.
OK?
So let's see if we can write down the equations that correspond to this example.
So it's going to be Newton's laws.
F equals ma on the cannonball as it travels.
So we can write that also as equals-- well what's the F?
F is going to be negative mg in the z direction.
That's gravity pulling the cannonball down.
And then you'll have some air friction on the cannonball.
And I'm going to call the air friction coefficient gamma m times v. OK?
So that's the simplest model for what the forces are on a cannonball just flying from my cannon, hopefully, to hit the pirate ship.
And I can rearrange us by dividing through by m. And acceleration is velocity, so dv dt is equal to negative g. So you have negative gamma-- sorry.
My notation is not so good.
OK. Everyone is all right with this?
Yeah.
So then we can write the whole differential equation system, d dt of xz vx vz equals dx dt is vx, dz dt is vz.
dvx dt is negative gamma vx.
dvz dt is negative g minus gamma v ez.
So that's a fine ODE system.
It's not too hard to solve.
It's actually linear.
It's a piece of cake, right?
You guys just probably just did this already when you were in high school, or college, anyway, for sure.
And let's write down the initial conditions.
So we know x0 is equal to 0. z of 0 is equal to h, the height of the canon.
vx of 0, oh, what's that equal to?
We don't know.
vz of 0, we don't know that either.
But we bought this canon with a certified muzzle velocity v0, so we actually know that vx of 0 squared plus the vz of 0 squared is equal to v0 squared-- whereas, when I bought the canon, the merchants went out to the local canon dealer and bought a canon, and the guy said, you put this much gunpowder in it, you'll get a muzzle velocity of v0.
So they know that that number is.
So we have a condition that's not really the way we expect it to be.
And we don't have enough conditions.
We need four conditions.
But of course, the most important condition is what happens at L. We really want to know that x of t final is equal to L and y of t final is equal to 0 at sea level.
So that way, our cannonball is going to into the right location at t final.
So we have two conditions specified way, way away from where we are.
And we have two conditions specified initially.
And no you have some oddball thing in here, some oddball fifth condition.
Now, OK. One problem here is, we don't actually know what t final is because before we shoot it, we really don't know how long it takes the cannonball to get out there.
So this is a kind of weird condition where t final is unknown, whereas, the ones you've done before, I think you've always known t final.
Is that right?
So let's talk about how to deal with that.
So one problem, one way to deal with this is to notice that, if we have an equation system dy dt equals f of y, we could change variables instead of using t. Since we really don't know much about time in this problem, t is not the best variable for us.
So we want to change it to some new variable u.
So suppose that we want to change to u.
And, if we change to u, we can write something like this-- dy du du dt equals f of y.
Can you see that?
So I can try to get the differentials in terms of u, and I just have to know what my du dt is.
And you guys might have even changed the variables before, in calculus sometime, you probably did this-- a long time ago, and chain rule, and stuff like that.
All right, so we can sort of pick whatever idea we want, and as long as we put in the du dt in the equation, then we'll be OK.
So let's just choose that u is equal to x because we know du dt would then be with the vx.
So we could we could write this out this way.
We could write d-- I'll keep it as u for a minute, so the notation won't be so confusing.
So du dt, dy du is equal to f of y over du dt.
And as long as du dt is never zero, we're good.
So we could any u we want as long du dt is never zero.
Then we're still going to have a nice simple question here.
And so I'm going to choose u is equal to x because du dt is vx.
And I know that vx is never zero.
They're kind of also always moving to the right.
So I'm going to divide by something.
It's never zero.
So I can write it this way-- dy du is equal to-- I can go down that list and divide each of these things by vx.
So it's 1, vz over vx, negative gamma, negative g over vx minus gamma vz over vx.
That's my new [INAUDIBLE].
OK?
OK with this?
How about energetic?
Keep nodding.
Yeah, yeah, sure, keep going faster.
It's going way too slow.
That's what I want you to say.
Yeah?
Apparently not.
OK. All right.
So the advantage to changing here is that now I can write the initial conditions, and all these conditions, in terms of u, which is x.
So I really-- I want to go from u is equal to 0, i.e.
X is equals to 0, to u is equal to L, which is x equal to L. So now, I could also look at this say, this is kind of stupid.
dy du, the first component of y is dy du is equal to one.
And that first component of y never appears in the rest of these equations.
So that's kind of not a very useful one.
So I can change, and I could write a simpler equation, dy tilde du is equal to vz over vx negative gamma, negative g over vx minus gamma vz over vx, where my y tilde is just equal to the last three variables-- z, vx, vz.
So I did a lot of algebra, but I ended up simplifying my life because now I have three different equations to solve instead of four.
And I can write that y tilde of zero of the first element of y tilde, which is z, is equals to h. So that's a condition I know for sure.
And I know that I really want this cannonball to hit the pirate ship.
So I want y1 tilde of L to be 0, so I want tilde as z.
And I want z to be 0 when x is equal to L. Or that's the point where the cannonball is hitting the ship at the water line there.
So those are definitely two conditions I have.
And then I need a third condition.
And my third condition is this crazy equation here.
All right, so now I have three conditions for three differential equations, and one of them is an oddball looking thing.
These two look kind of normal.
However, this one is a t0, and this one is some goofball value, but at least I know the value.
I know what L is because I know how far away the pirate ship is.
So far so good?
All right.
But how am I going to solve this?
So how do we know how to solve differential equations right now is we know how to do initial value problems.
We have these nice programs like ode45 that work great, integrate differential equations.
So we should be able to use that somehow to solve this.
But we don't have all the inputs because ode45 would expect us to give values of all three of the y tilde values at 0.
I only know one of them.
So how can I approach it?
Sorry, I'm going to put my beautiful drawing down below here.
We'll save it for posterity and look at it later.
So I could approach it by saying, well, I don't know what these two initial conditions would be, what vx and vz should be at 0.
I'll treat them as unknowns.
I want to figure out what they should be.
And so, I'm going to vary those unknowns and then, for each value I choose, then I can shoot the cannon with those v's, and I can see where the cannonball went, and if it didn't go to the right location, I can readjust those v's and then try again.
Just like, if you were running the cannon, you might shoot at a certain angle, and then adjust the angle of the cannon, and then shoot again.
And then, when you change the angle, you change the ratio of vx to vy initially, right?
So that's called the shooting method-- for good reason.
So the shooting method, that's what we do.
We guess the missing initial conditions, then you solve using the guess.
You solve the ODE-IVP problem.
And then you see what it computes.
And if the solution you get doesn't satisfy the conditions you demand, you go back and adjust your guess.
And you loop around.
So that's the idea.
Now, this thing of adjusting some parameter values, some initial conditions, to try to satisfy some condition, that's like an f solve problem.
That's a system of nonlinear equations kind of problem.
So we're going to have f solve as the outside loop, and that's going to adjust some parameters.
And then, what's f solve do?
So f solve tries to solve problems of the form g of-- I usually write x, but I'll call it v here.
No.
V's not good either.
What letter have we have not used yet?
Any letter.
g of w. There we go.
g of w equals 0.
I want to figure out what w's I need to use.
And actually, v is right.
Because we're trying to adjust the velocities, right?
Sorry, I'm messing up here.
It's good to use the eraser in this class-- especially with this nice big eraser.
All right, so g of e-- we're trying to adjust the velocities to make something 0.
So what's our g that we want?
We really want that the z value at L-- we want that to be equal to 0.
And this thing is, like, implicitly a function of the v's-- the v0 values we give.
If you put the right v0 values in, then z at L would be 0.
So this is what we're really trying to solve.
Now, to get z of L, we have to solve a system of differential equations.
So it's a little complicated to go from v0 to zL, but we can do because we have ode45.
So we can write a function to do this.
Running out of words.
All right so let's try to write the program.
So we want to write the program g. So we want to write z of L is equal to g of v0.
And let's try to think what this function is.
So the meat of this is, we have to solve a differential equation, an IVP.
So somewhere, we have an ode45 call.
And what does ode45 computing?
It's computing the x positions and the z positions.
And it's going to use some function f, which was-- do we still have it on the board?
Maybe I lost it here.
Nope.
Sorry, it's under here.
I'm a terrible board worker.
So this is our function f, f f of y.
So this is really f tilde.
That's the function that I'm going to integrate.
And I can't remember what are the arguments, but it's going to be something like, from 0 to L, and then I have to give it a y0, and then maybe there's some other stuff up there.
All right, do you guys remember the ode45?
Is that in the right order?
I don't remember.
OK.
So I need to specify the y0.
So y0 is going to be equal to the initial value of z, the altitude.
So it's going to be H. And then it's the initial value of vx.
So I don't know what that is, but it's-- maybe you just want to call it v0.
vx0.
And then I do have one more condition here, that vz is related to vx.
So I can use that.
So I can have the last one be the square root of v0 squared minus vx0 squared.
So that's the line for what y0 is.
And then I solve this.
And then I write the number of steps in the time stepping program that did, in order to get here.
It was x stepping in this case.
Num steps is equal to the length of x vec.
And z of L is equal to z vec at N step.
That's it, right?
So that's the function that you would feed to f solve.
And your correct value of vx 0 is the one that makes zL 0, which is what f solve tried to do.
Yeah?
[INAUDIBLE]
You can, but in the physical problem, usually cannons always shoot upwards.
So we'll just shoot upwards.
But yeah, it could be either way.
Yeah, and, in fact, in this problem, you might do better to change-- instead of using vx0 as your unknown, you might want to change to theta, the angle, the elevation angle of the cannon, and that would be more physical of what the actual user of the cannon is using.
They're adjusting theta, not adjusting vx0.
It's no problem, actually.
From the point of view of the f solves, you can make this theta instead.
And then you'd have this be v0 times cos of theta, and this b v0 times sine of theta, and it would work just as well to get the solution.
All right, so that's the shooting method.
So this function g is the g that's going-- you'll have f solve, so is going to say, vx0 is equal to f solve of g. And then you have to have a guess, right?
For f solve.
So that's how you would invoke this function.
All right, and so if we wanted to figure operation count of how many operations this is to find a solution this way, how's it go?
How many iterations does it take to solve an ODE IVP.
That depends on the step size, right?
It's adaptive.
Usually adaptive stepping.
So you have some experience with that, maybe 100 steps?
Something like that?
Is usually enough?
Maybe 1,000 if it's not stiff?
And then how many iterations does it take f solve to solve something?
10?
So you have maybe 1,000 time steps, maybe 10 iterations, means 10 different initial values that you try before you convert to a solution.
And then you may have to actually compute the Jacobian of g, inside f solve, if it's going to use the Newton-Raphson step, it's going to need the Jacobian of g. The Jacobian of g takes the number of unknowns times the number of function evaluations to compute it by finite differences-- maybe 2 times that number.
So it's going to be something like n squared.
It'll be n squared to evaluate the Jacobian.
It takes n operations to evaluate f. So order of n, where n is the number of variables, n times that, to the Jacobian, so it would be like order of number variables squared times 10 times 1,000 is kind of an order of the cost.
And it's only that cost because we can do an explicit solver.
If you had to do an implicit ODE solution, it would be a lot more because you'd have to compute the Jacobians more often.
All right.
Is this fun?
OK.
So this is the shooting method.
You guys should be able to do this no trouble.
Conceptually, this is not so different than the homework problem you did where you did an optimization of the parameter values.
Do you remember?
So you had some differential equations, and you optimize the parameters to try to find the best solutions.
Here, instead of optimizing them, you're doing f solve.
You're trying to solve them to make something happen.
Now how can this run into troubles.
Why would this shooting method not always work?
Ready why this might not work?
Yeah?
[INAUDIBLE]
And a common case where that might happen would be if your ODE system was unstable, Or your ODE solution method was numerically unstable.
In either of those cases, who knows what you're getting.
And actually, you don't know if the solution is even real.
You don't know if the computed final values zL is really what corresponds to that input value of vx0.
Because, if you have a big kind of numerical sensitivity and instability.
So that's one kind of problem.
So this is just like all the ODE IVP problems.
If the ODE's problem, intrinsically, is unstable in a big way, you're in trouble because your errors are going to amplify.
And then also, if you have a problem where the delta t's are too large compared to this numerical stability of the solver, you'll also have a problem.
So those are two kinds of problems that we always have in ODE solutions.
And so, you might actually want to use this method for some regular ODE IVP problems.
So for example, if you had a problem-- I don't want to wreck my beautiful artwork.
You guys know this.
All right.
If you had a problem that said dy dt is equal to 10 to the 6th times y.
Yes?
I have a question about the Jacobian.
When you calculate the Jacobian do you have to solve the ODE system?
Yes.
Well, two ways.
You can either solve it for repeated values, different initial values, of v0, and by finite difference, get the Jacobian.
Or you could use the sensitivity method that we did in the second problem, a problem set ago, and analytically get the derivatives without respect to the parameters either way.
But either way, it's a lot of work.
So the from I guess this is a very simple problem, and suppose you wanted to go all the way out to t equals 1.
Now, the problem is that the solution of this is Ae times 10 to the 6th times t or something.
So this is really not looking so great.
As a problem to solve, this is about as unstable as you can get.
The tiniest little deviation anywhere along the way is going to make the thing blow up.
And so this is not so great and so even if initial condition is that y of t0 is equal to 10 to the minus 12, so actually this is not so bad, this is still hard to solve, numerically.
But you could flip it around, and treat it as a shooting problem, and solve it in reverse.
So you could change variables that u is equal to negative t. And then you'd have dy du is equal to negative 10 to the 6 times y.
And now this is really stable.
And now you might want to integrate in.
So supposed the original question was, what is y of 1 in the differential equations?
You could do it in reverse and say, I'm going to guess, y of 1 is equal to y guess.
And I'm going to integrate it and demand that this is my final condition.
So now this is actually y-- oh geez, now I'm confusing myself.
y of negative 1, I guess.
So I just flipped the sign, t. And to make it easier, I would do it this way.
1 minus t. Same derivative.
So there we go.
y0 is y guess.
And y of 1 is 10 to the minus 12.
So this is my final condition that I'm trying to shoot to, and I'll vary the y guess.
And they're great.
And now I have a perfectly stable differential equation I can solve.
OK.
So this same idea of shooting comes up in multiple applications.
And this whole thing about flipping the change in the variables is often an extremely useful trick to just recast the equations in a way that's more numerically stable to solve, has fewer number of equations, you can write explicitly what you want.
But this takes some cleverness to know how to recast them.
All right, so this motivating example is a good one.
Yeah?
[INAUDIBLE]
You could, but I was just trying to get back in the form of t0 equals zero.
I guess I could have t0 equals negative 1 do it.
I just couldn't do the arithmetic on the board
[INAUDIBLE]
y of 0 is known, but then I have to start from t0 equals negative 1.
So I could do-- so suppose I did u equals negative t dy du is equal to negative 10 to the 6 y.
And I know y of 0 is equal to to-- no, what do I know?
Yeah, y of 0 is equal to 10 to the minus 12.
And y of negative 1 is equal to y yes, and I can integrate from negative 1 over.
So this will t0 if I was using the shooting method, and this would be t final.
Or you could shift it over by 1 if you want, and do it the way I did.
Either one's fine.
Is that all right?
Sorry to confuse all the arithmetic.
I was getting confused too.
All right, now I wrote this-- another situation where this will not work is you can have systems where you have both positive and negative eigenvalues of the Jacobian, and that means that either way you integrate-- in the plus t direction or the negative t direction, either way it's always unstable.
So no matter what you do, you're doomed.
The IVP method is not going to work because you're numerically unstable in both directions.
And so that's really a bummer, and it means our ODE IVP method's not going to work for us, and so then we have to find a different method.
But before I go into that, let's just write the more general version of this.
OK, so this is-- I think you guys all know this.
So let's try a more general version of the ODE IVP problem.
And now we'll allow for the fact that things don't have to be explicit, and they have to be as simple as we thought.
And you can see it even in the problem I just did-- we had a vx squared plus vz squared had to be equal to something.
And so that's not a simple boundary condition like we saw before.
So you get a more complicated conditions, and what you really can have is a set of equations, gn of dy dt y, t, and is equal to 0.
And corresponding, qn of dy dt evaluated at tn, y of tn, tn is equal to 0.
So this is the general ODE system that's written in implicit form, and this has to be true for all t in the domain.
All t that are a member of t0 to tf.
So all the t's that are inside this interval, the domain we are about, this is the differential equation, written as just as general can be, and n is equal to 1 and N in these equations
Is there a reason you switched to q of n for the--
I just want to make it clear that this is the conditions, and this is the general equation.
I don't know.
You can write it anyway you want.
Do you like some variable letters instead of q?
I can change this
No, I was just checking
This one, I'm just trying to emphasize.
This one is only true at tn only.
This equation, this is the condition at one particular time I demand something be true, and this is the general difference equation that has to be true at all times.
If I have n differentials, I need n differential equations, and n initial conditions.
By the way, you can write the DAEs this way as well-- the differential algebraic equations, and then you just won't have as many initial conditions because every time you have an algebraic equation instead of a differential equation, then you don't need the initial condition because the algebraic equation has its own condition.
Does that make sense?
All right, so you can write this in general, and just by depending on how many differentials you have, that's how many conditions you need.
A lot of times, you can write them explicit.
This thing is equivalent to dy dt is equal to f of ty.
Many times, you can solve for dy dt, and write this way, but not always.
OK All right, and the reason why we need to have n conditions is just like when you're doing integrals, you'd have n constants of integration.
Every time you do an integral, you get one, now you have three.
We have n differential equations.
We'll have n conditions.
All right?
Now, a bad thing about these problems, in general, is that the solutions depend a lot on the boundary conditions.
And in fact, you can write down boundary conditions that are not achievable, which means there's no solution.
So this is really a bummer if you don't realize you did it because you could try to solve it all day, and you'll never solve it because it doesn't have a solution.
So for example, in this problem, if I set L to be too large, the cannonball can't go that far.
I can't set L to be 3,000 kilometers and get a solution because the muzzle velocity is not high enough that the cannonball is going to travel 3,000 kilometers.
But when I wrote the problem, you didn't see that, right?
You didn't come and tell me, whoa, watch out.
There's no way L could be bigger than such as such.
And so you might not know it.
You might specify a condition that's just not achievable.
I used to work an industry, this happened a lot.
My manager would specify boundary conditions which were not achievable.
I want 99% yield and a 10% cost reduction.
Go!
And I'm the engineer, I'm trying to work it out, and then sometimes it didn't even come out.
So in some ways, it's a lot better to just say max.
Please maximize this as best you can, subject to your constraints, instead of specifying this.
But of course, I don't know why, there's a school of management that says we weren't definite targets.
We don't want to be just the best, we want to be perfect.
So the no child left behind.
Every single student in this class is going to really achieve.
Anyway, that's the way targets gets set.
It doesn't have to be achievable.
And there's actually a big branch of mathematics about trying to detect if the boundary conditions are indeed consistent so that they can be achieved, and it's not so obvious to do it.
So I'm not going to get into that at all.
You can take a PDE class.
There's a math department, and that's all they talk about all day is about how to figure out are these boundary conditions consistent or not consistent.
You also have boundary conditions that are repetitive, and don't have enough information.
In those cases, you might have an infinite number of solutions.
And I think you can also have cases where you have multiple solutions.
So like in the cannonball problem, if you could have a negative vy, and if the ship was close enough, you might be able to shoot it two ways.
You can shoot it down at it, or you could just blow up high and come down, and they'll both be solutions that satisfy all the conditions.
This is like general non-linear equation problems, that you don't know how many solutions you got, you don't know if you have a solution, you might have an infinite number of solutions, you might have some finite number of solutions, you don't know how many, and it's just that's the way life is, and you'll have to live with it.
But it's kind of annoying sometimes.
For now, let's just assume that we somehow know that we have a unique solution.
There's a unique physical solution.
We're just going to try to find that, and we're not going to worry about this other problem about the fact there could actually be multiple solutions, or no solution, or whatever.
When you actually run into a case like that, then you'll really worry about it a lot.
Is there a question?
No?
OK. All right, so we had an example in actually the zeroth homework, the very beginning, that had a problem like this.
The homework 0P1, we had a problem where it gave you-- Do you remember this?
It was a cylinder, and you were trying to compute t of r. Do you remember this?
And it gave you some conditions about here.
It gave you t when r w was equal to r, and it gave you dt dr-- actually, it gave you some equation.
It was actually a function of this and t, all evaluated at r. It was equal to something.
Do you remember this?
So we gave those two, but then we also told you in the most recent homework, oh, by the way, dt dr has got to be equal to 0 at r equals 0.
So this is one of these ones where we should only have two conditions, but we told you three conditions.
So were any of you able to achieve all three conditions?
No.
OK, so this is highlighting the problem.
So we gave you three boundary conditions that were apparently incompatible, that you couldn't actually solve them all at the same time.
So beware, I guess.
Now, in that problem, it might have been more natural to give you this equation, which we are sure is exactly true by the symmetry of the situation as one of the conditions, and then only one of these other conditions outside.
That whichever one we were more sure was correct, might be the thing to do.
And so we kind of gave you some goofball conditions.
I guess, in some ways.
Yeah?
So if all three of those conditions are true, why wouldn't they converge to a solution?
They could all be true, and if they were all true, then if you solve the numerical exactly, you should get the real solution.
It should satisfy all of them
So does that mean one of them wasn't true in our case?
If we weren't able to do--
Yeah, I think one of them wasn't true, and I think it's maybe due to round off.
But it's highlights a problem.
If you have three conditions that should be true, but they involve numbers in them, then often they won't all be exactly consistent with each other.
But there's actually a worse kind of problem because it's close to being consistent to all of those conditions, but you're not exactly consistent.
But then if you get into it, trying to figure what's going on, a lot of times you'll get near singular matrices and stuff because you have things that are nearly linearly dependent in some way.
So really, it's better if you just have the right number of conditions that you're really sure really specify everything correctly, and just go with those, and treat the ones that you think would be nice if they were true as approximate things that are sort of physical checks to see whether your solution is reasonable.
And we did it sort of like that in the problem you did.
We said well, dt dr should really be equal to 0, but we didn't demand that in your solution.
Your solution didn't actually satisfy that, and we use that as a measure of how far off your solution is.
That's actually pretty useful in practice.
You get a solution.
It's good to have some additional condition that you can test for physical reasonableness of the solution, and particularly because there's a problem that you might have multiple solutions, and maybe one of them is a physical solution, and one of them is some other crazy solution.
And so you need to have some way to detect the crazy solution by some other condition that you can test.
Does that make sense?
All right, so I said you could have cases where the Jacobian of your equation-- so you could be trying to solve this, and it could be, for some problems the Jacobian always has both positive and negative eigenvalues together.
And in that case, using IVP is not going to be a smart idea because no matter which direction you integrate in, you're going to have a positive eigenvalue, and so then the IVP is not good.
So instead, we're going to use a different method, and we'll talk about that next time, and those are called relaxation methods.
And the concept of these is different than it was for shooting.
So the relaxation method is you have a wide guess that you're going to have a guess for the entire y throughout the whole domain.
Actually, a lot of times you might set this up so that y guess actually exactly satisfies the boundary conditions.
So you know it satisfies those equations right from the beginning just from the way you set up y guess, but it won't satisfy the differential equation.
And then what you'll do is you'll say how badly does it fail to satisfy that g is equal to 0?
Because we really want g is equal 0 for the true solution.
And we'll call this failure of our y guess to solve this, we'll call that the residuals.
So we can actually plug in y guess in here, and y guess in here, and evaluate this.
And normally it will not be 0 because our y guess is not perfect, and so it's going to deviate from 0.
But how big it deviates from 0 is going to be our measure of how bad that guess was, and then we're going to try to change y guess to try to make it closer and closer to 0.
So the idea is relax it.
You get an incorrect solution, and you adjust it to relax it to make it become the correct solution.
All right, and so that's the general idea of the relaxation method, and we'll talk about several different forms of that.
Normally, what we have to do is you have to write y guess in terms of some parameters.
So y guess of t depends on some parameters, and we're going adjust the parameters, and that's going to change what y guess is.
And so for example, we could do a basis set expansion as we did in one of the earlier classes.
We talked about how you could expand y guess as a sum of some basis functions weighted by p's, and then you can adjust the p's, and find the best possible set of p's to make this g as close to 0 as possible, for example.
We'll give examples of that next time.
There's several choices about that there.
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to keep talking about boundary value problems.
We'll do that again on Wednesday.
On Friday we'll start partial differential equations.
And next week we'll work with COMSOL, which I hope you guys all have installed on your laptops.
Please check that you have it installed and you can turn it on, your licensing things work and stuff like that.
We'll have a COMSOL tutorial I think on Monday in class.
And you'll use that for one of the homework problems not immediately but coming up.
Speaking of the homework, I received some feedback that the homeworks have been taking an inordinate amount of time.
And so I've discussed this with the other people involved in teaching it and we decided to drastically cut the amount of points given for write-ups to try to encourage you not to spend so much time on that.
And instead I'd really rather you spent the time and did the reading instead of spending an extra hour doing write-up.
You won't get any more points for doing it.
So if you want-- I mean, I love beautiful write-ups.
I'm sure the graders appreciate the beautiful write-ups.
It's good for your thinking to write clearly but it's not worth many, many hours of time.
And in general, the instructions at the beginning of course, are on average over the course, your 14 weeks, it should be about nine hours per week of homework.
So that's maybe 13 hours per assignment, so you have 10 assignments.
And so if it's getting to be 15 hours, you've spent too much time on it and just forget it.
Just draw a line and say I'm done, time ran out, and that's fine.
This is-- getting the last bug out of your Matlab code is not the main objective of this course.
And really, the purpose of homework is to help you learn, not that we need to know what the solution to this problem is.
This is now-- I'm getting the solution from somebody else in the class too so this is-- I don't need it from you.
The TAs probably already did it already and they have the solution too.
So we know the solution.
It's not so essential.
Its purpose is to help you learn and how to figure it out.
And beyond a certain point, it's not that instructive in my experience.
Though sometimes the fifteenth and a half hour you suddenly have the great insight and you learn a lot, but most the time not.
And by the way the reading is in [?
Beer's ?]
textbook pages 258 to 311.
And there's also some nice short readings by Professor [?
Brautz ?]
that have been posted, only a few pages long but definitely worth a look.
Both on [?
PPPs ?]
and also in ODEs and [?
DAEs.
?]
So today what we're going to talk about is relaxation methods.
And we talked last time about the shooting method.
So that's a good method.
At least one famous numerical methods book recommends you always shoot first then relax.
So try the shooting method.
If it works, you're done.
If it doesn't work for you, then you may have to use a relaxation method.
And we'll talk a bit now about the relaxation methods.
And the general idea is you're going to write your y-- which is an approximation, it's not going to be the real solution-- and you're going to try to write that typically as an expansion to some basis functions.
So let's say the nth component of y.
And then you're going to vary these coefficients, the d's, to try to make your solution as good as possible.
And now we're talking about different definitions of what good is for the solution for a problem.
And you have to be aware that almost always, this will not exactly solve the differential equation system.
So it's always going to be wrong everywhere, typically.
And you can get to decide sort of-- if you wanted to be good at some particular spot, you can do something about that.
If you wanted to, on average, be good in some way, you can decide that.
And it's sort of how much error you can tolerate.
And in general, you have to do a finite sum of a finite basis set.
For many, many ODE, [?
PPP ?]
problems, there's math proofs that if-- and the limit is this goes to infinity if you add an infinite number of functions here.
You can always make it work to get the true solution.
But you can never afford that so you're always finite truncated basis set is what you're using.
And so a lot of the accuracy things have to do with exactly what functions you choose and exactly how many of these terms and sum you include.
But typically, these problems will start from the beginning, you'll say I'm only going to include so many, how big my computer is.
And so you're going to be stuck with some error, so that's just the way this.
Let's recall the problem we have.
If we write it in the-- as a first order ODE system, it's dy/dt-- well, there's some function of dy/dt and y and t that's equal to zero which often can be written something like this, dy/dt is equal to an f [INAUDIBLE]..
Right?
And so if you plug in this approximation for y, then you'll get something here that will be some g of t that's generally not going to equal zero, but you would like it to be zero.
If you put the true solution in, you would get zero.
If you put your approximate solution, it is not going to be zero.
So you'll get g of t and you want this equal to zero for all t. And then in addition, you have boundary conditions.
And so you have boundary conditions on the solution and you want those things to be satisfied.
And again, you can write them in a form that something has to be zero.
Often, you'll exactly satisfy them.
So you'll choose your solution to make sure it satisfies the boundary conditions and it won't satisfy in the domain.
It won't really satisfy the differential equation.
That's the most common thing to do.
But it could also be that it doesn't satisfy the boundary conditions either, but you want to be close to satisfy the boundary conditions.
We have to think of how are we going to judge if the solution is accurate or not, if our approximate solution is good?
And from the way we wrote it, we have our parameters, d. These are numbers we can adjust.
And we're going to try to adjust them to make the solution as good as possible.
And now we just define what a good means.
There are several definitions of good that are widely used.
And one of them is called collocation.
This is like option number one.
And that is you choose a set of ts, of particular time points.
And for those particular type points you demand that g of the time point is equal to zero.
So you're forcing the residuals-- this is called the residual, it's the error.
And you're forcing the error to be zero at some particular time points.
And generally between the time points it will not be equal to zero.
But you can pick your time points and depending on which ones you pick, you'll get a slightly different optimal choice of the d's.
Because the d's will be adjusted to force the residual to be zero at your time points you pick.
So that's one option.
Another one is called Rayleigh-Ritz.
And that one is you minimize overall your d's.
The integral in t zero to t final of the norm of g of t. So this means you try to make the average of the square of that deviation to be as small as possible.
So it's like sort of like a least squares fit kind of thing.
So that's another option.
And then a third one that people use a lot is called Galerkin's Method.
And his method is you choose some functions-- some of your basis functions typically-- and you integrate them with each of the elements of the residual.
And you demand that that has to be equal to zero.
Yeah?
[INAUDIBLE] for [?
this method, ?]
what do you [INAUDIBLE]?
The d's, your coefficients.
And these ones, I didn't write it out but this g depends implicitly on the d's, so you can write it that way.
A lot of d's.
And so you optimize the d's, you very the d's to force g to be zero at certain ts.
And this one also, the g depends on d's.
And so you're going to optimize the d's to force this integral equation to be satisfied.
Yeah?
[INAUDIBLE] all changing d's?
Yeah.
They're all changing d's.
And it's just trying to-- your criterion, your error measure, how do you measure or what do you think is good?
You want to make something small, some kind of error small, but you have to figure out what are you going to define your error to be.
And you'll get different solutions depending on which error measure you use.
Is this OK?
Now this one's pretty straightforward.
I'm just going to write it down a bunch of algebraic equations that depend on my d's and then I'm going to solve them.
And this looks a lot like [?
an f ?]
[?
solve ?]
problem.
This is a Newton problem or something like that.
So it's just some algebraic equations.
Should be OK?
So that one you should be pretty well set up to do right now.
Let's just think of how many equations there are.
So we have-- say we have our basis set yf t dni fit.
That's our basis set.
And this is a sum over i equals one to say the number of basis functions.
What do you want to call that?
N, sound good?
So we have n basis functions.
And so we have how many unknowns here?
Maybe n's not good because n's this one.
Let's call it something else.
k, OK. All right?
In fact, I'll even change this to k too just to make life easier.
So there are how many dnk's are there?
There's n where n is the dimension, the number of od's or the number of components in y times k. That's how many d's we got that we're going to adjust.
We want to have an equal number of equations and unknowns.
We have to have as many equations as that.
So what equations do we got?
How many equations?
So if we just have a ODE [?
BBP ?]
we typically have n boundary conditions.
So that many boundary conditions because we need one boundary condition for each differential equation, right?
One integration constant for each one.
So we took an n boundary conditions.
And then we have-- in collocation, we have however many capital M time points we chose.
So we have M n equations-- no, that's not right.
We have an equation like that for every component of g. So it's M times n equations from the collocation.
Is that all right?
So just looking at this, it looks like we have n times M plus one equations and we have n times k unknowns that we're trying to adjust.
And so therefore, this says we should choose k to be equal to M plus one.
So if we choose we want to say we want 100 basis functions, then we need 99 time points to do collocations at it in order to exactly determine everything.
Is that OK?
How many people think this is OK?
OK.
He agrees.
It's OK.
The rest of you, no opinion.
This is like the American political system.
Only 5% people vote.
Everybody else just listens to Donald Trump.
So this is how many equations we need.
Now sometimes people will choose the basis functions so that, say, some of the boundary condition equations might be satisfied automatically.
And I'll talk a little bit in a minute about cleverness in choosing basis functions.
So one possible thing is you can try to cleverly choose basis functions so that no matter what values of d's you choose, you're always going to satisfy some of the boundary conditions.
And so long as you don't get a full value of n because some of these boundary conditions don't help you determine the d's, any values of d's will work.
And so in those cases, you need a few-- might need another time point or something.
Get some more equations.
So that's collocation.
And I think this should be perfectly straightforward.
You just evaluate your y's.
You need to have your dy/dts.
You'll need them, because they appear in g as well.
And they're just going to be the summation over k [?
of ?]
[?
dnk ?]
v prime k of t. And so you choose basis functions that the analytical derivatives of.
So now you know these answers.
And now you can evaluate this at any time point tm.
So you evaluate your time points.
You evaluate these guys at your time points.
You plug them all into your g expression over here, and your force is equal to zero, by varying the d's, right?
No problem.
So that's collocation.
That's pretty easy.
And because it's so easy, it's kind of pretty widely used as one way to go.
All it requires is that you have to know the derivatives of your basis functions.
In particular, doesn't require any integrals.
When you see the other methods, they're going to involve integrals.
So we'll have to figure out how we're going evaluate those.
So if you have functions you don't know how to integrate then this is definitely the way to go, to do collocation.
And if you use collocation with enough points, you're forcing the error to be zero a lot of points, then probably it won't get that big in between the points.
At least you can hope that I won't get that big between the points.
And you can try it with different numbers of points.
And different size numbers of base functions and see what you can do.
See if it converges to something, if you're lucky.
All right.
So that's one way to go.
And it's just f solve problem.
And all that's happening is it's just forming a Dracovian Matrix inside there.
Now, you still have to choose which basis functions you want.
And there's a lot of basis functions you know that you know the derivatives of.
So there's some issues here about what choice is best.
And we'll talk a little about that in a minute.
One thing you have to watch out for, is you don't want your basis functions to be linearly dependent because then you'll end up with indeterminate values a d. Different sets of d's will give you exactly the same y.
If these phi's-- if one of these phi's is a linear combination of other phi's in your set, then you could have different values of d's that would actually correspond to exactly the same y.
Because of that, when you a Jacobian Matrix as part of it your Newton solve, the Jacobian's going to be singular, and then the whole thing's not going to work.
So that's one thing to watch out for.
And actually, it doesn't really matter if the functions are orthogonal in the sense of being functions orthogonal, it's really whether the vector, vk evaluated tm, if those are independent of each other or not.
I feel like we talked about this before, one time, yes?
Yes.
All right.
So anyway, as long as the vk evaluate of tm, if those things are vectors which are not linerally dependent on each other, this should work fine.
OK?
All right.
Now, really [INAUDIBLE].
This one here often gets to be kind of messy.
Part of it is that inside this norm of this vector, it's square, so you end up getting squares of your d's.
So, the whole thing is definitely non-linear in d to start with.
And then you have to be able to evaluate all the integrals that come up.
So this is really sum over n, of g and of td squared that you're trying to integrate.
[INAUDIBLE] Right?
So, you have a lot of these function squared.
You have to add them up.
You could do it, but the algebra gets a little complicated.
So, this is not so commonly done unless the g has some special form that makes the algebra a little easier.
But there's no reason you can't do it in principle, but it just is a little bit of a mess because you get a lot of integrals of cross terms between the g's.
And in the d's inside there.
And so, not so commonly done.
Though one case where it is done a lot is in actually quantum chemistry.
Methods called variational methods in quantum chemistry use real [INAUDIBLE] to figure out the coefficients, and like your orbitals, and stuff like that.
So, it'd be certain methods.
But they've actually gone out of favor recently.
So, you probably won't use it that often.
But in the past, that was a big deal.
Galerkin is used the most of anything, so let's talk about that one for a minute.
Now, the concept, where does this equation come from?
I guess that's one thing.
So maybe we can back up and say, well, where did the collocation come from?
Collocation is actually like a special version of this where I choose, instead of these basis functions, I use delta functions.
So, if I integrated with delta functions, t minus tm.
Another way to look at the collocation is demanding that the integral of delta function t minus tm g n a t dt is equal to zero.
OK.
So, now instead of using delta functions, I'm using these functions.
Basis functions.
So I don't know if it's particularly obvious why you'd use one or the other, but anyway, you have a choice.
And any time you do this kind of integral with some number of functions, you get some number of equations that can help you determine the d's.
This particular choice of the basis functions-- one way to look at it is, you can say, well, suppose my solution g g n, suppose I could write this as a sum of some different expansion coefficients.
So I can relate this plus OK, so, there's two terms.
One is the expansion of the residuals in the basis that I'm using, and one is all the rest going on to infinity of all the other basis functions in the universe.
And if I think my basis set is very complete, that it kind of cover all the kinds of functions I'm ever going to deal with, both in y and in g, in the residual, then this would be a reasonable thing, and you might expect that this term-- if you'd pick big K big enough, this might be small.
So that's sort of where the idea is.
So then, if I think this is small, then I want to make sure I make this part as close to accurate as possible.
And this condition is basically doing that.
It's saying that I want the [?
error ?]
to be orthogonal to the basis functions, in the sense that, for two functions, the integral of the two functions like this, is like an inner product, just like the inner product between two vectors.
So I'm saying, like, in the vector space, in the function space that I'm working in, I don't want to have any error.
That's what this is saying.
But I'm going to have is that there's other g terms which are the rest over here.
These guys will not be orthogonal to the first part.
Does that make sense?
So there's still some error left, in my g. But the part in here, I can make a good [?
answer.
?]
So that's where this equation comes from conceptually.
All right.
And we'll come back-- when we do least square [?
setting, ?]
we'll come back to that same idea.
So this is what the equation looks like, and the disadvantage of this one is it involves some integrals.
And so you have to be able to evaluate the integrals.
All right.
So that's the downside of this function.
So cleverly choosing using your basis functions to make it easy to evaluate the integrals is like the key thing to make this a good method.
And just like we needed analytical expressions for the derivatives here, now we need analytical expressions for integrals.
That we're going to get from this guy.
OK.
So, let's think about what basis functions we can choose that might make it easier to evaluate the integrals.
Suppose we can write g explicitly.
Like, this.
So, g is equal to d gx dyn/dt minus f/n.
Suppose.
OK?
Then what integrals am I going to get?
Well, dyn/dt is the sum of the derivatives of the basis functions, and y is just this sum up there.
Some of the basis functions.
And so, I'm going to have integrals that look like this is like I have an integral phi j summation d phi prime.
No, k. t. dt.
That will be the integrals from the first term.
And then I'll have minus some integrals from the second term here.
So, phi j fn ft y, where y is the sum-- it's actually like a matrix.
All right.
Because this y is a vector.
It's all the yns.
And so, the whole y vector is d times the phi vector.
All right.
Where d's that big matrix.
The elements are [?
dnk.
?]
And so, if you cut these integrals, this thing here is a summation of [?
dnk.
?]
The integral of phi j. Phi k prime.
And so, if you choose your basis functions cleverly, you might be able to know all these integrals analytically.
OK?
But if you don't choose them cleverly, who knows what kind of horrible mess you'll end up here.
All right.
Ideally, you really want to get these all analytically so you don't have to do numerical integration.
As a loop inside, all the rest of the work, you're going to do in this problem.
OK?
And then, these ones also, now knowing something about this is really important or you may have to do the numerical integration here, because this could be really a mess.
You have a lot of phis inside some function, which could be a non-linear function of these guys.
And so, in principle this could be really horrible.
So you may have to do numerical quadrature for those guys.
All right.
OK, and so, if you do Galerkin's method, a big part of that is thinking ahead of time, oh, what basis function am I going to use?
How can I make a basis function so I can evaluate the integrals easily, and then I might have a chance to do it?
All right, questions so far?
Phi j
Phi j.
Sorry, phi j of t. Here, I'll get rid of the t's.
These are both functions of t. You're integrating over t. And these integrals from t0 to to t phi.
Your domain.
All right.
Now in Galerkin's method, in addition to these integral equations, I still have the integrals, the equations, that the boundary conditions.
So I still have some equations that look like collocation equations that are evaluated at tm.
Do I have that anywhere?
Nowhere.
I have an equation like this, except it's not g, it's q, it's the boundary condition equations have to be true at the boundary conditions.
Right?
So they're the same as before, q [?
or just ?]
before qn of dy/dt evaluated at tn y of tn.
tn.
This is equal to 0.
This is sort of the general way to write a boundary condition.
And so there'll be some special ends, the boundaries, where I want to have extra conditions.
I'll get some equations like this.
These have to be satisfied in Galerkin's method as well and they will not be integrals.
They are just that, tn.
And so in addition to the integral equations, that you have to solve over here, you want these things to be zero and you also want to satisfy the boundary conditions.
So then, all these methods, a big part of it gets to be cleverness of a basis function, a choice of basis functions.
And there's kind of two families of approaches.
So, one family is global basis functions.
So basis functions that are defined on the whole domain.
And there are special ones, sines and cosines, Bessel functions, all these functions inferred from your classes, all the special functions.
And a lot of them, the integrals are known for a lot of cases.
There might be special tricks that make the integrals easy to evaluate.
They can satisfy the boundary conditions automatically.
And some of them, for example, many of the problems we have with like the heat flow equation, so you have-- actually it won't be d. I'm probably going to get this wrong.
It's, like, kappa or alpha.
Which one is it?
Alpha.
Alpha d square of td, x squared minus, there's some source of heat that might depend on the temperature, and this has to be, say, equal to zero.
Right, does this seem what you've seen in classes before?
So this is a common one.
So, in this case, you might try t to be a sum of, say, sines.
So [?
dnk ?]
sine of k something something something.
In there, and the cleverness of this is that d squared phi k dx squared is going to be equal to some number times phi k. Because the second derivatives of sines are also sines.
Right?
So all your differentials will solve and in fact, the derivatives will be really simple.
Now, the q terms could still be a horrible mess.
For example, this could be an Arrhenius thing where's the t's up in the exponent.
And then the whole thing might be horrible.
Anyway.
But at least the differential part is, like, super easy.
OK. And you might have a boundary condition say, at one end, that dt/dx at someplace like d final x final is equal to zero.
Right?
That might be like, you're up against a x insulator.
Or something like that.
So, you don't have a heat flow.
So that would be a boundary condition you might have.
And then, by cleverly choosing your definition of the sines, you could force that all the sines satisfy this derivative condition.
OK, so, rescale your coordinates so that ends up with pi over 2, an all sides of everything and pi over two is always zero, the derivative, so you're good.
So, you can do some clever trickiness to try to make the problem easier to solve.
And so, that's one whole branch of these basis function methods, is clever basis functions to match the special problem you have.
OK?
So that's one option.
Then, the other option is the non-clever approach, where you just say, well, I've got a computer.
Who cares about being clever?
Let's just brute force it.
All right.
And instead, you just want to write a general method, any problem you can solve.
And this is more or less what COMSOL does.
And you're going to do it.
And so the distinction here is that this kind of thing, this sine function, has a value everywhere, all across the domain.
So this is kind of like a global function.
OK?
And the alternative is to do local basis.
Try to have basis functions that are only defined in little tiny areas.
And then, at least when I integrate the integrals, I don't have to integrate over the whole range.
I don't have to have to integrate right around my little basis function.
So, this global basis function-- this is sort of similar to what we're doing, interpolation?
Do you remember you could use high order polynomial to interpolate, or you could alternatively do a little piecewise interpolations, say, with straight lines between your points.
And, you know, it's not so clear, actually, which would be the best way to do it.
Or maybe you want to do some combination.
Do little parabolas between little triples of points, or something.
That might be a good interpolation procedure.
It's the same thing here.
Some problems, you can find a global basis set that works great and you should use it.
In other problems you might do better to just break up the domain in little tiny pieces and then do simple little polynomials or something in those little domains.
All right.
So this is the global basis.
Let's see a local basis.
It's OK to delete this?
So, local basis functions.
A really common choice for these guys are the b-splines.
And, in particular, first order of b-splines and these functions have the shape phi phi k, it looks like this.
So, at zero, all the way up to ti minus 1.
Then it goes up to one at ti, and then it goes down to zero again, and goes out.
OK.
So this function is a b-spline.
It's also called a tent function because it looks like a tent.
Some people call it a hat function.
I don't have a pointy head so it doesn't look like a hat to me, but they call it a hat function so I guess they must have hats like that.
And this is a very common basis set.
And the nice thing about this basis function is, it's zero except in this little tiny domain around it.
OK?
And it's the only basis function in the whole set that has a non-zero value of ti.
And so if you want the function to equal something in ti, it's going to be equal to the coefficient of this basis function, right?
Because we're going to write y n of t is equal to summation [?
dnk ?]
phi k of t. And so, if I really care about yn evaluated at ti, the only one basis function this whole sum is going to have a non-zero value there.
So that's going to be equal to dn dnk, k where this is the special k. k prime matches up to ti.
OK, so there's one base function that looks like this.
There's another one over here that's the one base center on this point, and there's another one over here.
So on this point, it's [INAUDIBLE].. And I have as many basis functions as I have points.
So this is like a way I discretize the problem, but I've kept my solution as continuous function because my y-- that's the sum of these guys-- has a value everywhere.
It's a continuous function.
The way this is written it looks like it might not be differentiable at all the points because it has all these kinks, but there's a clever trick you can do to deal with the kinks.
So, actually, it's not a problem
So, for these basis functions, how would you define if you had the boundaries?
Like t0?
So, you'll have a basis function at the very end.
Suppose this is t0 here.
You have one like that.
OK, so it's just a half of a tent.
Can you get a [?
sparse ?]
d-matrix?
Yes.
So locality is really good because it makes the Jacobian matrix sparse, the overall problem.
So, when I compute the integrals of this thing, for example, the integral of phi i of t, phi i minus 1 of t dt, this turns out to be equal to 1 or 2 times ti minus ti [INAUDIBLE].
No.
I take that back.
Just one half.
Just half.
So it is like a brilliant thing.
Is that right?
It does have a ti [INAUDIBLE],, not a [INAUDIBLE]..
I'll have to double check.
Possibly including delta t. I can't remember.
All right.
But the integrals are very analytical, and only certain ones are non-zero.
So only one that the two is when the two is differ by one unit, do they have a non-zero integral?
All the rest of them are non-zeros.
So, when I write down these equations, I mean, many, many, many, many zeros, so it looks horrible when I write Galerkin's method, I get so many integrals, but actually a zillion of them are zero, and then there's a bunch of special tricks I can do that make it even better than that.
OK?
So then, the Jacobian sparse.
So, that'll save you a lot of time in linear algebra, which allows you use a lot of points.
So, you can use a very large basis set because you end up with sparse Jacobians, I mean, it's not going to fill your memory storing all the elements in the Jacobian, even if the number of points is very large.
And also, there's vast numerical solution methods for those.
So that's the idea of this.
I guess should we try to carry one of these out?
You guys are [INAUDIBLE] trying to do a lot of algebra on the board?
Do you think I can do the algebra on the board?
That's the real question.
All right.
Should I do it for collocation, or you guys confident you can do collocation?
You're all right with collocation?
You're fine with collocation.
Great.
OK.
So we'll just go right into Galerkin.
So, Galerkin.
That way, we use a local basis.
So, most of the equations we have to solve are this type.
Phi j t times the residual function of summation [?
dnk ?]
phi k prime.
Summation [?
dnk ?]
of phi k and t. Is equal to zero.
All right, we have a lot of equations like that.
We're trying to find the d's that are going to force all these integrals to be zero.
OK?
So we have a lot of different j's we're going to try.
We want this to be true for all the n's.
And then we're going to adjust these [?
dnk's ?]
to try to force this to be zero.
All right?
That's the main problem.
And then, on top of this, there's some equations with boundary conditions.
So, now we have to look at what the form is.
If the form is, oh, I erased it, if I can make this explicit in the derivatives, which I can do very often, for example, this could be dyn/dt minus [?
fm.
?]
So then, dyn/dt is just this.
And then I'll have another term, [?
fm, ?]
which will depend on that [INAUDIBLE].
And so the integral phi j, and I'll just remind you that phi j is not equal to zero.
If tj minus 1 is less than tj plus 1.
All right.
That's the only places where my local basis function is non-zero.
That's where the tent is.
And all the rest of it's zero.
So when I have this integral, originally have it t0 to t-final, but actually I could replace this with tj minus 1 to tj.
And it's just the same.
Because of all the integral outside that domain is zero.
Because this function is zero.
All right?
So at least I have a small little domain to do the integral over
[INAUDIBLE] tj?
tj plus 1.
Thank you.
Yes.
Yes.
Plus 1.
Yes.
Is that all right?
OK.
So, I have this integral, and then I have this times the derivative term, so it's dnk dk prime minus the same integral, j minus 1.
J plus 1.
Phi j sorry [?
fm ?]
[INAUDIBLE] summation of dn.
k phi.
k [INAUDIBLE] t. OK. Now, this derivative of phi k, well, I just told you what it was.
It's just like the [INAUDIBLE] set functions.
So, this integral, you'll know analytically.
Like, it's either 0 or it's 1/2.
Maybe minus 1/2.
Whatever, but it's nothing complicated, and you'll just know it right off the bat.
So this is all that can be known.
And sparse.
It's only going to be when k is equal to j minus 1 or j plus 1 that this will be non-zero, and all the rest of it would be zero.
OK?
So, that's mostly zeros.
This one over here, in principle, I have quite a huge sum here, k equals 1 to k, where I have all my basis functions, which is all my points where I put my little tents down.
So I have a domain and I've parked a lot of tents all on the domain, that's my basis functions.
And I'm trying to figure out, sort of, how high the tent poles are.
On all those tents.
And my tunnel functions, the sum of all the heights of those tents.
OK?
But this guy is only non-zero in this little domain, and these guys, most of them are zero in that domain, because they're mostly tents that are far away from where my special tent phi j is.
OK, so I'm going to draw a picture.
Here's the domain from t0 to t-final.
Over here is my tent corresponding to phi j, which is centered around tj.
Hence, this function.
And then, these guys are all the other tents.
There's a tent here, there's a tent like this, there's another tent like this, another tent like this.
All these tents.
Those are all the basis functions starting from k equals 1 and going up.
All of these guys are zero in this domain because they all have zero tail.
So, almost all of these, the f's doesn't really pick up anything from any of those guys in the domain I care about.
The only domain I care about is this domain right here.
And this whole sum is all zero except for a few special k's when k is equal to j minus 1, j, or j plus 1.
So I only have to worry about three terms.
The inside contributing to the y in the region of t that I care about.
All right?
So, when I want to compute the Jacobian with respect to d, the only terms here where the Jacobian's going to be non-zero are for d I have a Jacobian, which is the derivative of this whole thing, ddd nk.
Right?
And that's only non-zero if k is equal to j minus 1 j, or j plus 1.
You guys see that?
How many people do not see this?
How many people are lying?
This is a really important concept for the locality, so this is the advantage of a local basis, that the Jacobian will turn out to be really sparse because it's only non-zero in this special case.
And you might have 1,000 points, 1,000 of these little tent, 1,000 basis functions in the sum.
But only three of them are non-zero.
For four each n. So, actually, it's 3 times n. [INAUDIBLE] non-zero.
Is that all right?
Because it just depends on the k. Three of the ks are non-zero.
There's 1,000 of these guys [INAUDIBLE].. Yep.
So that's the big trick of locality.
And then, also, because these integration ranges are so small, because you choose your points really finely spaced, then you might get away with simple polynomial expansions of f, for example.
Around the points, or [INAUDIBLE] expansions, there are all kinds of little tricks.
Or you could even do quadrature and determine some points, do a [INAUDIBLE] quadrature to evaluate the integral.
But you only need a few points, because you know it's a little tiny range of dt.
And you would hope that you've chosen so many fees, so many time points, that your function doesn't change much from one time point to the next.
So, more or less, first order is constant.
Your function's constant with respect to t in this little tiny domain, and then maybe has a little slope.
And then, if you're really being fancy, you might be able to put a parabola on it.
But it's not going to change that much.
If it's changing a lot, that's telling you you don't have enough time points and you should go back and put some more basis functions in.
And then you can get a good [INAUDIBLE].. Because what we're, really doing here is we're using this basis set.
If I add up these guys, what I'm really doing is piecewise linear interpolation between all the points.
So I'm approximating my y versus time.
It's going to look like this.
All right.
It's piecewise linear, because that's the only thing you can make from adding up a bunch of straight line segments.
A bunch of tents is a bunch of straight lines.
And so this is what the approximate function is.
Well, you can see, this is really bad if these functions are too different from each other.
But if they're all like this-- and you think, well, maybe it's not so bad to use piecewise linear.
Yeah
[INAUDIBLE] confused at where we're using [INAUDIBLE]
Inside [?
f ?]
solve, what is trying to solve for the d's, it's solving each of this giant set of the equations, a whole lot of equations like this, they all come in to a gigantic f. That's the f of d that need to make equal to to zero.
And so, I'm [?
burying ?]
the d's, I need the Jacobian of that gigantic f. Now, the problem, because it's gigantic, I need a lot of points, a lot of closely spaced points to make my function look smooth.
That means the number of base functions is really huge.
So the Jacobian principle is huge number squared.
I have 1,000 points, say, discretizing my domain.
And then it's 1,000 by 1,000 Jacobian.
It might be really hard to solve it, or cost a lot of CPU time, or use a lot of memory.
But, fortunately, almost all the elements are zero.
So, it's like, has a sparsity of 0.3% or something is the occupancy.
So, it's almost all completely sparse, and therefore you can solve it even though it's gigantic.
Yes
[INAUDIBLE] basis function [INAUDIBLE] the Jacobian [INAUDIBLE] should be [INAUDIBLE]
We're trying to find d, so we're really trying to solve a problem that's f of d is equal to zero.
That's our fundamental problem we're trying to solve.
We're doing Galerkin.
We have something equal to zero, and it depends on d. And we're trying to find the d's.
So this is really the function we're trying to solve.
In order to evaluate the elements in this, we have to compute a whole bunch of integrals with the Galerkin method.
So that's the complexity.
But the basis functions, we know what they are.
We've pre-specified them.
So, the whole question is just what the d's are.
And the d's get multiplied by a whole lot of integrals.
Is this all right?
I think I may have misunderstood the question.
So you have f of d is [INAUDIBLE],, so therefore we probably want to know j with respect to d. OK. Time ran out before clarity was achieved.
We'll try to achieve clarity on Wednesday morning.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to talk about partial differential equations.
Let me just give a quick review where we are.
So we're going to have a PDE lecture today.
Then Professor Swan is going to do a review on Friday.
There's no lecture on Monday, but you have the quiz, too, Monday night.
Wednesday, next week is Veterans Day so it's a holiday.
There's no classes at MIT.
So the next time I'll be lecturing to you will be a week from Friday.
The homework following the quiz will be posted and so you guys can get started on that.
That homework involves a COMSOL problem which, for those of you who had trouble with COMSOL, might take some extra time, since it's the first time you're doing it.
So I would suggest you at least give it a try early.
I'm sure the TAs would be very happy to help you to force COMSOL to cooperate.
Bend COMSOL to your will.
All right, so we're going to talk about partial differential equations.
And, as it sounds, what's different about a partial differential equation is, it has partial derivatives in it.
And so we have a lot of equations like this.
So for example, I work a lot on this equation.
It's a poor choice, isn't it?
OK, so this is the conservation equation for a chemical species in a reacting flow.
So the change in the mass fraction of the species z, at a point, xyz at time, t. That differential is given by a diffusion term, a convection term, and a reaction term.
So this is a pretty famous equation.
This is part of the Navier-Stokes equations for the whole reactive flow system, which I hope you guys have seen already, and for sure you will see, if you haven't seen them yet.
And there's a lot of other partial differential equations.
I work a lot, also, on the Schrodinger equation.
That's another partial differential equation.
And what's special about the partial differential equations is that, in this case, this partial derivative is respect to time, holding all the spatial coordinates fixed.
And these partial derivatives are respect to space, holding time fixed.
A very important thing about partial derivatives, which you probably have encountered in 1040 already, is, it really depends what you hold fixed.
And you get different results if you hold different things fixed.
Now the implication, when you write an equation like this, is that whatever partials you see-- When I wrote this, if I have this and it has a term that's like d squared, dx squared, is one of the terms in this thing.
This says t. This says x.
The convention is, oh boy, I better hold x fixed.
And here must be, I must hold t fixed.
Because there's another one in the same equation.
Now, you can change coordinates however you want.
Just really watch out that you carefully follow the rules about what you hold fixed, when you change things.
Otherwise you can cause all kinds of craziness.
Now you guys probably took multivariable calculus at some point in your distant past.
And they told you very carefully what the rules were.
So follow them.
And I suppose, when you're doing thermo right now?
Have you started to do all this partial derivative stuff?
Yeah, so you've seen how completely confusing it can be with negative signs showing up all over the place.
And other things like this.
I don't know if you've encountered this yet?
At least, I thought it was confusing when I took it.
So just be aware.
It's the same thing.
Now, often you do want to change the equations.
So you can write down an equation like this.
But, for example, if you have a special symmetry of the problem, like it's cylindrical, for example, then you might want to change the cylindrical coordinates.
And then you have to just be careful that you write that correct Laplacian and cylindrical coordinates.
Make sure it doesn't mess up anything about what you're holding fixed.
Sometimes you might want to be crafty and use some weirdo coordinates.
For example, if you wanted to solve the Schrodinger equation for h2 plus, it turns out there's a special coordinate system called the elliptic coordinate system.
And in that coordinate system, the partial differential equation is separable.
And so, if you do it in that coordinate system, you can actually solve analytical solution, which is pretty unusual for the Schroedinger equation.
But it's a pretty goofball coordinate system and you really have to watch out, when you convert, to make sure you do it just right.
But it's just the rules, chain rule and the rules about how to keep track of what's being held constant.
And when you change, will tell them, how, being held constant, how to change it.
Now in principle, this problem is like no different than the ODE BBP problems we were just doing.
You just have a few more terms.
And so, you can use finite element.
You can use finite differences.
You can use basis set methods.
And it's really no different.
So all those methods basically look the same.
You just get equations that have more unknowns in them.
Because, for example, if you're using finite element methods or basis set methods with a local basis set, you need to put points both in the x direction and the y direction, and in this case, the z direction, and then maybe also in the time direction.
So now you have a lot of mesh points in all different directions.
And so you get a lot of mesh points.
And so fundamentally, there's no problem, but in practice, there's a big problem if the number of unknowns gets to be so large.
That's the main problem.
So think about it.
You have some kind of equation like this.
It's in three spatial dimensions and in time.
How many points do you think you need to discretize your solution in the spatial coordinate?
There might be somewhere, might be 100, I don't know, points might be enough.
And so now I have x and y and z.
So I'll need, well, just in COMSOL, you saw this on Monday.
Suppose I just have two dimensions.
Suppose I just have x and z.
And I start putting points in.
Well, I want to have some points along this way.
And I want to have some points along this way.
And so, I actually have an unknown here and an unknown here, and one there and one there, and one there and one there.
There's quite a few of them.
I think of how many points I have.
Each of those is a point where I have a little basis function.
And each one of those has a magnitude of how much weight I have on that basis function, things they called the d's before in the basis functions.
And I have a lot of them.
Now, it's going to be, if I have 100 points this way and 100 points this way, I have 10,000 of these little points.
That means I have 10,000 unknowns I have to solve for.
10,000 unknowns are a lot harder to solve for than 100 unknowns.
and if you saw, on some of the problems we did with the ODE BBP problems, even with 100 unknowns, we were having a lot of trouble, sometimes, to get the real solution.
All right?
So you can just see how it just gets to be, like, a big numerical problem.
So that's two dimensions.
The third dimension, you'll get another 100 times as many points.
You might have a million unknowns.
And once you get up to a million unknowns, we're starting to talk serious math, now.
OK?
Even for your good computers that you have.
In addition, it's not just you have the number of mesh points, but it's the number of mesh points times the number of variables at each mesh point.
So in the Navier-Stokes equations, what variables do you have?
[INAUDIBLE]
Louder
[INAUDIBLE]
Velocity and pressure, OK.
So you have pressure.
We have the different components of velocity.
What else we got?
[INAUDIBLE]
Sorry, what?
[INAUDIBLE]
Yeah, though, yeah.
Density or temperature.
Yeah, temperature maybe, so.
So maybe temperature would be what you might use.
And then you'd have a state function maybe with these guys.
What else would you have?
So then you'd have all these species, right,?
The mesh fractions are all the species.
So you have, like, y, species one.
Mesh fraction, species two.
However many you got.
So that's a pretty big set of variables and you need to know all these numbers at each point.
So this might be-- how many have we've got?
One, two, three, four, five, plus however many species we have?
So this is n species plus 5 times the mesh.
And I told you the mesh was what, a million?
So maybe I have 10 million unknowns.
So the issue, to a large extent, has to do with just, can I solve it?
Can I solve a system of 10 million unknowns?
10 million equations, 10 million unknowns.
All right, so that's going to be one big issue.
Now the other issues, we still have the same problems we had in ordinary differential equations, that we have to worry about numerical stability.
We have to worry about actually, is the whole problem even stable to begin with?
Physically?
Because if it's not stable physically, then probably we're not going to get a stable solution.
So you have all those kinds of problems that we had before in the ODE case, and now it's just sort of amplified by the fact, now we have boundary conditions in more than one dimension.
So we have multiple chances to screw it up.
And we're trying to figure out how to set the boundary conditions correctly.
And we have to worry about stability in all the different directions.
So those are the different kinds of things we have to worry about.
But it's all the same problems you had before, just amplified.
Now, because of this problem of the very large number of unknowns, currently, if you have a problem that has two dimensions, you can kind of very routinely solve it.
And things like COMSOL on your laptop, you can usually do those.
When you get up to three dimensions, you might be able to solve it or might not be able to solve it.
And it could be, like, a really big challenge sometimes to solve it.
And then in reality, we have problems like the Navier-Stokes is four dimensions, right?
There's time, as well.
So then that gets to be really hard.
And then the Schrodinger equation has three times the number of electrons dimensions in it.
So that gets impossibly hard.
And so, specialized methods are developed to handle those systems with a lot of dimensionality.
So, like, for the Schrodinger equation, people almost always do it with basis set methods.
And there's been a huge effort, over decades, to figure out really clever basis sets that are really close to the real solutions, so that you don't need to use too many basis functions to get the good solution.
So that's the way to keep the number of variables down.
In the Navier-Stokes world, I'll show you what people do to try to deal with that.
There's many other problems.
I don't know all the problems in the world.
But in general, when you have three or more dimensions in your PDE system, you're looking up special tricks.
You're looking up, how do people in this field deal with this particular PDE?
Special ways that are good for that kind of PDE.
All right, so I said, one problem was that you could have, that the PDE system, itself, can be intrinsically unstable.
So one example I know like that is, detonation problems.
So if I have a system that has some explosive in it, or a mixture of hydrogen and oxygen even, and I have it in a tube or something, if I change my initial conditions, I can go from the case where it's stable, that it just sits there, a hydrogen and oxygen.
Or it could turn into a regular burning.
Or if I change the conditions a little bit differently, it could change into a detonation, where it actually sends a shockwave down the tube faster than the speed of sound, totally different situation.
So those kind of systems can be really sensitive to the physical initial conditions.
A very small spark can make a really big difference in a system like that.
And also, in your numerical solution, if the thing is physically sensitive, it'll typically be sensitive also to numerical noise.
If you get numerical noise at some point, that might be enough to kick it over from, say, not igniting at all to having a detonation, which is a completely different solution.
But another case, like in biology, if you're trying to study the growth of a cell culture or the growth of a tumor, as you know, if somebody has a tumor and one of the cancer cells mutates and does something different, it can have a really different outcome of what happens to the patient, what happens to the tumor.
The whole morphology can change completely.
Everything about it can change.
So that's the kind of system, also, that can be very sensitive to small details.
A small change in one cell, for example, could be a really big difference.
So this is not just in combustion problems you have these kinds of instabilities.
All kinds of problems have this kind of thing.
You can have just regular chemical reactors, like in multiple steady states, little tiny fluctuations can make it jump from one steady state to another one.
That's another very famous kind of problem.
There's another class of PDEs where it's stable in some sense, but it causes a problem.
We call those hyperbolic.
And those are like wave equations.
The solutions are propagating waves.
So the equations of acoustics, the equations of electromagnetism, are wave equations where, basically, a wave propagates more or less without any damping.
So what happens, then, is you have some numerical noise that will make a little wavelet that will propagate, more or less, without any damping, too.
And it'll keep bouncing back and forth inside your domain numerically.
And if you keep introducing numerical noise, eventually the whole thing is just full of noise.
And it doesn't have much relation to the real physical solution.
So hyperbolic systems of differential equations need special solution methods.
That in the numerical method, it's carefully trying to damp out that kind of propagating wave that's coming from the noise.
And they set them up a special way.
I'm not going to talk about how you solve them in this class, but there's a special, whole group of solution methods that people use who are solving acoustics equations and solving electromagnetic equations.
If you get into solving shockwave equations-- I actually work for Professor [?
Besant.
?]
He has, like, electrochemical shocks in some of his systems.
You have the same kind of thing.
You have to use special mathematics to correctly handle that, special numerical tricks.
But we're not going to get into that in this class.
If you get into those things, you should take the PDE class for hyperbolic equations and they'll tell you just all about the solvers for those kinds of things and special tricks.
And there's journal papers all about it.
Here, what we're going to focus primarily on two kinds of problems: elliptic problems and parabolic problems.
And those problems are ones where there's some dissipation that's very significant.
And so, in an elliptic problem, you introduce some numerical noise and as you solve it, the numerical noise kind of goes away.
Sort of like intrinsically stable all throughout.
Now the real definition of these things, parabolic, hyperbolic, elliptic, has to do with flow of information.
So in a hyperbolic system, information-- there's regions of the domain that you can make a change here and it doesn't make any effect on some domain part over there.
And so that causes-- because that is the real situation, that also propagates into the numerical solution method.
So for example, if I'm modeling a shockwave that's moving faster than the speed of sound, I can introduce any pressure fluctuation I want behind the shockwave, and it has no effect on the shockwave.
Because the pressure waves from that pressure fluctuation are traveling at the speed of sound.
They won't catch up to the shock.
So they will have no effect whatsoever.
So if I was judging what was happening by looking at what's happening in the shock, I can introduce any kind of random noise I want back over, downstream of the shock, I guess.
Yeah?
And it won't make any difference at all.
And so, that's part of the reason why I need a special kind of solver.
If I have elliptic equations, every point affects every other point.
A famous case for that is, like, this steady state heat equation.
You have some temperature source here, a heat source here, and you have some coolness source over here.
And there's some heat flowing from the hot to the cold.
And the heat flow is kind of slow enough that everything sort of affects everything else, since you actually have, the heat is trying to flow by, sort of, every path.
And there's some insulation that's resisting its flow.
And it has been in the connectivity of flow a certain way.
And those kinds of equations are called elliptic.
And the methods that we use for the relaxation mesh that we showed are really good for those kinds of methods, for those kind of problems.
And then another kind of problem that we get into a lot is like this one.
This is called parabolic.
And typically, in the cases we see that are parabolic, we have time as part of the problem.
Always what we think should happen is that what happens in the future depends on what happened in the past, but not vice versa.
So you would expect that when you're computing what's happening at the next time point, it's going to depend on what happened at earlier time points.
But if you try to do it the other way around, it would be kind of weird, right?
You wouldn't expect that what in the future really affected stuff in the past.
So it naturally leads us to sort of pose it as an ODE IVP, or IVP kind of problem.
And we know initially, some time zero something, and we're trying to calculate what happens at some later time.
And so those kinds of problems we call parabolic.
And the methods that we're going to talk about mostly are ones for solving parabolic and elliptic problems.
And you can read, in the middle of chapter 6, has a nice discussion.
Just has the mathematical definitions of what these things are.
Now, even in a steady state problem, the problem can have a sort of directionality to it.
So if you have a problem that's very convective.
So you have a flow in a pipe.
And you have a nice, fast velocity flow down the pipe.
What's happening downwind is very much affected by what happened upwind, but not vice versa.
So maybe you're doing some reaction upwind.
Say you're burning some hydrogen.
You're going to get water vapor made upwind and it's going to flow downwind.
And you're going to have steam downwind.
If I squirt a little methane in downwind, it doesn't actually do anything to what happened upwind.
I still have a hydrogen and oxygen flame up there and it's been burning and making steam.
Do you see what I mean?
So it has a directionality, even though I might do it as a steady state problem and not have time in it at all.
And you've probably already done this, where you can convert, like, a flow reactor.
You can write it, sort of, as a time thing or as a space thing.
Time and space are so related by the velocity of the flow.
We'll have similar kinds of phenomenon then.
It's almost like a parabolic time system that, if the flow is fast enough, the fusion backup, anything moving upstream is negligible.
It's all moving from upstream to downstream.
And so then, you'll have the same kind of issues.
In fact, you may even try to solve it by just starting it at the upstream end and competing stuff and then propagating what that output from that first one does to the next one downstream, and then do them, one at a time.
That would be one possible way to try to solve those kinds of problems that are highly convective.
As you slow the velocity down, then the diffusion will start to fight the velocity more and more.
If you make the velocity very slow, the Peclet number very low, then you really have to worry about things diffusing back upstream.
And then it's a different situation.
And that would be more like the elliptic problems we have.
Because things downstream would actually affect upstream, if the flow is really small.
But typically for our problems, the velocity is always really high.
Peclet numbers are really large and so not much goes from downstream to upstream.
It's almost all upstream to downstream.
This leads to a famous situation.
If you look in the textbook at figures 6.7, 6.8, it's kind of a very famous problem.
That if you just do this problem.
So this is just an ODE problem.
So I just have convection and diffusion, no reaction, very simple.
This problem.
You'd think that I could just use my finite differences, where I would put, I could use the center difference for here and here and change this into this obvious looking thing.
You think that that would be a reasonable thing to do, right?
And you can do the same one here.
You can write-- I'm terrible with minuses, sorry.
So these, the finite difference formulas.
And so, you can write these like this.
This stays equal to zero.
And now it's just an algebraic problem to solve.
A system of equations, you have equations like this for every value of n. And there's the mesh points.
You guys have done this before, yes?
Yes.
OK, so famous problem is, if you actually try to solve it this way, unless you make delta z really tiny, you get crazy stuff.
So on figure 6.7, they actually show what happens for different values of delta z, which is different values of what they call the local Peclet number.
So there's a thing called local Peclet number, which is-- OK?
If you make the local Peclet number too large, actually anywhere bigger than 2, and you try to solve this equation, what you get is oscillations, unphysical oscillations.
They'll make the phi go negative.
It's crazy.
It looks like the simplest equation in the world, right?
It's a linear equation, so what's the problem with this?
But it doesn't work.
And so, then you could think, well, why doesn't it work?
And there's, like, multiple ways to explain this about why this doesn't work.
I think that the best way to think about it is about the information flow.
So if the local Peclet number is large, that means that phi is just flowing downwind and diffusion is not doing much.
Because the Peclet number is big.
And so, you really shouldn't use this formula to calculate the flow.
Because what happens downwind, at point n plus one, I guess.
Here we have the flow this way.
So here's n minus one.
Here's my point n, n plus 1.
I really should not include, is this right, n plus 1 in this formula.
Hope I have those signs right.
I apologize if I had them backwards.
Because I don't think that what happens downstream should really affect this at all.
So really, I would do better to change from this center difference formula to what's called the upwind difference formula.
Where they would say, OK, instead, let's use phi of n minus, the phi of n minus 1 over delta z.
Just use that instead of using this formula.
Now, you wouldn't think that that would make much difference.
But it makes a gigantic difference.
And so if you look at figure 6.8 in the textbook, you see you get perfectly stable solutions when you do that.
But where you do this one, you get crazy, oscillatory, unphysical solutions, unless you choose delta z really tiny.
Now, there's other ways to look at this.
So I think this is the correct way, the best way to think about it is, it has to do with information flow.
If you try to make your solution depend on stuff that it doesn't really depend on.
So upwind does not really depend on downwind.
If you force it to, by choosing to include that information here, you end up with crazy stuff.
So that's one way to look at it.
Another way to look at it is that, if you really want to compute these derivatives, you've got to keep delta z small.
Because we're taking a differential.
We're turning it into a finite difference.
And if you take your delta z too big, then it's a terrible approximation to do this.
And you can look at it.
You can see that what we're doing is, when we have this problem where the local Peclet number is getting too large, is we're actually choosing delta z bigger than the, sort of, the thickness of the solution.
So if you look at the solution, the analytical solution of this problem, it's like this.
Where this has a very, very thin layer at the end.
It's like the flow has pushed all your stuff to one side.
And if you choose your delta z to be from here to here, that's how big your delta z is, and then you're computing the derivative of this point halfway along from here to where this is.
You compute your derivative like that.
It's not a very good, not a very accurate representation of what the real derivative is here.
And in fact, you really should have a bunch of points here and here and here and here and here to really resolve the sharp edge.
And so you did something really crazy to use a big value of delta z to start with.
So that's another way to look at this problem.
Now, the fix, by doing this upwind differencing, I think the best way to look at this is saying, well, I'm just going to make sure I only depend, I only make the equations depend on what's upwind, with the convective term.
Because there's nothing convecting from downwind.
So just leave that out.
So that's a conceptual way to think about it.
Another way to think about it, which is in the text, is that by doing this, you're introducing another thing called numerical diffusion, where you're actually, effectively increasing the diffusivity.
Because this is a very poor approximation to the differential.
This is a asymmetrical finite difference formula.
It's not a very good value estimate of the derivative, if delta z is big.
But you can write it out carefully and write this out and say, oh, this actually is sort of like this plus an effective diffusivity chosen very carefully, so the terms cancel out just right.
And it turns out, if you look at it that way, the effective diffusion you've added, by using this instead of that, is just enough to make the local Peclet number, if you compute it with the effects of this d effective, to stay less than 2.
Which is what you need as a condition to make this numerically stable.
So I strongly suggest you read that.
It's only two pages long in the textbook.
It's very clear.
Just read, just look at it if you haven't seen this before.
There's a similar thing and it might be relevant to the COMSOL problem that Kristyn showed you on Monday, is you can-- In that case, there was a region of the boundary here that had some concentration c naught, and then there was, over here, the concentration was zero.
The boundary is zero.
Here, it's some number, 17, whatever number it was.
And if you think of how to model what's going on, one way to look at it is, I have a diffusive flux coming in from the drug patch, diffusing the drug into the flow.
And I have this big, giant flow, flowing along here.
And the flow gets slower as I get closer to the wall because of the friction with the wall.
And so if I looked somewhere right in here, I could just draw a little control volume, and look what's happening.
I have a diffusive flux coming in here.
I don't have any drug up here, so there's no drug coming in here.
But I have drug leaving.
So basically, it's coming up and it's making a right-hand turn.
And so then I could think, well, if I was going to try to figure out, what's the steady state concentration of drug in there, I could say, well, there's a certain amount of drug entering from the diffusive flux, which would be just this times dcdy.
This is the y direction.
And then this part over here would be the velocity, actually, just times the concentration I have in here, right?
So how much is flowing out.
And it's probably, my units are screwed up so there's-- Well, maybe not.
That's OK. Is that all right?
Units are screwed up.
You guys will get it right.
It's probably an area.
Something to do with this size right here.
This one here, too.
Is that right?
So you could compute what the steady state concentration would be if it's just coming in and flowing out.
And so, this is the finite volume view of this kind of problem.
And so you can write the equations that way, instead.
And what you really care about are what the velocity flows and the diffusive fluxes are on these boundaries around the point.
So you don't try to compute things right at the point.
You compute around it.
And one way to look at this is, we just did finite differencing.
We were looking at what was happening coming in.
And then, this is actually a center difference around this interface, which is halfway between the two points.
And so actually, this is a good approximation for the finite volume point of view.
This is all right?
Now, these are all different ways to write down the equations.
All of them are correct in the limit that delta z goes to zero.
If you have an infinite number of mesh points, all these are exactly the same.
But they lead to really different numerical properties when delta z is large.
And they're all inaccurate when delta z is large, so it's all about what happens with the inaccuracies.
Now, we can go ahead and take any problem, even really complicated 3-D problems or 4-D problems like this, and we can write them with relaxation methods.
And what we're basically doing in a problem like this is turning it into some function of y is equal to zero, where y is the value of all the unknowns.
And there's a lot of equations.
So I might have 100 million equations and 100 million unknowns that are the values, or every state variable at every state point at all times.
And I can write it down, by finding differences, by finding elements, by finding volumes.
Any way I want, I can write down an expression like this.
And I know in the limit, as I make the number of elements of y go to infinity, it will actually be the real solution.
And the problem is, my computer can't handle infinite number of unknowns.
In fact, it's even going to have trouble with a 100 million unknowns.
And so, I have to figure out what to do.
So how would I solve this normally, is I would take the Jacobian of this and I would do, I'd take an initial guess and I'd update it and have the Jacobian times my change from the initial guess to the equal negative of f, right?
You guys have done this a million times.
As Newton-Raphson is how you'd find improved from an initial guess.
You have to provide some initial guess.
This is actually pretty hard if you have 100 million unknowns.
You have to provide the value of the initial guess for every one of the 100 million.
But somehow you did it.
And now you want to refine that guess and this is the formula for it.
And so, you just use backslash, right?
But the problem here is that now f is a 100 million long vector and this is a 100 million squared matrix.
And a 100 million squared is pretty big, 10 to 16th elements in it.
So I'm not going to just write that matrix down directly like this.
Now, we cleverly chose local basis functions, so this is going to be sparse, this Jacobian.
So most of the numbers, are those 10 to the 16th numbers in this matrix are zero.
So we don't have to represent them.
But there still might be quite a few.
Probably the diagonal events are non-zero, so that's like 10 to the 8th of them right there.
And we've got, probably, a couple of other bands.
So I don't know, might be 10 to the 9th non-zero elements inside this thing, which is pretty nutty.
And depending on how much memory they provided in the laptop they gave you, you might be filling up the memory already just to write down j.
And certainly, if you tried to do Gaussian elimination to solve this, you're going to have a problem because you get what's called fill in.
Do you guys remember this?
Even if you start from a very sparse Jacobian, as you run the Gaussian elimination steps the number of non-zero entries in the intermediate matrices gets larger and larger.
Remember this?
And so, even if, initially, you can write down j and store it in your computer, you could easily overwhelm it with the third iteration or something.
So you're not going to just use Gaussian elimination.
So what do you have to do?
You want to find a method.
They're called direct.
Direct methods to solve this kind of problem, in order to get your increment, which is going to be your update, to get a better estimate of what the solution is.
And the direct methods are ones that you don't actually store the gigantic matrix.
So you're trading off CPU time for memory.
So you don't have enough memory to store the whole thing.
Which we, normally you would do this with Gaussian elimination and huge steps.
It would solve it perfectly, right?
Right, Gaussian elimination's great, backslash.
You guys have used it a few times.
It's a really nice program.
It always works.
It's really nice.
But you're going to give up that because you can't afford it because the memory, it's consuming too much memory and your cheapo department didn't buy enough RAM in your computer for you.
So you're going to instead try a direct method which is, so it's trading off CPU time versus RAM.
So you're going to reduce how much memory you actually actively have, but you're going to expect that it's going to cost more CPU time.
Because otherwise, everybody would do something else besides Gaussian elimination all the time, and they don't.
All right, so what's the method we're going to use?
It turns out that variance on the conjugate gradient method turned out really well.
So we mentioned this earlier, briefly.
What you do is, instead of trying to solve this, you try to solve this.
So you just try to minimize that, right?
You know at the solution this is going to be zero.
Right, jy plus [INAUDIBLE] zero when this is solved.
So you try and vary and find the delta y that makes this go to zero by minimizing it.
And then this method, it turns out that the conjugate gradient, if everything works great, this is guaranteed in n iterations to go directly to the solution of this kind of problem.
Now n is large, now, because we have 10 to the 8th unknowns.
So that's 10 to the 8th iterations.
You do anything in 10 to the 8th iterations, you're not really sure it's really going to work.
So that's a warning.
But at least it should get, even after a few iterations, you should get a smaller value of this than you had before.
And so this is the method.
And what's really nice about this method, if you look into the details of how it works, it never has to store any intermediate matrices.
So all it needs is the capability to multiply your matrix times some vector.
And from that multiplication, it figures out a step direction.
That's the trick of this method.
And because it only has to do a forward multiplication, you don't have to actually store j.
You can just compute elements of j, as needed, in order to evaluate this top product.
This j times v. And you don't have to ever store the whole j at once.
So it's really great for RAM to do this method.
Now, as I said, when you do 10 to the 8th iterations, things don't always work out so well.
And so people have found that when you go more than maybe 10 or 20 iterations like this, you tend to pick up numerical problems.
And so, they've worked out better methods.
And there's one that people use a lot for these problems called bi cg stab.
Which is biconjugate gradient stabilized.
The applied mathematicians went crazy trying to figure out better methods like this.
And this is the one that, I think currently, people think is the best.
Though probably, there's a lot of research on this, so maybe there's even better ones now.
But inside Matlab, I think this is the best on they have.
And this is a method for doing this.
Now, it's iterative.
Now, let's remember what we're doing.
We're trying to solve this problem.
We started with a guess.
We're going to break it up into an iterative problem like this, where we're going to do some steps, delta y.
This is now doing iterative procedure in order to compute delta y.
So we have an interim procedure and another iterative procedure inside the first iterative procedure.
So this is going to be a lot more CPU time.
So just to warn you.
And it's also, it's not guaranteed like Gaussian elimination, to beautifully come to machine precision solution.
It's going to get you something.
But anyway, this is what people do.
So that's one good thing to try.
Now, this whole approach is sort of based on Newton-Raphson.
We know that doesn't have the greatest radius of convergence.
So you need to provide a pretty darn good initial guess.
So how are we going to get a good initial guess if you have a 100 million unknowns?
So what methods do we know?
So one idea is you could do things like homotopy, stuff like that.
If you could somehow take your PDE system, get rid of the nonlinear terms.
You're really good at solving linear systems of PDEs.
You start with that and then you gradually turn the non-linear term on.
Maybe you could coax it over.
So that's one possibility, if you're really good at PDEs.
Another possibility that I've run into a lot is the problem you're trying to solve, for example, might be a steady state problem like this.
Where this is zero and you try to figure out what the steady state situation is in a flow reactor, for example.
And so, it's actually, your steady state problem came from a time dependent problem.
And if you think that your time dependent problem really converges to the steady state problem, sort of without much concern about what the initial guess is, then you could put any random initial guess in and march it along in time enough, and eventually, it should end up at the solution you want.
So that's the idea.
So that's called a time marching idea.
So there's one idea, sort of the homotopy continuation kind of idea.
That's one idea about what to do for initial guesses.
Another idea is the time marching idea.
And the key about it is, you have to really believe that, physically, the system does march to the steady state you want.
If it does, then this is a good idea.
If it doesn't, then you're not going to end up where you want, because it's not going where you want.
And so, you have to watch out.
But in some situations, this is true, that things work.
So I do a lot of combustion problems.
A lot of these things, you light a spark on a stove burner.
No matter how you light it, it ends up basically the same flame.
So that's a good one.
But I have some other ones where I light a candle and I'm in a convective field, a turbulent field, and it flickers.
And no matter what, it always flickers.
And I never get a steady state, so I'm never going to find a steady state solution with that one.
And so I could time march until I retire.
My computer's still burning CPU time all that time and it'll never get to a steady state solution.
So you really have to know what the real situation is.
But if you have one where it's going to converge, then this is a reasonable idea.
Now let's talk about the time marching.
The time marching, you have, say, dy, dt is equal to some f of y where this is the discretized spatial vergence.
I took all the space dimensions and replaced them with the finite difference expressions for the derivatives.
Or I did colocation or something to convert this into algebraic equations.
So the right-hand side is now algebraic equations, no longer differentials.
And now, I'm going to time march this and I just-- This is very long.
It's a 100 million.
So how are we going to time march that?
Well there's kind of two schools of thought for this.
One school of thought is, if I can use an explicit method, an explicit ODE solver, those are pretty nice.
Because all I have to do is evaluate f and then I can march along and I never have to save any matrices.
Because remember, look at those formulas, y, y new, by the explicit method, is equal to some function g of y old.
And this is the update formula.
It can be forward Euler.
It could be RK 2.
It could be whatever.
But there's just some explicit formula.
I plug in the old vector.
I compute a new vector.
I never had to do any inversions of matrices or anything.
So that's good.
And this is, indeed, the way that the Navier-Stokes equations are solved currently by the best possible solver.
So the best solver in the world discretizes in space and then does ODE IVP using explicit method for marching it.
They do problems that are pretty big.
So they'll have like 100 million mesh points and then 100 species.
So they have 10 to the 10th unknowns.
And at each time step, you're computing another vector that's 10 to the 10th long.
So it's a pretty big problem.
And they use, like, 40% of one of the national supercomputer centers will be running at one time for one job like this.
But it works.
You never have to store anything that's too big.
The biggest thing you have to store are the vectors.
Which, you're always going to have to store some object the size of your solution, right, if you're going to get your solution.
So that's one possibility.
Now, this is really good if an explicit solver can solve it.
So that means it's just not stiff, your problem.
And also, if you want a time accurate solution.
So in that case, they actually want the time accurate solution.
But suppose we're trying to really solve the steady state problem over here.
Then we really don't care about the time accuracy.
We're not going to report anything about our time steps.
In the end, we're only going to report our steady state solution.
That's all we cared about.
So in that case, there's no reason to try to have a really high accuracy, explicit formula and try to make sure we all our time steps are exactly right.
We're just going to throw away all those time points anyway.
We'll just keep the final one as our initial guess for a Newton-Raphson solve to find the steady state.
So in those cases, you might instead want to use-- So this is time accurate.
There's another method.
I don't know if I should call it time inaccurate.
If you don't care, if the solution is really, what the solution yft is, all you care about is where you get to, then you can do other methods.
And one of them is the backward Euler.
So you can say, well, y new is equal to y old plus delta t times f of y new.
Now, the advantage of this is, I can make delta t pretty large.
This is guaranteed stable as long as the ODE system is stable.
Remember?
And so, this will go to this, go to the solution pretty well, to the steady state solution.
It won't go there in the right amount of time because this is a very low order of formula.
It's not really accurate.
But it will end up to the real steady state solution.
This is what's actually used a lot.
However, the problem with this is, this is now an implicit equation.
So we may have to solve it with some method like Newton-Raphson.
But we got into this because we couldn't solve our Newton-Raphson steps because we didn't have a good initial guess.
So then the question is, can we find a good initial guess for this problem?
And in this case, you can because you can use the explicit formulas to again provide the initial guess for this implicit formula.
So that's what they do, also.
And so now, if we're doing an iterative procedure in order to get the initial guess for our other iterative procedure ever there, which we're going to break up into another iterative procedure inside it to solve every step.
But this is currently what people do.
So you guys are smart.
Maybe you can figure a better way to do it.
The world is waiting.
I think I'll stop there.
By the way, actually just names-- This kind of thing is called method of lines.
And if so, if anybody ever says I solved something by method of lines, what they meant was, they discretized in some of the coordinates and they kept one of the coordinates as a differential.
And then they solved it with an ODE solver at the end.
And in certain systems like this, it's a smart thing to do.
You need to be kind of lucky that the set up is right, so you can use it.
But if you can, it can be pretty good.
All right.
See you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, shall we begin?
So on the syllabus, today's class is supposed to be a review for the next quiz.
And I'm happy to answer any questions you have about the material or about the quiz to the best of my ability.
And when we've exhausted those questions, I have some notes on another method for solving partial differential equations.
We didn't really cover it in class, but I think it's quite interesting and applicable for chemical engineers in particular because of the classes of problems that we study.
And then notes on how to construct a simulation of a PDE that follow from what Professor Green discussed during your last PDE lecture.
And we'll do an example where we actually develop a code to solve the PDE and maybe refine it so that it runs more quickly by noticing certain things about the structure of the problem.
OK?
I have your gift cards.
I remembered them.
So I'll hand those out at the end the class, just in case more people trickle in.
But it doesn't look like there's 53 people here, so maybe the TAs are going to be eating burritos for a while.
There's going to be a lot left over.
We'll see.
I'll hold onto them.
Try to make sure everyone gets their reward for responding to the class survey.
So let's start with questions about the quiz, about the material that we've covered up to this point.
I'm really going to leave it to you to point out things that you feel like you need help with, you think need to be reviewed.
I know you've had a review session with the TAs already.
Maybe that's been sufficient.
OK?
Questions.
It's up to you.
Yeah
Will the test contain chemical engineering material like last year's test, or will it be like Professor Green's test
Oh, good question.
So I'll tell you that I believe the problems on the quiz this year are motivated by chemical engineering problems, OK?
The reason for that is, unlike linear algebra problems or optimization problems, those can be crafted out of whole cloth.
But workable, right, boundary value problems, workable initial value problems really need to stem from something physical.
You are not asked to derive anything from your extensive chemical engineering knowledge.
This is not a qualifier for reactors or thermo or transport.
We provide you with those details.
OK?
But they are-- I think they are of a chemical engineering origin.
So you have equations that are reasonable to work with.
OK?
More questions.
Yes
I would love to briefly go over some boundary value problems, specifically like the collocation and--

--Galerkin methods, maybe
Sure.
So OK.
So let's just discuss boundary value problems first.
Turns out PDEs are a class of boundary value problem, too, largely, at least the ones that chemical engineers are interested in.
So if we understand boundary value problems in general, we should understand how to solve partial differential equations too.
Many classes of partial differential equations that we're interested in can also be solved in these ways.
So we talked about relaxation methods in general, which were these methods that rely on somehow discretizing the continuous function that we're trying to describe in this boundary value problem.
So usually, you've got some-- linear function, some linear operator that operates on some variable that we're trying to understand in a boundary value problem.
Maybe it's temperature, right, equals 0.
So it's going to be something like thermal diffusion.
This equation could represent that.
Could represent some advection diffusion problem in some bounded domain.
Could be a PDE.
It could just be an ordinary differential equation, right, but a boundary value problem.
And with this class of relaxation methods, we try not to exactly satisfy this equation.
We relax that assumption.
We try to somehow discretize this function.
It could be temperature, could be energy, could be velocity or momentum.
It could be lots of different things.
Try to relax the constraint that this is satisfied everywhere.
And we try to satisfy in some approximate sense instead.
So we discussed collocation methods, where we take the domain we're interested in and we divide it up into a series of points.
And we try to satisfy this equation at those points.
And we have some approximation for the function, its representation at those points.
That approximation could be a finite difference approximation.
That approximation could be a basis set expansion.
It could be a finite volume approximation.
We could expand in terms of local basis functions.
That's the finite element method.
We covered all these different methods.
That involves chopping the domain up into a series of points and trying to satisfy the function at each of those points.
There was an alternative method, this sort of Galerkin method, where instead of trying to satisfy the function at each of those points, we have some basis set expansion.
So we have some approximate representation of our function in terms of these basis sets-- in terms of this basis set.
And instead, we try to make it be the case that the integral of our operator applied to our approximation-- I'm going to break chalk-- multiplied by each of the basis functions and then integrated over this entire domain is equal to 0.
So we try to satisfy this equation with respect to each of these basis functions, essentially, right?
There's some-- this isn't going to exactly satisfy the equation.
This is an approximation.
It's not going to exactly satisfy the boundary value problem we were after.
But the projection of the residual, or how far off we are with this operator, on each of the basis functions, we're going to try to force to be 0.
So rather than trying to get a pointwise solution of the problem, we're going to try to sort of smooth out the error with respect to the basis functions and make this equation is satisfied as it possibly can with respect to these basis functions.
And that's the Galerkin view.
So it's different.
It's like you're trying to minimize sort of a global error with respect to the basis functions, as opposed to a pointwise error, OK?
And again, these basis functions can be chosen in a lot of different ways.
They can be chosen as sort of global functions.
They can be things like sines and cosines.
We may know that the solution to this equation looks a lot like sines and cosines, so that makes sense.
Or we may choose local basis functions instead, little tent functions in 1D.
They're actually tent functions in many dimensions as well.
You can imagine sort of extending that to look like a tent in many dimensions.
That's the finite element method.
So we can choose those basis functions however we want.
And the goal is to just minimize this residual with respect to each of the basis functions that we include in our approximate solution.
And we would call that a reasonable approximation.
Presumably, as the number of basis functions goes to infinity, this is going to converge.
That's the applied mathematician's job, to show us that it's going to converge to the true solution of the equations.
OK?
So those are the methods that we-- those are the methods that we apply.
Good choices of basis function are important.
So there was this discussion about quantum chemistry, where you're trying to solve the Schrodinger equation for n electrons, there's 3n degrees of freedom.
So a very high-dimensional space.
How are you going to do that?
You do it with basis set expansions.
And the choice of those basis functions is really important.
So if you read a quantum chemistry paper, you'll see they give a long series of digits and numbers that are meant to indicate which basis set expansion they used.
And for certain classes of problem, some are better than others.
And they make the problem actually solvable when it's not, if you were to just choose your basis functions without care for the physics of the problem.
Sam
The last homework, in the Galerkin method, where you multiplied by the complex, how did you get one basis set, basis functions?
Where does that fit into this?
Well, it should have been right here.
So technically, the basis functions don't have to be real-valued functions.
They can be complex-valued functions.
And then this is, in some sense, a dot product, an inner product in the space of these basic functions.
And the proper form for an inner product when we have complex-valued things is to take the real part of one and multiply it with the complex part of another.
So that should have been right here.
This should be the complex conjugate.
OK?
Yeah
So when we do the Galerkin method, we necessarily have to approximate these derivatives and things using basis functions
Yes
But in the collocation method, there was lots of other ways that we could do it, like find differences.
I know you mentioned a few of them earlier
Yes.
Yes

That's right
That [?
LOT term ?]
kind of contains also the basis function in the Galerkin method, or in the collocation method
Well, in the-- if we have an expansion of the sort of objective that we're trying to calculate in terms of those basis functions, L is the real L, right?
It is what it is.
This is the model equation to be solved.
We operate on our approximate solution with the real L. In the collocation method, it may be the real L, or it may be some approximation for L. If we do finite differences, we're approximating all these derivatives.
If we do a basis set expansion and we try to satisfy that at a certain number of points, we're not approximating L. We're approximating T instead.
So they're a little bit different.
More questions?
So that's BVPs.
This picture applies for PDEs, too.
So this is the integral over the domain of interest, the domain that's bounded by our boundary conditions.
These basis functions don't have to be in one dimension.
They can be in any number of dimensions we want.
We just want them to be good basis functions in principle.
So for finite elements, you use little tents.
They're multi-dimensional tents, but they're still tents.
The same as these hat functions that Professor Green talked about in 1D.
Just do them in many dimensions.
They work exactly the same way.
They're easy to take the derivatives of.
These intervals are, for the most part, simple to calculate.
And that's why COMSOL, for example, uses that methodology.
It's based on this sort of Galerkin picture.
So PDEs, BVPs look very similar.
Usually, when we're talking about spatial dimensions, they look very similar.
When we go to the time domain, things get a little more confusing.
That mixes together what we know from ODE IVPs with what we know from boundary value problems now.
So you talked about the method of lines last time as one way of studying, for example, parabolic partial differential equations.
You have the spatial discretization like this.
And then in time, you have some ODE IVP technique that you integrate forward.
You rely on these robust ODE methods that know how to control the numerical error, can detect, maybe, when there's some sort of an instability and try to damp it out by altering the time step so that things stay stable.
Yes?
I can't visualize the local basis.
Can you give me an [INAUDIBLE] expression for the local basis?
Sure.
So first of all, there are a lot of local basis functions we could write down, OK?
This is just one of them
There are many of them.
So yes, let's do the hat.
OK?
Let's suppose I have this line in x, and I target in my collocation method a series of points, column x i minus 1, xi, x i plus 1, and they extend well beyond here.
It doesn't matter where they are.
I choose where they are based on the physics of the problem.
I'll put them where I want them to go.
And in fact, you might even try to optimize where they go.
So you might start with some distribution of them, and then move them around in some way to improve the quality of your solution, right?
Maybe you distribute them uniformly at first, and then you notice, oh, my solution is changing very quickly over here.
I would like more points in this region.
In MATLAB, there's a function called bvp4c, which is a boundary value problem solver.
It solves 1D boundary value problems.
And it does precisely this.
It uses finite differences and collocation.
And the good thing about it is it's trying to adaptively change where these points are positioned.
So it solves it, it looks where things are changing quickly, and tries to shift its points around and resolve it in order to improve the quality of the solution, hit certain targets for the error.
So what are these-- what would these local basis functions look like?
So we talked about these hats.
They're 1 here.
They're going to be 0 on either side of this point.
We could call this our hat i, which is centered at point xi.
And it's going to be a piecewise sort of function.
It's going to be an increasing linear function between x i minus 1 and xi.
So for x between xi and x i minus 1, it'll have one form.
Then it's going to switch.
It's going to be a decreasing function on the other side.
So between xi and x i plus 1, it'll have a different form.
So here, it's going to increase between 0 and 1.
So it's going to have to look something like x minus x i minus 1 divided by xi minus x i minus 1.
So it goes from 0 to 1.
And then on the other side, it's going to have to have the opposite sign.
So this is going to have to look like-- let's see, x minus x i plus 1 divided by xi minus x i plus 1.
Will that do it?
I think so.
OK?
You want me to make it look like this one.
So we could have plus 1 xi, and then put a minus sign in front of it if you want.
You decide how you want to write it.
But it describes this line here and this line here.
But there are other choices.
So it doesn't have to be piecewise linear necessarily.
A lot of times, people choose different types of splines, different types of interpolating functions.
They might not be localized to one point.
They might be localized over a few points.
Maybe something about the physics of a particular problem that tell you, you can't get away with these piecewise linear functions.
They're not good enough to describe, say, higher-order derivatives in your equation.
So you might instead interpolate with, I don't know, like a quadratic basis function that spans a few more points.
That'd be fine.
It would be easy to do the integration in your Galerkin method.
You just do these integrals out.
They're complicated, but you do them once for a generic point xi, and you know their value for every point in the domain.
So you choose them.
You choose the basis set.
But that'll give me an equation for the tent function centered at xi.
That's the tent function centered at any point in the domain, in principle.
In many dimensions, right-- so if I have two dimensions, you see in COMSOL, you have a sort of a triangular mesh.
And what does my tent function do?
It sort of stands up, stands up above the triangle.
It's a real tent now.
It's coming out of the board.
Can you see it?
Put on your 3D glasses.
You'll see it.
But it's coming out of the board.
It has height 1 above this triangular domain.
It sits at the center of the triangle, for example.
It's one way to draw these elements.
There are actually many ways to draw these elements.
This is just one way of drawing these elements.
OK?
Good.
Other questions?
Could you please explain why central differencing is second-order accurate and not--
Oh, good question.
Good question.
So why is central differencing second-order accurate?
So the central difference formula says that the derivative of a function x could be written as-- let's see, f at x i plus 1 minus f at x i minus 1, divided by 2 delta x.
Or let's say these aren't equally spaced points.
Well, let's make these equally spaced points for now.
Let's make it easy.
So let's say x i plus delta x and x i minus-- it'll make our lives easier if we do it this way.
It can be generalized, but let's do it the easiest possible way.
And this is the derivative at the point xi.
Graphically, here's x, here's xi.
Here's my function.
I want to know what the slope here is.
And I'm going to approximate it by looking one point upstream and one point downstream.
And I'm going to connect those points with a line.
You would agree that as the spacing gets smaller and smaller, it will be a good approximation for what the derivative is.
So why is it second-order accurate?
So you can show that.
It's not too hard.
You can show that using Taylor series expansion.
Like everything else in this course, the Taylor series is really a key.
So let's look at it.
So f at x i plus delta x. I want to write this as a Taylor series expansion about the point xi.
So tell me the first term.
f of xi.
OK, tell me the second term
f prime at xi
f prime at xi.
Times?
Delta x
Delta x.
Tell me the third term
1/2 f double prime
1/2 f double prime at?
Xi
Times?
Delta x squared
How big is the third term?
[INAUDIBLE]
OK. Is this always true, by the way?
This?
Not always, right?
There are lots of functions, non-analytic functions that don't have these sorts of expansions.
So we're sort of-- we've got to have a function that has these derivatives.
They need to exist at the point xi.
Let's assume that's true, OK?
Good.
OK, now I'm going to do something really simple.
Let's write f at x i minus delta x.
And let's just point out the differences between these two formulas.
So this is the same.
This switches sign because delta x switched signs.
It was plus delta x.
Now it's minus delta x.
This is the same, delta x squared, same as minus delta x squared.
Doesn't matter.
This one down here-- well, OK.
It's order delta x cubed.
It'll actually look the same but have the opposite sign.
OK?
We can't resolve it, though.
That's sort of the tricky part, right?
We don't know.
We don't know for sure what value that takes on.
This just gives us a bound for how big the error is in this expansion.
So let's now compute what's in that formula.
So if you take the difference between this and this, these two will cancel.
These two will combine.
I'll get 2 f prime xi times delta x.
These two will cancel.
So I'll get 0.
These two, I can't rely on them necessarily canceling each other out.
In fact, they want to combine in principle, right?
We've just bounded the error.
Here, we don't really know what it is, but it gives me something that's order delta x cubed in size.
So this is the-- this is 1 minus 2.
It's 2 f prime xi times delta x plus the error, which is order delta x cubed.
If I divide by 2 delta x-- I know you guys hate it when the instructor does this, but you know, you can't help it.
Divide by delta x, the error becomes order delta x squared.
So if the derivative is this thing plus some order delta x squared piece, it's second-order accurate, we would say, right?
The error is going to be reduced like the square of the spacing.
Now you can imagine all the finite difference formulas out there in the world are constructed in this way.
You use Taylor series expansion.
You've got as many terms as you need to go for the level of accuracy you're trying to achieve for different points based in different ways.
And you try to dream up some linear combination of these things so that you get a certain level of accuracy.
They're also not unique.
I can construct these in a lot of different ways and have the same sort of error level out here, but different coefficients associated with the error, but the same overall scaling.
And then you might choose different approximations for other reasons.
Sam
What would stop me from going on further and then just subtracting f prime from both sides?
I mean, it wouldn't really tell me anything, except maybe the order of [INAUDIBLE] approximately 0
Well, I mean, yeah.
So let's say you go out further.
What's the next term here?
It's going to be 1 over 6 f triple prime xi delta x cubed.
And here, you'll have minus that because I cubed the delta x.
So when I combine them, I'll actually get 1 over 3 f triple prime.
So I could do a little better.
If I knew what my function was, I can tell you exactly how big the error was.
I can tell you that it was as big as the third derivative of f. My function doesn't have a third derivative at that point.
The error might be huge.
And it's possible to construct functions that lack that particular property, OK?
So you can go further, and you can use it to sort of quantify exactly how many you think the error might be in the approximation.
But it's all constructed from linear combinations of different Taylor expansions about nearby points.
And the related problem, actually, the equivalent problem is polynomial interpolation.
So the sort of-- if you can somehow, in your mind, transpose this problem, it looks a lot like polynomial interpolation.
So these two are intimately related to each other.
OK?
Yes
How much do we have to know about polynomial interpolation?
How much do you have to know about polynomial interpolation?
Yeah, like, do we have to be able to compute the primes and polynomials?
Say that again
Do we have to be able to compute the primes of polynomials?
[INAUDIBLE]
What do I want to say about that?
I wouldn't expect that level of detail.
I think it's quite difficult to construct Lagrange polynomial expansions by hand.
I don't know anyone who does that professionally or for pleasure.
I think that's a bad idea.
You should understand polynomial interpolation.
You should understand the context in which it's important.
What is polynomial interpolation important for, numerically?
Do you recall?
Hersch
Numerical integration
Numerical integration, right.
So this was really the key to numerical integration, was trying to construct appropriate approximations of our function from a finite set of data and figure out the area under the curve.
So you should understand how one uses polynomial interpolation to do numerical integration.
But I wouldn't expect us to ask you to actually compute a polynomial interpolant and numerically integrate it by hand.
I think that's too much.
OK?
OK. More questions.
Hersch
Now do you mind showing how the trapezoid rule is over delta t squared accurate?
Sure
For ODE IVPs
For ODE--
IVPs
ODE IVP, OK. OK, so we have to have an ODE we want to solve.
Can I just do-- can I do a one-dimensional unknown?
Do I need to do a vector here?
Can I just-- would you be satisfied if I wrote this as f of y and t and that it's sufficiently general?
That way, you don't have to blame me when I forget to underscore things along the way.
And we don't have to think about the Jacobian or anything like that.
OK.
So oh, time derivative, right?
So there's the ODE we're going to solve.
The trapezoid rule.
OK, so let's do this the following way.
It's actually going to wind up looking very similar to what we just did, which is useful.
I might even be able to pull that board down and reuse it at a certain point.
So that'll be good.
Let's do this.
So what is-- ultimately, what is ODE IVP solving like?
What are you trying to do when you solve one of these things?
Extrapolate
It's extrapolation, right?
I know the value of y initially.
I'm trying to figure out where the hell this thing is going.
I've got to extrapolate for it.
And you should be very uncomfortable with the notion that I can do that, right?
How good a job can I do predicting the future knowing how things are now?
It's really quite complicated.
In fact, it's so complicated, there are certain circumstances in which it's not really possible.
Discovered here at MIT, actually.
So there was weather research going on at a certain point at MIT.
It's a very famous case.
A researcher, a very famous researcher was doing numerical simulations of a weather model in an ODE IVP.
He ran it for a while, and he stopped.
He looked at his end state, the last value of y in his simulation.
He put in three digits, used that as the initial condition, and started his simulation again, and it went wacky.
Didn't look anything like the initial solution.
OK?
How did that happen?
It seemed fine up to that point.
Solution's cranking along.
He stopped.
He took three digits accuracy out of the computer, and he started again.
And it, I don't know, did something else.
So there are problems that are inherently chaotic and very sensitive to their initial conditions.
They're hard to resolve.
So the entire point of doing this sort of analysis of ODE IVPs is, at least in the best-case scenarios, we want to try to manage the error associated with these kinds of extrapolation.
In some case, it's really not very possible.
You don't know the initial conditions well enough to predict what the final state is.
So there are problems that are inherently unstable or sensitive to their initial conditions.
They don't have to blow up either.
They can find themselves moving through the phase space in some way that's very, very sensitive.
And two points that start very close together can wind up very far apart from each other.
So that's deterministic chaos.
But outside of those problems, we try to do this extrapolation and figure out sort of a best approximation for where we think our target function is going.
So what are we trying to do?
In principle, we know y at a certain point.
And we want to know where it's going to go.
And so you might say, well, maybe I know dy dt initially.
And so I should just move a small increment along that slope.
And that's a good approximation.
That's the forward Euler approximation.
Trapezoid rule-- or equivalently, the midpoint rule-- says, well, maybe that initial slope isn't really the right one to use.
Maybe y does something like this, and we should actually look sort of halfway between the point we're trying to extrapolate to and the current point, and use that value as the best approximation of the derivative and extrapolate for it along that.
And it turns out that's more accurate.
So what does that look like?
So one typically says, you know, y i plus 1 is equal to yi plus delta t times f y i plus 1/2 comma.
And we won't worry about the t here, but it's going to have to be t i plus 1/2.
So I look halfway forward in time.
And I say, that's the slope, that's the derivative of y.
So let me step forward along that line.
It turns out this is going to be reasonably accurate if I can get a good approximation for y i plus 1/2.
OK?
OK.
So let's-- [COUGH].
Excuse me.
Yeah
Is this the midpoint method or--
They're functionally equivalent to each other.
They work in precisely the same fashion.
So whether it's the trapezoidal rule or the midpoint method, they have fundamentally the same pictures there.
OK?
Here, if you do forward Euler, you're doing numerical integration, assuming the slope is the value at this point.
If you're doing the midpoint method or the trapezoid rule, you're trying to get a better approximation for the slope, right?
So you're trying to build a trapezoid and integrate the area under that trapezoid.
So they're fundamentally the same thing.
Now the question becomes, I don't know f at y i plus 2.
So I need to use a Taylor expansion of this quantity in terms of the value of f that I know.
So I need to expand this in terms of f at yi.
And I'll know the derivative of y with respect to t at yi.
I know it from this formula.
So if I take this Taylor expansion and I carry it out to the first term I don't know, which is the order delta t squared term, and I carry through all of the errors, I'll find out that there's a numerical error proportional to order delta t squared.
So I need to do exactly this and substitute in terms of quantities I know and quantities I don't know.
And the leading-order contribution of the quantities I don't know will give me this order delta t squared piece.
And that'll give me the local truncation error in the solution.
Does that make sense?
So it's still Taylor expansions.
Good.
Other questions.
Yes
Why is the central difference 0, not [INAUDIBLE]??
Why is the central difference--
No, why's the outlook [INAUDIBLE] the one we did [INAUDIBLE]?
That one is not implicit, right?
I don't think I said anything like that.
So implicit versus explicit really depends on what you're trying to do.
So if I'm trying to use a central difference formula of some sort to solve an ODE which progresses forward in time-- it has a direction associated with it that things always progress in-- then this might be an explicit formula.
Because I could solve for f at x i plus 1 in terms of what's going on in the past.
If we tried to do finite differences in a boundary value problem instead, and that boundary value problem doesn't have some direction associated with it anymore-- so it's not like space has some preferred direction.
Time goes in a preferred direction.
Space does not.
You may find that the pointwise equations you write down cannot be solved strictly in terms of one of these.
You can't just move this to the other side of the equation and solve in terms of everything else.
So you'll have a coupled set of equations that can't be diagonalized, can't be simply reduced.
They won't have the form of, say, of triangular matrix, for example.
This will happen.
And in that case, you would say this formula is implicit.
So it really depends on which problem you're trying to solve, whether you label them explicit or implicit.
Explicit is going to mean that the data I want can be found with a simple equation solve in terms of the data I know.
Implicit is going to mean I've got to do some complicated equation-solving methodology.
I've got to go do Gaussian elimination, or I need to go to fsolve and solve this thing.
The data that I want is not available through some simple scheme where I can just write explicitly a thing I want equals some function of things I know.
OK?
Other questions.
OK. Yeah.
Yeah, yeah
[INAUDIBLE],, how many questions will there be?
And for each question, how many subquestions?
Could you just roughly--
Oh, man.
The question was, how many questions and how many subquestions on the quiz?
Yeah.
The quiz is-- in my opinion, it's now shorter than the last one that you took.
We've been pretty sensitive to time concerns.
We know that was something that was an issue on the previous quiz.
I think there's only going to be two problems on it.
They have many parts, but we've really tried to refine the questions in those parts.
That's what you should expect.
It's comprehensive.
It Covers all the things from this section.
OK?
Yes
Also, DAEs problem--
Yeah
It says that as index increases, we have less degrees of freedom
Yes
Could you explain?
Sure.
Sure.
So higher-index DAEs come up all the time.
And the degrees of freedom refers to your ability to choose initial conditions for the problem.
And we saw several examples in which, if we tried to convert that DAE to an ODE, we found along the way that there were certain hidden conditions that all the initial conditions would need to satisfy in order for this ODE, the system of ODEs that we built from the DAEs, to follow the same solution path or solution manifold as the original DAE system would follow.
The higher the index is, the more of these hidden constraints you find.
And those hidden constraints tell you that the initial conditions have to be very particular.
So there are-- I would encourage you to look at the specific examples in the notes.
And you'll see there was a simple CSTR example that was index 1 and almost unconstrained, basically.
There was an index 2 DAE in which the consistent initial conditions, you actually couldn't specify any of the initial conditions.
They were fixed by the equations themselves.
But remember, that sort of happened because certain DAEs are pathological in nature.
They often ask us to do something we can't do physically, like try to control the input by-- or try to predict-- let's see, how do I say this?
It's easy for us to predict the output knowing the input.
It's very hard for us to predict the input knowing the output.
And these higher-index DAEs were essentially asking us to do that.
There's a lag in how we measure things.
And it's easier to go forward through a process than backwards through a process and make these predictions.
There's inherent sensitivity in doing the former.
It's the same sort of sensitivity you get from these systems of ODEs, in fact.
A small change in the initial conditions can lead to big changes in the outlet.
So if I measure the outlet, it's hard to predict the inlet.
It's just very inherently difficult to do.
And so formulating a model which has this high index, you will naturally find that your choice of initial conditions for the solution of that model are constrained.
And they're constrained because of the way you formulated the model.
If you're actually trying to do control of a chemical process, it's going to be very difficult to try to somehow control the-- try to make the input be a certain thing by controlling the output.
That doesn't make sense almost, right?
It seems very, very strange.
You would really want to change the input in order to hit a target in the output.
How do you change the output and hit a target in the input?
The process doesn't actually care about things in that way.
And you saw that.
Because in that particular CSTR example, you couldn't fix any of the initial conditions, actually, right?
There was no free choice in the problem.
Everything was automatically fixed, and you had to have perfect knowledge of everything in the system.
I wasn't actually able to do control in a sense there, right?
I could just measure the output, and I can tell you what the input was supposed to be.
And it was super sensitive to my measurement.
So these high-index problems tend to come up when we want to constrain things in a way that's, let's say, physically quite challenging.
High-index problems are incredibly common in mechanical engineering, things like robotics where you want to make sure the robot's arm doesn't whack someone in the head.
This is important.
And so you're trying to constrain all these degrees of freedom and the paths that it's going to move on.
And all those constraints-- I mean, there's actually a very narrow envelope in which this thing gets to operate.
So accurately simulating the motion and then making the robot move that way is very empowering.
In chemical processes, that's true, too.
But oftentimes, we can make choices in model formulation, right?
Like how you try to control the process that can ease those constraints.
If I had some control or it doesn't control in an instantaneous fashion, even if it's a very high-bandwidth control, there's still a finite time for signals to propagate.
There's dynamics associated with the controller that can convert that DAE system into a system of ODEs, for example.
You don't have-- they may be stiff ODEs.
DAEs are the limit of sort of infinite stiffness in ODEs.
But they may not be as troublesome to solve as the DAE system would be.
OK?
It's about model formulation.
OK, can I talk briefly about PDEs?
Is that OK?
I'll tell you a little bit about it.
I like this finite volume method.
I think it's very interesting.
You've got some notes on it.
I think I like it because I think it comes very naturally, actually, to chemical engineers.
Because the problems we like to solve are of the type that finite volume eats for breakfast.
These are problems of conservation.
So we have a conserved quantity b.
Is that what Dean uses in his book?
The new version still uses b as the conserved quantity, right?
So the time rate of change of b is equal to minus the gradient in the flux plus some, I don't know, some volumetric change.
Maybe a reaction, maybe there's some injections somewhere of mass or momentum, or you know, some source.
And our numerical solution, oftentimes, we would like it to also conserve b, right?
Conserve mass, conserve momentum, conserve energy, conserve species.
These collocation methods in general, or relaxation methods in general, don't do that.
They're not designed to do that.
You might be able to construct, for example, basis functions that have that property that they will satisfy conservation.
You might be able to construct certain meshing methods, certain collocation methods-- and this is one of them, the finite volume method-- that respects conservation.
And here's how you do it.
So you worked really hard in transport to derive these differential forms of the transport equations.
But finite volume actually likes the integral form.
So you've got to go back.
So you've got to take the integral of these equations over some volume V. And then the time derivative of the amount of b in this volume is related to the flux of b through the surfaces of that volume plus the amount of generation or consumption of the conserved quantity within that volume V, right?
And this volume is my finite volume.
I take the domain of interest, and I chop it up into all these little volumes.
And then I just need to conserve b within that volume.
So I have the first equation you learned in chemical engineering, right?
The amount of accumulation of the conserved quantity is related to the fluxes in and out of the control volume plus the amount of generation or consumption within the control volume.
This comes very naturally to chemical engineers.
It fits perfectly in the problems we like to solve.
Yeah
Is the nominal vector pointing outwards or inwards?
Well, it all depends on how we choose to write j, actually, right?
So is j the flux into or out of this volume?
So I won't prescribe that here.
It depends on what you chose as the flux.
So maybe, I don't know, what does-- Dean writes the stress tensor with the opposite sign of Bird, Stewart, and Lightfoot.
So these guys, that's the flux of momentum, right?
They choose a sign.
Then the normals have to respect that sign.
Doesn't really matter.
The physics doesn't care.
You just have to choose the flux in the right direction.
If we think of j as the flux into the volume, n should point in.
OK, so you have this conservation equation.
You've got to solve for each of your finite volumes.
And the finite volumes are coupled together, because a flux out of one volume is a flux into another volume, right?
So you can think of these volumes as being little CSTRs.
They're all sitting next to each other.
And they're all linked together by these volumetric flows from one to another.
And that's, in fact, what they are.
The key is, how do I model these fluxes, basically?
The accuracy of the method is going to be set by how big a volume am I going to model.
Can't have infinitely small volumes.
There are going to be too many of them in my domain to solve the problem accurately.
And then, do I have good models for these fluxes?
And you can think about these terms in this way.
That sometimes is useful.
So this big B, the amount of b in the volume, is like the volume times the density of b, the volumetric density of b in the volume.
The average density in the volume.
R is like the volume multiplied by the average rate of generation or consumption.
F is the flux in or out.
It's the sum over all the faces of the flux through each of the faces, which is the area of the face multiplied by the average flux through the face.
So this is the normal to the face, and this is the flux through the face.
And so I just-- I'm going to solve this equation.
I try to figure out, in each volume, what's the average b?
That's what this tells me.
On average, what is b?
On average, what is the concentration?
On average, what is the amount of momentum?
On average, what is the total energy?
That's fine.
So finite volume, very simple.
What's nice about it?
Well, you know, it's really suitable to any sort of geometry you want to look at.
We want to solve this equation, say, in this square domain.
We divide the domain up into a bunch of little square volumes.
And in each volume, we just make sure that the amount of this b balances the fluxes coming through the volume and the amount of consumption or generation of that quantity within the volume.
But it didn't have to be a square domain, right?
A square domain, that's OK.
But maybe I want to solve it on some circular domain.
Well, I can divide that up into volumes, too.
And actually, these equations are the same.
They look identical.
I've just got to calculate the flux through each of the faces.
OK?
But maybe I'm interested in, I don't know, pollution in one of the Great Lakes.
And I want to know how concentration of some pollutant is transformed.
I'd divide this up into a bunch of little volumes, too.
And now I've got to calculate the flux through each of these faces.
And that links all these domains together.
Maybe I'm interested in transport within some porous medium-- say, a bone.
I think it's someone's hip joint here.
OK, well, I divide that up into a bunch of volumes.
And I still solve the exact same conservation equation.
So I've sort of decoupled the problem of meshing and equations, the governing equations associated with the meshes.
As long as I have a way to divide the domain up.
It may be very complicated.
I'm going to go ask a buddy in applied math to do that.
But I'm going to come to him with the right engineering equations for each of those domains, a little equation for a CSTR.
OK?
Turns out this-- you know, one of the most common fluid mechanical codes out there that people use now is OpenFOAM.
So it's very common.
Lots of people use it.
It's a very general solver.
Does finite volume.
That's the method it chooses to use.
Why does it choose to use it?
Because it wants to conserved momentum?
A guaranteed momentum conservation.
Can't lose momentum, can't lose energy.
Can't lose mass.
I just have to pass the right amount between each of these faces.
As long as my calculation of that is accurate, I've conserved the quantity.
Never evaporates.
Finite difference, finite element, different basis set expansions-- they will have this problem that at low levels of accuracy, you can have sort of artificial consumption of, say, the quantity, or artificial production of the conserved quantity of interest.
So this can be a very nice way to solve these problems.
It's another relaxation method.
It's a type of collocation.
I divide the domain up.
And then I try to satisfy conservation equations on each of these little volumes.
Hence finite volume.
OK?
OK.
So how do you solve PDEs by hand or write your own code?
There's a nice little tutorial here.
We'll see how far we're able to get through it before I've got to give you guys your cards.
But I'll illustrate some key points that I think are really important.
So step one, no matter what method you choose, whether you choose collocation, whether you choose Galerkin, whether you're using basis sets or you're using finite elements or finite volumes, you've got some domain you're trying to solve your PDE in.
And you've got to discretize it.
You've got to figure out how to decompose the domain.
So for finite differences, you might decompose it into a series of nodes at which you want to know the value of your function, whatever that function is.
For finite volume, you would decompose it into cells in which you would like to know the average value of your conserved quantity.
For finite elements, you divide the domain up into some points, across which local basis functions span the region.
We would like to know the weighting of these local basis functions near each of these points.
But you've got to divide the domain up.
And the most important thing is you always choose the spacing of the nodes, or the dimensions of the cells, to match the physics.
A very natural thing to want to do is to pick-- say I've got this square grid-- to pick the number of grid points to discretize with.
I say, I'm going to use 100 points in this dimension and 100 points in that dimension.
Actually, that tells me nothing.
That's an irrelevant number.
The physics has no idea how many grid nodes you put in there.
Doesn't care.
There are length scales that are intrinsic to the problem you're trying to study.
And the discretization should resolve those physically relevant length scales.
So it might be the case that I only need a resolution of one part in 10 along this dimension and one part in 10 along this dimension because the physics are such that that's good to get a description of the solution to this problem.
It might be that I need one part in a million in this dimension and one part in 10 in that dimension because the physics are that way.
So you really want to think about length scales, not numbers of nodes.
It's a very common mistake.
You say, well, 100 and 100, that makes sense, or 10 and 10.
Or you know, you pick these randomly.
But you want to think about physics instead.
Here's a good example.
This is my fault.
You'll solve a problem like this.
You already saw it in some sense in the finite element or COMSOL tutorial.
But there's a little drug patch here near this red zone.
It's leaking drug into a sheer flow, and it's all being advected down.
Here's my solution to the problem.
I think it's a pretty good one.
Here's a solution from when I taught of course two years ago that the TAs produced.
And I didn't talk with the TAs about this issue with length scales.
And it sort of got the right structure.
You can see, here's the patch, and there's some advection and diffusion.
But it's not a very stable solution, it turns out.
It's also not very accurate.
It doesn't reflect the essential physics.
And the point is actually the scales.
So here, the patch was 1 centimeter in diameter.
And so here, you got many centimeters long this direction, where you get advection.
Here, you've got 2 centimeters in this direction.
But actually, the solution is really confined down to this small space.
The physics all happens at fractions of a centimeter.
You took Transport, so you know that this is a problem in which there's a boundary layer.
And all the change in concentration is confined to that little boundary layer.
So really, when you do your numerical solution, you'll do like was done up here.
Now the lengths are made dimensionless on the width of the patch, but they're-- you know, you can think of these as measured in centimeters.
But look at the dimension here.
We need to resolve the solution down at the 0.1 centimeter and smaller scale.
I want to put all my little discretization or my nodes down here so that I can see how the concentration field changes as a function of distance.
I have to know something about the physics to do that, though, OK?
Can't just solve this blindly.
It may or may not be the case that COMSOL recognizes the physical situation and remeshes your problem to put mesh points down where they matter.
It may do that.
It may not do that.
It just doesn't know.
People build heuristics to try to solve these remeshing problems, but they may not always apply.
So it's good to understand the physics of the problem.
This drug patch problem, this is a pretty common transport problem.
I mean, did Professor Bazant give you guys a talk during your visit weekend?
Was that this last year?
Was that two years ago?
It was two years ago.
He does problems with bromine flow batteries.
And they worked exactly this way.
You have flow of bromine in.
An electrochemical reaction takes place at the surface.
This entire thing is premised on the fact that you have a very thin boundary layer in which the electrochemical-- in which the reagents are confined.
And if that boundary layer gets too thick and reaches the other electrode, it screws up the reaction and doesn't work as a battery anymore.
So being able to resolve concentration gradients in this small domain, it's not just important for this drug problem.
It's important in general.
It's a common sort of transport-y type problem.
And the numerical solution is really going to hinge on your ability to recognize the physics and resolve the solution over the length scales that are important in the problem.
OK?
So that's the number one most important thing for doing these PDEs.
The rest of the notes will take you through the construction of a numerical solution to the diffusion equation in 2D, and some different scenarios that you might try to employ in MATLAB to solve them, and how you can accelerate the solution using different MATLAB functions.
It's fun to try out and play with.
You'll see that I can make the solution to this problem go very fast for a domain that's quite highly discretized by recognizing certain forms of the equations, choosing certain solvers to solve with.
OK?
Good.
All right, I'll meet you guys in back, and we'll hand out these gift cards.
And good luck on your quiz.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
WILLIAM GREEN,
All right, let's get going.
So today, I'll say a few more words about PDEs, and then we'll leave that topic for a while.
I think I'll actually come back and have a little lecture about that right before Thanksgiving, for those of you still around.
And then we'll start talking about probability, and then that will lead into several lectures about models versus data, which is a very important topic for all of you who either plan to generate data or plan to generate models during your stay here, which probably is everybody.
So we'll talk about that for a while.
So PDEs.
I guess the first comment is, homework 7, have any of you looked at this yet?
[INTERPOSING VOICES] [LAUGHTER] WILLIAM GREEN,
Not a single one
[INAUDIBLE] [LAUGHTER] WILLIAM GREEN,
So homework 7 is the same problem that Kristin showed in class in the demo for COMSOL, and I want you to solve it both ways.
So solve it with COMSOL and then solve it writing your own finite volume code.
And I want to warn you.
This problem has a characteristic length that's way smaller than the dimensions of the problem.
And so in principle, you might need to use an incredibly fine mesh to resolve the gradients in the problem.
So just remember, in case you haven't looked at it, the problem is we have a drug patch.
It has some concentration of the drug.
The drug is diffusing slowly out of the patch into the flow, and we have some flow here like that.
And the characteristic length is at the-- this is a velocity boundary layer.
So the velocity in the x-direction is equal to y times something, dv dy, I guess.
This is some number, and so this has units of-- what is that-- per second, so like a strain rate, OK?
And the diffusion here is controlled by diffusivity D, and that has units of, say, centimeter squared per second.
And so the D over dvx dy, this thing gives a characteristic length squared, which is sort of the natural length scale of this problem.
And the problem is that for the drug molecule, it's a big molecule.
It has a very small diffusivity.
And so therefore, this is a really tiny ratio, and so the L is very small.
And similarly, if you look at it from the point of view in the x-direction, the Peclet number is really gigantic.
And so both of those will tell you that you have to watch out, there might be very sharp gradients in the problem.
And if you just think of it physically, right over here we think the concentration's 0, somewhere over here.
And all of a sudden right here, the concentration is going to be close to the concentration in the patch.
So there's almost a discontinuity in the concentration.
So there's a really sharp gradient on the upstream edge.
And then something funky is going to happen down here at the end of the patch, too.
It won't be quite as abrupt, but could be pretty strange.
All right, so you got it?
OK. And in the problem, we want you to figure out the drug diffusing all the way over to here somewhere, way far over there.
And so you may need quite a few mesh points in the y-direction as well.
All right.
And this kind of problem, this is a very simple case, right?
There's no reactions.
The velocity's just in one direction, and this is not a very hard case.
But you'll see it's actually still pretty tricky to get the right solution.
So don't just believe what the code tells you.
Just run COMSOL and just-- don't believe it's showing the truth.
And don't believe you just write down some finite volumes that you'll get the truth.
So mess around with it and try to convince yourself it's really converged, and you really have the real physical solution.
Because we expect a sharp gradient, say, in the upstream edge, you might want to play with using a finer mesh in the x-direction here than you would down here, because down here presumably the gradients in x-direction are much smaller.
So you don't have to use square finite volumes.
You could use rectangles.
Yes?
I don't understand what you have written [INAUDIBLE] as vx equal y dvx dy.
WILLIAM GREEN,
Yeah
So what-- WILLIAM GREEN,
So that's because the velocity-- the vx is 0 at the wall, and the velocity increases with y.
So vx increases the further you get from y.
This is y equals 0, always going up that way.
As you can increase the height, the flow gets faster.
And that's typical, because you have a no-slip boundary condition at the wall, right?
Is that OK, or did I misunderstand your question?
Is that good?
OK?
So y is a constant.
WILLIAM GREEN,
Yeah, in this particular problem this is just a number that we tell you in the problem
[INAUDIBLE] WILLIAM GREEN,
Right.
If you had a flow in a cylindrical pipe, it wouldn't be-- wouldn't be a number.
Be more complicated, yeah, all right?
So this is-- I mean, again, this is like the simplest you got.
It's only 2D.
It's really simple.
But you're going to see here that even here you could have a lot of trouble.
And if you just don't think about it-- the computer will give you an answer.
Whatever-- you put in some finite difference equations or finite element equations, finite volume equations, fsolve, whatever, it's going to solve and give you some numbers.
It doesn't mean it has any relation to the physical reality.
So this is a problem to really pay attention to whether you're really converged.
All right.
And so what I suggest you do in this problem is go through a sequence where you vary-- the real problem wants you to do this all the way out to-- this is 1 centimeter.
But I suggest you instead solve a simpler problem, where you put the-- here's a simpler problem.
Put your boundary right here, really close, and then solve that problem.
You won't need as many mesh points across to get from here to here.
And this should be-- this is-- suppose you choose this is c equal 0 boundary condition for this wall, then this should be an upper bound.
Because if you put an absorbing layer here, it should drive the diffusion faster, right?
So as you increase this distance-- let's call it little h-- as you increase little h, you should converge to the true solution that you want sort of from above.
Does that makes sense?
Yeah?
Is this OK?
All right.
And similarly, you can vary the mesh, how big your boxes are, say, in delta x and delta y.
And again, for each of those, you could think about how should things converge.
And so look and see if it's actually converging the way you think it should be converging, right?
Any more questions about this?
OK, so this problem might-- this homework problem might look like a MATLAB coding problem.
It has MATLAB coding, but it's not really-- that's not really what it is.
It's really like a-- it's a conceptual problem about what you're doing.
All right.
All right, in this problem, I want you to use finite volumes.
So let's talk about finite volumes for a minute.
So finite volumes is to imagine that we have little control volumes, and each one of them has a little propeller in it, stirs it up, like little CSTRs.
And so we do it just like you did back in your intro ChemE class and mass/energy balances a million years ago.
You know that you have some flow coming in here, and maybe some flow going out there.
And maybe you have some flux come in here and maybe something out there.
And you can add up all the flows in and all the flows out, and then that-- the net of all these fluxes has got to be equal to the accumulation.
And if we're doing a steady-state problem, then there's no time derivative, so the accumulation term should be 0.
And you did a lot of these problems a long time ago, right?
OK, so you just do it like that.
The only problem is now we have a million of these little boxes all coupled to each other.
So you get one equation for each box.
And in this problem, we're going to-- when you do this method, what people assume is that you have a uniform concentration sort of across here or the number you use as the average of the concentration in the cell as [INAUDIBLE].
And so it's not really exactly realistic.
So that's where the approximation is.
What you're doing at the boundaries is very realistic, because you're actually computing the fluxes across control volumes is exactly the way you should do it.
So there's different methods, right, finite element, finite difference, finite volume.
The nice thing about finite volume is you're really treating exactly what's happening across the boundaries of the mesh area, the mesh volume.
And we'll find that a lot of ChemE problems, this is how people prefer to do it.
And because you're treating the fluxes exactly, if you have your mesh volume sitting on the wall or on top of the drug patch, suppose if we're sitting right on top of the drug patch here.
Then if you're sitting-- first, let's do a wall.
Suppose you're sitting on an impermeable wall, then we just know that the flux here will be 0 if it's a wall.
So that's easy.
No flux here.
So we only have to worry about this one, this one, and this one.
That one's not doing anything.
This case, if you say in the drug patch there is a flux, stuff's coming in and you'll need to figure out how to write that boundary condition.
So the boundary condition as written is that C, the drug, is equal to some number, C drug, whatever is in the patch.
But that's not so easy to impose here.
And so there's two ways to look at it.
One way is people compute this flux by considering-- suppose I know this is C drug here.
I'm trying to really figure out what's the average concentration here.
And so one way to compute this is to say it's the diffusivity times C drug minus C the middle over delta y over 2.
That would be the flux.
So that's one way to look at it.
Another way people look at it is they draw what they call a ghost volume.
And so here, here's my C that I care about, C middle.
Here's C ghost.
And here is the line between them.
Now, I can use the same equation here that they would have used before.
But I don't know what C ghost is, because there's no real cell here.
This is below the patch.
Here's the patch.
But imagine the patch is not there for a second, and I write down the same equation I would have here.
And the flux diffusively would have been D C ghost minus C middle over delta y.
That's what you would have written as the diffusive flux.
And I don't know what C ghost is.
So now, I have to think about how do I estimate what C ghost is.
I can say, well, let's do a linear interpolation from this concentration to that concentration.
So let's say that the C boundary is equal to C ghost, the average of these two, plus C mid over 2.
That's what we got if we did a linear interpolation between these two guys to figure out what it is here.
But here we know what it is.
We know what a C boundary is.
That's the concentration of the drug patch so that we can solve for what c ghost is
How do you know what C middle is?
WILLIAM GREEN,
C middle is the unknown.
That's what we're going to compute.
We just want an equation that involves C middle, because we have c middle as an unknown.
We need an equation the involves C middle, just like we would for-- if we have a-- If we have finite volume that is in the interior somewhere, we have c of this grid point, Cij.
We just want equations that involve Cij.
But we need an equation that somehow connects it to the boundary conditions.
Yeah
When would you use that second [INAUDIBLE] if you know C boundary [INAUDIBLE]??
WILLIAM GREEN,
Right.
And actually, I think if you do it this way, in this case, you get the same formula.
You're running out of
So when would you use the C ghost?
WILLIAM GREEN,
The C ghost thing is handy when these conditions-- actually, in the finite volumes, this is it.
I don't think you need to do it.
Maybe if you had a flow, velocity flow here.
I don't know.
But you won't in a wall.
So I don't know if, in the finite difference method, you can use those ghost points as well to do the flux gutter conditions.
And that's often useful to do the ghost thing for if you have a symmetry-imposed flux boundary condition.
Yeah
[INAUDIBLE] WILLIAM GREEN,
That's the flux.
The flux is coming from across this wall.
And we'll have to add that with the flux coming this way and the flux coming this way and the flux coming this way to get our total flux, which is going to net out to 0 instead of C. Is that all right?
But the other ones, you all know how to write them?
They're just velocity flows.
The nice thing about the finite volume method is that when you're trying to consider the velocity, the velocity that matters is actually the velocity here.
It's the flux that couples the two finite values together.
So your velocity is being evaluated halfway between the two mesh point centers.
This turns out to make the whole procedure more numerically stable.
You compute the flux by, say, the difference between these two.
And you're implicitly evaluating right at halfway between.
Other methods, like finite difference, you're trying to get the velocity or the flux at the same point here.
And it doesn't really make much sense, from the point of view of trying to compute how much material's close to here, to use the velocity right there.
And so when you work out equations, and particularly the equations that involve pressure, so suppose you try to discretize the Navier-Stokes equations and you have a pressure gradient that's driving the flow, then you have to figure out where are you going to evaluate the pressures and where are you going to evaluate the velocities.
And when you try to evaluate them at the same point, it turns out that you get numerically unstable problems.
But if you do it by this funny volume method, then it just works out naturally.
It's fine.
And then you need special methods, if you're going to try to do it where you have the velocity of the mesh at the same point as the pressure mesh.
You'll notice in most of the problems we give you, we just leave pressure out.
We try to rewrite the equations so no pressure ever appears.
And that's because pressure in general is a problem, because you can have acoustic waves physically.
And if you do the equations, which allow for acoustic waves, you'll get them in the numerical solution as well.
But usually, we don't care about that.
So you have sound waves racing around your solution, and that causes a lot of trouble numerically.
So a lot of times, people rewrite the equations to try to remove the pressure from the equation.
So you write down the Navier-Stokes equations.
You guys did this transport.
Maybe you'll study that today in the class, in the test.
I don't know.
Anyway, there's a pressure term if you just write naturally.
But oftentimes, you can remove that by an equation of state, for example.
And then you can get rid of it.
And that turns out to be better from the numerical solution.
All right.
Yes
I have a question.
So you said that for this problem, [INAUDIBLE] was very small.
So that means that these finite volume cells [INAUDIBLE] you have to do a lot of them.
[INAUDIBLE] gets around this by having a variable volume.
Do we have to address this on that lab that have some variable volume?
WILLIAM GREEN,
OK, so from the point of view of writing a code, it's a lot easier to write it with fixed mesh because all equations look exactly the same.
So I suggest you start that way.
And what you do, if you use a very small value of each, then I think you'll be able solve it, no problem.
You'll have enough mesh.
Then, once you figure out how to do that, now you might be able to see, OK, can my solver solve this?
And then that gets into another question, actually.
What solver are you going to use?
So any ideas about this problem?
[INAUDIBLE] WILLIAM GREEN,
So you could use fsolve But I'm telling you, you're going to have to use a lot of mesh points.
That means that fsolve's going to have to solve for a lot of variables
WILLIAM GREEN,
Backslash it might be.
So the key thing, the nice thing about this problem is it's a linear differential equation.
There's no nonlinear term today.
So when you rewrite the equations of the finite volumes, it's all going to be linear in the unknowns.
And what else is nice about this problem, when you use local finite volumes as your-- It's going to be super-sparse.
So the matrix that comes in is going to be really sparse.
And so you'll want to use some method that can handle gigantic sparse matrices.
So you wrote a code like that earlier, so that would be one possibility.
If you know how to use the right flags in MATLAB to help their built-in solvers handle sparsity, then that should be good.
If you just ask it to solve it by dense method, by, like, LEU or something, you're going to have lot trouble, once you've put your mesh points in there.
But you can just experiment.
Try bigger and bigger matrices.
And then at some point, if you have backslash, it will give you a warning or something, unless you tell it that you're sparse.
All right?
Any more questions that about this?
How would people solve it do you think professionally?
Suppose I was doing a problem like this in 3D instead of 2D.
How would people solve it?
What solver would they use?
[INAUDIBLE] WILLIAM GREEN,
Not fsolve.
No
[INAUDIBLE] gradient.
WILLIAM GREEN,
Yeah, so they would use conjugate gradient.
So probably-- I think it's called this, BiCGSTAB.
That's the program that we would use if you have a really, really gigantic sparse matrix.
So that's the conjugate gradient.
And Professor Swan talked about, the advantage of that is you never have to actually store the whole matrix.
You only need to evaluate the matrix elements, and then you can throw them away.
And so if you have a very sparse matrix, that's pretty cheap to do.
So it's a really good code.
I think in this 2D problem, you can probably get away with other solvers.
You don't have to use this.
But this is a definite possibility.
This is a built-in MATLAB program as well.
Be warned, though, this is an iterative solver.
It's not just going to be one solve, boom.
And It might have troubles.
So you might want to go with the other ones, but anyway.
May I ask you a question?
How about, do you need initial guess?
What do you think?
Depends.
WILLIAM GREEN,
Depends on the solver.
So if you solve it with backslash, do you need an initial guess?
If you can solve it with fsolve, do you have to initial guess?
If you can solve it via BiCGSTAB, do you have to use initial guess?
Yes.
WILLIAM GREEN,
Yes.
OK.
So then you have to think about how you really get your initial guess, too.
So this is things to think about.
All right.
What else to tell you about?
One last thing about PDEs, and we'll come back to this later, so far we haven't done really anything that's very time-dependent.
But a lot of real world PDEs have a time dependent in them.
And there's is a very important concept, a thing called the CFL number.
And this is named after a 1928 paper, and I'll write the guys' names down.
And what they showed was that, if you're trying to solve the PDE system, where you're discretizing in both x and time, that you have a number that they defined as delta t times the velocity and the extraction divided by delta x.
So that's a dimensionless number.
So that's a CFL number.
You see a lot of papers that will say what CFL number they used.
What that means is the ratio of their time mesh compared to their space mesh.
And conceptually, let's think about what's happening here.
So suppose we have a flow flowing in an upwards direction, and we have a bunch of little finite volumes.
So we've discretized the delta x already.
And this is x.
And there's a flow here.
And I've already decided, somehow, what length scale I want to use.
So I've decided my delta x.
And now, I'm trying to figure out what delta t I should use.
Now, from the point of view of saving CPU time I want the delta to be as giant as possible, because I want to be able to zoom along or predict for long periods of time what's going to happen in my system.
But if I make delta t really large, let's think about what happens.
Suppose I choose delta t to be 10 times delta x divided by u.
So it means that in my one time step, I have some guess or some current value of the concentrations in all these finite values.
And then I wait through a time step that's 10 times delta x over ux.
So I had some stuff that was here.
Where is that going to be 10 times later?
10 time steps later.
10 blocks up, right?
So it's going to be, like, way up here somewhere.
And so what's going to happen there is that my numerical methods are all computing stuff locally from the spatial derivatives.
But it's crazy if, between my time steps, this stuff completely left the picture.
It's already convected all the way off the screen.
And some new stuff, which was way down here before, is now the stuff that's here.
Should be there if I was physical.
Numerically, who knows what will happen if you try to do this.
But it won't be good.
So the condition is that you need this number to be less than 1, or same order of magnitude as 1, and you try to make this much bigger than 1, then you're doing something crazy because you're convecting stuff over multiple mesh points.
And so that turns out to be a very serious limitation if you try to do simulations for, let's say, a reacting flow for a long period of time, because you might have to use a really tiny delta t. And then people have developed all different fancy methods to try to get around that.
But if you just do the obvious things to do, you'll always run into this limitation.
Then you need to choose the time steps small.
And also, it's bad, because as you make delta x smaller, which improves your accuracy, You'll have to make your delta t's smaller, too.
But of course, making delta x smaller increases your CPU time because you have more finite volumes to compute.
And then you'll also have to make double t smaller, which means you'll have to do more time steps, too.
So it's even like a double whammy.
So getting more accuracy is going to really cost you badly.
And so this another reason why people used to always refer to color fluid dynamics.
You can make a pretty picture, but it might not be physical at all.
Because you can make it solve equations that maybe, for example, didn't impose this, then who knows what kind of crazy stuff you'll get.
You'll got something.
It'll compute something, but it may have no relation to the real problem.
I think that's all I was say about PDEs.
Are there any questions about PDEs before I start talking about probability?
You got it totally down.
I'm looking forward to some really awesome solutions.
How about that?
Just one last comment.
If you decide you wanted to do adaptive meshing and you want to change your mesh size, you can choose measures like this if you want.
And you can even do things like this, where you have a bigger mesh, and then maybe have two smaller meshes underneath it in the next trial.
So you can just have stuff flowing here, stuff flowing here stuff flowing there.
So you can do all kinds of crazy stuff like.
This can really help improve the accuracy of the solution a lot.
But it's, I would say, very prone to bugs.
So if you do this, be really careful and don't do it too often, I would say.
You might have a few boundaries where you do something funky like that to change that mesh size.
But don't go crazy with it.
[INAUDIBLE] is smart.
It has a really nice way of doing the meshing for you.
So that's its advantage, that somebody very carefully coded how to handle this kind of stuff in a general case.
But if you guys are doing it for the first time, it might not be so good.
So that's enough of PDEs.
Let's talk about probability.
So probability is everywhere, except in undergraduate ChemE homework problems.
So when you do problems as an undergraduate, they always had some nice solution.
It was 2 pi, it was 3.0.
Everything was, like, deterministic, is definite.
The grader could go through.
Oh, no, you're off by 0.1.
It couldn't possibly be right.
That's, like, not reality.
So any time you actually make a measurement, you always had measurement noise.
And if you try to repeat a measurement, you don't get the same result.
So that's, like, completely different than an undergraduate problem.
But this is the reality.
So the reality is the world is, like, not so nice as you think.
But actually, it's even more fundamental.
It's nothing about-- I mean, there's one problem about how good an experimental those people are, and how fancy an apparatus you bought that can make things exactly reproducible.
But even if you do that perfectly, the equations we use really don't correspond to the real physical reality.
So we always use the continuum equations.
You guys probably are studying them a lot in 1040 and 1050, especially 1050, I guess.
But those equations are really all derived from averages over ensembles or little finite volumes or something if you look at the derivation of the equations.
And reality is that the world is full of molecules.
And so they're all wiggling around.
And if you look in a little finite volume, you look right now, you'll see that there's 27 molecules in there.
If you look a second later, there might be 28.
Then a little later, maybe 26.
It's always fluctuating around.
But according to our average equations that we use, like in the Navier-Stokes equations, it always 27.3.
But of course, there's not 0.3 of a molecule.
So, I mean, explicitly, it's the average is what we're computing, and the reality is fluctuating around the edges.
Same thing in thermo.
We say that such-and-such has a certain amount of energy.
But you guys have some [INAUDIBLE] already, yes?
So you saw that's not true?
So really, all that's saying is that's, like, the probability, the average.
If you've had many, many ensembles that were exactly the same and you average them all, you get some number and that's your average energy.
But for any actual realization, it has some different value of the energy.
And it's even worse than that.
That's because we have a lot of particles.
You can even go down to, like, the microscopic level, where you have one molecule.
And you calculate things about that, it turns out you have to use the Schrodinger equation for that.
And the Schrodinger equation explicitly only gives you probability densities.
So it just tells you the probability the molecule might be somewhere, the electron might be somewhere, the energy might be something.
But it's not actually whether it really is.
It's just saying a probability distribution.
And every time you do the experiment on the molecule, you get a different result.
Now, this is super-annoying, but it's the way life is.
Einstein got so annoyed, he has a famous saying, and it's "God does not play dice."
He was, like, just completely annoyed at these equations.
But it's the way it is.
So the reality is that things, all we know about, really, are probability distributions.
In most of our work, in our lives, we always talk about, like, the mean or the median.
And we're talking as if it's a real number.
But really, it's always some distribution.
So it's time, I guess, you're in graduate school, it's time to, like, face up to this.
And that's what we're going to talk about for the next week or two
[INAUDIBLE] WILLIAM GREEN,
This also makes it makes you wonder what you're doing when you make a measurement.
So if you make a measurement, first of all, the fact that when you measure something repeatedly, you're not going to get the same number, that's alarming.
Because I want to say I'm 5 foot 9", this should be pretty common, I'm always 5 foot 9".
But actually, if you measure me multiple times, sometimes you'll get 5' 9 and 1/4" Some people will get 5' 8 and 3/4" So it might make you worry, did I change?
Did I grow between the measurements?
So that's one issue.
Because our experiments are not repeatable and we get different numbers every time we make a measurement, then we have a big problem.
Somebody says, well, I really want to know how tall Professor Green is.
I've always wondered, how tall is he?
And everybody's told me a different number.
And when you go measure it again, you get a different number again.
What's going on?
And so you'll have to then-- then we have, like, a concept that there is a true height of Professor Green and we just don't know what it is.
And then we'll try maybe to make repeated measurements of my height, and then maybe take the average.
That would be the obvious thing to do.
And we take the average and report that.
We'll tell the boss, we'll lie to him, say, oh, Professor Green's 5 foot 9".
When really, we never actually measured 5' foot 9".
Every time we measured, it was 5' 9 and 1/8", 5' 8 and 3/4".
Every time, it was something different.
But we just say, OK, it's 5' 9".
And the boss, he doesn't want to know about all this complexity, anyway, so he just believes you.
So it takes your average number.
But you know that you're not really sure I'm exactly 5' 9" And so you have a probability distribution, and you're doing your best guess.
And in fact, if you're honest to your boss, you'll give him an error bar.
You'll say he's 5' 9", plus or minus 1/2 an inch.
And that way, what you're saying is you're pretty sure the true height of Professor Green is somewhere in that range.
Is this OK?
Yeah.
Now it might be that I'm actually changing height.
I get a good night's rest, I lie down for a long time.
Maybe when I stretch out a little bit.
When I stand up in front of lecture here for a long time, I'm shrinking.
My vertebrae are being compressed by standing here so long.
So it's, like, a combination of things.
One is your measurement system is not perfect, and one is that I actually might be fluctuating.
I had a big breakfast, I'm growing.
So is it true with everything?
Every experiment you do is like this, that there is a real fluctuation of the real, physical apparatus of the thing that you're trying to measure because mostly, things we're trying to measure have fluctuations intrinsically.
And then on top of that, your measurement device is fluctuating, which means-- and then the combination is what you're measuring.
It gives you fluctuation.
If you have a very nice instrument, the fluctuation of your instrument is smaller than the fluctuation in the real system and you'll go down to the limit.
Anybody with a really good instrument should measure approximately the same probability distribution, which is actually the real fluctuation.
My height, for example.
So if you bought a laser interferometer and mounted a mirror on my head, and measured my height to the wavelength of light, you're pretty sure that it's pretty good.
It's within a wavelength of light.
So the error bar there is just due to the fact that I slouch sometimes.
Is that OK?
So let's talk about some basic things about probability.
So we're always saying there's a probability of an event.
And so we want to give a number.
So for example, I'm flipping coins.
I flip a penny, it could be heads or tails.
I'll say the probability of heads is approximately equal to 1/2.
So I flipped the coin, and if you flip the coin 100 times, you might expect to see 50 heads.
Now, it could be 49 heads, it could be 51 heads.
So you have to worry about, like, exactly how precisely you know it.
But you think it's something like a half.
Now, just to warn you, I actually didn't specify any more significant figures here, and it might be really hard to figure out what those significant figures are.
And this is related to the measurement problem.
So we think that a coin has about a 50/50 chance of being heads or tails.
But if you really wanted to prove it, that might be really hard to do.
You can have joint probabilities.
So suppose I have a penny and I have a dime.
And I try to think, if I flipped them both, what could happen?
I could have that they both come up heads.
I could have this guy come up heads, this guy tails.
This guy tails, this heads, tails.
So there's, like, four possible outcomes, and we think they all have about approximately equal probabilities.
So the probability of any of these things happening should be about 1/4.
So you can write the probability of event 1 and event 2.
So this would be, for example, the probability that I got heads for penny and heads for dime, and I think that this is about 1/4.
Now, I could also say the probability of heads, and heads is equal to the probability of heads for the penny times the probability of heads for the dime if I got heads for the penny.
Now, if these two coin flips are completely uncorrelated, then the probability of heads on the dime and heads on the penny is the same.
It's just the probability of the heads on the dime.
So they don't matter.
But many things that we'll study are correlated.
That the probability of something happens depends on whether something else happened.
So this kind of expression is very important.
Now, this is just an equality.
It's like a definition of what these are, right?
And I'll just notice you can write that the other way around.
So it's the probability of heads on the dime times the probability of heads on the penny.
This is OK?
So these two guys are equal to each other, and you can rearrange that equation any way you want.
And we'll come back to a very famous way to rewrite that equation.
It's called Bayes' theorem, and that turns out to be really important in model versus data comparisons.
Instead of doing AND, do OR.
So maybe you can say, then, what's the probability that I see at least one head?
So I flip my two coins, and I have the probability of, I see at least one head.
So we know we intuitively the answer is 3/4.
But let's try to think of where does that really come from.
So-- [HIGH-PITCHED SOUND] What is that?
Sorry.
Really threw me there.
So probability of at least one head, it's not equal to the probability of head for the penny plus the probability of head for the dime, because we know this is really 3/4, and this is 1/2, this is 1/2.
You add them up, 1/2 plus 1/2 does not equal to 3/4.
So be careful.
There's a lot of things you can say quickly that are not true.
So anyway, you really have to consider the whole thing.
And in the best case, if you can enumerate what's going to happen, it's very simple to add up things.
Otherwise, you have to be very careful with the algebra to make sure you add all things correctly.
Let's see at least what this should be equal to.
So this should be equal to the probability that a head on a penny times the probability that I have head for the dime if I had a head for the penny plus probability of the tail for the dime or for the penny plus the probability that I have for the dime and then times something like this, too.
And in a case like this, where I'm summing over all of the possibilities.
So in this case, I'm either going to get a head for the dime or a tail for the dime.
It's not going to balance on its end.
I'm assuming that that chance is 0.
Then these two things just add up to 1.
Is that all right?
It's really saying the probability of the dime will do something if I have a head for the penny.
Is this all right?
So you want to practice doing little algebra things like this to make sure you know how to do it.
Yes?
No?
Maybe?
Let's see what else I've got here
Professor.
WILLIAM GREEN,
Yes
What you just wrote, [INAUDIBLE] WILLIAM GREEN,
Yeah, what's the correct thing to write in here?
What is the right thing?
Tails.
WILLIAM GREEN,
Yeah.
So it's tricky.
Yeah
The probability [INAUDIBLE] WILLIAM GREEN,
Yeah, it's the easy way to do it, for this case.
And so there's, like, a lot of-- for some cases, it might be easier to write it this way.
In some cases, write it that way.
It depends on how many different options there are.
So anyway, I'm just trying to warn you by writing this out.
It's like, you might write down stuff quickly without thinking about it.
And it's easy to double count.
Like for example, if you put the other term in here, you double count.
Because you already have, they had head, you already had the head case here.
Yeah, maybe-- Did I write the-- I'll just write the base theorem down the way you usually see it.
So this is the general expression, which is always true.
The base theorem way to write it is the probability of A given B is equal to the probability of B given A times the probability of A over the probability of B.
And that's just rearranging this equality.
Rearrange again.
And we'll come back to this with the situation that's like, what's the probability that Professor Green is 5' 9", given our measurements, is related to the probability that if Professor Green was 5' 9", we would have made the measurements we got.
So this is like that way to invert that statement.
And if the thing that's here is exclusive, it means there's, like, many possible things that could happen.
This is the probability that one of them happened.
There's a lot of other things that could have happened.
For instance, like heads and tails, it's either heads or it's tails.
It's exclusive.
If it's like that, you could rewrite that probability of B is equal to probability of Aj, probability of B given Aj summed over j.
This is something, whatever A measurement it is, if it ends up with B, that's probability B.
Is that all right?
So you can put that into the denominator here.
You can substitute that in so you can rewrite this as probability of Ai given B is equal to the probability of B given Ai times the probability of Ai divided by the sum of the probability of Aj [INAUDIBLE] Aj.
And this is the form that you'll normally see based here.
A lot of times, we have a continuous variable instead of discrete events like this.
And so then we talk about probability distributions.
And so instead of having a sum there, we might have an integral.
And this is a topic that also is quite confusing to many people.
So suppose I had a Maxwell-Boltzmann distribution.
And I have, like, the probability density of having a certain velocity in the extraction instead of the particle.
And so that's going to be something like e to the negative 1/2 mvx squared over keT divided by something.
And maybe it actually is [INAUDIBLE]..
I'm not sure.
Maybe there's a vx here.
Something like that.
So you have an expression that you get for [INAUDIBLE] for probability density.
Now, what does this mean?
We want this thing to be the integral of P of vx dvx over from negative infinity to infinity.
We want this to equal something.
What do we want this to equal?
1.
So that means that the units of this, this is units of centimeters per second, what's the units of this?
[INAUDIBLE] WILLIAM GREEN,
Centimeters per second now minus 1.
It's, like, it's per the unit to this to get a dimensionless number there.
So this has units of seconds per centimeter, which probably none of you thought until I just said that to you.
So probability densities are tricky, and they always have to be multiplied by a delta.
When I talk about this, I really need to talk about P of vx, delta vx.
I need to have something here to make this look like a probability again.
And so the issue is that the probability that the velocity is exactly something is, like, 0.
It's really the probability that's in a certain range, plus or minus something.
Then you get a nonzero probability.
There's another quantity you'll see a lot called the cumulative probability distribution.
Let's see, what letter do I use?
Call it F. And this would be like the integral from negative infinity to vx prime, or vx of P over vx prime, dvx prime.
And this is the probability that the particle has vx less than something.
So this is F is equal to the probability that vx-- let's call this vx star-- vx is less than or equal to vx star.
And that can be quite an important property.
So for example, you're designing a supersonic nozzle.
You want to know what gas molecules are going to come out at a certain speed.
You really need to know that probability.
How many of them are going to be bigger than that speed, how many less than that speed?
So these are two different ways to express a similar thing.
This is, like, the probably that it does have that speed within a certain range.
This is the probability that it has anything less than or equal to that speed.
And in a completely different way, this is an integral.
This has units of dimensionless.
This has units of per velocity.
All right?
All right, we'll pick up more of this on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so I know some of you have succeeded to do the homework and some of you, I think, have not.
Is this correct?
Yeah
OK.
So I was wondering if someone who has succeeded to do their homework might comment on how small a mesh do you need to converge
[INAUDIBLE]
It's about l?
L?
OK, so you need something on the order of l to converge.
Is that correct?
So if you're trying to do the problem using mesh much bigger than l, you should probably try a tighter mesh.
Yes?
[INAUDIBLE]
All right.
Yes?
[INAUDIBLE]
Yes.
Yes.
All right.
And has anyone managed to get the [INAUDIBLE] solution to actually be consistent with the [INAUDIBLE] solution?
Something like 3% or 4% or so
3% or 4%, OK. And I assume that the [INAUDIBLE] is also using a mesh of similar size?
Hard to tell?
I used like a triangular system--
Yeah, yeah, but I mean, it's really, really tiny ones at the bottom?
If you want me to blow it up, I can just take a look and see to be sure.
All right, and is backslash able to handle a million by million matrix?
Like 10 seconds with [INAUDIBLE]
[INAUDIBLE] OK.
So you need to-- so do the sparse allocation.
And MATLAB is so smart that it just can handle it with a million by million, which is pretty amazing, actually.
That's a pretty big matrix.
All right, sorry, this is too loud.
All right, so last time, we were doing some elementary things about probability.
Actually, any more questions about the homework problem before we get started?
What's the answer?
What's the answer?
You could ask your classmates.
Any other questions?
All right.
So I had you confused a little bit with this formula probability of either A or B.
So I asked what the probability of-- I flipped two coins-- that one of them would be a head.
And I could see a lot of consternation.
The general formula for this is it's the probability of A plus the probability of B minus the probability of A and B.
It can't just be the two of them added together because if you have 50% chance for head of the penny and the dime is 50% chance, this would add up to 100% chance that you'll get a head, but you know sometimes it's true.
So this is the formula.
And then the probability of A and B is often written in terms of the conditional probabilities, the probability of A times the probability that B would happen given that A already happened, which is also equal to the other way around.
And this has to be read carefully.
It means B already happened, and then you want to know the probability of A given that B already happened.
So it's sort of like this way-- I don't know-- the way I think about it.
Like, this happened first.
and now I'm checking the probability that that's going to happen.
Now, a nice little example of this is given in [INAUDIBLE] textbook.
And I think it's nice enough that it's worthwhile to spend a few minutes talking about it.
So he was-- [INAUDIBLE] who wrote the textbook, was not actually a numerical guy.
He was a polymer chemist.
And so he gave a nice polymer example.
So if you have a polymer and the monomers have some big molecule, and at one side, they have a sort of acceptor group, and the other side, some kind of donor group-- we'll call it D, I guess.
And these are the monomers.
And so they can link together.
The donor can react to the acceptor.
So you can end up with things like this and so on.
So this is the monomer.
This is the dimer.
Then you could keep on [INAUDIBLE] like this.
And many, many, many of the materials you use every day, the fabrics in the seats that you're sitting on, the backs of the seats, your clothing, the binder holding the chalk together, all this stuff is made from polymers like this.
So this is a pretty important, actually, practical problem.
And so you start with the monomers, and they react where you have A reacting plus D, over and over again.
And we want to understand the statistics of what chain lengths are going to make, maybe what weight percent or what would the average microweight be, something like that would be the kind of things we care about.
So a way to think about it is if I've reacted this to some extent and I just grab a random polymer chain, any molecule in there, and I look and find, let's say, the unreacted D end-- so any oligomer is going to have one unreacted D end.
You can see no matter how long I make it, there will still be one unreacted D end.
And I'm neglecting the possibility this might circle around and make a loop.
So assuming no loops, then any molecule I grab is going to have one unreacted D end.
So I grab a molecule.
I start at the unreacted D end, and I look at the A that's next to it.
And I say, is that A reacted or not?
So if it's a monomer, I grab the D. I look over here.
The A is unreacted.
So the probability that it's a monomer is going to be equal sort of like 1 minus P where P is the probability that As react.
So it didn't react, so just like that.
This one, the one next to it has reacted.
So this is just going to be the probability of a dimer is the probability that my nearest neighbor reacted and next neighbor is unreacted, right?
Is that OK?
So I can write this way.
I could say, what's the probability that my nearest reacted times a conditional probability, next unreacted if nearest is reacted?
So far, so good?
You guys are OK with this?
So I grabbed a chain.
I'm trying to see if it's a dimer.
I'm going to calculate the probability that this next acceptor group has been reacted to a donor group.
If it has reacted, then I'm going to check the next one after that.
So this is the nearest neighbor.
This is the next nearest neighbor.
And I want this to be unreacted.
If that's both true, then I have a dimer.
If either one of those is false, [INAUDIBLE].. Is that OK?
So now I need to have a probability.
So what's the probability that the nearest one is reacted?
There's some probability that things have reacted.
So this is going to be my P, probability that things reacted.
And I wanted this to be unreacted.
Now, there's a question.
Are these correlated or not?
Now, in reality, everything's correlated to everything.
So probably, they're correlated.
But if we're trying to make a model and think about it, the fact that this thing reacted at this side doesn't really affect this side if this is a big enough [INAUDIBLE] So to a good approximation, this is independent of whether or not it's reacted or not.
So this is still going to have the ordinary probability of being unreacted, which would be 1 minus P. So I could write down that the probability of being a monomer is equal to 1 minus P. The probability of being a dimer is equal to P times 1 minus P. What's the probability of being a trimer?
P squared times 1 minus P. And in general, the probability of being an n-mer is equal to P n minus 1 times 1 minus P. So now you guys are statistical polymer chemists.
So this derivation was derived by a guy named Flory.
He got the Nobel Prize.
He's a pretty important guy.
If you want to learn a lot about him, I think both Professor Cohen and Professor Rutledge teach classes that are basically, learn what Mr. Flory figured out.
Well, maybe that's a little bit too strong, but pretty much.
There's another guy named [INAUDIBLE] that did a bit too, so [INAUDIBLE] and Flory.
Basically everything about polymers worked out by at these guys.
And all they did was just probability theory, so it was a piece of cake.
And so this is the probability that you have an n-mer.
So now we can compute things like, what is the expectation value of the chain length?
How many guys link together?
And that's defined to be the sum of n times the probability of n. So that, in this case, is going to be sum of n times P to the n minus 1 times 1 minus P. Now, a lot of these kinds of simple series summations, there's formulas for it.
And maybe in high school, you guys might have studied series.
I don't know if you remember.
And so you can look up.
And some of these have analytical formulas that are really simple.
But you can just leave it this way too, because you get a value numerically with MATLAB, no trouble.
You can also figure out what is the concentration of oligomers with n units in them.
And so that's going to be equal to the total concentration of polymers times the probability that it has n. So this one, we just worked out.
The total concentration, a way to figure that out is to think about there's one monomer or one monomer-- I'll call this a polymer too.
This is a polymer with one unit.
There's one polymer molecule per unreacted end, unreacted D end.
So it's really, I want to know how many are unreacted.
So that's going to be 1 minus P times the amount of monomer I had to start with.
It could be A or D. It doesn't matter.
It's like, how many of them-- I started with a certain amount of free ends.
What fraction of them have reacted based on 1 minus P. Yeah, it's 1 minus P. So as P goes-- well, yeah, it goes backwards.
Yeah, as P goes to infinity-- I think that's right.
Yeah, when P is-- well, I'm totally confused here now.
1 minus P sound right?
Maybe I did the reasoning backwards.
This is definitely the right formula.
I'm just confusing myself with my language.
This is a, at least for me, endemic problem with probability is you could say things very glibly.
You've got to think of exactly what you mean.
So the concentration of unreacted ends, so initially, this was equal to A.
It was all unreacted ends.
And as the process proceeds, as P increases, then at the end, it's going to be very small.
So this is right.
And the concentration of unreacted ends is equal to the total concentration of polymers, the number of polymers [INAUDIBLE]..
So it's this times P n minus 1 times [INAUDIBLE].. All right, and this is called the Flory redistribution.
And that gives the concentrations of all your oligomers after you do a polymerization if they're all uncorrelated and you don't form any loops.
It's often very important to know the width of the distribution.
If you make a polymer, you want to make things have as monodisperse as possible.
It's because you'd really like to make this pure chemical.
There's some polymer chain length which is optimal for your purpose.
You want to try to make sure that the average value, average value, this is going to be equal to the value you want.
So you want to keep running P up until you reach the point where the average chain length is the chain length that's optimal for your application.
If you make the polymer too long, then it's going to be hard to dissolve it.
It's going to be hard to handle and it's can be solid.
If you make it too short, then it may not have the mechanical properties you need for the polymer to have.
So there's some optimal choice.
So you typically run the conversion until P reaches a number so that this is your optimal value, but then you care about what's the dispersion about that optimal value.
And particularly, the unreacted monomers that are left might be a problem because they might leach out over time because they might still be liquids, or even gases that come out.
So this famous problem, people made baby bottles and they have some leftover small molecules in the baby bottles.
And then they can leach out into the milk, and the mothers don't appreciate that.
So there's a lot of real practical problems about how to do this.
So anyway, you'd be interested in the width of the distribution.
So we define what's called the variance.
And the variance of n is written this way.
And it's just defined to be the expectation value of n squared minus the expectation value of n squared.
These two are always different or almost always different, so it's not 0.
So this is equal to the summation of n squared times the probability of n minus-- all right?
And a lot of times in the polymer field, what they'll take is they'll take the square root of this and they'll compare sigma n divided by expectation value of n. This is a dimensionless number because sigma n will have the dimensions.
Sigma squared is dimensions n squared.
This is dimensions of n, so it's a dimensionless number.
And that's-- I think they call it dispersity of polymer, something like that.
Now, notice that when we use these [INAUDIBLE],, when we wrote it this way, it's implicitly that these things are divided by the summation of the probability of n. But because these probabilities sum to 1, I can just leave it out.
But sometimes, it may be difficult for you to figure out exactly what the probabilities are and you'll need a scaling factor to force this thing to be equal to 1.
So sometimes, people leave these in the denominator.
There's another thing you might care about, which would be like, what's the weight percent of Pn?
So what fraction of the weight of the polymer is my particular oligomer, Pn?
[INAUDIBLE] sorry, some special one, Pm.
And I want to know its weight percent.
So that's going to be equal to the weight of Pm in the mix divided by the total weight.
So that's equal to the weight of m times the probability of m divided by the total weight, which is going to be the weight of all these guys, times the probability of each of them.
And you can see this is different.
This is not the same as-- not equal to, right?
It's not the same thing.
So just watch out when you do this.
And in fact, in the polymer world, they always have to say, I did weight average.
I did number average, because they're different.
Is this OK?
Yeah?
So I would-- my general confidence, at least for me, if I skip steps, I always get it wrong when I do probability.
So don't skip steps.
Do it one by one by one, what you really mean.
Then you'll be OK. All right, now, this is a cute little example.
It's discrete variables.
It's easy to count everything.
Very often, we care about probability distributions of continuous variables.
And we have to do those probability density functions that I talked about last time which have units in them.
And so as we mentioned last time, if you want to know the probability that a continuous variable, that x is a member of this interval, the probability this is true is equal to Px of x hat times dx.
And so this quantity has units of 1 over x, whatever the units of x are.
And then you have to multiply it by x in order to get the units to be dimensionless, which is what the probability is.
And this is like obvious things, like P Px of x prime dx prime.
[INAUDIBLE] value as possible of x has got to be equal to 1.
It's a probability, which is the same as saying that probability of x is some value anywhere is 1.
So there's some [INAUDIBLE] you measure.
And you can also have a probability that x is less than or equal to x prime.
And that's the integral from negative infinity to x prime of Px of x dx.
And the mean is just the integral of x Px.
And you can compute the x squared.
The average of x and x squared, same thing.
You average of anything.
You can put these together.
You can get sigma x squared is equal to x squared minus the average squared.
So that's the variance of x.
You can also do this with any function.
So you can say that the average value of a function is equal to the integral of f of x Px of x dx.
This is an average value of a function of a random variable described by probability density function with P of x.
And then you can get things like sigma f squared is equal to the integral of f of x, quantity squared, Px of x minus-- all right?
Everything's OK?
Yeah.
All right, so a lot of times, people are going to say, we do sampling from Px.
So sampling from Px means that we have some probability distribution function, Px of x, and we want to have one value of x that we draw from that probability distribution.
When we say it that way, we mean that we're more likely to find x's where Px has a high value and we're less likely to draw an x value that Px has a low value.
So that's what's sampling from.
Now, you can do that mathematically using random number generators in MATLAB, for example, and we'll do that sometimes.
But you do it all the time when you do experiments.
So the experiment has some probability density function that you're going observe something, you're going to measure something.
And you don't know what that distribution is, but every time you make a measurement, you're sampling from that distribution.
So that's the key conceptual idea is that there is a Px of x out there for our measurement.
So you're trying to measure how tall I am.
Every time you measure it, you're drawing from a distribution of experimental measurements of Professor Green's height.
And there is some Px of x that exists even though you don't know what it is.
And each time you make the measurement, you're drawing numbers from that distribution.
And if you draw a lot of them, then you can do an average.
And it should be an average that's close to this.
If you drew an infinite number of values, then you're sampling this.
You can make a histogram plot of the heights you measure of me, and it should have some shape that's similar to Px of x.
Does that makes sense?
All right.
So actually, everyday you're drawing from probability distributions.
You just didn't know it.
It's like [INAUDIBLE] street.
The probability the bus is going to hit me or not and the bus driver is going to stop, I think there's a high probability, but I'm always a little worried, actually.
Good.
I'm drawing from-- it's a particular instance of that probability distribution about whether the bus driver's really going to stop or not.
And if I sample enough times, I might be dead.
But anyway, all right.
Often we have multiple variables.
So you can write down-- you can define Px hat.
So now I have multiple x's.
It's like more than one variable of x.
And I wanted the probability density function of them.
I'm going to measure this and this and this and this, all right?
And this is equal to the probability that x1 is a member of the set, x1, x1 plus dx1, and x2 is a member of x2, x2 plus dx2, and that.
That's what probability density function means with multiple variables.
So this is very common for us because we often measure and experiment more than one thing, right?
So you measure the flow rate and the temperature.
You measure the yield and the absorption at some wavelength that corresponds to an impurity.
Usually when you experiment, you often measure multiple things.
And so you're sampling from multiple observable simultaneously.
And implicitly, you're sampling from some complicated PDF like this even though you don't know the shape of the PDF usually to start with.
And so then when you have this multiple variable case, you can define a thing called the covariance matrix where the elements of the matrix Cij are equal to xi xj, the mean of that product, minus xi xj.
And so you can see that, for example, sigma i squared is equal to Cii, but the diagonal elements are just the variances.
But now we have the covariances because we measured, let's say, two things.
All right, so suppose we do n measurements and we compute the average of our repeats.
So we'd just repeat the same measurements over and over.
So suppose you measure my height and my weight.
Every time I go to the medical clinic, they always measure my height, my weight, my blood pressure.
You've got three numbers.
And I could go back in there 47 times, and they'll do it 47 times.
And if a different technician measured it using different [INAUDIBLE] and a different scale, I might get a different number.
Sometimes, I forget to take my shoes off so I'm a little bit taller than I would have been.
So the numbers go up and down.
They fluctuate, right?
You'd expect that, right?
If you looked at my medical chart, it's not the same number every time.
But you'd think, if everything's right in the world, that I'm an old guy.
I've been going to the medical clinic for a long time.
If I look at my chart and average all those numbers, it should be somewhere close to the true value of those numbers.
So I should have that the average values experimentally, which I just define to be the averages-- this is the number of experiments.
OK, so I can have these averages.
And I would expect that as n goes to infinity, I hope that my experimental values go to the same value of x that I would have gotten from the true probability distribution function.
If I knew what Px of x is and I evaluated the integral and I got x, I think it should be the same as the experiment as long as I did enough repeats.
So this is almost like an article of faith here, yeah?
It's what you'd expect.
Now, the interesting thing about this-- I mean, probably you've done this a lot.
You probably did experiments and you've averaged some things before, right?
If everybody in the class tried to measure how tall I was, you guys all wouldn't get the same number.
But you'd think that if you took the average of the whole classroom, it might be pretty close to my true height, right?
So the key idea here is that the sigma squared of the x measurement experimental, which we define to be this-- maybe we should do this one at a time.
[INAUDIBLE] Then I can have a vector of these guys for all the different measurements.
So there's some error in my height.
There's some error in my weight.
There's some different error in my blood pressure measurement, but each should have their own variances.
I can have the covariances.
OK, so these are all the experimental quantities.
You guys maybe even computed all these before in your life.
And we expect that this should go like this as n goes to infinity.
Now what's going to happen to these guys as n goes to infinity?
That's the really important question.
So there's an amazing theory called the central limit theorem of statistics.
And what this theorem says, that as n gets large and if trials are uncorrelated and the x's aren't correlated, which is the same as saying that Cij is equal to 0 off the diagonal, then the probability of making the measurement x is proportional to the Gaussian, the bell curve.
All right?
So this is only true as n gets very large.
It doesn't specify exactly how large has to be, but it's true for any Px, any distribution function, probability distribution function.
So everything becomes a bell curve if you look at the averages.
And sigma i squared in that limit goes to 1 over n sigma xi squared experimental.
And this is really important.
So what this says is that the width of this Gaussian distribution gets narrower and narrower as you increase the number of repeated experiments or increase the number of samples.
So this is really saying that the uncertainty in the mean is scaling as 1 over root n where n is the number of samples or number of experiments that's repeated.
Now, sigma, the variance, is not like that at all.
So this quantity, actually as you increase n, just goes to a constant.
It goes to whatever the real variance is, which if you're measuring me, it might how good your ruler or something.
It'll tell you roughly what the real variance is.
And that number does not go to 0 as the number of repeats happens.
I mean, I could get the whole student body to measure how tall I am at MIT, and they're still not going to have 0 variance.
It's going to still be some variance, right?
So this quantity stays constant as n increases or goes to a constant value once it sort of stabilizes.
You have to have enough samples.
But this quantity, the uncertainty in the mean value, gets smaller and smaller and smaller as the square of n. Now, this is only true in the limit as n is large.
Now, this is a huge problem because experimentalists are lazy, and you don't want to do that many measurements.
And it's hard to do a measurement.
So for example, the Higgs boson was discovered, what, a year and a half ago, two years ago?
And I think altogether, they had like nine observations or something when they reported it, OK?
So nine is not infinity.
And so they don't have infinitely small error bars on that measurement.
And in fact, who knows if it really looks like a Gaussian distribution from such a small sample, but they still reported 90% confidence interval using the Gaussian distribution formula to figure out confidence intervals.
So everybody does this.
If n is big, it should be right.
And you could prove mathematically it's right, but the formula doesn't really tell you how big is big.
So this is like a general problem.
And it leads to us oftentimes misestimating how accurate our results are because we're going to use formulas that are based on-- assuming that we've averaged enough repeats that we're in this limit where we can use the Gaussian formulas and get this nice limit formula.
But in fact, we haven't really reach that because we haven't done enough repeats.
So anyway, this is just the way life is.
That's the way life is.
And I think there's even discussions in statistics journals and stuff about how to make corrections and use slightly better forms that get the fact that your distribution of the mean doesn't narrow down to a beautiful Gaussian so fast.
It has some stuff in the tails.
People talk about that, like low probability events out in the tails of distributions, stuff like that.
So that's a big field of statistics.
I don't know too much about it, but it's like-- I mean, it's very practical because-- now unfortunately, oftentimes in chemical engineering, we make so few repeats that we have no chance to figure out what the tails are doing, maybe [INAUDIBLE] our tails.
And so this is a big problem for trying to make sure you really have things right.
So I would say in general, this is an optimistic estimate of what the uncertainty in the mean is.
Uncertainties are usually bigger.
So you shouldn't be surprised if your data doesn't match a model brilliantly well as predicted by this formula.
Now, if it's off by some orders of magnitude, you might be a little alarmed.
And that might be the normal situation, too.
But anyway, if it's just off by a little bit, I wouldn't sweat it because you probably haven't done enough repeats to be entitled to such a beautiful result as this.
We can write a similar-- actually, so here I assumed that the x's are uncorrelated.
That's almost never true.
If you actually numerically evaluate the C's, usually they have off-diagonal elements.
For example, my weight and my blood pressure are probably correlated.
And so you wouldn't expect them to be totally uncorrelated.
And so there's another formula like this.
It's given in the notes by Joe Scott that includes the covariance.
And you just get a different form of what you'd expect, OK?
And the covariance should also converge roughly as 1 over n if you have enough samples.
So you should eventually get some covariance.
You can write very similar formulas like this for functions.
So if I have a function f of x and that's really what I care about-- remember, I said that I have the average value of f is equal to f of x Px of x dx.
And I could make this vectors if I want.
And I could repeat my function, and I'd get some number.
And I could repeat the variance.
I have a sigma f. And this is something I like to do a lot of times.
Then if we do experimental delta f-- so we don't know what the probability distribution function is usually, or often.
So we'll try to evaluate this experimentally.
This is going to be 1 over n, the values f of x little n, the n-th trial.
And we could write a similar thing for sigma f, which I just did right there.
You can do the same thing.
Just make these experimental values now.
The sigma f squared experimental should go to 1 over n times the variance.
And this was the sigma in the mean of f, 1 over n times the variance of the sigma.
All right, so this is the same beautiful thing, that the uncertainty in the mean value of f narrows with the number of trials.
So you have some original variance that you computed here, either experimentally or from the PDF.
Experimentally is fine.
And then now you want to know the uncertainty in the mean value, and that drops down with the number of trials in the number of things you average.
So this all leads in two directions.
What we're going to talk about first is about comparing models versus experiments where we're sampling by doing the experiment.
So that's one really important direction, maybe the most important one.
But it also suggests ways you could do numerical integration.
So if I wanted to evaluate an integral that looks like this, f of x P of x dx, and if I had some way to sample from Px, then one way to evaluate this numerical integral would be to-- sorry, I made this vector [INAUDIBLE] a lot of species there, a lot of directions.
If I want to evaluate this multiple integral-- it's a lot of integrals for every dimension of x-- that would be very hard to do, right?
We talked about in [INAUDIBLE],, if you get more than about three or four of these integral signs, usually you're in big trouble to evaluate the integral.
But you can do it by what's called Monte Carlo sampling where you sample from P of x and just evaluate the value of f at some particular x points you pull as a sample and just repeat their average.
And the average of those things should converge, according to this formula, as you increase the number of samples.
And so that's the whole principle of Monte Carlo methods, and we'll come back to that a little bit later.
And you can apply that to a lot of problems.
Basically, any problem you have in numerics, you have a choice.
You can use deterministic methods or stochastic methods.
Deterministic methods, if you can do them, usually are the fastest and more accurate, but stochastic ones are often very easy to program and sometimes are actually the fastest way to do it.
In particular, in this kind of case, we have lots of dimensions, many, many x's.
It turns out that stochastic ones are pretty good way to do it.
But we're going to talk mostly about [INAUDIBLE] data because that's going to be important to all of you in your research.
So let's talk about that for a minute.
I'll just comment, there's really good notes posted on the [INAUDIBLE] website for all this material, so you should definitely read it.
And the textbook has a lot of material.
It's maybe not so easy to read as the notes are, but plenty to learn, for sure.
So we generally have a situation where we have an experiment.
And what do we have in the experiment?
We have some knobs.
These are things that we can change.
So we can change some valve positions.
We can change how much electricity goes into our heaters.
We can change the setting on our back pressure regulator.
We can change the chemicals we pour into the system.
So there's a lot of knobs that we control.
And I'm going to call the knobs x.
And then we have parameters.
And these are other things that affect the result of the experiment that we don't have control over.
And I'm going to call those theta.
So for example, if I do a kinetics experiment, it depends on the rate coefficients.
I have no control of the rate coefficients.
They're going to [INAUDIBLE] by God, as far as I know.
So they're some numbers, but they definitely affect the result.
And if the rate coefficient had a different value, I would get a different result in the kinetic experiment.
The molecular weight of sulfur, I have no control over that.
That's just a parameter.
But if I weigh something and it has a certain number of atoms of sulfur, it's going to be a very important parameter in determining the result.
So we have these two things.
And then we're going to have some measurables, things that we can measure.
Let's call them y.
And in general, we think that if we set the x value and we know the theta values, we should get some measurable values.
And so there's a y that the model says that's a function of the x's and the thetas.
Now, I write this as a simple function like this.
This might be really complicated.
It might have partial differential equations embedded inside it.
It might have all kinds of horrible stuff inside it.
But you guys already know how to solve all these problems already because you've done it.
You've been in this class through seven homeworks already.
And so no problem, right?
So if I give you something-- I give you some knobs.
I give you some parameters-- you can compute it, all right?
And so then the question is-- that's what the model says.
So we could predict the forward prediction of what the model should say if I knew what the parameter values were, if I knew what the knob values were.
And I want to-- oftentimes what I measure, y data, which is a function of the knobs, it's implicitly a function of the parameters.
I have no control of them, so I'm not going to even put them in here.
So I set the knobs I want.
I get some data.
I want these two to match each other.
I think they should be the same thing if my model is true, yeah?
So this is my model, really.
But I don't think they should be exactly the same.
I mean, just like when you try to measure my height, you don't get exactly the same numbers.
So these y data are not going to be exactly the same numbers as my model would say.
So now I have to cope with the fact that I have deviations between the data and the model.
And how am I going to handle that, all right?
And also, we have a set of these guys, typically do some repeats.
So we have like several numbers for each setting in the x's, and they don't even agree with each other because they're all different.
Every time I repeated the experiment, I got some different result-- that's my y's-- for each x.
And then I change the x a few times at different knob settings.
Then I make some more measurements.
And I have a whole bunch of y values that are all scattered numbers that maybe scatter around this model possibly, if I'm lucky, if the model's right.
Often, usually I also don't know if the model's correct.
So that's another thing to hold in the back of your mind is like, we're going to this whole comparison assuming the model's correct.
And then we might, at the end, decide, hmm, maybe the model's not really right.
I may have to go make a new model.
So that's just a thing to keep in the back your mind.
But we'll be optimistic to start with, and we'll assume that the model is good.
And our only challenge is we just don't have the right values of the thetas, maybe, in my model.
And this is another thing, too.
So the thetas are things like rate coefficients and molecular weights and viscosities and stuff that are like properties of the universe, and they're real numbers, maybe.
They're also things like the length of my apparatus and stuff like that.
But I don't know those numbers to perfect precision, right?
The best number I can find, if I look in the database is, you know, you could find like the speed of light to like 11 significant figures, but I don't know it to the 12th significant figure.
So I don't know any of the numbers perfectly.
And a lot of numbers I don't even know at all.
So like there's some rate coefficients that no one has ever measured or calculated in the history of the world.
And my students have to deal with that a lot in the Green group.
So a lot of these are quite uncertain.
But there are some that are pretty certain.
You have quite a big variance, actually, of how certainly you know the parameter values.
So one idea, a very popular idea, is to say, you know, I have this deviation between the model and the experiment.
So I want to sort of do a minimization by varying, say, parameter values of some measure of the error between the model and the data.
Somehow, I want to minimize that.
And I have to think about, well, what should I really minimize?
And the popular thing to minimize is these guys squared and actually to weight them by some kind of sigma for each one of these guys.
So this is-- we should change the notation, make this clearer.
These guys-- one model, and it's the i-th measurement that corresponds to that n-th experiment.
So I think that the difference between what I measured and what the model calculated should be sort of scaled by the variance, right?
So I would expect that this sum has a bunch of numbers that are sort of order of one because I expect the deviation to be approximately scaled of the variance of my measurements.
And if these deviations are much larger than the variance, then I think my model's not right and what I'm going to try to do right here is I'm going to try to adjust the thetas, the parameters, to try to force the model to agree better to my experiment.
And this form looks a lot like this.
Do you see this?
You see I have a sum of the deviations between the experiment and a theoretical sort of thing divided by some variance?
And so this is the motivation of where this comes from, is that I want to make the probability that I would make this observation experimentally would be maximum if this quantity in the exponent is as small as possible.
So I'm going to try to minimize that quantity, and that's exactly what I'm doing over here.
Is that all right?
OK, so next time when we come back, I'll talk more about how we actually do it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we're going to talk about Bayesian prior estimation and prior estimation in general.
So the last time we were writing down the expressions for the probability of observing a mean measurement if you know what the model is.
So let's try to do that again.
So suppose I have a model that predicts some observable and it depends on some knobs, and it depends on some parameters.
And suppose because I have great powers of faith that I believe this model is 100% correct with every core of my being.
And also because I have tremendous confidence in all the people who built my apparatus, and the knobs that I turn actually correspond to the real values, and I have tremendous confidence in all the literature that reports the parameter values.
And so I'm absolutely certain that this is the truth.
So we'll start from a position of absolute certainty, and then we'll degrade into doubt as the collector goes on.
So let's start from the position of someone who has absolute faith that this is the truth.
Is true.
So I have a model, I really believe this model.
So for example, I believe that the kilogram weight in the SI Institute in Paris weighs exactly one kilogram.
I believe that with every core of my being.
I'm completely confident that model is correct.
So there are some things I'm really confident on.
That's one.
And maybe guys have some things you really believe, too.
So let's go with things we really believe.
So I plan to conduct some experiments that measure this observable and are related to this model.
And so I'm going to do 10 repeats of measuring y.
So I'm going to get to the kilogram blob that's in Paris, and I'm going to stick it on my really expensive scale that I really believe is great, and I'm going to measure its weight.
And then I'm going to put it back, and then put it back, and put it back, and put it back.
And I'm going to get another really great scale that I really believe is great, and I'm going to measure it there, too.
So I've got a lot of repeats of measuring the weight of this kilogram, and I believe it's really a kilogram.
But the stupid measurements don't say a kilogram.
They say, you know, 1.0003, 0.99995, all kinds of numbers not equal to one kilogram.
So now I'm going to try to figure out what the probability is that it would have measured some particular value y.
So what is the probability that my experimental is between some value, say, y and y plus dy.
So that's a question for you.
So what's the probability?
Sorry, what?
[INAUDIBLE] OK, so we think that the probability that y is in this interval given that the model is true.
And I know the theta values perfectly, and I know the x values perfectly, is equal to some integral of what?
The bounds integral, probably y to y plus dy, believe that?
What's the integrand?
Sorry, what?
The probability [INAUDIBLE] AC function of y
Right, so what is it?
[INAUDIBLE] You wrote it down last time, I think.
So this [INAUDIBLE] is large?
Standard normal, right?
So it should be one over sigma root of 2 pi.
Does that sound OK?
I mean here.
It's probably the same, it's fine.
Yep
What does that notation mean, if your model is true?
So this means, given that the model is true, and I know these data values are exactly certain numbers, and the x values are actually certain numbers, what's the probability that I would make a measurement whose average would fall in this interval?
So this line means given that this is true, what's the probability of that?
OK, is this right?
Is this surprising?
This is OK?
So this is what I mean.
So we say that our probability distribution converges to a Gaussian distribution, this is what we expect.
So we expect it to have been large enough for this to be true.
Yeah?
This is very important.
This is like the whole course, actually.
This is the whole section, is this one equation.
So I just wanted to make sure you really get what this says.
And if you don't like the integral, you can make dy really small, and then it's just this times dy.
OK?
Actually, this notation's like [INAUDIBLE] I think I should do this way.
I should do this.
This is a number.
Let's get rid of the integral.
Let's make dy really small.
I'll make it [INAUDIBLE].
That all right?
So this is the probability density that we would observe, this is the experimental value y that we observe from the mean, and this is the little with of our tiny little interval.
Is that all right?
Yes?
So is sigma the [INAUDIBLE] on there?
Ah, what is sigma?
That's a great question.
We didn't write down what sigma was.
What is sigma?
Standard deviation?
It's not the standard deviation exactly.
Standard deviation of the mean, right?
So there's two sigmas.
We have the sigma of y, of the measurements, and that's equal to average value of y squared minus.
So we just figure that for how many experiments we do, we just compute the average of y squared, the average y, subtract them.
That's the variance.
And then sigma that I used in that equation there is 1 over n times sigma y.
And we call this the variation of the mean, it's the uncertainty in the mean value of y.
And the central limits theorem said that as long as n gets really large, we expect that this should converge to this.
And we talked last time about how when n get bigger, these averages don't really change when it gets big.
They're just the average.
But this number declined as n gets big because of this one over n formula.
And to understand that, suppose I measure the weight, and I measure, it should be around one kilogram.
But in fact my measurements are all over here.
Lots of measurements.
So they have a variance something like this.
But if I make a plot of-- as I run, I compute the running average.
So when I run the first two points, I get some average value here.
After I run 27 points more, the average value is here.
After I run 1,000 repeats, the average value is here.
It's getting pretty close to this, and the uncertainty in this number's getting smaller and smaller as I'm doing better and better averages.
The average more and more repeats.
Does that make sense?
OK.
So from this key equation, I can derive a lot of things.
And it depends what you want to do.
So one thing people do a lot is it was called model validation.
And what does this mean?
It means I have a model, I believe it's true.
I have some parameters, I believe they're true.
But there are some foolish skeptics out there who don't have the faith that I do.
And they think that my model's baloney, or my parameter values are wrong, or something.
And so to prove I'm right, I'm going to make some experiments.
And I'm going to show that I make a plot that looks like the experiment and model agree.
Some of you might have done this in your life, yes?
Everybody might make a parity plot or something.
You've seen these things before.
Now, this is like a confidence builder.
You're trying to get the skeptics out there to believe that there's some evidence to back up your faith that this model is perfect.
And what you really want to know is like, if the measurement that I measure, the average for my 10,000 repeated measurements, I expect that this quantity should be pretty big somehow in some way.
By then quantitatively saying what that means exactly what's a good fit, what's a bad fit, this is actually kind of a difficult question, and we'll come back to this one.
But that's a very common use of this equation is to try to do validation.
Now because it's kind of complicated, most people don't actually do it.
So instead what they do is they just plot some data points, and they plot your model curve.
And as long as they look good, then you're done.
So that's the normal way that it's done in the literature currently.
But of course, that's completely unquantitative.
It doesn't really say whether the model and the data really agree, it just means they look sort of like each other.
So that's like a human qualitative thing.
Now, if the purpose of validation is just to convince humans, then you've done the purpose.
Now, if your purpose is to try to quantitatively say something, then you really have to get into this equation, which usually is not done but would be the right thing to do for validation.
Now the alternative view is disproving a model.
But I just say that there's several ways this can happen.
You can try to disprove a model, but you might also show that the theta values are incorrect.
Or you might show that the experiment is wrong.
These are all possibilities, reasons why the model and the data might not agree with each other.
So this equation, it only holds if the model is really true, if the parameter values are all perfectly correct, if we know exactly what all the knob values are perfectly.
If any of those things are not true, then you should have some discrepancy, and there should be a way to show it.
And really what you're showing is that you'd observed some y that is very unlikely to be observed.
So probably observing that y is very extremely unlikely if all these other things were true.
So if all these things are true, and you compute this value, and this value's very tiny, then it makes you think that it's unlikely that you would have observed that.
And therefore, you might try to use that as an argument to say that something must be wrong.
The model's wrong, parameters are wrong, the knobs are wrong, something's wrong.
My y values are wrong.
It could be any of those things.
So this is often the most exciting papers to publish.
You publish a paper, you take some model that a lot of people believe.
You tell them they're full of baloney, it's completely wrong.
My great experiment shows you are completely wrong.
And so you'll see a lot of these in Nature.
I should warn you, a lot of those get retracted later, a very high retraction rate in Nature.
Because they want to publish papers like that that show that the common view is incorrect, and sometimes it's true.
But oftentimes the common view is actually correct, and there's something wrong with the experiment, or the interpretation, or how they computed this equation, or whatever.
And so actually it turns out the common view is perfectly fine, and it's just that the foolish authors went off on a tangent.
And then they have to six months later publish a retraction, by the way, sorry, paper was completely wrong.
And so you see a lot of that.
So that's a second kind of thing.
And we'll talk more about that a little bit later, too.
And then another thing is I'll relax my assumptions.
So I'll say, well, I'm sure that the model is true, and I'm sure that my knob settings are perfect, and I know what they are.
But I'm not really sure about all the parameters.
And therefore I want to use the experiment to try to refine parameter values.
So I'm trying to take my y's that I measure and somehow infer something about the thetas.
And this is a very common thing to do.
So in my group we've tried to measure the rate coefficient for a reaction.
We believe there is value of that theta, and in fact, we probably have an estimate of what it is.
But we're not sure of the exact number, and we'd like to do an experiment to refine the number and get it more accurately determined.
So that's another useful thing to do.
And this leads into two somewhat different points of view about this.
One you've probably done already called least squares fitting.
That's one view.
And the other is this Bayesian view that I'll tell you about next.
So there's sort of A and B.
There's one that I'll call Bayesian, and one I'll call least squares.
They're sort of related to each other, but not exactly the same conceptually.
So I'll try to explain that.
So the Bayesian view is probabilistic, so it's actually pretty straightforward to write down.
Remember that we wrote that the probability of A and B is equal to the probability of A times the probability of B given A, and it's also equal to the probability of B times the probability of A given B.
And what we have here is one of these conditional probabilities, if the thetas have a certain value, this is a certain probability.
So I should be able to use that formula somehow.
So I can write down that the probability of measuring y given theta is equal to the probability of y times the probability of theta given y divided by the probability of theta.
So I just took this formula, and I plugged in y's and thetas instead of A's and B's.
So I said, these two are equal to each other.
Rearranged it so then I can rewrite this.
This is the way we have it in here, probably of measuring y given theta.
Let's flip it around.
So probability of theta given that we measured y is equal to the probability of theta times the probability of observing y if theta was true divided by the probability of y.
Terrible handwriting there.
That's just algebra.
So this is what we want to know.
We want to know, what's the probability distribution of the parameter values y?
Because some of them are uncertain.
Now, before we started the experiment, we had some idea of what the ranges were for all the primary values.
Like I'm trying to measure a rate coefficient.
I know from experience with other similar reactions, from a quantum chemistry calculation, from some indirect evidence, from some other more complicated experiment, I have some idea that this rate coefficient has to be in a certain range.
Now, it could be pretty uncertain.
It might be five orders of magnitude uncertain.
But I know it's not less than 0.
I know it can't be faster than the diffusion limit, how fast things can come together.
So for sure I know some range, and oftentimes I know a much narrower range than that.
So I have some information about these parameter values before I even start.
Some of the parameters of the model I know perfectly, or pretty well.
So you know maybe there's a Planck's constant, or the heat of formation of one of my chemicals or something like that shows up in the numbers, and I might know that parameter really pretty accurately.
Whereas the particular rate coefficient I care about is the thing I really don't know very well.
So some of these have tight probability distributions ahead of time, and some of them have loose ones.
And this thing has a name.
It's called the prior.
And it's our prior information before we did the experiment.
And this one, after we've done the experiment, we're going to change it.
So we're going to say previously people thought that the parameters all lied in these certain ranges.
And now I'm going to get a tighter range, because I have some additional experimental information.
So this is called the posterior.
This means before, it means after.
So this is what I know about parameter values before and after the experiment.
This is the formula that I have over there.
It's a probability that if the thetas had a certain value, probably would have observed what I saw.
Yeah?
Which one refers to which?
Sorry, this is the prior, this is the posterior.
And those of you who are paying attention to notation realize I'm not doing this very nicely.
Because these are continuous variables, and I'm writing capital PRs, and they should be not be capital PRs, it should be probably density functions instead.
So let's rewrite it nicely.
So the probability of theta given y, probability density is equal to the probability density of theta initially times the probability of y given theta, [INAUDIBLE] density divided by the probability density of-- all right?
And what I just basically did was this is the correct equation the previous other one was all multiplied by d theta dy, it shouldn't be done that way.
So this is OK?
Now this is the prior information I have about the parameter values.
I know that they have to fall into some ranges.
And really all I'm doing is I'm correcting that information.
I'm improving the information to tighten the distribution.
So initially I know that my rate constant, here's my rate constant.
I know that it's got to be greater than zero, and I don't think it's really down there at zero anyway.
I think it's somewhere in here.
I really don't know much.
And I really don't think it's all up at the diffusion limit, and no way it's higher than the diffusion limit.
So that's my initial information that I have about the probability distribution of K. So it's the rate coefficient I want to know, and I know it's bigger than zero, and I know it's less than infinity.
And actually I know there's some physical limit, it can't be higher than something or other.
And you can do this for any problem, right?
I give you any parameter, you should be able to tell me something about it.
You might be uncertain by 20 orders of magnitude, but at least you have an error bar some width.
It can't be anything, right?
A lot of parameters have to be positive, for example.
You know that.
And you usually know something.
you might not think you know anything, but you do, you actually do know before you start.
So you actually know, this is the P of theta to start with.
And after I've done the experiment, hopefully you're going to know more information about it.
I might know that this quantity here is going to be like a Gaussian distribution.
It might have a kind of goofball dependence on theta.
I should comment that.
Notice how theta appears inside F. So theta's up in the exponent.
It's sort of inside a Gaussian, but it's like processed by F and so the observable might have a pretty goofball dependence on this rate coefficient.
So this thing could be some weird thing.
But for sure, when I change theta so this changes a lot, it's going to make a pretty big difference.
Because up inside the exponent of a Gaussian, so it's going to drop off a lot somewhere.
So I should get something that looks something like this maybe for my experiment.
So this one is P of K initially, the prior.
This one is P of yk.
And what this equation says is I want to multiply those two together.
And so I'm going to multiply this times this, and I'm going to get some new thing that's something like that when I multiply this time that.
Is that OK?
And so that's my new numerator of this equation.
Now this denominator doesn't make too much sense.
This says, what's the probability that I measured the mean I measured, given nothing?
So this is sort of like the prior probability that I would have measured it or something.
I don't know what this is.
So instead what people do, is they say, forget this.
But instead, let's multiply this by a constant that's going to normalize it to make it probability density so that it integrates to one.
So that's the way Bayes' theorem is used.
This is called Bayesian analysis.
And so what it's telling you is how to take your experimental information as expressed in this formula and use all your previous information about the parameters, put them all together, now we have a cumulative information about everything.
So we have some parameters that came into our problem into my experiment, but from previous work, I also knew something about those parameters.
Now I put it all together and I get a new value of probability distribution of those parameters.
And if my expert was really good, it would make this really tight [WHOOSHING SOUND].. And then when I multiply these two together, it's going to make this really sharp, and we have a really good value of k. So that's like the ideal case if I have a really great, well-designed experiment executed perfectly with great precision, then I can do this.
More generally, when I don't think about it, I get some distribution like this.
I still learn something compared to what I had before, but it might not be much.
So now I can end up with some distribution that's a little tighter than before.
So is this OK so far?
All right, now this is super simple.
I didn't have to solve anything, all I had to do was multiply two distributions together.
So in some respects, this is what you should always do.
All you do is you take your experiment, you multiply the probability distribution corresponds to your experiment times the prior, and you get some posterior, and that's why new information about the distribution.
And if I have a distribution like this, suppose this is my new distribution here, I can still get it central value, that's my mean value, k. I can get an estimate of the range of k. So I end up with a k plus or minus dk maybe, from just looking at the plot.
In fact, I never even have to evaluate what this constant is in order to do this.
I can just go look at the plot, see where the peak is, figure out the width, and I can report now because in my experiment, k plus or minus dk is more precisely determined than it was before.
Now, a practical challenge with this is that theta is usually a lot of parameters.
And I only drew the plot here in one dimension, but really it's a multi-dimensional plot.
So really what looked like, suppose I had two parameters.
I had my k I care about, and I have some other parameter, theta 2, that also it shows up in my model.
And say, before I started, I knew theta 2 fell in this kind of range, and I knew k fell in this kind of range.
So really before I started, if I think what it looks like, I really had sort of a blobby rectangular contour plot, where I think it's more likely that the k value and the theta 2 value are somewhere in this range.
And the most likely one is maybe somewhere in the middle there.
But I really didn't know much.
So it could be anywhere in this whole blob.
Now, when I do the experiment, the experimental value depends of both k and theta 2.
And commonly what'll happen is that the distribution from the experiment-- need color chalk here.
Let's get rid of these guys.
So this is my probably distribution, there's my prior.
If I do the experiment, maybe I'll have something like this.
That the experiment says that the guys have to be somewhere in the contour plot like this.
Because I can get pretty good fits of the data with different values of k as long as I compensate with the value theta 2.
Now I multiply these two dimensional functions.
The original is a blob function, and this is a stretched out blob.
And I multiply a stretched out blob times a fat blob, I get some stretched out blob that looks something like the intersection of these guys.
And so I end up with some kind of blob like that.
I'll draw it really thick.
So this is my posterior, some kind of blob like this.
So now I know a little bit more about these two parameters than I did before I started because of my experiment.
Is this OK?
I really can't say I know what the real value of k is, or the value of theta 2.
But I know that combinations of k and theta 2 that are sort of in this range, all of them will give me pretty good fits to my data, and also be consistent with all the previous information I have about those parameter values.
Is that all right?
Now, I drew it with two parameters.
In a lot of models we have, we have five parameters, six parameters, seven parameters, nine parameters, 14 parameters.
We have a lot of parameters.
And so then we try to make this plot, even how to display the plot is going to be a little problematic.
But it's there, right?
And somehow, we still narrowed down the hypervolume in the parameter space from whatever it was to begin with to now we know something a little bit better.
We have a narrower range of the parameters that would be consistent with all the information available, including my new experiment.
And then the next guy does his experiment, and he does an experiment that shows that these guys have to be somewhere in this range in order to be consistent with his experiment.
And so now I can narrow down the range to be something like that.
And the next person does their experiment, and they get something else, and something else, and something else.
And eventually by 2050, we have a pretty nice determination of the parameter values.
So that's the advance of science, as drawn in chalk by Professor Green at the board.
So this is a very important way to think about it, is what you're doing when you do experiments, is you're generally restricting the range of parameter space that's still consistent with everything.
And when we mean consistent, we mean that the probability that you would have observed what you did observe is reasonably high.
We'll still have to come back to quantitatively figure out what reasonably high means.
Now, when you did this before when you were kids, nobody mentioned the word Bayes, or Bayesian, or conditional probabilities, right?
So they just said, oh, just do a least squares fit.
How many of you did that before?
So somebody told me before, forget this stuff, we're going to never even mention this stuff.
We're just going to do a least squares fit.
Now, where did the least squares fit idea came from?
It came from looking at this formula and saying, you know, this is the deviations between the experiment and the model prediction, and I weigh them somehow, and I have the square.
And that's the thing I want to make small.
If I have a high probability that what I observed really happened, or the probably I'm going to observe this, it's got to be that these guys have to be reasonably close to each other.
They're really different.
And it's going to be very small, because it's inside an exponential.
And if those guys are really different, and the squared thing is really large, then the probability is incredibly small that I would have observed that.
So we think that this thing should be small.
And in fact, if I want to get the very best fit I can get, which means like the probability was the highest of what I observed in the real observation or something, then if I'm free to adjust one of these thetas, I can adjust the theta, try to make this thing like equal to zero, or small as I can.
So that's where the concept of least squares comes from.
Now, when you're doing least squares, you almost always have multiple parameters, and therefore you're going to have to have multiple data.
And they can't just be a repeat of one number.
Can't be your data, it's not sufficient to determine the parameters.
So normally when you do an experiment, you have to change the knobs.
We have to make measurements in a couple different conditions.
Like for example, kinetics.
You often want the Arrhenius A factor and the EA.
And so I got to run the experiment in more than one temperature or I'm never going to be able to figure that out.
So I have to change the temperature in my reactor.
Make some measurements at one temperature, and make some measurements at a different temperature.
And for almost everything in life that you want to measure, you're going to have to do this.
You vary the concentration of your enzyme if you want to see how the enzyme kinetics depends on something.
you can't just keep running exactly the same condition over and over.
You'll get that number really precise, but it's not enough information to really fill out the parameters in your model.
So you're going to have to run several different experiments with different knob settings.
Also, normally we don't just measure one quantity, one observable in each experiment.
We usually try to measure as many things as we can.
So we actually have several observables at each knob setting, and we have several knob settings, so we have quite a lot of data.
And each one of those is repeated a whole bunch of times so that we're confident that we can use this Gaussian formula.
And so what we really have is the pi-th observable measured at the l-th knob position.
Well, I'm sorry, l's not good either, it's used in your notes for something else.
M, there you go.
The m-th knob position.
Now, normally you have several knobs, so that's a factor.
And we have a lot of observables we can make at each position.
So this thing is a measurement.
And we repeated this multiple times so I can get the average.
And we're also going to have a corresponding sigma I M, which is the variance of the mean.
So it's variance, that's going to be divided by the square root of the number of repeats for that particular experiment and that particular observable.
So this is your incoming data set, and you also have your model which predicts y model, it predicts the observable i for the sequel to fi of xk theta.
So if you have certain knob settings, like certain temperature, and you have your parameter values, then you can calculate what the model thinks should be the observable value, and then you can actually measure it and measure its variance.
so that's the normal situation.
And now you want to figure out, are there some values of the theta that make the model and the data agree?
And that's the least squares fitting thing.
So what we can define as a new quantity, weight of the residual vector EJ, which is defined to be jk, to be consistent with Joe Scott's notes
Is k the same as m?
M is in oppositions, I'll tell you what k is in a second.
m, sorry.
Too many indices.
OK, so this is the residual between the model position.
And now-- oh man, I'm sorry, [INAUDIBLE].. K is an index over i and m. So k is just going to list all the data you got.
Some of the data came from the same knob settings, some came from different knob settings.
Yeah?
So is x the m the y model i [INAUDIBLE]??
Thank you, yes.
y model i, I guess this is now k. And so k is one of these indices that carry-- you can bind two indices together and put them together just like you did in your PD problems.
All right.
Now, I wrote down this sigma.
But actually if you're measuring multiple things at the same experiment, you should expect them to be correlated.
So really what we should worry about is the c, the covariance matrix, that we defined last time.
So you should also compute that thing.
And so what you should expect is the probability density that we would measure any particular residuals if the model is true.
And if we have these certain parameters, theta, this should be equal to 2 pi negative k over 2 the determinant of c negative 1.2 exponential of negative 1/2, epsilon transpose c inverse epsilon.
So this is the multi measurement version of the same equation here.
So this is the quantity that we think should be small if we have good parameter values and we did a good experiment.
Actually, even when we did bad experiments, still should be small if we have good parameter values.
And that's because the c's, if we did a bad experiment, we'll have a high variance or something, then we should see the c's will give us weightings that will reflect that.
Yeah?
[INAUDIBLE]
Is that--
So you have the next [INAUDIBLE] K [INAUDIBLE]
Oh I'm sorry, this is the capital K, this is the number of data points.
So little k is equal to 1 to capital
So does capital K count for both experiments?
It's the number of distinct data values after you've already averaged over repeats.
So you do m experiments, at each experiment you measure capital I observables.
So it's like m times I, so K. If you measured everything in every experiment, it's equal to I times m. Now there's two ways that people approach this in literature.
The fancy way is you say, you know, this covariance matrix comes in in a pretty important way into this probability distribution function.
And so maybe I need to worry a lot about whether I really know the covariance matrix.
And my uncertainty in the mean drops pretty fast as I do averaging, but I'm not so confident that my answer in the covariance matrix was small.
So what people do sometimes is they'll try to vary both c and theta, and try to get a best fit where they're varying c. But then they have additional constraints on c that c has to satisfy the equations you gave last time about how you calculate the covariance matrix from data.
And so I was saying, well, I want this c to personify these equations pretty well, but true covariance of the world of the system is not the same as what I actually measure by just measuring, say, five repeats of an experiment.
And so I might want to vary the c. You try to vary the c, turns out to be kind of complicated math, so not many people do it.
Even though conceptually it makes some sense, you should worry about the fact you're not really sure about the covariance.
So what a lot of people do is they say, let's just use the c that's computed from the formulas I gave you last time experimentally.
So just say, let's just take c experimental, put them in here.
And now this is a constant.
And now the only thing that varies in this problem is thetas which come into the epsilons.
Because the epsilons depend on theta.
And so in that case, I can just try to maximize this probability.
And what that happens to do is to minimize this quantity in the exponent.
And so all I need to do is say, for example, theta best is equal to arg [INAUDIBLE] theta epsilon of theta c epsilon.
And so this is the least squares fitting problem that you guys have probably done before.
And probably what you did was you assumed I had perfectly uncorrelated data, and all my errors were the same.
And so c, which is the identity matrix, and I took it out.
Probably did that before?
Yeah, OK. That's pretty dangerous to do, I'd say.
What people do a lot, which is a little bit less dangerous, is at least say, well, you know, when I measure the concentration of species x by GC, I have an error bar of plus or minus 5%.
And when I measure the temperature with my thermocouple, I have an error bar of plus or minus 2 degrees.
And so the variances of these guys should be a lot different, temperature and GC signal.
And therefore I definitely need to weight my deviation somehow.
And it's really what you do is you keep the diagonal entries of this.
That's often done.
And we just forget the fact that they might be covariant.
But if you've done the experiments, you actually do have enough information to compute this thing anyway, so you might as well just use the experimental value.
So this is the least squares thing.
And let's think, what the heck is this doing?
We're saying, all of a sudden we grabbed all the parameters in the model, which might include things like the molecular weight of hydrogen or something.
And we can find the very best values that would make our data match the experiment as best as possible.
And in some sense, that's great, we know the best values and parameters for our experiment.
But of course, if we vary the molecular weight of hydrogen, it's going to screw up somebody else's experiment.
Because somebody else did some other experiment that depended on the molecular weight of hydrogen, and they had to get some other value to match their experiment.
So in these parameter set, anything I do to vary those parameters, I got to watch out that maybe some of those parameters are involved with somebody else's model and [INAUDIBLE] some other experiments.
And I'm not really free to vary them all freely.
So this is the idea from the Bayesian of having the prior intimation is so you know some of the ranges on these thetas already, and some of them you might know really sharp distributions, like the molecular weight of hydrogen.
You might know that to a lot of decimal places.
And so when people do this, normally you don't vary all of the thetas.
Usually what you do is you select a set of thetas that you feel free to vary because they're so uncertain, and other thetas that you think, oh, I better not touch them.
Because if I adjust them, I may go to crazy values that are inconsistent with somebody else's experiment.
So a lot of times like the molecular weights, you would not touch them.
You would just say, I got to just stick to the recommended values and the tables.
I'm not free to vary the molecular weight of hydrogen, even though if I did it would make my data match my experiment better.
Makes my model and the experiment match more precisely.
So deciding which parameters to vary in this is a really crucial thing.
And that's a lot of the art of doing this has to do with that issue.
Also, you don't have to keep the thetas in the form you have them.
You could do a transformation.
So you could change to, say, W's that's equal to, say, some matrix times the thetas, and I could express the equation in terms of the W's.
So I could transform my original representational parameters as some other parameters.
And often times, your experiment might be really good at determining some of these W's, even if it might be incapable of determining any of the thetas.
So you often might know some linear combination of parameters, or maybe not linear combinations, some non-linear combination of parameters might actually be determinable very well from your experiment, even though you can't determine things separately.
And this gets into the idea of dimensionless numbers.
So your experiment might depend on some dimensionless number very sensitively.
And you can be quite confident from your external data what the value of that dimensionless number must be.
But if you look inside the dimensionless number, it depends on a lot of different things.
And you might not have any information about them separately.
All you know is about your experiment just tells you the value of that one parameter very accurately.
So this is another big part of the art of doing the model versus data is setting up your model in terms of parameters that you really can determine, and getting out all the ones you can't determine and fixing them.
So we're really going to generally change do this kind of thing.
But we're going to say that some thetas are fixed, and also we might change to a different representation, change to W's instead.
Yeah?
Can you explain where this transform-- I don't really know what's up with--
Yeah, let's get an example.
Suppose I was doing a reactor that had A equilibrium of B.
And I was really interested in kf, the forward rate for A going to B. I'm a kineticist, I love to know A goes to B.
However, if I setup the experiment wrong, it might be that this readction ran all the way to equilibrium.
And what I see in the products is actually just the equilibrium ratio of A to B.
So what I'm measuring might be something that's dependent really on kf over kr, and that might be the quantity I can really determine.
Because that's equilibrium constant.
If I didn't think about it, I could just try to have the model fitting procedure, just optimize to find the very best value of kf.
And in that situation, it might have a lot of trouble, because it might be quite indeterminant what the kf is, because really all that matters is the ratio.
Also I think about this some more, suppose I run at short times, and I measure the time dependence.
What I'm really measuring is kf plus kr.
Do you remember we did the analysis of A goes to B, one of the early homework problems?
The time cost was actually kf plus kr, not kf separately.
And so if I measure the exponential decay time constant, I'm really determining kf plus kr, I might be able to determine that very well.
Actually, in my lab, I can do a great job for this.
I have an instrument that can measure the time constant of the exponential of k really precisely, but it's determining the sum.
It's not determining either one of them separately.
And I might have to do a separate experiment, say a thermo-experiment to get the ratio.
And then from the two I can put them together and get the two values distinctly.
So this will be an example of this would be a W. My W is kf plus kr, the matrix would be 1001, something like that.
1-1, something like that.
Where I add these two guys, kf, kr.
These are my two parameters, 1 plus 1.
And I can determine W1 now very accurately.
sorry, this is m, this is W. So now in terms of W, this has two parameters now, W1 and W2.
I can't determine W2 from my experiment, but I can determine W1 really well.
So then when I do the least squares fitting, I should vary W1.
I can fix it it for my experimental data, and just leave W2 fixed at some value.
I can't do anything about it.
That all right?
Now, do you get the difference in these two points of view?
This is like, two completely different ways to look at the problem.
You can think about it as, these parameters are free for me to vary, and I just have to be careful to select the ones I'm really free to vary.
And that's the least squares fitting point of view.
Or I could say, I'm not really determining anything in particular, all I'm doing is taking the whole range of uncertainty that we have about parameters, and by my experiment, I narrowed it down in the Bayesian view.
So it's the two different points of view.
To do this one, I need to make sure I have enough data to determine something.
So I have to have enough determine some parameter, at least one, otherwise there's no point in doing this.
This one I can do even if I can't determine anything, because I could still narrow down the range of parameters.
But this might be harder to report in a table.
Because all I have at the end is a new probably density function of multiple parameters.
All right?
OK, we're done.
See you guys on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So welcome to our special class with half the people gone for tonight's hearing.
You guys are the chosen few, so I'm pleased you're here
[INTERPOSING VOICES]
So I'm just going to say a little bit about models versus data and then I'm going to go back to undervalue problems and talk a little bit about how you handle really big ones.
Yeah?
Are these going to be [INAUDIBLE]
Yeah.
Yes, sir
All right
And in fact, these are even going to be on the video, because the guy just this fixed the video, so it will be recorded.
All right.
Yeah, I'm sorry.
You said the video was broken last time so that recording didn't work.
All right.
So when we're talking about models versus data a really critical question is whether the model is really consistent with the data or not.
And how do we know this is really true?
So we talked about how we can figure out some sort of chi-squared maximum that we would tolerate.
So the maximum that we tolerate, we call it Q, capital Q in a lecture before.
And so if we can find a chi-squared of theta that's less than this, then it's at least plausible that our model is true.
Right?
Or, alternatively, if all chi-squared thetas for any value of thetas are greater than chi-squared max, then we would say the model is false, or something else is really wrong with our experiment.
Right?
That's what we say when we're saying we have a maximum chi-squared that we would tolerate.
So a critical question with us is whether we really found the very best fit value of the parameters.
In some simple problems you might have linear dependence on the parameters, and then there's only one best fit usually, as Long as you have a non-singular set of equations and everything we find there's no trouble.
However, more commonly you have equations which are singular or close to singular, and also you normally have non-linear dependence between the model predictions and the parameters.
So, for example, I do kinetics.
We always care about the ray coefficients, the K's and they always come in inside a differential equation.
Right?
You always have like d concentrations dt is equal to some terms, and some of them are, you know, k concentration I times concentration j. whatever one it is, ij.
Right?
You have a lot of like terms like this by molecular reaction terms.
And these K's, some of them I don't know.
And so these would be one of my thetas.
So this would be, you know, theta 17 would be one of these K's that I want to determine.
And when I have a differential equation like this, when I integrate it, the K is always going to come in non-linearly in the solution.
Right?
So when I integrate this solution, even in the simplest case you get things like c of t is equal to ae to the negative kt.
That might be a really super simple case if you have a linear kinetics.
The K is still coming in non-linearly to the predictions.
Right?
And if you have anything complicated you can't even write it down in a closed form expression, and you have to integrate with oed45 or ode15s and so the relationship between K and the concentrations that you measure can be really complicated.
So it's always non-linear.
In my world all the models are non-linear.
And I think there's many other situations like this.
So if you try to measure a diffusivity, it might be possible to set up an experiment where you'd expect a linear dependency to something you'd measure in a diffusivity, but most likely not.
So most likely it will come in non-linearly somehow and you'd have some model for it, and you'd have a non-linear dependence between that number and the numbers you would measure.
Right?
And almost all the properties are like that.
So because you have a non-linear dependence, then you have to worry about how many minima do you have in your surface?
So chi-squared of theta is a non-linear function of a lot of variables.
And typically it won't just have one minimum, you'll have a whole lot of local minima.
Now one of those is the global minimum.
And that's the one you want, that's the very best possible fit.
But you normally won't know how many minima it has and so no matter how many minima you find you don't know if you're done, and you don't know if you really found the best fit.
OK?
So just because all the ones you found so far bigger than chi-squared max you might not be ready to write that article in Nature to claim that the laws of physics are incorrect because you're not sure you really found the very best fit.
OK?
So that's one of the many problems of trying to figure out if a model of the data are really consistent.
Now that particular one, you can overcome if you can find a guaranteed way to find the global optimum, to find the very best possible that there is.
And some people spend their life working on this, and one of those people is professor Barton in this department.
And so he teaches a class on how to do this, and there are actually methods to do it.
So if you want to see the application of this to kinetic models there's a paper by me and professor Barton.
And the lead author is one of his former students, Adam Singer and J. Phys.
Chem.
and it just shows how you can use global optimization to guarantee you have the very best possible via the parameters, and then they'll show you what happens.
So this is a case where the experimental data was measured by one of my students, James Taylor, and that's the thing with the error bars, and he did a lot of hard work to make sure he had pretty good error bars and he really knew what they were.
And so he had those points.
So he measured a lot of points.
This measuring a lot of points is a key thing.
So it means his degrees of freedom is very large.
Because he uses many, many different time points along the way there and he has error bars on all of them.
And so because the degrees of freedom are very large it actually makes it a very good constraint on the model.
It tested very thoroughly.
Where if you only measured one point, then that's not a very good constraint.
Now what James had done, and we published a paper in 2004 where he did it, where he said, you know, when I do my fits this is what I get.
I get these kind of purple and green things, and therefore they're all outside the error bars, and therefore I publish in the Journal of Physical Chemistry, I claim this model is wrong based on my experimental data.
OK?
So we published that in 2004, it was accepted by the referees.
It's in the literature, you can read it.
Don't believe it though, because Adam singer went back and did global optimisation using the same model and his red line goes right through all the points really well.
Do you see that?
You can see the red line there.
See this red curve?
That's the true best fit.
And so all the fits that James had found were all local minima in the chi-squared squared surface.
And when he actually found the best fit with Adam's fancy method, then it turns out that actually the model matched the data perfectly.
So that's definitely true.
So what we published the first time was completely wrong, because we said the model disproved the data.
But in fact the model proved the data.
I mean the data proved the model, or was completely consistent with the model in every way in great detail.
So I was a little embarrassed.
So then James, also in the same paper, had found-- and they measured another condition in a slightly different case, and actually was looking at a different solvent.
And in this case he found this green curve as his best fit, and it looked pretty good by eye, though you can see it sort of skittering on the tops of the error bar bands.
And so this is one where he said, you know, we do the statistical test, there's like a 5% chance that you would measure data like he measured if the green model was the truth.
And so we said some wishy-washy words in the paper, basically because we weren't really if it was consistent or not consistent with the data.
But in that case Adam used his fancy global optimization techniques and found that James' fit was actually not that far from the global optimum, and he got the red curve there, marked by the red arrow, and that one looks pretty good, it actually goes through the data.
But it doesn't really do it right, and found that the confidence intervals are only 16%.
So it's only a 16% chance, one chance out of six, that James would have measured the data he measured if that model was true.
And now we're sure that we have the global best fit, so now we're, like, thinking well, you know, maybe we should say that the model is not true for this case.
Because it's an 84% chance it's not true anyway.
So actually we got it wrong both ways.
So in the first case we said that our data disproved the model.
And in the second case, we said that our model was pretty good with the data, but actually they were both wrong.
So one was wrong that actually-- the data did prove the models correct for the first case and it actually looks a little fishy for the second case.
So anyway this is just a concrete example to make you be a little bit more worried about what you do when you do these fitting things.
Now there's a variety of ways to try to do the least course fitting to try to find optima.
There was a guy named Carr from University of Minnesota.
And he published this [INAUDIBLE] way to just try a whole lot of different initial guesses, and that usually works to find the global minimum.
But if you really want to have a guaranteed global minimum that you're absolutely sure is definitely is the global minimum, then you really need to use global optimization methods and you really should talk to professor Barton.
And for these kinetic models he actually has a distributed code called GDOC that will just find the global optimization as long as you don't have too many kinetic parameters.
I think if you go to about 6-- around 6 adjustable parameters it can handle.
If you have more than that then usually typically not.
Because the problem with global optimisation methods is that they scale exponentially with the number of adjustable parameters, or the number of things you're trying to optimize.
Now on the other hand, if you have a model that you need to use more than six adjustable parameters to fit your data, then you might want to try to think of a different experiment.
Because you know the famous saying is like, you know, give me N parameters I can fit an elephant.
So if you have more than six parameters and you're trying to fit some data set, probably if you do it right you'll find a fit, but it may not really prove anything.
So from that point of view professor Barton's code, GDOC, is pretty convenient for doing these non-linear optimizations.
So that's a code for non-linear optimization with differential equations embedded.
And they can be stiff and they can be large, it doesn't matter.
What really matters it's just the number of adjustable parameters, that's the cost.
All right now there was a really good question, that you asked I think, in class last time, maybe.
Maybe the time before.
And it was what happens-- so let's look at different case you get.
So one case you get, you know, if chi-squared square, theta best, your best fit parameters is a lot bigger, think chi-squared max, then you're ready to say the model disproves it.
So that one you're OK.
If chi-squared of your best fit is less than, or significantly less than chi-squared max-- it has be much less than it, really less, not equal.
--then you're really happy to say that the models are consistent.
The model and data are consistent to each other.
And, in this case, what you would normally do next is determine the confidence intervals on the thetas that you adjusted.
Right?
And if you read the chapter in Numerical Recipes, for example, some of the notes online, it could tell you how to do this.
Right?
And so that was OK?
It's OK.
So the case that's really problematic is you find chi-squared theta best, and it's less than chi-squared max, so you think the model is consistent with data, but it's pretty darn close to theta-squared max.
And so then you have to figure out, well, the weird aspect of this is I now have-- here's my famous plot of theta one and theta two.
So I have two parameters I'm adjusting.
If they want theta two, here's my best fit.
My best fit.
If chi-squared max-- if the curve of constant chi-squared max is something like this, this is chi-squared of theta equals chi-squared max.
If it's like that then I have some big region of a lot of different theta values that give pretty good fits, and I'm happy to draw confidence intervals, like draw lines here and here, for example, and say that's the confidence interval on theta one, and I'm all right.
OK?
And it's sloped here so I might want to tell people about the co-variance between theta one and theta two.
All right?
But I know what to do, it's fine.
Now suppose this is the case that's like this.
Where chi-squared theta best is significantly less than chi-squared max.
Now suppose instead chi-squared best was very close chi-squared max.
So then here is chi-squared max.
This is chi-squared of theta equals chi-squared max, this is the good case.
Here's the bad case.
So in the bad case, there is some region of thetas that the model is consistent with the data, but it's very tiny.
There's a little tiny range of theta values that would make the models consistent with the data.
Now what's weird about this is that my best fit, it's not very good.
The chi-squared is almost up to the maximum chi-squared I would tolerate.
So it means my deviations between my model and data are pretty bad.
Like this plot actually.
OK?
And so I'm, like, you know, it could-- it's possible.
I was a little bit unlucky and my data set is pretty far off but still within the possible range.
And so maybe the model data are consistent.
But then when I try to compute the confidence intervals on my parameters, there's only a really small range of parameters that are going to give good fits here.
Because the best possible fit is this red line.
There's no choice of the parameters that's going to go through all the circles with this model.
So now I'm like, what's going on here?
So if I went ahead and do the same way I did before, I draw my confidence intervals like this and say, OK. Say, wow, I've done an amazingly great job of determining theta one.
I know theta one very, very, very precisely.
And the same for theta two.
Here's theta two, it's got to be in that range and I've really done a great job.
So I'm awesome.
But I'm only awesome because my original best fit wasn't very good.
So this might make you feel a little sick in your stomach if you're getting ready to publish in nature.
Right?
And then if the reviewers are on their game, you might get something really scathing back from the reviewers.
So there's kind of two ways to play it at this point.
One way is, you are absolutely sure that your equations are correct.
You are Sir Isaac Newton, you have just written the law of gravitation.
You are measuring the orbits of the moons of Jupiter, and you are sure that your equations are right.
And so, you know, my problem is my antiquated telescopes of 1600 are not that great, and so I think I have maybe-- maybe I misestimated my error bars on my measurements.
Maybe I have some aberration or something, the angles are off, right?
My twisted telescope looks as if the star moves or something, right?
OK?
So that's one possible way to go, say I really believe the model, and so I believe this is the truth, and I really believe my chi-squared max to be what it is, and I double checked the error bars and I think it's true.
So I think have done a great job and I should publish it in Nature because I have determined these parameters with 17 more decimal places than anybody ever could before.
OK?
So that's one way to look at it.
The other way to look at it is what's recommended in the Numerical Recipes book.
And they say don't use-- instead of writing the equation to be chi-squared of theta has to be less than chi-squared max, they say when you're computing the confidence intervals, instead write chi-squared of theta has to be less than chi-squared best fit plus chi-squared max.
So they say don't use a stinky little circle here, draw a nice big one around there.
And what is this saying?
I guess what this is saying is I don't believe that my poor agreement between the model and data justifies me declaring very narrow parameter values.
So I just use the kinds of parameter value confidence intervals that it would have actually computed a priori when I do my experimental design.
I think that's really how well I can determine stuff.
And I think something is wrong that my chi-squared best fir is so bad.
Maybe I didn't do my global optimization right and so I have the wrong parameters.
Maybe I mis-estimated my error bars and so my chi-squared are all off.
Maybe-- I don't know what else happened.
Something happened, and I don't believe I'm that good, so therefore I'm going to quote much broader confidence intervals, which means much less precision in my determination of parameters.
And that's what the Numerial Recipe guys recommend you should do.
But you know, it's up to you.
And I see it both ways.
So the people who won the Nobel Prize for the dark energy, do you guys know about this?
So if you ever saw the data there.
So that's one where they said-- they were using Einstein's theory of relativity, General relativity.
They say we believe Einstein, we believe that equations true.
And we're not going to, you know, the fact that the data only matches when you choose a certain value of the cosmological constant, we're just going to go with it.
And we're reporting that's the value of the cosmological constant.
Now some other people might say that's kind of goofy theory.
Einstein changed his mind six times about whether to put the cosmological constant in the equation at all, so I'm not so confident and I would report something different.
But they went with their gut.
They said we believe Einstein.
They went with it.
The Nobel Committee agreed, and they got the Nobel prize.
So you know, you're taking your chances.
The Numerical Recipe guys are more like engineers, they say, oh, we don't believe models that much anyway.
Let's just use more conservative confidence intervals.
Any questions about this?
OK. All right.
So now let's change topics.
And we'll change to [INAUDIBLE] splitting.
So if you remember back when we were talking about boundary value problems, we're often trying to solve problems that are like this.
All right.
So this is the conservation equation for a species in a reacting flow situation.
So you have some convection, you have some diffusion, you have some reactions.
Now in my life I have a lot of models.
I have about 200 species in them.
So this K is 200 different variables.
And then you have these equations, and on top of them you have [INAUDIBLE] equations for that momentum conservation and continuity.
And so this is actually a pretty bad set of equations.
And typically the chemistry is stiff.
Because I have chemistry time scales it'll be, like, in picoseconds, sub-nanosecond time scales.
And I'm caring about simulating-- well in my case maybe I have an advanced new engine burning some fuel.
And so we're running over a simulation of a piston cycle, maybe 10 milliseconds, but the chemistry is sub-nanosecond, so then you multiply it out.
So that's like I'm going to 10 to minus two seconds down to 10 to minus 10 seconds.
That's eight orders of magnitude difference in time scales.
So I'm in a bad way trying to solve this equation.
And I have 200 of these guys coupled to each other.
And then I have to think about what my mesh looks like.
And I'm trying to simulate inside a piston, actually I have a moving mesh because the pistons compressing, so the equations are kind of complicated even with what the mesh is doing.
And then I need a bunch of mesh points there because the-- do you guys hear the Kolmogorov scale?
You guys learned this yet?
It's a scale like the minimum eddy size if you have a turbulent flow.
And that's pretty darn small also.
So it might be sub-micron.
And so I have a natural, lens scale that's like microns maybe.
So 10 to minus six meters, but the piston diameter is couple centimeters, so again I have like four orders of magnitude between the mesh size I need to resolve the physics and the size of the thing.
So I need about 10,000 points across the cylinder, but it's a 3-D problem, so I need 10 to the fourth, times 10 to the fourth, times ten to the fourth, so I need 10 to the 12th mesh points.
And then I have 200 variables, say variables at each mesh point, so I have 10 to the 14th stayed variables.
So how much memory you have in your computer?
How much ram?
You have probably like 16 gigabytes.
So you have ten to the ninth- tenth to the ninth bytes, is that right?
No-- what's giga?
Ninth?
OK, so you have ten to the tenth.
Sorry.
Ten to the tenth byes, but I have 10 to the 14th stayed variables.
So I have a problem.
So then you need to figure out how you solve it.
So there's two schools of thought of how to solve these equations.
The school thought which has actually been the most successful so far, but is not readily available, is to get the government to build you the biggest possible computer in the world with as much memory as possible.
And then you can actually keep track of your state vector, all 10 to the 14th elements, and you reserve this whole computer for yourself for, your know, a month.
And just run it.
And you figure out how to parallelize all the calculations, and you just use an explicit solver.
And so that way it's just straightforward.
All you're doing is you're computing, you know y of T plus delta T is equal to something, and it's just an explicit formula ode45, fancy version of that.
OK?
And it's just algebra, and you just add them up and it's no problem.
But this is a limited feasibility thing.
This only became possible about three years ago when they built a computer that actually had enough memory to store the state factor.
It's also-- if the equations are too stiff it still doesn't work, because the step size and time step you need to use is so small-- so say it's 100 picoseconds and you're trying to get it out to a millisecond, so you need 10 to the eighth something time steps, which means you need to double precision numbers all the way through.
But if the government builds you a 64-bit 128-bit machine then you can do it.
But anyway this is like a very special kind of approach.
So now most people, including me I don't have access to that kind of computer power.
Actually if you want to read about that there's a woman who does that, who is Jacqueline Chen, and you might want to look on the internet and see about her.
And somehow she became buddies with the people in charge of the super computers, and so they're happy to give her a month of, you know, the best computer in the world every year, and she runs some really awesome calculation that's exactly right.
Solves everything, has all the state variables ten to the 14th, and that's fantastic.
And currently that's the benchmark for how people test approximate methods for how to solve this.
Because she actually has the full solution numerically converged.
But this again is a limited applicability.
So a lot of people are working on approximation methods to try to solve this equation if I don't have access to such awesome computer resources.
And the general idea of them-- well the two ideas.
So one idea is somehow get rid of the turbulence.
OK?
And so what they do is they use a mesh coarser than what you what you would need to resolve the Kolmogorov scale, and then they use models for what happens for itty bitty tiny eddies.
And those are called sub-grade scale models, and this whole approach is called large eddy simulation.
And the idea is you keep track of the large eddies, the big recirculation zones that's modelled explicitly, but you don't keep track of all the itty bitty tiny swirling off little eddies that lasts for a little second, a little bit of a period of time before they dissipate because of diffusion.
OK?
So that's the concept.
And there's, I don't know, half of all of the mechanical engineers who work in large eddy simulations are trying to figure out how to do this.
Maybe Jim as well, I don't know.
No.
OK, but at least he knows some of them probably.
So that's one whole branch.
It's to try to get rid of the turbulence.
And basically that's trying to reduce the density of mesh I need.
So trying to get rid of this-- use the spatial mesh.
But the other issue is the time mesh.
And the time is even worse, right?
Because I said I have ten to the eighth time points and I only need ten to the fourth spatial points in the dimension.
So in some ways, if I can do something with the time, that really can buy me a lot.
I have eight orders of magnitude I'd like to try to reduce to three orders of magnitude if I could, that would be really nice.
That idea is to say, well let's separate the fast timescales from the slow time scales.
And in fact all the fast time scales are mostly all chemistry, particularly after I changed the eddy simulation where I get rid of the really small spatial scales, then the time scale of diffusion over this big mesh is kind of slow.
And so I can go to a time scale that's completely controlled by chemistry.
So we already told you, how do you solve stiff differential equations?
Use a stiff solver, right?
And there are special methods, there are implicit methods, and there's specialized solvers.
And what they do is they allow you to take larger time steps than you would be allowed to use with explicit methods where you would run into numerical instabilities.
If you try to use the explicit method with a big time stamp, it's much bigger than the real physical time scale, you'll get some crazy oscillations.
But if we use the stiff solvers you can get away with time steps that are much, much larger than the real physical scale.
So that allows you to go from 100 picoseconds up to maybe 100 nanoseconds.
Get rid of three orders of magnitude of time step by using a stiff solver.
So that's pretty good, three orders magnitude reduction?
Well go for it.
It's not only reducing the CPU time potentially, but it's even more importantly reducing the number of points you have to save, because when you run this calculation what Jacqueline Chen ends up with is a list, y at every x, xn, yn, el, times e, something like this.
Alpha, I don't know.
That's the output of her program.
OK, so how big is this object?
This is 200 times 10 to the fourth, times 10 to the fourth, times 10 to the fourth, times 10 to the eighth.
Right?
That's how many numbers she's going to get out of her simulation.
So, actually, her biggest problem is not actually solving the simulation.
As long as she can get the one month of CPU time she's good.
She's got to solve.
Her Problem is how to analyze her data.
Because now she has this much data that she has to figure out to say something rational about it.
And so she has like a whole army of post-docs whose whole job is trying to invent new day interrogation methods.
And also this data set is so large it cannot be stored on a hard disk.
In fact it cannot be stored in the biggest farm of hard disks.
So she has to, as the calculation is running, send the data to different farms of hard disks around the world and filling them completely up, and then sending them to the next one and filling them up.
So she has fragments of this data set stored all over the place.
And then if she wants to make a graph she has to have a special graphs software that looks inside this data farm, get some numbers, look inside this data farm and puts it all together in some way.
So this is a pretty hard project.
OK?
Now if I can use a stiff solver I don't need to use so many time stamps.
So I can get this down, say 10 to the fifth at least I got rid of three orders of magnitude in my data.
Right?
So now I just need one data farm maybe, instead of, you know, using all the ones at Amazon and Google own or something.
OK?
So this is a pretty big advantage to do it this way.
Now the disadvantage, as you know, is solving implicit differential equations is way more expensive, right?
Because you have to solve an implicit non-linear algebraic equation at every time step.
And in this case it can be a pretty complicated equation.
Because if I have this many state variables, so 10 to the fourth, times 10 to the fourth, times 10 to the fourth, so 10 to 12th times 200, so 10 to the 14th at each time step, that's my current state.
And I'm trying to solve a non-linear algebraic equation, 10 to the 14th unknowns, you need a pretty good solver.
I mean, I know backslash is good and F [?
solved ?]
and stuff, but I really don't think it's going to work.
So that's a really serious problem.
So the next trick is to say, OK, well this stiffness is only happening because of the chemistry.
So I don't-- I could solve all of the finite volumes separately.
So the idea is to split the problem up.
I'm going to solve the chemistry using a stiff solver only at one point a volume at a time.
And then I'm going to do the transport some other way.
So what I'm going to do is I'm going to rewrite this equation as My unknown is dy to t, is equal to some kind of transport term and some kind of reactant term.
And this term is local.
All this in this equation, this is really same dy at position x depends on this at position x.
Whereas if you compute these guys I need to do finite differences or something from adjoining finite volumes.
Right?
I have to do fluxes from the adjoining volumes.
So these guys are sensitive not just to my local value of y at this finite volume, but also to the neighboring values of y.
But if I don't worry about them, if I break it up I have this local thing and this non-local thing.
This one is local, it's stiff.
This one is non-local, but it's not so stiff.
And often it's linear or nearly linear.
Because the transport terms are mostly linear.
And so I can use-- I have specialized solvers that people have worked on for many years for each of these things.
So for the stiff thing I have things like ode15s, I have DAEPACK and [?
DASPAK ?]
and [?
DASL ?]
and sundials and all those programs that people have written and spent a lot of effort to solve this problem.
And similarly there's a whole bunch of people who have spent their whole life trying to solve the convection diffusion problems, the non-reacting flow problem.
And so I can use specialized solvers that they have.
So I want to split this up.
And what I really would like to solve is I'd prefer to solve dy/dt equals r and dy/dt equals t completely separately if I could.
Because I can use specialized methods that are the best for each.
So not to figure out, well this is not really the real equation because it has both of them added together.
How can I handle that?
So this operator splitting recipe is solve dy/dt equals to t for a while, and at the end of this process I'll have a [?
yat ?]
final.
And then I'll solve this equation with y equals y0.
That's the equation system of solving, of transport.
I do some transport for a while, I get some values for the y, then I take these guys and I solve the reaction equation starting from y equals y final.
How to write this?
One or two 0.
OK?
And I solve this one for a while and I'll get a different, you know, a somewhat different [?
yrt ?]
final.
All right?
And the simplest way of putting things is just keep going back and forth.
Boom, boom, boom, boom, boom, boom, boom, boom.
And in the limit where I make this t final minus t0 to be very small, then maybe I might expect that they might converge to the same solution as the couple of questions.
All right?
So that's the concept.
Now, if you do it this way it doesn't work very well.
You need to make this delta t incredibly small to get any accuracy.
However you can do a slightly more clever version, which is I saw dy/dt equal t starting from yt0, so y0, and I solve this to me y at t plus delta-t over 2.
Then I solve dy/dt is equal to r, starting from y of t0 equal to, call this y star.
y star and this I integrate out to y of t plus the full delta t. I'll call that y double star.
And then I solve this one again, and I solve starting from y of t0 equal to y double star, and I integrate that out for another delta t over 2.
And this one I called my y final.
And that's the one I use.
So I take a half of a transport step, sort of like I'm starting to do the transport, then I suddenly-- oh, hold on.
I got you my reaction.
I do my reaction, then I do the other half of my transport after that.
And if you can do the analysis of this, it turns out this thing converges to the true solution of the couple equations with second order accuracy.
So you might make this delta t, if I cut it in half the precision increased by factor of four.
If I cut if by a factor of 10 it increased by a factor of 100.
So this is looking more promising as a way to do it.
This is the most popular way to do it.
So this recipe here for operator splitting is called Strang splitting.
Named after Gill Strang, he was a professor in applied mathematics department here.
Former head of CIAM, the international body for applied mathematics.
And also author of a fantastically great linear algebra textbook which maybe some of you guys might have seen some videos of his.
So he invented this 1968, I was five years old.
He didn't know that his invention was of any practical use, being a mathematician.
So he just published that paper.
Now in the meantime, this paper is like a citation classic.
It's cited like, I don't know, 20,000 times.
He had no idea.
He never looks at citation list, he's a mathematician.
So I was working on this problem and-- let's see-- let's back up.
So this is how he usually solves it.
Right?
We usually solve the problems, the-- splitting, sorry.
Just be able to see it.
And we like to solve it this way because I can use specialized solvers.
I use a transport solver, I use a stiff solver here.
This one I can parallelize because I can do a different ode solution at each finite volume.
So if I have a big computer, a big parallel computer, say I have 10,000 nodes, I can solve 10,000 finite volumes simultaneously in this step, so the CPU time is reduced a lot because it parallelizes easily.
So this is really popular.
This is the main way that people do things.
And then in the ideal case you do it with a certain guess of delta t, then you try delta t divided by five.
And if you get the same result then you publish it.
OK?
And you've converged your solution.
However, what I found doing this is I was trying to find flames.
I was trying to calculate the positions of flames.
And what I found was that with the delta t's that I would try, sometimes I would get a qualitatively different solution depending on the delta t I chose.
And so in particular what I found was that I was solving burner flames.
So I have, like, a burner here, and it had a flame on top of it like this.
And there's two importantly different cases.
One is where the flame is floating above the burner.
And one is where the flame kind of goes all the way and touches the metal.
And you've probably seen both those cases, like a bunsen burner or something like that.
Sometimes if you get the flow rates high enough, the whole thing lift off and be floating there.
And sometimes if you slow the flow rates down they say that they will anchor to the metal.
Some piece the flame will actually touch the metal.
Usually it's like if you have a-- a lot of times you have a case like this.
You have like a metal piece, metal piece, here's your mixture coming in, out here is the air, and a lot of times the flame will actually touch this metal piece here.
You know, the flame sort of looks like that.
Have you ever seen a flame like this?
Anyways, this is a common thing, anchored-flame.
If you just have a slow speed burner that's what you get.
If you go high speed on the burner you can make the whole thing lift up and sort of dance around if you look at the flame.
All right?
You could probably see the same if you look at a piece of wood burning.
If it's burning well the flame will be lifted off the wood, you'll see a gap between the flame and the wood.
If it's not burning that well you can see bits of the flame actually touch the pieces of wood.
OK?
So that's a very important thing for a commission guy.
So was really-- to me this is a big deal.
So when I solved it with one choice of delta t I get the solution.
When I solved with another choice of delta t I get the solution, with a nice big gap.
Now they both can't be right.
And so I was a little bit alarmed because it's like, it's really different, right?
So I thought, you know, I didn't really care that much about the dynamics of my problem.
I just wanted to make sure that the steady state flame I ended up with was the right one.
So if you're interested in steady state problems this is a really important thing.
You want to make sure that your solution really converges to the true steady state.
So what I demonstrated primarily is the Strang splitting method is not reliable to converge steady state.
Now I believe I made the delta t tiny enough, probably I would converge the [INAUDIBLE] steady state.
But I don't know how tiny I have to go, and I can't really judge my convergence criteria very well, because at some point of delta t is it suddenly jumps from one solution to another solution that's qualitatively and completely different.
So the convergence is not smooth and continuous, it's like it's converging, converging, converging, jumps to some other solution.
And I don't know where the jump happens.
And I don't know if I keep going further, will it jump to something else?
So this makes me very loathe to publish the paper.
Right?
So I don't know.
So I was agitated about that.
So I start working on trying to fix this splitting.
The problem was that Strang is really smart.
So his splitting method is really good.
It's stable, it's pretty accurate, it's really hard to beat it actually.
And so I published the papers and they were that good because they really didn't really improve it that much.
But then I had a really smart post-doc named Ray Speth, now works in the AeroAstro department.
And he came up with a different way to look at this.
And he said, you know, if I had a problem that's like this, I can always add some constant to this and subtract the same constant from this, and I'm really getting the same problem.
And so now the thing is, can I cleverly choose this constant so that, for example, my steady state solution is going to be good?
So how to figure out what constants to choose?
And so the methods where you add these constants are called balancing methods.
So it's trying to balance the operator splitting scheme.
And what Ray's method-- pretty simple balancing method.
So this is called simple balancing.
Let's see if I have a slide that shows it.
There we go.
Here we go.
Simple balancing.
You choose a c to be the average of your right hand side at the previous time step.
OK, now idea here is that as you're getting close to a steady state, the previous time step is not going be that much different than the current time step.
And so this average number is going to stay more or less study as you as you get close.
And so what you're really doing is adding the, sort of, the average of these two guys to each of them from the previous step, and that has the consequence of really changing the solution.
So let me just show you.
So the left hand side is what Strang splitting does.
The blue chips are the transport, and the green steps are the reaction.
And what's happening is I did a half a transport step, then I do a reaction.
The reaction sort of jumps me off the trajectory, then brings me back to the trajectory, then overshoots because I do a whole delta t step of reaction.
And then I have to do a half step of transport back.
And so I get from the black-- one black dot is the first point, and the other black dot is what I call y final.
And that's showing what happens.
And it just repeats like that over and over again.
And it makes sense.
If you're close to a steady state, then if I'd just take one of these terms, without the balancing theta transport, it's going to head away from steady state, because at steady state the transport and the reaction are balancing against each other.
Typically you have something transporting in and is being consumed by the reaction.
Or something is being created by reaction and is being transported away.
And they have opposite signs.
If I only put one in that has a certain sign, then I'm going to move away from the steady state.
And then I need the other one to bring it back.
And the reason why Strang splitting has trouble is it's doing these huge excursions away from the steady state at each time step, with the hope that it ends up back at the steady state.
Right?
That's the way it's supposed to work out.
And to second order it does, that's because it's pretty clever about how it works.
But it does it's crazy excursions.
Now if you have a non-linear model, the further you do excursions from the real trajectory the bigger the area you get.
So if you do that balance method, you can see what happens is that when you get to steady state all the steps are just moving you along the steady state trajectory.
They're basically maintaining the steady state, and that's because I added-- it's like transport plus half of transport plus r or something like that.
And what is it?
There's probably a minus sign here.
And so I'm getting rid of half of my transport and I'm adding in half of this.
So I changed-- before I had dy/dt is equal to t, but now I have dy/dt is equal to 1/2 of t. I feel I need a sign to stick here.
So half of t minus f of r, something like that.
And the-- this can't be right.
I must have a sign.
Maybe this is not the average it's a half.
It should be minus.
It's like the-- these things should be added, and these guys are you cancelled out, so basically [INAUDIBLE] is going to be zero, which it should be as a steady state.
So I'm sorry, that previous slide had the sign wrong.
This is what happens as the solution.
If you do Strang splitting on the left you get more accuracy as you cut delta t, but it's only-- it's not getting any better.
It doesn't matter if it's near steady state or away from steady state, it treats everything equally, the Strang splitting.
In the balance splittings, as you get to steady state, all of a sudden the errors drive exponentially.
You see this litte logarithm of the error?
Logarithmic plot.
So you get tremendously great convergence to the real steady state once you get anywhere close to the steady state.
Now the cost of this is that because the formula, which has a sign mistake I think, it depends on yn, it makes the whole thing more numerically unstable.
Because it's like an explicit thing.
We're using what happened at a previous time to correct our equation for future times.
And so you are more likely to get numerical instabilities.
So then Ray invented an implicit version of that, and also a higher order method, and that's called re-balance splitting.
The equation is way too complicated for me to write down, since I can't even write down the minus signs correctly in this one.
But you're happy to read the paper, it's in CIAM.
And he has the equations for that.
And when you put the second order one in, that's implicit, then the thing is just as stable as Strang splitting, but it also has this great property that as you get close to steady state it exponentially converges to the solution, which is sort of like how Newton's method is really so good.
It's sort of like that.
You get a lot more decimal places really fast once you get close to steady state.
All right, that's enough.
Enjoy Thanksgiving.
I posted a bunch of readings and things about metropolis Monte Carlo and stochastic stuff that we're talking about next week, and also a little bit more about models versus data.
So do the reading when you get a chance.
Say hi to your families, say hi to your friends.
Have a nice time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so today we're going to keep on working on stochastic.
And I guess maybe we start-- if you guys have questions from the homework set, since I'm sure you're thinking about that a lot right now.
Are there any questions?
Should I take this to mean that you have it completely under control?
That's good.
All right-- so since we have Monte Carlo integration metropolis totally under control, let's talk about time dependent probability distributions.
So in many situations, we'd like to predict what's going to happen in an experiment.
And the fundamental equations of that experiment are time dependent, so we have different things happening at different times.
And so we really want to know what d dt of the probability of some observable-- this is going to equal something.
Let me fix the lights.
All right, so we want some equation like this.
And then when you actually made your measurement, you'd be sampling from P. So probability of the observables-- and so if you made the measurement, you'd have a certain probability that you observe something.
And you made a different measurement, you'd have same probability if you made the measurement at this time.
So this is going to be probability of the observable depending on the time.
And you made the measurements at different times.
So let's conceptually think about this.
Suppose we had a box.
And it had a uranium atom in it that's radioactive.
And it can decay.
So we look in the box.
And at time zero, I look in there.
And there's the uranium atom sitting there nicely.
And I go away for a while.
They come back.
And I open the box.
And I look again later.
And there's some probability that the uranium atom is still there.
And there's some probability that it fissioned during the time and it's gone, right?
OK so there must be some kind of equation like this, how the probability that the uranium atom is still there in the box.
Right, is this OK-- yeah?
So just for that case, what do we think the probability-- how does that probably behave-- any ideas?
So uranium has a half-life or something, right?
You guys heard this before, right?
So let's just try to think of how this should scale.
Whatever it is, it probably should have something to do with a time constant for how fast uranium decays, yeah?
You guys are looking at me, like, so blankly.
This is not that hard.
It's just one uranium atom.
It's OK. All right, so let's just write a plausible equation.
Maybe it's dP dt, where this is the probability that the uranium is in the box.
This might be the probability divided by two.
Something like that-- would that be-- all right, I guess we need a negative sign maybe.
So then the probability-- the solution to this would be that the probability that uranium is still there is going to be equal to some initial probability e to the negative time over tau.
Does that look right?
Does this look very reasonable?
You might have seen this before, yeah?
Maybe you didn't think of it probability.
Maybe you thought about it as number of uraniums in the box.
If I had written it as the number of uraniums is equal to the original number of uraniums e to the negative t over tau, you guys would have believed that right away, right?
OK, so I guess maybe this is highlighting a issue here, is that we use a macroscopic thinking.
We're used to thinking of things as continuum, right?
But if I only have one uranium there, it's either there or it's not there.
It's 1 or 0.
So the problem with this equation-- if you compute this, it computes irrational numbers, fractional numbers as the number of uraniums.
But of course, the number of uraniums, it's just either-- it's integers, right?
There's one uranium atom, or there's two, or there's three.
So this equation is not right, right?
So this is incorrect.
So that was what you learned in high school, right-- that equation?
But that's not right.
So this is really the correct equation, that the probability, that you have some number of uraniums that does something.
You know-- well this works for one atom, anyway.
So that's the equation for one.
So this is probability I have one uranium.
It's the initial probability that there's one uranium in a box, which is basically-- my case would be one.
I know I put one in there.
And then it just decays.
The probability decays.
But it doesn't mean uranium decays.
So it means when I do-- if I do the experiment once, and here I have the uranium-- the probability of the uranium is a simple exponential decay-- time.
I know I had a uranium atom there when I first put it there.
And at later times it's gone away, right?
'Cause at longer and longer times it's more and more likely it would have fissioned.
They Now if I actually do this.
And make a-- buy a million boxes.
And put a million uranium atoms-- one in each one to start with.
And then I just keep coming back once in a while and checking, what I'll see, is that initially I had-- here's the number of uraniums.
Initially I had a million uranium atoms.
And for some time period I still had a million.
And then something happened and one of them went away, because one of the fissioned, turned into lead-- or whatever they turn into.
All right, and so now I had a million minus one uraniums.
They live for a while.
And then maybe another one went away.
And then this time, another went away.
And that time it lasted longer-- right?
This is what you really expect to observe.
Is this OK?
And so this is sampling from the probability distribution that you would use.
Does that make sense?
So I'm sampling a million times, so I have a million boxes.
This is the probability distribution for one box.
And so I'm able to figure this out.
Is this OK?
All right, so we'd like to write equations like this for more complicated systems.
But you'll still going to have the same confusion, that the probability is going to be continuous variable.
And it's going to behave in ways that seem perfectly sensible to you.
But then every time you do the experiment, different things will happen in time.
Because you have something at the time-varying probability, and each time you sample from the probability distribution-- every time you do the experiment, it's like a particular instance of sampling from that probability distribution.
And you might get that there is 87 uranium atoms left.
And you might get that there's 93.
And it's not because of your measurement error.
It's because the intrinsic system has its own randomness to it, right?
You know, you look at one uranium atom-- some uranium atoms live for a million years.
And some uranium atoms might decay in the next second, right?
And you have no idea when they're going to decay.
All you know is statistically, on the average, uraniums decay with a certain half-life, which is pretty long, right?
All right, is this totally obvious?
I can't figure it out.
Is this, like, totally obvious, or totally confusing?
It's very funny-- with your eyes and your expressions, totally obvious and totally confusing has the same blank expression.
Can someone ask a question.
Maybe that would help us to have a-- this is OK?
All right, this is the
It's
This is OK. All right, so it's totally obvious-- good, hard to tell sometimes.
All right, so Joe Scott wrote some brilliant notes trying to explain how do this for chemical kinetics, OK?
So you should definitely read them.
They're in the-- posted under the materials, general section-- Stochastic Chemical Kinetics-- nice 20 page long thing, something like that.
It's definitely worthwhile to read it.
He was very nice at writing very clear notes.
So I'll try to explain his notes inexpertly, in a short period of time.
I strongly recommend you read them.
So the idea is that the chemical kinetic equations you guys have all used are also incorrect, right-- just like this is incorrect.
Because this is assuming that something that's really a discrete variable is a continuous variable.
And we do this all the time, right?
We know that material is made of atoms.
And so when you measure our mass in a certain volume, there only can be certain discrete values.
But we always ignore that.
And we always treat is a continuous variable, right?
So you have some flow-- some amount of liquid-- and you have some amount of mass.
And it could be any number.
And we treat dm dt, or things like dm dx, d rho dx-- we write those down as if everything is perfectly continuous variables.
And that's because that in a lot of systems we have, we have so many atoms-- and we can't measure that well anyway-- that we couldn't tell if we were missing one or two atoms.
Whether we end up with an extra half an atom in our equations or not makes no difference, because we're not that precise, right?
So we don't worry about it.
So we use contiuum equations everywhere and-- but in many cases, what we really should be using are continuous equations for probabilities.
And then as going to show you how you do it for chemical kinetics in a minute.
I'd say that this is super fundamental, actually.
So quantum mechanics says that everything is wave functions, which is actually like the square root of a probability density.
And the real equations are really probability densities.
And so that's the real fundamental equations.
Really, everything is probabilities.
And when you make an experiment, you're just sampling from the probability.
And all our continuum equations are like approximations.
They're only valid in, like, the large number of atoms limit, OK?
I know this takes maybe a change of thought, because you're so used to those continuum equations you start to believe they're really true, right?
You've probably used them a lot.
And you start to feel like you love those equations.
They're your favorite equations.
You know, you have a relationship with them.
Some of them you might hate, some you love.
But at least, you have a strong connection with them.
And you just find out that actually they're not even related to the real equations.
The real equations are actually something different.
This can be very annoying, but it's true.
And when you get to smaller and smaller systems that have fewer and fewer molecules in them, then you get more and more sensitive to the fact that the regular equations are incorrect.
And this comes up a lot in small systems.
So many of you probably will do research that has to do with biology.
In biology there's cells.
Inside cells there's subcellular structures.
Inside those subcellular structures, there's not very many molecules.
And so you often are at the limit where you have, like, one molecule of a certain type inside this mitochondria, or something.
And that's it.
And so you wouldn't expect the continuum equations to work.
And in fact there's a big field-- many people at research do in here-- called single molecule spectroscopy, where you can measure the motion of one DNA molecule or something like that.
And so you really see the individual molecules, and the effects of the individuality of the molecules.
And this is not only true in biology.
For a long time-- there's a field called emulsion polymerization.
You guys ever hear of this?
So like, all the paint on the walls here is made by that process, where they polymerize methyl methacrylate.
They want to make the little beads of polymers all have a small dispersion as possible, to make all the molecules the same.
So what they do is they make a colloidal suspension-- I'm going to draw a little picture.
So they have the reactor.
They make little bubbles of the monomer that are suspended in some solid.
And inside each of these bubbles-- because the way they make them, they're all about the same size.
So they all have a lot of bubbles, all about the same size.
And then they polymerize.
And they polymerize each of them.
And they polymerize.
And basically, all the material that's there polymerizes into one molecule.
Now how do they make that happen-- is that they introduce, say, a free radical.
And one free radical gets into one of these bubbles.
And it can polymerize the whole thing, because the reactions are a radical plus monomer goes to a polymer-- bigger polymer, right?
So polymer radical size n goes to polymer radical size n plus 1.
That's the polymerization reaction.
Now normally, if you did this in the bulk, there would be a lot of radicals.
And some of these guys would react with each other.
And the radical will be gone, because the two radicals would react, make a new chemical bond, and this process would stop.
And so this gives a big dispersion in the size of the polymer chains you get.
So to beat this, they do emulsion polymerization.
And they make that number of radicals so small that statistically, at any time, any of these droplets either has zero or one of radicals in it.
And there's not-- during the time it does the polymerization, there's not enough time for a second radical to come in.
So it's only got one radical, so that one radical is going to eat up every single monomer molecule in the whole thing.
And so it's going take a giant polymer, that's a whole size of the emulsion.
If you made the drops too big, you get two radicals in there.
And you get some of this happening.
And then you get a dispersion.
So this is emulsion polymerization.
All right, so cells have small little volumes.
Colloids and emulsions have small little volumes.
They all have-- that the-- some of the molecules-- the important molecules are so-- have such low concentrations-- the volumes are so small, that the concentration times the volume is less than 1, or about 1.
So that's for like-- you know, your copies of DNA in you, in your cells.
It's the same kind of thing for radicals-- free radicals in a polymerization system.
And similar things could happen with, like, explosions-- different kinds of rare events, that you might have a very small number of some really important molecule.
And then whether or not you have one or not is really important, OK. All right, so we want to model those kinds of systems, we have to worry about the graininess.
So we want to write down what the equation is dP dt.
So first you've got to think, well what is P?
Let's do a case where I have the reaction of A plus B goes to C, OK?
Suppose this is the only reaction that can happen in our system.
The only molecules we have in our system are A's, B's, and C's.
And so our probability distribution-- our probability that we want to know is how many A's, how many B's, how many C's do we have in our system, OK?
So that's the probability we're interested to know.
We could have 5 A's, 2 B's, and 0 C's.
We could have 3 A's, 7 B's, 11 C's-- who knows, right-- different numbers like that.
If it gets out to be 10 to the 23, then there's no point.
You don't have to do this, because you can just go back your continuum equations, right?
But if you have 5, or 3, or 2, or 1, or 0, then you really need to worry about the graininess.
So you want to write down an equation.
So here's a probability of n A n B n C. And it's going to equal to reactions which consume-- if I'm in this state that has a certain number A's, B's, and C's, and I can react away, so I only have reactions-- for example, I'm in this state.
An d plus B could react with each other, and will react away.
So I'll have some reaction.
I'm going to write a k, but we'll come back to what this really means exactly in a minute.
So this is the k for A plus B going to C. And we think that this is going to scale with the number of A's and the number of B's I've got, right?
And this depends on the probability that I have A B C. So if I'm in this state with n A's n B's n C's-- that probability is going to go down, because the A's and B's can react with each other, right?
Now on the other hand, I have plus the same k. And this is going to be n A prime n B prime.
If I have some other state that can react so that it makes this state, then I'll get a plus sign, because probability of this state would increase.
So for example, if n A prime is equal to n A plus 1.
And n B prime is equal to n B plus 1 and n C prime is equal to n C minus 1, then that state can react by one of these A's and one of these B's combining together to make a C. That will get me to this exact state here.
Does this make sense?
All Right, so that's the master equation if I have a system that could only do the irreversible reaction A plus B goes to C. So the only-- I can go away-- because the probability density of this-- the probability that this state can reduce, because that reaction occurs starting from this state.
And probability this state can increase, because I started from this other state here-- this special state here.
And that would make the state interest.
All right and so I have equation-- this equation over and over again, for every possible value of n A, n B, n C. Is this OK?
Now I have to actually-- remember for chemical kinetics, that whenever you have a forward reaction, you also have the reverse reaction.
So there's actually a couple more terms.
So now I have another loss term.
This is C goes to A plus B.
This is going to tell of the number of C's.
And that's P of n A n B n C. And then I have another source term.
And this is where I start from the state with one more C. Is that OK?
So this set of equations is called the kinetic master equation.
And this is the equation that describes the real system.
So this is the real probability of finding the system with any number of A's, B's, and C's.
And I have an equation like this for every different possible number of A's, number of B's, number of C's.
All right?
And it's not bad as an equation goes.
It's a linear differential equation, because it's linear in P's, right?
There's just first-- everything's first power in P. So in general, I can rewrite this as dP dt is the some matrix times B.
And the matrix entries are these prefactors-- things like this and this-- sorry, this, this.
These are the different elements in the matrix, m. So you should go ahead and be able to write any kinetic equation system-- rewrite it in this form if you want.
It's called the kinetic master equation.
Now if I do an experiment like I talked about with uranium, where I put one uranium atom in there.
And all uranium atoms can react to do is fall apart to lead plus neutrons, or whatever the heck they fall apart to.
So I can write down-- there's like one reaction that uranium nuclei do.
And so I can write it out.
It will just be something like this.
And this equation will work for a case where I put one uranium atom in there to begin with.
I could do the case where I put two uranium atoms in there to begin with.
I could write this down.
Is that all right?
OK, now if I had a case where I had 27 A's, 14 B's, and 33 C's to start with, then you can see that the number of possibilities here might get pretty large.
'Cause I have to consider, well, I could lose one A, and lose one B, and get one more C. But then I'll have an equation for this term which will look just like this.
But it'll go to ones where I have lost two A's and two B's and gained two C's.
And then all the way down, until you get down to one of the n's to be zero.
And so it could be quite a lot of possible equations, right?
So the dimension-- the length of the vector P might get really long, right?
So the length of P is something like the max number of A's, max number of B's, max number of C's.
All right, so if I have-- I'll allow-- consider systems with up to 10 A's, 10 B's, 10 C's-- then this is 1,000-- P's a length of approximately 1,000.
And it might be less than that, because maybe there's some of them-- some particular states I can't get to.
So maybe it's less than 1,000, but something like 1,000.
On the other hand, if I allow it to a million A's, a million B's, and millions C's, then I, like, 10 to the 18 length of P. So P gets pretty long, pretty fast when the number of possible states goes up.
Is that all right?
So-- now, it's linear though.
And it's linear.
And this is constant coefficients.
And so this is really not-- and sparse.
So this is really-- it's not that bad.
And in fact Professor Barton's group has worked on these kind of problems.
And he has to solves that can work with P has dimension up to 200 million.
So if you can keep the number-- say it's 200 million or less, you can actually solve this exactly.
Now solving this exactly is really good, because then you actually have the exact probability of any observable measurement that you would possibly make at any time, all computed, OK?
So that's really great.
But as I just pointed out, if I make these numbers, like, 10 to the 6, 10 to the 6, and 10 to the 6-- this is like 10 of the 18.
That's a lot bigger than 200 million.
And so Professor Barton's numerical methods are going to poop out at some point, and not going to work for you anymore.
So there's a limited number of problems you can do.
So if your problem small enough, you can just solve this-- for really small ones you can solve this with OD45 or OD15s But because it's linear, there's special solution methods.
I don't know if you remember solutions of linear equations this, you can relate them to the eigenvectors and eigenvalues of m. And so you can do-- that's one way to do it.
And then there are special methods you can do.
Talk to Professor Barton.
If it's sparse-- and it has certain kinds properties, you can make this really-- you can make quite large systems, up to 10 to the 8 kind of size.
If we get a system with more than 10 to 8 possible states, that means more than 10 to the 8 possible things could be observed or the result of experiment could have more than 10 to 8 possibilities, then this is-- you're not going to be able to solve it this way.
And so then we're going to have to use a stochastic solid solution method to try to figure-- to do-- so we're going to try a sample from P of t. So I guess I should comment, again, I wrote it this way.
But the thing-- P is definitely-- it depends on time.
So it's you'll have different P for every time.
Yeah?
[INAUDIBLE]
Yeah maybe I'll just do better to-- I can substitute it in here.
Is that all right?
So, like, if you transfer all the neighbors in it, that's--?
Yeah, 'cause a key idea is that chemical kinetics only does one step at a time.
So if you have a reaction like this, at any instant the probability that two reactions are going to be happening exactly the same instant is negligible.
So at any instant you're only changing the numbers of the atoms by one unit, yeah
Then shouldn't you add another A minus 1 and an E minus 1?
Maybe I got this backwards.
n A plus 1 plus 1 minus 1, minus 1 minus 1 plus 1-- I think got-- I think this is right.
So this is for the forward reaction that always reduces the number of A's and B's.
And this is the reverse reaction that always increases them.
And so I think you generally get like this.
You always get-- for any reaction, you'll have four terms-- so the forward and reverse, starting from initial state, and the forward and reverse that make your state.
And then if you add more reactions, you have a system where you've got a lot more reactions.
Maybe you have C's that can just decompose into B's or something-- different reactions.
Then you'd have a sum like this with terms like this from every reaction.
And you'd go through all your list of all your reaction.
I guess there's one funny thing I should point out to you about this, which is very important for this emulsion polymerization case-- Is that if you have a reaction of the type A plus A goes to-- I don't know-- B.
That might be a reaction you could have.
That when I write down the equation, if I only have one A in my sample, this probability of this happening is 0, right?
Because I need two A's for this to happen.
So really, the equation for this case is n A times n A minus 1.
So it's k A plus A goes to B times n A times n A minus 1.
And this is crucial for, like, the emulsion polymerization case, 'cause the fact that I don't have the second one is really important.
Otherwise I always miscompute that this radical could react with itself and terminate itself, which is not true.
Is that all right?
I commented that these k's are a little bit odd.
So if you remember, if you have a k for a unimolecular reaction, like C goes to A plus B, what units would it have?
What are the units for regular kinetics?
Inverse seconds
Right, inverse seconds, right?
So you would normally say this thing has units of inverse seconds.
And that works fine here, right?
Because I have dP dt, and it's in for a seconds times P, so it's fine.
However, this guy-- to make the units work out, still has to also be inverse seconds, but that's not normally what you would have for a bimolecular reaction.
And the other thing to watch out for here is that I have n's.
But normally you would write them with concentrations, right?
You have the-- you'd would write something like k times the concentration of C. It's the normal way you write your equations, the right equations.
But this is not the same, right?
This is n. This is unitless.
Now it turns out, in this case, actually, you can use exactly the same as k and it works.
But for the bimolecular case, it definitely does not work.
The units are wrong.
And also this concentration thing implies that we think the rate really has-- suppose we have k A k B. I could rewrite this as k times n A over the volume, n B over the volume.
Is that all right?
Actually one thing I should warn you about-- that usually when I write this, I write moles per liter.
So I have factors of 10 to the 23 between this and this, because these n's have got to be the individual molecules for this to work.
So watch out.
First of all, there's a 10 to the 23 somewhere in here, right-- Avogadro's number.
And then this is what-- how we would normally write it.
But this k has the wrong units for us here.
And so this guy here is the normal k divided by the volume.
I think that's right.
Maybe it's the other way around.
No, this is normal k times the volume.
That's right.
So normally we would write units k normal-- normal kinetics, macro-- k for A plus B reactions, the units are centimeters cubed per mole second.
But we want them to be units of-- the new k has got to be units of per second.
And so the new k is equal to the old k new old times the volume.
Is that right-- no divided by the volume.
I was right the first time.
This is volume-- yeah.
Thank you-- centimetres cubed, centimeters cubed-- yeah, right.
Cancelled out, good for a second.
So this is k per volume normal k divided by n-- correct.
And you may wonder, well how come it's only volume to the first power?
Before I had volume to the second power in the denominator.
And it's because I'm using n's up in the numerator.
And it's just like here.
I'm using n's instead of n over v. So I've sort of multiplied through by v everywhere.
That's one way to look at it.
All right, no questions yet?
Yeah?
So when you [INAUDIBLE]
Yeah, so I need a 10 to the 23 as well.
So there's got to be an Avogadro's number.
And you'll have to help me where the Avogadro's number goes.
But it's got to be in there somewhere, to get from moles to molecules.
So the numbers are a lot different.
Is this OK?
And this factor, this n minus 1 thing-- again, when these are macroscopic numbers like 10 to the 23, this makes no difference, right?
Because it's minus 1.
So we can fit our data ignoring the 1-- It's 1.
Perfectly well, the law of mass action-- it works fine.
So you never notice this unless you get down to where n is very small.
But this is actually the truth.
This is really what the correct equation is.
OK so now we have these-- the true equations for predicting how the probability distribution evolves in a reacting system.
By the way, you can add transport in as well if you want.
So you have control volumes, and you know formulas for the rate at which stuff is transported in and out of the control volume.
You can add those terms in as well.
And they'll affect the probability distribution.
No problem, you should be able to do the same conversion.
So just-- we've converted the reaction equations.
You can write a similar thing.
Once you get right the macroscopic transport equation, you can figure out how you would reduce it down to a time constant for transport that would work for the-- for this microscopic version of the equations.
So transport reaction-- you can write the whole thing down as a master equation.
That's a real solution, the real equation.
And the problem is that the real equation oftentimes is too big, because P scales up so rapidly with the number of possible states.
So then we have to figure out how are we going to solve it.
And the way to solve it is by-- called kinetic Monte Carlo.
And this method was invented by a guy named Joe Gillespie.
And the journal paper is posted.
And this is just like with Monte Carlo-- Metropolis Monte Carlo that you just did.
We're trying to find a way to sample from the true probability density without ever computing the true probability density.
So in the Monte Carlo integration you're doing for homework, you're sampling from the true Boltzmann probability density, but you never actually supposedly write down what that P is, right?
And so we're going to do the same thing here, is we're going to-- from the differential equations-- from this differential equation system here, we're going to try to somehow never ever solve this directly, but instead just sample from the solution, P, which depends on time.
And the way it's done is really to try to follow individual trajectories of what could have happened, OK?
So at any time-step, we have some number of A's, B's and C's.
And we're going to say, well in the next time period, some small time period, what could happen?
And we're going to make the time period so small that the only thing that can happen is nothing happens, and A, B and C stay the same, or one event happens which will change one of the numbers, the A's, B's, and C's, by the same one reaction amount, OK?
And then after we accomplish that, now we have a new state-- A, B, and C with some new values.
And we'll do the same thing again.
And we just repeat that over and over.
So it's very brute force.
You're just doing, like, what a single system would do.
So for example, for the uranium atom case, we have one uranium atom in there.
We wait for some time period, say 20 minutes.
We compute the probability that the uranium would have decayed in the last 20 minutes.
It's pretty small.
And then we'll say, OK, now we'll draw a random number.
If the random number is less than that probability, then we think that the uranium is still there.
And if the random number is bigger than that probability, then the uranium is gone.
If it's still there, then we'll do it again.
And we'll just keep doing that over and over again.
And we'll sample what could happen.
The uranium atom could live for 38 years and 14 minutes, and then disappear.
And that would be one example.
And then we go back and we do it again from a different trajectory.
And we do it over and over and over again.
And we do it over and over again.
Those trajectories are sampling from what could happen, with the right probability density.
Does this make sense?
So the way Gillespie says to do it is figure out the expected time until something would happen.
We'll call this time tau.
And we'll pull-- actually let's not use tau, because that's the only one that-- I'll try to be consistent with Joe's notation.
He calls this thing 1/a.
So a is like the expected rate per second.
And 1/a is the expected time until something would happen.
And then we'll draw a random number r-- he calls it r1.
So we just call the rand function.
It gives me a number between 0 and 1.
And I'll say that my time-step, delta t-- which Joe calls tau, is 1/a the logarithm of 1/o.
So I think there should be a negative sign.
No, 1/r is positive.
That's right-- positive.
All right, so I choose a random number.
Suppose it's 1/2.
Then this fraction is 2-- 1 over 1/2 is 2.
Logarithm of 2 is some number bigger than 1.
And I pull that.
And I get some-- actually, some number less than 1-- logarithm of 2-- anyway, some positive number.
Multiply it by the expected time until something happens-- and that's the time, in this particular simulation, when the next thing happened.
And then I repeat this again, and again, and again.
And I get the period of time intervals between when something happened.
Now every time something happened, I have to figure out, well, what happened?
So if I look at my-- plot back here when I had the uranium, I waited a long-- in this particular case, here's my tau.
I'll call this tau 1-- the time that it took for the first thing to happen.
And then, well-- what could happen?
The only thing can happen with my single uranium atom case is it decayed.
So it went away.
All right, and then I do the random thing again.
And I draw.
And I get a different number this time.
How long it is 'till something happened.
Now notice-- maybe it's not so obvious-- that in the time until something happens changes as you run.
So for example, if I started out with 100 uranium atoms in 100 little boxes.
And I'm watching them.
After I've run this for a long time, I only have 50 uranium atoms left.
And that means that the average time period until something happens will be slower, because I don't have as many.
The more I have, the more rapidly something will happen.
Does that make sense?
So I have to compute this a thing at each iteration.
So I compute the time until something happens.
Then I'm going to compute what happened.
I'll tell you what that the second.
In the uranium case, the only thing that can happen is one of the uranium atoms decayed.
So I took that away.
Now I have to compute a again.
So a is the sum of all the things that can happen.
So in my case where I have 100 uranium atoms, I have 100 different boxes.
In each one of them, there is a rate at which uranium atom probability is decaying.
It's like 1 over the normal half life uranium-- or something like that.
And I add them up.
So a is the sum of the rates of everything that can happen.
So we just list out all the different things that can happen.
So in the case-- suppose I have two boxes-- two uranium atoms.
I can have decay of uranium atom number one or uranium atom number two.
So I have-- this thing would be equal to-- so two uranium atoms-- a is equal to 1 over tau normal for a uranium atom plus one over tau normal.
Because they're both normal uranium atoms.
So it's equal to 2/tau.
And so 1/a is half of the normal lifetime.
And that's when I would expect that something would happen by then, OK?
One of them would probably decay.
So that would be the a value I would use here.
Draw my random number, get my tau until something probably happened, then figure out what happens.
Now in the kinetics case, several different things can happen, right?
I wrote down if I'm in state n A n B n C, I-- the state can change by two different ways.
I can move out of that state this way or this way.
Two different reactions can happen-- a forward or reverse reaction.
And they would both take me out of the state.
So those are two different things that can happen.
And so I have to add those rates.
So I would add this number and this number.
And they would be the two things that could happen.
And that would give me the a value.
Now I figured out something happened.
But now I'm not doing uranium.
Now doing, say, number of A's.
Now I figured out something happened.
But now I have to figure out what happened.
And the number of A's could have gone up or gone down, because if it was a forward reaction, the number of A's went down.
If it was a reverse reaction, the number of A's went up.
So I have to decide which way is going to happen.
So I could have either the number of A's went down, or the number of A's went up.
Those are two possibilities.
I have to figure out which one happened.
I have to figure-- draw another random number to figure out which one probably happened.
So I'll list out the list of all the things that can happen with their rates.
And one of them, say, is twice as big as the other.
So that'll be twice as likely to happen.
So I have to choose a random number, and use that to choose the next one.
So first I figured out-- this is actually what we did first.
I figured out what my rates are.
Second thing is I figure out the time until something happened.
Third thing is I'm going to figure out what happened.
And what I do is I say-- I draw a random number called r2.
And if there's only two possibilities like in this case, then I can say if r2 is less than the rate of process one divided by the sum of the rates, which is the same as the a, right?
So this is-- it this is true, then process one happened.
On the other hand, if the rate-- if the random number I chose is larger than this ratio, then I'll say process two happened.
And that's how I decide whether a number of A's went up or went down.
And then I would recompute the a-- depending on what happened.
Suppose I do this.
And I found out that the number of A's went up-- because I-- by drawing the random number.
Then I'll go and I'll recompute the A.
Now the rates will all be different, because n A is bigger.
So the rate of the n A reaction is faster.
So I recompute it.
I redraw a number here, and get some different time, tau 2.
It's not the same as tau 1.
And then something happened here.
I do the same process here out what happened.
Maybe this time A reacted away and it went back down.
And then I do it again.
And this time I draw a longer time, tau 3.
And I do-- what happened?
And maybe this time, again, something reacted away.
And it went down.
And on, and on, and on-- and after my computer is done doing that for a while, like, after a long enough time, I say, OK, I'm done.
Oh-- I'm not really done.
I have to go back and start it again, and run another simulation to run through all the possibilities.
And I want to save all these results of all these simulations, because these guys are samples from the probability distribution.
And if I built histograms of stuff from those guys I can figure out what really is going to happen.
All right, so it's a way to represent the P. Questions?
All right-- perfectly clear?
All right, so what's nice about this is actually, it's not that hard once you do this once.
It's not hard to do it.
And you can do it for any problem at all.
And so it's like-- often cases people do this even for cases where you actually could solve-- you might be able to solve this equation.
A lot times, people won't do it.
They'll do this instead just because it's so easy.
So though maybe it looks hard to you right now, it's only a three step thing.
And just do it over, and over again.
You're not doing it.
The computer's doing it-- no big deal.
And so this one you have to, like, look at the sparsity pattern of your m-- and some complicated stuff.
So this is actually done extremely commonly.
All right, see you on Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right today, I would like to do a Kinetic Monte Carlo example with you in class and, also, talk a little bit about the connection between the master equations, master equation solution, the normal kinetic equations we use, and the-- how you handle these trajectories, you'll get how to Kinetic Monte Carlo.
All right, so let's just pick a simple example.
As we mentioned, in the real world, it's pretty easy for these examples to get completely out of hand, because the number of possible states can be so gigantic.
Let's all do a really super, super simple one here.
To try to keep that underhand, not worry about that.
When you do the real problems, that will be a major issue-- is if he number of states get out of hand.
So the simple example I'm gonna do is a real one from inside your body.
So inside your body, you have a thing called low density lipoprotein.
And there's little vesicles of fat, lipids, inside your body, actually, in your blood vessels.
And these are bad.
If you go to the doctor, they measure your LDL and your HDL.
And if your LDL number is too high then they yell at you for eating too much fat food.
And they make you exercise and stuff like that.
This has happened to me, so I know.
And there's a reason for this, is because the LDL is susceptible to oxidation.
And if too many of those lipid molecules peroxidize, it causes inflammation in your blood vessels.
And that can cause your blood vessels to constrict and stuff that eventually can lead to heart attacks.
And so that's why your doctor is alarmed if you have too much LDL, because you have a lot more material available that could be oxidized, that could kill you.
There's actually a really complicated, interesting story about what vitamin E does in the LDL, and how it interacts with vitamin C. And if you take my kinetics class, I'll tell you about that.
But we'll do the simplest case now.
There's no vitamin E, there's nobody Vitamin C. You just have some lipid, and it's oxidizing.
And let's model what's happening.
So what you have is a little bubble of fat.
So out here is water.
And here's my fat which I'll politely call lipid.
That sounds like not quite as fat, if you call it lipid, slimy.
And the lipid is susceptible to attack by this reaction.
So if I have some peroxy radical, and I have the lipid molecules, which are some hydrocarbon, they can react and make peroxide plus a radical.
And in your blood vessels, if you're not under the water or being killed, you have a lot of oxygen.
And so the carbon set of radicals you form immediately react with oxygen to make the peroxy radical back.
And so the net process-- we can kind of combine these steps, because this is super fast.
So it's like a microsecond times scale.
For that to happen in your body, because the oxygen concentration is so high in the blood.
And so this reaction is really roo, plus RAH, plus O2, to rooh, plus roo.
So the radical is just catalyzing the oxidation of your lipid into the peroxide.
When this level gets too high, then your body will detect it.
There's an inflation, and you'll have all kinds of trouble.
OK, so we like to understand this.
Now, these vesicles are very small, so there's a probability you might have only a very, very, small number of radicals in there.
Also, your body has a lot of antioxidants in it, like vitamin C and vitamin E and some other ones too.
And the antioxidants are trying to get rid of radicals.
So the number of radicals you'd expect the concentration-- background concentration is very tiny.
And because of that, this is tiny and the concentration is tiny.
So the concentration times the volume might also be tiny.
So the total number of radicals might be very, very small.
And so therefore, you might need to worry about the stochastic kinetics being different than the continuum kinetics.
OK, so that's the idea.
Now, on top of that, you also have a lot of these vesicles in your body.
So you might be able to somehow work up some continuum model, treat them all.
And in fact, I think that's what people did originally, is they just said well, let's take the total amount of LDL you got in your body and do a continuum model.
But as you will see, it's different, because the fact that the vesicles are small, each one might only have, say, one radical in it.
And that will actually behave quite differently than if you had a lot of radicals.
All right so, this is the kinetics, and the oxygen is almost always high concentration, so I'm not going to worry about that.
And I'm going to assume that we're not going let your lipid oxidize so much that the amount of lipid changes significantly.
So let's just, for now, we can just ignore this.
OK, maybe later we might go back and figure out how you'd account for the fact that these cells are being consumed, which would change the rates a little bit.
So we'll just assume that concentration is constant.
So you have a tiny amount of radicals.
And if you had the radicals in your vesicle, they're creating roh.
And you see, the ro is catalytic.
And so what we're gonna see is basically D. If you did it in classical kinetics, you'd write DR dt this is basically equals some constant times roo.
And then, I lumped these concentrations into this constant.
[INAUDIBLE] That all right?
All right, so that's the import process.
We also have another important process, is that if you get two radicals in there, and they bump into each other, they can kill each other off.
So two peroxy radicals can combine and make stable molecules.
And we don't care what stable molecules they really are.
What's important is that they get rid of the radicals, because the radicals are catalyzing the unfavorable oxidation.
In this program I started writing, I called this k3.
This one is k4, reaction four.
There's two other things happening.
Out here, I had some peroxy radicals.
They're floating around the water.
Some of them might go into my lipid.
I'll call that process K1.
There's some time constant, which new radicals arrive.
And then, I also have the possibility these guys could diffuse out.
So I call that k2.
And k1, I'll just treat as a constant.
So as soon as a background concentration radicals that's constant, and so this is a sum-- every 10 seconds or something, a radical arrives in a vesicle.
And this will depend on the concentration, the number of ro's.
So the more you have inside here, the faster the rate which they go out.
That all right?
And this just mass transfer.
But from the point of view of the equations, it doesn't care if it's mass transfer or chemical reaction.
They arrive in the same way.
All right, so that's the system.
And so let's look at the-- I should try to write the MATLAB code for this.
And maybe, we can try to finish this together in class.
So I have four processes-- arrival of new radical, diffusion of the radical and the vesicle, reaction to make the peroxide, and self-destruction of the radicals.
And I wrote down what each of these does.
So the first one increases the number of radicals by one.
The second one gets rid of a radical because it's diffused away.
The third one increases the amount of peroxide.
The last one gets rid of two radicals.
So we're going to try to compute a trajectory by the Kinetic Monte Carlo method.
And trajectory is going to be some time points, the number of radicals at that time point, the number of peroxides I have at that time point.
And as I run, I'm going to get many, many, different time points.
And each one will have a different number of radicals or a different number of peroxides.
And that's what I want to compute.
And then, at the end, after I have trajectories like that, big tables of these things, I want to somehow put it together to figure out on average what happened or something, try to understand it.
And so let's see, so my inputs are the initial numbers of radicals and peroxides in the vesicle, the vector of rate coefficients, the max amount of time I want to run the trajectory for-- not clock time, but time in the simulation.
So it's like how many microseconds or milliseconds or seconds I look at my lipid protein and see what it's doing.
And the maximum number of steps, because I don't want to get a trajectory list that's 10 to the ninth long, because over a while, my memory, my computer would cause me trouble.
So here they are, all the setups.
Everything is fine so far.
So here's the loop.
So the outermost loop is just to keep track of how many steps we have, which is going to be the number of entries we have in the trajectory table.
And the second one is really-- what we care about-- is well, the time is less than the maximum time of the simulation.
First, we want to figure out the time [INAUDIBLE] until something happens.
So following Joe Scott's notes, we compute this quantity called A, which is the sum of all the rates.
And that's equal to the rate of the first process.
And then, they're in the second process, which depends on the number of radicals.
And the rate of the third process all depends on the radicals.
And the rate of the fourth process, which is a number radicals times the number of radicals minus one.
Now k4 has to have units of per second.
But bi-molecular rates would normally have units of volume per second, per mole even-- volume is per molecule per second.
And so that k4 has to really be a normal kind of k divided by the volume.
And we're going to have to figure out what to do about that in a minute.
And then, we do the formula to get the-- Gillespie's formula for how to sample from an exponential decay, as good as that is.
So Gillespie thinks that the probability that something is going to happen, or the probability, the time, until the next thing happens is going as e to the negative a times t. So we're trying to sample from e to the negative, a, t. And that's what that crazy log of the random number is doing-- it's sampling from that distribution.
And so we get a [INAUDIBLE] that how long we've waited from the time the last thing happened until the next thing happened.
And now, we have to figure what happened.
So there's four different processes that could have happened.
A radical could have arrived.
A radical could have left.
A radical could have reacted with one of my hydrocarbon molecules, with one of my lipids, or the two radicals could have killed each other.
So we list these guys out.
A is the sum of all those processes.
P1 is the probability that the first process happened, that a new radical arrives.
So it's going to be the k for the first step, divided by a.
That one was zero water, because it's subterfuging it from outside.
So it's just a k of 1.
Probably the second step is k2 times n rad.
That's the rate of the second step divided by the total rate of all the processes.
And I'm going to add that onto p1 to make a vector of possible things that could happen.
So I'm picking a random number from zero to one.
And I want to make a little bar.
Here's the probability the first process happened.
And here's the probability that the second process happened.
And here's the probability the third process happened.
And here's the probability the fourth process happened.
I know that these guys have to add up to one, because I computed that something happened.
And so I'm going to try to compare my random number from zero to one to these breaks and sort it out into these four possibilities, and then figure out which thing happened.
OK, so this is the second step of Gillespie's algorithm.
All right, so maybe you guys can help me finish the code here.
If you want to go to Broadway in Chicago, you can go there.
So I need to write the next one.
So p3 is equal p2 plus what?
So step three is-- sorry?
Say again
[INAUDIBLE]
All right, that's enough, right?
So now, I get to choose a random number, so, say, x is equal to rand.
And then, I have to say if x is less than p1, than something.
What's it gonna be?
Then step number one happened.
So that means that n rad is equal to n rad plus one, so one radical transported in.
If x is less than p2-- [INAUDIBLE] Is that good?
[INAUDIBLE]
[INAUDIBLE] space.
Thank you.
What happens here?
So this is b, n rad is equal to n rad minus one?
[INAUDIBLE]
I think they also take care of that, right?
Is this right?
X is less than p3, than nroh.
Else and rand.
Does that look right?
Do I need an end?
Semi-colon
Semi-colon.
Semi-colon, thank you.
That would be really bad.
All right, yes, Ziggy?
So in the first problem, you have step size longer than that [INAUDIBLE] size
So that's not good.
It should be less than.
Thank you.
Any more bugs?
I think you need spaces after the brackets
Spaces after the brackets-- where?
Is that here?
[INAUDIBLE]
[INAUDIBLE] separate line
[INAUDIBLE]
I can do that.
It's easier to read anyway.
So that's good.
I definitely agree with that.
[INTERPOSING VOICES]
Is that right?
Yeah
You like this?
With all these [INAUDIBLE] sets, I don't need any n's, is that right?
[INAUDIBLE] after the other like that, no problem.
Just one end at the end.
[INTERPOSING VOICES]
That's good.
All right, so we're OK with this?
We think this is going to work?
[INAUDIBLE] [INTERPOSING VOICES]
All right, so now, we've computed what's going to happen.
Now, we need to store results somehow.
So I'm gonna say trajectory is equal to-- or trajectory of steps-- so step was zeros.
Now, step is one.
[INAUDIBLE] step is one, to start with.
[INTERPOSING VOICES]
Does this look good?
They had only commas, is that right?
No commas?
[INAUDIBLE]
All right, one step is zero.
I already have the first line filled in here.
[INTERPOSING VOICES]
Is that good?
We'll see.
I don't know if it's OK or not, but I'll believe you.
All right, so right now, we have a program that maybe or maybe not might actually compute something.
Is that right?
OK, now, we to figure out-- are we going to run this program?
So we need an initial.
So any suggestions?
[INTERPOSING VOICES]
All right, so let's do an initial-- how about n rad is equal to 1 and nroh is equal to 0.
Is that OK?
We're cool with that?
And how about k?
So we need a vector of k's.
Yup?
For your line 18, I think that's just the [INAUDIBLE]
Yes, thank you.
Yes?
All right, so let's think about what's reasonable for the k's.
So I think we can make the k for this stuff diffusing in very, very small.
So I don't know-- [INAUDIBLE] make a small, 1e minus 5 something.
Isn't that small-- only once an hour.
You think it might be a little too small?
And then, the next one is how fast do things diffuse out?
Now, what's that going to physically depend on?
It should be something like the diffusion.
Diffusion [INAUDIBLE] get back to the wall, back to the outside.
So it should be-- by one radical in here, what's the average time for it to diffuse back out?
Any ideas?
So if it does, say, just a random walk, maybe-- then you say that delta x squared is like d times tt.
OK, so delta x squared, maybe make this like the radius of our lipid vesicle, so that delta t is like r squared over d. Seems like a reasonable scaling.
And we actually want to rate, so we want the other way around.
So k2 is going to be something like d over r squared.
All right, so we need to pick a size of our LDL vesicles.
I don't actually know what they are.
I used to know this, but I don't remember.
Anybody want to make a guess, how big is a vesicle inside your blood vessel?
[INTERPOSING VOICES]
Micron
Micron, OK.
So let's pick a mircron.
So let's say r is equal to 10 to the minus 6 meters.
What's a diffusion constant in liquid phase?
[INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE] meter squared per second.
OK, so feasible.
[INTERPOSING VOICES]
Anybody know if it's meter squared or centimeter squared?
What?
[INTERPOSING VOICES]
You guys did the one of the drug.
What was the drugs one?
Was it 10 to the minus 6, meter squared or centimeter squared?
[INTERPOSING VOICES]
Meters squared.
OK, so for a light, smaller molecule, it's reasonable.
So this number should be something like 10 to the minus 5, divided by 10 to the minus 12, so something like 10 to the 7.
1, 8, and 7.
All right, now for the reaction, roo plus RH, we can go look it up on the [INAUDIBLE]..
They'll have a list of reactions like that.
For now, let's just guess.
And so let's say that k3-- k of roo plus rh-- these guys typically have 8 factors of 10 to the eighth leaders per mole second.
And they have ea's, about 15-- 15 k cals per mole over rt.
OK so now, we need to figure out how to turn this into the k3 we want.
Now, the k3 we want is really-- K3 is like this-- times the concentration of the rh, because we took the rh out of the problem, because we want this to have units of per second.
Is that right?
And this is in units of volume per meters cubed per mole second.
And this would be moles per meter cubed.
And so it's per second, which looks right.
OK so now, we need to have-- I understand we have a calculator?
We can try to calculate this number for room temperature.
And this does not mean room temperature is r time t-- actually for body temperature.
And the concentration of hydrocarbon-- it's actually the number of h's, because every h can be attacked in a hydrocarbon.
So you typically have densities of 0.8 grams per centimeter cubed.
It's for organic.
And you typically have two h atoms per 14 grams of ch2 groups, because you only see h2 groups in there.
S two h atoms.
And then, if we're going to try to keep moles here, we need 6 times 10 to the 23rd atoms per mole.
All right, and this is centimeters cubed, but this one has leaders, so I need 10 to the 3 centimeters cubed [INAUDIBLE]..
So again, verifying what your high school teacher taught you, that the first thing is to learn units.
[INAUDIBLE] So we take all these numbers.
This is the RH, and this is the k. And we can multiply them all together, and we should get something reasonable, maybe.
And since we have MATLAB we can make it do it for us.
So let's just do that.
So I think it's-- you have to help me, I can't read this very well.
1ea star exponential, [INAUDIBLE] 15,000, slash 1.987 times-- what's body temperature?
40ec you told me?
So it's 310 Kelvin maybe-- times all these numbers, 0.8 times 2.
Divide by 10 to the 23rd.
More factors in there.
OK, so 5 times 10 to the minus 25.
We think we got this right?
[INAUDIBLE]
But I want to get moles, because I have this in moles-- it's [INAUDIBLE].
Three times mole [INAUDIBLE] This is really mole of [INAUDIBLE] atoms.
OK, so we'll try and see what happens.
So 5 times 7 minus 25.
It does seem pretty small, so we have a problem.
And then the last reaction-- those reactions are typically 10 to the fifth, liters per mole second.
This is for a recombination of peroxy radicals.
So now, we have to get rid of the volume unit.
This is for roo-- sorry, [INAUDIBLE] Yeah?
Yes, question?
[INAUDIBLE]
One mole per [INAUDIBLE]..
Very good-- that's right, because [INAUDIBLE] that's just two atoms.
It's actually two moles.
Yes?
That would change the [INAUDIBLE] Only a factor of 10 to the 23rd.
So we'll get back.
Thank you.
Area 0.3, somewhere like it.
OK, so now let's go do the same thing with this one, ro plus roo.
So typically these reactions, the normal way they're written are numbers that are about 10 to 5th meters per mole second, for two radicals recombining.
So now, I need to figure out how can I change the volume here.
And the thing that is key is that I know the volume of this lipid particle.
So what I really want is this.
My k4 is going to be equal to normal k, divided by the volume of the reactor, because the rate we would write normally, which would be droodt-- it would be like negative 2k roo plus roo, times the moles, the concentration of roo squared.
This how we would normally write it.
And so we have to have the-- this unit is all right.
So this is the volume of the reactor.
So we were having nroo over v, so we might need Avogadro's number over here, because the rate here is in moles.
But now, we are going to do single molecules.
Yeah?
Do you ever actually use k4 in calculations?
Yeah, for a. I think an a is there.
Good question.
All right, so we think this should be like a volume per molecule or something like that, for one molecule.
So this should be k4, should be 10 to the 5th, meters per mole second, divided by 4/3 pi r cubed.
And then, we need a mole-- [INAUDIBLE] 23rd molecules.
And we're gonna have to make sure that the leaders and these guys match up, which is not going to be so easy.
You might use 10 to the minus 6 meters.
This is going to be meters cubed, so I need 10 to the 3 liters per liter cubed.
And I think that will all work out to be per second.
So let's try it.
So this would be 10 to the 5th over 4/3 pi is about 4-- 10 to the negative 18, 6 times 10 to the 23rd.
So all the big ones cancel each other out, is a 1 over 24.
Oh, I lost it.
That's bad.
OK, so one over 24,000.
It's a lot easier to do with a lot of people checking your work.
All right, so four times n minus 5.
So this is what we think k is.
Anything else we need.
We need a t-max, you want to guess the time?
You want to wait?
The time for the main reaction, the time constant is like seconds, right?
So if we made 1,000 seconds, a lot of stuff is going to happen, right?
So maybe 1,000 seconds will be OK for t-max.
So let's try this all.
Where's the main function.
The main function-- [INAUDIBLE] my n initial was simple 1, 0.
And k, which was decided was 1e minus 5, [INAUDIBLE],, 2.3, 4e, 25.
Now, if you just look at these rates for a second, I think you can see a problem we're going to have.
How many steps are we going to do?
I don't know.
10,000.
The time scales here-- this 27 is pretty fast.
So the stuff is going to diffuse out of there, because it's going to disappear, and then nothing is gonna happen.
Yeah?
[INAUDIBLE]
Centimeters squared, I thought so, yeah.
Thank you, so we think this is 10 to the minus 9.
So this is actually [INAUDIBLE] lower, is that right?
We're still gonna have a problem.
But it's not quite bad [INAUDIBLE] problem.
So let's fix that.
Um, so I think our problem is that the 1e minus 5 is actually too slow.
So radicals are diffusing in from outside at some rate, and it's got to be a rate so that if there was no reaction, basically, the concentrations of the stuff here and here might be about the same, same order of magnitude, of the stuff that dissolved in the water, dissolved in the lipid.
Maybe a couple of [INAUDIBLE] are different, but not a million times different.
And so we need to have the rate of this stuff coming in to be something reasonable, that's going to be consistent with having a chance of having some radical in there.
So now we to guess what we think the real outside world radical concentration is.
So maybe-- I don't know, what?
1e minus 10 moles per liter?
1e minus 6 moles per liter?
I don't know.
Any biologists here?
Do you know how many free radicals you have in your body?
[INAUDIBLE]
They can measure it, right?
So it can't be zero.
People talk about reactive oxygen species in your body, ROS.
Yes, so I don't know what it is, 10 to the minus 6, maybe-- 10 to the minus 8.
So we had the number for-- 1e3 e3 is for going out from the-- we have that rate leaving is 1e3 per second times the number in there divided by the volume, really.
Well, it's times the number, that's the rate that's leaving.
But we would think of this as a k. I don't know how to think of this, actually.
But for sure, the volume matters.
So in order for us to compute this, we use the r squared.
Now, another way you look at this is say well, is diffusion times the gradient in the concentration?
[INAUDIBLE]
[INAUDIBLE] Well, it's OK.
They would live in their little tiny thing for a millisecond, and then they diffuse out.
I think that actually sounds very reasonable.
It's only a micron.
It's a very tiny, little thing.
So the question is how do we complete the reverse?
So if we have stuff outside, and suppose we didn't have anything inside, we'd have some rate of diffusion of the radical from the outside coming in.
How do we think about it?
It's really got to be the inverse of this.
So if we think the time constant is 10 to the minus 3-- a millisecond to come out, it's probably about a millisecond to come in.
I don't know.
You think it's got to maintain whatever initial concentration we put there, one per volume, then it's got to be about the same.
So this can't be as tiny as 10 to the minus 5.
So if we take the average concentration as one, then this should 10 to the 3, I think.
If we think the average concentration is 10, then it should be 10 to the 4.
If we think the average concentration this 100, then it should be 10 to the five.
So let's try 10 to the 3.
So every millisecond, something comes in or goes out, some radical comes in and out.
You guys buy this?
Would you defend this to your boss?
You can blame it on Professor Green.
All right, here goes nothing.
Do you think it's going to work?
[LAUGHTER] Aww, aww, line 29.
What's wrong?
What's wrong with that?
Parentheses off?
[INAUDIBLE]
Maybe I didn't save the latest version, let's try again.
Ah, I didn't say the latest version.
That's what it is.
Now, here goes nothing.
That [INAUDIBLE] help.
Leave me alone.
All right, trajectory, the first column is the times, so it uses the x-axis.
And the last column is the oxidation, the products.
So let's see if anything peroxidized or not.
[LAUGHTER] [INTERPOSING VOICES]
Oh, it should have all the zeros, right?
So how can we do it when we [INAUDIBLE] that way.
That's really weird.
Try it again.
[INTERPOSING VOICES]
The time is 2 plus tau
[INAUDIBLE]
I guess if you look at the [INAUDIBLE] So trajectory-- so it only [INAUDIBLE] 2 points.
That's why it looks like that.
I don't know why it doesn't show the zeros.
Maybe they'll show on top of the origin there.
So why did it only give us two points?
[INAUDIBLE]
Yeah, also-- seems like a 307 oxidations, but why 307 all of a sudden?
That makes no sense, right-- because it should be one at a time.
I should be seeing one at a time coming in
[INAUDIBLE]
Yes, it's very worrying.
I agree, so we have a bug.
So where is our bug.
This is where I need your help.
OK guys, let's figure out why didn't this work.
So I'm storing-- by stepping the steps, they're going up.
Ah, I know what's wrong.
I need this step thing inside the loop here.
So all of this-- so let's see if I can explain this.
I'm storing them according to the step, but right now, the wild loop is just zipping around, and then, it ends at the step and just gets one.
So I'd rather start again.
It's gonna work this time?
You guys don't have any faith.
If people say that scientists and engineers don't have any faith, I think we have more faith than anybody else.
We believe stuff like this is going to work.
It's doing something now.
Who knows what?
[INTERPOSING VOICES]
Yup?
[INAUDIBLE] time until your arrival time, 7:15 [INAUDIBLE]
[INAUDIBLE] is, not good, right?
[INAUDIBLE]
Yeah, it's a very good point.
I have a nested loop that should not be, right?
[INAUDIBLE] You don't necessarily want to sample every arrival time.
You might want to sample [INAUDIBLE] So you might want to have two.
You don't necessarily want to [INAUDIBLE] every single--
Yes, yes, I agree
[INAUDIBLE]
OK, so let's see how we can fix this.
So we really want-- is it really a jump out?
If the number of steps gets too large, we just want to jump out, so we want to go to?
What do you think?
[INTERPOSING VOICES]
Break, that's what we want.
So while is not the correct thing to do here
Why would t [INAUDIBLE]
So we're stepping time, every time we'll do the steps.
And we just want a break.
[INAUDIBLE] step greater than max steps
Why don't you do it with a while loop [INAUDIBLE]
What happens if you do a wild loop like that?
Do you guys know?
Does it work?
[INTERPOSING VOICES]
It'll work.
OK, that's easy.
Double and-- I tired it with and this morning.
It didn't work, so that's why I stopped doing it.
Stuff like that will kill you, doesn't it?
All right, we gotta get rid of one of the ends at the end
[INAUDIBLE]
I think it's likely to cause a problem, why it's taking 10 million years.
So really, is there a reason it should be nested loop, right?
Yes
[INAUDIBLE]
All right, try again.
[INTERPOSING VOICES]
That's too fast.
Try that again.
That's not good.
See we have a [INAUDIBLE] here
Oh no
Oh, jumps to one.
That doesn't look very good.
Try that again.
Boom
Oh!
Oh, that was pretty interesting, so much more like what you expect, so that at some time steps, I have one, two, three, or four, or five radicals in this vesicle.
I only have one time separate because of the nine.
So I think this actually looks like what I expect.
I should see it jumping up and down, stuff goes in and out, sometimes it reacts, sometimes it goes away.
So this one is correct.
I don't know what's happened with the other one.
So I'll have to figure out what's wrong with that equation.
All right, we're almost done time.
Well, I want to talk about for just one minute is-- what are we gonna do after we have this working?
So we get this working, we have the trajectory, we really want to run 10,000 trajectories, because all we're doing is sampling from the probability, just the distribution of time.
So we need to run zillions of them.
So this whole thing we just wrote would be inside a loop that runs a lot of different cases, because every time you run this, because the random number, you're going to get a different trajectory.
And what you care about is some kind of average behavior.
Or you might want to know what's the percentage chance that I'm going to have [INAUDIBLE] oxidized and die?
So you might say I want to count what fraction of the trajectories end up with number of peroxide greater than some number, that means I'm dead.
And if that probability is too high, then I know I'm in big trouble.
So that'd be one possible thing.
You might have whatever objective function you want, but you need to run a lot to get good statistics for anything.
So you're gonna have to embed this whole calculation inside a loop, and then, you're going to get a zillion of these trajectories out.
And you figure out how are you going to analyze them in order to figure out what you want?
So one way is if you could add the trajectories up-- so I have a trajectory-- suppose I have the number roh's versus time.
And I do it once, and I have none.
Then I have one, and then I have two, and then, I wait longer, and I get three.
And then, I wait longer, I get four.
Whatever, something like that.
That's what it really looks like for one trajectory.
And then, the next time I run it, it starts in a slightly different time.
And this time it runs longer before something else happens.
And then, it gets here, and then maybe, it goes over here.
And I have a lot of them like that.
I have 10,000 trajectories, all look like that.
So if I could add them, then I can do an average, to get an average trajectory, so that would be one possible thing.
Or I might want to histogram, so I might pick one time point.
So like after 20 minutes, I want a histogram of what this looks like here or here.
One of them has four peroxides, and one of them has five peroxides.
And then the other 9,999-- some of the three, some of the whatever.
Yes, question?
[INAUDIBLE] you're not gonna get the same time points--
You're not going to get the same time points, that's right.
So see the first time it stopped, the first time point was here, the second time, the first time point was here.
And they'll all be different.
So one is I was asking a few of the students before the class started, there's got to be a program in MATLAB that will let you generate these kind of plots with the flat lines.
Or even a linear interpolation would be OK too.
But you need to have some way to add them up as continuous functions, because they all have different time points.
So that's one practical issue about it, because you want to pick some special time you care about, and you want to know what does the trajectory say, what does this trajectory say?
But you actually only have numbers here, just really you have that number and this number and this number.
That's all you got.
Yeah?
You can plot like a-- an out of time to get to n amount of roh's and then, that would be [INAUDIBLE]
OK, so then you plot [INAUDIBLE] versus time instead, that's right.
So that you just think of what you want.
You're going to have all this trajectory information.
And then you need to figure out what do you want, and then, what are you going to plot?
And what are you trying to compute?
And you might want to compute not only the number but the standard deviation of that number, because you don't know how many trajectories you have to run before the center of deviation is narrow enough that you'll be confident with it.
So this is a general problem with this whole approach-- is that what you're getting out are just samples of things that happen.
It's just like if you were doing a Monte Carlo, and you just got some of the energy values from the hydrogen peroxide calculation you guys did, you have 47 of those energy values.
What are you going to do with that?
So you have to figure out, how are you going to take the [INAUDIBLE] of this calculation.
It's sampling from the real solution, so it should be OK.
But what is it?
How are you going to handle that and use it as a means-- it's sort of like as if you ran a zillion experiments, and then, what would you do with all that data?
So that's like issue number one.
Now alternatively, you can rewrite this as the master equation and solve it as an ODE, in which case, you'll get explicitly p of each of these ends.
So n rad, nroh time-- it would be continuous in principle, but actually, the ODE solver gives you out random time points also.
So you get a similar kind of thing out from the ODE solver, except it'll give you probability for every possible range of these guys.
So if you have one radical, two radicals, three radicals, four radical, fie radicals, and number of peroxides from one to 1,000, or however many peroxides you get, you're going to get numbers.
So you have all these numbers, all these probabilities, at different times.
Again, what are you going to do with that?
So one thing people do a lot is they would plot, say, the number of roh's versus time.
So at different time points, compute the average over all your trajectories.
Were with, if you had this solution, you can compute that as nrooh, p, nrooh.
So like this, you're gonna have some kind of average if you do it that way.
And either one are fine, but you have to think of what you want, and then in both cases, you'll probably be interested in the dispersion.
So you [INAUDIBLE] want to worry about the n squared too.
We'll talk more about this on Monday.
All right and I posted a homework problem that was given last year about this, for Kinetic Monte Carlo for catalysis.
It's used a lot in heterogeneous catalysis.
And if you have extra time, feel free to do the problem.
It's a really good problem.
Those of you who are feeling like you don't have any extra time, and you're about to kill yourself, please don't kill yourself.
And instead, just send an email to me or Professor Swan asking for an extension.
And we can give you some more time to finish up the homeworks that's due tonight.
All right, talk to you later.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's get started.
This is my last lecture of the class, and I want to thank you guys.
This has been a really good class.
I really enjoyed the good questions on the forum especially.
So I don't know if you guys enjoyed it, but I liked it anyway.
So I wanted to give-- my lecture here is going to be a wrap up on the stochastic methods, and then professor Swan's going to give a review on Wednesday.
It will be the last lecture of this class.
And then the final is a week from today, I think.
Morning?
Yeah, morning.
All right.
All right.
So as you did in the homework and we talked about, there is a lot of multidimensional integrals we'd like to be able to evaluate.
And a lot of them have this form that is a probability density time some f that we're trying to evaluate.
That's the integrand.
And then I didn't draw in the millions of integral symbols around it.
But usually x is a very high dimensional thing, so you have to integrate a lot of things.
And often we actually don't really know p. We know a waiting factor, w. So I just drew three of the integrals there, but there might be 10 to the 23rd integrals there.
So w is a lot easier for us to deal with than anything with all those integral signs.
So for example, the Boltzmann distribution is one of these things.
And also, just recall back earlier, we were talking about models versus data.
The Bayesian analysis of experiments says that you take your likelihood function, the likelihood that you would have observed the data you did if the parameters were true for certain value, theta.
And then you multiply that times the prior knowledge of the parameter values, and that gives you the weighting factor really for any integrals involving parameters, because it's really giving you the w of data given everything you know.
Prior information, and you also know the new data you measured.
And put them all together, you get the weighting factor for the parameters.
And so you can evaluate all kinds of multidimensional integrals over all the parameters in the problem.
So very often you might do experiments where you maybe have four or five or six adjustable parameters, theta, some of which you know something about ahead of time, and you want to put them together this way.
And that should be a summary of everything you know.
But it's sort of a goofball summary, because it's a multidimensional function, right, a function with a lot of variables.
But now you know how to do integrals of functions with lots of variables.
So you can compute things like some linear combination of-- what's the expectation value of some function of those parameters.
So if f, f here was a function of theta-- why is it bouncing?
OK.
I wonder why you guys are always look like this.
All right.
So f of theta, of any function you want, you can evaluate-- it's a function of the parameters, like prediction of what will happen in some new experiment that depends on the parameters.
You'd have it as f of theta, and then you could integrate it over w of theta to get the expected value of the result of the new experiment.
And also actually the distribution you have to get that way.
Does this make any sense?
I see one person thinks it makes sense.
OK.
So just because we did Monte Carlo using Boltzmann, which is a common one to use, there's other weighting factors.
I guess that's my comment.
And anytime you end up something that has some integrals with weighting factors, then you should think, oh, I can use Metropolis Monte Carlo.
And also, anytime you have experiments, a lot of times you think of the, OK, I want to get the best fit least squares approach, which is what you already knew before I taught you anything.
But it's very important to remember that you can also write the actual probability density, the shape of the probability density the parameters.
And that way you can get all the correlations between the parameters and include all the previous knowledge of the parameters and stuff like that.
So it's a very important thing to remember from this course is this Bayesian formula.
All right.
But Metropolis Monte Carlo, as you saw in the homework, is not always that easy.
So you need to choose the step length in every dimension.
It's sort of the factor you're going to multiply times random numbers to figure out how far to step from one state to the next.
And it's not so obvious how to choose that before you start because you may not know that much about the shape of the integrand, right?
Actually, even with the strength of the p in the probability distribution, you might not be that sure about it.
And so if you accidentally choose it too large, what will happen is it tries to take big steps away from a probable state and ends up at really improbable states, and then most of the times it won't accept that transition, right?
Metropolis Monte Carlo will just repeat the same state over and over again.
So if you see you just keep sitting there getting the same value over and over and over again, then that's warning you that you must have chose your stepsize too large.
Alternatively, if you choose it too small, all your steps will be accepted because you're not moving anywhere.
So you're basically staying at the same point over and over again, except it's slightly different, because you took a little tiny step.
But that way you won't necessarily sample the whole range.
And you can see that by plotting.
They say the distance from the initial state position.
You initiated the Monte Carlo Markov chain at some state.
And if you plot probability distribution of density, the distance from all the new states to the original state, and if that number is super teeny tiny, then you might be a little concerned that you really didn't cover the whole space.
But again, you have to know what is super teeny tiny, so you have to have a range in mind.
Yeah?
Can you do this adaptively?
Yeah, yeah, yeah.
So good algorithms for this try to pick it up that there's a problem, like if it keeps on either not accepting any states, it would try to shrink the step, the delta.
Or if it's always excepting the states, then it might try to increase it.
You want accept actually most of the transitions because it makes it more efficient.
But you don't accept them all.
So you want to do some-- I don't know-- keep it to 0.9 probability or something like that, or 0.8.
And there is probably a whole journal paper about what's the optimal way to do the adaptive stepsizing to get to convert the rest.
Yeah?
How did you prevent your initial guess [INAUDIBLE]??
Well, to me, I don't think that throwing them out is a good idea.
I don't know.
People do it.
I think it's that your really just need to make enough steps that you are sampling everywhere, and then it shouldn't matter.
And a good thing to do is actually start from a completely different initial state and make sure you get the same value of the integral, and that will give you a lot more confidence that you're not sensitive to it.
If you're sensitive to it, you're in trouble.
Because you really won't know how many to throw away.
And you don't know if you actually even achieved the real integral.
I think in the hydrogen peroxide example, actually it was symmetrical, so it was a little bit didn't matter too much.
But it I had given you an asymmetrical one, there's like-- here's like the dihedral angle.
It has to do with the orientation of the h, like sort of out of the plane.
And it should be symmetrical that you can sort of plus and minus, and there's a high probability region over here and a high probability region over here.
And you start one of them, say, over here, and you just sample around here, you get a lot of points all have basically the same value dihedral angle.
And you never see any of these, then it's no good, right?
Now this particular case, you might still get the right answer by luck, because they're exactly symmetrical.
If this one is a little bit asymmetrical, then you'd get the wrong answer.
Does that make sense?
So you really want to make sure you're really sampling over all the physically accessible region.
But again, this is a common problem for us all the time, is that when you're doing a calculation, you want to know what the answer is before you start.
This is a absolutely critical thing.
Because how the heck you going to know if you have a bug?
Right?
The computer's going to give you some number at the end.
You have to know actually what the answer is before you start.
I know this seems weird, right?
But anyway, it's the truth.
If you're going to calculate anything, you really want to know what the answer is roughly.
What the units are, what the order of magnitude is.
Some things about it.
And ideally you should try to think of some simple test you can do to try to check whether your code's right, whether things are reasonable.
The reasonableness test, is it reasonable that what which you get.
So this is the same kind of thing.
Like you draw the H2O2 and say, oh, I think the dipole moment should be about such and such, because I know the typical charges on an H atom and O atom.
And if you get some number that's way off from that, then obviously you made a mistake somewhere, right?
It could be a mistake in your code, or it could just be you just didn't sample correctly because your delta was wrong, for example.
All right?
Now I know I've told this to students every time, and every time I tell this to students, they totally reject this idea that you should know the answer before you start.
But it's absolutely critical.
I mean, my whole job as a professor is I tell my students, please calculate this.
But before I tell them that, I know what the answer is, right, roughly.
And so that way when they come back and show it to me, I can say, oh, it's reasonable, OK, I believe them.
Yeah.
So I think the number should be 20 and they come back with 14.3, I say, 20, 14.3, they're pretty close.
OK, so I believe them.
And when I think it's 20 and they come back, it's 10 to the minus 6, then I'm pretty confident, I tell them, I think you must have made a mistake.
And they're like, no, no, no, I did a great job.
I'm sure my calculus is perfect.
And then I'm like, now we're really probing, and I think I'm like-- I don't know-- I have it in for them.
But no, actually I knew what the answer is, so obviously it can't be right.
So they must have made a mistake.
Anyway, you should be like it, too.
You want to be able to be self-critical about what do you expect your answers to be before you do them.
All right.
What else is problematic with Monte Carlo?
This is very problematic.
If you want to achieve high accuracy, you need a really large number of states.
And it's funny.
It's like you can get a pretty good accuracy with not very many samples because of the behavior of one over square root of n. So it has very big changes, that small values of n. And just a relatively small number of samples, you get something that's halfway reasonable.
But then if you have three significant figures and you want to get the fourth one, it's a killer because you have to do 100 times more effort, 100 times more samples to get one more significant figure.
And so you've already done 100,000, and now all the sudden you have to do 10 million samples.
And then you only get one more [INAUDIBLE] of that.
Now you have to do a billion samples, and it's like, this gets really out of hand.
So that's a really unfortunate thing about it.
But the nice behavior at the beginning is you get roughly-- if you only want a few significant figures, you can get them pretty darn fast with Monte Carlo, certainly a lot better than trying to do the trapezoid rule in multidimensions or something like that.
All right.
So you guys just did Monte Carlo.
Are there things I should tell you about or we should talk about, problems you ran into?
Now you see, when I set that homework problem up for you, I helped you out a lot.
I don't know if you noticed that.
So the original problem had, it's four atoms, they each have three xyz positions.
So there's 12 coordinates.
And then I did clever tricks to get it down to, I think, six, maybe.
OK.
So going from six dimensions to 12 dimensions, that's actually a pretty big deal.
And so if I have given you the original 12-dimensional problem, you could still compute it and actually use the same kind of code as you wrote.
But the number samples you'd need to get a good number gets really wildly different.
Right, and so again, this has to do with knowing the answer ahead of time.
You know, I knew, OK, that the magnitude of the dipole moment doesn't depend on the orientation of the molecule, so therefore, I'm going to get rid of all the rotational degrees of freedom.
I know it doesn't depend on the transitional position of the molecule, so therefore, I get rid of all those.
And so before I even did any calculation, I can tell you I can get rid of six degrees of freedom, so that'll help you a lot.
But it's a similar kind of thing.
You want to do that all the time yourself, is try to think what can I do that's easier than just doing the brute force problem?
Yeah?
So on that problem, changing the max stepsize, you could change the percentage of step-- or [INAUDIBLE]
Yeah
And I think the mean state roughly around the same, but the shape of distribution actually changed a little bit

So how is--
So I think it might be this kind of thing is one part of it.
It's like if you only integrate one lobe of the dihedral, you actually get a dipole moment value that's pretty reasonable, even if you totally missed this other lobe.
But if you did a different stepsize, you might start getting some samples over here, too.
So the distribution would look a lot different, but you still end up getting roughly the mean.
But also it's partly that the-- the good thing about the Monte Carlo method is that, the first few samples, no matter what they are, give you something that's on the order of magnitude of what the value of the thing is.
So even a really lousy sampling at the beginning still gives you something that's halfway reasonable for the average
So if you're trying to recreate the overall distribution histogram, then how do you know which max stepsize to choose, because [INAUDIBLE]
Yeah, so the rule of thumb I've heard is that people try to get an acceptance ratio between 0.2 and 0.8.
So it means you want-- Yeah.
But I really don't know.
I'm not an expert in this field.
But I'm sure you read papers and people have big discussions about what the acceptance ratio should be in order to taking big enough steps that you have a chance to get the weird stuff.
In this kind of one, it might be a problem if you're doing steps, say, in the dihedral, you may have to step quite a long distance to find the other lobe.
So sometimes having some pre-knowledge of what you think the shape of the things are is a big help.
Yeah?
No?
Sorry, maybe I didn't answer your question.
Is that--
Yeah
Yeah.
All right.
All right, so that's Monte Carlo.
To Then we talked about the more difficult problem of where the probability distribution is not stationary, and in fact, we don't even have the w written out.
All we have is a differential equation that divides the probability distribution.
And we did it here for the case of discrete states, and that's the right one to use, for example, for the kinetics equation, if you want to get them exactly right.
Now that differential equation looks really easy, right?
Just a the times p. It's a linear differential equation system.
You knew how to do that one probably in your second semester of undergraduate taking differential equations class, right?
They did dialyze the matrix.
Yeah?
Remember this?
So that looks really simple.
But the problem is that the number of states is so big.
So have any of you tried problem two on the optional homework?
You haven't tried?
Has anyone even looked at it?
I know somebody looked at it because I got some question about it.
All right.
At least one person looked at it.
Let me tell you guys briefly what this problem is.
It's a really relevant problem.
If you go work for Professor Jensen, Professor Braatz, maybe Professor Roman, you might ed up doing this calculation.
This is not a crazy calculation.
It's very simple.
All they have is like a chess board of sites on a catalytic surface.
And they some number of sites.
And this is what-- if you take a crystalline catalyst and cleave it, you'll have a whole repetitive pattern on the surface, and somewhere along there is some active site that does something.
You can tell that experimentally because you stick that catalyst in the presence of some gases, you make products that you didn't make when it wasn't there.
So it did something.
So the way people model this is they say, OK, there's some active sites on here, and there's probably one of them per unit cell maybe.
And the site can either be empty, like if I ultra high vacuum, I pump on it and heat it, should be nothing there.
And then if I expose it to a little bit of some gas A, some of the sites might have A molecules sticking on them now.
And I also exposed to some sites gas B.
Maybe one of these sites might have a B molecule on it, too.
And if A and B can react and make my product, C, then maybe they're sitting next to each other and they might react with each other.
OK, so this is the whole way you look at this problem.
And so all I want to know is, if I have some A's and B's sitting on the surface, sort of how often are they going to react?
And the problem is a little bit more complicated.
It says, well, suppose I also know if I put a whole lot of B on the surface, what I end up seeing is coke.
My whole surface gets totally coked up.
So let's model that by saying, well, if suppose two B's are next to each other, then some probably reacting to turn to some coke product that sticks there permanently and just poisons the catalyst.
OK?
And this is real life, too, right?
So if you're trying to run a catalytic process, a lot of times, you have to run with one reagent in great excess to keep the surface kind of clean, keep it covered by the unharmful reactant.
And then you let little bits of the other reactant come down and react with the A really quickly.
But you don't let too much of the second reactant come down, because it might cause a problem like dimerize or coke or something.
OK, so that's the model.
Some of these guys have A's, some of them have B's.
And in the unlikely case that two of these guys react, they both turn around and then turn them into S's.
Coke.
Soot.
And then that part's dead forever after that.
OK, so that's the model.
So in the case that they have, they had 100 sites.
I think it was 10 by 10 if I remember correctly.
So it's 100 sites, and each site can either be empty, or have an A molecule, or have a B molecule, or have a coke molecule, S. So there's four different states on each of 100 different sites.
So how many states are there altogether?
Its' four to the what?
100.
OK.
So that's a pretty big number.
So that's how many states.
It's about 10 to the 60th, I think.
All right.
So this is a very large number.
This is bigger than Avogadro's number.
This is really a lot of states.
You wouldn't have thought that just a stupid little problem with just a 10 by 10 piece of catalyst, and three different things that can stick there, would give you so many numbers, but it does.
So now I have a problem that if I have-- my p vector is now a probability that has a number for each of the 10 to the 60th possible states.
And then the matrix m is that number squared, right?
Dimension.
So it's 10 to the 120th power elements inside the matrix m. So even though the matrix m is very sparse, this might still be a problem, because 10 to the 120th is really big, OK?
And your computer can only hold 10 to the 9th or 10 to the 10th numbers in it, so this is going to be a problem.
And also that's not just what p is.
That's what p is at this instant, say, 1 millisecond after t naught.
And then if I wait to 1.1 milliseconds after t naught, p will be different.
So I have 10 to the 60th numbers that change with time in some way that I don't know.
OK?
So this is actually a really hard problem to solve it.
And so everybody uses Kinetic Monte Carlo to do it, because there's no way I can possibly even list all the elements of p. And in fact, if I sample, if there's 10 to 60th states, no matter how good my sampling algorithm is, I'm never going to sample 10 to the 60th states.
So there's a lot of states I'm never going to get-- not even get one sample of.
I'm just never going to encounter them at all, because there's just too many states.
But anyway, people do it anyway.
So we go ahead and we'll do the Gillespie algorithm, which you guys-- we talked about in class.
Now to compute a Gillespie trajectory, we have to compute two random numbers, right?
We compute one random number, it tells us how long we wait until the next thing happens.
And then we compute a second random number that tells us which of the many possible things that happened actually happened.
And so if I have a case like this where I had the-- to the original case.
What can happen, the A could react with the B, the B could react to the B.
The A could come back off.
The A can move to the next adjacent site.
The B can move to the next adjacent site.
A lot of things can happen, right?
So quite a variety of things can happen from this initial state.
And so you have to, if you're solving this problem, you have to write that down.
Yeah, John Paul?
So the special waiting times are [INAUDIBLE] process where we haven't decided there's going to be an arrival, [INAUDIBLE] arrival times by that [INAUDIBLE]
Yeah
And then we make something happen.
But I mean I feel like you could run the exact same simulation without counting the times.
Get the same answer and then after the fact come and [INAUDIBLE] into the system.
[INAUDIBLE] doesn't seem to be [INAUDIBLE]
Yes, that's right.
Yeah, that's true.
So you only do the time calculation because you care about the time.
If you don't care about the time, you only care about the steady-state solution, then you can probably do it some other way
You don't need to calculate the times inside that.
You could calculate the entire state space trajectory without any reference to the time, yeah?
Yes.
Yeah, so you can get the sequence of all things that happened if it didn't give you the time, because it doesn't matter, right?
For the time.
It just means the sequence of what happened
Yeah
The sequence of states is all that matters.
But if you want the kinetic information, then you also want the time, because it might be it sat in one state for a million years, and the other state--
[INAUDIBLE]
Yeah.
Yeah, that's right.
Let's go ahead.
OK, so maybe make this cheaper.
Make it one random number for j.
No?
OK. All right.
So things to just keep in mind about this.
So the cost here is to compute in random numbers.
We have to compute at least-- well, some number of random numbers.
I don't know how many.
And that number depends on the length of time I want to simulate and what my delta t is.
Where the delta t is sort of like the average time between the events happening.
It's like one over a in the [INAUDIBLE].. And so if that delta t is very small, and my time I want to simulate is very large, that means that each trajectory is going to have a lot of events.
And that means I'm going to have to generate a lot of random numbers.
And so generate one trajectory is going to be really expensive.
On the other hand, if the delta t were about the same size as the total time I'm simulating, then I might only get one or two events, and so I only have to generate four random numbers for each sample.
All right?
And I just have to keep in mind that I'm going to have a lot of low probability states that I'm not going to sample at all.
I might even have some high probability states I don't sample, because I just can't do enough samples.
And I had the same exact problem I have with the Metropolis Monte Carlo and with all these stochastic methods that the scaling is one over square root of n. So I can get a rough idea pretty easily, but I don't get a really precise number of anything, then I'm going to have a problem because I'm going to have trouble to do enough samples to really refine the number.
The key thing in this is that with anything like this where I'm going to have so many states, I'm going to run a lot of trajectories to get at all-- sample all the things that can happen.
And so I'm probably not going to be able to store all the trajectories I ran on my computer, because I'm going to run so many.
And so I really want to compute things on the fly, which means I should decide ahead of time all the averages, all the f's I'm trying to compute.
So I might only compute f and f squared.
And I don't know, whatever else I want to compute.
The average value of the number of A's on the surface.
I mean, there might be a lot of diagnostics I can compute to just check everything's OK.
Think of that beforehand, coded to compute them as it runs, and then we only have to store the running averages of all those quantities, and I don't have to store anything about exactly which state sequence I hit.
Is that OK?
All right.
Now if you remember, in this equation, there was an initial condition of the right-hand side, which we totally ignored so far.
But that's actually pretty important.
And in a lot of real cases, from your macroscopic continuum view of things, you have an idea of the average value events.
And what you don't know is about the discreteness and you don't know about the correlations.
All you know is sort of an average.
I expect to have two A's and three B's in the surface or something from the partial pressure of A and B and the binding constants of A and B.
So I have some idea.
So I think I know these averages.
And so what people usually do is they use a Poisson distribution for the probability of the exact number of A.
So I suppose I think on average I'll have two A's.
, Well, I might have one A, I might have three A's on the surface.
And so I'll just use the Poisson distribution of n to get an estimate of sort of the expected width of distributions.
And so that's the formula for the Poisson distribution.
You have to know the average values of Na, Nb, Nc.
And then you can see that the N appears in the-- on average, N appears in the exponent and in the factorial [INAUDIBLE]..
Right?
And so really every time you start a new trajectory in Kinetic Monte Carlo, what you should do is sample from the Poisson distribution to get a new initial condition as well.
OK?
And that way you're sampling over both all the possible initial conditions and all the possible things that can happen to them to really get the real average of what you might want.
All right.
Now we just realized how horrible this problem is because it has so many states.
And so then we're going to have to think immediately, what can I do to make this faster and easier?
And so here are some things people do.
One thing is that people try to figure out what are the really fast processes, and do I care about them.
And some of the fast processes, you might say, well, like diffusion of A moving to here, and then it moves to here, and then it moves back to here, then it moves back to here.
That's of no interest to me.
I don't care where the A is bound on the surface.
All right?
The only way I care about it is whether the reacts of the B when it's sitting next to it.
In other respects, I really don't care about the diffusion.
So having the time concept for the diffusion be the real time concept might not be that important.
So then you can do different things.
One thing is you can assume it's infinitely fast, and you say, well, every time I look at a site, I assume that this A has a 1/10 chance of being here or here or here or here or here, all these different spots, as if it's equilibrated around all the empty sites.
That would be one possible way to do it.
That's the infinitely fast diffusion idea.
Another idea is to say, well, let's slow down the fusion to make it slower just to help out our computation.
So I say, well, on average, I might get one reaction a millisecond.
In reality, the diffusion time is a nanosecond.
But I don't care about all that nanosecond stuff.
Let's pretend that it's a tenth of a millisecond.
So it'll still be pretty much equilibrated on the time scale on the reactions, but that way I'll be able to accelerate my calculation by seven orders of magnitude by going from a nanosecond time scale to a tenth of a millisecond time scale.
So there's a lot of different tricks like that.
If you read a paper that does Kinetic Monte Carlo, you've got to read exactly what they did.
But they usually do something like this, because it's just totally out of hand if you try to model everything perfectly.
Also, the low probability events, the really low probability events, you're never going to sample.
So if I have a process that happens on my catalyst that takes 10 hours to happen, and my main reaction happens on a millisecond time scale, I'm never going to be able to run out to 10 hours.
So I might as well just forget it.
So if I know experimentally that the coking thing only happens on 10-hour time scale, I'd just take it out of there.
And I don't have to clutter up my calculation with all these S's.
I'm not going to form any of them in my time scale anyway.
And so therefore, I cut the number of states.
Instead of being 4 to the 100th, now it's 3 to the 100th because I got rid of all the coke.
And so now I've drastically reduced the size of my problem.
Not often you can do a reduction like this.
That's a pretty big reduction in the size of a problem.
All right?
And then you have to have an idea of what adequate sampling is.
So you're going to see some lower probability processes compared to some higher probability ones.
And you have to know, when are these low ones so small that they're statistical noise.
And this is the margin of error problem.
Do you ever see the polling, like in the political polls, they always say, so-and-so many people believe in evolution plus or minus something, right?
Well, the way they get the plus or minus is from the square root of the number of samples with a positive result.
So the margin of error on low probability things is much larger.
So the number of people who believe that Professor Greene is God is a very small number.
So if you find somebody like that, you have to figure they're within the margin of error, right?
OK?
But the number of people who are going to attend 1034 this morning is a big enough number that the margin of error in that is maybe two or three people.
It's not going to be 100 people.
That make sense?
All right.
So the more likely the event is, the more likely you'll get a lot of samples of it if you count the errors roughly the square root of the number of samples you got of that event, then the statistical error in your sample of the high probability events won't be that large.
Whereas for the low probability events, it could be really gigantic.
All right, so just for an example.
In the main reaction, A plus B is really fast because it's a good catalyst.
So I'm going to get a lot of trajectories going to show that reaction.
So that's good.
And I should get good samples [INAUDIBLE] But the coking reaction is really slow.
At least I hope it's slow, otherwise the catalyst is no good.
And so if it's really slow, I might not get very many of those guys.
And so even if I left it in the calculation, I might not be able to reach any conclusion because I might only see one when coking even out of all of the 100,000 trajectories I run.
And so then I won't know whether to say anything about that or not.
And then if I let the diffusion in and it's too fast, then my delta t is going to be too large, which means that my CPU time to compute a single trajectory is going to be too large, which means that I won't be able to get good sampling because I won't be able to run very many trajectories.
So I might want to do something to get rid of that.
All right.
Now there's another method that a lot of people use.
I think actually Professor Swan uses this sometimes.
Is that correct?
Yes.
So this is another method.
And what it is, is you solve the equations of motion of the atoms or clumps of atoms typically using Newton's equations of motion.
And typically people use it using force fields that were fitted to some experimental data, and maybe with some quantum chemistry data as well, to get some idea of the forces between the atoms and the force with which the molecules, when they bump into each other, how they interact.
And typically, it's done classically using Newton's equations of motions.
But if you don't like that, you can put in the quantum mechanical effects in a couple of different ways, and my groups, worked them one way called RPMD.
And you can get pretty good agreement with the quantum chemical results by doing this fancier version of molecular dynamics.
So there's some different equations you solve, but basically the same.
And so you can do it.
And there's a nice algorithm called the velocity of Verlet algorithm that almost everybody uses in this field.
And what's nice about that one is that it can do a lot of steps.
A lot of steps of moving the atoms around.
And after you do a million steps like that, you compute the energy by adding up all the potential energies and kinetic energies of all the atoms.
And it'll still be about the same energy as you started from.
Whereas a lot of methods, if you do the integral like that and you calculate the energy at the end, it won't be the same as energy you calculated because the methods have little round-off errors, and they kind of accumulate in a way that messes up the energy.
And so this particular algorithm is a nice O to E solver method that is good for-- has a property that is good for conserving the energy.
And then a lot of people use a thermostat, because they care about samples that are in contact with a thermal bath.
So you're sampling a few molecules, maybe 100 molecules or 1,000 molecules, but not 10 to the 23rd molecules.
So you have your 100 molecules or your 1,000 molecules, and you pretend that they're in contact with some bath, and you're watching those 100 molecules wiggle around, and their energy is not exactly conserved because they're exchanging energy with a thermal bath.
And so there's different things called thermostats which are computer ways of adjusting the velocities of the atoms periodically as if they got a kick from the thermal bath that makes them go up or down.
And that's very important trying to do, say, chemical kinetics, because their reactions are so slow that most of the time you'll watch the atoms wiggling around, nothing happens.
You need the unusual case when you get a big kick somewhere that gives you enough energy that you overcome a barrier to make a reaction happen.
All right.
So that's what molecular dynamics is.
And numerous people in the world and in this department and on the campus do these kind of calculations.
And there's two ways that they're used.
One way is used as an alternative to Metropolis Monte Carlo.
So you're trying to compute basically multidimensional integrals, basically it's from statistical mechanics integrals, more or less.
And instead of using Metropolis Monte Carlo, you decide to use molecular dynamics.
This is a tricky choice between those two options.
Nice thing about the molecular dynamics is I didn't have to choose any stepsize, basically, right?
it's like the time scale was set by the real physical motions of the atoms.
And if I don't want to think about it, I can just put in, what's the real physical time scale of the vibration of some atoms.
And I don't have to think about it at all and just run it.
But something should be a little bit thinking about is the molecular dynamics equations we're solving, it's really a time accurate method.
We actually get real time dependences.
We're using the real physical time, which turns out to be pretty darn fast.
So molecular vibrations or time scales are like tens of femtoseconds.
And so that's really fast.
And so you have a really tiny delta t. But if you're trying to actually compute a steady state property, then maybe this isn't-- it's not necessarily the best way to do it, OK?
Because you're doing something where you're doing extra effort to keep the time accuracy.
It's sort of along the lines of John Paul's question.
If you're doing Kinetic Monte Carlo, if you didn't really care about the time, then you don't have spend time computing the time, right?
And the same thing here is if you don't really care about the time, then you might want to use the Metropolis Monte Carlo instead, because if time's not really in your problem, then you can tailor it, take steps that don't really have to be physical or related to physical amounts of time, and you can still get the right integral.
Whereas here, it's going to necessarily do things that are exactly the physical amount of time.
And some processes in the real world are pretty darn slow, at least compared to 100 femtoseconds.
So you might have problems trying to compute by a time accurate method.
Now on the other hand, sometimes you want to compute time dependent properties.
And this is more or less an exact simulation of what the molecules really do on the time scale that you're simulating.
So if that's what you care about, like what's happening on the time scale of picoseconds and nanoseconds, this might be exactly right, because you're actually simulating with a tool that's time accurate on the time scale of what you're really trying to measure.
And so some chemical reactions, some kinds of energy transfer processes, like Professor Tisdale's group has exciton transfers that are happening on picosecond time scales.
It's tailored to that kind of problem.
OK, but this is the limitation.
So you have to use a very small delta t. Therefore, your total time is typically limited to nanoseconds as far as you can integrate, because you have to-- if you just count how many time steps you would need.
So if you're trying to determine some kind of static equilibrium property, you start from some initial gas at the positions of the atoms and the initial velocities, and then physically, it takes some time scale for that initial gas to relax to the real equilibrium, because you don't know the real equilibrium situation of the system.
And that time scale, if it's longer than nanoseconds, you're in trouble, because it's not going to be done before you've done the calculation.
And you never have even achieved the equilibrium situation.
Also, if you have a situation like that hydrogen peroxide case we talked about, suppose it takes a millisecond for the hydrogen peroxide to change confirmation from the dihedral angle being one way and to be the other way.
If I can only say, well, simulate for a nanosecond, then I'm never going to see that happen.
So I'm never going to jump from the one conformer to the other.
On the other hand, if I care what happens inside the one conformer, everything inside there probably happens on nanosecond time scales, and so I'll get the really good sampling of what's happening inside the one conformer.
And similarly for the dynamic processes, if you have a process that happens on nanosecond time scale, this is really the ideal way to do it.
If you have a process that's happening on millisecond or seconds or hours time scales, then this is not really the way to do it at all.
And then there's an initial condition problem with this.
It's kind of related to the Kinetic Monte Carlo initial condition problem.
So in the Kinetic Monte Carlo, we had to sample over all the possible initial conditions.
We did it with the Poisson distribution.
There is a similar issue here, is how do I start the initial conditions?
Where do I arrange the atoms to be to start out?
I really want to sample over all the different possible ways the molecules could be arranged.
And particularly if I have some-- say I have a protein.
And the protein has some conformation it likes to sit-in and then it can unfold and go to some other conformation.
Maybe there's two or three conformations like this.
The time scales for those changes, again, might be milliseconds.
I'm not going to be able to follow them with the molecular dynamic, so I need to have some sampling method to set me up in each of the different conformations that I want to sample.
And then I can follow very accurately what would happen over a couple nanoseconds after it's in that confirmation.
But I'll hardly ever see it actually achieve the other confirmation on the time scale.
So I guess what I want to say to you guys is these are different tools for really different purposes.
The Metropolis Monte Carlo, the Kinetic Monte Carlo, and the molecular dynamics, they're all good for some kinds of problems.
But none of them is good for all problems.
And I often get journal papers where somebody uses the wrong method for the wrong problem.
And so I have to reject it.
I've often had people come that want to do postdocs for me.
And they're talking about their thesis, and I see the poor kid has spent five years of his life using the wrong tool for the problem he's doing.
It's very sad.
So don't that be you, OK?
So don't just use a tool because you know it.
Say, does this tool work for this problem?
If not, I've got to find a new tool.
And just make sure you're using tools that match the problems you want that you're trying to solve.
I think that's all I got.
So I have 10 more minutes left in class.
Any questions you guys have?
Yeah?
Just have one on weighting functions for the Monte Carlo simulations.
I'm still a little shaky on how you always determine them.
It seems like it was given to us on the problem we did on the homework and then in the example in class, you just set it to an even distribution, or a uniform distribution?
Yeah, yeah, yeah
How do you choose what's best, how do you know [INAUDIBLE]?
Yeah.
Well, yes.
So you get some integral.
This integral of g of x something.
All right?
Where this is a lot.
And then you have to figure out, what am I going to do so I can solve this thing?
And the clever thing is to figure out, hmm, can I rewrite this as p of x times f of x, because I know how to solve those kind.
And actually, I don't even-- Yeah, and even if I don't know what p is really, maybe I can write it as w of x over some number of integrals here.
That will be my p of x.
So most of these things you can do with uniform distributions if you want.
But then your sampling can be extremely inefficient, because you'll sample a lot of regions that have very low probability of being really there.
But this is like a cleverness thing about can I figure out what's that p times f that's equal to g that's going to work the best for me.
And work the best, that's a good question.
So one way of working the best is, if f is most constant, because if f is perfectly constant, then I get the right answer in the first sample.
OK?
So that's one thing is trying to figure out f to be really pretty constant.
The second one is to try to figure out p that's very sharply focused.
So I don't need to sample a lot of x values to get it.
Now these two are kind of at odds with each other, the sharp p and the flat f. Probably again, probably whole people wrote their PhD thesis in applied mathematics about what's the optimal choice of p and f. A lot of times, I'm not that smart, so I just like, if I have a problem in stat mech, I just-- I'll always do a Boltzmann factor.
It may not really the best thing to do, but that's what I would do, right?
And if I was doing a Bayesian problem, it's sort of like the w is given to you, right?
Where's that?
That's the formula I know, so I'm going to use that w. Maybe there's a more clever way to do it, but that's what I normally would do.
Actually, one thing that I don't know if we've ever talked about explicitly, but very important to know, is that this kind of formula, these formulas have w of theta.
That's the joint probability that theta one is something, and theta two is something, and theta three is something, and theta four is something.
It's that probably density.
But a lot of times, you don't care about that level of detail.
Like you may have only a few of those parameters that matter to you.
So you really care, you would like to know p of-- I don't know, theta one, theta two.
And you'd like to get rid of all the rest of the thetas, because these are two thetas you really care about.
Maybe these are the ones you think you're controlling in your experiment, you're trying to determine.
In the other ones, somebody already measured the mass of the proton, so you really don't want to determine the mass of proton again, and you're not going to really say anything about it even if you did.
If your calculation says that it made the mass of proton a little bit different than what the standard book says, you might believe that in your heart that it's true, but you're probably not going to say it, because you're like, I better double check before I do that.
But I'm sure that it determines the length of my reactor, because I measured that with a meter stick, and nobody else knows that number, so I'm sure that's the theta I got that I'm really going to be in control of here.
So oftentimes you do this.
You want theta one, theta two, but you actually know theta one, theta two, theta three, quite a few of them from my formula.
And so you can do what's called a marginal integral.
Suppose I have-- I don't know-- theta three, theta four.
I could do D theta three, D theta four.
And this is like integrating out these degrees of freedom to get the probability density that I want, all right?
So if you have some case where you all you care about is the variance of something, and you don't care about all the rest, you can kind of integrate them out.
That's a very handy trick to do.
Well, you can do similar things with the Boltzmann things, like for example, a lot of the Boltzmann distributions, we don't actually care about the momenta, because what we measure, say, is like a crystallography thing.
We see positions of atoms.
We don't see velocities anyway.
So a lot of times, people will just integrate all the velocities out of the problem right at the beginning.
It depends on what you want, right?
You're in charge of your problem.
Any more questions?
Yes, Kristen
OK, well, I just have an announcement

[INAUDIBLE] today at office hours, we'll talk about KMC, probably from about 5:00 to 5:30, but come any other time [INAUDIBLE] questions.
And we just posted a poll for the final review session.
It's either going to be Wednesday evening or Friday morning, so if you could vote on that as soon as possible, that would be [INAUDIBLE]
OK, got that?
So final review is either Wednesday evening or Friday morning, you have to vote.
And if you come today, 5:00 to 5:30, it'll be all about Kinetic Monte Carlo.
And other times today are about anything you want to talk about.
And the homework solution will be posted shortly, I think, for the last homework that was graded.
Anything else?
All right, good luck as you study and good luck on the exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this is our last 10.34 lecture of the year.
And we're just going to use it for review.
So I'm going to give a brief recap of the course.
We've done a lot, actually.
I hope you've learned a lot over the course of the term.
We've had lots of opportunities to practice with different numerical methods.
I've checked in from time to time on the homework submissions.
I've been really impressed with the quality of the work that the class has produced as a whole.
It tells me that you guys do understand what you're doing, that you can apply these methods to problems of engineering interests.
And I hope that the things you've learned over the course of the term can be applied to your research down the road.
Right, that's sort of the goal here.
Whether you do experiments or modeling or theory, at some point, computation is going to play a role.
You should feel comfortable after that things that we've done in this course on reaching back to these tools and utilizing them to either solve simple transport models of a drug delivery problem that you're working on, or fitting data reliably, doing hypothesis testing of models versus data.
You have the tools now to do that.
The homework assignments that you've completed over the course of the term sort of lay out the framework in which that can be done.
And you seem very competent.
We'll test that competence on the final exam, which is Monday from 9:00 to 12:00 AM.
It's not in Walker, though, OK.
It's in 56-154.
I'll remind you of that again at the end.
But it's not in Walker, so don't show up there at 9:00 AM.
You'll be taking the wrong final exam.
It's in 56.
There's going to be a review session with the TAs on Friday.
It's going to be in this room.
It's going to be from 10:00 to 12:00 AM.
And the TAs asked me to ask you to review your grade on Stellar.
Make sure that everything is entered accurately.
So check that your assignment grades match the grades that are entered there.
Check that your quiz grades match the grades that are entered there.
You're going to take your final on Monday.
And we're going to grade it on Tuesday and submit your final grades, because some of us have to travel right after the final exam period.
So we want to make sure everything is as accurate as possible when we enter the final data.
OK?
Good.
OK, so a course recap.
We've done a lot this term, actually, right?
You should be really pleased with yourself for having made it this far.
Any one of these subjects could comprise an entire course in and of itself.
And you've gotten a broad overview of all of them.
So I'll remind you.
We started with linear algebra.
I told you the first day that linear algebra would be the underpinning of all the numerical methods that we utilized along the way.
And that was true.
I didn't lie.
You really wanted to understand linear algebra well if you were going to solve any problems numerically.
We moved on to things that were more interesting from an engineering context, systems of nonlinear equations and optimization.
So being able to solve engineering problems, being able to do engineering design by optimizing over various uncontrolled degrees of freedom is important.
And we did both constrained and unconstrained optimization.
Then we moved on to dynamics.
We did ODE initial value problems and DAEs, which are the combination of nonlinear equations and initial value problems.
We discussed boundary value problems and partial differential equations, which largely are a class of boundary value problems as well, at least the ones that chemical engineers are typically interested in.
And we switched a little bit.
We went into probability and models and data.
And then we came back to more modeling in the form of stochastic simulation.
And all along, this was threaded through with lots and lots of MATLAB.
So hopefully, you learned to do programming if you didn't know that before.
MATLAB's actually a very useful programming tool to know.
It's not so specific.
There aren't specific quirks and details you need to understand like you would with a lower level programming language like C or Fortran.
It's very high level, and the concepts that you utilize there can be translated to other programming languages as well.
So you may find at some point in your research you want to do a lot of data handling.
MATLAB's not great for handling data actually.
Parsing large data sets is quite difficult in MATLAB.
But there are tool kits that are built into programming environments like Python that are very efficient at doing that.
And Python is also underpinned by similar numerical methods that are readily accessible.
Everything you learned in MATLAB can be applied there as well.
So you should have a broad understanding of how to draft an algorithm, how to modularize code, how to integrate several different numerical methods together to form a solution to a more complex problem.
So we did linear algebra.
This was the basis of all the problem solving we did.
Principally, it was concerned with the solution of systems of linear equations, though oftentimes, that's sort of disguised.
It isn't a direct system of linear equations we're trying to solve.
But somehow, we need to decompose a matrix.
And in the process of decomposing it, we're essentially doing solution of systems of linear equations.
One thing we're really concerned with was how linear algebra operations scale with the size of the problem that we're interested in.
The bigger the system of equations gets, the slower the calculation is going to be.
And how slow can that get?
This will introduce fundamental limits on what we can do computationally.
When we try to solve partial differential equations, and we decompose them into linear equations, and we have huge numbers of states that we'd like to be able to resolve with a partial differential equation, the problem can get so complex so fast that a computer can't approach it.
So being able to estimate the computational cost of these operations is critical for deciding whether a problem is feasible or unfeasible.
You'd like to know that before you start writing down a numerical solution to the problems.
We focused a lot on that.
There was fundamentals like arithmetic, solving linear equations numerically using Gaussian elimination, for example.
We discussed decomposition in the form of eigenvalues and eigenvectors and, more generally, the singular value decomposition.
And we discussed briefly iterative solutions of linear equations.
That was sort of our segue into iterative solutions of nonlinear equations as well.
So that was things like the Jacobi method or the Gauss-Seidel method, which were ways of the really operator-splitting methods of the same sort of class as we talked about in PDEs, where we took that matrix and we split it into different parts.
We split it in a way that would make it convenient to solve a simpler system of linear equations.
And depending on the properties of the matrix, that solution might be achievable or not achievable.
The iterative method may not converge.
But for certain classes of problems, we could guarantee that it would converge, and it might do it very quickly.
Things we didn't cover, but that are important, I think.
There's a notion called linear operator theory, which is an extension of linear algebra to infinite dimensions.
You've seen some of this already in your transport class, for example, where you solve differential equations with sets of functions.
You have some infinite set of functions that you solve a differential equation with, an ordinary differential equation or a partial differential equation.
And all of the underpinnings of that fall under linear operator theory, which is an extension of what you've learned already for linear algebra.
We talked about Gaussian elimination and the LU decomposition.
You'll find in the literature there are lots of matrix decomposition methods.
And the ones that are chosen are chosen for often problem-specific reasons.
So there's things like the QR decomposition or the Cholesky decomposition or the Schur decomposition.
These are technical terms.
You can look them up.
And you should feel confident that, if you read about them, you should be able to understand them given the basis in linear algebra you have now.
But we didn't look at those in particular.
And the state of the art in linear algebra and solving systems of linear equations lies in what I refer to as Krylov subspace methods.
We talked about conjugate gradient as an iterative method for linear equations.
There's a broader view of that that says Krylov subspace method.
You may see that in the literature from time to time.
It's a more nuanced discussion of the linear dependence of columns in a matrix or linear independence and how you can take advantage of that to rapidly generate iterative solutions to those equations.
Any questions about linear algebra?
No?
Things are clear.
You've done a lot of it this term.
Systems of nonlinear equations.
So these are, of course, essential for any engineering problems of practical importance.
Linear equations tend to be easy to solve overall.
Most engineering problems that are interesting are nonlinear by nature.
That's what makes them interesting.
And we also saw later on that solving systems of nonlinear equations was intimately associated with optimization.
And so we discussed fundamentals like convergence of sequences and the rate of convergence of those sequences, stopping criteria for iterative methods.
The best sort of nonlinear equation solvers with multiple equations and multiple unknowns are Newton-Raphson-type methods.
They're the ones that are going to, if we're close enough to a solution, give us quadratic convergence, which is often more than we deserve.
It's exceptionally fast, but it's not a globally convergent sort of methodology.
So we also discuss quasi-Newton-Raphson methods that can help enhance the convergence of Newton-Raphson.
And then we knew that we needed good initial guesses for solving these problems.
We don't know how many solutions there are.
We have no idea where they're located.
And so we talked about homotopy and bifurcation as ways of generating those initial guesses and ways of tracking multiple routes or multiple solutions to the system of equations.
I mentioned briefly during that lecture the concept of arclength continuation, which is a more advanced sort of homotopy method.
And we didn't get to discuss that in detail because, actually, the underlying equation for arclength continuation is a DAE.
OK, it's a fully implicit DAE that one has to solve in order to track the solution path as a function of the homotopy parameter.
So we didn't cover this.
We didn't try to solve any arclength continuation problems.
So it's a very common method that's applied to solutions of nonlinear equations.
Then we moved on to optimization.
So we need this for design.
We use this for model fitting.
And we discussed both unconstrained and constrained optimization.
In the unconstrained context, we derived optimality conditions.
How do we know that we found the solution to this problem, at least a locally optimal solution?
And then we discuss steepest descent methods that could get us to that solution with linear conversions.
We discussed Newton-Raphson methods that can get us to that solution with quadratic convergence.
We discussed dogleg methods that are heuristic by nature, but meant to mix the properties of steepest descent and Newton-Raphson to give us reliable convergence no matter how far we are from the solution.
We moved on to constrained optimization.
Those problems came in two types, right, equality constraints and inequality constraints.
And we discussed the method of Lagrange multipliers to handle equality constraints.
We discussed interior point methods to handle inequality constraints and their mixture to handle a combination of equality and inequality constraints.
We left out a lot.
Optimization is a huge field.
So there are special classes of problems called linear programs, which are solved in a particular fashion.
These are problems that have a linear dependence on the design variables.
There's problems called integer programs where the design variables can only take on integer values.
These might be very important.
You can imagine trying to design a plant and asking what integer number of heat exchangers or separators is optimal.
Associated with constrained optimization and inequality constraints, there is an equivalent set of optimality conditions that we didn't try to derive.
Their called the KKT conditions.
So this comes up a lot in literature.
So it's sort of the fundamentals of knowing that you found the optimum to the inequality constrained problem.
They're not difficult to derive.
But we have a limited amount of time in the class.
It's something fundamental that's sort of missing.
And then these sorts of methods of all find local optima.
But oftentimes, we're interested in global optimization instead.
And that's not something that we discussed at all.
That's quite complicated and beyond the scope of the course.
But things like genetic algorithms, that's a term that will come up a lot.
That's one way of trying to seek out global optima in a landscape.
ODE-IVP, right, so now we want to move from static engineering situations to dynamics.
And we were really concerned, when modeling dynamics, with both the accuracy and the stability of these methods.
We were concerned about stability with other algorithms, too, for solutions of nonlinear equations.
We were concerned with whether the Newton-Raphson method would converge or not.
That's a stability issue in a sense.
OK, so we're concerned with stability there.
But here we really focused on it in detail, in part because some of these differential equations are inherently unstable.
In engineering circumstances, that happens.
And we would be unwilling to accept unstable solutions in situations where the equations themselves are stable.
In the case of Newton-Raphson, we can do these sorts of quasi-Newton-Raphson methods to change the path the solution follows.
And we didn't really care what path it followed.
But here, the path is important.
We're not free to change the path.
So the method itself has to be inherently stable.
So we discussed numerical integration.
We discussed two classes of method, explicit methods and implicit methods.
So explicit methods, we mentioned these explicit Runge-Kutta formulas that have been derived for all sorts of accuracy levels, both local truncation error and global truncation error.
Explicit methods, unfortunately you can't guarantee that they're always stable.
They lack a property called A stability.
Implicit methods, on the other hand, can be made to be stable under all circumstances.
So one of those is the backward Euler method.
That's a nice one to use.
The penalty for using these implicit methods is you likely have to solve a nonlinear equation at each time step.
So there is a cost to pay associated with enhancing stability.
We discussed local and global truncation error.
And we talked about stiffness and linear stability criteria.
So linearizing the ordinary differential equation, assessing what the eigenvalues associated with the Jacobian of that linearization is.
And using those eigenvalues to tell us, for example, what kind of time steps we need with our method in order to achieve stable integration in time.
Things we didn't cover that are sort of interesting and maybe important in engineering context, one of those is event location.
You may want to know the time at which something happens.
So I may want to know when does the solution hit a certain point in time.
Maybe, I even want to change my dynamics when the solution hits a particular point in time.
There's a broader class of methods that can do that.
They're built into MATLAB too, actually.
You can set up these events and have the dynamics change or have a report that an event occurred.
But we didn't really discuss that at all.
And in numerical computing, parallelization is important for studying classes of big problems.
It's sort of funny, but you can even do parallelization in time.
So this is something that's been discussed for a long time.
And for certain classes of problems, it offers computational advantages over serial integration of the dynamics an ODE-IVP.
So you can imagine how do you parallelize in time.
So you can take the time window that you're interested in and cut it up into different slices.
And you could try to solve in parallel the dynamics over each of those slices.
But for each slice, you need an initial condition that has to be precisely prescribed.
And you don't know those initial conditions.
That's what the serial integration would tell you.
One way of doing parallel in time is to say, well, those initial conditions are unknown.
And they match up to the terminal conditions of the previous slice.
So why not wrap this serial integration process in a Newton-Raphson solver to determine each of these points?
With certain classes of problems, certain ODE-IVPs that are inherently stable, you can do that.
And you can get a computational speedup over serial integration just by doing the integration for each of the slices on different computers and bringing all of the results together.
The bigger the farm of computers you have, the more slices you can utilize.
That's sort of interesting, right?
Parallel in time, it's bizarre, because we usually think of time integration as being sort of causal.
I have to know where I was to predict where I'm going.
But you can actually divide these things up in more sophisticated ways.
We talked about BVPs and PDEs.
We treated them as the same class of problem broadly.
Usually, when chemical engineers are looking at these problems, we're interested in spatial variation of some sort of a conserved quantity, so momentum or energy or concentration of a species or mass.
And here we are also concerned with both the accuracy and stability of method stability now for when we had also time integration of the solution, in addition to spatial variation.
And so for BVPs, we talked about shooting methods.
So can I recast a boundary value problem in one dimension as an initial value problem with an unknown?
That's a very common way of solving these problems.
And they are sometimes stable and sometimes not.
When they're unstable, you can do what's called multiple shooting, where you divide the domain up into subdomains, and you shoot from the initial condition of each of those subdomains.
It looks exactly like the parallel in time integration that I described for you before.
And then you try to match up the initial conditions with the terminal conditions to get a continuous solution.
Then we discussed relaxation methods for solution of BVPs and PDEs.
So collocation, Galerkin.
Collocation, you'll recall, was we want to try to satisfy the governing equation at various points in the solution domain.
So we might write our solution as the superposition of a set of functions.
And then try to satisfy at various points that we've laid out in the domains.
Maybe, there's even an optimal place to put all these points in order to minimize the error in the solution.
Galerkin, instead, we tried to make the projection of the error in our equations on different orthogonal functions that represented the solution be zero.
So that was sort of a global error that we tried to minimize over different orthogonal functions.
Whereas collocation is sort of a local error that we're trying to minimize, a local truncation error that we're trying to minimize.
We discussed finite difference and finite volume methods as well.
So ways of discretizing in the equations.
Finite difference, I would say, is the easiest one to reach for.
If you had to pick a quick and dirty method to try to get a solution, finite difference can be really easy to go to.
It's easy to figure out how big the errors are in your finite difference method.
But it may not be the optimal approach to the problem.
For certain geometries, finite volume is really far more preferable, because it's easy to estimate the fluxes through surfaces of funny shapes where your grid shape now conforms to your geometry.
Finite volume also had the advantage of being strictly conservative.
So I can never gain or lose a conserved quantity in the finite volume method, at least to within the numerical error.
And that isn't true of finite difference or finite element.
So if I'm concerned about those things, finite volume is really the approach to use.
And you'll find one fluid mechanical solver that's freely available.
You can go download and use it today, and it is widely used in research, is OpenFOAM.
And that's all based on the finite volume method.
So they're trying to conserve momentum, conserve mass, and conserve energy associated with the fluid.
OpenFOAM is good at drawing bizarre grids of the domain and giving you very accurate integration of fluid mechanical equations.
And it's good at doing that because it uses finite volume.
You discussed the method of lines, which was a way of discretizing space, but leaving the time dimension undiscretized and then applying ODE-IVP methods for solving in time.
It's an incredibly useful way to approach those problems.
Because it leverages expertise in two different techniques.
There are specialized methods for doing time integration with some spatial discretization that are vetted for their stability properties and their accuracy.
And those are fine, but I would say they're a little antiquated.
They're reliable for particular classes of problems.
But for general problems, you may not have any idea whether they're going to work or not.
Whereas method of lines may be a really good approach to a parabolic partial differential equation you're working with where you can just rely on the adaptive time stepping and the air control and an ODE-IVP solver to keep everything stable in time as you integrate forward.
We did application of commercial software.
So you used COMSOL to solve a problem.
I don't know.
Would you say was COMSOL easier to use than writing your own MATLAB code?
Harder to use than writing your own MATLAB code, do you think?
No
Easier
Easier to use.
But the result, I would say, wasn't as good without some careful refinement of the COMSOL solution.
So for that particular problem, if you wanted a solution that looked like your MATLAB solution that was carefully resolved in space, you really had to go in and refine the grid in particular places.
So it gets harder and harder the more detail you want.
But it's an easy way to get an answer.
You should check whether that answer is right by comparing it with other methods.
You can't guarantee that it's giving you the right result either.
To give you an example, I had mentioned an unsteady advection diffusion problem to one of my grad students.
And he's very good with COMSOL.
I mentioned that it was a hard problem because there were boundary layers.
And the boundary layers changed thickness and time, and resolving those things can be quite challenging without any physical insight.
And he says, well, I think I can just do it in COMSOL.
And he tried it, and COMSOL was happy to report a solution in time and space, but the solution was nonsensical.
We could visualize it and see that it didn't look like what other numerical solutions look like, and in certain limits, didn't correspond to what those solutions are known to be in those limits.
So the black box is great in that it reports an answer.
But it can also be really problematic as well.
So maybe, you have expertise with COMSOL.
You can get a solution to converge for a difficult problem.
But you should really have other methodologies that let you assess the quality of that solution.
For simple problems, simple elliptic problems and simple parabolic problems, it's going to do great, because it's built to eat those for breakfast.
But for complicated engineering problems, you've got to vet your solutions.
Oh, what didn't we cover?
We didn't talk about hyperbolic equations.
Not really.
We talked a little bit about advection equations, which have a hyperbolic character associated with them.
So you have like a pulse of [?
solute, ?]
and it's advected along in time.
And you want that advection to be stable, so you want to use things like upwind differencing to ensure that stability.
But there's a broader class of hyperbolic problems that relate to things like the motion of waves in elastic solids.
We don't typically encounter those in chemical engineering, but chemical engineering is a broad discipline.
And maybe you're engineering soft materials for some sort of device.
And you want to look at the elastic response of that material.
The PDEs that govern that are a completely different class than the ones that we typically look at.
They're not parabolic.
They're not elliptic.
They're hyperbolic, and they are these wave-like solutions associated with them.
They require their own particular solution methods in order to be stable and accurate.
And then there are all sorts of specialized methods that are dreamed up to apply to, maybe, a broad class of problems, but it's kind of hard to ensure that they're going to work.
But nonetheless, you'll see a lot of this out in the literature.
So there are things called multi-grid methods.
They try to discretize a partial differential equation with different levels of fineness.
And you use course solutions as initial guesses for finer and finer solutions in space.
For some problems, this works great.
And in fact, you can show that it can work as well as is possible.
So depending on the size of the problem, say you have n points at which you want to find the solution.
You can find it in order n time with some of these methods depending on the particular problem you're looking at.
That's good.
But oftentimes, engineering problems aren't of that sort.
There's something more complicated going on.
And it's hard to ensure that you get those sorts of rates of convergence.
But nonetheless, they exist and something that we didn't cover you should be aware of.
We did DAEs.
So now, we coupled ODE-IVPs with nonlinear algebraic equations.
Most dynamic engineering problems are of this type.
We showed that DAEs are really an issue with model formulation ultimately.
We write down conservation equations, and we write down certain constraints on those conservation equations.
The constraints can be specifications like we want particular flow rates in certain places.
Or they can be fundamental.
They can be thermodynamic constraints on the coexistence of different phases of a particular material in a particular unit operation.
If we could solve those constraints for certain unknowns and substitute them in the equations, we might be able to produce ODE-IVPs.
Probably we can't come up with those solutions.
So instead, we've always got this coupled system of initial value problems with nonlinear equations.
So you have that sort of a circumstance.
How should you think about that?
One way to think about it was to say that the algebraic equations are essentially infinitely stiff.
They have to be imposed exactly at every time step.
That they never relax in time with the finite time constant.
And so we knew that those sorts of methods are hard to resolve with typical ODE solvers.
And we tried to assess that by computing something called the differential index.
So we asked how many times do I have to take derivatives of the algebraic equations or any new algebraic constraints in order to get a system of ordinary differential equations to represent the same model.
I may want to solve that system of ordinary differential equations.
Or I may just want to know the differential index so that I can choose a particular method to solve this problem.
We saw the higher the index was, the more sensitive the solution was to perturbations in different input variables.
So the differential index played an important role in telling us something physical about the problem and the responses that could be elicited by DAE systems.
We also talked about consistent initialization.
I need to prescribe the initial conditions for all the differential and algebraic variables.
Can I prescribe any of those independently?
Well, the answer is actually no.
Depending on the differential index, there are underlying algebraic constraints that have to be satisfied at all points in time.
And if they're not, then that can break the method that's trying to integrate the DAE.
So you have to prescribe the initial conditions consistently.
If they're not prescribed consistently, and you go to a piece of commercial software, it may give an error.
It may tell you nothing.
You may get a reasonable result or an unreasonable result.
It really depends on the methodology.
So it's up to you to know in advance that this is an important aspect of the problem.
We solved index-1 DAEs typically.
Index-2 DAEs can be converted into index-1 DAEs and solved pretty easily.
Index-3 and bigger DAEs are difficult because of their high sensitivities.
So there's special software that's out there.
I mentioned some of it during our DAE lecture.
And that's really what you want to reach for if you have a high index DAE.
Examples, we were able to craft all sorts of examples where we tried to do funny things with causality, where you tried to change the input by affecting the output, which doesn't make a lot of sense.
But that led to very high index DAEs.
Mechanical systems often have very high index DAEs associated with the mechanical constraints, typically lead to index-3 DAEs.
So these pop up in places that are important.
And being able to look at the model and assess the index of it is kind of essential.
We did probability.
So the physical world has inherent uncertainty.
It's not just uncertainty in our ability to measure things, but fundamental, physical uncertainty is built into the world around us.
So measurements have uncontrolled or even uncontrollable sensitivities to this uncertainty.
And we wanted to know how to handle it.
So we talked about things like sources of randomness.
We did fundamentals.
Maybe this overlapped with 10.40.
But I think overlap is good, because humans tend to have bad intuition about probability in general.
So we discuss probability densities, means, covariances, expected values, joint probabilities, conditional probabilities.
We talked about common probability densities.
You covered the central limit theorem.
You talked about the difference between sample averages and the population mean.
One thing that's important that I don't think that gets covered much in detail, but if you're going to be working with randomness computationally, oftentimes you have to generate random numbers.
And you'd like to generate good random numbers.
Turns out a computer doesn't generate proper random numbers, only pseudo random numbers.
And these generators have various properties associated with them.
And some of them are good.
And some of them are bad.
And you want to use good ones.
So if you have to generate random numbers for your research, you'd like to know that they are high quality, that the sequence of numbers that's being generated doesn't repeat over the time that you're using it, that it won't overlap over many different uses of this methodology.
So high quality random numbers are important.
We discussed models versus data.
So we wanted to regress, estimate parameters from the data.
We wanted to do things like hypothesis testing.
I have a model.
Is the model consistent with the data, or is it not consistent with the data?
So you covered things like linear and nonlinear least squares, maximum likelihood estimates, confidence intervals found using the chi-square distribution.
We did Bayes' theorem and Bayesian parameter estimation as well, so a way of using prior information about parameter values to better estimate the probability that those parameters will take on values given the data.
We didn't cover design of experiments.
That's a 10.551 topic.
But there are specific ways that one can design the experiment.
Which data values do I take in order to make the fitting of parameters optimal?
And there are other problems besides regression that are important.
But we don't typically utilize those in chemical engineering.
So one of those is classification.
So if I have a series of points, and I do regression, I'm in some sense asking for like what's the best curve due to this model that can be drawn through those points, right?
What parameter values give me the curve that's closest to all these points, that's least squares.
Classification might ask instead, what's the curve that divides these points most evenly, or sits furthest from all these points in an equal amount instead.
Which side of this line do the points sit on?
Are they of type A or type B?
It's kind of a related problem to regression.
But it's a different sort of problem.
The same sorts of methods can be applied to classification that you apply to doing regression and parameter estimation.
It's an important problem.
And then we finished up with stochastic simulation.
So you learned that sampling of random processes can be used for engineering calculations.
And they're even inherently random physical processes that you might want to model reliably.
So we did Metropolis Monte Carlo.
This was a way of basically computing high dimensional integrals.
That's what it boiled down to, where we wanted to sample from different places in the domain, weighted by some probability density function.
You did kinetic Monte Carlo, which is really a sort of event-driven algorithm.
You are expecting that certain events are going to occur with certain rates.
They're actually nonstochastic versions of that, the same sort of event-driven algorithm.
I know certain things should happen at certain points in time.
And rather than integrating the dynamics of the system all along these points in time, I just advance the dynamics until the next event occurs.
That event occurs.
I keep a list of what the next event is going to be.
And I follow those events along.
So it could be something like billiard balls on a table.
I know the trajectories of the billiard balls, given their position and their momenta.
And all I need to do is keep track of when they collide and where they're going to go next and what the next events are going to be.
Well, kinetic Monte Carlo was the same thing, except the events are now stochastic.
They occur with prescribed rates.
And so I just need the sample from this distribution of rates to get the right sequence of events.
There's funny things about kinetic Monte Carlo.
So applied to chemical kinetics, you can estimate these rates reasonably well.
You've got to know the rates for this process to look right.
If you don't have the rates correct, or you don't have the relative magnitudes of the rates, then the simulation is kind of meaningless.
The result is going to be nonsense or garbage.
And so there's going to be limitations on your ability to do this based upon how accurately you can estimate these rates.
It also means that if you know that there's disparities in the rates, say one rate is very high and one rate is very low, you can make certain assumptions about the process.
So things that happen very quickly you may treat as though there are equilibrated and not try to resolve them at all and look more at infrequent processes that occur, slower rates.
So the precision with which you have to know these things is important.
The relative rates are what's really important in these problems.
And you want to get them right in order to resolve these physical processes correctly.
Then we discussed molecular dynamics after that, which is another type of stochastic simulation methodology.
Here you just integrate the equations of motion associated with whatever particles you're interested in.
Kinetic Monte Carlo and molecular dynamics are sort of fundamentally far from equilibrium, but may relax towards equilibrium, depending on what constraints you put on the particular rate processes that you're modeling.
Metropolis Monte Carlo is usually sampling an underlying-- if we're applying it in the statistical mechanical sense, it's usually sampling an underlying equilibrium Boltzmann distribution.
So they often get used in fundamentally different ways as well.
We didn't cover stochastic differential equations.
So depending on which degrees of freedom you want to represent in a model like molecular dynamics, there may be some degrees of freedom you simply don't care about.
But they have an influence on the problem.
So in my research, for example, we model nanoparticles and particulates.
We're not very interested in the details of the solvents surrounding these particles.
There's a huge disparity in length scales associated with nanoparticles and the molecular dimensions of the solvent that surrounds them.
The solvent has an influence nonetheless.
It's fluctuating all the time.
Solvent molecules are colliding with these particles, and they're imparting momentum.
One way of resolving a system like that without having to look explicitly at the solvent is to try to integrate out the solvent degrees of freedom and treat those collisions of the solvent with this particle in a stochastic sense.
So at various points in time, there's momentum that's randomly transferred to particles dispersed in a solvent.
The physical manifestation of that is Brownian motion.
When you look at small particles in the fluid, you see them diffuse around, and that's where that's coming from.
So you can simulate processes like that without having to resolve molecular scales.
So as you go up in scale, you integrate out degrees of freedom, but oftentimes, that introduces inherent stochasticity in the underlying dynamics of whatever you're trying to model.
That's very interesting.
And then for Monte Carlo methods, there's advanced sampling methods.
And we didn't discuss any of these in detail.
But you saw some of this with the model you saw with the problem you approached in your homework where you had two populations or two peaks in the probability distribution that were widely separated.
And it may be hard for a Monte Carlo method to traverse between those peaks.
And umbrella sampling as a way of biasing the probability distribution to make those paths from one peak to another more probable.
So it's just a way of seeking out rare events like the transitions between these two different populations.
And it's pretty important, ultimately, for exploring complex high dimensional spaces.
But you have to understand something about the underlying physics in order to figure out the biases needed to enhance the sampling.
So it's so quite complicated.
So I'll remind you.
Your final exam, right, it's Monday.
It's in 56-154 at 9:00 AM.
Don't show up to Walker.
It's not going to be there.
It's three hours.
It's going to be comprehensive.
You can expect roughly one problem each thematic topic.
We've written it now.
I think it's nine problems long.
So we're in the process of revising it for difficulty and time.
It's going to be short answer format.
We sort of aim at 10 to 15 minutes per problem.
I think that's the sort of frame you should be looking at.
The problems are structured in such a way that that's about how long they should take.
You should use that to guide your temporal budget on the exam.
If it's taking too long, switch to something else.
Don't let yourself get hung up focusing on one thing.
Move around and come back to it later.
There will be some fundamentals.
There's going to be some translation of engineering into numerics.
And there's going to be some MATLAB.
So it'll look like a blend of the first exam and the second exam.
Yeah
When you say it's on MATLAB, do you mean [INAUDIBLE] MATLAB?
Yeah
[INAUDIBLE]
Yeah
No computers
No computers.
It's just write a little script.
We've asked problems like that before.
It'll be of the same type.
Yes
You said nine problems.
[INAUDIBLE]
I think there's nine, yeah.
Well, that's how many we've written now.
I'm not planning on writing anymore
Plus or minus one
Plus or minus one, yeah
[INAUDIBLE]
Oh sure.
Listen, if we gave you nine problems of the same length as on the first two quizzes, you wouldn't get through the problem.
So they're not written to be so long.
But there are multiple parts.
But multiple parts are usually intended to guide you to the solution we're looking for.
Usually, we're testing for a particular understanding.
It also helps you out when there are multiple parts.
So there's lots of partial credit available.
You don't have to get the whole thing.
You've just got to get two out of the three parts.
You did pretty good.
OK, more questions?
So when you say short answer, does that just mean that similar types of problems as what we had on the first two quizzes.
They're just shorter
Yes, that's right.
Yeah
[INAUDIBLE]
Yeah, they're not multiple choice, true or false.
They're short answer.
You want to hit a level of detail that shows us you understand what you're talking about, right.
We don't need pages of information on these things.
But we need to have enough information that we can see you know what you're doing.
We're trying to assess your understanding of the material.
That's all.
More questions?
No more questions, OK. Well, good.
It's been a real pleasure teaching you guys.
I'm really pleased with the outcome of this course.
This is my third time teaching it.
I think this is the strongest group that I've seen come through 10.34.
So you guys should be really proud of yourselves.
Good, all right, good luck.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
When it comes to public housing, current concept of mixed income housing really appear very late in the game even though public housing proponents were very focused on incomes right from the start.
Public housing was not always for the poorest and has had a complex relationship with ideas about neighborhoods and neighborhood renewal.
It's really too simplistic to think of US public housing as a single failed experiment and as something that's now been replaced by mixed income housing.
I like to think of it as really a three faced social experiment.
There's a first phase that lasted I think from about 1935, when the program started, up until 1960 or so.
And it was not about finding the poorest people from the worst slums and re-housing them.
It was more about finding the worst slums and tearing them down but not really re-housing large numbers of the displaced.
Rather, it was a chance to seek out large numbers of the barely poor and to build public housing communities selectively from those people.
We don't call them low income because they probably had incomes between 50% and 80% of the area of median, but they're not very low income and they're not extremely low income the way that we would talk about it today.
But by 1960 or so, and lasting until 1990 at least, public housing really took a different attitude towards those who ought to be housed.
By the mid and late 1960s, public housing was predominantly serving a very low income constituency.
And by the 1980s, an extremely low income constituency, meaning that it went from 50% or 60% of the area median into between 30% and 50% and then all the way down to about 15% or 20% of the median income by the 1990s or so.
And so that's the phase of welfare housing that we think about as a second phase.
And since 1990 or so, there's really been a third face, a series of initiatives to return more of public housing to that first phase of selectivity to really try and find again the people who have less extremely low incomes to give work preferences and things like that and to engage in mixed income housing.
And that's really had a lot of demolition, new communities that are built with different income structures, and rule structures.
The images that you see show three phases using Chicago's Cabrini Green as an example.
I'm not going to talk much here about Cabrini, but when we get to class, it'll be one of the key case studies, especially since I've recently completed a book that features this place.
But you can see it's a slum.
The blueprint's to build high rise public housing that you see in the middle.
And more recently, the demolition that that has occurred.
A lot of people have had different responses to public housing.
And I think it's worth looking at three of them.
First is that journalists and scholars did have some initial enthusiasm for the program and then a lot of controversy, leading to a second phase that one might call the design in decline and fall literature, where everybody seemed to be telling a story about failures of one kind or another, whether it's a failure of design or management or something else.
But since the late '90s, 2000 or so, there's been a series of revisionist efforts, to really rethink the complexity of public housing and the values of what has taken place.
And not see everything as a complete disaster.
I tend to go really far back in thinking about it.
And you don't have to do it entirely to the 1600s to make sense of this.
But I think it's important to talk about ideology and institutional origins of public housing and not treat this as simply something that emerges in the 1930s out of nowhere.
You may not want to go all the way back to the 1630s as I tried to do in my book.
And you certainly don't need to do that if you come from places that are very distant for many Puritans, but I see a long continuity in housing over questions of really moral judgment.
On the one hand, there's a reward tradition, a sense that the government can help low income people in ways that reward them for certain kinds of behaviors.
So in the upper right, you can see an African American family taking advantage of the Homestead Act in the mid 19th century.
The system that enabled you to get entire plots of land if you would promise to work and live on it and then it would become yours after a certain number of years.
So there's a tradition that goes with pensions for veterans and other kinds of reward that tie housing to good behavior and particularly good working behavior.
And then there's another tradition that I might call the coping tradition.
If you look at the bottom right, there's a picture of one of the [INAUDIBLE] houses in Boston.
This one's from about 1800, designed by Charles [INAUDIBLE], the same guy that did the State House just a few years earlier.
And it had a wing for men and a wing for women and a chapel in between.
And this was the place where people went if they really couldn't afford to live on the town.
They didn't have friends or family that could support them.
So the state built a grand edifice and built a lot of them.
So you have on the one hand, certain housing that is given as a reward for certain behavior and then another tradition that's really about coping with a set of people that are not behaving as needed.
By the end of the 19th century, you get one of my favorite books.
This is called Civilizations Inferno-- Studies in the Social Cellar by a man named Benjamin Orange Flower.
It's the equivalent of the Jacob Riis book about New York, How the Other Half Lives.
But this one's about Boston.
And if you look at it, it's a kind for the society, the frontispiece piece of the book here.
It shows the happy, wealthy people in the top dancing in their town home while increasingly levels dire circumstances of poverty lurking below them.
The immediate next lower level are the well intentioned men that are out of work.
The Depression of 1893 was happening.
And they were people for whom no fault of their own, they were out of work, in trouble, and poor for that reason.
And then there's a second deserving poor.
The widows and orphans that you can see in the third lower level.
And those two were a piece of the poor of a city like Boston.
But then there was the social cellar.
They the undeserving poor.
The criminals, the misbehaving masses tht couldn't quite be trusted.
And this is the kind of thing that made it very complicated to decide what is the role of the government when it comes to housing.
So when you get housing authorities, and here's a picture of the men who formed the five person Boston Housing Authority, meaning the board of the authority in the 1940s, you had a sense of judging poverty in a different sense.
They wanted to make sure that the people who came into public housing were from that second and third tier and not from the fourth tier.
And even better, they wanted to find people that were ready to move up into the top.
That were not going to be employed very long and were not just deserving poor, but actually ready to move onward into the middle class and wouldn't need public housing very long.
So you can see if you look at a variety of graphic imagery from all over the country, the different housing reports and other city agency reports were setting a contrast between public housing and slums that was really very important.
The 1930s and '40s were a time when people focused on sub clearance and employment in the building trades.
And so when you got to the Housing Act of 1937, it linked all new low rent housing to slum clearance.
It mandated an equivalent elimination agreement.
In other words, public housing construction had to be accompanied by what they called elimination by demolition or condemnation or effective closing or the compulsory repair and improvement of unsafe or insanitary dwelling situated in the locality or metropolitan area.
And that that was going to be substantially equal the number to the number of newly constructed dwellings provided by the project.
So in other words, public housing was intended not to compete with anything in the private sector because it wasn't going to make any gains in the low rent housing stock.
Even if elimination was achieved by rehabilitation rather than demolition, the improved properties were probably going to cost a lot more and demand substantially higher monthly rentals.
So what this did was to require or at least encourage public housing construction in inner city neighborhoods rather than the more affluent peripheral areas.
So it was almost a form of neighborhood renewal in the form of replacement housing even though many other neighborhoods were cleared for other purposes that included more private purposes rather than public ones.
So the Newark example, the green one near towards the bottom right, is a pretty good one.
It's a booklet that shows the changes between people in the slum in the upper left side of it and the public housing in the bottom right side.
It's almost a environmental determinism from badly behaved people that need the police and sit on stoops and don't do the things they're supposed to do to the Boy Scouts and the happy woman at her sink and the people playing in their yard and the sunshine and the lighter colors that predominate.
It's almost implying that the new appliances are leading to new behavior.
But it's really in practice more changing of people rather than changing in them.
At least it's worth thinking about.
It may be just a different set of people.
I was struck by that constantly when I looked at this.
So slum clearance had multiple forms of justification, mostly having to do with issues of crime and health.
Chicago, in this brochure, depicted itself, meaning the housing authority, depicted itself as having what they called a dirty backyard.
And it is a very revealing evidence for how one thinks about the relationship between the haves and the have nots and the sense that the dirty backyard had to be cleaned up for the benefit of those that were valuing it.
The goal was to deal with what were called streets of dreariness and the dangerous people that lived in them.
It's a sense that these were a menace to themselves.
So they did charts like once that compared the prevalence of fires and tuberculosis and violence and juvenile delinquency between slum areas and non slum areas.
And there's a picture on the left of a family that has a caption and another place that makes clear that this is supposed to be telling you about the evils of broken families in the slums because the husband has run off and abandoned these women and girls in that place.
So these attitudes towards the slums of people and places to get rid of were implied largely to do with things like disease.
The health risks of bad housing were promoted constantly in all the brochures and annual reports and report-- and other documents that came out during the '30s and '40s.
The health risks were paramount.
But there was also a set of rewards that were still part of the tradition.
The public housing had really inherited from both sides of that reward and the coping mechanism.
For the most part, in the '40s, it was still part of the reward tradition.
It was used to reward returning veterans, like you can see in the upper left in Los Angeles.
It's an effort to, as the middle one says, make juvenile gangsters take a back seat.
And it's still that 19th century sense of public housing as a reward for good behavior and an accompanying mistrust for many of the poor.
It really is constant across the country.
Here's one of my favorite examples.
In Boston, the Boston Housing Authority used dramatic photographs in their annual report.
This is from the 1940s or so.
And these binary juxtapositions.
Out of the shadows, into the sun.
It says, if you can't quite read it, after they built the first eight of these projects, the housing authority wanted to take stock in the 1940s.
And they said it beheld eight, clean shining developments rising fresh to the sun where once, in dreary dirt filled dilapidation, slum dwellings had shambled in contaminating hopelessness against a gray and somber sky.
So even the weather was going to get better from public housing.
But something's really missing here, or at least it's very misleading.
The implication of the text and the image is that the children from the shadowy alleys are being rescued by transporting them into the sunshine and the openness of public housing.
But I guarantee you none of those kids in the left picture ended up being the happy kids in the sunshine in the right.
Or at least it's very unlikely.
When I looked at the records, I saw that of the first four public housing developments in Boston, between 50% and 80% of the people actually sought entry into them that were the ones completed under the Housing Act of 1937.
But when I actually examined the lists of tenants in these places, I found that only between 2% and 12% of the projects had actually gained a place-- the people gained a place in the project.
And this is not atypical.
I've seen statistics for New York that showed that all the way up to 1957, only about 18% of the former residents of public housing sites were re housed in the new public housing.
If the patterns were consistent around the country as I think they may have been, it's more evidence then that the goals of public housing were not really centered on serving the people who were displaced to create it.
There was just intense scrutiny about who was wanted in public housing.
You had to have a head of the household that was a US citizen, it was segregated by race, it had family sizes that had to be between two and nine persons so you couldn't be a single person.
You couldn't have a large family or an extended family.
You certainly couldn't be a gay or lesbian couple.
There were no rooming houses available, no living with friends, nobody without a stable employment to pay the rent.
It was a very selective effort to find a public citizen.
And I like this image in the upper left of the young child and his picket fence in the new Norfolk, Virginia public housing called a citizen of Norfolk's Merrimack Park.
They were very pleased with the people that they got, whether it was toddlers or teens.
In the upper right, the caption says boy meets girl in a Brentwood Park home.
All three look nice.
That's the thing that was going on.
But if you look at the bottom chart also from Norfolk, it's an analysis of who really is intended to receive public housing.
They said they've got 47,000 dwellings and only 17,000 of them are substandard.
But of those 17,000 that are substandard, which would seem to be the good place to look for public housing residence, they found that 4,000 earn too much for public housing, meaning they really weren't all that poor.
But 7,000 earned too little.
In other words, only 6,000 of 17,000 were the target audience.
What does it really mean to earn too little for public housing?
It means that they were targeting not the lowest of the incomes, but a near poor group of people.
And not worrying so much about the people at the bottom.
They were more interested in seeing public housing as a social progression.
In many cases, all the way from a slum by way of public housing up to home ownership.
This thing appears repeatedly, whether in text or in graphics.
The upwardly mobile working class were intended.
Built in a mixed income community, if you want to call it that.
It's just that nobody was very rich and nobody was very poor.
The assumptions were there about upward mobility towards home ownership.
Race is undergirding all of this, of course.
This is a Miami example of the situation prevailing in what they inelegantly called the central negro district, not the central business district, but you can see that off to just the right of that.
But actually an area that was seen as a district defined by race.
And really vilified by race.
If you look at the bottom right the section of the Coconut Grove negro area as they called it, the blocks are classified by what was called the block median penalty score, with red and black being-- red and brown being the worst cases.
And so you could see a lot of it was there.
And that had to do with lending and the things that were going on and a way of self-fulfilling a prophecy of disinvestment in the place.
So this kind of a situation where white and race were tied into it were really important.
And there certainly were efforts going on into the '40s and '50s to cope with white racism and deal with the extent to which public housing still needed to be seen as entering into a segregated city.
The right side is the Baltimore urban league's effort to try and downplay the fears of a changing neighborhood.
But those things were very much a part of it.
And so public housing gets built.
And after a short while where many people were very positive about it, you enter into decline and fall literature, where a lot of the scholarship from the 1970s and '80s was just relentlessly negative about public housing.
Lee Rainwater's book Behind Ghetto Walls from 1970 about Pruitt-Igoe, written just before the demolition of it.
Or the book by Arnold Hirsch, Making the Second Ghetto in the 1980s.
And then by the 1990s, Alex Kotlowitz's book, There Are No Children Here, about the impossibility of childhood in the Henry Horner Homes of Chicago.
Those things were happening.
But what I find really interesting is that there have been revisionist lenses that really do matter.
Since about 2000, new ways of looking.
Thinking about what success has been or could be, taking the role of tenants much more seriously.
And also really thinking about the role of design.
So if we say a little bit about each of those.
In 2008 and 2009, I was fascinated to see two books with almost diametrically opposed titles come out.
Brad Hunt wrote about Chicago with the title Blueprint for Disaster-- The Unraveling of Chicago Public Housing.
At almost the same time, Nick Bloom is writing a book called Public Housing That Worked-- New York in the 20th Century.
Now, one could think it's just the time period.
The middle volume here, When Public Housing Was Paradise is a book about Chicago in its first 20 years of public housing.
But Bloom's book says public housing worked in the 20th century, meaning all the way up until pretty recently.
Since 2000, I think there's been a lot more publicity for New York and [INAUDIBLE] problems.
And it suggests that some places, including the city that has the most public housing, is very far from the worst case situation.
But Chicago was probably the worst for the longest.
Though several other cities had lawsuits and receiverships and HUD takeovers and all sorts of things like that.
In Boston, for instance, there was a receiver, really the first of this who took over in 1980, Harry Spence.
And the court put him in charge and told him to try and fix a very broken system.
The Boston Globe magazine cover here says that, I think, in some really interesting ways.
Think about what you're seeing here.
What is this suggesting about how positive change comes about?
Can this man save public housing?
Is that the right question?
What is it actually asking?
I want to return to this image when we get to class and talk about it and try to deconstruct it a little bit more.
So think about it.
Think what you're seeing here.
I think that picture and that sentence is encoding the whole history of American housing very neatly in a single composite image.
So think about what you're seeing here.
It's not the only picture like, that strangely.
A decade later, another housing authority in crisis, another newspaper, Sunday Magazine, is asking a similar question.
Can this man save the CHA, the Chicago Housing Authority.
This time, the man is the head of the CHA, Vincent Lane, a former developer.
And at least he gets to pose on top of the buildings, it's Cabrini Green.
I'm not sure that's an advantage, though.
Is he dwarfed by the challenge or is he on top of his world?
You can make your own guess.
He didn't last very long despite making some very highly publicized interventions of the mixed income housing at Lake park place.
The anti-drug sweeps that brought him national and international attention.
And by 1995, the CHA was taken over by HUD from '95 until 1998.
And then they launched the famous Plan for Transformation in 2000, which we'll talk about a bit in class.
So there's a very mixed record of achievement from the successes to the failures to the ambitious plans to try and turn failure back into success.
But one of the things that I find so striking in recent years is that there's a latter day revaluation and revaluing the activism of the tenants, the residents.
It's really a change in the scholarship about public housing.
More of it has become tenant centered.
It's not just about public politicians and housing authority leaders.
But it's about the coping mechanisms of tenants who've had to endure terrible conditions.
It puts people like Thelma Smith, who's in the middle picture, a tenant leader at Boston's Franklin Field development, it puts them at the center of the picture and not shunted to the margins.
Rhonda Williams's book about Baltimore, The Politics of Public Housing, is one.
Roberta Feldman and Susan Stall's book, The Dignity of Resistance, about women tenant activists at Chicago's Wentworth Gardens.
Sudhir Venkatesh's complex view of the complexities of gang culture and politics and economics at Robert Taylor Homes in Chicago, American Project, a project that is now demolished and we'll talk about what's happened to that as well.
My own reclaiming public housing book about the 50 year struggles of three communities in Boston to try and get past dire conditions and return to desirable communities.
And I think a lot of people have started asking the question that's posed in the title of the book by Larry Bennett, Janet Smith, and Patricia Wright, Where Are Poor People To Live?
It's not so obvious as public housing itself has come to change.
The last of the real changes that I think is starting to take place is a more complex view about the role of design.
More of the thinking and the writing has emphasized the importance of design.
There was always a long history emphasizing the negative consequences and design was always a key part of that.
The fact that it was initially built to emphasize the distinctiveness of public housing, not to just be bare and austere and off putting, but to appear as a security and safe and modern alternative to the wooden fire traps of the aging slums.
You can see in the middle of the picture of the Orchard Park development built in 1942 in Boston, although the pictures from 2000 and then a picture just a few years later of Hope Six redevelopment of Orchard Gardens.
Pretty much the same spot in the place.
But there have been other aspects to the conversation.
When I see the design centered revisionism that's taking place has often taken, again, a environmental determinist stance.
There was a book that is little remembered now by [INAUDIBLE] and the better known Christopher Alexander, called Community and Privacy, that's 1965.
But it also had the attack on project centric mentalities that was much more wildly and widely influential by Jane Jacobs in 1961.
And then by 1972, there was Oscar Newman's work, Defensible Space-- Crime Prevention Through Urban Design that really was trying to suggest that it was possible to use design and urban design to shape behavior.
It was about crime reduction and showed early cases of how public housing projects could be retrofitted and redesigned to make them safer through residence by encouraging a territoriality.
And it was fascinating to me in the mid 1990s the way HUD embraced Oscar Newman's defensible space ideas and had him author an entire manual in the 1990s.
And soon after that, HUD embraced the new urbanist wing.
You can see a co-authored publication, Principles for Inner City Neighborhood Design at the left.
And particularly emphasizing the neo traditional development side of new urbanism and was really trying to rethink the whole overall image of public housing developments.
So rather than public housing being this deliberately different looking modernist alternative to decrepit and ill provisioned rows of wooden or brick townhouse that that seem to signify mid century blight and slum conditions, the new urbanist paradigm wanted to sanitize an update the reputation of the pre modern urban models and make future public housing look much more like private sector neighbors.
Not standing implacably, in part from the [INAUDIBLE], the colors in the Orchard Garden look pretty different than anything you'd find in that part of the neighborhood.
It still stands out, but it stands out in a different way.
The goal if you take defensible space and new urbanism together is a tableau of middle class Americana, that normalizes the appearance of public housing to a point where it really could be accepted again into the fabric of existing, market friendly neighborhoods.
And that's pretty necessary if you want to start thinking about mixed income.
But it raises a question.
Does Hope Six and HUD's program with its picket fences and pastel facades, it's a new look for public housing and it's buried within a larger concept of mixed income housing, it's managed by private firms rather than the public sector, but is it actually displacing the poorest again just like they were displaced from the slums to build public housing back in the '30s and '40s?
There's a book by Edward Goetz that's coming out in 2013 called New Deal Ruins-- Race, Economic Justice, and Public Housing Policy.
And I had a chance to look at an advance copy of it.
And it's an entire book about public housing demolition.
And it really strongly critiques the extent of displacement caused by Hope Six and suggests that displacement has been disproportionately born and has had a disproportionately negative impact on African Americans.
My own book, which is also due out in 2013, is called Purging the Poorest-- Public Housing and the Design Politics of Twice Cleared Communities.
And despite its title, it's probably a little more evenhanded than the Goetz book, but it's attempting to understand why the leadership of some cities, particularly Atlanta and Chicago, has been so intent on introducing mixed income communities to replace the public housing projects of the past.
So really, what I've wanted to do here is to frame some of the current challenges of mixed income housing.
I hope you're going to keep in mind some of these deeper cultural issues that I've been discussing here.
It's not just public housing about our attitudes towards the housing that is important.
It also matters what we mean by the public in public housing and which parts of that public are expected to benefit from housing redevelopment and what form those benefits ought to take.
And those are the things that I hope you'll think about in the days up to class and we'll get a chance to talk in person very soon.
Thanks.
I was an undergrad at MIT.
And I majored in both economics and management.
And while I was taking classes in these subjects, I also had the chance to tutor and teach the subjects for a couple of semesters over a couple of years.
And while I was tutoring and teaching the subject, what I noticed is that there were a number of problems that would always trip up students in certain and specific ways.
And so what I'm hoping to do in these problem-solving videos is I'm hoping to take some of the more difficult problems from the PSETS, and take you through the solutions, so that these problems that come up won't affect you.
And what we're going to do is I want you to not just take away a quantitative solution to the problem.
But more specifically, I want you to be able to take that quantitative solution, link it to a conceptual understanding through figures and diagrams, and ultimately build a greater intuition about economics principles in general.
I hope you enjoy the videos, and I look forward to working with you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So what I want to do today is I want to talk about what the heck this course is.
What is microeconomics?
What are you going to be learning in this course?
And just, sort of, set us up for the semester.
OK.
So basically, microeconomics is all about scarcity.
It's all about how individuals and firms make decisions given that we live in a world of scarcity.
Scarcity is key because basically what we're going to learn about this semester in various shapes and forms is a lot of different types of constrained optimization.
We're going to learn a lot about different ways that individuals make choices in a world of scarcity.
OK?
That is, this course is going to be about trade-offs.
Given scarce resources, how the individuals and firms trade off different alternatives to make themselves as well-off as possible.
That's why economics is called the dismal science.
OK?
It's called the dismal science because we are not about everyone have everything.
We're always the people who say, no, you can't have everything.
You have to make a trade-off.
OK?
You have to give up x to get y.
And that's why people don't like us.
OK?
Because that's why we're called the dismal science, because we're always pointing out the trade-offs that people face.
Now, some may call it dismal, but I call it fun.
And that may be because of my MIT training, as I said I was an undergraduate here.
In fact, MIT is the perfect place to teach microeconomics because this whole institute is about engineering solutions which are really ultimately about constrained optimization.
Indeed, what's the best example in the world we have of this?
It's the 270 contest.
Right?
You're given a pile of junk, you've got to build something that does something else.
That's an exercise in constrained optimization.
All engineering is really constrained optimization.
How do you take the resources you're given and do the best job building something.
And that's really what microeconomics is.
Just like 270 is not a dismal contest, microeconomics is not to me a dismal science.
You could think of this course like 270.
But instead of the building robots, we're running people's lives.
OK?
That's, kind of, the way I like to think about this course.
Instead of trying to decide how we can build something to move a ping pong ball across a table, we're trying to decide how people make their decisions to consume, and firms make their decisions to produce.
That's basically what's going to go on in this class.
OK?
And that's why basically modern microeconomics was founded at MIT in the 1950s by Paul Samuelson.
The father of modern economics was a professor here, and he basically founded the field.
He basically introduced mathematics to economics.
And through teaching this course, 14.01, 50, 60 years ago, actually developed the field that we now study.
Now, what we're going to do in this class, is focused on two types of actors in the economy: consumers and producers.
OK?
And we are going to build models of how consumers and producers behave.
Now, technically, a model is going to be a description of any relationship between two or more economic variables.
OK?
That's a model.
A description of any relationship between two or more economic variables.
The trick with economics, and the reason many of you will be frustrated during the semester, is that unlike the modern relationship between say energy and mass these models are never precise.
They are never accurate to the 10th decimal.
OK?
This is not a precise, scientific relationship with modeling.
We'll be making a number of simplifying assumptions that allow us to capture the main tendencies in the data.
That allow us to capture the main insights into how individuals make consumption decisions and how firms make production decisions.
But it's not going to be as clean and precise as the kind of proofs you're going to be doing in some of your other classes in freshman and sophomore year.
OK?
So basically, we have a trade off with the simplifying assumptions.
On the one hand, obviously we want a model that can explain reality as much as possible.
If a model can't explain reality, it's not useful.
On the other hand, we need a model that's tractable, a model that I can teach you in a lecture or less.
OK?
And basically, what we do is we make a lot of simplifying assumptions in this class to make those models work.
And yet, what we'll find is despite these assumptions, we'll come up with incredibly powerful predictions of how consumers and producers behave.
So with consumers what we're going to do is, we're going to say that consumers are constrained by their limited wealth or what we'll call their budget constraint.
And subject to that constraint they choose the set of goods that makes them as well off as possible.
OK?
That's what we're going to call utility maximization.
We'll say the consumers maximize their utility, consumers are going to maximize utility subject to a budget constraint.
That's going to be what we're going to develop the consumer decision.
They have some utility function which is going to be a model of their preferences.
OK?
So I'm going to propose to take everything you love in life and write it down as a u function.
OK?
Then I'm going to propose you take all the resources at your disposal, write them down as a budget constraint and then I just do constrained maximization to solve for how you make decisions.
Firms, on the other hand, are going to maximize profits.
Pi is profits.
Firms are going to maximize profits.
Their goal is to make as much profit as possible, to earn as much money as possible.
OK?
However, that's going to be subject to both the demands of consumers, we get to firms it's a lot harder, subject to both consumer demand and input costs.
So firms have to consider, consumers have to consider look what does stuff cost and what do I like, I'll make my decision.
Firms is a little more complicated.
They've go to consider, what do consumers want and how do I make what they want?
So they've got to consider both the output side, what a consumer is going to want me to make and what's it going to cost me to produce that good?
And how do I combine those to make the most profits?
OK?
So from these assumptions, we will be able to answer the three fundamental questions of microeconomics.
OK?
The three fundamental questions of microeconomics will be, what goods and services should be produced?
What goods and services should be produced?
How to produce those goods and services?
And who gets the goods and services?
What goods and services get produced?
How to produce those goods and services?
And who gets them?
And what's amazing, we'll learn in this course, is that all three of these questions, the three fundamental questions that drive our entire economy, are all solved through the role of one key state variable, which is prices.
Prices in the economy resolve all of these problems.
OK?
Consumers and firms will interact in a market, they'll interact in a marketplace.
And out of that marketplace will emerge a set of prices, in a way we'll describe.
And those prices will allow firms and consumers to make the relevant decisions.
OK?
So let me just give you one, and we're going to do all this rigorously throughout the semester, let me just start with one casual example.
OK?
Let's think about the development of the iPod.
OK?
Let's cast our minds way back, lo way back to the development of the iPod.
OK?
Now, when Apple was thinking about making the iPod, they had to ask, would consumers want this?
So consumers had to decide given their limited resources, given the fact that they were buying a certain set of things would they be willing to forsake some things they were already doing to spend the money on the iPod?
OK?
It was a non-trivial amount of money.
Would they be willing to forsake things they are already doing?
OK?
To spend money on the iPod.
And clearly they were.
Clearly, consumers were willing to spend money, to spend a lot of money, to get an iPod.
They were originally what?
$300 Back when $300 meant something.
OK.
So basically, what the firm will do is they'll say, OK, we get a signal from the consumer that they're willing to pay money to get the iPod.
They're willing to pay a high price to get the iPod.
Now, the firm will say, well, should we make iPods?
Well, that will depend on what it cost to make them.
So we have to then assess what are the inputs that we'll need to make an iPod?
Well, to do that we have to look at the prices of the various inputs that we'll need, of the chip in them and the metal and all the other stuff that goes into the iPod.
OK?
So they can shop across different countries, to different kinds of chips, they can look at different kinds of monitors, et cetera.
But, once again, what they'll do is they'll use the prices of those different inputs to decide how to produce the iPod.
So whether to produce the iPod will depend on the price people are willing to pay for it.
How to make the iPod will depend on the prices that firms have to pay for the chip and the casing and all the other things that go into the iPod.
OK?
And then finally, who's going to get the iPod?
Well, they're going to make a certain amount.
What decided who gets them?
Well, the person who gets them are the people who are willing to pay the price that Apple decides to charge.
Some people are willing to pay that price, they're going to get an iPod.
Some people are not willing to pay that price, they will not get an iPod.
So the price in the market will ultimately decide who gets the iPod, as well.
OK?
So basically, prices will determine what gets produced, how it's produced, and who gets the goods that are produced.
OK?
Of course, this is a very, very simplified example, as you can already tell.
There are lots of cases where prices don't decide these things.
So my favorite example is the fact that there are lines for hours to get tickets to see a Lady Gaga concert.
OK?
Now if it's really true that prices determine everything, we shouldn't see any lines.
It should just be that those who are willing to pay the most to see Lady Gaga should.
Those who want the most to get Lady Gaga should get the tickets.
Those who aren't willing to pay shouldn't get the tickets.
Why should there be a line?
Not to mention the fact that people shouldn't be willing to pay anything, but that's a different issue.
That's a taste issue.
We'll come to taste later.
OK?
So basically, clearly this is not working perfectly.
If the world worked in the way I just described, then what should happen is there should be essentially an auction and whoever is willing to pay the most for Lady Gaga tickets would get them.
And whoever is not willing to pay wouldn't.
It wouldn't involve any waiting in line or other things.
Now, what's very interesting is we've actually seen an evolution from my youth to your youth towards the economic model.
When I was a kid, if you wanted, so then it was Cars tickets, OK, to date myself, OK, you had to go and camp out at 3:00 in the morning outside the store where they're selling them to get the tickets.
Now, of course, you don't do that anymore.
Now you go on Stub Hub or Ticketmaster or these other secondary sellers and there there are prices that determine it.
So how many people have waited on line to get a concert ticket?
That's amazing.
So if I asked this question 30 years ago, 90% of the hands would have gone up.
OK?
So what that means is the price mechanism has started to be used.
It has replaced the line mechanism as a way to allocate those tickets.
And we see prices working.
That wasn't true.
There wasn't StubHub.
There weren't these secondary ticket sellers 30 years ago.
You had to wait on line to get the tickets.
Now, so that's basically, sort of, an overview about, sort of an example, of how we think about the role of prices.
Now, let me draw a couple of important distinctions, terms I'm going to use this semester that I want you to be comfortable with.
OK?
The first distinction I want to draw is between theoretical versus empirical economics.
Theoretical versus empirical economics.
OK.
Theoretical economics is the process of building models to explain the world.
OK?
Empirical economics is the process of testing those models to see how good a job they do in explaining the world.
OK?
We could all make up a model.
OK?
Anybody with math skills could make up a model.
But it doesn't do any good unless it's actually doing something to explain the world.
And so basically, the goal of theoretical economics is essentially to build a model that has some testable predictions.
To build a model that says, look here's my simplified model of how consumers decide whether or not to buy an iPod.
OK?
I have a model of that, that I've built theoretically.
Well, that has some testable predictions.
And the role of empirical economics is to gather the data and go and test them using statistical methods.
Specifically, typically regression analysis like the kind you learn about in advanced statistics.
OK?
So basically, what we're going to do is we're going to is 95% of this course will be about theoretical economics this semester.
It will be about understanding how economists develop the models to model how consumers and firms behave.
But I will try to talk somewhat about empirical economics and what data we can bring to bear to understand whether or not these models explain the world.
OK?
The other distinction that's very important is positive versus normative economics.
Positive versus normative economics.
And this is the distinction between the way things are, which is positive economics, and the way things should be which is normative economics.
Distinction between the way things are and the way things should be.
OK?
So let's consider a great example of microeconomics at work which is auctions on eBay.
OK?
Auctions on eBay, economists love studying auctions on eBay because it's a textbook example of what we call a perfectly competitive market which is what we'll focus on the semester.
A perfectly competitive market.
OK?
And by that we mean that basically that producers in this market offer up their good to a wide range of consumers.
OK?
A number of producers offer up their goods to a wide range of consumers.
OK?
And the consumers bid up the price until the person who has the highest value for the good gets it.
So price serves exactly the signal it should in allocating goods.
OK?
So really eBay's really sort of about this third thing of who gets the good.
OK?
I offer my alarm clock or whatever on eBay, OK, and then people bid on that.
And whoever values that the most, that rare Jon Gruber alarm clock the most, they get it.
OK?
So it's a perfect textbook example.
OK?
And basically, because on eBay the price is used, or now with also StubHub and concert tickets, the price is used to allocate the good to the person who wants it the most.
OK?
Now, a recent example of an auction on eBay that a lot of attention, not so recent anymore a couple years ago, someone tried to auction their kidney on eBay.
OK?
Someone offered their kidney for auction on eBay and said, I have two kidneys I only need one.
So I'm going to auction my kidney, you pay for me to fly to wherever you need my kidney and the operation, they take it out and they give it to you.
And that's the way it goes.
So what happened was person offered their kidney and they said the starting price will be $25,000.
They didn't do a buy it now.
They said the starting price will be $25,000 and the bidding went on.
The price got to $5 million before eBay shot it down.
eBay shut the auction down.
And eBay said no, in fact, you can't do this.
Now there's two questions here.
The first is, why did the price of the kidney go so high?
That's the positive question.
The positive question is, why did the price the kidney on eBay get so high?
And here, we'll talk, and you'll learn more starting Friday, about the twin forces of supply and demand.
The twin forces that drive the economy of supply and demand.
And you'll talk more rigorously about these on Friday.
Basically, they're what they sound like.
Demand is how much someone wants something.
Supply is how much of it there is to have.
And the intuition here is surprising.
OK?
The more that there's demand for a good, the higher will be the upward pressure on prices.
The more people want a good, the higher prices will go.
And the less supply there is of a good, also the higher prices will go.
So if everybody wants something but it's common, the price will be low.
And if no one wants something but it's uncommon, the price will still be low, and vice versa.
In fact, the development of the model of supply and demand framework was from Adam Smith, the, sort of, so-called first economist who wrote The Wealth of Nations in 1776 which is, sort of, viewed as the, kind of, first serious book about economics.
And he posed what he called the water diamond paradox.
What Smith said in that book is, look, it's clear water is the most important thing in life.
We can't live without water.
And diamonds are completely irrelevant to life.
You can live totally fine without a diamond.
And yet, the price of diamonds is astronomical and water's free.
How can this be?
How can it be that water which is so much more of a fundamental building block of our life is so much cheaper than diamonds which are not.
And the answer, of course, is that so far you've only considered demand and not supply.
Yes, it's true.
The demand for water is much higher than the demand for diamonds.
But the supply is even larger.
So that basically, yes it's true that while water should be worth more, in fact, in the end the price of water is much lower, because of the twin forces of demand and supply.
The demand is higher, but the supply is much higher.
So the price ends up lower.
And that was his diamond water paradox.
OK?
Well, in this case, it's a similar thing.
What determines the demand for a kidney?
What determines the demand for a kidney is going to be the fact that you die without it.
OK?
If you have no kidneys, you're having kidney failure.
OK?
You'll die without it.
So basically, what will determine it is people are willing to spend all their wealth, as much money as they can have, to get a kidney OK?
So the demand will be quite high.
The supply will be quite low.
Sadly, not many people are willing to be organ donors.
More relevantly, a lot of people aren't in good situations to be organ donors.
OK?
As a result, the supply is much lower than the demand.
So we have a situation with a high demand, a low supply and the price went through the roof.
That's a positive analysis.
OK?
So we can understand pretty intuitively.
We don't need this course to understand why the price went up.
OK?
It's just the twin powers of demand and supply.
But what about the normative question which is, should eBay have allowed this sale to happen?
EBay at $5 million cut it off and then passed the rule saying you can't auction your body parts on eBay.
OK?
Should they have done that?
That's the normative question.
That's economics gets really interesting, which is you all are smart enough to figure out why the price went up.
But this is where it gets interesting is should people have been able to auction their kidney on eBay?
On the one hand, many, many people in this country die for want of a body part.
OK?
Thousands to hundreds of thousands of people die every year waiting for a transplant.
OK?
If someone is incredibly rich and they want a body part, which to me a surplus because I have two kidneys, why shouldn't they be allowed to buy it from me?
I'm better off because they can pay me a ton of money.
They are better off because they live.
So I've just described a transaction that makes both parties better off.
Why shouldn't that be allowed to happen?
So you tell me.
Does everyone think eBay was wrong?
Yeah, go ahead
Say there is another person who doesn't have as much money, and that person also dies
You mean the person who, what do you mean the person doesn't have as much-
--so obviously somebody doesn't get the kidney
So in other words, what you're assuming is, let's say that if I hadn't done the auction on eBay, I would have just given my kidney away to the transplant center.
Then that's one less kidney that can go to the transplant center.
And that means the rich guy gets the kidney, and someone else implicitly doesn't.
That's a trade-off.
You've just described a trade-off.
The trade-off is that basically now we've allocated the kidney away from the poor person to the rich person.
Now, but why do we care about that?
I mean one person dies, another person lives, why do we care?
Yeah?
There would be some sort of case of severity in condition.
Like there might be someone who's poor who would get the kidney if it went to a transplant association because they would die in a couple of days without it.
Whereas the rich person might just be able to afford it, and it might make their life more convenient.
But they might not be in any more peril
They might be a collector.
So basically, that's right.
So one reason we might care is because we think that kidneys should be allocated on the basis of who needs it the most.
OK?
So a great example of this, of course, was Mickey Mantle with a liver transplant.
Mickey Mantle, famous ballplayer, raging alcoholic, who had liver failure because he was basically drinking himself to death, and jumped the queue and got a liver above a bunch, a lot of people and then he kept drinking and killed himself and wasted the liver he'd gotten.
OK?
So basically, you can think that doesn't make sense.
We should give it to people who need it the most.
For who it would do the most good in terms of increasing their life.
OK.
So we've got the substitution point.
OK. Let's come back to the substitution point though.
Tell me a situation in which that's wrong.
Can someone tell me a situation in which, in fact, that not a valid point.
Yeah
Well, if the guy is only going to sell it.
He's not going to give it away
Exactly.
You're assuming that the guy who did sell would give it away.
But, in fact, if it's sell it or keep it then there's no trade-off.
And similar here, if it's sell it or keep it then you might as well let the rich guy get it.
Or is there another argument?
Is there another reason why you might not want this to happen?
Yeah
It would encourage people to use illegal ways of getting kidneys
So the other reason could be that we don't trust people to make good decisions when money's involved.
That we think that, gee, if it's really true I can get a couple million bucks for a kidney, I might give mine up even if I haven't really thought through the ramifications of doing so.
Even if there's a risk to the surgery, if there's a risk that my other kidney will then fail then I'll be screwed.
OK?
So basically, we might have a paternalistic attitude that will lead us to not want to allow people to engage in this kind of risky behavior.
Yeah?
There may also be some legal ramifications associated with that if someone sells their kidney and then their other kidney fails, they might then blame eBay
Want it back.
Like that Repo movie.
That's right.
There could, but let's leave the lawyers out of this, OK?
I don't like lawyers.
I'm going to rag on lawyers this semester.
We're going leave the lawyers out of this.
But, in any case, you're right.
That's, sort of, a ramification of the same thing.
So we've talked about the fact that there's substitution.
We've talked about the fact that it's not allocated to those who need it the most.
We've talked about the fact that people might be making bad decisions in doing this.
But there's another factor, as well, which is we may just as a society feel it's unfair that rich people can get things poor people can't.
There may be a pure equity component here.
OK?
Which is simply that we as a society value equality, value income inequality.
And we think people should not have an extra shot at getting a resource just because they're rich.
Now that is a very deep and hard concept, and we'll spend a couple lectures talking about equity towards the end of the semester.
By and large, we won't consider it.
OK?
But it turns out to be behind much of what we'll discuss, OK, in much of what we'll discuss this semester and much of what goes on in economics.
OK?
Just take a look at the debate that's going on right now in terms of President Obama trying to decide whether or not to extend tax cuts to wealthy individuals in the US.
Some people argue that allowing those tax cuts would promote the economy.
OK?
But others argue it's unfair for rich people to get tax breaks.
And that fairness argument matters a lot in terms of driving the kind of economic policy decisions we need to make.
So this semester, we're going to focus a lot on efficiency and optimization and how to get resources to the right place.
But you have to remember behind a lot of this is deep normative issues about what should be happening, how should an economy function, and, in particular, how should we think about these kind of equity issues that are so important.
OK.
The last thing I want to talk about is I want to talk about why micro is not just an abstract concept for things like you might say, oh this is all pretty funny and it's like selling kidneys on eBay and tax cuts for the rich and why do I care?
OK?
Well, you care because literally every decision you make is made through the kind of framework we're going to think about this semester.
OK?
Now, different decisions may follow our models more closely and less closely.
But there is not an economic decision.
OK?
Sorry, let me back up.
There's not a decision that people have made that economists haven't tried to model.
From whether to produce iPods, to how many times to have sex each week.
OK?
These are all things economists have tried to model with varying degrees of success.
OK?
Because economists think that these all come from the same decision theoretic framework that we can discuss.
Let's talk about a simple example from this course.
Your decision of whether or not to buy the textbook.
There's a textbook and what's it cost?
$140?
What's it cost?
Does anyone know?
This line says $180.
And this one says $180, but it's available for $130
$130, fine.
So $130 for Professor Perloff out at Berkeley.
He doesn't get it all.
I wrote a textbook too.
He gets a small share of it.
OK.
So you have to decide, now has anyone bought a used version of the fourth, well it's the fifth edition now, does anybody use a version of the fourth edition?
Does anyone know what the used price is?
You can be honest.
I don't care
I know some people found it for like $85 or so
So $85.
So you've got to decide.
So let's say you can buy the previous edition, the fourth edition, for $85 or the current edition for $130.
OK?
You've got to make that decision.
OK?
How do you make that decision.
Well you may think, gee I just make the decision.
It's not really about microeconomics.
But it is.
We're going to model how you think about a decision like that.
Well how do you think about a decision like that?
Well, the first thing you consider is your preferences.
How much are you willing to take a chance that there's new stuff in the fifth edition that you need to know?
OK?
If the fifth issue was identical to the fourth edition then you'd be an idiot to not just buy the used fourth edition.
But it's not.
Textbook writers are smart.
They update their book.
OK?
So basically, the fifth edition is updated.
There's new things in it.
So you have to ask yourself, what are the odds that I need some of the new information in the fifth edition and not in the fourth edition?
And in thinking about that, you're going to think about your preferences.
In particular, are you very risk averse, are you afraid to take a chance?
Or are you risk loving?
Are you willing to take a chance?
OK?
That's one side of the equation.
If you're someone that says, you know I will not take a chance in life.
I just have to make sure I learn the most possible from this course.
Then you're going to want that new edition.
If you're someone that says, you know what screw it, I'll just figure it out later.
I'm going to the lectures.
I don't care.
OK?
Then you might not want it that much.
So that's the first factor, is going to be your preferences and break down how much you are willing to take a risk that you need this fifth edition.
The second factor is going to be your constraint: how much money you have.
OK.
The more money you have, the more you're willing, or more relevant perhaps your parents have, the more willing you are to go ahead and buy that new edition.
The less money you have, the less willing you are.
OK?
So basically, it's going to depend on whether you're paying or your parents are paying, in which case what the hell you might as well buy the fifth edition.
OK?
And then finally, you're going to take these preferences and this constraint, your preferences and your resources, and go to the market and look at what does the market tell me the difference is.
So you're going to say, here's how much I kind of care about fifth versus fourth edition, here's the resources I have, now I'll go to the market and say, aha there is a $45 difference between these two editions.
So now I will solve that constrained optimization problem.
And I'll decide whether I want to get that book.
You are going to be thinking to yourself, this is stupid.
I never think about it that way.
But the key point is you don't have to think about it exactly that way.
There's a famous example in economics of what's called the as if principle.
I don't know if kids still say this one, as if.
So it's the as if principle.
OK?
And basically it's from Milton Friedman, the famous economist from Chicago, who said, look, when you're playing pool, technically you could compute the optimal angles of which to shoot the ball every single time to get the appropriate bounce and get the balls in.
You could do the mathematics and compute it.
But professional pool players are not in this class with you, I'll only say that.
OK?
They're not guys who are able to do that computation.
They just know how to hit the ball to get the same outcome they would get if they mathematically solved for the optimal trajectory to hit the ball.
OK?
They behave as if they've solved the constrained optimization problem.
And your decision when you buy this book, you may not think it through the very framework that I just laid out very heuristically and will lay out more rigorously this semester.
But you're going to behave as if you do.
Because those facts are going to be in your mind, and you'll be thinking about it when you make that decision.
So while you may feel the models we learn in this course are rather abstract and don't really explain how you behave in an everyday basis, you're going to behave as if those models are really applying to you.
And if you think about, over the next few days, think about the decisions you make.
From should I bring an umbrella today, it looks like rain?
Well, on the one hand, I might lose it.
It's a pain to carry.
On the other hand, how much do I care about getting wet?
That's a constrained optimization decision.
To should I have an extra drink at a party Friday night?
On the one hand, that could have some pros and cons.
OK?
These are all decisions, constrained optimization decisions, you're going to make.
They're going to affect your life.
And what we'll learn this semester is about how you make them and how we model, how economists can use what we learn about that to understand the function of the economy.
OK?
So I'm going to stop there.
One other announcement, there will, in general, be handouts every lecture.
I'm not going to do PowerPoint.
I'm going to do handouts.
So when you come in every lecture, please look at the back banister.
Though typically, not today.
But typically, they'll be handouts that you'll need to follow along with in class.
So remember, go to section on Friday, first problems that will be posted on Friday and it will be due in section the Friday after.
And I'll see you all back here on Monday, next Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now, what we're going to do today is build on what you learned in section on Friday.
On Friday, you learned about how supply and demand interact to yield equilibrium.
So the sort of fundamental concepts we talked about with Adam Smith and his water-diamond paradox of supply and demand interacting to yield a market price and quantity.
And the way we're going to build on your understanding of this today is talk about what happens when you shock that supply and demand equilibrium.
When things change, how does that change that supply and demand equilibrium?
And that'll help build your understanding of what you learned.
So to review, let's go to our basic supply and demand graph, Figure 2-1 in the handout.
You talked in section about the market for pork.
That's the market Perloff uses.
Seems good as any, so we'll continue to rely on that, the market for pork.
And in this pork market, you have a demand curve and a supply curve.
Now, it's critical to remember what these curves represent.
Once again, for this course, ideally, you understand all concepts on three levels, intuitively, graphically, and mathematically.
But the intuitive is the most important, especially for those of you who aren't going to go on and do a whole lot of economics.
For instance, where it's just one of the few economics courses you'll take, it's very important for your lives and using economics you understand this intuitively.
So intuitively, let's talk about what the demand curve represents.
Can anyone tell me, what's the demand curve represent, from what you learned on Friday?
Can anyone just take a shot?
We're just-- yeah, go ahead
The willingness of consumers to buy a certain product
Exactly.
The willingness of consumers-- I'm going to change your words just a bit, because words are important here.
The willingness of consumers to pay.
The willingness to pay.
The difference with buy is it's really because the different amounts.
They're going to be willing to pay different amounts.
So it's the willingness of consumers to pay for the good.
The willingness of consumers to pay for the good.
So what this says is that each point in the demand curve represents the price consumers are willing to pay for that quantity.
And it's downward sloping.
The demand curve is downward sloping because as the price goes up, consumers are willing to buy less of a good.
So their willingness to pay changes.
So that's the demand curve.
Now what about supply curve?
What's a supply curve represent?
Anyone?
Yeah?
Pretty much the opposite.
The willingness of producers to produce, like, how much they are going to charge for--
Yeah, the willingness of producers to supply the good.
So how much they're going to charge for a given quantity of the good.
And once again, that is upward sloping because as the price rises, they're willing to supply more.
So as the price rises, consumers demand less.
As the price rises, producers produce more.
And equilibrium is that point where supply equals demand.
Equilibrium equals happiness.
It's the point where suppliers and demanders are both happy.
They're both happy because at a point such as e, the amount that consumers demand at that price is equal to the amount suppliers are willing to supply at that price.
So jointly happy.
You've reached a point where consumers want a certain amount at a certain price.
At that price, producers say, great, you want e, I'm happy to produce e at that price.
So we've reached equilibrium.
Now, let's talk about what happens when we shock that equilibrium.
This is the market for pork.
Imagine that the price of beef rises.
The price of beef rises.
Can anyone give me a guess as to what that does to the market for pork?
The price of beef rises.
What does that do to the market for pork.
Yeah
It reduces the demand
It what?
Reduces the demand
Why would it reduce the demand for pork?
Because the price [INAUDIBLE]
But I didn't say the price of pork rises.
The price of beef rises.
And how does beef relate to pork?
They're substitutes
They're substitutes.
Exactly.
So when the price of beef prices, how does that affect people's demand for pork?
Increases it.
Exactly.
Because they're substitutes.
What we're going to learn critically as we go through is what's going to determine these demand curves importantly is going to be substitutability across goods.
So here we have a situation where the substitute for pork has gotten more expensive.
As a result, people want more pork.
That is a shift out in the demand curve, or a shift up in the demand curve.
A shift up or out, depending on it's out into space or up vertically in the demand curve.
So what happens here is that folks shift to consuming pork, so they want more of it.
So we're initial equilibrium-- we just made up some numbers-- we're initial equilibrium here at 220 millions of kilograms of pork a year and a price of $3 a kilogram.
That was the initial equilibrium.
That was the point where demanders and suppliers were happy.
Now the price of beef has gone up, maybe because-- I don't know.
Mad cow disease or something like that has raised the price of beef.
So now people are saying, hey, we want more pork.
That shifts the demand curve out.
Initially, if the price stayed at $3, if the price remained at $3, what would happen?
What would happen is people would now-- and once again, these numbers are made up.
But this is just to illustrate an example.
People would now say, gee, at a price of $3 and this further out demand curve-- now I'm on Figure 2-2, if you haven't picked that up-- I want 232 millions of kilograms of pork a year-- not I, personally.
That'd be like Homer.
Everybody wants 232 millions of kilograms of pork a year.
I hope you guys don't get tired of Simpsons references.
So what that says is consumers say, gee, I want a lot more pork at that price.
If it's going to be $3, and the price of beef just went up, I'm going to want a lot more pork.
So what you're going to have initially is excess demand, because at that price of $3, suppliers are only willing to supply 220 million kilograms.
They were happy, right?
They were happy at point e1.
When you come along, and you say consumers now want more, they say, well, we're not happy to provide more at that price.
If you want more, we're going to have to charge a higher price.
So they say, if you want more pork, we're going to have to produce more.
So we are going to slide up the supply curve.
So if you start at a point like e1, they say, gee, you want more pork?
Well, we're going to have to slide up that supply curve.
So we're moving up the supply curve and saying, we're going to have to charge you a higher price if you want more.
Well, as the price goes up, consumers say, well, wait a second.
If the price is going up, I don't want quite as much more.
And you meet at the new equilibrium e2.
So what happens is producers say, gee, if you want more, I'm going to charge you a higher price.
Consumers say, well, gee, if you're charging a higher price, I'm going to go back up the demand curve.
I don't quite want as much at that higher price.
And the new equilibrium is e2.
That's where consumers and producers are now happier with that outward shift in demand curve.
But the key point is consumers are not getting as much as they originally wanted at the original price, because the original price will not hold.
The price is going to change.
Given the price is going to change, the quantity is going to change.
And you end up with both a higher price and a higher quantity for pork.
So think about it.
This is pretty interesting.
The price of beef rising raised the price of pork.
It didn't just increase the demand for pork, but through increasing the demand, it raised the price.
So price in one market affects the price in another market, not just the quantity but the price in another market.
And that's our new equilibrium.
That's the shift in the demand curve.
And once again, suppliers and demanders are happy.
They're happy at this new point e2, this new equilibrium, because given the new demand curve, at a price of $3.50 a kilogram, producers are happy to produce the 228 million kilograms of pork a year that consumers want.
And at that price, that's what consumers want.
If they're going to say, oh, it's $3.50, I want about 228.
I want more than I wanted before, because beef's gotten more expensive, but not as much as I would have wanted if you hadn't increased your price.
And that's the new equilibrium is e2.
Yeah
But is it possible that changing the price of pork could affect the price of beef?
Excellent.
So the question is, could there be a feedback effect where, basically, now the pork market's changed, could that go back and affect the beef market?
In principle, yes.
In practice, we're going to make your life easy by saying no.
In principle, it could.
And that gets into something called "general equilibrium."
And that's way too complicated for this course, so we are going to ignore those kinds of feedback effects for now.
That's something you can learn about more in future courses.
But you're right.
In principle, it could affect the demand for everything.
You could imagine that as the price of beef goes up, that means that, for example, the cost of restaurant meals goes up.
So I'm not going to eat at the restaurant as much, so maybe I'll go to the movies more.
So in principle, the price of beef going up could raise the demand for movies.
It's crazy.
It's hard to know where to stop.
So we're going to make your life easy by stopping at one level here.
But you're right.
In principle, there could be these wide range of feedbacks.
And once again, I talked in the first lecture about simplifying assumptions.
We're going to try to teach you the basic concepts but try to make simplifying assumptions that make it more manageable.
That's the kind of simplifying assumption we'll make for our modeling here in this course.
So that's a shift in the demand curve.
Yeah
How do you quantify the shifts in the demand curve?
Excellent.
And that will be exactly-- basically, the way we're going to learn in this course, we're going to do things a couple of times.
When we talk about producer theory, we will teach you where this supply curve comes from.
Basically, firm profit-maximizing decisions will determine the supply curve and how much it shifts, and we'll teach you that when we teach you producer theory.
So this is sort of a big overview.
You're good MIT students.
You're immediately thinking, where the hell do these demand and supply curves come from?
That's where we're going to get to.
That's the whole point of consumer theory will give us the demand curve.
Producer theory will give us the supply curve.
All right.
Good questions.
So now, let's talk about a different case.
Let's talk about a shift in the supply curve.
So now, let's suppose there's a drought.
And let's suppose it's a drought that only affects pork.
Because if the drought affected beef, that would then have these other feedback effects.
Let's imagine there's a pork drought.
I don't know what that would mean, but let's imagine that happened.
So suddenly, it becomes more expensive-- or let's just say there's mad pig disease.
Let's imagine there's mad pig disease.
So suddenly, it becomes more expensive to produce pork.
Because there's fewer pigs, it's more expensive to produce it.
So what that would do is that would cause a shift in in the supply curve-- inward or upward of the supply curve.
Suddenly, on you get the fact that given that it's more expensive to produce pork, suppliers say, well gee, to produce a given amount of pork, I need to get paid a higher price.
So we get, once again, initially, if the price stayed constant-- so now I'm on Figure 2-3.
If the price stayed constant, we would not begin with excess demand.
Because if the price stayed constant, then consumers would want 220 million kilograms of pork, which is what they wanted before.
Nothing's changed from their perspective.
But producers would say, look, at that price, I can only now produce 205, because my supply curve shifted up.
So once again, you have to move to a new equilibrium where suppliers and consumers are happy.
That will happen at a point like e2.
Because at e2, given the higher price of producing pork, producers will now say, OK, I'm happy to sell 215 million kilograms at $3.55.
And consumers will say, well gee, at $3.55, I don't want quite as much as I wanted before.
I only want 215.
And they're happy.
Here's what's striking.
We had two very different phenomena.
We had a demand shift and a supply shift.
Both led to the same outcome.
I'm sorry, both led to a higher price.
Sorry, not the same outcome.
Both led to a higher price.
One led to higher demand, to higher quantity sold in the market.
So you go back to Figure 2-2.
The price went up, and the quantity in the market went up from 220 to 228.
In Figure 2-3, when the supply shift happened, the price also went up almost exactly the same amount.
But here, the quantity fell.
So you can't tell from a price increase what happened.
If the price of pork goes up, you can't tell me whether that was a demand or supply shift.
You need to know both the price and quantity to be able to tell me that.
Both changes led to the same outcome in terms of prices.
Questions about that?
Now let's make it interesting.
So we have the supply and demand equilibrium.
Supply shifts, demand shifts, you move the points.
You guys can all do it graphically, even if you don't quite get it intuitively yet.
Now let's ask, what happens if the government comes in and messes this up?
Now, step aside to my other hat.
The other course I teach is called Public Economics, which is all about the role of the government in the economy.
In this course, most of what you'll learn is there's a clear role of the government in the economy.
It's to screw it up.
Economics is fundamentally a right-wing science.
Most of my colleagues are Democrats, but that doesn't change the fact that what we teach is fundamentally very conservative, which is the market knows best, and governments just mess things up.
Now, later in the semester, we'll talk about why that might not be true.
And the whole point of my other course, 14.41, is to talk about why that might not be true and where governments can make things worse and better.
But from your basic perspective, you need to be indoctrinated with this fundamental position, that the market gets it right.
The market equilibrates.
Governments mess things up.
And that's sort of where we'll start.
So let's talk about the classic example of government messing things up, the minimum wage.
You've all heard of the minimum wage, the thing workers get paid more.
That seems like a good thing.
Well, in fact, to economists, it's a bad thing.
And let's talk about why.
To many economists, at least this initial analysis is a bad thing.
We'll come back later as to why it might not be so bad.
To do that, let's talk about equilibrium not in a market for goods, like pork, but in a market for inputs, which is labor.
Labor is an input to the production process.
We'll talk about this at great length later in the semester.
But the key thing is, just like there's a market for pork where the equilibrium outcome is the price you pay for pork and the amount of pork sold, there's a market for labor.
And in that market, things are flipped.
Who are the consumers now?
Well, they're the firms.
The firms demand labor.
Who are suppliers now?
Well, they're you.
They're the people.
People supply labor.
So now we've flipped things on its head.
The suppliers are the people.
The demanders are the firms.
But the analysis is the same.
This basic supply and demand framework, which will turn out to be so powerful this semester, can be used, even though we flipped the labels of who's on what side.
So you see in Figure 2.4, we have a labor market with an upward-sloping supply curve.
What does that mean?
Why is the supply curve upward-sloping?
Because the higher your wage, the more you're willing to work.
And we have a downward-sloping demand curve.
Why is it downward sloping?
Because the higher the wage, the fewer workers the firm wants to hire.
It would rather use machines instead.
So basically, you've got that same supply and demand curve, different stories as to where they come from, but the same effect in terms of determining equilibrium.
We get an equilibrium at w1 l*, which is at a wage w1, l* workers are willing to work.
Or l* hours are supplied of labor.
We think about hours supplied, not necessarily number of workers.
l* hours of labor are supplied.
And at that w1, l* workers are willing to provide l* hours, and at that w1, firms demand l* hours, and that's the equilibrium.
Same analysis as before, just different stories as to what's behind the curve.
Now, let's say the government rolled in and set, as in Figure 2-5, a minimum wage.
The government says, look, that wage w1 is too low-- or w* in this graph-- is too low.
Workers are not paid enough in equilibrium.
We need to make sure that they get paid a minimum amount.
So we are going to mandate that workers cannot be paid below w lower bar, which is critically above w*.
Well, what happens in that case?
Well, in that case, at that wage, workers are thrilled.
They say, look at that wage.
We are delighted to work-- god, the numbers are-- I'm getting old-- ls hours.
I have my prescription for my bifocals.
I've had it for, like, two months, and I just can't quite take it to the mall and get them.
So eventually I'll get them.
I just can't quite admit I'm that old.
So workers are like, great.
At that wage, we'll supply l sub s hours.
That sounds great to us.
We're all good.
But the firm says, wait a second.
No, we don't want to hire l sub s hours at that wage.
That's way too high.
We only want to hire at that high wage l sub d hours.
So suddenly, here you have a case of excess supply.
Not excess demand, excess supply.
Workers want to supply more hours than firms want to hire.
Now, in our previous example, what would happen?
What would happen is, if there's excess supply-- well, let me ask you.
If there's excess supply in the market, how would the market equilibrate?
What would happen?
Yeah
It would lower the price so that the--
In this case, the price is what?
The number of--
The wage.
The wage.
So exactly.
In our previous example, the wage would fall.
Workers would suck it up and get a lower wage.
Firms would be happy to pay a lower wage.
You'd reach a point where they were happy.
You'd reach a point like e. But you can't now.
You can't, because the government said you can't pay less than w lower bar.
So you end up with excess supply.
And we call that excess supply "unemployment."
You end up with unemployment, because at that high wage, firms are not willing to hire as many workers as are willing to work.
And you end up in disequilibrium.
You end up not in equilibrium but disequilibrium.
Now, disequilibrium is kind of hard to think about, because in some sense, this course is all about equilibrium.
So I'm not talking about chaos or cats and dogs living together, breakdown of social order.
That's not what I'm talking about.
Disequilibrium, there is still a market outcome.
The market still settles at a point.
In this case, what happens is the market settles at a wage of w lower bar because it has to and at an amount of labor delivered at l sub s, that low amount of labor delivered.
Because when you're in disequilibrium, the outcome of quantity and prices is determined by the constrained party.
That's the way I like to think about.
You can think about it however you want.
But here's the point.
Workers can't work if firms won't hire them.
So if you want to say, look, we're going to end up at w lower bar, how much labor is going to be in the market?
Well, it's up to the firms.
They get to decide.
They're going to say, look, we don't want more than l sub d, so that's how many jobs they're going to provide.
So the new outcome is going to be at a wage w lower bar and an amount of hours l sub d. So basically, you end up with this unemployment situation where only l sub d workers are hired and the wage is w lower bar.
Now, let me ask a question.
What would happen if w sub lower bar was set below w*?
What if I'd redrawn this diagram so w lower bar was below w*?
Yeah
The market would equilibrate and--
Yeah, nothing would happen at the end of the day.
And here's the key point-- very important point to remember-- which is that markets are very robust.
Economic markets are very robust.
And if they can undo the effective government intervention, they will.
So if the government tries to come and intervene in a way which markets can undo and get back to where they wanted to be, they will.
That original point e is where the market wants to be.
That was where it was happy.
It can't get there in the example I taught here with w lower bar above w*.
But if w lower bar was below w*, it could get there and be happy, so government intervention would make no difference.
That's one example of a government intervention.
Let's do another example.
Then I want to talk about the pros and cons of these government interventions.
Let's do another example.
Gas price ceilings.
So let's consider a cap on how much can be charged for gas.
Seems a pretty sensible policy.
Gas is crazy expensive.
We all like to drive.
So let's consider a cap on the price of gas.
Imagine, for example, we're initially in equilibrium, in Figure 2-6, with demand of Q2 and a price of P1.
We're all happy.
Now let's imagine there's an oil crisis, because, say, oil companies decide it's a good idea to drill eight miles underground, and everything explodes, and we suddenly run out of oil.
So all of a sudden, there's a constraint in the supply.
Suddenly, there's a-- I'm sorry, my bad.
We're initially in equilibrium at e1.
My bad.
We're initially at equilibrium at e1, with a price of P1.
There's a bit of mislabeling here.
On the vertical axis, that upper price should be P2.
So on the vertical axis, you see there's P1 equals P. Then above it, it says P1 again.
That should say P2.
So we're initially in equilibrium at point e1 with a price of P1 and a quantity of Q1.
Now oil tanks blow up all over the world.
Suddenly, that means there's a restriction in the supply of gas.
Suddenly, there's not as much gas as can be produced.
So that's an upward shift in the supply curve, just like we talked about before.
And absent any government intervention, the supply curve would shift up to s2, consumers would want less gas, and you'd reach a new equilibrium at e2.
And once again, let's talk about equilibrium as being where people are happy.
Now, you could say, well gee, people aren't happy paying a higher price for gas.
Well, that's why I'm a bit glib using the term "happy."
It's a point where the suppliers and demanders are jointly willing to make the deal.
And they're willing to make the deal at e2.
They're willing to say, look, given how many oil rigs have blown up, we are happy to equilibrate the market now at a higher price and a lower quantity.
But then it's September, 2010.
There's an election two months away.
President Obama isn't very happy about this.
And he says, forget it.
We're going to cap the price at P. We were paying P before.
We can pay P again.
Those stupid oil companies can suck it up.
They're the ones who drilled eight miles underground.
They're the ones who blew everything up.
They can suck it up.
We're keeping the price at P. We'll help the consumer.
We'll keep the price.
Well, what happens?
What happens is that consumers are now, great, we continue to want Q1.
That's where we were before.
We're happy.
But producers say, well, you know what, President, we can't supply that much at that price.
Because it costs us so much to drill oil now that if you're going to force us to keep the price at P1, we are only going to supply Q sub D because we just can't supply at that low price.
We're going to supply Q sub D. So what happens, you end up with excess demand.
Consumers want Q sub D gallons.
Producers are willing to produce only Q sub S. And you end up with disequilibrium.
And in this case, once again, the amount sold is what the producers are willing to sell.
It doesn't matter how much consumers want if producers aren't willing to sell it.
You end up with disequilibrium, and you end up with much, much less gas being sold.
So yeah, the price stays low, but the amount of gas being sold is much lower than if the president allowed the price to rise.
So that's another example of a government intervention causing disequilibrium.
Any questions about that?
Yeah
In the last chapter, we talked about, can the government then come in and give a subsidy to the oil producers to produce the same amount?
Great.
Hold that thought for 14.41.
But that's exactly what the government could do.
The government has lots of tools.
So it can come in and say, well, on the one hand, we'll set a price cap.
But then we'll come in on the other hand and give a subsidy to oil producers to make sure they do that.
That has two problems.
One is, not so good with the voters.
Like, gee, we're going to give subsidies to oil companies.
Isn't that a great idea.
Two is, you've got to raise the money somehow, which means you've got to raise taxes.
Also not so good with the voters.
So it's, in general, not a good idea, because you screw it up on the one hand to screw up again.
Two wrongs don't make a right, something I learned from an early age.
If the government's messed up the market on the one hand, it's not, generally, a good idea to try to mess with it again to fix that.
But that's exactly the kind of stuff we'll discuss in 14.41
I've got a question about that.
Will that mean that there's a [INAUDIBLE] because [INAUDIBLE]?
Great segue into what I want to talk about next, which is, what are the costs and benefits of these kinds of market interventions, of a minimum wage or gas price ceiling?
Why do we do this?
The government, these aren't stupid people, by and large.
Why do governments do things like this?
What are the costs and benefits?
Now, later in the semester, we'll talk about welfare economics.
By welfare, welfare has two meanings.
We often think of welfare as being money distributed to poor people.
That's one meaning.
But when we say "welfare economics," we actually mean the well-being of society.
And we'll talk about the well-being of society later on in the semester.
But let's talk for a minute about this general topic.
And how do we think about the kind of welfare economics of these restrictions?
Well, there's two costs and one benefit to these restrictions.
The first cost is the efficiency loss.
So the costs of things like the minimum wage and gas price ceilings, the first cost is the efficiency loss.
And we are going to be much more precise.
This lecture is sort of a chance to be loosey-goosey about things that I'll then make much more precise throughout the semester.
We'll be precise about what we mean by this throughout the semester.
But the key point is, in economics, whenever there is a trade that can be made that makes both parties better off, and it is not made, that is an inefficiency.
So in economics, we define "efficient" as when all trades that can make both parties better off are made.
And whenever anything comes up that interferes-- so a trade that could make both parties better off is not made-- that is inefficient.
There is an efficiency loss.
And think about it.
It makes sense.
If both parties could be better off by a trade and don't let it happen, then society is worse off.
That's the idea of an efficiency loss in economics.
Economics is all about trading to make things work more efficiently.
When you don't let that happen, you've hurt society.
You have a welfare loss, because you have something that could've made both people better off, that would've been a good thing to do.
You haven't allowed that to happen.
And if we think about it, in both these examples, Figure 2-5 and 2-6, there are trades that would've made both parties better off that we're not allowing to happen.
So in the labor market case, think about a wage that's above w* and below w lower bar.
So a wage in that interval.
At that wage, workers would be happy-- and take unemployed workers.
Unemployed workers would be happy to work it a wage somewhat below w bar.
Firms would be happy to hire them at a wage somewhat below w lower bar.
But it isn't happening, because the government has interfered.
That's an efficiency loss.
Likewise, look at Figure 2-6.
Consumers would be happy to pay a somewhat higher price and get some more gas.
Producers would be happy to produce more at that higher price.
But it's not happening.
That's an efficiency loss.
So an efficiency loss is whenever there are trades that can make both sides better off that don't happen.
Once again, this is loosey-goosey here.
We'll make this all more precise as the semester goes on.
But it's important to get the big picture concept of what we mean by efficiency in economics, which is trades being made that make both parties better off.
The second cost here is what we call "allocation inefficiency."
Remember in the first lecture, I talked about how prices play three roles in the market.
They determine what is to be produced, they determine how it is to be produced, and they determine who gets it.
That's called "allocation."
Price plays a critical role in making sure that the people who want the good the most get it.
Because remember, that demand curve is a willingness-to-pay curve, or a willingness-to-buy curve.
What that lets us know is the further up the demand curve, the more people want the good.
And we should make sure that those people are allocated that good.
So in a world of equilibrium, we make sure that the allocation happens.
Anybody who wants the good at a price that consumers are willing to sell it gets it.
Someone who doesn't want it at that price doesn't get it.
Equilibrium takes care of that allocation problem.
But in disequilibrium, it doesn't.
Let's consider the gasoline example.
Now we have a case where there are many, many people-- in fact, all the folks who lie between Qs and Qd on that x-axis-- all those folks want gas at that low price but can't get it.
There's only Qs being supplied.
So what happens?
Well, what happened-- you guys weren't alive for this, but in the 1970s, does anyone know what happened when we did price ceilings of this type?
Does anyone know what happened?
Gas shortage
Gas shortage.
And how did people respond?
How was gas allocated in that gas shortage?
There were lines
Peopled waited in huge lines.
Basically, we couldn't use the price mechanism, so what did we do?
We used the wait mechanism, just like they used to do in Russia all the time and still do.
The point is the gas that's limited-- that Qs-- has got to get allocated somehow.
In the market, it gets allocated by the price rising until the price is high enough that the set of people who want it at that price get it.
With this gas price ceiling, that can't happen.
So what happens?
Lines for gas.
Huge, hours-long wait.
This is sort of inconceivable now.
Sometimes, you have to wait at a gas pump, but pretty much, you drive up, you get your gas, you leave.
In the 1970s, literally, you would wait hours-- multiple hours-- in line to get gas.
That was how the gas was allocated in the face of these gas price ceilings.
Yeah
What about things like the government saying you can't shut off someone's gas in the winter, even if there was a price ceiling?
OK, let me come back to that.
Hold that thought for a couple minutes.
I want to come back to that.
But I want to focus on these gas price lines and why they're bad.
Well, these gas price lines are themselves a source of inefficiency.
And why?
Why is it inefficient to have people waiting in line for gas?
Yeah
Because they could be working or doing other things
They could be working.
They could be out making trades that make everybody better off.
Instead of being in line waiting for gas, they could've been at work, working for a wage that they were happy to earn and their employer was happy to pay.
So a trade is not being made.
Unless they're equally happy sitting in line waiting for gas, which is doubtful, a trade is not being made, which makes both parties better off.
What else?
I have a question.
Did the government know about this?
Yeah, believe me, they did.
But they said, well gee, we can't let people pay those high prices.
Governments face really hard decisions like this all the time.
There's another program, of course, which is, what happens with people waiting in line for gas?
They're idling and using up gas.
So in fact, there was a direct mechanical inefficiency as well, which is all the gas that was wasted while people idled in line, waiting to get their gas.
So that's the kind of inefficiency you're used to thinking about as engineers.
There's a mechanical inefficiency, but the main thing we care about is the allocative inefficiency, which is trades are not being made because people are sitting in their cars waiting for gas.
And that's inefficiency.
And that inefficiency arises because we have to allocate the gas somehow.
You can't get around that problem.
Remember, we are the dismal science.
We point out problems that cannot be surmounted.
You can't get around that problem.
That gas has to get allocated somehow.
And if you don't let the price mechanism allocate it, some other, more inefficient mechanism will arise to do so.
Now, the trade-off, of course, is then you keep the price low.
And the government's got to decide-- once again, this gets into the political economy of how the governments make these decisions.
And that's not really the point of this course.
But that's the kind of decision they have to make.
Now let's come-- and this will touch on your question-- oh, do you have a question about this?
I have a question, actually, about the minimum wage.
So isn't it a little imbalanced, because when people need work more than companies need people-- because, for example, there are a lot of instances when companies are paying hunger wages, a dollar a day.
[INAUDIBLE] they're not happy for it
Yeah, basically, in some sense, the bottom line is, equilibrium is where people are going to work for what the company's willing to pay.
Now, you can say, if people want work when the company wants to offer it, that's sort of a difficult subjective judgment.
But at the end of the day, if people are willing to work for a dollar and the company's willing to hire them for a dollar, then that's a trade which should be made.
Except, that's a great example to point out-- same person who just said that, well then, tell me what's the benefit of a minimum wage?
[INAUDIBLE]
It's equity, that thing economists like to not think about, because it's tricky.
It's fairness.
It's equity.
It's unfair that you would work for a dollar a day.
And people might be exploited and work for unfairly low wages.
Likewise, it's unfair that we pay a huge amount for gas.
So people, we should keep the wage high and the prices low to make it fair.
And that's the pro of the minimum wage.
Yeah
I didn't quite get [INAUDIBLE] it is giving a few people a higher wage, right?
But it's causing unemployment, so how is that better?
Wonderful point.
Wonderful point.
That's because in economics, there's the direct effect and the indirect effect.
And the direct effect is what voters understand.
And the indirect effect is what we understand.
In the case of gas, it was easy to see.
The question, why didn't the government realize this?
Everyone saw the direct effect, which was low prices, and indirect effect, which was long lines.
With a minimum wage, it's harder, because there's lots of reasons for unemployment, not just the minimum wage.
So the indirect effect is a lot harder to see.
If you're a politician, you're saying, look, I'll raise the minimum wage.
I'll make sure you're paid more.
Everybody goes, yay.
And then the economist says, well, you have to understand, according to this diagram, that would lead to unemployment.
People are like, whatever.
Shut up.
So basically, the point is that, basically, yes, you're right.
But for perceived equity, this is the case.
But you're right.
There's a trade-off.
Just like they recognize the trade between the price and the lines, there's a trade-off between the higher wage and the employment.
And that comes so what we'll talk about, empirical economics, which is measuring that trade-off.
Well, how big is that trade-off?
How much unemployment does a higher minimum wage cause?
In fact, we'll learn in about nine lectures that it actually doesn't cause that much unemployment.
It actually doesn't.
So maybe the trade-off isn't that bad.
But in principle, there's a trade-off.
Now, that comes to your point about shutting people's water off and things like that, and the government making sure that the suppliers make sure people's water doesn't get shut off.
Once again, one way to do that is just say to the water company, you can't shut people's water off, even if they don't pay their bills.
Well, that's going to mean the water company's going to lose money on those people.
That'll raise the cost of supplying water, and that will lead to the same kind of problems we've talked about
But if a price cap and a--
But if the government then says, OK, a price cap, and then we're going to pay you for the people who don't pay, that's kind of like the idea before.
You can have these countervailing interventions.
But then it starts to get messy.
You've got to raise the money to pay them, et cetera.
So that's why equity is so hard.
If it's efficiency, it's easy.
You just don't do anything.
With equity, it gets a lot harder, because government has to intervene to address equity, and then that causes other problems.
It causes these equity efficiency trade-offs.
And that's a lot of the problems it raises.
So now let's talk about one last example to stop.
I don't have a diagram for this one, but let's talk about the great example-- it's a real world example that happens a lot-- which is water shortages.
How many people here from California?
You guys know water shortages.
You guys know about how this works.
You guys know the drill.
Which is that there'll be a drought, and the government will say, you can only use x gallons per day.
You can't water your lawn or-- actually, let me ask the Californians.
So does the government monitor your meter or just tell you?
Is it enforced?
Tells us
Just tells you.
So the government says, you can't do this, you can't do that.
Maybe they enforce it.
Maybe they don't.
Whatever.
But the government comes in and says, look, there's not enough water.
So as a result, we are going to limit your use of water.
Yeah
They also have tiered pricing
OK, hold on.
Time out.
That's because the government got smart.
So let's go back 10 years ago.
So what they'll do is they'll come in, and they'll say, you can't use as much water.
Now, this has two problems, just like we talked about.
First of all, there are households which would happily pay more to make sure they got to use the same amount of water.
And those trades are not being made.
The first inefficiency.
Moreover, it's got the problem that you're not allocating the water appropriately.
Some guys are just dirty and don't like to shower.
Some guys are clean and care a lot about showering.
I want to allocate the water to the clean guys.
But the government typically doesn't do that.
It just says, don't use more than x gallons.
So there's an allocation inefficiency as well, where the people that value the water the most aren't getting it the most.
Now, tell me about tiered pricing
So it's like if you use 0 to 80 gallons per year or per month or whatever, you pay a certain price.
And then if you use 81 to 120, you pay extra on those gallons.
And then you pay even more down--
Exactly.
So what the government can do-- the right answer is that the government can use the price mechanism to deal with the shortage.
And the way it can do that is by saying, we're going to let the price increase.
We're going to let the price increase, and we are going to allow the price to be a function.
You see us pricing, but you make that a function of the underlying conditions and water supply.
So basically, you have a tiered pricing, and if there's a drought, the prices all go up.
So the government sets the price for water, and it says, look, the price this year will be higher if there's a drought.
And what that will do is that will make sure whoever wants the water gets the water and makes sure we allocate it to the people who need it the most.
Can anyone think of another way you can do this?
Never really worked in reality, but it's kind of a fun way to think about it.
Well, imagine that what I did is I said, every Californian gets a certain amount of water permits.
Every Californian gets-- I don't know how many gallons of water people use.
I don't know.
You get 1,000 gallon a year water permit.
How many gallons of water people use in a year?
I've got no idea.
1,000?
Let's say 1,000.
1,000 gallon a year water permit.
We're going to give you 1,000 pieces of paper.
Each one permits you to use a gallon of water.
Now, what we're going to let you do is trade those pieces of paper with your neighbors.
So what we're going to do is we're going to say, in normal times, basically, there's enough water, everyone gets 1,000.
Then we're going to say, when times are tight that basically, you can only use 900 gallons.
So what that's going to mean is that you're going to have to have permits that allow you to use those gallons.
And you're going to have to buy those permits off your neighbors.
So basically, we can allow neighbors to trade.
And the neighbor that says, I don't care about water so much-- so what's going to happen now is the government's going to say, this year, we only issue 900 gallon permits to each of you, instead of 1,000.
Now, you're a clean person.
You say, wait a second, I wanted to use 1,000 gallons.
So you go to your neighbor and say, look, you're dirty.
You don't need more than 800 gallons.
Sell me your extra hundred.
The government gave you 900.
I want my 1,000.
The government gave me 900.
Sell me your extra 100.
The neighbors says, sure.
And they set a price, and they sell to you, and it works out to be exactly the same outcome as if the government had used the proper pricing, because that secondary market-- somebody asked about black markets-- this secondary market can evade the government regulation.
So once again, the market equilibrium is very robust.
And if you can figure out a way to evade the government regulation, you will.
If there's some way with permits, if there's some way to trade-- now, in reality, you can't really trade water.
But if you could, then you could use a market mechanism to overcome this problem.
Yeah
Is that what-- one of the suggestions was for the global warming type thing, the carbon credits--
Exactly.
So global warming-- actually, I flew with Al Gore over to Kyoto in 1997 and negotiated the global warming treaty.
So something near and dear to my heart.
And with global warming, that's exactly the solution that's part of the Kyoto protocol, the framework that was set up at that meeting, which is that basically, there will be a certain limit on how much carbon dioxide can be emitted into the air.
But there'll be permits, and countries can actually trade across each other.
And the idea is, look, in the US, it's incredibly expensive to reduce the emissions we have.
Because what you have to do is you have to take a coal-fired plant and retrofit it to use natural gas instead.
Well, in China, they haven't built the plant yet.
They're building the plant.
And it's not that much more expensive to build it to be natural gas instead of coal.
So in China, it's pretty cheap to reduce emissions.
You just say, OK, I was going to build a coal.
I'll build a natural gas instead.
It's pretty much the same cost.
I've just reduced emissions.
So the idea is that we would trade with China, we would pollute more, China would pollute less, but the global total would be the same.
Now, as you hear that, you might think that's kind of controversial.
But in fact, it makes total sense.
Just like it might be controversial that dirty neighbors are selling water to clean neighbors.
You can imagine the newspaper articles, the outrage.
People forced to be dirty to make ends meet.
But that's not right.
What's right is you want to allocate to people for whom it's most efficient.
It is more efficient for China to reduce emissions because it's cheaper than for the US to reduce emissions.
So the efficient system, we say, here's the total amount of emissions we're going to reduce, and we're going to let China do extra reduction, or pollute less, and the US gets to continue our slovenly ways, and that's the efficient outcome.
One last thing on this note.
Many of you may know who Larry Summers is.
Larry Summers was actually my thesis adviser who then went on to be Secretary of the Treasury, had a somewhat failed stint as president of Harvard and is now Obama's top economic advisor.
Well, Larry Summers has gotten himself in trouble two times.
You all know about the time he said women are stupid.
But the other time he got himself in trouble was in 1990, he actually signed his name to a memo.
He didn't actually write it.
Signed his name to a memo saying that the efficient thing to do is we should ship all our garbage to poor countries.
And he said that's efficient because poor countries, what do they have?
They have lots of space and not much money.
What do we have?
Lots of money and not much space.
So the efficient thing to do is we should take our garbage and put on barges and send it to Africa.
He was right.
That is the efficient thing to do.
Africans would be better off, because they have lots of space they're not using, and they'd be richer.
The US would be better off, because we would be able to get rid of our garbage, and we're happy to pay to do it.
That's the trade to make it better off.
And yet, it got him fired, writing that memo.
That's why economics is the dismal science, because we point out things like this.
All right, let me stop there, and we'll come back on Wednesday and start talking in more detail about consumer demand.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
So today what we're going to do is continue our discussion of supply and demand.
This is sort of introduction week, if you will.
We've kind of talked about supply and demand, and you guys, rightly, immediately were on to where do those curves come from.
And that's what we'll start next week.
But what I want to do today is talk some more about what determines the shapes of supply and demand curves and just think about an overview of how we think about supply and demand interacting in a market and what determines how responsive individuals and firms are to prices.
And, once again, remember everyone should have a handout that you should have picked up in the back on your way in.
So everyone should have a handout.
What we talked about last time was the sort of qualitative effects, the qualitative version of the supply and demand model.
We talked about what happens when a supply curve shifts, what happens when a demand curve shifts.
We talked about how either a supply shock or a demand shock could lead to the price being increased.
But they could have very different effects on quantity, et cetera.
What we didn't talk about is how big these effects are.
I made up some numbers.
I threw them on the graphs.
But I didn't talk about where the size of those effects come from.
And where they come from is the shapes of the supply and demand curve.
And that's what we'll talk about today is what determines the shapes of supply and demand curves.
And that will be the focus of today's lecture.
I'll talk both theoretically about what determines these shapes and empirically about how economists go about figuring out the shapes of supply and demand curves.
So, to think about this, let's start with Figure 3-1, which is a standard market diagram we had last time.
With an initial equilibrium at point E1, with an initial price P1 and a quantity Q1.
That's the equilibrium that's stable.
Because at that price P1, consumers demand Q1 units, and suppliers are willing to provide Q1 units.
So that's a stable equilibrium.
Now we have some supply shift.
Last time we talked about somehow a pork-specific drought.
That leads the supply to shift inward.
So the supply curve rises to S2.
At that new price, initially, you would have excess demand.
But quickly the price increases to shut off that excess demand.
And you end up with a new equilibrium with a higher price, P2, and a lower quantity Q2, and new equilibrium point E2.
OK?
And we talked that through last time.
What I want to talk about this time is well, what determines the size of that shift from Q1 to Q2 and that price increase from P1 to P2?
What's going to determine it is the elasticity of supply and demand.
The elasticity of supply and demand is how much do supply and demand respond?
Do the quantities supplied and the quantities demanded respond when the price changes?
When we say, how elastic is demand, what we mean is how sensitive to price is the quantity demanded.
Or, alternatively, what is the slope of that demand curve?
So the slope of the demand curve will be the sensitivity of quantity demanded to the price consumers face.
And that will determine the market responsiveness.
In economics, it's always true that the best way to think about things is to go to extremes.
You have to remember that extremes don't exist in the real world.
But it's a useful teaching device to think about extremes.
So let's think about one extreme case in Figure 3-2.
Let's think about the case of perfectly inelastic demand.
Perfectly inelastic demand, that's where there's no elasticity of demand.
What that means is that demand for a good is unchanged regardless of the price.
So perfectly inelastic demand is a case where demand for the good is unchanged regardless of the price.
That would lead you to have a vertical demand curve at a given quantity.
What this says is regardless of the price, people always demand Q.
Can anyone tell me what would cause demand to be perfectly inelastic?
In what types of situations would demand be-- it's never perfectly inelastic-- would demand be relatively inelastic?
Yeah?
[INAUDIBLE PHRASE]
It's all about substitutes.
When there's no substitutes, when there's nowhere to go, it doesn't matter what the price is.
When there's no substitutes, demand will be perfectly inelastic, because you have to have Q.
It doesn't matter what the price is.
Because there's no substitute for that good.
So if you wanted amount Q of that good for any reason, you're always going to want that amount Q no matter the price.
So a perfectly inelastic good would have no substitutes.
So you'd always want Q no matter what.
Can anyone think of an example?
There's no perfectly inelastic good in the world.
But what sorts of goods?
Yeah?
Medicines
Medicines.
Now, not necessarily all medicines.
So give me an example of a medicine which would be more or less inelastic.
So I don't even need a medical name.
What sort of treatments?
Like heart attack maybe?
Yeah, something which is sort of lifesaving.
The best thing that we often use is insulin for diabetics.
Diabetics without getting that insulin to manage their diabetes will die.
That seems like that's something where there's not a whole lot of substitutes.
The substitute is dying.
So basically that's where demand is relatively inelastic.
Or a heart transplant, when you get a heart transplant or any kind of transplant, you have medicine you take so you don't reject the transplanted organ.
That sort of medicine demand should be very inelastic.
Elastic drug, well, our favorite example is always Viagra.
It's something where you'd think that you can probably survive without it.
And people would want less Viagra if you charged a lot more for it than if you charged less for it.
So elasticity is going to be about substitutability.
And that's going to determine inelastic demand.
Now, what happens with inelastic demand when there's a supply shock?
When supply increases, what happens?
Well, in that case, there can never be excess demand, because demand doesn't change.
So all that happens is price just increases.
If there's inelastic demand, and there's a supply shock, then all that happens is an increase in price and no change in quantity.
So with inelastic demand, quantity doesn't change for a price increase.
Price just goes up.
From a supply shock, prices just goes up.
Now, let's consider the opposite.
Let's look at Figure 3-3 and think about perfectly elastic demand.
Perfectly elastic demand is demand where consumers, essentially, don't care about the quantity.
They just care about the price.
That is, there are infinitely good substitutes.
A perfectly elastically demanded good would be one where there are, essentially, perfect substitutes.
An inelastic good is where there's no substitute.
A perfectly elastic good would be where there's perfect substitutes.
Technically, if a good is perfectly elastically demanded, then you are completely indifferent between that good and a substitute.
Well, if you're completely indifferent, then if the price changed at all, you would immediately switch.
And so the price can't change.
What's an example?
Once again, there's no good example of a perfectly elastic good.
Yeah?
Candy
What?
Candy
Candy.
OK.
So you've got your Wrigley's gum.
I like the sugar-free, minty gum.
You've got Orbit and Eclipse.
And I go to the store, and they're all pretty much the same price.
If Orbit was more than Eclipse, I just buy Eclipse.
They're the same.
They're minty gum.
It doesn't make a difference.
So basically the price is the same.
If there's a supply shock, I don't know, they're made with the same shit.
But let's say that Eclipse has some magic ingredient.
And let's say the Eclipse magic ingredient got more expensive, so the supply curve shifted up.
Well, Eclipse could not respond by raising its price.
Because I just switched to Orbit.
Or we often think of McDonald's and Burger King.
Now, they're less perfect substitutes, but pretty perfect.
If McDonald's started charging $10 for a hamburger, you wouldn't go there anymore.
You'd go to Burger King.
So if there's a supply shock to a provider that's facing a perfectly elastic demand curve, they cannot raise their price, because people will just switch.
So quantity will fall a lot.
Because if I'm supplying Eclipse gum, and it suddenly costs a lot more to produce Eclipse gum, but I can't raise my price, because I will lose all my business to Orbit, I'm just going to produce a lot less Eclipse.
Because I'm losing money now.
So with perfectly inelastic demand, the quantity didn't change.
With perfectly elastic demand, we saw a big quantity change.
So, more generally, what determines the quantity change in response to a price change is the elasticity.
More generally, we're between these two cases of perfectly elastic and perfectly inelastic.
And what's going to determine the price change is going to be the price elasticity of demand epsilon which is going to be the percentage change in quantity for each percentage change in price or, in calculus terms, dQ/dP.
So it's, basically, the percentage change in quantity for the percentage change in price.
So, for example, if for every 1% increase in price quantity falls 2%, that is a price elasticity of demand of minus 2.
The price elasticity of demand is the percentage change in quantity for the percentage change in price.
So inelastic demand is an epsilon of 0.
There is no change in quantity when price changes.
Perfectly elastic demand is an epsilon of negative infinity.
Any epsilon change in price leads to a negative infinite change in quantity.
Immediately, the quantity goes to 0 if you try to raise your price.
So the price elasticity of demand will typically be between 0 and negative infinity.
And the larger it is the more quantity will change when prices change.
Questions about that?
Yeah?
So that formula, shouldn't it be dQ/dP times P/Q because dQ/dP just refers to the change of the quantity with respect to price, not necessarily the percent change
Yeah, you're right.
I was trying to get too fancy with my calculus.
You're right.
Let's just stick with the non-calculus formula.
I never should deviate from my notes.
So let's just stick with the non-calculus formula.
OK, other questions about this?
OK.
So, basically, that's the elasticity.
That's going to be the elasticity.
Now, an interesting point about elasticity is now, we're not going to get into producer theory for a couple of lectures.
But as a little peek ahead about producer theory, let's think about how elasticity determines the money that producers make from selling their goods.
Well, if a producer sells Q goods at a price P, they make revenues R. Revenues are the price times the quantity.
The amount of money a producer makes when it sells goods, its revenues, this isn't its profits.
We're not having profits.
It's just the amount of money it makes, not the amount of money it takes home at the end of the day.
I'm ignoring the cost of making the goods.
The amount of total revenues it makes is price times quantity.
Well, we can then say that the change in revenues with respect to price is what?
It's Q plus dQ, plus delta Q-- let me put it this way to make my math clearer-- plus P times delta Q over delta P. That's how revenues change with respect to price.
Or, in other words, plugging in from the elasticity formula, delta R over delta P equals Q times 1 plus epsilon.
So, in other words, what this says is that if you're a producer, and you're trying to decide whether to raise your price, whether that will increase revenues, it all depends on the elasticity.
If the elasticity is between 0 and minus 1, then raising prices will raise revenues.
If the elasticity is greater than minus 1, then raising prices will lower revenues.
We're often faced with the issue of why did they charge this much for this good, or should they raise their prices or not raise their price.
Well, that's all about the elasticity of demand.
The elasticity of demand will determine whether they're going to make more money by raising their price or lose money by raising their price.
For Eclipse gum, their elasticity of demand is well above minus 1 in absolute value, so they're going to lose money by raising their price.
If they take the current level of Eclipse, for every penny they raise, they'll lose money.
For insulin, for every penny they raise, they'll make money.
And then you might say well, then how come the price of insulin isn't infinity and the price of Eclipse gum isn't zero?
Well, that's what we'll talk about in a few weeks.
Because it also depends on the costs of producing it.
But at the end of the day, that's what's going to determine the money that's made by producers when they change their prices.
Questions about that?
OK.
So now, that's how we think about the shape of supply and demand.
The shape of supply and demand is determined by these elasticities.
So now we have to get into OK, well, where do we get these elasticities from?
And that is the main topic of empirical economics which is estimating these kinds of elasticities, estimating these types of elasticities.
So one of the first distinctions I drew in the lectures is between theoretical economics and empirical economics.
Theoretical economics can tell us this is what a graph looks like and supply and demand.
Theoretical economics can't really tell us how big, for example, an elasticity is going to be.
It can tell us, there's more substitutes or less substitutes so we can rank them.
We know the elasticity for Eclipse gum has got to be higher than the elasticity for insulin.
But from the theoretical model, we can't say what the elasticity actually is.
To say what an elasticity actually is, we need to go to an empirical model.
We actually need to bring data to bear on the question.
And this is very difficult.
Because here we face the fundamental conundrum facing the empirical economist which is distinguishing causation from correlation.
And the whole guts of empirical economics is all about this question, distinguishing causation from correlation.
The classic story that illustrates this, it's due to my colleague, Frank Fisher, from a textbook many years ago, was the story of in ancient Russia there was a cholera outbreak, and many people were dying.
So the government decided to send doctors out to try to solve the problem.
And where there were more people sick, they sent more doctors.
Well, the peasants said, wait a second.
We observe that where there's more doctors, more people are dying from cholera.
So the doctors must be causing the cholera.
So they rose up and killed the doctors.
The peasants confused causation with correlation.
They thought that the fact that you saw more people dying where there's more doctors meant that doctors were causing the disease.
Clearly that's wrong.
That's why they were peasants.
But it's not just peasants that make this mistake.
For example, in 1988, Harvard University, our illustrious neighbor to the south, I guess, west, east, I don't know.
Which way is Harvard?
I don't know directions, down the street.
A Harvard University dean conducted an interview with a set of freshmen.
And they found that those that had taken SAT preparation courses-- now, you all took SAT preparation courses.
But in 1988, not everyone did.
Those who'd taken SAT preparation courses scored an average of 63 points lower-- this was back when the SAT was 1600 points-- 63 points lower on their SATs then those that had not taken preparation courses.
The dean concluded that preparation courses were unhelpful, and that the testing industry was preying on the insecurities of students to provide a useless service.
Why was the dean confusing causation with correlation?
What did the dean get wrong in drawing that conclusion?
Yeah?
What had probably happened is the students who got worse scores realized that they wanted to try and improve their scores by taking an SAT prep class.
So that's why there is a lower average score for the people who had taken the class
Generally, the people who needed the help the most took the most courses.
And so they had an underlying lower score.
So, in fact, you can't tell anything from the fact that the people who took the prep course scored worse.
It's just another excellent example of confusing causation with correlation.
And that's another example.
Another example I like quite a lot is studies of breastfeeding.
There are numbers of studies of breastfeeding, especially in developing countries, where they found that the longer children were breastfed the sicker they were.
So they concluded that breastfeeding was bad for kids.
Well, that's not the truth.
The truth is the sicker kids need to be breastfed more, because breastfeeding is actually good for kids.
And they just confused the causation with the correlation.
Now, these are all fun examples.
But the truth is this is a common mistake made by citizens, policy makers, everyone in the real world.
It's taking two things that move together and assuming one causes the other.
And this is the fundamental conundrum facing empirical economics in trying to address these kinds of things like measuring elasticities.
So to understand that, let's think about the issue of trying to estimate the elasticity of demand for pork.
Let's say you have the exciting job of estimating the elasticity of demand for pork.
That's your assignment.
Well, you say, wait a second.
What we learned in class, as shown in Figure 3-4, is that the price of pork can rise for very different reasons.
Figure 3-4, we start at an initial equilibrium like E1 with a quantity like Q1 and a price P1.
Now, imagine that there was a shift in demand, because the price of beef rose, remember?
The price of beef rose.
That shifted demand from D1 to D2.
What did that lead to?
A higher price and a higher quantity.
So if you took that diagram-- forget the supply shift for a minute, just imagine that's the change-- and you said, aha.
I can measure the elasticity.
I see here there's a change in price.
I can then look at how quantity changed.
And I'll get the elasticity right after all.
It's delta Q over Q or delta P over P. So I just look, and I take Q2 prime minus Q1 over Q1.
That's the percentage change in Q. I take P2 over P1 over P1.
That's the percentage change in price.
And what do I get?
A wrong signed elasticity is what I get.
I get a positive elasticity, because Q is going up and P is going up.
Why?
Because I'm confusing causation with correlation.
It's not the price change that caused quantity to change.
In fact, it's the opposite.
It's a taste shift, which caused quantity to increase which drove up the price.
It was a demand increase which caused the quantity demanded to increase which drove up the price.
So it's the quantity driving the price, not the price driving the quantity.
So if you looked at that simple example, as many people in the real world do, they'd say, hey, look.
Higher prices cause higher quantities.
You're getting the wrong answer.
Because you're confusing correlation which is the higher price is correlated with the higher quantity.
Because there was a common factor causing both of them which is the demand shift and not causation.
The higher price did not cause the higher quantity.
What do we need to do?
We need to distinguish why the price increased.
We need to distinguish why the price increased to measure this.
If, instead, we looked at a shift in supply such as the case that's shifting from S1 to S2 and moving the equilibrium from E1 to E2, then you would get the right answer.
Because then you'd say, look.
Something independent to consumers shifted up the price.
Some shock to the supply of pork shifted up the price.
And we saw that their quantity fell as a result.
What's the key?
The key is that to measure an elasticity of demand, you're measuring the slope of the demand curve.
So you need to shift along a demand curve, not shift the demand curve itself.
So if you look at this figure, what's the concept we want?
We want the slope of the demand curve.
Well, you get that by shifting from E1 to E2, because you shift along the demand curve.
So by looking at what happens to quantity as price rises from E1 to E2, you get the slope of the demand curve.
You get that delta Q over delta P you want.
But from E1 to E2 prime, you're not shifting along the demand curve.
You're actually measuring the elasticity of supply.
You're measuring the elasticity of supply.
You're shifting along a supply curve.
So you're actually answering a different question, a relevant question, but a different one.
That question is, what's the elasticity of supply?
How willing are pork producers to supply pork as the price goes up?
So it's the same delta Q over delta P. But here we did the elasticity of demand.
There's a corresponding elasticity of supply which is measured the same way.
It's delta Q over delta P, but it's for a different kind of shock.
It's what you get from moving along the supply curve.
So if we went from E1 to E2 prime, we can use that to measure the elasticity of supply or the slope of the supply curve.
And we do that if something shifts demand to move us along the supply curve.
From E1 to E2, we measure the elasticity of demand as something shifts supply and moves us along the demand curve.
So what we need to measure the elasticity of demand is something which shifts supply but does not, itself, affect demand.
And the best example of this that we use in economics, a great example, is government policy which comes along and changes the supply conditions for a good.
So, for example, let's think about a tax on pork.
So if you go to Figure 3-5, imagine the government came along and taxed pork.
The government comes along and taxes pork.
Let's think about what a tax on pork does.
The government comes along, and let's say the pork market is initially in equilibrium at $3.30 with 220 million kilograms of pork sold.
Now the government comes along and says that it's going to charge $1.05 in tax for every kilogram of pork.
So it's going to impose a tax of $1.05 per kilogram on producers.
So it's saying to producers of pork, for every kilogram of pork you sell, you have to send a check to the government for $1.05.
For every kilogram of pork you sell, you have to send a check to the government for $1.05.
Now, somebody talk me through how a supplier thinks through that.
How does a supplier react to that?
What do they think?
They're initially happy at E1 selling 220 million kilograms at $3.30.
What happens when the government comes in and says you have to pay $1.05 for every kilogram of pork you sell?
What happens?
Yeah
The producer decides that the current amount of money they have will not be able to buy as much inputs to create their products.
So they can produce less
Exactly.
So, in other words, the cost of producing just rose.
So what do they do?
So, in other words, what they say is look, effectively, if I was happy before selling 220 million kilograms at $3.30, to keep me equally happy selling 220 million kilograms, I'm going to have to raise the price.
We should add this to graph, actually.
If you draw a vertical line up for me, one to the S2 curve.
Draw a little dashed line up from the E1 to the S2 curve and then over.
That price intersection will be $4.35.
So in other words, if you want me to keep producing 220 million kilograms of pork, I'm going to have to get $4.35 a kilogram.
And you might say, what gives you the right to get that?
And it's not about rights.
It's about what producers are willing to do.
That same mathematics, that same supply curve that tells us they're willing to sell 220 million kilograms at $3.30 says, if you want them to keep selling 220 million kilograms but also pay $1.05 to the government, they're going to have to get $4.35 a kilogram.
So what happens is that's a supply shift.
And with the same reaction we saw last time with the drought, the price goes up, consumers demand less, and you reach a new equilibrium at the price E2.
You reach a new equilibrium where you sell 206 million kilograms for a price of $4.00.
So someone tell me how I use this example to find elasticity of demand.
Yeah
I guess you need to know that the change in price traveled along the demand curve.
So you know that it's not [INAUDIBLE PHRASE]
OK, so tell me.
You don't have to do the math in your head.
But how would I compute it?
You would take E1 and E2, and then you would do the price over the quantity change
Right, exactly.
So the quantity change delta Q over Q, is what?
It's minus 14 over 220.
It fell by 14 million kilograms over 220.
The price change, delta P over P, the price rose from $3.30 to $4.00.
So the price change is $0.70 over $3.30.
And using those, you end up with a price elasticity of minus 0.3.
Or, in other words, there's a 6.4% change in quantity.
This is minus 6.4% for a 21% change in price.
So quantity falls by 6.4% when price goes up by 21%.
That's a price elasticity of minus 0.3.
Or that's a relatively inelastic demand.
It's not perfectly inelastic, but it's relatively inelastic.
In other words, at that point, pork producers could make money by raising the price.
Now, you might say well, why didn't they?
That's something we'll discuss in a couple weeks.
But at that point, demand is relatively inelastic.
And you've got a convincing estimate, because you moved along that demand curve.
You used the supply shift.
Now, we're going to talk about taxation much, much later in the semester.
Let me just talk for one minute about what we learned from this graph.
What happens?
Well, the shaded area is the money the government raises from its tax.
The government has a tax of $1.05 at 206 million kilograms.
So it raises $1.05 times 206 million kilograms which is that shaded area.
There are two points to note the we'll come back to later in the semester.
The first point to note is the amount of money the government raises will depend directly on the elasticity of demand.
Can anyone tell me how much money the government would raise if you had a perfectly inelastic demand?
Yeah
[INAUDIBLE PHRASE]
Right.
If we think about this demand curve being perfectly flat, if we think about this demand curve being perfectly flat, then basically the producer can't charge any more for their good.
So it's going to depend on whether the producer is willing to sell at $1.05 less and how much less they're willing to sell.
If they're willing to sell a lot less, they're going to make a lot less money.
It's going to be where that second supply curve intersects a flat demand curve.
So that quantity is going to be a lot smaller.
We don't have it on the diagram.
But you see where that dashed line at $3.30 intersects S2, that's way to the left.
Quantity is going to fall a ton in this market.
When quantity falls, the government is going to raise a lot less money.
Because the government raises $1.05 on every unit sold at the end.
So if the government taxes very elastically demanded goods, it's going to raise less money.
If it taxes inelastically demanded goods like insulin, it's going to raise more money, because the quantity doesn't change.
Yeah
So cigarettes are relatively inelastic
Yes, exactly.
Cigarettes are relatively inelastic.
The elasticity is around minus 0.5.
So the government will actually raise money by raising the cigarette tax.
Those of us, as good liberals, think we should tax yachts.
Let's tax yachts.
Only rich guy have yachts.
The problem is yachts are incredibly elastically demanded.
So you raise a lot less money taxing yachts than you think.
Because guys buy fewer yachts, and you don't raise as much money as you think you would.
You still raise some, and it still may be worth it.
But you raise less than you think.
So that's one sort of observation about this.
It's basically how much money you'll raise will be a function of how elastic the demand is.
The other important observation to make is why it's actually hard for governments to figure out how much money they're going to raise for a tax.
Because, to figure it out, they need to know these elasticities.
That is, the naive thing to do would have been to say what?
Well, we're selling 220 million kilograms of pork.
That's $1.05.
We're going to tax each kilogram.
So that's 220 million times $1.05.
And that's how much money we raise.
Well, that's wrong, we know, because that assumes inelastic demand.
If demand's elastic, they'll raise less than that.
Well, if we want to figure out how much a government is going to raise from a tax, they've got to know what these elasticities are.
And those are actually pretty hard things to know.
So that's why there's uncertainty.
That's why when politicians will say, this tax will raise x and you'll hear the New York Times report, the tax will raise x, that is a guess.
Those are guesses, because they depend on our best estimate of the key elasticities that determine how people respond.
Yeah
But in Washington you have tax cuts that raise money
Well, some claim you do.
You don't actually.
But some claim you have tax cuts that raise money.
That's because they think the elasticity is very large.
If the elasticity is large enough, a tax cut can raise money.
So, basically, that's all about that some people think that elasticities are large enough that tax cuts can raise money.
Those people are wrong.
But that's what they claim.
Yeah
[INAUDIBLE PHRASE]
Yes.
Excellent point.
You'll go through that in section on Friday.
So what I've done is I've done an example of a constant elasticity curve.
Actually, I've done something here which is logically inconsistent.
This curve is linear which means it can't be constant elasticity.
If it's constant elasticity, it would have to curve.
So what I've estimated here is a local elasticity.
I have estimated the elasticity around that price change.
But the elasticity, if this curve is true, would be different at different points on this curve.
If the elasticity is going to be constant all over the curve, and you're going to do a constant elasticity of demand, that's going to be a curve that bends, not a linear curve.
So a linear demand curve is not constant elasticity of demand.
We will typically ignore that issue and focus on local elasticities.
But that is an important issue.
We'll discuss that in section on Friday, the difference between constant elasticity of demand curves and linear demand curves.
But, typically, we're think about local changes.
So if it's local enough, it doesn't really matter.
But, for a broad change, it will matter what the shape of the curve is.
Good point.
Other questions?
OK. Let me then turn to another problem we face in empirical economics.
So this is an example of a problem we're facing in empirical economics.
Let me turn to an example of another problem we face in empirical economics estimating elasticities.
It is that individuals often choose the price they face.
Individuals, typically, often don't just face a price that's given to them.
And then you can say, OK, they're given a price, and we see how they respond.
They often choose the price they face.
Let me explain what I mean by that.
A classic example of an elasticity that matters a lot for policies is the elasticity of demand for medical care, the elasticity of demand for medical care.
That is how much less medical care will you use if you have to pay for it?
So, for example, most of us have insurance through MIT or maybe through our parents.
And the way health insurance works is you pay a certain amount per month or your parents do, and, in return, that health insurance covers the cost of your medical care, most of it.
But, typically, you have to pay some of it.
So how many people have gone to the doctor in the last six months?
Did you have to pay something?
How much did you pay?
Did you pay a copayment?
No?
None of you?
Yeah.
How much did you pay?
I think like $20
$20, $10, $5, that's what's called the copayment, or $0.
Most insurance these days has what's called copayments.
A copayment is what you pay when you go to the doctor.
Insurance picks up the rest.
You don't know.
You didn't know how much the whole doctor visit cost.
You just went, you gave them your card.
They said your copayment is $20.
You gave them $20.
You don't know.
The visit might have cost $100, $200, $500, $1,000.
You don't know.
Your insurer picks up the rest.
You pay the copayment.
Copayments are rapidly on the rise in health insurance.
There's a rapid rise in copayments.
Increasingly, insurers are saying, look, health care costs are out of control.
One way we're going to combat them is by making people bear more of the cost that they use.
I could go on forever about how I'm a health care economist.
I could go on about health care forever.
But just to fix ideas on why this is an issue, in 1950, the US economy spent 5% of our gross domestic product, 5% of our size of the economy went to health care.
Today it's 17%.
By 2075, it's projected to be 40%.
That is of every dollar that's made in America, $0.40 will go to medical care.
By 100 years later, it's about 100%.
Literally, if we do nothing, the entire economy will be health care.
Obviously, that can't happen.
We've got to deal with this.
And one way that insurers and some policy makers are saying we need to deal with this is we need to make consumers bear more of the costs of their medical care.
We need to make consumers pay more when they go to the doctor, so that they understand the consequences of their decision.
Well, if we're going to do that, a key question we need to know is well, does it affect their behavior?
If we make consumers pay more, and it doesn't at all affect their demand for medical care-- it's just a tax on them, essentially-- then that's different than if it causes them to use less medical care.
It may be good, may be bad.
We'll come back to that.
But the key empirical question is what is the elasticity of demand for medical care?
If you pay $20 and you pay $0, how much less like are you to use the doctor when you pay $20 versus when you pay $0.
Well, we can all introspect this and think about it.
But, in fact, to answer this we have to go to the data and ask, well, what's the difference?
So people, for many years, went to the data.
And they said, look, there's all sorts of differences out there across people and what they pay for their copayments.
Some people have insurance where they pay nothing, some where they have $20.
Some people have what they call high-deductible plans.
A deductible plan is where you pay the full cost of your visits until you reach some limit.
So a $2000 deductible plan will be one where you pay all of your medical costs until you've spent $2,000.
It's a big copayment.
So we look across those people, and people did.
And they found, look, the people that have plans where they spend more for health care, where they have a high copayment, use a lot less health care than where they don't have to spend anything.
The elasticity of demand looks very, very high.
What is wrong with those studies?
What is wrong with the conclusion those people drew?
They drew it by comparing people who had plans where they paid a lot to go to the doctor, and therefore use a lot less care to people who didn't pay anything when they went to the doctor and used a lot more care.
I pick the $20 person, because I picked on you already
Probably they chose to have a high-deductible plan, because they don't often go to the doctor already
The rational choice, if you're young and healthy, for almost everyone in this room, is going to be a very high-deductible, high copayment plan.
Because it will cost you less money, because the insurer is shifting the money to you.
But you don't use the doctor anyway.
So who cares?
So the healthier people are going to choose the plans where they pay more.
So, of course, you're going to find in the plans where people pay more they use less medical care.
But is it because they're paying more, or is it because healthy guys choose those plans?
It's causation versus correlation.
We don't know.
Well, how can we figure that out?
Well, if we were doctors, what we'd do-- real doctors, not a doctor like me, a real doctor, a medical doctor-- what we'd do is we'd run a randomized trial.
So if doctors want to figure out whether a drug works or not, they don't just look at guys who take the drug versus guys who don't.
They run a randomized trial.
They randomly assign some people to take the drug and some people not.
Now, when you run a randomized trial, by definition, you get a causal effect.
Well, this room isn't quite big enough.
We all know the law of large numbers.
But imagine there were four times as many people in this room or five times as many people in this room.
OK?
And I had you come up to the front.
I flipped a coin and said half of you are going to take the drug, and half of you are not, randomly by the flip of a coin.
Then, by definition, any statistically noticeable differences I get between the group the takes the drug and the group that doesn't is caused by the drug.
And how do I know that?
Because I know the groups are otherwise identical by the law of large numbers.
By the law of large numbers, I know that as long as I have enough people, they're identical.
So if the only difference between them is that one's taking the drug and one's not, that's a randomized trial.
That would be how I could solve the causation versus correlation problem.
In medicine, thousands of randomized trials every day are being run.
In fact, the FDA, before it will approve a drug, will typically require a randomized trial.
Well, in the social sciences, it's harder to run randomized trials.
Because we're actually trying to understand things like people's demand for medical care, not whether a drug works or not.
But, in fact, one of the most famous social randomized trials in history was called the RAND Health Insurance Experiment run in the 1970s.
This is where some innovative health economists who understood this problem that we laid out about the fact that you can't just compare more or less generous health insurance policies, actually randomized health insurance policies across people.
They recruited volunteers, and they literally said, we're going to randomize.
Some people are going to have policies where the health care is free, and some people are going to have policies where they have to pay, essentially, all the costs of health care.
So they, essentially, randomized across these different groups.
And, therefore, they can assess what the price elasticity was.
Because they knew the price difference between groups.
For one, the price was zero.
For one, the price was one.
They actually had a range of prices they varied it across.
They could look at the quantity response, and they knew that was a quantity response to the price, because people weren't choosing their prices.
The prices were being assigned to them.
What did they find?
Well, they found that medical demand is elastic, although not as elastic as the previous study.
It's somewhat elastic.
It's not as elastic as the previous studies found.
They found that the elasticity of demand for medical care is around minus 0.2.
So when the price goes up, people use less medical care but not that much less.
Now, let's be clear.
Remember what elasticity is.
That delta Q over Q.
The same study showed that if you take someone who paid nothing and make them pay almost everything, their utilization of medical care falls by 45%.
That's consistent with that small elasticity.
Because that's a huge delta P, percent delta P. So, basically, that's comes to the question about local versus global elasticities.
So it's not saying that prices don't matter.
But it's not a very, very elastically demanded good.
So that's how they measure that price of elasticity of demand.
That experiment, which was run over 35 years ago now, that result drives much of what we do in health policy.
So a lot of the estimates that we saw for the recently passed health reform bill derived from how do we get that estimate.
We'll have to figure out how people are going to respond with their medical care when we give them health insurance.
The recently passed health care bill just gave 32 million people health insurance.
Well, how are they going to respond to having health insurance?
We go back to the RAND estimates and say, well, we have this elasticity of demand.
We know what we're doing to the price.
We figure out how much medical care is going to go up.
But here's the other thing.
Here's the question in the lecture that that we'll close with.
Is that a good thing or a bad thing that medical care fell when the price went up?
And how would we tell whether it's a good thing or a bad thing?
So we know when we raise the price, people use less medical care.
How can we tell if that's a good thing or a bad thing?
In the same experiment, how could we tell?
What could we do?
Yeah
Maybe you'll get death rates or like--
You look at their health.
You say, look, the same trial can answer a different question.
We know that when you charge someone for health care, they use less.
Well, are they sicker?
The answer, not at all.
People use less health and were no sicker.
Why?
Because we waste a huge amount of health care in the US.
A huge amount of health care is wasted.
So, in fact, we could cut back quite a lot on health care, and we'd be no sicker.
And that's what the RAND experiment showed, that we can charge people to use medical providers.
And they'll use less medical care, and they won't be sicker as a result.
Which suggests that, actually, as we try to think about getting our health care costs under control in America, making people pay something to go to the doctor is not a crazy thing to be thinking about.
How much?
Well that depends on efficiency versus equity.
We can't make someone who has no income pay $1,000 to go to the doctor.
That, clearly, is a mistake.
But we can take a rich guy like me and make me pay $50 to go to the doctor.
There's no reason not to do that.
So, basically, that's a lesson of how you can use elasticity of demand to help inform the kind of policies we need to make.
OK, let me stop there.
By the way, if you at all find this stuff interesting, and you haven't yet read Freakonomics-- how many of you have read Freakonomics?
That's amazing.
OK.
If you haven't read Freakonomics, you should.
It's a great book.
If you're lazy, the movie is coming out.
And Freakonomics the movie is premiering on Friday the 30th at LSC.
So if you're interested in learning more about empirical tools in economics, you can watch Freakonomics the movie on Friday the 30th.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at MIT.edu
We've been talking so far about basically overviews of supply and demand relationships and understanding how markets work.
Now we're going to step back and get behind the supply and demand curves and understand where those curves themselves come from.
So we talked about, given that we have supply and demand curves, how they interact.
Now we're going to get behind that and see where these curves actually come from.
Thank you, by the way for coming down.
I appreciate it.
So what we're going to do is, we're going to start with the demand curve, and we're going to spend the next few lectures talking about consumers and how consumer preferences are ultimately what leads to the construction of the demand curve.
Then after that and after the first exam-- that will cover what's on the first exam-- after the first exam, we'll start talking about firms and what determines the firm supply curve.
So today we'll talk about consumers and we're going to talk about where the demand curve comes from.
And where it comes from, and where all consumer behavior coming from in economics is from utility maximization.
That's where everything with consumers starts is with utility maximization.
That's the basic building block of consumer behavior.
And basically, utility maximization-- that's what this lecture will be about, describing it.
But basically, an overview is, we posit some type of preferences.
We posit consumer preferences, what consumers would like.
We posit some budget constraint, what resources consumers have to get what they'd like.
And then we do a constrained maximization problem that says, given your preferences, given what you'd like, subject to the resources you have available, what choices will you make?
And in particular, we're going to ask, the term we'll use is we'll ask what bundle of goods makes you the best off?
Given your preference, given your constraints, what bundle of goods?
So think about consumers choosing across a set of goods.
Typically, we'll think about two goods because graphs are easier to think about two dimensions than more.
So we'll typically think about trading off two goods.
So think about consumers with preferences across two goods, some budget they can allocate, and how they make those choices.
But this basic framework applies to the multiplicity of choices we all make along many, many dimensions.
So doing two dimensions as one of the simplifying assumptions I'll talk about.
But that's just a simplifying assumption.
So basically what we we're going to do is we're going to go through this in three steps, not just in this lecture, but over the next few lectures.
Step one, is we're going to talk about what assumptions we make about preferences.
So I'll talk today about preference assumptions.
So the axioms that underlie how economists model consumer preferences.
We'll then talk about how we translate these preferences assumptions into mathematical tractability through the use of the utility function, which is basically a mathematical representation of underlying consumer preferences.
So we'll talk about how we basically take these preferences and translate them into something that we can work with here at MIT by making it mathematical, by making a utility function.
And then finally, we'll talk about budget constraints.
And armed with these three things, we'll then be able to model how consumers make decisions.
Now, importantly for today's lecture, we are not dealing with budget constraints.
So this is not happening today.
So today we're not going to worry about the budget constraints.
Today we're in a world where we're just going to talk about what people want and we're going to put out of our mind whether or not they can afford it.
So just talk about people want, and we'll put out of mind for today.
We'll come back next time to whether they can afford it.
We're just going to think about unconstrained preferences for today's lecture.
So let's talk about our preference assumptions.
So, to model consumers' preferences across goods, we're going to impose three preference assumptions.
Three preference assumptions.
Assumption one-- now once again, let me remind you from the first lecture, this is getting to some of the harder material.
I'm going to write messily and talk quickly, so stop me if anything is unclear.
And if you don't stop me, I'll just go faster and faster until I explode.
So basically, feel free to interrupt and stop me with questions and such.
Three assumptions on preferences.
The first assumption is completeness.
The first assumption is the assumption of completeness.
When comparing two bundles of goods, you prefer one or the other, but you don't value them equally.
OK when comparing two bundles of goods, you prefer one or you prefer the other, but you're not indifferent.
Completeness is the same as no indifference.
So what we're saying is whenever I offer you two bundles of goods, you could always tell me what you like better.
Now it could be infinitesimally better.
I'm not saying you have to have strong preferences.
But you cannot say I'm indifferent.
You can never be purely indifferent.
There always at least some slight preference for one bundle of goods over another.
That's the completeness assumption.
This is an assumption we make.
Now in reality, oftentimes we are indifferent.
Well once again, this is one of these simplifying assumptions that will make the model work.
And in fact, in reality if forced, you can always decide whether you like one thing better than another, we just often follow heuristic rules which say we're roughly indifferent.
We're just going to say, more precisely, you are never purely indifferent.
So, I'm not sure is not an option.
You can never say I don't know, I don't know which I prefer, I'm indifferent.
I'm sorry, let me back up.
I'm using the wrong word.
Forget I said indifferent, because we'll want to use that word in a different context later.
You can't say, I'm not sure.
You can't say, I'm not sure.
You can't say, I'm not sure, can't say I don't know, I don't know how I feel about that.
Scratch what I said a few minutes ago, because I want to use indifference differently.
Completeness is not about not being different, we're going to use that.
What I'm saying is it's about not being sure.
You've got to value every bundle of goods.
You've got to be willing to value every bundle of goods that's given you.
So you can't say that I don't know, I don't know how I feel about that.
You've got to have some feeling about stuff.
You can't say I'm not sure.
You've got to have a complete set of preferences over all bundles of goods that are given you.
OK, that's completeness.
The second is transitivity.
Which is something we've been learning since kindergarten about transitivity, right?
And also, it's a different context.
That's just if you prefer x to y, and y to z, you've got to prefer x to z. OK, you guys should do transitivity in your sleep by now.
OK, so the standard transitivity we always assume in math class, we're going to assume here as well.
OK, that should be pretty noncontroversial.
OK, and then finally, and probably most controversial, is we're going to assume non-satiation.
Or the famous economic assumption that more is always better.
OK. More is always better, that is, you never would turn down having more.
Now we're going to talk later today and tomorrow about why you might not like the next unit as much as you like the current unit.
But you'll always like it greater than zero.
You're always happy to have more.
You never say, I've had enough, I literally value at zero the next unit.
You may value it as epsilon, but you'll always value it as greater than zero.
That's the non-satiation assumption, more is always better.
Now, this is the most controversial.
And obviously we can think of many contexts in which that's not true.
But if we don't allow for this assumption, the modeling gets a lot trickier.
So once again, let's put it out of our mind.
Realistically, we know once we've eaten a certain amount, we literally do not want any more.
OK, so we're going to put that aside.
Assume we're always in a space where we can always eat a little bit more.
OK, we'll call it the Jewish mother space.
OK, you can always eat a little bit more.
OK, you can always eat a little bit more.
We're just going to assume we're in that space for now.
OK. And so, for large ranges, we can see it is not an unreasonable assumption.
Although, I think in extremes, you could see this becomes unreasonable.
OK, so those are assumptions.
Completeness, which once again, I screwed up in describing.
Come back to the second way I described it, which means you can't say you're not sure.
You always have preferences over things.
That doesn't seem unreasonable.
Transitivity which we've been living with since we were kindergartners.
And non-satiation, which could be a little controversial, but we'll live with it for now.
Now given these, we're going to talk about the properties of what we call indifference curves.
This is why I screwed up before.
Of course you can be indifferent between things.
That's the whole point of economics.
I don't know why I got that wrong.
I haven't taught this course about six years, so I lost track of things.
Properties of indifference curves.
So indifference curves are our name for what you could also think of as preference maps.
In economics, we like to be able to describe everything, as I said, three ways, intuitively, graphically, and mathematically.
Preference maps are the graphical representation of people's preferences which we do through graphics that we call indifference curves.
So now let's go to the example I'm going to use that I'm going to use throughout these next couple lectures of a decision you have to make.
Now I tried to think of a cool way to make this example cool, and I just couldn't.
So its going to be a boring example.
It's going to be, imagine your parents gave you some money and you had to decide whether to buy pizza or see movies.
I tried to make it at least a little bit relevant even if I couldn't make it cool.
You've got to decide whether to buy pizza or see movies.
That's your decision.
That's the trade-off you're making.
We're in a world with only two goods, pizza and movies.
And you're deciding how to allocate the money your parents gave you over pizza and movies.
Now let's say we're going to consider three choices of pizza and movies.
So go to figure 4-1a.
We're going to consider, you could have two pizzas and one movie, that's point A.
You could have one pizza and two movies, that's point B.
Or you can have two of both, that's point C. That's just three choices you're facing.
Once again, we're ignoring paying for them.
Budget constraints is next time.
Now we're just saying I'm giving these three choices.
Well how do you feel about them?
Well let's assume that you're indifferent-- and this is why you can be indifferent.
What I said before, just strike.
Let's say you're indifferent between two pizzas and one movie, and one pizza and two movies.
Let's say, if you had two pizzas and one movie, or one pizza and two movies, you pretty much feel the same about them.
But clearly you like two pizzas and two movies better than either of the first two combinations.
Then what we can do is we can draw what we call indifference curves.
And that's in figure 4-1b.
These are maps of your preferences.
An indifference curve is the curve showing all combinations of consumption along which the individual is indifferent.
And I'll say that again, very important concept.
An indifference curve is a curve showing all combinations of consumption along which an individual is indifferent.
So you have an indifference curve.
I said you were indifferent between A and B.
So you have an indifference curve that runs between A and B.
That means that all, and I'm assuming that all combinations along this curve, you're indifferent.
So you're equally happy getting two pizzas and one movie or one pizza and two movies.
But point C, which is two pizzas and two movies is on a different indifference curve.
You're not indifferent between point C and points A and B.
You're indifferent between A and B-- I'm just assuming this, I'm not saying you are.
But I'm just assuming, let's imagine you are.
But you clearly like two pizzas and two movies better than one of one and two of the other.
Yeah?
Does that break the completeness rule for the--
Does that break it?
Why would that break it?
Do you prefer pizza over movies or movies over pizza?
No.
Because this is my screw up before.
Completeness just means you know how you feel about everything.
So strike from the record my initial description.
Completeness means you just know how you feel about everything.
You're allowed to be indifferent.
Completeness just means you can't say, I don't know, I don't know how I feel about pizza.
You've got to have feelings for pizza.
OK. You've got to know how you feel about stuff.
That's what completeness is.
So armed with those assumptions, there are four key properties of indifference curves that we have to keep track of.
Four key properties of indifference curves.
The first is that consumers prefer higher indifference curves.
So you prefer higher indifference curves.
Prefer higher indifference curves.
What I mean by that is, the further out the indifference curve, the more you prefer it.
And this comes naturally from the non-satiation assumption.
Given that we've assumed non-satiation, you must always prefer an indifference curve that's further from the origin because it's more, and more is better.
OK so given non-satiation, you will always prefer an indifference curves that are further from the origin.
That follows directly from non-satiation.
The second point is that indifference curves are always downward sloping.
Indifference curves are always downward sloping.
Indifference curves are always downward sloping.
And that, once again, comes from non-satiation.
To see this, let's look at the next figure, an upward sloping indifference curve.
Why does an upward sloping indifference curve, someone tell me, violate non-satiation.
Yeah?
Because you're indifferent to getting more
Yeah.
Because this would say you're indifferent between (1,1) and (2,2).
It's not quite drawn right.
We ought to just have this go through to point (2,2).
But basically, this would say you're indifferent between getting one pizza and one movie or two pizzas and two movies.
You can't be because that violates more is better.
So indifference curves can't be upward sloping, they've got to be downward sloping by the non-satiation assumption.
OK, that's the second property of indifference curves.
The third property of indifference curves is indifference curves cannot cross.
Indifference curves cannot cross.
Why can't indifference curves cross?
Well here I forgot to have Jessica do a pretty diagram, so you'll have to deal with my ugly handwriting here.
So why can't indifferent curves cross?
Well imagine a situation where you have your pizza and your movies.
And imagine a situation where you have one indifference curve that looks like this, and one indifference curve that looks like this.
OK, two indifference curves.
And you've got, let's label these points A, B, and C. Now could someone give me, based on the properties of indifference curves that we talked about over here, given these three properties, can someone tell me why this is a violation?
Yeah?
Because A and B are on the same curve, meaning you're indifferent between A and B.
A and C are also on the same curve because you're indifferent between the two.
But that means you're also indifferent between B and C which can't be true because more is better
Exactly.
So transitivity says I must then be indifferent between B and C through the logic you just laid out.
But I can't be indifferent between B and C because B dominates C. B has a basically the same number of movies, but more pizza, so I must like B better.
So by the combination of transitivity and non-satiation indifference curves can't cross.
And finally, completeness, which is the most awkward of these assumptions, it simply means you can't have more than one indifference curve through a point.
So basically, the idea of every possible bundle has one indifference curve.
You can't have two indifference curves through it sayin, I'm not sure which indifference curve I'm on.
I'm not sure how I feel about this.
You know how you feel.
There's one indifference curve through every bundle.
There's not two indifference curves through a bundle.
So this is the way we think about preference maps which is the sort of core building block of utility theory.
Now I was an undergrad here, took this course, but I never really understood indifference curves until I had a year off with a grad student who was trying to decide where to take a job and he did it through just showing me an indifference map.
He said look, I'm trying to decide where to take a job, and I care about two things.
I care about how good the place is and where it is.
So he said here, he had location and he had academic rank.
And he said look, I'm indifferent between Princeton which has a shitty location but a wonderful academic rank.
I'm from New Jersey, but it's still a shitty location.
OK, and Santa Cruz.
And Santa Cruz which has not such a good academic reputation, but a pretty awesome location.
And he said here's my indifference map.
And where did he end up going?
He ended up going to the IMF, the international monetary fund in DC which had a better location than Princeton-- worse than Santa Cruz, but a better reputation than Santa Cruz and worse than Princeton.
So he decided he was indifferent along this map, and he ended up choosing a point in the middle.
But indifference curves are just a way of representing two dimensional choices.
Now very few choice in life are really two dimensional, but that's a nice example.
Question in the back?
I was wondering if IMF, the point would be actually not on the curve, but further out?
If it were further out.
A great question.
So imagine if IMF were here.
What should he have done?
Definitely go to IMF.
Here he was indifferent.
He could flip a coin and be equally happy at all three.
But if IMF were out here, and maybe it was because that's what he chose.
That's a good point.
I don't know if IMF was here or here.
The fact that he chose IMF, it can reveal it wasn't anywhere in here.
It's a very good point actually.
It can reveal it wasn't anywhere in here.
That we know.
But I can't tell if it was on the curve or outside the curve.
It could have been on the curve because he's indifferent, so who knows, he could have flipped the coin.
Or it could have been outside the curve because it's better.
We can't tell that.
That's a good point.
All right.
So that's a preference map.
That's indifference curves.
Now let's step from indifference curves, which is a building block of preferences, to utility.
Now everything you need to know about preferences is represented in those indifference maps.
The problem is they're pretty awkward to work with when we need to actually prove theorems and solve and understand how people make decisions.
That's a lot easier if we have a mathematical representation of those preference maps.
And that's the utility function So the utility function is a mathematical representation of preferences.
That's all it is.
You're going to be hearing this term in your nightmares for the next semester.
Utility functions.
But remember, it's just a mathematical representation of people's underlying preferences.
Don't be scared of it.
And the key thing is that we assume individuals have these well-defined utility functions, and by maximizing those utility functions we can tell what choices they're going to make.
So for example, suppose that I said that your utility function over pizza and movies was the square root of pizza times movies.
That's a utility function.
I'm going to say, what the hell does that mean?
Well, it doesn't mean anything, it's a utility function.
It's your preferences.
It's a mathematical representation of your preferences.
What does that mean?
What it means is-- it doesn't mean anything inherently, but it tells us about your preferences.
What it tells us is that your preferences can be represented.
If you flip back to figure 4-1b, it tells us those are your preferences because you're indifferent between two pizzas and one movie and one pizza and two movies.
Of course you're indifferent.
They both give a utility square root two.
But you prefer two pizzas and two movies because that gives a utility of two.
So this is a mathematical representation consistent with those utility indifference curves.
Not the only one.
There's other mathematical representations that could be consistent with those indifference curves.
But let's posit that this is your utility function.
This is a mathematical representation of your tastes.
Now what does utility mean?
Utility means nothing in the sense that it is not a cardinal concept.
It's only an ordinal concept.
So if I say to you that you get two utils from two pizzas and two movies, that doesn't mean anything.
It just means that you get more than from one pizza and one movie.
And we can even get the ratio that you get square root of two more, than you get from one pizza and two movies.
We can do ranking and ordinality, but we can't assign cardinality.
I can't say how happy you are in some abstract absolute sense from two pizzas and one movie.
I can't give a cardinal form preference.
But this is an ordinal ranking of preferences.
I can tell what you like better than what else.
That's why utility function is a representation of indifference maps.
They're just a mathematical tool for comparing bundles, they're not some inner answer to the value of your soul or something like that.
Don't imbue these with too much magic.
They're just mathematical ways of representing preferences.
The key concept, the single most important concept, for consumer theory for understanding how consumers make decisions is the concept of marginal utility.
We'll talk a lot this semester about marginal this and marginal that.
And this is our first example.
Marginal utility.
That is how your utility changes with each additional unit of the good, or the derivative of the utility function.
If you want to do it in calculus terms, marginal utility is the derivative of your utility function with respect to one of the inputs.
But if you don't want to put it in calculus terms, it's as you add each unit of one of the elements of the utility function, how does utility change So to see this, let's do an example of marginal utility.
Imagine for a moment that you have two pizzas, p equals two.
You've got two pizzas, they're there.
Your roommate's got them or something.
OK, now I want to ask, how does your utility change as you see additional movies?
And to show that, let's look at figure 4-3 which isn't here.
Whoops.
There's no figure 4-3.
Do you got that figure 4-3?
There was never any figure 4-3
There was never any figure 4-3.
So let's go to 4-5.
So basically--
Figure 4-4?
No but-- actually fine.
4-4.
So basically what this is showing, what figure 4-4 is showing, is it showing how-- no actually, let's go to 4-5.
They're out of order.
Let's go to 4-5.
What 4-5 is showing-- no, that's not going to work.
OK, back to 4-4.
What figure 4-4 is showing, is it's showing how your marginal utility for movies evolves, how your utility evolves as you get more movies.
Given that you have two pizzas, this is the evolution of your utility as you get more movies.
So each additional movie increases your utility.
The slope is positive.
By more is better, we know that.
Even if it's some date movie, it still improves your utility.
So it still improves your utility, but at a diminishing rate.
And that's the key is that we assume diminishing marginal utility.
The key assumption underlies everything we'll do for consumers is diminishing marginal utility.
We assume that additional movie increases your utility, but at an ever diminishing rate.
So basically, we can actually graph your margins.
And that's what figure 4-5 is, is a graph of your marginal utility.
So basically, when you have two pizzas and one movie, utility is square root of 2, right?
Now what I'm saying is if you get one more movie, your utility is going to rise from square root of 2 to 2.
So the marginal utility of that next movie -- is that right?
Two movies.
1.4.
Yeah, it's going to rise by the square root of 2.
You're going to multiply your utility by the square root of two, so your marginal utility-- you're going to go from the utility of square root of 2 to utility of two.
So utility is going to increase by the square root of 2.
Utility is going to increase-- I'm doing this wrong, hold on.
One second.
From one movie.
I see.
I see.
So, I'm sorry.
This isn't the delta, this is the level of marginal utility.
So I'm graphing the actual level of marginal utility.
Back up.
OK, so I'm graphing the actual level of marginal utility.
So when you have two pizzas and one movie, your marginal utility, your actual utility-- I see, that's what this is.
This is the actual utility I'm graphing.
So I told you a minute ago, we can't measure utility as a cardinal concept, but actually here I'm doing it anyway because it's to illustrate marginal utility.
So your utility, OK.
When you have one movie is 1.4, square root of 2.
That's your utility.
Now when you move from one movie to a second movie, your utility goes up from square root of 2 to 2.
Your utility goes up by 0.6.
So the marginal utility of that second movie is 0.6.
Utility was 1.4, was a square root of 2.
Now it's increased to 2.
So the marginal utility of the first movie is 0.6.
Now let's say you add another movie, you go to three movies.
What's your utility now?
It's the square root of 6.
So it's gone from 4, to the square root of 6, which is 2.45.
So your marginal utility of the third movie is 0.45.
This graph is messed up because the first one is an actual utility level.
So the first one I say, for one movie, you have a utility 1.4.
And then for the second movie, I give the marginal utility, the third movie marginal utility.
So, this graph sort of-- yeah?
It shows the marginal utility of the very first movie
Yeah, I guess that's right because you're zero.
You're zero movies.
OK, right.
You're right.
OK, so the first one is the marginal utility of the very first movie, you're right.
So the very first movie gives you marginal utility of 1.4 because you go from 0 to square root of 2.
That's right.
My bad.
So you go from 0 to square root of 2 to get a marginal utility of 1.4 for the first movie From square root of 2 to 2, you get 0.6 the next movie.
From 2 to square root of 6, you get 0.45 for the third movie.
For square root of 6 to square root of 8, you only get 0.38 from the fourth movie, and so on.
So the key point is that these marginal utilities are ever decreasing.
Each additional movie gives you less incremental utility.
And if you stop and think about it, it's kind of intuitive.
Just stop and think, think about the movies you want to see right now.
The four movies you want to see.
Presumably whichever you ranked first would give you more utility to see than whichever you ranked second.
And if you think the movies that are out right now are pretty crappy like I do, by the time you get to the fourth movie, you're not getting much utility from it at all.
Thinking about movies that are out now, you're getting a lot of utility from that first movie you see.
Marginal, extra utility from the first movie you see.
But each additional one is giving you less and less.
And that's the idea of diminishing marginal utility.
Likewise with pizzas, if you haven't eaten all day, that first pizza can give you a very high marginal utility.
The enjoyment you get from eating that first pizza can be very large.
But the second pizza, not so much.
You're already pretty full.
Third pizza, even less.
And then fourth pizza would probably violate non-satiation.
So that's the basic idea.
Yeah?
I have a question.
Do we assume that the goods are homogeneous.
Is it the same movie watched four times?
Or different movies?
Actually, that's a great question.
And you have to specify that as part of the problem.
I haven't specified that here.
Obviously it can't be the same pizza eaten four times.
It could be the same kind of pizza eaten four times.
But do you see the same movie?
I haven't specified that here.
So there's not a general assumption about that.
It depends on how I define movies.
Did I define movies as-- I don't know.
God, I'm terrible.
All I know that's out now is the Guardians of G'ahoole because I've got a little kid who is interested in it.
Whatever movie's out.
Do I define movies as Guardians of G'ahoole, or do I define movies as seeing a movie?
And I didn't specify that.
Implicit in my examples, I specify movies as seeing a movie.
But you have to specify that to be more precise if you're actually trying to figure out-- it depends what you're maximizing over.
If you're maximizing over seeing any movie or maximize over seeing the same movie.
And I didn't specify here
It can work in both cases
It would work in both cases.
Clearly you could imagine, actually it's a very good point, where do you think your marginal utility would diminish more?
Seeing the same movie.
So what your example points out is that different goods will have different rates of diminishing marginal utility.
OK, so marginal utility will always be diminishing, but at very different rates for different goods.
So the general principle is that they'll be generally diminishing marginal utility.
But at different rates for different goods.
So after all my mess ups, let me just review.
Marginal utility is diminishing because each good is worth less to you.
It's always positive because of non-satiation.
And this graph represents the marginal utility you get from each movie you see conditional on having eaten two pizzas.
Marginal utility is the increment from the next unit consumed.
Now let's get back on track here.
Now let's go to thinking about-- now that we have this concept of utility and marginal utility, let's now bring utility back to preference maps.
Let's ask, given what we know about utility, what can this teach us about the shape of preference maps?
What's the linkage between utility and preference maps?
And that linkage comes through something we call the marginal rate of substitution.
The marginal rate of substitution is the mathematical concept that links preference utility with preference maps.
The marginal rate of substitution technically is the slope of the indifference curve.
It's delta P over delta M. The slope of the indifference curve is the marginal rate of substitution.
That's what it means graphically, but here's what you have to understand at a deeper level.
What it really is, it's the rate which you are willing to trade off.
The rate at which you are willing to trade off the y-axis for the x-axis.
The rate at which you're willing to trade off pizza for movies.
So that's what it means intuitively.
The slope of the curve tells you that you're indifferent.
Remember, you're indifferent between any points along with this indifference curve.
You're indifferent between four pizzas and one movie, you're indifferent between two pizzas and two movies, and four movies and one pizza.
You're indifferent along all those combinations of figure 4-6.
The MRS is the slope of that curve telling you the rate at which you're willing to trade off pizza for movies.
Now just a side note here, you're never, of course, actually trading.
There's not some market where you bring a pizza and get a movie.
So I didn't say trade, it's not like baseball cards.
I said trade off.
What I mean is ultimately you have some budget, and you have to allocate that budget.
So if you decide to allocate it on pizzas, you can't allocate it on movies.
Or the more you allocate on pizzas, the less you can allocate on movies.
So there's always a trade-off.
Remember, I said, economics is always about trade-offs.
Given your limited budget, there's always a trade off.
And the rate at which you're willing to trade off is your marginal rate of substitution.
Given that you're going to have to trade off-- and we haven't got a bunch of constraints yet, we'll get to that next time-- the rate at which you're willing to is your marginal rate of substitution.
Yeah?
Is that rate usually related to the price?
Ultimately no.
I'm sorry.
The marginal rate of substitution purely comes from your preferences.
Ultimately to decide how much you actually consume, you'll need to bring in the price.
So remember, I haven't talked about prices here, we haven't talked about that here.
But this is a preference concept.
This has nothing to do with prices.
But you're getting ahead of us.
We'll see next time, to decide how much you actually consume, you're going to relate the marginal rate of substitution to the prices you face in the market.
And that will decide how much you consume.
This is just a utility concept.
Yeah?
Did you say it was the y-axis or the x-axis?
That would be negative?
It's negative.
Yes.
Of course.
Right, of course.
The point is how many movies are you willing to give up to get another pizza?
How many pizzas are you willing to give up to get another movie?
MRS, it's very hard to remember what's on the top, what's on the bottom.
Be very careful on this.
But that's why I said remember it's the y-axis or the x-axis.
It's how many pizzas you're willing to trade off to get another movie.
Basically remember when I say trade off, here, this is not that you're literally trading, it's that ultimately you're going to have to make that trade-off.
Ultimately when we come to the next lecture and face a budget constraint, you're going to have to decide how do I want to allocate my budget across pizzas and movies?
The way you're going to decide that is by the relationship of how you feel about trading off one for the other.
Now here's the key feature of the MRS which is the MRS is yeah?
Question?
Yeah
That and exchange rates are always changing depending on how much you happen to be trading
Exactly.
The MRS is diminishing.
Technically when you go to grad school, you realize that marginal utility isn't actually technically always diminishing.
I said it is.
For this course it is.
But if you want to get mathematically correct, really what's always diminishing that you prove is the marginal rate of substitution is always diminishing.
So we have diminishing marginal utility for the purpose of this course, but the really important concept is you have diminishing marginal rate of substitution.
The rate at which you're willing to trade off pizza for movies is going to fall as you have less pizza and more movies.
So to see that, look at this graph, and let's compute the marginal rate of substitution along each segment.
So let's localize.
Imagine the segments were linear, imagine we had two linear segments between these points.
We don't.
But imagine for a second we did.
So the marginal rate of substitution from the first point, four pizzas and one movie, to the second point, two pizzas and two movies, the marginal rate of substitution is -2.
You are willing to give up two pizzas to get one movie.
This is the same graph, figure 4-6.
This isn't on the graph, you have to write it on.
So going from that first point to that second point, you're willing to give up two pizzas to get one movie.
So that rate of marginal substitution is -2.
However, when you're at two movies and two pizzas, and I say OK, how about giving up one more pizza to see movies?
Now you say, wait a second.
To give up one more pizza, I need to see two movies.
My marginal rate of substitution on that second segment is -1/2.
The marginal rate of substitution on the first segment is -2.
The marginal rate of substitution on the second segment is -1/2.
Once again, assuming they're not linear, so it's actually changing everywhere, but if they were linear, that's what it would be.
Can someone tell me why?
Why is the marginal rate of substitution falling?
Why is the marginal rate of substitution lower on that second segment than on the first?
Because marginal utility increases the fewer of something you have
Exactly.
So go ahead, flesh it out, the fewer of somthing you have, so tell me in terms of the trade you're willing to make
You value it more, so you want to trade more of something else for it
The point is when I have four pizzas my marginal utility of that last pizza is not very high.
And I'm fine to give up two pizzas-- and plus I'm only seeing one movie, there's a second movie I really want to see.
So you say to me, look, I've got four pizzas, I'm seeing one movie.
You say hey, there's a second movie out I know you want to see.
I know you don't really value four pizzas.
At the end, you're totally full.
Would you be willing to give up two pizzas to see the second movie?
And you're like, sure why not?
Well once you have two pizzas and you've seen two movies, you're not that interested in a third movie and you'll be hungry if you have less than two pizzas, so then you say, wait a second.
If you want me to give up another pizza, you've got to give me two movies.
Because my marginal utility of pizzas is rising, my marginal utility of movies is falling.
And that's why the marginal rate of substitution diminishes along the indifference curve.
So that allows us to write mathematically the definition of the marginal rate of substitution is the negative of the marginal utility of movies, or more generally what's on the x-axis, over the marginal utility of pizza, or more generally what's on the y-axis.
The marginal rate of substitution, the first key formula you need to know for this course, the marginal rate of substitution is equal to the ratio of marginal utilities.
Now this is tricky.
Maybe you guys don't find it tricky, it's the kind of thing I find tricky.
Which is I defined it as delta y-axis over delta x-axis.
And yet, when I defined here the marginal utilities, I flipped it.
I did the marginal utility of what's on the x-axis over the marginal utility of what's on the y-axis.
Why is that?
Can anyone tell me why that is?
Why is it flipped when defined in terms of marginal utilities?
Yeah?
It's a denominator.
So utility over movies--
Well, let me try for slightly more, how does marginal utility relate?
Yeah
Marginal utility is delta P over P. So it gets flipped.
Because of--
OK. Yeah, you're giving the same answer, which is technically right.
What I was more looking for but it's the intuitive version of that, marginal utility is a negative function of quantity.
Marginal utility is a negative function of quantity.
So the fact that it's a ratio of the quantity of pizza over the quantity of movies is the same thing as the marginal utility of movies over the marginal utility of pizza.
Because marginal utility is a negative function of quantity.
The more quantity you have, the lower is your marginal utility.
And that's the key to understand.
So it's the slope of the indifference curve which is the ratio of the marginal utilities, but it's the marginal utility of movies over pizza.
Because what that's saying is that as you get more movies, you care less about each additional movie and ditto with pizzas.
Let's just look at this for a minute, think about it intuitively for a minute.
We've seen it graphically, we're seeing it mathematically, let's make sure we understand it intuitively.
What this is saying is that as you get more movies-- so let's relate this to the graph.
As you get more movies and less pizza, as you move down that curve, more movies, less pizza, what's happening?
What's happening to the marginal utility of movies as you move down that curve?
What direction is it heading?
It's decreasing
What?
It's decreasing
It's decreasing.
Because you're getting more movies and marginal utility is a negative function of quantity.
Likewise, the marginal utility of pizza is increasing because you're getting less pizza so you care about each pizza more.
And that's why the marginal rate of substitution diminishes.
That's why it diminishes because as you move down that curve, the numerator is falling, the denominator is increasing.
And that's why we have everywhere diminishing marginal rates of substitution.
So another way to think about this is imagine for a moment what life would be like if we didn't have diminishing marginal rates of substitution.
And once again I'm going to try, once again Jessica, next year we'll let you make this pretty.
But I'm going to try to draw it crudely here.
Let's do pizzas and movies again.
Let's do pizzas and movies again.
Movies and pizza.
And that's one, two, three, four.
One, two, three, four.
Now let's imagine that instead of diminishing marginal utility and instead of indifference curves being convex to the origin, imagine if indifference curves were concave to the origin, which is what increasing marginal rate substitution would imply.
So that would be something where you'd be indifferent between four pizzas and one movie, between three pizzas and two movies, and between one pizza and three movies.
So your indifference curve would look like that.
Not quite to scale, but you get it.
It would be concave to the origin instead of convex to the origin.
In this case, marginal rates of substitution would be everywhere increasing.
That is, basically I'd be willing to give up one pizza to get one movie.
But to get that next movie, I'd give up two pizzas.
But as you can see, that doesn't make sense.
It doesn't make sense that given that as long as you're ranking movies, or even more in the example of seeing the same movie over and over again, it's maybe more compelling.
That basically what you can see is that if you're willing to give up one pizza to see that movie a second time, why would you possibly give up two pizzas to see it a third time?
That makes no sense at all.
If you only like it so much you only give up one pizza to see it a second time, why would you possibly give up two pizzas to see it a third time?
You wouldn't.
It doesn't make sense.
And that's why marginal rate of substitution has to be everywhere decreasing, it can't be increasing.
Yeah?
Could it remain constant?
It could actually remain constant.
Yes, that's right.
You can be indifferent.
My indifference curves-- how many of you guys have seen Toy Story 3?
I think it's one of my 10 favorite movies of all time.
The greatest children's movie ever made.
I've seen it three times.
My indifference curve is virtually-- I've enjoyed it the third times as much as the first-- it's virtually flat with respect to Toy Story 3.
I could see it 10 more times and feel pretty much the same.
So that's certainly possible that it would be constant, that I'd be willing to give up whatever I pay-- $10 a shot to see it.
It's possible.
So basically, almost always, inequalities will be greater than or equal to, or less than or equal to in this course.
It's more fun to talk about the not equal to case, the non-linear case.
But linear cases will exist as well.
It's just a can't be can't be opposite sign.
You can't have an increasing marginal rate of substitution.
Another question over here?
What about addictions?
You could want it more the second time
That's interesting.
So how would addiction work?
so basically--
Well it's not really decreasing.
You need more the second time, right?
So it has to--
That's very interesting.
I mean in some sense.
So you give up one pizza for the first shot of heroin.
And then, you're hooked, so then you'd be willing to give up two pizzas for the next shot of heroin.
Yeah, I guess so.
I guess that's right.
I guess we're going to have to stay away from addiction in this course.
I guess an addictive good could look like that.
That's a very good point.
Other questions, comments?
So what we're doing is we're going to stop here, understanding that we're going to have-- leaving this example aside-- we're going to have diminishing marginal-- yes one more question?
[UNINTELLIGIBLE]
Basically, we're assuming by non-satiation that ever happens.
So once again, that would violate the non-satiation.
The problem with the addictiveness example is the reason it wouldn't work in this course is eventually you'd violate your budget constraint because you'd want more and more and more.
Maybe not.
But in any case, we're going to ignore that example, assume diminishing marginal rate of substitution, and we'll come back next time as I put this together with a budget constraint to actually dictate your choices.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so today we're going to continue our discussion of consumer choice.
If you remember the set-up from last time, the main motivation is you're trying to understand what underlies demand curves, how consumers ultimately decide to trade off price and quantity of goods.
We said that ultimately that came from the principle of utility maximization, and that utility is maximized when individuals maximize the utility function, which is this mathematical representation of preferences.
And last time we talked about how if individuals were unconstrained how they choose what they want, they would just like more of everything, and their ranking across different bundles would depend on that underlying utility function.
Now, of course, what's stopping individuals from consuming everything they want is their budget constraints.
And so today we're going to turn to the second part of the problem, which is talking about budget constraints.
Now, we're going to make a very simplifying assumption here for most of the semester, which is we are going to assume that your income equals your budget.
That is, you spend your entire income.
That is, we're going to ignore the possibility of savings until about the third lecture from the end.
Now, this turns out not to be a terrible assumption for the typical American.
The typical American doesn't save.
So actually, it's not a terrible assumption for us to work with if we think about typical consumers.
In practice, savings is going to turn out to be a very critical part of what we're going to do to think about economics, so we'll come back to that.
But we're going to ignore savings for now and assume that your budget equals your income.
So let's say that your parents, probably a good model is you guys, you guys probably aren't in saving mode.
You've got some budget saved from your parents.
Let's call it y.
And let's say that your parents give you some budget at the start of the semester, y, and they say this is your money you have to spend, say each month or for the whole semester.
And let's imagine that you have to allocate that budget only across two goods, pizza and movies.
So once again, unrealistic, but this is the kind of simplifying assumption that lets us understand how people make decisions.
So that gives you your budget constraint.
You've got some income y that your parents have given you, and you can allocate that across pizza and movies.
So how do you allocate that?
Well, you can buy movies, the number of movies you can get, plus the number of pizzas.
Well, how many of each you can get, that depends on their price.
In particular, budget constraint is the number of movies times the price per movie plus the number of pizzas -- plus the number of pizzas times the price for pizza.
That's your budget constraint.
It's the number of movies times the price per movie or the number of pizzas times the price for pizza.
And this is easiest to see graphically.
If you go to figure 5-1, this is a graphical illustration of a budget constraint.
Now, let's just carefully talk through this for a moment.
You're going to be really good at dealing with budget constraints.
You're going to have to be this semester.
So let's carefully talk about where this comes from.
OK, the x-axis is going to be how many movies you could have if all you did with your income was consume movies.
Well, if all you did with your income was consume movies, you could have y over p sub m movies.
If you decided to devote your income solely to movies, then you could have y times y over p sub m movies.
If instead you decided to devote all your income to pizza, then you could have y over p sub p pizza.
So the y-axis is going to be the point where you consume zero movies and all pizza.
It's going to be where you devote your entire budget to pizzas.
And then there'll be some combination in between, which is our budget line.
Which is the combinations of pizzas and movies you can consume given your total income y.
And the slope of that line is going to be the price ratio.
Or the negative of the price ratio.
The slope of that line is going to be minus pp over pm.
The slope of that line is going to be the change in the ratio of the price of pizza to the price of movies.
OK, what am I doing wrong here?
Negative of the price ratio.
Have I got this right?
Rise over run.
Yeah.
Have I got this right?
Yeah
I think that the pp and the pm's are in the denominators, because it's y over pp [INAUDIBLE PHRASE]
Right.
It's the denominators, that's what I did wrong.
Right, so it is pm over pp.
Sorry, my bad.
OK, right.
Because they're the denominators.
Because it's the rise over run in terms of the quantity.
So that's what I did wrong.
OK, so basically it's the negative of the price ratio, minus the price of movies over the price of pizzas because they're in the denominators as you said, because as the price goes up, the quantity goes down.
So the negative of the price ratio of the price of movies to the price of pizzas is the slope.
So let's just do a simple example.
Imagine that income equals $96.
Imagine your parents give you $96, say a month or a semester.
Imagine that the price of movies is $8 and imagine the price of a pizza is $16.
It's a good pizza.
So what this means is that with your income of $96, you could either get eight pizzas or 12 movies.
So that means that the price ratio of the slope of your budget constraint is minus 1/2.
The price ratio is minus 1/2.
So the slope of that budget constraint is minus 1/2.
Now, we have a name for this slope.
We're going to call this the marginal rate of transformation.
The marginal rate of transformation is our label for this slope.
Now, why do we use that name?
Well, it means that's the marginal rate at which you can transform pizzas into movies.
The rate at which you can turn pizzas into movies.
Now, once again, like I talked about last time, you're not an alchemist.
You're not actually turning pizzas into movies.
But the market essentially is giving you a rate at which you can do that given a budget, given that you have a certain amount of money.
Given that you have a certain amount of money, $96, and given the prices that you face in the market, you could transform pizzas into movies by trading one pizza for 1/2 a movie.
Now, once again, you're not actually doing the physical transformation, but that's the trade-off that you face when you're trying to transform one to the other.
So effectively, it's the same as if you're trading them for each other.
As I talked about last time.
it's essentially the same as you're trading, and that's because of the key economic concept we'll come back to over and over again in this course-- the concept of opportunity cost.
The opportunity cost is the value of the forgone alternative.
The value of the forgone alternative is the opportunity cost.
So basically what that means is if you decide to forgo a pizza, that's the same as forgoing two movies.
Likewise, if you decide to forgo a movie, it's the same as forgoing half a pizza.
So the opportunity cost of a movie, what essentially the movie is costing you, is 1/2 a pizza.
Now, really it's costing you $8 and a pizza costs you $16.
But when we think about trading off goods, the opportunity cost of that movie is that you've forgone the ability to eat half a pizza.
That's the opportunity cost of the situation.
So that's basically how we're going to think about this trade-off.
We're going to think about trading off goods as the opportunity cost of consuming one good instead of another.
The opportunity cost of that movie is that you haven't gotten to eat 1/2 a pizza.
The opportunity cost of the pizza is that you've forgone seeing two movies.
And the reason is because you have a fixed budget.
If you had an infinite budget, there'd be no opportunity cost.
But because you have a fixed budget and you have to allocate that budget, there's an opportunity cost.
If you choose not to decide, you've still made a choice.
I don't know whether that quote's due to Shakespeare or Rush, I'm not sure.
I have to look that up.
But basically, if you choose to have one thing, then by definition you're forgoing another.
Now, to understand the budget constraint, let's talk about what happens when we shock the budget constraint.
Let's imagine the price of pizzas rose from $16 to $24.
Pizzas got really expensive.
We decided we only want gourmet pizzas or something.
The price of pizzas went from $16 to $24.
What does this do?
Well, let's look at figure 5-2.
It'll show what that does.
What that does is it says our new budget constraint, instead of being 16p plus 8m equals 96, which is what the budget constraint was in our example, it's now 24p plus 8m equals 96.
That's the new equation for the budget constraint.
Or, more relevantly, the slope of the budget constraint has flattened from minus 1/2 to minus 1/3.
The slope has fallen from minus 1/2 to minus 1/3.
The price ratio has been reduced from 1/2 to 1/3.
Now, forget utility for a second.
Forget the fact that we thought about utility.
Just looking at this, can you tell whether you are better or worse off from this price change?
You shook your head no, why not?
Because you don't know if you like pizzas enough to get any
OK, well in particular, you can almost tell.
Who's the only person who doesn't care about this price change?
[INAUDIBLE]
No, no no.
Think about consumers, people with different preferences for pizzas and movies.
What would your preference for pizzas and movies have to be for you not to care about this price change?
All movies.
So long as you care about pizza at all, you're worse off.
So in fact, the answer is your opportunity set has been restricted.
So we can think about the opportunity set.
Your opportunity set is the set of choices you can make given your budget.
Before, you could make choices all the way up to the upper line.
Now your set of choices that are available have just fallen.
Now, you're no poorer-- it's not like your parents have cut you off.
They still give you the $96.
But you're effectively poorer.
You're effectively worse off, and why is that?
Because the set of things you could afford with that $96 has just been restricted.
And unless you truly have no value on pizza, unless all you care about's the movie-- you're gluten and cheese allergic or something, you have no value on pizza-- then you're worse off.
Your opportunity set's restricted.
And that's the key insight here, is that you are worse off because the price has increased.
A price increase makes you worse off.
It restricts your opportunity set, because with the same amount of income, you can now afford fewer goods.
Your opportunity set has been restricted.
Likewise, now let's talk about what happens when your income falls.
That's the next figure.
Now, let's suppose your parents are pissed at you and they cut you down to $80.
Because you didn't do something.
You don't write enough or call enough, so they cut you down to $80.
Well, here the slope of the budget constraint has not changed.
Because what determines the slope of the budget constraint?
It's prices, and no prices have changed.
The slope of the budget constraint is unchanged because prices haven't changed, but your opportunity set is once again restricted because you now have lower income.
So you can now afford fewer pizzas and movies.
So now, instead of being able to afford up to six pizzas and up to 12 movies, you can now only afford up to five pizzas and up to 10 movies because your income has fallen.
So once again, you're unambiguously worse off.
Your opportunity set has contracted.
So your opportunity set will contract whenever income falls or whenever price increases.
And how it affects the graph will depend on whether it affects prices, which affects the slope, or just income, which affects the intercepts.
Now, questions about the budget constraints and opportunity sets?
Armed with that-- Yeah, I'm sorry, go ahead
Is the area under the curve at all indicative of utility?
No, it's not, because that's going to be determined by your preferences.
It's indicative of, if you will, potential utility, because that's your opportunity set.
But as the example here points out, if you don't like pizzas at all, you'll feel very differently than if you like pizzas a lot.
So it's indicative of sort of your potential utility, but not your actual well-being.
But now that's a great segue to the next step.
Let's put that together and talk about constrained choice.
Which is now, let's put together-- we know what your preferences are, we've mathematically represented those by utility function.
We know what your budget set is, we've mathematically represented that based on your income and prices.
Now let's put them together and talk about how you make choices.
And the basic question you want to ask is, what's the highest utility you can achieve given the constraints your budget constraints put on you?
Or, graphically-- so let's say you wanted to understand this intuitively, graphically, and mathematically.
Intuitively the idea is quite simple, I think, which is just, what's the most you can have given the constraints that are placed on you?
Graphically, we represent that as asking, what is the furthest out indifference curve you can achieve?
Because remember, more is better.
Indifference curves that are further out make you happier.
So what's the furthest out indifference curve that you can achieve given your budget constraint?
So to do that, let's actually do an example.
Let's imagine, as last time, your utility is the square root of pizza times movies.
Once again, this has no fundamental meaning, it's just a mathematical representation of your preferences.
So your preferences are mathematically represented by utility equals square root of pizza times movies.
And let's have the same budget constraint that we have up here.
Income is $96, price of movies is $8, price of pizza is $16.
Now let's go to the next graph.
Figure 5-4.
What this does is put together our indifference curve analysis with our budget constraint analysis.
It's a little complicated.
The budget constraint line is the vertical line running from a y-intercept of six to an x-intercept of 12.
The-- not the vertical line, the straight line running from a y-intercept of six to an x-intercept of 12.
That's your budget constraint.
We saw that before.
Then we have here a series of indifference curves.
These curves are drawn-- these are a mathematical representative of this utility function.
These are points among which you're indifferent if you have that utility function.
And what we see is that the best you can do is to choose point D. Point D, with six movies and three pizzas-- OK, that should be p on the y-axis, not C. It should be movies on the x-axis and pizzas on the y-axis.
That should be p on the y-axis.
The best you can do is to choose a point D. Now, to see that.
And that gives utility.
What's the value of your utility at point D?
We understand value's meaningless, but just so we can compare, what's the value of your utility at point D?
The square root of 18.
The value of utility is the square root of 6 times 3, which is the square root of 18.
Which is going to be square root of two, or three times square root of two, but we'll just call it square root of 18 for comparison.
Now, let's talk about why that's the best point for you.
Let's think about some alternative points.
For instance, why is that better than point E?
Somebody raise their hand and tell me, why is point D better than point E?
Yeah?
E is unattainable with your budget
E would be better, that'd be great.
We'd love eight movies and four pizzas, but we can't reach it.
So E's unattainable.
Why is it better than point A?
Point A you can afford.
So why's point E better than point A?
You could afford point A just like you can afford point D. Yeah?
Because the utility's only root 10
Because what?
It's only root 10
Yes, exactly, because the utilitity's only the square root of 10.
You're on a lower indifference curve at point A.
So it's true you could afford point A, but you're on a lower indifference curve.
Your utility's a lower value, it's only square root of 10.
So point A is dominated by point D. What about point C?
Well, point C you have the same-- point C is just an inward shift from point D, but here that's a dominated choice.
Once again your utility's lower.
It's the square root of 4.5 times 2.2.
And basically that's dominated because you could afford more.
So basically, the point is that the point which will make you happiest is the point at which your indifference curve is tangent to the budget constraint.
Because that is the point of the farthest out indifference curve that you can reach given your budget constraint.
The tangency of the indifference curve and the budget constraint is the point which makes you best off given your available budget and the available prices.
And that's the point where the slope of the indifference curve equals the slope of the budget constraint.
The tangency is the point where the slope of the indifference curve equals the slope of the budget constraint.
Or, more relevantly-- OK, let me stop there.
That's the graphic intuition.
With a sloping indifference curve because the slope of the budget constraint is the optimum, because by definition, that is the point of the furthest out indifference curve you can reach given your budget.
The point of tangency is the point of equal slopes.
Are there questions about the graphical analysis here?
This is very, very important, so.
Yeah?
This is sort of about the graphical analysis, but if it only matters in terms of the ordinal values of the utility function and p and M are always positive, does it matter if you got rid of the square root?
Would anything change if it was u equals pm?
Because the the marginal rate of substitution, transformation, all that would still be the same, right?
In this particular example, it would not.
So actually, you're asking a great question.
Because it's ordinal, you can typically do transformations for the ranking of bundles.
You will always get the same ranking with a monotone transformation of the utility function.
That's exactly right.
Later in the course, we'll show different ways why the functional form matters, and I'll show you why I did square root, because it's going to turn out that that's going to matter.
But for other things-- but for the ranking bundles, you're right.
The ranking of bundles is consistent with the transformation.
So that's the graphical.
Now let's come to the mathematical derivation of this.
So let's talk about the mathematics of utility maximization.
Now what I'm going to do here is, I'm going to do this sort of casually, as is my wont.
Friday in section, you're going to work on the underlying calculus that lies behind the mathematics that I'm going to present here.
Now, let's talk about what it means that these slopes are equal.
Well, remember, what is-- does anyone remember what the slope of the indifference curve is?
What do we call the slope of the indifference curve?
Yeah?
The
The MRS.
The slope of the indifference curve is the marginal rate of substitution.
Which is defined as what?
What is the marginal rate of substitution?
The ratio of what?
[INAUDIBLE]
No, the marginal rate of substitution.
This is just about preferences.
What's the marginal rate of substitution defined as?
It's the ratio of what to what?
Yeah?
The amount of one good you have to get to give up a unit of the other good
I'm sorry?
The amount of one good you have to get to give up a unit of the other good
So graphically that's what it's defined as, exactly.
It's the slope of the indifference curve.
Mathematically, what was it?
What was it in terms of utility?
Does anyone remember?
Yeah
The ratio of the partials
Ratio of the marginal utilities.
In particular, it's the negative of the marginal utility of movies over the marginal utility of pizza.
Remember, it's the negative of the marginal utility of the x-axis over the marginal utility of the y-axis.
So marginal rate of substitution is the rate at which you're willing to substitute between movies and pizza, which is a function of your marginal utilities.
If your marginal utility for movies is very high, then you need a lot of pizzas.
Then you wouldn't trade a movie unless you get a lot of pizza for it.
If your marginal utility of movies is very low, you'd be happy to give up a movie even for a small fraction of a pizza.
So that's the marginal rate of substitution.
That's about preferences only.
At the same time, we're saying that that marginal rate of substitution is equal to-- this slope is equal to the slope of the budget constraint.
Well, the slope of the budget constraint we call the marginal rate of transformation, which is the price ratio.
The slope of the budget constraint is the negative of the price ratio.
That's where you were sort of one step ahead of us here.
So preferences give us this, the marginal rate of substitution.
The mechanics of the market give us the marginal rate of transformation.
And utility maximization gives us that those are equal, because they're equal at that tangency.
At that tangency is where you get to the highest possible indifference curve.
So at the optimum, you get that the ratio of marginal utility equals the ratio of prices.
Now, I want to try to see you understand this a bit more intuitively, given this mathematics.
The way I like to think about this is, think about the ratio of the marginal utilities as the marginal benefit.
So it's the benefit of another movie in terms of pizza.
The marginal rate of substitution is the benefit of another movie in terms of pizza.
It's how much you like that next movie relative to how much you like that next pizza.
The marginal rate of transformation, the cost, is the price of that next movie relative to the price of that next pizza.
So what we're saying here is we're setting benefits equal to the costs.
In particular, we're setting marginal benefits equal to marginal cost.
The marginal benefit, the benefit to that next movie in terms of pizza, has got to be equal to the marginal cost, the cost to you in terms of that next movie in terms of pizza.
And this notion that the optimum will be where marginal benefit equals marginal cost will pervade through the whole course.
When we do firm maximization, it'll be the same thing.
Any maximization we'll do in this course, any optimization, will be about equating these margins.
Setting the marginal benefits equal to marginal costs.
Now, this is different than benefits equals cost, because it's about the next unit.
It's saying, how do we feel about that next movie compared to the price of that next movie.
Now, prices here we have being constant.
You could imagine prices of movies changing as you see more, but that gets complicated.
We'll worry about that later.
For now, the price is constant.
But the marginal utilities are not constant.
Marginal utilities are obviously changing the more moves you see.
Once again, with intuition, you have to develop your own intuition.
The way that I like to think about this intuitively is to actually rewrite this a little bit, and rewrite it as saying that the marginal utility of movies over the price of movies equals the marginal utility of pizza over the price of pizza.
At the optimum, this'll be true.
I like this because to me this term sort of says, look, the bang for the buck has to be the same across all goods.
For each dollar of movie expenditure, what's it buying?
What's that next dollar of movie expenditure buying you?
This is saying, what's that next dollar of pizza expenditure buying you?
And they've got to be equal.
If the next dollar of movie expenditure buys you a lot more happiness than the next dollar of pizza expenditure, then you're not at the right place.
You should shift your money and spend more on movies and less on pizza.
If the next dollar of pizza expenditure buys you a lot more happiness than the next dollar of movie expenditure, then you're not in the right place either.
You should see fewer movies and buy more pizza.
So basically, it's where the marginal benefit to you-- the bang for the buck of that next movie is the same as the bang for the buck of that next pizza.
So to see this, let's go back to figure 5-4 and let's actually think through the mathematics of a couple of these points.
Let's take point A.
At point A, you have two pizzas and five movies.
So pizza equals two, movies equals five at point A.
So utility is equal to square root of 10 at point A.
Now, in particular, your marginal utility for pizzas at that point-- what's the marginal utility for pizzas?
Well, we can differentiate.
That's the derivative of the utility function with respect to prices, du/dp.
Which is going to be 0.5 times movies over the square root of p times m, which at these values is going to be one over the square root of 10.
That's your marginal utility of pizzas.
Your marginal utility of movies is going to be du/dm, which is going to be 0.5 times p over square root of p times m, which is going to be 2.5 over square root of 10.
So the marginal rate of substitution between these two is 2.5.
The marginal rate of substitution, 2.5.
What does that mean intuitively?
Can someone tell me what that means intuitively, the marginal rate of substitution is 2.5?
What does that mean?
Someone explain that like you'd explain it to someone who's speaking English.
What does it mean that the marginal rate of substitution is 2.5?
Yeah?
You'd give up one pizza for 2.5 movies.
Yes.
No, actually, the opposite.
You'd give up two and 1/2 pizzas-- the marginal rate of substitution is 2.5-- no, that's right.
You would give up 2.5-- one second, let's make sure I have this right.
Right, so you would give up one pizza to see 2 and 1/2-- no, it's the other way around.
You're getting a lot of pizza and not enough movies.
So you would give up two and 1/2 pizzas to get one more movie.
This is confusing, OK?
You'd give up two and 1/2 pizzas to get one more movie.
That's what it means.
You would give up two and 1/2 pizzas to see one more movie, and why is that?
Why at a point like A would you give up two and 1/2 pizzas to see one more movie?
Yeah?
Well, if you just look at the line tangent to the indifference curve at A, it's really not a downward slope
It's really steep, which means what?
Which means you get a lot more benefit--like you don't care if you give up a lot of pizzas to see more movies
Exactly.
Actually, that's a great way to bring the graphics and the intuition together.
Here's thinking of it intuitively.
I'm getting a lot of pizza at point A. I'm not getting a lot of movies.
So I would happily give up a lot of pizza to get my next movie.
What you pointed out is the tie to the graphics here.
The indifference curve is very steep at that point through A.
A steep indifference curve in that way means I don't really care a whole lot at this point about how many pizzas I get, but I care a lot about getting more movies.
So at a point like A where it's very steep, you are willing to give up two and 1/2 pizzas to see a movie.
But what do you have to give up?
What's the market telling you you have to give up to see a movie?
How much pizza do you actually have to give up to see a movie in practice?
You're willing to give up two and 1/2 pizzas to see a movie, but how many pizzas do you actually have to give up?
1/2
1/2 a pizza to see a movie.
So that can't be the optimum.
You're willing to give up two and 1/2 pizzas to see a movie, but you only have to give up 1/2 a pizza to see a movie.
So you can't be at the right place.
You should be changing your consumption bundle.
You should be changing your consumption bundle, because the market is only asking for 1/2 a pizza to see a movie, but you're willing to give up two and 1/2 pizzas to see a movie.
So at a point like A, you're clearly not at the optimum.
Because you are willing to make a trade.
Remember, we talked about-- we can go all the way back to the first lecture.
The key point was-- the first or second lecture-- inefficiency happens when trades aren't made that people value.
Here's a trade that you value.
You're willing to give up two and 1/2 pizzas to see a movie.
The market only wants 1/2 a pizza to see a movie.
You're not making a trade you value, so that's not the efficient outcome.
Likewise, let's do the same mathematics at point B.
Well, at point B, if you do the math and work it out, you see that the marginal utility of pizza is five over square root of ten.
The marginal utility of movies is 0.5 over the square root of 10.
So the MRS is 0.1.
At this point, you'd only be willing to give up 0.1 pizzas to see a movie.
At a point like B, the indifference curve is very flat.
You're only willing to give up 0.1 pizzas to see a movie.
But remember, the market is willing to say, look-- you can flip it around, the market's willing to say, look, you can get a movie.
You're only willing to give up 0.1 pizzas to see a movie.
Well, that means clearly that you have too many movies and not enough pizza.
You're clearly at that point happy to say, wow, you mean that I can gain a whole pizza by just giving up two movies?
Heck, I'd be willing to give up 10 movies to get a pizza.
At the point I'm at right there, I'd be willing to give up 10 movies to get a pizza.
You're telling me we only have to give up two movies to get a pizza?
Great.
I'm going to do that trade.
I'm going to move back towards point D. So that's why this idea of what you're willing to do, which is the MRS, and what the market's making you do, you want to equilibrate those to decide how much you want to consume.
Now, obviously a point like-- let's talk about point C. Point C is interesting, because at point C, what's true?
The marginal rate of substitution is equal to the marginal rate of transformation at point C. The slope of the indifference curve and the budget constraint are equal.
That's why you have to check two conditions for optimization.
First of all, those slopes have to be equal.
Second of all, you've got to spend all your money.
So it's true there's a whole host of points-- in fact, there's a vector running through CDE, all which are points where the margin rate of substitution equals the margin rate of transformation.
But only D is optimal, because you also have to remember more is better.
You never want to leave money on the table.
So the two conditions you have to meet is that you're at the point where your desired trade-off between pizzas and movies is the same as the market's, and where you're spending all your budget.
And that's the optimum.
OK, questions about that?
So that's basically how we think about optimization.
That's how we think about consumers making their decisions deciding between consuming pizzas and movies.
Now, let's come back-- however, this is a particular case we've looked at.
This is a case in particular where we've imposed that there's an interior solution.
In fact, in practice you could end up in these kinds of choices with corner solutions.
So let's take a look at the last figure, figure 5-5.
We've chosen a case where your optimal bundle includes both pizza and movies.
But you could imagine a situation where your optimal bundle-- and once again, that should be a p, not a c in figure 5-5, that should be p on the y-axis-- where your optimal bundle includes only one or the other.
So this is a particular case-- we have the same budget line as before, which is you're trading off pizza and movies.
You have an income of 96, the price of pizzas is 16, the price of movies is 8.
So same budget line as before.
But now your indifference curves look very different.
Now your preferences are such that you've got these linear difference curves of the form I1, I2, I3.
You've got these linear indifference curves.
What that means is you've got-- these indifference curves mean that you have a constant rate at which you're willing to trade off pizza for movies.
If we go back to figure 5.4, the rate at which you're willing to trade off pizza for movies changes.
Your preferences are such that-- because the square root function as pointed out-- your preferences are such that you are willing to make different rates of trade at different amounts.
In figure 5-5, your preferences are constant, no matter how many movies or pizzas you have.
You're always willing to make that trade-off at the same rate.
Well, in that case you can end up with a corner solution, where in fact, you're going to consume only six pizzas and no movies.
And why is that?
That's because this is a person who loves pizza relative to movies.
It's a very flat indifference curve.
They love pizza relative to movies.
And they love pizza so much relative to movies that given the prices they face, they'll just go ahead and choose six pizzas and no movies.
So that's a corner solution.
So mathematically, as you'll go through a section on Friday, you're going to have to check for corner solutions.
You may solve these problems and end up with negative quantities and be befuddled about what happens.
Well, if an answer looks wrong, it is wrong.
If you solve problems with negative quantities, that's probably because there's a corner solution to the problem, and actually the optimal quantity is to have zero of one thing and spend your entire budget on something else.
Questions about that?
So now let's think about applying this to the kinds of decisions that you all have to make.
So I talked about this particular example of pizza and movies.
And in fact, you might say, well, that's sort of unrealistic.
Gee, I spend my budget on lots of things, and how do I do this?
Well, in practice, what you'd have to do is you'd have to draw a multi-dimensional graph and solve a multi-dimensional problem.
And that's a bear.
But in practice, in fact, we can often think about breaking down the choices we make into pairs of choices.
In practice, you could think about saying, look-- many people do what a lot of psychologists call mental accounting, where basically they say, look, yes, I have a whole budget and lots of things I can buy.
But in fact, I like to think of my budget in sort of subcategories.
I think of a certain amount I'm willing to spend on entertainment and a certain amount I'm willing to spend on food.
And I take my budget and I mentally put it in different buckets.
And within each of those buckets, you can then do the same kind of optimization problem that we've done here.
So even though in reality we choose across a whole host of goods, in practice what you're going to see is that people will do this kind of mental accounting, where they sort of divide their goods into different buckets and optimize within each of those buckets.
What this means in practice is that in fact, if we now stop for a second and think about the government, and how it affects our consumption decisions, what this means is that in practice, the government-- so, one way we typically of the government affecting consumption decisions is through the power of taxation.
So let's say for example, the government decided that pizzas were bad.
They caused obesity.
That there's too much obesity because people are eating too much pizza, and we need to deal with that through a government policy that involves taxation.
Somebody talk me through the analysis of how we'd think about analyzing a tax on pizza given these diagrams.
Let's go to figure 5-4.
And imagine I said that we're going to place a 50% tax on pizza.
So we're going to say that every dollar you pay on pizza, you're going to have to pay $0.50 to the government.
Because we're really worried people are eating too much pizza.
What would that do the budget constraint?
Yeah
Well, effectively you're increasing the price of pizza
Effectively you're increasing the price of pizza to the consumer
The budget constraint would shift down like this
It's actually going to have the same effect as we saw in figure 5-2.
In fact, I've just replicated figure 5-2.
Because in figure 5-2, the price of pizza went up by 50%, from $16 to $24.
That's the same thing that the government's just done.
It's raised the price of pizza effectively from $16 a pizza to $24 because instead of paying your $16, you're also going to pay $8 to the government in tax.
So what's that going to do?
That is going to, in general, lower consumption of pizza.
So that kind of price increase is going to, in general, lower the consumption of pizza.
So the government has tried to accomplish its goal by shifting people away from pizza towards movies, away from pizza toward other things.
Yeah?
Would it be meaningful to say that it also affects the utility curves?
It would not be meaningful, actually-- it will affect your optimal choice.
In general, you will choose a different amount of pizza and movies.
We'll talk about that next time.
But it's very important, it would not be right to say it affects utility.
Utility is like what you're born with, it's an innate concept about your underlying preferences.
However, you're actually getting to my point, which is in fact, the way economists typically think about this is the government can't affect your preferences.
But in fact, if people do mental accounting, the government maybe can affect your preferences.
So let's say that basically the way I think about it is, let's say I think I have a budget for food and I have a budget for entertainment.
And let's say I think my budget for entertainment's pretty small because I'm low income.
I've gotta have a budget for food.
And so let's say I put pizza in my budget for food, so I allocate some of my budget for food to pizza.
And the government can then cause me to eat less pizza by taxing it.
But what if somehow the government could get me to think of pizza differently?
What if somehow the government could get me to think of pizza as entertainment?
And suddenly I put it in that bucket, where I'm trading it off not against other food, but against the fact that I want to see a movie and I want to download stuff from iTunes, et cetera.
Maybe then I'd buy less pizza at the same price because I'm putting it in the bucket where I have less money.
So in other words, we've imagined a world with two goods.
And the only way you can affect the choice across those two goods is to lower your income or your price.
But in fact, if people have lots of different bundles of goods, somehow I could shift you mentally from considering pizza in a bucket where you have a lot of money to putting pizza in a bucket where you don't have as much money.
I can lower your consumption of pizza without affecting prices, without an obtrusive government tax policy.
This is the kind of thing that we call in economic policy a nudge.
And there's a new book called Nudge by Richard Thaler, who's a famous behavioral economist, basically bringing psychology into economics.
There's this very important field now in economics called behavioral economics, which is all about, how can we bring the lessons of psychology into economics?
We don't do that in 14.01.
14.01's all about, we assume everybody's these perfectly rational people who would never really be fooled into thinking about pizza differently just because of what the government told them.
But in fact, in reality people think about things differently based on the kinds of information they have.
And given that tax policies can feel very intrusive-- imagine some of you are like, wow, they're raising my pizza to $24, that seems very intrusive.
If the government could somehow through nudging you to think about pizza differently change your pizza consumption, that might be a much more acceptable and palatable policy to many people.
And that's the kind of role the government can play, or policy-makers can play, is not just by changing your prices or income, but by actually changing how you categorize things mentally, they can change the choices you make.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're going to start with an interesting application of demand curve analysis, of the kind of indifference curve and constrained choice analysis we've been doing, the case of food stamps.
And then we're going to move on and talk about deriving demand curves.
So let's talk about food stamps.
It's an interesting case.
This is a policy that's been in place in the US government for a very long time, which are essentially coupons that individuals can use to buy food.
It used to literally be coupons.
It used to literally, you'd get a stamp, a coupon, and you'd go to the supermarket and hand this in instead of cash to buy your goods.
Now it's actually a debit card.
And it's given to low-income individuals as a way of redistributing income in society.
So individuals of income below a certain level, typically, say, a family with income below about $25,000 a year, below what we call the US poverty line, will be eligible for food stamps, which is a debit card they can use to charge their food purchases.
Now, what I want to talk about today is how we can use the kind of analysis we've done so far to think about the effect of food stamps.
So let's start with figure 6-1.
And let's think about someone with an original budget line.
If someone has a budget line, they have income of $1,000.
OK, a very poor person, they have income of $1,000.
And let's talk about them, their choices between food and all other goods.
Once again we have this analysis.
We put everything in two dimensions.
What we think about is via mental accounting, as we talked about last time.
However you want to justify it to yourself, the way we model it is thinking about people having this choice along two dimensions, food and other goods, and they have income of $1000.
Let's say we'll have two individuals, individual x and y.
Individual x doesn't care that much about food.
They really like consuming other goods.
So they spend $600 of their income on other goods and $400 on food.
Individual y cares a lot about food.
They end up spending $900 of their budget on food.
And that should be 100.
100 on other goods.
So that y-intercept should be 100 for person y.
So they spend 900 on food and 100 on other goods.
Now let's say the government comes in and wants to consider two options.
The government decides, look, we want to help poor people.
And particularly we want to give people like these people $500 in resources.
Let's think of two different ways the government could do that.
One way the government could do it is it could give them cash.
It could say, look, we're going to send you a check for $500.
What would that do to their budget constraint?
Well that would shift it out, as we talked about last time.
It would be an outward shift of $500 at every point.
So their new budget constraint would be the line that runs from $1,500 of all other goods to $1,500 dollars in food.
So include that.
So both the solid and dashed portions would be their new budget constraint.
So the new budget constraint would run from 1,500 to 1,500.
Person x would choose to move from x1 to x2.
I'm sorry, they would choose to move from x1.
It's not labeled as a point.
But they would actually choose to move from indifference curve 2 to indifference curve 3.
OK, that's where they'd choose to move if they could choose along this entire line.
Let me sort of, well, I'm not going to draw it, because I'll do a bad job.
But basically, if you can think about the new budget constraint running from 1,500 to 1,500, person x would move from indifference curve 2 to indifference curve 3.
They would choose-- Actually, you know what?
Let's put this graph aside.
It's not quite right along the number of dimensions.
I'm going to draw this, because there's a number of problems with that.
So you've got an original budget line that runs from 1,000 to 1,000.
And we've got person x up here, and they're choosing to spend 400 on food and 600 on other goods.
And we've got person y down here.
This is person x, and they're in section x1.
And person y intercepts at y1 where they choose to spend 900 on food and 100 on other goods.
Now, the government first says, look, we're going to give people $500 in cash.
That just shifts the budget constraint out parallel, but now runs from 1,500 to 1,500.
Let's say that the choices people make-- Person x would say, great, I'm going to take that money, I'm going to spend almost all of it on other goods.
So I'm going to move to a point like x2, where I'm going to consume $1,200 on-- I'm going to consume-- They were consuming-- No, I'm sorry.
15, right.
They're going to move from spending $600 on other goods and $400 on food to spending $1,100 of their dollars on other goods, and-- let me think for a second.
Let's say they would go vertically.
So let's say they'd choose to spend all of it on other goods.
So they'd take the whole 500, and they'd go from spending-- they'd continue to spend 400 on food, but now they'd spend 1,100 on other goods.
So person x, they would continue to spend $400 on food.
They'd say I'm going to take that entire $500, instead of spending $600 on other goods, I'm going to spend $1,100 on other goods.
Let's say person y, they would say, well I'm going to sort of split it.
I'm going to spend some on food.
So this is their new intercept x2.
I'm going to spend some on food, and I'm going to spend some on other goods.
So I'm going to go out here.
I'm going to spend now-- instead of spending $900 on food, I'll spend $1,200 on food.
I'll take $300 and spend it on food.
Instead of spending $100 on other goods, I'll spend $300 on other goods.
OK, so that's person y2.
OK, so that's what they'd do if we gave them $500 in cash.
Now say the government came in and said, you know, we're going to give you $500 but in the form of a coupon that you can spend on food.
So the first question is, what does that do to the budget constraint?
This is a bit tricky.
We've got to think about this.
Think about their budget constraint.
What that says is for anyone who wants to continue to, at least spend $1,000 on-- anyone who wants to at least spend $500 on food, it does not change their opportunities at all.
So the new budget constraint looks like this.
It's a solid line to here, and then it goes down.
This intersects at 500.
It's a solid line to the point at $500, and then it goes down and follows the old budget constraint.
Why does it do that?
Because for anyone to this side, they are $500 richer regardless of whether you give them cash or food.
Either way they're $500 richer.
Why?
Because as long as you intended to spend $500 on food anyway, it doesn't matter the form in which the government gives you the money.
Think about that for a second.
It's very important.
If you were going to spend $500 on food anyway, it does not matter if the government gives you a check for 500, or a food card for 500.
Why is that?
That's because your budget is what we call fungible.
You can always move money around within your budget.
So let's say you're spending 500 on cash.
Let's take a person like y.
Let's take a person like y.
They were spending 900 on food and 100 on other goods.
The government comes in and gives them a food card for 500.
Well to them that's the same as $500 in cash because they're already spending $900 on food.
They can just take some of the cash on food, spend it on other things, and use the card for the food instead.
So for them, there's no difference in giving them cash or giving them the food card.
It has the same effect.
They move to y2 either way.
Now let's take a person like x.
Well, if you gave them cash, $500 cash, they'd spend none of it on food.
So they've been consuming $1,100 of other goods and $400 of food.
But they can't do that if you give them a food card, right, if you give them food stamps.
That's not a choice.
They are now constrained to move to a point like x3.
They're now constrained to move to a point like x3 where they have to go to this intersection.
Because this point is not attainable.
This point is not attainable.
If you give them food stamps, the new budget constraint is this.
So x2 is no longer attainable.
They have to move to a point like x3.
So what do we know about their level of happiness at x3 versus x2?
Someone raise their hand and tell me.
Yeah
It's lower
It's what?
Lower
It's lower.
And how do you know that?
Because it's kind of like, if it was on the full curve, they would be elsewhere
Well OK, so there's two different ways to see it.
One is you could say, look they're at a lower indifference curve.
You can see what's wrong with this graph.
The indifference curves cross.
The indifference curves can never cross.
So that's wrong there.
They're on a lower indifference curve, OK.
But what's the other way to think about it?
The marginal rate of substitution and transformation aren't the same
The marginal rate of substitution and transformation aren't the same.
That's another way to think about.
And that's always true at the optimum.
But what else do we know about a point like this?
They could have chosen that point before and didn't.
Right?
When they had the cash, they had the option of choosing this point, but they didn't, they chose a different point.
So we know by something called revealed preference that they're worse off.
This is a very important concept.
If someone makes a choice that they turned down before, then by revealed preference they're less well off.
We've revealed that they're worse, because they could have chosen this point before, but they didn't.
When you gave them the cash, they chose this point.
So by revealed preference we revealed they're worse off.
So it's the same as saying their indifference curve is lower.
We've revealed they're worse off.
So what we've learned is for person y, they don't care if you give them cash or a food card, food stamps.
It's called food stamps, but it's now a debit card.
Person x is made worse off if you give them the food stamps instead of the cash.
Why do it?
You're the government.
The US government spends $35 billion every year giving people food stamps instead of cash.
Why don't we just take that $35 billion and give it to them in cash?
Yeah?
Probably because the government doesn't trust people to spend it on what they actually need.
And that will just lead to more poverty and people wasting it on things they don't need
Or to put it more succinctly, what if this axis was not labeled other goods but labeled cocaine.
Then we might be sad that you took the whole $500 and spent it on cocaine.
We might want you to take that $500 and spend it on food.
So it's paternalism.
The reason we give the guys food stamps instead of cash is we don't trust them with the cash.
If we trusted people with the cash, there'd be no reason not to give them the cash.
We are unambiguously making them worse off by forcing them to consume a bundle that's on a lower utility curve, lower indifference curve.
But since we don't trust them, since we're paternalistic, we are willing to go ahead and force them to do that.
So then the interesting question becomes, well, how much are we costing them?
In fact, it's not obvious.
If everyone in the world looks like y, then there's no cost to food stamps.
There's no good done either.
Then it doesn't matter if we give them cash or food stamps.
But if a lot of people look like x, then there is a welfare cost to people.
They are worse off, from their own perspective, getting food stamps.
Society may think they're better off.
So how do we tell?
How do we tell?
Can anyone take a guess?
If you're now an empirical economist, and you want to test, how would you tell if people are like x or like y?
Any ideas?
It's tricky, but let's see if we have any budding empirical economists here.
Yeah?
You'd see if they're spending any cash beyond the $500 that you gave them, because then you'd basically [UNINTELLIGIBLE]
See if they're buying food beyond the $500 you gave them
They might spend $100 cash on food
Excellent.
So that's one way you'd do it.
You could look at people who get food stamps and see if they're spending more.
That's a great idea.
The other thing you could do is you could actually literally run an experiment where you take people who are getting food stamps and replace them with cash or vice versa and see what happens to their behavior.
When we do this, we find that about 15% of people are like x.
Or in other words, the way to say it, is about $0.15 more precisely.
When you give people food stamps instead of cash, they spend 15% more on food than they would if you just gave them the cash.
So there's about 15% lower utility compared to what they'd want for spending it on the food instead of the cash.
So the question is, is it worth it?
We're basically taking people and making them spend $0.15 more on food than they'd want to.
That's the right way to think about it.
If you give them food stamps instead of cash, they spend $0.15 more on food than they would if you just gave them cash.
Is it worth it?
That's a great question.
It depends on how stupid we think people are and how paternalistic we want to be.
If we think people would really waste the money, then $0.15 is not much to give up to make sure they eat.
If we think nobody would waste the money, then we're just throwing $0.15 down the toilet by making them buy food instead of goods they prefer.
And that's the interesting kind of public policy question we have to struggle with.
We think about government policy and redistribution.
That's exactly the kind of question we need to struggle with.
And we'll come back to that again later in the course when we talk about efficiency versus equity.
OK, questions about that?
Yeah?
[INAUDIBLE PHRASE]
Sure.
I mean, so basically you make a good point.
We sort of like to know.
Actually that's a very good point.
You say, when we run these experiments and replace the food stamps with cash, we like to know what they spend the cash.
We want to know not just what happens to food consumption.
So if you run the experiment, and you say, I was giving you food stamps.
I now cash you out and give you cash.
And I find you spend 15% less on food.
Well what do you spend more on?
If it's clothes, maybe we're not so worried.
If it's cocaine, maybe we are.
So that's a very good point.
That's something we could look at.
Excellent.
OK, so that's an example of how we can use the kind of analysis we did last time to think about policy making.
Once again, this is an incredibly simple framework, right?
Yet I just described to you a succinct way to think about the implications for society of different government policies.
That's the power of this kind of simplified framework.
Now let's move on, and let's get to the core of why we did all this.
The reason we did all this is we wanted to figure out how we come up with demand curves, where demand curves come from.
The stork doesn't bring them.
Demand curves come from underlying utility maximization.
And we'll see that now.
And basically the way to do this is to return to our example from last time.
Your parents gave you $96.
You could buy movies at $8 a pop or pizzas at $16 a pop.
So we said last time, if you turn to the next page of the handout.
What we said last time is if given your utility function, u equals square root of p times m, you would choose a point like a.
If the price of pizzas was $16, the price of movies was $8, your income was $96, you would choose a point like a, where you consumed-- At point a, you're consuming six movies and three pizzas.
Once again that should be p on the y-axis.
You're consuming six movies and three pizzas at point a.
Now let's say the price of pizzas rises.
I'm sorry, now let's say the price of movies rises to $12.
So the price of movies rises from $8 to $12.
Well what does that do to the budget constraint?
That steepens the budget constraint, moves it inward.
Because now think about your opportunity set.
For the same income of $96 you can buy the same number of pizzas you could have before, but now you're buying fewer movies.
Same number of pizzas you could have bought before, but now you can buy fewer movies.
So your new budget constraint, you have a new constrained opportunity set.
With a steeper budget constraint, the slope, instead of being minus 1/2, is minus 3/4.
And given the preference I wrote, u equals square root of c times m, you should be able to show yourself that you'd now choose a point like b, where you have three pizzas but now only four movies.
So you reduce the number of movies, you keep the number of pizzas constant.
And you check that we still spent our total budget.
Well, 4 times 12 plus 3 times 16 is still $96.
So we're still spending our total budget.
The marginal rate of substitution you can compute if you write it down from that utility function, will be minus 3/4, which is the same as the marginal rate of transformation with this new price.
So you will choose a point like point b.
Now let's say instead the price of movies fell from $8 to $6.
Well in that case, your budget constraint would flatten.
It would move outwards.
Your opportunity set would expand in that case, because effectively you're richer.
Your opportunity set expands.
You move to bc little 3.
You move to bc little 3.
And given those preferences I wrote down, u equals square root of p times m. u equals square root of p times m. You end up choosing point c, with the same three pizzas but now eight movies.
Once again, how do we know that's right?
Well first of all the marginal rate of substitution, you can compute, will equal the new marginal rate of transformation.
And also you can see you spend your entire $96 income.
You're still roughly splitting it with $48 on movies and $48 on pizza, exactly splitting it.
So all we've done here-- Forget the bottom diagram for a second.
All we're doing in this top diagram is saying, given your utility is u equals square root of p times m and given your income and the prices, these are the choices you would make as prices change.
Are there questions about that?
Now armed with that, we can draw a demand curve.
Because what have we done?
We've just given you three different prices for movies and three different quantities of movies you choose.
We know when the price of movies was $8, you chose six movies.
That's point b. I'm sorry, that's point a.
The points are mislabeled too on this.
I'm sorry.
If you go to this bottom graph, these points are mislabeled.
So it should go b, a, c. It should go b, a, c. So when the price of movies is $8, that's point a in the middle, you choose 6 movies.
When the price of movies rises to $12, your demand for movies falls.
You only choose 4 movies.
When the price of movies falls to $6, you choose 8 movies.
Thus the demand curve.
And we're done.
That's where demand curves come from.
They just come from utility, constrained utility maximization.
You just take your utility function, you maximize it, given the constraint the budget constraint places on you, and boom, you have a demand curve.
Now note that this a particular case we did.
And it's a particular case that's interesting.
In this case as we change the price of movies, what happened to demand for pizzas?
Stayed the same
Stayed the same.
That is a particular case.
It's basically the case that will happen with utility function of this form.
It's a case of what we call no cross-price elasticity.
This example has no cross-price elasticity.
What that means is that in this particular case we chose, as the price of one good changes, it does not change your demand for the other good.
That's a special case that will not in general be true.
You can imagine if your income was only $96, and the price of movies was swinging around, that might affect your taste for pizzas.
That might affect your demand for pizzas as well, because you're only have a fixed budget.
That's a more general case.
We've chosen a particular case here with no cross-price elasticity.
But don't think that's general.
This is not a general lesson.
There's the price of one good changes the other goods unaffected.
In fact, in general, both goods will be affected when any one price changes.
That's a more general result.
You should be able to check at home that you can do this exact same exercise for pizzas and draw the demand curve for pizzas the exact same way.
You'll still get a flat-- you'll still get no cross-price elasticity.
You'll still get this flat curve-- well now it will be vertical curve with respect to movie purchases.
But you can see as you change the price of pizzas, you'll find a well-defined pizza demand curve as well.
OK?
So that's where demand curves come from.
We basically maximize utility at different prices given your income, and we end up with a demand curve that shows us the relationship between how many movies you choose and the price of movies.
Demand curves themselves can also shift.
We talked about that in the second lecture.
We talked about demand curve shifting.
And one reason demand curves can shift is because you get richer.
So let's talk about how that can happen.
Let's now turn to figure 6-3, which is really tiny.
My bifocals are in at the mall.
I just haven't picked them up yet.
So let's take the glasses off for this one.
Now let's take a case.
Once again, originally you're at point a, where you're choosing six movies and three pizzas.
Now let's say your income rises.
Your parents are feeling generous.
And instead of giving you $96, they're going to give you $128.
Once again, on that y-axis it should be labeled p, not c. You can now afford up to 8 movies and up to 16 pizzas with your $128 income.
So your budget constraint has shifted outwards from bc1 to bc2.
At that new higher budget constraint, you're going to choose, instead of choosing a, which is six movies and three pizzas, you'll choose b, which is eight movies and four pizzas.
You're richer so you choose more of both.
Likewise if your parents cut your income to $64, you're budget constraint will shift inwards.
Your opportunity set will be constricted.
You move to budget constraint three.
And you choose fewer of both pizzas and movies.
So you can see as your budget constraint shifts, how you choose different amounts of both pizza and movies.
We can translate that to shifting demand curves for movies.
So if you draw that down to the next diagram, you say, look, I can now graph that at a given price of movies-- prices have not changed in the example.
The slope of the budget constraint is the same.
Only your income has changed.
At a given price of movies of $8, as my income changes, I am on different demand curves.
You can see the demand curve for movies shifting out and in as my income changes.
So as my income went up, the demand curve for movies went out and moved from point a to point b.
As my income fell, the demand curve for movies went up, shifted in, moved from point a to point c. We can then drop that down one more level, just to make life especially interesting, and draw the relationship between your income and your demand for movies.
And that's the third figure.
Here we graph the relationship of your income and your demand for movies.
This is not a demand curve.
Demand curves only relate price to quantity.
This is what we call an Engel curve.
Those of you who studied your socialism theory will remember Engels worked with Marx.
It's not him.
It's Engel not Engels.
Different guy.
So basically this is an Engel curve.
And basically it shows the relationship between your income and your demand for a good.
And this turns out to be a very important concept.
Because an important thing that we'll focus on now is what we call the income elasticity of demand.
We've talked about price elasticities.
What's the price elasticity of demand?
Someone quickly, someone raise their hand and tell me, what's the price elasticity of demand?
Get some other folks involved here.
Yeah?
How demand changes with the price of the item?
Right, so as price changes, how demand changes.
The income elasticity is the same concept.
It's a change in demand as your income changes.
In the book it's this fancy letter.
I can't write, so I'm going to call it gamma.
But it's some c thing that I can't draw in the book.
Which is delta Q over Q over delta y over y.
So just like the price elasticity is the percentage change of quantity, percentage change in price, the income elasticity is the percent change in quantity with percent change in income.
Once again, just like price elasticities are locals.
You talked about it in section.
You talked about, sort of, local versus global price elasticities and how it's really local to that segment of the curve.
Income elasticities are local too.
Your income elasticity will obviously change along an Engel, could change along an Engel curve.
But the key point is that for most goods the Engel curve is upward sloping.
That is for most goods, this is greater than zero.
Just like we talked about the price elasticity being less than zero in general, this is less general, but for most goods, the income elasticity is greater than zero.
We call these normal goods.
Normal goods are goods for which the income elasticity is greater than zero.
As you have more income, you buy more of them.
On the other hand, if the income elasticity was less than zero, we would call those inferior goods.
Inferior goods, goods where as your income goes up, you buy less of them.
Yeah
Is there any term for when income elasticity equals zero?
If it equals zero, you're just income inelastic.
It's in between normal and inferior.
I don't think there's a precise term.
You're just income inelastic.
So can someone tell me how you could get an inferior good?
Does anyone have a good idea of an example of an inferior good?
How could a good be inferior?
Yeah
Canned food, because if you have a low income, you buy canned food because it's cheaper.
But [INAUDIBLE]
Exactly, so canned food versus fresh food.
As your income goes up, you'll substitute away from canned food to fresh food.
So actually as you get richer, you'll consume less canned food.
So canned food is an inferior good.
Excellent.
The classic example uses potatoes.
Potatoes is a good cost-effective, cheap source of nutrition.
But, you know, no one wants to eat potatoes all the time if they don't have to.
So when the income goes up, we substitute away from potatoes towards arugula or whatever.
So basically, essentially we could think of inferior goods as goods-- Once again, more is always better.
There's no goods we don't like.
More is always better.
But there are goods we'd like to substitute away from.
We'd like to have others instead.
And goods you substitute away from as you get richer are inferior goods.
Goods you move towards as you get richer are normal goods.
Moreover, we can break this down further.
Within the class of normal goods we can talk about gamma less than one and gamma greater than one.
Any guess as to what terms I'll use for gamma?
Any examples in, sort of, the class of goods where it would be less than one versus greater than one?
What's an example of a good that would be less than one?
Think about what that means.
Yeah
Perhaps food, because if your income [INAUDIBLE PHRASE]
Right
[INAUDIBLE] if your income increases
Excellent.
So we call these necessities.
And we call these luxuries.
Goods where the income elasticity is less than one are necessities.
You want more of them as you get richer.
But you don't want as much more as you get richer.
So you've got the food as the classic example.
As your income doubles, you're going to eat more food but not twice as much food.
Likewise luxuries are things where as your income doubles, you'll buy more than twice as much.
So you think about fancy cars.
You might buy one with your first million but three with your next million.
So those are luxuries.
They'll increase more than proportionally as your income goes up.
Necessities will increase less than proportionally as your income goes up.
Now, of course, it's a very hard distinction to draw.
And of course it varies by person.
So take clothing.
Is clothing a necessity or a luxury?
Well, some clothing is probably a necessity, and some clothing is probably a luxury.
You know, Dolce and Gabbana is a luxury.
You know, Keds is a necessity.
Or whatever, I don't know, what, The Gap is a necessity.
So basically we could think about, it's actually a subtle distinction what makes luxury and what's-- Normal versus inferior is kind of stark.
Luxury versus necessity, that's a little bit trickier.
That's going to depend on the person and depend on the type of goods.
But it's important to understand that concept.
OK, questions about that.
Yeah
I have a question in general.
Is there a way to relate income elasticity to, like, own-price elasticity?
Actually that's a great question.
That's a great segue to what we're going to do next.
It's actually a fundamental determinant of own-price elasticity, and we'll talk about why next.
Other questions about this concept?
Yeah
[INAUDIBLE PHRASE]?
Yes, income elasticity is, once again, just like price elasticities are not necessarily constant.
You could have a constant income elasticity curve or a non-constant income elasticity curve.
It can absolutely change.
In general it will.
But it might not.
Good questions.
But that other question in the back, gee, how does this relate to own-price elasticity?
Great segue.
In fact the income effect is going to be one of two key determinants of own-price elasticity.
Now what we're going to do is we're going to go even further behind the demand curve.
We talked about the demand curve comes from.
We talked before about how the slope of the demand curve is the price elasticity.
Remember, the price elasticity was the slope.
Now we're going to talk about where does the slope come from.
Where do price elasticities come from?
So I've shown you where the demand curve comes from.
Now let's talk about what determines the underlying slope of the demand curve.
What determines the price elasticity.
And what's going to determine it is two different effects which work generally together but sometimes in opposition.
There are two effects that determine price elasticity.
The first is the substitution effect.
The Substitution Effect is the change in quantity demanded when price increases holding utility constant.
So the Substitution Effect is delta p-- it's in percentage terms-- delta p over p, over delta q over q, holding utility constant at a fixed level u bar.
So given that your utility has not changed, how does your demand for the good shift?
We're now getting kind of deep.
Think of this as, as a good gets relatively expensive, how do you shift away from that good?
Think of this as the shift away from the good as it gets expensive.
But at the same time, there's a second effect which we just introduced, which is the Income Effect.
The Income Effect is the complement of the Substitution Effect, which is the change in quantity demanded because of a change in income holding prices constant.
So this is the Income Effect we just introduced.
So it's delta Q over Q over delta y over y.
But this is holding prices constant.
And these two put together determine your own elasticity demand.
Think of it intuitively.
We'll do it intuitively, and graphically, and mathematically.
Intuitively, it's when a price goes up, two things happen.
On the one hand, you're like, gee, at that different price ratio, I might want to substitute my consumption bundle.
The second is, gee, the price just went up, I'm effectively poorer.
And that's also going to affect my demand.
So to see that let's go to the graphical analysis and figure 6-4.
And we're going to actually decompose income and substitution effects.
This is one of the things that's sort of hard to do it intuitively.
The graphics kind of makes it the most clear.
We're going to start at a point like a.
Point a is our initial equilibrium at our budget constraint one, where we're choosing six movies and three pizzas.
Once again, this is pizzas not CDs.
We're choosing six movies and three pizzas.
That's at point a.
Now we're going to say imagine the price of movies has risen to $12.
Well we know that ultimately you'll end up at a point like c. We demonstrated that before.
That was when we derived the demand curve for movies.
We know that if the price of movies rises from $8 to $12, your demand for movies will shrink from six movies to four movies.
We already established that.
But now what we can see is that that's actually a composition of two effects.
The first effect is the Substitution Effect.
And the Substitution Effect is the change in prices holding utility constant.
How do we hold utility constant?
You have to be on the same indifference curve.
So the way to measure the Substitution Effect is we effectively draw an imaginary budget constraint.
We say, look, imagine you're on the same indifference curve, but prices changed.
So we draw budget constraint-- you're originally on budget constraint one.
Now we draw a new budget constraint, budget constraint three.
Budget constraint three, it's sort of hard to see.
Budget constraint three is drawn so that it has the new price ratio.
That is, it's parallel to the final budget constraint bc2, but it's tangent to the original indifference curve.
This is hard, so let me just walk this through again.
You've got your original budget constraint, bc 1.
You chose your point a.
Now the price of movies has just increased, so you move to bc2 in reality.
And at bc2, you choose point c. But for the Substitution Effect we're going to say, let's hold utility constant and ask what package you'd choose at these new price holding utility constant?
Well the way to do that is to draw an imaginary bc3-- bc3 never existed in reality-- that is parallel to the new budget constraint, that is the new set of prices, but it's tangent to your old indifference curve, that is utility is constant.
If that were the case, you'd choose point b.
You'd choose 4.89 movies.
Therefore we say that the Substitution Effect is 1.11.
We reduce your movies by 1.11 through the Substitution Effect only.
The price change only, holding your income, holding utility constant, is 1.11.
Then we say, well, what's the Income Effect?
The income effect is given prices are fixed, what's the effect of just being poorer, because movies are now more expensive.
We know how to make someone poorer, we just lower their income.
So we shift from bc3 to bc2 and from point b to point c, and we're done.
We get the total effect of the price.
So we have to shift from a to c is all that you see in the real world.
But behind that is the Substitution Effect which shifts you from a to b and the Income Effect, which is this parallel shift inwards of the budget constraint, which shifts you from b to c. I'm going to go through that again in a minute.
But let me first answer these questions, then I want to show you some of the math of this.
Are there questions about what's going on here?
Yeah
The income effect can have either sign, right?
The income effect can have either sign.
Excellent point.
What have I illustrated here?
What kind of good is this?
This is a normal good.
Excellent point.
This is a normal good.
I've assumed a normal good.
And this should actually say in the title income and substitution effects for normal good.
So I've assumed a normal good.
And we know this student aptly pointed out the normal good, because we know that as your income fell, moving from point b to c, you consumed less of it.
This comes to the question that was in the back, which is as the price changes, whether we consume more or less is related to the Income Effect but doesn't tell you precisely what the Income Effect is.
But as your income changes, if you consume more or less, that's directly Income Effect.
So we see it's a normal good.
So the Substitution Effect moves us from a to b.
The Income Effect, from b to c. What I want to do is one more thing before we stop.
And then I'm going to come back next time and go back over this and then talk about some applications.
What I want to do right now is I want to-- The Income Effect, say the following, the sign is ambiguous, because it depends on if it's a normal or an inferior good.
The Substitution Effect is unambiguous.
Substitution Effects are always negative.
Holding utility constant, a price increase of a good always shifts you away from that good.
Negative or less than or equal to zero could have no effect.
But once again, I always talk in inequalities even though they're typically inexact.
The key is the Substitution Effect is always negative.
We can think about this in two different ways depending on how you want to think about it.
Graphically we could show this by the fact that if the price-- think about it graphically.
If you're on the same indifference curve as you were before-- that's the definition of Substitution Effect where I keep the same indifference curve-- but you're tangent to a steeper budget constraint, which is what happened when prices go up, you must be choosing less of the good.
Think of it graphically.
Just look at the graph.
I'm in the same indifference curve.
But to be tangent to a steeper curve, it's going to have to be to the left.
So graphically it's going to have to be that I'll choose fewer movies when the price of movies goes up.
That's graphically.
Mathematically we can just say, look, what do we know is our rule for utility maximization?
We know our rule for utility maximization is that the marginal utility of movies over the marginal utility of prices equals the price of movies over the price-- marginal utility of pizza.
I'm sorry.
Marginal utility of pizza.
So the price of movies, so the price of pizza.
Or the MRS equals the MRT.
We know that mathematically.
That's our maximization condition right?
Well if the price of movies goes up, holding the price of pizza constant, then the right hand side has risen.
If the right hand side rises, then the left hand side has to rise to get this equality.
How does the left hand side rise?
The left hand side rises by either the marginal utility of movies going up or the marginal utility of pizzas falling.
And how do you do that?
By shifting away from movies towards pizza.
How do you make the marginal utility of movies go up?
Consume fewer movies.
How do you make the marginal utility of pizza go down?
Consume more pizza.
Now in this case, pizza doesn't change in this particular case, but in general it can.
But the key point is you are going to see this Substitution Effect is going to shift you towards fewer movies to try to get-- basically because given utility constant, to try to equilibrate this, you're going to have to move to a higher marginal utility of movies.
Given that the price of movies has gone up, you're going to have to move to worlds where you care about movies more, or you're not in equilibrium.
Think about it this way.
If the price of movies goes to $100 a movie, and you're going to be indifferent to where you were before, on the same utilities you were before, it can't possibly be true that you consume the same number of movies.
You'd have to be sadder if you consumed the same movies.
You're going to have to move away from movies.
And that's why the Substitution Effect is always negative.
You're always going to move away from the good where the price increases, holding utility constant.
But then we have the second effect with the Income Effect, which then basically, if the good is normal, reinforces that Substitution Effect.
It says not only do you not want movies because they've gotten more expensive, you also don't want movies because you're poorer.
Your opportunity set's restricted.
And when your opportunity set's restricted, you buy less of everything including movies.
So there's two reasons.
We only saw one own-price elasticity demand, one shift.
But there's two reasons behind that.
The first reason is because the relative prices have changed.
And the other is because you're effectively poorer.
You add those up, and you get the final effect.
Now the first, as I said, is unambiguous.
That Substitution Effect will always move you to the left.
You're always going to want less of a good if its price goes up from Substitution Effect.
The second is ambiguous.
That depends on whether the good is normal or inferior.
Next time we'll talk about what happens with inferior goods.
With inferior goods, we can see that we can actually get what we call a Giffen good.
A Giffen Good is a good where it's inferior, so the income effect goes the other way.
And you can actually technically get a good where when the price goes up, you actually want more of it.
When the price goes down you want less of it.
That is, you can get a wrong sign, wrong slope demand curve.
I say demand curves always slope down.
I might even refer to it as the law of demand.
That's not technically true.
Technically there exist goods, we'll talk about the next time, called Giffen Goods where you can actually get the demand curve sloping the wrong way.
The price increase can actually cause you to want more of it.
In fact, these are based-- I like the name Giffen because it's close to griffin.
And griffins are imaginary and so are Giffens.
In fact, there's no evidence such goods exist, but it's a nice theoretical concept to build your understanding of Income versus Substitution Effects.
So for next time think a bit about how that could be.
And we'll come and show you graphically how you can get Giffen goods depending on the sign of the Income Effect.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
All right.
So today we are going to start by reviewing income and substitution effects.
Because that's a pretty hard concept and pretty central to a lot of what we'll do for the rest of the semester.
And then we're going to dive in and talk about an application, a more interesting application, of income and substitution effects which is the effects of wages on labor supply.
So let's review.
If you take the handout, grab the handout and look at the first figure, it's the same as the last figure of the previous lecture.
To review, remember, whenever the price changes, a price change can be decomposed into two effects, the substitution effect and the income effect.
The substitution effect is the change in the quantity demanded when the price changes, holding utility constant.
And as we proved last time, that is always negative, 0 or negative.
It is always non-positive.
It's always true that when a price goes up, the substitution effect is negative.
We proved that both mathematically and graphically last time showing that if you're going to hold utility constant, and the price of a good is going to go up, you're going to shift away from that good.
OK. That's the substitution effect.
In our example, we showed graphically how you measure a substitution effect.
You draw a new imaginary budget constraint, BC3, which is parallel to the new budget constraint, BC2.
So it's got the new price ratio but tangent to the old indifference curve.
So the key thing to understand is the imaginary budget constraint, BC3, where it comes from.
It's parallel to the new budget constraint.
That is it's got the new marginal rate of transformation, the new slope, but it's tangent to the old indifference curve.
That gets you to point B.
And so the movement from A to B is the substitution effect.
Then we have an income effect which is, in fact, utility isn't held constant when prices change.
In fact, utility falls, because you're effectively poorer.
You're effectively poorer.
Utility is falling.
And since you're effectively poorer, that further reduces demand if the good is normal.
So if it's a normal good, if it's a good where lower income causes less consumption of it, the fact that you're effectively poorer further lowers the consumption from point B to point C. So the total price effect is the one we demonstrated at the beginning of the last lecture.
We raised the price of movies from $8 to $12.
And we saw the number of movies consumed fell from 6 to 4.
But what we can see now to understand what's underneath that is two things, an effect of the fact that prices change holding utility constant, and the fact that you're effectively poorer.
And that's the key thing.
No, your income hasn't actually gone down.
But that $96 your parents gave you can buy you less.
Your opportunity set has been restricted.
And that makes you effectively poorer.
And so you buy less for that reason.
And so you get the total movement from A to C. Now, as we emphasized last time, this will be the case if it's a normal good.
So substitution effects are done.
Substitute effects are always negative, nothing fun about that.
Income effects are a little more interesting, because goods can be not normal but inferior.
We have inferior goods which are ones such that they're crummy stuff that as your income goes up, you want less of it.
And that can change the analysis.
So if we look at Figure 7-2, now we're talking about the price change with an inferior good.
And now imagine someone choosing between steak and potatoes.
So now the choice is between steak and potatoes.
And steak costs $5 a pound, initially, and potatoes cost $1 a pound.
Initially, you have an income of $25.
So someone has an income of y equals $25.
The price of steak is $5, and the price of potatoes is $1.
So your budget constraint, your original BC1, runs from you can either have 5 steak, or 25 potatoes, or something in between.
And so individuals choose point A where they're consuming 8.3 potatoes.
They choose point A. I don't know what the number of steaks is.
We probably also ought to label that, the number of steaks that comes from that.
But whatever.
It comes out of the utility function.
So then we say, now let's imagine that the price of potatoes rises to $3 a pound.
There's a blight on the potatoes like there was in Ireland in the 1800s.
There's a potato blight, and that shifts in the supply curve for potatoes raising the price of potatoes from $1 a pound to $3 a pound.
Now, what we know is that that will move consumers, given the utility function that we've chosen here, that will move consumers from point A to point C. Once again, that's not labeled, but some lower amount of potatoes.
That will move them from point A to point C. So, ultimately, they'll choose fewer potatoes and fewer steaks.
But, in fact, what we can see is that's the composition of a substitution effect which is negative, and an income effect which is positive.
So if we do our standard decomposition, we draw a new monetary budget constraint BC3.
It's parallel to the new budget constraint, BC2, so the same price ratio.
It's parallel.
But it's tangent to the old indifference curve at point B which is actually to the left of the ultimate choice at point C. So the substitution effect takes us from A to B.
The income effect actually takes us back from B to C. That is as that budget constraint shifts from BC3 to BC2, as you get poorer, you choose more potatoes.
So the substitution effect would say that from the price change of potatoes alone, we go all the way to point B.
We massively reduce our consumption of potatoes.
But because we're poorer, effectively, we now consume more potatoes.
Because we're effectively poorer, we now consume more potatoes.
And so, on net, you get a reduction in potato consumption.
But it offsets the substitution effect.
So that's when income effects can be a little more interesting.
It's going to be a little more interesting exercise.
When you think about substitute effects in the same way, it's not that interesting.
It's just look, quantity fell.
It doesn't really matter why.
You don't see, in the real world, substitution income effects.
What's interesting is when they're opposed to each other.
That's when it gets more interesting.
And so you see this small reduction you get from the substitution effect alone.
By the way, there's two handouts.
Right?
Jessica, is there two handouts?
There should be.
There's tables as well.
I didn't actually get it.
Jessica, grab me one of those.
There's tables as well as graphs.
So make sure you have both handouts.
Anyone else need tables?
Am I the only one who didn't get it?
OK, good.
So, in principle, the income effect could be so large it could offset the substitution effect.
There's no reason, theoretically, that couldn't happen.
That is, in principle, you could derive preferences such that the income effect is so large it offsets the substitution effect-- thank you-- and the price increase actually leads to more potatoes being consumed.
That is what we'd call a Giffen good as I talked about last time.
So if you look at the table, the top table, this sort of lays out our possibilities.
So look at the top table.
It sort of maps out the possible sets of things that can happen.
So if we have a normal good, and the price of that good rises, then we know that the substitution effect is negative.
The income effect is negative.
So the total effect is negative.
Quantity falls.
That's the law of demand.
We talked about that last time, downward sloping demand curves.
Likewise, if the price falls, the substitution effect is positive.
The income effect is positive.
You're now richer because the price of the good fell.
And so, therefore, demand goes up.
Quantity consumed goes up, once again, downward sloping demand curve.
Price rises, you consume less of it.
Price falls, you consume more of it.
That's what we learned about in the first lecture.
However, once goods are inferior, all bets are off.
Because now the income effect is the opposite sign of the substitution effect.
It's possible it could be larger.
So the total effect is ambiguous.
You could actually get an upward sloping demand curve.
You could actually get that a price rise leads to more of a good, and a fall leads to less of a good.
Now will you?
Only if it's a Giffen good.
And, in fact, there's a lot of controversy in economics about whether any good in the world has ever been a Giffen good.
At most, there's maybe one or two examples people can find.
Even then, it's controversial.
So I think it's fine in life to assume that demand curves slope down.
I think, in fact, I don't see convincing evidence that any subset or set of goods are Giffen goods.
I think it's just generally fine it life to assume demand curves slope down.
Nonetheless, it's important to understand this theoretical possibility even if it's just theoretical.
Because it's important to understand income and substitution effects.
OK.
Questions about that, either on substitution effect or price changes?
OK.
So now, armed with that, let's go onto the more interesting case which is labor supply.
It's more interesting, because as I'll come to in a few minutes, we talk about labor supply, labor is typically going to be an inferior good.
So things are going to get a little more interesting.
So let's talk about that.
So the question you want to ask here is how hard do folks decide to work?
How many hours of labor do folks decide to provide?
As we talked about when we talked about minimum wage, just as we all have to decide between consuming pizza and consuming movies, or consuming steak and consuming potatoes, we also have to decide between how much labor we're going to provide and how much we're going to consume.
The more labor you provide to the market, the more you consume, but the less fun you get to have.
Fun, we call leisure.
The less labor you provide to the market, the more fun you get to have, the more leisure you get, but the less you get to consume, because you have less income.
And that's the trade-off we talked about when we talked about the effect of minimum wage.
Now let's come back and get underneath that labor supply curve.
So we talked about the minimum wage.
We talked about the labor supply curve which was how the hours you provide respond to the wage and a labor demand curve, which was how the hours that firms want respond to the wage.
Now let's get underneath the supply curve.
A minute ago, we were talking about the demand curves and getting underneath the demand curve for consumers.
Well, now let's get underneath the supply curve for labor.
Now, the key thing is that when we talk about labor, it's not a good, it's a bad.
The typical person doesn't want to work.
The typical person is not in this room.
You guys like to work.
The typically person actually doesn't like to work.
Leisure is a normal good.
For the typical person, leisure is a normal good.
They like time off.
Leisure is a good, which means labor is a bad.
They don't like to work.
The problem is we don't know how to model bads in economics.
It's just we're used to trading off between two things you want.
When I used to trade-off, we know how to model something you want to get something you don't want.
Indifference curves wouldn't work, because more wouldn't be better.
If you drew an indifference curve for labor, it would violate the more is better assumption.
Because you wouldn't want more.
You'd want less.
So the modeling trick we're going to use whenever we're modeling bads, is to model the complementary good and then, in the end, solve for the bad.
We're not going to model labor.
We're going to model leisure.
And given the total amount of hours you have to supply, the total hours minus the amount of leisure is the amount of labor.
So we're going to model leisure.
We're going to model the good and then solve for labor at the end.
So, in other words, if you have 24 hours a day you can work, then your amount of hours of work is 24 minus the amount of leisure N. Call it N or call it L. We'll call it N because L, typically, we think of as labor.
Let's call leisure N for reasons I don't quite understand.
Let's just use that.
Basically the amount of hours you can work is 24 minus leisure.
So if we solve for the optimal amount of leisure you want, we can obviously get the amount of labor you supply.
So the trick when modeling a bad is not to model the bad.
It's to model the complementary good.
In this case, the complementary good is leisure.
So we're going to model the trade-off between leisure and consumption and use the result of that to solve for the amount of labor you supply.
So it's the general modeling trick you need to understand, which is turn a bad into a good.
That's the modeling trick.
Because we know how to model the trade-off between two goods.
We don't know how to model the trade-off with a bad.
So to think about that, let's go to Figure 7-3, and let's talk about what's underneath a labor supply curve.
What's underneath a labor supply curve is the trade-off between how much leisure you want and how much consumption you can have.
So you see here, here's a trade-off.
On the y-axis is the amount of goods you can have.
You earn a wage, w. The y-axis is the amount of goods you can have from a day's work.
So you earn w per hour.
That means the most goods you can have from a day's work is 24w.
If you worked all 24 hours at that wage, you can have 24w goods.
On the other hand, if you work not at all, then you take 24 hours in leisure and have no consumption from that day.
So we see as you move to the right on the x-axis, that's leisure.
That's the good.
As you move to the left, that's labor.
That's the bad.
OK?
That's just illustrating.
But we're going to model the good.
We're going to model leisure.
Your trade-off is between how much you want to consume and how much leisure you want to take.
Now, here's what's interesting.
In general, what determines the slope of a budget constraint?
What determines the slope of a budget constraint?
Marginal rate of transformation
Which is what?
Ratio between prices
Ratio between prices.
Prices determine the slope of the budget constraint.
But here's what's tricky.
What's the price of leisure?
Wage
The wage.
Why?
Because for every hour you take having leisure, you are effectively using money that you could gain at work
Exactly.
The key is the economic concept of opportunity cost, which we've talked about and will continue to talk about this semester, opportunity cost.
By not working, you are forgoing earning a wage.
So that is the price of leisure.
You may not think of it this way, but, once again, that's why we're the dismal science.
When you go home today, and you sit on the couch, and you watch TV for an hour, you have just paid a price.
And that price is what you could have earned by working that hour.
Every action has a price.
And the price of leisure is the wage you forgo, The wage you forgo by sitting around is the price of leisure.
Let's assume here that the price of goods is $1, that the goods you're going to buy cost $1.
Whatever your consumption, it costs $1.
That's the trick we always use with modeling.
Make as many things $1 as you can.
That makes the model easy.
So let's assume that the price of the goods you're going to buy are $1.
So the slope of the budget constraint is minus w over 1.
The slope of the budget constraint is just the price of leisure which is minus w .
So the trade-off with the price of goods of $1, the trade-off between taking leisure and consuming is that if you take leisure, an hour of leisure, you get w fewer goods.
And if you work an hour, you get w more goods, but you lose an hour of leisure.
And that gives you the trade-off between how much you consume and how much leisure you take which determines how much you work.
OK.
Questions about that?
Now let's take this framework and ask, what happens when the wage changes, Figure 7-4.
So we have an original outcome with the budget constraint BC1.
We have an original budget constraint, BC1.
Now imagine the wage goes up, so we move to BC2.
BC2 is a budget constraint with a higher wage.
The wage goes up.
So what we're going to see is you're going to move from point A where you work N1 hours to point C where you work N3 hours.
That's where your indifference curves are tangent with the new budget constraint.
Not work, take leisure.
I'm sorry.
We take leisure of N1 hours to leisure of N3 hours.
The wage going up has reduced your leisure which makes sense.
If the wage goes up, you work harder.
Right?
So your wage going up, we always first take if there's a leisure and then convert to labor.
Wage goes up, leisure falls from N1 to N3, which means labor goes up.
But actually two things are happening here, the substitution and income effect.
The substitution effect, which we see by drawing the imaginary budget constraint BC* which is parallel to BC2 but tangent to the original difference curve, the substitution effect is a very large reduction in leisure.
It moves all the way from N1 to N2.
The substitution effect is a very large reduction in leisure.
The income effect is that leisure is a normal good.
I'm now richer, because my wage has gone up.
So I want to buy more of it.
So I buy more leisure.
And that moves me from N2 to N3.
So, basically, now the income effect offsets the substitution effect even with a normal good, or with a normal good.
With a normal good, the income effect offsets that substitution effect.
And that's because the money you're getting, you're using to buy leisure.
So, in fact, if you flip to 7-5, you can see a case where the income effect dominates.
And you actually get that a wage increase leads you to work less hard.
Now, think about that.
If I'd said to you-- I probably should have started with this-- if you increase the wage, will people work more or less hard?
Your initial instinct would have been more hard.
You would have thought, well, if your wage goes up, you work harder.
But that's because your instinct was focused on the substitution effect.
You're thinking about the income effect.
Here's a case where I started at N1.
The substitution effect leads me to N2.
But I feel so much richer from that higher wage that I actually move all the way to N3.
My leisure goes up, and I work less hard.
Now, unlike a Giffen good, this is totally plausible.
Why is it plausible?
Well, let me do give you a simple intuition for why it's plausible.
Let's say that you're someone who has a certain amount of things you want to buy every week.
You don't save.
You have a certain amount of things you want to buy every week.
You have to pay your rent, you have to buy your food, you have to buy your other goodies, a certain budget.
A lot of people live on a budget.
You have a certain budget.
And the truth is you're happy with that budget.
That's kind of what you want to do.
Now let's say I doubled your wage.
Well, now to meet the budget you can work half as hard and still meet the same budget.
So you'll work less hard.
You could say, look, I can get more leisure and consume the same amount of goods as I did before.
So I'll work less hard.
That's a totally plausible case.
That's a case of what we call target income.
If someone has a target income, and their wage goes up, they'll work less.
Now, that's not necessarily the truth.
But it's, at least to me, sort of a plausible case of how people might behave.
And that's a case where income effects can dominate.
So if we, once again, go to the second chart on that page, now we see the income and substitution effects for labor supply.
Once again, we're assuming leisure is a normal good.
We're always going to assume leisure is a normal good.
We're never going to assume people don't like leisure.
Assuming leisure is a normal good, then as the wage rises, the substitution effect is you take less leisure.
This table is a bit different than the other table.
Instead of the first panel being normal and the second panel being inferior, the first panel is what happens to leisure.
The second panel converts it to what happens to labor.
So for instance, in the first cell, when the wage rises, the substitution effect on leisure is unambiguously negative.
You clearly take less leisure when the wage rises.
So, likewise, you have more labor.
So on the bottom panel, labor is clearly greater than or less than 0.
But the income effect is positive for leisure.
You're rich, you take more leisure.
Or, likewise, negative for labor, you're richer, so your work less hard.
And, therefore, the net is ambiguous.
So with goods consumption, we needed goods to be inferior for there to be a Giffen good type phenomena.
Here, even with leisure being normal, you can have a Giffen good type phenomena.
It's much less random.
And, in some sense, this is why we learn income substitution effects.
To be honest, they're just not that interesting for consumption.
The book makes a big deal out of them and talks about consumer price indices and all that.
It's just not that important for consumption.
Because we know in consumption if prices goes up, you consume less.
It's just not that interesting.
It's much more interesting for things like labor supply.
And we talk about savings in a number of lectures.
It's the same thing.
There, it's more interesting.
Because now they can often offset each other in meaningful ways.
And so now this is why the tools of income and substitution effects become much more important.
OK?
So if we put this together, if we go to Figure 7-6, we can now think about deriving where labor supply comes from.
Where does labor supply come from?
Well, first, you've got the consumer's decision of how hard to work.
So here's a case.
It's sort of small, but you can take a look.
Here's a case where you've got someone initially working, taking 16 hours of leisure and, therefore, working eight hours, at a wage of W1.
Now their wage goes up to W2.
They choose to take 12 hours of leisure and, therefore, work 12 hours.
This is someone who works harder when the wage goes up.
That is, the income effect does not offset the substitution effect.
Now, we can take that to draw a demand for leisure curve just like we drew any other demand curve.
It's the same technique as last time.
Just bring those point and say, look, at a wage of W1, leisure is 16.
At a wage of W2, leisure is 12.
We have a downward sloping demand for leisure, standard downward sloping demand for leisure.
But we can convert that to a supply of labor, which is what we care about.
Nobody cares about the demand for leisure curve.
We care about the supply of labor curve.
You just subtract these from 24.
You use the supply of labor curve which is upward sloping.
So as long as substitution effects dominate income effects, we'll get an upward sloping labor supply curve.
But it's certainly possible that if income effects dominates substitution effects, you could get a downward sloping supply curve, if you will, what we call in labor economics, a backward-bending supply curve, a supply curve that goes the wrong way.
Instead of sloping up like supply curves are supposed to, it goes the wrong way and slopes down.
And we can see that's plausible.
The target income case I just described to you would deliver that.
The target income case I just described to you would deliver a downward sloping supply of labor.
As the wage rose, people would work less and less.
That's a totally plausible case.
And that's why income and substitution effects are interesting.
Because they can deliver this weird result.
They can get the wrong signed supply curve.
Questions about income and substitution effects or labor supply?
So what I want to spend the rest of the lecture on is talking about well, what is the case?
Do labor supply curves slope up or down?
And what do we know about that?
Well, this is probably the major focus of a field we call labor economics.
And there's an excellent course on labor economics, 14.64 taught by Josh Angrist, which goes into much detail in the entire field.
But one of the main focuses of the field is understanding the elasticity of labor supply, and is it positive or negative, and how big is it.
So, basically, measuring the slope of the labor supply curve is the focus of this literature, the elasticity of the labor supply.
Now, what I want to do is start with a historical fact, and then I'll come to the modern age.
Let's think about 30 years ago.
30 years ago, all men worked and less than half of women worked.
It was more normal for women not to work than to work, married women.
I'm sorry.
Less than half of married women worked.
Now, married women could work.
I'm not talking 60 years ago or 80 years ago when there were marriage bars.
Literally, firms wouldn't hire you if you were married.
It's true.
If you're interested in that, you can actually read Claudia Goldin.
She's a labor historian who's written about the early 20th century when, literally, women could be fired for being married.
We're not talking about that era.
I'm talking about 30 years ago when you could work if you were married.
It's no problem.
But most women chose not to, maybe 40 years ago now.
So, in that case, let's think about two groups.
Let's think about married men, and let's think about married women.
And let's just posit, hypothetically, how big we think their substitution and income effects would be.
Let's start with substitution effect.
Do we think the substitution effect would be bigger?
This is the change in the wage holding utility constant.
Do we think that would have a bigger effect on leisure and, therefore, labor for men or for women and why?
Don't yell it out.
Somebody, raise their hand and tell me.
Do we think that the substitution effect would be bigger for men or women and why?
Remember the name.
It's the substitution effect.
That's the key to the answer.
Yeah
I think it would be the same, because they each have equal use for the goods.
Maybe their income effect would be different
OK. [UNINTELLIGIBLE PHRASE].
They each have equal use for the goods.
Well let's deal with where the substitution effect comes from.
Let's break it down.
So you're someone who's deciding.
You've got you and your wife, and you're each deciding how to respond to a change in the wage.
Now, you both value the goods the same.
But it's goods versus leisure.
What's the other feature that you're going to be thinking about?
Think about a married man 40 years ago, and the wage goes down
They have to work more
No.
We're just doing substitution effects.
That's unambiguous.
If the goes down, they work less.
We're just doing substitution effects.
That's ambiguous.
The question is, if they work less, what do they do?
Whereas think about a married women 40 years ago.
If she works less, what does she do?
What does a married man do?
Nothing.
There's nothing to do.
Your friends are all at work.
You can't go play golf.
You can't do anything.
You don't take care of kids, because men didn't take care of kids 40 years ago.
What do you do?
There's nothing to do.
Whereas a woman, married woman, if the wage goes down 40 years ago, you can take care of kids instead.
You can hang out with other women who aren't working.
There's plenty to do.
Based on that, now change your answer.
Where do you think the substitution effect would be bigger?
[INAUDIBLE PHRASE]
In women, it would be bigger.
Because men, there's less of a substitution effect.
Because it's all about substitutability of options.
There's no good alternative option to work for men 40 years ago.
It's either work or nothing.
Basically, everybody worked.
So, basically, there's no good substitution effect option for men.
For women, there's lots of outside options.
There's sociability, there's child rearing, et cetera.
The substitution effect will be larger the more things there are you can substitute to.
Men don't have anything to substitute to from work.
Women have options to substitute to from work.
For men, this is going to be very small.
For women, this will be big.
We know the sign.
The smallest this can be is 0.
We know the sign.
But it's going to be a very small substitution effect, because I don't have a lot else to do if my wage goes down.
Women, if it's a low wage, why work?
You can be much more effective taking care of the kids or hanging out with your friends.
Why work for a low wage?
Men, there's nothing else to do.
So that's the relative size to the substitution effects.
Now, the income effect, I think, is a little bit harder.
And let's come back and think about what drives an income effect.
We talked about the income effect as being delta q over delta y, how much a quantity changes when your income changes.
But, in reality, what's going to matter for your income effect given when you start today, is going to be not only delta q over delta y, how your taste for work changed or income changes, but also how hard you worked to start.
Think of it this way.
The income effect is how much richer you feel if your wage goes up, or how much poorer you feel if your wage goes down.
If you are working 0 hours, the income effect is 0.
You don't feel any richer if the wage goes up, because you don't earn any money.
The more hours you work the bigger the income effect is, because the bigger that shock is to you.
So we can think of the income effect, a shorthand for the income effect, is going to be h times dh/dy, the hours you work times how your hours change with your income.
OK?
Now, to prove this it involves using complicated algebra.
We're not going to get into it in this course.
I worked hard last night to see if I could make the algebra less complicated, and I can't.
I just have to try to work intuitively on this.
The notion of the income effect is bigger the more you're in the market.
You can think about it for goods too.
Think about the income effect of a change in the price of something you buy a lot of for something you buy very little of.
So let's say you're someone who's buying two Starbucks a day, and you very rarely go see a movie.
Well, if the price of a movie goes up 10%, or the price of Starbucks goes up 10%, which is going to make you feel poorer?
The price of Starbucks going up, because you buy a lot of Starbucks.
So how much poorer you'll feel, or the income effect, will depend on your starting point.
The more you're in a market, the more you'll feel the income effect.
Now, based on that, who's going to have a bigger income effect, men or women?
Same person, what do you think?
The income effect is going to be stronger the more you're in the market.
So who's going to have a bigger income effect?
Married men would have a bigger income effect
Exactly.
Married men would have a bigger income effect, because they're in the market.
Married women, most of them don't work.
So there's no income effect.
So this is going to be big for men and small for women.
There's another issue, which is does dh/dy differ for men and women?
I'll leave that alone.
Let's assume they both have the same underlying income elasticity.
But, certainly, the initial hours are much bigger from men than for women.
So what does this mean in terms of the labor supply curves you would see for married men and married women 40 years ago?
Based on these facts, what would you think?
Yeah?
They would have opposite slopes
Yeah.
So, in particular, the female labor supply curve would look like what?
It would slope up or down?
It would slope up
It would slope up.
You'd have an upward sloping curve, because you'd have these big substitution effects and small income effects.
So it would look much more like Figure 7-4.
You'd have the big substitution effect when the wage goes up and a small offsetting income effect.
Think about the woman who is not working at all.
She's now working at all at $8 an hour.
You raise her wage to $12 an hour.
She's like, hey, I wasn't working at all.
So there's no income effect.
But now I'm going to go to work and make some money.
So it's upward sloping.
But for men, it's going to look more potentially like Figure 7-5.
There's a small substitution effect but a potentially big income effect or bigger than women.
Now, how big it is, that's not clear.
Because, once again, men have nothing to do if they don't work.
So it could be this ends up being bigger and smaller.
It's not clear how big this ends up being.
But it's at least possible that you could have men having a backward-bending or downward sloping labor supply curve.
Because the income effect could even more than offset the substitution effect.
But, in reality, given the way I set up the example, you'd think men would basically have a pretty inelastic labor supply.
You'd think, 40 years ago, these things would basically both be 0, both offset.
And, basically, you'd have a situation where the change in the wages didn't matter much for men.
And, in fact, that's what people found.
This is a wonderful case of the convergence of truth with theory and a wonderful chance to see the power of some pretty simplistic theory.
The intuition is exactly borne out in the data, which is males, 40 years ago, would have a very inelastic labor supply.
Their labor supply curves were virtually vertical and maybe backward-bending.
There's some controversy on that.
Some estimates got backward-bending.
Some didn't.
But there were certainly not upward sloping.
It was basically vertical.
Women had a very few elastic labor supply.
The elasticity is estimated to be around 1.
That is every 1% change in the wage lead to 1% more labor supply.
So that's a fairly elastic labor supply for women.
Where, for men, the estimate was basically 0.
And that's kind of neat, because we're actually getting confirmation in the data of what the theory would have told us.
Now, someone else tell me what do you think has happened over the last 40 years relative to these elasticities of married men and married women.
Yeah
The elasticity of married women has gone down in the last 40 years--
Why?
Speak up so the class can hear you.
Why is that?
Because women work more often now than they did before
Women work more often now.
So the income effect is going to be getting bigger for them.
So the income effect is going up, because their initial h is bigger.
Plus there's actually now, in some sense, less good opportunities if you're not working.
So when we had our first kid, and we lived in Brookline, which is sort of an urban city, and my wife decided to stay at home, she didn't have moms to hang out with.
It was just nannies at the park.
And it wasn't that much fun.
And so, basically, the substitution effect is shrinking, because the outside options aren't quite as good as they were, as the norms shift towards work.
Whereas for men, actually it's becoming more normal for men to be engaged in child care.
My best friend is a stay-at-home dad.
It's becoming more normal for that to happen.
And so the substitution effect is rising.
It's not implausible that if you cut a man's wage down, he'll just say forget it.
My wife's going to work.
I'm taking care of the kids.
That would be socially ostracizing 40 years ago.
But it's not that odd now.
And, likewise, as men are less engaged in the labor force and spending more time at home, their income effects are falling, because their initial h is smaller.
So you're getting a convergence in these labor supply elasticities.
What really seems to be happening is mostly convergence down for women, not much up for men.
So men are maybe going from 0 to 0.1.
Women are coming from like 1 to 1/2.
So what you're seeing is that men aren't actually working that much less.
There's a few stay-at-home dads.
But they're still not the majority.
Women are working a lot more, and kids are in child care a lot more.
So what you're seeing over time is you're seeing men being a little more responsive, but not that much more responsive.
They're still, basically, working all the time.
Women are working a lot more and being more responsive to wages.
And there's a reduction coming in both women's leisure and production of child care at home.
Now, that raises a very interesting question of is this is a good thing?
Now this is a very deep and hard topic.
In economics, we think if people do something it's good, or they wouldn't have done it.
It is true that if you look at data on self-reported well-being or happiness data, married women report a general decline in happiness, over the last 40 years, as they've entered the labor force more and more.
And the issue is, is this something which is a good way for society to spend its resources, to have everyone working?
We're consuming more.
Consumption has gone up.
We're consuming more, but we're getting less leisure as a family.
Because the men aren't working that much less.
The women are working a lot more.
So we're getting less leisure as a family.
How do we feel about that outcome.
That's an interesting question.
And we'll talk about that some more later on in the semester.
OK.
Questions about this?
Yeah
[INAUDIBLE PHRASE]
That's a really good question.
And let me talk about that for a couple of minutes.
The definition of unemployment is those employed over looking for work.
If the number of people employed does not change, and women suddenly want to work, and they report to surveyors that they're looking for work-- that's the employment rate.
I'm sorry.
The unemployment rate-- I'm sorry-- is going to be those looking over employed.
My bad.
The unemployment rate is going to be those looking over those employed.
So the unemployment rate is how many people are looking for work over how many are employed.
If women start suddenly looking for work, and there's no jobs to be had, that will raise the unemployment rate.
So one thing that's been a focus of a lot of research has been do increases in the supply of labor lead to increases in unemployment?
What you've expressed is what's often called the lump of labour view.
The lump of labour view is basically the view that there's a fixed box of production in the economy.
And as more workers come in to fill that box, there will be more unemployment.
The alternative view is that the economy is dynamic.
And as more women are working, and earning income, and buying stuff, that makes more jobs.
So our standard of consumption is way higher than 40 years ago.
We all have much cooler stuff than 40 years ago.
You have no idea how bad life sucked 40 years ago.
We have way better stuff.
We have that stuff, because women are working and making the income to buy it, which means people have to make it which makes jobs.
So, in fact, the existing evidence is labor supply shocks do not cause unemployment increases.
This is something I've worked a lot on.
What you see, a very interesting case is in Europe.
In the US and all over the world, we have assistance of what we call Social Security, a term you've all heard, I'm sure.
The Social Security program is a program which provides income when you're retired.
So it provides income when you're retired to help you deal with the fact that you don't have a source of labor income anymore.
And that's a program that virtually every country, and all developed countries have a very generous social security program.
But they're different in the US than in other countries.
In the US, the way the social security program works is when you hit 62, you get a choice.
You can stop working and get your benefits from Social Security, and then you get them every year until you die.
Or you can keep working, delay getting your benefits, but they'll increase what you get to offset the delay.
So, in other words, if I retire at 63 rather than 62, given that I'm going to die at the same date, I'm going to get one fewer year of benefits in my life.
But they raise them by 6.7% to compensate for that.
So I get one fewer year of benefits, but every year it's 6.7% higher.
And it turns out, given life expectancy, that works out to be a roughly fair deal.
So, basically, at 62, your choice is I can get one more year of benefits or I get higher benefits for one fewer years.
And that's a choice that's a roughly fair deal.
OK.
Questions about that?
Am I making sense of that?
In Europe, it's not a fair deal.
In Europe the way it works is they say, you can get one more year of benefits.
But if you decide to work this year, we're not given you any more in the future.
So let me describe how it works in the Netherlands.
At age 55, the Netherlands says, if you decide to retire this year, we will replace 90% of your wages in social security payments to make sure your income doesn't suffer when you retire.
If you don't retire and work, you're going to give up sitting at home earning 90% of your wage.
That is the opportunity cost of working.
It's that you have forgone the ability to sit at home and get 90% of your wage.
So what is your net wage if you work?
10% of what you would have earned.
So if you're earning $20 an hour, then your choice is you can sit at home for $18 or work for $20 an hour.
So your net wage for working is $2 an hour.
The return to work, the opportunity cost of leisure is only $2 an hour.
You're only forgoing $2 an hour by sitting at home.
But wait, there's more.
If you sit at home, you don't have to pay the payroll taxes of financing the system that are almost 50%.
If you work, you have to pay the payroll taxes.
Which means that if you work, you lose money.
Because if you work, you forgo getting to sit at home at 90% of your wage, and you pay a tax that's about 40% of your wages.
So, actually, you will lose 30% of your salary by working relative to sitting at home.
Guess what people do in the Netherlands at 55?
They sit at home.
No one works after 55 in the Netherlands on the books.
They work off the books painting houses and doing odd jobs.
No one works on the books after 55.
Economics works, guys.
If you pay your guys to stay at home, they stay at home.
Now, if you ask European politicians, why do you have this screwed up system?
They'll say, well, it's easy.
We want to get those old guys out to make jobs for the young guys.
We need to pay those old guys to stay at home to make jobs for the young guys.
And then you point out, have you noticed that Europe has higher unemployment than American, even though we don't do that and you do?
And that's because you're wrong.
It doesn't work that way.
Because by paying the old guys to sit at home, you have to have such high taxes that no one makes new businesses.
And so there's not jobs for the young guys to have.
So it's true.
In theory, you've made jobs for the young guys by leaving the old guys at home.
But by imposing the 40% tax rate that you've had to impose to make it possible to pay the old guys to sit at home, you've killed job creation in your country.
And, as a result, there's not the jobs for young guys to get.
That's a very long-winded way of answering your question that supply, in substance, creates its own demand.
So more labor supply will not necessarily cause more unemployment.
And we're going to talk about one more thing before we stop.
I've just talked about a vast empirical literature in how people understand the effects of wages on labor supply.
Well, how do they do it?
Well, you could say, look, we can just look at how you earn a higher wage than you do.
And we'll ask, do you work harder than you?
And we'll say, the guys who earn higher wages work harder.
If guys who earn higher wages work harder, that means labor supply slopes up.
If guys who earn higher wages don't work harder, that means labor supply slopes down.
What's wrong with that?
Yeah
Those who are getting paid more probably are getting paid because they want to work harder
Yeah.
Maybe you guys are different.
Maybe you're talented, and you're not.
And maybe because you're talented, maybe you're driven, and you're not.
And because you're driven, you work harder and get paid a higher wage.
So I'm not learning anything about the causal effect of the wage on your labor supply.
I've just documented a correlation between wage and labor supply.
How can we get the causal effect of your wage on your labor supply?
Well, once again, ideally we'd run an experiment.
We'd assign you a higher wage.
We'd find someone just like you.
Not you, you're not driven.
We find someone just like you.
No offense.
You know I'm joking.
We'd find someone just like you and, randomly, by a flip of a coin, assign them a lower wage.
And we'd see how your labor supply differed.
Now, it seems like you couldn't do that.
But, in fact, the US did that.
In the 1970s, we ran what was called the negative income tax experiment where we literally assigned people different wage rates through taxing them by different amounts.
And that was part of what gave us this very convincing evidence from 40 years ago of these responses.
So where we get this is from a real experiment we ran 40 years ago.
The problem is that's a pretty hard experiment to run.
It's pretty expensive, and there's some ethical issues.
So what do you do today to estimate that?
What you can do today is say, well, we can't run the experiment.
But the government runs it for us every time they change tax rates.
Because if you take two people that are identical-- so let's say you and you were identical-- and I change your tax rate because you live in Massachusetts.
I don't change your tax rate because you live in New York.
I can see what happens to you relative to you.
Because I've now essentially run this experiment by the government changing someone's tax rate and not someone else's.
That's the way we do it if we can't run a true, randomized experiment.
And that gives very, very similar answers.
Let me stop there.
And we will come back.
Next lecture we'll talk about applying this model.
So I guess in section on Friday, we review for the exam.
In section on Friday, we review for the exam.
So show up to that.
And the exam is next week.
The exam will cover through my next lecture.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now what I'm going to do is, I'm going to turn and start producer theory, OK, which will not be in the exam, but be the next subject to the course.
OK, and we're going to spend a little bit longer on this.
So what we've been doing so far is saying, look, we introduced the course with supply and demand curves, elasticities, all that.
Hint, all that's on the exam too.
OK, and all that.
Then we say, well gee, where do supply and demand curves come from?
Well, demand curves come from utility maximization.
And we talked about how where indifference curves come from, how the tangency with the budget constraints leads to the demand curve.
That's where demand curves come from.
We talked about what underlies demand curves is income and substitution effects.
OK, now let's come to supply curves.
What underlies supply curves?
Now on the one hand, this will be much easier than demand curves, because a lot of the logic is the same.
So we can march more quickly through the analysis, because it's basically the same kind of tangency of curves with straight lines that yield supply curves that we've seen with consumer theory.
On the other hand, supply curves are a ton harder, because now we don't just have the price as given to consumers.
The suppliers actually make up the price.
With consumers, we said the price -- you just went to the store.
you're given a price.
And you choose what to buy at those prices.
Well, who set those prices?
Producers do.
And that's what determines the underlying supply curve.
So life gets a little bit more difficult with producers.
And that's why I'll probably spend about twice as many lectures talking about producer theory as we've spent talking about consumer theory.
Now, let's go to the basics.
The basics are, just as we had consumers making decisions, we thought of a consumer as somebody choosing across a bundle of goods, pizza versus movies, now we're going to think of a producer very simply as a black box, where inputs go in and outputs come out.
So think of the firm.
Let's think of a firm, a producer, as just some black box.
We're literally thinking a flow chart.
You've got inputs that go in and outputs that come out.
And that black box, that firm, just as individuals have a simple goal which is to maximize their utility, producers have a simple goal which is to maximize their profits.
And profits are defined as revenue minus cost.
OK?
Producers are these black boxes, where their goal is to maximize their profits, which is revenue minus cost.
And the key to maximizing profits is efficient production.
The key to maximizing profits is going to be to produce goods as efficiently as possible.
OK, so profit maximization requires production efficiency.
To maximize your profits, you need to produce as efficiently as possible.
Now, you might ask yourself, gee, I can read about all these companies that have executive corporate jets, and the guys in these lavish lifestyles, and that doesn't seem very efficient production.
I'll come back to that.
OK, in a couple of lectures, we'll talk about do firms actually maximize profits, and whether they do or not.
But for now, let's take as a given that they do.
OK, just as it is a given that consumers maximize utility, let's take as a given that firms maximize their profits.
OK?
Now, to decide how to efficiently produce goods, we're going to turn and today discuss firm production functions.
OK, today, we'll focus on production functions.
That's essentially the technology by which a firm takes inputs, or what we call factors of production, and turns them into outputs, is through this production function.
So just like your utility function is a tool for which we take bundles of goods and turn them into happiness, a production function is a tool for which we take bundles of inputs and turn them into outputs.
OK, but something that's a little easier to understand this, you could think of a factory where stuff comes in, and think of a belt, a mechanical belt, going through and stuff goes in and other stuff comes out.
You could think of that production function.
We're going to think about two different kinds of inputs that firms are going to use.
Once again, to make life easy, we're going to assume firms only use two kinds of inputs.
Later, we'll expand this, but from mostly intuition you can get from thinking about this-- two kinds of inputs, labor and capital.
Labor and capital are the two kinds of inputs firms use.
Labor is clear.
It's just hours of labor, hours of work.
OK?
It's just hours of work in production.
Capital is a lot trickier.
And we'll spend a lot of time this semester struggling with what capital really means.
Basically for now, think of capital as everything else that goes into production, the machines, the buildings, the land, everything.
Think of capital as everything else that goes into production.
So basically, when you produce stuff, you produce it with workers working with stuff.
Capital's the stuff.
Now, this capital is sort of a composite of everything else that goes into production.
And once again, we'll add some more detail on this later.
OK, and the output is some output, q.
That's the units of production.
That's the output that is produced by the firm.
So basically, we can think of a production function is q is some function of l and k. That's a production function, that the output you produce as your firm-- and by the way, we're going to use little q to represent a firm's output, and big Q to represent market output.
OK, so little q, and if I slip in this, yell at me, OK?
But we're going to try to consistently use little q to represent a given firm's output, and big Q to be the market's output-- So little q is some function of the amount of workers you have and the amount of capital you use.
Now, the important distinction we're going to make here is between variable versus fixed inputs.
Variable versus fixed inputs is an important distinction we're going to draw.
Variable inputs are inputs that are easily changed, like how many hours somebody works.
In principle, you could just have some work five hours one day, one hour the next day, 10 hours the day after that.
It's easy to change hours of work.
So that's a variable input.
Fixed inputs are things which are harder to change quickly, like the size of the building that the workers are building in.
Once that building's built, it's pretty hard to change it.
You can't like lop off 2/3 of it on one day and add it back the next day.
OK, that's more of the fixed inputs.
And this will lead to a critical distinction for production theory, which is the short run versus the long run.
The short run versus the long run.
The long run is the period over which all inputs are variable.
The short run is a period over which some inputs are fixed.
Let me say it again, it's very important.
The short run is the period over which some inputs are fixed.
The long run is the period over which all inputs are variable.
Now, what does that mean?
I can't tell you.
OK?
I can tell you that, clearly, tomorrow is the short run.
Clearly, you can't vary all your inputs over one day.
And probably next month is the short run.
And probably even next year is the short run.
There's a lot of inputs that it takes more than a month to change, or a year to change.
On the other hand, 10 years from now is almost certainly the long run.
There's very few inputs to production you can't change over a 10-year period.
So we know a day are the short run.
And we know 10 years is the long run.
We don't really know where the transition is, but in substance that doesn't matter for you guys.
What matters is the definition of short run, long run.
When someone asks you what the long run is, you say it's the period of time over which all inputs are variable.
That's what matters.
So it doesn't matter if you define it in days or months or years.
It's a theoretical concept, which the break from the short run to the long run is the break between when some inputs are fixed and all inputs are variable.
OK?
Now of course, once again this is tricky.
And economists recognize this subtlety.
A lot of times, economists will talk about quasi-fixed factors of production, which are things which could change in between the short run and the long run.
So for instance, take labor.
OK, we'll talk about labor as a variable unit of production you can change in the short run.
But in fact, we know in practice you can't.
Most jobs, you can't ask the guys to come for an hour one day, and 10 hours the next day, five hours the day after that, maybe some jobs like hourly construction.
But most jobs you guys will have, OK, your labor isn't that variable.
We're typically on some kind of reasonably set work schedule.
Now that work schedule can evolve.
OK, but it can't change day by day.
None of us are going to have jobs where they're going to say, look, work two hours today, 12 hours the next day, et cetera.
OK, we're all going to have jobs with a fairly smooth distribution.
There'll be peaks and valleys, but a fairly smooth distribution of our labor effort.
So really, truly, there's very few inputs which are truly perfectly variable.
OK?
Just like there are no inputs which are truly perfectly fixed, but for the purposes of this model, let's think about labor as a variable input.
Let's think about labor as being like hourly labor, like hourly construction, OK?
And let's think about capital as being a fixed input, like a building that you can't pare down or rebuild overnight, but over a 10-year period you can.
OK?
Questions about that?
Once again, we talked about simplifying assumptions at the beginning of the semester.
This is our set of simplifying assumptions.
Simplifying assumptions are that firms produce goods with two inputs, labor and capital, that labor is variable, which means you can change it minute to minute or day to day, and capital is fixed, which means you can only change it in the long run.
OK?
All right, so armed with that, let's now talk about how firms make short-run production decisions.
And once again, these are a lot of assumptions.
But at the end of the day, I'm going to be able to teach you in a couple of lectures how firms make decisions.
And I'm going to be about 80% right.
So that's pretty good.
OK, so there are some assumptions, but we're going to go a long way with these assumptions.
So let's start by considering the short run, and considering that period of time over which labor is variable but capital is fixed.
Labor is variable but capital is fixed.
That is, you have a given plant, but you can adjust how many workers you use every day in that plant.
And now the firm has to decide, given that that plant exists, how many workers should I hire to produce my good?
How many workers should I hire to produce my good?
And the key concept that's going to determine that is something we'll call the marginal product of labor, which is the change in total output resulting from the next unit of labor used-- that is, delta q, delta l-- is the marginal product of labor.
It's going to be the change in total output from another unit of labor, once again holding capital constant because that's fixed.
So really, it's really at a given k bar, but that's implicit in the fact it's the short run.
So the given level of capital, what is the change in output for another unit of labor?
And once we get to the short run, I shouldn't have to write this.
If I tell you short run, you should know this is true.
But technically, we would write that out.
Now, what we're going to do here is we're typically going to assume this-- hint, hint-- if you're comfortable with consumer theory, this is like marginal utility.
Marginal product is like marginal utility.
Just as the marginal utility was your utility from another unit of one good, holding the other good fixed, marginal product is the marginal production from another unit of an input, holding the other input fixed.
So just sort of a parallel to keep in mind.
And just as we've assumed and discussed the intuition for diminishing marginal utility, we're going to typically assume diminishing marginal product.
So we're going to assume diminishing marginal product.
That is, from a given level of labor, the next worker you add increases your total product by less than the previous one.
Now once again, just like we have a non- satiation rule in utility, we're not going to say the next worker doesn't help.
Every worker helps.
But every worker helps less and less and less, just like every pizza means less and less and less to you.
Now, the trick here, once again, this is like consumer theory, only more subtle.
With consumer theory, it's pretty clear that once you had one pizza, clearly, the second pizza means less.
And once you've seen one movie, your first-place prize, the movie you most want to see, clearly the next movie you want to see means a little bit less.
Producer theory is a little more subtle.
And you can imagine ranges over which more workers help.
They can work together.
So that actually two workers can do more than twice what one worker can do.
And we'll talk about that.
But we're going to focus in the main on ranges over which each additional worker does less and less and less.
And the reason is because, to remember, this is less than two-- Yeah?
Is there going to be a point where actually additional workers won't do anything, because of like constrain to the amounts of other inputs?
Once again, we're not going to get-- just like there's a point at which extra pizza will make you barf-- OK, we're assuming we don't get to that point.
But yes, technically of course, at some point more is not better.
At some point, extra workers just sit around.
And so I agree.
So basically, it's a little tricky.
You could imagine an initial range where more workers would help.
Then there's the main range we'll focus on each additional worker does less and less and less.
And you could imagine that running out, which is a worker does nothing.
But let's just now assume we don't get there.
So basically, what's the intuition?
Now the intuition for in utility theory, for diminishing marginal utility, I thought was pretty easy, which is each pizza means less.
The intuition for why each worker does less, I find a little bit harder.
You could say, well, gee, why should one worker do less than the next worker?
Why shouldn't the next worker do less than the first?
And the key is this part, that we're holding capital constant.
The reason each worker does less is because they only have the same amount of stuff to work with.
And the classic example we use here is the example of digging a hole.
You go to dig a hole and your capital's a shovel.
And let's say for some reason the shovels are out, so you can't get another shovel for a while.
OK, so there's one shovel.
So you have one worker digging a hole, then the next worker comes along and that's where he can help because he can spell the first worker.
And maybe the next worker's just as good because he can work more hours, but probably it's a little bit less good.
But certainly by the time you add a fourth and a fifth and a sixth person, with one shovel, they probably each help because they can rotate and rest each other.
But certainly the sixth person is not going to help dig the hole as much as that second person did, or the third, or fourth, or fifth person, because only one shovel.
So they can share a little bit and share the burden a little bit.
But at some point each additional worker helps less because they have to share the same shovel.
So that's what diminishing marginal product is, because capital's fixed.
With a certain amount of capital to work with, each additional worker just can't help as much as the one before.
Now eventually, you can add more shovels.
And you could have wheelbarrows.
So some people could run the wheelbarrows, some people could run the shovels.
But that's the long run.
In the short run, there's the one shovel, so each additional worker does less good.
And that's the intuition.
Once again, not as clean as with consumption, where it's easy to see each additional pizza is worth less, because you could imagine the second worker might actually help.
They could rest, et cetera, and you could imagine eventually the ninth worker does nothing because there's nothing left to do.
But let's focus on that range where it's intuitive.
We're staying between the second and sixth worker, which is when a worker would help, but they help less and less and less.
OK, and that's the diminishing marginal product.
Questions about that?
That's short-run production.
And we're going to come back to this.
But I just want to introduce these concepts.
Now let's talk about long-run production.
Now in the long run, all inputs are variable.
That's how we defined the long run.
So now a firm doesn't just choose how many workers to hire, or how many hours of labor to buy.
It chooses both l and k, and has to trade them off, just like you chose both pizzas and movies and had to trade them off.
So the long-run production theory is the same, basically the same mechanics, as utility theory.
There's a production function.
You have two inputs.
You trade them off.
Just like you're a consumer.
You have two goods to consume.
You trade them off.
The difference is going to be that ultimately production is going to self-- the difference is, when you decide how to trade them off as a consumer, you're given a budget constraint.
The difference with production is the budget constraint is going to be itself determined by the same system.
So you're not only going to develop your production function, but you're going to develop your budget constraint and, you're going to decide both.
It's a little bit funky but we'll get to it.
But for now, let's just think about the parallels to consumer theory, and think about a production function which is q-- little q-- is the square root of k times l. That same functional form I used with pizza and movies, where utility is the square root of pizza times movies.
Now I'm going to say what you produce of your good is the square root of k times l. Now let's go to figure 8-3.
And what we see here is, if you're trading off k and l and deciding to produce, then you get what's called isoquants.
Isoquants are the parallel to indifference curves.
Once again, this is all the same mechanics.
Just as there were sets of goods across which you were indifferent, two pizzas and one movie versus one pizza and two movies.
Isoquants are sets of inputs along which production is the same.
So along a given isoquant, q is fixed.
Each of those isoquants is a different level of q, but they show how you can vary k and l to get the same amount of q.
So producing q equals square root of 2.
I can use two units of capital and one unit of labor, or one unit of capital and two of labor.
So I can choose lots of combinations of k and l along that isoquant to produce a given amount of output.
And isoquants have all the same features as in indifference curves.
The further out the better because you're producing more.
They can't cross.
All the same set of things we have in the indifference curves are true with isoquants as well.
And they slope downwards because there's a trade off between capital and labor.
Now, what's going to determine the slope of an isoquant?
Can someone tell me what determines the slope of an isoquant?
What determines if isoquants are steep or shallow?
Well, what determines the slope of a indifference curve?
Yes, go ahead
It's based on the ratio of how much labor is worth versus capital?
Right, and what do we call that?
What determines how much they're worth from each other?
What determines the slope of a indifference curve?
The marginal rate of substitution, it was the substitutability between goods.
Likewise, the substitutability between labor and capital will determine the slope of these isoquants.
So to see an example, let's do an extreme example here.
Let's consider goods that are perfectly substitutable.
So like, I don't know, just like a Harvard undergraduate and a beanie baby.
OK, perfectly substitutable.
Figure 8-4a shows the case of perfectly substitutable inputs.
In that case, you would have a linear isoquant, because what that would mean is you don't care if you have three capital and one labor, or three labor and one capital.
You don't care.
You don't care if you have two labor and two capital, or three labor and one capital, as long as you get a total of four.
That's all that matters.
They're perfectly substitutable inputs, which would say that it would be something like q equals k plus l would be the case of perfectly substitutable inputs.
You don't care if it's k or l, you just care about the total.
That'd be perfectly substitutable inputs.
That would be a linear isoquant.
OK, on the other hand, let's think about goods which are not at all substitutable like cereal and cereal boxes.
The cereal wouldn't be any good unless you have a box to put it in.
The box doesn't do anything unless you have cereal to put in it.
That would be like in 8-4b.
8-4b would show you non-substitutable inputs where basically, given the amount of one input, it doesn't matter how much you have of the other.
So for example, if you take these-- these we often call non-substitutable isoquants-- Leontief production functions.
Leontief production function, for Wassily Leontief, some old economist.
And basically, the Leontief production function is that your production, q equals the Min of k and l, is the Leontief production function.
So given how much k you have, given you have 10 cereal boxes, once you have 10 chunks of cereal, it doesn't matter if you have 10, 11, 12, 1 million.
You only have 10 cereal boxes.
So given an amount of k, then it doesn't matter how much l you have.
Once you get to the amount of l you need to fill the cereal boxes, it doesn't matter, and vice versa.
So basically, they're perfectly non-substitutable.
So all that determines your output is which you have the least of.
OK, so substitutabilities determine the slope of the isoquants and of this production function.
Now in general, we'll be in between these cases.
There'll be some substitutability.
Goods won't be perfectly substitutable, but they'll be somewhat substitutable.
So in general, we'll be in between these cases.
And more generally, just as the slope of the indifference curve is the marginal rate of substitution, the slope of the isoquant we will call the marginal rate of technical substitution.
The marginal rate of technical substitution-- the rate at which you can substitute one input for another in a production function is the marginal rate of technical substitution-- which we'll define as delta k, delta l for a given q bar, the rate at which you can trade off k for l to hold q bar fixed.
Now, as with marginal rate of substitution, the marginal rate of technical substitution will change along the isoquant.
So if you go to figure 8-5, here we've drawn a typical isoquant for the production function q equals square root of k times l. So the production function is q equals the square root of k times l. And here's a isoquant.
This is the isoquant of all combinations which produce two units.
This is the q equals 2 isoquant.
Now, unlike utility-- remember utility?-- we said where u equals 2 was meaningless.
Utility was an ordinal concept, not a cardinal concept.
Here, quantity is meaningful.
If you produce four, you would have only produced twice as much as if you produced two.
We can care about both the ordinality and the cardinality of these outcomes.
So we can say, what are the combinations of inputs which lead you to produce two units?
What's all these combinations of inputs?
So whereas you can have one unit of labor and four units of capital, two of each, or four units of labor and one unit of capital, all will produce two units of the output.
And what you can see is that the marginal rate of technical substitution varies.
So for instance, when we start with four units of capital and one unit of labor, and we think about adding a second unit of labor, then the marginal rate of technical substitution is minus 2.
That is, one unit of labor is worth two units of capital.
In other words, we can produce the same amount of widgets of q, but if we replace two units of capital with one unit of labor, so at that point we're very capital-intensive, and that unit of labor is very valuable.
On the other hand, now if you imagine we're down at 4-1, at the third point-- we are probably using a, b, c-- now we're at the third point, where you have we have four units of labor and one unit of capital.
Now, if you'd be willing to give up two units of labor just to get one unit of capital, that's the marginal rate of technical substitution is now minus 1/2.
When you're very labor-intensive, you'd be happy to give up a lot of labor to get a little capital.
Once again, the principle of diminishing marginal product, just like the principle of diminishing marginal utility, implies that the marginal rate of technical substitution is going to be falling as you go down the isoquant.
Just like the marginal rate of substitution fell as you went down the indifference curve, the marginal rate of technical substitution is going to fall as you go down the isoquant.
And why is this?
Once again it's because of this diminishing marginal productivity.
That is, as you add more and more labor, given capital, each unit of labor can do less and less.
And likewise, as you add more and more capital, given an amount of workers to use it, each unit of capital can do less and less.
So it's a very different approach, but gets you the same answer, which is with consumption, each piece does less and less for you, but we can see that as consumers.
Here's the notions of labor, each worker does less and less for you, holding capital fixed.
And each machine by the same logic, if you have one guy in the hole, it doesn't matter how many shovels you throw there.
He still is only one guy.
So each machines is doing less and less by the same logic.
And those diminishing marginal products lead to this decreasing marginal rate of technical substitution as you move along the isoquant.
That's about the most technical statement I'll make all semester.
OK, questions about that, about what's going on here?
That's production theory.
That's what you need to know to know basically how firms produce things.
Now, what we're going to do is, now take this basic production theory, and turn it into actually understanding how firms make decisions on how much to produce.
But before we do that, I want to talk about one other concept, which is very important for thinking about production theory.
And it comes back to these assumptions that we make which might be a little bit unrealistic, which is the concept of returns to scale.
The concept of returns to scale.
So here the question is, what happens if we increase all inputs proportionally?
Return to scale.
By scale, I mean, what if we just double everything, twice as much labor and twice as much capital.
That's an increase in scale, increasing all inputs proportionally, twice as much labor, twice as much capital, half as much labor, half as much capital.
So a change in scale is an equal increase or decrease in all inputs?
Equal proportional.
So what happens if we increase all inputs, increase or decrease proportionally?
And this is an interesting case, because it's like, OK, what if we just make the firm half as big?
What happens?
Well, the answer is, it depends on the production process.
Some production processes will exhibit what we call constant returns to scale.
This is a convenient form.
It's convenient because the way constant returns to scale works is, it says that f of 2l, 2k, is equal to 2 times f of l, k. That is, if it's constant, you can just pull the 2 out.
Doubling the inputs leads to doubling the outputs, which equals to 2q, I'll write here.
Doubling the inputs equals doubling the outputs.
So if I have twice as much labor and capital, that's the same as twice producing what I had with the original labor and capital which will get me twice the original production.
That's the constant returns to scale.
So every time I double my firm, I get exactly twice as much stuff out.
We can contrast that with increasing returns to scale, IRS, which says that f of 2l, 2k, is greater than 2 times f of l, k. Or it's greater than 2q.
That is, when I double my firm, I produce more than twice as much stuff.
That'd be increasing returns to scale.
On the other hand, we could also have decreasing returns to scale, which not surprisingly means that f of 2l, 2k, is less than 2 times f of l, k, or less than 2q.
That would be decreasing returns to scale, which is when I double my firm I get less than twice as much stuff.
Can anyone give me an example or some reason why returns to scale is going to be increasing or decreasing?
Think about firms out there.
Give me some that, by returns to scale, would be increasing or decreasing, any examples you can think of.
There's no right answer here.
Yeah.
Go ahead, on the end
Oh, if you're mining something, you can only get to stuff at one time, so having more equipment and more people doesn't mean you can get more out of it?
Right, so if there's a limited resource, you could imagine decreasing returns to scale.
Good.
What else?
Yeah
If the company gets harder to manage, that could be decreasing returns to scale?
Right, exactly.
If you're an entrepreneur, you've got a great idea, but it turns out that idea only works if you really are hands-on about it.
You're not Mark Zuckerberg, who can just farm the thing out, it's really your idea.
And you've got to be there.
Then after it expands and you lose control of the production, it might not be as effective because you're not there making it happen.
Any ideas why return-- Yeah
If you or [UNINTELLIGIBLE]
So that's an example of increasing returns to scale.
Increasing returns to scale would be, gee, maybe if I'm bigger, I can get a better deal on the inputs.
But that's not quite right, because we haven't really got into prices yet.
I'm looking-- that's really a market response-- I'm looking for a more technological story, a technological story of increasing returns.
Go ahead
If you're bigger, you can hire a specialist to do work, like a manager who can [INAUDIBLE]
Exactly.
So it's the opposite, which is, let's say you have an idea which is pretty good, but in fact it's replicable.
And right now, you're trying to manage some guys.
And then all of a sudden, when you get twice as big, you bring other people in who can effectively replicate your idea, and really expand it in a very effective way.
You can have increasing returns to scale.
Now, so an example of what that looks like, if you go to the last page in the handout, figure 8-6a shows constant returns to scale.
So let's start with 8-6a, that's constant returns to scale.
So here-- and these are from the textbook-- for example, when we doubled the inputs from 100 labor, 100 capital, to 200 labor, 200 capital, you doubled output.
Those are constant returns to scale isoquants.
The last page shows decreasing and increasing returns to scale.
So for instance, tobacco is an example of decreasing returns to scale, because there's only so much leaf that you have that you can produce.
So here, when you double the inputs from 100 to 200, your output only goes from 100 to 142.
OK, and if you want to double the 200 units, you've got to go way the heck up there on inputs.
On the other hand, primary metal production is something where you can have increasing returns to scale, because there's such high costs of building a plant that once it's built, so just like I said, gee, in fact that second digger of the hole might actually make it more productive.
Once that plant's built, there's one worker banging around the plant, a second worker adds a huge amount to productivity because they can specialize.
Basically, increasing returns to scale is going to come from specialization.
So if you build this big plant to build primary, to build steel, one worker in that plant is no good, because he's got to pour the steel, then run over and cool it down, et cetera.
Once you have two, they can specialize, and three, they can specialize more.
You might think of something like that would have increasing returns to scale through specialization.
And that's illustrated here, where doubling the inputs actually leads to more than doubling the output.
Now we're going to talk about all these cases.
Typically, economists are suspicious of increasing returns to scale.
Anybody, anybody who ever has an IPO, whoever starts a firm will always tell you that they have increasing returns to scale.
Well, we say, I've got a great idea and the bigger, the more, the better it's going to be.
And they're generally wrong.
Generally, we think of increasing returns to scale as being like a free lunch.
And we don't like free lunches, we economists.
We think, gee, if it's really increasing returns to scale, you would have already expanded there.
OK, if it's really such a good idea to get bigger, why aren't you bigger already?
Now, it doesn't mean you can-- Yeah
Well, they need capital
Well, they need capital, and so that would be their argument.
And that's great.
And typically, they're wrong.
But that would be why you would argue that they need capital to get bigger.
So typically, we like decreasing returns to scale.
That's going to be our sort of default assumption of the world.
In other words, here's a way to think about it, think about we're dealing with mature firms.
We're modeling, for instance, mature firms.
We think that for mature firms, at least, there's not increasing returns to scale, because in a mature firm they would have hit that point already.
Now they're on the decreasing part.
So that's where we're going to focus for the rest of the semester.
OK, let me stop there.
And good luck tomorrow night.
And we'll come back to talk more about production on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so today we are going to finish up our discussion of consumer theory and then move on to producer theory.
That is, we're finishing up what's behind the demand curve and moving on to what's behind the supply curve.
But before we do that, I want to talk about an application of why income and substitution effects are important.
Income and substitution effects, we talked about this imaginary budget constraint.
It seems like sort of a vague concept.
We talked last time about labor supply and how you can get some interestingly different answers.
But I want to talk about an application now, of why understanding income and substitution effects can really help you understand the world a bit better.
And the application I want to talk about today is the case of child labor in developing countries.
Now this is a terrible problem worldwide.
In developing countries in particular, children are pulled out of school to work.
They then don't develop the skills that can allow them to earn high wages later in life and as a result, the cycle of impoverishment continues in these nations, where these children in turn have low wage jobs and make their kids go to work.
And it's a major problem in the developing world.
And we all know that one way to help countries out of poverty is to help them get their kids to get education.
Education is the key to getting out of poverty.
And you can't get education if you're working.
I think that's a generally recognized fact.
A more interesting, controversial fact is well, what does this imply for the benefits or cost of free trade?
We'll talk about free trade later in the semester and we'll come back to this, but basically the idea is look, if you have free trade, then poor countries can sell more goods to rich countries and that will allow them to raise their standard of living.
But many of you will point out that that's contradictory with our concern over child labor.
Which is if a poor country is going to sell more goods to a rich country, then maybe kids in that country are going to have to work harder to make those goods.
And one concern people have when they talk about free trade agreements is essentially we're dooming the children of these impoverished countries to work harder to make the goods that rich countries want.
And that if these impoverished countries weren't engaging in making all these textiles and all the things these rich countries want, maybe their kids would be getting an education instead.
So that's a concern that's been raised about free trade, but in fact, what we can understand from understanding income and substitution effects is that, in fact, that claim may not be true.
In fact, the effect of free trade on child labor is ambiguous.
And that's what I want to talk through now, an application of why understanding income and substitution effects matter.
And there's a very interesting study by two professors at Dartmouth who studied the impact of trade liberalization in Vietnam.
Vietnam used to have a system where they would not allow domestic producers to sell rice abroad.
Domestic rice producers had to sell that rice only in Vietnam and then they introduced free trade provisions in Vietnam which allowed them to sell that rice abroad as well.
And one concern that was raised is that would mean kids in Vietnam would have to work harder producing rice so it could be sold around the world.
Let's see whether that was true.
Well, to think about that, let's first think about it theoretically.
So let's go to figure 8-1.
Figure 8-1 shows the demand for rice, D sub V the demand for rice in Vietnam, D sub W is the demand for Vietnamese rice worldwide.
D sub V is the Vietnamese demand for Vietnamese rice, D sub W is the world demand for Vietnamese rice, and S sub V is the supply of Vietnamese rice.
And we assume demand curves are downward sloping, supply curves are upward sloping, as usual.
So if there was a worldwide market where Vietnamese rice farmers could sell their rice to the entire world, then you'd have an equilibrium price P sub W and an equilibrium quantity of Q sub W. What happened was until 1989, that was not the case.
Before 1989, the government imposed a quota which said that rice producers in Vietnam could not sell effectively, could not sell outside Vietnam.
It wasn't quite that easy, but you could think of it this way, they could not sell outside Vietnam.
Instead, they could only sell the amount that Vietnamese citizens demanded of rice, which meant that they sold Qv units of rice at a price, Pv.
So there was a government imposition which said effectively-- not quite, but effectively-- you can only sell rice in Vietnam, so we're going to sell Qv at a price, Pv.
So then what happened was in the early 1990s, the government weakened this quota so that by 1997, there was no longer a quota.
Rice farmers were allowed to sell wherever they want.
So effectively, the country moved from Qv, Pv to Qw, Pw.
Essentially, it expanded the market for Vietnamese rice so that they were able to sell a larger quantity at a higher price on the worldwide market.
And the difference between Qv and Qw were exports.
So basically, the government allowed them to export an extra amount of rice, Qw minus Qv, and allowed them to obtain a higher price for their rice.
So we'll talk later about whether this is a good idea or a bad idea in general.
We'll talk about free trade and get on to free trade later in the semester.
Hint, we like free trade as economists.
I think most of you probably knew that, and we'll talk about that later in the semester.
But now I want to focus specifically on one question, which is what did this do to child labor?
Well, if we go the next figure, if you look for a second-- forget the shifting supply curves.
The market for child labor is initially at equilibrium at a supply of child labor of S1-- that is, the higher the wage, the more the kids will work-- and demand for child labor of D1-- that is, the higher the wage, the less the demand there is for child labor.
And we're in some initial equilibrium where L1 kids work at a wage, W1.
And let me highlight an important thing about the supply of labor of kids.
If the kids aren't working, they're in school.
The notion is not if the kids aren't working, they're sitting at home.
If the kids aren't working, they're in school.
So basically, the more kids are working, the fewer kids are in school.
So you have this supply, S1, and this demand, D1.
So what free trade does is it shifts out the demand for child labor.
Because now you need to produce more rice, you need kids to produce it, so it shifts out to D2.
It's not marked in this diagram, but you would end up, if you could see where D2 intersects with S1, you'd end up with a lot more child labor at a higher wage.
And that's the argument for why free trade is bad for child labor.
Because it increases demand for child labor and therefore increases the amount of child labor that's used.
But what that argument misses is the income effect, which is that the rice farmers in Vietnam are now richer.
And one thing they'll do with their wealth is buy their kids more education.
That is, think about the utility function of these rice farmers.
They don't want their kids to work.
They understand that their kids are better off with education.
So what would happen if they won the lottery?
What would happen if they won the lottery is they'd use some of that money to say hey kids, you don't have to work anymore.
You get to now go to school instead and build yourselves a brighter future.
Well, the higher price for rice that exists through world trade is like them winning the lottery.
Suddenly, rice farmers are richer because each unit they produce is sold at a higher price.
What are they going to do with that higher wealth?
They're going to allow their kids to work less.
They're going to say, we're effectively richer because we're now selling our rice at a higher price.
As a caring parent who's richer, I'm going to send my kids to school instead of having them work.
How do we manifest that in this diagram?
As an inward shift in the supply of child labor.
That is, children get pulled off the market because their parents now say, I'm going to send them to school instead.
So the supply curve for child labor shifts inwards.
Now, how much it shifts inwards determines what ultimately happens to child labor.
If it shifts inwards a little bit from S1 to S2, then child labor on net still increases from L1 to L2.
That is, the demand shift exceeds the supply shift and on net child labor increases.
That is, on net, free trade does increase child labor.
But if supply shifts in a lot, if parents are a lot richer and send their kids to school a lot, then actually the supply of child labor could fall from L1 to L3.
You could have less child labor, even though the demand for it's gone up.
And that's because of income effects.
The importance of income effects is that the families are now richer, so they're pulling their kids out of the market.
And they're pulling their kids out of the market in such numbers that it exceeds the excess demand for child labor.
So once again, we see the power of income and substitution effects.
We have the standard market operation, which is g. We want more rice, we need to hire more kids to produce it.
But we're forgetting the income effect, which is the price of rice has gone up.
Farmers are now richer, they're now going to send their kids to school instead of making them work in the rice paddies.
OK?
So that is the importance of understanding the subtle argument that comes forth when you think more completely about income and substitution effects and when you don't stop and just say, gee, more rice exported means more child labor.
So the question is, which of these is right?
OK?
Which of these is right?
How do we know whether free trade increases or decreases the demand for child labor?
Let me first stop and ask a question about the theory.
People understand why an increase in free trade can either lead to more kids working or less kids working.
Yeah?
I'm not sure how you would sustain that increase in demand for rice.
Because if their kids won't work anymore, how will they keep up that increase in demand of rice, keep them wealthy?
Well, see, here's the key point, which is that the price has gone up.
So the price has gone up, so they don't have to increase the supply that much.
So the kids could work less, they could work more, or they could just not supply that much more rice.
But the fact that they sell it in a worldwide market increases the demand so that they're getting a higher price and that makes them wealthier.
So now we have to ask, what's the truth?
What happened?
Well, the answer is that how do we tell this?
Well, the way we tell this is we can look, we can use the fact that the increase in the price of rice varied across the country.
In some parts of Vietnam, those that were very close to the border and it was easy to export, there was a big increase in the demand for rice, a big price increase.
In other parts of Vietnam, those more internal where it's a huge transportation cost to get to the coast and therefore export it, it's like nothing happened.
So you can actually ask, what happened to child labor in the areas close to the border where the price of rice went up a lot and therefore there's a big effect of this free trade, versus the areas more internal to the country where price of rice didn't change much?
And by comparing those two, you can ask, what happened to child labor in those two areas?
What you find is that child labor went down in the areas near the coast.
They got so much richer from these higher prices that on net, they used fewer kids to work.
That is, the supply shift was larger than the demand shift.
So actually, free trade lowered overall use of child labor.
Free trade lowered the overall use of child labor so that freeing up trade was good for kids, not bad for kids.
We'll talk later about other arguments about why free trade is good or not, but this is an important point where you can get beyond a simple intuition of, gee, more kids are going to work, to say wait a second.
That's offset by the fact that the higher worldwide prices led parents to allow their kids to go to school rather than go to work.
And that's an example of the power of income and substitution effects.
And the power of this theory for understanding how you might not get what seems, initially, an intuitive answer.
OK, questions about that example, how it relates to consumer theory, income or substitution effects, any of that?
OK.
So that's where we are going to stop on consumer theory, and that's where the material for the exam will stop.
You can't leave, but that's where the material for the exam stops, OK?
So the exam will cover everything up to through this point.
The exam will be on consumer theory.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
At the beginning of the lecture, we're going to actually talk about productivity, one of the most important topics in economics.
And really one of the most famous applications-- or, more generally, misapplications-- of the principles of diminishing marginal product in the history of economics.
Many of you may have heard of a guy named Thomas Malthus.
He was a famous philosopher who, in 1798, posited the theory that we're all in big trouble.
And he did so following the basic tenets that I've taught you so far.
Malthus pointed out look, we've got-- he said, think about production of food.
He said, with the production of food you've got, as with any other production process-- he didn't put it in these terms, but basically he was appealing to what we learned last time.
He said, like any other production process you've got two inputs, labor and capital.
But with food, the capital is land.
And unlike other kinds of capital, it's fixed even in the long-run.
That is, we talked about the long-run being defined as the period of time over which all inputs are variable.
Well, land is never variable.
There's a certain amount of land on earth, that's not variable.
And at the end of the day, production of food is essentially just short-run, there is no long-run.
At the end of the day, production of food, capital's fixed, it's only labor.
Moreover, in that situation labor has diminishing marginal product with a given amount of land.
It doesn't matter how many workers you have, there's only so much you can grow on it.
Obviously, as you increase workers you can grow more originally.
But eventually, you'll run out of useful use for those workers, yet the demand for food will not stop growing.
So basically, the demand for food is going to continue to grow unabated over time as population grows.
Demand for food [?
it fuels ?]
proportional to population, so it's growing over time.
Yet the production of food eventually has to slow down, because there's a diminishing marginal product of labor without an increasing capital.
So basically what you've got is a forever growing demand, but a gradually slowing production, because the marginal product of labor's diminishing with this fixed capital or land.
The result is mass starvation.
So Malthus predicted that by about where we are now, if not before, the world would be suffering from mass starvation.
Through the basic principles-- not because he's a crazy nutcase-- but the basic principles we've studied so far.
Which you've got ever-increasing demand, but diminishing marginal product of producing food.
And in the end you get mass starvation.
Well, as we all know Malthus was wrong.
World population has risen about 800% since he wrote his article at the end of the 18th century, and yet we're fatter than ever.
Our problem is we eat too much, not enough.
Now that's not true around the world, there's starvation elsewhere.
But there's clearly no more starvation worldwide than there was at his time despite the fact that the world population has grown eight-fold.
So what did Malthus get wrong?
What Malthus got wrong is what I haven't taught you yet.
Which is that aggregate production is not just about k and l, but also about productivity.
It's also about productivity.
That the production function really looks like-- the form of the production function, which we wrote last time as q equals f of k and l. Really more generally, can be written as q equals A, times f of k and l, where A is aggregate productivity.
Really, let's say that this is big Q.
If we think about the big Q for society, now let's think of aggregate quantity for society or else we wouldn't talk about a specific firm.
But if we think about aggregate product, aggregate quantity produced in society, it's a function of the aggregate capital and labor of the society, but also a function of productivity.
It's also a function of the fact that we use our inputs more effectively over time.
So for example, one thing Malthus missed is that the acreage of land-- it's an empirical fact, the number of acres of land on Earth are fixed.
Earth is not growing-- but the arability of that land is not fixed.
We get better and better at figuring out how to grow more and more stuff on the same amount of land.
That's the factor A, that's a productivity improvement.
Likewise, agricultural technology has improved.
We have disease-resistant seeds, we have better land management.
The bottom line is we are making more and more of a given plot of land compared to what Malthus saw in his time.
So while k if it's defined as land may be fixed, and l therefore there's diminishing marginal product of a given production function, the production function itself is improving over time because of productivity improvements.
Productivity, the arability of land, disease-resistant seeds, and other things are making that given quantity of land more productive over time.
So effectively, in the long-run if A goes up faster than the marginal product of labor diminishes, then overall quantity can increase even though k, the underlying level of land, is fixed.
That's what Malthus missed, is that there's two factors going on over time.
The marginal product of labor's falling, it's true, for a given plot of land.
But we're making each plot of land so much more productive, it's overcoming that.
And as a result, food production is actually rising per capita.
So since 1950, world food consumption per capita has gone up 40%.
Despite the fact that the Earth's not gotten any bigger, and despite the fact the population's grown a lot over that time.
And basically this huge increase of agricultural productivity has overcome the diminishing marginal product of labor.
There's actually a great little box in the Perloff Text about a single individual and his contributions to that.
A scientist who led what's called the Green Revolution.
He experimented in Mexico with different methods of improving agricultural productivity, and then essentially brought those to Southeast Asia-- India, Pakistan and other places.
And they estimate, saved about a billion lives through the increase in agriculture productivity he made possible for this Green Revolution in Southeast Asia.
Really, just changed the entire trajectory of that part of the world through the agricultural productivity improvements that he put in place.
So it's very interesting putting a personal face on this impersonal letter A, how one scientist can really make a difference in that case.
This also leads to the larger question which this course doesn't spent a lot of time on, but which is more of a macro question, which is what determines the overall standard of living in our country?
The standard of living in our country, that is basically for a given level of labor we supply, what determines the level of our utility, of our social welfare, given how much labor we can supply?
Well, ultimately, what's going to determine-- or another way to think of it is what determines the amount of stuff we can have for a given amount of labor effort we put in?
Well, that's society's productivity.
Society's productivity is how much more we can have for each given level of labor input.
So what determines how much stuff we can have?
Well, it's k and A.
Given a fixed amount of labor input, given how much we work, what determines how much stuff we can have, with how much capital we have, and how productively we make use of it?
Now, productivity in the US has followed a very interesting trend.
So productivity, which is how much we produce for a given amount of inputs, has followed an interesting trend.
From World War II until about 1973, productivity grew rapidly in the US.
Productivity grew at about 2.3% per year-- 2.4% per year-- from the end of World War II through 1973.
That is, working no harder and having no more machines, we can consume 2.4% more stuff every single year.
That's pretty impressive.
That means we can just sit around, work no harder than we were, and have no more machines, and produce 2.4% more per year.
Now, of course, over time we worked harder and had more machines, so overall output in US economy grew much faster than 2.4% a year.
It grew more like 7-10% a year over that period.
Yet, the point is that a lot of that we can get for free, essentially, without any harder work or any more capital.
However, starting in 1973 until the early 1990s, productivity growth fell dramatically to 1% per year.
That is literally we lost 1 1/2% per year of stuff we were getting before.
We were getting 2 1/2% a year up to '73, all of a sudden it's down to 1%.
That's 1 1/2% a year less stuff we can get unless we work harder to make up for it.
Why did this happen?
Well, we don't exactly know, but there's two good candidates.
We know the two candidates, we just don't know the right proportions.
One is that we have less capital in our society because savings fell.
The amount of savings US households do fell dramatically.
And the US has a very low savings rate.
The US savings rate over this period averaged about 3%.
That is, every dollar we earned we saved about 3% as a society.
Compared to countries like Japan, where it's more like 20%.
Every dollar they earn they save about 20%.
Now why does that matter?
Well, we'll talk about this later in the course, but essentially the amount we save determines the amount of capital we have in society.
Because essentially, where do firms get the money to build machines?
They get it by borrowing from households who save.
And the less we save, the less money there is that firms could invest in building machines.
And we'll talk about that at length later in the semester.
But the bottom line is, the more we save as a country, the more money we have available, the more firms can take that money and build machines that improve our standard of living.
And that saving fell a lot, and that's one reason.
And the other reason is that productivity fell for reasons we don't quite understand.
We know that productivity slowed down, but we don't quite understand why that is.
But then in the 1990s, productivity shot up again.
So productivity went back up towards our historic levels, from 1% back up to over 2% a year.
Why is that?
Well, it's unclear, but we think it's basically the IT revolution.
Essentially, we think that the slow diffusion of computers, which people were predicting should increase productivity as way back as the 1980s, suddenly in the 1990s it really happened.
And this IT revolution led to a big productivity increase.
It's not clear if that's dying down now again, or if it's going to continue.
It'll be interesting to see what happens over the next 15 years.
So we have this period of high productivity growth, slowed down from '73 to the early '90s and then picked up again.
We're not quite clear if that year's coming to an end or not, but that's sort of where we are now in that time path.
What's very interesting-- so that's what happens to productivity, that's all I'll talk about it for this course, it's more of a macro topic.
But I will mention an interesting micro spin on that.
Which is, if society's more productive, that's like found money for society.
That's like saying with all our resources we suddenly get extra money.
Society then has to decide what to do with that.
The US and Europe have followed very different paths in what to do with that money.
In the US, we've taken that money and bought a lot more stuff.
We have the highest standard of living in the world.
We buy the most stuff per capita of anyone the world.
In Europe, they took a lot of that money and took more leisure with it.
They decided we're not going to quite have as much stuff, but we're going to have six weeks a year of vacation instead of two weeks a year of vacation.
So if we go back to our discussion of what determines labor supply is the choice between leisure and consumption, and you think of the wage as the opportunity cost of leisure, well, what they've decided in Europe is to choose more along the leisure axis, and less along the consumption axis.
In the US, we've chosen less among the leisure axis-- we work way harder than Europe-- but we have more stuff.
And the question is, how do we feel about that choice?
Has that been ultimately a welfare maximizing choice?
Now an economist will say of course it's been, because it's a choice we made.
Of course, it's been welfare maximizing.
We talk about revealed preference, and people's choices reveal what they prefer.
So our revealed preference, we just prefer stuff more and leisure less than Europe.
But in fact, it's not clear that that is each individual's optimal choice.
If a given individual says, look, I'd rather have less stuff and more time off, it may be hard to find the job that lets them do that.
So while that may be the choice we've made as a society with our social institutions, that may not serve the interests of every individual in society.
And that's the kind of trade-off we need to think about.
So anyway, that's sort of what I wanted to say on productivity.
Yeah, question?
Does higher productivity translate into more income, or more income for individuals who will buy stuff [INAUDIBLE] taking more leisure?
Because basically the point is think of our economy as a pie.
That basically the idea is let's think of you have a start up, and your start up is such that you can make this product, and you could make $1 million a year with 10 workers.
You could make $1 million worth of stuff with 10 workers.
So each of your workers takes home $100,000.
Now imagine that you discover new technology which lets you, with the same amount of workers, make $2 million a year.
Well, some of that you'll keep, but some of it you'll pay your workers more.
So suddenly they have more money, because you've suddenly managed to make twice as valuable stuff with the same amount of resources.
So that's the situation which improves our standard of living.
Other questions about that?
Comments?
OK, so the bottom line, coming back to sort of micro-theory we're talking about, is we have to think about production functions as having a productivity adjustment.
Macro raises these big issues about sort of ultimately what determines our standard of living in this country, and how do we want to spend that money?
So, with that as background, we're now going to stop talking about production and move on to cost.
Cost is-- quite frankly this is perhaps my least favorite thing in the whole course.
It's a little bit boring, but you need to understand how cost structure in a firm works to understand how firms make the decisions that ultimately get to be a lot more interesting again, so just sort of bear with me.
Now, so we talked about costs, let's start with a couple of definitions.
Basically, let's back up, where are we coming from?
I talked about what the firm's decision is, the firm has to maximize profits, which is revenues minus cost.
So we have to ask what are costs if we're going to make this profit maximizing decision.
Well, costs are going to have a few components.
The first component, costs are going to have really two major components-- fixed costs, and variable costs.
Fixed costs and variable costs.
Fixed costs are the costs of inputs that cannot be varied in the short-run.
Remember, I said that the short-run is defined as a period over time which only some inputs can vary.
Well, fixed costs are the costs of those inputs that can't vary in the short-run.
Variable costs-- so that's like capital in the short-run-- variable costs are the cost of goods that can vary in the short run, that's like labor.
So total costs is the sum of these two, so total costs equals fixed cost plus variable cost.
Finally, another definition that's important is marginal cost, which is the change in cost with a change in output.
So the marginal cost is just like-- remember, we want to think in terms of marginal decision making in this course.
So the marginal cost is the change in cost with the change in-- actually, that should be a little q.
The change in a firm's cost with the change in the firm's output is marginal cost.
And then finally, average cost is just what it sounds like.
Average cost is just c over q, it's just the average.
So the difference between marginal and average cost, is basically average costs is the average over the whole set of goods produced.
Marginal cost is the cost of that next unit of production.
So those are our key definitions.
Now with those in mind, let's ask how do we get costs?
And the answer is we get them from the production function.
Once we do a production function, we can derive costs.
So if we have some production function, q equals f of l and k, then we can say the cost of producing q is equal to f of wl plus rk.
Where w is the wage rate, or the rate you pay per unit of labor, and r is the rental rate, or the rate you pay per unit of capital.
Now, let me just pause here for a second to talk about pricing capital.
It's easy to think the cost of an hour of labor, it's the wage you pay for an hour.
It's harder to think about the cost of a unit of capital.
Because we buy the machines, right?
So how do we think about the cost?
I'm going to cover this later in the course, for now imagine all machines are rented.
Imagine you rent every machine you use.
And think of r as the rental price of that unit of capital.
So with buildings it make sense, firms often rent the buildings they're in.
Think of r as the rental price of that unit of building, or that unit of machine.
We'll come back later to see why that's a sensible way to think about it.
The key point is, the reason we have to do this is the wage is a flow measure, every hour I pay you a new wage.
If I use the cost of buying the machine, that be a stock measure, so you couldn't really compare it to wages.
So we want to use a flow measure.
The flow measures is what we have to pay every period to rent the machine.
Yeah?
[INAUDIBLE] just take the cost of the machine and estimate the amount of time we want, and then divide it?
Sure.
No, and I'll cover that later.
You could think of the rental-- if I bought the machine today and sold it tomorrow, that'd be like I rented it.
And this would be the cost difference between what I paid for it and what I'd sell it for.
But it's just easier to think of it as the rental, because the flow measure-- like the wage-- is a flow measure.
Now, in the short-run, capital is fixed.
So in the short-run, our fixed costs are rk bar.
That's our fixed cost, the rental rate times the fixed amount of capital in the short-run.
And our variable costs are w times l, which is a function of q.
That is, the more you produce the more labor you use in the short-run.
So total costs in the short-run, short-run total costs, are rk bar plus wL of q. k is not a function of q because k's fixed in the short-run, but the amount of labor used is a function of how much you produce.
This implies that the marginal cost, the key concept we want to work with, marginal cost, which is the derivative of total costs with respect to quantity.
So dc dq is going to be equal to w-- or, let's do it in deltas, because we're not doing calculus here.
Delta c delta q is going to be w times delta l over delta q.
That's going to be the marginal cost.
The marginal cost-- so I'm just differentiating the total cost function-- is going to be the wage times delta l delta q.
So the marginal cost of producing the next unit is going to be how much labor I have to produce to produce the next unit, times the wage I pay per unit of labor.
Now, does anyone remember what we call this?
I know this wasn't on the exam last night, so you may not-- cast your mind back to the lecture on Monday.
Do you remember what we call delta l over delta q?
Anyone?
Bueller?
No?
It's the marginal product of labor.
Remember from Monday?
So this is the wage times the marginal product of labor.
So what we say is that the marginal cost is equal to the wage times the marginal-- I'm sorry the wage over.
I'm sorry, it's one over.
That delta q does-- l was the marginal product.
The wage over the marginal product of labor.
So marginal cost is the wage over the marginal product of labor.
Marginal product of labor was delta q delta l, so wage over the marginal product of labor is the marginal cost.
So think about this intuitively.
What we're saying is the cost of the next unit of production is declining with the marginal product of labor, it sort of makes sense.
The more productive is a worker, the less expensive is producing the next unit.
The less productive is the next worker, the more expensive is producing the next unit.
So it's an inverse relationship between the marginal cost and the marginal product where the wage is the constant that scales that relationship.
So basically, when workers are very, very high marginal product, then it's going to be cheap to produce the next unit.
When workers have a low marginal product, it's going to be expensive to produce the next unit, and that's going to depend on what you actually have to pay the worker.
Questions about that?
So basically, the first key thing we want to derive here is that the marginal cost is directly related to the marginal product of labor, and the marginal product of labor we saw last time comes out of production function.
So if you're given a wage, and given a production function, you should be able to derive the short-run marginal cost.
You might someday be asked to do that.
Now what about the long-run?
The short-run's no fun, what about the long-run?
In the long-run, firms can choose their mix of labor and capital.
Remember, in the short-run the capital is fixed, so fixed costs rk bar.
The only thing they could change was the amount of labor, so we could derive their marginal costs.
What about in the long-run?
Well, the long-run's a little more interesting because in the long-run firms get to choose their input mix to maximize their production efficiency.
So input mix is chosen to maximize production efficiency which equates to minimizing costs.
Maximizing production efficiency equates to minimizing costs.
So we talked last time about isoquants, and the notion that isoquants were combinations of labor and capital that delivered the same output.
Just like indifference curves are combinations of pizza and movies that deliver the same utility, isoquants are a combination of labor and capital that deliver the same output.
The key point is that, technologically, any choice of labor and capital produces the same q, so there's nothing that tells you technologically which of those to use.
We just know, technologically, there's a set of choices which deliver the same q.
Well, how do we tell which to use?
Well, we want to choose the one which is minimizing costs.
So to do that, we're going to have to bring in the cost of those inputs.
Just like we said there's a set of pizza and movies, all of which leave you indifferent.
How do you decide which pizza and movies to choose?
Well, you bring in the relative price of pizza and movies.
Here, we're going to bring in the relative price of capital and labor to determine how we choose between capital and labor.
So to do that, we're going to draw isocost lines which are going to be just like our old budget constraints.
Isocost lines which represent the cost of different combinations of inputs, just like our old budget constraint represented the cost of different consumption goods.
So if you look at figure 9-1, here we're going to have isocost curves which are going to represent-- and we're going to assume here that the wage is $5 an hour, and the rental rate is $10 per unit of capital.
So, in other words, the $50 isocost line in figure 9-1 shows all combinations of labor and capital that cost $50.
So you could spend $50 in production if you had 10 units of labor, and no units of capital.
Or five units of capital, and no units of labor, or any combination in between.
These are all the combinations of labor and capital that cost $50.
Likewise, the $100 isocost is all combinations of labor and capital that cost $100.
So each of these isocosts give you the combination of inputs that cost a certain amount.
Just like a budget constraint gave you the combination of pizza and movies on which you spent your income.
Now, you may have said well, wait a second, the difference with consumers is we knew their income so we knew what their budget constraint is.
Here we don't know whether to choose the $50 cost, the $100 cost, $150.
We don't know what the total amount is.
That's what makes firms hard, that's why we have an extra step.
So hold that thought, we'll come back to that next lecture.
For now, let's just say there's a set of trade-offs that the firm can choose from, and a set of isoquants that they have.
And what's the slope of this isocost line?
It's the negative of the wage rental ratio.
The slope of the isocost is minus w over r. The slope is minus w over r. It's basically the trade-off between labor and capital's going to be determined by the relative prices of those inputs, so slope is going to be minus w over r. So basically, how many units of capital do you have to give up to get the next unit of labor?
Well, what this isocost tells you is you have to give up 1/2 a unit of capital to get a unit of labor.
So the slope is minus 1/2.
Likewise, you could say you have to give up two units of labor to get one unit of capital.
So that's why the slope is minus 1/2, that's what it's telling us.
Once again, budget constraints are about opportunity costs.
How much labor do you have to give up to get another unit of capital?
Or how much capital do you have to give up to get another unit of labor?
Now, armed with isoquants, which are like indifference curves, and these isocosts which are like budget constraints, we can then figure out what is the economically efficient combination of inputs for the firm to use.
The economically efficient combination of inputs for a given level of output.
So the economically efficient input combination for a given level of output is going to be determined by the tangency of the isoquant with the isocost, as you see in figure 9-2.
Here we're going to use our same isoquant we had before, which is we're going to assume that q equals square root of k times l. So same production function we had before, which gave a series of isoquants last lecture.
So basically, what we see is that the efficient-- if you want to produce a given amount of q, then basically what you're going to do is you're going to look for the tangency of that isoquant with the isocost.
And you're going to say that the efficient way to produce that is going to be to use 2 1/2 units of capital and 5 units of labor.
It's going to say look, given the relative prices that are given to us by this budget constraint, the production technology is given to us by this production function from which we derived isoquants last time.
So the optimal combination of inputs to get this level of output is going to be 2 1/2 units of capital and 5 units of labor.
And that will produce basically square root of 12 1/2.
So basically the quantity will be equal to the square root of 5 times 2 1/2, or the square root of 12 1/2 units of production.
So basically, that is going to give us the efficient way to do that.
Now, once again as always, we want to think about things intuitively, graphically, and mathematically.
Let's think about for a second the mathematics.
We know that the slope of the isoquant-- we talked last time-- the slope of the isoquant at any given point.
The isoquant slope was the marginal rate of technical substitution.
We defined that last time.
The slope of the isoquant was the marginal rate of technical substitution which is the marginal product of labor over the marginal product of capital.
And what we're saying is we want to set that marginal rate of technical substitution equal to the input costs ratio w over r. That's what we're saying, the efficient thing to do is to set the marginal rate of technical substitution equal to the price ratio.
That's what happens when the slopes are equal.
Now, once again, I find it easier to rewrite this equation-- once you've developed the intuition, I find it easier to think of it this way.
Rewrite this as the marginal product of labor over the wage, equals the marginal product of capital over the rental rate.
What this is telling us is the efficient place is where essentially for every dollar you spent on workers, you're getting the same return as a dollar spent on machines.
The marginal product of labor over the wage is sort of the bang for buck of workers.
What are you getting for your next dollar of wage?
The marginal product of capital over r is the bang for the buck of machines.
What are you getting for your next dollar of rent?
And the efficient point is where these are equal.
If they're not equal, then you have too much of one and not enough of the other.
So basically, what we can do is we can solve in this example-- in this example, we could say the marginal product of labor is 1/2 k over the square root of k times l. The marginal product of capital from this production function is-- once again I'm using this production function q equals square root of k times l. Marginal project of capital is 1/2 l over square root of k times l. So the ratio of the marginal products is simply k over l. The marginal rate of technical substitution, given this production function, is k over l. That's the marginal rate of technical substitution.
So this says that given this production function and these prices, at the optimum you should set k over l equal to w over r, which equals 1/2.
So what this says is given this production function, these price ratios, the optimal thing to do is to use half as much capital as labor.
Half as much capital as labor is the optimal thing to do, and that's what we see in figure 9-2, is the optimal thing to do is use half as much capital as labor.
Now, in other words, let's say, to now develop the intuition.
Imagine you told me no, I should use as much capital as I should use labor.
As much capital as I should use labor.
Imagine I told you that.
Imagine I said no, in fact, the efficient thing to use is why not have one machine for every worker?
How would you tell me intuitively why that's wrong?
Why would I be wrong to say use one machine for every worker?
Why would that be wrong, given the prices prevailing in the market?
Someone can tell me this.
Yeah?
Well, renting machines is a lot more expensive than paying more workers
Twice as expensive to rent a machine as get a worker
So it would be more cost-effective to have the workers share machines rather than get a whole new machine
The key point is the machine costs twice as much, but the machine doesn't do twice as much.
The machine and the worker do the same thing.
The marginal rate of technical substitution is one.
You're indifferent between one more machine and one more worker, but the machine cost twice as much as the worker.
So you want more workers and fewer machines, right?
Given the machines and workers, this is a perfectly substitutable production function.
The marginal rate of technical substitution is k over l. You're perfectly indifferent between these two, given that-- not perfectly substitutable, but at this point you're indifferent between the two.
So given that you're indifferent and the machines cost twice as much, why not buy half as many machines?
Yeah?
But then, if the machines cost twice as much, why buy any machines?
Oh, that's very good point.
Because it's not a perfectly substitutable function.
My bad.
If it was, if the production function-- great question.
Let's say the production function was of the form q equals k plus l. That's perfectly substitutable production.
Then you're right, in that situation you should only buy workers because they do exactly the same thing.
But that's not the case here.
This exhibits diminishing marginal product.
So if you only bought workers, eventually each worker would do so much less that you'd be better off getting a machine.
It's not perfectly substitutable, I misspoke before.
At the margin they have an equal effect.
But as you get more and more laborers, they'll be less and less productive, so eventually you're going to want to buy a machine.
But you're only going to buy half as many machines as workers.
You never want to buy one machine per worker.
But you also don't want no machines per workers, because the workers won't have anything to do then.
Here you'd want no machines per worker, right?
The optimal thing to do, if you have a perfectly substitutable production function, you'd only just buy the cheaper input.
But that's not the case when you have diminishing marginal products, then you're going to use a combination of inputs.
But the combination used will be determined by the prices in the market.
Other questions about that?
So now we can ask, just as we asked in consumer theory, how does a price change in the price of goods affect your consumption decisions, we can ask how does a change in the price of inputs affect your production decisions?
You could see that in the next page, figure 9-3.
Imagine that wages went up.
So imagine now wages, instead of being $5 an hour, are $7.50 an hour.
They pass a new minimum wage, and wages go up to $7.50 an hour.
What does that do?
Well, that steepens the isocost.
Your trade-off is now you're going to get fewer workers for every machine you give up, or more machines for every worker you give up.
And so at the same isoquant, that's going to shift you to using less labor and more capital.
By the same logic as before, you're going to use less labor and more capital, because you're going to see this shift in relative prices.
This figure shows why the minimum wage leads to unemployment.
We talked about it last time.
We did in a graph, we just said supply and demand and showed you.
But actually this is the underlying mechanics of how minimum wage leads to unemployment.
Because the minimum wage, by change, is relative input prices.
If the only way you could produce things was with labor, there wouldn't be much unemployment for a minimum wage because basically you wouldn't have anything else you could do.
You'd still have to hire the workers.
But, in fact, that's not the only way to produce things.
You can substitute to capital.
As a minimum wage goes up, firms will substitute towards capital, and that's why the minimum wage will lead to unemployment.
So this is sort of the underlying mechanics of how that happens.
All right, now armed with that-- so basically when we did consumer theory we were done here.
We basically said, look, we now know you have a budget constraint, you have indifference curves, you're fine with their tangent, you're done.
The reason firms are one step harder is you don't have a budget constraint.
q is not given to you, q is ultimately decided by you.
You the firm are going to decide on little q.
With our example for consumers, your parents gave you $96, you had no choice.
Well here the firm isn't given little q, it's going to decide little q.
What that means is we're not done yet.
There's one extra step we need to do with firms, which is figure out where little q comes from.
So to do that, we're going to have to then say well, how does a firm think about the set of choices of little q?
And how does it think about how it changes production as little q changes?
So to see that, go to figure 9-4a.
This shows the long-run expansion path for a firm.
This shows how, as it produces different amounts of goods, it will choose different units of inputs.
So for the first level of production, it chooses five machines and 10 workers.
Then if it wants to double production, it chooses 10 machines and 20 workers.
So if it wants to increase production by another 50%, it chooses 15 machines and 30 workers, and so on.
This is a linear expansion path.
This says this firm is a production function, and prices are such that basically they always want these inputs in fixed proportions.
So it would be a fixed proportional expansion path.
No matter how much you choose to produce, you always want to use twice as much labor as capital.
However, that doesn't have to be the case.
So this long-run expansion path is going to be what becomes our underlying cost curve.
This is where underlying cost curves are going to come from, and hopefully where supply is going to come from, is this long-run expansion path.
This long-run expansion path is going to show us how much more we have to spend to produce different amounts of quantity.
Now in this case, what you see here is that you have these fixed proportions.
That as you increase quantity, that the input portion stays the same, but that doesn't have to be.
For instance, figure 9b, you can imagine a world where, as you produce more units, capital becomes less productive.
So you want more and more labor, but not that much more capital.
So this might be the example of like McDonald's.
If McDonald's wants to produce more burgers, ultimately there's only so many fryolators it can use.
Ultimately, it needs more people to package up the burgers and sell them.
So you might think that capital becomes less and less productive.
And as a given McDonald's franchise expands its sales, it might want to increase the ratio of labor to capital.
So this is a case where capital's becoming less productive.
And as you see, as you expand production you're going to more labor and less capital.
In other words, the marginal product of labor is still steep, and the marginal product of capital is flattening.
So you want more and more labor, and not as much capital.
That's one kind of expansion path.
Figure 9c shows a different kind of expansion path.
Here's one where labor becomes less productive.
So this might be, for example, something which is a mass production process, like producing automobiles.
Where basically as you produce more and more automobiles, you need more and more machines to produce them.
The people just run the machines.
So it's much more efficient to have to do it through more machines and less through more workers in automobile production.
So in that case you could have a steeper expansion path, where basically the marginal product of labor is falling relative to the marginal product of capital, so you want to increase the ratio of capital to labor over time.
The bottom line is as firms produce more, they may hold constant or may change the ratio of their inputs, but they'll clearly use more inputs.
They're going to use more inputs, but the mix of the inputs they'll use will change with their production levels.
So the question we have to ask is, well, what's going to determine their production level?
Where does q come from?
I'll have to leave that as a teaser for next time.
Let me just say where q comes from, is q is going to come from market competition.
We're going to get q-- I'm not done, I have one more thing to cover.
But we're going to get q from market competition.
Now there is one other thing I want to cover though related to costs.
Which is an important concept that we have to have in the back of our mind, which when we come back, we think about competition.
Which is fixed versus sunk cost.
Fixed-- my wife always thought I was saying some costs, I'm not.
I'm saying sunk costs.
Fixed versus sunk costs.
Fixed versus sunk costs.
Sunk costs are costs which are fixed even in the long-run.
Fixed costs are costs which are fixed in the short-run, and variable in the long-run, so capital.
Sunk costs are costs which are fixed in the long-run.
That is, they're foregone once you produce.
The minute you produce one unit, those sunk costs are gone forever, and they cannot be changed even in the long-run.
In other words, importantly, they cannot be changed by how much you produce.
So in the long-run, you can change the cost of capital by building bigger or smaller plants, producing more or less.
But some costs cannot be changed.
So what's a classic example?
Well, the classic example for example would be medical education, or any professional education.
Once you've gone to med school and done all your grueling years of staying up all night, you've paid those costs.
They're now paid for, and it doesn't matter if you see three patients the rest of your life or three million patients the rest of your life, you've already paid those costs.
Think of that as the capital of a doctor's office.
Now when you take your office as a doctor, if you want to see more patients in the short-run, they might be crammed into your office, and in the long-run you might build a bigger office.
So in the short-run, how hard you work is variable.
In the long-run, how big your office is is variable-- how many secretaries you hire, et cetera.
But your medical school spending is gone.
That's not variable in the long-run, that's sunk.
And that's a very important distinction is between basically these fixed costs, what we call fixed costs.
Which are costs where, like the costs of the office and the machinery the physician uses, which can be changed over a 10-year period, versus sunk costs which once paid are gone forever.
And the key reason, just to give you a hint about why these will matter, is because when firms set up this-- we may see firms in the market losing money.
You may see firms in the market losing money.
In fact, in any point in time we see lots of firms in the market losing money.
You might say, why don't they go out of business?
The reason they don't go out of business is because they've already pay huge sunk costs.
It's not efficient to go out of business.
They've already invested a certain amount.
It's not going to be efficient to go out of business, because then they'll give up the cost they've invested.
So if you're a doctor, and you've spent all this money on med school, and you're not making money as a doctor in the first couple of years.
If you quit and go do something else, you've just given up all the investment you made in med school.
So if there's any prospect that eventually you'll make money, you might want to hang on and keep being a doctor.
So that's the difference between a fixed cost and a sunk cost.
So I'm going to come back to that, but it's important to remember that distinction when we talk about competition.
So let me stop there, and we'll come back on Wednesday, I guess.
Have a good three-day weekend.
We'll come back on Wednesday and we'll talk about competition.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
All right.
Let's get started.
Today we are going to talk about competition.
Just, once again, I want you guys to try to have in mind a flowchart of where the course is all going.
Where does this fit in?
We talked last time about how firms decide on the cost minimizing way to produce a given level of output through the tangency of the isoquant and the isocost.
And we talked about how if we were consumers, we'd be done then.
Consumers just find the tangency and indifference curve in the budget constraint, and they're done.
But firms aren't done.
Because the level of output, unlike the budget which is given to you as a consumer, the level of output is not given to the firm.
They get to choose it.
So with firms, we have to go one step further.
We actually have to choose the level of output that we produce.
We don't just choose the way to produce it but actually how much to produce.
So, for consumers, it would be sort of like consumers choosing what their budget constraint is.
And the way we do that is we bring in-- since we now have a third variable, which is how much to produce-- we have to bring in a third equation.
And that equation we bring in is essentially the market.
Essentially, we say the market imposes conditions on the firm which helps them figure out how much to produce.
So we take all the stuff we did before.
We then take that and put that in a market.
And the market interacts with the firm.
And from that market setting, the firm derives how much to produce.
So, basically, the level of production for a given firm, little q, will be derived from how firms behave in different market settings.
And the market setting we're going to start with today is the classic starting point for economics which is the market setting of perfect competition.
We're going to start talking about perfectly competitive markets, a benchmark of perfect competition.
With perfect competition, this is basically a case where many firms are selling goods to many consumers.
We're then going to, in the next few lectures, talk about more realistic alternatives like monopoly which is not a game but rather a market structure.
It is basically a case where one firm sells to many consumers.
Or oligopoly, my favorite word in all of economics, not because I like the topic so much, I think it's a cool word.
This is when several firms sell to a large market, which is probably the most realistic setting of all.
And we're going to come to talk about those.
But, today, we're going to start with our benchmark of perfect competition.
Now, what is perfect competition?
I can give you a technical definition.
Technically, perfect competition exists whenever firms are price takers on both the output and input markets.
They're price takers on both the output and input markets.
So perfectly competitive firms are price takers.
That is no action that they take can affect either the price at which they sell their goods or the price that they pay for their inputs.
They're price takers.
They're not price makers.
No action they take affects either the price at which they sell their good.
No action the individual firm takes affects either the price at which they sell their goods or the price they pay for their inputs.
Well, when will this be true?
Go back to lecture one.
Technically this would be true if a firm faced perfectly elastic demand for their goods, and if they had perfectly elastic supply of inputs.
Under those conditions, firms will be perfectly competitive if they face perfectly elastic demand for their goods and perfectly elastic supply of inputs.
OK.
So let's focus on the first of those which is perfectly elastic demand.
Let's take a look at Figure 10-1.
These should be little q's by the way.
This isn't a market.
This is a firm.
These should be little q's.
So, basically, what you have here is that you have a firm facing a perfectly elastic demand.
What that means is the firm's quantity is pegged by their supply curve.
Or, in other words, the point is the firm cannot change the price one iota from that level P. So, in other words, if this is a supply shift, say the price of the firm's inputs go up, the firm doesn't get to charge any more for their goods.
They just sell fewer goods.
So the supply shift from S1 to S2, the firm is going to be-- it should be little q1 to little q2.
They're going to reduce the quantity they sell, but they cannot change the price.
They face perfectly elastic demand.
So when does this make sense as a description of the world?
Well, it makes sense as a description of the world under four conditions.
So there's four conditions under which perfect competition will exist.
The first condition is identical products.
In a perfectly competitive market, when the firms in that market sell identical products, now let's be clear.
They don't have to literally be identical.
They have to be perceived by consumers as identical.
So when I say identical products, they don't have to literally be identical.
But consumers have to consider them identical for purposes of their demand across firms.
So firms have to sell identical products for there to be perfect competition.
Because if products aren't identical, then firms will be able to charge different prices from each other because they have something different to sell.
So firms need to be identical.
Second of all, consumers have to have full information on all prices.
Well, let me write down the next two conditions, because they're related.
And the third is low transaction or shopping costs.
OK, these two are critical.
Because the way perfect competition is going to work, its consumers are going to shop across firms selling identical goods.
And they're going to buy from the cheapest one.
And if there's any failure of either of these, the consumers might not know if you're the cheapest.
And, therefore, you might be able to charge extra.
So perfectly elastic demand, once again we're getting to the microfoundation of something we discussed in the second lecture.
We discussed perfectly elastic demand.
Now we're talking about where that comes from.
What conditions do you need?
Perfectly elastic demand is going to require that consumers know all the prices and can cautiously shop across all the options.
Otherwise, firms might have some opportunity to charge different prices.
And finally-- and we'll come back to why this is important-- there needs to be free entry and exit of firms.
This one I can't really give you intuition for yet.
Just take my word for it.
We'll come back to why that's important.
So, basically, what you want for an example of a perfectly competitive market, you want a market where producers are selling homogeneous goods in an easily informed, easily shocked arena.
So I think the best example of a perfectly competitive market is those guys selling like shlocky touristy things around Port Authority in New York or in any large open air market.
Basically, in that area, these guys all sell the same crap.
You can go from one to one quite easily.
And it's quite easy to find out what the prices are.
That's not perfectly easy, but it's quite easy to find out what the prices are.
That is a condition for a perfectly competitive market.
It's easy to shop, because they're all in the same area.
Prices are pretty easy to observe, and the products are all, basically, identical.
There are only so many little Statues of Liberty you can buy.
So, basically, that's an example of a perfectly competitive market.
Now, in reality, no perfectly competitive market exists.
There's never been a perfectly competitive market.
But this is sort of as close as we can get.
Questions about that?
Now that provides a good moment to pause and talk about Peter Diamond who just won the Nobel Prize in Economics.
This is my MIT economics Peter Diamond, MIT Econ Peter Diamond shirt.
Peter Diamond just won the Nobel Prize in Economics.
Peter Diamond is the greatest economic theorist of his generation, sort of the heir of the Paul Samuelson, Bob Solow generation that founded this economics department and made it great.
Peter Diamond was sort of the next generation that led this economics department forward and kept it great.
And Peter has made contributions throughout economic theory.
He should have won the Nobel 10 times over.
What they finally gave it to him on Monday for was for search theory.
And search theory is essentially about what happens when markets don't work like these vendors around Port Authority in New York, when markets aren't perfectly competitive.
Where, basically, you have markets where there is some mismatch and some search costs that sellers have to pay to find the right buyers, and buyers have to pay to find the right sellers.
So the best example here, and the example of which they really gave him the Nobel Prize, was the labor market.
It was about search costs in the labor market.
And what Peter Diamond and his fellow co-winners talked about was about how in the labor market, your firms have vacancies.
They have jobs they want to fill.
Individuals have labor supply.
They want to provide themselves to these jobs.
And so there's unemployment.
There's people out of jobs looking for jobs.
But there's vacancies.
There's jobs that are empty.
And both exist at the same time.
And how can that be?
In a perfectly competitive market, that couldn't be.
You couldn't have both jobs looking for people and people looking for jobs.
But what Diamond wrote down, and these other theorists developed in their models, is basically how you get these frictions in the labor market.
Where since these jobs have specific characteristics employers are looking for, you can't quite match the vacancies to the unemployed workers.
There's a sorting process where some are easy to match.
You can take the high school dropout and put him in McDonald's.
That's easy.
But the job which requires some computer skills, you have to find the right guy to take that.
And, basically, it's these frictions that lead to what we call the natural rate of unemployment in our economy, which is the notion that no economy would ever get down to 0 unemployment.
That's impossible.
And the reason is because there will always be some frictions and some mismatch.
There will always be some inability of people to find the right people to fill their vacancies.
Now, we don't know what the natural rate is.
When I was in grad school, we learned that the rate was 7%.
In the 1990s, the natural rate seemed to fall to about 4%.
That is we got to about a 4% unemployment rate.
Now who the heck knows what the natural rate is anymore.
And a big debate right now among economists in macro is how much of our 9.6% is an increase in the natural rate, which is something the government can't really fix very well, versus short-term demand reductions, which the government could fix by pumping more resources into the economy.
And all that is informed, theoretically, by the work that Peter Diamond did.
Now, I hope what I just described sounded pretty obvious to you.
And that's good, because that's what great theory does.
Great theory ex-post sounds obvious.
It's just ex-ante, before Peter Diamond did this, people always said, well, we have these perfectly competitive markets.
This is how they should function.
And he's the guy who really taught us how real markets should function like this.
And that's why he gets to go to Stockholm.
So that's very exciting for our economics department, very exciting for the profession.
And it's just a great moment, really, in taking economics-- I saw it described very well in one article-- these last few Nobel Prizes are the beginning of recognizing that economics is not what we teach in this course anymore.
You need what we teach in this course to go on in economics.
But probably the first couple dozen Nobel Prizes were for about what we teach in this course.
And the last few have been about what you teach in the subsequent courses.
And that's a real evolution of Freakonomics, to understand that we need to mature as a science and move beyond the basics that you learn in 14.01 and move beyond to these other things.
So we're giving you the basics here.
But the excitement happens elsewhere.
So it's a very exciting time for our department and for the profession as a whole.
Now with that little diatribe aside, let's go back.
So we have these.
So now, having said all that, forget it.
Forget Peter Diamond existed.
We're now going back to perfect competition.
And, once again, as I said in the first lecture-- and Peter would be the first guy to say it, this is how he taught me to do economic theory-- you've got to make simplifying assumptions if you want to get anywhere.
So we're going to make a simplifying assumption of perfect competition.
We'll weaken that as we go along.
But, for now, imagine it's perfect competition.
And we have the situation of perfectly competitive firms.
Now a very important distinction to draw-- and that's why it's important to remember that these are little q's not big Q's-- is the distinction between firm demand and market demand, firm versus market.
And this is something which is confusing.
It confuses me at times.
I may even get it wrong at times.
I'll need you to correct me.
Even if a given firm faces perfectly elastic demand, it doesn't necessarily mean that market demand is perfectly elastic.
That is the overall demand for little, fake Statues of Liberty around Port Authority in New York is not perfectly elastic.
As the price goes up, fewer people will buy them.
As the prices goes down, more people will buy them.
But for any given vendor selling them, it is perfectly elastic.
Because there's always someplace next door you can go.
So it's very important to distinguish between the demand facing the firm being perfectly elastic and the demand facing the market not being perfectly elastic.
And the way to think about this is to think about the concept of residual demand.
We have a demand function for market D of p. We have a demand function for a market which is that as the price goes up demand goes down.
Now the demand function for a given firm, we'll call the residual demand D super r of p, is equal to what?
It's equal to the demand for the market minus the supply that all other firms in the market provide, S super 0 of p, the supply that all other firms in the market provide.
So the demand for my product as a firm is my residual demand.
It's the market demand minus what other firms supply.
Well, if you differentiate this with respect to price, you'll see that dDr/dp equals dD/dp minus dS0/dp.
This first one is the market demand curve.
We know that's a negative number, because demand curves slope down.
We're not assuming Giffen goods.
We know that's a negative number.
But this is a positive number.
Supply curves slope up.
The amount that other firms in the market will supply as the prices goes up is positive.
Supply curves slope up.
So this is a negative number.
But this is a positive number which means, by definition, this is a very negative number.
The firm's residual demand responds more to price than the market's demand does.
Because the firm's residual demand is after all the supply of other firms.
So we can rewrite this in terms of elasticities.
So let's assume, for a second, that all firms are identical.
Assume, for one second, that we're in a market where all firms are identical, that little q equals big Q over N. Assume that all firms are identical.
And so, therefore, the amount produced by other firms, Q super 0 is (n - 1) x q.
So, basically, the amount that's produced by other firms is (n - 1) x q.
So the last is demand facing a given firm, epsilon sub i is n times the elasticity of demand for the entire market minus (n - 1) times the elasticity of supply for the market.
So, for example, let's say you've got a market with 100 firms in it.
It's a big market but not outrageously big.
We have plenty of markets with more firms than that.
And let's say that the elasticity of demand for this market equals minus 1.
So it's in between elastic and inelastic, not a crazy number.
And the elasticity of supply is 1.
Let's just say that's the example.
Then what you get is that for a given firm, if you used this formula, the elasticity of demand facing a given firm is minus 199.
It's a huge negative number.
So even though the market demand is modestly elastic, minus 1-- it's elastic but not crazy-- the demand facing the given firm is crazy elastic.
So, basically, the point is that even if a market does not have super elastic demand, a given firm can face very elastic demand.
And that's what can lead to perfect competition.
It's very important to keep those distinct.
When we talk about demand, think about demand at the firm level versus demand at the market level.
Demand at the market level, that's about substitutability with other goods and the things we've talked about deriving demand curves.
When we derive demand curves, we're not deriving firm demand curves.
We're deriving market demand curves.
And so the demand curve was a function of elasticities and substitutability across goods.
The firm demand curve is a function of all that but also how many firms are in the market.
If there are a lot of firms in the market, it's going to be very elastic in a perfectly competitive market.
Questions about that?
It's an important distinction to keep in mind.
Now, with that as background, let's now come to profit maximization which is what this is all about.
Remember I said we assume that every decision consumers make is driven by utility maximization.
Every decision producers make is driven by profit maximization.
So let's talk about profit maximization in the short run.
How do firms maximize profits in the short run?
Now, the first question we have to ask is what is profits?
Well, that seems pretty straightforward.
I defined those already.
I said, the profits were equal to revenue minus costs.
Profits are equal to revenue minus costs.
Well, the trick is that there's two different types of people who measure costs.
Revenue is revenue.
It's just, basically, the money you make.
And anybody can measure that.
But there's two different ways of measuring costs.
There's accounting costs, and there's economic costs.
And these are different concepts.
Accounting costs are cash flow costs what you actually pay.
So your accounting costs are what you actually pay.
So if you buy something for x dollars, that's your accounting costs.
The economic costs are about opportunity costs, which is not about what you lay out in cash, but what you could have done with that cash.
It's not just about what you lay out, but what you could have done with that cash.
So to give you an example, let's just do a simple example.
Imagine that you graduate.
You're going to graduate.
I don't mean that.
You're going to graduate.
Imagine that after you graduate, you decide you're going to start a website design firm on the side.
You're going to do this while you decide what to do with the rest your life.
You're trying to figure out where to go to grad school or whatever.
You'll start a website design firm.
That seems easy.
Basically, how does the website design firm work?
Basically, you work full-time, and you hire some slave programmer who works for you.
He does all of the grunt work.
And let's say that you have to pay him $40,000 a year.
So it's going to be you working full-time plus some slave programmer you're going to pay $40,000 a year.
And let's say that you have a computer that's like six months old, a year old.
It's still in pretty good shape.
It's not brand new, but it's still in pretty good shape.
And you just let the slave programmer use that.
So you don't have to buy a new one.
So you've got the programmer, you're paying him $40,000.
You're letting him use your computer.
You buy some new computer you're working on.
So you do your work, and he does work.
You put it together.
At the end of the year, you tally up all the receipts you've had from your website design, and you've made $60,000.
So you sit back and say, well, that's pretty good.
I put in $40,000.
I paid $40,000.
I brought home $60,000.
That's $20,000 in profit.
That's not a bad profit.
If we think about profit margins, we'll often think about profit relative to revenue.
Well that's $20,000 of profit on $60,000 of revenue.
It's a 33% profit margin.
Most companies would kill for that.
So you say, that's not bad.
I made a 33% profit margin.
But what opportunity costs did this calculation miss?
So if you were an accountant, you'd stop there.
That's why accountants don't make as much as economists.
Because we're better.
If you're an accountant, you'd stop there.
But if you were an economist, what do you recognize?
What did this calculation miss?
What opportunity costs were involved in running this firm that the cash flow calculation didn't capture?
Yeah?
If you plot your grunt worker and your computer, you might have been able to do more work, because the computer might be more efficient
Yeah.
OK. That's sort of a different issue.
That's sort of about the fact that you might not have produced as efficiently as possible.
I'm not asking that question.
I'm asking, given the numbers I gave you, why did I misstate your economic profit?
Yeah, in the back
You could have gotten another job that paid more
I could have gotten another job that paid more.
Here's another thing about it.
I just spent an entire freaking year, and I made $20,000.
You're hoping you do better than that with an MIT degree.
At least your parents are hoping you do better than that with an MIT degree.
So, basically, the first source of opportunity cost that this has missed is the value of your time.
That doesn't show up as an accounting cost.
But it's a real opportunity cost, because you could have had another job.
So let's say you could have gone out and made $60,000 a year.
You could have easily found a job making $60,000 as an MIT graduate.
Well, then that's a cost of running this website.
By spending the year setting this up, you forgo-- I don't know what the past tense of forgo is-- forgoed?
$60,000 in income you could've earned.
You've forgone $60,000 you could have earned.
That's a real cost.
It's not an accounting cost.
It didn't show up on anybody's books.
But it's an opportunity cost.
What else?
Yeah
Also the $40,000 that you gave the program, you could have invested it somewhere else.
And it could be growing
You could have invested the $40,000.
You gave him the $40,000.
You paid it.
It's gone.
If you put it in the bank, you could've earned interest on that money.
That's an opportunity cost.
And we'll come back and talk about capital markets and interest later.
But that's opportunity cost.
What else?
There's one more
You could have sold the computer
I could have sold the computer.
I gave it to him, and he used it for free.
But if I could have sold that for $1,000, that's an opportunity cost as well.
So, in fact, if I could have worked for $60,000.
On the $40,000, I could have made $2,000 of interest.
And I could have sold the computer for $1,000.
Then, actually, my opportunity costs were $63,000 plus the $40,000 I paid the guy.
So, actually, the entire cost of the operation was $103,000.
So I actually lost more than $40,000 running my little website business.
So opportunity costs represent the fact that you could have done other things with your resources.
It's not just the fact that your cash flow said positive $20,000.
Your economic flow said minus $43,000.
Because while you're plus $20,000, you were minus $60,000 that you could have earned.
Now you're down to minus $40,000.
You're minus $2,000 that you could have earned in interest on the money you paid the programmer.
Now you're down to minus $42,000.
And you're down another minus $1,000 you could have made by selling that computer.
So you're at minus $43,000.
So you actually lost money.
So when we talk about profit, we want to think about economic profit, not accounting profit.
Now, I'm not going to make that distinction much now.
But I want you to keep that in mind as you go out and think about starting your business or think about whether firms are profitable, remember we use an economic concept which accounts for opportunity cost, not just an accounting concept which follows the dollars.
Question about that?
OK. Now, armed with this, we now say, OK, how does a firm maximize profits?
Well, that's easy.
We say that profits is a function of quantity produced.
Revenues is a function of quantity produced minus cost as a function of quantity produced.
Remember, what was our goal when we laid out the start of this lecture?
It was to figure out what little q a firm chooses.
Well, what little q a firm chooses is dictated by maximizing this equation.
So a firm will choose little q such that dR/dq equals dC/dq.
That's the profit maximizing equation.
A firm will choose its quantity such that dR/dq equals dC/dq or, to put it in economic terms, where marginal revenue equals marginal costs.
The firm will choose to produce a quantity q where its marginal revenue, which is the revenue made from selling the next unit, equals it's marginal cost, which is the cost incurred by making the next unit.
Well, in a competitive market, we know what dR/dq is.
Because remember in a competitive market, dR/dq is given to the firm by the market.
In a competitive market, dR/dq, or marginal revenue, equals the price.
The price is given to the firm.
It comes from God in the competitive market.
We'll talk later about where it comes from.
But, for now, let's consider it God.
They're price takers.
They just get some price handed to them.
So in a competitive market, we know what marginal revenue is, it is price.
So what this says is that in a competitive market, the profit maximizing equation is price equals marginal cost.
Memorize it, put it under your pillow.
In a competitive market, price equals marginal cost is the profit maximizing condition.
You will produce until the marginal cost of producing the next unit is equal to the price you can sell that unit for in the market.
So, to see that further, let's look at an example.
Let's go to the next figure, Figure 10-2a.
For the next few examples, I'm going to use a particular cost function.
The cost function I'm going to use to make this all concrete is C equals 10 plus 0.5q squared.
That's the cost function I'm going to use.
So armed with that cost function, let's say the price in the market is 6.
Let's say the price in the market 6.
I just pulled that out of my hat.
It's a market for whatever.
We're just doing an example.
Now, what we have on this graph, we have a cost curve and a revenue curve.
The cost curve literally graphs that function.
So, in other words, if you produce two units, then your cost is 10 plus half of 2 squared or 12.
If you produce 2 units, your costs are 12, and so on.
You've got this cost curve.
You've also got a revenue curve.
Well, the revenue curve is just 6 times the number of units produced.
Because the price per unit is 6.
So it's just a straight line with a slope of 6.
So each unit you produce, you make 6.
Now, what you'll see here is that for this cost curve, what's marginal cost?
Well, if we differentiate this, we see that marginal cost equals q. I made this an easy example.
If you differentiate that, you'll see that marginal cost equals q.
That is just differentiate this cost equation.
Marginal cost equals q.
So what that says is that the profit maximizing point is going to be to set marginal cost equal to price.
So that says set q equal with 6.
And you're done.
That's how much the firm should produce.
The firm should produce six units, because it's marginal cost function is q.
It's a linear function.
The price is six.
That's a horizontal line.
So they only intersect in one point where quantity equals 6.
Now, what you notice in this graph is that this happens to be the point where the gap between the revenue curve and the cost curve is largest.
These are not marginal curves.
This is revenue and cost.
But notice that at 6 that's exactly where the revenue curve and the cost curve have the largest gap between them.
I think a more intuitive way to see this is to flip to Figure 10-2b.
This shows the marginal profit that you make on every unit you sell.
So, in other words, if you sell fewer than 2 units, you lose money.
Why?
Because if you sell 1 unit, your costs are 10 plus 10.5.
If you sell 1 unit, your costs are 10.5.
Your revenues are 6.
You lose money.
You lose 4.5.
If you sell 2 units, you make 0.
Your costs are 12, your revenues are 12.
You make 0.
If you sell 3 units, your costs are 14.5, 10 plus half of 9.
Your revenues are 18.
So you make money.
So each unit, you can calculate how much you're making on that next unit.
On the 3rd unit, you made money.
4th unit, you make even more money.
5th unit, even more.
6th unit, you make the most money you can make.
In the 6th unit, your costs are 10 plus half of 36, or 18, so 28.
Your costs are 28.
Your revenues are 36.
So you make a profit of 8.
You make a profit of 8 on that 6th unit.
Think of yourself as climbing this hill.
In economics, optimization is a hill climbing exercise.
Think of yourself as climbing up this hill and asking yourself, should I make the next unit?
Does the hill keep going up?
Yes, it keeps going up.
Make that next unit.
Then you get to the top.
And now you say, well, should I make the 7th unit?
Well if I make a 7th unit, my costs are 10 plus half of 49, so 34.5.
My revenues are 42.
So my profits are only 7.5.
I make less profit on that 7th unit.
So I shouldn't make it.
Now, you might say, wait a second.
Why wouldn't you make it?
You still make profits.
Why wouldn't you go ahead and make all units all the way down to 10?
You still make profits on those units.
And the answer is because of opportunity cost.
The answer is that yes, you make profits.
But given where your marginal cost curve is, you could do better by that point going and producing other goods.
That's why I care about accounting costs versus opportunity costs.
Accounting costs says look, you're going to make money until you go to 10 units.
But opportunity cost says, no.
At that point, the opportunity cost has gotten high enough that you could do better devoting your resources elsewhere.
And that's why you want to stop at the point where you're at the top of the hill, where the price equals the marginal cost.
So what are the profits you make at the top of this hill?
Well, if you go to the next diagram, we can see what the profits you make are.
So this next diagram, Figure 10-3, illustrates this example.
So what we have here is an example with cost curves for this cost function, once again, 10 plus 0.5q squared.
Average costs is that line in the middle there.
That's the average costs.
You have an average variable cost that's a line, that's linear, an average variable cost that has a slope of one.
You have an average fixed cost that's everywhere declining, because your fixed costs of 10 is everywhere declining.
As you produce more and more, your fixed costs are declining.
And, as I said, you have a marginal cost of q.
So your marginal cost of 1 unit is 1.
Your marginal cost of 2 units is 2, et cetera.
So this draws out the cost curves that correspond to that cost function.
You see them drawn out here.
Now, we also, on this diagram, have a demand curve.
The demand curve is perfectly elastic facing this firm.
It's a perfectly competitive market.
And that perfectly elastic demand curve is horizontal at price equals 6.
So what does the firm do?
It chooses to produce where marginal cost equals price.
When it produces where marginal cost equals price, then what profits does it make?
On each unit, it makes a difference of the profits between the price and average cost.
Now it's an important distinction.
We went over this in lecture and section.
But let me go through it again.
Marginal cost is the cost of the next unit.
Average cost is the average cost of all the units you've made.
So if I make 6 units, what profit do I make?
Well, on that 6th unit, I make a profit of 1 and 1/8 or 1 and 1/4.
I make profits of 1 and 1/4 on that 6th unit.
But I make those profits on all 8 units I sell.
So what that means is, in total, I'm going to make a profit of 8.
The area of this rectangle is eight.
Here's the key.
You cannot choose a production level that produces a bigger rectangle than this.
So if you produce 7, your rectangle will be longer.
But the gap between price and average cost will be smaller.
So your total rectangle size would fall.
The largest rectangle is produced at a production level of 6.
That's the most efficient use of your resources is when you produce at a point where marginal cost equals price.
Because when you produce at marginal cost equals price, that causes the maximum gap between price and average cost.
Flip back for a second to the first figure.
Not the first figure, I'm sorry, 10-2.
This relates back to 10-2.
As I noted, the largest difference between your revenue and your cost curve was that 6 units.
That's where the gap is large between revenue and cost.
That's where you produce.
Now we flip forward again to Figure 10-3.
You see that corresponds to the point of the largest gap between price and average cost, I'm sorry, the largest of the points on the marginal cost curve between price and average cost.
So if you produce on the marginal cost curve, if you produce 7 units, then you're going to have a smaller gap between price and average cost.
And, therefore, even though you produced 1 more unit and are making profit on it, your total profits will fall.
The key point is that yes, climbing back down that hill I still make money.
But I make less and less money.
And that, as a result, that rectangle is being reduced as I climb down that hill.
That rectangle is maximized at the top of that hill.
That's the point at which I make the most money.
So that's why we say profit maximization occurs at the point where price equals marginal cost.
Because that is the point of greatest gap between revenues and costs.
Where price equals marginal cost is one of the greatest gaps between revenues and costs.
That's the profit maximizing point where that rectangle is largest.
And you can demonstrate for yourself, and you should, that at any other production level, that rectangle will be smaller.
Questions about that?
Yeah
What do you do if you have a linear cost function.
So that means marginal cost is a constant.
Then how do you determine the line?
You don't.
It's an indeterminacy.
That's right.
I'll try to avoid giving you problems like that.
Basically you get a corner solution.
In a perfectly competitive market, you'd get an indeterminate solution.
In a nonperfectly competitive market, you will have a determinant solution.
But in a perfectly competitive market with a linear marginal cost, you'd have an indeterminate solution.
Either you produce 0 or infinity.
So you'd have an indeterminate solution.
Now, just a further drill this in, imagine, for a second, there was a cost shock to the firm.
Imagine there was a cost shock to the firm.
Imagine that there's a tax on the firm where the firm has to pay a tax of an amount t. Let's say t is $1.
The firm has to pay a tax of $1 on every unit they produce.
Well, what is their new cost curve?
Somebody tell me what the new cost curve is.
If you have a tax of $1 on every unit you produce, someone tell me the equation for the cost curve.
I didn't write it down here, did I?
No.
OK, what's the equation for the cost curve.
Yeah
Just add plus tq to the top thing
Exactly C equals 10 plus 0.5q squared plus tq.
Because for every unit you produce q you pay a tax t. And since t is $1, it would just be plus q.
If t is $1, it would just be plus q.
So the cost function has now shifted.
So what we see is that it has shifted average cost and marginal cost both upwards.
Average cost has shifted upwards by that amount.
And average cost has shifted upwards.
And marginal cost has shifted upwards by precisely $1.
It's just a linear shift of marginal cost.
Because marginal cost is now q plus 1.
If you differentiate this with respect to q assuming t is 1-- t is now $1-- if you differentiate this with respect to q, marginal cost is q plus 1.
So your marginal cost curve has shifted up by $1 or by t, in this case, in the more general case.
Well, what does that do to the first production decision.
Well, now it doesn't change their maximization formula.
They still want to set marginal cost equal to price.
So now they say, set q plus 1 equal to price.
Well, the price is 6.
So it says set q equal to 5.
Now the profit maximizing level is 5 units.
The profit maximizing level is 5 units.
What this is saying is because the tax increases your marginal cost, you are now producing less.
And what we can see now, if we flip to Figure 10-4, we can see what that's done to your profits.
Your profits have now fallen to the dotted rectangle, the much smaller dotted rectangle.
Your profits used to be that entire slashed rectangle plus the dots.
Now your profits are just where the price exceeds average cost which is just that smaller dotted rectangle.
Your profits have fallen dramatically.
So by imposing a tax on this firm, we've dramatically reduced their profits.
Now, this is the economics behind why taxes can lower production.
Now, you might say, wait a second.
In reality, if we tax a firm, the don't have to lower their profits.
They just pass it on and charge consumers higher prices but not in a competitive market.
In a competitive market, they can't do that.
If you tax a firm in a competitive market, it comes out of their profits.
Because they face a perfectly elastic demand, so they can't raise the price.
All they can do is just say, well, the price is the same.
My marginal cost has gone up.
I'm going to produce less, and I'm going to make less profit.
And that's life.
So a tax in a market like this is just going to lower the firm's profits, and it's going to lower their level of production from 6 to 5.
In noncompetitive markets, different things can happen.
We'll talk about that.
But in a competitive market, this is what happens with the tax.
You basically set the new marginal cost equal to price, and you get that they produce 5 units instead of 6 at a lower profit level.
Yeah?
Would that be a good cause for entering the market--
You're right.
That's an excellent point that I should have pointed out.
What do I mean by short run?
What I mean by short run is remember, labor is variable.
Capital is fixed.
What do I mean in this context?
Think about the short run as being the period over time over which firms cannot enter and exit.
When we talk about competitive markets in the short run, we talked about short run being the time in which capital is fixed.
Now we're going to add another condition.
We're going to say the short run is the period of time in which there's no firm entry or exit.
That's the short run.
We'll come back and talk about what entry and exit does.
And that's going to have some funky effects.
We'll talk about that next time.
But what we mean by short run here is no firm entry and exit.
Whichever firms are in the market at the beginning of the short run people are in the market at the end of the short run period.
Yeah
But in your conditions, you had like free and fluid entry and exit of firms
Right, exactly.
And that's why I said, don't pay attention to that yet.
In some sense, those are the conditions of perfect competition.
I didn't say if that was short run or long run.
Those are the full long run set of conditions.
That's why I said we're going to ignore four for now and just focus on the first three.
In the short run, only the first three are relevant.
Because in the short run there's no firm entry or exit.
In the long run, which we'll talk about next time, that fourth one will be relevant.
Good question.
That's a good point.
Thank you for pointing that out.
Other questions or comments?
One other thing I want to cover before we stop which relates to the long run, is that in the long run, this is sort of the transition, the bridge to talking about the long run.
We also have to decide, ultimately, whether or not we want to shut the firm down.
So, in other words, we want to set price equal to marginal cost.
That's one condition.
But, actually, short run profit maximization has a second condition.
Short run profit maximization has two conditions.
The first is to set price equal to marginal cost.
The second condition of short run profit maximization is to check whether the firm wants to shut down.
Why would a firm want to shut down?
It might want to shut down if it actually loses money by continuing to produce.
And that's because firms may lose money but not shut down.
Firms may lose money but not shut down.
Or firms may lose so much money they shut down.
And we need to consider that.
Let's get rid of the tax.
Let's go back to the marginal cost with the tax.
Imagine the price in this market suddenly fell from 6 to 3.
The price of the market is now $3 per unit you sell.
Well what would the firm's profits be?
Well, if the price fell to 3, the firm would choose to produce 3 units.
You would still have this condition, marginal cost equals price.
Price is 3, and marginal cost is q.
So q is 3.
You still produce 3 units.
If you produce 3 units, its costs are 10 plus 4.5 which is 14.5.
At a price of 3, it makes 9.
So its profits are negative 5.5.
It would lose money from this production.
Remember marginal cost equals price.
That doesn't vary with what the price is or anything.
This is a maximizing condition.
If a price changed, it's not like you change which equation you follow.
You always follow this equation.
The efficient production level is always marginal cost equals price regardless of what the price is.
So if the price is 3, the efficient thing to do would be to produce 3 units and lose money.
Now, you might say, well, then that's stupid.
If that goes negative, wouldn't they just shut down?
And the answer, is in the short run, no.
And the reason you wouldn't shut down in the short run is what I talked about last time which is about the notion of sunk costs.
In the short run, the fixed costs that you paid to produce are sunk.
They're unchangeable in the short run.
In the long run, they're changeable.
You can just leave.
But in the short run, you've invested fixed costs of 10 in being in this market.
You've paid a fixed cost of 10 to produce in this market.
Given you're in this market, and you've paid 10 to produce, you will not exit unless you lose more than $10.
You will not shut down unless you lose more than $10.
You will not shut down unless you're losing so much money that you can't cover your fixed costs.
Because you've paid those fixed cost.
In the short run, they're sunk.
So unless you're actually losing more than your fixed costs, you will not shut down your firm.
OK.
This is actually pretty confusing, and we're out of time.
So what I'm going to do is we'll pick it up on Monday exactly at this point.
And I'll go through some more of the intuition for this.
And that will be our segue for talking long run profit maximization which we'll talk about on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
All right.
So today we're going to continue our discussion of the perfectly competitive market outcome.
And remember, once again, where we're coming from here.
We're trying to figure out how firms decide how much to produce.
We talked about the firm's production decision in costs.
Then we said how the firm decides how much to produce is going to get dictated by the market.
We're going to talk about different market structures.
We'll start with our benchmark of perfect competition and then move on to some more interesting and realistic cases next.
Now, I just want to finish up where we were last time.
Remember last time we were dealing with a firm that had a cost function of the form 10 plus 0.5q squared.
And if you remember, the key condition we derived last time for profit maximization with a perfectly competitive firm is that price equals marginal cost.
If you differentiate this with respect to q, you get that that means that p equals q is the profit maximizing condition for this firm.
It sets the price equal to quantity it's going to sell.
That's the profit maximizing condition with this particular functional form of the cost function.
Now, before we left last time, I said, this was not enough.
There's one other thing you have to consider which is the firm's shutdown decision.
In the short run, a firm might not shut down even if it's losing money.
And the reason is because the firm has already paid its fixed costs.
It's already paid 10.
So even if it's losing money, it might still not shut down.
So, for example, imagine that the price, as I said last time, imagine the price fell from 6 to 3.
The price equals 3.
If the price equals 3, the firm will choose to produce 3 units, because it will still follow the profit maximizing condition.
It will still choose to produce three units.
If it produces 3 units, its profits are its revenues which is 9-- 3 units at a price of 3-- minus its cost which is 14.5.
So that equals negative 5.5.
So its profits are negative.
So now I say, well, if profits are negative, maybe I should shut down and stop doing business.
Well, what are its profits if it shuts down?
Negative 10.
If it shuts down, its profits are 0 minus 10 equals negative 10.
So it actually makes more money by staying in business than shutting down, because it has these fixed costs.
So because it's going to pay the 10 anyway, as long as it's going to lose less than 10, it might as well stay in business.
More generally, what we say is a firm will stay in business in the short run as long as its price covers its variable costs.
So a firm will stay in business so long as the price is greater or equal to its variable costs.
Then it will stay in business.
If we go further-- let me just derive it for a second-- as long as they cover its fixed costs, that means a firm will stay in business as long as its revenues are greater than its variable costs.
As long as its revenues are greater than or equal to its variable costs, it will stay in business.
And that means that it will stay in business at long as its price is greater than or equal-- this should be a little q, sorry, this is just a firm-- as long as its price is greater than or equal to variable costs over quantity, or as long as price is greater than or equal to average variable cost.
As long as its price is greater than or equal to its average variable cost, it will stay in business.
As long as its revenues cover its variable costs, it will say in business.
And that's saying the same as long as its price is great than or equal to average variable cost, it will stay in business.
Now, what are the average variable costs for our firm?
Well, the variable costs for our firm are 0.5 times q squared.
The variable costs for our firm are 0.5 times q squared.
We know that in equilibrium, if it's profit maximizing, it will produce where q equals p. So we can replace the q with the p. The variable costs are 0.5 times p squared, because we know we'll produce where q equals p. So its average variable costs are 0.5 times p. Its average variable costs are 0.5 times p. Well, by definition, p is always greater than 0.5 times p. So our firm will never go out of business in the short run.
In the short run, our firm will never go out of business, because at the profit maximizing price, at p equals q, it will always be producing a point where the price is greater than its average variable cost.
So it will never shut down.
So, more generally, when we think about a short run supply decision, a short run supply decision for a firm, there's two steps.
The first step is set price equal to marginal cost to figure out what the firm is going to produce.
So step one is set price equal to marginal cost.
And that will give you the firm's q*.
That will give you what the firm is going to produce.
The second step is check that price is greater than or equal to average variable costs.
Because you may solve for an optimal quantity that turns out to be a money loser for the firm.
So they'd rather shut down.
So it's a two-step process.
You've got to first solve for the optimal quantity that the firm is going to produce.
But then you've got to make sure that the firm actually makes money on that quantity, or it won't produce at all.
And that's how we do the profit maximization decision in the short run for the firm.
You've got to produce at the efficient point and make sure the firm actually makes some money.
All right, questions about that?
Now, armed with these rules, we can now, finally, derive the supply curve.
Remember we derived the demand curve a number of lectures ago by getting the tangency at different price ratios with the indifference curves.
Well, to derive the firm's supply function, what we now need to do is say, OK, at different prices, how much will the firm produce?
Well, we can now get that if we go to Figure 11-1.
We can now see the supply curve for this firm.
What we see is that at a price of 3, it will produce 3 units.
At a price of 4, it will produce 4 units, et cetera.
The supply curve is the marginal cost curve.
So now we know where supply curves come from.
Supply curves are marginal cost curves above the point where price equals average variable cost.
So the definition of a firm's supply curve is the marginal cost curve above p is greater than or equal to average variable cost.
That is the firm's short run supply curve.
Now, in our case, p is always greater than average variable cost.
So the second condition is irrelevant.
The firm's supply curve is just literally that marginal cost curve.
With different functions, which you may someday see in a problem set or an exam, that won't be true.
So, in that case, you'll need to check that shutdown condition.
But the supply curve is the marginal cost curve above that 0 profit point.
And that's where supply curves come from.
So where supply curves comes from is the same kind of maximization we did with consumers.
But instead of their parents giving them their income, the market conditions firms face are dictated by the competitive nature of the market.
And that's the firm's supply curve.
Now, this is the firm's supply curve.
Now, of course, what we talked about in the first lecture was not firm supply curves but market supply curves.
So now let's take the next step and say, well, where do market supply curves come from?
We now know where firm supply curves come from.
The marginal cost stork brings them.
Now, where do market supply curves come from?
Well, to do that, we need to now imagine that there's not one firm in the market but many firms in the market.
And we need to recognize that the market demand may not be perfectly elastic.
But, as we talked about last time, the firm's own demand will be close to perfectly elastic.
Or, in this case, a perfect competition will be perfectly elastic.
So, basically, the way to get market demand is to say, look, we're going to take each firm.
It's going to take a market price as given.
I'm sorry, we get market supply.
I'm sorry.
Each firm is going to take a market price as given.
Based on that market price, it's going to decide how much to produce.
We're going to add up that production.
That will make a market supply curve.
And that market supply curve will then interact with market demand to give you a price.
If that price is the same one the firms were using, then the whole thing is in equilibrium.
Let me explain that in less steps just to make it clear.
Let's talk about the steps involved in getting to short run equilibrium in the market, the steps involved in getting to short run market equilibrium.
The first step is each firm chooses an amount of capital.
So the first step of the short run is you're going to enter this market.
And to enter this market, you're going to have an amount of capital you're going to pick.
So each firm is going to have some cost function which involves picking some amount of capital or fixed costs.
It's going to say, I want to build a building this big.
Having built that building, we're going to get the firm's supply curve which is p equals MC.
That's step one.
That's a step we've derived.
The second step is we're going to add up the firm's supply curves to get a market supply curve.
We're going to add up the firm's supply curve to get a market supply curve.
So, for example, suppose that there's five firms in the market.
Suppose that there's five firms in the market.
Those five firms are going to produce.
Now to see that, let's go to Figure 11-2.
This is the second step.
It's how we get to that short run market supply curve.
Each firm has a marginal cost curve.
Here we're using our same cost function we've used.
That same cost function up there where price equals marginal cost, where p equals q, is the supply curve.
So each firm has that supply curve you see in the first panel.
Then what you see is as you add more firms, the second panel gives you the market supply curve.
So if there's only one firm in the market, the market supply curve would be S1.
Now, if there were two firms in the market, the market supply curve is S2.
That is, at a price of 2, you're now producing 4 units in the market.
If there's three firms, the curve is S3, four firms, S4, and so on.
As you add more firms, that market supply curve shifts out and becomes flatter.
Remember firms are identical here.
We're adding identical firms.
That was an assumption of perfect competition.
We're adding more and more identical firms producing the same good.
You can see that market supply curve is shifting out and becoming flatter.
That is the supply of goods is becoming more elastic as there are more firms.
The more firms in the market the more elastic the supply.
And that comes to what we talked about last time when I derived residual demand.
It's sort of the flip side of that.
Basically, the more firms you have with a given supply curve in the market, the more elastic it's going to become.
Why is that?
Well, just think about it.
Think about what elasticity of supply is.
It's saying, if I increase the price by $1, how much more production do I call forth?
Well, the more identical firms I have, every time I increase the price by $1, I call forth production from all these firms.
So the more firms I have, the more production I call forth.
So for every increment in price, the more firms in the market, the more production I call forth.
Therefore, the more elastic is the supply.
So as there are more firms, that market supply curve becomes more and more elastic.
And that's the market supply curve.
That's the second step.
The third step is we intersect market supply with market demand to get the equilibrium price.
So, in other words, we say, look, there's some market supply, which we've derived.
Now let's imagine there's some market demand.
And that will give us the equilibrium price.
So, for example, in our case, market supply is what?
Let's say, for example, there's five firms in the market.
Just to make an example, let's say five firms have entered.
There's five firms in the market.
Well, the total market supply Q is 5 of the little q, because there's five identical firms in the market.
Five identical firms are in the market.
Well, we know, from the marginal cost condition, that's the same as saying Q equals 5 times p. So our market supply curve, which is actually S5 on Figure 11-2, is Q equals 5p.
You can see that.
Because when the price is 2, Q equals 10.
When the price is 5, Q equals 25.
So you can see that S super 5 is the market supply curve.
Big Q equals 5p.
Let's just make this up.
So this is the quantity supplied.
Let's say the demand function is that the quantity demanded is 30 minus p. I just made this up.
I'm making all this up.
But this is just an example demand curve.
We have a downward sloping demand curve with a slope of negative 1.
The quantity demanded is 30 minus p. So to get equilibrium, we set these equal.
And we get that 30 minus p equals 5p or p equals 5.
30 minus p equals 5p or p equals 5, that's the equilibrium price.
Given the market supply curve, given the demand curve, I've derived the equilibrium price of 5.
Now, at a price of 5, what's the quantity demanded?
At a price of 5, quantity demanded equals 25.
So, at a price of 5, the market wants 25 of these things, whatever the heck it's producing.
At a price of 5, the market wants 25 of them.
That's the quantity demanded.
Then the final step in solving for equilibrium is that each firm then decides how much to produce.
Well, what is each firm going to decide to produce?
How much is each firm going to decide to produce?
Somebody raise their hand and tell me.
Yeah
p
p which is?
5
5.
So each firm is going to produce 5.
How many firms are there?
5
How much gets produced?
25
Which is exactly what people want.
We're done.
That's the magic of the market.
Through these four steps, we've gotten equilibrium which is defined as the quantity supplied equals the quantity demanded, just by following these steps.
The firms didn't come at it saying, hey, let's figure this out beforehand.
Let's all get together and coordinate and figure out how much we're going to produce at what price.
They didn't do that at all.
The firms just entered the market.
They said, this is my supply curve.
We added it up.
We interact it with demand.
We find an equilibrium price.
At that equilibrium price, the quantity demanded equals the quantity supplied.
We're done.
We've now finally gotten to where we started this course.
We started this course with a supply and demand graph.
I told you how to derive demand.
You took a test on that already.
Now I've just told you where supply comes from.
Now we interact, and we're done.
And this tells us now, given this price, how much each firm is going to produce.
So, basically, what do you need to find equilibrium?
What do you need to find the short run equilibrium?
To find the short run equilibrium, you need a demand function, a cost function, and a number of firms.
You have to be given a number firms, because there's no entry and exit in the short run, remember.
So, basically, the firms are magically put on the earth in the short run.
So if you're asked about the short run equilibrium, you have to be given the number of firms.
You can't derive that yet.
That comes in the long run.
But given a number of firms, given a cost function, and given the demand function, you can find the short run equilibrium.
Questions about that?
Now, with that in place, now let's get to where it gets really interesting.
Let's talk now about the long run.
Now what makes the long run interesting is you no longer have to be given the number of firms.
Now, in the long run, we're going to actually figure out how many firms are in the market.
So at the end of the day, in the long run, all we'll need is two things, a demand function and a cost function.
And then we'll be done.
Now, here's the key thing for the long run.
The key point in the long run is that in the long run, no one can lose money.
So we don't have to worry about the shutdown condition anymore.
The shutdown condition goes away.
In the long run, nothing is fixed.
So in the long run, no one loses money.
There's no reason to be in the market if you're losing money in the long run.
So, in long run, the first thing is now we only have one condition to worry about, which is price equals marginal cost.
Price equals marginal cost is what we need to worry about.
And you're only going to be in a situation, in the long run, where you're only going to be making either 0 or positive profits.
If you're making negative profits, you'll be gone.
Now, the key difference in the long run, is now we can't take the number of firms as given.
Now we need to derive the number of firms.
And the way we do that is by thinking about entry and exit.
Now, what's going to determine entry and exit?
Well, it's quite simple.
If in the market, as it stands today with some number of firms, there's profit to be made, new firms will enter.
If in the market, as it stands today with some number of firms, there's losses being made, some firms will leave.
Remember, no one stays in making losses anymore.
And that continues until you reach a situation where all firms make zero profit.
And here's the key lesson.
In a perfectly competitive long run equilibrium, all firms make zero profit.
It's the fundamental lesson about perfect competition.
Obviously, there's no place that works like this in the world.
This is an extreme.
But, nonetheless, you should understand this extreme.
In a perfectly competitive long run equilibrium, all firms make zero profit.
Why?
Because if there's any profit to be made, a new firm will enter and take it away.
And if there's any unprofitable industry, a firm will exit until the profits go back to zero.
So profits will always be zero in the long run equilibrium.
Now, to understand how this works, let's think about a realistic example.
Let's think about the PC market circa 1990 when all you youngins were being born.
It's circa about 1990, the PC market, cast your mind way back.
It's a history lesson for you guys.
In 1990, not that many folks had PCs.
There was still a vibrant use of mainframe computers.
And, basically, you had big firms like IBM who were producing these mainframe computers, which is what I did my computing on.
A lot of people did their computing on it in 1990.
And there was starting to be a market, however, for personal computers.
The chip strength had gotten large enough that it was actually viable to have desktop computing that was powerful enough.
Firms like Dell were starting out-- not starting out, necessarily-- but were starting to make money making desktop computers, making PCs.
Let's actually think about how that market might look.
So let's actually talk about, in Figure 11-3, this is the market for PCs circa 1990.
In 1990, if you were making PCs, that was a great business to be in.
Because people wanted them, there weren't that many firms making them, and you were making a killing.
So if you were Dell in 1990, that was a great place to be.
So let's say Dell in 1990 was facing a demand curve D and the supply curve SR1.
The supply curve was pretty steep because there weren't many firms making PCs.
So the market price was P1.
So on the right, you have the market.
We should label this actually.
On the right, you have the market.
On the left, you have Dell.
On the right, you have the market.
In the market, in initial equilibrium, there's a price of P1 with big Q1 being sold.
So Q1 PCs are being sold at a high price of P1.
It's the novel technology.
People want it, but not many firms are doing it.
What happens with Dell?
Now let's go to the left-hand side diagram.
Well, Dell is producing where that price equals their marginal cost.
That price equals their marginal cost at little q1.
So Dell's producing little q1.
But its average costs at that point are all the way down.
It's not really labeled.
But you can see it's where that vertical line for Q1 intersects average total cost.
That's where their costs are.
So, in each unit, they're making the height between marginal cost and average total cost at that Q1.
They're making that vertical bar.
So they make that entire rectangle of profit.
So Dell makes a big profit, because not many firms are in this business.
And yet demand for PCs are high.
And that's where things are circa 1990.
So what happens?
Well, Gateway arrives.
Does Gateway still exist?
Do people still buy Gateway?
Gateway is gone, right?
So, while Gateway was big, they were sort of a big upstart.
They had the boxes with cow colors on them and stuff.
You guys didn't even know Gateway?
Well, they were big.
They were the first cheap knockoff computer makers that competed with the big folks.
And they came and said look, we can do this.
OK, we can produce.
This is a profitable business.
We can make PCs.
It's not that expensive.
It's largely a variable cost business.
You just have to build your plant first.
So they built their plant, and then they come in.
Well, what happens when a new firm comes in?
The market supply curve flattens.
Because now, at any price, you're producing more.
So the market supply curve flattens to the point SR2.
In fact, maybe it's not just Gateway.
Maybe you get a bunch of entrants until you get the market supply curve SR2.
Well, SR2 intersects demand at a new higher market quantity big Q2.
Well, that higher market quantity going to the left, there's now no longer profits to be made.
Because at that market quantity, Dell is going to produce little q2.
Little q2 is exactly at the minimum of the average total cost curve.
It's where the marginal cost curve intersects the average total cost curve, the minimum of that average total cost curve.
So Dell no longer makes profits.
Dell shrinks its production.
The price has fallen.
It doesn't lower its price.
Dell doesn't set the price.
Remember, this a price taker in a perfectly competitive market.
The price is given by that diagram on the right.
Dell gets a price of P2.
It says look, at P2, I have to produce along my marginal cost curve.
I have no choice.
That's what's profit maximizing.
I can't choose a point not on that marginal cost curve.
That will not be profit maximizing.
Well, where does that price intersect that marginal cost curve?
At little q2.
So I'm producing at little q2.
And at little q2, I make no profit.
So the entry of Gateway and other firms into the PC business has removed the profit from the PC business.
Now note what's interesting here.
Market quantity has gone up.
Big Q2 is bigger than big Q1.
But Dell's quantity has gone down.
Little q2 is smaller than little q1.
That's because more firms are in the market producing.
So as more firms come in, total market quantity goes up.
But any given firm is going to produce less.
And that will continue until profits go to zero.
That is how firm entry wipes out profits.
That is how firm entry wipes out profits, by driving firms to the point where price equals average cost.
So, in the long run, firms make zero profit because, first of all, entry drives price down to average cost.
Entry drives price down to average cost.
And when price equals average cost, profits are zero.
Profits are zero when price equals average cost.
Because profits are pq minus C. So if you divide by qp profits are p minus average costs.
So if price equals average cost, profits are zero.
So entry drives profits to zero.
It drives price to equal average cost.
Since price equals marginal cost, it's the point where marginal cost equals average cost.
That's the technological outcome in a perfectly competitive long run equilibrium.
You'll end up producing where marginal cost equals average cost.
That's what will end up happening naturally through the forces of entry.
Likewise, you see this through the force of exit.
Now let's go to Figure 11-4.
And now let's look at IBM.
I guess we're calling this the broader computer market now.
It's not just a PC market.
It's the broader computer market.
So IBM, they're producing these mainframes.
This is the mainframe market.
This isn't the PC market.
This is the mainframe market.
In the mainframe market, now people don't want mainframes much anymore.
But there were a lot of firms producing.
There was IBM and tons of other firms producing these mainframes.
Because that's what everybody wanted.
So the original supply curve was very flat at SR1.
So, initially, in the mainframe market, we're in equilibrium with quantity Q1-- big Q1-- and a price P1.
And what's happening there?
Where does that price intersect marginal cost?
It intersects marginal cost for IBM at little q1 which is below average total cost.
So IBM is losing money.
In the initial short run equilibrium, IBM is producing at little q1, and it's losing money.
Now, why is it still in business?
Because it's the short run.
And as long as those losses are less than its fixed cost, it's staying in business.
So, in the short run, you can lose money.
So IBM is losing money.
Because it's built this big plant.
It's cranking out these mainframes.
People don't want them anymore.
The price has fallen so low that they can still make enough money than it costs to produce the next mainframe.
Price is still greater than marginal costs, but price is below average costs.
I'm sorry.
Prices are greater than average variable cost.
But it's lower the total average cost.
They're losing money.
So what happens?
They leave.
There was a company called [?
Deck ?]
that went out of business.
And what happened was that then raised the supply curve.
It steepened the supply curve in the mainframe market.
It steepened the supply curve, because now you have fewer firms producing mainframes.
That supply curve gets steeper.
That raises the price.
And, indeed, exit will continue until you raise the price to the point where marginal cost equals average total cost.
And what you'll see is the market will shrink from big Q1 to big Q2.
The remaining market participants will increase from little q1 to little q2.
And profits go to 0 with price going to the minimum average cost.
So through both entry and exit, we get this condition that's illustrated in Figure 11-5.
In 11-5, we see that in the long run, firms always supply not on a single curve but at a single point.
In the long run, with a perfectly competitive market, for a given firm, there is no longer even meaningfully a supply curve to a firm.
There's just literally a supply point.
Every firm produces at exactly the point where marginal costs equal average costs.
So in some sense, once again, for a given firm-- this is not the market-- but for a given firm, there's not even meaningfully a supply curve anymore.
For a given firm, in the long run, they literally choose one production point which is technologically given.
So this is interesting.
For a given firm, the market doesn't matter.
For a given firm in a perfectly competitive market, we don't need to know anything about demand.
All we need to know is the firm's production function.
That's all we need to know.
We don't even need to know anything about costs.
Well, we need to know cost.
We need to know their cost function.
All we need to know is their cost function.
And then all we need to do is derive where marginal costs equals average costs, and we're done.
This is the power of the perfectly competitive equilibrium.
This is why economists love it so much.
Because we don't need to go through all this.
This is all short run stuff.
In the long run, it's easy.
You just say, give me a cost function, I'll tell you what the firm will produce.
And I'll tell you what the price is.
The firm will produce where marginal cost equals average cost.
And the price will be where marginal cost equals average costs.
I can tell you the p and the q in equilibrium just if you give me a cost function.
And that's the beauty of the long run perfectly competitive equilibrium.
That's why it's so attractive to economists for modeling purposes and other things.
It's incredibly easy to work with.
Because all you need is a cost function, and you're done.
The key lesson is what is true at the point where marginal costs equals average costs?
Well, look at our graph.
That is the point of cost minimization.
Note that that is the very minimum point of the long run average cost curve.
So where marginal cost equals average cost is the point of cost minimization.
So we're saying further-- this is even more powerful-- we're saying that in the long run of perfectly competitive equilibrium firms will, by definition, minimize their costs.
They will produce as efficiently as possible not because God told them to, but through the power of the market.
Because what happens if you start a firm and you aren't cost minimizing?
What happens?
What happens is, in the short run, you might make money even if you aren't cost minimizing.
But, in long run, you'll get driven out of business.
Because if there's someone else who can produce more cheaply than you, they'll be able to charge a lower price and drive you out of business.
Your price will end up above the long run equilibrium price if you're not cost minimizing.
So any firm that is not cost minimizing will get driven out of business.
And the equilibrium will be a market where all firms are producing at the cost minimizing level.
And that's why we get the high tech Figure 11-6, which is that the long run market supply curve is perfectly elastic.
Now this comes all the way back to what I talked about at the beginning of the last lecture.
Remember I said, what determines perfect competition?
Two things, the demand curve to the firm was perfectly elastic, and the supply curve to the market is perfectly elastic.
And we talked last time about why the demand curve to the firm is perfectly elastic.
Because with lots of firms, any given firm has a perfectly elastic demand.
Now we've just derived why the market supply curve is perfectly elastic.
It's perfectly elastic at the cost minimizing point.
If the price ever rises above that cost minimizing point, what happens?
What happens if the price should suddenly rise above it?
What happens?
Firms enter and drive the price back down.
If the price ever drops below that cost minimizing point, firms exit, and the price goes back up.
So through the power of firm entry and exit, in the long run, you end up with a horizontal or perfectly elastic supply curve.
And that's perfect competition.
Questions about that?
Yeah?
You said that if it's not profit maximizing, then it cannot [INAUDIBLE PHRASE].
But [INAUDIBLE PHRASE]
Yes, they are.
So that's a great point.
So what will happen is it's not that a firm will charge a lower price.
I shouldn't have said it that way.
It's that another firm will enter.
And, by definition, the price will then fall, because the supply curve will flatten, and the price will then fall.
So if you're in there producing inefficiently and I say, hey, your firm sucks.
I can come in and produce much more efficiently than you.
I'll hop in, that will flatten the supply curve.
The price will fall.
At that price, if I'm not cost minimizing, I'll be losing money.
So I'll leave.
That's a good clarifying question.
So if we have a market with a bunch of guys in it, and one of them is not cost minimizing, well, that means someone else can come in.
They'll, in the short run, expand the market.
That will flatten the supply curve, drive the price down.
At that lower price, the non-cost maximizing firm says, I'm losing money.
So I leave.
The price goes back up, and it goes up and down and up and down until it settles at this point where costs are minimized.
So perfect competition leads to a perfectly elastic supply and cost minimization.
So that is our extreme.
That is the theoretical point of no return.
Of course, in reality, we never get there.
In reality, there's no such thing as a purely perfectly competitive equilibrium.
Why not?
Well, in the long run, supply is actually upward sloping.
In the long run, market supply will be upward sloping.
And that's going to be for at least three reasons.
So why is long run supply upward sloping in reality?
Well, in reality, if would be for at least three reasons.
The first reason is that entry and exit are not free.
There could be barriers to entry or exit.
Even in the long run, there could be barriers to entry and exit.
There could be features of the market which make it hard to leave or hard to join.
A classic example is something that we introduced a couple lectures ago, the notion of sunk costs, costs which even in the long run might be fixed.
If they're large sunk costs, if one firm's incurred them, another firm is going to say, look, it's not worth it for me to get in.
If you have to build such a massive plant to produce your good, then it takes an unbelievable amount of capital, it's a huge investment, and once you're in, it's going to be really hard to drive you out, because you made that big investment, other firms might say, forget it.
Dell made this huge investment in this huge plant.
They're never going to leave having built that plant.
That's virtually a sunk cost.
So I'm not even going to bother entering.
So Dell can exist making some profit, maybe not too much profit.
If they make too much profit, then another firm will build a plant.
Well, as long as they're not making too much profit, they can make some profit.
Because another firm says, you know what, I can't fight that battle.
I can't build a plant that big.
Or there could be other things like that.
That's sort of a natural barrier to entry.
There are some artificial barriers to entry we see.
For example, take med school.
The number of slots to be doctors is limited by the physician profession.
So even if docs make lots of money, which they do-- especially specialists-- you can't just compete and have new doctors enter.
Because the number of slots are actually limited.
You have to be licensed by an organization which is run by the people making all the money.
So if you're a doc, and you're in, this is a pretty good deal.
You say, hey, let's have a system where new docs can't be licensed.
They'll have to come to me, and I can charge whatever I want.
That's a barrier to entry.
We call that occupational licensing.
We see that in lots of professions.
Plumbing, taxi drivers, we see it everywhere.
It's occupational licensing.
A second example, of course, is patents.
And we'll talk about patents more in a few lectures.
If I invented a new drug, and it's patented, nobody can sell that same chemical compound for 17 years.
That's a barrier to entry.
There could be more informal barriers to entry.
Let's say you're around Port Authority setting up these little shlocky stands, the ones we said were perfect competition.
You've got yours, and the guy comes in next to you, you just beat the crap out out of him.
That's an informal barrier to entry.
You say, you come in, and you're going to get beaten up.
The guy says, well look, if it's a big profit, it's worth getting beaten up.
Or I'll hire protection if it's a big profit.
But it's not that big of a profit, I'm not going to bother getting beaten up over it.
So I'll let you make your profit.
I won't come in.
So barriers to entry exist all over the place.
And they're a big reason why we don't get a perfectly elastic supply curve in most markets because of things like patents or thuggery.
So that's one example of why you don't get it.
Another example of why you might not get a perfectly elastic supply curve is that firms might differ.
In particular, we have assumed critically, through the last lecture and this lecture, that firms are identical.
We have assumed that firms are identical.
But, in fact, of course, firms aren't.
And one firm's cost minimizing production level might be different than another firm's cost minimizing production level.
Not all firms will have exactly the same cost minimizing production level.
In particular, some firms may have a lower minimum average cost than others for a while.
So it may be that as long as I'm producing less than x units, I have a lower minimum average cost than you do.
But once I produce more than x units, my minimum average cost rises to above yours.
Well, in that case, I might be able to make money for a while staying in.
But then once I produce too much, I'm going to have to raise the price.
So to see this, there's a great example in Perloff which you see in Figure 11-7, where he talks about the international long run market supply curve for cotton.
And he says, look, in Pakistan, you can produce cotton incredibly cheaply.
This is a dated example, but in Pakistan, you can produce cotton incredibly cheaply.
You can produce it at $0.71 per kilogram.
So if the world demand for cotton is less than $2 billion kilograms per year, or less than $1.8 billion kilograms per year, then Pakistan would provide it all, and the price would be $0.71.
But let's say the demand is more than that.
Well, Pakistan just runs out of cotton.
They can't do that.
Well, then you have to go to the next cheapest country.
Well, the next cheapest country is Argentina.
It costs a lot more to produce cotton there.
And then comes Australia, Brazil, Nicaragua, Turkey, then finally the US.
And then Iran is the most expensive.
So this is, effectively, an upward sloping supply curve.
It's stepwise, but it's an upward sloping supply in the sense that as you want more quantity, the price goes up.
So if the market wants $5 billion kilograms of cotton a year, then that means that the marginal producer is the US even though they're much less efficient than Pakistan.
Because Pakistan hit a constraint.
You have to go to that next less efficient producer.
So that's taking an upward sloping supply curve.
Basically you're getting an upward sloping supply curve, because you have constraints on how much any given firm can produce.
Those constraints can make you move onto less efficient firms as you go on.
And so in a market, you'll end up, in reality, with a distribution of firms ranging from most efficient to least efficient.
The most efficient would produce as much as it can at that efficiency level.
But then some less efficient ones will get in the game as well just depending on where demand is.
So you can see if you put in demand curves at different points in the supply curve, you get different prices.
That's an upward sloping supply curve.
That's a second reason.
And then a third reason-- these aren't a comprehensive list, but types of reasons why supply curves will slope up in reality-- is that input prices might rise as the market expands.
We've assumed fixed input prices.
I gave you an r and w, and I assume they were fixed.
But, in fact, that might not be true.
It might be that, in reality, as you want to produce more, you need to buy more of the input.
Well, if you need to buy more of the input, and the input has an upward sloping supply curve, then you'll have to pay more to get more of that input.
So to see that, let's run through an example.
Let's imagine that you want to produce something in the long run, and you need more labor to produce it.
So as you produce more of it, you need more labor.
So now let's go to Figure 11-8.
You're initially, in your firm, demanding L1 units of labor at a the wage of W1.
And let's say that at that point, at that wage, you're cost minimizing.
That's the cost minimizing point, and you've got this flat supply curve.
You're at that point.
Now let's say you want to produce more.
Well, to produce more, you've got to go to the market and hire more labor.
If the supply curve for labor is upward sloping, if labor is not a perfectly competitive market and, therefore, is an upward sloping supply, they'll say, fine.
If you want more workers, you've got to pay more.
You've got to pay W2.
You've got to pay more for your workers.
Well, think about what that does.
Now, go to Figure 11-9.
What that means is that as I produce more units, I have to pay more for the labor.
Now let's start on the left-hand side figure.
That says that if I'm producing little q1 as a firm, my marginal cost is MC super 1, and my average cost is AC super 1.
So I'm at P1.
Now if I want to produce more, if I want to produce q2, my average cost is going to be higher, and my marginal cost is going to be higher, because I have to pay a higher wage to workers.
So that's going to shift me up to have to charge a higher price.
Now, I'm still cost minimizing.
Given the wage the market gives me, I'm still cost minimizing.
There's nothing non-cost minimizing about this.
I'm still cost minimizing.
But to cost minimize, I have to charge more.
Because the market's charging me a higher price.
If you go back to solving our initial production decision, you'll see that because this is a higher wage, that's going to shift my cost function up.
A high wage is going to make my costs higher.
That's going to make my cost minimizing price be higher.
So I'm going to shift.
I'm going to need to charge a higher price.
That, itself, will also yield an upward sloping supply curve.
So an upward sloping supply curve comes from the fact that as I produce more, I've got to pay higher prices for my inputs.
That means I've got to charge higher prices for my outputs.
So these are three examples of reasons why, in reality, we don't see a perfectly flat long run supply curve.
So once again, to review, because this is the end of this particular topic, to review where we are, the way it works is firms are given a cost function.
Well, they choose a technology.
That gives them a cost function.
They enter the market.
In the short run, they're stuck with that technology.
So they decide to produce where price equals marginal cost as long as they're not losing more money than they've paid in fixed costs.
In the long run, firms come in and out until the point where every firm is producing efficiently.
As long as entry is free, as long as there are not barriers to entry, every firm is producing efficiently.
Then every firm is producing at a single point, which is where marginal cost equals average cost.
That yields a flat long run supply curve at the technological minimum.
In reality, supply curves slope up because there might be barriers to entry which leads to non-cost minimization.
So this leads to non-cost minimization.
And then there's two reasons, even if you're cost minimizing, you still could have capacity constraints, which is like our cotton example, or you could have upward sloping input supply.
So that's why, in reality, we draw the upward sloping supply curves that we started with at the beginning of this course even with a perfectly competitive market.
So let's stop there.
That's a lot of stuff to digest.
We're going to come back next time and talk about why all this is crap, and firms don't really cost minimize or maximize profits or any of that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
Today we're going to continue discussing firm behavior.
But today we're going to deviate from our fantasy land of 14.01 to come to the real world a little bit which is to talk a bit about whether firms actually maximize profits.
The whole presumption of producer theory is just as we think with consumer theory, consumers maximize utility.
Except producers maximize profits.
That turns out to be an incredibly useful shorthand for lots of things.
And it will turn out to be, as with any simplifying assumption, largely right and largely will help us draw a lot of interesting conclusions about firms and markets.
Nonetheless, it's clearly not right in reality for every firm.
For example, we see lots of firms doing things which look pretty wasteful like big corporate jets or other things like that.
More relevantly, huge pay for CEOs seems pretty wasteful.
So, for example, on 2004 the net income of Eli Lilly, the drug manufacturer, fell by 29%.
Yet the CEO got a 41% raise to to $12.5 million a year.
Or even more so in the financial crisis, in 2009, Citigroup and Merrill each lost more than $27 billion.
It's a pretty bad year for the financial sector, you might think.
And yet their CEO's pay in each company rose by more than 10% to about $15 million.
So in the worst year for the financial system since 1929, the CEO pay increased in these two companies.
So you might think that this sounds kind of off.
How could we explain that?
Well, there's two possible explanations.
So one explanation is that things which look wasteful really aren't.
That is that we pay CEOs a huge amount of money, but they're worth it.
So, for example, even at his ultimate salary of $33 million per year, Michael Jordan was vastly underpaid.
By any reasonable economic analysis of the value he added to the Chicago Bulls, it was well in excess of $50 million.
So even though he was paid $33 million, Michael Jordan was underpaid.
Similarly, just because someone gets paid a lot of money, doesn't mean that they're overpaid.
These CEOs are very talented people.
They may be underpaid.
Now this is an argument you might have made with a somewhat straight face maybe 10 or 15 years ago.
It's hard to make that with a straight face now.
It's hard to believe that these CEOs of these companies aren't overpaid given their company performance and even the kind of facts we just talked about.
So a better explanation for things like excessive CEO pay is something which we call the agency problem.
And that's what I want to emphasize in today's lecture.
In reality, we think of a firm as something where there's a guy who's an owner.
He hires some machines and some workers.
They make stuff that he thought of, and you produce it, and some profit is made.
So the model we have in mind when we talked about the firms so far in this course is the model of what we might call a sole proprietorship or a partnership.
It's a model where one individual, where there's basically a commonality.
The key thing that we've discussed so far is a commonality of interest between the owners who own the company and the control of production.
So the people who earn the money from the company also control the production.
So when you start your start-ups when you graduate, you will start them as a sole proprietorship or a partnership where you own the company, and you control what's done.
You're there 18 hours a day making sure the guys are doing what they're supposed to do.
And you're who gets the money if they succeed or lose the money if they fail.
And that's the model we've had in mind as we've thought about corporations or firms so far in our class.
But, in fact, most production in the US doesn't happen that way.
Most production in the US happens through the guise of corporations.
Corporations are marked by a separation of ownership and control.
What defines a corporation is a separation of ownership and control.
The owners of corporations are the stockholders or what we call the equity holders, people who have invested money in the corporation.
So the owners of the corporation are its investors, its stockholders, its equity holders.
All of you, most of you, probably somewhere-- if you're Jewish, it's in some Bar Mitzvah gift you got, if you're not, it's in some other gift you got-- probably have some equity ownership in something somewhere.
OK. You maybe have a little bond which says you own five shares of GM that someone gave you, because they thought it'd be cute or whatever.
That's now worthless, but maybe some other company.
We're equity holders.
We invest in companies.
But we don't make the production decision.
The production decision is made by the firm's managers who are employees of the owners, the equity holders.
Now these managers may have some ownership.
And we'll talk about that in a little bit.
But, by and large, the primary ownership of the firm is done by people who do not control the everyday decisions of production.
And it sort of makes sense it has to be that way.
When a company's as big as Microsoft, you couldn't have me.
Somewhere, I'm sure if you add up my various stock funds I own, I own 0.000000000001% of Microsoft.
It'd be crazy to have me involved with a 0.000000000001% of the decision making.
I don't know anything about anything that Microsoft does.
It makes sense that there's a separation of ownership and control once companies get big enough.
But the fact that there is a separation of ownership and control can lead to this agency problem and can lead to firms not maximizing profits.
So, basically, the point is that managers care not just about the profits they make but also how happy their life is.
And things which make their life happy may be things which aren't profit maximizing.
The managers who are in charge of production may have their own agenda that deviates from profit maximization.
And the problem is it may be hard to figure that out if you're the owner of the firm.
So the agency problem is the first example of a problem we'll talk about later in the semester of imperfect information.
This is the first time we'll see imperfect information.
We've assumed in our class so far, and we'll assume in our class going forward that there's perfect information the world, that everyone knows what stuff costs, that everyone can shop costlessly across providers, et cetera.
But, in fact, the world is marked by imperfect information.
As an owner of Microsoft, as a trivial owner of Microsoft, I don't know for sure that every Microsoft employee is doing what's profit maximizing for the company.
I can't.
It's just impossible for me to own that.
So that creates an agency problem.
And the agency problem arises because it's in the employees' interest, perhaps, to do things which aren't profit maximizing.
So so as a simple example, imagine that you're an investor in the company that's a medium-sized company.
And the head of that company is jealous because all of his friends at bigger companies have their own jets, and he's got to fly commercial.
He says, I want my own jet, because I'm not cool enough.
There's this great episode of the Simpsons where Burns goes to the billionaires club.
And he's in the billionaires club.
And then eventually his wealth drops so low he gets thrown into the millionaires club, and it's these yokels in the millionaires club.
He's like, I want to be back in the billionaires club.
The point is managers of companies care about how they look and who they hang out with.
And you've got this manager of a medium-sized company who wants his own private jet.
But he knows that that's not profit maximizing.
Because he knows that if he figures out his travel, and prices it out at competitive prices, it'll be cheaper to just travel commercial.
So what does he do?
He puts together a glossy PowerPoint which he presents to you, the owner of the company, showing that financially it would be more prudent to own a jet than to fly commercial.
What he doesn't point out and put into that PowerPoint is that in putting together the commercial prices he used, he chose the highest rates from flying from point A to point B, not the lowest rates.
Now, unless you're an expert on the cost of flying from point A to point B, you won't know that.
You'll say, well, that's a pretty impressive presentation.
Yes, you can show you've saved 20% by having your own plane.
Go get it.
But it turned out he didn't save 20%.
He just cooked the numbers in a way to make his life nicer even if it wasn't profit maximizing.
And that's an example of the kind of problem that arises with imperfect information.
And I could go through examples all the time.
But I think you guys probably all understand what I'm talking about, which is there's lots of ways employees of companies might make their lives nicer that aren't profit maximizing.
Now, corporations have recognized this from time immemorial.
And they've recognized that their owners are too diffuse to monitor this.
So what corporations do is they have set up an intermediary between the owners and the producers called the board of directors.
The board of directors is a set of individuals appointed by the owners that are supposed to actually pay attention to what's going on in production.
The owners say, look, we can't afford to pay attention.
But we're going to appoint you, the board of directors, and we're going to pay you quite a lot of money.
A typical director on the Fortune 500 company will make $100,000 plus a year.
It's a part-time job.
We appointed you to keep an eye on the tools of production.
The problem is the board of directors aren't that good at it either.
Because it turns out to be really hard to figure out exactly what's going on.
And this is made worse, because the members of the board of directors are often chosen by management of the company, by the people running the production, not by the owners.
So they'll make selections of who's going to be on the board of directors.
And they'll pack it with their friends who will be nice to them.
So, for example, Richard Grasso was the CEO of the New York Stock Exchange.
He started out in the mail room, worked his way up.
He became CEO of the New York Stock Exchange and retired with $187 million severance package which was outrageous and totally unearned.
But the board of directors were a bunch of his buddies who'd known him since he was a kid.
And they said, he seems like a nice guy.
Let's give him a bunch of money.
And a lot of the board of directors, when actually grilled on it, didn't actually know he'd gotten that much.
They were like, wow.
We didn't realize his retirement package was that big.
So it turns out having the board of directors does not solve the agency problem.
Because there's still too much of an information imperfection between the managers and the board of directors, not to mention the managers and the owners.
So what do we do?
What do we do with this agency problem?
Well, this was known for a lot of years.
And about 25 years ago, the thinking was, look, the way to solve this problem is we need to align the incentives of the managers and the owners.
If we're going to solve this problem, we need to get the managers, the producers, to actually care about maximizing profits enough that they'll do that even if it makes their life a little uncomfortable.
And how do we do that?
We turn the managers into owners.
We give the managers, the producers, an ownership stake in the company.
We say look, your income is going to be directly derived from the profitability of this company.
So now you'll have a direct incentive to do what's profit maximizing even if it makes your life a little uncomfortable.
Even if it means flying commercial, you recognize if I have my own jet, then I'm going to suffer.
So you align the incentives.
So we want to give them an ownership stake in the company.
Well, how do you do that?
How do you give managers an ownership stake?
Well, there's two ways you can do it.
One way is you can directly give them stock.
So you can directly say, look, instead of paying you in cash, we are going to pay you in company stock.
So your fortunes will directly be tied to the performance of this company if you pay them in stock.
Or, alternatively, you can do something which turns out to be cheaper.
You can give them stock options.
It turns out to be cheaper, at least, in principle.
We can give them stock options.
So let's talk about what a stock option is.
And some of you may deal with these along the way.
A stock is a piece of paper which says you only 0.00001% of the company.
A stock option is simply a piece of paper which says we are granting you the right to buy the stock at some price.
Typically, it's today's price.
We're not giving you a share of the company.
You have a piece of paper which says if the value of stock today is $100, you have a piece of paper which says no matter where the stock ends up, you can always buy it for $100.
Why is that valuable?
Well, that's valuable because let's say you give a CEO of a company a million options at $100 each when the price of the stock is $100.
Well, if the price of the stock rises to $150, that CEO can then take his million options, say, I want to buy the shares for 100, which I'm allowed to by these pieces of paper, and I'll turn around and sell them for $150.
I've just made $50 million.
So a stock option is valuable if the stock goes up.
But it's worthless if the stock goes down.
So it's kind of the ultimate incentive, if you think about it.
Because what it says to the CEO is, look, we're going to incent you to do better.
And you make a lot of money by doing better.
But if you do worse, you're not going to get anything.
And as myself as the owner it seems great, because it's cheaper.
Because if the stock goes up, I share some with the guy.
But I don't care.
I'm happy.
The stock goes up.
If it goes down, I don't have to give him anything.
And, in fact, giving someone a stock option costs, effectively, about half as much as giving them stock.
So if I say today's stock price, if I instead of giving you a share of stock, I give you an option to buy a share of stock at today's price, the value to that, what it costs me in the market is about half of what it costs when I actually give you the share of stock.
So it seems like the best deal.
You get all the incentives for them to raise the price, but it costs you half as much.
And, in fact, there's been tremendous growth in use of stock and stock options.
So if you look at Figure 12-1, now, this sort of changes.
Before 1992, there's only two categories.
And after '92, there's three.
So before '92, we just have to divide into salary and bonus versus options.
So you see there was essentially no options in '84.
This is the mean CEO pay.
So the typical CEO in 1984 made $500,000, and it was all cash, all salary and bonus.
By 1992, they made about $1 million, and a decent share of it started to be stock options.
Now, starting after '92, we actually break into three categories which is salary, bonus, and options.
But that doesn't really matter.
The main thing is to pay attention to the option share, which you see grew astronomically.
And so by its peak, in about 2002, first of all, the typical CEO was making $3.5 million.
Second of all, the majority of that was in stock options.
In fact, if you look at salary and bonus, it didn't grow that much over time.
If you look at 1992 versus 2002, over that decade, salary and bonus barely grew, but options went through the roof.
So there's a huge increase in use of these.
And the motivation is pretty basic economic intuition.
What we're going to do is this is a cheap way to align the incentives of owners and managers.
And that was why they took off.
And everyone thought it was great.
And people like Michael Jensen at Harvard Business School made fortunes off having suggested this, et cetera.
It was like we solved the problem.
And, remember, we did pretty damn well in the 1990s.
And a lot of it, people attributed it to exactly what's in this graph.
That by finally figuring out how to align the incentives of producers and owners, we'd figured out a way to change the economy.
And people thought we sort of entered this new economic era of massive productivity.
Well people, as we know now, were wrong.
Things slowed down in the 2000s and then now have completely crashed.
And why were they wrong?
Well, they're wrong because what makes economics fun is unintended consequences.
And these stock options had two kinds of unintended consequences.
The first unintended consequence is that they lead to excessive gambling.
And what I mean by that is that once your stock option is out of the money, once, if you've got it for 100, you're below 100.
You don't care how low it goes.
If you're at 99 or 20, you don't care.
All we care about is being 100 verses above.
What that means is as an owner, as a manager, you'll take excessive risks to go into the positive territory.
Even if there's a huge risk you'll get nailed on the downside.
Contrast to guys working for a company that's currently worth $100.
One is given stock, and one is given stock options at $100.
Imagine they're both going to leave the company in one year.
Because there's dynamics with the long run issues.
Let's put that aside.
They're both going to leave in one year.
One today is given a share of stock, and one today is given an option to buy at $100.
And then they're given the choice of a risky investment.
Someone comes to them and says, have I got a deal for you.
I've got an investment that has a 10% chance it will double the value of your company and a 90% chance the value of your company will fall by 20%.
So there's a 10% chance of a 100% return and a 90% chance of a 20% decline in your company's value.
Now, what is the profit maximizing thing to do in this situation?
Well, to do that, we have to consider the concept of what delivers the company the highest expected value.
Expected value is a concept we'll use-- and we'll come back to this later when we talk about uncertainty-- it's a concept we use to measure how you value things when there's uncertainty.
So the expected value of a gamble is the probability that you win times the value if you win plus the probability that you lose times the value if you lose.
That's the expected value of a gamble.
And the profit maximizing thing to do would be to take gambles of expected values of greater than zero.
The expected value is the summation of the value of that gamble.
This gamble has an expected value of 0.1 times 100-- 100% return and a 10% chance-- plus 0.9 times negative 20-- a 90% of losing 20-- or an expected value of minus 8%.
That is if you take this gamble, and you took it an infinite number of times, you would end up losing 8% on average.
Any one time might work out.
But by the law of large numbers, if we took this enough times, you'd lose 8% on average.
So you will lower the value of your company.
You will reduce profits by taking this gamble.
Your company is worse off if you take this gamble.
Well, what would the person who got the $100 in stocks do?
They won't take the gamble.
Because their $100 in stock will be worth, on average, 8% less if they take this gamble.
So I'm not taking the gamble.
This is stupid.
But now let's look at the guy who got the option.
His gamble is a little bit different.
Because he doesn't care whether the value of the company is 99 or 80.
He just cares about how much it goes above 100.
So what is his calculation?
Well, his calculation is he has a 10% chance of making 100% and a 90% chance of what?
Zero, ending up with zero.
He can't lose.
He has a 90% chance of zero.
So what is his calculation?
Well, he uses the positive 10% gamble.
He makes money on this gamble, because head he wins, tails he walks away.
The guy who owns the stock loses if it's tails with this weighted coin that only hits heads 10% of the time.
This guy only wins.
So he says, sure.
Any gamble which has a huge upside, I'm going to take.
Because I get all the upside, and I don't bear the downside.
So what this does is it leads to excessive gambling, excessive risk taking.
And that's exactly what we saw in economy, was companies gambling on things which sounded very good, managers gambling on things which sounded very good, some of which worked out very well and some of which didn't.
And that's the first problem we have with the use of stock options.
And questions about that?
Yeah
What about using a package of options like a number of stock options [INAUDIBLE PHRASE] package of stock options [INAUDIBLE PHRASE]
Certainly there are more sophisticated ways you can do that.
I mean, there's certainly more sophisticated ways you can do that.
But that's a great segue to the second problem with these.
The second problem is who decided the structure of the stock options?
The managers.
So the managers had executive compensation committees that designed the stock options the managers got.
So the second problem was there was just outright cheating.
So what you just described would be a great structure which would incentivize them maximally, but it would not make them the most money.
So the second problem we had was outright cheating.
So there's a wonderful investigation by the Wall Street Journal that investigated a whole list of stock option arrangements.
And what they found was companies would frequently do what's called backdating stock options.
What they'd say is, look, the value of the company just went up from $100 to $15 last week.
We're going to give you an option at $100.
And let's pretend it was issued before the company value went up.
We've just given you money.
You haven't done anything to earn that.
We've just given you money.
And the Wall Street Journal found incredible evidence of backdating stock options.
So they had one CEO.
They Yeah, I'm sorry
Is the company paying the money?
I thought it was like the market who gives it
The owners are paying the money, the owners
People who are buying the stock are exercising the option to
No.
But here's the thing.
When I gave it to you, then, basically, I gave you something of value.
I have to pay for that thing of value.
Now, ultimately, the transaction happens in the market.
But, basically, in other words, I give you this option to buy the stock at $100.
This is an important point that I wasn't clear enough on.
I give you an option to buy a stock at $100.
The stock is now worth $150.
That means that stock I could have had at $150 as an owner.
And it's worth $150.
But by letting you take it at $100, I let you keep the extra $50 instead of me getting it.
I have a share of stock which [INAUDIBLE] $150.
I've given that to you.
I've, essentially, transferred to you $150
So when companies [INAUDIBLE PHRASE]
They'll keep some amount of shares.
The owners, they will sell stock and they'll do offerings.
But the owners own the majority of the stock.
That's what makes them the owners.
Maybe that's the right way to put it
[INAUDIBLE PHRASE]
No.
Well, in principle, the board of directors approves stock.
They don't decide.
An executive management committee decides.
The executive management committee consists of all the golf buddies of the CEO.
They decide.
The board of directors is supposed to make sure nothing untoward happens.
The board of directors, they're just a bunch of retired former executives who are also golf buddies of the manager.
So they don't really get it right.
So there's outright cheating.
So, for instance, one CEO, he got six stock options.
Each one happened to be issued, happened to be issued the day before-- at least on paper-- was issued the day before a huge increase in the stock price.
Now, the CEO said he was just lucky.
But the Wall Street Journal did a simulation and calculated that there's a 1 in 1 billion chance he could've been that lucky.
That, in fact, what clearly happened was he got the stock options later, and they were backdated to make it seem like he got them before the price went up.
But that's not the best example.
The best example is Cablevision.
They granted and backdated options to their vice chairman from 1997 to 2002.
So from 1997 to 2002, they gave them options and they kept backdating them to make them look good.
The only problem was that he died in 1999.
So, basically, they were just giving money away to this guy's heirs.
Clearly they weren't incentivizing his performance.
As the quote said, trying to incentivize a corpse suggests they were not complying with the spirit of shareholder approved stock option plans.
They were just giving money to his heirs, because they felt bad for him, because they weren't keeping their yacht up or whatever.
So, basically, this is an example of a kind of cheating.
Of course, then you can go further, and there's outright fraud.
The backdating is sort of cheating.
You can go even further.
I don't know what determines fraud versus cheating.
You can just do what Enron did.
What Enron did is the executives there just lied about how much money they were making.
They just went public with figures that were wrong.
The stock price went up.
They cashed in their options and quit.
And as long as you don't get caught, that's a great deal.
And if you do get caught, well, they fine you for 1/3 of what you made.
So who cares?
So basically it was outright fraud.
And this was a lot of what happened.
A lot of the wealth that we thought was created in the 1990s in the stock market was really just false wealth created by people saying their company's were worth more than they actually were.
Yeah
Who decides the salary [INAUDIBLE PHRASE]
No.
I mean, once again, it varies by company to company.
But, basically, the dividend payout would be something that a different set of managers would decide, once again, subject to approval of the owners.
But, once again, the point is that the owners of a company, of a big company, are just a diffuse set of people who don't know what they're doing.
The managers know exactly what they're doing.
The board of directors is in between and is half clueless, half knows what it's doing.
And that's what leads to the agency problem.
What's so interesting about this example is it sort of points out, like in many things in economics, why we're called the dismal science.
There is no right answer here.
On the one hand, if you don't incentivize your owners, then they might not do what's profit maximizing.
On the other hand, if they incentivize your owners, they might do cheating to maximize their profits as opposed to the company's profits.
And that's why you need complicated structures, the type that was suggested in the back.
And that's why, for example, they find that companies with much more concentrated ownership, the ones that Warren Buffett owns 20% of, they do a much better job in their CEO compensation than the ones where it's just a bunch of people owning a tiny, tiny percent.
So that's exactly the kind of difficulty you face in setting up executive compensation.
Now, this was a real departure from the kind of things we talked about, just one example.
Partly what we want to do in this course, I want to teach you about basic economics.
I partly want to excite you about what else you can go on and do in economics.
What I've just done in the last half hour is a microcosm of the field of corporate finance.
There's an entire field in economics called corporate finance.
We teach it in course 15.
And it's a terrific course.
15.401 and 15.402 are the introductory courses.
And if you find this stuff exciting, as I'm sensing some of you are, that's a great place to take what we're learning in this course and go on and actually dive in.
Realize that half the stuff I've told you, I'm talking out of my ass.
I don't really know for sure.
But those guys do in course 15.
And you can really get into these agency relationships and understand our corporations work and how you set up these kinds of compensation arrangements.
So this is kind of just a very exciting area.
Now, we're going to go back after this and assume corporations maximize profits again.
And, once again, as with any assumption we make, by and large, it's right.
And by and large it will deliver the kind of important knowledge that we need to know about how firms function.
But it's important to remember that these kinds of things are behind it.
What we do in this course is teach you the basics.
Then you go on in these other electives and actually learn about this kind of more interesting tweaks and what we do here.
And other questions about this before I move on?
OK. Now we're going to stop with that topic, and we're going to move to sort of an in between topic.
That is, we've been talking about firms and firm profit maximization.
And we talked about how that depends on the structure of the market and perfect competition.
Starting in two lectures, we're going to start talking about alternative market structures like monopoly and oligopoly.
But in order to talk about them, we need to introduce a new concept we haven't used before.
And that's what we'll do the rest of this lecture and next lecture.
We need to move from positive economics to normative economics, from positive economics to normative.
Positive analysis is the study of the way things are.
So positive analysis is explaining why do firms produce this many widgets?
Why do they hire this many workers?
Why did I buy this many CDs?
That's a positive analysis, trying to explain the way things are.
That's what we've done, by and large, so far.
Normative analysis is the analysis of the way things should be.
Is it good that I bought that many CDs?
Is it good that the company made that much profits?
Would be better if we have a minimum wage or don't have a minimum wage?
Normative analysis is the analysis of the way things should be as opposed to the way they are.
So, for example, we showed in the last lecture that in the long run, with free entry and exit, all firms are driven to zero profit, and the supply curve is horizontal.
Well, is that a good thing?
We don't know.
We just said it happened.
I described why it happened.
But I didn't tell you if it was a good thing or a bad thing.
Zero profits, I make a lot of money off the profits that the companies I own make.
I don't know if I want to work with zero profits.
I've got a lot of stock and stuff.
And I'm making money because these companies make profits.
I don't think I'd be that happy to work with zero profits.
How do we think about whether that's a good world or a bad world?
How do we think about whether we like these outcomes or don't like them?
These are the tools of normative economics or what we will refer to as welfare economics.
Now, let me be clear here.
The term welfare, as you might have heard it, often refers to money you give to low income populations.
And we'll talk about that kind of welfare later in the semester.
Here, when I say welfare, I mean well-being.
Welfare is the term economists use as the measure of well-being.
We need to start talking about welfare economics, measuring the impact of economic outcomes on well-being, not just studying what happens when firms maximize profits or consumers maximize utility, but measuring how we feel about it, measuring whether it's good or bad, and measuring whether some outcomes might be preferred to others.
Because if we don't do that, we can't ever talk about the role of the government in the economy.
Until you start talking about welfare, you can't talk about whether we should have government interventions or not.
You can only talk about what happens, not whether it should happen.
Now, the problem of welfare economics is that it's a lot harder because of something I mentioned when we talked about utility.
Utility is an ordinal concept, not a cardinal concept.
Utility is an ordinal concept, not a cardinal concept.
That is, utils are meaningless.
We use utility functions to decide whether you prefer package A or package B.
But the actual amount of utils you get, unlike profits, is sort of a meaningless concept.
It's just sort of an index.
So what we do to do welfare economics is we turn from utils to dollars.
We measure welfare in dollars which is easy on the corporate side.
That's profits.
How do we do it on the consumer side?
We do it through the concept that we call compensating variation, which is one of these things where economists use fancy terms for something which isn't that hard to understand.
But you need to know the fancy terms, because that's kind of how we teach, compensating variation.
That is instead of asking, how sad are you about outcome x, we ask, how many dollars would it take?
Instead of asking, how sad would you be to not have a CD, we ask, how much would you pay to avoid being in that situation?
So instead of asking, how sad are you that you end up with this car instead of that car, we don't want to measure the utils of driving a Hyundai versus a Lexus.
We can't measure those.
What we can ask, is how much more will you pay to get the Lexus than the Hyundai?
And that's $1 measure of your utils.
That's a compensating variation.
How much do you have to be compensated?
So, in other words, instead of asking you how sad are you that you couldn't get tickets to a Lady Gaga concert-- maybe that's not a good example because no one's sad about that.
But let's say you were.
Let's say I am because my daughter wants to see her.
How sad are you?
I ask, instead, how much would you pay to get the tickets to the Lady Gaga concert?
That's a measure of how sad you were you couldn't get them.
Because you'll pay an amount to compensate yourself for the sadness of not getting them.
So it's a way of turning utils into dollars.
And that leads to the concept that we'll refer to throughout the semester.
That leads to the concept of consumer surplus.
Consumer surplus measures the benefit that a consumer gets from consuming a good above and beyond what they paid for the good.
So it's the benefit of consumption beyond the price.
So your consumer surplus-- surplus means extra-- is how much you enjoy consuming something above and beyond what you had to pay for it.
In other words, the idea is if you have to pay exactly as much as you'd enjoy the good, there's no consumer surplus.
So let's do an example.
So consider my demand to buy a Lady Gaga CD.
I've got to update my examples.
You guys don't buy CDs anymore.
How many of you have bought a CD in the last year?
Wow.
OK, let me ask another question, just one for the record.
Just because I have to decide how hard a time to give my 16-year-old son.
How many of you typically pay when you buy a song?
Wow.
OK. Let me ask another one.
How many of you typically don't pay when you buy a song.
Well, when you get a song, when you download a song, you know what I mean, when you download a song.
So the majority of you don't pay when you download a song.
Interesting.
OK.
I'll update this next year.
But, for now, we're still 10 years ago.
Consider my demand for Lady Gaga CDs.
I'm trying to decide whether to buy the marginal CD.
And let's say that it's worth $15 to me.
I'd be willing to pay $15 for the Lady Gaga CD.
And let's say the price is $15.
In that case, there's no surplus to me.
There's nothing extra.
I've paid what I was willing to pay.
So that's a case of a zero consumer surplus.
The consumer surplus is how much extra I got from consuming it.
I've got no extra.
It was worth $15.
I paid $15, no surplus.
Now let's say that the price was $10.
Then I've got a surplus.
I've got $5 in surplus.
I was willing to pay $15.
It makes me $15 happier to have this Lady Gaga CD.
But I only had to pay $10 for it.
So I've derived some surplus.
Well, how do we know what people's willingness to pay for a good is?
The demand curve.
That's the definition of the demand curve.
The demand curve measures your willingness to pay.
So let's go to the second figure.
Think about the demand curve for CDs.
There's some demand curve for CDs.
What this measures is the utility maximizing choice.
Every point in this demand curve represents my utility maximizing choice at that price.
This is backwards.
That should be a $15 on the y-axis and a $5 on the x-axis, Sorry, guys.
It should be $15 on the y-axis and a-- no.
Wait, hold on one second.
Time out.
At $5, I get no consumer surplus.
Yeah, this graph is kind of messed up.
Let me draw a new one here.
So you've got my demand curve for CDs.
You've got the quantity and the price.
And I've got some utility function that delivers this demand curve.
And let's say the way this works, this demand curve for CDs, let's say the way this works is that I am willing to buy 1 CD.
I'm willing to buy if the price is $20, I'm willing to buy 1 CD.
So at a price of $20, I'm willing to buy 1 CD.
So if I get that 1 CD at $20, then basically that's my willingness to pay.
It equals a price, so there's no consumer surplus.
Really imagine it's right next to that y-axis.
Now let's the price falls to $15.
Sorry.
That's a 1.
That's the 1.
That's a 20.
At a price of $20, I'm willing to buy 1.
At a price of $15, I'm willing to buy 5.
I'm willing to buy 5 CDs.
Now, what is my consumer surplus?
Well, let's ask.
For the first CD, I got no consumer surplus.
But for the second, I was willing to pay.
When the price was $20, I got no consumer surplus.
Now let's say the price is $15.
So what was I willing to pay for the first CD?
$20.
So what's my consumer surplus on the first CD?
5.
So on that first CD, I got a consumer surplus of 5.
The second CD, I was willing to pay less because I've got diminishing marginal utility.
But I was still willing to pay more than $15.
So I've got some surplus there.
Third CD here, et cetera.
But on the fifth CD, I derive no surplus, because I was willing to pay $15 for that fifth CD, and I paid $15 for that fifth CD.
What that means is that all of this is my consumer surplus.
Because on every unit, because I paid a fixed price of $15, and for my fifth CD, that delivered no surplus, that was indifferent.
But in my first through fourth CDs, that made me very happy.
Because I was willing to pay more than $15 for those.
So my consumer surplus is the amount by which I derive utility above what I was willing to pay, above the price-- I'm sorry-- the amount that I was willing to pay about the price.
The amount I'm willing to pay is every point on this curve.
So anything I buy that I was willing to pay more for the price for, I derived surplus on.
So this becomes my consumer surplus.
So, basically, consumer surplus is the area under the demand curve above the price.
So, graphically, consumer surplus is going to be the area under the demand curve above the price.
Intuitively, why is that?
Intuitively, what's going on is that every unit above the price under the demand curve are units which I valued higher than the price.
So I get surplus on them.
OK.
Question?
Yeah
If you're [UNINTELLIGIBLE] this, would you start at zero?
Yeah.
Really, in some sense, you'd want to start at zero.
It's really the integral of this whole area.
The problem is it's discrete.
I should have a stepwise curve here.
I made it discrete.
But, really, it's the integral of this area.
Yeah
[INAUDIBLE PHRASE].
upward sloping demand curve?
I'm sorry
With an upward sloping demand curve
With an upward sloping demand curve, well, as I said, we don't believe in Giffen goods.
We think they are a fantasy.
So with an upward sloping demand curve, how would that work?
So an upward sloping demand curve, then that would say that if the price is here, then you'd have an infinite consumer surplus.
It wouldn't be well-defined.
It wouldn't be well-defined.
So I don't know how you'd do that.
That's why we don't like upward sloping demand curves.
Yeah.
Question?
No?
OK.
So, basically, what you end up with is this consumer surplus concept which, basically, as pointed out mathematically, is the integral of this area.
But we're not going to make you integrate.
It's basically this triangle which is basically all the units above the price, all the units for which you're willing to pay more than the price.
And you get consumer surplus on every unit you buy up to the point where your willingness to pay equals the price.
And that's our normative measure of welfare.
That's our normative measure of how happy you are.
And the key that drives this is diminishing marginal utility.
That's why we don't like upward sloping demand curves.
What drives this is diminishing marginal utility.
It's that with each additional unit I'm willing to pay less and less for.
So I'm going to get less and less surplus off each additional unit I buy.
And that's going to be the key thing.
The key intuition that you guys have to remember is that the last unit that I'm willing to buy at that price, by definition, I get no surplus on, or I get a $0.001 of surplus on.
Because if I've got a lot of surplus, I'd buy 1 more.
So, by definition, the marginal unit is the one where surplus is driven to 0.
And units before that are the ones with positive surplus.
And that's why an important intuition to have is the amount of surplus you're going to get from a purchase is going to be about how far away from the price point you are.
Those first units are going to give you more surplus.
And that's going to dwindle as you get to the actual last unit you buy at that price.
Any questions about that?
OK. Let me stop there.
We will come back next time, and we're going to spend the whole lecture talking about welfare economics, we'll introduce producer surplus, and talk about how we then think about whether changes are good or bad.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We want to turn this course to now start talking about some normative economics, which is how do we feel about the actions firms and consumers take.
And to do that we need to pause, and starting last lecture now through this lecture, talk about welfare.
Well-being.
How do we measure the well-being implications of individual and firm actions?
Now, we talked last time about the first concept of this, which was consumer surplus.
Consumer surplus was the amount of utility consumers derived above and beyond the price they had to pay for the good.
Now, since it's utility, remember, it's not ordinal, it's cardinal.
We redefine that.
That is the consumer's willingness to pay above the price that they pay for a good.
So, anything a consumer's willing to pay that was higher than the price they actually paid is consumer surplus.
And that was at the individual level.
That was the individual consumer surplus.
Now what we want to do is move on and talk about market consumer surplus.
So for an individual, consumer surplus is a willingness to pay above the price actually paid.
At the market level, it's the same thing.
It's just the aggregation of individual consumer surpluses.
So let's think about a simple example.
Let's think about the case of the consumer surplus that you derive from my lectures.
Let's think about the consumer surplus you derive from my lectures.
So we have some demand curve for my lectures.
Let's look at figure 13-1.
Let's say this is the demand curve for my lectures.
So basically, this is a willingness to pay.
So the notion is that there's some student who's willing to pay a large amount because they recognize the brilliance of what they're hearing here.
And they're willing to pay a large amount for my lectures.
Then the next student's willing to pay slightly less.
All the way down to the student who falls asleep every time who's not willing to pay anything for my lectures.
So you've got some demand curve for my lectures.
And, once again, the key to all this, the key to the welfare analysis, is to remember what this demand curve represents.
It's a willingness to pay curve.
But in this case, it's not an individual's willingness to pay.
It's the market's willingness to pay.
Where each point is a unit of demand.
So in this case, one unit equals one person.
So each point represents the willingness to pay of that person.
So the point on the y-axis represents the willingness to pay of the person who most enjoys my lectures.
Or derives the most value out of them.
The intersection of that line with the x-axis is a willingness to pay of zero, someone who's not willing to pay anything for the lectures.
Obviously, could go negative.
People could be willing to pay not have to come to the lecture, but we won't consider that.
We'll just say zero's the minimum.
So, basically, what is the key determinant of the equilibrium outcome?
It is the point where, equilibrium is the point, where the price is set equal to the willingness to pay of the marginal consumer.
So if you think about typical supply-demand diagram, and we get some equilibrium price.
So here's our equilibrium quantity, Q-star.
And here's our program price, P-star.
Now, let's focus in on this demand curve.
With that price P-star, it represents-- it's on the demand curve, so it's a willingness to pay.
Whose willingness to pay is it?
It's the willingness to pay of person Q-star.
It's the willingness to pay of the person who's exactly willing to pay the price.
That is, the equilibrium in a perfectly competitive market, of the kind we've discussed so far, the price represents the willingness to pay of the marginal consumer.
The willingness to pay of the consumer who exactly is indifferent between consuming and not consuming the good.
So this consumer, Q-star, this person Q-star in our example, is the person who's indifferent.
Who derives, what, zero consumer surplus.
This person, since their willingness to pay equals the price, equilibrium is achieved where the price is set-- In equilibrium, you have that the marginal consumer, the last person consuming, is deriving zero consumer surplus.
They're indifferent between consuming the good and not consuming the good at that price.
If it was free, they'd be happy to consume it, but at the price, at that equilibrium price P-star, they're indifferent.
So basically, looking at the diagram, what this is saying is that the price was $100 a lecture.
Then person 100, that hundredth person, would be the marginal consumer who's indifferent between paying the $100 and hearing my lecture or paying nothing and skipping my lecture.
So that person 100 is the person who's indifferent between paying $100 and hearing my lecture or paying zero and not hearing my lecture.
That person drives no consumer surplus.
Because their willingness to pay equals the price.
What that means is that every consumer up to the first 100, by definition, must be deriving some consumer surplus.
As long as the demand curve is downward sloping.
As long as the demand curve is downward sloping, every person to the left of person 100 must be deriving some consumer surplus.
In particular, the first person is driving enormous consumer surplus.
How can we prove that?
Well, the proof is simple.
We know their willingness to pay is higher than person 100.
And we know person 100 is willing to pay $100.
Therefore, by definition anyone to the left is deriving consumer surplus from this.
Because we know they like it more than person 100, and person 100 is willing to pay $100.
They only have to pay $100, it's one price to everybody.
So by definition, they're deriving consumer surplus.
So the first guy, he or she gets a big consumer surplus.
Then it dwindles and dwindles and dwindles until the hundredth person derives zero.
But, if you integrate, and the entire consumer surplus for the market is that entire triangle, that entire shaded triangle.
So market consumer surplus is defined the same as individual consumer surplus, as the difference between the demand curve.
It's the area under the demand curve above the price.
So it's the integral of that area, under the demand curve, above the price.
But here we're thinking of it not in terms of a person's decision but the market's decision.
And when you think about that, it is the last unit be representing the person who's indifferent.
And everybody else deriving surplus from it.
Questions about that?
Now, let's ask what, then would happen if the price of my lecture rate rose.
So let's say I was charging $100 boxes at the door.
You guys are all the guys to the left of person 100.
Those people who were here for the first lecture and don't show up any more, they're the people to the right of person 100.
And now let's say I change.
Instead of charging $100 at the door, I charge $110 at the door.
So let's look at figure 13-2.
What happens there?
Well, in that case, if I charge $110, the hundredth guy is no longer willing to come to lecture.
Remember, the hundredth guy, he or she was indifferent between paying $100 and hearing my lecture and paying nothing and missing my lecture.
Well at $110, clearly, they'll say forget it, it's not worth it any more.
They'll drop out.
Likewise, the way I've drawn this diagram, everybody above person 90 will no longer attend.
That is, person 90 is now the person, where at a price of $110 they're indifferent between paying and attending or not paying and not attending.
Everybody to the left of person 90 still gets consumer surplus.
It's not shaded, but if you shade the triangle above the $110 dashed line, the triangle under the demand curve and above with a dashed line at $110, that's the new consumer surplus.
That's the new consumer surplus.
Those people still derive consumer surplus because they love me so much that they're willing to pay more than $110.
So even $110, and they prefer to pay $100.
But even $110 they'll still come and they'll still be happy about it.
But now 10 people have dropped out because they're not willing to pay $110.
They were only willing to pay $100.
So what we see is that the total consumer surplus has shrunk.
It's shrunk by this trapezoid, by this shaded trapezoid.
And it's shrunk for two reasons.
One reason is that some people who used to derive consumer surplus now are out of the market.
That's the triangle.
That's the shaded triangle.
It's also shrunk because even people staying in the market are now sadder.
They've lost consumer surplus.
The consumer surplus is still positive.
They're still coming to the lecture, but it's smaller than it was, and that's the rectangle.
So the triangle is the people who drop out of the market.
The rectangle is the reduced consumer surplus, the reduced consumer surplus of people who stay in the market but now have to pay $10 more.
The consumer surplus is still positive, but it's smaller.
So, a question to you.
What is the key economic concept that's going to determine, for a given market, whether the consumer surplus is large or small?
Yeah?
Elasticity
The elasticity.
And why is that?
Because it changes how steeply--
Exactly.
So if we look at figure 13-3, now here is a case with a steeper elasticity of demand, where consumers are more inelastic.
If you compare it to figure 13-1, you'll see the consumer surplus triangle is much bigger.
Likewise, if we drew a flatter demand curve, a more elastic demand, the consumer surplus would be much smaller.
So hopefully you can see graphically what will determine consumer surplus is the elasticity of demand.
Now, can someone give me the intuition for why that's true?
Graphically, I hope you can see it's true.
The triangle will be smaller as that curve is flatter.
Can someone give me the intuition for why that's true?
Why is it that consumer surplus is larger the more inelastic is demand, and consumer surplus is smaller the more elastic is demand.
Want to give it a try?
I guess if it's more inelastic, then consumers don't have--
They don't have what?
[UNINTELLIGIBLE] What determines it?
As many choices, I would say
They don't have as many substitutes.
So with inelastic demand is when consumers don't have as many substitutes.
In that case they get a lot of surplus from consuming this good.
So think about insulin versus McDonald's.
The consumer surplus from insulin is very, very high.
Because regardless of the price, I die without it.
And there's no substitute.
So it's a very inelastically demanded good, and as a result, at any price, I derive a huge consumer surplus because as long as that price is less than the value of my life, I derive a big consumer surplus.
Now let's take McDonald's.
I can always go to Burger King and be equally happy.
Maybe not quite equally happy, the curve isn't perfectly elastic.
I like McDonald's a little bit more.
I like the prizes in their Happy Meals better.
So it's a little bit elastic, but the point is, if McDonald's raises the price, I'm not that much sadder because I just go to Burger King where I was perfectly happy as well.
So there's not much lost consumer surplus if McDonald's raises the price.
There's not much consumer surplus arising.
There's not that much consumer surplus-- forget the loss.
Go back.
That's a different issue.
I'm talking the sides of the consumer surplus.
There's not much consumer surplus arising from consuming at McDonald's.
Not much consumer surplus arises from consuming McDonalds because I can always just go to Burger King instead.
So what determines consumer surplus is the elasticity of demand, which is fundamentally about your willingness to pay.
Inelastic goods have higher willingness to pay.
And so as a result, the consumer surplus is essentially inversely related to the elasticity of demand.
The higher the elasticity, the smaller the consumer surplus.
Now, let's shift and talk about producers.
The analysis for producer surplus is exactly the same type of analysis, just flipped to the other side of the market.
Now, here the question is, what determines my producer surplus?
Well, what determines consumer surplus?
It's the difference between my marginal willingness to pay and the price.
What determines producer surplus, that's going to be the difference between the firm's marginal willingness to supply, and the price.
So my marginal willingness to pay as a consumer is what determines my consumer surplus.
Producer surplus will be determined by my marginal willingness to supply.
My marginal is supply, which is simply represented by the supply curve.
A little bit easier for producer surplus.
The supply curve represents the price at which I'm willing to supply a good.
That's what a supply curve is.
As we learned a couple lectures ago, in the context of competitive firms that's my marginal cost.
So the supply curve is my marginal cost.
That says that is how much it costs me to produce the next unit.
In a perfectly competitive long-run equilibrium, or short run equilibrium, that is basically what I'll set my price to.
So if we go to figure 13-4, now let's look at my producer surplus.
My surplus from delivering lectures.
Let's say that-- this is a bit harder to imagine-- but imagine that I prefer deliver smaller lectures.
Maybe because as the number of kids gets smaller I can learn your names.
I don't have to look around as much, I don't have to pace as much, whatever.
So imagine that I'm most happy delivering a lecture to one student.
And that's the lowest cost to me, the lowest effort.
I can just come in and sit down, and we have coffee and I just riff, and I don't have to worry about notes or any of that stuff.
But when there's two students I feel a little bit guilty doing that, so I make some notes to myself.
When there's three students I make some more notes.
By the time there's all you students, I have to type up all these notes.
So every student that adds to my lecture, imagine, is a marginal cost.
Now, you know that's not true, of course.
I wouldn't change my notes if six more students walked in.
But let's just imagine that it's linear.
Imagine with every student that comes in here, I have to put in a little more effort in my lecture.
So it's a marginal cost to me with the additional students.
So that delivers the supply curve, this upward sloping supply curve.
Now let's say at a given price, P, I'm willing to lecture-- so that, given the supply curve, if the price is equal to p, I'm willing to lecture to Q students.
That is, if you're going to give me a price of P, at that price my marginal willingness to supply is that I'll supply Q lectures.
So another way to say it is that if you want me to supply Q lectures, you've got to pay me a price P. That's the point where I derive no producer surplus.
I'm indifferent at that point.
If at that point you said, would you want the Q-th student or not, I'd say I don't really care.
I'm indifferent.
I get zero producer surplus.
But, I got a huge surplus on that first student.
Because I was willing to work with them and it wouldn't have cost me anything.
But I'm getting paid P to work with all of you.
So I make a producer surplus on that.
So producer surplus is made on every unit to the left of Q, because those are units with a positive producer surplus.
They're the units above the supply curve and below the price.
They're the units where it's above the supply curve, so I'm willing to supply, but below the price which means that I'm getting paid more than I would have to to supply that unit.
Can anyone think of another name for this triangle, roughly?
How else you might think about that triangle.
Well, I can talk about producer surplus.
And I said, what's the difference between the price producers receive and the cost that they have to produce it
Profit
Profit.
Roughly speaking, producer surplus is profit.
Now, technically, in the short run, that's not true.
Because in the short run there's fixed costs and you might lose money on fixed costs and still make a long run profit.
So in the short run it's not technically true.
In the long run it is technically true, and for the purpose of this course we'll say it's technically true.
So producer surplus is profit.
It's a lot easier to think about.
Consumer surplus is this vague concept we have to measure my willingness to pay versus what I pay.
Producer surplus is easy, it's profit.
So producer surplus is profit, it's the difference between the price at which I'm willing to produce the good and the price you actually pay me for it.
So that's producer surplus.
Questions about that?
OK.
Putting this together, we can now measure the total welfare of society.
We now have it.
We can measure the entire happiness of all of society.
And we define social welfare of society as consumer surplus plus producer surplus.
Now, that doesn't have to be.
You could say, gee, Jon, don't you care more about consumers than firms?
Or gee, Jon, don't you care more about firms than consumers.
I'm going to leave that alone for now.
We're just going to do the simplest thing and just say, we're going to define social welfare as simply the sum of how much surplus consumers get plus how much surplus producers get.
OK?
This is a particular representation.
When we talk about efficiency and equilibrium, we're talking about efficiency-- I'm sorry-- versus equity, which we'll talk about towards the end of the course.
We'll talk about alternative definitions, and how we weigh different definitions of this.
But for now, this is the standard economic definition.
Which is, let's not draw a judgment about who's better than who.
Let's just talk about the total amount of surplus in society.
So the amount of social welfare, is the total amount of surplus in society.
And here's the key result.
That the competitive equilibrium, where demand equals supply, in competitive equilibrium, is the welfare maximizing outcome.
The competitive equilibrium of the market, which is where demand equals supply, is also the welfare maximizing outcome.
And that's the key thing we want to go to now, which is that basically moving away from that equilibrium point, where demand equals supply, will by definition lower the amount of social welfare.
So to see this, let's go to Figure 13-5.
And we've got some supply and demand curves.
This is from the book now.
We've got some supply and demand curves.
And we're initially in equilibrium at P1 Q1, at the point E1.
So we're initially at equilibrium at E1.
At E1, at that initial equilibrium point, consumer surplus is equal to R. Now, going by the letters that label the areas, R plus S plus V. That's the amount of consumer surplus.
The amount under the demand curve, above the price.
That's consumer surplus.
Stop me if this is not clear, this is important stuff.
So under the demand curve above the price, R plus S plus V. The producer surplus is T plus U.
The profit is the amount above the supply curve and below the price.
And so total social welfare is the sum of all these.
R plus S plus V plus T plus U.
That's our starting point.
That's our competitive equilibrium starting point.
Now, let's see.
You should be able to immediately see, those of you who are good with your geometry, that there is no point you could choose which can make social welfare larger.
As a simple comparative statics exercise, imagine I had the government come in and say, we're going to raise the prices in the market to P2.
We decide that producers aren't making enough.
Look, the consumers get three letters, the producers only get two letters, that's unfair.
So we're going to raise the price to give the produces more letters and the consumer fewer letters.
That sounds like about the rational basis for government policy making these days.
So we're going to do that.
And so we're going to raise the price to P2.
With a price of P2, we'll have a new equilibrium.
If you force the price up to P2.
You have a new equilibrium at little e sub 2, and a quantity of Q2.
What happens now?
What's happened to consumer surplus?
Consumer surplus, you've now lost S and V. Consumer surplus is now just R. Because at that new price, P2, that's the area under the demand curve above the price.
Producer surplus has grown, however.
Now, instead of being T plus U, it's now T plus S. So what's happened, effectively, is you've transferred S from consumers to producers.
And you've lost V plus U, forever.
So what's involved in this change is a transfer and a loss.
The transfer is the area S, which used to belong to consumers now belongs to producers.
But now we have what we call a dead weight loss.
Dead weight loss, of V plus U.
That's welfare that has disappeared.
Welfare has disappeared.
And the definition of dead weight loss is a net reduction in efficiency from trades that are not made.
Remember we talked about efficiency early on in the course.
We said the efficient outcome is one where trades that make both people better off are made.
Here we have trades, which absent the government would have made both parties better off, and they're not happening.
That is surplus that's just gone.
It's into the ether.
That is social surplus that is now social welfare, that is now gone because there are trades that would have made both parties happier that are now not happening.
And that's a total waste from society's perspective.
Because society's best off if all the trades that make both parties happier, happen.
So the bottom line is, any price you would impose other than the market price of P1, and you can work this out for yourself, any price you would impose would by definition lead to a lower social welfare.
It may shift.
It may lead to a bigger or smaller consumer surplus relative to producer surplus.
Once again at this point we're just using the sum of them as a measure of social welfare.
Social welfare has fallen.
So social welfare is maximized in this case.
So this gives us a framework to think about.
We started the course with supply and demand, and talking about how things like the minimum wage reduce efficiency.
Well, this gives us a welfare framework, a more formal welfare framework, for thinking about that.
I said it reduces efficiency before because you had less labor in the market.
But now we can actually more formally say why does the minimum wage reduce efficiency?
We can actually look at that.
Let me just do the always-risky thing of trying a freehand diagram.
You remember our market for labor, you had the amount of labor on the x-axis, the wage on the y-axis.
You had some supply of labor that comes from workers deciding to work as the wage goes up, they want to work more.
You've got demand for labor.
That comes from firms demanding workers.
As the wage goes up, they want fewer workers.
And you have some initial equilibrium, L-star, W-star.
When the government came in with its minimum wage and the government said, we're going to impose a minimum wage of W-bar, W-super bar.
OK Remember, we said what happened was, well, of course then firms are only going to want L1 workers.
We talked before about how that led to some unemployment.
What we didn't mention is how this leads to an efficiency loss to society.
This area is now dead weight loss.
These workers, who would have been happy to work at the prevailing wage, and firms would have been happy to hire the workers at the prevailing wage, and those trades no longer happen.
So here's an important question to help you think-- you're going to have to draw dead weight loss triangles in your sleep now.
So here's the trick with these.
Why is the dead weight loss triangle smallest here and grow like that?
Why does the dead weight loss triangle grow as you move away from the competitive equilibrium?
Yeah
There is less amount of people who are willing to work at that price
Say it again?
So there's less people that are willing to work for that smaller price
Well, here we're imposing a higher-- less people are willing to work for, are you referring to here or here?
To all the way to the right
All the way.
So basically, the point is-- another way to put it is, at that wage the consumers there are pretty indifferent.
So in other words, they're willing to work but barely.
So at this wage, at this point here, the L*-th worker is getting no surplus from working.
It's not a crappy wage, he's happy to take it.
But he'd also be happy to sit at home.
He's indifferent.
So for that L*-th worker, you make him not work, he doesn't care.
So if you raise the minimum wage and this guy sits at home instead of working, he didn't care.
He was getting no surplus from working anyway.
Likewise this firm, who's hiring the L*-th worker, they were paying this worker exactly what he was producing.
The marginal cost of that worker exactly equalled what they were paying him.
What that worker's producing was exactly equal to what they paid him.
They were earning zero profit on that worker.
So they don't care if he stays home.
So if the government set a minimum wage, such that one guy stayed home, the minimum wage was so close to market wage that one guy stayed home, there would be no social welfare loss.
Or infinitesimally small.
Because that last guy, there was indifference on both the worker's side and the firm's side.
However once you start displacing more and more, these workers aren't indifferent anymore, right?
These are workers who were making a lot of surplus at that wage.
And firms that were earning a lot of profit at that wage.
Now they're not indifferent.
So as you move farther and farther from the competitive equilibrium, the distortion gets larger and larger.
Very important intuition to have.
That for that last person, there's no distortion from moving epsilon away from the competitive equilibrium.
Because they weren't earning any surplus anyway.
And the firm wasn't making any surplus on them.
But as you move away, the loss in social welfare gets larger and larger because these are people who are making all sorts of surpluses on these transactions.
And you're stopping them from happening.
So if the government interferes with the transaction-- So imagine, this guy and I were negotiating over a baseball card and I ended up paying him exactly what the baseball card was worth to him.
And I ended up paying exactly what it was worth to me to have it.
And then my parents come in and say, you can't do that.
Then it doesn't really matter, because we weren't making any surplus of that trade anyway.
But, if he had three of these cards and was delighted to get rid of it for $50, and I have always wanted this card and valued it at $200, then if my parents come and sink this trade, then that's a real bummer.
That's a huge loss in social welfare, because the trade that made both parties better off is not happening.
Questions about that?
Let's look at a particularly good example of this.
Of when the government interferes with trades and the implications of that.
The TAs on Friday are going to go through a bunch of interesting examples.
I'm just going to do one today.
And then you'll do some more in section, because this is hard and important.
I want to today focus on the example of taxicab medallions.
It's the example in the book, which is a particularly good example.
Taxi drivers, we've all taken taxis, we know how they work.
But taxi drivers in virtually all cities, you cannot just start a cab and drive people around.
In virtually all cities, you need to get something from the city that allows you to call yourself a taxicab.
And that's typically called a taxicab medallion.
It originally was literally something you had on the hood of your car.
Now it's a certificate you have in the back of your car.
So you see the certificate whenever you ride in a car, which says, So-and-so is licensed by the city of whatever to drive this cab.
The government issues a certain amount of these taxicab medallions.
And almost always issues less than would be demanded in the free market for taxicabs.
And let's see what effect that has.
So go to 13-6.
This is getting kind of complicated, but this is an example of the kind of welfare analysis we can do.
Once we understand these concepts.
Let's say we start at point E1.
Big E1 on the right, little E1 on the left.
So on the right-hand side, this is like our other profit diagrams.
The right-hand side is the market.
The left-hand side is an individual cab firm.
Initially, if the government doesn't interfere, you are in equilibrium at point E sub 1.
Which is, that the price is P1, and the individual cab firm delivers q sub 1 rides.
And let's assume cab firms are identical, just to make it easy.
And that there are n of them.
Well in that case, the total amount delivered is N1 Q1.
So Q sub 1 is q sub 1, which is the amount delivered by a given cab firm at that price, times the n cab firms in the market.
N1 cab firms in the market.
So there's N1 firms.
Each delivers little q sub 1 of rides.
And you can see at that point, they each make some profit.
So what you can see at that point is that at E1, at a price of P1, they're making some profit.
The price is set equal to their marginal cost.
I'm sorry, they're not making a profit.
My bad.
You see at that point they're not making profit.
Because what you can see is price is equal to the minimum of average cost.
Remember, the no-profit condition is where marginal cost equals average cost.
You can see at that point, little e1 in the diagram at the left, there is no profit.
Because price equals marginal cost, equals the average cost.
So we're making no profit, and that's a perfectly competitive equilibrium.
That's what we derived last time.
Now, let's say the government comes in and says, you know what?
And the welfare here.
Just to do the welfare here.
You see that the welfare of society is, producers make no surplus.
There's no producer surplus, there's no profits.
Consumers have surplus of A plus B plus C. So consumer surplus is the area on the demand curve above the price.
That's A plus B plus C. Producer surplus is profits, which are zero.
Because it's perfectly competitive.
So you end up with a total social surplus of A plus B plus C. And it all goes to consumers.
Now let's say, the taxicab owners aren't so happy about this.
They don't like making zero surplus.
And they manage to get a restriction.
Such that the government says, there's only a certain number of medallions.
And we're only going to let people drive the cabs if they have a medallion.
Now, let's say that the taxicab owners say, we're only going to limit the number of medallions to N sub 2.
Instead of there being n sub 1 cab firms-- let's say medallions aren't for cabs, they're for cab companies.
Instead of being n sub 1 cab companies, we're going to limit medallions so there can only be n sub 2 cab companies.
So what happens?
Well, it's a little complicated so let's follow along.
If there's only n sub 2 cab companies, then what that means is that given the same market demand, that means firms are now going to be able to make some profit.
So up to, if you look at the right-hand side diagram up to N2 Q1, the supply curve is the same.
But at that point, once you pass N2 Q1, then you have a point where that's-- at the old, efficient level of cab rides, the efficient level is, each cab company provides Q sub 1 rides.
Well, you then run out of rides, but people still want more.
So what happens?
Well, cab companies start to be able to charge more.
The supply curve becomes upward sloping.
They start to be able to charge more for their rides.
Because now you don't have extra cab companies entering and competing those profits away.
They start to be able to charge more for their rides.
And you see that that upward sloping supply curve meets the demand curve at N2 Q2.
So up to N2 Q1 it's the supply curve which it was, which is flat.
Once you get beyond that, now firms can charge a higher price.
Because they don't have to worry about entry.
So you get a new supply curve that's flat until N2 Q1 then starts to slope up.
That's S super 2.
And S super 2 intersects the demand curve at point E sub 2.
And that is the new equilibrium, with the higher price of P2 and a lower total quantity of Q2.
Now let's what that does to the firm.
Well, a given firm at a price P2, they now-- they always set the price equal to marginal cost.
That's the rule.
Every profit maximizing firm does that.
Well, they now are at a point little e sub 2, making a huge profit.
Because at that point, their marginal cost is well above their average cost.
So at that point, they make the profit of the shaded area pi in the left-hand side diagram.
They make the profit pi, because once again [INAUDIBLE] go to the right-hand and then back to the left-hand side.
The right-hand side is, the new supply curve intersects the demand curve.
At point E2.
That's the price of P2.
Carry that over the left-hand diagram, you see at a price of P2, they make a profit, the difference between that price and the average cost curve, which is that shaded pi box.
So firms now make profits.
And they didn't before.
What's happened to welfare in the market?
Well, what's happened is consumer surplus has now fallen from A plus B to just A.
Because the price is now P2.
The consumer surplus is the area under the demand curve above the price.
That's just the triangle A.
Just the triangle A.
At that new price, that's the consumer surplus.
What's the producer surplus?
Well, now producers are making profits.
And what they're making a surplus is the area below the price above their supply curve.
The supply curve is this funky kinked thing I just described.
So their new producer surplus is this, I don't know what the name is for, like a trapezoid with a curved side.
Does that have a name?
It's still a trapezoid?
I don't know.
Anyway.
Curvazoid.
B.
Their new surplus is B.
So producers now make surplus B.
Consumer surplus is reduced to A.
And what's happened to C?
C is gone.
C is dead weight loss.
Consumer surplus is now A.
Producer surplus is now B.
The darkly shaded area C is now the dead weight loss.
And that's the dead weight loss, because once again, at the old marginal cost curve, at the old supply curve, these are trades which both the producer, which is the taxicab company, and the consumer, which is a person riding the taxi, were happy to make.
And they're not being made now.
So we've now lost an amount C, dead weight loss.
Sorry about the confusion.
Questions about that?
Now, here's my question for you.
You can go home and think about that, and hopefully it'll be clear in the examples on Friday.
But let's ask a slightly deeper question.
Does this mean that if you become a taxi driver today, you will derive producer surplus of B from driving a taxi?
Why not?
Because you have to pay that lump sum to get--
Because the point is, you have to get into the market, and to get into the market you need a medallion.
And the guys with the medallions isn't going to hand them over to you for free.
So let's say this guy's retiring as a cab driver.
He's got his medallion.
You come up him and say, I want to be a cab driver because I see this big producer surplus area B.
[INAUDIBLE] medallion.
What's this guy going to say?
He's going to say, well, wait a second.
If I give you my medallion, you're going to make that profits of pi.
That's the profit you make by being a taxi driver in this market.
So what should he do?
What should he do?
Sell it to you
He should sell it to you.
And how much should he sell it to you for?
Pi minus a penny.
And you might say, well, that's ridiculous.
I won't pay pi minus a penny.
Then I'll make nothing.
Well, but he will.
He'll pay minus a penny because at least he'll make a penny.
Now, you might say a penny, a dollar, $5, $10 whatever.
But the point is that he'll sell it for very close to pi.
Because there's someone out there who's willing to pay close to pi to get that.
As long as there's a little bit left.
So it's a little bit less than pi.
They'll pay it.
But what that means is having paid it, you don't make any money.
So let's do the extreme example where he's willing to sell it for $1 less than pi.
What that means is you having bought it for $1 less than pi, you don't make any money as a cab driver.
So wait a second.
This is weird.
We've just restricted the market.
We've said there's all this profit to be made.
And yet you're a cab driver and you make no profit.
What happened to the profit?
Who got it?
[UNINTELLIGIBLE]
The cab drivers who were originally issued the medallions.
The first generation cab drivers who got the medallions got all the money.
So in fact, taxicab medallions do nothing [INAUDIBLE] taxi drivers.
All they do is enrich the set of people who originally lobbied for them and got them.
OK. Yeah
But wouldn't he make his money back the same day he sold it, given that he can get the same price for his--
Wouldn't he-- No, but the point is that basically, what he's done, or you're saying that when he goes to sell it.
So, in other words, the taxicab medallion is worth a certain amount of money.
But that would be embedded in the money he would charge you.
So he's not dumb.
He's straightening out the supply curve, he's a smart guy, you should have picked someone else to negotiate with.
He's going to say, look, I'm going to charge you enough so that you'll only make $10 forever on having this.
I'm going to charge you enough so that basically, you will not make any money even when you go to get rid of it.
So basically, what you'll do is, you'll have to pay him so much.
So let's say the way it works.
Let me give you an example.
Let me come back to that one in the context of an example.
So what we know is that basically-- and this is in Perloff, give you some interesting facts on this.
We know that taxicab medallions are really limiting.
For example, we know that Tokyo has five times as many cabs as New York City, despite the fact that New York City's bigger than Tokyo.
And Washington DC has ten times as many cabs as San Francisco.
Despite the fact that Washington DC is smaller than San Francisco.
I'm not from San Francisco.
You can always get a cab in Washington, I don't know how hard it is to get in San Francisco.
But there's 10 times as many cabs in Washington.
And this is reflected in the value of a permit.
So in San Francisco, what you do, and this is sort of an easier way to think about it.
What he does is, he doesn't sell you the permit.
In San Francisco they don't sell it.
He rents it to you.
So you never get to own it.
He rents it to you.
And in San Francisco, the typical permit costs $42,000 a year to rent.
So if you want to be a cab driver, you've got to pay $42,000 off the top before you earn a dollar.
And then he rents to you, when you're done, he takes it back.
So in substance the way-- now, if he was selling it, I can describe it to you.
Involves the fact that he would embed your future asset in his sale price.
That involves some complicated finance that we'll get to later in the course.
So think about renting it, that's easier.
He'll rent it to you for the entire surplus pi, or pi minus $10.
And in San Francisco that amounts to $42,000 a year.
In New York, New York originally had 12,000 permits they issued in 1937.
They issued 12,000 permits for $10 each.
They have not issued a new one since.
Literally, there are no more cabs allowed in New York City than there were in 1937.
A typical taxicab medallion, which sold for $10 in 1937, now sells for $400,000.
Now once again, you typically don't sell.
You typically rent it out.
But the point is, that taxicab medallion, which embeds the entire future stream of having the right to drive that taxicab, is worth $400,000.
So what have we done here?
What we've done-- and in Boston, by the way, a taxicab medallion's worth about $250,000.
Think about that the next time you're standing in the rain waiting for a cab.
You're standing in the rain waiting for a cab because some guy in 1930-something got a grant from the government worth $250,000.
But the current cab drivers don't get crap.
A taxicab driver in your city makes $10-12 an hour.
It's not a very fun job being a taxicab driver in New York City, unless you like taking your life in your hands every time you go out on the streets of New York City.
They make $10 to $12 an hour.
After paying the enormous amount they have to pay to rent their taxicab medallion.
Now, this is not the only example we have in the world of something we call occupational restrictions.
There's lots of examples.
Probably the most prominent example, that may be relevant to some of you.
I hope the taxi cab driver won't be relevant to you guys.
But the doctor example might be.
A great occupational restriction is the AMA and the education, the institute that educates doctors puts a limit on the number of medical residency slots, that determines how many doctors there can be in America.
As a result, we all pay more to go for our medical care, than we would if more doctors were allowed.
And you might say, that's outrageous.
But why, obviously, how would the AMA defend this?
And how would you defend any occupational license?
Yeah
The smaller the amount of [UNINTELLIGIBLE]
Yeah.
You don't want everybody being Dr. Nick.
You want-- come on, you guys got to get Dr. Nick.
Raise your hand if you know what I mean when I say Dr. Nick.
Good lord, what is wrong with you people?
OK, homework for this course, you've got to watch one episode of The Simpsons.
Before the end of the term.
This is crazy.
Dr. Nick is the terrible doctor on The Simpsons.
You don't want this terrible doctor.
You don't want these terrible doctors operating on people, so we have these restrictions to make sure doctors are good.
And it sounds like a good idea.
But next time you hear it, remember.
It's not just making sure there are good doctors.
There are probably plenty of people who would be good enough doctors who aren't let in because there are not enough slots.
It's also a way to make sure doctors earn lots of money.
And it's not just the government that does this.
There's no government involvement here.
These are private associations which license.
And they restrict, plumbers, and doctors and optometrists, and all these other things are limited.
Ostensibly to keep quality up, but in reality often to make sure that there's some surplus being earned by this initial generation who puts puts them in place.
Yeah
So is the relative effect on the market smaller for doctors because they're not selling each other permits?
Well, what's different with the doctors is, since it's not a permit but an ongoing limit, then every doctor makes the pi.
There's no selling.
So the effect on the market is no different.
So imagine here, instead of it being a medallion, there's a limit on how many cabs could run.
And that limit was randomly reallocated.
Cab ran out, and then new people went in.
Then each generation would-- then you wouldn't get the thing where the first generation wins.
Each generation should win.
But you still get the same distortion of the market.
The same profit being made.
It's just that instead of the profit all accruing to the first generation, there'd be an ongoing approval of that profit to each generation of doctors.
Yeah
Professors and tenure, is that also related to--
Professors and tenure would be another very good example.
Basically, an occupational restriction.
The difference is there's no limit to how many tenured professors there can be.
So basically there's no sense in which, if there's more demand for education, a new university couldn't start up and have tenured professors.
There's no situation in which a university couldn't just say, we're given tenure to any Tom, Dick, and Harry who walks in off the street.
There's no stopping that, because tenure is determined by the institution.
So in that sense it's not an occupational license, because there's no board which determines tenure standards.
Thank goodness.
So we'll come back.
So that's what we were talking about welfare.
You'll review this more in section on Friday.
On Wednesday we'll come back to talk about monopoly.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so today we are going to continue with our discussion of producer theory.
And today we're going to move beyond the unrealistic case of perfect competition to the somewhat more realistic case of monopoly.
Now, we've been discussing perfect competition thus far as a form of market organization, and that makes sense in some context like fast food and other things.
But in most contexts, we think perfect composition is not the way the world works.
There's some limits on competition.
And we think that markets-- many markets, many of the goods we consume have only a few firms.
Operating systems or cars or a lot of things we consume, typically have only a few firms in them.
So the most realistic model of markets would be one which accounts for the fact that there's less than an infinite number of firms, there's only a few firms.
That turns out also to be the hardest model.
So what we do is we sort of iterate there.
We started with one extreme, which is a competitive market where there's an infinite number of firms, that allows us to draw some interesting conclusions.
Now we're going to reverse field and talk about the other extreme, monopoly, which is only one firm.
We'll talk about monopoly markets with only one firm.
Then we'll talk about oligopoly, that middle case, which is multiple firms.
So let's talk about monopoly, a market where there's only one firm.
The key thing to remember for monopolies is they're no longer price takers, they're now price makers.
Competitive firms are price takers.
They were given a price by the market and they reacted to that.
And as we saw in the long run, that price settled at the minimum of average cost.
So basically, they were given the price which dictated production efficiency.
They reacted to that in how much they produced.
You got a flat long run supply curve.
However, in a monopoly market, we don't meet the conditions for perfect competition.
In particular, one condition was that consumers had perfect substitutes between your good and other goods they could buy.
That's not true in a monopoly market.
So if we think about that era, sort of maybe, I don't know, I don't have my computer history isn't as good.
Maybe 10 years ago when Macs were sort of at the nadir of their popularity.
And really you had to be a pretty high tech guy to be using Linux and things like that.
That Windows had virtually a monopoly on operating systems.
Not really, but virtually.
Certainly unless you were a high tech guy who could do Linux and things like that, pretty much if you wanted an operating system, you got Windows.
Windows pretty much had a monopoly.
And that's may be as close as we've come in the modern economy to thinking about an example of monopoly.
In that case, Bill Gates had to decide-- wasn't given a price, he had to decide how much to charge for Windows.
And that decision determined in turn how much he would produce.
And that's exactly what we'll talk about today, is how does a monopolist decide both what to charge and how much to produce of their good?
So to do that, we're going to turn to a new concept.
Well, not a new concept, but a different way of looking at something we've talked about before, which is marginal revenue.
If you remember, the profit maximizing condition that we derived when we started our competition lectures was that marginal revenue equals marginal cost.
That was our profit maximizing condition, was that marginal revenue equals marginal cost.
Now we're going go to-- in perfect competition.
And the way we thought about this marginal revenue equals marginal cost, remember the logic was this notion of climbing this profit hill.
You want to produce any unit.
Think of yourself as climbing this profit hill.
You're deciding, do I produce the next unit?
And I produce the next unit as long as the money I make off that unit exceeds what it cost to produce that unit.
So I climb that profit hill.
If marginal revenue is greater than marginal cost, I keep going up.
At the peak, marginal revenue will equal marginal cost.
Once I go beyond that peak, marginal revenue will fall down below marginal cost and I'll stop producing.
So the notion is I climb this hill, which in the peak is dictated by marginal revenue equals marginal cost.
Now for a competitive firm, we said marginal revenue was just price.
So in perfect competition, the rule was set price equal to marginal cost.
But that was a particular case of marginal revenue.
So for example, to see that, let's look at Figure 14-1.
This is another way.
Probably next year when I teach this, I'll put this in the lecture on competition.
Just a way to think about how marginal revenue is priced.
Remember, a perfectly competitive firm faces a perfectly elastic demand curve.
They face a perfectly elastic demand curve.
So they have to think about what the implications are of the marginal unit they sell.
Well if they sell little q units, their revenue's a.
If they sell one more unit, their revenue is b.
And the marginal revenue is the height of that rectangle B times the base.
The base is 1 because it goes from q plus 1.
The height is p. So the marginal revenue is price.
This is a pretty basic diagram.
So marginal revenue is price for the perfectly competitive firm.
Now let's look at the monopoly case.
The difference with a monopolist, as you see in Figure 14-2, is they no longer face a perfectly elastic demand curve.
They now face a downward-sloping demand curve.
Why is that?
Well remember, this is the graph for little q.
The reason the graph for little q was perfectly elastic with a perfectly competitive case was not that we said the demand for the entire good was perfectly elastic.
It's just that the residual demand facing anyway one firm was perfectly elastic.
Well now, a monopolist is the only firm in the market.
So their residual demand equals total demand.
Their residual demand equals total demand.
So as long as total demand is downward-sloping, as we typically think it is, then they'll face a downward-sloping demand curve.
So unlike a perfectly competitive firm, which faces a perfectly elastic residual demand curve, a monopoly firm will face a downward-sloping market demand curve.
They face the entire market demand curve.
So this is demand curve.
Now we're talking about big Q's not little q's anymore.
Or we could say little q equals big Q.
There's only one firm.
So little q's and the big Q's are the same.
So now the firm reacts not to its residual firm demand curve, which is flat, but the downward-sloping market demand curve.
Now, we're going to make one assumption here that's very important.
We'll come back to this at the end of the lecture.
We're going to assume that the monopolist can only charge one price for their good to all consumers.
So most of this lecture we're going to assume a non-price discriminating monopolist.
We're going to say that Bill Gates can't look at you and say, look, you look like you really want Windows.
I'm going to charge you more than her.
He can't do that.
He has one price.
He sells it to the stores at one price.
So it's a non-price discriminating monopolist we're going to work with for the first 2/3 of this lecture.
That monopolist has to set one price.
Now, for that monopolist, for Bill Gates with Windows circa 10 years ago, let's think about his decision to produce another unit.
He's originally producing at Q, big Q at a price p1.
He's originally producing a big Q at a price p1.
If he wants to sell one more unit, he's going to have to lower the price.
Because he now faces a downward-sloping demand curve.
So if he wants to sell one more unit, he's going to have to lower the price to P2.
I have a big cockatoo at home.
This is his feather.
So he's going to have to lower the price to P2.
What's that going to do?
Well, on the one hand, what that's going to do is that's going to mean on that next unit he's going to make p2.
So he's going to get the rectangle B.
On the other hand, on all the units he was selling at p1, he now gets a lower price p2.
So he loses the rectangle C. So the marginal revenue for this monopolist, the marginal revenue is equal to the rectangle B minus the rectangle C. The marginal revenue is rectangle B minus rectangle C. Or alternatively, you could say that if we just write that out, write out what that is, that's p2 minus p1 minus p2 times Q1.
Or rewriting, we could rewrite this then as marginal revenue equals p plus delta p delta q, how much the price changes when you change the quantity, times the original quantity Q1.
Or one more time, for those of you who prefer calculus.
If revenue equals p times q, and q is a function of p, then marginal revenue, differentiating that, is p plus dp dq times Q.
So marginal revenue is the price plus the change in price from selling another unit times the initial quantity.
Once again, the graphics are in this graph.
The math is here.
We just differentiate the revenue equation.
This term is positive.
Price is always greater than 0.
But this term is negative because demand curves slope down.
So there's now two effects.
There's a positive effect, which is if I sell another unit, I make money on that other unit.
There's a negative effect, which is to sell that other unit, I have to lower the price because I face downward-sloping demand.
So there's two effects a monopolist as he thinks about wanting to sell another unit.
There's the money from that unit, but the lower willingness to pay for all previous units.
And that's what makes a monopolist a little more interesting.
We basically think of the monopolist as basically having to work down the demand curve.
With a perfectly competitive firm, they don't have to work down the demand curve.
The demand's flat to them.
They can sell as much as they want at that price because they don't affect the price.
They want to sell 10 of that price or a million at that price, it doesn't matter.
Their demand curve's flat.
Not true for the monopolist.
If the monopolist wants to sell more, he has to face the wrath of the market.
And the wrath of the market is such that if he wants to sell more, he's going to have to lower the price to do so.
Remember, once again assuming he has to charge the same price to everyone.
Assuming he's going to charge the same price to everyone, if he wants to sell more, he's going to have to lower the price to do so.
There's different ways of [UNINTELLIGIBLE] intuition on this.
I like to call this the poisoning effect.
That's how I like to think about it.
That basically, if I want to sell another unit, I'm going to poison the money I made on all previous units.
Because if I want to sell another unit, I have to lower the price.
That's going to take away from the money I was making on my previous units.
So it's sort of a poisoning effect is how I like to think about it.
But you can have your own intuition for it.
And basicallly, this poisoning effect did not exist for the perfectly competitive firm because they couldn't affect the price with their action.
They could sell however many units they wanted at that flat price and their actions did not affect that price.
The poisoning effect only exists with monopolists because to sell that next unit, they have to lower the price.
Now, basically what that means is to find equilibrium for a monopolist, it's going to be a little bit trickier.
And so let's go to Figure 14-3 and slowly walk through Figure 14-3.
And we'll start walking through.
What the monopolist is going to want to do is draw a marginal revenue curve.
With the perfectly competitive firm, marginal revenue curve was just a price, it was given to them.
There was no marginal revenue curve.
For a monopolist, there is a marginal revenue curve.
So here I have a demand curve.
The demand curve I've drawn here in this example.
The demand curve I've drawn here is q equals 24 minus p. That's a typical demand curve, downward-sloping.
As the price goes up, people want less of it.
And now we're in market demands.
Because remember, little q equals pq.
There's only one firm in the market.
Now, here's the trick with monopoly: mathematics.
The first thing you're going to want to do is you're going to want to invert this.
You're going to want say, OK, that takes q, but what takes the price a monopolist is going to charge?
The price a monopolist is going to charge is therefore going to be 24 minus q.
That's going to be the price that the monopolist is going to charge.
Therefore, revenues, pq, is 24q minus q squared.
So we inverted the demand equation.
We do this because we want to write down a revenue equation.
So that's the trick.
This is the mathematical trick here is to invert this, so then you can write down a revenue equation.
We then differentiate this revenue equation to get that marginal revenues equals 24 minus 2q.
That's marginal revenues.
The next unit you sell, you make 24 minus 2 times the amount you're now selling.
And that's the marginal revenue curve graphed here.
Now, basically what you see is in this case the marginal revenue curve starts at the same point as the demand curve on the y-axis and lies everywhere below the demand curve.
Now that first fact, that it starts at the same intercept, is not always true.
That is true because in this case we assumed a linear demand curve.
With a nonlinear demand curve, marginal revenue curves can start at different points on the y-axis.
The second point about the marginal revenue curve always being below the demand curve is always true regardless of the function.
The marginal revenue is always below demand.
Marginal revenue is always below demand.
So that marginal revenue curve will always be below the demand curve because of this poisoning effect.
Now, what I want to highlight here is this means there's a very important relationship between marginal revenue and the elasticity of demand.
So let's take our marginal revenue equation and put it back in change terms.
p plus delta p over delta q times Q.
And let's multiply and divide by p. So marginal revenue you can rewrite as p plus p times delta p over delta q times Q over p. So I just took this second term, multiplied and divided by p, the second term.
I just multiplied and divided by p. The reason I did that is because that means you can rewrite this.
This now starts to look like an elasticity expression.
Remember the expression for elasticity.
This looks like the inverse of an elasticity expression.
Remember what elasticity of demand was, delta q delta p times p over Q.
So that's the inverse to the elasticity demand.
So we can rewrite this as marginal revenue equals p times 1 plus 1 over the elasticity of demand.
Marginal revenue equals p times 1 plus 1 over the elasticity of demand.
Think about what this means for a second.
What is the marginal revenue in a perfectly competitive firm?
Well, as a perfectly competitive firm, what's the elasticity of demand facing a perfectly competitive firm?
Infinity.
Perfectly elastic.
So marginal revenue by L'Hopital's rule equals p. So for a perfectly competitive firm where elasticity is infinity, marginal revenue equals p. Now instead, if we took a firm where the elasticity of demand was minus 1, the electricity demand was minus 1, the marginal revenue would be 0.
Why is that?
What that says is, if you're a monopolist facing an elasticity of demand of minus 1, then you make no money by selling the next unit.
Because these two effects exactly cancel.
It turns out with elasticity of demand of negative 1, these two effects exactly cancel.
Exactly what you make by selling one more unit is offset by how much you have to lower the price on all your previous units.
So an elasticity of demand of minus 1, marginal revenue equals 0.
And as you can see as the elasticity of demand gets below minus 1, as it approaches 0 from below.
As the elasticity of demand approaches 0 from below-- OK, I should have said perfect competition, I'm sorry, was negative infinity, not infinity.
Negative infinity.
As the elasticity of demand approaches 0 from below, then you're going to see that the marginal revenue-- as you approach 0 from below, marginal revenue is going to become negative.
So for example, if the elasticity of demand equals minus 0.5, then the marginal revenue equals minus p. So if this is minus 0.5, then this becomes minus 2.
So marginal revenue equals minus p. You lose money.
So as that elasticity of demand approaches 0, you're going to have a negative marginal revenue from selling the next unit.
And why is that?
With a very inelastic good, you have to push the price down so much to sell the next unit that you lose money.
Think about a very elastic versus very inelastic good.
With a very elastically demanded good, to sell another unit you don't have to change the price much.
Because the demand curve's very flat.
So there's not much of a poisoning effect.
dp dq is small, or dq dp is big.
OK, this is the inverse.
So dq dp is big with elasticity, so dp dq is small.
With a very inelastically demanded good, to sell one more unit you're going to lower the price a ton, which is going to poison the revenues you get from selling that extra unit.
So that's why marginal revenue will be higher, or will be a larger fraction of p as this elasticity becomes more negative.
Yeah
[UNINTELLIGIBLE PHRASE] elastic that means [UNINTELLIGIBLE PHRASE] irrespective of the price.
So couldn't you just charge a higher price and get marginal revenue [UNINTELLIGIBLE]
No, because here's the thing.
You should have already been charging that high price.
The point this is the margin.
So you go into a market for insulin.
You say, look, these guys are going to die without it.
I'm going to charge $500,000 a shot.
But now the question we're asking about marginal revenue.
And at $500,000, you sell to everyone who can afford it and everyone else dies.
Now you want to ask, what's the marginal revenue as you're trying to sell that 500,000 and first unit?
Well to sell that, since it's inelastically demanded, if that person could afford anything like $500,000, they would have bought it already.
They can't.
They can only afford $400,000.
We have to lower the price to $400,000.
You're going to sell one more unit at $400,000 but lose $500,000 on all those other units you were going to sell at $500,000.
The point is it's about the margin not the level.
Yes, monopolists make a huge profit when it's inelastic.
I'll talk about that in a minute.
But that next unit they're going to lose money on.
So that's actually a good point to segue to now, let's talk about with this in place, let's talk about how monopolists maximize profits.
Let's talk about monopoly profit maximization.
Let's go to Figure 14-4.
Profit Maximization for a Monopolist.
Now, this is a lot more confusing than perfectly competitive firms, so let's follow along here.
This is a case the cost function here is 12 plus q squared.
So I'm doing the cost function, which is 12 plus q squared.
That's the cost function.
And the demand function, as before, is Q equals 24 minus p. So that's what's graphed here.
Now, recall the rule that profit is maximized where marginal revenue equals marginal cost.
Well, we know marginal revenue.
We know marginal revenue-- we derived that above-- is 24 minus 2Q.
What's marginal cost with this expression?
Well, marginal cost, differentiation of the cost equation, which is 2Q.
So the optimization term for a monopolist is going to where marginal revenue, which is 24 minus 2Q, equals marginal cost, which is 2Q.
Or Q equals 6.
That's going to be the optimal production level for the monopolist.
So we can see that graphically that's where the marginal cost curve hits the marginal revenue curve.
If you go downward from that point, you get that the sales are 6 units.
So marginal revenue equals marginal cost at 6.
You should be able to see that graphically, it's just where the curves intersect.
Mathematically I just did it here.
It's actually pretty straightforward.
Here's the hard part.
What's the price?
We might say, well, gee, marginal cost and marginal revenue intersect at 6.
I'm going to draw the dashed line over.
That means the price is going to be 12.
Why can that not be the price?
Why is that wrong?
What would that violate if the price was 12?
If you tried to sell 6 at a price of 12?
Yeah
[INAUDIBLE]
It's not on the the demand curve.
The monopolist still has to respect the demand curve.
So monopolists in setting their quantity, gets the intersection of marginal revenue and marginal cost.
But then in setting the price, they still have to read off the demand curve.
They can't change consumer tastes.
So they charge a price of 18.
That's where you sell a quantity of 6.
So monopolists it's a little bit trickier in a perfectly competitive firm.
You set marginal revenue equal marginal cost to derive Q.
But then to get p, you've got to go back and plug that into the demand curve.
So with a Q of 6, I have my Q of 6.
Well, what's the p?
Well, to get that p, I've got to go back and plug this in here.
At Q equals 6, p is 24 minus q, or 18.
So I've got to respect the demand curve.
The monopolist has to respect the demand curve.
The monopolist picks both price and quantity, but he has to pick them such that you get a point on the demand curve.
And the way we solve it, is the monopolist chooses a quantity to set marginal revenue equal to marginal cost, and then chooses the price that's consistent with demand for that quantity.
Questions about that?
Now one last thing.
In the short run, we still have another condition for profit maximization, which is the shutdown rule.
Remember the shutdown rule we talked about perfectly competitive firms in the short run, which is even if profits are negative, you might not shut down.
You only shut down if price is less than average variable cost.
So there's still the shutdown rule.
So you only shutd own if price is less than average variable cost.
Now in this case, what's the monopolist profits?
Well, the monopolist made a profit of 60.
How do we see that?
Well that's graphically the box, the rectangle, that's the difference between the average cost curve and the price they get.
So they're charging 18.
Now once again, marginal revenue is gone.
Think about marginal revenue like an imaginary concept.
Marginal revenue isn't something that actually exists in the market.
Marginal revenue is just something the monopolist draws to pick what they're going to do.
But then it disappears.
What the monopolist cares about then is price.
They're charging 18.
Their average cost for that unit is only 8.
So they're making a profit of 10 per unit on 6 units.
So they're making a profit of 60.
And what you can see, what you should be able to demonstrate to yourself is, if a monopolist sold 5 units.
I'm sorry, selling 6 units.
If the monopoly sold that seventh unit, what you'll be able to see is they would lose money on the seventh unit.
Because yes, if they sold that seventh unit, what happened if they sold the seventh unit?
Well then their price would have to be what if they wanted to sell a seventh unit?
The price would have to be 17.
The price would have to be 17.
So to sell a seventh unit, they'd have to have a price of 17.
So basically at a price of 17, the price was 17.
Then what would happen?
Well, they'd sell one more unit at 17.
That'd be good.
But they'd lose $1 on the previous 6 units, which is bad.
So how much revenues would they make?
What would be their marginal revenue?
Well, the marginal revenue would be they make 17 minus the 6 poisoning effect.
So marginal revenue equals 11.
What's their marginal cost?
Their marginal cost is 2Q.
Marginal cost is 14.
So they lose money.
So you should be able to walk through this exercise yourself.
You might say, gee, the marginal cost of that next unit is only 14.
They sell it for 17.
Gosh, they should do it.
What you're missing is by selling it for 17, they've lost the dollar extra they make on each of the previous 6 units.
And that poisoning effect makes it unprofitable to do this.
And that's why the monopolists stop short of what would be the perfectly competitive outcome.
What would the perfectly competitive firm do?
The perfectly competitive firm would set marginal cost equal to demand.
And they would end up producing where marginal cost equals demand.
So demand here is 24 minus p. Marginal cost is 2Q.
So they would end up producing where marginal cost equals demand at a much higher level charging a slightly lower price.
So what you see is the monopolist ends up selling fewer units at a higher price.
Questions about that?
Yeah
How does this work for Microsoft where their marginal costs are very low or nonexistent?
Well, then what would happen, if their marginal costs were very low or nonexistent.
Think of that marginal cost curve then as being much, much flatter.
It would intersect demand at a much higher quantity.
Or it'd intersect marginal revenue at a somewhat higher quantity.
Not that much higher.
So if marginal cost is very low, they produce more but they make even more profits.
So it's a good question actually, a good comparative statics exercise.
You bring that marginal cost curve down, what's going to happen?
Quantity is going to go up, but not as quickly as profits are going to go up.
That's why Bill Gates is the richest man in the world.
That's what happens.
You get really rich.
So basically, when you're a monopoly, low marginal cost you get really rich.
But that's a great thought exercise to understand how this monopoly example works.
Other questions about that?
So this is a good opportunity to introduce an important concept with monopolists, the concept of market power.
What monopolists have, what Bill Gates has that my local McDonald's does not is market power.
Or what he had, has less of now, is market power.
Market power is the ability to charge price above marginal cost.
The summary statistic of how much power a monopolist has is how much they can drive their price above marginal cost.
When Bill Gates marginal cost dwindles to 0, his market power gets bigger.
Price above marginal cost.
Now to think about this, remember the condition for profit maximization.
It was that marginal revenue, which we wrote as p times 1 plus 1 over epsilon equals marginal cost.
So we can rewrite this as marginal cost over price equals 1 plus 1 over epsilon.
Now let's define the markup.
Let's define the markup as price minus marginal cost, how much money you make on the next unit.
You sell for p, you get marginal cost.
It's money you make the next unit.
If you define the markup, p minus MC over p, that's the percentage markup.
It's how much you make on the next unit, the percentage markup.
Then you can see that that markup equals minus 1 over epsilon.
So the markup for a monopoly firm equals minus 1 over epsilon.
This comes to the question before about the insulin example's sort of confusing.
Here we see your intuition on insulin.
The lower its elasticity, the more the monopolists can mark up their price.
So your intuition is shown here.
Yes, the monopolist will charge an incredible price for insulin.
They'll still lose a lot of money if they try to raise that price, if they try to sell one more unit.
But the first initial price they'll set will be incredibly high.
Basically, what is the constraint on Bill Gates?
What is the constraint on Bill Gates?
It's Steve Jobs.
It's substitutes.
The only constraint on a monopolist is the extent to which people can sub-- actually, let me go back, that's not a good example.
Let's [UNINTELLIGIBLE] Bill Gates circa 10 years ago.
The constraint on Bill Gates circa 10 years ago was a mainframe or some other form of doing a set of-- or 20 years ago it was a typewriter.
It was basically the fact that there was some other way to do what Bill Gates was letting you do.
If there was no other way to do what Bill Gates was letting you do, he would charge an infinite price.
Clearly if there's some other way-- and also, elasticity of course, comes from substitutes or one substitute is just not to compute.
So if Bill Gates tried to charge infinity for Windows, people just wouldn't own computers.
So the reason Bill Gates can't charge infinity, and the reason he can't charge infinity for insulin is that there's some elasticity of demand.
People at some point will just stop buying.
Either because they'll choose to use a typewriter instead or they just won't compute.
They'll write by hand or something.
So basically at some point, there is some elasticity because there's a market demand curve.
And basically what's going to determine how much market power the monopolist has is going to be how elastic it is.
Basically, how close the substitutes are for that good.
If there's close substitutes, the monopolist won't be able to charge a very high markup.
If there's not close substitutes as of Window circa 10 years ago, the monopolist can charge a very high markup and become very, very rich.
Yeah
But if there are substitutes for the market, then it's not a monopoly anymore
No, no, this is the key thing.
Substitutes for that producer.
So basically, that's why I said Steve Jobs is not a good example.
Because then it's not a monopoly anymore.
But the typewriter is a good example.
That's a different good, that's a different market, different good that substitutes.
So my point is any given good, insulin being an exception, but any good there's always something you can do instead.
Insulin there is something you can do instead, you can be sick.
There's always something you can do instead.
We don't have only one thing in life.
So the elasticity of demand is never perfectly inelastic.
It seems silly 10 years ago, but 20 years ago it actually was a legitimate decision whether to have a PC or not.
A lot of people just didn't have computers.
You could always just not have one.
That gives you inelasticity of demand.
So basically, it's important to recognize when we talk about substitutes, I'm talking about here not substitutes within the market, but substitutable activities, other things you could do with your money.
And the more other things are you could do with your money, the less markup that Bill Gates can make on his Windows operating system.
Questions about that?
OK, now we can ask, OK, gee, John, this is all good and interesting, but why did you just waste the last lecture and a half teaching us about welfare if you're just going to go back to producer theory?
Well, the reason is because now we come to what the welfare effects of monopoly.
And ask, what effects do monopoly have on society?
And in fact, we can show you that there's a deadweight loss on society imposed by monopoly.
To see that, let's go to Figure 14-5.
And here we can show the deadweight loss of monopoly.
And here's the same example we were using.
Demand is Q equals 24 minus p. Marginal cost is 2Q.
The cost function is 12 plus Q squared.
So marginal cost is 2Q.
As we saw before, the monopolist chose to sell 6 units at a price of 18.
6 units at a price of 18.
The perfectly competitive firm sets demand, which is 24 minus Q, sets price, I'm sorry, equal to marginal cost.
Well, price comes to demand curve as 24 minus Q.
Marginal cost is 2Q.
So the perfectly competitive firm sets Q equal to 8.
The perfectly competitive firm sets q equal to 8.
They choose to sell 8 units at a price of 16.
So you get the competitive quantity Q sub c is eight and the competitive price piece p sub c is 16.
That's where graphically demand equals marginal cost.
Or price equals marginal cost.
The monopoly firm sells 6 units at a price of 18.
So what is the welfare effects of monopoly?
What we see is we know that the competitive firm maximizes welfare.
We learned that last time.
We know that the best you can do is to sell 8 units at a price of 16.
What happens when you sell 6 units at a price of 18?
What happens is consumer surplus falls from A plus B plus C. So with perfect competition, consumer surplus is A plus B plus C. With a monopoly, consumer surplus falls to the area A.
So you lose B plus C with monopoly.
Producer surplus under perfect competition was the area D plus E. Now under a monopolist, the producer surplus is equal to D plus E plus B.
The monopolist , in this case, gained the rectangle B, but gave up the rectangle E. The consumer lost the rectangle B, that was a transfer to the monopolist.
So there was a transfer of the rectangle B from the consumer to the monopolist.
But C plus E have disappeared.
They're a deadweight loss.
They're deadweight loss because in the perfectly competitive equilibrium these are trades that would have made both parties better off.
That is, these are trades which socially should happen.
They are trades where the value to the consumer exceeds the cost of producing that unit.
Those seventh and eighth units are units-- so take the seventh unit.
What's that worth to someone?
Well, it's worth 17.
We can read that off the demand curve.
That's a willingness to pay curve.
People are willing to pay 17 for that seventh unit.
What's it cost to produce?
It cost 14.
So you have a unit which people want more than it costs to produce, yet it's not getting sold.
That's deadweight loss.
So monopolists induce deadweight loss because units that people value above their marginal cost doesn't get sold.
Units people value above their marginal cost don't get sold.
And that's because this poisoning effect.
Because while it's socially optimal to sell those units, while society is better off, it's privately sub-optimal.
From the monopolist's perspective, it's bad to sell that unit because of this poisoning effect.
So basically, the monopolist is underselling, underproducing.
In general, monopolists will underproduce goods.
They'll sell too few goods because to sell the right amount would not be profit maximizing.
Because remember, what's the profits for the perfectly competitive firm?
Profits for the perfectly competitive firm?
Well, we know the profits of perfectly competitive firm.
We know cost if they sell 8 units.
We know the cost function is, the cost here is 12 plus Q squared.
So if they sell 8 units, their costs are 12 plus 64, which equals 76.
Their revenues if they sell 8 units are 8 units times the price of 16.
8 units time the price of 16, which is 128.
So what are their profits?
Their profits are 52.
So their profits are 52.
The monopolist's profits are 60.
So the monopolist is better off than the competitive firm would be.
The competitive firm would only make profits of 52.
Obviously the short run.
The long run they make no profits.
But in the short run they make profits of 52.
The monopolist makes profits of 60.
So the monopolist is better off than the competitive firm.
The difference of course, is to do so they cause a social deadweight loss.
Questions about that?
Yeah
Is that what the OPEC is doing right now?
I'm going to come to that actually.
Time out on that.
Because OPEC is more of an oligopoly.
And we'll come to that when we talk about that in a couple of lectures.
But I want to talk about one more thing before we stop, which is I want to talk about the key assumption we made here, which was the monopolist could only charge one price to everyone.
In fact, we know that's not true.
In fact, we know in the world, there's a large amount of what we call price discrimination.
There's a large amount of price discrimination.
We know that for many goods, different prices get charged to different consumers.
If you ever tried to book an airline ticket the last minute, you know exactly what I mean.
Basically, different prices in many, many contexts get charged different consumers.
Everything from discounts for senior citizens, to higher price last-minute flights, to specials, two for one specials.
People who buy two get a special price on the third, et cetera.
There's all sorts of price discrimination just out there in the world.
And in fact, there's very few goods that are sold at just one price.
McDonald's hamburger is typically sold at just one price.
They don't say like, fat people got to pay more for McDonald's hamburgers or something.
But many, many goods we buy in the real world are sold at many prices.
And that's an example of a price-discriminating firm.
And here's the crazy part.
Here's the crazy part.
It turns out that a price-discriminating monopolist maximizes social welfare.
A price-discriminating monopolist is as good as a competitive outcome.
How can that be?
Let's go to Figure 14-6.
Here's the price-discriminating monopolist.
Now, the price-discriminating monopolist, what does he do?
This is a perfectly price-discriminating monopolist, someone who can charge a different price to every single consumer.
Well, if you were a price-discriminating monopolist, perfectly price-discriminating monopolist and you could charge a different price to every consumer, what do you charge the first consumer?
24.
What do you charge the second consumer?
23.
Third consumer, 22.
You literally charge them their willingness to pay.
If you're perfectly price-discriminating, then what you do is literally charge every consumer exactly their willingness to pay.
You say look, I know your willingness to pay function.
Your willingness pay function is p is 24 minus Q.
That's your willingness to pay function.
So I'm going to literally charge you that.
I know that about you, it's stamped on your head.
So I'm going to say, ah, you're willing to pay 24 for the first unit, 23 for the second, et cetera.
In that case, what will the perfectly price-discriminating monopolist do?
Will they stop at 6 units?
No, they won't.
Because for that guy, there's no poisoning effect.
There's no reason to stop at 6 units.
That seventh unit, as we just did the math, there's money to be made on that seventh unit.
Because that seventh unit is worth 17, but it only costs 14 to produce.
So the perfectly discriminating monopolist will sell it at 17.
Likewise the eighth unit, people willing to pay 16 and it cost 16 to produce.
So they'll sell it or not.
They're basically indifferent.
So we typically say they'll sell it.
The point is, the perfectly price-discriminating monopolist will work all the way down the demand curve to the competitive outcome.
They will move to the competitive market outcome because there's no poisoning effect.
There's no reason not to.
No reason not to sell as many units.
No reason not to climb the same hill the competitive firm climbs and sell any unit where the price exceeds the marginal cost.
Well, what's interesting is let's ask what's happened to social welfare with this perfectly price-discriminating monopolist.
Well, consumer surplus is what?
What's consumer surplus with the perfectly price-discriminating monopolist?
Somebody raised their hand.
Yeah
Zero
Zero.
Why is it zero?
Because they're charged exactly how the value is
Exactly.
Consumer surplus is defined as willingness to pay minus price.
But your price is set equal to your willingness to pay.
So by definition, consumer surplus is 0.
With a perfectly price-discriminating monopolist, there's no consumer surplus.
But what's producer surplus?
Same person, what's producer surplus?
Everything else
Everything else.
A plus B plus C plus D plus E. There's no deadweight loss.
You get exactly the same social welfare as you got with perfect competition.
It's just divided differently.
With perfect competition, consumers got A plus B plus C. Producers got D plus E. With a perfectly price-discriminating monopolist, the monopolist gets everything.
But the total shaded area is the same.
So really fascinating because here we have the ultimate screw on consumers.
We think about competition as being the best thing for consumers.
Lots of firms selling goods at a competitive market where you can shop and do what's best for you.
It's not surprising intuitively that that's the best thing for society.
What's very surprising intuitively is having a producer who can screw every single consumer out of every penny they value something is equally good for society.
And why is that?
That's because we've made a particular assumption, which is social welfare is the sum of producer surplus and consumer surplus.
The linear sum.
Since it's is sum, we don't care in that function who gets the dollars.
We just care about the total amount of dollars, the total size of the pie.
We don't care who gets what slice of the pie, we just care about the total size of the pie.
And the total size of the pie is the same with a perfectly price-discriminating monopolist and a competitive firm.
What this highlights is that that's a pretty stupid way to think about social welfare.
Clearly, we don't feel the same way about a market where people get everything they want and a market where people-- all they're willing to pay is sucked out of them by a greedy monopolist.
Clearly we don't.
And that's why we're going to need to think more richly about equity and think more richly about the division of resources in society.
Because it turns out that you can have equally good outcomes from an efficiency perspective that are very, very different from an equity perspective.
And this is the first example we'll see of that.
What we'll do when we get towards the end of the course lectures, like 23, 24, lectures like that, we're going to start talking about equity.
And what are different rules we can think of for dividing this pie that might give us a different answer?
OK, questions about that?
All right.
OK, so anyway, we'll stop here then.
Let's remember that perfectly price-discriminating monopolist is obviously also a silly concept just like a perfectly competitive firm's a silly concept.
What we're going to do next time is come back and talk about price discrimination in reality and what firms do to try to approximate this golden outcome.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.
mit.
edu
Once again to put this in the context of where we are.
We're talking about producer theory.
And we're talking about how firms decide how much to produce.
We started with a perfect competition, and that nailed down the fact that with under perfect competition we got the zero profit long run perfectly elastic supply curve.
We then, last time, talked about monopoly as another extreme and talked about how monopolists choose both their price and their quantity.
They set margin around or equal to marginal cost.
That gives them a quantity.
And then they choose the price off the demand curve and how that leads to profits and how that leads to monopolists producing less than the competitive level.
And we also ended by talking about a perfectly price discriminating monopolist, one that can actually charge each consumer their willingness to pay and therefore, absorb all the social surplus.
Now, I want to come back and start, this morning, by talking again about price discrimination and realizing that, in some sense, we know that there's no such thing as perfect price discrimination.
There's no monopolist out there charging us literally exactly what we're willing to pay.
But there are lots of examples of price discrimination in the real world.
And sometimes they're kind of obvious.
Sometimes they're not.
But what they all have in common is the notion of trying to find some signal of our willingness to pay and using that to set the price.
So instead of being like our original monopolist, just setting one price.
Monopolists are forever trying to find some signal, some underlying signal of our willingness to pay and using that to set the price.
OK.
So for example, a classic example we know is airlines.
OK. What airlines do-- Airlines charge you more, for example, if you book your ticket the last minute.
And the notion of course-- or towards the last minute.
At the very last minute they may end up charging you less.
But as you get close to the date when you want to leave, the price goes up.
The notion, of course, is that who is going to buy tickets close to the date they want to leave.
It's largely going to be business people, not the vacation traveler.
The vacation traveler is going to plan ahead and buy far in advance.
The one that's going to buy a week before is the business person who just got a meeting scheduled.
Business people are going to be less price sensitive.
They're going to have a higher willingness to pay.
They've got to get to the meeting.
They're not paying anyway.
The company's paying.
And they're just to do what they need to get to the meeting.
So that's a less price sensitive population.
So that's the population you want to hit with a higher price.
So you want to price up tickets that you buy towards the end because those are more likely to be the people who are least price sensitive.
So let's ask another question.
Somebody tell me why do movie theaters charge less for matinees than for movies at night?
Why do movie theaters charge less?
If you go see the first movie of the day-- and you guys probably don't anymore.
But when you were kids you did.
And when I take my take my kids, I do.
And that afternoon matinee movie costs less.
Why is that?
Yeah.
Anyone want to take a chance at that?
And related to it, you can also ask the question.
Why do restaurants have early bird specials?
Why, when you go to restaurant, often will they charge you less if you go there in the early hours as opposed to later?
So what's going on with that?
Someone tell me.
Yeah
They're not using up all their seats in the afternoon.
So they might as well lower the price
In the afternoon they're not.
Although the truth is, a lot of the time they're-- If you look at a movie theater today, it's almost never full, except for like the opening night of a very popular movie.
So that's part of it.
But let's talk about the demand side.
What else is special about the type of people?
Why don't you go on what's special about the type of people who want to see movies or go to restaurants at those different times?
They're not willing to pay as much
Yeah, they're basically people with a lot of free time on their hands.
Right?
The people who are going out to dinner early, the people who see movies in the afternoon, these are people who are unemployed or have little kids.
And they're just looking to kill time.
And so basically, these are people who are willing to shop.
These are people who are going to say look, I've got to fill in some time.
I don't have to go right now.
I can go any time.
So I'm going to be more price sensitive.
The people who are going to dinner later, who are going out to the movie at night, they're working all day.
They have no choice.
They have to see the movie at night.
Or they're in school all day.
They have to see the movie at night.
They're less price elastic.
So that's why you want to set the movie price higher at night.
Alternatively, another reason could be-- One signal of willingness to pay is price sensitivity.
What's another signal?
It's going to be income.
Who's going to be seeing the movies during the day and going to the dinner early?
It's elderly individuals and families with small kids who are the most resource strapped.
People who have plenty of disposable income are going at night.
They're working.
They're young.
They're singles and young couples.
They work all day or are in school all day.
They go at night.
They have more disposable income.
The people going during the day are the elders and the people with young kids who have less disposable income.
And as a result they're going to be more price sensitive.
So you want to charge them a lower price.
So that's another example of price discrimination.
OK, why does Disney World charge you less if you live in the area than if you do not.
This is true fact.
Disney World charges you less if you're from Orlando than if you're not from Orlando.
Why is that?
Someone tell me.
Yeah
I think the Disney can go any time.
So you might as well you might as well take them to other places
I mean, basically the idea is look, if I just spent a grand on a plane ticket to take my kids to Orlando to Disney World, I'm not going to kvetch over $10 either way, getting in or not getting in.
But if I live in the area, I may or may not go.
I can go to a park, instead.
I can go do something else, instead.
It's just one of many choices for me.
The point is, having paid the fixed cost of getting to Orlando, I'm going to be much less price sensitive about getting into the park than other people who live there, who have lots of other choices.
Likewise, having paid the money to get there, I have shown I have more disposable income because I paid the money to get there.
I've shown I have enough disposable income that I'm willing to drop money on a plane ticket to take my kids to Disney World.
So I'm going to have more income that we are willing to pay.
OK, so basically the point is companies are forever doing this.
The ultimate example was, briefly, Amazon.com started pricing by IP address.
For a while, it was a controversy over Amazon started literally charging, getting people's IP addresses and literally adjusting prices based on saying, oh, this person is from a university library.
University libraries aren't price sensitive.
We'll charge them more, et cetera.
That got shut down.
But that was the ultimate form of trying to be price discriminatory.
That was as close as we'll come to perfect price discrimination.
The point is, in general, people with market power, people that set the price above marginal cost will in general be looking for parties which are high income and low elasticity to which they can charge a higher price.
Yeah
Another example, like MIT, they actually ask for your income statement.
And then, based on that, they give you their price
Well, see that's very interesting.
That's very interesting.
MIT-- What's tricky about that is-- Is that about exploiting price sensitivity?
Or is that about equity?
That's a bit harder because MIT could claim-- And probably, actually, it's true.
They're not doing it.
Because the truth is, MIT could charge whatever they want.
They'd still fill the class.
Right?
So it's not really about-- I think in MIT's case, it's not really about price sensitivity.
It's about income redistribution and some notion of making sure low income people can come and afford to be here.
But that is a good point.
The difficulty with empirical economics is often, correlation versus causation.
The same fact can have many different descriptions, many different explanations.
OK?
So just wanted to give you some brief examples to think about the fact that we see price discrimination in reality all the time.
Nonetheless, we do our classic monopoly model.
We assume there's one price.
And the truth is the one price monopolist is probably closer to the truth than the perfectly price discriminating monopolist.
But the truth, as often in this course, is going to be somewhere in between.
OK, questions about that.
What I want to focus on primarily, for this lecture, is where do monopolies come from.
How do monopolies arrive?
How do we end up in this situation, where instead of many firms competing to sell a good, we only have one?
And there's basically two sources of monopolies, two sources that give birth to monopolies in modern societies.
The first source is cost advantages.
The first source is cost advantages.
Some markets come with natural cost advantages, natural reasons why there's a benefit to being one player in the market.
So for example, it could be that there's, essentially, only one usable input.
So let's say there's a rock quarry in town.
You've got to get rocks from.
And there's just one rock quarry.
And rocks are really expensive to transport.
So basically, if you live in that town, the person who owns that rock quarry is going to be the monopolist.
No one else is really going to compete because it would cost them so much to transport rocks from another quarry in another town.
OK?
Basically, the point is.
Sometimes, there are these fixed costs.
And whoever has paid the fixed cost has an enormous natural advantage over other players, over other entrants to the markets.
And we call those natural monopolies.
Natural monopolies-- Natural monopoly is a firm where for all the relevant quantities over any relevant range, the average cost for one firm is always below the average cost of a new entrant.
The average cost for the one firm that's in the market is already always going to be below the average cost for any new entrant.
And that is going to be true in markets with very large fixed costs and very small marginal costs.
A natural monopoly will occur when there's very large fixed cost and very small marginal cost.
In that case, what we'll see is average cost is always declining.
Natural monopolies arise when average cost is always declining.
If average cost is always declining, it's never going to make sense for another entrant to come in the market.
And an example we could think of-- The best example I could think is like a water utility, the folks who deliver water to our houses.
The fixed costs of delivering water are enormous.
You've got to lay the pipe to deliver the water all through the town.
Those are enormous fixed costs.
The marginal costs are trivial.
I mean, water is cheap.
You get the water.
You send it through the pipes.
OK?
It would never make sense to have two companies deliver water throughout the town.
It wouldn't make sense to have another company come in, rip up the roads, lay a competing set of pipes all through the town to compete.
It just wouldn't make sense because those fixed costs are so huge relative to the marginal costs.
They could never make money.
That first company-- If a second company came in and threatened to do that, the first company could always keep the price low enough that that second company could never recover their fixed costs.
Because the first company has already paid the fixed cost, they're already in.
A second company could never come in and cover those fixed costs.
So if we see figure 15-1.
Here's an example of a firm, of a water utility, low marginal cost.
There was a high fixed cost.
So the initial average costs are very high.
But the average costs are declining constantly because the marginal costs are low, relative to fixed costs.
OK?
So if demand for the good goes up, over some relevant range-- If demand gets to be infinite, it may make sense to lay another set of pipes.
You know, the first set of pipes may get overwhelmed.
But as long as demand can be met by the set of pipes that's been laid, there's absolutely no reason for another firm to enter this market.
If another firm ever threatened to do so, the first firm would just lower price and say, look, you're going to have to spend so much money putting that set of pipes in through town you'll never be able to make it up.
Because having laid those set of pipes-- Remember what the shut down condition is.
As long as that first firm can charge price greater than marginal, can charge price above average variable cost, which is marginal cost in this case.
As long as that firm can charge price above marginal cost, it will always stay in business.
And that second firm can't make any money at a price near the marginal cost because they can't cover their fixed cost.
So they'll essentially chase them out.
So essentially, the fact that they've had that set of pipes already laid down gives them a threat, a credible threat, that that second firm cannot enter.
Thus, we call natural monopolies are places where other firms face barriers to entry.
Natural monopolies are where there are barriers to entry, where other firms cannot enter because they'd have to pay those enormous fixed costs.
And it would never make economic sense to do so.
OK. And in that case, you're going to get a monopoly.
And we call it a natural monopoly because, in some sense, the monopoly is the right thing to do.
It is actually economically efficient for there to be only one provider of water in the town.
That's the economically efficient outcome.
It is economically inefficient to pay a second set of those enormous fixed costs.
Now, then you say, well what about the fact that monopolist will under produce water, charge too much.
We'll get to that in a minute.
But when there's natural monopolies, the goods should be delivered by a monopoly.
That's the right thing to do.
But there's a second reason why monopolies arise, which is a little more pernicious, which is government actions.
Sometimes monopolies are unnatural and created by the government.
So a classic example is when governments actually decide that they're going to deliver a good and only they'll deliver the good, like the postal service.
When a government decides look, we are just going to deliver this good.
We're just going to have the postal service.
And for many, many years, if you wanted to send a package from one place to another, you had to do it through the postal service.
Well, it turned out of course, that wasn't a natural monopoly.
As we know through the growth of FedEx and others, there was actually quite a good distance to compete in that market.
And eventually, those other folks delivering packages came in and competed in that market.
And in fact, in the US, it's very rare to have a government provided good, a purely government provided good.
It's very rare.
Historically, it wasn't.
And in many other parts of the world, it's not.
In many other parts of the world, governments run the banks, the airlines, all utilities.
So in many other parts of the world, governments will actually set up these monopolies through government action.
And basically, that is something which is not done much in the US but done elsewhere.
Now however, one place we do set up monopolies in the US, which is pretty important, is through the issuing of patents.
If you invent, say a new drug, and you patent it.
You then have the exclusive right to sell that chemical compound for 17 years.
So if you come up with some new chemical compound to treat whatever, you have the exclusive right to market that chemical compound for 17 years.
That is a government sanctioned monopoly.
Now, is that a good thing or a bad thing?
Are patents-- The government has just created a monopoly.
Could someone tell me why that might be actually a good thing for the government to do?
Yeah
Schumpeter talks about how it's a way to cause innovation.
Because no one would put that many dollars into drug research, if they weren't going to be able to make money.
Because the marginal cost was so low that other people-- The cost of establishing a pharmaceutical company [UNINTELLIGIBLE] is really low.
So other people would come and cut out all their profits that they had
Exactly, if you put all these resources into inventing a drug, it's not like laying pipes.
Once you invented it, someone could just copy it the next day and produce it at the same cost you can.
So why would you ever do that?
Why would you, as someone who, even if you cared about the benefits to the world, still want to make some money.
Why would you spend billions and millions of dollars inventing these new drugs?
If the next day, someone else can just say, that's a great idea.
I'll produce that for $0.47 a pill, just like you will.
And you won't make any money off it.
And you'll have lost all this money you invested.
So the argument for patents is you want an incentive for innovation, which suggests there's a trade-off here.
On the one hand, you want an incentive for innovation.
On the other hand, once you create this monopoly, they can charge outrageous prices.
There is a drug.
There are drugs.
Genzyme is very big company started here in Massachusetts.
And the whole reason they exist is to create drugs for very, very rare diseases.
Diseases that strike like 100,000 people a year but are deadly.
Like Gaucher's disease is a famous example of this one.
So they create these drugs.
They put billions of dollars into research, create these drugs.
And then, having created them and gotten a patent, charge like $100,000 a year to use the drugs.
Drugs that cost, you know, $500 a year to produce.
Now on the one hand, if these drugs didn't exist, people would die.
These are literally lifesaving drugs.
These are people who used to die, who now live because of these drugs.
And these drugs never would have been invented if Genzyme couldn't have made some money off them.
On the other hand, you've got a product which costs $500 a year to make, and they're charging $100,000 a year to people to use.
And in particular, when it comes to drugs like for treating AIDS in the developed world where people cannot-- There's no way they can afford that kind of money.
That's a problem.
Yeah
But aren't you not only paying for them to produce the drug but paying for all the research that went into--
Exactly, that's right.
You're paying for all the research.
That's why they never would have invented it if they couldn't.
So you're paying for the research.
They also take home a nice, hefty profit at the end of the day.
But that is true.
You are also paying for the research that goes into that.
And people wouldn't put that money up front.
That research is funded by investors.
And investors would not invest in Genzyme and give them money to do research if they weren't going to get a return.
So if Genzyme-- Of that $99,500 Genzyme makes on that Gaucher's disease drug, most of it is going back to pay the people who funded the research.
But a healthy chunk is not.
A healthy chunk is market power monopoly profit.
Yeah
Could the government then have subsidies for some types of licensing to ensure that the consumers are not affected because everyone cannot afford the drug without it
I'm going to come to that in a second.
Now actually, let me just hold that thought for a second because I want to actually show you how we think about whether patent is a good idea or not.
So let's go to figure 15-2.
I just want to, sort of, show you how what you guys seem to understand intuitively, we can illustrate in the kind of graphs we've been using.
Once again, we want to think about going back and forth between the intuition and the graphical and mathematical aspects of this stuff.
So here we have a monopolist.
Imagine that you have a good where the original demand curve is D1.
A good where the original demand curve is D1 and the original consumer surplus is that dotted area C plus 1 plus that slashed triangle.
So it's based on the area above the competitive price PC under the demand curve D1.
So you have a good.
It's some drug, which people like.
And there's some demand for it.
And it's produced competitively at a price PC with the consumer surplus of that triangle.
Now, let's say a monopolist comes along and says, we can make this drug way better.
We could make a much better drug, but we're going to need a patent for it.
Well, two things happen.
On the one hand, the demand curve shifts way out.
Now you've got something which people value a lot more.
They're much more willing to pay for this better drug.
OK?
And that's the benefit.
That's the innovation effect on demand.
On the other hand, the price goes up to the monopolist's price.
Where marginal revenue equals marginal cost, they're going to produce at Q sub m and charge a higher price P sub m. Whereas, if this is a perfectly competitive firm, they produce much more, where D2 hits the marginal cost and consumer surplus would be much larger.
So the trade off is, on the one hand, you end up here.
In this case I've drawn, you end up with a larger consumer surplus, even with monopoly.
But you could imagine drawing the same graph, in the case where consumer surplus falls.
You can imagine demand doesn't go up that much.
But the monopolist gets the monopoly price.
And you could imagine drawing this triangle such that the new consumer surplus triangle is smaller than the old consumer surplus triangle.
OK?
So it's not obvious which way this goes.
You have two effects.
On the one hand, you have the effect that demand shifts out.
That creates a potential for more social surplus.
On the other hand, you have the case that the monopolist artificially prices too high.
That causes a reduction in social surplus.
In this case, the former dominated the latter.
But I hope you can see you can draw this in cases where that wouldn't be true.
And that's the tricky thing with patents.
It's going to depend.
If you're patenting something which is really demand increasing, that's great.
But if you're patenting something which is just a copycat of something that exists and all you're doing is turning a competitive market-- In the limit, imagine I somehow snookered the government into giving me a patent for something which literally does no more than what's existed.
But somehow I could fool-- In that case, have to do a little more or people wouldn't buy it, just a little more than what existed.
It wouldn't increase demand much.
But I would then have these monopoly rents.
OK?
So that's basically the trade off with patents.
All right, question about that.
Now, let me move on then and talk about-- The question was raised about-- Well, couldn't the government do something about this, in terms of trying to address this problem.
And we just talked about the government as bad guy in creating this problem.
Although, it's not necessarily a problem in the case of patents.
But there's also the role of government as potential good guy, which is can the government help in this case of natural monopoly?
So natural monopoly, we've got this awkward situation where clearly a monopoly makes sense.
It doesn't make sense to have competitive people delivering water in town.
But once you've got one person delivering water, they'll charge you a fortune for it, well above marginal cost.
And that's inefficient.
That lowers social surplus relative to the optimal level of provision.
So can the government solve this problem?
Could the government, for example, come in and say look, we recognize there's a natural monopoly.
We recognize that you, Rubrico, are the only company that makes sense to deliver water.
But we are going to regulate you in a way to maximize the social surplus, given that natural monopoly.
So to see that, let's go to the highly complicated figure 15-3.
OK, we'll have to walk through this slowly.
It's a lot of stuff going on here at once.
OK?
15-3.
We're back to the usual curves we were using last time.
The demand curve is P equals 24 minus Q.
The cost curve is 12 plus Q squared.
So the marginal cost is just 2Q.
The competitive outcome was to produce eight units at a price of 16.
The monopoly outcome was to produce six units at a price of 18.
OK?
And you ended up with the monopolist causing a deadweight loss of C plus E. That was where we were last time in this example.
And let's say this was a natural monopoly.
So that's why there's a monopolist there.
Now, what if the government came in and mandated that the monopolist can charge no more than the competitive price?
Let's say the government knew what the competitive price was.
The government says, look, I took 14.01.
I see this graph.
I know the competitive price is 16.
So I'm going to say, monopolist-- You're the monopolist.
You go ahead.
And no one else can deliver water.
It makes sense for you to do it.
But I'm going to regulate that you cannot charge consumers a price above 16.
What you can see in that case, is the government will turn the monopolist from a price maker to a price taker.
And in doing so, the government will undo the poisoning effect we talked about last time and cause the monopolist to behave as if it was a competitive firm.
So to see that, we have to ask, well what's the marginal revenue curve now for the monopolist, given this government mandate that it can charge no more than 16?
Well, the marginal revenue curve, up to the point it sells six, is the old marginal revenue curve.
So we're working down this marginal revenue curve.
And up to that point, where marginal revenue intersects marginal cost, that's the marginal revenue curve.
And that's where the monopolist would stop.
But the problem is the monopolist isn't actually able to charge more than 16.
So now, if it's charging 16, now it asks itself well, should I produce that seventh unit.
Remember last time we did the math.
The monopolist should not produce the seventh unit.
Because last time we showed that, because of that poisoning effect, profits would fall if it produced that seventh unit.
But now it's not true because now it's only charging 16.
So if it's charging 16, and if it produces that seventh unit, it's still going to charge 16.
The poisoning effect has gone away.
There's no more poisoning effect.
There's no loss to producing the seventh unit because the most it can get is 16.
It would like to price it much higher.
It can't.
Since the most it can price at is 16 he says, well gee, then there's no poisoning effect.
I might as well produce at the competitive level.
And basically, what you can see is if you tell the monopolist they can't charge more than the competitive price, then they will produce at the competitive level.
And you'll end up at that competitive outcome, even with a monopolist, because that will be the profit maximizing thing for the monopolist to do.
So sounds like we've got a pretty good solution here.
Right?
If there's a natural monopoly, we just come in and say you can't charge more than the competitive price.
So in theory, government can fix this.
We've been talking about government as a bad guy.
Here's government as a good guy.
Government can come in and fix this.
What's the problem with that?
What's the problem, in practice, with carrying out what I just laid out here in theory?
Yeah
How does the government know what the competitive price is?
How does the government know what the competitive price is?
How does the government know what these curves look like?
Now, let's think about both curves.
OK, first the government needs to know-- If it wants to know the competitive price, it's got to know the demand curve and the supply curve.
Well the demand curve, in principle, the government could learn.
The government could go out and survey people and get a sense of kind of, could collect some data, could run some experiments and try to find out from people.
Gee, how much are you willing to pay for another unit, another unit, another unit?
So in principle, the government could actually measure demand curve by doing market research.
Although in practice, this is quite hard.
It turns out in practice-- The first rule of economics is believe what people do, not what they say.
Turns out, if you ask people what they're willing to pay, you get very silly answers, relative to actually do it.
So for example, this comes up a lot in when the government needs to value environmental damage.
So if we need to ask how much should we charge BP for the damage they did to the Gulf, we need to value how sad are people that the wetlands are ruined.
Well, the only thing to do is to ask people.
And it turns out when you do that, you get incredibly silly answers.
Things like, for example, if you ask people how sorry are you if 10 ducks died, they'll say I'm this sad.
Say how sad are you if a million ducks died?
They'll say, I'm the same amount sad.
OK, that doesn't make sense.
You ask people what's it worth to save a duck versus saving a whale.
And you ask in that order, they'll say saving a duck is worth $100, saving a whale is worth $100.
Or it's worth $150.
You ask the other order.
You say, how much is it worth to save a whale and then a duck.
Well then, saving a whale is worth $300.
And a duck's only worth $50, same question just a different order.
People do not give sensible answers when you ask them what things are worth.
The best way to know is to actually vary the price, is to actually run experiments where you actually say, OK, we have to actually give people different prices and see how they behave, actually trace out the demand curve.
That turns out to be very hard to do, especially for things like valuing damage to wetlands.
But even for things like valuing prescription drugs, you've got to literally-- It's hard to run the experiment where, literally, I give you a different price than you for a prescription drug.
That's a little bit tricky.
So it's hard to get the demand curve.
So that's the first problem the government faces.
But in some sense, that's the less difficult problem.
It is tracing out the demand curve.
In principle, there are ways to do it.
In principle, you could run experiments where you vary the price and you get that demand curve.
In practice, the difficulty is firm supply curves are much harder.
And why is that?
Because let's say the government comes to you and says, look I'm doing great.
I've nailed down the demand curve.
I just need to know your supply curve, so I can set your price.
What's your supply curve again?
OK, what are you going to say?
[INAUDIBLE]
Aw, this stuff costs a fortune to make.
You would not believe what it costs this stuff to make.
My supply curve-- 12 plus Q squared?
Ha!
That's a joke.
It's 30 plus 18Q squared.
Are you kidding?
My supply curve is crazy high.
And the problem is, the government doesn't know.
Because with consumers, at least I can run experiments to get the demand.
With the firm, I can't get inside their books.
I can't really know.
And if I get inside the books, they can cook their books and make it look like stuff costs more.
How can I really know?
Just like it's difficult for the board of directors to figure out whether the CEO of a company needs his own plane, it's very difficult for government regulators to figure out what it actually costs to produce these drugs.
And as a result, it's very hard to get this right.
And the problem is if the government gets it wrong, it can make it worse than not regulating at all.
So for example, if the government came in, and let's say, the government said look, I can't believe this firm.
12 plus Q squared sounds way too high for cost.
I think marginal costs are much lower than that.
And I think the competitive price should be 10 because I know the firm is going to lie to me.
I know they're going to tell me numbers too big.
So I'm going to say that equilibrium should be competitive, should be a price of 10.
And at a price of 10, the firm is now operating off the marginal revenue curve at that low price because it's below 16.
So at a price of 10, the firm is going to set marginal revenue equal to marginal cost.
Well, marginal revenue at a price of 10 is the price.
That's all they can charge is 10.
So they're going to set price equal to marginal cost.
The marginal cost with a production of 10-- The marginal cost is 2Q.
So set price equal to 2Q.
Or 10 equals 2Q.
Or Q equals five.
They'll produce five units.
They'll say look, if you're only going to let me charge 10, I'm only going to produce five units.
That's what I'll produce.
That's what's optimal for me at a price of 10.
What you see is-- Now if you look at this graph and you look at the firm producing five units.
What you see is the firm produces five units.
Now, the total surplus has fallen.
It's just A plus B plus D. The consumer surplus is very high.
We don't have the horizontal line of 10 here.
But draw yourself that horizontal line of 10, which is where the dashed line of five intersects the marginal cost curve.
You could see there's a big consumer surplus.
It's A plus B plus half of D and a little producer surplus, which is that triangle that's the lower half of D. But look what's happened to social surplus.
Before the government got involved, the deadweight loss was just C plus E. Now the government, by misregulating the price, has created deadweight losses of C plus E plus F plus G plus H. The government has just increased the deadweight loss a lot, by getting involved and misregulating.
And in particular, that area that increases it-- Here's the interesting thing.
Falling from eight to six caused that little triangle.
Falling from six to five, just one more unit, causes a trapezoid, which is probably bigger than that triangle.
Why?
Because we're moving farther and farther away from the optimum.
We're losing more and more trades that were socially beneficial.
Remember at the optimum, deadweight loss is very small.
But as you move farther and farther away, that deadweight loss gets bigger and bigger.
So this government misregulation, instead of moving us to what's optimal, by just moving us one below where we were as a monopolist has caused all this huge extra deadweight loss and made things worse.
Indeed, it could be worse.
Imagine the government chose a price so low the firm just said, forget it.
I'm going to shut down.
You could lose all surplus in this market.
If the government came in and said, no, we don't believe you.
The price should be whatever.
It wouldn't be, in this case, because in this case, price is always greater than marginal cost.
But let's say the government came in another case and set a price that was too low.
It could just cause the firm to shut down.
Then all social surplus in the market would disappear.
OK?
So basically, there's these difficulties with the regulation, which is it's hard to know how to set the price.
Now in practice, of course, what happens is governments set the price too high.
In practice, the government would much rather err on the side of letting monopolists make a little money than err on the side of there being too little of the good.
So government, facing uncertainty about what the right level to set is, will typically end up erring on the side of monopolists, partly because of the way of the nature of regulation.
So let's say you are president whatever.
And you want to regulate this new good.
OK, somebody made the new good and you want to regulate it.
Well who are you going to hire to be the regulator?
Well clearly, someone who knows something about the market.
So there's a new good.
My favorite example of an outdated technology is the umbrella.
I really think we've got to figure out a better way to do umbrellas.
They suck.
In the wind, they blow up.
They're useless, right.
There's got to be a better umbrella out there.
Let's say somebody finally figures out a better umbrella.
But they've got a patent on it.
And so you want to regulate it.
It's got some natural monopoly producing it.
It uses unobtanium.
And the only can find it in one place, Pandora.
So they've got to-- They're the only guys-- Isn't that the stupidest name ever for a material?
How could they have not come up with a better name than unobtanium?
Anyway, so it uses unobtanium.
And basically they have this natural monopoly.
So you want to regulate it Well, who are you going to find to regulate it?
Well you know, once there was an expert in making umbrellas with unobtanium.
That person who you're going to hire.
That's the right person to hire.
Unfortunately, that probably means they're probably friendly with all the guys that make umbrellas with unobtanium.
And they're not going to want to be too tough on their buddies.
So the natural person who's going to regulate, by their nature, is going to tend to be someone who's going to think sort of on the industry's side.
Moreover, what are they going to do after they're done being the regulator?
They're going to go back into the business of making umbrellas with unobtanium.
So they're not going to want to be too mean to the business because they're going to be making money off it five years hence.
So you've got this problem that regulators are going to have a natural tendency to be sort of friendly towards industry they're regulating.
And that's going to lead to an upward bias.
The question is, how much is that upward bias in the price relative to what the monopolist would actually do.
And much like a patent, it depends.
You can draw examples where the government regulation would improve things and examples where the government regulation would make things worse.
You can do examples either way.
And what, at least, is interesting here is this is the first example we've seen or one of the few examples we've seen where the government, at least, could possibly make things better.
Even though it's not clear they will.
At least, they could possibly make things better.
And that's because-- Why is that?
This is a very important insight.
Why is that?
That's because everything we've dealt with so far, the market does everything well.
We haven't needed a government.
What natural monopolies do is they say, wow, here's a case of what we call a market failure.
The market has failed to maximize social surplus.
And that opens the door for potential gains for government intervention.
Basically, as long as markets are functioning well in a competitive manner, there's no door open for the government to make things better.
The government can just make things worse.
It's only when there's market failures, of which natural monopoly is one example, can the government come in and potentially make things better.
It won't necessarily do so.
But at least there's a door open to doing so.
Questions about that-- OK, the last thing I want to talk about, today, is an example that's in between a natural monopoly and no natural monopoly, which is what we call a contestable market.
A contestable market-- A contestable market is one which says that there is a natural monopoly, but it's not so big that someone couldn't come in and compete if the profits got too large.
Or another way of saying this is, just because there's a monopoly doesn't mean there's a whole lot of market power.
Just because there's a monopoly doesn't mean there's a whole lot of market power.
So imagine a natural monopoly market with a very modest fixed cost, a fixed cost that's, you know, real but not enormous.
As long that monopolist who gets in first keeps their price near marginal cost, no will ever enter.
As long as they get their price near marginal cost, no one will enter, even if they're making a small profit.
No one will enter because no one could ever make money then and pay the fixed.
No one could ever come in, pay the fixed cost, and still make money.
So no one's ever going to enter.
But if that price ever got too far above marginal cost, then someone would say, ah, there's so much profit to be made.
That'll cover my fixed costs.
I'm coming in.
That's what we mean by contestable market, a market which can exist with a monopoly and can exist with someone making money.
And yet ultimately, there's sufficiently easy entry that, basically, the monopolist is forced to behave almost like a competitive firm.
So it's sort of like market pressure on monopolists.
It's a weird outcome where you get a monopoly market but pricing close to competitive levels because of this threat of entry.
Now the most famous example of this is in airlines and airline deregulation.
Until the 1970s, the way airlines worked in the US is there were private airline companies.
Many of them don't exist anymore, Eastern, Pan Am.
Ones you don't know about, don't exist anymore.
And they were regulated.
It was a regulated oligopoly.
We haven't really talked about oligopolies yet.
It was a few firms competing.
But think of it as a monopoly.
It's basically regulated monopoly.
In particular, they had monopolies on different routes.
So what happened is the government would say New York to Boston.
That is the route that Eastern Airlines owns.
And they fly that without competition.
But in order to make sure they don't rip off the consumer, we're going to regulate their prices.
So Eastern Airlines, you fly this without competition.
But we're going to regulate what you can charge the consumer.
And this was because the government viewed airlines as a natural monopoly.
It said, look, airplanes are really big and expensive.
There's huge fixed cost to becoming an airline.
So we think this is a natural monopoly.
And we're going to regulate it.
However, economists started pointing out, and experts started pointing out that no, in fact, this was actually a very contestable market.
That it turned out it wasn't that expensive to produce airplanes.
There were a lot of older airplanes you could refurbish, et cetera.
And that basically this is a market where, if the government opened it up to competition, you could actually get some-- That threat of entry eventually could drive prices down.
And so basically, after a lot of debate, the government, in the late 1970s, did deregulate airlines.
One of the most important changes in government policy in the 1970s was they deregulated airlines.
And what happened?
Well, three things happened.
First of all, prices fell enormously.
Prices fell by about a third.
Flying from point A to point B fell by a third, other places more extreme.
When I was an undergraduate at MIT, I could fly from Boston to Newark for $19 each way.
There was this airline called People's Express that was introduced in the wave of deregulation in the 1980s.
People's Express-- It was fascinating.
I don't know what their planes were held together by.
And the thing was, you didn't even buy a ticket.
You just went and waited on line.
They let you on the plane.
And when the plane was full, they took off.
And they made you pay on the plane.
I still, to this day, don't know what happened if you didn't have the money, if they like threw you off, get parachutes or something.
I don't know what they did.
But literally, they'd come down the middle of the aisle with a little credit card thing.
And they'd make you pay on the plane.
I mean literally, it was cheaper than taking a bus.
It was incredible, $19 each way.
So prices came way, way down.
The second effect was many more routes were offered to consumers.
Suddenly, routes which the government had said, no that's not profitable.
You shouldn't fly that.
New entrants said, no, in fact, it is profitable, government.
You just had the cost structure wrong.
And it is profitable to fly from Pittsburgh to Akron, or whatever, wherever they now fly.
There are hundreds more places you can fly now than you could in the 1970s.
Hundreds and hundreds of routes that just didn't exist because the government regulators assumed they weren't profitable.
But in fact, they were.
Once there was competition in these contestable markets, they did turn out to be profitable.
And the final thing that happened was that quality of airline travel deteriorated massively.
When I was a kid, and you flew on planes.
It was really nice.
I mean there was tons of leg room.
You got tons of free food, free movies, free drinks.
And I didn't do that when I was a kid.
My parents did.
Basically, it was an unbelievably nice experience.
It's not so nice now.
I think as any of you who have flown will attest to.
Now they're charging you for everything.
And you can barely fit your legs in if you're over 5'2".
Now, why did this happen?
Somebody tell me.
Why did this happen?
Why did deregulation lead from a world where flights were unbelievably comfy to a world where flights are uncomfy?
Yeah
It's more profitable to have more people taking it.
And--
Sure, but that was always true.
It was always more profitable to have more people on a plane.
It was always more profitable to not give them nice stuff.
That hasn't changed.
What has changed?
Yeah
You have-- there's competition.
So they need to reduce their cost more
What's that?
There's more competition.
So they need to reduce their cost
There is more competition.
But there was some competition before.
But you're right.
Your about halfway there.
What else?
Consumers aren't willing to pay for leg room and all that stuff
Basically before, if there was a route with two airlines on it-- A lot of them, I said there's two airlines.
And you wanted to compete.
How did you compete?
You couldn't compete on price.
Right?
If Eastern and Pan Am were flying New York to Boston, they couldn't compete on price.
The price was regulated.
So how did they compete, by being as nice as possible on everything else, even though consumers didn't really value it.
Well, once they competed on price, they said, look, consumers don't really care so much about this crappy airline food.
We're going to get rid of it and charge 20% less.
And lo and behold, they did.
What's ironic, of course, is everyone bitches about how bad airlines are.
But they don't bitch about how cheap they are.
It is so cheap to fly now, compared to when I was a kid.
Based on what happened is they went from competing on non-price factors to competing on price.
There's always competitive pressure.
There's always competitive pressure.
It's just before, the competition had to be on things which consumers didn't like.
But they liked it enough that you might as well compete on it.
Because it was not that consumers didn't like it all.
Obviously they liked it some, or they wouldn't have bothered.
Now they've moved to a more efficient form of competition, which is to compete on price rather than on quantity.
Yeah
When you said the government regulated the price, only wasn't that just on the upper level that you got that?
No.
The government just regulated the price.
Now, so this is a big victory for economists.
Yay for economists-- We did a great job.
But we whiffed on one thing.
We whiffed on one thing, which we did not foresee, which has led deregulation to be much less beneficial than we thought it was going to be.
What we whiffed on was the hub and spoke system.
We whiffed on the fact that, while building airplanes is contestable, building slots at airports is not.
That there's a limited number of slots at airports.
And it's incredibly hard to build a new airport because of environmental regulations and other things.
What that meant was there is still a natural monopoly in airport slots, even if there's not much of a natural monopoly in planes.
The result is that now what airlines do is funnel everyone through their hub and then out to the spokes.
And as a result, because of that they have developed new quasi monopolies.
So to fly from point A to point B, for many A's and B's, only one airline flies it.
Because it's economical for them because point A is their hub.
But airline B can't get into that hub.
They can't get a gate at that hub.
So as a result, they can't effectively fly from A to B.
So my wife is from Minneapolis.
We had to go to Minneapolis.
Northwest had a monopoly on going to Minneapolis.
So I could fly from Boston to LA for half the price of Boston to Minneapolis.
Why, because Northwest controlled all the gate slots in Minneapolis.
They had a natural monopoly on that resource.
And that's what economists missed.
So deregulation worked in some markets with a lot of competition.
Like the New York to LA market is very competitive.
It did not work in other markets, where there was this natural hub.
And so one airline could dominate.
Or a few airlines could dominate.
And that's why you could see some crazy pricing differentials.
That's why, for example, I could fly to Baltimore for $127.
But to DC, it cost me $500, even though they're less than an hour apart by car.
This is because the constraint on slots has limited the amount of competition that can go on.
And so that's sort of an example of where competition can work and where it can't to try to deal with these natural monopoly problems.
All right, let me stop there.
And we'll come back on Wednesday and talk about oligopoly.
Good luck tomorrow night on the exam.
It's tomorrow night?
Yeah, tomorrow night.
Good luck.
Everyone looked confused for a second.
Next week-- Oh-- OK, just joking, April fools, Halloween fools.
Good luck next week on the exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
Economics-- oligopoly.
Which is basically trying to move towards the most realistic modeling of markets that we can.
We've talked about two extreme versions of modeling markets.
One is perfect competition, which is the extreme case of perfect entry and exit.
Free entry and exit.
Perfect consumer information.
An idealized market.
We know that doesn't really exist in practice anywhere.
The second extreme was monopoly, which we do see in practice in some places.
In particular, when there's natural monopolies.
We do see that.
But that's still doesn't describe most markets.
Most markets are better described as oligopolies.
These are markets where there's more than one market player, yet where each firm is large enough to actually affect the price.
So an oligopoly market is where there'll be a small number of firms in the market with substantial barriers to entry from additional firms.
An oligopoly market where there's a small number of firms with enough barriers to entry that additional firms don't enter.
So the classic example of an oligopoly industry is the auto industry.
Here's a market with a small number of dominant players.
There's been some entry and exit over time, obviously, but it moves pretty slowly.
By and large it's a market where there's very limited entry.
And the question is, how do firms behave in this market?
Obviously it's not like perfect competition where they can lazily take a price out of the market and just produce based on that price.
But it's also not the same as monopoly where they can just get to set the price and not worry about what other people do.
They're in this in-between situation where they have price setting power.
They have some market power but in a context where they have to worry about competitors.
And so in this context there are two different ways firms can behave.
It's important to lay out to start.
There's two different ways for firms to behave.
They can behave cooperatively or non-cooperatively.
If they behave cooperatively we say that they form a cartel.
So our cartel is what happens when oligopolistic firms, when firms in an oligopolistic market behave cooperatively to determine the outcome.
We call that a cartel.
The classic example, of course, here being OPEC.
The Organization of Petroleum Exporting Countries, which is a cartel that drives the price of oil.
Those countries cooperate in how much oil they produce to move the price up or down according to what the group desires.
And what cartels do is essentially turn oligopolies into monopolies.
So what cartelization does, what a cooperative equilibrium does is essentially say, let's all get together and behave as if we're one big monopoly by cooperating.
And therefore, if you cooperate you can get all the wonderful things monopolies get: huge market power, huge markets et cetera.
But as we'll talk about next time, it turns out to be pretty hard to get a cooperative oligopoly.
There's lots of reasons why it might fall apart.
And that's why in most oligopolistic markets firms behave non cooperatively.
In most oligopolistic markets firms are behaving non-cooperatively.
They're competing with each other, not cooperating.
And that's what we're going to spend today analyzing is the case of non-cooperative oligopolies.
Yeah?
[INAUDIBLE]?
Depends on the context.
In the US, and I'll talk about this, in the US there's anti-trust legislation which can make it illegal in many contexts to cooperate.
Obviously OPEC is not subject to some world legislation.
But even in the US there can be implicit cartelization and implicit cooperation, and we'll talk about that.
It's a good question.
So technically you're right.
Technically it is illegal in the US in most contexts to form a cooperative, to form a cartel.
Whether in practice those laws can be enforced is an interesting, legitimate question.
So today we want to focus on the case of non-cooperative oligopolies.
And to do so we're going to turn to a new tool.
And one of the fundamental tools of economics in the last 30 years which is game theory.
So today we're going to talk about game theory.
Game theory is a tool of economics that was not really used early in economics but has come to dominate theoretical economics over the past 30 or 40 years.
And basically the way that game theory works is to think, literally, of oligopolistic firms as engaging in a game.
So when you play, don't want to say play Monopoly, that's confusing terms.
When you play Sorry!
or whatever with someone, or play some online game with someone, you're competing to win.
You're behaving non-cooperatively.
You're competing to win.
So basically what the insights of game theory are that all the tools we used strategically to make decisions in playing games can actually be used in modeling how firms compete non-cooperatively in oligopolistic market.
The key insight is that each firm will develop a strategy.
Just as when you're playing chess you have some strategy going in, firms develop a strategy.
And that based on that strategy they will determine their behavior.
And what is going to determine that behavior is going to be how firms' strategies combine to determine a market outcome.
When firms come in with different strategies, or when a bunch of firms with strategies come in to compete with each other, what determines the market outcome.
And what that's going to depend on is something we call an equilibrium concept.
Which is how do we measure-- essentially the equilibrium concept is in game theory terms, think about how do we determine when the game's over?
How do we determine when we've decided on the outcome of the market.
What's the equilibrium concept?
So when you're reading the rules of a new game the first thing you look for is how do you decide who wins.
That's kind of like what the equilibrium concept is.
It's what determines whether the game has ended.
What determines whether you've reached equilibrium.
Where you've reached a point where the market is stable and therefore the game has ended.
Not ended in the sense the firm shut down, but ended in the sense that you know what everybody's doing.
So it's not quite like winning or losing.
It's more just like what determines when you're at the point where that market is at equilibrium.
Now the most famous concept is due to John Nash, who many of you heard of from the movie and book A Beautiful Mind, and that's called the Nash Equilibrium.
A Nash equilibrium is the point at which no firm wants to change its strategy given what the other firms are doing.
I'm going to say that again.
A Nash equilibrium is the point where no firm wants to change its strategy given what the other firms are doing.
It's a little bit bizarre, but we'll work it out.
In other words, more formally, the idea is that holding constant, given the strategies all your competitors use there's nothing that I can do to raise my profits further.
Given the strategies all my competitors are playing, there's no strategy I can choose that will make me more profitable than the one I'm choosing.
And likewise for every player in the market that will turn out to be true.
So have players sitting around a game board, going around the circle.
Each player says, given what I know each of the rest you are doing, I'm doing the best thing I can.
And they go around the circle and everybody says, OK, I'm at that point.
You've reached a stable Nash equilibrium.
This was named, of course, for John Nash.
You all know the story of John Nash.
He was a famous, actually mathematician.
We use his tools in economics, but he was a mathematician.
Developed these incredible theories, and then developed schizophrenia, went crazy.
But not before he developed some of the most important concepts in both mathematics and economics.
The most important is the Nash equilibrium.
Now the best illustration we use of the Nash equilibrium is an example that we refer to as the prisoner's dilemma.
And many of you will be familiar with this from more popular reading you've done in economics.
But let me just go through it because it's important to understand it.
The prisoner's dilemma.
The prisoner's dilemma the title comes from the old way they used to make police movies.
Where the idea is you catch two guys at a crime.
You can't put them away unless one of them fesses up.
So what you do is you put them each a separate room.
And you say to the one, your buddy's cracked.
He's going to he's going to sell you down the river.
You better, I'm using all my '50s analogies, he's going to put you away for good.
But if you admit that he's guilty and he did the crime, we'll let you off with a light sentence.
Then they go to the other room and say the same thing to the other guy hoping they'll both rat on each other and they'll both get a sentence.
So basically the idea is, let's say that you walk into each room and you say to each person, look, we have enough evidence right here, and you show them, to send you each to prison for a year.
We would have enough evidence to send you each to prison for a year.
But we aren't sure about this other thing.
If you'll admit that your friend did this other thing, then he'll go for five years and you'll go free.
And then we go to the friend and say the same thing.
If you'll admit that your friend did the other thing he'll go five years and you'll go free.
But if they both admit then they both go for two years.
So if they both admit, they both go for two years.
Now the way to write this down, what we do here to explain this, is we write down what we call a payoff matrix.
So write down a payoff matrix.
So the idea is we have prisoner A here and prisoner B here.
And they each have an option.
They could remain silent or they can talk.
Silent or talk.
Now if they both remain silent, if they both say I would never rat out my friend, I'm happy to go to jail for a year rather than rat out my friend, then they each get one year in jail.
a equals 1, b equals 1.
However if prisoner A rats out his buddy and prisoner B chooses not to rat out his buddy, then A gets-- so I'm sorry, if prisoner A talks and prisoner B remains silent.
Prisoner A talks, prisoner B remains silent, then A gets zero but B get five years in prison.
On the other hand, if prisoner B rats out his friend, but a prisoner A is true and doesn't say anything, then prisoner A is going to get stuck with five years and prisoner B is going to get nothing.
And if they both rat each other out then they both get two years.
So people understand the payoff.
Are there questions about the set-up here?
This is complicated so got to make sure you understand the set-up.
Now, could someone tell me, if A and B could truthfully cooperate, what would be the optimal cooperative strategy?
Yeah
You both go for a year
Right, you'd both be silent.
So the optimal cooperative strategy is clear, which is to both be silent.
So if the cop said, you know what, we'll let you guys get together and discuss what you want to do first-- which the cops would never be stupid enough to do-- but if they did and the guys can trust each other, then that's the optimal cooperative strategy.
What we call optimal in the language of game theory, we call that the dominant strategy.
A dominant strategy is the best thing to do no matter what the other guy does is the dominant strategy.
So the dominant cooperative equilibrium strategy is to both stay quiet.
But what's the dominant, non-cooperative strategy?
What is the dominant non-cooperative strategy?
So the ways with dominant strategy, we run through.
Take prisoner A, ask the following.
The dominant strategy is, is there a strategy that makes him better off regardless of what B does.
Is there a strategy that makes A better off regardless of what B does.
Let's go through.
If A remains silent, if B remains silent he gets a year.
If B talks he gets five years.
Now compare that to prisoner A strategy of talking.
Well if he talks he's better off than if he doesn't talk, if B's remaining silent.
If he talks he's better off than if he doesn't talk if B talks.
That is regardless of what B does, he's better off talking.
Regardless of what B chooses to do, his dominant strategy is to talk.
Because if B is silent, he's better off if he talks.
If B talks he's better off if he talks.
So matter what B chooses to do, A is better off talking.
What about B?
Well B, by the same logic, is always better off talking as well.
No matter what A chooses to do, B is better off if he talks.
So where do we end up?
What ends up as the Nash equilibrium?
The Nash equilibrium is that both prisoners talk.
The dominant strategy for both prisoners is to talk.
And they both end up worse off than they could have if they could have cooperated.
So the dominant non-cooperative strategy is to both talk, even though if they could get together they'd be better both not talking.
And this is basically how game theory works.
Game theory math gets incredibly hard.
And if you're interested, 14.12 is one of our most popular undergraduate courses, game theory.
It's a great course where you take this and go run with it for a whole semester.
It gets very complicated mathematically.
But the basic idea in game theory is pretty straightforward, which is just ask, are there dominant strategies that can be played by each player.
If each player has a dominant strategy and those dominant strategies lead to a Nash equilibrium then you're done.
Here we're in a Nash equilibrium.
Why are we in a Nash Equilibrium?
Because given that A has talked, B's strategy, which is talking is the optimal thing to do.
Given that B has talked, A's strategy which is talking is the optimal thing to do.
So given what the other person's doing, each person is doing the right thing.
So you're in Nash equilibrium.
Given that B has chosen to talk, A is talking.
That's the profit maximizing thing to do.
Given that A is talking, B is talking.
That's the profit maximizing thing to do.
So given the strategy the other player's chosen that's an equilibrium.
Yeah
If they both talk and got 10 years would that be the Nash equilibrium
No if they both talk, the 10 years only happens if one talks and the other one doesn't
No, I mean like, [INAUDIBLE]
Oh, if they both got 10 years?
So let's ask that.
Let's change this.
Now let's just rework it.
So now what's A's choice?
Well A if he talks and B's silent, then he's better talking.
But if he talks and B talks, he's worse talking.
So then what he does depends on what B does.
What B does is going to depend on what A does.
And we can't obviously see the Nash equilibrium here.
Because there's no dominant strategy.
What you do depends on what the other person does.
So there's no dominant strategy.
Dominant strategies only occur if there's something that you should do no matter what the other person does.
So in this case were these are both 2, there is a dominant strategy, It's to talk.
If those are both 10 there's no longer a dominant strategy so we can't quickly get the Nash equilibrium.
It's a good question.
Now this is not just a cute example that you can use for prisoners, but actually explains firm behavior in many contexts.
So the best example I like to think of of this is to think about advertising.
So imagine if Coke and Pepsi, and imagine a world where Pepsi was as popular as Coke.
That should never happen.
Coke's way better.
But imagine that was that world.
So imagine a world where if there's no advertising then basically Pepsi and Coke would split the market 50-50.
Imagine that set up.
So if Pepsi and Coke could agree not to advertise they'd split the market 50-50.
However, it's going to turn out that while that may be the dominant cooperative outcome, that's not the dominant non-cooperative outcome because each firm is better off advertising if the other one doesn't, or regardless.
So for example, imagine the following payoffs matrix.
I'm just making this up.
But here you have Pepsi and here you have Coke.
And imagine the payoff matrix.
And the payoff matrix is if they don't advertise, so Pepsi can choose not to advertise, or it can advertise.
So if they both don't advertise Pepsi gets 8 and Coke gets 8.
I don't know what 8 is.
8 is billion dollars.
I'm just making up numbers here, doesn't matter.
So $8 billion each.
If they both do advertise then they still end up splitting the market.
Because basically they're just as good as each other.
So they both spend all the money advertising and just back up where they would have started, except they've wasted all this money in advertising.
So if they both advertise they each earn $3 billion instead of $8 billion.
That is, they each split the market.
They end up back where they would have started but they pissed away a bunch of money advertising along the way.
However if Coke advertised and Pepsi doesn't.
Then almost everybody sees Coke, Coke, Coke everywhere and is like, Pepsi?
Never heard of that.
Coke makes $13 billion and Pepsi loses $2 billion.
And likewise if Coke doesn't advertise but Pepsi does, people are like, I've never heard of Coke.
I'm going to drink Pepsi.
So Pepsi makes $13 billion and Coke makes minus $2 billion.
Once again I just made numbers here so it would work, but these aren't real world examples.
So once again we can see there is a dominant cooperative strategy, which is they both should agree let's not advertise.
But if they can cooperate, then in fact what's the Nash equilibrium?
Let's work it through.
And there's no shortcuts here.
You've just got to work this through.
Let's look for Pepsi.
Well Pepsi says if Coke doesn't advertise I'm better off advertising.
If Coke does advertise, I'm better off advertising.
So my dominant strategy is to advertise.
Coke says, well gee, if Pepsi doesn't advertise I'm better off advertising.
I make $13 billion instead of $8 billion.
If Pepsi does advertise I make $3 billion instead of negative $2 billion.
So I'm better off advertising too.
So my dominant strategy is to advertise.
So for both firms the dominant non-cooperative strategy is to advertise.
So they both advertise and you end up in this equilibrium.
It's an example of how a non-cooperative equilibrium can lead to what we call a race to the bottom.
You can think of it as a race to the bottom.
In other words, if they could cooperate they could just be better off.
But because they can't trust the other, there's this race to the bottom where they both end up worse off.
This is pretty striking.
I thought they brought this out well in A Beautiful Mind, both the movie and the book, which is that all we've learned about economics so far is that competition is good, right?
Competition is beneficial.
Well here's a case, where in fact, at least in the firm's perspective, competition is bad.
If they could just get together and cooperate they could make more money.
Now, in fact, this example is not so far-fetched.
I don't know when it started but it was during your lifetimes.
When you were young, hard liquors, scotch, bourbon, whisky, et cetera, did not advertise on television.
You never saw a Johnny Walker ad or anything on television until, I don't know when it changed maybe five or six years ago.
Maybe 10 years ago, I don't know.
But certainly in your lifetimes.
That was not by government regulation.
Many people thought there was a government regulation, they couldn't.
That was not.
That was a cooperative equilibrium where the makers of hard liquors got together and agreed not to advertise on TV.
And they said it was in the "public interest" yada yada yada, but that wasn't.
It was just they recognized the benefits of cooperating and not wasting money advertising on TV and competing with each other.
Well that broke down some number of years ago.
And now you see whiskey ads and scotch ads and other things on television.
And they've moved to this non-cooperative equilibrium where they're all losing money by having this advertising.
So that's in advertising.
For me, once again, this is a hard thing, what intuition works for you.
For me the intuition works best maybe from my scars from my dating life is thinking about personal decisions.
So imagine that there's some girl named Allison.
And Allison has a potential problem with her boyfriend.
They've had a fight and now she's deciding whether or not to make up or break up.
Now we can think of this just like Coke and Pepsi.
Allison is going to be thinking, well gee, if he wants to make up with me and I want to make up with him then we're both better off.
But if I want to make up with him and he doesn't want to make up with me, that makes me look like a total idiot.
Whereas if I break up with him preemptively, yes it would be sad if he wanted to stay with me, but at least if he wanted to break up with me I look better.
So she preemptively breaks up with him.
John, her boyfriend, thinking the same thing, behaves in exactly the same way.
So it could be that even though they both would be better off if they just said, look the honest truth is, we're both wrong, let's make up and we'll both be happy.
Because they're so afraid of being the one being dumped, they end up breaking up.
Now we all know of examples like this from life, where people stupidly if they could have just cooperated had a better outcome because they're so afraid of being left with the short end of the stick, because they don't cooperate you end up with the worst outcome for both.
That's an example of a non-cooperative equilibrium in real life.
Now in real life there is one aspect of oligopolistic non-cooperative equilibria that allow them to be enforced, however.
That allow you to overcome the prisoner's dilemma.
There's one thing that allows you to overcome the prisoner's dilemma and that's repeated games.
Repeated games can help you overcome the prisoner's dilemma.
Now let's take the Coke and Pepsi example and let's imagine that they're making the advertising decision every period.
Every period they have to make an independent advertising decision.
They can advertise or not advertise every period.
And imagine Coke says to Pepsi, I've got the following deal for you.
I commit to not advertise as long as you don't, but the minute you advertise I'm going to advertise forever.
So as long as you don't advertise I won't.
But if you ever run an ad, the bet's off and I'm never going to cooperate with you.
I'm going to advertise forever.
Let's think about Pepsi's choice in period one if Coke presents them with this deal.
Let's think about Pepsi's choice if Coke presents them with this deal.
Well one thing is they could say, ha ha, great Coke, good job trusting me.
I'm going to screw you in advertising period one.
So Pepsi could say, great, Coke's laid down arms in period one.
I'm going to go and advertise.
I'll make $13 billion in period one because Coke's wimped out.
And then I'll make $3 billion forever after, because forever after we're both going to advertise.
But at least I' beat them up that first period.
However, if Pepsi says, wait a second, if Coke's really right, honest, then I can get an equilibrium where we make $8 billion forever.
And certainly $8 billion forever a better deal than $13 billion one year and $3 billion forever.
So in fact if Coke's willing to live up to that promise then that is a good deal.
And actually we can turn this non-cooperative equilibrium into a cooperative equilibrium through the enforcement of a repeated game.
Through the fact that this game gets played over and over and over again you can enforce a cooperative equilibrium.
We can come back to relationships again.
If you're in a committed relationship and you know that if I say it's your fault over and over again, eventually person's going to leave.
Then people say gee, I'm willing to take some of the blame and not always say it's your fault and we have a fight.
Because I know that if I always say it's your fault ultimately that will break the relationship up and that makes me worse off then admitting it was my fault some of the time.
It's the same logic.
Yeah
Is the point of the game to make more than the other guy or to make as much as possible?
Make as much as possible.
That's a good point.
I presume that the point is to make as much as possible.
I'm ruling out cut off your nose to spite your face equilibria.
So in other words, I'm assuming the goals is to maximize profits not relevant mark position.
That's a good point.
I should have made that assumption clear.
It's the assumption we're always making whatever we confirm behavior is assuming it's profit maximization.
So repeated games can enforce a cooperative equilibria, essentially.
Even in a non-cooperative set up.
But it turns out this only works if the game goes on forever.
This only works if the game is going to go on forever.
So, for example, imagine that Pepsi knows that in 10 years the US government is going to ban the advertisement of sugared sodas.
The US government is going to finally say, look, people are too obese, no more advertising sugary sodas.
Pepsi knows this.
In 10 years that's going to happen.
Well Pepsi's thinking, gee, that means in year 10 I should advertise.
In that last year I should advertise because Coke can't punish me the next year because no one can advertise the next year.
So I know starting after 10 years we're both in this equilibrium because the government's going to enforce it.
So in ninth year, right beforehand, right before that government ban I should advertise and Coke can't get me the next period because they don't have the tool to punish me.
Well Coke, of course, knows Pepsi is going to behave that way.
So Coke says, wait a second, if Pepsi's going to behave that way then I better advertise in the ninth period too or I'm going to get hit with a minus $2 billion.
So I'm going to advertise in the ninth period too.
Well Pepsi says, look, if Coke's going to advertise in the ninth period for sure, I may as well advertise in the eight period because Coke's going to advertise in the ninth period for sure.
And by the same logic you can work it back that they'll both advertise right away and you'll end up with a non-cooperative equilibrium.
That is a repeated game that does not enforce the cooperative equilibrium.
Because by working that logic backwards if it ends in some realistic time frame, then you better off just breaking it now rather waiting for that period where you're the one who loses.
Yeah
[INAUDIBLE] what if Pepsi [INAUDIBLE]?
Exactly.
If they don't then you could imagine that if Pepsi knows this and Coke doesn't, then Pepsi's optimal strategy will be to cooperate to the next to last period then go ahead and screw Coke and then Coke will lose.
But presuming symmetric information, repeated games cannot enforce these equilibria if they end.
So this is just an incredibly quick introduction to the fun that is game theory.
We're going to go on now and do more rigorous versions of these models.
But this is just to get you excited about the tools you can do with game theory with fun examples.
Game theory is about taking these fun examples and thinking a lot harder about a lot.
What if there's asymmetric information?
What if the game ends long enough in the future that you're willing to be patient?
How far off in the future does the game have to end to still enforce the cooperate equilibrium?
What if there are three players?
What if players move in different orders?
What if one player goes first, the second player responds to that, is it different than if they go simultaneously?
These are all really interesting issues that are very relevant to the real world firm behavior that you learn about game theory.
So this is just to whet your appetite for that.
Having done that we're going to now turn, leave aside these more interesting dynamic issues and focus on a specific example of a non-cooperative oligopoly.
Because, once again, this is all fun intuitively, but we want you to be able to work through a problem.
And the way to work that through is we're going to have to move to a specific, simplified example.
And the example we're going to focus on is called the Cournot model.
The Cournot model of non-cooperative oligopoly.
The way we're going to do this here, is we're going to return to the example we had with the prisoner's dilemma.
But instead of just facing two choices, talk or not talk, we're going to talk about firms facing a whole continuum of choices.
Firms choosing how much they produce in a non-cooperative equilibria situation.
So, for example, let's take the example they use in the book.
Let's imagine there's two airlines that fly between New York and Chicago, American and United.
And let's imagine for simplicity those are the only two airlines.
Because of the hub and spoke system we talked about last time, let's say all the gates in Chicago that are available to come from New York are taken by two airlines-- United and American.
Those are your only two options flying New York to Chicago.
And the question we want to ask is, get in that world, how do United and American decide how many flights to run and what price to charge?
If they're monopolies we'd know.
If it was a perfect competition we'd know, but how do they decide this all oligopolistic.
The way we figure this out is by looking for the Nash equilibrium in this case, which we also call Cournot equilibrium.
Which is basically the quantity is chosen by each firm such that holding all other firms' quantities constant.
So each firm chooses quantity such that holding all other firm's quantities constant they are maximizing profits.
So I choose a quantity such that holding all the other firm's quantities constant I'm choosing a profit-maximizing quantity.
And if each firm can choose a quantity that makes the market function where this is met, then you're in Cournot equilibrium.
You're in Cournot equilibrium when each firm has decided, I'm happy.
It's the same as the Nash concept.
I'm happy with what I'm producing given what everybody else is producing.
If everybody feels that way then you're in a Nash equilibrium or a Cournot equilibrium.
So to see this, this is not immediately intuitive.
Let me just talk you through the steps of how you'd solve for this.
How do you actually solve for a Cournot equilibrium?
So basically what are the steps to solving?
Step one for solving a Cournot equilibrium is to create each firm's residual demand.
So compute residual demand.
We talked about residual demand curves earlier.
Which is, that's the demand for my firm given the quantity absorbed by other firms in the market.
In this case it's quantities absorbed by the one other firm in the market, but in general you do this with multiple players.
So first you calculate residual demand.
Then having computed your residual demand you develop a marginal revenue function.
You calculate your marginal revenue which will be a function of other firm's quantities.
So your residual demand will lead you to calculate a marginal revenue function.
It's a function of other firms' quantities.
You then do the same, do one and two for all firms.
So for each firm you end up with a marginal revenue function and a function of all the firm's quantities.
Step four is you have n equations and n unknowns and you solve.
So you develop a series of equation where each firm's marginal revenue function is a function of each other firms quantities.
You get one equation like that for each firm.
That leaves you n equations and n unknowns you solve.
If you can solve it then you reach equilibrium.
If you don't have a solution then there is no stable Nash equilibrium.
But if you can solve that there is a Cournot equilibrium and you solve for it.
So what we're going to do here is I'm going to illustrate this to you graphically today and we'll work through some more of the math of it next time.
So we'll start by doing this graphically.
So let's start by considering the case of American Airlines.
Let's start with figure 16-1.
Start by considering the case of American Airlines.
And let's say that the demand curve in this market in our example we're going to do, let's say that the demand curve is of the form p equals 339 minus q.
So there's 339,000 flights that are demanded each month.
Each month there are 339,000 flights demanded in the whole market.
So 339,000 people want to go from New York to Chicago every month.
And let's also assume the marginal cost is $147.
I don't know where Perloff came up with these numbers, but let's just go with them.
The specific numbers don't matter.
Now what would American Airlines do if it was a monopolist?
If American Airlines was a monopolist, it would set marginal revenues which are 339 minus 2q by the same math we did before.
You just multiply it through by q and then differentiate and you get 339 minus 2q equal to the marginal cost which is 147.
So if it was a monopolist it would choose a quantity of 96 and it would choose a price of $243.
A prices of $243 which we would just get out the demand curve.
If the quantity is 96, the price is $243.
And that's what we see here.
The marginal revenue curve intersects the marginal cost curve at a quantity of 96,000.
You then go up to the demand curve to read off the price.
Remember for a monopolist you've got to still respect the demand curve.
You get the demand curve to read off the price, that's $243.
That's what American would do if they were monopolist.
So if they were the only folks flying New York to Chicago, they fly 96,000 people a month at a price of $243,000.
However, now let's say American recognizes that United is in the market.
And let's say American recognizes that United is going to deliver some amount of flights q sub u.
They know American is going to do some amount of flights q sub u.
They don't quite know yet what it is, but they know there's going to be some amount of flights q sub u.
So the residual demand for American is q sub a equals total demand minus q sub u.
That's their residual demand.
So, for example, let's say that American just guesses that United will fly 64,000 passengers.
Let's say Americans says, look, I just know, I've got some corporate spy who's told me that United will fly 64,000 passengers.
So what you want to do is then you just re-solve the problem but using residual demand.
So then you say, well if United is going to fly 64,000 passengers then my residual demand is that price equals 339 minus the quantity I sell, q sub a, minus q sub u, which I think is 64,000, which is 64.
So my new residual demand is p equals 275 minus q sub a.
That's my new residual demand.
Because I thought United is going to sell 64,000.
So instead of my demand being 339 minus q, now 275 minus q sub a.
That's what's left.
So if I use this as my new demand function and re-solve, if this is my demand function, my marginal revenues are then 275 minus 2 qa.
My marginal cost is the same which is 147.
So instead of my equation being 339 minus 2q equals 147.
Now it's 275 minus 2qa equals 147.
If I do that I'm going to get a qa star of 64,000 flights.
I'm going to say, well look, if I was a monopolist I would have deliver 96,000 flights.
But given that United is delivering 64,000, that's it's going to be optimal for me to also deliver 64,000.
At 64,000 flights what's my price going to be?
Well my price is 275 minus qa.
So my price is going be to be $211.
If I think United is delivering 64,000 flights then I'm going to deliver 64,000 flights at a price of $211.
So that's basically how American would function.
Now what's strange about this is American doesn't know how many flights United is going to deliver.
There isn't such a thing.
In fact, it's not like there's not some rule which says we're going to go 64,000.
United is trying to figure this out too.
So, in fact, simultaneously to American making this decision, United is making the same decision.
And they're going through the exact same math.
They're saying, well gee, given how much American flies, how much should we fly?
They're going through the same math.
And in fact if we assume that both firms have to face the same marginal cost and the same demand curve, then in fact they're solving a symmetric problem.
They are also creating a residual demand function, but instead of being qa equals d minus qu, now United is making qu equals d minus qa.
They're making a parallel residual demand function and they are solving as well.
And both firms, therefore, are ending up with choices of quantities that depend on the other firm's quantity.
And in particular what they're developing is what we call a best response curve.
So figure 16-2, I skipped over figure 16-2.
This just illustrates what happens when American thinks United is committing 64,000.
Let's go through that in one second.
Get through it mathematically.
This is an example where American knows United is doing 64,000 flights.
So they say, well look, my residual demand is essentially this new line d super r. And that's what I choose on that new line.
So that creates a new marginal revenue curve, mr super r. That new marginal revenue curve intersects marginal cost at 64,000 and that's why I fly only 64,000 flights at a price of $211.
So you see here is one example of how given an amount United is flying, how American chooses how much to fly.
Questions about that graphic that ties the math I did here?
What figure 16-3 does is say, look, we can actually do this for a whole host of possible production levels by the other firm.
And we can develop what we call best response curves.
Best response curves are, given what the other firm does, what should I do.
So, for example, American's best response curve is given that-- so on the x-axis we have how many thousands of flights American's passengers are flying per quarter.
On the y-axis how many thousands of flights United passengers are flying per quarter.
So, for example, if American decides to fly zero flights then United should fly 96 flights, right?
Then United is a monopolist.
So that's the point on the y-axis, to 96,0 point on the curve.
With a 0 on the x-axis, 96 on the y-axis.
If American decides to fly 0.
United should fly 96.
That's the monopoly case we just solved.
If American decides to fly 64, United should fly 64.
That's the case we just solved as well.
Likewise, American's best response curve is this steeper line.
But it's the same thing.
If United flies 0, American should fly 96 then at 0 on the y-axis, 96 on the x-axis.
So if United flies 0 American should fly 96.
If United flies 64, American should fly 64.
So you can actually, literally trace out these curves asking at every single point, given what the other guy's doing, what should I do.
So we solved for two points on this curve.
We solved for the other guy producing zero point, which is you produce 96.
We solved for the other guy producing 64 point which is you produce 64.
That same math can be used to solve for every point on this curve.
Yeah?
[INAUDIBLE] 192 point
The 192 point is it's the question is the following.
At what point would American produce zero.
How much when United have to produce for American to produce zero.
Well they'd only produce zero if United was producing 192,000.
Only at that point would they actually say, forget it, we're just going to produce zero.
That's what the 192 point means.
Only if they knew United was producing all that much would they just drop out of the market.
So that's the 192 intersection.
A backwards way to read the curves.
But the bottom line, is essentially we can write these best response curves.
They're basically the quantity I'm going to produce given the quantity the other firm produces.
And the key thing is that these are symmetric in this example.
Since the costs are the same and they both face the same market demand then these curves are symmetric.
What that means is that these curves are having figured out one you can automatically draw the other.
A trick for solving these problems is that if you have a symmetric Cournot equilibrium you don't need to calculate the math to find each firm's best response curve.
Once you calculate one you know the other firm's just a complement of it.
So having calculated American's best response curve we could have automatically drawn the United best response curve as a complement of that.
The other key point is by drawing this diagram we can see the Cournot equilibrium.
Remember Cournot equilibrium.
Cournot equilibrium is where I'm happy with my quantity given what the other firm's doing.
Given what the other firm's doing I can't make any more money.
Once again, I don't care about market share, I just care about money.
So given what the other firm's doing I can't make any more money.
Well that happens at 64,000 each.
Because when American is producing 64,000, United is happy to produce 64,000.
That's their profit maximizing choice.
If United is producing 64,000 American's happy to produce 64,000.
That's their profit maximizing choice.
So that point of each producing 64,000 we are in a Nash or Cournot equilibrium.
Both firms are happy given what the other firm is doing.
Both firms are profit maximizing given what the other firm is doing.
Now basically, for example, another way to look at this is that you're only in equilibrium if you're on both firms' reaction curves.
So, for example, American might say, look the equilibrium I like is where I do 96,000 flights and you none.
So the equilibrium I like is on the x-axis the point 96, 0, where I do 96,000 flights and you United do none.
United says, however, no that's not optimal for me.
Because if you're doing that, then you're charging a price of $243.
So there's money to be made for me.
I can come in and start stealing some of your flights.
So that's not an equilibrium from my perspective.
Might be an equilibrium from your perspective.
You're delighted you're a monopolist.
But not from my perspective.
At that price I'll come in and start stealing some your business.
And I'm going to start stealing some of your business.
As I steal your business you are going to have to move up your best response curve because your residual demand is shrinking.
And you'll only reach equilibrium when you're both happy with the outcome.
If only one firm is happy with the outcome the other firm can always change its behavior, raise its price up or down or its quantity up or down to change the market share and change the outcome.
So equilibrium will only be when you're at both firms best response curves.
You'll only be at both firm's best response curves where they intersect.
Let's stop there I'm going to come back next time.
Jessica we should have the same handout next time.
Let's make sure this figure is in the handout next time as well.
And we'll come back and talk about this last figure and we'll do the math behind it as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
OK, so what we're going to do today is continue our discussion of oligopoly which we started last time.
I want to return to the example we were using last time to discuss the example of Cournot competition.
We started last time by talking about general game theory and the prisoner's dilemma and the concept of Nash equilibrium.
Then we turned to a specific example of a Cournot competition, the way we can specifically model two firms competing in a market.
And so if you recall the example from last time, we had demand curve of the form p equals 339 minus q where this is thousands of passengers per month.
And we had a marginal cost of $147.
And we have flat marginal cost, so average cost equals marginal cost equals 147, equals average cost.
So flat marginal cost.
And what we said, so therefore if the firm was a monopolist, if American Airlines say was a monopolist, they would set marginal revenue equal to marginal cost which would be 339 minus 2q equals 147.
And so the monopolist was going to set a quantity of 96.
That was the monopolist quantity with the price.
So the quantity of monopolist is 96 and the price of monopolist is $243.
So that was the monopoly case.
Something you should be very facile with for tomorrow night is solving those kinds of monopoly problems.
And then we said, OK, but what if, in fact, it recognizes that it's not a monopoly.
There's another firm in the market.
United is in the market as well.
Well in that case it has to consider its residual demand.
So in that case we said the qa, its residual demand, for American was equal to the total market q minus the quantity absorbed by United, qu.
And we showed last time what this leads to graphically.
And so we re-handed out figure 16-3 just to review.
The notion was that each firm develops a best response curve.
Each firm says, based on what the other firm's going to do, here is my best level of production.
Here's my profit maximizing level of production.
And each firm having developed a curve, there's an equilibrium where those curves intersect because at that point both firms are happy.
You've achieved your Nash equilibrium because at that point both firms are satisfied with the strategy they're playing given what the other firm's playing.
So that point of intersection, both firms' best response are consistent with each other.
And that's we developed last time graphically.
Intuitively, I think it follows from the same logic, the idea is we're playing this game and the game will only have a stable outcome if we're both satisfied with the outcome.
If we're not both satisfied we'll continue to change our behavior and it won't be stable.
And so what I want to do now is talk about how we solve for this mathematically.
So let's just think about the mathematics of solving for Cournot equilibrium.
So what's the mathematics now?
What's American Airlines's residual demand function?
Well their price, p sub a, is equal to the total demand, 339, minus what they supply minus what United supplies.
So the price in the market, they don't have separate prices, the price in the market is going to be 339 minus what each of them supplies.
So now American can't control q sub u, American has to take that as an outside given factor.
That's what they're responding to.
So American says, I need to optimize this with respect to what I can control which is q sub a.
So how does it do it?
Well, it computes marginal revenue.
Well marginal revenue is p times q sub a.
So that's 339qa minus qa squared minus quqa.
Just multiplying through.
That's their revenue function.
So marginal revenue, differentiating with respect to qa which is all they can control, is going to be 339 minus 2qa minus qu.
That's going to be their residual marginal revenue function.
And they're going to set this equal to marginal cost.
So I'm going to set marginal revenue equal to marginal cost.
So they're going to set this equal to 147 which is marginal cost.
And solving.
you get that qa equals 96 minus 1/2qu.
qa equals 96 minus 1/2qu.
Once again reviewing, what we're doing is we're saying we set up their residual demand function.
We compute what that implies in terms of revenues.
We differentiate with respect to their control variable, q sub a, to get a marginal revenue function.
You set that equal to marginal cost.
And you solve for their best response curve.
Essentially you just solve mathematically for the curve you see here graphically.
You go to the graph and you look at American's best response curve.
That is exactly this line.
When qu is 0, it intersects at 196.
When qu is 192, qa is 0.
So you can see we've just mathematically developed where this best response curve comes from.
We've just solved for that mathematically through the similar mathematics of monopoly profit maximization.
In this case we're essentially saying, imagine you're a monopoly except for the stuff produced by the other guy.
That's the way to think about it.
Imagine you're a monopoly except for what the other guy does and you get this best response function.
Now, since we set this problem up to be symmetric-- that is we said the marginal cost identical for both firms, and the market demand applies to both firms-- since we set this up to be symmetrical, then we know that we can also write a symmetrical best response function for United, which is q sub u equals 96 minus 1/2 q sub a.
We can solve for this but there is a shortcut.
If it's symmetrical problem they'll have symmetrical best response functions.
So we can just write the symmetrical best response function.
And then we can find the equilibrium.
Remember the steps I went through last time.
Now we have n equations and n unknowns.
We have two equations and two unknowns.
We can just solve by plugging one into the other.
And we get that qa star equals qu star equals 64.
Which is in fact where these curves intersect.
So that's how I found the intersection of these curves was solving mathematically for the optimal level of production given what the other guy's doing.
And the price, what's the price?
Well the price is 339 minus q sub a minus q sub u.
So it's 339 minus 128 or 211.
So you end up with each firm producing 64,000 units a price of $211.
And this is the Cournot equilibrium.
Because it's on both firms' best response function.
Both firms are maximizing profits.
And in maximizing profits they both have chosen the same point to produce and so you're done.
They're happy.
That's the Nash equilibrium.
Yeah
So in the last lecture was that game theory box.
And this would be the lower right hand box, correct?
That's a very good point, yes.
This is sort of like game theory but with a whole distribution.
But in some sense it's like at each point you've reached that point.
But I really wouldn't draw that parallel actually.
The parallel that's important from that is the equilibrium concept.
The reason we choose the prisoner's dilemma instead of choosing the equilibrium concept-- and the equilibrium concept is, you're in equilibrium when each firm's dominant strategy yields an equilibrium outcome.
Here we've just taken that intuition and solved mathematically for each firm's dominant strategy.
And that yields an equilibrium outcome
That top left box, there's no way-- we shouldn't think about that at all?
No, actually it's a great segue to what I'll talk about next which is a cooperative equilibrium.
So what we've solved for is we solved for a non-cooperative equilibrium.
Where they're not cooperating.
So it's like the prisoner's dilemma in the sense they end up in that lower corner.
But in some sense I think that parallel, I'm not sure how much that helps other than thinking about the strategies and about thinking that that's non-cooperative.
We want to turn to next what happens if they can cooperate.
But before I do so, two things.
First of all, these problems do not have to be symmetric.
I've just solved symmetric problem often, and you'll see symmetric problems, that makes the math easier.
But they don't have to be.
American and United can be different and you could solve the same math.
And in fact Perloff goes through an example like that.
Symmetry is not guaranteed in these problems, it's just a feature of the way I set this up.
So this question is a great segue to the next topic, which is what about cooperative equililbria.
Can't American and United get together and form a cartel?
Remember we talked in the prisoner's dilemma about how that would be the best outcome if the two persons could just trust each other.
Well here can't American and United just trust each other and form a cartel?
And what they do here mathematically is pretty easy.
They would just say, look, let's get together and pretend we're one firm.
Then let's act as that one firm monopoly would and split the profits.
It's the way a cartel could work.
They could say let's get together, we'll produce as if we're a monopoly and let's split the profits.
Well what would happen if they did that?
Well we know that the total monopoly production if they had been a monopoly is 96.
So that's a big Q.
The quantity for the market is 96 and the price if they're a monopoly is $243.
So what they did is they said, look, we'll each do 48 flights, for a total of 96 and we'll charge $243 and we'll agree to do that.
But what would happen?
Well their profits to each firm would be their 48 flights they do times the price they get minus the marginal cost.
Marginal cost is average cost here.
The price they get minus the marginal cost.
That would mean per firm the profits would be $4,608.
So each firm would make $4,608 in profits.
Actually that's four million.
This is in thousands, right.
So $4,608,000.
Let's compare that to the profits they made as Cournot competitors.
Once again comparing the upper box to the lower box, going back to the prisoner's dilemma.
I think that is actually useful to think about.
Let's compare that to what happened when they were Cournot competitors.
When they were Cournot they each took 64 flights.
They each did more flights when they were competing, but they only got to charge $211 with that same marginal cost.
So their profits as Cournot competitors were $4,096,000 each.
So by monopolizing and splitting the market they raised their profits by 12.5%, by an eighth.
Their profits went up by an eighth, by 12.5%, which is a major jump in profits by getting together and forming this cartel.
Just as the two prisoners were better off not ratting each other out, they were both better off not ratting each other out.
These guys are both better off getting together and forming a cartel and splitting the profits, same logic.
So why don't we see these cartels everywhere?
Why do we have American and United, why don't they just do this?
Well that's for two reasons.
Two reasons why we don't see these cartels everywhere.
The first reason, so why not cartels?
The first reason why we don't see cartels is there's a fundamental instability.
Cooperative equilibria are fundamentally unstable because any one party has an incentive to cheat.
Any one party in a cartel can make money by cheating because they stick the other firms in the cartel.
So let's just talk about this for a second.
Imagine what would happen if American decided to cheat.
Imagine they had this cartel, it's running along, they're each making their $4,608,000 per month profit.
They're all very happy.
Now imagine American says, wait a second, what if on the sly I increase my number flights from 48 to 50?
What if I just did that and did it in a way which wasn't immediately transparent to United?
So American takes its q sub a which was cranking along as 48 and it raised it to 50.
Well what happens?
Well if they do that the total quantity in the market instead of being 96 rises to 98.
And if it rises to 98 that means the price is going to fall, right?
This is not a price discriminating monopolist, this is a one price monopolist.
So if they want to sell 98 flights, the price is going to have to fall.
So the price is going to fall to $241 from the current level of $243.
Well you might say, gee American just shot itself in the foot.
Its just lowered the price by selling more flights.
But look what happens to American's profits.
American's profits are now 50 times 241 minus 147 which equals $4,700,000 which is higher than it was making ib a cartel.
American just increased its profits to more than they were making when they cartelized splitting the profits.
And what's happened to United?
So this is profits for American.
United's still doing 48 flights but the price has now fallen to $241.
So United's profits fall to $4512.
United's profits fall.
American's profits go up.
So basically American has stuck it to United by cheating.
It's raised its profits at the expense of United.
And why is that?
What's the intuition?
Well it goes back to our monopoly intuition.
Because when one firm cheats in a cartel it gets all the benefit of the extra quantity, but only feels part of the poisoning effect.
Remember when a monopolist increases its quantity.
It's got these two effects.
It sells more but poisons its previous sales.
Well here when American sells two more flights, it sells more, it's got all that effect like a monopolist would, but the poisoning effect is shared with United.
They don't feel as much pain from that price going down because they're sharing that pain with United.
They get all of the benefit from quantity going up and only part of the pain for price coming down.
And that's why it makes sense to cheat.
A monopolist would never cheat itself.
It'd make no sense for the monopolist suddenly to go from 96 to 98 and say, ha I've cheated myself because it would feel the entire pain of the poisoning effect.
But American doesn't.
That's shared with United.
So it actually makes sense to cheat because they get all the benefit of the more quantity and only part of the cost of the lower price.
And that's intuitively what's going on.
Questions about that?
Well knowing this, clearly United says, wait a second, if we form a cartel American will cheat.
So I'm going to cheat first and the whole thing falls apart.
And we get back to the repeated game intuition we talked about in our game theory lecture.
Where if they can commit to never cheat, if they can commit to punish each other if they cheat, you might be able to make this work in a repeated game, but only if the repeated game never ends.
So if there's either not a repeated game, or it's a repeated game with an end, then there's going to be incentive for someone to cheat.
Once there's an incentive for someone to cheat, the whole thing falls apart.
And that's why cartels are fundamentally unstable.
Now there's a second reason why we don't see cartels which economists would always put second.
We always put the economic stuff first and the legal stuff second.
The second is they're illegal, but we think that's less important as economists because we think folks are often smart enough to figure their way around laws.
Technically these are illegal.
In fact, in the late 1800s cartels were very common in the US.
They were called the trusts.
There were trusts in sugar, railroads, other areas.
This is how these robber barons made all their money-- the Rockefellers and the Vanderbilts and stuff-- was by building monopolies.
That's what made their money.
And then in the early 20th century Congress passed what was called anti-trust laws which explicitly forbid the creation of cartels or these kind of trusts.
So technically they're illegal.
In fact, there are lots of ways, and Perloff talks about this in his book, there are lots of ways that firms can implicitly cartelize.
There are ways that firms can say, look we'll get together not under the government's auspices and we'll agree that you raise your price and I'll do it two weeks later and look like we're not a cartel.
I'm just following you, but in fact we've agreed to do it.
So there is some implicit cartelization and basically that's why there's an anti-trust division exists that's quite large.
The Department of Justice is basically to prosecute exactly those cases when firms try to do that.
Now in fact, however, the government has a bit of blood on its hands in this respect, because sometimes the government actually promotes cartels even though it might not mean to do so.
And the best example was the 1981 voluntary export restraint policy, the Reagan administration.
1981 was the worst recession before this one.
Actually unemployment was higher than this one.
This is a very bad recession, but actually unemployment was higher in 1981.
It was well over 10%.
And a big place where unemployment was high was in the manufacturing sector, particularly auto manufacturing.
We were getting killed by Japan.
Basically this was the first wave of Japan really producing much better cars than the US.
And US car manufacturers were suffering enormously as a result.
So basically the Reagan administration was trying to figure out what to do about this.
One thing you could do, and we'll talk about in international trade in a few lectures, you could impose a quota and you could actually say to Japanese producers, you can't sell in America.
But Reagan was a Republican.
And Republicans are for free trade.
And he really couldn't do that politically.
Plus economists have often said that's a bad idea.
And we'll talk about why that's a bad idea.
What Reagan did instead was said to the Japanese auto manufacturers, look, I'm a charismatic guy.
I'm Ronald Reagan.
Why don't we get together in a room and I'll convince you to just export less to America.
Let's have a voluntary export restraint agreement.
Where I'm not going to put on a quota, because that would be bad, I'm going to actually suggest to you, I'm going to make you an offer you can't refuse that you just export less to the US.
It was basically a back door way of imposing a quota without looking as politically bad as actually imposing one.
And in fact, Japanese automakers agreed to this and reduced their sales of cars in the US.
Now why would they agree to this?
Now there's two explanations.
One is they could have thought if they didn't then a quota was coming.
They could have said, look, this is kind of an olive branch before they lower the hammer.
That's a really bad set of metaphors, but you get the idea.
That basically look, if we don't do this, then he's going to impose a quota.
The other is they could have said, thank you Ronald Reagan, you've just formed a cartel for us.
We can't stop from competing against each other.
We can't stop cheating.
We'd like to form a cartel but we can't.
But you've told us we have to cartelize because you've limited how much we can sell.
You've essentially said, look you can't sell more than this.
You have to agree to sell more than that.
You've essentially allowed us a framework where we can get together and implicitly cartelize in a way where it's hard to cheat because then we'd be violating the agreement with America.
So basically what Reagan gave the Japanese was a cartel enforcing device.
What American and United need is some way where American and United can monitor each other to make sure they don't cheat.
That's what Reagan gave the Japanese.
Said, I'm going to report on how many cars are selling in the US and you're going to agree to limit it to this.
And they're like, great, that gives us the tool we need to enforce the cartel we wanted in the first place.
Thank you very much.
They made a ton of money.
American consumers lost out.
And I'll talk more about international trade on why that happens.
But the bottom line is, this end up being just like a quota.
The bad thing about quotas and things like this is that essentially they raise prices and hurt American consumers.
And in this case they were bad for American consumers and very good for Japanese producers.
We essentially increased profits for Japanese producers which they used to develop better cars and then came in later and killed us in the car market anyway.
So it was dynamically a pretty terrible policy.
So that's the legal issues around cartels.
Now what I want to do now is step back for a second and say, we've now talked about three different kinds of market forms.
We've talked about perfect competition.
We've talked about monopoly.
And we've talked about oligopoly.
In particular, non-cooperative oligopoly which is a more interesting case since within cartels it's going to be hard to sustain.
Non cooperative oligopoly.
We've talked about three.
How do these compare?
So let's just do ourselves a little chart.
Let's consider the three cases.
We've got monopoly, oligopoly, and perfect competition, three cases.
Let's ask how they compare on quantity and on profits per firm.
Quantity produced in the market and profits per firm.
And let's do it for this American, United example.
Well we know the monopoly case, we told you that in the monopoly case 96 units get produced in the market.
And the profits per firm are $4,608,000.
In the oligopoly case that's the Cournot case, in the Cournot oligopoly case we said the total amount in the market was 64 per firm or 128,000.
And the profits per firm were lower.
They were $4,096,000 per firm.
Now what about in perfect competition?
What's the quantity that's going to be sold, what's the price going to be in perfect competition?
We haven't done perfect competition.
You can tell me.
You can just do that intuitively.
What's going to happen in this market if there's perfect competition what's this price going to be?
Somebody raise their hand and tell me.
Yeah in the back
[INAUDIBLE]
147, because price equals marginal cost in perfect competition.
So quantity would be 339 minus 147 or 192.
There will be 192 units sold.
And what will profits be?
Zero.
Zero profits in perfect competition.
With a flat marginal cost there will be zero profits with perfect competition.
So we can now compare this and we could say, an oligopoly case leads to less output and more profits than does perfect competition.
And a monopoly case leads to even less output and even more profits than does oligopoly.
And basically how do we think about welfare in this case.
Well to think about welfare you'd have to actually draw the diagrams to compute producer and consumer surplus.
What I could draw and show you is basically social welfare is highest in the perfect competition case, lowest in the monopoly case, and we talked a couple times about the deadweight loss of monopoly.
This is not a price discriminating monopolist remember.
This is a one price monopolist so there's deadweight loss.
And it turns out if you write down the graph that it's in between the oligopoly case.
Oligopolies cause some deadweight loss, it's worse than perfect competition but better then monopoly in terms of total welfare.
And in fact, the bottom line is, we can actually pretty much tell what happens to welfare.
A shortcut for thinking about welfare is to look at quantity.
Because remember what social welfare is about, what causes deadweight loss, is trades that aren't made.
Under perfect competition every trade that has a positive social surpluses is made.
So the closer the quantity produced comes to the perfect competition level, the smaller will be the deadweight loss, roughly speaking.
So it isn't always true, and you can find bizarre deviations, but a good rule of thumb is that welfare, the distortion of perfect competition will be proportional to how much quantity falls relative to perfect competition.
If quantity falls a lot relative to perfect competition, that's going to be a big loss in welfare.
If quantity is falling just a little, it's going to be a small loss in welfare.
And that goes back to that deadweight loss triangle.
About the fact that it's smallest right around the perfect combination equilibrium and gets bigger as you move further away.
So basically perfect combination is the best, monopoly is the worst, oligopoly would be somewhere in between.
This leads to the question.
I think you might look at this chart and ask yourself is well, gee, does this sort of say that the number of firms determines welfare?
You've got one firm, two firms, infinite firms so can we make a more general statement about the role the number of firms.
And it turns out the Cournot model we can.
Basically in the Cournot model, the more firms there are, the closer you get to perfect competition.
In the Cournot model as n increases you approach perfect competition.
And in particular, and the book does the math in this, you don't need to know the math, it's more the intuition.
But the book shows that the markup in a market, price minus marginal cost over price, in a Cournot market is equal to 1 over n times the elasticity of demand.
That is that we've said before that markup was inversely proportional to elasticity of demand.
I might have used a different letter, I might have used nu for that elasticity.
But this is elasticity of demand.
I forget what letter I used last time.
Same elasticity I've been using.
So basically, we said before we had this equation, right?
We said markup was inversely proportional to elasticity demand.
The more inelastic was the good, the higher market power the monopolist had.
Well here we now divide by n. So the point is, that for a given elasticity of demand the more firms that are competing in the oligopolistic setting, the closer and closer you're going to get to a zero mark-up with a Cournot equilibrium.
So it's just this intuition you see from this table.
This is one firm.
Then oligopoly can imagine a number of branches.
This is oligopoly with two firms.
You should be able to show yourself.
If there were three firms, which you can solve, that's three equations and three unknowns.
It's the same math we just did.
It's high school math.
You can do this if you just put in three firms, do three equations and three unknowns, you get an even higher quantity and so on.
So as the number of firms goes up you're going to move that quantity up towards the perfect competition level and profits will fall towards the zero that we get from perfect competition.
And this is consistent with our rough notion.
We talked about extremes, about perfect competition being lots of firms, the monopoly being one firm.
But this confirms the rough intuition of the more firms, the more competition.
That's a good solid intuition.
Now this raises the interesting question about should we ever allow the number of firms in a market to shrink.
Should we ever allow the number of firms in the market to shrink?
What we call that is we call that a merger.
You've all heard of mergers.
When two companies merge to form one company.
Well by the logic I just told you that's going to be bad.
I just said that the more firms you have, the higher welfare.
The more you get towards perfect competition.
And yet mergers happen all the time.
The government does not stop them.
The government typically investigates them if they're big mergers, the government would typically investigate them.
But by and large they often let them go through.
Why is that?
Why should the government not stop all mergers?
And in particular, if anyone for extra credit, can anyone tell me what concept we covered which would explain why it might make sense to allow firms to merge.
Yeah
The economy of scale
Economies of scale, exactly.
That's the term I was looking for.
It could be that you have two firms that are producing inefficiently relative to having combined production.
So you've got one plant that produces the left shoes and one plant that produces the right shoes.
And that's not a good example because they wouldn't really compete with each other.
But you know the point.
The point let's say you've got two firms competing with each other and they're both running a plant at half capacity.
There's two separate plants running half capacity, much more efficient to have them merge and have one plant run at full capacity.
And yes, if they merged they would then be a monopolist, but it's possible the cost efficiencies could be so high a monopolist with lower marginal cost might be better than oligopolist with higher marginal cost.
This is kind of like the patent example I did last lecture or the lecture before.
Where I talked about how a patent could be a good thing or a bad thing.
It depends on how much demand increases due to the patent.
It's the same notion here.
The tradeoff is on the one hand you get marginal cost down by allowing a merger, on the other hand you reduce the number of firms in the market by allowing a merger.
And there's that tradeoff.
You're going to raise each firm's mark-up ability but by lowering marginal cost you might actually in essence, might in the end lower the price.
And that's exactly what the Department of Justice does every day.
And this is what is the hundreds of economists that work for the Department of Justice do, they have quite an interesting job.
They have to go ahead and evaluate these mergers.
People want to merge and they have to say, well which effect is going to be larger.
How do we decide whether the savings from economies of scale exceed the cost and increase market power?
And that's just an incredibly hard thing to do.
That turns out to be an incredibly hard thing to do.
And part of the reason it's hard is exactly comes back to what I talked about when I talked about regulating monopolies.
Because the data is controlled by the market participants.
And so you can have necessarily good data on what it's going to do.
So a great example is hospitals.
Over the last 20 years there's been enormous mergers in the hospital sector.
Even here in Boston we've seen huge mergers of things like the Beth Israel and Deaconess hospitals merge.
Other hospitals merged.
These mergers were big and had to go before regulators.
And the hospitals rightly argue, look, there's huge economies of scale in hospital production.
Partly because hospitals are rarely full.
They have to have excess capacity in case somebody comes in injured.
They have to have an extra bed around.
They don't want people sleeping in the halls.
As a result, in a situation where you're rarely full, where you're constantly under capacity, there could be huge economies of scale from combining and using your combined space more effectively.
So pretty compelling argument.
And for that reason, pretty plausible.
Most hospital mergers were allowed through.
Very few hospital mergers were rejected.
Well what happened?
It turns out the hospitals just lied.
They kept exactly the same production process they had before the merger and just raised prices.
So in fact, Beth Israel and Deaconess each kept their building just like they had before the merger.
They just charged more.
Essentially they just cartelized.
But the government let it go through.
And what we've done, essentially, is we're fooled by the theory of these economies of scale, when in fact there was nothing to force the companies to necessarily do much to realize them.
We just allowed them to have market power.
So basically this is an incredibly tricky and difficult issue in deciding whether to allow mergers or not is basically evaluating whether in practice you'll see the economies of scale that justify the increased market power.
And that's why the Department of Justice has a hard job.
Questions about that?
One other point I want to make here is about the size the market is, I've talked about how mathematically the size of the market will lead to more competition.
There's also another reason, which is the bigger the market, the harder to enforce a cartel.
If there's two players in the market and one cheats, the other one has the first one whacked or something.
You know what's going on, you can keep an eye on each other, you know who's to blame.
When there's lots of players in the market it becomes-- so let's say there was 10 airlines flying, it becomes hard to figure out who's cheating.
Or more still, imagine the most important cartel in the world which is OPEC.
The Organization of Petroleum Exporting Countries.
This is an organization of 20 plus countries who control the world's oil supply.
But there's no good way of tracking who's selling what oil.
We just know the total amount of oil sold out there.
So if a country like Venezuela run by some nutty guy decides to cheat and throw a bunch of oil on the market, it's hard for the other members of OPEC to actually view that.
OPEC's efficiency has varied over time.
At times OPEC has not done such a good job of controlling the supply of oil in the world, at other times it has.
But basically the more players, the harder it becomes.
And as a good example, which is the cartel to produce mercury was an existing cartel.
Italy and Spain essentially had controlled the market to sell mercury.
They had a very well functioning cartel.
Other countries entered and the cartel broke down.
So we can see this.
And this becomes a big issue in thinking about the benefits of enforcing competition.
And becomes a big issue in government policy.
Here's another government policy issue which you may followed lately which is the rare earths issue.
You guys all know as scientists better than I, the importance of rare earths, but these are materials which are very important for putting in cell phone batteries and other things.
These are important new materials for the modern economy.
The US used to be the dominant producer of rare earths and then China turned out were able to produce it much more efficiently, so we just let China take it.
So now 97% of the world's rare earths are produced in China.
China's now started saying things like, Japan we're mad at you, we're not selling you rare earths, which is a huge, huge problem.
And basically we've allowed China to essentially monopolize this market.
Now we tend to think in economics it's a bad idea for the government to subsidize businesses.
The government should just let the economy work and decide what works.
But this may be a case where the government wants to say, no, we are going to actually subsidize rare earths production in America, to make sure on national security grounds we have enough in case China decides to shut us off.
And that's exactly the kind of issue the government has to face in terms of trying to decide whether or not to try to actively get in there and promote competition versus just letting the market work.
Questions about that?
Now I want to talk about one last thing before we stop, which is the fact that Cournot competition is not the only model of competition out there.
In fact it might not even be the first one that comes to mind.
In fact, if I said to you, imagine two firms competing to sell something, the first thing to come to your mind would probably not be best response functions.
If you say, well they just compete over the price.
And if there was really tough competition the price would just come down.
And the limit, if they're incredibly good competitors, the price would have to come down close to marginal cost.
Well that is a different model of competition that we call Bertrand or price competition.
Cournot competition is competition over quantities.
You get together and decide how much quantity to produce given what the other guy is producing and then the market gives you the price.
Bertrand competition is the flip of that, which is firms set their price and they produce whatever the market demands at that price.
Once again, Cournot competition is firms set their quantity and then they set the price reading it off a demand curve.
Bertrand competition is totally different.
They say, we're just going to compete over price and we'll just produce whatever quantity you want at that price.
We don't really even care what demand is.
We're just going to compete over price.
And the idea is that you have some marginal cost that firms will never go below in the long run.
And the question is how much will they compete down to that marginal cost.
In practice, what's crazy about Bertrand competition is it's possible with two firms you can get to the perfectly competitive outcome, with only two firms.
Because those two firms are competitive enough in setting the price, they can fight it all the way down to the marginal cost.
That is if I ever try set it to marginal cost plus $1, you'll come in and set it to marginal cost plus $0.99, because you can still make money.
Now I'll come and set it to marginal cost plus $0.98, because I can still make money.
And we'll compete it all the way down until we both are charging marginal cost plus a penny.
And if anyone tries to sell a higher price they can't sell the product because people will only buy the lower priced good.
So you could have a world with Bertrand competition where essentially firms compete over price.
And where unlike this case, oh I just covered it up, unlike our comparison where with two firms there's still lots of profits to be made.
In Bertrand competition it's possible that two firms are enough, just two firms are enough to get us to perfect competition.
You don't need n firms.
Two firms are enough.
And in some sense, if we thought about it, that's probably the first model we have in mind if we thought about firms competing is they compete over where to set the price.
And that's essentially what the Bertrand model is.
So two questions about that.
The first is, well jeez, which model do we use?
How do we know whether it's Bertrand competition or Cournot competition?
How do we know which model to use?
We've got a market with two firms in it, you tell us the models are going to give very different answers, how do we know which one to use?
And the answer is, basically we don't know which one to use for sure.
We can talk about the conditions under which quantity competition is more likely and under which price competition is more likely.
Basically quantity competition is going to be more likely when there's lags in the production process.
So that you can't flexibly say, here's my price, I'll produce as much as I want.
If an airline said, I'm charging this, I'll fly as many guys as I want, you can't do that.
You only have so many planes.
Airlines can't just set a price and then meet whatever demand comes out.
Airlines have to actually sent a quantity.
They have to say there's this many flights we can do.
So when there's lags in the production process it's going to be hard to do pure price competition.
It's going to be more like quantity competition.
That's going to be like airlines or cars.
On the other hand, take selling cereal at a supermarket.
The supermarket if it runs out of cereal the next day it can get a whole bunch more.
The supermarket can almost instantaneously replenish any supply it's out of.
So there, you're more likely to see same price competition between say cereals in the supermarket.
People can just compare very easily.
And if one's cheaper they'll just buy it up and if it runs out the supermarket will just reload the next day with that cereal.
Essentially it's pretty easy to meet quantity.
So this is not a very clean distinction.
But the bottom line is quantity competition is going to be more likely when it's a hard good to produce on demand.
And price competition more likely when it's an easy good to do that.
That's how you're going to think about which kind of competition makes the most sense in which context.
That's point one.
Point two, and the last point I want to make.
What can firms do to avoid Bertrand competition.
Let's say, for example, you're a cereal producer.
You're in a Bertrand market.
Well this is a shitty market to be in.
I just said two firms can drive your profits to zero.
What do you do?
What do cereal producers do?
Yeah
Differentiate
They product differentiate.
So the way to beat Bertrand competition is through product differentiation.
Once your products are different, then you can price above marginal cost.
Then they're not perfect substitutes anymore and you can price above marginal cost.
Have you ever asked yourself why the hell there's so many kinds of cereal.
I mean have you been to the supermarket?
It's crazy.
Why are there so many kinds of toothpaste and shampoo.
It's all the same crap.
Why?
Because it's product differentiation.
It's to avoid Bertrand competition.
Because once you can differentiate, you get market power.
Differentiation provides market power.
In the book, this is too detailed for the course, but in the book Perloff talks about this as monopolistic competition.
Which is sort of an odd oxymoron, right?
Monopolistic competition.
But these are models of differentiated products where essentially you can say, look I'm different enough that essentially I can charge a higher price than my competitor.
This comes back the concept of contestable markets we talked about.
The idea is that, look, if you're different you can charge a little bit higher price.
Once it's too high people are going to switch back to the other food.
But if you make it different enough, people will pay something.
The most famous example is Apple Cinnamon Cheerios.
Cheerios have been around forever.
And what happened was people bought Cheerios, it was very popular.
But then supermarkets started producing generic Cheerios.
And Cheerios are like pretty freaking easy to produce.
And it turned out they were identical.
And supermarkets charged like half as much for the generic Cheerios.
And Kellogg's, no Kellogg's?
Not Kellogg's, the other one.
General Mills.
General Mills was all like, any of you guys from Minnesota?
Mall of America has this cool General Mills thing in Minnesota, I remember it.
So General Mills said, well gee, we're getting killed here.
These generic Cheerios are going to wipe out our profits.
So they invented the Apple Cinnamon Cheerios.
And this was a product differentiation where basically they could still charge a high price for Apple Cinnamon Cheerios because it was different enough from the generic cheerios the supermarket was making.
And they could patent them.
They could say, hey, we've got a different thing we can patent our secret Apple Cinnamon formula.
So supermarkets now just can't copy us.
So for a while we get price greater than the marginal cost.
Or even if supermarkets can copy them, they at least get for a while price above marginal cost until supermarkets catch on and make the product.
So product differentiation is the way that firms fight Bertrand price competition.
So is this bad?
Once again we don't know.
It depends how much you like Apple Cinnamon Cheerios.
Just like patents, it depends on whether the new good they invented delivers sufficient extra consumer surplus that it's worth the fact that they can market prices on it.
In fact, one of the most famous studies in this field was done by my colleague Jerry Hausman, who actually measured the demand curve for Apple Cinnamon Cheerios and the cost of producing Apple Cinnamon Cheerios and found that the introduction of Apple Cinnamon Cheerios raised consumer welfare by $67 million a year.
That even though the prices were high because of product differentiation, people were so happy with this wonderful new taste, it shifted demand out so much that actually consumer welfare went up.
Even though this was a tool introduced by the companies to avoid price competition, which should make consumers better off.
So it's all about these trade-offs between how much does it make consumers better off versus how much more market power does it give the producers.
And product differentiation is another example of that.
Why don't we stop there and we'll come back.
Good luck tomorrow night and we'll come back on Wednesday and move on to factor markets.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
OK, so what we're going to do today is the last in what I'd say are the core set of lectures.
Our core set of lectures, we started with talking about the market.
We then moved on and talked about consumer theory and did a series of lectures on that.
Now we're doing producer theory.
This is the last in our series of lectures on producer theory and then basically we move on to topics.
So the remainder of the section we'll talk about things like international trade, uncertainty, equity and efficiency, asymmetric information in insurance markets.
We'll move on to in the last part of the course showing you how you can apply what we've learned in the basics to answer a bunch of interesting, real world questions.
So this is the last of our core basics lectures.
What we're going to do here is fit in something that's fallen through the cracks, which is we've talked about firms and their decisions about how much to produce.
And we've talked about the output side.
But we haven't talked about the input side at all.
How do firms decide what kind of the inputs to use and in what ratio to use et cetera.
We talked a bit about it.
We talked about isoquants and isocosts and doing that tradeoff between inputs.
We haven't really talked about the input markets themselves.
So firms go and they say, look I've done my isoquants and isocosts and I want 63 workers.
Well they've then got to go to a market for labor and hire those workers and how does that actually work.
So today what we want to focus on is the demand side of input markets.
That is, what's the actual market analysis by which a firm having maximized its profits and deciding how many workers it wants goes and actually finds those workers.
So we're going to do is talk about demand for factors.
In particular today we'll focus on the demand for labor.
Although the demand for capital, the analysis will be very similar.
But today we're going to focus on the demand for labor.
And what we're going to do is begin by focusing on the demand for labor in a competitive factor market.
So we're going to begin by talking about competitive factor markets.
What I mean by that is that a perfectly competitive factor market is one where, just as perfect competition and output markets means there's lots of sellers selling the same good, a perfectly competitive factor market means there's lots of sellers, in this case workers, selling an identical good.
That is their labor.
So the notion is we're in a market where there's many, many workers firms could hire, all of whom are equally qualified for a job.
So this is not, obviously, a high-tech market.
This is some low-tech, construction, other sort of blue collar market, where there's lots of workers out there who could equally well be qualified for a job.
In fact what we're going to assume is that there's a perfectly flat labor supply curve.
Let's assume a perfectly flat labor supply curve.
Perfectly elastic labor supply just to make life easy.
Obviously it's not true in reality.
Let's assume we're looking at some market with perfectly elastic labor supply.
Now how do we think about what happens in factor markets in that world?
Well once again let's start with the short run.
So in the short run capital's fixed.
So a firm has said, look, I've done my short run profit maximization, my isoquants and isocosts.
I've decided how many workers I want given a fixed level of capital.
And that gives me some demand for labor.
I can derive a demand for labor curve by essentially saying, at different wage rates, given a fixed capital price, that will shift my isocost curve, going back to the producer theory, at different wage rates that will shift my isocost curve, that will cause me to demand different amounts of labor.
So that traces out a demand curve for labor.
And we can see that graphically in figure 18-1.
So you've got a perfectly elastic labor supply curve and then you've got a downward sloping labor demand curve.
And that labor demand curve comes from the profit maximization.
The other way to think about how we get there though is interesting.
You say, how do we think about the marginal benefit versus the marginal cost of hiring another worker.
We know the marginal cost is the wage, that's easy.
What's the marginal benefit of hiring another worker?
Well recall that another unit of labor raises output by the marginal product of labor.
Remember the marginal product of labor.
We talked about this a while ago.
This is delta q delta l. So the next worker raises your output by an amount marginal product of labor.
That's what you get from the next worker.
But that's a quantity.
The firm cares about profit not quantity.
So what it cares about is revenues.
So the revenues from the next worker would be the marginal revenues that are made on that next unit times the marginal product of labor.
That's the marginal benefit to the firm of the next unit of labor is the marginal product that that worker produces times the marginal revenue the firm raises from selling that next unit.
So they have to consider two things, two margins.
How much is it worth them to sell that next unit and how much will be produced by that next unit of worker?
So if we have a perfectly competitive labor market, the marginal cost is going to be the wage.
So we're going to set this equal to the wage.
So in a perfectly competitive labor market the marginal cost of another worker is the wage and this is the marginal benefit of another worker.
So this is going to be the condition the firm's going to use to decide how many workers to hire.
We're going to call this the marginal revenue product of labor.
So you're going to set the marginal revenue product of labor equal to the wage.
That's going to be your profit maximizing condition-- the marginal revenue product.
The marginal product is about quantities.
Marginal revenue product is about dollars.
What's the dollars that the next unit of labor produces for you is your marginal revenue product.
Now if the output market is also perfectly competitive.
That is, imagine for a minute now, take one further step.
Not a perfectly competitive labor supply but also a perfectly competitive output market.
So it's not a monopolist.
It's selling in a perfectly competitive output market.
Then we know what the marginal revenue is.
We know the marginal revenue is the price.
So for a perfectly competitive output market we could rewrite this as price times marginal product of labor equals the wage.
Because we know the marginal revenue is the price in a perfectly competitive output market.
And basically this makes it even easier to see which is to say, look, how many workers do you hire?
You hire until the wage you pay that worker is equal to the price you sell your good for times how many goods that worker produces.
If one worker produces 100 goods and each good sells for 100, then you'll only pay the worker $10,000.
Basically that is going to determine your labor demand curve.
And so the labor demand curve is also labeled the marginal revenue product of labor curve, I'm sorry.
Why is this diminishing?
Why is it downward sloping?
Because remember the marginal product of labor's diminishing.
The marginal product of labor's diminishing.
As a result this curve is downward sloping.
Now here price is fixed, so it doesn't matter.
Marginal revenue also is diminishing so that's going to make it even more downward sloping than the more general case.
But in this specific case of perfectly competitive output markets where this is just a constant price you get this downward sloping marginal revenue product of labor curve because the marginal product of labor is diminishing, as we talked about.
So that's the analysis of what we see for a perfectly competitive factor market.
So the equilibrium is where the labor supply curve, which is perfectly elastic, intersects this marginal revenue product curve, which is determined by how productive the workers are and how much money they're making for you with each unit they produce.
And that gets you the short run equilibrium.
Questions about that?
Questions about what we're doing here?
So that just says the underlying analysis of where demand for labor comes from or where the equilibrium level of labor is going to come.
It's going to come from intersection of this demand with the supply, which is flat.
Now how is this going to differ in the long run?
Well let me ask the question this way, forget the math, forget the graphs, I'm just going to ask you intuitively.
In the long run, do we think the long run demand for workers will be more elastic or less elastic than the short run demand, in general.
Will the long run demand curve, this is short run demand curve, will the long run demand curve typically be more elastic or less elastic than the short run demand curve for workers?
Somebody take a guess.
Yeah
More elastic
More elastic.
Why?
It's more elastic.
But intuitively, don't worry about the graph, I'm just looking for intuition.
Yeah
Because in the long run you can substitute capital
Exactly.
More substitutability equals more elasticity.
General intuition you want to remember for this course.
In the long run if I can substitute towards capital, then that long run demand curve for labor will be even more elastic than the short run demand curve.
I'm not going to work through the math or anything.
I just want you to remember that intuition that when you have more margins you can use, that's more substitutability, that means more elasticity.
So the idea is in the short run, if the wage increases for workers all you can do is if you hire fewer workers you just produce less, so you're sort of stuck.
But in the long run if the wage decreases you just say, fine, I'll just use machines instead.
So in the long run you can substitute away from workers towards machines.
So in the long run your demand curve is going to be more elastic.
Questions about that?
Now this is all relatively straightforward, just follows from producer theory.
The sort of stuff you had to do in the exam last night.
Let's now talk about a little more interesting case which is one of my other favorite words in economics which is the case of monopsony.
Not monopoly, but monopsony.
We've been talking in lectures about monopoly which is the case where one firm is the only seller in the output market.
One firm is the only seller in the output market.
A parallel case in input markets is the case of monopsony which is when one firm is the only demander in the input market.
So monopoly is when one firm is the only seller in the output market.
The parallel in input markets is monopsony which is where one firm is the only buyer in the input market.
And the key thing that's going to drive monopsony is that when there are barriers to exit from a factor market it's going to create a monopsony.
And any time there are barriers to exit, any time workers are stuck and cannot leave a market, workers are stuck working one place, that will give the employer market power over those workers.
The classic example is the company town.
In the 1800s when there were mining operations and they'd come in and they'd hire people.
And basically there was nowhere else to work in the area.
These were areas which were dying agricultural areas there was nowhere else to work.
You'd go work for the mining company.
That mining company had tremendous monopsony power over you because basically there was nowhere else to work within a decent area.
There weren't cars yet.
You couldn't just commute to somewhere else.
A modern example is MIT's monopsony power over me.
MIT has monopsony power over me.
Why is that?
Well that's because my life's pretty comfortable now.
I'm in a house I've lived in a long time.
My kids are in schools I like.
I've got a lot of friends.
It'd be a real pain in the ass for me to move.
And as a result MIT has some, not unlimited, don't tell them, but they have some monopsony power over me because they know that that's a barrier to my exit.
To my exiting MIT a barrier is that I'm very comfortable and satisfied.
And given the nature tenets of psychology, it is harder to move someone with a pull than with the push.
So someone could come and try to pull me away, but they're going to have to blow me away with an offer-- once again, don't tell MIT this-- they'll have to blow me away an offer because I'm pretty satisfied.
And that satisfaction inherently gives MIT some monopsony power over me.
Now the question is what implications does this have.
And the implications are quite interesting.
And what they are is a complicated flip of the monopoly case.
Let's look at figure 18-2.
Here's an example of a company town.
Now let's imagine a labor market where you've got some labor demand curve, the marginal revenue product of labor, MRPL, and some labor supply curve.
And now labor supply is upward sloping.
This is a not a perfectly competitive labor market now.
This labor supply is now upward sloping.
So the demand curve for labor we're going to say is 60 minus l. So in our example the marginal revenue product of labor is going to be 60 minus l. The notion is you have some downward sloping demand curve for labor because of diminishing marginal product.
The wage you're willing to pay is diminishing in the number of workers.
To get that first worker you'll pay a high wage, because you got to produce something.
But as you hire more workers the wage you're willing to pay falls.
So the demand curve is downward sloping.
And let's say that our labor supply curve is that the amount of labor workers are willing to supply is the wage over 2.
I'm just making this up.
These are just made up numbers just to make the math work.
So this is it just an upward sloping labor supply curve.
A higher wage calls forth more labor.
So let's ask what happens in the competitive case.
Well in the competitive case you set the supply equal to the demand.
Well supply is that firms are going to set the wage equals 60 minus l. Labor supply is l equals w over 2.
So we could just have two equations and two unknowns we can solve and get that the amount of labor supplied in the competitive case is 20 units.
20 hours, 20 days, weeks, whatever.
20 units.
That's the competitive outcome.
And the wage we can read off the labor supply curve.
If they're going to supply 20 units, they're going to need a wage of 40.
So once again we can read that off the labor supply or the labor demand curve if there's going to be 20 workers the wage is going to be 40.
So the wage, labor competitive and the wage competitive is 40.
So in a competitive market with this demand and supply curve you should know by now you just set them equal, you solve, you get an outcome of 20 workers working a wage of 40.
Now let's imagine this isn't the competitive case.
Let's imagine it's the monopsony case.
Let's imagine this firm has monopsony power.
Workers can't exit.
And let's further assume that the firm cannot wage discriminate.
Just as we talked about monopolists that couldn't price discriminate, we're going to talk about a firm they can't wage discriminate.
It has to pay one wage to all of its workers.
And we'll come back once again with this assumption.
Just as we talked about price discrimination we'll come back and talk about wage discrimination.
But for now assume a non-wage discriminating monopsonist.
They have to pay one wage to all their workers.
Well what that means is just as when a monopolist wanted to sell more goods it had to lower the price, if a monopsonist wants to hire more workers it has to raise the wage.
Parallel thing.
Just as the monopolist had to lower the price to sell more units because it had to respect the demand curve, a monopsonist if it wants to hire more workers has to raise the wage because it has to respect the supply curve-- parallel.
And what this will do is that will lead them to under hire workers ct too low a wage.
Just as monopoly led firms to under produce at too high a price, monopsony will lead to under hiring at too high a wage.
So once again, to think about this, let's think about the firm's decision to hire an extra worker.
What is the firm's total expenditure on labor?
Its expenditure on labor is the wage, which is a function of the amount of labor, times the amount of labor.
That's the expenditure on labor.
So its marginal expenditure, if you take the derivative, is going to be w plus dw/dl times l. That's its marginal expenditure.
If you want to hire an additional worker what's the marginal cost.
Well I want to hire an additional worker, what's the cost?
I've got a pay him w and to hire him I have to raise the wage, so I have to pay all my previous workers more as well.
So to hire one more worker I've got to pay that worker a wage and in order to entice him I've got to raise the wage, which means I've got to pay a higher wage to all my previous workers too.
Once again, remember, I have to pay one wage.
So if I'm going to hire that worker there is the same poisoning effect that we saw with monopolists.
With monopolists the poisoning effect was if I want to sell one more unit I'm going to have to undercut my price on all previous units.
For a monopsonist, if I want to hire one more worker I have to pay all my previous workers more.
And that's going to mean that there's a poisoning effective in reverse.
That's going to cost me a lot of money to hire that extra worker.
So we can actually derive now a marginal expenditure curve just as we derived a marginal revenue curve for the monopolist.
The marginal expenditure curve.
So we know expenditure is w of l times l. And we know from the supply curve, we can rewrite this as w equals 2l.
So that says that the expenditure on labor is 2l times l. So plugging in from the supply curve the expenditure on labor is 2l times l. Its wage is a function of labor times labor.
So 2l times l. So that means that marginal expenditure is 4l.
The marginal expenditure is 4l.
So we can now draw a marginal expenditure curve that's steeper than the labor supply curve.
Once again it's confusing but it's all parallel.
It's just we're flipping everything around from monopolist.
Instead of drawing that marginal revenue curve that was steeper than the product demand curve, now we're drawing a marginal expenditure curve which is steeper than the labor supply curve.
And once again to find the outcome we'll find the intersection of that marginal expenditure curve with marginal cost.
Well here we'll find the intersection of marginal expenditure curve, I'm sorry, with labor demand.
I'm sorry, so the parallel was we found the intersection of marginal revenue with marginal cost.
Now find the intersection of marginal expenditure with labor demand.
That intersection is going to happen at 12 workers.
So the firm is going to hire 12 workers.
But what ways you're going to pay-- once again our first temptation is to look at that high intersection and say, well at that intersection what's the wage.
We didn't put that on the diagram for a reason.
Maybe we should have to throw you off.
Remember, to figure out the wage you've got to respect the labor supply curve.
Just like you have to respect the product demand curve to figure out the price.
So what's the wage when I hire 12 workers they pay 24.
So the monopsonist is going to hire 12 workers and pay 24.
It's going to hire the number of workers where marginal expenditure equals the labor demand.
That's going to determine the quantity.
And the wage they're going to read off the labor supply curve.
And as a result this firm is going to under hire at too low a wage relative to the perfect competition.
Relative to the competition they're going to hire fewer workers at a lower wage.
So if you think about this, let's come back to the example of MIT.
MIT, say, would like to expand the economics department.
But to do so it's got to poach an economics professor away from another university.
Well, it poaches another professor away from another university.
And let's say that at MIT pays all its professors the same.
If they're going to poach a professor from another university they're going to have to pay the rest of us more.
And that's going to cost them a ton of money.
So they think think we'd rather have a little bit more crowded undergrad class, we don't really see it here today, but maybe in general, more crowded undergrad class and not get that extra professor to avoid having to pay a higher wage to all our existing professors.
So as a result MIT will under hire professors and they'll under hire professors at too low a wage.
And once you can determine how big monopsony power is, what determined how big monopoly power was?
What was the key thing I don't want to say what it is because it will give it away, what's the key thing that determined the size of monopoly power.
Yeah
Elasticity demand
Elasticity of demand.
So just like that, the key factors going to determine the market power of monopsonist is going to be the elasticity of supply of labor.
The elasticity of demand for a good is what determined the market power of monopolist because as goods were more elasticity demanded, they had less ability to jack up the price.
The elasticity of supply of labor is what's going to determine the market power of monopsonists.
Because if I have more options.
they can't underpay me.
So if I'm very willing to move to another university.
That is as my labor supply curve gets flatter, then there's less market power that they have.
In particular, in a perfectly competitive labor market there's no monopsony power at all.
So monopsony power is a function of the options facing their workers.
Questions about that?
Now the key question this all raises, at least when I first learned about it and think about it, is it maybe was plausible to think that monopolists can only charge one price for their good.
That the iPod is what the iPod costs.
And you couldn't start charging different amounts for iPods to different people.
That would get bad press and stuff.
But it's seems a little bit stranger to think that employers have to pay one wage to all their workers.
As a matter of fact we know that MIT doesn't pay the same to all its professors.
There is some wage discrimination.
In particular, it's a well known fact that the way to get a raise as a professor is to get an offer from another university.
Because MIT, the pay structure professors at competitive departments like economics is typically they will underpay you until the university comes and says you're worth more, and then they'll ratchet up to try to match them.
So there is wage discrimination in practice at universities as there is in most workplaces.
There's very few workplaces where all workers make the same.
There's wage discrimination.
So does that mean this model is irrelevant?
And the answer is, no, it doesn't.
Because there are still major barriers to perfect wage discrimination.
There's some wage discrimination, but there's a lot of barriers to perfect wage discrimination.
The most important one is workplace norms or fairness.
So what MIT should do, here's the MIT optimal strategy.
The older the professor, the less they should pay them.
Not because they're less productive.
Marginal productivity is constant, it's not, we get less productive as we get older, but put that aside.
It's because the older you are, the less likely you are to get up and move.
And the less likely, quite frankly, other universities are going to want to hire you and take you away.
Because no one gets that excited about hiring a 60 year old.
So what you should do is take all the 60 over professors and say we're cutting your wage in half or by a third.
Because the truth, we've written our lectures already, the wage is already well above our marginal product.
We're all doing pretty well anyway.
And the truth is people wouldn't leave.
So why doesn't MIT do that?
MIT doesn't do that because I'm going to someday be one of those old guys.
I'm getting there rapidly.
I say, wait a second, if they're going to do to me when I'm 60, I'm going to get out of here while I'm 45 because I don't want to be in that situation.
Because that's unfair.
Workplace norms matter.
Employers really do not like to discriminate within the workplace because it breeds bad blood and ultimately can lower productivity.
And this is something that we miss in our basic models.
We don't have fairness and workplace norms in our models.
So wage is just about setting the wage that maximizes the profit, which means screwing the 60 year olds.
But in fact, in reality, that's not the way workplaces work.
And that's what the point of labor economics is.
If you're interested in this we have an excellent course in labor economics that follows up on these issues.
But a key issue is how much wage discrimination can be done given workplace norms, given the notions of fairness we have.
And the answer is it might be kind of tough.
Because basically MIT doesn't want to worry about upsetting all its younger faculty by mistreating its older faculty.
And part of that could be solidarity.
Some of those guys are my friends and I feel bad for them.
But partly it could be just more selfish which is, I don't want to be at a place that's going to discriminate in that way against me when I get older.
And that's just something that's missed by the basic 14.01 models.
Questions about that?
Yeah
Couldn't you model the person's life wage or something?
So yeah.
So what you can do is you could say to people, look, when we're hiring you we're going to overpay you relative to other universities when you're young and we'll underpay you when you're older.
And so on a lifetime basis you'll be fine.
What would the problem with that be?
So let's say the say-- yeah
You'd leave
I'd leave as soon as I reached that point where I was being paid more elsewhere that didn't pay this downward slope.
And, in fact, there's a lot of interesting labor economics theory which says the optimal formal labor contract is actually the opposite.
It's to overpay when you're old and underpay when you're young.
And the notion is to get people to want to stick around.
And so in some sense, common labor theory says exactly the opposite of what you suggested.
You want to overpay older people to get them to stick around.
And for many years in America that's often the way labor markets worked.
We had very generous pensions and health benefits and high wages for older workers.
That equilibrium is now breaking down because in this more competitive labor market you can't afford to overpay those older workers because then you can't attract the younger workers in the first place.
And we're moving towards a flatter profile by age.
But that's exactly the set of interesting issues we have to deal with in labor economics in setting pay that we don't really get into here.
Other questions or comments on that?
Of course there's another every reason why MIT couldn't do this, which is it's against the law.
We have age discrimination laws in our country which say you cannot discriminate against people based purely on their age.
MIT could say, well look, I could demonstrate a productivity difference.
This guy is publishing fewer articles than he did 20 years ago et cetera, but they do have a legal hurdle to overcome as well as an administrative hurdle.
It would be a pain in the ass for MIT to have to figure out exactly how to shift their wage schedule to do all this.
And that causes administrative costs and extra Deans and extra things and they just don't want to deal with it.
So there are other barriers as well which is administrative costs and legal costs.
But I think probably the main barrier is just the difficult issue of workplace norms and how it affects the productivity of the workers who are behind while you get rid of these older workers or underpay these older workers who aren't as productive.
Questions or thoughts on that?
So now with this monopsony model in mind I want to go back and revisit a major topic that we talked about early in the course.
And a couple times in this course we talked about as an example of how governments can screw up markets we talked about the minimum wage.
We talked about if you take a competitive labor market and impose a minimum wage above the competitive level, that could lead to deadweight loss.
Because what will happen is at that higher wage firms will want fewer workers.
Workers who would be happy to work at the competitive wage will not be able to work.
Trades which would make social welfare higher won't be made.
And there will be deadweight loss.
The monopsony model says that may not be the case.
Because in the monopsony model, a minimum wage can play the same role that optimal price regulation paid with monopolists.
Remember with monopolists we said if you regulated a monopolist and forced them to charge a competitive price, that you could actually force them into the competitive outcome.
Well if a minimum wage is set above the prevailing wage but that's because a monopsony prevailing wage is too low, and the minimum wage is set at the competitive level, then you could actually increase employment and improve outcomes with a minimum wage.
Sort of counter intuitive.
So let's look at it, pretty confusing.
Let's look at figure 18-3.
Figure 18-3 once again parallels a figure you saw on the entire flip side for price regulation of a monopolist, this is the parallel which is wage regulation of a monopsonist.
Let's walk through this.
It's pretty confusing so let's walk through this slowly.
Initially you have a monopsonist who is hiring at the point where their marginal expenditure curve, which is the dashed line and then the solid line.
So the line me1 is original marginal expenditure curve.
It's the me1 plus the me2, it's that segment.
That line which is, once again, more elastic than the supply curve.
That intersects the demand curve at a labor supply l1.
So they hire l1 workers and they pay a wage w1.
That's the initial monopsony equilibrium.
The competitive equilibrium is where supply equals demand.
That would be hiring l2 workers and paying a higher wage w2.
So the monopsonist is under hiring relative to the competitive firm.
Now let's say the government rolls in and says we're going to set a minimum wage and we're going to happen to get it right and set it at the competitive wage level, w2.
Well now let's think about the monopsonist calculus.
The monopsonist new marginal expenditure curve is the old one for the solid segment.
So where it says me2, that solid segment is still the marginal expenditure curve.
So as they think about hiring additional workers, they're working down that curve.
But once they get to l2 workers they can't lower the wage anymore.
So that marginal expenditure curve, they can no longer lower the wage below w2.
So their new marginal expenditure curve hops down and becomes the minimum wage.
So the new marginal expenditure curve is the two segments labeled me2.
It's the horizontal segment from the y-axis to e2.
And then it jumps up to that upward sloping segment to the right of l2.
So the new marginal expenditure curve is basically at a wage that's above minimum wage they continue to behave like a monopsonist.
But once you hit the minimum wage and they can't lower the wage anymore, what happens?
The poisoning effect goes away, just like we talked about with the monopolist.
Essentially what this has done has killed the poisoning effect.
Because it said as you're thinking about hiring that next worker to the right of l1, typically say I want to hire them because I'm going to have to raise my wage to everybody else.
But you're already paying everybody else a higher wage.
You're already paying everybody else the minimum wage.
So there's no poisoning effect.
You're not going to have to pay them a higher wage to hire that next worker because you're already paying them that higher wage.
So just as optimal price regulation undoes the poisoning effect on the demand side, optimal wage regulation undoes that poisoning effect on the supply side and can lead you to the optimal outcome.
So minimum wage can actually increase employment.
Pretty bizarre.
Minimum wage is our whipping boy for this course about how it causes dead weight loss and leads to lower employment.
Here we're saying, actually, if we start a monopsony equilibrium, a minimum wage could increase employment.
Of course, as you should know, the minimum wage could reduce employment even in a monopsony setting if it gets set too high.
So if the minimum wage got set very high-- Jessica maybe this is something we should actually add for next year-- so if you can imagine a minimum wage that's set very high, that could lead to a level of labor supply that's actually below the monopsony level.
So just as with optimal price regulation, we talked about how setting a price too high can make things worse, setting a minimum wage too high can make things worse.
So it's ultimately an empirical question of does the minimum wage raise employment, which would require two conditions, a monopsony market and a well-set minimum wage.
Or does it lower employment?
Which can happen either with a competitive labor market or it could happen with a poorly-set minimum wage.
This pretty confusing so you'll probably have to go home and think more about this.
But are there questions now about anything that's not apparent from the diagram?
So now this is why we have empirical economics.
We have empirical economics, well really we have two reasons.
One, is sometimes we know the direction of what we're looking for, we want measure its magnitude.
Sometimes we don't even know the direction, not to mention the magnitude.
Here's a case we don't even know the direction.
Will increasing the minimum wage raise or lower employment and by how much?
Well how do we test this?
Well the traditional way to look at it to say, OK, the minimum wage changes over time, let's look at what happens to employment when the minimum wage goes up.
And what people found was an increase in minimum wage tended to be associated with lower employment.
When the minimum wage went up, employment fell.
So people took that to mean that there was a situation where we're either in a competitive market or we're screwing up the monopsony market by setting too high a minimum wage.
Because we raised the minimum wage, employment fell.
And what is wrong with drawing that conclusion from that evidence?
Or could someone tell me a story about why that might not be a convincing piece of evidence?
Why the fact that when we raise the minimum wage employment falls, why that may not be by itself be compelling.
What problem you might have with that piece of evidence.
If you read that in the New York Times tomorrow, look at this graph, we raised the wage and employment falls.
Clearly minimum wage is bad.
What should your first thought be upon reading that article?
Well what do we care about?
We care about causation not correlation.
So what could be causing this to be correlation but not a causation effect?
Someone want to try?
It would depend on when you raise the minimum wage
Right, in particular what story might cause this effect?
If they're in a depression and they decide to raise the minimum wage
Right, what if governments, worried about workers in bad economies, that's exactly when they raise the minimum wage.
What if the government raised the minimum wage in bad economy, because that's exactly when they're worried about workers suffering.
Then you would see that a higher minimum wage is associated with lower employment.
But it's got nothing to do with the higher minimum wage.
It's got to do with the fact that you raised the minimum wage when unemployment is falling anyway.
It's causation versus correlation.
You have to be critical reader of evidence like this.
The minimum wage is not handed down by God.
It's determined by legislators who are subject to political pressures which may depend the state of the economy.
If the minimum wage increases when employment happens to be falling, then it will look like the minimum wage has harmed employment when it really hasn't.
So what we do about this?
What we do about this is to try to think about a way we can find a causal relationship between the minimum wage and employment.
And one way to do that is to try to find cases where a minimum wage increased and yet we know there was no independent change in economic activity.
And the way economists have done that is by looking at state minimum wages.
Turns out a lot of states actually set minimum wages higher than the national minimum wage.
Not really much anymore because the national minimum wage has gone up a lot the last decade.
But about a decade ago the national minimum wage, or about 15 years ago, the national minimum wage was at a real historical minimum in real terms.
And a lot of states exceeded that in setting their minimum wage.
So take two states, for example New Jersey and Pennsylvania.
New Jersey raised its minimum wage, Pennsylvania doesn't.
But any economic shock is going to hit New Jersey and Pennsylvania pretty similarly.
They're both right next to each other on the East coast.
Any recession's going to hit them pretty similarly.
So what you could do is you could ask what happens to employment in New Jersey when they raise their minimum wage relative to Pennsylvania, which suffers the same economic shocks but doesn't raise its minimum wage.
We try to achieve the gold standard.
What's the gold standard?
The gold standard is a randomized trial.
What we'd like is literally a trial where we change the minimum wage randomly at different places and different times.
That's ever going to happen.
So we try to approximate in what we call a natural experiment or quasi experiment which say, are there experimental interventions that nature gives us, even if they're not perfectly randomized trials.
And this is one.
Here's a situation where we have two states, very similar, one raises the minimum wage and one doesn't.
And they're set by economic shocks.
You can look at that.
Another way people have taken this approach is say, well when the minimum wage increases it's going to have different effects in different places.
And why is that?
That's because the minimum wage is going to be higher relative to the market wage in some areas than in others.
So, for example, when they raised the minimum wage a dollar nationally, most people in Massachusetts already make more than they raised it to.
So it's not going to affect Massachusetts much.
Whereas in Mississippi that's a big hit.
So given a national raise we can compare states that were hit harder by that raise to states that were hit less hard, another way to try to do this.
A number of studies have done this and they've come up with the striking conclusion that the minimum wage, if anything, raises employment, at least at the levels that we've seen over the last 15,20 years.
Changes in the minimum wage are actually associated with modest, but increases in employment.
And certainly no decreases.
They've been associated with modest increases in employment and not decreases.
And this is really, really striking because this is something that economists just never even really took seriously.
We always taught that minimum wage was the bad boy of economics, and this is saying, no, in fact, if you take it seriously and look at the evidence carefully you can actually see that a minimum wage can actually increase employment as this model shows.
Now, this is still the subject of some controversy, there's still a lot of work on it.
But it then raises the question of, well how can this really be?
Don't we have a pretty competitive labor market?
How do we really have a monopsony labor market?
And the answer is that if you look at who's getting the minimum wage it's largely younger and low-skilled workers who don't have a lot of employment options.
So basically McDonald's in a given area in a city could have some monopsony power.
Because people don't have cars.
These low-income urban youths don't have cars.
They can't go somewhere else to work.
They don't have skills so they can even work retail because they don't have good enough skills to work retail.
So McDonald's has them.
McDonald's has some monopsony power over them because McDonald's is a job they can get.
Or McDonald's and Burger King together maybe are the jobs they can get.
That gives them some monopsony power.
So given that's where the minimum wage is going to bite the most it's maybe not implausible that the minimum wage could actually increase employment which is what we see in the studies.
Questions about that?
Yeah
[INAUDIBLE] anti-trust would it be possible [INAUDIBLE]
Yeah exactly.
So let's take my inner city example.
McDonald's isn't the only employer.
There's Burger King and Wendy's and lots of other low-skilled employers.
The notion is if there's a few of them that should be enough to break any monopsony power unless they collude.
So likewise just as there's not supposed to be collusion on the output side, there are laws against collusion on the input side in the same way.
But once again, just as those laws are hard to enforce on the output side they're hard to enforce.
Because what you can do is you can get together in the back room, or they can just say, Wendy's and Burger King can wait and see what McDonald's does and just follow in lock step.
So there's lots of ways to get around those rules.
But yes, just as there's antitrust laws on the output side, there are the labor market laws on the input side which get in the way of collusion.
The difference is those are more on a sector by sector basis.
So, for example, the unionization of the workers affects the collusion ability of the employers.
So if the workers are unionized it's more lax in terms of allowing employers to collude as well because the workers are colluding.
If workers aren't unionized it's less lax.
And there's a complicated body of labor law about that.
Other questions?
Yeah
So it would be bad for unions to try and get an industry-wide standard wage or--
Well that's about efficiency versus equity.
If a union got an industry-wide standardized wage then that's going to penalize the talented workers who would want to go work elsewhere and help the less talented workers.
And basically a lot of the complaints employers have about unions is that they lose their talented workers because the union doesn't allow them to pay those differentials.
Teachers is a great example that's very controversial right now which is, should there be merit pay for teachers.
A lot of teachers say, no, there shouldn't be merit pay because that's going to violate workplace norms and it's unfair.
But we might not be getting the most talented people into teaching as a result.
So we'll come back next time and we'll start our topics part of the course by talking about international trade and whether it's good or bad.
Answer, it's good.
But we'll talk about why.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The first 2/3 of the course were covering sort of what you need to know to know basic microeconomics-- consumer theory and producer theory.
And basically you can now, if you understand the material, go forth in the world as a qualified micro economist.
What we're going to do for the rest of the semester is apply what we've learned and show you how you can use the tools that we've learned from basic consumer and producer theory to understand a broader range of phenomena.
Really, you can think of this as-- as I talked in the first lecture about we make simplifying assumptions-- this is sort of as we bend those simplifying assumptions, and consider more and more realistic applications of these models.
And the hint of what the sort of stuff you can get to see as you move on in economics and move to our other courses beyond micro.
So what we're going to start with today-- and of course, unfortunately, since I'm going to cover a lot of topics, I'll give each way too little time.
Including today, which is one lecture on international trade.
You could take several courses on it.
We have an excellent undergrad course, 14.54 on international trade.
And I'm going to sort of try to shove down your throats in one lecture the key things you need to know about international trade.
But if you find it interesting I urge you to follow up on this.
So, thinking about this-- a good way to think about international trade is to think about an example.
So let's think about Valentine's Day.
Valentine's Day sort of presents an interesting conundrum.
Because Valentine's Day happens in the winter.
And yet, the thing you're supposed to do is give roses.
Which don't grow in the winter in the US.
At least not very conveniently in many places.
So you've got this difficult issue that basically we're supposed to represent this holiday with something that doesn't actually come that time of year.
So historically what that meant it was if you wanted to get roses for Valentine's Day you to buy them from specially heated greenhouses.
Where they set up largely to supply the roses for Valentine's Day.
There wasn't really a large purpose for them otherwise.
However, over the past couple of decades, what's happened is instead-- instead of growing these in these specially heated greenhouses, we've started flying them in from other places, from Colombia.
Where of course, Colombia's on the other side of the equator.
So February is a wonderful time to grow roses in Colombia.
And as a result we've started flying them in.
And the typical rose you will give on Valentine's Day this year will come from Colombia.
Now the issue is-- is that a good thing or a bad thing?
Now on the one hand, we get cheap roses.
That's good.
Especially for poor college students who want to impress their valentine by sending a dozen roses.
It's good they're cheap.
And roses are way cheaper now than they were when I was-- even in dollar terms when I was in college giving roses.
Roses are just incredibly cheap now compared to 20 or 30 years ago.
On the other hand, a lot of rose producers have lost their jobs.
A lot of people whose livelihoods and source of income was growing these roses are now out of jobs.
OK, these are typically people who are not high-skilled people who can go find another job easily.
These are people who have been really displaced for something which was a specialized skill which they cannot easily use other places.
And basically this trade-off is sort of a microcosm of the debate we have over international trade every day.
A debate that's ongoing.
Obama just came back to the G20 summit.
Where there was huge discussions of the issues of international trade.
It's an ongoing debate.
And it's a particularly important topic right now in the US because the US is running what's called an enormous trade deficit.
The trade deficit is the difference between how much we export, that is how much of our goods we sell to other countries, minus how much we import.
Which is how much of goods from other countries that we buy.
Currently we export about $160 billion worth of goods every month.
So every month we send out $160 billion worth of goods around the world.
We import about $200 billion of goods every month.
So that means we have a trade deficit that's running about $40 billion.
Now the question is-- is that a problem?
Is it a problem that the US is systematically buying more stuff from the rest of the world than they're buying from us?
And the answer is it's not necessarily a problem.
And really, it might in fact be a natural outcome because of the principle that we'll focus on today-- the principle of comparative advantage.
Comparative advantage is saying if some other place is particularly good at producing roses in February, then we shouldn't be that stressed about the fact that we're running a deficit of roses.
That's something which is OK in terms of total efficiency.
So to see that, let's focus as this rose example in a particularly simplified way.
Imagine there's two countries-- the US and Colombia.
And there's only two goods in the world-- roses and computers.
Two countries, two good models.
The standard model we work with with international trade.
With two country two good models you can develop almost everything you need to know.
There is no need to make it more complicated.
Now, as I mentioned it's really hard to grow roses in February in the US.
It's a lot easier in Colombia.
On the other hand, it's much easier to produce good computers in the US than in Colombia because we have the high skilled labor force that can produce computers.
So we have a thing where the US is relatively bad at producing roses in February.
Colombia's relatively bad at producing computers.
So the key point is that means that the opportunity cost-- remember the opportunity cost, this key concept we've come back to a couple times.
The opportunity cost of producing a rose in terms of producing computers is relatively high in the US.
That is to produce a rose we have to use so many resources.
Those resources can be much more effectively deployed to producing computers.
Likewise, in Colombia, to produce a computer would use a ton of resources that would be much more effectively deployed to produce roses.
As a result we see that Colombia has a comparative advantage in roses.
And the US has a comparative advantage in computers.
The point is that if a country is relatively good at something then they have a comparative advantage.
And it's all about relativities.
Because people are going to want both.
But the key thing is who's relatively good at producing one versus the other.
So to see that let's go to figure 19-1.
To see how we diagram this, let's go to figure 19-1.
Figure 19-1 shows production possibility frontiers.
You learned about these a while back.
Let me remind you, a production possibility frontier we talked about in the context of a firm.
It shows the trade-off between the firm's ability to produce one good versus another good.
So for a firm producing two goods a production possibilities frontier is the combination of the two goods they could produce at a given level of inputs.
So we talked about it from the context of firms.
We can also talk about this in the context of countries.
That says, we can draw a US production possibility frontier.
Which is given the resources the US has, it could produce up to 2000 computers and no roses.
Or 1,000 dozen roses, boxes of roses, and no computers.
And let's assume it's linear in between.
So the US production possibility frontier is given the resources we have-- and this is a very simplified example-- but just bear with me.
Given the resources we have, we can produce 2000 computers and no roses or 1,000 boxes of roses and no computers.
Or any combination in between.
That's our production possibility frontier.
Columbia has a production possibility frontier illustrated in the second panel.
They can produce 1,000 computers and no roses.
Or 2000 boxes of roses and no computers.
That is, Columbia has a comparative advantage in roses.
Meaning that their production possibility frontier is a lot flatter than ours is.
We have a comparative advantage in computers.
Meaning our production possibility frontier is a lot steeper than is Colombia's.
Ignore panel C for the moment.
Now remember what the slope of the production possibility frontier is.
It's the marginal rate of technical substitution.
It's the marginal rate at which the producer can substitute one good for another.
So basically, for the US, the marginal rate of substitution of roses for computers is -2.
That is you have to give up two computers to get one box of roses.
In Colombia it's -1/2.
You have to give up 1/2 a computer to get a box of roses.
So since the marginal rate of substitution is so much higher in the US, we say that Colombia has a comparative advantage in producing roses.
Now let's go further and impose tastes on consumers in each country.
Let's say that tastes are such that given these production possibility frontiers, consumers in the US choose 1,000 computers and 500 boxes of roses.
We choose production over love.
Colombia chooses love over production.
Given their production possibility frontier, this is not inherently about taste necessarily.
Because you have very different slopes here.
But given their tastes and their production possibility frontier, they choose 500 computers and 1,000 boxes of roses.
Now this we call the outcome.
We call this outcome the autarchy outcome.
Autarchy.
Which is the word-- I don't know what the hell it means, but it basically means no trading.
Autarchy.
I don't know where it comes from.
Must be some Russian term or something.
Autarchy.
Which means no trading.
So the no trading outcome is consumers in the US consume 1,000 computers and 500 boxes of roses.
Consumers in Colombia consume 500 computers and 1,000 boxes of roses.
Now the key point is that both the US and Colombia can be better off if we introduce trade.
And how is that?
Well if we introduced trade, then each country can specialize in their comparative advantage.
Trade allows for specialization.
That is the key advantage of trade.
Their comparative advantage naturally yields specialization.
Comparative advantage naturally yields specialization-- it makes sense for the US to be a computer producer and Columbia to be a rose producer.
It doesn't make sense the two countries to produce both.
But absent international trade they have to produce both.
Because consumers want both.
So if we're shut off from the world and Columbia's shut off from the world, then we end up as in figures A and B.
But once we introduce trade, then we can take advantage of our relative expertise.
And we get a new production possibility frontier which looks like panel C. That is, if you want more than 2000 computers-- so if you want 2000 computers and 2000 roses-- then you simply have the US produce just computers and Colombia produce just roses.
And you can get 2000 of each.
Now you can get 3,000 computers and no roses by having everybody produce computers.
Or 3,000 roses and no computers by having everybody produce roses.
So you know production possibility frontier has its sort of wedge point that 2000-2000 intersection.
You can label that point the point of specialization where those two dashed lines hit the solid line.
That's the point of specialization, pure specialization.
That's the point where the US does just what it's good at.
And Colombia does just what it's good at.
Of course you could have other combinations too.
And that's what leads to this bent production possibility frontier.
But the key point is this joint production possibility frontier is further out than what any country could have produced on its own.
We've increased the opportunity set.
Specialization has led to a larger opportunity set.
A larger opportunity set-- we've expanded the opportunity set for the world by allowing countries to specialize.
And the result of that you can see in the next figure.
In figure 19-2 which shows gains from trade.
So what this figure shows-- this is the same autarchy figure from panels A and B before.
I've just added some more labels.
So what we see is, in autarchy, we're at point C- sub US.
With the US producing and consuming 1,000 computers and 500 boxes of roses.
If we move to specialization, move to international trade, what happens is the US moves to producing 2000 computers at Q-US.
And, likewise, looking at the second panel, Colombia moves to producing 2000 box of roses at Q-CO. And US consumers now increase their consumption of both roses and computers.
As do Colombian consumers.
And you end up with total consumption of 2000 computers and 2000 roses.
So the US consumes 1250 computers and Colombia consumes 750.
And the flip for roses.
So we are learning that Colombians, even at the same price, do prefer love over production.
But nonetheless, the bottom line is, both sets of consumers are better off.
Both sets of consumers are consuming at a higher point than was possible without trade.
It's magic.
The magic is that simply by letting them trade we have made both countries better off.
And the magic, the key to the magic, is specialization and comparative advantage.
If the US and Colombia were identical in terms of their production possibility frontiers then you should be able to see that there would be no gains from trade.
If they had the same production possibility frontier, if in A and B the slopes were the same, then the joint production possibility frontier would be identical to what's in each country.
There would be no gains from trade.
Gains from trade come from the fact that these production possibility frontiers have different slopes.
That there's comparative advantage in one country and not another.
Which allows specialization.
So, basically, the key insight of international trade-- and once again I'm doing now in 50 minutes what you do in 12 lectures when you get it right.
But the rough insight is that comparative advantage yields specialization, yields gains from trade.
That's sort of the chain of logic to be thinking about this.
Questions about that?
Yeah?
If we're going to adjust to international situations?
Or could it be any sort of specialization between two companies?
Any specialization between two companies.
So just we learned about this originally in the context of companies.
Same issue.
In the business world they have a term for this.
What do they call it?
Synergy.
You all know that word, you've got to if you're going to business school.
Synergy.
Synergy means that somehow you put two companies together and they can produce the same stuff better.
Synergy is a fancy name for this.
Which is the idea is that there may be gains from specialization even within a company.
But if you take the old example, take two shoe companies, both producing left and right shoes inefficiently and they could specialize.
And one could do better producing left shoes and one at right shoes.
You put them together and overall you can have more shoes.
That's actually a pretty stupid example, but you get the point.
That basically the same principle can occur whenever there are gains from trade.
Whenever there's comparative advantage of specialization you make gains from trade.
Good question.
Other questions?
Now, that raises the interesting question of-- well these comparative advantage things sound great.
Where can I get one?
Where do comparative advantages come from?
And there's really two sources of comparative advantages.
Comparative advantage in international trade.
So where does comparative advantage come from?
One is differences in factor endowment.
Differences in factor endowment.
What that means, is that for example, Canada has a ton of trees.
Everywhere.
Canada is endowed with an enormous amount of lumber resources.
With that factor very well.
So Canada is an enormous exporter of lumber and paper products.
Because they happen to have the main thing you need for that which is unbelievable amounts of trees, everywhere.
So Canada can specialize in exporting lumber and paper.
So now that gives them a comparative advantage in that area.
Now let me ask you another question.
Why does China export most of the world's clothes now?
What?
[INAUDIBLE]
Cheap labor.
It's not that they have cheaper textiles.
It's not that the cloth itself-- and I realize that the silkworms are in China.
But not like the actual production of cloth is that much cheaper.
It's that clothes are primarily a labor intensive good.
And the labor is cheapest in China.
So they have a factor endowment.
They have an advantage, comparative advantage, in labor intensive goods.
So China, with international trade, will produce a disproportionate share of labor intensive goods.
Because they can specialize in labor intensive goods.
And make them cheaper for the rest of the world.
So likewise, a sweatshirt that you would go and buy today-- especially if you go and buy it at a not top end store.
At an Old Navy or even at a Costco.
Is literally in dollar terms cheaper than what I paid for that same sweatshirt when I was in college in the early 1980s.
Because they're just produced incredibly cheaply in China now.
We bring them in and they're just cheap.
Goods where you can specialize in that are going to be-- when you take advantage of specialization will be a lot cheaper.
So that's one reason why you see comparative advantage.
The second reason is going to be technological leadership.
Technological leadership.
So, for example, Japan has no natural comparative advantage of producing cars.
There's no reason why Japan should have a comparative advantage of producing cars over the US.
Except they developed the technology to more efficiently mass produce the modern automobile.
As a result of that technological leadership they gave themselves, essentially, comparative advantage.
Now, once that technology becomes public, the production shifts elsewhere.
So now China is a major producer of cars.
Basically copy-catting the technology developed in Japan.
So it does move elsewhere.
Unlike factor endowments-- if the US wanted to compete with Canada I guess we could plant trees and in 50 years we would compete.
But factor endowments are kind of hard to compete on.
Technology is potentially a little bit easier because you can reverse engineer things.
So once again, the technology's shifting to China.
So really the answer is in the long run everything's going to be made in China.
Because they've got the cheap labor.
And they're adopting the technology.
And once again, to quote towards our most important sense of cultural relevance, there's the episode of the Simpsons where Homer says, don't worry we're fine.
We're going to rule-- our country's going to rule the world.
We're fine in the future.
He goes, wait a second we're China, right?
So basically China is doing very well because they've got these factor endowments-- these great factor endowments and because they are adopting technology as well.
What's interesting about this is this really leads to some more interesting policy issues.
Once again, factor endowment you can't do a whole lot about.
The interesting policy issues are in technological progress.
This is an argument that many people make for subsidizing new technologies.
So you often hear President Obama saying, we need to subsidize green technologies.
This is the wave of the future.
The idea, what he's saying implicitly is if we get the technological leadership, we can make the green products that are exported.
Not the color-- I mean environmental stuff-- that can be exported to the rest of the world.
And that's the argument that he's making.
Question about that?
Now let's talk about where we started the lecture, which is is trade a good thing or a bad thing?
I just talked about how trade can greatly increase consumption possibilities.
But why haven't we talked about the consumers?
What about overall social welfare.
And the answer is that trade unambiguously increases social welfare.
That trade is unambiguously a good thing to do.
So to see that let's go to the next series of figures.
Start with figure 19-3.
So let's start with the market for roses and imagine we have-- [INAUDIBLE] autarchy is sometimes spelled with a K, sometimes with a CH, I don't know what the right way to spell it is.
But anyway, you have autarchy.
So the US-- imagine the old days where we produced all our roses and we're at some equilibrium with some consumer surplus and producer surplus.
So we produce Q sub A roses at a price p sub A.
Now, in figure 19-4 we're going to introduce international trade.
It's a little confusing, so let's go through it slowly.
Figure 19-4.
You can find on that figure the domestic supply and the domestic demand.
And they intersect at point A.
Figure 19-4, domestic supply and domestic demand intersect at A.
Which is once again a quantiy of Q sub A and a price of P sub a.
Now let's say that what happens is we now allow imports of roses.
We now trade with Colombia for roses.
What that does is that means now instead of just drawing on the domestic supply, we can now draw on the world's supply of roses.
We don't have to just rely on domestic supply.
Well, that by definition has shifted further out.
And it's shifted further out because other countries countries can produce roses so much more cheaply than we can.
So we can rely on the world's supply of roses.
Remember my cotton example where you first buy from the cheapest country, then the next cheapest country, et cetera.
This is what we're saying-- if the US was the cheapest producer of roses, this wouldn't shift out.
But since the US is not the cheapest producer of roses this shifts out to world supply.
And we get to a new equilibrium at quantity C sub t. That's the new quantity of roses we consume.
And a lower price P sub w. The horizontal line's a little distracting, actually.
But basically what you have here-- the horizontal line is just showing the price in autarchy and the world price.
But the bottom line is, what you get you get is you get this new equilibrium with a quantity C sub t and a price P sub w. And in particular, what we're seeing is that total domestic consumption of roses has increased.
But domestic production has fallen.
Because look, the new price intersects the supply curve at Q sub t. So what happens is at that new lower price P sub w, US rose producers aren't actually producing as much.
So US domestic production falls from Q sub a to Q sub t. But US consumption rises from Q sub a to C sub t. The difference is imports.
Production falls from Q sub a to Q sub t. Consumption rises from Q sub a to C sub t. The difference is imports.
And that's what happens when we allow Colombia to send us roses.
What are the welfare implications of that?
Let's go to figure 19-5.
And what you can see is we can show you the welfare implications.
Previously consumer surplus was W. Producer surplus was X plus that other white triangle below X.
Now what's happened?
Consumer surplus has gone up by X, because consumers have gained that plus z.
So consumers have gained that entire trapezoid, X plus Z.
Producers have lost x.
Producers have lost that trapezoid, X.
So in total we've gained Z.
We've gained that entire triangle, Z.
Z is the entire triangle on both sides of the dashed line.
So we've gained all of Z.
Consumers gained X plus Z, producers lost X.
We've gained Z.
So overall we've increased welfare in the US.
So basically consumers win, producers lose.
And that's the problem.
The political problem we have.
And I'll come back to this.
What you can see is consumers win, producers lose.
But by definition, consumers win more than producers lose.
So overall, social surplus has gone up.
Now that's the case of imports.
Now what about exports?
Well if imports make us better, do exports make us worse?
We just said imports make us better.
What about exports?
Well let's look at that in figure 19-6.
Now figure 19-6 shows what happens with exports.
Once again we start at point A, the domestic outcome, point A.
And now we're talking about computers.
So now the US is going to export computers.
What that means is that the supply of computers to people in the US is going to fall.
why?
Because a bunch of them are going to get sent away.
So it used to be US consumers got to consume on that domestic supply-- so US producers used domestic supply curve.
And we had domestic demand intersecting at A.
Well now, domestic producers are shipping a bunch of the computers to Japan, and China, and Colombia.
As a result, the supply has shifted in.
Because they don't have as many computers to sell in the US.
That means the price rises.
So because of exports we pay more for our computers.
And it sort of makes sense, right?
They produce a bunch of computers.
If there's been a great demand for them elsewhere and we want them too, we're going to have to pay a higher price.
Because we're competing with other people for those computers.
So now that supply curve has shifted up.
And now we end up that US consumers only want to C sub t computers at a new higher price a P sub w. But the world is now saying, well we are interested in the total amount of computer we want is Q sub t. So what that world price of sub w-- the domestic producers say great, at that higher price P sub w, I'm delighted to produce Q sub t computers.
So what I'm going to do is I'm going to produce Q sub t, I'm going to ship Q sub t minus C sub t elsewhere.
And US consumers will consume C sub t. So consumers are worse off.
So unlike imports which make consumers better off, consumers are worse off.
So is trade bad?
Well, no.
Trade's not bad.
But the bottom line is, domestic producers are going to gain more than domestic consumers lose.
The bottom line is we'll be better off from exports.
But in this case, consumers lose and producers win.
Society gains either way.
Society gains from imports because consumers gain more than producers lose.
Society gains from exports.
Because producers gain more than consumers lose.
So what that means is that any form of trade is going to make the US better off.
But also inevitably create losers.
Any form of trade will make the US better off but inevitably create losers.
And the problem is that those losers are very loud in the political process.
And the winners are not so loud.
So, for example, if you polled US consumers and said, how much are you saving on your sweat shirts?
Because we import textiles from China.
They wouldn't know.
I mean they'd say a couple bucks what in fact it's maybe-- they'd say I don't know, 10% of the price when it's maybe more like 50% of the price.
But if you polled US textile manufacturers, or guys who have left the business, and said how much did you lose because of imports of China, they'd tell you exactly how much they lost.
They'd tell you 10 times as much.
But they know how much they lost.
And they go to their politicians and they say, hey we're losing jobs here.
And the consumers don't come saying, hey I'm getting cheaper sweatshirts here.
As a result, there's huge political backlash against imports in particular.
So that leads to policies which limit imports into countries like the US.
Like tariffs, which are essentially taxes.
Tariffs, which are essentially taxes on goods imported into the US to us.
Which are taxes on goods that are imported into the US.
Or quotas, which are limits on how much companies can send to the US.
What I think what you should know now is by definition-- since I said free trade is good, these things are going to be bad.
And they're going to be bad-- things like tariffs are going to be bad because they're going to hurt US consumers more than they're going to help US producers.
So to see that let's go to figure 19-8.
Let's show what happens with a tariff.
This is saying that now we're starting the world with trade.
So the world with trade we're producing C sub 1, which is where domestic demand equals world supply at a price P sub w. That's where we were with trade.
Now the US government comes in and levies a tariff.
And says that that tariff is going to be the difference between P sub w and P sub t. So that's the amount of the tariff.
They are going to raise the price essentially to P sub t. They're essentially going to levy a tax on roses that come in from Colombia.
What that does is that means that consumers now, at that higher price, only want C sub 2 roses.
That's where that intersects demand.
So they wanted C sub 1 roses without the tariff.
But now with the tariff at that higher price intersects demand at C sub 2, they only want C sub 2 roses.
Producers, given that they have to pay the tariff, are only going to produce Q sub 2.
Domestic producers are now going to produce Q sub 2 roses because they get a higher price.
So domestic consumers are consuming less.
Domestic producers are producing more.
Right, because now that price is higher.
So now you could reopen those heated greenhouses because it's worth it now.
And we end up with a much smaller amount of imports.
So in that sense the tariff worked.
So in the sense that the goal of the tariff was to reduce imports, it worked.
Consumers wanted less from Colombia.
Producers were happy to produce more.
So imports fell.
So in that sense tariffs worked.
They've lowered imports.
But what do they do to welfare?
Well that's our final figure 19-9.
What do they do to welfare?
Well what you can see is domestic producers gained a trapezoid A.
Their producer surplus, which used to be that little white triangle below A, became that entire triangle that includes A and the white part below it.
So their producer surplus became A.
They gained A in producer surplus.
But consumers lost a ton.
They lost the entire trapezoid A plus B plus C plus D. They lost that entire trapezoid gone up to higher prices.
And now we have one other player which is the government.
The government made some money.
This is a tax.
How much did the government make?
Well we taxed all imports by the amount P sub t minus P sub w. So the government made the rectangle C. The government made C because we got to tax all the important at that tariff.
So on net, society as a whole is worse off by B plus D. Consumers have lost the entire A plus B plus C plus D. Producers got some of that, the government got some of that.
But some of it's dead weight loss.
Trades that would have made consumers better off, and worldwide producers better off that are not happening.
So this tariff, it had its effect.
It lowered imports.
But at the same time it lowered social surplus.
So restrictions on trade lower welfare.
And this is why economists like free trade.
Because restrictions on trade lower welfare.
Are there questions about that?
Moreover, this is just what we call static analysis.
What else could happen if the US government voted a tariff on roses coming from Colombia?
What else might happen?
Yeah?
Colombia might vote for a tariff on the
Columbia might vote for a tariff against computers coming from the US.
Which of course would hurt Colombian consumers.
But would also hurt US producers.
So from the US's prospective, that would be even worse.
So think about it-- we put a tariff on Colombian roses which hurt us.
Then Colombia puts a tariffs on US computers which hurts us even further-- hurts Colombia too, but we don't care about them, we care about us.
It hurts us even further.
So dynamically restrictions on trade can be even worse than it looks in this diagram.
If they cause a trade war you can actually end up with things being even worse than you look at this diagram.
So we talked about why economists like free trade-- partly so that the gains to consumers exceed the losses to producers.
But partly it's because free trade begets free trade.
And that exports make us better off too.
So by allowing free trade not only do we increase our welfare by having more imports, we also increased our welfare by allowing more exports.
And in fact, producers as a class, if they could all get together and have a cooperative equilibrium, they might be fine without restrictions of trade.
Because the computer producers and the rose producers get together and say, wait a second, we can trade, on net we win from this.
On net we win from free trade.
Let's get together and make a deal.
But of course we know that's going to be an incredibly hard cooperative equilibrium to enforce.
And that's why you have different sets of producers.
You have the rose producers lobbying the congressmen.
Then they leave and the computer producers walk in his office and lobby him the other way.
So you have producers lobbying for different effects depending if they're competing with imports or they're exporting.
They're going to lobby for and against trade restrictions.
Moreover, there's yet another thing we've missed.
There's yet another thing we've missed.
So we've talked about the dead weight loss.
We've talked about trade wars.
But let's say for a second-- let's think about a third thing.
Let's say for a second we're not just heartless selfish Americans.
But we actually do care about the rest of the world.
Well the third thing we miss is allowing trade from Colombia makes Colombia better off too.
So not only have we improved our welfare by importing Colombian roses, we've improved their welfare too.
And they're a poor country.
And we should be happy to improve their welfare too.
We think about free trade in Vietnam.
That is the application for child labor.
By allowing the import of Vietnamese rice we improved the lives of the Vietnamese children.
So independent of the fact we've made our lives better off we've made these other country's lives better off too.
So there's three reasons why we should have free trade.
There's the simple welfare gain, there's the dynamic welfare gain, and there's the fact that we might care about the welfare of other countries as well.
And as a result, there's been a huge emphasis over time in trying to increase free trade.
By economists saying this for years, things going back to the early economists.
But we've made a lot of headway in the last 20 years.
And in particular, one big source of headway was NAFTA-- The North American Free Trade Agreement.
Which was passed under Clinton.
Which basically demolished trade barriers between Canada the US and Mexico.
There used to be a lot of trade barriers.
We would tax Canadian lumber.
They would tax our computers.
We would tax Mexican whatever they sent to us.
They would tax our stuff.
It was a mess.
So basically NAFTA said let's just get rid of all of it.
Let's have a cooperative agreement.
Let's have a cooperative agreement whereby we all agree to get rid of trade barriers and thereby making all of us better off.
And it seems like it would be a pretty simple thing to do.
But it was a mess.
And it was very hard.
And it was very hard because now you had parties in every country opposing this.
You had you had the Canadian computer producers getting upset.
You had the Mexican computer producers getting upset.
You had the US lumber industry getting upset.
And you had huge difficulties because you had parties in every case opposing it.
And the reason free trade agreements are so difficult-- now eventually NAFTA did pass.
And has been by most measures a huge success.
But ultimately the major difficulty here is in another failure of government policy.
Which is the inability of the government to effectively tax the winners and compensate the losers from international trade.
That is-- if the government could work as a perfect-- in the way that I design it to work-- what would happen is that we would tax sweatshirts and take the money from taxing sweatshirts and send it to compensate the guys who lost jobs in the textile industry.
And if we did that, we can make life much better off than with the tariff.
So if consider two outcomes.
One is we don't import Chinese sweatshirts.
The other is that we import them but we levy a small tax on all sweatshirts not just Chinese lectures.
Levy a small tax on all sweatshirts.
We import Chinese sweatshirts but then we levy a tax on all sweatshirts.
And we use that sweatshirt tax money to pay off the guys who lost their jobs in the textile industry.
That would be better for society than would not allow sweatshirts to come in in the first place.
So the fundamental failure in trade policy ultimately is a failure is that inability of the government to effectively compensate the losers by taxing the winners.
Because just as the losers know to go to lobby Congress, the winners will get pissed off if you start taxing sweatshirts.
Because they don't realize that they're saving all this money from all these imports.
And in fact the typical American voter would probably vote to block Chinese imports before they'd vote to have a tax on their sweatshirts.
So really in some sense when you have a very concentrated set of winners or losers competing against a very concentrated set of losers, defeating against a very diffuse set of winners, that's a hard policy to get in place.
Unless you can figure out a way to get those winners to compensate the losers.
And that's the tricky thing.
Are there question about that?
About free trade agreements, things like that?
Now what I want to talk about for a couple minutes then one subtlety in these free trade agreements.
And why things like NAFTA are hard.
And the reason they're hard is because this makes enormous sense.
Free trade agreements make enormous sense if it's true comparative advantage like Canada has more trees.
But what if Mexico's source of comparative advantage is that they treat their workers like shit.
What if China's source of comparative advantage is that they have incredibly bad working conditions?
And they have incredibly bad environmental conditions?
China has an incredibly bad environmental situation.
It is incredibly dangerous just to live in major Chinese cities.
The working conditions are awful.
The wages are terrible.
Nonetheless, the result is very, very cheap labor.
Then things get a little dicier.
Because if the Chinese labor was cheap or Mexican labor was cheap just because inherently that was just the way life was.
At the same working conditions and the same environmental conditions they were just cheaper, then that's great.
That's the comparative advantage.
But if they're cheap because they exploit their population and put them in horrible working conditions and facing horrible environmental conditions, then the welfare gets a little trickier.
From the US's prospective it doesn't get any trickier.
The US-- our welfare is exactly the same.
We get these static and dynamic gains from trade.
But from a world perspective, now we're maybe not so sure we want to promote Mexico producing more clothes or China producing more clothes.
Because it could just lead to more exploitation of the population.
So this comes back to when I talked about free trade in Vietnam and child labor.
Which is if free trade actually let to an increased use of child labor in Vietnam we might worry that gee maybe this isn't such a good thing.
So really what it comes down to is we have to think not just about the simple dead weight loss triangles and squares.
We have to think about some broader social issues as well.
And once again there is a right answer here.
And this is what a lot was fought over with NAFTA.
Which is one of the conditions for NAFTA was the US actually lobbied and got significantly improved work and environmental standards in Mexico.
This was like an opposite of a trade war.
Instead of being a vicious cycle, this was a virtuous cycle.
We actually improved the life for workers in Mexico because Mexico is willing to do that in order to get the benefits of trade with the US.
So just like limiting trade with Mexico will hurt us and hurt Mexico, expanding trade with Mexico and then tying it to improve work conditions and environmental conditions can make both countries really much better off.
And so that's where, once again, trade becomes very interesting.
And why it's worth studying further.
Because it's not as simple as these boxes.
It's thinking about where comparative advantage comes from.
And whether we really want to promote trade with countries that have comparative advantage depending on the source of what that comparative advantage is.
And that's sort of the free trade versus fair trade argument.
The bottom line in that argument-- the free trade versus fair trade argument-- the bottom line is free trade is always a good thing.
But fair trade considerations can be used to try to make it work better than just simply unfettered free trade.
And so really both sides have a point in this debate.
So let me stop there.
That's international trade.
We're going to come back on Wednesday to talk about uncertainty and whether you should play the lottery.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
Modeling decision under uncertainty turns out to be a critical part of what we do in economics.
And I'll spend today's lecture talking about this set of issues.
And, let me just say, the uncertainty you face now is nothing compared to the uncertainty that you'll face later in life.
So you have uncertainty now about whether you should study for the final, or carry an umbrella, or go on a date with this person.
I've got uncertainty about whether I should refinance my mortgage, or which college to send my kid to, or how much life insurance I should buy.
Uncertainty only get more and more important as you move on in life.
This is an important issue.
Now, how do we think about uncertainty?
Well, the tool that we use to think about uncertainty is, once again, to make simplifying assumptions which allow us to write down sensible models, but which capture the key elements of what we're thinking about.
And the simplifying assumption here is we move to the tools of what we call expected utility theory.
And so, basically, the way we think about expected utility theory is the following.
Imagine that I offered you guys in this class a choice.
And I'm just going to say right now, there's no right answer to this.
But I do want you guys to answer me.
There's no right answer.
Here's the question.
I'm going to give you a choice.
I'm going to flip a coin.
I have a coin in pocket, and I'm going to flip it.
And I'm going to offer you guys the ability to make a bet.
If it comes up heads, you win $125.
If it comes up tails, you lose $100.
Heads, you win a $125.
Tails, you lose $100.
There's no right answer.
How many would take that bet.
How many people would not take that bet?
Very good.
That's the typical set of responses I get to this.
Now, what's interesting is to think about the parameters of that bet.
And to think about it, let's take a step back to something we've discussed already this semester, the concept of expected value.
What's the expected value of that gamble?
The expected value, if you remember, is the probability of each outcome times the value of that outcome.
That is you remember expected value, which you defined before, is the probability that you lose times the value if you lose plus the probability that you win times the value if you win.
That's the expected value of a gamble.
So, in this context, the expected value is there's a 50% probability that you lose, so 0.5.
And if you lose, you lose minus $100 plus a 50% value that you win.
It's flipping a coin after all.
And if you win, you won $125.
So the expected value of this gamble is $12.50.
On average, if I did this enough times, you would win $12.50 per time.
Statistically, if I did this enough times, you'd win $12.50 per time.
So, in other words, we say that this is more than a fair bet.
A fair bet is one with an expected value of 0.
A fair bet has an expected value of 0.
So a fair bet would be tails you lose $100, heads you win $100.
This is a more than fair bet.
There's more than 0 expected value.
Yet, the majority of you would not be willing to take this bet.
In fact, the majority of people would not take this bet.
Why is that?
Why is it that I've dictated a bet which has a positive expected value and yet, people won't take it.
Yeah
But wouldn't that also depend on how much money you have
It will absolutely depend on how much money you have
Right.
So if I were a richer person, then losing $100 isn't as important to me as the chance of getting $125
OK.
So flesh that out.
Why is that?
Why is it that basically it would matter how much wealth you have.
Because no matter how much wealth you have, this math is impeachable.
It's always a good bet.
So why is it that your state without much wealth, your state as college students without much wealth, what is it about you that causes you to not want to take this bet that's more than fair
So, basically, for me, the risk of losing or the state I will be in after I lose is much greater, well, for me, a lot more than what I would be in if win
Exactly.
And there's two possible reasons for that.
One we're going to push off to the very end of the lecture.
The main reason we're going to focus on is because individuals do not consider expected value, they consider expected utility, and individuals are risk averse.
Expected utility is going to differ from expected value when individuals are risk averse.
Expecting utility is not going to be the probability times the value if you lose.
Expected utility is going to be the probability that you lose times the utility if you lose plus the probability that you win times the utility if you win.
And utility is not the same as value, importantly, because utility functions exhibit diminishing marginal utility.
Utility functions are not linear.
Utility functions are nonlinear.
And, in particular, there's diminishing marginal utility.
And with diminishing marginal utility, you're going to not want bets where there's the chance you lose is equal to or even a bit smaller than the value that you win.
And the basic point is that the joy of winning is smaller than the pain of losing with diminishing marginal utility.
Yeah
Isn't there also a statistical side to this then?
Because we don't know how many times we're going to bet.
It might just be once.
We're a lot more comfortable if, let's say, use the law of large numbers and say, OK, it's going to eventually even out so we'll win $12.50 a game.
But for the first, let's say 10 or so games, we might get really unlucky and flip eight tails and two heads
But, once again, if you weren't risk averse, you wouldn't care about that.
Hold that thought.
I'm going to explain why that isn't true.
So just hold that thought.
So now let's imagine that your utility functions are the typical form we've worked with before, the typical diminishing marginal utility form we've worked with before where utility is the square root of consumption.
You're casting your mind back to consumer theory here.
You're going to have to start integrating the course now, both consumer and producer theory.
So remember we said the typical diminishing marginal utility function we worked with was u equals the square root of c. Now, let's say you start with consumption of $100.
Imagine you consume your income.
Let's say you have consumption of $100.
Well, then utility is 10.
If you start with a consumption of $100, your utility is 10.
Now let's calculate the expected utility of this gamble.
The expected utility of this gamble is that there's a 50% chance that you lose.
And, if you lose, what is your utility?
Well you lose $100.
So consumption goes to 0.
So utility is 0 plus a 50% chance that you win.
Well, what do you get if you win.
Well, if you win, you go from $100 to $225.
So your utility is the square root of $225, or $15.
Your utility is the square root of $225 or 15.
It's half chance of having 15.
I'm sorry.
So this is a negative.
Yeah.
So utility, if you take this gamble, is you end up with a utility of 7.5.
So utility falls.
You move from a utility of 10 without the gamble to a utility of 7.5 with the gamble.
Utility is lower with the gamble, which is why people decided they didn't want to take that gamble.
Utility is lower.
And the reason is because given a utility function of this form, you are sadder about losing than happier about winning.
To see that, we can see that graphically in Figure 20-1.
This graph's utility against wealth-- we don't usually graph utility, because it's not cardinal.
Remember it's just ordinal.
But the sort of gives you a sense of the intuition.
This is a graph of utility against wealth levels.
So you start at point A.
You start with $100 in wealth, which is consumption.
and utility of 10.
Now, I give you a choice of a gamble.
That gamble has a 50% chance of leaving you at 0 and a 50% chance a leaving you at point B.
So your utility and expected value is the midpoint of that chord that runs from 0 to B or point C. Your expecting utility is lower than your initial utility.
Why?
Because utility is concave.
You are made so sad by getting to 0 that it vastly overcompensate the happiness you feel moving to $225 because of the diminishing marginal utility.
Because, basically, think of it this way.
Imagine it's your actual income.
Let's take the point about the size of the gamble relative to income seriously.
Imagine, literally, I was asking you to gamble your entire income for the year.
And if you lose, you starve to death.
And if you win, you get to eat extra nice.
Well, clearly, the disutility of starving to death vastly outweighs the extra utility to eating well.
So, in that extreme example, if this was your entire wealth, you can see why you would have a situation where you wouldn't want to take that gamble.
Because if you lost, you'd die.
And, basically, risk aversion arises because, basically, with diminishing marginal utility you're made so much sadder.
That steepness at the bottom, you get so much sadder as you get towards 0 that it vastly overcompensates the flatter part as you move above your initial point.
So, as you can see, you are going to end up not wanting gambles even if they're fair.
Gambles that are fair, that is positive expected value, might still lead to a reduction in your expected utility.
Indeed, let me go further.
You dislike this gamble so much that if I said the following, I as your teacher am going to force you to take this gamble-- imagine it's like 100 years ago where teachers can beat students and stuff-- I'm going to force you take this gamble unless you pay me, you would actually be willing to pay me to avoid taking this gamble.
How much would you pay me?
Imagine utilities in dollar terms.
Imagine we're actually measuring utility in dollar terms.
How much would you pay me to avoid taking this gamble.
If I said you either take the gamble, or you pay me.
You're starting with a utility of 100.
Yeah?
The difference between the two utilities
Well, the difference between the two utilities.
So utility is 100 here.
Here utility is 7.5 squared, so 56.25.
So you would actually pay me $43.75 to avoid taking this gamble.
Think about that.
I've offered you a more than fair bet, a very good bet, which, on average, will yield you a positive $12.50.
Yet you will pay me $43.75.
You will pay almost half of your entire wealth to avoid taking that gamble.
That's pretty incredible if you think about it.
I've offered you a more than fair bet, and yet you will pay me more than half your wealth, almost half your wealth, to avoid taking that bet.
So another way to see this, let's look at this another way.
How large would I have to make the positive payoff for you to take the bet?
Let's look at it that way.
Right now I said you win $125 with heads.
How much would you have to win with heads if you were going to take that bet?
Yeah.
And tell us how you figured that out
Because you need to have at least the same utility as you had before from the unexpected utility.
So more than half of his per year utility would be 20 if he wins.
20 squared is 400.
[INAUDIBLE  PHRASE]
Right.
You'd need to win 300.
Because I'd need to take your utility to 20 if you win.
Only then would you be willing to take this gamble.
So another way to say it is that's how fair a gamble would need to be, how more than fair it would need to be before you take it.
You'd need me to pay off 3:1 on a 50% chance before you'd take the bet.
And this is just with a typical looking utility function of the kind we worked earlier in the semester.
You didn't look at this earlier in the semester and say, wow, that's a bizarre utility function.
We got sensible answers on our problems, and problem sets, and tests, and things, examples from square root of c. That seemed like a sensible function.
And yet it yields these incredibly wild predictions that you would pay people almost half of your wealth to avoid engaging in a more than fair bet.
And that you would need the odds to be like 3:1 before you even consider taking a a bet.
That's the power of uncertainty and the power of risk aversion.
Really, risk aversion, it's just the power of diminishing marginal utility.
The power of diminishing marginal utility is so key to driving our decisions.
It's the fact that that first pizza means so much more to you than the fifth pizza, that you really hate outcomes that don't let you get the first pizza.
And, as a result, you will pay a lot to be forced into a situation where you don't get any pizzas.
You'll need to be paid a lot in the state where you do win to deal with the state where you don't.
Questions about that?
Now, we can change the example in some interesting ways to understand it.
So let's change the example to say, instead, let's talk about some alternatives to this example and how they affect our intuition.
First alternative, imagine your utility function instead of being square root of c, your utility function was 0.1 times c, a linear utility function, not a non-linear utility function.
We can now say that, in that case, you actually would take the gamble.
There's a 0.5% chance of 0.
And I chose 0.1 times c, because your initial utility is still 10 then.
I normalized this.
So starting with your bundle of 100 you still start at 10.
It gives the same starting point as the square root function.
But now your expected utility from his gamble is 0.5 times 0 plus 0.5 times if you win 125, your utility is 12.5.
I'm sorry.
It's 22.5.
So your expected utility is 11.25 which is higher than your starting utility.
So you would take this gamble.
What's changed?
No diminishing marginal utility
No diminishing marginal utility because now we are no longer risk averse.
We are what we call risk neutral.
A linear utility function yields risks neutrality.
And once you're risk neutral, you only care about expected value.
Risk neutral consumers would only care about expected value.
And so a linear utility function will lead to risk neutrality since you don't have diminishing marginal utility.
Then you take any bet that's fair.
You don't care.
You're indifferent between winning a dollar and losing a dollar with this utility function.
It doesn't matter if you go up or down.
The joy you get from winning is the same as the pain you get from losing.
Whereas with this utility function, the pain you get from losing exceeds the joy from winning.
We can see that graphically in the next figure, Figure 20-2, the case of risk neutrality.
Here, you start at point A.
You have 100, and your utility is 10.
Now, I've offered you a gamble where there's a 50% chance of getting 0 and a 50% chance of getting B.
Well, that yields an outcome of c, which is a higher utility.
So since your utility is linear, you're risk neutral, and you'll take any fair bet.
We can go further.
What if utility, instead, was of the form u equals c squared over 1,000?
What if this was your utility function?
Once again, your initial utility u of 100 is 10.
It's the same starting point.
But this is a utility function which now if you do this gamble, your expected utility is 50% times 0 plus 50% times $225 squared over 1,000 which is 25.3.
That's a huge increase in utility from this gamble.
So your expected utility with the gamble is 25.3.
It's a huge increase in utility.
And that's because this is an individual where the shape of the utility function has change where they don't have diminishing marginal utility, they have increasing marginal utility.
We've never worked with utility functions like this before.
These are individuals we call risk-loving.
That is, they are made happier by winning $1 than they are made sadder by losing $1.
It's the opposite of all the intuition we developed earlier in this course.
It's a crazy utility function.
But the notion of a risk-loving utility function is one where literally $1 that moves you up makes you happier than $1 that moves you down makes you sadder.
You can see that in Figure 20-3.
Here's a risk-loving utility function.
The individual starts at point A.
They have a choice of a gamble where they can have a 50% chance of landing at 0 and a 50% chance-- Jessica, that B should be down at the intersection of dashed lines-- a 50% chance of landing at B at the intersection of the dashed lines.
You take the average of those two, and it's c. Their utility is way higher with the gamble than it was without the gamble.
In fact.
we can go further.
With a risk-loving person, they would actually take an unfair bet.
Consider the following bet.
Tails you lose $100, heads you win $75.
That's a bet with a negative expected value.
Neither the risk averse nor the risk neutral person would take that bet.
But a risk-loving person would.
If you work out the math, that bet gives them a gain in expected utility.
That is a bet with a negative expected value that gives them a gain in expected utility.
Why is that?
Because it's the opposite of diminishing marginal utility intuition.
They're made so much happier by winning that they're willing to take a bet even if it's a negative expected value.
Just like the risk averse person is made so much sadder by losing, they won't take a bet even if it's more than fair.
So you can actually develop all the opposite predictions from a risk-loving person.
They'll even take unfair gambles.
Now, by the way, I skipped over your earlier question about risk neutrality.
With risk neutrality, you see it doesn't matter if you do it 100 times or one time.
If you're risk neutral, you should take the bet anytime, because the expected value is still positive.
Now, you're thinking about risk aversion where, in substance, you're more confident as the numbers go up.
But if you're risk neutral, you'll take it no matter how many times I offer you that bet.
So to extend this further, let's go to a third extension which will develop this intuition further.
Now imagine that I offer you guys a different gamble.
And, once again, I really want you to answer honestly.
Don't try to game me.
Answer honestly.
Now the gamble is if I flip a coin, tails you lose $1, heads you win $1.25.
Now how many of you would take that gamble.
How many would not take that gamble?
OK.
I hope you're answering honestly.
But maybe you're just thinking ahead and realizing that that gamble is very different.
And why are people more willing to take that gamble than they were willing to take the previous gamble, the same risk averse people.
Yeah
The difference in [INAUDIBLE PHRASE]
Exactly.
In particular, the utility function is locally linear.
Let's go back to Figure 20-1.
As you get closer and closer to A, you could draw, essentially, a linear segment.
So for an infinitesimal bet, utility is linear.
So it's linear at point A.
So for small bets, you become risk neutral.
Even a risk averse person moves towards risk neutrality as the bet is small relative to their resources.
This was the point that you were making.
Basically if you're a rich person, you'd probably be happy to take the $100 and $125 thing.
I'd be happy to do that.
I'm a rich guy.
I'd be happy to do that.
So, basically, what determines your willingness to take a bet is going to be about what's at stake relative to your resources.
And what you can see is that if you solve the math here, that basically expected utility even with a square root of c is positive for that smaller gamble.
Because as it gets smaller relative to the $100 you start with, you become roughly risk neutral.
And then you'll go ahead and take the gamble.
So at the end of the day what's going to determine whether you're going to take a gamble is going to be your level of risk aversion and the size of the risk you're taking relative to your resources.
The more risk averse you are, and the bigger the gamble, the less likely you are to take it at a given level of fairness.
Questions about that?
All right.
So now that we all understand expected utility theory.
Now we're going to go on and talk about why this matters in the real world and how we use it.
And I want to talk, in particular, about two applications, insurance and the lottery.
Let's start by talking about insurance and why people have insurance.
Because, in fact, given what we learned in this lecture, there would be no reason for insurance.
This lecture tells us why people have insurance.
Because there's diminishing marginal utility, and you're made so much sadder with a negative outcome, you're willing to pay you avoid it.
Remember we talked about that you would be willing to pay almost $44 to avoid being forced to take that bet?
That's what insurance does.
Insurance allows you to avoid taking gambles.
That's what you can think of insurance as.
It's a way to avoid taking a gamble.
You're gambling you're going to get sick.
You're gambling your house is going to burn down.
These are gambles you face that are forced on you by nature.
What insurance does is allow you to avoid taking those gambles.
And just like you'd pay me to avoid the $100, $125 gamble, you're paying Aetna to avoid gambling that you might have to go to the hospital.
So let's say there's a 25-year-old who is deciding whether to buy health insurance.
And let's say they're 25-year-old guy, totally healthy.
I say guy because there's no risk they're going to have a kid.
So he's basically totally healthy, basically zero chance they're going to use the doctor except if they get hit by a car.
So imagine the situation is that you've got 25-year-old with an income of $40,000.
And let's say that there's a 1% chance that they'll get hit by a car.
It is Cambridge after all.
So every time you cross the street, there's a 1% chance you get hit by a car.
And if you get hit by a car, you're going to suffer $30,000 in hospital bills.
And let's say your utility function is square root of c. So you're a risk averse guy.
So let's say that I then come to you and say, look, each year there's an expected cost to you of getting hit by car of $300.
How did I calculate that?
Well, every year there's a 1% chance you get hit.
They're independent draws, let's say.
If you get hit this year, it doesn't mean suddenly you're safer.
It's random.
It's just crazy drivers.
So there's a 1% chance you're going to get hit every year.
And if you get hit, there's a $30,000 cost.
So every year there's an expected cost to you-- the opposite of expected value is expected cost-- of $300.
So let's say I offered to sell you insurance for $300.
I offered to sell you insurance in a way where, on average, if you lived an infinite number of years, you would pay out in premiums what you'd get in benefits.
If you paid $300 a year and lived forever or lived for many, many years-- the law of large numbers enough years-- then basically you would pay out in premiums what you would collect in benefits.
You'd get hit once every 100 years.
And ever 100 years you would have paid $30,000 in premiums, and you'd collect $30,000 in benefits.
So that's what we call actuarially fair insurance.
Actuarially fair insurance is insurance where the price of the insurance equals the probability of the bad outcome times the cost of the bad outcome.
That's actuarially fair insurance where the price you pay is the probability of the bad outcome times the cost of the bad outcome.
That's fair because, over a large enough population, the premiums that get paid in will get paid out in the form of claims.
Now, let's ask what is your utility if you do not or do buy insurance.
So for the first thing you say if I'm a 25-year-old.
Screw it.
I'm never going to get hit by a car.
I'm not going to buy insurance.
What's your utility with no insurance?
Well, if you have no insurance, there's a 1% chance, 0.01, that you'll lose $30,000.
You'll get hit by a car and lose $30,000.
Your income is $40,000.
So there's a 1% chance that you'll end up with a utility, which is the square root of 10,000.
And there's a 99% chance you'll end up with a utility that's the square root of 40,000.
You work this out, and the answer is you get 199.
Utility without insurance is 199 which is pretty close to utility just if you weren't going to get hit by the car.
Because it's so rare that you get hit by the car.
So utility is 199 without insurance.
Now, let's ask the question, how much would you be willing to pay to have insurance?
How do we figure that out?
$300 is the actuarially fair premium.
But now let's do a different question.
I'm an insurance company, and I want to make money.
I don't want to just charge the actuarially fair premium.
The insurance company makes no money with this premium of $300.
So the insurance company wants to make money.
How would we figure out how much would you be willing to pay, this 25-year-old, be willing to pay to get insurance?
How do we figure that out?
Yeah
Maybe you could keep the utility function constant
Keep the utility value constant
Value, yes
Exactly.
You'd have ask well, how much would I be willing to pay to have insurance which would protect me and leave me at the same utility level.
Obviously it would have to be a little bit higher.
But let's just set it equal.
So, in other words, if I bought insurance, my utility with insurance, there's a 1% chance that I will get hit by the car.
In that case, what happens to me?
Well, if I get hit by the car, I get $10,000.
I make $40,000.
I lose $30,000.
Let me actually write it out.
If I get hit by the car, what happens to me?
Well, I make $40,000.
I always make $40,000 each year.
I lose $30,000, because I get hit by the car.
But then the insurance company pays me $30,000.
They pay off my debts.
So then I gain $30,000.
So these things cancel.
But I have to pay the insurance company premium.
So I have to pay some amount x.
If I don't get hit by the car, I get my $40,000 income, but I still have to pay the insurance company premium.
I have to pay them whether I get hit or not.
It's insurance.
I pay them either way.
So that's my utility.
So my expected utility with insurance is the sum of these two.
And I want to set that equal to 199.
I want to say what x am I willing to pay that would leave me at the same utility as if I was uninsured as per the answer here?
Well, it turns out that if you do, that if you solve this, you get that x equals 399.
That is you would pay $399 for insurance that has a value of only $300.
You'd pay $399 for insurance even though the actuarially fair price is $300.
You would pay you insurance company $99 more than they expect to pay out to you.
Why?
Because you're risk averse.
Because you're made so much sadder than being left with $10,000 than you are by having to pay $300.
If it doesn't work out, you pay $300.
Who cares?
That's tiny compared to your income.
But if it does work out, you're safe from having to starve.
You pay $400, I'm sorry.
You pay $399.
You're like, look, I'll be bummed if I have to pay $400.
That's a percent of my income basically.
That would be a shame to pay a percent of my income for something that doesn't happen.
But, boy, would I be happy in that 1 in 100 chance where I get hit by a car when I'm not out $30,000.
So you will pay $399 for insurance that's only worth $300.
That extra $99 we call a risk premium.
We call that a risk premium.
The extra $99, we call a risk premium.
That is the amount that you are willing to pay above and beyond the fair price, because you're risk averse.
And what you should go home and show yourself using the same kind of mathematics is that, for example, the risk premium will rise the bigger the loss is.
Hopefully you can see the intuition on that.
The bigger the loss is for a given level of income the bigger the risk is.
Likewise, for a given loss, the risk premium falls with income.
So the bigger is the loss of relative to income the more risk premium you're willing to pay.
You should also, obviously, see that the more risk averse you are, the bigger premium you're willing to pay.
A risk neutral person would not pay a risk premium.
Only a risk averse person will.
So the more risk averse you are, the bigger risk premium you'll pay, and the bigger the loss is relative to your income.
These are the same principles we talked about before.
So the $43.75 we were willing to pay to avoid that gamble I was going to force on you, that was the risk premium.
You were willing to pay $44 to avoid that gamble.
Here, you're willing to pay $99 to avoid the risk of ending up in that bad state where you get hit by the car.
And that's why people buy insurance.
And that's why insurance companies make ungodly amounts of money.
In the US we have a health insurance industry, for example, that earns about $800 billion a year.
Why do they make all that money?
Because people are risk averse, and they're willing to pay to have someone else bear the risk of their injury or illness.
Any questions about that?
Now, I don't mean by that to say, insurance is a bad thing, and we shouldn't do it.
Risk aversion is the nature of our utility functions.
We should be willing to pay a risk premium.
It's just that you need to understand why, in fact, it makes sense to have insurance in that case.
The second application is the lottery.
The lottery is a total ripoff.
I hope you knew this already.
The expected value of $1 lottery ticket is roughly $0.50.
So for every $1 you spend in the lottery, in expectation, you get about $0.50 back.
This is an incredibly bad bet, incredibly unfair, an incredibly unfair bet.
On average, you lose $0.50 for every $1 you bet.
So, basically, despite that, lotteries are wildly popular.
They've become a huge source of revenue for state governments.
A lot of the money that state governments now take in is through state lotteries.
What accounts for the fact that lotteries are so popular?
Well, there's four different theories for why lotteries are so popular.
The first is that people are risk-loving.
We have it all wrong.
Actually people like taking risks, and the lottery feeds that.
This, of course, we can immediately rule out.
How?
How do we know this is wrong?
That the answer is that people play the lottery because they're risk-loving.
How do we know people aren't risk-loving?
The same people don't take [UNINTELLIGIBLE]
And they spend $800 billion a year on health insurance.
Basically, as a society, we spend, in total, about $1.5 trillion a year on insuring various risks that face us.
We're not risk-loving.
So that's clearly not the answer.
However, there's an alternative.
People could basically alternate between risk-loving and risk-aversion.
This is a theory due to Milton Friedman, the famous economist from Chicago and a co-author named Savage, the Friedman-Savage preferences, where the notion is that basically people are risk averse over small gambles but risk-loving over large gambles.
So to see that, go last figure in the graph.
This is sort of a complicated case.
Basically, the notion is if you take someone, they have a utility function which is initially risk averse and then becomes risk-loving.
That is in the segment between W1 and W3, that looks like a risk averse utility function.
But once you get above W3, it looks like a risk-loving utility function.
So the notion is that for things which can make me very poor, I'm risk averse.
I want to insure against events which will leave me in that bottom segment.
But once I'm going to be above W3, then great.
I'm happy to take risks.
Then I become risk-loving.
Now, this is a not crazy idea.
Graphically, what I'm showing you here, is that b* is utility without the gamble and b is with.
So you see you're happier without the gamble when your income is low.
Once your income is a lot higher, you're happier with the gamble at d then you are without the gamble at d*.
That's not a crazy theory.
The notion is that once I'm rich enough, I become risk-loving.
But when I'm poor, I don't want to take the risks.
The problem is that this is inconsistent with lottery behavior in the following sense.
Most people who play the lottery don't play the Mega Millions.
They play tiny scratch lotteries where you bet $1 to win $10.
And people spend huge amounts of money on lotteries with very, very low payoffs.
That is inconsistent with this.
Because this would say that you'd only play lotteries that have big payoffs.
Lotteries that have small payoffs, once again, there's no reason to play that and still buy insurance.
So if you're buying insurance against being low income, why are you playing these small lotteries that are a ripoff.
Because those small ones are a ripoff too.
So the existence of the fact that the most popular lotteries are actually the small lotteries is inconsistent with this explanation.
Yeah
So I'm confused.
Is it risk-loving on large gambles?
Yeah, risk-loving on large gambles.
It's not the size of the gamble.
You're risk-loving on gambles which leave you in a high wealth state.
The point is that if I'm gambling over winning Mega Millions.
Yeah, I'm a little risk averse.
But the truth is winning Mega Millions would make me so happy that I could move into the risk-loving part of my utility function.
But this would not explain why people ever play something that pays off $100.
This is a fancy way of the intuition you probably have.
It's I'd think differently about something which would completely change my life and make me a multi-billionaire, that's something that would make me raise me, than the bet I offered you guys before.
People are systematically taking terrible bets like the kind i offered you guys before.
And that's inconsistent with these preferences.
The third explanation is entertainment.
It's that the utility function has in it the thrill of the risk.
We only write down utility functions that are a function of consumption like how many pizza and movies you see.
But people have utility over lots of things.
One thing you may have utility of the thrill of being able to scratch the thing off and seeing if they won or not.
That would actually be consistent with the fact that people play a lot of small lotteries.
If it's a thrill of winning that matters, if it's the scratch off thrill that matters, then the optimal thing to do, in fact, would be to not play one Mega Million.
It would be to play lots of little lotteries.
And that would be consistent with that behavior.
So one story that is consistent with what we see is that people actually view this as entertainment.
On the other hand, once again, it's really expensive entertainment.
Because you're throwing away $0.50 of every $1.
So you've got to get a lot of enjoyment out of that scratch off relative to when you go to see a movie.
So that's another theory.
I'm going to put this in here.
It sort of inserts in here.
We talked about the fact that people can't be risk-loving because they buy insurance.
And this alternating thing doesn't work, because they play small lotteries.
But another theory that might fit here is a theory we call loss aversion.
This is sort of a different version of the Friedman-Savage preferences.
It's that people are, in general, risk averse.
But, in fact, they're really risk averse on the downside, and they don't care so much on the upside.
So, in other words, the point is that when I initially offered you that bet of win $125, lose $100, part of your reaction was about the risk aversion.
But a lot of you are thinking, I'd be really bummed if I lost $100.
It's not just that I don't have it to spare.
It's just like, god, I would kick myself.
It was one flip of the coin.
How could I possibly have been so stupid?
Whereas if you won, you'd be happy.
But then you'd go on to the next class.
The notion is that basically it's an extreme version of risk aversion.
It's not only that you're risk averse, it go further than that.
Relative to the starting point, anything which is a loss really pisses you off.
So, in fact, even that little gamble I offered you, win $1.25 lose $1, you still might not take.
Some of you still wouldn't take it.
And the reason you wouldn't take it can't be risk aversion.
Because it's just too small for risk aversion to plausibly work.
It's that you'll just be bummed that you did that and you took that chance.
You'd be made sadder by the loss than you'd be made happier by the win.
In that case, that could explain why people spend a lot of money to buy insurance.
Because they'll be so bummed if things go badly.
But they might play the lottery because, in fact, around that point, they don't view the money they're spending as a loss.
They think of it differently.
They think of the loss of being my house burned down.
That's a loss.
That would make me really sad.
But the $1 I paid to pay the lottery, that's not really a loss.
So I'm risk neutral going up and really risk averse going down.
So I'm willing to take gambles that push me up.
It's sort of like Friedman-Savage.
I'm willling to take gambles that push me up, not gambles that pull me down.
But, once again, that doesn't really explain the small ones.
That doesn't really explain the small ones.
That's more the entertainment theory.
Then finally, the last theory we have is that people are stupid.
The lottery is, after all, its official motto is a tax on the stupid.
And that's what it is.
It's a tax on the stupid.
Basically many of your public schools are financed by taxes paid by stupid people.
It's sort of ironic.
But people just don't know.
You probably all had a vague sense that the lottery wasn't a sensible thing to play.
But how many people actually knew it was that bad a deal as I said.
That is actually was $0.50 expected payoff.
A few of you knew.
But most of you knewm had a vague sense it was a bad deal.
You didn't know how bad a deal it was.
This is sort of hard to figure out.
Meanwhile, you see on TV that these guys win these bazillion dollars, and you get the thrill of scratching if off.
So, basically, if people are just stupid, then that could explain it.
The problem is it matters a lot for government policy which of these is right.
Because if A through C is right, if one through three are right, then the government should go ahead and allow lotteries.
And there's no reason why the state shouldn't run a lottery.
In fact, let's take the entertainment theory.
If this is really entertainment, and the state can make money off of my entertainment, then that's a win-win.
I'm happy, because I'm playing the lottery.
The state is happy, because it's financing schools.
That's a win-win.
So if these are right, you're going to want to encourage state lotteries.
But if this one's right, we don't want to have them.
Because, A terrible way to raise government revenues is to tax stupid people.
There are much better ways to raise government revenues.
We'll talk about taxation in a couple of lectures.
But, clearly, taxing the stupid is not going to be an optimal tax.
Yeah
I can maybe sort of understand why people would prefer smaller lotteries over bigger lotteries.
Because they are thinking that in smaller lotteries, they have a much bigger chance of winning than in bigger lotteries.
So, in that sense, their expected payoff in terms of utility or other [INAUDIBLE PHRASE] is a lot higher than the antes in the bigger ones, even though the bigger ones might end up being a lot heavier--
So that's sort of an entertainment theory, which is my utility derives from the win.
You have a theory in mind my utility derives from the win.
Because if it's just about dollars, that wouldn't explain it.
Because I win so many more from the big one that it would compensate from the frequency at which I'd win the little one.
But if I actually, in my utility function, have the joy of seeing that winning thing, then that would explain it.
That's an entertainment theory.
You're saying, in my utility function, I actually get joy from scratching off and seeing that it's a winner, and so much joy that I'd much rather take a 10% chance at a small win than a 1% chance at a huge win.
Because then, at least, with the first one, 1 in 10 times I get that joy of the scratch off and seeing it's a win.
So that's sort of an explanation.
And that would say that lotteries are good.
The other way economists might think about lotteries is they're voluntary taxes.
The public doesn't like taxes.
Here's a voluntary tax.
You never hear policy makers getting up and railing against a horrible evils of the lottery.
Sometimes groups do.
Sometimes outside groups do and stuff.
But politicians don't.
But those same politicians will go on and on about how terrible taxes are.
I'm going to cut your taxes.
Taxes are terrible.
Well, the lottery is a voluntary tax in that sense.
And I might say, look, there's no reason to oppose it, it's a voluntary tax.
It's those involuntary taxes that cause problems in society.
Well, whether we want to buy that story or not depends on how much we think it's being played because people are stupid or not.
OK. Let me stop there.
So that's a great example of how a little bit of an extension of our model can really enrich our understanding about a lot of decisions that we make in the real world.
We'll come back and talk about another version like that later.
And that is the case of thinking about savings decisions and thinking about individual decisions on how much to save and how much to spend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Our topics part of the course by revisiting another important topic that-- one of the most important topics in economics which is where does capital come from.
This is a topic that is essential in economics.
It's also really central, the basis, for what is in finance.
So a lot of what we'll do today is really-- if you want to learn more about it you take more in economics but also more courses in Course 15.
So, basically, we spent a lot of time this semester talking about one input to the production function which is labor.
We talked about labor supply, labor demand, monopsony models, et cetera.
We haven't talked much about the other input into the production function which is capital.
Now, partly, that's because that's a more awkward concept.
It's clear what labor is.
Labor is the workers working in the production process.
Capital's a little bit harder because capital's sort of everything else-- the machine, the land, the buildings, other physical inputs.
And we know where labor comes from.
Labor comes from our working.
But it's less clear where capital comes from in some aggregate concept.
So, basically, the key thing is that all forms of capital have a common feature.
All forms of capital have a common feature which is what capital represents is a diversion of current consumption towards future production and consumption.
So capital's about diverting current consumption towards future production and consumption.
So the original concept of capital came from farming where the notion was that farmers every year would take some of their grain, and, rather than eating it, they'd put it aside to become seed to plant for the next year.
That was their capital.
And so they diverted their consumption this year which was eating the grain they grew to produce future consumption through planting those seeds and creating consumption for next year.
So, basically, in a modern economy the idea is the same.
And so when we think about capital, what I want you to think about is I want you to think about, basically, the capital as money.
Think of the capital in the production function as the money that we invest in all these other things that aren't labor-- that we invest in machines, and we invest in buildings, and we invest in land.
So we want to think about capital not as physical capital but as financial capital.
That's the way to be thinking about that one aggregate letter k is this financial capital.
It's the money that's invested in producing goods, in building machines, and building buildings, and stuff like that.
OK?
Now, basically, where do firms get that money?
So you're a firm.
You want to build a building or new machine or something like that.
Where do firms get that money?
They get that money in capital markets.
Capital markets are basically pools of money that firms draw on to invest and create capital.
So a capital market-- literally think of it as a pool of money that's out there that firms can tap into if they want to build a building or build a machine or buy some land.
They tap into capital markets to make those investments.
So while capital physically represents lots of different things, financially it represents one thing which is the pool of money that firms tap into to invest, to divert current consumption to future consumption.
OK?
It's the pool of money firms tap into to invest.
That's what we mean by capital.
By capital market we represent that pool of money.
So think of capital as financial capital, and think of where you get financial capital in a capital market as being that pool of money that firms tap into to make investments to divert towards the future.
Now, where does that supply of money come from?
Where does that supply of money come from?
Well it comes from households' decisions on how much to save.
OK?
So the pool of money in capital markets, the pool of money that firms draw on to build capital, comes from households' decisions on how much to save.
So now we see the tie to labor, the other input in the production function.
Just as households' decisions on how hard to work determines the labor input into the production function, households' decisions on how much to save determines the capital input into the production function.
So just as my decision how much to work determines how much labor is available to firms, my decision on how much to save that's what fills up this pool.
OK?
So this pool of financial capital is filled up by household savings and then drawn down by firms' demand for investment.
And that's the way a capital market works.
So, basically, if we think about capital market equilibrium-- if you go to Figure 21-1-- we have equilibrium in capital markets.
Now this is just like we talked about-- this is the other factor markets.
Just like we talked about labor markets and determining what determines the wage rate and the optimum amount of labor hired, it's same with capital markets.
You have some demand for capital.
That comes from firms' demand for investment.
Firms want new machines.
They want new buildings.
That's downward sloping because, initially, there's very high demand for capital.
But there's a marginal diminishing product.
The more capital I have the less valuable it is on the margin, the less you are willing to pay for it.
So there's a downward sloping demand curve for capital and an upward sloping supply curve.
And the price, in this market, is the interest rate.
What is the interest rate?
The interest rate is the rate you have to pay households to get them to lend you money.
So the interest rate, i, is the rate you have to pay households to get them to lend you money.
So if that interest rate is very high, firms will not demand much investment because they'll have to pay a lot of money to get the financial capital to finance that investment.
But households will be delighted to supply lots of savings because they're getting paid a high price for it.
So, basically, the interest rate serves as the equilibrating price in this market.
Just as the wage serves as the equilibrating price in the labor market, the interest rate serves as the equilibrating price in the capital market.
As the interest rate rises, folks want to save more, filling more money into that pool of capital.
And firms want to borrow less, taking less money out of that pool of capital.
And when that supply and demand is equilibrated, at point e, is going to be where the firm's drawing on the pool at exactly the rate people are putting money into the pool.
And that's going to be the equilibrium.
OK, so we want to focus on-- for today's lecture and next lecture as well-- is what determines the money that goes into that pool.
We know what determines the rate at which firms want to draw out of that pool.
That's basically going to be determined by the production function and all the stuff we learned in lectures on production theory.
You can get your optimal demand for capital.
It's going to be determined by isocosts and isoquants.
And you get some k star.
But what's going to determine what goes into that pool?
That's going to be households' decisions to save.
And households' decisions to save, we say, are determined by a process we call intertemporal choice.
Added some extra letters there-- intertemporal choice.
Intertemporal choice-- which is basically, instead of thinking about someone choosing between apples and bananas, we think of them choosing between consumption today and consumption tomorrow.
So think of different periods like different goods.
And I'm choosing between consumption today and consumption tomorrow.
That's my intertemporal choice-- the rate at which I choose to trade off consumption in different periods.
So for example, I'll illustrate how this works.
Let's say that I'm deciding whether to just tell MIT, "Look, I don't want to work next year.
I want to stay home and take care of my kids.
You're not going to pay me.
I'm taking an unpaid leave for a year."
Something professors can do with enough advance warning to their chairman and such.
So I'm going to take an unpaid leave next year.
I'm thinking about doing that.
And now I have to say, OK, fine.
Next year I'm going to take this unpaid leave so I have to decide how to allocate my-- and then I'm going to come back and life will be the same thereafter.
So it's just about next year I'm going to take this unpaid leave.
I have to decide how to allocate my consumption across this year while I'm working and next year while I'm taking an unpaid leave.
And let's say my salary is $80,000 a year.
So one thing I could do is I can consume all $80,000 this year and consume nothing next year.
That would not be a very satisfactory outcome as I die.
That's obviously not going to be a satisfactory outcome.
But what's the alternative?
MIT's paying me this year, and they're not paying me next year.
What's the alternative?
Well, the alternative is I can save.
And by saving, what we mean is I can loan some of the money that I make out to firms to invest in their physical capital in return for which they'll give me interest.
And next year I can live on the interest I've earned from making that loan.
Now, I don't literally go to Genzyme and Microsoft and Apple and say I want to loan you money and negotiate with them.
That obviously would be impossible.
What I do is I implicitly loan to firms through drawing on various aspects of the capital market.
So does anyone know, how can I implicitly loan to a firm?
Let's say I want to-- Yeah?
Banks
Banks.
So explain what you mean
You deposit money in a savings account for the bank.
The bank pays you some interest rate so that it can use your money to loan to bigger companies that want to take money out of the bank.
And they, in return, get money from the other companies by the companies paying some interest on what they took out
Exactly.
We call banks financial intermediaries.
What that means is they basically are the folks who can get a hold of firms and make those loans.
So, in other words, I don't loan directly to Genzyme.
I loan to the bank-- Citizens Bank, my bank, and Citizens Bank loans to Genzyme.
Citizens Bank pays me an interest rate on my savings-- now close to zero, we'll come to that-- but basically pays me some interest rate.
Genzyme pays them an interest rate to borrow money, higher than what they're paying me, and the difference is bank profit.
So, basically, one way I can loan money is I can put in the bank and get paid interest.
we don't think about putting in the bank as a loan, but that's basically what you're doing.
You're loaning it to the bank.
And they're paying you interest rate, i for that loan.
What else can you do?
How else can you-- Yeah?
You can purchase stocks
You could purchase stocks.
So, in other words, what I could do is I could directly go to a public company, and I could take some of my $80,000 and buy stock in that company.
There I'm essentially directly loaning to them.
I'm directly giving them money.
Now, it's not a loan that's paid back like a bank loan.
It's loan that's paid back hopefully with my stock becoming more valuable or with a dividend.
So how can I loan?
One is I can invest-- I could put it in the bank.
The other is I can buy stock.
I can put it in the bank, and the bank pays me interest.
I could buy stock, and that stock pays off in two ways.
One is many companies pay what we call a dividend, what is called the dividend, which is a quarterly payment that companies make to their shareholders.
So if I invest in a company that pays a dividend, then I'll be getting a quarterly check from that company that's a portion of my investment.
The other is what we call a capital gain which is the stock could go up in value.
So next year, if the stock goes up, if the stock market moves steadily-- it doesn't, it jumps up and down, we'll come to that-- but if it went steadily up I could just sell some of that stock next year and have extra money.
So that's the other thing I could do with my $80,000.
I could loan to a company by buying their stock.
How else can I loan to a company?
Yeah?
I don't know exactly what the difference is, but couldn't you also invest your money in a mutual fund or something?
A mutual fund-- that's a good point.
That would be loaning-- a mutual fund is essentially loaning money to an aggregate collection of companies.
So there are very different ways I can do stock.
I can do a mutual fund, I can buy individual stocks.
There's lots of different ways, but those are all different ways to buy stock.
Yeah?
You could buy bonds
You could buy bonds.
You could buy company bonds which is I literally loan directly to the company.
Stocks aren't really a loan.
I'm literally buying an ownership share in the company, and they're paying me back.
I could buy corporate bonds.
I could buy corporate bonds, and the way those work is it's literally cutting out the middleman.
I don't loan to Citizens Bank, and they loan to the company.
I just loan to the company, and they pay me back.
That's a corporate bond.
I can also buy, by the way, I can also buy a government bond.
You may know the government's running more than a trillion dollar deficit right now.
Somebody's got to finance that.
So you can loan to the government and get paid back by the government.
OK, let's put the government aside for a minute.
Let's focus where we just-- we haven't really had a government sector in our models.
We're where it's just you and the companies.
So let's leave the government channel aside.
But the other thing I can do with my money is I can loan it through bonds.
The point is that $80,000-- yeah, I'm sorry
With stocks, if I were to buy a stock from a company, wouldn't it be primarily and usually through the secondary market?
I wouldn't be giving any money to the company.
I'd just be giving money to the previous stockholder
That's true.
It basically depends on whether the marginal stock comes from a new issuance as stock by the company or through stock that's already floating around the secondary market.
That's a good point.
So in some sense the-- likewise with bonds.
A lot of bonds are traded in a secondary market.
So I'm thinking about a simple model where basically new stock gets issued by the company, I buy it.
More technically you're right.
It's just trading among people, but that sort of makes things complicated.
Let's put that aside for now.
So, basically, the point is is my $80,000-- there's lots of things I can do with it.
All of them yield me some rate-- the key point is all of them have the feature that I'm diverting today's consumption for tomorrow's consumption.
I'm taking some of my money and, rather than eating it this year when I'm working, I'm loaning it out in some way, shape, or form and getting payback in next year when I'm not working.
And we can summarize.
Now this is a very complicated set of mechanisms, not to mention the secondary market issues.
And this is basically a semester of 15.401.
This is basically a semester of finance theory.
But basically what we're going to do is compress this all down and say that I get some interest rate on my money.
However I do it, let's just say that somehow I divert my money through one of these mechanisms, and it yields some effective interest rate, i.
And you can know behind that there's lots of ways I can get that interest.
But for now just simplify it down and say the main thing is I'm diverting my consumption now, and it's yielding some interest earnings on that diverted consumption, i.
So what that means is that for every dollar I divert, I get 1 plus i dollars the next year.
So for every dollar of consumption I divert, in one of these forms, I get 1 plus i dollars next year.
So, basically, I could literally-- if I wanted to-- let's say the interest rate was 10%, just for example.
Now what that means is instead of consuming $80,000 this year and nothing next year, I could consume nothing this year and $88,000 next year.
Obviously, that's not very satisfactory either.
So how do we think about that?
We think about that in Figure 21-2 shows-- now this is a complicated diagram we gotta use to figure 21-2-- this shows the intertemporal choice model, intertemporal substitution we also call it.
So the deal is that now instead of the x-axis being pizza and the y-axis being movies or all the other wacky things we've done, now the x-axis is first period consumption.
The y-axis is second period consumption.
You might say what's a period?
Well a period's whatever I want it to be-- a day, a year, 10 years, whatever.
Sometimes I'll say a year.
Sometimes I'll say a period, but the point is it doesn't matter.
It's about the trade-off.
So in my example, c1 is consumption this year.
c2's consumption next year.
And my trade-off is I can consume $80,000 this year, or, given the interest rate, I can consume $88,000 next year.
Now the trade-off-- that's a typo, by the way.
That should be minus 1.1.
OK?
This is a 10% interest rate.
The key point is the trade-off is that, basically, I can trade off for every dollar I don't consume this year, I consume 1 plus i-- 1 plus r there, should be 1 plus i dollars-- next year.
We use r and i interchangeably for the interest rate, so 1 plus i, 1 plus r dollars next year.
So, basically, what does the interest rate represent?
This is important.
The wage rate I defined as the price of leisure.
Remember what the wage rate was?
It was the price of leisure, that basically by working I forgoed the ability to-- I'm sorry, by taking leisure I forgoed the ability to earn a wage, w. So, literally, that was a price of sitting around on the couch was the wage, w, I could've earned.
Likewise, the interest rate is the price of first period consumption.
By consuming today, I'm forgoing the fact that I could've earned the interest on that money had I consumed it tomorrow or next year.
So the interest rate is the price of first period consumption, just as the wage is the price of leisure.
Yeah?
You said before that r and i are used interchangeably for interest.
Does that play into the cost function at all?
Is the cost for capital going to be the interest rate?
I'm going to come to that.
That's exactly what I'll talk about next lecture.
So, basically, this is the key thing, but the key thing to understand intertemporal choice-- and the other important point to understand on why it's a bit harder than labor is there's an extra-- well it's not harder.
It's the same thing.
Remember, we said we don't model bads in this course.
We model goods.
So we're modeling your choice of how hard to work.
We model the trade-off between consumption and leisure.
And then we said define labor as the total amount of hours available minus leisure.
Same thing here.
We don't model savings.
That's a bad.
Now you might not think [?
some of these ?]
things are good, but savings really by itself is not a good.
Unless you're Scrooge McDuck-- does anyone know who Scrooge McDuck is?
Wow, that hurts.
OK, he was this old cartoon character when I was a kid who used to, like, fill a swimming pool with money and swim in it.
Basically, unless you're like that, the savings itself does not give you utility.
We don't have savings entering utility functions.
We have consumption entering utility functions.
Savings is a bad.
Savings is the mean by which you translate consumption period one into consumption period two.
But from the effect of today you wish you didn't have to save.
You just do it because you want to make sure you eat tomorrow.
So we model the good.
The good is consumption in period one, and savings is the difference between income and consumption in period one.
So we don't model savings.
We define savings as y minus c1.
We model c1, and define savings as y minus c1.
You can see that there in the diagram.
Now what happens when the interest rate changes?
Let's go to Figure 21-3.
What happens when the interest rate changes?
Actually, go to 21-4.
OK?
Skip 21-3.
Got to 21-4.
What happens when the interest rate changes?
So, initially, we're at a point like a and then the interest rate goes up from r to r2.
The interest rate goes up.
Now what does that do?
Well, graphically, it steepens the budget constraint.
What that means is it's raised the opportunity cost of first period consumption.
First period consumption is now effectively more expensive because I'm forgoing a better savings rate by eating today.
The more of my $80,000 I consume today, the less I get to save for tomorrow.
And that's now a better deal to save for tomorrow because I'm getting a higher interest rate on that.
So what does that do?
Well that has two effects.
Just like a change in the wage rate has two effects-- a substitution effect and an income effect.
The substitution effect, which we can unambiguously sign, is the fact that now first period consumption's gotten more expensive, so we do less of it.
Substitution effects are always price goes up, you do less of the activity.
The substitution effect is first period consumption's gotten more expensive, there'll be less of it.
Now here, once again, don't slip into thinking about savings yet.
You'll really get yourself in trouble, and it's a natural tendency.
Model consumption that makes savings a residual.
So the activity we're modeling here is first period consumption.
The price of first period consumption's gone up, so you do less of it.
That's the substitution effect.
The income effect is you're now richer.
And you might say what do you mean I'm richer?
I still have the same $80,000 in income.
But any given dollar of savings yields more income in the second period.
So, overall, you're richer.
If you take the perspective of saying I have two periods in this model-- first and second period.
Any given level of savings makes me richer in the second period.
That means I'm richer.
If I'm richer, I consume more of everything including first period consumption.
So first period consumption goes up from the income effect.
I find this confusing.
I don't know if you guys do, but once again-- run through this again.
I'm richer because for any given amount of savings I now have a total larger sum of money over both periods.
When I'm richer I consume more of everything.
One of the things that I consume more of is first period consumption.
So, actually, first period consumption goes up, and I save less.
It's sort of bizarre.
Because I'm getting more return to my savings I save less.
Here is the way I like to think of the intuition to make it easier.
The way I like to think of the intuition is imagine that you have a goal for saving-- something I call a target savings level.
Imagine you said, look.
Imagine you said that I really want to make sure that I have certain level of savings to live on next year.
Well if the interest rate goes up, I can save less to get to that target.
Right?
If I have a target, c2, and the interest rate is higher I can consume more c1 and still hit my target of c2.
So that's the income effect.
I can effectively consume more c1 because I'm made richer because any given level of savings allows me to consume more the next period.
Now the target's an extreme case, but I find it a useful intuition for thinking about what's going on.
And that's the income effect.
Now, obviously, as with anything, this is ambiguous.
If you go to Figure 21-3 now here's a case where the income effect dominates.
Well, actually, go back to 21-4.
Let's finish this example.
So here with the substitution effect dominating, when the interest rate goes up I consume less in period one which means I save more.
And that was prior intuition.
A higher interest rate means you save more.
But we'll work through the mechanics of how we get there.
And the reason the mechanics is important is because of Figure 21-3.
Which as in 21-3, the interest rate goes up, but I save less.
The interest rate goes up, but I consume more in period one and therefore save less.
And that's consistent with this target notion.
That basically I'm so-- All I care about-- let's say in the limit if all I care about in the limit is exactly what I consume the second period, then, basically, my period one's consumption will definitely go up from a raise in the interest rate because, basically, I'm saying, look, all I care about is what I get in the second period.
Now I can save less and get to that target.
So my consumption first period goes up.
The income effect dominates.
So the bottom line is, just like with labor supply, we can't tell.
Unlike with goods where it's rare to see a Giffen good, here we honestly don't know whether a raise in the interest rates will raise savings or lower savings.
It all depends on the strength of the substitution and income effects.
Let me actually say one of the most disturbing things in empirical economics is we actually do have no idea.
Literally, there's no convincing study out there which even tells us which way the effect of interest rates goes on savings.
We think probably the substitution effect dominates, but it's been very hard to find a convincing estimate of that.
So it's a little bit disturbing for empirical economics.
We'll typically assume it dominates, but don't necessarily assume that in the real world.
OK, questions about that-- intertemporal choice framework?
OK, now, with that in mind, let's now talk about how capital markets work.
How do capital markets work?
And the key concept for thinking about capital markets is the concept of present value.
And the concept of the present value is simple.
It's that $1 tomorrow is worth less than $1 today.
$1 tomorrow is worth less than $1 today.
And why is that?
It's because if I had the dollar today, I could've invested it in something productive and had 1 plus i dollars tomorrow.
So if you give me a dollar today I could have 1 plus i dollars tomorrow.
If you give it to me tomorrow, I just have $1.
So by definition, $1 today is worth more because I have the opportunity to save it.
Whereas a $1 tomorrow I don't have the opportunity to save it.
It's too late.
So the key point is you can't add up dollars that you receive in different periods.
So, in other words, if I said to you-- if this intertemporal choice graph was back pizzas and movies, and I said you have nine pizza plus movies.
You have a total of nine.
You'd be like, what the hell does that mean?
It matters if it's nine pizzas and zero movies or five movies and four pizzas?
I don't know what that means.
Those are different things.
You can't add them up.
You can't just say I have nine.
Well consumption over time-- it's the same thing.
You can't just add up your consumption tomorrow and consumption today or a dollar tomorrow and a dollar today.
They're different things.
And so you have to account for the fact that $1 tomorrow is worth less than $1 today in trying to add them up.
And the way we do that is we actually do it through the concept of present value.
And the idea of present value is to translate all future dollars into today's dollars.
Translate all future dollars into today's dollars recognizing the fact they're less valuable in the future.
So the concept of present value is the concept of any future payment's value from the perspective of today.
And you should know that any future payment will be worth less than a payment today.
How much less-- that's what present value tells you.
How much is a future payment worth in today's terms?
So suppose that the interest rate is 10%.
Suppose the interest rate is 10%, and you want to have $100 next year.
So you know next year there's something you want to buy, and you have to decide how much do I have to save today to have $100 in period two.
How much do I have to save today to have $100 in period two?
Well if you put in an amount, PV, into the bank-- you put PV into the bank, then you know next year you're going to have-- if you put in PV in period one, next year you're going to have PV times 1 plus i.
You're going to have your PV plus all the interest you earned on it, or in our example, PV times 1.1.
So what that says is that, basically, you have to put in 100 over 1.1 into the bank today, or 90.9 dollars, $90.90.
If you put $90.90 in the bank today, you will have $100 tomorrow or $100 next year-- whenever the periodicity of the interest rate.
So, basically, more generally, the present value of any stream of payments is equal to that stream's future value-- I'm going to write it over here.
It's bigger.
The present value of any stream of payments is that stream's future value over 1 plus the interest rate to the t, where t is the year in which you get the money.
So any future money you get in year t is worth that amount you get over 1 plus the interest rate to the t. So, basically, the point is you have to weigt.
Any money you're going to get, you to weight by how far into the future it is, just like if you're adding up these different goods.
So, essentially, this is kind of like saying let's add a converter machine which can convert pizza into movies.
Then I could say well I'll just take pizza, put it through the converter machine, and that'll tell me how many movies I have.
That's what a utility function basically does.
This we're saying the interest rate is the converter function.
This present value formula is the converter function by which we convert two goods that are different into the same good.
You convert them through this formula.
So, basically, suppose that you say to me, "Look, loan me $30, and I'll pay you back $10 a year each of the next three years."
Well I should say, "Wait a second.
What's the value of that to me?"
Well the present value of those repayments is I'm going to have $10 in one year, so that's $10 over 1 plus i.
Let's say the interest rate's 10% again.
Next year I got $10.
That's worth 10 over 1 plus 1.
The year after, you're going to give me 10 more dollars, but that's worth 10 over 1 plus 1 squared because that's in two years.
If I had that money today, I could've invested it, earned 10% and then 10% on that.
And, likewise, the money you give me in the third year is worth 10 over 1.1 cubed.
If you'd given me that money today, I could've invested it and and earned interest three times on it.
So the bottom line is your repayments are only worth $24.87.
So I've just given you $30 today in return for a stream of payments that's only worth $24.87 today.
I've lost money from that loan because I gave you the money today, and you're paying me back in the future when the money's worth less.
So, basically, the general formula we have is that the present value of any stream of future payments is that the amount of the future payment-- let's call it f for any fixed stream of future payments, $10 forever or $15 forever-- fixed stream of that amount, f, times 1 over 1 plus i, plus 1 over 1 plus i squared, plus da da da da, plus 1 over 1 plus i to the t. So if you're going to pay me a fixed amount, f, for t years, here what it's worth to me today.
You pay me a fixed amount, f, for t years, it's worth this much to me today.
I'm accounting for how far off in the future it is.
Now one important trick we're going to do now for the rest of the semester is we're going to take the trick of saying this is actually-- well this is a messy formula.
It's actually a rather easy formula to write down if the future stream of payments is infinite, if we have what we call a perpetuity.
If we have what we call a perpetuity.
A perpetuity is a future stream of payments that goes on forever or long enough that we'd consider it forever.
Fifty years is probably good enough.
If you have a perpetuity, then this formula can be reduced to-- the present value of any perpetuity is the amount of that perpetuity over the interest rate.
It's just taking the infinite sum of that product.
They're just taking the infinite sum.
Those mathematically inclined will know this already.
But it's just taking the infinite sum here, you can get this formula.
So any perpetuity, if you're getting a payment forever, the value of that payment today-- so if I said I'll give you $10 forever and the interest rate's going to be 10% then you'd say that's worth $100 to me.
If I'm going to give you $10 forever at a 10% interest rate, then you'd say well that's worth $9.90 the next year and $8 something the next year, et cetera.
And if I add all those up, I get approximately the amount of the payment over the interest rate.
So, basically, that is what determines present value.
Now we can flip this around and we could say, OK, well, if that's present value what determines future value?
Well the future value of getting a payment today-- So that's the present value of getting payments tomorrow.
The only other thing is what's the future value?
What's the future value of getting a stream of payments starting today?
So let's say, starting today, I'm going to get $10.
I'm going to get it for a certain number of years.
What's that going to be worth at the end of the day given that I can save it along the way.
So, in other words, if you give me $10 today-- if you give me $10 today, well then in one year, next year, I have $11 because I got to save it at the interest rate.
So if you give me $10 today, next year I'll have $11.
Now let's say that I then keep it in the bank for another year.
Well the next year, it's worth 10 times 1 plus i squared, so 10 times 1.1 squared, 10 times 1.1 squared, or $12.10 and so on.
So, basically, the point is that at the end of each year I earn the interest on my original $10, plus I earn the interest on the interest I earned the previous periods.
So in the long run, at the end of t years, which of the future value, is the amount invested, f, times 1 plus i to the t. That's your future value.
So if you invest a given amount of money today for t years, you end up with that much.
If you invest a given amount, f, today you end up with that much-- And the key point, the key insight, is the miracle of compounding.
The miracle of compounding is the point that you earn interest on your interest.
And what this means is the earlier you save, the more you'll have later on.
So there's an example in the book which is very important.
So it's actually thinking about-- let's think for a minute about retirement.
You might say this is sort of crazy.
Maybe not crazy for an old guy like me, but you're thinking I'm just starting on my career.
Why would I think about retirement?
Here's why you want to think about it.
Let's say, for example, that you plan to work full time from age 22 to age 70.
You've got a great idea.
Screw grad school.
You're going right to work.
You've got a great idea, and you know you want to retire at 70.
So you plan to work full time from age 22 to age 70.
And let's say that you want to save money for your retirement because you're going to retire at 70.
You're going to live forever.
You're a healthy young person.
You think you're going to live forever at 70.
So you want to have money around when you retire.
And let's say that the interest rate you can save at is 7%.
So you can save money for your retirement at 7%, and that's your choice.
Now let's consider two different savings plans.
Savings plan one is that I'm going to save $3,000.
You're going to save $3,000 right off the bat.
And for the first 15 years of your working life-- so from 22 to 37-- you're going to save $3,000.
You're going to save $3,000 a year savings for 15 years, and then nothing.
And then once you're 37 and you've got to start worrying about kids' college and mortgage, you're not going to save anything.
So from 22 to 37, when you're living high on the hog, you've got no obligations, you're going to save.
Once you're 37, you've got a house, you've got kids, you've got things that are expensive, you're not going to save anymore.
So then you go to zero.
Zero savings from age 37 to age 70.
That's a pretty bold plan.
You're just going to save 15 years then zero savings.
Well what do you get?
Well after 15 years, if you use our future value formula, if you save $3,000 every year, you work out that you will have $75,387 in the bank.
So it's more than 45-- is not just 3,000 times 15 because you get the compounding.
It's not just 3,000 times 15.
You actually get more than that because you got the compounding along the way.
Then if you just let it sit there in the bank-- you don't do anything.
No more active savings.
You let it sit there.
Then by the time you retire, you have that $75,387 times 1.07 to the 33 because you let that money sit there for 33 years.
You let that money sit there for 33 years.
That works out to be $703,010.
So by saving for 15 years-- all you did was save $3,000 for 15 years, a fraction of your career.
And you retire with $703,000.
Now we'll contrast that with an alternative plan.
My alternative plan is I'm going to save nothing for the first 15 years because I figure, like, I'm young.
I'm gonna party.
I'm going to use the money now.
I'll save later.
I'm nowhere near retirement.
Then I get to 37.
I say, wait a second, I'm starting to see more mortality.
I better worry about retirement.
Then I start to save, and I save for the next 33 years.
So the new plan is zero per year from age 22 to 37, and then $3,000 a year from 37 to 70.
So it's a lot more savings.
You're saving for more than twice as long, more than twice as long.
What do you end up with?
You end up with $356,800-- half, just slightly more than half, than what you end up with the first plan, even though you save for twice as many years.
This is like the parents' lecture why you should save.
The point is that saving early lets you ride the wave of compounding for many, many years.
Savings late does not let you do that.
And as a result, you end up with less money.
I take my kids to the science museum, and at the science museum they have these little ramps.
And you can drop a ball.
And one is flat then steep, and one is steep then flat.
And the one that's steep then flat always wins because there's compounding in acceleration the same way.
The point is building up early and then riding that velocity going forward is a lot better than starting late.
Questions about that?
So make sure when you get those jobs, and they offer you-- we'll talk next time about savings incentives-- and they offer you those good 401K packages, that you take them, and don't say I'll worry about that later.
Now, one last thing.
Last thing I want to cover is that we've ignored, so far, the whole concept of inflation.
When I've talked about savings, I've presumed that you've saved the money and it's worth something.
But who the hell knows what $703,000 will be worth in 48 years?
What's that even going to be worth?
How do we even think about that?
Well we have to account for the fact that stuff's going to be more expensive.
So we have to account for inflation in doing this.
And the way we do this is by recognizing that what we've done so far is we've talked about the nominal interest rate.
By the nominal interest rate, I meant the interest rate that you actually see posted in the bank.
But what matters, ultimately for your well-being, is the real interest rate which is what your money can do in terms of actually buying goods.
So I should not care about how much money I have next year.
I should care about how many goods I can buy next year.
The money's just paper.
What matters is what I can get with it.
That's what matters.
So let's say, for example, I want to use all my money on Skittles.
That's just what I want to use my money on.
So let's say I have $100, and I want to spend that money on Skittles.
And let's say Skittles today are $1 a bag.
And let's say the interest rate, once again, is 10%.
And let's say there's no inflation.
Inflation equals zero.
So my choice is I can spend $100 on Skittles today and get 100 bags.
So I could have 100 bags today.
Or I can save it, have $110 tomorrow and get 110 bags of Skittles.
That's my choice.
That's my trade-off.
Now let's say there's inflation.
Let's say that prices are rising at 10% a year, as well.
So the prices are rising 10% a year, as well.
What that means is next year Skittles cost $1.10 a bag.
So now what's my trade-off?
That means I could have 100 bags today or 100 bags tomorrow.
I don't get any more Skittles tomorrow.
I get 10 more dollars, but who cares?
Everything costs more.
If everything cost 10% more and I get a 10% interest rate, that interest rate is effectively zero in terms of what I can buy.
The real interest rate is the nominal interest rate minus inflation.
What I care about is what I can buy.
So I have to take out of the interest rate what happens to prices.
Because if prices go up, it offsets what I'm earning in the bank.
And so what I care about is I care about what the bank posts minus what inflation will be.
So it's even trickier, right?
Because it's not about what inflation was, it's what inflation will be.
You'll have to guess what inflation is going to be.
And so what we care about is this real interest rate.
And that's why the interest rate that banks pay, a primary determinant of it is inflation.
Right now, we are in the lowest inflation period this nation's seen since World War II.
Core inflation-- we don't really get into inflation in this course-- but core inflation, which is inflation minus some things which fluctuate a lot, is basically zero in the US.
So, basically, nominal interest rates are the same as real interest rates, and that's why the interest rates you see posted are so incredibly low because there's no inflation.
In the late 1970's, when inflation was running at 10-15% a year, interest rates were 15 to 20% a year.
Now it wasn't that you could get so much more for your savings in the 1970's.
It was just that stuff was going to cost more next year, so banks, if they wanted to induce you to save, had to pay you a higher interest rate.
So, essentially, banks are going to have to pay you to get you to put your money in.
If in 1978, when the inflation rate was 15%-- if banks had offered a 3% interest rate no one would've put money in the banks because you would end up losing effectively.
Effectively, that's a negative 12% real interest rate.
So what matters is how much the bank pays you in cash minus how much more stuff is going to cost.
And that's often what matters.
Now that's a distinction we won't spent a lot of time on later on.
I'll just say interest rate.
I won't say real versus nominal.
But you've got to know in your head that what matters is the interest rate is the real interest rate, what the bank pays you, minus how much more stuff's going to cost.
Questions about that?
Alright, we'll come back next time.
There is class on Wednesday.
It does matter.
And on Wednesday we're going to talk about the rest of capital markets and people's savings decisions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
So today we're going to continue our discussion of capital markets.
If you remember the introduction from last time, what we talked about was we talked about labor as an input and where it came from.
And these lectures are about capitals and input, where it comes from.
We talked about the fact that while capital, ultimately, is machines, and buildings, and things like that, what we think about in this course, we're going to think about financial capital and what's behind all the different kinds of capital that businesses use.
We talked about peoples' savings decisions as creating a pool of capital from which businesses draw.
And then we talked about present value and the notion of considering the fact that dollars in the future are worth less than dollars today.
So with that as background, what we want to do now is talk about how firms and individuals should make choices over time.
We talked a couple of lectures ago about how firms and individuals make choices when faced with uncertainty.
So that was sort of a choice across two different states of the world.
You could get hit by a car.
You could not get hit by a car.
Now we'll talk about choices across two different time periods, today versus tomorrow, and how people make those choices.
And the answer is going to be pretty simple following what we did last time, which is whenever you're faced with two choices that pay off at different times, you just want to choose the choice with the highest present value.
So if you're faced with two streams of payments, you know that you can't just add them up.
What you need to do is you need to add them up in a way that gets present value.
So, for example, consider the example of a professional athlete who's considering two contracts.
One contract pays $1 million today, and one contract pays $500,000 today and $2 million in deferred payments in 10 years.
So that's the two contract options facing the player.
If you read it in the newspaper, they would describe this is a $2.5 million contract and this as a $1 million contract.
However, the newspaper is wrong, because they haven't accounted for the fact that some of those payments are deferred.
So they're worth less.
So how do we compare them?
Well, to compare them, we have to take the present value of these two streams.
So the present value of the first stream is just $1 million, because it's paid today.
So putting it in today's dollars, it's worth $1 million.
The second stream is worth, the present value is $500,000 plus $2 million over 1 plus i to the 10th, because it's being paid in 10 years.
And actually here, I'm going to not use i. I'm going to use r for the real interest rate.
Remember last time we talked about how what really matters is the real interest rate, the interest rate you get minus inflation, minus the costs of price increases for goods you have to buy with that interest.
So remember we want to use the real interest rate here.
So I'll use r now.
So, basically, whether this second contract is a better deal or not depends on what the real interest rate is.
So if r equals 5%, then the present value of this second contract is $1.73 million.
So that is a good deal.
On the other hand, if r equalled 20%, if there's a 20% interest rate as there was back in the late '70s, early '80s, then the present value is $0.82 million.
So it's not a good deal.
So the key is if you want to ever compare two streams of payments that pay off in different times, you have to bring them back to a comparable unit, to today's dollars.
And the way we do that is by calculating their present value.
And this is a problem people have.
A common mistake that's made is people don't consider this in evaluating streams of payments.
So a classic example is when you hear someone wins $100 million in the lottery.
It's actually a lot less than that.
Because $100 million lottery win, what it really is is $5 million a year for 20 years.
And you have a choice.
You could take a lump sum now, or you take the $5 million a year over 20 years.
So basically, the $5 million a year over 20 years is, of course, worth a lot less than $100 million.
What's it worth?
Well, it's worth $5 million plus $5 million over 1 plus the real interest rate plus $5 million over 1 plus the real interest rates squared plus dot dot dot dot plus $5 million over 1 plus the real interest rate to the 20th.
So, for example, for an interest rate of 5%, if r equals 5%, this lottery is really worth $65 million.
Now, that's still pretty good.
But it's a lot less than $100 million.
When you hear a number about a player's contract or a lottery winning, you need to always recognize that the effective value is going to be lower than what you hear on the news because of the time payment that takes place.
Now whether that's a big deal or not depends on the interest rate.
So Victor Martinez just signed with the Detroit Tigers for four years and $50 million.
Now, he could have signed with the Chicago White Sox for three years and $42 million.
Or the Boston Red Sox were offering.
So the Chicago White Sox were offering three years and $42 million.
Now, you might say that you can't really compare those two, because one is out three years, and one is out four years.
But the truth is the interest rate right now is so low that you can pretty much compare them.
Right now, the interest rate you can get in a bank is close to 0.
So, basically, if the interest rate is 0, the present value is the nominal value.
So right now, you can sort of compare players' contracts like that.
On the other hand, when the interest rate is higher, you can't do that comparison.
And, in particular, what you'll hear a lot of times is these contracts have very deferred payments.
So even for a low interest rate, it's still worth a lot less in the end.
So that's the key point to remember when you hear values.
It's to know that you have to discount them by when they happen.
And that's going to depend on how high the interest rate is.
Now, this leads us directly to the important implication of present value which is how firms and individuals make investment decisions.
So we've talked about firms choosing a level of capital and a level of labor.
And we talked about in a simple isoquant-isocost framework.
But, in reality, firms don't really deal with that simple framework.
In reality, for any given investment decision, they essentially want to compare the cost of that investment to the benefits of that investment.
And that depends on things like the isoquants and the isocosts.
But it also depends critically on the time frame over which that investment will pay out.
And what firms consider when they make an investment decision is the net present value of that decision.
On net, what's the present value?
Because often, for the investment decisions, you have to lay out money up front to recoup that money later on.
So this adds an extra dimension.
Which is not just you're adding up a stream of future payments.
You're actually contrasting a debit now versus credits in the future.
And the math is the same, but it's a little bit more complicated.
So the net present value of any investment is going to be the revenues from that investment in period 0 minus the cost of that investment in period 0 plus the revenues from that investment in the period 1 minus the cost of that investment in period 1 over 1 plus r plus dot dot dot dot dot plus the revenues in year t minus the costs in year t over 1 plus r to the t. So the net present value is going to be a function of at every year is the investment making or losing money and then adding those future gains or losses up discounting by when they occur.
And the bottom line is firm investment decisions are very simple.
If this is greater than 0, then it's a good investment.
If it's less than 0, it's not.
And this matters a lot, because you'll often see investments that have cash losses up front and gains later.
In fact, that's sort of the definition of investment.
It's that you're investing some money up front to yield returns later on.
So, for example, if you have an investment with an upfront cost of $100 in year 1, it's going to cost you $100 in year 1 to buy some machine, and you get no revenues.
So it's minus $100 in year 1.
But in year 2, you're going to earn $200 from that machine minus you're going to have a $50 maintenance on the machine, so $150 in year 2.
So in year 1, you buy the machine for $100.
In year 2, the machine produces some widgets which you can sell for $200.
But, along the way, you have to maintain that machine.
And that costs you $50 in year 2.
The net present value is minus $100 plus $150 over 1 plus r. That's the net present value.
Whether that's positive or not is going to depend critically on what the interest rate is.
So the key thing is we want to take that stream of payments, positive or negative, and put them in today's terms.
Are there questions about that?
But this raises the question of well, what interest rates should a firm use?
So the firm has got to make this decision.
Should all firms just walk by BayBank, or Fleet Bank, or whatever the hell it's called now?
What is the big bank called?
Fleet, I guess.
Should they walk by now and look in the window and say the interest rate is 1%, or 2%, 3% and use that?
What should firms do?
Basically, the key issue is that different firms will have different interest rates that they want to use.
Or firms will have different, what we call, discount rates.
You can have a firm-specific discount rate.
You can have a firm-specific discount rate that firms might want to use.
And what's going to determine that is going to be the opportunity cost of money to the firm.
What is the firm's next best use of that money they're investing.
So, for example, let's say a firm has a bunch of cash sitting around.
Let's say the firm has $100 in cash sitting around, and it's trying to decide whether or not to make this investment.
How do we decide on what the opportunity cost on that investment is, what the right discount rate is to use?
How should the firm think about that?
If it's got the cash sitting around, how should the firm think about whether it should invest $100?
Yeah
Well, whether or not it has some other place that it can invest it.
Or it's just going to rot in a safe somewhere
Well, at a minimum, we can put it in the bank.
So at a minimum, it can always earn 1% or 2% of whatever banks are paying now.
But what you're saying is right.
It may also have other investment opportunities.
So, in some sense, the discount rate for any investment opportunity is what you could earn on the next best investment opportunity.
So if I've got $100 sitting around, and I've got a guaranteed 10% I can make somewhere, then if I decide whether or not I should take this investment, I should use a discount rate of 10%.
If I can make 20% somewhere, I should use 20%.
So, in some sense, what you want to do is you want to stack up your investment opportunities from worst to best in terms of rate of return.
And then you start at the top of the list and ask, OK, discounting at the best alternative opportunity, is this one worth it given the steam of payments?
And then work your way down the list.
So, basically, the key thing is that for any firm, their opportunity cost, their discount rate is the next best thing they could have done with the money.
Similarly, if a firm doesn't have the money and is deciding whether or not to borrow the money to do this investment, then the discount will be the cost of borrowing the money.
Right now the bank only pays 1% on your money.
But if you want a loan, you still have to pay 4% for it.
So it may be that you could end up in a situation where your $100 doesn't really do you that much good saved.
And it would cost so much to borrow the $100, that it's better to use the $100 that's sitting around.
So, basically, let's say you have $100 sitting around, and you're trying to decide how to finance its investment.
Well let's say you have an alternative investment that yields 10%.
What you do is you put the $100 on the alternative.
And then you say, well, should I then borrow to finance this investment?
Well, I have to ask, is the borrowing rate low enough that this net present value is positive?
So the firm wants to consider all of its options it can do with its money that it's borrowing and consider a discount rate for each, which is the alternative next best use of the money.
Now it's not just firms that make these investment decisions.
People have to make these decisions too.
In fact, I faced one, a number of years ago, when I was first teaching this course.
So it's good for me to work it out in the context.
I was decided this just about the time I was delivering this lecture.
And here was my choice.
Basically, I had to decide on whether to invest in insulation for my 100-year-old house.
I've got this old house, windy, crappy, whatever, and I had to decide whether to invest in insulation.
And the math was that my heating bills were costing me $2,000 a year.
I was spending $2,000 a year in heat.
It's a lot more now.
Back then it was $2,000 a year.
My best estimate was that if I insulated my house, I could lower my heating costs by about 25%, or about minus $500 per year.
But to do that insulation would cost $4,000.
So my question was, do I make a $4,000 investment to lower my heating costs by $500 a year?
Well, I know how to do that.
I say, I take my $4,000, that's a negative in year 1.
So let's say there's no effect in year 1 of the heating costs, because they're putting it in.
So the heating savings start in year 2.
So then in year 2 I'm going to save $500.
But that I'm going to have to discount at some interest rate.
In year 3, I'm going to save another $500.
I'll discount that at some interest rate, and so on.
Now, the tricky thing about this calculation is twofold.
What are the two things I still have to figure out to do this calculation?
Yeah
The time frame that you're going to spend in the house and the interest rate
One is the interest rate.
Now, for that one, I have to think well, if I had money sitting around, what was the next best investment opportunity.
This was the early 2000s when the stock market didn't look like a very good investment opportunity.
The next best was the interest rates were probably on the order of about 5% back then.
So let's say I could have put it in the bank at 5%.
But the other tricky thing is how long will I have the house for?
If I'm going to own the house forever, or for long enough that it's equivalent to forever, then I know I can just rewrite this as minus $4,000 plus $500 over r. So if the interest rate was 5%, then that's minus $4,000 plus $500 over 5% or plus $10,000 which is well above 0.
So at a 5% interest rate, if I was going to own the house forever, this is a great investment.
Yeah
Wouldn't you also have to consider how much it would add to the value of your house when you attempt to sell it?
Excellent point.
Even if I'm not going own the house forever, it still might be a good investment.
Let's say I was going to sell the house in two years, then it would be minus $4,000 plus $500 over 1.05-- Well, let's say I was going to sell it after two years.
So this is the first year and the second year.
So you might say, well, this is clearly a bad deal.
I'm spending $4,000, and all I'm saving is $500.
And that's in this future where it's worth less.
But then the question is, how much does it raise the value of my house?
In a perfect world, given that my house will live on forever, it should raise the value of my house by $10,000.
Because what I've done is I've saved $500 forever.
So whoever buys it after me should be willing to pay $10,000 more for it.
But the question is that information, will that percolate down?
Will people actually include that in the value?
So in some sense, the decision I had to make was, will I live there long enough that even if the next people don't value it, I'll still end up having the positive net present value.
If I lived there for more than about 10 or 12 years, that would be true.
And the second thing is if I'm going to live there less than that, will the price go up enough that the net present value will be positive?
I decided to do it.
I've been there at least 10 years since.
So it was a good decision.
But this is exactly how consumers face these kinds of decisions every day.
Just as uncertainty affects decision making of people every day, so does thinking about streams of payments over time.
Yeah
Did you leave out the square on the $500 over 1+r?
Yeah.
You're right.
Good point.
OK. so now another very interesting application of this that may be more relevant to you.
You guys aren't thinking about insulating your houses right now.
But your family has recently thought about a very important decision.
It's not about investing in physical capital, like investing a machine, but investing in your human capital.
An important theory due to Gary Becker, Nobel Prize winning economist at Chicago, was that just as we can think of firms buying machines as investing in physical capital, we think of people investing in education as like building your human capital.
You're spending money to improve your long run productivity just like buying a machine improves the firm's long run opportunity.
So it's exactly the same thing.
It's just instead of investing in a building or a machine, we invest in you.
And, likewise, human capital investment decisions are subject to the same net present value considerations that physical capital production decisions are subject to.
Let's think about this with an example from the book.
So imagine that if you don't go to college, you're going to work for age 18 to age 70.
And if you do go to college, you work from age 22 to age 70.
You get the extra year dinking around Europe or whatever.
No, that's right, 18 to 22, four years.
So you either work from 18 to 70 if you don't go to college or 22 to 70 if you do go to college.
Imagine, moreover, that college costs $10,000 a year, obviously not MIT.
Some state school, $10,000 a year is what college is going to cost.
And the cost of going to college is twofold.
One is you pay the $10,000 a year.
Second, you forgo earning while you're in college.
The benefit of going to college is you learn a lot more once you graduate.
On average, at age 22, the typical college educated person-- once again, not MIT-- but the typical college educated person earned about $30,000 while someone with a high school diploma only earned about $20,000.
So you earn a lot more thereafter.
So, basically, the trade-off is you pay tuition up front, and you lose earnings up front, but you earn a lot more starting at age 22.
Well, how do we think about whether that's a good investment or not.
Well, let's look at Figure 22-1, the present value of education.
What you see is a diagram of the net present value calculation.
So from age 18 to 22 there's a huge cost, which is your forgone earnings plus the amount that you had to pay to go.
Then starting at age 22, there's a net benefit, which is you earn more.
Basically, whether the net present value is positive depends on comparing the shape of these and discounting the fact that the earnings you make from going to college are worth a lot less.
So what's striking here is that basically if the discount rate is more than 5.1%, it turns out not to make sense to go to college.
It's pretty amazing, if you think about it.
You earn 50% more if you go to college, 50% more.
And yet if the discount rate is more than 5.1%, which it's been, typically, in many years in our society, it's going to end up not making sense to go to college.
Why?
Because the upfront costs are worth so much more.
The upfront benefits of excluding college are now.
And the benefits of your education are distant.
So, as a result, because of net present value considerations, unless the discount rate is very low, it's not going to make sense to go to college.
It turns out now, of course, the discount rate is very low.
Now we ask ourselves, OK, what is the discount rate your parents face when they decide whether to send you to college?
Or maybe you faced it?
Maybe you're paying for your own college.
What's the discount rate you face?
Well, once again, it's the opportunity cost of the money you use to go to college and the opportunity cost of what you could have done with the money you made if you were working now at gap.
The opportunity cost of the money you could have made, is you could have saved that at some interest rate.
But the truth is whatever you could have saved it at, nothing is yielding much more than 0 right now.
There's no investment that yields anything.
So the interest rate is very low on that savings.
Moreover, you had to borrow the $10,000 a year and, in your case, the $50,000 a year, to come to college.
And that's at the borrowing rate which is still above 5% even in this economy.
Yeah
Can you explain to me what the discount rate is
Oh, the discount rate is the interest rate in this net present value calculation.
So when you're considering whether to go to college, it's the rate at which you discount those future extra earnings that you're going to get.
So the discount rate is your version of the interest rate.
It's the opportunity cost, the opportunity cost to you of what you could have done with that money.
What you could have done with the savings from working is basically nothing.
It would have just sat under your mattress.
However, the money you had to borrow to come to college, that you paid 7%, 8%.
That's money that has a pretty high discount rate.
That's money that's worth a lot less in the future.
So at the end of the day, with today's interest rates, it almost certainly makes sense to go to college.
However, when interest rates are high, it might not.
And that's an argument for why the government may want to subsidize student loans.
The government is in the business of subsidizing student loans and making student loans artificially cheap.
That's a pretty big government expenditure.
It's on the order of $30 billion a year.
Why is the government doing that?
Well, this table tells you why.
If the government thinks that there's social benefits for having a more highly educated population, then, essentially, by intervening to lower the discount rate through subsidizing college loans, the government can encourage people to go to college.
OK.
Questions about that?
So that's how we think about net present value in investment decisions which is basically to put everything in today's dollars and then, on net, ask if it's positive or negative.
Now, the other thing I want to talk about, in terms of government policy, and in terms of important issues in this area, is about increasing savings.
I want to talk about increasing savings in the US.
Why do we care about increasing savings in the US?
Why do we care about that as a goal?
After all, I said savings is a bad.
Consumption is the good.
Why do we care about savings?
Well, the fact is the US government does.
The US government spends hundreds of billions of dollars a year encouraging individuals to save in ways I'll describe in a moment.
Why do they do that?
Well, the reason they do that is because savings becomes the engine for growth in our economy.
Basically, as savings goes up, that increases the pool of capital available for firms to draw from.
In other words, that shifts out the capital market supply curve.
In terms of the diagram that we made last time of the capital market, that shifts out the capital market supply curve or increases that pool of capital.
As the capital market supply curve shifts out, what happens to the interest rate?
It falls.
That leads the real interest rate to fall.
Because there's a bigger pool.
It's cheaper to get from it.
As a pool grows, it's cheaper to draw from it.
That lowers the interest rate.
What does a lower interest rate mean for any given investment to its net present value?
It means it's higher.
In means that any given investment has a higher net present value now.
Because a lower interest rate means you might as well invest instead of just putting the money off to the future.
And a higher net present value means more investment.
So more savings becomes the engine of growth for our economy.
Because savings promotes investment.
When people save, it lowers the price of borrowing.
Firms are more likely to borrow, and they invest more.
And so that's why savings is so important.
And this is basically a lot of what Bob Solow won his Nobel Prize for.
A famous professor from MIT won his Nobel Prize for pointing out the critical role in savings is an engine of growth.
It's that basically savings becomes the key engine to growth.
And as a result, society should care about how much individuals save, not just how much they consume.
And that's why savings is so important.
The problem is we don't save a whole lot in the US.
In China, for example, the savings rate, depending on how it's measured, is on the order of 30%.
So for every dollar people earn in China, they save about $0.30.
In the US, it's about 3%.
For every dollar we earn, we save about $0.03.
It was negative for a while.
Yeah
Are those rates including taxes and other things that people have to pay.
So is it like revenue?
Its a share of your disposable income you save.
The check you take home, how much of it do you save, and how much of it do you spend?
US citizens spend about $0.97 on every dollar we take home.
Chinese citizens spend about $0.70 on every dollar they take home.
You might have noticed that China's grown a hell of a lot faster than the US over the last 30 years.
It's no coincidence.
They're basically building up a stock of capital that's allowing their firms to draw on it at a cheap rate and invest.
And that's why something like 20% of all the world's cranes are in Shanghai right now.
Because they have a huge pool of capital they can drawn to build all of those new buildings and invest.
So this is a big problem for our place in the world economy.
The US is falling behind in the world economy, because we're not saving enough.
We're not building up that pool of capital that our firms need.
As a result, there's a huge public policy effort to promote savings.
And, in particular, there's an enormous amount of emphasis on tax subsidies to retirement savings.
The basic idea here is that when you put money in the bank to save for your retirement, the interest you earn on that money is taxed.
The interest you earn on that money is taxed.
That lowers the rate of return to savings.
And, assuming substitution effects dominate, that means you save less.
So in today's system, since we tax the interest you earn in the bank, you save less people think.
We still don't have quite convincing evidence on that.
But people usually presume substitution effects dominate.
There's less savings partly because we're taxing savings.
And, as a result, there's less investment.
Yeah
When we're talking about savings here, are we just talking about just private savings?
Or are we also concerned with what the government is doing?
Excellent question.
Let me actually come back to that.
Let me answer that, but I want to come back to that.
That's important.
What we care about, of course, is the total net pool of savings.
And if the government draws from that, that's less money that firms can draw from.
So you're absolutely right.
We about the net level of social savings which is people plus government.
If people save a lot, but the government has a huge deficit, that cancels out.
And I want to come back to that.
It's very important.
This is a good point you raise.
We take these taxes.
We tax your interest.
That discourages savings, but at least it raises money for the government.
So on net, it's not clear if it's a bad thing or a good thing for savings.
On the one hand, at least we get money for the government which can reduce the government's deficit.
On the other hand, we discourage your savings which may reduce your savings.
So on net, it's not clear.
But, generally, the assumption is that on net, total social savings falls.
But as that question points out, that assumption relies on two things.
The first one relies on the substitution effects dominating income effects.
And it relies on the substitution effects being so strong that the reduced savings because we're taxing it exceeds the extra revenues we get from taxing it.
But under those two assumptions, the government might want to offer some tax subsidies or try to offset this taxation of capital income by offering tax subsidies to individuals.
And the way we do that is we say that you can save for your retirement tax-free.
Actually, it's not tax-free.
Let me say it again.
We allow you to save for your retirement on a tax-deferred basis.
So, for example, we have employer-provided pensions.
Pensions are plans that your employer sets up.
where you take part of your salary, and that is not taxed.
Instead it's saved.
And when you retire, you get that savings, and then it's taxed.
So, in other words, right now MIT takes $2,000 from my salary every year and puts it aside.
When I report my income to the IRS, it's $2,000 lower.
So I'm not paying taxes on that even though I earned it.
That money is in an MIT pension account that's building up.
MIT has invested, and it's building up.
And when I retire, I'm going to get that money and then pay taxes on it.
Now, you might ask yourself, so what?
What the hell is the advantage of that?
You pay taxes now or you pay taxes later.
Who cares?
Why is that a benefit to me?
Because the present value of the money in the future is less.
The after tax amount is going to be less assuming that taxes don't rise by a substantial amount in the future
Exactly.
Assuming tax rates stay the same.
Let's leave that.
The point is just as money in the future is worth less than money today, paying taxes in the future is better than paying taxes today.
So if MIT take the $2,000 aside for me and saves it, and I pay taxes on it in 25 years when I retire, that's 25 years with a 5% discount rate.
That's nothing.
25 years at a 5% discount rate, those taxes are worth like 1/3 of what they are if I pay them today.
So basically by deferring taxes, we are wealthier.
Just like by deferring payments, we're poorer.
We don't have the table here
[INAUDIBLE PHRASE]
What?
No.
That's OK. Let me just sort of explain an example.
Let me just do an example here and explain that.
So imagine you've got an individual who's a 70-year-old, so someone about to retire.
They are a 70-year-old, and he earns $100 on his job.
And he wants to save it for 1 year and then spend it.
And he has two choices.
He can put it in the bank or he can have his employer hold it aside as a pension payment, and he can get it next year.
And let's say the interest rate is 10%.
And let's say his tax rate is 25%.
So the bank route, what happens?
First of all, $100 he gets is going to get taxed.
So he's going to have $75 to put in the bank.
He's going to put that in the bank.
And that's going to yield $7.50 in interest after one year.
But that will also be taxed.
That $7.50 of interest will also be taxed.
He has to pay $1.88 in taxes on that $7.50 of interest.
So on net, he's going to end up with $75, $82.50 minus $1.88.
So he's going to end up with $80.32.
Is that right?
$80.22.
So he's going to end up with $80.22.
Now, if instead he asks his employer to hold it aside as a pension, then he gets the whole $100, and that gets saved at 10%.
So he earns $10 of interest.
So at the end of the year when he gets the pension payment from the employer, he gets $110.
That is then taxed at 25%, and he ends up with $82.50.
He ends up with more money.
Even though it's just paying taxes one year later, he ends up with more money.
And the reason is because he got to earn the interest on the taxes instead of the government earning the interest on the taxes.
Here, he pays taxes up front.
So he has less to save.
So he ends up with less.
Here, by paying taxes later, he gets to earn the interest on that extra money.
And he ends up with more.
So by deferring taxes, he's effectively richer.
And this is a very simple example.
If you did the same example, and instead of making it 1 year, you made it 30 years, then you'd end up with twice as much money if you go the pension then if you go the regular savings route, twice as much money just from deferring taxation for 30 years.
You get taxed either way.
It's just that you get to earn the interest instead of the government earning the interest.
Yeah
Can you earn interest on a pension?
OK.
So let's come to that.
Let me make sure people understand the math here and why it's a better deal.
OK. Do you earn interest on a pension.
It's a good question.
There is three ways that you can have tax subsidized retirement savings.
And let me go through them clearly now.
One route is the pension.
Under a pension, your employer takes money out of your salary, saves it in some account with your name on it, but they invest it.
And then you get the money when you retire.
An alternative, which firms are slowly moving to, is 401(k) accounts, which you've all heard of by now.
401(k) accounts are just like pensions, except you control the investment of the money, not the firm.
So under a pension account, your employer takes money out of your salary, puts it aside, and invests it as he likes, and you get the return when you retire.
A 401(k) account is the same thing except you decide how much to take or the employer decide how much to take, and you decide how to invest it.
And then you get the return at the end of the day.
But the tax treatment is the same.
Yeah
Isn't it also one's defined benefits and one's defined contribution
Well, within this, within pensions, there's two kinds of pensions.
There's defined benefits and defined contributions.
We can think of this as old style and new style.
Defined benefits is what the auto companies and the steel companies had.
It's dying away.
A defined benefit pension is one where you don't actually get an account from your employer.
They just take money out of your salary.
And then when you retire, they pay you some amount which is unrelated to what you put away.
It's related to something like how long you worked at the company and what your wages were.
So defined benefit pension is they literally define the pension benefit you get.
And it's got no direct relation to how much money they took away on your behalf.
It's related, instead, to what you earned and how long you worked there.
A defined contribution pension is what I described.
It's literally an account with your name on it.
And that's what most pensions are now.
Almost any firm you guys will work for will have a defined contribution pension, which is one where there's literally an account with your name on it.
And most of you will have 401(k)s where you actually control the investment.
And then finally, the last option is IRAs.
That's not the Ireland thing, but individual retirement accounts.
These are accounts which operate outside the employment setting.
You can literally set up your own pension, effectively, by taking money, putting it in the bank, and calling it an IRA.
The thing about IRAs is IRAs are not special investment vehicles.
They're just a label for savings that you do.
Any kind of savings can be an IRA.
You can have gold in your IRA.
You can have cash in your IRA.
You can have whatever you want in your IRA.
Any kind of investment is an IRA.
What an IRA means is you say to the government, this money, up to $5,000 a year, don't tax me on now.
I'm going to put it in a specially labelled bank account, and you'll tax me on it when I retire.
So it's just like a pension, but you set it up on your own.
And the firm is not involved.
Let me just say something on the IRA.
If you're not wealthy, if your income is below about $75,000 a year, it operates just like I described.
If you are wealthy, if your income is above about $75,000 a year, then you can't get the tax break on the IRA.
So the IRAs are really more focused towards lower income populations.
Now, here's an interesting question that you all face.
You're going to get jobs in a couple of years.
And your employers are going to offer you pensions, or a 401(k), and you can set up an IRA.
And you're going to have to decide, do I want to do that or not?
Now, what are the considerations?
Well one consideration is what I mentioned last time, which is that savings earlier in your career is a lot more beneficial than savings later in your career.
The second advantage, of course, is that there's the tax break which, of course, the earlier you do it, the more valuable it is than the later you do it by the same logic.
On the other hand, there's a huge disadvantage to these forms of savings.
You can't get them until you retire.
If you take them out before you retire, you pay a tax penalty on them.
So this trade-off when you think about setting these up is you only want to put in money you're sure you're not going to need until you retire.
So it's a good idea to set these up.
It's a good idea to take advantage of this tax breaks.
But, in doing so, you have to remember that there's different kinds of savings, and there's different needs for savings.
You can't put savings for a house in these.
You don't put savings in case you lose your job in these.
These are for savings which you can honestly say I won't need for 30 years.
And that's the trade-off.
You should do it early and as much as you can.
But you should also recognize that you don't want to leave yourself with no money in the bank to do this.
Because then, if you lose your job, there's nothing to draw on.
And that's how one sort of thinks about these things.
Yeah
So you said there was a tax penalty if you [INTERPOSING VOICES] early.
Is that greater than the tax rate that you would normally pay on them?
Because then, even with that penalty--
It depends on how long you have the money in.
If you have it in for 20 plus years, even with the tax penalty, it's a good idea to do it.
But if you're going to have it for five years, it's not worth it
So it's bigger than a tax [INAUDIBLE PHRASE]
Yes.
It's 10% of the balance.
You pay 10% of the balance for the tax.
So, basically, whether that's a good thing or not depends on how long the money's been in there and what the forgone interest rate is.
But for a short investment, it's going to be a bad idea.
If you have an IRA for 20 years and then have to take it out, it's still a better deal then having not put it in there.
One last point about a 401(k) that's very interesting.
Basically, in the theory I've taught you in the last two lectures, what determines your savings is the interest rate.
If the interest goes up and substitution effects dominate, you save more.
If it goes down, you save less.
But, in fact, in the real world, savings is determined by lots of other factors that economists are just starting to think about and model.
One which you've know for a long time, of course, is precaution.
The bigger risk you face in your life, the more you'll want to save.
So, for a given interest rate, we've found that individuals who face greater risk in their lives save more.
Likewise, we found that government programs which protect you from risk cause you to save less.
So this is another tricky thing with government programs.
On the one hand, we like government programs like unemployment insurance or social health insurance that protect us in case we lose our job or get sick.
We have to recognize the social costs of those programs is people save less.
And that's less savings equals less investment.
So precaution is one reason why people save.
But there's other factors which fall in the realm of behavioral economics, which I've mentioned earlier in this course, which drive why people save.
And this is one of the most important economic findings in the last couple of decades.
Studies of firms that change the structure of their 401(k)s in a particular way that shouldn't matter but does a lot.
So for most firms, when you go decide to join the firm, they'll say, here's a benefits package.
You can sign up.
You can sign up for health insurance.
You can sign up for life insurance.
You can sign up for a 401(k).
And that's the way it usually goes.
And we typically see, as a typical large firm, is with new employees there's about a 25% sign-up rate for 401(k)s. And over the next 10 years, that grows to about 70%.
So people don't join right away.
They join slowly.
Some firms experimented with changing the contract in a very small way that should not matter in this course.
They've instead said, welcome to our firm.
You are now signed up for the 401k unless you tell us you don't want that.
Now, that's the same thing.
I can sign up or I can tell them I don't want to be in.
Those are identical things.
It's just a question of the default.
It's just a question of the default that I have to say affirmatively yes or I have to say affirmatively no.
What they found when they switched, is the initial sign-up rate from 401(k)s went from 25% to 75% simply by this relabeling.
Simply by shifting the default, you've got an incredible change in people's behavior.
And what this says is that economics is about more than things like interest rates.
It's about more than prices.
In this course, economics is all about prices.
I talked about it in the first lecture.
But in the real world, we bring psychology into it.
Economics is about more than just price.
It's about important behavioral factors.
And this says that important policies that there may be a much more significant policy than spending all this money on tax subsidies and raising the government deficit.
These tax subsidies, by the way, add up to on the order of $200 billion a year that we spend on these.
So that's $200 billiion a year that we're increasing the deficit by to have these tax subsidies.
What if, instead, we got rid of them all and just defaulted everyone to 401(k)s. We'd probably raise savings more and potentially save the government a lot of money.
So, basically, the point is that we have a lot of tools at our disposal besides prices.
And our government policy makers have to be thinking about those.
OK so let me stop there.
Thank you all for coming.
Have a great Thanksgiving.
And we'll come back after Thanksgiving and talk about equity and efficiency.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so what we're going to do for the next few lectures, really through the end of the course, is turn away from what we focused on for most of this course, which is efficiency, and start talking about equity.
So most of this course, we've talked about efficiency and the principles that lead to efficient outcomes.
We haven't talked a whole lot about fairness and equity.
We've mentioned it in passing for things like why you want a minimum wage, et cetera, but we haven't really focused on how society should consider efficiency and equity as two different goals, because clearly it does.
So for example, a classic example of the fact that efficient outcomes may not be equally equitable is the perfectly competitive market versus the perfectly price discriminating monopolist.
Remember that both outcomes, both a perfectly competitive market and a perfectly price discriminating monopolist, both lead to maximum social welfare.
So in the terms we've used in this course, both provide equal social welfare.
The difference, of course, being that in the perfectly price discriminating monopolist case, the producer gets it all, whereas in the perfectly competitive case, it's shared between producers and consumers.
So we actually care.
We actually in the end are going to care as a society, not just about social welfare, but who's getting the shares of the pie.
In other words, we're not going to care just about the size of the pie, but who's getting what shares.
So first of all, even for a given size of the pie, we have to decide who gets what shares.
Moreover, it gets much more complicated than that because the size of the pie isn't fixed.
And typically, in a society's efforts to redistribute shares, they shrink the pie.
That is, societies typically face the classical equity efficiency trade off.
Which is that in any effort to redistribute resources, inevitably we'll shrink the total amount of resources available.
And that's roughly speaking, because the efficient outcome is where you start from.
You start from an outcome where you've maximized social welfare.
So any intervention that tries to deviate from that by moving resources across groups, by definition will move us away from that efficient outcome.
We're starting with the efficient outcome, let's say a permanently price discriminating monopolist, that's efficient.
Any efforts we have to take some money away from that monopolist and give it to other people will end up causing distortions and reducing the total size of total social welfare.
So basically, we're always going to face this equity efficiency trade off, and really the question we want to ask in this lecture is, is it worth it?
How do we think about whether it's worth it to redistribute from rich to poor, even if along the way, we shrink the total size of the pie, even if we have to deviate from the welfare maximizing point.
And the best way to think about this I think is due to an analogy developed by a famous economist named Arthur Okun.
And this is Okun's leaky bucket analogy.
So Okun's leaky bucket analogy is the following, he said, look, let's say as a society, we agree that we should redistribute from rich to poor.
And I'll come later to why we might agree that, but let's just say we agree as a society that we should redistribute from the richest members to the poorest members of society.
And let's think of that redistribution as being literally the rich people put money in a bucket, and you carry that bucket and then dump it out in front of the poor people.
Let's say that's the way redistribution happened.
Well, probably we'd have general agreement that if Bill Gates could put $1 in a bucket and it could be carried and given to a homeless person, that's a good thing.
We probably in general agree that if you distribute from the richest people to the poorest people, and $1 from a rich person becomes $1 for a poor person, that's a good thing.
However, what if along the way, some of that dollar leaked out?
What if Bill Gates put $1 in the bucket, but by the time it got to the poor person, it was only $0.80.
You might say, OK, that's still fine.
$0.80 to a poor person is a lot.
Bill Gates isn't going to miss $1, but $0.80 to them might matter.
What if it's only $0.50?
What if Bill Gates put $1 in, by the time you carry it over, it leaks out so much so the poor person only gets $0.50?
What if it's $0.10?
What if literally every dollar Bill Gates gives up, the poor person only gets $0.10, then you might say, gee, it's not entirely clear it's worth it anymore.
I mean, yeah, $0.10 is worth more to a poor person than a $1 is to Bill Gates, but that's really wasteful to have $0.90 just leak out along the way.
So basically, what we have to do is ask basically how does society think about transfers when some of what happens when you transfer from rich to poor is you lose some efficiency along the way, something leaks out of the bucket?
And that's basically what we're going to talk about in the next two lectures.
We're going to do that in four steps.
We're first going to ask, how does society value transfers?
How does society think about the value of $1 to Bill Gates versus $1 to a poor person?
How do we think about the value of those dollars in different hands, something we haven't talked about this semester?
We then are going to talk about the facts on income distribution.
How has the distribution of dollars changed over time in the US, and how do we compare it internationally?
Then we're going to talk about the sources of leakage.
That is, what causes this bucket to be leaky, in fact, what causes the equity efficiency trade off in practice.
And then finally, we're going to talk about what governments do, what government does to redistribute resources.
How does the US government in particular redistribute resources from higher to lower income groups?
So that's what we're going to talk about the next two lectures.
So we're going to start with the question of how does society value transfers, or more generally, how do we think about the socially optimal allocation of resources?
The key word here being allocation.
So we're no longer just going to talk about the entire size of the social welfare triangle, we're now going to talk about who gets what, and how do we think about the socially optimal distribution of those resources, allocation or distribution of those resources.
And to do so, we're going to have to take an extra somewhat uncomfortable step we haven't taken before.
Before, we've just talked about a purely mathematical concept of maximizing the size of a triangle.
Now we've got to actually put some value judgment on who gets what within that triangle.
And the way we make that value judgment is through the introduction of what we call a social welfare function.
A social welfare function, which is literally the mathematical representation, because we're all about the math here.
It's a mathematical representation of how society values different groups.
So social welfare function is some function of the utility of individual one, the utility of individual two, all the way to the utility of individual 350 million, however many people we have now in the US.
It's literally a mathematical representation of how we take every individual's utility and come up with a measure of social welfare.
That's what the social welfare function's going to be.
So think about that.
Let's look at figure 23-1, which shows what we call isowelfare curves.
Think about a society with two people, Ned and Homer, and think about the government's decision, or think about society-- excuse the word government, think about society right now-- society's valuation of the allocation of resources across these two individuals.
Each of these curves is meant to represent a social indifference curve.
In other words, think of society as having a utility function.
Think of this being a social utility function, and the utility function has indifference curves.
So society has some indifference curves.
So what this depiction says is society's indifferent between Homer having u1h and Ned having u1n, that is, with Ned having a lot and Homer having a little.
Or Homer having u2h and Ned having u2n, that is Homer having a lot and Ned having a little.
So I've just made this up.
I've drawn this curve such that society is indifferent between those two allocations.
And basically, these isowelfare curves will look just like our indifference curves.
So we've talked in this course about choosing between goods.
We've talked about choosing between states of the world, injured not injured.
We've talked about choosing between periods of time, today versus tomorrow.
Now I'm talking about the hardest one of all, choosing between people, choosing between the utility of people.
But it's the same damn principle, it's the same stuff we've always done, which is there's some utility function, now we call it a social welfare function.
You maximize it across its elements-- here it's the utility of Ned and Homer, and you develop some indifference curves.
And they have all the properties indifference curves always had, further out is better, that a society is happier if both Ned and Homer have more, and along the curve, you're indifferent.
So these are representations of how society feels about transfers across people.
So the big question then becomes, what does this social welfare function look like?
Mathematically, how do you represent this incredibly hard question of the trade off across people?
And this is an incredibly deep question, but the standard social welfare function that's used is something due to Jeremy Bentham, a famous English philosopher.
You can actually still see his head if you go to University College London.
Actually, it's not a representation of his head, they actually had him stuffed and his head on display, but then apparently, students would take it at night and use it for soccer.
So they now just have a representation of his head there at UCL.
But Jeremy Bentham developed what we call utilitarianism, the utilitarian social welfare function.
And utilitarianism simply says that the social welfare of society is simply the sum of each individual's utility.
So u1 plus u2 plus dot dot dot plus u350 million.
That's social welfare.
That is the social welfare of US society, is the sum of everyone's utility.
Very straightforward.
So it's a linear social welfare function.
So what this says is this says you would maximize social welfare by transferring from any one individual to any other individual, if the first individual has a higher utility that the second individual.
So basically you maximize social welfare, is that basically if you-- I'm sorry, let me say it another way.
If transferring $1 from me to you makes you better off than it makes me worse off, then we should do it.
Now, your first instinct might be, well, this isn't very liberal in a sense, this is just saying everybody's utility is the same.
That is, we consider Bill Gates is one of these people, and I'm one of these people, and you're one of these people, and the homeless guy in Harvard Square is one of these people.
And they're all just added up.
How is that fair?
In fact, let me ask the question differently.
Why despite that is utilitarian wealth, social welfare function can it be perceived as fairly liberal, despite the fact that everyone just gets added up?
Why in fact is social welfare function that's utilitarian call for transfers from Bill Gates to homeless person?
Yeah?
[UNINTELLIGIBLE]
Exactly.
If you maximize social welfare, what it's going to say is set marginal utilities equal.
If you differentiate this with respect to everybody's utility, you're going to get that the rule that you get from maximizing, the first order condition mathematically is going to be that everyone's margin utility should be equal with utilitarian social welfare function, which is very radical.
That says that we should redistribute until Bill Gates has the same marginal utility as the homeless person.
Thought of that way, that's incredibly radical redistribution.
It's not weighting Bill Gates equally to the homeless person, it's weighting his utility equal, but it's saying we should have massive transfers.
In fact, basically we should equalize income in society, effectively.
That basically, what we should do is-- in fact, let me be more precise.
If individuals are identical, which they're not, but if individuals had identical tastes, the utilitarian social welfare function would say that social welfare is maximized with an equal distribution of income.
Pretty radical.
If individuals are identical, social welfare is maximized with the equal distribution of income, despite the fact we started with this pretty un-obnoxious looking function, we're simply going to weight everybody equally.
We're not even saying we care more about poor people than rich people, we're saying weight everybody equally.
That fairly innocuous looking function delivers the radical implication that if everyone is identical, we should have an equal distribution of income in society.
And that's a typical social welfare function that we look at.
But that actually is not considered really a particularly lefty social welfare function.
In fact, if you talk to the left side of the philosophical distribution, they'll say this is not an appropriate measure to use.
They much prefer something like the Rawlsian criteria, named for the philosopher John Rawls.
The Rawlsian social welfare function is that the role that society's goal should be to maximize the well-being of its worst off member.
That a fair and just society is one which maximizes the well-being of the worst off person in society.
So that says social welfare is equal to the min of u1, u2, dot dot dot dot.
So it's a max-a-min criteria.
You want to maximize the min, maximize the minimum utility in society is the Rawlsian social welfare function.
Now, in some situations, this can deliver-- in fact, if everyone's identical, this would deliver in many cases the same outcome as utilitarianism.
But in some cases, it's much more radically lefty.
Think about it this way.
Suppose we had a situation where everybody but me and you, but me and this guy have equal incomes.
Everybody in the world's equal income is at $40,000.
But my income is $1 million, and his income is $39,999, he's got $1 less income than the rest of you.
And let's say that a new government is running for office that proposes to take away all of my money, so I'm down to $40,000, and give him $1 and bring him up to $40,000.
Now in utilitarianism, that would not be an appropriate thing to do, because clearly my utility is going to fall more from going from 1 million to 40,000 than his is going to go from going 39,999 to 40,000.
So under utilitarianism this would be a bad thing to do.
But in Rawlsianism, it'd be a good thing to do, because he's the worst off member.
So it doesn't matter what happens to me, it only matters what happens to him.
So Rawlsiansim is an incredibly radical theory, which posits that we only care what happens to the poorest guy, even if we have to like take all the money away from the richest guys, we don't care.
We only care about what happens to the poorest guy.
So that's a very radical sort of lefty view of how we should think about distribution.
On the other hand, we have a very radical-- well, some would say a more radical view on the right, which is a Nozickian social welfare function, named for the philosopher Robert Nozick, also at Harvard.
Rawls and Nozick were both at Harvard.
Nozick said this is just way too radical.
He said utilitarianism is way too radical, way too far to the left.
Forget Rawlsianiasm, that's just nutty.
Even utilitarianism is way too far to the left, because he says, basically what should matter in society is an equal distribution of resources-- I'm sorry, an equal distribution of opportunities.
And conditional on equal distribution of opportunities, if individuals take those opportunities and choose to do outcomes which give them different incomes, that's their problem.
So for example, let's say that we started with everyone in the world started with an equal distribution of income.
But let's say that MIT students loved to hear me lecture, and were willing to pay $100 to hear me lecture.
And so at the end of the year, every MIT student ended up $100 poorer and I ended up a ton richer.
Well, under both utilitarianism, and certainly under Rawlsiansim, that is an outcome which is not social welfare maximizing, because I don't care about that last $100 whereas you guys care a little bit about giving up the $100.
So clearly social welfare would be maximized if I gave some of that money back to you.
Under Rawlsianism, clearly I should give it all back to you, because all we care about is you guys being $100 poorer.
Nozick says this is crazy.
This is a voluntary transaction.
You guys voluntarily paid me $100, making me richer.
Why should there be any redistribution?
Why should social welfare be any lower from that transaction?
How could a voluntary transaction lower social welfare?
And this is sort of a right wing approach to thinking about social welfare, which is look, everybody should have equal opportunities.
It's not fair to have some person not have access to education and things.
But once everybody's got equal opportunities, there's no reason why if you make voluntary choices that lead to unequal distribution of income we should care.
Now this is actually very compelling.
It has a lot of merit, and really should make us think about our emphasis should be on opportunities rather than outcomes.
But it has one flaw, which is that in fact, most of the differences in society aren't due to choices, but rather due to outcomes outside people's control, often just luck.
So basically, if it was true that we all started equal and that any differences in income came through our choices, then this would be a very powerful way of looking at the world.
But in fact, if you look at most differences in income across people, they're not due to their choices, or to effort, or to other things you can actually figure out how to measure, they're due to unmeasured things.
Often it looks like luck.
And in that case, you would care about the fact that I end up richer than you.
So if I end up richer than you because you're willing to pay me, that's one thing.
But if I'm richer than you just because I won the lottery and you didn't, that's a different thing altogether.
And so basically, whether how we feel about the equality of opportunities view versus the equality of outcomes view depends very much on what we think is the source of difference in incomes.
If we think it has to do with effort and choices, then people can often favor this view, that if people by their effort and choices made more money, let them keep it.
But if people made more money through luck and things beyond their control, then people tend towards more this view, saying look, if you got your money through luck and things, there's no reason why we should have such an unequal distribution of income.
And that's sort of another tension in thinking about these social welfare functions.
There are no right answers here.
I don't mean through my tone or word choice to imply there's any right answer.
I'm just trying to lay out the arguments for these different ways of thinking about it.
Then finally, the last way we could think about it, and to many people, really the most compelling, is through what we call commodity egalitarianism.
Commodity egalitarianism, which is a fancy name for saying look, it doesn't matter who has what.
It just matters that everybody can survive.
This is kind of like a reinterpretation of Rawls.
It's almost a mix of Rawls and Nozick in a way.
It's saying look, what matters is that people aren't starving, that they have decent shelter, decent health care, et cetera, you can define your minimum.
But once you've defined a minimum, we shouldn't care about how rich people get beyond that minimum.
So basically, the key thing is that we should not care-- what commodity egalitarianism focuses on, the insight it brings, which is an important one, is it's not clear why relative should matter.
What should matter is absolute.
It's not clear why I should care how much richer I am than you.
We should just care that you have enough to live a decent, socially acceptable life.
And then if I want to get rich, God bless me.
So it's kind of like saying we care about the minimum, but it's not all we care about.
We care about making sure there's a minimum decent standard of living, defined however you want to do it.
Different people choose choose different things.
For some people, it's just food and shelter.
Other people could include health care, or clothing, or other things.
I'm not saying what the right minimum is.
Let's define some minimum, and then say beyond that if guys get rich, God bless them.
That's their prerogative, as long as the poor people have enough to survive.
And this is an incredibly interesting, compelling view, because it's a radically different view, which says relatives don't matter, absolutes are all that matter.
All they're making sure is that the absolutely people at the bottom have a decent standard of living.
And that's a very different view as well.
So basically, social welfare, when we talked about utility functions, I didn't really talk much, I just said, you write down a utility function, we often use square root of c, whatever.
I didn't really talk much about the form, I just said it should have diminishing marginal utility, and otherwise I said sort of all bets are off.
With social welfare functions, it's a lot more open-ended and a lot harder, and we don't even really know what the right standards are.
But it's just saying that we can't get away from asking this awkward question.
Because ultimately society does end up with an unequal distribution of resources, and given that, we can't get away from asking the question of how do we feel about that?
Economics is about asking uncomfortable questions.
And this is an uncomfortable question we have to answer, which is how do we feel about different distribution of income.
And the reason this is an issue-- Yeah?
[UNINTELLIGIBLE]
That's a great question.
I don't know.
I wouldn't ask you to write it down.
I mean, it's a much more complicated-- there's not a simple mathematical function with your Nozick or commodity egal.
Commodity egalitarianism, I guess the function would be that-- the social welfare function would be that there's some minimum u bar.
Below u bar, we have an infinite weight on you in our social welfare function.
But once you get to u bar, we only care about you the same as everybody else.
That would be sort of commodity egalitarianism.
Nozick, I don't know how you write it down mathematically.
Why does this matter?
This matters because it turns out in society we do have a very unequal distribution of resources.
So to see that, let's go to the next page, figure 23-2, this is from my textbook that I use for my course on public policy.
And this shows the income received by quintile in the US.
Let me explain what this table means.
Each row is a fifth of the population.
If we had an equal distribution of income, then each of these numbers would be 20%.
Each fifth of the population would have a fifth of the income.
But that's not true.
So if you look at 1967, the poorest 20% of people only earned 4% of the income.
And the richest 20% of people earned 44% of the income, or 11 times as much, 43.8, about 11 times as much.
That's an unequal distribution of income.
What's interesting is you can see over time how it's changed.
From 1967 to 1980, things actually got a little more equal.
The poorest share actually grew, and the richest share fell a bit.
Then this since 1980, things have gotten a lot more unequal, to the point where in 2007, the poorest 20% of people only controlled 3.4% of our resources.
That is, 1/33 of all resources in society go to the bottom 20% of the income distribution, whereas 1/2 go to the top 20% of the income distribution.
So the richest 20% of people get half the income that's earned in the US.
The poorest 20% of people get 1/33 of the income that's earned in the US.
So this is why this stuff matters, because there is inequality in practice.
In fact, it matters a lot in the US in particular because the next table shows how we do relative to the rest of the world.
So this compares us to what's called the OECD countries.
OECD is the organization for economically something something, it's a French acronym.
And basically, it's sort of close to developed countries.
It's a first and second world if you will.
It's different years because the data is collected at different frequencies.
But the rough facts are constant over time.
This shows the share of income in each of the quintiles across all this list of countries.
And the bottom next to last row shows the unweighted average across these developed countries compared to the US.
What you see is we are the second most unequal nation on this list other than Mexico.
We are more unequal than any other nation on this list.
On average, the poorest 20% of society has more than twice as much in other countries, whereas the richest 20% has about 20% less, 40% of the resources instead of 50% of the resources.
And so what you see is we have a much more unequal distribution of income than any other major economy except for Mexico.
So basically, this says we have an incredibly unequal distribution in American society by some absolute standard, just if you look at the numbers, it seems unequal, and relative to the rest of the world.
No country in history has ever had all these numbers be 20%.
Societies are always unequal, the question is just how unequal.
And compared to the rest of the world at least, we seem pretty unequal.
So that says that under a utilitarian function, or especially in a Rawlsian function, we should worry that we're not maximizing social welfare.
Under a utilitarian social welfare function, at least, it's going to be hard to think whether it maximizes social welfare when there's such an unequal distribution of resources.
Certainly in a Rawlsian function we're not.
Nozick, who the hell knows.
That depends on where we start with opportunities, how much is due to choice and how much is due to luck.
But of course, this doesn't speak at all to the last view, which is a commodity egalitarianism view, which says none of this matters.
All that matters is the measure of absolute deprivation.
And we have such a measure in the US, it's called the poverty line.
The poverty line is a concept that was actually thought up by the mid-level government bureaucrat in the 1960s.
What this woman, Molly Orshansky, did is she said look, the typical family in the US spends about a third of their income on food.
Let's figure out what a minimally nutritionally adequate diet costs.
That is, how much do you have to spend on food to have at least a minimally adequate diet.
No Starbucks, no Outback Steakhouse, just enough to actually have a nutritionally adequate diet, and then let's multiply it by three, and we'll call that what a family needs to live in the US.
Food's typically a third of consumption, let's figure out a minimally adequate diet, multiply it by three, call it the poverty line.
And basically, we've taken this exercise and used it ever since.
We simply update it by inflation to see how the poverty line changes over time.
So basically, we've just taken this concept and updated it by inflation over time.
The result of what you get is in figure 23-4.
This shows the US poverty line in 2006.
Actually, this is mislabeled, it's 2009 actually.
It's a typo in my book.
What this says is that for a family of one, the minimum standard that you need to live is $10,830.
And it goes up with family size because more people need more to eat.
Now, if you are from Mississippi or maybe North Dakota, this might look like kind of a reasonably large number to you.
If you're from Boston, this looks like insane, like how could someone live on $10,830 a year in Boston, New York, or DC, or San Francisco, or any major city?
And the answer is they can't.
The answer is that the poverty line has not been kept up appropriately, because we've simply taken this number and inflated it by inflation over time, but it hasn't accounted for the fact that people's consumption bundles have changed.
In particular, food is no longer a third of the typical person's consumption bundle, it's now 1/6.
So in fact, the poverty line should be much, much higher than it is, but it hasn't been updated over time in a meaningful way.
So you can think this is a very low minimum standard for what people need to live, and certainly below what anyone needs to live in any major metropolitan area.
That said, we can look at what's happened to the share of population in poverty, and that's in figure 23-5.
So here's the sort of commodity egalitarianism thing to look at, which is what's happened to poverty.
And what you see is there was a massive decline in poverty for everyone in the 1960s to early '70s.
We call this the War On Poverty under the Kennedy and Johnson administrations, a huge reduction in poverty.
What you see is that since about 1970, that reduction has continued for one group, for the elderly, their poverty rates continued to fall.
So in 1959, 35% of elderly people lived in poverty.
Then through massive expanses of a program we'll learn about in a couple of lectures called Social Security, that fell, so that today, only about 10% of elderly people live in poverty.
On the other hand, for everyone else, and especially for kids, it's basically flattened out or increased since 1970.
So basically, overall poverty rate hit about 13% in 1970, and it's been pretty flat since.
For kids, it fell to about 15%, and it's been bounced up and down.
It's now back up to about 20%.
So from a commodity egalitarianism view, even if we take the poverty line as a minimum acceptable level, there are currently about 40 million families in the US living below poverty.
40 million people, I'm sorry, living below the poverty line.
So even at this minimum standard, we've got a failure of redistribution.
We're not maximizing social welfare given that we've got 40 million people living below this standard.
So based on whatever standard you want to use, except the hard to evaluate Nozickian standard, we're clearly not maximizing social welfare at the current distribution of income.
We've got an incredibly unequal distribution of income, and even if all we care about is making sure people have a decent standard of living, we're also failing on that standard for upwards of 40 million people.
For a more realistic poverty line, it might be more like 60 or 70 million people.
So clearly there is an argument here for some income redistribution, whatever your social welfare function is.
So we can write down-- I'm not going to ask you to write down mathematically social welfare function and maximize it and solve for outcomes and stuff like that, but what I want to leave you with here is simply two points.
First of all, there are lots of reasons why we might want income redistribution in society under any of these alternatives.
That's point one.
Point 2 is our current distribution of resources seems sufficiently unequal that we likely do want some redistribution of resources in society.
For most social welfare functions you can write down, given the facts I've shown you, we would not be maximizing social welfare at the current distribution of income in society.
I'm trying to couch this in a way that doesn't make me particularly lefty or righty.
I'm just trying to couch this in a way which just draws on the uncomfortable fact that we have to talk about this.
We have to talk about income redistribution.
The only way to do so is to have some framework.
I've given you four frameworks and said under all of them, or at least most of them, we would be uncomfortable with the existing distribution of income in society.
Questions about that?
So what do we do about that?
What do we do about the fact that we have unequal distribution of resources in society?
Well, we redistribute.
We say look, let's get out Okun's bucket and start carrying some money around.
We don't like the fact that there's people living in poverty who can't eat.
We don't like the fact that the poor have so much less than the rich.
Let's start redistributing.
Once you do that is you have to recognize the leak in the bucket.
And recognize there's a trade off, that it's not so simple as saying, fine, let's take from the rich and give to the poor, because there's a leak in the bucket.
And so that as we take from the rich and give to the poor, we shrink the total size of social welfare.
And that gives a trade off.
And in particular, there's two sources of leakage that we have to think about.
The first source of leakage is that when you tax a rich person to take their money away, they work less hard.
And when they work less hard, there's less goods produced in society, and that shrinks the pie.
The other source of leakage you have to think about is that when you give money to poor people, they may themselves quit their jobs to qualify as poor so they can get the money, and that further shrinks the pie.
So both the taxing the rich and the giving to the poor are sources of leakage in the bucket.
And that's a trade off with dealing with these enormous distributional problems that I laid out.
So let's just go through one example, general example to make this point.
Let's say we have a society where everyone is equal, and everyone earns a wage of $20 an hour.
Equal society, everyone's equal in terms of their underlying preferences, and everyone earns $20 an hour.
But people work different amounts of hours.
Now they could work these different amount of hours because they're lazy, Nozick might say.
They could work these different amount of hours because they just can't find a job, or because they're disabled, or because they're not skilled enough to work a full-time job.
Whatever the reason is, we're going to stay away from what that reason is.
Although obviously, what the reason is matters for these social welfare functions.
Let's just say we have a society where everyone earns $20 an hour, but we have a distribution on how many hours people work.
And let's say that we want to start with a commodity egalitarianism view, and say we want to make sure that everyone has enough to eat.
We want to make sure that everyone has at least $10,000.
So we come in and we say look, we're worried about this distribution of resources in society.
We want to make sure everyone has at least $10,000.
So what we're going to do is we're going to give everyone a transfer.
And that transfer function is going to be equal to the max of 0 and $10,000 minus your income.
In other words, if your income is 0, we're going to give you $10,000.
If your income's $5,000, we're going to give you $5,000.
If you income is $9,999, we're going to give you $1.
Once your income is $10,000, we give you nothing.
So this is very much like a commodity egalitarianism view.
We're going to bring you up to 10,000.
We're going to bring you up to 10,000, but once you're above 10,000, we don't care about you.
So this is much less radical than the utilitarian solution.
This is just saying, we're just going to bring the poor up.
We're just going to deal with that.
However, if we're going to have this program, we've got to pay for it.
We've got to give money to poor people, where's that come from?
Let's say we finance the program by taxing higher income people.
What we're going to say in particular is we're going to have a tax rate tau-- so regular t is transfers, tau is taxes-- which is 0 if your income is less than $20,000, and 20% if your income is greater than $20,000, greater than or equal to $20,000.
So we're going to have a tax schedule.
We're not going to tax you if you're less than $20,000.
But once you earn more than $20,000, we're going to tax you at 20%.
So this is our redistributive scheme in society.
This is our Okun's bucket.
The rich put the money in.
We tax them to get the money to put in the bucket.
We bring it over and give it to the poor in the form of bringing everybody up to $10,000.
That's our redistributive scheme, the simplest possible redistributive scheme.
What does this do?
Let's now go to figure 23-6.
Pretty complicated figure, so let's walk through this slowly.
We initially have a budget constraint that runs from the intercept at 40,000 on the y-axis to the intercept of 2,000 on the x-axis.
So the initial budget constraint is the outer budget constraint here running from 40,000 on the y-axis all the way down to 2,000 on the x-axis.
That's because we have a $20 hour an hour wage-- I'm sorry, let me go back.
This is a consumption leisure trade off, like we do whatever we do in labor supply.
When we do labor supply, we do consumption leisure trade off, so you're deciding how hard to work.
How do you decide how hard to work?
You can do it by trading off consumption and leisure.
You can either take 2,000 hours of leisure and consume nothing.
Or you could take zero leisure, work 2,000 hours to consume $40,000.
That's your trade-off, or combinations in between.
And we have three individuals, A, B, and C. A is someone who works very little.
They take a lot of leisure and have very low consumption.
C is someone who works a lot.
They have low leisure and high consumption.
And B's in the middle.
Basic labor leisure trade off, questions about that?
Now, how does this new government policy affect the budget constraint?
Well, it affects it in two ways.
First of all, for those above $20,000, we now lower their wage.
Instead of taking home $20 hours an hour, they're only going to take home $16 an hour, because we're taxing them at 20%.
So for everyone above $20,000, we shifted the budget constraint in.
We have lowered the price of leisure.
We've lowered the price of leisure by taxing them.
So we've shifted that budget constraint in.
It's the same budget constraint up to 20,000, but now it pivots, and at 20,000, you're now on the lower segment.
With a slope, you can write there, the slope is $20 at the higher segment, it's only $16 for that lower segment.
That's the first change you've made to the budget constraint.
That's the tax part.
The second change is that for anyone with income below $10,000, we've now said no matter how hard you work, we're going to give you $10,000.
So now we've said, once your leisure is 1,500 hours, or your work is 500 hours, anywhere to the right of that, your income is always $10,000 no matter how hard you work.
If you earn $5,000, your income is $10,000.
If you earn $1, your income is $10,000.
If you earn $9,999, your income is $10,000.
So the new budget constraint is a flat segment starting at the intersection of 10,000 and 1,500 and moving all the way to the right.
That is, no matter how much leisure you take, above 1,500 hours of leisure, your consumption is always $10,000, so it's flat.
So the new budget constraint is the inner segment above, then the old budget constraint from $20,000 to $10,000, and then a flat from point B to point D. I'm sorry, not from B to D. Kill that last comment.
From the intersection of 10,000 and 1,500 all the way to the right, that's the new budget constraint.
Questions about that?
This is very important.
This is just an application to understand how these things affect budget constraints.
What does this do to people's choices?
Well, for a person like C, they are now taxed on their labor.
Person C doesn't care about the welfare program, they're rich, they don't care about this $10,000 thing.
But they do care about the fact they're now taxed.
Assuming substitution effects dominate, the price of leisure has fallen, so they choose more leisure and less labor.
So person C, who was at C moves to point E, working less assuming substitution effects dominate, more leisure, less labor.
Person A, for them, this is a no-brainer.
They can have both more leisure and more consumption by moving from point A to point D. Under this new budget constraint, they used to have about-- it's not labeled there-- but have about 1,700 hours of leisure and consume maybe $5,000.
Now they can have more leisure, they can have 2,000 hours of leisure and consume $10,000.
So they move to point D. They work less hard as well.
Person B, the way I've drawn this, this is critical.
The way I've drawn this, their indifference curve cuts through the horizontal segment running from the old budget constraint to point D. Why is that important?
Because that means that there'll be a higher difference curve out at point D, because indifference curves can't cross.
So if their old indifference curve cuts through that horizontal segment, it must mean that they would be happier out at point D. They're giving up some consumption.
This isn't a no-brainer like for A, but they get so much leisure it's worth it.
So you could have drawn B, if I'd drawn B higher up, their indifference curve wouldn't have crossed this horizontal segment, and then they wouldn't have wanted to move out to point D. So for A, it's a no-brainer.
they move to D. For B it depends on their indifference curve.
Yeah?
For C, doesn't he change to working harder and consuming less?
Because he moves [INAUDIBLE], so his leisure [INAUDIBLE]
Yeah, you're right.
This is drawn wrong.
You're right, we draw it so that income effect is dominating.
Good point.
I'm sorry, that should have been-- good catch.
Point E should be to the right of point C, that's a mistake here.
We should have drawn it with substitution effects dominating.
That would have been more leisure and less consumption.
He get less consumption, but he works more because of the tax.
But you're right, that should have been to the right of point C. Please correct that.
They should have been to the right of point C. Substitution effect should lead him to earn less.
So he earns less, although once again, it's ambiguous.
A clearly works less, that's unambiguous.
And B, it depends on their difference curves.
But the way we've drawn it here, B works less-- in fact, B works a ton less.
B goes from taking maybe 1,400 hours of leisure to taking 2,000 hours of leisure.
This is a huge change for B.
What this means is this tax and transfer system has massively reduced the amount of labor supplied in society.
Why do we care?
Go to figure 23-7.
We care because we initially were in equilibrium at point e1.
But when people work less hard, that's a shift inward in the labor supply curve that reduces the total amount of labor supplied and causes a dead weight loss.
We've distorted the economy by people working less hard.
Trades that would've made both parties better off are not happening.
Trades that would have made both parties better off are not happening.
We've distorted the economy and caused a dead weight loss.
And that is the equity efficiency trade off in a nutshell.
The equity efficiency trade off is we knew we wanted this tax and transfer system.
We were very upset about the number of people living in poverty.
But by putting this in, we've caused a dead weight loss, because it caused the rich people to work less hard and the poor people to work less hard.
There's leaks in the bucket on both sides.
This dead weight loss triangle is the leak in the bucket, the social waste that comes from this redistribution system.
Questions about that?
So that leads us to mask, was it worth it?
Is a transfer system like this worth it?
Well, there's a simple answer.
You solve the social welfare function.
Once again putting Nozick aside, we can't write that down.
But if you're utilitarian, you just solve it, because, in other words, what you say is look, two things are going on here.
Everyone's utility is falling because there's a social waste.
So each of those u's goes down, but the rich u's go down a little bit, and the poor u's go up a lot.
And you have to basically ask which of those effects are bigger.
In other words, if the dead weight loss triangle is very small, then clearly-- or if there's a small leak in the bucket-- then clearly, total utilitarian social welfare will go up with a scheme like this, because the poor people will be made so much happier, and the other people won't care much.
But if that dead weight loss triangle is huge, let's imagine half of society's output disappears for this transfer system, well, then probably social welfare will fall, because so many people are made sadder that the fact a few poor people are made happier isn't worth it.
Remember, 40 million people are in poverty, that means 300 million people aren't.
So if 300 million people, if they see their incomes cut in half so that 40 million people can see their incomes raised some, for most utilitarian social welfare functions, that won't be a good deal.
So basically, we can literally mathematically represent whether this transfer was a good thing or not by using this social welfare function to evaluate it.
Under a Rawlsian function, we know the answer.
We know this is a good thing, because Rawls doesn't care if we destroy all of society, as long as that bottom guy gets pulled up.
So basically, Rawls we know that this is a good answer.
Utilitarian, we don't know.
That's going to depend on the loss to the rest of us from a smaller pie versus the gain to the poor from getting more resources.
Questions about that?
All right, what we'll do is we'll come back next time, and we'll talk about what is the government actually do to actually affect redistribution in society, and how does it look?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so let's continue our discussion today of equity and efficiency.
We talked last time about the equity efficiency trade-off and the problem of the leaky bucket.
And we talked about how society would value transfers from one group to another and what the sources of the leak in the bucket might be.
In today's lecture, we're actually going to talk about what governments actually do to transfer resources across income groups and what effect that has.
Obviously this is a very big topic.
I'm sort of summarizing in one lecture what takes about half a semester in the course I teach on public policy.
But this will give you sort of an overview of kind of how we think about these transfer issues in the US.
And certainly if you want to learn more about it, you can learn more in 14.41.
So I want to start by talking about the first side of the transfer equation, putting money in the bucket, and that's taxation.
Talk about taxation, putting the money in the bucket.
We have a number of sorts of taxation in the US.
I mean, look at the first page of the handout.
This pie chart gives a breakdown of where we raise our money as a government in the US.
So if you want to redistribute, first you've got to raise money.
How do we raise it?
Well basically, the majority of the money we raise is raised through the income tax.
The income tax is a tax on families' incomes.
And importantly, it's what we call a progressive, so the majority of money raised from the income tax.
The income tax is what we call a progressive tax.
What that means is the richer you are, the higher share of income you pay in taxation.
Progressive.
As opposed to regressive tax, is one where the richer you are, the lower the percentage of your income you pay in taxation.
And once again, obviously progressive/regressive has some normative feel to it.
And the notion is once again, under most social welfare functions, we're going to want a function which redistributes from rich to poor.
And that's a progressive income tax system does.
So if you look at the next page of the handout, this shows for current year, I think it's for 2010, maybe 2009, what tax rates look like in the US.
Now it's important to remember a distinction between marginal tax rates and the taxes you actually pay.
What this graph shows is your marginal tax rate.
What that means is for the next dollar that you earn, what percent do you pay in taxes?
So for example, for someone who earns less than $16,700, for every dollar they earn, they pay $0.10 in taxes.
In fact, everybody pays that on the first $16,700 they pay.
So Bill Gates, on the first $16,700 he earns pays $0.10.
Then on every dollar beyond that, you pay $0.15 till you've earned $67,900.
Then you pay $0.25 and so on until on every dollar above $373,000, you're paying $0.35.
And the key point is these are the marginal rates.
So if your income is $350,000, then your marginal rate is 33%.
You're on the next to the last bracket.
But along the way, you've paid lower rates.
You pay the 10% on your first $16,700 and so on.
But the bottom line, this is a progressive system where the higher your income, the more you pay in tax on the next dollar earned.
OK, and that's the way the income tax works.
And there's a lot of other complicated features we can get into.
But roughly speaking, you take your income and you tax it progressively.
And that determines what you pay the government.
Now if we flip back to the first page, the second major source of revenues for the US government is payroll taxation.
This is different from income taxation in that this is a flat percent tax.
So this is not progressive or regressive, it's neutral.
It a flat percent of your income you pay in payroll taxation.
So unlike the income tax where the richer you are, the more you pay.
Here you pay a certain flat percentage regardless of income.
And this money goes to finance the nation's-- what's called the nation's social insurance program.
And we'll focus on that on Monday.
But basically goes to finance programs that help people if they suffer negative risks, like get unemployed or need health care, et cetera.
That's financed by the payroll tax.
The third source of taxation is consumption taxation.
Consumption taxation.
As we can see in the US, that is the third largest source of revenues, about 15.7% of government revenues come from consumption taxes.
Now these are of two types.
These are taxes on consumption.
There are two types.
One type is the sales tax.
So in Massachusetts this is the 6 and a quarter percent.
Everything you buy in certain categories, you pay 6 and a quarter percent extra that goes to the state.
The other is excise taxes, which are specific taxes that are levied on specific goods.
So there's an excise tax on cigarettes.
You pay a certain dollar amount per pack of cigarettes in excise tax.
Excise tax on alcohol.
Excise tax on gasoline.
So these are specific taxes on goods.
And the important thing is these consumption taxes are often called indirect taxes.
Because unlike income and payroll taxes where you earn $1, you pay tax on it.
Here you don't pay the tax till you spend the money.
So the consumption tax does not tax you directly on your income.
It taxes you as you use your income to buy things.
So it's often called the indirect tax.
The fourth major source of tax revenues is the property tax.
This is a tax that you pay on your actual wealth.
This is the third kind of tax.
So the first kind of tax we tax you on your earnings, either through income or payroll taxes.
A second kind of tax, they tax you on your consumption.
A third kind of tax, they tax you on your wealth.
So literally every year, you pay a certain fraction of the value of your house, for example, in a property tax to your local government.
So it's another form of taxation.
That's where we get about 10% of our revenues.
And then finally, there's the corporate tax, which is a tax-- this is sort of akin to the income tax.
But instead of levied on individuals, it's levied on corporations.
It's money that corporations pay as they earn more in profits.
So we tax lots of different sources of income.
We tax you lots of different ways in the US.
If you add it up overall, we pay about 20% of our income in taxation.
That is every dollar that's earned in the US, about $0.20 goes to the government on average.
Now obviously it's different.
If you're richer, it's higher.
If you're poor, it's less.
It depends on how much you consume, et cetera.
And your wealth, et cetera.
But overall across everyone, on average, about 20% of our income goes to taxation.
About one fifth of our GDP.
The problem we have right now is if you look at government spending, that's more like a quarter.
It's 24% of GDP.
So we collect about a fifth of our national income in taxes, but we spend about a quarter.
Thus we have a more than trillion dollar national deficit.
So the problem we have right now is we're collecting a lot less in taxes than we're spending as a government.
Now, we're not going to get into what's behind that.
That has both some structural sources, which we'll talk about.
Most notably the incredible rise in medical care spending.
And it has some cyclical sources, which is in a recession, naturally you spend more because people need more help from the government and you tax less because there's less income to be taxed.
So some of the reason we have this huge deficit is that we're in a recession still.
We haven't come out of it yet.
Some of it is more structural in that we have fundamentally a system which is spending beyond our means.
And we'll talk a bit more about that.
What I want to focus on now is given this large set of different things we should tax, I want to focus on one specific question.
There's lots questions we could focus on.
And once again, in 14.41, we talk about a lot of them.
But I want to focus today on one question of particular interest.
Which is, what should we tax?
I've just laid out here five different things we can tax.
We can tax your income, either progressively or in a flat tax.
We can tax your consumption.
We can tax your property.
We can tax corporations.
What should we tax?
If we're going to raise this 20%, why do we do it in all these different ways?
For example, in Europe, taxation is very different.
In Europe, they raise much less through income taxation and much more through consumption taxation.
They have something in Europe called the value-added tax.
You guys may have dealt with it if you've traveled there, traveled abroad-- the VAT.
The value-added tax is basically their version of the sales tax.
It's basically a sales tax, but each level of producer is tax on the value they add to production.
And so in Europe, they tax consumption a lot more and income a lot less.
Is that a good idea or not?
For example, one of the two major deficit commissions that's just reporting these last couple weeks on ways to get down the deficit, has suggested we actually introduce a national sales tax to move towards more like Europe and have more of our taxation based on consumption and less based on income.
What's the major argument for this?
Well, the major argument for taxing consumption instead of taxing income comes back to what we talked about a couple lectures ago.
Which is that it promotes savings.
Remember, income can be defined as consumption plus savings.
You take your income and you either consume it or save it.
OK When we tax income, then we tax both your consumption and your savings.
When we tax consumption only, we don't tax your savings.
Assuming substitution effects dominate, that will therefore promote savings.
Taxing consumption rather than taxing income will promote savings.
Once again, assuming substitution effects dominate.
And the argument is, we know through mechanisms we talked about last time how important savings is as an engine of growth.
So the notion is that by moving from a system of taxing income to a system of taxing consumption, we can say to individuals, hey, you will have a tax benefit to saving rather than spending.
And that tax benefit you have from savings will cause you to save more.
And therefore, we'll increase savings in society from doing this.
And that's why many economists favor moving away from an income tax to a consumption tax.
Actually, this was first proposed by the depressing philosopher Thomas Hobbes back in 16-something where he said, "A man should be taxed-- because it was all men back then.
"A man should be taxed not based on what he earns, but what he takes out of society through consumption."
That's sort of the philosophical underpinnings of saying let's not tax people on what they make, let's tax them on what they use, which is their consumption.
So that's got both a philosophical merit to it and also this sort of efficiency argument of promoting savings.
So why not do this?
Well, it's our friend that we've been talking about these two lectures, the equity efficiency trade-off, which is a tax on consumption is very regressive.
It falls much more heavily on the poor.
And why is that?
Well, quite frankly because the poor don't save and the rich do.
The typical American lives pretty much hand to mouth.
They pretty much spend what they earn.
They don't save a whole lot.
Most of savings in society is done by the richest people in our society.
The vast majority of wealth is controlled by a small share of the population.
As a result, if you went from an income tax system, which is progressive, to a consumption tax system where you're just taxing people on what they spent, you would end up moving vastly towards a much more regressive system.
Now partly you could address this by taxing consumption progressively.
But at the end of the day, the rich just don't consume a lot of what they earn.
They pass it on to their kids.
So at the end of the day, the rich will just pay a lot less in taxes if you move to a consumption tax system.
And that's the issue.
Now in Europe, what they do is they address this problem by saying, fine, our tax system is regressive, but we're going to spend a lot of money on the poor.
So I talked last time about a system making sure nobody lived in poverty.
Everybody got $10,000.
No one lived in poverty.
That's more of a European-style system.
So in Europe they say, yes, we have a more regressive tax system, but a much more progressive spending program.
And put together, it's a fairer system.
And that may be something to consider.
But within the tax realm alone, moving to consumption taxation will probably promote efficiency, but would hurt equity.
And once again, we have that trade-off that we're always facing.
Questions about that?
A more interesting case where this trade-off might not be quite so stark or a little more subtle is thinking about excise taxation of "sin goods."
"Sin goods."
So if we think about what's taxed by excise taxation, they're on things like cigarettes, alcohol, gasoline.
Basically, goods which produce what we call negative externalities.
What a negative externality is, is an activity which produces a negative consequence that is not borne by the person engaged in the activity.
It's not borne by the person engaged in the activity.
There's a negative externality that's associated with these goods.
So for, example let's take smoking.
When I smoke, part of what I'm doing is just killing myself.
And that's not an externality.
I'll come back to that.
But if all you do through an activity is hurt yourself, that's not an externality.
An externality is the cost imposed on society.
The key insight from basic economics is that anything you do that hurts only you is not society's business.
So for example, every cigarette you smoke lowers your life by 7 minutes.
Now not specifically, but on average, every cigarette smoke lowers your life by 7 minutes.
However, in a world with rational consumers, the type we deal with in 14.01, that's not a problem.
When you go to buy that pack of cigarettes, you should say, look, in addition to the $5 I have to pay, I'm lowering my life by 140 minutes.
I will decide whether my enjoyment of smoking is worth the shortening of my life plus the money I have to pay.
If it is, I'll buy it.
If not, I won't.
I'll go off and smoke and kill myself or not, but that's my problem.
That's the standard economic view of this, which is that basically what matters is not the damage to yourself because that's a trade-off you make.
You have an indifference curve across life and smoking.
You have an indifference curve.
And if you like smoking a lot, you'll choose to have a shorter life to smoke.
If you don't like smoking, you won't.
But that's a choice you've made and the government has no role to interfere with that.
Where the government has a role to interfere is when there's a negative externality.
When the consequence of my action affects other people and I don't bear the cost.
So, for example, when I smoke, if I'm, for example, on Medicare, over 65.
If I smoke and get sick, then the medical costs that are borne are borne by the taxpayer.
Because I'm on public insurance.
So when I'm over 65 and I'm on free public insurance, which a lot of people aren't over 65.
I'll talk about that in a couple lectures.
And I smoke, than those costs are borne by society because they have to pay the cost of my medical bills.
Or more relevantly, take secondhand smoke.
If I smoked in this classroom and you all got lung cancer as a result, that would be an externality because I wouldn't be bearing the fact that you got sick because I smoked.
That's a negative externality.
Higher medical costs alone associated with smoking are $80 billion a year.
Not to mention the secondhand cost of smoke.
Take drinking.
What's the externality for drinking?
Now many, I would gather, venture most people in this room have had consumed alcohol.
Often we've consumed it and quite enjoyed it and consumed it responsibly.
However, there's enormous negative externality associated with alcohol, which is drunk driving.
Every year about 13,000 people a year are killed by drunk drivers and about 400,000 are injured.
Once again, that's an enormous negative externality because I drink, I get drunk, I kill someone.
That's a cost I've imposed on society that I don't bear.
I'm going to be guilty and stuff, but I'm not bearing that cost.
That's a negative externality of drinking.
Consuming gasoline clearly has a negative externality, which is the more I drive, the more carbon I emit into the atmosphere, and the more I cause global warming.
The externality is that basically by the best estimates, global temperatures due to global warming, which a large share of it is caused by driving, global temperatures will be up 5 to 10 degrees by the end of the century.
Now if you're from North Dakota that may not sound like such a bad thing.
But if you were say, from Bangladesh, it might be because it'll be underwater.
Or if you happen to like visiting Cape Cod, it might be because it will also be underwater.
And these are the aspects of global warming that are largely caused by activities such as driving.
It's a negative externality.
By my driving, I'm putting Bangladesh underwater.
I'm not paying for that, so that's a negative externality.
And then finally, we get to the toughest one and the most interesting one.
And perhaps the most important one going forward, which is obesity.
Individuals who overeat and get fat and overweight as a result cause an externality because they have extra medical costs that society has to bear.
Currently, one third of our nation is obese.
And one in three children born today will get diabetes.
Largely from overeating or poor lifestyle.
That's an externality on society and the extra medical costs that we'll bear.
Since society bears these costs, society then has the right to say, well, I'm going to tax you on the costs you're imposing.
We call that corrective taxation.
If you exert a negative externality on society, then society has the right to come and say, OK, we are now going to correct that by taxing you on the cost you're imposing on society.
So that's an argument for excise tax that goes beyond the normal equity efficiency trade-off.
Any tax has the normal equity efficiency trade-off.
Sin goods have this extra argument in the pro column, which is the negative externalities.
Which is that even aside from the standard equity efficiency trade-off, because consuming sin goods imposes negative externalities on society, we should tax them more.
And that's why we have excise taxes above and beyond our sales taxes.
Questions about that?
Now, traditional economics often stops there, but I hope that my discussion of shortening your life by smoking left you at least a little bit uncomfortable.
I hope that when I said, well, you shorten your life by smoking, that's your problem.
You might say, well, gee, that leaves me a little uncomfortable.
And the reason you might say it leaves you uncomfortable is you might think, well, maybe people don't understand that.
Maybe people don't realize that every cigarette they smoke is shortening your life seven minutes.
And maybe they don't realize it in particular when they're 16 and start smoking.
And then they get addicted and they can't stop.
In that case, we have actually understated the argument for taxing these goods because then we actually want to tax them to help you from killing yourself.
So for example, you take high school seniors who smoke a pack a day of cigarettes.
And ask them, will you be smoking in five years?
Of the ones that say, yes, I will be smoking in five years, if you actually follow them up five years later, 72% of them are smoking.
So you got it pretty much right.
Of the ones that say no I will not be smoking in five years, when you follow them up five years later, 74% are smoking.
They completely got it wrong.
Clearly, the kind of underlying rationality assumptions we make in this course don't hold in some context.
In that case, there may be a role for the government to tax these sin goods even above and beyond externalities.
Basically, there maybe a role for the government in helping people help themselves, which is a dangerous place to go for economics.
We tend to think of the government as rolling in and fixing mistakes that might affect society.
But people, what they do for themselves is perfectly fine.
If they want to do crazy things, that's their business.
What behavioral economics leads us to, where these sort of facts leads us to is thinking, well, if people make mistakes, there may be a role for the government in addressing those as well.
That's a new area that economics is pushing in, is thinking about, well, gee, if individuals are actually not behaving in the perfectly rationally way we learned about in 14.01, then is there an even more aggressive role for the government in correcting their behaviors?
And this is discussed a lot in my course 14.41.
It also is discussed a lot in 14.13 Behavioral Economics, which talks a lot about these issues, about these sort of issues of sort of how moving beyond 14.01 might actually impact our thinking about the proper role of government policy.
So the bottom line is, we should clearly tax these sin goods more because of the negative externalities they impose.
Because they impose costs on society through higher medical costs, or more drunk driving deaths, or global warming.
And perhaps we should tax them even more than that if people are actually making mistakes in their consumption decisions, and the government has a role to come in and help correct those mistakes.
Questions or comments on that?
Now, I just talked about what we should tax and some of the issues in deciding what we should tax.
The other, of course, issue is, well, how much should we tax?
So not just what's the right tax base, income or consumption or excise taxes or whatever, but what's the right tax rate?
And this is, of course, a very important issue today because the number one public policy issue we're dealing with now is the expiration the Bush tax cuts.
So if you go back to your chart, the second page of the handout, the marginal tax rates.
Before 2001, these tax rates were all about 5% higher.
So the poorest paid 15% and the richest paid 40%.
In 2001 and in 2003, the Bush administration cut tax rates a lot, cut these tax rates, cut a bunch of other taxes, including corporate taxes, as well as the property tax and other things.
But everyone's focused right now on the individual tax rates.
They did so, but to make-- due to some sort of budgetary trickery they had to do to make it work, those tax cuts actually expire in a month.
What that means is if nothing is done in a month, everybody's income tax rate jumps up by 5%.
So you can imagine that's not delightful politics.
Politicians are very upset and worried about this.
I don't know that people are so much, but politicians are.
And the current debate right now is, well, what should we do about this?
Should we extend the Bush tax cuts?
Should we continue to keep tax rates lower?
Or should we let them expire and grab the extra revenues, as well the increased progressivity that comes along with that?
Well, should we?
Well that depends on two issues, equity and efficiency, the same issues we've been discussing.
The efficiency issue is, well, what will the impact be on the economy from allowing tax rates to rise?
And in particular, some argue there could be such a negative impact that you actually end up hurting the government by allowing tax rates to rise.
And this argument that appeals to a famous notion known as the Laffer curve, named for a conservative economist, Arthur Laffer, who advised Ronald Reagan on his tax cuts in the early 1980s.
The Laffer curve argument is illustrated in Figure 24-3.
So the point is, imagine two tax-- I want you to consider three possibilities for tax rate: 0, 100, and something in between.
A tax rate of 0 clearly raises 0 revenues.
But a tax rate of 100% also clearly raises 0 revenues.
Why?
Because if you're taxed 100%, you'd never work.
If literally everything you ever made simply went to the government, people just wouldn't work.
As long as leisure's a normal good, why would you work?
Why would you reduce your leisure if you didn't get to see any increased consumption from it?
So at a tax rate of 0, the government collects no revenues.
At a tax rate of 100%, the government also collects no revenues.
And the tax rates in between, the government collects at least some revenues.
Given those three facts, we know there must be some function that looks like the one that's drawn here.
Some parabolic function.
We don't know its exact shape.
But basically where it hits the x-axis at 0 and 100 and is above the x-axis in between.
It's just a theoretical truism that you'll get a curve this shape.
The question is, which side of this curve are you on?
If you're on what I've labeled the correct side, then what that means is by raising taxes, we raise revenues.
But there is inevitably an incorrect side, a wrong side, where taxes are so high that by raising taxes, we lose money.
How is that possible?
Well, think about what tax revenues are.
Tax revenues are the tax rate times the tax base.
Well, what happens to tax revenues when we increase taxes?
dR d tau is equal to B plus dB d tau times tau.
That is, when you increase taxes, you raise more money because you're raising more money on the existing tax base, but if the tax base itself shrinks-- this is negative-- then there's an offsetting effect.
The tax base could shrink so much that you end up losing money.
And that's what happens on the wrong side of the Laffer curve.
Taxes get so high that when you raise a tax rate, people work so much less it's like the monopolist poisoning effect.
The poisoning effect overwhelms the initial effect and you actually lose money by raising taxes.
Just like a monopolist can lose money by raising prices-- by lowering prices.
I'm sorry, a monopolist can lose money.
Here the government can lose money by raising taxes through the same kind of poisoning effect.
And that's how we get this Laffer curve.
The first issue is, where are we on this curve?
And the answer is that we're clearly currently on the correct side.
We're clearly at our current rates, and even if the rates go up under the Bush tax cut, we're clearly on the correct side.
Evidence is clear on this point.
More generally, the evidence is that the efficiency cost of taxation today is around 40%.
So the deadweight loss of taxation is around 40%.
That is for every dollar we put in the bucket, about $0.40 leaks out by the time-- for every dollar we try to get from a rich guy, about $0.40 leaks out before we can get to the poor guy.
So we've got about a 40% leak in the bucket.
The Laffer curve would imply we had more than 100% leak in the bucket.
That we try to take a dollar from the rich guy, we actually end up getting negative money because we actually lose overall.
So we're clearly far from the wrong side of the Laffer curve.
But clearly there's still some leak in the bucket.
And this comes to the issue of, well, how do we feel about that?
This is what we started last lecture with.
And that depends on our social welfare function.
So here's the way to think about it.
Currently what the Democrats have proposed, for example, is to get rid of the Bush tax cuts for everyone making more than $250,000 a year.
If you do that, you would raise on the order of-- you get about the fifth of the money you get from getting rid of all the tax breaks.
That is, if we extend the tax breaks for people below the richest group, but get rid of the tax breaks above the richest group.
So basically, the tax code stays the same except this top rate jumps up.
So basically the top two rates, about halfway through the next to the last bracket and the last bracket, they jump up and everything else would stay the same.
You'd raise a lot of money.
You'd raise about $700 billion over the next decade, which would go a long way toward solving the deficit problem.
Not solve it, but go a long way that way.
And you do so in a highly progressive fashion in the sense that you would only tax the richest people.
On the other hand, there'd be a very big efficiency loss.
So $0.40 on every dollar raised.
So the question is, is it worth it?
Well, that just depends on the social welfare function.
My guess is for a utilitarian social welfare function, it would say it's worth it.
Because basically the marginal utility of the rich would be so much lower than the marginal utility of the poor, that even with a 40% loss, it'd still be worth it.
But you could certainly write down social welfare functions where it's not worth it.
And that's basically what the debate needs to be about.
The debate needs to be about, how do we feel about the benefits of redistributing from the rich to the poor and raising this money versus the cost in terms of the leak in the bucket?
And that's essentially what the debate needs to be about.
Now it's not what it is about in Washington.
It's much more about other political factors, not just economics.
But that's the right way to think about evaluating it.
Questions about that?
Yeah
I don't see how, if you say the deadweight loss is 40%, it's completely lost.
If people are choosing to go to leisure, they're still getting some value from that leisure.
Couldn't it actually be that that prevents being [UNINTELLIGIBLE]
That's right, they're getting value.
This is above and beyond the value from the leisure.
So remember our deadweight loss triangle.
That very first person who switches from work to leisure, you're right there is no deadweight loss.
Because that person was indifferent between working and being in leisure.
But as your deadweight loss triangle grows, remember now you're talking to people who are no longer indifferent.
They would rather work than be in leisure.
So by forcing them out of work by taxing them, you are losing the gap between how productive they'd be at work and how much they value their leisure.
So you're right, if all this tax did is take the one person in society and cause them not to work, then we wouldn't worry about it.
It's people away from that where they're really much more productive at work than they are at home.
That's where the efficiency loss comes from.
Other questions or comments?
So that's one side.
That's putting the money in the bucket.
What about taking the money out of the bucket?
That's the other side of the efficiency story and the equity story.
Well what about low income transfers in the US?
So we talked about taxation, now let's talk about transfers.
We have several types of transfers in the US that we make.
The most prominent is what we call categorical cash transfers.
Or what's often called welfare.
Now we use welfare in a different context in this course, so it's confusing.
We're hearing the term "welfare."
What they mean is money that's sent to poor people.
So when a regular person, not an econ geek says welfare, they don't mean social well-being, they mean money that's sent to poor people.
A categorical cash transfer.
The categorical term means that we don't in the US just send money to you because you're poor.
We send to you because you're poor and other things are true.
So for example, we have a program called TANF, Temporary Aid to Needy Families.
This is cash that's sent to single parent families who are low income.
So if you're low income plus you only have one parent, then you qualify for TANF.
Actually the biggest cash transfer program we have is called SSI, Supplemental Security Income.
This is sent to families that are poor and are disabled.
Sent to people, I'm sorry, that are poor and disabled.
So if you're just poor, we don't give you anything.
But if you're poor and disabled, you get money.
If you're poor and a single mom, you get money.
Why?
Why do we impose these conditions?
Why do we do this, what's called targeting?
Why do we do this targeting rather than just saying if you're poor, you get money?
Now basically, the reason is because of what we saw the last lecture.
Which is we saw in the last lecture the hazards of just giving money to people because they're poor.
Which if you just give money to people because they're poor, then people will people poor to qualify.
So we saw last time in that diagram was we said anyone below $10,000 gets bumped up to $10,000, then everyone quits and says, I'm a zero income guy, give me $10,000.
However, if we, for example, knew for sure-- let's say that we are born with stamped on our forehead our underlying earnings ability.
And I could look at you and say, you're a $5,000 guy.
Here's $5,000.
You're a $20,000, you don't get anything.
OK, you're a zero guy, here's $10,000.
If I could read that off a forehead, then there would be no efficiency loss from transfers because I wouldn't change people's behavior.
I'd know what they were going to do anyway and I'd just give them the money.
But there would be no efficiency loss.
There'd be no movement of people could earn money moving to not earning money because I wouldn't give you any more money than you could earn.
So if we could perfectly target, we could get rid of the leak in the bucket that comes from transfers.
But of course, we can't.
So what do we do?
We try to find things which are correlated with having a low earnings ability.
Like, for example, being disabled.
If you're disabled, we know you're much less likely to earn a living than if you're not disabled.
Therefore, we can transfer money to you and cause less deadweight loss.
So if we take someone who literally we know for sure, someone who is mentally incapacitated, quadriplegic, basically can't function.
We know for sure they're going to earn zero.
So there's no deadweight loss in transferring money to them.
There's a deadweight loss of raising the money.
But in terms of their behavior, there's no deadweight loss.
So the more we can target in that way, the more we can safely redistribute.
Now, the trick with this, of course, is finding the targeting mechanism.
And a good targeting mechanism needs to have two features.
The first is, it has to find the poor people.
You want a target on something which is actually like being poor.
Like we could say, I'm going to redistribute to everybody who is a natural blonde and I could have some test for a hair color.
And that's immutable.
What your natural hair color is immutable.
But that wouldn't redistribute resources in the way you necessarily want in society.
Natural blondes aren't necessarily any poorer than non-natural blondes, or other people.
So first, somebody that's poor.
And second of all, we want something ideally that's unchangeable.
That is, I could redistribute to everyone who's blonde.
Let's say blonde people were poorer.
But I can change that by dying my hair if you can't tell if I'm a natural blonde or not.
So we want something which goes to the poor, but which is also unchangeable.
So for example, someone who's severely disabled, that's a good example.
Someone who's severely disabled, nobody's going to become severely disabled to qualify for some cash.
Actually, there is an exception.
There was a place in Florida they called Nub City where people were actually chopping off their limbs to qualify to get cash benefits.
There was a point in the US where one third of all limb loss accidents in the whole nation came from one city in Florida.
So that disturbing example aside, we don't think people will disable themselves to get cash.
Plus, they're poor people.
Single motherhood, that's a bit trickier.
Being a single parent, isn't it possible that people might become single parents to qualify for the cash?
Let's say my wife and I are poor.
We say, look, let's divorce.
We won't be married, we can still see each other.
But you'll get a cash transfer because you'll be a single mom.
Well then that isn't unchangeable.
So the question then becomes empirically, to what extent is single motherhood, for example, a good targeting device?
And the answer is, it's pretty good.
It turns out there's very little of this behavior.
We have lots of clever ways of testing whether people are doing this.
Turns out people don't do that.
Single motherhood is something people really don't want.
And basically if someone's a single mother, it's a pretty unchangeable indication that they're poor.
In fact, it turns out that the biggest problem we have in targeting is not poor disability, but another program we have, which tries to target money to people who hurt themselves on their job, something called workers' compensation insurance, which is insurance for people who get hurt on the job.
Turns out there, it's pretty easy to fake getting hurt on the job.
And as a result, that's not unchangeable.
Even disability is not perfectly unchangeable.
There's a lot of evidence actually because most disabilities today are not people who are quadriplegic or other severe, extremely sad cases.
It's people who have things which are hard to measure, like mental disability or back pain where there's no quantifiable, truly quantifiable test.
In that case, it's hard to know if you're really targeting to someone who need its or someone who's just good at faking.
And that leads once against, to the difficult trade-offs in equity versus efficiency.
On the one hand, we'd like to target to people who really need it.
On the other hand, if someone's just faking, we don't want to be giving them money.
And that's exactly the trick that we have to face in these kind of transfer programs.
Now in this world of difficulty, there has emerged a clear winner.
And the clear winner is something we call the Earned Income Tax Credit, the EITC.
So once again, the problem is if we just give money to poor people, we have the problems we said last lecture.
If we try to target to needy groups, we have a problem that sometimes targeting devices aren't perfect and people may change their behavior to qualify.
The third approach is something called the EITC.
Which is instead of just giving people money, we actually give them a transfer conditional on their work.
This is a conditional cash transfer.
What the EITC is, is it's a wage subsidy.
It's literally the more you work, the more you get from the government up to a certain point.
So to see this, let's go to the last figure in the handout.
Figure 24-4 shows the structure of the Earned Income Tax Credit.
Here's how it works.
If your income is below $12,570, this is for a family with two kids, I believe.
If their income is below $12,570, then for every dollar they earn, they get $0.40 extra from the government.
So it's a negative tax.
Instead of being taxed, they actually get a subsidy for every dollar they earn.
So for every dollar you earn, you get $0.40 from the government.
Until you've got the maximum of $5,028.
But then, once your income's above $16,400, that's then taken away at a rate of $0.21 So it's an after tax now of $0.21 per dollar you earn.
Until by the time you've earned $40,295 it's gone.
So this is what we call a targeted conditional cash transfer.
It's targeted to low income groups in that it phases out as your income goes up.
But it rewards work by basically saying that the more you earn up to a point, the more you will get.
Now the EITC effects are somewhat complicated.
I hope you can see right away.
You might say, wait a second.
This is a little bit complicated.
Because on the one hand, for anyone earning less than $16,400, you're subsidizing their work.
Or anyone earning, sorry, less than $12,570, you're seeing the more you earn, the more you get.
But once your income's above $16,400, then you're actually taxed.
Because we've given you this check and we take it back away.
That's the same as taxing you.
So for example, consider a guy who's at $16,400 who's considering earning $100 more.
Or consider a guy at-- yeah, so a guy at $16,400.
And he's thinking about earning $100 more.
So currently his income is $16,400 plus the $5,028 check he's getting from the government.
So his income now is 21,248.
I'm sorry, 5,428.
My bad.
A little dyslexia.
21,428 is his income.
Now let's say he decides to earn $100 more.
If he earns $100 more, his wage income goes up to $16,500.
His wage will go up to $16,500.
But his EITC falls by $21 to $5,007.
So his income only goes up to $21,507.
His income goes up by less than $100 when he earns $100.
So while the EITC incentivizes you to work when you're low income, it disincentivizes work when your income goes up.
So this is once again why public policy is fun and hard, which is there's a trade-off.
There's always a trade-off in this course.
It's really annoying.
There's a trade-off, which is on the one hand, we want to target our program to give the money to the lowest income groups.
On the other hand, if you target something, that means you have to take it away.
Targeting equals taking away.
What that means is by targeting to the lowest income groups, we are taking it away from these middle income people and taxing their work just as we subsidize the work of the lowest income groups.
Now, so how do we think about this?
Well, what we do in this case is we go to empirical evidence.
The EITC has grown enormously over time.
And nicely, it's grown differently for different groups.
For example, it's gotten a lot bigger for families with lots of kids than for families with no kids.
So what we can do is we can actually study what's happened to those kinds of families over time as the EITC has gone up.
What's happened to their labor supply.
And what you see, it's quite striking.
Which is there's been an enormous increase in the share, especially of single mothers who have gone from not working to working because of the EITC.
But you don't see them working less hard because of EITC.
That is, you see a lot of people pulled-- if we go back to our diagram.
See a lot of people pulled from the zero earnings point into the positive earnings range.
But you don't see a lot of people in the positive range earning less.
So the good part of the EITC has worked and the bad part hasn't really cost us anything.
Why is that?
Well, there could be two reasons.
One could be because people are a lot more responsive in their decision to work than their decision how hard to work.
For example, you may not even choose how hard you work.
Maybe you work 40 hours at McDonald's or you don't work.
So the first answer is the decision to work or not may be the most elastic, more elastic than how hard you work.
The second could be that people just don't understand this program and they know they get a check, but they don't know how to do this kind of math.
So they say I now I get a check if I go to work.
I'm going to work.
But they don't figure, by the way, if I work one hour less, maybe I'll lower my tax bill by $21.
Whatever the reason, the EITC has worked.
It has solved the problem of the leaky bucket.
In fact, it's put money into the bucket at the bottom.
In the sense that when the government transfers you $1, you actually earn more rather than earning less.
So we're not only ending the leak in the bucket at the bottom of the income distribution, we're actually improving, offsetting some of the 40% leak that comes from the top by getting people to work harder under this EITC.
We're transferring in a way which actually improves efficiency.
So basically, EITC we can think of as sort of a patch of the bucket.
It's a patch to the bucket.
You're actually helping address leaks in the bucket by doing this.
Now you might say that's great.
That means that we have solved our problems.
We'll just get money to the poor through the EITC and there's no leak to the bucket at the bottom.
But the problem is, once again it's not that simple because some people truly can't work.
Some people truly can't work.
So you need to have something for those that truly can't work.
So an ideal system would be one where if you truly can't work, we give you money.
If you can work, you get the EITC.
Once again, the problem being we have to decide who it is that truly can't work and who can work.
And so basically, how leaky the bucket is at the bottom will depend on how good a job we can do at telling who the people are who can work, who the people are who can't work.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We began our discussion by talking about how society would choose across different income distributions, which involve the trade-off between preferences for more equal distribution of income and the efficiency cost of transferring income through both taxation and welfare programs.
We then moved on to talk about what the government does in practice in terms of taxation and income redistribution.
What we're going to do today is actually recognize that, in fact, most of government spending, most of what we raise taxes for is not pure income redistribution.
We talked about things like the TANF program, which gives money to single mothers, or the SSI program, which gives money to disabled people.
That's not most of what the government raises money for.
In fact, 95% of the transfers that the government makes are not income based transfers.
They're transfers of what we call social insurance.
So most of what the government does today is not take from the rich and give to the poor.
Most of what the government does today is take from everyone to provide insurance for everyone.
Now basically, we talked in the uncertainty lecture about why people would value insurance.
So essentially we think about people choosing across states of nature like they're choosing across goods.
Then they will want to optimize across those states, hit by a car versus not hit by a car, the way they optimize across goods like CDs and movies.
And we said the way they optimize across states is by buying insurance.
That's the way you essentially transfer resources from the state where something goes bad to the state where something goes-- I'm sorry, from the state where something goes good to the state where it goes bad.
The way you effect that transfer, effectively, is by paying insurers when things go good and getting from insurers when things go bad.
That's insurance, and insurance is an enormous share of the US economy.
Currently, US as a whole spends about $2 trillion a year on insuring ourselves against adverse risks of various types.
There's health insurance, which is money we pay up front to insurers, and they cover our medical costs should we get sick or become injured.
There's disability insurance, which is money that you pay in the form of taxation, and the government uses that money to give you a benefit should you become disabled.
There's unemployment insurance.
We know about that.
Unemployment insurance is money that your employer pays in taxes, and in return, should you lose your job due to a layoff you receive income when you're laid off.
That's certainly a topic of much conversation these days, et cetera.
So there are a number of social insurance programs that are out there to help individuals with the adverse risks they face.
The big question we have to ask in starting this lecture is, well, why does social insurance exist?
After all, there's actually an enormous robust private insurance market.
There's health insurance, there's life insurance, there's casualty insurance.
The private market insures us for lots of risks.
The question is why does the government come in and also provide us this social insurance?
So we individuals will demand insurance for risk.
We know private insurance exists for things like health insurance, life insurance, fire insurance, auto insurance.
We know it exists, and that adds up to about $2 trillion of private spending.
Yet at the same time the government also provides on the order of another trillion dollars of social insurance spending every year for things like health insurance, and disability, and other things.
So the issue is why does the government need to do that?
Why does the government need to roll in and provide insurance?
We know individuals want it.
We know a private market exists that provides it.
What's the market failure that justifies a government role here?
Think about the role of the government in the economy-- and this is a very small part of this course-- but the focus of the other course I teach, 14.41.
The basic question is in this course we've largely learned the governments serves to screw up the market if the market's working well.
The only time the government can help is if there's a market failure, if there's something wrong with the market.
Well in this case, what could the market failure be?
And the market failure, in the context of insurance, is the problem of asymmetric information.
Asymmetric information, information that is held differentially between parties on either side of a transaction.
And that when there's asymmetric information, when different parties know different amounts about a transaction, that can lead to market failures.
So the classic example of this comes from the Nobel Prize winning economist George Akerlof.
It's called the lemons problem.
The lemons problem is, basically, he did the example of the used car market.
Imagine you've got a used car market, which is a set of individuals who want to sell their cars and a set of individuals who want to buy those cars.
And let's start with the full information benchmark.
Let's imagine there was perfect information about the exact quality of every car.
Imagine there was some test you could do which would perfectly tell you exactly how long the car is going to last, how well it's going to run, et cetera.
In that world there will be a set of transactions.
Some individuals will want to sell their cars because they want to buy up, other individual will be happy to pay that price, there will be a set of transactions.
Now imagine we go to the real world, especially a real world when Akerlof wrote his article in the '70s when, basically, if you went to buy a used car you had to look at it and decide does it look OK.
There's clearly asymmetric information.
Clearly the seller of the car knows much more about its underlying qualities than the buyer of the car.
And in particular, the buyer should be suspicious saying, wait a second, why are you selling this car?
If it's such a good car why are you selling it?
And the very fact that I see you selling it makes me suspicious that something's wrong with it.
Given that I'm suspicious something's wrong with it I'm going to be willing to pay less for it than I would if I knew for sure it was OK.
The fact that I don't have full information as a buyer means that I'm going to pay less for it than I would if I had full information.
Well if I'm going to pay less for it but the seller has a certain amount he's willing to sell it at, the seller will say, well, I'm not going to sell it to you for less.
I have a certain supply curve at which I want to sell.
If I can get x for the car I'll sell it, but if you're not going to pay me x I'm not going to sell it.
So the sale doesn't happen.
So what you have is a car where in a full information world a sale would happen, both parties would be happy with that sale.
But in an asymmetric information world, since the first party, since the buyer's suspicious of the quality and is therefore willing to pay less, and the seller won't accept less the sale doesn't happen.
And that's a market failure.
Relative to the 14.01 world we started with, where everybody knows everything, where there's full information, the market has failed because sales which would make both parties better off do not occur due to asymmetric information.
So that is an example of the market failure caused by asymmetric information.
Now let's flip this around and talk about insurance.
In insurance it's the opposite case.
In insurance, I, the person demanding the insurance, know more about me than the insurers who's selling me insurance.
I know whether I'm someone who gets sick a lot, whether I like skydiving, whether I engage in other risky behaviors that may have health problems for me, whether I am someone who smokes in bed so I'm likely to catch my house on fire so that I really want house insurance because I'm going to catch my house on fire, et cetera.
I know all that stuff about me, the insurer doesn't.
So the insurer, as a result, might be unwilling to sell me insurance at a fair price.
So to put this in numbers, let's consider an example.
Imagine that you are starting up a health insurance program for MIT graduates.
You want to say look, MIT graduates are a pretty healthy lot, and so I'm going to start up a health insurance program for MIT graduates.
I'm going to make money since they're basically pretty healthy, but they're risk averse so they will buy the insurance.
So imagine the facts are the following.
Imagine that for every for every 100 MIT graduates, over the next year, 90 of them will spend nothing on medical care, and 10 of them will spend $1,000.
But we don't know which ones, there's uncertainty.
So I'm left with this 90% chance of zero, 10% chance of $1,000.
So everybody graduating, since we're a pretty healthy crop, 90% of you will spend nothing but 10% of you will spend $1,000, and you don't know which.
And you're going to start an insurance.
You could say look, I'm going to start an insurance.
What I'm going to do is I'm going to say is what's the expected payout that I'm going to have as an insurer over the next year?
Well, the expected payout over the next year is $10,000 total, right?
And let's say there's 100 students you are going to sell to.
The expected payout the next year is $10,000 total.
10 students will cost you $1,000 nine students will cost you nothing.
Expected payout next year is $10,000, or per student you're selling to the expected payout is $100.
Right?
Have I got that right?
Yeah.
$100 per student you're selling to.
So if you sell to 100 students at $100 you will break even.
You will collect $10,000 at an expectation you'll pay out $10,000.
So if you roll in and offer insurance for $100 and you sell to all these students then you'll break even.
Let's say you then said fine, I'm going to offer it for $110.
Well, as long as people are somewhat risk averse, in our standard model they'd still like insurance for $110.
They'd still be willing to pay a little risk premium-- remember about risk premiums to get health insurance.
So you still sell it, let's say, and you make $1,000.
Then you're going to sell to 100 students at 10 extra dollars to make $1,000 profit.
So this is a pretty good product.
I'll make a $1,000 profit on this.
Now, what's in fact the problem with this analysis?
The problem is that what if some of the healthy guys say, look-- what if people know whether they're healthy or sick?
What if it's not uncertain?
Or what if people have a good inkling?
What if, in fact, half of the 90 people who will spend zero, what if 50 of them actually know they're going to spend zero?
What if, in fact, the story is different?
It's not totally uncertain, but 50 people know they're going to spend zero.
They're really healthy and they don't like the doctor, they know they're going to spend zero.
Whereas the other 50, well, 40 of them might spend zero and 10 might spend $1,000.
Well in that case, the 50 that knows they're going to spend zero won't buy the health insurance.
They'll be like why should I buy it?
I know I'm going to spend zero, why should I pay $100 or $110 dollars for health insurance?
So then, how much will you sell?
And let's say you charge $110 to make a profit.
You'll sell to 40 guys, 40 healthy guys, and 10 sick guys.
So 50 guys times $110, so you'll make $5,500.
What will you pay out?
You'll pay out $10,000 because the 10 sick guys are definitely buying, right?
The 10 guys who'll end up getting sick are definitely buying along with 40 healthy guys.
So you will collect $5,500 but you'll pay out $10,000.
You'll lose money.
Why will you lose money?
You'll lose money because you'll be collecting as if you're dealing with a full probability distribution but you're only going to see a truncated probability distribution.
You're only going to see the distribution for the people who have some chance of getting sick.
The very healthy people drop out.
As a result you lose money.
This is the insurer problem.
If the healthiest people don't buy insurance then they can't price it fairly.
How would you overcome this?
Well, you would have to raise your price a lot, right?
So let's say you say fine, I'm going to overcome this.
If I raise my price to $200 or to $210 then I'd make money again.
If I raise my price $210 that's 50 people buying at $210.
Then I get $11,000 in premiums and I don't have to spend $10,000, and I make money again.
What's the problem with that solution?
Yeah?
People will probably be dropping out because you just--
Some of these 40 will say, well, I was happy to pay $100 but I'm not happy to pay $200.
The chances are really low I'm going to get sick.
So this goes down.
So you collect less money but you still pay out to the sick people.
And you never win.
You can never win.
As you raise the price the healthier and healthier people drop out until you just simply can't make money.
Yeah?
Is there any situation where if you have a large enough number of people that you're supplying to you can set the price at a certain level such that you're guaranteed that there are at least going to be a large number of people left after dropping out so that you won't lose money?
Yes, and what's that going to depend on?
So one extreme example is what I've described here, where as you raise the price more healthy drop out and so you can never make money.
Under what conditions will that be a particular problem or not be a problem?
Well there's two key factors.
First of all, it's going to be how imperfect the information is.
For example, if people know for sure whether they're going to be in the zero bucket or the $1,000 bucket then by definition you can't make money.
If you know for sure, with 100% certainty, then why would you ever buy insurance unless you're going to be in the $1,000 bucket?
And why would you ever pay more than $1,000 for it?
So there's no way to make money if people know for sure.
So the first thing is where nobody has any idea at all.
Then this is going to be less of a problem because healthy people won't drop out because nobody knows who's healthy.
So the first issue is going to be the asymmetry of your information.
The second factors is going to be what?
The level of risk aversion, because if people are very risk averse then they may be willing to pay a lot for insurance even if the odds are low it's going to hurt them.
So in this case, once again if people know for sure, risk aversion doesn't matter.
But if people don't know for sure, if there's some uncertainty, then you may get healthy people buying health insurance because they're risk averse.
They may buy it anyway, and so then you may not have a market failure.
So basically, the more asymmetry of information there is the more likely you're going to lose money?
What happens if you're going to lose money?
You're not going to start your insurance company and MIT students will end up uninsured.
That's a market failure.
The market for insurance will fail, the market for insurance will fail because you will end up not being able to make money.
You're not going to be able to make money because only the sickest people will buy, and you lose by definition.
That's the problem that we have with private insurance.
If the healthiest people stay out, then the insurer can't make money.
The insurers need a distribution of healthy and sick to make money, they can't make money if the healthy stay out.
So what can we do about this?
What can we do about this?
Well there's two classes of solutions we can have.
The first is we can subsidize.
Imagine if we said, we MIT, will roll in and provide the insurance for free to everyone.
And then we'll pay the insurer what they want.
So MIT will come in, they'll absorb the cost of the insurer and give it to MIT students for free.
Well, in that case all the MIT students would take it, it would be free, and the insurer could charge $100 or $110 based on the negotiation with MIT.
And the insurer will provide the insurance because everyone would take it.
And MIT would just pay the cost.
So the market failure would be solved, but of course at a cost of MIT having to pay $10,000 or $11,000, or whatever it's going to be.
So the market failure would be solved by subsidizing the healthy people to come in.
So essentially, if you pay off the best risks to come in the market you solve the market failure because then the insurer knows they're getting a good distribution of risks.
What's the other thing you can do?
Well the other thing you can do is the centerpiece of the current health care reform bill, which I'll talk about next time, which is you can mandate that everyone buy health insurance.
If there was a law that every MIT student had to have health insurance when they graduate then, once again, the insurer would know what the distribution of risks are and they could sell insurance at a fair price.
They wouldn't have to worry if the healthy drop out because the healthy couldn't, it would be illegal.
Now, these two solutions-- so you can subsidize or you can mandate.
Now these solutions-- Yeah?
Weren't a lot of health insurance companies also looking at medical records of people before deciding-- were they unequally pricing health care based on someone's history?
And I know that caused a lot of controversy say over pre-existing conditions
Yeah.
Let me talk about that next time.
What you're asking is isn't it in fact possible asymmetric information could flip?
Isn't it possible insurers could now know so much about us from public records that they could start to have more information about us than we even have about ourselves?
And then they could decide to get rid of us because they know we're a bad risk even if we don't know.
So could we be soon living in a world of reverse asymmetric information?
And that's possible, and I'll talk about that next time.
But for now let's assume we live in a world where you know more than the insurer.
So you can subsidize the guys, you can mandate it.
Now each of these have pros and cons.
If you subsidize them then the MIT students are happy.
They get insurance for free, but MIT is out $10,000.
Or if it's the government, the government's out a lot of money.
How does the government raise that money?
It taxes people, and that has efficiency costs.
So one thing we could do is make health insurance free for everyone.
That's what Canada does.
Health insurance is free for everyone.
And the problem is to do that in the US would cost something like a couple of trillion dollars.
That's a lot of money to raise in taxes.
And that would cause inefficiency from raising that money in taxes.
On the other hand the mandate doesn't cost the government anything.
But it makes healthy people very upset because suddenly you're an MIT student, you know for sure you're not going to spend zero but the government mandates you have to pay that $100 premium.
Well now you're upset.
So basically there's no good answer here to solve this asymmetric information problem.
Either we have to spend a lot of money and then people aren't upset but taxpayers are.
Insurers aren't but taxpayers are.
Or we have to mandate people to buy, in which case the healthy will be upset they're being forced to buy something they don't want.
And that's the difficult position we face ourselves in with social insurance programs.
But the bottom line is what we have is this failure of private insurance markets due to asymmetric information and the need for the government to come in and take one or both of these routes to fix the problem.
If the government doesn't then you don't get people getting insurance that they'd like to have.
You get a market failure.
You get a market failure in people not having insurance that they would value.
And that's why we have governments come in and provide social insurance.
Now, as I said, the government comes in and provide many types of social insurance of many types.
There's medical social insurance, and here the government does that through the form of what we call two programs, Medicare and Medicaid.
Medicare is universal, heavily subsidized health insurance for the elderly.
Medicaid is like a welfare program.
It's means tested, categorical, free health insurance for the poor.
So Medicare's for the elderly, Medicaid's for the poor.
In neither case is there a mandate, in both cases the government, essentially, is making it free.
Essentially it's gone the subsidy route, as a result at an enormous cost to the US taxpayer.
Medicare is now about a $500 billion a year program.
Medicaid is approaching a $400 billion a year program.
It's almost a trillion dollars US taxpayers are paying to get people to buy health insurance.
So by solving it with this route these are incredibly, especially Medicare is an incredibly politically popular program because why not?
People are getting it for free.
But the cost is an enormous budgetary pressure on the US government.
That's for health.
We also have disability insurance, DI.
That's for people who have a career-ending injury.
If you fall off your motorcycle and get paralyzed and can't work anymore you can get disability insurance, which is money which compensates you for being unable to work having a career-ending injury.
So this is another form of social insurance.
We have workers' compensation, which is insurance that you have to insure yourself against injuries on the job.
So if you slip and fall on the job and have to miss a couple weeks of work this covers your medical costs and your lost wages.
And we have unemployment insurance, which as I mentioned, is insurance for people who lose their jobs due to economic circumstances.
Unemployment insurance is not for people who get fired because they're crappy workers.
It's for people who lose their job because they're laid off for economic circumstances.
They get some replacement of wages under that unemployment insurance programs.
These are the kind of social insurance programs that we have in the US.
And the arguments for them is basically more complicated version of the arguments I just made over there.
The argument for all these programs is that private insurance markets for things like unemployment or injury will not function well due to asymmetric information so the government needs to come in and provide that insurance.
But at the same time all of these insurance programs have a cost.
Not just the cost of raising the money but the cost of what we call moral hazard, which is that when you insure people against adverse risk you encourage adverse behavior.
When you insure people against adverse risk you encourage adverse behavior.
And the problem here is another form of asymmetric information.
So one problem of asymmetric information is since insurers don't know enough about you they'll under insure you.
So it's a market failure.
This is a flip side market failure, which is since insurers can't perfectly be sure about what's wrong with you, you will be encouraged to misrepresent your state in order to collect money.
So a classic example here is workers' compensation insurance.
Workers' compensation insurance is about a $50 billion a year program that provides money to people who are injured on the job.
The problem is injured on the job is very hard to verify.
Most of the injuries are not losing an arm.
Most of the injuries are straining your back.
As anyone with back problems knows it's pretty darn hard to verify if your back's hurt or not.
My first job in my first summer after MIT was I was a filing clerk at a doctor's office.
The doctor did workers' comp-- this is a real story.
Doctors do workers' comp claims.
It was one doctor with a staff of five secretaries and an enormous filing staff.
And the reason was the doctor saw a patient every three minutes all day.
They'd come in, he'd say, yeah, you're hurt, here's your chit for workers' comp, off you go.
All day.
So basically, because it's incredibly hard to verify, this doctor was just-- basically his role was saying yes to people so they could get out of work.
This is the problem we have with a system-- this is another information asymmetry, which is now you, the insured, know more about what's wrong with you than the insurer does.
But it's a different form of information asymmetry.
Now it's that you may actually not be as injured as you claim or need insurance as you claim.
As a result you over claim insurance, and that costs the insurance industry money.
Now, moral hazard, theoretically, is unambiguous.
But obviously it's going to vary across situations.
There's not going to be a whole lot of moral hazard in programs that insure you against blindness.
And in that case that's because it's pretty easy to observe if someone's really blind.
There's not a whole lot of information asymmetry.
There's going to be a lot of moral hazard in programs that insure you against having a bad back because that's harder to observe.
And in fact we have lots of evidence that moral hazard's a big problem with social insurance programs.
So one type of evidence is sort of fun stories.
So there's stories about, for example, the Massachusetts prison guard who got about $25,000 in workers' comp claims because he claimed he hurt his back at work until people went online in order to see that the same guy was running a karate school.
And there was videos of him doing karate moves and stuff online running his karate school.
Or the other guy, who unfortunately had good instincts, he was on workers' comp because he was injured.
But then there was footage of him at 9/11 flying down and carrying buckets back and forth and helping put out the fire, which was a nice thing for him to do, but clearly illustrated that he was not too injured to be able to carry out his job.
So basically, there's lots of observational evidence.
But there's also very clear statistical evidence, which is if moral hazard doesn't exist then when the program gets more generous I shouldn't be more injured, right?
My injury should be something that happens or doesn't happen.
But if more generous programs lead to more injuries, or people staying out of work longer, or people using more health care then that suggests moral hazard.
The generosity, whether you're sick or not, shouldn't be affected by the generosity of health insurance.
But if you suddenly see more generous health insurance leads to more people being sick that suggests moral hazard.
And there's hundreds of studies of this type showing, in all these realms, enormous moral hazard.
Showing, for example, for workers' comp, that when you make workers' comp more generous people report more injuries on the job.
Here's my favorite fact about that.
Someone did a study of type of injury reported to workers' comp by day of the week.
And what they found was on Mondays compared to other days of the week there was a very high share of sprains and strains and a low share of lacerations, of cuts.
So every other day of the week, say it was 50% cuts and 50% strains.
On Monday it was 75% strains and 25% cuts.
Why is that?
Well, that's because guys got hurt over the weekend playing softball and they came in on Monday and they said they got hurt on the job.
Now you can't do that if you cut yourself on the weekend in your shop because it's scabbed over by Monday.
But if you hurt your back on the weekend you can drag yourself over and say, ooh, I hurt my back at work.
So that's exactly the kind of evidence that's consistent with moral hazard.
And we see that everywhere.
There's huge amounts of moral hazard.
Moral hazard, for example, in health insurance the best estimates are that the amount of extra money that we spend on society on health care because of moral hazard on the order of $500 billion a year in terms of extra care that's used because people are insured.
That does nothing to improve their health, they just use it because they're insured.
It's on the order of $500 billion a year.
This is an enormous problem.
So here we have what we call the classic social insurance trade off.
On the one hand, people value insurance.
Insurance is very valuable to help you in the state when you get in an accident or are injured in some way.
On the other hand, giving people insurance against those bad outcomes will encourage bad behavior.
And that bad behavior has two forms of costs.
There's two costs to that bad behavior.
The first cost is the tax costs and the deadweight loss of taxation, which is if there's $500 billion extra in medical care used because people are insured then that means that's $500 billion of taxes we have to raise, or $500 billion in resource costs to society.
Well, not all that's government.
But a couple hundred billion in taxes we have to raise, that causes deadweight loss.
The more important cost is really the overall efficiency loss from people using resources inefficiently, from people not behaving optimally because of the incentives of social insurance.
So let's go back and think about the case of workers' comp, and let's think about our standard leisure consumption trade off.
We have our standard leisure consumption trade off where you have leisure on this axis, consumption on this axis, some budget constraint, and people choose how much leisure to take and how much consumption to take.
Now the slope of this budget constraint is minus your wage.
That's the price of leisure.
So the price of sitting at home is the wage you could have earned out in the market.
What this means is you will sit at home until the value you get from sitting at home falls below the value you get out in the market.
So in other words, you wouldn't work 24 hours a day because that 24th hour, the sleep would be so valuable that it would clearly exceed the wage you could earn.
So when you decide how hard to work, when an individual decides how-- presuming we can freely choose our hours-- we decide how hard to work, we basically trade off how much we value not working versus how much we'd earn if we do work.
If you're in a high wage you might work more hours, if you're in a low wage you'll earn less because if you're in a low wage you might as well sit at home and watch Oprah.
If you earn a high wage you're not going to watch Oprah you're going to go earn the money.
And that's how we make our labor supply decisions.
We set the wage equal to the marginal value of leisure.
That's how we make our labor decisions, is we say if the marginal value of the next hour of leisure exceeds my wage I'll stay home.
If the marginal value of the next hour of leisure is below my wage I'll going to work.
And that's how we make our labor supply decisions.
What does workers' comp do to this, or UI or other programs like that?
What that says is wait a second, now if I sit at home I don't just get the marginal value of my leisure, I also get a check from the government.
So now if I sit at home I get the marginal value of my leisure plus a workers' comp check.
Whereas if I work I still just get my wage.
So what's that going to do to the amount of hours I'm going to want to work?
It's going to reduce them because now it's makes sitting at home more attractive because now I'm going to work only until my wage exceeds my marginal value of leisure plus workers' comp.
So if workers' comp is big enough and my wage isn't that high, in the limit, if workers comp replaced all my wages I would never work.
If workers' comp had what we call a 100% replacement rate, replacement rate, if it replaced 100% of my wages I would never work.
Why?
Because by definition this side would always be less than this side, so I'd never work.
And that's the efficiency loss of these social insurance programs.
That people are not engaging in productive trades.
Society just cares about this part.
Society just cares about your wage versus your marginal value of leisure.
So if you work less than that because of this part then that's a distortion, that's an efficiency cost to society.
You are sitting at home when you should be making a productive trade of your labor for output goods.
And that's the efficiency cost of social insurance.
So what do we do?
We want social insurance, on the one hand, because private insurance markets fail.
We don't want social insurance, on the other hand, because people are going to sit at home.
What do we do?
Well the answer is we compromise.
We have social insurance but don't make it that generous.
So for example, the unemployment insurance program, that's insurance that is valuable because there's no way to get private insurance against losing your job.
I'd never want to sell that product.
We know who'd take that, right?
On the other hand, we know there's moral hazard.
We know that people sit at home longer rather looking for a job because they get UI checks.
There's great evidence of that.
For example, if you look at how long people sit at home, everyone sits at home and then suddenly gets a job the week UI runs out.
That's an exaggeration, but roughly speaking, there's a huge amount of people sitting at home collecting the UI checks.
What do we do?
Well the UI replacement rate is around 50%.
We say, look, we know you're going to want to sit home so we're going to make your life uncomfortable.
We'll only give you half of what you'd earn at work, but we don't want you to starve so we'll give you half of what you earn at work.
And so the replacement rate is chosen to trade off the benefits of insuring people against these bad events against the costs of distorting their behavior through moral hazard.
And that's the social insurance trade off.
That's how we set up these social insurance programs.
They sort of embody that trade off.
Questions about that?
Now, so what I want to do in the remaining time is I want to talk about the nation's largest single social insurance program, one you've heard about a lot, which is Social Security.
We're going to talk about Social Security.
I'm going to come back next lecture and we'll go over a bit what's going to be on the final in the last 15 minutes next time.
But the first part of the lecture I want to talk about health insurance, which is our single largest government expenditure.
But the single biggest government program right now, social insurance program, it remains Social Security.
Medicare is going to pass it in about two years, but for now it's still Social Security.
What is Social Security?
Social Security is insurance against the income loss from retirement.
We know that when you have to retire you're going to suffer a big income loss.
What the government does is insures you against that income loss by providing you Social Security.
And here's how it works.
How many of you have seen a pay stub and you've seen the word FICA on it, and then there's a charge that's coming out of your pay?
FICA is the Federal Insurance Contribution Amount.
That's the money you're paying in to support the social insurance program.
You pay that in, and that's currently 12.4% of wages, well 6.2% on you and 6.2% on your employer.
You each pay 6.2% of your wages.
That goes into a trust fund and that money then gets paid out to retirees, and hopefully paid out to you when you retire.
So the way Social Security works is you pay in money when you're young, that money gets paid out to you when you retire.
Not that same money because your money going in now is going to today's retirees.
Future workers will pay in and that money will go to you.
And the Social Security replacement rate is around 50%.
So to try to deal with this problem the Social Security replacement rate's around 50%.
So what's the benefit of Social Security?
Well, clearly the benefit is people don't want to starve when they retire.
We'd like insurance against the income loss of when we retire.
Now, our own savings can do that to some extent but it's not perfect because you don't know exactly how much to save, you don't know when you retire, you don't know how long you'll live.
So the government social insurance program helps with that.
But there's also a moral hazard cost, which is if you insure people against retirement you encourage retirement, because now we're saying is if you work you get your wage but if you retire you get your Social Security.
So now, you're eligible for Social Security at age 62.
So at age 62 you're suddenly eligible for about half your wage in Social Security.
So now your decision is, gee, do I work?
Well that's contrasted against my marginal value of leisure plus half my wage.
So unless the marginal value of leisure is less than half my wage I'm going to quit.
So what Social Security is doing is distorting your decision and causing you to retire early.
That's the moral hazard part.
Now in the US we recognize this, and we do sort of a neat trick.
What we do in US is we have something called an actuarial adjustment to your Social Security benefits.
What that means is the following, you're 62 and you say, wait a second, I can retire now, and if I retire now I'm going to get half my wage.
So why not retire now?
We say, wait a second, we've got a deal for you.
If you work one more year and retire next year then we'll give you 7% more every year for the rest of your life.
Your check will be 7% higher for the rest your life.
If you work two more years it will be 14% higher, 3 more years 21% higher, et cetera.
We recognize the moral hazard, and we raise the check the longer you work to make up for it.
So the US recognizes that, and as a result there's not a huge moral hazard from Social Security in the US.
It's pretty modest.
In Europe this is not the case.
Let's take the country-- let me back up.
So what the actuarial adjustment does is says we recognize that Social Security is essentially taxing your work.
To offset that we're going to subsidize you to work more by raising your Social Security check the more you work.
In Europe they're not as smart as us.
They don't recognize this, or if they do they're not smart enough to reflect it.
They don't have an actuarial adjustment.
So here's how the Social Security system works in the Netherlands, for example.
When you reach 55 in the Netherlands your choice is you can work or you can stop working and get a Social Security check which is 90% of your wage.
So literally, your choice is work or sit at home and get 90% of your wage with no actuarial adjustment.
So unless the value of your leisure is less than 10% of your wage you can just sit at home.
But wait, that's not all.
This is a very generous program, right?
How do they pay for that?
They pay for it by taxing workers.
So if you work you get your wage but then you pay a 40% payroll tax.
So actually, you can work and get 60% of your wage or sit at home and get 90% of your wage.
Guess what?
No one works after age 55 in the Netherlands.
Literally.
Now people will do some underground work that's not taxed.
They'll paint houses or things which they can do on the underground economy.
But literally, nobody works after 55 in the Netherlands.
Economics works.
When you make the incentives that blatant economics works.
And that's the moral hazard.
That's an example of not respecting the social insurance trade off, erring on the side of insuring people over respecting the problems of moral hazard.
And the answer is part of the major problem that they have in these countries, part of the major deficit, prolonged deficit problem they face is that older people don't work.
Now these countries have started to recognize this, and they're starting to try to deal with this.
In fact, the country of France, you may have noticed, recently tried to deal with this.
They have proposed raising their retirement age, which is age 60, beyond which, essentially, no one works in France, to age 62.
What happened?
You may have heard, massive strikes, enormous political strife because people don't understand economics.
People don't understand that, gee, yes that'll be a bummer if you work a couple years longer, but that's what's going to make our economy fiscally sound and more efficient.
And in the long run we all benefit from that.
So the government's, by the way, not proposing that today's retirees are hurt, saying starting in 20 years you have to work two years longer.
So overall everyone's going to benefit, but the bottom line is people are too short-sighted to understand the economics, and that leads to the rioting and things we saw in France.
This stuff is hard to a normal person.
You guys are smart and you spent a semester learning it.
If you take a regular person in the street they just don't understand this trade off, and that's the problem.
That's the problem in trying to deal with these moral hazard issues by making social insurance less generous.
Now, let me talk about one other issue with Social Security because it's been in the news and will be in the news, which is the fact that Social Security is running out of money.
Now, Social Security is running out of money because here's a dirty secret about Social Security you may not know.
How many of you have ever heard of a Ponzi scheme?
Anyone watch Boardwalk Empire?
They're doing that now, Boardwalk Empire.
A Ponzi scheme was named after Charles Ponzi, a Boston investor, and here's how the Ponzi scheme worked.
I teach two classes.
I teach now and I teach again at 2:30.
I teach 14.01 now, 14.41 at 2:30.
Let's say I walked in here this morning and I said, tell you what guys, you each give me $1 and on Wednesday I'll give you $2.
Now you're thinking, well, that's bullshit, but you know what, I want a good grade so I'll give him the dollar.
So you give me the dollar.
I then go to my 2:30 1441 class and say, hey guys, you each give me $3 and on Wednesday I'll give you $5.
They are like same thing, whatever, but they're higher classmen than you, they have more money, they give me the money.
I take their $3, I pocket $1.
I come back on Wednesday and give you each $2.
You're like, holy shit, that worked.
I'm like, I've got a better deal for you.
You guys give me $6 and next time I see you in class-- semester's ending, but you get the point-- I'll give you $10.
You're like, great, this guy's trustworthy, we'll do that.
You give me the $6.
I pocket $1, I go back to my 14.41 and say here's the $5 I owe you.
They say, wow, that's great.
I say, so I've got a better deal for you.
You give me $11, and so on and so on.
And as long as you're both willing to play I can continue to pocket the money and make money.
This is also called the pyramid scheme.
You might have heard a guy named Bernie Madoff.
Never invest your money with a guy who's last name is Madoff.
Bernie Madoff, for 20 years ran this kind of Ponzi scheme.
What he'd do is he'd find investors, he'd take money from them and promise them a return, he'd then find more investors, take the money from the new investors and pay off the old investors, and keep going.
There was nothing real behind what he was doing.
He made it work for 20 years until someone didn't want to play anymore.
What if we did this, and it was the last class and you guys said, well, see you Professor Gruber.
I'm like, wait a second, if you give me $20, and you're out the door.
Now I got to go to my next class and I don't have any money for them.
Then the whole thing collapses And it collapses because a Ponzi scheme is not a real investment.
The asset behind a Ponzi scheme is not a bank account.
The asset behind a Ponzi scheme is trust.
And basically, as long as someone's no longer trusting or can no longer pay in, it goes away.
Now, here's a little secret.
Our Social Security system is a Ponzi scheme.
The nation's largest social insurance program is a Ponzi scheme.
How is that so?
Well because you guys are paying in now, and those checks are going to today's elders.
There's no guarantee that 40 years from now kids will pay in to support you.
No guarantee at all.
You can pay in your whole life, and then people could vote out Social Security when you're retired and you're screwed.
It's a Ponzi scheme.
There's nothing behind this except trust in the government's programs.
The only asset behind this is the trust the government would never screw the voter, the elderly voters, by ending the program.
As long as the government is willing to force young people to pay checks to support older people the program can never end.
But if the government ever is unwilling to do that then a whole generation is going to suffer from the Ponzi scheme.
Now that's pretty unlikely because elderly voters are pretty powerful.
It seems unlikely that Ponzi scheme will ever collapse.
That's why we don't think of it in those terms.
But it really is just that.
It's just a Ponzi scheme, in this case supported by trust in the government.
The problem is this Ponzi scheme works well so long as more people are paying in than are collecting money.
So right now we have, in the US, we have about 10 workers for every elderly person, or about eight workers for every elderly person.
So we can collect a lot of money to pay off the elderly.
The problem is that number's diminishing rapidly because of the aging of the baby boomers.
There's a huge bulge of individuals who are about to become elderly and collect their Social Security.
By the year 2025 that 8:1 will ratio be more like 4:1, and we're suddenly going to be short on money.
Here's the way I think about it most scarily.
How many of you guys have grandparents in Florida?
OK, you've got a few grandparents in Florida.
The US in 2025, the US as a whole, will be older than Florida is today.
Because if you think of Florida as full of old people, that's the US in 20 years, in 15 years.
We're moving to a very aging society, and that's going to put enormous pressure on this Ponzi scheme.
And that's why, in about 30 years or in about 25 years, Social Security goes bankrupt.
We actually run out of money unless we fix the program.
So currently we have a situation where we are projected for that program to go bankrupt-- it's actually in about 30 years-- the program to go bankrupt because we have to pay out so much to this large group of elderly.
And that's a problem the government needs to deal with.
And it's going to have to deal with it in one of two ways.
It's going to either have to raise taxes.
We currently pay about 12.4% in payroll.
To solve this problem forever that tax rate would have to go up by about three and a half to 4%.
So we'd have to pay the three and a half to 4% more of our wages to solve this problem.
Or we're going to have to cut benefits, do things like saying to people they can't get their benefits at 62, that goes up to 64.
And you can see US style rioting rather than French style rioting.
That's the difficult problem the government faces going forward is that something has to give.
We either have to raise taxes to support this program or we have to cut the benefits that elderly people get.
Yeah?
This might just postpone the problem, but would it be possible for the government to taper off the benefits?
So for instance, at 62, when you're eligible, you could get let's say 50% of what you'd normally would be getting, and that amount would slowly increase as you get older.
So when you're making your labor decisions you would work, but maybe only half or quarter time what you normally would work and that might--
Certainly there are compromises along the way but that still goes in the category of benefit cuts.
You're just talking about a different way to do a benefits cut.
There's lots of very subtle and interesting ways-- and believe me, when the time comes I'm not just going to say we're cutting everybody's benefit 10%.
It will be something much more subtle along the lines you're describing, sufficiently complicated that people don't understand it so that they can try and slip it by people.
If they're going to cut benefits it's going to be that complicated, and that's the trade off we're going to face.
All right.
OK.
So next class we'll talk about health insurance and we'll do a final session.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
So to review, what we've been doing in the last two lectures, we talked about equity versus efficiency, and the equity efficiency trade off.
We then talked about redistribution and the leaky bucket, and basically how both taxation and distortions and welfare programs lead to leaks in the bucket.
And we talked about we do in the US to try to transfer income.
The last lecture, we focused on the fact that most of the US government transfers are not about just the rich to the poor, they're about fixing problems with our insurance markets in the form of social insurance.
And we ended last time by talking about the nation's largest social insurance program, social security.
What I want to talk about today is a particularly timely topic, which is the biggest single issue in social insurance, indeed the biggest single fiscal issue facing the US government over your lifetimes, which is health care.
Health care in the US today consumes about 17% of our nation's income, so that one in every $6 goes to health care.
By 2075, so when you guys are elders, when I'm no longer with you and you guys are elders, health care will consume about 40% of our nation's product, so $4 in every $10 will go to health care.
By 100 years later, when your grandkids are elders, health will consume 100% of our nation's product.
That is literally if nothing's done, every single dollar we earn will go to health care.
Now, obviously that is not sustainable.
That's not a legitimate outcome to envision.
So something must be done, and that has led to an enormous discussion about this area.
Another way to think about this is, if you think about the US, we haven't talked about budgetary issues in this course, think about US government having a budget, money it collects through taxes and money it spends through revenues.
We all know the US government has a big budget deficit right now, about $1.3 trillion.
Another way to think about that is if you look at the US government and all the promises it's made to pay for health care alone, forget everything else, just for health care, we've promised to pay, above and beyond any taxes we're planning to collect, $100 trillion in health care promises to the Medicare and Medicaid program, above and beyond any taxes we'll collect.
So if we do nothing, we're facing a $100 trillion long term deficit in health care.
Now, a year ago, that would have sounded more stunning because a year ago, we didn't use to talk in trillions.
We only talked in billions.
Now, we're all talking trillions, so it doesn't sound that impressive, but believe me that's a big number.
So basically, we've got enormous problem facing our country with health care, both in terms of the country's devotion of resources, and in terms of the government's devotion of its budget.
Now, you might say, well, the government should just get out of this activity, but that goes back to the last lecture, which is why is government in this activity?
Because there's a market failure for health insurance.
We talked last time about adverse selection, and the problem that, if the government doesn't get involved, health insurance markets might fail, and in practice we've seen that in the United States.
We've see a failure of health insurance markets.
Now, most people get their health insurance through big companies.
So I get mine through MIT.
You get yours through MIT or through your parents and the big companies where they work.
And in those markets, health insurance works well.
And health insurance works well through a simple statistical fact many of us learned about in high school, which is the Law of Large Numbers.
The Law of Large Numbers simply says that for most distributions we know, certainly normal distribution, as the sample gets large enough, the mean is predictable, and basically, that's the principle under which insurers operate.
So I talked about you, last time, starting your MIT insurance company, where you would go and insure MIT graduates.
Well, as long as you have a large sample of MIT graduates, and you knew they were all going to buy health insurance, then you could clearly predict what you'd have to spend, through the Law of Large Numbers, and you could have a functioning insurance market where you charge that plus a little money to make, to pay for your profits.
And that's how large business works.
So MIT has insurers, the MIT insurers know MIT has about 10,000 employees altogether, 12,000 maybe.
The insurers say, look, that's a large enough sample that we can basically predict what MIT is going to cost, and we can basically figure out what to charge them for premiums.
MIT says, well gee, we wish it was less, but that's fair.
We understand what health care costs, and we're done.
That's not true if you don't work for a large firm, particularly for an individual.
So let's say you've got your MIT health insurance company.
You've graduated, you're successful, you set up a program insuring all MIT graduates.
Let's say, MIT forces everyone to buy health insurance, so you've got a nice, predictable distribution of risks, and you're making your money.
And then I walk in the door, and I don't have to buy health insurance from you.
I'm just a regular guy off the street.
I say, hey, I hear you've got this great health insurance product.
I want in.
And you look at me say, wait a second, why do you want in?
You probably only want in because you're sick, and you know that you're going to need health care, and that's why you want to buy from me.
So no I'm not going to sell to you.
And that's the problem individuals face who want to buy health insurance on their own, is the problem of asymmetric information goes away when you've got a large pool of people who are all buying health insurance.
For any one individual trying to walk in off the street, the problem of asymmetric information means that insurers will be unwilling to sell to them, and if insurers are unwilling to sell to them, those individuals might end up uninsured.
Currently in the US, we have about 50 million uninsured people, so that about 18% of our non-elderly population is uninsured.
Remember, the elderly, as I discussed last time, get universal health coverage through the Medicare program.
But about 18% of our non-elderly population, around 50 million people, are uninsured, and one of the reasons they're uninsured is because of this insurance market failure.
And to see that insurance market failure, you can just look at what happens when people try to get health insurance outside of large employers, through what we call the non-group insurance market.
In the non-group insurance market, when people try to get health insurance, it's typically incredibly expensive because insurers are worried you're sick, so they charge you a high price.
I talked about that last time.
That the lemons problem.
Insurers also, in trying to deal with asymmetric information, the way they deal with it is by being discriminatory.
So if you're an insurer who's allowed to, what you can do is to say, fine, I'm going to insure you, but I'm not going to insure you for any sickness you previously had because I know that's something that might flare up again.
That's called a pre-existing condition.
I'm not going to insure you for any pre-existing condition.
Moreover, I'm going to go even further.
If you get sick, I'm going to drop you.
You might say, wait a second, the whole idea of insurance is you cover me if I get sick, but in fact, in most states' insurers, it's totally legal to do.
Insurers can simply say we're not renewing you.
You got sick.
Well, that clearly is a problem.
That clearly means that adverse selection's led to market failure because of what we discussed in the uncertainty lectures.
You should go back and review that, in the uncertainty lecture, we talked about why individuals would value insurance, why they pay a risk premium for insurance, well, if insurers simply won't sell to them once they get sick, the individuals aren't getting something they value, and the market's failed.
So we've got a situation in the US where we've got insurance that works for many, many people, those that work for large firms.
But insurance markets that are failed for individuals, and small firms are sort of in between, and that leads to 50 million uninsured people.
So we got this problem.
We got all these uninsured people on the one hand.
On the other hand, we have incredibly rapidly rising health care costs.
We, by far, devote the largest share of our economy to health care.
We're about twice the average of industrialized countries, and our costs are rising incredibly rapidly, although that's true around the world, as well.
So that's the sort of situation we face ourselves in, and that's thte motivation that underlies the discussion you're hearing now about health care reform.
And in particular, you're hearing a lot of discussion now about the Patient Protection and Affordable Care Act, which passed last March 23.
This was the single most important piece of government legislation, perhaps, since World War II.
Certainly, the most significant piece of domestic social policy legislation since Medicare was introduced in 1965.
What does this bill do?
Full disclaimer, I'm going to describe objectively, but I helped write it.
I'll try to be objective, but just full disclaimer, I was involved with writing the legislation.
So there is some bias involved here.
But what does this piece of legislation try to do?
What it tries to do is deal with the second of the problems I discussed, which is the insurance market failure, and it makes a vague stab at the first, which is the fact that health costs are getting very, very expensive.
Since more economics is involved in that second, in the insurance market failure, let's talk about that.
So how do you solve insurance market failure?
You've got this problem that there are 50 million people without health insurance, often because this asymmetric information problem.
50 million people without health insurance.
Many more are in this market that sort of can drop you when you get sick.
We discussed the reasons last time, so what are the two options we discussed last time for what the government might do if they want to solve this problem?
Yeah
Subsidizing the insurance companies--
Subsidizing the people and the insurance companies.
So one thing I could do is I could say, look, let's just make insurance free for everyone.
That way everyone will sign up, and then insurance companies can offer fair prices.
And that's a solution that's pursued by Canada.
Canada has free universal health care coverage.
Everyone in the country is covered by birth for their health insurance expenditures under one, single government insurance company.
But it wouldn't have to be one, single government insurance company.
That part's distinct.
We could separate who provides insurance versus who pays for it.
OK, so let's just focus on the who pays for it part.
If we make health insurance free, sign up everyone from birth, then we've solved the adverse selection problem, but that costs a lot of money.
To sign up everyone from birth and cover them with insurance at the full government cost, would cost on the order of $2 trillion a year.
That's a lot of money.
That's pretty hard to imagine the US government raising taxes enough to cover that kind of new program.
So the second solution we talked about was a mandate, was to say, OK fine, we can't make insurance free for everyone.
What we can do is we make sure everyone buys, so that we fixed this adverse selection problem.
So that insurers can know they are getting everyone, so they can price insurance fairly.
And basically, that's what PPACA does.
It does three things.
The first thing is it says to insurance companies, we're going to give you an individual mandate.
So starting 2014, every person in the US will have to buy health insurance.
Now, it's not quite every person.
There's exemptions.
For example, it doesn't apply to illegal immigrants, it doesn't apply to people for whom health insurance is particularly expensive, et cetera.
But the estimate is this mandate would cause about 60% of the uninsured get coverage.
So about 47% of the uninsured will slip through the cracks, but about 60% will get coverage.
The second thing it does, it says, OK, insurance companies, since we're giving you the mandate, we're going to say you can no longer discriminate in health insurance.
OK, no more discrimination in health insurance, so what that's saying is, now, you can no longer, for example, drop people when they get sick.
You can no longer exclude people for their pre-existing conditions.
You have to basically offer insurance the way we described it when we talked about insurance in the uncertainty lecture.
Once you get sick, you're covered, and anyone can get it, and it's priced fairly.
So basically, the trade-off is we're saying, insurance companies, we'll give you a broad distribution of risks.
On the other hand, you can't discriminate anymore in health insurance.
Let's give everyone the same kind of health insurance that MIT gives people, that large companies give people.
But then finally, the final thing the bill does is, to make that work, we have to recognize that you can't mandate people to buy health insurance if they can't afford it.
Health insurance is pretty expensive.
My MIT health insurance policy for my family cost about $14,000 a year.
MIT pays about 10, I pay about four.
Remember we talked about poverty.
The poverty line now for a family is about $22,000.
You can't take a family earning $22,000 and say I'm going to mandate you to spend $14,000 on health insurance.
That's just impossible.
They couldn't live.
So the third thing the bill does is it introduces a whole bunch of subsidies, so in some sense, it uses both the solutions we talked about last time.
Both making insurance cheap for people, so they can afford it, but instead of spending $2 trillion a year, it spends like $120 billion a year on this.
So it just subsidizes the very poorest people, and mandates the other people to buy.
So it's sort of a mixed solution.
We talked about two solutions you could do to solve the asymmetric information problem.
You could subsidize insurance and make it cheap, or you could mandate people to buy.
This bill does both.
It subsidizes it for the very lowest income people and mandates it for people who can afford it.
So that's essentially what the bill does to try to solve this problem, and we don't know whether that will work.
We have our best estimates, but this is a radical new intervention.
The best estimates suggest it'll cover about 60% of the uninsured, but this is a pretty brand new intervention, so we'll have to see, actually, how that shakes out.
But that's the basic structure of what the bill's trying to do to try to solve this asymmetric information problem.
Are there questions about that, how it relates to what we learned in the last couple lectures?
So that's one problem that we're trying to address with this legislation, is solving that asymmetric information problem using both the mandate and subsidy tools.
And as I said, if you want to learn more, I talk a lot more about stuff like this in my course 14.41.
This is sort of an example of how government can use its tools to address the kind of fundamental information failures that we talked about last time.
But we don't get into, in this course, politics, but of course, we can't talk about a government intervention this massive without talking about politics for a minute, and recognize that this is going to involve some sacrifice.
I made it sound pretty easy.
Well, we've got this asymmetric information problem, we put in the solution, we're done.
But there's two problems with the solution.
The first problem is you've got to pay for these subsidies.
And this comes back to our discussion of equity efficiency, the bill features an enormous tax increase on the wealthiest Americans.
The bill raises about $450 billion over the next decade from the richest, about 5% of American families.
So going back to our equity efficiency discussion, is that a good thing or bad thing?
That depends on our social welfare function.
If our social welfare function is very progressive, Rawlsian or some forms of utilitarian, that's probably OK.
If it's not, if it's a Nozickian world, where we just think that people should just be left alone and not taking money away from the rich just because they're rich, then that's not OK.
So that ties into our other discussion of equity efficiency.
So we have to pay for this, but obviously, politically, that raises problems.
That's going to upset a large set of people who are going to have to pay those taxes.
The other issue of course this is with the mandate, we talked about, well gee, you can't do it by subsidies alone, that's too expense.
You have to mandate.
Well, the mandate isn't free either.
The mandate basically amounts to a tax on people who don't want to buy health insurance.
What's the mandate?
A mandate is saying you didn't want to buy health insurance, and now you have to.
That's going to upset a lot of people, as well.
A lot of people out there don't want to buy health insurance, and you're telling them they have to.
You don't make a change this big without causing some problems, and these are the problems that arise.
This isn't a painless solution, and so that's, once again, why they call this the dismal science.
We're all about the trade-offs, and the trade-offs here are, you fix the market failure that we learned about last lecture, but at, well, the potential cost of the inefficiency of taxing people.
Whether that's a good thing or not depends on our social welfare function.
And also the inequity, potentially, of forcing people to buy health insurance who didn't want it.
Yeah
How do they suggest that you enforce the mandate?
The mandate is enforced through a tax penalty.
We actually have Massachusetts.
We put in this system in Massachusetts a few years ago.
We pioneered it, basically.
And the way it works in Massachusetts is, every year, I get something called the 1099-HC from my insurance company, which says here's proof that you have health insurance.
I attach that to my tax form, and I'm done.
If you don't attach a 1099-HC, you have to pay a tax penalty.
In Massachusetts, it's about $1,000 a year if you don't have health insurance.
The federal tax penalty will be about $700 a year.
So it's going to be enforced through a penalty if you don't show, on your taxes, that you have health insurance.
Other questions about this?
Now, of course, I haven't talked about the bigger problem, which is what about the fact that we have enormous cost control problem.
Now, last time I talked about two kinds of asymmetric information, two problems in the social insurance trade-off.
On the one hand, there's adverse selection, that fact that insurance markets might not fail, but I also talked about moral hazard.
I also talk about moral hazard, and the fact that, if you insure individuals for adverse events, they will act adversely.
Now, there's several kinds of moral hazard, and health care is rife with them.
One kind of moral hazard is a type we talked about, which is the classic moral hazard, which is sort of individual precaution.
So the story here is, if I have fire insurance for my house, I don't bother buying a fire extinguisher.
Or if I have health insurance, I don't bother wearing a helmet when I bicycle.
So these are basically individuals not taking caution because they're insured for this adverse event.
We don't think that's that big a deal.
I mean, we don't think that your decision about whether to wear a helmet or not is much affected by whether you have health insurance.
It's probably not that big a deal.
The bigger type of moral hazard is resource over-utilization, which is that, basically, because people are insured, they will overuse medical care.
So basically, here's the way we think about that.
It's a standard dead weight loss kind of argument.
So think about the market for medical care.
Think about the market for doctors visits.
There's some quantity of doctor's visits and some price of doctor's visits.
And let's say that there's a flat marginal cost of doctor's visit.
Let's say delivering a doctor's visit costs $100.
So delivering a doctor's visit costs $100.
And let's say there's some demand for doctor's visits.
And importantly, I'm going to make this downward sloping.
Now, this is important.
You might say, gee, wait a second, shouldn't the demand for doctor's visits be inelastic?
Shouldn't people just go to the doctor when they're sick, and not when they're not?
And why would the price matter?
In fact, it's uncertain, from theory, whether this should be inelastic or elastic.
It's uncertain how elastic demand should be, if people only go to the doctor when they're sick.
I hope you understand, the thing about elasticity, it's a good thing to understand for basic principles, if people only go to the doctor when they're sick, that curve should be vertical because it shouldn't depend on price.
But if people care about the price, and they decide to go to the doctor, then it could be downwards sloping.
In fact, there's excellent evidence from a actual experiment that was run in the US.
In the 1970s, there was something called the Rand Health Insurance Experiment.
Rand is a company out in California, a consulting company.
They ran a health insurance experiment where they literally, experimentally, gave people health insurance of different generosity.
You've got to plan the coverage, everything, you've got to plan with a big deductible.
So they gave people plans of different generosity and asked how much different medical care did they us.
Remember when we talked about, when doing empirical work, what you'd like to do is think about a randomized trial.
They essentially ran a randomized trial, where different people had different prices, and they found demand was downward sloping.
People who were charged more for health care used less of it.
So there is a downward-sloping demand.
And so what that says is, if this is the marginal cost, and this is the marginal benefit of getting health care, then the optimal amount of doctors is just Q*, that basically, the socially efficient level of doctors is Q*.
However, when they're insured, people don't face this $100.
When you go to doctor, you just pay $10, typically.
Well now, this is the social marginal cost, but your private marginal cost, what it costs you, is only $10.
So what do you do?
You use a level of health care, Q super p, for private.
You use too much health care.
You overuse the doctor's office.
Now when you think of this intuitively, forgetting the graph, you just say intuitively, if a doctor's visit is going to cost $10 instead of the $100 it should, you use too many of them.
But graphically, the point is that the underlying cost of the doctor's visit is $100, but because you're insured, and the marginal cost to you is only $10, you overuse the doctor's office.
You use too many doctor's visits.
This leads to a dead weight loss.
This leads to a dead weight loss from people over using the doctor's office.
OK And that the second kind of moral hazard, which is resource over-utilization.
Questions about that?
Yeah
How do you define too much?
If everybody else-- you said Qp is just right, and Q* is like people go to the doctor too few times?
That that's a great question.
If everything is working perfectly, then Q* is, by definition, right.
We talked about, earlier in this course, about how the private market competitive outcome is welfare maximizing.
The private market competitive outcome is where marginal cost equals marginal benefit, supply equals demand.
That's the optimal outcome if there's no other market failures.
So in this course up to a few lectures ago, you shouldn't have even asked that question, we know Q* is optimal because we know the private market outcome's optimal.
Now, you might say, OK well, there's lots of reasons to think Q* might not be optimal.
For example, many people aren't fully informed about the right level of medical care to use.
Then you're right, then it might be that Qp is optimal, but the point is, absent other market failures-- We know Q* is the socially maximizing point where marginal cost equals marginal benefits.
We developed that earlier.
So if people use more than that, that's inefficient.
So you might say, well gee, how do we know?
Well, actually it turns out we can tell.
Let's go back the Rand Health Insurance experiment.
Same person, see if you can figure it out.
How could we use that same experiment to tell whether it was optimal that people used fewer or more doctor's visits?
What could you do?
Or anyone?
What could you do using that same experiment to see whether Q* was optimal or Qp was optimal?
Yeah
Take a look at who gets sick less
Look at their health.
You've got a randomized experiment.
Some people are at Q*.
Some people randomly are moving to Qp.
Look at their health.
Well, it turns out, health is totally unaffected, that the health of people at Q* was the same as the health as people at Qp.
People who had big deductibles and used health care a lot less were no sicker than people who used health care a lot.
The answer is because in America, we overuse health care.
We do know Qp is too much.
In America, we overuse health care because it's so cheap, and so the way we can prove that, this experiment showed it, that we caused people to say stop overusing.
I'll make you actually bear the full price.
Do the Q*, people were no sicker, and they used less health care.
And that was a striking finding out of this experiment, that in fact, we were overusing health care.
There was a real problem of Qp being too big.
That's not the only form of moral hazard.
There's another one we even talked about, which is provider moral hazard.
We've blamed everything so far on people, but doctors are people, too.
And in fact, much of what's done to you was determined, not by your decision, but by your doctor's decision.
So now we have to think about the doctors and how they make decisions.
We can teach a whole course on that, but basically, if we think about doctors, we're going to recognize that what doctors primarily care about is making their patients better.
But at the same time, doctors are people, too, and they also care about how much money they make.
Let's say for example, doctors didn't care about how much money they make.
All they care about is making their patients better.
And let's say, I come into my doctor's office, and I've got a headache.
I say, Doc, I've had a headache for like four days that's not going away.
Well, the doctor says, well, there's a point 0.0000000000001 chance you've got a brain tumor.
I can find that with a CAT scan.
Now, it's really so small, it's not worth it, but the same time, why not?
It doesn't cost you anything.
It just costs you Qp, which is $10.
Just costs you $10, so you don't care about getting it, and I'll just order it.
It's fine, why not?
So because I'm over insured, that doesn't just cause me to demand more medical care, it causes my doctor to give me more medical care.
Now, add on top of that the fact the doctor makes money off the MRI.
The doctor makes money off the CAT scan.
Now, he might say, gee, if it doesn't do you any good at all, I still might give it to you.
If you're interested at all in health care, the first thing you have to read is a terrific article in the June 9, 2009, New Yorker by a guy name Atul Gawande, who's a practicing physician and excellent author, who wrote about a place called McAllen, Texas.
McAllen, Texas, is the place in America that where people spend the single largest amount per person on health care in the Medicare program.
He compared McAllen, Texas, to a nearby town, El Paso, Texas.
Very similar demographically, very similar towns, but in McAllen, they spend twice as much on health care.
Why?
Because they get six times as many CAT scans, they have three times as many surgeries.
Basically, doctors go nuts treating people in McAllen, Texas, and the doctors are really, really rich.
They're not hurting people.
There's no evidence that the over treatment is hurting people.
There's just no evidence that it's helping them.
So doctors are like why not?
Doesn't cost the people anything, and I make money off it.
So that's another kind of moral hazard.
By providing insurance for medical care, we've induced this over provision of medical care, both because people want it because it's cheap, and because doctors want it because they get paid for it.
And that's a lot of the problem in our health care system in the US.
And once again, going back to the question back there, if you look at McAllen, Texas, where they spend twice as much on health care as El Paso, people are no healthier.
They're no healthier despite the fact the spend twice as much.
And indeed, based on facts such as this, the best estimates are that the US wastes about 1/3 of all of our medical spending, that we could literally spent 2/3 as much.
Or literally, at this point, we could say we spend almost $1 trillion less per year on health care and be no less healthy.
And that is the thing we have to grapple with now.
It's the fact that if we can deal with that problem, we could solve this long-running cost growth problem, and our long-running fiscal problem, if we just stop doing health care that didn't make us any healthier.
The problem is that's a lot easier said than done.
It's easier said than done for two reasons.
First reason is because people make money off that health care we don't need, so the first thing's political.
The only way we're going to do that is to tell some people they can't make money they've been making.
That's a hard thing to do politically.
The second problem is scientific, which is we don't exactly know which is the 1/3rd of health care that's being wasted and which is the 1/3rd that's not.
So let's put it another way.
Since 1950, the share of our economy we spend on health care has more than tripled.
But you know what?
Health care is a ton better than 1950.
Health care sucked in 1950.
In 1950, the typical baby born had more than twice as high a chance of dying in the first year of life as today.
Someone had a heart attack, lived 1/3rd as long after a heart attack as they do today.
Or to put in terms young healthy people might care about, in 1950, if you hurt your knee skiing, if you tore your ACL skiing, you went into the hospital for a week, had major surgery, were on crutches for six weeks, couldn't really do sports or anything effective for about six months, and arthritis for the rest of your life.
Today, if you hurt your knee skiing, you go to outpatient surgery for an hour and a half, you get it scoped, a little camera, they fix it, and then you're back skiing two weeks later.
No permanent long run damage.
Health care is just better today.
It just is.
That's why we don't see any health insurance plan saying we offer 1950s health care at 1950s prices because no one would want it.
Even though it would be 1/10th as cheap, no one would want because it would suck.
So basically the problem is health care's better over all, but at the same time, we're wasting a lot of money, and that's why health care cost control is really hard.
That's because moral hazard is hard to deal with.
Moral hazard is hard to deal with because it's hard to find because it's an information asymmetry.
If every procedure had attached to it a magic identifier, which said this is a waste and this is not, we'd have no moral hazard.
Imagine every procedure hospitals did had a magic identifier, which said this is worth it, this is not, then you could set up the optimal insurance company.
You could simply say, for those that are worth it, we'll insure you.
For those that aren't worth it, we won't, and you'd solve moral hazard.
Then we could be fine.
We could insure people, we'd solve both information problems.
But of course, we don't know that.
And so as we solve the first information problem, which is adverse selection, by getting more people health insurance, we're making the second problem worse, moral hazard, by putting more people into a system which wastes a lot of money because of these problems.
And that's the difficult part about health care.
So what do we do about that?
Well, I told you what we do to solve the first problem.
What to do to solve the second problem is a lot harder, and we don't really know.
But we have some ideas about what might work.
The first thing we know might work is a lot more work on what's call comparative effectiveness, which is a fancy way of saying understanding what works and what doesn't, and in particular, what works better than what else.
Many of you know about the FDA, the Food and Drug Administration.
They approve medical devices, so if you want to introduce a new drug or a new medical device, it has to be approved by the FDA.
The way their approval runs is they say, is it effective in curing what it's supposed to cure, and if it is they approve it.
They never say is it any more effective than something that already exists, or even more is it actually cost effective.
So for example, in the treatment of prostate cancer over the last decade, we've moved to more and more expensive radioactive laser-based treatments for prostate cancer.
We used to treat prostate cancer about a 10th the cost of treating it now with no evidence that we're actually doing men any good through these treatments.
But they don't do harm, they work, they do what they're supposed to do, so the FDA keeps approving them.
So comparative effectiveness will be to actually say we're only going to pay for what works better than something that exists already more effectively.
That's one direction we can go in.
The other direction we can go in is we can start working to change the financial incentives for both patients and providers.
For patients, we can make them pay more of the cost of health care, so that they move back up this curve a little bit.
Still insuring them, but making them pay more.
So for example, we could move to systems where poor people are fully insured because they can't afford it.
But someone who's higher income, they should have a high deductible, they should bear the cost some of their health care because they can afford it, and that will make them make their health care decisions more wisely.
We can work on the individual moral hazard by making people pay more for their health care.
We can work on the provider moral hazard by removing the financial incentives to providers to provide excessive care, and the way we do this is by something called prospective reimbursement.
Prospective reimbursement is basically saying to providers, look, the way things work now is you do a CAT scan on the guy we pay you for the CAT scan on the guy.
You do a surgery on the guy, we pay for your surgery on the guy.
How about a different way to approach it?
How about we say this guy is your patient?
A guy like him, a 25 year old healthy guy costs, on average, $1,500 a year.
We will give you, at the end the year, a check for $1,500.
You do what you want.
You do nothing, you keep the $1,500.
You go crazy and you make 1,000 CAT scans and it costs $100,000, you eat the difference.
You doctors bear the risk.
Pros, instead of we, the public and the insurance companies and the individuals, bearing the costs when you give me an extra CAT scan, you, the doctor, bear the cost when you get the extra CAT scan.
We've turned the financial incentives on their head.
Now, instead of making money by giving CAT scans, you lose money by giving CAT scans.
Yeah
But might that have some negative side effects on the health care of patients?
Doctors are going to want to check how much they are going to make [INAUDIBLE]
Great question, like the question before.
How do we know that doesn't go too far?
Right now, you could think about it, we're in a place for both patients and doctors where there is clearly over treatment.
There, of course, exists under treatment as a problem, and we know that both making patients pay more and putting risk on providers, through making them bear the cost of CAT scans out of their pocket, will move us in this direction.
The question is, does it move us past the happy medium, or does it just move us towards the happy medium?
In the back was asked a question about patient payments, and here we know the answer, which is making patients pay more reduces utilization but still leaves us on the right side of a laughter curve if we come to our tax analysis.
It still leaves us in the place where we're still, on average, still probably doing too much care, but less too much care.
What about providers?
Well, we have evidence on that, which is we've put in prospective payment systems in a number of different contexts, and typically we also find it does not hurt health, that providers still provide, if anything, too much care and not too little.
Now, we've never put in a pure system of the kind I've described.
We've always put in sort of mixed systems, where you get a $1,500 check, but if you do a lot of stuff, we'll also pay you extra, a little bit.
So we've never put in a pure prospective system.
So for example, we put in a system like this for Medicare, the way Medicare pays hospitals.
Medicare use to pay hospitals, that when you went to the hospital, and the hospital billed Medicare, they just paid it.
Medicare then put in something which said, look, if someone goes in the hospital with this given diagnosis, we'll pay you this fixed amount, so it's more prospective.
What they found was an amazing reduction in how intensely elders were being treated.
The average length of stay for an elderly person in the hospital fell 20% in one year.
There's an enormous reduction in how long elders were kept in hospital and how intensely they were treated, yet elders were no worse off.
So we moved in this direction, but we clearly haven't moved passed the middle.
Questions about that?
So basically, what the bill does is it tries to set up incentives, on both patients and providers, to behave more economically, but the answer is it's still very small.
We're still years away from really putting those incentives in place in a major way, and that's sort of the next round of health care reform that has to go on.
And that's all.
I don't want to go on too long.
I just wanted to give you an overview of one example of how we take a public policy tool and deal with these problems that we've raised earlier in the course.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses visit mitopencourseware@ocw.mit.edu
Hi, welcome back to the 14.01 problem solving videos.
Today we're going to work on p-set 1 problem number 3 from Fall 2010.
And we're going to work through all four parts of this problem.
But to start off I'm just going to read through part A.
Consider the market for apple juice.
In this market the supply curve is given by quantity supplied equals 10 pj minus 5 pa, and the demand curve is given by quantity demanded equals 100 minus 15 pj plus 10 pt, where j denotes apple juice, a denotes apples, and t denotes tea.
Part A asks us to assume that pa is fixed at $1 and pt equals 5.
We need to calculate the equilibrium price and quantity in the apple juice market.
So to start off this problem, I wrote down both the supply and the demand functions.
But before we get started with the algebra, I wanted to come over to this graph and I wanted to think about conceptually what we're going to be doing.
When we solve for an equilibrium price and the equilibrium quantity, all we're doing is we're finding the point at which the quantity supplied and the quantity demanded is equal.
Looking at the graph, that point is going to be right here where the two curves intersect.
So this will be q star, our equilibrium quantity, and this will be p star, our equilibrium price.
So for part A we're just solving for the equilibrium price and quantity.
And they try to trip you up on this problem by throwing in the price of apples and the price of tea.
But since they tell us what these prices are initially, we're just going to plug these into our supply and our demand functions.
And once we do that, we'll have isolated the pj variable and the q variable so we'll be able to solve through for this problem.
So starting off with part A.
We're going to go ahead and we're going to set the quantity supplied equal to the quantity demanded.
And so for my supply function, I've already plugged in pa. And after plugging in pa I found that 10 pj minus 5 is the supply curve.
And the demand curve is equal to 150 minus 15 pj.
Solving out for pj I find that the equilibrium price is equal to 6.2.
And since I know this is an equilibrium price I'm going to go ahead and I'm going to label this with a star.
Solving through for the equilibrium quantity all we have to do is we have to take this equilibrium price we found and plug it back into either the supply curve or the demand curve.
I'm going to go ahead and I'm going to plug it into the supply function.
And that lets us solve for the equilibrium quantity denoted with the star.
And that in this case is 57.
So looking at our graph, the equilibrium price and the equilibrium quantity, we can now label them.
We can label the price, 6.2, and the equilibrium quantity 57.
Let's go ahead and move on to part B.
Part B is going to be the exact same scenario as we started off with in part A, only what we're going to do now is we're going to shift the supply curve by changing the price of apples.
Part B states, suppose that a poor harvest season raises the price of apples to pa equals 2.
Find the new equilibrium price and quantity of apple juice and draw a graph to illustrate the answer.
Now what's happening in this scenario is that the demand curve is completely unaffected.
The only thing that's changing is our supply curve is shifting.
So when we look at our supply curve we have to think about conceptually what do apples represent.
Well they're an input for the suppliers.
It's something they have to use to make the apple juice.
And if the price of apples is increasing then we intuitively know that this quantity, or this q star that they produce before, it's going to be more expensive for them to produce it.
And, in fact, it's going to be more expensive for the suppliers to produce any given quantity.
So this means that the supply curve for part B is shifting up and to the left.
I'm going to denote this by labeling our new supply curve sb for Part B.
So for Part B all we're going to do is we're going to plug in this new pa price.
And then we're going to do the same thing.
We're going to set the quantity supplied and the quantity demanded equal.
When we solve through for a new supply curve we find that 10pj minus 10 is our new supply curve.
And we know that are demand curve is going to be exactly the same as the scenario that we started off with in Part A.
Solving through for the new equilibrium price we find that pj is going to be equal to 6.4.
I'm going to label this new equilibrium price with a B.
And then we can take this equilibrium price, we can plug it back into either our new supply curve or our demand curve.
And we can find that the equilibrium quantity is going to equal 54.
And when we look at our graph we can think about what these quantities and these new prices looks like.
We can see that the quantity for part B clearly should have shifted down, and it did drop from 57 to 54.
And we can see that the price is higher than what we started off with.
It rose from 6.2 now up to 6.4.
So we can see that our intuition that the supply curve would shift up because it's more expensive for suppliers makes sense according to both our algebra and to our graph.
Let's go ahead and try part C. Part C is going to be another shift, but what's happening now is this new shift is going to affect the demand curve not the supply curve.
Part C states, suppose that pa equals 1, but the price of tea drops to pt equals 3.
Find the new equilibrium price and quantity of apple juice.
So for this scenario now, our pa is back to 1, like it started off with in part one.
But now t is dropped from five to three.
Now before we start, let's think about what t actually represents to the consumers in this market.
If I'm a consumer and I'm debating what I want to drink in the morning, one of my other choices might be tea.
Now the price of tea drops, then maybe I'll be less willing to pay as much for the apple juice because they can just go out and get the cheap tea now.
Looking at our graph, if the consumer is less willing to pay as much for each quantity of apple juice as they were before, this scenario means that the demand curve has shifted down.
So, for example, the equilibrium quantity that we started off with an point A, they used to be willing to pay this much, now they would only be willing to pay this much for the same quantity because tea is cheaper.
Now we're going to be using the same supply curve that we started off with before.
So in this scenario we're looking to set supply and demand equal and find this new equilibrium point.
When we go through and we actually plug in the new pa and the new pt, we're going to find that the supply and the demand functions.
When we set them equal we're going to set 10 pj minus 5 equal to 130 minus 15 pj.
Again we're going to solve through for pj.
And we're going to find that our pjc for part C is going to be equal to 5.4.
And we're going to find that the new equilibrium quantity, again either plugging back into our supply curve or demand curve, is going to be equal to 49.
And when we think back to our original part A, we found that the price was 6.2.
Now the price dropped down to 54 or 5.4.
And what we see here, is that we did see a price decrease, or we could have predicted a price decrease according to our graph.
And we also find out the quantity dropped from part A in 57 now the 49.
And according to our graph, we would have predicted that quantity would have decreased as well.
So the algebra and our graph both match up.
Let's go ahead and move on to part D. Now we're going to look at the effect of a government intervention on the market for apple juice.
Part D says, suppose that pa equals 1 and pt equals 5 and there's a price ceiling on apple juice of pj star equals 5.
What is the excess demand for apple juice as a result?
Draw a graph to illustrate your answer.
So now we're back to the original scenario we started with in part A.
We have the price of apples is equal to 1.
And we're back to the price of tea equal to five.
And now the only thing that's different in this scenario for part D is that the government said that these suppliers can't charge any price higher than 5.
So let's go ahead and let's think about what this looks like conceptually on a graph.
In this scenario we started off with an equilibrium price of 6.2.
If we were to have a case where the government was saying you could charge only as much as 7 in the market for apple juice, the suppliers wouldn't even be affected.
Because they would say, that doesn't affect us, we're charging 6.2.
That's too high.
We're not going to have to change anything we're doing.
But in this scenario the government is saying the most you can charge is 5.
Now at the price of 5, the quantity supplied, or the intersection of the price of 5 with the supply curve, is going to lead to a qs that's different than the quantity that's demanded.
And more specifically we're going to find that too many people are demanding the product and there's not enough being supplied in the market.
This is what we're referring to when we talk about excess demand.
We're talking about the space between the quantity that's supplied and the quantity that's demanded.
And that's what we're going to be solving for.
So by plugging in the price cap of 5 into both our supply curve and our demand curve, we'll be able to find the difference between the amount that's supplied and the amount that's demanded.
So plugging into our supply curve we can find that the quantity supplied is going to be equal to 45.
And plugging into our demand curve we can find that the quantity demanded is going to be equal to 75.
Now one of the biggest problems that students run into on this problem is they think they're done when they reach this point.
All right, I found the quantity supplied, I found the quantity demanded, I know they're different.
I'm done.
But really what you need to find is exactly how different the quantity supplied and the quantity demanded is.
You need to find out how many consumers are left out of the market.
So our excess demand is the difference between the quantity supplied and the quantity demanded.
And that's going to be 30 in this case.
So just to review what we did in this problem, we started off by setting a quantity supplied and a quantity demanded equal to solve for a basic equilibrium price and quantity.
We then looked at shifts in supply and demand and looked at how that affected the price and quantity in a market.
And finally, we ended with part D with looking at the effect of a government intervention.
How does a price cap affect how many people can get a product compared to how many people want a product.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, and welcome back to the 14.01 problem solving videos.
Today, we're going to work on Fall 2010, problem set one, problem number four.
And in this problem, we're going to be working with elasticities.
But instead of starting with a demand function, and starting with a supply function, and calculating the elasticity given those functions, we're going to be given the elasticity of demand and the elasticity of supply.
And we're going to have to back out what the demand functions and the supply functions should have looked like.
So we're basically just working in reverse from what we did in lecture.
Let's go ahead and read the full problem up through part A.
You have been asked to analyze the market for steel.
From public sources, you are able to find that last year's price for steel was $20 per ton.
At this price, 100 million tons were sold on the world market.
From trade association data, you are able to obtain estimates for their own price elasticities of demand and supply on the world markets as negative 0.25 for demand and 0.5 for supply.
Assume the steel has linear demand and supply curves throughout.
Part A asks us to solve for the equations of demand and supply in this market, and to sketch the demand and supply curves.
So looking at the formal definition of elasticity of demand and elasticity of supply, we basically are going to have three different parts to it.
We have the derivative of either demand or supply function with respect to P, in this case the own price of P, or the price of steel.
And we also have the equilibrium price, or any price at the point on the curve, and a quantity.
In this case, it's going to be the equilibrium quantity.
So basically, what we have now is we are given-- for the elasticity of demand, we're given three variables.
We're given the price, the quantity, and the elasticity of demand.
And that means the only thing that we don't know is the derivative of the demand curve with respect to P. So if we can isolate this derivative, then we can integrate the number that we're able to solve through for.
And then we can solve out for what our demand curve is going to look like.
So let's go ahead and walk through that process together.
Substituting in for the elasticity of demand P and Q, we're gonna have this equation.
And the one thing that I want you to notice is since the derivative of the demand curve with respect to P is negative 0.25, in this case, we know that it's linear.
But just because it's linear at the point where price is 20 and quantity is 100, that doesn't necessarily mean it's gonna be linear throughout.
So it's useful to know that at any point on this line, it's always going to have the derivative equal to negative 0.25.
So that's useful.
We know we can integrate and have the correct answer.
Solving for dQD dP, we're gonna have negative 1.25.
And we're just going to integrate this with respect to P. And after we integrate, we're going to be left with a constant.
In this case, we're going to call the constant a.
This is how much the demand curve has actually shifted up to begin with, shifted up or down.
And to solve for a, all we have to do is we can just plug back in for the $20 and the 100 quantity, and we can solve through for what a is going to be equal.
When you solve through plugging in Q and P, you're going to find that a is equal to 125.
So we're gonna have that our final demand function is gonna be negative 1.25P plus 125.
Now we can go through this exact same process with the elasticity of supply now.
And all we have to do now is use the number 0.5 instead, and we can solve through.
We can integrate.
And then we're going to solve for the other constant to, again, get our supply curve.
Substituting in the information we have, we're going to be left with this equation.
And we're gonna go ahead and isolate the derivative that we have.
And when we integrate again, we have to remember that we are going to have a constant that we're gonna have to solve for.
And I'm gonna just call this constant c. Again, plug in the price of 20 and the quantity equal to 100 and you're gonna find that the supply curve is gonna be equal to 2.5P plus 50.
And we can do a quick sketch of this on our axes here.
We're just gonna go ahead and draw our upward-sloping supply curve, our downward-sloping demand curve.
And we're gonna mark the equilibrium point and label the equilibrium quantities and the equilibrium prices, as well.
And before we move on to the second part of this problem, we can pause here.
And we can think about what did the elasticities that we started with actually mean.
Well, if we were to look at this point of intersection at the equilibrium of the demand curve, we're looking at the percentage change at this point in quantity per percentage change in price.
So we're basically just saying, for that tiny change, an infinitesimally small change at this point for the demand curve, how much does quantity change, percentage-wise, relative to price?
And that's also what we're looking at with the supply curve.
So when you're given a elasticity, if you have an elasticity of supply, it makes sense that it's gonna be positive, in this case 0.5, because when price increases, suppliers are willing to supply more.
And it makes sense that the demand elasticity that we're given is negative, or negative 0.25, because when price begins to increase, the consumers are gonna want less of the product.
Now, the second part of this problem is going to give us new elasticities of demand and supply.
And I'm gonna just quickly run through the actual calculation, because it's gonna be the same as our calculation that we just did.
And instead, we're gonna think about possible causes for the shifts that we see in the supply and the demand curve.
Part B says, suppose that you discover that the current price of steel is $15 per ton and the current level of worldwide sales of steel is 150 million tons.
The most recent elasticity estimates from the trade association this year are negative 0.125 for demand and 0.25 for supply.
Describe the change in the supply and the demand curves over the past year using your diagram from part A.
What sort of events might explain the change?
Now, I've given us the information for this part of the problem on this board.
And you'll notice that our inputs, or our variables, have changed now.
The price has dropped from $20.
Now it's gonna be down to $15.
You're gonna notice that the quantity has actually increased from 100 up to 150.
And our elasticities of demand and elasticities of supply have changed, because the price and the quantity are different, and we're at a different point on our graph.
The process we're gonna do to solve for our demand curves and our supply curves are going to be exactly identical.
And when you follow the same process-- I'll just do the first step up here-- you're gonna substitute in for the information that's given in the problem.
And all you're going to do is you're, again, gonna solve through for the derivative.
You're gonna integrate.
And you're gonna find the constants.
After you do that entire process, you're gonna find that the demand curve is given by this equation.
And you're gonna find that the supply curve is given by this new equation.
Now, if we look at this new demand curve and this new supply curve, we'll actually notice that the slope, with respect to P, is going to be identical in both of the cases that we solved for, both the beginning case and the case in the end of the problem.
The only thing that's shifted between our quantities demanded and our quantities supplied, or the curves, is there's been a shift.
And the shift for the demand curve-- it went from an intercept of 125 now to an intercept of 168.75.
So our demand curve is shifting up and out.
So we can represent this shift in demand like this.
Notice that the slope is going to be exactly identical.
I'm going to write a small db for part B.
And then we can do the same sort of interpretation for our supply curve.
Looking at our supply curve, the intercepts, now, is 112.5.
But before, it was only at 50.
And what this means, this means that the supply curve is going to shift in and down.
And so my graph with the equilibrium price that I've drawn-- it's a little bit off, but what you should see-- you should see that the new equilibrium price has fallen.
In this case, it's fallen to 15.
And the equilibrium quantity has increased from 100 to 150.
So since we had both a shift in supply and a shift in demand, necessarily we see that quantity is going to increase.
But if the demand curve had shifted way up here, we could see that price could have increased.
So the effect on the price in this market is ambiguous.
We can say that, necessarily, the effect on quantity is going to be clearly towards an increase.
So to wrap up this problem, we saw that changes in elasticities can also represent changes in the underlying demand and supply functions.
Let's wrap up by just thinking about what could have caused the demand shift that we've seen.
And what could have caused the supply shift that we saw?
Now, there are a couple of ideas that we can have for demand.
The first idea that we could have is we could just have had an increase in the income of a consumer.
If a consumer has more income, then they might be willing to spend more on steel.
A second idea that we have, we could have that the price of a substitute-- perhaps you're considering building a bridge out of iron instead of steel-- if the price of the substitute has increased, then perhaps the consumers are going to be willing to pay more to get the steel since the iron is more expensive.
A third possible idea is that the number of goods that you need to make from steel is increasing.
So if you suddenly find new uses for steel, then the price that you're willing to pay at any given point is going to be higher.
Basically, to affect the demand curve, you have to think about why would people be more willing to pay more for a fixed quantity.
And I just listed off a couple of ideas there.
We can also think about reasons about why the supply curve could be shifting in.
In this case, why is it-- why are sellers willing to offer a cheaper price at any fixed quantity?
And one idea that we could have for this is just that there are more firms in this market.
If this market isn't perfectly competitive to start off with, then increasing the number of firms is gonna increase competition, and the producers are gonna have to drop their prices.
A second idea for why we've seen the supply curve shift out and down could be the fact that input price for steel has dropped.
Perhaps the way of manufacturing or getting the raw material is cheaper because the machine they're using to get the steel is cheaper.
Basically, when you're thinking about the shift that's making it cheaper for suppliers to produce the good, all you need to think about is what could make it so that they're more willing to produce at a lower price.
So again, with this problem, we went through working with elasticities and demands.
We've seen that we can go from a demand curve or supply curve to elasticities, or we can go from elasticities to demands.
And then, once we've had the supply and the demand curves, we looked at how do we interpret the shifts and shocks?
And we looked at possible explanations for those shift and shocks.
I hope you found this problem helpful.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, we're going to be working on fall 2010, problem set number 2, and we're going to be doing problem number 4.
I'm going to start off by reading the problem.
There's a lot of information here, so I've written up some of the key points on the board for you guys already.
It is exactly 24 hours before Lauren's physics final.
She has an economics final directly after her physics final, and has no time to study in between.
Lauren wants to be a physicist, so she places more weight on her physics test score.
Her utility function is given right here.
Where p is the score of her physics final and e is the score of her economics final.
Although, she cares more about physics, she is better at economics.
For each hour spent studying economics, she will increase her score by three points.
But her physics score will only increase by two points for every hour spent studying physics.
Studying zero hours results in a score of zero on both subjects.
Although natural log of zero is not defined, assume her utility for a score of zero is negative infinity.
Now, we're going to go ahead and we're going to work through parts A, B, and C, A, B, C, and D and then we'll do part E, which is a new scenario afterwards.
Part A, we're going to find the constraints that Lauren faces in her test score maximization problem.
And part B, we're going to find how many hours does Lauren optimally spend studying physics, how many hours does she spent studying economics.
And hours are divisible, so we don't need whole number solutions for the hours spent studying physics and the hours spent studying economics.
So for part A, all we're looking for is we're looking for the constraints.
And it sounds like the constraint that Lauren is really facing in this scenario is the amount of time that she has.
It seems like Lauren's pretty intense about studying, so in the 24 hours before the test, she's not getting any sleep.
She's going to spend all of this time studying.
So we're going to have two hours variables to represent the hours she spends studying physics and the hours she spends studying economics.
And we know that the hours, when we add them together, they're going to have to be less than or equal to 24.
Now, if this is our constraint, we really know that if she's trying to maximize her scores and if that's all she really cares about, she's not going to spend less than 24 hours studying.
So we can actually just say that the sum of those two hours-- her hours spent studying physics and her hours spent studying economics are going to be equal to 24.
So that's the first constraint that Lauren is going to face.
The second constraint that she's going to face, and it's not necessarily an intuitive one-- or it is actually really intuitive, but it's also a trivial one.
It's just that she can't spend less than zero hours studying physics or studying economics.
So we're going to add those constraints in as well.
And the final constraints are actually production constraints.
If the hours spent studying-- these aren't actually what she's interested in.
What she's interested in are the p and the e. That's how she's going to get her utilities-- through the test scores.
So what we need is we need to find how she produces her physics score.
If she gets two points for every hour she spent studying, her physics score is going to be p equals 2 times Hp-- that's the production of her physics score.
The production of her economics score is going to equal 3 times He, since we know that she's a little bit better at economics than physics.
So these are the constraints that we're going to face in the problem.
Let's go ahead and move on to part B.
And for part B, what we're going to try to do is we're going to try to take these constraints and we're trying to maximize the amount of utility she's going to get from studying.
So what we're doing for part B is we're going to maximize utility, and we're going to plug in some of these constraints.
Before we can do this, we really need to get down to first-order conditions.
We need to be able to take the derivative with regards to only one variable.
So we have to get-- instead of p and e here, we're going to try to get only one variable.
And the variable we're going to get in there is going to be the hours spent study economics.
So we're going to get it all in terms of He.
So we need to find a way to replace both e and p with He.
In this case, replacing e with He is really easy.
We can just say that e is going to equal 3He.
Replacing p with the He or the hours spent studying economics is a little bit more difficult.
We're going to go to this equation.
And we can say that Hp is going to be equal to 24 minus He.
And then for this Hp, we're going to plug this into our production function for the physics score.
So we get that p is equal to 2 times 24 minus He.
So we're going to take these two equations that we have in and we're going to substitute for e and p in our utility function.
And so we're going to maximize, and the way we're going to maximize is by just taking the first-order conditions or the FOC.
And the FOC in this case is just going to be the derivative with respect to He.
When you take the derivative with respect to He, what you're going to get is 0.6 times negative 2 all over (48 minus 2 He) plus 0.4 times 3 all over 3He.
And the reason this first-order condition is going to help us is because we know to maximize.
We just set the first-order condition or the derivative with respect to He equal to zero.
And from here, we can solve out for He.
And when we solve for He, we find that the hour she spends studying economics is going to equal to 9.6.
And we know that any time she spends not studying the economics, she's studying physics, since all she cares about is her test scores.
She's going to spend the remaining 14.4 hours studying physics.
This is our answer for part B.
Now, part C just asks us, "What economics and physics test scores will she achieve?"
so we're looking for the e star and the p star.
We can just take the hour she spent studying physics and the hours she spends studying economics and we can plug these back into her production functions for her physics score and an economics score.
So, for part C, her economics score is going to be equal to 3 times 9.6.
She's going to get a 28.8 on her economics test.
And for her physics test, she's also going to get a 28.8.
Probably the not the best test scores in the world.
Let's go ahead and look at part D. Part D asks us, "What utility level will she achieve with these test scores that she has?"
Now, to find the utility levels, all we have to do is we have to go back to our utility function that we wrote down in part A, and we're just going to plug in the two test scores that she received, and we can solve through for overall utility.
And so when you grab a calculator, and you solve through for this equation, what you're going to find is you're going to find her overall utility level is going to be equal to 3.36.
And when we're measuring utility, we don't have any units on this.
The units, we usually think of as utils, but we can just keep it like this.
Just know that in relativistic terms, her utility is that 3.36.
And this is going to be useful for part D. Because in part E, we have the option of sending Lauren off to an economics tutor.
And we're going to be comparing her utility after going to the economics tutor to this utility of 3.36.
And by comparing her new utility with the old utility, we can see if her utility's higher after going the economics tutor.
We can see if that's a good decision for her.
So now that we finished calculating the utility for Lauren in the initial case where she had 24 hours and no help, now we're going to move on to part E. Part E says, "Suppose Lauren can get an economics tutor.
If she goes to the tutor, she will increase her economics test score by five points for every hour spent studying, instead of three points.
But will lose four hours of study time by going to the tutor.
She cannot study while at the tutor, and going to the tutor does not directly improve her test score.
Should Lauren go to the tutor?"
Now, for this problem, we can go back to our initial scenario.
Where we had our constraint for the amount of time Lauren can spend.
If Lauren goes to the tutor, it sounds like she's going to spend about four hours in the car driving to the tutor's house and back.
So instead of having 24 total hours of study time, her new constraint is that her total study time Hp plus He is equal to 20.
That's the downside.
The upshot of it is that her production function for economics test scores is no longer three points for every hour she spent studying, now she gets five point for every hour she spends studying.
So we're going to go through the exact same process for A, B, C, and D that we did before, only now we're going to use our new constraints to solve for the maximization problem.
Before we start off for our maximization problem and solving for the first-order conditions, what we have to do is we have to get it all in terms of one variable.
And the variable we're going to get it in terms of is the hours of economic studying or He.
So from the utility function, to solve or to plug in for e, the He, we have that e is going to equal five times He.
For p, we know that Hp is equal to 20 minus He.
That just says that any time she's not studying economics, she's going to be studying physics.
We can plug this into our p equals 2 times Hp.
So we know that p is going to equal 2 times 20 minus He.
Let's go ahead and plug this and this into our utility maximization problem and solve for the first-order conditions.
So we're maximizing utilities-- so we're going to take the derivative with respect to He again.
When we do that, we get our FOC.
And again, for the maximization problem, we can just set that equal to 0.
Solving out for He, we find that she's going to spend eight hours studying economics.
And we also find that any time she's not spending studying economics, she's going to be studying physics.
So she's going to spend 12 hours studying physics.
Now, we're not done with part E because really all we have to do for this problem is we have to solve for her new utility using these hours that she spends studying.
And we have to compare her new utility to her old utility of 3.36 to see if she's better off or worse with the economics tutor.
In her production function for economics test score, we know that for each of those eight hours you spend studying, her test score's going to improve eight points, so her new economic score is 40.
And we know that for each of those 12 hours she spends studying physics, her score's going to improve two points.
So her physics score has dropped a little bit to 24.
When we plug in for e and p into our utility function, we can find her utility level with the economics tutor.
When we solve through for these natural logs, we're going to find that her new utility is equal to 3.38.
And since the problem asks us to actually make a qualitative judgment-- is she better off or worse off, we need to compare her utility before of 3.36 to her new utility of 3.38.
This is the comparison that we're interested in.
And when we make this comparison, we can see that in relative terms, she's a little bit better off going to be economics tutor because her utility has risen two hundredths of a point.
So in this case, for part E, we found that her utility increases even though she's lost time because her production function for her economics to score has gone up, and that she's better off going to the economics tutor because of that increase in utility.
So the point of this problem was to look at the different constraints that consumers face when deciding where to spend their time or where to spend their money, and to see how those different constraints affect their utility levels.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, and welcome back to the 14.01 problem solving videos.
Today, we're going to do Fall 2010, Problem Set 3, Problem Number 5.
And we're going to go ahead and we're going to work through parts A, B, C, D, and E, and then we're going to finish up parts F and G. Problem 5 says that Xiao spends all her income on statistical softwares and clothes.
Her preferences can be represented by the utility function where her utility equals 4 times the natural log of S plus 6 times the natural log of C, where S is software and C is clothes.
Part A asks us to compute the marginal rate of substitution of software for clothes, asks us if the MRS is increasing or decreasing in S, and also asks us how we interpret the MRS.
So before we start with this, we should really think about, conceptually, what the marginal rate of substitution of software for clothes looks like on our graph.
So on this graph, I have clothes on the y-axis-- the quantity of clothes-- and the quantity of software on the x-axis here.
Looking at this graph, this line that I've drawn is one utility level.
So at this place, she might have a utility equal to 1.
So she's indifferent on this indifference curve between being at point here or at a point here.
And what the marginal rate of substitution is really asking us, it's asking us how much clothing is she willing to give up to get one more unit of software?
So she's going to have to give up a certain amount of clothes to get one more unit of software.
And the marginal rate of substitution tells us exactly how much clothing she's willing to give up.
To calculate this algebraically, all we're going to do is we're going to take the marginal utility of software and divide it by the marginal utility of clothes.
So we're going to take the derivative with respect to software and the derivative with respect to clothing and divide.
When we do this, we find that the MRS is going to be equal to 4 over S, which is our marginal utility of software, all over 6 divided by C, which is our marginal utility of clothes.
Solving through, we find that our MRS is 4C over 6S.
Now, we have to think about, conceptually, what happens when software increases?
When we have S increase, since it's in the denominator, we're also going to have the MRS decrease.
So what this means is as software is increasing, or as she has more software, she's going to be willing to give up fewer clothing, or less clothing, to get another unit of software.
So looking at our graph, when she's at this point, she's more willing to give up clothing to get more software.
But when she has more software down here, she's less willing to give up the clothing.
Let's go ahead and move on to Part B.
Part B, find Xiao's demand functions for software and clothes-- so we're going to call those QS and QC-- in terms of the price of software PS, the price of clothes PC, and Xiao's income.
Now, before we move on with this, what we want to do is we want to solve for one of the variables C or S in terms of the prices and the other variable.
So to do this, we're going to set the MRS equal to the price of the software over the price of the clothes.
From here, we can solve through for C, and we find that C is going to be equal to 3/2 times PS over PC times S. Now, since we have two variables-- we have a variable for clothes and a variable for software-- we're going to have to introduce another constraint into this problem.
And the constraint that we're going to introduce is going to be the income function.
We know that Xiao has some sort of income that's going to be fixed, and she's going to spend all of this on either clothes or software.
Now, the amount of money she spends on software is going to be equal to the price of software times how much software she's going to buy.
The rest of her income is going to be spent on clothes, so the price of clothes times the quantity of clothing.
Now, to solve for the demand function for software, all we're going to do is we're going to plug in for C in the income function here, and then we're going to solve through for S. I know it's a little bit messy, but this says PS times 3/2PS over PC, what we solved for here, times S. Now, when we solve through for S from this equation, we're going to find that the demand function for software is going to be equal to 2/2 times income over the price of software.
Now, we can go through the same process solving for the demand function for clothing.
And all we'd have to do now is we can take this S right here that we just solved for, we can plug this back into our income function, and then we can solve for C. When we solve for C, we're going to find that the demand function for clothing is going to equal 3/5 times I over PC.
Part C asks us to draw the Engel curve for software.
Now, all an Engel curve is, it's a relationship between the income and the quantity that's demanded for a product.
And it shows us that as income increases, it shows how the quantity demanded is going to change with changing income.
So to start off our Engel curve, we're going to draw an axes, we're going to put software, or the quantity that's demanded, on the x-axis, and we're going to put the income on the y-axis.
Now, the nice thing about the software demand function that we solved for is that it's linear with respect to income.
Now, before we can graph this equation, however, we have to get it in terms of income.
So when we solve for this, we're going to find that income equals 5/2 PS times S. So all our Engel curve is going to look like is it's going to be a straight line.
And the slope of that straight line is going to be 5/2 PS.
And the way to interpret this conceptually is to say that with each one unit increase in income, the amount that's demanded is going to increase by 2/5 divided by PS.
Let's go ahead and move on to Part D. Part D says, suppose that the price of software is PS equals 2, and the price of clothing is going to equal PC equals 3.
And Xiao's income is going to equal 10.
What bundle of software and clothes maximize Xiao's utility?
Now, we've already found the conditional demand curves for both software and clothes.
So we can start off this problem by writing down those conditional demand curves.
The conditional demand curve for software was given by 2/5 I divided by PS.
And the conditional demand for clothing was given by 3/5 I divided by PC.
All we have to do now is we have to plug in these variables to solve for the software and the clothing that's going to be demanded.
When we plug those in, we're going to find that she's going to demand two units of both software and clothes.
So this is in the scenario for Part D. Part E gives us another scenario that we can solve for.
And all that's going to happen now is that the price of software is going to change.
And we're going to look at how that affects the bundle that maximizes her utility.
For Part E, it says, suppose that the price of software increases from PS equal to 2, and now it's going to be PS is going to equal 4.
What bundle of software and clothes does Xiao demand now?
Again, we're just going to solve through.
With our new PS equals 4, we're going to solve for the software and clothing that Xiao demands.
We're going to find that S is going to equal 1 now, and that the amount of clothing is going to equal 2.
So let's take a pause right here.
And we're going to come back in just a minute.
And we're going to look at the more interesting case, which is given the fact that she's consuming less-- she has one less unit of software to consume-- how do we get her back to the utility that she had before?
What amount of money or income do we have to give her so that she can be as happy as she was in this initial scenario with two units of both software and clothes?
Welcome back.
So we're going to continue onto Part F. Part F says, given the price increase, how much income does Xiao need to remain as happy-- have the same utility-- as she was before the price change?
What bundle of softwares and clothes would Xiao consume if she had the additional income given the new prices?
So we want to find out, how can we give her as much income so she can be as happy as she was to start off with?
To start this problem, the first thing that we're going to have to find out is we're going to have to find out exactly how happy Xiao was to begin with.
So we need to know her initial utility.
So let's start off with that calculation.
To calculate her initial utility, we're just going to start off by saying that her utility is equal to 4 natural log of S plus 6 natural log of C. And we can plug in 2 and 2 for S and C. In this case, when we solve through, we find that her initial utility is 6.931.
So we're going to set her utility equal to 6.931.
And what we want to find out is we want to find out what income we have to give her so that she can get up to this utility, given the new prices.
So we're going to take this utility function, and we're going to plug in the conditional demand curves for S and C so that income is now a function, or one of the inputs for her utility.
When we do that, we're going to get this function.
And remember, we said that we want, given this input and the new prices-- so we're going to set this PS is going to be equal to the new price in the problem.
And we're going to also set the PC equal to the price that was in Part E as well.
And flipping back to the problem, we know that the price of clothing is going to be equal to 3, and the price of software for the second part was equal to 4.
So we're going to plug in for PS and PC, we're going to set utility equal to 6.931, and we're going to solve through for I.
When we do this, and when we solve through for I, what we're going to find is we're going to have 6.931 is going to be equal to 10 natural log of I minus 4 natural log of 10 minus 6 natural log of 5.
Solving through, doing the inverse natural log function for I after isolating this variable, we're going to find that the new income that she needs to be supplied is 13.19.
So the income that she needs to be just as happy with these prices has increased by 3.19.
Now, we can go back to our conditional demand curves that we had here.
We can plug in PS equals 4, PC equals 3, and we can plug in for income 13.19.
And we can solve S double prime, which is the new software that she's going to demand, which will be 1.32, and C double prime, the new amount of clothes that she's going to demand, which is 2.64.
Now, the final part of the problem, which we're going to move on to now, which is Part G, is actually the most important part of the problem, because what we're going to do is we're going to tie together the three scenarios that we did.
We did this scenario where we were giving her income so that she would be just as happy.
We had our initial scenario before the price increase.
And we had the scenario after the price increase.
And we're going to look at this conceptually on a graph, and we're going to see, how do we relate these three bundles of consumption?
Part G says, going back to the situation in Part E, where PS equals 4 and I equals 10, we need to decompose the total change of softwares and clothes demanded into substitution and income effects.
In a clearly-labelled diagram, with softwares on the horizontal axis, show the income and substitution effects of the increase in the price of software.
Now, we're going to go back to this graph that we started off with.
And what we're going to do here, is we're going to illustrate the three bundles that she selected.
I'll illustrate the first bundle where she consumes 2 units of each.
And we already have our utility curve, or indifference curve, drawn up here.
Now we need to draw the budget constraint that shows how much she can spend on each product.
If she were to spend all her money on clothes, she would be up here at this corner solution.
If she were to spend all her money on software, she would be down here.
When we connect a line through here, this is her budget constraint that shows all the possible bundles of goods where she could potentially spend her money.
And this first bundle is the point 2,2.
This is where she starts off to begin with.
Now, when the price of software increases, she's not going to be able to buy as much software with her money.
But she can still buy the same amount of clothing.
So her new budget constraint in this scenario is going to look like this.
So in this scenario, which is in our problem's Part E, her utility has moved in towards the origin.
And she isn't going to be as happy.
And we can represent this on a utility curve as well.
And we can see from this point on the utility curve the way that I've drawn it, that at this point, she's still consuming the same amount of clothing.
But the amount of software she's consumed has been cut in half.
This is the total effect of the price change.
It's the difference between where she started and where she's ending up without giving her any money to change where she actually is.
So the total effect is just that she's losing one unit of software.
Now, we can break down the total effect in the substitution effects and income effects.
It's important that we really understand conceptually how to define substitution and income effects.
And what we're going to think about is we're going to think about, on this graph, we're going to represent the substitution effect by the movement-- if we were to just have the price change, and we were to give her income so she could stay up at this utility level, the substitution effect is how her bundle changes with that movement.
The income effect is going to be-- since she's poorer because the prices are higher, it's going to be the shift downward.
So I'm going to draw in the scenario that we calculated in Part F right here.
By drawing in this scenario with the higher income level and the price change, we can represent this bundle as the substitution effect.
So what this looks like is she's going to have the same budget constraint, only it's going to be shifted back up.
This is going to be the bundle in Part F. And we can label the bundle 1.32, 2.64.
And this is where it's going to get a little bit tricky.
The substitution effect is just the movement from 2.2 to the same utility curve but at a different point with a different bundle.
So it's when we've given her income to keep her at the same utility level, but we've had the price change.
This is going to be the substitution effect.
I'm going to label it 1.
Now, the income effect is the next movement.
It's the movement that says, well, we don't really give her more income.
She's actually poorer.
It's the movement down from 1.32, 2.64, down to 1, 2.
And then the total effect is just this movement from 2.2 to 1.2.
So I can label the substitution effect 1, the income effect 2, and the total effect 3.
So to calculate the substitution effect, all it's going to be is it's going to be the difference between 2.2 and 1.32 and 2.64.
So in this case, our substitution effect is going to be equal to 0.68, negative 0.64.
And you can see that we actually had an increase in the consumption of clothes for the substitution effect.
And then the income effect, using this equation and what we calculated the substitution effect and the total effect to be, we find that the income effect is equal to 0.32, 0.64.
So what this problem basically had us do is it made us look at the effect of a price change on the consumption decisions of a consumer.
So when a price increases, two things are basically happening.
The first thing that's happening is the price of that product is more, so the person, in most cases, shifts their buying away from that product and towards the less expensive product.
That's what this substitution effect shows us.
The other effect that the person feels is since the price is higher, they can't buy as much stuff with the money that they have.
So they feel poorer, even though they have the same amount of money, because the prices are higher.
That's what this income effect represents.
And the total effect is just the summation of the fact that the price is higher for one good and that they feel poorer.
And so we looked at the total effect broken down into substitution and income effect.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome back to the 14.01 problem-solving videos.
Today we're going to work on Fall 2010 Problem Set 4, Problem Number 3.
And this problem is really going to take us through two scenarios.
We're dealing with producer decisions.
So now instead of dealing with utilities, we're going to be working with cost functions.
And we're going to first go through the short-run scenario.
And then we're going to talk about the long-run scenario and the implications of both of these cases.
Problem Number 3 says, suppose the process of producing corn on a farm is described by the function q equals 8k to the 1/3 times quantity L minus 40 raised to the 2/3, where q is the number of units of corn produced, k is the number of machine hours used, and L is the number of person-hours of labor.
In addition to capital and labor, the farmer needs to pay a $15 transportation fee to deliver corn to downtown.
So the cost can be written as total cost equals 15 times the quantity produced plus the rental cost of capital plus the wage rate times the quantity of labor.
Part A says, suppose in the short-run the machine hours rented are fixed at k equals 8, and its rental rate equals 64, and the wage rate equals 16.
Derive the short-run total cost and the average costs as a function of output level q.
So to start off this problem, we're going to start by working with the short-run scenario.
And typically, the only difference between the short-run scenario and the long-run scenario in economics problems is that in the short-run, the amount of capital that a firm can use is going to be fixed.
It means that because machines are a fixed cost in the short-run, you can't actually change how many machines or how often you use the machines.
So we're going to set that equal to 8 for this scenario.
And we also know that each hour that we use this machine is going to cost us 64.
And we know that for each hour that we're using labor, it's going to cost us 16.
We also know that in addition to the cost of the capital and the cost of the labor, which is represented in our total cost function here, for each unit q that we produce, we have to transport it to market.
So we also have this 15 times q added into our total cost function, which is something that we might not always see in all of our cost functions.
So let's start off by solving for the total cost function.
And to do this, the first thing that we're going to do is we're going to plug in to our production function here what we know the capital is fixed at.
And we're going to solve for labor, or L, in terms of q.
So plugging in for k we're going to be left with this equation.
And from here, we can solve for L in terms of q.
And this is going to be useful for us because what we're going to do is we're going to take this L and we're going to plug it into the total cost function, so that our cost function is no longer in terms of k and L. But it's only going to be in terms of q.
So isolating L in this equation, we're going to have that L equals 40 plus q/16 raised to the 3/2.
So now let's go to our total cost function.
We're going to plug in for k and r. So we know that r is 64 and k is 8.
We're going to plug in for w 16.
And now for L we're going to plug in what we solved for using our production function.
So from this equation when we do the algebraic manipulation, we're going to get the total cost function in terms of only q.
And solving out for this, we find that the total cost-- and I'm going to denote that this is in the short-run with the capital fixed by a little sr as a subscript.
The total cost is going to be equal to 1,152 plus 15q plus 16 times quantity q/16 raised to the 3/2.
Now to find the average cost, all the average cost is it's the total cost divided by q.
So it's per unit, how much on average does the producer have to spend to actually produce one unit?
So to find the average cost, we're going to divide this whole thing through by q.
And we're going to find the average cost in the short-run is going to be equal to 1,152 divided by q plus 15 divided by q/16 raised to the 1/2.
In part B, what we're going to do is we're going to take the total cost function, we're going to plug in a fixed quantity, and we're going to find, what is the actual amount of money that a producer would have to pay to produce a fixed quantity?
Part B says, suppose the farm wants to produce 64 units of corn.
Based on the answer to part A, what is the total short-run cost?
So using our solution from part A, all you have to do now for part B is you're going to plug in for q the number 64.
So for part B, plugging in for q, you're going to find that the total cost in the short-run is going to be equal to 2,240.
Now the more interesting part of this problem is what actually happens when instead of having to fix the capital at 8, what happens when the producer can change the amount of capital that they're producing?
Part D says, in the long-run, the farm can change its capital level by minimizing the cost subject to the production function, derive the cost-minimizing demands for k and L as a function of output q, the wage rates w, and the rental rates of machine r. So now we're going to go back to a similar problem like we saw with consumer theory where we saw the marginal rate of substitution had to be equal to the price ratio.
In this case, we're going to use something called the marginal rate of technical substitution.
Simply put, the marginal rate of technical substitution asks us, how many machines or how many people would we be willing to basically lay off for one additional machine?
And we call that the marginal product of capital, or how much we're actually getting from each unit of capital, divided by the marginal product of labor.
We're going to set that equal to the price of the capital and the price of the labor.
To find these marginal products of capital and the margin marginal product of labor, what we're going to do is we're going to take the derivative of our production function with respect to q, or with respect to k, and with respect to L. And when we do that, we find that the marginal product of capital is going to be equal to.
And we can also solve for the marginal product of labor.
And we're simply going to divide here and we can find that L minus 40 over 2k is going to equal r/w.
And finally, the last thing we're going to do here is we're going to solve for the amount of labor and the amount of capital rented in terms of the other variables.
We're going to plug this back into our production function that we have over here in the middle of the board.
And then we can solve for how much capital and how much labor is demanded in terms of w and in terms of r. So I'll go through this process first for solving for the demand function for capital.
To start off, we're going to get this equation in terms of L. So we have that L is equal to 2rk over w plus 40.
And then we're going to take this L and we're going to plug it back into our production function.
And when we do this, we're going to have that 8k to the 1/3 times quantity 2rk over w raised to the 2/3.
And now we're just going to solve through for rk.
And when we solve through for k, we're going to find that k equals q/8 w over 2r raised to the 2/3.
This is our conditional demand for capital.
And we call it the conditional demand because it's dependent on the price that we're going to pay for labor, the price that we're going to pay for labor, the price we're going to pay for capital, and the quantity that we're going to produce.
Now to do this same process only solving for the conditional demand for labor, you're going to go back to this equation right here.
And instead of solving through for L, you're going to solve through for k. And when you solve through for k, you're going to find that k equals quantity L minus 40 times w over 2r.
You're going to take this k, just like we did for labor you're going to plug it back into the production function.
And you can solve through for the conditional demand for labor.
And when you do that, you're going to find that labor is just going to equal q/8 times quantity 2rw raised to the 1/3 plus 40.
So again, to summarize this problem, we started off with the short-run scenario.
And what we found with the short-run scenario, we were able to plug-in for the capital that was fixed at 8.
We were able to find the total cost and the average cost functions.
Given a fixed quantity that they wanted to produce, we solved for the total cost in the short-run.
And then finally we said, let's let the producer change how much capital they are actually using.
And let's figure out based on letting the wage rate and the rental rate of capital change, let's get a conditional demand that lets those variables change that would let us then solve for the amount of capital and the amount of labor that would be needed to produce a certain amount of quantity.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, welcome back to the 14.01 problem-solving videos.
Today I'm going to be working on Fall 2010 Problem Set 5, Problem Number 4.
And I'm only going to be working the last few sections, E, F, G, and H in this video.
But if you need help with the earlier sections, you should go ahead and should look at PSET Number 4, Problem Number 3.
And in that problem, we work through the production function, and we go through and we find conditional supply and the conditional demand curves.
But this problem is going to have us looking at aggregated supply in a market.
We have to consider what production should be occurring in the market given a set number of firms in the market.
And then we're going to think about the case of perfect competition where firms have to be operating at the absolute best efficiency possible.
And we're going to look at how that affects the production level.
Part E is introduced by saying, consider now that r equals 4 and w equals 1.
And that the market demand for coffee is given by quantity demanded equals 20 minus p. There are eight other companies operating in this market and all companies have the cost structures identical to Sebastian's company, the company that we've been dealing with earlier in the problem.
Part E asks us, what is the aggregate supply in this market?
And if you look back earlier in this problem, the other piece of information that we're going to need is we're going to need this cost function that gives all of the cost in terms of the rental rate of capital or how much it cost to use each machine per hour, the wage rate or how much labor cost per hour, and q, the quantity, that's output by a specific firm.
Now, to find the aggregated supply, what we're first going to find is we're going to first find the supply curve for one firm within this market.
And then we're going to set the demand or the supply curve in terms of quantity in terms of price.
We're going to multiply by eight to aggregate it.
And then we'll have our aggregated supply curve.
But before we do that, we also have to think of a limiting case as well.
When we're representing the costs for a firm, we're going to represent both the marginal cost and the average cost.
The marginal cost is the cost of one single additional unit, while the average cost tells us all the costs including the fixed costs divided by the total that we're producing, what does that look like?
If the price in a market is below the minimum of the average cost of a firm in the market, they're not going to produce in the market.
So if the price is below this critical p star, since the firm, even if they're producing right at the minimum of average cost, they can never recover their cost.
So a firm is only going to produce if this p where the price that's being charged is above the p star, the minimum of average cost.
So we're going to find the supply curve in two cases, one where the price is above this minimum of average cost.
And two, we're going to find it when the price is below that minimum.
Let's start off by finding the marginal cost to get our supply curve.
Taking the marginal cost, the derivative with respect to q.
Or before we can take the marginal cost, sorry, let's plug-in for the variables w and r. We're going to find that our cost curve is given by 4 plus 4q squared.
Now we can find the marginal cost, which will be our supply curve for a single firm.
So in most cases, we know that this supply curve is going to represent the supply for a single firm where marginal cost, the price, is going to equal 8q.
So in most cases, this will be our supply curve for one firm.
Putting it in terms of q, we'll have q equals p/8.
Now since we have eight firms, we multiply by 8.
And in this case, we're going to have the aggregated quantity, which we represent by a capital Q.
So that's the quantity produced by all eight firms in the market.
And that's going to equal price.
So this is one part of the supply curve.
And what we need to know is, what's the critical price at which this will represent the supply curve?
So when the average cost curve crosses the marginal cost curve, that's the minimum of the average cost.
It's always like that for all cost curves for a producer.
So if we set marginal cost equal to average cost, we find this critical p star at which the firm is going to produce at any price above that p star.
So we're going to go back to our marginal cost.
And what we have to do is we have to find the average cost as well to set it equal.
To get the average cost, we're just going to go back up to our cost curve and we're going to divide through the whole thing by q.
So the average cost is going to be 4 divided by q plus 4q.
Now we're going to set average cost and marginal cost equal.
And when we set marginal cost and average cost equal to find the intersection point on our graph, what we're going to find is we're going to find that critical p star is going to be equal to 8.
So Qs is going to equal p for any price that's greater than or equal to 8.
But what happens in the case where the price is less than 8?
In that case, no single firm can make a profit by being in the market.
So for any price less than 8, the production level is going to be equal to 0.
Now we're going to move on to the next case.
Now we're going to take the demand curve that we're given and we're going to calculate the equilibrium price, the aggregate quantity sold, and the quantity sold by each firm, and the economic profit of each firm.
So let's start off in solving this problem, we're going to just assume that the price, the equilibrium price, is going to be greater than or equal to 8.
And as we're solving through the problem, if we end up with a price that's less than 8, then we're just going to go back and we're going to say, OK, there's going to be no production, and we're done.
But let's work with the assumption that we're working with this supply curve to begin with.
All we have to do in this case is we're just going to set the supply curve we just found equal to the demand curve that's given in the problem.
Solving through for p, we're going to find that the price in this market is going to be equal to 10.
And plugging in the price back into the demand curve, you can find that the aggregate quantity is going to be equal to 10.
So the price for each unit is 10 and the aggregate quantity is 10 as well.
Now to find the quantity produced by each of the eight firms, all the firms, since they have identical cost structures, are going to be producing the same amount.
So we're just going to divide this quantity by 8 to find the quantity produced by the individual firms.
So each firm in this case produces 5/4 of a unit.
The last thing that we have to do is we have to calculate the economic profits for each of the single firms.
So the profit is going to be equal to the revenue, which is just price, times the quantity for a single firm.
A big mistake here would be to use the aggregated quantity.
And then you're going to subtract out the cost for each firm.
And we're just going to use the cost function that was given after plugging in w and r. And this is going to represent the economic profit for each of the firms.
And this leads right into the next part of the problem.
It asks us, can this be a long run equilibrium where we have these prices, quantities, and profits?
And why or why not?
And how will the supply side of the market adjust in the long run?
Now when other firms are considering entering the market, the only thing that they're going to consider is they're going to consider, is a firm that's existing in the market currently making profit?
If they're not currently making profit, then the firm that's considering entering would have to have a better technology, a better way of producing at lower cost, to enter the market and be able to actually produce with a profit.
But if there is profit being made, in this case 2.25 for each firm, then more and more firms are going to enter until profits are driven down to 0.
So is this a long run equilibrium?
The answer is no.
And why not?
More firms are going to enter on the supply side until we're driven to equilibrium.
The last part that we're going to do is we're going to do part H. Part H asks us, what is going to be the price in the long run?
How many firms will be present in this market in the long run?
And how much will each firm produce?
Now in the long run, we know that profits are going to be driven down to 0, and that each firm is going to have to be leaner and meaner.
They're going to have to produce at maximum efficiency to be competitive within the market.
So if we go back to the graph that we started with, we can look at, at which point are firms operating optimally?
It's when marginal cost is equal to average cost.
It's at this critical p star that we calculated to be equal to 8 earlier.
So in the long run, all the firms are going to have to be lean and mean.
They're going to have to operate at a very low average cost.
And we're going to calculate how many firms are going to be in the market producing at this point where marginal cost intersects average cost.
So to start off H, we know that p is going to be equal to the minimum of average cost, which we calculated in one of the earlier parts of the problem to be equal to 8.
When we have this equal to 8, we can plug-in to our demand curve that price.
And we can find that the quantity demanded is going to be equal to 12.
Now what we can do now is we can go back and we can look at the individual supply curve for each firm.
And each firm, when we had our individual supply curves that we calculated in the first part of the problem, had a supply function that was equal to q equals p divided by 8.
And in this case, if we know that the price when the firms are operating optimally is equal to 8, then we know that each firm is going to be producing one unit of the good.
So to find the total number of firms, we just have to take this aggregated amount, the total amount that's being produced, and divide through by the amount each firm is producing to find out the number of firms that have to be producing.
So in the long run, we're going to have 12 firms each producing one unit at a price of 8, which is the optimal price where marginal cost is equal to the minimum of the average cost.
So just to summarize what this problem had us look at, we looked at the case where we had instead of just one firm, we had multiple firms operating in a market.
We saw that when we have multiple firms operating that if there's any economic profit that more firms are going to enter until the firms that exist in the market are forced to operate optimally with no economic profit.
I hope that you found this problem helpful.
And go ahead and again, you can do the earlier parts, and you can look to PSET 4 Problem Number 3 for help on those problems.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu
Hi, and welcome back to the 14.01 problem solving videos.
Today we're going to do Fall 2010 problem set 6, problem number 3.
Moldavia is a small country that currently trades freely in the world barley market.
Demand and supply for barley in Moldavia is governed by the following schedules.
The demand is given by quantity demanded equals 4 minus p. The supply is given by the quantity supplied equals p. And the world price of barley is $1 per bushel.
Part A asks us to calculate the free trade equilibrium price and quantity of barley in Moldavia.
How many bushels do they import or export, and on a well-labeled graph depict this equilibrium situation and shade the gains from trade relative to the autarkic no trade equilibrium in Moldavia.
So what we're going to be doing in this problem is we're going to be working with three different functions.
The first is the domestic demand.
This is how much people in Moldavia want barley.
The domestic supply tells us how much the suppliers within the country are willing to supply.
And the international price is telling us if we open up the borders to trade without any tariffs or any barriers for trade.
This is what the equilibrium price, or the new equilibrium price will become.
So let's pretend for a second that we're in autarky where there's no trade at all.
In that case the supply function, which is our domestic supply, and demand function are going to be equal.
And in this case we're going to have a quantity supplied that'll be right here at the equilibrium point.
Now what's going to happen is that we're going to have the international price come in when we open up or borders.
And it's going to function like a price cap.
So instead of the price being way up here when we only have domestic suppliers, we're going to see that the price is going to shift down to p equals 1 to the equilibrium price.
And what's going to happen is consumers are going to be able to consume more out to this point which we'll calculate.
Suppliers domestically will only be willing to supply a quantity at this point.
And that means that all of this in the middle, which in earlier problems we would have thought of as the excess demand.
It's no longer excess.
These consumers can actually get a product.
And the way they're going to get this product is through imports.
And we need to calculate how much people are going to demand, how much the domestic suppliers will produce, and what the difference is made up by the international importers.
So to start off, let's think about what's going to happen when we have free trade.
Well in free trade we're going to start off with our demand function.
And instead of setting this demand function equal to the supply function, we're just going to plug in the international price for the free trade scenario.
So we can see that in free trade people are going to demand three of the products.
Now at the price of one, however, the suppliers aren't going to be willing to supply these three.
So we can calculate how much they'll actually be willing to supply at the price of one.
So just plugging in the price we find the quantity that they're willing to supply is going to be equal to 1.
So that means that the difference here is going to have to be made up by imports.
So importers are going to be equal to 2.
Now compared to the autarky scenario, what we had is we would set the quantity demanded equal to the quantity supplied.
And we would have found that the price would be equal to 2 and the quantity supplied would have been equal to 2.
Now we can represent on the graph in this autarky situation.
I'm going to outline in blue what the total consumer and producer surplus would've looked like.
So we would have had a consumer surplus which would have just been the space below the demand curve up until the equilibrium price of 2.
So this would have been our consumer surplus.
And we would have had a producer surplus up to the price.
It's a triangle up to the price but above the supply curve.
So the total surplus beforehand was this box.
Afterwards what we're going to have is we're going to have a new consumer surplus because more people are accessing the product.
So our new consumer surplus is right here.
Our new producer surplus is this triangle out here.
And looking at our graph, the only difference between the free trade scenario and the autarky scenario is this box right here that I'm shading in.
So you can see that what actually happened here conceptually is that the domestic producers were worse off.
Their producer surplus decreased.
But the consumer surplus increased so much that overall, the total surplus within this country increased by an amount equal to the area of this box, which we could calculate if we needed to.
Let's go ahead and move on to part B.
Part B says the prime minister of Moldavia, sympathetic as always, believes he can help those hurt by free trade in barley relative to the situation and autarky.
He taxes the party that has benefited from free trade equal to the amount per bushel that is the difference between the autarkic price of barley, which we calculated right here, the difference between that price and the free trade price of barley, which is equal to 1.
Furthermore, he rebates the entire government revenue of the tax back to the party harmed by free trade.
In a new, well-labeled diagram show the post-tax equilibrium situation.
Calculate and show the new equilibrium price and quantity of barley in Moldavia, the changes in the quantity of imports or exports, the amount of revenue collected by the prime minister, and who pays the larger burden of this tax, consumers or producers in Moldavia and why.
So there's a lot of things that we need to answer in this problem, but the first step is going to be to really think about how this tax is going to affect the equilibrium that we calculated.
And so this tax is going to be paid by the group that benefits.
So looking at our graph we said that the consumers are benefiting.
We saw that their consumer surplus changed from this triangle to the much larger triangle.
So they're going to be the group that's paying this tax.
So we're going to have a new domestic demand curve for this scenario.
And so we started with our demand curve of qd equals 4 minus p. I'm going to get the inverse demand so that it's p equals 4 minus qd.
And now instead of their inverse demand being equal to this, we have to add in the tax.
So the demand curve is going to shift so that t plus p is equal to 4 minus qd.
So basically when they think about how much they're willing to buy, it's going to be reduced.
The whole demand curve is going to shift down by the amount of the tax.
And so we can represent this graphically.
The demand curve is going to shift down an amount equal to the tax.
I'm going to put dt represent the demand curve after the tax.
And the distance from here, from our initial equilibrium, down to where the demand curve is now is going to be equal to t. And so we can go ahead since we know the tax is going to be equal to the difference between the autarkic price and the free trade price, or 2 minus 1.
We know that t is going to be equal to 1.
So now we have a new equation for our quantity that's demanded.
And we can again set, since we are still open up to trade, we're going to set the price equal to 1 and we can solve for the new quantity demanded.
So in this scenario since we're taxing the group, they're not willing to buy as much.
The quantity that they're demanding has shifted from 3 down to 2.
And how we can represent that is initially we had the international price right here at p equals 1.
So in our initial scenario they were demanding qd and domestic suppliers were supplying at qs.
And now in the new scenario what we're going to see is see that the qt, or the quantity that's demanded with the tax, has shifted down because of the tax shifting the demand curve down as a whole.
Now the last thing, or the other things that this problem asks us is how much tax revenue are they going to receive and how are the imports and the domestic supply going to change.
Well the quantity that's supplied by domestic producers given that the demand is still above 1, the quantity that's going to be supplied in this new scenario is still going to be equal to 1.
And what we're going to see is this reduction in demand is only going to affect the importers.
So before, we had 3 and then minus 1 for the amount that's supplied.
Now instead, the imports are going to be reduced by 1.
And a total tax revenue that's going to be collected is going to be equal to the quantity that's demanded times t. So we have that the total tax revenue in the situation is equal to $2.
So in part A we saw a scenario where we calculated and looked at what quantity was supplied and what price was given when there was no free trade at all.
When it was complete autarky.
Now what we're going to do is we're going to look at the free trade scenario where there's the tax.
And we're going to specifically look at the producer surplus.
We're going to compare the producer surplus in autarky to the producer surplus when there's free trade but they're receiving the $2 rebate from the government.
So part C asks us, are the free trade losers better off or worse off after the rebate than they were under autarky and why.
Let's start off by drawing graphs to represent both of these scenarios.
In autarky what would happen is there would be no international price.
And instead we would just have the equilibrium price right here, with a quantity demanded of 2 and a price of 2 as well.
So in the autarky situation we can calculate the producer surplus as this triangle right here.
To calculate the producer surplus in autarky it's just going to be 1/2 times 2 times 2.
So beforehand, the producer surplus, the area of this triangle, is equal to 2.
Let's look at the scenario after they're open up to free trade but with the producers getting that $2 rebate from the government.
So in this scenario, the international price of one caps the price that the suppliers are going to get.
And so the suppliers in this scenario are also only going to supply a quantity of one as well.
So the producer surplus in this scenario is a smaller triangle.
But the added benefit is that a chunk of the producer surplus, the $2, is also being added in to the producer surplus that would have existed under free trade.
So we're going to calculate the area of this triangle, add in the $2 government rebate to get the new producer surplus in the free trade situation.
So normally the area of that triangle would only be 1/2.
But since we're adding in the government rebate of $2, we find that in the free trade scenario the producer surplus has increased to 2.5.
So since the producer surplus increased to 2.5 we can say that the producers are better off under the free trade system with the caveat that they're receiving a government revenue or a tax.
So to quickly summarize the parts of the problem that we saw.
What we saw here is we looked at the autarkic situation where there's no free trade.
And then we looked at how producers and consumers are affected when borders are open up to free trade without any government intervention.
After that we saw what happens when the government has a policy of taking away from the group of consumers or producers that benefit and giving set revenue back to the other group.
And we compared the producers' surplus before and after the new government intervention.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to do Fall 2010, P Set Six, Problem Number Four.
And for this problem we're going to shift away from what we've usually been talking about, where we're dealing with a straight equilibrium, setting the demand curve equal to the supply curve.
And now we're going to think about what happens when the supplier has market power.
When they're the only competitor in the market, and they can decide how much quantity they want to produce.
And they don't have to worry about other producers coming in and producing.
Problem Number Four-- I'll read through Part A-- states, "A monopolist firm faces the following cost curve.
The cost equal Q squared plus 15, where Q is the output produced, the demand for its product is given by P equals 24 minus Q.
We need to calculate the non-price discriminating consumer surplus, the producer surplus, and the deadweight loss associated with the monopoly."
Now, what this problem's really going to look like-- we can think about it starting with this graph.
Is, instead of producing output to the point of the equilibrium right here, the supplier can actually make more money by saying I'm only going to produce to a point right here, so this is what we're looking for.
We're wondering how much is the supplier actually going to constrict the supply in the market.
And when they constrict this supply, what happens is this small triangle right here becomes the deadweight loss.
This is potential surplus that would have existed when this whole big triangle was the consumer surplus plus producer surplus.
So now nobody's getting that triangle.
But the producer surplus is much bigger than it would have been when it was just the space below the price level to the supply curve.
So the producers are better off.
The consumers are going to be worse off.
And society as a whole-- adding together the producers' and the consumers' surplus-- is going to be worse off.
Now, the way the producers actually make their decision on how much to produce is, when they're moving this line back and forth deciding how much they want to constrict the quantity that they're going to supply, and when they're supplying more, the quantity they're supplying is going to be increasing.
But as they supply more-- since the demand curve is downward sloping-- the price is going to be going down.
And now the way the producer actually makes their production decision is to say, OK, I know I'm going to lose some money if I'm producing more, because the price is going to be falling.
What I want to know is I want to produce as much as I can so that, at the margin, the cost of producing that one additional unit is the same as the revenue that I'm going to be taking in for that additional unit.
The point where I'm producing, and the additional cost of that unit is more than the additional money that I'm taking in, I'm going to stop assuming that there's no competition.
So the monopolist firm is going to set the marginal cost equal to the marginal revenue.
So we have a total cost function, so calculating the marginal cost is pretty straightforward.
I'm just going to take the derivative with respect to Q, and the marginal cost for a monopolist firm is going to be 2Q.
Now, it's tempting when we look at this revenue function-- revenue just being the total quantity I'm producing times the price I'm receiving.
It's tempting to just take the derivative here with respect to Q, and say that marginal revenue is going to be equal to price.
But that's not what the monopolist does.
Because in a competitive situation, we were setting marginal cost equal to P. In the monopolist situation, the monopolist is going to look at this P right here, and they're going to say I know how the consumers are going to respond based on my decision to produce.
So I'm going to replace this P with the demand curve, 24 minus Q.
So I can plan how much I'm producing based on what I know the consumer's response to my production choice is going to be.
So instead of taking the derivative of this function, I'm going to plug-in 24 minus Q and we're going to find the marginal revenue using this function.
When we do this, we find that the marginal revenue is equal to 24 minus 2Q.
And all we have to do now is we have to set the marginal revenue and the marginal cost equal, and we can find the quantity that's going to be produced at the monopolist outcome.
Solving for Q, you find that the quantity is going to be equal to 6.
And then we can solve for the price just by going back to the demand curve that's given on our graph.
The price here in the monopolist case is going to be 18.
So now we can come back to our graph.
We know that the monopolist level of output is going to be 6, so we can label this 0.6.
We know the price that's going to be charged-- which is not the intersection with the supply curve, it's going to be the intersection with the demand curve-- this price is going to be 18.
And on our graph it's going to also be useful to label two more points.
That'll just make it easier for us to calculate consumer surplus, producer surplus, and deadweight loss.
We're going to want to label this point right here so the equilibrium quantity is 8-- and you'll see in a second why I'm labeling that.
And you're also going to want to label where, when the quantity is 6, the intersection with the supply curve.
So when the quantity is 6, you know that the marginal cost curve is given here.
So that means the intersection right here is going to be 12.
And this is going to make our calculations of the area of PS, the area of CS, and the area of DWL just a little bit easier.
Now, to calculate consumer surplus, I'm just going to multiply the height of this triangle right here by the length of the triangle, and I'm going to take one half of that.
So consumer surplus in this situation is going to equal 18.
And now we're going to do the same thing for producer surplus.
We're going to add the area of this rectangle to the area of this triangle at the bottom as well.
So the first term here that's given is the area of the rectangle, and the term here is the area of the triangle.
Adding these together, we're going to find that the producer surplus is 72.
And now, to calculate the deadweight loss you really have two options.
One, you could find the total producer and consumer surplus at equilibrium-- so the area of this large triangle right here.
And you could subtract out the new consumer surplus and the new producer surplus, and you'll be left with only the deadweight loss.
For our purposes, it's going to be a little bit easier to just take the height of the triangle and the length of the base, and to multiply through.
When we do that, we're going to find that the deadweight loss is going to be equal to 6.
And so you can see that the producer surplus is pretty high in this situation.
And so the government's going to come in in our next problem, and they're going to say we have an intervention that might be able to correct this problem that we see in the market.
Part B says, "How does charging the monopolist a specific tax of $8 per unit affect the monopoly optimum, and the welfare of consumers, the monopoly, and society, where society's welfare or surplus includes the tax revenue?"
So, what's basically happening in this new case is we're going to start off with the same sort of problem where the monopolist gets to decide how much they're going to output.
And we're interested in the marginal cost and the marginal revenue.
Now, the marginal revenue is going to be represented by the same equation, and we're going to substitute in for price the demand curve again.
And when we solve through, substituting in for the demand curve and taking the derivative, we're going to find that the marginal revenue is again going to be equal to 24 minus 2Q.
So the marginal revenue hasn't changed at all.
What is going to change is going to be the total cost curve for the monopolist.
So this was the cost curve that we started off with, but now for each unit Q that the monopolist produces, it's going to be taxed at a rate of t. So we can add in the cost of the tax.
And in the next step, I'm going to take the derivative with respect to Q, and I'm going to substitute in for t the price of the tax, or 8.
So now our new marginal cost is equal to 2Q plus 8.
And to solve for our new equilibrium, we're just going to set marginal cost and marginal revenue equal.
And when we do that, we're going to find that Q is equal to 4.
And plugging into the demand curve, you're going to find that the price is equal to 20.
Now in this problem, you could go through and you could go ahead and you could calculate the consumer surplus, the producer surplus, the deadweight loss and the tax revenue, and you could figure out quantitatively how much they've changed.
But we're just going to draw a new graph, and we're going to look at the changes in consumer surplus, producer surplus, and deadweight loss, and make a qualitative assessment of how those quantities have changed.
So on a new axes, I'm going to draw the old supply curve and the demand curve.
Now, what's essentially happening is, since the suppliers know for each unit they're producing they're going to have to pay a tax of 8, the supply curve is essentially shifting up by 8 units.
And I'm representing the new supply curve with an s prime.
So now, instead of the suppliers making their monopolist decision based on this supply curve and the demand curve, they're now making it over here.
And so what's going to happen is they're going to have some new monopolist output.
And in this case, we know the new monopolist output is 4.
We know that the new monopolist price is going to be 20.
And so we can see on this graph that producer surplus is going to be represented by this four-sided figure right here.
We know that the tax revenue is going to be-- since the distance from this point to this point is 8, from 0 to 4-- this is going to represent the tax, this box right here.
And we know that the consumer surplus is going to be this small triangle up top.
Meanwhile, the dead weight loss is anything that's not represented that would have been in our original producer surplus, plus consumer surplus.
So the deadweight loss is this large triangle over here.
So basically what's happened is, compared to our initial case, the government's come in and for society-- since there's less being produced-- we've basically shifted the monopolist quantity over to the left.
This means that, overall, the deadweight loss, this triangle over here, has increased in size.
So if the deadweight loss is increasing, we can say that society is going to be worse off in this situation.
In another case, it's also clear to see that the consumer surplus, since the consumers are paying a higher price for a lower quantity, we can also say that the consumer surplus is going to decrease.
So we can safely say that the consumers are going to be worse off as well.
And then the last interpretation is knowing that in the first case the producers were allowed to make their production decision just given the demand curve and their original supply curve.
Now their production decision also has to take into account the government taking away some of their profits.
If the government is taking some away some of their profits, the producers are necessarily going to have less surplus.
So the producers are going to be worse off as well.
So overall, the only person who might possibly benefit from this policy would be the government.
But overall, the producers, the consumers, and society are going to be worse off.
Now, the last part of this problem is part C. And instead of implementing a tax on the per unit production decision for the producers, now the government's going to consider a different tax policy.
Part C says "How does imposing a tax on profits-- profit after tax equals 1 minus t-- affect the monopoly optimum, and the welfare of consumers, the monopoly, and society?"
Now basically, what's happening in this situation is the government's going to come in.
And they're going to say all right, after you've made your decision on how much to produce, we're going to take a set percentage of the producer's surplus.
So if you get a producer's surplus of this amount, then a certain chunk of it is going to go to Uncle Sam at a tax percentage which we can say is just x percent.
Now that percentage of tax, it doesn't change the fact that the producers want to have as much surplus as possible.
Just because they're going to lose, say, 10% of it because of Uncle Sam, it's not actually going to affect the fact that they want their producer surplus to be as big as possible.
So what happens in this situation is that this after profit tax will not affect the equilibrium at all.
And we're going to be left with the same consumer surplus.
Producer surplus is going to be lower because of the tax, but overall, society is going to be left with the same social welfare.
So really what this problem is looking at through its three parts, we look at the monopolist situation, and we look at how the government can try to adjust with tax policy what's happening in the market.
And what we saw in our second scenario is that when they charge a per unit tax on the producers, societal welfare is going to go down.
But in the third case, when they're just taking a set percentage from the producer surplus, the overall welfare for the society is going to stay the same.
So bundled in this problem we had the monopolist situation, setting marginal cost equal to marginal revenue.
And we also looked at tax implications on a per unit basis, and on a profit basis with a set tax after the production decision is made.
I hope you found this problem helpful.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, and welcome back to the 14.01 problem-solving videos.
Today I'm going to do Fall 2010 Problem Set 7, Problem Number 2.
And we're going to work through both parts A and B to begin with, so I'm going to read the beginning part of the problem.
Suppose Napster's considering selling music via email.
There are two types of users, students and non-students.
Each non-student has an inverse demand function of 200 minus x and each student has an inverse demand function of 160 minus x, where x is the number of songs delivered by email, p is measured in cents.
The marginal cost to Napster of sending an additional song via email is 0.
Suppose Napster can identify all users as either students or non-students.
If Napster offers a fixed number of songs per year to each person, what is the profit maximizing level of songs offered to a student and a non-student.
In other words, what is the equilibrium level of output for each type of person under first degree price discrimination?
Part B is going to ask us, given the equilibrium level of outputs that we've calculated, what is the dollar price charged to students and non-students per year?
Now what this problem is really asking us about, it's asking us about a situation in economics called a two-part tariff.
Now in some cases, when you have two groups of consumers that have differing demand functions, a monopolist can actually capture the vast majority of the consumer surplus.
Or in most cases, all the consumer surplus if he can differentiate between the two groups.
So in this case, Napster is probably going to set up a situation where they can differentiate between students and non-students.
So if they can discriminate by a student by having them enter their school ID or their school email address, then they'd be able to do the situation that we're doing for parts A and B.
The two parts of a two-part tariff, the first part is an access fee that's equal to the consumer surplus.
So you can say to a consumer, sure, you can have this set bundle of songs, this set number of songs.
But you're going to have to pay all of the potential benefit that it would bring you.
So if it's going to bring you 160 units of benefit, we're going to take that from you and we're going to give it to the producers instead.
The second part, in addition to the access fee, the producer has to decide the price per unit when the consumer is consuming the bundle.
And they also have to decide how many songs to bundle.
The way the producer makes this decision is by setting the price per unit equal to the marginal cost.
So they're basically going to say, I'm going to throw in as many songs into this bundle, get you using as many of the songs as possible, or listening to the songs as possible.
But I'm going to only thrown in as many songs until it starts costing me more than it could potentially bring me by capturing your consumer surplus.
So we're going to set the price per unit equal to the marginal cost equal to 0 in this problem.
Now looking at our graph, I've tossed up the demand curves for the student and the non-students.
And what's going to happen in this problem is in the equilibrium case, our supply curve is actually straight along the x-axis because our marginal cost is equal to 0.
So if this was a competitive equilibrium, the producer's surplus would be equal to 0.
But what we're going to do in this case, or what the producers are going to do in this case, is they're going to say I can bundle for the students 160 songs, so that the quantity produced is here.
I'm going to charge you the marginal cost equal to 0 for each song you listen to.
But then I'm going to come back and I'm going to make you pay all of your consumer surplus each year to download those 160 songs.
So again, I went to the marginal cost, which is right at 0.
Then I set the price where it intersects with the demand curve.
And then again, I said that we're going to charge an access fee equal to that area.
The same thing is going to happen to the non-students.
In addition to charging the consumer surplus for A, we're going to make a larger bundle with 200 songs, so that we can capture this additional consumer surplus out here.
And I'm going to break this area up into two pieces called B and C. And you'll see why I'm doing that in just a second.
So for part A, how big are we going to make the bundles for the students and the non-students?
The bundle for the students is going to be equal to 160.
And the bundle for the non-students is going to be equal to 200.
Now for part B, we know that we're going to charge a price per unit equal to 0.
We just need to calculate what the access fee for each of these groups is going to be.
And so the access fee for the group of students who have a song bundle of 160 is just going to be the area of triangle A.
So their fee is going to be consumer surplus of students, which is equal to A, which is going to be equal to 1/2 times 160 squared.
So we're going to charge the consumers a fee that's going to be equal to 12,800.
And a common mistake on this problem would be to just have this answer and put dollars here.
But if you look back on the problem it said that all prices are given in cents.
So we know that the fee for the students is going to be $128.
And now we know that the fee for the non-students is going to be higher because their consumer surplus is greater when they have a bundle of 200.
So the fee for the non-students is just going to be the consumer surplus of the non-students.
And that's going to be the area of A plus B plus C. Which again, that's going to just be equal to 1/2 times 200 squared.
So in this case, our fee for the non-students is going to be equal to $200.
So when the monopolist has a way of telling who's a student and who's a non-student, then they can charge the non-students a higher price because their demand curve is further up and they're willing to pay for more songs within a bundle.
Part B is going to ask us, or part C, moving on to part C. It's going to ask us, what's the maximum price that students are willing to pay for this bundle?
So for the 160 song bundle, we're asked, what's the maximum price students are willing to pay?
Well, if we charge them a price higher than their consumer surplus of $128, then they're just going to drop out of the market.
They're going to say, I'm not going to even buy this bundle.
You're taking away all of the benefit and then an extra one penny.
It's worth it for me to just go someplace else and take my money elsewhere.
So the answer to part C, the maximum price you can charge a consumer in a student for the 160 song bundle is just the $128 that we calculated before.
Part D brings up an interesting implication, however, for the non-students who can look over and see the maximum price that we're charging the students for their bundle.
Part D says, what is the gross consumer surplus that non-students enjoy if they consume 160 songs per year at the price from C?
What is the net consumer surplus?
So gross consumer surplus is just going to be the situation that we had before.
In this case, we're looking at this point here.
We're dealing with the non-students.
They're going to now be consuming the 160 song bundle.
The consumer surplus for this group of students is going to be both A and B.
So working with part C, we know that the gross consumer surplus for the non-students when they're consuming 160 songs is going to be the area of A plus B.
And we know that the net consumer surplus, they're going to be charged a fee equal to A.
So the net consumer surplus is just going to be what we calculate for the gross consumer surplus minus the fee the area of A, 128.
So even though the non-students will get no consumer surplus in net if they go with the 200 song bundle, maybe if they go with the 160 song bundle, they'll be left over with some consumer surplus.
Now to calculate the area of A plus B, all we're going to do is we're going to say that we know the area of A plus B plus C. Now we're just going to subtract out the area of C. And if we were to go back to our demand curve for the non-students, plugging in the quantity of 160 into that demand curve, we would find that it intersects this axis at 40.
So the area of A plus B plus C is going to be equal to 200.
And then the area of C is going to be equal to 1/2 times 40 squared.
And so that means that the gross consumer surplus for a non-student is going to be equal to $192.
So that's the area of A plus B in our diagram.
Now to find the net consumer surplus, we're just going to subtract out 128.
And we're going to find that the net consumer surplus for a non-student who's consuming 160 songs is going to be equal to $64.
So if the monopolist was still to price the same prices that they had before, if they were to make the 160 song bundle $128 and the 200 song bundle $200, in that case, all of the non-students would look at their two choices between the bundles and they'd say, even though this non-student bundle is targeted towards me, I'm better off in net consumer surplus if I go with the bundle that's targeted toward students.
But now the monopolists aren't going to be stupid in this situation.
They're going to go back and they're going to think, well, if you're going to go and buy the non-student bundle, then what I have to do is I have to make sure-- if you're going to go back and buy the student bundle, I have to make sure that the non-student bundle gives you a higher net consumer surplus, so that you're still willing to buy the non-student bundle with 200 songs.
Part B asks us, what is the maximum price Napster can charge for 200 songs per year if it offers 160 songs per year at the highest price that students are willing to pay?
And so simply put, what they can charge for the 200 song bundle is they can charge for an area of A and C. So we're going to subtract out.
We have to leave the non-students with at least enough consumer surplus for the area of B.
We know from our previous problems, we know that the area of A is just going to be equal to 1/2 times 160 squared plus 1/2 times 40 squared.
This is the area of A and this is the area of C. So we can find that the maximum fee is going to be $136.
The other way of calculating this is to just say, well, I know that the non-students are starting off with 200 hours of consumer surplus.
I know that I need to leave them with at least $64 of consumer surplus.
So you can just think intuitively.
You know the maximum price is going to be $136.
And so what we're really going to look at in this situation where they can't discriminate between the students and the non-students, this is the price that they're going to be able to charge for the 200 song bundle.
For the 160 song bundle, they can charge the maximum price the students are willing to pay.
And if we were in a scenario where there were only two customers in this market, one non-student and one student, then we could calculate the overall profits for the monopolist.
And so in this scenario, the overall profits are going to be $264.
The last part of this problem is going to give you another scenario that the monopolist might consider.
The last part of these problems is going to ask you to look at, what happens if instead of designing a student bundle that has 160 songs, what happens if we design a student bundle that instead is only going to have 140 songs?
And again, when you're designing this bundle, you're going to have to plan the prices so that only students will want the 140 song bundle and only non-students are going to want the 200 song bundle.
And the process is going to be the same.
So we're going to stop here on this problem.
And just to summarize what we looked at here, we looked at the consequences and the implications of having a price-discriminating monopolist.
When you can actually discriminate, you can capture all the consumer surplus and you can charge different prices to different groups in the market.
But when you can't discriminate, what you have to do is you have to make packages, bundle them and price them, so that the consumers will discriminate themselves by picking the bundle that gives them the highest net consumer surplus.
I hope you found this part of the problem helpful.
And go ahead and continue through with part I and check your answers against the solution.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome back to the 14.01 problem-solving videos.
Today I'm going to be working on Fall 2010 PSET 8, Problem Number 2.
And we've seen the case with the monopolist where we've only had one producer in a market, and we've seen how that affects consumer surplus, produce surplus, and deadweight loss.
Now we're going to look at the case where we have two firms competing.
And specifically, for the first part, we're going to have the Cournot equilibrium, where they're going to be competing not by setting the price of their product.
Instead they're going to compete by setting how much they're going to produce.
Let's go ahead and read part A of this problem.
Consider a market in which two firms produce a homogeneous product.
Market demand is given by quantity equals 200 minus p. The cost functions for Firm A and Firm B, the total cost for A equals 5qA and the total cost for b is going to equal 1/2 qB squared, respectively.
Find the Cournot equilibrium quantity supplied by each firm.
We're going to graph our results using reaction functions and we're going to find the market price, and then calculate the profits for each firm.
Now in the duopoly model, both firms in the Cournot equilibrium are going to set their quantities at the same time.
But what they're going to do is they're going to know what each other's revenues and costs look like.
So they can know, I know that if I make this move and set this quantity, I know that my competition is going to have this reaction.
Since they can plan for each other's reaction, they can decide how much quantity they're going to produce at the same time and they're going to reach an equilibrium.
What this looks like on our reaction curves, up here we're going to have qA.
And down here is going to be qB.
We're going to graph the reaction function for Firm A first.
And we're now going to graph the reaction function for Firm B.
Now all this tells us is if we're at this point, if we're Firm A and we decide to produce this much, then we know that Firm B's best reaction is going to produce this quantity here, the intersection of how much I'm producing with their reaction curve.
Similarly, if Firm B is going to decide to produce this much, then my best reaction going over to Firm A's reaction function straight across, is going to be to produce this much.
Now since they're both choosing at the same time, neither of the firms can decide to declare a larger amount.
And they have to kind of use their intuition to determine what they're going to produce.
So since they're choosing at the same time, they have to plan for each other's reactions.
They're going to produce at this point where the two reaction curves meet.
And that's what we're going to be calculating.
We're going to be calculating the quantity that Firm A produces and the quantity that Firm B produces when the reaction curves are set equal to each other.
Now to get the reaction curves, we're going to start off with our revenue function for both Firm A and Firm B.
And revenue is just going to be the price times the quantity that either Firm A or Firm B is producing.
Now instead of saying that we are just going to take the marginal revenue according to this P times Q, we're going to plug-in for P from the demand curve 200 minus a disaggregated quantity where we disaggregate into the amount Firm A is producing and the amount Firm B is producing.
We're going to plug this in to the revenue function.
And so just like the monopolist did where they're maximizing by setting marginal revenue equal to marginal cost, Firm A and Firm B we're going to do the same thing.
We're going to set marginal revenue equal to marginal cost.
And then we're going to solve through for qA in terms of the quantity that the other firm is producing.
That's why it's considered a reaction curve because it's in terms of what the other firm is producing.
So solving through for Firm A's marginal revenue, we're going to find that marginal revenue for Firm A is equal to 200 minus 2qA minus qB.
And it makes sense that the more that they're producing, Firm A and Firm B, the lower the revenue that they're going to be taking in.
Now we're going to also calculate the marginal cost for Firm A by taking the derivative with respect to the total cost function.
We're going to find that the marginal cost is going to be equal to 5.
Now we're just going to set the marginal cost and the marginal revenue for Firm A equal.
And we're going to solve through for qA.
When we do that, we're going to find that qA is equal to 97.5 minus 0.5 qB.
And we're going to repeat this exact same process for Firm B.
But when we do it for Firm B, instead of taking the derivative with respect to qA, we're going to take the derivative with respect to qB.
So the marginal revenue for Firm B is going to be equal to 200 minus qA minus 2 qB.
And the marginal cost is just going to be equal to qB.
Again, we're going to set marginal cost and marginal revenue equal to each other.
And now we're going to solve through for qB.
And when we do this, we're going to have Firm B's reaction curve.
And now what we're going to do since we have these two reaction curves, we have a reaction for Firm A and a reaction for Firm B, all we're going to do is we're going to plug-in for this qB, qB's reaction curve.
And when we do that, when we plug-in 66.67 minus 0.33qA, we can solve through for just qA.
Doing this we're going to find that Firm A is going to produce approximately 77 units.
And then taking this 77 and plugging it in to Firm B's reaction function, we can solve for Firm B's production amount.
And we're going to find that Firm B is going to produce approximately 41 units.
Now to find the equilibrium price, we're just going to come back up here to our disaggregated demand function, and we're going to plug-in for qA and qB.
And we can solve through for the price being equal to 82.
Now what we've just done for this problem is we solved on our graph for the intersection of the two reaction functions.
We found that qB is going to be equal to 41 and we found that qA is going to be equal to 77.
So we just calculated the intersection point for the Cournot equilibrium.
Now the last part of this problem asks us to calculate the profits for both the firms.
The profits for Firm A are just going to be price times the quantity that A is producing minus the cost as a function of qA.
So we're just going to take the total revenues minus the total costs.
For Firm A, we're going to find that the total profits are going to be about $5,929.
And doing the same process for Firm B, we can find that the profits for Firm B are going to be equal to approximately $2,521.
Now part B of this problem is going to ask us instead of having this Cournot equilibrium where neither firm can go ahead and produce a higher quantity or move first in the market, we're going to look at something different than the Cournot equilibrium.
We're going to look at the case where one of the firms gets to decide how much they're going to produce before the other firm.
And if you get to decide first, you get to produce a higher quantity and get more of the profits.
Part B says, now suppose that Firm A chooses how much to produce before firm B does.
In this case, Firm A is a Stackelberg leader and B a follower.
We're going to calculate the quantities, the market price, and the profit for each firm.
Now coming over to this side of the board, we see that I'm going to keep Firm B's reaction function the same.
So Firm B is going to be reacting in the same way to Firm A's decision.
Only now, the only difference is when we calculate marginal revenue equal the marginal cost for Firm A, instead of just saying the qB is going to be random, we can't account for it.
We're going to plug-in, we're going to take into account Firm B's reaction when we're maximizing or taking the derivative with respect to qA.
So instead of having qB in here, I'm going to plug-in this reaction function.
So in this case, the equation that I'm going to be maximizing is going to be this one right here.
I'm going to take the derivative with respect to qA to find the marginal revenue for A.
And again, I'm just going to set this equal to the marginal cost, which we found earlier is equal to 5.
And when we set these equal, we can solve through for the quantity that Firm A is going to produce.
And we're going to find just like we predicted that the leader is going to produce more.
In this case, Firm A has increased their production to 96.25.
And then plugging in this quantity in to Firm B's reaction function, we can find that Firm B in this case, is going to produce 34.6.
Now in this case, we can again calculate the price by taking the demand function that we have.
We can take the demand function that we're given in the problem.
Plugging in for qA and qB, we find that the new price in the Stackelberg problem is going to be 69.15.
And again, we can calculate the profits going through the same process of doing total revenue minus total cost.
And we're going to have that the profit for Firm A is going to be equal to about 6,174.
And the profit for Firm B is going to be equal to about 1,794.
And so what we can do here is we can compare the profits that we had in the Stackelberg case to the profits that we had at the start of our problem.
So before we can see that Firm A was not as profitable when they had to choose their quantity at the same time as Firm B.
We can see that their profits have increased.
But we can see that Firm B, their profits have actually decreased because they're a follower in the Stackelberg model.
Now the last thing, and the thought I want to leave you with is, how do we actually interpret this when we look at the reaction functions on our graph?
We're no longer at the point where we're setting the two reaction functions equal.
What's happening now is we're way up here and qA is choosing their production way up here.
And qB is forced to react by choosing their production right here.
And what happens is since they both increased their production or since qA has increased their production and qB has decreased their production, but since production has increased overall, the price has dropped compared to when they were at the Cournot equilibrium.
So total profits have actually dropped as well.
So really what the first two parts of these problems were having us look at, they were looking at two different situations of duopoly where we have two competitors in the market.
The first one they were choosing their outputs at the same time.
And in the second problem, one of the firms had the advantage of getting to choose a higher quantity and making a credible threat that they were going to make that quantity to begin with.
For the last parts of these problems, you're going to go ahead and you can look at what the implications are when we think about what the total quantity is produced in a competitive market aggregating the supplies of these two firms.
And then you can compare the output in the three different scenarios.
But for now, I'm going to leave you here.
Go ahead and finish the rest of the problem, and I hope you found this part helpful.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So let's get started.
So before I get into any poker I want to talk about the mentality I want everyone to try to have when analyzing poker in this class.
So I call it the decision mentality.
I'm going to start with a story.
Who here has heard of credit card roulette?
It's like a game you play at the end of a guy going to restaurants.
So what happens is poker players, they're going to split the bill by instead of everyone paying their own bill, which is annoying.
You have to keep track.
You might have to Venmo people after the exact amount.
And sometimes the waiter or waitress doesn't want to split the bill per person.
So poker players get around this by just picking one person at random to pay the bill.
And we like making this exciting.
So what we do is we ask everyone to put in their credit cards.
And then we pull out the credit cards one at a time.
And if your credit card is pulled out then you're safe.
And the last person in has to pay for the whole table.
So it's a pretty fun game.
Yeah, I think I'm pretty lucky.
The biggest one I lost was in Hong Kong.
I once had to pay around 1,200 USD.
It was a pretty big table.
But overall I'm pretty good at this game.
It's a game of skill for sure.
But sometimes this results in some funny stories with non poker players.
So this is something that happened to some poker players.
So poker pro Matt, he goes to dinner with poker pro Steven.
And that brings Emily, who's a close friend whom he also has a romantic interest in.
So when the bill comes Matt's like OK, I'm going to pay for it Emily.
So he puts in two credit cards.
He's like the second credit card is for Steven-- is for Emily.
And then Steven pays for himself.
So Stephen puts in one credit card.
So they play credit card roulette.
And then Matt, being a very lucky guy, pulls out both of his credit cards before Steven's.
And Steven ends up paying for all three of them.
So now the question is, who should Emily thank.
So who would you thank if you were in Emily's shoes?
Does anyone want to say to thank Steven?
Yeah
Because Steven actually paid for the meal.
So I think it's a totally reasonable thing to do as a reasonable person to thank Steven who actually had to pull out his wallet.
So in this class we want everyone to think in terms of the expected results and not actual results.
So Emily should be thanking Matt because, on average, Matt put in the card for Emily.
And on average Matt is going to be paying for Emily because Matt's going to be paying the one third of the time that Emily would be paying.
But at the time, Emily thanked Steven for her meal and then didn't say anything to Matt.
And then Matt was upset about it and told the entire poker community.
That's how I found out about this story.
So we want to think about in terms of on average what your decision would have whether you would have made money in expectation or on average.
So roughly, the law of large numbers says, over your lifetime, the amount you end up paying for credit card roulette is the same as you would have paid from splitting the bill.
So you know why split the bill?
You might as well just save a lot of time by playing this fun game every single time.
And over your lifetime, the amount you pay in credit card roulette is roughly going to be what you would have paid from splitting.
So all randomness eventually averages out to it's expected value.
That's what this is saying.
So what does eventually mean?
So basically when we say a gamble is very risky I'm not mathematically defining anything here.
But I just want to throw out some intuitive concepts.
So a risky gamble is a gamble where it takes a long time to converge at your expectation.
But the point is, no matter how risky it is, eventually it will get you.
So there's a saying that death, taxes are the two things that eventually get you.
As poker players, we would like to think that three things eventually get you.
It's death, taxes, and the law of large numbers will eventually-- you're going to reach your [INAUDIBLE]..
So here's another hypothetical situation.
So let's say you get off at the wrong bus stop because you were distracted.
And then you were upset yourself you analyze how to not be distracted in the future and get off at the right bus stop.
But then after you get off at the wrong bus stop you find $1,000 on the ground.
And then you immediately, you're no longer upset and you marvel at your riches.
So this is sort of an absurd story.
But situations like this happen all the time in poker.
You're going to make a bad decision but bad decisions still get a good result 49% of the time.
And if you make the right decision you're still going to get a bad result 49% of the time.
So it's very important to analyze your decisions without being biased by the actual outcome that occurred.
So you really want to be obsessed with this self-improvement, analyzing your decisions.
If you made $10,000 in a situation where you could have be $12,000 then that's not good enough.
So I want everybody to think in terms of what's the maximum you could of made and analyzing what's the best decision you could possibly have made in every situation.
And sometimes it's hard because if the result is exactly correlated with the decision then you can just go back and look at the result and know whether you made a good decision or not.
But that's why learning poker can sometimes be very, very hard because you don't have immediate feedback.
You're not sure whether the decision you made is what caused you to make that money or you just got lucky.
So with that being said, now let's talk about some ways to reason about poker hands.
So roughly there's three levels of reasoning of poker hands.
Level one my hand versus your hand.
But by this I mean, you can see what your cards are.
And you look into your opponents eyes and you say, OK, I can tell your cards must be pocket kings or whatever.
Your hand must beat this other hand.
And you played your hand exactly against your opponent's specific hand because you have a soul read on them.
So let's see the example of this.
So we'll watch an episode of Poker After Dark here.
[VIDEO PLAYBACK] [MUSIC PLAYING] - Raise to 1,200.
- I think you would call it this time, Patrick.
- Button raises.
Never anything
Contrary to what Patrick might think Jennifer has a real hand and it just got better.
She's flopped top set.
Patrick flopped a pair of tens with a gutshot straight draw.
- I've got two pair.
- Check.
- Full house.
- I can't beat that.
- I thought you had pocket kings.
- I almost thought I had you.
[END PLAYBACK]
So yes, this is sort of well known poker term from way that in the day.
If you're a Jennifer Tilly hand-- I'm sorry if I'm making fun of her-- but basically she put her opponent on a specific hand.
She looked at Patrick and Antonius and had a feeling that he had pocket kings for some strange reason.
And then what happened was, so she had pocket jacks here, which is a really, really good hand.
It's a full house.
And she just checked the turn and checked the river instead of trying to get Patrick to put more money in because she was so certain Patrick had pocket kings.
Just mathematically speaking, out of all the possible combinations of cards you have, to put your opponent specifically on pocket kings in this example is basically unfounded.
So this gets to level two reasoning.
So level two reasoning is my hand versus your range of hands, versus your probability distribution of hands.
And another name for this is exploitative play.
So let's look at a different hand.
And I'm going to show you how to reason about this hand using level two exploitative reasoning.
So I'm not going to go through the whole hand.
But this is actually a real hand I played in Macau five, six years ago.
So we get to the river and I had ace 10 in this situation.
So the pot is 21,000 roughly.
And my opponent bets 8,000-- sorry, so the pot is 21,000 after my opponent bets 8,000.
So basically the pot was 13,000 and my opponent bet 8,000 making it 21,000.
And I have to decide whether to call with a pretty good hand.
I have a pair of aces here.
But I know my opponent is [?
Rain ?]
[?
Kahn, ?]
who is a very tight player and doesn't really like bluffing.
So I don't think he's really ever bluffing.
If I'm beating him here it's because he's betting a hand that he thinks is good but is actually worse than mine, like ace eight basically.
So I model my opponent's range as ace king through ace eight.
I don't think he has something even stronger, like pocket nines because he would have raised more earlier in the hand.
So let me run through this calculation slowly for those of you seeing pot odd for the first time.
So here basically, I want to think of it like this.
The pot, which includes the 8,000 he just put out, is 21.
We're considering calling for 8,000 so if we lose, if we call and her turns up ace king, which is a better hand, then our net result from this decision is negative $8,000.
If we win and we get our 8,000 back as well as the 21,000 in the pot, so we make, our net is plus 21,000.
So therefore, our win to lose ratio needs to be at least 8,000 to 21,000 in order for calling to be positive expectancy.
So we'll do the exact calculation.
I'll try to do it somewhat slowly to show an example of how to do this calculation.
So essentially what I do is I count the combinations of each hand that exists.
So ace king, there's eight combinations because there's two aces remaining that I haven't seen and there's four kings remaining.
So it's two times four, there's eight combinations.
Same with ace queen, ace jack, and ace eight.
Ace 10 and ace nine, there's only six combinations each because I have the 10, which makes him having ace 10 less likely.
And for ace nine there's a nine on the board.
So ace nine is also slightly less likely.
So ace 10 we actually get half the pot.
So I'm just going to roughly make that equivalent to us winning three of those times and losing three of those times, which is equivalent in expectation.
So basically, if you do this counting, there's 33 combinations that beat us, there's 11 combinations that we beat.
And so our equity is basically our probability of winning, which is equivalently only the fraction of the pot that we own.
So our equity in this case is 25%.
And the pot odds are 21 to 8.
So we need we basically 1 over 3.56.
So 21 to 8 is 2.56 to 1.
We need equity 1 over 3.56 to call, which is 28%, which we don't.
So I folded.
So unfortunately, this calculation is a bit ugly.
You just have to do it a bunch of times and convince yourself that are doing the right calculation.
Basically you got to remember to add one when you're converting the odds ratio to an equity.
So it should be fairly simple math.
So in this case, I folded because I didn't have the desired.
So is there any questions about that?
That was maybe a bit confusing but I don't know of a great way to show the exact conversion from pot odds into equity.
But I just want to point out that they're slightly different.
One is a ratio of when to lose and one is basically a fraction of win over all possible outcomes.
So just don't get confused by this.
That's all I'm saying.
I'm going to quickly interchange between the two throughout the course.
And you should try to get in the habit of being able to quickly complete between the two.
So this is an example of level two reasoning.
And hand reading is about using your opponent's past actions and your knowledge of their tendencies to tweak your probability distribution of over what you think their hand is.
So hand reading is not about pegging your opponent on a specific hand.
It's a marketing message.
Poker players, they wear sunglasses, they have earplugs to prevent people from reading their soul.
It matters a bit, but really, these things, things like I scratched my ear before I raised, it should affect your belief about my cards a very small amount, way less than what Lady Gaga and the media makes it seem like.
That's how you reason about a hand exploitatively.
You build a model for how your opponent behaves and you make the decision that's the most positive expectancy against that model.
And you can go very, very far with level two reasoning.
You can basically build a career out of level two reasoning.
And it's best targeted towards individual opponents with specific tendencies that you're trying to take advantage of.
So this sounds very good.
This sounds like a great way to play poker.
I figured out what my opponent's doing right.
I just figured out the probability distribution and play in a way make the most money from him.
So does anyone have any problems with this type of reasoning?
So I've got some $20 Amazon gift certificates from [?
Acuma ?]
Capital.
Someone can point out potentially something you don't like about this type of reasoning I'll give you this gift certificate
You reveal a lot of information about what your hand is when you think like this
Right.
But what if your model incorporates the fact that they are going to behave based on what they think is in your hand.
Yeah
It might take you a long time to build this model.
And if the players leave your table, or whatever, you spent like-- because before you have a model you can't use this line of reasoning.
So it's expensive to do it
That's a good point.
So first of all, level two reasoning is not easy.
Even though it has flaws, even playing very good level two, poker is not easy.
Building the model can be very hard and you could be wrong.
But that's not really what I'm looking for.
Colin, yeah
Because if your opponent knows your reasoning then he could maybe bet a certain way to push it to 28%, where you'd only call if it's--
Right.
OK. Yeah.
I think that's be good enough.
I' going to give a certificate.
I'll give it to you at the end.
So essentially, the problem is it's this assumption-- the fatal flaw in every plan is the assumption that you know more than your enemy.
Does anyone know where this is from, by the way?
If you do I'll have another $20 gift certificate.
Does anyone know where this phrase is from?
Yeah
Is it from Kasowitz?
No.
Yeah
The Art of War?
No.
No one knows.
I'm going to keep the gift certificate.
That's OK.
So essentially, the problem with this line of reasoning is your opponent does not play according to a fixed, static algorithm.
They're also an intelligent human who's taking this class who's building all this for you and adapting the strategy to beat you.
So if you assume your opponent plays with a fixed strategy, but what if they're doing the same for you?
Now I'm going to get to the third level of reasoning, which I call optimal play.
So let's analyze the exact same hand using level three reasoning.
So don't get me wrong.
Level two reasoning is great.
If you can do level two reasoning very well you can make a ton of money.
But level three reasoning is a completely different way to analyze this hand that can also make you a lot of money.
So level three reasoning I'm going to think like this.
Level three is my range of hands over your range of hands.
So instead of looking at my specific cards and figuring out what my cards are, I'm actually going to also put a probability distribution over myself and myself, to my opponent's eyes, what is the probability distribution of my hands?
And in this specific spot, let's just say, I decide that my range in this spot is ace jack through ace seven.
So not including ace nine.
It's basically ace 10, ace eight, ace seven.
So this is the way to do the calculation.
So let's just assume I know my opponent's propensity to bet 1 over 1.6% of the pot on the river.
So the pot was 13,000 on the river and I know that my opponent's going to bet 8,000.
So I must call with the frequency such that their EV from bluffing is zero.
So that's essentially my goal with optimal play.
So essentially what I'm saying is, let's say they have a really bad hand.
They have like jack 10 for jack high, which is almost certainly going to lose the pot if they don't bet to try to get me to fold.
Then I want to call with the frequency such that with those hands, regardless of whether they bluff, their expectation is zero.
So how do I do that?
I just make my call to fold ratio 1.6 to 1.
Sorry, in all these computations, to make it easier, I'm assuming 1.6 is approximate for the ratio.
It's actually 13 over 8.
It actually turned out to be mostly Fibonacci numbers so the ratio is around 1.6 to 1 for all of them.
[INAUDIBLE] So my call to fold ratio needs to be 1.6 to 1 so that if they have that a bad hand their expectation from bluffing is zero.
It doesn't matter whether they bluff.
So if I do this calculation then ace 10-- sorry, there's a typo-- ace 10 is definitely in the top 61.5% of hands I can have so I need to call.
So I'm not building a probability model for them.
What I'm essentially saying is, if I'm not calling ace 10 here then they can exploit me by bluffing too much and I'm just folding all my hands here.
And they're going to be able to make money off of me.
To prevent this I must call ace 10, which is a 61.5 percentile hand in my range, so they can't exploit me.
So I'm going to make an analogy with RPS.
Rock, paper, scissors, does everyone know how to play this game?
So you either throw a rock, scissors, or paper.
So there's no way that you would not know this game.
I know people, if you grew up in a different background you might call it something else or you might have different rules.
But the rules I'm playing with is rock beats scissors, scissors beats paper, and paper beats rock.
So exploitative play you think like this.
You say, since my opponent just played rock three times in a row I think they're probably not going to play a rock a fourth time in a row.
So I'm going to play scissors because I know I can't be beaten.
Optimal play just says I'm going to memorize a sequence of random bits and always play each a rock, paper, and scissors with probably a third.
So this is an analogy with rock, paper, scissors.
So one question I often get it is, if when you play optimally you're making all your opponent's decisions the same then your opponent essentially is never making a mistake because regardless of what they do it's the same.
Then how do you make money playing optimally?
In rock, paper, scissors that's true.
If my strategy is just play rock, paper, scissors, each with a probability a third I'm never going to beat you more often than I should.
But in poker there's enough opportunities to essentially be inconsistent.
For example, sometimes you will see players fold seven six suited and then, because later they're bored and feel like playing a worse hand, they'll call six five suited in the exact same situation, which is like basically making a strictly dominated strategy.
Or like check raising a circling theory range.
Essentially, a theoretical optimal strategy you will still extract money slowly from even the best players.
Because even the best players in poker right now, there's going to be slight inconsistencies that are not theoretically optimal.
So the optimal strategy, another way to think about it is a Nash equilibrium, if you've heard of the term.
Because the best response to the optimal strategy is the optimal strategy itself.
Whereas the best response to any exploitative strategy is going to be a different strategy.
Whenever you play an exploitative strategy you stand to bee beating by a different exploitive strategy that re-exploits you.
This is a defense optimal plays.
We play each with the same probability.
Optimal play you're indifferent to your opponent's move.
So optimal play, you're making a lot of money when your opponent does something strictly suboptimal.
Whereas in exploitative play you make a lot of money when you're winning the mind games and you lose a lot when you're losing them.
So exploitative play is sort of like I know that you know that I know that you know that I know this.
But you know that I know that you know that I know that you know that I know this.
You get into these mind games.
And if you're good at them then it's pretty good.
If you're playing rock, paper, scissors against a four-year-old child, probably you don't want to play optimally.
Probably you want to look at their face and try and figure out what they're going to do and try to beat them.
But if you're playing rock, paper, scissors against the rock, paper, scissors world champion, which there is one actually, then probably you just want to memorize the sequence of random bits and play each probability a third.
So another good thing about exploitative play is it's intuitive.
We sort of grew up thinking in terms of exploitative play.
I think that's a fair statement.
In most things it's like what do you think might happen?
OK, if that might happen, OK, I'll do this.
But optimal play is sort of a weird mentality thing because it's sort of like I'm just going to analyze what my opponent could potentially do.
So you need an opponent essentially.
And then I'm just going to perform an action that makes it so that my opponent can't really beat me.
So that's enough about those two general concepts.
I'm going to take a short break right here since it's good timing.
So I'm going to continue and we'll get in some actual poker.
But first we're going to play one last game.
We're going to play a game called Who's Taller?
Anyone can join the content for a dollar.
We're not actually having this so don't get out your wallets.
And then the tallest person who joined the game gets the entire pot.
So all the people who join the game, we're going to measure who's the tallest, and then that person gets everyone's money.
So again, let's play.
So everyone, close your eyes so that you can't look in here.
Everyone close your eyes.
No peeking.
No peeking at how tall-- So I want everyone-- close your eyes.
Is everyone's eyes closed?
So put up your hand if you want to join the contest.
Put it nice and high so that I can see if you want to join the content.
So raise your hand high.
It's OK.
There's no embarrassment.
The whole point of poker is where-- OK, cool.
So now everyone open your eyes.
So I think five people joined.
So you guys come down here.
Let's see who-- [LAUGHTER] Let's see who won the pot
[INAUDIBLE]
That's OK.
It's totally OK if you lost.
So I think the four of you.
Sorry, what's your name?
Justin
OK. Justin.
So I think Justin voted one.
So good job Justin, you would have made $3.
All right.
Cool.
So what's the point of this game?
The point of this game is so from a game theoretic point of view no one really should be playing this game.
So this is only known as the k-beauty game from game theory.
So how tall are you Justin?
6'5"
6'5".
So why did you play the game?
Because he thought there's probably someone who's going to be 6'2" who might play the game, right?
But if everyone knew that if you're only 6'2" you shouldn't be playing this game because someone might be 6'5" then you wouldn't play the game because someone else is only going to play the game if they were like 6'8" or something.
So let's say we played this again.
Probably only Justin would play.
Eventually this always devolves into a situation where basically no one is going to join.
Because you know that no matter-- even if you're like seven feet tall, you're LeBron James, you would know that someone would only played this game if they were the tallest person in the world essentially.
So basically poker without blinds, which is the money that's put into the pot at the start of every hand, would essentially be like Who's Taller game.
So when you play poker you want the motivation of every hand to be stealing the blinds, stealing the money that was forced into the pot without the choice of the person.
So you would always fold king king, which is the second best hand in poker pre-flop, if there were no blinds.
Suppose we were to play Who's Taller again.
But I told you that I'm going to force Lee Marie to play.
OK So now we've got a game because now, even if you're not sure whether you're the tallest person in the room, if you're taller then Lee Marie then you have a chance of winning.
So that's essentially why we need blinds to have a game.
So you always want to think how many chips you have in terms of the blinds.
So having $400 in front of you in a game where the blinds are $1 or $2, for our purposes, is equivalent to having $4,000 in front of you in a game where the blinds are $10 and $20.
Because essentially, everything you're wagering is relative to stealing the blinds that were forced into the game at the beginning.
So in both of these situations we say that you have 200 bets, or 200 big blinds, or 200 BB.
So that's how to calculate your stack size.
But what's actually important isn't stack size it's effective stack size.
So effective stack size takes into account the stack sizes of the people remaining in the pot as well.
So I'll give an example.
So in this case, we're the player with the ace jack.
And we decided to go all in.
So what is our exact stack size?
The big blinds is 2,000, the small blind is 1,000.
And we wagered in total 42,000.
So what's our stack size?
20--
21, right.
It's around 21.
It's around 21 big blinds.
But we didn't really risk 21 big blinds here, did we?
Can someone-- OK, I'll give a $20 gift certificate.
How much did we actually risk in some sense.
Yeah
I can't see quite well, but I think 12 and one half big blinds?
Yeah, exactly.
I'll remember it.
I'll put it here.
So we only risked 12 and a half big blinds because everyone in front of us has already folded.
And the true players behind the small blind and the big blind, one of them only has 7 and a half big blinds and the other one only has 12 and a half big blinds.
Can everyone see that?
So even though UTG plus two has 32 and a half big blinds could have theoretically covered us and could have theoretically taken all our money.
But on this hand, because they already folded, we're essentially only risking at most 12 and a half big blinds.
That's the most we can lose.
So we can't be eliminated from the tournament this hand.
So here's another example.
So here you see this guy that I'm going to call Low Jack for now, you see him go all in for 16 big blinds.
So in this case, technically he could lose all his money because we have more than 16 big blinds and we do cover him.
But in reality, I'd say he's probably not on average risking 16 big blinds because most of the people who could call him, other than us, have way less than 16 big blinds.
Does that make sense?
So essentially, there's not like an exact formula for effective stack size.
But you want to sort of think of it in terms of you look at the stack sizes of all the people who could potentially play the hand against you.
And you want to look at roughly how much am I going to be risking in this hand.
That's effective stack size.
Does that make sense to everyone?
So the second thing I want to talk about that's very important is position.
So position is basically where you are at the table relative to the blinds.
So how many players are remaining to compete versus me for the blinds?
Essentially, the fewer players that are left the less strong my hand needs to be attack the blinds.
So I'm going to give names for the positions.
Basically the thing that matters is how far you are from the button.
So in this example with ace jack, when everyone folded to you and you're the dealer, which is also known as the button, essentially to steal the blinds, you just need to get through the two blinds themselves.
So that's essentially two players.
And then if you're in one position earlier, when it was folded to cut off, then he had to deal with three players.
So you want to name everything relative to the dealer essentially.
So we can just quickly go through the names.
They'll get more familiar as time goes on.
So the first person to act, we call them under the gun.
So that's this guy here.
And then we go around the table.
It's under the gun plus one, under the gun plus two.
There can be different names for the same position.
So in this specific hand, where there were only six players at the table, you could have called Low Jack under the gun because under the gun is the first person to act to the left of the big blind.
But essentially, it's more clear to say Low Jack because when you say under the gun you have to say under the gun at a six-handed table.
And then people will know that means you got to get through five hands.
But if you say Low Jack it's very clear.
You have to get through five hands.
So under the gun and then plus two, and then Low Jack.
You can also call under the gun plus two, Low Jack minus one, if you want.
And then High Jack, cut off, button.
Don't ask me where these names came from.
I actually have no idea.
And then small blind, big blind.
So that's position.
And the importance of position is basically the later you are the fewer hands you got to get through to steal the button, to steal the blinds of everyone who's already folded.
So the third thing that matters in a poker game is equity.
So we talked about stack size, we talked about position.
The last thing is actually your cards.
So the equity of your cards is basically your secret height for the Who's Taller game.
So you can think of it like that.
Your cards is like your secret height.
And the probability of you winning the pot, or equivalently, the fraction of the pot you would win once the remaining cards are dealt is called your equity.
So I'm going to give some examples of calculating equity.
So here's an example where we get it all on the turn.
And so I have five four states spades here.
So I'm not in a great position.
But let's count how many outs I have.
So this one I want to just name a river card that would help me win the pot
Seven of spades
Yes
Seven of spades
Seven of spades.
So that gives me a straight and a flush actually.
So let's go along those lines.
So how many spades are there left in the deck?
There should be nine spades
Right, nine spades.
And then how many cards help me make a straight here?
Six more
Seven or a two
Right.
A seven or a two.
So it's six more because there's eight sevens and twos.
But I'm double counting the seven of spades and the two of spades.
So that's nine plus six.
And then I actually have to subtract one more card.
Does anyone see what it is?
Yeah
The queen of spades
Right.
The queen of spades because that's actually a disastrous situation because then if it wasn't all in I would probably put more money in the river thinking I have a flush and lose to a full house.
So it's 14 outs, 17 minus 3.
Our equity is around 32%.
It's 14 over the 44 cards that could still come.
Did that make sense to everyone?
So that's one way of calculating equity.
It's just a very simple probability distribution over the remaining cards in the deck that could come.
And if you're worried, what if someone folded the queen of spades so I shouldn't be counting that?
Essentially, you just want to pretend that the cards that are in the muck, like the cards that were folded by other players, they're essentially irrelevant because they could have folded the queen of spades.
But they could have also folded an irrelevant card, like the jack of hearts or something.
So essentially you just want to all the cards that you haven't seen, it's easier to just assume that they don't affect anything.
In theory they affect things a bit because if you know that the dealer button folded two cards, even if you don't know what they are, they're probably more likely to be really crappy cards, like twos and threes rather than aces because if they had an ace they would have played it.
But I think not worrying about that is fine.
You don't need to worry about it.
Another example of equity calculation.
So it's the exact same example as before.
But in this case, the probability distribution isn't over cards that could come.
All the cards have already come.
The probability distribution is just over my opponent's hand.
So this is the same calculation.
We calculated our equity is 25%.
And that's not a probability distribution over cards to come.
That's a probability distribution over our beliefs of our opponent's hand.
So example three of calculating equity.
So this one you can't really do by hand.
Let's say you get it all in pre-flop, with ace king suited against pocket twos.
And you basically need a calculator.
There are certain websites that help you do this.
And ace king suited against pocket twos, if they don't have a two of your suit is actually a very small favorite which is cool.
But if you have the two of hearts then you're actually a slight underdog.
So just make sure for all the different things equity could mean.
Equity could mean a probability over river cards, probability over the unknown.
A very good calculator in general is PokerStove, which all have lots of examples of in my slides.
And you can download it via this link here.
So this is an example of using PokerStove.
You can put in exact calculations.
Let's say you get it in with two of diamonds, two of spades.
So pocket twos on the flop of 5, 3, 2 versus a range where you know your opponent is going to have a big pair, like pocket jacks plus, you can actually run it and your equity is 85%, which is actually surprisingly low, I think.
Because it seems like you've got three of a kind they've only got a pair.
They have two chances to hit one of two cards.
But there's a lot of random stuff that can happen, like if they have aces they can make a straight, they can make a back door flush, the board could come two more fives, which counterfeits, or three of a kind deuces.
So this is another example of calculating equity.
So this is just a summary of the different situations you might want to calculate equity.
And I recommend you download PokerStove.
But you don't really have to.
And I guess one question you could ask is how do you actually do this at a table?
So essentially, if you do this a lot while you're studying about hands and thinking about poker, eventually-- I've been able to-- you just memorize a lot of situations in general what the probabilities are, or at least what the correct decision is.
So now let's talk about raising two win the blinds and antes.
So the antes, an ante is an extra small bet that each player must put into the pot each hand.
And these sum to around a big blind.
And they come in during the later stages of a tournament.
And they're inexistant in cash games.
I stuck this under the rug in the earlier examples.
So in-- let me go back-- so like in here.
Do you see the 1,200 in the middle?
So actually stuck this under the rug.
But that comes from each player being forced to put in 200, which is 1/10 of the big blind, at the start of every hand.
And if you're playing tournaments then this would usually be the case.
Antes will come in fairly soon.
If you're playing cash games, where you just sit down and you can leave whenever and you sit down and just play for your own play money, then there's usually no antes.
But antes actually make a world of difference in terms of what you want to do.
It's not just the stakes are bigger when there's antes because you don't want to just think of it like there's extra money in the pot every hand.
It's equivalent to the blinds being bigger.
That's not true because you also have to put in an ante.
So if the blinds were proportionally bigger to cover the antes you would have to raise to a bigger size to try to steal the blinds.
But with antes you don't need to raise to any bigger size to try to steal the blinds.
So it's actually very action driving.
It's very exciting.
It makes it so that your basically want to play a very wide range of hands.
And you really just want to be trying to win the blinds as often as possible because winning one pot with the antes is so big.
So this is what it looks like.
So the first thing I'm going to say is if you're going to start playing tournaments tonight is if no one has raised yet you really, you don't want to call.
You want it to raise to give yourself a chance of winning the blinds without seeing a flop.
So I'd say the most common beginner mistake I see beginner poker players make is not raising when no one has raised before.
So in this case, it would be just calling for 2,000 and trying to see a flop.
But the main issue is you're giving the big blind they can just check and see the flop for free.
Whereas if you raise you put them to a decision.
And they might fold, and you might just win the pot for free without having to do anything essentially.
So how much do you raise when I say raise.
The minimum raise size you can do is raising to two big blinds.
However, this is a bit small because you give the blinds fairly good odds to make a profitable call.
Although, it's not even that bad.
So let me just talk in general about raising big versus raising small.
So the advantage of raising small is that you're risking less.
Like let's say you raise and then the next person re-raises, and then the next person goes all in.
So you know probably those two people have pretty good hands and you want to fold.
If you raise to a small size then you can fold and you don't lose that much.
If I raise to five big blinds then I'm losing a lot more.
But what's the benefit of raising big is you give other people worse odds to call.
If I only raise to two big blinds, let's actually do this calculation.
Let's say here, instead of going all in I raise two big blinds, which is 4,000.
What odds am I essentially giving the big blind you call to call?
So this is actually, in this very exact situation, this is a very common mistake I see beginners make, which is you raise to only 4,000.
Yeah
So there's 8,000 that's in the pot and they have to call 2,000, so it's 21?
Right.
So it's 41.
But there's also a small blind.
So there's actually going to be 9,000 essentially in the pot.
Yeah.
So it's going to be 8,800 in.
It's going to be 8,800.
So approximately 9,000.
And they only have to call 2,000 to see the pot.
Does that make sense to everyone?
So the odds are actually 4.5 to 1.
And ace jack off-suit is a great hand.
But there's no hand that ace jack off-suit is more than a 4.5 to 1 favorite against.
So in some sense, basically, even if they have jacks two, or whatever, ace two, think of the best possible case, you're still not doing better than the odds you're giving them.
So on the other hand though, it is really risky to raise more than 2.25.
So a reasonable rule of thumb, I'd say, is to raise to the 2.25 big blinds in tournaments.
I know it's pretty close to do.
If you just raised to two it's probably not that bad.
But roughly speaking, I think this is a reasonable rule of thumb.
Earlier you could try to raise to more.
I think that's sort of customary, although I don't think it's theoretically optimal.
You'll often see players raise to like three blinds, especially when you watch pro players.
They'll raise to bigger.
But the main reason is because they're the better player by a lot and they're just trying to make big pots, which is reasonable.
If you're trying to just play big pots and win big pots then raising to the bigger size is fine.
But I don't think there's any theoretical reason to raise to more than 2.25 big blinds in most situations.
Other than when in cash games, where all players have a lot of money, then it's a bit different because also in cash games you're not worried about risking more because you're not worried about losing out of the tournament.
So the other thing is instead of raising you should just go all in if the effective stack size is 12 big blinds or less.
So recall, the rational for raising big is to prevent others from calling for cheap.
But the rational for raising small is to lose less if we get re-raised and have to fold.
But 12 big blinds is sort of the point where it's small enough that you never really want to fold after committing 2.25 big blinds.
So if I only have 12 big blinds, I raise 2.25, I've only got 9.75 left after.
OK, fine, if I get raises and re-raised maybe I'll fold.
But even if I get re-raised once, if I'm raising in the first place my hand is going to be reasonable.
If it's a Who's Taller game I'm not going to raise in the first place if I'm 5 feet, or something.
So your hand will be reasonable.
So this is beginner mistake number two is being too scared to go all in pre-flop.
So that's why this ace jack hand, I actually cheated a bit.
So technically here, the effective stack size is-- the big blind has 12 and a half big blinds but the small one only has 7 and a half.
So I roughly said, the effective stack size is less than 12, and I just went all in.
So that's definitely a beginner mistake number two in tournaments is to be too scared to go all in pre-flop.
So what your goal essentially should be-- oh, sorry.
I should say the rule is-- so all of the numbers I said assume there's antes.
If there's no antes you should change to 10 big blinds.
So the threshold for going all in should be less because when there's no antes you want to be risking less because the pot is smaller.
So overall, when I talked about position I talked about stack size.
Essentially what I'm trying to get the point across is if you're just starting out, players tend to make all decisions based on their cards.
It's just like, I have a pair of jacks.
I see my pair of jacks on this board.
I have a pair that's pretty good.
And you sort of tend to ignore what the effect of stack size is, how much you're wagering, and position.
But experienced players, the cards actually matter much, much less to the.
In fact, if you're doing optimal play you don't care in some sense what your cards are.
You just care what your range of cards are at that point.
So experienced players, they're willing to raise the blinds with much weaker hands from good positions rather than early positions.
And they're going to risk going all in a lot more frequently with a lot worse hands if their stack size is low.
So I want you to think about that.
So if you're just starting out that's fine.
If you're still trying to figure out whether you have a straight and stuff like that, that's fine.
And it's totally fine if you try to play based just on your cards.
But I want everyone's goal by the end of the class to be able to play based on these other factors more so than your cards.
So that being said, I'm going to give you some concrete suggestions for those of you who might want to start playing tonight of what hands you should be playing from each position.
So most of the tournaments will have nine players per table.
So this is going to be roughly a range I recommend to open.
So these are the range of hands you should be playing from the worst position, so under the gun at a nine-handed table.
So there's eight players left behind you.
You can pick up really good hands.
So it's aces, kings, queens, jacks; ace king off-suited; ace king off-suit; 10s; ace queen suited; ace queen off.
So that's basically the list of hands.
These are only premium hands.
This is like 6.2% of hands, I think.
It's a very premium range.
So note how tight this is.
Because the thing to remember is only the best hand out of nine hands at the table is going to get the pot.
When there's nine hands, even though the average hand is really bad the best hand out of nine hands is always going to be pretty good.
So to even think that you have a chance you need to start out with a very premium hold 'em.
Roughly, what do you add as you go around the table?
So for the second position-- I'm going to post all these slides before the tournament starts, by the way.
So you don't feel that you scramble to write all this down.
I'm going to post it.
From the second position, I put in black the hands that I would open from the previous position.
And I've put in red the hands that I would open from that position in addition.
So second position there's one less player.
OK.
I'll gamble with sevens, ace, jack, king, queen, ace 10 suited.
So we'll get around the table and then the range will slowly increase.
Also note that these are very conservative ranges.
If you watch high stakes poker, probably the players will open a lot more hands than what I'm recommending here.
But my general experience has been when you're just starting out to play poker it's much easier to err on the side of being too tight than being too loose.
Because it's in some sense a lot easier to play pocket aces than seven five suited.
With pocket aces you're always going to have a pair of aces or better.
You just back that bet.
And then with seven five suite you need to be able to bluff very carefully, you need to sometimes be able to try to get value when you just hit a pair of sevens.
And it's just a lot harder.
So the ranges I'm suggesting are very tight by most poker standards.
By tight I mean conservative.
But I think that's the right side to err on when you're just starting.
So these are the ranges.
As we get to the later positions, so this is Low Jack, we're opening more hands now.
Any two suited Broadways.
So that's hands like jack 10 suited, queen 10 suited.
It's basically any two cards that are the same suit and both 10 or higher.
So maybe I should just run through this quickly.
So why is having your cards be suited good?
You get a flush
Right.
So if it suited you're more likely to make a flush.
But if have two unsuited cards then there's two different flushes you could get.
If you have queens or diamonds, 10 of spades is going to make a diamond flush or a spade flush.
Essentially, that argument is wrong because there's two reasons why it's wrong.
One is it's not more of a play because you need four of the suit to make a flush.
And two is, when you make the flush it's also more obvious you have the flush.
like If you only have one diamond, even if it's the ace of diamonds, once there's four diamonds on the board it's way more likely that you have a flush.
So you're going to get paid off less.
Whereas if you have two diamonds in your hand and there's only three diamonds on the board it's less susceptible that you have a flush.
So high jack is three to the button.
So this is what I would recommend.
So any pair, pairs are pretty good.
Any suited ace, so suited ace means any card with any hand with an ace and another card of the same suit.
Any suited connector.
By that I mean hands like basically two cards that are next to each other and also the same suit, like 10, 9-- 10 of spades, nine of spades.
Or any two unsuited Broadway cards.
So I do use a lot of terminology here.
So yeah, please stop me if something is unclear.
You can google most of these, I think.
And suited connectors are good.
So like a hand like seven, six suited is often better than a hand like 10, 6 suited, even though the trend is bigger than the six because ten is sort of a small enough card where it's not that relevant, the fact that it's a big card.
But the fact that seven, six is connected and can hit a lot more straights is very relevant.
So we're if we're cut off I'm just going to show you on PokerStove because there's too many to list.
But the thing I want you to know is the percentages.
30%.
So remember, under the gun I recommended playing 6%.
So we've multiplied how bad our hand could be for us to play by five.
And I think this is what I really want you to try to do.
And I think what a lot of new players don't do enough is they always play the same cards regardless of their position.
And then even crazier, on the button, if everyone in front of you folds, I recommend them playing about 55% of hands.
This is huge.
I want you to play jack three suited.
Who thinks jack three suited is a good hand?
Or king four off-suit, or queen six off-suit.
What hands do we play from the small blind if it's folded to you?
So let's compare opening from the small blind to opening from the button.
So for small blind it's a bit different now.
So I should mention this because when you're raising from the button, from the dealer, if you get called you get to act last post-flop.
So I'm going to talk more about this in future lectures.
But this is basically called having position post-flop.
And having position is basically good.
Yeah
For all these hands, are you assuming that everyone's folding behind you?
Yes, yes.
If people haven't folded, essentially, I'll talk more about this in future classes but essentially what you need to do if-- so let let's say this guy raised and you're the button.
Essentially you need to consider what his range is and calculate your equity against his range, which you can do--
The calculation is completely different
Right.
So the calculation is essentially completely different.
So essentially the first guy who raised determines what the barrier to entry to the pot is.
Because essentially the first guy that raises, if it's from under the gun, he's telling you, my range is pocket eights plus.
Even if he's sort of loose it's still going to be pocket fours plus and queen 10 suited plus.
Basically it's saying, if you're playing eight six suited then you're basically in idiot, because my range is way better than that.
So essentially the first guy who plays this hand sort of sets the tone on how good your hand needs to be.
Thanks for that question.
So I'll talk more about this in a future lecture.
But essentially that's what you need to do.
So from the small blind let's compare opening the small blind from opening the button.
So the issue with opening the small blind is if the big blind calls you actually don't have position post-flop.
The big blind gets to act after you post-flop.
Pot.
So it's actually a lot worse in some sense than opening from the button because you don't get you act last.
But it is better in the sense that there's one fewer person to get through.
And also, you have less to wager, which is actually really relevant because from the button, if the blinds are 2,000 or 1,000 or OK, let's just dictate it.
So if the blinds are 20, 40, if you want to steal the lines from the button and you want to do 2.25 you need to put in 90.
You raise me 90 and you need to wager 90.
But if you're raising from the small blind you only need to put in 70 more.
So you're actually paying a smaller price to try to steal the blinds.
But the fact that you're out of position is a very important negative.
So I'd say all these factors balance out.
And you can roughly play the same range of hands from the small blind as you would from the big blind.
I think that's a reasonable rule of thumb.
But the fact that you're out of position hurts a lot less as stacks get shallower, essentially as effective stack sizes get a lot smaller.
So you could really drastically increase this range if you only have 10 big blinds from the small blind.
So let me just give a few caveats about this.
So actually, I'll leave it to you guys.
If someone comes up with a very good complaint about these recommendations I'm giving you I'll give up the last $20 gift card.
So what are some problems?
Normally I get lots of complaints in this section.
You already got one but you can answer.
Yeah
They're very predictable so it's unusable
Good.
That's a very good point.
A lot of people ask me, why would you follow these?
If I follow these everyone knows what I'm doing.
Well the point is, for most of these, even this range, it sort of encompasses enough hands.
So this is where I'm talking about optimal balance play.
I think even though humans haven't solved poker, I think roughly, if you want to ask, I would guess that roughly the optimal range of hands you open from this position is something like this.
It's going to be looser.
But it's something roughly like this.
Probably you want to probabilistic play some smaller cards just so when the flop comes 2, 2, 4, you can potentially have a two.
But essentially you're not doing bad here because if the flop comes three small cards then you were going to be doing pretty well with all your big pairs.
And if the flop comes the big cards you've still got ace king, ace queen, ace jack in your hand.
But not that is a good complaint.
So in theory, it's in theory optimal to play seven seven some fraction of the time, six six some fraction of the time, and two two some fraction of the time.
But that's making things really complicated.
I don't want to add two two to the list because if you play two two every single time then that's way too loose.
You're just spewing money by raising with two two against this many players behind.
So ideally maybe with two twos you should do a random number generator and 10% of the time you play two two.
That would probably be theoretically optimal.
OK, good.
That would have gotten this.
But OK, I'll give it to-- Yeah
I think another problem with playing really tight all the time is that you are also very easily bullied around the table.
If someone bets and you really want to play, you really want to stick to your tight range, you'll fold and you'll maybe miss out on something that you shouldn't have depending on your position, and things like that
OK. Good.
So yes, these are pretty tight in that it does give the other people incentive to raise a lot because everyone's going to be folding a lot because you are just waiting for these really good hands.
OK. That's good.
Are there any other questions about these suggestions I gave you guys?
Or are there other potential complaints?
I'm going to give this gift certificate to you unless someone thinks that they have another complaint.
So I'll give this to you at the end.
This is one thing that I didn't really address.
I talked about going all in as well as just raising.
So how do these ranges differ?
The answer essentially is they don't really differ.
So let's say, here, these are the hands I told you to play.
Let's say you were sort of small enough, you only had 12 big blinds with antes and you were supposed to go all in instead of raise, which of these hands do you go all in with?
It's roughly the same range.
I think that's a decent rule of thumb.
If you just go all in with the same range you're not going to be doing badly.
But when the effective stack size is much smaller, like let's say you only have three big blinds, then you can drastically increase this range.
So when your effective stack size is actually say 10, or let's say your effective stack size is five big blinds, intuitively you might think, I can wager a lot, I can wager a lot wider range of hands.
So here's the argument.
The argument is I'm only risking five big blinds.
In the other case I was risking 12 big blinds.
I should be way more aggressive with five big blinds than 12 big blinds.
So that's the argument.
But the reason why that argument is wrong is because with five big blinds you're going to get called every time.
With five big blinds you can't just go all in with seven two off-suit because your opponent has such good odds to call, they're going to call you.
Whereas with 12 big blinds, even though you're risking more you also have more chances of getting everyone to fold.
So even though it seems like you're risking way more, in some sense you're not.
Because the more chips you are risking the more likely it is that everyone else folds.
Does that argument make sense to everyone?
So make sure you don't get tricked by that because it is easy to get tricked.
But the fact that when you're risking more you're getting people to fold more means you're actually not risking as much as you think you are.
And that's a common fallacy.
So we're near the last part of the class.
So are there any other questions?
I know some of you probably want to start playing tonight, and I definitely did not address everything you need.
But are there any other questions roughly, in terms of opening and going all in that you might want to know before playing tonight?
Because I do think we have maybe five extra minutes.
So I can answer some questions.
Does everyone not know what the term "bluff" means?
I said bluff a bunch of times but I realized I never explained.
So bluffing essentially means you have a bad hand and because you have such a bad hand you know the only way of winning that is to get your opponent fold.
So basically you bet big to hope that your opponent folds.
If you have a better hand, like an average hand, often you actually don't want to be betting because there's no point.
Because when you bet you're essentially just getting adversely selected, where your opponent calls you when their hand is better and you lose more money to a better hand.
And your opponent folds when their hand is worse so you're not making any money from a worse hand.
And then with your best hands, you want to bet to try to get your opponent to call and increase the amount that you win.
And we call this value betting.
So I'm going to get more into this in future lechers.
But essentially in poker, the paradigm you want to play is you want to value bet your very best hands and then you want to basically check and try to get to show down, like try to show your hand on the river with your medium hands.
And also bet with your bad hands to try to get your opponent to fold better hands.
I'm going to get to the last part now.
So quick test.
So here what would you do?
It's folded to you.
You're in the high jack.
You have ace, five of clubs.
So someone want to suggest what would you do?
What would your play be?
Call
Bet
Raise
Raise.
OK, good.
Raise.
So not call.
That was mistake number one to not do.
You always want to raise the blinds if it's folded.
But it's fine, it's fine.
That's fine.
We're all here to learn about these hands.
Now should I go all in or should I just raise to 450, which is 2.25?
I should go all in.
OK.
I think it's not terrible if you just raise to 450.
But I said if it's 10 big blinds-- there's no antes so you need 10 or less.
And you have 10, so going all in I think is fine.
So we decide to go all in.
So there are Nash calculators.
They're complicated.
But if you run this through a Nash equilibrium calculator this is roughly what you should be going all in with.
So we go all in.
So this range is a bit different than what I showed you before.
This is an actual Nash calculation just not just my recommendation.
It's going to be wider than what you saw before but that's fine.
So now you're this guy.
You're seat nine.
So I'm going to show you how you do a quick calculation of what equity you need to call here.
So what equity you need to call here.
A straight up pot odds calculation says you're getting 23 to 20 or 1.15 to 1.
So roughly you need 46.5% equity to call as this player.
But there's two players behind.
There is a graphics glitch because seat one has not folded.
Seat one and seat two both has cards.
So you actually need more than 46.5% equity because the calculation of 46.5% assumes seat one and seat two can't wake up with pocket aces.
PokerStove is very good for this.
And you can do this right now.
I've taught you everything you need to do these calculations.
And PokerStove is also easy to figure out how to use.
You can essentially just plug in.
You look at pocket fives, it has 48%.
You need 46.5%.
And I said you need a bit more than 46.5 due to the fact that there's two players behind so pocket fives, it's good enough.
Well, I'll call ace nine suited.
I'll call.
So I'm just showing you some calculations on PokerStove.
It really is a really good tool.
So you can do some calculations.
So basically they call.
And the hands I showed you here is actually sort of the worse hands that it's plus CV to call with.
Pocket fives, ace nines suited plus.
So aces nines suited plus means ace nine suited, ace 10 suited, ace jack suited, all the hands that are strictly better than ace nine suited.
Ace 10 off-suit, king queen suited.
As the last player here, do you call with king queen suited?
It's just a rough guess.
How many of you intuitively would call with king queen suited?
Two players have already gone all in.
So most of people would fold.
Oh, you guys would call.
So it turns out, at least according to the Nash calculator-- So here you calculate the pot odds.
And this is an exact calculation because there's no more players behind.
So just calculate, do I have 29.5% accurate?
Basically if I assume that player one and player two are playing according to Nash equilibrium or optimal ranges, you actually have way more than enough.
You actually have 34% where you only needed 25%.
So this is a trick question I set up.
The trick was king queen suited doesn't seem like a good hand.
But against two other hands it's actually relatively a very good hand.
Against one other hand, they could just have a pair and you're behind.
They could have an ace most of the time and you're behind.
But against two other hands you actually have really good equity because when you hit a king or queen you have the big pair.
You can also hit straights and flushes with king queens suited.
And your equity against two other hands is very good.
So you have way more than enough so they call.
So we're going to get in this all in freefall.
So who are the people I gave these cards to?
You guys can come up to the front and I'm going to hand you your cards.
The people who I gave these gift certificates to, I said
Do you want me to stay up here?
Yeah.
Stay up here.
So what I want you guys to do-- these are the hands people went all in with.
Who was the first person to answer?
OK.
So you get to pick.
Which hand do you want?
I'll go with jack jack
OK.
So Colin is going to jack jack.
Sorry, what's your name?
Kevin
Kevin.
Kevin, which hand do you want?
So he took jack jack.
Which of these remaining-- you were second, right?
Yeah, you were--
I was last
OK.
Which hand do you want?
I want king queen
OK.
So he's going to take king queen.
So you're going to take ace five plus.
This is sort of mean but I want to show you guys this how poker works.
When you've already won, you've won $20.
You guys are going to put in your hand and whoever wins this hand is going to get all $60.
[LAUGHTER] We're going to go for it.
So this is the free flow of equities.
So it is pre-determined.
So it looks like you guys did very well.
You guys picked correctly.
You got to pick first.
You picked the best hand with equity of 34.73.
You got 32.95.
You're only a bit behind with 31.1.
So the flop is 6, 7, 8.
So let's look at the probabilities now.
Jacks is still ahead, ace five has picked up a lot of outs.
King queen of diamonds is not looking too good but they did pick up one diamond.
Notice the nine of hearts.
So, sorry.
Like Queens.
I'm sorry.
So you're out of the running
He needs a 10
2 to 1 to cheer.
Is there any card you're hoping for?
What are you hoping for?
I think a 10
A 10 or a heart
So you want to be screaming.
I'm going to press it on the count of three.
You want to be screaming 10 or higher.
And you want to be screaming--
Anything else
So I guess you got all three of us.
There's going to be more of these good card stories in class.
So participate in class, you're going to get gift certificates courtesy of [?
Acuma ?]
Capital.
All right.
Thank you guys.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, guys.
I'm going to get started.
If you haven't done attendance yet, you can do it at the break or afterwards.
I think it will work.
OK, so just a few quick announcements.
So, homework 1 is posted and due Friday.
I have it up on here.
Are there any questions about homework 1, or anything about the procedure?
So, Lee Marie's going to have office hours shortly after class, 4:30 to 5:30 in E51-145.
So if you have any questions, you can ask her as well.
Or you can ask me now if you think they're applicable to the general class.
OK, cool.
So, OK, so I'm going to proceed then.
So I hope-- so most of you I think have started playing in the online tournaments.
I hope you guys-- if you haven't, I hope you plan on starting soon.
Yeah, I hope they've been fun so far.
OK, so today, I'm going to focus more specifically on poker itself.
I know last class, I talked about a lot of general concepts on decision making and how to think about exploiting your opponent versus playing within a Nash equilibrium way.
Today, I'm going to focus mostly on poker, and try to give a lot of hand examples.
And I'm going to introduce post-flop play, which I didn't really talk about last time-- so basically, what happens after you see the flop.
OK, so I'm going to just start with an example hand.
So, we get ace-queen offsuit in hijack minus 1, so I think I called it lojack in the earlier slide-- in last class, but basically, it's four away from the button.
So it's how many people-- the thing that matters is how many people behind have cards and could be left to act.
So we raise a bit bigger than 2.25 big blinds, but I'm not going to nitpick that for now.
So, we're a bit deeper stacked, so we want to build a bigger pot because we think we have an advantage, I guess.
OK, so, it's folded to both blinds.
And both blinds call OK?
So, the flop comes on 9-8-deuce.
So we sort of missed the flop.
And the small blind checks, and the big blind checks.
OK, so what happened is clear so far?
OK, so now, we have a decision here, essentially.
So first, I want to talk about checking to the preflop aggressor.
So this is sort of-- this is sort of a conventional way to play.
So, they both checked, so we were the preflop aggressor.
So in this case, I mean by preflop aggressor, the person who put in the last raise preflop.
And the reason why this is significant is because in some sense, we have a stronger range of hands than they do because we could potentially have pocket aces, which is the best hand.
And it's going to be good hand on pretty much any flop.
Whereas it's much harder-- it's much more unlikely for them to have pocket aces, because if they had pocket aces, it would be a good play, and a tempting play to re-raise preflop.
So basically, they're sort of just recognizing the fact that on average, we're going to have a better hand than them.
So they're going to check us and let us bet because in general, on average, they would prefer if the pot was smaller.
And we would prefer if the pot was bigger because our range is better.
Yeah, so they both check to us.
And right, so, like let's say they had a pretty good hand.
Like, let's say small blind had 10, 9 suited or something, which is the top pair-- so the highest pair on this board.
That's a pretty good hand.
But that's still a hand that loses to pocket aces.
And there's still the risk if he bets out with 10, 9, we could just raise.
And then he has to be worried, you know, do we have pocket aces?
There's basically always going to be this worry of pocket aces and pocket kings, and just big pairs that he has to worry about that we have to worry about less.
Occasionally, someone there probably should slow play pocket aces preflop and just call.
But most of the time, it will be a better idea for them to re-raise to get more money in preflop.
So that's why they both check to us.
So now, we have the opportunity to bet or to check back.
So, let me first check what-- how big to bet if we decide to bet.
So here, we bet 200 into 375, which is about half the pot.
And so, between betting big and betting small, it's sort of the same principles I talked about last class between raising big and raising small preflop.
So the advantage of betting small-- so, the disadvantage of betting small is that you give your opponent better odds to call.
So like let's say they have, I don't know, like Jack, 10 here, a hand that's losing to you.
They have the better odds to call and a cheaper price to call to see another card to try and beat you.
But on the other hand, our hand isn't amazing.
Our hand isn't good here, essentially.
We've missed the flop.
We haven't paired our hand.
And if we get raised, we're probably going to have to fold.
And the more we bet, the more we lose when we get raised.
So it's the same sweet spot principles.
If you bet too small, you get called too often by worse hands that just have good odds to call.
And if you get too big, you're just losing a lot when you get raised.
So a good rule of thumb is to bet half the pot on the flop and then like a slightly bigger percentage on the turn, like 60%, and then an even bigger percentage on the river, like 70% or something.
Something like that is about reasonable.
You could bet up to like 100% of the pot on the river.
So I'd say just somewhere between 50% to 100% of the pot is a good amount to bet.
And the fraction should increase as you get closer to the river.
So this might seem slightly contradictory, because what I said last class was you were able to-- last class, I said you want to give your opponent-- you don't want to give your opponent great odds.
But in some sense, if we get to the river, right, all the cards have come out.
There's no more unknown cards.
So either they had the better hand or we have the better hand.
But there's going to be no uncertainty in what cards could still come.
So in some sense, you might think it's more intuitive to bet smaller on the turn and river and bigger on the flop, because on the flop, they have more chances to outdraw us, whereas on the turn and river, they have fewer.
But let me just find the-- all right, now I don't have to stand there.
OK, so this might seem contradictory, right?
So I'm saying that bigger on the river, but on the river, they actually have in some sense-- on the river, there is-- it's more determined whether their hand is good or not.
So why is this?
Why do we want to bet bigger on the turn than the flop, whereas on the flop, any hand sort of has closer to 50% equity, because anything can still happen with two cards to come?
So the main reason is because when you bet the flop, they're not just deciding to call your flop bet for the equity there.
They also potentially have to call a turn bet and a river bet.
And essentially, this factor-- the fact that they have to, you can threaten to bet again on the turn and bet again on the river, and they have to call all these bets, is a bigger factor than the fact that they have-- there's slightly better odds to call on the flop.
OK, so another important thing to say is when I said bet between 50% to 100% of the pot, I mean I think that's-- it's very important to bet at least some fraction of the pot, especially when they're checking you.
Because in some sense, betting small is strictly worse than checking.
Like in this case, you get 50, which I guess isn't nothing.
But imagine you bet $1 here.
The issue with this is the rules of betting safe, right, if anyone bets any positive amount, then the guys who checked now have a chance to check-raise.
So if you're betting small, which is almost the same as checking, all you're doing is giving your opponent a free option.
You're allowing them the opportunity to say, OK, I couldn't just-- like if you check, you see the turn, and the turn comes.
But by betting, now they have the opportunity-- a second chance to bet.
Yeah?
How does this change if you have the nuts?
Should you bet?
Like, you want calls at that point, right?
Right, right.
Exactly.
Yes.
So, I'm going to get to that, yes.
So if you have a good hand, yeah, you'll bet too because you want calls.
Right now, I'm just talking about a general strategy to play with your entire range of hands.
And in general, I guess that's all I'm trying to encourage people to think about poker in this class.
Like instead of thinking about what cards do I have and what cards do my opponent have, I try to talk in terms of what is the probability distribution over my opponent's hands, and what's the probability distribution over my hands in his eyes, essentially.
So I'm just talking about a general scenario, yeah.
OK, cool.
So, right.
So you never want to bet too small, basically.
It's better just to not bet.
There's no point to bet.
Really, it's not, because you're giving away free option.
OK, so yeah.
So this is what I just said, essentially.
Because remember, just because they checked doesn't mean they have bad hands.
They're often checking regardless for the reasons I described earlier.
OK, so in this situation with ace-queen offsuit, I think both betting 200 and checking are reasonable plays.
And let's analyze the advantages and disadvantages of both.
So the advantage of betting 200 is that you don't give some garbage hand like king-jack a chance to outdraw you.
So in this situation, if we bet 200, probably if someone had King, Jack, they'll probably just fold because it's almost one of the worst hands on this board.
And if we bet 200, then they're going to fold.
And then we could just win the pot right there.
If we don't bet, maybe the turn will come a king or a jack and they could beat us.
So what's the advantage of checking?
The advantage of checking is we get to see the turn card for sure.
We can't get check-raised.
Like right now if we bet we get check-raised, we probably have to fold with no pair.
But if we check, we might see a queen or an ace on the turn that should be great.
And another advantage of checking is, in some sense, when you bet in this spot, you're only called by hands that beat you.
You're going to get king-jack to fold, which is still fine, because they have six outs.
But you are ahead of them, so it's not the end of the world to just check him.
If the turn comes like a deuce, you're still ahead.
And if it checks all the way to showdown, like all the way to the point where you have to flip over your cards, you're going to beat king-jack.
Whereas when you bet, you're essentially getting all the hands you're ahead of to fold.
Like if they have ace-7, which you're also ahead of, they're probably going to fold.
And pretty much all the hands that beat you probably aren't going to fold.
Like, maybe someone will fold pocket 3's or something, but for the most part, you're going to get called by a 9.
You're going to get called by an 8.
So it's this idea sort of adverse selection.
You're getting called only by hands that beat you, and you're just folding at everything else.
So overall, I'd say these about balance out and you can-- I think betting or checking are both fine.
OK, so now let's suppose it's the same spot.
And we had king-queen.
We had king-queen instead of ace-queen.
So pretend all the action was the same.
All the preflop action, the flop action-- everything was same, except just our cards change.
So in this case, if we do the same comparison again, we'll see that betting has a bigger advantage now, because we can get hands like ace-jack and ace-7 to fold.
Those hands previously we were beating with ace-queen.
But now, we don't want to risk getting a situation where it's checked to showdown and everyone flips over their cards, and we lose to ace-jack or ace-7.
So I'd say in this case, now with king-queen instead of ace-queen, betting has a lot more advantage because we're getting better hands to fold.
OK, and this is what you were talking about.
Yeah, what if you have the nuts?
So in this case, if-- so suppose now we change our king to a queen.
So we actually have a really good hand.
We have pocket queens, which is an overpair, a pair above any card on the flop.
So now, let's look at the advantages of betting versus checking.
So now, one advantage of betting that we never had before because we always had bad hands is, now we can get worse hands to call us.
If they have 10-9 or ace-8 or pocket 5's, they're probably going to call.
And we're beating them, so we're just going to win more money.
And the advantages of checking is essentially none.
So the correct play, I think, is clearly to bet in this situation.
So, essentially what I'm trying to get at is-- so I call this the Fundamental Theorem of Poker.
It's sort of a made-up term.
But essentially-- so one of the main things you should realize when you're betting postflop is that a bet basically accomplishes one of two things.
So with your good hands, you want bet because you can get them to call with worse, but still good hands.
And we call this value betting.
So with the pocket queens, this is-- what you're doing is value betting when you bet.
And then with the bottom hands in your range, you should bet to get them to fold better but still pretty weak hands.
So in this case, when we had king-queen, we would be betting to get them to fold ace-jack.
So, this is called bluffing.
So essentially, the point of a bet should either be a value bet because you have a really good hand, or a bluff because you have a really bad hand.
And you want to sort of think in terms of that way.
And if you ever find yourself making a bet and you're not sure whether it's a value bet or a bluff, then chances are you maybe shouldn't be betting.
So with the middle hands in your range, you should check because if you bet, you fold out all the weaker hands and you keep in all the stronger hands.
OK, does that make sense?
So this is like the fundamental concept in poker, essentially, that we should try to realize.
OK, so I sort of made this diagram.
So, the best hands are at the top of the rectangles.
The worse cards-- the worst hands are at the bottom.
So once again, we're thinking in terms of our entire range of possibilities.
So postflop, we want it sort of-- so that's on the left.
Postflop, we want to bet the best hands, bluff the worst hands, and then check things in the middle.
And then preflop, basically we fold all the bad hands and then we raise, or if someone's already raised, we re-raise the best hands and we just raise the pretty good hands.
I'll talk more about preflop playing in a future lecture.
But I just wanted to contrast this because I don't want someone to think, you know, if I have 7-2 offsuit preflop, I should be bluffing.
Because it's a different scenario.
Because preflop, you're choosing what percentage of hands to play.
And it's strictly better to just choose the best 30% or the best 40% instead of choosing the best 10% and the worst 10%.
OK, so going back to this principle, with 7-7, so it's still the same situation.
So I'm analyzing the same situation many, many times with different cards.
With 7-7, the right play is definitely to check.
Because so, with ace-queen, it's in a similar spot to ace-queen in that it's also you get called only by better hands and you mostly only fold out worse hands.
But with ace-queen, at least there was the advantage that if they called with like an 8 or 9, we could turn-- we have more chances to beat them.
We could turn an ace or a queen and win more money, or alternatively, we could turn a jack which gives us a straight draw, and try to bluff them off [INAUDIBLE] or something.
Whereas with pocket 7's, there's less room to play.
So, I think with pocket 7's, you should definitely check on this flop.
So, OK, so some other stuff about continuation betting on flops.
I wouldn't bluff super scary flops.
By super scary, I essentially-- mathematically, it just means there's a lot of uncertainty because so many things are possible.
And basically, this happens when there's three cards of the same suit and they're all connected.
So whenever the board is like this, it looks scary.
It looks very coordinated.
But mathematically speaking, it's essentially saying, anything can happen.
And no one should fold if you have anything at all.
Because you know, if you have any 10, you have a chance of making a straight.
If you have a reasonable heart, you have a chance of making a flush.
Just any hand is going to have a reasonable chance.
Same as, you know, even if you have like an ace, like I guess as we do, you have a chance of turning a higher pair.
So I would just check, and basically give up on this flop.
Or check, and hope you hit an ace or a queen, I guess.
And another thing is continuation betting into too many people, if you have nothing, it can be an advanced play in certain situations if everyone else also recognizes that if you're continuation betting into five people, you just have such a good hand that they're going to fold a hand as good as say, like, pocket 8's here.
But for the most part, in practice, I would say if the pot has like six or seven people, I wouldn't try to bet on the flop with overcards to try to get everyone to fold, because someone's going to have two hearts, or someone's getting have a 7.
Someone's going to have a pair.
OK, so I talked about sort of when you want to bet postflop.
So now, I'll talk a bit more about when you want to call bets postflop.
And I'm going to try to give a lot of different situations.
So now, let's look at calling bets.
And I think newer players especially tend to call with hands weaker than you need to.
So the general rule of thumb is, unless your opponent bets extremely small, which they should not do by what I explained, you should fold most of your hands to a half-pot bet.
So in some sense, most hands miss most flops.
And when you miss, there's really not a reason to do anything other than to fold.
There's a lot of tournaments.
See I always play the next tournament and get dealt and new hand and try to have a good hand, so, OK.
So now, I'm going to change this situation up a bit.
But once again, I'm going to analyze the same betting situation over different hands.
So now, it's everyone folds to-- is folds to the button.
We're still 40 bets deep because the blinds are 2,550 and we have-- we have 2,000.
And so the button makes it 125, and we decide to call with our hand.
OK, so let's say we have queen-9 suited in this case.
And, OK, let's say the flop is 10-7-2.
So, we check.
We check as a form of respect to the preflop raiser.
And then they bet half-pot.
So here this is a situation where we've got no pair, no draw.
I guess we have an overcard.
We have an overcard in the queen, and we can turn a pair of queens to beat a tier of 10's, but that's only three cards that help us.
Basically, the correct play is just to fold.
You essentially have nothing here.
You know, you have some like, speculative straight draws where if you hit a jack and then an 8, like if you have two cards that come for you, then you get a straight.
But for the most part, we've got nothing.
We just fold.
Here are some other situations where you should just fold.
So here, we have ace-high but the board has three pretty big cards.
And there's a decent chance he's going to have one of them.
And if he doesn't, he's going to have like queen-10, or 10-9, or something with straight [INAUDIBLE],, or some kind of straight draw with a lot of outs.
So there's just no reason to play hands like these.
So it's totally fine if you lose a lot of hands, as long as you win more chips on the hands that you do win.
So, just fold this.
Not much reason to play.
Here, you have this.
This board is a bit better for you.
You have ace-9 high on a 6-high board, but there's a lot of straight draws, and there is a flush draw, a spade flush draw on this board, so I would still fold.
Here, we even have a pair, but this is an example where I think I'd fold even though you have a pair.
So we have a pair of 9's here which is decent, but once again, there's three hearts.
And it's easy for him to make a flush or make a straight.
And even if he doesn't, we're going to be-- we're going to have to check to him on the turn, check to him on the river.
And then they can bet, and then we have to decide whether we call it or not.
So, just overall, there's a lot of-- it's an [INAUDIBLE] play.
And this is why what-- I'm talking about being out of position.
So where we have to act first on every straight, so it's going to be tougher to make good decisions.
OK, so now, we're getting some better hands.
But I think I would advocate still folding these.
So here, our hand is-- it's not bad.
We basically got a four-out straight draw, which is usually called a gutshot.
You hit an 8 to turn a straight.
And also we've got diamonds.
If we get two diamonds in a row, we can make a flush.
But overall on this spot, I would still fold just because even though we've got four outs to our gutshot, we've got no combinations-- we've got no other combinations to help us.
Like, a 9 is essentially useless when there's three higher cards on the board.
Turning a 9 doesn't help us at all.
Turning a 7 doesn't help us at all.
And furthermore, even if we turn an 8, it's possible.
So if we turn an 8, then queen-9 makes a better straight than us.
And also, king-queen, which already has a straight, is a better straight than us.
So even when we hit an 8, it's not even-- we're not even 100% sure we're going to win the hand.
And we could just put our chips in after hitting an 8 and lose all our chips.
So, I would fold this as well.
A four-out gutshot is bad unless you have other stuff to go with it.
OK, so in this case, we have a better straight draw.
So in this case, we have an eight-out straight draw.
Because an 8 gives us a straight-- 8 through king.
But a king also gives us a straight.
We have 9 through King.
But in this case, I would still-- I would still recommend folding just because if a king comes then-- you know, we do have a straight, but we lose to an ace.
So, 10-jack-queen-king-ace is going to just be a higher straight than our 9 through king straight, so even though we have a lot of outs, the outs aren't that good, and it's possible we just once again, we make a second-best hand, which is the worst possible situation in poker because you're going to put a lot of money in and you're going to lose.
OK, so this is another example I would fold.
Pocket 5's on queen-jack-10.
You have a small pair.
You started with a small pair, but you didn't improve at all.
The flop did not help your hand at all.
And it most likely helped your opponent's hand.
OK, so, here's a question you can ask.
So in all these spots, I say fold, but what if I think bluff-raising will work?
You know, what if I think I can get him to fold if I raise?
So what I want to- what I want to say is, when you're bluffing on the flop, it's usually because you have some kind of draw or some kind of nice speculative hand that has a chance of turning into a good hand.
In all the examples I showed, not only are our hands bad, there's not that many good cards for us, with the exception of maybe this 7-9 hand, where if you turn an 8, it's pretty good.
But yeah, even this-- even this hand, if you turn an 8, it's not that good, because the board is then going to be 8-10-jack-queen.
So we're only playing with one-card straight, which is still fairly easy to put aside.
OK, so before we get to spots, or maybe we want to raise, potentially bluff-raise, let me just go-- let's go through some spots where I think calling is a pretty good play.
So here, I think calling with ace-9 high on this board is reasonable.
In the example where I said you should fold, the flop was 6-5-4, and there was a flush draw.
So there were straight draws and flush draws.
In this case, the board is paired, which really helps us because there's just fewer combinations of pairs that he could make.
And a 3 is a pretty small card, so it's pretty unlikely they have a 3.
So ace-9-high is pretty good here.
You have two overcards.
You even have a diamond [?
back your ?]
flush draw.
So I would call here.
OK, so we would get-- so getting better, here, this hand is even better.
We have ace-3.
We have bottom pair on this.
Pretty safe flop.
It is three different cards, and they could have a jack or a 5, but just overall against a range, we have tremendous equity here.
So I would call.
So note, when I say call, do not raise.
I'll give you examples of how good, roughly speaking, our hand has to be to raise.
All right, so here, I would call.
The board is very scary, but we have second pair.
And I think second pair is just-- second pair with the best kicker would be ace.
So that's definitely good enough to call.
With ace-9, I think this would be borderline, whether you call.
I think calling is OK, but not mandatory.
OK, here's another example-- second pair on a safer board.
This is a pretty good hand.
I mean, if they've got an ace, we're not drawing to that many outs.
But if they don't have an ace, they're not drawing with that many outs.
So it's because whenever big cards come on the flop, like an ace and a king, in some sense, even if there's flush and straight draws, more of the-- more of who's going to win has been decided.
Because if you've got a big pair, like if you've got an ace or a king, then there's not like an even bigger card that can come in to try and beat you.
And if you don't have it, then, you know, it's like if you have 10-9, even if you turn it's hand, you're not going to outdraw an ace-7.
So here, I would just call.
Here, you have a pretty good pair, pocket 8's on 9-3-3.
Obviously, you have a very good hand here, but I just think, in this case, where there's no flush draws-- no straight draws, there's not really that much reason to raise because it's not like you're going to get called by worse hands that often that I would still only call.
Even though in some sense, your hand is very, very strong here.
Like, I would expect you to have the better hand more than 80% of the time.
But the problem is when you raise, they're only playing the 20% that beats you, which is hands 9's, hands with 3's, and pocket 10's plus.
So here, I would also recommend calling even though we have top pair.
So top pair, the highest pair on the board.
It's a pair of aces, which seems-- you know, it seems like we have a good hand we should raise.
But once again, our other card, which is very relevant, is very small in this case.
So we're going to lose to ace-queen-- any kind of big ace.
And furthermore, there's not that much reason to raise because we're not that scared of letting them see a turn card with, say, 10-9.
Because even if he turns a 10 or a 9, he's not going to beat us.
So, right-- so, the reason you raise is essentially to not let your opponent see more cards and give them outs.
But they don't have that many outs if we're ahead of them.
So I would-- so this is, you can say this is like, a way-ahead, way-behind situation.
That's some terminology that you might hear, which basically means you're either way ahead because they don't have an ace, or you are way behind because they've got ace-queen.
And when you're way ahead or way behind, you don't really want to be raising because he's just going to call you when you're way behind, and he's just going to fold when you're way ahead.
OK, so here we're getting some pretty good hands.
Ace-jack and jack-10-9 with a flush draw.
I think if you raise, it's not-- it wouldn't be the end of the world.
The issue is I think 2,000 chips when the blinds are only 2,550 is still a bit much to put in your entire stack on this flop.
And just because even though you have top pair with the best kicker, there is a lot of straights.
And even if he's-- like even if let's say they have queen-9 of hearts-- I think that's behind of you currently because they only have a pair of 9's.
There is just going to be so many straight and flush outs they're going to have that you're never really getting it in ahead of equity by that much.
So I would just call and see what the turn is.
But that's maybe the best-- one of the examples of the strongest hands you can have where I would still recommend calling.
In this case, we've got a small overpair.
But I would still call even though, once again, we have the best hand probably 80% plus of the time.
But once again, when you raise, you're only getting played back at by the 20% that beats you.
So, OK. Yeah, so, and notice there, when I went through these examples, the stack size was very important, right?
So in the first class, the important things are position, stack size, and cards.
In this case, the position was the button.
The cards-- I've showed you what the cards are.
But the stack size-- the fact that you're playing for potentially 2,000 chips, is very important.
And if you're playing for fewer chips then you can risk gambling a lot more, as I'll show you later.
OK, so another note is you could argue for check-raising in some of these call situations especially the ones where your hand was more vulnerable to overcards.
So let me just quickly run through them again.
So, I'll describe what they are.
So this is one of them, because here, you could argue, OK, I want to raise because if I call, what if he has jack-10?
Then he has 6-7, right?
Here, this is another example.
What if they have 10-9?
They can turn a 10 or 9 and beat us.
Here, this is not an example.
Here, I think calling is clearly correct because the only bad cards are like a queen, because an ace gives you two-pair.
So, there's just very few scary-- the board is already scary and there's not that many overcards that can come.
This is another example because in this case, there's actually no scary cards in subsets.
No card is-- there's no card that you're afraid of giving him.
This is an example where you could argue for raising.
You know, you don't want jack-10 to see another card.
That's reasonable, yeah.
Here, there's no risk.
It's completely safe, so I would never raise.
Here, I guess there are some bad cards, but if you raise, they can just call with your draws anyway try to hit.
So here, this is an example where you could argue for raising to not let them see a turn card.
OK, so, yeah.
These are reasonable arguments and I think raising some of these call hands would not be a terrible play.
But I think for most of these, the advantages of calling still outweigh the advantages of raising, especially if your opponent is also capable of bluffing.
Because let's say you check-raise.
So let's say they make it 150.
You make it 400.
And then let's say they go all in.
But they're going to go all in not only with hands that beat you, like pocket 7's plus.
They'll also occasionally do this with hands like ace-3 or ace-4, hands that have an overcard in the ace, but also have a gutshot straight draw.
Like ace-3 or ace-4 can make a straight.
Then you're just in a very tough spot.
Basically if you call, you're not in a good situation because you're often behind.
But if you fold, then you're also folding to a lot of weaker hands.
So there's just a lot of ways for the hand to go badly if you raise because your hand sort of-- it's too-- it's not good enough to raise and call if they re-raise.
Yet, it's too good to sort of waste as a check-raise bluff and just fold if they re-raise, right?
Like let's say my play here with pocket 6's is to check-raise and fold if they do anything else.
But then you could argue, why am I doing this with a hand as good as pocket 6's?
Why don't I do this with 10-9?
Because it's going to be a similar result most of the time.
They're going to either fold and I'm going to win the pot with 10-9, or they're going to raise and I'm going to have the fold with 10-9.
Occasionally, they'll call, but you could argue in that case, I would almost rather have 10-9 than pocket 6's because pocket 6's I'm going to always lose.
And 10-9, who knows?
I can turn a 10.
So, anyway, so the long story short is I would just recommend just calling with most of those hands although there's definitely lots of arguments you can make for raising.
Yeah?
I didn't get why we'd assume once-- should re-raise if you have the top pair, the top kicker of either because if you give them the draws, at least once you make the shorts bigger, then it will turn out all right
You're talking about this situation?
Yeah, it is
Yeah, yeah.
OK, yeah, that's a good point.
So, yeah, I think that the main reason is just your overall equity against their range is pretty low on this board compared to an average situation if you have top pair or top kicker.
Like if the board is jack-7-2, your equity against their range is probably something like 85%.
Whereas on this board, your equity is just overall a lot less.
There's a lot more hands that beat you.
So there's a lot higher probability that if you raise and call and put all your money in, that you're behind, basically.
So there's just a huge risk of that.
So, but yeah, I do agree.
It does feel very unsatisfying to only call and let them see those cards.
But I think it's sort of the lesser of two evils in this situation between putting all your money in with his hand versus letting them see some cards.
Yeah.
All right, cool.
OK, so just how good does my had have to be to raise?
And the question depends on the effective stack size.
But when the effective stack size is fairly deep-- so let's say 40 big blinds, I think it's about the cut-- 40 big blinds or deeper So, you know, you have a lot of chips.
You're playing for a lot of chips relative to the blinds.
You need a pretty good hand to want to put all your money in.
So a general rule of thumb is two-pair or bigger.
And yeah, once again, this is only applicable when the effective stack size is large.
And a lot of the online tournaments, especially the ones where the blinds go up quickly, the effective stack size can be around 10 or even lower pretty quickly.
Yeah?
How much do all of these decisions depend on their position or your position?
OK, that's a very good question, actually.
Right, so in this situation-- so the answer is essentially-- yeah, it depends a lot in this.
The examples I gave are specifically for the button.
Yeah, so, I would say if they were under the gun instead of the button, you need to play somewhat differently because they're going to have-- because on the button, they're going to have 55% of hands.
And let's say, here, like 10-9 is a possibility.
But actually, maybe a better example is-- now, I think this is a board where it's really different.
Because against a-- if they raise from the button, you know they could have 55% of hands.
And ace-3 is a pretty good hand here.
But if they raise from under the gun and they're pretty tight and their range is-- pocket 8's are better and ace-queen, ace-jack, ace-king, then you're doing a lot worse.
Yeah.
So yeah, so it definitely matters a lot.
And yeah, I mean-- yeah, so I guess I can't go through every single possibility, right?
Like this position, this stack size, with this [INAUDIBLE]..
So yeah, so you do have to do a bit of extrapolation.
Like if they're-- right now in these examples, they're the button.
If they're the cut-off, their range is going to be a bit stronger.
If they're the hijack, their range will be a bit stronger, yeah.
Yeah, that's a very good question.
Yeah, very, very good point.
Yeah, because in all the analysis I just did, I'm assuming their range includes hands as bad as like, 8-7 suited and stuff, which isn't going to be the case if they were an earlier position.
OK, so-- all right, so how good does my hand have to be to raise?
So, two-pair or better, but by two-pair or better, I mean two-pair where both of your cards make a pair.
Here, where one pair is on the flop, I wouldn't count this as two-pair.
And this two-pair is complete trash because any card beats you.
So I wouldn't go on here.
This one is also pretty bad.
So, OK.
So now, when would we raise?
OK, so I think this is what-- this is basically the best thing that can happen to you in poker, pretty much-- almost this exact situation.
So here, we've got three of a kind.
And there's also a lot of draws.
So we're going to get played back at a decent amount because you could have hearts.
He'd have like queen-jack.
There's just so much stuff that isn't going to fold, and we're always going to be way ahead of him.
Pretty much always going to be way ahead.
So, it's sweet.
I actually think this is very close to the dream situation.
Does anyone have an idea of why this situation is almost, in some sense, even better than say, we had pocket 10's?
Yeah?
If you have pocket 10s, that's why you tap a 10?
Exactly.
OK, I'm going to give you a $20 boost.
[CHUCKLING] No more playoffs today.
So you're going to see-- OK, right.
So it's almost even better in some sense you have pocket 5's because as Ben said, if you have pocket 5's, he could have a 10 and put in a lot of money.
If you have pocket 10's even though you're crushing even harder, it's almost like you've taken all the good cards away and then he's just always going to fold.
So, OK, here's another situation where your hand's very good, although I think you can make a case for just calling here.
Yeah, I don't want to say you have to raise because your hand is just so invulnerable.
And even though the chances that he has that the remaining ace and a better card to go with it is pretty small, there is still a lot of, I guess, adverse selection in this case because there's no draws.
When you raise, he will fold a lot of stuff.
And the stuff that he does play-- it's not going to only include the remaining ace.
Like, he will call with pocket 7's almost certainly, but it does include-- it does significantly make his range more likely to be the ace that beats you.
So, I think just calling is OK.
Here, obviously, you have the best possible hand.
Although, once again, I think just calling as a sort of a trap is reasonable.
Although I would advocate just raising.
I think it's the simpler play.
You know, a calling-- I think you'll see a lot of high stakes players do as a trap, but it's a very, very tricky play.
I think raising definitely has lots of merits because if they've got king-- like even ace-king, they're not going to fold.
They just want to get all the money in.
Because they're probably going to think you're more likely you have a single heart than two hearts.
So if they had ace-king, especially when it's only 40 big blinds, they're probably just going to be happy putting all the money in, and before you can see a fourth heart.
So, like in this case, even though you don't need a fourth heart to make a flush, a fourth card is a really bad card for you, because that sort of kills all the action that you're getting from a king-- from a pair of kings or something.
Yeah, OK, so this is a similar situation.
Although in this case, I would say a raise is mandatory.
In the previous case, I think it's not mandatory because your hand is so safe.
If you decide to try to trap your opponent, you'll never end up trapping yourself.
Whereas here, if they just have any heart higher than a 6, if a fourth heart comes, they are beating you now.
And worst of all is, let's say they have 10 of hearts, 9 of diamonds.
If you raise, you could have easily gotten them to fold that 10 of hearts, probably.
Although, that 10 of hearts, 9 of diamonds is a bit good.
Let's say like 9 of hearts, 8 of diamonds.
They almost certainly would've folded their higher heart, and now you just lost an entire pot because you didn't raise.
So here, I would raise.
You have a straight.
You are drawing dead if they have a flush.
But with only 40 big blinds, I think the chances of a flush is unlikely enough that getting in is OK. With 100 big blinds, you could still raise, but probably not go all in.
Two-pair-- the board is sort of scary, but two-pair is definitely good enough here.
So-- All right, so, these are examples where you had really good hands.
And I said you should just raise and get the money and then be happy about it.
So do I need such a good hand?
There's some other situations where a one-pair hand is sort of good enough.
And so, coming back to the ace-jack example, you asked.
So, we'll see some of that.
OK, so in this case, I think if you have pocket aces, that's definitely good enough to get it in.
Although, you shouldn't have pocket aces that often here because most of the time, you should be re-raising preflop.
But if you decide not to raise pocket aces preflop and you end up with it here, then I guess raising is good.
You're going to be having the better hand.
Although, if you were trapping them preflop, maybe you just want to trap them again and call again since there's not that many scary cards.
So here's an example of an ace-jack situation where I think getting in is fine just because there's still draws and there's still lots of worse hands that will give you action, but overall, your equity against their range is much, much higher on jack-7-deuce than jack-10-9.
So I would just raise and be happy getting in 40 big blinds here.
Here's a similar example-- king-jack.
But we've also got the second best flush draw, so it's just very hard to get in back.
OK, so we'll see some examples where we have fewer chips.
So in the last case, all the hands, we essentially had 40 big blinds-- essentially had a lot of chips.
So, OK, so now, let's look at a situation where all the players-- so we only have 1,250 to start the hand, so that's 25 big blinds.
And, OK, so in this case, even if you've got top pair with a mediocre kicker, I would just be prepared to go all in because we risking a lot fewer chips.
We're risking a lot fewer chips relative to the blinds.
So here, I would just go all in.
So here are some-- yes, so I'm showing you some examples where if I'm 40 or more big blinds deep, I wouldn't be happy getting all my money in.
But when I'm only 25 big blinds deep, I'm very happy to get all my money in.
This is another example-- top pair, top kicker.
It is a scary board in some sense, but you've just got to gamble for it when the pot is already so big on the flop relative to your stack.
Same thing again here.
So earlier, I had some examples of this on-- similar to this where I said, you just call it, but when you've only got-- when you've only got four to five times the flop-- I'm sorry, 45 times the pot on the flop, I would just go for it.
OK, I'll quickly run through this section.
This is a slightly more advanced play where-- it's where you don't check to the preflop raisers.
So this whole time, I've been talking about checking to preflop raiser as a form of respect-- respecting that the preflop raiser's range contains aces and yours probably doesn't.
So when do we want to lead?
So from a theoretical point of view, so from a trying-to-play-optimally point of view, essentially we want to lead flops that hit our range better than his, even though his range contains aces, which is like a universal best hand.
So they're rare, but they do exist.
And I think it's OK if you never lead, but I'm going to run through this section showing some examples where I think leading is fun.
But I think it's an advanced, unnecessary play.
So, OK, so here's an example.
I hope this is visible.
So we won't-- we're pretty shallow here.
So most of the situations where leading is good, you're fairly shallow.
So a cut-off two [INAUDIBLE].
So one from the button makes it 1,200 at 300, 600.
And there's antes.
So, we call with 9-deuce suited with 4 and 1/2 to 1 odds.
OK, so-- and so, here, we flop 2-5-6.
And I lead a third of pot, planning essentially to get all my money in because we're so shallow.
Like, I only started the hand with 10 big blinds.
Essentially any pair is going to be good enough to get all my money in, although, maybe not if you raise from under the gun.
But from cut-off, I'm happy to put all my money in against this.
So what's the rationale?
So, our hand's good enough to put all our money in.
And it's essentially a better flop for our range because we're short enough where the immediate equity preflop means so much to us what we're calling a very wide range of hands.
Basically, a lot of hands that includes 2's, 3's, 4's, 5's, and 6's, whereas his range, he opened from button-- sorry, he opened from cut-off.
You know, even though it contains more monstrous hands than ours, like pocket aces, it also just contains a lot of hands like jack-10, queen-10-- basically, hands that don't connect with this board.
Because if he had 6-5, he probably isn't going to open, at least, I don't think by my recommendations he should be opening 6-5.
Whereas we could have hands like 6-5, we can have hands like 4-3.
So I'll lead here.
So it's basically essentially saying, you know, I'm no longer respecting you because the flop came in a very specific way where now my range of hands is actually better than your range of hands.
So I'm essentially the preflop aggressor.
I'm going to bet and try to make the pot big and force you into a tough situation with your weaker range.
And yeah, and we can play like this with our good hands and with our bad hands, like 7-3.
We have a gutshot, but we can bluff a bit.
So here's another example.
So, under the gun raises.
We call with ace-9 suited, which I think is a bit speculative, but whatever.
We just had to call.
And we lead a 10-7-5 flop where we have a flush draw.
And once again, I think it's reasonable.
So it's deep-- it's fairly deep, but we can have-- even though they can have aces, we can have more hands like-- we can have more pocket 5's, pocket 7's.
Tighter hands than him.
He might not open those.
And also, he almost certainly isn't ever going to have 9-8 of diamonds or jack-9 of diamonds-- hands that are basically monstrous draws, whereas we can.
So once again, I'd bet because I don't really want him to check back ace-king or pocket 8's or something, which he probably will, because if they have ace-king, they're going to think the board is fairly scary.
They're probably not going to fold that much because there's so many draws if I bet.
So they're just going to check back ace-king.
And that essentially has-- that essentially has two problems, is we're not beating ace-king unless we improve, so by betting, we basically we just make the pot bigger against ace-king and 9's, and we can get them to fold.
Whereas if we check, it might just checked down.
And yeah, so-- so yeah.
One caveat about leading though is if your opponent knows leading is part of your strategy, then your range is significantly weaker when you actually check.
So as a result, if your opponent knows you're the type of player who, let's say, on a-- you know, so this is really getting higher-- next and next level.
But if they know that you're the type of player you will lead a lot of your hands on a 2-5-6 flop in this specific situation, then when you check to them with a hand like jack-10, they can basically just bet and know you'll almost certainly fold, because you're leading a lot of your good hands.
So that's another caveat about leading.
It's very tricky to balance correctly because you need to-- you know, if you're leading bluffs, you need to also lead some good hands to balance it out.
But then you also need to check some good hands occasionally to balance out all the bad hands you're checking so that your opponent can't just bet for free.
So it's a very tricky play.
I wanted to quickly run through it to just say it's out there, and when you watch poker and you see people not automatically check to the preflop aggressor, just sort of giving you an idea of why.
But it is a fairly advanced and tricky play.
And it's easy to make mistakes leading, which is why I think-- you know, I think it's fine if throughout this course you never lead.
OK, so now we're going to talk get some implied odds and reverse implied odds.
So, OK, so here's a situation.
So note that-- so we have the worst hand in poker.
We've got-- we've only got three big blinds, OK?
And hijack raises to 2,400.
So the question is, do we want to call in this situation?
So we're putting in 1,600.
So we can do a quick equity calculation.
So if we call, we're all in.
So the only thing that matters is the equity of our hand against his range.
That's essentially the only thing that matters because he has no more decisions.
So optimal play doesn't even matter in this case.
It's a strict mathematical expectation calculation.
And we're actually-- so basically, we need a call two big blinds.
So 1,600-- that's two big blinds.
To win how much is already in the pot is 5 and 1/2, right?
5 and 1/2 big blinds.
So they made a big raise here.
They made it 3x.
So we need 2 over 7.5, which is 27% equity.
And if we put him on a reasonable range for that position, we have 28% equity, which is enough to call.
So I think calling is a fine play, OK?
I'm not winning in front of this person who I think probably was me.
So, yeah, calling's a fine play.
And so, now, but I want to emphasize, OK?
So if calling is a good play because it's positive expectancy, why don't we call when we have more chips?
So in this situation, if someone called, they would probably be ridiculed, right?
Why are you calling with 2-7 offsuit against a hijack raise that isn't even giving you good odds, because they raised it 3x, which is a big raise.
So why is it bad in this case, when you have more chips?
And the reason is essentially what's called reverse implied odds.
So in the first situation, we called because we were all in, and the equity calculations said we could.
But in the second situation, even though the equity calculation is the same and we do have the right odds to call preflop, the hand isn't over.
There is still potentially 46,000 more chips to be put in postflop.
And with a hand like 7-deuce offsuit, not only is it a weak hand, it's very hard to play.
When we hit a pair, we hit a small pair.
We're often not going to be sure whether we have the better hand.
We're going to have to decide whether to fold or call them down.
And we're often going to be wrong.
So, it's a very, very easy fold in this situation even though you have the odds, because these reverse implied odds hurt you so much when there is still potentially 46,000 to play for postflop.
OK, so here, let's say instead of 7-2 offsuit, I had ace-2 offsuit.
I would still fold.
So if you do the same equity calculation, we actually have 43% against his range, which is tremendous.
But you only need 27% equity to all in, right?
So, why would we fold when we have 43% equity and we only need 27% equity?
That reason is because, once again, reverse implied odds.
So, there used to be actually be this way you could cheat in online poker.
So they had this thing called disconnect protection which is someone disconnects-- this is really stupid.
I got cheated by this so many times.
Basically, if you disconnected, instead of your hand getting folded, essentially there's no more betting and all the cards are dealt.
And the person who wins the hand just wins the pot.
OK, so essentially by disconnecting, you could say, OK, screw you.
There's no more betting.
We're just going to see until the river, and whoever wins the hand is going to win the whole pot.
So basically, you could cheat in this situation even if you had 7-2 offsuit, by just calling and then disconnecting immediately.
And then none of this reverse implied odds exist.
But you can't cheat like this anymore, so don't try to make that play.
So yeah, so in this case, even though you have 43% equity, I would fold.
Here, with 9-8 suited, it's a whole different story.
We only have 37% equity, which is worse than 43%.
But I would call.
With 9-8 suited, you can make a flush and make a better hand than him.
You can make a straight and get him to put in a lot of money with a good pair.
And yeah, you're still out of position.
You still have to act second.
But overall, it's just a much better situation.
OK, so now, I want to talk about implied odds.
So with the 9-8 suited, you know, I sort of talked about-- the implied odds, the fact that you still have to play for more chips after on the turn and river almost works in your favor because you can make straights and flushes and have a very good hand and be fairly certain you're winning, and get him to put in more money with a worse hand.
So implied odds is basically, yeah, it's when the odds are in your favor because you're playing a hand that plays well postflop.
OK, so-- OK, so let's look at an example where implied odds, I think, are good.
So this is called set mining.
This is a play that's, I think-- so, I'm going to go through it even though it's a play that's sort of dying in poker.
I think it's a play that no longer works that well in practice nowadays, at least at like, at fairly high stakes.
But it's in theory, I think it teaches you a very good thing about implied odds.
So, in this case, under the gun raises to 3x-- 2,400 when the blinds are 400, 800.
And we decide to call.
So if you do the calculation, we don't have enough equity to call, but why do we call?
So essentially, we're in a position because we have the button.
And also, we can win essentially a huge pot when we hit a third deuce.
So now, this only happens a small percent of the time.
And if you do the calculation, this happens roughly 1/8 of the time.
You're going to hit a third deuce 1/8 of the time.
And essentially when you don't hit a deuce, you're against an under-the-gun raise, so he's going to have a really strong range.
You're going to lose the pot 7/8 of the time, essentially, roughly speaking.
So what this is essentially saying is if I'm only winning the pot 1/8 of the time, I need to win on average 8 times as much as I lose for it to be worth it, right?
But the point is, at least back in the day in poker, you essentially do, because when you do hit the deuce, they just-- they're going to have pocket aces or pocket kings or, you know, the flop will come ace-6-2 and they'll have ace-king.
They'll just have a tremendously good hand that's still crushed by your hand so often that you definitely, essentially, 1/8 of the time, you lose 2,400 chips, and-- I'm sorry, 7/8 of the time, you lose 2,400 chips, and 1/8 of the time, you win their whole stack of 80,000 chips.
Essentially, it's what the best situation is.
So, yeah, so you need a lot of chips for this to be a good play.
You know, even if you have 40 big blinds which is, by our standards, fairly deep, it's still not that good because, once again, you only win 1/8 of the time.
So, you really need to win a lot when you win.
And like when you hit a 2, it's not 100% sure that he's going to put in all his money.
So the fact-- so basically, you're only winning a huge pot even less than 1/8.
Like you need to hit a 2, and you need to have-- you have aces or kings.
So you know, that's going to be like 1/20 or whatever.
But the point is if the stack is so deep, then it's still worth it, essentially.
And the reason why it's sort of dying in poker is because I think nowadays, players are good enough where even if they have aces under the gun, they realize the optimal strategy isn't to always put all your money in on a 7-5-2 flop.
But yeah, long story short, I think this is a fine play.
I think it's fine if you do it in the current league.
Because it's tough to fold aces on a 7-5-deuce flop.
And yeah, I think it's a good play to know about at least.
So, here is another hand.
So, cut-off raises to 2,000.
We have jack-10 suited on the button.
So once again, we don't have direct odds to call.
But we have good implied odds because we're in position and we've got a suited connected hand.
OK, so I'm going to go through the whole hand because there's going to be more examples of implied odds here.
So both blinds fold, and the preflop aggressor, the cut-off continuation-- that's about a bit less than half of pot.
And we have a flush draw here.
So, what is the effective stack size?
Even though we started the hand with 80,000, they only have-- they only started the hand with 30,000.
So the effective stack size is only 30,000.
So we're definitely not folding.
We could call or we could raise.
The problem with raising is when we raise and they go all in, we're sort in a really crappy situation where our hand's too good to really want to fold, but too weak to have the odds to call his all-in.
So I think just calling is the right play.
And yeah, and raising gives him the opportunity to re-raise on.
If we call, he can't say, oh, because right now, I'm putting in more chips, right?
So, OK, so we miss on the turn, which kind of sucks.
And they bet fairly big.
They bet 8,000 into 12,000.
So let's do an analysis.
So, OK, so if I'm simplifying an assumption, let's assume they have it, OK?
Let's assume we modeled them most fairly tight.
When they're making this bet, they've got us beat.
We're not going to win the hand by rivering a jack or a 10.
We need to river a diamond.
So essentially, let's assume they just have ace-king, or they're going to have a pair of aces, at least.
So we need to call 8,000 to win 20,000 in the pot, so we need 8 over 28, or 29% equity.
And if you count, there's 9 diamonds out of 46 cards, so we only have 20% equity, and we need 29% equity.
But I would still call.
So, because when we call and we hit our flush, we can actually-- we can bet on the river, and there's a decent chance that we're going to get called.
Whereas if we call and we miss our flush, we can just happily fold.
Does that make sense to everyone?
So we don't really have the odds to call here.
But we call because if we hit our diamond, we can go all in on the river.
And there's a decent chance they're going to call us, and we're going to win more money.
So we actually do have the odds, essentially.
We have the implied odds.
OK, so here's a similar scenario.
So pretend the same thing happened, but this was the situation instead.
In this case, I think we should definitely fold.
So what changed was instead of having two diamonds on the board and two diamonds in our hand, now there's three diamonds on the board and one diamond in our hand.
So it's worse for many, many reasons.
So one is when you only have one diamond in your hand and there's three diamonds on the board, if they have ace-king, it's going to be more obvious that we probably have a flush if we bet on the river and they have like ace-king.
So they're probably just going to fold this king.
So, we're not going to get paid off when we hit the diamond.
And even worse, we're not even drawing to the best hand.
Like, what if they have ace-king with the king of diamonds?
Then we're just crushed.
We're drawing dead in this situation.
There's no cards that can help us win.
So that's just such a huge risk.
So, yeah, and also, a small thing, but the board is paired.
So it's possible they have like pocket aces, or ace-5.
They already have a full house.
And even if we hit the best flush, we're not going to win.
OK, so-- OK, so, that was a good example, I think, of when you have the implied odds to call with your draw to try to hit.
When you don't have the implied odds to try to hit.
But we can also bluff with draws.
So in all the examples I gave you before where you check-raised the flop, we had a good hand, right?
But obviously, this is predictable.
If they know that we only check-raise their flop that with two-pair or better, they can just fold whenever they have one-pair or worse, right?
So draws, not only are they good for implied odds purposes where you call and then only put in more money when you hit, they're also very good for bluffing in certain situations.
So essentially, what you'll realize is bluffing sort of equivalent to having a draw, in some sense.
Because the hands you want to bluff with are the hands that have some draw value.
So, here's an example.
The button raises to 1,600.
So we're 20 big blinds deep.
And one of the big blind with 10-9, and we call with our 4.5 to 1 odds.
OK, and the flop is jack-10-5.
So we've got what I would consider a good draw.
So a good draw, I would say, usually means either an open-ended straight draw.
So four-in-a-row do a straight.
So there's eight potential cards that complete your straight, or a flush draw, which it usually has 9 outs.
OK, and then they bet 2,500.
And then we go all in in this situation.
Essentially, we're bluffing.
We're hoping they fold.
But why is this a good play?
So, you know, it seems pretty sane to just call instead, right?
Because we can call, we can have perfect information of when we hit, when we don't, and only risk more money if we do.
But it's actually also quite a good play to go all in because even though it looks like we have a bad hand to be risking all our chips because we only have 10-high.
The point is the fact that we have 10-high doesn't matter because they're never calling our all-in with a naked queen-high or king-high, right?
They're not to call with a random hand.
So what matters isn't how good our hand is, in this case, 10-high.
What matters is how many outs we have against the hands that they called.
So in this case, we're always going to have 8 outs.
Even if they have the best possible hand, like pocket jacks, we're still going to have 8 outs.
So, this is a great hand to bluff with because if we call, we might just lose the pot to queen-high by the river.
Whereas if we check-raise all in, it's essentially the opposite of what I was talking about before with like being averagely selected.
Here, it's just essentially being beneficially selected because all the hands that barely beat you, he's going to fold.
And if they have a hand as good as j-- ugh.
When they have a hand as good as jack-jack, it's almost, like, wasted, you know?
They haven't-- they got their jack-jack with-- if they had jack-jack, they're hoping we have like, pocket 8's that only has one out against them.
But here, we have a much worse hand than pocket 8's with 8 outs against them.
You can contrast this with having, say, like 6-5 on this flop.
6-5 is a really bad hand to raise with.
You just want to be calling because it's sort of the opposite.
Any hand that beats you is going to be crushing you, and you're going to fold out the hands that are already using to your 6-5.
But yeah, so 6-5, if you're against like ace-jack, you only have five outs.
And you have close to 0% equity against flopping jacks.
So, right, so this is essentially the bluffing epiphany.
So when you're bluffing, the-- when you're bluffing before the river, the thing that matters isn't how good our hand is, it's how many outs we have.
OK, so-- right, OK, so this is essentially just me talking about the bluffing epiphany.
So you want to always bluff with hands that have a high number of outs against the hands that you get called by.
So, here's an example of something that's not bluffing, OK?
So the cut-off raises to 90 and we call with 7-6 of diamonds on the big blind.
And this happens randomly.
You know, sometimes you'll see someone make a play like, what if ends-- You're going to-- so someone will make a play like this.
And it's going to work.
And they're going to look like a genius.
So, we check to the preflop raiser.
We've got nothing, but we have a read that they, like, scratch their ear or something.
So, we check-raise hoping that they fold.
So we get called.
And, you know, we still got absolutely nothing here.
And we bet on the turn because that's the only way we could potentially win the hand.
And they call.
And then we still have nothing.
And then as a last resort, we go all in on the river.
And, you know, a good percent of the time, they're going to fold.
And we're going to look like a genius for making this bluff, right?
So basically this is not-- so yeah, this is not bluffing.
This is essentially opening your wallet and giving your money to your opponent.
So when you're check-raising, you're essentially-- the reason why you were check-raising in the first place as a bluff is because you had some chance of making a good hand.
And essentially, a bluff is because your-- the hand that you could have potentially made, the flush that you potentially could have made, didn't get there.
It's not because you started with a hand that could never make anything on this board and it didn't get there.
So, OK, so here's a similar situation where I think it makes a lot more sense.
So it's going to be a similar board, but we have 10-9 of hearts.
And here, we check-raise their flop bet.
So this situation's a bit different than the jack-10 diamond situation from a while ago, where we only called with a flush draw.
The reason why raising here is better is because the stack sizes is deeper.
So with the jack-10 of diamonds hand, the problem with check-raising was there's a risk they would go all in.
And we would fold because we don't have good enough odds, yet our hand is so good.
Here, our hand is similarly good, but because it's so deep.
So, we actually started the hand with, I think, 100 big blinds instead of 40 big blinds.
They're pretty unlikely to go all in.
So here, check-raising 10-9 of hearts has a lot lower chance of getting blown off the hand.
OK, and we get a called.
And we turn a 15-out draw.
So we had 9 outs here, right?
But the jack now essentially gives us 15 outs because it gives us the 8 open-ended straight outs, plus the 9 flush outs, minus the 2 from double counting-- so 15 outs.
And we bet 650 here.
So it's the same hand as a 7-6 suited.
It's the same hand as the stupid play.
We're betting here.
And we get called.
So, some notes.
Here, our hand was good enough where even though when bet 650 into the pot on this turn, they could go all in.
If they have like ace-jack, they're probably going to go all in.
And you know it's going to suck.
But we have good enough odds in this case, because we have so many outs, that we can even call an all-in.
So it's fine.
We can just call.
And yeah, so, note that by check-raising the flop, betting turn-- so the good thing about playing draws like this is that now, if we play draws like this, we can also play our good hands, like ace-8 or pocket deuces like this, and get paid off a lot more.
So, OK, so we get called.
And then the river is a king.
It's not the scariest card, but we can have queen-10 of hearts.
We could easily get a queen-10 of hearts.
So we go all in here.
And even if the king wasn't scary at all, like even if the king was like a 3 or a deuce, I'm fine with just going all in because that's the only way we can win the hand.
But it's a fine excuse here to go all in with 10-high on the river, and say that's the only way I could've won the hand because there was a reason I was check-raising and representing all the strength on the flop at turn, right?
It's because I had a heart draw.
My heart draw didn't get there, so now, I need to desperately, to have a chance of winning the hand, go all in.
But it's reasonable.
The reason why I got to this situation, this bad situation, is because essentially I got unlucky, right?
I had a chance of hitting a flush, and I didn't.
It's not like the 7-6 case where I knew I was going to get it in this situation, which is not a good situation.
But here, I got a bit unlucky to get in this situation.
OK, so-- all right, so I think this is the last example.
So, last example of when you give up on a bluff.
So sometimes, you're bluffing.
You don't have a great hand.
But sometimes, you just give up because you figured out your opponent's not folding, or the board comes out in a way where it's not theoretically a good idea for you planning to fold.
So hijack minus 1 makes it 90.
Cut-off calls.
And we're 100 bets deep.
We call with jack-9 of hearts from the big blind.
OK, so the flop is a 10-7-deuce.
And so we have a-- we get a decent flop.
So once again, you know, we have a reasonable draw here.
We have some reason to be bluffing.
We don't have nothing.
We have some reason to be bluffing because even though-- essentially, we're hoping to turn 8, even though the chances that is only 4 out of 47, which is low.
But it's still something.
And it's much better to bluff with a 4 out of 47 chance of making a monster hand than a 0 out of 47 chance.
So, OK, so here, this was interesting.
The preflop aggressor checked, which is fine.
As the preflop aggressor, you're not forced to grant the flop.
So, but this-- but note that the preflop aggressor is not the guy who got the flop.
So the cut-off bets the flop, but the cut-off was not the preflop aggressor.
OK, so the cut-off shouldn't have pocket aces.
They could occasionally, but it's less likely than usual that they have pocket aces.
So they bet, and we decide to check-raise here.
And yeah, I think this is a very reasonable play.
So what are the reasons?
One-- so yeah, we have something.
We can try to turn an 8.
And also, turning a jack is fine.
We also turn a heart.
And so we had lots of backdoor draws.
We can turn a heart.
That will give us four hearts.
And we could turn even like a queen will give us an open-ended straight draw.
A king will also give us an open-ended straight draw.
So that's the weird case where-- so if the turn is a king, then the board will be king-10-7.
And we'll have the jack-9.
And then an 8 or a queen will give us a straight.
It's one of the rare cases.
It's called-- it's called a double gutshot, or a double gutter.
So it's one of the weird-- it's one of the strange cases to watch out for.
It's possible that you have an 8-out straight draw, even though you don't have four to a straight.
So one example is where you have like-- it's like this one, where you have a king and then jack-10-9, and then 7.
And another example is where, let's say you have like, king-jack on an ace-10-7-6 board.
Then essentially the board-- essentially, it's ace-king-blank, jack-10-blank, 8-7.
So then you have it hit a 9 or a queen.
So there's a couple of weird straight draws you can have.
But anyways, the point is-- so yeah, I think this is a good play, not only because we have a crappy hand that's basically-- so basically, a crappy draw is a really good hand to bluff with, but also the fact that their range isn't particularly strong here.
Because they can't really have aces.
So, I mean, they can have pocket 7's or pocket deuces, I guess, but also we're attacking the fact that he's not the preflop raiser.
So their range isn't that strong.
OK, so, so, hijack minus 1, the preflop aggressor, gets out of the way.
And then cut-off calls us.
And then the turn is a 7, which is basically the worst card in the deck for us.
I'd say the 7 of clubs.
Even if it was like the 7 of spades, it would've been better.
So, why is this the worst card?
So first of all, it doesn't help us at all, right?
It doesn't give us any kind of better straight draw or better flush draw or a pair.
And furthermore, it's fairly safe.
You know, even if the turn was an ace, we could at least pretend we have an ace.
The 7 is a fairly safe card.
If they have a pair 10's, they're not jeopardized because of ace-king.
Furthermore, the main issue is his range is going to be a lot more likely to contain a 7 than our range.
Because, remember what I said earlier, you don't really want to be raising-- check-raising the flop with your middle-of-the-road hands.
Because your middle-of-the-road hands, you want to just call.
Because when you raise, you're getting called by worse-- sorry, you're getting called by better and you're folding out worse.
So, probably if I had a 7 on the flop here, I'm never check-raising.
So basically, I can't have a 7 here.
And furthermore, they're a lot more likely to have a 7.
So this just really hurts us.
Essentially, it's just a bad situation where their range is a lot stronger than our range.
And basically the best thing we can do is just fold our hand.
There's no point-- you know, there's no point to try to continue bluffing.
Like, it's sort of like playing the "who's taller" game where you don't get to see my height, but you know that I was like malnourished as a kid.
Like you just know, on average, there's no way I'm going to be taller than you.
So the right thing to do here is just stop our bluff.
And it was like the worst card that came, but we just give up the hand, which is fine.
And then they take down the pot.
Right, so, bluffing epiphany 2 is basically, other than counting how many outs we have, it's important to analyze what we're representing.
What else is in our range?
You know, we're bluffing.
We don't have a good hand.
But what's the good hand that we could potentially have in some other parallel universes that we're representing?
And in this case, there just really isn't anything.
So yeah, so like an ace-turn would've been fine.
Even a 6-turn would have been OK, because we could have represented 9-8.
So, one of the best times to bluff is when your draw missed, but a different draw completed.
So like if you had a straight draw and you were bluffing the flop, and then the turn completes the flush, it's reasonable to bet.
I mean, you do run into the risk that they have the flush, but also it's just they're going to be more scared of the flush.
And similarly, yeah, like if you have a flush draw and the straight completes, you can also bet.
OK, so, yeah, so I'm running through a lot of poker concepts today.
So that's the end.
I'll give a quick summary.
So hopefully, this will give people more ideas on how to play online.
I know I sort of left you guys in the dark last time.
I mean, I ran through things pretty quickly this [AUDIO OUT],, but hopefully, as you play more, it's OK if you didn't understand everything immediately.
But hopefully, as you play more, you'll be like, ah, implied odds, or you'll be reminded of examples from this class.
So yeah, so we covered continuation betting, the preflop aggressor getting the respect, being able to bet, the Fundamental Theorem of Poker about how you bet your good hands and bluff your bad hands.
We analyzed different flops, and I tried to give you a general idea of what sorts of-- how good your hand needs to be on certain kinds of flops.
And yeah, so when I showed you the different flops, you know, what I told you-- I didn't like mathematically prove that calling is good or raising is good.
It's basically just like from experience and from doing calculations, I know roughly your equity is good enough.
But I didn't like, mathematically prove it.
So you essentially just have to trust me in that sense for the example flops that I showed you, because I didn't show any calculations.
Yeah, we talked about when to lead.
We talked about implied odds, reverse implied odds, set mining, bluffing with draws, and then the two bluffing epiphanies.
And yeah, so it's important to-- so basically, it's much easier to play good hands than bad hands in poker in some sense, because when you have a good hand, you just bet and hope everyone calls.
But it's important to know how to play bad hands, a.k.a bluff, so that your opponent also might call you when you have a good hand.
Because if you're the type of player who never bluffs, then the game is uninteresting, right?
So, because then, your opponent just always folds whenever you do anything.
So that's why it's important to think about the bluffing epiphanies.
OK, cool.
Yeah, thanks.
And hope you continue to play online.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I guess I'll get started.
So all right, cool.
OK, so before I get into any hands, I wanted to-- this is really loud.
Before I get to any hands, I want to talk about tournaments versus cash games since I haven't really made the distinction yet.
So essentially-- so this is a big chart with all the differences.
So when you guys play in the things online, those are called tournaments.
So in the tournament, you sort of buy-in for a fixed amount of play chips.
case, it's not that relevant.
It's like one play dollar.
And then you get, for coming in the top 20% a round, you're going to get some amount of play chips paid out to you.
And like, if you win, you get the most.
And if you just barely make it, you'll probably make back like two play chips.
So you'll profit one play chip or something.
So that's how tournaments work.
And that's sort of what, throughout this class so far, I sort of assumed all the hands are taken from tournaments.
So there is a different way to play poker.
And a lot of you might have actually been introduced to poker like this-- so, you have like sort of a home game, where there's just a bunch of friends, you're playing at someone's house.
And you all buy in for some amount of play chips.
And basically, you can play, and people can join the stock any time.
If you lose all your play chips, you can just buy in for more.
It's not like tournaments where once you're out, you're eliminated.
And you can never go back into that tournament and cash in to go buy in whenever.
You can hit and run people.
You can come try to make a bit of money and then just leave as soon as you make a bit.
So what are the main differences?
So I wanted to highlight the differences between the two types of games and talk about why we mostly choose students tournaments in this class.
So everyone knows the definition of what a tournament is, what a cash game is.
So let me talk a bit more about, what are some differences?
In cash games, usually, the stakes are fixed.
So usually, it will say we're playing for like one play dollar, two play dollars, or something like that, whereas in tournaments, I'm sure as you've noticed, the blinds keep going up.
The blinds start 10/20 in your tournaments.
And then they become 20/40, and then like 40/80, or 30/60.
Actually, there's a 15/30 level, I think.
So it's 15/30 and 20/40, and then 25/50.
And it keeps going up.
and the reason this is the case is so that eventually the tournament ends, because if the blinds don't go up, the tournament will take forever to end.
So what's a big factor about tournaments versus cash games?
In tournaments, you have no control over your table.
You just join in the tournament, and you're put at some table, and you've got to do your best at the table you're placed at.
And the fact that there's not this much metagame, I think is one of the reasons it's much easier to analyze poker hands in terms of tournaments, because in cash games, if you're at the cash game, and everyone's really, really good, you could just choose, I don't want to play in this cash game anymore.
I'm just losing my money.
And you could choose to stand up and leave.
And there's a lot of decisions of this form.
So there is more metagame required.
And in tournaments, the general goal is to survive.
You're just trying to get eliminated as late as possible to maximize your payout, whereas in cash games, a lot of it comes down to, if there's one bad player at the table, you're trying to target that specific player.
You're trying to do everything you can to get in hands against that player, and exploit them, and win a lot of money against them, and risk a lot and stuff like that.
There's less focus on the math of poker and more focus on specifically targeting certain players.
Tournaments-- there's frequent but fixed losses.
So if you were playing tournaments for real money, how it works is most of the time, like 80% of players don't get paid anything back from the tournament.
So when you play tournaments most of the time, you're going to lose, even if you're really good.
You're not going to win more than 50% of the time.
So you're going to lose, lose, lose, lose lose.
And then you're going to get a big win and then win a lot.
And then it's going to be lose, lose, lose, lose, and you're going to win a lot.
Sort of like playing the lottery, whereas cash games, anything can happen.
You can lose a lot.
If you are having a bad day and you decide not to quit, you could lose all your money, buy back in, lose all your money, by back in, eventually, lose all the money in your bank account.
That could happen.
You could also win a lot.
So anything can happen.
But in some sense, tournaments is more variance.
So like when I was talking in the first class about the law of large numbers and reaching the long run, it does, overall, in some sense, take longer if you're playing tournaments, because in tournaments, you do need to occasionally get the big score in the tournament-- it's sort of like winning the lottery-- to make back all your money.
And if you don't, then you won't make back all your money.
I argue that tournaments, I think, are more fun.
They're more exciting.
If you make it far in a tournament, eventually, it comes down to just two of you.
It's pretty exciting.
Also, tournaments have a wider range of situations.
So in cash games, usually, it'll be a fixed number of players at the table.
And players will have a similar number of big blinds, because whenever you lose a bunch of big blinds, you can always buy back in and get those big blinds back by putting more money on the table, whereas in tournaments, we have to learn how to handle I, think interesting, mathematical situations where you have one big blind.
How do I play it optimally?
And stuff like that.
The last thing is if you play at a casino, another good thing about tournaments is relatively, the casino makes a lot less from tournaments and cash games.
Cash games, just as a fraction of how much money you wager, the casino takes way more compared to tournaments.
Tournaments-- what the with the house takes is very small.
OK so those are the main differences.
But I did want to comprehensively go through the differences, because if you do decide to play poker, these are essentially the two ways to play it.
So I wanted to make sure everyone is well aware of the differences.
So yeah, so why I chose tournaments for the class-- I think there's less metagame.
I think it's more exciting.
I'm mostly a tournament player, myself, although I have played cash games for extended periods during my career.
Yeah, there's a wider range of scenarios.
And I think it's more applicable.
I think the MIT Poker Club-- the event they're running in January is a tournament, because tournaments are exciting.
OK, so are there any questions about this chart?
OK, cool.
Yeah, our club actually has both-- I mean the points-only tournaments count.
But if you look at this thing above, then actually, the pen works now.
I don't have to draw in here.
OK.
If you look there, then these are the cash tables.
And you could, in theory, sit down there and play for play money.
And if you do lose all your play money in those cash tables to a shark in the class, and you have no more play money, and you can't join tournaments, you can always refill your play money and join the tournaments.
So that's why it's called "play money".
Only tournaments count for standings.
And if you want to see the standings, you have to-- oh, I see.
They annotate.
So you see that you've got to click the blue arrow, and then you can see the standings.
Some people ask me about this.
So if you want to see your standings, how many points everyone has, how many points you have, you can click that.
You can click there.
OK, let's get into some poker hands now.
That being said, I highlighted all these differences about cash games and tournaments.
But good poker is essentially good poker.
If you're good at one, you're probably good at the other.
The difference is still rather small.
It's more based on, how much risk do I want to take?
What kind of schedule do I want to play?
And that determines which one you choose to play rather than, am I better at tournaments or cash games?
At least once yours-- except at the very top levels, there there really is no difference between-- there's no such thing as a cash game specialist, except at the very, very top where it's possible the best cash game players are good at cash games compared to the best tournament players.
But when you're just starting out, if you're good at one, you're good at the other, essentially.
All right, cool.
So today, I'm going to talk about some preflop play.
Last class, I focused on post-flop flight.
The first class, I talked a bit about what hands you should open with preflop But I didn't really say that much about playing preflop.
But it's of the most important parts of the hand, because it determines what cards are playing for the rest of the hands.
So first of all, I wanted to run through some numbers.
It's good to have a sense of these.
You don't have to memorize all of these.
But it's good to have a reasonable idea of equities, of specific hands versus specific hands, when you get it all in preflop.
And I've classified this to make this roughly easier to memorize.
So the first example, which is sort of the dream situation, is you have a bigger pair against a smaller pair.
What's the best situation that you can be given preflop in poker?
So, what's the best hand you can have?
It's aces, right?
But you don't want aces and for everyone else to have 3/2 offsuit, because they're just going to fold.
So you want aces, and you want someone else to have kings.
One of my screen names is actually, please give me ace ace against king king or something, on a site, because it really is the best situation you can hope for, because they're going to put all their money in with kings.
And so roughly, when you have a bigger pair against a smaller pair, your an 80/20 favorite.
So you win 4/5 of the time.
And the edge dwindles a bit as you get to like, 33 versus 22, because there's a higher chance both pairs will be counterfeited.
Like the board can come 66 77 8 or something.
And then pocket threes ties pocket deuces.
And an equally good situation is one pair against zero overcards.
So ace ace against ace king off it's also a very good situation.
You're actually 93% there, because you dominate their ace.
So there's very few ways for them to win.
Pocket aces against 6-5 suited-- this is actually the hand that's best against aces, other than aces itself, is 6-5 suited.
And aces is only 77.5%.
But it's still ludicrous.
So pocket jacks against 10-9 suited, 81.7%-- so, some rough numbers.
10-9 suited is pretty good.
But pocket jacks crushes it because having two jacks blocks a lot of their straight-outs.
Pocket queens against 7-4 is very good, because 7-4 off is not suited, not connected.
And the best you can do preflop is actually king king versus king two off.
If you have pocket kings against king deuce off, I think that's the highest equity you can possibly get in preflop is 94.6%.
And pocket kings against king two suited is 89 points.
So look how much better king two suited does than king two off against king king.
The fact it's easier to make a flush is so relevant.
We'll talk more about this later.
OK, the next few categories-- so a pair against one overcard.
So you're still a favorite, but less big of a favorite.
Instead of an 80/20, you're a 70/30.
So queen queen against ace jack off is 71.7%.
Queen queen against ace jack suited is a bit worse.
Queen queen against ace queen suited is actually much worse than queen queen against ace jack suited.
I think this is a bit counterintuitive, because with, against ace jack suited, I always thought they have more outs.
They can hit three jacks.
But it actually turns out, the fact that they have a queen to prevent you from hitting three of a kind in the situation they've already had an ace, is actually a lot more relevant.
So queens is much worse against ace queen than is jack.
And pocket eights against ace two off-- 70.2%.
Pocket threes against ace two off is a bit worse, because once again, small pairs can get counterfeited when you all-in preflop.
OK, a similarly good situation, which is the 70/30, is when you are, quote, "dominating" the other person.
So "dominating" means one of your cards is the same.
But the other card is different.
And yours is higher.
OK, so ace king off against ace queen suited-- pretty good situation to get it in poker.
You're at 70%.
If you are suited, and they are not, then you're 75%.
If neither of you are suited, it's 74%.
Yeah, ace king off versus king queen off.
You're actually a bit better.
So, ace king off against ace queen off, you're at 74.4%.
Ace king off is king queen off, you're 74.8%.
And you're a bit better, because it's harder to tie with the ace.
So when you both have an ace, it's possible the board will come with two pairs, so like 6, 6, 7, 7, 8.
And then both of you will have two pairs of sixes and sevens with the ace kicker.
But that's less likely to happen if they don't have-- like if your bottom card is their top card instead of your top card being their top card-- One thing to keep in mind is ace five off against ace two off is barely a favorite at all.
So it's nowhere near as good as ace queen off against jack off.
Anyone have any idea why that is?
Is it draw?
Right.
Exactly, yes.
So the five and two are going to get counterfeited.
The five kicker is going to get counterfeited a lot more often, because if four cards on the board come, that's higher than five.
The five is going to be irrelevant.
So that's a good thing to remember.
Let's look at some more numbers.
This is one thing that I've always thought is really cool.
Two overcards versus a pair is actually very close to a 50/50 for the most part.
Ace king suited against pocket twos it's 49.9%.
I actually showed you in the first class, if they don't have a two of your suit, then it's actually like 50.1%.
So ace king off versus pocket twos is 47.4%.
Note that 10/9 suited against pocket twos is actually a 54% favorite.
It's much better than ace king against pocket twos, because 10/9 can make a lot more straights and flushes.
And the fact that you're making a pair of tens instead of a pair of aces, is irrelevant when the thing you're trying to beat is a pair of deuces.
Ace king off versus pocket queen is only 43%.
Even though people call it a coin flip, it's actually much worse than a coin flip.
So yeah, this is actually one of the first things that drew me to Hold'em.
When I found this out, I was curious.
I was wondering, the person who invented the rules of Hold'em, did they invent it so that this is a 50/50?
Or was it just by coincidence that is [INAUDIBLE] 50/50?
But I think it's very cool that it turns out to be a 50/50.
OK, so the last cases-- there's some weird cases where all four cards are different.
Roughly speaking, when all four cards are different, the guy with the highest card is going to be a 60/40.
I mean, this is very rough, because it's hard to encompass all the different types of scenarios under the same umbrella.
But the thing that matters is having the highest card.
So let's see.
So AB versus CD, which means you have two cards above their two cards.
You can be as good as 67.7% if it's like ace king off versus queen seven off, because queen seven off has no straight-outs and no flush-outs.
AC versus BD, it's about 60 versus 40.
AD versus BC, it can be as low as 50/50.
So ace two off against 10-9 suited is only 51.6%.
So that's sort of the worse case for AD versus BC.
You can look through those numbers more on the slides.
And you can also get it yourself on PokerStove.
Or a lot of websites, you can compute the equities of a hand versus another.
But it's good to have some rough ideas of these categories.
When is it an 80/20?
When am I a [?
four to one ?]
favorite?
When am I a 70/30 favorite?
When am I a 50/50?
When am I a 60/40?
OK, so one thing that I hope you sort of got from these numbers is that suitness matters a lot when you're behind, and matters not that much when you're ahead.
So if you look at these numbers, the gain in equity of ace king suited when-- sorry.
The gain of equity of ace king when ace king is suited, is only 1%.
You only increase your equity from 74.4 to 75.4.
But if ace queen is suited, then the equity of ace king drops by 4% instead of 1%.
And the reason, essentially, is if your hand is bad, you need as many ways as possible to try to get lucky.
And being suited is one of them.
But if your hand is already very good, you just really need your opponent to not get lucky.
And you being suited is only going to be relevant if they first hit a pair to beat you.
And then you need to hit a flush to beat them.
So that's one thing to keep in mind.
So like with ace king, the difference of being suited and unsuited doesn't matter that much.
But with a hand like 9-8, being suited is way better than being not suited.
And shrewdness is also important for implied odds, as I talked about last class, because you can make much better decisions post-flop if your hand is suited.
So that's roughly some hand [?
equities ?]
to remember.
Now, we're going to talk about some preflop all-ins, so going all-in preflop.
OK so this class-- most of this class, this is going to be a bit boring maybe.
These are not really highlight reel plays.
And this is about routine, making good decisions that increase your preflop equity by a couple percent that will-- but it's really these boring decisions, I think, that make you win right, not the occasional brilliant thing that you see on TV.
So if your goal is to get on TV, then studying this is maybe pointless.
But if your goal is to make expectancy, win money, essentially, then it's really the simple, boring decisions that affect whether you win, not the one-time crazy bluff you pulled off against Phil Ivey that makes you a winner.
Let's talk about this a bit.
So the main thing is, don't be afraid to go all in preflop.
I didn't really stress this yet.
But one thing I tend to see people do is, you are too afraid to put all your money in preflop because you would think it's gambling or something.
It is a lot of luck.
You essentially put your tournament life-- you let it ride on a coin flip.
So it does seem scary.
But the thing to remember essentially, is late in the tournament, the antes and blinds are so big.
And if you ever can win the blinds by going on, that's such a large gain relative to what you risk, and also, any two cards of a chance against any other preflop.
So what was the rule I said?
So the rule I said in the first class is, when the effective stack size is less than 12 big blinds, when there is antes, then just go all in if you're going to play the hand at all.
And if there's no antes, then you need to be a bit shorter, because you don't want to risk 12 big blinds when there's no antes.
So you only want to risk like 10 big blinds without antes.
But this is the rule that I outlined in the first class, roughly, and when you can go on.
And one thing I wanted to talk about, too, before we get to the first example, is from the small blind-- in the first class, I said, your button-opening range is similar to your small blind opening range.
And why was this, even though from the small blind-- so when it's folded to you in the small blind, you may have to get through one more player, whereas when it's folded to you on the button, you need to get through two more players.
So it seems like you could play a lot more hands from the small blind.
And another advantage of playing from the small blind, is you've already put in half a blind.
So the amount more you're going to put in to play the pot is less, is half [?
blindless.
?]
But we said that being in position-- especially [?
when we sell this last class-- ?]
being in position is so important, that the fact that you're in position on the button and not in position was the small blind, meant about balanced-out the advantages of playing hands from the small blind.
So we said, roughly speaking, you should be similarly willing to raise hands from the button and raise hands from the small blind.
But for all-ins, position doesn't matter.
If all the money goes in the pot preflop, then who gets the [?
act ?]
last postflop?
Doesn't matter, because there's no more actions to make postflop.
So for all-ins, your small blind all-in range can be a lot wider than your button-on range.
And furthermore, I think the threshold of how many bets you can have to go all in is a lot higher, because even with like 15, even with 20 big blinds, I often go all in from the small blind if I know the big blind is a competent player.
From the button, I would pretty much very rarely go on for 20 big blinds, because I'm not that sad if I just raise it to 2.25 and then call, because I play the rest of the hand in position.
But from the small blind, when I know that if they call, I'm playing the hand out of position, I really want to avoid this situation.
So going on for a huge amount from the small blind is usually OK since there's only one player behind anyway.
Let's look at a sample situation.
And I'm going to first analyze it theoretically.
And then I'll try to go through a bunch of examples.
So do we go all in or fold here?
And let's suppose we're reasoning exploitatively.
So this was level 2 thinking, not level 3 thinking.
So if we're reasoning exploitatively, what we're asking ourselves is essentially, what's exploitative thinking?
We model our opponent, give them a probability distribution, and then playing in a way that maximizes our expectation relative to the Bayesian probability distribution we put on our opponent.
You can ask yourself, is a blind a gambling Player?
Is it likely he or she will call with a wide range of hands?
How crazy have I been playing?
Will he give me any credit if I go all in?
Suppose the pay bubble doesn't matter.
Suppose it's far from having to try to survive to make the next pay increase or whatever.
So let's do this.
What do we think he's calling?
OK so I want to see what you guys think.
Does someone want to give me a hand that that you think he's calling, or definitely calling, or definitely not calling?
You give me a no-brainer.
I just want people to-- so what's the hand he's obviously not calling?
2-7 offsuit.
OK, good
2-7 offsuit.
What's the hand he's obviously calling?
[INTERPOSING VOICES]
OK, so let's try to zero in a bit more, OK, don't worry.
It's OK.
So tell me what hands you think he might be calling that are a bit worse than pocket aces?
What do you think is the smallest pocket pair he's calling?
If you're not sure, let me ask you.
What's the smallest pocket pair you would call if you don't know anything about the small blind?
So it's 15 big blinds.
It's a 15 big blinds all-in-one.
OK, who would call pocket fives?
Who would call, call pocket twos.
Who would call ace seven suited?
Who would call a queen jack offsuit?
OK, so I guess we have a rough idea of-- so assuming you're playing against someone in this class, let's just say when you look at that sample, just sort of guess what they're calling.
So I had this range.
Maybe it's a bit more ambitious than what people put up their hands for.
I said they would call with any pocket pair, any ace, king ten offsuit, king eight suited.
So this is what I said they would call with just 25% of hands.
Maybe in reality, they call with even less.
Maybe in reality, they only call 20%, 15% of hands.
But this just further emphasizes my point.
So we assume, they call it 25%, let's say.
OK, let's do the math.
Let's now assume we have 10 offsuit.
OK, so let's do the math against this.
Oh, sorry.
So what is the equity of ten eight offsuit against their 25% Range?
So when we get called, we're not going so well, because they're only calling 25% of hands.
So their hand is going to be good when they call.
We're only 36%.
So let's do the math.
So 75% of the time, they're going to fold.
And we're going to win 2.5 big blinds.
Why is it 2.5 big blinds?
There's a big blind that we're winning, there's a small blind that we're winning, and then there's one big blind from the antes that we're winning.
And it's 3.5.
We count the small blind, because even though we put in the small blind, the fact that you put it in the pot-- it doesn't matter who put it in the pot.
The fact that it's already in the pot means we're winning the money.
That make sense?
So 25% of the time, he's going to call.
And then when this happens, 36% of the time, we're going to win the all-in.
And we're going to win 16.5 big big blinds.
Because we're going to win the 1.5-- we're going to win the small blind, the antes plus his entire 15 Big Blind stack.
64% of the time we're going to lose, and we're going to lose 14 and 1/2 Big Blinds, which is how much we wagered to do this all-in.
So you can do the calculation.
And it's actually very positive.
We're making an expectation, an entire Big Blind by going all in.
So now let's suppose we had a worse hand.
So let's suppose we had 3-2 offsuit.
We can do the same calculation again.
The only thing that changed is, now we only have 28% equity instead of 36% equity.
So 3-2 offsuit is actually a worse hand than 7-2 offsuit for a lot of all-in purposes.
Because 7-2 offsuit is, like, the worst poker hand against a range of strong hands.
But against a range of any two cards, 3-2 off is actually worse than 7-2 off because seven high is relevant if they had six high.
Anyways, so you do the calculation, and you find that this is still an excellent play.
So going on with 3-2 offsuit, it's actually not a crazy bad play.
It looks like a crazy bad play on paper.
And I'll tell you, in reality, it is a crazy bad play.
But according to this calculation, it's actually not a crazy bad play.
You're actually earning 0.42 Big Blind.
OK, so what's wrong?
What's wrong?
Why does our calculation say 3-2 off is a good play?
And there's no mistake in the calculation.
It's not because of a mistake in the calculation
Because of the high probability that he's not going to call you
Right, exactly.
OK, good.
Yeah, so basically remember, we did this calculation using Level 2 reasoning, exploitative thinking, right?
We built a model for him.
And if our model is correct, then actually 3-2 offsuit is a good all-in.
So what this is showing us is actually-- the point is, if our model is correct, then they're making a mistake.
Then he's making a mistake by only calling 25% of hands.
It's just too small a fraction.
It's too small a fraction of hands to call with.
So I guess the lesson sort of is-- so when I took the poll around the class, you guys wanted to call even less than 25%, right?
25% still includes Ace-2 offsuit, and still includes hands like-- let's see again.
It still includes hands like Queen, King-10 offsuit, Queen-10 suited, King-8 suited.
So even 25% is sort of too low in the sense that it allows a Small Blind to shove any two cards-- shove means all-in, shove any two cards profitably.
And in reality, it might be the lower if people were being truthful when they put up their hand for what they're calling with, right?
So basically my point is, I think people are too afraid to go all-in preflop.
Going all in preflop is fine, basically.
So let's suppose we're the Big Blind now, and we're considering adding Queen-Jack off to our calling range.
So I'm saying, OK, you're making the mistake as a Big Blind of only calling 25%.
But what's wrong with my reasoning, right?
The thing that's wrong with my reasoning is, that's only mistake from a Level 3 reasoning point of view, right?
That's only a mistake in that it's not theoretically optimal because in theory, the Small Blind could shove any two cards profitably.
But if the Small Blind isn't actually shoving any two cards, then maybe this is actually the optimal range to call.
If the Small Blind is shoving too small, then maybe 25% is the optimal strategy for the Big Blind from that point of view.
So we can do a calculation with Queen-Jack.
So let's assume the Small Blind only shoves top 25% of hands instead of 100%.
Then your equity is only 42%.
And what equity do we need to call all-in?
We need to call 14 Big Blinds, right?
We need to call 14 Big Blinds.
And if we call, the pot will be 31.
So we need 14 over 31, which is about 45% equity.
We only have 42, so we actually shouldn't call anymore.
So it's possible the Big Blinds play is correct.
But the point is, someone's play isn't correct.
So I also wanted this example to give you another idea of what sort of optimal Nash equilibrium play is.
So I'm going to-- so let's draw a chart of what is in game theory called an iterative best response.
So how this works is, I fix a strategy for one player.
I take the best strategy for the other player at exploiting this strategy.
And then I fix that strategy, and I come back.
So I fix the strategy for the Big Blind, which is only call 25% of hands.
And then I ask myself, what's the best way for the Small Blind to exploit this?
Well, it's to shove 100% of hands, right, by the calculation.
So now, if the Small Blind is fixed on this strategy, what's the best way for the Big Blind to exploit this?
Well, it's to call with anything basically, because you know they're shoving anything.
So it's a call with a very wide range, say, 67% of hands.
So once the Big Blind is calling too much, how does the Small Blind exploit this?
Well, now they tighten up, right?
They only shove their good hands.
And that's fine, because they're getting paid off very often when they shove their good hands.
But once the Small Blind realizes they shove only their good hands, then the Big Blind's like, oh crap, I shouldn't be calling 67% anymore.
I'm going to call less, 30%.
So overall, let me draw sort of a diagram of how it's going to work.
So you can see, right?
0, 100 Small Blind.
So essentially how it works is-- let me just put some markers.
So how it's going to start out is, the Big Blind calls 25%, right, which is here, 25%.
And then the Small Blind, to exploit this, shoves 100%.
Then the Big Blind goes to call 67%, and then the Small Blind will respond by shoving 40%.
These numbers are approximate, right?
Then the Small Blind will shove too little, and then the Big Blind will sort of call too little.
And then it'll essentially go here, and then go here, and then go here, and then go here.
So essentially, the point is it's going to converge.
It's going to zero in on this blue line, on these blue lines.
And the thing that it zeroes in on is basically what's called the Nash equilibrium.
So you can actually compute the Nash equilibrium with a computer in this case, which is what I did.
And this is what the Nash equilibrium is.
So the optimal thing to do is for the Small Blind to shove about 66.8% of hands, and the Big Bind to call about 38.5% of hands.
And I think if you play reasonably, reasonable stakes online poker tournaments nowadays, this is what most people know how to do because most people will have a computer program that can do this calculation for them.
But it's quite loose, essentially.
So remember, the situation was 15 Big Blinds, there are antes, and you're in the Small Blind.
You should be shoving 2/3 of hands, roughly.
And the Big Blind should be calling almost 40% of hands.
And this is the ranges.
And as you can see, the Small Blind can shove 4-3 suited.
So suitedness is very important.
But if your hand is unsuited, then 7-6 offsuit is a shove.
Oh, 6-5 offsuit it as a shove, too.
But 9-5 offsuit is not a shove because it says 9-6 off plus, which means 9-5 off is not a shove.
So it's always a good idea to know the Nash thing just for preflop.
Because even if you're trying to exploit your opponent, you want to make sure you're never going too far off from the optimal play.
Because whenever you get too far off, you could potentially be getting exploited.
Yeah, but I mean, you don't have to do exactly this.
I'm not saying everyone should do exactly this, right?
If you think you're smarter than your opponent, and you have a good model than your opponent, if you think the Big Blind is calling way fewer than 38.5% of hands, then you can go in with a lot more than 66.8% of hands.
Just make sure if you do decide to try to exploit your opponent, you do the exploitation in the right direction.
If they're calling too little, then you need to shove a lot.
If they're calling too much, then you need to shove very little.
That makes sense to everyone, right?
That's the right direction to exploit your opponent.
And similarly, if the opponent is shoving too much, then you're going to call a lot.
And if they're shoving too little, then you call very little as well.
But Yeah, but what's good about the Nash equilibrium strategy is you can't be exploited.
You know no matter what they do-- the optimal, basically the reason why this is called the convergence point is because if the Small Blind is shoving 66.8%, the best for the Big Blind to do against that is 38.5%.
And the best you shove against 38.5% calling range is 66.8%.
So it essentially converges, that's why it's called an equilibrium.
So hopefully this gives you a better, a more concrete example for those of you who haven't seen what a Nash equilibrium is exactly, of what I mean by optimal Level 3 reasoning play.
OK, so that's enough of that.
So now let's go do some concrete ranges.
So now you can ask me, how do we learn these ranges, right?
I showed you the Nash range for one specific situation.
But the situations you can enumerate.
There's going to be a lot, right?
So I showed you the range for Small Blind versus Big Blind, and there's 15 Big Blinds.
But what if it's 14 Big Blinds?
What if it's 13 Big Blinds, right?
What if it's 10 Big Blinds with no ante?
So essentially it's a combination of memorization, understanding theory, and then extrapolation slash interpolation.
So let me just show you a few more.
So just to give you a rough idea, this is where the memorization slash interpretation comes in, because if you have a few baselines to go off of, you can sort of use rules that theoretically make sense to extrapolate, right?
So here's button, 10 Big Blinds.
43.9%, and this is roughly the hands.
Oh sorry, is it clear what this range of hands means?
I guess I never really explained.
Basically s mean suited, o means offsuit, and plus just means any hand where the denominations are strictly higher.
If you don't know exactly what it means, it's not a huge deal.
But roughly, you want to sort of know what this is, what this is saying.
So there's a few weird cases, but roughly, if it says Ax+, that means any Ace, any hand with an Ace, suited or unsuited you can go all-in.
So 43.9%-- So in terms of extrapolation and interpolation-- I just want to make sure everyone gets the directions right.
So if I move from button to the cutoff, does the fraction of hands go up or down?
Down
Down, right?
So the fraction of hands goes down because from the cutoff there's more players to get through, right?
So you need a better hand.
You can't-- so the fraction of hands you can shove goes down, right?
That make sense to everyone?
So make sure you get the direction right.
And then what if the number of Big Blinds goes up or down?
So let's say it's 15 Big Blinds.
I guess 15 is sort of too much to go all-in with, but let's say it's 12 Big Blinds.
Does someone have a guess what the percentage might be?
You just have to get the direction right.
Someone?
Yeah?
40%
Yeah, 40%.
That's about right.
That's about right, basically.
So the percentage will go down a bit.
And what if you only have 5 Big Blinds?
Does someone want to guess the percentage?
60%
60%, yeah.
That's about right.
So try to get the directions right.
If you have a few baseline points, and if you can get the directions right, that's a very important first step.
So here's another data point.
Cutoff, 7 Big Blinds.
It's 38.8%.
So notice that the cutoff, this situation compared to the first one, you're one position worse because you're cutoff instead of button.
But you're risking a lot less, so you only have 7 Big Blinds.
But the percentage is still going down.
So I think I said this first class as well, the position matters a lot more than the number of Big Blinds because for the number of Big Binds, even though you are, in some sense, risking a lot more as you have more Big Blinds, it's also harder to get called.
If I go all in for 10 Big Blinds, they're going to fold a lot more than if I go all-in for 7 Big Blinds.
So that's why the position actually matters a lot more than how many bets.
OK, so lojack-- So that's hijack minus one.
So that's two over from the cutoff.
It's only 28-- 10 Big Blinds is only 23.4%, but it's still nothing.
Not nothing.
So under the gun nine handed with 3 Big Blinds, it's actually pretty high.
It's 24.1%.
So you can go look at those on your own.
You can also calculate on your own in certain websites.
Yeah, so this will roughly give you an idea of the shape of the graph.
So this is under the gun all-ins.
It's a bit approximated, but with 15 Big Blinds-- I know I recommended with 15 Big Blinds not to go all-in.
And I think that's a good recommendation, but let's say you told the computer they were forced to go all-in with 15 Big Blinds, and they had to calculate the Nash equilibrium, this is what it would spit out.
It would say, you go all-in with this 6.2%.
And with 10 Big Blinds, it's 13.4%.
With 5 Big Blinds, it's 33.3%.
So it actually increases quite a lot for under the gun.
For under the gun, the amount of the fraction of hands you can wager as a function of how many Big Blinds actually goes up quite a lot
Question
Yeah?
For example, you have 15 Big Blinds, and you get an Ace-Jack offsuited, what would be-- how much would you raise?
Oh, I would raise to 2 Big Blinds, or 2.25 Big Blinds
That's it?
Very little?
Yeah.
For all hands, I essentially raise to 2.25 Big Blinds
But it goes from 2.25 Big Blinds to all-in?
There's no middle range?
Right, right.
There's no in between, yeah.
So the reason essentially is, there's-- so let's say you raise to 5 Big Blinds, right?
This is essentially equivalent to an all-in, except worse.
So why is it equivalent to an all-in?
Because if your opponent has pocket Aces, or a good hand, not necessarily pocket Aces, they're going to go all-in.
And pretty much, you have to call.
I mean, if you knew they had exactly pocket Aces, you could fold.
But it's possible for them to play a balanced strategy where they could have pocket Aces, or Ace-King, or pocket 10s, or whatever, and you're forced to call anyway.
So essentially if you make it any more than 2.25 Big Blinds, you put in enough chips that your opponent can just play a strategy that forces you to go all-in when they want to.
And so it's basically strictly worse than just going all-in yourself.
Yeah, good question.
But yeah, with 15 Big Blinds, I would recommend just raising small, although I don't think going all-in is the end of the world.
OK, so this is roughly what the shape of the curve looks like.
So here's another example.
So for hijack, from 15 Big Blinds to 10 Big Blinds, it actually changes very little.
I guess it changes somewhat-- 23.4% to 28%.
And then with 5 Big Blinds, it goes up very fast.
So as the number of Big Blinds goes down, it increases a lot.
But the difference-- but the change from 15 to 10 isn't that much.
I guess under the gun it's a lot, but for hijack, it's not that much.
So just give you some ideas.
I don't expect anyone to do this perfectly.
I'm trying-- I gave you a bit of data points.
Just try your best to extrapolate a lot, try your best to extrapolate slash interpolate.
If you're off by a lot, that's fine.
The biggest thing is, just try not to get the direction wrong.
Don't think, if I'm one position worse, I can shove more hands, or something.
So a few more pointers about going all-in preflop.
So here, this is definitely a good play, I think.
So you have pocket fours from hijack.
You have 15 Big Blinds, which is more than what I said you should go all-in with.
But I think-- but going all-in is a good play, instead of just raising to two times the Big Blind.
Because small pairs, while having good implied odds when very deep-- so in the last class, I talked to you about set mining.
I said small pairs are great when you have 100 Big Blinds because you can try to hit three of a kind, and just win a massive amount of money when you do.
But they actually have terrible implied odds when it's 20 Big Blinds deep because you only hit three of a kind one eighth of the time.
When you do, the total effective stack size is not enough for you to recoup your losses.
And when you don't, you're just always going to be sitting there with a pair of fours, and there's always going to be higher cards on the board.
It's going to be Ace, Jack, 10, and you're always playing a guessing game.
Does my opponent have a better pair?
And you just can't make good decisions.
So that's why with small pairs, just going all-in, even for a lot of Big Blinds, is reasonable.
So here's another example.
Queen-8's good.
Normally you wouldn't go all-in with that.
You normally would just fold under the gun if you had a lot of chips.
But with only 5 Big Blinds, going all-in is fine.
Another factor that's actually sort of relevant here is-- this is sort of a higher level thing, but it's good to keep in mind is, when you're under the gun, in some sense you have the least to lose by busting the tournament.
So by going all-in, you're risking busting the tournament.
So this is not true in a cash game, but in a tournament it's true.
You want to go all-in from under the gun when you're really, really short.
You have very little to lose by busting the tournament because if you don't bust the tournament, the next hand, you're the Big Blind, which really, really sucks.
So even though you busted the tournament, you busted the tournament in a situation where the next hand you would have had to pay the blind, which is one fifth that your stack, in this case.
So going all-in is OK. Ace-2 suited is another example of a hand that's pretty good to go all-in with.
Because if they have a better Ace, you're still suited, so you have chances of catching up.
And the fact that you remove an Ace, there's less Aces out there for them to have, is quite relevant, actually.
The fact that there's three Aces out there for them to make a good hand to call you with, than being four.
Everyone know what I mean by that?
So if you have an Ace, there's one less out there to go around.
Yeah, so we go all-in here.
And yeah, here we have 17 and 1/2 Big Blinds from the button.
But I think going all-in with Ace-4 off is fine, basically.
So it's a lot.
It's 17 and 1/2 Big Blinds, but once again, Ace-x offsuit has terrible implied odds.
It also protects your small pair shoves a bit.
So when I said this pocket twos hand-- sorry, this pocket fours hand, I said you can go all in for 15 Big Blinds.
But what's one problem if you follow this strategy exactly?
It's whenever you do this, your opponent knows you have pocket twos or pocket threes, right?
Then what can they do?
They can call with 10-9 suited, which is not a good hand.
Like 10 high against a 15 Big Blind shove shouldn't be possible, but they can.
So if you also do this with Ace-x, it protects you a bit.
Yeah, with 7-6 suited, I think it's also OK.
This is a similar story, pocket Deuces.
7-6 suited has great implied odds when you're 100 bets deep, and making straights and flushes is very relevant.
But when you're only 20 bets deep, and you're just trying to make a big pair, essentially, small cards just aren't good at making big pairs, can't make big pairs.
So once again, going on is acceptable.
So one thing to watch out for is-- I actually swept this under the rug in the last couple of slides.
But in all these cases I showed you, there's actually a pretty big disincentive to go all-in.
Does anyone see what it is by looking at the stack sizes?
So they've always been the same actually, but does someone see a pretty big disincentive?
Yeah, Vincent?
Other people have much larger stacks than you
Right, although actually I should say this, other people having much larger stacks than me is actually an incentive to go all-in.
The reason being, it's possible-- let's say you go all-in and then someone calls, and then another guy re-raises all-in and the first guy who called folded.
Then what would happen is, you're still all-in.
And if you win the pot, you now triple up instead of double up, and you still only have to beat one hand.
Did that make sense?
I can draw it if it didn't make sense.
So you go all-in for $5,400.
So let's say this guy calls for $5,400.
Let me use a different color.
This guy calls for $5,400.
And then let's say this guy goes all-in for $12,000.
And then this guy folds-- which he shouldn't be doing because he definitely has odds to call, but let's say he does fold.
Then the situation is great because how the rules work is, now the pot-- so now this guy is going to get back $6,600.
And the pot-- But $5,400 of his is still in the middle.
And essentially, there's this $5,400, this $5,400, and your $5,400.
And you're only against his hand.
And if you win, you get all the money.
So you actually triple up instead of double up.
So that's actually a good incentive.
But yeah, that's a good observation.
Yeah?
Because your going all-in is profitable because there's a high probability of other people folding.
But if two people call or three people call, you get a lot of people can call it, and then your odds are not very good when they call
Right, right.
Yeah, that is relevant, yeah.
So multiple people can call, but the Nash ranges that I presented do take that into account.
So the Nash ranges, the Nash calculations do take into account that multiple people can call.
You have to look carefully at the stack sizes.
Yeah?
[?
They now only ?]
have $2,100?
Right, good.
Good.
Damn, I wish I'd brought one of those gift certificates.
I still have some, but OK.
So this is actually very relevant.
And I swept this under the rug, but let me clear the annotation.
The Big Blind only has four bets, only has 4 Big Blinds in this situation.
And they're basically guaranteed to call.
That's not a terrible result for you because there's a good chance that just have absolute trash, and Queen-7 suited is better than them.
But it's still worse than being able to steal the Big Blind for free.
So the fact that the Big Blind is so short here that they're basically almost guaranteed to call, maybe they'll fold 3-2 offsuit is really, really bad for you.
Because your chances of just winning down-- taking out the blinds without having to win an all-in is a lot lower.
That make sense to everyone?
So it's hard to do when you're just starting out, but it's good to pay attention to all the different stack sizes.
And if there's someone behind, someone in the blinds who is very short and who's almost guaranteed to go all-in, then you should be a lot less incentivized to try to steal the blinds.
Yeah?
All right guys, I'm going to get started again.
So I'm going to give you a few more examples of extrapolation.
I know this is a bit boring, but to be honest, I think-- this is less exciting than playing postflop, and reading your opponent, and trying to read what cards they have, and trying to make tricky bluffs.
But really, in tournaments, the best skill you can have when just learning, just starting out is knowing when to go all-in.
Because even if you're pretty good at it versus very good at it, the fact that, because most hands just come down to going all-in preflop, if you can increase your equity every time you do this by even 1%, that's a huge game in terms of how profitable a player you'll be.
So yeah, it's a bit boring.
But just try to bear with me.
This is, like, the most important skill in tournaments, essentially.
So I wrote down on this graph the most number of Big Blinds I'm willing to risk all-in for from each position.
So I want to sort of illustrate a concept.
These numbers are actually quite ambitious, by the way.
I think I made this maybe four years ago when I was trying to play exploitatively against the fact that the average player was too tight, the average player was not willing to call often enough.
So you would be shove more hands than Nash because most players don't call enough.
Nowadays, I think people are getting closer to Nash, so you can't really be this ambitious anymore.
All these numbers should go down a bit.
You shouldn't be going all-in for that many bets.
But nonetheless, the graphs illustrate the same point.
So I roughly-- I wrote down in black how many bets I'm willing to risk from each position with Ace-4 offsuit.
So notice that from the button it's huge, it's 22.
But under the gun, it's a lot smaller-- it's only seven.
And roughly, it decreases as a 1 over x, curve as a inverse curve.
And that's sort of intuitive because if you shove from the button, you need to be the best out of three hands, so it's like 1 over 3.
And then you got to be the best out of four, so it's 1 over 4, and then 1 over 5, 1 over 6, et cetera.
So the graph is roughly-- I mean, I don't know, let's say 88 over number of hands remaining.
Or I don't know, something like that.
But I want to show a similar curve with 7-6 suited.
So what do you notice about these two curves?
Looks more linear than the other one
Right.
So this one is very curved.
This one is like 1 of x.
This one is maybe there's like-- this one it's a lot flatter, right?
It's 9.5 under the gun, which is more than 7.7 for Ace-4.
But Ace-4, it's 22 from the button, whereas for 7-6, it's only 19.
So the thing is, these two hands are a bit different.
So essentially with Ace-4 off, it's a hand that does poorly against good hands, but well against bad hands because against a bad hand, your Ace high is going to be good.
And against a good hand, you're always going to be dominated.
Like, if they call with an Ace-Jack, or a pair of 7s, you're going to be at 30/70, right Whereas with 7-6, even if they have Aces, I mean, it's bad, but it's the best hand against Aces.
But basically 7-6, you're going to be better on average.
If they call with Ace-King, it's not a good situation, but it's much better than if you had Ace-4, whereas if they have 8-2 offsuit, which is a terrible hand, that actually, theoretically is ahead of yours.
So what's the most important thing about 7-6 suited and going on with it?
Essentially it's having a lot of chips, it's having a large stack.
Because you want them to fold all of their trash hands.
And it's good when they're not going to call you very often.
So you want to have a lot of bets.
Whereas for Ace-4, the most important thing is-- you don't mind if they call, as they call with a bad hand.
So the most important thing you want with Ace-4 is, you want to be later in position so that you're against your hands so that there's less of a chance that someone behind is going to pick up Ace-Jack or pocket 10s.
So when you're extrapolating slash interpolating, try to remember this principle.
So with a suited connector hand, it matters a lot more how many bets you have.
Whereas with a hand with two unsuited but sort of big cards, like Ace-4 offsuit, King-8 offsuit, I guess Queen-7 offsuit or whatever, the thing that matters most is having few players left to act behind.
So I talked to you a lot about how do you initiate all-ins from preflop.
So what about calling other people's all-ins?
So another thing that I sometimes see people do is fold with ridiculously good odds preflop.
And similarly how you can shove any two cards preflop, you can also call a lot preflop because you'll always have some equity.
So yeah, it's difficult to get less than 30% equity preflop no matter what your cards are, unless your opponent has a really strong range.
Like they're really tight, and you know Jacks plus an Ace-King or something.
It's difficult.
to have less than 30% equity if you have an Ace in your hand, right?
Because if you have an Ace, then you're either going to have an over card, at least, to their pair, or be dominated.
But both of those are still 30/70s.
Only if they have Aces are you actually in a really bad situation.
So I wanted to show you a hand here that looks maybe a bit like bad play to some of you, but I think it's very reasonable play.
So we have 9-8 suited, we have 15 Big Blinds so we don't go all-in, right?
Which is fine.
We may get $1.600.
And then it's folded to the Big Blind, but the Big Blind goes all-in for 10 Big Blinds.
So normally, if the Big Blind had covered us, if they had 15 Big Blinds, you would probably fold because you have 9 high.
You probably-- you don't have enough odds to call.
But in this specific case, the effective stack size is not 15 Big Blinds, right?
They only have 10 Big Blinds, so the effective stack size is 10 Big Blinds.
So if you do the math, when the effective stack size is only 10 Big Blinds, basically you have enough equity to call.
Even if they had 12 Big Blinds, maybe it's very borderline.
But even if they have 12 Big Blinds, roughly speaking, you have enough equity to call, right?
This is sort of the same calculation as me saying, if you had 12 or less Big Blinds and you raise, and someone goes all-in, you basically have to call.
This is sort of a similar calculation.
It's the same thing.
You raised, someone went all-in, it's effectively 12 Big Blinds, and you can call.
So this is a totally reasonable play.
And then you just call.
If you do the math, you need 37% equity to call.
It's possible in some corner cases you can convince yourself that you don't have 37% equity, but for the most part, I see a lot more bigger mistakes by people folding this spot.
If you call, you're never really making a huge mistake.
OK, fine, if they're sort of tight, maybe you only had 35% equity, and you needed 37.
You made a small mistake.
But there are situations similar to this where you might have 45% equity and you only 37% equity.
So it's a huge mistake.
Folding can be a big mistake, but calling is rarely a huge mistake when it's 12 Big Blinds or less.
So any hand that we would raise here in the first place, we probably would have enough equity to call.
If I had 7-2 offsuit here, I probably would not call.
But I would not raise here with 7-2 offsuit.
So any that you would've chosen to open here, which is about 30% of hands, or 25% percent of hands I think you can call.
Here's another situation that we sort of talked about already.
You have 4 Big Blinds, and you had to pay a Big Blind.
Basically you pretty much have to play close to any two cards in this spot.
If you do the calculation, right, you only need to call-- I mean, calling is equivalent to going all-in, sort of.
If you go all-in, you know you're getting called because they're going to have almost four to one odds, so they're going to call.
So essentially the calculation is, do I want to wager my entire stack on this hand?
And you're risking 3 Big Blinds for a total pot of 9 and 1/2 Big Blinds.
So essentially, you only need 30% equity to call.
And 30% equity is very easy to have preflop, essentially.
So basically you have to play any two cards.
Queen-8 off is way more than good enough.
Yeah, so this is basically the calculation I just went through.
And yeah, like here I think I would probably play 95% of hands.
I think I'll fold 3-2 offsuit, 4-2 offsuit, but I'll probably play any hand with a card that's 10 or higher, and I'll probably play every suited hand.
So let's talk a bit of re-raising all-in preflop now.
And all the situations I showed you were just going all-in, and someone calling preflop.
So I wanted to show you.
So don't do this, don't call preflop.
I already talked to you about this, but especially when you have 10 Big Blinds.
So once again here, don't just raise when you only have 10 Big Blinds.
That's less than 12, so you can go all-in.
Yeah, so you basically either want to go all-in or fold.
But I want to talk about raising now.
So why is this so bad, but this is OK when we have more Big Blinds?
So now everyone has 19 and 1/2 Big Blinds.
So why is this OK?
The main reason is because-- so in the first case, right-- so I talked about this.
If you get caught, if someone goes all-in, basically you have to call anyway.
So you might as well go all-in yourself because you're not getting away if they pick up a good hand.
So that's the same calculation.
I'm just going to skip that part.
It's the same calculation.
Essentially it's just saying there's no point to not go all-in because if someone else goes all-in, we're essentially roped into calling anyway.
So the second case, we can get out.
We can fold.
So let's keep this in mind, right?
So in this case, we can definitely fold.
So we can do the calculation.
In this case, we need to call 17 Big Blinds to win a total pot of 40.5.
We need 42% equity.
And we almost certainly don't have this because we raised from early position, they went all-in against an early position raise, so they probably have a good hand.
We're almost certainly going to be a 30/70.
So let's look at whether we call or re-raise.
So for re-raise sizing, so now let's put ourselves in his shoes.
If he's re-raising-- sort of the same principles apply when you're re-raising.
So the advantage of re-raising to a small size is that, what if it's deep enough where he could re-raise, and we could four bet, re-raise again, and go all-in.
Then they could fold, if they were intending to do that.
The advantage of re-raising to a large size is you deny your opponent the odds to call profitably.
And once again, if your re-raise size would cause you to commit a critical portion of your stack such that you can't escape, you're going to go all-in anyway, then you might as well just re-raise all-in.
So it's essentially the same principle, but let's see this in action.
So this is what you don't want to do.
And I see some players doing this, which is fine because I've is never really explained this in class.
But if someone raises, let's say to $2,000, you don't really want to just click the raise button.
This is what would happen if you click the raise button.
Because he raised $1,200-- from $800 to $2,000.
So if you click the raise button, this is the minimum amount you could lose.
You can raise it to $2,000 plus $1,200.
But if you look at the odds you're giving him, it's actually ludicrous.
They have to call $1,200.
So they would have to call $1,200, and the pot will be $5,000, $5,200 $6,400-- essentially $7,200.
So it's $6,100, right?
They got to call $1,200 into a pot of $7,200.
So this size is basically way too small.
So how big do you want to make it?
So roughly speaking, you want to make it 2.5 times their open.
I think that's a reasonable rule.
There's a lot of factors that can affect this, but if you're not sure what to do, a reasonable rule for re-raising preflop is 2.5 times what the original guy raised.
If you're out of position, you might want to raise a bit more because there's more incentive for them to call if they can play in position postflop.
But I think in this specific spot, this $5,000 is a good size.
I'm going to skip to this slide.
But in this case actually-- can we see our stack size?
So our stack size is only $20,000.
So in this case, I think going on, it's a bit big.
So the thing is, if you're going to re-raise, it's the same principle again where if the amount you re-raise to is more than a quarter of your stack, which sort of means you're committed and you can't escape, then you might as well just go all-in yourself.
So I'd say in this hand, if you had less than $18,000, I would just go all-in.
Because you can't really escape.
Even in this specific case where you have $20,000, I don't think it's terrible to just go all-in instead of making it only $5,000 because it's pretty hard to fold a hand is good as Ace-King, even if you make it $5,000, and then he goes all-in for $20,000.
But I think $20,000 is enough that you can make it $5,000.
And you can do this with your good hands.
You can also do this with your bad hands, sometimes as a bluff to balance it out.
And you can fold when you do have a bluff and he goes on.
So that's roughly the sizing you want to make at preflop.
And another thing I want to say is, why is calling a preflop raise OK?
So I said in the first class, if no one has entered the pot, and you're just attacking the Blinds, you always want to raise.
You always want to raise to give yourself a chance to win the Blinds for free.
You don't want the Blinds to be able to see the flop for free.
But if someone has already raised, I think it's OK to call.
I mean, a hand like Ace-King is sort of too good, but if you have an Ace-Jack suited or Ace-Queen suited, I think just calling is fine.
So why is this?
So it's because-- the reason why, in the first case, there's no real advantage to not raising, just calling is because if a Big Blind has a good hand, they can still raise anyway.
So it doesn't matter.
But in this case, there isn't a huge incentive to not re-raise, right?
What is the huge incentive here?
So let me go back here.
What's the huge incentive to just call for $2,000 instead of make it $5,000 is, if you make it $5,000, then you give him the option to go all-in, right?
This is a huge difference.
In the case where you're just raising the Big Blind, they can't go all-in.
But here you're giving him an option.
So that's why you just want to call sometimes so that they can't have the option of going all-in.
So this is a good example.
So this is now a different scenario, but it's very-- the Blinds are only 25/50, and players have $8,000, which is, I guess, 160 Big Blinds.
And here, calling is definitely a positive expectancy play because you're in position, it's very, very deep, there's a lot of money still to play for, and you're in position for all the rest of the hand, which is great.
This is a good play.
So if you raise in this spot, it could lead to disaster, right?
So what happens here is you raise, and then now hijack minus one, the guy who originally raised, they now have an option to re-raise again, which is what they do.
And now it's just terrible.
We basically have to fold.
We could call if we're feeling lucky, but we basically have to fold.
And I think calling is actually, in reality, a reasonable play.
But basically, it's a much worse situation than this one where we just called.
So yeah, that's a demonstration.
So if someone's already raised, you can just call.
I'm going to talk more about this next class as well, a preflop strategy against someone who's already raised.
But that's one thing to keep in mind.
Another thing I want to go through quickly is how do you deal with callers.
So I've said that you should never be just calling a pot when the pot hasn't been raised.
But people will inevitably make this mistake.
And usually if it's folded to someone, no intention to pot yet, and someone just calls, we call-- the term in poker is a limper, which is a bit of a derogatory term.
Poker actually has a lot of derogatory terms, which is a bit unfortunate.
But I sort of have the tough decision of either making up my own terminology, but then if you go into the poker world, no one understands what you're saying.
Or I use the same terminology, which is derogatory.
But OK.
So we're just going to call them limpers.
Either way, you need to be prepared and raise their limps.
But you do need to change your raise size if there's limpers.
So let's say normally when it's deep, you decide to raise to 3 Big Blinds, which is fine.
I mean, I said raise to 2.25, but I think it's fine if you raise to 3.
It doesn't matter that much when it's deep.
You can raise to 3 to play bigger pots.
But if there's a lot of limpers ahead, then you want to make it more than 3 because there's already money in the pot.
So if you only raise it to 3, you're actually giving them to good of odds to call.
So let's say in this case, I only make it 3.
If I only make it $150, then by the time it's folded to the last guy, their odds of calling are going to be-- they put in $100 into a pot of #375, right?
So that's like 3.75 to 1 odds, so that's too good.
So roughly, the odds-- the rule is, take whatever size you were going to raise to, and then add a Big Blind for each limper.
This essentially makes it so that the odds are the same as before.
So in this case, if you were going to make it 3, I would make it 3 plus 1 plus 1 plus 1 because there's three limpers.
And of course, if you only have 15 Big Blinds, then in this case, instead of raising it to 6 Big Blinds, you might as well just go all-in, right?
The fact that you only raise to 6 Big Blinds instead of going all-in is predicated on you having at least 25 [AUDIO OUT] to start the hand.
Or something where it's enough to escape if something weird happens after.
I'm going to finish off showing you some more simple preflop situations.
I think-- so the theme this is being boring.
Oh, I'm sorry about this slide.
So being boring, being simple, just playing on simple probabilities and using calculators can get you very far in poker.
And I will go through crazy bluffs and exciting hand reads and stuff like that as well.
But this class, I wanted to stress the simple things that I think are good.
So here, this is a situation.
So here you have 4-3 suited.
You're too scared to go all-in.
Can I raise to 2 Big Blinds as an alternative?
Oh, so you're the button.
So the button only has 8 Big Blinds.
They started the hand with $1,600, and they make it $400 with only $1,200 behind.
So I want to show you why this can go wrong.
Why is it bad if you don't follow the all-in rule?
Because now let's say the Big Blind has 4-3 suited.
Well, if he went all in for 8 Big Blinds, they would have to fold their 4 high.
I'm not going to run through the calculation again.
But the problem is now, if you make it 2 Big Blinds, they could actually call your small raise with 4-3 suited.
And they have 4.5 to 1 odds, and that's definitely good enough odds for them.
So if our hand has 40% equity against his range, then it's going to be +EV.
So essentially they're going to call, and they're not going to raise the 4-3 suited.
So basically they could-- so the fact that we didn't go all-in gives them an option, here.
With their good hands, they can go all-in, and rope us into getting all-in anyway when we're the button.
And with their bad hands, like 4-3 suited that they don't want to go all-in with, they can still call and see a flop.
But the point is, we're essentially-- this is essentially their strategy.
You can read it.
I'm not going to read through this slide now.
But the point is, we're giving them an opportunity to make a better play than they otherwise could've.
If we just went all-in, the choice is theirs.
They either call or they fold.
But here, essentially they can take their best hands, and they want to get all-in with and go all-in, and we call.
Or with their worst hands, like 4-3 suited that can still call to see a flop, they can call and see a flop.
And we're just letting them realize a lot more equity than we need to.
So let's say, OK, fine.
If you want to get around that, you can make it $800.
And if we make it $800, it's more or less equivalent.
So if they were going to fold, then they were still going to fold.
If they were going to re-raise all-in, then we're still going to call.
And then-- it's mostly equivalent.
I'd say in this exact situation, making it-- if you raise to $800, it's pretty much the same as making it $1,600.
But on some off chance, it's still possible you're giving them a free option.
Like maybe their better play with 4-3 suited is still to just call.
And then if the flop comes really, really bad for them, like Ace-Ace-King.
Maybe that's not even that bad.
But 9-8-7 or something, I don't know, they could maybe fold.
So basically I think just keeping it simple is good.
So one concept I'll finish on is, there's this idea of-- so I have talked about-- all this stuff I'm saying about letting them go all-- giving them the option is bad.
But what if you think they're so stupid that if you give them the extra option, they're actually going to make a mistake, and giving them strictly fewer options is actually bad?
You actually just want to give them more options even though it's free options because they might make a mistake.
So this is slicing in from a while ago, but Bill Chen sort of gave a very good theoretical example of this.
So Bill Chen, he actually might give a guest lecture.
The last lecture, we haven't decided yet, but he might come and speak for the last lecture of the class this year.
He wrote the book called The Mathematics of Poker.
He's one of the world experts on the math behind poker.
He's a math PhD.
He's currently a trader at Susquehanna International Group.
Anyways, this is what Bill Chen calls a sucker bet.
And when you see your opponent do this, you should almost be a bit offended.
Like I sometimes-- this occasionally, very rarely happens.
But when it does happen, I'm sometimes very confused because my opponents shouldn't be doing this.
So maybe they're stupid, and they don't go all-in because they're stupid, and they're actually giving me a free option.
But maybe I'm actually the stupid one.
Maybe they think I'm so stupid that if they gave me the free option, I'm going to make a mistake.
So if you do get in a situation like this, it's actually sort of an interesting theoretical question, I think.
If you're sitting in exactly Big Blind's shoes, you saw.
Let's say you know the button is a competent player, too.
This makes it worse, right?
If you know the button is a competent player, then you're saying, I guess this sort of reveals what they think of me, if they're giving me a free option to see the flop here here 4-3 suited.
They must I think I'm going to mess it up so bad that they'd rather give me this free option than not.
But it's an interesting concept to think about.
And sometimes it does happen.
Another sort of example of a sucker bet is, I guess I was talking about last class on the flop, or even on the turn or whatever, you never really want to bet a very small fraction of the pot.
Like, if the pot is $100, and then your opponent checks you, and you bet $1 into $100, it doesn't-- it's not a real bet.
It's $1 into 100.
It's essentially an epsilon bet.
It's essentially a zero bet.
But what does it do?
It gives your opponent a free option, right?
Now they can check-raise you.
If they-- now it's saying, OK, I bet $1.
Now you have a second chance to bet on the flop, right?
You check the flop, but I'm going to bet $1 into $100.
And now if you want to make it $50, you can re-raise my $1 bet to $50.
And now if you want, you can.
And if you think your opponents going to make a mistake, you can try to do this.
It is a very specific exploitative strategy, but it's interesting in that it's so obvious as an exploitative strategy.
Anyway, I think it's an interesting theoretical thing.
So I think one-- a few more examples.
So once again, these are all simple hands, but I just wanted to show you there's-- the only way you can mess up this hand is by not going all-in.
You only have 15 Big Blinds, someone's already raised a lot.
You don't really want to try to do something cute like slow playing, especially-- if you had Aces, maybe slow playing is OK.
But with just Ace-King, any hand is so much equity against you.
And you just want to go all-in.
Here's an example where someone in lead position raises, and you have a pocket pair.
You can go all-in.
And this is pretty good because you do pretty well against their range that calls you.
So this is a pretty good play often.
I'll talk more about this in future classes.
But if you're defending your Blinds often with small pairs, re-raising all-in, even if it's for a lot of bets, is a reasonable play because the hand-- they might even fold pocket 3s, pocket 4s.
I think it's quite likely.
Yeah, I think the Nash equilibrium is to fold pocket 3s, pocket 4s if you have enough chips.
But when they do call you, even if they have a hand like Ace-King, you still have 50% equity, right?
So small pairs are pretty good at just re-raising all-in against late position steals.
So all-in here is good.
So the last example is also easy.
It's essentially saying, so under the gun raises, hijack minus one calls.
You have a great hand on the button, and only 20 Big Blinds.
You just go all-in.
There's no reason to be cute and make it $8,000, I guess unless you were trying to sucker your opponent.
I'm going to stop here, and then-- so the next class, I'll maybe run through a tournament history next class.
I'll show you guys a history of me playing a tournament, and try to discuss some of those decision.
That will be one of the next three classes.
All right, cool.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so today I'm going to take a more empirical approach.
I feel like I've done a lot of theory in this class, so today I'm going to play through the hands of an online tourney, so that I played, and sort of show you the most interesting hands and show you exactly what I did.
I don't think I'm going to finish the whole tourney today.
But I'm going to go through the first part, and I'm only going to talk about a few [INAUDIBLE] concepts, but mostly just go through the hands.
OK.
So it's a $55 buy-in tournament on Party Poker.
And it's a fairly small field.
So this is a tournament I played recently.
OK, so let's just get into the hands.
I'm only going to show you the hands where I did something at the start.
And we will start with a bit of theory, OK.
So this is the situation.
It's folded to this player and-- so we have king and queen here.
And it's at hand where we're in position, and he raised from late enough position where I think our hand is too good to fold.
So if Dreams crushed, if the guy who raised was from under the gun, let's say, then I might have-- I probably would have folded.
Because against an under-the-gun range, king-queen off suit kind of gets crushed.
But against a range from high jack minus one or low jack, I think king-queen is good enough to play.
But the question is sort of what do we do.
Do we raise?
Do we call?
I don't think folding is the end of the world.
I think it's slightly profitable to play, but it's not like you're giving up a ton of equity by folding this.
So what do we do?
OK, so I'm going to start out with a bit of theory.
So I haven't really talked about this.
I want to talk a bit about preflop re-raising theory.
So in this situation, I don't want to just talk about what to do with this hand, right.
This is how throughout this class I've been saying we shouldn't think about poker.
We should always think about the general situation.
What's the range of hands we could have?
What's the range of hands they could have?
And how should both players play their overall strategy with their entire range of hands?
OK, so there's a few interesting things to think about here.
So there's sort of a dilemma.
So if we think about our overall strategy-- so first thing you notice is how many bets deep is it effective.
What's the stack size effectively?
It's about, it's about 28, right, I think.
So yeah, we have about the same amount of chips.
We have like 28 and 1/2 big blinds.
That's clear to everyone?
So it's important that it's fairly deep here.
So the stack size, the effective stack size is 28 and 1/2 big blinds.
So what's the issue?
It's sometimes in this spot, we have a hand that's good enough implied odds to play in position.
But it's not really like a good hand that we're trying to raise and make the pot bigger with.
Like an example of this is like king-jack suited or like king-queen off, I think, is a good example, too.
It's a hand that's good enough to play, but not a hand that's good enough that we'd necessarily want to raise and make the pot bigger.
On the other hand, sometimes we have a hand that you want to raise and hope you get all-in with, like aces, right.
So we have these sort of two types of hands that we want to play in this situation.
But if we play according to our hand, it can become too predictable, right.
So this is like a somewhat intuitive strategy.
So you know, let's suppose this was our strategy.
Let's suppose our strategy was with the best hands that we're hoping to get an all-in with, let's say in this spot, pocket 10s plus and ace-king.
I think that's reasonable as the set of hands we're willing to and happy to go all-in with.
So pocket 10s plus, ace-king, and then we call with pocket fives plus, ace-jack off, ace-10 suited, king-queen off.
I'd say this is a reasonable strategy to play.
But what are some problems with this sort of intuitive, exploitative strategy?
Yeah?
People are just going to fold to your aces and pick up money from you when you don't have the top hands
OK, great, yeah, good.
So that's a good point.
So basically, if we only raise with pocket 10s plus, people could just fold pretty easily, because they know that we have a great hand whenever we raise.
Good.
OK, what's another issue?
This one's a bit more subtle.
What's sort of another issue?
So one issue is when we raise, it's obvious that we have a great hand.
And so on a similar line, another-- yeah.
I'm sorry, did you--
Yeah, if we call them, they know we're weaker
Right, exactly.
OK, good.
So an almost bigger issue here is if we play like this, when we call, people know we can't have aces.
So you know one of the guys behind, like this guy, [?
Aight ?]
might just be able to raise.
And they're going to know we're going to be fairly weak.
And we're going to have to fold most of the time.
So basically, this is not a really balanced strategy.
So what can we do?
Well, we could raise some assortment of hands from the call category to balance it out.
We could call some hands from the raise category a small percent of the time, like play randomized strategy where you raise aces 80% of the time and call aces 20% of the time to balance it out.
There's some other more extreme things you can do that I don't think are good in this situation but are reasonable when the stack size is different.
So I think if it's shallow or, let's say, only 20 big blinds deep, then just raising all hands, just raising the top X percent of hands is a decent strategy, because 20 big blinds is sort of shallow enough where if you raise, you're kind of committing your whole stack anyway.
And if it's deeper, like if it's 100 big blinds deep, some players will like to call 100% of the hands.
They'll just never raise.
I don't think it's like the optimal strategy, but it's not an unreasonable strategy.
If you're 100 big blinds deep, and you have position, it's not terrible.
And then your range is completely disguised, because you're playing all your hands the same preflop.
So these are some reasonable solution ideas.
So here's sort of a better solution.
So this will sound really cool compared to the previous slide, which is instead of raising some of the call hands as a bluff essentially to balance out the times we raise with a good hand, what if we actually-- we bluff-raise the best hands that aren't good enough to call.
So let me just, so we can look at this again, right.
So our strategy was we're raising for value, aka we're raising, hoping to get called with 10s plus and ace-king.
We're calling with sort of the pretty good hands like pocket fives plus, ace-jack, ace-10 suited, king-queen off.
And then we're going to bluff-raise hands that aren't positive expectancy to call, so like pocket fours, ace-10 off, ace-nine suited, hands that are even weaker than the call hands, but in some sense, if we bluff-raise with these hands, the benefit is we're sort of not wasting them, right.
Like let's say with these better hands, like let's say with ace-jack off, I raise, and my opponent goes all in.
Well, I'm probably going to have to fold in this situation for 28 and 1/2 big blinds.
So I sort of like wasted my ace-jack, whereas if I did it with ace-10 instead, then I get to call with my ace-jack, which I know is plus EV to call and not waste such a good hand bluff-raising.
So this is a very important concept in poker.
It's called polarizing.
And in general, you know, it's a sound strategy, right.
By this wasting argument, I'm saying, I'm talking about-- polarizing sounds like a pretty reasonable strategy, right.
Because it ensures that you're not wasting your medium-strength hands to bluff-raise.
So what's an issue with it?
It can be exploited if your opponent calls your three-bet.
So does anyone actually see, does anyone actually see, so let's say, in this case, suppose this was my strategy.
So I'm raising pocket 10s plus and ace-king.
And I'm bluff-raising the hands that I listed here.
OK, I'll give a $20 gift certificate to you.
So like what's a specific board, let's say, where someone can explain my strategy if this was my specific strategy?
This is just for illustration.
So look at my exact range, and can someone say sort of what's a very bad flop for my range?
Yeah
If flop's high cards, because they'll be thinking your range is generally high cards, but those are a bit lower
Right, so if there's high cards, but we do have lots of aces and kings.
Like if the flop comes, like if there's an ace, then it's pretty good for our range, right, even though they might be able to fold easily, but we're still going to win the pot.
But what is like a specific, what's a specific issue-- yeah
Well, if five, six, seven, or in between
OK, good, so five, six, seven is kind of bad for us, right.
Because our range basically contains no fives, no sixes, no sevens.
But even five, six, seven, we still have like a bigger pair quite often.
So what's like an even worse case?
Yeah
High flush draw or a high straight draw
Yeah, but we can have those, right.
We have ace-nine suit in our range.
We got king-10 suited in our range
So then like a low pair, like four-four or eight
Yeah, we don't have fours or eights, but you know on those boards, like pocket jacks, even like ace-king high is a pretty good hand and pocket jacks.
There's something very specific we're missing.
I mean this is just for this example, but yeah
Maybe a pair of queens
Right, OK, good.
So that's sort of what I was looking for.
You can come get this now if you want.
Essentially, suppose this was our exact strategy.
If the board comes queen high, then essentially they know that our range is pretty bad, because we don't ever have a queen.
I mean we could have kings or aces, but all of our other pairs are going to be smaller than queen.
So I mean this is just a specific example, but I'm saying polarizing is, in general, a good concept by this argument that you're not wasting your good hands.
But it can be exploited if your opponent knows the specific way in which you're polarizing, and they just call your reraise and see the flop.
And they'll be able to play very well on the flop still, even if you have some good hands and some bluffs.
It can also be bad against unpredictable opponents, because by the ace-jack, ace-10 argument again, let's say you three-bet with ace-10 instead of ace-jack as an attempt to polarize.
But let's say your opponent is kind of bad, and they might make a loose call in this situation with ace-10.
Then you would have really hoped to have ace-jack instead of ace-10 if they're going to call your raise with ace-10.
So if you're polarizing, you're sort of assuming your opponent is going to play reasonably.
So the point is the reason why I don't want to sort of waste ace-jack by raising is because my opponent's probably going to fold ace-10.
But what if they might call ace-10?
Then I really would have wished I reraised with ace-jack.
OK, the short answer is I'm not going to give a conclusive formula on how to play this situation, because the short answer is it's complicated.
And I don't think anyone essentially knows what the optimal strategy is yet.
But in practice, I think just raising a mix of hands from the call category, mostly the off-suit hands which have worse implied odds, is pretty good as part of your bluff-raising range.
And calling the best hands from the raise category, like aces, like trapping a small percent of the time, just calling aces maybe a fifth of the time, so that sometimes you have it when you call, so that people can't just reraise when you call.
And also raise some of the best hands that aren't plus EV to call.
So polarize a bit.
Essentially, the short answer is it's complicated.
You need to do a bit of everything with all sorts of hands.
But I just wanted to sort of talk about how complicated this situation can be.
All right, so OK, so let's get back to the hand.
So what I decided to do here was following the thing I said about reraising with off-suit hands, I decided to, so I decided to raise here to 555.
And so I think this is about the right sizing, because we're making it about 2.5 times his raise.
And essentially we're bluffing here.
And our hand is good enough where it's not the end of the world if they call, because we have a lot of chances to hit top pair.
And if we do, we're probably good, because they're probably going all in with ace-king or ace-queen preflop.
But essentially we're bluffing.
I'm definitely hoping he folds, and he does.
So OK, we take down the pot.
OK so the very next hand, I actually just noticed this during, when I was replaying through the hands.
So I didn't notice this.
I didn't notice this during the tournament actually.
So what do you notice?
We're actually in the exact same situation again, right.
But we have different cards.
So it's good that we thought about our overall strategy when we played that hand, because you will end up in the same situation again with different cards.
OK, so along the same lines-- but at the time I was probably playing like 15 tables or more.
So I wasn't really paying attention.
I probably didn't remember that I reraised my opponent the hand before.
But playing through this, it actually yielded a really good result.
Because what happened was they didn't believe my three-bet the second time around, and they went all in with pocket sevens.
It's not a terrible hand, but it's definitely, I think, a weaker hand than they would usually go all in there with.
But I guess they didn't believe my reraise after the previous hand.
So this also sort of illustrates the importance of like you said, of sometimes having a bad hand when you reraise, so that people will give you action when you do have a good hand like pocket jacks.
OK, so let's go on.
I'm only playing through the hands that I essentially played at the start of the tournament.
So we doubled up.
The starting stack was 3,000.
We've got 6,000 now.
We get ace-queen, we raise, and we get called here.
And here, this guy makes it-- so this is a different player, because we eliminated the guy who used to sit here.
And they make it 755.
I decide to fold here.
I think ace-queen offsuit is just a bit too weak.
I think going all in is-- it wouldn't be terrible, but I think when we raised from fairly early position, under-the-gun plus one.
And this high jack called.
They're not going to be doing this as a bluff very often.
And their reraise to 755 I think usually commits them.
It's not guaranteed that they're going to go all in if we-- they're going to call us if we go all in.
But I think most likely they will, and they're going to be ahead of our range.
So I decide to fold.
It's a fairly tight fold.
So it is a stack.
I'm just going to continue playing through my hands from the tournament.
If you have any questions, just stop me and ask.
So this is sort of a weird hand, and I actually think I played this hand quite terribly.
So this guy calls, which I said we should never do, but I guess he doesn't take my class.
We get queen-10 here.
We check, we see a flop.
So we flop two over cards, a backdoor flush draw and a gutter straight draw.
So I decide to lead into it, sort of lead my draws out, hope he folds.
He calls, and the turn is a three.
And then here, so this is where I sort of, he decided he suffered at me.
So I talked about how he should only be doing this if he doesn't respect me.
Because I mean in this case, it's not a completely insignificant fraction of the pot.
It's like one sixth.
So he is actually getting some substantial payment into the pot, but it's still fairly small.
And the main thing it does is it gives me an opportunity to check-raise.
But I'm not going to.
I don't think they're going to fold often enough.
There's a lot of draws they're going to call me with, and I'm going to have to bluff again on the river.
So I decide to just call, which is reasonable.
So this is where I think I played the hand poorly.
So the river completes a lot of draws, but it doesn't complete my hand.
And I just check it down.
I just check it, which at the time I didn't lead out, because I thought the fact that I only called the turn and suddenly decided to bet big on the river would probably get called.
But I think it's just not a balanced way to play the hand, because plenty of times I will have clubs, or I'll have jack-10 or something.
I'll have a jack, and I will want to bet for value.
And here when I have a hand that essentially has no chance of winning the hand at showdown.
Like essentially, I'm giving up the pot for sure if I check, because my queen high will never be good.
I should definitely be bluffing.
So I didn't, which I regret, because I let him take down the pot with a pair of threes.
Yeah, I guess he kind of played that hand well.
He called eight-three suited.
I guess he made money from me, because I didn't play very well from the big blind.
OK so the next hand, we get king-jack offsuit here.
We open, probably one of the weakest hands I'm opening from this position.
I think probably in my recommendations, this is not in the list you should open from, high jack minus one.
But I decide to open, because I'm bored I guess.
And we get three callers.
OK, so the flop is four, six, four with two hearts.
So I decide to continuation bet here.
I think it's a fairly marginal spot.
I think just giving up is OK.
I definitely wouldn't have stabbed at the pot if I didn't have the King of Hearts.
But the fact that I have a backdoor pretty good flush draw, I decided to bet and just hope that everyone folds.
And we do get that which is pretty nice.
OK, so you know our stack is hovering around the same size.
We won some hands and lost some hands.
It's still early in the tournament.
The blinds have gone up a bit, but it's still far away from the payouts.
OK, so we get we queen-jack on the button.
And this guy just goes all in for a lot of bets.
I guess it's not that many.
It's like 35.
No, sorry, it's-- yeah, it's it's about 30, 35 right.
So I fold here obviously.
But I thought a bit about his range, and you know it's possible his range is like entirely-- so I talked about this in a previous class.
When do you want to be going all in for a huge size?
It's essentially what hands that are pretty good to get all in with preflop but have terrible post-flop implied odds in this situation.
And what were the hands I listed there?
It's essentially small pairs, like pocket twos, you really don't want to see a flop.
It's quite a good hand preflop but you really don't want to see a flop, because it's pretty much always three higher cards are going to come, and you're just going to be playing this guessing game, trying to guess whether your opponent has a pair.
And the other type of hand is like ace-two offsuit where once again, you have an ace preflop which is decent.
But you're just going to be playing a guessing game on most flops.
So you know probably if I had a very strong read on him, and if I had jack-10 suited here, calling is maybe not that bad, because you're like a coin flip against ace-two offsuit.
And you're like 54% favored against pocket threes.
So I folded here, because my hand isn't suited, and just because there's a small chance they're going to have ace-queen.
And my read was nowhere near strong enough to conclude that for sure.
There range was only small pairs and a six.
OK, so we lose a bit of chips.
We're blinding down a bit.
Here we get 10-2 suited, so I call from the big blind with 4 and 1/2 to 1 odds.
I think this is probably one of the weakest hands I'd call.
I might not call eight-two suited although it's close.
I mean, you could make a case for calling eight-two suited.
So once again, my odds were really good, but the reverse implied odds do sort of work against me quite a bit since I'm going to be out of position, and I have small cards which will usually make a smaller pair than them.
But I decide to call, and we flopped a flush draw, which is pretty good.
But I'm still going to check, because this is a board where I'm just going to check to the pre-flop raiser and let them continuation bet.
So I talked about leading in an earlier class, and I don't think this is a board where we should be leading.
I think never leading is OK as well, so.
I decide to check, and they check back.
So we turn the flush, which is great.
We bet 600 into 1080.
So unfortunately, we don't get any action.
So we take it down.
So for the next-- I think this is one more a bit later.
So under-the-gun raises, small blind queue is very, very short.
He only has 7 and 1/2 big blind calls.
And we decide to call with our 7-- with our 6 and 1/2 to one odds here, or six to one odds.
So we flop queen, jack, five.
So we have second pair with a backdoor straight draw and a backdoor flush draw.
And the small blind goes all in for 1054 into 1380.
So this is sort of I think a pretty close spot.
I think if I knew for sure that Arden1977 didn't have a hand here, I think calling is definitely fine.
We're given about 2.3 to one odds.
And I think second pair with these backdoor draws is more than good enough against his range.
Because I think his range will mostly be draws and top pairs and second pairs, probably not bottom pair that often, because they did call preflop from the small blind, which I don't expect most players would do with a hand like seven-five suited.
So we're not even that happy to call him to be honest, but I probably would call.
But the fact that this guy here is behind, and they could raise, or they could call, and we'd have to make more decisions on future streets, I think really hurts us.
And I think we should have folded.
At the time I decided to call.
I don't know why, but I don't think this is a good play.
I think it's just too risky essentially.
It'll work most of the time.
Because most of the time [?
Arton ?]
will fold, because they're going to see two players putting money in.
But the times where they do have something, I'm going to have to fold, and it's going to suck.
So we do call.
They actually had the same hand as us.
It worked out well.
I guess we sort of took a bad beat, because we could've made a flush, and they couldn't have.
We were technically ahead, but we get half the pot.
So I'm not going through my tournament history from one month ago and saying I played every hand perfectly.
When I look through here, I definitely-- there's a lot of mistakes.
There's a lot of hands I wish I could replay.
So this hand, I guess, is fairly simple.
So we have ace-king, and I think it's definitely good enough to get it all in here with.
So as far as what my range is I think-- [INAUDIBLE] I think-- sorry.
Did you have a suggestion or?
Oh, no, no, I just saw this guy's handle, and I thought it was funny.
OK, so you thought it was who?
Funny
Sorry, sorry.
OK. No, yeah, don't ask me what my hand means.
My handle is--
What does it mean?
[LAUGHTER]
I can't even read it.
I think it stood for something.
I probably made it when I was like 17, and I think it stood for something.
I probably don't remember what it stands for, OK.
So we go all in here.
So this is sort of a no-brainer hand, but let's stop and think a bit about what our range should be.
So I think with ace-king, ace-king and pocket tens, I think it's fairly easy get it all in.
I think with ace-queen suited, I would get it in.
I think if I had ace-queen offsuit, I would probably fold.
Although with ace-queen suited actually if this Jet Ski Fun guy had as many chips as us, then maybe I would have folded.
But the fact that we're actually not risking our entire 6250 against the two players who are in the pot is very relevant.
Because [?
Armond ?]
covers us, and they could in theory pick up pocket aces.
But it's much less likely for them to pick up pocket aces than this Jet Ski Fun guy who's already in the pot.
So the fact that they may have [?
SEAM ?]
less than 20 big blinds as well helps our case a lot.
So I would probably get it in with ace-queen suited plus and then pocket nines plus, probably, is what I would get it in with here.
So we go all in, and we beat ace-jack, which is good.
So I think ace-jack is slightly on the loose side.
I mean, I think it's a fine play definitely to get it all in with ace-jack, but maybe they could have-- because they do have more than 20 big blinds here.
So I think maybe they could have considered calling or raising small and folding if someone like us entered the pot.
But nonetheless, I think they played their hand fine.
OK, so the next hand we have a pocket nines.
So [?
Manny, ?]
this guy, raises.
So we're getting deeper now, because we won the last hand.
So now we've got, we've got 40 big blinds, and we call.
This guy also calls and [?
Liamkid ?]
goes all in for 30 big blinds.
So they get out of the way.
I think this is actually a very close spot.
I do think one big incentive to get it all in is I think it's not that likely that [?
Liamkid ?]
has pocket kings or pocket aces.
Because I think there's a good chance they would just raised smaller, maybe, to try to sucker people in.
So I actually think I'm not losing to too much here.
And same with [?
Armond.
?]
I think it's also very hard for [?
Armond ?]
to have a good hand.
So even though in the first example, we talked about how sometimes calling with pocket aces here is a good play, but you know, I said if one guy raised, and you're the second guy, you should maybe call with pocket aces like 20% of the time.
When it's two guys who have-- like when it's one guy who has raise, me who has called, and now you're deciding what to do with pocket aces.
Instead of 20% you should be calling, it's literally like, maybe like 2% or 1% we should be calling with pocket aces.
So if you multiply 1% by the probability of getting dealt pocket aces, which is around one in 200, it's literally like one out of 20,000 that he has pocket aces here.
So I'm not too afraid of him.
So I really think getting it in is fine.
But at the same time, I don't think I'm ever a favorite here.
I think their range is essentially ace-king, ace-queen, maybe ace-jack suited, and then, essentially, pocket 10s, pocket nines, pocket jacks, pocket queens, maybe occasionally pocket eights.
So I could have done an exact equity calculation on PokerStove.
If I did, it probably would have been very, very close.
But I decided to fold, and they take down the pot.
OK, so the next hand we get a ace-jack.
We raise from under-the-gun.
This guy goes all in for six big blinds.
We're basically committed here, even if we think he's crushing us.
So I call, and we lose to ace-king.
Not really too much we can do here.
Although this does illustrate one concept, which is when there are stacks behind like [?
Zarbizan ?]
who are short enough that they can commit you by going all in, you should be slightly less incentivized to open speculative hands here.
Like say I had 10-nine suited, I think while normally I would consider opening 10-nine suited here as a steal.
It's a huge disincentive to steal here with 10-nine suited when someone like [?
Zarbizan ?]
can just pick up even a hand as weak as ace-jack suited or ace-jack offsuit and go all and beat our 10-nine suited.
But ace-jack is more than good enough to still raise here.
We unfortunately run into ace-king.
Next hand we get ace-jack again.
The same guy who just doubled up through us goes all in, and here I call him.
Because even though there's more chips, the positions are a lot later, and specifically, they went all in for 14 big blinds when no one else was in the pot.
So their range is, their range isn't super duper wide, but it's definitely wide enough that ace-jack I'm happy to call here.
As far as what my range is, I think I wouldn't call with ace-10 offsuit, but I would call with ace-10 suited.
So probably ace-10 suited, and I think I'd call with king-queen suited but not king-queen offsuit.
And the smallest pair I get it in with is probably pocket sixes, I think, maybe pocket five is OK. Yeah, probably pocket fives is too loose, probably pocket sixes.
So we call.
So unfortunately, we lose to king-10 suited.
So he doubles up through us again, and we're kind of short now
I have a question.
On that last hand, why go all in yourself?
Why not just call his all in?
Oh OK, good question.
So yeah, this is a good question.
So in this case, the two plays I'd say are fairly close to equivalent.
Because OK, so what's the defense, right?
They're going to be the same almost always, because Check Raise is just going to fold.
[LAUGHTER] So what's the difference?
The difference is if we don't go all in, then Check Raise let's say, he picks up aces, right.
Then he can just go all in, and we're basically forced to call him, because we can't-- if we put in 4,118, and they go all in, we can't really fold for only 3,000 more.
We can't really fold preflop with four to one odds
Would that change if you're the big stack?
If like, you had the Check Raise stack, and he had your stack?
Oh yeah, for sure.
Yeah, that's a very good point.
Yeah, for sure.
So if we had the same stack as Check Raise here, I would just call.
And then if Check Raise goes all in, I would probably fold.
But I mean, I'm also going to call here with pocket aces, so they can't just look down at pocket twos and go all in and bully me around.
So my strategy if we had their stack would be to call my entire range that I plan on playing.
But in this case, the only thing calling does is it gives him more options.
Like let's say they had a very marginal situation, like let's say they had king-queen suited.
I think if I only called, they could maybe argue for calling themselves with king-queen suited and seeing a flop.
Whereas if I go all in, I think they're forced to fold king-queen suited
Basically, you have a better chance at making him fold
Exactly.
I'm just giving him fewer options.
But it's very, very marginal.
It affects it basically specifically for me the king-queen suited, like it effects basically for one hand or maybe like two or three hands.
But you might as well minimize the options you give your opponent.
Yeah, like I'd say the minimum stack size I would need before I call instead of go all in with is maybe, I think maybe like 13,000.
I think at 13,000, it's sort of more than I want to risk against the pot.
So now we're kind of short.
So we blind out for a bit.
We don't get cards.
And when you don't get cards in a tournament, there's not really much you can do other than blind down, not just go all in with crap and just lose.
So we blind down for a bit, but we're fortunate to pick up kings.
So this is sort of a no-brainer.
We go all in.
There's no point trapping, because they're guaranteed to call us.
And we get called by ace-five, and we win, which is good.
The next hand we get eight-seven offsuit, and against a button raise, I think calling is fine.
I think their range is just going to be wide enough, where even though the reverse implied odds, once again, really don't work in our favor, we just have to sort of realize the equity from 4 and 1/2 to 1 odds, so I'm going to call.
So we flop nothing.
I'm planning to just check-fold.
So I check.
They check back.
The turn's a four.
Once again, we have nothing.
I could bluff here, but I figure I'm not really looking to make a big bluff with this hand, because I have no draw, no pair.
There's no point.
But I will probably look to make a bluff on the river if they also check, just because whenever you have no chance of winning the pot and it's the river, then making a bluff is not unreasonable.
But in this case, bluffing is kind of bad, because with any club, they might just call me.
Because any club is going to have some equity.
So I check.
They check back.
The river is a fourth club, which is, I think, a pretty good card for me.
Because I can easily represent a small club, and if I did have a small club, I am definitely value betting.
So it's a pretty good card for me to bluff, and they're probably going to have to fold king high.
So I bet, and I don't think I bet fairly big.
I'm just looking for a size where they have to fold sometimes, and they'll have to pay me off sometimes when I have a small club.
So they raised me here, which is pretty weird.
I still don't really have a good idea of what I think he has.
I mean I'm certain they have me beat.
You know so I just fold.
We could think a bit about doing something crazy, like go all in, but it would just be weird.
I think it's unnecessary.
We've got nothing.
So I just fold.
I don't really know.
Maybe they just had like a slow played flush.
It's possible they had like a king high flush here.
And they slow-played the flop.
They decided to slow-play the turn again, because the turn sort of didn't hit anybody.
Yeah, I'm not sure.
So I fold here.
I mean, it's possible they just had a high club.
But I feel like with a high club, they should have bet at some point, just because a good flush draw is just as enough equity.
But nonetheless, I guess we'll never know.
That's one benefit of calling.
You get to find out what your opponent had.
So the next hand, we have queen-jack.
We've got 12 bets.
I just go all in.
12 is around the cut off, like I said, 12 is sort of the cutoff where if you have 12 bets or less, instead of raising, just go all in.
And that's what we do.
So we get called, but-- so this is actually a pretty sick board.
Yeah, so they flopped a full house, but we rivered a higher full house.
[LAUGHTER] Yeah, it's sort of a waste, because normally if this happens, you can win like 100 big blinds from them probably.
But instead we only won how many, eight big blinds.
So it's sort of a waste but whatever.
OK so the next hand, I got pocket fives, and we raise to a thousand.
And [?
Liamkid ?]
goes all in again.
And you know, it's irritating, but once again, I have to fold.
I think I just-- I don't have enough equity to call, given his range.
Once again it's a situation where it's a fairly big all in.
So I could maybe not put them on aces, although given that they've done this once, they're more likely to just do this with their entire range and play a strategy where they're just going on preflop a lot.
So maybe they could have aces in their range, but nonetheless, regardless, I think it's not even that close.
I just fold.
Yeah, we don't really want to call in frustration, because you saw what happened to the guy with pocket sevens when he ran into our pocket jacks in the earlier hand.
OK, so I'll take a short break there.
And then I'll finish up the rest of the hands in like two minutes.
OK, so I'm going to get started.
The next hand we get we king-eight offsuit.
And once again, it's one of these situations where we're not thrilled to be calling, especially now that our odds are even a bit worse, because they actually made it 2.25x, actually a bit more, like 2.35x instead of 2x.
So against Degenerated, but king-eight offsuit, two big cards, I think, with when we're sort of not that deep, the reverse implied odds don't work against us that much, so we call.
And we get a good preflop.
[LAUGHTER] So we check, because once again, I'm checking my entire range.
That's how I choose to play my strategy.
And they bet.
And so this sort of, I think this is a somewhat close situation, because there are some draws.
There is a flush draw, and there are a bunch of straight draws if someone has jack-10 or queen-jack or queen-10.
So I do think there is a decent amount of incentive to just check-raise all in.
So on the second lecture, I showed you a bunch of examples where on boards like this, you can check-raise all in with your good hands and check-raise all in with your draws, and it's a pretty balanced strategy.
Because when you have a good hand, you have a good hand.
And when you have a draw, you have lots of outs when you get called.
And another advantage of going all in is-- let's say I had like 10-nine, I would probably want to be raising.
Because with 10-nine, even though we have a pretty good hand, we sort of want to protect our hand.
We don't really want a jack or queen or an ace to come that much.
So it's close, but in the end, I decided to call.
And I think I decided I'm just going to play pretty close, pretty much my entire strategy, I'm just going to call here.
As tempting as it is, like let's say I had ace, Two of Diamonds, or let's say like queen, Six of Diamonds to check-raise all in, I think it's a board where it's already paired.
And if I have a flush draw, I still have lots of opportunities to bluff on the river.
And if I have a good hand, there's not that many outs, especially when I have a king.
So I'm just going to call and hope that the board comes out well, which I guess it does.
So I check again, because this is how I'm playing jack-10.
This is how I'm playing all my hands essentially.
So they check back.
The river is kind of bad for us.
But I think it's hard for them to have a king at this point, so I just bet out about a bit more than half the pot, maybe 60% of the pot, yeah, 60%.
Oh, so they actually call me with ace-seven, which I think is pretty reasonable.
I think ace-seven is a fine call against my strategy, because I could have easily had jack-10 or queen-10 or queen-jack here.
Although I probably also play a nine the same way, so they are also paying off my nines.
So one good thing, actually.
So this is one interesting thing to think about that's sort of advanced, but I think it's cool is the fact that they have ace-seven, I think, makes their call a lot better than if they had, say, ace-10 or ace-jack.
And the reason for-- so even the ace-seven is in some sense a worse hand than ace-10 and ace-jack, they're the same on this board.
And the problem with ace-10 and ace-jack is if they have a 10 or a jack, it's less likely I have a 10 or a jack.
So when they're calling with ace-seven, essentially what they're hoping for is hoping for me to have jack-10 or queen-10 or queen-jack.
So if they have a 10 or a jack, it just decreases that chance.
So it's almost like ace-seven is a better hand to call with then ace-jack.
But nonetheless, with any ace, I think calling here is reasonable, because I do have plenty of bluffs.
Any missed straight draw, any missed flush draw, I will bluff in the same way on this river.
OK, so the next hand.
We won a bunch of pots, which is nice.
So we built our stack up.
So we get pocket fours here, and this guy raises it to 1,200.
Pocket fours is definitely a hand where I want to be bluff-reraising with.
So I'm not I'm not hoping to get called when I reraise here, but it's a hand that does quite well against the range of hands I get called by, because a lot of times it'll just be ace-king, ace-queen.
And they will fold quite a large percent of the time.
So I definitely want to raise.
The only question is how do I raise, right.
I could just go all in, which has the benefit that they can't call and try to hit a pair.
Because that would actually be bad.
Let's say I only reraised to 3,000 here.
It's actually kind of bad if this guy calls with this queen-jack trying to hit a pair.
Because that's actually the best strategy.
Basically, their obvious strategy is good against my hand.
So I want to either go all in, which sort of denies them that opportunity, but the problem with going all in is let's suppose Check Raise picks up a hand.
Then I'm going to be losing with my pocket fours.
So there is this delicate balance here.
And this is why I think tournaments are interesting, because there's plays that are good against big stacks and plays that are good against small stacks.
But in reality in a tournament, there's going to be big stacks and small stacks at the table, and often you do have to compromise between the lesser of two evils.
And that's sort of what I do here.
So I decide to raise to 3,600, which is bigger than I'd usually make it.
Usually, I'd make it maybe like 3,000.
But when this guy is this short, [?
RoflLolBoom, ?]
I don't really want to let them just call, but on the other hand, I want to be able to fold if one of these two guys, Check Raise or [?
Vampson ?]
picks up a hand.
So I sort of compromise between making it 3,600.
They both folded.
This guy also folds, which is, I guess, an ideal result.
The next hand we have eight-queen offsuit, and we make it 1,600.
So I know I said make it 2.25x, and I stand by that.
I think that's a good strategy.
I think at the time I'm just, so once again, I'm probably still playing a lot of tables, and I'm kind of lazy.
So I'm too lazy to type in 1,800 manually, whereas the software defaults to just 2x.
So I tend to just 2x, but I think it's probably not the theoretical optimal strategy.
I would guess that the optimal thing to do is closer to 2.25x.
So [?
Vampson ?]
calls us in position.
So immediately I could think of it like, what is their range?
Because it's a spot where I expect him to go all in with most of their good hands.
So the question is are they trapping me, or do they literally just always have a hand like jack-10 suited, king-queen, or something like that.
So immediately, it's sort of an interesting spot, because I expect them to mostly be raising when they want to play their hand.
So we flop four, 10, nine.
And it's definitely a pretty bad board for us, overall.
Like I think if, like regardless of their strategy, like if their strategy is to trap us sometimes, then this board is just terrible.
And even if their strategy isn't to trap us, they've always got a straight draw on this board, because they're going to have king-queen or queen-jack or something.
So I think objectively, it's a pretty bad board.
Maybe they could have ace-jack or ace-queen themselves.
Although ace-queen, I think they're definitely raising preflop, so probably just ace-jack.
I do think we're not in great shape here, but I decide to bet anyway, essentially just banking on my Ace of Clubs.
I definitely wouldn't be betting this without Ace of Clubs.
But my strategy is to bet, and if I get called, I'm going to go all in on any club turn, any jack, queen, or ace turn.
So essentially, any club turn I'll have the nut flush draw.
Any jack turn, I have a good straight draw with two overcards, and any queen or ace turn pairs with my hand.
So I just think there's enough turns where I can basically continue bluffing, continue barreling on.
That one barrel has to be OK.
But I'm only making this play, because I have the Ace of Clubs.
Even with the Queen of Clubs, I probably would've just given up this hand.
But we do get a fold.
So I guess they probably had ace-jack.
That's probably their most likely hand, maybe something like pocket sevens they folded.
Yeah, so that sort of shows you the importance sometimes of backdoor draws.
Because let's say they had pocket sevens here.
Let's say they called with pocket sevens, which I don't think is unreasonable.
If I didn't have the Ace of Clubs, I'm going to just have to end up giving up on a bunch of turns, and they would take down the pot with their pocket sevens.
But because I have the Ace of Clubs, it's just very, very unlikely they end up winning the hand with the pair of sevens.
Because I'm going to shove a lot of turns, and they're just going to have to fold.
OK, so the next hand we get queen-jack again.
And we raise here.
And so this same guy, [?
Vampson, ?]
they call.
And we get a good flop.
So I bet-- OK, so I'm going to talk about this.
So I bet 1,600 here, which is quite small.
It's like 40%.
It's like less than 40% of the pot.
I might have done it out of laziness, or I might have just done it because of my cards, because I have two pair, and I'm trying to sucker him in and get him to put in all this money slowly.
But looking back, I definitely don't think this is how I want to be playing my entire strategy on this flop.
I think I want to be betting at least half the pot when there's a flush draw, a straight draw, like a bunch of straight draws.
I definitely think this bet is way too small.
On average, even if I have a pretty good hand, let's say like queen-nine, there's just a lot of bad cards for me that I don't want to be betting this small on this board.
I don't know if at the time I just misclicked and messed up, or I was lazy and I just wanted to make it 2x, or I specifically had a read that I could exploit this player by betting small.
But this is not how I would like to balance my strategy.
And If you ever see me doing this, most likely I had a really good hand, as I do in this case.
So unfortunately it doesn't work.
So I'm going to talk about this a bit more.
So I'm going to jump back to theory just for a bit.
I don't think I've talked about this before, but I want to say that bet sizing depends on board texture.
So let me explain what I mean here.
So essentially, there's two types of boards.
There's dry boards and drawy boards.
So I hate this terminology, because these are two words that mean opposite things, and they sound very similar, especially if you don't enunciate.
But it's dry and drawy.
Maybe I can't pronounce them well.
Dry boards is basically boards where the winner is mostly decided before the river, and you're either way ahead or way behind.
Drawy boards is basically boards with a lot of flush draws, straight draws, where every subsequent card can change the board a lot.
And it's hard to fold, because all hands have equity.
So in reality, most boards, they're somewhere in between.
It's a spectrum.
So what are some characteristics of dry boards?
Well, a board tends to be dry if it's paired, like four, four, eight.
That tends to be pretty dry, because not much is going in.
Most will have missed that board.
A board is dry if the highest card is big.
Like if the highest card is an ace, then against someone with an ace, if you don't have a face flush or straight draw, you're already kind of drawing close to dead.
Whereas if the highest card is a 10, then if you have a queen, you always have an opportunity to turn a queen and beat them.
It's dry if there's no middle cards for straight draws, if there's no flush draws, or I'd say, if there's already four to a flush or already four to a straight, then it's also sort of dry, because-- like if there's already Four of Diamonds, then it's also sort of decided who wins already.
Like if you have a big diamond, you're pretty much guaranteed to win, barring some full house.
Same with four to a straight.
If it's like nine, 10, jack, queen.
If you have a king, you're just pretty much guaranteed to win.
And so on the other hand, what are drawy boards?
It's sort of the opposite, right.
Small cards, why small cards?
Because everyone's going to have overcards, which is six outs.
Flush draws or three to a flush is also very drawy, because if there's three to a flush, then someone with just any card of that suit has a flush draw.
Boards with straight draws.
So how should your play change on these two types of boards?
Well, on dry boards, you can bet pretty small fractions of the pot on the flop and turn.
And your opponent might not have the odds to call.
So like if the flop is ace, ace, two or maybe ace, ace, seven with no flush draws, you can bet like a fifth of the pot, and your opponent is still going to have to fold a lot of their hands.
Because if you've got an ace, they're just way behind.
What's another characteristic?
Is any draw might be good enough to make a bluff.
So like if the board is ace, ace, seven with no flush draw, and you've got like eight, nine with a backdoor flush draw, that's a pretty good hand to make a bluff.
Because you've got three to a straight and three to a flush, which is sort of the most you could possibly ask for on that flop with respect to draws.
And another aspect of playing dry boards is slow playing.
Tricky plays are good.
Because the thing that matters isn't preventing your opponent from seeing the turn and river and not letting them see outs.
The thing that matters is sort of convincing your opponent that you have a good hand when you don't or convincing them that you don't have a good hand when you do.
So it's all about tricking your opponent, disguising these tricky-- I mean, I shouldn't say that.
Simple play can still be good on dry boards, but in general, slow playing is reasonable And tricking, trying to trick your opponent is more important.
Whereas on drawy boards, you just want to bet large fractions of the pot before the river, because every hand has so many outs.
You really don't want to be giving them good odds to call.
And on the flop and turn, if you don't have anything, then just don't put any money in.
Because bluffing is pretty moot, because so many hands aren't folding, because they have outs.
And yeah, don't really slow play.
If you have a good hand, similarly, just put all the money in right away, because they're going to call very often, because they're always going to have some outs.
So that's roughly how the play changes.
OK, so I'll come back to this.
We'll see these hands in a bit.
So coming back to this queen-jack hand, the reason why I think my bet is too small is because I would consider this queen, jack, three with a flush draw to be a fairly drawy board, fairly close to the drawy end of the spectrum.
And I said you have to bet big fractions of the pot and here I am betting close to a third of the pot on the flop.
So I think it is not a good strategy, but I take the pot down.
OK, so the next hand, we have ace-queen offsuit.
So this guy calls.
My rule was roughly raising it to 3x plus one.
So you could count how many limpers there are before you, and you add three to that, and you multiply that by the big blind is roughly what you raise it to.
So in this case, 4,000.
I make it 4,250.
If they decide to do some weird plays, by the way, like if they go all in here, I'm definitely calling.
I am just not going to believe them, and ace-queen offsuit is definitely good enough.
So they decide to just call, which is a bit weird considering their odds aren't that good, and they're out of position.
We flop three hearts, and we have the Queen of Hearts.
And in this situation, I'm just betting pretty big into the pot.
So I consider this to be a fairly drawy flop, so I bet big.
Even though I'm in position-- so position is sort of an incentive to maybe bet smaller, because you have position on later streets.
And you can control the betting better on later streets.
So this is a fairly big bet considering I'm in position.
But I'm just happy to get it in here.
So contrast this with an earlier hand.
I think I also had ace-queen, except I only had three to a flush.
And I said it was very important that I had the ace of the suit.
But here it's just very hard for me to be in terrible shape.
Like in theory they could have, I suppose, ace-nine with an Ace of Hearts.
Like that hand would be crushing me, because I would be drawing to literally three outs to carry my queen.
But it's just there's not that many combinations, because it would involve an offsuit hand, which they're way less likely to have when they call me out of position.
Or it would involve a hand like pocket kings with the King of Hearts, which is also pretty unlikely, because they probably would have done some raising preflop.
So I just think no matter what they have, I'm going to be in great shape here.
I just have more than enough outs to just gamble.
And so they fold.
OK, the next hand we had queen-10 off.
So I opened from under the gun.
I think it's fine.
It looks a bit loose, and it is, but I think in terms of raising some smaller cards to balance it, so that I will sometimes have smaller cards in my hand.
You know you can't really ask for more than a suited connector.
So queen-10 suited, I raise from under the gun.
And the big blind calls when they only start the hand being 18 bets deep.
So we flop two, two, 10.
So this is a good example of a very dry board, where there's really not much going on.
And it is possible they have a deuce, but just in general, there is no straight draws, no flush draws.
I guess there are some overcards, but still, it's pretty close to as dry as you can get.
And they checked to me, and I bet 2,000, which I think is actually too big here.
To be honest, I don't think I needed to risk this much.
I mean, I have a good hand but if I didn't have a pair, I only want to be betting like 1,500.
I think that's a fine size.
I think betting 1,500 is more than enough on this board, just because there's really nothing going on.
But I guess I was lazy.
I bet 2,000, and they call.
So as you can see, there's a huge difference between my flop bet sizing depending on the texture of the board.
And that's sort of-- that's the point I wanted to drive across through some of these examples.
So they call me.
So we turn a six, and they check.
I'm just going to keep it simple and bet.
I'm not going to try anything tricky.
And obviously I'm not folding with top pair and a flush draw.
So they go all in.
I call.
So we river a flush, and it turned out to actually be necessary, because they had a deuce.
So it is possible here, I think, for them to have a deuce.
I think their play preflop is a bit loose with two-nine suited but not completely terrible.
Because with 4.5 to 1 odds, nothing is really that terrible.
But I guess we get lucky.
But I do think their call is a bit marginal.
Because we are under the gun, so our range is fairly strong.
That's a nice pot to win.
So we're a fairly big stack now.
I guess we're chip leader at the table.
And Check Raise mid-raises from there.
And we have pocket deuces.
I think definitely our hand is too weak to go all in with as a bluff against an under-the-gun opening range.
So the only options really are call or fold.
Basically our hand does have terrible reverse implied odds post-flop though.
So I think if you wanted to fold here, it wouldn't be terrible.
But I guess I think calling is fine, too, with 4.5 to 1.
And also in some sense, you do have odds to set mine here I think.
The chances of hitting a third deuce on the flop is about one in eight.
My odds are only 4.5 to 1, but there are implied odds in the case I do hit a set.
So literally if my strategy is call and fold unless the flop has a two on it, I don't think I'm losing a ton of money there.
And if there ever is a case where they check down, and I get to take down the pot with just a pair of twos, then I'm just printing money by calling.
So I decided to call.
The flop doesn't have a two, so it's bad.
But out of the flops that don't have a two, it's probably not that bad, considering the three and seven is not that likely to pair him when he's under the gun.
But nonetheless I fold.
You know like if I could do the cheat I talked about where I could disconnect and call and see the river, I definitely would.
But if I call, I have to decide again on the turn whether to call and decide again on the river whether to call, and it's just not going to be a good situation, even though for this immediate bet, I think I definitely have odds to call with a backdoor flush draw, and often they're just going to have ace-king.
It's just impossible to play the hand good.
This sort of just demonstrates why it's very risky when you decide to play a hand like pocket deuces out of position, because it's just very easy to get roped in.
Like let's say I call here, and then I call the flop thinking they might have ace-king.
They bet small, so I have good odds, and I have a backdoor flush draw.
So let's say I call, and then the turn is like the Five of Clubs or something, and they bet like 40% of the pot.
Well, am I going to talk myself into a call again?
Maybe, because now I hit a third club.
But if I do then, if the river's something random-- it's just all these decisions that I'm not going to be able to make well.
Whereas for my opponent, it's going to be fairly easy.
They're going to just bluff their bad hands, value bet their good hands, and check down the hands in the middle which beat me.
OK, so the next hand we get ace-king offsuit, and we raise, and the big blind calls.
The flop is nine, two, seven.
So note that the big blind just got moved to the table, and they actually cover us.
So they're probably one of the biggest stacks in the tournament.
So they check to us.
I decide to check back here, which I don't think is terrible, but I think is a bit of a scared play that I-- I think I would have rather saw myself bet.
If I was shallower, like let's say we only had 20,000 chips, then I think just checking is good.
Because if I bet, they're probably going to be check-raising with a lot of their flush draws, straight draws, and then like pairs of nines or pairs of sevens.
And I'm going to have to fold with a pretty good hand where I have two good overcards and a backdoor flush draw, but when it's this deep, they really can't just check-raise a nine, because they're going to have to be worried about pocket aces on our part.
So I think it's-- we're not going to get check-raised that often, and by betting, we just make the pot bigger when we do hit a king or an ace and also make it more likely that we can bluff them off a nine or a seven by the river.
So I would have liked to see myself bet, but I don't think checking is terrible, by the argument that I will most of the time, you know by betting I am folding out basically all the hands that I've beat and keeping in all the hands that beat me.
It's close but I would have liked to see myself bet.
But as played, you know, if I'm going to check the flop, I'm not going to bet this turn.
Because the whole point of betting the flop is so that I could potentially bet the turn and bet the river and get him to fold a pair of nines or a pair of sevens, but by now I'm not going to really have any hope of getting him to fold a nine.
And I'm just trying to win the pot with my ace-king high.
And so yeah, I just check it down.
The river check, I think, is very, very obvious, because I'm literally only getting called by better hands if I bet the river.
They have queen-10.
I think they probably should have bluffed the river.
I think queen-10 is weak enough where they're just very rarely winning the pot with queen-10 high.
They basically need to bluff and then put me to a decision.
But nice to win that.
The next hand we have jack-10, not too exciting.
So this same guy who's now the button, raises from the button.
And we call, and we flop nothing.
So the best thing to do is bluff off all your chips, but no, I just fold.
[LAUGHTER] OK, so the next hand, it's also against this [?
GarlaEDU.
?]
So the two big stacks at the table are actually playing a lot of hands against each other.
I just call here with jacks.
I don't think it would be terrible to just raise and bet it all in, but I think it is sort of unnecessarily risky.
I think it wouldn't be a terrible play to just raise and get it all in, but it's definitely thin.
Like you're not printing value by raising and getting it all in, because if they reraise all in, their range is going to be pretty strong.
And pocket jacks is not really crushing them.
I think you need to have queens or better to really be hoping to get it all in against this guy here.
So with jacks I just call.
And also, you know, this protects my calling range of it.
Let's say, like Check Raise thinks I'm weak and decides to make a play here.
And they reraise to like 10,000, then if [?
GarlaEDU ?]
folds, I can come back around and go all in and sort of surprise them.
So it does protect my calling range a bit as well.
I call.
We get a nine, six, five flop.
They check, and I bet.
I think betting is a fairly natural play.
I don't want to give ace-king outs.
And I just want to win the pot right now, but I was thinking to myself after that if they check-raised here, that's really a gross spot.
Actually, I would hate my life if they check-raised me.
Because I literally don't know what I should do.
On one hand, I would be inclined to fold, because it's kind of weird for them to want to check-raise me here without having like a bigger pair or a flush draw with lots of outs.
But at the same time, it's just why would they check pocket aces to me here, maybe to try to trap me.
It would have been a really gross spot, I think, if I got check-raised.
And by that argument, I think against a very tricky player, if you could be check-raising here as a trap and as a bluff with a balance strategy, maybe checking is better just to protect myself from that.
But I think against him, I was reasonably confident nothing crazy was going to happen.
And yes, so they just fold.
OK, so it's against this guy again.
I guess we're playing a lot of pots against the same player.
I think it's getting fairly late into the tournament at this point.
It's not a huge tournament, maybe like 80, 90 players, and it's probably down to like 20, 25 at this point.
So they raise.
We call with queen-eight off.
We're a bit deep here, so our reverse implied odds are actually sort of bad.
But we decide to call, and we flop two, three, five.
So I'm just going to fold here.
But they decide to check.
And then we turn a flush draw, but I'm still just going to check and basically bet pretty much every river, maybe not an eight river.
So we river a six, and I essentially bet as a bluff, because our hand has no hope of winning if it gets a showdown.
And it works this time.
So yeah, we played a lot of pots against him.
And it seems like they are fairly willing to just check and fold their hands, so not too aggressive.
So I guess one thing to keep in mind is maybe we can get out of the way if we do see them raising.
OK, so this situation is similar to the situation in the very first hand we discussed where we had king-queen, and we had roughly 30 big blinds.
And I talked sort of about the benefits of raising versus calling.
It's pretty much the exact same situation.
And I decided to sort of raise, essentially as a bluff, but also king-queen plays pretty well when they do call.
Because I'm always good when I hit top pair.
So make it 8,000.
But unfortunately, this time this [?
GerarDocks ?]
goes all in.
So we basically just have to fold.
Yeah, unfortunately, 8,000 is small enough where we can just happily get away here.
And it actually turns out this guy also goes on, which makes the decision even easier.
So it looks like we ran into pocket aces and pocket kings that time.
It doesn't always work when you reraise with king-queen.
All right, so still at the same table, we get ace-jack offsuit, under the gun.
We reraise, and this [?
GerarDocks ?]
guy calls.
We flop king, nine, deuce with two hearts, and I think this is sort of pretty much the worst flop, one of the worst flops I could get.
If there was like two higher cards like a king and a queen, I would have a gutshot straight draw.
And if it was all low cards, I'd have two overcards, but here there's a flush draw that I have no part of.
And also, they are somewhat likely to have a king, and if they do, I only have three outs and my aces, so I'm just going give this up.
So I talk about the power of continuation betting before where continuation betting is good, because my range is stronger than theirs.
I could have aces.
I can bluff, et cetera.
And yeah, in general, continuation betting, being aggressive, especially when you're the pre-flop aggressor, is very good.
But I think this is just one of the worst flops, and I'm just going to give up.
So I check.
They bet, and I fold.
I mean, I think sometimes I will check a reasonably good hand here to protect the times-- so that they can't just bet here with any two cards and know for sure I'll fold.
But most of the time if I'm checking here, I'm going to be fairly weak, which is maybe exploitable.
But I give up the hand.
So eight-five suited.
I decide to call on the small blind.
I know this goes against my recommendation from the first class.
I think calling from the small blind is actually, both theoretically and practically, a pretty good strategy for a lot of cases, but it's a bit complicated.
Because you have to sometimes limp raise as a bluff.
You have to sometimes limp raise and trap with aces.
So it's a bit complicated.
So I just said for simplicity, you should just raise or fold, which is still a reasonable I think.
But I decide to call here, and it's checked.
We get a pretty good flop for our hands but probably a better flop for his range, because when we call, we're less likely to have really small cards than he does.
But I mean, we have a good flop for our hand, so I am still going to bet my hand.
So I bet half pot, which is maybe not big enough, to be honest, considering how dry the board is.
And we get a call.
We turn a nine, which is a great card, because it gives-- not only does it complete our hand, there's lot of draws out there, and I bet quite big here.
So this is where I was talking about if the board is drawy, you need to bet big and not give your opponent odds.
So I bet 688 into 10,800.
The reason for the eights is because it's easier to type like 688 then 6800, so you'll see that quite a lot.
And we get called.
Yeah, and another, sometimes you can get more creative, you can do like 1234567.
That's a popular bet, because you can just slide your finger across the keyboard.
Yeah, there's a lot of weird idiosyncrasies, OK, so I bet 15,555 here, and he folds.
I'm playing like this with my missed spade draws, and I missed heart draws.
Unfortunately, we don't get paid off.
OK, so jack-eight suited here.
The small blind raises.
Definitely I'm going to call, and we flop top pair.
They bet, and-- so I think this is a spot where I think my hand is-- there's benefits of raising and benefits of calling.
And overall, I decided the benefits of raising were higher.
So I decided to raise and just get it all in, although I think calling is not unreasonable, because your hand sort of isn't good enough where you are just really, really happy to get it all in.
I mean, it's good enough where I'm going to get it all in, but I'm not like thrilled if they go all in, and I have to call.
So that's one disadvantage of getting it all in, but I just think compared to the advantage of not giving them a chance to suck out on me and beat me in the hand is a pretty big advantage, not letting a queen, king, or ace roll off.
Another advantage of calling is being able to pick off bluffs.
But I just think in this spot, I want to play a strategy where I'm going to raise to a small size and put him to the decision, both with my top pairs like jacks and also my hands like seven-nine or whatever.
So I decided to raise, but I think calling would have been fine and [INAUDIBLE]..
The next hand we have ace-king.
All right, so we got like seven more hands to get to.
I'll go a bit quickly here at the end, but I think we'll get through all the hands.
This hand is fairly easy.
They make it 5,000, and we have ace-king, and we just go all in.
There's no point slow playing essentially, especially when if they have jack-10, they have lots of outs.
So we go all in, and they fold.
So it is a bit curious here, because they actually only had 12 big blinds here.
So they could have just went all in themselves, but they decided to raise small, so that they could fold if someone behind picks up a monster.
And I guess they were paid off this time.
But yeah, like for this hand, like one disadvantage of their play, suppose we had king-two suited.
We would have folded if they went all in, but now we can call and see a flop.
But because we had ace-king, I guess they profited from not committing their whole stack.
OK, we have king-jack offsuit.
We make it 6,000, and this guy goes all in.
I fold here.
I think it's not that close.
I think with king-jack suited, it's close.
But I'd still fold with king-jack suited.
I'd probably call with ace-jack offsuit plus.
I would probably call with ace-10 suited, and I'd call with king-queen suited and probably king-queen offsuit as well.
In this case-- so in the last class, I said being suited is very important when you're behind.
But when you're potentially ahead, just having a slightly better hand is very important.
So I think in this situation, I would fold king-jack suited but call with king-queen offsuit, because a decent part of his range is hands like king-jack and king-queen.
Here I have king-queen and Degenerated-- so for some reason, I love making this play with king-queen.
It's actually not that unfounded.
It's just a good hand to bluff with that at the same time sort of protects your range in the case that he calls, and also your hand doesn't play that well if you just call.
Because you're not suited, so it really is king-queen-- I mean in this tournament, we've happened to have been dealt king-queen a lot and gotten into situations where we can reraise-bluff people.
But it really is my favorite hand to do this with.
It might even be too predictable, because I just always have king-queen whenever I do this.
But we do take it down here.
Ace-jack offsuit.
This guy, so this Degenerated guy who we just-- at this point, it's probably late enough in the tournament.
I am paying attention, and I am aware that I just reraised this guy three hands earlier.
So they make a big raise here, do 9,000.
And I call with ace-jack, which I regret quite a bit.
I think it's just too weak a hand to play.
And if I'm going to play it, I think I should just go all in and try to win the hand preflop.
Even though I'm in position, I think it's just going to be hard to play.
Even when I hit an ace or a jack, I'm still going to quite often be behind.
And if I don't hit an ace or a jack, I'm pretty much always just going to have to fold.
And that's basically what happens.
So I don't know why I just gave away 9,000 chips there.
OK, so queen-jack suited.
I just go all in from the small blind, because it's-- so even though we have 15 big blinds, which is more than 12, I said last class that from the small blind, you're really, really incentivized to go all in, because if you don't then the other guy has position.
Whereas like from the button, if you just raise, the big blind calls, you have position.
So I go all in.
Queen-jack suited is more than good enough.
And we get a fold.
And the next hand, OK, so this is actually a great illustration.
The very next hand we get queen-jack off, and we just raise from the button.
And it's exactly the reason I said, because we have position.
So we don't mind if the pot gets played post-flop.
So the small blind goes all in.
And note that they're actually fairly short here.
They only have 14 big blinds to start the hand.
So I would have easily with queen-jack suited.
Even with queen-jack off, I'm not certain that folding is a better play than calling.
It's just your odds are basically good enough here, and their range will be fairly loose when I raise from the button.
Because they're going to think I don't have much when I raise from the button.
So it's quite possible calling here would have been the right play.
It's very close.
I think I would definitely called with ace-nine offsuit.
I would have definitely called with king-10 offsuit.
Yeah, it's pretty close.
I decided to fold.
Maybe I also knew that they were kind of tight.
Normally when I play but I'm not showing here is I have like statistics on every player, of like overall the hands, what percent of the time they raised, what percent they folded.
So I have a rough idea while I'm playing of how tight everyone is, and maybe I just knew he was tight.
Actually let me see if I can turn it back on here.
Yeah, let me see.
That might be why.
So OK, so these numbers-- Actually, he's fairly loose.
Actually, I'm not sure why I folded.
So this 10.7 is his reraise percentage, and what this says is 10.7% of the time he could have reraised, he reraised.
It's only over a sample of 187 hands, but it's-- Yeah, I think it's definitely looser than average.
Although it's a bit biased, because we're playing at a five-handed table.
Nonetheless, it's close.
I think calling probably would've been OK, but I folded.
Let me turn this off.
It's quite annoying.
So this is near the final table now.
This is, I think, the second final table.
There's maybe like 11 or 12 left, and I'm just going to show you the final hands before the final table.
This hand is a no-brainer.
We go all in, and fortunately, we hold against pocket twos.
OK, so the very last hand, also fairly no-brainer.
This guy goes on.
So yeah, poker is easy if you just get dealt pocket jacks, pocket aces all the time.
So we win this one.
OK so, this is the last hand before the final table.
So in a future class, not the Friday class, because Jennifer Shahade is speaking.
But in a future class I'll go through the final table, and I'll maybe also go through all the hands, not just the hands where I got in a big pot.
So let me just quickly go back to the slides.
Right, so I really only talked about two concepts in this class.
It was this drawy versus dry boards.
So try to remember that while you guys are playing, changing your bet sizing depending on the board pressure.
And also this pre-flop reraising strategy, even though I didn't give you a conclusion here on what you reraise with.
I should give you some rough guidelines and the advantages and disadvantages of each play.
Yeah, OK, cool, so that's it.
Come on Friday's class.
It should be really exciting.
Jennifer Shahade, she'll probably talk a bit about chess as well, but I'll convince her not to talk too much about chess when it's a poker class.
But thanks, guys.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today I promised I would play through the second half of the tournament that I started last time, so two times ago, but before Jennifer Shahade's guest lecture.
But before I start, so remember last time we got to the final table.
This is essentially the start of the final table.
And but before I get into the hands, I want to do a bit of theory first that I've been forgoing so far in this class.
And it's essentially what's called the independent chip mode, or ICM theory.
And essentially what it's used for is in tournaments, essentially the value of chips is essentially nonlinear because you're not really trying to just get all the chips.
You are trying to last as long as possible and move up the payouts.
So I want to now formally quantify what this means and how you calculate the EV of your game rules.
So far I think I've just told you everything that's done so far in this class has been chip EV.
And I've told you just play to maximize chip EV.
Maximize your expected number of chips.
And that's a pretty reasonable way to play.
In cash games, it's what you want to do.
And in tournaments it's not a bad approximation.
So I don't want everyone after this ICM lecture to think, you know, I've got to completely change my playing in tournaments because that's not the case.
Essentially, playing and maximizing expected number of chips is a pretty good approximation of how you actually should be playing.
But there are some small differences in extreme situations, and I'd like to highlight some of that today.
So OK, yeah.
So this is essentially-- so it's this idea of minimizing risk to stay alive and move up to the escalating payouts.
So I want to make an important distinction, though.
When I'm saying, we're still trying to maximize our expected amount of money.
What I'm saying, you should be minimizing your risk in a tournament so that you can move up the payouts.
That's different than saying you should minimize your monetary risk because I'm still assuming we're trying to maximize our expected money without worrying about risk.
But we're only minimizing tournament risk to maximize your expected amount of money.
But I'm not saying we should be overly conservative in a tournament and just trying to move up the money ladder slowly when you should be gambling because it's positive money expectation.
If you find yourself in this situation, then if you're trying to play professionally and just maximize your expected amount of money in every situation, then you should probably play smaller tournaments, where this is not really the case-- where this doesn't occur too often.
I can understand in practice, if you play poker for fun and you made it far in a tournament and it's the top, let's say, 50K, and there's 55 left, I can understand why you might not want to make a positive money expectation play because you've played this tournament.
You're probably not going to be in the situation that many times in your lifetime.
And you just really want to cash the World Series of Poker main event or something.
And I think that's a reasonable life decision.
I'm not criticizing that life decision.
But I'm going to talk about all of these decisions assuming we don't care about that.
We're still trying to maximize expected amount of money.
And if you are playing professionally, where you're getting into that situation very frequently, then it is very important to maximize risk in some cases and to maximize your dollar EV because in tournaments, you're losing most of the time.
You really have to win a lot the times that you do win.
You can't be content because we made $30,000 if we could have made $50,000.
So OK.
So what is the exact formula for ICM/ So I'm just going to demonstrate how to calculate the value of chips in a tournament with an example.
So ICM is essentially a formula to calculate how much your stack is worth in a tournament.
So let's suppose there's three players left and that payouts are 5, 3, 2.
This is a common structure in sit-and-goes if you've played those, which are 10-player tournaments.
So this is the statement is essentially your chances of winning the tournament is proportional to your percent of the total chips.
And essentially we're assuming this is true, and we're going to derive from it all the formulas.
And this statement is approximately true.
There are more advanced ICM calculators that also take into account your position if the stack sizes are rather shallow because there are some inaccuracies because this statement, it calculates only based on chips.
But if you're folding and then you're the button the next hand, which is the best position, then that's actually an entire big blind better almost than being under the gun the next hand.
But we're not going to worry about that.
We're just going to assume your chances of winning the tournament is proportional to your percentage of chips.
So how do we calculate this?
So you can't really do it by hand in practice.
But for this example with three people, I'll show you how to do it by hand.
So let's suppose the chips stacks are also 5, 3, 2.
OK, so the payouts are 5, 3, 2.
And the chip stacks are also 5, 3, 2 ratio.
If chip EV equals money EV, then the guy with 5,000, their equity from the tournament should be $5.
And the guy in third place, their equity should be $2, right?
But this is clearly not correct because the guy in third place is guaranteed $2.
And clearly there's a nonzero chance of them winning the tournament.
So clearly their equity is actually more than $2.
And then similarly if you're player A, your equity is clearly not $5 because you're not guaranteed to win the tournament.
So it's clear from this example why if you just do chip EV equals money EV, that's clearly wrong.
Is that clear to everyone?
So how do we calculate it?
So let's calculate it for player C. So essentially what it's saying, your chances of winning, your chances of coming first is 2,000 over 10,000, which is 20%.
And then to calculate your chances of coming second, you just condition on each of the other player's way.
So conditioned on A winning, which would happen 50% of the time, we assume that our chances of coming second is 2,000 divided by 5,000, which is the remaining number of chips besides player A.
Does that make sense?
So player A has 5,000 chips.
So between you who's player C and player D, there's 5,000 chips in total, and you got 2,000 of those.
So it's 2,000 over 5,000, so it's 0.4.
And then if player B wins, then your chances of coming second is 2,000 over 7,000.
So overall, your chances of coming second is 0.4-- So it says, 50% times 0.4 plus 30% times 2/7, which equals 2/7.
So then you can calculate your equity is-- you're guaranteed $2.
And you have a 20% chance of winning, which would give you three additional dollars.
And then a 29% chance of coming second, which would give you an additional dollar.
So your equity is actually 2.89 in this case.
is actually closer to second place than third place.
Does that calculation make sense to everyone?
So if there's five players, you just have to recursively condition.
You have to condition on player A winning and then go through all the possibilities of who can come second.
Then go through all the possibilities of who could comes third, et cetera, et cetera.
So it's not a calculation you can really do in your head.
But that it's quite easy to find a calculator to do it for you.
You just Google ICM calculator, and you punch it in.
And even though it's not-- it grows exponentially.
But there's only going to be nine players.
So it's still fine.
It won't take that-- the computer can do it instantly, even for 9-10 players.
OK, so what are some cases where ICM is easy to calculate?
So like I said, in cash games, there is no such thing as ICM.
You're just playing to maximize expected number of chips.
And your goal is essentially, you're sitting out at the cash game looking to get into all-ins, looking to get into favorable all-ins, hopefully winning the person who you're targeting's money.
And in winner-takes-all tournaments, it's also the same because you're just looking at your chances of winning the tournament.
And that's always just your number of chips.
So you're just trying to maximize expected number of chips.
So as a corollary, once it gets down to two players, there's no more such thing as ICM.
Three is the minimum number of players you need for ICM to be relevant.
Once it's two players, it's just maximize your chip EV because there's no sense of trying to survive in hopes that someone else busts first.
Because with only two people, for the other guy to bust, you have to take all their chips.
OK, so what are some mathematical corollaries of ICM formula?
So big stacks have money EV less than chip EV.
Small stacks have money EV greater than chip EV.
Small stacks also tend to have positive chip EV in general.
This is something I didn't mention to you much.
But in general-- so you might have seen this if you guys have been to a casino and played cash games.
So let's say you sit down at a table in a cash game and everyone buys in for 100 big blinds.
At some tables, usually they'll allow you to buy in for as low as 20 big blinds.
And if you do this, in general, that's going to be positive EV for the guy with 20 big blinds because there's situations where they have to-- Where, let's say, they have to essentially sometimes one guy will raise.
One guy will call.
And then you can go all in.
And one guy will go all in on top of you.
And the other guy will fold.
And then you have dead money.
So another thing is early on in the tournament, ICM is irrelevant.
You just want to accumulate chips.
So if you try to do this formula early on in a tournament, what you'll find is essentially you're so far away from the money that none of the things really matter to maximize your chances of even getting to the money.
You just want to maximize your number of chips.
So where is ICM most relevant?
It's most relevant on the exact payout bubble and on the final table.
So roughly speaking, in every poker tournament I've seen other than strange ones that are winner-takes-all or satellites, the payout looks roughly like this.
So it's 0.
You go in, and once you hit about top 20% is the bubble.
And then there's a jump here where suddenly you get paid.
This is about 80% get 0.
So here there's a jump, and then it escalates slowly until you get to the final table.
Final-- sorry, I'll actually switched to color-- final table.
And then it escalates really fast.
So essentially basically ICM is most relevant at the points where essentially, the derivative, or the rate of change the payout, is highest.
And those points, as you can see, are here and here.
So essentially it matters most at the exact money bubble and in the final table.
OK, let me clear this.
OK, so yeah, so some more extreme examples of ICM you have one chip left late in the tournament when everyone else has thousands of chips.
So clearly this ship is worth a lot more than its value as a fraction of the chips.
And you're just trying to survive with this one chip essentially instead of trying to win more chips.
Maybe I'll calculate this in a bit.
So in satellites-- what satellites are are there are tournaments where you're trying to qualify for a bigger tournament.
So essentially the important thing is there will be maybe 100 players.
And there will be 10 seats, which means the top 10 get an invite to the bigger tournament.
And then everyone else gets nothing.
So I guess the qualifiers for the MIT Series of Poker Main Event are kind of like this.
I guess there's other prizes, to.
But you're essentially trying to come top 10.
So this is how that works.
And in that case, the payout is essentially this.
So there's not really much incentive to win.
You're just trying to be one of this top 10% that gets paid.
So that's another case where ICM has extreme importance.
And you often find-- well, not often-- but you can often make a case for folding pocket aces preflop even though you know it's going to have positive chip EV if you call because it's the best possible hand.
But because you just really don't want to bust, then just you want to fold aces.
OK, we'll talk about this a bit more later.
OK, so let's get back to the tournament.
So this is the final table.
So ICM is going to be quite relevant here.
As I'm talking through my decisions in the hands, I'll talk about the ICM decisions as well.
This is the first hand, or I think maybe one orbit passed.
We're at the final table.
We're the second biggest stack.
We should maybe look at the stack sizes.
So we look around at the stack sizes.
So what do you notice about the stack sizes?
Or specifically, how are we doing?
How many chips do we have?
[INAUDIBLE]
Real.
Right.
We have quite a lot, right?
Yeah
We have quite a lot, but we're not in first place.
There's this one guy who covers us.
So this Panpancrisy guy who covers us.
So specifically, the thing I'm trying to avoid is getting in a big pot with Panpancrisy.
Let's say he goes all in, and I have pocket kings.
I'm going to call.
But I'm not going to be that happy about calling in a situation where otherwise I should be very unhappy.
Just because chances are he's going to have an ace.
He's going to have 30% equity.
It's still going to be positive money EV call, but it's so bad if I basically bust right now in eighth place against Panpancrisy.
So essentially I'm really incentivized to avoid getting in big pots with him.
And I'm essentially hoping for some of these smaller stacks to bust so that then, even if I lose a big pot to him, I don't get the low eighth place payout.
So OK, so it's folded of us.
We raise here.
I think this is one of the weakest hands I am raising from the button.
But so far players have seemed to be reasonably tight, so I'm going to raise.
And then I get the blinds.
OK, so I'll skip some hands at the beginning.
Near the end, I'll play every hand.
This hand we don't play, but I'll try to show the hands were someone busts.
So this hand small blind goes all in.
Big blind calls.
And so small blind wins with 10-2 suited against ace-4, which is basically great for everyone at the table.
Because someone busted, you're always cheering for people who bust essentially.
And I think also both of their plays are fine.
So I think small blinds play-- it looks a bit silly.
But realistically not only does 10-2 suited have reasonable equity when you're called, it's also-- the ICM is very, very relevant.
This Gerardocks guy, he's not the shortest.
He is one of the shortest, so he does need to take some risk, but he's still really very much would like to bust after I show to Heap78.
So he can't call with anything.
With ace-4, I think that's definitely good enough to call.
But let's say they have king-2 offsuit.
I think that's the situation I'd normally call if it was not a final table in a tournament.
But at the final table, where I'm not the shortest and there's someone shorter than me, I would consider folding 10-2-- sorry, king-2 offsuit here just because of ICM essentially.
All right, so the next hand-- so the big blind busts.
And then there's no small blind in this hand, which is great because people are less incentivized to steal.
And indeed we get our big blind back
That works
Very nice.
OK, so not much happens here.
I'll skip forward a bit.
OK, this hand-- I guess it's quite convenient for us.
But this still illustrates a point.
So this guy, who dealt a bit earlier, now he goes all in for quite a lot.
This is quite a big shove.
And if you play by my recommendations, I would never recommend shoving this many bets.
It's 24.
That's quite a lot-- actually, 23, sorry.
It's 23 big blinds, which is way too much to shove.
There's no need to risk this much.
But once again, I think his play is justifiable because of ICM.
So let's see what he had.
So we call.
Here it's a bit different because even if I call and lose, I'm not out of the tournament.
Even if I would be out, even if he had 200,000, I'd probably still call with queens.
It's just too good to fold.
So we need ace-queen here, which is great for us.
But if you look at the play from his perspective again, it actually seems quite reasonable if you think about it because for him, ace-queen is a great hand from this position.
It's definitely good enough where if it was between folding and shoving for 23 big blinds, you would definitely rather shove for 23 big blinds.
So the main reason why normally one would criticize his play is because why doesn't he just raise small with ace-queen?
Well, understandably, he doesn't really want to incentivize me to reraise or Panpancrisy to make a reraise.
He essentially wants to minimize the chances that he has to play for his stack in this hand.
And the way he does that is essentially by going all in.
Looking at back at it, I think his play is fine.
The only weakness of his play, I think, is it is a bit predictable what his hand is in this situation because I think if he had a hand as good as pocket aces, then he shouldn't be doing this.
He should just be raising small because it's good enough where he's happy getting it in against me or Panpancrisy.
Whereas also if his hand was slightly worse, like ace-jack offsuit I think is a good example, I think ace-jack offsuit is not strong enough to make this all-in play with.
But you still want to be raising.
So I think there's a small range of hands that you want to be making this all-in play is.
I think it's basically exactly he's queen offsuit, may ace-king, maybe ace-queen, ace-king.
And then middle pairs, let's say between like pocket 8s and pocket 10s or something.
So the only downside of this play I think is essentially his range here is very predictable it's going to be ace-jack suited plus ace-queen and pocket 8s through 10s or something like that.
But nonetheless, I think, because it's a final table, because of ICM, it's fine.
He gets unlucky, though.
He runs into queens.
All right, here we've got to walk.
OK, so let's see this hand.
So I showed to-- raises goes all in.
So we call here.
We're deep enough where I want to give myself the opportunity to fold if Panpancrisy he goes all-in.
And we call.
And we win.
So OK, so I guess I've been running pretty good in this tournament.
But I just want to point out, there is a lot of selection bias here because I'm not going to play through a tournament where I get all-in on the first hand and lose.
So I mean I'm going to pick a tournament that I [INAUDIBLE]..
So it's in some sense not the best pedagogical example because there is a lot of selection bias.
You know that I'm going to run very good in the tournament I choose to show you guys.
But if it's between that and having be over in five minutes because I lost-- [LAUGHTER] --this is the lesser of two evils.
All right, so here, yeah.
So once again, this is a pretty-- it's not that big, but it's a 17 big blind shove.
And once again, it's justified by ICM prefold.
OK, here, queen-7 off.
Yeah, I shove.
I think it's probably barely a-- I think it's pretty close if chip EV equals money EV.
It would be a borderline shove with the smallish antes.
But given ICM, I know he's less incentivized to call than usual.
So all borderline hands I'm going to be shoving, so I do.
OK, so this hand we get king-10.
So once again, I just shove.
Once again, I think I-- basically it's hard for them to call if I just shove and because of ICM.
So I'm essentially bullying him, bully them.
So when you're the big stack, you can bully people at the final table because of ICM.
We do get lucked up here, though.
And we get lucked up at ace-8.
And so he wins.
I'm not going to do an exact calculation because I'm going to do an exact ICM calculation later.
I would guess his call is very borderline.
It's definitely positive chip expectancy, but the fact that there's this guy with only 57,000 and this guy with 70,000, I think it's a call.
But I think ace-6 is definitely a fold.
So I think it's probably one of the weakest hands he should be calling with.
I think ace-8 is probably too good.
He has to call, but I think it's very borderline in either case because of ICM.
So he does fold, though.
So now ICM is a bit trickier because now we're smaller.
We're getting it all in with any of these guys can really damage us.
And there's still two short stacks that everyone is hoping will bust soon.
So In this situation, I could go all in and bully this Panpancrisy guy, but ICM isn't that relevant.
He's not going to fold ace-queen if I go all in.
So I just give this one to him.
Calling is probably bad anyway, but calling, I think, is especially bad because it just increases the chances of getting in a bigger pot.
OK, we'll skip forward a bit.
Let's see this hand.
So ace-10 off here.
So this guy makes it 12,000.
And I guess I've skipped a bunch of hands.
So Nicholas Brody, I think, has been very active.
And overall, I think people have become very active.
Everyone's playing a lot of pots.
So overall, I am going to put people on wider ranges than I normally do.
But nonetheless here, I still fold ace-10.
I think it's ace-jack I would probably get it in.
And ace-10 suited, I think I'd get it in.
Ace-10 suited is borderline.
Once again, when you're tied between getting in a smaller pot and getting in a bigger pot, you want to be folding.
Yeah, so I guess at a final table, I should make this clear.
It's not like you want to fold more.
It's you want to go all in more when you're first to act.
But call other people's all-ins less.
So that's exactly what's going on here.
He's already moved all in.
And I'm going to fold to his all-in more often than in a normal game.
But as you can see, I haven't really been tighter in every sense because when I get to go first, I think actually going all in more often and risking it more often because I know my opponents are highly incentivized to fold.
So they go all in.
I fold here.
So Nicholas Brody also folds, which I guess is fine.
Definitely he had pretty odds he called there.
But once again, ICM just makes-- it changes things up a bit.
Jack-7 suited, once again, is a pretty big shove, but jack-7 suited plenty good [INAUDIBLE]---- ICM relevant.
Pocket 9s here.
We make it 12,000 on the button.
So here I think pocket 9s is good enough where I'm happy if one of them reraises all-in, and I call.
So essentially my strategy here is I'm going to shove all my medium strength hands and especially hands that don't play well postflop ace-2 offsuit.
And then I'm going to raise small with some of my hands that are less good preflop but better postflop, let's say, 10-8 suited, and also my monster hands like pocket nines that are also reasonable to play postflop.
So Nicholas Brody calls here.
And the flop is ace, jack, 7 with the hearts.
So it's a bad flop for our hand, but it's a good flop for our range because I think, given the way the table was going, I was opening a lot.
And Nicholas Brody is very aggressive.
I would guess unless he had like pocket aces, which he might slow play, he would reraise most.
I wasn't 100% sure-confident, but I was reasonable-- I'd say maybe 85% confident that if he had an ace preflop, he would have just went all in preflop.
So overall, I'd still say it's a very good flop for my range because I think he's way less likely to have an ace than me.
There's a small chance maybe he would chicken out with ace-2 offsuit and just call.
But given how aggressive he's been, I would be surprised if he did that.
So I decided to bet here.
And I'm still not sure how good this bet is.
So I think at the time I bet, because of my logic that it's really hard for him to have an ace, and so I think the best strategy for me on this flop and in this situation is to just bet 100% of my hands.
And he's just forced to fold basically very often because he can't really keep up with an ace.
I can just continue betting.
That being said, though, I do think if I do decide as part of my strategy to check any part of my range here, pocket 9s might actually be one of the best possible hands to check back.
So I think checking back would have been fine.
I guess I'd bet here because I told myself, I want my strategy to be bet 100% of my hands, which I did.
And he folds, which is reasonably smart because he knows that I can bet again.
And he's going to have to make a decision for all his chips probably.
And it's just going to be tough.
OK, this hand-- BeMySponsor goes all in, gets [INAUDIBLE].. Yeah?
Why would you say, pocket 9s is one of the best hands to check back here?
I very rarely get called by worse, and I get called by everything better.
I think he's always calling a jack and usually folding a 7.
Maybe he won't fold 7-8 with some back door draws, but I think he'll usually call a jack.
And maybe he'll call a 7 sometimes.
But I still think mostly it's just he'll basically fold everything that beats me and call everything that-- sorry, he'll fold everything that I beat and call everything that beats me basically.
Yeah, but I still decide to bet, nonetheless, just to win the pot there and not give him extra outs if he happens to have 10-6 suited or something.
So OK, so we'll go through this hand.
So Panpancrisy raises.
He doesn't go all in, which is curious.
But LPMBox calls.
He calls again.
It's checked.
And then LPMBox goes on in the river [INAUDIBLE].. OK, jack-7 here?
Yes.
I guess, so once again, Nicholas Brady raises.
And then LPMBox goes on over him, wins a big pot.
So I guess I do the same for Nicholas Brady.
He's been raising a lot, so pockets 6s good enough to risk going all in there.
OK, this hand-- so we get ace-jack suited.
And I raise from cutoff.
So I've been raising to a fairly small size because the antes are smaller.
So that's one thing to remember at a final table is the pots are actually a bit smaller because with only five players the ante, the sum of the antes, instead of normally the 6,000, is only 3,000 in this case.
So the big blind has worse odds to call.
So Nicholas Brady goes all in.
I look him up ace-jack suited-- definitely good enough.
Also at this point, because a number of pots have gone my way, I'm big enough where even if I call and lose this, I won't be chip leader, but I'll still have a decent amount of chips.
So unfortunately we lose the ace-8 here.
So now we're second place, but we're still doing reasonably well.
I could just maybe gone all in here.
I guess this guy has a lot of chips.
But I could've raised to a bigger size to disincentive Nicholas Brody from doing this with the right wide range of hands and then protecting my stack more from an ICM point of view.
But I'm happy with-- I think ace-jack suited is just good enough where I'm happy calling his all-in, which I should have been in this case.
I guess he had ace-6.
OK, so jack-10 suited here.
So yeah, here I just decided to fold.
I mean we could consider doing a crazy bluff all-in, but I don't think this guy's folding that often.
They made a rather big reraise preflop from going three times the initial opening.
So Nicholas Brody goes all in here, and they do fold.
So Nicholas Brody picking up a bunch of chips.
So he just takes it down again.
He's been raising quite a lot.
Here we get a raise and a call.
So LPMBox has been raising over other people's-- opens quite a lot.
So he gets called here, but he holds and wins, which is definitely bad news for everyone.
If I could pay 10,000 chips into the pot to have this all-in go the other way, it almost certainly would be profitable.
The amount actually we'd be willing to pay you make this all-in go the other way is probably 25,000.
It's a ridiculous amount because just the value of having people bust and moving up the payouts is so significant, especially when I'm not such a big stack anymore.
OK, so now this guy, he just doubled up.
So I guess we were running pretty good.
But I'd say in the last two or three orbits, as you can see, a lot of things haven't been that favorable.
We've lost some all-ins.
And also people haven't really busted.
The chips are just being passed around the table, which is basically not what you're hoping for.
So king-10 offsuit.
We get a walk.
So walk just means that everyone folds to you, and the big blind.
I think we've actually gotten quite a lot of walks at this final table.
So something we can keep in mind if we're trying to play exploitatively later, queen-jack, go all in.
He folds.
So at 8-7 I guess, LPMBox, after winning that, so as you notice, the big stacks are the ones who win most of the pots.
And this is expected.
So whenever ICM is relevant, you would expect that-- so although the small stacks had a higher chip EV, then-- sorry, had a higher money EV than chip EV, usually the chips get transferred from the small stack to the big stacks because the big stacks have more license to go all in first and bully people around.
So he takes this one down.
Next hand, we get jacks here.
So Panpancrisy is actually short now even though he started the final table as the chip leader because he's lost a bunch of all-ins.
He gets an all-in here, and we hold against a 6.
So there's actually an interesting play here.
I don't think he should have done it.
But this is actually often a good ICM play.
So normally I would say in this situation, if you have a good hand, just go all in because he knows I'm going to call.
So when ace-6, he just went all in.
But often there's this play called the stop-and-go, which is in practice not very good.
I don't think it would have been here, but it's an interesting theoretical concept, which is instead of going on, he just calls, OK?
He calls, and then he basically blindly goes all in on the flop.
He doesn't actually have to, but this is what players used to do.
And you can do this in live poker.
What you would do in this situation is you would say, call, and before the dealer even deals the flop, you would just say, all in.
And this is a legal play.
The point is essentially-- it's an ICM.
The justification, which I think is mostly wrong but still interesting, is he wants to maximize the chances that I fold.
So essentially, OK, this is strictly better, right?
Sorry, the fact that he would call, let me see the flop, and then I get to decide whether to put the remaining chips in sounds like it should be better for me than me having to put all the chips in preflop.
But the main reason why he does this is because he wants me to have an opportunity to fold my cards and avoid getting in a situation where both of us are playing for our entire stack.
So this is the justification for it.
So even though you're giving your opponent more information, you're also decreasing the chances that they decide to call to the river card and have more chances of beating you.
So that justification is correct, but it's just that if you've crunched the numbers, usually this situation won't be favorable enough.
And usually you don't want to be giving your opponent that extra information.
OK, so finally someone busts.
We get jacks off again.
So here normally I think I would often consider calling this, but I think just avoid-- so now the stacks are even-- we're the chip leader but not by a ton-- and I don't really want to get in a big pot here.
So I decide to give it to them, even though normally I would definitely call this.
King-7 suited.
I make a pretty big all-in for 21 and 1/2 bets from the small blind.
And take it down.
All right, we'll see all the hands.
We fold 6-3 suited here.
BeMySponsor decided to raise small, which is, once again, a bit unusual.
You usually just want to go all in in a situation where ICM is so relevant.
And then Nicholas goes all in.
And he folds.
Jack-5 offsuit.
Here we fold.
So once again, this guy raises small.
So maybe he doesn't really want to do this ICM thing and just he wants to play pots.
He takes that one down, though.
Nicholas Brody-- so as you can see, it's a bit boring.
There's not too many hands here.
That's what happens with there's ICM because you don't really want to splash around.
You just want to survive in the tournament by risking all your chips once in a while instead of slowly bleeding your chips in a variety of pots.
So once again, queen-9 suited, big all in.
Plenty good enough.
Queen-7 suited, I decide to raise it up.
But unfortunately, I guess, even if just one guy went all in, I'm clearly folding.
So they both go all in.
And ace-king beats kings, which is really bad news for everyone essentially because if kings held, then he would have busted.
But unfortunately he doesn't.
So pocket 8s, I'm raising here.
This would have been a gross spot, by the way.
I think if BeMySponsor went on here, I don't know what it would have done.
it would have been very gross because on one hand, I definitely don't want to be folding a hand as good as pocket 8s.
But on the other hand, ICM is just so relevant, I think I probably have to fold.
So yeah, this is a case where I could've made an extreme, a hero fold essentially.
I folded a very good hand if BeMySponsor went all in.
So king-jack off-- you get another walk.
OK, jack-10 off.
I'll tell you what happened, and then we'll take a break.
And then I'll analyze it after.
This hand I want to analyze in a bit more depth because I was actually surprised going through the replays looking at how I played this.
So LPMBox makes it 16,000.
And I reraise to 44,444, so essentially I think this-- I don't have a great hand.
But I think this hand is too good not to make a bluff with.
So I do that.
And then he goes all in.
And then I think this is a very tough spot because my hand isn't good.
But when I reraise, I definitely had the-- if this guy had fewer chips, I would have just went all in myself.
I definitely didn't reraise to this size thinking I would even consider folding if he moved on.
So yeah, the reason I made this size is so that if this guy went all in, I could fold.
But as it turns out, he folded, and he went all in.
So let's analyze this hand after the break.
We'll take a short two-three minute break.
OK, I'll get started again.
So OK, so let's calculate this hand.
So I actually decided to go back and calculate this hand from an ICM perspective because ICM is basically very relevant here.
OK, so let's see exactly how to do this in practice.
OK, so let's first do the chip EV calculation, OK?
Let me first do the chip EV calculation.
You can trust that I copied the numbers correctly.
So essentially there's three possibilities, OK?
There is if I fold, this is the amount of chips.
This is the amount of chips I'll have.
OK, so right now I'm only caring about chips.
So if I fold, this is the amount of chips I'll have.
If I call and win, this is the number of chips I'll have.
And if I call and lose, this is the amount of chips I'll have.
So we could basically do a calculation here where I do essentially it's-- so let's say my chances of winning is x.
So it's x times how many chips I have if I call and lose, which is 107-- I'm going to not write the last three zeros-- plus 1 minus x times 402.
Sorry, I did this in reverse.
OK, x is my chances of losing.
So again, I'm basically comparing this number versus how many chips I have if I fold, OK?
And then essentially I'm just solving for x.
So the way I do it is I set these to be equal, and then I calculate x.
And that's the breakeven x, which tells me what's the minimum equity I need against his range to call.
So we can do that.
So I think it turns out to be-- so this formula is equal to fold minus call over-- hang on-- I did it before.
OK, so this is easy.
OK, so I Ctrl+Z, so essentially I had 2 to 1 odds, right?
I had 2 to 1 odds, so it makes sense that I needed a 33% equity call.
OK, so how do we do this side?
These are essentially the ICM numbers.
So essentially the way you calculate this is you Google ICM calculator, and then you just go to the first one.
It's usually quite easy to find.
OK, so it's a bit tricky because you have to put in the exact payouts.
So I think at the time they were 18.5.
This is all as a percentage of the total pot.
So I forget exactly the numbers, but it was something like this.
OK, and then you type in the four-- you type in the stack sizes.
So it takes a bit of time and even if you're fast at this, if you conserve your entire time bank from the whole tournament for this one calculation, you'd probably have enough time to do it because you'll probably have like two or three minutes of time bank saved up.
So you probably can do it.
You have to essentially, right?
So there's three possibilities.
I have to calculate if I fold, not only how many chips I have but how many chips everyone else has.
And then I take them in, and then I can essentially calculate the value of my stack in terms of as a fraction of the total money in the pot.
And then similarly, I do it for the other two cases.
And then I calculate the ICM numbers.
And roughly speaking, this is my-- this is all normalized.
This is roughly my dollar EV in each of these three scenarios.
If I do the same calculation, I'll find that I actually need 39.6 equity.
So far does the calculation make sense to everyone?
I didn't do all the steps in practice, but I explained how to do all the steps.
Yeah, if you're fast, it still takes like a minute-a minute and 1/2.
But the last part is now we have to open Poker [INAUDIBLE].. And now we have to type in jack-10 offsuit.
And then we have to put him on the range.
So OK, let's just put him on a-- OK, I suspect that I'm probably going to have to fold because 39.6% is quite a lot.
I'm going to put him on the loosest range and see a 539.6.
So let's say he's pretty loose.
So he's willing to gamble with, let's say, pocket 4s plus-- I mean, another thing that's relevant here is I think it looks like he doesn't have fold equity.
I don't think when he's going all in here he suspects that I'm ever folding.
Or at least he probably thinks I'm calling him the overwhelming majority of the time.
So I think he's rarely going to turn up a stone bluff like 8-7 suited because of this.
I would be very surprised if he ever had 8-7 suited unless he read into this and realized that I could actually fold some hands here, which maybe he did.
I should give him enough credit.
OK, let's just do something somewhat quick.
Let's say he gets ace-8 offsuit, ace-5 suited plus, king-jack suited, king-queen off, and pocket 4s plus.
So that's 14.6% of hands.
So let's do this.
So against this we have 36% equity.
So that's not enough.
We need 39.
Let's see how wide his range has to be for us to have 39.
All right, let's say he gets in with 20% of hands, which is quite a lot, which is, I think, way too much.
Let's say 18.6 percent of hands.
Oh, we actually have worse against this range because it has-- oops-- because it has more hands that dominate us because I put in a lot of hands like king-10 and removed a lot of the hands like ace-4.
So basically OK, you can play with it however you want.
But I think it's hard to play with it in a way to give yourself 39% equity unless you were very optimistic about how wide his range was.
So in either case, so at the time I did fold.
And it looks weird to me at first, but after analyzing it, I think this is definitely the right play if you take into account ICM.
So I think it's a good example of ICM because if you remember when we were playing with it, our equity was always essentially between 33 and 39.
If we put them on a looser range, our equity would be 37.
And if we put them on a tighter ranger, our equity would be 34.
But basically it's a situation where I would always call if ICM was not relevant and always fold if ICM was relevant.
And this isn't even a case where it looks like ICM should be that relevant.
Yeah, so we do fold.
So with hindsight, maybe my initial strategy was wrong.
Maybe I should have should not have even bluffed here.
Maybe I should have just folded in the first place.
That's a different story.
But given that I did bluff, I think the best thing I can do here is fold.
OK, let me just jump back to the PowerPoint for a bit.
So I just want to talk about a bit more about this theory of satellites and going all in first.
So in game theory, this is called the traffic intersection game.
So ICM does not say, play tighter.
It allows you to play looser if you can go all in first.
So this is where I actually think it's very interesting because in satellites, you want to be under the gun.
So normally in poker, that's the worst position because you've got to act first.
Everyone else gets to act after you.
But in satellites, basically you just want to be the first person to be able to threaten, say, look, I'm putting all my chips in.
You pretty much have to fold.
So in satellites, let's say there's 11 people left, and 10 people get a seat.
If you go on, no one can call you even if they have aces because to call you, they essentially need to be more than 10/11 sure that they're going to win the hand.
And even if they have pocket aces, they're not going to be 10/11 sure that they're going to win the hand.
So basically going all in first is very, very good.
So it's called the traffic interception game because you can imagine the situation where there's two cars trying to pass each other at an intersection.
And you want to save time, get there first.
But you're worried that the other guy might crash.
Both players really want to avoid disaster, but both players really want to be the guy to get through first.
Well, if you can act first, and if you just tell the guy, look, I'm closing my eyes.
I'm tying up my hands.
I cannot stop my car, OK?
I'm just going to go through the intersection no matter what.
If you're the first person to be able to do this, then the other guy, he's like, OK, I guess I've got to stop my car now and let you go first.
That's essentially what it's like.
If you're the first person to go all in and you've already said, look, I'm all in, and there's nothing I can do to take my chips back, then the other guy is like, all right, I guess I'll follow.
And you can win the blinds.
So it's sort of how it works.
Of course, some opponents are not rational.
If you do this enough times, they might be like, OK, screw this.
I'm just going to kill this guy and speed up.
So it's a bit weird, and in practice, this happens a lot, too.
I've definitely had very bad situations where I'm going all in with any two cards, and I know this is a game theoretically optimal strategy.
But my opponents are sped up.
And then he calls me with ace-jack or something.
And then I'll have jack-2 offsuited.
And he looks like a genius, and he beats me when his play is just terrible basically.
Yeah?
Yeah, so interesting.
So does it matter if, say, you're under the gun, and you're in the low stack.
Can you do this?
Right.
So the stack size does matter.
So I'm assuming if everyone had the same stack size in that extreme example, where I said other people should fold aces, I'm assuming everyone has the same stack size
I guess where's the balance between knocking you out versus the chips differential?
Right, exactly.
So the reason, in that case where it's correct to fold aces, is because let's say they call and knock me out.
It doesn't help them that much because first place in a satellite does not get paid any more than 10th place.
But in real tournaments, you basically never fold aces because even if you're risking losing all your money, just first place will be hire than second place by enough.
And winning with those aces will increases your chances of coming first by so much that you should just gamble anyway.
So this is only in satellites.
Yeah, please don't fold aces in like a normal poker tournament.
Yeah?
If you're a short stack in a satellite or one of the shorter stacks, does going all in still make sense?
Because people can call you without risking a tournament
Right, exactly.
So in that case, you actually need a good had to go all in.
You actually need to have positive chip EV as well to go all in.
Yeah, so that's why when ICM is relevant, the small stacks win.
And the big-- sorry, the small stacks lose.
And the big stacks win.
Because the big stacks have more license to bully small sacks around, whereas the small stacks can't because the big stacks can call at them more often
And also does it matter if other people have raised behind you?
Does it just matter who goes all in first?
So if someone's already raised, then it's more likely you going all in-- it's more likely they are already committed so that if you go all in, they won't fold.
So that is relevant, yeah.
OK, and cool.
Right, so exactly.
So in practice, it's very tricky.
And in fact, it can be beneficial if you can somehow convince your opponents that you're not rational.
I've never personally done this, but I've heard of people making a bad call sometimes in a satellite just so people know not to mess with him essentially.
But I mean it's a bit weird.
There is this weird metagame with this traffic intersection-type game, although this is only relevant in the most extreme ICM situations, which are satellites.
Actually, I should say, one scenario where I think you could consider folding aces preflop in a normal tournament is actually this.
Let's say there's 55 until-- let's say 54 get paid.
And there's 55 left.
And you've got one chip, and everyone else has thousands of chips.
And you know that someone else has already gotten eliminated that hand.
So if you fold, you're guaranteed to get in the money.
Then I would definitely fold aces because even if you win in double up, congratulations, you now have 10 chips instead of one chip.
Everyone still has thousands of chips.
There are cases where you could construct an extreme case where you should fold aces preflop in a tournament, in a normal tournament as well.
OK, so let's get back to the tournament now.
OK, where were we?
So we lost a bunch of chips here.
So the next hand we get king-8 offsuit.
I raise from the button.
So we are committed here.
It sucks because our hand sucks, but it's way too good odds to call.
And ICM is nowhere near that relevant when it's against the small stack.
It's the guy you're trying to eliminate.
So we call.
We get pretty lucky here.
Good redemption for that terrible jack-10 hand played by me.
OK, so there's three players left.
So ICM is still very relevant.
We're the second stack.
So we're trying not to get in a big all-in with this guy, although it's less significant because this guy still has a decent amount of chips.
If this guy only had 50,000 chips, then I would try a lot harder to avoid getting in a big pot with BeMySponsor.
So-- oops.
So yeah, so he raises, and LPMBox goes all in, takes it down.
So this hand, he makes it 3x.
And I decide to call it here with queen-6 suited, which is gutsy I think.
But I think it's in position, and it's fairly deep.
And being able to put ICM pressure on him is reasonable.
I think it could have gone either way.
Folding is definitely the safer play, but queen-6 suited is plenty good enough here.
So I call.
He just checks.
He might just be giving up on this board where there's a decent amount of-- it's fairly dry, right?
So from what I said last class, if the board is fairly dry, where every hand has something, if you've got absolutely nothing, then there's not even much point to make a bluff like if he's got ace-2 offsuit here.
So he just checks in, and I bet, and he folds.
So he probably had ace-2 offsuit some hand like that.
Next hand we get ace-king.
I decide to raise small here once again.
So I guess this protects that jack-10 hand.
If I'm doing this with jack-10, I do need to do it with good hands as well to protect the times I do have jack-10.
Unfortunately, he guesses correctly.
He doesn't fold, and I have jack-10 and folds.
And I have ace-king.
So he still take that one downs.
And we win the blinds the next hand again.
So suddenly we're worried chip leader by a decent margin after those three hands.
9-4 off-- I'm not going to call, so fold to him.
So jack-7, so once again, because of ICM, this guy just goes all in knowing we can't call that much.
Takes it down.
7-6 offsuit, I decide to fold.
I think that's very reasonable considering they can just go all in on top of that.
Queen-9, queen-9 off, get a walk.
So one thing I should pay attention is I do think these guys-- by now I've collected enough data points of getting a walk way more often than I should, where I think it's not just a coincidence.
I'm statistically confident enough that they're actually just giving me too many walks.
So when they don't give me a walk, I should put them on a stronger range of hands than I normally do.
Pocket 10s, we three-bet this guy again.
He gets away again.
So 6-2 off, so we've gotten dealt good cards even though we haven't gotten too much action from them.
We're still winning by two chips from getting good cards.
So jack-4 off, I'm going to fold to this guy.
So as you can see, it's fairly boring, but I think this is what essentially should happen in the final couple of people left in the tournament.
Ace-10 off, so I decide to not go all in here.
I guess that would be 27 big-- 26 and a 1/2 big blinds, which is a bit much.
So I decided to just raise small.
If he three-bets, if he reraises to, say, 60,000, I'm always going all in here.
If he goes all in himself, I'm basically not folding even though it sucks from an ICM point of view.
I think ace-10 offsuit is just way too good.
So I'm essentially just raising small looking for action here.
He folds.
7-2 offsuit, we raise-- no.
[LAUGHTER] So LPMBox gets a walk.
Queen-3 off, we get another walk, which is nice.
Jack-5 off, I raise here, which is definitely an exploitative play.
It's definitely not-- definitely from a Nash equilibrium optimal play point of view, there's no way the optimal strategy is to raise a fraction of hand so big that it includes jack-5 offsuit.
But I think at the time, I probably told myself, this guy was being overly scared.
Even though he was playing somewhat aggressive earlier on, he is maybe overadjusting for ICM and is respecting me too much in the presence of this short stack.
I can look at the HUD stats at the time, although they won't be that accurate because they're going to include all the hands.
So normally I do play with all these numbers on.
So we can see that his-- yeah, but it's going to be biased because this includes the hands with six or seven people at the table.
But it says that he calls from the big blind only 20% of the time.
But that's not a great-- yeah, that's going to biased.
So it doesn't tell us too much.
But I guess at the time, I must have told myself I'm going to try to exploit this guy.
That's the only way I can see why I decided to raise jack-5 offsuit here.
So we get a call.
But I'm going to continuation bet.
We do have a backdoor heart draw, and this is our best chance of just winning the pot.
And I could maybe barrel on-- by barrel, I mean bet again as a bluff on a bunch of turns.
So I just bet here.
He does fold.
Next hand with queen-9, we open the button, but we get shoved on.
And I guess we have to fold.
Queen-4, we get another walk.
Yes, so this is actually getting ridiculous.
It seems like we've literally gotten five or six walks in a row.
And I would guess probably we should be getting a walk less-- I guess at a three-handed table, it's more often but still less than like 30% of the time.
Yeah?
Is it possible you're getting walks because someone has raised your big blind and you defended a lot?
And so they're scared to raise your big blind?
Maybe.
I have been fairly active.
But I did fold a bunch of-- I did fold some borderline situations, though, like this jack-- I think it was a different jack-7.
But yeah, maybe.
That's possibly.
Yeah, very good point.
But that's good, right?
It's good that we're like-- I'm not complaining that we're getting a lot of stuff.
And we are the big stack, so in general, it is harder for them to bully us around than us to bully them around.
So it does make sense from that point of view, too.
I saw that it was just too many.
Anyway, so I just continue trying to raise a lot, especially against this guy.
So jack-8 offsuit, we take it down.
Jack-2, maybe we could have folded.
But by now this guy was getting short enough.
He's blinded down from 15 and 1/2 big blinds, I think, to 13 and 1/2.
He might have had more before, like 17.
So he might be looking for a spot to shove now.
BeMySponsor actually raises a small here and gets him to fold.
And jack-6 off here.
We get another walk.
I should just skip the hands on big blinds since nothing ever happens, which is great for us.
So pocket 3s here, I just go all in.
So I talked about this before.
What's a problem with what this play in some sense?
Reverse implied odds?
Right, good.
So yeah, so if I don't do this, if I just raise small, I have bad reverse implied odds, right?
But one particular issue is every other time we've made it 25,000.
And really nothing's changed, and suddenly we go all in.
So it is a bit suspicious where he basically knows we have a hand that has very bad reverse implied odds.
He basically knows we have pocket 3s or ace-4 offsuit or something like that.
But it's still hard for him to exploit.
I mean sure, you can call ace-9 and exploit us, but ace-9 is already a great hand.
Or you can call pocket 4s and exploit us, but it's not that easy to get dealt pocket 4s or a better pair than pocket 4s.
So here yeah, even though against a good player, he knows my range is a very small specific number of hands.
It's still hard for him to-- he can't really just call jack-10 suited here.
Maybe he can, but probably jack-9 suited or queen-7 suited he can't call because there's still an off chance we have queen-9 offsuit or something.
Oops.
So anyway, so he does fold.
But it definitely raises suspicion.
Jack-10 off, so jack-10 off, we just go all in here.
So this is actually important.
So normally I would just raise from the button, but by now, the big blind has gotten so short that I'm willing to just go all in because the big blind is short enough where even if he picks up a hand and calls and beats me, I'm still the chip leader, even if that happens.
So the ICM doesn't hurt me that much because-- yeah, and also just the fact that I'm risking less.
The other important thing of the big blind getting short is now the small blind is much less incentivized to gamble than before.
Before, when the big blind had 180K, he was incentivizing gambling against me because it's not that easy to wait for the big blind to bust.
But now that the big blind only has 10K, it's easier to just wait and hope that he busts.
So basically ICM becomes a lot more favorable in both cases.
And shoving, I think, is very good now with the stack sizes.
So we do shove, and we take it down.
9-2 off, so no one usually wants our blinds, but then they both want our blinds.
LPMBox takes it down.
So ace-2, I just go all in here.
Basically same justification as the pocket 3s.
8-7 off, I think I probably should have raised this.
Even going all in honestly might not be that bad.
I'd have to do an exact calculation, but I think probably-- it's a pretty bad hand.
But I think probably I shouldn't be folding given that I'm the big stack and given that they've both been giving me lots of respect.
But I fold, which can be a big mistake.
But I think probably I shouldn't have folded.
The next hand, LPMBox goes all in, and we fold.
Next hand, we 8-7 suited, so by now, this guy, BeMySponsor, so we've just won enough pots where this guy is also short now.
So it's a pretty good situation for us from an ICM perspective.
They're both short enough where we can just go all in preflop very often and they have to fold quite often.
10-7 off is a bit too weak, but I decide to raise.
But yeah, this is a bit inconsistent play from my part because I think for most purposes, 8-7 off is a better hand than 10-7 off.
But for some reason I raise 10-7 off and fold 8-7 off.
He's 5-suited.
We get a walk again.
Queen-5 off, yeah, so I just go all in here.
So this is definitely not a plus chip EV all-in, but once again I think ICM by now is relevant enough where it's fine because he just has to fold, weakening it down.
10-6 off, I decide to fold.
Big blind gets a walk.
Get a walk with 6-deuce pretty good.
Ace-2 off, once again, we go all in.
They fold.
Pocket kings, so I basically--
Wow
Yeah, so this is a bit suspicious again.
Normally I've been going all in.
He raises small.
Obviously yes, pocket kings.
So yeah, that's basically true.
But I mean also I think I will do this with some of the weakest hands that I'm not willing to gamble all in with but still want to raise maybe something like 9-7 offsuit maybe.
But we don't play it, so we take it down.
So he goes all in.
We fold jack-2 offsuit.
Queen-3 suited, go all in.
So I'm going to go through this fairly fast.
There's a lot of same stuff.
6-5 off, we fold because yeah, by now they're short enough where they might actually just call us pretty often.
So I fold 6-5 off.
So here, pocket 3s holds against queen-jack.
So this is an extreme ICM spot.
So he calls here, and he does double up.
So the next hand, we do have an extreme ICM spot because this guy only has one big blind.
So essentially their correct strategy is going to be I'm supposed to shove any two cards.
And he's only supposed to call, I would guess, maybe the top 7% or 8% of hands in a situation.
He normally should be calling top 30 or something.
So I just go all in.
Even if he has ace-2 offsuit, I'm certain he has to fold.
Even if he has, say, king-queen, and I turn up my hand and show him jack-8, he probably still has to fold just because he really can't bust.
I haven't done the exact math, but I wouldn't be that surprised if he told me pocket 9s is a fold for him.
I wouldn't be that surprised, so yeah.
So once again, I go all in because this guy probably can't call.
Oh, OK, so yeah.
We win against ace-queen, which is crazy that they picked up ace-queen with one big blind, right?
Anyway.
[LAUGHTER] Some players like to do this thing called cooperative play to get that big blind to bust.
In the history of poker, it's what was almost customary for the players who have chips in here to basically just all agree to just call and not raise each other and to maximize their chances of busting the big blind.
It's a form of cheating in some sense.
It's collusion in some sense.
But it's customary maybe five-six years ago for everyone to do this because it benefits everyone else.
It benefits all of these guys if they all agree to do this.
And it just maximizes the chances that the big blind is eliminated.
But the issue with this is unfortunately there is an option to be selfish.
Unfortunately, if we both trusted each other completely, we could collude and be like, OK, we're just going to call, check down every flop turn and river no matter what to match so that we can both see the river with any two cards and maximize the chances of busting this guy.
The issue with this is when you play a game, you're selfish.
If there is an opportunity to bet because you have a good hand, you're going to want to bet.
If there's a draw and you have a good hand, you want to bet to get the other guy to fold.
And basically from a game theoretic perspective, since we have these options, there's no way for us to essentially agree that we're going to just check down and see the river, which is why I consider collusion of players actually do this.
So nonetheless, I do the most selfish line, which is just go all in.
But yeah, if this guy was my best friend and I wanted to cheat, then I could just call and let him call and then just check it down even if I flopped the nut flush.
OK, so now it's heads up.
OK, I'll try to show every hand.
OK, so now ICM doesn't matter.
But it would be good to show a little heads up play because I don't think we've done too much heads up play in this class.
So we call here.
And we check.
He makes it 32.
And I just have to check-raise bluff this flop.
I think it's close.
I think I probably shouldn't be folding.
I think jack-6 suited just has too many backdoor draws on this flop to ever fold just as part of my strategy.
I think I decided that I wanted my strategy to be I'm going to raise good hands, like 8s and 4s.
And then I'm going to also raise hands with lots of backdoor draws like jack-6 of hearts.
So I decide to do this.
So unfortunately he goes all in, so we fold.
It doesn't quite work.
So queen-10 off, we raise.
Oh, so one thing I should say is-- I think most of you know this-- but once a tournament gets heads up and it's just a small blind and the big blind, the small blind is actually has position.
So normally when it's small blind against big blind, the bid blind has position postflop.
But when it's heads up as a small blind, you actually have position.
So you're very strongly incentivized to play from this small blind.
It's just by far the better position because you have position, and you have to put in less of a blind.
So another cheat that-- so I'm teaching you guys all these different cheats.
Another cheat that people used to do-- it's not really a cheat.
But another angle shoot people used to do back in the day is there would be people who normally play pretty low stakes, let's say, $0.25, $0.50.
They would set up these $25, $50 tables.
So where the blinds are $25, $50, which is really high stakes for them.
They really don't have the bankroll to gamble that.
But their strategy basically is if you're the first to sit down, you get to be the dealer on the first hand.
So their strategy is basically just sit there and then wait for someone else to sit down.
And then play the first hand as the dealer, where they have a huge advantage.
They can just raise, and usually the other guy will fold, and they'll win 50 bucks, which is a huge deal to them.
And then they'll just leave after the first hand.
So this is something that happened quite a lot.
So my friends and I, we've talked about playing a crazy strategy, where you would sit down knowing this is what the other guy's going to do.
And then sit down with him.
We would have the capital to take on risk.
And then we would basically just play the first hand extremely aggressively.
And knowing that the guy has been fold most of the time, because if he calls, that's all his money on the site.
So yeah, it led to a lot of weird dynamics.
But OK, so here we have-- This is why I chose tournaments, not cash games.
There's fewer ways to do these when you use cheats.
So queen-10, we raise.
We take it down.
Next hand we get ace-queen suited.
So unfortunately we get a walk.
So queen-8, we raise.
We take it down-- sorry, he calls.
And then it's checked to us.
We continuation bet.
And he raises, which is weird because it's a flop that's a lot better for my range than his because I can have ace-king and ace-queen.
And even ace-jack, I think, if he had ace-jack, he would have reraised preflop.
So in some sense it's weird because it's really hard for him to have, I think, pocket aces or pocket 8s.
Or even pocket 3s, he probably would have reraised all on preflop.
So really it's like, what do you have?
It's like, I don't have much.
But really it's like, what do you have?
So it's weird because I expect him to check-raise this flop so rarely that what he does, on the one hand, it's just strange.
But on the other hand, he must know that.
I'm assuming it's somewhat balanced, and he will occasionally turn up ace-3 or something here.
So I decide to call and see one more card.
I don't know if that's great.
Maybe I should have just folded and just believed him or just reraised if I didn't believe him.
Reraise essentially is a bluff.
Make it 100,000, and just hope he folds.
But I decide to call.
The turn is a 9 putting out some straight draws but no flush draws.
That's fairly big, and I fold.
I don't really know how I feel about how I played that hand.
I think I mostly played exploitatively.
I just called the flop because I wanted to see what he did, and I folded the turn because maybe based on his bet sizing or his bet timing, I just felt like he had it.
So I'm not sure I'm extremely happy with how I played that hand.
But probably just at the time, I decided to play exploitatively and try to play best against how I expect him to act.
So here 10-7 off, I decide to reraise.
So the reason I reraise a hand as bad as 10-7 offsuit-- I talked a bit about this two classes ago about your preflop raising strategy.
So essentially what I'm doing is I want to be reraising my good hands and also my hands that I'm not sad if he goes all in, and I have to fold, and I don't get to see a flop with.
So if I had 10-7 suited, that's too good of a hand.
And I really don't want to reraise.
And then if he goes all in, fold my pretty 10-7 suited and not even get to see a flop with it.
But 10-7 offsuit is a bad enough hand where I'm not that sad if he just goes all in, and I have to fold.
So that's why I'm raising my good hands and also my weakest possible hands that I'm OK if I don't get to see a flop with.
He does call unfortunately, but we do hit a great flop.
So I'm just going to bet and get it all in.
Heads up basically any top pair is, in general, just way more good enough to put all your money in with.
He folds, though.
So ace-6, we raise.
I'm going to go fairly quickly because I'm running a bit short of time.
We continuation bet the flop and take it down.
He raises here.
I call with king-6, which is pretty borderline.
It's too good a hand to fold, but probably it's too bad a hand to call.
Probably I should have just raised by the same logic as before.
But we call.
And I decide to donk out on this flop because I figured there's a number of draws I can have.
And here I do have a weak draw.
A 7 gives me a bad streak but still a very good hand in the two-player pot.
He calls.
And the turn is a jack, which I still have a straight draw because now a queen gives me a straight.
So I continue bluffing.
So he folds, which is good.
6-2 offsuit I just give one to them.
So yeah, I think you should be playing 90-95% percent of buttons.
Maybe not that high when it's getting shallower, but at least 80% of buttons.
But 2-3 offsuit is one of the hands I'm OK with folding.
8-5 suited, I'm just check-raising this flopping, getting it in.
He folds-- yeah?
Let's say you had an opponent that from the big blind was very likely to three-bet you.
How would you adjust your range opening from the button or the small blind?
Oh, OK.
So if they're really loose, you would basically raise only your good hands that you're hoping they reraise all in with.
And--
So would that be from maybe 60% of your hands down from 80?
Oh, yeah.
You would basically raise less.
So it'd just have more value essentially
It's hard to calibrate that range, though, as what's a good hand
Yeah, and also another thing you can do is call more.
I don't think it gets relevant in this heads up match.
But sometimes it gets shallow enough where you actually want to have a calling range preflop from the button.
OK, so king-10.
OK, so I do call here.
I think I essentially call, thinking I would call if he went all in but also just to make it so that I can also sometimes call a hand like 10-4 offsuit.
And he can't just shove any 2 and expect me to fold.
So I decide to call here.
We get a great flop, so I check it back as a slow play.
He checks again on the turn.
So I bet.
He calls.
The river he leads out quite big.
And I just call because I think raising I'll usually only get called by better.
And he actually turns up a flush here.
So it's a weird hand.
It seems like we could have lost a lot more.
But I'm not saying our opponent played the hand badly, but we were very lucky to lose this little when we have a two pair against a flush.
And we only lost, essentially, four big blinds or something.
I guess I'll be happy to get away with that.
OK, I'll probably run two or three minutes over time.
I'll try to hurry.
But I don't want to not finish the tournament today.
So we fooled this.
Maybe I'll skip forward a bit to some of the bigger pots.
So what happened here?
So we raise jack-6.
He goes all in.
We fold.
All right, let's go do this one.
OK, so king-- so OK, so the match has gone on for a bit.
He's won a bunch of pots.
He catching up.
He's catching up.
King-3 off, we raise.
He calls.
So here the flop I think is-- so once again it's a situation where I don't think he ever has an ace.
So I'm basically betting at 100%.
He calls.
The turn's another ace, which is a bad card for us because now it's less likely we have an ace.
But normally, I still think it's a fine spot to bluff just because he essentially can't have an ace and we can.
But I think king-high is good enough here where I am going to win the pot by checking it down with king-high often enough that bluffing is superfluous.
So I just check.
And it's checked down.
And he was actually trapping with king-9 here.
So he wins that one.
Yeah, and probably he wouldn't have folded.
Let's say, I bet the turn and went all in on the river.
Probably he has to call just because it's almost the best hand he can possibly have.
It pretty much is the best hand you can possibly have other than maybe 7-6.
I think nothing happens there.
So let me see.
OK, so yeah, we lose a bunch of pots because we raise and he reraises and we fold.
So things aren't looking too good.
Oh, so I actually made a hero fold here.
I think, once again, this is an exploitative play.
I would probably never fold this from an optimal strategy point of view.
I think an ace in heads up with only 20 big blinds is way too good.
But he did just reraise me two hands ago.
And I don't know.
I think at the time I just felt like he almost certainly had it when he did this.
I made an exploitative fold based on what I thought he had.
So the next hand, he raises.
We go all in with pocket 5s.
And we've never [INAUDIBLE] the ace-jack which is pretty stupid.
Yeah, I was very surprised.
I was basically ready to just leave.
I was probably in some other tournaments.
But I basically was ready to close the table and focus on the other tournaments.
But somehow we get there.
And then the tournament just is easy from there.
We get ace-9.
We go all in.
He folds.
And then he goes all in into our pocket 6s.
And we hit a straight, so we win.
Right, So I played through all the hands at a tournament.
Once again, there's a lot of selection bias there.
But hopefully you learned some stuff from that.
So there's two classes left.
Next class, I'll wrap up the theory and the exact poker example.
So I'll talk a bit more about cash games and show you how to play when you're really deep.
I talked about postflop raising.
And then Friday we have a guest lecture from Bill Chen, which should be really exciting.
He always talks about very interesting stuff.
So I'll see you guys around.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, guys, I guess I'll get started.
So I guess one quick announcement is, so I guess everyone has taken a look at the prizes.
If you can take a look at the prizes and, especially, if you're near the top and expect yourself to be getting some prizes.
So we haven't announced the final ordering yet, because it'll also depend on the homework and then we'll decide how much.
It'll mostly just be number of points, but we might give a bit of weighting to people who have like a high points per game or something.
But if you think you have a decent chance of getting prizes, it'd be good if you looked them over for when you research who this Mike McDonald guy is, how much is his coaching worth-- you can do that beforehand so that people don't take forever deciding which prize you want on Friday.
OK, so I'm going to get started then.
So this is going to be the last class I'm going to teach.
So the final class, on Friday, is going to be guess-lectured by Bill Chen.
So I want to go through an in-depth combinatorial hand analysis of a cash-game hand-- go very in depth on a hand where you have deep stacks, and there's a lot of action, and there's a lot of deduction that can take place.
And then I'll finish with some general poker stuff and answer any questions you guys might have about the poker economy, how certain things work, the history of poker-- I'm happy to answer whatever questions.
OK, so first, I'll do something more technical and more practical to the game.
So an in-depth combinatorial hand analysis in cash games.
So I'm going to look at the following hand.
And the board is visible?
It's good?
OK.
So this is a cash game.
This is no longer a tournament now.
And everybody has 100 big blinds.
So the blinds are $1 and $2, and everyone starts with $200.
So it's 100 big blinds.
The cutoff opens near six, and we have eight-seven suited on the button.
And when it's this deep stacked-- so in a tournament, this might not be a good situation to call, because it just puts a lot of risk on you to bust.
And also, you're not really that deep stacked.
But here, with 100 big blinds, we can really realize a lot of our implied odds given that we're both in position, and we have a hand that plays very well-- a suited connector-- that can make straights and flushes.
So we decide to call, and the big blind also calls.
OK, so the flop is 10, eight, six with a club, and the cutoff continuation bet's 14.
So one thing I'll talk about first is, continuation bet sizing.
So I said a bit about this, but I wanted to talk a bit more about bet sizing.
And this is really just a review of what I've been talking about all class.
It's the same principles when deciding your flop and turn bet sizing.
So you don't want to bet so small that your opponent has the odds to call with anything, but you don't want to bet so big that you are risking a lot of chips when you're bluffing.
And also, if your raise size commits your entire stack anyway, like if you're roughly, let's say, betting 40% or more of your remaining chips, then you might as well go all-in if it's the flop or the turn because you probably aren't going to fold.
Given that your hand is good enough to put in 40% of your chips, you probably have enough equity that you're OK putting in 100% of your chips.
Remember that raising your bet gives them the option to reraise, so that's always an incentive against raising a flop bet.
The sizing should be bigger on dry boards and smaller on dry boards.
We talked about this.
And you want to bet?
Bet a bit bigger when they're out of position.
So he's kind of out of position here against us, and he bets 14 into 19, which is a reasonably big bet, but I think it's fine.
So with our hand here, eight-seven of clubs, I think we definitely don't want to be folding.
So let's talk about the differences between raising and calling.
So we decide to call, and so what's the analysis?
So we're definitely not folding.
Raising, I think, is a bit suicidal when we have a medium-strength hand that can play well in position on a lot of turns.
And we have a backdoor flush draw; and our straight draw's very legitimate when there is no flush draw out there.
So we decide to just call.
We have to be a bit worried about a big blind check-raise, but I think it's OK.
It's not that big of a deal, especially when there's no flush draw.
So the big blind folds.
And on the turn is the queen of clubs, and he bets 30.
OK, so this is sort of a tougher decision now than the flop, I'd say.
So let's analyze this.
So he bets 30 into 47, we've got 180 behind, which is about four times the pot, and we've got a pretty bad hand right now, but we still have a pair, and we have lots of chances to hit a straight, or hit a flush, or hit three of a kind or hit two pairs.
So tons of outs-- tons of river cards that help us.
And so what are the benefits of each?
OK, so I think it's fairly clear we're not going to fold here with this much equity.
So what are the benefits of calling?
Well, we get to see the river for sure, right?
If the river's a club, then we don't want to fold in situations and not see the river when it could be a club, which is the best card for us, essentially.
Yeah, so we really want to see the river because if we raise, then maybe they could reraise all-in, and then we would have to fold.
So that's one clear benefit.
And another sort of benefit is, we already have a pair to start with.
So we don't really need to bluff to win a hand.
So this is something I've been talking about a lot, too.
If you don't need to raise to win the hand because you already have a pair, then there's a lot more incentive to call.
So what are some benefits of raising?
Well, if you get him to fold something like pocket jacks-- if we think he might have like pocket jacks-- it can definitely get him to fold some better hands.
And we can bet the river and win a bigger pot when we hit.
And we can also maybe make some bigger bluffs when we miss.
So we'll go through a more detailed analysis in a bit, but I like calling in this situation.
I think with a draw that has no showdown value, like say I had nine-five of clubs instead of eight-seven of clubs, which has a pair, I think I would raise for two reasons.
One is, I would need to bluff the river anyway if I miss the river.
So if I raise now, I just give myself a chance to win the pot right now.
And another benefit of raising is that I can-- I'm not that unhappy if he goes all-in because my jaw is less strong than eight-seven of clubs is.
OK, and then maybe with a hand like ace-jack of clubs, I would raise just because we can maybe get it in against weaker flush draws.
OK, so yeah, this is a general principle that I haven't really talked about for playing on flops and turns.
And this sort of goes along with most of the stuff we've talked about in this class but, in general, when you have like the stone-cold nuts and when you have weaker draws, you want to be the one-- you want to be betting and raising in a way such that your opponent gets the last bet in.
So one person is always going to go all-in first, right?
If you're raising each other or betting and raising, one person is going to go on first, and the other person, basically, has to decide whether to call or fold and play for your stack.
So if you have the stone-cold nuts, then you want to be the person having your opponent go all-in first.
Because obviously, you have a very easy decision.
You just call and you're going to win.
On the other hand, if you have like a weaker draw, which is, essentially, your bluff-- because we don't want to be bluffing with absolutely nothing.
We want to be bluffing, basically, with our weaker draws.
That's also an easy decision because if, let's say, we have a gutshot straight draw.
So we have four outs only.
We're not that sad if our opponent goes all-in on the flop or all-in on the turn.
We just fold.
We have an easy decision; whereas if we have a more vulnerable good hand-- like you have top pair with a medium kicker, then it's a tough decision.
If our opponent goes all-in, we have a tough decision.
We're not sure what to do.
And same if we have a stronger draw-- like say we have a flush draw, and we have nine outs, and then our opponent goes all-in on the flop, and we have like 1.61 odds to call or something.
It's a difficult decision.
And so in these hands, you definitely want to-- you generally want to bet and raise in a way that allows you to get the last bet in.
Because with a stronger draw, your opponent gets to make a decision.
But even if they have the nuts, you still have nine outs.
And with a more vulnerable good hand-- I mean, if your opponent has the nuts, I guess you're kind of dead, but you're sort of protecting the times you go all-in with your stronger draws.
So following that principle, I decide to just call here because if I raise, then I have to make a decision when he goes all-in-- whether to call with my medium-strength hand and pretty good draw.
OK, so we call, and the river's an ace of spades.
So we completely miss, and our opponent checks to us.
So we decide to bluff the river for 70.
And this is sort of the main decision I want to analyze, and it's going to be quite a complicated decision.
So let's do some quick analysis.
So yes, we do have some chances of winning with our pair of eights.
So maybe you can make a case that there's no need to bluff because we do have a pair, but there's a lot of reasons to bluff.
So one is, there's a lot of higher cards on the board now.
The pair of eights-- they looked pretty good on the flop, but now it's a lot worse after a queen turn and an ace river.
So we can get him to fold a queen.
He can have a queen, and he could very willingly be able to fold it.
Also, the pot is big.
They bet the flop and the turn, and we called the flop and the turn.
So both ranges are just very strong because so much money has already gone in.
So a pair of eights is not so good.
If you knew that he had two random cards, sure, a pair of eights will win a decent percent of the time.
But when you know you're up against a hand that was willing to bet the flop and bet the turn, it's less likely the pair of eights is good.
And then, the ace is always a scary card.
And another advantage of bluffing is-- this is why position is so great-- especially when it's deep stacked in a cash game.
We're in position, and we know that he checked the river.
So maybe he's trapping with like king-jack, but it's more likely than not that it's a sign of weakness that he checked the river.
So we could try to bluff him off his hand.
OK, so what about river bet sizing?
So okay, so one thing is whether the bluff.
Two, is if you bluff, how much do you bet?
So I decided to bet 70 into 107.
And so what are the things to consider when you're deciding your river bet sizing?
So one is, there's no more cards to come on the river.
So in some sense, there's less of a worry to betting really, really small, because there is no such thing as letting your opponent see a draw.
But it still has a disadvantage.
Like if your opponent checks to you and you bet small, it still does give them the option to check-raise.
So that is one thing you worry about if you're trying to bet really small.
And another thing is, you should bet big if your hand is polarized.
By polarized I mean, you either have the stone-cold nuts or you have a bluff.
Basically, if you bet small and your range is polarized, your opponent can just call with a very wide range of hands because their odds are too good.
By betting big, you give your opponent worst odds to call, and since you have a stone-cold bluff some percent of the time, you don't want your opponent to have the odds to call with like third pair.
OK so, it doesn't matter what they did, because our decision is betting 70.
OK, so now what I sort of want to do is, a lot happened this hand, right?
We know they raised the preflop, bet flop, bet turn, checked river.
So in some sense, we have a ton of information.
So in the tournaments that you play, maybe a hand like this doesn't show up that often because often it's just an all-in preflop, and you can't really deduce that much about your opponent's hand.
It's just like, he has the top 30% of hands or something.
But here, we can actually do a lot of deduction.
So let's replay the hand from our opponent's perspective and sort of exploitatively put them on a range.
So let's go put ourselves in our opponent's shoes and consider all the actions they did.
So I'm going to use exploitative analysis here.
So there are some flaws with this.
We are sort of arrogantly assuming we're one step ahead of our opponent-- we can build a mathematical model for our opponent.
But still, this is a useful exercise even though, yes, there are flaws.
Maybe he's one step ahead of us.
Maybe there's flaws in our probabilistic model.
But let's just assume for the sake of the exercise that we can do this pretty well.
OK, so first of all, preflop, what's our opponent's range?
Let's just say, roughly, he's opening from the cutoff.
Let's say he's opening about 30% of hands.
I think that's about reasonable for like an average player, and I think it's consistent with the guidelines I gave in the first lecture.
So this includes any pair, any suited ace, any two Broadway cards, which are cards 10 and higher.
It includes suited hands as bad as five-three suited.
This is maybe one difference between cash games and tournaments, where five-three suited is a pretty bad hand in tournaments when it's so short; but when it's 100 big blinds deep, a hand like five-three suited is a lot better than a hand like 10-8 offsuit.
So yeah, so it includes five-three suited, but not like nine-eight off or like king-seven off, which are just terrible hands when it's 100 big blinds deep.
So OK, so what's our opponent's range here?
So they decided to continuation the bet.
They continuation bet into the flop against two players after the big blind check.
So what can we put him on?
OK, so this is a complicated analysis, and you can't really do this during the game itself, but it's very useful to do it after the hand.
So let's review the factors of why a player would continuation bet and then consider why he continuation bets.
OK, so this is also sort of a review class since the last class.
So what are the incentives for continuation betting?
Well, your hand is good enough that it beats most of their calling hands.
So you're betting for value or your showdown value is poor, but you have some equity.
You have some backdoor equity.
You have some overcards or draws, and you're, essentially, bluffing.
Another incentive is, you're out of position like he is, and it's not like he can see a free turn card by checking because I can still bet if he checks.
So you want to bet-- few opponents, you want to bet.
Incentives against continuation betting-- one is, your hand is so dominant that you need to give him a turn card to hope that he improves.
So this is like the trap-- the slow play.
You have decent showdown value, but not amazing showdown value-- so like, middle pair.
You don't want a continuation bet if you have like zero equity on a reasonably dry board like this one.
And there's too many people for you to fold out or-- like you're in position, and you can see a free turn by checking.
So there's a lot of things but, still, let's look at the situation again keeping some of those factors in mind.
He continuation bet into 10, eight, six rainbow, which means three different suits.
OK, so actually, before I show this slide, I'm going to use the annotate feature, and we'll try to write on here what hands we think he can have.
So this is going to take a while, but I plan on spending the next 20, 25 minutes discussing this hand.
OK, so does someone want to say a hand that you think is in this range at this point in the hand?
Pretend we don't know what he's going to do on the turn.
Yeah, just at this point in the hand, can someone tell me a hand you think is in his range?
Yeah?
Ace-jack
OK, yeah, ace/jack.
OK, I think that's reasonable.
It's actually, I think, a bit weak.
But let's say like ace/jack suited with a backdoor flush draw.
OK, I'll write down ace-jack for now.
OK, so someone go and tell me another hand.
Yeah?
[INAUDIBLE] pair.
Maybe 9-10, 10-jack suited?
Ace-10?
OK, ace-10-- sure, jack-10-- 10-9-- OK.
Someone want to tell me another type of hand maybe?
Yeah?
[INAUDIBLE] pocket jacks?
OK, good.
Pocket jacks, yeah.
So that's a clear value hand.
Someone want to tell me another type of hand?
What about queen-jack?
Queen-jack.
OK, that's a good idea.
Yeah, that's a good one.
So Queen-jack I think is one of the best draws-- not the best draw-- what is the best draw, would you say?
Queen-jack of clubs?
Queen-jack of clubs, yeah, but queen of clubs only-- it's not the best straight draw, right?
Yeah, what's--
Jack-nine
Yeah, jack-nine.
Jack-nine or queen-nine.
I wrote some hands here.
OK, jack-nine, or queen-nine, or like five-seven
One pair of sixes.
Is it possible?
Pocket sixes?
Yeah
Yeah, yeah, OK, that's good.
So I'll write that down.
So pocket sixes, good.
OK, here's a question, what do you guys want to put him on if he has a really, really good hand?
So this is open ended.
I didn't come with a [INAUDIBLE].. You guys want to assume that he checks with pocket 10s and like nine-seven-- basically the absolute best hands on this board to trap us?
Or do you want to assume that he bets?
Yeah?
I think if he has nine-seven, he'd probably bet because someone might get a better straight if he doesn't bet
OK, yeah that's a reasonable assumption.
I think it can go either way with nine-seven and like pocket 10s-- depending on our opponent.
Pocket sixes, I think, is a clear bet, whereas pocket 10s is not.
Because pocket 10s-- you're taking away so many of the top hands from your opponents, whereas pocket sixes just gets so much value when you bet.
OK, so roughly speaking, I sort of categorized all the hands he could have into five categories.
So the first-- the top is hands that are so good that he checks to trap or check-raise.
The second category is like the hands that are good but not that good-- he value bets.
Then it's like medium-strength hands that he checks to check-call.
So this doesn't agree with exactly what we wrote, but that's OK.
But this is just roughly-- and then the next category is hands that he bluffs.
And then the last category is hands that he just check-folds because they have no equity-- like pocket threes, where he would just give up the hand.
So we basically crossed off these three categories because with these three categories, he would check, whereas the top one, he's checking to trap; the second one, he's checking for a showdown; and the last one, he's checking to give up.
And then the two types of hands he's betting are, essentially, value bets and like draws or bluffs.
OK, so now let's consider the situation on the turn.
OK, so once again, let's consider his incentives first for betting again on the turn.
Sorry if this is a bit hard to read.
Maybe I should do it on notepad so that the annotate isn't annoying.
Sorry, just give me a sec.
OK, so what are the incentives for betting again on the turn?
So one is, you bet a good hand for value on the flop, and your hand is still good on the turn.
So you're getting for more value.
Two is, you bluffed a speculative hand on the flop and now you either hit the turn-- like the queen helped you-- so like if you had jack-nine or you improved your draw-- or like an overcard to the board came, which was the case, and maybe you thought bluffing the turn could be good.
So OK, so he bet the turn.
OK, so let's maybe put the hands we think he could have into PokerStove.
So yeah, I think that's the easiest way to do it.
OK, so let's do this.
OK, so roughly speaking-- can everyone see this?
This is easy enough to see, right?
OK, so let's do this.
So bear with me.
This is a meticulous exercise.
OK, so let's say preflop, 30%-- OK, this is about reasonable.
Actually, let me give him a few more pairs.
I'll take away ace-six off.
Let's say he goes suited connectors down to five-four suited, suited one gappers down to five-three suited-- suited two gappers.
So suited two gappers means suited cards that are three apart, essentially.
Yeah, you have like eight-five suited, something like this.
Something like this is fine, probably, but let's say he plays 10-6 suited.
OK, something like this-- maybe take away some of the worst offsuit hands.
OK, so something like this.
OK, good.
So on the flop, what happened?
On the flop-- we said, OK, so let's eliminate hands.
What can we eliminate?
So this is his entire range of possibilities, preflop.
OK, so on the flop, we're going to eliminate pocket twos, threes, fours, and fives because he just gives up.
Pocket sevens and nines we're going to eliminate because we're going to assume he checks.
Five-three is bad enough we can assume he gives up.
Let me just display the flop so that we can see.
OK, so what else?
OK, so let's go through everything.
I'll deal with this a bit fast.
OK, let's say all the middle pairs-- he's going to check.
So I'm going to remove most of the hands with an eight in them-- yeah, so ace-eight, king-eight, queen-eight, jack, OK, so 10-8 he's going to bet
You do that just because you have an eight?
Pretty much
Yeah, so we will take that into account.
But he could still have an eight, right?
But yeah, you're right.
It is less likely, and we will take it into account at the very end.
So ace-five, let's assume-- let's assume a lot of these offsuit aces, he gives up because even with overcards but with no backdoor flush draw on a dry board, I think is kind of scary.
OK, so let's say something like this on the flop-- eight-five, let's eliminate that.
Eight-seven, I think he'll check.
So nine-seven, we're going to assume he bets.
Let's assume he checks pocket 10s.
Let's assumes he also checks some of the weaker 10s-- like jack-10, let's suppose he checks.
Maybe 10-9, he checks as well.
10-7, he checks.
OK, nine-eight he checks.
OK, am I missing anything?
Did someone see something you disagree with?
King-queen off, king-jack off-- I think those are too weak, too-- that end of this board-- queen-nine, king-nine suited, ace-nine down here.
OK, I think this is roughly-- it'll get easier on the turn, basically.
OK, So let's say this is what he's going to have on the flop.
Actually, some of these weaker aces probably should be eliminated.
OK, let's eliminate everything down to ace-five and below.
Ace-seven and ace-nine we'll keep in because he has a straight draw.
OK, so on the turn, he bets again.
So did someone want to suggest a hand that maybe we should eliminate because he bet the turn?
Yeah?
Only weak 10s, I guess?
OK, good.
Yeah, I think that's a very good answer.
So let's get rid of ace-10 and king-10 because those are no longer good hands-- or they're no longer, I think, hands that you want to really continue betting for value, right?
Queen-10 is good because you hit two pair.
OK, is there another hand that's of a similar category?
So I guess pocket jacks is of a similar category, right?
It's not as good as it once was.
OK, so what are some other hands maybe you don't want to bluff with-- you don't want to bet with anymore?
Are there any hands you think maybe he would just give up because--
The six-five
Yeah, I think that's reasonable.
Most of these, we can assume he's going to give up on the queen, like five-four, especially.
So on the flop, he didn't even have a great hand.
It was just drawing to a seven.
But now, it's sort of worse.
So yeah, let's get rid of five-four.
Do you think they would continue betting like nine-six or seven-six?
Yeah, even seven-five is a pretty bad-- is a lot worse after the queen because seven-five was like drawing to a nine.
But now, when the queen comes, the nine is not a good card because you still lose to a jack.
So maybe we can get rid of that one.
Do you think they definitely continue betting with ace-king, ace-queen, ace-jack in all those?
Yeah?
Maybe if they had ace-queen, they would keep it [INAUDIBLE] pretend to [INAUDIBLE]
Yeah, so ace-queen, I think, is good.
All right, so I think this is [INAUDIBLE] good.
Ace-nine-- some of these hands, I think they would probably stop bluffing with.
Like I think jack-seven suited is pretty-- maybe some of the sixes also, I think, it's hard to-- even like ace-seven suited, I think it's hard to continue bluffing.
Because as we see, there's already enough hands with draws, right?
There are already a lot of bluffs here like king-jack, ace-king-- I guess like queen-nine, nine-six, some of these-- so yeah, maybe he'll bluff with these, but OK, let's get rid of these.
OK, so we've sort of cut down his range a bit.
So now it's a river situation, and he checks.
So what does this mean?
Well, it seems improbable that their opponent would be trying to trap us with this check because he's already been showing plenty of aggression.
So it's hard for him to pretend he suddenly has a bad hand by checking.
And with the pot already so big, he can get a large percent of our remaining stack into the pot just by betting instead of check-raising.
So it seems like it's less likely he's checking as a trap.
And also, the ace is sort of a better card for his range than ours because he's the one who's been betting, so having draws, having overcards like aces-- and for us, we're much less likely than him to have an ace.
So if it's a better card for his range, he should be betting both for value and with his bluffs.
So that being said, so let's suppose we assume it's sort of unlikely he has a really, really good hand, but he could so easily have a hand that beats ours and calls our potential bluff.
So OK, so now let's go back to PokerStove and eliminate some more hands.
OK, so what are some hands we can eliminate after he checks the river?
Aces
Aces, yeah.
I think that's a good one.
I think aces, pretty clearly, he'll bet again on the river.
What's another hand?
King-jack
Yeah, king-jack I think he might.
OK, so let's eliminate all other good hands that we think he's going to continuation bet, right?
I think with any straight, I think it's reasonable to assume they're just going to keep betting.
Like nine-seven-- even though it's nowhere near the monster it was on the flop, it still only loses to jack-nine and king-jack.
So let's get rid of that one
Certainly like ace-queen-- all those
Yeah, I think most strong two pairs I think, definitely, he'll keep betting.
So let's get rid of that.
Let's get rid of ace-queen.
Let's get rid of queen-queen.
Let's get rid of queen-10.
I think queen-10 is good enough.
Ace-eight-- let's get rid of that one-- yeah, that's good enough.
Pocket eights, pocket sixes I think is good enough.
OK, good.
So OK, we've got some hands here-- basically, all the good hands we removed-- all the really good hands that he bets again.
What do you guys think about 10-8?
I guess it doesn't matter too much.
Let's assume he checks 10-8 because by this river, 10-8 is no longer the powerhouse it was.
Oh, I've got to get rid of jack-nine.
OK, what are some other hands you think we can eliminate?
So there there's actually another category of hands we could, potentially, eliminate here.
Yeah?
He would probably bet an ace
Well an ace isn't that good of a hand.
So this is one problem with this analysis is, it is very subject to debate what we think about our opponent.
But for the sake of this exercise, I'd say-- I mean also I think in practice-- I think it's not clear that you should value bet an ace here.
Because there's just a lot of stuff that beats you-- a lot of straights, a lot of two pairs.
I think that even if you have a ace-king, you shouldn't really be confident enough in your hand to bet.
Like maybe ace-king is marginal but definitely not like ace-seven.
Yeah?
Bluff bets-- so like weak hands, maybe six-four
Great, great.
Yeah, exactly, so that's what I wanted to hear.
So we can also eliminate some of his really weak hands.
This is crucial.
This is an easy thing to miss.
We can probably assume he's not going to have king-seven because the ace is a reasonable card to bluff.
It's not great-- it's not like the ace of clubs, which would be a better card for him to bluff, but if he's got nothing-- like if he just completely missed-- probably-- I mean, let's assume you know these guys are pretty good players.
He's probably going to bluff.
So let's assume he at least has a pair.
He has some showdown value, because if he had zero showdown value, he would have bluffed.
So let's get rid of king-seven, let's get rid of king-nine.
Six-five six-four-- I think, definitely, he should be bluffing, but maybe some players will see a pair of sixes and say, oh I have enough showdown value.
So let's get rid of one of them but not the other.
Maybe that's reasonable.
So let's get rid of two.
So let's get of six-four and seven-six but keep in six-five and nine-six.
OK, so we leave ourselves with these.
OK, so this seems about reasonable, I think.
Let's eliminate ace-seven.
I think we should have eliminated it on the turn.
Maybe like ace-seven of clubs we can-- oh no, ace-seven of clubs he can't have because we have the seven of clubs.
I think any other ace-seven other than the ace-seven of clubs is kind of bad to bet on the turn because the seven isn't really that good of a card to-- isn't that good of a draw.
So let's get rid of ace-seven.
So let's just suppose it's this.
OK, so that was a lot of work.
OK, so now roughly speaking, we can conclude his range on the river is something like this.
So let's assume that he's going to call with top pair or better.
Once again, this is a pretty strong assumption, but let's suppose this is the way we think he's going to play.
He's going to call if he has an ace or better and fold other ones.
So we're risking 70 to win 100.
So let's do an exact calculation now of how many combinations of these hands.
I'll do this on Notepad.
OK, let me pull up the board.
OK, hopefully, people can see this.
So when I said combinatorial hand analysis, I mean we're going to, basically, count the combinations.
And like you pointed out, this is-- oh crap, did I accidentally get rid of that?
Oh, that's not good.
OK, let me quickly put it back.
So OK, we had kings left, ace-king, ace-jack, ace-nine, king-queen, queen-jack, queen-nine, what else?
10-8, 10-6, eight-six-- we had some sixes, right?
Seven-six, six-five-- and is there something else I'm missing?
Does anyone remember if there's something I'm forgetting to check off?
I knocked queen-nine off, I guess.
I think this about right.
Maybe I'm missing like one or two things.
No one sees anything?
I think this is approximately what we had, right?
OK, hopefully I didn't miss enough that it drastically changes.
It shouldn't.
OK, so let's count the combinations.
Because you can look at the square on the PokerStove and-- let me clear this annotation-- sorry, I should've done that a while ago.
OK, so you can see the yellow hands on the square and roughly see how many hands of each type.
There's, essentially, three types of hands he could have.
There's hands that call, there's hands that beat us and call, there's hands that beat us and fold, and there's hands that we beat and fold.
But the thing is, if you just look at these yellow squares, it won't give you the right probabilities because we have certain cards in our hand.
And also, because of the specific ways suited combinations work, there's different combinations of each hand.
So OK, let's go through this.
So how many combinations of ace-king suited are there that-- so we assume he had a backdoor flush draw on the flop, but-- so how many combinations of ace-king suited are possible?
Don't worry about the backdoor flush draw part.
It doesn't matter for this example.
So how many combinations of ace-king suited, Will?
Three.
Because it has to be suited.
And it can't be spades because the ace of spades is on the board.
So it's three combinations.
OK, ace-jack suited?
Three
Three, OK, good.
Ace-nine suited?
Three
Three.
OK, good.
All right, pocket kings?
About six?
Right, so pocket kings is six.
Because there's no kings; we don't see any kings, and there's no such thing as suitedness.
So yeah, so one thing you realize in this-- oh, so pocket kings we're assuming he's going to fold-- one thing you will realize in this exercise is, you've got to put a lot of weight on the offsuit hands instead of the suited hands.
Because the offsuit hands, combinatorially, have way more possibilities-- essentially three times as many possibilities-- as the suited hands.
So like this queen-jack offsuit in here has a huge weight in our calculation.
That's by far his most likely hand.
Because it's the only offsuit hand we're putting him on.
Yeah, it's the only offsuit-- is it really only the offsuit hand that he could have?
I guess according to our analysis, we're assuming the only offsuit hand he could have is queen-jack off.
So how many combinations of queen-jack off are there?
12
12
OK, so let's just say queen-jack in total.
Queen-jack suited plus offsuit is 12.
Because we see a queen, so there's four jacks, three queens, three times four is 12.
It could be suited or not suited.
So OK, that counts for both of those.
What about king-queen suited?
Three as well?
So it's three as well, but I'm going to put it as two because I can assume if he had king-queen of spades, which has no flush dras-- like no runner-runner flush draw on the flop-- I'm going to assume that he would have folded king-queen of spades on the flop.
So I'm going to give it two.
OK, so which ones haven't we done?
Queen-nine suited, how many combinations?
Three?
Yes
OK, 10-8 suited, these get five.
So the reason why 10-8 is suited is because we're assuming he doesn't raise 10-8 offsuit preflop from that position.
So that's why 10-8 has to be suited.
So how many combinations of 10-8 suited?
Two.
OK, how many combinations of 10-6 suited?
Two as well
Two.
How many combinations of eight-six suited?
One?
One, yes.
OK.
I miscategorized this one.
So queen-nine is in this category that he's folding.
These are all the hands that call our bluff and beat us.
And so the last category is the hands that we can actually beat if we check it down.
So it's very small.
It's just seven-six and six-five suited.
OK, so seven-six suited, how many combinations?
Two?
Two.
Six-five suited, three?
OK. Did I miss anything?
I think this is it, right?
OK, so basically we have to tally it up.
So here, there's 14.
Here, there's 23.
And here, there's five.
OK so now that we've done this calculation, we need to, basically, run the numbers with these calculations.
So essentially, roughly speaking, they're going to fold more than half the time.
So the numbers-- I think this will actually come out close because with the way we did this specific calculation, I think we gave him a lot of combinations of hands that we can actually check and beat.
But I think this is still-- this will still definitely call for a bluff.
But it close.
If you've done the homework, the homework sort of addresses this, right?
It's not just the probability he's calling our bluff, it's the probability he's calling our bluff versus the probability we win the hand by checking.
And the probability we win the hand by checking is roughly 5 out of 42-- so which is pretty small, and he folds more than half the time.
So if you run the numbers, you'll see that bluffing is overwhelmingly profitable.
And I think this is a good example of why this combinatorial analysis is good-- is because if you look at PokerStove and you look at this, you might actually think bluffing is bad because you look at most of the hands-- ace-king, ace-jack, ace-nine, 10-8, 10-6, 8-6-- it's hands that call us and beat us, but when you do this combinatorial analysis, you'll realize there's way more combinations of queen-jack than there are eight-six.
So looking at this itself isn't quite enough because this doesn't weight the probabilities of each one correctly.
So this is a good exercise to do, I think.
Even though we made a lot of assumptions in our model, but it's a very good exercise to do after the fact to sort of just analyze what are the different-- all the different possibilities.
OK, so I'm going to take a quick break here.
And then for the last half of the class, I'm going to just talk about some general poker topics.
So no math.
No poker hands.
But just some general stuff about poker and some ending remarks since this is the last time I'm teaching since Bill Chen is coming next time.
If you want to, during the break, think about any question you might want to ask me about poker in general-- or specific-- and-- yeah-- so we'll take like a 10-minute break.
Make sure you hand in the homework.
All right, so I'm going to wrap up with some general thoughts about poker and poker in general.
So one question that I get asked a lot is, it's called the Moneymaker effect.
So what happened-- and this is sort of a big catalyst for poker becoming really popular-- was, I think in 2003, Chris Moneymaker, that's actually his name-- Moneymaker is his last name-- he spent like $1 playing in this satellite on PokerStars, won a seat to the World Series of Poker main event, and then went there, and then won the whole thing for like $2.5 million.
It was a beautiful Cinderella story, and then people heard this.
And also, they made it seem like, you know, it's not like the lottery where you've got to get really, really lucky to become rich.
It's literally, this guy's really smart.
He can read people, and it's all skill.
And he did this.
And it's sort of like, anyone can become a poker star just like Chris Moneymaker.
And this was a huge driving force in the popularity of poker.
So a question that I wondered myself is, how could there actually be so much money?
Because someone has to be losing this money for you to be winning this money, right?
So where is the money coming from?
Can anyone smart and motivated succeed in poker?
Et cetera.
OK, so what's unique about poker?
Why can't you make this much money playing like chess or something?
Why can't you make this much money playing hockey?
So what's unique about poker?
So I guess four unique aspects that allows for there to be so much money.
So I'm going to talk about them individually.
One is, I think everyone is overconfident, myself included.
Two is, gambling self-control-- aspects of that.
Three is, it's a very fast-evolving game.
And four is, what I sort of started the class-- the very first class I talked about being credit card roulette-- not being results-oriented.
And I think that's very hard.
So I'll go through each one of them separately.
So yeah, one is this huge overconfidence thing, which is very prevalent.
So it's normal, in general, for people to be overconfident.
I don't know how many of you have seen the experiment where it's like, you know, write down your 95% confidence intervals for all these things.
So if they're actually 95% confidence intervals and they're correctly calibrated, then actually 19 out of 20 of your intervals should contain the real thing.
But like you do this for a normal person, it'll be like, 30% of their things actually lie neutral.
So they're just drastically overconfident.
And so yeah, it's normal to be overconfident, but especially in poker.
I think poker is among the things I know where it's easiest to overestimate your own abilities.
Does someone want to suggest any others?
Is there something else you think where like, everyone thinks they're good at this thing?
So I can think of two examples that I think are sort of true.
Raise your hand if you think you're a below medium driver.
No, but some people actually are willing to do it.
Because I very rarely talk to someone who admitted to being bad at driving.
Everyone I've ever asked about their driving skill thinks they're really good at driving a car, myself included.
I definitely don't think I'm a below medium driver.
Another thing is teaching.
I've also barely talked to someone who thinks they're worse than medium at teaching.
I guess maybe Professor [INAUDIBLE] can speak about this.
Most things someone will admit to being bad at this, but poker is similar to these two examples I thought of-- driving, teaching, sort of.
Where it's just very easy to think you're really good at it, and it's hard to admit to yourself that you're not good at it.
Yeah, so one saying in poker is, if you can't spot the fish at the table-- the fish is like the losing player that everyone is trying to win money from-- then you are the fish.
This is sort of a popular saying.
So why is overconfidence so common in poker?
It's like a mental battle.
Your mentally battling against your opponent.
Is he bluffing?
Is he not?
And it's just easy to always assume-- which I do throughout a lot of this class, right?
I talk about exploitative play.
I'm building a model for our opponent and assuming that they behave as a probabilistic machine according to my model.
I'm assuming that I'm ahead of my opponent.
I'm assuming I'm better.
There's also a lot of selective memory, I think.
It's easy to remember your bad beats-- the time they hit a king on the river and beat your aces-- and it's easy to forget the times you got really lucky on the river and beat them.
So selective memory, I think, is a big aspect.
I think lack of clear benchmarks is a big aspect as well.
So it's not like running 100 meters.
No one who takes 30 seconds to run 100 meters is going to think, oh, I can compete in the Olympics, right?
Because it's very clear.
There's a very clear benchmark how fast you need to be.
But in poker, even if you don't win tournaments, it's easy to complain about luck.
And there's different types of tournaments.
And it's easy to like bend the benchmarks in a way to convince yourself that you're better than you are.
It's easy to hit a lucky streak and consider it all skill, and you can blame all your losing streaks on luck.
So those are some reasons I think.
I think poker is just a very well-designed game for this purpose and to, basically, tricking and deceiving people into overestimating their own ability.
And another aspect of it is, and this is what's really tough poker, I think, is, even though it's so easy to be overconfident, and it's such a flaw to be overconfident, it's also, in some sense, necessary.
You know, if you're not confident, how can you take risks?
Poker-- you're gambling, essentially, and you have to do risky things.
And if you're not confident, how can you do risky things, right?
And also, if you're trying to mentally read your opponent, if you're constantly remind yourself that you're dumber than you think you are, then how can you actually think that you can outread your opponent?
And I talked about this results-oriented.
You need to be confident to trust that you made a good decision even though your result was you lost $10,000 on the day.
So confidence is also necessary, but it's also so easy to be overconfident.
So this balance of being exactly the right confidence is something that I think even like professional poker players strive to try to strike the balance between every day.
And I think it's very tough.
And it's something I struggle with a lot, too, to try to figure out when I'm being overconfident and when I'm being underconfident.
So I think one good mentality-- so back to this driving thing is, I think it's a good mentality-- you know, if you play poker and you enjoy it.
I think it's a great game.
It's a beautiful mathematical game and a fun game to play with friends if you're just are willing to admit, you know, I play in this home game every Friday night.
I'm probably a below average player.
I probably lose, on average, $10 every time I play.
I think that's totally fine.
It's just like spending your Friday night going to see a movie.
You pay $10 and you're happy to take $10 out of your wallet and not see it again, right?
I think it would be a healthy mentality for more people to be willing to do this and just admit to themselves, yeah, I enjoy playing poker, and I lose a bit of money, but you know, I have fun doing it.
And it's the same as seeing a movie but, in reality, I think this isn't the case.
Even at these friendly Friday night games for low stakes, probably, if you ask all the people there, do you think you're beating your friends?
Everyone's going to say yes.
Everyone's not saying, oh, I'm going there, essentially, paying $10 to play poker with my friends.
Everyone's saying, I'm winning a bit of money or whatever, right?
So I think that's an important mentality to try to have that.
And I think it's fine.
You don't have to be the best at everything, right?
So one story I like to tell is, so David Einhorn-- he's like a billionaire.
I think he founded Greenlight Capital or something.
And he actually won $4 million in a $1 million buy-in poker tournament and donated it all to charity.
And after the fact-- he's always like a super smart guy-- so he played a $1 million buy-in tournament.
So everyone else playing this tournament is like a poker professional-- the best in the world-- and he's just like this rich billionaire who doesn't really-- like he wasn't bad, but he just started playing poker.
Like, he's a smart guy, and he got coaching to play it, but he was the clear loser in the tournament.
And he just admitted it.
He just said you know, I'm a billionaire.
I'm happy to spend a million dollars playing this tournament.
I understand I'm probably expecting to lose like $200,000 or whatever playing this tournament with all these guys, but I'm fine with it.
I got a million dollars, who cares, right?
And he was just very honest.
And even after he actually won $4 million, he didn't come first, he came like third-- he just admitted, you know, he said like, I was very lucky.
I came in expecting to just lose this $1 million that I don't care about, and then he just donated all his winnings to charity, anyway.
But I wish more people were like him.
I think that's a very good mentality to have to just admit that you're playing poker for fun, and that's totally fine.
OK, so the second thing I want to talk about is gambling self-control.
So I think this can also be the downfall of a lot of smart and motivated people trying to get into poker.
So these are some things that can happen.
So after getting unlucky in the previous hand, you play the next hand poorly because you're upset.
You're tilted, as poker player say.
Or playing when you're tired just to get unstuck.
I've definitely done this many times before in my poker career.
If I lost money, especially when I feel like I'd gotten unlucky playing against someone who I'm better than and just like, I've lost money because I got unlucky, I won't stop playing until I get unstuck, which is like, win back the money I lost until I'm in the black, essentially.
And it's terrible because even if I'm a bit better, if I'm playing with this mentality trying to get unstuck, I'm just going to be making below average decisions, and it's just not a good thing to do.
And also from the other side, it can be hard to rationalize gambling.
So it's hard to stop yourself from gambling too much, but it's also sometimes hard to rationalize that, you know, gambling-- there is a lot of stigma around gambling.
You have to convince yourself that, I shouldn't be too scared.
I'm playing this game that involves-- you know, you can call it gambling-- I'm playing this game with a lot of luck.
And you've got to convince yourself that, this is what I'm fine with doing.
It's like a lottery.
And you've got to sometimes make decisions under pressure as well.
And you've just got to rationalize to yourself that, yeah, there's a lot of luck in this, but this is the game I chose to play.
The third thing is-- so fast evolution, I think, is another big aspect of how there was so much money in Texas hold 'em for so long.
So it was a relatively new game, also with a lot of hidden depth.
So something like this can never happen in chess, where chess has been studied for hundreds, thousands of years, and it's not like a new game.
And even though the best player today is still better than the best player 20 years ago, it's not like the game is evolving super fast, whereas hold 'em was like a new game-- a relatively new game at the time.
And just-- there was so much hidden depth, and the best players just kept improving so fast that-- so like you say, like the best player in the year 2000 would be like a terrible player by 2004.
And like the best player in 2004 would be a bad player by 2008, et cetera.
So it's just, even if you're on top of the game right now, if you stop studying, you're going to be nowhere near the top of the game in like a year.
So it was just so fast.
Can anyone else think of other examples of things that are kind of like this?
CrossFit was-- well, if you get like-- the winner of the 2009-2010 games wouldn't even compete
Oh, wow.
OK I didn't know that.
That's cool.
I didn't know CrossFit was that good, yeah.
That's pretty crazy.
There's one other thing I can sort of think of.
How many you guys have done math contests as a kid?
I think those, to my knowledge, have gotten a lot harder, just in terms of how good you need to be to say, be on the United States national team is way harder now than it was, say, 30 years ago from what I've heard.
But yeah, so this aspect of Texas hold 'em-- so this was also a huge driving factor.
Because it would be easy to tell yourself, I was the best player in the world four years ago.
I could obviously sit down at this $1/$2 game and make money, right?
And the answer would often be, no.
So this was another reason why there was a lot of money in it.
Because it's easy to just remember that you were the best a year ago and just forget that the tides are rising so fast.
So that being said, so I wanted to also take this opportunity to suggest some further readings for-- I know I've been asked this.
Yeah?
So why is that?
Based on your understanding?
I think because of your strategies-- you have to improve your strategy?
Yeah, yeah, I also think there was a lot of hidden depth.
So I mean, I think with any new thing, this will be the case.
But I think in poker, there were many times where the consensus amongst the top players was that they were close to solving the game.
And then a year later, they would realize they missed completely viable strategies that completely messed up all their calculations, and they had to start over.
So I think that was big.
It's easy to think that you've solved the game and then suddenly realize, oh, I'm nowhere near solving the game because I forgot about all of these different strategies.
This might be sort of true with a lot of-- I'm trying to think, I mean-- the biggest thing is, it's a new thing.
Like it's hard for this to happen like-- I think in most sports that people have done for a long time, people get gradually better, right?
You look at like, the 100-meter sprint.
It's like, yeah, the world record is getting broken, but it's not like it's going to go from 10 seconds to five seconds.
It'll go from 10 seconds to 9.99 seconds or something like that.
So I think just the fact that it's new is a big aspect.
But yeah, Texas hold 'em would be the equivalent to the world record in the 100 meters being 10 seconds then like five seconds a year later, then like two seconds like two years later, it was just insane.
OK, so that being said, so some people have asked me, if I want to continue learning about poker and reading poker, what should I do?
So that being said, I think the best resources, by far, are online.
A lot of people have asked me about books, and I will recommend some books, but I think books, basically, blew out of date way too fast.
So I think, in general, the best resources are online.
So cardrunners.com is a website where, basically, poker pros can make videos of themselves while playing and talking.
You do have to pay to get a subscription.
OK, so this is a bit of a biased advice, just a warning, because I'm a pro at CardRunners, but, I mean, amongst like 50, 60 other pros.
They also donated some free memberships to students in the class if you guys looked at the prizes.
So you can get some free memberships to CardRunners.
A good free resource is Two Plus Two forums.
I think there's a lot of garbage on those forums nowadays but, still, most of the best poker players in the world still do use those forums and posts on those forums-- so twoplustwo.com.
There is a lot of garbage and banter, but if you are just trying to improve, there is good content on there.
You just have to find it.
So this is a new thing, but on Twitch streams-- so Twitch is a website you can go on to watch, essentially, poker pros play in real time.
They share their screens and talk through their hands.
It's a pretty good free resource.
It's sometimes more entertaining than educational, but I think it's quite good.
And they would do like a 10-minute delay so that you can't watch them and you know what cards they have.
In case anyone was considering trying to do that to get an advantage.
So these are some online resources, and I will suggest some books, although I do think most of these books, I think, are reasonably outdated.
But I do think they're very-- in my opinion, out of the books I've read, these are the best-written books.
And even if they're a bit outdated, I think the theory and the way they're written is very good and also somewhat entertaining and sort of gives you an idea on the history of poker and the evolution of the game.
So my favorite book is the first one.
It's called, Small Stakes Hold 'em.
It's by Ed Miller, who's a MIT graduate; David Sklansky; and Mason Malmuth.
So this is actually limit hold 'em, which is, in some sense, a solved game nowadays, and no one plays limit hold 'em anymore.
So limit hold 'em is where you can't bet any amount.
You have to bet a very specific size.
But I think this is one of the classic books.
So this is one of the classic books in poker.
It's very well written and written by mathematicians.
And it just goes through the basic concepts of poker, I think, very, very well, even though it's for limit hold 'em.
And it's a good read.
Yeah, it might even be a collector's item nowadays.
So one story I'd like to tell about this book is-- so my good friend, Mike McDonald, he was-- so he's the guy who got me into poker.
He's probably read this book, he says like-- maybe like 20 times.
And then he's also a guy who doesn't really like reading.
So I think there was a point in his life where he's read less than 20 different books in his life but then this specific book more than 20 times.
So it's a very good book.
Harrington on Hold 'em is on tournaments.
It's outdated.
It's really, really badly outdated, but I still think it's very well written and has some good concepts.
That's how I first started learning to play tournaments is Harrington on Hold 'em 1 and 2.
Kill Phil/ Kill Everyone I think is pretty good, pretty up to date-- it's decent, I think.
Every Hand Revealed is-- Gus Hansen's a very famous poker player who's been around forever.
So it's a book where he goes through every hand he played in this tournament that he won.
And I think it's more entertaining than educational, but he's definitely a really good player.
Mathematics of Poker by Bill Chen.
It's not that practical-- don't tell Bill this when he comes Friday-- it's theoretically very interesting if you like math-- if you like game theory.
It's theoretically very interesting.
Building a Bankroll is a recent book that-- this year, he didn't have any leftovers to donate to our class but, in the past, he's supported our class and donated a lot of copies of the book.
And I think it's pretty up to date, and it's for cash games, so I think it's very good.
OK, so back to the four things about poker that I think makes it unique.
So the fourth point is this idea of not being results-oriented.
So yeah, so we talked about this a lot in the first class already-- this decision mentality, where you need to care about the decision you made, not the result you got.
And I think this actually is a barrier to a lot of smart and motivated people getting into poker because it's almost antithetical.
If you're a smart and motivated person, you're used to like-- you study hard, and then you get a good result on your tests, right?
If you get a failing grade in your test, you're never going to be able to go up to your parents and be like, oh, I make good decisions, but I just got really unlucky and failed the test, right?
No one's going to believe you.
In poker, this can happen all the time.
So it is a bit antithetical to sort of being motivated.
One thing poker players like to laugh at is, I don't know if you put the word, results-oriented on your resume?
I've seen seminars that teach you to become "results-oriented," and advertising this as being a great thing.
You achieve results.
You work hard, make decisions, and you achieve results.
But we always laugh at this.
Because results-oriented is always a negative term in poker.
If a poker player says someone is results-oriented, they don't respect them very much.
But I've seen resume's where people say, I'm very results-oriented.
So a related thing is, it's easy to underestimate the variance in poker.
And this is a big aspect, too, that goes on with being results-oriented.
So this statistical experiment involving making up sequences of coin flips-- I've heard of this where a professor would ask students to make up a sequence of 200 coin flips.
Can we get half the class to do this?
And then can we get the other half to actually flip 200 coins and write down like heads, tails, heads, heads, heads, tails?
And he could tell with like 100% certainty which ones are made up and which ones were real.
And the way he did this is basically looking at which of the-- what's the longest sequence of heads in a row or tails in a row?
And basically, in reality, if you flipped 200 coins-- I forget the exact numbers, but I think it's something like, you're like 99% to get at least like six or seven of the same thing in a row, whereas in all the made-up ones that people would be like, oh if there was heads seven times in a row, that can't be random, right?
So we can't put seven hands in a row.
So I think it's known that it's easy to underestimate how long you can get unlucky looking for.
Like it's easy to think, I've been unlucky the last three times; that means I must deserve to get lucky now.
But this is not true, mathematically.
And it's just very easy to underestimate variance and underestimate how likely it is that you're just actually going to lose 10 coin flips in a row in poker.
It's going to happen to you, and you're going to get tilted.
And it's very important to control tilt.
And yeah, pretty much every poker player thinks they're unluckier than mathematically possible.
I've definitely felt like this in many parts of my career.
I still feel like this sometimes now.
It's very easy to get this feeling where you just, how can it be possible?
How could I actually lose 20 coin flips in a row?
Well, it can happen.
And one of my poker players that I was talking to, John Cannon, he plays a lot of online poker, and this is actually a picture of his desk.
He, basically, was so angry I think he smashed his mouse and made a big hole in his desk.
So it can really frustrate you.
And I hate to say this, but I think even if you're working very hard, you do need some lucky big scores along the way, especially at the start to get into it, even if you're very smart and working in the right way.
And this is also antithetical and smart.
Antithetical to smart and motivated people, right?
Because we want to believe that if you work hard and you're smart, then you'll make it for sure.
But I wouldn't say this is true in poker.
I mean, I think I'm willing to admit that I think I was very lucky at the start.
I had some big scores at the start that really drove my interest in the game and really propelled me into professional poker.
And I think there probably, in a parallel universe, could be another copy of me that did the exact same things and just didn't get as lucky as I did at the start and just never got into poker.
So this fact that you do still need to get lucky, even if you make all the right decisions [INAUDIBLE] make it is-- It's sort of a tough thing to swallow.
This is sort of a different story, but I'd like to argue that that's sort of true in life, too-- the fact that being smart and working hard doesn't guarantee success by any means.
So although that's a negative thing for poker, I'd like to end with what I call, "the joy of making good decisions."
So even though not everyone makes it in poker, but poker players-- we like to talk about sort of, you're not really there.
You shouldn't be trying to think about making money or whatever.
You're sort of just there to enjoy the game, and have an honest opinion on your ability, and calibrate your confidence.
I've told the Bill Gates/Doyle Brunson story but yeah, basically, Bill Gates, I think, was a pretty good poker player for quite a while.
And he would make money.
And clearly, it was not his best way to make money.
He could probably just go to his company and make like $500 a second or something like that.
But when asked about it, Bill Gates would just say, I love making good decisions.
I love thinking about this game and making decisions.
And even though, clearly, the money means nothing to me, it's still important to me, as a personal goal, to succeed at this game and make money, even though it doesn't mean anything.
So it's about this joy of making good decisions.
And I think part of the reason Bill Gates was able to be quite good was because the money meant nothing to him.
He didn't really care about his results and how much he made or lost, clearly.
He just focused on making the best decisions.
And so it is true that there's a decent amount of luck in poker but, yes, I would argue, life is luck, and you only live once-- there's all these sayings but, overall, the thing to keep in mind is, like Jennifer Shahade talked about this with the "Goldilocks" video-- so you want to think of your life and/or your poker career or whatever as one long session where you're just trying to make the best decisions.
And even if you don't get the best results today, hope that you get better results throughout the course of your life.
So yeah, that's the end of what I wanted to say about the general poker and what I hope you take away from this class from a non-mathematical, non-poker point of view.
I guess there is a bit of time for questions.
There's a lot of things I didn't talk about, but I'm happy to answer questions about.
Yeah, I'm happy to answer any questions, now.
Yeah?
Now that there are no longer any class tournaments, where can I go to play to get better and improve?
So there's an MIT poker club, and they run tournaments on PokerStars similar to the way the class works.
The way you can get into the poker club is so [INAUDIBLE] is part of it.
She's like one of the execs.
Maybe you should ask her.
Send her an email.
I can ask her.
I'll ask her, and I'll send something to the class if you guys are interested in continuing playing poker on PokerStars and-- yeah, so there's the MIT Poker Club.
And I know in the past, sometimes the classes continued playing tournaments after the class is over.
We would have like a tournament every Saturday or something for people to play, but I don't know if-- they didn't last very long.
I think people quickly lost interest in it.
But yeah, I think just the poker club.
Actually, Martin, you're in the Poker Club, right?
So maybe you can something to answer that question
So sometimes we have [INAUDIBLE] games where people can play like $0.25, $0.50 [INAUDIBLE]..
So there's like a million you can get on
OK, I'll find some information from [INAUDIBLE] and Martin.
I'll send something to the class, and I'll give you guys some pointers
How do you, personally, deal with downswing?
Yeah, so definitely, I've gone through bad downswings and times where I really hated the game.
I think I mostly just took a break.
So yeah, I mostly just took a break.
My first couple of downswings when I was first starting, I just took a break from the game for a while, like a month or something, and then came back.
That's sort of what I did.
Nowadays, I'd say like the last four years, I've been-- I'd like to think I was fairly good at dealing with downswings.
I sort of went through enough downswings where I kind of was able to just keep it together through it for the most part.
Like maybe I'd play slightly less, but I wouldn't quit altogether
So for small cash games online, what kind of edge do you think you have?
For small?
And then what kind of edge-- what is your like, borderline edge where it suddenly wouldn't become worth it anymore?
Oh, so how much edge do I think I have?
So in online cash games-- the thing is, I don't really specialize in cash games, so I never really play them.
So I'd say it's probably very little.
I mean, I'm sure I can find low enough stakes where I'll have a significant edge--
Or even in tournaments, then?
Online small stakes tournaments?
Yeah, yeah, so in tournaments, I'd say there's still a decent edge, and I'll play it for-- but mostly, I'd I'll play poker, mostly, for fun nowadays.
I rarely force myself to do it for making money, I'd say.
Most of the time I'm playing, I'm still trying to make good decisions because I enjoy doing that, but I'm not really doing it for the power degree or whatever
So what about maybe the best cash-game player in the world, how much would they make in about $0.25/$0.50 online?
Oh, $0.25/$0.50 online?
So I don't have a great sense of this, but let me think.
So the thing is, the players who are sort of considered the top players in the world who play like very high stakes-- like nosebleeds stakes with like $100/$200 big blinds or something-- it's sort of, the variance is so high that it's not really accepted who's the best because if it was accepted, you know, someone had a win rate over someone else, then the guy who was worse just wouldn't be playing.
So the only reason those games run is because they all think they have a win rate.
So it's hard for me to say the number because I'm not going to know better than what they're true win rates are against each other.
And they're all going to tell me something different, because they all think they're beating the other guys.
As far as how much a really top player can win at a low-stakes game, it's still going to be very significant.
I think making something maybe like, five or six big blinds per 100.
Yeah, five or six big blinds per 100, I think, is maybe doable at a reasonably low stakes-- if you're like one of the very best.
It's quite a lot.
It's quite generous.
But let's say every 100 hands you make five big blinds.
So let's say you play $0.25/$0.50, five big blinds is $250.
You make that every 100 hands.
If you play six tables, you can get in about 100 hands per hour, 120 hands per hour.
So that's 600 hands per hour.
So it's six times $250; so it's like $15.
So it's not a lot, but I mean those guys can play stakes way higher than $0.25/$0.50 and still have a slightly smaller edge.
So if you imagine they play 250/$5.
Then suddenly, you multiply, then that'll give me about $10.
I mean you would be decreasing it because they probably wouldn't make five big blinds per hour-- five big blinds per 100.
But I think those numbers are ballpark correct, yeah.
All right, yeah?
What's a good for playing at a casino, like exploiting people?
What's a good strategy for playing at a casino and exploiting people?
One general thing that's going to do is, you go there-- you sleep at night, and you go there at like 7:00 AM and play against people who have been tilted and angry for-- [LAUGHTER] And you just woke up at 7 AM in the morning.
And you're in such a good mood, and they're all in a bad mood, and you can win a lot of money.
In terms of specific strategy, I think it depends a lot on the player.
I think, in general, playing tight is pretty good.
At least I'd say I consider myself, in general, a tight player, and I think a decent amount of money can be made at casinos against recreational players just by them getting bored.
Like at a nine-headed table, you're only paying the blinds-- 1 and 1/2 blinds every nine hands.
You can easily get away with only playing like pocket nines plus, ace-queen plus.
I mean, that's extreme, but they'll probably still call you because they're bored.
They're not going to think like, this guy's-- I'm going to just fold-- so one disadvantage of that strategy is, they're just always going to fold to you.
But in practice, I don't think that's what happens.
So I think, in general, playing tight will always be a pretty good strategy at casinos against recreational players
Yeah?
Do you have any advice for avoiding this pitfall of thinking you're good and you're not actually?
Not really, because I think it's happened to me a lot of times.
Yeah, I think it really is hard to judge, especially in tournaments.
In cash games, it's slightly easier to judge because you can just look at your data points, and it'll converge faster.
In tournaments, the variance is so high you can't really use your tournament results.
I think the most important thing, for me personally, was just talking to players and having very honest friends who would tell me if they thought I was worse than someone else and tell them not to play them.
And I think that's probably the thing that benefited me the most-- just being able to talk to players better than me or at a similar level to me and just getting their opinion from them talking to me.
If they think I'm not good enough, I want them to tell me.
And I think that's one of the most important things is just getting good feedback from your friends and honest feedback.
Yeah?
Since we can't play online here, and we can't do it at the casino like every weekend-- so we can't play for money-- what's some good motivation to keep the Saturday game?
Is this like poker mentality, and these kids will have development in whatever they like [INAUDIBLE] jobs?
OK, so I'll address a few things.
So one thing that I didn't really formally announce-- but yeah, so one thing you pointed out is, playing online poker for money is illegal in Massachusetts.
The police can't come arrest you, but the website will take all your money if they find out you're playing from Massachusetts.
But I don't believe it's written in Massachusetts law or something.
To my knowledge, the police can't arrest you.
But yeah, so the question was, if we can't play online, what can we do?
So I think it is legal in New Jersey and Nevada.
So you can move to New Jersey or Nevada.
OK, yeah, so if you don't want to go to the casino every weekend, so what else can you do?
OK, so what's the motivation to keep playing?
So I think I've tried in this class to give examples of where I think poker is very useful, just in terms of how poker is related to nonpoker stuff, like calibrating your confidence, figuring out what your biases are, and stuff like that.
And I also think it's a fun game.
It's a very fun game.
It's mathematically interesting.
But yeah, I mean, if you don't enjoy playing the game much, and sense you can't really make money playing it in Massachusetts, yeah, then, if both of those are true, then there's not that much reason to play it if you don't enjoy it.
I think that's just the biggest thing.
If you enjoy it and you enjoy the decision-making, I think it's very fun, and it's also very good for you
Thank you
All right, cool.
Yeah, so if there's no more questions, then I guess we'll call it here, and then Friday will be the last class.
There's two more nights of tournaments left.
And yeah, I'm excited for Friday for Bill's lecture and to hand out the prizes.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Some funny things happened on the way to the moon, as a takeoff of Funny Things Happened on the Way to the Forum.
And probably not too many people in this audience know that.
Also, in particular, a thing that's happening this year is the 40th anniversary of the lunar landing.
And we're going to talk a little bit about how MIT got involved.
And let's see if we can move this along.
Sputnik was about the size of a large, maybe oversized beach ball, or a basketball.
And it was launched in June, of 1957, or later, maybe not June.
I thought I had it on the screen.
But anyhow, if it hadn't been for Sputnik, I doubt if anything would have happened in terms of the United States getting interested in the space program.
So we have to have a lot of gratitude for the Russians for doing this.
I remember Wernher von Braun in a meeting was asked, this was after maybe six months of the program, says, "Wouldn't we do a lot better if we collaborated with the Russians?"
And he, without hesitation, said, "If there were collaboration with the Russians, there wouldn't be a program in either country."
So it had all to do with the race, the race for the moon.
It was October 4, I'm sorry, 1957.
Right after the lunar landing, Dave Hoag and I were invited to the Soviet Union as their guests, all expenses paid.
It was really quite an experience.
And they gave us a good show.
Everywhere we went-- we had some movies of the lunar landing.
And as soon as they knew we had movies, they wanted to-- they took them and set up their screen.
If the didn't have a screen, they'd put up a bed sheet.
And they oohed and aahed in all of the right places for the landing.
And it was really quite an experience.
And also they asked us, when will the Americans go to the moon again?
Well that was Apollo 14.
And the date was sometime in November.
And I was always grateful that it went on time and on that date, because the Russians never gave any indication of what they were planning on doing until it was all over.
And then they would take bows, or they would hide the results.
But anyhow, I happened to buy this little device.
It's actually a music box.
It plays the Russian national anthem.
It used to, but it's all rusted out in there now.
It's been around a long time.
And when we were in Moscow, we went to the GUM department store.
And I saw this there and bought it.
I'm so glad that I did.
And I brought it home.
1957 is particularly interesting to me, it's because my son, my youngest son, was born in that year.
And so he really became-- everything that happened in space, he was right there to watch it on TV, or whatever news there was.
And he's still much like that today.
This the MIT Instrumentation Laboratory responded to-- we're going to go into space, so why don't we design a spacecraft which will indeed take us, not to the moon-- we aren't going to think about the moon-- but actually into interplanetary space.
In 1957, when the Sputnik was flying, the MIT Instrumentation Laboratory was-- that building on the right is the Whittemore Shoe Polish factory.
And the one on the left is the old Hood's Milk factory.
And the railroad tracks there are the same tracks that cross Mass Avenue today.
The buildings, of course, are gone.
And Doc Draper's office is indicated as a little, the corner of these warehouses.
So we didn't really have plush surroundings.
The thing we wanted to do was to build a Mars probe.
We would like to launch this spacecraft.
And it would coast to Mars and pass close by, take a picture, and then return to Earth so we could collect the photograph, not a whole series of photographs, just one picture, because we didn't want to have all that time and effort invested and find that the wheel which would change pictures was screwed up.
So it was just the simplest thing to do, would be to just have it open the shutter and close it.
And that was the only thing that had to be done.
And we'd do that at the point of closest approach to Mars.
The round trip from Earth to Mars and return, there it is.
We were serious about this.
We were thinking of launching it in December of 1962.
And we would intercept Mars on February 15.
And then we would pass by the planet.
It's not a two-dimensional problem.
This is the relative motion of the spacecraft with respect to the planet during the planetary contact.
Then after that, we have to get back to the earth.
And unfortunately, we had to make two trips around the sun to get back.
And so the total trip time was more than three years.
That did not play very well with those who are interested in getting some attention for the work that the laboratory was doing.
But they did get some.
This is a picture from the newspaper.
And I'm on the right hand side, a younger version of me.
And there's Hal Laning and Milt Trageser, all of whom were involved in this program.
And it said, this project may solve the life on Mars mystery.
But it didn't.
In fact, it did do a lot of things.
But it never flew.
The Mars probe involved always a three year round trip to photograph the planet.
And we had to have onboard, self-contained navigation.
For three years, this little vehicle was going to be cruising and making measurements to determine where it was and when to make velocity corrections, all done on board the spacecraft -- unheard of at that time.
And after the three-year trip, it would splash down in the Gulf of Mexico.
And this little model, which is a good size, is hanging in the Draper Laboratory.
I used to know where it was, but then they moved it.
But they said, it's somewhere in the building.
So if you want to take a look at it, and you can get into the Draper Lab building, then you can see this.
The big red, that's the entry vehicle, which was, in it was the film of the picture that we took of Mars.
The major problem was how to design a small computer and its memory in 1960 to survive a three year round trip to Mars, with no data uplink and no inflight service.
That doesn't mean what you think.
Inflight service means if something breaks, you can't fix it.
It's got to be perfect.
But then came 1961.
Alan Shepard was ready to fly the Mercury-Redstone on May 5 of 1961.
It was the first American in space, not the first human being in space, the first American.
The Russians had already capitalized on their Sputnik, and they had a man in space somewhat before we did.
And you look at the picture, there's the blockhouse.
It looks like about the size of a garage.
And the space vehicle, not very big, all it was going to do was just fly a 20-minute flight and land in the ocean and be fished out.
That was what we were going to do, when in fact, the Russians were already putting men in orbit.
Now, I will never be as amazed that three weeks after Alan Shepard made his flight, that John F. Kennedy said that he believed that this nation should commit itself to achieve the goal, before this decade is out, of landing a man on the moon and returning him safely to Earth.
Now, that's pretty gutsy to do when your only success is a 20-minute splashdown in the Atlantic with one person at the helm.
Now, 1961, let's look at some of the other things.
Alan Shepard's flight on May 5; May 25, President Kennedy's message to the Congress and to the world.
And then on August 10, MIT is awarded the first NASA Prime Contract for Apollo.
NASA had just barely been created.
And in response to President Kennedy's "we're going to go to the moon," they gave the job to MIT.
That's pretty amazing when you think about it.
And we'll say a word more about that in a minute.
The Mars probe never flew, but the onboard computer did evolve into becoming the Apollo guidance computer.
There was a computer that we had to have in our little space probe, and that did evolve into the guidance computer, the Apollo onboard computer, for the lunar mission.
We moved into better quarters.
This is the home of the Apollo project from 1962 on.
That building no longer exists.
It was originally a storage warehouse for underwear.
And we took over the whole building, refurbished it, and my office was on the other side of the building.
And it was on the corner.
It was a nice office.
Here's what the place looks like today.
This is the luxury waterfront condominiums that they made long after the space program ended.
And the underwear factory or underwear warehouse was replaced by these condominiums.
Now, MIT was chosen for the GN&C.
And quite a big question is "Why?"
Well, Jim Webb, who was the administrator of NASA, knew Doc Draper.
And this is Doc's version of what happened.
Jim Webb called Doc Draper on the telephone, says, "Doc," he says, "Can you develop the guidance and navigation system for Apollo?"
What would you say if you were Doc Draper.
You would say, "Yes, of course.
We'll do that."
"When will it be ready?"
"Well, when you need it."
(Laughter) That's a good schedule.
"And how do I know it will work?"
And you say, "I will go along and operate it for you."
Now that's the story that Doc always told.
And I believe it, because he did know Jim Webb.
Jim Webb was not a scientist.
Jim Webb was good at getting money out of Congress.
He was a lawyer.
And his job was to make sure that the Apollo program was adequately funded.
Now, was it Doc Draper's phone call from Jim Webb, or was it Bob Chilton's memo?
Bob Chilton worked for NASA all of his life.
He's retired.
He lives down in Texas, at Texas A&M.
And I hear from him once a year.
And here's what Bob Chilton did.
He wrote this memo to the Chief of the Flight Systems Division in November 7, 1960.
He said, "Development of the onboard capability to determine position and velocity by optical means is important to the principle of onboard command for Apollo.
The Instrumentation Laboratory of MIT has shown a strong interest in the problem of self-contained navigation.
The position and velocity determination schemes proposed in the reference report by Battin and Laning called A Recoverable Interplanetary Space Probe in 1959 are alike in principle to that desired for Apollo."
At the time, Bob Chilton was the head of the Flight Dynamics Branch at Langley Field.
Bob Chilton's memo, the memo was stamped Confidential.
Everything was classified in those days, and Bob gave me his original carbon copy, which I still treasure.
Here we are sending men to the moon, and we had not even invented a good copy machine.
Why was onboard navigation a basic requirement?
Well, one of the reasons would be that the Russians might not play fair.
They might jam the communication links and we would not be able to communicate with our spacecraft.
1961 was a long time ago.
We had to use typewriters and carbon paper.
Xerox had not been invented.
You could watch TV, but not in color.
There were just black and white TVs and not very good quality.
You could do calculations using electromechanical machines, no personal computers or even pocket calculators.
These things would cost about $800 to $1,000, and all the would do is add and subtract and divide.
If you spent $500 more, you could get one that did square root.
And that was a pretty amazing thing, because every day, just about, the repairman would be there to fix it, because it was such a complicated piece of equipment.
When we'd go on business trips, no jets.
We had propeller-driven aircraft.
And the per diem rates were $12 a day.
With that $12, I was supposed to pay for my hotel bill and buy three meals, and do it for $12 a day.
And that was plenty good enough.
In fact, if you just said, give me the $12, and then if you didn't spend it, you could keep the dollar or two which was left.
Those were different days than they are today.
Now let's look at, this came about only relatively recently, the numerology of 1961.
It's an invertible number.
That is, if you turn it upside-down, it's still 1961.
Earlier invertible years, 1001, 1111, 1691, 1881.
When will the next invertible year take place?
6009.
You've got a long wait for the next invertible year.
OK, that's one thing.
But there's more.
President Kennedy's Special Message to Congress was 5/25/61.
5 plus 25 plus 6 plus 1 is 37, which is a prime factor of 1961, which is only interesting.
But if you take the prime factor, which is 37, there's only one other prime factor, and that's 53.
So 53 times 37 is 1961.
If you take 53, and you get the month of August, 5 plus 3 is 8, and the 10th, 3 plus 7 is 10, and it's all written there forever that MIT would get the Apollo contract on August 10 of 1961.
Isn't that-- that's pretty amazing.
I was going to give a talk on Apollo.
And I had it in the afternoon.
And I was fiddling around, and I came up with this.
And I was really impressed.
And so were the audiences when they realized that this was, as I pointed out, this is not an accident.
This is the way it was all planned ahead of time.
Now also, Apollo is Greek mythology.
Apollo is America's Program for Orbital and Lunar Landing Operations, so even the Greeks knew that we were going to do the job.
1961 was a vintage year.
A lot of things happened then.
There they are.
In January 26, the first multiple fly-by orbit of the planets was determined, by going from Earth to Mars via Venus.
You go from Earth to Venus to Mars and back to Earth, all in one launch.
And that was discovered by me in January 26 of 1961.
I have to emphasize that that date is very important.
Because there was a lawsuit by somebody whose name I must not mention, who brought suit against me, because he said he did it first.
But he didn't really.
In the spring of that year, the first offering, that is the first class in Astronautical Guidance, is what my course used to be called until it was changed to Astrodynamics.
On April 12, Yuri Gagarin flew three orbits of the earth.
And then to respond to that, on May 5, Alan Shepard did a quarter-orbit flight.
And then, three weeks later, President Kennedy commits us to going to the moon.
And in August, Gherman Titov flies 16 orbits of the Earth.
And MIT is awarded the first Apollo contract.
In October 13, the American Rocket Society had a meeting, because Kennedy said he wanted to know what progress had been made.
And so we had a spaceflight report to the nation of all the stuff that we were doing.
Later, on November 21, Draper, who had said to Jim Webb that yes, he would make sure it worked, because he'd fly it, he volunteered formally to be an astronaut.
And then, I thought of this much later, December 23 was Christina Shue's [?]
100th birthday.
Who in the world was Christina Shue [?]
?
My grandmother.
So in 1961, a lot of things happened.
Here is the first round trip to Venus.
We were a little discouraged by taking three years to get to Mars and back home.
So we said, maybe we could go to Venus and return in less time.
And as a matter of fact, you can.
If you see the launch, and you go past Venus, and then you're coming back to Earth.
But look, you're almost out to the Martian border.
And wouldn't it be nice if Mars were there?
And that was what we did.
We had a Venus fly-by.
And then following that is the return to Earth.
And it is all done in just a little over a year.
And we visit two planets for the price of one, first to Venus and then to Mars, and then back to Earth.
And this was the very first one of these that was ever done.
We do that all the time today, using fly-bys of planets to change the velocity to get to another location to accomplish something else.
But this was the first time it was ever done on a computer.
Now saying that the first offering of astronautical guidance, it's kind of interesting.
Because there are the people who took that course, Buzz Aldrin, who flew in 1961, in Apollo 11, and then Dave Scott, who flew Apollo 15, in 1962, and Ed Mitchell, who was on Apollo 14 in 1963.
Each of these gentlemen, and they're all still alive and well, they walked on the moon.
And they took astronautical guidance.
Apollo 14, I always thought that Ed Mitchell, it had more people watching that launch than any time in the past or the future.
Why?
Why?
It was Apollo 14.
What was the flight before that?
Apollo 13.
And that was pretty serious.
We almost didn't get them home.
And then to be a crew for the next flight, I think, took a little gutsy.
Here's the textbook for the graduate course in astronautical guidance.
The book is Astronautical Guidance.
The diagram, if you take my course, you'll find out how to navigate in space by making measurements of angles between stars and near bodies.
And so that little diagram was on the flyleaf, and was also on the inside of the book in color.
Now when the book came out, we were having a heavy load of important visitors.
And Doc, he took my book, and he said, "Battin," he says, "I want you to buy a lot of these.
And I want you to autograph them.
We're going to have important people here."
And sure enough, let's see who the important people are.
Well, there's Doc.
And then there's Kurt Debus.
He was the head of the Space Center in Florida.
Wernher von Braun needs no introduction.
Bob Gilruth, who was the head of the NASA team in Texas, and then me, looking smug.
And everybody, you see, has got a copy of the book in front of them.
And I was pretty excited about that.
There was an unauthorized version of the book, Astronautical Guidance, in Russian.
And this was about four years later.
And everything is in Russian.
There is Battin in Russian.
And the diagram is exactly the same diagram that appeared on-- but it's an unauthorized, that is, they didn't ask permission to do this or give us any royalties.
But they did give us an all expenses paid, round trip visit to Moscow and Leningrad and Tbilisi after the Apollo 11 landing.
MIT awarded the first Apollo contract, and here was the team.
We have Jim Webb, the one in the center.
And to the left is Hugh Dryden.
He was the deputy administrator, but he passed away in December of 1965.
And then there's Bob Seamans, the associate administrator.
He was promoted to deputy administrator.
And Bob Seamans, as you are well aware, passed away.
I was very happy that when I give this talk at this time last year, Bob Seamans came.
He had not heard it before.
But he came and listened.
And I was so pleased that he was there.
But he passed away shortly thereafter.
MIT was chosen.
Why?
Well, Bob Seaman's book, Aiming at Targets, says, it doesn't give you much information.
It just says, "When the decision was made to go to the moon, the Draper Lab got the contract for the navigation system."
He really knew why, but he didn't want to put in print.
Because after all, he was from MIT Instrumentation Laboratory.
Doc headed the laboratory.
And then all of a sudden, we get the job for the GN&C for Apollo.
But we deserved it.
And we performed well.
So was it Doc Draper's phone call, or was it Bob Chilton's memo?
I guess I got this in there twice.
I shouldn't.
I have to do a little surgery on this.
Doc Draper volunteers to be an astronaut.
He really did.
And he had a letter.
This is part of his letter.
"I would like formally to volunteer as a crew member on the Apollo mission to the moon, and also for whatever sub-orbital and orbital flights that may be made in preparation for this lunar trip.
I realize that my age of 60 years is a negative factor in considering my request, but General Flickinger tells me that this is no sure bar to my selection as a crew member."
And we had a system, a real system, put on the roof of one of the buildings on the river, where you could practice navigating.
And there's Doc Draper making a navigational fix.
And you can recognize the Boston skyline, or what the Boston skyline looked like in those days.
Some buildings are missing.
And you could actually make measurements, and the computer would calculate where it was on the basis of that.
And the result was always the particular spot on the roof of the building, even though you could start off with very bad guesses of where you were.
And then by utilizing the navigation algorithm, you could to get a navigation fix.
The guidance navigation laboratory system, you could see that today, if you get somebody to take you through the Draper Lab.
This is actual-- it's not a mock up.
It's really the equipment.
And you have the means of communicating with the computer on the right, that was the DSKY, the display and keyboard.
where you could get information, and you could also call up different things that you wanted to do in flight.
For navigation purposes, we had a scanning telescope and a good sextant to make these measurements.
You could get a view of the stars and the planets through the sextant and then make the actual measurements.
You'd try to line them up, line up these two images, press a button to get information to the computer.
In fact, what we have here does not show behind, where the ball for getting information with respect to attitude and velocity, all done on the back side of the-- but this is the front side, is this what the astronauts saw and used in their flights.
There's the Apollo guidance computer.
The fact that it's approximately one cubic foot, it's a really interesting story.
When the spacecraft contractor was chosen, you got a phone call from them, saying, "Oh, I understand that you're going to have a computer on board.
How big is it?"
Well, we had no idea.
All we could see was all this equipment that was on the wall.
And somehow or other, we were going to have to package this down to something that would fit in the spacecraft.
So they said, "Well, we've got to give him a number."
And so I said, "Well, just tell him it's a cubic foot."
And it was.
We were able to squeeze it.
You couldn't make it two cubic feet, because that's all they had room for, planned for.
It was a computer of that size.
And it weighed about 70 pounds, consumed 55 watts, which, that's pretty good.
And then this up close is the display and keyboard.
The core rope memory was a-- we're not going to spend much time dwelling on it.
But it was an invention of Hal Laning, primarily Hal and others, that you could have a memory system which you would wire in place, so that if you lost electricity, you wouldn't lose your memory.
And there is the way it was done.
Each core would store a word.
In fact, we got it down to where a single core could store 12 words, and each word was 16 bits or 2 bytes.
And to read, to get information out, all you have to do was select the core and switch it.
And then all of the sense wires that went through the core would produce 1s, and those that did not go through the core would produce 0s.
And lo and behold, this was our system.
The trouble is that you had to make these things.
You had to manufacture them.
And after you've made it, and if there was a mistake, you couldn't change it.
Here's the speed in kilobytes.
For our Mars probe, it had eight read only memory kilobytes, and change, this would be Random Access Memory, is a half of a kilobyte.
The Apollo Guidance Computer had 74 ROMs and 4 read and write memory, and it was actually a little slower in speed than the Mars probe.
Now today, the kind of memory that we're talking about, you can carry around in your pocket.
In fact, I do have it in my pocket for this talk.
And it's just absolutely amazing that first of all, that they had the idea that this was the way to do it, because in Apollo 12, the spacecraft was struck by lightning on the way up.
And all of the read and write memory vanished.
They had to put that back in by hand.
But the read-only memory was fine.
There was no trouble with it whatsoever.
Here's a photograph of the core rope memory up close.
And how would you manufacture something like that?
Well, we used the LOL method, the Little Old Lady method, which I'll show you in a second.
The computers and the memory were manufactured by Raytheon.
The flight software was written and tested at the Instrumentation Laboratory.
The core ropes required six weeks to build, and you couldn't make any last-minute changes.
And there's one of the women who's wiring this thing.
It was set up to be like a weaving machine that the computer would tell you where you're supposed to put the next wire.
In fact, it would set it up for you so that you could put the wire through the right core.
And all this was done at Raytheon.
And it must have been very tedious for these women to do that kind of work.
But they were always so proud when the astronauts would come and visit.
Oh, it gave them the feeling that we can't screw up.
We have to do this right, because we just met the men whose whole lives depend upon.
How many computers?
Well, we wound up with one for the Command Module and one for the Lunar Module.
And there would be no inflight repair.
There would be no built-in redundancy.
That is, if one of the circuits fails, that's it.
There would be no uplink from the ground.
And there were many possible single-point failures, but none of them ever happened.
The quality control people couldn't calculate the mean time between failures, because they never had any failure of the AGC at all.
Software, the early NASA manned flights.
The first one was Mercury flight.
There were 6 Mercury flights.
And there were 10 Gemini flights.
The first one was May 5.
We've already seen Shepard's.
And the last one of the Mercury group was May in 1963.
Then there were Gemini flights, where there were two astronauts in a capsule, rather than just one.
And there were 10 of those, from '65 to '66.
George Mueller was the Associate Administrator for Manned Spaceflight.
And he was a NASA person.
And he was very much concerned with the computer software.
In fact, he was in charge of the software for the Mariner 1 spacecraft to Venus, which was launched in 1962.
And it went off course due to a software error.
There was a missing minus sign in the code.
He had this minus sign framed and hanging in his office.
Do not take for granted.
That you've got to check everything to make sure that it's going to work.
The astronauts used to come and visit us.
And in the fall of 1966, we had six astronauts.
There were all sitting down there in the front row.
And we had them autograph these.
And then the people in the back were the ones that were working on the job.
There is me.
And there is Dave Hoag, and there's a whole bunch of other people all there to be photographed with the astronauts.
This is a very sad picture, because Virgil Grissom and Roger Chaffee and Ed White, who where in that picture, were in Apollo 1.
What happened to Apollo 1?
It burned up on the ground.
And so those guys, next to them was Dave Scott, who was one of my students, and Jim McDivitt and Russell Schweickart.
Russell Schweickart was an MIT student, but he didn't take my course.
And they flew on Apollo 9.
Apollo 9 was the first flight using the Lunar Module.
Apollo 8.
This was the round trip trajectory to the moon.
And you notice you're going from Earth to moon, go around the moon and back.
And of course, what you want to do is to go on an orbit which is going to arrive at the lunar orbit ahead of the moon, and then we'll go around and then be swept back to Earth.
Instead, if you had gone behind the moon, then the energy exchange would have been such that you would go off into the solar system someplace.
You had to do this just right in order to get on an orbit back to the Earth.
This picture is also in my book.
And you can look at it.
And they made a big deal out of the fact that it is a figure eight.
And Apollo 8 was the first manned flight away from the Earth.
And so Apollo 8, which was the Christmas flight, you may remember that, maybe you don't know.
You don't have to remember it.
But it was the time when the astronauts were for the first time in orbit around the moon.
And they were in this particular mission.
Of course, you always worry about, what if the engine doesn't work.
You're going to stay there forever.
You're not going to come back to the Earth.
There's nothing the ground could do to help you out.
And this design for the Apollo 8 is really a Earth around the moon, just as the real mission showed.
Here's the crew.
Frank Borman was the commander.
And Bill Landers was the Lunar Module pilot.
Even though we didn't have a Lunar Module, they had work to do.
And Jim Lovell was the Command Module pilot.
And he was the one that did the navigation.
He did all the measurements to do the orbit determination, the navigation, for the first time, on a flight to the moon.
And what he had to do, this is the navigator measuring an angle between a star and a landmark.
And you would put a mark in the computer, which would register the time and the direction of each of these, and then it would calculate the angle, the shaft angle and trunnion angle, or two angles.
And you'd get one piece of information.
But when you added all these different measurements together, separated in time and position, you were able to do a very credible job of navigating to the moon.
As a matter of fact, it was so close that it was decided that we didn't need an update from the ground.
In the rule book, it said, before you make any velocity change, you have to get your state vector information from the ground.
And Chris Kraft said, he couldn't see any difference.
The onboard computer and the ground were almost identical.
But they did have to follow the rules and make the correction.
And Jim Lovell navigated all the way back.
So he was the very first person to ever navigate a spacecraft by making telescopic measurements on board.
Jim Lovell, he trained at MIT using the Earth horizon simulator.
He was calibrated.
That is, he would tell you where he thought the horizon was, and it would be different than what the computer thought the horizon was.
And so eventually, at least he was consistent.
And so he had demonstrated that his consistency was such that we would calibrate him.
That is, when he looked at the horizon, and there was a little difference, that that particular offset would be Jim Lovell.
And other astronauts would have their own.
And it says here that Jim was the first to demonstrate that man could navigate in space without assistance from the ground.
He wrote a book called Lost Moon.
This was the Apollo 13 book.
And he was the Command Module navigator on Apollo 8.
There he is when he was younger.
And that's just kind of sad, because on Apollo 8 navigational measurements, he said, "After leaving Earth orbit, the astronauts took a few rapturous sightings of the receding planet, and then turn their spacecraft around to fly in a proper, nose-forward attitude."
That always, whenever I see that, it's sort of upsetting.
We went to all kinds of trouble.
And he was the navigator.
And when he wrote the book, he didn't mention anything about how he did this.
He just took some rapturous sightings of the receding planet.
Oh, well.
Here are the flights.
Apollo 1 was destroyed by the fire.
Then there was Apollo 7, which was to try it out in Earth orbit.
Apollo 8 went to the moon.
Apollo 9 was a flight for that the first time, the lunar module flew in Earth orbit.
Apollo 10 was a dress rehearsal to fly to the moon.
You do everything except land.
And then Apollo 11, you land.
Then there were a bunch of others, as you can see.
12, 13 was, it took a while before they were ready, over a year and a half, to go from Apollo 13 to 14.
And at the very end, they had Apollo 17.
But there were 18, 19, and 20.
The rockets were there.
The spacecraft were there.
They could have flown.
But I think that probably the reason was that with all these successes, why don't we quit when we're ahead?
And there's another reason why you might want to quit.
And that is numerology, again, more numerology.
1 plus 9 plus 6 plus 1 is 17.
And Alan Shepard's flight was on 5/5/61, which is 17.
So Apollo 17 was going to be the last flight to the moon, all set by the very beginning.
But not only that, 17 is an important number.
It's a Fermat number.
It's a Fermat prime.
Fermat numbers are 2 to the 2 to the, and so on up the line.
And Fermat had suggested that all such numbers would be prime.
And they're not.
In fact, the only numbers which are known to be Fermat primes are the ones at the bottom 6, or 0, 1, 2, 3, and 4 are the only known primes, and F2, which is 2 to the 2 to the 2 plus 1 is seventeen.
And so it is one of the only known primes generated by this formula.
So obviously, 17 had to be the last flight to the moon.
Then Apollo 11.
Here are the guys that flew Apollo 11.
There's Neil Armstrong.
And there's Buzz Aldrin.
These pictures were taken when they were young.
They don't look like that anymore.
And Mike Collins stayed in orbit while Neil and Buzz landed on the moon.
And there is a picture of, this is the arm patch for Apollo 11.
It's the Eagle.
The spacecraft, the Lunar Module was the Eagle.
And there it is, landing on the moon.
Now, in the aftermath of Apollo 11, this is Newton's tomb in Westminster Abbey.
And I recently went to Westminster Abbey to see, again, Newton's tomb.
And they wouldn't let you go up to it.
Before, you could walk up to it.
And you could pat the-- but now you couldn't.
And I was talking to one of the guides there.
And I told them what had happened in Apollo 11.
I said, there was a message left on Newton's tomb with flowers.
And it said, "The Eagle has landed."
And his eyes just lit up.
He said, "I was here when that happened.
This was my job.
And I saw the symbol, the flowers.
And I saw the note at that time."
He said, "It really did happen."
And there it is.
There is Newton's tomb.
It's a huge thing.
If you stand in front of it, you're no taller than the base.
And so that was the story about Westminster Abbey and "Sir Isaac, the Eagle has landed."
It was Howard Johnson who knew about this.
Howard Johnson was president of MIT not too long ago.
And a friend of his was killing time between flights.
And he went to Westminster Abbey, and he saw this note.
And he told Howard Johnson about it.
And Howard Johnson told me.
And it's very, very-- brings tears to your eye when you think that they were able to actually do this.
And I'm still pretty emotional.
"Sir Isaac, the Eagle has landed."
So thank you.
[APPLAUSE] If you have any questions, fire away.
If you don't have any questions, I had a good time.
I hope you did, too
First of all, on the numerology, was that all your personal touch?
Or was that an active discussion topic?
No, that was just me talking to nobody.
No, I was really struck by the factoring 1961 and getting the date out of that.
That just didn't seem like it should be, unless somebody was moving the hands, and we were able to get the job.
And the number 17, I was just fiddling around one day, and I said, oh, maybe that's why 17 was the last flight to the moon, because there's only one other prime Fermat number beyond 17.
And if you used that number, it'd be much too big for having anything to do with the number of the flights
I have a question about the orbital mechanics that you were looking at of that early period.
I was in high school several weeks ago, and somewhere around 1951 or 1952, in an advanced math class, we were doing celestial orbital mechanics.
And we were doing it from a small textbook imported from England.
And it was written by somebody who I didn't recognize as a mathematician that year, Arthur C. Clarke
Oh yes, of course.
He just passed away recently
Yeah, and now I've got to go home and find out if multiple body orbits were in it or not
No, he didn't-- you mean the one I was showing you?
No, he said he was not impressed at all by that.
I didn't send it to him.
Somebody else did.
And he said, well, that's not a problem.
Well, it's not a problem until you try to do it.
So he was just not interested
Work it on the blackboard for him.
Thank you
OK.
Yes?
Where were you when Neil Armstrong walked on the moon?
I was in Houston.
And I was in the center there.
And we were all wringing our hands.
The alarm that came off just before landing, none of us knew, except the one person who had to know, and that was the, what's his name.
I can't think of his name.
But he was the one who said, he knew what the alarms were.
Forget it.
They're OK. Because everyone else, if he hadn't spoken up, they were ready to scrub it, because they didn't know what happened.
And they would have just decided to play it safe and not land.
But it was not a problem.
Actually, what was happening was that the computer had a lot of jobs to do simultaneously.
It can only do one thing at a time.
But it would do a piece of a job, and then it would switch to another one.
And all the time, it had to be reading the accelerometer information and calculating what the x-acceleration was, lots of things to do.
And it had more to do than it had time to do it.
And the reason was that somebody, and Buzz Aldrin has admitted to it now, that he put on the ascent radar, just in case they were told to not land but leave, they'd be all set with their radar.
And by leaving that on, which the computer would do, if they were actually launching to the moon, it was just an additional task which was not supposed to be done at that time.
And so fortunately, the guy who was in charge knew that that was what the problem was, and he just said forget it.
Let's go.
It's one of those things that you don't want to be responsible for.
You don't want to have to be the one who says, go ahead, and then have it smashed into the surface of the moon.
You'd be very serious about what you would do with the rest of your life.
Any other questions?
Dr. Battin?
On your left.
I worked at the lab on the guidance system.
And I think the astronaut students, your old students, might be interested knowing, it wasn't just one program.
But the Lunar Module and the Command Module were very different, because of the ascent and landing.
And also, the lunar module was a very early digital autopilot.
It worked on a phase plane to decide when to fire its jets.
And for ascent, it didn't have a "gimbal-able" engine, so the whole scheme for controlling it was very different
Well, the interesting thing about the LEM computer was that the people who were responsible for the LEM said, "OK, well, we'll put a computer in.
We'll do our own."
And we kept saying, and NASA kept saying, why do we want to do that?
I mean, if the Command Module computer will functionally do the same things that the Lunar Module computer is supposed to do, then let's do it.
Let's just change the program.
Let's don't reinvent a new computer for that job.
And fortunately, the decision was that there would only be one computer.
We had two sets of software to design, but we didn't do it for two different computers.
Actually, the Lunar Module had, I forget what it's called.
But it was a little computer which could do basic things.
And if you couldn't do anything with the big computer, at least with that, it could run the engine, for example.
It may not calculate what the result's supposed to be.
You'd just give it to the astronaut.
And the simple jobs that you expected the computer to do, you would do.
And again, they never, ever had to do that.
They never had to use it.
But it was always there.
And they were always trained for it.
But it didn't ever have to be used.
Yes?
I had hear about Draper attempting to be an astronaut.
Whatever happened with that?
Did they just decide--
Draper what?
Wanting to be an astronaut.
Did they just determine--
Oh, well, I mean ha ha.
Actually, Draper would have gone if they had picked him.
The Instrumentation Laboratory existed before Apollo.
And he personally designed instruments for aircraft.
And then he would test-fly them in the plane, and he would be the pilot.
So he was a test pilot in that sense.
But the fact that he was 60 years old, that's unheard of.
I remember when I was 60 years old.
It was a while ago.
But no, I don't know whether he was actually serious about it.
He did write a letter to Bob Seamans saying that he would like to be considered.
And they could say, well, we considered you, but you couldn't do it, physically couldn't do it
Also, in those days, all of the astronauts were military pilots
And there weren't any women.
There weren't any women, because there weren't any women test pilots.
Had there been women test pilots, that'd be OK. Janice Voss was the first female astronaut from MIT.
And she helped me with my course.
She was really a very good student.
And when she graduated, she applied to be an astronaut.
And she was turned down once, twice, and she was turned down three times.
The fourth time, she was selected.
And so she says, don't ever give up.
If you want to do something, make sure that you don't give up.
It may not come right away, but it could come eventually.
So she was very excited about that.
And she was actually the first female astronaut from MIT.
We've had other-- well, you just saw some of the guys who went to the moon were all MIT students.
Incidentally, when Buzz Aldrin was a student, he did not have a label on saying I'm going to be flying to the moon.
He was a military pilot.
And he was interested in, I guess they had begun to talk about it.
He was interested in doing some of the rendezvous calculations.
But he was not representing NASA.
He was just there to get a Master's degree.
And all the other astronauts, none of them arrived and said, I am an astronaut, and I want to [UNINTELLIGIBLE].
They were all just students.
Mostly, they were from the military.
They might be from the Air Force Academy, that sort of thing.
But it was only afterwards that they got an assignment, actually being an astronaut.
Yes?
Can I ask your opinion on the Constellation program?
I imagine you think it's long overdue, but are they doing it the right way?
Do we still have the skills to go back to the moon?
I don't know.
I really don't know.
They're doing so many things which are just unbelievable, the Messenger program of getting a spacecraft to Mercury.
And they did that by at least a dozen or more fly-bys of Earth and/or Venus in order to get the energy lowered down to where you could actually contact Venus and maybe go around, orbit it, or take pictures of it, or what have you.
But I don't know any more about that today than anybody who reads the paper.
The Constellation is, I'm not sure.
Is this is a new program or what?
No, that's the return to the moon, and then to continue on to Mars
They got a name for it now, oh.
Well, I think it's certainly a great idea to do what we did.
And it would be wonderful to be able to do it again.
And I had a student, she was actually a freshman at the time.
And she was Japanese.
And I was a freshman advisor.
We were going around to ask, what would you like to do?
What would you like?
And it got to her.
She said, I would like to be the first woman on Mars.
And she was serious.
I want to be an astronaut.
The fact that I'm Japanese shouldn't have any problem.
I said, well, that's great.
I'll be there to cheer you on.
But the Japanese do seem to be getting more interested in space, as are the Chinese.
And the only thing that's going to get us really interested is when it looks like they're going to do it, and we're going to be left in the dust.
If as von Braun said, if you had tried to cooperate with the Russians, there wouldn't be a program.
What was I going to say?
I've forgotten what I was going to say.
But anyhow, there are so many opportunities that we hopefully will not ignore, just because there isn't any particular competition.
The whole business of Apollo, we learned how to make these computer chips.
And the chips were manufactured by Raytheon for Apollo, and the largest number of computer chips that were being made were the ones for Apollo.
There were very few applications.
And so you might say that the Apollo computer really was the original device that we have here.
And I think that that is true, that with an application, demand, you're going to learn how to do it better and cheaper.
OK. [APPLAUSE] Thank you for coming.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome to 16.90.
I'm Professor Qiqi Wang and this is Professor Wilcox.
So we are going to be together teaching 16.90 this year.
So 16.90 is Introduction to Computation Methods for Engineering Aerospace Engineering.
So what I'm going to do today is [INAUDIBLE] But before that let's go over about what this class is about.
[INAUDIBLE] OK.
So I'm first going to go over the syllabus.
And then professor Willcox is going to show you something new this semester that is very exciting for you to-- actual a great way to guide yourself to learn the material yourself.
Because this lecture is really-- this class is really taking advantage of the [INAUDIBLE] classroom structure, we really want you to learn the material before you come to class.
And in class we are going to be assuming you have read the material and we are going to be solving problems together.
So this is really not a new style.
This style has been practiced in this department for a while.
So lecture one.
We are now using computers to do a lot of things.
Every one of you are using computers checking email [INAUDIBLE].
The reason we have computers is because of things we are going to be talking about in this class.
So the first real computer is designed not for checking emails but for computational simulation to help engineers solve problems in ballistics.
And these are really aero problems.
So we are going to see something very similar to the world's first computer applications in this subject.
OK.
So let's fast forward in time and come to today.
Today's computer simulations are-- haven't become less important but increasingly important for studying, interpreting, and predicting many different processes.
And we are in aerospace engineering so I'm just going to be talking about aerospace examples.
But outside aerospace you can find even a lot of broad application for computational simulation.
So this is a class even though you not end up doing aerospace, it is going to be also [INAUDIBLE] --related to either science or engineering.
This is going to be useful.
So two examples are simulation of spacecraft reentry.
And this is a real thing.
[INAUDIBLE] Another example is designing [INAUDIBLE] this is the simulation.
And we're going to be talking about how to perform this kind of simulations.
And these are [?
PD's ?]
because you have multiple spacial dimensions where you have x, y, and z.
And if you are unsteady you also have a fourth dimension which is time.
So, to solve problems using numerical methods some problems like that.
It really involves two steps.
So this subject we are going to be spending a lot of money talking about how to solve differential equations.
But let's not forget the first step.
That is how you would actually get this differential equation.
OK, so the first really is to reduce a real system or process or [INAUDIBLE] into a mathematical model.
And in a lot of times you are going to find out the mathematical model is going to be a differential equation.
So to figure out what kind of mathematical model is suitable for the system or process you are studying you have to first know what are the quantities of interest.
[INAUDIBLE] What do you want to know?
OK. And what are the key processes involved?
What approximations can we reasonably make that is going to lead you to [INAUDIBLE] And the model we are going to derive [INAUDIBLE] from the differential equation-- [INAUDIBLE] it can be linear or non-linear.
Or you could look at the [?
method outcomes, ?]
distinguished between linear and non-linear equations-- [INAUDIBLE] It can be deterministic or probabilistic.
At the end of this chapter we are going to be looking at [INAUDIBLE].
It can be static or dynamic, involve time or [INAUDIBLE].
It can be discrete or continuous, either based on [INAUDIBLE] principle [INAUDIBLE] of math, --of momentum, [INAUDIBLE] --or empirical.
And once you have that you design a computation code and solve the mathematical model.
So let's take a look at an example of a computational model.
The model is a thermal problem.
So this is really-- this guy is a [INAUDIBLE] thermodynamics problem.
Problems like this [INAUDIBLE] where you have something spinning up.
Different parts are going to heat up at different rates and expand at different rates.
You want during the entire process of different parts expanding at different rates, nothing is going to heat another part.
Right?
You know on the spinning-- spinning [INAUDIBLE] or turbines which actually expands-- heats up faster than the outside of the turbine engine to actually keep the outside safe.
So you need to really predict well how fast different parts heat up during the process.
So our model problem is this.
This is aluminum.
It's a 3.89 cm cube, initially at room temperature.
It is going to be heated up by this [INAUDIBLE].
starting at time equals t0.
We're going to be turning off our air at time equals t1.
So here comes the quality [INAUDIBLE] the measured temperature.
So you see here it's a thermocouple basically stuck inside [INAUDIBLE] how this thing heats up.
No, what I want you to do is form a team of either two or three.
[INAUDIBLE] And answer this question.
I've already given you what is the quantity of interest, the temperature in the middle of the cube.
Now the question is what are the important [INAUDIBLE], and how to model the [INAUDIBLE] mathematically.
OK. And last, I'm actually going to show you because this is what you are going to be learning in this class.
How to use the mathematical model to make the prediction which means how to solve this equation [INAUDIBLE].
So, when we're here and start with these two questions, at the same time as you form a team I'm going to start [INAUDIBLE].
The last time we actually had more time for you to solve these so this time we are running out of time.
I know a lot of you haven't gotten to the bottom yet.
A lot of you I think given more time is going to derive the equation, especially some of you are looking up online and taking a lot of time to calculate.
So here I think I'm just going to wrap up by telling you how I would model.
There is no single way of modeling this.
But, my way is just a-- OK. All right.
So, what is the physical process that determines the quantity of interest which is the temperature of the cube?
I'm saying the temperature of the cube because aluminum is very good conductor of heat.
So If there is uneven heat in the temperature distribution inside the cube, it is going to even out very fast.
Especially for a cube of this small size.
All right?
So one can assume that the temperature distribution inside the aluminum is basically uniform throughout.
When you look at the time history it heats up slowly over the course of many minutes.
The temperature-- the conduction inside the aluminum is actually much faster than that.
So we can say that the temperature-- [INAUDIBLE] OK the temperature of the aluminum is only a function of time.
So this is the temperature of the cube.
OK. Now we have the temperature of the cube.
So what governs the temperature of the cube?
What makes the temperature of the cube change?
Heat convection.
OK.
So we have heat convection.
OK?
We have heat convection that makes the delta of the internal energy of the cube.
Right?
Equal to what?
So let's say that been two time points the change of the energy of the cube is equal to what-- what has contributed to the energy change?
Yes, what I'm looking for is the change in the energy is really equal to the heat that goes into the cube plus what's done to the cube.
Right?
And nobody did any work to the cube.
Right?
I didn't, I don't know if you did.
So it is only equal to the heat convection into the cube.
All right?
So this is how you start putting the physical process into equations.
Now let's look at how do we relate both sides of the equation to the temperature of the cube.
OK?
How do you relate the delta in the internal energy to the temperature of the cube?
So the change in the energy relates to the change in temperature.
Right?
How does it relate to the change in temperature?
Delta t times what?
C?
Cp?
Times the mass.
Right?
OK and the mass we-- although we don't know exactly what it is, we can look up for the density of the aluminum and the size of the cube.
Right?
So this is going to give us the changing energy.
And what is the-- what determines how much heat has convected it into the cube?
[INAUDIBLE] OK.
So I think I should write it like this.
What determines Q?
What determines the heat that goes through the surface of the cube into the cube?
So that is Q from the gun minus Q-- so this is gun to cube right?
So this is cube to air.
So this is gun to cube minus cube to air.
Now how do you model-- how does the rate of heat transfer from the gun to the cube depends on the temperature.
And how does the rate of-- how does the heat convect from the cube to the air?
[INAUDIBLE] Yeah, they linearly depend on the difference in the temperature.
So if you look back on unit five notes-- OK.
So this is-- these are heat convection right?
Heat convection.
And the way you blow air from the gun to the cube is forced convection.
OK.
So this is C-- the rate of the forced convection, it is proportional to the temperature of the hot air minus the temperature of the cube times the coefficient.
And the coefficient is the forced convection coefficient in the air times the area of the convection.
And for example, [INAUDIBLE] So maybe we can just say 2 times the square is the temperature of the heat.
And maybe the cube to air is C free convection times now four [INAUDIBLE] now is T minus T air.
All right?
And this is the rate of heat conversion.
Now we time the delta of time.
We are going to get the amount of heat that went into the cube during a small time unit.
Now this is going to give us a differential equation.
Because if we let the delta T, the amount of time we are looking at goes to zero, we are going to get to the equation of Cp times rho d cubed dT/dt.
So this is the rate of change in the temperature equal to C force times 2 d squared T H minus T minus C free, 4 d squared T minus T air.
right So this is the differential equation we get.
Now what changes when we turn off the heat gun?
C force is gone and the C free becomes [INAUDIBLE].
Right.
So that's the difference.
And let's go to Matlab, and I wrote a code for simulating this.
So here is the parameters.
Ambient temperature we said today is a little bit colder than I thought.
And the hot air temperature, if we look at this, I think is like 106 or something.
Let's put 106.
And the start time in seconds I actually don't know, so let's just put 0 here.
And the T stopped in seconds.
What time is it like 11 minutes?
What is 11 minutes in-- so this is 11 minutes right?
OK. tend I think I'll just do this.
OK.
So these are what I looked up online.
Some of the values for the free air convection and the forced air convection.
And the [INAUDIBLE] of the aluminum cube the density of aluminum, and the specific heat capacitance.
All right.
And these you don't need to look at them because they are never used in the code because radiation exactly is not important.
So heating and we are using [INAUDIBLE] so solve the equation.
I'm basically saying that the heating rate is proportional to two sides, and the cooling rate et.
cetera, and we are also doing the cooling.
So let's do the simulation.
We are getting a plot like this.
And let's compare our temperatur.txt-- [INAUDIBLE] Oh, OK.
I need to get rid of this.
Let me push the stop.
Not responding.
Error writing.
[INAUDIBLE] OK. Now we are plotting.
So look at the difference between the simulation and the experiment.
It's huge.
Right?
So what is causing this Difference It is because now here what I said is the force convection coefficient is 10 watt per meter square plus K. Right?
Now let's go to the internet and search for force convection coefficient of air.
Engineering toolbox.
Is it this one?
Yes.
Forced convection of air goes all the way from 10 to 200.
OK. And here I am actually putting the lower bound.
If I put something like 200 and do the simulation again, and do the experiment again, this is what I got.
OK.
I've got a much better match and I also have my free convection to be 5.
If you look over the internet free convection goes from 5 to 25.
OK. And also if I go to 25 I think I am also going to be getting a much better match.
OK.
So the rate of decaying is much better now.
And another thing I assume there are only two sides of the cube are being heated.
[INAUDIBLE] But actually if you switch over to the next problem you will see more than two sides are being heated.
So that is actually another motivation for why we need to look at not only one simulation but several simulations.
And that is leading into the fourth part of this subject, probabilistic simulations.
Because when you look at a lot of physical phenomena there is a lot of uncertainty in it when you look at the range of coefficients you can possibly get in modeling physics.
And convection is a primary example if this because there are so much uncertainty in what rate you can get.
All right.
So this is our first lecture.
I will see you next Monday.
But remember, one thing is you need to read and do the homework, do the embedded questions before you come to class next Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's get started.
So here, as I said, I assume that everybody [INAUDIBLE] has already done the reading.
OK, what I'm going to do today is not [INAUDIBLE] repeat any of the things you have already read.
But I also don't assume you already mastered all the material [INAUDIBLE].
So what I'm going to do is kind of [INAUDIBLE] to give you a view of more or less the same material, maybe a little bit more.
But from a slightly different angle.
And which means we are going to [INAUDIBLE] but like [INAUDIBLE] may look like this one.
The syntax may be different and may denote things with a different symbol.
But, like, any [INAUDIBLE] is the same and you should be able to read the style.
You should read what I'm going to be writing.
And in addition to that, I'm going to ask you to work out some problems for yourself or actually in teams based on what you read and what I'm going to be discussing today.
So I brought these things.
I'm going to be actually [INAUDIBLE].
And I also brought [INAUDIBLE] each of you are going to be working [INAUDIBLE].
OK, and [INAUDIBLE] say which is the most active [INAUDIBLE].
OK, to start this, I'm going to be-- wait, OK.
I'm going to be briefly reviewing what you read, which is how to basically write the function.
And then in addition to writing down formulas, I'm going to be demonstrating things in MATLAB, OK?
Discretize a function-- let's start with a function that, like, most of you very commonly see [INAUDIBLE] ODEs.
When you write down f-- let me change to a different color.
OK, if you write down F of T equal to E to the minus lambda T-- OK, can somebody tell me why this function is commonly ODEs?
It's a solution to essentially the simplest ODE you can find in the world-- df dt equal to what?
[INAUDIBLE]
Minus lambda times f of t, right?
So if I solve this ordinary differential equation, I'm going to get this function.
OK, if I draw the function, if I draw this function as a function of time, at t equal to 0, the function has a value of 1, right?
This is f of t, this is t. And how does the function look like for positive lambda?
It's going to decay exponentially.
That is what we call exponential decay to the minus lambda t, right?
OK, this function is a function.
It is a continuous function, which means it is, in some sense, infinite dimensional.
You need infinitely many numbers or infinitely much memory to memorize this function in a computer, right?
Because for every t, every single t, you have a function value f of p. But computers have finite memory, although we have a lot of memory, but still finite.
So we need to find a way to represent this function using a finite number of bits in a computer.
How do we do that?
OK, we do that by first selecting a range.
We cannot discretize the function for infinite t. We have to select a range and let's say from 0 to some big t. Is that enough?
No we also need to discretize this interval into small time steps.
And the first time step is usually just 0.
The second time step is delta t and the 2 delta t, 3 delta t, 4 delta t, et cetera.
And at t is the last time step.
All right, so this discretization, when I do this thing in MATLAB-- let me actually start up MATLAB.
And when I do it in MATLAB, it is usually how we actually draw the function in MATLAB, right?
I believe a lot of you have done this before in some other classes.
Here, I'm just trying to warm you up and get you, if you're not familiar, get you familiar again with how do we do things in computer.
I think I started MATLAB.
Right, so OK.
So let me set a big T equal to 5.
You see this?
Is it big enough?
Shall I change it to a little bit bigger?
Anybody want it bigger?
No?
Good?
OK. What [INAUDIBLE] do you want?
What one?
Good, OK. And what I want-- how I discretize the time is t equal 0 dt t, all right?
This is going to give me a t 1 by 51 double.
So if I click on it, it is going to show that the first one is 0, the second one is 0.1, et cetera, et cetera.
And the last one should be-- whoops-- is 5, all right?
And now the function value f of t is equal to any set of lambda of equal to 1, OK?
Lambda equal to 1.
And then my function value is equal to exponential of minus lambda times t. So that way, I computed discretized version of the function f. How would I show the function?
I can go to plot and to do this plot.
Like here, I'm just going to use the command line version to log t and f. Yeah, this gives us exactly the function we expect, right?
Exponentially decaying function.
And I would actually also prefer to plot this and look at what kind of discretization it is.
So if I plot this function with a dash and the o, it is going to display a circle at every discretized point so that we are going to see that this function is actually a discretized function.
It is not a continuous.
In MATLAB, the plot is approximated by drawing a linear line, drawing a line, between these circles.
But because delta t is small, 1.8, it looks like a small function, all right?
OK, let's try another one.
So here-- let's see.
How do I insert another?
Each option insert [INAUDIBLE] insert.
OK, so let's try another function.
We did df dt equal to minus lambda u.
How about minus lambda f?
How about minus ft squared?
Anybody know how to solve this ODE analytically?
Anybody who reviews ODE before this class where you are reading the readings?
The original variables?
Good, we are going to be dividing f square on each side, right?
So we have df over-- not on this-- f square equal to minus dt, right?
We're going to be integrating both sides.
And on this side, we get minus f 1 over f plus an arbitrary constant equal to minus t, right?
So we are going to get f of t is going to be equal to 1 over-- we have t-- here is actually plus t, right?
Agree?
OK, in particular, if I set the same initial condition, which is f of 0 equal to 1, then it determines my constant c and my f of t is equal to 1 over, what?
One over t, right?
OK, so let's Test this function.
And it is going to be useful later on, right?
So my f2 is going to be 1 over t. OK, I'm going to be-- let me hold on and plot t f2 [INAUDIBLE].
Oh, OK. That's one, right?
I'm going to have serious trouble if I do this.
So let me just do this again-- equal to 1 divided by t plus 1.
Yes, thank you for paying attention, OK?
And plot t f2.
Let me do dash dash and s. So, dash dash means the [INAUDIBLE] line and s means squares.
So we are going to see how different these are going to be.
So, one function is the solution of df dt equal to minus f. The other is df dt equal to minus f square, right?
OK, so here is discretization function.
But like, this is nothing new, right?
We all know this.
What's new is how do we approximate the derivative of these functions.
Because if we can approximate the derivative of these function, then we can start to solve these ODEs numerically, right?
These two ODEs I just showed you have exact solutions.
But the reason we take this class is because we want to solve ODEs that do not have analytical solutions.
If we have a way to approximate the derivative of any function we want, then I'm going to show you that we can solve any differential equation we want.
OK, so approximated derivative-- not from analytical function, but from a function discretized like what we just had.
Right, so pretend we have MATLAB.
We have this app.
We have f2 but we don't know where they're computed from.
They are just some kind of discretized function.
Now, how do we compute the derivatives?
OK, any ideas?
Any ideas?
Any ideas?
Any ideas?
Any ideas?
The linear definition of derivative-- what is the linear definition of derivative?
f2 minus f1 over f1-- There is a [INAUDIBLE].
OK, you did, like, partial t's.
Anybody wants to [INAUDIBLE] what's your name?
[INAUDIBLE] OK, anybody wants to complete this [INAUDIBLE] answer by maybe changing something?
[INAUDIBLE] is f of t plus delta t minus f of t divided by delta t, right?
And this is, of course, the definition of exact derivatives.
But in the computer, we cannot take delta t and go to zero, right?
That would again require infinite memory and infinite computation time.
We have to approximate it.
We have to be satisfied with delta t being small enough.
OK, so here, we have a function like this.
We have a function like this.
Each-- let's look at this one, the one with squares.
These two squares is based over delta t in the horizontal side.
On the vertical side, they are spaced by f2 minus f1, right?
So if we take the vertical distance, divide by the horizontal distance, that [INAUDIBLE] approximation of this, right?
f of t plus delta t minus of of t over delta t, all right?
So that's a good way of approximating the derivative.
Now, anybody who can transform this formula into MATLAB?
I have two functions, f and f2, in MATLAB.
How do you guys transform this formula, which is-- this is like one of the most important skills you are going to learn in this class-- that is, how to incorporate, how to translate, a mathematical formula you can write down on a piece of paper into a code like MATLAB.
[INAUDIBLE] see [INAUDIBLE] is that going to keep me a code I can type into MATLAB.
Like, here, I'm writing [INAUDIBLE] type.
Please tell me what to type.
Yeah, find the function-- the difference between functions between one point and other points.
Let me do a [INAUDIBLE].
Yes, the difference and the approximate derivative-- OK. OK, you guys are good.
So OK, so I'm going to do something new.
I'm going to use if.
I'm going to use if of f. What is that going to give me?
So let me just say df equal to diff f, OK?
And let me double click on df to show me what this is.
And I'm going to see, compare this with f-- f and df.
So what I can see is that df gives me this minus this, right?
The first element of df is the second element of f minus the first element-- sorry, this minus this.
Or what I can do is the same thing can be computed by f of 2 to end.
Do you know what this means?
Exactly-- it gives me all the angles from the second to the end minus f1, 2, and minus 1.
What will this give me?
The first to the second last-- so this gets me exactly the same answer, right?
And of course, I can just do this divided by et, all right?
And so this df dt.
This is an approximation of the derivative.
OK, and this is an approximation of the derivative at which point, if I speak to this formula and the discrete time instances instances, but like [INAUDIBLE].
So if look at length of df dt--
[INAUDIBLE]
Yes, this is an approximation to the derivative from 0 to t minus delta t, right?
Because here, I'm taking the difference between t plus delta t, which goes all the way to big t, minus ft, which actually doesn't go all the way to big t, right?
So I have the derivative from 0 all the way to big t minus delta t. I don't have the derivative of the last one because at the last one, I do not have f of t plus delta t. OK, now let me plot-- let me make another figure.
I'm going to plot t1 1 to n minus 1 because I don't have the derivative at the very end as df dt.
I'm going to plot them using circles.
And it's the derivative-- yeah, this is the derivative.
It goes all the way to close minus 1 to 0.
And you can see it is roughly an approximation of the slope of the circle line, right?
The slope is negative.
Therefore, the derivative is negative.
Let's compare this against the real derivative.
Now, what is the real derivative of this function?
The real derivative of this function if f of t is equal to minus-- e to the minus t, df dt is just equal to negative of e to the minus t, right?
OK, so let's take this bigger two and hold on.
And the plot t exponential of minus t negative-- let's plot it using black, [INAUDIBLE] black.
OK, we see we had a pretty good approximation of the real derivative.
We can also see that it is not exact, right?
So if the black line goes right through the center of the circle, then I have a very good approximation.
But, like, [INAUDIBLE].
All right, so we can see we do have some approximation error, which in this case, we call truncation error.
OK, so this, how big the truncation error is, is the real meat of this lecture, OK?
Truncation error.
And here, we start the Taylor series analysis.
So, Professor Wilcox last lecture said you're going to have so much fun with Taylor series.
So I hope you have already started looking at Taylor series and it is important you guys do look at Taylor series.
Because of the importance, I use red, OK?
Taylor series analysis-- OK, so we want to know how much approximation error do we have by approximating the limit with this finite delta t, all right?
We want to know what is the difference between df dt and our approximation.
So this is at some t and this approximation is t plus delta t minus f of t. OK, how do I figure out this difference?
How should I figure out this difference?
Then we can compute both because-- yeah, we can compute both.
And then we can prove a difference.
We can [INAUDIBLE] this line with this line [INAUDIBLE] lambda.
Like, this is probably 0.05 or something like that.
Yes, this is a translation error, right?
Because this only works when we do have an additional solution, right?
And so what-- is there any way to activate how big the trunctation error is in a more general way?
Taylor series
Taylor series-- OK, good.
Taylor series-- what is Taylor series?
Anybody can summarize what Taylor series is in, like, one sentence?
[INAUDIBLE]
Yeah, an approximation of a function at some point using the values and derivatives of this function at a nearby point, right?
OK, so here, what we-- we have three things.
If you look at the formula we have, three things.
[INAUDIBLE] The derivative at t, the functionality at t [INAUDIBLE] the functionality, like, we have two points.
One point is at t, the other points [INAUDIBLE].
At [INAUDIBLE] t, we have a derivative of [INAUDIBLE].
At another point, we [INAUDIBLE].
We can employ Taylor series in two ways.
One way is more convenient.
The other [INAUDIBLE] is a little bit more inconvenient but still works.
The most convenient way is [INAUDIBLE] f at t plus delta t using Taylor series.
The more inconvenient way is to send both derivatives and the value [INAUDIBLE] using Taylor series [INAUDIBLE].
So let's do the convenient way first.
And I'm going to show you and we're going to get something similar using the slightly more convenient way, all right?
So the easy way-- let me use green to denote the easy way.
OK, so easy-- f at t plus delta t. So, the start of Taylor series-- the Taylor series I write as a formula is k goes from 0 to-- let me actually-- equal to infinity of f to the k-th derivative at t divided by k factorial delta t to the k-th.
This is what Taylor series is.
And in this case, we do not want to keep anything above the first derivative.
So I'm just going to be writing that-- I'm just going to be keeping two terms.
The first term is k equal to 0.
What is f to the 0-th derivative?
It's just f itself, right?
This is k equal to 0.
What is 0 factorial?
One.
What is delta t to the 0-th?
one.
OK, so that is the first term.
Now let's do k equal to 1.
What is the first derivative of k?
It's df dt.
What is one factorial?
It's one again.
And delta t to the one-- that is delta t. Anything above that has a delta t to the at least second order, right?
right?
At least delta t squared.
square.
So I am going to write all these terms a a big O times big O delta t squared.
Has anybody seen this big O [INAUDIBLE]?
You did, OK. What this means is that this [INAUDIBLE] compares [INAUDIBLE] as long as all these terms have a delta t squared or delta t cube or delta t [INAUDIBLE].
[INAUDIBLE] more of the [INAUDIBLE] square.
OK, which is true for all the other terms [INAUDIBLE].
Or maybe green is a little bit too-- doesn't show really well on this.
So let me see if I can-- let me change to this.
OK, yeah, that's better.
All right, OK.
So, once we do that, what does it change?
[INAUDIBLE] action [INAUDIBLE].
Yeah, actually don't know [INAUDIBLE].
OK, once we do this, we can plot this back into the truncation error we call tau.
Tau is going to be df dt at t minus something divided by delta t. And that something is-- first of all, it's a Taylor expansion of ft plus delta t minus f of t, right?
And now it's time to cancel the terms.
So I'm going to write this as it is and, first of all, try to cancel the terms here.
So delta t is [INAUDIBLE] here.
This ft cancels with this ft, right?
So we have a df dt times delta t plus O delta t square, OK?
And this delta t cancels with this delta t, which makes this cancel with this.
So the only thing we get is, oh, delta t square divides by delta t. [INAUDIBLE] square, delta t square [INAUDIBLE] all the terms that has delta t square or higher.
Then [INAUDIBLE] divided by delta t. What do we get?
[INAUDIBLE]
Order delta t, exactly.
All right, so what we know is that the truncation error, whatever it is, is order delta t, which means it decreases f delta t. So [INAUDIBLE] to kind of-- to visualize what this means, if I go back to here, so this is [INAUDIBLE] 0.05, right?
[INAUDIBLE] 0.05.
This for delta t of 0.1.
If I increase delta t to 0.2-- let me exaggerate.
If I increase delta t t 1, what do you think this is going to get?
Yeah, much bigger of course-- how much bigger?
No, no, the area is O delta t. Let me say, if I can make this [INAUDIBLE] if I may [INAUDIBLE] Now the [INAUDIBLE] 0.05 [INAUDIBLE].
It's 1.
It is going to be o delta t, which means delta t increase by a factor of two.
This area increases by a factor of two.
Let's just try to do this to see if what I said is true.
0.2-- OK, I'm going to be repeating what I'm going to be doing.
f is equal to-- no, I need to do t plus t equal to this.
f is equal to this and the df dt is equal to this.
Right, and I'm going to do a hold on and the plot.
I'm going to be plotting t from 1 to n minus 1 df dt.
And this time, I make it red.
All right, so o means I still want to plot in circles.
r means red.
So Let's see what I get.
Is it clear?
The blue is the derivative I get [INAUDIBLE] delta t 0.1.
The red is the derivative [INAUDIBLE] delta t 0.2.
The error [INAUDIBLE] the difference between the circle and the black line.
The black line is the exact [INAUDIBLE] derivative, which is only available if we have [INAUDIBLE].
And where [INAUDIBLE] the difference between the circles [INAUDIBLE] derivatives and the black line.
This is real derivative.
It grew by a factor of two.
OK, so this is using the easy way, Taylor series analysis.
I'm also going to show you briefly the hard way, which is kind of pointless here.
But as you are going to see later on when we get to more complex schemes, we have to start using the more difficult way of doing Taylor series.
I'm going to rewrite what is the [INAUDIBLE] error we're interested in here-- f of t plus delta t minus f of t divided by delta t, right?
This is at t. OK, Taylor series-- we're going to do it again, but we're going to represent f of t as a Taylor series of k goes from 0 to infinity f to the k-th t plus delta t divided by k factorial.
Here should be, what?
So I'm extending f of t with a Taylor series that is based on t plus delta t. But there is something to the k power.
And what is this something?
OK, I'm just going to write a box here.
f of y is equal to summation of k equal to 0.
Taylor series k is power fx k factorial what to the k-th power.
Let me explain to you y at x. Y minus x-- exactly, right?
Previously, we have x being t, y being t plus delta t, OK?
So y minus x is delta t. How about in this case?
Negative delta t, exactly.
OK, does this make sense?
All right, yeah.
So this is going to be very useful, like for making sure you know every different view, every different manipulation of the Taylor series.
It's going to be useful.
I'm going to show you another manipulation of the Taylor series.
That is, if I take a derivative to both sides of the Taylor series, take derivative with respect to y, what I'm going to get-- if I take derivatives on the left hand side, what I'm going to get?
df d-- yeah, let me just say, yeah, df dy is equal to summation k equal to 0 to infinity.
What is this term, taking derivative y?
It's a constant.
There is no derivative.
So this term doesn't depend on y.
Only this can depends on y, right?
So what I'm going to get is k, after k-th power [INAUDIBLE], write x divided by k factorial.
I'm going to have y minus x to the k minus y power times another k here.
So this is what happens when you type in [INAUDIBLE] derivative as a Taylor series.
And now, if you expand this term at t plus delta t, you can write the same conclusion as we previously arrived, which is our scheme has a truncation error of order delta t. All right, let's take a mini break of like five minutes and then come back to our lecture.
All right, so [INAUDIBLE] to you a small entry into what we are going to be doing with Taylor series.
It's not going to be obvious in the beginning, but like, this is going to be getting better and better method to exercise.
OK, [INAUDIBLE].
OK, let's get back to [INAUDIBLE] again.
I'm sure I lost all of you when I talk about Taylor series of t [INAUDIBLE] plus delta t. But let's not get [INAUDIBLE] onto this.
It just requires you to go back and look at Taylor series with a little bit more familiarity.
OK, so here are-- I'm going to say a little bit on why we want to approximate the derivative and how is that going to give us a way of solve [INAUDIBLE].
OK, so remember, we are approximating the derivative df dt at time equal to t at f of t plus delta t minus f of t, right, over delta t. And that's not an idea I proposed.
It's proposed by-- sorry, what was your name again?
Ariya
Ariya, OK. OK, so yeah, if we stick to this scheme, we can actually start to integrate ODEs.
OK, let's call this 0 equal to t0.
So that's actually how each time step has a name-- so, t1 equal to delta t, p2 equal to 2 delta t, et cetera.
And the function values-- f of t at t equal to 0.
I'm going to call it f0.
The function value at f1, I'm going to call it t1 equal to f1.
The function value at t2, I'm going to call it f2.
Now let's say I have an ODE and I only have the solution at t0, t1, t2.
I want to know what is the solution at the next time stamp.
What is the value of f3?
So that is what happens when we solve ODEs, right?
We start with the initial condition.
We start with f0 that is given.
The test case will compute what f1 si.
And now, once I computed f1, the task is to compute what f2 is.
Once I compute f2, now the task is to compute what f3.
How do I compute f3?
Yeah, how do I compute f3 using Taylor data points.
So here's what is given.
I have the ODE.
I have df df equal to a function of f. Let's say-- I'm going to say minus lambda times f. So here's my single ODE I need to integrate.
And I can approximate-- I have an approximation to the derivative.
And I also have f0, f1, f2.
How do I compute f3?
So this is given.
This is given.
This is given.
This is given.
This is given.
How do I put them together to compute [INAUDIBLE]?
[INAUDIBLE]
Right, we have our function value at f2.
We also have an estimate of the derivative at f2, which actually involved the unknown.
So let me write down-- let me write down-- this is important.
Let me write down what is the derivative estimate at f2-- at t2, sorry.
It is equal to f of t plus delta t, which is t3, right?
This is t3 minus f at t2 divided by delta t. This actually involves an unknown in the derivative.
And this is df dt by the differential equation is equal to minus lambda times f at, what?
Is t2.
Now, through this, by plotting both the approximation of the derivative and-- by basically putting both the approximation of the derivative and the function together, we converted the differential equation into a what equation?
[INAUDIBLE]
Yeah, into a difference equation, into a [INAUDIBLE] equation.
We can just compute.
So let me color this.
This is unknown, this is known, and this is known.
We can just move all the unknowns to one side.
So f of t3 is my f3 is equal to all the known on the other side.
f of t2 is f2 minus delta t times lambda times f of t to [INAUDIBLE] 2.
Right, by doing this, I compute F3 out of f2.
Now, what do I do when I-- now I have f3.
What do I do when I want to compute f4?
Same thing, right?
It's kind of recursive because my f4 is now equal to f3 minus delta t lambda f3, right?
And I just could go on.
The only thing I need is to plug in the derivative approximation here into the differential equation.
Let me do it again using another different scheme.
[INAUDIBLE] me do this [INAUDIBLE].
OK, different scheme.
OK, another way to approximate the function-- I mean, I have been calling the function f, but like, because we started by discretizing a function.
But, like, in [INAUDIBLE] ODEs, it is more convention to call the solution u.
So from now on, I'm just going to call my function as u of t instead of f of t. This is just a-- but, like, everything else is the same.
[INAUDIBLE] with this, we'll just call it u of t. OK, u of t, I have du dt equal to minus lambda times u of t. I'm going to devise another scheme of approximating the derivative.
I have du dt at a certain time t equal to u at t plus delta t minus u at t minus delta t divided by 2 delta t. OK, so let's do two things.
One is, why is this a [INAUDIBLE] approximation to the derivative?
Two-- now, if this is a valid approximation of the derivative, how do I make this a [INAUDIBLE] of [INAUDIBLE] ODE?
So, anybody have an idea to how to answer either of these [INAUDIBLE]?
Yes?
[INAUDIBLE]
It's the definition of the derivative, but the definition of the derivative is also [INAUDIBLE], yes?
[INAUDIBLE]
Good.
After the [INAUDIBLE] problems, it can be solved using Taylor series [INAUDIBLE].
OK, it's good.
So, answer to the first question-- y is a good approximation.
OK, to answer that, again we use Taylor series.
u at t plus delta t-- as usual, we are going to be expanding it using u and t plus du dt at t times delta t plus o delta t square, right?
This is the same thing as we did of the other scheme.
The other scheme is [INAUDIBLE], right?
Remember your reading.
And ut minus delta t-- how do I expand that?
Yes
[INAUDIBLE] Good.
It's the same thing except for the distance between this t minus delta t and t is minus delta t, right?
So in here, the distance between this variable and this variable is delta t. Here, the distance between this variable and this variable is minus delta t. Therefore, all the values and derivative is the same except for here, I put minus delta t. And o, it should be minus delta t to the q.
But does it matter?
No, in the big O notation, the constant coefficients, [INAUDIBLE] the terms doesn't matter, right?
So minus O delta t square is the same as o delta t square, is the same as five times o delta t square.
It's the same as 1 million times over delta t square.
As long as the terms have delta t square in front of it, no matter if it's multiplied by 1 million or 1 million or 1 trillion, it doesn't matter.
It's o delta t squared.
If it's multiplied by minus 1, it's still o delta t squared, right?
OK, so now, when I plug in both approximations into this formula, t plus delta t minus u t minus delta t over 2 delta t, is equal to-- these two terms cancel out.
These two terms-- they actually have different signs.
So they become [INAUDIBLE] 2 times delta t. Du dt plus-- I'm just going to write this down.
What is this plus?
What is the difference between these 2 o delta t's?
Do they cancel out?
No?
[INAUDIBLE]
It's still o delta t square.
Why?
[INAUDIBLE]
Exactly, because they don't know what the coefficients are.
Maybe one of these [INAUDIBLE] has 1 million in front of it.
The other o del t has a 0.001 in front of it.
So no matter the summation of them or the difference between them, they are still o delta t. So this is going to be-- these two cancel out.
du dt, this is at t, plus-- again, this o delta t squared divided by delta t, it becomes o delta t, right?
OK, so this is still a good approximation, because the difference between them is o delta t. And it is actually stronger than o delta t because in these two Taylor series expansions, you can expend more terms.
And you're going to find out that coefficients before [INAUDIBLE] these o delta t squares before the delta t squares still are going to cancel out.
So you're going to get o delta t cubed out of these Taylor series [INAUDIBLE].
And over here, you can actually prove that-- it's in the reading.
You can prove that this is not only o delta t but also o delta t square.
It's a confusing [INAUDIBLE] conflict itself.
We're saying that this is both o delta t and and o delta t square.
Do I contradict myself?
No?
Hm?
[INAUDIBLE]
Yeah, how does it go?
How can this term be both o delta t and o delta t squared?
Again [INAUDIBLE] I want-- somebody else, yeah
Because o delta t is the [INAUDIBLE] delta t squared and [INAUDIBLE]
Yeah, because o delta t actually includes o delta t squared.
o delta t means it can contain delta t terms, delta t squared terms, and delta t [INAUDIBLE].
But any or all of these terms can have zero as their coefficient.
If I have an o delta t term and it happens that the coefficient before the delta t term is zero, then it is automatically going to be o delta t squared, right?
So I can say that o 1 is a superset of o delta t, which is a superset of o delta t square, which is a superset of o delta t cube, et cetera.
Any questions?
Questions?
All right.
OK, and so now, the second question is how to make a scheme out of it.
We already have an approximation.
That is du dt at t is equal to u of t plus delta t minus u of t minus delta t divided by t delta t. We have a differential equation-- du dt equal to minus lambda u.
Again, the only thing you need to do is to plug this into this.
So we have u of t plus delta t minus u of t minus delta t divided by 2 delta t is equal to minus lambda u at t. And when you look at these terms, when you are at time t, when the solution at time t is already known but anything beyond t is unknown, then this is known because this is the before t. This is known because this at t. The only unknown is this.
So again, we are going to rearrange the terms and say u t plus delta t is equal to u t minus delta t minus 2 delta t times lambda times ut.
If t is going to be exactly on a time step, what I'm going to say is that u of i plus 1 is equal to ui minus 1 minus 2 delta t lambda times ui.
OK, this notation is assuming I have discretized u into a uniform grid of space in delta t
[INAUDIBLE]
Yeah
[INAUDIBLE]
The question is, am I using a scheme that has a mathematical definition?
[INAUDIBLE]
Oh, OK.
So what does a scheme mean?
A scheme is a numerical method, a method that you can implement into the computer to solve a differential equation.
It just means if I have an initial condition, a scheme must be able to tell me where is the solution at the next time step and then tell me again, what is the solution at the next time step, et cetera, et cetera, right?
A scheme is basically, you can think of it as an algorithm
[INAUDIBLE]
Right
[INAUDIBLE]
We're making what?
[INAUDIBLE]
Exactly, exactly, yeah.
Good question.
I was just so familiar with what scheme is, I didn't explain it.
So it's a good catch.
If you find something like this, please to raise a question.
Because if you have a question, your classmates probably also have the same question.
OK, so here, we basically derived two schemes.
By the way, this scheme is the midpoint scheme.
We have tow rite two schemes-- the [INAUDIBLE] scheme, which is the scheme that we have-- yeah, the scheme we have here.
We can run it in general as f of i plus 1 equal to f of i minus delta t lambda f i.
So this is the Forward Euler.
This is called a forward Euler scheme in your reading.
And we have also derived the midpoint scheme both from-- they are different only by that they're different in how to approximate the derivative in time.
Once you have an approximation of the derivative in time, you plug into the differential equation.
You can [INAUDIBLE], OK?
Now what I want to talk about next is what is the local order of accuracy.
And I'm going to say that the local order of accuracy is how accurate the scheme approximates the time derivative.
It is related to what tau is.
So for example, in forward Euler, how is o delta t, right?
This is what we derived right before we take the break.
So tau is-- tau in forward Euler is the df dt at t minus-- let me call it u because it's a small-- it's more consistent to the notes-- minus u at t plus delta t minus ut divided by delta t. This is o delta t for forward Euler.
And the tau is the same derivative minus a different approximation for the derivative, ut minus delta t divided by 2 delta t. This is actually o delta t square for the midpoint and through Taylor series analysis.
OK, the local order of accuracy is the exponent on top of delta t of this truncation error.
So forward Euler is first order accurate because it's o delta t. Midpoint is second order accurate because it's o delta t square.
So the local order of accuracy is 1 for forward Euler.
The local one of accuracy is 2 for midpoint.
Usually, the higher the order is, the better scheme is.
Why?
Because you can make the scheme more accurate by only increasing or decreasing delta t a little bit.
We just need an analysis of forward Euler.
If I make delta t half as small, how much smaller the truncation error is going to be?
Half.
Now, if I have a midpoint, if I decrease the-- if I decrease the delta t by half, how much more accurate is the midpoint going to be?
A quarter-- exactly.
So-- yeah, so this is how we usually kind of visualize the order of accuracy.
OK, so let's say the [INAUDIBLE] is delta t. OK, so let's say this is forward Euler.
So let's say this is first order accurate.
This is delta t and delta t, let's say, is 1 here, 1 to 1 here, 1 to o1 here, 1 to oo1 here.
So let's say when you have 1 I get a pretty big error [INAUDIBLE].
When I decrease delta t to 0.1, what do you think the error is going to become?
[INAUDIBLE], right.
So I should be right here, right?
I should be right here because it's o delta t. Now, if I decrease it to 0.01, [INAUDIBLE] should decrease by another 10 and by another 10.
So if I link it, it will be a line with a slope of 1 in a [INAUDIBLE] All right?
Now-- so this is forward Euler.
This is first order accurate.
How about a second order accurate scheme?
So again, [INAUDIBLE] delta t is 1, 0.1, 0.01, 0.001 here.
And again, if I am [INAUDIBLE] large error at delta t equal to 1, what do you think is the error going to be at delta t equal to 0.1?
[INAUDIBLE]
[INAUDIBLE]
Why should it be here?
It's delta t square [INAUDIBLE] smaller.
Delta t square is 100 times smaller.
So I should be here when delta t decreased by a factor of 10.
I should be here when delta t is decreased by another factor of 10.
If I link it, it will be a slope like this and I'm kind of out of touch when I'm 0.001.
So you can see that second order scheme is something much better than the first order scheme.
Essentially, I have the luxury of making delta t very small-- OK, so going back to MATLAB, if I can say df dt equal to-- now I cannot no longer do this, right?
I have to do f of 3 to n [INAUDIBLE] 3 to n OK, let me clear the workspace so that you can see better.
[INAUDIBLE] window [INAUDIBLE].
OK, so df dt-- let me say midpoint is equal to f of 3 to n minus f of 1 to n minus 2.
What does this mean?
I'm picking from the third value to the last value-- minus the first value to the third last value
[INAUDIBLE]
I'm taking the different space by two.
OK, so that is like-- there is how I view f at t plus delta t minus f of t minus delta t. So I should divide this by how much?
2 times delta t-- exactly.
And here, I think I'm actually doing a really big delta t. It doesn't matter.
[INAUDIBLE] OK, so when I plot it I should be plotting it against the t going from 2 to n minus 1, right?
Because this is now my t. 2 to n minus 1 is my t. 3 to n is my t plus delta t. 1 to n minus 2 is t minus delta t. So this is my t. I'm going to [INAUDIBLE] this against df dt midpoint.
I'm going to be square and what color do you want?
Pink
Pink?
Magenta?
OK, [INAUDIBLE] square,
[INAUDIBLE]
Yes, so [INAUDIBLE] even though [INAUDIBLE] delta t [INAUDIBLE] 0.2 [INAUDIBLE] kind of goes right through the exact derivative.
OK, so this is how a second order scheme is better than a first order scheme.
And how to assess that?
Taylor analysis, right?
OK, maybe I will just show you how did I get the certain order.
So if I further extend this, [INAUDIBLE] what I get is the d square u dt times delta t squared over 2, right?
[INAUDIBLE] squared over 2 and then plus the cube.
Here, what I get is plus d square u dt times minus delta t square, which is the same as delta t square, over two plus this to the cube.
And you can see when I subtract the u of t plus delta t by u [INAUDIBLE] delta t, the [INAUDIBLE] order term cancel.
The first order term, because they are the same, so they add together.
And the second order term also cancel.
The third order term again is going to have different signs and subtracting them actually makes them bigger.
So it is order two.
So here, I can modify this to delta t cube because the square term actually cancel each other.
And here is going to be square.
All right, and we see that by actually observing the increased accuracy.
OK, so let's-- we have a little bit of time.
Let's actually start the contest.
And I would actually not end it today.
I would have you start it today and maybe we'll come back and report your findings in the beginning of the next lecture.
All right, so the concept is like this.
I want the best X scheme.
How do I [INAUDIBLE] my best?
The highest local order of accuracy-- OK, what does that mean?
I want to o delta t to as high power as possible, right?
OK, and the X-- what does X mean?
X means one of these-- one of these four.
And I'd like you to form a eteam with one person or two other persons and commit to either one, two, or three or four and figure out how do you design a scheme to make it as accurate as possible
[INAUDIBLE]
OK, so I'm going to give you the formula now.
Best implicit one step scheme means I want u at [INAUDIBLE] so u as i plus 1.
Or let me actually use u [INAUDIBLE] t plus delta t equal to something times du dt at t plus delta t plus something times u at t plus something times du dt at t. All right, this is the best implicit one step scheme and your task is to figure out what teh questions are.
OK, best explicit two step scheme-- it means this.
It means you add t plus delta t has to equal to-- because it is explicit, it does not allow a derivative [INAUDIBLE] order t. So you have to do a question mark of ut plus a question mark at du dt at t. And now, because it's two step, two step means I allow one more step to be used in determining the solution t plus delta t, which means I can use u at t minus delta t and du dt at t minus delta t. All right, so this is the degrees of freedom you have in your best explicit two step scheme.
And what is the difference between explicit two step scheme and an implicit two step scheme?
Yes, the answer is you can use du dt at t plus delta t. So I'm going to circle the term that makes a scheme implicit-- right, so implicit.
So with the implicit scheme, you are allowed to use u plus delta t equal to-- the increased term is a question mark times du dt at t plus delta t. If I don't have this term, it will be explicit.
All the other terms are the same-- question mark times ut times question mark times du dt at t plus question mark u at t minus delta t plus question mark times du dt at t minus delta t. So let's just choose between the three-- the first three options.
OK, and time is over, but I'd like you to find a team mate.
Commit to one of these axes and come up with an answer.
And we'll star to discuss this in the next lecture.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at [?
ocw.mit.edu.
?]
All right.
So last time I did this using the video recording, there was some complaints about the audio quality, so I'm wearing this today.
It feels a little bit weird, but I'll see how much better the audio quality came through.
And you can ask your questions [INAUDIBLE].
All right.
Can you pass this on, like everybody have one piece?
So the purpose of this is for you to just write down anything that is not clear onto this piece of paper, and hand it to me after the lecture.
What I'm going to do is, I'm going to draft these questions in the next section so that you're not going to stay confused on what might confuse you.
So this important [?
Monte ?]
cuts.
You can write anything that is [?
Monte, ?]
and I'm going to be lacking them and try to earmark your [?
Monte ?]
points.
So we are continuing our discussion integration of ordinary differential equations.
Let's just do a brief review of what we did in the last lecture.
So we are solving differential equations like du over dt equal to f of u.
All right.
We discretized the time into t0 equal to 0, t1 equal to delta t, t2 equal to 2 delta t, et cetera.
We discretized the solution, u, into u0 equally to u at t0, et cetera and ui denoted as the solution, u at ti.
Forward Euler, solves the equation like this.
It is basically taking-- I'm going to change the color.
It is essentially taking this differential equation and approximate the time derivative into ui plus 1 minus ui divided by delta t. And on the right-hand side, it is the same thing.
So this is forward Euler.
A midpoint rule, we are going to be doing something very similar, but instead, I'm going to discretize the time derivative term in a different way.
I'm going to discretize the time derivative term into ui plus 1 minus ui minus 1 divided by what?
[INAUDIBLE]
2 delta t. Exactly.
So this is a consistent approximation of the time derivative.
So that's the only difference between forward Euler and midpoint rule.
And how to analyze the accuracy of these two methods?
Taylor series expansion.
In midpoint rule, we are expanding ui plus 1 equal to ui plus-- you should become very familiar with this as you learn more.
So ui prime, which is a shorthand for denoting du dt at ti time delta t plus half of ui double prime, which means du d squared u, dt squared, the second derivative of u times dt squared plus all the other terms.
The third term is ui triple prime delta t cubed plus et cetera.
Is a little bit too wide, maybe
[INAUDIBLE]
You can't see.
Let me change this to a little bit darker green.
And ui minus 1 is equal to-- let me scroll this up a little bit-- is ui minus ui prime delta t. Why minus?
I'm going backwards.
I should test delta t to minus delta t, right?
This is why there is a minus here.
And this term should have also a minus here because minus delta t is squared.
So I have, again, altered it here.
And this term?
Should I have a minus here?
Yes.
Because when I cube it, the negative sign is preserved [INAUDIBLE].
So when I take the subtraction between ui plus 1 and ui minus 1, this got canceled this gets doubled, this gets canceled, and this gets doubled.
So what I ended up having is ui prime plus 1/6 of ui triple prime delta t squared.
I mean, this cube cancels with this delta t and delta t squared equal to f of ui.
And this is n plus 0 delta t cubed, so this is the leading term of the truncation error.
Remember my origin original differential equation.
My origin or differential equation is du dt equals f of u.
The midpoint rule gives me du dt plus something times delta t squared and plus an even smaller terms equals f of u.
So these additional term that is append from the discretization is the truncation error of the scheme, is the error made by the scheme, is the approximation.
How much accuracy do I lose by approximating the derivative using discretization scheme?
All right.
This is the truncation error.
And if you do forward Euler, this truncation error, would it be delta t squared?
No.
It will be on the order of delta t. So forward Euler is less accurate than midpoint.
All right.
What we are talking about is this is called local truncation error.
The reason why we call it local truncation error is going to be clear in the next section because there is a different way of assessing the error is the global error, and we're going to be talking about that next one.
Yes
[INAUDIBLE]
Good question.
So the question here is, why did I start expanding here.
What if I happened to expand all the way here?
I can write a hundred terms after this.
There is infinitely many terms in the [INAUDIBLE] I stopped over here because I did this, and I know that this term is not attainable.
If a term is not unattainable then this is going to be the leading term in your local truncation error.
The way you do your own is if I do an analysis of a scheme I've never seen before, I would expand it somewhere maybe onto here, and then dug it in to see if I get something that is not canceled.
What happens if I see everything cancels?
What if I see everything cancels, and all I have is exactly the [?
C.
?]
Why I might have an [?
o ?]
delta dt?
[INAUDIBLE]
I should expand more.
Exactly.
That means that delta t2 may be the leading term of the truncation error, but would it be the leading term of truncation error if it is given less than delta t2?
I don't know.
I only know that the scheme is at least third order accurate [INAUDIBLE].
Only when I expand it more terms, I can know it happened in what order of accuracy this means.
All right.
So when you [?
file ?]
a scheme, when you're not sure, expand it to whatever you like and see if everything cancels.
If everything does cancel, then it means you need to go back and expand more terms.
Is it clear?
In the last lecture, I asked you to do this to find out who the best X schemes.
Can I have a show of hand of who did this?
Who did this?
Who attempted?
Good.
Who attempted the first one?
Let me see, 1, 2, 3, 4, 5.
I don't need you to be successful, just tell me what is the best you got
The first order [INAUDIBLE]
No.
What is the scheme?
So I want the product is, what is the--
[INAUDIBLE] coefficients?
Yes
I just got r1
Did you get--
I got u of t sub delta t
Good.
You get u of [INAUDIBLE]?
Yeah
--t plus delta t
--is equal to u prime
--is equal to u prime at--
--t plus delta t
--t plus delta t
--plus u of t
--plus u of t
--plus u prime of t
--plus u prime of t. Anybody get anything different, or everybody agrees with that?
Five people got the same thing?
[INAUDIBLE]
OK. Let me write this in a more concise manner.
t plus delta t is [?
as ?]
in f times x. t is at the current time set.
So let me denote everything, t plus [INAUDIBLE] i plus 1, i plus 1, and this is i and i.
So it's just clearer not having all the tn, t plus delta t. If I know-- come on.
Come on.
And you are sure there is no delta t in front of this?
I mean, forward Euler, and at midpoint, I have delta t's.
Yeah?
[INAUDIBLE]
OK. All the derivative terms should have delta t
[INAUDIBLE]
You don't have that?
[INAUDIBLE]
I was listing delta t's.
Sorry for that, but if I do the theta series and I just [?
carefully ?
], correctly, I should test that.
And if you got something different, let me write it here.
You've got something ui plus 1 equal to 1/2 of ui plus 1 prime delta t plus ui plus, you also get 1/2 here.
Yes?
You got something different?
[INAUDIBLE]
u prime of delta t. You get negative 1/2 here.
All right.
So we get three answers.
Is that all?
Anybody get anything different?
Good.
We have five people got three answers, so at least there are some agreements.
Who did the second one, best explicit, two-step scheme?
Who attempted the second one?
You?
So we have 1, 2, 3, 4, 5, 6, 7.
More people did this.
What is the best scheme you guys have got?
[INAUDIBLE]
The question is, can she-- what's your name?
[INAUDIBLE]
OK. Tren's question is, can we express the derivatives as f as well?
The answer is yes.
So whenever I see a derivative, I can plug in the derivative into the differential equation.
The differential equation is u prime equal to fu.
So whenever I have f prime of i plus 1, I can replace this as f of ui plus 1.
Whenever I have f prime of i, I can replace this as f of ui.
So what is your answer of the best explicit two-step scheme?
[INAUDIBLE]
Negative 4ui?
[INAUDIBLE]
f ui and--
--plus 2 f times u [?
sub ?]
i minus 1
OK. Good.
Anybody who get anything different for best explicit two-step scheme?
You all got the same thing?
[INAUDIBLE]
Delta t's.
And I guess delta t here.
You must have that too.
All right.
Oh, wow.
That's amazing.
Everybody got this?
No disagreements?
So the explicit two-step people are more in agreement than implicit one-step people.
Who did best implicit two-step scheme?
Who attempted?
You also did that?
OK, great.
And what is the answer?
[INAUDIBLE]
OK, you
[INAUDIBLE]
I'm going to append a delta to for you.
Plus--
[INAUDIBLE]
Wonderful.
So can you tell me how you did that?
I did a bunch of Taylor [INAUDIBLE]
Nice.
I get to have you show me how you did that later on.
Who the best explicit three-step scheme?
Anybody try that?
You also tried that?
OK, great.
I'm going to have you as an instructor here.
What is the answer for that?
[INAUDIBLE]
Minus what?
Sorry?
[INAUDIBLE]
18 and sorry, what?
9
9
9, 18, 10, and 3
18, 10, and 3.
So what is multiplied on this one?
[INAUDIBLE]
ui minus 2 and ui minus 2 prime.
So all the primes have delta t, I guess.
Great.
So I'm going to do some analysis of what this is.
I'm going to go back to this.
So I saved the documents.
Even if I lose it, I'm going to go back to this.
But what I want to show right now is first of all, I'm sure some of you did this, so you know already how to do this.
But I also want to see especially resolve the disagreement on the first one.
So that is what I'm going to do first.
What is the best implicit, one-step scheme?
I'm going to be having a demonstration of that right now, and then I'm going to show you how to implement a scheme once you derive it.
You derived a lot of schemes, now how do I put this scheme into a code, into a production, into something actually wrong.
So first of all the best implicit one-step scheme.
All right.
I'm going to append something after this.
The best one-step scheme.
So the scheme has to be ui plus 1 equal to something times ui.
All right.
Did I say implicit, one-step scheme.
So one-step scheme means everything has to be written in terms of i plus 1 and i.
And it can depend on ui, ui prime, and ui plus 1 prime.
This is because it can be implicit.
If it's explicit, then it has to always depend on i.
All right.
So let me put in three unknowns, x plus y plus z.
And these are unknowns.
And what do I do to make the scheme as accurate as possible?
Minimize the error?
Minimize the error.
Exactly.
And the error can only be [?
appends ?]
from Taylor series expansion All right.
So Taylor series expansion, again, there are two options.
One option is to expand all the i plus 1's at i.
So for example, ui plus 1 is equal to ui plus ui prime delta t plus 1/2 of ui double prime delta t square plus-- I can do more terms-- triple prime delta t cubed plus o delta t fourth.
But actually, in this case, another equally convenient way is to expand ui's in terms of ui's plus 1's because there are as many terms that is i plus 1 as there are many terms of ui.
So it is equally convenient, so equally cumbersome, whichever way you think about it.
u prime of i plus 1, I can also do a Taylor series expansion also by just taking the derivative.
Now, I'm going to write everything [?
correspondence ?]
here, but every term has one more derivative than what is above.
So when I have third derivative here, I actually have fourth derivative.
And this is still to the fourth.
So all the delta t's to the power is the same, but all the derivatives have one more order because I'm expanding the derivatives.
And when I should take the second load of derivatives, I should be taking the second derivative of the derivative, which is the third derivative.
All right.
Is it clear why I'm having one more derivative here?
Now, this is all I need to expand.
And to figure out the error, we need to plug these extensions into the formula.
We also need to plot something else in the formula.
Oh, in this case, we don't need to plug anything else.
If I were writing the scheme not as u prime, but as X, then I also need to plug in the differential equation into the formula.
But I'm already writing the scheme in terms of u prime, then I don't need to plug anything else into the formula.
I just need to plug in the Taylor expansions.
When I plug in the Taylor expansions, what I get is, I'm going to write down the truncation error, which is the left-hand side.
Well, this case, it is actually the truncation era times delta t is equal to this.
I'm going to let you know why I'm writing this.
But the truncation error times delta t is the left-hand side minus the right-hand side.
And let me just write down term by term.
The first term is ui plus, which is this.
Let me actually start by changing the color.
It is ui plus ui prime times delta t plus ui double prime times delta t squared over 2. plus ui triple prime delta t cubed over 6 plus our delta t to the fourth.
The second term is minus x times ui-- let me changes the color to this-- minus x times ui.
Now, what is the third term?
It's minus y times ui prime.
So it is minus y times ui prime.
I'm writing down here so that everything responding to prime is on this column.
And the third term is minus z times ui plus 1 prime, and we again plug in the Taylor expansion.
But everything starts at the prime, so I have a minus z times ui prime here.
I have minus ui dot double prime times z delta t. So this is double prime.
This is z times this, and I also have this.
I have minus ui triple prime times 1/2 of z delta t to the cube.
And I don't even need to write this.
I'm just going to write to this as o delta t cubed times z.
All right
[INAUDIBLE]
Up more?
No, the other way
The other way.
Scroll up.
Right.
So what I'm doing is I'm expanding this term as the first line, this term at the second line, this term as the third line, and this is the fourth line.
I'm just writing things down, and it's different lines, and I'm also aligning the same derivatives of u on the same color.
Is it clear?
So how do I choose the coefficients?
I'm going to choose coefficients to cancel as many terms as possible.
I'm going to choose, first of all, the first priority is to cancel anything that is ui, anything multiplied on ui.
So what x should I choose?
[INAUDIBLE]
Exactly.
x has to be equal to 1 because I want these two terms, I want them to be canceled.
The only way to cancel this is we choose x equal to 1.
Is it clear?
And what should I do to ensure that this term also cancels?
[INAUDIBLE]
What?
Somebody?
[INAUDIBLE]
y plus z equal to 1
[INAUDIBLE]
Equal to delta t. Exactly.
y plus z has to equal to delta t in order for this, this, and this to cancel.
y plus z has equal to delta t. Do we have a question?
No?
All right.
What can we do to cancel this?
[INAUDIBLE]
We need to have minus z delta t to cancel with the last delta t squared over 2.
So we have z delta t has to equal to delta t squared over 2, which means z has to equal to 1/2 of delta t. Now, the question is, can I cancel more?
Can I make sure I can cancel more?
No.
I only have three terms to play around with.
I only have three unknowns, x, y, and z.
So I can only satisfy three equations.
On [INAUDIBLE] that of the three equations.
There is [INAUDIBLE] that satisfy automatically.
So unless I'm super lucky, I can only tune the three unknowns to satisfy three linear equations.
All right.
So in this case, it is pretty clear that z has to equal to 1/2 of t and y because of the second equation also has to be equal to 1/2 of t. Or I can just take these three equations and make it a matrix.
I can take these three equations and particularly, I can make, if I can-- well, let me see.
If I can divide the second and third equations by delta t so that the unknowns are x and y over delta t and z over delta t. If I make them as the unknowns, then I can write the equation as x, y, z equal to 1, 1, and 1/2.
How can I do that?
How can I fill in this matrix?
[INAUDIBLE]
Well, the first line is 1, 0, 0.
Thank you.
It's just the first equation x equal to 1.
What about the second line?
I mean, this is not y, z, but y over delta t and z over delta t. What is the second line?
[INAUDIBLE]
0, 1, 1.
Exactly.
And what is the third line?
0, 0, 1.
I mean, this looks obvious, but as you get to more complex schemes, as what's your name again?
Jacobi
Jacobi.
Jacobi has discovered in his [?
one ?]
contact schemes, you cannot just by looking at equations and get all the coefficients.
You have to solve a matrix equation like that.
And in MATLAB, let me set up this matrix.
This matrix is 1, 0, 0, the first line.
The second line is 0, 1, 1.
The third line 0, 0, 1.
This is the matrix.
Exactly.
The matrix.
And the right answer is 1, 1, and 1/2.
Yeah, that's the right-hand side.
And by a backslash c, does everybody know what the backslash means?
Yes.
It's matrix division.
It is solving this equation.
It is solving ax equal to b.
It solves for x, and I push.
It give me the coefficients, 1, 1/2, 1/2, which means x equal 1, y over delta t equal to 1/2, which means y equal to 1/2 delta t, and z also equal to 1/2 delta t. All right.
So some of the scheme we have is ui plus 1 equal to ui plus 1/2 delta t ui prime plus 1/2 delta t ui plus 1 prime.
All right.
So good.
I think we resolved the conflict.
There is only one scheme that satisfy this matrix equation.
And there is only one scheme that can make sure the first term cancels, the second term cancels, the third term cancels.
All right.
And remember z is 1/2 delta t. So what is the truncation error here?
What is the truncation error?
What is the other of the truncation error of the scheme?
What local [?
order of accuracy ?]
does the scheme have?
[INAUDIBLE]
Second order delta t square.
Because this is square, so sorry.
This is square, but z is equal to 1/2 of delta t, so this term is delta t too.
So the truncation of the leading term of the delta t times tau is delta t too, which means tau is delta t square.
All right.
And why did I say this is delta t tau?
This is because if you look at how accurate I am approximating the derivative, I mean, I'm approximating the average of the derivative between i and i plus 1 set.
I'm basically approximating the average of the i-th derivative and i-th plus 1 derivative.
It is equal to what?
ui plus 1 minus ui divided by delta t. And if you look at the truncation error of this scheme, so if I do this rigorously, this is delta t cube here.
And if you transform, go from this equation to this equation, you have to divide everything out by delta t. So you get delta t squared.
So the truncation error of this scheme is delta t squared.
All right.
I'm basically taking these two terms, dividing by delta t, and getting this.
Well, I'm moving this term to here and divide by delta t to get this.
And what is that here?
Because I'm dividing by delta t, I get delta t squared.
So the scheme is second order accurate.
Any questions?
No?
Here?
Let's try to implement the scheme into one of the very simple problems.
Let's do a pendulum problem.
In a pendulum problem, we have a pendulum that is vibrating on a small angle.
So today we restrict ourselves to small angles so that the equation is not going to be linear.
And we're going to be going to [?
nulling ?]
equations, and we're going to see why [?
nulling ?]
equations is going to be more complex especially for implicit schemes.
So if it's explicit scheme, actually, I'm going to show you later that it's very easy to do a non-linear scheme, but for implicit method, like the one we just derived, one that actually involves a u prime of i plus 1, it is going to be much more difficult to do a non-linear scheme.
But here the variable is theta.
And what determines the acceleration of theta?
What is the second derivative of theta?
What's that going to be determining?
[INAUDIBLE]
Force balance, right.
What is the acceleration of this ball?
Let me just ask, what is the gradual acceleration of this ball?
[INAUDIBLE]
[INAUDIBLE].
This is [INAUDIBLE], so the only way to accelerate is along the [?
duct.
?]
So it is [INAUDIBLE].
is acceleration?
[INAUDIBLE]
It can be moving.
Let's assume there is no friction.
The only force it has is the gravity force and the tension of its spring, which is [?
non-elastic.
?]
g sine theta
g sine theta.
Why is that g sine theta?
[INAUDIBLE]
Yeah.
a is equal to F over m. It's force over mass, and what is the force?
The force is actually here.
This is the force in interaction.
So it is mg sine theta over m, which is exactly g sine theta.
So you are right.
Sorry.
What's your name again?
Libby
Libby.
OK. Libby is right.
So this is acceleration.
And what is the acceleration in theta?
What is the second derivative of theta?
Just a geometric relation between this and deceleration
[INAUDIBLE]
Divide by L. Exactly.
So this is a over L. So it is going to be g sine theta over L. All right.
And we are going to be linearizing this equation because sine theta, when theta is very small, is what?
Again, Taylor series expansion sine theta is equal to sine theta at 0 plus theta times the derivative of sine theta at 0, which is equal to 1, which is cosine theta at 0, et cetera.
So this term is 0.
This term is theta, so it is approximately equal to theta.
I mean, this is probably sine theta can be approximated by theta a lot of times, but in order to know this is, again, a result of Taylor's rule expansion.
So sine theta can be an approximate of theta, and now I'm going to write the question theta double dot is equal to g theta over L in a pretty strange way.
I'm going to write it as theta doubled dot equal to-- no, I'm not going to write it like this.
I'm going to be writing this as d theta dt equal to theta dot, which is just a definition of theta dot.
It's equivalent.
d theta dot dt is actually equal to, again, the second derivative, which is equal to g theta provided by L. You may be asking, why am I doing this?
Again, I'm sorry.
So there is a negative sign on this.
I have forgotten because-- so if theta is positive, the acceleration is in the negative direction, so I'm using a negative sign.
Yeah, if you discover something like this, please shout.
I'm writing it this way because remember the schemes we have been deriving are also first-order ODEs.
What are first-order ODE mean?
It means I can only have first-order derivatives through time.
All right.
So if I have a second-order ODE here, I need to convert it into a first-order ODE.
Actually, I cannot convert it into first-order ODE.
I have to convert it into two coupled first-order ODEs.
Whenever you have a second-order ODE, you can always convert it into two first-order ODEs.
Whenever you have a third-order ODE, you can always convert it into three first-order ODEs by introducing basically a separate variable, theta dot.
So here I am having theta and theta dot as two variables.
The derivative of both theta and theta dot is a function of theta and theta dot.
Is it clear?
Let me do this.
Let me go to MATLAB and start to write this.
Let me do this.
I'm going to create an empty file here.
I'm going to find out why MATLAB doesn't listen to me.
It's called pendulum.m.
Open Pendulum.
That's good.
So when Windows doesn't work, you can always rely on MATLAB.
So what do we do?
We want to code up this differential equation first.
Whatever scheme we use, we need to have a function that computes f of u.
The function should do like this.
You give me u, the function returns f of u or du dt.
I just call this function du dt equal to f of u.
And what is u?
u actually contains theta and theta dot.
Let me do this.
Theta equal to this first element of u. Theta dot is equal to the second element of u. I'm going to be computing theta double dot-- I'm just going to call this d dot-- is equal to minus g times theta divided by L. L is equal to minus g times theta divided by 5.
So now what I need to return is du dt.
And what is du dt now?
Remember u is theta and theta dot.
So du dt is actually theta dot and theta double dot.
That's all.
That function encodes the differential equation.
If it will give me a u, I'm going to return a du dt.
Here u is a [?
matter ?]
of 2. du dt is going to be a [?
matter ?]
of 2.
Clear what this function does?
Now, let's see what forward Euler does.
Let's define dt equals 0.1, and we want to simulate this thing for how long?
[INAUDIBLE]
60 seconds.
All right.
I have to initialize a u0.
So what u0 you want?
I mean, it has to be a theta and d theta dt.
Anybody?
[INAUDIBLE]
If you look at this, when I am approximating sine theta as theta, is it in degrees or radius?
It can be either, right?
It can be either, so it doesn't matter.
Oh, yes, it doesn't matter because it's a linear equation.
It's a linear equation, which means if I change a unit, I'm just scaling everything by the unit conversion factor.
And the linear equation still should be satisfied.
That's like the definition of the linear equation.
If you scale everything by some factor, it's still satisfied.
So it can be either degrees or radius
0
0.
And 1?
[?
It's ?]
really boring
It's really boring
[INAUDIBLE]
So this is my initial condition.
And here's the forward Euler.
Did you see this?
Actually, I think I should do this.
I should make another file so that you don't have to look at the bottom.
forwardeuler.m.
I have to open forwardeuler.m.
So I'm going to be setting u0 1, 0.
I'm going to set dt equal to 0.1, t equal to 60, and I'm going to forward t equal to 0, 0, dt t. So this is a loop.
I'm going to compute du dt is equal to actually, I think pendulum of u0.
And what is forward Euler?
Forward Euler is my u at the next step is equal u0 plus u dt times dt.
And when I go to the next step, u0 should be set to u.
All right.
And now what I want to do is I want to record the history.
All right.
I want to record the history.
What I want to do is history is equal to an empty array, and the history is equal to history u0.
I'm going to run this.
It doesn't run.
I'm going to forward Euler.
It's done, and I have an array history, and I can plot history.
It is done to give me a history of the numerical solutions.
Does it look good?
No.
So this is forward Euler, and I'm using a pretty big time step.
Let's see what can I do better if I use a smaller time step.
You see, this is better.
And a difficult solution should be an oscillating pendulum with no increase in the magnitude of the oscillation.
And now can I do even better by having an even smaller delta t?
All right
It's still working
Still working?
Am I having a reasonable delta t?
Yeah.
It should be-- It actually takes a pretty absurd value of delta t to make this like not even super accurate.
And you can visually see the empty array still growing.
It is visually different from the analytical solution of the ODE.
So let's go back and try to implement something that is better.
Just name a scheme that is better than forward Euler
Midpoint
Midpoint.
Sure, let's do midpoint.
To avoid repetitive work, let me just rename the Forward Euler using Midpoint.
And just copying the Forward Euler and to rename it as Midpoint.
I'm going to load this guy.
I'm opening midpoint.m.
So this is Forward Euler, but I'm going to change it to Midpoint.
How should midpoint be different.
What is the difference between midpoint and forward Euler?
[INAUDIBLE]
If you look at the formula, midpoint, instead of ui plus 1 equal to ui, plus delta t times fu, I have ui plus 1 equal to ui minus 1 plus 2 delta t times fu.
So two things, instead of ui, I need ui minus 1.
Instead of just f of ui, I need 2f of ui.
The second one is easy the change.
I can just multiply by 2.
Now, how do I do about this?
How can I get ui minus 1?
[INAUDIBLE]
Good point.
I need to do one step of forward Euler or something to get me one actually at time step first.
So let me do this.
Let me do one step of forward Euler.
u dt equal to pendulum of u0.
I'll just put a comment here, one step of forward Euler.
And the u is equal to u0 plus du dt times dt.
u0 is equal to u.
Is that enough?
[INAUDIBLE]
Exactly.
I also need to store something else.
I need to do this.
See, I need to store an extra step.
That is why I think now we get to why it's called a two-step scheme because you need to store two steps in your computer's memory.
While for the forward Euler scheme, you only need to store u0.
You only need to store one step in your memory.
So implement of midpoint scheme, the two-step scheme, you need to store two time steps in your memory.
You have to store u0 and u00.
So here I'm just going to use u00, and u00 equal to u0, and u0 equal to u.
You keep two previous time steps in memory.
So this is going to work, and let me go back to 0.1 L of t, for which forward Euler did a pretty crazy job.
It went over to something like 1,000 or something.
Let's try midpoint.
Let me Close All.
Did I Close All?
Midpoint, but I didn't plot it.
Plot History.
How about this?
Is this much better?
It's pretty.
So let me actually plot it not just the history.
Let me actually make the primes to be accurate.
Let me make 0 dt t. Let me actually plot the x-axis as the real time instead of time steps.
If I plot x-axis as unset, it would be like 1, 2, 3, 4, et cetera.
[?
It's ?]
the number of time steps.
Now, this is plotting [INAUDIBLE] solution of the ODE.
So here we get visibly the same answer as an original solution, which is sinusoidal oscillations.
We got half an hour left, but I want to take a short break now to get our minds back.
And next what we're going to be doing is [INAUDIBLE].
I'll see how it goes.
Let's continue.
Is everybody back?
Before I try your schemes, let me just show how easy it is to change what we did with the linear differential equation, change it to a non-linear one.
To change it to a non-linear one, we just keep the sine theta here.
Just replace this with sine of theta.
As soon as we change this to sine of theta, we lose the analytical solution.
I don't know how to solve this anymore.
Do you?
But once we have a computer, what I need to change is let me open my pendulum.m.
What do I need to do?
I just need to replace theta with sine of theta.
And let me do this again.
Midpoint.
I get a solution as easy as I'm solving the linear equations.
And when I plot it, I want to plot this with history.
Here I get the solution to non-linear equation.
I mean, it may look similar, but let me zoom in a little bit to see.
Oh, man, MATLAB doesn't listen to me.
I'm going to the x lim, which is kind of a scale to x limit to 0 and 5 here.
And well, it doesn't really show a lot of difference, but let me change the initial condition.
Open midpoint to m. Let me change the initial condition to how much?
1 is like 60 degrees
[INAUDIBLE]
[?
Reads ?]
very close to pi, so you get like-- so for a let's try it.
Let me plot it again.
Oh, yeah.
So we get [INAUDIBLE] So it will stay close to the top for a while and then drop down and stay at [?
0 ?]
for a while and then [?
conditions ?]
the other part of the chart.
To get over the middle of the area [INAUDIBLE].
So you get some very nonlinear behaviors where you have a big initial [?
areas ?]
as easy as you're solving a linear equation.
And that is why we use numerical methods.
This is when we don't have an easy way to get analytical solutions, and it is certainly true for most of the engineering problems we are dealing with.
And it's quite rare you can find any of the solutions.
And most of the time you have to use numerical solutions.
So let's go back and start to try-- we did forward Euler.
We did midpoint rule.
Let's try best implicit one-step scheme.
And I think I remember what it is.
Please correct me if I'm wrong.
ui plus 1 equal to ui plus 1/2 of delta t ui prime plus 1/2 of delta t ui plus 1 prime.
This is the one we just derived from writing down all the rows, having all the columns aligned and canceled three of the most important terms.
And we get this.
We get x being 1, y being 1/2 L of t, z being 1/2 L of t. Now, let's try this, and for this, I have to say for now I am going to go back cowardly to a linear equation.
And we are going to be talking about how to do the same with nonlinear equations.
I think three lectures from now or something like that, we are going to be using something called the Newton-Raphson.
But for now, let's stick to linear equations.
What we have is we have u prime is equal to something times ui.
ui is 2 by 1 vector.
ui prime is going to be 1, 0, 0, minus g over L times ui.
Why is that true?
You just need to look at the code.
du dt equal to theta dot, which is u2.
And theta d dot, which is u1 times minus g over L. This is the same as saying du dt equal to u2, u1 times minus g over L. I can basically rephrase all of this by this.
Or all the stuff I just commented out, made green is just this.
Agreed?
And this is just saying that du dt is equal to this matrix times u.
And this is going back and forth between the formula and the code, the formula and the code.
Doing this is the same as a matrix multiplication.
Now, the same thing happens for ui plus 1 prime.
Agreed?
So now if you plot this into the equation, we removed all the primes and replaced all the primes with ui and ui plus 1.
We can actually collect the terms.
We can say ui plus 1 equal to-- I'm going to use ui plus 1/2 delta t times ui prime, which I am going to write like this plus-- and switching to red, and you're going to see why I am doing this-- ui plus 1.
This is really important.
It is ui plus 1.
Why am I using different colors?
What is the difference between the blue terms and the red terms?
[INAUDIBLE]
Yes
[INAUDIBLE]
The current step versus next step.
The known versus unknown.
When I have the i-th step when I want to go to the next step, ui plus 1 is unknown.
ui is known.
All right.
So what I can do is I can rearrange and say that something times ui plus 1 is equal to something times ui.
Can somebody tell me what these two matrices are?
If I move all the red terms to the left and let me actually write it as blue because [?
there-- ?]
and all the blue terms to the right, what do I get?
What is the red matrix?
What is the blue matrix?
The red matrix
[INAUDIBLE]
1
[INAUDIBLE]
Great.
The two 1's are because ui can be represented as identity matrix times ui.
Identity matrix is 1, 0, 0, 1.
Now, these two terms are basically this matrix times delta t over 2.
Any questions about this matrix?
I mean, this is just combining these two terms.
How about the red one?
[INAUDIBLE]
1
[INAUDIBLE]
Negative delta t over 2, yeah.
Exactly.
It is basically taking this matrix takes the negative sign.
And here?
[INAUDIBLE]
1.
Great.
So to implement the scheme, we need this.
We need to basically multiply this matrix with ui and invert this matrix.
All right.
Clear?
I'm going to make another file.
I'm going to call it-- this scheme by the way, the best one step implicit schemes you guys invented is called the trapezoidal rule.
Not tx, t. Sorry-- dot m. Yes.
Oh, I should be copying.
this.
Trapezoidal rule is a one-step scheme, is a single-step scheme, so I don't need to do any forward Euler or anything like that.
All right.
And I'm going to be doing t equal to 0 to dt equal to t. [?
I'm ?]
going to type m first.
What I'm going to do is I'm going to talk to construct two matrices.
One matrix, I call it A.
Let me just call it red matrix and the blue matrix.
The red matrix is 1 minus delta t, delta t over 2-- I'm just going to type it.
1 delta t over 2, delta t over 2.
I think that I get a minus here.
That will be over 2 times g over L. and 1.
The blue matrix is roughly the same, but there is a minus sign here, and there is a plus sign here.
All right.
Then what do I do next?
u is equal to what?
[INAUDIBLE]
Inverse of the red matrix times blue matrix times u0.
So actually here I need to do u0 transpose because u0 is a row matrix.
I need to make it a column matrix, and the whole thing has to be transposed again.
All right.
And I'm going to be doing history is equal to history is 0 and u0 equal to [?
u.
?]
Is it clear what I did?
Anybody have any comments on this?
Is it a good way of doing matrix inverse?
[INAUDIBLE]
Yes.
u is ui plus 1
[INAUDIBLE]
Yes
[INAUDIBLE]
Good point
[INAUDIBLE]
Inverse red matrix is equal to inverse red matrix
[INAUDIBLE]
Sure.
I can also combine blue matrix
[INAUDIBLE]
Sure.
A good suggestion.
I'm just going to be u equal to a times-- all right.
And let me actually make u0 a column matrix by replacing this with this.
All right.
So I don't need to do all of this.
Here, instead of when I recall the history, I transpose it.
Great.
Let's see how it works.
Trapezoidal.
Yeah.
Let's plot this again.
It works perfectly.
All right.
So good job, guys.
The scheme you came up with works.
That's it for today.
And so next class, what I want to do is bring your own computer because we're going to be doing something-- next class, not I'm going to code up things like this.
Instead I'm going to ask you to code up something like this, and you are going to be working class and hopefully, show things on this screen.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hello, everybody.
So I guess yesterday was the a really serious reading due.
How is that going?
How do you like the reading?
What's consistency?
Hm?
What's consistency?
What's consistency?
Consistency is the scheme being at least first order accurate.
OK?
So the it's local-- consistently is related to local order of accuracy.
So if the scheme is at least first order local accuracy, then it is consistent
[INAUDIBLE]
It means the truncating error.
If you do the truncating error analysis, the data series gives you a truncating error of at least delta t squared if you don't divide by delta t. At least delta t if you divide by delta t. So that's what it means by being consistent.
I mean the word consistent basically means if you look at the scheme, it is somewhat consistent to the differential equation.
So the scheme on one hand, the differential equation on the other hand.
They are consistent, right?
So that would mean that the scheme is self-satisfied with the [INAUDIBLE]?
The scheme here-- so the question is does that mean the scheme satisfies the differential equation?
The answer is the scheme is never going to satisfy the differential equation exactly.
But as you make delta t smaller and smaller, it is going to satisfy the differential equation better and better.
OK. That's what it means by being consistent
So if it wasn't consistent, it would make the [INAUDIBLE]
Yes.
So, right.
If a scheme is not consistent, what does that mean?
That means if you make the delta t smaller and smaller, the difference between the discrete scheme and the continual differential equation is not going to become smaller.
It may become larger.
It may remain constant.
It may go around but it doesn't go to zero
How does that differ from the definition of stability?
How does that differ from the definition of stability?
We're going to see there are-- when you talk about stability, there are two types of stability.
They are different in the sense that a consistency concerns the Taylor series analysis.
You can figure out consistency by doing data series analysis.
Stability has nothing to do with data series.
Right?
We are going to see when you check stability, you're plugging in a half solution that grows exponentially and seeing can the exponential be a growing exponential?
Or the exponential has to be a decaying exponential?
If all the exponential solutions have to be decaying exponentials, then the scheme is stable.
If there is one mode that can be a growing exponential, then the scheme is unstable.
OK.
I clarified a question about consistency.
That is basically the scheme is at least first order accurate.
What is the difference between consistency and stability?
Consistency is by data series analysis.
Stability is by plugging exponentially growing solutions.
So basically asking does the scheme allow exponentially growing solutions?
Any other questions that arise in the reading?
If one person has a concern, probably your classmate also has it.
So please raise it
Maybe one other way to think about the definition of stability is analysis.
[INAUDIBLE] the solution behaves mostly [INAUDIBLE].
When we think about stability, meaning that we're taking the relationship.
We're looking at how [INAUDIBLE] analysis.
Other things [INAUDIBLE].
The other is [INAUDIBLE].
Because as w goes to 0, [INAUDIBLE].
Because I think more and more [INAUDIBLE] really [INAUDIBLE].
You could probably think there's a few different [INAUDIBLE].
You're going to need both of those in order to analyze [INAUDIBLE]
Yes.
Thank you.
Thank you for the [INAUDIBLE].
Any other questions?
No?
Then I basically want to very quickly review and recap what we did in the previous lectures and also what we read before today.
So we did local order of accuracy.
That is basically by plugging Taylor series into the numerical scheme and figuring out how, which is the truncating error.
Is it the left hand side [INAUDIBLE] right hand side of the numerical scheme?
When you plug in both the differential equation and the data series, does it go-- what order of accuracy does it go?
So as delta t goes to 0, how fast does my inconsistency between the differential equation and my numerical scheme go to 0?
OK. And the local order of accuracy is closely related to consistency.
Actually the definition of consistency is by looking at is the scheme at least a first order accurate or not.
OK.
Global accuracy.
You should have read about global accuracy.
Could somebody answer me the question of what is the difference between global accuracy and local accuracy?
Or are they the same?
Yes?
[INAUDIBLE]
So the global accuracy is the long term accuracy while the local accuracy is not the long term?
Local accuracy
The local order of accuracy you can figure out by looking at only one time step, right?
The global order of accuracy, you have to look at really by taking the number of time steps to infinity.
And as Professor Willcox said, if you look at the solution U at a particular time-- let's say U at t equal to 1-- and compare the numerical solution-- let me say V at t equal to 1-- right?
If you look at the difference between them, let's just take the absolute value of the difference.
That difference is really looking at how much error has the solution accumulated over one time step all the way from t equals 0 to t equals 1?
Right?
And if I look at that area as I take delta t to infinity, I'm actually accumulating more time steps, right?
So if delta t is to 1 I'm accumulating 10 time steps.
If delta t is [INAUDIBLE], then I'm looking at the total area that has been accumulated over 100 time steps.
Now we can for sure see that this puts a more stringent requirement on how my scheme behaves.
Because when I only look at local order of accuracy and just look at one time step, now I'm looking at not only how much area have I produced in one time step but also how much has the area grown?
I mean, does the area made in one time step actually only matter locally?
Or is this area actually amplified by the later time steps?
So it puts additional requirements on a scheme.
A scheme is globally accurate only if it is both locally accurate and what?
I hear two answers.
One says stable.
Another is consistent.
Who votes for stable?
And who votes for consistent?
Stable?
One, two, three, four.
Consistent?
One, two, three, four, five.
OK.
I said a scheme is globally accurate only if it's locally accurate and what?
So if we the answer is consistent, then it doesn't add anything, right?
Being consistent means being locally accurate.
OK?
A scheme is locally accurate and consistent means it's just locally accurate.
It doesn't say anything additional to being locally accurate.
That's the definition of being consistent.
It is just to be locally accurate.
A scheme is globally accurate only if it's locally accurate and is stable.
Actually zero stable.
And what does zero stable mean?
You should have just read about it.
I'm just going to say I mean you have read about it.
I'm just going to say another way of understanding zero accuracy is that when you solve the differential equation du/dt equal to zero, the scheme is going to be stable.
OK?
That's what it means by a scheme being zero stable.
In this sense, it's a very weak requirement.
Right?
You only need to be stable when solving this trivial differential equation.
Right?
What is the solution for a differential equation?
Confidence
Confidence, right.
I mean, if you're so very [INAUDIBLE], even your scheme can't even solve that differential equation, what good is the scheme?
Right?
OK.
So this is, yes.
It is an additional requirement on top of local accuracy.
But it is a pretty weak requirement.
All right?
That is why I think it is such a great result that we can say being locally accurate [INAUDIBLE].
And being zero stable, which is such a weak requirement, you can immediately have the conclusion of global accuracy.
Right?
So what theorem or result gives you that conclusion?
[INAUDIBLE]
What is it called?
Dahlquist Equivalence Theorem.
Yes.
Right.
So that theorem basically says that these are locally accurate, which only looks at one time step.
And if you have are zero stable-- which means you can solve this trivial differential equation-- then your scheme is globally accurate which means you can solve the differential equation for a fixed amount of time as it takes delta t to go to zero, which also means the amount of time steps goes to infinity.
Your global solution becomes more and more accurate.
OK. And in addition to this, zero stability gives you the result that the local order of accuracy is the same as the global order of accuracy, right?
So if you can do Taylor series analysis, you can not only know the local order of accuracy.
You also know the global order of accuracy.
OK. Eigenvalue stability is a step beyond zero stability.
This tells you that your scheme has to be not only solving this equation but also du/dt equal to lambda U.
Right?
And then this is also a pretty easy differential equation.
You have analytical solutions to that.
The zero stability, you can think of as a special case.
It's a much weaker requirement than Eigenvalue stability.
Eigenvalue stability means you need to solve now nontrivial differential equations.
OK.
So today what we're going to do is we are going to exercise this local order of accuracy, global order of accuracy, and the zero stability, and Eigenvalue stability by looking at a pretty simple problem.
It's a ballistic trajectory prediction problem.
I hope most of you brought your computer.
Does everybody have your computer with you?
And do you have MATLAB installed?
Yes?
OK.
So what I want you to do is, I want you to do mathematical modeling which is deriving some ODEs.
I'm not going to give you any ODEs.
You need to derive it.
And you're going to solve the ODE using forward Euler.
Solve the ODE using midpoint.
And solve the ODE with one of the homework, the first homework question you did, which is the most accurate two-step explicit scheme.
Anybody who can tell me what that scheme is?
V of m plus 1 equal to minus 4 V m plus 5 V m minus 1
[INAUDIBLE]
Plus 4 delta t F of V m plus 2 delta t F of Vm minus 1.
So I mean, yeah.
It is the most accurate two-step scheme.
But why doesn't it have a name?
Something like this should have a name, right?
Let's try to see why it doesn't have a name by applying this onto the ballistic trajectory prediction problem.
All right.
And let's look at accuracy, stability, convergence and how they relate to each other through the Dahlquist Equivalence Theorem using this example.
OK.
Ballistic trajectory predictions.
Here is some history.
It is really the motivating factor of developing the first real computer, the first real electrical computer is what they call it.
They are really designed to help engineers solve this kind of problem.
And here we are going to be programming a MATLAB to solve such a prediction problem.
So we are thinking if the motor fires a small 3 kilogram and 10 centimeters in diameter spherical cannonball, so it is fired over here.
At sea level in standard atmosphere, you can look for the air density or whatever from the internet.
And so I want to derive the differential equation.
That is going to allow me to predict the impact if I give you the initial velocity, the x velocity and y velocity of this thing.
Here is some additional data.
Aerodynamic drag has to be considered.
The cannon ball, the force on the cannonball is gravity and drag.
Right?
Those are the only two forces.
And let's assume it's fired at a subsonic speed.
Subsonic Cd, that coefficient.
Let's use 0.5.
And yeah, actually we give you the air density.
So you don't have to find it out.
So what's the differential equation?
The differential equation such that you can [INAUDIBLE] in MATLAB.
For deriving the equation, you can work in groups of two or three.
Writing the code has to be done individually.
All right?
So if you're sitting alone, please tend to gather.
Form groups.
And continue working out the differential equations together.
Any questions before we dive into the lab?
[INAUDIBLE]
Uh huh
[INAUDIBLE]
What is the angle?
This is [INAUDIBLE].
And I'm going to tell you that.
Let's leave it as the parameter [INAUDIBLE].
And I'm going to tell you where the [INAUDIBLE] what happens [INAUDIBLE].
OK.
Lot of you have figured out the ODE.
So I'm just going to write it down here so that everybody is on the same page when you go into implementation.
OK. What happens is that the force-- if you look at the cannon ball, if the velocity is Ux and Uy, it has a force of gravity, m times g. It has a force of drag, right?
And the magnitude of the drag is equal to Cv times half of rho U squared.
U squared is Ux squared plus Uy squared.
And the drag also has an angle.
The angle is proportional to the angle of Ux and Uy but in the opposite direction.
So if you write down the differential equations, it's dUx/dt.
The force on the x component only has a component of drag on it.
Right?
So it is minus D times Ux divided by square root of Ux squared plus Uy squared.
This is the angle.
And as you plug it in, it is-- oh, and times the area.
When you plug it in, it is equal to minus 1/2 of Cd times row S times square root of Ux squared plus Uy squared times Ux.
And dUy dt is equal to first of all-- Wait.
I think I'm missing an m here.
m times d Uy/dt.
That is the acceleration on the y direction, which is also equal to the force in the y direction.
It is equal to minus m times g minus a similar drag component which has the same Cd Row S over two.
It has the same square root of Ux squared plus Uy squared.
But the direction is in the y.
So it is proportional to the Uy.
All right.
So these are the two differential equations.
And when you want to compute the trajectory, you also need dx/dt equals what?
x and y is the location of the cannon ball.
Ux have and dy/dt equal to Uy.
So these are the four differential equations you need to implement and solve.
So let's try fourth order first.
And if you checked your email over the last half an hour, you're going to see I shared a Dropbox folder with you.
How many people got that?
OK.
So if you are happy with it and if you have Dropbox on your computer, you can make the subdirectory in the shared folder and you can pull in that directory so I can take a look from here after you put it up.
So if you look at your email, you are going to see a folder called 1690shared2014 being shared with you.
If you accept it, everybody is going to see the same content here.
And I see people have already uploaded code over here.
Great.
Thank you very much.
I'm going to open your model
[INAUDIBLE]
That's a what?
Let me upload my [INAUDIBLE]
Oh OK. Sure.
Please
[INAUDIBLE]
Are you going to also--

Oops.
OK.
I'm going to wait.
Oh OK.
So somebody must have not had Dropbox before.
Because if I shared with you and you actually--
[INAUDIBLE]
Yeah.
[LAUGHTER]
[INAUDIBLE]
I get more space if I shared.
And as a result of my sharing, somebody who didn't use Dropbox before now uses Dropbox
Oh
Yes
So you don't get access to anything?
All right.
OK.
So I have codes here.
And let me first run it to see if it also runs in my computer.
Mmm
What is [INAUDIBLE]?
Oh.
I see, I see.
OK.
So let's go through the [INAUDIBLE] here and explain to us what is your code doing
Right.
So first, I put in some constants that we need-- the diameter, the coefficient drag.
I apologize for any errors.
I'm definitely not immune to errors.
I don't think there are any.
But we calculate drag.
And then I calculate the cross-sectional area, rho, gravity.
And then I figure out the initial conditions.
So I didn't actually hear if you gave us an output.
So I just put in pi over 4
OK great
But I could have put anything in there.
So I calculated the initial x and y velocities here.
And then I create vectors with plenty of space to hold all my [INAUDIBLE] and set the initial conditions in the velocity.
And then I create a time step.
And then this is the integration.
So what I say is the position at 1 times 7, the future x position is the current x position times et times the x velocity.
And same thing for y.
And I have to calculate the changes to the velocity of both x and y.
And then I add those, again multiplying by the time step.
And then here it hits the ground again, I think.
And then I plot
Great.
Thank you.
Any questions about this [INAUDIBLE]?
Is it perfectly clear, everything?
[INAUDIBLE]
Here.
Let me run it again to see if it actually still works.
I am going to click Run.
It works
And there's some residuals here [INAUDIBLE]
So thank you.
And let me take a look at another code that is by same thing.
Did you update?
Yeah
Let me close it.
So--
Probably ballistic
Ballistic
Yeah.
That's been upgraded

Try again
No.
OK.
I got another code from-- all right.
Thanks.
And which one should I look at first?
[INAUDIBLE]
[INAUDIBLE].
Do you mind coming here and explaining what you did?
I think it's a little bit different from what-- so you organized the code in a different way.
I think this is more consistent with what I did in the last section
Yeah
Of the other one.
Yeah, the other one
So what I did was to find all my initial values, I chose a state of 45 degrees.
And this is all [INAUDIBLE], actually.
[LAUGHTER] I defined my rate on each side as 1,000 elements.
And then I chose a dt of 0.01.
And then I made an initial vector which has my x0, y0, and the dx0 and dy0.
And then we go into the while loop to do our forward Euler method.
And then to calculate our next element in our forward Euler method, I go to this function called trajectory that I made
So trajectory t is a function you made.
So let me open this also.
All right
Yeah.
And then here, you can see the equation [INAUDIBLE].
And so what this trajectory does is it takes in the position and velocity.
It applies them to the [INAUDIBLE] we derived, and then outputs two velocities and two [INAUDIBLE], which we then multiply by a change in our time step to solve for the new position
We have any questions on this?
And in the other file, I believe I need to change this to this.
Because I think you changed the name of this which you also have to change this too.
So that way, [INAUDIBLE].
So [INAUDIBLE], you can do the derivative that you did
After we did the derivative, we have to multiply it by our time step to go up an order to change it from velocity to speed and position.
We add that to our old position and get our new position.
And then we store that in our position vectors.
And then we [INAUDIBLE] the position
OK. Let's try to see if it works.
OK. Let me close everything.
Let's click Run.
Yeah.
We get a [INAUDIBLE]
[INAUDIBLE]
Yeah.
[LAUGHTER] And we do see that [INAUDIBLE], right?
Because the angle here is much deeper than the other angles.
[INAUDIBLE] slower than when it was shut off.
OK. Now my question to you is, what if I want to challenge you with a question?
What if I want to change this to midpoint?
From last lecture, we saw that at the same time step, midpoint was much more accurate than forward Euler.
Right?
So what should we do to change this to midpoint?
I'm asking the whole class.
Anybody have any idea?
Yes?
[INAUDIBLE]
Good
[INAUDIBLE]
Thank you
[INAUDIBLE]
[INAUDIBLE].
OK. OK.
So, right.
Now the question is what if we want to change this to midpoint?
If we change this to midpoint, the first thing is that midpoint is a how many step scheme?
It's a two-step scheme.
Forward Euler is a one step scheme.
So in terms of implementation, what do we need to change?
What do we need to ignore when we switch from a one step scheme to a two-step scheme?
Why do we call it a two step scheme?
[INAUDIBLE]
Because we need to store two time steps.
Here we only need to have that.
That is one previous time step.
Now we have to have two previous time steps.
Right?
So we need this.
And we need one step of forward Euler to actually get to the next time step.
So I'm just going to say a one step of forward Euler.
OK.
So in doing one step of forward Euler, I'm just going to copy this and copy this.
Right?
Here, I'm just going to pass my vect 0.
And the vect equal to vect 0 plus Vt times d vect dt.
Any questions on this one step of forward Euler?
OK. Now we are using midpoint.
OK.
In mid-point, now what we want to change is these two lines.
Right?
Let me scroll it down.
We need to change these two lines.
Can somebody tell me how do I change lines when [INAUDIBLE] to make this midpoint instead of forward Euler?
Yeah?
You have the vect.
And you add it to 2 times U vect dt and your time step
OK.
So first of all, in midpoint, we have 2 times delta t times F of v. Right?
So we have a 2 times.
What else do we need to change?
[INAUDIBLE]
For which one?
In the previous scheme?
Yeah.
You are right.
When I do this, I need to also somehow store the old vect.
So I need to do vect of 0.
Let me actually do this.
Let me actually call this vect 00 as the previous time step.
And the vect 0 as-- so, at any point, that 00?
Let me call it k minus 1.
OK. vect 0.
Let me call it vect k. All right.
So this is one step of forward Euler going from k minus 1 to k. All right.
Now in midpoint, where do I compute the derivative?
x time step k?
Or k minus 1?
K, right.
So that's k. So this d vect/dt is the time derivative at step k. Now my vect is equal to vect k minus 1 plus 2 times delta t times F of v. Right?
This is because in midpoint-- so forward Euler is V at k plus 1 equal to Vk plus f of Vk.
Midpoint, we have Vk plus 1 equal to be Vk minus 1 plus 2.
There is a delta t here.
2 times f Vk times delta t. Right?
This is what we are implementing right now.
All right.
So we have this.
And everything else I think is same
[INAUDIBLE]
Right.
Then I need to also update.
Because I need to set vect of k equal to vect.
Right?
And before I do that, I need to have vect k minus 1 is equal to vect k. Right?
So I'm done.
Should we run this?
Is it clear like if when you guys have fourth order implemented on your computer how to change your implementation into midpoint?
Good?
Let's run it.
Let me close out.
Now I'm going to run it.
Is it the same as before?
No
[INAUDIBLE]
Oh
[INAUDIBLE]
Oh.
Is this something to--
Yeah.
[INAUDIBLE]
You mean remotely?
Yeah.
[INAUDIBLE].
[LAUGHTER]
OK. OK. What did you do?
I didn't [INAUDIBLE]
Oh.
OK. OK.
So let me just make it 10 more time steps
[INAUDIBLE]
Oh, yeah.
Right
[INAUDIBLE]
Yes.
OK. Let me run it
[INAUDIBLE]
Let me-- I think this is too much.
Let me change it to-- I think I just do this much.
It may be enough.
Let me see it.
And when you plot it, let me make sure to do x is equal.
Do you know what this means?
This just means that the x-axis and y-axis are going to be on scale.
Right?
OK. That's good.
So midpoint also works.
So first of all, I want to use the remaining few minutes, let's try to implement the most accurate scheme we got to see-- we already have a two-step scheme.
Right?
And it should be pretty easy to change to this scheme we have over here.
So instead of here, so we have a d vect dt at step k, right?
We also need a d vect dt at step k minus 1.
That is just called in the same function at step k minus 1.
Right?
You see what I'm doing?
I'm computing two time derivatives.
Now in this update, I have a minus 4 times Vk and the plus 5 times Vk minus 1 plus 5 times this, right?
OK. And then the second line is plus 4 delta t f of Vk plus 2 delta t f of Vk minus 1.
So this is 4 plus 2 times dt times d vect dt k minus one.
Right?
So that completes our change from midpoint to an even more accurate two-step scheme.
And by more accurate, I mean global or local order of accuracy?
Local
Local, right?
Because we did this using Taylor series analysis.
So let's see if it translates into global order of accuracy.
Let me click, let me close this first.
And let's run it.
You see this?
You see this?
What does this mean, 10 to the 162?
That's where my cannonball bounced.
What happens?
What happens?
Let's look at our scheme solution not for this many time steps.
But let's just go 19 time steps.
And let's put a circle at every time step to see what actually happens.
OK. After just 19 time steps, we've got 10 to the 22.
That's amazing.
And we basically just shoot out and diverge, right?
So why do you think that is the case?
Do we have any global order of accuracy?
I mean, we can do that by changing the time steps to be even smaller.
And we would change-- don't change it to this.
Let me change it to twice smaller.
And the time steps to also twice as much.
That is a good way of assessing the global order of accuracy, right?
It will decrease the time step by a factor of two, increase the number of time steps by a factor of two, right?
To see if the solution gets more accurate.
OK.
It gets wilder.
Instead of 10 to the 100 or something, it's 10 to the 200 and something.
All right.
So something is wrong.
What is wrong?
Can somebody just shout out?
What we reviewed in the beginning?
Our scheme is not zero stable.
It can't even solve this differential equation.
Let me repeat.
The most accurate two-step scheme can't even solve du/dt equal to 0.
You want me to prove that?
I'm going to prove that analytically.
OK. We have the following scheme.
I'll see if I can do it in five minutes.
It's equal to minus 4 Vk plus 5 Vk minus 1.
And the rest of them on the right hand side are f of V, right?
If I solve the differential equation du/dt equal to f of U equal to 0, then f of U is equal to 0.
I have nothing after this.
And this is my scheme.
I want this scheme to reproduce a constant solution.
Right?
The question is why can this scheme not reproduce a constant solution?
The answer is like this.
The answer is because this scheme is going to allow an exponentially diverging solution.
OK.
If I have a solution Vk is equal to V0 times a zeta-- let me just write out z because I can't write zeta-- times Z to the k. I'm going to see-- if I plug into this equation and the equation is satisfied and my V is something greater than 1-- then what does it mean?
It means I have a very, very small V0, even if I have an extremely small V0, I can end up having a huge solution after sufficiently many time steps.
Right?
And when I plot this thing, I'm going to have a V of k plus 1 is V0 times V to the k plus 1 is equal to minus 4 times Vk V0 times V to the k. And sorry, this is k minus 1.
Right?
This is my scheme.
Times V0 V to the k minus 1.
I cancel this.
Cancel this.
Cancel the V0.
Because they are the same on both sides.
And though all of these have V to the k minus 1 at least-- some of them are k, some of them are k plus 1.
So I remove that vector.
I have V squared is equal to minus 4 Z plus 5.
We get a quadratic equation.
Right?
This quadratic equation can have real or complex solutions.
But let's see what it has.
So a is equal to 1. b is equal to 4. c is equal to minus 5.
So we get a minus b plus minus b squared, which is 16.
Minus 4ac, which is plus 20.
All right?
So that's my two solutions.
And I can see it is-- right?
I'm basically factoring the two into these.
And I minus 2, plus or minus 3.
So that's equal to what?
It's equal to either minus 5 or 1.
All right.
OK.
If I only look at the one, that means good.
It allows a constant solution, right?
Z is equal to 1 basically means Vk is equal to V0 period, or whatever k. So in this sense, it does allow a numerical solution of this differential equation.
du/dt is equal to zero.
But it has another solution of Z equal to minus 5.
And it is that solution that makes the scheme diverge.
You only need one bad solution Z to make the scheme diverge.
And that's what we saw over here.
The scheme is not zero stable.
Therefore, although it is locally accurate, it is not globally accurate.
But because it's not zero stable, it can't solve this equation.
All right?
I'll see you tomorrow and continue discussing this and also look into Eigenvalue stability.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
All right, OK, let's start.
1690, Numerical Matters.
Local accuracy and global accuracy, again, this is an distinction between local accuracy and the global accuracy.
What is the distinction?
What is that we ultimately want in a numerical method?
Do we ultimately want global accuracy or local accuracy?
We ultimately want global accuracy, right?
Local accuracy, actually, I mean if a scheme only has local accuracy, it's useless.
It is useful only [?
if it has ?]
global accuracy.
Global accuracy means the solution using numerical method actually matches the true solution as my delta t goes to 0.
So global accuracy is what we want.
But how can we analyze global accuracy?
We can only look at global accuracy when we of apply the scheme to solve a differential equation.
Right?
Is there a way to show global accuracy without actually coding it up and solve a particular differential equation?
Is there a way to show global accuracy without applying this definition?
[INAUDIBLE]
OK, so global accuracy means V at a particular time converges to u at t. So Vt is the numerical solution.
ut is some true solution I may or may not know, depending on what equation it is.
So this is global accuracy.
I mean, of course I can check global accuracy by applying my scheme to a particular differential equation.
But is there a way I can know if a scheme is globally accurate or not without [INAUDIBLE] apply?
Yes?
[INAUDIBLE]
Yes.
That's the answer I'm looking for.
If the scheme is locally accurate, which is also consistent, and it is 0 stable, then I know whatever differential equation I applied it to, I'm going to get global accuracy.
This is a good thing because it is independent of what differential equation I look at.
If I can show local accuracy, which doesn't require me to implement anything, I just do what?
How do I show local accuracy?
Parallel theory?
Parallel theory is right.
You can show local accuracy by applying parallel theory as analysis.
And can you also show zero stability without implementing anything?
[INAUDIBLE]
Exactly.
You can plug in the du dt equal to 0.
So 0 stability is if a scheme can solve this.
Usually, whatever scheme you have, you're plugging this previous differential equation into the scheme.
You can analyze this behavior analytically.
You can plug in an exponentially growing solution to see if that solution is possible.
If an exponentially growing solution is possible, then the scheme is not zero stable.
If an exponentially growing solution is possible, then it's not zero stable.
If an exponentially growing solution is not possible, then the scheme is zero stable.
Now you can show local order of accuracy and zero stability all without implementing the scheme on any differential equation.
And once you have these tools through the [INAUDIBLE] equivalence theorem, you know the scheme is globally accurate when it is applied to any differential equation.
Is it clear, the relationship between these three?
Global accuracy is what we want.
This is what we want.
What we want.
And the zero stability and local accuracy is easy to analyze.
And once we have the two easy to analyze things, we check these, then we automatically know global accuracy.
So the relationship between these three are this simple.
So when we have these two, that leads to global accuracy.
So we discussed zero stability.
But zero stability is a pretty weak criterion.
So zero stability means it can solve du dt.
Yes, if a scheme is locally accurate and zero stable, we are going to have global accuracy.
But sometimes, global accuracy may not be enough.
I'm going to show an example of why global accuracy is not enough.
So global accuracy only says that [INAUDIBLE] equal to zero.
[INAUDIBLE] solution [INAUDIBLE] analytical [INAUDIBLE], right?
It doesn't say anything on if my [INAUDIBLE] is finite, how big the error is.
It also doesn't say how [INAUDIBLE] my solution is.
And it turns out we can get [INAUDIBLE] solutions for finite delta t even if a scheme is local order of accuracy and zero stable.
I'm going to show you such an example.
The example is using a second order accurate scheme and also zero stable.
Can you guys think of a zero stable scheme that is second order accurate?
What scheme have you learned that is second [?
order ?]
[?
accurate?
?]
Midpoint
Midpoint.
Exactly.
Midpoint is locally accurate.
It's zero stable.
And in something we implemented last lecture, right?
So let me go back to my Dropbox.
[INAUDIBLE] Wait, where is my shared folder?
[INAUDIBLE] up there, but, like-- OK, it's over here.
OK.
I would ask you, who has completed these implementation we started last class?
Of course [INAUDIBLE] here I have finished.
Who else did it?
Can you raise your--
[INAUDIBLE]
No, not including [INAUDIBLE].
Great, great.
So we have a bunch of people who did this.
Anybody willing to try their code here?
No?
[INAUDIBLE]
[INAUDIBLE]
The first two
The first two.. All right.
So I'm going to copy your code into today's folder.
And which one should I run?
[INAUDIBLE]
[INAUDIBLE], OK.
Thank you.
Right.
So I see you did [INAUDIBLE] and midpoint.
OK, you are also doing this.
So I'm just going to use-- I see you did this double percentage thing.
Does everybody know how to use that?
Yeah, so basically, [INAUDIBLE] according to sections.
So I think it's Control-Enter, right?
To just around these sections?
I think it's already done.
And I can run midpoint.
[INAUDIBLE] That's nice.
What if I change the delta t to 0.5?
OK, what if I change delta t to 0.5, and I'm going to close all in the beginning of this so that, like, I'm going to close my [INAUDIBLE].
I'm going to run it again.
I get this.
And this doesn't actually defeat the purpose of the analysis of the scheme being globally accurate.
The scheme is still globally accurate because, as I decrease my delta t, as I make delta t equal to 0.3, for example, the oscillation gets smaller.
My solution gets better.
If I decrease to 0.2, the solution gets still better.
[INAUDIBLE] it gets better and better.
And when I decrease it to 0.1, it's almost invisible.
But I'm going to assure you.
If I let these things drop further, the oscillation is going to show up.
You want to try?
To try, OK?
p is equal to 100, let's see.
If p is equal to 100, oh, you did the [INAUDIBLE] thing.
I'm going to remove the [INAUDIBLE] him and do axis equal.
That's too much.
[LAUGHTER] p is equal to 50.
What?
OK. Did I change anything?
Why do I get this?
Do I need to do this again?
Or not?
Why do I get a--
[INAUDIBLE]
Variables.
The first one, OK?
I ran it, and then we're on the next point.
Oh, there it is.
So let me go back to my p equal to 50.
You see what I get over here?
I don't get a solution that is smooth.
I get a solution that-- [INAUDIBLE].
So what happens-- still, it doesn't defeat the purpose of being globally accurate.
If I make delta t equal to 0.05, this oscillation becomes smaller.
We can still see it, but it becomes smaller.
But when I increase t to be 100 again, this is going to show up again.
I mean, I see something is wrong.
OK.
So I'll do that later.
But the point is clear, right, although the skin is zero stable, it is second order accurate, the midpoint scheme.
But it doesn't really give the desired behavior.
It's even worse than the forward Euler which is only first order accurate.
If you look at the forward Euler, with a delta t of 0.5, which shows horrible behavior for midpoints, right?
I get an error somewhere.
Oh, I need to run this first, and run this.
So yeah, forward Euler works fine with a delta t equal to 0.5.
So in this case, forward Euler actually works better than midpoint, even though midpoint is a second order accurate scheme.
So why is this the case?
Both schemes are zero stable.
Midpoint is more accurate, supposedly more accurate than forward Euler.
But applied to this specific equation, unlike the equation we saw last week, the pendulum equation-- on the pendulum equation, midpoint actually works much, much better than forward Euler for a large [INAUDIBLE].
Here, a forward Euler works better than midpoint or [INAUDIBLE].
The reason is linked with eigenvalue stability.
It turns out that the particular case, we are looking at, for which forward Euler is actually eigenvalue stable, and the midpoint is not eigenvalue stable.
So eigenvalue stability is a requirement that is more strict, more stringent, than 0 stability.
Although midpoint is zero stable , it is not eigenvalue stable for that particular equation.
OK, any questions before we move on?
[INAUDIBLE]
Exactly.
Good question.
So zero stability is per scheme.
A scheme is either zero stable or not.
It's independent of the differential equation, because it just tells you, can scheme solve this previous differential equation or not.
Eigenvalue stability, on the other hand, depends not only on the scheme, but also on the equation.
In the simplest case, eigenvalue stability looks at, are you able to solve this equation?
So you can see zero stability is like a standard case of eigenvalue stability, where lambda is equal to zero.
So if a scheme is eigenvalue stable for lambda equal to zero, then it is zero stable.
But a scheme being zero stable doesn't mean it is always eigenvalue stable for all differential equations like that.
OK, so let's start discussing eigenvalue stability.
And at the end of the lecture, I'm also going to start talking about implicit schemes and the Newton Raphson iteration.
All right.
So eigenvalue stability analysis, the first thing I want to show you is this link.
If you go to this link, which I already opened, I think, you're going to get to the website called mathlets.org.
And it hosts a bunch of what they call Mathlets.
They are basically Java applets that allows you to visualize these various things.
And if you click on it, it is going to open a window.
You need a statue like Java security or something like add exception to mathlets.org.
Otherwise, your browser may actually block the thing.
But once you have it-- let me make it full screen.
OK.
Right.
So what it shows you is for what lambda-- remember, we are talking about eigenvalue stability-- is a scheme stable or not for this equation.
So we have a lambda here.
In the mathlet, a lambda can only be real values, but complex values.
So this is the space of our lambda times delta t. We can think of a point here is 1, 0.
That means lambda times delta t equals to 1.
So, for example, at this point, this lambda is equal to 10.
Delta t is equal to 0.1.
Then you're at this point.
If lambda is equal to minus 1, [INAUDIBLE].
The blue regions here shows you that for what values of lambda, for what combinations of lambda and delta t-- actually, it only includes the lambda.
What lambda times delta t is, for a particular scheme, eigenvalue stable.
So here is the forward Euler.
For forward Euler, the scheme is stable only if lambda [INAUDIBLE] times delta t was using this very small circle.
OK, what does this mean?
This means that if you have a lambda greater than zero, would forward Euler ever be stable?
No.
Actually, I would expect, if lambda is greater than zero, even the analytic solution of the PDE is unstable.
Why?
What is is the analytic solution for TDE?
[INAUDIBLE]
e to the lambda t, right?
Each of the lambda t is the analytic solution of [INAUDIBLE] unstable by itself.
So actually, not the only real one, but also the whole [INAUDIBLE] the analytical OD is unstable.
Because if the lambda is here, then the lambda is 1 plus i, how does the solution look like?
It's an oscillatory growing solution, right?
The imaginary part determines the frequency of the oscillation.
But real part determines the rate of growth.
As long as the real part is greater than zero, it's a growing oscillatory solution.
So that means, for the whole [INAUDIBLE] of the complex [INAUDIBLE], the analytical solution [INAUDIBLE] is unstable.
So you shouldn't really expect a numerical solution to make it stable.
I mean, you're going to see some, the numerical scheme actually does make some of them stable, but you shouldn't expect it.
What is more intriguing is that, even on the [INAUDIBLE] for which the analytical behavior of the ODE is either just a monotonic detail [INAUDIBLE] on the real axis into something negative times t. So is du dt equal to negative lambda times u.
The solution is u equal to u to the negative [INAUDIBLE] times t. So you get a monotonic case.
If you are over here, we'll have a negative real time, and a non-zero imaginary [INAUDIBLE], then you have an oscillating and decaying solution.
So you would also expect it to be stable.
But forward Euler, actually, is only stable within this region [INAUDIBLE] lamba delta [INAUDIBLE] over here, it's unstable.
OK, so that is interesting.
I'm just going to do a very brief analysis of showing why that is the case.
Forward Euler, V of n plus 1 minus V n over delta t applied to half of V n, which, in this case, is lambda times V n. So let's move all the knowns to the right hand side.
So this term is known, and this term is known.
So move this to the right hand side.
We have V n plus 1 over delta t is equal to V n over the delta t plus lambda times v n. And the [INAUDIBLE] is, if we [INAUDIBLE], V n plus 1 would be equal to 1 plus delta t lambda times v n. We can easily see if lambda is greater than 0, we have a growing solution.
If lambda is greater than zero, this term is greater than 1, we grow.
What if lambda delta t is less than minus 2?
We get an oscillatory growing solution because 1 plus delta t lambda would be less than minus 1.
And the next iteration I'm going to get a solution of the opposite sign, but larger than what it is at the n times f. That is why forward Euler is unstable when we have a lambda delta t less than minus 2.
Same thing happens when lambda is imaginary and 1 plus delta t lambda has a modulus of greater than 1, that means we are [INAUDIBLE].
So that's how you do eigenvalue stability analysis.
You plug in f of u for the lambda u into the differential equation.
You collect all the terms that is [INAUDIBLE].
And in general, you plug in the solution that is exponentially growing to see if the script scheme allows an exponentially growing solution.
That is how you would analyze eigenvalue stability, similar to how you would analyze zero stability.
Any questions?
[INAUDIBLE]
Yeah, the absolute value of that one has to be less than or equal to 1.
Yes.
Yep?
So, when you said that the [INAUDIBLE] has to be less than minus 2, [INAUDIBLE]?
[INAUDIBLE]
OK. Yeah.
[INAUDIBLE]
OK, good question.
When you are implementing forward Euler on the real thing, how do you know [INAUDIBLE]?
It's a good question.
So most of the time, you can analyze the eigenvalues-- let me back up.
Let me say, if you look at a scalar differential equation, if you look at, like, just one single differential equation, not a [?
coupled ?]
system of differential equations, even if the equation is non-linear, you can usually linearize the differential equation and look at what lambda is for linearized differential equation.
And the key is not to manipulate lambda, but ensure that delta t is small enough.
So let's look at here.
Imagine [INAUDIBLE] lambda delta t. But imagine your lambda is taken over here.
If your lambda is, let's say, minus [INAUDIBLE].
Can forward Euler be stable?
No?
[INAUDIBLE]
If my lambda is here, can forward Euler be stable?
[INAUDIBLE]
We still have delta t, right?
If lambda is here, what is to make delta t with a 0.1?
Then lambda has delta t which [INAUDIBLE] way over to here, into the stability region.
And we are fine, right?
OK, what if lambda is minus 500 plus 1,000 times i
[INAUDIBLE]
Yeah, even if lambda is minus 500 [INAUDIBLE] to make the delta t equal to 0.0001, you're going to be fine.
You're going to shrink your eigenvalues times delta t into here.
And you're fine.
OK, what pieces are not fine?
What are the cases where, no matter how small you make the delta t, you're going to be unstable?
[INAUDIBLE]
Right, if lambda has a positive real cost, if lambda is always here, and no matter how small you shrink it, you get a small delta t-- because delta t has to be real, right?
The constant has to be real.
You're going to shrink it to over here, but it's still unstable, right?
OK, what are the cases where forward Euler is not working?
[INAUDIBLE]
If it's just an oscillation, if the eigenvalue lambda is just a dual imaginary value, it's a non-decaying oscillation.
So if the analytical solution is a non-decaying oscillation, just like what we had on [INAUDIBLE], like the pendulum, if we have a pendulum that has no density, forward Euler would never be eigenvalue stable.
And that's what we saw, right?
I mean, even when we made delta t to be small, it's still mildly growing.
So yes, forward Euler will work if you could have a positive real eigenvalue, or you could have a zero real eigenvalue.
Otherwise, you can always shrink your delta t so that the scheme is stable.
Let's look at another scheme, midpoint.
OK, what happens on midpoint?
What is the stability region of midpoint?
It lies exactly on the imaginary axis from [INAUDIBLE].
So the case we have to analyzed saying two oscillations would never work for forward Euler, would never be stable for forward Euler, it works perfectly fine for midpoint.
That's why, when we did the pendulum case, midpoint worked so well.
It was, like, amazingly better than forward Euler.
But when we applied to this case, forward Euler works better because-- try to guess where the eigenvalues are for the case we had on [INAUDIBLE].
Anybody wants to take a guess of what lambda is for that problem?
Something with a very small imaginary component?
[INAUDIBLE]
[INAUDIBLE] means you have negative real part, right?
Yeah.
[INAUDIBLE]
If you look midpoint, as long as the eigenvalue lambda has a non-zero [INAUDIBLE], then it wouldn't be stable, no matter how small you make delta t-- I mean, if the eigenvalue is here, over here, no matter how small you make delta t [INAUDIBLE] was the origin.
But no matter how much you shrink it, you will never be exactly on the imaginary axis between [INAUDIBLE], right?
So as long as the solution is not pure oscillation, midpoint wouldn't work.
So forward Euler and midpoint are like two schemes that have complimentary values, but they never overlap.
A question would either work for forward Euler, or it'll work for midpoint with a small enough time stamp.
If the lambda is exactly on the imaginary axis, with a small enough delta t, you can make midpoint work.
If it has a negative real part with a small enough [?
timestamp, ?]
you're going to make lambda times delta t into this region.
What we're not concerned about is where lambda has a positive real part because the differential equation is unstable, and we don't want to solve it anyway.
I mean, we don't expect it to be stable.
Yeah, right.
So right.
So we don't really expect our scheme to be stable.
If it's stable, then it is artificially stable because the real solution here is going to be unstable.
Backward Euler, [INAUDIBLE] OK, we can see that, for any differential equation that is actually stable, backward Euler is stable.
The blue region is [INAUDIBLE] no matter how big delta t is-- you can make delta t equal to 1,000 if you like, backward Euler is [INAUDIBLE] stable.
So even in this region, where the real differential equation is unstable, artificially makes the solution stable, which actually gives you a wrong answer, but [INAUDIBLE] stable.
[LAUGHTER] All right.
Trapezoidal rule.
That is, like, yeah, you said that's what we want because, if you look at the stability ratio of trapezoidal rule, it is exactly the same stability region as the analytical ODE.
It doesn't always mean you're going to get the expected behavior because you're going to see is my lambda delta t is always here, the rate of decay of the trapezoidal rule versus the real ODE is going to be very different.
So like, at least in terms of stability reading, it is exactly what you have for the analytical ODE.
And the wrong [INAUDIBLE] schemes is something you are going to learn later on for this week's reading.
But you're going to see these weird shapes.
The [INAUDIBLE] schemes are explicit schemes.
So for non-linear differential equations, you're going to see very soon that explicit schemes are a lot easier to implement than implicit schemes.
And, for example, is great because it combines the advantage of forward Euler and midpoint.
Why?
Because as long as my lambda has a non-positive real part, as long as I make delta t small enough, I'm going to get into the stability region.
Forward Euler requires me to have a negative real part.
Midpoint requires me to have a zero real part.
[INAUDIBLE] requires me to have either 0 or negative real part.
And I can make delta t small enough.
It is going to work.
All right.
Questions?
Yeah?
[INAUDIBLE]
Eigenvalue stability concerns about an equation that is dx dt equal to lambda x.
We are looking at, if I apply the scheme, we [?
sub ?]
delta t to this equation.
Am I going to get a growing solution, or I'm going to get a stable solution?
Is there a way [INAUDIBLE]?
The stability region is different for each [INAUDIBLE]

So you have to-- yeah, it's separate from the [INAUDIBLE] solution.
For each combination of scheme and lambda delta t, you have an answer of yes or no
[INAUDIBLE] eigenvalue.
[INAUDIBLE] eigenvalue.
[INAUDIBLE] eigenvalue.
[INAUDIBLE]
Yes
[INAUDIBLE] and then you linearize it.
But because of your error [INAUDIBLE].
This sort of method [INAUDIBLE]
I don't have a definite answer to that, but it's possible.
And what I want to emphasize is that, if you linearize the equation, and eigenvalue stability shows that it is unstable, then it's very likely, when you apply to your [?
normal ?]
equation, that it's going to be unstable.
So eigenvalue stability, I would say before you apply the scheme to differential equation, it's a good idea to check eigenvalue stability.
And this also gives you a guidance of when you see something being unstable.
Is it because you are using the wrong scheme?
Or is it because you have a too large [INAUDIBLE].
Right?
You can take a look at these eigenvalues and figure out which case it is.
Any other questions?
OK, let me give you a challenge.
Now, if I get a couple differential equations [INAUDIBLE] where y is equal to x, where x can range all the way from 0 to t. For what x [INAUDIBLE] delta t is forward Euler table.
For what values of [INAUDIBLE] is backward Euler table.
And for what value of epsilon is it midpoint stable?
I mean, not for epsilon, but for what combination of epsilon and delta t is each of these [INAUDIBLE].
Form groups of two or three, and figure this out.
All right.
A lot of you have got the answer, or almost get the answer, and let me do it here to make sure everybody is on the same page.
First of all, the first step is to write the differential equation into a matrix form.
The du dt of x and y, let's write it in a similar way as du dt equals lambda u.
But here, the lambda is no longer a number.
It is a matrix.
And what is the matrix?
Just fill in the blanks.
dx dt minus y minus epsilon x.
So minus epsilon x minus 1y.
dy dt equal to x, so 1x plus 0y.
OK, so I get a matrix here.
And how do I go from the matrix to eigenvalue stability?
Take the eigenvalue.
Take the eigenvalue.
So the eigenvalue of this matrix, and I may be lazy, so let me do it in Matlab.
[LAUGHTER] OK?
Yeah, so for this 2 by 2 matrix, you can do it by hand.
But if you get a large matrix, what do you do?
Let me do the bad example
[INAUDIBLE]
OK, I'm just going to make epsilon go all the way from 0 to 10.
So linked space is putting a bunch of epsilon.
And the matrix here is going to be like this.
So let me do this.
Let me put e to be a symbolic variable.
And the matrix is what?
Minus e minus 1, 1, 0.
So that's what the matrix is.
And e means epsilon.
And you can do the eigenvalue of this matrix.
It gives you the formula, which you guys have figured out.
Minus epsilon over 2 plus or minus the square root of-- I mean, to the 1/2 power is basically square root, right, of this thing.
So let me just again say, epsilon e is equal to linked space of 0, 10, 100.
And I'm going to say lambda 1, the first eigenvalue is-- I'm just going to copy the formula here and paste it.
And I need to convert the products into dot and exponenting to dot exponential.
And everything else is going to be fine.
I'm going to also look at the second eigenvalue.
I'm going to paste it and also do the dot product and dot explanation.
OK, I have L1 and L2.
Let me plot them.
I'm going to plot a line and dot.
So what you get is, in the complex plan, let me do x is equal-- in a complex plan, I'm plotting the trajectory of lambda, of lambda 1, as my epsilon goes from 0 to 10.
I'm going to say hold on and plot the evolution of lambda 2.
Actually, I want to plot it in a different color.
So you get a [INAUDIBLE] of lambda 1, but this way, [INAUDIBLE].
And remember, this is, in the context [INAUDIBLE], the two eigenvalues, right?
OK, anybody remind me what is the stability region of forward Euler?
[INAUDIBLE]
Yeah, so a circle around negative 1.
So a circle around negative 1, right?
Going to the Matlab plot, it's a circle around here.
But remember, the stability region is a stability region of what?
Of lambda t. Lambda times t. Well here, I'm just plotting the lambda.
So what does that mean?
If I have, let's see, epsilon equal to 10, one of my eigenvalues is here.
Is there any hope?
I mean, the [INAUDIBLE].
Is there any hope of making forward Euler stable?
[INAUDIBLE]
How?
[INAUDIBLE]
By a big delta t equal to how much?
Or a small delta t?
A small delta t equal to how much?
Huh?
[INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE]
As long as I can make this point with being the [INAUDIBLE], where the [INAUDIBLE] here is minus 2, right?
So as long as I can have a delta t that is less than 1, 2, I can make forward Euler stable.
Any questions on that?
[INAUDIBLE]
Yeah, right, sure
[INAUDIBLE]
OK, you have beat me
[INAUDIBLE] that lambda [INAUDIBLE]
Yeah, right.
OK, thank you
[INAUDIBLE]
Right.
So we have to make [INAUDIBLE] stability region.
So, question?
OK. And so we can see, when epsilon is very large, it's difficult to-- you have to have a small delta t to shrink it [INAUDIBLE].
Another case is when epsilon is very small, actually adds epsilon equal to zero.
I get two eigenvalues equal to [INAUDIBLE].
How can I make the case stable for forward Euler?
[INAUDIBLE]
I can never make forward Euler stable.
I can make delta t goes to very, very small, [INAUDIBLE] shrink over here.
But they are still on the [INAUDIBLE].
And remember forward Euler, it doesn't include any point on the imaginary axis other than 0.
So when 1 epsilon is very small, it's impossible to make forward Euler stable.
And when epsilon is very small, it takes a very, very small delta t [INAUDIBLE].
So when a system is [INAUDIBLE], it's almost pure oscillation.
It also takes a lot of [INAUDIBLE] for very small delta t for forward Euler to accurately resolve its behavior.
Yes?
[INAUDIBLE]
Good question.
Does both Eigenvalues need to be in the stability region?
[INAUDIBLE]
Yes.
Both eigenvalues, or all of the eigenvalues, in this case, with two eigenvalues, [INAUDIBLE] all of the eigenvalues have to be in the [INAUDIBLE] region.
And the eigenvalues of a matrix doesn't always conjugate each other.
And in this case, [INAUDIBLE] conjugate each other.
And as our epsilon [INAUDIBLE].
And if you have a bigger matrix, that's even more [INAUDIBLE]
[INAUDIBLE]
Right.
The instability analysis of the analytical system, the analytical system is stable for the whole [INAUDIBLE].
So you are looking at the most dangerous [INAUDIBLE].
Where we have a numerical scheme, the most dangerous ones are not necessarily the one that lies towards the right.
It made be the ones that lies more towards the left.
Or most [?
low, ?]
most dangerous, of the stability region.
They are not necessarily the ones that is closest to the imaginary axis
[INAUDIBLE]
Yeah
[INAUDIBLE]
Yeah, for nonlinear problems, sometimes [?
they are ?]
very extremely important to adapt the timestamp according to-- because they have different [?
phase, ?]
and different parts of the solution, when you linearize the nonlinear equations, you get different eigenvalues.
So when you get the big eigenvalues, or [INAUDIBLE], it's important to make [?
them ?]
small.
Any other questions?
OK. Let me pose another change question.
Oh, right.
OK.
Yes, what about midpoint?
If you look at the eigenvalues, will they work for midpoint?
Only when epsilon is equal to 0.
Only when epsilon is equal to 0, we get equal eigenvalues.
And for what delta t would midpoint be stable?
Less or equal to 1, right?
Because the stability region for midpoint, when looking at lambda times delta t, that [INAUDIBLE] midpoint to midpoint.
If you make delta t lambda equal to 1, then the [INAUDIBLE] pinpoints was [INAUDIBLE] into the [INAUDIBLE].
If you make delta t greater than 1, you will push this point in the lambda delta t [INAUDIBLE] out of the [INAUDIBLE].
You'll find a scheme where it will be stable for all of the actions-- trapezoidal, right?
Because the trapezoidal rule has a stability region that covers the whole left panel.
And we are looking at the whole left panel over here.
What else?
The [INAUDIBLE] is called a [INAUDIBLE].
OK, that has a stability region somewhere like [INAUDIBLE] or something like that.
We can have a small delta t that would bring the whole thing into the [INAUDIBLE].
OK, yes?
[INAUDIBLE]
How do you find a stability region for various methods?
Good point.
I have showed you how to find the stability region for forward Euler.
I have showed you how to find the stability region for forward Euler by basically plugging du dt for the lambda u into the scheme.
I'm going to show you another example of doing this for midpoint, all right?
OK, if you have a midpoint, you have a Vn plus 1 equal to Vn minus 1 plus 2 delta t times half of Vn.
I can do this example, but you've got to help me.
What's next?
When I analyze eigenvalue stability, what do I do next?
What is eigenvalue stability?
[INAUDIBLE]
Yes.
Eigenvalue basically shows is the skill [INAUDIBLE] for [INAUDIBLE] equal to lambda u.
So basically, we are asking, is the scheme stable for this differential equation.
So when we look at these as f of u, we get lambda times Vn.
OK, so I'm going to write it down again.
This is what we want to analyze [INAUDIBLE] exponentially growing solution or not.
How do I analyze if it [INAUDIBLE] the exponentially growing solution or not?
I'm going to try one.
I'm going to try to say Vm plus 1 is equal to V0 times Z to the n plus 1.
Vn equal to V0 times [INAUDIBLE] minus 1 equal to V0 times Z to the n minus 1.
I'm just going to try 1, try an exponential growing solution to see if it is possible for Z to be something greater than 1 or less than minus 1, or having an imaginary having a complex value that has a modulus greater than 1.
But if it's one of these cases, I'm going to get an unstable scheme.
OK, let me plug this in, and I'm going to cancel all of these zeroes.
What I'm going to get is Z to the n plus 1 power.
Power is equal to z to the n minus 1 power.
I mean, these are time steps.
These are time steps.
Now what I get is to the powers.
I'm going to do parenthesis to denote time steps and the ones without parentheses to denote power.
Oh, this is a plus, sorry.
Plus 2 delta t times lambda Z to the n. OK, what do I do next?
[INAUDIBLE]
Factor out Z to the n minus 1.
I'm going to get a quadratic equation equal to 1 plus 2 delta t lambda times z. OK, now I can't get the solution for z to see if it can lead to an exponentially growing solution or not.
So I believe I get 2 lambda delta t plus minus square root of 4 lambda squared delta t squared plus 4.
So let me go [INAUDIBLE].
OK, so this is the Z I got.
I want this Z to have a modulus less or equal to 1.
[INAUDIBLE] And this 2, divided by 2, if lambda is-- so let's proceed.
If lambda is a real value, if lambda is anything real, and nonzero, then lambda squared is going to be something positive.
And what I get is something greater than 4.
And the square root is going to be something greater than 2.
So because I have a plus or minus, one of them is going to have a magnitude greater than 2.
When divided by 2, I'm going to get something greater than 1.
So for any real lambda, I'm going to be on unstable.
And you can also figure out, for complex values of lambda, what values is going to make it stable, what value is not going to make it stable.
And in fact, [INAUDIBLE] for all the values of lambda delta t, that will make this Z having a modulus of exactly 31.
So if this has a value exactly 31, my [?
schema ?]
is going to be exactly on the boundary of stability.
So if I have all this level 1, I'm going to be stable.
If I have 1 z that is greater than 1, if I have [INAUDIBLE] modulus, [INAUDIBLE], I'm on the boundary.
So by [INAUDIBLE], I'm going to compute the boundary of the stability region, which gives me something like this.
So if I take the boundary [INAUDIBLE] modulus 1, I'm going to block its boundary of stability.
[INAUDIBLE] Yeah?
[INAUDIBLE]
Yeah, Professor Wilcox is saying that, when you look at this equation, and plug in Z is equal to e to the ith theta, you're going to get Z squared, which is e to the 2i theta is equal to 1 plus 2 times delta t lambda times z, which is e to the ith theta
[INAUDIBLE]
Right.
And then--
[INAUDIBLE]
Right, and you can figure out the stability boundary for delta t lambda using [INAUDIBLE].
All right.
OK. Let me show you one last thing.
What if you have a du dt equal to minus u squared?
We get a non-linear equation.
For what scheme would this be stable?
What scheme this would be unstable?
The analysis of this scheme lies in linearization.
OK, so let's say, if I have a u of 0 at t equal to 0 equal to 1.
OK. What if I linearize this question?
What if I say, u, I'm going to put [INAUDIBLE] a u to u plus delta u, or plus a very small v. What is the result of the linearization?
The linearization can be seen by plugging both the u and the u plus V into the differential equation.
If you plug just the u, you get this.
If you plus u plus v, you get du plus V dt equal to minus u plus V squared.
And here is linearization V is very small.
And because v is very small, you can derive du dt plus dv dt.
This is just factorizing u plus v out of the derivative, is equal to minus u squared minus 2 uV, minus V squared.
But because V is very small, this term is negligible.
This term is very, very small, or very square small.
OK. Now, when you look at the answer to the equation has a du dt.
The right hand side has a minus u squared.
The right hand side has a minus u squared.
These two terms cancel out because of the answer to the equation.
The only thing you get is the linearized equation dv dt equal or minus 2u times t. No, yes?
No, yes?
That's how you would linearize a nonlinear equation-- by perturbing the variable into the variable something very small and remove all other very, very small terms, and the very, very, very small [INAUDIBLE], if you have cubic terms.
So now we have the linearized equation, which is linear with respect to V. Can somebody tell me, if I use forward Euler, when would this scheme be stable?
When would my forward Euler be stable?
[INAUDIBLE]
Good.
In this case, this is my lambda.
I'm solving dv dt equal to minus 2u times v. My minus 2u is my lambda.
This is where, like, you have a question of adaptive timestamping, right?
This is when lambda actually depends on the solution U of the nonlinear equation.
As I solve the nonlinear equation, my lambda of the linearized equation actually changes.
So in the beginning, I mean, the solution of this equation looks like this.
Actually, we figured out ut, I think, is equal to 1 over t plus 1 or something like that, right?
In the first or second lecture we figured that out.
So in the beginning, u is large, and minus 2u is kind of on the negative side, and big.
We need to use a smaller timestamp.
And later on, as you become smaller and smaller, we are allowed to use larger timestamps for forward Euler to be stable.
This is a case where we actually can benefit from adaptive timestamps.
The case is minus 2u has to be less than 0, equal to 0, and greater than or equal to minus 2, minus 2u times delta t because I am lambda delta t. This is the stability for forward Euler.
Questions on this?
Is it clear how did I linearize this and apply eigenvalue stability?
All right.
OK. Now the last thing I want to do, not sure I have time, but, like, I am going to pose these as a question-- is how do I apply backward Euler on this problem.
How do I apply backward Euler on this problem.
What is backward Euler?
Can somebody help me write down what backward Euler is?
OK, what backward Euler is?
What is forward Euler?
If I use forward Euler, what would it be equal to?
[INAUDIBLE]
Forward Euler would be e f of u to the n. What would backward Euler be?
For f of u n is Forward Euler
[INAUDIBLE]
Yeah, backward Euler is f of u to the n times 1.
So that that is backward Euler.
And in this particular case, I'm going to get minus u n plus 1 squared.
I'm going to get an equation that looks like this.
How do I solve that equation?
If I get any answer that is good, I'm going to end this class.
u n is known, right?
I'm at the nth concept.
So u n is known.
Let me put unknown in read.
u n plus 1 and the known in blue.
Yeah, that makes it a quadratic equation.
We get an unknown square, a linear, and a [INAUDIBLE].
So we can see we need to solve a quadratic equation now in order to go from one step to the next step.
We need to solve [INAUDIBLE] nonlinear equation to go from one step to the next step.
And what if this is q?
Then I need to solve a [INAUDIBLE].
What is this for?
I need to solve [INAUDIBLE].
What if this is final u times exponential u.
So this is what we are going to be reading from now to Monday.
You are going to be reading about one of the most important techniques in numerical analysis-- that is how to solve a general nonlinear equation like this-- something that is even more complex than a quadratic equation, more complex than cubic equation, something that you may not have analytical solutions.
And it is extremely useful [INAUDIBLE] of nonlinear equations, such as [INAUDIBLE] over here.
So I'll see you on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So let's start.
So first of all, it looks like there are many questions regarding the reading and homework.
So if possible, I like to answer some of your more [INAUDIBLE] questions.
Anyone want to raise a question I can answer?
Yes
So something that we were running into was the definition of local ordered [INAUDIBLE].
Local and stationary and global [INAUDIBLE] and how to figure out [INAUDIBLE].
OK.
So what we ended up doing was finding [INAUDIBLE] by taking dd plus 1 minus [INAUDIBLE]
Yeah
And then saying that that was the [INAUDIBLE] location of it
OK. Yeah
And then that was equal to-- whatever order [INAUDIBLE] that was was equal to the order of delta t times [INAUDIBLE] plus 1 where t is the local order [INAUDIBLE]
That's exactly what you should be doing is finding the local order of axis.
Did anybody else understand what [INAUDIBLE]?
OK. [INAUDIBLE] saying.
So that's exactly the right way of finding out the local order of axis.
Looking at the schema as [INAUDIBLE] n plus 1 minus the [INAUDIBLE] of the [INAUDIBLE].
That is you're tau.
That is your [INAUDIBLE] area.
And tau should be on the order delta t to the t plus 1 where p is a local order of [INAUDIBLE].
OK. That's basically repeating what [INAUDIBLE]
And how does that relate to the global order?
How does that relate to the global order of [INAUDIBLE]?
Anybody want to attempt to answer that?
[INAUDIBLE]
So say that there is an if
[INAUDIBLE]
If [INAUDIBLE] under what conditions does [INAUDIBLE] theorem hold?
[INAUDIBLE]
0 stable, right?
As long as t is great or equal to 1, which usually it is, you have a consistency.
So consistency is usually is not that much of problem when you already have the local order of [INAUDIBLE].
0 stable is something you have to separately.
But as long as the schema is 0 stable, then the local order of accuracy is equal to the global order of accuracy as long as that order of accuracy is great of equal to 1.
Is that clear?
Could you repeat that one more time?
If a [INAUDIBLE] is 0 stable, then the local order of accuracy is the same as the global order of accuracy
[INAUDIBLE]
Exactly.
When a scheme is not 0 stable, it's [INAUDIBLE] equal to 0.
And there is no global order of accuracy at all.
Yes
[INAUDIBLE]
Yes.
If a schema is not 0 stable, then the scheme doesn't work.
Remember when we had the supposedly most accurate two-step scheme, [INAUDIBLE] scheme?
What happens when we rebuilt the delta t?
The error, instead of decreasing, it increases.
So that's what happens with a not 0 stable scheme, where you refine your time step.
Instead of the error decreasing, you have an increasing error.
All right.
Any other questions?
[INAUDIBLE]
OK.
So the question is can we go over eigenvalues [INAUDIBLE] again?
[INAUDIBLE] page-- yes.
All right.
So in doing eigenvalues, stability you plug in du dt equal to lambda u, right?
So that is the question you use to analyze eigenvalue stability.
Then you write down an equation of Un plus 1.
Again, if it's a midpoint rule, then the scheme I'm plugging is plus 2 delta t times f of Vn.
And in this case, f of Vn is lambda times the Vn right?
So that is the equation you get by plugging in the differential equation into a scheme.
Now, the next thing to do is to assume my Vn.
And actually V n plus 1, Vn minus 1, and all other terms in your scheme satisfies an exponentially growing pattern.
So when I whatever time stamp it is, it is equal to the 0 time stamp times a growth factor to whatever time stamp [INAUDIBLE].
So once you plug all of these into the scheme, you can find the quadratic equation for z if it's a two-step scheme.
If it's a one-step scheme, you find just a linear equation for z.
If it's a three-step scheme, you're going to find out with a cubic equation for z, et cetera.
And then the next is to see, does that equation allow any z, any solution, who's modulus is greater than 1?
If that equation for particular combination of delta t and lambda-- delta t and lambda is always multiplied together.
So for whatever delta t times lambda if this equation allows for z that has a modulus greater than 1, then it is not eigenvalue stable.
The combination of the scheme and that lambda delta t is not eigenvalue stable.
Yeah
When you say the modulus greater than or equal to 1, I guess whenever I see modulus [INAUDIBLE]
OK.
So the question is modulus is.
Here the modulus is off a [INAUDIBLE] number
Oh, so do you mean like [INAUDIBLE]?
Yeah.
The [INAUDIBLE] to it is the real of z squared plus imaginary of d squared.
What is the significance of that?
The significance of that is that if that modulus is greater than 1, then z to the nth power is going to grow larger and larger as you make n larger.
If the modulus is less than 1, then z to the nth power is going to become smaller as a take n to infinity.
What happens if the modulus of these equal to 1?
Then the modulus of these always going to be equal to 1 whatever [INAUDIBLE].
OK. Any other question?
Yeah
[INAUDIBLE]
OK.
So last week's [INAUDIBLE], why does du dt [INAUDIBLE] negative lambda times u.
So in analyzing eigenvalue stability, you are looking at equation of this form.
Right?
So if I write down equal to lambda minus lambda times u, it is still in this form.
It's just the sign of the lambdas are kind of wrong.
I mean, you can-- maybe I should use another symbol.
Maybe I should say dudt equal to minus theta times u.
In that case, then you just have to [INAUDIBLE] lambda equal to minus theta.
And you can do the analysis the same way.
Does that answer your question?
Yeah.
[INAUDIBLE]
OK. Any other question?
No?
[INAUDIBLE] more.
OK. OK.
So for the reading recap, we have read about stiffness and the Newton-Raphson.
Let me recap what they are.
Stiffness are really orders of magnitude distance in timescales.
OK.
So a stiff problem means there is one timescale that is a very, very small compared to the timescale of the main phenomenon we want to simulate.
Maybe we want to simulate some phenomenon that is interesting.
But in order to simulate that, there is something of a hidden timescale that is much, much faster than the one we are interested in.
And I'm going to show you two examples of what that very small timescale is.
And so that is expressed in terms of physical phenomenon, two different timescales.
we're presenting maybe two different physical phenomenon that you have to simulate at the same time.
In terms of mathematics, where you write down OD, when you write an Ordinary Differential equation, you already kind of abstracted from the physics.
So in that case, how do you interpret [INAUDIBLE]?
OK. We can say that when you transform a physical phenomenon into an ordinary differential equation, the timescale is really translated into eigenvalues.
And we are also going to see examples.
So if you have two timescales, one slow, one fast, we are going to see two eigenvalues where you write down the equation in its matrix form and [INAUDIBLE] do eigenvalue analysis of the matrix.
So we're going to see one large eigenvalue, one small eigenvalue.
And the fast process responds to the larger one or the smaller one, what do you think?
Large?
The large, why?
Because it's [INAUDIBLE]
Exactly.
So the large eigenvalue corresponds to the fast process.
So if you think of dudt equal to lambda u, what unit does lambda have?
That's another way to think about it.
What unit does lambda have?
In engineering, one of the most important things is to look at the units of everything, right?
Huh?
[INAUDIBLE]
Yes.
The unit of lambda is 1 over the unit [INAUDIBLE].
It's 1 over [INAUDIBLE].
Lambda really has a unit of frequency or rate, right?
Actually, lambda is a-- imaginary lambda it is exactly the [INAUDIBLE] frequency of the solution, right?
So lambda has the unit of frequency.
A very fast process is going to have a high frequency, therefore a large eigenvalue.
And a very slow process is going to have a low frequency, and therefore a small eigenvalue.
All right, does it make sense?
So stiffness can either be interpreted in terms of order of magnitude difference in timescales or order of magnitude difference in eigenvalues.
OK. You also read about Newton-Raphson.
And Newton-Raphson is the matter of solving nonlinear equations using increasing schemes.
So the [INAUDIBLE] is, as you're going to see, is going to benefit a lot from using an implicit scheme instead of explicit scheme.
And when you have a nonlinear equation and you want to use an explicit scheme, as you dismay discover when you are doing the homework question, at every time stamp you need to solve a nonlinear equation, a nonlinear algebraic equation.
In the homework, the equation happens to be a quadratic equation, and you can solve it using [INAUDIBLE].
But in general, that nonlinear equation you have solved every single time stamp is not going to happen to be an equation with an analytical solution.
And a Newton-Raphson is a method people have invented centuries ago to solve such nonlinear equations numerically.
All right, any questions before we dive into stiffness and using Raphson?
All right.
Stiffness.
The first [INAUDIBLE] when we illustrated stiffness is this guy.
So we did in the first lecture, [INAUDIBLE].
Right?
And this is a very, very similar set up.
And to remind you, we have the same [INAUDIBLE] over here.
But instead of an aluminum cube, we have two metal plates sticked together like [INAUDIBLE].
So this blue [INAUDIBLE] is very [INAUDIBLE].
And one is copper, the other is aluminum.
They are 2 inch by 2 inch big.
And each only is a [INAUDIBLE], so one aluminum, one copper.
And I'm going to heat only this aluminum with a heat lamp.
So I'm going to turn on the heat lamp.
And there are two thermal [INAUDIBLE] that matches the [INAUDIBLE].
What differential equation would you use to model this problem?
When I'm heating only the aluminum, how would the [INAUDIBLE] of this and this going to behave?
How would that be [INAUDIBLE]?
You can discuss in pairs or in groups of three.
And we'll see really answer the question of if we want to predict it, is it a stiff problem?
If so, why?
If not, why not?
You can use a computer to look up any concepts or anything like that.
You can see the two colors, one is bronze-- sorry, sorry, one is copper.
One is a copper temperature.
One is a aluminum temperature.
So we have some very good conclusions here.
Why don't you just help us understand [INAUDIBLE]?
So this is a stuff problem, because there are two different things that are happening.
We have the convection, which we [INAUDIBLE] the heat [INAUDIBLE] and the metal and then the metal in the air, and then the conduction between the two metals.
The metals are both very conductive.
So that process happens much faster than the rate of convection
All right.
Does that make sense?
All right, thank you.
So we have to physical phenomenon, convection and conduction.
And if you'd think carefully, the two metals I exposed over here, within like the common metals, are the most heat conductive ones that you can reasonably [INAUDIBLE], aluminum and copper.
So I deliberately made the rate of conduction to be very fast.
I deliberately made the conduction to be very fast.
And I also made them very thick.
So if you look at-- let me start writing some equations.
OK.
So if you look at the conduction, rate of conduction, the rate of conduction, heat conduction, is equal to what?
Can somebody tell me?
There is definitely a k somewhere, the conductivity times the area times delta t over delta-- in this [INAUDIBLE].
Let's say the direction across the plates are x.
So it is delta t over delta x. OK. And changing the Q-dot is also equal to-- actually, I shouldn't say this.
I should say the mass of each plate times the thermal capacitance, mc, times the partial T, partial T. So this is the rate of temperature change is also equal to Q-dot convection minus this is the conduction, Q conduction.
So this is for the aluminum plate.
And if you just look at if, let's say, we isolate these two phenomenons if we only look at the conduction, what we get is partial T partial T equal to kA divided by mc delta T over delta x.
So remember our eigenvalue analysis?
This part would become our eigenvalue.
If we already have the conduction process, then this would be our lambda.
OK.
I think I get a minus sign here.
So this will be our lambda.
So we have a large k. OK We have our large A.
But have a large [INAUDIBLE] compared to-- we have a large A compared to our delta x, two [INAUDIBLE].
So we have deliberately made these to be large.
And if you plug in the numbers, this is significantly larger, about one to two orders of magnitude larger than the forced convection rate [INAUDIBLE] table.
So this is a stiff process in terms of physics, because we have a very fast speed conduction and a relatively slow speed convection.
In terms of mathematics, let me say this is the aluminum.
This is delta squared delta T is equal to the T aluminum minus T copper.
So we have m aluminum c of aluminum equal to this.
And we have aluminum equal to, let me say-- and let's set this k to be the average rate of conductivity in the aluminum and the copper.
So we replaced our delta T by TA minus Tc.
All right.
So this is conduction.
And we also have convection.
And the rate of convection, can somebody tell me what the rate of convection is?
We have the convection coefficient times A, area-- sorry-- times the difference between the T error.
So let me just say to [INAUDIBLE] T error minus T aluminum and divided by aluminum mass and the heat capacitance.
And our copper is going to satisfy a similar equation, where the heat conduction is the same.
So what this is c instead.
And we have TA minus TC here divided by delta x.
And here we have a removal of heat.
So this is hot air.
And we have a removal of heat.
Cold air divided by Mc Cc.
OK.
So as you're going to see, we are going to get a 2 by 2 matrix.
We are going to get a 2 by 2 matrix.
And the 2 by 2 matrix is going to have the 2 by 2 matrix.
We write down the T of the aluminum temperature and copper temperature is going to be equal to if you regroup the terms.
And we know that this term is going to be large.
So let me call this big A.
And these terms are going to be small.
Let me color these small terms by blue.
OK.
So this term is going to be small.
This term is going to be small.
And let me color the last terms in red.
So this term is going to be large.
So let me just write qualitatively here.
We have a minus large term over here and a minus small term minus a small term in blue.
So this is a multiplied by Ta and the Tc.
So on top of Ta, we have a negative large term and negative small term.
On top of Tc here, we have a plus large term.
And in the second equation, the evolution of the copper temperature, we have a large term in front of here and nothing in the small term.
And in terms of the copper temperature, we have a minus large term and a minus small term.
So the matrix looks like this.
That's how the matrix would look like when you write these two equations in terms of matrix form.
Now, if you do the eigenvalue analysis of a matrix that looks like this, you can take the-- what happens to the eigenvalue if S infinitesimally small compared to L?
Let's say what is the eigenvalue of minus L, minus L, L and L?
[INAUDIBLE]
Yes.
Or it is L, because eigenvalues are linear.
It's an L times the eigenvalue of this matrix.
And what is the eigenvalue of that matrix?
We can try to--
Well, at least one of them [INAUDIBLE]
OK. At least one of them are 0.
Because this matrix is [INAUDIBLE], right?
The first line is negative of the second line.
What about the other one?
0 minus 2
0 minus 2 exactly.
Wait, is it minus 2?
Yes, it is minus 2.
OK.
So it is 0 or minus 2, right?
Minus 2L.
So you can see the eigenvalues, one of them is 0.
The other is minus 2.
And if you add these small terms the eigenvalue-- because the small items are just minus S on the diagonal, right?
So how do they affect the eigenvalues?
It's a test to your linear algebra
[INAUDIBLE]
It is going to be put a minus S on both of the eigenvalues.
It is the eigenvalue of an identity matrix times S times minus S. The eigenvalues of identity matrix is 1, right?
So let me see-- [INAUDIBLE].
I'm not sure.
Let me take it back.
I'm not sure exactly what the eigenvalues are.
OK.
So when I'm adding a [INAUDIBLE] matrix, I'm basically perturbing this by minus S. So this is going to be my eigenvalues.
And because I have a small eigenvalue, I have a large eigenvalue, I would get a [INAUDIBLE].
So basically, we are looking at the example of a [INAUDIBLE] problem.
We can analyze from the perspective of physics.
We have a fast conduction of slow convection.
And if we put these physically into equations and analyze the eigenvalues, we can get a slow eigenvalue and a fast eigenvalue.
The orders of magnitude difference in eigenvalues is also going to indicate the system is going to be stiff.
Questions on this?
Maybe you don't quite get through the mathematics, but it is going to be leading to the same conclusion
Does the name derive from [INAUDIBLE]?
Good question.
Does the name derive from stiff springs?
So let me actually give you-- the answer is yes.
The name does derive from stiff springs.
And it's actually going to be a homework question.
But like I can kind of hint you a little bit right now.
So what if you have a system like this?
What if you have a system like this?
Instead of a thermal problem, let's think of a pendulum.
So a pendulum is we have a mass over here, right?
And a regular pendulum would have an inelastic string attached to the mass.
But what if you think of you have a spring here, so that the distance between the hinge and the mass can actually become longer or shorter.
So if you have a system like this, in what condition-- so let's say the stiffness of the spring is K. Would you have a stiff system in terms of ODEs?
Or you have a non-stiff system?
In what condition would you have a stiff system?
In what condition would you have a non-stiff system
[INAUDIBLE] stiffness [INAUDIBLE]
It depends on the stiffness of the spring, exactly.
We have two physical processes over here.
One is the motion of the pendulum driven by gravity.
The other is the oscillation of the spring.
If the spring is very stiff, then the frequency of the oscillation is going to be very high, which means the timescale of the oscillation is going to be much faster than the motion of the pendulum due to gravity.
And then we get two very different timescales.
Therefore, we get a stiff system.
So that is an example of, like, how does the name stiff really comes into the picture of all this if you have, let's say, a large structure where you are interested in the motion of the whole.
But inside the structure, there are some very stiff components, I mean, really stiff in terms of physically stiff components.
When you derive the whole ODE, you discover two timescales.
One is the timescale you're interested in.
But there is a much shorter timescale, much higher frequency timescale due to the stiff component inside the system.
And therefore, you get a very difficult problem to solve.
And [INAUDIBLE] this problem.
Does that make sense?
Yeah
[INAUDIBLE]
Yeah.
I guess my interpretation is that it's stiff, because of the stiffness of the spring.
I mean, yeah, it's hard to solve.
But it's because of the stiffness of the system it is hard to solve.
It's like hard to solve is a result of it being stiff.
Any other questions?
[INAUDIBLE]
You got it?
Yup
OK.
So before I give you another example of this system, let's talk about how to solve it.
OK. As you read in your reading, in order to solve a stiff system, you really need implicit methods.
Why is that the case?
Because in explicit method, it is only stable for a limited region of lambda delta t. And if there is some phenomenon in your system that is very fast, much, much faster than the phenomenons you want to resolve-- OK, for example, in this case, we may want to resolve the scale of how the temperature rise and how the temperature [INAUDIBLE].
That is the scale of the convection.
But in order for you to solve it, you have to resolve-- one of your lambdas is the lambda corresponding to a much, much faster process.
What does this mean in terms of the plot?
It means that if you have the stability region like this-- so this is the peanut shape of the active [INAUDIBLE], that's one of the best explicit schemes out there.
But if you have a lambda that is a thousand times the process, the frequency of the process you are interested in, that means you have to have a delta t that is a thousand times smaller than you would want to use.
Because if you want to really resolve the process you want to resolve very well, you kind of only need a lambda times the smaller lambda times delta t to be in a reasonable range.
But that wouldn't be stable, because of the existence of a very large eigenvalue.
So in order to make all the lambda delta t's to be within the stability region-- and remember, you have to make all the lambda delta t's within the stability region.
Otherwise, you're scheme is going to be not eigenvalue stable.
So in order for you to do that, you would have to have a delta t that is some times ridiculously small.
OK.
So that is explicit.
And for explicit schemes, all of the schemes only have a limited stability region.
But if you have an implicit scheme, the story's much different.
You typically have a stability region that looks like this.
It is stable for an infinitely large region.
OK. You can easily make this scheme to be a stable for an infinitely region, so that even if some of the lambda are very large, you can still have a stable scheme.
You can still use a delta t as large as the timescale you are interested in.
Does it make sense?
Now, implicit method has a lot of benefits.
But it also have some disadvantage, especially for [INAUDIBLE] your system.
As you saw, [INAUDIBLE] system each step involves solving a nonlinear equation.
So for example, I'm using the most simple scheme backward [INAUDIBLE].
In backward [INAUDIBLE], the difference between V n plus 1 Vn minus over delta t is equal to the right-hand side of V n plus 1 instead Vn.
If it's Vn, then it's forward [INAUDIBLE].
But if it's Vn plus 1, it's backward [INAUDIBLE].
It looks like a very innocent difference.
But if f is nonlinear, this is hell a lot different.
Because the unknowns are also within here.
So this is unknown.
V n plus 1 is unknown.
Only Vn is known.
If this is Vn, I can evaluate f of Vn.
But if this is V n plus 1, I don't know what V n plus 1 is.
So I cannot evaluate this.
I have to solve the whole thing as a nonlinear equation.
OK. On the other the day, we said if f is a linear equation, if f is linear, we can express this in a matrix form and rearrange the equation and compute it by solving a linear equation.
But if f is nonlinear, how do you do it?
Of course the answer is Newton-Raphson.
But to understand what Newton-Raphson does, I think it benefits first to review what we would do in the linear case.
In linear case, f of V n plus 1 can be represented using a matrix times the n plus 1.
OK. Then what we can do is the following.
We can move all the known terms on the left-hand side, I over delta t minus A times V n plus 1 equals to the-- this is unknown equal to the knowns, which is Vn divided by delta t. So we are moving [INAUDIBLE] this is [INAUDIBLE] and moving this term, which is now here, onto the left-hand side.
Once we arrive at a matrix form, we can use [INAUDIBLE] matrix and solve for the Vn plus 1 right?
Now, let's dive into the nonlinear case.
OK. We don't know how to solve the nonlinear equation.
So why don't we approximate it using a linear equation?
Why don't we approximate the nonlinear term f of V n plus 1.
Try something linear.
How do you do that?
[INAUDIBLE]
Small perturbation Taylor series.
This is another use of Taylor series completely different from how we used it in discretizing ordinary differential equations.
We linearize f of V n plus 1 at some kind of initial guess.
So we first guess what V n plus 1 is.
And usually, a good first initial guess is just Vn.
Because if I know I'm taking small time steps, then I know the next time step I have a V n plus 1 that is really similar to what Vn is.
So I'm going to set f of Vn plus 1 equal to Vn equal to f of Vn plus delta V where delta V is V n plus 1 minus Vn times the derivative of V. So instead of writing it like this, I'm going to write a matrix, partial f partial V at Vn times delta V. So I'm going to write it like this.
Now, in this Taylor series expansion, I only can write a problem.
Because I'm ignoring the [INAUDIBLE].
Now, in this Taylor series expansion, what is known?
What is unknown?
This is known, right?
And the unknown-- well, the unknown is only this.
The matrix is also known.
So somehow we reduced the nonlinear equation into a linear question.
Because all the terms that involves the unknown is linear, right?
That's write this down and convince ourselves.
Let's plug this into the original equation
[INAUDIBLE]
Yes
[INAUDIBLE]
So half of [INAUDIBLE] is unknown.
But once we [INAUDIBLE] like this, we found the only unknown in terms of a linear term is [INAUDIBLE].
So let's do this.
Let's plug into the scheme.
What we found is that so Vn plus 1 minus Vn is my delta v, right?
My delta V is unknown.
Delta V divide by delta t is equal to this.
So the first term is known.
And the Jacobi is known.
The Jacobi is the derivative of f to V. And my delta V is over here.
OK. Now, I can solve this equation using exactly the same way which I solved the linear equation.
I can say I, which is identity over delta t minus my Jacobi, partial f partial V at Vn times my delta V is equal to f of Vn.
OK. Now, I can solve for the delta V. And I'm going to say my f Vn plus 1 [INAUDIBLE], which is my guess, is equal to Vn plus this quantity I just solved for, delta V. This Vn plus 1 [INAUDIBLE] hopefully is a better approximation to V n plus 1 than my initial guess Vn.
But how can I be sure?
How can I be certain it is actually better?
[INAUDIBLE]
I calculate the residual, exactly.
Now, I have this quantity.
I can plug this quantity into the equation.
I'm going to say is it equal to f of V n plus 1 tilde.
If this v tilde n plus 1, when I plug it in, is equal to the right-hand side, more approximately than if I plug in Vn as V n tilde plus 1.
Then I know I have a better approximation.
If this is almost exactly equal, then I know I already got a very good answer.
What if I improved, but still didn't have an as accurate answer as I want?
[INAUDIBLE]
Yes.
I keep iterating.
Now, I'm going to define another delta n. So if not good enough, I'm going to say, OK, what I really want is equal to my guess plus my new delta V. This is a new delta V. And do this again.
And do this again.
How do I do this again?
The unknown, which is f Vn plus 1, now is equal to f of v tilde n plus 1.
Now, this is known, because I already computed the better approximation.
Plus the derivative of f to V at this tilde V n plus 1 times my new delta V-- so this is iteration two.
And previously, this is iteration number one.
And here is my iteration number two.
In iteration number two, I constructed a new Taylor series expansion, I constructed a new approximation based on my updated guess.
And then the next thing is the same.
I'm going to plug in this equation into the formula.
And the formula is V n plus 1 minus Vn.
Now, my V n plus 1 is equal to this, right?
So I actually have a V tilde n plus 1 minus Vn plus my delta V divided by delta t is equal to-- oh, let me actually color the terms that is known and unknown.
These are known equal to f of v tilde n plus 1 plus the same thing times the unknown delta V. Sorry, should be blue.
And again, I'm going to move all the known terms into one side and the unknown terms to the other side.
It's equal to f of V tilde n plus 1, which is known, minus delta t of [INAUDIBLE] minus Vn.
I'm going to solve for that delta V again.
Once I solve my delta V, I'm going to update my guess to be my first iteration guess plus the update again.
And then here goes my iteration number three.
I'm going to keep iterating until the residual drops to into a tolerance level that I think is small enough.
So the Newton-Raphson iteration is really another way of using Taylor series analysis by using the fact that we are able to solve these increases scheme equations.
So if we derive a algebraic equation using implicit scheme, we know how to solve that if it's a linear equation.
We don't know how to solve that if it's a nonlinear equation.
But we can approximate it using a linear equation and approximate it again and again.
When I have a better estimate, then the Taylor series approximation is even better.
And that should give me an even a better approximation at the next step.
All right is it clear?
I mean, one of the Newton-Raphson iterations is one of the most useful numerical techniques that really goes well beyond this subject.
It really accomplishes what you can-- if you go to MATLAB, you can use fsolve.
fsolve usually helps you solve nonlinear equations.
But fsolve only works if you have a small amount of equations you want to solve.
When we go to [INAUDIBLE] when we want to solve-- [INAUDIBLE] had [INAUDIBLE] you're going to find out maybe we have millions of equations we want to solve at the same time.
And the fsolve it going to be helpful in the situation.
And in these cases, you really need Newton-Raphson equations [INAUDIBLE].
Any other questions on Newton-Raphson.
Do you guys know how to implement a Newton-Raphson equation?
Sure?
Because I just got some consensus right before class on some of the questions.
When you try to solve-- let me say one word.
When you try to solve this equation, you want to solve a matrix times delta v equal to a residual.
So this is the residual, right?
It is really important to construct the A in the right way.
Because if you don't be careful, you will get A transposed instead of A. OK. And when you solve delta V equal to A inverse times R, it is really important that A inverse is on the left-hand side of R instead of on the right-hand side.
Because in linear algebra, the order really matters.
OK. And how the entries in a matrix A is placed is also important.
Let me make a new page just to make my point.
This I think can save a lot of headaches.
If delta V is arranged at V1, V2, et cetera to Vn and R is arranged as R1 et cetera to Rn, and the A is arranged as the derivative of R1 to V1 et cetera to derivative of R1 to Vn and the derivative of Rn to V1 derivative of Rn to Vn.
It is important you [INAUDIBLE] rather than the transport of the matrix.
Because it's correct Taylor series expansion is this matrix times this vector.
If you do the transpose of this matrix, then you would be modifying the derivative of Rn to [INAUDIBLE] Vn, which is going to be [INAUDIBLE].
And then you do delta V equal to A inverse times R. So just want to make sure you have the right denominator and the [INAUDIBLE] inside the matrix.
Is this clear?
When you do this in MATLAB, it's A backslash R. That gives you A inverse times R. Questions?
No?
OK.
So the rest of the 20 minutes, I want to have some fun.
So I want to kind of have you try in your own computer a real example of a stiff system.
And I have a joystick over here, so you can use.
And if some of you have Simulink installed on your computer, feel free to grab your computer, come to here, and see what is your own implementation of the system.
So this is system is like this.
So let' build a MATLAB flight simulator.
OK. [INAUDIBLE] build a flight simulation in MATLAB.
In class, I won't be able to simulate the whole airplane.
I'm only to be able to simulate the pitch motions, the longitudinal motion of the airplane, so no turning.
All right.
So OK.
So assume the [INAUDIBLE] coefficient is equal to 2 pi times alpha.
So we're assuming we have [INAUDIBLE] air foil-- if you do think [INAUDIBLE] theory, that is exactly what I'm going to get, 2 pi times alpha angle of attack.
And we have the lift and drag to be proportional to Cd and proportional to Cd and Cl.
And the dynamics is that dvdt, the derivative of my velocity, is equal to minus D, the drag times the gravity component in the backwards reaction.
So if the my airplane has a drag, has a pitch of theta, I'm going to have deceleration mg cosine theta.
And the rate of change of theta, which is my pitch, is proportional to the lift minus the gravity component in the cosine direction.
So that is my dynamic.
So see [INAUDIBLE], I have two couple differential equations already where the terms d and l are expressing in terms of these.
Now, the rate of change of alpha, that is the angle of attack, I am going to model this as input minus alpha divided by Tau.
So that is the model of the longitudinal stability.
OK.
So this is the case where I actually have a longitudinal stability.
Otherwise, the sign would be the other way.
So alpha is my input.
I'm going to use the [INAUDIBLE], add my input in.
And alpha is the angle of attack
[INAUDIBLE]
Tau is a value I'm going to set.
Tau is going to be-- so what do you think the value of Tau should be if I have a very stable system, if my Cd is way in front of my aerodynamic sample?
[INAUDIBLE]
So if I have a very stable system, Tau should be very large or very small?
[INAUDIBLE]
Large?
Are you sure?
So what happens if I have a very stable system?
If I pitch the airplane off, is it going to come back fast or come back very--
Fast
Fast, right.
So if I have a very stable system, Tau is going to be very small.
If I have a marginally stable system, Tau is going to be very large.
So Tau has the unit of pi.
It's really kind of how long that it takes for the airplane to cut to static stability.
I have two files.
One is the longitudinal [INAUDIBLE] n. So this function is really [INAUDIBLE] exactly the same differential equation I just showed on the slide.
So I have four variables.
They are the velocity, the pitch angle, the angle of attack, and my altitude.
I have all the [INAUDIBLE] defined here.
I have my Cl equal to 2pi times alpha.
I can do the drag in this.
[INAUDIBLE] And I have the dvdt, the theta dt, the alpha dt, and [INAUDIBLE].
So if you guys implement your own computer using [INAUDIBLE] style, [INAUDIBLE].
And here, I have Tau equal to 0.001.
Does that mean I have a very stable system or a not very stable system?
I got a very stable system, right.
I get a very stable [INAUDIBLE]
[INAUDIBLE]
And if you succeeded [INAUDIBLE] you can [INAUDIBLE] input [INAUDIBLE] alpha [INAUDIBLE].
It's kind of like visualizing [INAUDIBLE].
For example, I can [INAUDIBLE].
It just kind of visualizes [INAUDIBLE].
This is a [INAUDIBLE].
I think I [INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE] your own [INAUDIBLE] on the joystick, you can really control your aircraft.
You can really control your [INAUDIBLE] simulated [INAUDIBLE].
So let's do this.
Just for the rest of the five minutes, I'll just let you have fun for yourself.
And one thing I want to show you is that-- so here, I [INAUDIBLE].
And the way I did is [INAUDIBLE] can save this file.
And the way I [INAUDIBLE] the joystick input is by setting [INAUDIBLE] equal to [INAUDIBLE].
That actually gives you a joystick.
It's like a handle.
It gives you a joystick handle.
And in [INAUDIBLE] from the dropbox, what I did is the input equal to axis.
So I did the input [INAUDIBLE] to axis [INAUDIBLE] 2.
So this is the axis when I [INAUDIBLE].
And the [INAUDIBLE] is the third axis.
And it happens to be this one.
The first axis I think we [INAUDIBLE].
If you guys have time, that'd help me explain this to you something more like real simulator, that's be great.
OK.
This is [INAUDIBLE].
[INAUDIBLE] my view which is the [INAUDIBLE] of the velocity, each angle, angle of attack, and altitude is equal to the last variable plus delta t times the function [INAUDIBLE].
And I've [INAUDIBLE] and did the display.
The [INAUDIBLE] is asking [INAUDIBLE] the whole thing.
But [INAUDIBLE] that [INAUDIBLE] very much depends on this thing, very much depend on my Tau.
So right now, my Tau is plus 1.
It's stable, but not that stable.
If I [INAUDIBLE] it to 0.001, it's going to be a more stable system.
But what does that mean?
That means it's [INAUDIBLE], right?
It has a smaller timescale that is much smaller than the one I'm interested in, which is the [INAUDIBLE] the longitudinal motion of aircraft.
So if I set to be that value in MATLAB for [INAUDIBLE].
If I want to simulate [INAUDIBLE], what happens is my altitude is 10 to the 16.
And basically, the airplane goes unstable.
Not unstable physically, but unstable numerically.
It's not the airplane goes unstable.
But it's my [INAUDIBLE] integrator goes unstable.
It is particularly unstable in the pitch direction.
Because in the pitch dimension, i have a timescale that is much, much smaller.
So if you guys are interested, take the [INAUDIBLE] I have and [INAUDIBLE] integrate on it using Newton-Raphson.
That way if somebody is able to do that and bring the demo next class, that's fantastic.
And that also saves you the homework this week.
Because the homework this week is going to be about that.
So if you do that before next class, you can demo in class, and you already solved a big part of this week's homework.
All right.
OK. And what you [INAUDIBLE] is already in the [INAUDIBLE].
So I will see you all Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Any questions so far?
Especially if you have any additional questions on the ODE part, if anything is not clear I'd like to have them answered before we move on to PDEs.
And you're going to see other things we do on PDE really depend on ODEs or prerequisite ODEs
[INAUDIBLE]
Oh yeah.
Good idea.
Let me open it using my Chrome.
Because the [INAUDIBLE] didn't work.
OK. OK.
These are the measurable outcomes.
These also updated.
So you have the ODEs.
Now you scroll down.
You have the Product Differentiated Equations.
So we can look at all the measurable outcomes again.
And for each of them, you can click on them.
And it'll show you all the sections that you can use by yourself on how to navigate through the materials and make sure you have learned what we want you to learn.
All right.
We have an identifier with the PDE using [INAUDIBLE] law.
And that is that one of the things we are going to be talking about today.
Qualitatively describe the solution of simple ODEs.
And that is also something that we are going to be describing today.
But you can read all of these also in the readings.
OK. Now let's dive into PDEs.
OK. PDEs versus ODEs.
When we solve for ODEs, we are looking at solution, we are discretizing a function of time.
Right?
So this is when we solve ODEs and when we want to aproximate-- The difference between an ODE and [INAUDIBLE] equation is because it compares a differential operator d/dt.
And a lot of things we'll learn is how to approximate this d/dt and how we use the approximation to solve the ODE.
Now once we go to PDEs, there are two cases.
One case is now I have a U function of both space and time.
So we have space and time.
That gives me a PDE.
And now when we take derivatives, there are two derivatives we can take.
We can take the positive derivative of U with respect to space.
And the reason it is a positive derivative is because when you fix all the other independent variables-- so it is a derivative to x while holding time fixed.
Or you can take partial derivatives to t. That is, taking derivatives to time while fixing the spatial location.
This is very important because we are going to see when we also can take derivatives of this function of space and time, [INAUDIBLE] neither hold space nor time fixed.
So we can also take something like a directional derivative when we are both moving in space and time.
And that is something we are going to see in the characteristics.
We can take the derivative in interaction and call it a characteristic reaction.
And that is going to give us some interesting results.
Another possible case for PDE is when there are multiple spatial locations, we can have U of x and y or U of x, y, and z.
And this is also a PDE because we can take the derivative to x.
We can take-- and of course, in most cases you can have a function of x, y, and z.
And t. That gives you a four-dimensional solution you can solve numerically.
And a lot of the simulations, the aerodynamics simulations if you want to simulate the flow around an airplane, it's actually the unsteady flow air on an airplane, you do have to look at this case.
You have three spatial dimensions and time.
And you need to solve partial differential equations involving these.
And the mass per volume is what?
It's density, right?
The unit amount of momentum per volume is momentum density.
The unit amount of energy per volume is energy density.
So these are the quantities we solve.
When these quantities can be spatially dependent-- the density of air, for example, in a compatible flow can be different from one spatial location to another.
In an unsteady compressible flow, it can be different at different times.
So this kind of view is going to be a function of space and time.
OK.
So how do we express this kind of conservation law in terms of partial differential equations?
First of all, what does it mean by, let's say, mass being conserved?
So let's take a special case where U is the density.
So let's say the unit amount of mass, the volume.
So can somebody tell me what does it mean by mass being conserved in the mathematical sense?
[INAUDIBLE]
OK.
So yeah.
That's a very good answer.
So first of all, you are choosing a controlled volume, right?
So first of all, you are choosing a control volume.
Let's say do my control volume, let's actually do the control volume like this.
It can be an arbitrary control volume.
Let's call it omega.
So mass being conserved means the amount of mass inside the control volume is not going to magically appear or magically disappear.
The increase has to be coming from outside.
And the decrease has to go outside.
Right?
That means-- And also let me express what the mass is.
The mass is the integral of the density integrated over the volume.
Let's express it as this integral so that it makes more sense.
Now this is the mass inside the control volume at a certain time.
The time derivative of this mass which represents in a very small amount of time how much mass increased or how much mass has decreased inside the control volume.
It has to be equal to the rate of mass coming into the volume minus the rate of mass going out of the volume.
Right?
And the mass going in and out can only happen on the surface of the control volume, on a boundary of the control volume.
And let me write like this.
Plus the rate over the surface of control volume.
Let me call it partial omega.
This is a notation of the boundary of the control volume omega.
It has to be equal to an integral at the boundary of the control volume times the flux of row.
Let me call it f of row d S. That will be equal to zero.
OK. Let me expand this a little bit more.
So this [INAUDIBLE].
It represents for a unit amount of time at a unit surface.
How much mass has gone through that surface?
It is directional because there is a direction of the flow of mass, right?
For example, if you know it's a fluid flow, then in fluid flow f of row is going to be equal to row times the local velocity, right?
So if you have a velocity vector and you know the density, then the unit among the mass going through a surface is equal to the row U times the area of the surface normal to the velocity.
Right?
So that basically is what we mean by a quantity being conserved.
All Right.
Now this is not a differential equation, right?
Anybody have a question?
Question on this conservation law?
OK.
This is not a differential question yet.
Right?
And honestly I don't really know how to solve it.
So how do we convert this into a differential equation that we can solve?
[INAUDIBLE]
Good point.
We can shrink this omega to something very small.
And this is exactly what we are going to see in the finite volume approach.
So in the finite volume approach, we are actually going to shrink this omega to something very small and use this form to solve this equation.
Another method we're going to discuss is finite difference.
So we're going to be teaching three methods.
Finite difference, finite volume, finite element.
So let's leave finite element to the last because it's the most advanced method.
Finite difference.
I'm trying to avoid, we're going to be discussing first.
And finite volume is exactly taking that equation and shrinking omega to very small.
The finite difference approach, on the other hand, has to first convert this equation into a differential equation.
In finite difference, we do not allow any integrals.
We have to express everything through derivatives.
Now how to do that?
[INAUDIBLE] theorem or divergent theorem, right?
With divergent theorem, we can convert the surface integral into a volume integral inside omega of the divergence of the flux.
Right?
So the conservation equation becomes like this, it becomes-- and we can also take the timed derivative inside of the fixed control volume.
So we have integral over a really arbitrary domain, omega.
Because this conservation law has to satisfy for any control volume.
So this is the time derivative then.
And we copy the divergence term into here.
The x has to be zero.
And this has to be zero for any control volume.
For any omega.
That means what is inside the integral has to be zero.
Because if the integral is non-zero anywhere, then we can make a very small control volume around it.
And that integral is not going to be zero.
So the integral has to be zero everywhere.
Now this is a differential equation, right?
This is a differential equation.
And it is a partial differential equation because we not only have derivatives in time, we also have divergence.
And divergence is a derivative in space.
So this is my PDE in the differential form.
Right.
This is my PDE in the differential form.
And that is what we have over here, differential consideration law of U.
So a differential conservation law of U.
In general, we have derived the conservation of mass, right?
In general, the differential form of the conversion law is dU/dt plus the divergence of some flux which is a function of U is equal to a fourth term.
And the fourth term is going to be non-zero for things like, for example, if you have energy-- if you have momentum-- for example, if you use momentum, momentum density, then the fourth term will be external force like gravity.
Right?
So you can also have a [INAUDIBLE].
So when we talk about a differential form of a conservation law, we are talking about equation [INAUDIBLE] like this.
All right.
Let's take a look at a few examples.
So the first example is what we do a lot of research on.
The conservation of mass, momentum, and energy with flows.
And by solving differential equations, that governs the [INAUDIBLE] conservation of three things-- mass, momentum, and energy-- we can really explain a lot of phenomenon that we are interested in in aerospace engineering.
Like the flow around airplanes, rockets, and missiles.
And also flow inside [INAUDIBLE] engines.
We can all solve these problem by solving differential equations that just governs the conservation of mass, momentum, and energy.
All right.
So this is one of the prime examples of conservation loss expressing partial differential equations.
Another example of conservation law is wave propagation.
So this is like surface waves.
And many of the simulations you can see actually inside [INAUDIBLE] where we are solving partial differential equations that govern really the conservation of mass and momentum of water, that is under the force of gravity.
Another type of conservation law is what you find when you open a website and look at the weather forecast.
So how do people predict the weather?
They solve differential equations.
They solve differential equations that govern the conservation of mass and momentum, weather molecules, energy.
Of course, you have [INAUDIBLE] from the sun.
And you have things from the ocean, things like that.
But essentially it is a differential equation governing the conservation of stuff that happens on the Earth's atmosphere.
Of course, it's a very large scale PDE.
But you look at the weather forecast because people are able to solve PDEs.
So this is what we are going to be looking at in the PDE section of this class.
We look at how to numerically solve partial differential equations.
OK.
So first of all, we are going to be looking at some examples of conservation laws.
And these examples are the very elementary type of conservation loss that a lot of them, you should be able to qualitatively describe the solution, how the behaves.
But really in complex differential equations like this, it's really the combination of multiple of these elementary conservation laws.
And if you are able to solve each of these elementary conservation laws and you know how to combine them together and how to do that in complex geometry, we can really solve these very complex differential equations.
So the first conservation law is advection.
Vachon And also many people in this field called convection.
So convection is a physical phenomenon that is usually created by the bulk movement of molecules inside some fluid.
Like, for example, the generation of the hurricane is really a convective phenomenon.
So let's look at-- at the most elementary form-- what convection is like.
So let me go over here.
I have some MATLAB [INAUDIBLE].
OK. And I want you to look at the first code there.
But I'm going to give the source code to you via Dropbox.
So here, what I'm doing is-- OK.
I'm opening in MATLAB a window that has nothing in it.
I like some of you to come up and design what's called an initial condition for this [INAUDIBLE].
All right?
And let me first describe what the advection equation does.
The advection equation describes the evolution of a concentration.
Let's say mass density of a certain species.
Density of, let's say, a pollutant or [INAUDIBLE].
Then how does the density evolve under the bulk movement of a fluid?
So convection, that is one field we are going to be going over here, describes the initial distribution of that species.
Or the initial density of the species.
And after [INAUDIBLE] is in a position, we are all going to be observing how does the distribution go under the bulk movement of some fluid?
There's somebody here [INAUDIBLE].
Somebody is speaking at the back?
[INAUDIBLE]
Come on.
[LAUGHTER] OK. Somebody-- [INTERPOSING VOICES]
[INAUDIBLE], people
[INAUDIBLE]
OK.
Thank you
[INAUDIBLE]
Come up and draw it
[INAUDIBLE]
You want to move [INAUDIBLE] starting from the left to the right

And it's going to be-- try to make it periodic.
[INAUDIBLE] on the screen
[INAUDIBLE].
[LAUGHTER]
You want it to be periodic?
Yeah

Going from the left to right, we want [INAUDIBLE].
Yeah
OK. [INTERPOSING VOICES]
Let me try it again.
Let me try it again
Is it going to freak out if it's negative?
No, no.
It's not going to freak out.
This [INAUDIBLE] with most [INAUDIBLE]

Good
That wasn't--
Keep going.
Keep going.
Keep going.
Keep going
It's not periodic
It's fine, it's fine

All right.
OK. We actually have two windows.
That's why it's-- so let's look at the right window first.
Thank you.
OK. Let's look at this window first.
So this is the evolution of the equation after a certain time, right?
It's accelerating, right?
This is the solution to the partial differential equation at a certain point.
So this is the spatial coordinate x.
And the time is not typical.
This is the [INAUDIBLE].
And this [INAUDIBLE] over here.
[INAUDIBLE] is how the solution evolved.
What is this solution doing?
Yeah?
It's moving all the [INAUDIBLE]
OK.
The answer is-- the solution, kind of, moves while also [INAUDIBLE].
But here the template [INAUDIBLE]
[INAUDIBLE]
Yes.
Good observation.
The amplitude has to be the same.
Right?
0.6 minus [INAUDIBLE].
If x equals stays the same and the whole thing just moves towards the right at a fixed velocity.
So this is a solution to the pure advection equation.
And it is the solution that keeps moving towards the right without diffusing.
So the question is partial U-- let me call it small u-- partial small u, partial t plus a big U partial U-partial x equal to zero.
Here there is a big distinction between the small u which is a function of space and time, is the unknown which is also what we plotted.
And the big U is neither function of space nor time.
It's a constant.
OK?
So big U is a constant advective velocity.
And small u is the unknown.
So two things.
One is, is this equation a conservation law?
Do you think, just from guessing from the solution you saw, is that a conservation law?
Yes.
Because it looks like the density or the solution is conserved, right?
It just moves around.
Whenever that goes into-- if you look at a control volume, that's because [INAUDIBLE].
Whatever that [INAUDIBLE] right?
So it is a conservation law.
Mathematically, how does that come into a conversation law?
Mathematically if I write the equation a little bit differently, if I write the equation differently-- partial u-partial t plus partial F of u partial x is equal to zero where the F of u, the flux of u is just equal to big U times small u.
Right?
Does that look more like a conversation law?
Why?
Because we already mentioned that derivative in x is-- Yeah.
This is what the divergence is having a multiple dimension, right?
The divergence of [INAUDIBLE] is e x component of the method dx plus the y component of the method dy.
And in one dimension there is no dy.
So it would be dx is the divergence.
That's what happens in one dimension.
The divergence is [INAUDIBLE].
And now the use of flux.
The flux is equal to big U times small u.
That makes sense, right?
Because if the small u is the solution [INAUDIBLE], then the [INAUDIBLE] of the density should not be the velocity times the local density.
Right?
The higher the velocity is, the faster the mass goes through [INAUDIBLE].
The higher the density is, also the faster the mass goes through [INAUDIBLE].
The flux indicated is velocity times the [INAUDIBLE].
Right?
OK. Now there is a conservation law, right?
It is a conservation law.
And it seems it has an analytic solution of everything that's moving towards the right.
Is that true?
Yes.
We do have an analytic solution.
And the analytic solution looks like this.
Let's take a look at this again.
So this is the solution at a particular time-- in this case, it is at time 0.5.
But if you don't know the solution at a certain time, the entire solution as a function of space and time, it looks like this.
It looks like this.
Now over here I already have this-- and [INAUDIBLE].
The solution as [INAUDIBLE].
But other solutions [INAUDIBLE].
OK. Blue is negative value and red is positive value.
[INTERPOSING VOICES]
Here you can see that at time equal to zero, this is [INAUDIBLE].
This is the initial condition.
As time increases, let's say, at 0.1, the peak has gone from here instead of here at the initial condition.
At the initial condition, the peak is here.
The bottom is here.
[INAUDIBLE] the peak has moved towards the right.
The bottom has moved towards the right.
Everything has moved towards the right.
And it goes over the [INAUDIBLE].
And we're going to discuss later, it depends on the conditions [INAUDIBLE].
And keep moving towards the right.
All right.
And you'll see the lines over here, the lines which really is the contour of the solution for space and time-- these lines are called the characteristic lines in the solution of a partial differential equation.
OK.
So why are they important?
Why does it have a name?
Do we see something particular about these lines?
[INAUDIBLE]
Huh?
They all have the same flow or-- first of all, they are straight, right?
We all see the contour of the random function of space and time, you would expect the [INAUDIBLE] to be a curve.
Right?
But these contours are straight.
And we're going to see later on that [INAUDIBLE] another equation we happen to be looking at.
The second is, the solution is [INAUDIBLE] along these lines.
And these are really the characteristics of characteristic lines.
And in this case, we can analytically derive what they are because there is an analytical solution of this differential equation.
U of x and t is just equal to whatever initial condition is-- let me call u naught the initial condition-- x minus U times t. OK. What does this mean?
It means-- remember u zero of x minus Ut.
It means at t little zero, it makes sense that at t little zero, my solution U of x, t-- that is [INAUDIBLE].
That is my initial condition.
When t increases, the solution at a particular x is the initial condition at a smaller x.
Let's assume U is positive.
If U is positive, then at a positive t, my solution at a certain x is equal to the initial condition at a smaller x.
That make sense?
At a particular time, my solution at one x is equal to the initial condition at a smaller x.
And how much smaller it is really equals U times t [INAUDIBLE] is [INAUDIBLE] of how fast these lines go towards the right.
And how fast those lines go towards the right is determined by U.
It's determined by how fast the waves are convecting towards the right.
Any questions on this equation?
So this is good example of a partial difference equation.
Because we know its analytic solution.
And when we use a numerical MATLAB to solve the differential equation, we can compare against this analytic solution.
It is like when we look at all these, we want to try our numerical methods on du/dt for the lambda U.
Not because it is the most useful equation to solve-- I mean, it is useful but the reason we want numerical methods is because we want to apply this to more complex equations-- the reason we want numerical methods for PDEs is because we want to solve free-flow equations.
But in order to have our methods and see how our methods behave, to analyze what method is useful to apply our methods, we're not going to have equations like this.
This one of the elementary types of differential equations-- the convection equation or advection equation.
All right.
And so the characteristic lines are x equals to some x zero plus U t. So this is the characteristic line.
And we are going to also see similar characteristic lines later on.
OK. Let me give you another example of conservation law.
Now this time, the equation we are solving is no longer linear.
It's called the Burgers equation.
It is the equation people use to model fluid flows in the higher fidelity than the advection, than the linear advection equation.
And one of the key features we're going to see in this equation is shock waves.
Shock waves as we see in supersonic [INAUDIBLE] that develop shock waves.
OK.
So the bulk of the equation looks like this.
Partial t plus u partial u-partial x equal to 0.
It looks the same as what we did before.
But there is a one key difference in what we did before.
This U is a constant that has nothing to do with this solution.
In both of the equations, this U was not really a solution.
So the bulk of the equation had a quadratic [INAUDIBLE] because this one is a [INAUDIBLE] which will make U twice as large.
That [INAUDIBLE] is going to be four times as large.
All right?
OK. Is this equation a conservation law at all?
Yes.
Can we write the equation into a form that looks like this?
Remember in order to write in conservation form, we have to have an equation like this.
Right?
Can we write this equation in some form with somehow defined F?
One half of U squared exactly.
OK. Because if you take this and wrap this into the [INAUDIBLE] the derivative of U squared, it is going to be two times U times the u.
And the [INAUDIBLE] is [INAUDIBLE] here.
You get back to the first equation.
OK. Now let's see how this equation behaves differently from the linear advection equation.
Can I get somebody else to draw me another solution?
Thank you
[INAUDIBLE]
Start on the left to right to the [INAUDIBLE] All right.
Thank you.
So let's see how this solution goes.
And [INAUDIBLE]
[INAUDIBLE]
Sorry?
[INAUDIBLE]
What do we see that is different from what we saw in the linear advection equation?
[INAUDIBLE].
[INTERPOSING VOICES]
Huh?
The magnitude
The magnitude is [INAUDIBLE].
But [INAUDIBLE]
[INAUDIBLE]
The bottom is coming up a little bit.
And the top is going down a little bit.
Right?
So there is something that [INAUDIBLE].
Right?
This is one of the things we didn't see in the previous equation.
We have developed a [INAUDIBLE] right?
And [INAUDIBLE] What else do we see?
Does the wave go towards the right at a uniform speed?
[INTERPOSING VOICES]
[INAUDIBLE].
Normally we have something that moves toward the right, we [INAUDIBLE] the solution has been [INAUDIBLE].
OK. And if we go back to this one, looking at the solution of the function of space and time, we can see that we no longer have parallel characteristic lines.
But we still have these characteristic lines.
Right?
And there you will see other things [INAUDIBLE].
This is initially not a shock wave, but now it has become a shock wave.
A shock wave is not [INAUDIBLE].
All the characteristic lines that are not shock waves [INAUDIBLE].
[INAUDIBLE] characteristics are [INAUDIBLE].
All right.
So OK. Let's let the animation go.
And let's take a five minute break.
And after the break, we're going to be looking at a more, we're going to be sort of animating these solutions all ourselves.
And we'll see what we get in terms of the solution.
OK.
So let's-- we have seen some solutions of partial differential equations.
All right.
So right now what I'd like you to do is have everybody come to the front of the classroom.
And we are actually going to be emulating the solution of the differential equation
[INAUDIBLE].
[LAUGHTER] [INTERPOSING VOICES] [INTERPOSING VOICES]
OK. Let's stand so that the two doors are boundaries to this.
So let's [INAUDIBLE]
[INAUDIBLE]
Yes.
Sure.
Can somebody there just push the off button?
All right.
OK.
So let's [INAUDIBLE] ourselves a little bit?
[INAUDIBLE] so that we can go in and out of the door.
[INTERPOSING VOICES]
Let me just have--
[INAUDIBLE]
Just in case
We're worried about [INAUDIBLE]
Yeah.
[LAUGHTER]
Good point.
[INTERPOSING VOICES]
OK.
So let's [INAUDIBLE].
So let me get one student here that's going to be an observer.
OK. [LAUGHTER] OK.
So if you stand over here, you can really [INAUDIBLE].
Let's say the height [INAUDIBLE] is the solution of my PDE.
OK. [LAUGHTER] [INTERPOSING VOICES]
OK.
So I have a distribution like this.
This is my solution.
[INAUDIBLE] space.
Right?
And imagine I have this [INAUDIBLE].
So what I want to do is I want to simulate the solution of an advective differential equation.
So when I say I'm going to the next concept, let's think of-- we won't move all at one time.
Right?
Let's just move our position.
What do you think we should do when we evolve to the next time slot?
[INTERPOSING VOICES]
Move that way.
Somebody has to be emulate the periodic-- [LAUGHTER] When I go, let's go.
Go.
One step.
OK.
Step one step.
Oh it doesn't [INAUDIBLE].
OK. Two steps.
OK.
So let's go there for two more steps.
All right.
[LAUGHTER] OK. Now it's a perfectly good solution because everybody moves there at a constant speed, right?
Let's stop for a moment.
Let's think of emulating the solution of a Burgers equation.
[INTERPOSING VOICES]
Does anybody feel like they're on a [INAUDIBLE]?
[LAUGHTER]
So we haven't talked about diffusion yet.
We're going to be talking about that later.
But in Burgers equation, what happens is that when we look at the characteristic lines-- [INAUDIBLE] actually following the characteristic lines.
Because your height is going to stay constant, right?
As long as [INAUDIBLE] your height is going to stay constant.
Now what is different is that in the purely advective equation, the characteristic lines stay parallel.
That means all of you-- no matter of your height or no matter what solution value you are at-- are going to be moving at the same speed.
Right?
In a Burgers equation, what is difference?
The solution-- remember we have U times du/dt.
And the U in front of the du/dt is no longer constant.
It is actually the solution itself.
So what does it mean?
How fast should you move when you are emulating the [INAUDIBLE] equation?
Four [INAUDIBLE]?
Exactly.
The [INAUDIBLE] are going to be moving.
It's going to be proportional to the solution itself.
So in other words, if you're [INAUDIBLE] for the solution, then the speed you move is going to be actually proportional to how tall you are.
[LAUGHTER] OK.
So let me do this.
When I say-- when I say step, let's just move for a single step.
And let's say the speed, the step size we're going to take, let's say, is-- let's estimate-- is like a third of your height.
Let's try to see if you can--
OK. --estimate how to do that.
And whoever goes out of the door has to come back as a periodic [INAUDIBLE].
[INTERPOSING VOICES]
OK. Everybody ready?
One step.
[INTERPOSING VOICES]
OK. [INAUDIBLE] [LAUGHTER] OK. Another step.
[LAUGHTER]
Um?
Yeah?
Can characteristic lines cross each other?
Or when they're right in front of each other?
OK. We have a good question here.
Can characteristic lines cross each other?
Because look at here.
What happened here?
[LAUGHTER] So what is the physical phenomenon that is happening at this point?
[INTERPOSING VOICES]
OK. Well what physical phenomenon is happening here?
We have a shock wave.
[LAUGHTER]
We have a shock wave.
Right?
We have a shock wave.
Because its characteristics in front is moving slower than the characteristics at the back.
So instead the molecules at the back catch up.
And you get a shock wave forming.
So actually once you have a shock wave forming, [INAUDIBLE] you all should move at the same time.
You also move at the same speed that is equal to the average of your speed.
OK.
So another step.
[INTERPOSING VOICES]
Oh there is another shock wave forming over here.
OK. Another step.
[INTERPOSING VOICES]
OK. Another step.
Another step.
[INTERPOSING VOICES]
All right.
So we have a lot of shock waves, right?
A lot of shock waves.
And we actually get either shock waves or a solution that is very smooth, right?
So we end up getting a solution not unlike what we saw previously on the screen.
So when the characteristic in front moves back up, they spread out.
And the result of the solution becomes smooth.
That is like an anti-shock wave.
It's a verification wave.
It helps when you have supersonic expansion.
[INAUDIBLE] smoother.
On the other hand, if the solution at the back is faster, then it catches up.
And the characteristic lines converse each other.
And [INAUDIBLE] form these continuities which is shock waves.
OK. OK. Do we want to move more?
Or-- [LAUGHTER] Or do we understand how the differential equations behave?
[INTERPOSING VOICES]
OK. Propagate.
Let's go-- [INTERPOSING VOICES]
Traffic jam.
OK. That's fine.
That's good enough.
Let's go back to our seats.
[INTERPOSING VOICES]
And interestingly the bulk of the equation looks something really similar to the Burgers equation.
If we use the model of the flow of cars on highways, or freeways, so imagine the lots of cars-- [INAUDIBLE].
Cars are conserved, right?
Think of a highway that has no entrance and exit.
The The cars are conserved.
No car disappears, no car is generated like spontaneously.
So the number of cars is conserved.
And the flux, which is proportional to the density of the cars and also proportional to the speed of the cars-- and the speed of the cars can usually depend on the density, right?
If you have really a lot of traffic, the cars move slowly.
If you have not a lot of traffic, they will move at slightly faster than the speed limit.
So really the flux depends on density.
So you really have conservation laws.
And you have behavior similar to this.
But in the case of traffic flow, the shock waves actually move backwards.
So think you have a red light, all the cars kind of bumping into each other and stopping.
We have a shock wave propagating backwards.
So this kind of equation is not exactly the Burgers equation but something pretty similar can model the evolution of the density of cars on a highway.
And maybe we can take a look at the Burgers equation again.
And who was the observer?
OK. You observed.
Can you tell us what [INAUDIBLE] what the initial distribution of the height was?
[LAUGHTER] And we can kind of a look at the solution evolution again, just try to recall where you were.
And was the solution kind of reproducing the kind of shock wave [INAUDIBLE] a little bit as well?
OK.
There was a group of tall people over here.
And some shorter ones.
A few tall ones.
[LAUGHTER] A few shorter ones
All right.
[CHATTER]
Yeah.
I see kind of a-- these [INAUDIBLE] performed shock waves really quickly.
Oh, these I don't have [INAUDIBLE].
Sorry.
So there are those people coming to the door.
I think those are [INAUDIBLE].
[LAUGHTER] So yeah.
[INAUDIBLE] There is the replication wave over here.
And you can see the [INAUDIBLE] shock waves already formed over here.
So there are already people [INAUDIBLE].
And here because there is a continuous [INAUDIBLE] over here, you can already see the [INAUDIBLE] are converging.
Initially they just spread out.
Now they are closer with each other.
And [INAUDIBLE] that have already formed.
OK. Let's wait until this shock wave forms [INAUDIBLE]
I think that's [INAUDIBLE]
OK.
So let's see the response and we can move onto the diffusion equations.
Yeah.
The diffusion equation, I think, is much more difficult to act.
Because we can't really diffuse our height into people around you.
[LAUGHTER] So if you think of a way to do that, I'll do it next year.
[LAUGHTER] OK. Now the shock waves are converging each other.
And now these three guys are moving at the same speed as each other.
OK. All right.
And sorry for not having periodic on the conditions.
So now let's look at the Burgers equation.
The Burgers equation has characteristics that are x equal to some x zero plus my U times t. And U is the local solution that stays the same along the characteristic.
OK. Why is it like this?
Because if you look at the derivative of the solution along the characteristic line, so du/dt-- let's look at this.
Let's look at the solution that we have like this, right?
Let's look at the derivative of the solution along this characteristic line.
So we're looking at du/dt.
Notice I'm using the total derivative because I'm looking at the solution along the characteristic which is at spatial location x zero plus Ut and at time t. So I'm allowing my space and time to change together as t increases.
I'm not fixing the space and taking the derivative through time.
I'm taking the derivative through time while the space depends on time as I go along this curve.
So this derivative is like taking the derivative of yourself as you move in, right?
And it's like taking the time derivative of a moving point in the solution.
So now what is it?
Anybody remember the chain rule?
How do you apply the chain rule to expand the total derivative into the partial derivative?
Sorry?
x naught is a constant, yes.
And looking at one particular characteristic.
Hm?
[INAUDIBLE]
Uh huh.
Du/dx?
dx dt.
And here my is x is this.
So dx dt is U, right?
So this is equal to U.
And plus partial u-partial t. Because this is also a function of time.
OK. Now if you look at this, du dt, du dt.
du dx times U. U times du dx.
So this is exactly equal to the left hand side of the partial differential equation.
And according to the partial differential equation, it is equal to zero, right?
What does this mean?
That means the derivative of this solution as we go along the characteristic is equal to zero.
Or the solution doesn't [INAUDIBLE] along the characteristic.
That is exactly what we saw in the behavior of the solution and exactly what we tried to emulate as a group along the classroom.
And the behavior is derived from the analytic form of the differential question.
And you can use the same math here to derive it in the advection equation.
You can use the same math to derive similar things for all non-linear conservation laws, all scalar non-linear conservation laws.
You're going to get-- all of the characteristics are going to be different.
The U is-- and you need to convert the conservation law into something like this.
You have to say-- you have to take this out, which is really partial df du, you have to take this out.
But [INAUDIBLE] as long as you have this df du, df du is the speed of your characteristic line.
And you can derive the same fact that the solution doesn't change along the characteristics.
OK.
So I only have a few minutes.
Let me do another example.
That is, what if we have advection and diffusion at the same time?
So when I have advection and diffusion at the same time, then I need to specify how much convection I have and how much diffusion I have.
So let me start to build a case where I only have-- so let me say, first of all, I only have a case of-- so u is the coefficient on the convection.
This is the coefficient on the-- kappa is the coefficient on the diffusion.
Let's take a large-- just one convection and one diffusion.
OK. Can somebody help me draw initial conditions?
Come on.
We just get less than five minutes left.
All right.
Thanks
Are the [INAUDIBLE] periodic or does it matter?
It doesn't matter.
All right.
Ooh.
OK.
Whether you draw a pretty [INAUDIBLE], then some very [INAUDIBLE].
That is the color of the diffusion.
OK.
So you see that kappa equal to one is very strong diffusion.
OK.
Thank you.
OK. Let me try another one that is-- let me make kappa equal to 0.01.
Can somebody come down to draw the last initial condition before we go home?
Thank you.
All right.
So now because we have a much smaller kappa tendency, the solution is smoothed out.
Right?
The initial discontinuity is down.
And the solution becomes smoother and smoother and smoother.
At the same time, its [INAUDIBLE] was the right periodically.
But because the coefficient is small, the [INAUDIBLE] is pretty apparent.
But the rate is much smaller.
And the main impact is like [INAUDIBLE] and there is a smaller diffusion, but still it is visible.
All right.
So this is the behavior of the equation with [INAUDIBLE] flux.
All right?
Any questions?
No?
All right.
Then I will see you next Monday then.
Or I'll see you on Friday.
I'm going to announce the office hours.
Yeah?
[INTERPOSING VOICES]
[INAUDIBLE]
Yeah.
The project now only has two questions that is relating to ODEs.
And I'm going to add another question that is going to be related to PDEs.
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
It's up.
We'll look here and we'll start.
The first real lecture on how to use numerical methods on partial differential equations.
So just a reminder, of the latter lecture.
We did a-- We looked at partial derivative equations in general, right?
We looked at the behavior of a few simple differential equations, where we know how it behaves.
We looked at [INAUDIBLE] equations, where the solution just keeps moving towards the right.
We looked at a diffusion equation, where the solution [INAUDIBLE] diffuse result.
It becomes that line afterwards.
We also looked at Burgers' equation, which is the equation that can generate shock waves.
Where the characteristics can go together they generate shock waves.
When they become, or when they diverge from each other they expand, or the solution expands.
And the wave will use a little bit of human kind of animation to really animate the solution of the partial differential equation.
So this is a good inclusion on the innovative behavior of the differential equation.
And so try to remember to us, because in this class, we are going to be using finite difference to solve them.
And I'm going to ask you where the solution and body on the screen is correct or not.
I mean, not all of them are going to be giving us the right answer, as you already saw in the audio section.
And some of them are going to give me, in this case, answers that are correct in some parts.
But not really correct in other parts.
And I also ask you, like which part is correct, which part is not correct?
OK, so let's first start with a finite difference approximation.
In this case with partial derivatives.
So let's look at the simple case where my solution U is a function of (x, t).
What are the partial derivatives?
What is a partial derivative?
Du, dx and du, dt.
Right?
Why do we call it a partial derivative?
Because when we are taking derivative of one variable, we are holding the other one fixed.
So in finite difference method, a good way to construct the grid to say is I think it's the first time we look at the grid.
Is to actually construct the grid first that, it is aligned with one of the variables.
So that, if you look at a particular direction on the grid, one of the variables are naturally fixed.
So we use in the last naturally, if you remember, we used these two axes, x and t, to visualize the solution.
Right?
Now let's use the same plot in one direction x, the other direction t, to visualize my mesh.
I'm going to design my grid or mesh to be like this.
I think in finite difference it's more common to use the word grid.
And the way you go to find a volume or finite element, we will use the mesh instead.
And you're going to see why that is the case.
So we have a grid like this.
And you see that I'm describing both space and time separately.
OK?
I'm going to call this-- this x is called a zero.
This is delta x, 2 delta x, 3 delta x, 4 delta x.
Et cetera.
And my timestamp is 0 delta t, 2 delta t, 3 delta t, et cetera.
Right?
So-- I have my solution right here on this grid.
And for example here, my space is equal to-- I'm also going to assign my grid points.
So this is what I call u of 4.
And one, two, three, four, five.
So remember this notation.
My subscript, you know, the spatial grid, and my superscript 5 denotes which timestamp I mark.
And here would be U of 4, 4.
Here would be U of 3, 5, et cetera.
OK.
So now I have the solution.
I have the discretized solution leaving all these grid points.
How do I approximate the spatial derivative of x at some grid point i, and some timestamp n?
Any suggestions based on What you learned being finite in ODE?
Taylor series, yes.
But can somebody give me a suggestion of what should I use?
[INAUDIBLE]
1 delta x further towards the left or right?
[INAUDIBLE]
OK, let's try to use the right.
OK so I can-- nope.
I can approximate the partial derivative by-- in this case, I'm fixing n, right?
In partial derivatives, I fix the other variable.
In this case, I fixed n, which is the same as fixing t. I took the difference in the x direction and the i direction, which is also in the x direction.
So this is an approximation of the derivative in x. I can call this forward in space.
OK.
I can also-- let me see.
I'm going to insert page.
Escape it turns out I want smaller than A4.
Come on.
I think this is good, OK.
I can also do backward in space.
Backward in space, I would get Uin minus Ui-1n over delta x.
So this is backward in space.
When you see it as-- forward order and backward order have different stability properties-- this is also going to have different stability properties.
And I can also do central difference.
So central difference would be U of i plus 1-- that is towards the right-- minus U of i-1-- towards the left-- divided by how much?
2
2 delta x, because that is the space in-between them.
So this is central in space.
How about time?
We also need to discretize time.
We also need to discretize partial U partial t at i and n. How do I approximate that?
[INAUDIBLE]?
Exactly the same way as we did in ODEs.
And we can approximate it as U of i n plus 1 minus U of i n, divided by delta t. And that could fall into what scheme we use in ODEs?
This grid is going to forward Euler, and we are going to see later what other things we can use.
So let's try to implement one of the combinations.
So which one-- [INAUDIBLE] because that's really the simplest type of [INAUDIBLE] scheme, I think.
And you want both or one of these?
[INAUDIBLE]?
Huh?
[INAUDIBLE]
What?
Central in space one
Central in space one.
All right, good.
Because you guys probably can do Taylor Series in your head and figure out the central difference is actually second-order accurate.
It is better than the other two.
So good.
You have a good pick.
OK, so I'm going to go to Matlab.
The first thing I want to show you in Matlab is-- let me show you in the next class what the difference between finite difference and finite volume.
But I'm just going to show really kind of how a finite different discretize a function.
So when we have a function in space, we have to discretize it, right?
So, because the function is really infinite-dimensional, we only have finite memory.
So if you draw a function like this, a finite difference only stores the values on the grid points.
So the red circles are what it stores.
What is in the middle are just forgotten by the computer.
And all the derivatives have to be approximated by these discrete points.
And this now seems natural to you for sure.
But, as we go to find the volume, you are going to see how different discretization schemes are going to be discretizing a function.
OK so let's try to implement a central difference plus forward Euler on the advection equation.
So I haven't said what equation I'm going to do yet.
So I'm going to implement it on this equation.
Partial U partial t plus partial U partial x is equal to zero.
With a periodic boundary condition, you add 0 is equal to U at 1.
So this is a periodic boundary condition.
So this is the same equation as we tried to animate last Thursday.
Anybody who goes out of these doors comes back from inside here.
So we should be seeing-- this is the linear advection equation.
And all the characteristic lines are parallel.
The characteristic lines are like the colorful lines in the solution in the x and t diagram.
So all the characteristic points go towards the right, and along each characteristic line, the solution is constant.
So let's try to implement the forward order central difference and see how the solution looks like numerically.
So I'm going to New file to Script.
So central space, forward time.
So this is my central space, forward time.
I'm going to set up my number of grid points.
Let's just use 100 grid points in space.
And because I have a domain from 0 to 1, my delta x is equal to 1 divided by n. So this is my delta x.
Let me choose the initial solution that looks somewhat like Gaussian.
OK, so we have a peak, and we see how the peak evolves.
So let me, first of all, set my x equal to 1:N times my dx.
So I have a grid, and accessed the spacial location of the grid.
And let's set my u0 my initial condition, to be exponential of minus x minus 0.5 squared, divided by 0.1 squared.
And I have to do this divide.
So we can take a look at how the solution looks like.
We can plot x and u0.
And we are going to run the script to see how the solution looks like.
So this is the initial condition.
We have a Gaussian curve.
The shape of the curve is exponential.
1 minus x minus 0.5.
So 0.5 is the central of the Gaussian.
And the reason [INAUDIBLE] standard deviation can be plus 1, [INAUDIBLE].
The width of the Gaussian or half-width of the Gaussian.
Any questions on this?
No?
OK.
So let's decide on a timestamp.
What do you guys want on the timestamp?
0.1?
All right.
I goes from 1 to 1/dt.
So let's simulate it for one time unit, because after one time unit, the speed of the solution going towards the right is 1.
That is because in my equation, I put if there should be a small c, then this c is equal to 1.
So my speed of the solution going towards the right is equal to 1.
So after one time unit, the solution should go through the domain once and end up exactly at the same location as the initial condition.
Now when I'm doing forward order, plus central difference, I need to first compute the dU/dx at each grid location.
And dU/dx, at each grid location, is the difference between the solution towards the right minus the solution towards the left, divided by 2 delta x.
And look carefully at how I'm computing the solution on the right and on the left.
du dx is equal to-- what I'm going to do is I'm going to do circshift of my u0, of my u. I'm first going to say u is equal to u0.
[INAUDIBLE] I am going to do circshift of u, and I'm going to be shifting 0 on the first dimension and a -1 on the second dimension.
So does this give me?
I'm going to go back to Matlab and tell you what it gives us.
So if I do 1 to 10, for example, I'm going to get an array going from 1 to 10.
If I do circshift of 1 to 10, I'm going to shift it-- I'll put 0 in the first dimension and 1 on the first dimension.
This is what I'm going to get.
I'm circularly picking these 1 to 10, towards the right.
So instead of 1 here, I get [INAUDIBLE], which is why this method-- the [INAUDIBLE] method-- comes out of the [INAUDIBLE] and goes here.
And the [INAUDIBLE] is that [INAUDIBLE] here.
Everybody's shifted towards the right 1.
Now if I want to grab the i plus 1 person at the i place, this is the opposite of what I need to do.
What I need to do is I need to put -1 here so that at the first place, I'm going to be grabbing the next solution and the next grid point.
And particularly at the last grid point I'm grabbing the solution [INAUDIBLE], which is actually the other way you're supposed to do it on the left [INAUDIBLE], which is 1.
So this is the way you can [INAUDIBLE] Ui plus 1.
If you have periodic [INAUDIBLE].
What about Ui minus 1?
This is my Ui minus 1.
This is my Ui-1.
Any questions on that?
Someone said it back there?
[INAUDIBLE]?
Yeah.
On this one, things are being rotated towards the left.
The 1 which is here now was the end.
And the 2 [INAUDIBLE] here now is here.
3 was here, here.
Yes?
[INAUDIBLE]?
So in the ODE part, why didn't we do a circshift?
The answer is in the ODE part, we didn't do a circshift, because we don't have the solution at ndx.
So when we are at the end timestamp, we don't have the solution at the n plus 1 timestamp.
While in the PV part, we have a grid like this, and we are given the whole initial conditions.
So I've given the whole initial condition at [?
p ?]
equal to 0.
So we have all the data at this timestamp.
Now our job is to go from this timestamp to this timestamp.
We want to compute from the solution here all the solutions on these grid points.
And then we go forward and forward and forward.
So there is a fundamental difference between space and time.
In time, you don't know what is going to happen tomorrow, right?
In space, we have access to the solution at all the spatial locations.
At the current timestamp, and if you stored the past timestamps, also at the past timestamps so you know all these things.
So that's the difference between space and time.
Although, doing Taylor series analyses on them are the same.
But in terms of coding, they are different.
In terms of the order of computation, they are different.
So going back to our Matlab code, we shift towards the -1 direction that is grabbing Ui plus 1.
If we shift the whole thing towards the positive 1 direction, we are grabbing ui.
And if we divide that by 2dx, we are computing du dx, right?
So now we have du dx, we want to plug this into this equation.
dU/dt, in this case, is equal to minus of du dx.
This is what this equation is.
If you shift du dx towards the right-hand side, du dt would be minus of du dx.
So that's what we are going to do.
du dt is equal to minus du dx We used the partial differential equation to convert a spatial derivative into a time derivative.
This is the result of using the partial differential equation.
So the first step, find the difference approximation of the spatial derivative, the partial derivative of x.
Use the differential equation to get the time derivative.
And anybody want to suggest what is next?
Are we back into the ODE world?
We are back into the ODE world.
We have computed the time derivative of the solution at all the grid points we had.
We can use forward order.
From the cutoff, we can use [INAUDIBLE] or other sexy things we learned in the ODE section.
So let's do the most elementary thing and apply forward order.
u is going to equal u plus dt times du dt.
This is exactly how you implement forward order in ODE.
So we already plotted the solution at the initial condition.
Let's also hold on so that we don't lose the plot of the initial condition and also plot my x and u at t equal to 1.
And let's see how the solution will look like.
And let's actually plot this in red, because the initial condition is going to be-- let's plot the initial condition black and the solution red and see how it looks like.
We get a pretty big solution.
This may not be the fault of spatial derivatives, though, because we know forward order has a pretty limited stability region.
But we know forward order is 0 stable.
Which means what?
Which means there it is always going to shrink the timestamp.
Which means it is globally accurate.
And global accuracy means when you make the timestamp smaller, the solution of the ODE at least converges to the [INAUDIBLE] solution of the ODE.
So let's see if it helps in this case.
Let's make delta t equal to 0.001, and let's run it again.
Let me close all before I do the plot.
This time, we get pretty good.
We do see the solution having a pretty good match with the analytical solution.
And let's also visualize how this thing progesses.
For every timestamp, let me actually do the plotting.
For every timestamp, let me do the visualization at clf which means clear the figure.
I'm going to plot the initial condition.
I'm going to hold on.
And I'm going to plot the solution.
So I'm clearing the figure, plotting the initial condition, plotting the solution.
And then you could do draw now.
So let's take a look at how-- let me actually pause for a little bit.
Pause for 0.01 seconds so that we can actually see the solution moving.
So the red one is my solution as my time goes from 0 to 1.
So does it look like we are capturing at least the qualitative behavior of the analytical solution?
Yes.
Although, it is not completely satisfactory, because [INAUDIBLE] and the solution seems to be growing a little bit
[INAUDIBLE]
And also, another concerning point you have [INAUDIBLE] is that the solution initially was all positive.
And you can look at my analytical graphing for the initial condition where [INAUDIBLE] quantity.
It's the exponential of something.
How can it not be positive?
But, like [INAUDIBLE], it becomes negative at some point.
So it overshoots and undershoots.
I mean, we're not going to discuss this today.
But we're going to be discussing the reason of this error, both on Wednesday-- Professor Wilcox is going to teach on Wednesday.
And on the error of this finite difference method.
How to analyze this error and convergence.
And another, very related issue is stability.
That is something we are also going to be discussing in the PDE section.
But this class, we are just happy we got a solution.
And we can also see that the solution is going to get better as I make my grid to be finer.
Instead of 100, let's do 200, and we run the same thing.
It's going to get slower, but the rate of growth is going to be not as severe as in the 100 case.
[INAUDIBLE] Don't really see the undershooting [INAUDIBLE] for the [INAUDIBLE] Question?
[INAUDIBLE] questions.
[INAUDIBLE] Why didn't we see the [INAUDIBLE]?
Right.
So we did a comparison between-- actually, let me ask you a question about it first.
So the question is on backward difference, what's the central difference?
So what do we need to change here to make it backward difference?
The ability of [INAUDIBLE]
First of all, it just needs to be delta x not 2 delta x.
And instead of ui plus 1 minus ui minus 1, we have u minus ui minus 1.
So we're just going to be putting u here.
This is going to be backward difference.
And when you run backward difference-- whoa.
Yeah, we get a similar solution, but now what does the solution do?
Instead of growing a little bit, it decays a little bit.
And we see we have the same number of grid points.
That's where the difference seems to have more undershoot than overshoot of the backward difference.
And again, Professor Wilcox is going to discuss the accuracy of these two methods on Wednesday.
So this is the implementation of the central in space and backwards in space finite difference scheme.
And we also discussed in the last lecture how to solve-- what is the behavior of a diffusion equation when we have diffusion instead of advection.
We have a differential equation that is like this, partial U partial t is equal to k times partial squared u partial x squared.
And we can even have a mixed differential equation.
We can have a mix, advection and diffusion.
So how do we approximate this term?
How do we approximate the second order derivative?
Now the second order derivative has a very concise approximation.
So the second derivative of x at a particular grid point at a timestamp can be approximated as this-- as u of i plus 1n minus 2 Ui n plus U of i-1 n. Divided by delta x squared.
Unlike the first order derivative, you have the choice of forward and backward difference.
For approximating second order derivatives, this is the single most popular scheme we use.
And it is very difficult to find an approximation that is as popular as this.
And let me describe to you why this is the case.
OK We are approximating the second order derivative.
We are only using three points, the center, left, and right.
And amazingly, this thing has second order accuracy, has very good accuracy.
And you can show that by doing Taylor Series on that-- it's U of i plus 1 n can be expanded as U of i n plus delta x times, in this case, the partial derivative of partial U partial x at i n, plus half of delta x squared of the second derivative of U at i n. I'm going to be expanding more terms.
It's the third of the derivative times delta x cubed divided by 6.
And I'm just going to write over here O delta x to the 4th.
Now this this term.
I am also going to be expanding this term.
Ui-1 of n is equal to Ui n. In this case, it's minus delta x because the difference between the i-1th grid point and the i-th grid point is minus delta x.
The second derivative term is plus again, because we are taking the square of minus delta x.
The third derivative is minus again, because we are taking the cube of delta x.
And again, we have plus delta x to the 4th.
Now we can start to see which terms start to cancel out.
See we have one of Ui n here plus another Ui n here.
Now we have minus 2 of Uin.
So this and these cancel together, cancels with this minus 2Uin.
All right, this is one term.
This and this is added up.
So i plus 1 i minus 1 is added up.
So the first derivative term cancels.
The third derivative term, instead of canceling, they add up.
So they, divided by delta x squared, serves as an approximation of partial squared U partial x squared.
It is a consistent approximation for partial squared U partial x.
And usually, when you have three terms-- when you are playing around the coefficient of three neighbors, you can make sure three terms add up to what you want it to be.
You've got rid of the constant terms.
You've got rid of the first order derivative terms.
You make sure that the second order derivative turns out to be what you want to approximate.
But in this case, you get a bonus.
The third order derivative term cancels out too, just by accident, and now, you only get a delta x to the 4th, which after dividing by delta x squared, you get a second-order accurate approximation to the second derivative.
So the error, which is the actual second order derivative minus the finite difference approximation goes like O delta x squared.
And this is pure luckiness.
Any questions on this?
This scheme is what we are going to be using.
Now let's go back and modify our code to also do second order derivatives.
So we have du dx.
We have d2udx2 equal to-- let me copy this.
And we have a Ui plus 1, which is circshift of U towards or the minus direction.
We minus 2 times u and plus u of i minus 1.
And divide it by delta x squared.
So this is our second order derivative.
If we want to solve the diffusion equation over here, the du dt would instead of being minus du dx we have the positive of d2udx2.
And let's put a cover here, a 0.1 cover so that it diffuses slower.
And then let's look at how the numerical solution looks like.
Let's change back to 100.
Oops.
Again, if the stability is coming up, let me make sure we have a small enough timestamp.
So we can see are the solution of this PDE agrees with the analytical behavior of the PDE, which is the solution diffuses out.
It becomes less sharp.
And we can even combine these two terms.
We can du dx plus this, or we can do a convection diffusion equation.
And we can see the solution diffuses and also, in this case, it's shifting towards the left, because I think that the right convection-diffusion equation should be minus du dx.
Let me make sure that is the case.
So we have a solution that is diffusion by the diffusion terms, and also advects toward light by this advection.
Now let's take just a 5 minute break before we go to the matrix form.
We did find a difference [INAUDIBLE].
Before we go into the matrix form of this finite difference method-- and we see the matrix form is going to come up a lot, and it's going to be very useful in, for example, in physics scemes of finite difference methods.
All right.
OK. Let's continue.
So the really [INAUDIBLE] like you haven't seen before.
There is, as I said, unlike discretization in time, you only have one stamp of information, right?
You can only view things one step at a time.
You don't really know what happens tomorrow, what happens at the next timestamp.
So you have to go one step at a time.
We find a difference when we discretize things in space we actually have much more data than we have in ODE discreditization because we have the entire timestamp.
At every grid point we have the solution in our head.
OK. That makes it possible to write and find a difference in a matrix form.
OK.
So when we write the finite difference in the matrix form, we first need to write our solution in a vector form.
We know our solution-- let's say we know our solution yields at the first grid point, u at the second grid point, u at the n-th grid point.
We have them all, right?
So that is what we have.
Can we compute not d but partial-- partial u partial x at the first grid point, partial u partial x at the second grid point?
Is that true of partial u, partial x at the last grid point?
As a joint matrix-- I said joint matrix because sometimes when you have, let's say, a million grid points, how big is the matrix going to be?
It's going to be a million by million matrix, right?
It's going to be a million by million matrix.
Think of n as a million.
And this is why-- I don't know if some of you have heard of the term spark matrix.
That is why spark matrices are invented because usually people don't have enough memory to store those million by million rate.
I'm going to talk about that later.
But sometimes you have this joint matrix.
Now, what is this joint matrix?
What are the elements in this joint matrix?
If it is a central difference-- let me write down the central difference formula here.
Let me write it here so that it's down here now.
In a central difference, partial u, partial x at i is equal to u or i plus 1 minus Ui minus 1 divided by 2 delta x.
And let's assume again we have here [INAUDIBLE] boundary condition so that if i is equal to n, then i plus 1 would be equal to 1.
If i is equal to n, if i is the last grid point, then your neighbor who was y-- which actually, the first y of the other end.
So n plus y is equal to y.
What is the first line of the matrix?
[INAUDIBLE] boundary condition.
What happens if I want to approximate of e to the x at what, when i equals to what?
[INAUDIBLE]
Yeah.
I have 0 here because this formula do not depend on u and i, right?
Now when I go to the second one, what is the modify on u2?
The modify on u2 is 1 over 2 down x, right?
And is there any other than 0s?
[INAUDIBLE]
All the way up to the end because I have periodic boundary condition, I have this.
OK. How about the second line?
The second one is actually a easier way to do it.
When i is equal to 2?
When i is equal 2, du dx at 2 is equal to y.
It's equal to u3 minus u1 divided by 2 down x.
So u3 and u1, which is based on this-- so this guy is now empty.
This guy is now empty, right?
We would get minus 1 over 2 down x here because u1 is being minused and u3 is being plussed.
So we have 1 over 2 down x over here.
We have 0 in the middle, 0, and 0.
Everything else is 0.
So I think under [INAUDIBLE] each line there is only going to be two non-zeros, only two non-zeros because each derivative only depends on two neighbors.
That means we should only have two non-zeros.
If you have more sets over here, then they are going to be more non-zeros.
Two terms means two non-zeros for each line.
And for the third one, my minus is going to be operating on u2.
My plus is going to be operating on u4.
And for the fourth line, my minus is going to be operated on u3.
My plus is going to be operated on u5, and et cetera, et cetera.
So diagonally, it's always 0 towards the very last line.
The sub-diagonal is always going to be minus 2, minus 1 over 2 delta x until the last line.
And the upper diagonal is always going to be 1 over 2 delta x.
Am I missing something here?
[INAUDIBLE]
So the last one, the positive, is over here because of the periodic boundary condition.
Yes?
[INAUDIBLE]
Exactly.
Because we also need a periodic boundary condition, this point, instead of lying over here, there is no longer anything [INAUDIBLE] anymore.
It is actually the first element of the left.
This entry, no matter how matrix multiply with a vector?
This entry is going to be multiplied [INAUDIBLE] instead of U n plus 1, which [INAUDIBLE].
Any question on this [INAUDIBLE] matrix?
So all of this matrix is really a million by million.
But it only compares two a million, 1, 0 entries.
Most computers have enough memory to store a million non-zero entries.
All right.
So this is central difference.
Let's just do an exercise of how, if we do a-- oh, I haven't talked about upwind yet, so backward in space difference
[INAUDIBLE]
Huh?
[INAUDIBLE]
Towards the upwind of the [INAUDIBLE].
Good guess.
Right.
So if I do a backward in space, I'm going to be writing the same thing, partial u partial x at 1, partial u, partial x at 2 essential, partial u partial x at n. I think I used a small-- yeah.
I used a small n-- is equal to a joint matrix of the same size times u1, u2, et cetera.
So nothing has changed, except for the content of the matrix.
And a backwards space difference is partial u partial x at i is equal to Ui minus Ui minus 1 divided by delta x.
All right.
Somebody help me fill in the blanks.
Fill the blanks of a million by million matrix.
Doesn't take that much time to fill in, actually.
What is the first top level entry?
One over delta x
One over delta x.
It is no longer 0 because the du dx at ix usually depends on Ui.
In the last case, the du dx at i does not depend on Ui.
So we have one other entry because we have two elements in each derivative.
What is the other non-zero entry?
At the very end
At the very end.
That one is the same.
All the others are 0's.
How about the second line?
We have diagonals to be always 1 over delta x because we always have 1 over delta x multiplied by Ui.
That going to be du dxi.
And we have 1 minues-- sorry.
We have minus of 1 over delta x always towards the left because the value at the left is always multiplied by minus 1 over delta x.
All right?
OK. Another x.
So OK.
So how about let's try to, in MATLAB, try to construct the matrix form.
When we construct the matrix form, we don't need to do it in every concept.
We an do it well in advance.
So after we settle initial conditions, let's actually-- before we settle our initial conditions, we know what n is, we try to set up matrix a. OK. Let's call it a ddt.
So that is the matrix that gives me-- sorry-- ddx, that gives me derivative to x.
So let me see what I did.
Here is actually backward in space.
So let's try to reproduce backward in space.
This is the matrix I have.
I already have delta x computed.
Yes, I do.
So then I need two matrices.
One matrix, the first matrix, is this guy.
I need, first of all, this matrix.
And somehow if I can add that matrix to the second matrix, if I can add that matrix to this matrix plus this little element over here, then I have the entire matrix.
So let's look at the diagonals, the blue one.
Can everybody tell me, is there an easy way to [INAUDIBLE] the blue matrix?
[INAUDIBLE]
Huh?
It's identity, it's identity matrix times one over delta x.
That one is actually super easy.
How about the other one?
How about the black one?
[INAUDIBLE]
Yes.
I can use my x. I can use diag if I don't have this one.
I'm going to see later on if I have not periodic boundary condition.
Periodic boundary condition is actually this.
If I don't have periodic boundary condition, [INAUDIBLE] then diag is a very good way to do it.
In this case, I'm going to use the same function, the third shift I just used to centrally shift the solution itself.
I can also shift the matrix.
I can take this matrix, shift it to the left or towards the bottom.
It's the same way.
I'm going to get the back matrix.
It is equal to i of n divided by delta x.
So that gives me the blue one.
I'm going to minus in by dx.
But I'm going to circshift the whole matrix by left 2 towards the bottom.
Towards the bottom is [INAUDIBLE] in the third direction, 1 in the other direction, 0 in the [INAUDIBLE] direction.
So I'm going to shift it by 1 and 0.
OK. That is my plan a difference matrix.
Now how do I apply it?
How to apply it-- when I computed du dx, When I compute du dx-- let me first do one thing.
I want to make my everything to be a color matrix first because by default, if I do 1 to n, it's a row matrix.
Where if I want to multiply the matrices that way, I need my solutions to be like a column.
So initially, I want to construct this by columns.
And everything should be in columns.
And this is no longer true anymore.
I'm going to remove this.
I'm going to just do a linear advection equation first.
But my du dx is equal to a_ddx times u.
That's how easy it is to apply this on a difference matrix.
Also, I have the final difference matrix, I don't need to do anything at each concept other than just applying the matrix just as if it is the mathematical operator.
I just think this matrix is my actual [INAUDIBLE].
I apply on top of u, I get partial u partial x.
So this a serves like a mathematical operator.
I'm going to round this.
I'm going to get the same solution.
Of course, this is backward in space.
I got a solution that moves towards the right but also the case.
So this is what I have as a backwards difference.
And I can modify this.
So I'm going to [INAUDIBLE] this out.
This is backward difference.
I'm going to do central.
And in the central, the difference is I need to circshift both.
I need to circshift this in the opposite direction.
And instead of dividing by 1 delta x, actually, let me divide this by 2 delta x.
So this is going to be my central difference, right?
Agreed?
If you look at this, I have one identity matrix.
shifted towards the right.
And I have another one shifted towards the left.
So I get two matrices subtracted.
I get the central difference.
Let me look at delta behavior.
Looks like the same as the central difference.
It does, right?
I have a solution where we go to right.
It doesn't really decay at all.
But [INAUDIBLE] a little bit.
[INAUDIBLE] if the maximum occasionally gets greater than 1, it still shifts down a little bit.
All right.
Any questions on the matrix form of these two final difference operators?
No?
Whenever you need them, I'm basically constructing a final difference operator just like a mathematical operator, just like ddx.
And when I need it, I can just apply it.
The interesting thing is that the operator I have already contains the boundary condition.
The operator I have here, the periodic boundary condition is encoding into the operator [INAUDIBLE].
If I didn't have periodic boundary condition, would I still have this?
No.
I wouldn't be able to say, if I want to-- I wouldn't be able to say if I only graph the solution towards the left, I just need to grab the one at the right hand.
I wouldn't be able to say that.
So what if we-- let's say we have a boundary condition at the left, but if it is not periodic, what if we have a boundary condition that says u at 0 equal to 1?
OK.
So let's see.
U at x equal to 0 at net equal to 1.
This is consistent with our matching question, because in this matching equation, the characteristics move towards the right.
So if you didn't have periodic boundary condition, you need to specify the solution.
You need to specify the solution also here and here, right?
You need to specify the solution where the characteristics originate.
Because if I'm here, I have the initial condition that p is equal to 0.
You also need to specify those conditions here, either boundary conditions.
That is when x is equal to 0 or x is at the left end of the domain, [INAUDIBLE] equal to 0.
So this is the left boundary condition.
All right?
OK.
In that case, how do we deal with that?
How do we encode that boundary condition into the matrix, into the matrix form of finite difference.
Let's look at this.
Let's take a look at the matrix.
And the only thing we need to change is which one?
Which row of the matrix do we need to change?
If we need to change one at all.
We only need to change the first row, right?
Because all the other rows of the matrix are still exactly the same because they are the interior.
Only the first row involves the boundary.
So let's look at the first row.
The first row has partial u, partial x at r equal to 1 at n [INAUDIBLE].
Originally, it is equal to u of i minus 1, which is in this case is 0 minus u1 divided by delta x.
This is my backward-- again, this is my backward in space.
This-- yeah.
It's the other way around.
Thank you.
It is u1 minus u0.
Yes.
OK.
In the previous [INAUDIBLE] we said the domain is periodic.
So 0 is just U n, right?
So what this is is that we have a boundary condition that is given to us.
You know what exactly U0 is.
We know that your 0 is equal to 1.
Then the derivative is u1 over delta x minus 1 over delta x.
We have a term that doesn't depend linearly on the solution.
Or, if you look at the matrix form, we have multiple conditions all the way.
We don't have this.
But in addition to the matrix, we have something else.
Obviously, we have a negative of 1 over delta x sitting over here as an additional vector to this matrix.
Let me make a new page.
OK.
I'm going to draw what the matrix is going to look like.
When we have a boundary condition, like partial u, partial x at 1, partial u, partial x at 2, partial u, partial x at n is equal to a similar matrix at minus delta x on the sun diagonal.
I want to leave enough space for the vector ends on the right.
My u1-- I'll think I'll use the big U for the first U n. This is the big U plus something else.
And that something else is going to be 0 almost everywhere except for the first row, which is minus 1 over delta x.
And it is going to be the 0 everywhere because I don't need to modify anything.
How id I achieve this?
I plucked the boundary condition into the finite difference formula at the grid point that is close to the boundary.
All right?
That is a general technique of locking in the value of the boundary condition into the [INAUDIBLE] difference here.
So let's modify what we did under here.
So let's copy this.
I think I should rename this as backward because this is backwards.
And I'm going to do probably another one and say boundary condition backwards space forward time.
That's correct.
So what do I need to do?
I need to change-- oh, OK.
I need to change the operator.
I need to change the operator to include the boundary condition.
How do I do that?
If you look at what the matrix is like, I still have the diagonal term to be exactly the same.
I still have my identity of n divided by delta x.
So that is this part.
Now, I also have my rare part, which is not exactly a circular shift of identity matrix.
So here I'm going to use the function code diag.
Diag, it's a function that you need to give to me.
What are the elements of a diagonal and how much I shifted.
If a is equal to 0, then elements aren't exactly on the diagonal.
If a is equal to minus 1 then I get exactly what I want because it's minus 1 shifted towards the opposite diagonal, which is 1 shifted towards the lower diagonal.
So the elements would be once with m minus 1 elements divided by delta x-- these are the values-- and minus 1 towards the shift.
OK.
So this is what the matrix is going to be like.
I'm going to paste it here and take a look at the matrix.
The 100 by 100 double on the diagonal [INAUDIBLE].
I get what I want, and I also don't have this-- I don't have a minus 100 over here in the periodic case.
Now, let's run it to see what we have.
OK. We still see these things going over here.
What happens on my left?
Oh, OK. Did I forget something?
I forgot the boundary condition.
So I forgot to add the vector v when I applied this derivative.
The end result here is I have a boundary condition of 0 here.
I'm setting the boundary condition to be 0.
[INAUDIBLE] As this thing advances towards the right, it's going to be no longer [INAUDIBLE] towards the left.
So yeah.
So I did forget something.
But we still have such a boundary condition that is not periodic.
Because you see this [INAUDIBLE] go out, I'm still going to have 0 here.
OK.
So let's add to the b.
My b_ddx is going to be 0n1.
And my b_ddx, the first element is going to be 1 minus delta x.
Minus.
All right.
When I apply it ddx is this plus [INAUDIBLE].
Yeah.
This is still going towards the right, and it doesn't come back from the left.
So now let's look at what if we do have a non-trivial boundary condition.
Why?
What happens?
Ah, undefine the functional variable, b_ddx.
All right.
I'm going to round it.
All right.
So we have a boundary condition of b equal to 1 over here as you can see.
This is the behavior of the differential equation [INAUDIBLE] so we'll see everything that wraps towards the right.
And [INAUDIBLE] there shouldn't be a discontinuity over here, but the [INAUDIBLE] actually moves this discontinuity out, and you get this finite jump from 0 to 1 over here.
But you still the boundary condition being forced correctly as things do move towards the right.
All right?
So the right way to put boundary conditions, one right way to put the boundary conditions is to substitute the boundary value into the differential operator, into the discrete highest differential operator and derive not only a more refined matrix form, but also a vector that added on to the matrix form
[INAUDIBLE]
OK.
So the question is, this form o the boundary condition looks like we are asking a [INAUDIBLE] into the differential equation.
And you are absolutely correct.
I think not only in this case, but in several others [INAUDIBLE] of laying the boundary condition, it's essentially adding a [INAUDIBLE], like [INAUDIBLE] boundary condition.
So sometimes [INAUDIBLE] the boundary condition is like knowing how to add the fore onto the right-hand side near the boundary to-- so that your emergence solution converges to the analytical solution with that boundary condition.
All right?
Did I answer your question?
Does that?
[INAUDIBLE]
So the graph I have is-- the graph I have is not with the boundary condition but due to the minor difference here.
Yeah, the scheme is on the-- gives you a truncation error.
It is marked on the [INAUDIBLE] derivative exactly on that truncation error.
[INAUDIBLE] smooths out any discontinuities, if there's computation error, you can actually see that a truncation error looks like a diffusion here.
It looks like you're diffusing your solution out instead of just advecting it.
And the behavior of the central difference scheme is very different.
All right.
OK.
I'm just going to very quickly also give you the finite difference form, the matrix form of the second order derivative.
The matrix form of the second order derivative is if you have the second order derivative of u at 1 and the second order derivative of u at n-- again, is equal to this joint matrix.
And again, let's assume is periodic boundary conditions so that we don't have to worry about the boundary conditions for now.
OK.
So the second derivative, partial u, partial square u, partial x at i is equal to three terms.
We have a u of i plus 1 over delta x squared minus 2Ui over delta x squared plus Ui plus 1 over delta x squared, right?
[INAUDIBLE] the same as the second order [INAUDIBLE].
So Ui plus 1 minus 2Ui plus Ui plus 1, everything divided by delta x squared.
Now if we have periodic boundary condition, how do I fit this into the matrix?
[INAUDIBLE]
Sorry?
[INAUDIBLE]
The derivative.
Oh, the derivative.
Yes.
Thank you.
Yes.
Yes.
OK. All of these terms going to that matrix.
So let's start with the diagonals.
The diagonals are all the same.
They are coefficients on this term.
This term goes into the diagonals.
Yeah.
This term goes into the diagonals.
And let me color that by blue.
This 2 over delta x squared--
[INAUDIBLE]
Negative.
Thank you.
Delta x square all the way to here minus 2 over delta x square.
Now, where does this go, Ui I plus 1?
Both with their upper diagonal goes over here.
It's 1 over delta x squared, 1 over delta x squared' all the way to 1 over delta x square, and here, 1 over delta x squared.
Because when i is equal to n, i plus 1 is actually equal to-- it's the same as ux1.
OK. And lastly, let's fill in the-- OK. Let's fill in this i minus 1.
This is i minus 1.
And fill in the i minus 1 term we have on the lower diagonal, it's the same as the i plus 1 term, 1 over delta x squared, 1 over delta x square, and 1 over delta x square.
The first term is actually over here.
It is in order to get the second derivative for the first grid point.
You have to graph all the way from the last grid point.
OK.
So this is the matrix form of the second order derivative.
OK.
So next class, we're going to look at how the finite difference schemes converge as we make the grid point more and make the spacing smaller, and also stop to look at two-dimensional problems.
All right.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So today is our lecture on finite volume method.
So we talked about finite difference method.
And we are going to be talking about finite volume method and finite element method.
So I'm going to-- there is a request for me to go over what did I do on the matrix form of the two dimensional finite difference.
So I'll go over that.
But before I do that, let me show you what is the difference between finite volume method and finite difference method.
Because there are two concepts, they are two conceptually different methods of describing and discretizing partial differential equations.
So I have a few demos.
I have a finite volume demo.
I have a finite difference demo.
I have also a finite element demo.
So they are really three different conceptually different methods of solving PDEs.
So first I have a finite difference demo.
So let me explain what is happening here.
We are representing a function of x, a function of space in solving partial differential equations.
And this function is going to evolve as a function of time.
But in order to [INAUDIBLE] that function of x, we need to discretize it.
Because a function is an infinite dimension [INAUDIBLE].
Like that.
Huh?
[INAUDIBLE]
I think I-- I think there is something I can get rid of.
I just don't know-- Advanced Settings.
Monitor.
Yeah.
I don't know why.
Oh.
That went away?
Oh OK. [INTERPOSING VOICES]
Don't close that dialogue box
OK. Advanced Settings.
Monitor.
I think I didn't do anything except for--
[INAUDIBLE]
Yeah, just like that.
Yeah
[INAUDIBLE]
OK. Wow.
[LAUGHTER] I have no idea.
This is weird.
Technology.
OK.
So I have a function of x. I need to discretize it.
Can somebody come and draw me a function?
And visualize how this is kind of different, how finite difference discretizes a function.
So can somebody come up and draw a function?
And maybe you can describe to me before you draw the function of how finite difference discretizes the function, how will you [INAUDIBLE] that function.
How would the-- in a finite difference method, how do we represent, like, obviously a function?
Let me see.
These blue lines are where the grid is.
[INAUDIBLE], Right.
OK.
So who wants to draw it for me?
We just did the project.
It's a time to relax a little bit.
So [INAUDIBLE] the difference in [INAUDIBLE].
[INTERPOSING VOICES]
All right.
Good.
So when we draw a function, it looks like this.
Right?
[?
If this ?]
curve is the function [INAUDIBLE] a little bit, right?
On the-- if not the [INAUDIBLE].
In finite difference, we forget about that the function is.
We only remember the value of the function at these distinct points.
This is finite difference, though.
Do we all have a mental picture of what this does?
What finite difference does?
And then when you have approximated derivative, right?
For example the derivative here, we can approximate it either as the slope of this line or as the slope of this line.
Depending on if we do backwards [INAUDIBLE] or [INAUDIBLE].
And in particular we may approximate the derivative here, [INAUDIBLE] taking this slope.
If it's backwards [INAUDIBLE] then we are taking this slope [INAUDIBLE].
So there are some approximations [INAUDIBLE].
This is what finite difference does.
OK.
So can I invite somebody to do something that is completely unpredictable?
Because we are going to be looking at the same thing.
But we're going to be using the same grid.
I'm also going to ask somebody to draw me a function.
But now we have a visual of how finite volume is going to discretize this function.
OK.
I'm going to type FvDemo.
It's the same plot before-- yes, I'm going to ask you to draw the function of the same plot.
But can somebody draw-- [INTERPOSING VOICES]
And let's see how a finite volume is going to discretize the function.
Whoa
[INAUDIBLE]
OK. Well, we all see the function in the background, OK?
See that?
This is how finite volume [INAUDIBLE] this function if we do that.
What do we do?
What does finite volume do?
[INAUDIBLE] the function value anywhere?
Would somebody describe to me what we did?
What finite volume did?
And we're taking like an average value in between the points, each two grid points.
Right?
In finite difference, we talked about grid points.
Because we are exploring the value of the function at these grid points.
In finite volume, we are talking about cells or controlled volumes, right?
Each space is in two points, or controlled volume.
We are discretizing the function by storing the average value of the function inside each controlled volume, or inside each cell.
OK?
So in particular, if we look at this one, we will have the [INAUDIBLE].
But [INAUDIBLE] for the average difference.
OK. And as we see, when we use finite volume, we need to evolve these averages.
And we know the function value.
In finite difference, when we look at the function value, and the function value, of course, will satisfy the PDE.
And the PDE in order to solve the PDE, we need to approximate the spatial derivative.
Here, we need to evolve this average.
And we are going to look at what the differential equations on this average can [INAUDIBLE], how these averages cannot satisfy the PDE.
It satisfies something else.
And this is what finite volume does.
It is going to derive from the PDE a different set of equations than describe the evolution of these averages.
And so this equation set describes the evolution of the averages.
OK.
So this is a conceptual difference between finite difference and finite volume.
And when we start finite element, I'm also going to-- well, let me not show you today.
OK?
If you're interested in it, you can just go run it at home.
I'm going to give you the three scripts.
If interested, you can run it at home and see what you get.
[INAUDIBLE] who don't have a tablet, you can go with your mouse on the function.
So I'm going to give you the scripts later on.
But right now let me first focus on and go to our last lecture folder which is 09.
And I'm going to show you a little bit of what exactly we did.
The two dimensional advection question.
OK.
So this is the two dimensional advection equation.
I think Professor Willcox ran that last class, right?
We have this wavelike thing going on there.
Of course, this is not as advanced as your project.
Because we are looking at the constant cx and cy.
We're not looking at a cx and cy responding to the [INAUDIBLE].
So this is actually a simpler script than what you guys wrote in the project.
But one thing I'm doing differently from the project is that I'm using implicit methods in solving this equation.
[INAUDIBLE] So in using the implicit time-integration method, I have to use the matrix form of the finite difference.
Right?
Does everybody understand why by switching from explicit to implicit I need to use the matrix form of the finite difference?
You don't?
[INAUDIBLE].
Does the matrix only have a times b plus something else times [INAUDIBLE]?
Exactly.
The reason I have to use the matrix form is because in an implicit time-integration scheme, the spatial derivative is operated not only on time step n but also on time step n plus 1, which is not known before I do the time step.
I have a spatial differential operator operating on some unknown solution.
So I cannot compute it explicitly.
I have to move that curve to the left hand side of the question and solve it.
So let me make a new page and show you what this is.
Page options.
Insert.
OK.
It's more like this.
[INAUDIBLE] OK.
So in an explicit scheme which would be partial u-partial t equals to, for example, a simplified version of what you guys did is this, right?
This is like the simplified version of what you did.
And if I do explicit time step, and if I use forward Euler, then I can do U of n plus one minus Un over delta t. That is an approximation of the time derivative.
OK.
I can do matrix form.
I can do non-matrix form.
But whatever I do, I'm making-- I'm either evaluating distance in matrix form or not in matrix form.
But I'm actually computing the differential operator operated on Un, which is known.
The fact that Un is known makes it possible for me to evaluate this without [INAUDIBLE] the matrix A.
Right?
This is explicit.
But what if I need to do this?
What if I want to use an implicit method?
For example, if I need to do backward Euler, then the difference is that these spatial finite difference operator is operated on not Un but Un plus one.
Well, yes.
Where a is finite difference, approximation of minus partial u-partial x minus partial u-partial y.
Right?
OK.
So in this case, I do not know what we have for [INAUDIBLE].
Therefore, I cannot just evaluate-- I can no longer do-- most of you did, which is taking a bunch of indexing with this U and subtract from doing the indexing or doing the circular shifting of U and subtracting it from each other.
I cannot do that.
Because I don't have the n plus one.
What I need to do instead is to move all the Un plus one terms, is I need to move all these blue terms to the left hand side of the equation and move the green terms to the right hand side of the equation.
So this is what I need to do.
I need to move these to this side.
So what I end up getting is identity over delta t minus A of Un plus one is equal to Un over delta t. So I [?
initially ?]
was so busy [INAUDIBLE] I did not [INAUDIBLE] must have [?
deflated ?]
A, constructed.
Right?
So that is why, when I am solving-- when I'm using an implicit scheme, I need to have the matrix form of the finite difference operator.
OK so this is how I did it in the finite difference matrix.
I am taking-- so I am constructing the same derivative matrix I constructed in the [INAUDIBLE] case.
This is-- if you let me-- OK. Let me run this.
I get a grid that looks like this.
And then when I run this, getting-- If I do a spy on Ax, I'm getting the same matrix as we get in a one-dimensional case.
On the y diagonal, I think we have something on the diagonal [INAUDIBLE].
Remember this?
And Ay has the same structure.
It is almost identical, except for the dimension are a little bit different because the y direction has a different number of grid points as the x direction.
Then I did something interesting.
I did a run of Ax with Iy.
So Iy is just an identity matrix.
So I did a [INAUDIBLE] of Ax in Iy.
To show what [?
connect a product ?]
does, I think I just want to do a spy of the product.
This is what gets me.
OK. You see, what I'm getting-- let me zoom in.
Let me zoom in using my pen.
I think it's a little better.
So what I'm getting is a bi-diagonal matrix.
But the [INAUDIBLE] I'm replacing each element in Ax with that identity matrix.
So now this [INAUDIBLE] this is a series of dots.
Each dot is one entry of a matrix.
There is a bunch of elements over here.
And there is a bunch of elements over here.
So when I'm eventually doing is, OK.
If I modify something with this matrix, I'm subtracting-- so let me look at this one-- I'm subtracting the value corresponding to this column from the elements corresponding to this column.
So I'm subtracting two values that are spaced apart with the spacing exactly equal to the number of x elements.
Right?
So in terms of spacing, if I have-- let me insert another one.
If I'm organizing U2 1, U2 2, et cetera, U 2nx, et cetera, to Uny1, et cetera, Uny2.
So if I'm organizing my array like this and I'm going through-- [INAUDIBLE].
OK. And if I'm organizing my array like this, I'm organizing my array as this one goes to this one goes to this one goes to this one.
So I'm storing this first in memory and then this one and then this one.
And then you want to two in memory and following this.
So I am storing one column the other column then another column, et cetera.
Then if my two elements are spaced apart with ny elements in memory, then it is the exact same, the adjacent elements in the x direction.
That is how my elements are stored in memory.
So which means, if I'm taking derivative in the x direction, my spacing should be ny.
And what happens if I take the derivative in the y direction?
What if I do kron of Ix with Ay?
Oh.
OK.
Yes, I mean this.
OK.
When I go to something like this, so if you look at each block-- so I'm getting a periodic structure.
And each block [INAUDIBLE] is exactly the same as the one dimensional derivative matrix.
And once we zoom in over here-- oh, you can't see anymore.
But these are individual dots.
So when we take the derivative in the y direction, we are taking [INAUDIBLE] adjacent values in the array, which is the adjacent value actually in the y direction.
So adjacent values in the x direction are actually spaced over by ny in the memory.
So these values in the y-direction are spaced with only one in memory.
That is why we need to use the kron in the matrix form.
OK.
Questions on that?
OK. Now let's go to our finite volume analysis.
OK.
In finding the volume, we are ultimately going to two dimensional too.
Today, we are just going to focus on one dimension finite volume.
And again, in finite volume-- OK. Let me go back to our last [INAUDIBLE].
In finite volume, we draw a function.
And obviously in the function we are only storing the averages inside each cell.
All right.
So let's look at why that is, what kind of equations can finite volume solve?
And why does it solve this kind of equations well?
Finite volume only works if you have a conservation law.
Finite volume is not for all differential equations.
It works only when you have something that can be formulated in terms of conservation laws.
And what does conservation law mean?
Conservation laws mean the equations that are in this form.
OK. You can have a [INAUDIBLE].
But like you can have a [INAUDIBLE] here.
All right.
So this is conservation laws.
It's some kind of a conservative quantity.
Can be mass, momentum energy, some molecules, number of [INAUDIBLE], number of people.
Something that can be described as conservative.
This F is the flux.
The flux is the number of quantities u that goes through a boundary per unit amount of time.
OK. That's what flux means.
All right.
And the specific force, the force is how much quantity, again described in u, is generated, comes out from nowhere, per unit amount of time.
This is a case, for example, if you have some kind of a-- if u is describing the amount of molecules and F is the number of molecules generated in an amount of time if you have a chemical reaction or things like that.
All right.
And if u is describing momentum, then F is the flux of momentum through the boundary.
And S would be something like the [INAUDIBLE] of gravity.
Something like that.
Right?
Is it clear what the conservation law means?
The conservation law really makes even more sense if you try to integrate it.
OK?
So this is the differential form.
Now if you try to integrate the conservation law, it even makes more sense.
And by integrate, I don't mean integrating time.
I mean integrating space.
So let's integrate over any control volume.
And control volume is described as an interval from L to R. L is the left hand of the interval.
R is the right hand of the interval.
So the integral of left hand side has to be equal to the integral of the right hand side of S, both are dx.
So let's do some manipulation to this integral form.
So first of all, the integrative time derivative in space.
Because time and space are two independent variables, the integral in space [INAUDIBLE] with the derivative in time.
So the integral in space of a time derivative is equal to the time derivative of the integral in space.
The first term can be written as this.
And this is the time derivative of [INAUDIBLE] using that control volume.
If U is density, then this is the amount of mass inside the control volume.
If U is the density of [INAUDIBLE] like molecules, density is the total number of molecules in that control volume.
OK.
The second term.
The second term is not a timed derivative.
It is a spatial derivative.
Now what is the spatial integral of the spatial derivative?
The function itself.
Exactly.
So it is F of U at R minus F of U at L. All right?
And let's actually move this to the right hand side.
Because it makes more sense.
Let's move it to the right hand side and put a minus sign here, put a plus sign here.
OK.
So this is the result of the spatial derivative term.
And the [INAUDIBLE] that's [INAUDIBLE].
OK.
So this is what the integral is going to give us.
Now let's try to interpret that integral.
The time derivative of the total quantity within the control volume is equal to the flux at the left minus the flux at the right.
This is what goes into the control volume.
This is what goes on the control volume.
So makes sense?
The rate of change of the amount of stuff inside the control volume is equal to the rate of change for the stuff going in minus the rate of stuff coming out plus the rate of stuff generated inside the control volume.
So this is the integral form.
This is the integral form of the conservation law.
Now why is it important?
And how does it relate to finite volume method?
Because this is the equation we are solving in finite volume method.
This is the equation we're solving in finite difference method.
Right?
We stick with this kind of equation.
Because in finite difference method, the U, we are storing the value of the solution U at individual points.
Right?
And that is why we need to approximate the distance, approximate the spatial derivatives [?
and it ?]
kind of a finite difference method.
And then we can do [INAUDIBLE] for U [INAUDIBLE].
With finite volume method, we are not interested in the solution at individual grid points.
Instead, we are storing the average of the solutions.
And the average of the solutions, what is a mathematical description of an average of a solution in a control volume?
What is the average when we look at the amount of stuff [INAUDIBLE]?
Yeah.
It's the amount of stuff in space.
If I ask you to write a mathematical formula of the value of the average as a function of-- in terms of the function within these intervals, what is it?
[INAUDIBLE]
Exactly.
The average would be equal to the integral of this function from 0.6 is 0.7 divided by the width of the integral, which is 0.1.
That is what the average means.
And now we have derived the integral form of the differential equation.
And it describes the evolution of the integral.
Do we have the evolution of the average?
Of course.
We just divide this thing by R minus L, right?
We just divide the whole thing by R minus L. Then this whole term actually-- let me include the time derivative-- this is the d dt of the U average.
Right?
This whole term would be the time derivative of the average value of the function between L and R. The time derivative of the average value between L and R is just equal to the amount of stuff going in minus the amount of stuff going out divided by L over R. Plus the average amount of stuff generated in the control volume.
Right?
So the reason we derive the integral form from the differential form is because this integral form is what is satisfied by the finite volume method, by the volume average.
So we use this form for the finite volume method.
Now this form has distinct advantage over this form.
Why?
Why is sometimes this form valid while this form is invalid?
Can you guys think of a case?
[INAUDIBLE]
Yeah.
Of course, this has derivatives [INAUDIBLE].
This has a spatial derivative.
This does not have a spatial derivative.
Right?
So in what cases can you not have a derivative?
[INAUDIBLE]
Yeah.
We can have singularities.
What kind of singularities have we seen that have no derivative?
[INAUDIBLE]
Huh
[INAUDIBLE]
[INAUDIBLE].
Shocks.
Shocks.
Right?
We saw-- when you say shocks in solving this kind of differential equation.
Once we have a shock wave, you have a discontinuity over x.
Do you accept you can take the derivative of a shock wave?
No.
Actually, I'm going to show you.
If it applies to you to define a different method to solve a differential equation with a shock wave, sometimes you can still [INAUDIBLE].
And you only get correct solution when you use finite volume method, because finite volume method doesn't care if it has shock waves or not.
It solves the integral form of the equation, which is immune to shock waves, and not even taking spatial derivative.
As long as I can integrate, I can solve the equation.
And the solutions with shock waves have their integral.
All right.
So before we go into finite volume method and look at its solution, let's review how the solutions behave.
And I think we can do that review best if we just ran the Burgers equation phase again.
All right.
So Burgers' equation.
I'm going to draw the initial condition this time.
I haven't drawn it.
So I'm just going to draw something that goes up and goes down and goes up again.
OK. And let's look at how the solution evolves.
OK.
So this is a typical conservation law.
[INAUDIBLE] is one of the simplest conservation laws.
And remember, the solution moves with a speed proportional to the value itself.
So if I have a high value and a positive value, I move toward the right.
If I have a negative value, I move towards the left.
Right?
This is even more dramatic than the case where everybody's [INAUDIBLE].
This is the solution we are [INAUDIBLE].
So if I have something moving towards the right and I have something moving towards the left, and you can very clearly see a shock wave forming over here.
And we can know what the solution is really intuitively because we know at each point, the solution moves proportional to the value, which if you look at the plot with [?
accuracy ?]
in space while at the same time, then the characeristic lines-- which are straight lines in space and time-- the flow-- or I should say the inverse flow because their speed is delta x over delta y.
So the delta in the x axis over the delta in the y axis
[INAUDIBLE]
[INAUDIBLE] is always delta x over delta y.
In this case, delta x is [INAUDIBLE] delta y. Delta t [INAUDIBLE].
Delta x over delta t is kind of the inverse.
Does that make sense?
I mean, a speed equal zero is like a vertical line.
A vertical line has no delta x but lots of delta t. So it's low speed.
And the line that is very shallow is the high speed line because it has a lot delta x over small delta t. So we see these are high speed.
And these near vertical lines are low speed.
And these lines sloping towards the left are negative speed.
They are moving towards the left.
Right?
OK.
The solution behaves like this.
The characteristics are proportional to the solution itself.
The redder it is, the larger the position is.
And the more quantity, the larger the speed is.
The bluer the solution is, the more negative the solution is.
And the solution moves backwards at a negative velocity.
This is how the solution behaves in a Burgers equation.
And we also have this shock wave that moves at an average velocity of [?
6.54 ?].
Right?
So we this shock wave moves kind of slowly towards the right.
Because if you look at the average left value and right value, it's slightly positive.
So the shock waves moves slightly towards the right.
OK. That's how the solution for the Burgers' equation behaves.
And we need to have a method to solve the Burgers equation.
And we're interested not only in the smooth part of the solution, but also in the discontinuous part of the solution.
OK let me just do another case and ask you how the solution is going to behave.
All right.
So if I have a function u of x for the Burgers equation, the Burgers equation is partial u-partial t plus U times partial-partial x equals zero.
Or in a conservative form, it's partial u-partial t plus partial u over partial x and half of u squared equal to zero.
Right?
So this is-- this form is [INAUDIBLE] u squared out of the spatial derivative.
After a while, can anybody tell me how the solution is going to look?
Yeah?
Is this periodic?
Let's say they are periodic,
[INAUDIBLE]
Yeah
The shock waves are formed and [INAUDIBLE]
Yes.
Exactly.
So this part moves at a positive speed.
This part moves at a negative speed.
So you can really see the solution later on should become something like this.
And later on, the shock wave should form.
I actually don't know if it's moving towards the left or moving towards the right.
But the shock wave is going to form over here
[INAUDIBLE]
Uh huh
[INAUDIBLE]
Yeah.
Good point.
This part is a little bit [?
difficult ?].
So the positive part is still dominant and the shock wave is going to move towards the right.
Good point.
OK.
So this is how the equation behaves.
And let's see how do we solve it using a finite volume method.
OK.
So let me write down the equation again.
Partial u-partial t plus partial-partial x of half of u squared equal to zero.
Right?
Or in the more general form, I should be writing partial u-partial t plus partial F of u partial x equal to zero.
And to specialize this general conservation law into the Burgers equation, we just need to set F of u equal to half of u squared.
OK. And again, you find the volume.
We are solving for the derivative.
OK. Now we need to, we are solving the-- we are tracking the evolution of this Uk average.
And the Uk average is defined as one over delta x.
So delta x in this case is the size of each control volume.
OK?
Control volume.
OK. Times the integral from the left of the k-th control volume to the right of the k-th control volume.
OK. And the size of the control volume is of course equal to Rk minus Lk.
OK.
The integral is the u of x and t dx.
This is my cell average.
And in general, this is delta xk, right?
Because each control volume can be different.
And here we are just going to talk about the case with a uniform, uniform mesh.
And the uniform mesh responds to delta Xk or r equal to delta x.
And the L of k is equal to k minus one times delta x.
And the r of k is equal to k times delta x.
All right.
So the L of 1, the left boundary of the first [INAUDIBLE] is at zero.
The right boundary of the k-th control volume is always the same as the left boundary of the next control volume.
That has to be satisfied for finite volume method.
You have to divide the entire space, the entire domain into non-overlapping into non-overlapping control volumes and leave no empty space in between.
You have to basically partition the entire [?
combination ?]
domain into non-overlapping control volumes.
And you can't leave anything open.
OK.
So now let's take a look at what is the time derivative of this average?
To figure out the time derivative of this average, we have to use the integral form of the conservation law.
It is equal to one over delta k times d/dt of Lk Rk Uxk dx.
And by applying the integral form of the conversion law, we get the flux at the right hand side minus the flux at the left hand side of the control volume divided by-- oh, we don't need to divide anything because we are looking at the derivative of the integral.
Right?
And plus the integral L to R of a [INAUDIBLE] which is zero in the Burgers equation case.
Because in the Burgers equation, we don't have any source then.
The source is zero.
OK. Now we have an evolution equation for the cell average.
And up to here, we've made no approximations.
Right?
In finite difference, we are approximating the spatial derivative.
But here, we don't need to make any approximations in getting to here.
And the flux in this case is half of u squared at the cell boundaries.
Now this is the point we have to start making approximations.
Because unlike in finite difference, we have the value of the function x at [?
these ?]
points.
But here we don't.
We don't have the solution at either Lk or Rk.
Right?
We don't have the solution at the control volume boundaries.
We only have the cell average solution values.
So we have to approximate, we have to do the finite volume approximation.
I have to approximate the flux at rk.
I have to approximate it as F of a function.
The most simple cases u bar of x Uk minus one and u bar of k. F at-- sorry, this has to be at Lk because I'm on the left.
Rk has to be approximated as a flux of Uk and Uk plus one.
So let me draw what is happening here.
So I have a bunch of cells.
This is, let's say, k minus 1.
This is k. k plus one.
k plus two.
Right?
OK.
So this point is the left boundary of k minus one.
Here is the right boundary of k minus one.
But it is also corresponding to the left boundary of k. This point is the right boundary of k but also corresponds to the left boundary of k plus one.
This point is the right boundary of k plus one but is also the left boundary of k plus two.
This point is the right boundary of k plus two but also the left boundary of k plus 3.
I need to attach a flux value at each of these boundaries in order to solve this equation.
For this equation, I need to not only resolve the cell average in each cell, I need to have the flux at these boundary points.
But I don't have the solution at these boundary points.
All I have is the cell average inside of the cells.
How do I approximate the flux at these boundaries?
It makes no sense to approximate the flux at this point using the cell averages around that cell boundary.
At the left hand side of [INAUDIBLE], and this is [?
like the cell interface ?].
On the left hand side interface is [INAUDIBLE] right of the interface [INAUDIBLE].
So I need to approximate the flux using the values.
OK.
The easiest way to approximate it is what we call first order upwinds.
And upwind means the reverse of the direction towards which the solution goes.
OK.
In this situation, can somebody tell me what is the upwind direction?
Yeah?
[INAUDIBLE]
This direction is the upwind direction?
Always?
It depends.
Right.
The upwind direction in the Burgers equation actually means almost all [INAUDIBLE] conservation laws ends on the solution.
In this part, the solution was here and now moved here.
The solution is [INAUDIBLE] towards the right.
It's kind of blown, like blown by the wind, and that comes from the left.
So what is the optimum direction for this part in the red region?
It's towards the left, right?
What is the optimum direction over here in the blue region?
Towards the right, right?
The wind is coming from the right.
So the wind is going this way, if this were the wind.
So upwind direction is looking where the wind is coming from.
And in the Burgers equation, the upwind direction is towards minus x if u is positive.
It is plus x if u is negative.
This is upwind.
It makes a lot of sense to do upwind because that's where the solution comes from, right?
So you want to-- I'm going to talk about upwinding, why upwinding makes sense really later when we talk about stability.
OK.
So upwinding schemes.
At this point, I just need to tell you it gives stability.
And in the upwinding scheme, I'm just going to say F of k-- so F at the interface-- so I'm going to define F of k plus 1/2, defined as F at the right boundary of the k value, which is also the same as the left boundary of the k plus one cell.
It is the flux at the interface between the k and k plus one cell.
Right?
I call it k plus 1/2, because it's kind of a halfway between the k and k plus 1 cell.
Right?
And the first order upwind scheme is to compute F of k plus 1/2 is equal to 2 cases.
It is either F of u bar at k, which in the Burgers equation is just u bar of k squared over 2.
If the minus direction is the upwind direction, I'm here again at-- I'm going to draw the same cells again.
So k minus 1, k, plus one.
OK.
I am looking at k plus 1/2, which is the interface between k and k plus 1.
I am going to use the average method of k to approximate here.
If k is the upwind direction of the cell, which means if the wind travels towards the right, if locally u is greater than zero.
And here locally-- I can say locally u is greater than zero if the average between the two cells is greater than zero.
And else, I'm going to approximate it using the upwind direction, which is now towards the right.
It is this.
Does it make any sense?
This is what I'm going to write in the code.
OK.
This is what I'm going to implement in my code.
That is, in the [INAUDIBLE] code [INAUDIBLE].
The F at interface is either the flux at the left value or the right value where [?
it is ?].
Any questions on this?
No?
OK. Let's do it then.
What I'm going to do is I'm going to pull up a skeleton.
And I'm going to ask you to fill in the blanks.
So I'm going to go to the shared border.
Open your computers if you haven't.
I'm going to create an order 2014.
Today is the 10th, right?
OK.
I'm going to start from scratch.
I'm going to make a function just called the du dt Burgers.
OK. And like our-- so we're assuming PDEs.
But as long as we can convert the PDEs into a set of ODEs, we know how to integrate it at this point.
I'm sure you guys all can.
So what is it going to be?
The input is the u bars.
And the output is d u bar dt.
And what I need to do is I need to compute from the u bars the du bar dt.
So what do I do first?
I'm first going to do F bar is equal to u bar squared over two.
What am I doing here?
[INAUDIBLE]
Computing the flux.
This is the only thing I need to change if I switch from the Burgers equation to some other equation.
OK?
This is really defining the flux.
If I have a flux that is equal to another form, this is what I need to change.
OK. And here, I'm going to have you fill in the blanks.
Compute the flux at cell interfaces.
So I need to compute F. Let me call this F of [INAUDIBLE] k. So this is F at k plus 1/2 equal to blah, blah, blah.
This is what you need to do.
OK?
And then what I'm going to write down is this is the finite volume method.
This is the same for all finite volume methods.
What is it?
It is the d u bar dt equal to-- according to our formula-- du bar dt is equal to one over delta x times the F, the flux, at the right boundary of k, which is k plus 1/2, minus the F at the left boundary of k, which is F of k minus 1/2.
Right?
So this is F k plus 1/2 minus F k minus 1/2 divided by delta x.
And I need to compute F of k minus 1/2.
It's basically a [INAUDIBLE] of F k plus 1/2.
Let's assume they are periodic boundaries for now.
So how should I compute F of k minus 1/2 from F of k plus 1/2?
I'm going to use circshift, right?
And which direction should I shift it?
So I already have F of k-- I already have F of k plus 1/2 over here.
I need to have-- so if I already have F of k up here, I need to shift it towards the right to get-- so that's the same point I have, F of k minus 1/2.
So we assume-- let's assume it's a row vector, so that when I shift it, I shift it to 0 and 1.
OK. And let's assume my boundary goes from 0 to 1.
OK.
So if I have a boundary that goes from 0 to 1, my delta x is equal to the length of the domain divided by the length of u bar, how many control volumes I have.
So that's my delta x.
The size of each control volume?
I have F of k plus 1/2, F of k minus 1/2.
Then I get the du bar dt.
OK. Go ahead and just copy what I wrote here.
Just rename it to whatever.
Just rename it to dudtBurgers underscore your name.
You can do it in the same directory.
And tell me what you get.
All right.
Anybody have anything I did wrong?
I just wrote a script.
So it's writing [INAUDIBLE].
So this is basically making [INAUDIBLE].
OK. And using all the input [INAUDIBLE].
So [INAUDIBLE].
All right?
So I'm [INAUDIBLE]
[INAUDIBLE]
OK. You can try.
All right
[INAUDIBLE]

[INAUDIBLE].
The function in--
Oh.
Oh.
OK. Oops.
I added just--
[INAUDIBLE]
OK.
So this is-- let's see.
[INAUDIBLE] So I goes from one to the length of u bar minus 1.
If u bar is [INAUDIBLE].
You also put I here
[INAUDIBLE]

It's be a good idea
Yeah.
I think I need to close this so that I don't get a [INAUDIBLE].
OK.
It's updated.
OK. Let's run the script again.
The driver.
Run.
Ah.
OK.
This is-- [INTERPOSING VOICES]
OK. Update it again
[INAUDIBLE].
[INTERPOSING VOICES]
Oh, you must-- OK.
So that's my problem.
I need to change it so that I [?
deleted ?]
a column vector.
Let me [?
delete ?]
a column vector and [INAUDIBLE] a column vector.
OK. Now let's see.
Ah.
You're returning a vector of length 63.
But the length I should get is 64
Oh, right.
So it's returning [INAUDIBLE]
Ah.
We are assuming the domain is periodic.
So at the last control volume, if we need to take towards the right, we should be taking the first value.
Right?
OK. Let me see, I see to [?
copy ?]
the network, oh
[INAUDIBLE]
Into the [INAUDIBLE] [?
space ?].
Let me see.
Hm.
OK. For the last cell, here I think I [INAUDIBLE] from the [INAUDIBLE], I think that will end after-- so let me see.
For r equals from 1 to 10, i n. Yet, if I do i again, it is still 10.
It is not 11
[INAUDIBLE]
Yes
[INAUDIBLE]
OK. [INAUDIBLE] update.
[INAUDIBLE] update.
OK. Let me do this again.
Oh.
I need a minus sign here.
Right?
[LAUGHTER] Is that right?
OK.
I need a minus sign in my-- OK.
I need a minus sign in my script because--
Yeah
Right?
Is that right?
Let me see.
So it is equal to the right value minus the left one, which I may be--
Yes, it would be--
Should have been the left value minus the right one.
Sorry.
I have been deriving it incorrectly.
So yes.
So for the time derivative of the average is equal to what comes in from the left minus what goes out from the right.
So I need a-- yes, I need a minus.
So don't change your code.
I just put a minus sign here.
All right.
Now it looks like it's behaving a little bit weirder at the end.
Otherwise it seems to be working
[INAUDIBLE]
Let me-- OK Right.
So we do see a shock wave developing, right?
So let me change the delta t to be a little smaller.
And then run it again.
Ooh.
Right?
That's the correct behavior of the Burgers equation.
Right?
OK.
Very nice.
So who else-- I think the-- let's see.
Yeah.
OK. Let me run it again.
All right.
Great.
That's great.
And let me see if I can break it without betraying initial conditions.
And let me take it as x is zero equal to draw periodic function.
Do I have that?
Oh no.
I don't have that in here.
So I'm just going to copy it
[INAUDIBLE]
Draw periodic function
[INAUDIBLE].
[INTERPOSING VOICES] [INTERPOSING VOICES]
Let me do that.
OK. Let me do something.
Anyway, OK.
So I see a lot of you are doing good work.
Anybody else?
I can try your solution.
OK. Ah.
[LAUGHTER] OK.
I think I'm going over time.
I had a lot of fun trying your scripts.
But let's meet on Wednesday and talk about more of the finite volume method.
So if you want to stay, I'm going to try a little bit more of your scripts.
[INAUDIBLE] Oops
[INAUDIBLE]
Oops.
All right.
OK .
Anyway so once you have a script, you can always try my driver and see how they work.
All right.
I'll see you on Wednesday.
Yes.
If you didn't send me a project here, please put it over here.
And the homework [INAUDIBLE] with the homework.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at osw.mit.edu
So today, we're going to be continuing finite volume.
We started on finite volume on Monday.
And just a very quick recap, finite volume doesn't solve all the differential equations, but it solves a class of differential equations that we care most, that is conservation laws.
Conservation laws are differential equations that can be returned in this form.
And today, I'm just going to be more specific.
Let's say, rho is the density of a conservative quantity.
The time derivative of rho plus the divergence of a flux, which is a function of rho, is equal to 0.
So this is a general conservation law when there is no [INAUDIBLE], when there is no generation or it's appearing off any material within the control volume.
Everything that increases has to come from outside the control volume.
Anything that decreases has to go out of the control volume.
And this is the differential form.
And in one dimension of cos, the divergence is just spatial derivative.
It's just a partial x of F of rho.
And today we have going to be solving one of the equations like this.
And in order to apply finite volume, we need to make it an integral form.
We have to really integrate this in space.
And instead of a divergence operating space, we get this-- d dt of an integral over any control volume.
Let me describe it as omega.
Omega is a control volume.
So we are basically integrating this whole thing in omega.
And this is the first thing we get.
And the second thing we get is that now we are doing this in impossibly multiple dimensions, like last week [INAUDIBLE].
Now if you integrate the divergence of something-- you're integrating a divergence of a vector field-- in a control volume, what do you get?
A listing something like the integral of a divergence of a vector in a control volume equal to something [INAUDIBLE].
Something can relate a divergence.
[INAUDIBLE] the divergence 0.
[INAUDIBLE] is so cool.
There are two theorems in method popular that I can see that are very important, both relating a spatial derivative to a spatial integral.
And the divergence theorem is the more fundamental one.
I can feel a more fundamental one, because it basically tells you the spatial integral of a divergence can be expressed with something that doesn't involve derivative at all.
It can be represented as the integral over the surface of the control volume.
The surface normal starts with just the flux vector is equal to 0.
So divergence theorem says that the integral of this is equal to this-- divergence theorem.
So what you see is that this is the 2D or 3D version of the integral form of the conservation law.
What we are saying is that now, again, this is the total amount of mass, or stuff, within the control volume.
And today we're going to be doing half.
We see that the total [INAUDIBLE] half within a control volume.
And this is the flux through anywhere in the surface.
So there is a normal.
And this normal is also at normal of the control volume.
So n points towards the outset.
So this F of rho is the flux towards the outset.
That makes sense, because if F points outward-- if the flux is towards the outside-- then this term would be positive, because F is going to be aligned with n. This term is positive.
And the time derivative will be negative because things are going out, the stuff should decrease.
If asked if the flux vector points inward, then it is going opposite to the outward point of normal, opposite to n. And this integral would be negative.
So if this thing is negative, then we have to add to 0 so this can have some positive, which means the amount of stuff is going to increase as a function of time.
Is it clear?
This is now generalizing that in 2D or 3D.
And of course, we can still write that in 1D.
In the one dimensional special case, what is this partial omega?
Partial omega is the boundary of omega.
What is the boundary of the one dimensional control volume?
The point, or two points-- two points, right?
And the two points, I have R and L. And outward pointing normal is going to be just a 1 at the right hand side at R, is going to be minus 1 at L. So I'm just going to be saying this is F of rho at R minus F of rho at L is equal to 0.
Or as we said on Monday, the time derivative of total stuff is equal to the stuff coming in from the left minus the stuff coming out from the right, if you moved both of this stuff [INAUDIBLE] side by side.
So this is the kind of differential equations we are solving, and the scheme is finite volume.
And finite volume basically follows exactly this formula.
Instead of integral over control volume, we are storing the average of the control volume, which is simply the integral divided by the size of the control volume.
It could modify the cell average with the size of the cell, with the integral.
And in order to think of the DDP of the cell average, you just divide this also by the cell volume, or cell size.
So you get the evolution equation that allows you to control using finite volume.
Now this equation is not approximate This is exact.
What we need to approximate in finite volume is this flux.
This flux is a flux evaluated after the cell boundaries or cell [INAUDIBLE].
And the finite volume is [INAUDIBLE].
You only have the cell average.
So you need to approximate the interface value from the cell average.
And that is the approximation we have been talking about.
So that's kind of a review.
The first new thing we're going to talk about is, actually, why do we use finite volume.
We use finite volume because it captures shock waves, because if there is discontinuity in the solution, finite difference doesn't work.
I'm going to show you with a-- what?
Cancel.
I'm still testing around with the code you wrote.
But let me go to today's directory, and I'm going to show you two demos.
I'm going to show you a simulate-- so first, I'm going to tell you what this is doing.
So I have two functions, one is simulate fv, and one is simulate fd.
So these two functions are identical, except for this single letter.
One is fd, one's fv.
But otherwise, I'm just doing the same.
I'm using [INAUDIBLE] 45 to integrate in time.
Conditions on that I'm [INAUDIBLE] having space, using either finite volume or finite difference.
I see there is a question on what this ''at'' do.
This "at" is basically take a function I wrote, which is either du dp fv or du dp fd and feed that into the ODE 45.
Anytime you use ODE 45 you need this "at" to make it work.
Now the difference between finding fd and finding fv is that in this we're using the standard finite difference to solve the differential form of the Burger's equation.
So du dp is equal to minus u times du dx.
This is du dx.
Now you should be familiar with this.
I'm taking u minus a specific version of u-- this is assuming [INAUDIBLE] conditions-- divided by dx.
So this is a [INAUDIBLE] space on a difference of du dx.
So here I am solving the differential form of the equation of the finite difference.
In finite volume-- let's review what we do in finite volume.
du dx [INAUDIBLE] flux.
Now, this is not using the differential form, but using the [INAUDIBLE] form because we are computing the flux.
And here, I'm assuming the opposite direction is always the left.
Here, I'm assuming like I have an initial condition that is opposite, and it's easy to illustrate using a case like that.
So the wind always come from the left, u is always positive.
That means when [INAUDIBLE] it is a minus [INAUDIBLE].
And I'm doing du dt equal to flux at the left minus flux at the right divided by dx.
So this is finite volume.
So let's see, what is the key difference between the results of these two schemes.
And in either one, I'm starting with something positive, so sine x plus 1 over 2 because I'm assuming also in finite difference and finite volume, the wind comes from the left.
So that I'm doing [?
backwarding ?]
space is a finite difference or finite volume.
So I have a positive solution.
I told them to see how it does.
So I'm going to [INAUDIBLE] on a different sort, and somebody new will tell me, what is wrong with a finite difference solution.
Watch carefully.
I'm going to start.
You need to tell me what's going wrong.
So it looks like any Burger's equation.
I mean, we have a shock wave.
Things are going.
So what's wrong with this?
Is anything wrong?
Solution [INAUDIBLE].
Let me run again.
Anything looks wrong?
Many different circles it'd be tracing, but it's not
[INAUDIBLE]
You have sharp eyes.
Now, let's look at the finite volume solution.
Exactly the same, but using finite volume.
What's the difference?
This [INAUDIBLE] the shock wave is correct in this case.
While the shock just [INAUDIBLE] it forms, the finite difference just stands there.
The speed of the shock wave is finite difference, is wrong in finite difference.
So by using the differential form of the equation and disregarding the conservative form, you are not guaranteed to have the correct speed of the shock wave.
It actually solves the other part correctly.
The only part that it doesn't capture correctly is how the shock wave is supposed to behave.
And if we use finite volume, because we are looking at the conservative form of the differential equation of this form, the other integral form.
This form of [INAUDIBLE] is to have shock waves.
Solve this one, and you take the divergence of the [INAUDIBLE] that is discontinuous, for it acts derivative of something that is discontinuous.
That is no longer a correct thing to do.
And therefore, you get wrong behavior when there is discontinuity like shock waves.
Any questions?
Oh, and by the way, another thing is this.
I should let you look.
In addition to the shock wave, there is another thing that is not right in the finite difference.
Lets look again at the finite volume.
The Burger's equation's a conservation law, which means that [INAUDIBLE] under the curve should be [INAUDIBLE].
The integral of the solution u, of course, the whole domain, should be the same, because anything that comes out of here should come back into here.
It's like when you step out of the door, you come back.
You're not disappearing.
But if you look at the finite difference, at this point, it [INAUDIBLE].
Things just disappear out of nowhere.
The solution is not conserved.
So two things that finite difference is missing.
One is, it doesn't have to behave as a shock wave.
Two, it is the solution is not conservative.
Any question on why we do finite volume?
So this is why we do finite volume.
And the next question is another question, probably, you have when you are going through the last lecture and reading through the material is, why the hell do we do upwind?
Why do we look at the upwind direction?
So the upwind of flux is this.
So we look at a few volumes.
k minus 1, k plus 1.
The finite volume, we store the average value in the volumes.
So we have k minus 1/2 and k plus 1/2 as the volume interfaces or the cell interfaces.
And the upwind flux is basically saying F at k minus 1/2 is either the F of rho at k minus 1 or F of rho at k, depending on where the wind comes from, depending on if, should say-- let me use C as the speed of the wind.
So if c is greater than zero, the winds have positive speed, which means the wind goes towards the right.
And then you look at the upwind direction, you should be looking at when you are computing the upwind value of the plus [INAUDIBLE] minus 2, you should look towards the left, [INAUDIBLE].
So when k is greater-- so when C is less than 0, that means the speed of the wing is negative, opposite direction is towards the right.
The winds are [INAUDIBLE] opposite direction for this, you should be looking at the right and use the k value instead.
And here, c is what?
c is this.
c is the c that appears in this.
c is the value that appears in the differential form of the equation, over here.
So as you can imagine, if we have [INAUDIBLE] like the [INAUDIBLE] vacuum equation we have, this c exactly determines whether your solution moves towards the left or moves towards the right.
So if this c is positive, the solution will go to the right and winds go towards the right.
If c is negative, it goes the opposite direction.
But in general, in a conservation law, how do we convert our conservation law, which is partial F rho partial x equal to 0, into to that form?
But if we convert this into this one, what is c?
Can we always convert the conservative form into this, like a vacuum scheme looks like, leaving a vacuum looking like form?
Just take the derivative
I just take the derivative, exactly.
If I take c equal to partial-- not partial.
I do too many pd's.
dF d rho, then I get my C. In the Burger's equations, c is [INAUDIBLE] rho because F is equal to half of rho square, right?
So when I take derivative, I just get [INAUDIBLE].
As you see if you're looking at all the equations, you'll see what we did [INAUDIBLE].
And by the way, you may be thinking of why did I use these here.
Because if you look at a gas dynamic equation, the C you get is going to be the speed of sound.
OK. OK. Why do we need do upwind?
I mean, the real mathematical reason is something I'm going to talk about [INAUDIBLE].
But, like, I think Dr. [?
Vikram ?]
[?
Dag ?]
is going to be giving the lecture right after the midterm.
But he's going to talking about stability.
And this is for stability.
I'm just going to demonstrate it empirically in the shared folder we had-- not this one-- we had from last lecture.
OK.
So I had a driver here.
And I have a bunch of people's schemes.
Who's scheme is working by the way?
Who knows?
Which one of them should I use?
Who kind of knows more or less your script is working?
OK. Yours?
OK. [INAUDIBLE]
[INAUDIBLE]
OK. All right.
So I'm going to change this to-- OK.
Thanks.
So I can run it here.
And I get a solution.
I mean, you probably didn't treat the boundary condition correctly, but that's fine.
So there is something on the boundary that is not working correctly.
I think that you might have set the [INAUDIBLE] here to be 0.
So you just set the [INAUDIBLE] here to be 0 instead of [INAUDIBLE] the [INAUDIBLE] blowing with the [INAUDIBLE].
Because I'm having a [INAUDIBLE] find the word, so [INAUDIBLE], right?
[INAUDIBLE] But that's fine.
OK.
So I think it should be relatively easy to modify.
F-- yeah, yeah, yeah.
OK.
So I just need to treat the last one.
So F_k_plus_half(and), right?
The last one is equal to-- OK.
I will just cheat here, F_bad_k(and).
Because I know my solution is positive, so I just take the left one.
OK.
So here is k_plus_half
That should have taken care of--
Oh, that should have taken care of that
[INAUDIBLE]
[INAUDIBLE]
Sorry?
k_plus_half of i minus-- that is end, right?
Yeah.
That's [INAUDIBLE]
I think you shouldn't be-- F_bar_k, right?
OK. And else it is equal to F_bar_k(1), right?
OK.
So let's see if that works.
Oh, that works.
OK, nice.
OK.
So we have this scheme over here.
We have an upwinding scheme.
So as we did in finite [INAUDIBLE], will know that backward in space or forward in space anything you [INAUDIBLE] have a one-sided [INAUDIBLE], you get first order accuracy.
But if you take essential difference or take the average of left and right, you get secondary accuracy.
So why don't we use that?
I'm going to change your code to do exactly that.
I'm going to instead of doing all this conditional thing, I'm just going to do this is equal to half of F_bar_k and F_bar_k plus 1.
This thing's divided by 2.
So I'm not going to be using your conditions.
I'm just going to do k_plus_half equal to actually k_plus_half, k_plus k_plus over 2.
And instead of doing this condition-- apologize first for going to make the [INAUDIBLE] unstable.
But k_plus_half(and) is going to be F_bar_k(and) plus F_bar_k(1) divide by 2.
So now if I do this, I always set the interface value to be the average between the two [INAUDIBLE].
What do I get?
Now, it looks like correct.
Oh.
All right.
So what we see here?
We see here in the beginning, it looks fine.
Let's look more carefully what actually happens.
Let's set dt equal to 0.001 to be more-- let's see what happens.
OK. Usually, the evolution- let me [INAUDIBLE] it, so it doesn't [INAUDIBLE].
[INAUDIBLE] I'm going to do ylim(0,2), so that doesn't [INAUDIBLE].
OK.
In the beginning, we see it is behaving correctly.
And I am probably getting secondary accuracy, right?
But as soon as the [INAUDIBLE] is formed, something bad is going to happen.
Now, [INAUDIBLE].
All right.
[INAUDIBLE] So, yeah.
So we've got these calculations.
And it looks pretty much like in [INAUDIBLE] analysis you get this [INAUDIBLE] phenomenon, like right or wrong [INAUDIBLE].
So this is actually why we need to take an upwinding scheme, especially if you have a nonlinear differential equation with potentially shock waves.
I mean, if you go to math class, I can probably talk more about shock waves being a dissipated phenomenon and a second order scheme, like essential difference that has no numerical dissipation.
And you need the numerical dissipation [INAUDIBLE].
I'm just going to demonstrate numerically.
This type of thing is going to happen if we don't do upwinding and you have shock wave.
So for something that doesn't have shock wave it's probably fine as long as you use an appropriate time integrator.
OK.
So today-- I mean, this is why upwinding-- we are going to be doing something like this.
So I tried to stimulate distance.
This is a lot of parts, right?
And if you do a simulation of individual parts, yeah, you can do this.
But [INAUDIBLE] this [INAUDIBLE] it really looks like-- I mean, each [INAUDIBLE] more looks more or less like a molecule.
So you really get a continuum of cars or [INAUDIBLE].
And one of the ideas is to instead of treating each car as an individual agent and simulate that, we just look at from a macroscopic point of view and simulate this like gas dynamics, where each car is like a molecule.
OK. And so a macroscopic description of the cars on a highway is just P as a function of x and t, right?
So x is space.
t is time.
And P is the number of cars per length of the highway.
So P is a converse quantity, because cars are converse, right?
Cars are not disappearing into [INAUDIBLE].
Yeah.
Cars are not disappearing as airlines sometimes do
[INAUDIBLE] [INTERPOSING VOICES]
OK.
So we are also assuming the velocity of cars depends on the density.
Well, let's first say so P equal to 0 is an empty highway.
P equal to 1 is this.
OK. P equal to 1 is like a parking lot.
P equal to 1 is like the maximum density you can pack onto a highway.
OK.
So this is P. And we are assuming the velocity of the cars is a functional P. So again, a velocity equal to 0 is like this.
So this car cannot move when the density is at [INAUDIBLE].
And let's pick the unit of velocity, so that velocity equal to 1 is something like 65 miles per hour, where you would have-- or 80-- where you would be driving if there is no other cars.
Or 85.
OK. OK.
So what would be an easy way or intuitive way you would model u as a function P?
[INAUDIBLE]
Yeah.
It can be more curved.
But let's make it simple.
Did you want to [INAUDIBLE]?
So would you [INAUDIBLE] code [INAUDIBLE] like something like this?
Or would you [INAUDIBLE] coding [INAUDIBLE] like U(P)= 1 minus P?
This captures the right behavior.
So when P is equal to 0, you drive at 85 miles per hour.
When P is equal to 1, even if you want to drive faster, you're stuck, right?
OK. Now, here's my first question.
What is the flux?
What's F(P)?
[INAUDIBLE]
Integral--
[INAUDIBLE]
Yeah.
I know what you're thinking.
You are thinking about U being the d f in P, right?
So it has to be the integral.
But that's not true.
It's easier than that.
You are thinking more-- you're too smart
[INAUDIBLE]
P times U.
That's that correct answer.
Can you say why that is this case?
[INAUDIBLE]
That's [INAUDIBLE].
Yeah.
Because flux is number of cars, the amount of converse quantity that flows through a point per unit amount of time, right?
So if you just [INAUDIBLE] the control volume like this, this is my-- oh, I shouldn't use green, because you can't see.
If you true the control volume like this-- and the flux of this [INAUDIBLE] is the number of cars that goes through this line per unit amount of time.
So this is cars that have density of rho goes through the [INAUDIBLE] U.
Of course, the flux is going to be P times U.
The flux is going to be P times U, which is P minus P squared in this case.
Because U is 1 minus P. P times U is P minus P squared.
Is it clear?
Any questions?
So we derived our differential equation, dP dt plus d of P-- I'm just going to write F(P), it's easier.
F(P) dx is equal to 0 where F(P) is equal to P minus P squared.
OK. Now, what is C?
What is the [INAUDIBLE] characteristic C?
Now, this becomes highly [INAUDIBLE].
What is C?
1 minus [INAUDIBLE]
It's 1 minus 2P.
It is equal to dF dP, right?
So derivative of P is 1 [INAUDIBLE].
OK.
So we have a relation like this.
We have a relation of F versus P. P is all the way from 0 to 1.
F is something like this, where the maximum takes 1/4.
OK. And we have a C that looks like this.
We have C that is equal to 1 when there is no cars and goes all the way to minus 1 when there is a parking lot of cars.
This is 1 and minus 1.
OK.
So what is happening?
C, again, is the speed of the local solution.
It is intuitive that when there are no cars, things move at 85 miles per hour.
So the [INAUDIBLE] should be 1.
What happens here?
When there is a parking lot of cars, things move backwards at 85 miles an hour?
[INAUDIBLE]
Yes.
It's kind of the-- so this C here is not-- I mean, if you look into physics you've got two kind of speeds, like a group speed and a [INAUDIBLE] speed, right?
So here, the cars move forward always have a positive speed.
But it doesn't mean if you look at cars in the backwards [INAUDIBLE] point of view, it doesn't mean the wave that appears in the flow of traffic is going to move forward.
Sometimes you are going to see waves actually moving backwards.
I mean, we're waiting to see some examples when it actually solves that numerically.
OK.
So the task for you is to hold up a simulation of this thing in a periodic [INAUDIBLE] solution.
So let's say this is our highway.
This is our highway.
It just goes in circles, right?
Assuming you have a bunch of cars driving in circles
I could never get off the [INAUDIBLE]
[INAUDIBLE]
Huh?
NASCAR, yeah, yeah, right.
Yeah.
We have a lot of cars driving in home circles.
And let's just [INAUDIBLE] how these work.
And then at sometime, we are going to put a red light over here.
And say, OK, the cars stop here.
What is the right way to model that numerically?
You can set the flux [INAUDIBLE] to 0, right?
I mean, at some point, that's a [INAUDIBLE] simulation.
And for example, time equal to 1, that's [INAUDIBLE] 0 [INAUDIBLE] before the red light.
And at some point, let's set the light [INAUDIBLE].
But let's do something simple in the beginning.
Let's just [INAUDIBLE] the solution of [INAUDIBLE] with no red lights or anything like that.
I'm going to give you the initial position.
And we'll see how it evolves.
And then we are using final volume.
And you are free to take-- the [INAUDIBLE] case we had in the last lecture is in the shared folder.
And I'm just going to make another shared folder here.
OK.
I'm just going to take a [INAUDIBLE] and the du dt.
I'm just going to take the [INAUDIBLE] stamp [INAUDIBLE] to this.
Because I know it's working.
And [INAUDIBLE] start by modifying [INAUDIBLE].
So instead of u_bar, you're going to solving for P_bar.
And what is the first thing you would want changed in this?
[INAUDIBLE]
Oh, yeah.
Yeah, make it upwind.
Yeah, right.
To make it upwind there are [INAUDIBLE].
Huh?
[INAUDIBLE]
It's no longer that equation.
The equation is [INAUDIBLE].
But there is only one line we can change to solve for a different equation and find the volume.
What line is that?
[INAUDIBLE]
Second line.
The only thing that satisfies a different equation is the flux.
You change the flux, you are now solving a different equation.
OK. Of course, another thing is what is upwind direction [INAUDIBLE].
In [INAUDIBLE] equation, we know U is positive plus [INAUDIBLE] the right.
U is negative [INAUDIBLE].
Now, this is different.
Now, this is different, because we have [INAUDIBLE].
What is left and what is right all depends on the [INAUDIBLE] instead of the [INAUDIBLE].
So any questions?
OK.
If you are done with the-- If you get the solution going, try to put a red light somewhere.
Let me kind of illustrate how I should modify the driver.
So I have this here.
That, I didn't call it [INAUDIBLE].
Because it's no longer [INAUDIBLE].
Let me [INAUDIBLE].
trafficDriver.
OK, trafficDriver.
And let me see.
So in this ODE 45, I'm going to set t0=(i-1) times dt and t1=i times dt.
So this is like the beginning and end of this interval.
And when I'm solving it, I'm going to be solving it from t0 to t1.
So now the difference is if I look at-- so let me do this.
OK.
So I just want to be able to passing the actual time.
So I'm just going to be passing also time into this.
So I also have a time into this.
And then now if-- OK, so let me just go through what you need to modify.
So first of all, I'm going to modify this, which is the different flux.
Right?
Instead of half of u squared, I have u minus u squared.
OK. k_plus_half is still the same.
I'm going to-- so if I don't modify anything, I think it all wrong.
But like once a shock wave forms, it'll go unstable.
So let me try it.
I doesn't even wrong.
[INAUDIBLE] dimension matrices being concatenated are not consistent
[INAUDIBLE]
Oh, OK.
I'm [INAUDIBLE] different [INAUDIBLE].
Yes, thank you.
I save this
[INAUDIBLE]
Oh, i'm solving-- yeah-- from t0 to t1.
Yeah, thank you.
OK.
So this thing goes.
And we're going to see [INAUDIBLE]-- wait.
I shouldn't initialize it to 2, right?
Because 1 is parking lot.
What is 2?
OK.
So I need to also modify the initial condition.
So what I can do is I can set this, let's say, divide by 4.
OK.
So this is going to be my initial condition [INAUDIBLE].
OK.
I also want to-- my ylim, when I show my solution, I just want to go from 0 to 1, right?
OK.
So let me run again.
OK.
So this is how my solution looks like.
This part goes really fast.
[INAUDIBLE] unstable.
And I need to change this to upwinding.
But we have the [INAUDIBLE] part of the solution correct.
Every [INAUDIBLE] towards a positive direction when the [INAUDIBLE] move faster.
That's [INAUDIBLE] car density, cars move slower.
So sometimes, a shock wave is going to form.
Because the faster parts is going to [INAUDIBLE] to the slower parts and decelerate.
So it becomes a shock wave.
So let's start to modify this to an upwinding scheme.
We computed F_bar.
And we have a c. c is equal to 1-u_bar times 2.
OK. And the c at k_plus_half is equal to c plus-- OK. Let me do like this.
So in the condition, instead of u_bar, I'm going to put c. OK.
Here, too.
Instead of the u_bar, I'm going to put c. OK.
So this completes my upwinding.
And let's try it again.
We still have the solution.
But the lower density goes faster.
High density goes slower [INAUDIBLE] high density, but now it's [INAUDIBLE] stability.
So that's good.
And if you want to get better solution, I can change this to 256.
And I round this.
And I [INAUDIBLE] the top [INAUDIBLE].
OK.
So now, let's say at this point, so at either the beginning [INAUDIBLE], I have time equal to 1, we get a red light.
And at time equal to 2, we turn the light to green.
[INAUDIBLE] there.
OK.
So we do that.
[INAUDIBLE] then that [INAUDIBLE] what we need to do to make that happen.
The key thing is now you have time.
You have t available to you.
So you can basically do a conditional statement on t. If t is greater than 1 and t is less than 2, set the red light.
[INAUDIBLE]
What about [INAUDIBLE]?
Oh.
Thank you.
That is wrong.
So it should be F_bar_k.
Yeah, it should be F_bar_k.
Thanks.
Yeah.
I should just rename it to something else, so that-- I'm just going to rename it as 2t.
All right
[INAUDIBLE]
Which one [INAUDIBLE]?
[INAUDIBLE]
Sorry?
[INAUDIBLE]
This one?
Yeah.
[INAUDIBLE]
Yeah.
[INAUDIBLE] plus 1, right?
So because I'm looking, I [INAUDIBLE] the interface.
I want to look at the interface [INAUDIBLE] decide which side I should upwind the flux at k_plus_half.
In order to make a decision, I want my [INAUDIBLE] speed at k_plus_half.
And the right way to [INAUDIBLE] the other [INAUDIBLE] here k_plus_half is take the average of the [INAUDIBLE] speed [INAUDIBLE] at k(and) k plus 1.
Just to make the code easier to understand, I just want to change-- I think I want to replace-- oh.
I think I want to replace my i with k. [INAUDIBLE] All right.
Yeah.
So instead of-- it's easier to replace [INAUDIBLE] k_plus_half than [INAUDIBLE]
[INAUDIBLE]
Why did I half t?
I wanted to half t, because what I like you to do now is to set a [INAUDIBLE] when t is [INAUDIBLE].
OK.
So what happens on the red light?
[INAUDIBLE]
Huh?
[INAUDIBLE]
Yeah, the [INAUDIBLE] goes to 0.
That's one way to say it.
Another way to say it is no cars can pass, right?
If no cars can pass, then the [INAUDIBLE] pass through the red light is what?
0, right?
So what I can do here is that before I compute k minus half, I just want to say if k is greater than 1 t is less than 2, F_k_plus_half at the end-- so k_plus_half-- the last one is basically the right hand and also the left hand-- is equal to 0.
I think I [INAUDIBLE].
OK. That's it.
That's what I need to do.
Now, in the trafficDriver, let me set it to-- ah, ah, ah.
Let me copy this trafficDriver for the [INAUDIBLE].
Yeah.
I think we're overwriting [INAUDIBLE].
And it's going to save trafficDriver [INAUDIBLE].
OK. Now, instead of 1 over dt, I'm going to simulate it for five time units, so that I actually see the light.
OK. We see the same thing here [INAUDIBLE].
At sometimes, the light [INAUDIBLE].
Yeah, OK. See, the lights is in effect here.
All the [INAUDIBLE] simulates here.
And then we have even [INAUDIBLE] that actually goes backward.
OK. Now, there is no cars moving.
All the cars are stopped over there.
So this is a parking lot while there is no cars here.
And at [INAUDIBLE] equal to 2 or [INAUDIBLE]
[INAUDIBLE]
Did I-- did I-- what's happening?
OK.
If t is greater than 1 and t is less than 2--
[INAUDIBLE]
And never goes passed 2, are you sure?
[INAUDIBLE]
OK.
I think I know what's happening.
So I actually have come to a situation where-- hm, let me think
[INAUDIBLE].
What
Yeah.
Let me do this.
I'm actually encountering a situation that cannot be handled by this [INAUDIBLE].
What is happening is that [INAUDIBLE]-- let me do this.
I think I can change this around by doing this.
I can change this around by not letting the red light to stop for that long.
So I only let the red light to-- if I only let the red light go from 0.2, I think I'm going to get rid of the problem.
The problem is that when the red light happens here, the flux-- the P at one side shock wave is 1.
At the other side of the shock wave, the P is equal to 0.
So both values of P [INAUDIBLE] equal to 0.
Because the flux is P times 1 minus P or P minus P squared.
So the other P is 1 or zero.
The flux is equal to 0.
That's why no matter if you take upwinding or downwinding, you'll get 0 flux.
So this is a case where [INAUDIBLE] upwinding wouldn't work.
It's like a singular case where [INAUDIBLE] upwinding wouldn't work over here.
So what I'm going to do here is am going to set something artificial.
And let's say when there is a red light, I think I'm just going to set the flux equal to 0.01.
So that there is a small leakage of cars over the red light, so that we actually want to see what is going to happen [INAUDIBLE]
[INAUDIBLE]
Yeah.
So if you take [INAUDIBLE], you're going to learn about the entropy conditions of [INAUDIBLE].
And the entropy condition tells you that there is a physical solution.
There is a non-physical solution.
And over here [INAUDIBLE].
But then over here it is a physical solution.
While this kind of an [INAUDIBLE] is going to be a non-physical solution.
And that cannot be dealt with by the [INAUDIBLE] scheme.
And you nee a more complex flux scheme to deal with situations like that
[INAUDIBLE]
There is no downwinding.
And what you need to do is this.
What you need do is you need add something called numerical dispositing into this.
So you need to do F_k_plus_half is equal to F_k_plus_half plus 0.5 times absolute value of c. So instead of taking the c's, you want to take the absolute value of c's and divide by 2.
So this is like a very similar concept to upwinding.
But it operates differently.
You need to take the absolute value of c, but multiply that with u_bar and plus_half.
So it's minus [INAUDIBLE] u_bar minus 1 0, minus 1.
So you need to kind of-- so instead of doing upwinding, you need to use a numerical dispositive or a numerical diffusion to basically achieve the impact of upwinding, but do this a little bit differently.
But that's kind of beyond the scope of this class.
So, yeah, If you want to learn that, you're welcome to take the [INAUDIBLE].
We are going to be learning all about [INAUDIBLE] speed.
So let's see if this goes [INAUDIBLE].
OK.
So we have the [INAUDIBLE].
OK. Now, the cars are starting to move.
[INAUDIBLE] the light goes green, the cars start to move.
To do it properly, you have to change this back to the central average.
So you do upwinding or add numerical dispositive, but don't do both.
And doing both, so the scheme works, but it's not as accurate.
It's suffers from additional numerical dissipation.
OK.
So I hope you have fun playing with this small example here.
So starting from our next lecture, the next lecture we're going to be doing a review for the midterm.
But starting from after the midterm, we're going to be talking about first of all, stability of solution of positive differential equations, and then talk about finite [INAUDIBLE] method, which is the third method we are going to start in fall for [INAUDIBLE].
All right.
We'll see you next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, I kind of prepared not for the whole lecture, but a little bit short of that.
So I really expect you to ask questions on this material.
It's supposed to be a lecture that helps you review the material we have already covered and prepare you for the midterm.
So instead of me just going mechanically through the material, I want to ask you to initiate what do you think is more confusing, or you'd like me to clarify again, and things like that.
If you feel something is confusing, it's probably confusing for the whole class.
So please raise it so that I can spend more time on it.
OK, so before I do that, I just first want to finish up the finite volume scheme we have been working on for the last two lectures.
We have been discussing finite volume schemes in one dimension.
But applying the same concept to two dimensions or even three dimensional is surprisingly straightforward.
So let me first give out what we did in 1D.
We started out in finite volume schemes-- we started out with the integral form of the differential equation.
We started out with d dt of the integral of a conserved quantity inside a control volume, omega, it is equal to minus of the flux out of the control volume.
The minus sign is because the time derivative is really the flux into the control volume.
But the definition of our n, normally, is usually out of the control volume.
So we have a minus sign to reverse the normal so that it points into the control volume.
Times the flux as a function of rho, ds.
So this is like the integral form of the conservation law, right?
And I can write the same thing in 1D that is the specific case of omega being just an interval.
Now, this is the flux at the left minus flux at the right.
Or it's really rho at left and rho at right.
Right, so in 1D, what we did was we set rho bar of k is equal to the size of the control volume of k. So it's the integral of the control volume divided by the size of the control volume.
And basically, by plugging in the definition into the integral form of the conversation law, what we get is 1 over the size of the control volume, the flux at the left side of the control volume-- I'm just going to say F of k minus 1/2 minus F of k plus 1/2.
That is what we use to denote the flux at the cell interfaces.
Now, let's go back to 2D, or the multi-dimensional form of the conservation law.
We're going to define rho bar of a control volume, k, being the same thing.
It is really the integral over the k-th control volume, rho dx, divided by-- in 2D, that is the area of that control volume.
All right, let me draw a typical mesh in two dimensions.
So let's say this is a triangular mesh in 2D.
And this is like the k-th control volume.
Right, for example, this is part of a mesh in two dimensions.
Right?
And we are computing-- we found the volume scheme.
We are tracking the average of the solution inside that control volume.
So this is my omega k. That's the control volume.
And the area of that control volume is my Ak.
And rho bar of k is the cell average there.
So what is the time derivative of that cell average value?
Because the area of the control volume does not change.
It is equal to 1 over the area times the time derivative of the volume integral, right?
And the time derivative of the volume integral, by the integral form of the conservation law, which you can get by applying divergence theorem, right?
We did that last lecture by applying divergence theorem to the differential form of the equation to get this integral form.
And just by plugging in, we get 1 over Ak times the integral over the boundary of the control volume.
We get a minus sign here.
And the outward normal dotted with the flux, ds.
Now, let's look at what this is on the control volume.
So we found the integral over the boundary of the volume.
If the control volume is this angle, what are the boundaries?
We have [INAUDIBLE], right?
Now, we can express that as minus 1 over Ak of summation of ni dotted with Fi times the length of each of the sides.
So i is, I'm going to say, like the sides of k. So Sk is the sides, the three sides of the triangle, all the boundaries of this triangle, k. So Sk is a set of these integrals that gives you these three boundaries.
And the i the normal of each edge.
So for example, in this direction, so this is the ni's pointing in this direction.
These are the ni's.
And the graph, the Fi is the flux on these edges, which we have gone through, again, approximately, that by applying the graph function of these two neighboring cells.
It's the same thing as we did in 1D.
In 1D, we approximate the flux at the cell interface by looking at the value of the function at the neighboring two cells.
In 2D, it's the same thing.
We approximate the flux of the cell interfaces, so yield between two cells, by looking at the flux, by looking at the solution of the cell averages over the different sides.
If we have to do [INAUDIBLE], we do [INAUDIBLE].
We look at which direction is the [INAUDIBLE] over this edge.
Is it from this side or this side?
And choose the value on either side of this edge.
If we go to three dimensions, it's the same.
Instead of edges, we have spaces between control volumes.
That is the case where we really become looking at interfaces, interfaces at each place in three dimensions instead of an edge in 2D or a [INAUDIBLE].
Now, we approximate using a numerical flux that is rho bar of k, rho bar of-- I'm just going to say neighbor.
And then, we're done, right?
We have approximated the time derivative.
So you know the [INAUDIBLE] of the cell average in the cell k. And the function of the cell average in k and the cell average in the neighbors of the k-th control volume.
[INAUDIBLE] other point is the area, just like normal, the length of the k-th [INAUDIBLE], the n-th normal in this space.
And the length of the Ses are all known quantities from the mesh.
So this is [INAUDIBLE] flux, a function of the cell averages.
So essentially, we have turned a PDE into an ODE.
What is that ODE?
That ODE is d dt of rho bar 1, rho bar 2, et cetera, to rho bar of the last control volume is equal to some joint function of all these rho bars.
OK, so this n is the function that connects the time derivative of one cell average to the value of a cell average of that control volume and its neighboring control volume.
Is it clear?
How we do the same thing in 2D?
There is a lot-- there is going to be a lot of bookkeeping just because of the mesh and for every cell, we need to be able to find its neighboring cells, and find which edge is connected to which cell, and who is whose neighbor.
Conceptually, this is exactly what we do.
OK, so we have started finding the difference, finding the volume, and all these methods are methods that are intended to turn a partial differential equation into an enormous ordinary differential equation, right?
And once we turn a PDE into an ODE, we can apply exactly what we have been doing in ODEs to study these PDEs, to discretize, solve, and study the behavior of these PDEs.
OK, so this is going back to the review part.
I'm going to really both the rule of the mesh and the outcomes and put particular emphasis on several of the points.
The first point is order of accuracy.
So the order of accuracy in an ODE scheme is found by looking at the equation, du dt equal to f of u.
And we discretize this into some operator.
We discretize the d dt into a finite difference using a finite difference operator.
So we discretize the d dt into some kind of a delta over delta t. I mean, this is going to be finite difference operator, operating on the u.
Now, this is not going to be-- if you know the analytical solution, if you know what u is, for example, if f of u is equal to lambda u, then you know what u is, right?
And you plug this into the finite difference operator.
This is not going to be exactly equal to 0.
Right?
This is not going to be equal to 0.
OK, so for example, if you're looking at Forward Euler, if you're looking at Forward Euler, u of k plus 1 minus u of k divided by delta t, if you subtract by f of u k, it is not going to be equal to 0.
If instead of u k you plug in the [INAUDIBLE] solution, you're going to get 0.
But if you plug in the real, analytical solution, you're not going to get 0.
And the order of that approximation is going to be the local order of accuracy.
So if this is equal to O delta t to the k-th power, then k is the local order of accuracy.
Now, here, the usual confusion that happens here is that when you do the truncation error analysis in another way, if you do the truncation error analysis by writing down the tau equal to u of k plus 1, if you plug in the scheme, say, OK, u of k plus 1 minus the right hand side of the scheme, which is u k plus delta t times f of u k, I'm going to get-- tau, for example, in Forward Euler is O of delta t squared.
Which, for any scheme, or for Forward Euler, it is because k is equal to 1.
Delta t squared is actually delta t to the k plus 1 power.
So why is there a plus 1 when I compute the local order of accuracy by figuring out the truncation error of the update scheme?
Well, I get k if I look at the differential.
If I look at the approximate, the time derivative could be [INAUDIBLE].
Yeah?
[INAUDIBLE]
Right.
We still need a manipulation.
You can base the finite difference approximation of the n-th derivative and the update formula.
So how do you go back and forth from this and this?
How do you go back and forth from the approximation or the PDE to the update scheme?
You have [INAUDIBLE] to multiply these two equations by delta t in order to get to here.
You need to multiply this by delta t to get the update scheme.
In other words, the approximation error of this update scheme is equal to the approximation error of the time derivative multiplied by delta t. Therefore, of course I get one more delta t in the truncation error.
So that is the reason we ask you to subtract 1 when we figure out the local order of accuracy.
If you figure out the order of the truncation error, it's the single step upward.
You can go from the truncation error of the single step updates to the truncation error of the time derivative.
You divide this whole thing by delta t. And dividing [INAUDIBLE] by delta t, you get O delta t to the k, right?
[INAUDIBLE]
Yeah?
[INAUDIBLE]
Oh, yeah, sorry.
Thank you for that.
Let me use p for the order of accuracy.
It is not by purpose
[INAUDIBLE]
Yeah
[INAUDIBLE]
Ah, sorry.
This is k. Those are k's.
Right, I confused myself
[INAUDIBLE]
All right.
Yeah, so the k is the time step number, and p is the order of accuracy.
Yeah, thank you
[INAUDIBLE]
Yeah
[INAUDIBLE]
Any other questions?
So that is, I think, the one order difference in the local order of accuracy in that case is a big source of mistakes in figuring out the order of accuracy local order of accuracy analysis.
So try to remember that.
OK, so one thing, in the PDE context, if we approximate a PDE by a set of ODEs and solve it using ODEs by [INAUDIBLE], what do you think is going to be the order of accuracy we'd get at the end?
How much area are we making in that approximation?
All right, what determines the accuracy if I approximate a PDE by an ODE and solve the ODE using, let's say, a Forward Euler?
[INAUDIBLE]
The accuracy of the ODE scheme?
[INAUDIBLE]
And the accuracy of the discretization, actually, both.
Actually, both.
So if that happens even for some advanced graduate students when they look at the order of accuracy of a PDE discretization, for example, it's quite usual for them to have a plot of the area versus the grid.
And often what they find out, as I refine the grid further, I no longer improve my accuracy.
So what happens?
[INAUDIBLE]
Yes, because then I'm replacing my grid spacing.
And if I'm not also decreasing my cell step size, if I'm not decreasing my delta t, even though my spatial discretization is extremely accurate, has no error, I feel that a truncation error would be proportionate to delta t. So if I don't decrease my delta t, I'm not going to reduce my discretization error further.
So the ODE discretization scheme makes an error.
And that error is actually sometimes overlooked when you do PDE analysis.
So remember, there is still going to be an ODE component, always, even if you look at PDEs, time dependent PDEs.
OK, so this is the local order of accuracy.
How does that relate to the global order of accuracy?
So what if I do global order of accuracy?
How does-- I mean, it's a very simple answer, so please answer me.
How does local order of accuracy relate to the global order of accuracy?
Global is one more than local?
They're the same
Or are they the same?
If it is consistent, yes.
OK, global is equal to local if the scheme is zero stable.
That's the Dahlquist Equivalence Theorem.
Yes.
Right, so local and global order of accuracy are the same, right?
There is no plus one or minus one.
The plus or minus one happens over here when you figure out the order of the one-step truncation error.
Then you can get local order of accuracy, you need to subtract one from the delta t. But there is no plus or minus one in the global and local order of accuracy
[INAUDIBLE] one or greater [INAUDIBLE].
You can't have like zero order of local accuracy?
Oh, yes, this is provided we have a consistent scheme, provided we have at least equal to 1 here.
Right, yeah, thanks for that.
So the whole thing is under the assumption that p is greater than or equal to 1.
I mean, that's usually the case.
We usually only look at vertical schemes that are consistent, right?
[INAUDIBLE]
Yeah, the definition of consistency is p greater than or equal to 1, local.
That is a consistent scheme.
Yes?
And the practical way of finding that is the sum of the--
The practical way of finding what?
If p is greater than or equal to 1 [INAUDIBLE] discrete surface [INAUDIBLE] equals 0
A practical way of finding out if a scheme is consistent is by doing the global truncation error analysis.
You have to look at the Taylor series expansion to find out if the scheme is consistent or not.
I don't think there is an easier way to find out if a scheme is consistent or not
There was something in the notes about the coefficients of the non-derivative terms sum to 0.
Might be a little bit [INAUDIBLE]
Yeah, OK, so there is a condition that the coefficients of the constant terms sum to 0.
But I think that is a necessary but not sufficient condition
[INAUDIBLE]
Right, right.
So if I flip a coin of [INAUDIBLE], we're going to get a sum of 0, that is not a consistent scheme
[INAUDIBLE]
Yeah, the sum to 0 only means that you get consistent approximation of the d by dt equal to 0 equation
[INAUDIBLE]
OK, again, if you look at a PDE, the global order of accuracy is going to be determined both by the spatial and temporal discretization.
So even though if you look at how the area decays as you refine your grid.
You still need to be careful with the time derivative term.
You need to refine your time step at the same time.
And maybe you have to refine your time step even more if your order of accuracy in time is less than the order of accuracy in space.
All right, any questions on accuracy?
And accuracy, if you need to figure out what a scheme is, the order of accuracy is a good way to check what the scheme is.
Because different schemes have different orders of accuracy.
Forward Euler, Backward Euler are first-order accurate.
Trapezoidal rule and midpoint are second-order accurate.
And there are more advance schemes.
Many of them have an even higher order of accuracy.
So that's a good distinguisher of different schemes.
OK, eigenvalue stability.
OK, can somebody tell me?
We looked at zero stability, right?
That is, what makes a scheme have a global order of accuracy equal to local order of accuracy?
How does eigenvalue stability, how is that different from zero stability?
Yes?
[INAUDIBLE]
Hm?
If the general case--
[INAUDIBLE]
Right.
So eigenvalue stability, in some sense, is an expanded version of zero stability.
So it basically says that my solution, Vk, is bounded for the question of du dt equal to lambda u.
So this eigenvalue stability, while zero stability says that the solution is bounded for du dt equal to 0.
So when you talk about eigenvalue stability, you have to give me two things for me to determine if a scheme is eigenvalue stable or not.
You have to give me the scheme.
What scheme are you are using?
Are you talking Forward Euler, Backward Euler, midpoint?
You have to also give me something else for me to determine if I'm going to have eigenvalue stability or not.
What is that?
The governing equation?
Hm?
The governing equation
The governing equation, or more specifically, lambda times delta t. Right?
You have to give me these two things in order to determine if a scheme is eigenvalue stable or not.
And if you search for MIT math links, eigenvalue stability, the first thing you're going to get on Google at least is this thing.
Always run, OK, run.
You have to do a bunch of Java things.
Come on.
Yeah, that is going to give you the eigenvalue stability.
So you need two things.
One thing is the scheme, right?
For different schemes, you are going to get a different plot of eigenvalue stability.
So if I get Backward Euler, that is my stability region.
And this plot is a plot [INAUDIBLE] depending on the distance lambda delta t, right?
That's the second thing you need to tell me in order to [INAUDIBLE] eigenvalue stability.
You also need to tell me lambda delta t, and that determines one point in the complex set.
And depending on [INAUDIBLE] and eigenvalue stable [INAUDIBLE] or the eigenvalue unstable [INAUDIBLE].
Right?
Any questions?
So questions over there?
OK, so yes, tell me a scheme, and tell me lambda delta t. [INAUDIBLE] I'm going to tell you I'm stable or not.
And one thing is for Backward Euler, as your delta t decreases, whatever is happening, for the same lambda, as my delta t decreases, I'm more and more zooming into small regions and in all regions.
And the smaller I get, the more I'm approximating the stability of the true ODE.
The value of the true ODE is the [INAUDIBLE].
If lambda has a [INAUDIBLE], then that's unstable behavior.
If lambda has a [INAUDIBLE] value, then it's stable.
Must be stable, right?
If I zoom in [INAUDIBLE], that's going to be more and more of what I get.
And same thing for Forward Euler, right?
As I zoom more and more into the [INAUDIBLE], that's what I get.
And the same thing for trapezoidal.
It's really depending on if I zoom in or not, but the same thing for RK2.
As I zoom in [INAUDIBLE], I get [INAUDIBLE] stable, right term being unstable.
And same thing for [INAUDIBLE].
I get the behavior that on the left hand side is stable, on the right hand side is unstable.
Yes?
[INAUDIBLE]
For the Backward Euler, I can easily [INAUDIBLE] delta t, and it actually is going to make an unstable equation stable, yes.
That actually happens with some PDEs.
Now, you can even sometimes, for example, [INAUDIBLE].
And a lot of the fluid flows are actually unstable.
So if you're looking at, for example, water shedding behind the [INAUDIBLE], what you see there is an unstable flow field.
But you can actually get a stable flow field if you use Backward Euler.
You can use Backward Euler with a really [INAUDIBLE].
If you do that, you can actually converge, force yourself to converge to an actually unstable closed solution
[INAUDIBLE]
The question is, are you still consistent?
The question of consistency is for ODEs.
If we find that lambda delta t goes to zero, consistency means the [INAUDIBLE] behavior of the ODE as lambda delta t goes to 0, in this case, that's always consistent because if your lambda delta t goes to 0, and approximating the true behavior [INAUDIBLE].
Consistency has nothing to do with if I give you a really big delta t. If I give it a really big delta t, then it has nothing to do with consistency.
Because consistency is behavior of the linear [INAUDIBLE]
[INAUDIBLE]
Oh, OK, yes.
I said converges in a different sense.
So the convergence we are talking about here, as I decrease my delta t, as the numerical approximates the analytical solution, what I mean over here is that when [INAUDIBLE] delta t is low, as I've [INAUDIBLE], grows closer and closer to a solution of the time independent equation.
So that's what I was saying when I spoke about convergence earlier.
It has a-- I think that this is not a very good equation, but convergence in [INAUDIBLE].
One thing is when I decrease my [INAUDIBLE] time and space, I might converge in the analytical solution.
The second thing is in an iterative scheme, I guess you're going to learn more when you go through more advanced classes.
When you apply an iterative scheme, trying to compute the solution to a steady state differential equation, these states are [INAUDIBLE].
The steady state never stops closing.
And as I increase the iterations, do I get closer to the solution of the steady state equation?
It's more like a convergence in the sense of we can apply Newton-Raphson.
We can apply Newton-Raphson to solve the founding equations.
That happens to be more [INAUDIBLE] of your iterative solution.
Convergence moves, as I do more Newton-Raphson steps, do I converge to the solution for the [INAUDIBLE] equation?
So there are two completely different concepts of convergence.
One is, as I decrease both t and delta x, I get closer to a solution.
The second is as I why iterate more, do I converge to a solution?
[INAUDIBLE] What about [INAUDIBLE] So what it's saying is that [INAUDIBLE]
Right
[INAUDIBLE]
OK, I'm--
[INAUDIBLE]
Yes, so I'm going to draw what [INAUDIBLE].
So if you look at one where there's [INAUDIBLE] over here, that means a lambda that is that type of property, the real part-- and if you look at the analytical solution, the analytical solution here is something that oscillates sinusoidally while growing in magnitude exponentially.
Right?
So that's an analytical solution.
If they used Backward Euler with a small time step, so that is like when your lambda delta t is going to be 0.1, with a small delta t, you're [INAUDIBLE] lambda to somewhere close to the origin.
So you may get a solution that maybe you wouldn't get exactly the right behavior because unless your delta t is infinitely small.
But you are also going to get something that grows exponentially larger.
So that is when you have a small delta t. Now, if you use a large delta t, like you are scaling the lambda delta t to somewhere larger along the same line, because the delta t is real, what happens is that we get a stable solution.
So although analytically the solution grows larger, you are expected to get a solution that looks more like this.
So it is going to get to the wrong answer qualitatively even.
And of course, this is because you're using a super large delta t. You're using a delta t that is actually much larger than 1 over the magnitude of the eigenvalue.
Right?
Does that make sense?
Yes?
[INAUDIBLE]
Yes
[INAUDIBLE]
So you're asking a very good question.
So for a system that is analytically unstable, what is a good way of telling my numerical scheme is doing a good job or not?
This is a much deeper question than I can answer.
Yeah, there is a [INAUDIBLE].
If the solution analytically is unstable, that means to approximate using numerical methods is extremely difficult.
If you make a small error in the beginning, even if you have a small error let's say in terms of [INAUDIBLE], even though we'll assume you are doing everything exactly starting from time step 0, you're still going to get a [INAUDIBLE] error as you come to here.
Just because the equation itself is unstable.
The equation itself being unstable means you can make a small perturbation over here.
That perturbation will [INAUDIBLE] grow larger and larger as we integrate more and more.
So treating a case like that numerically is very difficult.
And if interested, let's talk more about that after class.
All right, any other questions?
OK. And by the way, that's something I'm actually looking at in my research right now.
So very good question.
I'm very impressed
[INAUDIBLE]
Yeah, that's why people can't predict the weather, right?
I mean, they try to solve a PDE to get the weather seven days later.
But you know it's going to be a not very good solution by experience.
So that's exactly what they're trying to do, solve an unstable system forward in time for something like seven days.
All right.
OK.
I got a question of, what is the advantage, what is the main advantage and disadvantage of explicit versus implicit methods?
Let's do a comparison here.
You all did the project.
So you all kind of know the disadvantage of an implicit method.
What is that?
[INAUDIBLE]
A lot of coding, right?
Why lots of coding?
[INAUDIBLE]
You have to solve a non-linear equation.
Solve non-linear equation, right, in every time step.
And the way we solve it is using Newton-Raphson.
So we have to apply Newton-Raphson equation within every time step.
That means a nested loop within every time step.
So you have an outer loop that is looping through the time step.
Within the outer loop, you need to have the inner loop that does Newton-Raphson iteration.
So of course, it's much more complicated than explicit schemes, right?
Where you don't need to solve any non-linear equations.
That's why it is explicit, right?
OK, but now, what is the advantage of implicit schemes?
[INAUDIBLE]
It's way more accurate.
But I wouldn't hold that as a rule.
I mean, in the problem, yes.
We used an implicit scheme that turns out to be way more accurate.
But the reason may just be our implicit scheme is eigenvalue stable, right?
The explicit scheme we were using was [INAUDIBLE].
That turns out to be eigenvalue stable only along the imaginary axis.
So accuracy is actually not the main driver of people adopting implicit schemes over explicit schemes
[INAUDIBLE]
Yes, the main driver is a larger stability region.
Larger stability region, as we were just looking at.
OK, just for example, compare Forward Euler with Backward Euler.
Forward Euler, tiny, right?
Backward Euler, the region where it's unstable is tiny, right?
That's kind of an extreme comparison, but gets the point through.
Yes?
[INAUDIBLE]
OK, good point.
What happens if Newton-Raphson doesn't converge in the implicit method?
And now, when you say converge, you don't mean as delta t and delta s go to 0, right?
You mean as my iteration goes to infinity, doesn't converge?
So it's a very good point.
Because yes, if you have a very non-linear problem, if you use a super high delta t, it's quite possible your Newton-Raphson doesn't converge.
So if you use implicit methods, there is actually an implicit restriction on delta t in which you cannot get through this eigenvalue stability analysis.
It is actually a delta t that is going to enable it to converge rapidly with Newton-Raphson equation.
So one thing I think you can-- by just a little bit of analysis you can find out is that as I decrease my delta t, my Newton-Raphson is going to have a much easier time to converge.
In fact, if my delta t is very small, then my Newton-Raphson, I'm going to have a very good initial guess of my Newton-Raphson.
Because if my delta t is very small, then my next step solution is going to be pretty close to my current time step solution.
Right?
The change of the state over the two steps wouldn't be that large.
And Newton-Raphson will always converge if your initial guess is close enough.
So that's the nature of Newton-Raphson because it uses a linear approximation to get the root of that linear approximation.
I can talk more about that.
But if you have a close enough initial guess, Newton-Raphson will always converge.
Therefore, by decreasing, if Newton-Raphson doesn't converge, a very straightforward recipe is decrease your delta t. And that is going to make the change between one state and the next time step state closer, and therefore give you a much better initial guess to Newton-Raphson.
And that is going to allow you to converge much easier.
So yes, Newton-Raphson actually can diverge if you have a super non-linear problem and you use a super large time step
[INAUDIBLE]
Oh, yes.
By the way, I can make the same argument for [INAUDIBLE].
If I get an unstable system, I can always decrease the time step so that I get into the stability region of the implicit scheme.
That is still except for the time step restriction for Newton-Raphson, we said over here, is set by a different mechanism as the largest delta t I can handle up here.
The largest delta t I can use in the explicit step is done by how large, by a larger eigenvalue.
It is really-- we can really obtain it from a linear analysis.
And linearizing the equation [INAUDIBLE] the largest eigenvalue, I can find out what is the largest time step I can take here.
So [INAUDIBLE] the implicit method is governed by if Newton-Raphson converges.
And Newton-Raphson will always converge in one step if I have a linear equation.
Newton-Raphson will converge in one single step if I have a linear equation.
So the time step restriction here is set by the convergence of Newton-Raphson, not by the eigenvalue step, right?
How non-linear the equation is.
So there are some integration methods where people actually separate out the linear part that has super large eigenvalues and the non-linear part.
So for a linear part that has super large eigenvalues, they do it implicitly.
And for the non-linear part, they do it explicitly.
And there's a class of methods for the [INAUDIBLE].
Fancy name, but short for implicit, explicit methods that creates different parts of the equation, either explicitly or implicitly.
And as you can guess, the part they treat implicitly is going to be the linear part that has super large eigenvalues.
So that you converge in one step, you can do Newton-Raphson, and you avoid the time step restriction set by the large eigenvalues.
All right, OK.
Right.
So this is really especially in stiff problems.
And what is a stiff problem?
A stiff problem is actually defined in terms of if you use an explicit integration scheme.
If you find yourself having to use a super small delta t not because you want super high accuracy but because it's still going unstable if you use a larger delta t, then you know you have a stiff problem.
That is really the easiest way, I find, to define a stiff problem.
That is like, you are forced to use a small delta t by the stability region, not for accuracy reasons, not for wanting higher accuracy, but simply trying to avoid [INAUDIBLE].
OK, so then you get a stiff problem.
Any questions?
Yeah, so any question, please raise it.
Otherwise, I'm going to review Newton-Raphson a little bit, and there is no more things I'm planning to cover.
So I'm kind of expecting questions from you guys
Finite difference [INAUDIBLE]
Yeah, finite difference, and finite volumes I include in the scope of the exam.
As you saw from the previous year's questions I posted, sometimes we include it in the actual exam, sometimes we don't.
So it is included in the scope, but because we just did it, I think it's a good idea to review some of the earlier stuff that you may have forgotten.
And another benefit is that we are also going over this now that we have [INAUDIBLE] to give you a sense that all the stuff we learned applies to PDEs because what we did in PDEs is just to approximate the PDE using a big ODE.
Right?
And you can apply implicit methods on the PDE also, except for the Jacobian.
You get it's going to be something close to the matrix form of finite difference, we did in the finite difference vectors.
Yeah?
[INAUDIBLE]
Oh, OK, the example format is that it's closed notes, right?
It's closed everything in the period from you get the exam to when you come to our office.
So it's closed everything, closed computer, closed cell phone.
Closed Wikipedia.
We are actually letting you work out, really think about the problem during the time you are looking at the problem.
But even though you don't get the answer, or during our face time, we are going to also interact with you when we ask you questions that you may not have expected, or we may actually help you going through some of these.
So that's kind of how [INAUDIBLE] goes.
Do you have something else you want to ask?
[INAUDIBLE]
Yeah, right.
You can write things down and bring whatever you have written into our office.
And in the office, you're expected to use a whiteboard or blackboard and explain to us like you're the professor, the other students, explain to us what you got.
Any other questions?
No?
The last thing I want to go through is Newton-Raphson method.
And it's another sort of confusing-- as I said, Newton-Raphson method is a method that simply solves regular non-linear equations.
And a non-linear equation can appear in 16.90.
It can appear anywhere else in your future career.
So what you're learning about solving non-linear equations really goes very far, even if you don't deal with numerical methods later on.
So as I said, it's a method of solving non-linear equations if the set of non-linear equations is relatively small.
Say you have two equations or three equations, you can go to Matlab and use F solve to get it solved like brute force.
But if you have a large set of differential equations you have to solve simultaneously, like what we're going to have if you use an implicit time integration method, apply it to a discretized PDE, then you're going to get at least 100 ODEs you have to-- which, if you have to solve using an implicit scheme, you're going to get hundreds, maybe thousands, maybe millions, or maybe trillions of algebraic equations you need to solve.
OK, so imagine you have to solve f of u equal to 0 where u, instead of writing down a simplified version of what you would get in an implicit scheme, your implicit scheme you would get u minus u k, which is the stuff you already know, over delta t is equal to some right hand side of u, and uk, and maybe something else, right?
And instead of writing the same f here, I'm just going to write a big F of u equal to 0.
So that big F in this case is really defined as u minus u k over delta t minus f of u, u k, et cetera.
So this is what happens if we solve, let's say, a discretized PDE [INAUDIBLE].
U is the next time-step solution we want to get.
And u k is the previous time step solution you already know.
Now, if F is non-linear, you have to find the root of its big F, which is essentially the left hand side minus right hand side of this implicit update [INAUDIBLE].
Right?
You need to find a u which is a vector of the solution that makes this f 0 for all the components of f. And f has the same dimension as u.
If u is a 3D time step, f is going to be a 3D time step.
And you need to find these 3D numbers that make the [INAUDIBLE] f equal to 0 simultaneously.
That's not an easy task.
And I think F solve is going to have a hard time dealing with this if u is a high dimensional vector.
Now, what does Newton-Raphson do?
Newton-Raphson starts with an initial guess.
So u, for example, I'm going to use parentheses to denote the information [INAUDIBLE].
Parentheses 0 is the initial guess.
I'm going to set it to u k. I'm going to set it to the solution at the previous time step.
Then, I'm going to approximate this non-linear function using a linear function.
Can I approximate F of u by something linear?
Let's do it one by one.
Let's approximate the first component of F. Think of F having components.
We're just going to approximate the first component and have all the other components follow.
OK, so if you just approximate the first component, I'm going to use Taylor series.
And I'm going to use a Taylor series of a function of the [INAUDIBLE] variables, all right?
So a function-- the periodicity of a function of a [INAUDIBLE] variable is pretty complicated.
But the lucky thing is that I don't need to keep all the terms.
I'm going to throw away anything that involves more than the first derivative.
In other words, I'm only going to keep the zeroth order term, which involves no derivatives, and the first order term, which involves only the first order derivatives.
OK, now what is the zeroth order term?
The zeroth order term is F1 at my initial guess, right?
What is the first order term?
Actually, in this case, because it's a multivariate function, I have more than one first order terms.
If I have as many first order terms as there are many u's, I'm going to be summing from i goes from 1 to the dimension of the system, let me call big N, partial F1, partial u of the ith dimension times ui minus u0i.
Right?
Does that make sense?
That's the Taylor series expansion of F1.
We are only keeping the zeroth order terms as opposed to all the terms.
If I start to write all the other order derivatives, it would get too complicated.
But I'm not going to write them, so I'm going to truncate them.
And I'll approximate [INAUDIBLE] method, it's a linear function.
It's a linear function because this F1 of u is probably a linear function of u.
But now, I'm truncating.
I'm removing everything that happens over here, so if it's quadratic, or cubic, or anything.
I'm only keeping the constant part that is independent of u, and this part, which is 0.
Now, I get a linear approximation of F. We all know how to find the root of a linear function?
Even though it's a million dimensional?
Right, that's the reason we have linear algebra, right?
That's why we have matrices.
That's why Matlab is called Matlab.
That is because we can write functions like this, we can write linear functions in matrix form.
And to solve linear equations like this, even ginormous ones, we just use linear algebra, right?
OK, so we are also approximating all the components of F, all the way to the n-th component using Fn u0, plus summation i goes from 1 to N, partial FN partial ui, ui minus ui0.
So there are big N equations.
We have big N of these variables.
How to solve them?
How to solve these coupled, N-coupled linear equations?
Yes?
[INAUDIBLE]
Yeah, it's just write it into a matrix form, right?
First, write it into a matrix form.
So I want to try to say, OK, if I want to set all these things to 0, I'm trying to set like 0, 0, 0, 0, is equal to F1 at u0, et cetera, to FN of u0.
This is this term.
And how about this term?
This term is just a joint matrix vector multiplication.
The matrix is other derivatives, which is called the Jacobin when I put that into a matrix form.
The first row is my partial F1, partial blah.
The first column is my partial blah, partial u1.
The last column is partial F blah, partial uN.
So each row corresponds to one [INAUDIBLE] the residual.
Each column corresponds to one thing in the independent variables, in the u's.
And multiplying that by order u1 minus u1 0, et cetera, to uN minus uN 0.
Do you all see these linear equations are the same as this matrix?
All right?
So what I did is the Taylor series expansion for all these non-linear equations and write them all into a matrix form.
And what I'm trying to do is I'm trying to solve for these u1 to uN.
How do I do that?
I just invert the whole Jacobin.
If I call this as J, then my u1, et cetera, uN is going to be equal to u1 0, uN 0 minus J inverse times my F1, FN evaluated at the initial guess.
Right?
So this is the solution [INAUDIBLE].
This is one step in the Newton-Raphson, right?
In Newton-Raphson, we compute the residual [INAUDIBLE] at the initial guess.
And we do the Jacobian.
[INAUDIBLE] the Jacobin inverse times the residuals.
Then, you stop at the result from the [INAUDIBLE].
And get your next step solution.
And what is the next step solution?
The next step solution is the zero of the linear approximation?
All right?
Which hopefully gives you a better initial guess than u0.
And what you do next is you call this set of u's to be u parentheses 1.
We call these as u parentheses 1.
And then, you linearize again on these.
So the one dimensional analog is this.
You try to find F of u. OK, you want to find F of u.
You start over an initial guess.
This is my u0.
I construct my first order Taylor series approximation, which is what in this case?
If the zeroth order term is going to be this, this is the zeroth order term.
It's a constant, right?
It's just F of u0.
The first order term is a derivative at this point, which is going to give you this.
So this is my first order term is going to give me the tangent line.
My u1 is the solution of the Taylor approximation equal to 0.
So this is my u1.
And then, I'm going to linearize around this again.
I'm going to construct another first order approximation, and go to this point, et cetera, et cetera, until I converge to the zero at the blue line?
Now, it's hard to draw this picture when both u and F are million dimensional.
But the concept is the same.
I construct a linear approximation.
And the linear approximation can be arranged into matrix form using the Jacobin.
Jacobin is just a matrix containing all the derivatives of Fu's.
And I keep iterating.
I keep iterating.
I solve the 0 of the linear approximation, linearize again at that new point, and then solve, get the zero of that linear approximation.
I linearize again at the new point, I get the zero of that linear approximation, and then linearize again at the new point.
Any questions on this?
General technique of solving large systems of non-linear equations.
Using the fact that we are already good as solving large systems of linear equations.
OK, now I'm all out of things.
Just time to answer your questions.
Yes?
Will we be expected to [INAUDIBLE]
No, you don't need to quote anything.
You don't need to have your hands on the keyboard.
Yes?
[INAUDIBLE]
What does big F represent in this case?
Big F represents the function of u if we want to get the zero.
If I want to solve [INAUDIBLE], if I use an implicit scheme and I want to evolve from this case to the next u, to the next time step, then my F of u would be the left hand side of the scheme minus the right hand side of the scheme, right?
Imagine if I have Backward Euler.
Then my big F would be u minus uk divided by delta t minus F of u.
That is the Backward Euler.
If I trapezoidal, my big F would be u minus uk over delta t minus half of F of u minus half of F of uk.
All right?
And the difference is implicit schemes are going to get a different value.
I have a Backward Euler, [INAUDIBLE], half of that F of u plus half of that uk.
And that is in the derivative.
Whatever the scheme is, F is the thing along with [INAUDIBLE] zero that big F is [INAUDIBLE]
[INAUDIBLE]
It's a [INAUDIBLE], yes.
Questions?
After this, we are going to [INAUDIBLE] the midterm
[INAUDIBLE]
All right.
Anything else I can talk to you about?
All these materials?
Oh, and just to remind, we are really looking at the measurable outcomes.
And don't forget to go [INAUDIBLE] something that [INAUDIBLE].
Analytical solutions [INAUDIBLE] what is the solution of u [INAUDIBLE]?
And things like that.
And if the solution is stable, when it is unstable, you need to know that and things like that.
So this is important.
Yeah?
[INAUDIBLE]
Should d of dt be equal to lambda u?
You need to solve it analytically.
Huh?
[INAUDIBLE]
Yeah.
For d of dt over the lambda u, you need to be able to solve it analytically, yeah
[INAUDIBLE]
Huh?
[INAUDIBLE]
Yes, yes.
D of dt equal to au, yes, right.
Any matrix, A, right?
You need to be able to solve it
[INAUDIBLE] [LAUGHTER]
Yeah.
My A can be a trillion by a trillion.
And you need to know how to solve it.
You are not expected to--
[INAUDIBLE]
Hm?
[INAUDIBLE]
Yeah, of course, you won't be able to do the eigenvalue problem of A in your head if A is a 1,000 by 1,000 matrix.
But you need to know how to get the analytical solution of the
[INAUDIBLE]
Hm?
[INAUDIBLE]
Yeah
[INAUDIBLE]
All right.
Anything else?
And homework here.
If you didn't [INAUDIBLE] homework for [INAUDIBLE], the previous homeworks, you can always come to my office and figure it out.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So you guys have read all about method of weighted residuals.
Yes?
Makes perfect sense.
Yes?
We need to go through things today?
A little bit.
OK. Just a little bit.
All right.
So-- actually, but before we do that, I want to-- [INAUDIBLE] [?
Let's ?]
put up a list of topics that we're going to talk about today.
So I want to start by just talking about the measurable outcomes for the module.
Remind you of what it is we're going to do.
Then we're going to introduce the model problem.
You read this in the pre-class reading.
But it's the steady 1D diffusion.
And we'll do a few things in general, but more or less we're going to develop the [INAUDIBLE] by thinking about this specific example.
We're going to talk about the idea of approximating the solution with basis functions, which is a fundamentally different way of doing the approximation compared to what you've already seen in finite difference and finite volume.
Then we'll look at the collocation method.
And then finally, we'll get to the method of weighted residuals.
OK.
So what I'm going to do on the board is more or less paralleling what's in the notes.
And so I can go quickly on the thoughts that are already clear, and we can spend more time on the parts that you have found to be confusing.
OK.
So I'm not going to put the screen down because it'll take a while to get the projector on, but I just want to remind you that there's the measurable outcome list embedded on the MIT site.
It's also on the mitosis.mitx.mit.edu site.
And so we're into a new module now.
There are four modules in the class.
The first one was integration methods for ODEs.
The second one was finite difference and finite volume methods.
And the third one now, which is finite element methods for PDE.
And so I think it's really helpful for you to go back and-- well to go forward at this point-- and look at the outcomes, and think about specifically what it is you're going to have to be able to do.
Because as you are going through all the notes, and I know there are a lot of notes for this section, it will help you think about what are the specific skills that need to take away.
There'll be homework problems and a project that will help you do that, too.
But for example, describe how the method of weighted residuals can be used to calculate an approximate solution to the PDE, we're going to cover that today.
Then to be able to describe the differences between the method of weighted residuals and the collocation method, and the least squares method for approximating a PDE.
So there's a lot of really specific outcomes.
I think there's probably about 12 outcomes that are associated with this module.
Go and take a look at them, and I think that will hopefully help you divide up the material, and think about what it is that you need to be able to do.
But let's start off by thinking about our model problem.
So a model problem just means a simple problem that' we're going to use to develop things on.
And so the model problem again is steady 1D heat diffusion in a rod.
And the PDE should be pretty familiar [INAUDIBLE] you by now.
It's d by dx of k [?
ct dx ?]
equal to minus 2.
We're thinking about a 1D domain.
So here's our domain.
That's the x direction.
The domain is going to have links l. So we'll have x in general going from minus l over to plus l over 2.
The k sitting in here in general is going to be a function of x, although you'll see that just to make things simple in the beginning we'll at some point make k constant.
That's the thermal conductivity of the material that the rod is made out of.
And this q on the right hand side also can be a function of x, is the heat source.
And it's actually heat source per unit [INAUDIBLE] to get units right.
OK, so simple problem set up.
PDE specify heat source.
We're also going to need to apply boundary conditions when we actually get to specifying a particular problem.
So there's one particular problem that we will use.
One that we can compute the analytical solution for it.
Because if we can compute an analytical solution, we'll be able to look at errors and things when we look at the numerical approximation.
So for the particular case where k is a constant and equal to 1, the length of the rod is 2, so that x goes from minus 1 to plus 1, and the heat source is q of x being 50 e to the x.
And again, this is the same example that's in the notes.
And boundary conditions, the temperature at the right end of the rod is set to be 100, and the temperature at the left end of the rod is set to be 100.
OK, so for that particular case we have an analytical solution you can get by integrating the PDE.
And that turns out to be t of x is minus 50x to the x, plus 50x the hyperbolic sign of 1, plus 100, plus 50 times the hyperbolic cosign of 1.
OK, so that's just what comes out of this analytic.
[INAUDIBLE] and that's maybe a little bit hard to think about, so let's just sketch what t of x looks like as a function of x.
So there's x going from minus 1 to 1.
We've taken x equal to 0-- I mean x equal to minus 1.
Then we should be getting the boundary condition back, t is 100.
So if we make this 100 on my t-axis, then the solution looks something like this.
And it comes back to 100 at the other end.
OK.
So that's an analytic solution that we can compute.
And again, we're just doing that because when we start looking at the [?
miracle ?]
approximations, we're going to want something to compare to.
So a simple problem where we can actually and integrate things analytically.
OK?
Yep.
Think I've told you anything.
All right.
So let's now start thinking about the solution approximation.
Can I-- again, everything I'm writing is scanned in online, so-- I know it's good to copy stuff down and help follow along, but also you will have copies of the full notes.
So now let's start thinking up this idea of approximating the solution.
So first let me ask, if we were going to solve this problem using finite differences, what would we do?
We'd run to Alex and ask him to help us?
What would we do if we wanted to solve this problem using finite differences?
We'd break up the domain into a bunch of cells, and then what?
How would we approximate things?
[INAUDIBLE]
Yeah.
Good.
So we would decide what kind of a scheme we wanted to use, and we would approximate the derivatives.
And that's kind of the key part of [INAUDIBLE], is that we would take this-- we'd take k equal 1 constant.
So basically we've got a second derivative the t sitting here.
We would approximate it using our favorite finite difference [INAUDIBLE].
So maybe the central difference for the second derivative.
So I think it's really key to keep in mind that finite element departs from that mentality right from the beginning.
That with finite elements, we're not going to approximate the derivative here.
We're not going to approximate this operator, this d squared by dx squared that's acting on t. With finite elements, we're going to assume a form for the solution t, and approximate the solution.
We're going to assume a form for the solution t, and we're going to substitute in and do a bit of mathematical magic, and then solve to find an attribute solution.
So it's already quite different to finite differences.
So how do we get started?
We start off by-- and let's call this 2a-- we're going to say, to approximate the solution [INAUDIBLE] used a sum of weighted functions.
OK. We know that t is a function of x. t varies continuously along the domain here.
A function of x.
That's an infinite dimensional problem, right?
t is a continuous function.
Best rather write our approximate solution, which I'm going to call t tilde of x.
So this guy here is the approximation of my exact solution, t of x.
And I'm going to write it as-- I'm just going to leave a bit of space here-- the sum from i equals 1 to capital N of some ai's times some ci's that are a function of a.
What are these guys here?
These things here are no-end functions, they are things that I'm going to specify.
And what you'll see is that these things are going to be referred to later on as basis functions.
These things are no-end functions of x.
So we're going to specify them.
The ci's, there's N of them-- capital N of them.
And the ai's here, these are unknown [INAUDIBLE].
OK, so what I've said is that t of x, an infinite dimensional problem, t is some continuous, or some function, whether it be continuous, some function of x on the domain.
Let's approximate it as a finite sum from i equals 1 to n of some weight, so some coefficient-- ai's-- just constant, times some functions that we're going to specify.
So with-- can you see that we've discretized the problem in a different way than we did in finite differences.
We have discretized the problem because now we have how many unknowns?
n. We have n degrees of freedom.
n [INAUDIBLE] ai's that we can use to create our approximate solution.
And I've left a bit a space here because I'm going to write this in.
I'm going to write in 100 plus that extension.
Why do you think I wrote the 100?
I heard the b word
[INAUDIBLE]
The boundary conditions.
Yeah.
In this particular example that we're considering, the boundary conditions [INAUDIBLE] conditions that specified to be t equal 100 at either end.
So let's put the 100 out in front, and then the rest it there, the some of the ai times the ci's is going to satisy 0-- 0 temperature at either end.
Right?
OK so in this particular example, this guy-- this 100 is chosen to satisfy a outbound [?
preconditions ?]
here for our model problem.
OK.
So by writing this-- and this sort of approximation of the temperature-- it's a simple equation, but I'm spending a little bit of time because it's really important it's clear in your mind, because this is really kind of one of the-- what's the first key step of making this approximation.
We've turned the problem of determining t tilde of x, the approximation, which again is really an infinite dimensional problem, into the problem of determining the coefficients, the a1, a2, up to a n where we have now just capital N unknown.
Kevin, yeah?
[INAUDIBLE]
Yeah, so if you had like 100 here and 200 somewhere else or something?
There's various ways that you can handle it.
And we'll talk probably in two weeks of times about how to do boundary conditions and the finite element method.
But one approach that we could use just with the simple idea here is, you could, for example, have a linear function that went from 100 to 200 and then plus the rest.
Right?
So you could still get the boundary conditions back.
But yeah, we'll talk once we've been through the theory about how to handle boundary conditions more generally.
Yep.
Generally speaking we will do something so that those ci's will satisfy 0.
[INAUDIBLE] OK.
So now the question that you should be asking yourself is, what are these?
I said they were no-end basis functions.
They are things that we specify.
So you guys have seen this kind of idea, this expanding a solution as a sum of weighted functions.
You've seen that before, right?
When have you seen?
Fourier series.
Good, yes.
So-- I mean this is no different really to Fourier series, right?
What are the ci's in Fourier series?
Sines and cosines of different frequencies.
So the idea of decomposing a signal into a sum of harmonic components-- I mean, signal processing-- the entire field of signal processing kind of hinges on that idea.
Here we're not going to use sines and cosines.
We're going to use polynomials.
So ci of x are going to be polynomials [?
in x.
?]
In other words, c are going to be constant plus maybe a linear term in x plus maybe a quadratic term in x, and so on.
And the reason that we're going to use polynomials is because that sets the groundwork for the finite element method.
Finite element method works with polynomials, basic functions.
So we're going to start off by thinking about polynomials.
OK?
Good?
Straightforward, right?
Easy.
OK.
So let's just be more specific for our particular example of what we want to choose for the ci's.
And again, I want to be clear, there are many choices for ci, and this is a decision that you make.
So this is [INAUDIBLE] choosing the ci's.
So again, here we're go to use polynomials, and that's because we're interested in the finite element method.
And you will see on Wednesday exactly what the basis functions look like in finite element.
But for now, not yet the finite element, we need basis functions that satisfy the boundary conditions.
But because we subtracted it off, or because we added in the 100 here, you can see it's what I was saying earlier, we're going to set these guys now to be 0.
So they're going to be 0 at the end of the domain, right?
So if we-- I think we're going to choose polynomials, so what's the simplest polynomial we could choose?
We could choose [INAUDIBLE] equal constant.
That's not very useful.
What's the next one?
x plus b.
You could use a linear function, but what's the linear function that satisfies this condition?
[INAUDIBLE]
Zero everywhere.
So that's not very useful.
So it turns out that the first one that would give us anything-- what's that?
[INAUDIBLE]
Yeah.
The first one that would give us anything interesting would be quadratic, or a parabolic function.
So we'll choose a quadratic function and-- again there is a variety of things we could do-- but here's one-- here's a quadratic function, which may be [INAUDIBLE] well it could be scaled.
1 plus x, 1 minus x, that's a quadratic.
So c1, the first basis function in our extension, the function of x, and [INAUDIBLE] is 1 plus x, 1 minus x.
That clearly satisfies the boundary conditions, the 0 boundary conditions at either end.
Yep.
And then let's also look at cubic.
So that's going to be our second one.
And there are a variety of cubics we could choose, but let's choose the one that does x, 1 plus x, 1 minus x.
And again, clearly if we satisfy [INAUDIBLE] x equal-- if we substitute x equal minus 1, or x equal 1 in there, we're going to get c2 equal to 0.
So if we sketched out physically what these guys look like, we sketch c as a function of x, then the quadratic looks something like that, and the cubic looks something like that.
So that's c1 of x, and this is c2 of x.
Right, that's 0.
OK.
So those are the choices we make.
And so in this simple example we're going to use only two degrees of freedom to describe the approximate solution.
And with those two degrees of freedom, what is our approximation?
Our approximation is t tilde of x being 100, plus that sum, which in this case is going to be a1 times c1 of x, where c1 is 1 plus x, 1 minus x, plus a2 times c2 of x, where c2 is x, 1 plus x, 1 minus x. OK, so physically we're saying the solution is 100-- so first of all, just shift it up.
100.
Then take some amount of a function that looks like this, plus some amount-- some other amount-- of a function that looks like this.
Add them together.
And now we're going to say, what's the right-- what are the right weightings?
What are the a1's and a2's that we should choose so that the function that we get is somehow reasonably a solution of that model problem that looks something like this.
Yep.
And that's now what we're going to talk about with numbers 3 and 4.
The collocation method and the method of weighted residuals are two different ways that we can come up with the conditions to help us choose a1 and a2 so that in different ways we get a solution that is a good, or just a different kind of an approximation, of the problem we're actually trying to solve.
OK?
Yep.
All right.
Any questions so far, questions based on things that are in the notes that weren't clear?
No?
All right.
So that's kind of the easy part.
So now let's talk about the two different ways that we can come up with conditions that will let us solve for a1 and a2.
And the first one is going to be the collocation method.
So number three, collocation.
Collocation.
OK.
So again, what are we doing here?
We're saying for the approximate solution t tilde of x being 100 plus the [?
extension ?]
[INAUDIBLE] in of ai times the ci of x, where we're, in our simple example, choosing [?
n equal 2 ?]
and the ci's to be these two guys here, quadratic and the cubic.
We need now to determine the ai's.
a1, a2, up to an.
In this case, just a1 and a2.
OK.
So first question, how many conditions do we need to come up with?
Two.
In general, n conditions.
Right?
We have n unknowns.
We said that over here.
We turned our infinite dimensional problem of solving the PDE into a problem with n degrees of freedom.
So now we somehow need n conditions.
And in this example here where we have just two basic functions, we need two conditions that will let us solve for a1 and a2.
So what is collocation say?
Collocation says let's pick n points.
We draw my rod back up here from l over 2, minus l over 2, to l over 2.
Let's pick some points.
In particular, let's take n points, so let's pick two points in the domain, this point and this point.
And let's enforce the PDE at those points.
Let's make sure that the PDE is satisfied.
Everyone know what the PDE is?
d by dx of kdtdx equals minus 2.
Let's have the solution satisfy the PDE at those points.
OK?
And by forcing that-- forcing the PDE to be satisfied with the approximate solution at those two points, we're going to two conditions.
Right?
Two mathematical conditions.
Two equations, two unknowns, we should be able to solve.
So that's the idea with collocation.
You guys have seen-- did you guys see collocation in aerodynamics?
[INAUDIBLE], yeah.
What do you do?
You have the collocation point at-- which-- I always forget which way it is-- you have c over 4 and 3c over 4, right?
[INAUDIBLE]
[INAUDIBLE]
At 3, [INAUDIBLE].
So it's the same idea.
The collocation point is the point at which you enforce the condition.
So it's exactly the same idea.
OK, so how does that work out mathematically?
So that's what collcation-- that's what collocation means.
We're going to enforce the PDE at n point.
OK. How are we going to write that?
So let's again write down the example that we're considering, which is the 1d heat equation.
And I'll write it in the general form here with the k still in there.
d by dx of k dt dx equals to minus q. OK, so now we're going to define something very important, which is the residual.
And this is a really important concept that we're going to use repeatedly all the way through this finite element module.
So what is the residual?
We're going to call it capital R. It's going to be a function of our approximate solution, and it's a function of x.
And for this 1D heat equation, what's the residual?
It's d by dx of [?
kdtdx ?]
tilde dx plus [?
q.
?]
OK, so 1D heat equation, d by dx of of kdtdx equals minus 2.
The residual for that equation, which is a function of the approximate solution, t tilde-- so you want to mark that this is your approximate solution to remind yourself-- function of the approximate solution n of x is d by dx of kd t tilde dx plus q.
So does someone want to give me in words, what is the residual?
Use some words to describe it
[INAUDIBLE]
Yeah, good.
So the residual is the amount that the approximate solution doesn't satisfy the PDE.
In other words, if we come back to our original PDE, we brought everything to the left hand side.
So it would read d by dx of kdtdx plus q equals 0.
Then if we take, instead of t, the actual solution, if we take an approximate solution, t tilde, we substituted it in, the residual is how much we get that's different from 0.
So it tells us by how much the approximate solution, t tilde, does not satisfy the [INAUDIBLE] equation.
Yes?
So someone tell me, how is the residual-- is the residual the same thing as the error?
If I define the error to be the difference between the exact solution, t, and the approximate solution t tilde, is the residual the same thing as the error?
Not the same thing.
So how is it different?
[INAUDIBLE]
Yes.
So let's think about that.
So if we define the error to be t minus t tilde, this will be our-- is this thing a function over the whole domain?
Is e a function of x?
Yeah.
Right.
So e is a function of x if it's t of x minus t tilde of x. Yeah?
How about the residual?
Is the residual a function of x?
Yeah.
So they're actually both-- they're both functions that we could plot-- we could plot over the whole the domain x.
We could plot e of x and we could plot r of t tilde, x.
So they're both functions that are defined over the whole domain.
But they're measuring different things, right?
[INAUDIBLE]
Yeah.
That's exactly right.
The error is measuring how wrong is t-- t of x.
What's the error in t. But the residual is measuring how far is the equation from being satisfied.
What's really amazing about the residual?
Yeah?
[INAUDIBLE]
Exactly right.
Which is real-- isn't that cool?
And that's kind of-- I mean, I think in a way that's kind of like the amazing breakthrough here, is that yeah we'd love to know the error, but if we knew the error we would know the true solution, so then why would we be doing all of this?
The really cool thing about the residual is that we don't need to know the true solution.
We can take our approximate solution.
We know the equation that should satisfy.
And we can measure how far off it is.
Now just because the residual is small, doesn't mean the error is going to be small.
And that's a whole kind of field of study.
Depending on the PDE that you're studying, for this particular PDE, it turns out the residual and the error will be nicely related.
But if you have a nasty PDE, the residual might not be a good indication of what's going on.
But it's still a very powerful thing that can give you a sense of how good is your approximate solution without actually knowing the truth.
And we're going to use this idea of a residual over and over and over again, because we don't need to know the actual solution.
Sometimes I know the residual can be something that gets people tripped up, because they're kind of-- it's pretty neat, right?
Take the approximation, substitute it into the PDE, and see how well you're doing.
Not the same thing as the error.
So one thing-- so, no it's not the same as the error, but one thing we do know is that what happens if I substitute an exact solution here, residual is 0 everywhere.
Yeah.
OK. All right.
So how is that going to help us with our collocation method?
So we defined the residual.
And what we're going to do-- you can see now what we're going to do with the collocation method, is when I said that we enforce the PDE, in point what we're going to do is we're going to set the residual to be 0 at n point.
So in other words we're going to pick n. In our case 2 values of x.
Right?
So remember we said that the residual was a function of x, something?
We're going to pick two values of x.
We're going to set the residual to be 0 at those particular points.
Pin them down.
And that's going to give us the two conditions that we need to solve for the two coefficients, a1 and a2, that are multiplying our basic function.
OK?
That logic is clear?
Yep.
So we can just work through the math that actually does this.
And I won't go in too much detail, but if you want me to back up and do any more of the detail, can do.
So in our example, remember we set k to be 1 and q was 50e to the x.
So this is a situation where we do have the exact solution but we don't want to use it.
And again, just to remind you that our approximate solution is this expansion that looks like 100 plus some amount, a1 times our quadratic basic function, c1, plus some amount a2, times our cubic basis function, c2.
And actually let me write that out.
It's 100 plus a1, 1 plus x, 1 minus x, plus a to x, 1 plus x, 1 minus x. OK.
So here's our residual.
It's d by dx-- the k is just 1, so it's actually second derivative of the approximate solution plus q.
So we're going to need a second derivative of our approximate solution with respect to x.
So the 100 is going to go away from this guy.
What am I going to get?
Only the quadratic terms are going to stick around, right?
I'm going to get a minus x squared multiplying a1.
So when I differentiate that-- where I differentiate that twice, I'm going to get minus [?
2a1.
?]
And then from this guy here I'm going to have the cubic term-- is that me beeping?
Oh that's you.
OK.
It's your shoe beeping?
[INAUDIBLE]
OK. All right.
Cubic term.
We're going to have minus x cubed times a a2.
So we're going to get minus [?
6a2 ?]
times x coming out, and then the quadratic terms actually drop away in that last one.
OK.
So there's the second derivative of the approximate solution.
And you know, again, we've still got these constants hanging around, because we don't know what they are yet.
So we can substitute that in to our expression for the residual, which again is a function of our approximate solution and x.
And it's just-- that second derivative is 1 plus q.
So it's going to be minus 2a 1, minus 6a2 times x, plus q, which is 50e to the x. OK.
So there's the expression for the residual for this problem.
So you look at this and you immediately see, well, there's no way the residual can be 0 everywhere, right?
There's no way that this linear term-- no matter what we choose for a1 and a2, there's no way we can make it equal to 50x to the x for all values of x between minus 1 and 1.
So that immediately tells you that those two basis functions we used are not rich enough to describe the solution exactly, which is not really a surprise, right?
But now the question is, what are good choices that we can make for a1 and a2 so that we get a decent solution?
And again, what are we going to do?
With the collocation method we're going to pick a couple of points for x.
And we're going to set the residual to 0 at those points.
That's going to give us the condition.
And then we'll solve for a1 and a2.
So-- yep?
[INAUDIBLE]
So if the source was 0-- if q were equal to 0, and this were the residual, what would you choose for a1 and a2?
You could put them to be 0.
So you could make the residual 0 everywhere.
What would then-- what would our approximate solution be then?
[INAUDIBLE]
Yeah, t equal 100.
So if you think about the physical problem, if you had 0 heat source and you pin the temperature to be 100 and 100 on either end, that would be the solution.
So there's an example of a source where-- I mean, it's kind of a trivial solution but-- of a source where you could get the exact solution.
Yeah?
[INAUDIBLE]
Yeah
[INAUDIBLE]
Yeah.
So the question is, if we have a q that basically we can't do the integration by hand, what would we do?
That's a very good question, and we're going to talk about that, too.
You're going to see some-- in fact you've already seen, probably in the pre-reading, some of the integrals that show up with two.
If you can't integrate analytically there's something called Gaussian quadrature.
So quadrature is a way to numerically do integrals that are basically too hard to do analytically.
So that's going to be in the pre-reading that's due next Monday, and probably [INAUDIBLE] will be covering it in class I would say probably on Monday.
So there are ways that basically we can numerically integrate things that we can't do by hand
Yep?
[INAUDIBLE]
Yeah.
So in that case, is there's going to be-- what you'll see is, when we actually get to talking about the finite element method, that there's a bunch of integrals we have to do.
By using quadrature you're going to actually end up having to approximate the integrals, but be doing integrals over sort of small elements.
But yes, the quadrature is going to introduce a little bit of additional error.
Yeah.
Yeah.
OK.
So we have the expression for the residual.
And so now what we're going to do is pick n equal [?
two ?]
collocation points.
So again, just to remind you, collocation method says enforce the PDE at n equals two points.
And enforcing the PDE, what does that mean?
Enforce the PDE means i.e.
set the residual equal to 0 at two-- and by points we mean two values of x-- two points in the domain.
And here's our domain.
There's x.
Remember, we're going from-- we've got [INAUDIBLE] so it goes from minus 1 to 1.
So there's 0 in the middle.
And the collocation points we're going to choose there and there.
That's minus one third and plus on third.
And those are the collocation points that spread things out the most, kind of away from putting up the boundary conditions here at minus 1 and 1.
So [INAUDIBLE] that we don't really use the collocation method.
It's not a good idea.
One of the reasons we're doing it is because it's a good stepping stone to see how the weighted residual works.
But that's a good question, because in some cases people do use collocation, and aerodynamics is a good example.
And the reason you use the 3/4 quad point is because for a particular kind of left distribution, that integrates you exactly.
And in fact, your question is somewhat related to the question about numerical integration.
When we see quadrature, you're going to see it's the same-- it's going to be different settings, I don't want to confuse you about quadrature-- when you do numerical integration, we're going to put points in the integration domain.
And those points are chosen in a particular way so that we can integrate certain functions exactly.
So it's-- here they've chosen to be spaced out-- there's actually all these rules, they're called quadrature rules, and they're all these different kinds of points, and they actually tend to be named after mathematicians who discovered them-- different patterns of points where it says, space things evenly, or distribute them out.
That's actually a pretty interesting area of study.
In fact, Alex I would say knows a lot more about these things than I do.
Is that true, Alex?
What's your favorite set of points?
[INAUDIBLE]
[INAUDIBLE]?
So depending on the function you're trying to integrate, there are sort of optimal choices of where you might put the point
[INAUDIBLE]
Yeah, something about the solution or something about the basis function that you're using to represent the solution
[INAUDIBLE]
Yeah.
So that's a really good--
[INAUDIBLE]
--that's a really good question.
So if you're trying to approximate things, but you know there's something in the middle that you would really care about-- so now what you're talking about is an adaptive strategy.
And again, this is a very current area of research.
In fact, a lot of [?
professor ?]
[INAUDIBLE] research [?
facilities ?]
I think mention there at the beginning that uses adjoint methods.
So that would be bringing in information that tells you where the regions in the domain that are most sensitive to what it is you care about, where might you put more points.
People who choose points to do this quadrature, or this numerical integration, sometimes will choose to adapt in these ways.
And what you would like is, you would like kind of general rules that work well, but then you would like to be able to adapt depending on your particular problem of interest.
And there are definitely a lot of adaptive strategies out there, but it's also an area that a lot of people are working on.
Good question.
So you guys are all ready for the lecture on quadrature.
In fact, when we come to talk about Monte Carlo integration-- Monte Carlo methods-- I mean, evaluating means and variances.
That's just integration as well.
That's all just about putting points in a domain.
And what you're going to see is Monte Carlo does it randomly, but in some cases you can do better by putting them in a very spec-- putting the points in a very specific way.
OK.
But here we're going to do just [INAUDIBLE] we're in 1D, so, in 1D you can usually get away with pretty much anything.
We're going to just put them at minus a third and a third.
So what that is, again, [?
sit ?]
the residual.
And I'm purposely writing out all the functional dependencies.
I keep writing r of t tilde comma x because it helps me remember that residual is a function of x.
And so what I'm doing here is I'm evaluating the residual at the point x equal to minus one third, that's a point on my domain.
So that means substitute x equal minus one third into my expression for the residual, which gives this 50e to the minus one third, and put that equal to 0.
Right?
So there's the first condition.
And then the second condition is original evaluated plus one third.
Again, substitute that in to the expression.
So we're just changing the sign here.
And then this is going to be 50e to the plus one third.
And that's been set equal to 0.
So now you can see we reduced-- we keep sort of reducing the problem.
The first thing we did was assume the functions-- these guys-- that we want to approximate the solution in.
Then we take-- we've got two degrees of freedom to unknown, so let's use a collocation method.
We set the residual to be 0 at two points.
We chose these two points.
We've enforced these conditions.
Now we have two equations, two unknowns.
We could solve that-- you could probably solve that analytically, or using mathematical, or whatever you like.
And what you find is that a1 and a2-- 26.4 and 8.5.
So those numbers don't mean a whole lot to us, but again, what's the key?
The key is that this is our approximation of the solution.
So we're saying that by the collocation method, the approximate solution of t tilde is 100 plus 26.4 times something that looks like that-- a quadtratic-- minus plus 8.5 times something that looks like that [?
cubed.
?]
And when you add those together you get something that is at least an approximate solution to the PDE.
So I want to take a look at what that solution looks like.
While I put the projector on, are there questions about the collocation method?
If we were actually using collocation method, we would use more than two points typically.
My simple example.
So-- so I have a code here that actually implements-- I guess I should plug my laptop in.
OK.
So the code here that's going to implement it, there are the two basis functions, c1 is 1 minus x, 1 plus x. c2 is x times that-- the cubic.
And the collocation method just gets implemented here.
It's kind of hardwired because all the derivations have been done-- sort of done offline.
But let's just run this.
Won't run that yet.
OK.
So one of the parts that we're looking at here-- so here first, of all, is a plot of the temperature versus x.
And the solid blue is the exact line, that's the one that we computed analytically for this problem, we can do that.
And a dashed line is what we got with collocation.
OK, so it actually doesn't do too badly, right?
We sort of assumed this [INAUDIBLE] of the solution with the solution being 100 plus a quadratic plus a cubic.
We figured out just with two degrees of freedom what good choices for a1 and a2 would be.
It's actually not too bad.
Right?
So there's a plot of the actual compared to the collocation's approximate solution.
Here's a plot of the error.
So that's t minus t tilde.
We can compute again in this case because we happen to have the exact solution.
You can see that error is 0 on either end.
And you could see from the previous part that the exact solution-- the approximate solution was always under-predicting in temperature [INAUDIBLE] to be exact.
So you see that with the error being negative.
OK?
Now for this problem we can look at these because we do actually know what the exact solution is.
We're not going to be able to do that in general.
But what we could plot is the residual.
So again, what is this?
This is r of t tilde comma x.
And now we're taking the t tilde that we determined.
We specified the form of it.
We chose a1 and a2.
And remember, it's a function of x.
So here's the residual plotted out as a function of x.
And now you should be able to see, this is where you can make sure you did things correctly.
This guy should be 0 where?
Here at minus a third, and here at plus a third.
The two points at which we asked the residual will be 0, and then everywhere else it ends up being non-zero.
Yep.
OK.
So that is it for collocation method.
Any questions?
Good?
It's clear?
All right.
So now we're going to talk about the weighted residuals method-- the method of weighted residuals.
And this is really the method that we want to focus in on because this is going to be the launching point for finite elements.
But again, we were talking about collocation just sort of as a way for you to start thinking about what it means to approximate a solution with a finite number of basis functions, and also to think about there being different ways for you to actually come up with the coefficients.
Yeah, [?
Libby?
?]
[INAUDIBLE]
Maybe start with the error.
You have to deal with the fact that error is 0 at the end point
[INAUDIBLE]
So let's try to put them side by side
[INAUDIBLE]
Yeah.
Yeah, that's exactly right.
So if we start with the error, shold the error-- it's warming up-- yeah?
[INAUDIBLE]
Yeah.
By construction the error is 0, right?
Because we constructed the basis functions to be 0 at either end.
So no matter what value we choose for a1 and a2, the approximate solution will be exact at the boundary condition.
What about the residual?
[INAUDIBLE]
Yeah.
Say it a bit louder
[INAUDIBLE]
Yeah.
So it's specifically what do you think is not what it should be at the end point.
So I'll put this up [INAUDIBLE].
So derivatives.
Yeah, exactly right.
So it basically is telling you that-- and you can kind of-- you can see it physically in the shape, but you can also see it mathematically because it has this form.
When you take a derivative, as soon as you move away a little bit, things are going to go astray because second derivative is not right.
and at the endpoint, the derivatives are not what they should be
[INAUDIBLE]
Yeah, it's not-- would only-- no, come on, [INAUDIBLE] turn it off.
We've only got two degrees of freedom, right?
We see the solution is a constant plus a quadratic term plus the cubic term.
We've only got two degrees of freedom.
So you only get to pick two things, right, to put them down.
And we said, let's make a residual be 0 at minus a third and plus a third.
I guess we could have chosen to make a residual 0 at the two end points.
I don't know what solution-- maybe Alex can quickly figure out in his head what solution that would have been.
That would be kind of a bizarre choice, but I'm not-- I'm not sure is that-- I'm just wondering if that might give a 0 solution everywhere.
But you can't you can't kind of have everything.
You only got-- we only got the residual at the minus one third and the plus one third collocation point.
There seems to be a lag, a big lag, in the projector-- there we go, get it off.
OK.
So any other questions about collocation?
[INAUDIBLE]
Yeah, so that's a good question.
If you look at the residual could you tell whether you're using too low of an approximation?
In other words, can you tell by looking at the residual how bad the error might be in the solution?
And the answer is, yes.
And there's some beautiful theory-- again, it's going to depend on PDE that you're talking about-- this theory that basically says that it relates the norm of the residual-- how big is the residual integrated over the domain-- compared to how big is the error integrated over the domain.
And they're related by a constant, which depends on the properties of the PDE you're solving.
And that comes back to what I was talking about earlier, that for a nice PDE like this 1D diffusion, it actually turns out that looking at the residual is a really good way to understand what would be happening with the error.
For nastier PDEs, that's not so much.
But what you're sort of asking about is to a lot of the theory of the error analysis in the finite element method-- that people do a lot of very rigorous analysis of error based on being able to compute the residual.
And in fact, the residuals also can then start being an indicator for figuring out how to do things like [INAUDIBLE] refinement, and all these things are kind of related.
And then the theory with adjoint methods.
So if you're interested in that stuff you have to take [?
16920 ?
], which Professor [?
Wong ?]
will be teaching in the fall, which is the grad class on numerical methods of PDEs.
I don't know how much you get into that stuff
[INAUDIBLE]
Little bit more than here.
But you have to take that class so that then you can take [?
16930 ?]
which really would get you into it.
Right?
Yeah.
OK. That's good.
That's great questions.
So now, if you guys are ready, we can talk about the method of weighted residuals, which hopefully won't be too much of a jump from where we are, but it is a different philosophy.
So everything is the same up until the point that we define this residual.
But where we're going to kind of take a fork in the road is, now we're going to use a different strategy for coming up with the two conditions to figure out what those two degrees of freedom are.
OK?
So same idea-- approximating the solution with the basis functions and [?
the extension ?].
But now what's different is, how do we come up with the conditions to determine the functions a1 and a2.
And-- so-- method of weighted residuals.
OK.
So in order to move forward-- in other words, in order to find these two conditions, we're going to define something else.
We're going to define what's called a weighted residual.
OK?
So we defined the residual-- it still sits over here-- which is the thing that we get when we substitute the approximate solution into the PDE.
Now I'm going to define the weighted residual, and I'm going to call it-- and actually, I was realizing when I was going through that the notation in the notes jumped around a little bit-- so let's call this capital R sub i.
It's going to be a function of the approximate solution t tilde, but not a function of x because we're going to integrate things out of x.
So we're going to define it as the integral over the domain.
And again, the domain in our simple problem is just x from minus 1 to 1.
We're going to take a [?
weighting ?]
function, which we'll write as wi, it's a function of x.
And we're going to take our residual-- you know what, I'm going to give this a little r just to make sure that they're different.
And that's going to be integrated over ex.
So again, little r sub i is going to be our i'th weighted residual.
And what you're going to see is that we're going to need n of these things.
So again, we're looking for n conditions to find out n, a non-coefficient, and it's defined to be the integral of a weighting function, which is our wi, times this guy here, which is our residual.
And we're integrating over the domain.
Here's x going from minus 1 to 1.
OK.
So residual is a function of x, weighted residual is not a function of x because I've integrated over, and the integration is weighted by some weighting function wi.
But the weight of residual is still a function of our approximate solution t tilde.
OK.
So now what does the method of weighted residuals do?
It says let's choose n different weighting functions, w1, w2, up to wn.
Let's define the n corresponding weighted residuals, and let's set each of those weighted residuals equal to 0.
Each time we set a weighted residual to 0 we're going to get one condition, right?
So we'll do that n times with n different weighting functions, and that will give us the n conditions we need to compute our n a coefficients.
Yes?
Seems like kind of a bizarre thing to them.
It's OK?
You guys are so quiet.
Does that mean that you think this is very easy, or kind of weird?
We don't know.
So let's write it out a little bit and see.
So again, the method of weighted residuals-- we're going to require n weighted residuals to be 0.
So that means we're going to have to choose n weighting function.
Those are the w1, w2, up to wn.
They're all functions of x.
And we're going to then get, again, n equations to determine these coefficients, a1, a2, to a n. OK.
So first thing we need to do is choose n weighting function.
So this is the first question, is now what do we choose for the wi's?
What do we choose here?
So the answer is that there is a variety of things we can choose, but there's a very special choice that results in what's called a Galerkin method.
What does the Galerkin method say to do for the weighting functions?
You guys read about this
[INAUDIBLE]
Yeah.
Exactly right.
Let the weighting function be the same basis function that you're using to approximate the solution.
OK?
So the same [INAUDIBLE], that we used-- we've erased it now-- but that we used in our in our expansion of t tilde-- choose the same basis function to be the weighting functions that you use to define your weighted residuals.
Turns out don't have to do that.
There's a bunch of other methods.
If [INAUDIBLE] Galerkin methods, we can choose different weighting functions.
But we're only in this class going to consider Galerkin methods where you choose the weighting functions to be the same basis functions that we're using to approximate the solution.
So the Galerkin method is a very special choice that says choose wj to be equal to [?
cj.
?]
And it turns out that this choice has really good properties for some PDEs.
And again, if you take [?
16920 ?]
with Professor [?
Wong ?]
in the fall, you would see how this plays out from an energy perspective.
OK.
So this just means the same functions that you use to approximate the solution, they're going to be used to weight the residuals.
OK.
So for our example, what does that mean?
That means-- coming back to what we were doing before-- that means that the first weighting function would be our c1, which was 1 minus x, 1 plus x.
And the second weighting function would be the cubic, which is x times all of that.
And then the first weighted residual, 1, which is a function of t tilde, would be the integral from minus 1 to 1 of w1 of x times r of t, xdx.
And the second weighted residual would be the second weighting function multiplied by the residual integrated over x from minus 1 to 1.
OK.
I don't want to work through all the math on the board.
But basically you can see what's going to happen here, right?
We're going to substitute in w1 as 1 minus x, 1 plus x.
We have the expression for this guy that we derived before that was minus 2a1 minus 6a2x plus 50 e to the x.
So now at that point it's just multiplying things together and integrating.
And you could do it all by hand for this example, or you could again throw it into [?
Mathematica ?]
or something and get it out.
And then what are we going to do?
We're going to set our 1 equal to 0, and we're going to set our 2 equal to 0.
It's going to give us two conditions, two unknowns.
Solve those.
And we're going to come out with, in this case, a solution that says a1 is 27.6 and a2 is 8.9.
OK, so I skipped over a lot of messy math there, but I don't think there's anything conceptually difficult.
Two conditions now coming from setting a weighted residuals equal to 0, where the weighted residuals correspond to taking the residual, multiplying it by the basis function, the Galerkin, integrating over the domain.
Plugging through all the math, solving the equation, gets us this solution which says that by the method of weighted residuals for this choice of the basis function-- the cubic and the quadtratic-- in a Galerkin method the solution we get is-- the approximate solution is 100 plus 27.6 amount of our quadratic basis function, plus 8.9 amount of our cubic basis function.
So the different solution to what we got with collocation, right?
We got to it through a different route.
So I'll follow it up, and we can look at that solution and we can compare it to a collocation, but first, questions about method weighted residuals.
Does that make sense?
[INAUDIBLE]
Yeah.
So that's a good-- that's a good question.
The answer is going to be, it's going to depend on the problem.
So there's no general guidelines that I can give you.
[AUDIO OUT] you're [INAUDIBLE] exactly right is that, because the-- what came out of the collocation method, and we'll see it in a second-- the residual was-- I forget whether it was positive or negative everywhere, but it always had one sign.
And we're going to see that the method of weighted residuals actually kind of balances out.
Now that's going to make sense, because what are we doing when we-- when we take the residual, and we weight it, and then we integrated it.
I think of this as kind of like-- it's kind of like we're asking the residual to be 0 on average over the domain, but on average weighted by w1.
Right?
Because if this were gone and I just integrated it and set it equal to 0, it would be like saying [INAUDIBLE] the residual on average 0 over the domain.
So it's like taking an average, but weighting it by w1.
So yeah, I'm going to expect this to come out with something that's going to be more evenly distributed, rather than with collocation where had all of one sign.
But there's not-- I mean you want to be careful about-- I think you want to be careful because it's going to depend on the problem.
You'll see when I [INAUDIBLE] in a second that it actually turns out our residual will be 0 at a particular point x.
Not because we set it that way, but because that's how it comes out.
So another [INAUDIBLE] check would be to go back and plug that point in and make sure that the residual-- actually worried about getting the integration wrong.
Take that point, x, plug it in, and make sure that indeed the 0 is satisfied.
But I said earlier we don't really use the collocation method, but we do use method of weighted residuals, that it's going to be, because it actually turns out to have really great properties.
It wins for this problem, and that's not really a coincidence.
And in fact it forms the basis of what we're going to do with the finite element method, which is incredibly powerful.
OK.
So I am going to run the same script that I ran before.
But now I also have the implementation of the weighted residual method.
So here are-- so this is basically, this is the combination of all those integrals over there-- amounts of the [INAUDIBLE] matrix-- this k here, this matrix, and the coefficients being [INAUDIBLE] times b.
So all the integrations amount to that.
And so we're going to run it and plot.
So let me-- clear all, close all-- OK.
So just like we looked at before, we can look at the temperature as a function of x.
Solid lines, just like before, is the exact solution.
Dash line is the method of weighted residual solutions.
So remember, we've got the same degrees of freedom as we had for collocation, right?
Still any two degrees of freedom.
And our same basis functions.
The only thing we've changed is the way we choose how much of each basis function to put in.
And you can kind of see visually-- I'll put them both up together-- you can see visually it does pretty well.
And actually what Kevin was just observing, it's not always under.
It actually is under a little bit here, and over a little bit there, and in under a little bit here.
It's kind of just the way it came out for this problem.
So there's the temperature compared to the exact solution.
We can look at the error as a function of x again because we have the exact solution here.
And again we can see it's negative, then it's positive, then it's negative.
And we can look at the residual.
And so now, remember with collocation, what did we see?
We saw with the collocation method that the residual was 0 at the collocation point, because those were the conditions that we imposed to get the coefficient.
Here it turns out that the residual is 0 minus 0.4 and at plus 0.5.
But that just kind of fell out.
And again, if you're looking for check to make sure you did the integrations right, you can always take your residual, substitute in whatever these values are, and make it came out to 0.
But again, that just fell out.
What we actually ask for was that this residual, when weighted by this guy and an integrator over the domain, is 0.
And this residual, when weighted by this guy-- the cubic-- an integrator of the domain-- that is 0.
And so then finally-- let's see-- plot the comparisons.
So I put them all on the same plot.
So there in black is the exact solution.
The red is the method of weighted residuals.
The blue is collocation.
Turns out method of weighted residuals did better for this problem, but again we want to be careful about making general assumptions.
They both have the same number of degrees of freedom.
They only differ in the choice of those coefficients.
Here's a plot of the difference errors.
And again, I would say the method of weighted individuals is kind of more balanced, again, because we are asking for this weighted residual to be zero.
And then lastly, here are the plots of the residuals as a function of x.
And now you can see clearly what I was saying.
There's the collocation [INAUDIBLE] to 0 at the collation point.
The method of weighted residuals happens to be 0 at a couple of other points.
OK?
More questions.
I'd say it's really important that you feel very comfortable with the method of weighted residuals.
The idea-- and let me just say it again, the idea that we're going to take the PDE-- we're not going to mess around with the derivatives, we're going to instead say, let's approximate the solution and an expansion with a finite number of basis function.
We're going to choose the form of the basis functions-- and they were even talking about polynomial basis function.
Now we need to figure out a way to determine the coefficients-- how much of each basis function.
And so the way we're going to do that is choose weighting functions, and with Galerkin methods we'll choose those weighting functions to be the same basis function that we're approximating the solution with.
We'll weight the residual with the weighting function, integrate it over the domain, define the weighted residuals, set those equal to 0, and get the conditions we need to find the coefficients on our expansion.
All those sets.
And what we've done today is define global basis functions, right?
So my c1 was a quadratic that varied over the whole domain, and my c2 was a cubic that varied over the whole domain.
Finite element method-- you're all ready for it now-- is just a, I guess kind of a simple, but really pretty huge next step, which is to say, let's not do this on the whole domain, let's divide domain up into little pieces, and let's use this idea.
Let's define basis functions-- special basis functions that are polynomials just on the little pieces.
And then we're going to have coefficients that go with those.
Yeah, [?
Tran?
?]
[INAUDIBLE]
Yeah.
So that's a good-- [INAUDIBLE] In that case, the weighted residual-- we're going to see all that on Wednesday.
So what's going to happen now is that this weighted residual is actually going to have very special structure, because the way we're going to define the basis functions is that they're going to be local on elements-- the finite elements of the finite element method.
And so when we do the integration, a whole bunch of it's going to disappear, and we're going to be integrating locally just over the elements.
But what you're going to see is that there ends up being a little bit of interaction between one element and its neighbors because of the way the basic functions are going to come out.
So we'll work through all those integrals.
I think you read maybe a little bit of it already.
How far did you guys get in the reading?
You saw the linear, [INAUDIBLE]?
Yeah.
Yeah, we're going to work through all of that.
And you'll see that's what's incredibly powerful with the finite element method, is this idea of using a polynomial basis and a local element, we're going to get these integrals with so much structure that we can handle-- it's going to sort of all work out in a really neat way
[INAUDIBLE]
That's exactly right.
We're still integrating a weighted residual over the whole domain, but a whole lot of it's going to go away, and it's going to turn out that we get sort of these special patterns that show up because of the way which is a basis functions.
And you'll see all of that.
We'll do all of that.
I think on Wednesday we'll sort of work through all the steps of the finite element method and have you go along and do it, so actually bring your laptops if you can so you can do some stuff on Matlab.
OK. Yeah, Ben?
[INAUDIBLE]
You know, that's a good question
[INAUDIBLE]
That's a good question.
The question is, is the fact that the method of weighted residuals, the resulting residual crosses 0 in time, is that a coincidence, or is it always going to happen?
I think that's a very interesting question, and I'm actually not sure, because I was wondering actually, is there a way to figure out where we could have put the collocation points for this problem.
We could have put the collocation points in a particular point and gotten the same answer that we got with the method of weighted residuals.
And it's going to have something to do with the 50e to the x that's in the [INAUDIBLE] function.
I just think the fact that it crosses twice is not a coincidence.
I think it probably has to cross twice, but I don't--
[INAUDIBLE]
You feel like it's kind of like a fundamental theorem?
[INAUDIBLE]
Maybe it's a great question for the final.
Really good final questions are when the professors don't know the answer What do you think, [?
Xixi.?
?]
[INAUDIBLE]
Yeah.
Well the thing is, we have an expression for the residual, which we wrote out, which was-- what's the expression for our residuall-- it's minus 2a1 minus 6a2x plus 50e to the x.
So I guess you can figure out how many crossings are possible because of the degrees of freedom that we have.
It actually turns out that even though things seem like coincidences, there are almost no coincidences.
In these kinds of problems, it's usually that there's so much structure in the problem that you could have-- but-- yeah.
OK.
If you have-- if there are parts of the method of weighted residuals that are a little bi uncomfortable for you, I would strongly suggest you talk to me, or to Alex, [?
Xixi, ?]
or to [?
Bikram ?
], before Wednesday, because otherwise things are going to get dramatically worse for you on Wednesday.
Make sure that this-- really-- make sure-- that's why I went kind of slowly through this lecture, is because it's really important that you have all these steps kind of clear in your mind.
And please if you have a chance to even just listen to a few minutes, especially those of you who said you like the audio recording-- if the combination of what's on the screen and me talking is good enough then I can keep using the blackboard, but if not please let me know.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu
All right.
So let's see.
I have the graded projects here which maybe you guys can grab after class.
Or you can grab them now if you want but it can take a while.
Vikram, maybe you want to give the projects back.
Just quietly ask people what their names are.
OK.
So today, we're going to talk about finite elements.
You guys have read a little bit in the reading.
I'm going to talk about a couple things first, just go through a couple of the very basic ideas.
And then we're going to do it interactively.
You're going to actually implement the one 1D finite element codes.
We'll do it piece by piece together.
So let me just put the topics up over here.
So again we're going to talk about 1D finite elements.
We're going to start off just with basically thinking about the key ideas.
We'll just look at a very brief history of finite elements.
We'll think about what 1D linear elements look like.
We'll talk about the nodal bases.
And then we will spend most of the session doing the derivation and implementation for the 1D diffusion equation.
Same derivation and implementation for 1D diffusion.
OK.
So by the end of today, you'll more or less have a basic shell of a code that implements at least the meaty part of the finite element.
I'm not quite sure how far we'll get today.
Then on Monday, I am unfortunately not going to be here.
But Vikram is going to run the class session.
And you guys will continue working with that code.
Probably Monday you'll implement the boundary condition and implement some quadrature schemes in it.
And so then, by the end of class on Monday, you should have a working 1D finite element code which will then hopefully be of help to do the 2D that you have to do for the project.
And on Wednesday of next week, Vikram will talk a little bit about 2D finite elements.
So let's just start off with key ideas.
I'm just going to write over there for a bit because I want to put something up on the screen in a second.
So remember on Monday, we talked about the method of weighted residuals.
And again, just to sort of go over what the steps were in the method of weighted residuals.
Kind of the fundamental idea is take the solution and to approximate it in a basis.
Right?
Remember we took the intradimensional PDE and we discretized it.
And this is already different to what we do in finite difference or finite volume.
We discretized it by saying let's assume that a solution t of x lives in this finite dimensional space described as a weighted sum of basis functions.
Then we said how do we figure out the coefficient of the basis function?
And we went through the derivation of choosing weighting functions to be the same functions that we used in the basis to approximate the solution.
What was that called?
What's it called when you choose the weighting functions to be the same functions that you think [INAUDIBLE]?
Galerkin.
Yeah.
It's the Galerkin method.
When we define the residual, multiplying by the weighting function using Galerkin.
Integrating over the domain, setting that equal to zero.
That's the weighted residual.
Those gave us the equations we solve to find the approximation.
So we're more or less going to do the same thing now with finite elements.
So what you are going to see today is that a finite element as we're going to view it is based on the method of weighted residuals.
But what is different is the first step.
Before we start with the approximation of the solution, the very first thing we're going to do is to discretize the domain into small cells.
OK?
So we're going to divide up our domain-- rather than thinking about the approximation of the solution globally over the whole domain, we're going to discretize our domain into small cells, elements.
That's the element of the finite element method.
They've been called-- I think you guys called them cells when you talked about finite difference.
Is that right?
Nodes and cells?
And for finite elements, the cells are referred to as elements.
Then once we've discretized the domain into small cells, then we're just going to go and basically do what we did with negative weighted residuals.
So now we're going to think about approximating the solution in each element.
So in each element, we're going to say let's assume that the solution can be approximated as a linear combination of basis functions.
And we're going to use polynomials just like we did on Monday.
But now the polynomials are just going to be defined over the little element, the little piece of the domain.
Not the whole domain.
E.g.
Here with polynomial functions.
Again, just like we did on Monday but now to serve a small element polynomial function.
And then again, it's going to be the same as what we did on Monday.
Once we've done the approximation, we're going to evaluate weighted residuals for each element.
We're going to use the Galerkin method to do that, to define the weighted residual.
That's going to give us a system of equations.
And we're going to solve to determine the weighting coefficient.
Weighting coefficients.
So basically exactly what we did on Monday, except that first step which is discretize the domain to small cells.
And then what you're going to see is that idea of breaking the domain up into small cells, we're going to make a very special choice for the polynomial basis functions.
It's going to be really neat because when we do the weighted residuals and we do all the integrals-- like we mentioned a little bit, I think, in response to one of [INAUDIBLE] questions-- a whole bunch of integrals are going to fall out.
And it'll turn out that everything has got a very special and really efficient and kind of neat structure to it
Did you pick the same basis functions for the entire problem or for each individual cell?
Yeah.
So the question is do you pick the same basic functions for the entire problem, or for each individual cell?
You're going to see it in just a second.
There are a lot of different ways to choose the basic function.
Today we're going to talk about a special choice that's referred to as the nodal basis.
And you'll see exactly what they are.
Before I start, are there any kind of residual questions from method of weighted residuals?
Residual questions.
Are there any things about method of weighted residuals that is mysterious or uncomfortable?
It's pretty straightforward, right?
So just before we start, I want to give you a little bit of a-- I hate this WebEx thing.
Because when you try to grab things, it goes down-- little bit of a historical perspective.
And actually I went to Wikipedia as you do, which actually doesn't have a bad-- this is the Wikipedia entry for finite element method, which actually surprised me.
Look how many sections there are.
Vikram, have you contributing to Wikipedia?
No.
[LAUGHTER]
Aha.
There's actually quite a lot here.
I didn't read it all.
So I can't verify how good it is.
Although Wikipedia appears to be pretty good, there's enough of a community of people.
So I was going to show you guys the history about technical discussion, general principles of weak formulation, discretization.
Then you can see all the different kinds of finite element methods that exist.
So that should give you a sense of just how big of a field of study this is.
We're going to be just a little tiny bit of it.
[INAUDIBLE] for finite difference method application.
I just wanted to mention a little bit just about the history.
So like it says here, it's difficult to quote the date of the invention of the finite element method.
More or less, its origins are actually from structures in both civil and aerospace aeronautical engineering.
So it has roots in aeronautical engineering.
And there were three kind of groups where if you trace back, that's where things seem to have really originated.
Courant is often the person whose is cited as being, I guess, the father of a lot of theories that led to finite elements.
So he was at NYU and then established an Applied Math institute, the Courant Institute which still exists today.
It is one of the best Applied Math computational places in the world.
And as you can see here, I think this is a German mathematician.
And also things were going on in China at the same time.
So there's a little bit here.
You can see there Rayleigh, Ritz, and Galerkin.
Those are all names of mathematicians that you guys have seen in various methods, things that came up.
Really the way the finite element method started to really sort of become like a computational powerhouse, I think, was here in the 60s and 70s.
So Agyris at the University of Stuttgart.
I think this is pronounced Clough.
Is that right, Vikram?
Yeah.
Clough at UC Berkeley.
And Zienkiewicz at University of Swansea.
And in fact University of Swansea is where Professor [INAUDIBLE] did his PhD.
And he was working with some of the later people there that were at the forefront of finite element methods and their development.
So it's actually a really kind of neat story because it really comes from applied math.
And you're not going to see a whole lot of it in class.
But there's a lot of incredible analysis and theoretical results that go with finite element methods, a lot of guarantees that can be made.
So there's really a hardcore math component.
But then it has gone on to be of such practical use.
So it started off in structural problems but now is applied to fluid problems, acoustics, all kinds of things.
You can see here NASA sponsored the original vision of NASTRAN.
I actually didn't know that until I read this.
So now there's a lot of software, things like NASTRAN ADINA is Professor Bathe from mechanical engineering package.
Professor Strang in the math department also plays a really important role in some of the rigorous results early on.
So there's also a lot of connections to MIT.
So we're going to see only just a little piece of the finite element method.
But it is certainly, I think, one of the really great examples of where applied methods made enormous impact on real problems and particularly on engineering and engineering design.
So let's get into talking about how it works.
We're going to talk about 1D linear elements.
I'm just going to go over a few things that you guys read about just to make sure that the notation is clear and that the ideas are clear.
And then we're going to start together deriving and implementing the methods for the 1D diffusion equation.
So we're at number three.
1D linear elements.
OK.
So I'm just going to draw a picture.
And again this is the picture that was in the online reading.
Board is not very clean.
All right.
So we're thinking in 1D.
So draw this line that's going to go in both directions.
So this is the x direction.
And I'm going to have nodes just like I had in finite difference.
So I draw some nodes.
This guy here is going to be node i.
And this guy here is going to be node i plus one.
So node i is located in at position xi.
Node i plus one is located at position xi plus one.
i minus one to the left and so on.
And then in between xi and xi plus one is element i. OK?
So when we talk about element i, we're talking about the region of the domain that lies between xi and xi plus one.
And--
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So first up, bootstrapping, which is really aimed at this question how do we get estimates of the standard errors and our estimators that don't have known distribution?
So remember we were talking on Wednesday.
I don't even know what day it is today.
Monday.
We were talking on Monday about how you can run the Monte Carlo simulation and then we know, for example, the mean estimate and the limit then goes to infinity by the central limit theorem.
Then the distribution of the estimator itself, estimate for the main is normal.
And the mean of that distribution is the actual mean that we're trying to estimate.
It's unbiased.
And the standard deviation of that distribution, remember, was the standard deviation of the quantity we're putting in an output divided by square root of n. But we also said that, for example, the estimate of the variance-- remember Alex told you three different ways to estimate variance-- that those estimators don't have known distributions.
They're not necessarily following a normal distribution or any other known distribution.
So bootstrapping is a trick, and actually, as I was putting together today's lecture, I [AUDIO OUT] tricks.
Statisticians tend to have lots of tricks up their sleeves-- just going to pull out the section on the notes because I think you should have read this-- as to how bootstrapping works.
And there's a lot of words here, but let's just come down to the [INAUDIBLE].
So what do we do in bootstrapping is perform our initial Monte Carlo sample, just like we've been talking about.
So take the inputs, draw in realizations from the input distributions, run them each through the model, produce the samples, the yi's, and compute the desired estimator So this theta here might be a mean estimate or think of it as a variance instrument, one we can't necessarily easily do the confidence interval analysis on.
Then what bootstrapping says is, well, now you've got these N samples.
And remember each one of these samples is being run through your finite element model, run through your CAD model.
Let's now re sample those values.
But N samples of y, N samples of the blade middle hot side temperatures sitting in the bin, let's just re sample from those ones.
And why is that efficient?
Because we don't have to run the model anymore.
We don't have to do the finite element analysis.
And we're just going to re sample from those with equal probability.
So probability 1 over N int, we do it with replacement, so you could pick the same sample more than once.
So they're pretty clear.
We had N samples.
Then we're going to sample N time from those, but we're going to get a different set of N because, I mean, maybe we'll pick each one once, but more likely we're going to pick that one a couple of times and this one not at all.
So we'll have a different sampling, we could compute the estimator, and we could keep doing that N minus 1 times.
In the end, we're going to end up with N values to the estimator.
And from there you could try to determine competence intervals.
So it's kind of a trick because it's kind of cheating.
And it's not the same thing as running a new Monte Carlo, but hopefully, you can see that this is something you can do with almost no more computational cost, and it actually turns out that if you do some analysis, this is a reasonable thing to do and it take a lot of statisticians to analyze it.
So when people talk about bootstrapping in this context, this is what they're talking about, re sampling from your samples, putting the information together in different ways, and then using that to get a sense of the distribution and say that they're [?
untestable.
?]
Is that clear?
So [AUDIO OUT] internet, so I didn't get time to go through it on Monday.
But I mostly want us to spin today talking about different ways to reduce variance.
And again, the really important idea that should be in your heads is the quality of the estimators that come out depends on how many samples in the Monte Carlo.
And if you can afford millions of samples, you can probably get really good estimates for the mean and maybe OK estimates of the variance.
But if, for example, you're interested in tail probabilities, so think about an aircraft design, a new flight critical system.
Maybe they has to be a 10 to the minus 9 failure probability that we have to satisfy.
So if we want to estimate that by Monte Carlo simulation, how many samples do we need?
A lot.
How many is a lot?
At least how many?
At least a billion.
If a probability is 10 to the minus 9 and we had a billion samples, then on average, you would expect one sample to be that failure.
So that's not even going to be enough because you might get 9, you might get 1, you might get 2.
So that means you're going to estimate this probability differently.
So billions, tens of billions, hundreds of billions of samples to estimate what are called [?
variates, ?]
but even just in general, even just means, if you're talking about running a CSC code or a finite element simulation that takes minutes to run, we usually can't even afford hundreds of thousands of simulations.
So the variance reduction method, trying to increase the accuracy of our Monte Carlo estimates for a given number of iterations.
And I think of them, their kind of tricks, and they're really clever tricks.
I'll try to show you how, at least, important sampling works.
And you can think of it in two ways.
Here, if you've got a good number of samples, I'm willing to run a million samples, how much accuracy can I get in my estimators?
Or you could think if I want to achieve a given level of accuracy in my estimator, how many samples do I need, and can we reduce the number of samples?
So there are a lot of different ways that people do variance reduction-- important sampling, what's called antithetic sampling, control variates, stratified sampling.
And I've decided I was going to talk about control variates and importance sampling, but I think we'll just talk about importance sampling today because it's already quite a lot to think about.
So in general importance sampling, is a general technique for estimating properties of one distribution while only having samples generated from another different distribution.
So keep this in mind as we go through, because it's a key idea.
We're going to have samples generated from one distribution, and we're going to figure out how to manipulate those samples so that they can estimate things about a different distribution for us.
And it seems kind of like a wacky thing to do, but what you'll see is that if you pick this distribution carefully, we can say things about the statistics of the other distribution in a more efficient way with the Monte Carlo estimates.
So let me get the screen up.
And so we're at number 3, important sampling.
Not a very good piece of chalk.
And so let's think about a random variable, x, that has pdf f sub x, and it has mean that we'll call mu sub x, which is the expectation of x.
And I'm going to put a little x on the subscript on the expectation to just denote that we're taking an expectation under the density f sub x.
So what I mean by that?
If you want to think about an expectation as an integral, and remember Alex talked about this on Monday, the expectation of x mu of x is the integral of what?
x times it's pdf dx.
And so again, you can see where we're going with this.
Remember we talked about having a different distribution.
We want to be clear that when we're talking about the expectation of the random variable, x, we're talking about taking the expectation with respect to it or under its pdf f sub x.
Is the notation clear?
Yes?
So we have this random variable, x.
We may be interested in estimating its mean, and we know that this is the definition of the mean.
It's an integral of x weighted by the pdf of x. I didn't put limits here, but this is an integral over this 1/4 of x, which would be minus infinity to infinity if x was a normal interval.
If it was uniform, it would be an integral over the ab range, what this 1/4 of x would be.
So now what we're going to do is we're going to choose a random variable that we'll call z.
And we're going to ask z to be non-negative, and we're going to choose it subject to the constraints that the expectation of z taken under the pdf-- still the pdf here-- of fx is equal to 1.
And why are we writing this?
Well, what's the 6th notation?
It's the integral of z f sub x of x dx.
That's the expectation of the random variable, v, under the pdf fx.
So if this is equal to 1, maybe you can see that all we've done is really define another pdf that we can write as f sub z. Yep.
Because it's going to satisfy the properties of a pdf.
It's going to integrate to 1.
Is that OK?
[INAUDIBLE]
z is going to be just another random variable.
Yeah, so you can maybe just think of z as being defined on the same support if you want to, or I think this thing is allowed to be 0 in some places, but it's not allowed to be 0 everywhere.
There are some conditions related to where z can be 0.
Is that OK for the lecturer to ask you a question?
[INAUDIBLE]
Yeah.
So I think the easiest thing is to think about x as having infinite support, then integrals away from minus infinity [INAUDIBLE].
So we haven't done anything yet.
We've just manipulated things.
So why are we doing this?
So if we did this, we'd have to think about the mean of x, mu x, the thing here, which is, again, the expectation of x with respect to under its own pdf.
And we said that this thing was integral of f times its pdf, weighted by pdf.
And so now what can we do?
We can divide by z and multiply by z.
Just multiply by 1.
And now we recognize that z f of x is just what we're now calling f of z, where this is x over z f of z dx.
So all we did was multiply and divide by z, and then say that we're going to group the z times the pdf's of x together and [INAUDIBLE] f of z.
And we can do that because [?
opposite ?]
constraint is that same as integrating to 1.
So what is this guy here?
This guy here is the expectation of something else with respect to the pdf of z, the density of z.
What is it the expectation of?
x over z?
So this is the expectation of x over z, but now taken with respect to the pdf fd.
All right.
So does this kind of feel like we're just moving chairs around?
Yeah.
That's why I said they sort of seem a little bit like tricks.
Is anybody uncomfortable with anything that we've done?
It's OK?
We've put a 1 over d in here, and we've changed the pdf, the weighting and the integral to accommodate to make it equal to same.
Yeah, Kevin?
[INAUDIBLE]
The constraint we have here is that this is the expectation of [?
d1 ?]
of f of x is 1.
Well, this is going to be what leads us to find the pdf's.
Yeah
[INAUDIBLE]
So why do we put [INAUDIBLE] underneath and then bring it about?
So let's look at what we have.
We have now two different ways to compute the mean of x.
That's what we're after.
We have [AUDIO OUT] what we normally think about, which is thinking about this mean as the expectation of x under the pdf.
And now we have derived this alternate expression, which is the expectation of x over z where the expectation is taken now with the pdf f of z, the z, and the weighting.
So this is what we've been talking about.
How do we estimate the mean this way?
Well, we sample x by drawing samples from the distribution f sub x.
We define the pdf of x input.
We draw samples from it.
We estimate the mean to be the sample mean.
And the variance of the corresponding estimator to the estimator variance is what?
What was on the numerator?
Not mu x
[INAUDIBLE]
Yeah, it's [INAUDIBLE] squared.
So we'll call it the variance of x.
And again, I'm going to put the subjects here just to denote.
So that's what we saw before.
Actually, we sampled inputs and we propagated them through to the outputs, but basically, we're just sampling from this density, and given the sample mean and our estimator and the estimator of variance with the [INAUDIBLE].
So what we've done here is defined a parallel pattern analogous [?
path.
?]
So what we do here?
We're going to sample x over z, and we're going to sample it under now the distribution or the density xz.
And again, we take the samples, we take the sample mean.
That would be our estimate.
And the variance of that estimator, the estimator variance, is now going to be what?
Variance of x over z under this divided by [?
N.
?]
And now maybe this is where you see why these tricks work because putting the d in the denominator and bringing it out here didn't change what we're estimating.
These two expressions are equal.
I like to think about it is that because of the way that variance behaves non-linearly, pulling the z under and putting it out here in front does not mean that these two things are going to be equal.
And maybe you see there.
You could drive me crazy when I was learning statistics is that things like means would behave in kind of this linear way that made thins, but the variance never seemed to be doing the sensible thing.
So basically we're going to exploit that.
So again, we can do this manipulation with this other variable, z, and not change the estimate, still be getting after the thing we want.
But if we choose z in the right way, and we'll talk about how to do that, we could make this smaller, which means, again, for a given number of samples, if this is smaller than this, then our estimates are going to be tighter.
Or to achieve a given confidence in the estimator, we could take less samples here than doing it this way.
Yeah.
So We'll talk about sort of the general ideas, and then I'll show you exactly how you might come up with a z for probabilities.
But the general ideas are first of all, that some values of x in the Monte Carlo simulation are going to be more important for estimating the parameter.
The mean is not a very good example, but you can imagine if we're talking about tail probabilities, we want to get more samples around that failure probability.
What goes on over here we don't really necessarily care about because it's like doesn't fail, it doesn't fail, it doesn't fail.
So the idea is that some values of x are going to be more important, and what we want to do when we choose the z here is to somehow emphasize those more important values.
And so really what it's going to come down to is how do we choose z again so that this estimated variance is going to be less than this guy.
And we will talk about that in flow probabilities.
So any questions?
I didn't ask you at the beginning of class, did Professor [?
Nguyen ?]
talk about importance sampling in 1609?
No?
Just as well.
All right.
So let's talk about-- this is 3.2, importance sampling for probability estimation.
And hopefully, maybe it will make this a little bit more concrete for you.
So what did we already see?
We saw that the probability of A, where A is some event that we're interested in is estimated by a Monte Carlo, or can be estimated by a Monte Carlo, with an estimator that we'll call P hat A.
So let's think about the case where, again, we're interested in the probability of failures, so A is the event where, say y is greater than y current.
So y is some output of interest.
It's the temperature in [?
blades, ?]
and we're interested in looking at when that temperature exceeds some critical value.
So this shows up often, and if we are looking at a probability of failure.
So we're going to define an indicator function.
Usually when you're working with probabilities and if you're told to find an indicator function, you guys should have seen those in 1609, yes?
In 6041.
So the indicator function, i, is the function of Ai.
So this is going to be corresponding to the i-th sample in our Monte Carlo.
So we're drawing a sample, where running it through our [?
planet ?]
element code, and we're looking to see whether event A happens.
And indicator function says if event A happens, then takes a value of 1, if not, take a value of 0.
So this is going to be if the corresponding sample of y, the yi exceeds y current, and if not, then we can set indicator function to be 0.
So indicator function is set to 0.
1 result.
Failed.
Didn't fail.
A happened.
A didn't happen
[INAUDIBLE]
i to the range.
i is either 0 or 1
[INAUDIBLE]
Oh, little i.
Sorry.
Yeah, that's right.
Little i from 1 up to n. Sorry.
So if we just think about these things in this way, then the reason we do this is because then our estimator P hat of A-- what was he estimator for the probability?
What was it?
Yep.
So the number of samples where A occurred just divided by the [INAUDIBLE] number.
So what does that look like?
What does that look like in terms of the indicator function?
I'll put the denominator in there.
What's the numerator?
Yeah.
Just the sum of the iA's.
So it's 1 over N. The sum equals 1 to capital N of iA's.
In other words, it's the sample mean of the indicator function.
And so we can think of the estimator this way, and we've already seen this.
We saw that the expected value of this estimator for the probability was equal to what?
Probability of A, unbiased.
This would be a big piece.
And do you remember the expressions to the variance of this?
Does anyone remember it?
Here, it's P of A [?
deserves ?]
1 minus P of A.
And what's always in the denominator?
N. Right.
Standard error of the estimator is 1/3 of that thing.
All right.
So that's what we saw before.
So now what we're going to do is think about how importance sampling can help us come up with an estimator, for P of A that's got smaller variance.
Do you remember when we did that analysis?
We also did an analysis where if P is really small and you want to have a tolerance that's relative to the size of P, you end up needing billions, millions of samples.
So we're going to see whether importance sampling or see how importance sampling can help us.
Again, I'm going to erase this, but I find it helpful to think about these two paths.
We could sample, take lots of samples and we're talking about the mean here, use our regular estimator.
Or we could think about introducing another random variable with another pdf and changing what we sample from and then weighting the integral to account for the fact that we sampled from a different pdf and coming out with an estimate of the same thing but changing the variance.
So let's see how that would work out.
So we'll go straight to a pdf.
So we'll introduce another pdf, and it's going to be called w. And it's a function of x, so it's defined on the same support as x, whatever x is.
What are we doing?
Yeah, we have [INAUDIBLE]
[INAUDIBLE]
Ah, fourth
[?
Don ?]
Ulrich just wanted to say hello
Hi, Don
How are you doing?
You've got chalk all over your face
I have.
You guys know who this is, right?
[?
Don ?]
Ulrich, Satellite astronaut
I'm one of the lucky guys who took a fall in space
They're excited in learning all about computational methods.
And today we're talking about estimating low failure probabilities, so things like 10 to the minus 9, things that can go wrong.
Do you know anything about that?
That's 10 to the minus 9.
Well, you don't need to worry about that
I'll be at the dinner tonight.
So I'll pick up a beer.
Thanks.
Where is the talk?
Marlar Lounge.
Second floor.
4:00
Or you could come to office hours
And make sure you learn this stuff because it's important
I don't if you guys know, but I made it to the finals at the astronaut selection last year, but didn't get picked so, Don was, yeah.
All right.
So we just introduced a bit-- I'd much rather teach that stuff.
It's more exciting than going into space, right?
[INAUDIBLE]
So we've introduced to pdf w of x, and I want you to think is this as being like an alternative pdf for x.
And x is just a generic at this point.
This thing is called the biasing entity.
And you can see what we're going to do is we're going to choose w of x so that this event, A, occurs more frequently.
So the idea is we don't want to have to wait around for a billion samples for the one event to occur.
We're going to choose this alternate pdf w, in such a way that this event, A, occurs more frequently.
So then what is our estimate to the probability look like?
Probability of A is maybe you can see it directly from this expression.
We could write it as being the expectation of the indicator function.
And it's the expectation taken under f of x.
So it's the expectation of the indicator function with expectation as with respect to f of x.
Maybe up here I should define that then f of x is but the pdf f of x.
And what is this?
This is nothing but the integral of the indicator function weighted by the pdf and then [?
graded ?]
up for the support of x.
So we're going to do the same trick that we did before.
We're going to multiply and divide by this w. So we've got our f of x.
We're going divide by w. We're going to multiply by w. And maybe now you can see that what we have now, we have an expectation of [?
sine.
?]
We can think about lumping this thing together now taken with respect to the pdf w. So this is just the integral of something times the pdf integrated over the support.
So we have to write it up here.
So the last line in that development-- the left-hand side is P of A, which we haven't discretized with Monte Carlo yet.
We're still doing things exactly-- is expectation with respect or under this pdf w of-- now, the indicator function with this waiting, f of x over w of x.
So what does this mean?
This means we could draw samples from w instead of from x.
So draw samples from the pdf w. But if you do that, in order to get the right estimate for the probability, you have to weight the samples by the f over w to make up for the fact that you drew from a different distribution.
Does that make sense?
So weight by f of x over w of x, I'd say to counter the sort of my informal way of thinking about it.
And if you do that, then the expectation works out so that you are still getting the probability you're after.
So then the last step is just to write down what our Monte Carlo estimator would be-- so then our Monte Carlo importance sampling estimator for this probability of A is what?
So let's call it P hat.
We'll put an IS down here to indicate that's the importance sampling estimator.
So help me write it.
What does it look like?
Take this risk part.
That's the easy part.
One over N-- is it Libby or Casey?
Oh, it's [INAUDIBLE].
All right.
Now, one of you guys have to do the harder part.
1 over N what?
There is a regular Monte Carlo estimator.
Summation from I equals 1 to N of what?
iA times f of xi over w of xi.
And it doesn't show up in the formula, but it's important to know that this is all when the xi's are drawn from the pdf w. That's that.
We never noted that before because we only ever had one pdf floating around, but now that we've got two, so here's the estimator.
If we drew the samples from f of x, which is the regular way of doing it, that would be the estimator, but if we draw the samples from the pdf of w, then this is our estimator, and basically we just have to re weigh it.
Is that clear?
So can you see-- yeah
[INAUDIBLE]
Here?
Yeah.
We divided by w, and we multiplied it by w
[INAUDIBLE]
Well, yeah.
w is a pdf.
Maybe I could have written it like if w might have been-- yeah.
Sorry.
w is a pdf.
Yeah
[INAUDIBLE]
This one is strange, actually.
We approached it here by starting off by saying w is a pdf.
So w itself is a pdf.
When I introduce a general importance sampling, we started off with a random variable and said that's why we put that constraint on so that we could get to a pdf.
But here we're just say let's introduce a secondary pdf or an accelerated pdf.
So w is a pdf, which means it has to satisfy the property of a pdf.
Greg, did you have some--
[INAUDIBLE]
This guy?
[INAUDIBLE]
That's right
[INAUDIBLE]
Yeah.
So this would be given to you.
So one thing that's just a little bit confusing and maybe I haven't been entirely, so I'm going to put it over here, is that when we think about Monte Carlo, I'm going to use this completely different variable so that I don't mix up with w's and f's.
The model has say, an input that might be d and an output that might be q.
And we talk about drawing from some distribution of d. Say d is normal propagating it through and getting the corresponding whatever q looks like.
So here when I talk about estimating the probability of A by drawing from x-- x is really q.
Because this is the pdf that defines if A is out in here, we want to draw from this distribution.
I mean, if I gave you just this distribution, q, and I asked you to estimate a probability, you would randomly sample from it and then you would count the number on the tail.
So there is kind of an extra step, which is that we're really drawing from the d and pushing it through the model to characterize q.
Is that clear enough?
The f point, is that all the samples, or is that just the [?
sales ?]
proportion?
For 10 points?
Oh, the N?
That's going to be all the samples.
Yeah
[INAUDIBLE]
Yeah.
So let me try to write a summary and draw some pictures to help you, because I don't want to mix up two things.
But what we would we do here is we would sample randomly, and more samples are going to end up in here than in here.
And every time we sample here, we generate a corresponding sample of this.
So when you think about the importance sampling, just don't really think about this part of it.
Think about it as we're sampling from this.
But it turns out we're sampling from this by something from this and going forward.
But we're still sampling from this.
So when we do that and we put samples, most of them are going to end up over here, and then one in a billion times, we get one out here on average.
Yeah.
So what we're saying is we could define a different distribution.
We're jumping here a little bit, but let's just conceptually say that distribution looks like this.
And instead of sampling from this guy-- which by the way we do by sampling here and pushing forward, but that's not really relevant-- this sample from this guy so that now when I sample from here, instead of 1 in a billion, maybe 1 in 10 or maybe half of them actually fall in the regions I mentioned for them.
Yep.
So now I'm going to have N samples from this guy.
But now when I compute the estimate of the probability of A, I can't just take the fraction that are here divided by the total numbers.
Clearly that would be the wrong estimate.
But if I take each sample that [?
folds ?]
in here, the 1's, and multiply them by the ratio of this guy to this guy, that's going to recalibrate that to be the right estimate.
And you can kind of see it graphically, because what happens is you get a sample here.
The f of x is basically 0, fairly small.
This guy is big.
That ratio is going to be really small.
So even though it's the 1, it gets weighted by a 10 to the minus 9 or whatever it is, or 10 to the minus 10.
So you can see how it works, and it all works out.
Draw from this guy, get more sample than here, but then each sample doesn't get a 1.
It gets a 1 times the ratio of this to this.
And by doing that, we'll be weighting, [INAUDIBLE] actually estimating the right probability.
But then the next question, that's kind of the next thing we're going to talk about is, how do you figure out what a good biasing distribution is?
How do you come up with the distribution?
And it would be easier if we didn't have this part of the problem, but because we are generating the samples from the inputs and pushing them through the model, we don't necessarily know how to bias the distribution here so that we get a good biasing distribution here.
And that's like saying do you know what combination of inputs makes your aircraft much vulnerable to failure.
And sometimes you do, and sometimes you don't.
So again, that's kind of the separate part.
Part of it is a little bit different to describe.
Yeah
So you could take like a Gaussian variance with a [?
unit ?]
and plot that over there?
Yeah.
You could
Would that be like [?
an ?]
example?
Yeah.
So really simple things that people do is that they, and we'll look, they scale this thing to shift more mass.
Well, what I drew here is a little bit extreme.
They scale it to shift more mass, and they basically make this thing have better tails.
And sometimes they actually just shift it.
We'll take whatever the distribution is and just shift it and scale it.
And then there are more sophisticated things you can do, but, in fact, this is somewhat of an open question is how do you, [INAUDIBLE] define a good biasing distribution to make your sampling really efficient?
I would think that it would [INAUDIBLE]
That's right.
And so usually in the cases where it's easy to come up with a biasing distribution, you kind of know what conditions including failure anywhere.
And so it's sort of obvious way to look.
The really difficult problems are when there's tens, hundreds, thousands of them that are through input here.
We don't even know which combinations of input to the ones that make you most vulnerable to failure.
And the only way to find out would be to sample them all, which we already said we don't want to do.
So aircraft design is a good one, also people who simulate earthquakes and all these kinds of things, I mean, where events of different weather stuff.
It's not even clear sometimes that you can simulate and get good estimates for some of these events
[INAUDIBLE]
Yeah.
In some places, it's really hard for us to characterize what q is out here.
The financial markets are another one.
I don't know if you guys realize The Black Swan book, so it's all about [?
fact ?]
tails and suit and models and don't account for things that are like way out here, that are really unlikely, but when they happen, they have a really big impact.
So if we knew what was going on here, we would already know the probabilities.
And so the trick is that we don't know what's going on here, but we want to learn about what's going on here by finding better ways to sample.
But to find better ways to sample, we need to know what we're looking for.
So then there are things, I mean, this still always happens, and this is what keeps us busy in research is that's where you're using activity.
You learn a little bit and sample.
So let's just quickly summarize the steps for importance sampling.
So we're going to define a w of x and however we come up, we're going to draw samples of x from that pdf.
So I'll just note here that this is a pdf.
If you prefer to call it f sub w, that's fine.
And again, the idea is we want to get more samples in the region of interest.
And then we're going to estimate this probability, but we're going to do it using this thing here that we define, the P hat, IS, where we have to do the weighting to account for the fact that we drew from the wrong distribution.
So importance weights if x divided by the w, that's the sample point.
And one thing I should say actually is if you don't choose a good w, you can actually make it worse.
You could make your Monte Carlo convergence worse.
So actually, I think there's one final result to write down is what are the properties of this P hat IS?
They are that the expected value of our importance sampling estimator for the probability of A.
What do you think it is?
Hopefully, it's P of A.
It is, indeed, P of A.
And that's actually, I mean, we didn't do anything over here except move things around.
We had equality all the way through.
So that's key.
We dumbed this down with the expected value.
In other words, it's unbiased.
But then the key is that-- you could put a w in here to be clear-- that the variance of the estimator, which again controls basically how good it is for a given number of samples, has got this more complicated expression.
So the 1 over N is still out there.
And then there is an expectation of i of A, f of x over w of x, and then there is a minus t of A squared.
I think I got all the pieces.
And that actually goes back to what I was just saying.
It's not actually obvious whether this is smaller or bigger than what we had before with the regular Monte Carlo estimator, but of course, it all comes down to the choice of w. I mean, the idea of choose a w to make this small so that you get away with fewer sampling.
That's kind of neat trick, no?
Multiplying and dividing by things, no?
[INAUDIBLE]
It's what?
[INAUDIBLE]
Yeah.
It used to always drive me crazy when I was an undergrad
[INAUDIBLE]
So next just briefly how to pick, and I'm not going to give, sorry, [?
Trey, ?]
I'm not going to give you any definitive answers on this, but how to pick the biasing distributions.
So what do you do?
So we'll just look at two yeah?
[INAUDIBLE]
Oh, here?
Yeah.
Just like all of the variances of the estimators have like sigma y or P of A or whatever it is
[INAUDIBLE]
Yeah.
Remember like the original.
Even the variance to the mean estimate is sigma y, sigma squared over N, and you don't know sigma.
So you can plug in the estimates, but then you're introducing additional error.
Yeah.
To actually compute them, you have to use an estimate for P of A. Yeah
[INAUDIBLE]
Yep
[INAUDIBLE]
Yeah.
So this is the theoretical expression for the thing.
How you actually compute this is a whole other question, but we've seen that with every single one of the variances of our estimator.
We had sigma squared over N. We didn't know what this was.
On the original one, we had t1 minus t over N. We didn't know what this was.
So all of them have got the theoretical result and how you compute them.
And you could compute the estimates.
You can do bootstrapping, and in this case you would be able to estimate everything.
Yes.
So one simple approach is just to scale.
So we're talking about picking w. And thank you.
These are homeworks 7.
Thank you.
And it's scaled in such a way that we shift probability into the region of interest, shift probability into the event regions.
Again, the idea being that we want to get more samples.
So how would that work?
In that case, w of x could be some 1 over a times the original pdf.
But now the function of x over a, where a is some constant that's greater than 1.
So I have to draw these for them to make sense to me.
So it's saying pick w to be the scaled version of f of x where we scale the argument and then also scale the result.
So if I can just sort of sketch emotionally what that might look like, I know it helps me to think about what it's doing.
So let's say x and this is the f of x, the original pdf.
So maybe it's Gaussian.
And just to make it easier to think about, let's just set it at 0.
It doesn't have to be.
And it's high, which is going to be something.
We'll call it q.
Then what is w?
Does anyone want to have a go at [?
throwing ?]
w?
So the height is q over a. OK, good.
Am I going to choose a greater than 1?
So it's going to be lower
[INAUDIBLE]
Good.
You're much better at visualizing this than I am.
I had to mentally do a few points because I'm not very good.
It's not a very good fit Gaussian.
All right.
So I had to actually sort of mentally thinking it through.
I'm like at this point, we're at 1 and say a were 2, then this guy at 1 is going to get the mass from 1/2.
Then it's going to scale it down by two.
But because this thing is falling off quickly, it's still going to end up being set or relative to what was in the middle.
So it's exactly a scaling that's going to shift probability mass out.
So this is what I was saying when I meant [?
heads ?]
or tails.
Any problems you see with this if we're trying to estimate a probability of failure, say the probability that x is greater than some critical value?
What's that?
[INAUDIBLE]
Yeah, it might not still be.
Anyway it's definitely going to generate more samples over here, but it's also going to generate more samples over here.
So we've made this tail fatter, but we've also made this tail fatter.
So
[INAUDIBLE]?
Can you do one if f of x is uniform?
I think about what w would be.
But basically w can be anything you want
[INAUDIBLE]
Yeah, that's just one way to do it.
But you've got to make sure that the support of w is it at least as wide [INAUDIBLE] actually going to be dividing by 0.
So if this guy was uniform, I mean--
[INAUDIBLE]
That's right.
But you couldn't have this be uniform and then say, I'm going to take my w to be this guy.
Well, what's generating from here would you-- you would be OK because you never generate points here.
Yeah
[INAUDIBLE]
You could but then they would just get multiplied by 0.
Then you'd be wasting [?
holds.
?]
Yes, I guess there's no reason why you couldn't do this.
Yeah.
So that's one simple thing to do is just to move mass out, maybe [?
theta ?]
tails.
Another simple thing you could do is translation, which just means that w of x would be f of x minus c, where maybe c is greater than 0 if we're thinking about a right tail.
And again, maybe this wouldn't make sense if you had a finite support like a uniform distribution because then you would be putting w, which you didn't care about.
But what would that look like notionally?
If that's x and that's f of x, we get a first term normal [INAUDIBLE] 0, then the w of x is just going to be shifted over by an amount, c. So again, all it's doing is putting more probability mass in this case in the right tail that we care about.
But thinking of biasing distribution, the simple problems, simple things probably work for really difficult problems with lots of input in very rare events, differently, still an open question.
Any other questions about importance sampling.
We sometimes talk about distributing analysis?
Does it all makes sense that's on this screen?
Yes
[INAUDIBLE]
Actually control variance are my favorites.
I've mentioned some of the other variance reduction methods.
Control variance is another way to do variance reduction that I think is also a kind of neat.
So if you're interested in knowing other variance reduction methods, I could give you some stuff to read.
So let's now just in the last 15 minutes talk a little bit about sensitivity analysis in the context of Monte Carlos.
So the question is now how do we use our Monte Carlo results to understand which uncertain inputs are contributing the most to output variability.
So I want you to put this picture back in your mind, this guy here, which is that we've got uncertain inputs, and maybe there's a lot of them-- d1, d2, d3, however many.
And maybe there's one output of interest, or maybe there is lots.
But we've run these Monte Carlos, we put pdf's in all these guys.
We draw samples.
We run through.
We generate some kind of output histogram on q that looks like however it looks.
But now we want to ask the question, well, which one of these is really contributing to the variability, or which combinations of these are contributing to the variability?
Which combinations are causing this big fat tail out here, which corresponds to [INAUDIBLE] that don't meet our design criteria?
So this is really now kind of the opposite question.
You've done a forward propagation of uncertainty, and you've got the uncertainty in q, and now you want to figure out where did it come from.
So forward propagation of uncertainty, sensitivity analysis is kind of like the reverse question.
And it's important for two reasons, one is that it helps us understand where we should focus our uncertainty reduction efforts.
But I think in many ways when you look say like an aircraft design and development program at a place like Boeing or wherever, so much of the process is about reducing uncertainty.
It's about running experiments or tests or running models and trying to figure out exactly what the performance is of the thing that you're designing and making decisions as well.
So understanding where this big uncertainty can help you decide what kind of experiments to do, whether it will improve your models, if you're thinking about uncertainty in finished products, it can help guide you one where you should tighten tolerances in the new manufacturing process.
It might tell you that you've got uncertainty because you don't know certain conditions.
So maybe it says you need a sensor in your engine or on board your aircraft or wherever.
Sometimes this is called factor prioritization, figuring out what the priority order of inputs.
Effect is just a term that is often used in the statistics community.
Which one should be priorities for uncertainty reductions.
So it's kind of half a story, but the other half of story is that it's also really important to understand where the uncertainties are not important.
That they could be distributions on d1, but this doesn't really matter because this thing is not sensitive to d1.
And that's important because then you shouldn't waste your time worrying about d1, and you shouldn't waste your time arguing with other groups about whether d1 should be 1 or 1.1 or 1.2.
So it's important to understand where things are actually not important.
And this is sometimes called factor fixing because this is where you can understand where it's not important to consider uncertainty, and you can just pick a deterministic value of an input and go forward.
And this is really important, actually in the policy setting.
As I mentioned before, you can work with FAA.
People argue about all kinds of things when they make policies.
It's nice to be able to show that some of the things that you argue about actually don't matter for the uncertainty in the policy decision.
Less things to argue about.
All right.
So how can we do sensitivity analysis?
There is something called VABO, vary-all-but-one [INAUDIBLE].
Think about doing.
So here are the steps.
First of all run your Monte Caro with all the inputs varying.
Think about d1, d2, d3, up to however many we have of them.
We've got pdf's for all of them.
We sample all of them.
We generate the corresponding-- it's called q on that order over there.
Then let's go and take input case, so it's like the first one, d1, and let's fix it to deterministic value so it's no longer got a pdf, it's no longer able to vary, and rerun the Monte Carlo with all the other ones-- all the other, however many there are d minus 1 inputs varying.
So now we have the results of two Monte Carlo simulations, one with a fixed, one was varying, and one where it wasn't.
And you could compare the statistic to the output.
You could look at the variance from run one, and the variance of run two.
And you could then attribute the difference in the variance to that fixed factor.
And then he would repeat it with 1, 3, 4, 5, 6.
It varies at 6 2, 6 3, 6 4.
So vary all but one.
I mean, maybe it's a useful thing to do, but of course, there's question.
The first question you'd say, well, what value should we fix factor k?
If we fix it to this value and ran the Monte Carlo, would we get a different result than if we fixed it a different value?
And the answer for a complicated system is probably yes.
And then the other question maybe you should be wondering is what about possible interactions.
If I fix input 1 at a value and analyze things, but what if I fixed input 1 and input 2 and there were interactions between them, and I never explored them.
Are there interactions that might be important that I would be missing?
So to address those limitations is something that's called global sensitivity analysis.
So let's get this off.
And I think when you think about global sensitivity analysis, it's useful to have this vary all but month's color in your mind because it's conceptually a little bit the same.
So we're on 4.
And 4a was the vary-all-but-one.
So this is 4b, global sensitivity analysis.
So I'm going to draw a picture here.
I'm going to use different notation, but let me not try to change otherwise, I will end up mixing something up.
So we've got a model.
So then I'll call the inputs x1, x2, down to-- we're going to have [INAUDIBLE], so d random inputs.
Now, let's just think about a single random output, so then just one random output.
So global sensitivity analysis defines [AUDIO OUT] called sensitivity indices.
And there are two different kinds of sensitivity indices.
So the first one is what's called a main effect.
Did I talk about main effects in 1609, 6041?
No.
Maybe in 62x, anybody seen effects?
So if you've seen effects sensitivity index for input i, usually the set of [?
sessions ?]
of u say people would use it with factor here, but let's just call it an input.
So this thing is [?
the limitation ?]
si.
Let's write the formula and talk about what it means.
It's the variance of Y minus the expected value of the variance of Y given xi all divided by the variance, Y.
So the variance of Y minus the expected value of the variance of Y given xi divided by the variance of Y.
Does anyone want to have a go at explaining what this is?
[INAUDIBLE]
Yes.
So if this happened to be 0, then overall sensitivity index would be what?
1.
And it would mean that all of the variance in Y was explained by xi, because when we fix xi the variance [INAUDIBLE].
What about if the variance of this thing were the variance of Y?
Doesn't have any effect.
Yes.
Why is there an expectation here?
So we go with [AUDIO OUT] So we're saying Y give xi
[INAUDIBLE]
Yeah.
If we picked any particular value which to effect xi, as I said given xie to some x star, then it would just be a number.
But because this thing is a random variable, we don't know where to fix it.
So this is addressing this question of we don't know where to fix it.
So at least the way I think of it conceptually is that we're going to fix it at all possible places and then [INAUDIBLE] over there.
And so it's the expected reduction in the variance is relative.
It's normalized.
So it's the expected relative reduction of the variance if we learned everything about the factor xi.
So it's giving us a measure of how much to does xi contribute to the variance in Y, but it's a measure where we're taking expectation over the possible values that xi could take on.
So that's where the word global come from.
[INAUDIBLE] So let me just write it in words.
So expected relative reduction in output variance-- and then output variance is variance of Y-- it's a true value of xi is [?
learned.
?]
And then another thing that's important to see is that this is a measure of the effects of varying xi alone.
And if we wrote this in a slightly different way, we could write it, we can see that would be averaged over variations in other inputs.
We'll see you Monday when we talk about [?
planet ?]
experiments.
It's another way of computing of effects that [?
finds ?]
out more with the second interpretation.
Let me say it again.
Measure, this is what we talked about.
Measure of the effect of varying xi alone averaged over the variations of other inputs.
And that's where the term main effect comes in.
So if we were to change xi, if we were to vary xi, how does it change the output averaged over everything else in the [?
x3, ?]
x4 varying?
Is that clear?
So there are two different ways you can think about that
[INAUDIBLE]
So xi can at most be 1 because the most we can get here is 0 if i is going to be between 0 and 1.
And the idea is that you could compute these then you would rank them.
And if you were looking for where you focus your efforts, which variable you try to control in the manufacturing process, before we go after the one with the biggest sensitivity index that you occupy.
So actually turns out we can also compute higher order interaction indices.
You can also compute effects for the interactions.
And I won't go into details, but instead of now just being si, there could be an sij, which would be two variables, two inputs interacting or three variables interacting or four, all the way up to all of them interacting.
And what the match shows, I mean, we're not going to talk about it, is this idea that you can take the variance of Y, think of it as this pi, and you can decompose this variance of Y.
So let's think about-- it's easier for me to draw fY depends on x1 and x2, so they're just two inputs.
There would be a part of the variance that might be due to x1 acting alone, there might be a part of the variance that's due to x2 acting alone, and there might be a part of the variance of the interaction between x1 and x2.
And if we computed the sensitivity indices, f1, f2, and f12, they would all come up to 1.
Or the variance due to x1, the variance due to x2, and the variance x12 would come out to be the variance of Y.
And you can show there is something called a high dimensional model representation that shows how all of this comes out.
So it's kind of a nice result that you can attribute.
It actually turns out that there's something else also called a total effect sensitivity index that you can also compute that tells you how much does x1 contribute, not just by itself but with all of the interactions as well.
So the total effect sensitivity index, what include x1 and x12, x1.
And the total effect x2 would include x2 and x12.
And then all the other ones, if you had more variables then I would add them all up, and it turns out to be a fairly easy way to compute that as well.
Last questions?
Great
[INAUDIBLE]?
Yeah, that's a good question.
So I actually haven't told you how to compute these things that all.
These are the expressions for them.
It turns out there are a couple different ways.
I think the most common way to compute these things is a method called the [?
Sogl ?]
method.
[?
Sogl, ?]
I think, was a Russian statistician, which involves Monte Carlo simulations.
And you basically end up doing one Monte Carlo simulation and then you do a second Monte Carlo simulation where you free some of the variables and redraw other ones.
And it's done in kind of a clever way so that you get to this.
So then the question of how [?
converge ?]
of the sensitivity indices comes to be a question of how converged are the variance estimates.
And we've talked a lot about mean estimate and how they converge.
It actually turns out that to get variance to converge, you usually have to take more Monte Carlo simulations.
Most of the time when we use these things, we do as many samples as we can afford, and that ends up being the [INAUDIBLE].
So that's a good question.
And it also depends on what you want.
Do you actually care whether you get the sensitivity index to four decimal places, or do you just care about saying number two is the biggest?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. We can go ahead and get started.
So today is going to be the first lecture on an introduction to design optimization.
You guys ready?
Can you talk about the project?
I have office hours right after this, so we can-- So here are three things I'm going to try to cover today.
We'll talk about the basics of the design optimization problem, how you take a design problem and set it up as an optimization problem.
Well talk about unconstrained optimization methods and how to compute the gradients that we will need for those optimization methods.
And then in the next lecture, we'll talk a little more about some of the different methods.
We'll maybe talk about some responsiveness modeling, so surrogate models and see what else we have time for.
So this is intended to be a little bit of just kind of a teaser for optimization methods.
This is something that I use a lot in my research.
So let's just start off with thinking about the basics of writing a design problem as an optimization statement.
So the idea is that a design problem, as I said already, can be written as an optimization problem.
And the reason that we're doing this is because then we'll be able to use numerical optimization methods to explore the design space.
So what we saw in the last lecture were the design of experiments, which were basically sampling methods.
But what we saw is that if you have a lot of design variables, it get's hard to explore the design space very much.
Because you can't [INAUDIBLE] to maybe too many levels.
So one thing we can do is try to write the design problem as an optimization problem.
And in doing that, we would define a few mathematical elements.
So the first are what are called objectives or objective functions sometimes.
And these are what we are trying to achieve, so measures of performance of the design.
We'll talk about some examples in a second.
We're going to have constraints.
And constraints are going to be things that we can't violate, so requirements on the design are cannot be violated.
So these are going to be what we cannot violate.
We will have design variables.
And those are the quantities that we can change.
Those are the things that as a designer we have to make decisions about.
The design variable are what we can change, what we can change and what we can control.
And lastly, we're going to have parameters.
And the word parameters is used often in different contexts.
Sometimes people use the word parameter when they mean design variable.
The way that I'll use it is that a parameter is going to be other quantities that are taking on fixed value.
So they're other quantities that affect the objective and the constraints.
But they're going to take on fixed values.
So they're not going to be varying as we search the design space.
OK. That's the four elements of the optimization problem, objective functions, constraints, design variables, and parameters.
And I'm going to go through each of these.
And we'll define it mathematically and also think about what it might be in terms of physical examples.
So let's start off thinking about design variables.
So again, the design variables, these are the qualities or the attributes of the design that we have control over as a designer, things that we can change.
And we're going to write the design variables in a vector, the design variable vector or the design vector.
We're going to give it the symbol x. I'll put a vector symbol above it just to denote that it's a vector.
And it's going to contain n design variables, n dvs, that form the design space.
So when we talk about the design space, we're usually talking about the space that's defined by these quantities in the design vector x, so n design variables that form the design space.
And so mathematically, the vector x will be x1, x2, down to x10.
OK. And the idea is going to be that as we run the optimization algorithm, it's going to be searching over different values of x.
So this is what's going to be changing.
And what you'll see is that different optimization methods use different rules and different information to figure out how to move around in the design space.
OK.
So what are some examples?
So if we think about aircraft conceptual design, say, what would be examples of physical quantities that might be in the vector x?
What?
Weight.
Weight normally would not be a design variable, because weight is not usually something that you can control directly.
It's something that typically would be computed as a function of other things that would be designed variables.
So weight is going to often show up as an objective
Wingspan
Wingspan.
Yeah.
So wingspan would be one.
What else?
What is it?
Number of rotors?
[INAUDIBLE]
Oh, motors, number of engines?
Motors?
[INAUDIBLE]
OK. Sure, number of motors or engines.
Yup.
What else?
[INAUDIBLE]
Payload size?
Yeah.
Payload weight or payload size or payload number of passengers
Color?
Color?
Who said color?
OK.
I'm going to spell it with a U just to have my objection.
Yeah.
[INAUDIBLE] mark number whatever.
OK.
So all the things that I think you can directly control and make a decision about.
And what you notice is that these quantities can be of different kinds.
So wingspan is going to be a continuous real variable that can take on values between 0 and whatever upper limit you set to it.
Whereas, number of motors is a discrete variable, right?
It can be 1 or 2 or 3, but it can't be 2.5 or 2.53.
And as we talk about the optimization methods, you'll see that particularly discrete or integer variables cause a lot of problems.
It's much harder to optimize a problem that's got something like number of engines or number of motors in it.
OK.
So will be your design vector x.
Next, let's talk about objectives or objective functions.
So the objectives can be a vector as well.
And it can be a vector j.
And I'll put the vector symbol on there to denote when I'm talking about a vector.
And it's going to be j of v system responses or characteristics.
And they're going to be responses or characteristics that we're trying to either minimize or maximize.
OK. All right, so these are going to be measures of performance, costs, schedule, anything that we might want to push either as high as possible or as low as possible in making the decisions about our designs.
So what might be some examples, again?
For the aircraft conceptual design example, what will be objective functions?
Range.
Yup.
Range might be an objective if you wanted to maximize range.
What is it?
Speed.
Yup, speed sometimes.
Fuel consumption, fuel burn, yup.
So weight often shows up as an objective.
Max take off weight is often used as a target for costs and fuel burn and everything kind of rolled up.
Cost might be another one, operating costs, or it might be entire lifecycle cost.
Could be environmental impact, could be noise or different kinds of things.
And so I most-- this is right here.
So we would write the vector j as being j1, j2, jz if we had z objectives.
Turns out that most optimization algorithms work with a single objective, with a single scalar objective.
There are some ways to do multi-objective optimization if you have more than one objective and you want to look for designs that are at the same time trying to maximize range and minimize costs, say.
There are ways to do it.
But most optimization methods work with a single scalar objective.
And if that's the case, then the way that you would normally proceed would be by weighting the different objectives.
So you would have weight 1 times j1 plus weight 2 times j2 and so on.
So you would roll all of the different objectives up, weight them, add them together, and get a single objective function that would then use in your optimization algorithm.
And these weights would represent both some kind of a normalizing factor, because range is going to be measured in meters.
And that's going to be in the order of thousands.
Whereas, speed would be measured maybe in meters per second or might be in the order of hundreds.
Costs could be in dollars.
So these things have different units and different scales.
But you could also use these weights to talk about your design of preference.
I care more about cost than I do about noise.
Or I care more about fuel burn than I do about noise.
And you could use the weights in that way.
OK.
So if we have multiple objective, either we have to roll them up into a scalar objective and you think about what these weights would be or there are some methods to do what's called multi-objective optimization.
I'll add one more to the left.
This is Professor [INAUDIBLE] preferred-- does he talk about [INAUDIBLE]?
I think it's payload fuel efficiency-- no, payload fuel energy intensity.
It's a measure of fuel burned per person traveled 1 nautical mile or 1kilometer.
So it's a measure of how far you fly, how many people you transport, how much fuel you burn, measure of efficiency.
OK.
So design variables and objectives, design variables are what we can change.
The objectives are going to be the measures of how good the design is that give us some direction for change.
Then we're also going to have constraints.
Again, we think constraints are what we cannot violate.
So these are going to act as the boundaries of the design space.
So here the design space is defined by those design variables.
So a nice visual picture that I like to have in mind is that the design space is like a landscape.
So think of your landscape.
It's got hills, mountains.
It's got valleys.
It's got flat regions.
The dimensions of the landscape are the design variables.
So you can think about two design variable, x1 and x2.
And then the height of the landscape is the objective function.
Right?
So the mountains, the hills, that's places where the objective is high.
And the places where you have valleys, that's where the objective is low.
And so if you try to minimize, say, cost, good designs you'd be looking for them in the valley.
And the regions where the space is flat, those are regions where changing the design doesn't really change the cost or doesn't change the weight or whatever the objective is.
So then what are the constraints?
The constraints are going to come in and tell you where in the landscape are you allowed to be.
So there's going to be boundaries that say you can't go on the side of the boundary.
You have to stay within this region.
It's going to define regions where designs are allowable, where they satisfy different kinds of constraints.
And it's going to tell you regions where your designs are not allowable.
And what kind of constraints can you get?
We're going to have inequality constraints, which we'll write as gj is less than or equal to 0, for j for 1 to m1.
So we'll have m1 inequality constraint.
And the constraint is written that g, gj for the j constraint has to be less than or equal to 0.
So this might be a constraint that's something like [INAUDIBLE] speed.
You need a constraint on [INAUDIBLE] speed.
And you don't require the speed to be anything in particular.
But you've got to make sure that you stay above the constraint.
You would bring everything over to the left-hand side and make the constraint less than equal to 0.
Or if you had a constraint on cost, you would say that the cost minus the maximum cost you can incur had to be less than equal 0.
So inequality constraints often come when there's some kind of limitation on the resources that you have or there's some kind of physical limitation in terms of the physics.
And then we might also have equality constraints.
And we're going to give those the symbol h. So we'll write hk equal to 0.
And in general, we'll have m2 equality constraint.
So what's an example of an equality constraint in an aircraft design problem?
Lift equals weight, yeah.
So that's kind of the classic one that shows up in the aircraft design problem, lift equals weight or left minus weight equals 0.
If you had sophisticated-- like, if you have structural models in there and you're using a finite element analysis, then the governing equation of the finite element analysis would also show up here, the laws of whatever the PDE that you discretized.
Or if there's a CFD model or a CFD equation, it's the conservation of mass, momentum, energy, and so on.
So often, we can have lots and lots of equality constraints.
So you have inequality constraints and we have equality constraints.
And then we can also have design variable bounds.
And even though these things are kind of like inequality constraints, they're usually separated out, because they're quite often treated differently.
And the design variable bounds for design variable xi-- remember, our design vector was x1 x1 down to xn.
We might bound design variable xi by a lower bound and by an upper bound, right?
So for example, wingspan was one of the design variables.
I mean, the lower bound could be 0.
But it might even be bigger than 0.
And the upper bound might be the limit that we know exists, because [INAUDIBLE] in the [INAUDIBLE] constraints, the 80 meter box for a big aircraft.
Or it may be that we know we want designs that are going to be between 40 and 50, and so we're going to put those bounds on so we don't waste time searching for designs elsewhere.
So we may have lower and upper bounds for some or all of our design variables.
OK.
So now, we have all the pieces.
I'm going to call the parameters p. What are things that could be parameters?
Material properties could be parameters.
Sometimes speed is actually a parameter.
Often, aircraft are designed with speed being fixed and not something that you can vary.
So often mach number might show up as a parameter-- anything that's going to go in the problem that's going to affect the constraints or affect the objectives, but is not allowed to vary.
So putting all that together, what does the optimization problem look like?
Minimize over x from j of x.
And x here is a vector.
But we're going to, again, consider just a scalar objective function.
So if we have lots of objectives, we're going to roll them all up together subject to the constraints, the hk of x.
And actually, let me write explicitly this is a function of x and p. And it's quite common to put the semi-colon here to show that j, the objective, depends on the parameters.
But the parameters are kind of different to the design variables.
But the things we can change, these are just things that affect [INAUDIBLE], but they do affect j.
So the same thing for the [INAUDIBLE] vector as well.
For the constraints, hk is equal to 0.
And that's k equal 1, 2.
I think we had m2 of those.
So gj-- again there, a function of x and p-- are less than or equal to 0.
And j goes from 1 to m1.
And then the xi lie between the lower bound-- how did I write [INAUDIBLE] that way-- and the upper bound for the design variables where some of these guys might be minus infinity or infinity if we don't actually have bounds.
OK. Yup
[INAUDIBLE]
Yeah.
Yeah.
So again, whatever the problem, however the problem shows up, we can always write it in this form.
So exactly, if we actually wanted to maximize something, if we wanted to maximize range, we would just minimize negative range.
So any objective that you want to maximize, you're going to multiply with a negative sign and try to minimize it.
How about the inequality constraints gj less than equal 0?
You can always just rearrange it and turn it around, right?
Multiply by minus 1 on this one.
And flip the sign if it's a greater than or equal to sign.
So the fact that we wrote this as a minimization and with these guys being less than equal 0, let's go to the standard form that for the nonlinear optimization problem.
It's sometimes called NLP for Nonlinear Program.
But you can always get your problem into this form.
So I forgot to scroll through this thing.
OK.
So let's see, any questions?
It's clear?
Mathematical statement.
This is a design problem written as an optimization problem, finding the xs that minimize my objective while satisfying the constraints.
And that will be a good design.
All right.
So what we're going to talk about now are how we might go about solving this problem.
So this is number two.
And I'm going to focus mostly on unconstrained optimization problems.
I'll talk a little bit about how to handle the constraints maybe in the next lecture-- so unconstrained optimization.
So many or most-- so let's say many optimization algorithms, in fact, almost all optimization algorithms are iterative-- [INAUDIBLE]-- which means that we're going to iterate to try to solve this problem.
So what do I mean by that?
We're going to have a guess for x.
We're going to update that guess in some way.
And we're going to keep iterating, keep updating the guess for x, get a new guess for x, new guess for x, new guess for x until we think we've solved this problem.
And the way that we can write the kinds of optimizations algorithms we're going to talk about is that xq is equal to x q minus 1 plus alpha q times xq.
And this is going to be true for q equal 1, 2, up to however many iterations we have.
OK.
So what are all these symbols?
So first of all, q is going to be the iteration number.
So the superscript q here denotes our guess for whatever quantity or our value for whatever quantity on iteration q.
And specifically, xq is going to be our guess for x on iteration q. OK.
So it's our current guess for the design variables.
And what you can see is that the guess on iteration q is going to be given by the guess of the design variables on t minus 1, so what we had before, plus this update turn.
And this update turn is about a scalar alpha q.
And it's got a vector Sq.
So let me explain it, and then we can write it down.
This thing here is called the search direction.
And this thing here is going to be out scalar [INAUDIBLE].
So again, what I want you to do is I want you to picture the design space as being like a landscape.
Right?
So that's the xs.
And again, the height of the landscape is going to be a measure of j.
So I'm standing in the landscape.
And I'm standing at point x q minus 1.
We can [INAUDIBLE] q equal 1.
So I'm standing at a point x0.
And I want to figure out a way to go.
I'm going to take a step.
I'm going to move in the landscape to get my next one, to get my x1.
And the way I move is that first of all, I'm to pick a direction to move in.
So if q is my search direction, so first of all, I'm going to look around.
I'm going to pick a direction to move in.
And maybe you have some intuition that if I'm trying to minimize something, I'm going to try to pick a downhill direction, right?
So I'm looking for a bottom of a valley.
I'm standing at a point in the landscape I might want to walk downhill.
So I'm going to pick a direction Sq.
And then once I've picked my direction, I want to figure out how far to walk in that direction.
And that's the alpha q.
So the Sq is the direction.
And then alpha q is going to be the size of step.
So this is scalar in that direction.
So standing in the landscape, pick a direction.
Take a step [INAUDIBLE] alpha q.
So that gets me to the new point in the landscape.
Then I'm going to stop, and I'm going to look around.
I'm going to find a new direction.
And I'm going to do the same thing, figure out how far to walk.
We're just going to keep walking around the landscape picking directions and figuring out how far to walk in those directions.
OK. And of course, then the trick comes.
How do you this?
And that's what the different optimization algorithms do, different ways of picking search directions and steps.
OK.
So Sq is, again, our vector search direction.
So it's got the same dimension as x.
Somebody is smoking out the window.
Do you guys smell it?
[INAUDIBLE] [AUDIO OUT] is [AUDIO OUT] [INAUDIBLE] q and alpha q is a strong consideration, too.
And then the other thing we're also going to need is we're going to need an x0, which is going to be our initial guess, all right?
So we're always going to have to initialize from some x0, so that we can start this whole process of walking through the design space.
OK.
It's very [INAUDIBLE] wavy.
Actually, that's a [AUDIO OUT] satisfy this problem.
[AUDIO OUT] You can [AUDIO OUT] make guarantees about whether the final x that comes out after you iterate satisfies the assumptions [AUDIO OUT].
So there are things that [AUDIO OUT] [INAUDIBLE] has to be true.
So [AUDIO OUT] I just want to minimize j of x.
Forget about [INAUDIBLE].
It has to be true.
Yeah, what's that mathematical condition?
What has to be there?
Yeah, [INAUDIBLE].
[AUDIO OUT] [INAUDIBLE] depending on [AUDIO OUT], we can guarantees about [AUDIO OUT] additional [INAUDIBLE] that you need [INAUDIBLE] derivative has to be positive for a minimum.
And we'll talk about what that means, because this is a vector [INAUDIBLE].
So I call these S's.
I mean, they're iterates.
They're on the way.
And again, depending on the what algorithm we use, we will get to at least a solution of that problem.
I want to-- that's strange-- I want to show you this idea.
It's a little MATLAB demo.
Because I think it will help you think about some of the issues.
And then we'll start talking about how we actually can-- well, we'll talk about some of the mathematical quantities that we're going to need to be able to compute these alphas and S's.
And then we'll talk about the different methods and how they do those.
So [INAUDIBLE] still simple [INAUDIBLE] script here that does optimization.
It does optimization on the function.
This is the peak function in MATLAB.
So this is the landscape.
There are two design variables.
Here, they're called x and y.
So this is x1 and x2.
And this is j on the vertical axis.
So this is the objective.
And we're going to try to maximize just because it's easier to see what's going on with maximizing.
So we've asked [INAUDIBLE] constraint to [INAUDIBLE] an unconstrained problem, we've asked the optimizer to find the design vector x, which in this case is xy, two components, that maximizes the objective j where the height and the color are the measure of j.
And this little asterisk sitting here is the initial guess that I gave.
When I called it, I gave it an initial guess of-- what did I give it-- minus 1, 1.
OK.
So this is x0.
And I'm going to start it running in a second.
And what you're going to see is a sequence of little asterisk.
And each asterisk is a new guess or a new iterate.
So this is x0.
It's going to compute x1, x2, x3, through the script all the way until, hopefully, it's gets up to the top here.
So we'll set it going.
And you can see it's doing what you would want it to do, which is it was asked to maximize.
So it's looking around.
It's picking search directions.
And then it's deciding how far to step.
On each iteration, it's moving in a particular direction and taking a step.
And you can see that it gets to the top of the hole.
So this is an optimization method that's called Nelder-Mead simplex.
And I'll show you just a little bit about it later on.
That's fminsearch in MATLAB.
And we'll talk in the next lecture about how to call the MATLAB optimization functions, because they're really powerful.
And they can really help you if you're doing design problems.
But that one's fminsearch.
Do you want to ask a question?
Yup
[INAUDIBLE]
Yeah.
We'll do that in one second.
Yeah.
I just want to show you this-- oops-- still running.
Hang on.
So it's still going.
So fminsearches, you'll see it-- where is [INAUDIBLE]?
It's interesting that it's still running.
So fminsearch, this is why you should always put maximum iterations in your code.
Because I'm pretty sure it's at the top of the hill, but the [INAUDIBLE] probably fit two [INAUDIBLE].
Yup
[INAUDIBLE]
No.
It changes each iteration.
Yeah.
And we'll talk about how to compute it.
So I think I'm going to kill that.
So fminsearch is actually-- it's called a Nelder-Mead simplex.
It's kind of a pattern search.
So it's sampling the design space and doing a pattern search.
And we'll talk about that.
But I also just want to-- oops, not that one.
How does it measure-- well, we'll talk about that as well.
Yup, 0
[INAUDIBLE]
Nelder-Mead simplex is the method that's implemented in fminsearch.
So there's another one-- oops, now I'm [INAUDIBLE] the wrong.
Actually, [INAUDIBLE].
There's another one called fminunc, which stands for minimization unconstrained.
And this is one that uses gradient information.
And again, we're going to talk about how that works.
But let me just run the same, so you can see.
[INAUDIBLE] after 49 iterations.
I don't know why it was continuing to go.
But let's run from the same initial condition.
OK.
So it takes a little bit longer per iteration.
But did you see how it basically jumped all way to top of hill in the second iteration?
And you'll see it when we talk about algorithms.
But that's because we actually computed the gradient here rather than just sampling locally, which is what now the Mead simplex does with triangles and flipping them on and taking a long time.
This one used a gradient.
And you can see just how quickly we got to the top of the hill, 6 iteration, 24 function evaluations.
And again, we'll more talk about the different aspects.
But this is definitely a message that if you can compute gradients-- which we can't always do, in this case we can do it analytically-- you can get really quickly to an optimal solution.
And so fminunc and fmincon is the constrained sort of version of fminunc can be really powerful.
So let's see, Kevin wanted to know what happened if we started from a different solution.
So what would you like?
OK.
So do you want to pick a-- so somewhere here?
So like minus 1, minus 1?
Let's see.
OK.
So there's the new asterisk.
So what do you think's going to happen?
So it goes to the local peak.
It's going to convert pretty quickly.
So while an advantage of gradient based optimization is that it's super efficient, because you use the gradient information and you go to the [INAUDIBLE] in the design space and [INAUDIBLE] you can [INAUDIBLE].
We'll talk about [INAUDIBLE] condition.
The gradient of j equals 0 and [INAUDIBLE].
We can't tell the difference between being here, being here, or being here.
And that's kind of a drawback of a local method.
At best, we can say that we are in a local maximum in this case.
So we have to be careful about where we initialize.
And you probably guess if we start close to this one, we're going end up up there.
If we can start somewhere over here, we have a good chance of getting up here.
And if we start where we started, we're going to go here.
It's one of the drawbacks.
[INAUDIBLE], do you have a question?
Yeah.
I was going to ask how could you know how far to go [INAUDIBLE]?
Yeah.
We're going to talk about that, yeah.
So we're going to talk about how it chooses x, which is going to use gradient information.
But it's not just the gradient you'll see.
And then how it knows how far to go is that once it's picked the gradient direction, it looks to see the shape of the function in that direction.
And we'll talk about how it actually knows how far to go.
Yeah.
Is there another question?
[INAUDIBLE]
Yeah.
This one here?
Yeah.
It's not quite clear.
If it were right on the settle point-- it's not quite clear.
And I'll see if I can-- I think it's a similar question.
If I started over here, so let's see, x will be close to minus 3.
And y will be close to 3.
So let's start it at, like, minus 2.5 and plus 2.5.
So I think that's over in a corner of the design space.
Do you see any potential problems here?
It's really flat, right?
So when we compute the gradient, it's going to be really flat.
It's going to know that it's not at a local minimum, because the second road is not going to be positive.
But it can be that the algorithm has difficulty figuring out which way to go, which might also be the same thing if you started in that shoulder between the two.
But so it turns out, I mean, it's going to have difficulty.
And you can see it's probably having a few issues.
It's running pretty slowly, which means it's probably doing a lot more function calls.
But these algorithms and especially MATLAB's optimization toolbox is really good.
So they have ways of recovering.
So that when things go wrong, if you can't figure out where to go, you do a little bit of sampling and there are many ways that you can recover.
And in this case, we managed to recover and actually get up to the top.
We're not there yet.
It'll get there, though.
It should get there.
It is-- yeah, you can see.
Even though it managed to get there, I mean, I don't know if you looked at the numbers before.
This is the number of times that it calls the evaluation of the surface.
And usually, it's on the order of 3, 6, but you can see it took 27.
And that second iteration, that's because it was in this flat region.
But you can see that whatever MATLAB, whatever [INAUDIBLE] built in to try and sort of make the algorithm robust, actually did manage to recover in that [INAUDIBLE].
These things can definitely fail to converge.
They can come back without finding a solution.
So there's all kinds of problems you can run into.
But conceptually, this is what's going on.
I think that a picture of a landscape is a really good one to have in your mind as we talk about the math of the algorithms and the computing of the alpha and the x. OK?
Is there any good way to approximate the landscape if the function is too expensive to call?
[INAUDIBLE].
We're lucky that we know what it looks like.
[INAUDIBLE]
Yeah.
So we know where it is.
And we're visualizing it.
But the optimizer didn't know that
Right
So--
[INAUDIBLE] are there [INAUDIBLE]
So it depends on your problem.
If your problem has got particular structure, it may be that you can come up with an approximations, and in particular what's called complex approximations where you can guarantee something about the solution.
So that's going to depend very much on the structure of your problem.
That's what a lot of people in operations research spend a lot of time doing is taking difficult problems and then coming up with approximations or relaxations of the problem that they can then say something rigorous about the solution and how it relates to solution [INAUDIBLE].
But I think the reality is that in engineering design the landscape usually looks a real mess.
And not only does it have multiple hills and mountains, but often there's cliffs or part of a-- because we're not even looking constraints here.
Part of the design space where there's like a region where there's just no feasible design.
And basically, people apply optimization methods to try to improve designs, but don't necessarily worry as much about mathematical optimality.
It's more of a tool to help improve design.
So think a little bit on what you're trying to do as well.
Do you have a question?
[INAUDIBLE] what happens like [INAUDIBLE]
Yeah
[INAUDIBLE] still [INAUDIBLE]
Yeah.
So that crinkled up wing was probably like a solution wing.
It was horrible.
I mean, this is now-- that was a design problem with many more designs, so we can't visualize it.
But even just thinking about it here, it was probably a design that was somewhere down here because it was so awful.
But you know, as long as you can get reliable [INAUDIBLE] gradients using [INAUDIBLE] methods, you get yourself out of that awful place, and you get to a good one.
So it might take longer.
Yup.
OK.
So Let's see where we are.
So I'll turn this off.
OK.
So [INAUDIBLE] a couple of mathematical things that we may need to [INAUDIBLE] concepts that we need before we actually talk about what's going on in those optimization algorithms.
So we're going to need to think about gradients.
So let's just think about what the gradient is in this case.
And so what we're talking about for j of x-- and remember, this is a scalar objective, but this is n dimensional design [INAUDIBLE].
So we're talking about the objective function j of x, a scalar function of n design variables.
Then the gradient is what?
First of all, what dimension does the gradient have?
n by n?
What do you think it is?
[INAUDIBLE]?
N?
n by-- n by 1.
It's a vector of link n, n by 1.
So the gradient-- but with the gradients, I'm talking about the gradient of j with respect to x.
So you should write that as gradj.
You must have seen this in 1802, no?
Gradient?
Yeah.
You're frowning, Jacobi
No, I was nodding
Oh, you were nodding?
You looked like you frown when I [INAUDIBLE].
So gradient of j, which is the vector of partial derivatives, right?
bj, bx1, bj dx2 down to bj dxn.
So the gradient is-- it's an n by 1 vector [INAUDIBLE] grad j, which is just a vector of partial derivative.
And normally, we'll be interested in the gradient evaluated at a particular point, because you know, again, we're going to be evaluating the gradient at the point we're standing in the landscape.
And so we would write that maybe as gradj evaluated at the point xk.
So [INAUDIBLE] going to use q.
You can leave q over there, which would mean taking these partial derivatives and evaluating them at the point x equal to xq.
So you guys did a reading problem that I know was lots of issues, because it broke the MITX platform's ability to take pictures as an answer.
But you computed the gradient vector.
And then you substituted a particular value of the design vector x.
And then the gradients just came out to be numbers, right?
Yup.
So that's the gradient.
So when we talk about the gradient of the objective, it's an n by 1 vector that contains the partial derivative.
And so if we had a gradient [INAUDIBLE] have [INAUDIBLE] is also going to be a vector of dimension n. OK.
So we know that we need gradient of j equals 0 to be at an optimal point.
Yup.
Minimum, maximum, all at [INAUDIBLE] point.
So gradient of j equals 0 means that all these partials have to be equal to 0.
But we also said that if we wanted to be sure that we're at a minimum, we need to look at the second derivative information.
So what is the second derivative of j with respect to x?
What dimension does it have?
n by n. It's an n by n matrix.
And it's called the Hessian.
So the Hessian matrix is a matrix of second derivative.
So it's an n by n matrix.
So we could write it as grad squared j.
And so it's just on the diagonal, we're going to be the pure partial.
Del squared j del x1 squared del squared j del x2 squared all the way down to del squared j del xn squared.
And then on the up diagonals are they mixed terms, del squared j del x 1 del x1 del x2 and so on.
Del squared j del x1, del x10.
And what's special about this matrix?
It's symmetric.
Because a mixed partial with respect to 2, 1 is the same as the next partial with respect to 1, 2.
So n by n matrix, it's symmetric.
And so in the scalar case, if you wanted to minimize-- how do I want to put it?
Let me put it here.
So in the scalar case if I asked you to find a minimum of f of x where x was a scalar and f was a scalar, you would have two conditions, right?
You would say, yes, the x equals 0.
And then second derivative, [INAUDIBLE] positive.
And that would be evaluated at the x, the optimum point.
So we could have made it evaluate it at x star.
So you've seen all this before, right?
This is 1801 or high school calculus.
So in our case, we are trying to minimize j which isn't a scalar, but now it's a function of the design vector x.
So the corresponding condition here is this grad j evaluated at x star equals 0.
We can put a [INAUDIBLE] symbol on there if you want to remind yourself that this is an n by 1 vector, and [INAUDIBLE] grad j equals 0 means putting all those entries to 0.
What's the corresponding case here?
What's the analogous condition to second derivative being positive?
Who's taken 1806?
Here it's a scalar.
And it's got to be positive.
Here, it's a matrix.
What have you ever heard about matrices and being positive.
Positive definite, yeah.
So the condition is that the [INAUDIBLE] matrix grad squared j, again, evaluated at the thing.
And [INAUDIBLE] write it that way.
In symmetric matrix, all eigenvalues have to be positive.
They're going to be real and positive.
So in another way, for any vector, say y, y transposed times this matrix times y has to be strictly positive, which you can show that it's equivalent to the eigenvalue [INAUDIBLE].
So turns out the eigenvalues of this Hessian matrix tell you an awful lot about the shape of your design space.
And maybe intuitively, I find it conceptually easy to think about maximizing.
If I'm standing at the top of the hill and I'm trying to maximize and think about the eigenvalues and eigenvectors, there's no direction I can move where things are going to increase, right?
So that means all the eigenvector directions are going to have to be associated with decrease.
And so this would be flipping, right?
To be the maximum, the Hessian would have to be negative definite.
And so it kind of makes sense that the eigenvalues of the Hessian are going to relate to what goes on as we move away from a minimum or a maximum point.
OK.
So we've got gradient.
We've Hessian.
We need one more thing, which is your old friend Taylor series expansion.
Yeah, Greg
Can you go over that definition [INAUDIBLE]?
OK.
This right here?
Yeah.
So let me write it out here to be-- So let's call it h. So let's write the Hessian-- I'm just going to call it h, so I don't have to keep writing grad squared j.
So the condition is that h is positive definite.
And what that means-- we write it this way, h is a matrix.
What that means is that v transpose hv has to be positive for av, for any v that's not 0.
So we take any vector v and do the b transpose hv.
And that has to be positive.
If that's true, then h is positive definite.
And then, because h is symmetric matrix, that is the same condition as saying all the eigenvalues of h-- and there are going to the n of them-- have to also be positive.
It's a property of the matrix
[INAUDIBLE]
Yeah
[INAUDIBLE]
So if not all the eigenvalues are positive, if some of them are 0, then it means--
[INAUDIBLE]
Well, so if some of the eigenvalues are positive and some are negative, it means you're at a settle point.
You've seen that in the 1dk--
[INAUDIBLE]
So I don't know how to act this out.
But in a landscape if I a settle point is [INAUDIBLE] point means that the [INAUDIBLE] goes this way in one direction, but this way in the other.
So there's still a direction that I can move to [INAUDIBLE]
[INAUDIBLE]
So maybe [INAUDIBLE] said another way.
An optimization algorithm that's trying to minimize or maximize something won't stop at those points.
Because it will be up to find a direction of improvement
[INAUDIBLE]
No, it'll keep going
[INAUDIBLE]
That's right.
Think about standing on-- you're standing on the edge and things are dropping either side of you.
But if you're looking front, you can keep going up the hill.
So it's just going to constrain the directions in which you move.
But the problem with [AUDIO OUT] as we got into the [AUDIO OUT] regions where [AUDIO OUT] you might know [AUDIO OUT] if you're minimal and not unique, then there's actually a ridge.
The top of the mountain would be the ridge.
And so there's a direction that you can walk along, and you're not changing an objective.
You're staying at constant elevation.
And that would correspond to one of the eigenvalues being 0.
But in that case, what you've got is a whole bunch of designs that are equally good.
And so that's actually kind of nice to know.
Because that would be different design decisions that you could make that were all as good in cost or whatever it is you're trying to do
[INAUDIBLE]
Yeah.
So, yeah
[INAUDIBLE]
That's right
[INAUDIBLE]
Yeah.
So you don't have to worry about this stuff.
This stuff is taken care of for you when you run something like fminunc and fmincon.
But what's important is to understand that there are sort of rigorous mathematical conditions that tell you when you're at optimal solution.
And if your problem sort of obeys the given structure, then that's great.
And that holds.
In reality, remember when we had our list of design variables we talked about having variables like number of engines?
That's something for which the gradient doesn't even exist, right?
It's not differentiable.
And so in many cases, our problems don't even satisfy the requirements for the algorithms that we use.
And so there are very few guarantees.
But they can still help us make progress
[INAUDIBLE]
That's the optimal solution?
[INAUDIBLE]
If there's one of these ridges, if there's a non-unique solution, yeah.
Yeah
[INAUDIBLE]
Yeah.
So integers becomes a whole other ball game.
And maybe I should have said right off the bat that when you start looking at the gradient and the Hessian analysis, [INAUDIBLE] j of x needs to be at least twice differentiable with respect to x.
Or otherwise, what we're writing here doesn't make sense.
When you have integers, the conditions become more complicated.
And how you solve the problem becomes more complicated.
One approach is to let the integers vary, exactly what you're saying.
And then at the end, you round.
And that might be effective, but it's certainly not guaranteed to find an optimal solution.
There are specialized optimization solution methods that are tailored for integer programs.
And in fact, MATLAB just released a mixed integer program, I think, as part of their latest release of the optimization toolbox.
And these methods will handle the integer variables in different ways.
It's something called [INAUDIBLE] bound, which is about searching down different integer combinations.
But yeah, integer variables make it really difficult.
Yeah, Alex
[INAUDIBLE] j [INAUDIBLE]
Yeah, so we're going to talk about-- that's sort of number three.
j is usually going to be computer code that we can put in the shape of the aircraft wing or the whole aircraft, and out comes range or cost or whatever it is, that's right.
So we can usually only get j of x through simulation not necessarily by analytic.
And so then we're going to need some way to compute gradients.
And what's really nice is finite differences, which you guys saw back a few months ago, is what we're going to use to do that.
Yeah
Like also, one thing we talked about [INAUDIBLE]
So it's all taken care of, because we're not sampling.
We're actually measuring the gradient.
And I mean, they're going to be-- I mean, we're doing something different here, right?
We're moving through the landscape.
We're not sampling, and then trying to say how this variable relates to this one.
We're actually just looking for an optimum solution.
It's a different thing.
So whatever interactions are there, how are they affected?
They're reflected in the shape of the landscape.
The shape that the landscape takes is a manifestation of how the design variables relate to each other and how they affect the objective
[INAUDIBLE]
Yeah.
There are.
And I'll show you some of those.
The problem is actually now to guarantee a global maximum.
So there are a whole bunch of methods called heuristic methods.
So genetic algorithms-- which Professor [?
Devic ?]
uses a lot in his research, which I happen to not particularly care for-- are ways to do that.
And I'll show you maybe in the next lecture.
They sort of spray points everywhere.
And then they use an analogy with natural selection and mutations.
And designs have babies.
And then the strong babies survive and the other ones don't.
So there will be [INAUDIBLE].
And you know, people sort of claim that's a way to do global optimization.
The problem is there are no guarantees at all.
I'll show you another one, which is a patent search, which sort of has a guarantee of eventually finding a global solution, but only [INAUDIBLE] that you search forever.
Yeah.
And it's a really hard problem.
If you know something about the structure of your problem, you maybe have to do something.
And certain problems, like we were talking about before, have the nice structure where you can be rigorous.
If you have truly just kind of this black box complicated aircraft design problem, there are no guarantees.
But again, often what you're trying to do is to find a good design that meets all the constraints that's better than what you could do by hand.
So [INAUDIBLE].
Yeah.
It depends.
Are you an engineer trying to make a good design decision?
Or are you a mathematician who wants to guarantee optimality?
And where do you fall in that?
So that's god.
So you guys have lots of questions.
You're trying to avoid getting Taylor series expansions I know.
Any other questions?
So Greg, does this sort of answer-- I know I didn't do it very deeply.
But it's probably enough
[INAUDIBLE]
So just the last mathematical ingredient that we need or that we will need are going to be Taylor series expansions.
And it's just, again, same thing.
You've seen them in the scalar case.
But let's just make sure it's clear in the gradient case.
And by the way, If you really do have lots of questions and you need to take the graduate class set [INAUDIBLE] and I teach-- which we're just teaching it this semester, and it's offered every other year.
But it's on design optimization, a whole class on this stuff, which is fun.
Taylor series expansion.
Again, in the scalar case, let's just do what we did over there and do the analogy.
So in the scalar case, if I had some f of-- let me use z. I probably should have used z over there.
It's OK.
If I have some f of z, what is that?
It's f. If I'm expanding about the point b0 plus the first derivative evaluated at the point b0 times z minus z0 plus the second derivative evaluated at the point z0 times z minus z0 squared plus blah, blah, blah.
Right?
So that's the Taylor series expansion that you've seen.
So how does it look in the vector case?
So when we talk about Taylor series expansion, we're talking about expanding j of x, where again x is now this n dimensional vector.
And we're going to expand it around the point x0 point in the landscape.
OK.
So what does the first term look like?
Gradient of j evaluated at x0 multiplied by x minus x0.
OK.
So we have to think about the dimensions.
It always helps me think about the dimensions.
So [INAUDIBLE] n by 1, right?
What do we need to do to this thing?
Transpose.
So it's in a product between the gradient evaluated at x0 and in the delta x, x minus x0.
Yup.
Scalar, scalar, 1 by n times n by 1.
That's a scalar.
OK. How about the second derivative term?
What do you think that might look like?
We'll put the Hessian in the middle.
And it's the Hessian, again, evaluated at-- let me write it thigs way-- x0
x [INAUDIBLE]
OK, good.
Yeah.
So x minus x0 on that side.
And then x minus x0 transpose.
And again, this is an n by n and n by 1 [INAUDIBLE] a scalar.
And then if you went higher, you would be getting a tensor for the third derivative
[INAUDIBLE]
You can.
And in fact tensors are very popular, lots of people working on them now.
I don't-- don't ask me anything about them.
OK.
So let me come back to this.
Then I can start showing you some of the optimization measures.
I'll show you pictures of some of them first.
And then we'll look-- yeah
[INAUDIBLE]
No.
Yeah.
So I could've written-- yeah.
So that's just the point about which we were doing, the Taylor series expansion.
And what you'll probably guess is that it's actually going to be xq.
We're going to lock locally
[INAUDIBLE]
Yeah, well, whatever.
q minus 1, yeah.
OK.
So we can close this.
So actually Professor Johnson, who's in the math department who teaches a really great graduate class on numerical linear algebra, it actually covers lots of things.
He has this NLopt package where he's implemented lots of optimization algorithms.
And he has them implemented.
And you can access them in MATLAB or Python or a variety of ways.
And also on the website, he was a really nice kind of description of the algorithms and what kind of problems they work for and issues with conversions and stuff.
It's really nice.
But [INAUDIBLE], you were asking about some of the different kinds of methods.
So this is sort of the four categories.
There are global optimization methods that sort of strive for this, being able to find the global optimum.
There are local methods.
And I'll show you how these work.
And in the local methods, there can be ones that are called derivative-free and gradient-based.
So these one use gradients, and these ones don't use gradients.
And then there are a heuristic methods, things like the genetic algorithms that kind of use a bunch of roles and some randomness to search the design space.
Yup
[INAUDIBLE]
So non-linear refers to the fact that j of x could be a general non-linear function and that the g of x and h of x-- yup.
To be a linear program, j would have to be a linear function of the x's.
So just w1 times x1 plus w2.
And then the constraints would also have to be linear functions of [INAUDIBLE].
And if you have a linear problem, then there's the [INAUDIBLE] simplex, different from the Nelder-Mead simplex, but the simplex method that's really efficient, can solve really big problems, lots of theoretical guarantees.
All right.
So he's the Nelder-Mead simplex.
So this was fminsearch.
Remember, that was the very first one that our little landscape was working on.
So this is a local method that's derivative free.
So it's not using any gradient.
And how does it work?
So a simplex is a special polytope of N plus 1 vertices in N dimensions.
For me it's easiest to think about in 2D [INAUDIBLE] a triangle.
So simplex is a triangle in 2D.
And here's how this Nelder-Mead simplex method works.
So you take your initial guess that, again, we have to supply.
This is the initial point in the landscape.
And you form an initial simplex.
So with our 2D landscape, we're going to put down a triangle, initial triangle.
And we're going to evaluate.
And remember, I'm talking about unconstrained optimization here.
So there are no constraints.
So you evaluate j, the objective, at each one of the points in the triangle, three.
So in two dimensions, we have three points.
So that's what can keep the function value.
The function value here is j.
[INAUDIBLE] the triangle-- the triangle can be--
[INAUDIBLE]
Yeah.
I mean, yes.
In terms of size?
You've got your initial point.
And you're going to put two other down here.
There would be-- they're close by though.
It's local.
So they're close by.
Maybe not super close, but they're close by.
And you'll see that's going to change.
So it doesn't actually matter too much what we start with.
We're going to order the vertices according to function values and discard the worse one.
So if we're trying to minimize, we have the highest value, the lowest value, and the middle one.
If we're trying to minimize, we would throw out the one with the highest value, right?
So we're going to throw away, in this case, x high.
Because it's got the highest value, and we're trying to minimize.
And we're going to generate a new point by what's called reflection, which means that I'm going to come across this line of the remaining two and reflect the triangle over and generate a new point that's kind of on the opposite side of the simplex.
And at the same time, I'm also going to-- I'm going to [INAUDIBLE] a new point over here.
This is the xr.
I'm going to run the function there and see whether things got better or not.
So we're trying to minimize.
We're going downhill.
Run the function here and see whether this is actually a better function value.
And then I'm also going to decide, depending on how steeply down the hill I'm going, whether I want to change the size of my simplex.
And if things are going really well and I'm generating points and I'm really going downhill quickly, I'm going to make the simplex bigger and bigger, so that I can go down hill faster and faster.
But if this guy's not really so good, then I might shrink the simplex so that I look more locally.
So maybe you can kind of visualize that what this optimization algorithm looks like is a bunch of triangles that keep flipping over just by measuring, by ordering the performance of the vertices, throwing out the old one, generating a new one on the other side, walking kind of through the design space.
And then the triangles are growing or shrinking depending on how well things are going.
So there's no gradient.
All it is is just a sampling and an ordering.
There are some convergence issues in this.
But actually, it's kind of a simple algorithm.
And it's pretty robust.
The function doesn't have to be differentiable, right?
j of x doesn't have to be smooth as long as a better point has a lower value of j of x than another one.
That's OK, right?
So when I ran fminsearch, way back up if I pull it up.
So you can see here in the MATLAB output it was [INAUDIBLE] things.
So that was telling me what was going on.
Every time it was evaluating new points, so two new points each time, that was the simplex expanding, reflecting, contracting, knew what was going on with points.
OK.
So that's a derivative-free method.
It uses a little bit of sampling, but it is local.
And that's fminsearch in MATLAB.
This one is sort of the same-- I don't know it sort of has the same feeling.
But it's a global optimization method.
It's called DIRECT.
And DIRECT stands for Dividing Rectangles.
And basically what it does is it divides the domain into these rectangles in 2D.
You'll have rectangles in multiple D. And it basically figures out where to look.
So if this one is interesting, then it would divide that more.
And then this guy is interesting, so then divide that one more.
And that one's interesting and keep dividing and dividing and dividing more.
And it's got all different roles for figuring out which ones to divide and which ones not to divide and how to progress.
So this is one that tries to go at global optimization.
Problem with this one is that it just really keeps running and running and running and running and running.
So we've used this a couple times.
But it tends to just take so long to run that it's not particularly useful.
OK.
So let's see what time we have left.
We have about 10 minutes.
So let's at least start talking about the gradient-based method.
So again, this is what we've been talking about starting with an initial guess, x0.
Then we've going to compute two things, the search direction, the Sq, and then the alpha-- this should have a q on it-- how far we step in that direction.
And with a gradient-based method, we're going to use gradient of j to compute this Sq.
OK. And we're going to check for convergence.
We'll talk about what that might mean and just keep going around this slope until we're done.
So these are the methods that we'll talk about.
I think maybe we'll just talk about steepest descent right now.
And then we can talk about the other ones on Monday.
But steepest descent is conjugate gradient of first-order method.
Newton method, which I think you've seen maybe in the scalar case.
Have you seen the Newton method for root finding in the scalar case?
You've seen Newton [INAUDIBLE] in [INAUDIBLE], a little bit different.
We'll see how Newton method looks in the [INAUDIBLE] case.
And then there are things called quasi-Newton methods that kind of sit in the middle between the two.
So let's look at steepest descent.
It's the simplest thing you could possibly do, possibly think about doing.
And it just says fix this search direction on iteration q to be negative the gradient of j evaluated at the current design iterate, Sq minus 1.
So why would you pick that search direction?
It's the steepest descent, yeah.
So we know that the gradient of j points in the direction of maximum local increase of j.
Negative gradient of j points in the direction of steepest descent.
So here's the algorithm.
But again, just think about the landscape.
What are you doing?
You're standing in the landscape.
You're looking around.
You're finding the direction of steepest descent.
And you're going to go in that direction.
And we'll have to talk on Monday about how to choose the alpha.
But we find the direction of steepest descent.
We choose the alpha.
We take a step.
We look around.
We find the direction of steepest descent.
We can pick a new gradient, find the new alpha, take a step, and keep repeating
[INAUDIBLE]
That's right.
That's exactly right.
And I'll show you how we do this on Monday.
That's exactly right.
It becomes [INAUDIBLE] the 1D search.
Because once you've define the direction, it's now just 1D.
So that's the first gradient-based optimization algorithm.
It's really simple.
It turns out that's not a great algorithm, that it converges really slowly.
But conjugate gradient is actually something that's not too different.
So now what does conjugate gradient do?
The first search direction S on the first iteration is the steepest descent direction.
Yeah.
But then on subsequent iterations, the search direction is the steepest descent direction minus squared j plus this term that's beta q times H q minus 1, so the steepest descent direction modified by the last search direction.
And the beta is the ratio of the gradients on the last two-- the ratio of the norm are the gradients from the last two search directions.
So what is going on here?
Again, I think you kind of look at the math.
But basically, what is happening is you think about the landscape.
So I'm standing at x0.
I find the direction the steepest descent.
I move in that direction.
Now when I get to the second iteration, I find the direction of steepest descent.
But I also look over my shoulder, and I see where did I come from.
I find the direction of the steepest descent.
And then depending on where I came from, I'm going to modify that direction a little bit, and then move.
And then I'm going to go, and I'm going to find the direction of steepest descent.
I'm going to look where I just came from and modify it a little bit.
So I'm incorporating information about where I came from.
And if you look at this example, this function is a really famous example that is an analytic example that's often used as optimization algorithms.
It's called the Rosenbrock function.
It's sometimes called the banana function, because it looks like a banana with really steep walls.
And what you can see here on the left is the steepest descent algorithm starting from the initial point here.
So we start here.
We compute the direction of steepest descent.
These are contours of objectives.
So the direction of steepest descent is perpendicular to the contours, right?
Yup.
Perpendicular to the contours, what you were saying, Dominic.
It looks along here.
And there's the point that would minimize.
It gets to the new point, completes the direction of steepest descent, moves along here.
It gets here, completes the direction of steepest descent, steepest descent, steepest descent, steepest descent, can you see what's happening to this guy?
It's going to take hundreds probably of little tiny steps going zz, zz, zz, zz, as it gets to the optimum solution.
What does conjugate gradient do?
So the first one is the same, the first iteration, steepest descent.
But now, when we get to this point, we compute the direction of steepest descent, which again would be perpendicular to the contour.
So it would be like that.
But now, we also look back over our shoulder, see where we came from, compute that data, add in that beta S1 term and modify it a little bit.
So it actually takes us there.
Not terribly different, right?
But enough different.
Now, we get here.
Steepest descent, again, would be perpendicular.
It would be here.
We modify it a little bit.
We skip all the way here and continue on.
So conjugate gradient converted in 1, 2, 3, 4, 5 iterations.
Whereas, steepest descent [AUDIO OUT] gradient [AUDIO OUT] much, much faster [INAUDIBLE] to converge.
And [AUDIO OUT] and you want to get down, don't use steepest descent.
Because you're going to end up going back and forth across yourself.
Always look to see where you came from [INAUDIBLE].
OK.
So let's finish there.
We'll talk about Newton's method and various other things.
We'll talk about computing gradients on a Monday as well.
But if you have questions, stick around.
I'm going to do office hours now if you questions about the lecture or questions about the project.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK everybody, let's get started.
So if you came in late, project three is here.
It's graded in here.
You're going to have project two back tomorrow.
Yeah.
The project threes were really good.
The one thing that seemed to trip up-- well, there was a few things that tripped up some people, but the one thing that seemed to trip up a few people was just the computation of the number of samples given the Monte Carlo simulation to achieve the required stated accuracy of plus or minus .01 on the probability estimate.
So some people got it really well.
Some people got it, but their explanations were a little bit muddled in some of their words.
And then some people didn't get it quite right.
And I think it's a really important thing, and it's really important thing to make sure it's very clear in your mind.
And one thing that I would recommend is that whenever you're talking about mean and variance or mean and standard variation that you make sure it's very clear in your mind of which random variable you're talking about, because just saying mean or just saying variance doesn't necessarily tell me or you anything when there's lots of variances floating around.
In this particular example, there's the variance of the output of the simulation that we're trying to estimate, which is how far the ball flies.
So how far the ball flies is a random variable that's got its own variance.
But then the probability estimates themselves have got variances because the probability estimates are random variables because you're estimating them using Monte Carlo, which is a random process.
So when you're asked to get the probability estimate to within plus or minus 0.01 with 99% confidence, then that says we want to look at the variance of the probability estimate.
How much does the probability estimate itself vary?
And the variance of that estimator-- so what's that estimator?
Let's call it p hat is the estimator of the probability.
Well, we know that saying by the central limit theorem, it's distributed as a normal distribution with mean equal to the probability that we're actually trying to estimate, and variance of that probability p of a, 1 minus p of a over n. Yup
[INAUDIBLE]
We'll also put the lights on.
That will help.
Do you mind going into my office and grabbing some of the big chalk that's on-- great, thanks.
I hate the little chalk.
We'll get some bigger chalk.
Can you see a bit better now?
So this is the variance, but it's the variance of what?
It's the variance of this estimator, p hat of a, and this is the mean of p hat of a.
So again, just any time you write variance, variance of what?
Variance of what random variable?
And I sort of got the feeling some people who knew it had to be this thing, but with defining this thing as the variance, and then just randomly or magically dividing by n, but make sure that this is really clear in your mind that the estimator itself is a random variable.
And then in order to figure out how many samples you take, we've got to control the variance of the estimator.
And then because we know this thing is normally distributed, and we want 99% confidence, that's plus or minus 3 sigma or plus or minus 2.58, some of you computed, so we need to make sure that this sigma, square root of this times 3 or times 2.58 falls within plus or minus-- is less than 0.01
[INAUDIBLE]
Less than here?
Yeah
So that's through the central limit theorem, because we know that we can write the estimate of the probability-- remember, we could write it as 1 over n, the sum from i equals 1 to n of i of ai
[INAUDIBLE]
Yeah, so this is another important thing to keep in mind is the output itself doesn't have to be normally distributed, but the estimator for the probability does have a normal distribution.
And the reason it does is because we can write it as-- this is like a sample mean of an indicator function, something that's either 0 or 1.
And so by central limit theorem, this thing for big enough n has a normal distribution.
The same is true for the estimator for the mean, because that's 1 over n, the sum from i equal 1 to n-- then we called this thing y, but I think we called it y bar in the notes.
The distribution of y bar, the estimator for the mean also has a normal distribution.
So remember, we saw estimators for mean, for probability, and for variance.
These two have normal distributions regardless of the distribution of the output, but the variance doesn't.
Is that clear?
And again, that's why I think it's important to say variance of what, or distribution of what.
This is the distribution of the random variable, which is the estimator of the probability.
It's sort of got nothing-- it's got nothing to do with the variance of the range of the ball in this case, or even the shape of the distribution.
But what it does depend on is the probability we're trying to estimate.
And so we use this result to figure out how many samples to take, but we don't have an estimate for this guy.
So a trick that helps people realize is that we know the worst case for this.
So what value of p will give us the biggest variance?
p equal 1/2?
If p is 1/2, this 1/2 times 1/2 is 0.25.
That's the biggest value that p1 minus p can take.
There's 0 at either end and 0.25 in the middle.
So one strategy is to compute this using p equal 1/2, which is the worst case, and I would just do plus or minus 3 sigma.
It's a little bit conservative, and that gives you something like 22,500 is the n that you need.
And then you know you'll be OK for anything.
And it turns out one of them, the probability came out to be like 0.52 or something.
And that would be the worst case, and you'd be a little bit conservative here, but that's fine.
You'd still be meeting your plus or minus 0.01.
So just make sure make sure that that's really clear in your mind as you go through how you take all these different pieces that you're clear about, the distribution and what is normally distributed and what isn't, and then you can work from this variance to identifying the plus or minus 3 sigma to give 99% confidence and then translate that into an n. OK.
So if you didn't get it quite right, take a look at it.
And then if it's not clear, I'm going to do sort of a summary of everything on Wednesday, and we can go through it again in more detail then if you want to.
And I'll also post some solutions tonight or tomorrow.
But any other questions?
OK.
So today we're going to finish talking about unconstrained optimization.
We'll just pick up where we left off on last Wednesday.
And we'll talk a lot about how we compute gradients.
We might talk a bit about the 1D search if we have time.
And then we'll finish up by talking about response surface modelling.
If you remember where we left off last Wednesday, so first up, this is a basic optimization problem we've been talking about.
Remember, minimize some objective j that's a function of design variables x, and we have x1, x2, up to xn design variables.
Parameters p subject to inequality constraints that we write as dJ is less than or equal to 0, and equality constraints that we write as hk equal to 0.
And in general, we might have m1 inequality constraints and m2 equality constraints, and then we might also have bounds on the design space.
So we saw this last Wednesday.
And we also talked about the general process of optimization.
And remember, I said think of it as a landscape where the directions in the landscape are the design variables, and the height of the landscape represents the objective function.
If you're maximizing, your goal is to get to the top of the highest mountain or highest hill in the landscape.
If you're minimizing, your goal is to get to the bottom of the lowest valley.
And the way that that works is that we start with some initial guess that we denote x with a superscript 0.
That's the initial guess.
We look around.
For a gradient-based method, we're going to also compute the gradient of the objective of where we're standing in the landscape.
Look around, find the search direction.
Once we've found that search direction, look along that search direction.
Perform what's called the 1D search and take a step alpha.
Compute a new design, figure out whether we've converged, and keep going.
So we're walking around the landscape iterating with our design variables.
And what we talked about on Wednesday was the steepest descent method, which says always choose the search direction to be the negative of gradient of j.
So that's the direction of steepest descent.
And we also saw conjugate gradient, where we modify the search direction according to where we have come from.
And remember that we saw that steepest descent-- in this plot where the objective function is represented with the contours, the steepest descent direction is always perpendicular to the contours.
And we saw that as we approached the optimum, steepest descent starts to take this zig-zagging path with really, really small steps.
Whereas conjugate gradient, which modifies each search direction depending on the previous search direction, is able to converge much more quickly.
And in this case, it's just [?
a quick consideration.
?]
OK, so that's where we are stopped on Wednesday.
So I want to mention just really quickly a couple other methods.
So if I go back actually, we talked about first order method that use just gradient information, and that was steepest descent and conjugate gradient.
So there are also second order method that use not just the first derivative, but also second derivative information.
And remember, we also derived on Wednesday that the second derivative of j is an n by n matrix.
That was the Hessian matrix, the matrix of second derivative that handled the pure second on the diagonal, and then all the cross terms on the off diagonals.
So how does Newton's method work?
Well, if you think about a Taylor series-- and again, we derived this on Wednesday.
The Taylor series in the case of a scalar objective that depends on a vector of design variables, so x is an n-dimensional vector.
If we expand j of x about the point j of x0, then the first term is the gradient times delta x. Delta x is x minus x0.
And then remember, the second term was this quadratic term that looked like delta x transposed times the Hessian times delta x.
Remember, we did that on Wednesday.
And this is an approximation, because we're truncating all the third order terms and higher.
So we can use this Taylor series expansion, and we can say, differentiate both sides.
So grad J of x can be approximated by grad J of x0 plus the Hessian evaluated at x0 times delta x. Yeah.
And what are we looking for?
We're looking for a place with the gradient of j is 0.
Remember, that was the condition for finding a minimum or a maximum, or it could also be a saddle point.
So if we set this left hand side equal to 0, then what's left is this condition here.
And so now we can rearrange and say that the delta x, the step that we take is going to be the inverse of the Hessian at the point we're standing at times the gradient of the objective at that point with a minus sign.
OK, so remember [INAUDIBLE] in the steepest descent method, we said that the search direction was just equal to minus grad J.
And we're writing that x at q plus 1 is equal to x at q plus alpha q-- let me write that as a subscript-- times [?
x ?]
q. OK.
So that's what we've been saying is that the new guess of the design is the old one plus [INAUDIBLE] term.
In the steepest descent method, where it's just minus gradient of J.
So now you can see with Newton's method, we have that the delta x, which is this guy here, the alpha q times [?
this ?]
[?
direction ?]
[?
s ?]
q is the negative of the inverse of the Hessian times gradient of
[INAUDIBLE]
Yeah, so we'll have to check that when we converge, when we get to-- we'll check that on convergence.
Yeah.
Typically you don't check it as you go along, but it actually depends a little bit on the algorithm.
The next ones that you'll see, you'll see that the desire for Hessian to be positive definite is worked in.
OK, so how does Newton's method work?
This says S, but it really should be alpha times S in the way I defined the update in delta x.
So maybe you can see that if you're trying to optimize a quadratic function, this extension is exact.
There's no approximation here.
And in that case, Newton's method would converge in one step.
So that means that if I was trying to optimize some kind of a quadratic function, or the landscape would be like a bowl, no matter where I started, I would get to the optimum in one step with Newton's method.
But if the function's not actually quadratic, then what we're basically doing is computing an approximation of the function here as a quadratic.
So however it would look, it would look maybe something like this, we'd be approximating here.
We would solve Newton's method.
We would jump to there.
We would create a new approximation locally that might look like this, and then we would come in and eventually we would converge.
So if J is not quadratic, you just keep performing Taylor series, getting a new point, and then repeating until it's converged.
So this turns out to be a really efficient technique if you start near the solution, if you're starting sort of close to one of the local bowls of attraction if you're minimizing.
But the problem is that you can see that now we need not just gradients, but we need also the second derivative.
I'm going to talk in a second about how to estimate the gradients even if is actually in many cases getting this Hessian ends up to be much too expensive
[INAUDIBLE]
How do we define convergence?
[INAUDIBLE].
Yeah, so for an unconstrained problem, remember we're looking for two conditions.
We're looking for the gradient of j equal to 0.
And we're looking for Hessian to be positive definite.
Or positive semi-definite depending on the uniqueness.
That's in theory.
It turns out that in implementation, it can be a little bit tricky to monitor convergence.
So often you would look for the sequence of the design variables to stop changing.
In other words, when the updates become really small.
And that's when you would terminate, and then you would try to check the optimality condition.
So let's talk about how to estimate the gradient.
And this actually will connect back to some stuff that you saw earlier in the semester.
So we're onto number two, which is computing gradients.
So, generally we're going to need definitely grad j, which remember was the partial derivatives of j with respect to all of the design variables.
And we might need the second derivatives.
If we wanted to do the Newton method, then we might also need the second derivatives.
And I won't write it out, but again, remember that was the n by n matrix of second order partials.
So methods to compute gradient.
Any ideas?
Finite difference, yup.
So that's finite difference, which we'll talk about.
That's probably the most commonly used in practice that people are using.
Any other ideas?
If you could, what would be the most reliable way to get gradient?
Differentiate, yeah.
So analytical gradients, if you can.
If you have a code, or if you have a model that analytic, that's all functions you can differentiate, then differentiate.
Analytic gradients is definitely the way to go.
Have you guys heard of the adjoint method, that Professor Wong talked about?
[INAUDIBLE] So this is a really powerful way to get gradients when you have particularly CFD models or [?
panelement ?]
models that have got lots and lots and lots of design variables.
So like if you're trying to do shape optimization of an aircraft wing and your design variables were all the surface points or all the properties of the sections in a wing, so you have hundreds and hundreds of design variables, people use what's the adjoint method to come up with gradients.
Has anybody heard of automatic differentiation?
So this is something that's becoming kind of increasingly popular, where people write these automatic differentiation codes, and basically you take your code-- so you take your code that takes in x and puts out J, and you feed it through this automatic differentiator, and it gives you back a code that takes in x and gives you grad J.
Sort of somewhere at the intersection of computer science and computational science, I guess.
They're write these codes that will go through your code and differentiate it line by line by line, and use the chain rule, and basically automatically differentiate it.
And so there's an example of this in [?
EETS ?].
There's a big model called the MIT GCM, which is the MIT Global Circulation Model.
I don't know if anybody's heard of it.
Nicholas, you work over in [?
EETS ?
], right?
Have you--
[INAUDIBLE]
MIT's been building for a long time this really massive community model that's doing a lot of modeling of the ocean and atmosphere and interactions for climate change.
And so they've used automatic differentiation so that you can take this massive model and come up with gradients.
And it's symbolic differentiation too, right?
Isn't it Mathematica?
There's a lot of different ways you can come up with gradients.
[?
And there's ?]
[?
another one ?]
[INAUDIBLE] something called the complex step method, which I'm afraid we don't have time to talk about.
It's a pretty neat trick.
I can't remember the formula.
Do you know the formula, Alex?
[INAUDIBLE]
Yeah.
I don't want to write it down and get it wrong, but if you can take your model, your J of x and put in a complex x-- maybe you can find the formula.
Anyway, it's a really neat trick where you can pretend that your variables are complex, and then it turns out that the gradient ends up being the imaginary part.
So that's also neat.
And if your code's written in Fortran, this can be a neat trick.
Do you guys even know Fortran?
You know, Professor [?
Jolley ?]
still writes all of his codes in Fortran
[INAUDIBLE]
You're painfully aware?
So in Fortran, you can define complex variables
[INAUDIBLE]
The imaginary part of J, x of [?
pi ?]
times [?
a ?]
[?
h ?]
is equal to the gradient?
[INAUDIBLE]
Divided by [?
J ?]
is equal to the gradient.
Yeah, so [INAUDIBLE].
It turns out if you take a complex step, if you take your variable x and add a complex step, and then divide by that h, it turns out the imaginary part of that result ends up giving you the gradient.
OK, but anyway.
Finite differences is the most commonly used because it's sort of the general purpose one.
So let's just look at how finite differences work.
And these formulas should look somewhat familiar, although you saw them in a different setting.
Remember, you saw finite differences when we were approximating partial derivatives in the [?
PEs ?
], right?
Remember way back?
You're look at the convection equation or the convection diffusion equation, where you had things like du dx, or [INAUDIBLE] squared [INAUDIBLE] squared, and you used finite differences?
The exact same idea can be applied now to find the gradient of J with respect to design variables x.
So for example, if we were looking for dJ dx i at some point, let's just call it x0, then we would just take J and we'd have x1, x2, all the way, and we would have xi plus delta xi.
So we're going to perturb the ith design variable.
And then we would have xi plus 1 all the way up to xn.
They would all be fixed.
And we're going to differentiate that with the baseline point, xi up to xn.
And all that's going to be divided by delta xi.
And so this thing here is the numerator.
So this is the baseline point, the x0 would be right here.
So there's the baseline point.
And then if we want to estimate the gradient in the direction xi, we just take xi and we perturb it by delta xi, take the difference between the function [INAUDIBLE] perturbed point minus the baseline divided by delta xi.
Yeah.
So again, it looks pretty similar to what you saw when we talked about finite differences earlier.
So this would be a one-sided difference
[INAUDIBLE]
Different ways of computing the finite difference?
Yeah, exactly.
So that's a one-sided.
You could also do a two-sided difference with the same partial.
So it would be J dx i evaluated at the point x0 could be J x1, x2, to xi plus delta xi minus J x1, x2, xi minus delta xi.
All divided out by 2.
[INAUDIBLE] central difference.
So you can evaluate the derivative at the same point using the point in one side, or you could use one little step forward, one little step back, and do it like a central difference.
And if you wanted to do higher order, you could take more points.
But now here's the catch.
So if we want to estimate [?
compute ?]
to compute grad J at whatever the current point is in the design space, how many function calls do we need if we use a one-sided difference?
How many times do we need to run our model?
2n if we were to use this one.
How about this one?
Well, almost 2n.
n plus 1?
So 1 for the baseline and then it's going to be a little step in x1, a little step in x2, a little step in x3, all the way through xn.
So with a one-sided, we would need n plus 1.
And for a two-sided, we would just need 2n.
We don't have the baseline.
Function evaluations.
Now remember, each function evaluation in the course of your simulation code, which could take minutes or hours or even days.
So if you have 100 design variables, each time you want a gradient, you're going to have to do of the order of 100 calls to your function.
And remember, you're going to need gradients at each point in the design space when you're trying to figure out where to move.
So it can get expensive pretty quickly.
How about if you wanted to compute the Hessian by finite differences?
Well, we didn't write down the formula, but you remember the formula for the second derivative, right?
If we were computing that guy, then it would be J, and we would do xi-- probably don't want to write out all the arguments.
xi minus 2J at the nominal point minus J doing xi minus-- do you remember that formula?
Divided by delta x squared.
The central finite difference for the second order derivative.
Do you guys remember that?
Am I confusing you, or is it OK to not write out all the x's?
Is that all right?
Yeah.
So to compute second derivative, first of all you're going to need the point in front and the point behind.
But how many entries in the matrix?
[INAUDIBLE]
Yeah, it's n by n. It turns out it's symmetric, so you don't need-- remember it's symmetric, so you only need to compute the diagonal and the upper triangle.
It still turns out to be n plus 1 over 2 entries, and then you need the extra function evaluation, so then you would need two for each of these.
So it basically ends up being [INAUDIBLE] n squared.
And so if you have 100 design variables, that's 100 squared, 10,000 calls to your function to get a Hessian matrix of the order every iteration.
Which is why I say the Newton method is powerful, but if you're doing gradients by finite difference, almost it means you can't afford to use the Newton method.
But the good news is that MATLAB takes care of all of this for you.
We can have a look at how MATLAB does that for you.
Is this clear how you estimate the gradients?
Yup?
So let me pull this.
So I showed you already on Wednesday.
[INAUDIBLE] So I showed you already on Wednesday this little [?
piece ?].
You might remember when we looked at the landscape and we saw the little asterisks climbing around.
So I showed you [INAUDIBLE], but here's the call.
So remember I talked about [?
Effman ?]
unc, which is the unconstrained gradient-based optimization method in MATLAB.
And I also talked about [?
Effman ?]
search, which I said was the [INAUDIBLE] simplex.
Remember that was the triangles that were flipping around and going through the design space.
So the thing with [?
Naldemede ?]
simplex is you don't need gradients.
It doesn't need gradients.
It just does these three triangles and all these rules to contract them.
It's only ever looking at the three points in order in the design.
So if you have a function that's not differentiable, you can still use [?
Effman ?]
search.
But if you have something that's not differentiable and you try to use it [?
Effman ?]
unc, MATLAB's going to be trying to compute gradients-- not second derivatives but first derivatives-- and could get itself into trouble.
But the really nice thing with these MATLAB functions is that you basically supply a function, that given x computes J.
And you supply an initial guess, an initial point in the landscape, and that's actually the only thing you have to give it.
If you don't want to give it anything more, MATLAB will take care of everything else for you.
You can give it just that black box function, given x computes J, and an initial guess.
If you know a little bit more about what's going on, you can also choose to manage options.
So you can do things like turn on see more information about the iteration.
You can play with tolerances.
You can even specify exactly which methods you want to use.
So here's the documentation for [?
Effman ?]
unc.
So you see there, there's just the basic call, giving it a function and an initial guess.
So you see all these different options in here.
So you can have the option to supply a gradient.
So if you had a function that was analytic and you could differentiate it, it would be a really good idea to give that gradient to MATLAB as well as giving it the function.
Or if you would use automatic differentiation and come up with a code that computed gradients, and you're able to specify that as an additional option.
And last thing I just want to show you is the methods.
Check it in here.
I'm not sure if it's telling us what the methods are.
[INAUDIBLE] and have a look, see what method it is.
So again, I didn't talk about them, but there's a class of methods called quasi Newton methods, which in a way are kind of like the best of both worlds between the first order and the second order methods.
So if you remember here back in the Newton's method, [INAUDIBLE] design space to be the inverse of the Hessian times the gradient.
But then we said that [?
computation ?]
was really too expensive.
So what a quasi Newton method does is that it basically introduces an approximation for the Hessian.
That's this A thing here.
And it builds up this approximation using the gradients that it's computing anyway.
So don't worry about the details of what is on here, because it's a little bit beyond what I want you guys to be comfortable with.
But more or less every time we compute a gradient, we can also use differences between gradients to get approximations to Hessians.
And I'm pretty sure that's the method that's sitting underneath here in
[INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE]
So we're not guaranteed because it depends on the problem.
So if you have a problem where there's a direction where the curvature is flat, then you're going to have a singular Hessian.
And then you just have to be careful
[INAUDIBLE] Yeah.
If J were linear in any design variable, when you differentiate twice you would get 0.
And you have to be very careful if you have [INAUDIBLE] inverse of the Hessian.
But there are ways you can handle that, more advanced optimizations.
OK.
So I'm not going to talk about constrained methods, but I did want to just show you briefly, and show you also in MATLAB how constrained methods work.
So there's something called the [?
buns ?]
function.
Actually the optimization community loves these little simple problems.
Remember I talked about [?
Rosenbrot ?]
the other day and called it the banana function.
So there's another one called the [?
buns ?]
function, which is actually a constrained problem.
So it turns out this is the objective, which I know doesn't really mean anything, but the difference of this one is it's got constraints.
And so if we look at the visualization, here's the design space, design variables x1 and x2.
And now the contours, they're called contours of the objective function.
And then the constraints are saying the solution has to lie above this one, has to lie to the left of this one, and has to lie above this constraint.
So again, when I was talking about the landscape, remember I said think of the constraints as being like the keep-out regions, telling you where in the design space you can be.
And so in this case our possible design region is this thing here traced out by the constraints.
So it turns out this problem has two local solutions.
One is also the global solution.
If we try and minimize, the global solution is actually up here in the corner of the design space, which you can see from the colors.
It's the bluest color.
But there's also a local optimum that sits here on this constraint down here at about 50, 20.
And I should say that there are bounds on the design variables, which is why this global solution sits up here, because the design variables are at their bounds.
So that's a simple constrained problem.
And I just want to show you running the algorithm.
So [?
fman ?]
[?
con ?]
is the MATLAB function that does constrained optimization.
And again, it's really super simple to use.
At a minimum, all you have to do is give it a function that given x returns J of x, give it an initial guess to start with, and give it a function that given x returns the constraints, the g of x or the h of x if you have them.
So given x, return J of x, return g of x and h of x, and an initial guess.
And there are lots more options that you can set if you want to.
And you can look at lots more things, but in the simplest implementation-- and then MATLAB will take care of finite differencing for you to do the gradient, the step size, everything.
Everything you have to worry about.
But of course, again, if you have an efficient way of computing gradients, you have the option to specify the gradients and make things run a lot more reliably and a lot more efficiently.
So just go back up here, so this one's actually using-- this is a quasi Newton method.
There's also a Newton method that [INAUDIBLE].
I think there are actually three levels of optimization in [?
fman ?]
[?
con ?
], two different kinds of quasi Newton method, and there's a Newton method that's in there.
In order to run the Newton method, you have to actually supply a gradient for it, so that it doesn't do all that finite differencing thing for the Hessian.
And what we can look at, what we're looking at here are different initial conditions.
So here's that [?
buns ?]
function that I was showing you.
And in this particular case, we initialize here, and what's being ported are the steps that the optimizer's taking in the design space.
I'm not going to talk about the algorithms at all, but just to get a sense that when you have constraints, things are much more difficult, because it's not just about going downhill.
But here we could just go downhill, and you can see more or less this looks kind of like a steepest descent direction.
But then when you get here and you're right kind of actually on the wrong side of the constraint, you can't just go downhill, because you've got to worry about the constraint.
And you can see what it's doing is it's kind of following the constraint and skipping around.
So things get a lot harder when you have constraints, because you have to worry about descent, but also about satisfying the constraints.
But again, if you take optimization class you'll learn lots of stuff there
[INAUDIBLE]
Yeah.
So it depends on the algorithm.
Some algorithms will allow it to violate the constraints a little bit along the way.
Some will not.
And there are all lots of different ways of handling constraints.
There are things called penalty methods.
There are things called barrier methods.
There are interior point-- the barrier methods related to interior point methods that force things to stay away inside the constraints.
It just depends on what algorithm you use.
But that sort of relates to the next part that I want to show you, which is now when you think about convergence, so here is the objective function value.
This is the iteration.
So you can see the objective is coming down, but you don't just want to worry about what's happening now with the objective.
You also have to look at the constraint violation.
And here it [?
skips ?]
[?
forward ?].
It looks like we found a pretty good design, but actually the constraint is violated.
So this would be like an aircraft wing that exceeds the maximum stress constraint, which would be no good.
And so that's why we have to keep going.
And actually you can see a little bit of an increase here in the objective to get the constraint violation satisfied.
And I'll say that when we try to apply these kinds of methods to realistic aircraft design problems that have lots of variables, they have even lots and lots of constraints.
And satisfying constraints can be sometimes just about the most challenging thing for the optimizer.
So it can often find good solutions, but then it turns out that they don't necessarily satisfy the constraints.
You see another one here.
In this case, we started with an infeasible design, one that violated these constraints, or violated this constraint.
So you see first of all the optimizer tries to get it back to the feasible region.
It tries to make sure the constraint is satisfied.
And then it starts searching.
And actually most of these designs are infeasible.
It's only at the end that we come here to the local optimum.
And so again, there's the function value.
And you can see we start off violating the constraint.
We satisfy it, and then we go back outside the constraint, and then eventually come back to it at the end.
And then the last one starts again over here, where two constraints are violated.
And you can see even though two constraints are violated, this one's actually able to get to that solution pretty easily.
One thing you notice is none of the ones that I started actually found the global optimum.
They all converge to the local optimum.
And again, if you like to think more in 3D, remember that landscape where there was the big mountain and the two smaller peaks.
In this case, we started here, we started here, and we started here, and they all ended up over here.
Probably we would have to start on the other side of this ridge here in order to get up into the global optimum.
OK.
So I know you don't really know very much about constraint optimization, but you could all be [?
dangerous ?]
[?
utilize ?]
for the next year if you wanted to be.
It's actually a really powerful thing if you're trying to do a design problem, or you have simulation code and you want to explore what's going on with the parameters.
The toolbox in MATLAB is really good.
And then if you say for grad school, you can take the graduate class that I teach that goes into this stuff in much, much more detail
Is there anything MATLAB, like other--
Yeah.
Absolutely.
So there's all kind of optimization toolboxes.
I mentioned Professor Johnson in [INAUDIBLE], where he's implemented a lot of some of the less mainstream optimization methods.
That's nice.
And then there's all kinds of other toolboxes, things like CPLEX that a lot of optimization people use.
What are some of the big ones?
I would say though MATLAB has really started to sort of dominate the market in a way because MATLAB's optimization toolbox has become really good, especially in the last 10 years.
And I think the gradient pace, they have the state of the art, the Newton methods with all the bells and whistles on them.
And then there are also a lot of user-contributed algorithms.
I'll show you.
Let me see if I can run it, something that I don't approve of
[INAUDIBLE]
That's right.
Yeah.
I gave you the--
[INAUDIBLE]
I gave you the link to that in the last lecture notes, to Professor Johnson's website.
And actually since you said Python, also pyOpt.
Right now pyOpt is starting to create all the optimization algorithms in Python
[INAUDIBLE]
What about [?
Julian ?]?
[INAUDIBLE]
Is that Professor Edelman's stuff?
[INAUDIBLE]
Oh, once you graduate?
But that's part of their model is for you to convince your employers to-- but that's why you stay in academia is you get perpetual very cheap MATLAB.
It's one of the great perks of being an academic
[INAUDIBLE]
So I want to show you something again of which I definitely don't approve, but again this triggered in my mind because I said user-contributed toolboxes.
There's a whole class of optimization-- I'm going to use the word "optimization" in quotes-- methods that don't use gradient, that don't really have any convergence theory.
A couple of them have some theories associated with them.
They're typically called heuristic algorithms because they use rules to search the design space.
And one of the most popular in aerospace engineering is something called a genetic algorithm, which basically uses all the rules of natural selection, and mimics the rules of natural selection to optimize the design space.
So this is the function that we were looking at on Wednesday.
And remember, we did a gradient-based method, or we did the [?
naldamete ?]
simplex, where we had a little asterisk, but with our initial guess, and then the asterisk marched and found the top of the hill, or got to one of the tops of the hills depending on where it started.
So a genetic algorithm is a population-based method, which means you don't have one design on each iteration, but you have many.
And I don't know how many there are in this population, maybe 20 or 30.
So there are 20 or 30 initial guesses, and that's the population.
And when I set it running, what you're going to see is that each iteration, it's evolving this population.
So if there are, let's say there are 30 there, on the next iteration it's going to be 30 more and then 30 more and 30 more, and the way it's evolving it's taking designs and then the parents and they're having children, then it's figuring out which children to keep and which ones to not keep.
And it's introducing a mutation that causes designs to jump around.
So there's a whole sort of thing.
And Professor [?
Divic ?]
knows a lot about genetic algorithms.
So if you're interested, you could ask him.
But let me set it running, and you'll see you'll see what I mean
[INAUDIBLE]
Yeah, the heuristic is all the things about-- you take designs and you have to encode them with chromosomes.
And then there are heuristics about the different mating strategies and then the mutations.
So there's a lot of-- and at the end of the day, you don't have guarantees about optimality.
But if you have a problem where you can compute gradients, maybe where the models are not differentiable, and you just kind of want to spray points everywhere and [?
work ?]
as well as you can, maybe something like this is OK.
I wouldn't call it optimization
[INAUDIBLE]
Yeah, if you want to avoid local-- but I think you avoid local minima by not getting to any minimum at all.
They work great for when you have 2D.
But you see the idea how dramatically different it is.
There's simulated annealing which does have some theory associated with it.
There's particle swarm optimization.
There's ant colonies.
There's lots of analogies with nature.
So there's this whole other class of optimization algorithms which can be kind of fun.
OK. And so if you wanted to find those, you could probably go find other toolboxes as well.
OK, so I think I'll skip over the 1D search.
It's not all that important.
But I want to spend the last little bit just talking about surrogate modeling and reduced [INAUDIBLE] models.
So maybe you have a sense that when we run an optimization algorithm, we have to call the code lots of times.
Every time we want to evaluate the objective or the constraints, and any time we want to get a gradient, we may have to call it n times.
Also, you've seen in the Monte Carlo simulation that you had to call the code 22,000 times when you were running the simulations for each ball, each pitch that you were analyzing.
So it turns out that when you have CFD models or finite element models or realistic models of aircraft or spacecraft or whatever it is you're modeling, it just gets too expensive.
And so we use what is called surrogate model.
So the word "surrogate" means that we're going to replace the real model with an approximation.
And the idea is that the approximation should be something that's much cheaper than the model that we're trying to optimize, but hopefully it's good enough so that we can make good design decisions and find good designs, and then maybe go back and analyze them with our more sophisticated model.
So I like to think of surrogate models in these three different categories.
There are data fit models like regression models, and we'll talk a little bit more about these ones.
There are simplified physics models, which actually we use in engineering all the time, that maybe we want to design according to a Navier-Stokes equation of, in this case, of a supersonic business jet.
But we could also use like a paddle method, or even just a simple empirical very high-level model of the aircraft so we can simplify the physics.
And then there are things that are called projection-based reduced model, which is basically a mathematical way of coming up with simpler models.
This little diagram here is supposed to represent an x dot equal x plus b u.
If you guys didn't see state space systems [INAUDIBLE] anymore, right?
No.
So mathematical ways for reducing systems that I won't talk about.
But I do lots of research in that third category.
So if we just want to think a little bit about the data fit category, so the idea is going to be that we're going to sample our simulation at some number of design points.
So we might use one of the design of experiments method that we talked about last week to somehow select some points.
So we're going to run our expensive model at a bunch of points, and then we're going to fit a model using that sample information.
We could try to fit a model everywhere, or we might want to do more of a local fit.
And then you could also think about updating the model when you get new points.
But I want to talk specifically about response surface modeling, and see how one would come up with a surrogate model using a response surface.
Because again, that's where we'll use some of the mathematical techniques that you guys should have seen before.
So we skipped over number three.
We didn't talk about the 1D search.
And number four was really just a brief intro to what is a surrogate model, and what are the different times.
So we're at number five, on a response surface model.
OK, so the setup is that we're going to have-- let's just draw the [?
block ?]
diagram.
So this is our expensive model where we can supply x, and out comes J of x.
Therefore with a constraint problem, it would also be g and h coming out of there.
So that's our expensive model.
And basically what we want to do is come up with a surrogate model said that when we supply x, what's going to come out will be some J hat, where this guy is hopefully a good approximation of J.
And one simple but pretty powerful and commonly used way to do that is with what's called a response surface model.
So response surface model, or RSM.
So if we wanted to do just a first order RSM, then what we're going to say is that this J hat, which is a function of x, is just going to be some constant, let's call it a0, plus the sum from i equal 1 to n of some ai times xi.
In other words, it's just a linear model.
It's a constant plus a constant a1 times x1, plus a2 times x2.
So it's going to replace the expensive model with a linear approximation.
That would be a first order response surface model.
Turns out this is not such a good idea, and we'll look at it graphically in a second and you'll see why.
So more often than not, people used a second order or a quadratic response surface.
And it's the same idea that the approximate model they had is going to be, again, a constant, and then it's going to be the linear term, so the sum from i equal 1 to n of ai xi's, and then it's going to be all the quadratic terms, so i equals 1 to n, j equals 1 to n of b i J x i x J.
And actually maybe this should be J equal i to n. So again, we're just replacing our J of x with a model, and in this case, second order case, we're saying the model is quadratic.
It can have a constant term, it can linear terms, and then it can have all these quadratic terms in the xi, xj, xi squared, x1 squared, x1, x2
[INAUDIBLE]
Well, because we don't need 1, 2 and 2, 1.
Yeah.
I mean, you can write 1 there if you wanted to, but because x1, x2 is the same as x2, x1, we would just lump them.
[INAUDIBLE] OK, so here we've got that the ai's are unknown coefficients.
And here we have the ai's and the bij's are unknown coefficients.
So we've said that we want to approximate our model with a linear or a quadratic function.
Now the question is how do we come up with coefficients for that approximation.
And that's where this sampling comes into play.
What we're going to do is we're going to generate samples using our favorite design of experiments method, or maybe we're just going to run Monte Carlo and sample randomly.
We're going to generate a bunch of samples, and then we're going to use least squares fitting to try to estimate those coefficients and come up with the response surface model.
The next step is going to be how to determine the ai and the bij coefficients.
So let's say that we have generated n samples, and each sample is going to have-- we'll call it j-- there's going to be a pair consisting of the design variables that we sampled, the value of x that we sampled, and that's again the n by 1 vector, and then the corresponding value of the objective.
So we'll have x1, j1, we'll have x2, j2, all the way up to xn, jn.
So I ran my expensive model n times.
So I just collected all the data.
I collected all the conditions that I ran it at, and I collected all the results that I got out of it.
Is that clear?
Yeah?
Let's think about this guy here, so just a first order [INAUDIBLE] J of x equal a0 plus the sum from i equal 1 to n of ai xi.
How many pieces of data do we need at a minimum?
How many unknown coefficients are there in the model?
n plus 1.
Little n plus 1.
So we need big N to be at least little n plus 1.
But in m what we would do is we would generate more than little n plus 1, and then we would do a regression.
We would do a least squares fit.
You guys did least squares fir somewhere, yes?
1806?
Did you do it anywhere else?
[INAUDIBLE]
[INAUDIBLE]
High school.
Did everybody here remember their high school regression?
No?
[INAUDIBLE]
You can do them on your calculator?
MATLAB backslash, like the most amazing operator?
OK.
So, well, let's think about what it would look like.
So I always like to think of the least squares fitting from just the matrix system.
Let me draw it over here so there's plenty of room.
This is a matrix system, and if we have exactly the right number of data points, the matrix will be square.
And if we have more data points than unknowns, the matrix will have more rows than columns, and it will be an overdetermined system, and you would do a least square solution.
So we'll set up a matrix system.
First of all, the unknowns, so the things we're solving for are these [INAUDIBLE] a0, a1, a2, down to a n, little n. Those are the unknowns.
And then each row in the matrix system, this is going to correspond one of the sample points.
So the first sample point says that x-- when I feed it in x superscript one into the model, I get out J1.
So here's J1, and here's the expansion.
And when I feed in x1, so the first thing I do is I get 1 times a0 plus a1 times x1 in the first sample point.
So this guy here is going to be x1,1 where the superscript is the sample number, and the subscript is the design variable number.
Yup.
And that's x2,1, xn,1.
Is everybody with me?
I just wrote down the equation 1 times a0 plus the first component of sample 1 times a1 plus the second component of sample 1 times a2, plus, plus, plus the nth component of sample a1 times a n equals to J1.
[INAUDIBLE] exactly does this equation apply to the first [?
here ?
], and I could write down so that this is the first sample, which we called x superscript 1.
So then I would do the second sample, and it would just look like this.
x2,1, x2,2, xn,2, and that's going to be equal to J2.
And then I would keep going.
So I'm always going to have 1's in this column, because that 1 is always going to multiply a0.
Down here eventually I'm going to have x, first component of the nth sample, and here I'll have x, nth component of the big Nth sample, and everything in between.
Yup.
So this is a big N by little n plus 1 matrix.
This is a little n plus 1 vector, and this is a big N vector.
Yeah.
And if we have exactly little n plus 1 samples, again, this matrix would be square, and we would just invert it, and it would give us the coefficients.
If we have more samples than we have unknowns, then again it's just a least squares solution, which you can do with matrix with MATLAB's backslash, or you can remember the formula involving ax equals b, and it's overdetermined, then you do a transposed ax equals a transposed b, and then take the inverse of this, and take it to the lefthand side.
[INAUDIBLE] OK, and if we're doing a quadratic response surface, so if I was setting J hat of x is this plus this plus the term, the b ij xi xj, how would the system change?
What would I have to do?
Would there be more unknowns?
Yeah, so there would be the b1,1, b1,2, b1,3, all the b's.
So more unknown means more columns in the matrix.
So what would be-- if I put b1,1 here, what entry would go in here?
[INAUDIBLE]
X1,1 squared, because in the expansion, it's b1,1 times x1 squared.
And we would be evaluating that for the first sample.
So we'd end up with x1,1 squared x1,1 times x2,1.
You'd end up with all the quadratic terms matched out.
So it looks just the same.
The regression's not really-- sometimes it seems more complicated, but I think if you just write out the matrix system and think about it as an overdetermined set of equations, it becomes clearer.
And then you can also see we're going to have more unknowns.
We're going to add more columns.
If we want it to stay overdetermined, or at least determined, we would have to get more samples, which is what we talked about.
So let me show you how that works.
Is this clear?
Is this notation clear?
[INAUDIBLE]
Say it a bit louder
[INAUDIBLE]
Yeah, that's right.
And in fact, using a first order response surface is a really bad idea, and we'll see it graphically.
So it turns out linear approximations in optimization are not a good idea.
Maybe you can think about why that would be with a linear approximation.
You think about what we're doing, we've got this landscape, we build a linear approximation and say "optimize."
Find a new point, and build a new linear approximation.
But where is the linear approximation always going to send you?
It's always going to send you far away in some direction, kind of by definition.
So I build a linear approximation here.
If it's sloping this way, it's going to take you all the way to that side of the design space.
If it's sloping the other way, so it's always going to send you [INAUDIBLE].
So you could combine it with a strategy that-- that's what a trust region does.
But actually it turns out that quadratic approximation, because they embed the information of curvature, which we know is really important for optimality conditions, are far, far better choices and more commonly used.
Put up on [?
stellar ?
], it goes through what we just wrote on the board.
But Let me just get rid of that horrible genetic algorithm and get to the right directory.
So I have a code here that is going to build some response surface.
It's the same demo.
It's this peak problem, the landscape problem.
So it's going to grab some samples.
Maybe I'll just do it and we can look at the code.
And we're going to build some response surfaces so you can-- so in this first example, again, here's our landscape, variable x1, variable x2, objective function.
We happen to generate these [AUDIO OUT].
Maybe they were random.
Maybe we had some kind of design of experiments method to do it.
And three points, two dimensions, what can we fit?
What kind of a response surface can we fit?
Linear one, first order?
n plus 1?
We don't have enough to fit a quadratic.
So there is-- in effect, it's determined.
It's three unknowns, constant, slope in the x direction, slope in the x2 direction.
So there's the response surface.
It's clean.
It goes exactly through those three points.
And you see kind of the problem that [?
Tyrone ?]
was asking about, which is if we use this optimization, it's going to send us-- and we maximize, it's going to send us over into the corner of the design space.
Clearly it's not a good approximation.
So we could generate more points.
So now we have these six points.
And now that we have six points, we could still set a linear model, but now it would be an overdetermined system.
The matrix has got six rows, because there are six sample points.
And it's got three columns because there are still three degrees of freedom, the constant, the slope in the x1 direction, and the slope the x2 direction.
So now you still have a linear model, but you can see that it doesn't go through the points.
And in fact, it's the least squares is like the line of best fit, but we're in multiple dimensions.
So it's like the plane of best fit.
Three of the points are above, and the other three want to be sitting somewhere underneath here.
And maybe if you've taken 1806-- they're sitting underneath here.
Maybe if you've taken 1806, you know that this solution is going to be the one that minimizes the sum of the squares of the two norm distance.
So it is the line of best fit in that optimal sense.
That's what the least squares fit is.
OK.
So now we have six points, which turns out to be n times n plus 1 over 2, which is how many points we need to fit a quadratic model.
So we have six points.
And you think about the quadratic model, what do you have?
You have the constant, a0, you have the linear terms, a1 and a2, and then you have of b1,1, [INAUDIBLE] squared.
You have b2,2 for x2 squared, and then you have b1,2 for x1 x2.
You have six degrees of freedom to fit a quadratic model in 2D.
I wrote out the expansion.
So six points, six degrees of freedom.
Again, our matrix system would be 6 by 6, so it would be uniquely determined.
And this is what the quadratic response surface looks like.
And as you can see, it's going exactly through those points.
OK, so now we're starting to get a little bit of curvature.
With a quadratic model, we can only ever have one minimum or one maximum for the models that look like this, whereas we know that the design space we're trying to approximate has got multiple blobs.
But a quadratic model;s not ever going to capture these multiple hills in this case.
I mentioned really briefly on one of the slides that we could think about adapting the model.
So maybe as we're going through the optimization, maybe we can make a genetic algorithm, even though we're not supposed to.
We're figuring out that this is the interesting region in the design space, so rather than having our points spread out everywhere, we might be saying let's kind of only on the points that have got good performance that we've found so far.
So see all these points around here-- [INAUDIBLE] this one does too because it's on the side of that other little hill that sits over here.
So if you build a response surface using those points, maybe you can see now that the model's really pretty terrible over here, but it's actually starting to be a pretty good approximation of what's going on locally around those points.
And that's a common strategy would be to build these local quadratic models in regions where you've sampled and use them to make progress, and then to keep updating them.
And even though a quadratic model seems really simple for a real problem, that strategy of adapting points and building local models [AUDIO OUT] updating the model as we go is actually a really, really powerful one.
And of course if we had more points-- I don't think I've got one that's got more points.
If you had more than six points, than the quadratic model again would be the quadratic model of this fit, where it wouldn't go through all the points, but it would cut through them on average equally.
OK, questions?
So that's a pretty, I think, neat use of regression that's actually very powerful and is used a lot in practice.
So that's one of the things that you need to be able to do.
So have a basic understanding of how a design problem can be posed as an optimization problem.
What do I mean when I say constraint?
What I mean when I say objective function?
Have a basic understanding of the steps, and the gradient-based constrained optimization algorithms are looking for the direction and how the different methods do that.
Be able to estimate a gradient, a first order or second order derivative using finite differences.
You could already do that.
You just did it in the context of PDEs.
Now we're talking about design problems.
And understand how you could construct a polynomial response surface, either linear or quadratic, using least squares regression.
Actually, we didn't talk about the quality of fit.
Do you guys remember what's the quality of fit metric for regression models?
r squared.
Yeah, so r squared tells you how well your surface approximates your data points.
But you have to be careful because it doesn't tell you anything about how good it might be in regions where you didn't sample.
All right.
Final questions?
You guys feel like you've learned enough in this class already?
No?
OK.
So on Wednesday, I will do a review of finite-- I think I'm going to focus only on finite element methods and then the probabilistic and optimization function.
If you have specific questions or topics that you want me to cover in more detail than others, then either bring them to class or you can e-mail me ahead of time so that I can prepare a little bit more.
But that will be a good chance to ask questions about things that are confusing before the final.
All right.
See everybody on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, now we're recording.
I think about half of my lectures are up on Stella, because about a quarter, I forgot to push record, and another quarter, something went astray with the microphone, and all you hear is like, blegh-egh-egh.
But there's about half of them up there, the most important ones.
OK, so the residual is the remainder when an approximate solution is substituted into the governing PDE.
And remember, what's so powerful about the residual is that we can take an approximate solution, we can see how close the PDE is to being satisfied, which gives us some idea of how good that solution is, but we don't actually need to know the true solution, right?
We don't need to know the exact solution.
So this is different to an error.
But it is some measure of how far our PDE is from being satisfied.
And so it turns out that this is a really useful thing when we're looking for finding good approximate solutions.
And the example that we worked with the most was the diffusion or heat equation, which we wrote as K dT / dx, all differentiated with respect to x, equal to minus q, where q is the heat source.
So I can just write the x's down here.
This is notation for differentiation with respect to x.
So if that is our governing equation, and we have an approximate solution that we call T tilde of x, then the residual would be [?
r.
?]
It's a function of that approximate solution.
It's also a function of x.
And it would be defined as bring everything over to the left-hand side.
So K, then substitute in the approximate solution.
So d T tilde / dx, all differentiated, plus q.
So that would be the definition of the residual.
And if the approximate solution were exact-- in other words, if we put not T tilde, but T in here, the residual would be 0.
And so remember, we define the residuals.
And then we said, let's say that this approximate solution, T tilde, is discretized and represented with n degrees of freedom.
And there are different ways.
We could use a Fourier-type analysis and assume that this thing is a linear combination of sines and cosines, or when we get to finite element, we could assume it's linear combinations of these special hat functions.
But whatever it is, if we have n degrees of freedom representing our approximate solution, then we need n conditions to be able to solve for that approximate solution.
And we talked about two different ways to get to that.
One was the collocation method, where, remember, we set the residual to be 0 at the collocation points.
So required the residual to be 0 at, let's say, capital N points, where this approximate solution here is represented with n degrees of freedom.
So that was the collocation method, pinning the residual down to be 0 at n points, which gave us the n conditions that let us solve for the n degrees of freedom.
Or the method of weighted residuals, which said, let's instead define the j-th weighted residual, where what we do is we take the residual, we weight it by another basis function, and then we integrate over the domain.
So that means the j-th weighted residual-- still a function of our approximate solution T-- is some weighting function, Wj, times our residual, integrated over [?
our ?]
1D domain.
So define the j-th weighted residual in this way.
That would be the first step.
And then the second step would be to require that n weighted residuals now are equal to 0.
You guys remember all of this?
Yeah?
And then remember, the key is now, OK, what are these weighting functions?
And you remember, we talked specifically about a kind of weighting called Galerkin weighting, which says, choose the weighting functions to be the same basis vectors, the same functions that you're using to approximate the T tilde and its n degrees of freedom.
And so that's what's key about a Galerkin method of weighted residuals, is that the weighting functions are the same as the functions used to approximate [?
as ?]
a solution.
And again, then requiring n of those weighted residuals to be equal to 0 gave us, remember, the n-by-n system, where each row in the system was the equation that said Rj equals 0.
And each column in the matrix system corresponded to the unknown degree of freedom describing our approximate solution, T tilde.
OK, so that's the method of weighted residuals.
But actually, I think once you understand that, you have, more or less, all the building blocks to then move to finite element.
And so what's the jump in terms of finite element is that finite element uses a very specific kind of basis function that represents a solution in this special way that then turns out to have really nice properties.
And there are a variety of basis functions that you can use, depending on what kind of an approximation you want, whether you want a linear approximation in each element, or quadratic approximation, or even higher order.
And what did we talk about?
So we talked about finite element basis functions in 1D and 2D.
And we talked mostly about the nodal basis.
And I think we talked even mostly about the linear nodal basis.
I'm not sure.
Did Vikram talk about the quadratic basis for 1D in the lecture that he gave?
No.
So I think maybe you saw only the linear nodal basis.
So e.g., if we look at the linear nodal basis in 1D, remember that these hat functions, they have the property that they are 0 at all nodes except their own node, where they're equal to 1.
So this is a plot of [?
bi.
?]
And it's got a value of 1 at node xi.
It goes to 0 at node xi plus 1.
And then it stays 0 for all nodes to the right of xi plus 1.
It also goes to 0 at xi minus 1 and stays to 0.
So the basis function is actually defined on the whole domain at 0, and then it's up to 1, down to 0, and then stays at 0 for the rest.
And then, just to run by the finite element notation, we talked about element i as being the element-- the piece of the x-axis that lies between xi and xi plus 1.
The element to the left is i minus 1.
The element to the right is element i plus 1.
And what's key about these basis functions is that they're 0 in most of the domain.
The only place where Ci is non-zero is on element i minus 1 and element i.
And we use that fact when we get all those integrals that come out of the definition of the weighted residual, that a whole bunch of integrals go to 0, right?
Because you're multiplying things together that are 0 on big parts of the domain.
All right.
Questions about [?
your ?]
[?
advisor ?]
remembering this?
Yes.
Any questions about either of those two?
So I will next just remind you about integration in the reference element.
What's the idea?
Well, first of all, in 1D we have things in the physical x domain.
So we have xj, say, and xj plus 1.
And the idea is that the elements could be all those different [?
links, ?]
depending on how we decided to make the bridge, based on what's going on with the physics.
So the j-th element might be a lot bigger than the j minus 1 element, or smaller, or however we'd like to do it.
And what we'd like to do is map to the [?
xc ?]
space where the element goes from minus 1 to 1.
So every element in the x space, element j going from xj to xj plus 1, could be mapped to the reference element in the [?
xc ?]
space that goes from minus 1 to 1.
And the mapping that does that is x-- as a function of [?
xc ?]
is xj plus 1/2, 1 plus [?
xc ?]
times xj plus 1 minus xj.
And you should be able to see that when I set [?
xc ?]
equal to minus 1, I get xj.
And when I set [?
xc ?]
equal to plus 1, these terms cancel out and I get xj plus 1.
And then everything in between is just the linear mapping.
The middle point here is the middle point here, and so on.
So that's what it looks like in 1D.
In 2D, we need to map to the reference element, which is a standard triangle.
So in 2D, we have a generic element that might look something like that, where it's got coordinates.
Now I'll say x1,y1, x2,y2, and x3,y3.
Those are the xy coordinates of the three nodes in this element.
And I want to map it to the reference element, which again, is in [?
xc ?]
space, which, remember, looks like this right triangle with this node at 0,0, this guy at 1,0, and this guy here at 0,1.
This is coordinate direction [?
xc1 ?]
and this is coordinate direction [?
xc2.
?]
And again, we can write down the generic mapping just like we did it here.
We now have to map x and y, and they are given by x1-- the mapping at x1,y1 plus a matrix that looks like x2 minus x1, x3 minus x1, y2 minus y1, y3 minus y1, times [?
xc1 ?]
[?
xc2.
?]
Just in the same way as here, we can see that varying xc from minus 1 to 1 gave back xj and xj plus 1.
You can see the same thing here.
If you vary [?
xc1 ?]
and [?
xc2 ?]
from the 0,0 point, clearly 0,0 is going to give you back x1,y1.
1,0 should give you the x2,y2, and 0,1 should give you the x3,y3.
So you can define these transformations.
And this is generic, right?
I can now give any element, any triangle, and you can always get me to a reference triangle.
And what that means is that, now, when you do the integrations that show up in the finite element method-- so an integral over, let's say, a domain, over element K, some function, function of x, these are the kind of integrals that show up in the 2D finite element method.
We can now do an integration over the reference elemental, [INAUDIBLE] omega [?
xc.
?]
We're now writing f as a function of x of [?
xc.
?]
And what we need to do is to introduce the Jacobian of the mapping so that we can now integrate with respect to the reference element.
And the Jacobian here, this is just the determinant of this matrix.
Can't remember, but I think you guys must have done this with Vikram, because I think I was away [INAUDIBLE] yeah?
So we end up with this j, and this j here is just the determinant of this x2 minus x1, x3 minus x1, y2 minus y1, y3 minus y1.
So basically, this accounts for the change in area, because you're integrating over this guy That gives you the reference element.
Sometimes I think it seems a little bit messy when you see it written down.
But the reality is it makes you code really clean, right?
Because you just introduced these mappings, and then every single integration in the code takes place over the reference element.
You just go account for the mappings from whichever element you happen to be in through the determinant of this matrix.
Is that good?
OK, so somewhat related to that, because we'll still talking about integration, was Gaussian quadrature.
A really simple idea, but quite powerful, and you had a chance to practice that on the last homework.
So remember, for Guassian quadrature, for the simple examples that we worked through in class, we could do the integrals analytically, because we were integrating constants, or linear functions.
So we were integrating things like e to the x, or xc to the x.
But in practice, for real problems, you may end up with integrals that you can't do analytically, either in the stiffness matrix or in the right-hand side system, on the system.
So if you can't do the integrals analytically, then you resort to Gaussian quadrature.
And remember that Gaussian quadrature in 1D looks like this.
If we're trying to integrate some function, g of xi, over the domain minus 1 to 1, we approximate that as the sum of the function evaluated at quadrature points xi i weighted by weighting coefficient alpha i, and it's the sum from i equals 1 to nq.
So nq here is the number of quadrature points.
Number of quadrature points.
[?
xci ?]
here is the i-th quadrature point, and alpha i here is the i-th quadrature weighting.
Do you remember-- where do the rules come from?
How do we figure out the [?
xci ?]
[?
alpha i ?]
[?
pairs?
?]
Perfect integration of a polynomial.
So if, for example, we wanted to just use one quadrature point, then we would say, with one point we can perfectly integrate a linear function, right?
Because if I take any linear function, and what I'm interested in is the area under the function, I can get that perfectly by just evaluating its midpoint, and the area of the rectangle is the same area of the-- what is this thing called?
Trapezoid
Trapezoid, yeah.
So with one quadrature point, I can typically integrate a linear function.
And I would do that by setting alpha 1 equal to 2 and xc 1 equal to 0.
So again, midpoint and the area.
If I am willing to have two quadrature points, it turns out that I can integrate linear functions exactly, I can integrate quadratics exactly, and I can actually also integrate cubics exactly.
And it turns out that the weights I should choose are alpha 2 equals 1 and quadrature points at plus or minus 1 over root 3.
And again, those points are chosen to give perfect integration of polynomials.
And of course, when we apply those quadrature schemes to non-polynomial functions, we introduce an approximation, but typically that's something that we're willing to tolerate so that we use not too many quadrature points in the scheme.
OK, questions about reference elements or Guassian quadrature?
Is that good?
Yeah, Kevin
[INAUDIBLE] just slightly [INAUDIBLE] quadrature.
This is on [INAUDIBLE] assuming there's no quadrature that can approximate [INAUDIBLE]
Yeah.
So the question is that-- so it's not only for polynomials, but these are the points that are derived by considering exact integration of polynomials.
And yes, absolutely, there are other famous quadrature schemes.
There are Chebyshev points, and there are all kinds of points that are derived in different ways.
Some of them are derived by looking at sinusoidal functions.
Yeah, absolutely.
Yeah, it's a whole study of math, quadrature points and quadrature schemes.
It's actually kind of neat stuff.
Yeah.
All right, so let's move on and talk about boundary conditions, and then we'll close by forming the system.
And then, that will be finite element in a nutshell.
Number five is boundary conditions.
And the main types of boundary conditions that we discussed were Dirichlet, where, remember, for a Dirichlet condition, you specify-- what do we specify?
Yeah, the value of the unknown.
The solution at a point.
And for a Dirichlet condition, we are going to replace the weighted residual equation.
So we're going to form the system, we're going to look at forming the system in just a second.
But we already said that setting each weighted residual equal to 0 gives us one row in our matrix system, right?
One row corresponds to the equation [?
Rj ?]
equals 0.
So if there's a Dirichlet condition in a node-- that's usually at the boundary, maybe at the first one-- we're going to come in and we're going to strike out that weighted residual equation that we got from multiplying by [?
c1 ?]
and getting the weighted residual to 0.
We're going to strike that out and replace it with an equation that might look like 10000 equals to whatever the Dirichlet condition is.
Then we're going to fit the value.
And we have to be a little bit careful and make sure that we're setting truly the value of the unknown.
If we're using the nodal basis-- I guess I didn't state explicitly when I drew the nodal basis, but what's so special about the nodal basis?
What does it mean for the unknowns?
Yeah, that's right, that the unknown coefficients that you solve for turn out to be the values of the approximate solution at the node.
And that's because, by construction, those basis functions are 1 at their own node and 0 everywhere else.
So if you're using the nodal basis, the unknown coefficients correspond to the value of the approximate solution at the node.
Which means that for a Dirichlet condition, you can directly manipulate that coefficient.
And then the other kind of boundary condition that we talked about was a Neumann condition, where now you specify the flux.
And what we see is that when we do the weighted residual, when you integrate by parts, you get a term out that is a boundary term that's where the flux shows up exactly.
And so a Neumann condition contributes to the weighted residual.
You'll see its place exactly in the derivation of your weighted residual, where you can go in-- if the Neumann condition is flux equal to 0, it basically strikes out that term.
If it's a non-zero, a non-homogenous Neumann condition, then you could go in and put that contribution as the flux.
So in this case, you leave the weighted residual equation, but you add the contribution.
So two different boundary conditions, two different implementations.
And so what this means is it's an additional contribution to the stiffness matrix.
So it's getting a little low.
It says, add contribution to weighted residual, additional contribution to stiffness matrix.
OK, so last thing.
I'll just kind of put all those pieces together for the 1D diffusion, and then finally, end with just showing how the matrix system gets formed.
Yep?
[INAUDIBLE]
That's right.
The Robin condition is a combination of the two.
So there's a Dirichlet component and a Neumann component.
I don't know if you actually did-- were they discussed in the finite element?
They were mostly discussed in, I think, the finite volume, finite difference section.
But yeah, it's just a combination.
We've put something in the right-hand side.
What's that?
We had them in the project
Oh, you had them in the project.
So then you know how to do them, right?
Or you know how not to do them.
[LAUGHTER] Yeah.
OK, so just as the last bringing it all together, let's think about finite element for 1D diffusion, and we need to make sure that you are clear that you can go through all these steps.
So starting off with the governing equation, K dT / dx, differentiated again with respect to x, equals minus q on the domain minus L over 2 to L over 2 [INAUDIBLE] x.
So let's see.
We would write the approximate solution, T of x, as being the sum from i equals 1 to n of sum ai times Ci of x.
These here are our finite element basis functions.
We might, for example, choose to use the linear nodal basis, those hat functions.
So again, those ai, the unknown coefficients that we're going to solve for, are going to end up corresponding to the value of our approximate solution at the nodes in the finite element mesh.
We want to write down, then, the j-th weighted residual.
We'll call it [?
Rj.
?]
And what is it?
It's take everything over on the left-hand side, substitute in the approximate solution.
So we're going to get a K d T tilde / dx, all differentiated, plus q.
Weight it with the j-th spaces function.
And because it's Galerkin, we're going to use the same C. So we weight it with Cj, and then integrate over the domain, integrate from minus L over 2 to L over 2.
I could write out all the steps, but I think you've seen it many times in the notes.
So we get that, and then what's the next thing we do?
What do we do?
What's the first thing-- what are we going to end up with?
We're going to end up with [?
Cxx.
?]
We can assume K is constant, if you want.
We're going to end up with [?
Cxx's ?]
multiplied by C, so we're going to end up with [INAUDIBLE] the integral of [?
C ?]
times the second derivative of C with respect to x.
What do we do?
Integrate by parts.
[?
Usually you ?]
always [INAUDIBLE] integrate by parts.
That's going to move one differential off the C, onto the other C. Yeah.
So get the approximate solution, substitute it in, bring everything to the left-hand side, multiply by Cj, and integrate.
Integrate by parts, and when you integrate by parts-- I can write it out, if you feel uncomfortable, but again, I think you've seen it many times, and it's in the notes.
I'm going to keep the T tilde, rather than writing the sum.
This thing is really the sum of the [?
ai's ?]
times the [?
Ci's, ?]
right?
But I'm just going to write it as T tilde, because I think it's a little bit clearer.
You can [INAUDIBLE] if you're working through things, feel free to leave the T tilde in there until you get to the bottom, so you don't have to write too many.
So there's the-- we're integrating v to u, so there's the [?
uv ?]
term that comes out on the boundaries, out of the integration by parts, minus now the integral where we have moved-- we're going to differentiate Cj, and this is the Cj that's coming from the weighting for the j-th weighted residual.
And then our other one has been integrated, so now we just have a K d T tilde / dx.
dx.
And then we have the q term, which looks like an integral from minus L over 2 to L over 2.
And again, it's the Cj times q dx.
We don't have to do any integration by parts.
This side just stays the way it is.
So I jumped one step there, but again, the [?
first line ?]
of the residual would have a Cj times this full term with the T tilde there.
One step of integration by parts gives us the boundary term minus the integral with one derivative on the Cj, the weighting Cj, and one derivative on the approximate solution.
Yes?
No?
Yes?
OK.
So where do all the bits go?
Here is if we had a Neumann condition.
This is where this contribution would go.
Yeah?
This term here is going to give us the stiffness matrix, and this term here is going to give us the right-hand side.
Yep.
Let's just remember how all that goes together, conceptually.
This guy here is going to go to our stiffness matrix.
This guy here is going to go the right-hand side.
And if we think about just integral xj minus 1 to xj plus 1 of Cj x K d T tilde / dx.
So why did I change the limits here?
[INAUDIBLE]
What is it?
Mumble a bit louder.
Yeah?
[INAUDIBLE]
Yeah, because we're looking at the j-th weighted residual.
We've got the Cj x sitting out in front here, and we know-- again, if you sketch the Cj x, you know that it's only non-zero over elements that go from xj minus 1 to xj and xj to xj plus 1.
So all the other parts of the integral go to 0.
So that's the only bit that's left.
And again, I'll skip over a few steps, but it's all derived in the notes.
If you then substitute in the T tilde [?
has been ?]
the sum of the [?
ai ?]
times the [?
Ci's, ?]
again, a lot of the [?
Ci's ?]
are going to be 0.
The only ones that are going to be non-zero on this part are going to be this one, so Ci equal to Cj, and then this one, right?
Ci equal to Cj minus 1.
So itself and its left neighbor are the only ones that are going to be non-zero over this.
Because in the next one-- oh, and this guy here.
And this guy here.
That's right, there you are.
Three of them.
Itself, its left neighbor, and its right neighbor are all going to contribute to the way I've written it, this full integral.
And so what does it end up looking like?
It ends up looking something like there's an a j minus an a j minus 1 over delta x j minus 1 squared, and here are I've left in the K. If K were a constant, it would just be coming out.
And then there's another term that looks like a j plus 1 minus a j over delta x j squared integral x j to xj plus 1.
OK, so I mean you could work through all of these, but what's the important concept?
The important concept is that-- let me draw this a little bit more clearly.
The important concept is that when you're an-- x j-- always draw the picture for yourself, because I think it really helps.
Don't get too embroiled in trying to integrate things and plug things in.
It's easy to make mistake.
Just think about where things are going to be going.
So there's Cj.
There Cj plus 1.
And there's Cj minus 1.
So when we think about this element, the j minus 1 element, which goes from x j minus 1 to x j, which two get activated?
It's j minus 1 and j.
So we expect the coefficients aj and aj minus 1 to be the ones that are going to get coefficients put in front of them in the stiffness matrix, right?
Nothing else.
And when we think about this element, xj and xj-- from xj to xj plus 1, this is the j-th element, it's Cj and Cj plus 1 that are going to get activated.
So we expect to see coefficients aj and aj plus 1, with something [?
multiplying ?]
them.
And so before we went through we worked out all the integrals, again, you could do it by just plugging and substituting, but you could also just think, conceptually, what is the system going to look like?
At the end of the day, you're going to end up with a big matrix, the stiffness matrix, multiplying these unknown coefficients, which correspond to the approximate solution at the nodes in the finite element [?
match, ?]
and then the right-hand side.
And again, remember that the j-th row corresponds to setting the j-th weighted residual equal to 0.
So each row in the matrix comes from setting a weighted residual equal to 0, and then each column-- so i-th column multiplies [?
an ?]
[?
ai.
?]
So how does this system start looking?
Well, we know if this were the first element, we would be getting what?
[?
c1 ?]
and a [?
c2, ?]
so we might get entries here and here.
Although, if we had a Dirichlet condition, we'd see 0 [AUDIO OUT] entries.
First, second, and third.
The matrix starts to look like this.
And again, it's because when we do the j-th weighted residual, which is here, we're going to trigger contributions to j minus 1, to j, and to j plus 1.
And that's because the j-th weighting function, again, triggers two elements, the j minus 1 element and the j-th element.
OK, so I think [INAUDIBLE] two sets of skills you have to have.
One is that you have to be comfortable doing the integration, writing down the weighted residual and doing integration by parts.
And then the other is not to lose sight of the big picture of forming this matrix system.
Columns correspond to the unknowns, rows correspond to j-th weighted residuals, and it has a very specific structure because of the way we choose the nodal basis.
Questions?
You guys feel like you have a good handle on the finite element?
Is that a no, or a yes, or you will have by Monday and Tuesday?
So the last thing I want to do is go back, and I want to remind you that I've given you a very specific list of measurable outcomes, which represent the things that I want you to take away from the class.
And so when I construct assessments like the final exam, that's what I'm doing, is I'm trying to measure those outcomes.
So if-- when you are studying for the final exam, going back to those measurable outcomes and going through them I think is, hopefully, a really helpful guide for you.
Because it really, again-- let me get plugged in here-- really, again, tells you exactly what it is that I want you to be able to do.
So let's go through them.
And I just-- so I've pulled them.
And I'll go back to the [INAUDIBLE] site, or the MITx site where they are.
But starting at Measurable Outcome 212, this one is, "Describe"-- and I also want to point out that the verbs are also important-- "Describe how the method of weighted residuals can be used to calculate an approximate solution to a PDE."
Hopefully, you feel comfortable with that.
"Describe the differences between method of weighted residuals and collocation method."
We didn't actually talk about least squares, so just the first two.
"Describe the Galerkin method of weighted residuals."
And we just went over all of that.
Then under finite element, describe the choice of approximate solutions, which are sometimes called the test functions.
So that's the choice of the [?
Ci's ?]
that you use to approximate the solution.
Give examples of a basis, in particular, including a nodal basis.
And I've crossed out the quadratic, because I don't think [?
Vikrum ?]
actually covered it in that lecture.
But if you're comfortable with the linear nodal basis, that will be good.
To describe how integrals are performed using a reference element.
Explain how Guassian quadrature rules are derived, and also how they're used, in terms of doing integrations in the reference element for the finite element.
We went through all of these.
Explain how Dirichlet and Neumann boundary conditions are implemented for the diffusion equation, or Laplace's equation.
So Robin not actually necessary for the finite element.
And then describe how that all goes together and gives you this system of discrete equations.
Describe the meaning of the entries, rows and columns.
So again, this is what I was meaning by, this is kind of the big picture, right?
Now, to do the integrals and come up with numbers is one thing, but explaining what the system means, and what all the pieces are, and how they go together, is important.
So I think that's essentially what we just covered.
And I also want to remind you that if you go to the MITx web page, so there's the Courseware where the notes are, but remember, there's this Measurable Outcome Index that has all the measurable outcomes.
And they're also linked to the notes.
And I know one piece of feedback I've heard is that it would be really helpful to be able to search on this web page.
I know you can't search, but this is actually one way to navigate, that we could come in here and click on one of these.
And I think our tagging is pretty thorough, but as you click on that, here are all the places in the notes where-- in particular one, describe how integrals are performed in the reference element-- these are the sections in the notes that relate to this measurable outcome, and then these are the sections in the notes that have embedded questions in them.
So again, if you're looking for a study guide, going through these measurable outcomes, and going back and looking at the notes, and making sure that you can execute whatever it says here.
Yep.
OK, any questions about finite element?
You guys look worried or tired.
OK. All right, so I'm going to do the same thing, then, for probabilistic analysis and optimization.
I'll go through a bit more quickly, just because hopefully that stuff is a little bit fresher.
But again, I just want to touch on the main topics.
We'll hit the high points on the board, and then we'll just take a look at the measurable outcomes.
And particularly, I just want to make sure you're clear on a lot the last bits, which are a little bit introductory, what it is that I expect you to be able to do.
Does it feel like you've learned a lot since spring break?
Remember on spring break, wherever you guys were, in Florida or whatever, you didn't know finite element.
You didn't know Monte Carlo.
You maybe didn't know anything about optimization.
It seems like a long time ago, no?
It's a long time, but a short time
It's a what?
Long short time
A long short time?
[INAUDIBLE]
All right.
So over there are the six main topics that we've covered in probabilistic analysis and optimization, the basics of Monte Carlo, the Monte Carlo estimators.
We talked about various reduction methods.
We talked about design of experiment methods for sampling.
We did a very basic intro to design optimization, and then we talked about responsiveness models in the last lecture.
So let's just go through and-- where are my notes?
So this is probabilistic analysis and optimization, so I'm going to start numbering from 1 again.
So again, Monte Carlo simulation, I hope by now, after the project, you feel pretty comfortable with the idea.
I think the little block diagram says a lot.
If we have a model, we have inputs.
Maybe we have x1, x2, and x3, and maybe the model produces one or more outputs.
So inputs, in this case, maybe just a single output is always considered.
That we want to represent our inputs as random variables, and so if we propagate random variables through the model, then our output will also be a random variable.
And the idea of Monte Carlo simulation is really quite simple.
Remember, we broke it into the three steps.
The first step is to define the input PDEs.
So once you make the decision that you want to-- not PDEs, PDFs.
Input PDFs.
Once you decide you want to model these things as random variables, you have to come up with some way where you can probabilistic analysis, some way of describing what kind of distribution these guys follow.
And for pretty much everything that we did, those were given to you.
Remember I said that, in practice, this is a really difficult thing to do.
You can use data that you might have collected from manufacturing, or you might query experts, but you somehow have to come up and define the input PDFs.
Then, once you have those PDFs defined, you sample your inputs randomly.
Each random [?
draw ?]
you solve by running through the model, and so what this means is that if we make capital N samples, then we have to do n solves.
So each solve is going to be one run of our model, which might be a finite element model or whatever kind of stimulation it would be.
So now we've taken N samples, we've run the model n times, we have n samples of the output, and the third step is to analyze those resulting samples.
So analyze the resulting samples of the output, or outputs, if there's more than one.
And in particular, we might be-- well, we're usually interested in estimating statistics of interest.
Statistics of interest.
And those statistics might include means, variances, probabilities of failure, or whatever it is that we want to calculate.
And if these inputs are uniform random variables, then drawing a sample from them is pretty straightforward.
You can draw them just randomly [?
under ?]
[?
01.
?]
If these things are not uniform random variables, then, remember, we talked about the inversion method, where you use the CDF, you generate a uniform random variable, which is like picking where you're going to be on the CDF, go along, invert through the CDF, and that gives you a sample of x.
And in doing that, by sampling uniformly on this axis, remember, we graphically saw that that puts the right [?
entity ?]
of points on the x-axis.
And again, you implemented that in the project with the triangular distributions.
What?
[INAUDIBLE]
Nothing awful.
Yeah.
I think the triangular is the only one that's really analytically tractable
[INAUDIBLE]
Yeah.
I'm not going to have you do Monte Carlo simulation by hand.
[LAUGHTER] Although it could be interesting.
We could correlate your grade with how many samples you could execute in a fixed amount of time.
See what the [?
process ?]
of power is.
But yeah, no.
Triangular is fine.
Triangular is fine, as long as you understand it generally
Can we build [INAUDIBLE]
If you can do that in the half-hour you have to prepare with the pieces of the paper that you have with you.
That would be impressive.
OK, so I think the more conceptually difficult part of Monte Carlo are the estimators.
And let me just-- I know, I feel like I've been harping on about these for a while, but let me just make sure that they're really clear, because I think it can get confusing.
So we talked about three main estimators, the mean, the variance, and probability estimators.
So if we want to estimate the mean, and it's the mean of y, y being our output of interest-- so let's call it mu sub y.
Then what do we do?
We use the sample mean as the estimator, and we denote the sample mean as y bar.
And it's equal to 1 over n, the sum from i equals 1 n of y i, where this guy is the i-th Monte Carlo sample.
The y that we get when we run the i-th Monte Carlo sample through our model.
So we're trying to estimate mu of y, which is the real mean.
To do that, we use the sample mean, y bar, which is just the standard sample mean.
And what we know is that for n, as long as we take a large enough n-- large, 30 or 40-- that this estimator-- so let me back up a second.
This estimator-- this thing is the true mean.
It's a number.
The estimator is a random variable.
And it's a random variable because we're sampling randomly, so any time we do this, we could get a slightly different answer.
So this is a deterministic quantity.
This is a random variable.
We need to be able to say something about the properties of this random variable to be able to say how accurate it is.
What we know is that for large n, where large means of the order of 30 or 40, this random variable, y bar, will be normally distributed.
And the mean of-- let's call it mu sub y bar, the mean of x-- mean of mu sub y bar, and variant sigma y bar squared.
OK, so this is the mean of y bar and a variant of y bar.
And what we can show is that the mean of y bar-- the expected value of our estimator, y bar, is in fact equal to mu of y.
It's equal to the thing that we're trying to estimate.
And so this is what's called an unbiased estimator, because on average, it gives us the correct result.
So that's great, on average.
But now, the variance of that estimator is important, because it tells us how far off could we be if we just run it one time.
And we could also derive-- that variance of the mean estimator is given by the variance of y itself divided by the number of samples that we drew.
And the square root of this thing, sigma y bar, is sometimes called the standard error of the estimator.
I tend to not use that term so much, because it confuses me a little bit.
I just like to think of this as the variance of the estimator.
So I really recommend that when you write these things, use the subscripts to denote mean of y bar, mean of y, variance of y bar, variance of y.
Keep it straight.
Yes, Kevin
[INAUDIBLE]
This is not an estimator.
The estimator is y bar.
This is the mean and the variance of that estimator.
We haven't yet talked about the estimator for the variance, which could biased.
Yeah, so this is not an estimator.
This is just the variance of this estimator.
OK?
Yep.
OK, so that's the mean estimator.
And the other ones we talked about were variance and probability.
So let's say we want to estimate the variance of y.
And again, y is the output of our code.
And so this is sigma y squared.
Then, here we have a variety of different options.
I think Alex presented three to you.
Maybe the one of choice is f y squared, which is given by 1 over n minus 1, the sum from i equal 1 to n of the [?
yi's ?]
minus the y bar squared.
OK, so again-- and I guess the notation is a little bit unfortunate, because this isn't what people use.
The f y squared is the estimator for the variance sigma y squared.
And again, this is a random variable, and we want to know what its expectation and its variance are, because that tells us how good of an estimator it might be.
And what we can show is that the expected value of this particular estimator is actually equal to the variance we're trying to estimate.
So this one is unbiased.
And Kevin, to your question, if we had 1 over n there, that would be biased.
And I think [INAUDIBLE] Alex also showed you 1 over n minus 1/2?
Oh, 1 over n minus 1.5.
That one, I think, turns out to be unbiased estimate of the standard deviation.
Yep, yep.
OK, and so that's the mean of it.
It turns out that the variance, or the standard deviation of this estimator is generally not known.
So the mean estimate, we know by central limit theorem, follows the normal distribution, and we know its variance.
For the variance, typically it's generally not known.
How could we get a handle on this?
We could do it multiple times.
Maybe we can't afford to do it multiple times.
How else could we get [INAUDIBLE] what's the cheating way of doing it multiple times?
Bootstrapping.
Bootstrapping, which is redrawing from the samples we've already evaluated.
That could be a way to get a handle on how good that is.
Turns out there's a case that if y is normally distributed, then you do know what this is, and this estimator follows a chi squared distribution that's mentioned in the notes.
That's a very restrictive case.
So in general, we don't know what this variance is, the variance of the variance estimator.
Yep.
And then, the third kind of estimator that we talked about was for probability.
So estimate mean, estimate variance, and then estimate probability.
So let's say that we're trying to estimate the probability of some event A, which I'm just going to call p, little p for short.
And so we would use as an estimator, let's say, p hat of a, which is na over a-- over n. The number of times that a occurred in our sample divided by the total number of samples.
Now, again, for this one, we know that by central limit theorem, for large n-- so again, this estimator, t hat a, is a random variable.
For large n, it follows a normal distribution, with the mean equal to the probability that we're actually trying to estimate, and the variance equal to the probability we're trying to estimate times 1 minus the probability we're trying to estimate divided by n. So this guy here means that it's unbiased
So you cannot say that in general, an estimator of [INAUDIBLE] variance, true population variance, you have to include for each type of estimator--
That's right.
Yeah, and I think that's a place where some people were a little confused in the project.
It turns out that this is a variance of n. But this is the variance of the Bernoulli random variable that's the [?
01 ?]
indicator that can come out of the code.
And so that's a way-- if you feel more comfortable thinking about it that way, you can think about it that way.
But I much prefer to think that this is the mean and this is a variance-- the mean of the estimator and the variance of the estimator.
And the variance of the estimator is this whole thing, just like the variance of this estimator is this whole thing here.
And there do turn out to be other variances divided by n, but that relationship is a little bit murky, and I think you could get yourself into trouble if you think of it that way.
It doesn't work over there.
Yeah.
So it's definitely-- it's not a general result.
Yeah.
Maybe the key is that here you can write a probability as a mean, because it's the average with that indicator function.
And so you can use the same result that you have used here to get to this one.
But again, that seems a little-- if the ideas are clear in your mind, you can think about it that way, if you want to.
OK, so once you have this, then specifying an accuracy level with a given confidence is pretty straightforward, right?
If we say that we want our estimate to be plus or minus 0.1 with confidence 99 percentile, then what we're saying is that this distribution's standard deviation-- square root of this guy-- has to be within plus or minus, if it's 99 percentile, 3-ish times that 0.1, or whatever I said it was.
Yep.
And so that will tell you how many-- should make this a bit of a curly n-- how many samples you need to run in order to drive-- so the more samples you run, the more you're shrinking this distribution of p hat a, and making sure that the one run you do-- shrink, shrink, shrink-- falls within the given tolerance that you've been given, with a given amount of confidence.
Now, of course, the trick is that you don't know p, in order to figure out what this variance is.
But like in the project, worst-case, variance is maximized when p is 1/2, so you could use that.
Or you could do some kind of an on-the-fly checking as you go, get an estimate for p hat, and then use that in here, and keep checking.
And so then again, it would be approximate, but if you built in a little bit of conservatism, then you would be OK.
I posted solutions for the project last night, where that's explained.
OK, questions about Monte Carlo estimators?
Three different kinds of estimators that I expect you to be comfortable with.
OK?
You guys are very talkative today.
Very talkative.
All right, so moving on, variance reduction methods.
The main one that we talked about was importance sampling, where again, it's kind of a trick to help control the error in our estimates.
So in particular, we looked at estimating a mean, mu of x, which is the expectation of some random variable x.
And remember, I was putting the little x down there to denote that we're taking expectation over x, because we're going to play around with the sampling.
And remember, we said that we could write this as the integral of x times f of x dx, where this f of x is the PDF of x.
And then, remember, we just play this trick where we divide by z and multiply by z, which means that we don't actually change anything.
But then we define z such that z times f of x is actually another density, f of z.
Which means that we can interpret the mean of x as being the expectation over x with respect the density f x, or we can also interpret it as being the expectation of x over z with respect to the density z.
And why do we do that?
We do that because now we have two ways to think about this mean that we're trying to estimate.
We can think about it as the expectation of x under the density f of x, which means that we would sample x, again, under this density, f of x.
And then we would get our mean estimate.
And more importantly, we would get our estimator variance that we just saw a couple minutes ago as being the variance, again, under f of x of x over root n. So that would be the variance of-- the mean [INAUDIBLE] so that's just the standard Monte Carlo that we've been talking about.
But what we just saw over there was that we can also think about this thing as being the expectation of x over z taken with respect to the pdfz.
So this would mean, conceptually, sample now x over z, under now f of z.
And then, the real reason that we do that is that we can then play around with the estimator variance.
And it now is the variance of x over z with respect to z over n. And while these guys are the same, because the mean is the linear calculation, these two are not necessarily the same.
So then, remember, we talked about different ways that we could come up with a clever f of z that would put more samples, for example, in a probability region that we're trying to explore, that could have the effect of reducing the variance of the estimator, meaning less samples for the same level of accuracy, while giving still the correct estimate of the probability in that particular case.
And we also talked about how you would modify the Monte Carlo-- remember, the Monte Carlo estimator had that x over z, and then the z scaling back [?
inside it.
?]
I think, unfortunately, that is one of the lectures where the audio got scrambled.
But I scanned in my notes, and they're posted online.
OK, how are we doing on time?
[?
46.
?]
So we talked briefly about variance-based sensitivity analysis, and I think I'll put it up on the board.
But I really just introduced you to the idea of doing variance-based sensitivity analysis of defining those main effects sensitivity indices that gave us a sense of how the variance in the output changes if you learn something about one of the inputs.
We talked about different DOE methods, the idea being that what we're trying to do is sample the design space, but not do full random Monte Carlo sampling, maybe because we can't afford it.
So in DOE [INAUDIBLE] that the idea is that we would define factors, which are inputs.
And we would define discrete levels at which we might want to test those factors.
And that we're going to run some combination of factors and levels through the model, and collect outputs, which are sometimes called observations in DOE language.
And that the different DOE methods are different ways to choose combinations of factors and levels.
And we talked about the parameter study, the one-at-a-time, orthogonal arrays, Latin hypercubes.
So again, different DOE methods are different ways to choose which combinations of factors and levels.
And then we also talked about the main effects, averaging over all the experiments with a factor at a particular level, looking at that contingent average of all experiments.
And remember, the paper airplane experiment, where we looked at the effects of the different design variable settings.
That gives us some insight to our design space.
OK, and then lastly, in optimization, not too much to say here.
Just a basic idea of unconstrained optimization, what an optimization problem looks like, the basic idea of how a simple algorithm like steepest descent, or conjugate gradient, or Newton's method might work.
We talked about the different methods to compute gradients.
And we saw particularly that finite difference approximations, which you saw when you were approximating PDE terms, can also be used to compute gradients, or to estimate gradients, in the design space.
And then, we also talked about how we could use samples to fit a polynomial response surface model, if we needed to approximate an expensive model, and use it in an optimization.
So that was the last-- really, the last two lectures.
All right, so let me put up-- quickly look at the outcomes.
Everybody got-- what's that?
[INAUDIBLE]
We'll go look at the outcomes for that last section, the probabilistic analysis and optimization.
And again, these are specifically what I expect you to be able to do.
I'm not going to ask you to do anything that's not in these outcomes.
If you can do every single one of these outcomes, you will be 100% fine.
So 3.1 and 3.2 related to some basic probability stuff.
3.2 described the process of Monte Carlo sampling from uniform distributions.
3.4 described how to generalize it to arbitrary univariate distributions, like triangular.
Use it to propagate uncertainty.
You've [INAUDIBLE] this one in the project-- [?
horray.
?]
Then, a whole bunch of outcomes that relate to the estimators.
Describe what an estimator is.
Define its bias and variance.
State unbiased estimators for the mean and variance of a random variable and for the probability of an event.
Describe the typical convergence rate.
So this is how does the variance, how do the standard deviations of these estimators behave as n increases?
For the estimators, define the standard error and the sampling distribution.
So this is what we're talking about with the normal distributions that we can derive for the mean and for the probability.
In the variance, we typically don't know what it is, unless we have that very special case that the outputs themselves are normal.
Give standard errors for the sample estimators of mean, variance, and event probability.
This one should really only be in that very special case.
And then, obtain confidence intervals, and be able to use those to determine how many samples you need in a Monte Carlo simulation.
Then on Design of Experiments and Response Surfaces, so just describe how to apply the different design of experiments methods that we talked about-- parameter study, one-at-a-time, Latin hypercube sampling, and orthogonal arrays.
Describe the Response Surface Method, and describe how you could construct it using least squares regression.
Remember, we did the points.
We construct the least square system.
Use it to get the unknown coefficients of either a linear or a quadratic response surface.
And then, we didn't talk about this in too much detail, but I know you've seen the R2-metric before, and you understand that that measures the quality of the [?
search, ?]
of the Response Surface to the data points.
Then lastly, on the Introduction to Design Optimization, be able to describe the steepest descent, conjugate gradient, and the Newton method, and to apply it to simple unconstrained design problems.
Describe the different methods to estimate the gradients, and be able to use finite difference approximations to actually estimate them.
And then, interpret sensitivity information, so things like the main effect sensitivities that we talk about, and explain why those are relevant to aerospace design examples.
Remember, we talked about how you could use sensitivity information to decide where to reduce uncertainty in the system, or where-- there was called factor prioritization, or factor fixing, where there are factors that don't really matter, and you could fix them at a value and not worry about their uncertainty.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good afternoon.
My name is Joe Leavitt.
My group members are Ben Ayton, Jess Noss, Erlend Harbitz, Jake Barnwell, and Sam Pinto, and we're going to introduce you to the topic incremental path planning.
Listed on screen are just some of the references we used in putting together this lecture.
The top three are the main references we used in developing our material on the D* Lite algorithm, but also take note of the bottom reference, which is a paper done within Professor William's group here at MIT.
It talks about all-pairs, shortest paths incremental search method.
So if you're interested in more incremental search methods than those we're going to talk about today, please get into this.
Especially if you're going to go ahead and do our P set, these references will be very helpful to you.
So today, we're going to first Monday problem, then we're going to introduce the idea of incremental search.
Then we're going to go into detail on the D* Lite algorithm, which is type of incremental path planning, an algorithm used for incremental path planning, run through an example of D* Lite, talk about when it is good to use incremental path planning when other path planners might be better, and then I'll go into some algorithm extensions and related topics, and some applications in mobile robotics.
So in order to motivate the problem, we're going to use a robot living in a grid world.
It's at a start location trying to get to a goal location.
It can move up, down, left, or right, or diagonally.
And it can only see the eight cells around it, as it's moving through this grid world.
And we're going to give some pretty fine map of the world.
This is a map that has the world before it starts moving.
And it's possibly moving through some of these, new obstacles could appear, or obstacles could go away.
And the idea is that it's going to make observations as it's moving to this world and update its plane to get to the goal based on what it sees as it's moving along.
So I must correct myself.
The map that you're seeing now is the map that the robot has ahead of time.
The previous map is what the environment actually looks like.
So as it starts moving, it's going to make a plan to get from the start to the goal, and that's what we want it to do.
And it's going to move to the next position and then realize that there's an obstacle in the path that it previously planned, and now it must replan in order to find a valid path to the goal.
And so now, it replans, and continues to move on the new plan, and now it finds another obstacle that it has to replan around, and it does the same thing.
And then it continues to walk through until it gets to a point where we have a change in the environment.
The change is represented by these two cells going black here, and now its original plane is no longer valid again.
And what we want it to do is to replan to find a new valid optimal path to the goal.
This is just showing here what happened to our environment.
And then the robot replans and continues to march towards the goal So this is the behavior we want from a mobile robot, whether it's a two-dimensional path-planning problem, or we're talking about multi-dimensional robot arms, planning, working with humans on a manufacturing environment, or an activity-planning process.
The idea is that we want the robot to be able to account for new obstacles or changes in its representation of the environment and continue to plan on the fly quickly and optimally.
So there's methods that accomplish this.
One method used to in path planning is rapidly exploring random trees, or RRTs.
RRTs are basically going to plan, replan, on every time step based on changes in the environment.
And one problem in my experience here, though, is that RRTs are suboptimal, even though fast.
See, here you see the robot has come up with a plan.
It's seeing new obstacles, the table in front of it.
The coffee cup.
And its planning for those things, so the plan it came up with is clearly not optimal.
So we can perform some additional computation.
We can use a method that builds upon the RRT, RRT*, that it finds an optimal path, but that requires a pretty significant amount of offline computation time, and then you can accomplish something that looks a little more normal.
Again, we have to allow the robot to perform a lot of the computation off-line.
So how do we accomplish what we want to?
We want to compute quickly.
We want to compute optimally, and then have the robot continue to move as it's received new information coming to the environment.
Some problems we're facing, there's changing environmental conditions.
There's sensor limitations.
You might not be able to see everything in the environment around you, and then we have limited computation time.
And the solution that we're going to explore here is to reuse that from the previous search, in order to speed up the search process, and continue to plan optimally.
And the method we're going to use to accomplish this is incremental search.
So now, we're going to talk about what incremental search means.
First to show you kind of how powerful incremental search can be, this is one example of an incremental search method, incremental all-pairs shortest paths, which allows you to online compute paths between any start and goal location.
The idea is that you have a robot that's trying to move around one obstacle.
That's what you see in the upper sequence of events there.
And another obstacle is added, and now it has to plan around this new obstacle.
In using this algorithm, you have a significant increase in performance and computation time using the all-pairs shortest paths versus an RRT.
RRT can act as another version of an RRT, where you're planning basically from both the goal and start and connecting your paths.
It's on the order of what you get with RRT, but this RRT connect algorithm doesn't give you an optimal result.
And then it's even fast than another algorithm probablistic road mapping, where you're randomly sampling the environment, and connecting points to produce a path.
So the idea here is that we can really leverage this idea of reusing results from previous searches to accomplish fast replanning online.
So the whole idea of incremental search is that we're going to form a normal graph search, just like anybody's used to.
And then we're going to repeat as the robot moves through the environment, or you're executing your plan, you're going to execute the next point in the plan, or the next action in the plan.
We're going to receive changes from the environment, either through our sensors or information from the outside.
And then we're going to use that information to update our previous search results instead of completely replanning, and then continue to form that loop, until you reach your goal.
So before we get into too many of the details of how we're going to accomplish this, first, we want to review the graph search problem.
So in graph search, you have a graph consisting of vertices or nodes.
We're going to use nodes throughout this lecture to talk about the vertices or nodes in the graph that are connected by edges.
And then, there's an edge weighting function that describes the cost going from one node to another.
And then, a heuristic function is an estimate of the cost to get from a current node to the goal node.
And then we a start vertex and a gold vertex.
In the graph search problem, we're also going to have this idea of a g value, or a cost so far.
We're going to use term g value, which is essentially the cost that it's taken to get to some predecessor node, plus the cost of the node that you're actually expanding.
So you would have this W start to S1 cost to get from start to S1.
And then the g value for S goal would be the weight from S1 of the goal plus the S1's g value.
Yes, go ahead
What is W?
W is the weighting function that I talked about previously here.
So the weighting function describes the cost to get from one node to another along any edge in a graph.
And then we'd use those waiting functions to develop the idea of the G value.
So W is just the edge cost.
And then our heuristic value, like, we use our heuristic function to establish a heuristic value from one goal to another, and we're going to call that our h value, or h. And then the total estimated cost to get from a node that we're currently expanding in a graph search is f, which is your cost so far, plus your estimated cost to go.
I think I might have messed that up in there.
Call g value associated cost so far, not cost to go, and then h is an estimate of cost to go.
So in order to reason about a real-world environment, we first have to take that environment and convert it to a graph, so we can form graph search, and then do everything we're talking about.
Some of the examples, we're going to talk about, and some of the ones we've already talked about, we're using this notion of a grid world.
You can have a four connected graph or a grid, where a robot can move up, down, left, or right, basically in x or y directions, or an eight connected graph, where you can also move along the diagonals.
And then had here, we just showed the graphs associated with those, with the nodes and edges.
And then you would have an associated admissible heuristic with those if you're going to form a heuristic search.
So just know up front that any time we're moving from a real world environment, that we're trying to reason about these search methods, we have to develop an associated graph.
And one more thing that is important as far as understanding incremental searches, is the notion or idea of relaxation.
We're going use Dijkstra's algorithm.
Everybody here familiar with Dijkstra's algorithm, I'm assuming, in the past.
We're going to use Dijkstra's algorithm to kind of illustrate the idea of relaxation and remind everybody of what that means, because it's an important concept when we're talking about incremental search algorithms.
So essentially, in Dijkstra's algorithm, it's a single source, shortest path algorithm, a best first search.
So we're going to make eight expanding nodes according to their best g value, or cost so far.
And we have our start node initialized to zero cost so far, because at your start, haven't accrued any cost.
And then, every other node is going to be initialized to infinity.
And the idea is we're going to find the shortest distance to all of the other nodes by relaxing the cost to those nodes from infinity down to their actual shortest or lowest cost value.
So we start by expanding the start node, and then you find the cost so far to each of the children nodes of the start node.
So you have an edge way to one, and so the node on the bottom as a value of 1.
Similarly for the middle node at 5, and the top node for 3.
Now, the next node we're going to expand, can anybody tell me what the next node we're going to expand based on performing best first search here using cost so far?
The 1?
1?
Yes.
So the node labeled 1 here, that has a g value of 1 is the lowest out of 1, 3, and 5, and infinity.
So we're going to go ahead and expand 1 next.
And now, we find that using this value of 1 plus 3 equals 4, which is less than what we previously had here as 4.
So now the center node has been relaxed from a value of 5 down to 4.
So now the shortest path we found to the center node is 4.
And we're going to continue to perform this by going through each node in order of the lowest cost.
And similarly, you saw how the top node was relaxed to 6 by going through 1, 3, and 2, where as previously, it had a lowest cost of 7 going through 3 and 4.
And if we format through the whole graph until all nodes have been unexpanded, then we've relaxed every node in the graph to its shortest path value.
And then, we can just do a greedy search from a chosen goal node, back through the graph to find shortest path
Is the term relaxation standard?
Because the fact that you're bringing numbers from infinity double to small value sounds more like tightening as opposed to relaxation.
So I was just wondering if people use it as like, a standard--
I'm not sure if it is in all areas, but I have seen it in multiple places.
When you're talking about graph search, specifically with respect to Dijkstra's algorithm, that is where the term relaxation is used.
And we're going to use that as we go throughout and talk about incremental search.
We're kind of repairing the results because of new edge weights and things like that.
We're going to figure out which nodes we have to relax back down to their shortest value to be able to find the new shortest path.
So that's the term we're going to use here if it's not used universally.
I can't say for certain.
But we will use relaxation to talk about reducing-- it's kind of related to-- think about like, a tension spring.
As you reduce the constraints on it, it'll slowly relax to its completely relaxed position.
That's kind of the idea here.
So now that we have talked about the ideas of relaxation and graph search, how do we reuse the results from a previous search in a new search field to employ an incremental search method where you have done a search, you found the shortest path, and now, you had some changes to your graph, and you want to use your previous results to come up with your new shortest path.
So the way we're going to do that, is we're going to store our optimal results from a previous search.
This can be done in a number of ways.
It's going to depend on the algorithm.
But you can soar like, a list of shortest paths as an incremental all-pairs shortest paths problem.
The one we're going to talk most about is storing g values, and that's what's done in a D* Lite algorithm, where you're finding the shortest distance from any given node that you've expanded so far in your search to the goal node.
And then, you'll use that data to find out, find the shortest path.
And then when you go to perform your next iteration, you'll look for inconsistencies in that graph.
And then in order to find a new shortest path, we'll leverage that previous data and the inconsistencies for relaxations to come up with a new optimal solution.
And that's all we're showing here.
On the right side of these two graphics is you have the initial search.
You have an obstacle that is now in a place that was wasn't there previously, which causes you to update edge weights in the graph associated with this grid world.
And then using these values which represent the g values in the graph search, you only update the ones you need to update to find a new optimal path.
And so completely reperforming our search.
So what incremental search methods exist, now that we've kind of talked about what they are and what we can do with them?
So the whole idea of this slide here is to show you that incremental search can be used across many domains.
It's not just specific to path planning.
It's used in temporal planning to determine temporal consistency.
It can be used propositional satisfiability.
Instead of every time you're updating another algorithm, you can use your previous results.
And the one we're going to focus on today is D* Lite, which is specific to mobile and robots.
But the ideas and concepts we're going to talk about there are applicable elsewhere, and you can use to help you learn about and expand your knowledge in other incremental search.
So the D* incremental path planning approach.
The whole idea behind it is that we're going to take the idea of incremental search that we just talked it about, and we're going to combine it with heuristic search, which is an optimal solution.
And the idea is that two things are orthogonal so we can combine them, and then we can form efficient incremental path planning, getting both an optimal result, and quickly getting a new path when things change in the environment without having to completely replan.
And so what that looks like, when we have a grid world, which is shown in four instances here up on the board, and we're trying to plan from a start node to a goal node, and showing each of these sections.
And in the upper left or on the left axis, we have-- whether we're performing a complete search or an incremental search.
And on the horizontal axis, whether it's an uninformed search or a heuristic search.
So the upper left is a complete search, just using best-first search and no heuristics.
And what it's showing is all the nodes that are expanded, and trying to find a path from the start to the goal.
In the upper right is a heuristic search, A*.
Employing the heuristic, we're able to collapse the number of nodes expanded by quite a bit.
In the bottom left is an incremental search method that's using uninformed search or best-first search as an underlying search algorithm, so it's going to expand the same nodes as the best-first search.
And then in the bottom right, you see an incremental heuristic search, the lifelong planning A* algorithm, which is kind of the baseline for the D* Lite we're going to talk about, that the initial search will expand the same nodes as the heuristic search, since it's using A* as its baseline.
So now let's say the robot moves.
We get an update to the environment.
There's a bunch of changed edges.
And now best-first search, as you can see, is going to have to expand pretty much the same number, if not more nodes, then our previous search.
And A* also is going to expand about the same number of nodes as it did previously.
The incremental search method using best-first search, by using previous results, reduces significantly the number of nodes that needs to be expanded.
And then, lifelong planning A*, the incremental search plus the heuristic search, reduces that number by even more.
Even when there's quite a few changes in the environment, you have very few nodes that you need to expand on the next search in order to find a new optimal path.
With that, I'll turn it over to Ben to talk about the D* Lite algorithm
Thanks.
So I'll be walking you through the D* Light algorithm today.
Now as we're emphasizing, the D* Lite is not the only incremental algorithm, as we mentioned previously.
But it is a widely used incremental algorithm, and you'll see it's quite efficient in what it does.
D* Lite is an algorithm that searches from a single start node to a single goal node.
But as we see changes occur in the path along the way, we'll adjust that.
So it's nice to think about D* Lite from two perspectives.
One is to think about it as a modification of the principle space.
Now, you'll see that D* Lite in its first iteration or its first run through before any modifications behaves very similar to A*, and that's where it's nice to keep that main model in mind.
And you'll see that in an example further along.
The second perspective to look at A* is from the perspective of Dijkstra's algorithm, from which we see many of the principles of relaxation that we covered beforehand.
From this perspective, we see that when we do reparations to the graph or changes to the graph, our methods of repairs the graph is essentially to relax down our changed edge weights until they reach their truth.
Now, where we differ from Dijkstra's algorithm with something like D* Lite is selecting only the nodes that we want to hit efficiently.
So whereas Dijkstra's algorithm tried to fit the entire graph, we only want to travel from a single start node to a single goal node.
And so D* Lite behaves like Dijkstra's, but a guided Dijkstra's.
So I'd like to just remind you of several concepts from A* first, because it's easier to introduce them in that context.
And then we'll modify them a little bit.
So Joe mentioned before that for A*, we're searching from a start node to a goal node.
We're sorting the nodes in our queue by total cost, which is the sum of what we're calling g, which are costs to get to a single node, and our heuristic, which is the cost to get from a node, s, to the goal node, your s. Now, what we don't usually consider with A*, but is very important for D*, is how to distinguish between ties when two different nodes on the search cube have the same total cost.
So here's what we're doing is we're introducing a couplet, which consists of two values.
The first, which I'm calling f1 here for any given state, is the same as what we're usually used to.
It's the linear combination of your cost to reach the goal node and heuristic function.
But we also introduced a second value for this couplet that's used in the case for only those to draw.
And that is simply the cost to go, that g value.
So the order of expansion of nodes from the q will order them in terms of f1 first, and if two nodes at the front of the cube tie in terms of their f1 value, we select the node with the lowest g value.
So looking at our graph down here, we're looking at our states s1 here and s2 here.
Can anyone tell me the total cost for our node s1 which, remember, is a couplet of two different values?
5, 3.
So five, yes, is our cost to reach this node one, plus the heuristic, which is 2 here.
And also, 3 is our second value, because it's only our g value there.
And so what would the value be by the same logic for s2 down here?
5, 2
5, 2.
That's exactly right.
And so, bearing in mind what I told you before, where should we take off the q first?
S2?
That's exactly right, because the first value, f1, for these two nodes is exactly the same.
But the second value is lower is f(s)2.
I'd also like to introduce the concept of successors and predecessors of any given node.
In general, a graph consists of directed edges.
And we could have a graph with undirected edges, but we can think of as just any edge being directed in both direction.
So here, we define the concept of a successor of a node, which is every node can be reached from a given node, along edges that can be found.
So the successors of this given red node are these two blue nodes here.
We also introduced the concept of predecessors, which is every node from which that node can be reached, meaning nodes from which we can follow our directed edge to our node.
So for our red node, the predecessors are these two nodes.
What I'd like to emphasize here is comparing the two, that this node here was both the processor and the successor of our red node.
And that's fine.
That occurs when we have undirected edges or edges that direct in both directions.
So think about D* Lite in the sense that is a repeated best-first search, which is where the A* principles come in through a graph with changing edge weights as that graph is traversed, meaning that as we travel from our start node to our goal node along a path that we are planning, we can see that edge weights are changing.
And in this example, I'm modeling an obstacle coming in place between this node here and the goal, which means that we have removed that edge.
And that's effectively the same as changing an edge weight.
We're just modeling it as changing this edge weight to infinity, which means that edge can't be traveled along.
And as I mentioned before, we also view this as replanning through relaxation of path costs.
So once we have moved to this point, how do we find the path from here to here when this edge has been removed, essentially?
So we want to make D* Lite efficient.
And it's a repeated search through the graph as we traverse it.
So you'll see that since we're using these g values, if we maintain a formulation where we're measuring the distance to go or to reach a node from our start node, this is not preserved when we move our start node.
And we move our start node when we're updating the position of our vehicle.
So let me show this through an example here.
The g value, or cost to get from one start node to our current node, s, here is the combination of these two edges.
And so, it sums to 4.
Now when we move along that path that we've planned to this next node, the g value to get to that same node is here too.
So this isn't preserved.
And that means that we'd have to potentially reformulate things in our cue, which would be very inefficient.
So what we do is we reformulate our search.
And it's perfectly valid to reformulate our search to search in the opposite direction.
We're essentially measuring for our g values now the cost to reach the goal from a specific.
And instead of heuristics, then measure the costs to get from this specific node, or to get from the start node to our specific node.
And so we search from the goal backwards through the graph.
In this case, we see that regardless of whether our start node, we preserve our g values, and our cost to travel from s to that goal.
That's what we like, and we we'll keep that reformulation to maximize efficiency.
So here, I want to give you a kind of overall understanding of the D* Lite algorithm before we dive into specifics of what it's doing.
First we want to initialize all nodes as unexpanded.
And what exactly, it means expand a node neighbor or cover.
And then we're doing a best-first search from the goal node to the start node until the start node is consistent with its neighbors.
And I'll again say what that means later.
We then move our start node to the next best vertex, essentially our node moving along the plan that we created.
And then we have to see if any of those edge costs change.
We track how our heuristics change, and then update the source nodes of those changed edges.
And we essentially repeat this process until, again, we've converged to a solution, and we keep moving.
So this best-first search, the A* Lite part of the algorithm, is where most of the computation is going to be occurring.
However, the incremental component is actually how can we adjust our edge costs to make sure that we're using most of the information that we previously gathered as efficiently as possible.
So I'm going to walk you through various steps of this.
First, I'd like to walk you through this step of moving to the next best vertex, just so you understand that.
So we can extract a path, given the path costs that have, through this simple argument, which says that we map the successor, by which I mean we restate what the start node is, because we're moving from one state to the next state, which becomes the new start node, by this form, which essentially says that we're taking the best path in the sense that we're minimizing the total cost that would be gained traveling by that path to this node.
So in a sense, the g value to get to this node, to the red from the green, or color stretching back through the graph, by this path here, is 7.
But to get to this node from the lower node is 10.
And so our g value that is optimal is 7 in this case, and know that we pick the successor as the green node up here.
So now to walk through some principles that appear in both of these steps.
I want to show how we handle weight changes locally.
And we do this because we don't want to track weight changes throughout the entire graph.
We want to handle this efficiently and separate them, so we only need to handle the changes that really impact our problem.
So this is the same formula that defines our successor, or our g values for a given node.
But we can see that this may no longer hold when edge weights change.
The example that I have here is that I've changed edge weight cost from 6 to 1.
And so now we can reach the red node, searching back through graph by a new optimal path, coming down from this node here, which would make our new g value five instead.
So this hasn't been preserved.
And then these changes then propagate to our predecessors, because the costs for the processors are each dependent on the g value for their successor.
And so when we hit a node, we have to propagate that change all the way back through the graph.
So to do this efficiently, we take an A* like approach, where we update the lowest cost nodes first, and we don't expand a node until it is the next lowest cost.
To help us do this, we're going to store additional fact.
We'll call this rhs.
Now the meaning behind rhs, it comes from the term right-hand side, which in the paper world, was first introduced and made more sense than it does so here.
What you can think of your rhs value as is essentially you're corrected your g costs.
When edge costs change, how G should be signed so that it's correct for a different graph.
And when your rhs value is not equal to your g value, that means that we have what is called a local inconsistency.
So given the logic that your rhs value should be what your g value would be corrected to be, what would your rhs value be for this graph here?
5?
5, yes.
So this is the same graph that I presented previously, and I walked through how g should be 5. rhs should be 5 here.
And what we're tracking is rhs and g differently.
So g is what it should be, but we haven't got the dated g to be 5 yet.
So we have two different types of local consistencies that we distinguish between.
We have what we call local overconsistency, where g is greater than your rhs value.
This is going to behave more similarly to Dijkstra's algorithm, where the value that we're associating to any given node now is higher than it should be, and we're relaxing down to the true value.
However, we have a lovely underconsistent case that we have to handle slightly differently.
So now I'm going to walk through updating and expanding nodes.
What I mean by updating a node is recomputing the rhs value, and then placing that node on the priority queue, if that node is locally inconsistent.
So to give an example here, I've changed this value, moving from 6 at the bottom to 1, and recomputing rhs, as we did beforehand.
Now we see that the node is locally inconsistent.
In this case, it's locally underconsistent.
And so we place that on the priority queue.
The priority queue values are based on this new formula.
This new formula essentially handles how we see G and rhs.
In essence, we want to take the minimum value of one of these, so that we can handle it the first time that it will cause changes that propagate through the graph.
And so this is essentially our ordering on our priority queue which is essentially the same coupling that we saw for A*.
But here, we just have g of s replaced by the minimum of g of s and rhs of s. And so, here your minimum of g and rhs is 5, plus a heuristic, leads to a total cost associated with this node of 9, 5.
So we expand our five prior nodes by then taking them off the priority queue and changing g. Now I said here that we have an inconsistent.
And so, we update this in much the way that we would expect.
Your g values are relaxing down to your rhs values, and so we just set g to be equal to rhs.
In this case, our node is now locally consistent, and it's going to stay that way.
So can anyone think of some good reasons why I go through the process of changing rhs and keeping track of that new value, only to put on the queue, and then take it off again, instead of just recomputing what g of s should be in the first place?
Does anyone have any ideas?
[INAUDIBLE]
What you can think of is that you're putting something on the queue.
If we update that value immediately, there are multiple nodes that could change the same node that we just considered.
By, which I mean more work changes as they propagate back through the graph, could lead to changes in rhs value of that specific node.
And so, what we would need to do if we can recompute g of s every time, is put it on the queue to signal that we're recomputing g of s, and do that recomputation, and then new changes we've done, perform that again, do that again and again and again and again.
And we want to avoid that situation.
So what we're going to do is we're going to keep our rhs as essentially a temp value that can be adjusted when it's in the queue.
So when something is on the queue, it's on there once, and it can be updated and taken off.
But we know when it's been taken off, that it's been taken for the correct time, and won't need to be handled again
So if I understand this correctly, so if you had a single edge cost that we changed, then you could just do what you just said, in terms of, you could just update the thing, and probably should be predecessor rate
Yes.
So if you knew for sure that nothing else was coming back and down, then we could just do that change and propagate to all the predecessors, because nothing upstream of that node is affected
So am I understanding correctly-- so you're basically running the programming, because you're keeping the costs to goal to devote, and then what you're doing is every time you have an edge that changes to a value that could potentially make that node better than it was before, so if it changes the value of that node to something better than it was before, you would change that node, and then queue all the predecessors, because those be changed as well
Yes
I mean, if those nodes do not change, you don't do anything.
You're moving from the queue.
They also change a better bell and keep propagating until you have nothing else to [INAUDIBLE]
Yeah.
But we only do the propagation, or we queue changes that occur earlier in the graph, but we only actually perform the change resetting the g value, when it's expanded
And then the reason for that is for efficiency when you have more than one edge changing?
Yes
So you could basically have multiple paths affecting the same node.
And in this situation, if you actually allow your changes will occur and multiple edges, that's when the importance of the queue comes
That's exactly
Which is actually, it prevents you from redoing the same work over and over again
That's exactly right.
Because if another change came through, then we'd put all the predecessors back in the queue again, and we'd do all of our updates for all of them as well.
We're avoiding that process
And the queue ensures that we're only updating-- and I think that'll cover it.
But in the manner in which we terminate, we don't necessarily-- we don't expand every node in the queue.
We only expand those we need in order to find the shortest, and then we retain the queue for that search in case we need to update those in the next round of the search
Right.
So I have to correct a mistake in my language that I said before.
I said that the node in my examples that I gave before was highly consistent, [INAUDIBLE] wasn't consistent.
I apologize for that mistake.
Other consistency is the easy case that we're handling here, where when we expand that node, we have to take g of s and set that to be equal to the rhs value for expansion.
And then we propagate that effect to all of the predecessors at that point, which means we update the predecessors and put those on the queue if it hasn't had an impact on them.
This means that the node is not low key inconsistent, and it's going to remain that way in that fashion.
The underconsistent case, which was the opposite of what I showed, is the more difficult case.
In this case, the old path cost was better than the new rhs value.
Rhs value has increased to above [INAUDIBLE]..
In this case, we can no longer relax down to that g value from our Dijkstra's algorithm respective.
So we have to think of a new way to handle this.
What we do is we essentially set our g value to be equal to infinity.
And this is an extra step that's going to cause the node to go back on the queue an additional time.
But what this mechanism assures is that that node goes back on the queue at most one additional time, which means that for D* Lite, we only ever examine any node at most twice on the queue.
Once we've set g of s to be equal to infinity, then we can essentially relax down to the rhs value from the perspective that we had before.
So we set g of s to be equal to infinity, and we update all the predecessors of s and s itself this time.
As I was saying before, we need to put that node node back on the cue so that we handle it when rhs has hit its minimal value.
And this means that that node is now going to locally consistent or overconsistent.
And we handle overconsistency in the case that I've shown you before.
So to give you an idea of how exactly this works, I'm showing you an example here.
We start off with this graph, and we introduce two changes.
We have increased this edge cost to 5.
And so rhs has now increased to a total of 10.
But I've also changed a value here for edges that you can't can see.
But assume that the rhs value of this node has also changed.
So now we can put all of these nodes on a cable ordered in this way, computed using the formula that we've seen for it.
We expand the node s1 first.
And because it's underconsistent, we set g to be equal to infinity.
Then we propagate to all predecessors and s1 itself, meaning those go back on the queue.
Now, assuming that node changes of any relevance happen in these nodes, we recompute the costs for s1, which has now changed to 14, 10, because we're using the minimum of infinity and 10, which is 10 plus 14, going behind the node s2.
Next, we expand this node s2.
And because that was an overconsistent case, we can just set g to be equal to our rhs value.
We then have to propagate that to its predecessors, which includes s1, which has updated its rhs value.
So if we didn't do this, we wouldn't have handled this effect correctly.
When we set our rhs value here, we then have to put this node back on the queue.
So previously, s1 was already on the queue, but the effect has to be propagated to this node, which puts it on the queue again, which would mean adjusting the values, because rhs has been reduced by 1.
Both the costs here have been reduced by 1.
Then we expand that node.
We have an overconsistent case.
We're setting g to be equal to rhs.
And finally, we've completed our graph search problem.
So now what we have to handle is an additional complication that occurs between several steps, tracking how our heuristics have changed.
So we want to carry over prior queue, which was what was mentioned previously, between our different searches as our stock node moves.
But the problem is that we're moving from our start node to a different point.
Recall that our heuristic value is measured from any given node to the stock node.
So when our stock node has changed, the heuristic value is not going to be the same value to the new stock mode.
So we want what's already on the queue to be comparable to what is being computed for our new value, so we can use the math and the algebra that we've already done.
So how we handle this is we introduce something called the key modifier, where we move from the previous start node, which here I indicated as s last, to the new start node, all the heuristics for admissible heuristic are lowered by at most a heuristic from the last start node to the new start node.
So now, when we add new nodes to the queue, instead of subtracting that value from everything that's already on the queue, what we do is we just increase for all nodes that we put on the queue, their values, by this new modifier.
Because essentially when we're taking off of the queue, all that matters is the relative differences between the two.
And so, instead of subtracting a value from everything on the queue, adding that value to all new values that occur on the queue will achieve the same purpose.
And so our key modifier is initialized to 0 at the start of the algorithm, and is increased every time that we update our start by this new value, by the heuristic between the last start node and the new start node.
And then we update our total cost to include the edition of the key modifier fire for our f1 term.
So now we've covered all of the D* Lite algorithm basics.
And we can actually walk through the slide that I showed you before with the math.
So when we start, we initialize all of our g values to be equal to infinity, and all of our rhs values to be equal to infinity, except at the goal node.
And that's to ensure that we always have a node to start from our search.
We best-first search from the goal node to the start node until that's locally consistent and expanded.
We should now know what that means.
And that signal that we have found our best path, we move according to our movement rule, and then if any of our edge costs have changed, we update our key modifier, and update our rhs and queue positions for the source nodes of change in edge costs, meaning the nodes from which those edges originated.
And then, we simply repeat from two, following best-first through the principles that I've walked you through for overconsistent and underconsistent nodes.
And as a reminder down here, we always search or sort by this very long couplet of two values.
So now, we're going to walk through an example of [INAUDIBLE]
OK.
So again, here's the pseudo code that Ben was just presenting.
And we're going to go through a quick example with five nodes, so this isn't going to show all of the properties of why you would want to use incremental path planning, because when you have only five nodes, you're going to end updating a lot of them a lot of the time.
But after the example, you'll see how it makes more of a difference on a larger graph.
So in this graph, we're going to have prior nodes that go A, B, C, D, and G is our goal node.
The robot will start at node A, and its goal is to find the shortest path to node G, except that the graph might change as the robot goes through, or the robot might gain new information.
So initially, the robot thinks that every node is available.
And it knows that the path lengths are all 1, except for from D to G, the path length is 10.
So this is a bi-directional graph.
Every edge goes both directions.
And the heuristic, as Ben described, will be with respect to the start node.
In this case, we'll just say that the heuristic is the number of nodes to the start node.
So we're ignoring the fact that this edge length is 10.
So first, we'll initialize all the g and rhs values.
So we have infinity at every node, except for the goal node, which has an rhs value of 0.
So this means that all of the nodes are correctly locally consistent, because their g and rhs values are the same, except the goal node.
So we put the goal node on the queue using the formula that is on Ben's slide, which I'll put here for reference.
So the key is going to be these two terms.
First, we have the minimum of g and rhs plus the heuristic value plus the key modifier, which is initially 0.
And then we have the second term, which is just the minimum, g of rhs.
So you can see how we would get 3, 0 here, because the 3 comes from the heuristic, and everything else was 0, so we just added a bunch of 0's.
So that's everything what happens for the initialization.
And now we have one node on the queue.
So we'll have to dequeue first.
This is an easy question
The node on the queue
The one node on the queue.
So we dequeue G. So we're going out to update its g value.
So as Ben said, the rhs value is sort of a look ahead.
You can also think of rhs as being the shortest path that we found so far, and G is the guaranteed shortest path to that node.
So this one is trivial.
The shortest path from this node to itself has length 0, because its the same node.
So now we're going to update the rhs values and its neighbors.
So it has two neighbors, C and D, and in each case, the rhs value will be the minimum-- this is not actual math, but its the minimum of the edge cost, the d value of a neighbor plus the edge cost.
Which is the same thing as the edge weight or the weight w. So we rhs is 10 here, because it's the distance from g to do, and then it's one, which is G to C. So again, we can calculate keys for those.
So now we'll dequeue another node.
What should we dequeue next?
C. Because it has a smaller key.
So we dequeue C. We update its g value.
So this is saying, now we know that the shortest path to C actually has length 1.
And we can also compare this to A* star algorithm by drawing a search tree.
So in the beginning, we had A*.
In the beginning, we just had node G, because we were searching from G backward through the goal.
And then we expanded that back to get C and D. And then A* to keep track of the path length to that node plus the heuristic at each node.
So the pathway to C was 1 plus the heuristic 2, give us the value of 3.
So if we go back a moment, we can see-- is that OK?
So you can see that this 3 is the same as the first point of the key, and path length one is the same as the second part of the key.
And then similarly at D, we have the path length, which is 10, plus the 2, give us 12.
So this 12 is the 12 here, and then the 10 is the path length.
And so you can see how if we were doing this the A*, you would also dequeue C next.
OK.
So now, we dequeue C, and we're going to update the rhs and its neighbors.
So B is pretty simple.
Previously, it's rhs was infinity.
Now the shortest path to B that we've found so far has length 2.
So here we have B was a path length 2, plus a heuristic of 1 gives us 3.
But what about D?
Previously, its rhs value was 10, but do we now have a shorter path to D, given that we're expanding C?
Or is 10 still the shortest path?
That is the shortest path, through
Yeah.
So now that we're expanding C, we see that you can actually get to D in only two units instead of 10 units.
So that's better.
So we'll update the rhs value to 2.
And that means that we're going to change D's key accordingly.
And the way that maps to A* is, so if you already have the path length is 2, plus the heuristic is 2, gives us 4.
And the fact that we're changing the key to 4, 2 means that we're never going to expand this D, because A* uses an extended set, so it would always expand this D with a shorter cost before this one.
So that's indicated by-- removed by changing the key on the queue.
So what do we do next?
Which node do we dequeue?
B?
Yeah, because it has the smaller key.
So we'll dequeue B.
We'll update its g value, which is again saying that we know now that the shortest path to B is 2.
So we dequeue B, and then we expand it.
How many neighbors does B have?
Three
Three.
So we're actually expanding to all three of those neighbors.
We have A, C, and D. Can we still see this?
We can.
So at A, it's pretty simple.
We have the rhs value is 3, because it's the shortest path through GCB to A.
So that's 3, plus the heuristic 0 plus 3.
What about-- oh, see, it doesn't change, because-- you could go through B and back to C, but-- oh, C shouldn't actually be on here.
Sorry.
C shouldn't be on here, because we don't want a loop in our path.
OK.
So at D, can we do better than this path length of 2?
No
No.
Because what happened was originally, we had the path length of 10 through G, and then we had the path length of 2, 3 C, and this path through B is not as good.
So we're going to keep the same rhs value and key at D. And we can see that in a lot of he A*, where we would have, if we did do GCBD, we would have a pathway of 3, plus the heuristic of 2 is 5.
But we would never actually want to expand that D, because it's not as good as this one.
OK. Now what would we dequeue?
Looks like we have two nodes in our queue currently

A.
So we dequeue A.
That's where we trying to get to, because that's the current start node.
So we set the g value to 3, which means that that's actually the shortest path.
And we're done.
We found the shortest path.
Well, we're done, except that the path might have to change.
So one thing you notice here is that we never have to actually set D's g value.
So it's possible, maybe there's still a shorter path to D, but it doesn't matter.
So now the robot will move to its next best place, which is B, and we'll update the heuristics accordingly, because this is our new start node.
So now, again, our heuristic is the number of nodes away from that green node.
And we're keeping the same items on the queue from before.
But we also have to update the key modifier.
So now what happens is an obstacle appears.
So there's an obstacle at C that the robot can now see because it's moved closer to node C, but it couldn't see it before.
So now, we're going to indicate that by keeping all the edges, but changing their rates to infinity.
And that means that the robot could try to get to see, but it wouldn't ever get there.
So what we do next in the algorithm is we're going to update all of the rhs values associated with the source nodes of these edges.
And in this case, the edges go both directions, so all four of these nodes are affected by the infinities.
So let's start with node C. What would be its new rhs value, given that rhs is the minimum from one of its neighbors to it of the g plus edge cost?
Infinity
Infinity.
because all of the edge costs going to C are infinity.
So what would be the key?
The tricky part here is the first part of the key is the minimum of g and rhs.
So the minimum of g or rhs is what?
So it'd be 3, 1?
Yeah
Yeah, 3, 1, because the minimum of g and rhs is g, which is 1.
The heuristic is 1, and the key modifier is 1.
OK. How about node B?
What's the new rhs value?
So it has a neighbor here with g as infinity, a neighbor here of g is 1, except the edge length is infinity.
And then this guy with g is 3
So it's infinity, 4?
4.
Yeah, because currently, we still think that you can get to A and B units.
So this is actually going to be 4.
So this is one of the sillier things that D* does, but luckily, it doesn't do this very much, so it doesn't actually really cost too much time.
And then how about D?
What would be the new rhs value?
5, 3?
rhs
So it has been 3 neighbors.
This edge length is infinity is useless, so that's useless.
This one has g of 0 plus 10 would be 10.
Yes, so this is the best one.
G is 2, plus edge length of 1.
So that would give us 3.
So now what do we dequeue?
C?
C. So this is one of those tie breaking cases, where both of them have the first value is 3, but C has a better second value in its key, so we'll dequeue C. Is this over or under consistent?
So the definitions of over and under consistent that we had earlier, we had if g is greater than rhs, then that's overconsistent, and we set g equal to rhs when we're updating g. So that was the case that we've seen so far.
But then if g is less than rhs, it's underconsistent, and we set g to infinity.
So what is this?
Underconsistent
Underconsistent.
So the reason intuitively why we're going to set g to infinity is because we're saying there's some sort of contradiction here.
This one doesn't make any sense.
It's less than the rhs value, so we're going to just reset g and sort of start over with this node.
So we set that to infinity.
And in this case, we don't actually need to add the node back onto the queue, because it's now consistent.
It's now locally consistent.
So next, we'll dequeue the one with the smallest key, which is B.
B is also underconsistent, so we're going to set its g value back to infinity.
So this is sort of resetting the weird thing we did earlier that didn't make any sense.
We're going to update its neighbors' rhs values.
So what would be the new rhs value for A, now that we know that we don't actually have a path B?
Infinity?
Infinity.
Yeah.
And then, how about rhs value to D?
Because currently, D got its rhs value by going from A to B to
10
10.
'Cause now the only way-- rhs is 10.
The key would have that one also.
Is that right?
Yeah.
Because currently, the best path to D, and actually, the best path to D you can see is from G to D, because you can't go through C anymore.
So now, we'll dequeue node A, because it has the smallest key.
It was also underconsistent, so we would set G to infinity.
So we'll propagate that to its neighbors.
And if we continue doing this, then we find-- eventually, we dequeue this node, and in this case, we had to dequeue everything, so now the node's are locally consistent.
So what's the best path that we've found from B to the goal?
B, D, G?
B, D, G. And that's also the only path in this case.
And we had to update these neighbors.
OK.
So here's our new shortest path.
Now, the robot will move to node D. But now, what's going to happen-- I think we updated our heuristics.
But now what happens is it turns out it's a moving obstacle, and it starts following the robot.
So now the obstacle has moved to there.
So as humans, we can see that, obviously, the shortest path is through C, but the algorithm doesn't know this yet.
So what would we do next?
So we just updated our edge weights.
So now all the edges to B are infinity.
But some of the edges to C are no longer infinity.
So we need to update the rhs values accordingly.
So we'll update all of the rhs values that were affected by this change.
In this case, G's rhs value never changes because the distance from G to itself is always 0.
And this one doesn't change because we still use the G value here.
OK.
So now again, we can start dequeuing and planning a new path, so we'll dequeue this node.
We'll propagate to its neighbors.
Dequeue that one.
Update the rhs values.
But in this case, we don't-- let me go back step.
So we dequeue this node, and we update its G value, but we don't actually need to continue dequeuing things.
We should-- oh, this rhs value should be updated.
Oh, it is updated.
Why is this rhs infinity?
All edges to it are infinity
Yeah.
All edges to it are infinity.
You can't get there.
But we don't actually have to bother expanding B, because we've already found the shortest path.
So this gives you sort of a hint at when D* might be efficient.
Because even if there were a whole bunch of nodes over there, I wouldn't need to consider them, because we found the shortest path.
So now, the robot can move up to C. And let's say the obstacle chases the robot again.
Do we need to find a new path?
No.
So intuitively, we don't, because the obstacle's not in the way of our current best path.
But we can also see algorithmically why that's the case.
So first of all, we'll update all of our rhs values accordingly, because we've changed all of the edge weights.
But the reason algorithmically why we don't actually need to consider any of these change values is because this node has the smallest key, which means that this is the node that has the shortest path to G anyway.
It is smaller key than all the other nodes that are on the queue.
So dequeuing things wouldn't actually help us find the shorter path.
So the robot can move to the goal, and we've succeeded.
Any questions on that?
OK.
So now then we'll talk about sometimes when you would want to use incremental path planning, other than this five node graph
Thanks.
So as Jess said, there are sometimes good reasons to use incremental path planning, but there are also good reasons not to in certain circumstances.
In certain circumstances, we can actually formulate graph problems, where a problem is worse by using incremental planning.
Remember that I said that D* Lite puts any node on the queue at most twice.
And so we can compare that to A*, or something like that, that does full replanning whenever a search occurs.
That pus every node on the search queue at most once.
So D* Lite is most beneficial when putting the nodes on the queue, potentially more than once, leads to less nodes being examined overall.
So here is essentially what I just said.
So A* might perform better for certain problems if we only have to consider a small amount of nodes anyway.
So putting nodes on the graph twice would actually lead to more expansions.
And so, to get at an intuition of why that might occur if changes are close to the start node, I want to show you an example here.
So here, we're actually showing an example that's derived from [INAUDIBLE].
So we're searching forwards here from the start node to the goal node.
And that's just a different reformulation, so don't let that confuse you too much.
When we walk forward through A* with a good heuristic, we might move directly from the start node to goal node, and find that path.
Now, if a change occurs somewhere upstream of the start node, when we replanning, we essentially have to replan the sections that are beyond the change that's occurred.
So everything that's here, to an extent, it's relatively insensitive to this change.
It could occur that we would have to take an entirely different route.
But for the most part, it's likely that changes are going to couple to our existing path and so forth.
And so we have an efficient search that will just go around from the path that we've already planned to our goal node.
So in this sense, we have an example where incremental path planning would be very efficient.
In this case, which is a different one, we introduce an obstacle very near our start node.
This means that most of our existing path is to charge.
So where we start out from our start node to our goal node, we have to do a lot of recomputation from scratch.
And so an incremental searching algorithm is going to have to undo all searches that we've already done, and then search through almost the entirety of the graph again.
So intuitively, in this case, just restarting with A* from a blank slate, which we would search directly down to the goal, instead of doing reparations that would send us in one path, would be a better idea in this case.
So since we're powering from the start to the goal, in this example that we're showing, it's worse to use incremental path planning algorithms when your changes happen near your start node.
For something like D* Lite, we search back from the goal node to the start node, it's better if changes occur nearer to the start node, which for our vehicles that we normally see that have limited sensor horizons, is what's going to occur in the most case.
So in those type of situations, incremental path planning is going to be very efficient
So I understand how this could be a problem for LPA* and D* Lite over finding a single path, where you have like a single source or destination or something like that.
But what if it's incremental, all-pairs shortest-paths, where you've computed-- I'd assume you wouldn't experience the same problems if your algorithm was computing choice paths between all pairs of nodes indirect
Yeah.
It already exists.
Well, that depends.
When you're redoing recomputation, between all the nodes in the path, it depends if our obstacle has changed [INAUDIBLE] of something there that we to conceive.
And by that I mean imagining this graph we've computed, D, the paths and path costs, to get from every single node to every single other node.
Now, if introducing an obstacle here only changed the path cost to get from our stock node to our goal node from example, then replanning that specific pair of paths from start node to the goal node would use data that already exists from the other nodes to the goal node.
We already have that.
But if introducing an obstacle would cause changes throughout the entire graph, then we can potentially create what would have to be a very well-connected graph, where again, incremental path planning would probably perform worse in that case
Yeah.
So you could still end up in the same situation [INAUDIBLE]
Now these algorithms are interesting, because they allow for some extensions that allow us to do some some very different things quite easily.
One example is greedy mapping, in terms of how can we design algorithm using what we know about incremental path planning, to hit every single node on our graph.
And what we do is then we introduce a faux goal node, essentially a node that doesn't exist that we can never reach.
And we add to our initial beliefs, our initial understanding of the graph connections between all nodes to this faux goal node.
Now, it's false in the sense that whenever we move our start node to one of the nodes that we've examined before, what happens in terms of the changes of the edge costs is removing that edge that connected that node to the goal node.
So what this means in principle is that our vehicle, our poor vehicle, can never reach the goal node.
Whenever it reaches a node that it thinks it can get to the goal node from, that path disappears.
But what that causes it to do is when it's replanning, it finds a path to a node that we haven't visited before, because it thinks there is a path that leads from that node to the goal node.
And so this means that it's efficient, but replanning a root from his current understanding to a node that it hasn't examined before.
And so it can apply something like the D* Lite or LPA* with this formalism in order to search through the graph in its entirety and make sure that in a greedily efficient manner, we hit every single node in the graph
Is this [INAUDIBLE] TSP problem, or the traveling solutions problem?
I'm not very familiar with the TSP problem
You try to-- and you try to cover all the Cs, and try to come up with a route that is the most efficient, and try to visit every known C value once
OK. Then in that case, yes.
It's not going to be globally efficient.
As you said, it's going to be an approximation for it.
In a sense, from each node that you visited, you're searching for the next best choice.
So it's possible you can corner yourself.
But it's a way that we use what we already implemented and the methods that we've already established to tackle a different type of problem
And all this [INAUDIBLE] usually if you have a strategy which is visit the closest unvisited node, instead of making this whole incremental path from this study.
If your policy is, always move to the closest unvisited node, how worse does that get compared to this?
I think it would behave quite similarly.
What we're just trying to show here is that we can implement that problem as an extension if this happened.
It may be an efficient way to do it
[INAUDIBLE] probably like, zigzag
Also, if you're just going to the nearest connected node, if you corner yourself as we showed here, then you could potentially run into issues if your algorithm doesn't then do an extended search for it.
If it only considers nodes in its immediate neighborhood that hasn't visited, once it corners itself and doesn't know where to go
Just keep going through the graph until you hit the next inhibited node, assuming that you know how many invocations you have to do
Sort of like the random walk type approach?
Well, I mean, kind of like the greedy version of it.
You kind of start from a node, and then you keep searching, you'll see, what is the next unvisited node, and you pick that.
Kind of like-- [INAUDIBLE] some you see what you have covered.
You see what's the next best move.
You make that move.
And you're done.
[INAUDIBLE] So you probably don't have this [INAUDIBLE]
This basically ensures you're going to find the shortest path back to the next unvisited node, I guess.
We also can use these incremental path planning algorithms for anytime planners.
And the benefit of an anytime planner is that we quickly reach a plan which is suboptimal.
And then from there, we are repairing this plan to approach optimality.
And this is useful in the real world where you want to get a solution that comes out quickly, and we want to start along our solution, and then repair it as we go.
And once we start it, we need o save some more time if your reparations might lead to small changes and that type of thing.
And this can be handled quite efficiently using an incremental path planning.
So what happens is we apply something, idea, called inflated heuristics.
And so these inflated heuristics are essentially increasing our heuristic values that exist on our graph, in our true graph, by a cost of scale factor.
And this is typically very large, so differences in heuristics become very large.
This means that once we start along a path, we continue along it.
And that causes us to very quickly find the first path that actually works.
Then, we can view reducing those heuristics back to their original values as essentially a form of changing our [INAUDIBLE] causes or-- so it's a form of replanning.
And then by applying the principles of an incremental planner, we can efficiently use the data that we already have in order to perform the search and repair our solution as we go.
Finally, we want to make explicit to you guys how we can actually apply these algorithms to mobile robotics.
Because it may not immediately clear how we go from a full space to a graph.
Joe on this kind of briefly when he talked about we discretize the world.
But we want to give you guys some ideas of how we might do this cleverly.
So in order to build a graph system out of our world state, we differentiate between holonomic systems and nonholonomic systems, which their difference is just in terms of whether your constraints are differential in nature or not.
Then we are going to outline three methods here very briefly, cell decomposition, using a visibility graph, and sampling-based construction methods.
So for cell decomposition methods, we might to do what you would expect, take the free space in this region and split it up into various different cells.
And each of those cells is going to represent a node in our graph.
And how we can do this is we split the region up based on each of the vertices of obstacles.
So we draw a line vertically from each of our vertices that goes in the direction away from the obstacle.
So at the top here, it's going up and out of the obstacle.
But in the lower left corner there, both up and down, don't intercept the obstacles.
We draw them in its entirely.
So we can continue this throughout the entirety of the state space, and then we can attribute to each essential cell that we've defined through these regions, a single point that represents the midpoint of that cell, which is going to be our graph node, because we attribute a single set of coordinates to each of our graph nodes.
And we also, to make sure that our transitions between these cells are consistent, additional nodes at the midpoint of each of these lines that we've drawn, where we make the distinction that from here, we've actually drawn two lines.
So we draw the midpoint between all of those.
This allows us to essentially create a graph through the entire system, and we connect each node at the center of any given cell to nodes that are on the boundary of any given cell.
This allows us to create a full graph through the system, that we can move through, guaranteeing that none of these paths, if we follow them exactly collide with the obstacles.
Now, this works in environments where the obstacles are 2D polygons, and the path is far from optimal, as we can see, but we can plan an optimal path over the graph that we have defined.
And this only works on holonomic systems.
We can also define what is called a visibility graph, where we essentially draw a boundary between both our vehicle down there at that bottom left, and boundaries around our obstacles.
These boundaries are sufficiently sized so that if the center of the vehicle that we're defining is along the extended edge of the obstacle, that the vehicle will not collide with that obstacle.
Then for each of the vertices of the extended boundary that we've defined, we add in a new node, and we draw all lines between all nodes, which can be connected without intersecting the extended obstacles, which we've drawn here.
And this leads to a graph that we can, again, plan over.
This, again, works in environments where the obstacles are 2D polygons.
But what's nice about this system is that we do get an optimal path, provided that we assume that we assume that our extension of the robot itself is valid, or doesn't act too much-- isn't too excessive for constraint.
Finally, going back to some of the random sampling methods that we mentioned at the very beginning of this lecture, we've introduced sampling-based roadmap construction, wherein we choose nodes randomly or deterministically throughout our environment.
The difference here, though, is that we then create a graph out of this system, and that we reason over that graph using something like an incremental planning algorithm.
And so when we create that graph, after we've sampled, we can use the [INAUDIBLE] neighbors of each node, or we can connect each node to every node within a certain radius, which is what we've drawn here.
Finally, in order to actually move between nodes, we need to have some knowledge about the dynamics of the vehicle, which is what we're discussing here.
And one method that we can do this for nonholonomic systems is we can distinguish between different paths that use turning right, turning left, and traveling in straight lines.
It turns out that the shortest path between two points can be defined using one of these systems.
And so we just have to search over a finite number of options in order to find the best path that can connect any two nodes.
Hopefully, that has given you some intuition into how we can apply these incremental path planning [INAUDIBLE] systems.
Are there any questions?
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, everybody.
Today we're going to talk about semantic localization.
So first I'm going to talk about what is semantic localization and what is the motivation for it.
Then we'll go through an algorithm that will allow us to localize.
And then we'll go into how to actually add semantic information into this algorithm.
So our focus what we're coming up with this was the orienteering Grand Challenge.
So in orienteering, basically you have a map and you have a compass, and you have to go around to various checkpoints.
So as you can imagine, this is sort of difficult, because you don't know where you are on this map except what you can tell from your compass.
And so we weren't sure how to do this either, so we asked some orienteering experts.
So basically the key ideas are, if you are disoriented, that's fine.
You just need to find a reference point so that you can figure out where you are.
You can think about where you last knew where you were, and estimate what your movement has been since you were there.
And that can give you some options as to where you are.
And you can look around for features that you can identify uniquely on the map.
And this is an example of an orienteering map.
If you just have what's over here, it's not very useful.
You can't actually tell what those things are.
You might guess that green is grass, but you don't actually know what they are.
This map isn't useful until you actually have a legend to go along with it so that you can say you have roads, or foot paths, or pits, or different things on the map that you can actually identify.
So this is the difference between how robots typically localize and how humans localize.
So humans think about places in terms of features, in terms of what you can do with them, things like that, whereas robots might measure distances with laser scanners to form a perfect map of the entire space, whereas humans might think about obstructions such as rooms and their relative location to each other.
So we can define semantic information as signs and symbols that contain meaningful concepts for humans.
Basically different sorts of abstractions.
And why might we want this semantic information?
Well, one very important application would be improving human-robot interaction.
So say you want a robot that can make coffee.
It has to understand commands like, go to the kitchen.
Get a mug.
Turn on the coffee maker.
This is the language that humans think in, and it's very useful if a robot can actually understand this sort of language as well.
Know what a coffee maker is, what it does, and where it is.
And additionally, you can get some performance and memory benefits.
You're not storing this full map with the distance to every single surface.
You're just storing locations of key information.
And this means that your search base is a lot smaller too.
So you can prune everything that's not related to kitchens or coffee makers, and get a performance that way.
Finally, the fact that you can use simply a camera instead of a complicated laser scanner means that this is a lot cheaper and more accessible for robots to have.
So we've talked about what semantic information is.
So now let's define what semantic localization is.
So basically, it's localizing based on semantic information rather than metric information, like distances.
So for our Grand Challenge, we have a map with labeled objects and their coordinates.
This is our semantic information.
And we want to basically look around and say, what can we see in our field of view?
And from that, we want to figure out where could we be on this map.
And it's important to note that map building is actually a very different and hard problem with a lot of other research on it.
In our case, we've been given a map, and we're simply talking about the problem of localizing based on that map.
And now I'm going to let Matt tell you about particle filters, which is an algorithm for localization
All right, so my name is Matt Deyo.
Now we're going to talk about particle filters.
So this is a tool that we're going to teach you to use, specifically for state estimation, using the given math and measurements.
So what is localization?
We just went over it.
Specifically, it's the question of, where am I?
For any system to work, the robot needs to know where it is on a map.
This is not so simple of an answer.
Metric localization.
As was said before, it is quantitative.
So how many meters are you from this wall or from your origin?
What's your orientation in degrees?
And here's some examples of that.
You can convert between coordinate frames really easily if you have metric.
So the localization is, well, in our case, mathematical.
The problem statement is, you have a control u.
You're going to observe something about your state.
It might be a camera image.
It might be laser scans.
And then essentially on your map, you're going to use probability to figure out where you are.
So this equation specifically is taking into account the probability of being out of position at your current time based off your previous position, the observation that you're taking right now, the command variable you gave it, and then your map, like I said.
So that's built on the observation noise, actuation, and then the belief.
We're specifically looking at the belief here for the particle filter.
So in our case, we're approximating it by several points, also known as particles.
So we're going to look at an example here.
I have to actually press the Play button.
OK.
Here's a quick demo.
[INAUDIBLE] So a quick YouTube demo of a particle filter in action.
So I'm just going to show it first.
So pause.
The red dot here is our actual robot position.
And it's trying to localize itself in a maze.
There's black walls throughout here.
And it looks like a grid world, but that initial distribution we had was completely random across all the walls.
So it's taking in observations of probably laser range finders.
So it essentially knows that there are walls around it, and it knows that there's not a wall in front of it.
So that centers all of these guesses on the center of hallways.
Obvious it got rid of things that were close to the walls.
There's a low probability that it's right up against the wall because of the observations it's taking.
And we're going to revisit this example at the end of the presentation.
Particle filters.
So the method that we're going to teach you has four easy steps.
One is not repeated.
So the initial step is your first sample of particles.
If you don't know anything about your state at the start, then you can sample just a distribution of your entire space.
And then the repeated steps are updating weights, resampling, and then propagating dynamics.
So we're going to show you a toy example.
This is a really simple example just to get the idea across.
We're focusing on just one dimension.
We have an aircraft at constant altitude.
So take that variable out.
The only thing we're trying to localize to is a distance x over a map.
The sensor is just a range finder down to the ground at this constant altitude.
And then the math is a known mapping of x location to ground altitudes.
And we're about to see what that means.
The goal here is determining where you are in the mountains.
So here we have our plane.
As you see, we know the map below, and we know that it has to be flying at this altitude.
So with a range finder, you can measure different depths down to the mountain.
So here we have long distances, a medium distance, and a short distance if it's directly on top of a mountain.
Constant altitude, unknown x.
That's what we're solving for.
And then we actually have a noisy velocity forward.
So we know how fast the plane wants to be traveling in this direction, but obviously with some noise, it could be travelling a little faster or a little slower.
So the first step, sampling.
Our initial belief here is that we don't know where it is with respect to these mountains.
So we're going to actually sample with a uniform distribution.
Well, this is as uniform as I could get it.
So over this range, those were our initial particles.
Essentially guesses for what the x location is.
We're going to update these weights based on the observation.
So we get our first observation from our range finder, and it's this length.
So it's a long length.
It's, I guess, green in this case.
This is our measured value.
Here are all the values that we expect to see at these particles.
So this is pretty easy to calculate, based on the map that you have.
The position of each particle maps directly to a range finder measurement.
So we're going to compare them.
So the likelihood of getting these.
So measuring the likelihood of each of these occurring.
And we're actually weighting the particles.
So based on what we measured, these are most likely, so they get a larger weight.
And we are most likely not on top of a mountain at this point.
So those get smaller weights.
So we're going to resample, given that.
So we're going keep the same number of samples the entire time.
That's just how this particle filter works.
Except we're going to resample across our new distribution.
So these are the weights that we just came up with.
And we're going to resample over them.
So now it's more likely that it's here, or here, or way over there.
And then the final step is propagating.
So this whole time that we've been filtering and updating our weights, the way forward.
So we need to take that into account.
So we have a forward velocity.
Let's say this range is 0 meters per second, to 5 meters per second, to 10 meters per second that you can see on this plot.
So we're most likely moving five meters per second.
So essentially, this is your change in time between sensor measurements.
Obviously if you're only getting sensor measurements at 10 hertz, then you can propagate in between each of those.
So here we are.
Using our new sample particles, propagating them forward.
For example, this one moving there is a low likelihood.
That's a large distance.
And these moving at a shorter distance is more likely, given our model.
So we have those new ones.
The new weights are now based in the probability of it transitioning to that point.
How likely is it for the plane to move that far, essentially.
And then we repeat.
So we're repeating the steps again.
We're going to get a measurement in from our range finder.
We're going to compare it to the map, to our particles, keep the ones that are most likely and weight them more, and then propagate them.
So as an example, here's time step 2, when we're measuring the uphill of this mountain.
Time step 3, now we're halfway up the mountain.
So positions that we've kept are at similar heights, so here and here.
And then we can slowly get down to differentiating it.
So now that we're on top of a mountain, the only pattern that matches that is here, and maybe over there.
And then eventually, we get down to these two.
And then eventually, as this mountain drops off, it's a unique position, where it goes farther than that one did.
So in the end, our particle filter thinks that we're right around here or here.
And finally, there's a pretty high chance that we're over that valley.
So we'll looking at this, again, now that you know how to do the filter.
And this will make a little more sense now.
So they started with the uniform distribution.
They had no clue where they were at the beginning.
And as the robot moves forward, they are resampling.
And the measurements are changing, obviously.
Because it's seeing these two walls, and eventually, it sees that there's doorway to the left.
And you can keep going forward as well.
So eventually, that geometry doesn't line up with any other spot on the map.
Here, we see it nose into this hallway.
I think this top hallway is the only one on this map that's that long.
But it still don't know which direction it's going.
It hasn't been able to differentiate that yet, but it's about to.
So the only remaining particles are here and here.
It knows it's in the middle of a hallway.
And it knows it's moved about two blocks now without seeing a wall directly in front of it, which that doesn't occur anywhere else, without having another turn-off.
So it's about to solve itself.
Because eventually, it sees this wall over here.
And those observations don't match up with what it's actually seeing.
So there we go.
There's a [INAUDIBLE] particle filter successfully working for a little robot with a rangefinder.
I went too far.
There we go
I'm David Stingley.
And now, after we've gotten an idea of why we want to use semantic localization, and how to use particle filters to get an idea of our guesses for initial positions, we're going to talk about how we can use these two to actually implement semantic localization onto a robot.
So hearkening back to the implementation idea, we have three important parts.
We have a belief representation.
We have the actuation model.
So once we have a location, how are we going to move?
As we said before, if you don't exactly know how fast you're moving, there's a probability you move to a bunch of different locations.
And then finally, we have an observation model, which is the probability that you see some certain observation, given you're in this newly simulated position.
If we continuously solve for our most probable x, then that's our location.
So that particle that is the most probable is the place that we guess we're going to be.
So let's look at a pseudocode for what a semantic localization would look like.
So as long as our robot is moving, we're going to make observations.
And those observations are going to be [INAUDIBLE] z1.
Then for those observations, we're going to start off our particle filter and guess a certain number of probable locations.
We're going to use our actuation to update it.
And then we're going to simulate observations at that location, and compare what we expect to see for that particle on our map versus what we actually saw that we made our observation.
This is going to be our update.
And this is what we're going to focus on understanding, since a lot of the rest of this is covered by a particle filter.
So what kind of models can we pick, because we need to define what our observation looks like.
And you get a lot of choices.
In metric localization, you'd use a labelled laser scan.
You'd have perfect information about the environment.
And so you can see everything.
It might be nice to use a scene of objects at specific locations, but that requires once again, a lot of information.
Now you need to know where the objects are.
You need to have an idea of its exact specific locations and orientations with respect each other.
Maybe it might be nice just use, like, a bag of objects.
I see four things in my view pane versus three.
These are all choices, and we're going to take a look just really quickly at what the facts of these are.
As we've stated before, there's a lot of choices in observation.
It can get complicated really quickly.
So just to impress that upon you, imagine if you used laser scanners when you have three objects here-- for instance, a house, a couple of trees, and a mailbox.
You check each line for an intersection.
And then you have to figure out what counts as your detection, since you're going to have to differentiate what's in your scene.
The problem with that might be is that, well, you saw an entire wall, for instance.
Where do you want the house to be?
So let's say we assume that objects were a point at some point.
You either completely see it or you completely don't.
That way, you don't have to half-see an object.
You just check to see if whatever you want your center of mass to be intersects with your current view plane.
If it does, then you're good.
The issue with that is that, for instance, something center of view isn't inside the scene anymore, you just completely ignore it.
We have the exact opposite problem.
So you might want to make it a bit more complex and have polygons, parts of objects.
Do you see some percentage of something?
How much of it is in the view plane?
It adds in a lot of chance for errors.
And that's, I guess, the big point here to remember, is that depending on how we characterize our observations, we have different ways to get things wrong.
So let's say, for instance, for an observation like distance and bearing to some new scenic object, what can be wrong?
Well, you have noise inside of your sensors.
Your sensors might have limitations.
You can't necessarily see to infinity.
So an object might be too far away for you to identify correctly.
It might be rotated in a way that you can't necessarily interpret it correctly.
How about if you want to have your observations in classes of objects, so of just everything being an obstruction, now you have different types of obstructions?
Trees are different from mailboxes, of course, in which case you have a classification error.
What if you see a tree and you just think it's a really, really big mailbox, or you see a mailbox and you think it's a really small tree with a funny, little, metal top?
These kind of errors can then make your scenes look incorrect.
If you decide to have sets of objects, well, what permutations matter?
If you don't have a way of differentiating elements in the set, then you don't know if two trees and a mailbox with the trees on the left and the mailbox on the right or vice versa aren't the same thing?
So with that in mind, we're going to be talking about how we're going to solve this question of given some position and your synthetic view, how likely is it that it's your actual observation?
And for that, we're just going to change what this probability statement looks like to make it a little more concrete.
In this case, Z is the set of observed objects that you have.
So we're going to say that there are objects in the scene, and we put them inside of this value, Z.
So for instance, you see a house and you see a mailbox.
That would be Z.
Your set of objects is just two things.
We're going to use the bag of objects approximation.
Y of x is going to be the set of objects you expect to see given your map for the position that you're at.
So given a position X, Y is a set of objects that you would see.
So you might be in a position where you can see a house and a mailbox.
Then Y is is a house and a mailbox.
And X is just going to be your position.
That's the element that we're getting from our particle filter.
So in our example, as you can tell by how much I talk about it, we're going to use just two things, trees and mailboxes, because I like them.
So here's my map.
There's going to be a long road, and off to the side, we have trees and mailboxes.
And let's say that your robot wants to be a paper delivery boy.
So he needs to be able to figure out where the mailboxes are and how far down the street he is.
Because if he were to throw the paper wrong, he'd throw a paper into a tree.
So in this world, our robot is going to be here, represented by an orange hexagon.
And it has this field of view.
So given this field of view, what does Z look like, our actual observation, one tree and one mailbox?
Well, for this case, what we're going to say is, we're assuming that as long as the thing intersects with our field of view, we're going to see it.
So simple finding that.
We're just going to say that we see both trees and this mailbox.
Once again, we talked about how difficult it is to do this observation.
So for a simplifying assumption, let's say that we just completely see the tree.
So that's great for our actual robot position.
But when we start spawning particles, we need to figure out what we're going to say they synthetically do.
So say we spawn a particle here, and we spawn one when we're just slightly off the road.
We deviated a little bit.
We're further forward than the actual position.
We drove into somebody's house, another guy's house in the woods.
Then what is each of synthetic observations for these given points?
Determining this is going to determine how we actually get that probability calculation.
So what do we need to consider here?
Well, the first thing we need to consider is classification.
Like we said before, with this set of objects approximation, it's important that you understand if you classify things correctly or not.
Past that, like we asked before, we said, wait, why didn't I just see a tree and a mailbox?
Well, we need to know if we saw everything inside of our field of view.
Maybe we did miss something.
Maybe instead that old scene saw just one tree, one mailbox, even though the other tree intersected it.
And noise can happen in reverse, so we could accidentally see a tree when there actually isn't one.
And finally, we could see things overlap.
We kind of ignored this before, but what if two trees were right on top of each other?
It might make it kind of difficult for you to successfully see that there are two trees there instead of one.
So to start with, we're going to strike this assumption.
It will become more evident later why this is important.
But for now, we're just going to assume that every observation corresponds to only one object being seen.
Otherwise, you could end up infinitely expanding your scene.
Think about it.
You see a tree.
There might be a possibility that that tree is two trees.
Well, you saw two trees, then.
So maybe there's a probability that each of those two trees is also two trees.
And you keep going and keep going, until the entire forest is behind one tree.
It would be kind of bad for doing probability calculation, because you'd eventually have to cut that off at some point, so that your algorithm actually finishes.
So for our purposes, we're just going to cut it off at the start and say that everything's just one object.
It's just error.
So now we need to talk about if we classify it correctly.
If we can solve this equation, what's the probability that our classification is right?
So for now, we're going to make two simplifying assumptions.
They're going to remove the two problems that we had before.
And don't worry, we'll relax them later.
For our first assumption, we're going to assume that we see everything inside of our field of view.
So that means we're not going to have any misconceptions.
And we never see something that doesn't exist.
So we have no false detections.
Everything that's in the scene, we see successfully.
If it's not in the scene, we don't see it.
So given that, and we spawn a robot here, and it has this field of view, What does this robot see?
Three trees
Yes.
It happens to see three trees.
So remember our assumptions.
We're going to say here that our actual observation for wherever our robot is is one mailbox, and two trees.
And we can see that the synthetic robot that we made saw three trees.
So what are some other forms of Y that we can make that would also map to this?
What's the way that we can take our Y and transform it so that it maps this Z?
What kind of action would we have to take on this?
If you were thinking that we'd have to misclassify one of these trees, you're correct.
And remember, this is just a set of objects.
So this doesn't have to be the first tree that got misclassified.
There's three of them.
We could have any permutation of these trees get misclassified.
So it becomes important.
And we're going to introduce this concept of this operator, phi.
And what phi is going to do is it's going to be a way to map the permutations that we could have of misclassifications for Y to look like Z.
That way, we don't have to try and write all the permutations down.
It's possible to do this essentially with a matrix, like a permutation matrix, that just reorders the elements that you have.
So what does this probability look like now?
We're going to use the lower case z, y and i to represent each individual element of those sets.
So for some element in the actual observation, z, what's the probability that some element in our synthetic observation matches it?
Well, for that we need to pick-- well, there's some probability of being wrong or a misclassifying, or classifying correctly.
So we're going to use c to represent a classification matrix.
So there's some probability that we classify correctly, usually a higher probability, and then some small probability that an object becomes another type of object.
So how often we misclassify is represented here.
But we could add another term.
Let's say that our classification engine gave the weighted values of things.
It's common for neural nets and other types of systems that observe images to kind of give weights to their classification.
So you might want to use those weights to represent our confidence.
And there's a problem that that confidence determines that our classification might just be wrong out the get-go.
In that case, we want to know what's the probably that the score is likely to be that type of object.
And then we could have other things.
For instance, let's say that our classification was way better at identifying mailboxes from the front.
And if we turn the mailbox to the side, it gives poor observations.
And it tells us if it thinks that the mailbox is on the side, so it doesn't really know.
In that case, maybe having the bearing of the object inside of our synthetic view allows us to determine another probability for misclassification.
The important thing to notice here is that we can keep adding more terms to this.
The more specific your classification can get, the more terms you can introduce into it.
So add some confusion to it, and make the probabilities of misclassification smaller, and smaller, and smaller.
So with that in mind, we're going to look at what the probability for these entire sets is going to look like.
And what is really is going to be is it's just going to be a product over all these classifications for some selection of phi.
So you're going to have to take all the different permutations that you could possibly have, and for each of them, you're going to multiply all of these probabilities by each other.
In the end, it's going to give you that entire probability.
So it's all the permutations that you have, and then just the probability of each of those objects being classified as that type-- that you just map one set to the other set.
So now let's take a look at another scene.
So we spawned a particle above, fake robot here.
And it has this field of view.
What does this field of view look like?
If you said it looked like two mailboxes and two trees, you're right.
And so we're going to assume that we have the same actual observation.
We still see a mailbox and two trees.
And we're going to remove our old assumption.
We're going to say that it might be possible that we don't see everything in a synthetic robot's field.
In which case, how can we amp this synthetic observation to the actual observation?
Well, we just add in a probability that we miss identifying an object.
If we just didn't see one of these mailboxes, it looks exactly like this.
So for that, we want to capture what's the probability that we see nothing?
So say we have a synthetic view with some number of objects.
What's the probability we just miss all of them?
So the probability that we miss all of them, we're going add an assumption here, which is that we're going to say that there exists some probability that we see an object for a given synthetic view here, and that we don't see it with a probability one minus that probability.
So essentially, there's just two states-- either we see the object or we don't see the object.
And we're going to say that probability of identifying objects is independent.
So if we see one object, then that does not change our chance of seeing the next object.
Both of these assumptions help simplify the math here.
In reality, these might be strongly interlinked.
For instance, if your robot's camera is broken, the probability that it doesn't see one object directly correlates with the other ones, because it won't see any of the objects successfully if it has no camera any more.
If your robot drives into a wall, and can't see anything because it's staring straight at the wall, the same kind of idea holds.
But for the purposes of making it so that you don't have a lot of covariances, and a really, really big conditional statement of a lot of probabilities of seeing things, we make these assumptions for our items to be independent.
Independence means we can multiply our probability successfully.
So if we just don't want to see any of the objects, we just take 1 minus the probability of seeing the object for all the objects in the scene.
So what goes hand-in-hand with not seeing anything?
Think about if you had a robot far off in the distance over here.
What can it see?
Just two trees.
So now, we're going to remove the idea that we can't see things that don't exist.
So to make two trees map to two trees and a mailbox, we just made up a mailbox of some noise.
So what's the probability that we see a full scene when we can't see anything?
And if you're thinking it's going to map to a very similar formula, you're kind of right.
But we need to figure out a way to capture our noise statement.
Specifically, we are going to use noise as a Poisson variable, which means that there's always some probability of seeing an object out of nothing.
And it's going to be coordinated and according to this factor Kz we'll get to on the next slide.
You could choose a lot of different things to represent your noise in this case.
A Poisson variable was just chosen by the specific method that we used and implemented from another paper.
But with testing, you could find that different potential distributions map to your noise for your particular sensor better.
So with a Poisson variable, what we're going to have is we're just going to have the product of all of our Poisson variables times this Kz factor for the given scene that we want to map to.
Essentially, it's just the product of all these independent Poisson variables for each of our different objects that we want to create.
So with that in mind, what's this Kz factor that we're multiplying everything by?
What it's actually going to be is it's going to be a set of uniform distributions.
These uniform distributions are going to be uniform over all the classifications we could get for an object we spawned from the noise, and all the possible scores, and all these possible bearings for this synthetic object.
And if you remember a few slides ago or you rewind the video, you'll notice that these map directly to the categories that we put into that classification engine.
In fact, they should.
So if you added more things or took them out, you changed this uniform distribution.
The idea is that when you synthetically create an object out of noise, you might get any of those types of objects.
But if you needed to create a tree synthetically from seeing it in the noise, you might get a tree or you might get a mailbox.
Either one of them might show up.
If you had a different or more intelligent distribution for how you might misclassify things-- for instance, let's say your noise, whenever it shows up, always identifies trees.
Whenever you accidentally see noise as an object, it's always a tree.
Then you might only want to have a probability of seeing trees over here, and it would change you distribution.
But for simplicity's sake, we're going to assume that it's equally probable that any type of object shows up out of noise.
So now, we're going to put it all together, since we relaxed our assumptions.
So we have a lot of different things that can potentially map to this actual scene.
We have scenes that lack objects.
We have scenes that lack objects and therefore, need to add the object in.
And when they lack objects and have to add the object in, there's a chance they add in a different object.
And then maybe, they just misclassify something, like our misclassification idea.
But wait-- I guess we can make this more complicated.
What if one of them just wasn't seen, so we'd take one of these out?
And then we just see two things from noise.
As you could see, these keep getting more complicated.
But if you think about it, with our previous probabilities, is every single one of these is going to add a lot of probability terms to be multiplied by each other.
You can keep getting as complicated as you want it to.
If we removed our first assumption, we'd be adding in probabilities of two objects becoming one object.
The idea is like higher power terms when you're doing approximations.
The goal is to make sure that any particle you spawn could potentially be what you've actually seen.
But we want the ones that are very low probability to really be very low probability.
So we make sure that these incredibly complicated transformations are going to be so insignificant that they'll usually return to almost 0.
So if we wanted to solve this, we need to start by using a little assumption, which is the fact that we now have to fold in the idea that we could have some missed detection and some false detections.
So false detections would increase the number of objects we've seen over the number of objects that are actually presence.
And missed detections would reduce the number of objects that are actually present.
Notice that this always has to be same size as the number of expected objects, because you're trying to map whatever your synthetic scene is to the actual thing that you observe.
When you do this, it really turns into a large number of multiplications.
So this should look familiar.
This is our term for successfully classifying all of our objects, and we're going to multiply it by the probability that we actually identify it, since we could now miss things.
Then, we have to multiply that by the probability that for whatever objects we didn't see, that we actually missed them.
And then finally, we have to add in the noise, because the noise is going to make the objects that we're missing show up, so that we have a classification of the same size as the thing that we want to see.
Now, one thing you should note is that we added in phi way down here at Kz.
That's because we have to map these false detections that added to the number of objects that we could see, as well as the actual detections.
Essentially, we have to take all the objects that are seen here, real or not, and map them to all the objects we expect to see, because they're going to be one to one.
So let's take a look at a little video that shows what semantic localization can do.
So in this video, for a little heads up, we have essentially a scene where they have mapped out two things inside of a suburban area.
They've mapped out cars and windows on the path of the robot as it drives around an area.
And they're attempting to use just the information of the cars and windows visible on the scene in order to localize where they believe the robot to be during its journey.
[VIDEO PLAYBACK] - This is a video extension to the paper--
So going to mute that.
It's on.
So this is a few steps into the process.
And then it resets.
It's spawn a number of points for potential locations.
And then it very quickly localizes and finds its spectacular rotation.
These are all the identifications that are happening inside of the scene.
You can see the boundary boxes for cars and windows showing up, the cars in red, and the windows in green.
It expands out its distribution of particles.
And then shortly thereafter, it gets a couple of approximations.
And then it settles in on a location.
It happens to use a kind of like a centralized weight for where it is for the set of particles, and then it draws the car as being in that location.
If you let it keep running, it occasionally expands back out to make sure it doesn't fall into a local minima, and then compresses once again to where its strongest belief is.
Notice that it has a couple of seconds of having a very, very spread distribution, and very quickly converges into a singular point.
That's because a lot of these locations become very, very low probability after you've seen a couple of scenes into the future.
And we're going to pause this.
[END PLAYBACK] I welcome you to go see the video, to see the video yourself if you want to.
You can check the title.
It's pretty easy to find on YouTube, because it's pretty much the only one.
So hopefully, it works now.
Step back over here to the other side.
So why would semantic localization be useful in this manner?
In the example that was shown, it was kind of done on post-processed data.
They took a scene.
They drove around it already.
Then they did the identification on the video feed, and used that to do a localization.
We talked about before.
But people kind of use sparse information to do a lot of their-- or if you can't walk into a room-- people don't walk into a room and produce an exact laser scanned map of the entire area themselves.
But they can store important information, like seeing objects and tables that they find with their shins, and use those to move around.
Robots store that perfect map, if they can manage to make it.
But they don't really have a good understanding of what that map might mean to a human.
So that means that we're actually kind of like better at doing tasks with the environment.
If we wanted to go directly to a location, we don't have to look around and figure out on a wall what our distance from the wall is before we can then localize and move ourselves back over to the table.
We just say, oh, I want to go to the table.
And you look over there.
Oh, there's a table.
Table, got it.
So how can we make robots think a little more like humans, so it's easier for us to give them instructions?
If we can make the robot use a map that has just scenic objects like we use a map that just has scenic objects, then order to move between scenic objects are as simple as turning, finding the object, and saying, oh, well, I know where the object is.
So I know roughly where I need to be.
And then we start moving in process towards it.
In conclusion, semantic localization has a lot of-- I'm going to leave the references slide up while I talk, so that if you wanted to go and take a chance to see any of these papers, you definitely can.
Most of the work in this slide comes directly from these sources.
Semantic localization offers us the opportunity to take robots, use sparser information to potentially localize, find ourselves inside of spaces.
It also gives us a chance to have really tweakable factors for how you might understand where you are in a space intuitively.
If you think certain things are important, you can add them in as probabilistic factors.
If you don't, you remove them just as well.
Thank you.
And this is essentially the conclusion of our presentation.
I definitely recommend taking, if you wanted to use something like this, taking a look at this particular paper here.
Notice how it says, "via the matrix permanent?"
As I said before, a lot of these operations are a series of multiplications and some permutations.
There's permutation matrices.
There's matrix multiplication.
And there's a lot of ways to make matrix multiplication faster.
This paper details how you can take the math that was shown before that's pretty inefficient, and turn it into something that you can do an estimate of very quickly.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so as this XKCD comic points out, in CS, it can be very difficult to figure out when something is just really hard or something is virtually impossible.
And until a couple years ago, people thought this idea of image classification would be something that was closer to the impossible side.
But with the advent of deep learning typology, we've made significant strides in image classification, and now the problem's actually quite practical.
So today we'll be going through how the process of image classification with deep learning works.
So we're first going to talk about what deep learning is, and then we'll move into some of the image processing techniques that researchers use, followed by the architecture of the convolutional neural networks, which will be the main focus in our presentation.
We'll also talk about the training process, and then go through some results and limitations of CNNs and image classification.
So what is deep learning?
Well, the term is particularly vague, and it's purposely so for a couple of reasons.
The first is mystery is always good for marketing.
But the second reason is that deep learning refers to a pretty wide range of machine learning algorithms.
They do have some commonalities.
They all seek to solve problems of a complexity that previously, people thought only people could solve.
So these are more sophisticated classification problems than traditional conventional machine learning algorithms can do.
So how do they go about doing this?
Well, all of these deep learning programs tend to take all the processes that need to happen, and split them up.
They've got different parts of their program working on different things, all while performing calculations, and then at the end, it all comes together, and we get a result.
Of course, this isn't unique to deep learning, and lots of distributed systems decentralize their calculations.
But the key thing about deep learning is that every part is performing these calculations.
The calculations are not simple calculations.
They're not we'll do this one simple operation over, and over again on a lot of data, and then we'll get a result at the end.
Each part is performing some particularly complicated process on all the little parts before they come together.
So why is this architecture a good idea?
Why did engineers come up with this sort of decentralized, multi-layered complex process?
Well, we take the example of image classification.
It turns out that the human brain does a pretty similar process.
So here's the human visual system, and it's pretty much a hierarchical process.
So you begin by moving from the retina into the first areas of the brain, and as the information gets processed, it moves from one region of the brain to the other, and each spatial element of your brain is performing an entirely different calculation.
For example, the v1 area over here is picking out edges and corners, and then over here, a couple steps later in v4, you're starting to group those figures together.
And so the brain kind of operates in a way that is very similar to the way these networks operate.
So let's talk about to classify a face.
If I asked you guys how would you classify a face, what is the first thing you might do?
Well, as I mentioned before, the first thing out brain does is it finds these edges.
The first thing to do is identify where the face is versus everything else.
Now, does anyone have any idea as to what we could with the next step?
Julian, you have an idea?
Maybe you could group these edges together
Right.
We could maybe identify some of these features that we're working with.
So these are things like noses, and lips, and eyes.
And then what do we do after we have these individual features?
Steve
Well, maybe we can group some of those together
Exactly.
Yeah, we can organize them into what we know the pattern to be.
We know that a face has to have two eyes, above a nose, and then above the mouth.
So that is precisely what a neural network actually ends up doing, and we'll walk through the process of how it does this later on in the talk.
But as you can see, the intuitive way that we classify a face, and the way our brains are wired to do it, is pretty similar to the way got these neural networks to operate.
So like I said, we're talking a lot about these convolutional neural networks.
There are other types of architectures involved.
Like we mentioned before, deep learning is a pretty wide variety of algorithms, but we're going to focus on these CNNs.
To give you a precursor of how good these CNN are, this results from ImageNet competition.
So the ImageNet competition is basically exactly what it sounds like, a bunch of computer scientists get together, and see how many images they can correctly classify.
And their error rate was pretty high.
Almost a third error rate over here in 2010, 2011, and then in 2011, the CNNs were introduced to the topic, and the error rate plummeted.
As you can see over here in 2015, we've got a significant improvement in these ImageNet competitions.
So clearly, the CNNs have been very effective, and it's definitely been something that is exciting in the field and happening right now.
All right, so now we're going to move into image processing
OK, so Ryan gave us a nice overview of where we get this concept of neural networks, but let's take a time travel, and go into a quick history lesson.
So suppose I had a chair, and I wanted the computer to classify this chair.
I have some a priori knowledge about what sort of things make up a chair.
So I might be interested in looking at arms, and corners of the chair, legs, things like that.
So I would go ahead, and feature engineer my discovery scheme to be looking for specific things.
So I'm going to talk about some techniques that are traditionally used.
For example, chairs, doors, these things have corners.
So I might use an image processing technique called a Harris Corner Detector, where we basically look at large changes in intensity as groups of pixels move from an image to an image that indicate the presence of corners, and you can use common corners to say that OK, all of these images are chairs, or doors, or whatever.
Similarly, I want to say I have a bunch of pictures of chairs of different sizes, but that they all must have so many corners or something.
So typically, we use a sift algorithm to scale invariant feature transforms.
It basically says that across different sizes, I still should be able to extract information about the placement of corners.
Another common technique that's used in image processing is what we call HOG, Histogram of Gradients.
So basically, for example, in this image, if I want to say I want to find all the images that have faces in them, or consist of faces, let's say, I might come up with a template of a face that basically assigns gradients to groups of pixels that form an outline of what looks like a face, and then scan it across my sample images, and say OK, a face is present in this image.
Obviously, there are some errors.
A mead cap, and a logo back here have become a face, but this is the traditional approach.
But here's the problem, what if I don't actually necessarily know what features are the most critical depending on the dataset that I get?
I want the system itself to figure out what techniques to apply without having any a priori knowledge about the dataset.
So this is exactly the idea of CNNs, the convolutional neural networks.
We want the techniques to be learned automatically by the process, by the system.
So if I'm trying to classify faces, I want the system to figure out that eyes, and ears, and nose, these are the most important things.
Or if I'm trying to classify elephants, the ears and trunks are the critical features, without me having to say OK, we're going to do corner detection, so on, and so forth.
So this is the idea.
So to be able to understand this process in greater detail, I'm first going to go into a little bit of math, and the idea is to present the most fundamental operation here, which is the convolution.
So this is the formal definition of the two-dimensional convolution, and since we're working with images, we're only considering the two-dimensional case.
So in a more graphical presentation, which is a little bit easier to understand than just seeing the formula, the idea is that we have a kernel, or a convolutional filter that we seek to apply on another image, and that extracts some information about that image that we can use to help us classify the convolution.
So assume that this is our kernel, or this is our filter, and suppose this-- oh, there it is.
So suppose we're applying the kernel [INAUDIBLE] here to the image that's in green.
So the idea is we want to slide this filter across the image, and what we're basically doing this is a succession of dot products.
So at each placement on the image, we multiply the overlayed numbers, and the sum becomes the output image on the convolt output.
So this is basically the way the process works.
You probably notice that there's a reduction in dimension, and Henry will talk a little bit more about why this is.
Let me get to it, and then [INAUDIBLE] So let's see some examples of what information we get by applying the convolution.
So you see the image of a tiger on the top left.
When we apply a filter that's a low pass filter, basically-- it's a Gaussian-- then we get low spatial frequency information about this image.
So basically, we blurred it, and this tells us something specific that we might want to learn.
So the kernel actually looks like a two-dimensional Gaussian function that's been distributed across this three-by-three kernel.
Similarly, we might be interested in high spatial frequency information.
So in this case, we're looking at sharp features.
So horizontal edges or vertical edges.
So a question for you is if I have this kernel, which of these outputs when this kernel was applied to the original image, which of these outputs do you think it produced?
The third one on the right
Yeah, that's exactly right, and it's probably pretty easy to see why that's the case, given that the numbers are all horizontal bands here.
Lastly, we also may be interested in extracting information at a particular frequency.
So we can take the difference of a high pass filter and a low pass filter, and add it to your frequency you can extract information about that as well.
OK, one last helpful piece of information is that there's another way that you can think of the information that's learned at each stage because a convolution can also be thought of as a Fourier transform in the frequency domain.
You can think of the image transformation that way.
So from an image perspective, what a Fourier transform is is basically a sum of a set of sinusoidal gratings that differ, say, in frequency, in orientation, in amplitude, and in phase.
So you can think about the zebra image here that's actually a composite of different gradients that might look like this, and the Fourier coefficients would be how much of each of these pieces come together to make that final image.
So just to get a sense of what kind of information this could convey, we typically take a Fourier transformation, and break it apart into the magnitude and phase representation.
So you see magnitude, and you see phase.
So those images weren't particularly clear, but this is a really good example for this.
So if we take the Fourier transform of all the horizontal text here, you see how the magnitude reflects this, and you can go back to the math to understand why it's reflected in a vertical marking.
And similarly, if I were to take that same image, and rotate it, and then ask for the Fourier transform, you see how that information is contained very clearly in the magnitude spectrum.
So these might be things that a network would learn at each stage to try to identify this as a text, or as as body of text that's tilted one way or the other, so on and so forth.
So with that, we can now go into what the actual architecture of a convolutional neural is
All right, so as it was said earlier, in order to classify or detect objects, you actually need certain features.
You need to be able to identify these features.
And the way you can identify these features is using certain convolutions or certain filters.
In many cases, we don't know what these features are, and as a result of that, we don't actually know what the filters are to extract these features.
And what convolution neural networks allow us to do is actually determine what these features are, and also determine what the filters are in order to extract these features.
Now, the idea for convolutional neural networks, or the idea for replicating how the brain works started in about 1960s or 1950s after some experiments by Hubel and Wesel.
And what happened in these experiments, as can be seen here, is a cat was actually shown a light band at different angles, and the neural activity of the cat was measured using an electrode.
And the outcome from this experiment show that based on the angle at which the light was shown, the neural response of the cat was different.
As you can see here, the number of neurons, as well as the neurons that were firing were very different based on the angle.
So what you can see also here is really a plot of the response versus the orientation of the light.
And what this has led Hubel and Wesel to is the idea that neurons in the brain are organized in a certain topographical order, and at each filter, it fills a specific role, and the only fires when its specific input is shown, or when the angle is show, or the angle is specified.
Now, the first step to actually replicating how the brain works in code is really understanding how the building block, the neuron, works.
That's a quick reminder of 7012 here.
So a neuron is actually a cell with dendrites, nucleus, axon, and a terminal.
And what the neuron actually does is aggregate the action potentials or the inputs it gets from all the neighboring neurons that are connected to it through the timelines, and it sums these action potentials, and then compares them to a certain threshold that it has internally, and that would determine whether or not this neuron would fire an action potential.
And that very simple idea can actually be replicated in code.
An artificial neuron looks very much like a natural one.
So what you would have is a set of inputs.
Here we have three inputs that are summed inside of a cell, or a neuron.
The sum here is not just a regular sum, it's a weighted sum.
So the neuron specifies some weight, which you can think of as how much it values the input coming from a specific neuron, and then the input is multiplied by its weight, and then the total sum that the neuron computes is then fed into an activation function that produces the output that the neuron then basically produces.
Now, what we just saw here is really a simple neuron, a single neuron.
You can't really do much with just one neuron, so what you would do is combined these neurons in a certain topography, or in that case, we have a network with seven neurons organized in three different layers.
And what you can think of that is really as one big neuron with 12 inputs, and one output.
So for example, in the case of the chair that was previously mentioned, if you're trying to identify whether a specific image has a chair in it or not, these 12 inputs here could be some sub images, or some small areas of the initial image that you feed into the network, and the output here could be a yes or no.
Whether the image has a chair, or doesn't have a chair.
And that is really the concept behind convolutional neural networks, which we'll go into details in a bit.
So what each neuron would be doing in that case, is really just performing a dot product, which if you aggregate that with the dot products computed by each of the other neurons, you would obtain a convolution.
So what we have here is three inputs.
If the input, in that case, is an image or a sub image, then the inputs would be pixels.
The weights that you would be using here would be the filter weights, which is the filter that you use in the convolution.
And then the sum here would be the dot product of the weights and the inputs, and that sum would be computed by a specific neuron in your network.
Then, that would be the convolution step, and then that convolution step would happen at the first layer in the network.
So you would be applying this to the input, but you also would be applying this at the second layer, and third layer.
In that case, we're only showing what happens in the first layer.
The next step after the convolution would be the activation step.
So the dot product computed here would be there's a function that would be applied to the sum, and then that function would produce the output of the neuron.
And this is where the activation layer is.
You also have another activation layer here, and then a final one here.
What we just went through now are convolutions and activations, but this is not the only thing that actually happens in a neural net.
What we also have is a step called subsampling, which we will be talking about next.
For now, we will dig deeper into the activation, and specifically, what activation functions to use.
In that case, we can see that that's a neuron, and what the neuron is doing here is the weighted sum that we talked about, or the dot product.
And then the output from this would be fed into a certain activation function.
Common activation functions are sigmoid, tanh, or rectify linear unit, and we will go through each one independently.
So here, we can see the sigmoid activation function.
So what this function essentially does is map any input to an output in the range of zero to one, and it's defined as one divided by one plus e to the minus x.
The other common activation function is tanh, and that's any input to an output between minus one and one.
And then finally, would be the rectified linear unit, which maps an input to itself if it's positive, or to zero if it's negative.
Now, in theory, you could use any function as an activation function in your network, but that's not what you want to do in practice.
You want your activation functions to be non-linear for one main reason, that the goal of the activation function is actually to introduce non-linearity in your system.
And if all your activation functions are linear, then you would essentially be having a linear system, which prevents you from achieving the level of complexity that you would ideally want to achieve with a neural network.
And there's a formal proof for as to why you need non-linear activation functions.
They don't all need to be non-linear, but you need to have a least a few non-linear activation functions in your network.
And the proof is available in the appendix, or the link to the paper that has the proof.
So after we've discussed what happens at the activation layer, now we want to talk about convolution.
So as I said earlier, an image is obviously a two-dimensional image, but we're using RGB images.
So actually need three channels.
So what this means is that an image is actually three-dimensional, and each 2D matrix represents one channel.
One of them corresponding to R, one of them corresponding to G, one of them corresponding to B.
So a 32 by 32 image would essentially be represented by a 32 by 32 by three matrix, as can be seen here.
So what happens at the convolution layer?
So here we have a nice animation that shows what is happening at each convolutional layer.
So assume we have a five by five by three filter.
So what this is, essentially, would be doing is covering a certain patch in the original image, which is 32 by 32 by three.
So what can see here is that for that five by five by three patch in the original image, we have a neuron that is actually performing the dot product on all the pixels in that specific patch.
So what is happening here is the pixel values, which in that case, we have five by five by three pixels, are being multiplied by the filter values, and this operation is being performed here.
Then, after that dot product is performed, it's fed into an activation function, as can be seen here, and this produces the output of this neuron.
Now, this is what this single neuron is actually doing.
It's just covering that area of the original image.
What you would have in a neural net is many neurons, each covering a certain area of the original image.
And if you aggregate the output of all of these neurons, what you would be doing or performing is, essentially, a convolution on the original image.
And to formalize what happens here, or what's the output that's being produced from that operation, we can look at that from a more mathematical perspective.
So if you have an input of size H1, W1, D1, and you're performing a convolution with a filter, then the output, W2, would be related to W1 with the following formula.
So W2 plus W1 minus filter width plus one, and the same formula applies for the height, and the depth would actually be the same because in that case, we're using a filter that has the same depth, or three, as the original image.
So what this would produce in aggregate is if you have 28 by 28 by one neurons, each one performing a dot product on some pixels in the original image, the output would be an activation map of size 28 by 28 by one, and the output of each neuron would be one pixel in the activation map.
Now, if we go back to the points that we made earlier, one thing we said was that the reason you use a neural network is because you don't know exactly what features you want to extract, and you don't actually have specific filters that you want to apply to the image.
So ideally, what you want to do is have multiple filters being applied to the first image, and perform multiple convolutions, and this is what you can do with multiple neuron layers.
So what we described before was just for one neuron layer.
In that case, we can assume we have five different neuron layers, each one performing a different convolution on the original image.
So in that case, we would have 28 by 28 by one neuron per layer, and then if we aggregate all these neurons together, we need to multiply it by five, and that would be the total number of neurons we have in that specific number.
So this actually leaves us with a pretty complicated system.
It would have many parameters.
The neurons have weights, the number of neurons is also a parameter So how do we actually formalize that?
If we have an input volume of 32 by 32 by three, which is our original image, and a filter size of five by five by three, then the size of the activation map that would be reduced would be 28 by 28 by one.
Then in that case, we also said we have five different neuron layers that perform five different convolutions.
Then the total number of neurons would be 28 by 28 by five, and then the weights per neuron are five by five by three, which is 75.
In that case, we're assuming that the neurons independently keep track of their own weights.
This could be simplified to each layer having their own weight, which would tremendously reduce the number of parameters.
But in that case, just to get an upper bound, this leaves us with a total number of parameters of 294,000.
And this is just using a 32 by 32 by three image.
You can think of this as a pretty small image.
So if you have a bigger image, you will have many more parameters.
Great.
So what we just saw now, and described, were convolutions, activations, and these steps happen sequentially in a convolutional neural network, specifically as can be see here.
One step that also happens occasionally is subsampling, and we'll discuss that step in detail here.
So there are two main reasons why you would actually subsample your input.
One is to obviously reduce the size of your input, and your feature space, but also because you want to keep track of the most important information, and get rid of everything else that you don't think is going to be relevant to your classification.
And the common methods used in subsampling are either max pooling or average pooling.
We will describe max pooling here.
So what happens in max pooling is, essentially, you are dividing the image into different sub images, non-overlapping sub images, and you perform an at max operation.
So in that case, if we consider two by two filters, we would split the image, which in that case is four by four.
We'd split it into four sub images, and for each two by two square, we would take the maximum.
In that case, for the first square it would be six, then eight, then three, then four.
And the reason that actually works is because what you want to do is really keep track of the response of the neurons that-- or the highest response produced by your neurons.
In that case, for example, the first highest response in the first square is six, and that means that if you get that high of a response, it means that something has been detected in the image, or has been detected.
And this is something you want to keep track of as you move forward in your network.
And although this moves around the location of pixels, because you can think of that as subsampling an image, it does keep track of the information you care about because you only care about the fact that something has been detected in the image.
At this point, you don't really care about where it's located in the image, and you want to keep track of all the features that your neurons have detected in order to be able to eventually classify the input correctly.
So if you have multiple feature maps-- so in that case, if you have 224 by 224 by 64, what your subsampling operation would be doing is reducing the height and the width so the depth would remain unchanged.
So in that case, you would go from 224 by 224 by 64 to 112 by 112 by 64, and that would be reducing your output size by a factor of four.
And formally, what this would look like is if you have an input of size H1, W1, D1, the size of your output would be related to your input in the following ways.
W2 would be W1 minus the pool width plus one.
The same applies for H2, and the depth would remain unchanged.
So these are, essentially-- these are the steps that happen in a convolutional neural network.
What you could be doing is repeating these steps on a certain number of times in your network.
But eventually, you have to make a classification, and decide in our case, whether our image has a chair or doesn't have a chair.
So how does that happen?
So after you perform all these steps, there's a step that happens here that would allow you to make that prediction, and that step is usually called a fully connected layer, or a multi-layer perception.
And what this essentially is is layers that are very similar, or exactly the same as what you had before, except that every neuron in the layer is connected to all the previous neurons.
So what it's allowed you to do is really consider everything you currently have about your input, or everything that's left about your input, and compute a dot product on that, rather than focusing on a subset subsample of your input like previous layers do.
In that case, if you're actually trying to classify your input into four classes, you would ideally have four different neurons in your output layer, each one corresponding to one of the classes that you have, and then you would perform the same operation as you would in a previous layer, compute the dot product, and then once you obtain the values at every neuron, you would perform a normalization operation on all the output.
This organization operation is called softmax, or normalized exponential, and what it does is really, put more weight on the highest value.
And by computing the softmax at the output, you're able to compute the posterior probabilities, and allows you to make a more informed-- or basically make a classification decision on your input.
Great.
So that's everything.
And now, the next step will be talking about back propagation
OK.
So now that Henry has given us an overview of the entire architecture of a CNN, I'm going to quickly spend some time, and talk about standard preprocessing tricks and tips that people might use on the image dataset before they actually feet it through a neural net to classify the images.
So let's suppose we have a dataset x, and there are n number of data points in the dataset, and each point has a dimension, D. So they have D features per point.
So in this example, we use these graphs as an example.
Basically, our original data here has just two dimensions, and it spans this range of values.
So for example, if we want to center this data, what we would do is a mean subtraction.
So we basically subtract the mean across all the features of all the points, and we basically center it, and you can see that transformation here.
And then we might, again, go for normalizing the dimension so that you have it.
The data points span the same range of values in both dimensions.
So you can see that transformation, and how it's taken place here.
And we just divide by the standard deviation to do this.
Something that's very commonly done is called PCA, or Principal Component Analysis.
And the idea here is sometimes we have a dataset that has a very, very high dimensionality, and we would like to reduce that dimensionality.
So basically, what our goal is is to project the higher dimensional space onto a lower dimensionality space by taking the subset of those features.
And if you've seen a little bit of 1806 from linear algebra, the way we do this is by generating a covariance matrix, then doing the single variable decomposition.
And I'll gloss over the math now, but that's the idea.
And you can see here how the original data spanned two dimensions.
I would decorrelate it so that it spans a single dimension.
And even with this data, you might want to ensure that it's widened, which is the same deal.
You want the values to span the same range in both dimensions.
So then you would just divide by your Eigenvalues to get the widened data.
This last bit is something that's very commonly done as a preprocessing trick, though people aren't entirely sure why it works very well, or that it really does help, but it's something that people do, and it's called data augmentation.
So basically, if I have a dataset that contains a bunch of images of chairs, a bunch of images of tables, and then a bunch of images of say, trees, I might want to intentionally augment that dataset further by introducing a few variations on these same images.
So I might take the chair image, rotate some, reflect it a few more, scale, crop, remap the color space, or just kind of have a process that does this randomly to create more variation on the same dataset.
And this is a good illustration of why this makes a difference.
I've taken an image here of what looks like a waterfall or some spot of nature, and simply just inverted the colors.
And what I see, if I were to just see this image alone, it maybe looks like a curtain, or a bit of texture, or something.
And the idea is even to a human perception, these images have two very different meanings, and so it's interesting to see what effect they would have on a neural network.
And with that, we'll go over to image classification results
So, so far we've seen how convolutional neural networks are built, and certain image processing techniques we can use on the input images to get them into formats that are there for the classification process, but so far, it seems a bit abstract.
It's good to know how CNNs work, why CNNs work, but why don't we take a look at some of the practical results from CNNs, and what they're used for so that when you're done watching this lecture, you can go home, and try classifying images on your own time?
With that, let's first revisit the ImageNet competition.
I hope you remember the graph at the bottom from the beginning of the lecture, where we used this to motivate the use of CNNs.
CNNs came onto the picture in 2012, but the winning CNN from 2012 was used on the 2010 ImageNet competition as well, and it managed to bring down the top five error rate to 0.17, which is pretty much on the same level as how performed in the 2012 competition when it was first used, which was at 0.16.
So this just goes to show that these convolutional neural networks are the state of the art when it comes to image classification, and that's why we're currently focusing on that.
But you might be wondering what the ImageNet competition exactly looks like, what the images looks like.
So why don't we take a look at that.
As you can see here, these are images from the ImageNet competition.
Underneath each image is a bold caption, which is considered to be the ground truth, or what the competition believes to be the correct classification of the image.
Underneath that ground truth, you see a list of five different labels.
Now, these five labels are produced by a convolutional neural network, and the different bars-- the different lengthened bars, some pink and others blue, represent how confident the CNN is that what it sees in that image is that specific label.
As you can see in certain examples, the CNN is pretty confident in that it has a correct answer.
For example, when we look at the container ship, it's pretty confident that what's in that image is exactly a container ship.
There are certain cases when it doesn't get the correct label on its first try, but it does have in its top five labels.
For example, you can see grill and mushroom.
Now, the funny thing about the mushroom image is that what it thinks the image should be classified as is agaric.
And if you don't know, agaric is actually a type of mushroom, and in fact, it's a mushroom that image.
And it make sense that their confidence levels are pretty much the same.
Agaric is slightly-- it's slightly more complex that what it sees in the image is agaric.
But there are certain cases when the CNN fails to classify the image correctly in its top five levels.
This will be registered as a top five error, as you just saw in the previous slide about the top error rate.
One example here on this slide is cherry.
Now, the ImageNet competition believed that this should be classified correctly as cherry, even though there's also a Dalmatian in the background.
The CNN, on the other hand, is pretty confident that what it sees in this image is the Dalmatian.
But if you look at some of the other results within the top five, although it doesn't guess cherry at all, it does guess certain fruits that it may think look sort of like cherries like grape or elderberry.
So the CNN does actually pick up on two different distinct objects within the image, but as a result of how it's built, or its training set, it ends up classifying it as a Dalmatian.
But it goes to show you that CNNs could also be used not just as an image classification, but also as object detection, which we do not touch up on in this lecture at all.
So I'm not going to go further into that.
Now, this is all fun and all, but what about some real world applications?
So this is a study that they did at Google with Google Street View house numbers, where they used the CNN to classify photographic images of house numbers, as you can see here, of certain examples of these house numbers-- what they look like.
So what the CNN was tasked with doing was that it was supposed to recognize the individual digits within the image, and then understand that it's not just one digit that it's looking at, but it's actually a string of digits connected, and successfully classified as the correct house number.
This can be quite challenging, even for humans sometimes when the image is quite blurry.
You might not exactly know what the house number exactly is, but they managed to get the convolutional neural network to operate around human operator levels.
So that corresponds to around 96% to 97% accuracy, and what that enables Google to do is that they can deploy the CNN such that the CNN automatically extracts the house numbers from the images online, and uses that to geocode these addresses.
And it's gotten to a point were the CNN is successfully able to do this process in less than an hour for all of the street view house numbers in all of France.
Now, you might asking where this could be useful for.
If you don't have access a lot of resources to actually do this geocoding process where you match latitude and longitude to street addresses, then your only resource might be actually photographic images.
So you actually need something, hopefully not human, but some sort of software that can do this successfully.
And so this is, for example, a place in South Africa, a bird's eye view.
Not sure if you can exactly see, but there are these small numbers on top of each of the houses.
All of these numbers were extracted and correctly classified using this previously seen CNN.
Another example from robotics is recognizing hand gestures.
So obviously, robots come equipped with a lot of different hardware.
They can sense sounds, they can also capture images of their surroundings.
And if you're able to classify what you see-- if the robots is able to classify what it sees, then it can actually act upon it, and take certain actions.
That's why it becomes really helpful to successfully classify the images.
So this is what they did using hand gestures, where there were five different classes.
Each class corresponds to the number of extended fingers.
So a, b, c, d, the top row, corresponds to the same class.
They all have two fingers sticking out.
The bottom row has three fingers sticking out.
So that's another class.
And they get the error rate down all the way to 3%.
So 97% of the time, the convolutional neural net correctly classified the hand gesture.
And you can use these hand gestures then to give certain commands to a robot, and it can train the CNN to act upon something else besides hand gestures.
For example, if it's in some sort of terranean and you train it on certain images that you might find in nature, then it can take those classifications, and act upon it once it sees, for example, a tree, or some sort of body of water.
It's all thanks to image classification.
Now, obviously, gestures are not necessarily static.
You could be waving your hand, and so that would require a temporal component.
So it's not just an image you're looking at, but a video.
And so if follows that we can probably extend image classification into video classification.
After all, videos are just images with an added component, specifically time.
Obviously, the added temporal component comes with a lot of additional complexity.
So we're not going to dive into any of that, but in the end, it comes down to the same thing.
You extract features from the videos, and you attempt to classify them using convolutional neural nets.
So why don't we look at a study done, again, at Google, where they extracted one million videos from YouTube, sports videos, with somewhere between 400 to 500 different classes, and they used CNNs to attempt to classify these videos.
Now, they used different approaches.
They used different approaches, different tests, different types of CNNs that I'm not going to go into.
But as you can see here, these are certain stills from these videos where the caption highlighted in blue is what the correct answer should be, and underneath it, the top five labels that the convolutional neural network producers.
The one highlighted in green is supposed to be the correct answer.
So you can see on all of these, it gets it within the top five, and for the most part, within the top two, and it's pretty confident when it does get it correctly.
Now, when I said that they use different types of classifieds, some of them were more stacked classifieds, where they were just trained on stills within these images, while others were what they called fusion ones, where they sort of add the temporal component by fusing in different stills from these photo images together.
Now, the current accuracy rate-- the best one they've achieved so far-- has been around 80% accuracy within the top five label.
Now, 80% accuracy is nowhere near what we saw with the ImageNet classification, where in 2015, they had managed to get it up to 98% or 99% accuracy.
But obviously, there's way more complexity involved in this.
So it makes sense that it's not quite there yet.
But it does provide a good benchmark, and something to improve upon in the future as well.
Now, that being said, convolutional neural networks do come with certain limitations.
They're not perfect.
And so Julian will now talk about the limitations
Thanks, Ali.
So Ali talked about the ImageNet competition, and talked about how the recent winners have been convolutional neural nets.
So before, the best was about 26% top five error rate, but now they've actually gotten it down to a 4.9% top five error rate, and that was-- the winner of that competition, the 2015 one, was actually Microsoft.
They've got the current state of the art implementation, and so because it's the ImageNet competition, that means they can identify exactly 1,000 different categories of images.
So there are few problems, actually, with the implementation, or just in general with convolutional neural nets.
So one of them is that 1,000 categories, well, it may seem like a lot-- ImageNet is actually one of the largest competitions-- that's not actually that many categories.
So it doesn't contain things like hot air balloons, for instance.
So these things that children would be able to classify, the neural nets actually aren't able to, even in the biggest competition.
And each of these categories also requires thousands of training images, whereas you could show a child a couple examples of a dog or a cat, and they'd be able to, generally, get a feel for what a dog or a cat looks like.
It takes thousands of images per category for the neural nets to learn, which means that the total number of images you need to train for the ImageNet competition is over a million.
And so this leads to very long training times, even with all of the heavy optimizations that Ishwari was telling us about like how efficient convolution is, it still takes weeks to train on multiple parallel GPUs working together to train the net.
There's actually a more fundamental problem with neural nets as well.
So here on the left, we have a school bus, some kind of bird, and an Indian temple.
And all of these images on the left side are actually correctly identified by convolutional neural nets.
But when we add this small distortion here in the middle, that doesn't change any of the images perceptively to the human, this actually causes the neural network to misclassify these images, and now all three of them are ostriches.
So that's a little weird.
How does this work?
How did we find those distortions.
So here on the left side, we see how a neural network typically works.
You start with some images, you put it through the different layers of the neural network, and then it tells you a certain probability that it is a guitar, or a penguin.
So it classifies it, and so we can use a modification of that method by applying an evolutionary algorithm, or a hill-climbing or gradient ascent algorithm.
We take a couple of images, and we put them through, it classifies it, and we see what the classification is.
And then we can do some crossover between the images.
So we take the ones that look a lot like what we're training for in the guitars or penguins, in this case, and we take the features of those that identify very strongly as a guitar, and we combine those together in the crossover phase.
This is for the evolutionary algorithm.
Then we mutate the images, which is making small changes to each one, and then we re-evaluate by plugging it back in through the neural network, and only the best images, the ones that looked the most like a guitar or penguin, are then selected for the next iteration.
And this continues until you get to a very high identification rates, even higher than actual images of the objects.
So using gradient ascent, these are some of the images that you could produce if you start with just a flat grey image, and then you run it through this algorithm.
So here on the side, we have a backpack, and we can actually see the outline of what looks like a backpack in there.
And over here, we have what looks like a Windsor tie right here, but all of these objects-- and perhaps, there are things in these other images, but they seem to be lost in the LSD trip of colors here.
So that's kind of strange.
That's definitely not how humans do it.
So let's try a different method.
What if instead of directly encoding, which is where we change individual pixels, what if we change patterns in the images, like different shapes?
Then this is the kind of output that we get.
So in the upper left, we have a starfish.
So you can see that it has the orange and blue of the orange in the starfish, and also the blue of the ocean of the environment that typical images of starfish are taken in.
And you can also see that it has the points, the jagged lines, the triangles that we associate with the arms of a starfish.
But the strange thing here is that they're not arranged in a circular pattern.
They're not pointing outwards like this, like we would expect of an actual starfish.
So clearly, it's not latching onto the same large scale features that humans do.
It's actually just looking at the low down features.
Even though it's a deep neural network, it doesn't grab onto these abstract concepts like a human would.
So the reason for this problem, or at least why we think neural networks aren't as good as humans at things like this is the type of model.
So a human would have more of what's called a generative model, which means if we have examples here, these dark blue dots, say, of lines, a few examples of images of lines, then we could construct a probability distribution, and say that images that fall somewhere in this region are lines.
And over here, we have a few examples of giraffes, say, and so anything that falls in this region would be a giraffe.
And so if you had a red triangle in here, that would be a lion.
But if the red triangle is instead over here, it actually wouldn't classify at all.
We wouldn't know what that is.
We would say that's something other than a lion or a giraffe, but neural networks don't work the same way.
They just draw a decision boundary.
They just draw lines between the different categories.
So they don't say that something really far away from the lion class is necessary not a lion.
It just depends how far away it is from the decision boundary.
So if we have the red triangle way over there, it's very far away from giraffes, and it's just generally closer lions, even though it isn't explicitly very close to it at all, and that will still be identified as a lion.
So that's why we think we're able to fool these neural networks in such a simplistic way or in such a really abstract way.
So the main takeaways from our presentation, and the salient points are that deep learning is a very powerful tool for image classification, and it relies on multiple layers of a network.
So multiple processing layers.
Also CNNs outperform basically every other method for classifying images, and that's their primary use right now.
We're currently exploring other uses, but that's generally where it's at, and this is because convolutional filters are just so incredibly powerful.
They're very fast and very efficient.
Also back propagation is the way that we train neural networks.
Normally, if you were to train a neural network that has a lot of layers, there's actually an exponential growth in the time it takes to train because of the branching when you go backwards because each neuron is connected to a large number of neurons in the previous layer.
You get this exponential growth in the number of dependencies from a given neuron.
By using back propagation, it actually reduces it to linear time to train the networks.
So this allows for efficient training.
And even with back propagation and convolution being so efficient, it still takes a very large number of images, and a long time with a lot of processing power to train neural networks.
Also, if you'd like to get started working with neural networks, there are a couple of really nice open source programming platforms for neural networks.
So one of them that we used for our pset was actually TensorFlow, which is Google's open source neural network platform, and another one would be Cafe, which is Berkley's neural network platform.
And they actually have an online demo where you can plug in images, and immediately get identifications.
So you can get started very quickly with that one.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so today's lecture-- we're going to be talking about probabilistic planning later, and in these cases where you're planning a large state spaces is very difficult.
You do the MVP planning.
It could be stress that activity planning, or the likes.
But you have to be able to figure out how to deal with these state spaces.
So Monte Carlo tree searches is one of the techniques that people can identify, over last five years, is having an amazing performance improvement over other kinds of sample-based approaches.
So entity is very interesting from that standpoint.
And then if we [?
link it to ?]
the last lecture, then the combination of something, we just learn about [INAUDIBLE] and combine it with search, is very powerful, in this case, through the state-of-the-art techniques for that, as much as tree search [INAUDIBLE] later [INAUDIBLE] PROFESSOR 2: Good morning, everyone.
As Professor Williams just said, we are going to be talking about Monte Carlo tree search today.
My name is Eann and I'll be leading the introduction and motivation of this presentation.
By the end of this presentation, you will know not only why we care about Monte Carlo tree searches.
As Professor Williams said, there's so many algorithms out there.
Why do we care about this specific one?
And second, we'll be going through the pros and cons of MCTS, as well as the algorithm itself.
And then lastly, we will have a pretty cool demo on how it's applied to Super Mario Brothers and the latest Alpha Go AI that built the second best leading Go player in the world.
So the outline for today's presentation is, first, we're going to talk about pre-MCTS algorithms.
There are other algorithms that currently exist out there, and just a few of them to lead into why we do care about MCTS and why these other algorithms fail.
And second, we'll talk about Monte Carlo tree searches itself with Yo.
And lastly, Nick will tell you more about the applications of Monte Carlo tree searches.
So the motivation of these kind of algorithms is we want to be able to play games and we want to be able to create programs to play these games, but we want to play them optimally.
We want to be able to win, but we also want to be able do this in a reasonable amount of time.
So these three can train itself leads to different kinds of algorithms, and different algorithms with different complexities and time, or times to search.
And so that's why today we're going to be talking about Monte Carlo tree searches.
And you'll figure out in a few slides why we do care.
So these are the types of games we have.
You have this chart where there's fully observable games, partially observable games, determinstic, and games of chance.
And so today, the games that we care about are the games that are fully observable and deterministic.
And these games are games like chess and checkers and Go.
And we'll also be talking about another example with Tic-tac-toe.
So these pre-MCTS algorithms include deterministic, fully observable games, like we said earlier.
And the idea of this, and the nice thing about these games, is that they have perfect information, and that you have all of the states that you need and there's no opportunity for chance.
And so the idea is that we can construct a tree that contains all possible outcomes because everything is fully determined.
And so one of these algorithms, to address this, is the algorithm Minimax, which you might have heard before.
And the idea of Minimax to minimize the maximum possible loss.
That sounds a little weird in the beginning, but if you take a look at this tree, this red dot, for example, is the computer.
And so in the computer's eyes, it wants to beat its opponent.
And we're assuming the opponent wants to win also, so they're playing their best game as well.
And so the computer wants to maximize his or her points, but also knowing that the opponent, or the human, wants to maximize their own win as well.
And so in the computer's eyes, it wants to minimize the maximum possible lost.
Does that make sense to everyone?
Yes?
OK. And so in the example of Minimax, we're going to start with a connect, or a Tic-tac-toe board, where the computer is this board right here, and the blue Tic-tac-toe boards are the states that the computer finally chooses.
It's anticipating the moves a human could play.
So if you take a look up here, here's the current state of the board.
The current state of the board.
And the possible options for the human are this guy, this guy.
Nope.
Possible options for the computer, we have three different options.
And so you'll notice that this is clearly the obvious winner.
But in the state of Minimax, it goes through the entire tree, which is different from depth-first search.
It goes through the entire tree until it finds the winning move and the minimize of the maximum possible points it could win.
So is there a way we can make this better?
Yes.
I'm sure you've heard about pruning, where, in our human intuition, it makes sense.
Well, why don't we just stop when we win, or when we know we're going to have a game that allows us to win?
And so this idea is the idea of simple pruning.
And so when we combine Minimax and simple pruning, we have-- anyone know?
Alpha, beta.
PROFESSOR 3: Yes.
Our 6.034 head TA knows about this.
We have alpha-beta pruning, where we prune away any branches that cannot influence the final decision.
So in other words, you wouldn't keep exploring the tree if you already knew that a previous term would allow you to win.
And so this idea in alpha-beta pruning, we have an alpha and a beta.
And so the details aren't important for you to know right now, but the idea is that we stop whenever we know we don't need to go on any further.
So in the games that have Tic-tac-toe and Connect 4 and chess, we have relatively low branching factor.
So in the case of Tic-tac-toe, we have 2 to the fourth branching factor.
But what if we have really large branching factors, like Alpha Go?
In Alpha Go, we have 2 to the 250.
Do you see that Mini Max, or even alpha-beta pruning, would be an optimal algorithm for this?
The answer is?
No.
PROFESSOR 3: No.
And this leads us to out next section.
Our goal is going to talk about how we can use the Monte Carlo tree search algorithm for games with really high branching factors, and using the random extension to allow us to see, ultimately, how Alpha Go, which is Google's AI, was able to beat the leading Go player in the world.
PROFESSOR 3: All right, guys.
So this is the part where we re-explain the algorithm itself.
And before we dive into this, I want to make something really clear, which is that because these are technical details and because we actually want you to understand them, and because I definitely didn't understand this the first three times I read the paper.
I really want you to feel free to ask any questions on your mind, with the knowledge that, in my experience, it is very rare that someone asks a question in class that's [INAUDIBLE] OK, so really, whenever you have one.
OK.
So why are we doing this?
Well, the ideal goal behind MTCS is that we want to selectively build up different parts of the tree.
So the depth-first search way, the exhaustive search, would have us exploring the entire koopa tree, and that our depth is limited by looking at all the possible nodes of that level.
But what we want is we want-- because the amount of computation required for that explodes really quickly.
With the number of moves that you're basically looking into the future, we wanted to be able to search selectively in certain parts of the tree.
And so for example, if there are less promising parts over here, then we care less about looking into the future of those areas.
But if we have a certain move-- in chess, for example, there's a certain move where in two moves, you're going to be able to take the opponent's queen.
You're really want to search that region and figure out whether that's going to end up being a significantly positive group for me.
And so the whole goal of our algorithm is going to be growing this asymmetric tree.
How does that sound?
OK, great.
So how do we actually do this?
We're going to go over a high-level outline, but before we do that, let's talk about our tree, which you're going to get very familiar with.
Can people see that this is red and this is blue?
So this is our game state when we start our game.
We can be given a Tic-tac-toe board with a [INAUDIBLE] place, a game of chess with the lose configured a certain way.
And so our player, which is the computer, has three separate moves that it can take.
And so each of those moves are presented by a node.
And each of those moves have response moves by the opponent.
So you can imagine that if one of these is a Tic-tac-toe board with just a circle, that one of these is with that circle and the next place right by it.
And as you go down the this tree, you start understanding basically, it's the way that humans think about playing these games.
If I go here, then what if they go there, and then what if I go right here.
You try to think through the set of future moves and try to evaluate whether your move will be good in the long term sense.
They way that are going to expand our tree, as we said, to create an asymmetric tree is first of all, we're going to descend through the tree.
We're going to start at the top and we're basically, jump down some sequence of branches until we figure out where we're going to place our new node, which seems like a key operation here.
To create an asymmetric tree it's all about how you [INAUDIBLE].
For example, in this case, we're going to pick this sequence of nodes.
And once we get to the bottom and find every location, we're going to create a new node.
It's not very hard.
Then we're going to simulate a game from this new node.
And this is the key part of MCTS.
Once you get to new a location, what you're going to be doing then, is you're going to be simulating a game from that new location.
We're going to talk about how you go about simulating a game from this more advanced game state that what we started out with.
Does anyone have any questions right now?
We will be going in depth into all of these steps, but just in a high level sense
Just a quick question.
PROFESSOR 3: Yeah
To create the new node, is it probabilistic, just creating a new node as the most probable [INAUDIBLE] PROFESSOR 3: No, no.
You're creating some new node.
We'll talk about how we pick that new node, but we're just making a new node and we're not thinking anything about probability.
The next thing is that we're going to update the tree.
So whatever the value of the simulation delta was-- delta, remember-- we're going to propagate that up and basically add that to all of the nodes that are in that parent of that node in the tree and update some information that goes in there and that they're storing.
This is going to be good because it's going to mean that-- it's a lot like in search algorithms where you have trees that then the entirety of the tree remains up to date with the information from every given simulation.
And we're just going to repeat this over and over and over again.
And slowly, our tree will grow out until whenever we feel like stopping.
This is actually one of the nice things about MCTS, is that whenever we decide that we're out of time, like for example, if you're in a competition playing a champion Go player, you can stop the simulation.
And then all you have to do is pick between one of the best first moves that you're going to make.
Because an the end of the day, after you're doing all the simulation, we're still right here.
And we're still only picking between the movies that go immediately where we started.
Yeah
Could this [INAUDIBLE] good tree?
And then on some initial region of interest, or is it arbitrary how you get to create it?
PROFESSOR 3: We'll go through how you pick where to descend right now.
I guess, it's any possible move that starts at your starting game state.
Does that make-- great.
Before we move on to the algorithm itself, let's talk about what we store in each one of these nodes.
So now we've added these numbers.
And these numbers represent is that nk, as in the value of the right, is the number of games that have been played that involve a certain node.
So for example, if I look this node, that means that four games have been played that involve this node.
A game that has been played that involves the node just means that one of the states of the board at some point in the game was the state of the board that this represents.
For example, if I have a game that was played here, if I know that I've played this once, then that guarantees to me that I played this game once because this is a precursor state to this one.
Make sense?
Yeah
How can the two n's below that node not add up to a value of [INAUDIBLE] PROFESSOR 3: That will come when we start expanding our game.
But that's a great question.
And intuitively speaking, it should
You're saying you're storing data from past games about what we've-- PROFESSOR 3: Yes
--done before
If past game's outside of the script simulation?
PROFESSOR 3: No, no, no.
Past game's in the script simulation.
And then the other value is the number of wins associated with a certain node.
And these are going to be wins for player one, which is red in this case.
It would get confusing if we put both of them, but they're complementary.
So for example, three out of the four times that the red player visited this node, they won in that node.
And these are the two numbers that we're going to store.
And we're going to see why they're significant to store later.
So first, descending the key part of our algorithm that we're talking about.
And when descending, there are these two counterbalanced desires that we have.
The first of them is that we want to explore really deeply into our tree.
We want to think about, OK, if they do this then I'll do this.
And then, well, then I'll do that unless I want it to forth.
And we want to think through a long term strategy.
But at the same time, we don't want to get caught in that.
We want to make sure that we're not missing a really promising other movie that we weren't even considering because we were really going down this certain rabbit hole of the move that we had thought about before.
This is illustrated by the x case [INAUDIBLE] SMBC.
The SMBC comic about academia and how someone tells you that a lot of really great work has been done in an area, that means nothing about how promising the future will be.
It's all about expansion and exploration.
And the way that we're going to balance expansion and exploration in order to create our really nice asymmetric tree is the following formula.
And it's fine if that looks really confusing and messy.
But actually, it breaks down quite nicely into two parts.
This formula is known as the UCB.
You don't need to know why it's the Upper Confidence Bound.
Let's just talk about what's inside it.
So first of all, you have this term on the left.
And this term on the left is the extension term.
It's basically proportional to the likelihood that the expected number of times that you're going to win, given that you are in a certain node and that you were a certain player.
It's basically the quality of your state in some abstract level.
If we knew this perfectly, then we would be doing great because that's the thing we're looking for on some grand level, The expected likelihood of winning from a certain state.
On the other hand, you have this exploration term.
And you may not be able to read the font there.
But what this is basically saying is that it looks at the number of games that I have been played through, and it was the number of games that my parent has been played through.
And it tries to preserve those numbers at a certain ratio, at a log ratio.
And what that effectively means, is that the number of times that I have been-- if I have been visited relatively few times, and the denominator is small.
Whereas my parent has been visited many times, which means that my siblings have gotten much more attention, then the likelihood that I will be visited again actually increases.
So this is biased on the one hand, towards nodes that are really promising, and on the other hand, towards nodes that haven't been explored yet, where there's a gold mine and all you need to do is dig a little bit, potentially.
We don't actually have an analytical expression for this.
But we can approximate it because you can think that the expected value from a certain node is, roughly speaking, approximately the ratio of wins at that node to the ratio of times that that node has been visit at all.
Let's talk about actually applying this statement.
Because what the statement is going to give you, is it's going to give you some number for here and some number here, and some number for here, and so on.
When we start descending through the tree, we're going to start at the top node.
And then we're going to look at the three children of that node.
And we're going to compute this UCB value for each of these children and pick whichever one is the highest.
So just as a thought for a moment, what if we ignore this one?
And what if we're just computing the UCB of these two?
Does anyone have any intuition on whether the UCB would be higher for this node or for this node?
The left node.
PROFESSOR 3: The left node?
OK.
So why is that?
It has a win [INAUDIBLE] PROFESSOR 3: Yeah.
It has a win
And they both have a [INAUDIBLE]..
PROFESSOR 3: Exactly.
And so clearly, you think the exploration term is the same because you know it's not that one child has been loved less than the other, but the expansion term is going to be different.
And so it's definitely going to pick this one.
In this case, what we're going to say is actually that this is so much more promising than the others that it's actually going to pick this left node.
And so it's going to expand, and it's going to look down.
And then when it looks down, it's going to compare between these two.
And this time, remember, that this is a parent.
A parent want to minimize the number of wins that we have.
Which means that our opponent is going to want to pick the one that were less likely to win in and they're more likely to win in.
This is the idea of mini-max, minimizing how well my enemy does in this game.
Although again, the expiration term might counterbalance it a little bit because, technically, this has been explored more.
We're going to pick the one on the left again.
And we're going to get to that location that we got to originally.
Now when we're comparing between these two, between a node that has been visited once and a node that has never been visited, can anyone guess which one of these it is going to pick?
Yeah
Never has been visited.
PROFESSOR 3: Yeah, exactly.
Because this number is zero.
And so if the parent has ever been visited but the node hasn't, this is going to be infinite and it's going to have to pick the node that it has never seen before.
So that's how we descend through the tree.
Does anyone have any questions on that.
Really, it's totally fine.
We're going to be talking about this for a while.
Yeah
With the left node that has the four for n sub k, wouldn't that be three because there's two and one below?
PROFESSOR 3: No because of the way that we're going to be updating the tree.
Next, we'll talk about some [INAUDIBLE]
I like the concept.
But if it's a deterministic game, why couldn't it hold it's [INAUDIBLE] pretty strictly?
PROFESSOR 3: That's a great question.
That's really up to computer memory limits.
As I think that Leah mentioned, the number of stakes in the game of Go-- it's a 19 by 19 board, and you can play something at every state.
It's only like 2 to the-- PROFESSOR 2: [INAUDIBLE] PROFESSOR 3: What?
PROFESSOR 2: 250.
PROFESSOR 3: 250.
You could never explore the entire search tree
[INAUDIBLE] over the first few layers or are we going polite.
We try to do this real time where you could have done something offline.
PROFESSOR 3: It's definitely true.
If you know a state that you're going to arrive at ahead of time, then you can totally do that.
But in a game that's large enough that to do that for all the possible states would take that much more time and take that much more memory.
It doesn't end up making that much sense.
Also, something to point out here, is that for most of the games that we're talking about, simulating a run through of the game is really fast.
So if you think about it-- let's actually get to that in next piece.
But the point is that building up this many levels of a tree for a computer takes probably on the order of less than millisecond.
So doing this for a really, really huge tree, it's peanuts because their such simple operations.
But it won't get expensive when we start building up the tree to serious depths
But a game like Go, how many nodes would you have?
PROFESSOR 3: On each level, in the beginning, we have something on the order of 400 nodes.
And we have a depth of about, I think most games have up to 250 steps, or something like that
So just to build, if you go in there blank, without any nodes built, you have to in the computer, like you said, it hasn't visited a node, it has to go there before it descends further.
Basically, like breadth first.
PROFESSOR 3: It's sort of like breadth first but not quite.
There's an important distinction here, which is that it doesn't have to build up this or this node.
It doesn't have to build up all of the nodes at a certain level.
All it has to do is, if it branches down to a certain sub region, then can't descend in that sub region below one of its siblings without having at least looked once at all its siblings.
After it looks once it can do whatever it wants.
And the point is, that it doesn't mean the tree has to be kept at an even level.
All it means is that the tree, in order to descend on a specific part of the tree, it has to have at least visited direct neighbors once before.
Any more questions on this before-- Yeah
What's the advantage necessarily of having to visit every single?
PROFESSOR 3: The advantage of having to visit every single-- the way that I think of it, is that you don't want to be missing out on potentially being interested in some of the things and not others.
It comes back to the exploration versus expectation distinction.
We do want to descend into the region of the tree that is really valuable to us.
But at least have explored a little bit, at least maintaining some baseline, which really isn't that costly compared to the size of the tree.
400 moves is not that bad compared with 400 and 250
Are these simulations, they're just random simulations?
PROFESSOR 3: We're going to talk about that in a minute.
Any more questions before I move onto that?
Next step is expanding.
And this is very simple.
You just create a node and you set the two initial values.
And the initial values are the number of times it's been visited is zero, and then number of times that someone has won from there is zero
[INAUDIBLE] So the easy part is solving it.
PROFESSOR 3: Now, simulating.
Simulating is really hard.
You can imagine that if you get to a single node and you've never seen that node before, and you don't know what to do from this node onward, that if we knew how the game was going to play out, that is exactly what were searching for, and we would be done.
But we don't.
And in fact, we have no idea how to go about simulating a realistic game, and a game that will tell us something meaningful about the quality of a certain state.
And so, as you correctly guessed, we're going to do it randomly.
We're going to be at a certain state.
And then from that state, we're just going to pick random nodes for each of the players until the game ends.
And if we, as player one, win then we're going to add one.
Then we're going to say delta equals plus one.
And if we don't win, or if we tie or lose, then we're going to call it a zero.
You can in this graph, we're descending randomly and not thinking about it.
And it turns out that this is actually great because it's really, really computationally efficient.
If you have a board, even if it has 400 open squares, populating it by a bunch of random moves doesn't take you very long, on the order of not that many machine can
That's why does you don't score-- if you go down a tree randomly, you already have a simulation.
So the node's going to get to someplace.
But you don't store it because it would lose the randomness?
PROFESSOR 3: You're totally right, actually, in this case.
I've thought through this, and I can't come up with a reason why you wouldn't store it, that's it's temporary values that you find all the way down the tree.
But they don't in most of the literature [INAUDIBLE] But you're totally right about that.
Does everyone understand that distinction?
The fact that we only hold onto the result here and don't theoretically make nodes for every place down in the tree just because we could, just because we've seen them before.
We don't, and it doesn't really matter in this case.
But it's theoretically a slight speed up that you could do
But you reduce that question to generalities?
PROFESSOR 3: Yeah, a little bit.
So we can look at an example of simulating out a running game.
We get some intuition for why a random game would be correlated with how good your board position is.
For example, here we have a Detecto game.
Circle is going to move next.
But as hopefully you can see, because you have played Detecto before, this is not a particularly promising board for x.
Because no matter what circle does, if x is an intelligent player x can win right now.
It has two different options for winning.
And so, if you simulated this forward randomly, what you'll get is that 2/3 of the time, x will in fact win, even if the players aren't really thinking of it ahead of time.
Yeah
Then why not do n simulations at a node instead of just a single simulation?
PROFESSOR 3: You totally can do that.
That's in fact, something that make sense to do and that some people do.
Although what you'll find somewhat soon, is that considering that we're going down the tree, and that sometimes soon we're going to explore all of its children, there's a good question of why you end simulations now when you could just descend through the tree n times and thereby do n simulations by going through the thing and also building out the children?
This case is-- yeah
This gives more importance to why you do randomness.
Because if you're doing random simulations you would ignore the possibility of the best one.
When you first ran a simulation here was that o wins.
If I ignore this node-- PROFESSOR 3: Absolutely.
Which is why it matters that we do this so many times that we drown out all the noise that is associated with playing a game out randomly.
Let's talk about that.
If there's a lot of distance between where we are right now and our end result-- For example, in this game, if I were to tell you how good is this board position, if you are one of those people who played out every game of Detecto, you'll know that this is great if you want it to be [INAUDIBLE] Anyway, the point is, that is not easy to do if you are doing random simulations from where you start.
The correlation between your friend's board state and the quality of that state actually drops precipitously.
And this for me is one of the hardest parts to study about Monte Carlo Tree Search.
Although, as Nick will explain to you, it actually works quite well.
And one of the reasons that it works quite well in practice for more complicated applications is they do away with the assumption of random simulation.
Because even the random simulations does allow you to explore all the states, if you have some idea of where a reasonable quality approach would be, then using that, as long as it's not that much more expensive computationally, can help you with your simulation.
Right now we're still talking about total randomness.
How are people doing with that idea?
Now we're going to update the tree with the results of our simulation.
So given that we had some result lambda, we're going to try to get up the parents.
And for each parent we're going to add that the game has been played there once, and that the result of that simulation gets added if it was a one.
So for example, if there was a win in this game, than this becomes one, one because now it's won once and it's been visited once.
And these two get incremented by one, and these two get incremented by one.
That in itself comprises a complete iteration, the complete single iteration of running Monte Carlo Tree Search, which means that now we can keep doing this over and over again, building up the tree and slowly making it deeper, and making it deeper in selective areas.
And having these numbers increase and increase.
And be more and more proportional to the actual expected value of the quality of the state, until-- does anyone have any questions about this idea?-- until we terminate.
And we have to come up with a way to terminate it.
Now again, we said we're going to pick what the best child is going to be, what the best immediate move from the start state is going to be.
That's the move that were actually going to play.
And so, how do we determine what the best is?
Well, the trivial solution is just the highest expected win given k. What that, in our case, is going to be is the ratio of number of times that I've win from a given early state to the number of times that I visited.
However, this doesn't actually work as well as we might hope.
Let's suppose the following scenario, which is that you have the Detecto game like this.
And you have been exploring the tree for a while.
And you're really mostly looking at these two nodes.
One of these nodes, if you think it through, this node is quite promising and you've been exploring it for a while.
There is a winning strategy from this node.
It's that circle goes here, and then x goes here, and then circle loses because x has two options to win.
However, if you explore this a bunch of times, and for some reason, due to the randomness, this is at 11 out of 20.
Whereas this state, which is inherently inferior, is at three out of five because of a bunch of randomness and because it hasn't been explored as much.
And if we had looked at this one as exhaustively we had at this one, that you probably would actually say that this state is actually better.
And so, you can create an alternative criteria, which is that it's the highest expected win value of one of the children.
But also, that value has to be the node that has been most visited so that they aren't explored by different amounts.
What this sacrifice is however, is that this means that we can't terminate on demand.
This is not always going to be true, and therefore, we're going to have to let the algorithm run until that's true for some start state, which means that maybe is not a criteria that we want to apply even though we know that it would be wise to do so.
Are there any questions about how we pick the terminating guide?
That was the whole thing.
And now we're going to do it lots and lots of times until you guys are sick of Monte Carlo Tree Search.
So this our tree.
It's more or less what we've had before.
The first thing we're going to do is we're going to look at the top.
And then we're going to pick one of these children.
Now let's say that we looked at this, and it turns out that the one on the left is really valuable.
I think it's the one.
Nope, yeah.
Never mind.
It's wrong.
The one on the left has been explored a whole bunch of times.
Remember, this term starts becoming larger than the ones that haven't been visited as much.
And so we're going to descend from this one.
And now we're going to descend, and we have these two options.
Given what you know, would you expect that this is going to pick is going to be the one on the right or the one on the left?
[INAUDIBLE] PROFESSOR 3: On the right because it's never been visited before.
And so, this term is going to explode.
And so, we're going to build a node there.
And then we're going to simulate a game.
And the result is a win, which is bad for this player.
That means that he probably didn't want to make that move.
And so we're going to propagate that value up.
And we're going to start the algorithm again.
And it's going to compare between these three.
And now it's going to pick the one on the left.
Now that it picked the one on the left, it going to compare between these two states.
Which of the two is going to have a higher expansion factor?
The left
Don't you invert it, though, because this is the opponent.
PROFESSOR 3: Exactly.
Because two out of three is actually better.
Because it's one out of three for the opponent that's currently making the move.
So the one on the left is going to have a higher expansion factor, and the one on the right is going to have a higher exploration factor.
Does that make sense for people?
It's OK if it doesn't.
So we're actually going to pick the one on the right because the other one was is doing three and has lots of it's mother's love than that one's.
Anyone else need a drink?
We're going to expand that node.
It doesn't matter.
They are both equally likely to be expanded.
We're going to simulate forward, and it's going to be one.
Which means that that was probably a wise countermove.
Yeah
So when it's the opponent's turn versus your turn, the exploration factor is the same but we complement the expansion factor, right?
PROFESSOR 3: Yes.
So the key here being that this takes in both the state that you're talking about and the player that you're talking about
But regardless of the player, the exploration factor will always be like this is.
PROFESSOR 3: Because it's only the number of visits it's.
It has nothing to do with results of exploration
If you win and you have the plus one, double plus one, and you've propagated out, but I'm wondering-- so if the opponent wins do you also propagate out the win increment itself?
If the opponent's winning, wouldn't you want to [INAUDIBLE] node here?
PROFESSOR 3: If the opponent wins then what you do is you propagate up a zero.
Which means that wk is not incremented, but nk is.
Have we seen a zero yet?
There's one soon.
But the idea is that rather than subtract or anything, all you do is propagate up the result of the game, which in this case is zero.
Which means that all of those states seems to become more valuable to the blue and less valuable to the red.
Because these numbers are lower than the other ones were
OK.
PROFESSOR 3: So we propagate this up and this becomes better.
What we've done here is we've figured out a theoretical countermove to blue moving here.
That's how you should think about this whole tree.
It's really a lot like the way the humans think about these things.
If I do this, then what if they do this?
Well, then I'll do this.
And I see that I'm successful when I do that.
We're going to look again at the top.
And we're going to pick the one on the left because it's really promising.
Five out of six is a good number.
And we're going to look at both sides.
And which one is blue going to pick now?
Well, it's going to pick the one that it's going to be more successful in, which is two out of three.
I realize that this is actually not the kind of thing where I could necessarily ask people because I'm the one who's decided which node to stop.
Then we go down here.
And there's an equal likelihood of picking either of those nodes.
And so we're going to pick one at random.
So that's going to be the left one.
And we're going to create an empty node.
Then we're going to play it out.
And it was a success for blue, which is amazing because what this means now is that suddenly, in this tree of this really good move that red could make the blue wasn't find a response to, suddenly there's hope because we're going to propagate this back.
And that means that blue actually has a response move to that sequence of red's moves.
And so it's going to propagate up.
And this state's going to be more promising to blue and less promising of red.
That region of the tree that we had dug into is a little less promising.
We're going to look back up.
And this time, instead, we're going to evaluate the thing that is both promising from the expansion factor, and also promising because we haven't looked at it very much [INAUDIBLE] exploration factor.
We're going to pick between these two.
Which one is going to be picked here?
[INAUDIBLE] PROFESSOR 3: Because the exploration factor is the same but the expansion factor is higher for the one on the left.
And it's going to show us a node.
And the result is going to be a win for a red, which means that red has found a good countermove to the thing that was previously promising for blue.
And we propagate it back up.
And finally, we're going to pick the one furthest on the right.
Because even though it's terrible for red, and even though it's never won when it's tried it, it has to obey his idea of the exploration mode to find out whether maybe there isn't something possible there.
So it explores, and it goes down, and it has to pick the one on the right.
And so it does.
And it plays this game out.
And it's a loss, again.
Which goes to show you, that blue has found yet another superior move to this really bad move of red, where probably this move of red, if this is a game of chess, is like putting my queen directly in front of the opponent's row of pawns, and I just leave it there.
There's nothing good that's ever going to come of it but we have to explore it just to find out whether there isn't some magical way that I should protect.
And as you can see, we've built up this tree over and over and over again.
And it's starting to look asymmetric.
And we're starting to see that there's really this disparity between exploring the regions that are crossing this tree and exploring the regions that are not and that don't really matter to us very much.
And that this is exactly what we wanted from Monte Carlo trees.
That was why we started the whole endeavor in the first place.
The next thing I'm going to talk about is the pros and cons.
But before I do that, does anyone have any more questions about the algorithm?
Yeah
It's still not clear how we're getting nodes with different denominators-- [INAUDIBLE] PROFESSOR 3: The reason for that is because of the way that we're simulating through.
We're actually not holding onto to the results of the simulation as we're going farther down the tree than the lowest node we expand.
For example, when you simulate from here, you're going to propagate that value here and here, and so on.
But then when we expand below, even if in the course of this guy's simulation it happened to go through one of the states that we expanded below, it will not have incremented the values of that state because we weren't keeping track of it.
Theoretically, if we were to keep track of all of the simulations that we have in fact run, the numbers beneath these things would be higher
If you've already run a simulation from that-- if you've already run a simulation from that red node when you first built it, and then when you created those two ones, each of those have [INAUDIBLE] PROFESSOR 3: OK.
I see
So would the denominator always be one more than the sum of the children?
PROFESSOR 3: Yeah, in [INAUDIBLE] Yeah
I understand how you built that.
Is there a rule of thumb, like it's time to choose a move?
And it seems like you have very low numbers here to make a [INAUDIBLE] Is there a rule of thumb on giving games like it's 2 to the 4 or 2 to the 350, whatever it is.
What kind of numbers do you need for that first row before you [INAUDIBLE]?
PROFESSOR 3: What we'll get to soon is that isn't one.
That's one of the problem with MCTS.
But in terms of which of the moves you will choose, there are actually variants of MCTS that suggest that you more selectively age or insert new children based on something more than just the blind look right now.
In terms of, if I'm here and it's creating my next children as the equivalent, then there are some intelligent guesses that you can make in terms of which one you should score first.
Although it doesn't particularly matter
I'm just saying computational time being what it is, you might say, OK, if this is the timeline of this game I can expect to do a million simulations, which will give me if there's 400 nodes, I'm going to have so much use.
In other words, is that enough time to say that I can play through a game?
I couldn't play through a game with 400 options if I've gotten five out of seven [INAUDIBLE] three out of four [INAUDIBLE] PROFESSOR 3: Absolutely.
And I would say that so far as I know, that's something that's basically very high experimentally.
They don't have good balance on it.
[INAUDIBLE] So let's get on the first comment because that is a computer element.
So why should you use this algorithm?
Even though we've seen tremendous breakthroughs in this algorithm, and you're going to have to ignore everything that I tell you and remember that this does actually work quite well in certain scenarios.
Should we use it or not?
The pros are that it actually does the thing that we want it to do.
It grows the tree asymmetrically.
It means that we do not have to explore.
And it doesn't explode exponentially with the number of moves that we're looking into the future.
And that it selectively grows the tree towards the areas that are most promising.
The other huge benefit, if you'll notice from what we've just talked through, is that it never relies on anything other than the strict rules of the game.
What that means is that the only weight of the game that's factored in is that the game is what tells us what the next moves we can take from a given state are, and whether a given state is a victory or a defeat.
And that's kind of amazing because we had no external heuristic information about this game.
Which means that if I took a completely new game that someone had just invented, and I plugged MCTS into it, MCTS would be a slightly or someone competitive player for this game, which is a powerful idea.
It leads to our next two pros.
The first of which is that it's very easy to adapt to new games that it hasn't seen before, or even that people haven't seen before.
This is clearly valuable.
But the other nice thing about it is that even though heuristics are not required to make MCTS work [INAUDIBLE],, it can work [INAUDIBLE].
There are a number of [?
advanced ?]
places in the algorithm that you can actually incorporate heuristics into.
Nick is going to talk about how AlphaGo uses this very heavily.
AlphaGo is not vanilla Go.
It has a lot of external information that's built into the way that it works.
But MCTS is a framework-- you can imagine your heuristics you can apply in the simulation, there are heuristics you can apply in the UCB in the way that we choose the next node.
There are places that it can fit in.
And this services as a nice infrastructure to do so.
The other benefit is that it's an on demand algorithm, which is particularly valuable when you're under some sort of time pressure, when you're competing against someone that's a mathematician, or when something is about to explode and you have to make a decision on which reactor to shut down.
And lastly-- or not lastly, actually, it's complete, which is really nice because you know that if you run this game for long enough it's going to start looking at a lot like a BFS tree.
No, it's actually going to start looking like an alpha-beta tree, if it is what it is converted to.
It's a nice property to have.
Although, this property does slightly get compromised if you remove the red in this idea, and if only simulate these [INAUDIBLE].. Yeah
You made an interesting comment when you said, oh, it looks like -beta tree.
So it looked like a mini-max tree.
But have they also incorporated notions of pruning in the MCTS, which would make it look like an -beta tree?
PROFESSOR 3: Sorry, you're completely right.
It does look like a mini-max tree.
I think I've seen variants where they do pruning, but I haven't looked into it as much.
But I would imagine that they would converge to whatever you know pruning a certain tree [INAUDIBLE]
But people have explored incorporating pruning into MCTS?
PROFESSOR 3: I think so.
I can't say [INAUDIBLE] And then lastly, it's really parallelizable.
You'll notice, none of the regions of this tree, other than the original choice, ever have to interact with each other.
So if you have 200 processors and you decide, OK, I'm going to break up this tree in the first 200 decisions and then have each one of those flesh out one of those decisions, that actually means that they can all combine information right at the end and make a decision [INAUDIBLE],, which is a really nice, powerful principle as you [INAUDIBLE]..
It does have its fair share of problems.
The first problem being that it does breakdown under extreme tree depth.
The main reason for this being that as you increase more moves between you and the end of the game, you're increasing the probability-- you are decreasing the correlation between your game state and whether a random playoff would suggest that you're in a good position or a bad position.
The same goes for branching factors.
One of the things that people sometimes talk about it as if MCTS AI's cannot play first-person shooters because the distance between the number of things that you can do at every given moment, and what would be a successful approach in the long term after meeting many, many, many moves that each have many branching factors, is that never begins to explore the size of the search tree.
For the most part, it's not really coming up with a long term policy.
It's really thinking about what are the next sequence of moves that I should [INAUDIBLE].
Another problem is that it requires simulation to be very easy and very repeatable.
So for example, if we wanted to tell our AI, how do I take over Ontario?
There's not a particularly good way that you can simulate taking over Ontario?
If you try it once, you're not going to have an opportunity to try it again, at least with the same set of configurations.
And actually, one of the things that we really took advantage of, if that random simulation happens really quickly, on the order of microseconds.
On other hand, the bigger your computational resources that you have access to, the better the algorithm works.
That means that I can't run it off my Mac particularly well.
It would be like large games.
It relies on this tenuous assumption of random play be weakly correlated with the quality of our game state.
And this is one of the first assumptions that is going to be thrown out the window for a lot of the more advanced MCTS approaches, which are going to have more intelligent play outs.
But those are going to lose some of the generality that we had before.
Something that goes off of that is that MCTS is a framework.
But in order to actually make it effective for a lot of games it does require a lot of tuning, in the sense that there are a whole bunch of variants.
And that you need to be able to implement whatever flavor is best suited for you.
Which means that it's not quite as nice and black boxy as we would want it to be as far as give it the rules and have it magically come up with a strategy [INAUDIBLE].. And then lastly, as you mentioned, there is not a great amount of literature right now about the properties of MCTS and its convergence, and what the actual proportion of time to quality of your solution is.
This is true of all modern machine learning things, is that there is certainly a lot more work that could be done.
But right now, that's a gap in terms of using this for a simulation that's supposed to be reliable.
Anyone have any questions on the Pros and Cons?
Before we jump dive into applications, let's talk through a few examples of what games could be solved and could not be solved by MCTS.
Do you guys think that checkers is a game that could be solved by MCTS?
Yes.
PROFESSOR 3: It's completely deterministic.
It's two-player.
It satisfies all of the criteria that we've laid out before.
Checkers is definitely a game that can and has been solved by MCTS, although not solved to the extent that you can defeat the thing that actually has the solution [INAUDIBLE].
How about "Settlers of Catan?"
This one's a little bit trickier.
Do you guys think that MCTS is likely to be able to play "Settlers of Catan?"
If not, let's throw out reason why or why not it would be [INAUDIBLE].
Yeah
No because there's randomness.
PROFESSOR 3: So yes, that is absolutely the criticism.
And that's why we can't apply it vanilla.
I put this on here as a trick question, though, because it turns out that MCTS is robust to randomness.
That you can actually play-- and I realize that's just me and we do.
[LAUGHTER] You can actually play through games.
If you think about the simulation, the simulation is actually applicable even if the game is not deterministic because it does give you a sense of the quality of your position.
And the MCTS-based AI to play "Settlers" is, I think, at least 49% competitive with the best AI to play, at least in the autonomous non-scale space.
So it does work.
Let's talk about the war operations plan response.
Who here has seen the movie "War Games?"
OK. Well, it should be more of you.
The idea of "War Games" is that one of the core characters in this world is this computer that has been put in charge of the national defense strategy with respect to Russia.
And that it needs to think through the possible future scenarios and decide whether it's going to launch the nukes or not.
Do you think that WOPR can be MCTS-based?
No.
PROFESSOR 3: No
It could, it just wouldn't be very good.
PROFESSOR 3: Absolutely.
Once you fire the nukes you're not going to get another chance.
So you can't particularly simulate through what the possible scenarios are going to be like.
Yeah
So what if you had-- I agree you can't simulate it in the real world.
But what if you had a really good model and you just simulated based on that model?
PROFESSOR 3: In that case, it probably depends on the quality of your model.
If you have a good model for how World War III is going to [INAUDIBLE].
[LAUGHTER]
It is the case that the military does have simulators and they do war games in simulation.
PROFESSOR 3: Yes, that's true.
They could certainly try it and run MCTS if they wanted.
And that's what happened in the movie.
[INTERPOSING VOICES]
And there you're putting your money in the simulation not in the--
It's like having an MCTS play SOCOM or something like that.
PROFESSOR 3: Yeah.
It's definitely about putting money into the simulation and you get really good simulation.
If you have a really good simulations then you [INAUDIBLE] to play WOPR.
Yeah
Back to "Settlers" for a second.
I'm curious if there's a way for the whole player training resources thing, or would it have to be only purely like using the ports.
PROFESSOR 3: That's a good question.
I haven't looked closely at whether they do that or not.
If it's playing a two-player game, then I would imagine that they wouldn't because you don't really trade in to play a game.
But if they weren't, I bet that you can incorporate it with
Is it limited to two-player games?
PROFESSOR 3: No, not at all.
In fact, there are lots of purchases that do only one-player games, where you think of what's the best movie that you can make
I know.
But I mean, couldn't MCTS handle three- or four-player games?
PROFESSOR 3: Yeah, it absolutely could.
I'm not sure how they computed their head-to-head.
That might be completely flat cursors.
I'm not even sure how the settlers interact.
Yeah
A quick question.
So at first you know if I reduce the chess board to only 4 by 4 or 5 by 5, and I run MCTS versus the traditional algorithm that AlphaGo offered as a tree.
Do you think MCTS will prefer theory and perform this computational requirement.
PROFESSOR 3: The thing about the way that Deep Blue is, which is the AI that ended the Kasparov thing, a bunch of his chess grand master, is that it has a tremendous amount of heuristic information.
There's a lot of external stuff that's incorporated into the system that makes it able to explore the best paths.
What I would say is that knoledgesless MCTS based on randomness, would take a very long computational time to even become competitive with those kinds of algorithms, and probably feasibly never would.
What if you incorporated heuristic information, I think that there's a bunch of hope in terms of getting MCTS to start performing better.
And you can look at what next I'm going to talk about, AlphaGo.
It takes inspiration for how we go about incorporating these new circuits
So only the circuit you [INAUDIBLE] PROFESSOR 3: It definitely seems like if you have a really good heuristic model for what good states in the game are, that if it's a smaller search space, that some other models could perform better.
Although, I'm probably going to eat my foot here because this is going to be on OCW some massive amount, massive chess playing algorithms.
Eat my shoe not my foot.
[LAUGHTER] One last game.
Does anyone know what this game is?
"Total War?"
PROFESSOR 3: Yes.
Nice.
This is "Rome, Total War II."
It's a simulator for this tremendous real time strategy game, where you play, I think, the Roman Empire.
And you're controlling armies and huge infrastructure systems that move and conquer states and continents, and meet in the field, and manage resources, and do all of these incredible diplomacy feats.
And so do you think that this game can be solved by MCTS?
Yes
Yes.
PROFESSOR 3: Lets say no.
But I guess I put it on here.
So that's good on you.
The way that the AI in "Rome, Total War II" is built is that it's built on an MCTS structure.
And it in fact does do resource allocation and a lot of its political maneuvers based on Monte Carlo Tree Search moves.
There are a bunch of reasons that they explain in the game for why they do this, or in papers released about the game.
But one of the nice ones is that it's random, which means that you're never going to play against the same kind of AI twice because every time the set of decisions that it's going to think about is completely different
I have a quick question.
PROFESSOR 3: Yeah
So if I want to model any game with MCTS, does it have to be that the actions in playing a game has to be able to discretize.
PROFESSOR 3: Yes.
So far as I know, I haven't seen many continuous variants in MCTS.
And so, I think that it is about choosing these reactions, which on it's most narrow level does actually bring it down to here.
I think one of the reasons that this is nice is that there are so many different decisions that could be made that MCTS is really the only approach that could even begin to handle the massive branching factor that's associated with the game Rome, Total War.
Yeah
This is also the consequence of this year you get the play off when this game comes.
PROFESSOR 3: That's interesting.
That's probably totally it.
That's cool.
That's everything about how the algorithm actually works.
I'm going to pass it off to Nick, and he's going to talk to us about some actual limitations for this game [INAUDIBLE].
PROFESSOR 3: So as you have said, I'm going to start diving into some applications here.
And not only applications but also some modifications or augmentations of MCTS.
It should hopefully clarify some of the side questions you all have been having on slight tweaks to MCTS.
Now let's get started.
Wait for it.
Now let's get started.
[LAUGHTER] Part III, applications.
First thing we're going to look at is an MCTS-based "Mario" controller.
And "Mario" might seem like some weird thing to test AI on, but there actually is a "Super Mario Bros" AI benchmark, which it used to test a lot of AI on how well they could play this platform.
In case any of you don't know what "Super Mario Bros" is, this is a screenshot.
Basically, you control this one character.
It's a single-player game.
The ultimate goal is to reach this flag at the end.
But along the way there's enemies, there's some bonus shrooms you can get.
If you break open some boxes you might get coins, things like that.
But first, let's just highlight some of the modifications that need to be made, or some of the differences between vanilla MCTS and an MCTS that's going to be able to work for "Mario."
First thing is that it's single-player.
The second is, we use a slightly different simulation strategy than the initial just vanilla simulation.
And someone actually hinted at doing more than one simulation because you, you're watching us to n simulations, I think.
We'll touch on that.
Then this also introduces what I would consider to be domain knowledge.
Then finally, there's a 50 to 40 millisecond computation time.
And that has to do with the frames per second of the game.
So you would think that "Mario" is a continuous game, but if we discretize time into these chunks, then we can use MTTS.
Now let's just think about how we could possibly formulate this problem.
Can anyone think of what each of these nodes would be if we're playing "Super Mario?"
Jump.
PROFESSOR 3: Sorry?
Jump.
It would be like, first node you're going to jump.
PROFESSOR 3: That might be a way to formulate it.
But I think that could get--
Oh, it's not your control at inputs [INAUDIBLE]..
PROFESSOR 3: Right.
So the node itself isn't going to be an action
Equal frames.
PROFESSOR 3: Yeah, basically.
So it's going to be the state of a game, what we'll call a state.
So it's basically just a screen grab.
And it take it, in this case, it's a 15 by 19 grid screen grab of the game.
And it will have information about-- it knows Mario's position, it knows the enemy's position, position of the blocks, et cetera.
And then, as Yo was saying, in MCTS we have values associated with our nodes.
And so it will also have a value.
But we'll get into the value in the next slide because I can't really fit it all in here.
With that being said for our node, that being the state of the game, what makes sense for the edge?
Does anyone know?
How do we transition from one state to another state?
Jump.
PROFESSOR 3: Yeah, exactly.
So this is where the jump and all the action have been played.
So the actions that you take-- I didn't list all the actions.
You can also have a jump left, jump right, all those things.
But basically, the actions are what takes you from state to state.
So I just drew out what a node might look like if you used the jump action.
You might have Mario go up in the sky.
Are there questions?
Does it just run the rest of it?
Because that little thing's moving as they move on?
PROFESSOR 3: Well, it's not moving in this moment in time.
We're discretizing time right now
But I'm saying, if your action is jump, just you would have 1,000 nodes because if you did plan out where that thing's moving, left or right, then it could be-- PROFESSOR 3: Yeah, right.
So in each state we have the enemy position.
And we know the speed and direction.
And so we know when we go from this node to one time step later, we'll know where the enemy's moving.
Any other questions?
Moving on.
Sorry.
Let me just preface this part real quick.
So in our other simulations, at the end of the simulation we would get either a one or a zero, if we'd won tic-tac-toe or we lost tic-tac-toe.
But that won't really work too well here because there's a lot of other factors that go into play when you're playing "Mario."
Also, if you're doing a simulation, more than likely, you're going to end up hitting an enemy and dying or falling into a gap and dying.
So a lot of these simulations might all return zero.
And that is, you can't really distinguish between them.
So this is why I say, this version of MCTS introduces what I would consider to be domain knowledge.
Basically, they're assigning scores to potential things that could happened along the way.
And this is basically telling the AI that collecting a flower is a little bit better than collecting a mushroom.
It's telling it that getting hurt is bad.
Right off the bat, all these things in the score are giving the AI some domain knowledge about "Super Mario Bros," that it's helping it calculate the simulation results.
As it says here, it's just doing a multi-objective weighted sum of all these things.
Throughout the simulation it's just adding up your score.
And then that's the score that is going to be propagated.
Are there questions about the score?
You said that it adds up all these guys and it propagates it over.
Is it possible to just propagate the multi-part sum [INAUDIBLE] as opposed to propagating one value that you create?
Are you essentially propagating all-- what's this?-- 15 values upwards at every node, or are you propagating one value-- PROFESSOR 3: Well, it's one value.
It's the collective--
Then you make them add it together and you got each one of them a sub factor.
PROFESSOR 3: Then also, just one thing to note here, is distance, you get 0.1.
And these are all parameters that have been tuned.
In the initial version, distance was, I think, a reward of five, but probably realized that that made Mario skip past a lot of coins and things.
And so he tweaked the score for that.
And also, time left is two.
So there's some weight there.
You want to get to the very end of the game
If you're pushing up this score, it's no longer a win over losses.
So it's not w over n. What is it affecting?
PROFESSOR 3: You can just use the score
The score is the-- PROFESSOR 3: Yeah.
In MCTS you have this idea of when you're propagating your q value, you could have that to be zero, one
It's like the sum of all the scores and the nodes below over the number of games you win.
PROFESSOR 3: So basically, what you would be getting when you divide by the number of simulations is your average score at that node

When you have killsByFire and [INAUDIBLE] like that, if you have a positive value, then isn't it good to be killed by fire, or something like that?
PROFESSOR 3: This is killing an enemy by fire.
Like Mario could collect a certain flower or mushroom?
I think flower, then you have a fire breath and you [INAUDIBLE]
So that's Mario's status if Mario never dies?
PROFESSOR 3: No.
Mario's status is-- I believe, Mario's status is the fact that you could upgrade Mario by collecting [INAUDIBLE] mushroom from a fire Mario.
So that gives you a lot of points.
Because if you become fire Mario, then you're more likely to not die by running into enemies because you have fire-spewing--
You said they spent a lot of time tuning these parameters.
Isn't it generally, though, just an optimization framework if that's some formula?
So they tuned the parameters just to make behave the way that we think is nice.
But if you change the values, they'll do the right thing for that equation.
PROFESSOR 3: Yeah
OK.
PROFESSOR 3: Yes.
But they were tuning this to make it play how they wanted
[INAUDIBLE] can't just be a reflection of [INAUDIBLE] PROFESSOR 3: That's a strategy.
If you choose that, I don't see why not.
That might affect certain things.
Obviously, you can change these to whatever you want.
It'll slightly tweak which simulations as to working better, in terms of changing which nodes you end up choosing [INAUDIBLE].
So we move on.
So we know about scoring simulations.
Now we're going to look at exactly the simulation type that's used to play this MCTS controller.
So the regular version that Yo talked about is just choosing a random node at each level in your simulation.
But there are some other strategies.
And someone brought one up.
The first is, look at best of n. So in this one, you choose three random nodes at each level, except that you stick with the best of those three.
Choose three random nodes, stick with this one.
Go to the next one.
You would choose n random three, take the best one, and then go to the next level.
You are able to do that in this game because of the way the scoring works, you don't have to get to the end of the game for your score.
You actually could collect a coin along the way.
If this is jump, and then it gets to be a coin versus moving left and right.
That doesn't give you any points.
Then this is the node that would give you the highest scores, so I would choose that one, et cetera.
And then the final one, which is the one that is actually used for this controller, is multi-simulation.
This was brought up by him.
I don't know your name.
Sorry.
But basically, you run multiple random simulations from your node.
And then you propagate up whichever of those simulations give you the highest value.
And the reason to do multiple simulations is to attempt to increase the accuracy of your simulations.
If you just do one simulation you might just get really lucky.
But if you do three then you can take the highest value use that as your value.
Since the whole point of this is to try make moves that get you the highest values, then that will make your random simulation value more accurate.
Are there questions about multi-simulation?
So what do you think about the simulation [INAUDIBLE] how many [INAUDIBLE] PROFESSOR 3: So there's a trade off here.
The more simulations you do the more accurate-- the more representative your simulation will be at the end of the game.
You could run two to the whatever simulations to try to get every single possible action and then take the max of that.
And that would give you the maximum value.
That would be ideal.
But obviously, that takes more time.
So there's a trade off between computation time and the number of simulations you run.
And that's just something that they probably just played around with
Do you use [INAUDIBLE] have to finish the decision losing a couple of minutes or 10 minutes or they're going to take your [INAUDIBLE] away.
PROFESSOR 3: In this competition there is different computation time budgets that you get.
And I believe the reason for the different computation time budgets is the frame per second of the game.
I told you all about the setup, we went over, the scoring, the nodes, what the advantages are, what the simulation strategy is used.
So you probably want to see it in action.
So this is always a risky move trying to get video to play
It's actually in the back up.
Hit Escape.
PROFESSOR 3: OK. Got it
And now, I guess, we-- PROFESSOR 3: And drag it over again?
Yeah.
PROFESSOR 3: Running this full screen
Hit the [INAUDIBLE] PROFESSOR 3: [INAUDIBLE] All right.
Here's this MCTS-based "Mario" playing controller.
You can see he's actually wrecking, so doing some serious damage here.
But those lines that you see, the reason they're different colors it's not showing different players, or anything like that.
It's just using different colors so you can see the different layers of this tree search.
You can see he actually went backwards there.
And that's because in a simulation, when one of the backward ones landed on an enemy-- and in fact gets you points from our scoring system versus if you had just gone forward you would have gotten some distance points but not-- also, he is just [INAUDIBLE] The simulation is quickly being able to figure out that he can jump on all his enemies.
So he's just wrecking all these guys.
Getting lots of points here, collecting the coin, et cetera.
You get the idea.
It's pretty awesome to watch.
There's that flower we were talking about.
So now he's actually a fire-spewing Mario demon.
He's doing some serious damage with that.
Stepping on missiles.
I didn't even know you could step on the missiles.
All right.
You could watch this for a while.
But we'll exit now.
It looks super promising in this video.
I don't know how close max stuff
Just click on back [INAUDIBLE] PROFESSOR 3: There it is.
OK.
The demo looks really cool, looks really promising.
Let's take a look at the charts here because we all want some quantitative stuff.
This is the chart.
The score is on the y-axis.
The bottom is computation budget, which is something that you were talking about.
I just want to highlight to make this a little more visually appealing here.
All of these things that I highlighted, it's labelled as UCT.
That's Upper Confidence Bound Tree.
Remember, Yo talked about upper confidence bounds.
That's essentially what's used in that TTS for guiding your tree search.
So these are all the methods.
But then UCT multi, which is this purple square, that's saying it's using MCTS but it's doing the multiple simulations.
And you can see this multi plus care is also in the top.
Both these use the multi-simulation technique.
And then the plus car is they added an extra scoring mechanism for carries.
I believe that's probably like carrying a shell.
That made it do better.
Then these ones that aren't highlighted are using plain Astar, and then a refined version of Astar.
With increasing time, the do increase scores, but they're even worse than just your UCT with just random simulation, no multi-simulations.
We're running low on time, which is not ideal.
But another thing that I want to point out is down at the bottom here, these are the multi-simulations.
They have the lowest maximal search depth, which at first would seem like, what?
I have the lowest search depth but my score is the most?
But that comes into play when you were saying about the trade off between the simulations and the amount time it takes.
So because I'm doing multiple simulations, I'm taking more time at each node.
But that's giving me a more accurate value assessment.
So that let's me choose my actions more carefully, or with more information.
And so that's what's able to give me this better scores.
That's all "Mario."
So we're going to moving onto AlphaGo.
Are there any questions about "Mario" before I go to AlphaGo?
Yeah
What's the table [INAUDIBLE] inference?
PROFESSOR 3: That's a good question.
I have a feeling it's because if you're doing best of n, that's really heavily relying on your scoring metrics.
Let's say at one step if I jump and collect a coin versus if I go left or right and play, I'll get more points if I get that coin.
But maybe, a missile is going to hit me in the face if I do that.
It gets rid of some of the-- it's forcing you to do certain moves
Is the A* heuristically using the same value, the same value that you're getting by your simulation?
PROFESSOR 3: Yeah.
I'm not exactly sure what the Astar heuristic is.
The whole reason that A* is difficult is because coming up with heuristics for these types of games are.
But this is not his version of Astar.
I believe this is the Astar that was used by-- I forget the name of the guy-- but he won the AI competition a couple of years ago.
I'm going to try to move onto AlphaGo.
Does someone have how many minutes I have left?
Four.
PROFESSOR 3: OK. We're going to power through.
Here's AlphaGo.
Hopefully, you all know the rules.
Just in case, I'll just go through a quick-- 19 by 19.
You alternate black stones and white stones.
You collect enemy stones by completely surrounding them.
You can surround a single stone.
groups of stones.
And your score is your territory plus the number of captive pieces.
So your territory is just the area that you're surrounding, and then you just add the stones you've collected.
The rules aren't super important.
The main emphasis is there's very few rules so you would think it's really simple.
But the complexity of the game is quite extreme.
At each turn you have about 250 options that you can play.
Each Go game lasts about 150 turns.
So that gives you a total of 10 to the 761 games, approximately.
And to put that in comparison, here's chess.
You can read those numbers.
Chess is also pretty complex.
But there's 35 options for turns.
Deep Blue.
I think you were talking about building out the whole tree.
So Deep Blue would build out the tree for six levels.
And then use this hard core chess master inputted heuristic evaluation that it used to find the best move.
Except with Go, you have 250 options, which already is adding a lot more complexity.
So that strategy won't work quite as nicely.
What do we do?
We use a modified version of MCTS.
Well, it's not what we do.
That's what Google's DeepMind team did with Go.
They combined neural networks with MCTS.
Coincidentally, we learned about neural networks last class.
Probably not a coincidence.
PROFESSOR 3: It's not a coincidence.
PROFESSOR 3: The we ordered two policy networks in the AlphaGo, and one value network.
And another big coincidence here, the two policy networks are actually CNN's, which we learned specifically about last class, convolutional neural nets.
And the reason for that is the input to the policy neural networks is an image of the game.
And remember, convolutional neural nets work really well with images.
What it outputs, though, is a probability distribution over the legal moves.
And the idea is, that if a move has a higher probability it will be a more promising move for you to take.
But another key point is that it's not deterministic.
It's not telling you to take this move.
It's just assigning a higher probability to this move.
And this network was generated by doing supervised learning on 30 million positions from human expert games.
Apparently, there's a giant database of Go expert games.
So that came in handy.
And there were two different networks trained.
One of them was a slow policy, the other was a fast policy.
The slow was able to predict an expert move with 57% accuracy, which to me was mind blowing.
Using this neural network, 57% of the time it could pin where the expert would place his move.
That took 3,000 microseconds.
Versus the fast policy, which suffered a bit in the accuracy, but it's 1,500 times faster.
And we'll see where they used each of these different policies later on.
But it could predict the expert move with 57% accuracy.
The other Go team was, that's not our goal.
We don't want to predict an expert move.
We want to predict a winning move.
And so to do that, they took their policy network, and then they would use reinforcement learning.
That's where you play the network against iterations of itself in order to hone in a better policy that's geared towards winning moves.
Then they tested this against Pachi, which uses-- for the camera, I have no idea if that's how you pronounce Pachi.
It might be Patchey.
I'm not sure.
But there's 100,000 MCTS simulations at each turn.
So this is purely MCTS.
If it were playing just the AlphaGo policy network, the policy network won 85% of the game.
So without any sort of trained search or anything involved, it won 85%, which is pretty great.
And that suggests that maybe intuition wins over long reflections in Go.
And interestingly, if you talk to expert Go players and you ask them why they did a certain move, they'll just say, It felt good, or I had a hunch in this.
That's indicative there.
Hopefully, I'm not going overtime.
Sorry.
Those are the two policy networks.
There's also a value network.
What the value network does is it takes in a board, and they'll give you a value, like how good is this board?
They'll give you a win probability number.
So 77%, it would say, 77% of the time you should win from the board.
That's similar to the evaluation that comes from Deep Blue.
But rather than a Go master coming in and telling you, well, if these are connected in this way, and down here we have this certain thing then here's the score we should expect, in chess, they had chess masters, like if the knight is here and the queen is here, all these specific things.
This was actually learned from the reinforcement learning that was happening when the policy networks were playing each other.
The value network was learning about those positions during that time.
And the predictions get better towards the end of the game, which I think Yo mentioned in his talk.
So how do you combine all these into MCTS?
The slow policy network, if you remember, is slower but should give us stronger moves.
It is used to guide our tree search in order to help us decide which nodes to expand next.
When we expand that node to get the value, the value of the state is the simulation, like before, like normal MCTS, except it's not a completely random simulation.
We use our fast policy network to give us a more educated simulation here.
But we're using a fast one, obviously, to save some computation time.
It's giving us probably a more indicative random simulation of what's going to actually happen.
And then we also combine that with our value network output.
So we run our value network on this node, as well.
And we add that to our simulation value and we propagate it.
Interestingly, the AlphaGo team tested out just using the fast policy simulation value and scrapping the value network.
And they also just used the value network and scrapped the simulation value.
And those both performed worse than if it had these.
And another added interesting point here, is that these two factors in our value have about the same weight.
They were both about equally important.
I think I'll get into that later.
But first--
Can I just ask a quick question?
PROFESSOR 3: Yeah
So when you said the policy network is used, is that used when you're navigating to the tree to get to a leaf, or is policy network being used to do the simulation once you're at the leaf, or both?
PROFESSOR 3: The slow policy is done for this part.
Then the fast policy is used for the simulation.
Because the slow policy does take 1,500 faster than-- or the slow takes 1,500 times longer than the fast policy.
You don't want to use that in your simulations.
That would just take way too long.
It's basically just a way of making it so our simulation isn't completely random.
It has some educated moves.
Why use policy and value network synergy?
Why can't we just use the policy network?
Why can't we just use the value network?
If we have the value network alone, we'll actually-- here's a side point.
Remember, the value network learned from the policy network.
And then also, later on, the policy network is improved by our values.
They work hand-in-hand.
But if we had the value network alone, when we're deciding on it the next move, we're going to have to evaluate every single move, which would take forever.
And so, what the policy network does is project the best move with a probably distribution.
And it narrows our search space.
And then, if we had the policy network alone, we'd be unable to compare nodes in different parts of our tree.
The policy network is able to tell us a distribution over which move we should take from a certain node.
But then, if I ask it if I'm in a better position here than in some other place, it won't know.
That's where the value network comes in.
It will give us an estimated number of the value assigned and open an evaluation of that node.
And then these values are later used to direct our tree searches based on updating the policy once it realizes, oh, I thought this would be a good path but the value is this, so update all that.
Then why do we combine neural networks with MCTS?
Remember, the policy network alone played against Pachi, which was purely MCTS, and it did pretty well.
So how does MCTS improve our policy network?
Remember, MCTS did win 15% of those games.
So already, that makes you think there's something there that maybe the policy network is missing.
Also, the policy network is just a prediction.
So by using this tree structure, we're able to use these Monte Carlo rollouts to adjust our policy to move towards nodes that are actually evaluated to be good.
And then, how do neural networks improve MCTS?
The point should probably be clear by now.
We're able to more intelligently lead our tree exploration.
Our simulations are more reflective of actual games.
And the value network and our simulation value are complementary, which I've mentioned before.
And just to highlight that, basically, the value network is going to give us a value that is reflective as if we've played the slow policy the whole time.
And the simulation is if we used a faster policy.
So they are complementary.
And I know I'm over time.
So I just wanted to skim through the stats real quick.
Distributed AlphaGo won 77% of the games against regular AlphaGo.
So it's the only thing that beat regular AlphaGo.
And then distributed AlphaGo won 100% of the games against all these.
In a rematch against Pachi, now that we've added MCTS to our policy network and we have our value network, we slaughtered Pachi 100%.
Then we decided to see how we fare against humans.
And by we, I mean not me, I mean Google.
And they won 4 to 1.
And Lee Sedol rating was 3,520.
Now AlphaGo's rating is estimated to be about 3,586.
So you're like, whoo, we beat the best dude.
Except we didn't because there's another dude who has an even higher score, apparently, 3,621.
This should be the last part.
Here's this timeline.
Basically, tic-tac-toe, checkers were conquered in '50.
About 40 years later, we conquered checkers, chess.
Then we scroll down to 2015, is when AlphaGo was able to beat Fan Hui, who was a two-dan player, which is considered lower down in the tier of professional Go.
But then, Lee Sedol was a nine-dan player.
And he was able to beat him literally last month
So good job.
PROFESSOR 3: We're done.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PRESENTER 1: The basic concepts of reachability-- I'm talking about how we can represent reach sets in continuous spaces.
[INAUDIBLE] is going to take it over and talk about some applications in robust motion planning.
And then Gerard is going to talk about how we compute reach sets and focus on these two, on flow tubes and funnels.
So reachability is a task of figuring out what stage a dynamical system can possibly reach or be at at a certain time.
So for example, if we have these two systems, we have a finite state machine on this reach system and a continuous system.
And these are the starting points.
And these are our axes.
We can move east or north here.
And we can move east or north with these velocities.
[INAUDIBLE] So after one second or one time step, we can be in either of these two states.
Or we can be anywhere in this area-- and after two seconds, after three seconds, and so on.
OK?
So this is what reachability does.
And the motivation for this-- one of the applications for reachability is for verification.
Verification is just the task of making sure that we are never going to reach any bad states.
So for instance, let's say we have a microwave.
A bad state is the microwave being open and on.
We want to make sure that these bad states are never reachable.
So what we do is we compute the reachable sets, and we check for intersection with the bad states to see if the microwave is on and open.
Or like over here, let's say we have these obstacles.
Let's say we have a wall or somewhere else over here.
We want to-- this is bad, right?
Because we can reach this state, and it's the same as a wall.
Any questions so far?
Feel free to interrupt me any time.
Another motivation for reachability is for robust motion planning, which [INAUDIBLE] talk more about.
And I have a video of this.
So what we can do with reachability is stuff like this.
So this is done using flow tubes, which is reach sets, reachability.
Gerard is going to talk more this example.
So now before we talk more about these cool examples, let's just formalize concepts.
Let's define some things.
So let's go back to the same system that we had earlier, this state machine, with a finite set of states, X; a finite set of inputs, U, which in this case are east and north; and a transition function.
And let's say this is just [INAUDIBLE] to keep things simple.
And we have a set of initial states, X0, which I didn't label as red.
So the reach set is defined as the set of states that we can possibly be in at time t. So more formally, there exists a sequence of t control inputs that will take us from this initial state to the corresponding reach set.
So let's say, for example, you have bad state, it's in the reach set at time 3 because there exists a sequence of three inputs.
In this case this is one such sequence.
There is two more that will take us from the initial state to that.
OK?
And then the reachable set, the reach set reachable set is defined as the union of all reach sets for time less than or equal to t. So the reachable set of time 0 is just the initial set of states.
And time 1, we have the unit and so on.
So the reachable set is what we use to test for intersection with bad states because that's all the states that we can possibly reach from time 0 to time something.
OK?
Now as some of you may have figured out by now, there's a simple way for finding segmenting so we can compute reach sets.
So for example, let's say we wanted the reach set of time 2.
One way-- the easiest way we can do this is to break this up into time steps, into single time steps.
So instead of computing it straight for time 2, let's first compute it for time 1.
And now we can use these two states as the set of initial states and compute the reach set for one more time step.
And that gave us this reachable-- the reach set.
OK?
Any questions?
OK, and the reason why these works is because reach sets are semi-groups.
And semi-groups satisfy this equality.
So we can not just break it up into single time sets.
We can break into any two times, s and t. OK, so all of this was for finite state machines.
But for continuous systems, things get a little more complicated.
It's a lot of the same concepts, but we need to handle a very important issue, and that is how to represent reach sets, or just set, regions in space for continuous systems.
Because in a finite state machine, we can just enumerate the list of states in the reach set or reachable set.
In a continuous system, we need some other way.
We can't just say it's every point for which there exists a sequence of control inputs, because there can be infinitely many points.
So what we do, we have two types of representations that we like.
One is convex polytopes.
And that's just a generalization of a convex polygon into n space, into n dimensions.
And we have two basic ways that we can represent these complex polygons.
One is with these hull vertices, like this, AB to F. [INAUDIBLE] And then the convex polytope is defined as the convex hull.
And in case someone doesn't remember what a convex hull is, let's say we have just a list of points.
The convex hull, imagine have a rubber band and you are going to stretch it out, and you let it go.
And what that's going to do is this.
And the convex hull is just the whole area inside this-- the rubber band.
OK?
And then another way we can represent convex polytope is with a set of inequalities, so for example, here we have four lines.
Here's the equation for each line.
So let's say this red line, we want anything below the red line.
This blue line, we want anything to the right, so anything greater than, and so on.
So this area is the intersection of all these inequalities.
OK?
And each of these representations has an advantage over the other.
Vertices are good to test for emptiness because when our set of vertices is empty, then our convex polytope is going to be empty.
And this inequalities is very useful for testing membership, because for any points, we can just plug it in to every inequality.
And if it's true for every inequality, then it is in the whole group.
And convexity, I'll just explain it briefly.
Convexity means that any two points inside our region-- our region is not non-convex is for the line connecting these two points.
All the points in that line are going to be in the region.
So here you can see that this is convex.
And here, this red area is outside the region, so this other region is non-convex.
And convexity is going to be important for the algorithms later on.
Gerard will talk more about them.
OK?
Other than convex polytopes, we can also use ellipsoids to represent reach sets.
So ellipsoids are just the extension of ellipses to n spaces, n dimensions.
And a convex polytope is defined by this.
x is just all the points inside the polytopes, inside the ellipse.
v is the center.
And A just tells us about this, how deform the ellipsis.
Any questions?
OK. And one of the reasons why we like convex polytopes, why we like ellipsis, ellipsoids, is because of their enclosure under linear operators.
So let's say we have P that is convex polytope or an ellipsoid.
Then, if we have a matrix A, then A times P will be defined that way.
It is still going to be a convex polytope or an ellipsoid.
So if we have something like this, then this can maybe getting formed, but it's still going to be convex, a convex polytope.
And the reason why this is nice is that, if we have a linear system defined this way or this way, and then if we start with a-- basically, if we started with a convex set of states, then our reach sets are always going to be convex because we're just multiplying by A.
And this is very nice to represent-- this tells us that our representation is going to be good because we can just use convex polytopes, ellipsoids, and stick with those.
And then just a technical-- OK, so even though the reach sets are going to be convex, the reachable set might not be convex.
But it will always be a union of convex polytopes or ellipsoids.
For example, let's say, in the previous example, we started here.
Then we need, after one time set, we were here.
So these are the two things.
And then our reachable set is the union of both of them.
This is clearly not convex.
But it is a union of convex polytopes.
So we can represent everything within it.
OK?
Any questions before I move on to [INAUDIBLE]??
OK, thank you.
PRESENTER 2: OK, so now that have understood what reachability is and how [INAUDIBLE] represents, and before we dwell into more theoretical aspects of [INAUDIBLE].
Here are some cool applications of reachability.
So there are many applications of reachability to robots are like complex systems [INAUDIBLE] systems.
One of the applications that has been talked about before is the robust motion planning.
The early idea is to more of a glider or an airplane through lots of obstacles, even in the presence of [INAUDIBLE] these and [INAUDIBLE].
They're already crashing into obstacles, even when they are [INAUDIBLE].
[INAUDIBLE] applications of reachability to control complex systems like by parallel working robots.
Here you can see that we are controlling [INAUDIBLE] simple soccer ball with something.
And more importantly, reachability is used for very fine safety properties, for safety reasons, like aircraft collision avoidance.
So in those [INAUDIBLE] we're going [INAUDIBLE],, robust motion planning.
If you're interested in other things, there are papers and there are [INAUDIBLE].
So the goal of motion planning is to find a set of control inputs that take you from the start state to the goal state.
So in this example-- without colliding with any of the obstacles.
So here, this is one possible path from the start state to the goal state.
So it just looks nice.
[INAUDIBLE] collective things of the obstacles.
But if you get the controls to this cart and then give it a real robot, you might find it going-- doing something like this.
So this is because of the path that you-- algorithm that you use for planning this path is-- that's not taking into account any of the uncertain areas in the environment or [INAUDIBLE]..
So there are different kinds of uncertainties.
One of them is environmental disturbances, like there is wind here.
So that can move you over off your planned path.
And there can be modeling errors.
The model that you use to represent your complex system might just be an approximate.
So the real world is different.
And there are state uncertainty.
So the things that you used for measuring your position or velocity are also approximate, so there are uncertainties.
[INAUDIBLE] there can be some randomness in the initial conditions.
You are thinking of starting the robot at 0,0, but in practice, you might actually be starting at [INAUDIBLE] 0, [INAUDIBLE]..
So there is randomness in initial conditions.
The goal of robust motion planning is to have some guarantees with some confidence that the system will reach the goal set without getting on with the obstacles even in presence of these uncertainties.
So this is the problem.
So let's look at some of the solutions that come up for solving robust motion [INAUDIBLE]..
So this is a timeline.
So there are three different systems.
And all of them have a goal concept where they use some representation to represent the reach sets, the set of possible state that the system be.
And then instead of searching for a single trajectory to a state, they search for these representations so that each final self-serviced representations that can go from the start state to the goal state, you won't hit any of the obstacles.
So earlier Bradley and Zhao and Frazzoli came up with this concept called the flow tubes.
It's [INAUDIBLE] presenting makes sense.
And we will know all about the [INAUDIBLE] of the [INAUDIBLE]..
So one approach that we use is that we have a set of initial states and we have a set of goal states.
So flow tube is a combination of all the possible paths that will get you from any point in the initial state and point in goal state.
So this is kind of representing all possible [INAUDIBLE]..
So and then you search for these flow tubes to be a path to get your [INAUDIBLE].. And [INAUDIBLE] constraints so that you can also reason about complex systems and revise spatial and temporal coordinations like the walking robot that you have seen before.
An [INAUDIBLE] about for using funnels for motion planning.
So funnels is also a similar concept to produce.
It's kind of like a [INAUDIBLE] region around your path.
So that if you are inside a funnel, you're guaranteed to be inside the funnel.
So in this picture, we're going to [INAUDIBLE] motion planning using funnels.
Again, there are references for the other people's [INAUDIBLE]..
So instead, you'll be using funnels [INAUDIBLE] for every path, you compute some region around the path that is a stable region.
And that's called a funnel.
[INAUDIBLE] your funnels [INAUDIBLE],, but for this part of the lecture, we are assuming we don't have a hundred of funnels, and that you'll see how to use these funnels to do motion planning.
So the previous example is here.
The [INAUDIBLE] funnel [INAUDIBLE] flied by.
You see that it collides with the trees and [INAUDIBLE].. Then [INAUDIBLE] like these funnels are [INAUDIBLE] uncertainty in the world that you have.
If [INAUDIBLE] everything with uncertainties, [INAUDIBLE] and then you compare funnels for that.
Any questions so far?
Can you please say again definition of funnel?
PRESENTER 2: So it's kind of something that [INAUDIBLE] around [INAUDIBLE] around-- like you have a [INAUDIBLE] around there.
So I think Gerard with explain more about it.
So the [INAUDIBLE] with funnels is that you are inside the funnel at the point.
When you [INAUDIBLE].
It's like [INAUDIBLE]
Yeah, you have an [INAUDIBLE]..
So a funnel is basically-- it's kind of like a reach set around a certain point too.
So the way that you do it is you have some [INAUDIBLE] trajectory.
And then at each node point in that trajectory, you compute some kind of reachable set at that node point.
And so you blow it up, and it becomes some kind of ellipsoid.
And as long as you're within that ellipsoid, no matter what disturbance you get, you're going to go back towards the nominal path.
And then so, when you join all of those ellipsoids over the trajectory, then you get a certain kind of funnel.
And as long as you're inside that funnel, you're going to stay on that funnel.
And you're going to be on the path you want to be at with some disturbance in this [INAUDIBLE].
So that the great thing that there would be an example of a funnel.
As long as you're in the funnel, you're going to stay in the funnel, basically
So is this considered the control, or is it only the dynamics?
PRESENTER 1: It's the dynamics with the control.
So it's the flow tube system.
PRESENTER 2: Yeah, so, you'll learn more of those [INAUDIBLE]
Let me just quickly add.
So a funnel is type of flow tube, right?
I mean, just different people either use the same terminology, use different terminology.
Each of the applications of flow tubes has some invariant, which is different.
And the particular invariant on this one is that if you are you in the funnel and you apply the LQR component, [INAUDIBLE] then you're guaranteed to stay within that tube
But I thought the flow tube is actually like getting smaller and smaller in the funnel [INAUDIBLE]
Not necessarily.
In general, a flow tube is simply a bundle of trajectories, which capture some common features.
And then maybe that they move to a limit of cycle, and maybe they stabilize to a point, and maybe that the move to a goal, maybe that they always maintain.
They're stable within a tube.
In a funnel they focus particularly on stability to disturbance and then staying within the tube.
PRESENTER 2: OK, so the [INAUDIBLE] example is kind of useful to combine the reachability motion planning.
So here is another situation where getting to the goal from the start [INAUDIBLE] and there are a couple of sort of trees lined up.
But there is a path that goes in between the [INAUDIBLE]..
This path looks kind of risky because it's pretty close to the two trees.
But there is another path we can take which travel around the trees.
And so if you don't need the [INAUDIBLE] safe.
But maybe that's [INAUDIBLE] too.
So that's why we do the combined reachability with motion planning.
So if you combine that, you might find that the first path, if you compute a flow tube for the first path, [INAUDIBLE] it does not hit any of the trees.
And it's indeed safe.
But it might happen that the second path is more susceptible to the disturbances in [INAUDIBLE]..
So that's why [INAUDIBLE] and also [INAUDIBLE] combined motion planning.
Because what's intuitive is not actual.
So far it's all about [INAUDIBLE] planning.
So the [INAUDIBLE] everything about the environment, like where the obstacles are.
So that's not always true.
[INAUDIBLE] So you don't always know all the obstacles beforehand.
You can't [INAUDIBLE] as you [INAUDIBLE]..
So one problem with doing online planning, there is that you can not do any expensive computations during runtime.
And all of these funnels can be places that are very expensive.
[INAUDIBLE] solving an optimization problem.
It takes a couple of hours to finish them.
So you can not compute funnels on the fly.
So the idea here is that you compute the funnels offline and do all possible funnels from the start state to the end state, and then use this library to [INAUDIBLE]..
So you have the same [INAUDIBLE] going from the start state [INAUDIBLE] the goal state in the archives.
So these are the set of possible paths from start state to goal state.
And then from each possible trajectory, you compute the funnel.
And the idea of online planning is to now find the sequence of these funnels that you can compose that takes you from the start state to end state without hitting any of the obstacles.
So an important component here is finding a sequential composition of funnels [INAUDIBLE]
So why plan-- you're doing a forward search with funnels.
Why use funnels instead of something like RIT?
PRESENTER 2: I actually don't know what RITs are
Can I guess at the answer?
[INAUDIBLE] thing
Why use funnels instead of RITs
Well, or why use funnels as opposed to anything else
Well, I guess RITs, you have like the classical example, where you want to go through a very-- well, if you want to go through a very narrow region.
Well, first of all, I guess RITs are going to have a hard time finding the narrow region because you'd need like a point to go through that.
And then, RITs by themselves don't really account for errors, maybe some extension.
But funnels, you can be sure that you're always going to stay inside the funnel.
So this if for [INAUDIBLE].
This is to make sure that we don't crash into anything.
Does that answer your question?
Do you want to answer it?
Yeah, I think what he just said, it's-- the point of the funnels is that they provide a control policy.
The actual reference trajectory that you have on the left is the motion planning.
But with the funnels, you get a control policy.
Because when you start executing a motion plan there's always going to be a disturbance.
So you want a control policy that keeps you on that trajectory.
And there's all sorts of control policies.
You could use simple EID controllers.
They work in some cases.
But funnels provide better guarantees for the power plants.
PRESENTER 2: Yeah, so the idea is that you have trajectory [INAUDIBLE] funnels [INAUDIBLE].
You're using this funnels compliance.
You compose them to find your path to those presets that you wanted to go in.
So you can not always compose funnels.
Some [INAUDIBLE] positions that are legal, and some of those are not legal.
So [INAUDIBLE] example, a time going from left to right.
So you have a situation where there is a funnel.
You're trying to compose a bigger funnel with a smaller funnel.
And you have another situation where you're going to compose a smaller funnel with a bigger funnel.
So what if I would have said, [INAUDIBLE]..
So which of these combinations do you think is a legal composition?
[INAUDIBLE] Legal or illegal, I mean that which of these [INAUDIBLE] do not receive the [INAUDIBLE]?
The bottom [INAUDIBLE].. PRESENTER 2: The bottom one is--
--is actually marked because-- with the top one, if you're at the very top [INAUDIBLE].. PRESENTER 2: Yeah, that's true.
So the first one is like illegal.
And the second one is a legal composition.
So why is that?
So if you are inside this funnel, what this guarantees that you will stay inside the funnel.
But it could happen that you can go through the funnel and make your outside [INAUDIBLE] funnel.
And after this step, you can go anywhere.
So you can [INAUDIBLE].
But if you're inside the second case, after you're through the first part, you're still set at the second part then we can decide [INAUDIBLE].
So now let's actually look at those online planning algorithms.
So this the case of your planning [INAUDIBLE].. First you can [INAUDIBLE] with quantum's chronicles.
It makes our analysis easier.
So this is the algorithm that's going through each of the steps one-by-one.
So this is online planning.
You will always one information beforehand.
[INAUDIBLE] And you get to know more about it as we go.
But you still have the initial planned funnel sequence that takes you from the start state to goal state.
And then you check for any new obstacles information when you go out from the sensor.
So this is the region that you can sense some obstacles.
And you also get the current state of your robot.
This is also something concern [INAUDIBLE].. And you check if your current funnel path collides with any of the obstacles you have seen so far.
And in this case, the answer is no.
So then you apply the control corresponding to this funnel for this [INAUDIBLE] location and time.
That take your one [INAUDIBLE] problem.
Then you put Goto and then do this again and again.
So [INAUDIBLE] update from certain information.
And one obstacle is here.
You can differentiate the other one.
When we check at the Replan collides with any of the obstacles-- in this case, yes.
So since it collides with the obstacles, you need to [INAUDIBLE] funnels.
So how do you delete funnels?
You have all this library funnels.
So you go to each of them, and then you find one set of funnels that has-- you find a set of funnels that starts with your current location and [INAUDIBLE] of the obstacles.
So this was a point, first of all, of a new funnel sequence.
And then, again, you apply controls corresponding to the new sequence and doing it two dimensional [INAUDIBLE],, In the position, if it collides, and again collides, so you redirect [INAUDIBLE] apply the control for that funnel.
And get a full solution.
And then it collides, so you have to replan that part.
Then you're OK.
So now you reached the goal.
So those are the planning plan using funnel libraries.
And questions about it?
Can you say more about the connotation of the funnel libraries?
So they generate a set of funnels, which covered the complete set space?
PRESENTER 2: Yeah, so that's like-- there's every possible [INAUDIBLE] end point.
And so then you can make [INAUDIBLE] or least, have it for every initial [INAUDIBLE] you can compose all these [INAUDIBLE]..
So if you're doing it for every initial [INAUDIBLE]..
So you might have a funnel that exactly starts at your initial position.
If there is a funnel that already started somewhere, where it goes through the funnel-- goes through your initial funnel, you can use that.
So it's a truncation of your funnel.
[INAUDIBLE] Any other questions?
Why can't you use-- if it really is an online problem that you're not going to run out of a funnel, that you won't reach a dead end, will it be adaptable enough to make a U-turn at all times, either that there's come latency that would perfectly time it.
And the robustness, that you'll always be able to find a path, even if we have guarantees that you'll always stay within a funnel.
PRESENTER 2: So are you asking what the situation, like the funnel, if you're going [INAUDIBLE] backtrack?
Yeah, just because these obstacles come out of nowhere, seemingly, that you don't have a guaranteed that the funnel is safe through the entire duration of our execution.
PRESENTER 2: Yeah, so that's a valid point.
[INAUDIBLE] you can use the adaptive [INAUDIBLE] backtrack [INAUDIBLE] combined with the [INAUDIBLE]
Even if the robots can't actually backtrack?
PRESENTER 2: If the robots can what?
Come back, or--
I mean, I guess, it's just like a UAB, it couldn't do that.
I'm just saying, what guarantees do you need to have about the obstacles , since it's an online setup, right?
PRESENTER 2: Maybe you'd have some initial information about some kind of obstacles.
You can find a [INAUDIBLE] trajectory-- you can find a trajectory that goes from the start sequence.
It's like [INAUDIBLE]
Well, I'm just going to say.
If you're in some kind of glider that can't necessarily-- like a quart of stop, go backwards, stuff like that, depending on how funnels you can plan backwards.
But if you're in some kind of glider going straight at a wall, there's no guarantee.
You're just screwed up.
I mean, even humans, they make errors.
There's situations that humans can't dynamically get out of.
So if your system is dynamically not going to be able to make a sharp enough turn or go straight into something, no matter how smart your planning algorithm is, no matter if you're like, there's a wall right there.
Stop.
If you can't stop, and you just go straight at it.
There's obviously situations where you can pretty much [INAUDIBLE] algorithm [INAUDIBLE].. You're just going to crash.
Hopefully you're in a situation where you're not [INAUDIBLE]..
It's also based on how far away your sets are received.
Your sensor can only see 5 meters ahead, and you're going 5 meters per second, [INAUDIBLE].. You're going to have make perfect decisions.
But if your sensor has perfect information about the environment, then what seems like the initial best path may not actually be once you know the full [INAUDIBLE]..
So it depends on the [INAUDIBLE] sensors.
PRESENTER 2: Any other questions?
OK, so finally I have some demos showing simulations of [INAUDIBLE].
Let's see how [INAUDIBLE].
So the [INAUDIBLE] applications.
So we have seen how to use funnels to do robust motion planning.
We also have the offline planning and come to do online planning using a sequence funnels and the funnel libraries.
I will talk more about computing these funnels and flow tubes and other kind of reach sets.
PRESENTER 3: Yeah, so I'm going to talk about how to actually compute flow tubes and funnels generally.
I'm going to give an example of what each one is used for.
So we had the question before, how are flow tubes and funnels different?
Flow tubes are-- so easiest way is they're both some kind of way to guarantee that you're going to-- the robustness guarantees.
So say you are here.
You want to be here.
They're both going to-- in most planning algorithms, you've got some kind of path from here to here.
But say there's some disturbance [INAUDIBLE] down here or something.
You don't know if you're going to be able to reach there or not.
What a flow tube is is it's some kind of set of trajectories that will basically go from some initial state, which are here, to some goal states over here.
And basically, [INAUDIBLE] if you're within this flow tube, and you get pushed out on a different trajectory, you know how to get from here to here anyway.
So you don't really care if you're pushed, as long as you're still within here.
A funnel-- let's see.
So a funnel is, say you compute some nominal trajectory around here.
Using the system theory and stability theory, you know that, with your control inputs, you're going to stay at each of these points.
And you're going to stay within here within these circles.
So if you get pushed out here, you know you're going to be able to get back onto your path when you're going to try to converge back to your nominal path.
So you have some sort of funnel.
Yes?
I have a question.
How complete is the funnel?
It sort of related to what Brian was saying.
PRESENTER 3: Completeness as in like these edges are--
Well, if you have a high dimensional state space, does the funnel cover it completely?
There's a set of funnels.
PRESENTER 3: A set of funnels--
So the question wasn't about if the individual funnel, but in the particular [INAUDIBLE]..
When he generates a set of funnels, does the set of funnels completely cover the space?
PRESENTER 3: It depends how much time you want to put into it, generally.
If you only compute two funnels, one of them goes over here and one of them goes here, then you're not you're not going to be covered in these states.
So it depends on how much computation you want to do offline previously.
So you get a set of funnels that's here, another one that's right here, another one that's here.
And you have your big library of funnels that cover the whole space
I think I skimmed that paper.
And, yeah, I think it is probabilistically [INAUDIBLE].. PRESENTER 3: OK, yeah.
All right, yeah, but I mean, as far as computing where you're going to go, if you compute that path of it, you should be good.
But yes, you're right.
The probabilistically-- So here is what I explained.
Normally you have your optimal trajectory.
But if you get pushed off of it, you may not get back to the goal state.
But if you have a set of trajectories that go from some initial states to your goal state, and you stay within that, you know how to get to your goal state, even if you get pushed.
Again, so some ways to approximate flow tubes, you can use some kind of polytope, which is as Gabriel explained previously, as long as it's a convex polytope, you can take that cross-sectional area and you can propagate it down here for the tube.
You can also use ellipsoids or rectangles as cross-sections.
And important point is to know that they're intercross-sectional approximations.
So as you see here, you may-- this would be your actually flow tube.
But your approximation has to be an inter one, because that guarantees that you're with a flow tube.
But it means you may also miss some of the outer trajectories , which also, if you want to compute a more thorough approximation area you get better flow tube representation.
OK, so robust planning, you plan a trajectory from here to here, and that should be good.
But then you get disturbed.
You're still within the trajectory, so you're still good.
But some work done by Professor Williams and Hoffman shows that-- so what do you do when you're actually outside of the flow?
Now what you can do is temporal planning.
So we're using the algorithm that we previously learned in class.
You can take your goal state and move the timeline back to a different time.
So here, this flow tube is, if you're here in a certain amount of time, can you get to here?
And if you get outside of it, then you can't do that anymore.
But you can just use our algorithm and move back the goal time.
And then when you move the goal time, the whole funnel shifts because now you're going to be over here.
And you don't care about getting here at this time.
You can here at this time now.
So then you move over.
And now where you, you get back into it because you're now inside the flow tube again.
OK, so one example is a humanoid footstep planning.
So this take a complex systems.
You have a bunch of different parts of the system.
You have your center of mass dynamics.
You have your leg dynamics, foot dynamics.
So what you're trying to do is plan your footsteps forward and through time.
So you have your original plan of where you want your footsteps to be for different parts.
So there's discrete things that happen in the system, as we've previously seen in a lecture.
So one thing that can happen is your left foot hits the ground, and then your right foot lifts off.
You're left foot lifts off-- or your right foot hits the ground, and then your left foot lifts off.
So those are different things that can happen.
So you have certain temporal constraints on the system that you want to [INAUDIBLE] by.
And so as you start planning, you have flow tubes to get you from one step to another step.
And that guarantees that you're going to stay within those.
Even if you get disturbed and get pushed, your feet can still get from one step to the other.
But then also, with the thing that we saw in the previous slide, you can move the timeline back and forth, as long as you're withing your temporal constraints that you had.
So those are flow tubes for the feet.
And that would guarantee that your feet go where they want to.
But you can decompose the system to also have flow tubes for the center of mass.
So what the center of mass does is to be broken down into separate flow tubes.
So does that make sense?
So this is the humanoid footstep planning that we saw previously.
And now you're going to understand how the feet are planned to go to each of the blocks.
[INAUDIBLE] go video
I have a quick question.
You said that you have a separate flow tube for the center of mass.
But aren't they really just the same flow tube, like a high dimensional-- PRESENTER 3: Yes, but that's-- when you decompose this and then you [INAUDIBLE] flow tube rather than a full system high dimensionality
Is it hard to recouple them?
How do you know that they're consistent [INAUDIBLE]??
PRESENTER 3: Well, it still adheres to the dynamics of the system.
So you can couple how you want, how you want it to move.
And you want to move any basic constraints.
But eventually, as you said, when you move your feet, your center of mass is going to move.
When you step forward, you're center of mass is going to move.
[INAUDIBLE] analysis of it
Can I make a comment about that?
Yeah, there's a technique called feedback linearization that can be used to decompose a complex non-linear dynamics assuming to the set of linear ones.
And that allows you to treat the center of mass as a [INAUDIBLE] separately from a control in the flow tube computations and while guaranteeing that [INAUDIBLE].. PRESENTER 3: So this is work done by Professor Williams experiment, which is actually [INAUDIBLE].
This is the robot doing the same kind of thing on Mars.
This is walking on stones and planning its footsteps through a pretty complicated environment.
The stones are not exactly in the path where you would normally want to walk.
But moving them around, as long as you're within the flow tube, you'll get to wherever [INAUDIBLE] Awesome.
So that was how to compute flow tubes and-- oh, and another way you can also compute flow tubes is by learning.
So you can have some kind of example, move through trajectories from one state to the other.
And you can estimate a flow tube based on where the trajectories are, as long as the flow tube encompasses all the possible trajectories
Is the dynamics always linear?
Because I though if your polytope, and it would be like kind of, is it a complex proposal when you would use a non-linear dynamic, then you move a little bit.
Then you would move a little bit, not being the convex shape then.
PRESENTER 3: I mean, you can plan non-linear systems too.
But as far as I'm concerned, [INAUDIBLE]
So, yeah, the polytopes generally require linearization with the system.
And there's a limit to the validity of the-- the range of the linearization.
And that's concluded in-- that's one of the things that limits the size of the polytopes
But it is the case-- I mean, you've just got to understand.
The thing that you do for non-linear systems, which is a piece-- if you're making a piece-wise linear, but you are-- you are linearizing them at the w step.
And there's other techniques that people have developed with flow tubes where you do require the input system to be linear.
Right here it can be non-linear.
But... it's an approximation.
PRESENTER 3: OK, so funnels, the goal is basically to find a region that guarantees you safety on a bounded uncertainty.
So there are regions of finite time variance throughout all time during that trajectory that you've defined.
And so in practice, there's a trade-off between computation time and guarantees.
So if your trajectory, if you compute more nodes, then you're going to be able to fill in these little gaps between each of the ellipsoid.
Whereas, if you lessen them, it's going to take less time to compute each ellipsoid.
But it's also going to not be as guaranteed.
And especially important when planning is that, if you're planning in some kind of point cloud, then some of the points may be right here, where one ellipsoid doesn't intersect the other.
And you might think that's a safe path, when actually it's not at all.
Yeah, so an example is this little glider, which is given in the paper by Anirudha and Russ.
And so what you start doing is you create your system, your function.
And this is dependent on your state, time, and w, which is some kind of bounded uncertainty.
And you choose the bounded uncertainty, which means that you choose some kind of reasonable estimate of how much can happen to your system that you didn't account for.
So you start it.
Your velocity can be different from what you want.
Say your nominal velocity is 10 meters per second.
You can have some kind of drag, or you can have some inconsistency in your motors.
And so what you expect is that there's a plus or minus 0.5 meters per second to your nominal.
So when you start running your equations, this is what you expect your robot to do.
And then that can vary forward and backwards depending on what your actual was.
But then you add some kind of wind on your system, which-- some kind of cross-sectional wind, [INAUDIBLE] direction, which we see there.
And so what that ends up doing is pushing you system away from where you want it to actually be.
So now you have some kind of velocity uncertainty.
And you also have wind that can push you one way or the other.
So how do you deal with that?
Well, you start with your nominal trajectory that has a plane flying perfectly straight.
There's no uncertainty.
Everything is deterministic.
And it acts like you want it to.
And then, so right there the wind hits it.
So what do you do?
You have to have some kind of control, some kind of controller to bring you back to your nominal trajectory and try to control you around the system.
So a standard way to do that is to have a cause function and create your optimal trajectory.
And this cause function basically, it kind of mirrors a LQR controller.
And by solving a lot of these equations, which is a pretty standard process, and to find out how to do that, you get a controller that tried to bring you back to your nominal trajectory.
And so you get back.
You're happy.
But you don't have any guarantees just with the controller.
You don't have any guarantees on how far you can get pushed.
You don't have any guarantees on where you're going to go and where your system really can be pushed.
So when you compute the funnel, what you do is you choose a Lyapunov function.
And I'll explain what that is in the next slide.
And you try to minimize the space around that function that tried to bring it back.
So what this is is-- so you have some kind of your initial state.
This would be the initial state.
And the whole region, what you're trying to do is minimize it around it.
So does that makes sense why you do that?
So your trajectory is basically going to go anywhere around here.
And what you're trying to do is find the region of space that minimizes-- that encompasses all of the places that your system is going to get to.
So that's easy to do when you it out and you know exactly where your system's going to go.
It's easy to just shrink your ball around it.
But you actually compute that.
No, forget it.
OK.
So when you have that for each point in the trajectory, you know that you're going to stay within here, because you're going to have some kinds of-- from your nominal trajectory, you're going to have different places where your system can go under the given conditions of velocity and wind uncertainty.
You're going to go anywhere withing that.
But as long as you know you're still in that ellipsoid, then you're going to stay within those ellipsoids as you go down the path.
Does it make sense?
Good.
So the reason they use Lyapunov functions is there's different ways to measure stability.
And so I'll move forward to this.
So there's different ways to measure stability in a non-linear system.
So in a linear system, you're either unstable or you're globally exponentially stable.
So you're going to converge back to your-- to whatever stability equilibrium, what you have at an exponential rate.
But there is also, in non-linear systems, there's global exponential stability.
There asymptotic stability.
And there's stability in the sense of Lyapunov, which is what we care about.
So the difference between that is-- so say this is your nominal.
And globally exponentially stable means that no matter where you start anywhere on the board, you're going to be converging back to this equilibrium going at an exponential rate.
So that's the best case scenario.
That's what you want.
Asymptotic stability means that you're going to be going from somewhere on the board, within a certain region, back to this nominal trajectory.
But it doesn't have to be an exponential rate.
So you can do something like that, as long as eventually you sort of converging back to it.
And then there's stability in the sense of Lyapunov, which is-- so, say you're here.
Your system can be doing anything, anything crazy, as long as it stays within some certain thing, which in the 1D case we saw was the little ball going around it.
So imagine there's like a ball for each of these.
and this is where your trajectory could possibly go.
As long as it stays within that, you can say your system is stable in the sense Lyapunov.
Does that make sense?
Cool.
So know this, does that--
I just have a basic question.
So the Lyapunov function is around a stable set point.
But the trajectory has non-zero velocity.
So what is the point that it's stabilizing around?
PRESENTER 3: You can stabilize around a trajectory, which also what the funnels do.
So we have your little point here.
And this is your nominal trick, some kind of state.
And you're wind can knock you this way.
Your velocity difference can knock you this way or this way.
But as long as you're still within that, You've got to converge back to it.
And this is the value for your Lyapunov function.
And these are the different states you can have.
An important thing to notice is the derivative is a negative definite-- or negative semi-definite in the case.
And that means that basically all your trajectories, no matter where you go, they're going to be going down this cone.
And eventually they're going to to down to some value.
So does that make sense?
That's like pretty amazing stuff and your system stuff.
Cool.
So once you know that, it makes sense to use Lyapunov functions as candidates for computing funnels.
So an ellipsoid is actually defined by a quadratic Lyapunov function, which is down here.
[INAUDIBLE] shown the equation for an ellipsoid is some V times S. So that was the equation for an ellipsoid that Gabriel showed previously.
The quadratic Lyapunov function has the same structure, where you have this being x to [INAUDIBLE],, which is basically the error of you state.
So it's some kind of exact mean of the state that you're actually in minus x0, which is the nominal point that you want to reach.
So this defines the region around your-- the nominal point, which would be somewhere in there.
And the ellipsoid is computed by doing that minimization that I showed you on the previous slides.
And then you can figure out the region inside of it by solving that equation.
And if it's less than 1, I believe you inside.
And those are all the red dots that show that you're inside of it.
So that's used when computing trajectories.
You solve for-- say that you're inside a point cloud, you know which points you're going to get inside of this ellipsoid.
So when you're planning, you solve that equation for each of the ellipsoids.
And you know if you're going to hit something or not.
All right, so this is a video that [?
Ani ?]
did.
And this is using the funnel libraries for collision avoidance.
This is showing for just one funnel.
The glider dynamics were shown in one of the previous slides.
They were pre-set by the glider dynamics.
But [INAUDIBLE].
So you see right there that there's objects, and the glider plans a path around them.
So the funnels-- this is one possible one.
Right here, I'm pretty sure they tell it where objects are, even though they're not actually there.
And it finds that funnel being the one the guy wants.
And you see that throughout the entire path, the glider will stay within the funnel.
And then when you're searching with the online planning algorithm that was shown previously, it goes through all of these funnels and eventually finds the best safe one.
You won't always find a safe one, as discussed.
In this case you did.
But it finds the best one around the path.
So one way that it finds the best one is by finding all of-- so if you're planning a point cloud, you find all the points that lie with the ellipsoids.
And the one with the least amount of points is probably going to be the safest one.
So here's a bunch of different obstacles.
And it safely maneuvers all.
And there's some pretty dynamic movements, which is cool too.
So within that there is-- let's see.
I'm going to back to-- is there a way to get back to the full screen [INAUDIBLE]??
[INAUDIBLE] PRESENTER 3: [INAUDIBLE] So with the online planning algorithm, here's one of the possible paths.
You have your point cloud of trees.
And those were shown in the previous, the little round thing with the green foxy tree-looking things.
Using a sensor model, you get a point of that.
And as you're going, you plan your path.
And the way that this works is every five meters, you plan a path.
And then as you're flying through it, you get new sensor information.
And you can replan your path using the algorithm.
And that will guarantee you that you're safe.
And when you're flying, obviously you don't see the funnels.
But right here you can see there's probably one funnel here and then another funnel that plans this way, another funnel that goes to here, and then another one that goes there.
And that's a full path.
And there's also a cool one , because it shows that you go through the tree.
And that's where one of the funnels is rather than going around.
It is possible that you start computing a new funnel-- or you start searching a new library here.
And you may not be able to make it up the way in time, given what speed you're going at.
Does anyone have any questions?
And this is the online planning algorithm that we saw before written out.
And some of the references, the paper from the video that we just saw, that's given here.
And that also go to the example [INAUDIBLE].. Frazzoli was one of the original people that computed a flow tube.
Hoffman and Williams did the temporal planning for footsteps and decoupled humanoid system.
And I did most of the stuff on [INAUDIBLE] for that years ago.
Another important thing to note that was a big thing in my personal research was that even though you may note be able to compute-- or that you may not find a funnel that necessarily goes-- finds a safe path, you can shift that funnel around.
You could even move it slightly up and down, as long as the first ellipsoid is still within-- as long as your initial state is still within better ellipsoid.
And that's easy to do because you can just give it a simple transformation between one state and the other.
And if you transform this S matrix, then that will transform your entire funnel and it will shift it around.
So you can find your best funnel, even though it may hit something.
If it was slightly shifted to the left, you can just shift it using that transformation.
So that's one of the [INAUDIBLE] funnels too.
Does everyone understand funnels and flow tubes now?
So I have two questions.
So the side of that funnel is computed by the transform script.
Is that right?
PRESENTER 3: Yeah, if you want to get it some ridiculous wind, like 1,000 miles per hour, then you're going to get huge funnels.
But that's under some kind of reasonable-- you know in this region of the forest or something there's only three mile per hour winds or something like that.
Yeah, that's based a lot on what your counts are
So do you have any algorithm that in your mind that is not using offline [INAUDIBLE]??
So this is like mostly you do the offline and the you do the online search.
But if you were in a wild environment, you do nothing.
Then do you have any [INAUDIBLE] that isn't-- PRESENTER 3: I mean, there are algorithms that will-- well, just the basic controller will try to get you back to nominal trajectory.
But there's nothing that I know of at least that will guarantee you safety and will give you an all-encompassing region like flow tube or a funnel.
So flow tube you find there's an infinite number of paths that will get you from here to here with your system, basically every little minute change that you have.
And that obviously takes awhile to compute and approximate.
And the funnels, it takes awhile to compute the regions where it can go.
So I don't know anything that will online compute this region.
But online you will be able to compute some trajectory.
And then let your controller will take your state error and control it.
But it won't guarantee [INAUDIBLE]
With the funnels, how do you compute your initial set of states, or the allowed initial set of states?
You have these uncertainties, like for the wind and velocity, but what about the initial state before those disturbances happen?
You'd probably want more than one initial state
Here's and eraser.
PRESENTER 3: Yeah, that'd be good.
Cool.
Thanks.
So with the funnels, so you have your nominal trajectory again, and then your ellipsoids around it.
So when your system starts, you have your initial state.
And your initial state is just-- it's going to be here at time t0.
But over the time computed, you can go over here, or anywhere around here really, depending on what happens
But I just me that you'd want to apply the funnel multiple times.
And you're initial state may not be exactly the same.
So it should be possible to use the same funnel even though the initial state is off by a little bit.
PRESENTER 3: Yeah, OK.
So say that you chose this funnel first.
And eventually what actually happens is you do this and you get pushed around.
And then you end up here instead of this funnel.
As long as your next funnel that you plan-- so you can start a new funnel at any point in this trajectory, as long as the next funnel is also-- as long as you have V intersection between the point and the next funnel.
And if you're within this region, and you plan the next funnel, what do you do is you plan the next funnel with this being the original x0.
So as long as that's you next x0, and then this will take on a new nominal trajectory
Yeah, I'm just saying, asking how you, like for the one all the way on the left, that one, yeah, how do you define how big that initial region is?
PRESENTER 3: Oh, OK, so that one, say, if you start, say, not moving or something and you got pushed?
Is that what you're asking for--
Well, I mean, I understand that you're using the V for uncertainty, and V and W. So that essentially defines how big those funnels are.
Do the also define how big the first one is?
PRESENTER 3: Yes.
Yeah, the do
OK. PRESENTER 3: So what you're doing, because what you want to do is use the funnels and have some funnel library and be able to use those throughout all time.
So you don't want-- so I guess what you're asking, the first funnel, you're going to start at some initial state 0 with no velocity, stuff like that.
And so you're not going to have any uncertainty on the first thing because you know where you start, the initial state
Yeah, I'm just asking independent of the wind and stuff like that, you might just be starting at a point other than what that nominal point is.
So you may want to define that initial funnel size by some other criteria besides wind.
Is there--
So [INAUDIBLE] sensor how sure you are about where initial location is
Yeah, and then-- I mean, there's sensor uncertainty.
But you just may happen to be in the-- if you're doing the glider experiment 10 times, you may not start it from exactly the same position every time.
PRESENTER 3: Yeah, you should-- your funnel, every time you plan a new funnel, you start at the nominal trajectory above that new funnel.
So you're not going to have this funnel.
And you're never going to start over here on the first one.
You're always going to start at this point.
[INTERPOSING VOICES]
I though the point of the funnel was so that you can reuse it without having to replan.
PRESENTER 3: Yeah, you can move the funnels around
Oh, you can move?
OK. PRESENTER 3: Yeah, yeah, that was-- so every time you start a new funnel, you start at this nominal trajectory.
But the next time you use the same funnel, you could be over here and just shift the funnel down to exactly the point.
So the initial state is never going to be anywhere starting at t0, it's not going to be somewhere other than the nominal trajectory.
But at t1 it could be over here.
And so you just shift the whole funnel around.
And as long as your t0 starts at the new funnel, and as long as t0 is within the funnel above t-- last funnel, then you just shift it around.
You can also turn it, as long as you can multiply this and this
It seems like the shifting and turning is a pretty complex search problem in itself.
Is that a hard problem?
PRESENTER 3: No, that was actually the-- [INAUDIBLE] I got rid of the slide.
I might have gotten rid of that slide.
I shouldn't because that was the work that I did for the [INAUDIBLE].
Let's see [INAUDIBLE].
But, yeah, there's a bunch of different ways to do it.
I did a pretty not very smart search, where I just looked for the funnels, the safest funnel being the one with the least number of points in the point cloud within that entire funnel.
And then-- oh, man, I did get rid of this.
But basically what I did was find the safest funnel.
And if there still was a collision within the safest funnel, the I admit I had a minimization problem as you shifted back and forth.
And there was a cost function that-- see if I can bring it up
So you're doing an optimization of the online running of the-- PRESENTER 3: Yeah.
That was an input that was implemented in realtime simulation.
But that wasn't implemented on the actual glider as of [INAUDIBLE].
So let's see, all right [INAUDIBLE].. And you also want-- I also made is so you minimize the amount that you shift it to.
So you don't want to shift it completely somewhere else because it won't be in your path.
And you'll end up doing some dynamic [INAUDIBLE].. Like, you'll be flying, and then if you compute a funnel that's here that's kind of safe, but you shift it all the way down to here.
Then you're gliders going to be doing something crazy.
And that may-- I guess it would be [INAUDIBLE]..
But I minimized that for at least [INAUDIBLE] shift it.
Does anybody have other questions?
Yeah, so does anybody have any questions on--
Can I ask one?
So you've given us models of bounded uncertainty in your dynamics.
So if you use probabilistic models, then your funnels start looking like risk allocations.
Is there an interpretation of one in terms of the [INAUDIBLE]?
PRESENTER 3: I don't know.
No, but I'm sure if you look it up, yeah.
I'm sure somebody's been working on that.
I'm not really sure that fits-- I know I haven't done that
Talk a little bit about the interaction between the flow tube selection or the funnel selections and the underlying [INAUDIBLE].
So let's say you have something that's planning the reference trajectory that's separate from your flow tube selection or your funnel selection.
And then you have to select the funnels and flow tubes that allow you to follow that trajectory the best you can.
How do the interact if you have a flow tube-- or don't have a safe flow tube or funnel [INAUDIBLE] reference trajectory?
PRESENTER 3: So you're asking basically the difference between planning with a funnel laddering and trajectory laddering
Yeah, kind of sort of-- well, because I know the checkoff planner that uses an incremental pathfinder, so the planned path or something like that.
And then you fit-- are you fitting flow tuber to the reference trajectory?
PRESENTER 3: Oh, you do that off and on.
So you compute-- they're called funnel libraries.
Or usually they're trajectory libraries.
So ever computing a trajectory in realtime for a fast-moving dynamic system is going to be expensive.
And then computing a funnel on top of that is going to be even more expensive.
So what a lot of people do is they compute the library offline for a trajectory and plan using that.
But on this one, you compute your trajectory, compute the funnel around that trajectory offline.
And then you search through the funnel library.
You don't search through trajectories and then try to fit a funnel around that one.
You just search through the funnel storage.
PRESENTER 2: If you don't have a very good library, then you will not get a very good trajectory.
PRESENTER 3: Also, a cool thing about funnels versus trajectories is a lot of trajectories will be discretized for speed, so you have your nodes.
And so, say this is what your trajectory does, and you have node points here, and you're trying to fly past a wall or something.
If your wall goes here, you're algorithm is going to search these points, and it's going to say, OK, well, this is a safe path because none of the node point intersect with this wall.
So you fly [INAUDIBLE] a little [INAUDIBLE]..
But a funnel, since you have this region around it, you-- no, I did a bad job of drawing that, but OK.
When we search the algorithm and see if it's less than or greater than 1, you're going to find all these points being in that region also.
I guess that depends on how well you want to discretize it.
If your trajectory is [INAUDIBLE] of really, really small points, then you're not going to run into that issue
Seems like you could-- like when you're building a normal RIT without funnels, from concave points, if you say, yeah, they're illegal, it seem that if you're automatically adding on to that funnel instead of generating an entire trajectory and then making a funnel, you could save computational time for [INAUDIBLE] things like that.
Do you know if that's done?
PRESENTER 3: No, I don't.
I do know that, I mean, computing, you obviously want to minimize computation time.
But it's not a critical thing when computing, I guess, funnels and flow tubes, because you do that all offline.
And you want to-- I don't think it's even close to the point where you go to online to compute a funnel.
It takes like six hours to compute it, so-- depending on how many dimensions your state has.
So, yeah, it's not like you're flying and you can compute one of these things.
You'll crash like [INAUDIBLE].
OK. [APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.MIT.edu
Self-driving cars are pretty cool.
It's pretty hot nowadays, so we're going to use this as an example.
And in the past we have seen how we can give an autonomous vehicle a goal-- some condition we want it to reach.
For example, we might have a car that wants to pick up some passengers along the way, get to some destination, and maybe navigate some worlds.
But let's consider that for something like a self-driving car, we also have a number of other goals or requirements that are sort of implicit along the way, in that we want our self-driving car to obey the rules of the road as it's driving.
And these aren't just goal conditions which is a sort of single goal that you're trying to reach, but these are sort of goals that you're trying to maintain throughout the path that you're traveling.
So one example of this is a traffic light.
So maybe you guys can help me out with this.
Just in plain English, how would you describe the rules of how you would want a car to behave when it comes across a traffic light?
Anything?
It's very simple.
Yes?
Slow down when you see yellow, stop when you see red, go if you see green
Yeah.
So pretty much stop if you see red, go if you see green, and specifically, if you see red and you stop, you're going to want to stop until you see the green, at which point that sort of condition that you're stopping goes away, and you're able to move on.
And in addition to things like obeying traffic lights, there might also be some other rules of the road that we want to follow.
For example, we might want to always stay within the speed limit.
And then there might also be some other practical conditions of driving down the road, which is that at some point we're going to need to refuel.
And if we consider a car that might be driving for a really long time, one logical statement we could say about the path that this car takes, and the sequence of states that it goes through is that at every point in time we want the plan to have some future state when we are going to visit a gas station.
So this isn't just a single goal that we're trying to reach, but over a very long time, even if we consider the car to be driving an infinite amount of time, at every point in the time, we're going to be thinking ahead that there's going to be some time when we reach a gas station and refuel.
So these types of conditions, things like staying at a red light until it turns green, always staying within a certain speed limit, or always having some path in the future-- some state in the future when we reach a gas station, are things that we can express with a type of logic called temporal logic.
And there are two things that we hope you're going to be able to do by the time you leave this lecture.
The first is be able to model these temporally extended goals using a specific type of temporal logic called Linear Temporal Logic, or LTL.
And secondly, we're going to hope that you'll be able to actually apply this to planning, create plans with these temporally extended goals, and in addition just sort of this regular planning we've been dealing with, actually incorporating this idea of preferences into your plans.
So a preference basically is not a required condition, but a desired condition that you can use to select between alternative paths.
So you know, you might have many different ways that you could reach your goal, but we can use some of these temporally extended goals, like maybe you prefer to break as few of the rules of the road as you can, and you can use those a preferences to choose between plans.
So just as an outline, first we're just going to do a little introduction to linear temporal logic.
We're going to talk about what this is-- what LTL is, why we want to use it, and we're going to go through the syntax and the semantics of different LTL operators.
Then we're going to walk you through some example LTL problems, and actually talk about complications to plans-- how you create plans with these linear temporal logic and temporal goals.
Finally we're going to incorporate preferences into this, talk about how you express preferences, and specifically talk about a language called LPP that allows you to plan with temporal logic and preferences.
So now for the introduction to linear temporal logic.
Temporal logic at its core is a formalism for specifying properties of a system that vary with time.
So these aren't just conditions that are true at single state, which is what we've mostly been dealing with prepositional logic.
Right, so we've been saying things like, condition A and B, but not C, are true at a given state, and that's prepositional logic.
Temporal logic is sort of a layer on top of that, when we're dealing not just with what's true at a single state, but actually extending over time as the system moves through a sequence of states, and expressing properties on a temporal level.
So you might have a system that can be represented as a state machine, and a [INAUDIBLE],, and it can go through a sequence of different states.
And while you might be able to represent the whole system like this, if we actually execute the system we're going to get one path through the system.
And that's often called a trace of the system, and you can also think of it as a timeline of the system.
So one example timeline of this system is it might start in state A, and then go to state C, and then go to D, and keep looping around in D forever.
So that's one trace, or one timeline of the system, and that's really just a sequence of states that it goes through.
And in the past we've seen these in some of the problem sets.
For example when we were modeling the warp reactor, we had the valves that could transition between open and closed states.
It could also be the whole system-- the starship going through different planets, transitioning through a larger set of locations, picking up passengers, dropping them off.
So these could be quite complex.
And so as I said, previously in our problems that we've been modeling, we've been going through a system, and we've been searching for a single state that satisfies all of our goal conditions.
Maybe we to reach [?
Lavinia ?]
and save a certain number of people.
But with temporal logic, we can actually express goals that are satisfied over a number of states, and this allows us to do some more expressive goals than we could capture just by looking at a single state at the end.
So for example, what if a problem requires some condition to be met until another condition is met?
And for example, you know, when you see a red light, that implies that we should stop until we see a green light.
And if we look at a timeline, or a trace of one of these systems, that might look something like this.
So we're going along through our system, and at one of the states, we see the red light.
So that adds this condition that we should stop.
And then that condition will hold forever until we see a green light, at which point that condition is dropped, and we can keep moving on.
Another example where we might want to use temporal logic is, what if our problem requires a condition to always eventually be met.
And this is what I was talking about with the gas station.
So we want to be going on forever, and there should always be some point in the future when we do have a state where we're at a gas station.
So we might start our system at a certain state.
And our plan says that at some point we do reach a gas station, so that's good.
But then when we get to the next state, we also want it to be true that from that point on, we're going to reach another gas station, and after that, going on and on and on into the future.
These are things that you can't express very easily with the types of logic that we've been dealing with before.
So one important distinction that we should clear up before we really dive into the syntax of how these linear temporal logics work is the difference between branching time and linear time.
They're really two different models of time that we can be working with.
There's two different types of temporal logic that exist to express these different types of time.
So linear time is more similar to the world that we live in, unless you're living in some sci-fi movie where time is branching into different realities, and all sorts of crazy stuff is going on.
You can think of time as just this linear timeline.
So right now, it's one future state that's going to happen for me.
And you know, regardless of how many options I have, there's one realization of the time.
And what this lets us do is that it gives us a single path for our system, and we can reason very exactly about the conditions that are met within that path.
And this lets us describe what will always happen in a system.
For example, we can guarantee that we always stop at a red light until we see a green light, or we always stay within the speed limit.
This is contrasted with a different model of time, called branching time.
And in this model of time, at each time instant, we consider all possible futures.
So if we have multiple actions that we could perform at a given state, we're going to consider different timelines that branch off for each of those possible behaviors.
And this results in alternate courses of time that we can reason about as a set.
And instead of thinking about this as always conditions-- so, our path always does this-- we can think about this more in possibilities.
And if we're using branching time, we can say, you know, at this state and time, there exists a future path in which we will always stay under the speed limit, or, at this point in time, all our future paths satisfy a certain condition.
So whereas linear temporal logic is more about guaranteeing what will happen on a given path, branching time allows us to reason about what might possibly happen in the system.
So for our purposes with planning, in part because it's more attractable, and in part because it gives us an exact analysis of a given path, we're going to be doing linear time, and for that reason, we use what's called linear temporal logic.
There's a different type of temporal logic.
There's one called CTL and CTL* that deal with branching time.
And those are sort of an extension of linear temporal logic.
They add a few other operators that allow you to reason over multiple paths.
But for our purposes, we're just going to be using the linear time model.
So kind of as a recap, what linear temporal logic involves-- we're using this linear time model.
Another important distinction is that we're actually going to be talking about infinite sequences of states.
So instead of us just reasoning up until a certain goal condition is met, we're thinking about systems that could theoretically keep transitioning over time, over and over and over into the future.
So if you have one of these state machines, you could think, at each time step, it's going to visit the new state.
And if you give it enough time, it's just going to generate an infinite sequence of states.
And we can actually express properties that can hold over an infinite number of states, not just the discrete number of states that we might usually think of to reach a goal.
And linear temporal logic can actually be modified to work with discrete set of states.
It just requires an additional operator.
But the ability to reason about infinite sequences of states gives us a lot of power in talking about guarantees for our system, regardless of how much time we're talking about it.
Finally, all the operators we're going to be looking at in our logic system are what we call forward-looking conditions.
So when we're given a state we're going to reasoning about what's going to happen in the future for a potentially infinite amount of time in the future.
There's sort of a symmetric version of this logic that reasons about the past.
So for example, we can say-- talk about guarantees that something will eventually happen in the future-- there's sort of a symmetric version where we can talk about, given a certain state, we can guarantee that something had eventually happened in the past.
But for us in that we're going to focus on planning-- we're typically planning about what to do in the future, and it makes more sense for us to use this future of updating the set for operators.
And so I guess one little nuance here that I sort of pointed out in the previous slide is that because we're using a linear model of time and not a branching model of time, we're going to be expressing properties that happen over a single path of states, and not properties that happen over several path of states.
So finally I'm just going to talk about a couple of the main application for linear temporal logic before Ellie will dive in, and actually talk more about the semantics and syntax of how we use this logic.
So the first menus is verification and model checking.
We can use this linear temporal logic to create guarantees about a system, and formally verify it for a couple different properties.
For example, we can talk about the safety of the system.
We can say, over the course of this system, we guarantee that it will never enter a state of a certain condition.
We could also talk about the maintenance of properties.
So over the entire lifetime of this system, we guarantee that a certain condition will always hold.
The other main application for temporal logic, the one that we're going to be focusing on, is for planning.
And this planning using these temporally extended goals that I've been talking about, and it actually extends nicely to talking about temporally extended preferences.
So this is given that we have a number of different paths that could reach a goal, we can actually look at all those and see which paths satisfy certain conditions over the sequence of states, and use those conditions as a way of choosing which ones we prefer.
So with that, Ellie is now going to talk about some of the syntax of linear temporal logic
Hi guys.
My name is Ellie, and I'm going to be talking about the syntax of LTL.
So an LTL formula is a series of states.
And at each state, you can have various propositions than can be either true or false.
So for example-- is it OK if I erase this, Steve?
Yeah.
Go ahead
So for example you can have a [INAUDIBLE] animation show that has a series of states that goes on to infinity.
And at each state you can have propositions that are either true or false.
So the traffic could be red could be an example for a proposition.
We could be stopped, et cetera.
And there's various logical operators that we can apply to these formulas-- these sets of states, and we can determine if these logical operators are either true or false about each set of states-- each formula.
So there's the not operator, the or operator, the and, the implies, if and only and, and like I said, the state-- each proposition of [INAUDIBLE] can either be true or false.
So just to go through some examples of these, the logical operator true says that-- essentially what the operator true says is it's not saying, oh, this state is true.
The operator just true says that there are states for all infinity.
So true is always true.
So no matter what you had for your state here, the logical operator true holds.
However, when you talk about the logical operator true with respect to a certain state or a certain proposition, then it can either be true or false.
So for example, here, the logical operator red light is true at the first state.
If this was green, then the logical operator red would be false at the first state.
Does that make sense?
All right.
One important thing to note about the logical o operators is that they just apply to the very first state in your statement.
So when we're discussing logical operators, we haven't gotten to the point where we can express temporal information about our sequence of states.
The next operator is, not.
So as you can see here, the traffic light is read at the first state.
So the traffic light's not green.
Hold, because the proposition green is not true at the first state.
And the logical operator, and, at this first state is we're at the red light, and we're getting gas.
And so both that we're at the red light and that we're getting gas holds for the first state.
And then the operator, or, essentially says-- I guess a lot of this is probably review for you guys-- logical operators-- but essentially if you have red or green then you could either have the traffic light be red or the traffic light be green at the first state.
And as long as one of those two hold, or both of them hold, then it's true.
And one thing to note about or lies in if and only if these can be re-written with just the logical operators, and, and, not.
So another way of expressing red and green is to say that you can't have the scenario where both is not red, and it's not green.
And I'm not going to go over that for, implies, and, if and only if, just for the sake of time.
So that's just a review of the logical operators, and now we're going to go into the temporal operators.
So when we're talking about our autonomous car, what are some useful temporal operators?
Because up until now, we just are describing of the first state in our sequence, which isn't really useful if we want to talk about the entire set of sequences [INAUDIBLE] states.
So what do you guys think are some mutual things that we want to be able to describe?
I know Ben touched on some of them at the beginning, but if you guys could recount some of them.
Yes
State connection
Mm-hmm.
State connections.
So exactly.
So if I'm in a state right now, might there be concerns about the next state?
Is that kind of like what you're saying?
Exactly.
Yeah.
So next is one of the things that we might be concerned about, the next state.
Are there any other tings you guys can think of?
Always or [INAUDIBLE]
Yes, exactly.
Always.
So like Ben said, we always want to follow the speed limit.
We always want to be under 35 miles an hour, or whatever the speed limit is.
Anything else?
Until
Until.
Yep.
So like he said, you want to be stopped until the light turns green.
Yes
Eventually
Eventually.
Yep.
So we eventually want to get gas.
And eventually.
Any other things?
Yes
At this state
At this state.
Yep.
And so, at this state, is kind of what the logical operators were doing at the beginning, but I agree that's something we have to be able to express.
So that's not a specific temporal operator, but it's included in the logical one, so I'll just write it up here for you
Can I ask a question?
Uh huh
So the logical operators, are they acting on the-- so they're acting for a state?
They're not acting on the entire chain
They act for a state.
And so if you just wrote the logical operators, just by the definition of LTL, it means let's just analyze the first state and see if it's true.
And so it won't tell us anything about the third state or the fourth state or the second state, which is why we have to look at the next state.
So yeah.
Any other questions?
OK.
So like you said, next is an important one.
So right here we consider OK, the next light will be green.
And until is the one that she mentioned, so the light will be red until it becomes green eventually.
So the light will eventually at some point in the future turn green.
So you might be waiting at the red light for a long time, but eventually it's going to turn green for us.
The light will always be red, so I guess you're stuck at the traffic light for a long time, or globally-- that's another name for always.
And lastly, this on is a little bit less intuitive.
So what this is saying is the light will always be red until another circumstance coincides with the same state.
So the light will be always red until the car gets gas.
And then after it gets its gas, then the red is released, and the light can be either green or red after that.
So you have red red red red red until we have both red and gas.
And then it can be green or it can be red [INAUDIBLE] it's not constrained to being red.
Does that make sense?
That one's a little bit less intuitive, so-- is that good?
OK. All right.
So now that we've gone over just the intuitive understanding of that, I'm going to give you the actual syntax for that.
So x is representative of next.
So in the first state, we don't care if p is true.
p could be not true, p could be true.
It doesn't matter.
But in the second state, the next one up in the first state, p, has to hold in order for xp to be a true statement about our sequence of states.
So what do you guys think if I wrote this?
What would that mean?
Different state [INAUDIBLE]
Yep.
So this one over here?
Sure.
[INAUDIBLE]
Yeah.
Perfect.
All right.
The until operator.
The until operator essentially says, like Ben said, we're stopped until the light is green.
So if you have p until w, you just p has to be true at every state until w, and after that point, p can be whatever it wants to be.
p eventually, or future operator, says that at some point in the future we will get gas.
However it's important to note-- I know when we talked about getting gas we said, oh, we have to always get gas at some point in the future, but the future eventually operator is just at one point in the future.
So once you get gas, you don't have to get it again.
Yes
On the previous slide you had a not at the bottom that said that w is required to become true.
Does that mean an until operator also implies a future operator on w?
Yeah
All right.
[INAUDIBLE]
Yep.
But it's the added condition that p will be true until the w
But like whenever you use until, you say p until w that you'd also have to add at some point in the future [INAUDIBLE]
Yeah.
Yep, you're right
Well, I mean, it would be possible that this stays p, right?
It could.
Yeah.
But then if your system-- that would be totally fine if you had your system just be p, but then the operator p until w wouldn't hold unless you had a w at some point in that sequence of p's.
Does that make sense
Yeah.
[INAUDIBLE] I guess it does if that's the way to do it, but it doesn't make sense [INAUDIBLE] it
So you're saying that we could have a bunch of p's, right?
Is that what you're saying?
And I'm saying that this is a fine state to have.
You can have whatever you want.
And we're just analyzing if a sentence is true about it.
So this statement, it's not true-- or, sorry.
I'm saying that it's not true that we have p and lw because there's no w in our state.
But if we add in a w somewhere and we keep all the p's, then it's true because we have p up until we get to a w. Did that make sense?
It made sense that that's the rule, but it doesn't make sense to me why.
Because the statement [INAUDIBLE] true and if w never appears.
You have p until w, which never happened
Yeah.
If w never happened, then it wouldn't be true
I mean, technically, for the English language it would be true
I guess it's not--
If you have p out to infinity, and you never have w's.
p until w is true
p until anything is true [INAUDIBLE]
Like if you say that I'm hungry until I eat, and you never eat, then you will always be hungry
Oh, I see.
So that makes sense in a logical sense, but that's not how it's described in
There's also an operator that's sort of unofficial called the weak until, and that's sort of saying p until w, or always p. So that kind of gets rid of the condition that it reaches w at some point
It aligns more with the English, too
Yeah
Yeah
But just the until operator on its own is just sort of like a strong until
Yeah
So it adds the--
Yeah, that's fine

This is the way it works
Yeah.
Sometimes-- yeah, I get confused about this sometimes.
I'll be like, why they necessarily [?
chose any of these, ?]
but, OK. And then the globally one, which we said-- this is the one that you were just describing here [INAUDIBLE] when you were talking about the p at every single state in the future.
And then the release operator is you have to have p until you have w, but p and w also have to share the same state when they switch from-- so in until, you don't have to have the p here, but with release, you have to have the p and w occur in the same state.
Does that make sense?
Yes?
OK.
So just like before, operators can be described using not and and.
The future, released, and the globally operators can also be described using other temporal operators.
And so just as an exercise, can any of you guys tell me which-- these are not in order-- which of these line up with the other-- like, whichever ones they line up with.
And I'll give you guys like three minutes to just think about it, and then we'll work through it together on the board.
All right, so have any of you guys thought about which ones might correlate to which ones?
Does anyone have any suggestions?
Yes
The first one's the top one
This one is the top one.
That's exactly right.
So let's continue this state again.
So the statement, true, doesn't say anything about what the propositions are.
It's just something that's true for all states, right?
So if we're saying, true until pi, that means that you can have whatever you want until p occurs.
So does that make sense that, at some point in the future, p has to occur, exactly because we had the strong until, right.
Does that make sense to everyone?
You guys want to see some [INAUDIBLE]??
All right and now what about this one?
We have not as not p. So this is saying that at some point, if you can't get a p set at some point in the future, you don't have p. So what do you think that would correspond to?
If at some point in the future, you can't have the situation where there's not p
The third one
The third one?
[INAUDIBLE] So that would be if you have p at every single state, then you would never have a situation where you don't have p, right?
And so that leaves this last one.
You don't have a scenario where you have not p until not w. So if you have not-- OK.
It's still a little hard.
But I'll come back to that one at the end, OK?
But it's just important to remember that you can describe these operators with similar to them so that your planner doesn't have to deal with the global operator or these operators-- eventually operator, it can have less [INAUDIBLE] to deal with.
So yeah, you guys were right.
Perfect.
And then this is just a recap of the different operators and the other ways that you can express them.
And so there's a few more really cool things that you can do combining these different operators, and the first is that-- remember when Ben was describing that we need to get gas?
And we need to get it in the future, but even after we get gas, we're going to have to get it again some time in the future.
So it's important that we can describe something that happens infinitely often.
And the way that we describe that is saying that we always have the scenario where you will eventually reach p. So I'm trying to think of another way to describe that, but does that make sense that right now in the future, I'm going to get gas.
And even after I get that gas, I will eventually in the future also need to get gas again, because your car will always be needing to get gas.
Does that make sense?
Cool.
And then eventually forever is another scenario that's important to describe.
So let's say that the traffic light will eventually turn green forever.
We need to be able to say that at some point in the future, although we don't know when, the traffic light will turn green.
So the future operator says that at some point in the future, you will have something come true.
And so at this state, p becomes true, but the global operators that happened from this state forever afterwards, which is saying that you eventually will always have this state B be true.
Do you guys have any questions on that?
Did I explain that good enough
It seems weird to me that the gas tank is becoming this occasional temporal operator instead of just like the state that we actually have gas, would just be the logical thing that [INAUDIBLE]
Oh, yeah, I--
Why is the act of getting gas showing up here like that, versus just, you always need quantity of gas
That's true.
We do always need some quantity of gas, and you would describe that, like you said, just with the always operator, but I guess my action of getting gas is I was thinking of actually driving up to the gas station and filling it up with gas.
And if you didn't do that, you wouldn't be able to always have gas.
Does that make sense?
Right, so I would think that always having gas would make [INAUDIBLE] of you plan [INAUDIBLE]
Yeah, and depending on how you make your model, you definitely could do that, and that definitely would satisfy the condition.
[INTERPOSING VOICES]
--the usual way to deal with it?
Well, it depends on the situation.
Maybe you [INAUDIBLE] want to say at some point in the future, the light will always be green, because you want to make sure that cars can go through, and there's a difference between red, and some point it will be green in the future.
So it really depends on your model and how-- I think the way you're saying it, where you always want to have gas, and the proposition gas is how much gas you have in your tank, and if you have it or don't, then I think that that would be a really good way to model it, too
Because this way, if you said, I have to get gas.
I planned it for next week, but I've run out of gas this week.
Do you know what I mean, like, you have to somehow be tracking the quantity of gas also
I agree.
I agree.
I was just using it as an example to explain the concept, but I agree that that definitely probably could be better with more work
Well, I think in both cases what it could represent is having gas all the time, or explicitly saying that you're going to be in gas stations eventually-- always eventually.
It depends on how-- maybe we'd want to always have gas from a station, and also getting gas from it.
So depending on how you want to model it, but I think both will work
So infinite and often, doesn't really care about how frequently--
Yeah, it doesn't care how frequently
But [INAUDIBLE] for instance something that happens every 60 years.
My system lifetime server is 30 years, which means something happens only once.
So in this case, infinite and often, how can you filter a switch operator could be the same, right?
Can you repeat that again [INAUDIBLE]
So for instance something happens every 60 years.
My system lifetime goes to 30 years.
After 30 years, I have to get a re-up on the system.
Then so during this [INAUDIBLE] only one thing happened.
And so it happens off into infinity, but in the lifetime of the system, it only happened once, and therefore in this condition, the infinite and often should be set the same as the future, right, in the current state.
Something happened in the future, I would say something could never happen again
Kind of.
So when you're talking about a system that's dying after 30 years, the LTL doesn't actual model something that ends.
It would model something to infinity, although there is an extension to LTL that can incorporate that into it, having it have an actual end goal state.
And I think in that case, then infinitely often would be different, and would-- I don't know exactly how that would find that situation.
Do you know, Ben?
I know you had kind of--
So there's something called metric temporal logic, which associates a time scale of each of these operators.
So I think you'd have to use that if you were trying to express that you had to always meet those conditions still within your lifetime.
You could have some sort of qualifier to these operators to specify an actual time [INAUDIBLE] where that operator [INAUDIBLE]
So which would you file for the instance infinitely often [INAUDIBLE] scenario something at a little bit more frequent of a general system by the time, right.
Now we have something you know [INAUDIBLE] 100 years ago for years
You know, it's important that the LTL we're explaining here is only dealing with infinite states.
So I agree, it's a little bit-- you have to think about that process of how do apply something that goes on forever to a real system.
It obviously doesn't model correctly
There has to be a way to-- I mean the future thing, if he has a system that lasts 30 years, and you say future p, somewhere in your design of the system is going to say that the constraint on future p is that it has to happen within 30 years, right?
Because future p here, for getting gas, has to happen before I run out of gas.
It's not just-- this representation says it just happens in the future, but somewhere in your constraint checking, it's going to say, hey, we didn't get gas in this plan soon enough, right?
Yeah
Somehow
Yes, that's definitely true.
And these are just tools to help you model it, but it's definitely not the entire model of your system obviously, and so I agree that we definitely would have to incorporate that in.
And for finance systems you have to account for that.
And that's why they have the extension of LTL which is the finance systems, which we can certainly drop a link of of the papers, and where we got the information, to the class if you guys would like that
Yeah, I think you guys have done a very good job of answering the questions.
The three extensions to LTL that I've found to be the most relevant kinds of things that we're doing is metric temporal logic, which is the first one that you mentioned, also to be able to have it over bounded time, which is related, and you mentioned.
And then the last one is to deal with uncertainty, and is then a problem of linear temporal logic
OK.
Cool.
All right.
So I meant to have this as two separate slides, but what are some true statements about LTL that you guys can tell me about to look at this, or do, and explain why if you look at it.
OK, so why is the next red true?
The next step is slow down
Yep, exactly.
Because the next one is red.
Why is it true that in the future, that it's green?
[INAUDIBLE] Because in the future, it's green.
And why is it true that there's red until green?
It keeps being red until it's green
Exactly.
So it makes sense that if all of these are true, that this culmination of the and statements between all of them is also true, right?
And so you could do that with or as well, or you could do that with-- if you had sequences behind here, you could put a future around this whole thing, and it would be true that at some point in the future, all of this would hold.
Does that make sense?
Yes?
Why keep it red there?
Your red is the next step
Yep.
Exactly.
I was just saying if you had more in the past, and you were considering it from the previous-- All right so now Nadia's going to talk about expressing LTL in PDDL3.
So Ben and Ellie have guided through how expressive LTL formulation is.
I'm going to formulate the LTL in a classical planner like PDDL3.
Now I'm using PDDL3, which is an composed extension of PDDL2.2, which supports some of the LTL operators.
So these are the basic operators that you can use to express the constraints and goals of your plan.
So you have at end, always, sometimes, within, at-most-once, sometime-after, sometime-before, always-within, and hold-during.
So again, now here is a numeric controls that you can specify, and the ellipses represents an already-existing [INAUDIBLE] goal.
This is a goal description-- [INAUDIBLE] So some of the operators may look familiar to you, because they can also be used to express the constraint in STN, the Simple Temporal Network, that we learned in class.
And there are some operators that are unique to LTL or other kinds, like metric temporal network, like always, for example.
So now we are going to take a look at how you can express the operators using the PDDL3 language.
So we can use, within one occurrence to the next, because there is no explicit next in the PDDL3.
And you use always until to express until.
And in the future, you use sometimes after, and globally, you can use always.
And for release, since it can always be omega or always until omega when you see p. You can use all costs, too, although this [INAUDIBLE].. OK, let's see some examples.
So if your goal is to have the traffic light turn red in the next state, you would want to formulate it using within one occurrence, the traffic light would turn red.
And let's take a look at a more complicated example.
So if you want to model a goal saying that the traffic light would be green until it turns red, at which point it would be red forever.
So this is temporal logic of predicates that express this goal.
And then if you want to model that using PDDL, there is a direct mapping.
You can see the direct mapping between the predicates and the PDDL.
So this is basically saying, it would always be green until it turns red.
And turning red implies it will always turn red.
So next, [?
Arleese ?]
will tell us how to map between the LTL to PDDL with a Buchi Automata which is a specialized automata
Cool.
So I'm [?
Arleese.
?]
I have bridged the gap between LTL and the planning world.
So this is kind of the framework for how you would go from taking a problem with temporally extended goals all the way to a plan.
So as you can kind of imagine from what Ben and Ellie were talking about, LTL is pretty expressive.
You can express a lot of different types of goals that include more temporal properties than just a time window for a goal to be completed.
So once we have a kind of defined problem with these temporary extended goals, then you want to model it in a language like LTL or PDDL.
So if you have a language like PDDL, and you have a planner that can do an algorithm called progression, which I'll talk about a little later, you can go directly to a plan.
Basically how progression works is it's kind of an algorithm that tells you when you're analyzing a state, and you're analyzing LTL-like formulas that are true in a specific state, how to push formulas that you need to evaluate later on to the next state, and how to keep track of things that you need to continue to evaluate for satisfaction in future states.
Another way you can do this is by taking LTL formulations and translating them into Buchi Automata.
So Buchi Automata are basically finite state machines that are extended to handle an infinite sequence of states.
And I'll get into more about how Buchi Automata work.
After you have a Buchi Automata, you can then translate that into PDDL2, which you guys are all familiar with, and then that PDDL2 can be used from the classical planner, and get a plan.
So a little bit more about Buchi Automata.
Again, like I said, it's an extension of a finite state machine.
Oh, yes
Why would have a Buchi Automata when you could just use a planner that will go from PDDL3 straight to the plan?
I guess it just depends on how-- what kind of planners we have access to, what other systems you have.
It's done both ways.
So PDDL3 is still kind of new, and so there's not a lot of planners that actually have progression and can handle PDDL3
And it's also the case that the particular algorithms that are trying to either be verification or planning, maybe exploiting particular properties of the Buchi Automaton, as opposed to the properties of the native language
So you guys are all familiar with regular state machines.
Buchi Automata has a very similar formulation.
Some slight differences-- you have a set of states, you have an initial state, and you have a transition relation, and then you have a set of accepting states.
These accepting states are essentially what replaces finite states in the state machine.
We also have a set of symbols that are again simply the transitions between states.
So what an accepting state is, is a Buchi Automata can only be valid if the sequence of transitions visits an accepting state an infinite amount of times.
So I'll get into a couple examples.
So let's say you want to model the ticking-tocking of a clock.
This is just a regular finite state machine.
As you can see, after some number of ticks and tocks, you get into S2, and you're in S2 because you can be in transition only [INAUDIBLE] So you're in S2 basically forever once you get to S2.
So we can model this as a Buchi Automata by making S2 an accepted state.
This example, making it a Buchi Automata, doesn't really do anything for you.
It's just kind of to illustrate what the accepting state does.
So the accepting words are a sequence of transitions.
In this case it would be an infinite combination of ticks and tocks that would make this Buchi Automata valid.
Does that make sense.
Questions about that?
Here's another example, for example, if I have changed what my accepting state was.
So if our accepting state was S1, I now have to visit S1 an infinite amount of times for this Buchi Automata to be valid.
That means the only valid sequence of transitions I could have is tock tick tick tick tick for an infinite amount of times.
So you can kind of represent different things and different sequences of inputs you might want to have based on which accepting state you use.
This is another example.
If you wanted your clock to be tick tock tick tock tick tock, you can have more than one accepting state, and now I have to visit S0 and S1 an infinite amount of times if I want this Buchi Automata to be valid.
So then the sequence of transitions I get is tick tock tick tock tick tock.
Does anyone have any questions to really think about in general with Buchi Automata?
OK.
It might help to see how we model LTL as Buchi Automata.
So here's an example of of these LTL formulas modeled as a Buchi Automata.
I guess take a few minutes and see if you can get which one this one might be
[INAUDIBLE]
Yeah.
Just so that everyone can kind of see that, and future events, we remember that means that eventually p has to be true at least once.
So from any sort of input state, you go to S0, and then I do have amount p for some amount of time, but as soon as I have p, I have to get to p, because I have to be in my subject state an infinite amount of times.
So I have to execute p at some point to get to my accepting state for this Buchi Automata to be valid.
And then once I'm at S1, I can have any other amount of inputs.
I'll always be in S1, and that's what the true label means
And it's also something stronger, right, which is you could-- it's eventually globally?
Right.
Yeah, so actually in this case, after it gets here, my input could be anything.
So this basically just says that for my initial state, I have to execute p at least once to get to this accepting state, and once I'm there, I can be there-- I'm always there.
Any input will believe me in S1
So just to make this clear for myself and maybe for others, the data string executed that's at R state something that's on the loop.
If you go from s0 to s0, that's essentially saying that one of the nodes is not p. If you go from s0 to s1, it's saying that one of the nodes is p
Yeah
If you go from s1 to s1 and take whatever it would be because you set it true
So if you put p where the true is right now, then you you won't have to go through
Right.
That's what I missed.
By the way, you guys have 25 more minutes
So that's what we talked about.
This is future.
So the accepted sequence of propositions would be not to not p. Then you have p. Then it can be anything after you get to p. And then in this case, for Buchi Automata states, which are like [INAUDIBLE],, slightly different than the LTL states we've been talking about.
In this case, you're in s0.
And then once you get to s1, you're forever in s1.
So globally, looks pretty similar to this.
Take a few minutes to think about how I might change this Buchi Automata to [INAUDIBLE] globally after [INAUDIBLE]
[INAUDIBLE]
Yeah, exactly.
So you have initially, your first state by your n because p has to be true forever after you get to this first state.
You have an accepting state in p. You can also model not p. If you want to, you can also just have this single see which would encapsulate exactly what Buchi is.
This is how you go from LTL to Buchi Automata.
In a real-world scenario, we have multiple LTL formulas all combined.
You're Buchi Automata are going to look a little bit more complicated than this.
Try multiple accepting states.
There's a full weighted model of what an LTL is trying to represent
If we have the not p transition out of there, Like it's possible you can come into s0 to accept the Buchi Automata because it would go to s1, right?
Accepting a Buchi Automata after that, it's a little confusing.
A Buchi Automata is only valid if the infinite sequence of states satisfies it.
If I were to go the p instead of not p, my sequence would accept a transition of p, p, not p. Then that would not be a valid sequence of transition for this Buchi Automata because I'm transitioning out of my accepting state, which I need to have been in for at least the amount of times.
And there's no way I can get back to success.
Also, a more defined algorithm [INAUDIBLE] an intuitive way to build a Buchi Automata, or is an algorithm.
The other one that's most popularly used was developed by [INAUDIBLE].
This is a pseudocode version of the algorithm.
We go from LTL to Buchi.
As you've noticed, it actually uses progression, which is something that planners that take PDDL use to generate plans.
An important thing to note, which I won't go into too much detail, this algorithm generates a generalized Buchi Automata, which you then change to a simple Buchi Automata.
The difference is, the generalized Buchi Automata has a set of sets of accepting states.
For a generalized Buchi Automata to be accepted, you need to visit an accepting state in each one of those sets of accepting states.
At least one of those states has to be listed.
For a simple Buchi Automata, you only have one set of accepting states.
And you can visit any one of those for the Buchi Automata to be valid.
On the translation between those two is a little bit more complicated.
We'll have paper in the preference that will walk through how to do that.
But we're not going to do that now.
This is a progression algorithm.
Basically, it just tells you how if have a LTL formula f and some current state N-- and usually they involve some time step because we're in the real world and we can't really model infinite amount of time, and we have to make it discrete.
So we have some time step, which means successive state.
How do you push certain LTL formulas onto next states to be evaluated later?
For example, I'll take this next f as an example.
So if f was a next f of some LTL formula, you need to append that formula to the next state to be evaluated in the next state to see if that's going to be true or not.
Similar things for LTL.
And I won't go through this [INAUDIBLE] it's here for reference.
The next step to this process is going from Buchi Automata, that's a PDDL2.
This process is confusing.
At least it is confusing for me to understand.
The first thing I'll say is that Buchi states are not equivalent to traditional PDDL states.
In a PDDL state, if you have two states, you have the same propositions that are true and false, those states are identical.
In the Buchi Automata, that's not necessarily true.
In the Buchi Automata you also have to encapsulate which transitions you can make out of each Buchi state.
So s1 and s2 couldn't have the same set of propositions that are [INAUDIBLE] hold.
But out of s1-- actually, back up.
This is the Buchi Automata for FutureGlobally, which is what [INAUDIBLE] was talking about.
So if I enter an f1, the star means I can have any transition that I want in s1.
At some point, I make the transition p and whatever the s2, I have to take transition p forever.
However, if I get a sequence of inputs, let's say I get p and then b, and p again, and then p again, it's not clear if I'm in s1 or s2 because in s1 I can execute any kind of state I want.
In s1 I can execute any transition I want in s1.
2 I have to execute only p. So if I got a sequence of p's, I could be in either state.
You need something else to activate PDDL2 to basically to determine which state you're and which transitions you can make from that state.
Does that make sense to everyone?
So there's two ways we can transform PDDL2 to encapsulate what a Buchi Automata encapsulates.
The first thing is you can create new actions that can encapsulate the allowable transitions of each state.
So basically, for every action you have in your traditional PDDL, you have create a repetitive action for each one of these states so you know exactly which transitions you can take out of that state.
This can get long because you have to make a new action for every single one of these states for every action that you want to take.
That is one way you can do it.
Another way you can do it is introducing derived predicates.
Derived predicates are predicates that don't depend the actions at all and just depend on some other formula in your plan.
For example, we can have a derived predicate that explicitly tells you whether you're in s1 or whether you're in s2.
And you can use these predicates in your actions in PDDL2 to make sure, even though which state, x1 or x2, you're in in your Buchi Automata.
And then, in your formulation of your plan, you set your final state to be one of the accepting states.
I guess that answers one of the questions, though, how you go from being infinite to finite.
The accepting state is usually what you place as your final state
Then in the derived predicates, you're purely in the pre-conditions?
Or do they also appear in the effects
They also appear in the effects because you need to know how to transition out of s1 to s2.
So you'd likely have a derived predicate for both s1 and a derived predicate for s2.
And that is also a lot of work to add to the PDDL2 problem.
But it's definitely shorter than the first way.
And it definitely encapsulates everything, by the Buchi Automata [INAUDIBLE] Does anyone have any questions about that?
So you're changing the action then?
Yeah.
You're changing the action but you don't need make a new action for every state.
In the first formulation, let's say I have some traditional action, like move blue block, or something.
I have to create a move block action for each one of these states because I need to know exactly which transitions I can take out of that action
In the second example?
And in the second one I can just have one move block.
But in my move block I'll have a derived predicate that tells me which state I'm in when I execute that action.
The number of actions you have to write is less because you explicitly have predicates that are telling you which state that you're in.
Now I'm going to hand it over to-- if there are no more questions-- hand it over to Mark.
And he's going to talk about preferences
We're getting a little low on time.
Now I'll transition to talking about-- we talked about temporally extended goals.
We said that goals are things that always have to be true.
So now let's talk about preferences.
Preferences are things that don't necessarily have to be true, but are things that you'd like to be true.
In the classical planning problem, we can formulate it like that.
We can set a state S, set a state s0, a set of operators, and a set of goal states.
The problem is transition from the initial states and any state.
But those are goal states using the operators [INAUDIBLE] Preference based planning problem introduces a traditional field R, which is a partial or total relation expressing preferences between plans.
And this is a little preference symbol right there.
That tells you, if there are multiple plans you need to accomplish your goal, which one do you want to use.
And again, preferences are properties that are desired but not necessarily required.
There are two [INAUDIBLE] different types of preference languages, quantitative and qualitative.
As you would expect, in quantitative languages, we actually assign a numeric value to the different plans in order to compare them.
So these plans are a weight of four, and a weight of two, and that tells you which one to prefer.
Here are some languages that do that.
For qualitative languages, they actually just have a plan with this property is preferred to this other plan with this property.
But we don't actually know any information that knows how much we prefer one to the other, or something like that.
An important element of this is when you have quantitative languages, they're totally comparable.
Any two sets of plans you can determine which one is preferred.
Whereas in qualitative languages, you could have situations where plan one is preferred to plan two, and plan one is also preferred to plan 3, but that doesn't give you any information about whether plan two is preferred to plan three.
So it's a little bit less expressive [INAUDIBLE].. To go into how you actually express preferences in PDDL3, like we talked about temporally extended goals, there is a PDDL3 syntax to do this.
There's a preference label here that you can put on fluents to represent things you prefer.
And then there's a function call is-violated that essentially returns the number of times that any fluent that has a preference label was not satisfied in your plan.
For example, if you have a preference, "Traffic light is green until it turns red."
Here's our PDDL3 template.
It extended this kid's preference, in which we label it with a preference label here.
And this is just a name.
And then or plan tries to minimize the number of times that this preference is violated.
That's one way to express preferences directly in PDDL3.
So now we talk about LDP.
LDP is a different language that is quite expressive in terms of types of preferences you can represent.
It is a quantitative language, which means we're going to have weights for each of our plans to express our preferences.
We can actually express the strength of the preference.
So Goal A is preferred twice or three times as much as Goal B.
It's an extension of an older language named PP.
Here is the paper if you want to actually check out these details.
The formulas are constructed hierarchically.
I'll go through quickly how we actually construct these preference formulas.
The lowest level is called a Basic Design Formula, or BDF.
That just expresses our temporally extended proposition.
So this is just a straight up LTL, basically, that we saw in the first section of the presentation.
I'm going to use that I'm cooking dinner example for these slides.
In this case, we always use the future operators.
At some point, I might want to cook this program plan.
Maybe I want to order takeout to eat dinner.
And then I have some eating spaghetti or eating pizza.
Those are two different things, two different options that I could have in my plan.
I don't have to have either of those.
Those are just options that I could have.
That's the lowest level.
The next level is called Atomic Preference Formulas, or APFs.
There are where we express our preferences between the BDFs that we formed in the previous slide.
So in this example I'm using weight, specifically to represent preference, with lower weight being preferred.
Here we're saying, we prefer to cook over ordering takeout.
This is you where you have to cook, this is where you have to order takeout.
And we apply weights to those two expressions.
The first one's preferred, and how much it is preferred over this plan.
And here is you prefer to eat spaghetti over eating pizza.
So that's the second level.
Third levels called General Purpose Formulas, or GPS.
These are where we can do conjunctions or disjunctions, or qualification of our previous formulas.
So for example, if we want to say-- because in the previous slides we had two APFs.
We had prefer to cook and prefer to eat spaghetti.
And here we can express that we really don't care which one of these we want to satisfy.
You can think of this as we actually tried to satisfy APF at the lowest weight.
So if we have this "or" operating here, that means that our planner is going to try and satisfy the lowest weight among all these different options here, which means that he doesn't prefer to cook.
That's its first goal.
You can also use a handoff rigor, for example.
And that's going to minimize the maximum weight against both of those options.
Basically, what's it going to try and do-- oops-- what it's going to try and do is minimize the highest weight among these two options.
So it's going to try and cook, and then it's going to try and eat spaghetti versus doing these other options.
So those are GPFs.
I'm now moving on to Aggregated Preference Formulas, which are the highest level.
So with APFs, these define the order in which our different preferences are relaxed.
You can, of course, express a lot of different preferences for your planner, but not all of them may be achievable, especially if you're trying to achieve a large number of preferences at the same.
You may not actually be able to achieve all of those.
So how do we produce our set in the correct order, in the preferred order so that our plan we end up with isn't even the most preferred plan?
That's what APFs like to do.
You can express, using this preference operator.
First, I want to try and satisfy both of these in the previous slide.
Then if I can't, I want to relax it so I only satisfy G2.
And then if I can't do that, then I want to relax it so I only try and satisfy G1.
So that gives us the order in which things are relaxed.
It's important here that if you have these situations where you can't distinguish from one another, or you don't really care, we can establish some order.
So at the very lowest type of level, we can sort them alphabetically, or something, just to provide some sort of order.
This is just a review of what I've been talking about.
BDFs are the lowest level, which express temporally extended propositions.
We can apply preference to those using APFs.
Then we can combine or join APFs using tell preference formulas.
And finally, we can aggregate those preference formulas to determine the order in which you shouldn't relax the propositions to allow yourself to plan it right.
So using this scheme in LPP, we're using both LTL syntax and rules that we talked about in the first section to express temporally extended preferences.
The plans that's actually used to solve these types of problems is called P Plan.
P Plans can actually handle templates, and the preferences, and a goal at the end, as well.
And it does basically a best first search among all your different options.
It's actually one of the planner that uses the left branch.
But the one we showed uses progression to take LTL formulas at each point in the plan and determine whether you've satisfied those formulas or not.
And if not, it will push them to the next state, so that in the next state you can evaluate whether you've satisfied those formulas.
That's how it prunes the search space and tries to always find the most preferred plan to meet your goal.
All right.
So that's our presentation.
Any questions from anyone?
[INAUDIBLE] question.
I have a question about [INAUDIBLE] presentation.
You guys [INAUDIBLE] and then believes omega.
Would you say that omega has to hold if p happens, and they have a conjunction in one state?
Omega doesn't have to hold until we [INAUDIBLE].. Omega just has to hold at the last state of p. Essentially what it's saying is omega releases p. So once omega happened, p has to happen at the state.
And the omega has to release p. Now it happens, so p, you can don't you want.
You can be true, you can be false
Sorry.
So p has to hold until omega happens
Yep.
And then omega-- [INTERPOSING VOICES]
So the definition is just switched to p [INAUDIBLE]
This is an occasion.
Occasionally, you have pR omega.
But like intuitively--
It's just the definition-- oh-- [INTERPOSING VOICES]
That's all I was wondering because then I saw that and it was confusing.
So that's all.
OK, sorry
That's fine
Any other questions?
Since there are multiple ways to express the same formula using the grammar, is there any notion of canonical forms, or a least complicated form, or something like that?
Yeah, there is.
Typically they actually-- let me find the slide.
So to simplify the algorithms they typically apply the kind of reductions that we showed in the slide to reduce the release and the globally and the future down to just the [INAUDIBLE] of an x.
Otherwise, your algorithms going to be able to handle fewer cases.
And then they apply the usual rules, trying to push nots out to the left-hand side.
All right.
Thanks, everyone.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.MIT.edu.
PROFESSOR 1: So as he gives an introduction, myself along with Catherine, David, [INAUDIBLE],, and Charlotte [?
are going to present ?]
you guys Probablistic and Infinite Horizon Planning.
To give you a brief overview of what we're going to talk about, we're going to start with the Quadrotor motivating example.
We're going to move into planning with Markov decision processses, give you a little bit about value iteration before discussing heuristic guided solvers.
And we're going to go into the more stochastic case, partially observable Markov decision processes and operating in belief space.
So we very often now see quadrotor motion planning as a problem, given, for example, with the Amazon fulfillment center.
We start with a goal configuration, start configuration, set of actions we can take, and some type of reward function or cost function.
So for instance, if we have a quadrotor starting at the Amazon fulfillment center, and we want to get to 77 Mass Ave, and let's say we want to take the shortest path to get there.
We would follow the red dashed line.
But, as we can see, it comes very close to these obstacles.
So we're looking at very higher risk for mission failure, crashes.
If there's any uncertainty in its path, we're going to have a problem.
So one of the ways that we can compensate for that is we can also done a green path, which is adjusted to give us a little bit of space.
Are there any more questions?
So as you can see, the level of uncertainty allows us to determine how easy or difficult the problem's going to be to solve.
On the easier side we have deterministic dynamics and deterministic sensors.
In this case our actions are going to be executed as commanded, and our sensors are going to tell us exactly where we are.
This will be something like dead reckoning validated through sensing, very redundant.
If we moved to a little bit more at a difficult case, we have deterministic dynamics.
Our commands are still being executed, but maybe we have a noisy camera.
So we have some uncertainty in the sensors.
These are the cases where we would see dead reckoning, but we would compensate with Kalman filtering to get rid of that noise level.
Now, down at the-- yes?
Sorry, what is dead reckoning?
PROFESSOR 1: It's essentially-- you are saying I want to execute this option, and it's going to execute it exactly
OK.
PROFESSOR 1: Then towards the bottom, we have stochastic dynamics and deterministic sensors.
So in this case maybe there's some uncertainty in our actions.
But we can validate through sensing.
This is where we're going to spend the next section on Markov decision processes.
And then briefly, later in the lecture, we're going to talk about this most difficult case, which is the stochastic dynamics and stochastic sensors.
Our execution maybe, maybe not, our sensors maybe a little bit of noise.
And this is where we see partially observable Markov decision processes.
PROFESSOR 2: Can we make just a brief clarification of the dead reckoning?
So dead reckoning is where you estimate you position using a probabilistic file, but you don't use any observation stuffs.
That's the thing.
PROFESSOR 1: OK.
So we talked a little bit about action uncertainty, where we're going to focus.
And this is a case where, for instance, even if you tell a quadrotor to stay in its place and a gust of wind comes by, it's not going to stay in that same place, right?
It's going to have a little bit of movement.
And we can see, this causes things in disparity where you have a command of trajectory and your actual trajectory and then you need to associate.
So in order to compensate for that, we want to model things with that uncertainty or else we have these higher situations where we command to follow some line.
We don't incorporate the uncertainty, and we see a crash.
So this allows us to introduce Planning with Markov Decision Processes.
So MDPs have a set of states-- some actions you can take-- a transition model.
So essentially, what's the probability of reaching some state if you take an action?
An immediate reward function and a discount factor.
This discount factor is important because it allows us to prioritize gaining an immediate reward as opposed to an uncertain future award.
So the concept of, bird in the hand is worth two in the bush.
Now we want to find an optimum policy that will essentially back an action-- the best action-- for each state.
And what we hope to get from this is maximized expected lifetime reward.
So we want to maximize the accumulative reward we get over the times.
So let's walk through an example.
If we have a quadrotor with a perfect sensor, and let's put it in this environment [INAUDIBLE] 7x7 bridge.
Our set of states are obviously [INAUDIBLE] space in them.
Can anybody tell me what some of the actions might be?
[INAUDIBLE] PROFESSOR 1: (LAUGHING) Yep.
Up, down, left, right.
In this case, we call them North, South, East, West, or Null.
The next thing we need for this example-- arbitrarily gave ourselves a transition probability.
So we said that you have 2% chance of following your commanded action with a 25% chance of moving to the left or the right.
Next we have the reward function.
And again, we arbitrarily decided that we want it to be a reward.
If you get to the state 6-5, the increment [INAUDIBLE]
Alicia, you had [INAUDIBLE] left or the right, is that clockwise or counter clockwise?
Let's say you had planned to go to the right, would that mean you have a 75% [INAUDIBLE] or 25% chance of [INAUDIBLE]?
What if you just waited [INAUDIBLE]??
PROFESSOR 1: We said clockwise, counterclockwise from the intended direction of action
Thanks.
PROFESSOR 1: Uh-huh.
And finally, we give ourselves a discount factor of 0.9.
So let's assume for a second that we have our optimal policy.
And let's say that our optimal policy says, from this state, we want to take the action North.
Right?
As we discussed, we have a 50% chance of going North and 25% chance of going to the left or right.
So after that time step, these are possible states that we could end up in.
Right?
So now let's assume for a second that we can take our next action.
And our next action says, go North.
Again, we have the same probability distribution.
And these are the states we could end up in after two time steps.
We can see that this starts getting very complicated, right?
And there are increasing amounts of uncertainty.
So does anybody have any ideas on how we could collapse this distribution?
Keeping in mind that our senses, at this point, are deterministic.
Yep
Fly to a corner?
PROFESSOR 1: I'm sorry
Fly to a corner?
PROFESSOR 1: We could do that
You just sense how far away the red box is.
PROFESSOR 1: We could do that
Quick comments.
So the blue state is not actually one.
PROFESSOR 1: I'm sorry
So the problem is you're not actually one
Yeah.
The issue is if you went to (1,3) and then you transitioned to the left, that's where the 0.0625 is coming from
[INAUDIBLE] then, shouldn't those numbers always are actually 1-- your probability distribution always are actually 1?
PROFESSOR 1: Yep
So it's just a point.
It's just off the screen.
PROFESSOR 1: We would add another section to the screen and just move the grid over.
We just cut it off for graphics
OK.
PROFESSOR 1: Yeah.
So those are all great points.
The easiest way to do it is just take an observation.
So at this point we say, after our first time set, we weren't sure which of these three states we were in.
So we took an observation and said, wait, we're actually here with complete certainty.
So to make this a little bit clear, we're going to look at it from a tree view.
Right?
We said that we started at a state.
We took an action.
And this is our possible states we could have ended up in.
We're going to collapse this by taking this observation.
And now have a complete study here.
And we take our next action and see that we have moved out here.
So this allows us to basically ignore the history of states.
We have the same percentage probability from each time set.
This will be really useful in completely collapsing the distribution every single time you take an observation.
Anybody have any questions on that?
OK.
So let's go back now and figure out how he came up with his optimal policy.
They way we do that is through dynamic programming.
There's two different ways.
You can do it either through value iteration or policy iteration.
For this lecture, we're going to focus on value iteration.
So let's take this same example we had.
We still want to maximize the expected reward.
And so to start, we're going to initialize the values of each state to 0.
Let's for example, start at (2,0).
We're going to focus on suite 6-5.
And we're going to say that we're going to take the Null action to start with.
From there you can see with a probability distribution 4, 6, 5, we have a 50% of staying.
We have 25% chance of going to 5-5.
And a 25% chance of going to 7-5.
The next item of information is the values at each of those states that we could end up in.
And currently, they're all set to 0.
And finally, the most important part here will be with the reward [INAUDIBLE] 6-5, taking the null action, regardless of what state you end up, is going to be 1, as we defined in our initial set up.
So let's see how we would calculate the next time step value of the state.
You'd start by taking probabilities, right?
And from there, we would add the reward here.
And because we said that the reward does not dependent on the state we end up in, [INAUDIBLE] should be across all three probabilities.
From there, we're going to add in the discounted value that we had from before.
So a way to look at this in a more generalized form is to say that across all the states that you can end up in, you're going to look at the probability of ending up in that state.
You're going to multiply that by the sum of the reward and the discounted lifetime value that it has.
So we want to make sure that we're getting the best possible values.
So we need to incorporate all the other actions that we can take from that state.
So what's going to happen is we're going to take that general formula, we're going to repeat it over all of the possible actions.
And then we're going to take the maximum of that.
So, for this example, the state is very easy.
All of them are the same for this case.
But we go fast, and we say we get a value of 1.
And we update it by showing the graph.
This gives us what's called the value [INAUDIBLE] Backup-- or Update equation.
This will be really important because it reaches across the entire states space and allows us to provide a history.
So what this would end up looking like is we're going to iteratively calculate the values across the entire state space So at t0, we determine that all the values are 0.
At t1, (6,5) gets a value of 1, and at t2, we see that value start to propagate out.
Make sense so far?
So the way this works is you would repeat those iterations until your changes in value become what we would consider small enough, which would indicate your approximation is close enough to the real value.
From there, you would extract the optimal policy from the lifetime values.
So you see the [INAUDIBLE] in the Bellman equation.
And you're now just taking the action from it.
And you would map those actions to your states.
An example of what this might look like propagating out-- if you say blue was the reward and red is an obstacle, et cetera-- you can see, as that value propagates out, you start seeing your policy by the arrows.
Anybody have any questions?
So one last thing to mention about this, though, is the complexity for each iteration is dependent on the size of the state space squared and the number of actions you can take.
So you can imagine, as your state space expands or you gather more actions it's very complex, in which case time and value depiction becomes very time intensive and costly.
So this allows us to move into the Heuristic-Guided solvers
Thank you.
[INAUDIBLE] transition, one quick question.
Can I just get a show of hands-- how many of you have learned value iteration before versus how many of you have seen it for the first time?
So how many have seen it before?
OK. And then, how many is this their first time?
[INAUDIBLE] PROFESSOR 3: Any questions on value iteration before we jump in?
It's going to be an essential part of how we're going to do [INAUDIBLE] solvers.
Anyone who hasn't [INAUDIBLE].
So the most important thing we said about value iteration is that it's super slow.
It's going to have to go over every possible state and every possible action that it can take.
Our state space is multi-dimensional, and we can take a lot of different actions.
That going to be really costly and really hard to [INAUDIBLE]..
So the approach we're probably going to want to take is using some sort of best first search applet?
Who can tell me some example algorithms that already do that?
A star.
PROFESSOR 3: A star.
So that's very good.
And that's exactly what we're going to base our stuff off of.
The A star is for deterministic graph search.
If we have a graph, we can use it heuristic, and we walk down it and search the space that we're most interested in until [INAUDIBLE] variable.
So going to introduce to new items focusing on the last one.
AO star is an algorithm we can use to search for graphs that have this "and" problem.
AO stands for And Or graphs as opposed to the simple graphs that we have [INAUDIBLE].
The And is a way to express probabilistic coupling between edges.
So if we explore one thing, we might have to explore other functions.
We'll discuss that a little bit more in the next slide.
LAO is a bit more of a generalization.
What it does is allows us to search loopy graphs and deal with the probabilistic coupling.
And allows us to search and find the best path with a heuristic guided algorithm over infinite time horizon, possibly revisiting states using the tools we just had-- valued iterations-- to understand what the next best thing to do is.
So we'll talk about exactly how to get our MDPs that we just saw, these little arrow examples to these And/Or graphs.
We'll talk about two cases.
One simple one where we could apply the AO star algorithm is where you have a qualicopter.
And if you command an action North, you have a high probability of going North, but you might go East.
And vise versa.
If you go East, you might go North.
This is expressed here with the growing tree.
And these And edgers that connect it.
But despite the action of reading my command from 0 to 0.
We might end up in (1,0) or (0,1).
Right?
As we propagate outward, you can see how we're never going to loop back to a statement we've been to.
We're going to be moving in the Northeast direction constantly.
Our [INAUDIBLE] and our probability distribution across this tree explands.
And that's the coupling of the edges.
Because as we explore down the tree, we have to explore all the edges together.
Anyone have any questions on just this kind of conversion formulation?
I have a broader question.
We mentioned that MDPs deal with finite states.
Do we always just discretize a continuous row into a set of planning states?
PROFESSOR 3: Yes.
That's our prerequisite for searching over the state space.
We can do that as finely as possible, but yes, discretization is there.
So now let's look at this case.
Instead of the Northeast, let's say it's not deterministic whether we command an action north or South that we go north or south.
Here we see this loopy structure begin to emerge.
We see that we might-- on our first action commanding North, we might go to plus 1.
And then commanding North again, but we have our And edge and our probability distribution is [?
back ?]
to 0.
What this does is this loopy structure.
And this is exactly what we're going to be exploring.
This is a very real world scenario where it's a very likely we might return to somewhere we just came from, just because of the uncertain dynamics we have.
This is the type of problem [INAUDIBLE]..
So we're going to use the LAO to start out with.
We're going to talk about the three main things that [INAUDIBLE] has a heuristic guided envelope.
And what that means is that we have our large state space here.
But we're only going to look at a portion of it.
This greyed-out portion.
We're only going to look at the portion that is interesting to us-- the portion that provides us with the biggest rewards-- the portion that's reachable if we follow an optimal policy.
We figure this out with some admissible heuristic.
We'll estimate our rewards just like A star.
The idea here was that we'll keep the problem small so we don't have to search the valuation for a giant state space.
What we'll do next is at the state space expands, we get a bigger picture of the states that we're interested in.
We're going to do [?
an audio ?]
[INAUDIBLE].. And then we're going to figure out what the best action is.
And in the ideal case, the states that we would never reach using an optimal policy are never going to be explored.
Because our policy is going to say, no don't go over there.
That's a dead end.
Or that's getting you farther away from [INAUDIBLE]..
So we're going to be searching in a very specific part of the state space that is useful to explore-- that will get us closer and give us a higher reward.
What's important here, the L stands for Loops.
It's an extension of the AO star algorithm, which is by itself is an extension of the A star algorithm.
We can handle infinite horizon problems, like transition in different ways.
And really models the real world scenarios we're interesed in.
Any questions so far on the broad scope of what AO star is going to do?
Yeah
Can you put the [INAUDIBLE] what you're doing over here, but-- PROFESSOR 3: Sure.
The AO stands for And Or graphs.
So that's graphs that we have here edges that are coupled.
If you took time to do this transition, you might end up here or you might end up there.
So that's the notion that of this probability coupling, that we'll see in action as we [INAUDIBLE] we're going to do, we're going to input this MDP or AO graph with transition probabilities or reward function heuristics.
These are all things that we defined prior to figuring out a plan.
What we're going to come out with is an optimal policy for every regional state.
It's a little different than what value iteration is, which is an optimal policy from every possible state.
But we argue that that's all we need.
If we know where we're starting and we follow our optimal policy, we're only going to explore a certain portion of the state space.
And we're going to explore that together.
[INAUDIBLE] plan a little bit more efficiently than iterating for a high [INAUDIBLE] heuristic.
Any questions on that?
So we'll define some terms that we're going to use throughout this.
We've already talked about our state space.
This is just a small portion of it that we're going to work with as we're going to walk through this example.
Next we're going to define something called our envelope.
And that's the sub-portion of our states that we're going to be looking at.
We're going to initialize that to just this zero [INAUDIBLE]..
But as we progress through the algorithm, it's going to grow.
It's going to grow only in the areas that we're interested in.
A subset of this envelope is the terminal states.
Now these aren't goal states, these are just [INAUDIBLE] of our space that we've added to our envelopes.
And this is what it would be in the expanded [INAUDIBLE].. And it's just the nodes that haven't been expanded yet.
Here they haven't drawn anything incredible but you can imagine the state space goes out further because [INAUDIBLE].
You can imagine that this goes out further.
And they're keeping track of the nodes that we haven't expanded yet.
Or likewise, we've showed that we initialize [INAUDIBLE]..
The other few things that we're going to define-- we've already defined the states that are in our envelope.
That's the blue or the red.
We're going to define cost heuristic or reward function, R E, and a transition probability matrix, or set of matrices, that we're going to use our optimal policy search on.
What's important here is that our reward function and transition probabilities are slightly altered to account for the fact that we haven't explored the entire state space.
We see here that if a node is in ST, in other words, it's one of those terminal nodes.
We're going to say we can't transition out of it, because we don't know what's beyond it so far.
And we're going to set it for reward to be a heuristic.
Whenever we think that the reward is going to be when we begin to explore and go further.
Like I said, we're just going to feed this into a policy search just like we discussed with value iterations on the sub problem.
And we're going to search for an optimal policy [INAUDIBLE]..
So far so good?
This is the general steps.
This is very text-y, but we're going to definitely walk through every single step.
We're going to do two full iterations of the algorithm.
So like I said, we're going to create RE and TE.
That's are reward function transition probabilities using the definitions we showed.
We're going to use value iteration to find the optimal policy of the sub space that we're interested in now.
And then this is probably the most important step.
Knowing this optimal policy, we take a look at what new states that aren't in our terminal states-- nodes that we haven't explored yet-- what new states we might visit now.
So let's say we have our policy, and it says we'll go North.
And we know that we haven't explored the North state yet.
We know this is the state that we're going to reach following what we consider now to be already an optimal [INAUDIBLE].
So that's the states we're going to expand next.
We're going to do some bookkeeping, adding them to the terminal states [INAUDIBLE] once we expand it, then adding them to our envelope.
What's important here is that we're only going to add states that we visited yet to our envelope.
And this is basically the little hack that allows us to deal with loopy graphs.
We're not going to continually explore nodes that we might reach a second time, probabilistically.
We're going to let value iteration handle that [INAUDIBLE]
[INAUDIBLE] states are expanded.
I'm the one who got confused on that.
Are you saying that you're going to just repeat until there aren't any more terminals to look at.
And if that's the case, how is that possible if you have an infinite horizon [INAUDIBLE] PROFESSOR 3: Sure.
So if you can imagine-- and we'll talk about termination at the end.
But you can imagine that as we have these terminal states, but you have a policy that guides you to a part of the state space that we've already expanded.
Imagine you've reached your goal.
Your optimal policy is going to say, stay put.
Right?
And it's not going to say, move North again
It's just the goal [INAUDIBLE] PROFESSOR 3: Essentially, the goal state is definitely an example in the more extreme case that there's nothing else you can do that's going to get you closer to our goal.
The idea is that your policy on your sub space never tells you to go to a terminal.
Nobody can [INAUDIBLE] inherently worse than [INAUDIBLE]
Planning optimal policy means running value iteration entirely.
PROFESSOR 3: Yes.
We were going to treat it essentially as a black box.
But the trick here is that we're doing it on a smaller portion of space of a different world.
All right.
I'm going to put the steps up there.
Hopefully you can see this.
But for now, we'll just walk through from the beginning of what we're going to do.
So we said that our envelope and our terminal nodes are just at 0 to start.
So very simply, we use these definitions and say, OK, the transition probability as 0 to any node right now is 0.
Because it's in that terminal.
[INAUDIBLE] That's just so that we don't transition out of it.
We can develop the policy based on only this portion of the space [INAUDIBLE].
And our reward function, we're just going to apply it to be the heuristic [INAUDIBLE].. Let's say that's 20.
And for a move-on from there.
[INAUDIBLE] started to value iteration.
And we'll run this value iteration.
So this is a very basic case.
We're just going to-- we're using this to kind of build up the machinery, understand what we're doing.
It's a very basic case where you only node we have is that zero.
We can't transition out of it.
All we have is some heuristic.
So the only thing we can do is nothing.
So the action we're going to take from S0 is nothing.
Very simple case just to get us warmed up and understand what's happening.
So using this policy and knowing that those 0s in our terminal node are node reset.
We're going to take the intersection between this terminal mode set.
And the nodes that we might reach following our policy.
And we know that we're at S0.
And we know that action that our optimal policy says to take is not.
So we know we're there.
And we know it's in our terminal node set.
So that's the only thing that we could reach so far using our optimal policy.
So far so good?
Clear?
So that's exactly what we're going to expand.
Just expand S0, A, B, and C defined those to our terminal nodes.
So that's up there the symbols [INAUDIBLE] from our terminal nodes added as children.
Then we added the children to the end.
Here we go.
We're going to do a little bit more.
We're going to do the same thing again.
But now we obviously have more nodes and a bigger sub space to explore.
So using these definitions we see our reward function is a tuple of the node that we start from.
And so this S0.
And one of these three actions.
And we'll take A1, A2, and A3.
When we have our instantaneous reward functions, 6,4, and 8 as being our rewards from doing those actions.
From A, B, and C, no matter what action you take, it's just heuristic.
That's part of it.
And likewise, we're going to take a look at this transition probability [INAUDIBLE].
This is for [INAUDIBLE] transitioning from a state S0 for saying that if we take actions A1, A2, and A3, what's the probability of being done in nodes A, B, and C?
If you look so far here, we're going to look at something deterministic.
If we take an action, we'll end up where we say we're going to end up.
And we'll see how this algorithm collapses down to A star if everything's deterministic.
We're also obviously going to look at the probabilistic case where we say [INAUDIBLE] small probability that it might end up with [INAUDIBLE].
That's going to necessitate that we're going to have to look at and expand B together with A if we were to decide we want to try [INAUDIBLE].. And likewise, if we try to take action A3, we have to expand them all three nodes.
So the tighter the probabilistic coupling, the more of the space we're going to have to explore.
So just off this, assuming that if you take action A1, A2, and A3, can someone tell me what the policy is for the rewards here?
And we're interested in a policy from S0 what we're going to do.
Take a look at the rewards and judge what the best action to take is from a purely deterministic sense
[INAUDIBLE] do you add [INAUDIBLE] or do you just [INAUDIBLE]?
You have the reward that you have gotten so far.
PROFESSOR 3: Does that help you answer the question?
Oh, yeah.
[INAUDIBLE] student.
PROFESSOR 3: So the policy preference says, from S0, take action A1.
And that's going to stay there.
[INAUDIBLE] from A to C, there's no action to take [INAUDIBLE].. What that means is that the nodes that we might reach that are in our terminal state set, using our policy, is node A.
All right.
So that's the node going to expand.
And this is where you really see that all we've done is collapse down to A star.
A star would say, OK. What's the best node using some heuristic.
Action a1 one takes us to that best node, and we're going to expand just that.
So when everything is deterministic, basically this algorithm collapses down to A star.
Nothing super interesting.
The interesting case does come up when we start doing more probablistic ones.
That's where nodes are probabilistically helpful to the scanned graph sense.
So now we have our policy.
The policies are politely going to remain the same, because they have very little probability on the edge actions that we might accidentally hit up.
What would I want to look at is what [INAUDIBLE] and how reachable following our optimal policy.
So we talked about that if we were to actually read.
We know some probability of ending up in [INAUDIBLE] nodes.
So taking action A3 makes C, 2, and A all reachable.
What's reachable in taking action A1?
A and B.
PROFESSOR 3: [INAUDIBLE] And that's where we get the notion on this probabilistic algorithm that we're going expand things together.
Explore the part of the state space that we reach both higher optimal polity and that we might accidentally end up in if we [INAUDIBLE] the policy.
And this is what guarantees that we'll have an action to take from any state that we intended to go to or that we might end up [INAUDIBLE].. That's how our state expands, our envelope expands to encompass only the reachable and interesting states that we want to look at.
It's not as simple as just examining not only the heuristic, but with A start because we have this probabilistic [INAUDIBLE].. You can see that if you have a tighter coupling, then you don't get to exploit this optimization as much.
For example, A3 we would have had to expand more as opposed to A1.
We can just stick to A and B and ignore expanding C for now.
[INAUDIBLE]
Our [INAUDIBLE] We know that when people do the statistic [INAUDIBLE].
So in this case, if you take action a because of a and b, [INAUDIBLE] PROFESSOR 3: So in the complete sense of running this algorithm, we shouldn't print it.
Because what if we do?
If we have that 2% chance, [INAUDIBLE].. We need to have a policy for correcting.
So we end up at B.
You can imagine that these are going to try to push back to whatever path that A was looking for.
I suppose that the probability is low enough, you can have some cutoff percentage where you've decoupled [INAUDIBLE]-- PROFESSOR 2: So just a quick point.
So I think that's an excellent point and an excellent answer.
We're going to talk about next week-- exactly what's going to happen and if you can prove lower probability on the paper right here will be lecturing on that.
Good question.
PROFESSOR 3: That also gets us into the sense that if every state in the world were probabilistically coupled-- let's say we had some transporter to go with the Star Trek examples.
If we had this transported that non-deterministically put us in any state in the world, we have to explore the whole world, because we could end up [INAUDIBLE].
So luckily that's not the case yet and we can take advantage of the fact that we ended up most likely where we commanded with some probability of [INAUDIBLE] This is exactly what we just talked about.
We coupled these nodes with this And edge.
And we expand those, too.
Is everyone understanding the holding intuition, and the logic for why both of them have the [INAUDIBLE]??
Even if there's only a small probability that we end up [INAUDIBLE]?
So and this is what we're going to repeat until the states are expanded.
You can imagine that the next time we run our value iteration, we're now running it on all of these colored nodes-- both the blue and the red.
We can imagine that next time, now that we have a little bit more information what lies beyond A and B, that our policy might say, oh, you know what?
Actually from S0 C, action A3 was the best to take.
What that does is say, OK, with a regional map, it's A, B, and C. And we expanded those nodes.
We'll we've already expanded A and B, so we move into the next part of the sub space, that as we gain more information, we run value iteration and we can expand on why.
Steve asked a good question.
When we did our dry run, about whether there is a way to save on the computation you did prior to the value iteration and add these [INAUDIBLE] states.
And we've looked at it a little bit.
I've seen some stuff.
But I haven't found a paper that specifically deals with it.
You can imagine how you've already run value iteration on your previous iteration [INAUDIBLE] and you add the new terminal edges which you'd expand on.
And you run them again until it stabilizes.
And that way you've saved the computation of having to run valuation multiple times [INAUDIBLE] state space
So I'm trying to think of how this is different from something like [INAUDIBLE] for [INAUDIBLE].
PROFESSOR 3: I think I don't know enough about that.
But my basic understanding says that what's useful here is it's this explicit [INAUDIBLE]..
I don't know how much [INAUDIBLE]
And also, as long as your heuristic is admissible, it's guaranteed [INAUDIBLE] not all [INAUDIBLE] algorithms.
Just like A star is optimal, as long as you've got a consistent [INAUDIBLE].
PROFESSOR 3: All right.
So that's definitely the idea here.
We coupled these in the only explored of the portion of the state space that we'll reach an optimal policy [INAUDIBLE]..
So we'll talk about quickly another determination.
We've touched on most of this.
So it's most likely when there are no more states to expand this when we've reached our goal.
It's when our policy that we run on our entire envelope from value iteration doesn't say that we should go to anymore terminal [INAUDIBLE] that we haven't looked at yet, that we haven't seen yet.
We've said that those are only the things that are reachable and needed.
Both, reachable because we're following optimal policy, and needed, only if we might accidentally end up in the probabilistic.
This is gives us the sense of rigorous that we get policy on the entire state space that we can end up in following the optimal policy.
The third bullet here touches on if we don't expand the states that are probabilistically coupled and we do accidentally end up there, we risk getting lost and not having a policy.
We can compute this all off line and have a plan before we even start planning to know exactly where we want to go even if our dynamics aren't [INAUDIBLE].. We're come back to this.
This was our motivating example.
And so we show that these real platforms can be modeled stochastically and then we can pretty easily deal with that.
Search our state space and deal with those probabilities and expand the nodes that we might end up.
Right?
And the heuristic allows us to not have to explore these areas of state space.
I never actually end up there.
We'll always be a commanding toward 77.
We're never going to try to command backward.
Sure, if there's a gust of wind and we have some probability there.
But you can imagine that we're going to only explore a small portion of this.
Because we'll always be trying to correct to get back to the top four blocks.
And using our reward function, we get to determine if we want to fly a quick path or if we want to fly a safer path.
For example, our time times our probability [INAUDIBLE] we want to perhaps reduce that.
All right.
Are there any questions about planning with MDPs, anything like that.
I love this stuff.
So the more questions, the more I get to talk.
Fine.
What I'm going to be talking about for the rest of this lecture is extending beyond MDPs to a broader class of problems called POMDPs, Partially Observable Markov Decision Processes.
I love this stuff.
I think it's really cool.
They're really fun problems.
We're going to talk about why they're so much harder to plan with, to execute-- but why they're important to at least know about so that you can model real world problems with them.
And then we're going to delve into a case study of a specific POMDP solver.
We're not going to go into as much detail as we did for MDPs, but we're going to look at what powerful results we can get by planning with POMDPs.
PROFESSOR 4: So first, I want to rephrase this in the overall talk.
Right?
We have this spectrum of uncertainty.
And coupled with uncertainty is difficulty of planning, of solving, of executing a problem.
And we've killed these first two cases.
That was really easy.
And then we just discussed the bottom [INAUDIBLE],, MDPs.
What I'm going to talk about is the case where both your dynamics and your sensors are stochastic.
Why is that important?
It's because when we first saw this slide-- our motivating example slide, we only saw the left hand side.
We said, our actions are uncertain.
But good news, we have a perfect sensor-- a perfect camera.
But that's unrealistic.
I think, we have all, to some extent, experienced the fact that no sensor is totally perfect.
Your camera might have fluctuated pixel values.
Your laser range finder is going to never read out exactly the right number all the time.
You can have a camera in different lighting conditions that will behave differently.
You might not be able to observe your full state.
That's, in a way, an imperfect sensor, right?
If I'm in this room, I have imperfect eyes.
I can't map out all of MIT's campus because I'm blocked by walls.
How can you deal with the fact that you can't see all your obstacles all the time?
We've already talked about some cases-- that there are some algorithms that can help us with that, like D Star Lite.
But can you reason about these things probabilistically?
And then finally, you might be in a non-unique environment where you cannot resolve your state with certainty no matter how good your sensors are.
Imagine you're in a building with two identical hallways.
You're dropped off in one of them.
How can you figure out where you are?
You can't unless you start exploring.
And so we've got to deal with this uncertainty, right?
it's Part of every single problem.
When observational uncertainty is slowing, you can maybe ignore it.
But it's there.
And so we're going to formulate this as a POMDP, a partially observable Markov Decision Process.
And this next slide is just like the MDP slide.
Hairy, but important.
Right?
We can formulate a POMDP, which is seven elements.
And MDP too five.
Most of them for all those are carried over here.
We've got our set of states where we can be.
We've got a set of actions what we can do.
We've got our transition model which says, given that I started in one state and then I took an action, what's the probability I end up somewhere else?
And like David was talking about, hopefully that distribution is pretty local-- we're not teleporting all over the world.
We've got our reward function.
This is exactly the same as for MDPs.
And we've got our discount factor down here.
The key difference of POMDP is these two elements.
We've got a set of possible observations and a probabilistic model for the probability of making an observation given your state and the action you just took.
Now it's important, I think, it matches up really well with real world sensors having this probabilistic model.
If you have a laser range finder, for example, and you're standing one foot away from the wall-- now a perfect sensor would always say, you're one foot away from the wall.
You're one foot away from the wall.
Every single reading is constant.
But realistically, there might be Gaussian noise, for example.
Or at a more extreme case, it says, your one foot away from the wall.
You're two feet.
You're right there.
There's this distribution.
And so you would ideally characterize this distribution.
And you plug that into this model and that formulates your POMDP.
This sounds really hairy, but if you work through just a sample iteration of living in a POMDP world, that's not too bad.
You start at some state, S. You take an action, A.
With some probability, you're going to end up in a bunch of different states based on your transition model.
At that point, we can use the lessons we learned from MDP land where we said, when we make observations, we reduce our uncertainty.
we collapse into a single state.
So we say, let's make an observation.
But this time, observations aren't guaranteed to resolve all our uncertainty.
So we make an observation.
And that observation is probabilistic based on our current state and the action we just took.
And again, obviously, it depends on your current state.
Because if you're one foot away from a wall, hopefully you'll get a different characterization of observations than if you're 20 feet away from the wall.
Otherwise, your sensor it's totally useless.
Are there any questions about this formulation?
Yep.
Quick question.
So we will take observation, any other observation and then you try to infer which state you're in, is it just a clustering problem?
For instance, the multi-cluster Gaussian [INAUDIBLE] models.
So class A, class B, class C, which are state, then taking an observation there's a high probability [INAUDIBLE] This is what we're tyring to do for each observation over here.
So we're trying to find clustering [INAUDIBLE]..
PROFESSOR 4: So you could, I imagine, implement an algorithm where, yeah, every time you make an observation, you then try to say, all right.
What's my most likely estimate, or maybe my [INAUDIBLE] least cost estimate?
But inherent with that is the risk that you're discarding a lot of information, right?
Because you're going to generate a probability distribution over your state.
And so, yes, you can say, I'm going to stick with the maximum likelihood estimate.
But if you can, you should probably try to maintain that distribution as long as possible.
OK. And we'll see that this is really computationally expensive unless you start making some assumptions.
And in the case study we're going to look into, that's exactly what [INAUDIBLE].
But I've seen a lot in the literature that as much as you can, you want to maintain these distributions for improved accuracy.
Any other questions?
All right.
Well, we're going to compare now the execution of a POMDP to the execution of an MDP.
We started out-- we're living in the same real world.
We've got our same transition model.
Everything is peachy.
We take action one, and we want to go North.
We have this distribution that we generate the states.
And at this point, what did we say?
We said, we hate the fact that we have to deal with three cases.
Three is two too many.
So let's make an observation to collapse this distribution.
Now I've described a lot about noisy sensors, right, where basically it's a true measurement plus some noise, maybe Gaussian distribution.
There's another partially observable sensor you can have in the POMDP which really feeds into the name, Partially Observable.
What if you can only observe part of your state?
For example, if you're living in an x-y grid, maybe you can only observe your y dimension.
This match up, in a real world example, to quadrotor flying down a hallway.
Catherine was working on a DARPA project with quadrotors flying down hallways.
If the hallway is too long, your laser range finder isn't going to be able to determine where you are along one axis.
But it can tell where you are along another.
And so that's what I've said.
I've said, pretend this quadrotor has that sensor.
We can only observe its y component.
And it says, my y component is 3.
I have no idea what my x component is.
Well, this sucks.
Right?
Because we got rid of this state, but then we couldn't decide, are we at state (1,3) or (3,3)?
There's no way of resolving this.
So we can re-normalize.
We can add in the effect from the observation probabilities saying, maybe, in fact, I'm far more likely to observe a y component of 3 if I'm at (1,3).
But in this case, we say it's equally likely to make that observation for those two states.
And so now we've got to deal with these two cases.
And so we can take our next action.
Instead of resetting to a single state, we've got to keep growing this tree.
And what's the key difference between this and when we were executing our MDP?
Except we didn't manage to collapse back to a single state.
We didn't manage to reset the problem.
And this is annoying.
Because you can't execute a policy and say, I'm certain that this is my configuration.
So your policy can't map from exact states to actions, because you never know your exact state.
Does this make sense?
Has everyone lost hope in planning, right?
Yeah
So in here because-- so from the left to the right, you're basically mapping from one belief state to another belief state.
So it's like a one arrow thing.
But and then from that second layer, you should have two arrows.
One with probably 0.5 that observes 3, and it gives rise to that belief state it just showed.
In one, with probability of 0.5 where the sensor is being 4.
Because if you have a 0 probability of looking at (2,4), you might as well just observe 4 as well as your y.
So in this case, I'm not sure if you were just trying to show one branch of your POMDP planning, but basically what you have to do is you would go like this to the second layer.
You would have a branch with a 0.5 probability on either.
One giving you 3, one giving you 4.
For the one that is showed, you've got that relief state.
For the other one, you get the state (2,4) with probably 1.
PROFESSOR 4: Yeah.
So you were perfectly describing planning, right?
I should have made this more clear.
This isn't planning.
This is executing.
We have this policy that we're going to execute.
And so if we were planning, we would have to consider all these branches and say, well, yeah.
There's a 50% chance here I'll end up here, in which case, my y value is going to read 4 and you have to grow this whole thing.
I'm saying, no.
This is real time execution.
yeah.
Great question.
Any others?
Well, this is a great time to transition to, well, we can't just magically be handed these policies.
How do we actually generate them?
How do we start planning in the belief space?
The belief space is the space distributions of possible configurations.
So I'm going to talk about a general class of algorithms.
A lot of planners in POMDP land and the belief space plan with probabilistic roadmaps- PRMs.
The goal is to generate a policy that maps from a belief state to an action.
And I'm going to go into a little more what a belief state is.
But the general algorithm in this graphic illustrates these four steps.
We're going to sample points from the configuration space, as if everything was deterministic.
We're going to connect those points to nearby points.
And define nearby however you want.
It could be your closest neighbors, all neighbors within a radius, whatever.
As long as those edges don't collide with obstacles.
Once you've done that, somehow-- and there's some magic in this-- you're going to transform your configured action space probabilistic roadmap to a probabilistic road map in the belief state.
Great.
Once you've done that, you can just do shortest path depending on whatever cost function you use.
And what's really cool is you get different paths for when you stay in the configuration space and when you go to the belief space.
So the green path, that bottom right figure, seems a lot longer than the red path.
And the reason is the green path was planned in the belief space and followed a lot of landmarks that the quadrotor could take measurements off of.
So it was really confident about its position whereas the red path is the shorter path that had a higher likelihood of a collision.
And we're going to look into that figure more later.
I'm going to segment this algorithm into two parts.
The first part-- these first two steps-- it's just probabilistic road maps.
Who here has heard about probabilistic road maps?
Raise your hand.
OK. 50/50.
I'm so excited for the 50% who haven't heard.
One of the top algorithms, in my opinion.
It's really simple, and it's really powerful.
Here's basically almost a complete implementation of probabilistic road maps.
It's pseudocode so don't copy paste, but it's almost there.
You're going to construct a graph.
You're going to add your start goal.
Your start configuration being the green dot and then goal configuration, the red dot.
And then you're just going to keep sampling notes those from the free space.
You say, how about (2,3)?
You're going to add that to your graph.
You're going to connect that node to a bunch of other nodes nearby.
And then you're just going to keep sampling until maybe you have enough nodes or maybe until you have your complete path.
Should be happening there.
And then once you've got this whole graph, you can just find the shortest path along that.
And there are some really cool results that if you sample in a good way, and so on, asymptotically.
As you start sampling more, you're going to asymptotically approach the best path in a completely continuous space.
The power of probabilistic road maps and a bunch of randomized algorithms though is that they scale pretty well to high dimensions.
So you don't need to actually consider the continuous space.
You can just sample [INAUDIBLE].
Are there any questions about probabilistic road maps?
Really cool.
If you are interested, and you just heard about PRMs.
You probably haven't heard about RRTs.
Those are also really cool
Just a quick [INAUDIBLE] question.
So for any node, [INAUDIBLE] at this uniform example [INAUDIBLE].
PROFESSOR 4: Sorry.
When you're choosing where to place the dot?
Or what to connect to?
[INAUDIBLE] there's a dot there on the side [INAUDIBLE] right?
So when I do the sampling, I just [INAUDIBLE] this node.
I uniformly choose one of them [INAUDIBLE]..
PROFESSOR 4: So in general, probabilistic roadmaps, you can throw in whatever sampler you want.
The way this particular one-- the way I implement this is you sample points uniformly from the entire space.
If it's inside an obstacle and you remove it, once you place it-- this was connect to the K closest-- I think K is 7 in this case.
And if there's an edge that if that edge collides with an obstacle, remove it.
I'd be happy to go into more detail for that PRMs later.
All right.
But that's not enough.
Because what we described as PRMs in the configuration space, but what we need to do is somehow elevate a PRM from the configuration space to a belief space.
And this is really hard.
We don't have access to these raw configurations.
Let's imagine we were in this really simple world where the quadrotor could be in three possible states.
One, two, three.
Really easy, right?
Sample a bunch of points.
They're going to end up in one, two, or three.
You could pretty quickly cover the entire space.
But this simple configuration space will transform to the belief space becomes infinite.
You have infinite possible distributions to consider.
There is the distribution where you have 100% chance probability-- 100% probability that you're in state 1, 100% probability that you're in state 2.
100% probability that you're in state 3.
And then everything in between.
We went from three to infinite.
This is not boding well.
And even if you start saying, well, I'm not going to consider the whole distribution.
I just care about the mean and the variance, it's still not pretty, right?
Well, this is where we have to start making approximations And this is where you start getting differences in POMDP planners-- where they make assumptions, where they make approximations.
So these images are from the belief road map paper from the Robust Robotics group a couple of years ago.
But I'm going to talk about a different planner soon.
But the idea behind a lot of these planners is maybe we can start saying what these distributions are going to look like based on our models.
So to our planning problem, if we know we started at (2,3) and we know our transition distribution, we can start saying, well, this is my probability distribution.
And then when I make and observation, I can build distributions off that.
And so, if you could exhaustively propagate these distributions forward, that would be great.
But it's unrealistic.
And I just want to point in terms of the visual way to represent these distributions.
A really nice way of saying, in the deterministic world, you have these dots and edges.
In the probabilistic world, these circles, these ellipsoids, represent uncertainty.
Typically, it's the one standard deviation or three standard deviations away.
And so you can start building into the map, these are the distributions and the variances I can see in these nodes.
Are there any questions about this stuff.
All right.
Well, we're going to delve now into a specific case study.
Feedback Based Information State Roadmaps-- FIRM.
From now on, that's the only way I'm going to refer to it.
The idea behind this is you're going to sample mean configurations from your configuration space.
Then you want to build an LQR-- that's Linear Quadratic Regulator controller-- around these mean points.
And that will generate what variance you can tolerate.
So LQR controllers, if you don't know, they're really nice.
Around a small region around a point, they can drive a quadrotor, for example, to that figure.
And so if you build these LQR controllers around points, you can say, all right, anytime I end up in this cloud in the belief space-- so any sort of distribution and can bring it back to that mean.
And so now, what we've done is we've generated every point, we just need to connect them.
And the idea is if you have a feedback based, they can get from one cloud to another anywhere in the clouds.
Then you can get from point to point.
All right.
This is how you generate the graph.
And what's cool is the way they formulated the problem is they said, well, set up the cost of executing an edge.
We switched from rewards to costs because we're pessimists now.
Well, the cost is going to be a linear combination of the expected time to execute that edge and the uncertainty along that edge.
Now, this is actually really cool to play with.
What do you think would happen if I set beta to 0 in this?
Like, what sort of [INAUDIBLE] would you get?
I will cold call because I know a few names here.
Anybody?
I don't want to cold call someone who may [INAUDIBLE]
You're going to get the shortest path.
PROFESSOR 4: Shortest path, that's right, right?
This turn goes away.
The cost is a function of the time-- shortest path.
Now, what's cool is one day I was messing around with this code.
I'm like, I wonder what happens if I set alpha to 0, right?
So your cost is purely a function of uncertainty.
In turns out what the quadrotor does it just hangs out where it starts.
It says, I'm in no hurry.
I know where I am.
I'm just going to stay here.
But I find this amazing, right?
Because this almost models behavior even.
You could start saying, do I want to be a risky quadrotor or a safe one?
Like, how important it is for me to get somewhere on time or be safe?
And it's just those two parameters.
Or you could even make it 1, like alpha and 1 minus alpha.
I think this stuff is really cool.
The one detail that I'm really going to into for FIRM is the cost equation, which is based on the Bellman backup equation that we had.
The cost to go, right, the expected cost from a belief state [INAUDIBLE] is-- well, you're going to take the best action.
So you're going to take the mide of this whole thing.
You're going to say, it's the cost of executing a specific action plus the cost of colliding with something, an obstacle, times the probability of colliding, right?
That's this term.
And then you've got to say, well, OK, and then once I reach a state, what's the cost from there?
And then I could weight that by the probability of ending up in that state.
Does this equation make sense to people?
There are a lot of symbols, and honestly, I hate notation, but it works.
You just plug in-- the cost is I'm going to take the best action.
It's going to be the cost of using that action, the cost of colliding times the probability of colliding given that I used that action and I started where I started, and then the cost from where I end up.
And since you end up in a probability distribution, we need to consider all these cases.
Yeah?
When you define like an action from one place to another, do you always think of it as starting from mean that you sampled or from anywhere [INAUDIBLE]?
PROFESSOR 4: So it's from the-- so formally, it was from the belief state, which is the mean, yeah, plus the variance once it stabilized to that point.
And the way that variance is generated, I should have said, is, you're going to have models of these quadrotors.
And so I spent a good time in the ACL with John Howell, in Course 16.
And you just like let the quadrotor hover.
You measure its position for a long time.
And you get a distribution over where it goes
So what does the letter M stand for?
PROFESSOR 4: What is the letter M?
Yeah.
PROFESSOR 4: Right.
So you're summing over-- these are the belief states that you could end up in, right?
So if everything were deterministic, this would just be ignore the sum.
It's where you end up.
Realistically you could end up in some other state we haven't considered
So just a quick question.
So those, if we're operating on a Gaussian space, then you have Gaussian observations.
So those are sums over the observation samples that are generated when [INAUDIBLE].
So that is a finite sum over the possible infinite observation states we might have.
PROFESSOR 4: Yeah.
So there's definitely [INAUDIBLE] in terms of where you could end up.
And even the action space, there's a set feedback controller that you're allowed.
I think the observation, if you modeled it as a Gaussian, if you made some nice assumptions, it can be tractable as continuous
Oh, yeah.
I was [INAUDIBLE].
So for the start, so if you start with Gaussian noise, then you have a linear model.
Then your prediction's going to be Gaussian.
PROFESSOR 4: Yeah
But then you have Gaussian observations, which is great because then your update is Gaussian but the observation space is infinite.
So basically not only you have two sample positions, but you also have two sample potential observations that you might get as you go along.
PROFESSOR 4: Yeah
So that you can basically-- and it's great.
But you end up basically reducing a possibly infinite branching, which is your Gaussian to kind of a like a Monte Carlo Tree search.
You generate a whole bunch of meaningful samples, and those are the ones that you consider.
So is that where the sum is coming from?
PROFESSOR 4: Yeah, basically.
And a fun fact, the theorem that I read and trusted was that if you just sample randomly right from your configuration space, you have zero probability of constructing a graph that without any assumptions will be connected.
Whereas for PRMs, you sample enough and things will turn out nicely, not the case with the belief state.
That's why we need to make these assumptions.
We're killing it.
We know POMDPs.
Now we get to look at some really fun graphics generated from real flights using FIRM.
The big takeaway is that FIRM prefers safer paths.
We've got two images that look really similar.
We're going to talk about one and then show why they're slightly different.
The test flight that we put this quadrotor under was we said, we're going to start at this configuration.
We're going to go this configuration.
And there's this big, blue obstacle.
And there are landmarks that you can take measurements off of.
So those are these red dots.
We want to compare two planners right, a PRM and a FIRM planner.
The PRM planner said, right, I just want to minimize time.
And so it found the first path is stay to the left of the obstacle.
But that's actually a really narrow path between the obstacle and the wall.
On the other hand, the FIRM planner said, right, I want to minimize the linear combination of time and uncertainty.
And so if you ramp up the term the wait for uncertainty, at some point the path sort of pops over the obstacle.
It's really cool.
It snaps.
And then you get this safer path.
But that's longer.
And how do we know it's safer?
Because these ellipsoids that we've drawn are the 3D versions of what we saw for PRM elevated to the belief space version.
It's the uncertainty that the quadrotor has over its true state.
Why is the PRM plan-- why does it have such big ellipsoids?
It's because when it's behind the obstacle, it can't make any of these measurements because it can't see the landmarks.
So it's basically using dead reckoning.
And the transition model, right, is not deterministic.
So it's uncertainty grows.
And so we can see these ellipsoids are bigger for PRM than FIRM.
And then the reason I have two images is they are slightly different.
It's that these landmarks were fictitious landmarks that we just said, you can-- and we would generate fake measurements off of them.
And so we could tune the noise of the landmarks.
Maybe a little bit of cheating, but it allowed us to say, would it increase the noise from some-- the dimension was number 0.05 to 0.15.
You can see the uncertainty ellipsoids from PRM grow when we increased the noise, whereas for FIRM, they stay about the same.
And importantly, these ellipsoids grow enough that they start overlapping with the obstacles.
That represents a very high probability of collision.
Whereas the FIRM, the way it managed to keep these ellipsoids so small is we kept the cost function the same, right-- the same weight on uncertainty, same weight on the time, right?
But the uncertainty was so much higher.
And so we decided, well, I can sacrifice a little bit of time.
I can take the slower path.
I can just hang out by landmark, really make sure I know where I am before continuing.
And so the path-- the duration of the flight took a lot longer for FIRM as time would increase but the uncertainty would stay about constant.
Do people understand these graphics?
Great.
All right.
We've got a graph that just represents a little more formally that growing uncertainly.
As noise increases, the variance along a single dimension, z, y, to x, for PRM, which is that, and FIRM.
The variance for PRM is always higher, and it grows.
For FIRM, it's lower, and stays about constant.
That's the big takeaway.
FIRM minimizes this uncertainty.
And then the final image from these results that I want to show is in simulation.
They said, well, let's actually measure how often it crashes.
We didn't want to do this in the real world because we don't want to crash the quadrotor that many times.
The gist of it is comparing a reactive planner and a deterministic one [INAUDIBLE].. As noise increases-- noise was simulated with wind strength-- the number of crashes increases for PRM, basically.
And for FIRM, it stays constant and low.
The reason there are two lines is there were two planners with different time horizons.
The important thing is FIRM is low and constant.
PRM grows.
We've talked now throughout all probabilistic planning you ever need to know, right?
No, not quite.
But we have covered a lot of stuff.
What are the big takeaways?
We've learned that real-world problems are stochastic, right?
Quadrotors are not these perfect machines that we wish they were.
But it's important to model them as stochastic.
The problem is once you start modeling them as stochastic, it becomes a lot harder to solve.
But if you make some assumptions, or even if you don't, if you're just get smart and you do the heuristics, you can resolve this complexity.
And so I hope you remember these three points, you remember this graphic or that graphic, the idea that if you take uncertainty into account, you get fundamentally different paths [INAUDIBLE].. And that can be a good thing.
What questions do you have, anything?
Yeah?
[INAUDIBLE] so far people have [INAUDIBLE] problems.
So how do the same, you know, [INAUDIBLE]??
So for instance, we suggest that this is the maximum risk we want to take.
Is it possible to integrate each of our constraints into your optimization problem and solve it?
Or [INAUDIBLE]?
PROFESSOR 4: So I imagine one thing that I would like to test is if we go back to this really crude version of our cost equation, we can imagine saying that if we want to come up with a bound for uncertainty that we can tolerate, you could maybe like setting an intercept for this to be that bound and then just ramping this up
Oh, [INAUDIBLE] multiplier [INAUDIBLE]..
PROFESSOR 4: Something like that
[INAUDIBLE]
Yeah, exactly.
So it only comes into effect.
Possibly what I said is a terrible idea.
But it's-- especially in simulation, you can just try it out and see if it works
And just another question, you said propagated probability solution on the networks is hard.
[INAUDIBLE] So therefore we can assume an illustrated area, and then [INAUDIBLE] normal distribution, they are contrary to each other.
Therefore you can [INAUDIBLE] observation and then updating [INAUDIBLE] and form a distribution, right?
PROFESSOR 4: Sure.
So I think that might be making some assumptions that we might not be willing to make about the nature of the distributions that you're trying to propagate.
I need think about this some more.
But I think intuitively, we can understand that any time you're propagating a distribution versus a single discrete value, it's definitely not going to be easier.
And so as your distributions become more complex.
Perhaps if you're modeling a real-world stochastic sensor, you might not be able to perform these efficient updates using conjugate priority.
Any other questions?
Otherwise we-- yeah?
So all these FIRM examples you're showing, this is all planning done offline.
[INAUDIBLE] offline, right?
Have you expanded this at all to like an online case?
Or does that all require re-planning?
And how long does it take?
PROFESSOR 4: Right.
So it's not good, I can tell you that much.
What's nice is FIRM generates a policy.
So if you construct your PRM to say don't just find the shortest path, keep sampling points until you're confident that you're always going to end up near some point.
You can construct a policy and then just online look up what's my belief that matches most closely.
It took for-- I think we sampled 600 nodes in like-- what are the dimensions on this?
So 2 to 3 meter by 8 by 3 meter thing-- it took about 15 minutes.
It was really slow.
Now, granted it was a virtual machine.
But it's not something we want to re-plan on the fly.
With PRM, typically a lot faster
Who was a lot faster in that case?
PROFESSOR 4: So when we just did PRM, I think it was one minute or something, at least an order of magnitude.
And you could use other planners as well, like RRTs.
And there are RRT versions of-- there are POMDT versions of RRTs.
But [INAUDIBLE].
All right, I'm going to turn it over to Steve so we can learn about this project.
[APPLAUSE]
So I'm just going to say a few good words about the Grand Challenge.
So again, the final assignment will be released tonight for this.
And as Professor Warrens mentioned, it's going to be descoped a bit from what we originally had in the syllabus due to time constraints.
But here's a preview of what you'll be doing.
So for an overview, it will be a class-wide collaboration.
So what we're going to do, you guys are going to stay in your advanced lecture teams and each basically apply what you've done, the great work you've done on that, onto our hardwork robot that we have.
so it's not a competition where each team will be competing against each other.
And so again, we're descoping it.
The syllabus originally said it was 20% of your grade.
It's probably going to be more like 10% or 15% in the end.
So this is the robot you guys will be using.
It's called an [INAUDIBLE] robot.
Usually it has sort of a robotic arm on it.
But we're actually not going to be using it for this [INAUDIBLE].
This base made from a company from Spain, Orbonix.
It's a pretty cool robot.
One cool thing about it is that it's actually omnidirectional.
So there are wheels on your wheels here.
And what that means is that it can drive sideways, left to right.
It would make parallel parking your car very easy.
[LAUGHTER] We're not going to be driving it probably this fast because it's actually super heavy.
And we don't want anyone to-- yeah, we forgot to screw in that [INAUDIBLE].. [LAUGHTER] It will be screwed in during the competition.
But it's a pretty fun robot.
So you guys will be working on this.
And so the actual challenge itself, as we've mentioned many times throughout this class, will be a modified orienteering your challenge.
So there's going to be a few different challenge stations that you have to drive to.
And at the stations, there'll be small computational challenges.
And those computational challenges will be a subset of the advanced lecture teams, not all of them.
And the goal is to complete as many of those as you can and try to do that as quickly as possible.
And it's also going to be held indoors in our lab space, where we just showed you.
So that way if it rains, we can still have the Grand Challenge at the end of the semester.
So this is sort of how it's set up as of last night actually.
So basically the robot will be abe to drive around in a small little LEGO maze that we set up.
And there's going to be sort of different things that you have to do with in different places.
So what are you actually going to be doing?
What assignment are you guys going to be doing?
Well, it's actually a little flexible.
Since each of you did different things for your advanced lectures, each team is going to have a bit of a different assignment applied to this.
I have some proposed ideas that I'm going to talk about in the next slide here.
But the big thing is that these are just ideas.
You guys have a lot of flexibility in this.
You'll be probably working with the [INAUDIBLE] a lot to have access to the robot.
We can arrange extra office hours for you guys to come use the hardware to test things.
So we'll be arranging all of that things as sort of on an as-needed basis.
If you want to-- basically it'll sort of depend on your team
And the people who'll be helping out are you, us--
Yes, me and Tiago, who gave a lecture earlier in the semester and possibly also a few other people from our lab.
But we'll be the main contact points for it.
So of course it should go without saying, but all the team members should contribute equally within your team.
So it'll be maybe less structure than in advanced lecture.
But just make sure that everyone's contributing equally in the assignment.
And it's going to involve using this thing called the Robot Operating System, or ROS, which is basically a software framework for communicating and it's used a lot in robotics.
Just a quick show of hands, how many of you have used ROS before?
Oh, wow, so a lot of used ROS.
How many have heard of ROS, if not used it?
OK, so a lot of people.
So that's a good starting point.
So here are the the proposed tasks for each group.
And of course, all of these are up for change.
If you guys want to change it, let me know.
Basically, so some of the groups, it's very clear how it immediately applied to the Grand Challenge.
So incremental path planning-- well, we have a mobile robot, so maybe we can actually print that on the robot and get it to change or plan if something gets in the way.
The semantic globalization group-- obviously very applicable to the Grand Challenge.
You need to know where you are.
So the robot that we have now can actually do normal metric localization.
So it can sort of know where it is.
But what will be interesting to see is compare the semantic localization to that one.
But how would you do semantic localization?
Well, we can use a camera.
And we can choose the visual learning through deep classification-- the visual classification through deep learning group.
So I think these two groups would have a really nice synergy and a really cool way to work together.
So the MCTS group are very cool, but I had a little trouble thinking about exactly how that would apply.
So maybe you look like you have an idea maybe
So to clarify, all these are separate runs of the robot?
So we're actually probably going to run all of them-- so the grid is-- this would be probably one run of the robot.
We're going to decouple these so that if one or a subset these don't work super well, the other groups will still be able to run.
So we're carefully planning that out too.
So the MCTS group, I was thinking maybe could solve-- that could be a really nice way to-- one of those computational challenges.
Maybe you have to play against a human.
And if it wins, great.
You can go faster or you get more points.
So you could implement on a different game other than connect four or possibly and sort of [wrap it around [?
rod ?]
that to call with that.
So each ability group I think would also be a great place to do one of these challenge stations.
So we could give you guys puzzle, say maybe it's some sort of maze-like state space.
And you have to see could we even the reach the goal here?
And if you get the answer right, well, you an move on to the next stage or get more points or something like that
But again, these are just suggestions.
So for example, they can do the reachability for doing motion planning
Yeah, so if you guys have other suggestions on how to implement your team's stuff into the Grand Challenge, definitely send me an email, preferably today or as soon as you think of the things.
And we can change these.
These are just suggestions for right now.
The last two are sort of more planning related.
So planning with temporal logic, which was Monday's lecture, I thought would be cool way to sort of control the robot's high-level action.
So maybe that could involve modeling are Grand Challenge with PDDL.
And maybe with linear temporal logic goals, models that you get [INAUDIBLE].
Then you could compile that and call up to turn around it and execute that plan on the actual robot, do [INAUDIBLE] high-level actions of the robot.
That's a possibility.
And for today's group, the infinite horizon probablistics planning, maybe you could do something actually similar.
But instead of modeling the domain as PDDL model it as an NVP and solve it with LAO star, and get sort of a policy on how to control the robot.
Maybe we can change it a little bit so that certain squares can be more risky than others or so on.
So again, there's flexibility in all of these here.
So these are just some of the suggestions.
Does anyone have any questions before we-- this all I have.
So anyone have any questions?
Yeah?
How are we working on [INAUDIBLE]??
Are you [INAUDIBLE]?
So it's basically up to you guys really divide up the work amongst yourselves.
It's really different for every team.
So we're--
Sorry.
So each team is developing [INAUDIBLE]
Right, yeah.
Each team is really in their own separate package FIRM.
For example, these two teams, there's probably going to be a common interface where the output of this one goes to the input of that one.
So for that one, we're going to give you sort of the interface [INAUDIBLE] message type package for that.
But other than than, like for dividing up the work, you guys [INAUDIBLE] that.
PROFESSOR 2: So your plans will be integration if you're trying [INAUDIBLE] pieces the group is doing.
That we think is, thing that uses soft engineering skills [INAUDIBLE].
It can be really unpleasant to do, take a lot of time.
So we don't want you to have that experience.
So that's why you can...
If not only what we wanted you to do is to be able to get your own capability [INAUDIBLE]..
If you guys choose to integrate with other teams because you're really excited about that and because it looks like the people that you're working for [INAUDIBLE].. Then that's purely your choice.
Is that fair enough?
Sure.
It's fine.
Any more questions?
OK.
Sounds great.
[APPLAUSE]
My name is Olivier de Weck.
And I'm a professor here at MIT in the AeroAstro Department, also known as the Department of Aeronautics and Astronautics.
And I've been in the department since 2001.
And my passion is aerospace vehicle, anything that flies, airplanes, rockets, satellites.
And it's been my passion since I was a kid.
So I grew up in Switzerland in the Alps.
And the Alps, especially in the wintertime when it's a clear night, are amazing.
You look up at the night sky.
And you can see the stars.
You can see the Milky Way.
And as a kid, it really inspired me.
I wanted to do something with stars.
Initially, I thought I was going to be an astronomer.
But then, I really liked building things.
And so aviation, rockets, spacecraft really became my passion.
Later, I did my military service.
And I worked on planes, flight operations, maintenance, repair, really learning planes in and out, the engines, the hydraulic system, the avionics, working with others and pilots to keep airplanes flying safely.
And that really was the origin of my passion for systems engineering.
OLIVIER de Weck: About five years ago, we started thinking about our classes in the area of aircraft systems and spacecraft systems in the department.
And we started realizing that there was a particular class that we didn't have, and something that seemed very important to us, and also very important to people in industry.
And that is a general introduction to systems engineering.
Systems engineering is essentially the art and science of coordinating and putting together the different disciplines in order to create safe, efficient flying systems-- not just flying systems, but systems in general.
And so we decided to create a six-unit class, which is a half class, that would focus on the methodology and the processes in systems engineering, which is going all the way from stakeholder needs and expectations, all the way to a functioning system that can be deployed in the field.
This class, 16842, is really a door-opener to the world of systems engineering.
And what we use in the class to orient everybody is the classic V-model of systems engineering.
Now what is the V-model?
The V-model takes you step-by-step through the life cycle of a system, starting with the left side of the V, which is, why should the system exist in the first place?
Who are the stakeholders?
What are their expectations?
How do you formulate a concept of operations for a system?
And then go step-by-step through the V, through the requirements definition, concept generation and selection, driving the system design, all the way to the details, to every little part, and piece, and component, and then moving up the right side of the V, where you integrate all these pieces into subsystems and modules.
You do verification, validation, testing.
You commission the system and bring it into operations.
And even then, the job isn't done.
You have to accompany the system throughout its life cycle through maintenance, upgrades, repair, making sure the system is really operating as efficiently as it can.
So what the class does is take students systematically through every portion of the V, so that at the end of it, they have a good overview over, what is systems engineering?
And what are the key steps in the life cycle of a system?
Many of you have heard of MOOCs.
They're very famous, Massively Open Online Courses.
MOOCs are very open and large classes, often with thousands or tens of thousands of participants.
And MOOCs are essentially a big movement to really democratize education.
SPOCs are a little different.
So 16.842 was run as a SPOC, which stands for Small Private Online Course.
So like a MOOC, a SPOC is also offered online.
You can take it in person in a classroom at one of the hosting universities or you can take it entirely online.
It's really up to you.
It's small because we have a few dozen students instead of thousands, and it's private because in order to take the class it's by invitation.
In this particular instance, we had two universities that co-sponsored this class.
One is MIT and the other one is called EPFL, Ecole Polytechnique Federale de Lausanne in Switzerland, which is also quite active in the world of online education.
The way fundamentally that a SPOC works, is that you have the course management system at each of the hosting universities and institutions.
And you essentially post the same materials, the lectures, the readings, the exercises, you post them in parallel at each of the hosting universities.
The lecturers themselves, however, are run online using an open collaboration platform, like WebEx for example.
You can do it with Google Hangouts.
There's a lot of different platforms.
And this is important because it is important to have live connections between the faculty and the students, but also the students amongst themselves can discuss.
The trickiest part of running a SPOC, of course, is the time zones and the time differences.
In this case, the two universities were six time zones apart.
And so it was offered in the morning here at MIT and then the afternoon in Europe.
If you're running this as a global class, it will be more challenging to find a time slot that works for everybody.
The feedback from the students has been very positive, because you can think of a SPOC in this particular class as a blended form of education.
Part of the class is live.
The lectures are given live, and every lecture is recorded.
So even if you miss a lecture as a student, you can watch it later.
And so there's a record of it.
The assignments are asynchronous, and there are some assignments that are individual.
And there are team assignments as well.
And so it's a blended form of education, which is an important trend in education where we take the best of the online experience, and we try to combine it with the best of the in-person experience.
Teaching a SPOC is fun and challenging at the same time.
It's fun because you get to converse and interact with students, not just in a physical classroom at one university but across multiple institutions.
And that enhances the class, because students will speak up with experiences from different cultural backgrounds.
But it's also challenging, because you have to keep track of what's happening at multiple places.
And you have a mix of students from different backgrounds and different institutional constraints as well.
And so it's both challenging and fun, but it's a very focused experience.
And the feedback from the students has been great.
In order to anchor the class, and provide an application that is at the same time realistic, but not overly complex, we chose to participate in the 2016 CANSAT competition.
And CANSAT is just what the name stands for.
It's the design of a satellite, that fits into a can, that is launched, not into space, but at relatively high altitude with a sounding rocket, essentially.
And so the CANSAT competition has been run for about 20 years.
It's quite well known, and there's teams from all across the world that participate.
There is now a CANSAT competition in the US.
There's also a European version of it.
The goal of the CANSAT competition, is for students to go step by step through the design process.
The starting point for the students, is a set of requirements, 47 requirements that are given by the organizers, about what the particular system has to do.
For example, the CANSAT has to fit within the payload fairing of the rocket.
It has to survive the launch.
So a certain number of g loads and vibrations have to be survivable by the payload.
The payload then has to separate into both the glider portion, and the actual container that contains the payload.
On its way down, the payload has to fly in a circular pattern, for about two minutes.
And during the descent, it has to record temperature and pressure of the atmosphere.
And you get extra bonus points if you record images, and transmit those images to the ground.
The students who are responsible for everything except the rocket itself.
So the design of the glider, the container, the ground station, and all the procedures.
So what's interesting about systems engineering and design, is that there are different approaches, how to tackle the problem.
One is an approach where you take it step by step, and you try to basically get the right answer at every step.
But, at some point you need to check yourself, as well, with milestones.
The other approach, and that's typically what we call a waterfall, or stage gate process, which is what's applied in very large systems, where it's too expensive to do a lot of prototypes, and you have to get it right the first time.
The other approach is spiral development, or agile, or rapid development, where you do quick prototypes, and you learn very quickly, and iterate your design.
And in some sense, what we're doing in this class, is a combination of the two.
What was very rewarding, is to see how students ideated their concepts, always keeping in mind the requirements, and the end goal in mind.
But, also testing those ideas, either through modeling and simulation, or with very simple prototypes that can be made out of paper.
One team actually produced a 3D printed version, of their glider, even though that wasn't officially necessary before the key milestone, which is the PDR.
And so it was interesting to see, how every team approached it slightly differently.
One of the key ways of learning is to look at examples, past examples, of both successful and failed projects or designs.
What we find in systems engineering is that often the problems begin at the requirements stage, where either a requirement is missing, or there's redundancy, or in some cases requirements are written that are overly ambitious and utopian given the time frame and the budget available.
So what you have is always a trade off between how ambitious you are technically, the schedule, and the budget that's available, and how much risk you are willing to take.
And so the students are pushing the envelope in these different dimensions and learning what's feasible, what's not feasible, and what can we learn from past failures.
We can learn from general examples of projects that had difficulties, either in aerospace, in the automotive industry, in consumer products.
In hindsight, one is always smarter.
But it is worth to look at these.
Specifically with the CanSat competition, since the competition has a rich history, and a very active set of sponsors, and quite a good website as well, there's a lot of opportunities to learn from mistakes that other teams have made in the past.
For example, designing a glider where there's not sufficient tolerances between the vehicle itself and the container, and basically the deployment doesn't go well.
Another example is designing a glider that does not fly properly in a circular pattern, and flies too well actually, and flies off into the woods never to be seen again.
There's, of course, a lot of lessons learned from establishing a stable communications link, from having worked out procedures.
How do you deploy your system?
How do you test it?
How do you communicate with it?
How do you collect it?
How do you post-process the data?
In each of those critical areas, we see that learning from the past is critical to avoid mistakes that could be avoided with these learnings.
Teamwork is an essential part of the class.
Because any great system or product is really the result of a teamwork, where in a team you have to balance essentially your own ideas and your own desires with what your teammates think.
And the dynamics of successful teams show that diversity is important, being able to openly share ideas, also the ability to compromise and not always getting one's way.
What we found over the years-- and I think this is also supported by research and scholarship in design-- is that the optimal team size is about five.
Four to six, maybe up to seven people is the ideal size of teams for this kind of project.
And why is that?
If you create a smaller team, you have less diversity of ideas.
And you also have less man or brain power, of course, to get work done.
Since once you decompose the project into its elements, each of these elements needs to be done carefully.
So you want to have a team that's large enough to do the work within the time constraints.
On the other hand, if the team is too large, then there's a lot of coordination, there's a lot of overhead.
Just being able to meet whether it's physically or online becomes harder.
And the efficiency of the team goes down.
And so we found that a team size of about five people is ideal for this kind of a project.
One of the first assignments that I ask students to do in the class is to write a team charter.
Now what is a team charter?
You can think of a team charter like a manifesto.
It's essentially defining the identity and the goals that the team wants to pursue.
And in this kind of class, there could be many different goals.
For example, as a team you want to learn as much as you can about the process of systems engineering and design and so forth.
Another charter could be that you are very ambitious and you want to win the competition.
So you want to score as high as you can.
And the ultimate goal is to be as highly ranked as possible in the competition.
Another goal could be just to have fun, you know.
To just have a good time, to learn, but essentially to really be together in this learning environment, to learn and have fun at the same time.
And so the flavor of each team is a little bit different.
And rather than just letting it happen without much thought or discussion, the purpose of the team charter or project charter is to encourage these discussions early, such that the team is on the same page as they proceed into the project.
Assessing student work as a team has different challenges.
Of course, the first thing you do is assess the output of the team's work, whether that's a requirements set, whether it's a set of initial concepts, whether it's a presentation that's given at a preliminary design review, whether it's a physical prototype that they present as a team-- so that always should be the first thing that is assessed is, what is the net output that the team has produced?
Within that, it is not always easy to discern what has been the individual contribution since the whole purpose of the team is that the sum that comes out is greater than the individual parts.
So in the best-functioning teams, there is a magic that happens.
There's a synergy that happens.
There's an output that's only produced by these synergistic interactions of all the team members.
In a sense, trying to dissect, what's the individual piece that everybody did, is countering this idea of team synergy.
That being said, we also need to assess individually the contributions and the learnings of students.
And we do this through a couple of mechanisms, for example, a peer review.
So you can do a peer-review process.
It's trickier in smaller teams than in larger teams.
But if it's done properly, peer review can be very effective in not only helping one understand, who are the contributors?
And who are the free-riders?
This is always a big issue in teams, is that the free-rider syndrome, where some team members contribute significantly less than others.
And this often can be a source of conflict.
And so understanding this is important.
But also, have other mechanisms in the class, such as an individual final exam, and then the other written exam, which is administered online.
And the other thing we started doing is oral exams, which seems old-fashioned.
But I have to tell you, in 15 to 20 minutes of a one-on-one conversation with a student at the end of the class, you really learn a lot.
You learn a lot about what they've learned, where they may still have confusion or misconceptions, and also the feedback that they give you.
So I highly recommend to do oral exams if it's possible.
Concept questions are a fairly recent instrument for checking the understanding of students in real time during the class.
One way to administer concept questions is using clicker systems.
They've been around for several years.
A concept question is typically a very short question that requires either reflection, or some very simple calculation, that can be done in less than a minute.
The concept question is intended to surface misconceptions, but also alignment and understanding.
And I do administer the concept questions using, in this case, Google Forms.
But there's other ways to do it, essentially providing the students a very short URL-- a link that they can enter either on their laptops, even on their smartphones or on their tablets.
And usually, a concept question is a multiple choice question.
Sometimes, it's very easy.
Sometimes, it's quite tricky.
And I give the students about 30 seconds or so to think about it, work out an answer, respond online.
And then, once the answer period is over, I display the responses in real time.
And what's nice about it, in the context of a SPOC, it doesn't matter where the students sit, whether they're with you in the classroom, whether they're online.
Everybody has the same link.
So you're getting everybody's responses at the same time.
So it's a great mechanism for engagement.
Sometimes, everybody agrees-- 100% or 90% agreement.
Everybody chose the right answer.
And in that case, there's not much discussion needed.
Because it's clear that it clicked.
You know the concept is well understood.
My favorite concept questions are the ones where the answers are all over the map.
You have five possible choices, and each of them got close to 20%.
That's great, because now, you have a hook for discussion.
So I would then, if that happens.
spend the next three, four, five minutes in the class reiterating the concept, and then asking students, who voted for A?
Who voted for B?
Why did you do that?
What was your opinion?
And so you get a great discussion going, which, again, can have the in-classroom, and then the online component, as well.
We also use the chat during those post-concept questions.
And it's a great way to activate it.
And there's a second reason I also do concept questions, at least one during lecture, which is, it's a great way to track attendance.
And I'm up front with the students about this.
Because you can see the time tag when students responded.
So you know exactly who responded to the chat during the class.
And I do use that for participation grading.
So at the end of the class, there are really two major forms of assessment.
One is an online written exam, which is an exam where the questions and the way to submit the answers is fully online.
And that link is unveiled, and then I give three to four days for the students to answer this exam.
It is an open book, open internet exam.
They can use any materials they want, but no collaboration.
So I don't have a formal way to check for this but essentially it's an individual exam.
And by submitting the answers online, again using for example a mechanism like Google Forms, you can relatively quickly compare the answers and grade these exams very efficiently and also see if there's any patterns.
So this is a way to essentially check individual learning with all the materials at hand.
The way I structure the online exam is a mix of multiple-choice questions, essay questions, but also some calculations.
And the exam starts with some very easy questions that can be referenced in readings or materials in textbooks, interpretive questions.
And then they get progressively more difficult.
And the last two questions-- in one of the questions, I give them a calculation that they have to do that has a very non-obvious answer.
And it's interesting to see some students that force an answer that's not actually the right answer because somehow they believe that any question given on an exam must have this one correct answer.
And other students scratch their heads and say, I don't think this question has the right answer.
And so it's a very interesting thing because, in the real world, you often get requirements from customers that are unrealistic or can't be fulfilled.
And so the purpose of these questions is to really get students to think and connect the material to real-world situations that they will encounter.
In the last question, I then ask about systems engineering-relevant things that have happened that they might have seen in the news.
So for example, one year I asked about the Volkswagen emissions scandal, which is very related to system engineering, trade offs between fuel efficiency, cost, NOx emissions.
It's very much current affairs.
This year, I asked the students about the problems that Samsung experienced with their Galaxy Note 7 batteries and what is the root cause of those problems.
And how are those problems related to the material that we teach in class?
So I recommend that one of the last questions connect the students with things that they're seeing in the news.
The oral exam is given either in person in the same room or through an online platform like Skype or WebEx or Google Hangouts.
The quality of the online experience, the image, the sound, is generally so good that there's virtually no difference between doing the oral exam in person or over the web.
And the oral exam is anywhere between 15 to 20 minutes per student.
That doesn't sound like a lot of time, but with well-structured questions you can learn a lot and have a great dialogue with the students.
I try to put the students at ease.
It is more a discussion than a classical exam.
Nevertheless, it's a great way for them to engage with the material again, to engage with us as instructors and TAs, and usually it sort of puts a nice bow on the class, and it's a nice finishing touch to the class.
Logistically, the way we do the oral exam is we open up typically one or two full days.
So I have to reserve a couple of days on my calendar just dedicated to the oral exams.
And then there's sign-up slots where students can just sign up online and show up at the right time.
And it's been working pretty well.
The personal reflective memo is a two-page document that I ask the students to write at the end of the class but before the oral exam.
So all the lectures we've had, all the team assignments, and all the material has been presented.
And at this point, this reflective memo asks the students to think back on the material.
What really was the most important thing they learned?
What of what they learned in the class are they going to build into their own future projects, their own careers?
If they're interested in systems engineering as a career choice, whether it's in research or in industry, why is that the case?
And so they submit this one or two days before the oral exam, and so we have a chance to read those reflective memos.
And usually, one of the very first questions in the oral exam is about some key ideas that they presented in their oral memo, so it provides a great hook into the oral exam.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
You have to make some design decisions.
And then once you've made those design decisions, then you can actually write your level 2, level 3 requirements and so forth, it's an iterative process.
But when you look at the classic v model, you don't really see those iterations.
But they actually exist.
So what we typically do at the system requirements review is we review and then agree on the high level requirements, level 0, level 1, maybe level 2 requirement, but not the lower level ones, because you can't, because you typically haven't done the design yet.
The other thing I want to say is a lot of the-- this is really a critical issue.
And a lot of the problems we talked about last time in the schedule, cost overruns, really, a lot of these things are traceable back to the requirements.
Either the requirements were over ambitious.
You can set requirements that were essentially unachievable.
Or you missed requirements that actually became clear later on.
But they weren't written.
So nobody paid attention to them.
A couple of examples.
So I have sort of one example of core requirements, and then a good example.
So this is a mission that you may have heard about.
A 1998 MCO.
So who was born in-- I'm looking at you guys.
Justin, right?
Justice, when were you born?
'94?
OK, so you're older than MCO by four years.
So MCO was a very well-known just about not quite 20 years ago, a very well known mission failure that happened.
A Mars climate orbiter, it was launched on December 1998.
And it had multiple functions study the Martian climate, weather and surface changes.
And it was to act as a relay satellite for the Mars polar Lander, which came after it and also failed.
Both failed.
And unfortunately MCO burned up.
We think contact was lost with a spacecraft after it basically entered the Martian atmosphere.
And you know nothing-- there was no more communication.
So we don't know for sure.
But their best guess is that the spacecraft burned up in the approach to Mars, that the actual altitude at which had entered the Martian atmosphere was about 57 kilometers.
And it was supposed to be about 220.
So even though the Martian atmosphere is very thin, when you enter at very high velocities, that's not a good thing.
This is the very famous-- you've heard about this-- confusion of units problem, like this mission that one part of the team used SI units, specifically Newton's seconds, this is momentum, right?
Force times time, that gives you momentum.
So when you burn your engines for x number of seconds the sum of the product of those two is your momentum.
And then another part of the team used English units, pounds of force times second.
OK, and so that was fundamentally the problem that caused the burn up.
Now if you read the accident report.
And I have a link here to the accident report, you will notice that the requirements were actually OK.
The requirements were written correctly.
So specifically, there was a document called the software interface specification, SIS, that specified everything should be in SI units on the ground segment and on the space segment.
But the problem is that this was not checked and implemented.
And so here's a quote from the accident report.
Items that the mission assurance manager, which didn't exist for this mission.
So the mission Assurance Manager role is to make sure the requirements are actually followed.
There was no mission Assurance Manager on the MCO mission.
Included ensuring that the AMD file, which is one of the files put out by the propulsion system, met the requirements of the softwares in the interface specification And that did not happen.
And so that was sort of the root cause there.
Here's a good example for requirements.
This is a little bit older.
This is the DC-3.
This is an old plane.
First flight in December 1935.
This was this was really an airplane that kickstarted civil aviation as a commercial service.
And so first of all, the DC-3 was based on an earlier evolution of the DC-2.
And I first saw this when I worked in St. Louis.
There was a little museum at the headquarters.
And they had like a, like a nice plaque on the wall with actually a replica of the contract.
You know, the original letter that left the DC-3 contract.
And the story is that this-- these requirements-- these are-- this is level 0 requirements, what's written here, was hammered out in a single quote, end quote marathon phone call between Smith and Douglas.
So Smith was, at that time, the head of what was today essentially United Airlines.
And Douglas, who by the way, is a graduate of this department here at MIT, Donald Douglas, was the head of the Douglas Aircraft Company.
And here are the requirements: range about 1,000 miles.
So this means that you don't have a trans-continental range, right?
If you want transcontinental range in the US, you need about how many miles?
Like Boston to San Diego, Seattle to Miami.
3,000?
Yeah, reserves.
Yeah, 3,000 miles.
So this means that, if you go transcontinental with the DC-3, you've got to refuel twice, basically.
But you know, that was OK. Today, we wouldn't-- we probably wouldn't accept it.
But that's OK back then.
Cruise speed about 150 miles.
20 to 30 passengers, depending on the configuration.
Twin engines.
And then something that we would say today.
That's a fuzzy-- that's a not a well written requirement, because it's fuzzy.
But it said, it should be a rugged and economical.
And based on this sort of high level requirements that were pretty clearly defined, the airplane was then designed, built, and very, very successful.
And over 10,000 copies of this airplane were built.
So what happened since then is the requirements explosion.
This is a chart from a document called the technical multi-disciplinary design optimization white paper.
And it's a little dated, but the basic message here is pretty stunning if you think about it.
So first, heavier than air flight, 1903, the Wright brothers, right?
1G flight.
What is the requirement for 1G flight?
Get off the ground.
Get off the ground.
And stay off the ground for at least a few seconds, right?
Not just ballistic.
That's the requirement.
Get off the ground.
Heavier than air flight.
Yeah, big success.
Big milestone.
Well, pretty clear after not too long, it was well-- we also want to turn.
We don't want to just fly straight on the beach and land again, so maneuvering, gust acceleration.
There are winds buffeting.
You've got to handle those gusts.
So that that's important.
And then what were the first airplanes made of?
Let's see an EPFL, guys.
What's the first airplanes?
What materials did they use?
[STATIC] [INAUDIBLE]
Wood
Wood, yeah.
What else?
Canvas
Canvas
[INAUDIBLE]
Exactly.
so then very soon after that, maybe in the 20s and 30s, we started using metals, right?
Metallic structures.
And the big issue there, especially if you're flying close to the ocean and salt water, is corrosion control.
Your airplanes can't rust.
And so a corrosion control became important.
And pressurization, what is that all about?
Higher, faster.
So at some point, you need to pressurize the cabin, right?
That's a new requirement.
The Wright brothers never thought about that.
Well, they didn't-- maybe they thought about it.
But they certainly didn't have to pressurize their Wright flyer.
So you see you see what's happening is we get greedy.
You know, we were excited.
We can fly now.
But now we want to go higher.
We want to stay longer.
So we get greedy.
And we go to more extreme environments and so forth.
And as we do that, it get harvested do and more and more requirements start piling up.
And you see a big step here, World War II, right?
Handling quality, radar transparency.
Radar was invented.
So now you want to have airplanes that are not visible on the radar.
Fatigue, rough field landing.
And then we have another big step here in the 60s and 70s.
This was during the Cold War, smart weapons, nuclear, fly by wire, right?
Replacing cables with electronic flight controls.
And then a lot of the "ilitys" in the last 20 years, reducibility, affordability, portability, et cetera.
So if you actually look at the requirements set for the new generation of airplanes, whether commercial or military, it's overwhelming.
I mean, it's thousands and thousands of requirements, because we've gotten greedy.
And we've gotten good at it.
So we keep adding more requirements.
And that's a big issue right now.
And we'll talk later in the semester about complexity management.
But the key message here is requirements have been growing over time.
More and more requirements added as systems grow and in performance and complexity.
So here's some standards.
I won't go through those in detail.
But people recognize, in the system engineering community, how important it is, these requirements.
So in the system engineering handbook, there's two sections, 4.2, which is about technical requirements definition, and then section 6.2, which is about requirements management.
So definition means that's the initial definition of the initial requirement.
And then management is the process of updating them, adding requirements, modifying them, making sure they're up to date.
And there are a couple of appendices.
INCOSE, the International Council of System Engineering has, in the handbook, a whole section.
There's even a requirement working group, people who really specialize in this, and then as well in the ISO standard, you have a lot written about requirements.
So this is really a big deal.
So the last thing I want to do here in this section is to sort of, at a very high level, communicate to you what requirement really are about.
What they are about is, like I said, don't set off in your ship without-- you don't know what port you're sailing to, right?
So requirements set goals and constrain the design in the objective space.
So whatever you're designing, you're always going to have two spaces that you're dealing with.
The design space, on the one hand, that's the things you can choose as a designer.
These are the knobs you can turn, the decisions you can make.
And then the objective space, which is the things that, essentially, your customer cares about.
I'll give you a quick example here.
So when we use the word shall, which I'll get to in a minute.
The English word, shall, means this is essentially a constraint.
You must accomplish this.
And when you use the word, should, it's more like a goal.
It would be nice to do this.
But it's not absolutely mandatory.
So shall is a hard constraint.
Should it is desirable as a goal.
So let me give you this quick example here.
So let's say you're designing a house.
You're about setting off to design a house.
And what would be some requirements for a house.
So I wrote four of them here.
And I'll map them to the design space and the objective space.
First requirement, the house shall sleep between four and six people.
Well, is that in the design space or in the objective space?
That's in the objective space, right?
Yeah, that's right.
So here, we have an axis called occupance, right?
So four is the minimum and six is sort of the maximum.
So that's the upper bound.
The next one, the total build cost should be less than $550,000.
Should be.
What is that?
Maybe an
It's a goal
It's a goal.
Yup, that's right.
And is it in the design space or the objective space?
The objective space
That's correct.
So I put this as a dashed line, OK, because it's a should.
So what I've done here, just by writing these two sentences, these two requirements, is I've essentially carved out a space, right?
And I can shape that box.
Basically, what these two requirements do is they put a box around in the objective space saying that, whatever house you're going to design, it has to fit within this box.
Then the third one is the house shall have at least at least three bedrooms.
What is that?
In the design space or objective space?
Design space, right?
Because defining a bedroom is a design decision, right?
And the house should have a fireplace.
So you can show the lower bound here, at least three bedrooms.
And then the fireplace is a-- it would be nice to have.
But you don't absolutely have to have it.
So you can sort of draw this-- it's more like a line here, right?
Yes, fireplace.
But it's a dashed line.
And then at least three bedrooms.
So just these four sentences, we've now put a-- we've defined the space that we're going to be designing in.
So we have constraints in the design space.
And we have constraints in the objective space.
And that's fundamentally the role of requirement is to constrain, to give us direction as to what we're going to design.
OK, so let's do our first concept question for today.
And that's the following question, do you think there's a fundamental difference in the meaning between the words, requirements, which we've talked about so far, and then the word, specifications, which I haven't mentioned yet.
So no, you think they're essentially the same.
Yes, there is a difference.
Requirements are like the input, and specifications are the output of the design process.
Yes, you think a difference.
Specifications include the requirements as a subset.
Or you're not sure.
So think about this.
And then submit your answers.
And as far as these URLs are concerned, for the concept questions, I went to all lowercase.
Hopefully, that's a little bit easier.
So secc3.
tiny.cc/secc3.
What do you think?
Requirements and specifications.
The same thing or not?
Who needs more time?
Anybody?
OK. OK, so we have nobody thinks they're the same.
62 thirds of you think that requirements are the input, specifications are the output.
9 of you think that requirements are a subset.
And then 6% are not sure.
So that's good.
I agree with that.
So a lot of people use requirements and specifications as the same thing.
They're really not.
OK, so think of requirements and you haven't actually started to design that.
This is your putting constraints in the design space, defining the direction.
But we don't actually know what the design will look like.
You may have some requirements may say you must use this box, or you must use this sensor, right?
And you might actually find some of those in the CanSat.
But basically the requirements are the input.
So any-- What are your experiences?
When have you heard those terms, requirements or specifications used interchangeably or not interchangeably?
Has this come up as an issue for you before?
[?
Marissa, ?]
you're worked at-- you're worked on human spaceflight, right?
Space tourism vehicles.
So what-- how did you guys do it at Virgin Galactic?
Well, I think something that you generally saw was like people will look at sensors.
And they would-- there's a specification for a sensor.
And they would not necessarily compare the requirements to the specification.
So or they would not understand how the two intersected.
So I think there was a bit of a misunderstanding of the difference between what the actual requirements were, versus what the specifications were designed to be
The sensors you're talking about would be sensors, I'm assuming, that you would purchase, right?
As commercial--
Yeah, absolutely.
So I think we were buying of the shelf.
And so I mean, sometimes the specifications were broader than what we wanted and sometimes they were narrower.
But you know, you can do additional testing on it.
So there was sometimes a breakdown between understanding really whether the sensor met what we are looking for or it didn't
OK. Good.
What about an EPFL.
Any experiences?
[?
Walker, ?]
you worked on slip rings, right?
Slip rings was one of the big specialties of your company.
I'm assuming that you guys had to make that distinction, right?
Between requirements and specifications.
What are your thoughts at EPFL on this?
Well, yes, from outside, we clearly had a separation also on an ether level.
But I was wondering here in the class, has anybody seen the difference between specification requirements?
Any comments?
Maybe some of the PhD students that have been around the block?
[INAUDIBLE]
Yeah, OK, great.
So I found here, I, you know, I wanted to sort of find a real example that everybody can relate to very, very easily.
So the answer is there is a distinction.
They are quite different.
But they're-- got to be careful, because you can mix them up.
So requirements specify what the product or system should-- shall or should do, right?
Functions it shall perform, how well it should perform these.
Also maybe the degree of autonomy, how automated is the system?
So what the operators must do, when the system is operating.
And also compatibility with other devices.
And then specifications are about how the system is built and how it actually works.
So the form, right?
What the materials that are used, the dimensions, schematics, blueprints, the details of the user interface.
Those are things-- those are all specifications.
So I try to look up a very simple consumer product that we all are familiar with, a microwave oven.
Kenmore Elite Countertop 2.2 cubic foot.
You go to Sears website.
And what's interesting is they actually do a pretty good job.
It's actually pretty consistent with what I'm saying here, except they don't talk about requirements.
They call it description.
So what's listed on their description.
And this is, I'm pretty much quoting here, verbatim.
This microwave is large enough to accommodate the big dishes, right?
So the idea is this is sort of family style, right?
This is not a small microwave just for a frozen meals.
This is-- you should be able to put a big casserole in it.
And reheat meals and so forth.
So the idea that what are the things you're going to use in the microwave.
What's the use case, right?
CONOPS.
CONOPS for the microwave.
You know, family, for kids, both parents are working, busy, you know, not time to spend two hours preparing dinner.
Therefore, big dishes, a lot of people.
It has to be quick, right?
That's a CONOPS.
And the requirements, this description, is essentially the requirement.
1,200 watts of power to reheat food quickly.
So time is important.
And then one touch settings for different food types, rice, pizza, frozen meals.
And that, what is that?
That's automation, right?
Rather than having to guess how long and at what power level to reheat, there's some partial automation built in.
That's essentially requirements, very user centric.
Do you see that?
And then when you look under specification, it says the following things: stainless steel exterior, right?
Again, you don't want it to rust.
The dimensions, the weight.
There's a general warranty one year.
The power cord is included.
It's very different.
Those are things that describe how it's made, the form of it, and so forth.
So make sure you keep those separate.
OK, so let me talk about the NASA requirements process.
And then we'll talk about challenges.
So getting back to the system engineering engine.
That's sort of the heart of the NASA system engineering process.
You remember, this is the-- this happens at every level of decomposition.
After stakeholder expectations.
The second step is technical requirements definition.
And there's some pretty strong language in the handbook that this really has to be done well.
So the center directors or their designees shall establish and maintain a process to include activities, requirements, guidelines, documentation for the definition of technical requirements.
So I think I've already said this, but I just want to make it clear again, this is kind of-- I like this cartoon.
This is Moses, right?
Up on thee-- Moses just got the tablets, right?
The stone tablets.
They-- the 10 commandments.
And one of them is, thou shalt not steal.
And God says, out of the clouds, no, they're requirements, not goals, right?
So what God is saying is that the 10 commandments are, shall, right?
And not should.
Even though, I think, some of us don't always succeed at that.
But that's basically the idea.
Shall is a very hard constraint.
And that's what you will be judged against.
And should is essentially a desirable goal.
But the degree of attainment is somewhat flexible.
Why are we doing this?
Why are we spending our time writing technical requirements?
It's essentially to transform those stakeholder expectations we talked about last time, transform them into measurable technical requirements.
Requirements come in different flavors.
And I'll mention those flavors.
And we express them in these shall statements.
They also provide a basis for agreement among the stakeholders and developers.
So you will often find requirements part of contracts, right?
Now it becomes serious, you know when the requirements set actually has legal implications.
You're legally signing up to develop a system that will meet these requirements.
That's pretty serious stuff.
By writing good technical requirements, you can reduce the development effort, because of less rework.
So a lot of rework and iterations and confusion is based on missing or poorly written requirement, right?
And writing those early and before the design begins is helpful.
A requirements set is also a basis for cost and schedule estimates.
And so some of these missions that overran.
Do you remember I showed you this chart last time.
We analyzed 40 missions, earth and space science missions.
Like 20% of them were responsible for 80% of the overruns.
A lot of that had to do with unrealistic goals or expectations and requirements.
And then the next point here is that the requirements provide the basis for verification.
In other words, the better, the more crisp, the better written your requirements are, the easier it is to test the system and to check whether these-- whether the system is in compliance with the requirements.
And then, eventually, the basis for acceptance.
Acceptance is also a big deal, right?
It basically means, I accept your design.
I take ownership of it.
And I say, you, as the developer of the system, have done your job properly.
And there's a legal transfer of the assets that happens.
So facilitates the transfer of the product to the users.
And then even later, if you're going to do it like a version 2 or a block upgrade, a next generation product, it's much easier to do when you have a clear requirements, because it tells you what the original system or the earlier version was able to do or not do.
Graphically.
So this is figure 4.0.1.
I briefly talked about this last time.
We start, essentially, here at the development.
We have mission authority.
We do the stakeholder expectations.
We talked about that last time.
And then here, right away, comes a high level requirement.
So that means your level 0, level 1 requirement.
And then as you try to get more detailed in the requirements, what do you think happens?
Ideally, you want to write all the requirements upfront, right?
That would be great.
But you try to do that, you hit a wall.
Why is that?
What's the issue?
Why can't we just write all the requirements and then be done with it in one shot?
Yes.
Go ahead
Well, your system changes over time.
And some of the requirements clash.
And you can't achieve all of them simultaneously
Yes.
so there's two things you mentioned.
So one is the sort of-- and I'll talk.
This is known as, requirements volatility, like the requirements are changing as you learn more about the problem.
And then the other is you detect conflict between requirements.
And you have to clean those up.
Those are two very valid issues.
But it's not quite what I was going for
There's another one, requirements creep, in which your customer levies these new requirements on you, once you've started the process
Yes, so new requirements get added.
Hopefully, you get more budget, too.
That doesn't always happen, right?
So that's another issue.
But it's a little different.
Let's see, EPFL, you guys.
Why don't we write like all four or five levels of requirements all at once?
Why can't we do that, typically?
Why do you think?
Usually, the other requirements aren't clear.
They don't exist yet.
Later on in the process, they become clear.
And you start to realize the details, which at first, you don't
That's-- I think that's pretty close to what I'm looking for.
So the issue is you do your level 0, level 1 requirements.
And then sort of as you get to level 2, you can't really write that level 2 requirement until you've made some key design decisions.
Are we going to use electric propulsion or are we going to use chemical propulsion?
You know, that's a huge decision in space system design.
And unless you've made that decision, you can't really write lower level requirements, because the fundamental working principles of chemical and electrical propulsion are quite different.
So that's the key issue is that you can only do the high level requirements in a solution neutral space.
And then you hit the wall, because you've got to make some key concept technology selection decision.
And that's shown here by this red box called functional and logical composition.
I don't like that nomenclature a lot.
This should really say, system architecture or concept selection, which we'll get into in the next couple weeks.
Once you've chosen a high level architecture and concept, then you say, OK, we're going to go for ion propulsion.
Well, then you can write the requirements for the ion propulsion system, which are we through the find in this yellow box here.
So this is your design and product structure, derived an allocated requirements at lower levels.
You see that?
That's the fundament-- all the things you said, requirements creep requirements conflict.
All those things are true.
But the fundamental reason why we can't write all the requirements upfront, because, at some point, the lower level requirements depend on design decisions made.
That's the fundamental issue.
OK, any questions about that point?
Yes?
It's not totally about that point.
But it's still like-- So requirements versus specification.
So if you're like buying a component from a vendor, some piece of hardware, and you want identify like the specific locations where it should attach to things, it's more of like a form as opposed to really-- so does that fall into requirements or is that like a specification?
Great point.
So there are-- what you're describing is a peculiar type of requirement that we call interface requirements.
And it's a perfect segue to this next-- you know, if I could only include like I don't five slides in this lecture, this would be one of them.
This basically is what are the different flavors of requirements or types of requirements.
And there are six of them here.
So first of all, the functional requirements.
I think we've sort of talked about those, right?
Define the functions that need to be done to accomplish the mission objectives.
There's some examples here around thrust vector control.
So basically, the idea here is that you have a thrusting system.
And it has or you can actually direct the thrust.
And in this case, you should control the thrust.
You shall provide thrust control around pitch and your axes.
So this statement is a high level functional statement.
And it's written in the actor verb, object form, right?
So that's sort of the classic, you know, the classic requirement and the functional requirement.
And then we have performance requirements.
The performance requirements are, in a sense, qualifiers on the functional requirements.
So a performance requirement will specify how well the function should be performed, how fast should it fly, how much thrust.
You know, this is where you have to actually put numbers in.
So in this case, this thrust vector controller shall gimbal the engine 9 degrees, right?
That's the deflection angle, at least 9 degrees, plus or minus 1 degree, degree.
0.1 degree.
That's the performance requirement.
Then we have constraints.
OK, yeah?
Isn't the performance department saying that the engine has to gimbal 9 degrees, isn't that kind of pushing toward the specification.
Or is that?
I mean, I guess, is can also be a requirement.
But isn't that?
Right, in this case, we have a set, and we're not specifying how those 9 degrees are achieved, you know, whether it's through a gimbal or the actual mechanism or how it's not described here.
All that's described is the angular [?
range, ?]
essentially, that this.
And that is a performance requirement.
But you'll see, there will be some examples of things that are putting limits on the form, which looks more like a specification.
So then we have constraints.
Constraints are things on like weight, mass, power, things like that.
So constraints are requirements that cannot be traded off with respect that cost, schedule, or performance.
So for example, the prospector controller unit module shall weigh less than 120 pounds.
So that's a-- that's not, if you think about it, that is not functional, right?
The weight of the thrust vector controller shall not be more than 100-- that is not performance.
It's not functional requirement, but it's a constraint on the form, essentially.
So this looks more like what we would call a specification constraint.
The fourth one is what [?
Marissa ?]
brought up.
This is an interface requirement.
OK, so in this case, our thrust vector controller shall interface with the J-2x The J-2x is a very famous engine.
And J-2x is a kind of-- this is sort of, this was written during the constellation days of the constellation program at NASA is the idea of a new generation of the J-2 engine.
So the idea is that whatever you do, however you design your thrust vector controller, it must be able to interface with the J-2x engine, according to conditions specified in this interface control document.
So this is called an interface requirement.
Then the fifth category are environmental requirements.
So the TVC shall use [?
Biber ?]
acoustic shocks and loads according, again, to some environmental document.
And by the way, for the CanSat competition, we have that, right?
There's two documents.
There's the mission guide.
And then there's this environmental testing guide.
So what this essentially says is under what conditions shall the performance, and functional-- the functions and performance that are specified in the first two type of requirements, under what conditions shall that be performed, right?
And if anything you think about it, designing whether it's a sensor or anything to operate between 0 degrees Celsius and 30 degrees Celsius or between minus 60 and plus 80, is a huge difference, right?
You can write that down.
But what that actually means to open up the range of environmental conditions, it has a big, big impact on the design.
And so this is a big deal in practice.
And then, at least in Space Systems design, you know a lot of these environmental requirements are of course, driven by the space environment or the launch environment.
If you're designing airplanes, whether you're flying in a small mountainous country like Switzerland or on the ocean off of an aircraft carrier, those are very different environments.
And they're going to influence.
You have to specify the operating requirement.
The bigger the envelope that you make for the environmental requirements, the more complex the system will be.
Also medical-- anybody working on medical devices here?
EPFL.
Anybody?
Medical devices?
So in medical devices, it's the same thing.
Is this medical device going to be used in a hospital setting, where everything is kind of clean.
You have clean power.
The nurses, everybody is very well-trained.
That's one thing.
Or is his medical device going to be used in the field, you know?
In Africa, in India, in a rural area, where people are not, maybe not trained, medically and professionally and so forth.
Very different.
The function may be the same.
But the environmental conditions are very different.
You have to really specify those.
And then we have-- the last category is kind of the other.
It's sort of a catchall.
But it can be very important.
So this includes a lot of the human factors, reliability and safety requirements.
Those are-- and you know, they're listed here as other.
And I think it's really important to say that just because they're listed here doesn't mean they're less important.
These are often neglected, unfortunately.
And it really can hurt you in the long term.
So pay attention to those human factors, reliability, and safety requirements.
So six types of requirements.
So let's talk about what makes good or acceptable requirements.
And there's a distinction here between a single requirement statement and then sets of a requirements.
So first of all, requirements should be written in natural language.
They should be complete sentences.
And each of the requirement statements should be clear and consistent, meaning that it's not a novel or it's not a poem.
But it's clear.
It's understandable.
It's well-written.
It's correct.
There's no errors in it.
It's feasible.
Now that's a really tricky one.
That's the first thing here that's really tricky, when you think about it.
Feasible means, what?
It means this requirement can be satisfied within the laws of physics and state of the art technologies and other project constraints.
So why is that a tricky one?
Go ahead
I was going to ask a question regarding that, in terms of how do you deal with a program where you're working on things or you're trying to actually maybe define the state of the art or figure out what is feasible
Right.
so if you're basically designing a product or a project or a system that's a repeat of what's already been done, then you can have pretty good confidence, right, that those poles are feasible.
But what if you're doing, let's say the-- I think I mentioned this last time.
You're going to Europa.
And you're going to drill through the ice.
And you're going to explore the ocean under the ice in Europa.
It's never been done before.
You know, can we actually do this?
So this is the tricky thing.
How ambitious, how ambitious can the requirements be and you still claim feasibility?
And that's also one of the big reasons why programs get in trouble is when they're actually defining requirements that are not really feasible within-- they're definitely way beyond the state of the art.
And within the time frame and budget allocated, you can't get there.
And many, many programs that run into this problem are the ones where the technologies that you're going to use are not really ready yet.
You know, there-- we'll talk later about technology readiness levels scale.
You're not really ready yet to do this, but you're going to try anyway.
So that's a tricky one, feasibility.
Flexibility is, you know, don't over specify how things should be done.
So don't say how it should be satisfied.
Without ambiguity.
That means if 10 people read this requirement, they should have the same or very similar interpretation.
Singular statement, one actor [?
verb ?]
object.
And then the last point here is verifiability.
How are you going to check whether or not this requirement will, in fact, satisfy you
With regard to that, the feasibility again for some of these programs I had large, [?
overrun ?]
schedule, overruns.
I mean, would it be almost better in a sense to have some of the requirements be more shalls, like-- or goal.
In other words, you know, if you can do this, go for it.
But if you're going to run over 10 years, maybe back off a couple of percent
That's right.
And what-- we'll talk about this a little bit.
But if you-- all of them are shall statements.
They're all hard constraints.
The objective space that I showed you may, in fact, have 0 feasible space.
So really knowing where, what is a hard constraint, what is a hard requirement, and what is flexible, that's very tricky.
And that's why it's important to actually not just accept requirements.
You know, if you're going to run a project, every requirement, you want to really understand it.
And if you think this requirement is infeasible, you have to negotiate.
That's where the upfront negotiation becomes really important.
Did we lose our EPFL?
Hello?
Go ahead, please
[INAUDIBLE]
Please, go for it
So in 1990, we got the requirements from NASA to build a space [?
bioreactor.
?]
And all the students will love this.
It was the smallest brewery that ever flew in space.
And the initial requirements was that all fluid containers had to be solid with double walls.
And this meant that we had another solution with bladders.
[INAUDIBLE] bladders.
So we had a real fight up front in the proposal phase to demonstrate the compliancy to the requirement of going the cell culture, yeast cell culture, and still being non-compliant, actually, to this requirement of hard double-walled fluid containers.
And so this is exactly [?
the point, finding these ?]
[INAUDIBLE] since the last 15 to 20 years.
And this was a breakthrough in technology.
And there, you will always have these challenges with the agencies, with your customers.
They are [INAUDIBLE] set in their ways.
And you have to demonstrate the feasibility to them to make them change the requirement that allows you to change the specification
No, I think that's a great example.
So in the end, you were successful to argue for the bladder the bladder solution
Well, the bladder was just how to keep the fresh medium and the used medium, meaning the beer.
The technology was to put in [INAUDIBLE] And actually, it needed to have flexibility on the pressure in the reservoir.
That's why the argument to put [INAUDIBLE] in space finally won the day, and allowed to relax the other requirements
Yeah, thank you, thank you.
A great example there.
Any other comments?
[COUGHING]
OK, so so all this is a pretty long list.
And this applies to single requirements.
And then there's a set of-- this is really important.
Then there's a set of characteristics that we want to see when you look at sets of requirements, groups that require.
Absence of redundancy.
This means that each requirement is specified only once, right?
You don't want to have redundancy.
Redundancy can be good in system design, but not in the requirements.
Consistency, using terms.
Completeness, this basically means not missing key requirements.
And then this idea of absence of conflict.
And this is also similar to the feasibility.
This is a tricky one.
Requirements can be in tension with each other, particularly their should goals.
But they shouldn't be in direct conflict with each other, like you know you shall use aluminum for this unit.
And then another requirements says, no metals are allowed to be used in this unit.
You can't.
That conflict is not solvable.
That's a direct contradiction.
That's different from having competing requirements or requirements in tension.
So let's do a quick exercise.
And then we'll actually take a break, as well.
So this is a turn to your partner exercise.
And then we'll have a break.
And we'll restart in like about seven minutes.
So here's what I'd like you to do.
I have three systems here.
They're very different in scale and complexity.
So A is sticky tape.
And I'll tell you this quick story here.
I grew up in Switzerland surrounded by farms, you know.
And I spent all my time at the farms and you know, big tables, big families.
And this is-- they have these things here.
This is basically-- these are flies sticking on tape.
This is to keep the flies off the dinner table.
It's called Mr. sticky tape for trapping flies.
Really kind of gross.
But I remember it.
B is I just a couple of weeks ago had a test drive and the new I3.
This is the BMW small electric city car.
Very cool.
And then C is something that at EPFL, you guys know very well.
This is the Rolex center.
It looks kind of like Swiss cheese, when you look from the top.
This is the equivalent of W-20 here at MIT.
This is the student center.
So there's a library there.
There's a cafeteria there and so forth.
So what I'd like you to do is turn to your partner, pick one of those three, and come up with one single statement, one good requirement that you think was possibly used in the development of that solution, whichever one you pick, OK?
So pick one of those three.
And then jointly discuss and write a requirement that you think led to this design, OK?
So take about five minutes.
Take a break.
And then we'll sample what you came up with.
So let's hear from EPFL first.
did anybody do A?
We did
OK, go for it.
Speak up
We had four requirements for
OK, go for it
All right.
So The tape shows [INAUDIBLE] of the tape should be less than 4 grand.
The tape should be able to catch up to 60 insects.
And the sticky tape should not be toxic for humans
OK, I like, yeah, the toxicity.
That's good.
Excellent.
Very good.
ow who did A here at MIT.
Anybody?
A?
Go for it
We said if greater than 10% of the surface area of the fly contacts the paper, it shall not be able to release itself
Ah, I see.
So this is a trapping requirement
Yeah, how effective should it--
There are different, very different flies, right?
There like these little day flies.
And then there's huge horseflies.
So which type of fly did this apply to
We said 10% of the surface area of the fly
Oh, any type of fly.
Any different fly species
Yeah
OK. All right, cool.
So go ahead.
Well, you know the point of this.
You get the point of this is, right?
A is like the sort of you think trivial system.
But once you really start thinking about it, it's pretty tricky, right?
Who else wants to, A-- something that hasn't been mentioned yet on A. Sam, go for it.
Make sure you push the button
Yeah, for the sticky trap we said that the product shall fit on a store shelf when packaged
OK, so that's kind of packaging, logistics, distribution requirement.
Very good.
Mike?
They kind of already touched on it, but the sticky tape shall not be toxic and allow for-- allow for removal of human skin without bodily harm
OK, great.
So that's in the same-- you know, human factors requirement, basically.
Yeah.
Very good.
Anything else on A at EPFL?
[INAUDIBLE]
OK, so installation requirement.
OK. Nobody mentioned the ones I remember as a child are bigger than this.
They were really long.
So capacity, right?
You could sort of have a length or capacity requirement, as well.
Good.
All right.
I think we want to move on from the sticky flies to BMW i3.
So we had a little discussion here during the break.
[?
Lucy, ?]
go for it
So the car shell meet environmental regulations through mechanical or software tweaking with a proposal
Could you guys hear this at EPFL?
You've been following the news, right?
With Volkswagon.
So what happened?
Why did Volks-- the CEO just step down, right?
It's kind of a big deal.
Trust in the company has eroded.
What happened there?
So the requirements that were set at the start.
So the car shell meet environmental regulations on emissions.
This requirement was probably not met.
But the realization that it wasn't met was probably too late in the design process, such that it cannot be changed.
And so the easy or cheap way to meet environmental regulations, in that case, was to tweak the software, I guess
Right.
So by the way this is kind of interesting.
But how do you measure environmental compliance of emissions for cars?
Do you know how that's done in practice?
Do you guys know?
So it's having [?
the car ?]
run on a-- having the engine run, but the car is not rolling.
And there's, behind the [FRENCH] the exhaust.
Behind the exhaust, you measure whatever is emitted
And it's like another rolling carpet, right?
It's a dynamo, basically.
And the way they do it is they have so-called drive cycles, like the [?
FT6 ?]
drive cycle.
You know, the highway, the city cycle, which was a lot of on and off.
And those drive cycles are actually different in Europe and the US and other countries.
Every country has different drive cycles.
So that's-- and what is a drive cycle, essentially?
What word would we, as systems engineers, what word would we put on it?
A drive cycle is just an, over time, accelerations velocity of the car, right?
It's a
It's a CONOPS A drive cycle is a CONOPS.
And so for diesel engines, this issue that happened with Volkswagen, it's only an issue with the diesel, the TDI engines, right, not the gasoline engine.
So the issue is that the CONOPS, the drive cycle, in the US with TDI engines, they couldn't meet it.
So they came up with this trick, right?
Whereas in Europe, they didn't have to do that, because the European drive cycle, there's a lot more diesel engines use in European cars.
They didn't have to do that in Europe, because the CONOPS, the drive cycle it's used for checking the environmental compliance is different in Europe.
So great.
Anybody else on the I-3.
Now the I3 is interesting, because that's actually an electric car, right?
And so it doesn't have an engine, a diesel engine or a combustion engine, except if you get the range extender.
So it's all electric, but you can actually, as an option get the range extender, which does use fuel.
OK, EPFL, the i3, any other requirements there?
So we have four [INAUDIBLE]
Yeah
As a third one, the car shall carry [INAUDIBLE]
OK, so recharging.
The third requirement is interesting.
You said of average build, right?
That's the word you use
Yes
Now that's great.
That's a human factors requirement.
Now of average build is a little fuzzy, right?
How would you make that more verifiable?
[INAUDIBLE]
Yeah, so there's actually people that, you know, the size distribution, male, female, you know, weight
[INAUDIBLE]
And [INAUDIBLE] the metric measurements are actually being recorded and updated.
People are getting taller around the world.
People are getting heavier.
We know that, right?
So the way you would write that third requirement to be verifiable is that the car shall accommodate for passengers in a, say, P10 male or P5 female and a P90 male, right?
And if you write it that way, then you know the actual weight and dimensions of the human body can actually be traced to a database that's pretty well known.
So then instead of saying average build, you say it's a PT50, P50 female.
And that's something that's very verifiable.
Does that make sense?
Yes
OK, great.
So what about the last one?
So by the way, here at MIT, anybody who's been to EPFL?
Been to the-- [INAUDIBLE] He has a Master's from there.
[INAUDIBLE] been there?
This is a very cool building.
It's pretty unusual.
I mean, we have some very unusual building on campus, too, here.
But if you get a chance, it's pretty.
Unusual so let's hear from you guys.
So what do you think was a requirement for the [?
Rolex ?]
center design?
The building should be a recognizable structure, that would be memorable for
Ah, interesting.
A recognizable structure that should be memorable.
So that's-- I think I know where you're going with this.
But it's a little-- you know, recognizable.
Every building is recognizable in a sense, right?
I think-- I like it.
I know where you're going with this.
But it's a little fuzzy still, right?
So why do you think it has holes in it?
Why do you think the holes?
[INAUDIBLE] unique might be a better word.
[INAUDIBLE]
Yeah.
So that's really.
The holes are, in some sense, inefficient, right?
Because you're putting holes in the middle of a building.
But they provide natural lighting.
And there's a symbolism there, right?
There's a symbolism, the Swiss cheese symbolism.
So if the holes, in a sense, have at least two functions, right?
That's good.
I like that.
That's very good.
What else?
Another example of a requirement for the [?
Rolex ?]
center?
You're taking and making [INAUDIBLE] category.
[INAUDIBLE]
Good.
No, that was great.
So you have functional requirement, you had interface requirements.
You had a lot of those six categories we talked about, right?
Good.
Excellent.
Yeah?
Go ahead
So how do you write a requirement about something aesthetic in a way that's not fuzzy or is there a way to do that?
That's a great question.
I'm not sure I'm the best person to answer that.
What I can't tell you is, in the automotive world, the aesthetics of automobiles and how people judge whether a car is beautiful or appealing, that's actually moved from being kind of just a very fuzzy thing to quite measurable.
You know, there's different shapes and then the building blocks of shapes and streamlines.
Those things are-- that's really a science today.
And eventually, you know, there's ratings.
These are people rate vehicles for aesthetics and so forth.
And usually, it's a five point scale, like a Likert scale, like JD Power and associates is a very well-known marketing firm.
And so they'll say, this looks-- this will prob-- they can actually, at this point, you can show them a-- not a sketch, but a model.
And based on past data and information, they'll tell you, this car will probably score between a 4.2 and 4.4 on the 5 point aesthetic scale, JD Power scale.
It's really pretty remarkable.
But I guess the bigger point is there's some things that really delight us, that have an artistic quality and surprise us in aesthetic quality.
And it's true.
That is one of the tensions is yet-- system engineering should-- you know, this very precise.
Write it down.
Make sure there is no fuzziness there or as little as possible.
And then on the other hand, we want delightful, surprising things that have an artistic nature to them.
And that is absolutely a tension.
And we just acknowledge that.
OK, any questions?
Comments before we move on?
I had a question about--?
Hang on.
Just one second.
OK, go ahead.
At EPFL?
And to which extent can we refer [?
to norms ?]
and [?
loads ?]
in the requirements?
Yeah, you should do that.
So compliance, being compliant with standards and just to make clear, standards and regulations are not the same, right?
A standard, like an IEEE standard or an ISO standard, it's not a law.
It's not a legal thing.
It's-- a standard it is something that maybe a group of companies or a group of organizations have agreed to.
This is how we will do it, right?
The IEEE Wi-Fi standard, what is it?
802.11g.
And that's not a law.
That's a standard.
And if you're going to be 802.11g compliant, you write that in the requirements.
The environmental emissions standards we talked about, those are actually laws.
If you are going to sell a vehicle in country x, it has to comply legally with these regulations.
But you're absolutely right, if you need to comply with these things, it needs to be part of the requirements set, because otherwise, it will not happen just automatically.
[?
Weston?
?]
Yeah, it's kind of along that vein.
If you have to comply, say, with ADA, instead you have to be called out, specifically, or do you say must comply with building codes or building laws?
And it's sort of a blanket requirement?
No, you have to be specific, because building codes are, first of all, a lot of these are local, as well.
And it's very chaotic.
So and some of these might be conflicting.
So you should be as specific as possible.
OK, let's move on here.
My sense is you're getting it just for what requirements are.
And why they're important.
So let me talk briefly about requirements, decomposition, allocation, and validation.
This is a figure from the NASA handbook.
And basically, what it talks about is the high level system functional requirements are broken down into the performance requirements.
And then as you make design decisions, as you decompose your system into different subsystems, each subsystem will have its own functional and performance requirements.
And then the important thing is the difference between allocated and derived requirements.
So allocated requirements are requirements that you choose to allocate.
And then derived requirements are calculated based on the dependent requirements, based on the allocated requirements.
And I'll get into this in a minute.
So requirements are hierarchical.
We talked about this.
Functional performance requirements are allocated.
And then from these, we can further decompose and derive requirements.
And then the total set of these requirements needs to be verified and then validated against the stakeholder expectations.
So let me briefly talk about requirements margins management.
So because you don't know everything upfront, there's uncertainty.
We typically build in reserves into our requirements.
And those are called margins.
So we put in margins for mass, power, maybe memory in computer systems.
So margins are essentially reserves that are not allocated to particular subsystems, but are controlled by the project managers or at a higher level.
So the idea is that you write the requirement in a way that is a little bit more stringent than it really needs to be.
And then by being more stringent, you've built some reserve into the system to handle unexpected things.
So I'll just give you the example with mass growth.
You know, this is very typical in aerospace vehicles.
Mass growth can range between, here, 10 to 60%.
I'll show you some examples.
And a lot of it depends on the novelty of the project.
So a typical guideline, specifically for mass margins, is about 30% reserve at the SRR, 20% PDR, 10% CPR, and keep about 5% right before you operate the system.
IOC is initial operating capability.
So here's some historical data.
This is for manned or crude vehicles, starting with Mercury, Gemini, Apollo, Skylab, and then the shuttle.
You can see the mass growth from the concept stage, which is phase A or prephase A, all the way to operations, you know, between 10 and 60%.
The new Orion spacecraft is not yet included on this chart.
So what you do is you essentially write the requirement, knowing that this growth will happen during the design process.
That's the basic idea of margins.
And then the next thing you get is that you monitor the satisfaction of the requirements during the design process.
And so let's say you have, in a performance requirement, you say that the system shall not emit more than x number of [?
NOX ?]
emissions, or the systems shall not be heavier than such and such.
That's on the y-axis here.
That's your technical performance measure.
And then you move through time and, you know, as the design gets more detailed, usually it gets heavier.
You add more things.
And you monitor that.
And then you have your final current estimate of where you will be at the end of the project.
And as you do that, your limits, your reserves, your margins, and here there is an upper and lower margin shown, is going to be narrowing down.
I'll give you a quick example.
When I was at McDonnell Douglas, the F18 EF version, the Super Hornet, was being developed.
And the key thing there was GTOW, GTOW, gross take off weight.
It's basically the weight of the plane with crew, fuel, any payloads all in.
The gross take off weight of the airplane, very important, because it determines the range.
It determines a lot of things.
And the number, not to exceed number, which is shown here on this chart as the maximum contract or allocated requirements threshold, was contractually specified.
Not only that, but there were penalties, financial penalties, associated with every kilogram that you would be over.
So this was a big deal.
And they had a huge the wall chart in the hallway, the main hallway, of the engineering building.
And every day somebody would actually manually update that day's best estimate of what the gross take off weight of the airplane would be.
And it's like a Brownian motion thing.
And as soon as you hit some critical threshold, you would see there'd be people with would come together and say, we've got to take weight out of the airplane somehow, again, right?
And then basically try to get the design to comply with that requirement as you move through the design process.
You can't do that with too many of the requirements.
But the most important technical performance measures, that's what you do.
It's a big deal.
So here's the flow chart, basically, for requirements.
I'm not going to go through this in detail.
But the basic idea is the inputs come from the stakeholder expectations, the stakeholder work we talked about a lot last time.
You go through the requirements definition process and outcomes invalidated set of technical requirements, measures of performance that you can measure, and then these technical performance measures that you can then track and validate against later.
Here's a question that I often get asked.
Well, OK, so we write these requirements.
You guys write them on paper or your tablet.
But how do these requirements actually get recorded, right?
And managed.
So I would say there's sort of two-- there's the low cost, and then there's sort of the professional version of doing this.
The easy way to do it is you just write them, you capture them in a document.
So that means Microsoft Word, Excel, Google Docs, just a document, a well written, organized document.
And then you capture and revise your requirements.
And the one thing I strongly recommend is using hyperlinks to link requirements.
And this is the idea that every requirement has to be linked to some other requirement.
So if it's a low level requirement, you want to ask, well, where does that come from?
Why did we write that requirement?
Well, it has a parent requirement.
You want to have a hyperlink there to get you from one to the other.
And I have a little example for that.
And I think this is OK for smaller projects, where you have dozens or a few hundred requirements, right?
And so here's my rule of thumb for this.
Do you remember the magic number 7?
So we talked last time, where does the world of really complex systems start.
And the argument was, well, if you need more than three levels of the decomposition.
What does that mean?
Well, 7 plus, minus 2 to the third power is somewhere between 125 and 729, right?
So if you're sort of in that world or fewer, right?
If a few hundred, a few dozen, or few hundred requirements kind of what we have in CanSat.
It's OK to do it this way.
It's still going to be a lot of requirements.
And you have to do a good job.
But you can do it that way.
If you have more than that, and that typically means more than 1,000 requirements to handle, and there are big projects that have 5,000, 10,000 requirements, there is no way you can manage that effectively in a kind of document based way.
So what you need then is a database.
You basically capture the requirements in a relational database where that allows you to link each requirement to other requirements.
And so this is not meant as an advertisement, but one of the most heavily used requirements tools out there is called, DOORS.
And this was relatively recently bought by IBM.
This was a separate company.
It was bought by IBM and included it in a suite of software tools called IBM Rational for System Development.
And so DOORS allows you.
It's a database, relational database, that allows you to write requirements, share requirements.
And the latest version of DOORS is actually web based, so you can have people in India, in Europe, in the US, you're co-developing a system.
You can all write requirements and manage them on this common database, right?
Because if you have a document, very quickly it's going to be confusing, as what's the latest version, who has the latest update.
Version management becomes a nightmare.
So just so you're aware of this.
We're not going to be using DOORS in this class.
We'll just do it document based.
But you know the rule of thumb here is, more than 1,000 requirements, you've got to go to some professional solution.
So here's a very sort of trivial example of hierarchical requirements with links.
Requirement one, the systems shall fit into a volume not exceeding one cubic meter.
And then we have sub-requirements.
The system's width shall be between 0.5 and 1 meter.
The height, the depth.
The system shall be made entirely from aluminum 60/60 alloy.
A sub-requirement here is the system shall not contain any internal voids or cavities.
Requirement three, the shape of the system must be a cube.
And then a sub-requirement, the angles between the sides shall be 90 degrees plus or minus 1 degree.
And so I did this here just in the slides.
But if you click on this, it'll actually transport you into another requirement.
This is the requirement four.
The mass of the system shall not exceed 2,700 kilograms, right?
Aluminum has a density of about 2,700 kilograms per cubic meter.
So I click back, it transports me back to the earlier requirement.
So the fact that it's made of aluminum and the volume cannot exceed one cubic meter, then this requirement four, the mass shall not exceed.
It's not an independent requirement.
It's an dependent or derived requirement based on the first two.
And therefore, they're linked.
Do you see how this works?
And to really manage requirements well and then link them, use these hyperlinks.
It's very, very effective.
So what would be an object that satisfies these requirements?
So here's our one cubic meter envelope.
So an aluminum cube with a side length of 60 centimeters this volume and this mass will satisfy the requirements.
It's not the only thing.
There's a lot of other geometries that would satisfy it.
But this particular instantiation would satisfy these requirements.
So that's the idea hierarchical requirements, linking them to hyperlinks.
OK, so let me talk briefly about the challenges now of requirements definition.
And there's-- this is not easy.
There's a lot of challenges.
The first one is requirements allocation.
You know, there's composition and flowing requirements to the lower levels.
And then the idea is that, whatever requirements you derive at a lower level, if you satisfy the lower level requirements, that should guarantee that you then automatically satisfy a higher level requirements from which the lower level requirements were derived.
That's the basic idea of requirements allocation.
And so the way you can think of this, graphically, is we start out at the high level.
Here's our stakeholders.
Stakeholder needs requirements.
This is level 0.
You set the system boundary.
What's the lifecycle?
And then we apply it.
That's our first application.
And then we apply it by decomposing the system into its function.
And so that's the second application.
And then you say, well, how do these-- how will these companies be carried out?
So we define subsystems.
And essentially, then, the subsystem requirements are derived from the functional requirements.
And you put numbers, key system performance parameters, behind these.
And depending on the complexity of the system, you may have to go multiple layers down.
But that's the basic requirements allocation process.
And I think I said this already.
It's difficult to do and stay solution neutral the deeper you go into this.
Let me briefly mention, this will not be, we will not sort of test you on this.
But I want to make you aware of this.
A methodology called ISO performance.
And that was actually the topic of my dissertation.
How do you allocate requirements to lower level parameters in systems, when a higher level requirement is defined, is clear.
So the idea is that you have a vector of desired performance requirements.
And you want to understand, first of all, you want to understand are those requirements feasible.
And if they are feasible, it usually means there is more than one way that these requirements could be satisfied, that the higher level requirements could be satisfied.
So find different non-unique feasible combinations to satisfy the high level requirements.
And so this is one of the readings.
Very quickly, we have our design space.
We have our objective space.
And then we have this cost risk objective space.
And so the idea is that, in the performance space, you have the shall statements.
You shall perform at that level.
And that's this point here.
That's your performance target.
And if it's in this gray area, it's actually feasible.
And it usually means there's more than one way to do it.
So if you can then map backwards to the design space, these points here are all ISO performance, meaning they all satisfy and provide this level of performance.
But they do it in different ways.
And then in order to select the final design or final requirements set, you want to look at other objectives.
This is where the should statements come in.
And these are typically cost and risk related objectives.
So let me give you, this sounds pretty abstract and theoretical.
Let me give you a very specific example.
This was my case study, the Space Telescope.
This is the actually this has now become the James Webb Space Telescope.
Hasn't launched yet.
This is 20 years ago.
We've been working on the James Webb for a long time.
But if it works, it will be exciting, because it's going to get us very, very close to the Big Bang, right?
That's what James Webb will look in the highly redshifted infrared to really see the formation of the earliest proto-galaxies in the universe.
So if this works, it will be very exciting.
So here's, essentially, a model of the spacecraft.
This is a precursor to James Webb.
And the big thing you need to do is you need to point and be very, very stable for a long time to take in these very, very faint images.
So we have wavefront error phasing requirements.
And then we have these pointing requirements.
You've got to a point very stable and in order to achieve this optical pointing performance, we have the structure of the spacecraft.
We have reaction wheels.
We have controllers.
And there is noise, different kinds of noise sources that are trying to basically prevent us from pointing with this precision.
So very quickly, this is what this Nexus precursor spacecraft looks like in the deploy configuration, here on the upper right in the stowed configuration.
Initial performance assessment.
So this is really trying to define the requirements for deriving the requirements for the structure, for the optics, for the controller, knowing that this is the point.
If you want to have this science happen, you've got a point with that precision.
That is known.
So let's flow that down.
So if we look at the pointing, for example, here on the right, this is kind of fuzzy furball.
What does that represent?
That's a time domain simulation of the centroid of the image, right?
The telescope is now observing a part of the sky, trying to get these early proto-galaxies.
And it's flexible.
It's a very lightweight telescope.
It has these reaction wheels are turning, trying to keep it stable.
You have electronic sources of noise.
So the blue furball here predicts that we would not meet the requirements.
This is some initial design, some initial set of requirements.
We need to find a way to get down to the-- this is about 15 microns-- [?
root mean ?]
square error.
So it's about three times worse than what we need to achieve.
And the requirement is we need to get down to 5 microns of pointing or jitter precision.
So how do we do this?
So what ISO performance does is it looks at the sensitivities in the system.
What are all the things that influence the pointing performance of the telescope.
And you can see here.
These are the two key performance measures on the left, wavefront error, and then line of sight.
And the bars essentially tell you what are the really sensitive parameters that drive performance.
So disturbance parameters, planned structural parameters, optics, and then controls.
So it's really multi-disciplinary.
And what's neat about it is you can, if you know these sensitivities, you can essentially calculate the Jacobian a matrix.
The Jacobian matrix is essentially the partial derivatives of your performance, which is your higher level requirement, with respect to these lower level parameters or requirements.
And you can then find, using this essentially the null space of the Jacobian matrix, that will tell you how do you move in that space to keep the performance fixed at the requirement level.
Let me explain this just using two parameters.
And this was a big deal in the Hubble Space Telescope.
So the two parameters I'm going to use are KR ISO and UD, the dynamic wheel and balance.
So these spacecrafts have reaction wheels that are turning to point the spacecraft and counteract any momentum that you get from, for example, from solar pressure.
So UD is the amount of imbalance that you have in these wheels.
If you have imbalance, that will cause chatter.
It will cost torques, which will cause that jitter.
And then KR ISO is the vibration isolation of the reaction wheel assembly.
How stiff or soft is the vibration isolation.
And what you can see in this clock is that, our initial design-- this is based on a simulation-- does not satisfy a higher level requirement.
In order to satisfy that, we need to go down to the blue line, labelled with 5 microns appointing precision.
And what you can see is that there are different ways to achieve that 5 microns.
We could go over here.
This is labeled as HST.
That's what Hubble Space Telescope is.
The Hubble Space Telescope basically went for ultra, ultra, ultra quiet reaction wheels, very, very low dynamic imbalance.
So [?
Marissa, ?]
you talked about buying sensors from a supplier and putting them in.
These reactors, there's companies that really specialize in making reaction wheels and things like that.
So if you're going to go for this point here.
Let me point that out again, so you guys can see it at EPFL.
If you're going to go, if you're going to derive and allocate this requirement, it means you put a lot of pressure on your supplier to achieve that level of dynamic wheel imbalance.
If they can do it, that's great.
But it could be very expensive.
But it makes your job easier as the spacecraft integrator.
Or you can go straight down to this point here called spec.
So you're essentially tolerating a noisier reaction wheel.
But then you need a very, very soft, very capable isolation stage, very soft isolator.
What's the disadvantage of having very soft isolation?
What do you think is the big issue?
[INAUDIBLE]
So the launch.
You probably have to lock it down during launch.
You have a big displacements.
You need, if you have a very soft isolation stage, you need volume, because the isolator is going to move a lot, right?
And you may not have that volume.
Or you could do a little bit of each.
So this test point here.
We're going to have quieter wheels, but not ultra quiet.
And we're going to have a soft isolator.
But we have the right combination of the two.
And we're going to go for that point there.
So now if you go for that point, you're going to satisfy your pointing requirement, bu you've allocate at the lower level requirement, in this case, to the isolation stage and the dynamic wheel imbalance in a kind of balanced way.
So everybody's job is roughly equally difficult.
That's the sort of idea here.
Yes?
Does this model assume that all of these variables are independent and, I guess, kind of continuous in terms of you can incrementally work better?
So they're not independent in the sense that they're coupled through the performance of the system.
But you can choose-- the assumption is you can choose these independently.
But they're coupled, because you need to achieve that requirement.
And yes, in this case, they're continuous.
But you can think of this as a discrete, as well.
So for example, the reaction wheels could be, you're picking from a catalog, in which case now you're really picking specifications for components that are off the shelf.
OK, so let me-- I'm going to skip here for time.
So basically, following this ISO performance process, you can go from some initial requirements, derived requirements that are infeasible, to, in blue here I'm showing you the results of this.
It's very close to 5.
There's some numerical tolerance here.
But by but essentially rebalancing the requirements within the system, we're achieving a higher level requirement.
But in a way that this [INAUDIBLE] sort of the challenge equitably across sub-systems.
Does that make sense?
At EPFL, did you guys follow this?
I know this was a bit fast.
But that's the basic idea of ISO performance.
Any questions on your side?
[INAUDIBLE]
Maybe I have one.
So given that you have [?
distance ?]
requirements and you probably have [INAUDIBLE] for each one, is there a way that we would integrate all those [?
lines, ?]
that you could see the [INAUDIBLE] for those?
If you don't mind, maybe it's a little bit [?
softer, ?]
but maybe there's another requirement where [INAUDIBLE]
[INAUDIBLE]
We don't hear you
Sorry, guys, ran out of battery here.
Can you hear me again?
Can you hear me?
Yeah?
So the question was, well if you have multiple of these performance, ISO performance lines, that's exactly right.
If you have a vector of higher level requirements, you're going to have ISO performance surfaces.
And so it gets, you know, the more of these you have to satisfy at the same time, the more complex this gets.
But we can handle that computationally.
So you have to do some simulation and computation upfront to make sure you pick.
These requirements are not just wild guesses.
They're actually based on some calculation and simulation.
OK?
All right.
So let's do this.
Let's do this in a kind of simplified way.
This is going to be our last concept question for today.
Here's the higher level requirement is a balloon.
Those of you that have done unified engineering here, you've done this, right?
We did this in unified.
A balloon shall lift the payload of 1,000 kilograms, which includes its own mass.
You can use either helium, which has a density of 0.2 kilograms per cubic meters.
This is in standard atmosphere conditions.
Or hydrogen, 0.1.
I've rounded these numbers, as a lift gas.
The standard density of air is 1.3.
Which of the following requirements is infeasible?
A, the balloon shall have a radius of 6.1 meters and the balloon shall use 99.9 percent pure helium.
B, a radius of 5.9 and the balloon shall use 99.99 point percent hydrogen as a lift gas.
And then 5.9 meters helium, 6.1 meters hydrogen, all these requirements actually are OK. Or none of these requirements are feasible.
OK, so I'll give you I'll give you three minutes to try and figure this out.
And then I'll show you the solution.
Don't cheat.
Don't go to the next chart.
So think about this the high level requirement is the balloon shall lift 1,000 kilograms, including its own mass.
And we're trying to allocate lower level requirements for the size of the balloon and the gas that we're going to use.
So try to figure this out.
All right, I think we probably need-- who needs more time for this?
OK, so we're going to leave this as a cliffhanger, OK?
We'll do the solution next time.
And take a little time to figure this out.
Try not to look at the next slide, OK?
So I'm going to skip this just for time.
And I want to talk briefly about the SRR, what it is, and then kick off assignment 2.
So SRR.
So the idea is you've really been working on these requirements hard, you know, as a team, with your customers.
And this takes-- this is not a quick thing.
This is not just two or three days.
Typically, writing your level 0, level 1 requirements is a process that takes weeks, months, sometimes at least a year.
But typically, it's on the order of three to six months in many projects, roughly.
And once you have your high level requirements, SRR is the milestone where you review those, right?
So here's an example.
So it's a social.
It's a peer review process.
And the main goal of SBAR is to vet the requirements as they were written to see if you have any missing, misstated, redundant, or otherwise unsatisfactory requirements.
This is a picture from JPL, from MSL, actually, mission.
And I want to just point out a good friend of mine who is a graduate here of our department, Richard Kornfeld.
[?
Voelker ?]
knows him, met him last summer.
He's been working on three different Mars missions over the last decade.
And we're going to hear from Richard later this semester about verification of requirements, because he verified the entry descent and landing requirements for MSL.
So here's essentially what it says in the NASA handbook.
SRR happens during phase A, before you go into phase B.
And you need at least your top two level requirements written before you can do the SRR.
I do want to mention the second reading post reading.
This is about requirements volatility.
So just because you past the SRR doesn't mean that the requirements are completely frozen.
First of all, you don't have a lot of lower level requirements yet, right?
Those are going to be added post SRR.
and even some of the-- you pointed out, some of you, the requirements creep and these things.
So this second reading is about requirements volatility, which you can actually quantify.
What are the sources of requirements volatility?
That's what this figure shows you.
And then what is the impact of requirements volatility?
What is the impact of requirements volatility on a project?
So specifically, on the number of system requirements, on rework, on project schedule, and so forth.
Really, it's a very, very recent paper on requirements volatility.
So let me just kick off assignment 2.
And then we'll be done.
So assignment 2 goes out today and is due in two weeks.
So that's October 9.
And it's very focused on requirements.
Basically, what I want you-- and it's the same teams that you're in for all the assignments.
Is essentially, first of all, first task is find a project or program where poorly written or managed requirements were a major problem.
So as a team, discuss an example and discuss that example as a team.
The second task is look at those CanSat 2016 requirements, those 47 base requirements and analyze those critically.
So that means are they feasible, are they well-written.
Classify them.
What type of requirement is it?
Is it an interface requirement?
Is it a performance requirement?
And then figure out whether there's a hierarchy there.
And basically, then, from that, generate your own set of requirements for the CanSat competition.
So you're going to either rewrite or organize these requirements in a way that's better and that works for you as a team.
And then the fourth requirement is to figure out where you want your margins, where do you think you need to have reserves and margins in these requirements.
So it's very much looking and reorganizing and critically analyzing the CanSat 2016 requirements set.
So let me summarize.
Good requirements are really essential.
It's the starting point for system design, system engineering.
It is a challenging thing, especially the flow down is challenging.
There are some methods and commercial tools for doing formal requirements management.
So I mentioned ISO performance.
And then I mentioned DOORS, these commercial tools.
And then the last point here is just because you passed SRR, and you have high level requirements approved, doesn't mean that's the end of the story.
The requirements will continue to be updated, refined.
But you have to be you have to be very disciplined to keep that in check.
If your requirements volatility is too high, then bad things can happen to your project.
And I really recommend the reading on requirements volatility, some very, very recent data on that, OK?
So that's the end of the session for today.
I'm going to be online on the WebEx in about 10 minutes.
If any of you have questions, or any comments, or you want to dive into this further.
I know this is not a great time for you at EPFL, because it's Friday night for you guys.
So I think what we're going to do, we'll figure this out with [?
Leighs ?]
and [?
Johana.
?]
We'll see if we want to do a separate time, a different time during the week for the office hours.
I'm happy to do this at a better time, if this doesn't work for you guys, because it's happy hour time for you.
So anyway.
OK, so great.
Have a great week.
And we'll see you next Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let me start.
Session 3 is about system modeling languages.
But before I start, I'd like to remind you that A1 is due today.
The first assignment is due today.
And I think neither [?
Yuanna ?]
or I or [INAUDIBLE],, did you get a lot of questions about A1?
No.
I didn't get any questions
Yeah.
So we didn't get many questions.
So I interpret that as a positive, but I guess we'll find out.
Everybody submitted.
Well, so we're hoping to have these graded in about a week, give you feedback.
And we'll also post a master solution.
And A2 is out right now.
And the other good news is there is no new assignment today that's being-- the next A3 is going to go out next week.
Any questions about A1?
Was there something that was particularly difficult or confusing?
Or was it straightforward?
Anybody want to comment on A1?
Wow.
Sam.
Do you want me-- push the button
No, our team thought it was fairly clear on what we had to do based on the lectures

We didn't have any trouble
Good.
All right.
Well, let's keep going then.
So the V-Model is our roadmap for the class.
We're starting to fill in the V. We're still in the upper left corner.
And today's lecture is actually a little different.
It's sort of in the center of the V, system modeling languages, as a precondition or precursor to what we've been calling MBSE, Model-Based System Engineering.
So what I'd like to cover today is why do we need-- or why have these system modeling languages emerged, particularly, what do we mean by ontology, semantics, and syntax?
And then we're going to give you a-- I would characterize this as a sampler of three different system modeling languages that have emerged really within the last 10, 15 years.
The first one is called OPM, Object Process Methodology.
The second one is called SySML, System Modeling Language.
And then the third one is called Modelica.
And then we'll sort of quickly wrap up with the question, you know, what does this mean now for system engineering today, and tomorrow in the future?
So I'd like to motivate this with a little exercise.
So you remember Mr. Sticky from last time?
You came up with some requirements.
So it's kind of the simplest system I could think of here.
So what we'd like to do is have you work in pairs again.
And the assignment here is to describe this system as clearly as you can, provide a description.
So last time the assignment was right, a requirement, come up with some requirements that led to this design.
But today I would like you to describe what the system is, how it functions, and so forth, as clearly as you can.
And I would like you to do this-- so hopefully you're on the Webex, logged into the Webex.
I would like you to do this in teams of two.
And as you're doing this, I don't know if you've noticed but on the Webex there's actually a Notepad feature.
Where is it?
Tell me.
Left?
Right?
[INAUDIBLE]
Oh, I see.
So I can't share and use the Notepad at the same time
[INAUDIBLE]
Annotate
[INAUDIBLE]
Yeah, yeah
[INAUDIBLE]
Stop sharing.
But then they can't see it
[INAUDIBLE]
Can they see this?
[INAUDIBLE]
So you can write text.
You can draw shapes.
So the reason I want you to do this on the whiteboard is such that we can then go around and look at some examples.
So the assignment is take five minutes, turn to your partner and try to describe this system.
And then we'll go around and look at some-- we'll sample people's descriptions.
Go for it.
All right.
So keep working on the assignment but do it locally on your computer, not on the whiteboard.
And then we'll sort of discuss it and share it and maybe not use the whiteboard, because I didn't realize there's only one whiteboard that we all share.
I thought that you have individually the whiteboard and then you can sort of pass it on to different people.
So if you do it locally on your machine, then we can share the screen so it'll work.
So do it in do it in PowerPoint, or Word, or sketchpad, or anything you want.
Sorry about that.
All right.
So let's do this.
We're going to sort of go back and forth between here and EPFL.
Let's start maybe over here with Narek.
Are you ready?
So I'm going to give you the ball, and then you can sort of explain how you guys describe the system
So what we decided to do was identify the primary function of Mr. Sticky.
So the primary function is to trap the fly, we thought.
This is enabled by a couple of other functions that are sort of at a lower layer of abstraction.
It's attracting the fly, immobilizing the fly, transporting the objects to where you need to immobilize the fly, and deploying Mr. Sticky.
And we mapped this to the physical forms that enable the function.
So the canister, the physical form of the canister is helping with the transporting function.
The sticky tape is helping with the immobilizing function.
The scented material, we thought, would be helping for the attracting function.
And the hook maybe on top that you use to hang it would help with the deploying
OK. Good.
So nice function form separation.
And you used primarily text, human language to describe it.
So let's see, at EPFL, who would like to share?
And we'll give you the ball
OK. We can try maybe
Who is speaking?
Maxim
Maxim, OK. Can we give the ball to Maxim?
Yes.
Do you see something?
Yeah, its good
Oh, perfect.
So we draw like the same system when deployed and undeployed.
So we begin with a container containing basically the sticky setup rolled.
Then when unrolled, we have the container that should be linked to the sticky setup, the [INAUDIBLE] whatever.
And then we have like an external input from the insects that were, that come to-- we it sticked onto the--
Yeah, go ahead
OK.
So that's all
So I'll note here that you guys used graphics.
You used some kind of graphical language.
And at the highest level, you have-- it's like a state diagram, rolled, unrolled.
So you showed the system in two different states.
Very nice.
Somebody else here on the MIT side, and then we'll go back one more time.
Who would like to share here?
Lucille?
OK.
So let's--
So we did a diagram showing the use of Mr. Sticky.
So we have a user because Mr. Sticky has to be rolled and unrolled-- well, unrolled.
The user installs or disposes of Mr. Sticky.
The flies are attracted to-- or Mr. Sticky attracts the flies, and the flies stick to Mr. Sticky.
And then we did compose Mr. Sticky into the different components that are below at a lower level.
And yeah, so it's basically at a higher level of use diagram
OK.
So here we have, again, a graphical description.
The states are sort of implied.
But you're focusing on decomposition, the elements.
Very nice.
Anybody else at EPFL?
Did anybody just write a paragraph of text, or more of, like, sentences?
[INAUDIBLE]
At EPFL, who wants to share?
Chris here
Chris,
So we didn't write the text.
The text seems a bit heavy to convey a description in efficient terms.
What we prefer to do is decompose in elements and for each element give some properties
OK. And did you do this in the form of a list, or in the form of a table?
Or how did you actually describe it?
So wait.
I'm trying to share the screen

All right.
So we just worked on the first half here.
We have the band.
And we give here properties.
Those that have to be made of paper or soft material has to be 1 to 1.5 meters long, 3 to 5 centimeters large.
It needs to have a coating which itself is a sticky material, and be centered in order to attract flies.
It needs to have visible color.
And then, well, the other part here, which I'm highlighting, would be related to the packaging, so a self-sealing linear container with a single-use opening, including a hanger.
We'd possibly give branding on the packaging and, well, the non-toxic material would refer to the sticky material
Good, good.
Thank you very much.
This is great.
So what you showed, that's more like a list.
And I would describe-- this looks like what I would call a bill of materials.
It's essentially a list of the primary elements of form of the system.
But there are some attributes that are associated.
So this is a list format in the form of a bill of materials with attributes attached.
So thank you very much.
That's great.
So I'm going to share here again.
And you're probably wondering what the heck?
Why did we do this?
Why did we do this exercise?
And that's the point I want to make next.
So here's a very simple system.
And we have four examples of descriptions that were quite different.
And of course, if you had more time, they'd become more complete.
But they would be really different.
So this is fundamentally the issue that we've been facing in systems engineering for a long time.
The means for describing our artifacts, whether it's something as simple as a Mr. Sticky, or an airplane, or a spacecraft, or a medical device, or even a service, how would we describe it?
Well, first of all, natural language, the human natural language.
And as we know, the human natural language is very rich.
There's very different ways in which we can express, essentially, the same facts, the same things.
That's a wonderful thing if you're a poet or a writer.
But it makes system engineering challenging, because it gets confusing when we're describing the same thing in very different ways.
Or graphical, so we saw some boxes, and we saw some great examples, sketches, drawings.
So fundamentally, the way we describe systems-- and this gets to the left half and right half of the brain-- is using language, words, sentences, lists, or graphical.
Those are the two fundamental ways of describing systems.
And then we put all these descriptions together in what we've been calling documents.
We aggregate this in documents.
So examples of documents would be a requirements document.
That's what, essentially, you're doing in Assignment 2.
Or a drawing package, even if it's in CAD, it's still essentially a document.
So typically in system engineering all of this gets assembled into what we call a TDP, Technical Data Package.
And fundamentally, when you're designing a new system, you're producing a technical data package that has software drawings, descriptions.
And that's the deliverable from the design process, is this TDP, Technical Data Package.
And from that, you should then be able to build and operate the system with as few errors, mistakes, misunderstandings as possible.
And fundamentally, as our systems have been getting more and more complex-- we're now talking about the systems that need the three, four, five layers of decomposition-- it's very easy to have errors, omissions, different interpretations of this information.
So that's fundamentally-- but there are advantages.
I don't want to say it's categorically bad to use natural language and graphics.
They're definitely an advantage, familiarity to the creator of the description.
So it's easy.
It's comfortable.
It feels familiar.
And also it's not confining, so you can be quite creative by creating descriptions in this way.
But the list of disadvantages is quite long for allowing an arbitrary description, room for ambiguous interpretations and errors.
It's quite difficult to update.
So if you make a change in one description, that change will not automatically propagate to the other descriptions.
Handing off these descriptions from one lifecycle phase to another, there's discontinuities in these hand-offs.
Uneven level of abstraction, so what I mean by that is you may describe one part of the system, and very detailed.
So the last example we saw with the list, with the bill of materials, there was quite a bit of detail there on the scent and the attributes of the tape.
But one of the other-- at least a couple of the other descriptions didn't have that level of detail when it came to the tape.
So the level of abstraction could be quite different in the different ways to describe it.
And then for a complex system, you can imagine that the amount of volume of information grows a lot.
And so you can walk into any program manager and systems engineers office and see bookshelves full of binders, dozens and dozens of binders with documents, thousands and thousands of pages.
And many of them are never read.
That's the big issue.
So that's been the kind of way in which we've been doing system engineering traditionally.
So the idea here is that in order to mitigate-- yes, Justice
[INAUDIBLE]
Phase A, conceptual design.
Phase B, preliminary and detail design.
Phase C is testing and launch.
Phase E is operation.
So usually not the same people do conceptual design, preliminary design testing.
So all the technical data package, these artifacts have to be transferred and handed off to new people who then work on the next phase.
That's what I mean by hand-offs.
And, yeah, so the idea is in order to mitigate some of these disadvantages of natural language and graphical description, there has been, and this has been recognized for a long time, a need to be more precise, perhaps more confining, but to create languages that allow us to describe systems much more precisely.
And so I'll mention a couple of the past efforts.
And you can read about these.
Each of these has-- so I'll mention bond graphs first, 1960.
This was actually invented here at MIT by a professor in mechanical engineering.
His name was Henry Paynter.
Professor Paynter created bond graphs.
You can think of bond graphs as block diagrams where different blocks have ports or interfaces where information, material, energy flows in and out.
And you can compose a system out of these blocks.
These bond graphs are essentially-- and Narik will talk about Modelica, which is sort of a modern version of bond graphs.
It has other features, too.
But this has been sort of one attempt.
Another one that's very well-known is IDEF, I-D-E-F, about 20 years later.
This was created by the Air Force, the US Air Force.
And this is essentially a description of systems that's very functionally oriented.
So it shows you what functions are involved.
And we saw one of the descriptions was very functionally oriented, and how the functions of the system relate to each other.
But generally these system languages have not fully been deployed.
Some organizations use them, others don't use them.
And the main reason for that is twofold.
First of all, some of these languages were incomplete.
They would focus only on one aspect, like functions or the block nature of the system, the block diagrams.
And a lot of them were not executable.
So they would be graphical, but you couldn't actually simulate and actually check whether that description was complete or accurate.
So since then-- and the other thing, of course, important is domain agnostic.
So what I mean by this is that the system modeling language should be applicable for any kind of cyber physical system.
Again, if you're designing a spacecraft, an aircraft, medical device, any kind of product, the language shouldn't have to be adapted.
The language sort of covers all these applications.
That's the idea.
So whatever language it is that you're using or developing, it has to have these three things.
Any language has these three things.
So the first is ontology.
And I reference here the Wikipedia articles on these things.
Some of my colleagues-- in academia it's a big debate is Wikipedia a legitimate source of information or is it not?
My position on this is that it is.
I think Wikipedia is definitely not perfect, depending on what topic you're looking at, but it's a sort of self-correcting system.
So I actually go to Wikipedia, and then there's references.
And you can dive deeper.
So I give you the Wikipedia links here for these three things.
First, ontology.
So ontology is a very fancy word.
What ontology actually is-- Mark, why don't you come up here?
You're an instructor today.
Ontology, fundamentally, is describing the entities that are allowed to exist in the language, subjects, nouns, adverbs, what are the objects, the entities, that can exist.
So it's a kind of very abstract thing.
But it's essentially the library of words and objects that are allowed to exist in that language and then how these entities can be grouped perhaps in a hierarchy and subdivided.
So it essentially constrains the universe of things you can describe in that language.
The shorter, the smaller your ontology is, the more confined the language.
Semantics.
Semantics is basically a branch of science of philosophy, which is fundamentally assigning meaning to those objects that are described in the ontology.
And so the way that we say this is that it's the relationship between the signifiers, so the signifiers are words, letters, symbols, graphical symbols.
So how do we describe a resistor, for example, in electrical engineering?
A squiggly line.
It's the zigzag line.
Well, if you don't know electrical engineering, you just see a squiggly line.
It's meaningless to you.
But if you know that semantically that means that's a resistor, that's the symbol for resistor, that's what we mean by semantics.
And then the third is syntax.
What is syntax?
It's the set of rules, the set of principles and processes by which the objects or the entities in the ontology can be combined to build up higher level information, like sentences, paragraphs, and so forth.
And so that's essentially the construction rules for the language.
So every language has these three things.
So any questions about this before we move into our first language?
So we're going to give three examples of system modeling languages, and you'll see the similarities and differences.
But as you see these languages, keep in mind they all have ontology, semantics, and syntax.
Any questions about that?
So OPM.
Let me describe to you OPM.
This is one of the younger languages.
And so OPM stands for Object Process Methodology.
And it was created by Professor Dov Dori at Technion, a colleague of mine.
Dov is essentially a computer scientist by training.
And you'll see the heritage here of OPM.
And the big news here about OPM is OPM is not that well-known yet.
If you ask around, not too many people know OPM.
But I predict that in the next decade that will change very quickly.
And one of the reasons is that OPM was just now adopted as an ISO standard.
And if you know ISO, the International Standards Organization, they're located in Geneva, it's a big deal to become an ISO standard.
It took like five years, the whole process, with committees and reviews.
And so the ISO standard is actually-- OPM was adopted as an ISO standard as a means to describe and develop other standards.
So it's kind of a metalanguage because, as you can imagine, when you read different standards which, by the way, have a lot of influence, they're also written in natural language and graphics and lots of inconsistencies.
So the idea is that future ISO standards should be written using OPM such that they're clear and consistent, and so forth.
So the history here is that if we go back further, UML, which is Unified Modeling Language which I'm not going to talk about today, is a software-- this is a language that was developed primarily for software engineering to consistently describe use cases, to consistently describe activity and flows in software, the structure of software.
But it's really software centric.
So from UML 2.0, we then sort of branched off into SysML, which Mark Chodas, who just joined us will talk about, and then OPM.
So these are sort of derivatives of UML.
And there's a book, it's not one of the mandatory books for this class, OPM 2002.
If you're really interested in OPM, I recommend you invest in that book.
It's really very well written.
So let me give you an example of how OPM can be used.
So like we said, typical product representations are sketches, engineering drawings, or UML diagrams and software, but the need for a unified representation.
And fundamentally, we have functions and then we have objects, form and function, in systems.
And so what we would like to do, and the premise of OPM, is that we can show everything in one diagram type, so the functions, the functional attributes, the objects, and there's different types of objects, operands, system components, consumables, the attributes of those objects, and then the links.
And I'll show you the different types of links between these that exist in OPM.
So it's a generic modeling language.
And it has been successfully applied to system architecting of complex products in different organizations.
I'm going to try to give you a pretty simple example here, which is a refrigerator.
So we're going to look at a small kind of household-level refrigerator through the eyes of OPM.
All right.
So here's the basic ontology of OPM.
It's very, very simple.
And that's the idea, is to have as few objects, as few entities as possible in the language to keep it simple.
So the first one is the idea of an object.
What is an object?
Objects are drawn as these rectangles.
Objects are defined as entities that have the potential for stable, unconditional existence for some positive duration of time.
And objects have states within them.
So what would be an example of an object that we've talked about today?
Go ahead
Maybe the sticky tape
Yeah, so the sticky tape itself.
That's an object.
It exists unconditionally.
It's there.
And what's important is it could be a physical object.
So it has a physical existence.
But it could also be a sort of informational object.
So for example, if you have an idea or a vision, that's an object, too.
It's not physical in that sense, but it does exist as an informational object.
What are states?
Let's see at EPFL.
What would be an example of a state that's associated with an object?
Yeah, that's OK. Can somebody give an example of a state?
Yeah.
Rolled and unrolled for the sticky tape
Rolled and unrolled.
Exactly.
Or furled and unfurled.
So that's kind of a binary state that could be halfway unrolled, or the sticky tape is full of flies, or it's kind of empty.
Those would be describing the object in terms of what state it is in.
Exactly.
So the form is then the sum of all these objects.
So that's one building block.
And then processes are the other.
So what are processes?
Oh, is there another example?
Yeah, go ahead
May I ask a question?
Is this [INAUDIBLE]??
Yeah, it's [INAUDIBLE]
Go ahead
What do you mean by positive duration of time?
Well, meaning that the object could be created.
It didn't exist before.
It's created, and it exists.
And then it could be destroyed again.
It could disappear, or it could be consumed.
But it means that the object needs to exist for a non-zero period of time in order for it to be called an object, so objects in the world that can be described with OPM, fundamentally, objects can be created.
Objects can be modified, particularly their states can be modified.
And they can be destroyed or consumed.
That's basically it.
That's a complete set.
Does that make sense?
So processes are-- what are processes?
Processes are really fundamentally different from objects.
Processes are shown as these ellipses.
And they're the patterns of transformation applied to one or more objects.
And processes change states.
So processes, unlike objects, cannot exist on their own.
Processes only make sense if they're associated with at least one object.
So processes are essentially-- the functions that we develop in systems are processes that transform or create, destroy, or transform objects.
So function emerges from one or more processes.
And then we have different links between objects and processes.
I'll show you two examples here.
One is the arrow with a pointy head.
That could be a consumption or production type link, or a link with this little lollipop symbol.
This is known as an instrument link.
And so the difference there is that if an object is linked to a process using an arrow, it means that something's happening to that object.
It's being created, or destroyed, or modified.
If an object is linked to a process using the lollipop symbol, the instrument link, it means that in order for that process to happen this object is needed, it's an instrument.
But the object itself, the instrument, does not get modified in any way by the process.
But the process couldn't happen if that object didn't exist.
Do you see the difference?
And so one of the really, I think, important things about OPM, but any of the languages, is that every box, every arrow, every link has precise semantics.
And usually when we kind of doodle, when we just think about-- we put arrows and links between boxes, we often don't really deeply think when I put a link in here, what does that link actually mean?
What does that link imply?
So when you do system modeling using these languages, you become much, much more precise.
Yes, please.
And would you push the mic button?
Yeah.
Mark, go ahead
Can an object be a process or a process be an object?
No
So what about if you have-- I guess I'm thinking in terms of if there's some process for doing some procedure and like you're assembling a satellite or whatnot.
You need to modify that process.
So how is that sort of thing represented in OPM?
The process modifies objects, but processes cannot modify other processes because processes are fundamentally, in OPM, acting upon the objects.
Now, processes can invoke each other.
So if there's a sequence of processes-- you have to do this assembly step before this other assembly step-- you can have what's called an invocation link.
But that's a logical dependency between processes.
But fundamentally, the processes act through the objects in

So let's go into some more detail.
So at a high level when you look at the economy, products-- yes, go ahead
I have a question.
Why create another language and not just stick with UML?
So good question.
So we'll talk about SysML, which is very similar to UML.
It's sort of generalized for cyber-physical systems, not just software.
So the reason that OPM was created, because UML was found to be somewhat too confining.
This is more general and also the idea of a unified representation, one type of diagram and description for any application.
So it's basically a kind of a more general version of UML.
But the other really important thing about OPM is that objects and processes are sort of-- the processes are often in object-oriented thinking.
Processes are embedded inside objects.
And in OPM, the processes have been emancipated to stand at the same level as objects.
Those are the main differences.
So let me go in a little bit more detail.
So if you think about the economy in general, goods and services, goods are objects, and services are processes.
So if you buy a new iPad, or a new pencil, or whatever it is, you're actually buying an object.
You're purchasing an object.
That's obvious.
But why are you purchasing that object?
So let's say you're buying a new tablet.
You're buying the tablet.
But why are you buying the tablet?
Sam, go ahead
You're buying the tablet to perform an action for a process on something else, to do something
Right.
So what do tablets do?
I mean, not stone tablets, but modern tablets
They allow you to work with software, communicate
Yeah, so they're information processing.
They're information-processing devices.
And there's an argument with tablets are great for consuming information.
They're maybe not as good for generating new information.
So fundamentally, you're purchasing the tablet, which is an object, in order to be able to do information processing and information consumption.
So the process is then implicit.
What's an example-- if you purchase a service, what would be an example of a service?
What would be an example of a service?
Let's see at EPFL.
What would be an example of a service that you could purchase?
Going to the dentist
Going to the dentist.
Yeah, one of our favorite things to do.
Have you been there recently?
Yeah, one month ago
So I don't want to be-- I don't want to violate your privacy, but can you share with us what happened at the dentist?
It's the yearly checkup.
You have to check that there's no hidden-- I don't know how you call that in English, carries?
Yeah, cavities.
Yeah
Yeah, cavities.
Check that wisdom tooth don't mess up what you've been working on tirelessly when you were younger with braces.
And [INAUDIBLE] get checked
Very good.
So going to the dentist.
The dentist provides a service which is either checking your teeth or filling cavities, which is a process.
And all the objects, the chair on which you sit, the instruments-- I guess we still use gold sometimes in some places-- those are objects that are used in the performance of the service.
You see the relationship?
So objects and processes always come in pairs.
Thank you for that example.
So let me talk about the links in OPM briefly.
So there are two types.
There's the structural links, which link objects to objects.
And we typically use arrows, you know, is related to or we can tag these as well.
So for example, something powers something else.
This is known as a tagged link.
It suppresses the processes.
And then there's these triangles that are-- essentially, there's a kind of hierarchy implied there and slightly different meanings.
So the solid triangle means decomposition.
So the higher level object is composed of lower-level objects.
So that's, Mark, you mentioned assembly, you know, are you creating the bus of the spacecraft and it has a whole bunch of stuff in it?
Well, you would use this filled in triangle to show that decomposition.
The second example is the characterization link.
So this is essentially relating an attribute to its kind of master object.
Specialization and generalization is the empty triangle.
And then this funny symbol here is instantiation.
So essentially, you have a general object.
And then you can instantiate that.
So I have two children.
I have two children, which is general.
And there's two instantiations of them.
One of them is called Gabrielle.
And one of them is called Christian.
And they're actual people.
So that's the idea of instantiation.
Processes.
Processes are these patterns of transformation.
They're tricky.
Processes are trickier to understand than the objects because we cannot hold or touch a process.
It's fleeting.
And the creating change or destruction of objects is what processes do.
They rely on at least one object in what we call the preprocess set.
A process transforms at least one object.
And the time is implied.
So processes take along a timeline.
And in terms of the description, in English we use the so-called Gerund form.
So all the processes, there's some examples on the right side, use the "ing" form of a verb.
So we can then put these together, objects and processes.
So here's an example of a machine.
This happens to be like a printer or copy machine.
It has a main switch.
The main switch has an attribute called Main Switch State, which can be on or off.
The process of switching transforms, in this case, the Main Switch State from on to off.
Or we could go the other way.
And in order for this to happen, we have here-- this is actually slightly different than the instrument link.
This is a filled in lollipop, which is known as an agent link.
So the operator is an active agent to carry out the switching process which changes the main switch state from on to off or off to on, and the Main Switch State is an attribute of the main switch.
So transporting.
This is another example.
Transporting changes the state of a person from being here, Location A, to being there, Location B.
So there are seven-- huh, coincidence, seven, seven-- object process links in OPM.
So P changes-- the process changes the object, say from State A to B.
That's the example we just looked at.
You can actually hide that.
If you're really not interested in all the states and details, you hide the states.
You don't want to see them.
And then you can replace that with what's called the affectee link, which is this two-sided arrow.
And all you know is that this process is affecting that object.
And it's a two-sided arrow.
But you don't know exactly how but you know it's affected.
A resultee link-- so this is an arrow pointing from the process to the object-- means that the process of transporting produces emissions that weren't there before.
So that's a resultee link.
But the process of transporting requires or consumes energy.
So the arrow is pointing from energy into the transporting process because it's being consumed.
I did mention the agent link.
So there's an operator of a vehicle.
And when we talk about autonomously driving vehicles, a big topic right now, actually, it was cool.
At EPA this summer, there's the autonomous shuttle on the campus, the electric shuttle.
Did anybody take that?
Did you guys try that shuttle this summer?
[INAUDIBLE]
Yeah?
Did you like it?
No, I didn't.
But a friend is working in this kind of shuttle, like sitting for hours waiting for people [INAUDIBLE]
So fundamentally, I mean, if you want to think of this in OPM language, a driverless vehicle is basically eliminating this, no longer needing an operator with an agent link and replacing this with a piece of software, which would be an instrument link.
So the instrument, the transporting process requires a vehicle.
And then we have what's known as a conditional link.
So this process can only occur if this object is in that particular state.
So this example here, obviously, ignores the existence of credit cards.
So you can do the purchasing.
The process of purchasing is conditional upon the state of the object money being in a state of enough for doing the purchase.
So here's an example of a little bit more complicated.
This is a Level 0 OPM diagram of a car, of a vehicle.
So you can see in the upper right is sort of a sketch of a vehicle.
And it has these different attributes.
ED is engine displacement, height, ground clearance, overall length, wheelbase.
There's a trailer here with a towing capacity.
So the way the way you would interpret this is that we have a transporting process.
That's our master, sort of the highest level process.
And it changes the attribute location for driver passengers and cargo from A to B.
And that's, fundamentally, where the value is for the owner of the vehicle.
And then we can zoom in to the transporting process and look at subprocesses, towing, propelling, and housing.
And if you think about what a vehicle does at the highest level, it protects you, it houses you, and it propels you.
And then you can break those into more detail.
And then on the left side here we have, essentially, the elements of form.
So the automobile, which is an instrument of the transporting process, can be decomposed into its major subsystems, powertrain, chassis, body, wheels.
And each of those are characterized-- you see those attribute links-- characterized by things like fuel capacity, engine displacement.
This is the design domain we talked about last time.
Ground clearance.
And so those are the design variables.
Those are the parts and assemblies.
And then on the right side, the processes, the internal processes can also be characterized by performance or functional attributes like towing capacity, fuel economy, acceleration.
PV stands for Passenger Volume and cargo volume.
And those are things when you're comparing different vehicles to purchase, those are the things you would compare vehicles against.
So they're the internal functions and then the functional attributes.
And then up here there is the fuel and emissions and safety-related issues, which that's often where the governments intervene and then regulate.
And this is sort of a highest level OPM of a vehicle.
And then if you want to see more detail, you would start drilling down into these.
And you'd have multiple levels of these, like a hierarchy of these diagrams.
Yeah
So here, what is the use of the-- or the meaning of the open arrows?
And it looks like there's a couple of different arrows here than what we had in the other diagram
Are you talking about these guys?
No
Oh, these here.
Yeah
Yeah
So it's just a visual represent-- there's no distinction on the arrows whether they're filled in or empty.
That's just a kind of graphical thing.
Yeah.
So one of the-- yes.
Veronica, do you want to push the--
How would you represent a process that creates kind of a temporary state?
So if you're saying this is acted on an object, and this changes the form of the object but the object will ultimately return to its original state kind of absent of a reversing process, if it's a natural tendency for the object to return, how would you represent that change?
Would you need to break it down as kind of a subprocess within the object?
Right.
So I mean, and sometimes there's multiple, there's non-uniqueness in sort of representing the same thing.
But there's one process that brings you to the temporary state

And then there would be a restoring process that restores you back to the original state
Does the process have to be a separate plan within the system?
Because there are certain objects that have a tendency-- I'm thinking primarily of kind of chemical states where reactions would happen naturally.
It's kind of a specific thing, but I was thinking about how you might model different systems.
And I was thinking about the engine of the car, just kind of how things might naturally return.
So do you have to describe the process explicitly if it's not something that's inherently designed in, if it's kind of a-- if it will happen anyway
I think, I want to say you have to explicitly define that

So if it's a man-made process, so to speak, then that's a process you want to happen.
And then if the restoring it back to some other state is a natural process, well, it exists.
So it will restore the system to a prior state.
That process would also have to be modeled
Is there a distinction between how you would indicate a man-made process or a natural process?
Not fundamentally

And in fact, OPM has been applied to modeling how a cell functions.
So there's been some pretty recent work on-- cells are incredibly-- the biological engineering is just really complex.
So there's some really recent work on describing even the RNA and cell division, using very much this language

So it doesn't matter whether it's an artificial process or a natural process
Thank you
Let me go a couple more minutes, and then we'll take a short break.
And then we'll talk about SysML and Modelica.
So the key thing in OPM is there's only one type of diagram.
And there's also natural language that gets auto-generated.
And I'll show you this very quickly in the tool.
So as you can imagine, as you're working on real systems, these diagrams, if you'd showed them on one sort of level, you'd have thousands of objects and links.
It would be a mess.
So how does OPM handle complexity?
There's three fundamental mechanisms.
One is known as folding and unfolding.
What does that mean?
It's basically related to the structure.
So folding/unfolding means that higher level objects, you can show the decomposition of the objects or you can hide it.
That's known as folding and unfolding.
Then the second one is in-zooming or out-zooming.
And so here's an example of a process and an instrument and an affectee that's affected by the process.
And I want to know, what are the subprocesses in that process?
So you can zoom into this process, and it will expose the subprocesses that are happening inside.
That's known as in-zooming.
And then going back the other way is called out-zooming.
And then the third one I've already mentioned, which is that states can be expressed or suppressed or hidden depending on your interest and what states of the system you want to look at.
So here's the sort of Level 0 OPM of our refrigerator.
I said that was kind of our case study.
So how does the refrigerator work at the sort of-- Level 0, that's what the stakeholder, what the customer sees.
Don't care about the details of what's happening in the refrigerator.
So we have food-- and we'll get back to this I think next week in the kind of creativity concept generation.
Why do we have refrigerators fundamentally?
If you've heard this before, you keep quiet.
Maybe EPFL.
Why do we have refrigerators?
Any ideas?
Go ahead
Keep food cold
Yeah, well, if you're a beer drinker, you want cold beer.
But if you really think about it deeply, that's not really the primary reason.
The primary reason is this state change, their shelf life.
So the primary reason why you have refrigerators is to extend the shelf life of the food.
So speaking as a systems architect, system engineer, a refrigerator is a food spoilage rate reduction device.
You see that?
So the attribute of the food is the shelf life, and we're going to extend the shelf life of the food.
If you think about it sort of architecturally, that's why we have refrigerators.
But I agree with you on the cold beer.
We all agree we want cold beer, not warm beer.
So you're right.
You're right, too.
So the refrigerator essentially is an instrument of extending the food shelf life.
So the food is the operand.
The food is the operand.
The extending of shelf life is what we call the primary value delivering process.
That's where the value is.
The refrigerator itself is the product system.
And then the operator sets the thermostat setting at which temperature the refrigerator should be.
And then here we have the primary operating process, which is what allows us to keep the temperature of the food at that level.
And in order to do this, we consume electrical power.
We produce waste heat.
And we also require, or we convect that waste heat to the exterior air at a certain temperature.
How well do refrigerators work in a vacuum chamber?
They don't.
They don't.
There's no way to-- well, I guess you could radiate the heat a little bit.
But they're not going to work very well.
You're not going to have conduction because you're sort of in the middle of the vacuum chamber.
You're not going to have convection.
So you only have radiation.
And that's not going to work very well.
So the exterior air is important for the refrigerator to work.
So then you say, well, OK. That's fine.
I buy that.
But now I want to really know, how does it really work?
So you say operating.
The refrigerator is operating.
But I want to do in-zooming and understand, how is it operating?
So what's the key to refrigeration?
What's the magic word there, or two magic words?
[INAUDIBLE]
Yeah, that's part of it.
That's just a sliver of it.
Heat exchange is part of it.
So the magic word is Carnot cycle.
So here's a little graphic that sort of gets into it.
So the Carnot cycle is actually a thermodynamic concept where you're compressing essentially a refrigerant.
A coolant is being compressed, absorbs the heat from the inside and then expands and condenses and radiates that or convects that heat to the outside.
So here's a-- I don't know if you remember your thermodynamics.
This is a classic PV diagram.
You've got the four legs of the Carnot cycle.
And actually, what's really nice here-- so we're going through this counter-clockwise.
What's really nice about it is that every leg of the Carnot cycle is one of our processes.
So compressing is this leg here from D to B. Condensing is from B to A.
Expanding from A to E. And then evaporation happens from E to D. So the Carnot cycle can be decomposed into four subprocesses.
These are the internal processes in the system that are governed by physics.
So if we take that operating process that we looked at before, we can actually zoom in and see the subprocesses emerging.
And so in cooling we have those four expanding, evaporating, compressing, condensing.
But I'm adding the absorbing process, which is that the heat then needs to be absorbed by the exterior air.
We have to power the device.
You can decompose that into grounding, protecting, supplying.
Regulation, keeping it at the set point, you can decompose that process into sensing, switching, and setting the set point.
And then we have supporting, which is we need to be opening and closing the refrigerator, retaining it, and then connecting all the pieces.
So at Level -1, we had one process at Level 0, which was operating.
The refrigerator was operating.
And then as we zoom into Level -1, four processes appear, powering, regulating, cooling, and supporting.
And then at Level -2, we have 15 subprocesses.
So this is sort of a view at Level -1, our four subprocesses, cooling, powering, regulating, supporting.
And then we can zoom in more.
So here's the general idea, and we've looked at many systems over the years, that most cyber-physical systems-- or it says optomechanical here, but I really mean it more generally-- have this kind of OPM structure.
On the right side, we have the output that the customer, the stakeholder cares about, the operand.
We have a set of specialized processes, and these can be often organized in a cascade.
And then we have supporting processes, like powering, connecting, controlling, that provide support for the specialized processes.
Most systems that we've seen have this generic architecture.
How do you generate an OPM?
Fundamentally, you can do a top-down.
So you start with your stakeholders.
That's what we did in the first lecture.
Where's the value?
You start thinking about requirements, what functions, how well the functions should be performed.
And you sort of go down.
Or if you already have a system, you can actually reverse engineer that system and from bottom up, like we started doing for the Mr. Sticky, and that's fundamentally reverse engineering.
So just for time, I'm going to skip this demo.
But what I will do is I will post a video.
I'm going to make a little video with the OPCAT demo and post that to Stellar so you can sort of watch that.
So this is one of the-- it's still not super mature, but it's a Java-based program called OPCAT that allows you to generate object process diagrams in a computer-supported environment and store them in an XML format, and so forth.
It allows you to create a hierarchy.
And the other thing that's very cool, it autogenerates text.
So the text is called OPL, Object Process Language.
And right now you can go from the graphics to the text, but you can't go the other way.
So they're complete sentences.
It's not like an exciting novel when you read it.
But it is semantically precise.
So we're going to switch to SysML.
We're going to take a very short break.
Are there any questions about OPM?
In the system architecture class, we spend like five, six lectures on OPM and you get to do detailed exercises.
We kind of don't have time for this in this class.
But hopefully you've seen what it is.
And if I've whetted your appetite for OPM, then the goal has been met.
Any questions about OPM?
Is it pretty clear?
All right.
So let's take-- yes
Compared to Modelica, because I've seen one of the links that OPM is just for describing the system, though.
It's not for making calculations or simulations
That's correct.
The latest versions of OPM you can do like a logical simulation.
So you can say, OK, this process enables and does this state, so it's kind of a discrete logical.
But usually it's not used for any mathematical calculations.
The purpose of OPM is really to support conceptual design, early conceptual design.
That's correct.
Mark.
So just a couple of words about Mark.
He's a doctoral student right now in the Space Systems Lab.
He's been working a lot on an instrument called REXIS.
I guess you're the chief system engineer for
Yeah, yeah
And that was also the topic of his master's thesis.
So Mark really knows what he's talking about.
He knows SysML quite well.
And thanks for doing this
So let me start by giving kind of a high-level overview of what SysML is and what it aims to do.
So it's similar to OPM, but there are a couple important differences.
SysML, as Olly said, it kind of is an extension or inherits a lot from UML.
And its aim is to really provide a language that enables you to capture all the different aspects of information about a system in one place.
And this concept of Single Source Of Truth is something that I'll kind of try and emphasize during my presentation.
The idea is if all your information is in this one model, then communication is easy.
There's no ambiguity between versions.
Everyone knows where to go to get the most up-to-date and correct piece of information.
So that's one of the emphases of SysML.
SysML is a graphical language, similar to OPM.
It's defined in terms of diagram types that I'll go into in a second.
It has more than just one diagram type, as compared to OPM.
But basically it aims to do things like capture functional behavioral models, capture performance models, capture the structural topology of your system, the parts of your system and how they're all interconnected, and any other engineering analysis model is one of the big emphases with SysML, is integration with external analysis tools.
So if you have a thermal tool, a structural tool, an electronics tool, or something like that, integrating this informational, descriptional model with that analysis model and enabling, making it easy to transfer information from your description model to your analysis model, do an analysis, and then incorporate those results back into your descriptive model is one of the things that SysML really is all about.
Then another thing, another difference compared to SysML from OPM is it incorporates requirements pretty explicitly.
And that's one of the other areas that people are really interested in is if you have good modeling of requirements, what sort of information can you glean about your system that you couldn't otherwise?
How do I advance the slide?
Oh, there we go.
So as I said, SysML is composed of diagrams.
I'll go into in a second kind of what each diagram, what all the diagram types are and what kind of their intent is.
But here's kind of a high-level overview.
It might be a little bit difficult to read.
So at the top you have a system model.
You have requirements, diagrams, behavior, structure and parametrics.
Within requirements, there's actually a specific requirements diagram that's supposed to represent the relationship between requirements in your system.
And I'll show an example of that.
In behavior, there are diagrams to describe kind of the activity of your system, the sequence of events that may happen.
There's a state machine diagram if you want to model your system as a state and transition between those states, things like that.
In the structure, there's diagrams that go over the decomposition of your system.
What is your system?
And what parts make up your system?
Both the logical decomposition and the physical decomposition.
And then there's topology.
How are they all connected?
What are the characteristics of the interfaces?
Things like that.
And then parametrics, which is kind of adding constraints and numbers to all these things, whether they be logical constraints, mathematical constraints, things like that.
Similar to OPM, SysML has no built-in analysis capabilities, so you can't like run a model or calculate an equation in SysML.
You can't really do that.
But very frequently the tools that implement system that I'll show you have that kind of analysis capability built into the tools, as opposed to the language.
So you can do things like use a parametric diagram with a bunch of equations to create a system of equations that you then can solve, whether it be in the tool, or you can move it to an external tool, like Mathematica or something and solve it, and then bring that information back into your system.
You also can do kind of sequence-based computation.
If you have an activity diagram that says first you have to build this part of your system and then this part of your system, there's things in sequence, things in parallel.
You can run simulations like that where it's all about, have you done everything you need to do to get to the next step?
Things in a more logical flow, as opposed to actually mathematical equations, you can do those sorts of computations as well.
One note is that these diagrams, although they are the main way to define your system and interface with the model, are not the model themselves.
So you can create links between diagrams.
If an element shows up in one diagram and that element shows up in another diagram, that's the same element.
If you make changes in one diagram, that's going to propagate to all your diagrams.
So there's kind of a database backend to this whole model that encompasses all the information.
So instead of having a bunch of isolated block diagrams, they're really just views into this model that's hidden in a backend database.
So we'll talk a little bit about the applications of SysML.
First is requirements engineering.
As I said, when you can explicitly model requirements in the relationship between requirements and your system, you can do a lot more.
The way that it's typically done nowadays is with tons of documents-- I'm not sure if you've ever actually developed a system, but there's an ungodly amount of documents.
I've experienced that firsthand.
It's a real pain.
There are tools like DOORS that will enable you to link requirements to other requirements, and things like that that help you manage your requirements.
But what if you had a really explicit tie between your requirements and your system?
You can actually represent in SysML.
And I'll show you a little bit about this.
You can represent in SysML a textural requirement, you know, the mass of your system must be less than 5 kilograms, or something like that.
You can tie that requirement directly to the mass property of your system.
You can [INAUDIBLE].
You can start building constraints.
Your requirements aren't just textual statements, they're actually constraints upon properties of your system.
Those are the types of things you can start to do with SysML.
Do you have a question?
[INAUDIBLE]
Yeah.
Yeah, so that isn't something that's built into the language.
But that is something you can do with-- basically there's a whole API.
And you can interface with the model.
I'll show you this, actually, in my demo.
But you can build in rules and constraints that say, check, for example, that all my requirements, at least how they've been defined, are satisfied.
You can run that check, and it will tell you have they been satisfied or not.
And that's something that's really powerful that you can't really do with existing kind of techniques.
Yeah
Thank you.
And can you also, for example, requirement changes, like the master system has to be this much, as opposed to this much, then would it go through and check until you have to now look at this, this, and this, and then that affect like-
So you're getting down in the weeds.
That's something that would be awesome if you could do.
That really-- again, that's not something that SysML enables you to do natively.
But it gives you the language and the syntax to be able to write queries that give you that type of information.
That's kind of where the cutting edge is right now is, can we do that?
Can we get that type of information from a SysML model?
That's something I'm really interested in for my PhD thesis.
So yeah, that's something that I think is possible and would be really great to have in the development process of a system.
So the next bullet here is on a system description.
So actually one of the fundamental questions is, how do you describe a domain-specific system within SysML?
I'll show you that SysML has a pretty strong notion of inheritance, and classes, and things like.
It's object-oriented.
And so one of the questions is-- I'm in the space system.
So how do you describe a spacecraft in SysML?
SysML is very general.
But how do you actually represent, for example, a CNDH system in SysML?
What are the types of attributes that are typically found?
How do you represent that?
How does it interface with other parts of your system?
Those type of questions.
That's another active area of research, domain-specific modeling.
And then finally, as I said, integration with external analysis tools.
So there's quite a lot of papers in the literature about going from a SysML model to MATLAB to SDK to Thermal Desktop, external modeling tools, taking that information out of the model, doing an analysis, putting it back in the model.
And actually, I think there I was going to talk about Simscape, which is a kind of analysis tool, external analysis tool.
And there's actually been papers written on, how do you take SysML information from a SysML model, pull it into Simscape process it, and put it back in the model?
So let's talk about the diagrams.
There are nine types of diagrams in SysML.
And I'll try and just give you a brief explanation of what they do.
I won't go into the syntax for all of them because there's quite a bit of detail in the syntax.
But I'll show you some examples of a couple of them.
So we'll go from left to right.
So there's two main classes, behavior and structure, similar to OPM.
In the behavior diagram, in the behavior diagram category you have activity diagrams, which basically represent flows of activities.
So you do this, then you do this, then you do this.
Those can be tied to system elements.
If this system element has this sort of function or property or performs this operation on another part of the system, you can represent that link as well.
There's a sequence diagram, which is more about logical ordering.
So if you have, for example, a multi-threaded software system, and you have different threads that may need different other threads to communicate with them or finish their computation before that, that way you can execute things like that, you can do that sort of interfacing between different threads of activities in a sequence diagram.
This is one of the diagram types that was kind of inherited directly from UML.
So there is a very kind of strong software element to that diagram.
There's a state machine diagram.
So obviously, state machines are very powerful.
If your system has various states, if things in your system have various states, you can represent that in a state machine diagram and then talk about, what are the criteria for transitioning between states?
What would trigger or cause a transition between states?
What are guards that must be met before you can transition states?
Things like that.
That's where you represent the state machine diagram.
These type of diagrams are very powerful for describing things like concept of operations.
So there has been some work-- I did an internship at JPL a couple of summers ago.
And they were trying to build up this capability to model a concept of operations for a spacecraft.
So what are all the power modes of everything?
What are the time-- you know, it spends this amount of time in this power mode, then it transitions here.
For example, I can give an orbit of a spacecraft.
That's sort of the thing that you can do with this set of behavior diagrams.
And then use case analysis.
Again, it's mostly focused on early concept development, stakeholders.
How do they interface with the system?
Where do they derive value?
How does the user interact with the system?
Things like that.
That's where you put a use case diagram.
Going over to structure, the block definition diagram is where you define the structure of your system.
So the logical or physical decomposition, I'll show you an example of this.
So if your system is a spacecraft, it has various subsystems, if you want to decompose it logically as a thermal subsystem, a structure subsystem, ADCS subsystem, things like that, you can also decompose it physically so your spacecraft has solar arrays, it has instruments, it has thrusters, things like that.
You can represent those types of things in a block definition diagram.
And then internal block diagram is where you describe the ties with the interfaces between all the components of your system.
And this can be at varying different levels of abstraction, as I'll show you.
Parametric diagram is kind of a subdiagram type of the internal block diagram.
So you can, again, put constraints, mathematical, logical, things like that, on your interfaces and begin to build up the infrastructure for doing computation in the model.
And then a packaging diagram is not terribly important.
It's really focused on the organization of your model.
How do you scope things?
It's kind of a modeling diagram as opposed to a representation of your system.
Then last of all is the requirements diagram up top.
So again, I'll show you a good example of this.
But that's where you represent, how are your requirements related to your system?
And then you can see here what's been modified and what's been taken from UML and the new diagram types of requirement and parametric.
There were a couple of diagram types that were eliminated from UML that were pretty software-specific.
I think there is one called the deployment diagram, like how has your software been deployed across various servers or users?
Things like that.
That's not really-- that's a pretty software-specific thing, so in a general system you might not care about that all that much.
So that diagram was removed.
So let me quickly go over some of the syntax.
So this is an Interface Block Diagram, an IBD.
And this is the type of diagram that I find is really interesting representing these interfaces.
So here's the system engineering ontology we typically talk about.
So this is basically a model of an avionics board.
So you have things like voltage converters.
You have memory, volatile, nonvolatile.
We have an FPGA, which is our main computational unit.
We talk about these as being parts of a system in terms of system engineering ontology.
Then these are these interfaces or these lines right here.
So in SysML, the way we talk about it is these are part properties of the system.
It's kind of like an instantiation type of thing.
What we're saying is all of these parts can represent independent of each other.
And then you define a property of that part as being part of a different part, if that makes any sense at all.
So for example, this is a board.
This main electronics board is this whole block.
And then that has some blocks within this block, which represent the subparts that make up the board.
And these are called part properties of this overall block.
We have these green little boxes, which are called ports.
And again, that's to support this kind of system-independent modeling.
So you can model like a voltage converter independent of any type of system as maybe an input voltage and output voltage.
You can define what ranges those are, things like that.
And you can model those interfaces using ports.
And then these connectors, which are called connectors in SysML which represent the interfaces, represent how each part is tied into the kind of larger system.
And you could, for example, check that you don't have any empty ports.
If a part needs an input voltage, you could run a script that checks that all the parts have all their ports kind of satisfied.
That's something you can do with SysML.
So before I get into the case study, I want to talk a little bit about what you're going to see.
As Olly said, I work on something called REXIS, which is the REgolith X-Ray Imaging Spectrometer.
It's an x-ray spectrometer that's flying on NASA's OSIRIS-REx asteroid sample return mission.
I've been working on it since 2011, when I was a senior here all throughout my master's, and then now for my PhD.
Basically we're going to measure x-rays that are fluoresced from the asteroid surface in order to tell what the elemental composition of the asteroid is.
So that's our main science goal.
And that will basically enable us to categorize where the asteroid is within the different meteorite types that have been defined on the ground based upon existing meteorite samples.
So what I did for my master's thesis was modeled the design history of REXIS.
So how has our design evolved from the very beginning where it was very open ended, very abstract, and you'll see this, to the current design, which current in this case was CDR which was over a year ago now.
Right now the current state of REXIS is we're almost ready to mount to the spacecraft.
So it's very exciting.
Just to give you an idea of a timeline, this is something I'm sure Olly will talk about in this course, is the flow through the system development lifecycles.
So we have a system requirements review back in January of 2012, system-- I think it's definition review, April 2012, preliminary design review January of 2013, and then critical design review February of 2014.
So I created models at each of these design points in SysML and looked at, what are the lessons we could have learned?
We didn't use the SysML in REXIS.
I was kind of looking back historically, what if we had used it?
Could we have designed our system better in any sort of way?
So here is kind of a CAD representation of how a design evolved.
And I think you can kind of get the idea.
Back at SRR, a lot of things we didn't really know what they would look like.
We didn't know what the interfaces would be.
We didn't know what all the parts would be.
We had a little more development for SDR.
You can see there's more arrows.
The CAD is a little bit more detailed.
PDR, we had even more detail.
This was actually like a buildable design.
This turned out to not even be buildable.
And then we had more evolution between PDR and CDR to get to pretty much where the design is.
There's actually been some evolution after this as well, as sometimes happens with a new system.
But you can see just graphically kind of the increase in level of detail and level of fidelity of the state of our system throughout its development cycle.
And I'll show you that and how that looks in the SysML model as well.
So one of the things you can do if you have a SysML model, is as I talked about, you can run queries on it and pull out information that's very difficult or impossible to get with our current deployment practices.
So this is just looking at the different subassemblies within REXIS.
What are the number of parts in each of the subassemblies?
So you can see the general trend is up for all of them, as you would expect.
Some jump up very high.
Some kind of stay basically where they were.
But in general, they all increase.
And this is something you might be able to do with looking at like a parts list or something like that with current methodologies.
But it would be very hard to get this information, which is the number of ports per assembly.
So each interface has two ports.
So these numbers divided by 2 basically equal the number of interfaces that we have in each subassembly.
And you can see, again, there's a general trend of increasing number of ports as you go through the lifecycle.
So this is a piece of information you might want to use if you want to manage the complexity of your system.
And you say, this subassembly is getting-- it's way too many interfaces, way too many parts.
It's way too complex.
We need to think about how we've logically arranged our system and maybe how can we rearrange it to make it more understandable and easier to work with.
And then you can divide the two.
And you end up looking at how many-- this is ports per part in each subassembly at each of the design reviews.
And you can see trends here, too, which are interesting.
So you can see in the beginning we didn't really know what we were doing.
Some of these had a lot of parts, ports per part.
Some of these had very few.
They all ended up stabilizing kind of in between the three and five ports per part range.
And then you can look at the literature and say, well, typically systems tend to be between five and six ports per part.
So what does that mean?
Does that mean that our system didn't model it correctly?
That's one possibility.
It didn't model it to the lowest level of fidelity possible.
Does it mean that our system is too simple?
Does it mean that we're missing something that we haven't thought about?
Does it mean that our system, which was intended to be simple and cheap and implementable by students is actually achieving that goal because it's beneath what you typically expect?
Those are the types of questions you can ask with this data.
But this data is not easy to get with the current methodologies.
So this kind of very simple queries you can do gives you power on managing complexity in your system.
So let me now transition quickly to the demo.
Let's see.
There's no sound
I know, but I need to sync it up to make it [INAUDIBLE]
So while Mark is setting up for the demo, are there any questions about SysML so far, any observations you guys have?
Maybe at EPFL.
Do you see the similarities and differences between OPM and SysML?
What's the biggest difference between the two?
There's two really important differences
No questions?
Yes
I was going to ask--
Hang on.
Hang on.
Is there anybody at EPFL who wants to comment on this?
No, there is no comment from
OK. Good.
That's fine.
That's fine.
Mark, are you set up?
Almost
What I would say is-- so first of all, OPM has only one type of diagram, and then you go really deep, sort of a deep hierarchy.
SysML has nine different types of diagrams split between behavioral and structural.
So that's one difference.
And then the other is that the SysML is fundamentally object-oriented because it comes from object-oriented thinking and software, whereas OPM has objects and processes sort of at the same level.
Those are two of the most important differences
All right.
So what I'm showing you-- can everyone see the screen?
Just let me know if you can't see the screen.
What I'm showing you right now is a tool called MagicDraw.
There are basically a variety of tools provided by commercial vendors that enable you to build and work with a SysML model.
So SysML is a language, and then it's implemented in tools.
And this just happens to be one that's fairly well, fairly widely utilized.
Unfortunately, it's quite expensive.
But anyway, so what I'm showing you right now, and this is going to be a little bit difficult because, as you can see, you need a big screen.
What I'm showing you right now is a requirements diagram.
So you can see that right here.
It's a requirements diagram.
And each of these blocks, as you can see by the tag here, is a requirement.
So this is one of our operating criterias.
Well operating temperature of all our components shall be maintained within operability limits.
Straightforward requirement.
And you can create these satisfy relationships between that requirement and the components in the system that must satisfy that requirement.
So right now, this is being done at SRR, as you can see up here.
So this is very early in the design.
So we don't have it broken down fully to all the components.
But we have-- here is our main electronics board.
It has to satisfy that requirement.
Here's our radiation cover.
It has to satisfy that requirement.
So you can tie these two things, the requirement and the part of the system that must satisfy the requirement.
And here I've tied it to parts.
So these are called blocks, which are the fundamental unit in SysML, is a block.
So these represent parts of our system.
But you could tie it to a property of that part if you wanted to.
So then you get into the situation I talked about earlier where you have the requirement is on the mass of it must be less than this.
You can tie it to the mass of the system itself, as opposed to the system.
You can tie it to the actual property, which is very useful.
So let me quickly show you one of the cool things.
If I delete these requirements, I'm not actually-- you can see there's no requirements.
I'm not actually deleting them from the system itself, but I'm just doing it as I'm removing it from the diagram.
But if you wanted to, you should be able to look at related elements of the different blocks.
So I just clicked on a block, and I can choose to show all the requirements, all the things that are satisfied, the requirements of that component of the system satisfies.
And these requirements pop up.
So this is showing how the diagrams themselves aren't the model.
There's actually backend to the model.
And you can kind of work in the diagram and show or hide things however you want.
But the information is actually kept behind the diagram, so to speak.
So my research looked into typologies.
So let me just kind of show you a little bit about what I did.
So this is a block definition diagram, again, defining all the parts of your system.
And I just want to give you kind of a high level idea of the type of things that we saw.
So it's big, first of all, very big.
These are all the parts of the system.
So we start high like the mission context, and the mission context contains things like the environment and the spacecraft and then REXIS.
And then you can break down REXIS.
We have these various subassemblies broken down to parts.
And that's how you get this tree structure.
And then this tree structure can be tied together.
I can zoom in, but it's big again.
So these are all the parts of our system.
Let me give you an example.
So we have a couple of boards that we call our detector electronics.
And those detector electronics have various ports.
One thing they had to do, they had this port in here and this interface here, which connects to our CCDs.
So this is showing how you can build interfaces and SysML.
So here are the green boxes, again, or the ports.
The lines are the connectors, and they're defining all the interfaces.
And you can see at a high level how complicated things get very quickly.
This is the earliest, most abstract version of our design.
And it already has a lot of complication.
One thing you can do-- I talked about running scripts.
I'll show you how that works.
It's quite easy.
So I just ran a script on the model that told me to find the number of parts in the scope that I defined and the number of ports.
And here's the output of that model right here.
So the script itself is not even very complicated.
It's like 50, 60 lines of code.
And immediately I can pull out how many ports, how many parts, things like that, information down my system.
So let me kind of take you briefly through the development process.
So that was, again, the highest, the most abstract, the earliest version of our system, SRR.
This is SDR.
You can see it's starting to get a bit more complex.
And then you can go over to PDR, and it starts to get really scary.
And you can go to CDR, and it's just a nightmare.
So I created all these systems, all these models by hand.
You can actually build the model with the script, if you would like.
You can basically do things like define a pattern and then apply that pattern to all the parts of that type.
That's all possible through the API.
And just to show you how much of a nightmare it was at CDR, let me run the same script on the CDR model.
You have 230 parts and 900 interfaces.
And this was not even modeling to the lowest level of fidelity possible.
I didn't, for example, model all the components, all the capacitors, resistors, op amps, and stuff on the board.
And now you can already see it's quite large.
Kind of the idea behind these models would be to extend this to the lowest level in a real system and use this, basically use all the capabilities that you had with the model to really manage your complexity in a way that is just not possible currently.
And there's no way you can really mentally keep track of all these interfaces and understanding how your system is working.
So having this modeling capability and querying capability is really, really powerful.
Yeah?
What was used during REXIS for the systems engineering?
You had applied this after the fact.
So what was used to create these block diagram or [INAUDIBLE]
Do you mean during when we were developing REXIS?
Yeah
We basically didn't have this.
So we were relying-- as you would typically do upon the capabilities of the system engineer or the team-- you'd to have documents.
We have a ton of documents.
But I was talking about how things weren't buildable.
I found a situation where because of the way we had done our thermal system, we were dumping spacecraft heat to space, which you don't want to do.
I can explain why, but you don't want to do that.
And that's something we didn't realize at the time.
If we had tried to build it, we would have had this property of the system that we didn't know actually existed until I went back and looked at the model.
So we definitely miss things.
And this should have improved the design process, if we had been using it
Great, so Mark, sort of to wrap up, because we've got to switch over to [INAUDIBLE],, what's your recommendation for, let's say, students in the class got sort of intrigued by SysML.
What's the next step?
There's a couple different ways.
Certainly if you're interested, let me know, and I can give you resources to further your understanding.
As I said, working with these tools can be expensive to get these tools.
So I can help you understand what that would take.
There are beginning to be some companies that will do like SysML training courses that will sit down for a day or a week and teach you SysML, basically, how to work with the model, how to build the model.
I took one of those courses.
It was really great.
Yeah, talk to me if you're interested, and I can steer you down the right path
Great, thanks, Mark
Sure
Very good.
So we did in 20 minutes what usually takes about a week, right?
There's a lot more depth.
There's a lot more depth
Great, so we're going to switch over now to Modelica, which I think is-- we're going to maybe run a couple of minutes over today, but I think it's important we cover all three languages.
So let's get the slides back up.
And I'm going to switch here to Narek.
He's another doctoral student in the group.
And so introduce yourself and then tell us about Modelica
Thank you
Are you using this computer for slides or this computer?
For the demo, I'm going to use this one
This one?
This one
This one
Thank you.
So hello, everyone.
My name is Narek.
I'm a doctoral student at AeroAstro here.
My background is in gas turbine engines, and specifically I've been looking at concept generation of gas turbine engines, automated concept generation.
And the way I got acquainted with Modelica was that I needed to be able to rapidly reconfigure different concepts and simulate them mathematically, do physics-based simulations.
So in contrast to the first two languages, Modelica is a lot more about rapidly being able to build physical models of systems and reconfiguring them and reusing them for later on.
So like I mentioned, Modelica is primarily about modeling physics-based modeling of systems and rapidly being able to reuse models and reconfigure them.
It's a language, again.
It's not a tool like the first two that you heard about.
There are many different tools which implement this language.
But I'm going to start off with just describing how the language works and then go on to describe which tools that you can use.
In contrast to SysML, there are a couple of really good free tools that you can use and rapidly get into.
And there are a lot of libraries that you can use with hundreds, even thousands, of actually basic components that you can use for modeling.
So it's a declarative language.
And what I mean by declarative is that in sequential sort of programming, you write commands.
And you make assignments to various parameters.
Here, you just describe the governing equations of the components you want to simulate.
And there's no particular order in which you do this.
The models are acausal.
There's no direction to flows.
All you really need to do is describe what ports, like Mark mentioned.
It's similar in this situation-- what kind of ports you have, what kind of interfaces the components can have with other components, and the governing equations and the parameters and variables.
It's a multi-domain modeling language, so it's like agnostic to what kind of domain you're working in.
It's not particular to electrical engineering.
For example, I'm going to be showing an electrical engineering example and also show a gas turbine engine example with aeroelastic vibrations.
So you don't necessarily-- you're not attached to any specific domain.
It's also object-orientated, and it enables you to decompose systems into subsystems or recombine them and look at them at various levels of abstraction.
It's designed to be efficient.
So this is a quote from Professor Peter Fritzen at the Linkopings University.
So these are just about the scale of problems that you can solve with the Modelica language.
Obviously, it depends on what kind of equations you're talking about, but it's designed to be a very efficient way of simulating systems.
So I really want to talk a little bit more about this idea of acausal modeling that I mentioned before.
So on the left hand side is an assignment.
And that's typically what you do when you program in MATLAB, just the MATLAB normal scripts.
f is assigned to ma or p is assigned the value of rho rt for the equation of state of a gas.
And what that means is that you know what the mass and acceleration are, and you figure out the force.
You assign that value to the force.
In Modelica, there's almost none of this.
It's equations not assignments.
And what I mean by that is this equation can be written in any which way, as long as your system has the same number of equations as unknowns.
The tool that you're using will interpret the language and will solve your problem.
So you can write this in any which way you want in any order.
As long as your problem is properly constrained, the tool that you're using will interpret it and solve the problem for you.
So just to go into a little bit more detail, all of Modelica's and also other acausal modeling language that I'll mention in a little bit-- which one of them is Simscape-- have essentially three parts.
They're designed to be extremely simple.
The first, like Mark mentioned, are ports.
It's essentially identical to SysML in a way.
Ports are the ways with which components can share information, material, or energy, for example.
You can define any kind of port you want.
All you need to really do is define what kind of parameters it carries-- like for gas, for example, temperature, pressure, and mass flow; or for electrical ports, voltage and current.
The second part of any kind of model that you're building in Modelica or Simscape, which I'll mention a little later, are variables and parameters.
So you just declare those.
And the third part are governing equations.
The point is that there's nothing else.
It's just that.
And I'll just brief briefly go through a very, very simple example.
So for example, a capacitor.
This is the entire code for a capacitor, and this is what will generate a visual image of a capacitor with the correct ports for you.
First you have pins that carry voltage and current.
So the key thing to notice here is that there are fundamentally two types of variables-- flow variables and normal variables here.
Flow variables are ones for which the Kirchoff's current law applies.
So mass flow, for example, every time you connect 15 components together in a network, mass flow into that network needs to be conserved.
So the sum of mass flows into any node has to be zero.
That doesn't apply to the standard variables.
So then you essentially define the parameters, the variables.
And you need to define the governing equations.
And that's the capacitor for you.
There are slightly more complicated components that you can use, for example a pressure drop component.
Did I lose my sharing?
No.
In this case, the main thing to take away from this one is that if you have complicated mathematics describing the fluid mechanics in a component, you can actually initialized with one model and then go to a full turbulent simulation.
That's what this is doing here.
So just to get to the tool side of the equation, the language I just showed you, it's the same across all the tools.
But there are many different tools which you can use free and commercial to actually run these models.
The main one that you'll be using if you want to get deeper into this is OpenModelica.
It's free.
It's actually become quite user-friendly.
And there's a link in the slides from which you can download it.
There's one from Wolfram.
It's integrated with Mathematica, which is quite useful.
And there's a free trial of it as well.
There's Dymola.
There are other ones.
But mainly, I think OpenModelica is the one that you guys will be using
OK, so in other words, I think we're going to wrap up and then have you back next week

Are you around next week?
Yeah, yeah.
sure
Just to stay around.
So basically we're going to finish this lecture next week.
I think it's important enough that you really see the demo and see sort of the-- and it actually ties in kind of nicely with next week's topic is concept generation.
This is the next step in the V. And since Narek, your research is also on concept generation, it'll tie in nicely.
So I think we're going to stop here for today.
So you heard about the general idea of system modeling languages that are rigorous, that have ontology, syntax and, semantics.
There's different of these that have been proposed, developed.
Some are used more than others.
There's really important differences between them.
So OPM is very conceptual.
SysML is based on UML and has these different type of diagrams and can really help you flesh out your design in more detail.
And then Modelica allows you to build these blocks.
It's acausal or declarative.
And you can actually simulate the physics of the system pretty readily.
So the big picture here, just to wrap up here, the big picture is the following.
And we'll come back to this next week.
The big picture is basically that system engineering is in a transition phase.
The classic way of doing system engineering really for the last 50 years is on the left, document-centric.
Write your requirements.
Do your drawings.
Even CAD, you know, computer-aided design, is great, but it only essentially does the mechanical part of the design.
And so the result of that is as you get-- even REXIS, REXIS is a box, like shoe-box size, basically.
And it's going to fit on a much bigger spacecraft.
And you saw how much complexity is there.
And it gets very, very difficult to manage all this information, to prevent errors, oversights, any change that you make.
It doesn't propagate automatically in these documents.
So the transition is happening to the right side, a model-centric way to do system engineering.
And think of paperless engineering.
Everything you're doing is in a model.
The models are linked.
The models are executable.
The models automatically propagate any changes that you make in requirements or design.
We're not quite there yet.
But that is where things are moving.
And so keep that in mind.
So there's no new assignment this week.
So next week, we have A-2, which is do, the requirements.
Please let us know if you have any-- we're here for you.
So I'm going to have office hours now on the WebEx.
You have the link.
If you have any questions about A-2, don't be shy to email me or [?
Yuanna ?]
or [?
Leislu ?]
at EPFL.
We're really here to answer your questions.
So next week topic, we're going to finish on Modelica, and concept generation is going to be sort of our-- creativity concept generation is our main topic next week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So let's see.
Mr. Sticky.
Let's see, EPFL.
Who picked Mr. Sticky?
Anybody?
Yeah.
So the definition is a cylindrical container containing a date covered with a [?
basic ?]
substance that can be deployed in order to attract and capture insects
Good.
I'm glad you added the last part because I was worried that you weren't going to talk about the function.
So it's good.
I think it's good.
You started describing the form, and then you explained what the function is at the end.
So that's good.
Anybody else do Mr. Sticky here?
Yeah.
Do you want to share?
So we said it was a portable canister deployable, fly-attracting sticky tape, fly-catching mechanism
All right.
Yeah, I think you've got the right elements.
I think it needs to flow a little better, but you've got the right ingredients there.
What about the i3.
Anybody?
Yeah, go ahead.
Justice
You said it was an efficiency and urban-optimized transport vehicle made by
Say that again?
Can you turn on the mic, please?
Does yours work?
Do it again
We said it was an efficiency and urban-optimized transport vehicle made by
Urban-optimized transport vehicle.
I think it's good.
I think you're missing the effect that it's an all-electric, the fact that it's fundamentally an electric vehicle.
I think it's key to the i3.
Because you have you have other urban-- like the smart car.
And so architecturally, distinguishing feature is definitely the electric drive.
So I think it's good, but I think you're just missing a little bit on the technology there.
What about i3 and EPFL?
Anybody do the i3?
Nobody
Nobody.
Anybody else here?
Yeah, Veronica
Thank you.
Sorry.
A transportation system responsible for moving people and products with an enclosed metal frame equipped with various safety devices using electrically-powered control and locomotion subsystems
OK. That's pretty good.
There's a lot there.
I would throw a question.
A concept you defined could almost apply to a Tramway as well.
If this was like a streetcar, don't you think it would apply to that as well?
So I think the fact that it's a personal vehicle, I think it's important.
So the key in this is, describe the concept using few words precisely, but to set it apart from neighboring concepts.
What about Rolex Center?
Yeah.
It's a single-layer building with multiple straw used as a library for people to meet and study
A single-story building using multiple--
--floors
Floors.
Oh, yeah.
So that's right
You have one layer.
However, it has this wave shape and therefore, you have something akin to two floors.
So the library is one something like the upper floor and the meeting areas, the noisy areas on the lower floors.
So you can connect both, but the noise doesn't connect that easily
Yeah.
No, that's very good, and I think that's the architectural.
And then the library is there.
That's for sure?
But isn't there a lot of other things or services provided in that building beyond the library?
Yeah, you have a bank, a cafeteria, student services.
But you could encompass it in a meeting area.
You meet and you do some stuff like eating, going to the bank
Right
I think we missed some functions
Yeah.
So I like your definition, but I think by pointing out the library, you're missing the other functions.
And my understanding is that the reason this was built was to put two things.
One, is it's a focal point for the community at EPFL.
We have a student center here at MIT.
People go there to do their banking, their eating, their meeting, and so forth.
So in that sense, it's pretty similar, but I think the Rolex Center is such an iconic building that it also serve a kind of a prestige function, to put the institution on the map in terms of it's a statement.
It's not just a utilitarian building.
Whereas, I would argue our MIT student center, it has very similar functions to the Rolex Center, but I wouldn't call it an iconic building.
It doesn't have that wow factor.
Would you guys agree with that?
Yeah.
Maybe it did when it was initially built.
I'm not sure, but very good.
So we could spend a lot of time on these, but really crisply refining and thinking about the concept is very, very important So let me very quickly go through the refrigerator case study to show how do we transition from concept to design.
So the first thing you do is understand where is the value-- the stakeholders and the stakeholder analysis and the requirements definition.
And so in order to do that, you ask the question, first of all, value identification.
Where is the value?
And so you understand that then who is the beneficiary?
That's a stakeholder.
The stakeholder has needs.
I'm using OPM here to describe this.
And then this is a funny thing here.
The needs have these little bumps.
This is meant to indicate a cloud.
This is not standard OPM nomenclature.
Just point that out.
So this means that the needs are somewhat vaporous.
They're not very well defined.
And then you interpret and incorporate some of the needs into goals, which become requirements.
And so the goals then are an instrument of the primary delivery value delivering process, which is your value proposition.
To then actually deliver that value you need to design the product, the product system, the product object, and understand the operand, the thing that is being operated on or transformed by the primary value delivery process.
That's a very abstract, high-level way to think about where's the value in the system.
But fundamentally, this is part of the reducing ambiguity work that you do as a system architect, as a system engineer.
And there is a recipe for doing this.
So first you start examining the operand associated with value.
What's really the thing that generates the value that the user, the beneficiaries care about.
Next you say, this is the attribute link.
What attributes of the upper are changing or being affected associated with value?
And so the attribute-transforming process is where the value is generated.
So for food, I think we briefly talked about this before.
Usually people will say, when you think about a refrigerator, keep the food cold.
But if you step back and think about it in a more abstract way, it's really about preserving food or reducing the spoilage rate of the food.
So the refrigerator effectively becomes a food spoilage rate reduction device.
and I know that sounds terrible, but that is really what it's about.
So when once you think that way, you can really start being creative and focus your creativity.
So here we have the food, and our goal is to reduce its spoilage rate.
And then you can ask, well, how can we do this?
What are all the different ways of reducing the spoilage rate of the food?
So from among the system operating processes, we then specialize and pick a particular one for our concept.
So beside chilling or keeping the food cold, we could irradiate the food.
We could we dry the food.
What are some other ways that we can reduce the spoilage rate?
So irradiating, drying, chilling.
Sam?
Preserving
Preserving.
So you add chemicals essentially.
EPFL, how else can we do it?
Spoilage rate reduction?
Using chemicals?
Yeah.
So that's chemical.
That's right.
What else?
Keep going.
Because food is pretty essential for humans, this is actually one of the areas where humans have been very creative.
There's a lot of ways to do this.
So keep going
Yeah, for example, for conserving grapes, maybe you do wine, which is a process to conserve this wonderful [INAUDIBLE]
Yeah.
Please keep some wine for us too, please
Beer is the same
Yeah.
So marinating it, vacuum packing, smoking, on and on and on.
My favorite is actually eating it.
What do bears do?
How do bears conserve food?
Fat
Fat.
Right?
So bears, they actually consume it and then transform it into fat, into a different storage form, store it inside their bodies, and then as they hibernate, that energy, that food is being gradually consumed.
That's a very different way, but it's essentially the same function.
So that's the key idea is start thinking in this abstract way, and all of a sudden all these other possibilities become possible.
And that's really the cool part about design and creativity.
But eventually, you have to pick a particular concept.
So the chilling part is not enough.
That's the function.
And then you say, well, we need a chiller.
In order to chill, we need a chiller, and there are different types of chillers, like a cooler or refrigerator.
And it's the combination of this specific way you're going to operate the system.
The element of form and then the specialized element of form, in this case the cooler, that combination is what we call concept.
So once we have that, we can start managing complexity, decomposing function and form, so our system operating process gets decomposed into the primary supporting processes like interfacing, powering, controlling.
And then our system object, you can decompose it into different elements, supporting systems, the operand, the operator, and so forth.
And then we can start connecting them.
So let's look at for chilling the food, let's look at a cooler and a refrigerator in terms of this decomposition.
So here's a picture of a very simple cooler that you would take out on a picnic.
It has the chilling function, and then the sub-functions when we zoom in are holding the food, exchanging the heat between the food and the environment, reducing the heat load, interfacing, connecting, powering, regulating the temperature, et cetera.
And then on the form side, the cooler itself is very simple structurally.
It just has a bottom, a box with a bottom and a top.
And then we have the ice, the food supporting the surface, the external heat load, ambient light, and the operator.
What's surprising here is when you map the two, there's a surprising amount of complexity.
It's not a very clear one-to-one mapping.
Let's just look at the ice.
So here's the ice, and you can just follow these links to see what functions, sub-functions does the ice support-- exchanging heat, powering, and regulating temperature.
So exchanging heat means, essentially, the ice itself acts as a heat exchanger.
Just the surface of the ice is where the heat exchange happens.
So what forms can you put ice into a cooler?
What are the different shapes of ice that you could put into a cooler?
Like an ice pack
You could put ice pack or you could put a block or chips.
And if you put the same quantity of ice, but you put a block of ice, you put little chips, what will be the difference?
Will there be a difference?
Surface area, right?
So the speed at which the ice will melt for the same given external temperature is going to be different.
So there's an attribute of the ice, which is essentially its quantity, but also it's form that will influence the-- but that's the heat exchanger function.
The powering function is pretty clear.
That's essentially the energy storage right there.
And then the third one is regulation.
How does that work?
How does the ice provide a thermal regulation in the cooler?
Physics 101.
Yeah?
Go ahead
[INAUDIBLE]
Right.
So as long as you have any ice left in the cooler, what will be the temperature inside?
Not opening and closing, but if you keep it closed, what will be the temperature in the cooler?
0 Celsius
0 Celsius.
As long as you have ice left, this is the phase transformation, you're at that point.
As soon as the ice is all melting, the temperature will rise.
So the phase transformation of the ice from solid to liquid is what, in fact, is the regulation mechanism inside the cooler.
So even though you think of a cooler as being something super simple and trivial, once you start listing its internal functions and how the top, the bottom the ice itself, how they interact and support those functions, it's pretty complex.
And you can go very detailed here, even for something very simple like a cooler.
One question and then we'll talk about the refrigerator and how it's different in a minute.
One question I often get is what's the difference between system architecting and system design?
Isn't that the same thing?
And I think they're somewhat different.
They're overlapping, but there's a distinction.
So architecture selects the concept, the decomposition, mapping of function to form.
And architecture essentially establishes the vector of design variables.
What are the key design variables and operating parameters of the system?
Design, then, given that, selects the actual values for those design variables, and then you can optimize.
So if we look at the example of the cooler here, we have our cooler with the box in the bottom and then the ice, and we can decompose the attributes of that.
So the box with bottom has length with height.
It has a wall thickness.
It has a type of material.
The top has thickness and material, and the ice, like we just talked about, has quantity, surface area, and maybe initial water content.
And those are the operating parameters, and on the upper right, those are the design variables.
So when you are making a material choice, we're going to use PVC.
We're going to have a 2.1-inch thick wall.
We're going to use this much insulation.
This will be the aspect ratio of the cooler.
It's going to have wheels maybe, so we can pull it easily.
Those are essentially design decisions.
You're instantiating that concept.
But the fact that it has a bottom and a top and it's hinged and it uses this phase transition as the regulation mechanism, that's conceptual design.
That's system architecture, and you need to do both.
But they're not quite the same.
Is that pretty clear?
That's a very important distinction.
So let's look at the form function mapping for refrigerators.
So this is actually a big difference between the US and Switzerland.
People in the US, we like to have big refrigerators, big gallon of milk, and refrigerators in Switzerland are much smaller.
I'm not sure why.
Maybe you go shopping more often, but it's definitely one of the big differences.
But here we have essentially the decomposition of the refrigerator.
So we have racks.
We have the air.
We have-- I guess freon is banned now, so we should use some other.
This is a refrigerant, a working fluid, the insulation, the feet and rollers, the frame, the electric motor, sensors, controller doors, lights.
And then we have those functions, essentially, the same functions we had for the cooler, holding the food, exchanging the heat, powering, regulating.
But the difference is that we have much more of a one-to-one mapping.
So in the refrigerator, each of these elements supports, essentially, one of the primary sub-functions.
So the form function mapping in the refrigerator is actually much simpler.
It's a much simpler form function mapping than the cooler.
And you say, well, I thought a refrigerator is much more complex.
How can that be?
Well, the real complexity comes in when you look at the form form mapping.
So this is then the decomposition of the refrigerator in terms of all the elements of form and then how they relate to each other.
And I'm not showing here all the sub-processes, but you can see that the mapping and the relationships are much more complex than in the case of the cooler.
So to wrap up on this piece, when we do a conceptual design or system architecture, you have two major activities.
One is concept generation.
So take the requirements and think creatively about how these requirements could be fulfilled.
So that means understanding the operand, the attribute of the operand that you're trying to affect-- it could be multiple here-- the intent attribute-- this means the required, desired state-- what is the system operating process, and what is your major element of form.
So that's generating this and specializing that.
That's concept generation, finding systems that do the right thing.
And then once you have several concepts, you've got to select among them, which we'll talk about next week.
That's our topic next week is concept selection.
So finding systems that do the right thing and do it well, deliver value, and comply with regulations, standards, and so forth.
So there's going to be consumables involved, side effects, like noise, waste heat, pollution.
The operator, how skilled does the operator has to be or how autonomous is the system.
What is the reliability, safety, cost, the quantity.
All those things are describing the concept and its instantiation in more detail and will help you do concept selection.
Do you do you see the difference?
Concept generation is take the requirements and come up with the fundamental form function mapping of how the requirements can be met.
Then you instantiate that in terms of designs and then you can evaluate those and compare concepts using other criteria like consumables, quantity, how skilled does the operator have to be, et cetera.
So those are fundamentally different activities.
So let me summarize on system architecture and then talk about the NASA approach and then creativity.
So architecture requires consideration of both function and form related through concept.
It's about starting with the operand.
What is the thing that the beneficiary, the stakeholder cares about, and how do we transform that?
Concept then elaborate these into architectures that have form function and structural complexity.
And then the goodness of an architecture is really a pretty complex concept where we have multiple objectives to satisfy, including performance, resource utilization, cost, operability, safety, capacity, and so forth.
And we'll defer that to next week.
So let me briefly talk about the NASA approach to this and then talk about some methods and tools for concept generation.
So the NASA approach is basically described in the system engineering handbook in the SE engine as step 3 called logical decomposition.
And so the logical decomposition process, as described in the NASA standard, is used to improve the understanding of the technical requirements and the relationship among those, transforming that initial set of requirements into a decomposition.
So the idea that we need to partition the system and then derive lower-level technical requirements based on that lower-level definition, and that's what's called architecting.
Getting back to our high-level system design process, this is, again, that diagram that we've looked at several times already.
You can see the red box is where this happened.
So we started with mission authority, stakeholder expectation, and then defining those high-level requirements.
Level 0, level 1 requirements, but then you get stuck.
You get stuck because you have to make some decisions before you can go to lower level requirements definition.
And that's what's known here as functional and logical decomposition.
Once you've decided on a composition and you can carry several compositions with you for a while, then you can do the lower-level trade studies, derive and allocate lower-level requirements refine your CONOPS, and so forth.
And then do functional and performance analysis to see whether you have enough detail.
Is it workable?
Is it safe?
Is it reliable?
And if yes, then you can select that as a baseline, if not, you might have to go back to the red box, which means that architecture didn't work.
We have to look for a different decomposition or different architecture.
And if that doesn't work after multiple iterations here, you might have to go back and change the requirements, because you come to the conclusion that the requirements are not really achievable.
So some examples here of decomposition models, here's a timing diagram.
On the right side, you have a state diagram, the different states that the system can be in.
So you can see this relates very strongly to the system modeling languages that we talked about.
So you use the system modeling languages to decompose and define the system in more detail.
And we really talked about much of this already.
And in terms of the logical decomposition flow diagram, you start with your basic high-level requirements and measures of performance.
You essentially do your decomposition.
And then on the right side, you come out with the lower-level derived technical requirements, logical decomposition models, which would be essentially a description of your different subsystems and the logical decomposition work products, which are essentially lower-level definitions of what these subsystems look like.
And then you can go off and do the detailed design and then the testing verification and so forth.
So it's essentially focused on decomposition, which is an important part of architecting, but it's not the only thing you do.
So let me talk about methods and tools for concept generation.
So I'm going to start with this.
This is another really fun thing we do in the system design and management program, which is a full-year program, is the we call it the creativity workshop.
So what are different ways of stimulating or organizing creativity.
And what I'm showing you here is-- that's essentially a mind map of how to think about the creativity space.
So I'll briefly go through this, and then we'll look at a couple of examples.
So one idea is that creativity is better if it's a group process, that people stimulate each other.
And so this whole group up there is called group dynamics.
These are all different methods for stimulating creativity using groups of people.
And some of these are authors that have written and methods, so de Bono, Six Hats, powwows, mind boggling, workouts, creativity workouts, these are all different variations of group dynamic processes.
The one that I'll talk about in some more detail is brainstorming.
This is probably the best known.
What's not so much known about brainstorming is that there's a right way to do it.
On the upper right branch, creativity and system architecture, this is just describing the importance of it, the three themes we talked about-- creativity, ambiguity, complexity, different types of innovation, radical innovation, modular, or incremental innovation, and so forth, and the high leverage that it has.
The next branch are called models of creativity.
What that essentially means, and I'll give you one example here, which is Leonardo da Vinci, is understanding people that universally are claimed as having been very creative thinkers.
Why were they creative?
What was their recipe for success?
Below that, we have structured processes, which are essentially trying to organize the creativity, which seems like an oxymoron, but there are actually ways to have a structured process to stimulate concept generation.
And we'll talk about very briefly mind mapping and then morphological matrices.
And then we have this whole area here, which I'm going to mention, but we're not going to do as part of the class, which is stimulants.
So this is the idea that somehow people are more creative when their brain, when you put yourself into some other state.
So bio-inspired design would be you go in nature, or you read books about seashells and animals and you really try to understand from nature-- and bio-inspired design is a very important field of research now.
It's pretty serious.
So you put yourself in nature and be inspired by what you see.
Random inputs, provocations, challenges, and then things like alcohol, and even drugs.
So a lot of the music that was done in the hippie age in the '60s, I mean, a lot of these artists were consuming large amounts of drugs and alcohol.
And there's a big discussion on is this fundamentally why they were creative.
So I'm not advocating that.
I'm just I'm just telling you that there is this idea that you can stimulate creativity in these different ways.
So let's talk about mind mapping.
So this is an example of a mind map that I really liked.
This is from several years ago.
This is from a student that took the system architecture class.
And so the idea of a mind map is that you look inside yourself and you try to put down on a map different ideas and concepts.
So in the core of it, you have the key focal point of the mind map.
In this case, it's system architecture, and then you have these branches coming off.
So the class itself, the skills, the concepts, the themes, and then it almost looks like a neural network.
You branch off into the sub ideas and sub concept.
And in order to really make it memorable, you draw it by hand even though there is software for doing this.
But I really like this, drawing it by hand the old style.
And then you add icons and symbols and colors to really make this sticky and memorable.
So you can look at that.
I really like this example.
And it probably takes a couple of hours to do a really good mind map like this.
But the idea is that by doing this, you're going to develop your ideas and concepts.
And there's books about mind mapping.
I mean, it's a whole industry almost.
Brainstorming.
So by the way, who has done brainstorming and organized brainstorming?
Who's been part of a brainstorming exercise?
Do you want to describe it, how that worked?
Make sure you use the mic
So we started with our problem, and if I remember it right, it was to redesign a coffee mug
This was for a class?
Yes.
And then basically, for the first part, any idea could go.
And the only role was you couldn't criticize anyone else's idea.
So he basically tried to come up with as much as we could, wrote as much as we could on the board, things along those lines.
And then once we had all of our ideas down and some people built off of other ones and then it was oh, but we can also add this.
And then we started to look at well, we can't really believe that.
This is going to be way too expensive and started down selecting from there
Did you take a break between the two parts?
Yeah
So yeah, you described it very well.
So the key idea is there's some rules for how to properly do brainstorming, and some of them are listed here.
And then I have another on the next chart, there's sort of a step by step.
So it's really try to remove creativity barriers, stimulate each other.
There's an ideal group size, and it says 5 to 10 here, but I should probably revise this to be-- what do you think?
7 plus minus 2.
If you try to do brainstorming session with 30 people in the room, it's too big.
It's not going to be that productive.
So use of intuition, associations.
What's important is that you have a clear idea of why you're doing the brainstorming session.
So there's some solution neutral question, like how can we improve a coffee mug.
What did you come up with, by the way, at the end?
Did you have a result?
I think it was we were going to redesign the handle so it would be more ergonomic.
But, yeah
So what can be done too, how can we improve.
There's got to be a driving question for the brainstorming session.
Actually, the first time this was described was by AF Osborn in this book in 1957.
There's why is brainstorming useful.
We can talk about that.
A lot of it has to do with this group dynamics.
How to organize and host a brainstorming session.
I'll talk about that next.
And then there's this killer sentences you should never say during a brainstorming.
Some of these are pretty funny.
And then what do you do with the results.
How do you actually then take the brainstorming results and use them for further refining or down-selecting concepts.
So here's a six-step process for doing a brainstorming session.
So you send out invitations a few days ahead of time.
And the idea is that people can think about this question so that when they come to the brainstorming session, their brain is preloaded with ideas.
That's the idea.
You don't just pull people in like five minutes earlier.
The idea is give a few days, not weeks, but a few days of warning so that people can think about this and come to the brainstorming session ready and charged to share their ideas.
7 plus or minus 2 participants.
There should be a facilitator.
This is somebody who is helping to moderate.
Participants take turns expressing thoughts, suggestions, ideas.
You should take notes.
So for example, these big whiteboards are great for that, with idea paint, the whole wall.
That's great.
Or you can do flip charts.
You can do different ways of capturing these ideas.
And then I think you mentioned this.
It's called the principle of delayed judgment.
So you're not allowed to criticize or particularly praise.
So you could, for example say, oh, this is the best idea we've had so far in this session.
Even though it's praise, it actually implicitly is criticism of the other ideas.
So avoid that.
Avoid the killer phrases.
And then the idea there is produce a large amount and diversity of ideas.
And then at some point, maybe 30 to 60 minutes, you end.
Brainstorming session that last four hours, the first hour is probably really good and then the second hour OK, and then the rest is everybody is kind of shot and, there's not a lot of new ideas coming.
Creativity killer sentences.
I highlighted a few.
This will never work.
We don't even need to talk about this.
Everybody does it this way.
I've already studied this problem for years.
Don't worry, I know I'm right, and et cetera.
How long have you been with this company?
Anyway, so that's the idea.
All right Leonardo.
Who's been to Italy or tour in France, or who's seen one of the exhibits know where his notebooks are on display?
[INAUDIBLE]
Yeah.
What about EPFL?
Have you guys seen these wandering exhibits about Leonardo's work and his notebooks?
Anybody had a chance to see that?
Go ahead
Yeah.
Actually, he do a lot of sketch, and he [INAUDIBLE] the fact that sketching is more important than writing
Yeah.
That's right.
So really, he didn't build a lot of his ideas.
So that's one of the-- did he actually then.
But he was a head of his time in many ways.
So he's really been identified as an exceptional individual.
So here's a book called How to Think Like Leonardo, Seven Steps to Genius.
And I'm not a big fan of these popular books, but this one is pretty interesting because what's been extracted from this is the seven da Vincian principles of creativity.
And they're here in Italian.
I'm just going to go through them very quickly.
So curiosita, lifelong quest for learning.
Dimostratzione, testing your knowledge through experience, trying things out.
Sensazione, continual refinement of the senses.
Sfumato, which is essentially also a style of painting, like the Mona Lisa is painted in sfumato style.
Mastering ambiguity, paradox, uncertainty.
Arte/Scienza is the whole brain thinking, left-right brain.
Corporalita, balance of body and mind, so a healthy mind and a healthy body.
And then connessione is interesting.
That gets close to system architecture, which is the appreciation of patterns, relationships, connections, and systems.
So the idea is that, this from Leonardo, his work, his way of thinking, these seven principles have been extracted.
And then you can say, well, which of these do I feel really resonate with me?
All right.
So we have only a few minutes left.
Let's move to some of the structured processes for creativity.
So the first one I want to mention is this is probably the simplest and the one that's used the most.
This is known as a morphological matrix.
So the idea there is that you try to define what are the key features, factors, or decisions that you have to make when you define a concept or an architecture.
So the key decisions are the rows.
Let's say there's n key decisions.
There are factors in the rows.
And then for each row you think about what are the number of possible alternatives for doing this.
And then you enumerate all possible combinations.
So an example here would be here's our morphological matrix.
And then one possible concept here would be A2, B1, C3.
That's our concept here.
And then you can see that for a full-factorial enumeration, you would have 27 architectures that you could generate.
So the number of architectures here is 27, based on this morphological matrix.
And I find this to be very, very helpful.
When the table gets too big, very quickly because of this being a product, this can really explode on you.
It can be very large.
And the big challenge with this, of course, is if you have many factors, you could generate many infeasible architectures.
Not all these combinations are actually feasible.
So the question then is, how do you prevent that, and that's where so-called architecture enumeration comes in.
And I'm not going to go in a lot of detail here.
But the idea is that through creativity, expert knowledge, and analysis you're going to define your components, which are essentially the rows in the morphological matrix.
But you're also going to establish rules that tell you which combinations are actually valid combinations and which ones are not.
And that, in fact, is [?
Narek's ?]
PhD topic is, how do you increase the number of physics-based rules rather than just empirical rules.
Because if you think about it.
If you apply rules that are just based on current practice, then you're just going to recreate concepts and architectures we already have.
You won't really come up with something fundamentally new because you've constrained the combinations to what people do now.
So the real challenge here is, between generating all possible combinations, many of which are infeasible and only the ones that we currently have, there's a middle ground there.
So that's architecture enumeration, and there's different ways of doing this at different layers of abstraction.
So here's an example of airplanes, different configurations of airplanes.
If you think about the tail of airplanes, we have the traditional tail with a lower stabilizer.
We have T tails.
We have we have V tails.
I mean, there's like 12 different tail geometries here.
And you could think of this for the wings, for the fuselage, for the engine locations, very quickly, you can generate thousands or even millions of architectures.
But at that higher abstraction layer, it's just a single tail.
So how do you combine these using compositional rules?
That's architecture enumeration.
So here's also an example from [?
Narek's ?]
work.
So at an engine, a turbo prop engine at a high level of abstraction, that's basically a propeller, an intake, a core, and a core nozzle.
And then to break that concept in further detail, the core itself gets shown at a lower level of detail.
And you can see that inside the core, you have, in this case, a single compressor, a burner, and then a turbine that drives the compressor.
And so one of the advances in engines has been from World War II to go from single-stage to two-stage, thee-stage engines, and so forth.
So the complexity has been going up, but you can actually generate through architectural enumeration, essentially, all architectures that have been built and that are known and have been certified through a organized architecture enumeration process.
Some of this can be done in Excel, for example, where you essentially list your components.
This is your library of components.
And then on a different sheet, you define all the different rules that allow you to combine different number of instances of these components into architectures.
And we'll post some information on this if you want to try this out for your concepts.
So let me summarize.
So system architecture is definitely very abstract, but it's also, potentially, the most influential activity that we do in system architecting.
The concept is mapping function to form.
We typically, in the conceptual design, don't do all the details.
We just go down two levels of abstraction.
So not all the details are defined.
The NASA approach specify or is really focused on this idea of logical decomposition, which is very important, but it's not the only thing we do in system architecture.
And then the really cool part, the exciting part in concept generation is the one that it's really a creative activity.
And when you look at the set of creativity techniques, you can think of group dynamics like the brainstorming.
That's used very heavily, but you have to do it the right way.
If you organize a brainstorming session, and there's some wiggle room, but if you violate some fundamental principles of brainstorming, you're not going to get the full benefit.
Models.
So thinking about really creative individuals and what drove them, what were their principles, and try to emulate some of that.
And then the structured processes, which include mind maps, morphological matrices, and then architecture enumeration.
So when you look at assignment A3, which is now out there, that's really what it is about, is you've done the stakeholder analysis and initial CONOPS, you have a requirements set.
So now be unchained, and within the constraints that are set by the competition, come up with different concepts.
And in the homework, what I ask you to do in A3 is try out at least two different techniques, a structured one and an unstructured one and then compare the results.
And this will be due in two weeks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so let's situate ourselves where we are.
So we're starting to really make a lot of progress here, moving in the V model.
Today's topic is session five: tradespace exploration, concept selection, and PDR-- preliminary design review.
So there's a lot to talk about, and I'm going to go relatively quickly through this.
So, first I want to talk about decision analysis.
Fundamentally, when you come up for PDR, you have to make a big decision which is, what concept are you going for?
What is your system architecture?
That's a big decision.
You don't have all the details yet.
The design is not fully done, but you've chosen your key architecture.
That's a big decision, and there's a whole branch of research and science called decision analysis.
So I want to tell you about that.
Talk about some of the issues in concept selection, and then give you some tools for doing that in a kind of organized way.
There's a couple of-- what I would say, simple methods Pugh matrix and multi-attribute utility-- that are relatively, I think, straightforward.
And then there's a bit more advanced concept called non-dominance-- Pareto frontiers, getting into multi-objective optimization.
And so, at a minimum, when you choose your concept, you want to choose a non-dominated concept.
And I'll explain what that means.
And then we'll close with discussing what is a PDR?
What do you do with the PDR?
What's the purpose of it?
So, here is another way to explain essentially the flow of things as we come up to PDR.
So the way you can think of it as, you start with a need, right?
And your requirements.
That's-- we've talked about that before.
And then you have this creative phase, which we talked about last time, right?
You come up with a lot of ideas, a lot of concepts, and the funnel that you see here is going up.
So each of these shapes here-- the circle, the triangle-- they're meant to represent fundamentally different concepts, different architecture, different way of satisfying the requirements.
And so there's a lot of different things you can do, and the funnel opens up.
This is sort of the size of options or alternatives you're considering.
And then you have to filter this down.
In the end, you can only build one system, right?
You have to choose one architecture.
And this filtering down happens typically in two steps.
One is what we call concept screening, which is more qualitative.
And then the actual concept selection is more quantitative, based on models, based on data, based on metrics.
And then you arrive at PDR.
So in this sort of simplified view, we've chosen the square as our concept.
And we've made that decision by looking at these different metrics.
So you can see that on this graphic here, we have metric one, metric two-- you can see the triangle is dominated, right?
The triangle doesn't do as well if we want to maximize metric one and two, so it wouldn't make sense to choose the triangle.
But the circle and the square-- circle does better on metric two, but less well on metric one.
The square does better on metric one, not quite as well-- and they crossover at some point.
So, whatever-- how you weigh this in decision making, the decision is made.
We're going to go for the square.
That's our concept.
And then you move to the actual system design, post-PDR, which is all the details.
So we're expanding the box now, the square, and designing all the details within-- the structure, the avionics, the software, the controls.
And that's design modeling and optimization.
And so here I'm showing you metric three, which is maybe another metric, and then design variable x1, which is some pretty detailed decision that you make within that concept, and you're looking for the optimal, in this case, minimizing metric three.
It's this point here.
And then that gives you your-- and I put this deliberately in quotes-- optimal design.
That's what you show at the CDR, and the critical design review, and that's what you go and build.
So that's roughly the flow of decision making.
Any question about that?
That general flow?
So what we'll talk about today is essentially this portion here, concept screening and concept selection.
So, since you make a decision, what is decision analysis?
So decision analysis, in general, is methods and tools for ranking and choosing among competing alternatives or courses of action.
And I just want to make-- the word alternative.
A lot of people use the word option and alternative sort of as synonyms.
They're not.
OK?
They're not.
Options and alternatives are not the same thing.
An alternative means if I choose A, I can't choose B and C. But the fact that you've chosen A, or the square, or whatever your concept is, means that you cannot choose the other ones.
An option means, well given that I've chosen A, I could also add something on top of A, right?
It's kind of like if you were buying a new car, are you going to get the winter package or not?
That's an option, but you've chosen a particular vehicle.
So, that's important.
So alternatives are mutually exclusive choices or courses of action.
So you need the alternatives, those are the components-- four components of a decision.
You need criteria, which means how are you going to evaluate and compare the alternatives?
Then you need value judgments.
You now apply the criteria to the alternatives to compare them, and then finally you need a decision maker, which is either an individual or a group with preferences to then make the choice.
Those are the four ingredients of decision analysis.
And if we show this somewhat graphically-- so here's our alternatives.
We can do A, B, or C. The criteria in this case are cost, performance, and risk.
Then we basically evaluate, in this case, option or alternative A, I should say, for cost, for performance, for risk.
And this gives us some score or number, and then we need to figure out a way to combine this to rank the alternatives.
And when it comes to ranking, there's a distinction between what's known as an ordinal scale and a cardinal scale.
So in an ordinal ranking all you really care about and do is say, what's the best choice, the second best choice, and the third choice.
You ordered the alternative.
But you don't know in an ordinal ranking or ordinal scale, whether one and two are really close, and then there's a big gap between two and three.
You don't know that, and you don't really care.
All you want is the rank order.
When you put things on a cardinal scale-- so this is a continuous line-- you actually get, in a sense, both the order but you also see how close the alternatives are.
So that's a cardinal scale as opposed to an ordinal scale.
OK?
So-- yes?
Please
How do you get a discrete value for something like risk or something like that?
And especially in an ordinal scale, if you have two things that are really close to each other, but the risk is kind of a blurry region, how do you--
So then they would be equivalent in terms of that criterion.
And you can actually have ties, right?
You can have alternatives that are tied.
And I'll talk about-- this is one of the issues in concept selection is, how do you do tie breaking?
Good question.
Any questions at EPFL?
I know this is very abstract.
But fundamentally, that's what we do.
Any questions over there?
Is it clear?
Yes
OK. Good.
So let's keep moving.
So what are the issues?
I think that's-- so multiple criteria, how do you deal with them?
We usually have multiple criteria and how do we-- and eventually, when you make a decision, when you pick a concept, you're somehow combining these criteria into a single measure, whether you're doing that explicitly or not.
So how do you deal with these multiple criteria?
We'll talk about that.
Here's the point that you brought up.
What if there's ties?
How do you break them?
Group decision making versus individual decision making-- who gets to really make the decision?
Right?
And this relates to the stakeholder analysis.
Who has the power to make the decision in the end?
And then uncertainty-- did we choose the right criteria?
Did we evaluate each option properly?
And then the big one is are the the best alternatives represented?
So you can choose among A, B, and C, but maybe there's this much, much better architecture concept D out there, but it's not in the decision set.
Because it didn't-- nobody thought of it, or some other thing happened, and it didn't come up in concept generation.
Right?
So there's a lot of uncertainties when you're doing this.
So you have to be aware of it.
So let me talk about two-- again, quote, end quote-- "simple methods" for doing concept selection.
The first one is called Pugh matrix, and then I'll talk about utility analysis.
So the Pugh matrix, essentially, is a very discrete method.
It uses essentially plus, zero, and minus to score alternatives relative to a datum.
A datum is a reference decision, a reference architecture.
And this is named after Stuart Pugh, who is a British professor of design, essentially.
And it's a very simple method.
And it's used extensively.
It does have some pitfalls, as well, but it's very well known.
So Pugh matrix And then the second one is called utility analysis.
So it essentially maps all the criteria to a dimensionless utility, this is between zero and one.
So zero is a completely useless system.
It has zero utility.
And one is a system that has perfect utility.
It can't really be improved.
You could try to improve it but your satisfaction would not increase, OK?
And this has very deep mathematical theory, particularly the Von Neumann and Morgenstern utility that was basically developed in the mid-twentieth century.
Anybody seen the movie A Beautiful Mind?
Yeah?
What is is A Beautiful Mind about?
It's about John-- mathematician John Nash who's a mathematician who won-- I forget exactly what prize he had won
He won the Nobel Prize
Was it the Nobel Prize?
Founded game theory
For what?
For game theory, right.
And so one of Nash's contributions was the idea of a Nash equilibrium.
So we have multiple decision makers in gaming, and the question is-- you have your current strategy, but is there a better move you could make?
And if you could make a better move that improves your satisfaction, or your utility, but by doing this, you decrease somebody else's utility, you may not go there.
So if you reach a Nash equilibrium, that means that everybody believes that this is the best they can achieve.
And how do you measure that?
Well Nash actually developed his Nash equilibrium and game theory building upon theories of Von Neumann and Morgenstern.
So there's a very deep theory behind it.
I'm not going to go very deep into it, but so you know that there's a lot behind utility analysis.
OK, so let me just talk you through the basic steps for doing a Pugh matrix analysis.
And I'll show you a very simple example.
First step, you choose or develop the criteria for comparison.
So, what's important?
How are we going to decide this?
Selection.
And obviously this should be based on a set of system requirements and goals.
Now, there are two flavors of requirements.
Remember, what-- let's ask here at EPFL.
You remember the two flavors of requirements?
What were the two flavors of requirements?
[INAUDIBLE]
I mean, there's actually six types, right?
There's functional requirements.
There is constraints.
There's interface requirements, and so forth.
What I'm trying to get at is there are requirements that use shall statements and then there's requirements that use should statements, right?
So the shall statements are what?
What do the shell statements imply?
So basically, for the shall, it's compulsory, and for should, it's a goal of the system
That's right.
Exactly.
So, in some sense, the shall state-- the criteria associated with shall statements don't necessarily make the best criteria for the Pugh matrix, because by definition, the concepts that you're evaluating should all satisfy the shall statements.
Because if they don't, they're essentially infeasible, right?
And you shouldn't select among them in the first place.
So architectures, or concepts, or alternatives that violate shall requirements do not satisfy the must have requirements.
And you shouldn't-- so you shouldn't use criteria that are associated with hard constraints.
That's what I'm trying to say.
Now the requirements that are associated with should statements, those are good criteria for this Pugh matrix, because the degree to which you satisfy this, the goals are somewhat variable.
Therefore, you can then compare amongst alternatives.
OK?
So that's number one.
Number two-- thank you, that was good-- select the alternatives to be compared.
So that those are coming out of concept generation and one little bit tricky aspect here is when you're putting together the alternatives, the concepts to be selected from, they should be represented at a similar level of detail of abstraction.
It is not good to have one concept that's really detailed, and you know a lot about it, and often it's kind of an existing system or something you've done before, and then there's this other concept that is very fuzzy.
Why shouldn't you compare something very detailed with something very, very poorly defined?
Why do you think that's not a good practice?
What do you guys think here?
Yes
Is it like-- it's not fair, because--
That's right.
It's a fairness issue, and because the concepts that are poorly defined often, you're too optimistic about them.
That's sort of the typical-- this looks really good, very promising, but it looks very promising because it's kind of ill-defined.
So that's important.
Try to represent all the concepts at about the same level of detail when you make the comparison.
That's number two.
Number three-- you actually go through and generate the scores.
And the key thing here-- this is maybe the most important thing about the Pugh matrix is you always use a datum.
The datum is a reference-- one of the alternatives that you're going to compare is your reference.
So what you do is you always compare every other alternative in the set against the datum, and you say for each criterion is it better?
Is it about the same, or is it worse than the reference, the datum?
So you don't compare all the alternatives pairwise against each other.
You always compare against the datum.
Then you compute the total scores and I'll show you this in the example, and then there's variations like-- rather than better, equal, or worse, you could go to a five-point.
Much better, a little better, or about the same.
So there are some variations on the scoring, but the classic method just uses better, the same, or worse.
So here's some--
Excuse me, just a question
Yeah, go ahead
Just about the number three, about generic scores and user datum-- by doing so, don't you exclude a solution that could have, under the generic picture, be better then others?
What do you mean by general picture?
You should take all the solution that you have, and at the end, one performs slightly better than the other, but you couldn't spot it by using the datum?
Yeah.
So it's a very good question.
And so this is a question of resolution right?
How good is your resolution?
The main purpose of the Pugh matrix is not for making the final choice right away.
The main purpose of the Pugh matrix is actually to eliminate concepts that are-- you can tell already that they're not competitive.
And you don't typically do Pugh matrix one time and then say that's the best one.
You use it, essentially, to eliminate purely inferior solutions.
Does that make sense?
I'm really sorry but the video cut just during your explanation
OK, so let me repeat this.
So the purpose of the Pugh matrix is not to select the final concept as a result of doing the Pugh matrix, but to identify inferior concepts to eliminate them from the set.
OK?
All right.
So here's a kind of generic example to explain how this works.
So here's our Pugh matrix, our evaluation matrix.
On the rows, we have our criteria.
It could be cost, performance, resilience, effectiveness, and so those are called A, B, C, D, E, and F. So we have six criteria.
And then the column-- each column represents a concept, a design, an architecture.
And so you know we're designing-- I guess we're designing beams here, or levers, or something like that.
So this is a solid cylinder.
This is a cylinder that's hollow.
This is a square, it's cross-sectioned.
This is a triangular cross-section.
You get the picture, right?
There they're qualitatively different.
And then you note which one is our is our reference here?
Which concept?
Number seven, right?
If you look at seven this sort of shallow triangle.
Seven, it says datum.
You see that?
Which means that this is our baseline, our reference, our datum.
And then what you do is, as you fill this in-- and you can do it two ways.
You can do-- pick, say, concept one, the solid cylinder, and then compare it against seven for all criteria.
So A, so concept one is better than concept seven in criterion A.
So it gets a plus.
It's also better in B.
It gets a plus.
It's worse in C and D, so this is a minus compared to seven, and so forth.
So the two ways of filling it in is you do concept by concept, column-wise, or you can do it row- wise.
And there's some arguments in favor of one or the other.
But essentially, you fill in this matrix by always comparing against number seven.
Alexis, what's your question?
Yeah, just a question.
Shouldn't we weight the criteria?
I mean, for example, if the criteria A is more important, shall we put more weight on it?
No.
No.
At this point, in the Pugh matrix, the criteria are unweighted.
This is very important.
You don't say B is twice as important as C. They're unweighted when you first do this.
Eventually, when you do the final selection, you will probably weigh them.
But when you first do this, there's no weighting.
They're equal.
OK?
Yes?
Go ahead
Are there any rules in picking your datum or do you just pick arbitrarily?
So I'll get into that.
It should be a design that you think is competitive, that [INAUDIBLE] maybe you know it, maybe it's an existing sort of system, there's data on it, a known quantity.
But if you don't have if you don't have any information, if they're all equal, then it's a random choice.
OK?
So, then once you've filled in this matrix-- so S here is for same, right?
About the same.
Then you essentially sum all the pluses, you sum all the minuses, and then you have the net result, unweighted.
Right?
So, here you can essentially compare that, you can see that concept one is better in three and worse in three than the datum.
You can see that concept two is inferior in three criteria, better in two, and the same in one.
But we're not actually summing.
We're not actually summing at this point, because we're not weighting these, OK?
But it gives you a basis of understanding.
OK so let's do a quick partner exercise.
What do you see as the main advantages and potential disadvantages or pitfalls of this method?
So turn to your partner, discuss this for a few minutes, and then we'll see what you came up with.
[MICROPHONE NOISE]
[INAUDIBLE]
All right.
Good.
So let's see advantages.
Let's get a couple here, and then we'll sort of go back and forth between here and EPFL.
So what are-- what's good about this method?
Yes, go ahead
So, Nate and I were talking that it is a relatively simple decision making process.
So instead of just going with some sort of gut feeling, or not looking at everything, you can get stuff down on paper and actually see these criteria and how they relate to each other
Yeah.
So simplicity.
What else?
Go ahead, Sam
It allows you to look at a large array of concepts very quickly
OK large samples.
Sorry.
Quick, right?
You said quick.
.
OK. What advantages, EPFL, what did you guys come up with?
So simple, it allows us to look at a lot of concepts.
It's quick.
What else?
[INAUDIBLE] we got [INAUDIBLE] the same things.
This method is simple, and it's pretty fast and pretty straightforward, too.
So this is a good way to get rid of some of the concepts pretty fast
OK.
Anything else that we didn't [INAUDIBLE]??
There's one more that I would like to put on this list.
Veronica, go ahead
[INAUDIBLE]
Qualitative
[INAUDIBLE] I kind of like [INAUDIBLE] improvements, rather than this is better because the requirement is--
Please put on the microphone
I think-- can you just quickly repeat that?
Just that it's qualitative
And that you see that as an advantage, actually
I think you can see it both ways
Yeah.
So I think what you implied with is it stimulates the discussion, right?
Stimulates debate
There may be another criterion on the positive side
Go ahead
And it would be that it's extremely easy to explain to [INAUDIBLE],, to [INAUDIBLE],, to whoever your customer is, because-- and they can relate to it.
If the criteria are mostly physical or scientific, I mean, larger, stronger, lighter, cheaper, it's probably very intuitive, yes, as I say
OK, great.
Now, downsides.
What are the downsides of the method?
Go ahead.
Make sure you--
I think it definitely depends on which you pick as a datum.
Like, if you pick the clearly best one as the datum, then it's not going to give you much information.
Or if you pick the worst one as the datum, then all of them will be better.
And you kind of don't really gain much information from that.
So you have to like iterate with different datums
Yes.
There's actually research on this.
There's some research-- this matrix, this Pugh matrix method, by the way, has been studied scientifically quite a bit.
There's quite a literature on it.
And the point that you brought up was studied.
They would basically give the same criteria and the same alternatives to different groups of people, but then give them a different datum.
And it's a little tricky-- is the result different because it's a different group dynamic?
Or is it-- but this has been done statistically, and the choice of datum has an impact on the outcome.
Very good.
That's actually a pretty subtle point.
That was excellent.
What are some other-- yes, please go ahead
While it's very easy to implement and anyone can do it, it is very subjective.
So if you have somebody that doesn't have a whole lot of experience in a certain area, they may get a completely different answer than somebody who is an expert looking at the same type of criteria
Subjectivity-- which basically, repeatability-- may be low, right?
You want to have robustness of the methods such that if you repeated this multiple times or you gave it to different groups of people, you want to have some confidence that the results will come up similarly.
Very good.
Let's see at EPFL, any downsides?
So datum dependent, subjective-- what else?
So what we are thinking is that the criteria depend, from group to group.
It doesn't seem to be clearly defined what you need.
Although, there are some that are straightforward and our other point--
Datum dependence?
That's what we--
Yeah, datum dependence.
If it was a very easy solution, or if a very bad solution as a reference, then all the others look good.
And finally, if you have a-- if you judge on the overall concept, and maybe you discard one or two solutions, but inside those concepts you have ideas that you could keep or use, then they might be lost
OK, so loss of sub-concepts.
OK. Good.
Anything else here?
Mike
We were talking about how it's very easy to tune your metrics to be-- like to be biased
I see.
Such gaming, right?
Gaming-- gaming the-- basically producing a matrix to give you the answer you want, that you already had preordained, sort of.
Right?
That essentially what you're saying.
.
OK.
Very good.
So these are all-- go ahead
Just sort of one last disadvantage of the method that I think is a bit interesting, because I think the way of grading everything with plus or minus can be [INAUDIBLE],, because you can imagine a system, for instance, that is very, very good in one criteria, and then pretty bad in all the others, and it could be a good concept that is disregarded nonetheless
What's your name?
Bastian
Pasqual?
Sorry?
Pasqual.
Is your name Pasqual?
Bastian
[INAUDIBLE]?
Bastian
OK. Keep that point until-- we're going to talk about exactly that point when it comes to non-dominance, OK?
So please reserve-- this is great.
Just, we'll come back to that in a few minutes, OK?
All right, so this is great.
I think you really got it.
This is exact-- this is a very good list.
So, you may say well, OK this is good for beams and you know very simple things.
So, I want to show you an example of an application of the Pugh matrix to what I would argue is a fairly complex architectural decision.
And what I'll talk to you about is a thesis that was done here at MIT in 2003, so a while ago, 12 years ago, by Brian Smith.
He was an STM student, and was one of the easiest students ever to advise.
Very knowledgeable in nuclear propulsion and nuclear power.
He's now the branch chief for nuclear power propulsion at NASA Glenn research center in Ohio.
And so, at the time, NASA had a mission that was kind of high priority called JIMO-- Jupiter icy moons mission.
And when you go out to Jupiter, and you want to not just do a fly by but you actually want to go to different orbits, you want to do high power imaging, you need a lot of power.
You're far from the sun.
RTG's only give you 100 watts or so, per RTG.
So nuclear power is pretty much the way to go.
But there's a lot of different choices of different nuclear reactor architectures that would potentially-- and then couple that with the propulsion system.
So that was the challenge was sifting through the large space of possibility for nuclear electric power, nuclear exploration class electric power and propulsion systems.
And you know, here's the way the study was set up is the expansion phase, the filtering, and then the screening phase.
So here's the architectural space the way it was defined.
So nuclear electric power and propulsion is what we're after.
And there's different ways to generate these alternatives.
So the most important is the design vector, the type of reactor, the operating temperature, the power conversion scheme, the heat exchange, and then the fuel type.
And then there were requirements.
So this is the power range, the delivery timeline, the fact that you should be able to do it in a single launch, and then the operational lifetime of the system.
So those are the shall requirements.
Every architecture had to meet these requirements otherwise it would not be considered.
And then there were some things that were assumed as constants.
They're shown here.
There was a policy vector in terms of funding profiles, international partnerships, at what altitude you could insert the system into orbit, and then influence on future missions.
And the objective vector is the criteria for the decision.
Technology-- TRL is Technology Readiness Level.
How much infrastructure do you need?
Complexity.
Strategic value to the nation.
This is a little fuzzy, but basically what this means is, could you use it for other applications than exploring Jupiter?
The schedule.
The launch packaging.
Power.
Specific mass, which is watts per kilogram.
The lifetime.
So it's interesting you have lifetime here and here.
So there's a minimum lifetime, right, and if you satisfy this, the extra life you get can actually be used as a decision criteria.
How does it interact with the payload?
Do you need a lot of shielding?
And then adaptability?
Can you adapt it for different missions?
Yeah?
In the previous slide, it listed both possible-- on the previous slide, we had both possible and feasible on there as different kind of criteria.
What do you have for a difference for those?
So essentially, possible is, this is sort of the combinatorial space.
And then, applying the hard constraints based on the requirements vector gets you from possible to feasible.
They satisfy your must have requirements, your shall requirements, and then you use, in screening, you use the objective vector-- these criteria-- you use to go from here to here.
So let me show you what this looks like, and I'll post this thesis, if you're interested.
It's really-- the whole thesis is about doing this process.
So here we have-- each column represents a different reactor architecture.
So you can see in the legend what these things mean.
So for the reactor type, we have a liquid metal, gas cooled, or heat pipe type reactor.
These are three types of nuclear fission reactors.
For fuel, we have two types of fuel, if I remember correctly.
We have UO2 or UN.
For the temperature at which the reactor would be run, we have medium or high.
For the conversion-- this is the thermodynamics.
How do you get the heat out of the reactor?
We have a Brayton or a Rankine cycle, or a thermoelectric reactor, which is basically directly converting the heat to electricity without a working fluid.
And then, D is the heat exchange architecture, which is either direct or indirect.
And so you can see that each of these columns represents a different kind of architecture, and they're colored here by-- the dark is a direct heat pipe with liquid metal reactor, and so forth.
So that's essentially-- what you're seeing here is the filtered set that is deemed to be feasible.
Potentially feasible.
And then the actual screening is done based on this.
So this is the actual Pugh matrix here.
So we have the concept combinations, which are the columns.
The datum here is the one that's shaded in gray.
So it's a liquid metal reactor with a thermal-- it's a liquid metal thermoelectric reactor with indirect heat exchange, UN fuel type and a high operating temperature.
And why was this chosen?
Well if you look here, it says SP100 reference.
So the US actually has launched-- at least, officially, as far as we know-- one nuclear reactor into space, and that's the SP100.
You can look it up.
There's a history behind it.
And that's the architecture of the SP100.
So it was launched.
We had data on it.
It's a known quantity.
The Russians, it turns out-- Russia has launched a lot of nuclear reactors over the years.
But this is sort of the US baseline, if you want to call it that.
And then all of these other combinations are compared against the SP100, the reference reactor.
And so, you can see here that the comparison is drawn again.
So zero means it's about-- so for TRL, it means it's about equally mature.
A plus means it's actually more mature.
A negative means it's less mature than the SP100.
And at the end of this, you can actually look at the total pluses and zeros and minuses, and then in this case, what Brian did, he did calculate a net score.
That's an unweighted score.
Basically, if you give one point-- so let's look at the first one.
So it has four pluses, right?
It has four pluses, and it has five equal to the SP100, to the datum, and has two minus.
So it's worse than two, better in four criteria, and the same in five.
So the net score would be plus two, because it's better in two criteria, net.
But it's unweighted.
And so in order to figure out what-- based on this net score, so a zero net score would say it's about equal to the datum, but it maybe in other, different criteria.
But essentially it's about the same.
If it's a plus two, then it is potentially better than the SP100.
But you don't know that 100%, because you haven't weighted the criteria.
It's really used for screening.
So the ones that are circled are better, essentially, than the datum.
And the ones that are-- this is-- I'm sorry, the rank.
This is the rank among the set.
So if you have a one here, you're in the rank one.
And there's two architectures, the first two are rank one.
And then we have this architecture, which is ranked two.
The datum itself, plus two other architectures, are rank three.
And then you see that there's some-- like, for example, this architecture here is rank eight, right?
So it-- why is that?
Because it's not better in any criteria than the datum.
It's the same in six.
And it's worse in five, right?
So that gives you an indication that this particular concept probably is not very attractive, and you would eliminate that.
OK?
So that's essentially an application of the Pugh matrix to a pretty real-world, complex problem.
So what the Pugh matrix is for is essentially structured-- structuring and representing the evaluation procedure.
It serves as a common visual.
It provides some amount of discipline, and helps break down self-sealing behavior, meaning-- what that means is people defending their concepts without sort of seeing the bigger picture.
Encourages teamwork.
It helps you eliminate weaker ideas, retain a set of stronger concepts.
And then this is interesting-- divergence.
It helps to identify opportunities for combination.
There's also a way to use this Pugh matrix methods multiple times.
And the way you would do this is you would say, OK so these are the stronger concepts here, right?
The ones that do equally well or maybe even a little better than the datum.
What are the strengths and weaknesses of each concept and can we hybridize them?
Can we create some hybrid concepts?
And so you eliminate the weaker ones.
The surviving concepts are kept, and then maybe hybridized with each other.
So you expand and create more concepts again.
Maybe a little bit more detailed.
And then you apply the Pugh matrix again.
So it's iterative application of the Pugh matrix, where you eliminate, you keep strong concepts, you hybridize them, and then you repeat it two or three times.
And there's actually research on this, as well.
Professor Dan Frye, who is here at MIT in mechanical engineering, has done research on that and shown that that can actually lead to very good outcomes.
So this sort of gradual-- so then, you don't have just one expansion and contraction of your concept space, but it sort of goes multiple times.
But eventually it does converge.
So this is a more refined application of the method.
What Pugh matrix is not good for is automatic decision-making, right?
Completely controlling the process.
So, you know, the idea of automating the Pugh matrix-- it should not be automatic.
And it's not really done-- it's not really good for trade studies.
Challenges-- [LAUGHS] so here's some quotes.
"People who have a lot of experience exhibit an impatience.
Get on with it.
This procedure holds us back."
Right?
"Strong-willed individuals who have a lot of experience and whose initial concepts have not emerged in the final selection commence a defense based on emotion, experience, and bluster."
Right?
So there's this social dynamics that gets unfolded here.
So, therefore, it is recommended to do Pugh matrix with a facilitator.
Somebody who-- similar to brainstorming, right, where you have a facilitator.
So you have a facilitator for Pugh matrix.
So somebody who controls the flow and pace of the session, records the results, tries to maintain some discipline.
Always compare against the datum.
I've seen Pugh matrix sessions where you start with the datum, and then by the third or fourth concept, people are starting to do pairwise comparisons, and the datum is sort of lost.
That should not happen, right?
Preventing tangents, but encourage clarification.
What do we really mean by criteria?
Clarification of the concepts, and then opportunities for divergence in hybrids.
So the critique of the method is-- I think a lot of these things you've already mentioned.
The ranking depends on the choice of datum.
The weighting.
You know, there's no real weighting in the classic method.
However, you can implement a weighted version of the Pugh method, but then you have to have the whole discussion about the weightings, a priori.
Next, I'm going to talk about multi-attribute utility.
And so Pugh matrix and multi-attribute utility could give you a different rank order of alternatives.
So how do you reconcile that?
The most important criteria may be intangible and missing from the list.
And my personal opinion on this is that the Pugh matrix is useful and simple to use.
It stimulates discussion about the criteria, the alternatives, but it shouldn't be your only means of concept selection.
But it's usually one of the first things you should do because you learn a lot from doing it.
OK?
Any questions about matrix before I move on?
Yes, Sam?
Should it always be done with a group that's working together and discussing the concepts?
Or could you also have people individually filling them out and then combining their results?
So yeah, usually you do it as a group because in the discussion with the group, there's a lot of clarification that happens.
Now if you do it separately, and you firewall people from each other and have people do this individually and then try to combine the results later, that's actually called-- that's closer to a method called the Delphi method.
Which I'm not going to talk about here.
But the Delphi method means you're asking these kinds of questions to experts.
They give you the answers, and then you anonymize the answers and reflect the combined answers back to the group.
And then you eliminate the social dynamics, and you can see whether was my assessment an outlier, am I close to the center of the group?
So the Delphi method has a lot of benefits, but it's not usually-- you don't have the social dynamic in the same way, which is sort of essential to make this work.
Good question though.
Any questions at EPFL about Pugh matrix before I move on?
No, it's clear
OK. Go ahead
It is useful to almost have an independent group do this, as opposed to everyone generates their ideas and then everyone's trying to fill out the matrix so that their idea wins
Usually not, because if you give it to an independent group, they'd know much less about the problem.
They probably weren't involved in generating the alternatives, they weren't involved in selecting the criteria.
They're probably not as knowledgeable.
And so there is an advantage of doing it within the team.
But then, you know, before you actually go and pull the trigger on the final-- that's the PDR.
You get vetted by independent people.
That's what happens at the
One point here is that you can invite external people to join your discussion.
So you have your design team.
And it's very good to have the experts to come, sit in the meeting, and then give their opinion, and with the Pugh matrix, as it's pretty straightforward and simple, you probably shouldn't then bias the result, but you could probably enrich it
That's a great point.
I agree with that, [INAUDIBLE].
Absolutely.
Yeah.
OK.
So let's switch gears to utility theory or multi-attribute utility theory.
So what's that all about?
So like I said, utility is a very deeply rooted concept in economics.
And so it's defined as, "Utility is a measure of relative happiness, or satisfaction, or gratification gained by consuming different bundles of goods and services."
This is sort of economic-- very-- this is how economists talk.
So it's-- the idea is you want to choose a concept or alternative that will maximize your happiness, your satisfaction, your gratification.
And whenever you buy something, when you go to Anna's Taqueria, or wherever you guys have lunch, or at EPFL you go to the Rowlock center, and you pick from the menu, you're actually doing just that, right?
You have-- how much money do I have in my pocket?
How hungry am I?
Am I a vegetarian or not?
That would filter out some menu items right away, right?
And you pick, every day.
You pick your lunch.
Well, you're doing exactly this.
You don't really know that you're doing it, but in your mind, as you're picking from the menu, and then making your choice, you're maximizing your utility at that moment.
So we do this every day.
Here, in this class, we talk about designing complex systems.
So we need to do it a little bit more deliberately, but this is not just some abstract thing.
We do this every day.
And so the idea is, essentially, we have this Consumption Set X, which are our alternatives-- mutually exclusive alternatives-- and then we map that to the real scale.
We rank each member of the Consumption Set.
So we map the alternatives on this utility scale.
In this case, the cardinal scale between zero and one.
So, the way this is done, in order to do the mapping from the criteria to utility, you need these mapping functions.
And we call these utility functions.
They're called utility functions.
So basically we have-- I'm going to use J here for-- J means, essentially, your attribute, your objective.
And then U is your utility.
And so there are different shapes of utility functions.
And different scholars call them differently.
So this is Cook.
He says, this is smaller is better, or larger is better.
And you see they're essentially monotonically increasing or decreasing curves.
Messac, another scholar in multi objective design, calls this class 1S, class 2S.
So what would be an example of a smaller is better utility function?
What would be an example of that?
Go ahead
An example would be the weight of a launch vehicle.
It's decreasing
The launch mass?
Yes
OK.
Launch mass.
OK. What about EPFL?
Smaller is better
Cost
Cost, right.
So, larger is better?
Revenue
Revenue.
Maybe range, endurance, reliability, right?
Then there's-- the next one is strictly concave or strictly convex, which is nominal is better.
So the idea there is there's a sweet spot, right?
You want the performance or the attribute to be a pretty specific value, and if you deviate from that on the up or down side then the utility decreases.
So what would be an example of that?
Size of a meal
Size of a meal, OK?
Well, some restaurants may beg to differ, right?
All you can eat.
[LAUGHS]
[INAUDIBLE] what their objective is
Yes.
All right.
Very good.
Yes, Sam?
[INAUDIBLE] has to interface with something else [INAUDIBLE] you're not [INAUDIBLE] it [INAUDIBLE]
Yup, so if there is a very specific interface condition.
Yeah.
OK, go ahead at EPFL.
Nominal is better
Like, for example, ambient temperature
Yeah.
So temperature-- ambient temperature, right?
Humans are pretty-- we have a fairly narrow range where we say, this is good.
Now we can put on a sweater or-- but it's a fairly narrow range.
So, great example.
Then the next one is range is better, which is concave or convex.
So this is the idea that as long as you're within this interval, within the interval itself, you're sort of indifferent.
Right?
But then when you drop outside the interval, then utility decreases or increases.
And then the last one is very exotic-- non-monotonic utility functions that have multiple peaks.
They exist in theory, but in practice, you almost never see them.
I can't give you a good example of multi-modal one.
But in theory they do exist
Well, there is one, and it's clearly the landing sites on a planet

Because you have the one injection trajectory, and then you have the primary sites of the primary elipse, then the secondary.
And it's non-monotonic, mostly [INAUDIBLE]..
It doesn't go up, it goes down.
So you have ideal.
Then you are out of it.
Then you would land on rocks.
And then you have the next plateau, and maybe a third one, on one pass by
That's an interesting-- that's an interesting comment.
I'm trying to figure out what's the attribute-- what's the engineering attribute that goes with that?
Well, it's the tolerance on the injection.
I know that, at JPL-- by the way, thanks for organizing the [INAUDIBLE]---- they made this presentation, also, of the landing of the rovers.
And they have the primary ellipse where they try to land-- well, try-- it depends on the entry.
And then they have secondary-- if they miss the entry, then they have very bad option.
And then, maybe a little bit later, they maybe can land on the next plateau, or in the next crater, which is useful.
So they have this non-monotonic, but degrading, function.
The first time is best.
The second would still work.
But in between, there's a part that doesn't go
OK. Got to think about that one.
But that's an interesting-- that's a pretty interesting example.
So, the main point here is that in order to calculate utility, you need a translation function between the engineering or financial attributes of each alternative and utility.
If it's not-- if it was directly that, you'd just have a linear, right?
It would be a 45 degree line, right?
Or negative 45.
But typically the mapping from your attribute values to the utility is non-linear.
And that's what this says.
So then the challenge is well, how do you get these utility functions?
Where do they come from?
And the answer is you've got to do interviews, you have to survey people-- the decision-makers, the stakeholders we talked about.
They're the ones who have these utility functions in their minds, even if they're not-- So you've got to make those explicit.
So here's an example.
You may have three different customers for your system.
User attribute, some performance attribute.
Customer one is shown here.
They need a minimum amount of performance, but once you reach this threshold, after that there's no more utility.
But then customer two and customer three are different.
They want-- their utility increases gradually, and in fact, customer three doesn't see any much utility until you hit this much higher level.
So one of the challenges, then, in designing these utility functions when you have multiple customers is to solicit this-- and there's interview and survey techniques for doing that-- and then combining those.
And then the other challenge in combining them is that, remember at the system level, utility is still between zero and one, right?
So if you had, for example, two attributes, and they both give you perfect utility, and you add them together you'd get a utility of what?
You have two attributes.
Two.
But that can't be because the total system utility can never be better than one.
So you have to normalize.
As you combine the utilities, you have to normalize them, and actually weight them as well.
So here's the equation when you have two utilities.
You basically have the utility of the system just from J1-- your first attribute-- times the utility of J2.
This is the mixed term, right, the combined term.
Plus the utility of just J1 plus the utility of just J2.
Then you have this K factor here-- this capital K factor-- which re-normalizes everything to zero and one.
And if you have more than two, then it becomes a matrix calculation.
So it's a little tricky for how to do this properly, but it's well-known how to do this.
So the steps in multi-attribute utility analysis are to identify your objectives and attributes, you develop an interview questionnaire, you administer that questionnaire.
You then develop your aggregate utility functions, you determine the utility of your alternatives, and then you analyze the results.
And there is one word of caution I want to give you, which is that utility is essentially the surrogate of value, but value often we express in dollars, in monetary terms.
But utility is unit-less.
So let me just comment about utility maximization.
It's very common and generally well accepted.
It's a non-linear combination of your criteria-- your decision criteria.
The downside of it is the physical meaning is often lost, because the engineering or customer focused attributes all get combined.
So you say this system has a utility 0.83, and this one has a utility 0.75.
So one is better than the other, but immediately you want to say well why is that?
What does that mean?
So then you have to backtrack and reverse engineer how those utilities were calculated.
You need to obtain a mathematical representation for all utility functions, and-- this is a probably not just US-M centric comment, this is probably everywhere in the world more or less-- but the utility function can vary drastically depending on the decision maker.
This is a big issue in government programs.
So you guys, it's uniform day, I guess, today.
So how long is a typical tour of duty of a program manager in the Pentagon for a big program?
What would you say?
It's usually three to four years
So three to four years, like it's written here.
And what does that that mean in practice for these utility functions and for programs?
Sorry I'm putting you on the spot here
Well, you don't usually have a lot of overlap between the PM.
So, as one goes out, another one's coming in, so you're losing a lot of experience, gaining somebody who doesn't have much experience.
So there's a learning curve here
Right.
That's right.
What I'm getting at is-- so there's definitely that effect-- but the priorities may be different.
You know, whereas the prior program manager really valued a lot performance or quick response, and the next program manager-- because maybe the context has changed-- is very, very cost conscious.
Really wants a system that is very, very affordable, and is willing to sacrifice performance for that.
So the way you can think of this is the shape of the utility curves for those decision makers has now shifted, and because of that the architecture or the choice of concept that you would go for may be different
This happens to NASA all the time, right?
Right.
And so this is not just a DOD issue.
And in commercial world.
You know, a new CEO comes in, new CTO comes in, and these utility curves actually shift.
So you've got to be to be aware of this.
So one of the big topics there is-- particularly now with NASA is-- how do you choose an architecture, a concept, that is robust to changing utilities and decision makers.
So maybe you don't go for the super duper best, most exciting, most capable concept, but you go for the one that's least likely to be overturned or disrupted with the next administration.
And this is a discussion-- active discussion-- right now at NASA headquarters.
You know, what are the investments you can make that are going to provide utility, even if the utility function of the future decision makers changes?
It's a big, big topic.
The other thing is, of course, this requires your formulation of preferences.
Those K factors, those weightings, a priori-- before you've actually scored the alternatives.
All right.
So the example I want to talk you through is essentially a space tug.
So we're going to do trade space exploration.
There's a trade space, a design space, of alternatives-- conceptual alternatives.
And we're going to try to understand that using utility theory.
And let me explain to you what we mean by space tug.
So a space tug is essentially a satellite that has the ability to change its orbital elements-- there are six orbital elements, semi-major axis, eccentricity, inclination, right ascension of the ascending node, and so forth-- of a target satellite by a predefined amount without degrading its functionality in the process.
So here's a picture of the earth.
There's two orbits.
The space tug is here in the black orbit, and then our target satellite is in the orange orbit.
And so the typical process is the space tug waits in its parking orbit-- the black orbit.
It gets tasked.
It transfers to the other orbit, searches for the target, identifies it rendezvous and approach, docking and capture, does an orbital transfer and then releases the target satellite at the new orbit, verifies its status and then either goes directly to the next target, or returns to the parking orbit.
And as you can imagine, depending on the plane changes that are required here this can be quite expensive.
It's less expensive in geosynchronous orbit, and there are some capabilities that we think heard about in the public that US and maybe other countries have as well to do this.
But there's also applications-- commercial applications for it.
So, for example, for space debris removal and things like that.
So that's what we mean by space tug.
So what are the attributes?
What are the-- so this is based on a paper from 2003 called Understanding the Orbital Transfer Vehicle Trade Space.
And so here's three attributes that are combined into utility: total delta V capability-- delta V is the change in velocity.
This essentially tells you where can it go?
You calculate that from essentially the rocket equation.
The second one is response time.
How fast can it get there after it has been tasked?
And there there's a big distinction between-- it's almost binary-- between electric propulsion, which is very efficient but slow, and then chemical propulsion.
The third is the mass of the observation or grappling equipment.
And so this tells you what it can do when it gets there.
So the size of target satellites that can be actually interacted with.
Those are the three attributes that define utility, and we combine those into a utility between zero and one.
And then in this case, cost or a surrogate for cost is kept separately.
This is the vehicle wet and dry mass.
These are the cost drivers, and this is calculated from some simple scaling relationships.
And so what we're interested in is looking at the trade space of utility versus cost of the system.
So how is utility defined for a space tug?
So we have response time, which is bad in electric systems.
Total utility is a weighted sum, and then we estimate the cost from wet and dry mass.
So the delta V utility is shown here.
This is in terms of meters per second.
So two, four, six, eight, 10 kilometers per second.
These are pretty large delta Vs. And what's interesting is this is a larger is better, but it's a step curve.
It's not a smooth curve, it's a step curve.
Why is that?
Because as you reach certain delta Vs, it enables operating just in GEO or a LEO-GEO transfer, or a LEO-GEO return.
Right?
So you can use the space tug more than once, the more delta V you have.
And it's a step curve because as you hit this value of delta V, you can now operate in a different regime.
The capability.
The payload mass utility is essentially discretized between low, medium, high, and extreme.
So low is for small satellites.
Medium satellites up to one metric ton, up to three metric tons, and then more than five metric tons, which are the big satellites in geosynchronous orbit.
And then the weighting factors are we're going to capability 30%, delta V 60%, and the time responsiveness of the system 10%.
So time responsiveness is not that important compared to the other two.
So once we have this-- I'm not going to go through all the details of the calculations.
That's in that paper, which I'll post.
You get this.
So this is a cloud of points.
Each point here represents a particular alternative or architecture, and we have the cost of the system in millions of dollars.
So this is not cheap.
All right?
This is a billion dollars, two billion, three billion.
And then we have this utility, this dimensionless utility.
And, of course, what we're particularly interested in is this lower right corner: high utility lower cost systems.
And what we did here is to identify some particularly interesting architecture, as they're shown here in this space.
So let me just explain two of them.
One of them, here-- and we gave them names that are recognizable.
So this point here, it's below 0.4, so it's a rather lower utility system, but it's also relatively affordable.
We call it the bi-prop low-earth orbit tender.
So it's intended for use in low-earth orbit and it has a dry mass of about 680 kilograms, a wet mass of 1,400.
So the difference is propellant-- about 800 kilograms of propellant-- and has reasonable size and mass fraction.
And it's fairly responsive.
Another alternative is what we call the electric geocruiser.
Sounds cool, doesn't it?
This is basically a space tug that operates only in geosynchronous orbit.
So you launch into geosynchronous orbit, but once it's there it can do a lot.
But it's electric, so it's kind of slow.
Seven hundred eleven kilograms dry mass, 1,100 kilograms wet mass.
And it includes return of the tug to a safe orbit.
And this is sort of a versatile space tug in the GEO belt.
So that's the idea.
You calculate utility, shown on the x-axis, you have the cost of the system.
You have this cloud of points.
And you start understanding what are the interesting architectures in that trade space.
This is the same trade space.
But now what we've done here is we've shown the choice of propulsion system is critical.
And what you can see here, it's really almost impossible to get to a utility of one.
It's very, very hard.
And the reason for this is the rocket equation.
You see how these-- so the blue are the bi-propellant architectures.
The purple are the cryogenic, so these are [?
lux ?]
hydrogen propulsion.
The yellow ones are the electric propulsion system, and then we also have nuclear propulsion, here, which clearly is challenging and has policy implications.
So you can see the nuclear propulsion gets you close to much higher utility, but it's also much, much more expensive.
But in all cases, as you're trying to get more and more utility, at some point you hit the wall because of the rocket equation, which is very non-linear.
You know, more fuel-- in order to carry more fuel, you need more dry mass, more fuel and dry mass requires more fuel to push, and the system blows up on you.
And you can see this in these curves.
So you get some physical intuition in terms of the shape of the space.
Yeah?
Is there a reason between this layering of cost?
Yeah.
They're essentially, if I remember correctly, the capability of the system.
Remember there's this small, medium, large, you know the size of the grappling equipment?
I think that's the tiering that you see
That doesn't change the utility?
Well, it depends on the weighting, right?
So as you change the weighting among the attributes, that space can get scrambled.
This space is valid for that particular weighting.
So any questions about-- this is utility theory, multi-attribute utility theory, applied to trade space exploration.
Any questions?
Go ahead.
EPFL, do you have a question?
OK.
So now what you see here is implicitly, is there's nothing here in the lower right corner.
There is no system that has perfect utility and is low cost.
Right?
There's an empty space.
We'd love to be here, but there's nothing there.
So the best we can do is get close to that.
And that's what I want to talk about next, which is this concept of non-dominance.
So what do we mean by non-dominance?
What is a Pareto frontier and what is multi-objective optimization?
The key point about this is that when you do non-dominance filtering, when you look for Pareto frontiers, you do not need to express your preferences ahead of time.
Right?
So when you do you a weighted Pugh matrix or when you do a utility theory, you had to define ahead of time, before you did the scoring, what's more important-- fuel efficiency is twice as important to me as cost.
Right?
The weighting, the preferences had to be expressed a priori-- before you did the scoring.
And there's arguments that that's not the best thing to do, but you can do it.
Here in non-dominance and multi-objective optimization, eventually you have to express your preferences, but you do it after you've scored the options.
That's a really important distinction.
So let me give you a little history here on this.
And what's really interesting is there's a Lausanne connection.
OK?
There's a connection with Lausanne.
So the word Pareto frontier is named after a an economist, who actually started as an engineer, Vilfredo Pareto.
He was born in Paris in 1848 and then graduated from the University of Turin in 1870.
Very much in engineering, civil engineering, his thesis title was The Fundamental Principles of Equilibrium and Solid Bodies.
Something we take-- that's pretty basic now.
Forces and torques in equilibrium on a body.
Now he worked as a civil engineer in Florence, and then he got interested in philosophy, politics, and economics, and he started to think about this in terms of how does this concept of equilibrium apply to economics?
And in 1893, he actually became a professor at the University of Lausanne, which is located right next to EPFL, and started applying this to economics and societal theory.
And then he did-- some of his work was a bit controversial, but the one that I want to talk about here is this the idea of the Pareto optimum.
And I will-- let me just read this quote to you.
"The optimum allocation of the resources of a society is not attained, so long as it is possible to make at least one individual better off in his or her own estimation, while keeping others as well off as before in their own estimation."
So what it means is if you can make an investment or make a change in society that makes a particular individual or group of individuals better off in their own estimation, without having to take the resources from another group and make them less well-off, then you've not reached an optimal allocation of resources.
You know, if there can be a win-win-- if everybody can be better off by making certain investments, you haven't yet found the optimal strategy.
Only when-- the only way to make somebody better off in their own estimation than somebody else is by taking-- and this is a big political debate still today-- by taking from one group and redistributing resources to the other.
When you're in that situation, then you are at the Pareto optimal point.
OK?
So that's the key idea underlying this.
So what this means mathematically is that an optimal solution -- X*-- is optimal if and only if-- for a feasible solution-- so, first of all the solution has to be feasible.
And this is when you have multiple criteria.
We're trying to do vector optimization, and x must be what we call an efficient solution, and it's efficient only if its objective vector-- J of X-- is non- dominated.
And what this means is that if you're looking at a point, it's only efficient when it is not possible to move from that point to another point without degrading at least one of the other objectives.
So if you can move from a particular design point to another design point, and all the objectives get better, you're not yet efficient.
You're not yet at a Pareto optimal point.
And this gets us to the notion of dominance.
So dominance-- let's say we have two designs, two alternatives-- J1 and J2.
Those are their objective vectors.
It means that-- this is for maximization.
We're trying to maximize.
It means J1 dominates J2 weakly if J1 is better or equal J2, and at least in one of the objectives, I, strictly better than J2.
So you could have two alternatives.
This gets back to Pugh matrix and the question-- who asked about the ties?
You did, right?
It's so long ago.
It's like, wait a minute, it's like an hour ago, you know?
So we have two alternatives, OK?
They're all equal.
They're tied in all criteria, except for one, where J1 is better than J2.
Which means that J1 will dominate J2, but weakly, because there's some ties.
Right?
And then there's a stronger definition, which is that J1 strongly dominates dominates J2 if and only if it is better strictly better in all attributes.
So for J1 to dominate J2 strongly, it's got to be better in all attributes than J2.
Does that make sense?
OK.
So I know this is a bit abstract.
So let's do a concept question that's sort of based on a hypothetical but real example.
So assume-- let's say you're in charge of designing in new commercial aircraft, and there's four criteria.
You want to maximize range.
You want to minimize the costs-- dollars per kilometer flown.
You want to maximize the capacity of the airplane-- number of passengers.
And you want to maximize cruise speed in kilometers per hour.
So it's a multi-objective aircraft design.
You do all the work.
You do concept generation.
You come up with eight concepts for airplanes, and you evaluate them.
And they're shown here-- one, two, three, et cetera, through eight.
So let's just look at number one.
What does this mean?
Airplane number one, concept number one, has a range of 7,587 kilometers.
It has a operating cost of $321 per kilometer.
It can carry 112 passengers.
And it has a cruise speed or max speed of 950 kilometers per hour.
So I'm going to leave this up.
This is like a five minute-- this is-- on the next slide, I'm going to ask you-- and we'll come back to this-- which of these airplane designs, these eight, are non-dominated.
Meaning that they're not dominated by any of the other.
And we're going to apply weak dominance here.
That was going to be your question, right?
Weak dominance.
So I don't think it actually matters.
I don't think there's too many ties in here, but apply weak dominance.
So which of these are non-dominated?
Meaning that you wouldn't discard them right off the bat, because there are some strong features that they have.
So work through this.
You might want to write down which ones are non-dominated.
And then we'll do the concert question, and then we'll show you the answer
I just want to make sure.
So non-dominated as in there is not one that could be considered better than it?
Yeah.
So, this is the definition.
All right.
So let me-- so please submit your answers.
So here are the choices.
Which airplane designs are non-dominated?
Five, six, and seven.
One, three, four, and eight.
One, two, three, four, and eight.
Two, three, five, and six.
One, three, four, and seven.
Or you need-- this is not answerable.
You need more information to answer the question.
OK?
So we have a pretty good distribution here.
So the wisdom of crowds I think prevails.
So the correct answer is the third one-- one, two, three, four, and eight.
So 36% of you got it right.
Now let me let me show you the solution to this.
So it is, in fact, possible to answer this without additional information.
So the way to do this is you have to do pairwise comparisons.
Now you said about-- I think your strategy-- What did you say, early on?
I was looking at maximums
You were looking at maximums.
So you were saying who's the best performer among the concepts for each criteria?
And you found those pretty easily, I would assume, right?
So what--
[INAUDIBLE]
OK.
So you've got to get the sign right.
Do you have the mic on, by the way.
So what would-- if that concept is the best performer in a criterion, what does that tell you?
It means that none of the other performers have something equal to or greater than that performer for that criterion
And this relates to the question-- is it Martin?
Martin?
In the red shirt, first row?
I didn't get your name right.
I'm sorry
Bastian
Bastian.
Bastian.
Bastian.
You remember at the start of the lecture, you said-- what did you say at the start of the lecture?
I said that if you evaluate all the concepts based on just plus or minus for the criteria, you might have one where you evaluate all the-- a bunch of criteria that are negatives, but there's just one that is positive.
But that one might be very, very positive
Right.
So that's exactly the point here.
That's exactly this point.
If you are-- if this concept is the best performer in just one of the criteria.
Like it has the best speed, or the lowest cost, but it's terrible, terrible in everything else, it doesn't matter.
In terms of dominance, it is non-dominated.
It cannot be dominated by any other concept.
If you are the best in class, even just for one criteria-- does that make sense?
You cannot be dominated by any other design if you're the best performer on just one criterion.
So let's think about track and field.
So the people-- who's doing track and field?
Any athletes here?
Like runners or javelin or-- Veronica.
I didn't know that about you.
Cross country and track.
OK.
So 5K?
All right.
So let's say-- I'm sure you were very good at 5K.
And how was your shot putt?
I never competed in the shot putt
OK.
So then we don't know, right?
But the point I'm trying to make is you could be a super duper specialist in one criterion, and that's-- you can will win a gold medal with that.
But you're very maybe not good in other things.
So that's the point here is in order to be dominated, if you're the best performer-- so, right off the bat, in this exercise here, in our little exercise, you can remove all the concepts that are best in class from the set.
Because they cannot be dominated.
So they have to be non-dominated.
OK, so the way you do the scoring with pairwise comparisons is you say, let's compare one and two.
Concept one and two.
Well, one is better in range, right?
And it's better in speed.
But then the second concept is better in two and three.
So the score here is two versus two, which means neither one nor two dominate each other, right?
Even if it's-- what if it's three to one?
What if it was a three to one score?
It would still be true, right?
That one doesn't-- one is, in a sense, better because it's better than the other.
But it doesn't dominate it, because dominance is a very crisp definition.
Now this is different, right?
Compare one versus number six.
Concept one versus concept six.
One is better in all four criteria to six.
So clearly, solution one dominates solution six, and as a rational decision maker, you can eliminate concept six from the set because it is completely dominated by concept one.
OK?
So in order to be dominated, a solution must have essentially a score of zero in a pairwise comparison.
Now, if we apply this to the full set of eight we can actually show this as a matrix.
I call that the domination matrix.
It shows which solutions dominate which other solution-- the horizontal rows and the vertical rows.
So the way you read this is so where you see these dots, these are dominance relationships in the pairwise comparison.
So this tells you that solution two dominates solution five.
This tells us, if we look column-wise, solution seven is dominated by solution two and solution eight.
So, if we do the row sum-- if we sum this along the rows, the row indicates how many solutions this particular solution dominates.
And-- but the question was about non-dominance, so we need to look at the column sum.
The column sum indicates by how many other solutions the kth solution or concept is dominated.
And when you do this column sum, you can see that concepts one, two, three, and four have a zero.
They're not dominated by any other design.
Column eight has a zero, and concepts five, six, and seven are dominated each by at least one other concept, which means that they're dominated, and the others are non dominated.
So when you do your concept selection, you can actually apply this to a much bigger set.
Your filter out all the dominated solutions.
And it turns out the bigger your number of alternatives or architectures, the smaller percentage-wise the set will be of non-dominated solutions.
And those are the ones you really want to focus on.
Is that clear?
I know this takes a little thinking about, what does this really mean?
But but that's a very rigorous way to do it.
And then this gets us to, eventually, this gets us to the notion of Pareto optimality.
So what's the relationship between Pareto optimality and non-dominance.
So let's say we have two objectives, J1-- we want to maximize J1.
Maybe that's performance, endurance.
We want to minimize J2-- maybe cost, for example.
So the Utopian point, or the optimal corner to be in, is the lower right.
We want to maximize J1, minimize J2.
There's nothing here.
It's empty.
There's nothing feasible here.
So the best we can do is get close to it.
So when you look at a set of discrete points, discrete concepts, those concepts can have three properties.
They can be D, meaning dominated.
So all the red points shown here are dominated.
And the way you can see this if you draw a little-- let me try to let me try to draw a box here.
Where's the eraser?
So let's pick a point, a particular point-- Let's pick this point right here.
Why do we know that this point is dominated, just looking at it graphically?
How do you know that?
Go ahead
The point is not at the front
It's not at the front, but the way the way to find first of all, whether it's dominated, which is relative to the other solutions, is we can just draw a horizontal line here, and a vertical line here.
And so in a multi-dimensional space, you're essentially drawing a hyper-cube.
And down here is our utopia.
This is where we would love to be, but we can't be.
It's not feasible.
And you see in this box, between the point and the utopia, there is this other guy here.
This point here.
And because it's there, it's closer to the utopia, and it dominates this point in both objectives.
Let's take this point here.
Let's do the same thing.
Draw a vertical line and a horizontal line.
And there's a whole bunch of points, all these points right here are all the box, closer to the Utopian point.
So this point here is actually dominated by one, two, three, four, five, six seven other points.
So that's essentially dominated.
Let me erase this again.
Then we have these darker red, purple points.
Let me just circle them real quick.
So we have this point here, this point, this point, this point, this one, and this one.
And those are what we call non-dominated, like we just did the filtering, meaning there there's no other point in the set that's better than it in all criteria.
So you would consider them as concepts to choose from.
But they're not exactly on that dashed line, which means that there is a way to improve them.
There's a way to tweak them, to optimize them further, if you could to get them right onto that dashed line.
They're close but they're not exactly on that dashed line.
So you could tweak these concepts to get them closer to the Pareto front.
But they're still pretty good, because they're non-dominated.
And then we have those grey points, which are shown here.
There's three of them.
Let me use a different color for these.
Let me use a different color, maybe green.
Here.
This point, this point, and this point, they are Pareto optimal.
Meaning they're on the Pareto front.
There's no way to improve them, such that you could make both objectives better.
Right?
If you want to make J1 better, you have to sacrifice on J2.
So the only way to modify these points is to slide along the Pareto front, meaning you're trading off the objectives.
Do you see that?
And so, in my spring class that I teach, called Multi-disciplinary System Optimization, we get into this pretty deeply.
Like how do you actually do this?
But the point I want you to keep for now is that when you select the concept, do not select a dominated one.
Get rid of the dominated ones.
Filter them out.
And then you still have to choose among the others, and that's where your preferences come in.
Right, so given the blue and green choices here, which one are you going to recommend at the end?
And that this is sort of the preferences between maximizing J1 and minimizing J2.
That's a social process of coming up with your preferences.
But you now determine those preferences after you've scored all the concepts.
Do you see the difference?
That's pretty fundamental.
OK.
So, one more thing and then we'll wrap it up.
So now you've gone through all this work.
You've done your concept generation, concept selections, scoring, and you found the one concept that you're going to go forward with.
That's when you do PDR.
So what is the PDR?
The Preliminary Design Review.
You explain what concept and system architecture was chosen.
You explain why it was chosen, and you want this to be approved as the design baseline for going forward and doing more detailed design work.
You compare it against the rejected alternatives.
Typically at the PDR, you will spend some time on saying, these are the other interesting concepts that almost were selected, but we didn't because of these reasons.
You show some quantitative analysis that gives confidence, especially you want to show that the requirements that you had defined and agreed upon at the SRR can be satisfied by your chosen concept.
There's maybe still some risk or some uncertainty.
So any risk reduction experiments or prototypes.
So you can actually do prototyping at this stage.
You don't want to spend too much on that, but you want to do enough of it so you can reduce risks.
And then you do a preview of the detailed design phase between PDR leading up to CDR.
So here-- I'm not going to read this-- but this is the description of what the PDR is all about.
In the NASA system engineering handbook on page 177, there's a very detailed description of entrance criteria, exit criteria, et cetera, et cetera.
So here's a description from a PDR from a European program, and you can see it's a pretty big group, right?
This is like 30, 40 people.
And because the PDR is such an important decision, that's when you also invite external stakeholders.
And you want to make sure that people come away from the PDR with confidence that, yeah, we have a good system architecture.
We've picked the good concept, and it's worthwhile now putting all this work into getting it to a detailed design that can actually be built.
OK. Any questions about PDR, or any experiences from PDRs that people want to share?
I have a question about the PDR.
So you said there's 30 people in that picture.
How do you reconcile that with seven plus, minus two rule?
Well, it's seven, plus or minus two, squared, right?
So you have a hierarchy of people.
Not everybody is a decision maker here
I see
So you probably have people who did some of the thermal design, some people did the cost modeling.
So, it's a great question actually.
You want many, many voices and people who contributed to make sure the PDR presents all the information you have.
You don't want to do a PDR with just seven people
But I guess you could say that in the presentation, I guess, only seven plus or minus two, will people be driving at that point in time
So there will be a leadership, right?
There is an organizational issue.
But the point here is that it's a big review.
It's a big milestone.
Please, go ahead
The big issue here is that all the people who are not decision makers, that are in the picture, they will have to implement the detailed design, so it's very important to have them in the process.
Otherwise, you have to go back to your organization and do the PDR again to explain the what's and the why's.
This is why you try to have these very large meetings.
When you do a pre-PDR, you do a dry-run, and you send stuff in advance and make sure that you're really all agreeing finally, if possible, you get the discrepancies off the table.
But then you all agree that this is how to go forward and then the worker bees can go and do it
Yes, that's another reason for being inclusive.
I agree with that.
Any other-- who has gone through a PDR in their prior work?
Anybody want to share?
Marissa.
Tell us about how that went.
I mean as much-- I know you can't probably we share everything but--
For us, I mean, it was a pretty big milestone.
You work a lot leading up to a PDR and then you finally have all the really big stakeholders in the room.
And a lot of them have been involved during the design phase, but when you actually have them all sitting there.
And I think getting the buy-in was also really-- I think that's a really big part of PDR.
Like normally you're feeling pretty confident in your designs, but then actually making sure whoever the agency is that you're working with, then your stakeholders are also kind of all on the same page.
It's pretty critical
Did you discover problems or discrepancies during the PDR?
Yeah.
I definitely think-- just generally we've had delta PDRs, if things weren't completely thought through.
Or you'd have a fair number of action items that come out of PDR to go and address before you really are allowed to start moving towards
Good.
Any comments or experiences with PDRs at EPFL?
I did a PDR at the European Space Agency during the REXUS BEXUS program three years ago, and as far as I remember it was very appreciated to bring as many details as possible.
And to be very close to that critical design review stage in the
That's interesting.
So, that's good.
I think there's a good to have detail and so forth.
But what's the downside?
Can you think of a negative what would be what would be not so good about almost being a CDR right there?
Can you think about a downside to it?
Probably not taking too much time in order to clarify the concept and the requirement because you are in a rush to finish the product
That's a good point.
So it takes more time.
But what I'm thinking about is it's very difficult to then undo the decision or backtrack.
Let's say that at the PDR that there's some-- it could be that you really forgot something very important.
And that your concept is actually flawed.
But you didn't know it as you were working inside your team.
And then if you have a design that's almost CDR quality, it's very difficult to undo that.
So I would say that a PDR-- I actually believe that at a PDR you should have some detail, but not too much yet.
I think a true PDR should be at the system architectural level, with some detail but not all the details.
But thank you.
That's a great comment.
Thank you very much.
So let me close it here.
So in terms of concept selection during conceptual design, we can use Pugh matrix selection.
And that's essentially useful when we don't yet have those detailed mathematical models.
It's qualitative.
And then as we move more into preliminary design, we use utility analysis, especially for noncommercial systems where a net present value-- a financial-- in commercial companies, you will often do a financial analysis as a measure of utility.
And then this concept of non-dominance is very, very important.
Generating many designs and then applying Pareto filters to try to come down to the non-dominated set.
So that's essentially the essence of it.
So I hope that was clear.
A-4 will be posted later today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's get to work.
The V-Model, well, believe it or not, but we've arrived today at the bottom of it, the pointy part of the V. We're arriving at the bottom of it, and the bottom of it is basically you driving down the design in all its glory and all its details.
And we call this design definition, and then I want to talk about multidisciplinary optimization, including what happens at the CDR, the critical design review.
Yeah, go ahead
All right, I do have a quick question about A4 based to this.
Do you want a CAD drawing or a hand drawing?
I'm fine with a nice dimension hand drawing, but we're gonna talk about CAD next week.
So I do think it would be good to have a CAD drawing, but you know CAD drawings can come in very different levels of detail, right?
So I think it would be good to have an idea, if the PDR package-- most of the PDR packages that you see that have been submitted in prior years at CanSat, they do have a CAD drawing in there for their sort of suggested configuration.
And you know, it is useful because you get to see whether the packaging concept really works.
You know, the sketching, you can get away with a lot just sketching.
So my answer would tend to be yes, but if absolutely, you don't feel like you need it or you don't want to do it, then a nice hand drawing with dimensions will do as well.
OK, so let's talk about the bottom of the V. The other thing I'm gonna cover today is this idea of multidisciplinary design optimization, and so where does that fit in?
So you remember we talked about concept synthesis, concept generation, screening, and selection, and then at the PDR stage, you know, we picked one architecture.
So today's topic is really how do you get this architecture, which is shown as this square here to an optimal?
And I put optimal in quotes here because we're gonna argue what that means, but you've refined the design, such that you can go present it at the CDR.
So you take this box here, which is your concept, and you open the box-- you can see this on the right-- and you really dive into the details of the various subsystems and components and how they're connected.
And what you're trying to do is fine tune your design.
So this is shown here schematically.
Let's say there's a design variable X1 and you want to find where is the optimal point?
This point here seems to be optimal to minimize metric three and that's the design that you're gonna propose.
So today's topic really is about fine tuning the design and coming up with a CDR level design, that's what we want to talk about.
And so what I'll cover is different topics.
So first of all, the NASA view of this.
It's called the design definition process.
Then I'll talk about MDO, multidisciplinary design optimization, the use of so-called CDFs, concurrent design facilities, and then finally, what happens at CDR.
So the general idea of the design solution definition process is that you start with something like this on the left.
This is a product breakdown structure, essentially your system architecture, your system decomposition.
And on the right side, you then translate this to a detailed design, OK?
So for a launch vehicle, we have, you know, the flight segment, this is what-- actually it's gonna fly and it has a payload element, the spacecraft bus, and then the launch and the launch accommodation.
So that's the general idea is you have high-level breakdown, but then you need to design every piece.
You need to have chosen your materials.
You need to have chosen the dimensions.
You need to understand the interfaces.
And for every part, you also need to decide what?
What's a big decision that you need to make when you go to the detailed design, beside, you know, material, dimensions, software?
What is always a big decision to make when you do a detailed design?
Go ahead
Maybe the interfaces between the two
Yep, interfaces, all that.
What I'm trying to get at is COTS.
Who knows this acronym, COTS?
Let's see if the, EPFL, anybody know COTS?
What does it mean?
I think we can't hear you right now
Commercial off the shelf
Commercial off the shelf.
So you have to decide in the detailed design are you going to pick components from a catalog, or are you going to custom design these components, right?
And so one of the things at the CDR, that I always ask the question-- so you present your CDR design.
What percentage, what fraction of your parts at the lowest level are COTS and what fraction did you custom design?
Mike, you've done a lot of these unmanned aerial vehicles and so forth.
Can you talk a little bit about, you know, what's the typical fraction of COTS, and how do you make that decision when you design, you know, a flying thing?
It really depends on the scale and how robust.
The problem with COTS in the unmanned world is that you can easily go to hobby grade parts.
And so if you want to get into more of the commercial, kind of more robust, almost more like defense kind of, like, robustness and you know, durability, you really have to start going to custom designed parts.
Because you know, if you can do a whole design space of, you know, all the COTS parts, but you know, they're always just kind of, you now, Chinese-made electric motors and batteries, and none of them that will perform to, like, commercial spec, but it just depends, really, what you need for the vehicle
I see.
So what you're saying is there's a gap.
You can get a lot of COTS stuff at the hobby shop level, right?
So they're pretty affordable but low reliability.
Or you have really high end, very expensive, you know, low production--
Like traditional aerospace grade, yeah
But there's not much in the middle.
Is that what you're saying?
Yeah, for sure.
Yeah
OK, good.
So you know, that's the reality of detailed design that you're gonna face.
And surprisingly, in kind of the formal system engineering methodology, there's not a lot of guidance on this.
There's not a lot of, you know, guidance to say what's the optimal fraction of commercial off the shelf parts?
This is really something you have to work out yourself, and depending on, you know, whether you're in aerospace or medical devices or automotive or making toothpaste, there's-- actually, yesterday there was also a big case study on diapers.
Diapers is a $3 billion business, believe it or not.
And they were going for-- their solution A was to go for a COTS solution, you know, a lot of vendors.
And then they said, no.
It's just gonna give us, sort of, average, mediocre performance.
We want something much better.
So they went for a much higher custom design solution.
Took them took them a while to do it, but it was worth it in the end.
So this is a big decision you have to make.
So the point here being, at the end, you have a very detailed design.
You have blueprints.
You have something you can actually build.
So you know, this is a list of all the things you need to do.
You define the solution space, develop your alternatives.
All the things we've already talked about at the conceptual level, but you do this at every-- you do it at the subsystem level, and even at the component level.
What are the alternate designs?
Cost performance and schedule.
Select your design solution, drive it down to the lowest level of detail, and then identify the enabling products, which would be like support equipment and things like this.
So the best illustration I can give you why this matters is these two pictures I'm gonna show you.
The top picture that you see here, this is a picture that was drawn in the '70s to sell the space shuttle program to US Congress.
This was the vision, right?
This is the orbiter, and you can see it's sitting there in this pristine hangar, right, very clean, shiny floors, maybe four or five people.
The payload bay is open.
There's, like, one or two carts of ground support equipment.
You just load up your new satellite, close the doors, and off you go for your next mission.
That's a reusable vehicle.
That's what we wanted, and then what we actually got is this.
You can't even see the orbiter, right?
But I promise you, it's in there.
It's hidden in that scaffolding.
So this was the-- it's retired now, but this was the orbiter processing facility at the NASA Kennedy Space Center.
And the reality was that to turn around an orbiter between flights was almost like rebuilding it.
And the two subsystems that required the most person hours of work were the main engine-- the main engine basically had to be disassembled and inspected and so forth-- and what was the other system?
What's that?
Heat shield
The heat shield, the TPS, thermal protection system-- if you've ever seen-- who's seen the shuttle close up?
OK, about half, half here.
At EPFL, who's had a chance to see the space shuttle, the orbiter, close up?
Anybody?
Go ahead
One or two
One or two.
So those of you that have seen it, what did you notice about the heat shield?
Did you notice anything about the protection, the thermal protection system?
Who's seen it?
We do not hear very well here
You can't hear me?
So the point I want to make here is that the thermal protection system, every tile is different.
Every tile has a different shape, a different curvature, and there were good thermodynamic reasons, aerothermodynamic reasons for that, but in terms of actually designing, manufacturing, and turning around the vehicle, it was a nightmare, right?
So that it turned out to be not a very reusable vehicle in the end, and why?
Because a lot of the detailed design decisions ended up being critical, in terms of not really giving us what we wanted, in terms of macro level properties of the vehicle.
So detailed design really matters.
Did you get that point, at EPFL?
Is it clear?
It is
OK, thank you.
So here's some-- I'm not going to go through this in detail, but here's some flow diagrams, process flow diagrams, from the System Engineering Handbook that essentially describe, you know, how this works.
So your inputs are your baseline logical decomposition model, which is essentially your system architecture, right?
And then your baseline derived, your detailed technical requirements.
You go through all these processes, through all these steps, and then you end up with a whole bunch of outputs, which are more detailed requirements.
The specifications, do you remember, we teased apart requirements and specifications?
Requirements are what the system should do.
Requirement specifications are how the system's actually been designed, right, blueprints.
So that's specifications at the subsystem level.
And then you get other things, like a verification, validation plan, and then logistics and operating procedures.
So very, very detailed work.
The other thing that's important is all the design considerations that go into making detailed design decisions.
So here's a fairly long list of them, and you can group them into performance, availability.
This is more related to reliability.
So you could have a system that has high nominal performance but is poor reliability, right, so it's not available very much.
That's not good.
We want high performance and high availability.
That gives you your technical effectiveness.
And then how do you actually operate the system?
So operations, maintenance, logistics, so that's process efficiency.
Together, that gives you system effectiveness.
And then, of course, you have the total cost of operating the system-- lifecycle cost, total ownership cost.
And then system effectiveness and cost together, give you what's called here affordable operational effectiveness.
So I would argue, in terms of the space shuttle program, it had high system performance.
It had OK system availability.
They had-- definitely there were some reliability issues, but it was manageable, but it had very poor process efficiency-- operations, maintenance, logistics very, very difficult.
So system effectiveness wasn't that high, and as a result, the lifecycle cost was also very high.
There's a very famous paper, where after the shuttle program was ended, a lifecycle cost analysis was done, and it turned out that one launch cost about $1.5 billion.
So during the program, people would say, oh, one launch is maybe $500 million, you know, the variable cost, and the rest is fixed cost.
But once the program is done, it doesn't matter anymore whether it was fixed or variable cost.
All you know is what was the total cost of the program, how many launches did we actually do, and you just take the ratio.
And it was $1.5 billion per launch of the shuttle.
And so I would say that's way off from what the original target was.
Yeah, Veronica?
Does that number account for crew and crew training?
Good question.
I think so.
I think it basically-- whatever was appropriated by Congress for the shuttle program, and I think that would include crew operations, as well, yeah.
So I'm happy to point you to that paper, actually.
Yes?
It also accounted for R&D costs as well, right?
Yes, that's correct.
It includes everything, all in.
R&D costs-- R&D took about 10 years and then operations was about 30 years.
OK, so you see-- and of course, within these are budget limits.
So part of the reason why the shuttle didn't have that high process efficiency was because the R&D budget was capped pretty tightly.
So rather than optimizing, you know, for maintainability and turnaround, they said, we need to have the system performance.
We need to have reasonable system availability, and then the rest, basically, is we're gonna deal with it later.
So what effectively happened is they capped R&D cost, and as a result, the lifecycle cost ballooned.
I would argue if they had spent, you know, two or three more years optimizing, you know, maintenance and turn around, really making it reusable, the lifecycle costs could have been much better, but that's not how it worked.
So you have a lot of these competing requirements.
And so I want to just ask you this-- this is the first concept question today, OK?
So imagine you're-- this is a very detailed design.
This is just a bracket here, OK?
And this bracket was designed so it has, like, a tip load.
Think of this as a cantilever beam.
So it's attached on the right side or left side here.
You have a load, and then this is your displacement under a given load.
And these three brackets all have the same mass.
They use the same amount of material, OK?
So you can think of this as you get from the very simple design to the more intricate design, you have more design freedom.
So you get better performance.
You see the displacement goes down, but you also have more complexity.
So it's more difficult to optimize.
It's more difficult to manufacture and so forth.
And so we can actually show these three choices here on a manufacturing cost, per unit cost, versus the displacement, structural displacement under load chart.
So our simple design, our one bar design, has a displacement of 2.5 millimeters under load, and it costs about $2 to make.
The two-bar design has much better performance, like 0.75 or something like that, costs about $3 to make.
And then this 17-bar intricate design has 0.6 millimeters, so the best structural performance, but its cost is about $8 per piece.
So my question to you is which of these three designs would you select and why?
So if you could go to that link, enter your answer, and then we'll look at the distribution.
So we have 39 responses.
That's good.
Ah, interesting.
OK.
So one of you said you're gonna go for the super-duper 17-bar design.
Most of you, 74%, said the two-bar design.
And one of you said one bar.
And then eight of you, 20%, said you're not sure.
I guess you need more information, right?
So who picked the one-bar design, the simple design?
OK, so please explain why you picked that
Well, for my application.
I'm planning on mass producing them, and it's for something that doesn't need the degree of displacement that the other ones are doing, and so I'm really only seeking to minimize manufacturing cost, and the one bar is the easiest to manufacture and also gives me room down the road to iterate if it's not working
OK, so So what would be an example, industry or application?
Like a children's toy, perhaps, where you don't need very strict requirements on displacement, something that you're mass producing
Something mass produced, you know, for sort of lower requirements.
OK, I think that's a valid answer.
Most of you said the two-bar design.
So who said two bars?
Let's see at EPFL, who picked the two-bar design?
Sure.
It gives you the biggest biggest bang for your buck.
For not that much more cost, you get a much more sturdy design and that's compared for the 17-bar one.
Really, it's still pretty sturdy, but you're paying much less, so it seems optimal
Yeah, so if you think about this in a Pareto sense, I mean, they're all Pareto-optimal, but this one is at the knee of the curve, right?
So you're getting, you said bang for your buck or value.
If you did, like, a displacement or one over displacement, I guess, per manufacturing cost, this one would do pretty well.
So that's also a very sensible answer.
Who picked the 17-bar design, the really intricate one?
Who picked that?
Oh, I think I might have picked that when I tried out the form.
[LAUGHS] I think so.
I can't remember now.
It's been already a while ago.
So if, indeed, I was the one, so how would I justify this?
OK, so here this bracket will go-- this will go on the Mars 2020 Rover.
And the Rover will see a lot of different thermal gradients.
You know, it has to be very precise because it holds scientific instruments that need to be, you know, pointing very accurately.
Even though the mass here is identical, but I think we could shrink this.
You know, we get the best performance structurally for the mass constraint that we were allocated.
And cost, you know, this is all taxpayer dollars, so this is a government program.
So we can spend the money.
Of course, I don't mean that really, but it is a high-performance application.
Therefore, even though the benefit is only small, that small benefit is worth it when you look at the whole mission.
So we're we're going to pay for that, and we're going to go for this high-performance design.
Do you see how that works?
So the right answer-- and I think a lot of you said, you know, you're not sure, and it's because-- who said you're not sure?
Nathan
I said not sure because you don't know all the specifications or what the tolerances are or any other information that would lead you to make that decision
Right, so you know, what are the requirements, right?
We don't know the requirements here.
Are the requirements shall or should requirements?
If it's a shall requirement, what it means is there would be some hard limit here, right?
The displacement shall be less than one millimeter.
If that's a hard requirement, then this solution falls out, does not satisfy.
And if it's a should requirement, then it's a goal that you're gonna trade off.
The point I want to make here, this lecture is about detailed design.
This is detailed design, right?
And you're gonna make hundreds or thousands of detailed design decisions just like this bracket and together, they're gonna give you, you know, the performance cost of your whole system.
And so every detailed design decision you make needs to be informed by the requirements but also the trade-offs between these properties.
Does that make sense?
OK.
So and that leads us-- and so you want to make sure you optimize your system.
You want to make sure you get-- and I liked, Katya, the way you said it-- get the bang for your buck, right?
You want to make sure you don't have a lot of inefficiencies in your design.
So that's what MDO is about.
So let me go to MDO here, what it is.
So there's a technical committee in AIAA, the American Institute for Aeronautics and Astronautics, that was founded about 20 years ago.
So it's defined as an evolving methodology, a body of methods, techniques, algorithms, and related application practices for the design of engineering systems, coupled by physical phenomena and involving many interacting subsystems and parts.
So the main things you need to do MDO are a mathematical model of the system, analysis capability, and in some cases, approximation concepts.
So what that means is not-- if you put all this together, it could be very, very slow and computationally heavy.
So you might approximate some of these things, and I'll explain a couple examples.
Sensitivity analysis, optimization processes, and a human interface-- those are the main ingredients of MDO.
Who's seen this chart before?
OK, about three or four of you.
Who's seen this chart before at EPFL, this particular graphic?
Anybody seen this before?
Apparently, no one
OK, so let me explain.
This is a cartoon, OK, and I've experienced this myself working at McDonnell Douglas in the '90s.
So each of these airplanes shown here is what the airplane would look like if that particular discipline or group could call all the shots, OK?
So let's look at a couple of these.
So the controls group is all about controllability.
So you see the flight control surfaces are very big.
Now, the problem is the linkages, you know, these control linkages are on the outside.
That's really bad for aerodynamics, but the controls people say, I don't care about drag.
I need to make sure my control surfaces are deflecting appropriately.
Here's the hydraulics group, right?
It's all about the hydraulic pump, the filters, and getting hydraulic pressure to all the different subsystems.
You know, the big engine here, the power plant group, the engine is the most important thing.
The loft group-- loft means-- lofting is surfaces.
You're dealing with the shape of surfaces.
So let's keep this simple.
So we have a little, you know, balsa airplane with little straight surfaces, very easy for loft.
Production is kind of similar.
The aerodynamics group wants a very nice high aspect ratio wing, very thin, very low drag.
Problem is, there's no room for passengers or cargo here.
You see this, the stress group?
They have a nice wing here in the shape of an I-beam.
I-beams are great, you know, this is good if you're a civil engineer but not so much-- none of these things would work on their own.
So what we really want is something like this, where all these considerations are blended and optimized so we get a high-performance system not just a high-performance subsystem.
Said another way, a high-performance system is not the same as just a sum of individually optimized subsystems.
That's, in a nutshell, what MDO is about.
So one of the readings for today, post readings, is this paper, 6A.
And this is based on a big workshop we had in Germany about five years ago.
That's actually longer.
It's probably seven years ago by now, but the paper came out five years ago called, "MDO, Assessment and Direction for Advancement."
And this is a pretty international group that put this together.
And what we do in the paper, we describe the history of multidisciplinary optimization, where it is today, where it's heading, and some of the key milestones.
So this chart is essentially a kind of history of MDO.
I'm just gonna point out a few things.
The roots of MDO are in structural optimization.
So you saw the bracket with all those-- so there's actually a field called structural optimization or topology optimization, and it's very mature now.
So you can buy, in some of the CAD packages, there's some specialized tools for optimizing structures.
And when you look at it, it looks like Swiss cheese with big holes, an Emmentaler, big holes.
You don't need a lot of structure that doesn't do work for you.
So a lot of these structures look like webs, and the reason is because the load path has been highly optimized.
For example, if you look at the A380-- I don't have a picture right here-- but you can look at an A380 wing spars and different structures.
They have more holes than structure because they've been highly optimized.
So that's the roots of MDO, starting in the 1960s.
Then, I would say, in the '80s, mid-'80s, we started having more complex decomposition techniques, and I will describe one of these to you today.
Optimization algorithms were started to be built into mainstream programs like Excel, MATLAB, Mathematica.
And then more recently, you know, in the 2000s, we had commercialization of multilevel algorithms, and the one that I'll describe to you is called BLISS.
So let's talk through a very specific example, and then one of those techniques.
So here's the example.
You're designing a wing, and in designing a wing, the two main considerations are structural and aerodynamic.
So here's a simplified wing.
It has a wing area, it has a sweep angle, it has an aspect ratio, and so forth.
And there's loads acting on the wing as you're flying because of the pressure differential between the upper and lower surface.
So there's some net force, P, acting on this wing.
And as a result of that, the wing will warp.
So you will have some twist angle in the wing, and that's just due to the structural deformation.
So that twist angle and the displacements, in turn, will change the drag that the wing experiences.
So the lift and drag of the wing will change as a function of twist angle.
But as the lift and drag change, the loads, of course, then change as well, and it feeds back into the structure.
And in this case, what we're looking for is maximum range.
We want to maximize the range of the airplane.
And this is a sort of simplified version of the range equation, the so-called Breguet range equation.
And you can see that the structure influences the range in two ways-- directly by weight, the actual weight of the structure enters here twice in the range equation, and then indirectly through its stiffness, right?
The softer it is, the more displacements, the more you affect the drag.
So it's not so clear then what is an optimal wing?
Is it the lightest wing possible?
But the lightest wing possible will have a larger twist and may have more drag, which will reduce your range, right?
So should you optimize the wing for lightness or for this minimum displacement or some combination or mix of the two?
And the answer, of course, is it's a mix of the two.
There will be a sweet spot somewhere, you know, between a very stiff and very heavy wing and a very, very flexible, lightweight wing.
There will be some optimal mix of the two, and that's the key here.
So let me discuss with you briefly-- this is just one of the methods.
There are several methods for doing multidisciplinary design optimization, but this is one of them called BLISS, Bi-Level Integrated Systems Synthesis.
And the general idea here in this method is that we have a lower level optimization of each subsystem but it's coupled through a system level optimization.
And what you want is, you know, an optimal design at the system level, not just optimal subsystems.
So the context here is we're gonna design a high speed, a supersonic business jet.
It's kind of shown in this gray picture.
And we have four major subsystems we're gonna consider-- propulsion, aerodynamics, structures, and then the performance, which, in this case, is range, OK?
And what's important here in decomposing the problem is the x's and the y's.
Now, the x's are the design variables.
Do you remember, we talk about the design variables, the knobs you can turn as a designer?
And so these here are-- the upper ones, we call them local design variables, you know, like the throttle setting, the tail sweep, the wing moment arm, the thickness of the wings, the taper ratio.
So the idea of a local design variable is that design variable only is used in one of the subsystems.
It's local to that subsystem.
So you basically can choose that in each of the subsystems, and it doesn't really affect directly any of the other subsystems-- indirectly, yes, but not directly.
And then we have the y's here.
The y's are the outputs from one subsystem to another.
So drag, lift, NZ is your maximum load factor, your range, your specific fuel consumption, your wing twist, and so forth.
So those are outputs from a subsystem or discipline that either matter at the system level or are required as inputs by another subsystem.
And then finally, the last group here, these are known as the shared design variables, so the wing aspect ratio, the altitude, mach number, wing sweep.
Those are basically required by more than one subsystem, so two, three, or four, in this case.
Then they are considered as shared variables, and they would be considered system level variables because they matter for more than one subsystem.
So what this particular method does is essentially decompose the problem into the various subsystems, so in this case, again, aerodynamics, propulsion, structures, and range.
And these subsystems are sending this information to each other and are all tied together at the highest level through the shared variables, the shared design variables that are shared by at least two subsystems.
So again, local variables are only unique to a specific subsystem, and then for the wise, these outputs, internal outputs, there's a distinction between the Y stars, which are coupling variables that are input to a particular subsystem and then the Y hats are coupling variables that are output from a particular subsystem.
So let's zoom in now on just one of the subsystems, let's say, aerodynamics, and see how this works.
So here we're zooming in on one subsystem, and that's the aerodynamics.
And the idea that you can actually optimize just the aerodynamic subsystem by itself.
So what are the inputs into it?
We have aspect ratio, wing twist, and W. So what is W?
And this is key to this particular method-- the W are the weights, the weights that you give to the outputs.
So what you're optimizing here is some objective function, f, shown here below, which is W1, Y1, plus W2, Y2 plus W3, Y3.
So you're either maximizing or minimizing that.
And what is this here?
It's lift, drag, and the lift to drag ratio.
So what's really cool about this method is that what it essentially is telling the subsystem is how to properly weight these different outputs in order to optimize the subsystem, taking into account the system level objective.
So the weights are not pre-defined, but the weights themselves are a design variable.
And so you can see the formulation on the right side.
So given Q, which is your shared variables, the Y stars, which are the inputs from the other subsystems, and the weights, minimize your objective function by varying your local design variables subject to a bunch of constraints, OK?
And so what is one of the consequences of being able to formulate the problem this way?
So this could be a very high fidelity, aerodynamics model, computational fluid dynamics, or it could be what?
Or it could be a lower fidelity code, right?
As long as the inputs and the outputs are the same, you can now plug in, you know, different fidelity levels of analysis, you know something simple or something based on experimental results, you know, panel method, very high fidelity CFD.
This becomes very modular.
The other thing you can do is you can pre-compute a lot of these responses, right?
So for a given set of inputs, you can pre-compute over a whole range of input variables what the response of that subsystem will be, and you can store that information and then use what are known approximation methods, like response surface models, to save that information, such that when you do the full system optimization, you don't have to rerun, you know, the detailed codes every time.
You can just pull from the approximation you've pre-computed already.
So that's one of the really powerful things about this method.
And that's what's shown here, is you substitute-- you have a series of approximation models, one for each of the outputs of the system.
You see that?
So these little wiggly things here are essentially an approximation of that input/output relationship at the subsystem level.
Could be an empirical function, a neural network, a response surface method, or you know, simplified engineering analysis.
So you do a lot of this work offline ahead of time for each of the subsystems.
Does that make sense?
Any questions at EPFL?
Is that clear how this works, at least in theory?
Everything's fine, thanks
OK, so what we have now is essentially these subsystem approximations, and now we can use them for system level optimization.
So now let's look at the system level optimization.
So that's essentially given approximation models for optimized subsystem outputs, we want to minimize some objective function.
So F, here in this case, which is a function of our shared variables, our relevant outputs, Y star, and the weighting factors, right, the omega-- not the omegas, the W's, which are the weighting factors given to each of the subsystems.
And we of course, have to satisfy constraints at the system level.
So let me show you an example then of what this looks like.
So here's our supersonic jet.
This is our cycle zero, our initial condition.
Here's the planned form of the jet, and I'm going to show you down here range and just one of the design variables, or a set of design variables, which is our wing box sheet thickness.
So you go through this, and it converges after 10 cycles already.
So this is what our business jet looks like after 10 cycles.
And you know, you see there's a pretty big difference.
So the fuselage is much longer.
This by the way, this is the same in ships, right?
Your speed that you can reach, your max speed is driven by your fuselage length, your water length.
You can see the wings have a pretty high sweep angle.
This is a supersonic jet.
And if we look at the details, we see that the range here, the range is about 3,000 nautical miles, and what you see is that the exact range that can be achieved and the approximate range merge.
So the approximation models that you have in this method are only approximate.
There's some error.
And you know, as you run the model several times, several iterations, the difference between the approximation and the actual high fidelity result gets less and less, so that it's actually a feasible design.
And then over here, we see the different thicknesses for the inner wing box, the mid span wing box, and the outer wing box.
And you can see that early on in the first four or five six iterations, things are kind of moving around, but then they're settling out.
And so for this maximum objective of range, you can see that the inner, mid, and outer wing box thicknesses are quite different.
So that's the general idea here is that this result-- this is a fairly detailed design was produced based on a system level optimization that included consideration of aerodynamic structures, propulsion, and so forth.
And there are several other methods, but this is just one of the methods that's been actually quite helpful.
The challenges now in MDO, in this multidisciplinary optimization are different trade-offs.
So one trade-off is between fidelity versus how expensive is it computationally.
So fidelity is over here.
So high fidelity means, you know, 3D finite element model analysis, computational fluid dynamics.
Then you have sort of intermediate fidelity or low fidelity empirical models.
And then on this axis, we have just doing trade studies, meaning you have sort of a baseline design and you just explore from that baseline design.
You do limited optimization or iteration or full MDO, exploring the full space.
And traditionally, you had to choose, am I gonna use high fidelity models?
But then I can only look at a few points.
So then the question is can you do better?
Is there a better solution out there?
Or you can do lower fidelity models and explore the design space very widely, and then the question is, can you believe the results because you've done this with approximate models?
So the real challenge here is we would like to do high fidelity and explore the full design space.
So that's one of the big research areas in MDO is high fidelity but also very computationally efficient.
And then the other challenge is quite similar.
So it's again the fidelity here, but then should you just focus on one subsystem?
How many of the subsystems, including, you know, the financial, operations, manufacturing considerations, so how many of those subsystems do you include in your optimization?
And so you know, ideally, you'd like to do the complete system, but then you may have to sacrifice on some of the fidelity.
So that's the breadth versus depth trade-off.
OK so any questions about-- I know this was very fast.
I do not expect you, by the way, for assignment four or five to do MDO on your CanSat.
This is really more to give you a feeling for, you know, how far things have come and some of the computational frameworks that exist to get you to a really fine tuned optimized design.
If you are interested in this, you know, we have the spring class.
And you know, I hadn't offered this class at EPFL in the past, but we actually do offer a distance version with the SDM program.
So there may be an option to do this at a distance as well.
We'll discuss it.
But any questions about MDO?
What it is, how it works, at least in principle?
Has anybody done MDO kind of work?
Sam, go ahead
I just had one question about if-- so here, we're looking at optimizing on one attribute, so range, but if we have multiple attributes, it's still up to someone to decide how important those attributes are in the end
That's correct.
That is exactly right, unless you have a way to combine all these into one objective function.
And to be blatant about it, to be-- you know, in the commercial world, that's actually clear.
It's profit or net present value.
So if you're designing a commercial product in a competitive industry, you can actually pretty easily combine all this into a long-- you know, as long as you include sort of the key drivers of, you know, your manufacturing costs.
If the product is not as optimal and not as performant, well, probably, you're not gonna be as successful.
Your market share might be lower.
So in the commercial world, people are doing MDO with profitability as the ultimate objective function.
Now, for situations where it's not about profit necessarily, but it's about a bunch of other considerations, then you can't easily combine it.
You can still do utility.
You can do a utility, you know, combine it in utility.
But I agree with you, I think a good practice is then, in that case, to do the multi-objective, and then you're gonna have a bunch of solutions that are all non-dominated, right?
And then it's a human decision making process to pick among those.
Any other questions or comments?
What about at EPFL?
MDO, anybody have experience with this before, multidisciplinary design optimization?
OK, no one, again
OK, that's all right.
So just to tell you, this is really a big deal in practice.
So you know, the structural optimization has been around, that's pretty standard practice, and you know, optimizing control systems.
So we've optimized individual subsystems for a long time now, at least two or three decades, but optimizing at the system level is relatively new.
And I think the best companies, the best organizations that are out there, they're really investing in this technology, and they're quite good at it.
And that's where you can really distinguish yourself from the competition.
So let's talk about concurrent design facilities.
And so I'm actually going to use some material here from EPFL because they have a very nice CDF, and I want to give credit here to Dr. Anton Ivanov at the Swiss Space Center for some of this material.
And actually one of the papers is also related to this.
Anton used to be at JPL.
He worked at JPL for close to a decade before going to EPFL.
So what is a CDF, a concurrent design facility?
It's essentially an environment where engineers of different disciplines come together to perform, you know, a system engineering study for a project.
Key elements that you need is the team, the process, the physical environment, which includes the A/V and the software, and then also some way to do knowledge management, meaning capturing the results of prior studies and ongoing studies.
And I'll show you some examples of CDFs.
We also do this in academia.
There's challenges for doing this in academia, a short learning curve, you know, synchronizing this with the academic schedule, and then a lot of turnover, of course, as students graduate and so forth.
So there's challenges.
And in practice, what we essentially have is, in a CDF, we have a set of models that are linked to each other.
And these models can be, for example, here's some Simulink models shown.
Here, there is some Excel models.
You can have CAD.
And one of the tricks is, you know, to link this together in a way that you can quickly execute different designs and look at the results from that.
So in industrial settings, probably the most famous CDF is the one at JPL, at the Jet Propulsion Laboratory, called Team X.
Team X has been around for about 20 years, and they just had an anniversary where they celebrated their 1000th study that was done at Team X.
A typical Team X study is about a week.
So you do a little bit of preparation, but people work together in the facility for about a week, and at the end of the week, you have basically a conceptual design pretty well worked out.
And the results have been impressive.
So studies have shown that the cost estimates that Team X came up with were within 10% of final mission costs, for those missions that were actually built and flown.
So of the 1,000 studies that Team X did, a handful of them were actual flight missions.
But every proposal that is written starts now as a Team X study.
So it's really a big deal.
The European Space Agency ESTEC has a similar facility in Noordwijk in the Netherlands.
As well, all the ESA projects are going through their CDF.
So one of these is [INAUDIBLE] which is a space-based observatory.
EPFL is involved in that project.
That went through the CDF as well.
Most NASA centers and then commercial applications in medical devices, painting, shipbuilding.
And like I mentioned in my introduction, even consumer packaged good companies are now building their version of a CDF, such that they can, you know, very quickly, within a couple weeks, come up with a new proposal for a product and all the major aspects, you know, the underlying physics and chemistry, the packaging, the business case, everything is considered at the same time.
Yes?
I was wondering why-- what makes the cost estimate so accurate with this method?
So there are three major ways of doing-- oh, Veronica should answer that question, since she's launching on cost modeling for her research.
There are three major cost estimation methods.
One is bottom up, you know, you have the product decomposition, and you add up all the costs.
One is called costing by analogy, where you look at a prior mission or a prior product and then you look at the deltas.
And then the third one is based on so-called CERs, cost estimation relationships.
It's a parametric method.
You can also blend these.
And you know, I think one of the reasons why it's accurate is because-- there are two reasons why, I would say.
So you always have a cost share.
One of the subsystems is costing.
So you have really people who been doing this for a while.
They know these models, and so there's some legacy information.
And the more missions we fly, the more products we make, the more robust, essentially, these cost estimates become.
And then the second, I think, is because people that typically work in a CDF, they're non-advocates, meaning it's kind of neutral ground.
So rather than pushing a particular idea or trying to be overly optimistic, you know, usually in a CDF, the calculations that are done are pretty non-biased, and that's not always the case.
So that's one of the advantages of the CDF is you're getting-- you know, people are trying to say, but really, let's make it look a little better.
No, no, we can't do this, because the model says, you know, it is gonna cost $280 million to do this satellite.
And that's based on, you know, within prior models, and if you overlay different models, you know, with some error or confidence.
That's the right answer.
So it's these two reasons.
There's experience, there's data, and non-advocacy, really taking a kind of neutral, unbiased position.
So benefits are, you know, improvements of quality, quick turnaround for ideas, better cost estimates, and then increased creativity and productivity in a company.
So let me give you a couple examples here.
This is a CubeSat.
So this is the ability to design quickly new CubeSats.
This picture here I'm showing you is the Swiss Cube that was launched in 2009, still operating.
I think it's the longest operating, continuously operating CubeSat so far.
There's like over six years of operational data now.
And so you know, for the next generation, which is shown here, how would you do a Cubesat in such a facility?
And there's three steps.
You define your decomposition level.
So you have your system level, your subsystem level, and then at the lowest level it's called equipment.
These are the actual components.
And you have your product decomposition here, and then for each of these components, you have different sheets that give you either component choices or give you different parameters like mass power with margins that allow you to then choose from this like a menu and pull together your overall design.
So that's a Cubesat example.
But I want to talk to you a little bit in more detail through a design of a suborbital space plane that was done in that CDF, and this was a project known as K1000.
So some of you may have heard of the Hermes, the Hermes space plane.
This was a project that the European Space Agency had starting even in the '80s, and it wasn't built, but there was a lot of studies done.
And this is essentially a Hermes-like vehicle but for space tourism, that's the basic application.
And so you know, how would you design a space tourism vehicle in a CDF?
Well, you have to start with requirements, right?
Without requirements, you're just-- it's like, you know, setting-- my analogy for designing without requirements is like taking your ship and going out to sea without a map, without a chart, with no destination port in mind.
You're just drifting in the ocean.
You've got to have requirements.
So here are the high level requirements for this particular vehicle.
Level one requirements-- you have to reach an altitude of at least 100 kilometers over sea level.
Why is that?
Where does the 100 kilometers come from?
Why is it not 80 or 90 or 150?
That's where space is defined
Right, and what happens to people who go above 100 kilometers?
They get their astronaut wings
They get their-- you have gone to space, and like, you know, you can claim that you've gone to space if you go above 100.
If you don't, if you are at 80 or 90 you don't.
That's kind of internationally accepted.
So that's really where that requirement comes from.
The second one is a zero g phase.
You should experience weightlessness for at least several minutes.
Who's done a parabolic flight before here?
The zero g, you know, the Vomit Comet?
Anybody?
Wow, we have to get you guys out there.
At EPFL, are you still with us?
We are
Has anybody done parabolic flights?
I know there's a flight campaign coming up in Switzerland later this year?
Yeah, it will be next year, I guess, but unfortunately, no one has done this right now here
All right, guys, we have to fix this.
This is unacceptable.
I mean, we have to work on this.
So it's very cool to do these parabolic flights, but you only get about 20 seconds, right?
20 seconds at the top.
It's like riding a big roller coaster, basically.
And then you go down, and then you suffer, you know, 1.7 g's or whatever.
Then you come back up.
But we want several minutes.
And then the next requirement is that the passengers should carry-- there should be six passengers carried on this vehicle.
Those are the top level one requirements.
Lower level-- level two requirements-- safety.
Don't exceed six g, positive six g's.
The vehicle must be controllable at all times.
So when you're out of the atmosphere, you have to be able to control the vehicle, that has certain implications.
The customer experience-- you should be able to see the Earth's curvature and atmosphere, that thin layer, at all times.
The spacecraft impact on the environment should be as small as possible.
And then the mass budget, the spacecraft mass should not exceed 11.6 tons with propellants.
Where do you think that comes from?
What kind of a requirement is that?
What flavor of requirement is that?
Yeah?
Transporting it for proper [INAUDIBLE]
Right, so in this case, it's actually a piggyback, right?
You either have a vertical launch or you have a first stage, an airplane from which you are dropped right?
So that's sort of an interface requirement.
OK, so once you know your requirements, you can really start developing your models.
So here's a model of essentially the kinematics of the vehicle.
So you have your environment, your atmospheric model, atmospheric density, pressure, mach number, speed of sound, a gravity model.
Then computing the forces and torques acting on the vehicle itself, looking at the derivatives, integrating that.
This is essentially the equations of motion of a vehicle in its environment that it's operating in, and then updating that with a certain time step, delta T. So you can zoom in on one of these, for example, this force model, computation of the forces.
And there's two frames.
There's the Earth reference frame and, then there's the body centric frame of the vehicle itself.
So you have, in this case, the aerodynamic coefficients.
You have the forces, lift, drag, pitching moment, and then you have the actuation.
This is assumed, you know, the pilots are here in control.
The rocket engine itself, secondary propulsion, the rudders, and then attitude thrusters.
You need those especially-- you know, once you're out of the atmosphere, the rudders-- the aerodynamic surfaces are no longer effective.
And we knew this in the '60s.
Vehicles-- like the X15 is very famous.
Well, it had thrusters that you could you could actuate once you're out of the atmosphere.
So you know, you can go pretty detailed in this.
You would then design the vehicle in a particular way, you know, particular engine, wing dimensions, aspect ratio, a particular flight profile, and then that would be a typical result that you would get in a CDF.
There's two curves shown here.
The blue curve is essentially altitude and kilometers.
So you see there's the 100 kilometers.
We need to get above this, right?
This was our requirement.
So we go above 100.
That's good.
We satisfy that requirement.
So looks like a parabola, and then we have a-- we glide here, essentially, to the ground.
The whole mission takes about 900 seconds, 15 minutes.
And then the yellow curve here is your load factor, your NZ, your vertical load factor.
And you can see that's a little noisier as a curve.
So we have higher acceleration as we accelerate to altitude.
And you can see that roughly here, roughly in the middle of the parabola, the load factor starts to-- you're basically in free fall.
You're basically ballistic, and the peak acceleration is about 4.3 g's here, so we satisfy that requirement.
And the whole reason we're doing it is for this period here, that zero g experience.
So when the load factor goes to zero, or near zero, that's when you get weightlessness.
And it turns out for this particular flight profile, we would experience 184 seconds, about three minutes of weightlessness.
This is pretty similar to the flight profile that Virgin Galactic is pursuing, by the way.
So you know, looking at this result, you would say, well, I don't know about the mass here, but from, you know, reaching 100 kilometers keeping accelerations below six g, this looks like a feasible profile.
Of course, the other thing that we care about is the view, since this is for space tourism.
We want to make sure that, you know, that the experience is great, not just from an experiencing weightlessness, which you can do in a closed can but you actually see something.
So this was essentially coupled with a virtual reality visualization.
So you can see, you know, the landing here.
You can see these three views here.
And you can see the vehicle itself.
So the pilots are up front, and then here is our six passengers.
Every one of the passengers gets their own window.
So this is the view that you would get from the front window.
This is the view from the middle window.
And then this is the view from the back window, from that last window here, which is closer to the back, to the engines.
So what would you, looking at these pictures-- you know, this comes directly out of the simulation and the CDF-- what would you say looking at this?
Anything you notice?
EPFL anything you guys notice?
Look at the view from the front, the middle, and the back window
From the front, looks better.
I guess it will be more expensive, again, for the customer
Yeah, so that's exactly what I was asking.
So it looks like either you need to redesign the vehicle so that the view is the same, more or less, from every window, or you need to start charging differential prices.
Because clearly, from the back window, half your view is obstructed by the wing structure, right?
Would you agree with that?
Would you charge different ticket prices for this ride, depending on which seat you get?
I guess that's offer and demand?
Supply and demand, yes.
That's right.
Go ahead
It could also merit a change in flight plan.
If your structure blocks the downward view, you might want to consider flipping over if like the Earth is what you're trying to see from above.
So it may have broader implications for the mission
OK, so the point here, is there a way to operationally improve the experience, keeping the vehicle design the same?
Absolutely, but what that means is, you would have to put the operation-- so the orientation of the vehicle, you'd have to put that into your design vector if you want to change it.
OK, so I don't know.
I think this is pretty cool.
So you can actually do a lot in this kind of CDF type setting.
And by the way, of the things that spun out of this project is this company called S3.
Their headquarters in Payerne in Switzerland.
This is actually-- I did a lot of my military service at that airport.
Their goal is not space tourism, you see there's no windows on this vehicle, but deployment of small satellites, custom deployment for small satellites with a reusable vehicle.
So you know, I think it's a very interesting concept, but the question is, is it feasible, right?
Is it feasible-- the aerodynamics, the economics, structurally, engine reuse?
There's a lot of-- this is a great picture, but you know, the devil is in the details.
So really working out the details of such a design, that's what a CDF is for, and you can really go deep.
OK so any questions about this?
I'd like to do a little partner exercise here.
Any questions?
So let's do a partner exercise, and the question is, what are your experiences with concurrent design facilities so far?
Whether you've done an internship at ESA or JPL, or some other place that has a facility like this, or even if you haven't been there, kind of speculate and discuss what it would be like.
Maybe, you know, what would be your favorite chair?
Would you pick propulsion or costing or flight trajectory?
You know, just sort of speculate and discuss with your partner, so for which project or application did you use it?
What went well?
What did not?
What could be improved?
And again, if you don't have much experiences, just think ahead what it would be like.
What would you like to get out of it if you have no experience in it?
So there is at ESA, a concurrent design facility, and I was talking with Katya that maybe also United Nations can be considered, like, the facilities there can be considered a concurrent design facility.
I don't know if it worked, this approach
Do you mean in Geneva, United Nations in Geneva or London or New York, or do you mean in general?
Exactly
But do they actually have a physical facility with models and screens, or are you saying this more metaphorically?
Metaphorically more
OK, but you know I-- go ahead.
Go ahead
Yeah, we're referring to the layout, to the site, we're looking at the ESA layout, where you have one big room, where you have all the people coming together, all the different models coming together.
So that's like all the different experts.
Then you have subrooms for different project rooms, and then you have, like, a big conference room.
That's kind of the structure of the layout of the facilities
But you know, maybe a place like the UN could really benefit when they deal with a refugee crisis or with, you know, climate change negotiations.
I like this idea.
Great suggestion.
What about here at MIT?
Any experiences, thoughts?
Yeah, please go ahead
Yeah, AFRL, we have a rapid innovation officer.
He's like a general equivalent as a civilian, a LOC BOS, and he basically is tasked with breaking all the rules in AFRL.
Whereas most projects take years to complete, he fields questions from the field, from the theater, from war fighters, and they have an urgent need.
And he gets them a solution in under 12 months.
So he pulls in all these experts from industry, academia, wherever.
He'll lock them in a room, and they don't leave until they come up with a solution.
And he has the money and the influence, the backing by the AFRL Commander to get it done
And what does that-- I mean, if you can share, roughly what does that facility look like roughly?
Does it have, like, models and computers and screens?
Just white walls, like, whiteboard walls, and they will just be covered with everybody's ideas by the time they leave there
Oh, I see.
So it's not as much computationally supported.
It's more of a human process
That's right
Cool.
OK. Great.
Interesting.
Any other experiences?
Anybody been at JPL or seen a commercial facility in a commercial company that does this kind of work?
Please, go ahead, Justice
Well it's less of a commercial-- I was interning at the Aerospace Corporation this past summer, and I was working in their concurrent design center.
And it uses a similar layout as JPL's Team X's design, where it's basically using models.
That they have, like, a group of engineers who sit at desks, and there's multiple engineers per subsystem, and they just discuss.
They're given a mission objective, and they discuss how they can best design the system based on that.
And they model it and optimize it in the subsystem, and then they feed it back to a systems level design
Did you participate in that?
Yes
So what did you think of it?
A lot-- it's a lot to take in at first just trying to understand how each of the models work.
You're not necessarily supposed to work on each model, but as an intern, they wanted me to kind of see what each of the models would do and not necessarily exactly how the optimization techniques work because I didn't necessarily understand it, but just more of have an idea of how it actually works as a whole
OK, good.
Anybody else at EPFL want to share any experiences?
Apparently not, but just a problem with the sound.
We didn't hear anything on what the guy before just told us
OK, Justice, did you push the mic button?
OK, yeah, so basically, he worked-- let me try to summarize.
He worked at the Aerospace Corporation.
He interned at the Airspace Corporation.
Was it in Virginia or California?
California, in California.
And Aerospace Corporation is like a think tank, and it was a very similar set up with different subsystem experts, different models.
And Justice was saying one of the big challenges is so much information, taking it all in and understanding all the models and coupling, and so forth.
So and I agree.
It takes a while.
If you haven't been working in this environment, it takes a while to really get used to it.
So the last thing I'll say here is that-- well, two things.
One is-- and they're linked-- one is that these CDFs are not cheap.
If you really want a high-functioning CDF, you have to have people that are dedicated to this.
You know, the software licenses expire.
The technology gets old.
You need to not just use it, but you need to constantly refresh a CDF, keep it up to date, and it just doesn't happen automatically.
You've got to have budget for that.
You have to have people who are dedicated to updating the models, updating the facility, and so forth.
So it's definitely something that needs to be invested in on a regular basis.
So what happened to us here at MIT in air astro, actually, when I was sitting where you are sitting as a grad student, we created a CDF in the mid '90s.
And you know, it was just like the one at JPL.
We had desktops, you know, with labels-- thermal, propulsion.
And we used it quite actively for maybe five, six, seven years, but we didn't put the budget into it every year.
We didn't have a dedicated person who was really.
looking after it.
So what happened to our CDF is that entropy took over, and it sort of got more and more dated.
And when you go and look in this room now, it kind of looks like a regular conference room.
So all the desktops that we had are gone.
I guess they'd be pretty old by now.
And there has been a pretty strong debate among the faculty whether actually we need this.
Should we recreate one?
And the argument goes as follows, the proponents say that a CDF is really something special.
When you enter in the CDF, you know this is a special place.
This is a facility that's designed specifically for doing rapid studies.
All the information you need is there.
The walls are plastered with reference information.
You know, really, that's what it's for.
The counter argument is that this is no longer the mid-'90s.
Now, everybody has laptops.
Everybody does mobile computing.
All the models and data should be on the cloud, not sitting anymore in a dedicated server facility, but it's sitting on the cloud.
And therefore, you know, physically, a facility that's dedicated is no longer needed.
What you do is you get together in a more kind of flexible ad hoc way.
Everybody brings their laptops.
You have the big white walls, like you said.
And you can just create a CDF on the fly, right?
Everybody brings their computing resources.
Stuff is on the cloud.
And then you get a lot done, and then you disband again.
You clean the white boards and you're done.
Therefore, no longer needed a dedicated CDF.
Those are the two countervailing opinions right now.
My view is, I think a CDF still has value today.
I think it is valuable as a dedicated facility, but it needs to be updated.
So yes, the models should be on the cloud.
Yes, you can bring your laptops, but it is still valuable to have dedicated computing resources, again.
So I now think a hybrid model is actually the right way to go.
So we'll see where it goes, but it's a pretty active debate.
There have also been, you know, theses written on CDFs, comparing different CDFs and so forth.
So take a look at the Anton Ivanov paper in the reading and just think about this.
I think this is a big deal, and it's going to be even more important than the future, whether you're doing space missions, aeronautics, whether you're designing commercial products, you know, or in the military, these facilities that allow you to make these design decisions in a very interactive, very quick way are gonna be more and more important in the future, and this is a big part of system engineering.
Any final thoughts on this before we move on?
OK, so here some lessons learned from the EPFL CDF.
So this is particularly in the university environment, so you know, giving access to a wide body of students.
You need to adapt this to the university's schedule, the learning curves, the analogy of a smoke-filled room or war room, optimal team size.
You know a CDF with 50 people on it is not as effective as one with, you know, seven plus or minus two.
And then distributed CDFs-- they don't typically work as well.
So I almost wore my-- I have a shirt that's almost the same.
Anybody know who this is or where this is from?
This is Sheldon Cooper in the Big Bang Theory, his virtual reality presence.
So the point here is that virtual reality is OK.
So you can imagine a CDF where half the people are in the room and half the people are dialing in remotely, but it's never quite as good as everybody is there physically.
So human interaction is critical.
OK, the last thing I want to talk about today is the CDR.
This is a big, big milestone.
This is a very big deal.
The critical design review, the main purpose of it is to approve the final design in all its glory, in all its details.
So generally, the way we talk about it is, if you pass the CDR, you've got the green light to quote, end quote, cut metal.
Of course, you know, our systems have software and hardware.
So the cut metal is essentially a euphemism for you can go ahead and manufacture the system now.
CDRs are typically the biggest review.
The largest number of people you will have at the CDR.
You know, if you had 10 people at the SRR and 20 people at the PDR, you're gonna have 50 or 100 people at the CDR.
Because any question that comes up on any of the parts-- say, is this really right?
Well, what could be the impact of this detail on the rest of the systems?
Somebody should be there who designed that part or who can answer the question at least.
And CDRs are typically longer than one day.
In fact, for a big project, a CDR can be a whole week of reviews.
So people get really tired.
Big CDRs are very, very detailed.
You really have to drink lots of coffee and stay awake.
So here's the description of what a CDR is.
One of the guidelines is that approximately 90% of the engineering drawings are approved and released.
So it's OK to have a CDR if you have not 100% of everything done, but you need to have at least 90% complete.
You can't do a CDR, with, you know, half the drawings done.
And it happens at the end of phase C, the final design phase C. For very large projects, you can have sub CDRs.
You'll have a system level CDR, and then you can have sub CDRs for every major subsystem in your project.
So an example, I put a link here.
This is from late 2014, the CDR for the James Webb Space Telescope that I mentioned earlier.
They passed the last major element level critical design review, which was a big deal.
So launch is planned for 2018.
But for big projects, what I'm trying to say is, there may be more than one CDR.
There may be the master CDR for the whole thing and then sub CDRs for different subsystems.
Now, if you don't-- well, let me get to this on the next chart.
Yeah, go ahead
Are the, like, CDR slides for NASA projects, are those available anywhere as a public entity and can you get access to any of that stuff?
It depends.
Not always, because especially if commercial companies, like, you know, Lockheed-Martin or somebody is doing, like, the spacecraft bus, they would consider those details proprietary.
So in general, I would say no.
But for CanSat, actually, you've probably already found them.
For CanSat, you can actually find a lot of this.
So prior PDRs and CDRs, you can see that.
I encourage you not to look too much.
You know, you kind of want to come up with your own thing.
But you know, for projects that don't have a commercial interest or proprietary technology, it's often available.
So to show you just-- this is a table from the Handbook, page 178.
This is the entrance criteria and the exit criteria for CDR.
So entrance criteria means what do you have to have in order to be allowed to go forward with a CDR?
And it's mainly three things-- successful completion of the PDR, and responses made to all the PDR RFAs and RIDs.
RFAs and RIDs are essentially deficiencies, open items that came up in the PDR.
So you have to close them out.
A CDR agenda, and then number three is all the technical work packages.
And so you see, it's a huge list here-- baseline documents, specification, fabrication, assembly and integration, technical data package, acceptance criteria, verification and validation plan, launch site selection, and operations plan-- super, super, super detailed.
This is a lot of information.
And at the CDR, you're gonna go through all this.
So you can imagine what that means.
And then success criteria, on the right, is, you know, in order to successfully pass the CDR, these are all the things that you have to basically have.
You have to have approval for all these things.
Now in reality, just like a PDR, at a CDR, things will come up that you missed or that people have concerns about.
And so you will make a list of those, and then you have to address these action items coming out of a CDR.
If those action items are serious-- if they're really serious, you could fail the CDR and say, you know, this is not gonna work.
We should not proceed.
And they send you back.
Or they cancel the project.
That's happened also.
More likely, is that they'll ask you to do a so-called delta CDR.
So they'll have you, you know, it's like retaking an exam or something.
They'll have you come back, and you'll have to do a CDR again but only on the delta, the portion that raised concerns at the main CDR.
You don't have to do everything again.
That's called a delta CDR.
OK, any questions about CDR?
This is really the-- this is the big one.
This is the big milestone.
And obviously, you know, the concept or the architecture you chose at the PDR turns out not to be a good one, it's very difficult to make changes to that here, right?
You're kind of locked in to whatever design or architecture you chose at PDR.
That's why PDR is also so important, even though you have less detail.
Yeah, Sam, go ahead
Do you have to have contracts in place already for the, like, manufacturing and operations?
Oh, absolutely.
I mean, basically green light CDR, the next day, you start manufacturing.
So you know, every portion, all the components, all the manufacturing, all of that has been bid, selected, contracted.
Everything's in place contractually, absolutely.
OK, any questions at EPFL about CDR?
Who has actually gone through a CDR on your end?
Has anybody done a CDR for any of the projects?
OK, go ahead.
What was it like?
I did also a CDR because I had the PDR last time at the [INAUDIBLE] program.
And one comment I want to make.
So the teams were allowed to fail the PDR and still fly.
If they failed the CDR, then they didn't fly at all.
So it was very strict and demanding
Yeah, so would you say that the CDR is more strict than the PDR?
The CDR is more strict
Yes, I agree.
I agree.
Definitely.
OK, anybody else?
Are any of you working in systems other than aerospace, like, I don't know, medical devices or you know, scientific instruments?
So the reason I bring this up is because you know, medical devices have to be approved by, here in the US, it's the FDA, the Food and Drug Administration, and then in Europe, there is equivalent, you know, European level thing.
It's very, very detailed.
It's really not that different.
So presenting your medical device design to a panel of experts-- doctors, you know, people in medical safety, regulators-- it is like a CDR.
You basically are going through a CDR when you're trying to get your medical device approved for sale and use.
So it's not often called a CDR, but it effectively is a CDR.
So this is, especially in industries where there's a lot of money at stake and people, you know, safety, people's health is at stake, this is common practice.
OK, so let me summarize here.
So this is the detailed design phase we discussed.
It's very important.
Basically, you take your PDR level design and you go define all the details in full maturity.
You create design documents and models.
And you know, as we talked about last time, we're sort of in this transition phase of producing documents versus producing models.
Typically, we have both now.
Examples would be detailed bill of materials, computer-aided design files-- we'll talk about CAD next week, computer-aided design-- software and control system definition, user interfaces, et cetera, et cetera, et cetera.
Multidisciplinary design optimization is a very powerful concept to optimize at the system and the subsystem level and trade off between disciplines and objectives.
The CDFs are the concurrent design facilities, and they've now become standard practice both in aerospace and product design companies.
If you have not had the chance to experience and work in a CDF, I really encourage you to do this.
I think it's a life-changing experience.
It really-- it's not easy, because like, you know, Justice said, there's a lot of information to absorb.
But once you've done this and experienced it, you realize if you don't have a CDF, if you don't design products this way, you really should.
And then CDR is the big milestone, the last gate before, quote, end quote, cutting metal.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare@OCW.MIT.edu
Seven miscellaneous topics, and you may be wondering, well, what are those?
So I'd like to start with a general status update, and then we'll go over the master solution for the online quiz that you took this week.
And then, I think, in about 10 minutes, we're going to have an interactive discussion with the INCOSE board of directors.
And so, we'll probably not get through all the master solution discussion by the time they dial in, and that's OK. You know, we'll just have a 15, 20 minute chat with them.
The reason we're doing this is because they're at MIT right now for a three day meeting, so INCOSE's the International National Council of System Engineering.
And so we thought it would be a nice thing since, you know, this is a systems engineering class, to tie in with them and hear a little bit of their perspective on what's happening in industry and in, sort of, the bigger picture.
And hopefully, some of you have got a couple of questions prepared, so we can have sort of a dialogue with them.
That's the general idea.
And then we have a presentation prepared.
The Octanis I project will be presented from EPFL, to us, and I think you'll find it very exciting.
And so, as we listen to that presentation, put on your system engineering thinking hat, and then hopefully we'll have time for discussion as well.
Let me just remind you briefly where we stand in terms of the class-- in the V-Model, in particular.
We made great progress, and we've essentially covered the whole left side, the bottom.
And what we'll do starting next week, is really climb up on the right side.
So next week's topic is System Integration and Interface Management.
Then we'll cover Verification Validation, Commissioning and Operations.
And then our final official lecture is Lifecycle Management.
We do have a 12th lecture on Prototyping and Manufacturing.
That window is optional because this is after the official end of classes at MIT.
So you're not expected to participate, but you're invited to.
So basically, the focus in the next month or so is covering the right side of V. So let me start talking about the midterm exam.
And we'll sort of go question by question, and please stop me if you have any comments or questions.
Let's make this interactive.
So question one, if a complex electromechanical system has 1,850 individual part numbers-- this is at the lowest level-- how many levels do you expect the drawing tree to have?
Everybody provided the same answer, four, right?
This is basically a straight application of the magic number seven rule.
And in practice, not everybody will go exactly for four levels, but that's the basic idea.
If you decompose the system with this human cognitive bandwidth in mind, for a system of that level of complexity, you should expect, roughly, a four layer decomposition.
So I'm very pleased that everybody got this answer.
But with the caveat that there may be deviations from this rule, and you can go for a flatter tree with fewer levels.
But you have to be aware of the implications that that may have.
OK, so that's great.
The next question was-- OK, so you're attending a cocktail party-- and cocktail parties, I don't think that's-- that maybe that's very 20th century.
I don't think we have cocktail parties anymore, so maybe more like a Google Hangout or the 21st century equivalent.
But the basic idea here is, how would you explain what system engineering is to somebody who is not a technical person, not an engineer?
And I think most of you-- I looked through all the answers, and most of you basically provided a definition of system engineering that we would use in this class, right.
So it was like, OK, well, not quite what I was looking for but not incorrect.
So I pulled out two examples that I kind of liked because they brought in analogies from other fields as to what system engineering is.
So the first one here is, OK, so system engineering represents a method but also a mindset that enables the design and so forth of successful systems, but systems engineering is like cooking a delicious cake.
OK, so it's like cooking a delicious recipe.
You've got your ingredients which is your part numbers, or your parts, right?
You have a process that you're following.
And at the end of it you should have something special, something delicious, something wholesome, something that is greater than the sum of the parts.
I don't know if any of you watch the Food Channel?
Anybody watching the Food Network here?
Yeah?
OK. We actually like to watch it.
It's kind of fun.
And what's fun is when you give the same ingredients to, like, half a dozen different people-- and same ingredients, same amount of time, and the result that comes out of it is vastly different.
Now, why is that?
Of course, you can say skills and so forth, but it is fundamentally, because those ingredients-- that details matter.
The way in which the ingredients get combined.
And you know, did you use the right heat?
Did you did you take it off the heat?
30 seconds too soon or too late, those little details make a huge difference to the final product, OK. And that's what I think what's cool about the Food Network and cooking in general.
Cuisine, it's an art.
It's not just follow the recipe.
And so, I think that's a great analogy.
I like that.
And then the other one is-- so you're talking to Cher, the singer, and explain to her that system engineering is like a concert.
So in a concert you have all your specialists, right?
You have your choir maybe.
You have your strings section.
You have your trumpets.
And I think it's kind of similar.
You could have great individual musicians, but if the conductor is not bringing everybody into harmony then you get a cacophony.
It's not a great result.
So a great orchestra is not just a bunch of great individual musicians, but it's the idea that you're orchestrating-- there's something special that emerges if you synchronize everybody in a way-- and you motivate them, and you synchronize them, and something really special happens.
I think that's another great sort of example of what we mean by system engineering in the technical world.
This idea of a conductor bringing everybody in harmony, and the end result is something really special.
That's a little bit what I was looking for in this answer, and I like both of these examples.
The third question was about the V-Model.
And I think, based on what I looked through the answers, I think most people really understood that.
And just to summarize, again, in a very, very simple way what the V-Model is all about-- so here's the V. If you wanted to really simplify it, we start at the upper left with a vision.
You know, what is it that we're trying to accomplish?
And that's where you need to understand, who's the stakeholder, who are the customers, the thing that you're about to create doesn't exist yet.
In some projects you actually start with an initial system, and you need to improve it.
So you're starting with generation N of a product, and the goal is to create generation N plus 1, which is better in several dimensions.
So that's possible, but let's just say we have a clean sheet situation.
And you have a vision that you're developing.
And then you [INAUDIBLE]
Hi, Ollie, is that you?
Aha,
All right, you are now connected on speaker with the INCOSE Board of Directors
Fantastic, can you hear me?
No
Yes, we can
Loud and clear
All right, this is great.
So I was just going over the midterm exam that we just had, and so I will stop doing that because we're very glad that we can spend time with you.
So let's make this bigger on our end.
And do you have a camera on your end, Tina?
Let's say no because of the demographics here, so we'll have to go on voice, Ollie
OK, Tina, can we see you?
Can we see one of you at least?
[INAUDIBLE] [INAUDIBLE] can stand there at look presidential
We're working on it
How about I figure it out while you give the intro?
There you go.
Thank you, Tina
Multi-task
It would be helpful to see at least who is speaking.
[SIDE CONVERSATION]
I know, we've got a problem here
All right, you stand over there.
I'll point this to you
Well, shall we go ahead and kick off here, Ollie?
I know your time is valuable
OK. Ryan can we make the INCOSE Board of Directors the main screen here?
Just give me one second on our end
Ollie, can I suggest we just press ahead despite the desire to see David's shiny face?
OK. All right, so David, it's great to talk to you.
So let me just give you a little bit of background here.
We have about 20 students here at MIT.
Currently, we're taking the Fundamentals of Systems Engineering class.
We're roughly midterm, a little bit past the midterm.
And then we have about 20 students, maybe 15 students, and staff at EPFL in Switzerland, and we're teaching this class in the form of the SPOC, Small Private Online Course.
And so, welcome to MIT, to the INCOSE Board of Directors.
I know you've already had very productive times.
And we're really excited to hear from you as to not just what is INCOSE, but where do you see Systems Engineering as a field of study and practice stand.
You know, what are the exciting things happening?
What are the challenges?
And whatever you would choose to share with us.
So, David
Very good.
Thanks, Ollie.
And thank you for making the time available.
It is always great to talk with the next generation.
I'll give you just a few comments up front and then really open it up.
For those who aren't familiar with INCOSE, it's the International Council on Systems Engineering.
It's about 25 years old, about 10,000 members worldwide.
And the value of INCOSE, fundamentally, is to serve as a connector across the systems community, across organizational boundaries, across geographic boundaries, so that we can connect the practice in the US to Europe to Asia and beyond-- they are applied differently-- and across domain boundaries, from military and aerospace, where we have our foundation, to automotive to medical devices and beyond.
If you look at it as systems engineers, that's perhaps the most exciting thing of what's going on.
If you look at how complexity continues to emerge and grow, the problems that we are being asked to tackle, the technologies at our disposal, the opportunities for wonderful outcomes, the dangers and risks of unintended consequences-- the challenges that the world faces are, absolutely, System's challenges.
And they are far beyond the traditional aerospace and defense domain.
They are everywhere.
I was with GM earlier this week, looking at automated driving and auto assist.
I was with medical devices in iRobot yesterday.
And wherever you look, whatever domain you work in, we see complexity, we see systems challenges, and we see tremendous opportunities for us, as professionals, but more than that, for systems professionals to have a profound positive impact on the world.
So it's going to be a very challenging journey.
We are still working to solidify our foundation to advance our theories, to advance our practice.
But it is a very fruitful time, I think, as well.
So those were the comments that I will offer at the start, and then open it up to the class to ask the questions of this board of directors
OK, David, that was great.
And so, who wants to be first?
Don't be shy-- either here at MIT or at EPFL.
Sam, go for it
I was wondering when you're creating, I guess, INCOSE standards for systems engineering processes, what considerations do you take into account in order to make them applicable for such a wide range of disciplines or domains?
Introduce yourself for whoever speaks
this is Paul Schreinmarkers.
I'm the technical director.
Thank you for asking.
Well, actually, INCOSE is not a standard setting body, but we are supporting a lot of standards like ISO 15288.
And what we at least try to achieve is that the standards are being applied in multiple domains as much as possible.
Does that answer your question?
Yeah, thanks
OK, and just so you know we do have 15288 as one of the-- in the class here, we covered, sort of, three standards.
One is the System Engineering Handbook, the [?
NASA, ?]
of course, the INCOSE Handbook, and then 15288.
So I think the students should be familiar with [?
15288, ?]
so that was good.
OK, can we have a question from EPFL?
Are you still with us, guys?
- Hello, do you hear me?
Yes, good.
- OK, yes.
[INAUDIBLE] just have a question about the complexity of systems with new systems being more and more complex over time, would system engineers, or engineering itself, have to reinvent itself in the future?
And what would be the direction?
Thank you
Hi there, it's Alan Harding here.
I'm president-elect at INCOSE, but more importantly, I co-chair our system and systems working group.
I think the answer is no to reinvention because I'm confident all the principles stand.
But what we've been doing as a working group, is identifying where engineering a system and systems are different.
We constructed a paper, which has been published, on the pain point to the system of systems, looking at the managerial and the technical differences, and therefore, helping people, when planning their work, see in which cases they can pick up their traditional ways of looking-- and where, if you, like, experienced practitioners, see problems or challenges in other areas.
So that working group is helping develop-- so understand and develop the state practice for complex system of systems, and that's proved quite influential.
And in fact, if you all using the latest edition of the SE handbook, in the section on system of systems, it does highlight those pain points.
So they are there helping shape what people do.
Hopefully, that's of use?
Thank you, Alan.
Very good.
I think we've only mentioned system of systems in the class so far.
We haven't really deep dived into that topic, but it's a great point.
OK, and back to
Hi, my question is about the systems engineering vision you have on the web site about 2025.
What has been done about the protocols or languages that could be developed to facilitate the communication between the systems engineering professionals and different stakeholders?
That's a really good question-- you know, this idea of having the same language.
By the way, I'm Mike Celentano, deputy technical director of operations.
The idea of having a common language is really important.
One of the things that I've struggled with in health care-- that's my industry-- is trying to convince people to speak the same language, and I've since given that up.
What I'm teaching people now is to understand the other people's language, so that they can communicate better.
And so when you go from domain to domain, it's important that you spend some time learning the language, a lot like it's important to learn how to say hello and goodbye in the language of the country that you're visiting.
It's important to understand the domain that you're exploring, so that you can make the interpretation and help those people with systems engineering.
Does that answer your question?
Yes, thank you
Ollie, if you don't mind, I'm going to follow up, and then Art will as well.
What that question calls out, and what I think Mike's answer calls out, is there are two very important dimensions, in my opinion, of systems engineering.
We are the glue to the project team, but we are the glue in two different dimensions.
There is the engineering aspect of what we do, and there is very real engineering analysis that needs to occur at the systems level.
We unify the subject matter experts, and we bring those understandings together whether it's in medical, whether it's in aerospace, regardless.
But the other aspect that we cannot avoid as systems engineers, is the communication role.
Because we are working with people who do not speak the same common language, whether that's speaking between aerospace and medical, or whether that's on an aerospace project speaking between computer science, reliability, mechanical, electrical.
As systems engineers, we must embrace the communication aspect and the translator aspect that Mike called out, because each of us speak a different language for a particular reason.
Art?
And to help in that regard, one of the products that INCOSE has helped create is a systems engineering body of knowledge, which is available on a website, which Ollie is quite familiar with-- which has several hundred articles about systems engineering.
But one of the projects that we are going to be-- and it's its domain neutral, so it's not tied to a specific business sector.
One of the projects that we are expecting to begin in 2016 is to create a variant of that, in some fashion, for the health care community-- tying it to the unique way they talk about things-- their approaches, but tie it back to the common underlying principles.
And if we are successful in that, as we expect, I can imagine us spinning up ones for transportation, for telecommunications, et cetera.
So that that can become a glue, in essence, for people across many different business sectors
Very good.
So I will make sure that we provide the link to the body of knowledge, which I know was a very big effort to put together.
We'll provide that information for the students as well
Great, that's excellent.
Unfortunately, for time, I'm going to have to ask Art to give us a wrap up.
But let me just say one thing because I'm not sure how well you can see the video over here.
But you guys are talking to a group of about 20 people from around the world.
We have people here from France, Sweden, Australia.
And the people that you just talked to are also senior engineering leaders that Airbus and BAE, and so they kind of arrange a broad spectrum here-- and also rose from different industries.
So I'll hand it over to Art to give you a good summary, but I hope that you continue to keep in touch with this group
Well, this is where I'm going to make my small pitch for membership.
So as students, you have an opportunity for only 38 dollars US annually to be members of INCOSE, which gives you access to all of our products, which includes videos that are recorded every month, handbooks, all sorts of documents.
And I'd really like to encourage you to consider that as an option.
It's great.
It's something like a quarter of what you'd have to pay as a full professional.
And so I'm going to let it go with that
No, Art, I think that was great.
And I'll just say from my perspective, I first got involved in INCOSE a little more than a decade ago, and I think both the industry-academic mix, the conferences-- you know, this is real.
This is not a community that's living in a bubble, and so I would I would fully endorse, Art.
All right, well, I think we're at the end of this interview.
You know, this is cool.
I mean, that technology has gotten to the point where, certainly, the audio part of this was very clear for all of us, and that's terrific.
So we want to thank you for your time.
Have a great continuation of your board meeting, and we're going to keep learning over here
Very good.
Thank you so much, Ollie
Thank you
Take care.
All right, well, I hope you found that useful.
And I'll provide you a link to the body of knowledge.
I think that's one thing that we haven't sort of pointed out.
So let's go back to-- EPFL, are you still with us?
[INAUDIBLE]
OK, good.
Thank you.
So let me just sort of finish on the midterm.
Yes, Veronica?
[INAUDIBLE] can I ask, a question I would have asked to the board, to you?
Thank you.
When I was looking over their web site, I see a lot about kind of communicating for the sake of engineers and everything else.
But I think that systems engineering, in particular, has an ability to shape the way that we kind of regulate policy developments.
I feel like a lot of times scientific advancement is working against international regulatory standards, where I feel like an entity like INCOSE has an ability to kind of shape international collaboration in a way that can help scientists achieve more together.
And so I was kind of wondering what the role they might have seen-- or what you might see as kind of an international collaboration, like that, taking, in terms of shaping the ways in which we engineer together-- not just the ways in which we kind of create in systems engineering naturally, but kind of more collaborative
I think it's a great question.
And I think the system engineering is sort of evolving and broadening.
The classic view of system engineering is you know, that's what a company or a big project that's producing a physical deliverable, a new airplane, a new computer, health care device, a new software system.
That's the out-- you know, and then that's a capability that you put out into the market and to the field.
And regulations and policies are given to you-- you know, their constraints, their requirements.
I think what you're saying is, well, the policies, themselves, are something to be designed--
I think so
--and negotiated.
And that's a broadening of the system boundary and the mandate for system engineering.
And so here, at MIT, over the last 15 years or so, the word we've used for that is, engineering systems.
So we kind of flipped the two words, and say that system engineering, in a more traditional sense, is scope, scope, smaller.
And then when you internalize the externalities like economics, policy, and you consider that part of the design space as well, then you're in the world of engineering systems or complex sociotechnical systems.
And so, absolutely, there's a-- and you heard the answer that Alan Harding, who's the President-elect of INCOSE.
He will take over from Dave Long next year.
When he started talking about system of systems-- well, you know, what our system of systems?
For example, air traffic management.
Let's take an example, air traffic management, air transportation.
In North America, there's this project called next-gen-- right, next generation-- that's a very complex system of systems.
And in Europe, the equivalent is a project called SESAR.
It's essentially the Unified European airspace.
Well, you've got-- it's not just about flight controllers, and radar stations, and GPS-enabled navigation.
A lot of it has to do with policy.
And how do you do the hand offs?
Who is responsible to report any anomalies?
How do you do backup operations?
And so forth.
So a good example when that came to the fore is-- I don't know if you remember the Icelandic volcano a few years ago?
The volcano erupted, and then there was this big ash cloud.
And then the question was, well, is it safe to fly through this ash cloud?
And who actually has the authority to decide which part of the airspace is closed and is open?
And then you've got the airlines saying, you know, open up the airspace because we're losing a lot of business every day.
And the regulators are saying, wait, we need to be cautious here.
So I think what you're saying is that's a system engineering problem that we could sort of address.
And so things are moving that way, but I think our methodologies for doing that work in a rigorous way are not quite there yet.
It's still evolving, but great question
Is that a type of advocacy that a group like INCOSE could move toward?
I can see from their mission statement and everything, that's something they are doing now.
But is that a place that you see that they could fill?
It could be.
And I think I think they're moving that way
Thank you
Yeah.
OK, any other points anybody wants to add from the discussion?
Yeah, one question from our side--
Go ahead
--if I can.
So I think that's a very good point, Veronica, in terms of how do you transfer this to policy or business management?
Some time ago, I was trying to see if there are actually jobs or careers that are called systems engineer, but besides IT, there's really not a profession.
For example, if you're doing this in the policy domain, could you just be the systems engineer of, say for example, the Swiss energy-- you know, what I'm working on currently-- kind of the Swiss energy transition-- you know, the systems engineer, and that would be your role.
Are their careers and career names that are explicitly, like, designing the systems and engineering them?
That's a question, right?
Katya, I think the mic is off.
Go ahead
Hear me now?
Yes
Are there jobs that do this?
Are there systems engineering jobs, or is that project management?
That's a good question.
So the job profiles of system engineers have been broadening.
There is also, system architects.
When we talk about concept generation here, concept selection, that's, in a sense, a specialty within system engineering known as system architecture.
So you'll have more and more people hand you a business card that says, I'm the system architect.
And what that means is they're in charge of the high level decisions, the upper left side of the V that's shown here.
So system architect, system engineers.
And there's also a very-- this didn't come up in the discussion with INCOSE, but there's a very active discussion going on right now between INCOSE, that has about 10,000 members and is 25 years old, and an organization called PMI, Project Management Institute, which is older and has about 400,000 members globally.
And that's the people running big projects.
So those job descriptions are still coming from different angles, but we do see this bigger discussion emerging.
And I think the organizations that are enlightened, that really understand that these big system decisions are critical-- and that you can't just make those arbitrarily.
You need people who actually think about the whole system, who model and simulate the whole system, who look at different scenarios.
You need dedicated people for that.
Who not just have the education and the experience to do it, but have the job title to do it as well, and the authority.
And so, I think we're seeing a transition to those new kinds of jobs, especially in the enlightened organizations.
OK.
Yes?
One more, I promise
One more and then we'll move on.
Yep?
So when you say that someone needs to have a designated title of a systems engineer to really do it well, is there an advantage or disadvantage to having everyone be a systems engineer with a specialty, so that you're training an entire organization to think about the big picture?
Because I can see pros and cons, but I'm wondering, is there a hard answer either way
My position on this is-- you remember, we talked a little bit about the value of system engineering, like how much should you invest in system engineering?
And we'll come back to this at the end of term.
But generally, you can see that there's empirical data that says you should spend roughly 10% to 20% of your budget-- maybe 15% is sort of the sweet spot-- on systems engineering, system architecture type activities, which happens to be 1 in 7.
1 over 7, that's 15% roughly, right?
So if you're going to, on average, decompose your systems into seven chunks at each level of decomposition then you need about one out of seven people-- or one out of eight or one out of seven people-- their primary focus should be on how the chunks fit together, not designing the individual chunks.
And so there's a lot of empirical evidence that suggests that if you spend a lot less effort on system integration than about 15% of your total budget, you're going to have real problems.
You will miss requirements.
You'll have poorly managed interfaces.
You'll have surprises during testing.
If you spend a lot more than 15% on system engineering activities, the project will be very heavy, too much bureaucracy, too many processes.
It gets slower.
So we'll come back to that.
But everybody should have some understanding of what system engineering is even if you're a subject matter expert specialty person, if you do cost modeling, or if you do control systems, propulsion, in the medical field, biocompatible materials.
Whatever it is you do, everybody should know something about system engineering.
But then you do need, that's my opinion, the dedicated people who really focus on the integration, the interfaces, how the chunks fit together.
And if you don't have that for projects that are of a certain level of complexity, you're going to have big problems.
OK, Katya, was that was that reasonable?
All right.
So let's move on here with the exam because we have a very, I think, exciting, interesting presentation to get to.
So the V is essentially-- you have this vision.
You then decompose requirements, subsystems.
And then the bottom of the V is the details.
Every little detail needs to be defined.
And then the right side is we go and then integrate everything together, and we go to operations.
So I think most of you really explained that quite well.
Question four, what is the most important aspect of the V-model that is not shown explicitly in the V?
There are two answers here, and I think both are correct.
The iterative nature of design, we'll hear about this in the Octanis project.
So the V-model does not imply that you're automatically doing a waterfall stage gate process.
You're only doing trade studies once.
You're only testing once.
That's not what it means.
It just means that there's a logical sequence to the activities, but you could actually spiral through the V multiple times, so the iterative nature of design.
And then the possibility of schedule and cost overruns, this is the programmatic side, is not shown in the V-model, but it's implied.
So both of these are correct, so this is very good.
Question five, at which of the following milestones should the main concept, or architecture, of the system selected-- so here we have a little bit of a spread of answers.
The correct answer here should be PDR, Polygyny Design Review.
That's when the concept, the architecture, the high level decomposition of the system, the assignment of functions to those chunks at level one or level two, should be defined.
SRR, it's too early.
All you're trying to do at the SRR, System Requirements Review, is agree on what the requirements are, what the system should achieve, but not necessarily how that will be achieved.
And then at CDR, I would say it's too late.
The CDR is where you lay out all the details of how the system will be built, operated, the software.
So that the high level design, the concept, the architecture, should have been decided and agreed upon way before the CDR, and it's the PDR where that's happening.
So I think about 3/4 of you got it right here.
The next one was a little subtle question.
What are the following attributes should a set of requirements meet?
And I put check marks here between what were the-- so there were multiple correct answers.
The set of requirements means multiple, an ensemble of requirements, as opposed to just a single requirement in isolation.
So, feasible, verifiable, and unambiguous, those three answers are correct for individual requirements.
Each individual requirement should be feasible, verifiable, and unambiguous.
But when you look at a set of requirements, that's when these other three come into play, absence of redundancy.
So what this means is that you shouldn't replicate the same requirement, essentially, across multiple requirements in your requirements set.
And one of the reasons for that is if you change a requirement then you might miss it somewhere else, right.
This redundancy is-- keep it as simple, as straightforward as you can.
Completeness, that basically means that the requirements set that you develop should cover all the requirements.
You shouldn't have big holes and missing requirements.
And then the last point is absence of conflicts.
If you have requirements conflicts, essentially, what this means is there's no feasible solution.
So if you think about optimization, you have an over-constrained problem.
Basically, that's what that means.
OK?
Any questions about these two?
Yes, I have a question for question five.
I think the question is a little bit ambiguous because if the main concept is selected on the PDR-- on a timescale on the CDR, you have the main concept.
So you can say that on the CDR, it's also the main concept selected already
Ah, I see what you're saying.
Yeah, so maybe I should have said, what is the earliest milestone at which the concept-- that's really what I meant, is what's the milestone where the concept has to be agreed upon, and then of course, at the CDR-- unless you had to overturn your concept selection, which happens sometimes-- you should stick with the same.
But the intent here was to say, this is PDR.
OK, go ahead
For this question, number five, I think I got concepts of operations confused with the concept architecture.
So just to be clear about that, the concept of operations is what the system does, and architecture just a list of the subsystems and what are the mass, energy, and information flow?
Is that correct?
That's right.
The concept of operations, that's actually an SRR type.
The concept of operations--
That's what I put
--how will the system be used, right?
And then the concept, the architecture is, given this concept of operations, what are the functions that have to happen, and then which chunks in your system support or enable what functions?
That's essentially what it is.
Yep.
OK, anything else?
All right, question seven.
I think most of you got this right.
We're going to get to verification and validation in a lot more detail, but there is a distinction.
Some people use verification and validation interchangeably.
They think it's the same.
It's different words for the same thing.
That's not quite the case.
So here you've got your verification loop and your validation loop.
To put it very succinctly, in verification, you're answering the question, did you satisfy the requirements as they were written?
That's the inner loop.
You've got your requirements set, and then you do verification, or inspection, or there's different ways of checking that.
But you check, do we meet the requirements as they were written?
That's the inner loop.
And then validation is the outer loop where you say, go back to the stakeholders, and then you say the CONOPS, right-- can we actually execute the CONOPS-- and does the system meet the expectations of the stakeholders?
Does it deliver value to them?
And so it's quite possible-- and there are examples of systems that actually successfully pass verification.
You can check every requirements, say, yup, yup, yup, we can do that.
And then you demonstrate the product, or the system, to the stakeholders, and they say, oh no, that's not at all what we were looking for.
And then the question is, well, did you change your mind?
You know, customer, beneficiary-- did you change the mind from today to when we started this project, or was there some error in translation going from the stake-- or the stakeholders didn't change their mind.
You know, their use cases, their CONOPS, the value they want to get from the system hasn't changed since you started the project, but there was some error in translation between those high level stakeholder expectations and then the requirements.
No, maybe you missed a requirement.
So that's the key idea here between V and B, and we'll talk a lot more about this in the next few weeks.
All right, now comes the one you've all been waiting for, I know.
Question eight.
So here's the rocket equation.
Delta v, the change in velocity of a rocket that you can achieve is equal to gravity times the specific impulse times the log of your initial mass-- so this is your rocket on the launch pad-- and over M i, which is the final mass after the burn has happened.
And so, one thing that's important is to define those masses.
And we got a lot of emails.
Right, [INAUDIBLE]?
Would you say this was probably the most emails we've gotten all semester-- is about this question?
Yeah
So a bunch of questions were just about the definition of mass fraction.
What is the mass fraction?
And there are really two different interpretations of it.
And it turned out it didn't matter which one you used because the answer was the same.
So the one that I'm showing you here is that the initial mass here is 1 plus alpha times M f, your mass of fuel, plus your payload.
So in this case, alpha is the percentage of the mass that you're launching, the fuel and the payload-- that's made up of structure, which includes tankage, support structure, the engines themselves, et cetera.
The other definition you could use which is equally valid, you just have to define it, is that alpha is only applied to the fuel mass, right.
That you've got the fuel, which in this case is liquid oxygen, liquid hydrogen.
And then you have your tanks and your engine.
That's your mass fraction.
And then the payload sits on top, and there's no overhead for carrying the payload.
And either way, it didn't matter in this question because the answer was the same.
So here the initial mass is 1 plus alpha times the mass of the fuel, which is unknown, and the mass of the payload, which is given, 20 metric tons.
And then M1 is simply alpha times mass of the fuel and mass of the payload.
So you can slightly rearrange-- there are different ways of solving this problem.
A simple way is this.
You just basically bring the constants over to the left side.
So delta V over g i s p, and then basically, the right side has to be larger than that.
So this is essentially your 1 plus alpha divided by alpha because the M f plus M p term falls out, if you formulated that way.
So you can just plug in the numbers that were given in the problem, and you can see that the left hand side does not match the right hand side, right?
And the conclusion from this-- some of you tried to solve for the fuel mass.
You know, there's all kinds of ways.
And it's super frustrating, and I know a lot of you spent a lot of time trying to solve this problem.
And no matter what way, you get negative fuel mass.
You get all kinds of weird things trying to make this work.
The bottom line is infeasible requirement.
It's not possible with the required-- this is an example of an infeasible requirement.
And I did this on purpose to show you that just because somebody writes a requirement down, it doesn't mean it's feasible.
So I think many of you got this and were frustrated by it, but I want to show you a real project where this happened.
So you could then ask, well, what would we have to do to make it feasible?
So some of you said, well, maybe we can achieve less delta V?
Or we need even better propulsion, higher i s p. Or-- which is maybe a little bit more straightforward, what would be the mass fraction, the alpha, that you would have to achieve to make this work?
And the answer is somewhere around 8%, depending on which definition of mass fraction you used, it's 7% to 8%.
So it means that if you want to achieve single stage to orbit flight with no staging, which is the current way we do it, you have to get the mass fraction below 8%.
And that includes everything, the tanks, the support structure, the engines.
And that's really, really, really tough.
So I know you were frustrated by this problem many of you, but let me tell you this story.
Those folks were really frustrated.
So there was actually a program here in the US in the late '90s that tried to as a small scale technology demonstrator demonstrate the technologies required to get to orbit with a single stage vehicle.
And that program was called the X-33, run by Lockheed Martin-- so, single stage to orbit, reusable launch vehicle demonstrator.
The target mass, this is your M 0, total mass, 130 metric tons.
The target mass fraction was 0.1.
And fuel was just the same as in this problem, LOX hydrogen.
And the project basically was canceled in 2001 after about $1.3 billion had been spent on it, and the thing that eventually really led to the cancellation was the fuel tank.
There was a composite material fuel tank that was embedded inside the wing here, inside this body, has very complex shape.
And those of you that know about-- hydrogen atoms are very, very small.
It's very tough to contain hydrogen.
It wants to leak.
Its natural tendency is to leak out.
So this composite fuel tank basically couldn't be built and tested under pressure to be leak proof, and they really tried.
They spent a lot of money on it, a lot of effort, couldn't be done.
And so NASA concluded, you know-- and the goal was to achieve, essentially, this mass fraction of 10% or below.
And at that time, in the late '90s, early 2000s, this was just not feasible with the state of technology.
And so, think about how frustrated-- I know you spent an hour or so trying to solve this, and we're not happy about it.
But think about how they felt.
So the purpose of this-- I think most of this midterm exam was kind of a gift up until this question
I was wondering about the mass fraction.
I read the text over there, and it says it's suborbital.
So was the plan to reduce the mass fraction even further if this was a success?
Right, so this would not have achieved orbital velocity.
So it's not a complete one to one map because I gave you also a higher delta V, right?
I gave you 11 kilometers per second, which is escape velocity, and you need to achieve about 7.5.
But with drag and gravity losses, you really needed achieve about 9, 9.5.
So it's not a one to one map, but you sort of get the idea.
Any question about this at EPFL?
Did you hear this?
Was it was it fairly clear?
Yeah, I think everything is clear on our side

Thanks
All right, so if you want to read more about the X-33, the Wikipedia article is pretty good and there's a lot.
So just to say, you know, I think we're going to try this again.
I think there's going to be new efforts.
Composite materials have come a long way-- composite manufacturing, 15 years later, we might try this again pretty soon.
OK, question nine.
So this was a little bit of computation here.
So the intent of this question was for you to think about competing objectives.
Right, the Pareto frontiers, we talked about this in the concept selection.
So this is the idea of, you're designing a can or container for containing liquids.
The shape is cylindrical as shown in red.
And so the two things you care about is volume.
How much liquid can you carry inside the can?
That's simply pi r-squared h-- and then your surface area, which you could say that's the cost of the container, right.
That's the amount of material you need to use, is 2 pi r. Those at the top and bottom.
And then 2 r pi h, which is your circumference.
And if you then plot four different combinations of radius and height, what is the volume and surface area, which is shown in this plot here?
Surface area as the y-axis and volume on the x-axis.
The idea is you want to maximize volume while minimizing surface area by changing the radius and the height subject to these constraints shown here, you should have gotten something that looks like this blue curve.
This is the Pareto frontier, and the question was, well, where's the Utopia Point?
The Utopia Point is down to the lower right.
Maybe you plotted it differently, and that's fine.
So you want as high a volume as you can with minimum surface area, but the best you can achieve-- there's nothing in this white space here.
The best you can achieve is this blue curve and above it.
And so then I'm plotting to a small and a big one.
So the Coca-Cola can is shown.
The soda can is in the lower left corner, and you can see it's pretty much on the Pareto front.
And then this is the standard 55 US gallon oil barrels.
You know, if you go to like a junkyard or a refinery, you'll see these barrels of oil.
Actually, it's not exactly on the Pareto front, but it's very, very close.
So that was the intent here.
Any questions about this?
Yes?
Are you trying to determine whether or not the oil and the soda can were on the Pareto front, didn't that depend on how you defined the assumptions?
Not really, one of the key assumptions that was given was that you should assume zero thickness.
Right, so it's just a volume versus surface.
There's no actual thickness
In terms of the-- I need to double check the-- so when I looked at the height of the different cans, were we supposed to leave the problem as defined in terms of the original units?
If you updated the units as the maximum height to be within a more reasonable range, not 2 meters versus 900 millimeters, neither point was on the Pareto front.
It just depended on how well you defined your upper limit to be
Oh, I see
And I was wondering if we were expected to maintain the original upper limit, or if we were allowed to change it
Oh, if you change it, that's fine

The basic idea of this question was to understand what is a Pareto frontier.
And you know, to actually think about something we use almost every day.
You know, we drink a soda or something like that, and that shape is not a randomly selected shape.
It's actually based on a trade off between volume and efficiency-- right, volume and surface area.
And here you go, here's where it falls on the Pareto frontier.
That's sort of the general idea.
Well, we haven't graded the exams yet, but I think we'll be fairly generous on this question.
OK, so then the last one was really the-- that's very recent happenings.
If you read the article, you realized, wow, there's kind of a prehistory here.
This is not just something that happened a month ago.
There's a long history leading up to this emissions scandal, and the key table in the article is this table shown here.
It's essentially these n o x numbers for emissions where you had the different-- so these are requirements.
These are regulatory requirements.
And a big part of the story is that the requirements-- this is just for diesel engines, the supplies for diesel engines in the US and in Europe are different.
And so you can see that the limit is essentially the regulatory requirement.
The dyno shown in green here is demonstrate-- this is verification.
Did you satisfy the requirement as written?
And you can argue that because these vehicles had software in them, the purpose of the software was to detect whether or not the vehicle is being tested for emissions.
And the reason you can do it is because the drive cycles are very tightly prescribed.
So if the vehicle is operating exactly at the speeds and velocities that correspond one to one to those test drive cycles, the software knew, ah, I'm being tested for emissions.
And you know, it's like knowing that you're being watched as a kid, and you're behaving because you know you're being watched.
And then whenever you think you're not being watched, you do something very different.
That's the general idea.
So the green was basically the dyno results with the cheating, essentially, with the software embedded.
And then the West Virginia University results, which were this independent testing, showed that the actual emissions during regular operations were about 30 times worse than this.
And the trade off here is, of course, with fuel efficiency.
That's the big trade off.
And so, we haven't read your answers yet to this question, but it's extremely relevant to system engineering and in particular, to the distinction between verification and validation.
So you can argue that verification was passed here because it passes the dyno test.
These are the regulatory tests that are prescribed, but it fails, essentially, in the field to satisfy the requirements.
And of course, Volkswagen has admitted that they cheated in essence.
But I'll be very interested to see how the legal side of this works out because if you interpret this very narrowly-- and you cannot say that those vehicles failed their emission standard tests because they've performed properly on the dyno.
So to what extent are these requirements, these environmental requirements, which are very, in a sense, narrowly defined in terms of how these requirements are verified legally binding in terms of being able to perform at those levels outside of the drive cycle test cases that are prescribed by law?
Did you see what I'm saying?
So even though morally, people generally say that Volkswagen has cheated because they're essentially reconfiguring the vehicle depending on whether it's being tested officially or not.
But from a legal perspective, I'm not sure this is fully done yet, so we'll have to see how the lawsuits and all these things play out.
Go ahead [INAUDIBLE]
Would you would you say that, looking at it from the regulators perspective, that they had set up an expectation that didn't flow into a requirement?
And if it was a requirement, it was unverifiable until WVU-- it was an unverifiable requirement in a sense
That's one way to say it.
And you know, what is the response to this?
Well, one response could be that a whole bunch of more test cases-- you know, the actual drive cycles that are going to have to be tested are going to be much more-- I don't know.
We'll see.
This is a big deal not just for Volkswagon, but I think it's a very big deal in the bigger picture of how do regulatory requirements and standards apply to all kinds of products, and how do you check them.
Let's see real quick if there is a question at EPFL.
Any comments, questions?
Regulation, actually, in Europe, it was forbidden and explicitly forbidden to introduce a [?
defect ?]
device in the car
Yeah, and there's penalties for that, right?
Yes, exactly
OK, good.
Nathan, you had a point?
I was wondering if you've seen any analysis on like WVU doing tests on other vehicles to compare what they show on their dyno results versus what they're admitting in real life, even though it might not be this extreme
I don't know.
I don't know what tests they've done internally
[INAUDIBLE] it's like a result [INAUDIBLE]---- what do you call it?
The test being so different from the actual real life driving because nobody accelerates from zero to 30 in 30 seconds.
That's what they've been testing
Yeah.
So there's a real, real strong connection here with system engineering.
And that was the purpose of this question, is to sort of push-- this question was not a gift.
This sort of really required you to dig in, and it was very timely, I think.
So cheat to meet requirements failed validation.
That's what I would say as a quick summary.
Any final questions, comments about the online-- the quiz, the midterm quiz?
OK, we've had we've had our interactive discussion with the INCOSE board of directors, so hopefully you enjoyed that.
And we are now right on time to hand it over to Octanis.
And Rafael will present this-- Rafael Fabian Tychui.
So essentially, you'll hear it's about the design manufacturing and deployment of a rover for extreme environment-- Antarctica, in particular.
And so, we look forward to Rafael's presentation.
And as you listen to Rafael, think about what you learned in this class so far, how does it connect.
And I think Rafael also tried to gear the presentation a bit to the system engineering concepts
So I'm Rafael.
I'm just going to present myself quickly.
I'm studying here at my master's in nano and micro-electronics, and I'm speaking on behalf of Octanis.
So just in case, the image quality wouldn't be perfect, you can find this presentation at MIT and on tiny.cc/octanis just to see the picture.
And well, Octanis is an association that was just recently founded.
We have certain key values.
So fascination is one of them.
We set in-shop goals and then get very passionate about this.
We want to build affordable products, so we use open source software, we 3D print parts, and so on.
We use off-the-shelf components.
And we want to do open design, so we share our code on GitHub.
We share our mechanical designs to the community.
And one point is also education, so we, ourselves, want to learn from what we're doing and want to make others benefit from our learning, and try to help them, as well, with their projects.
And how we tackle things is kind of summarized in this statement from Tom Chi from Google X.
So we minimise the time to try things and maximize the time of learning.
That means we iterate very fast.
We do a prototype very fast.
We go on with the idea, with a prototype, with a sketch, a design, and so on.
And we try it.
If you fail, just move, try the next thing.
And you know, that's basically how it is.
And the first project that just emerged out of this procedure is Octanis I, so that's what I'm going to talk about today.
So maybe quickly about the team-- so the rule 7 plus minus 1-- plus minus 2 applies here as well.
So we are eight members now working on that rover-- from different fields, from engineering, physics, life science as well.
And the story began actually when Sam was here as well.
For the QHN, just in case.
And Anna found about this competition or that call from Ecuador that was about building-- or sending a student project, or whatever, to Antarctica.
And they thought, well, let's [INAUDIBLE] take this challenge, and we're going to put a rover on a weather balloon, send it from South America to Antarctica, deploy it there, let it drive around for one summer.
And let's see what happens.
And well, they started immediately with doing simulations.
There's the online tool that allows to kind of predict the optimal time for a trajectory in the air, so a particle or an object in the air that will have a desired trajectory.
And you can kind of find out the best moment in the year to do this.
And it was actually demonstrated, kind of, to show that this is possible to have this trajectory.
Well, I heard about this later, and they asked me if I could do the electronics design of this-- so the main board and so on.
And I was fascinated by the project, so I joined immediately, but at the same time, I saw all the challenges that we're going to have to solve.
So I thought about all the problems we have.
For instance, what happens if we lose the balloon in the ocean?
What happens if we run out of power and the temperature drops-- if an arm breaks, and you know, we cannot get up anymore?
But at some point, I just realized that, well, all we do is we just send a balloon with the rover to Antarctica, and we're going to deploy it there, and it will drive it down.
And I think that that's something we saw also in this class because I'm following this class passively on WebEx and with the videos.
So yeah, the thing that you have to see is the big picture as a system engineer, right.
You see the whole mission, and you don't care about the details at this level.
But you're going to decompose the problem into specific sub-problems and have groups of people working on this, and that's it.
So it just looked easy to me.
In the end, we're just going to do it, and we started with the first problem.
And just to give you an idea, that's one of our last prototypes.
So that's how the rover is going to look like.
We see two struts with wheels on each side, solar panels and 3D printed parts for 3D printed wheels and struts.
And you see the balloon in our logo.
And we're not going to do the balloon right now, but the first mission will just be the rover itself.
And that slide basically summarizes the goal.
So the mission of Octanis I is to build a reusable autonomous solar powered and open source for outdoors, and in this case, for an extreme environment like Antarctica.
So let's go down to V-model, right.
We'll start with the stakeholder analysis.
In our case, it's pretty simple.
It's the Antarctic Institute from Ecuador, which we contacted and showed our project, and they were pretty happy about this, and I want to learn more.
So we presented our project.
We sent in a proposal and are right now, writing the final documents for them.
And what we basically have to provide is just the goals of the mission and the scientific findings we're going to have, so we will produce a paper at the end of this and write about what we found.
And further down, the V-models, will be the requirements.
So we can look at some Shall requirements.
So the power system of the rover shall provide 10 watts on average, so we figured that out after like riding on a power budget.
Should We want to drive around that will cost that much power, and that will be that much time during the day, and so on.
So that was the number we came up with.
Oh, and the interior temperature shall not drop below minus 20 degrees Celsius, so about 10 Fahrenheit.
That's basically the point where you cannot charge the battery any more because of the chemical composition of it.
So standard batteries, you shouldn't go below minus 10 degrees.
That's really a hard requirement we have to provide.
The total weight shall not exceed 2.5 kilograms.
Basically, we look already for the mission, for the balloons-- so it shouldn't be too expensive to put the helium-- like, to buy the helium and so on.
And the rover shall be able to communicate to us from any point of the Earth.
For that, they basically used the Iridium Network, which you can pay like for a text message.
You can pay something and you can send your message from any point as well.
There's a bunch of satellites covering the whole planet basically.
And [INAUDIBLE] managed to climb the slope, a certain slope.
That's just to assure that we can not just drive on flat terrain.
There's also some should requirements
Rafael, quick question
Yes?
Where did the 14 degrees come from?
Did you do like a terrain analysis on Antarctica, or how did you come up with 14 degrees as the requirement for this hill climbing?
We basically looked at the motors we would have in mind to take, and then figured out that that's basically what they can do.
And we tested it.
So I'm going to talk about this later.
We didn't do this up, down only, but we looked at what we have and what we can do.
And that's what we figured out would be a reasonable kind of choice, but yeah, good question.
So the should requirements cover reproducibility, so any non-expert should also be able to build this rover without being an engineer user as well.
The cost should stay below $1,000, so the material costs only, of course, not the R&D.
The range should be around 1 kilometers a day.
That's kind in the order of magnitude of the Curiosity Rover, so we're not stressed.
We don't have to go fast.
We have weeks or months to be there, so one kilometer per day is just fine.
There should be for a scientific payload.
Room means physical space and also certain conditions like temperature, humidity inside, depending on the experiment.
And well, it should survive one Antarctic summer, so that implies also the conditions that we have there like the weather and the sun, the radiation, and so on.
So with all of this done, we had our first diagram, so pardon the [INAUDIBLE].
Our representation is not what we learned in this class, but that's what we had, right?
So this graph is mostly on the electronics.
It's focused.
There's a main board in the middle.
I'm not sure if you see that clearly, but a main board with a bunch of sensors up here, and also a camera, and RF modules for communications, and maxillary board for the scientific payload, for instance.
And we kept going with this, and we basically designed a PCB with all these components.
And most of it worked as well.
We also thought about the mechanics of course.
This design is inspired from a NASA project that was, in the end, abandoned because it was too high in cost, but still-- this manner of [INAUDIBLE] was kind of having the same structure-- so two struts that would allow to flip the rover entirely around if it gets clipped by the wind, and also put it more flat or align it to the sun, so that it gets the maximum of the energy.
And yeah, that's what we had.
And we went out with our first prototype.
It's pretty cute, right?
We went to Sweden with that.
It has a bunch of cheap Chinese solar cells from the eBay.
And the rules and everything are already 3D printed.
Printed It has our first main board, and it was just capable of driving around somewhere on the snow.
And we [INAUDIBLE] testing the parachute, so [INAUDIBLE] came back.
And it worked a few times, but in the end, it had kind of an incident.
So we could really test it and analyze what happened, and so really what we can improve for the further iterations.
And I stopped before at the remodel.
I want to go down a bit deeper there and show everyone a few examples.
So that that's the electoral power system.
And I want to summarize again, that the relevant requirements that are 10 watts on average that should be produced and the temperature that should be held inside the rover.
So we have this choice of having the small, one chip solution on the left.
That was cheap.
It was easy.
It was reliable.
Or we could have that more complex system where we have more flexibility.
We have, actually, more power output than the small chip, and it's basically adaptable to what we need exactly.
And we, in the end, went for the second-- for the right option mainly because of the power output because the 10 wants imply-- a few amps, actually, of output and a solar powered input that would have to be managed.
And that's just not possible for the small chip we had in our first prototype actually.
So we had a concept that was inspired from the Swiss [?
Cube ?]
Satellite because I remember going to work there for his masterpiece, and he got inspired from his concept of having this particular architecture that implies having a shared VSS buss.
So you had your 3 volts that was feed [INAUDIBLE] to your system, and directly to that, connected to solar panels and the battery.
And if the power bus would drop below the 3.3 volts, it will get regulated or stabilized by discharging the battery or by solar power, but you cannot kind of steer this.
And if it will be too high then 3.3 volts, it will [INAUDIBLE] it again with charging the battery.
And that was basically the main concept that he implemented then.
And the next steps, if you go further, would be the component selection, the PCB design, right.
The [INAUDIBLE] materials-- right now, we're programming the framework.
And testing, that's also ongoing.
And at the same time, optimized for having cheaper components-- components with the smaller footprint, so you save space on your PCB and so on.
So that would be really at the bottom of the [INAUDIBLE] right now.
Also, maybe, a good example that shows this trade space exploration would be the mechanical design.
So the driving mechanism, that could be-- well, wheels like we had it, but it could also be like caterpillar or any other kind of mechanism.
The motors could be cheap ones from eBay for $3 a piece or like high end Maxon motors we all love here in Switzerland, right?
And well, you could print that with a 3D printer, the structure.
Even the gears, you can print them out, actually, yourself, with plastic.
It works actually.
Or you could select metallic parts.
We have also a few-- like, the worm guy for instance, we need to take a metallic one.
In the end, we had a combination of all of these things.
So we had the struts.
We also have the Maxon motors that were sponsored luckily, so the price isn't too high-- and the metallic worm guy, as I said, 3D printed struts, 3D printed wheels.
And in the end, there was a trade off between costs, reproducibility, reliability, the range of the robot as well, and the maximum slopes you could do as well.
So what we could have done was this [INAUDIBLE] metrics, or the further points, to optimize this design, but we figured out that kind of without this, we just decided, kind of, to take one of these options, I think.
So that was what we learned from the prototype one.
And then the second one, we had a few adaptions.
So we changed our main microcontroller.
We changed the RF module to a standard that is now emerging called LoRa.
And we also added that power system I talked about.
But mainly, it remained the same on that diagram.
And just to show you how we actually are looking at that goal that we are having the rover that is reproducible by someone.
This looks pretty much like a kit, right?
Its all 3D printed parts.
You can just assemble them, the solar cells and you know, the structure to build the body.
And looking at this, you might see that it's not many parts.
It's almost, as we discussed, within the range of a non-complex systems that could be managed by one person.
But in the end, for instance, the solar panels that are, again, decomposable in separate parts.
We manufactured them ourselves taking blemished cells from eBay, putting them into arrays, soldering the connection.
So in the end, you have a lot of individual parts-- also, all the parts on the PCB and so on, so it quickly explodes in size.
The second prototype had more power because of the solar cells-- and also improved mechanical design, so that the wheels were lighter and more stable-- also, the struts and so on.
And we tested that again, this summer, on a glacier here in Switzerland.
And to wrap that all up, just a few points, we didn't maybe do like an industry or like we saw it in this class.
So as I talked earlier about this before, we mainly had this bottom up approach.
We also had a global vision, at first, to define the subsystems and so on, but we didn't strictly iterate from the top of the model to down part because sometimes we just saw that great components-- for instance, the RF module-- that will provide us with all a lower power consumption, a higher range, and was just perfectly adaptive for our mission.
So we are stuck with this one, and integrated it in the design, and built the rest of the components around it.
There's also, of course-- we didn't have like formal PDR or CDR because we were a small team and also because we're not doing this-- let's say, as a main-- well, yes, it is kind of a main occupation, but we're still studying next to it [INAUDIBLE].. Also, we may change our design decision on the way, not like the complete design but minor, minor things of it.
And we mainly focus on this approach to do rapid prototyping and do a lot of iterations on one thing instead of making first concept studies, and then select the best one, and only then really go to build it, and cut metal-- that's [INAUDIBLE].
That we can do because we basically have all these low cost components, so it doesn't cost us a lot to build one prototype because everything is 3D printed, everything is electronics that is pretty cheaply doable right now.
And yeah, we can do this.
And to motivate us, or to give us some pressure, we set these artificial deadlines.
So we put the design freezes in our calendar.
So from this point on, we will not change these parts of the systems any more.
That allows us to kind of move on on this and get some things custom, for granted, so we don't have to adapt other things because something else changed again.
And we have these field tests like I talked about in Sweden and on the glaciers, so that gives us kind of pressure as well.
Because I remember before the week in Sweden, we basically relayed in the lab, and we spent the night to keep the 3D printer running.
And as soon as the part was done, put the next one on, and you know, changing our-- being there in shifts to keep it running.
And that was a great experience because you really have that thing to be done because it cannot be 3D printed in [INAUDIBLE] method
Rafael, so why don't you go one more minute because we want to do a couple of questions also, yeah?
Yeah, I'm almost done actually.
So how we structure all this data is basically on the Wiki.
That allows you to go deeper into the different parts, and we use Trello for defining milestones.
It's an online tool-- can use for free.
And the major systems engineering challenges I saw here was especially the time management, and human factor as well because you kind of need to have this boundary between someone who's doing the mechanical design, someone's who is doing the electronics design.
Don't cross the boundary because that person is an expert in his field.
And if you just come up with something-- hey, why don't you do this with this?
First of all, it will annoy the person.
And second of all it's like, that person has made a lot of thought in this, and you shouldn't just come with your thing you have done in five minutes.
Ask the right questions.
And [INAUDIBLE] thought, it's difficult to just put the design on paper for components you haven't even tested before.
So just like a few seconds left for this.
We go out to [INAUDIBLE] as well.
So we're going to test this rover in February in Antarctica.
We can only send one person to there, so we have this agreement within 30 days.
And they allowed us to send one person in February 2016 to Antarctica, and it's none of us who's going to go.
But it might be someone or someone else.
And you have that chance to go there, all expenses covered, and you will be there as a field scientist, collaborate with the bases, and basically produce a paper at the end of this.
If you're interested, octanis.org/apply.
You'll find all the information's there.
And the deadline is tomorrow actually
All right, just in time.
Thank you, let's give Rafael a hand.
OK, so we have like three to four minutes for questions, so let's start with questions at EPFL and then one question-- we'll go back and forth
And just maybe a critical question, why did you choose to switch from your first communication [INAUDIBLE] after?
[INAUDIBLE] thing we had before?
Yes, and why did you change?
Before we had kind of a DIY component that was called AHX 1 that there you basically had to modulate the signal that you're going to transmit yourself, and it was just way too complex.
The new component, you can just send zero commands by [INAUDIBLE],, and it will just do its job.
And it's higher range, as well, and lower power consumption
And how you shoot to have a [INAUDIBLE]??
Oh, sorry, yeah, that component's only to communicate from the rover to the base, so that will be like 20 kilometers.
For the coverage to the land, we'll use the Iridium network.
Any questions at MIT?
Sam?
This is really interesting, thanks.
So I was wondering, with your focus on openness and affordability, I guess, do you foresee any participation by the public in terms of like prototyping and testing or even improving the design-- and/or data analysis?
Yeah, well, we're completely open, so we are based here in [INAUDIBLE] in a place called [INAUDIBLE] hacker space, maker space, whatever you want to call it.
We have open nights over Wednesdays, so anyone can come.
And all our design is on GitHub, so if there's going to be remote collaborations, we would really appreciate that.
So anyone who is interested in the project, you can go on our website, octanis.org/join, and we [INAUDIBLE] to collaborate, of course.
Does that answer your question?
Yes, thank you
I'd like to ask you a question.
So you know, I think one of the things that was quite-- first of all, the enthusiasm that you guys have is infectious.
It's very clear you guys are very passionate about this mission, and it's very exciting.
The fact that you didn't do these milestones, you didn't do a formal PDR or CDR, but you're doing essentially, kind of spiral development.
So two questions, one is how many more spirals do you think you'll be able to do before you actually deploy the rover?
Because you said you did two spirals.
It sounds like so far.
So how many more spirals?
And how will you know whether your mission was successful or not?
So what are the success criteria in the end?
When you look back at the actual operation, how will you decide whether you were successful or not?
OK, thank you.
So for the first one, we had our design freeze for the last iteration already.
The truth is we're still adapting some points of it because technically, we decided on all the parts, and we're just implementing it right now, and refinding it, and correcting the bugs.
So that was actually done.
And we're now building the last prototype or the last, actually, robot that is going to go there.
And on the thing about achieving our goal, so what are the mission criterias, that's going to be written down in the project proposal, or the paper, that we're going to give to our stakeholders.
So the Antarctica Institute of Ecuador.
So one part will be the power autonomy and also the communication-- like, it has a reliable communication.
And one part will be also the scientific payload, there will be insights of conducting experiments on the field.
You want to add something, Sam?
Yes, 100 meters per day [INAUDIBLE] Very concrete goals, can we drive 100 meters per day, yes or no?
Can we log every minute, every data point that we collect temperature pressure, et cetera?
Can we log that, and can we send that to the base?
Like, does the 15 kilometer range of the LoRa is that [INAUDIBLE] or not?
Does that work?
And of course, does the satellite communications work?
So once they will send back a summary of the data, and we want to see if that works, if we can calculate all the stuff that we want, on the rover, and if we get that stuff back [INAUDIBLE]?
Good, thank you.
One final question?
Anybody at EPFL?
Here at MIT?
OK, Sam has one more
I just have a question about-- because it's going to be, I guess, situated near our base, is there any thought about, sort of like, how maintenance would work when it's in operation or if something happens?
Well, the person who is going to be there will be in charge of doing maintenance, so if something happens, if it gets stuck somewhere, he will rescue it.
And if some parts fall, he'll provide the replacement parts, of course, so we will not just send that thing there and hope it works.
Yeah, we have some backup plans as well

Great
Sounds like Sam is interested
We have another question here at EPFL as well
OK, do one more, and then we'll close the session.
Go ahead
Well, I was just wondering if you believe-- if you had followed all what we have seen here in this class about the procedures, PDR, CDR, and so on, would that have hindered you in being so reactive?
Or do you feel that you can be reactive while still following all those procedures?
A good question, thank you.
Certain points of the concepts that we have seen would have improved, especially the efficiency in the design procedure.
So going down step by step would help us to exactly create what we defined before and not go too much into different directions.
But on the other hand, the fact that we explore our traits, those [INAUDIBLE] in the beginning, makes that we might have an even better solution because we wouldn't have thought about this before when we were in the upper part of the [INAUDIBLE]..
So it's kind of a mix.
I think a mix of this is really the perfect approach for our project.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's get going on system integration and interface management.
So we're essentially moving up the right side of the V. And the idea is all the details have been designed.
The concept has been selected.
We know what we're doing, but now we have to re-integrate the system and make sure it works-- it performs the functions that are intended and satisfies the requirements.
And so what I'm going to cover today is three things.
First of all, why do we care?
Why is interface management important?
Why is it important?
There's two main reasons.
One is because a lot of failures originate-- so a lot of failures originate at interfaces.
And the other reason is because in order to design and manufacture a lot of systems, we need to work with partners and suppliers.
And I'll talk about interface management, sort of the details of it, the types of interfaces.
I'll Introduce the DSM, the Design Structure Matrix, as an important method.
And then we'll talk about ICDs, Interface Control Documents.
They're a very big deal in practice.
And then finally, I'll talk about system integration-- in particular, the sequence which you integrate systems, and then the role of standards.
All right, so here, the point that I want to make here is when you talk about interfaces, the first thing you need to be clear is, is it an internal interface or an external interface of the system?
And so I think you remember this chart here that has-- is this you guys-- that has a system boundary that's defined.
So the system boundary, in this case, we have this digital camera here.
And the internal interfaces are essentially the interfaces that are within this boundary.
So those are internal interfaces within the camera, within the peripherals, that are included in the system boundary.
And then we have external interfaces, which would be the interfaces between the system boundary and anything that's on the outside that you don't design or control directly.
So that's an important distinction.
The next point I want to make is that you have to really carefully think about interfaces as seeds of potential system failures.
And so I have three examples here for you.
So the first one is-- this is actually-- Katya, sorry, this is from Russia, from 2007.
This is traffic.
This is one of the big-- not [RUSSIAN],, but prospekt, the six lanes or eight lanes on one side, and there's a crossroad coming in here.
This is a traffic intersection.
And you can see traffic is completely blocked on this side.
So traffic engineers think a lot about interfaces, which are mainly your intersections, the roundabouts.
Interfaces are really where a traffic system makes or breaks the traffic system.
So that's a bottleneck.
An interface is a bottleneck, basically.
The second example here in the middle, this is actually an example from MIT from an undergraduate class in design where the students were designing an airfoil.
So this has been produced with a foam cutter.
And you have a spar that goes through here.
And this was intended for the MIT Formula SAE race car to create downward pressure on the rear axle.
They tested it in the wind tunnel, and it worked well at 20 miles per hour, 40 miles per hour.
And then at 60 miles per hour, there was a failure, and the airfoil just basically disintegrated and was carried off into the wind tunnel and completely shredded by the propeller.
Why did that happen?
Well, they thought about the shear loads.
So when you have an airfoil, there's a force that's acting orthogonally to the airfoil.
So they thought about that.
The problem is there's also a torque, like a pitching moment, a torque that's generated.
And the higher the air speed is, the higher the torque.
They didn't think-- there was no real good way to react against the torque that was being generated.
So at that high speed, the airfoil, all of a sudden, tilted and created a lot of drag and basically was destroyed.
So the airfoil itself was OK, but the interface was not.
The third example here is a software example or a software interface, very famous.
This was the launch failure of the Ariane 501.
The very first launch of the Ariane 5 basically led to an auto-destruct of the rocket upon ascent.
There was an accident investigation.
And I'm just quoting you here from the key passage, which said that "the active initial reference system transmitted essentially diagnostic information to the launcher's main computer."
And so the main computer was interpreting this information as though the rocket has gone off course and trying to correct for it when, in fact, it hadn't.
It was actually physically going exactly the right trajectory, but it was misinterpreted.
And this is also a very famous failure due to software reuse.
So three examples of interfaces-- bottlenecks in traffic, structural failures, and erroneous function calls in software.
And so be careful about interfaces.
The second reason we care is working with manufacturers and suppliers.
So the picture here is essentially the Dreamliner, the Boeing 787.
And you can see here the different colors essentially represent the different countries or the different suppliers that are providing different parts of the airplane.
Now, does anybody-- let me ask this question to MIT, see if we can hear them now.
No?
Well, maybe we can use the chat.
What's different between the Dreamliner-- and you could have seen a similar picture, say, for the 737 or prior generation aircraft.
So what's the real difference here with the interfaces, with the 787?
[INAUDIBLE]
Yeah, that's true.
Composites is one thing, but that's a structural thing.
I have something else in mind.
There's some other fundamental change that was made in the 787 beyond composites
[INAUDIBLE]
Nope
[INAUDIBLE]
This has to do with the way the airplane is assembled, with assembly
[INAUDIBLE]
Yes, yes, exactly.
So the supply chain is very different.
And I guess I lost my-- hm, did I reverse this?
So the point I want to make is that in the 787, the modules are essentially pre-outfitted.
Like, all the systems, like the electrical system, the hydraulics, everything is already installed by the time the module arrives at the assembly plant in Everett and close to Seattle.
Before, you got the structural sections, but then all the electricals, all the inside was done on site.
So the modules are pre-outfitted with all the subsystems.
And the final assembly itself is very short, only a few days.
So a big, big step, and that's exactly right.
That's the supply chain impact.
OK, so then the final point here is this is essentially a launch vehicle design, a launch vehicle that's being designed at NASA, the SLS.
And if you think about a launch vehicle, you have the first stage, second stage, third stage, payload, the fairing.
Actually, a lot of these fairings are made here in Switzerland.
And defining these interfaces very clearly is critical to reduce complexity.
Understanding the different types, which we're going to get into next-- identifying the interfaces is a way to reduce risk.
And I just talked about these problems originating at interfaces.
And so hopefully, it's pretty clear now that this is a critical issue.
So what I'd like to do now-- and we'll try to troubleshoot the audio issues in the meantime-- is for you to turn to your partner exercise and share with each other, what is an instance in your past experience, maybe during an internship or in a project that you worked on, where interface definition and management was critical?
Some of us went this summer to Russia, to Moscow, to follow a summer camp on system engineering and space engineering.
And we had a project there where we had to design a mission to deep space.
And everyone was assigned to a group.
The groups were around five to seven people, and there were five to seven groups.
And each group had a subsystem to design, but no group was assigned to system engineering.
So actually, there was no one to make sure that all the interfaces were OK.
So we had to go to each other group and ask for the interfaces.
And then when we had to go back, it was a mess.
And at the end, every group had something good.
But when we tried to put all the systems together, it was a big mess and nothing was working
That's a pitch for systems engineering.
So that's a good point.
So even if you negotiate bilateral interfaces amongst subsystems, there's no guarantee that it will work at the system level.
OK, can we get from MIT?
Can somebody give us what you discussed?
Is this the mic working?
Yeah, now we can hear you
Great.
So my grad school project here at MIT is a secondary payload on a big NASA mission.
And as a secondary payload, the interface control is a really big deal.
So we're getting all of our power and all of our communication from the spacecraft, and we're also getting a heater circuit from them.
So any change is a big deal.
We had to change our heater circuit and we had to go through a lot of paperwork just to splice a wire.
So no issue with it, but it's not as easy as it sounds, and there's a lot of control over it, I would say
OK, maybe another example from MIT.
Somebody else?
Does this work?
Yeah
So the--
We've got it fixed out now, by the way.
I think this is all working now, so good
The last project I was a part of was building an aircraft for a Red Bull Flugtag.
So it was like a giant glider that we'd push off a platform.
But we were kind of building it in a hurry.
The time constraints were pretty quick, and we didn't really have all the requirements until a month or two before the competition.
So we winged a lot of the design and construction.
And so we thought about how to build the wing and how to build the empennage and how to build the cockpit, but we didn't think about how to put them all together.
And so it turned out just being wrapped with a lot of carbon fiber and epoxied.
And in some cases, we used wood.
And it was a very heavy and poor-looking assembly once all that was together.
Even though each part individually looked great, they didn't really come together perfectly.
So we'll change that next year, but could have thought more
OK, good.
Great example.
All right, so hopefully, this is sort of motivation that, in a sense, a system is the functions you want, the components, and the interfaces.
You've got to have all three.
If you only have great components but you don't understand the interfaces, you stick them together, I'm not sure what you're going to get.
Yes, do you want to--
[INAUDIBLE] Is it also important the interfaces with testing facilities?
Yeah, absolutely.
So testability requirements, how you're going to test it, is critical.
Also, ground support equipment-- you know, like maintenance equipment.
So not just the interfaces, the system when you're operating it, but during testing, during maintenance.
And those are often forgotten as well, so great point.
So let's keep moving here.
I want to talk about the types of interfaces, and then we'll get to DSM.
So here's a set of examples of interfaces that we encounter.
Let's see if we can get the slides back up here.
So essentially, here's some examples.
In the upper left, we have a valve, and then a tank on the right side.
And we essentially have mass flow from the valve to the tank, and m dot, dm dt.
We have a rocket.
If you think about the rocket interface with the payload, it's transferring momentum, right?
The rocket provides thrust, force F, for some amount of time, delta-t, and that's the momentum imparted.
We have a heat exchanger that is maybe giving heat off to the air, the surrounding air.
So we have a heat flux, q dot.
We have solar cells, PV cells, that are producing electrical power, voltage times current.
And this power flux is being sent to a battery where the energy is stored.
So these are all pretty obvious examples.
And then on the right side, things get maybe a little bit less obvious.
You go to a URL and you download an HTML file into your browser.
That's an information.
You're getting data.
You have a motion sensor attached, say, to your house, and it triggers an alarm.
That's a command, different from data.
NPR is National Public Radio.
This is kind of the high-quality radio news reporting.
And you listen to this.
So there's a cognitive interface.
And then if any of you have had any psychotherapy, this is also an interface between the patient and the psychotherapist.
This is an effective interface.
So these are very different.
And you can say, well, there's hundreds of types of interfaces.
Well, no.
The argument here is all interfaces can be reduced or described by these four canonical types.
The first one is physical connection.
So A connects to B.
B connects to A.
A physical connection typically implies that there's a force or torque that can be transmitted across the interface, and it's always symmetrical.
And then we have energy, mass, and information flows.
And I'll give you some examples of each of these types of interfaces.
And these energy, mass and information flows typically are asymmetric.
It flows from one to the other.
Physical connection is always symmetric.
So let's go through these, and I'll give you examples.
And then we'll see what can we do with it.
So here's physical connection examples-- rollers, brake pads.
A finger touching a touch screen is a physical interface.
The interface can be reversible.
In other words, the connection is temporary-- and examples of that would be electrical connectors, USB ports, latch mechanisms, nuts and bolts-- or permanently connected, such as in-- so one of you mentioned riveting, spot welding, fusing.
And a fun question that comes up, well, I'm a software engineer.
What is physical connection mean for software?
Well, it's a kind of tricky question.
I think the closest analog we have is compiling.
If you have pieces of source code and you compile your program, then you've essentially fused this together into something new, right?
So I'll show you the example.
So how do we describe it?
Well, here's an OPM model of-- I'll give you a couple examples from xerography.
So we have a main motor here, number one, and then two different clutches, number two and number three.
And the process linking them is engaging, right?
The main motor engages through the clutch.
We can hide the process, and then we just have this bi-directional interface between one and two and one and three.
And then if we show this as a mini-DSM here, one connected to two, one connected to three, but two and three do not connect directly with each other.
They don't have a direct physical interface.
And you can see that because those cells are empty.
So physical connection implies symmetry in the design structure matrix that we can build up.
Yes, [INAUDIBLE]?
[INAUDIBLE]
Oh, DSM-- so DSM means Design Structure Matrix.
And I will introduce that in a minute.
We've briefly mentioned it before in the class, but I'll get into it in some detail.
Here's some more picture examples of physical connections-- so a welding joint, irreversible, reversible, nuts and bolts, RJ45, an electrical connector.
And then here's your-- hey, I'm an old guy.
I learned Fortran.
Well, Basic was my first language, and then Fortran.
I did a lot of Fortran.
You guys probably don't even know what that is.
But Fortran-- right, [INAUDIBLE]??
Fortran's kind of a big deal still in some--
[INAUDIBLE]
OK.
So basically the way this works is you have your source code, which are .f files.
These are basically Ascii files, right?
And then first, you transform them into binary files, object files.
And then these get merged together into an executable.
So that's the process of compiling.
Any questions about physical connections, this type of interface, physical connection?
Is it clear?
OK, any questions at MIT?
Can you still hear me?
Yes, we're good
OK, good.
So let's move on to energy flows.
Energy flows are present when there's a net exchange of work between two components.
So power is flowing across the interface dw dt, right-- joules per second, watts.
And the energy, this energy flow, this power flow, can take different forms-- electrical power, which is very common in many products.
And so the electrical power usually comes either as DC, Direct Current, or AC, Alternating Current.
And if you know a little bit about the history of electrical systems and networks, there was a huge war.
It's called the AC/DC wars.
It's not the band.
This is not the rock music.
This is the war between Edison, who is a big proponent of direct current, and Tesla and Westinghouse, who were really pushing AC for big power distribution.
So we won in the end?
AC won.
It looked initially like Edison was really-- and there's still, in New York City, there, for a long time, and other places, you still had DC power systems.
The problem with DC power is there's huge losses with distance.
And with AC, you can go to very high voltages and transmit power with relatively little loss over large distance.
That's the main reason that AC won out, essentially.
The thing that's important here-- and power, of course, is current times voltage.
The thing that's important is that these voltage levels-- and in the case of AC power also, the frequency is very important, 50 or 60 hertz.
This is very standardized, OK?
So if you're going to design a satellite system, or a rover, or an automotive system, or a medical device or a consumer product, the voltage levels are pretty-- there's a few choices, but you can't just choose some arbitrary number, because your whole industry, your batteries, your connectors, everything is now pretty standardized.
So for the Octanus 1, Rafael, what did you guys choose for your power system?
What's your--
[INAUDIBLE] 3.3 volts
3.3-volt bus across, OK. And then if you need more energy, you just have to add batteries.
But that will essentially-- and you can do transformations as well.
You can have different voltage levels in your system, but that adds to the complexity.
Thermal flux is essentially heat flux, dq dt.
There are three fundamental mechanisms of heat transfer-- conduction, convection, and radiation.
In spacecraft, radiation is probably the most important because that's the only way to get rid of heat out of your spacecraft.
And then you can do conduction and convection internally.
RF power, so microwaves.
Transmitting or transferring power through microwaves is becoming-- of course, we now had microwave ovens for a long time.
But even doing power beaming over larger distances with microwaves is definitely something that's been demonstrated, and there's a lot of interest in it.
Here also, we are pretty standard frequencies, so 2.4 gigahertz, 5.8 gigahertz.
You have a whole industry built around these standards.
By the way, why the 2.4 gigahertz?
Let's see if-- at MIT.
This 2.4 gigahertz for microwaves, where does that come from?
Anybody know where that comes from?
Is that the frequency that can be transmitted easily through the atmosphere?
Actually, no, it's not good.
It's absorbed.
It's absorbed in the atmosphere.
It's the opposite.
This 2.4 is very important because it's a resonant frequency of the water molecule.
So your microwave oven will work at 2.4 gigahertz.
Why?
Because if you radiate microwaves at that frequency, it makes the water molecules vibrate because that's a resonant frequency, and you transfer a lot of heat or a lot of energy into your water molecules, which are the majority of your food.
So it's actually really bad for-- you'd like to use 2.4 gigahertz, but if you try to use that frequency for atmospheric power transfer, you're right at the resonant mode.
So if you have a high level of humidity in the air, you're going to lose a lot of that power, just heating the atmosphere up and not transmitting the power.
Does that make sense?
Yeah, thanks
OK. And then another very popular frequency is 5.8 gigahertz.
And then we can go to X band, which is closer to 8 gigahertz, et cetera.
But anyway, so this is a kind of growing area is microwave power transfer.
And then of course, the more old-fashioned, perhaps, but still very important is mechanical power, so transferring energy and power through a mechanical interface.
So if it's linear, it's force times velocity.
If it's rotary, it's torque times the angular rate of rotation.
And then finally, energy-- typically, if you want to have energy flow, it does imply that there's a physical connection as well, like some kind of wires or a gearbox or something like this.
However, a lot of interest now in wireless power transfer.
But it's still kind of a-- for it to really work reliably at high power levels, this is pretty immature technology still.
So here's some examples of-- again, this is from xerography.
And you'll understand why I'm using these examples in a minute.
So we have a paper.
We have toner, unfused toner.
And we want to fuse those two together.
We do this through a heat roller and heat and a belt.
And essentially, we're transferring energy from the heat roll to the paper, which allows it to fuse the toner.
So we have transfer of energy from one to two, which is shown by this green mark here in the DSM.
So heat energy is transferred from system one to system two.
One question that often comes up is, well, what if we have like waste heat?
There's heat losses, and there's parasitic energy flows.
Do we care about those?
Do we represent those?
And the answer is yes, you should, especially if they affect your performance or your requirements.
So the first thing you typically do, as you map out your interfaces, is you map the flows that you want to happen, that need to happen.
And then on top of that, you map out the flows that happen even though you didn't design for that, but you need to account for them.
So heat losses would be an example of that waste heat flux.
Any questions about energy or power flows?
MIT, any questions, energy or power flux?
We're good, thanks
OK. Next one, mass flows.
Mass flows is when matter, physical matter, is being exchanged between two elements or subsystems.
So mass flows is dm dt, kilograms per second.
And the form of the matter can be fluids, gases, solids.
I guess plasma as well.
There's a big plasma center here on campus.
But the primary three are fluids-- so if it's fluids, it could be like a cooling liquid, a refrigerant, fuel, water.
If it's gases, air, exhaust gas.
And then for solids, examples would be toner, paper.
If you're in the mining industry, it's iron ore. And typically, mass flow implies an underlying physical connection, like some kind of channel or conduit.
So Rafael, your example of your sucking up and melting the snow and you said there's a hose, so the hose is itself a component, but it enables the mass flow of the melted snow into some holding tank or reservoir.
And so the hose is the physical connection.
And then the mass flow that happens through that hose is what's shown here.
Mass flows can be open.
So there's a source and there's a sink.
The source and the sink could be inside or outside your system boundary.
You have to think about it.
But also, mass flows can form continuous loops, particularly if you recycle things.
Like for example, in the refrigerator, your cooling liquid continuously cycles through your system.
So here's some examples of mass flows.
We have a heat exchanger.
This is a U-type heat exchanger.
So on the top, we have what's called here the shell side.
A fluid is coming in and is running out on this end.
And then we have horizontally, a fluid that's coming in on the lower side, cycling through, exchanging the heat, and moving out.
So a heat exchanger is interesting because what kind of flows happen in a heat exchanger of the flows we've discussed so far?
Energy and mass flow, right?
So the two of them happen together.
In order for the energy flow to happen or transfer to happen, the mass flow has to happen as well.
So these two flows are tied to each other.
Here's an example from xerography.
So we have our photoreceptor belt.
We have residual toner on the photoreceptor that needs to be removed from the photoreceptor.
And we do this through a cleaning lamp, and then a blade.
So in this case, you can show this as an OPM, and then you can show it here where that residual toner is transferred from photoreceptor one to the cleaning blade two.
And this is shown in the DSM as a mass flow from one to two.
And then we have one more, which is the information flows.
So this is really where the big revolution has happened in the last two decades since a lot of functions that used to be done mechanically have been replaced with software.
And here's some examples of these information flows.
So if your system has a user interface, a GUI, a graphical user interface, or some kind of I/O, input/output, with the user, that's critical.
That's an information interface.
And so Rafael, again coming to your presentation-- so I'm going to pick on you a lot today.
Not pick on you, but I'm going to cite you a lot.
You said this rover is going to be controlled somehow, right?
And the big question is always, how much autonomy do you put inside the thing, and how much do you rely on a user that's remote?
And that's clearly an information interface.
The nature of the information, is it analog?
Is it digital?
Is it wireless?
So on the analog side, we're essentially talking about some signal, some voltage, and there's been a lot of work done on ADC, analog-to-digital conversion.
And then the opposite is digital to analog.
And depending on how many bits you have available to you, you're going to have some discretization errors.
A purely digital interface, we refer to it as DIO, Digital Input/Output.
And then of course, wireless, this is the standard, the 802.11 standard, which is really very important these days.
If you think about why we need all this, mainly, the main reason is control, right?
We have sensors, actuators, controllers.
And because this information is never perfect-- there's gaps in the information or there's noise superimposed-- we have to also filter and amplify the information.
And so if you think about the type of information flows-- for example, in spacecraft, we could have telemetry, sensor data.
And telemetry fundamentally means, I want to know how is my system doing.
Is it healthy?
Are all the subsystems working?
Are there some anomalies?
Are components within their temperature limits?
That's what we call telemetry.
And then commands are when you are sending a command to the system and say, you know, turn this way, turn that way.
And fundamentally, telemetry and commands are treated quite differently.
So where do you need better quality information?
Where is the link more important, on the telemetry side or sending commands?
What do you think?
Actually, no.
So if your telemetry is imperfect-- I mean, it depends a little bit what you do with it.
But the commands are typically more critical.
Like, if you have a bit error in a command, and it says fire the engine but you didn't want to fire the engine, you can lose your mission.
So typically, what's known as the bit error rate has to be lower, meaning the link budget is much better for commands.
So there's an asymmetry.
OK, so here's some examples.
So here's a typical control loop.
You have your plants, your system, your actuator, sensors, a comparator that compares your reference signal-- this is how you want the system to behave-- and sends that to a controller, which then closes the loop.
The example here, again from xerography, is you have some original document.
You want to make a copy here.
So there's a lamp that shines light onto your document.
It goes through a lens, a laser diode, and then to your marking system.
The OPM, you have your original, your optical system.
You create a digital image file which then is sent to the marking system.
So if we simplify all this, we can say there's information flow from the optical system to the marking system.
From one to two, information is transferred.
So any questions?
So those are the four fundamental types of interfaces and examples with that.
And so the theory, what the theory of design tells us is that this is universal.
Any kind of interface falls into one of these types.
And in fact, even in biological systems that's true.
Yeah?
When we have a system [INAUDIBLE]??
I'll show that to you in the DSM.
You actually track them separately, but they're within the same-- it's one big interface, but it has these different subflows within it, which means that-- it's not just the fact that there is an interface, but the complexity of the interface is important.
So the most complex interfaces obviously would be the ones where you have all four types, right?
It's a physical connection, and there's an information flow, and there's an energy flow, and there's a mass flow across the interface.
Usually, it's only a subset of those, but you can have interfaces where you have all these four present at once.
Any questions about these types of interfaces at MIT?
No, we're good.
Thank you
OK.
In general, can you hear better now?
Is it pretty stable over there?
Yeah
Good, all right.
So let me talk about DSM.
[INAUDIBLE] was asking about DSM.
So DSM is a fairly recent method, the last couple of decades, for really making a map of your system that shows explicitly these interfaces.
So here's a very simple example.
This is a sample system with five components.
It has a pump, a controller, a motor, a valve, a filter.
And in this case, we have mass flow from the pump through the valve into the filter.
And so this block diagram on the upper left, the exact same information is contained in this matrix.
So a DSM is always a square matrix where your components are the rows and columns, equally labeled.
And the diagonals here, there's no information on the diagonals.
So the component connects with itself, obviously.
So you keep the diagonals sort of blank.
And then all of those interfaces and flows we mentioned are shown as off-diagonals.
You have to be a little careful because there's two definitions of DSM depending on the inflows and then the outflows.
So the definition that we mainly use in North America is where the inflows into a component are horizontal.
So the controller sends information to the valve.
This is that blue square right there.
And you can see that's shown here on the block diagram.
But the valve also sends information back to the controller.
And that information would probably be, valve should open or close.
And when the valve is actually opened, it would confirm that back to the controller that the status of the valve is open or closed.
And then we have the energy flows.
The numbers that are shown here, I don't want to get too much into this, but there's a really cool binary scheme for putting a number that you can then have one number in that cell.
And based on that, you can, through the binary code, reverse engineer what kind of interface is it.
Is it just physical?
Is it just binary?
It's a kind of trick for replacing these colors and cells with just a number that allows you to then reverse engineer what kind of interface is it.
So if it's empty, it means there's no direct connection.
And then you have mechanical, mass flows, information, and energy flows.
So let me show you this on a real example in a minute.
So this method, design structure matrix method, there's some synonyms for it-- design dependency matrix, n-squared matrix, n-squared diagram.
In matrix or graph theory, it's called an adjacency matrix.
Don't be fooled; it's the same thing.
It's just a square matrix that shows you what connects to what.
And then the simplest one is just a binary, 0's and 1's.
But the richer DSMs, they contain more information about the type of interface.
So a lot of work was done originally by Don Steward on this.
This is an original reference from '81.
And then at MIT, Professor Steve Eppinger at the Sloan School has done a lot of work on DSMs.
So fundamentally, it's a matrix representation of your product architecture.
Most of the literature you find just uses the simplest, this binary DSM.
But I'm actually a fan of adding information into the DSM that tells you about the physics of the interface as well.
So let's look at an example here.
Do you remember the refrigerator case study we did a few sessions ago?
So this is a liaison diagram, level one decomposition.
So the refrigerator's been decomposed into 10 components-- the door, the condenser, power supply, all the way to the compressor.
And all we're doing here is saying what is physically connected to what.
And the reason it's called a liaison diagram is because it's going to help you think about the assembly sequence of how this should be put together.
So another way to think about this is like the skeleton.
This is the skeleton of your system.
So if you do have a liaison diagram, shown again-- this is the same one on the left here-- you can just translate that to a matrix.
The matrix, this DSM here, I'm using X's instead of 1's.
But these two contain exactly the same information.
One is a graph view, and the other is a matrix view, but they're identical-- the same thing, the same information, just presented in a different way.
And of course, if you pay close attention to the matrix, you'll see it's actually symmetric.
So then we can say, well, what kinds of interfaces?
And here come the four types that we just discussed-- the physical connection, energy flow, mass flow, and information flow.
So if we apply that idea to the refrigerator, here's our 10 by 10 physical connections.
So we have a design scripture matrix with only the black marks.
So it's symmetric.
And this reflects the liaison diagram.
And then we can add, on top of that, the mass flows, which are in red.
So the mass flows, the way you read this is we have a mass flow from the compressor to the condenser, and then from the condenser to the evaporator, and then from the evaporator back to the compressor.
Do you see that?
And we could rearrange these to be closer to each other.
We could regroup the DSM.
But essentially, these red marks represent the refrigerant cycle within the refrigerator, within the machine.
And so you can check by tracing these from row to column.
You can actually check whether the loop is properly closed.
So then energy flows, since the refrigerant loop that I just talked to you about is important, you can see that those connections we just talked about always also carry a energy flow with them.
And then finally, information flow is very little.
So in this case, there's only one that I've represented, which is the flow from the thermostat to the power supply.
The thermostat defines the temperature at which you want to hold the refrigerator.
And if temperature goes up, the power supply kicks in and drives the compressor.
And as soon as the temperature falls below some threshold, then it reverses, and the compressor turns off.
So when your refrigerator kicks in, that's an information flow from the thermostat to the compressor to the power supply.
If you own one of those fancy, new Samsung refrigerators where the door has a flat screen TV in it and you can push buttons, there's a lot more blue here.
There would be a lot more blue in a really high-end refrigerator these days.
OK, any-- yes, please
[INAUDIBLE]
Yeah, so the question-- I'm just going to repeat the question-- is, can you also map the external interfaces?
And the answer is, absolutely.
So what you have to do, though, then is to kind of draw a bigger box around this and add here the external elements-- so the human user, the wall outlet, the grocery store, wherever you're including in your external world that's interfacing with it.
You would have to represent at least those elements that have a direct connection with any of it across the system boundary.
You would add them here, and then show-- but you would have to then put a big box around it to make it clear what's inside the system and what's outside
[INAUDIBLE]
That's correct.
That's correct.
And we can argue-- of course for the refrigerator, the main interface that you typically have-- well, there are two.
Let's say we go for what you're saying.
We're going to add some external interfaces.
For a refrigerator, what are the two or three most important?
[INAUDIBLE]
Correct.
So where would that enter here?
[INAUDIBLE]
The power supply, right?
So we would add, like, wall AC power.
If we added that, then in this cell corresponding to that, you would have an inflow of power.
And then--
[INAUDIBLE]
So this is the power supply inside of the refrigerator, inside the system boundary.
But if you had the wall outlet, then that would be an external.
And then you would have an interface across, but into the power supply.
So that's one.
What's the other one?
Do you remember we talked about the Carnot cycle?
Well, yes, what do you have to do with the heat?
Burn?
Yeah, so the heat has to be removed, right, radiated out.
And where does that happen?
Which element here is where the heat gets removed, which one of these 10?
[INAUDIBLE]
Not really.
The refrigerant does the internal heat transfer, but the heat transfer to the outside, to the air.
Refrigerators don't work well in vacuum chambers.
Remember that.
It would be the evaporator.
It's at the evaporator that you're radiating the heat to the surrounding air, the environment.
So you'd have the surrounding air as an external system, and then and energy transfer from the evaporator to the outside.
But it's a good question.
So you have to always define your system boundary.
So the question then is, when do you generate these DSMs?
And fundamentally, there's two ways, top-down or bottom-up.
So top-down, you start with a model of your system.
And we talked about those system modeling languages, like SysML, or in this case, OPM.
And then you hide some things.
You hide attributes and states.
You collapse all your processes into these tagged structural links, and then you generate your DSM.
And then the other way to do it is bottom-up where you already have a design or a system.
You decompose it.
For example, you do a product dissection.
You do your bill of materials or your parts list.
Then you do your liaison diagram.
You show the physical connections.
And then on top of those physical connections, you map your mass, energy and information flows.
And once you have that, then you can start manipulating the DSM.
You can group components differently.
You can show modules and so forth.
So I want to show you a real world-- this is not just sort of an academic example, but this is a real-world example of a DSM that we built for a research project with Xerox.
The machine that's represented here is in the upper right.
This is the iGen3 digital printing press.
And represented here is its architecture, its interfaces, as an 84 by 84 DSM.
So the machine has been decomposed into 84, really, subsystems, because you have a lot more than 84 parts.
But at this level of abstraction of 84 by 84, we have a total of 572 physical connections, 45 mass flows, 167 energy flows, and 165 information flows.
And there's a little more detail here in terms of the types of flows.
So in red, we have mass flows-- paper, toner, air, ozone and dirt, because you've got to keep dirt out of the-- the printing engine is shown here in the middle.
Let me just show this.
So you're probably scratching your head now.
Ozone, really?
Ozone?
Why do we keep track of ozone?
What do you think?
Why do we care about ozone?
[INAUDIBLE] at ground level
It's a pollutant at ground level.
And where does the pollutant come from?
[INAUDIBLE]
So it's generated inside the machine.
The reason is these machines use very high voltages.
Xerographic processes use pretty high voltages.
So if you apply a very high voltage, you actually generate ozone through a-- it's an electrochemical reaction that happens inside the machine.
Nobody used to care about this until people working in these print shops started to feel symptoms and started to complain.
So in the US, it's called OSHA.
That's the Occupation Safety and Health Administration.
They came in, and they took some measurements.
And wow, the ozone level is through the roof.
It's multiple times above the allowable maximum limit.
And the reason is because of this kind of xerographic process.
So now, these machines have to control the ozone.
And essentially, this box here, the printing engine, is sealed.
It has its own air conditioning system to prevent the ozone from leaking out.
It's a great example of how occupational safety drove new requirements which then increased the complexity of the machine.
So we keep track of the ozone flows.
And then in green, we have the usual suspects-- high voltage, low voltage, DC, mechanical energy, and then heat energy.
So there's some coding that goes on here.
So in total, we have 1,033 non-empty cells here, which is about 3.7%, so about 4%.
This DSM has a density of about 4%.
And that doesn't sound very high.
But believe me, this is a very complex machine.
This DSM that you see here is version 31.
So it took 31 versions to create this DSM.
And so why did it take that long?
Because this was published as an official document capturing the architecture of this machine.
So what that means in practice is of these 84 components or subsystems, every one of those components or subsystems is owned by a different person or a different team.
And we went through a process where all of these 84 teams, for their component, had to sign off and say, yep, you've accurately captured my interfaces.
My physical, mass, information and energy interfaces have been correctly captured in this matrix.
And oh, by the way, the empty cells have also been certified.
So if there's an empty cell here, there's no question whether, well, maybe it's just somebody forgot to put the interface there.
No.
The owner of that row and column certified that there is no direct interface between this component and the other component.
That's why it took 31 versions.
But now that you have this map, you can do a lot of things with it.
You can do architectural analysis.
You can benchmark against other products.
You can do what we call technology infusion analysis, which was done.
You can say, well, what if we add this feature in the machine?
OK, well, where is that feature going to go?
What are the interfaces of that new feature?
Do we have to change the software?
Do we have to physically integrate it?
This is a fantastic, very powerful tool for showing the interfaces and managing them.
Any questions about this DSM approach?
The real way to do this is to actually build one yourself.
And you can see it's a lot of work, but once you have it, it's very powerful.
Yes, [INAUDIBLE]?
[INAUDIBLE]
So there's templates, like Excel templates.
But there's also software.
There's a website called DSMweb.org.
And I'm happy to send that out for you as a resource.
There's even yearly conferences.
There are DSM conferences where people get together from academia, from different companies-- like BMW, for example.
I'm a big fan of BMW.
I think they do a great job.
Their system engineering is very, very strong.
They use this on all their vehicles, on all their subsystems.
And so this DSMweb.org, you'll see sort of the software that's available and so forth.
There's also some commercial software available.
So yeah, please, go ahead
[INAUDIBLE]
Ah.
So let's think about this.
The 7 plus or minus 2, how does the 7 plus or minus 2 relate to the 84?
This is an 84 by 84 DSM.
So how do the two relate to each other?
[INAUDIBLE]
Right.
But what's the relationship of that rule with the 84 by 84?
[INAUDIBLE]
Yeah, so this machine might have-- if you really want every little detail, this is probably a level 4, level 5, right?
But what's shown here, this is essentially the equivalent of a level two representation.
So do you remember 7 plus or minus 2 squared is somewhere between 25 at the low end and 81.
So we're just slightly above this.
So my argument here is this is essentially a level two DSM where you decompose the system down two levels, and then you show the DSM the interfaces at this level of abstraction.
You could go level three, but now you're going to have a 500 by 500 DSM, which is almost impossible for a human to really understand and get any use out of it.
There's also been some work in hierarchical DSMs.
So for example, if you look at the feeder system-- you see in the upper left here that says "feeder"?
Well, what is the feeder?
It's all this upfront stuff that feeds your paper and your media into the print engine.
So you could take-- this is just two rows here.
You could take those and break those into much greater detail.
So you'd have-- kind of like you guys did the requirements tree, the requirements hierarchy.
You can have a hierarchy of DSMs.
I'm not showing that here, but that's the relationship.
This is essentially a level two decomposition DSM.
Good question.
MIT, any questions?
Yeah, I just have one question about the liaison diagram
Yeah?
Is there, I guess, any meaning or information that comes from the distances between-- like when you draw it, the way that you draw it?
No, there really isn't.
So the tricky thing is if it's a very complex system, like if we did the liaison diagram for the 84 by 84-- let me just go back here-- it would be pretty messy.
And so you want to sort of do the layout of the liaison diagram so you minimize the number of crossing lines, for example.
There's actually some graph visualization algorithms that try to create as clean as possible of a graph.
One of those is-- it's called a spring energy algorithm.
It basically treats these links as though they were springs.
And then depending on how far things are, there's spring energy.
And you want to minimize the total energy of the system.
And by doing that, usually, the graph looks pretty clean.
But there's no inherent meaning in how you lay it out.
Does that makes sense, Sam?
Yeah, thank you
I have a question too--
Go ahead
--about the liaison diagrams.
I was doing something similar, at least where I had each block as a functional aspect of the system.
It was for an avionics system.
And I found that I needed to, like, make power a separate thing because it was more of like a tree.
We had one power distribution system.
But the thing I wanted to bring up was the information flow.
Originally I just had one color for that, but I didn't find that too useful.
What I plan on doing is splitting it up into command and telemetry.
So is drawing information just as one entity typically done?
Because I think it's more useful to have command and telemetry, or like data and commands
So you know, that's fine.
And depending on what you're going to use it for, you can redefine the colors.
There's no global standard for how to properly do a DSM.
That's still kind of evolving.
So if you have different types-- and you saw in the Xerox example, we used letters as well for the different types of mass flows-- P for paper, T for toner.
So the way you're going to code it is really up to you.
There's no global, right one way to do it yet.
So that sounds good.
Actually, you did the right thing there
Cool, thanks
How are we doing time-wise?
[INAUDIBLE]
OK. Any other questions?
Yes?
What kind of [INAUDIBLE]??
So why are we doing this?
Yes, exactly
Well, pretty pictures.
No, I'm sort of half-joking.
Part of it is definitely a graphical, visual, powerful way to convey the whole system architecture.
If you think about it, there aren't too many ways of actually visualizing the whole system.
This is a very abstract view.
But there's a-- and then this DSM here at Xerox, it printed out as a huge, wall-sized poster.
And you put it on the wall.
And people actually go up there and say, ah, that's my subsystem.
And then they argue and so forth.
So part of it is definitely a visualization.
But the other thing you can do with it, particularly if you start putting the numbers in, you can use this for complexity quantification.
So you can quantify the complexity of your system and then compare it against a competing system or a prior generation system
[INAUDIBLE]
Correct
You could not have a macro view on it
Right.
So this is a macro view, but if you really-- then you need to put numbers for the component complexities, the interface complexities and so forth.
And there's quite a bit of research in this.
So you can then move this from a purely visual tool to becoming an analytic tool.
Yes?
[INAUDIBLE]
Yeah.
And one thing we'll talk about next is interface control documents as an important-- so in theory, in theory, every one of these interfaces, you can have a document or a digital file if you click on this.
And this is where the tools are not fully mature yet.
But you'd like to be able to click on this, and then it links you to another document.
And what is that other document?
It's the ICD, the Interface Control Document that defines that interface.
And then you have a lot of details.
Like if it's an electrical connector, how many pins, what's the function of each pin?
And that's really where things become useful, OK?
All right, let me move on.
But I did upload for you guys.
So it's really up to you if you want to do a DSM at your PDR.
It would be nice, but I don't absolutely-- if you like this, if you think it's useful, try it out for the PDR.
But it's not a must-have requirement.
But in the full-year SDM class that we teach, we're just doing that right now.
And so you can use it for reverse engineering the architecture of a software package.
You like open source software.
You just download it.
You generate a DSM, and you can start seeing what are the different modules in the code, what do they do, how do they talk to each other.
It's incredibly powerful.
But what I did do is I uploaded for you guys-- I said there's a top-down, and then there's a reverse engineering or bottom-up way of doing it.
This is a 17-step procedure, pretty detailed, a document that I uploaded under the reading section.
So if you go on the website, under the reading section, you will find-- this is like a four or five-page document that says step by step how do you do a DSM.
And again, so at a high level, then, what you get out of it is-- here, I've sort of hidden the details from what you saw before-- is the high-level architecture you can see.
So in the case of the Xerox example, we have our frontend system, the feeder system, the main marking engine.
You can see the modules.
And then you have the finishing system down here.
And so many systems and products have similar architectures.
OK, any questions before I move on.
Let's talk about ICDs, Interface Control Documents.
And in the system engineering engine from the handbook, this is right here in the middle.
This is the technical management processes.
Number 12, interface management.
It's a huge deal.
If you don't do it well, if you don't do interface management well-- and we heard some examples-- it's really trouble.
So the purpose is for controlling your development effort.
It's really important-- I just highlighted a few words here-- if you have geographically diverse technical teams.
And maintaining interface definition, once you've decomposed your system, is absolutely critical.
So let me talk briefly about the different-- so this is a little bit more like traditional system engineering where you define the interface with a document.
We've said that system engineering is moving to become more model-based.
We don't want big piles of paper anymore.
But the reality is that many projects and programs still have a lot of these documents.
Ideally, you'd just have a model, a digital model of the interface.
Then you don't need a separate document.
But if you are working with suppliers or contractors that don't have the same modeling capability as you have, these documents are critical.
So the first one, it's called IRD, Interface Requirements Document.
And this is-- do you remember when you did the requirements analysis in assignment two?
This is one of the six classes of requirements is interface requirements.
So that's the idea, is all the interface requirements are pulled together into an IRD.
It defines the functional, performance, electrical, environmental, human, physical requirements and constraints that exist at the common boundary between two or more functions or system elements.
So you haven't designed the interface yet, physically, but you've defined what the interface should be able to do, its functionality.
Then we have-- this is the most common.
You've probably heard this acronym before-- is ICDs, Interface Control Document or Interface Control Drawing.
And so that is essentially the specification of the interface.
What does the interface look like?
So if you look at a ICD of an electrical connector, you'll see the pin-out, you see the voltage levels, you'll see the purpose of each signal for each pin.
Maybe there's some pins that are unused that are reserved for future use.
That's also very clearly defined.
And the key about the ICD is it's a two-sided document.
So there's a subsystem A and a subsystem B, and they share a common interface.
And the ICD, the Interface Control Document, will define both sides of the interface as opposed to IDD, Interface Definition Document, which is only a one-sided or unilateral document.
So for example, let's say your company makes a very fancy camera or sensor to be used on a spacecraft, but you don't know who is going to use that camera, how are they going to use it.
So what you're defining is essentially your side of the interface-- physical, functional.
And then it's up to whoever your customer is to take the IDD and turn it into an ICD.
But the IDD is only one-sided, unilateral.
And the ICD is bilateral, interface definition.
Does that make sense?
OK.
So here's-- again, this is from the handbook, the system engineering handbook-- interface management.
On the input side, you start with your interface requirements, any changes to the interface that come up during the lifecycle of the project.
You go through the steps and out come your interface control documents, any requirements changes, and then interface management work products, which might be other supporting documents.
For example, a user manual or an operator manual that will actually tell you how to properly use the interface would be an example of a supporting document.
OK, any questions about that before we get to system integration?
So who's actually read an ICD or had an actual ICD in their hands?
You have, yeah?
[INAUDIBLE],, I'm sure you've had that many times.
Can you tell us what it was?
[INAUDIBLE]
Let me give you the mic.
So-- [INAUDIBLE]
So we were working on a project where a Chinese asked us to contribute a camera payload for their satellites.
And so we had to define all the electrical and data interfaces.
I was in charge of the data interfaces.
And there were design documents saying how will the interaction work between the two systems.
Let's see.
Because [INAUDIBLE] protocol, I had to tell the satellites what to do if the camera fails and how should the camera fail if there is a problem on the computer, so all the failure cases and this kind of thing
So it sounds like [INAUDIBLE]
Yeah, it's an
Good.
And was it used then, or--
[INAUDIBLE]
Great, excellent.
Any examples of an ICD or IDD or IRD on the MIT side?
Anybody have experience with this?
No
Was that you, your honor?
Yes, I personally have experience with ICDs, but no one in the room does
OK, well, why don't you tell us about yours?
Because I know you worked on this gravity gradient sensor
Yes, exactly
So maybe that example.
Yeah, why don't you tell us about that?
So each system had its own ICD.
And that was primarily because the rest of the team needed to understand the inputs and outputs and how the specific system works.
So it not only works for system integration.
It just works for the team as a whole in its understanding of the project and the particular systems
OK, good.
All right, so this is a really, really big deal in practice.
If you do something that's-- so for example, if you make a design, you modify the interface because you think, ah, it's better for me.
I'm going to change this.
And it's different from the ICD.
And then there's a failure in your system.
And then you say you're going to blame the supplier and say, your box failed.
My mission got screwed up because of your box.
And then it turns out, ah, you actually implemented an interface that's in conflict with the ICD.
The supplier will step back and say, that's not my problem.
You didn't follow the interface definition, and just changing one pin or one polarity can lead to a system failure.
So really, you have to pay a lot of attention to the details here.
OK, let's talk about system integration.
So what is it?
It's essentially the process of deliberately-- so let's say you've designed your system.
The interfaces are clear.
When you physically put together your system, you have to integrate the system.
So that means different things.
Physical assembly of the parts, that's the liaison diagram, what fits into what.
And we'll talk about sequencing in a minute.
You need to connect your different conduits and hoses.
You need to fill in various consumables-- fuel, lubricants.
An example in space flight would be if you're going to use a cryocooler, you might have to use some gases or liquids for the cyrocooler that are consumables, finite consumables.
Connecting all your electronics to the power sources, the avionics.
Wire harnesses are still used a lot.
We would love to get rid of wire harnesses, right?
They're heavy.
They're cumbersome.
They're expensive.
And we've been trying to replace wire harnesses with wireless, but it's really hard to make it work reliably.
So we still use wire harnesses quite a bit.
And then you need to upload and test your operational software that sits inside your system.
So that's a lot of work, and it has to be done very carefully.
A lot of this is done in facilities that are very clean, clean rooms, because you really need to do every step properly.
You need to document every step.
There's quality control.
And that's why it's expensive and slow.
So the next point here is the sequence in which the integration occurs may be important.
And I want to-- so the reading, the post-reading associated with this lecture, is a paper by Ben-Asher, who's a professor at Technion in Israel who I think has done some of the best work, theoretical work, on system integration.
And unfortunately, in complex systems, a lot of problems are errors.
We often discover them during system integration and testing.
So that's also a key point here.
OK, so let me-- I hope you have a chance to read this Ben-Asher paper, because it's pretty interesting.
But I'll give you sort of the summary here on one slide.
So what they looked at was, does it matter whether you do a single-stage integration, everything at once, or you do what's called incremental integration?
So the idea of incremental integration is you just integrate two components or subsystems in isolation at a time.
You check them, and then you gradually integrate as opposed to doing it all at once.
And the example in the paper is UAV, Unmanned Aerial Vehicle.
And there's not a lot of detail, but there's four major subsystems-- the engine, the flight control system, the optronic payload, and then the payload control system.
And so in one case, you would do a single-stage integration.
So here's your four subsystems, and you just integrate them all at once.
And that's shown here in this upper graph.
So this is not the graph of components.
This is the graph of integration steps.
And then you have to put the system, once it's integrated, these step seven, eight, nine, through different tests.
And then the idea is that the amount of time that it takes to do each of these integration steps is somewhat stochastic.
It's not a deterministic number.
And so this upper graph here that looks kind of like a ski jump, this is time, number of days, versus the probability.
So what you see here is that it's right-skewed-- or left-skewed, depending on how you define it.
But the peak here is to the right.
And so the idea is that if everything goes super smoothly, actually, single-stage integration could work very well.
But the likelihood of that happening is not high.
And if you think about it, if you do all at once, single-stage integration, troubleshooting becomes more complex as well.
And so that's why this curve is actually shifted to the right.
Incremental integration, in this case, you're just integrating two and two, two and two, and then you integrate your pre-integrated subsystems together.
So it looks more gradual.
And then here is-- again, using their model.
They have made certain assumptions about this-- you get a somewhat broader distribution of outcomes.
But the mean, the expected amount of time for integration, is actually less.
So you have a faster integration time on average with incremental approach.
But in the worst case, you may be a little bit longer than the single-stage integration.
But what really matters is, what's the average?
So the conclusion in the paper is that the integration sequence is important, that the integration sequence can be optimized, that there's advantages of doing incremental integration, primarily from a risk perspective.
So again, Octanus 1 project, maybe you guys haven't thought about this too much yet.
Maybe you have.
But when you do your final vehicle, the sequence in which everything gets integrated and tested is important.
That's really what this says.
And an incremental approach can be very helpful.
The more components you have, the more the combinatorial space of integration sequences gets bigger and bigger.
So you have to do this at some level of abstraction.
Is that clear?
[INAUDIBLE]
Yeah.
And so the way you describe-- this is driven by your supply chain.
When are things becoming available?
But you have to be careful because it could be sort of risk-driven as well.
So OK, good.
Any questions at MIT?
Any comments about this?
I have a question
Yup
So I guess the takeaway is that incremental integration is usually preferred for most things, on average?
Yeah, I'm not sure.
Read the paper.
I think the conclusion is that for new systems that you've never integrated before, where your level of knowledge and the TRL levels-- or you've never done this before, first of a kind of integration, that incremental is better because if there's a problem, then troubleshooting is easier.
Now, if you're integrating a very well known system-- so it's sort of the 10th system, but you've sort of figured out where the bugs are-- so the risk level, the uncertainty is low.
Maybe then, single-stage integration is OK.
So it really depends on the novelty and risk.
So I don't think the conclusion of the paper is single-stage integration is bad, always.
I think it's that you should think carefully about the sequencing, and it's driven by the amount of risk
OK, thanks
All right, good.
So we're almost at the end here.
I want to ask you a quick question.
This is our concept question for today.
And this is about interface standards.
So over time in different industries, rather than reinventing the wheel and coming up with custom interfaces for every project you do, you say, do the interface according to this standard or that standard.
So I'm just curious to know from you guys-- and I asked this question last week from another class I teach at MIT.
So please answer this, and then I'm going to show your answers and the answers of the other class combined.
So the interface definitions here are 802.11, [?
MIL-STD-1553, ?]
RS-232, BB_aJ23100.
And then if you know of any other interface standards that you think are important, just click Other.
This is a multi-- there's not just one right answer here.
This is a multi-checkbox question.
All right, so here's the responses.
So 56%-- so this is your results, and then my class from last week, kind of mixed together.
So 56% know the 802.11.
So what is-- I thought it might be more than 56%, but what is 802.11?
Wi-Fi
It's the Wi-Fi standard, right?
And now, G I think is the latest, I think.
It's sort of ABC, 802.11 ABC.
So it's kind of evolving standard-- very important, obviously.
[?
MIL-STD-1553, ?]
24% know about that one.
Anybody here, any [INAUDIBLE],, [?
MIL-STD-1553?
?]
So you've done rural engineering, I can tell.
So what is the 1553?
It's [INAUDIBLE]
Yes, it's a DoD.
So that's what's a military standard.
It's essentially a data bus.
So for example, on military planes, on many satellites, it's essentially the architecture of the avionics.
It's a bus architecture.
So you typically have like a mission computer that's the brain, and then it directs how the messages are sent from the different-- like the flight control system or the payload.
It's kind of an old standard, but it's a very robust standard for avionics architectures.
OK, at MIT, RS-232?
This is also kind of an older standard.
Anybody chose that?
It's a--
Did anybody-- go ahead
--communication protocol for electronics
Right, but it's a very specific one.
There's a keyword I'm looking for
Serial
Serial, right.
It's a serial interface, as opposed to a parallel interface.
So it's a serial interface.
And still, it's kind of old-fashioned, but it's still used.
All right, now comes BB_aJ23100.
And nobody, out of 119, knew what that is.
So-- and this is kind of a trick question-- this is a BioBricks interface.
So are you familiar with synthetic biology?
So synthetic biology, the basic idea is that you can do what we do here for biological systems.
So there's libraries of parts, standard library of biological parts, like different proteins, that you can actually assemble organisms.
There's an annual competition.
It's been going on for almost a decade now.
It's kind of scary, actually.
But so far, nothing really bad has happened, I guess.
But the idea is that in biology, it's going this way.
So you can look this up.
You can google for this particular reference, and it'll guide you to essentially a protein interface.
It's essentially a molecular, biological interface.
So the prediction here is that 10, 20, 30 years from now, people will know what this is.
And there will be a very internationally recognized set of biological interfaces that people actually use and know.
And then anybody chose Other here?
Anybody choose other?
Yeah, you did?
Like standard USB and HDMI?
And USB has evolved as well, right?
We had USB 1.0, and then USB 2.0.
Isn't that the latest still?
USB-
C,
It's reversible
Yeah, so standards evolve as well, right, as the technology behind the standards-- you know.
So this is a huge deal in practice.
Anybody at MIT chose Other, other examples?
Yeah, I chose Other.
Things that come to mind are like SPI and I2C, more like serial electronic communication protocols that are useful for talking to chips
OK, good.
So those are sort of inside the IC world, right?
OK, excellent.
So all right, well, we are actually at the end of today's lecture.
So I just want to summarize the key points.
Why is interface management important?
There are many reasons, but I think the two top reasons are-- think of interfaces as a potential Achilles heel.
They're potentially seeds for failures.
We saw bottlenecks.
We saw structural failures.
We saw the wrong kind of information being passed through software.
And so be careful.
And if there's nobody in charge in your project to really look after the interfaces, that's a problem.
So you want to really think about this because almost any project today, you're decomposing it, and you're not doing everything in-house yourself.
You're going to deal with suppliers.
You're going to deal with partners.
So of course you have an organizational, contractual interface with them, but there's a physical interface as well.
The longer you wait to define that interface with your partners and suppliers, the fuzzier it is, the more problems you will have.
So critical to really define the interface, and then stick with the interface very sharply.
Interface management has different aspects to it.
So first of all, types of interfaces, there are many, many dozens or hundreds of types of interfaces, but they can all be boiled down to the four canonical types-- physical connection, mass, energy, and information flows.
I've introduced you to the DSM, the Design Structure Matrix method.
And that's where you basically decompose your system, and you show it as an n by n matrix.
And then you put all the interfaces into that matrix that you know about, not just the interfaces that you want to have, but the interfaces that are actually there.
So if there's noise, or vibration, or EMI, Electromagnetic Interference, or waste heat, those are also important because you're going to have to deal with them in some way.
Finally, ICDs.
So ICDS-- and we said there's IRDs, Interface Requirements Documents, which are the requirements for the interface; IDDs, one-sided; and then ICDs are the most important.
It's the two-sided interface definition.
That's the NASA approach, but [?
ISA ?]
uses that.
Many, many industries now use these interface control documents.
Ideally, in the long-term, we want to get rid of them.
We don't want documents in system engineering.
We just want to have models that we share.
But it's still moving in that direction.
OK, and then finally, the point about system integration-- do not just do it randomly.
Think about, how will you assemble the system physically?
How are you going to connect all your electronics, the consumables?
How will you load your software?
How will you test it?
We'll talk about testing, verification and validation next week, but the way in which you integrate, the sequence of integration could be very important.
And it's a pretty active area of research, so in sort of system engineering research, is really understanding and modeling system integration as something you can optimize.
And then the last point was industry standards.
Really understand what are the key standards in your industry, what are the key interface standards.
And if you're going to deviate, if you're going to choose something else than an industry standard, you better know why you're doing it.
Because the minute you decide not to use an industry standard, you've probably just inflated the cost by a factor of 10 because now there's no longer any off-the-shelf components.
You have to basically do everything yourself, but it's possible that there's good reasons for doing that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK so let's start on the material.
We're following the V-model and the empty boxes are getting fewer and fewer every week here.
So we're on the right side moving up toward lifecycle management and operations.
And today's topic is via V&V, verification validation.
And I also want to discuss the FRR as one of the key milestones, and that's the flight readiness review.
If you're not building something that flies, this is your launch to market, right launch to market review.
Are you ready to launch your project or product to market?
The outline is-- we have quite a few things to cover.
So first of all, I want to drill into verification and validation.
What's the difference?
What is their role?
What's their position in the lifecycle?
Then we'll spend quite a bit of time on the issue of testing.
What kind of testing is done?
Why is it done?
We'll talk about aircraft testing, flight testing, we'll talk about spacecraft testing, but also some caveats.
Testing is not always the-- it's not free of challenges and difficulties.
Then I want to talk about technical risk management, which is often covered in classes on project management, but I think it's essential here as well as a system engineer to have a good grasp on technical risk management.
So I'll cover the risk matrix, the Iron Triangle, cost, schedule, scope, and risk, and then I added a small section on system safety, which is a very important topic as well.
And we'll finish up with discussing the FRR, the flight readiness review.
So the readings related to this lecture are sections 5.3 and 5.4 in the handbook, the System Engineering Handbook, and then there's a couple of appendices there, Appendix E And I that are very, very helpful.
And at least one of these I'll mention in the lecture.
Plus one of the papers, and this is a paper about a decade old-- and I know a lot of work has been done on this since then, but this is a paper by Professor Leveson.
Nancy Leveson was a colleague of mine here at MIT who's really an expert in systems safety.
So the system safety model that she's developed is the subject of that paper.
OK, so let's talk about V&V, verification and validation, and how they fit together.
You've seen this diagram before, but I just want to talk through it in some detail again.
So the idea is you start your project, you're undertaking, in the upper left.
You do your stakeholder analysis.
Who are the stakeholders, the customers, beneficiaries, the regulators, the suppliers, the partners on the project?
So you really have to do a good job doing your stakeholder analysis then in order to write your requirements.
And I do have to say I've been very pleased, especially with your assignment A-2.
You really dug into those 47 requirements for canned set.
You grouped them, you scrubbed them, you did a great job.
And the idea is that for each of these requirements, you also have target values.
There are certain thresholds or target values that have to be achieved.
And then you actually do the development.
You do the conceptual design, the detailed design, and that's written here as functional deployment.
In other words, especially for the functional and the performance requirements, how will you actually implement those, embody those, in technology, hardware, software, and so forth.
So that's your intended function, your concept, and then you're implemented design solution.
Now the question is, do you actually satisfy A, the requirements, and do you satisfy your stakeholders?
And it is possible that you satisfy the requirements, but not the stakeholders.
So the way to think about this is we're going to close the loop.
In fact, we're going to close two loops.
An inner loop-- and I put testing here.
Testing is really one of the ways to verify whether you meet requirements.
There's other ways too, but testing is often the most important.
So we close this inner loop.
So what we ask is-- we test our implemented solution, our implemented design, and ask the question, did we deliver?
Did we actually satisfy the requirements as they were written?
Do we satisfy the requirements as they were written?
And are these attainable?
Were these requirements attainable?
So this loop here, this inner loop, is the verification loop.
You verify whether you design as implemented satisfies the requirements as written.
That's what verification is.
And then there's an outer loop where you take your implemented design solution, and you essentially bring it all the way back to the stakeholders.
And usually that also means you're employing it in a realistic environment, like in the environment that the stakeholders will actually use the system, not in a pristine lab environment.
And you have the stakeholders try out your system and see whether they're satisfied, whether this meets their original intent.
You remember the CONOPS, concept of operations?
Can you actually do the CONOPS the way you had envisioned it in a realistic environment?
And that's what we call validation.
And that's the outer loop.
You see the difference?
So a lot of people who don't know system engineering who have never been exposed to this when they hear verification and validation, they think it's basically two different words for the same thing.
It is it is different, it's not the same thing.
And then if you successfully verify and validate, you end the SE process and you deliver, which is good.
So this is something I pulled out from the handbook, which is the differences between verification and validation.
And I'm just summarizing this here.
So one way to ask is, was the end product realized right?
Meaning, did you do the right thing?
Or did you implement it correctly?
So verification is often done during development, so you verify components, subsystems, you check if the requirements are met.
Typically, verification is done in a laboratory environment, or on a test stand, or some environment that allows you to very carefully control the test conditions.
And verification tends to be component and subsystem centric.
OK, and then validation is the question was the right end product realized?
Did you actually build the right thing?
Did you deploy the right solution?
This is often done-- so validation focuses more on during or after system integration.
It's typically done in a real or simulated mission environment.
You check if your stakeholder intent is met.
And it's often done using the full-up system.
It's difficult to do validation on a subsystem basis alone.
Typically, validation implies you've got to use the whole system to do it.
Or, you basically use dummy subsystems.
You basically replace the actual subsystems you're going to have with something temporary so that you can go back to the stakeholder and give them as close to the real experience as they'll have with the actual system.
OK, so that's essentially the distinction here.
So I want to do a quick concept question on this to see whether this point, this distinction, came across.
So here's a link, SE9VV, these are all caps.
And what I'm listing here is different test activities or different type of activities.
And I'd like you to check the box here whether you think this is verification, whether this is validation, or you're not sure.
All right, testing and handling of a new car in snow conditions in Alaska.
90% of you said this is validation, and I would agree with this.
So many car companies, I think all car companies, once the vehicle has been finished essentially, the design, it doesn't go to market right away.
There's a very extensive-- usually it's at least six months of field testing of a new vehicle.
And you go to the desert where it's very sandy and hot, test your air conditioning systems right at the limit.
And then you go to really cold climates.
In Europe they go up to Sweden and Norway.
And here we tend to go up to Michigan, Minnesota.
And so the idea is that you really utilize the vehicle in a extreme environment, but that's realistic of actual operation.
So I agree with this.
Frontal crash test in the lab.
Most of you said it's verification, and I would agree with.
So those very standardized crash tests that you've seen in some of the commercials.
The vehicle is prepared, instrumented, you have the crash test dummies.
And then you have, typically, there's at least three different kinds of crash tests.
There's frontal, there's side impact-- like the T crash-- and then there's a rollover tests, that are more and more standardized.
And the test conditions in these are there they're highly stylized tests.
They're very prescribed exactly the speeds, the angles, everything is prescribed.
And real accidents, the variability of conditions is much, much bigger than in these tests.
So I agree because the test conditions are so tightly defined and constrained, this is verification, not validation.
Testing of a new toy in a kindergarten.
OK, so here you're-- and this is the toy companies, and they essentially-- before, again, they make a big million or billion dollar decision to mass manufacture toys, they will actually have kids play with them in realistic environments.
And so I agree with that this is validation.
Vehicle emission testing.
Obviously, this was the big Volkswagen scandal that we had.
So basically, what this cheating that happened is essentially software that was embedded in the vehicle, such that when the vehicle experienced exactly the test conditions of these drive cycles that are very well known, very well defined, the vehicle would internally switch or reconfigure to a verification mode and really emphasize low emissions at the expense of fuel economy.
And as soon as the vehicle would detect that it's in a more general driving conditions, it would essentially switch that mode off.
So verification, again, is on a dynamo in the lab.
Satellite vibration testing on a shake table.
We'll talk about this.
This is often-- we refer to this as shake and bake in the spacecraft business.
The spectra, the load spectra, that are put into these shaking tables are, again, very stylized and different for each launch vehicle.
So here we can debate a little bit whether-- is it closer to validation, because the actual test conditions are so much adapted to each launch vehicle.
But I agree, this is primarily a verification activity.
And then the field testing of the Google glasses.
So you basically produce an initial batch of your product and then you give it to like lead users.
You have them try it and give you feedback.
This is much closer to validation.
So I think by and large your answers here are very good.
And the real distinguishing factor is whether this activity happens in a lab, in a very controlled environment, under stylized conditions, or whether you're actually going out in the field, in a realistic mission environment with real users or real potential users that are not especially knowledgeable or not specially trained about the system.
So, very good job.
I think most of you really understand that distinction.
OK.
So let me talk briefly about the-- yes, please, Veronica?
Got it today.
OK, are there any has that really bridged the gap that really could be seen as both.
So I'm thinking about products in particular that are to be used in a lab setting where you have a very specific kind of user, where meeting the requirements is more about how the tool is employed, and I see the user in that sense is part of the system.
So I'm wondering if in a more clinical sense there's an action that is both validation and verification
Yeah, and so you said clinical.
So I think there are situations where the distinction is not as sharp, not as clear.
You said clinical, so I think in hospital, you know for medical equipment, like if you think about surgical equipment and things like this, where it's very hard to really-- it's very hard to do verification in a stylized way.
The only way to really check it is to have the equipment embedded and used in a pilot study, for example, in a hospital.
And in that case, because the human is so involved, and it's not a general consumer product, but it's really a tool for specialists, the only way to really check your requirements is to actually embed it in a realistic environment to begin with.
So whenever it's very difficult to design, very specific, isolated tests where you can check for each of these requirements one by one, you almost have to move straight into validation.
And I think in medical equipment that's often the case.
Yeah, go ahead
Would you say that for a spacecraft really true validation isn't possible?
You have to recreate the conditions in some way, in some kind of a laboratory
I think it depends on the novelty of the spacecraft.
If you're launching something like a standard communications satellite where you've launched dozens before, and you know the actual pitfalls and the operating conditions, you've experienced failure modes in the past and eradicated most of them, I do think that you can do a lot in verification.
But then I'll show you one example of a spacecraft we've actually talked about before, where there's going to be a lot of residual risk.
And the first time it's deployed, people are going to sweat, because there's still a lot of unknowns to be resolved.
OK, so let's look at the product verification process in particular.
This is from the-- so this is the product verification process from the NASA System Engineering Handbook.
So what are the inputs?
The end product to be verified.
So you have to have the artifact that you're going to verify.
The specified requirements baseline, you need this as a reference.
What are you going to check against?
The product verification plan which is essentially your test plan.
What test cases are you going to run?
How long are you going to run them?
How many repetitions will you do?
And then product verification enabling products, which would be test software, test equipment, things that are not part of the product itself, but our enabling of the verification process.
You then, essentially, go through this process, and what are the outputs?
The verified end product, the product verification results, so these would be test protocols, things like this, product verification report, and then product verification, any other work products that come out of it.
So you could have, for example, discrepancy reports.
You failed some tests.
Well, that would be an important output.
And then the question is, is this significant enough that you have to go redesign or retest?
Or is it a minor issue that you can waive essentially to move to the next stage?
So let me just give you a quick example here from my own experience about this verified end product.
One of the things on the Swiss F18 program that we did is not just by airplanes and equipment, but also models of the plane itself.
In particular, finite element models, very detailed finite element models of the structure.
And these models were very expensive.
Like, some of these models were millions of dollars.
And so I got a phone call, I was a liaison engineer at the time in St. Louis.
I got a phone call from Switzerland saying, this is crazy.
How can we be charged millions of dollars for this particular set of models?
And I said, yeah, that seems pretty expensive, so I'm going to go negotiate this.
And so I started negotiating, and I guess either I'm a bad negotiator or it was really, really clear why these were so expensive.
The reason these models were so expensive because they were on the right side not on the left side.
So every one of these models that we were purchasing had been verified using actual physical tests.
So every location was guaranteed under the load conditions to produce a stress and strain prediction at that location that was guaranteed to be correct within plus or minus 5%.
So the model had been very carefully calibrated and tuned against physical reality as opposed to a finite element model that's just anybody can make a model and put some load cases and boundary conditions on.
And you don't know how closely does this mean.
So there's a huge difference in value between a product, a model that has gone through verification where at the end of it there's actually a report, there's a protocol, there are data that says all these features, all these requirements that you had against it have actually been checked.
This is a certified product.
And that's the main reason for the price, because the actual process of verification is very, very resource intensive.
So even though when you look at it physically, you might say, I can't tell the difference between pre-verification and post-verification because physically it's the same.
But in actuality, there's a huge difference because once it's been verified and certified against a set of requirements, it's a much more valuable asset.
Does that make sense?
So keep that in mind when you think about these products on the right side.
Now what are the types of verification?
So tests we'll talk about, so you're physically testing.
But there's other ways to do it through analysis, through demonstration, and through inspection.
So analysis essentially means you're doing a calculation with a mathematical calculation or a simulation that satisfies you that this requirement is met.
And you're doing this with the input parameters into the simulation are as accurate as possible based on the physical reality of the system you have.
But for whatever reason, either because you don't have the funds for it, or you can't simulate the operating conditions well enough, you have to do it through analysis.
Demonstration essentially means you're operating the system, you're demonstrating the functions that you need, but you don't necessarily have a lot of instrumentation on the system.
And you don't certainly do destructive testing.
In other words, a demonstration simply means you're operating the system as intended and demonstrating physically that it performs its purpose.
Inspection essentially means you are physically inspecting the artifact either visually inspecting-- there's also a lot of techniques called NDI, nondestructive inspection through with X-rays or eddy current sensors.
You're checking for the lack of manufacturing flaws or [INAUDIBLE], whatever it is.
But inspection essentially is you're not physically operating the system, but you're inspecting the artifact to make sure that it satisfies a certain set of requirements.
And then testing typically means that you're putting a stimulus into the system.
You're operating the system under some test conditions.
You're recording data, which you then analyze in terms of comparing that to your prediction or expected behaviors.
So these are analysis, demonstration, inspection, and tests.
They are all different ways of verification.
Yeah?
How do I know when I'm supposed to use more than one type at the same time?
I mean in and or or
Yeah, that's a good point.
There's no real general rule of this, but in general, I would say the more crucial, the more critical a particular requirement is to the operation of the system, the more intense the verification will be.
Whether that's just using one of these types, you know you just run more tests or more different tests, or you doing a combination of inspection and testing.
There's no there's no general rule in terms of two out of three, or two out of four, but the purpose of the V&V plan, the verification and validation plan is-- and you did a little bit of this in A2.
You did a little bit of thinking into how would we actually verify this requirement.
The purpose of a V&V plan is to say for each requirement which of these four methods are we going to use for verification, and then actually write down each test that you're going to perform, what kind of equipment you'll use, what kind of test conditions.
It's a lot of work.
In fact, I think it's fair to say that the people that do this kind of work, verification and validation, are typically different people than the people that do the writing of the requirements or that do the actual design work.
This is a pretty specialized activity and the people are a little different.
If you've met people who would do testing or quality inspection, they're quite different.
It's a different mind set up.
Go ahead
According to what's happening in ESA, actually this ADIT will be imposed in the specification prior to your proposal.
And it will give you to a minimum requirements to just test against.
And they will give you a rough matrix for every requirement line, whether it's [?
ABIT, ?]
and then you have to answer with a validation intense plan, usually, unless your agency and you are defining the specification and you have to do it, and it's mostly based on experience.
And it's some people that have really lots of knowledge that then make these specifications.
But I think for all of you engineers here in the next 10 years, you'll be just hoping that not so many of these ADITS in the specs you will get, because you will have to answer as part of the specification actually
Yeah, and what--
I'll just demonstrate it
Well, and the point you're making, Voelcker, is that this is a contractual requirement.
This is not optional
[INAUDIBLE]
Yeah
It's not optional and it has to be followed, the pricing, right in front of [INAUDIBLE]
Yes, very good points.
So the outputs of all of this are discrepancy reports, if there's any discrepancy reports, waivers, the verified product itself, and then the compliance documentation, which is essentially your test protocols, et cetera, et cetera.
And as you can imagine now if you had 47 requirements with can set, and then by the end-- [?
Uonna ?
],, what would you say the number of requirements that we ended up with at the end in A2 were closer to 100, right?
Most people were around 80, 90.
OK, now imagine-- the good thing is if you can do tests that actually check multiple requirements at once, that's a good thing.
If you can do tests that help you verify multiple requirements through the same tests, you can save some money.
But the whole testing strategy, the contractual requirements that Volcker was mentioning, it's a big, big, big deal.
It's a really critical part of system engineering.
OK.
So in terms of the lifecycle phases where this fits in, most of this activity happens during phase D. So you remember this was the NASA lifecycle model, and so phase D is system assembly, integration, test, and launch.
So much of the testing that we talked about happens during phase D. So the system has been fully designed, it's been assembled, it's been integrated the way we talked about last week, and now you're really putting this system through its paces.
And so this phase D is intense, it's expensive.
And if something goes wrong, it sends you back to the drawing board often.
And you have to do-- you have to figure out whether a failed test, a failed verification, is a showstopper.
If it is, then you have to redesign the system, you have to retest.
But if it's a minor thing, then you might be able to request a waiver and you can say, OK, we didn't achieve this requirement, or we failed this test, but we think it's a minor issue.
And instead of holding up the program, we're going to get a waiver for it, which means you get an exemption essentially, and you can move on.
And whether or not a waiver can or cannot be granted is a big deal and that goes under risk management, which we'll talk about in a few minutes.
Yes?
I had a question on-- so if you're a system integrator, and you created statements of work for other people to procure a large optic, or something, they go and build that [INAUDIBLE] requirements, and then they do all the verification and testing, and then they provide all that documentation.
And then like from my experience with anything like procurement out stuff, that comes back in your in house, and you go to assemble it, and you essentially do a lot of that verification and testing again to double check that supplier.
There is a little bit of a conflict of interest, obviously, with that supplier doing the work and also verifying their own work.
Is there any good way to get around that?
So it seems like a very expensive process
So my experience-- basically, you're talking about separation of powers.
The good suppliers, they will have internally separation of powers.
In other words, the people that are doing the testing and the Q&A, they're usually people who really enjoy finding mistakes and faults.
And that's why when I'm saying-- I'm trying to be diplomatic when I said it, but people even in software, people who do software verification, they love to find bugs.
They love to find problems, because that's their job.
And so the good suppliers, it's in their own self-interest not to do shortcuts.
Now if you don't trust that, and you do all your same testing and Q&A again, that's a duplication of effort.
The way I've seen that done effectively is that you, as a customer, say you're going to buy the subsystem or the engine for, example, from it.
What you do is you send liasion people, you send representatives, who are knowledgeable people to the supplier while the testing is being done.
And so they're present when the testing has been done.
They're very involved with it.
And therefore, you don't have to do it twice.
So there are ways around this.
OK, so what I did here is just search for the word test and the list of milestones.
And where does it come up?
So the first time it really comes up in a major fashion is at the CDR.
OK, so let me just read this to you.
So the CDR demonstrates the maturity of the design and is appropriate to support proceeding with full scale fabrication, assembly, integration, and tests.
So in other words, even at the CDR you're blessing the final design, you should say something at the CDR about how the testing will be done.
In fact, test planning often is way before the CDR.
But at the CDR, you should you should really talk about the testing.
Then we have so-called TRR, which is a test readiness review.
And so for each major test, you would have a separate TRR.
The TRR ensures that the test article, hardware software, the facilities, the support personnel, the test procedures are ready for testing data acquisition reduction, meaning data post-processing and control.
And then at the system acceptance review, the SAR, that's when you essentially transfer the ownership of the asset.
And that's at the SAR, at the system acceptance review, that you're going to review, not just the product and its documentation, but all the test data, the analyses that support verification.
So at CDR, you say this is the testing we'll do.
At the test review itself, you say everything is ready for the tests to happen, and then you do them.
And at the system requirements review you look backwards and you say, what tests actually happen?
What's the documentation?
What were the results?
Are we ready to own the asset now?
Does that does that make sense?
OK, so with that in mind, let's talk about testing itself.
What kind of testing there are, and so testing is one of the four methods of verification.
And it's the one that we often spend the most money on.
So this is from the handbook, section 5.3.
This is basically an alphabetic list of the testing that we typically do.
And I'll just highlight a few here, and then I have a group exercise for you.
So aerodynamic testing, burn-in testing, which is often done with electronics.
Make sure that you use you burn-in your electronics, you get them running at the right conditions, drop testing, pressure testing, pressure limits, thermal testing, G-loading, human factors testing, thermal testing, manufacturing random defects-- that's when you do nondestructive inspection-- thermal cycling, vibration testing, and so forth.
So this is 20 or 30 types of testing.
And then within each there's even subtypes, so there's a lot of different-- and there's a whole industry actually that is primarily focused on providing test equipment, sensors, data logging equipment.
It's a big industry not just in aerospace, but throughout.
OK, so I'd like to do a little turn to your partner exercise.
And the question is I want to ask you what kind of testing have you been involved in the past.
And if this was like in product design, product development, that's fine.
If it was for an internship, but even at the University itself, if you did some experimental work and experimental testing as part of research, that's fine too.
You can talk about that too.
So what kind of testing have you been involved in the past?
What was the purpose of the testing?
What were the challenges?
What went well?
What were the results?
Maybe if it didn't go well, talk about that too.
All right, good.
So let's see, we're going to go back and forth.
So who wants to start here at MIT?
Who has a good story to tell?
Go ahead
So I worked on the ground station side of the Lunar Laser Communication Demonstration program that recently flew.
So I was involved in assembling an integration, but also doing verification testing in the lab and at our field site, but then we did validation when we be moved out to the field site in New Mexico.
So I was involved in the whole process and it was neat to see.
And we had to go into the clean room a few times to adjust the optics, because we saw that there weren't meeting requirements
The reflector that was left by the astronauts, are you using the reflector that was left on the surface?
No, this was-- we were using just like [INAUDIBLE] like optical alignment stuff in the lab.
And then when we were out at the field site, we were utilizing guidestars to align optics
So what was-- what went well?
Was there a big difference between indoor and outdoor?
What surprised you in these tests?
Yeah, so once you can do the alignment of the individual telescopes which were 20 inches in diameter, you can do that well in the laboratory, but then every time you assemble and disassemble the system, you change the alignment of them relative to each other.
So there was a lot of attention paid to making sure that we could replicate the alignment to a certain extent.
So that was very difficult in a laboratory setting to get done, but once we did that, we were able to have fair confidence that when we were out in the field that we could match that
Very good, very good.
What about EPFL?
Well, I did an internship in an aluminum-roll product factory.
So basically, I was doing natural science there.
And there was a whole bunch of tests to do.
And well, all the tests was about heat treatments and different tempering.
And actually, the alloy that was already produced in the factory was not the best of what we can have of it.
And it [?
applied ?]
to change the [INAUDIBLE] with the heat treatment for a few seconds, naturally, on the line of production.
Adding this amount of time was totally critical because it was continuous.
And the rolled aluminum, if it spends a bit more time in the oven, it would melt.
And that's a bit like for the Swiss plane, actually.
Like I discussed with my boss, saying that we should maybe change the original power meter.
But at the end, it was really critical to change something on the line because it could have cost a lot, like in the modification of the oven or the general machine
So were those tests successful?
Were these heat-treatment changes eventually implemented on the line?
Or did the tests reveal that it would be too difficult?
Unfortunately, I don't know because I finished my internship before
OK. Well, you should find out whether it worked out in the end.
Very good.
Back to MIT, any other examples people want to mention?
Test experiences?
Yes please, go ahead
We bought the CASA-295.
It's a small cargo aircraft.
And we get we got involved in the development of its simulator.
It was pretty different, the simulator.
Because as the aircraft has no fly-by wiring, it is pretty light.
So lots of hydraulics to de-motion.
And they brought the flight model from the factory.
And we applied the flight model to the simulator.
But it was not real enough.
So we had to go for flying, like 60 test flying points.
And we have to go back to the simulator to apply these points to tailor the simulator to meet reality
I see.
So the purpose of this testing-- because the plane itself had already been certified, it sounds like
Yes
It's Spanish, right?
Spanish airplane?
Yes
You tested it specifically to get flight dynamics and other data to then tune the simulator to be more reflective of reality
Exactly.
Because the flight model from the factory was not close to reality at all,
Very, very cool, very interesting.
So different purpose, of not testing for certification of the first airplane, because it had already been certified, but to get the simulator to be matching more closely
It was development for the simulator.
Because it was sold afterwards as a type delta simulator.
So it was the development of the simulator
OK. Great, thank you for that example.
This all sounds pretty good.
Anybody involved in test failures?
You know, things that didn't go well?
Yes, [?
Narik?
?]
Well, it was an interesting experience.
We were designing a wind turbine that we were 3-D printing in undergrad.
And we had certain requirements on the wind turbine.
And we were supposed to test in the wind tunnel afterwards.
What happened was that the wind turbine matched our performance prediction fairly closely.
But the generator and the electrical power system that the test operators consisted of wasn't designed to handle the current that we were outputting.
So we caused a small fire
OK. [LAUGHS] So this was the test equipment itself?
The test equipment
Not the artifact you were testing failed--
Yeah
--but the test equipment around it, because overload
The interesting point was that we had no control over the test equipment.
It was managed by the university.
So within the requirements that they gave us, the power output possible didn't match what they had
OK, great.
Great example.
So the test equipment and the test artifact need to be matched to the test conditions.
Excellent, good.
I do hope that you those of you that have not had a lot of test experience, that you get to experience it.
It's a lot of work, slow, tedious.
But in many cases, despite modeling and simulation, there's still a big role to play for actual testing.
OK, so let's talk about aircraft testing.
Typically we distinguish between ground testing and flight testing.
Weights and balance, I had some experience with this on the F-18 program.
You think this is the most trivial testing there could possibly be, you just put an airplane on a scale and that's it.
Well, it turns out it's actually more involved than you think.
First of all, airplanes are very big.
They're heavy, multi-tons.
And typically it's not just one scale.
You have several scales you put on the landing gear.
So the scales need to be properly calibrated.
If you have differences in calibration of the different scales, you have an issue.
You need to determine the mass.
Not just the mass, but the CG.
And then the most difficult thing to experimentally determine, at least in a 1G field, is the inertia matrix, if you need to experimentally get the inertia matrix.
Do you remember your Ixx, Ixy, Iyy?
The inertia matrix is tricky because you typically then have to suspend the airplane.
And just the presence of the cables and the suspension will pollute the real inertia matrix.
And you have to subtract out the effect of the suspension system.
So something that seems super trivial, weights and balances-- you just stand on the scale in the morning, there it is-- is actually very tricky.
And there are people, that's all they do.
They do weights and balance testing for spacecraft, aircraft.
And it's basically a science.
Engine testing, I'll show you some pictures.
This is done in what's called the Hush House.
So Hush House is heavily insulated.
You run an engine through all of its test conditions, its operational conditions.
And then you integrate it into the airplane and you run it outdoors.
Fatigue testing, this has been a big issue on the Swiss F-18.
But in general, making sure that the airplane can satisfy all the static and dynamic structural load conditions.
Avionics checkout, this is very, very involved.
As we get more and more displays, mission-control computers, all of the avionic suite needs to be checked out.
Essentially every function, every button, every menu item needs to be tested.
And the tricky thing is interactions among different pieces of avionics.
So you can't just test each box in isolation.
You also have to look at the interactions of different pieces of avionics, the flight control software, or the flight software that is loaded in each of these avionics boxes needs to be in the right configuration.
It's a very big combinatorial challenge to do avionics checkout these days.
And then finally, pre-flight testing.
So this is everything you can do on the ground, run the engines, taxi with the airplanes, basically turn all the equipment on, turn it off, do the cycling.
You could do a lot of testing before you actually fly.
Flight testing itself falls into different categories.
So flight performance testing, rate of climb, range, can you meet each point in your prescribed performance envelope?
Stability and control, this is where test pilots typically earn their living putting airplanes into stall conditions, recovering from stalls, trimming.
Flutter testing is a big deal.
So flutter is a phenomenon whereby at high speeds you have a coupling between the structural deformations of the airplane and the actual excitation of, for example, the wings.
Flutter can be very dangerous.
If you hit a resonance at high speed you can actually destroy the airplane because of an instability.
So flutter testing is also very, very interesting and very tricky.
And then finally, this is primarily for military airplanes, weapons testing, both guns, missiles, bombs, live fire testing-- sometimes also using airplanes that are towed-- simulated targets, and then LO stands for a Low Observability.
So this is essentially all the new generation of military airplanes have measures to reduce their radar signature, or even make them invisible or quasi-invisible to radar.
And you know, a lot of this stuff is classified.
But actually checking that an airplane is invisible on radar or has truly low observability, there's a lot of testing involved in that.
And that's also quite expensive and very involved.
So let me show you just some pictures that I've collected over the years.
This is a wind tunnel test model.
This is a model that was developed as part of the F-18 program.
This is about 1995, vintage.
This model, it's a subsonic wind tunnel model.
And you can see in yellow, you have all these probes and radomes and things like this.
So it's basically to check whether any modifications you make to the airplane will affect its performance in airflow.
This is for wind tunnel testing.
This model, by the way, just building this model is about half a million dollars.
It's very accurate.
It's very precise.
This is a picture-- yes?
Go ahead?
Is that model full scale or half scale?
No it's, I want to say, like 1/8 scale, something like this.
Yeah.
OK, here's the Hush House that I was talking about.
This is in St. Louis.
So you can see that the airplane is not painted yet.
And only one engine at a time.
So the engine is being, in this case, with full afterburner, you can see the airplane itself is secured with these chokes here.
And there's load cells in these chokes.
So as you fire up the engine, you can measure the thrust by the load cells that are attached to these chokes here.
You also see that there's these cables running in and out of the airplane.
So all the sensors, everything, and the engine is put through its full different operating profiles.
And a lot of sensor data is recorded to make sure that the engine responds appropriately, it has the right thrust for the right throttle setting, the fuel consumption, all the temperatures in the engine, that everything is nominal, essentially.
Live Fire Testing, this is a Maverick missile being fired, an air-to-ground missile.
As you can imagine, there are special ranges and test sites for doing this kind of work.
So in the US, one of the most well known as China Lake, out in California.
You have to reserve months and sometimes years ahead.
So if you want to do like a live fire test campaign, you have to reserve the range at least 18 months to 2 years ahead of time.
Because a lot of other services, a lot of other programs are using the same facilities.
In Europe, it's a little harder.
Definitely in Switzerland, because the country is so small and dense and highly populated, you can't test live missiles in Switzerland.
You can do guns, air to ground.
But in order to do missile testing, typically that's done here in the US, or in a more limited fashion, in Scandinavia, like in Sweden, in northern Sweden, there are some test ranges up there.
This is the most expensive kind of testing you can do.
So a single test like the one shown here, a single test like this will probably cost several million dollars.
Not just the airplane and the weapon itself, but all the test procedures, the protocols, airplanes that observe it from all kinds of angles.
It's very, very involved.
And because it's so expensive, you will typically only do it for something new that you haven't done before.
Either a new weapon, or a new weapon integrated on a new platform and so forth.
And obviously, it's very interesting.
But it's very involved.
Yes?
Are they just testing accuracy, Or that the two things work together?
What are they looking for, really?
The first thing you look for is does the weapon fire?
So do you have all the electronics?
All the signals?
The wire bundles?
Did you get it right?
Is there an end-to-end functionality?
That's number one.
Number two, safety.
Does the weapon separate properly from the aircraft?
The worst thing that can happen to you is if you release the weapon and it collides with the airplane.
And so you can see, the various angles and release conditions are very tightly prescribed.
And so there's a separation, has to be proper.
And then the third, of course, is accuracy.
So within each of these tests, there are multiple sub-objectives that you would test for.
But safety always comes first.
OK, any questions?
This was a little bit military-aviation heavy.
If you're testing, whether it's a CASA airplane or a new Airbus or Boeing commercial airplane, many, many months of testing.
They actually fly the routes.
You'll fly New York to Singapore, to London.
You would actually fly the real routes.
You would record fuel consumption, a lot of parameters.
Some of this testing is not very exciting.
It's many, many, many, many hours in the air.
But the key is that you have a lot of instruments and sensors during these tests that you may not have during regular flight operations, to really make sure there's no surprises.
The airplane flies at least as good as the requirements that you promised your customers.
And even then, when you think about what happened to the Dreamliner, the 787 had a lot of battery problems.
Because they used a lot of lithium ion batteries.
There were overheating issues.
Some of these problems didn't show up in testing.
They only showed up in early operations, once you had a fleet going.
So it's not a guarantee because you're doing a lot of testing that you're going to catch all the problems.
But you want to catch as many as you can.
Yes?
So my question was about the risk posture for larger airliners, for Boeing, the Airbuses.
So for military aircraft, there is an escape method for the pilot.
But for these larger aircraft, how much analysis do they do before they decide to go ahead and put a person inside?
They do fly by wire beforehand?
Is that possible.
for such large aircraft?
So that's where the ground testing, pre-flight testing becomes very important.
So you basically taxi for many, many hours.
All the flight control surfaces, all the engine, you have the Hush House testing.
So you essentially try to do as much as you can on the ground before you do the maiden flight.
All right, let's move to spacecraft.
And it's kind of a similar thing.
You can distinguish the ground testing versus on-orbit testing.
So the ground testing is really not that different, weights and balance.
The biggest thing is if your satellite is heavier than the launch capability of the launcher, you have a real problem.
So the mass constraint is even tighter in spacecraft.
Then, a lot of testing on antenna and communications.
This is typically done in anechoic chambers in the near field, and then later in the far field.
Vibration testing, that's the shake part.
Thermal and vacuum-chamber testing, that's the bake part.
And then you also have pre-launch testing, so off-pad and on-pad.
Off-pad testing is the satellite or the spacecraft has already been shipped to the launch site, and it's hooked up.
Like a patient in the hospital, it's hooked up to a lot of cables and power and cooling and so forth.
And then when it's on the pad, it's pretty limited.
So on the pad means the satellite or the spacecraft is already integrated into the launch vehicle.
It's on the launchpad.
And then that amount of testing you can do is very limited.
So that's when we say off-pad, on-pad is, is the spacecraft already been integrated on the launcher or not?
Once you launch to orbit, you got your eight minutes of terror.
And hopefully the launch goes well and the spacecraft is released into its initial target orbit.
And then you do a lot of other tests, like thruster testing.
Can you do station keeping?
Can you turn on and off the thrusters?
You deploy all of your mechanisms, your antennas, your scientific instruments, and then your communications, communication and instruments.
And we'll talk more next week, but this typically is called commissioning.
You're commissioning a spacecraft before you actually turn it over to the users.
And that commissioning phase could be anywhere from a few days to several weeks, or even a couple months.
And again, you don't want to randomly put commands into your spacecraft.
These test sequences, deployment sequences, are very, very, very carefully worked out.
Every command, the order in which you send the commands have been worked out ahead of time.
They've been simulated.
And all you want to see here is confirmation that the spacecraft behaves as planned.
Some pictures, so this is what typical spacecraft integration testing looks like.
So this is in a cleanroom environment.
You have people in bunny suits.
And the idea is to not damage the spacecraft while you're doing the testing.
This is a picture of the Clementine spacecraft.
This is a radio frequency anechoic chamber testing.
So you see these funny cones here.
These are essentially foam cones.
And the idea is to prevent multipath to prevent echoes in the test chamber, to test all of the antennas, EMI, electromagnetic interference and compatibility, charging and discharging of the spacecraft.
This is one of the failure modes, is that you have high electrostatic charges that build up on a spacecraft, create a lot large voltage potential across the spacecraft.
Some spacecraft have failed because of that.
So all of these things you want to test in a very controlled environment.
James Webb Space Telescope, I'll send you a link.
I'll send you a link through email.
There's a simulation of this on-orbit deployment.
It truly is amazing.
This spacecraft will be launched in a box, essentially.
And the deployment sequence is very carefully choreographed.
First, typically, you deploy your solar panels because you need power.
Because you're only running on battery initially.
So if your battery runs out before you've had a chance to deploy your solar panels and get fresh power into it, you're in big trouble.
So typically, solar panels first.
Then communications.
And then you start deploying the other subsystems.
So for James Webb, also very tricky, is this-- this is called the Sunshield.
It's essentially thin layers of insulation.
And the geometry is very important.
The primary mirror, the secondary mirror, all these things need to be deployed with very, very high precision.
And it's even to the point where this particular spacecraft is so lightweight that it cannot support its own weight in a 1G gravity field.
So there is no way to test, end-to-end, the full deployment sequence on Earth.
The first time it will happen is in orbit.
Now, they've tested sub sequences or scaled models.
For example, the Sunshield has actually been deployed at a smaller scale in a 1G field, but never the full thing.
So this will be kind of scary, after an $8 billion investment.
So let's hope for the best, 2018.
So testing is good, testing, testing, testing.
But testing also has its caveats.
So caveat means limitations, essentially.
So testing is critical, but it's very expensive.
Think about test rigs, test chambers, sensors, DAQ is Data Acquisition Equipment.
All this stuff is very expensive.
And if you can reuse things between different programs, that helps.
But still, how much testing should you do of components?
So one of the comments, who mentioned the vendor, the supplier?
One of you.
You talked about it.
And this is a key question.
Do you trust the parts that come from your vendors?
Or do you retest everything yourself?
Calibration of sensors and equipment, if you've done some testing and you forgot to calibrate your displacement sensors, your thrust sensors, you didn't calibrate them or they're out of calibration, that's a big problem.
That's a big problem.
So before you start your tests, make sure that all your sensors are properly calibrated, or you can get the wrong conclusions.
This is a mantra that's well-known.
Test as you fly, fly as you test.
Fundamentally, this means that the configuration of your item-- a spacecraft, aircraft, medical device-- the one that you test should be the same configuration as what you're actually going to fly.
And it's often failures occur when the test went well, but then somebody tinkered with it and modified it before it actually flew.
And that change actually caused a big problem.
So make sure that your test conditions reflect the actual operations as closely as possible.
Simulated tests, what do we mean by this?
So simulated tests use dummy components.
Maybe your full spacecraft or aircraft isn't ready yet, you don't have all the pieces.
So you can still start testing, but you have to replace the missing pieces with dummy components.
At least, they should reflect the right mass distribution.
But maybe you can do more.
Simulated operations, so the 0G versus 1G, is it representative?
And then, what's often true is that you pass all your tests and then you still have failures in practice.
And the failures often happen outside of the test scenarios that you had tested.
So you have to be ready for that.
But try to avoid that.
So here's from Appendix E. This is called a Validation Requirements Matrix.
Essentially what this is, is an organized way to organize your V&V activities, in terms of what's the activity?
What's the objective?
Which facility or lab will you do it in?
What phase?
Who's in charge?
And what are the expected results?
It's pretty straightforward.
It's just a table to organize these activities.
And then appendix I is your more formal V&V plan.
This is a suggested outline for it.
And I'll just say this.
The degree to which you take Verification and Validation seriously and the resource you make available for it are critical for success.
So how many dedicated Q&A personnel?
What is the interaction in working with suppliers?
Are you planning ahead for these tests?
How close are you getting to actual end-to-end functional testing?
Can you piggyback on existing facilities and equipment?
How well do you document all the outcomes and follow up with discrepancies?
And my last comment here is this work is often not glamorous, except for some of the very cool flight testing that I showed you.
Most of this work is really hard work.
It's very detail-oriented.
It's not glamorous.
But it's essential.
If you cut corners, you often pay the price for it.
So any comments or questions?
We'll take a short break, like a five-minute break.
But any questions about testing, verification, validation?
Yes, go ahead
Not really a question, but I wanted to say that also, flight testing is not so fancy.
I mean, many people think it is.
Well, if they actually fly, maybe it's fun.
But there are not that many.
And you need to prepare them weeks and weeks ahead.
And it's actually very, very boring.
Because you need to make sure that you don't waste time at all.
Well, at least you can waste a little bit more time in the lab.
So it's very stressing and not so fun to fly
Do have experience with this?
I wasn't flying, because you need to be certified.
But I was preparing.
And I think it's even worse than testing in the lab
Yeah, yeah.
Which airplane, or which system were you involved with?
I was in with the power plane system for [?
Airbus, ?]
particularly.
And the aircraft was an A330
A330, OK. Great.
But I think it's healthy to have this experience.
It really makes you humble.
And you also see, for the things in design, did you design well?
Did you design for testability?
Really, I highly recommend for every one of you to try to get on some kind of test campaign, at least once in your career, because it's eye opening.
So thank you for that comment.
Any comments at EPFL?
Any of the students?
[?
Voelker?
?]
Did you want to add something?
Katya?
Go ahead
Yeah, I guess the comment is, it seems like sometimes you can meet all of the requirements in the verification process.
But when you get to the validation part, for example, maybe the customer has some expectation that the range for the time of flight would have been on the maximum edge and you were on the minimum edge.
But actually, it seems like sometimes you can meet all the requirements but they're still not going to be happy.
When do you find that middle ground?
And it's going to be constant, to continue iterating this over and over again.
Do you try to involve them earlier on, during the testing process too?
How do you handle that, that difference?
Yeah, it's tricky, you know?
So that's when you do need contractual agreements in place.
You need to have the requirements, baseline.
You need to have the contractual agreements.
And hopefully, any problems that occur will not lead to some kind of legal dispute.
But sometimes that's unavoidable.
But as a designer, as a manufacturer, unless you have agreements in place and clear baseline, how do you decide in the end, is it successful or is it not successful?
And if there's problems, try to isolate these problems and say, OK here, by and large, the testing went well.
But we have like, three, four, five issues that need to be addressed.
And you can tackle these issues one by one.
But if you don't have a contract in place that's really a good contract, if you don't have a clear requirements baseline, and then if you don't have a good relationship with your customer, you're setting yourself up for big, big problems.
[?
Voelker?
?]
Go ahead
Yeah.
There is also the one big difference between commercial operators or commercial customers that are becoming more and more frequent compared to the institutional ones.
And often I remember, for some of the [?
Global ?]
[?
Store ?]
[?
Iridium ?]
series, the customer was only accepting the hardware six months after they had been commissioned in orbit.
So there you have a validation that is still your responsibility in orbit.
You can't go and fix it, but it still has to work.
And he was retaining up to 10% of the full contact value, even up to the N minus-2-tier-level suppliers, until he was satisfied it was working in orbit.
So these considerations, it's really the proof of the pudding when you have to test this up there and can't fix it.
So there's no recall of a satellite constellation, like, sorry we messed up with the software.
It's not possible there
Yeah.
And of course, those terms and conditions you've probably negotiated years before.
So you've got to be careful.
That's where risk management, which is actually-- thank you, [?
Voelker, ?]
for that comment.
Let's take a short break.
And then we'll talk about risk management, which is really what this ends up being.
So let me talk about risk management.
And this is actually quite prominent in the System Engineering Handbook.
This is right in the middle here of your System Engineering Engine, Technical Risk Management, Section 13.
Why is it important?
So first of all, what is risk?
So risk is the probability that a program or project will experience some undesired effect or event.
And then the consequences or impact or severity of that undesired event should occur.
And so think of risk as the product of probability times impact.
And the undesired events could come from a number of things, technical, programmatic.
So cost overruns, schedules slippage, safety mishaps, health problems, malicious activities-- cybersecurity is a big thing these days-- environmental impact, failure to achieve the scientific or technological objectives or success criteria.
And so technical risk management is, therefore, an organized systematic risk-informed activity centered around decision making to proactively identify, analyze, plan, track, control, communicate risks to increase the likelihood of success of a program.
And so what risk really does is measure the future uncertainties of achieving your program goals-- technical, cost, schedule goals-- and think of risks in a holistic way, all aspects of the technical effort, technology maturity, supplier capabilities, performing against plan, and so forth.
And so the idea of risks is that risks have some root cause.
There's something that gives rise to risks.
And then the actual quantification of risks happens in terms of likelihood and consequences, which are kept separate, separate dimensions.
So the first thing to think about is where do risks come from?
Where is the source of risks?
And I want to show you a couple of models for thinking about this.
The first one is this idea of layers of risk, that there are layers of risk.
And I want to credit one of my colleagues here at MIT, Don Lessard from the Sloan School, who really developed this Layer of Risk Model and applied it to different industries.
So there's a version of this for the oil and gas industry.
You could make a version for medical, medical technologies.
So this is the version for Mars missions, so if you're designing a new Mars mission, a new Mars Rover.
So you have, in the bullseye here, the narrow interpretation is technical or project risk.
So the airbag technology.
If you're using airbags for deployment, will it work?
The rover/motor performance, are you going to have software bugs?
Those are the risks we typically think about.
And the idea is you have high influence over these risks as a system engineer, as a project manager.
Then you have a layer around it, which we call industry or competitive risks.
Will your contractors perform?
Will you have budget stability?
And then there's sort of more country and fiscal risk.
So in the US, we have a budget cycle.
We have four-year administrations.
Will you get your budget?
What is the priorities between human and robotic space exploration?
And then working with international partners.
And then there's another layer of risk, which are called market risks.
So if you think in Mars missions, who's your market?
Well, the science community and maybe the public.
So will these missions hold their attention?
Are there new science requirements?
We discovered there's water, probably flowing water on Mars, maybe with a lot of perchlorates in it.
It's not pristine water.
But that could change the priorities for your mission.
And then finally, the most outer is what we call natural risk.
So this would be things like cosmic radiation, micrometeorites, uncertainties in the atmospheric density of Mars as you're doing entry descent and landing.
And you have very low influence.
That doesn't mean you can't protect yourself or take measures to deal with these risks.
But fundamentally, the occurrence or the probability is something you can't really do much about.
So that's one way to think about risks.
And the seeds of risks is in these layers.
I know this is very high level, but I find this to be a pretty useful model.
Yeah, go ahead
So this references the influence you have, not necessarily the amount that each of these are a risk to the program?
That's correct.
And That will be program specific.
Just the stuff that's in the bullseye here, you can do a lot about it, and perhaps both in terms of probability and impact.
And then as you move further out, there's less and less influence you have as a system engineer, as a project manager.
Here's another way to organize your thinking around risks.
And this is around the "Iron" Triangle and project management.
We talk about the "Iron" Triangle of cost, schedule, and risk.
And we call it "Iron" because the idea is that if you constrain all three too tightly, it can be very difficult.
And it's also referred to as the triple constraint in project management.
So the three dimensions here are technical risks, cost risks, and schedule risk.
And in the center we have programmatic risk, which means it's kind of the combination of all three.
And the idea that even if you do a great job on keeping your budget under control, schedule, and you're meeting your technical objectives, you can still fail because the program as a whole isn't doing the right thing.
Or the market that you had been targeting is no longer really attractive by the time you launch.
The key idea here is that these risk categories are not independent of each other.
So let me mention a couple of examples.
So cost risk might limit your funds.
And that could, in itself, induce technical problems which cause you further cost risk.
So one of the big initiatives at NASA in the '90s was the faster, better, cheaper program.
We're going to launch more missions, cheaper.
And out of 10 missions, maybe 2 or 3 will fail.
And then seven will succeed.
But we'll get more value out of this as a portfolio.
Unfortunately, it didn't work very well.
Because when the one, two, or three missions fail out of your portfolio, the media and the public focuses on the failures rather than the aggregate value of the whole portfolio.
And eventually, that's probably the main reason why faster, better, cheaper was abandoned.
So for example, we just talked about testing.
If you have a very limited budget, what is the first thing people typically cut out?
What's the first thing to go?
Testing.
Testing is very important.
Sam asked me during the break, what's your typical budget for testing in V&V activities?
And in many programs, it's very substantial, you know, 40% of the budget, maybe 30%, 40% of the budget easily.
And so you start cutting out tests.
Well, what you do is you introduce technical risk.
And if you have failures because you didn't test, that could cause you additional rework and more cost.
Similar, schedule slips can induce cost risk.
So as you slow down, you have what's known as the standing army cost, right?
People are going to charge to your program, even if it's at a reduced level.
And that will also increase your cost.
So lots of coupling here between risk categories.
This is a very useful Risk Management Framework.
It's essentially a controls framework.
And the idea is you start in the upper right.
You anticipate what can go wrong in your program.
So that's risk identification.
You then analyze these risks, in terms of prioritizing them, which of these are important?
You plan to take action.
This is often called risk mitigation.
You track these actions.
And then you correct any deviations from your plan and you communicate throughout, and you cycle through this.
So typically, risk management will happen on a weekly basis, a monthly basis, at least quarterly basis for big programs.
Now, how do you actually do this?
First of all, the risk ID and the assessment.
So the risks are typically brainstormed.
So you think about risks.
You have to imagine all the bad stuff that could happen to you and your program, the probability that these things will happen, and then the impact or consequence if they do happen based on the requirements, the cost, the schedule, the product and its environment.
And this is where actually having a mix of younger engineers and more experienced engineers really comes in handy.
The experienced engineers, they will have been through several programs.
They will have seen failures in the past.
They will really be able to point to potential risks that less-experienced people may ignore or just not understand how important they could be.
So the next step, then, is to aggregate these into categories, typically not more than 20-or-so categories or risk items.
Projects often keep so-called risk registers, which is just a database or a list of risks.
If you have hundreds and hundreds of risks in the risk register, it's too much.
It's just a long list and really it's just a check-the-box exercise.
To really take risk management seriously, you have to focus on few of the risks that you think are important.
You score them based on a combination of opinions and data.
And you try to involve all the stakeholders in the risk management.
And eventually, risks are placed on this matrix of uncertainty and consequence.
So let me zoom in on the matrix.
There are many, many different versions of the risk matrix.
This is one that NASA typically uses.
And I like this particular version for several reasons that I'll explain.
But basically, the way it works is you have the two dimensions, impact and probability.
So probability is how likely is this to occur?
And then impact, if it does occur, what will happen?
What's the consequence of that?
So of the two things that I really like about it, the first one is that each of these levels, there's actually some definition behind it.
Not just guessing at the level, but there's some criteria.
So for probability, a level 3 means it's about equally likely that it will happen and not happen.
So a level 3 means it's about 50/50, whether this will happen in your program.
And then 4 is very likely.
So maybe that's, I don't know, 75%.
And then near-certainty is like 90% or more.
Improbable is like 10%.
Unlikely is 20% to 30%, something like this.
And then more importantly, the impact so a level 1 impact is negligible.
It has almost no impact.
A level 2 means your mission performance margins are reduced on the technical side.
Do you remember margins?
I asked you to assign margins in assignment A2?
So it means you're eating into your reserves.
But you should still be able to meet all your performance.
There should be no visible impact.
Your safety cushion is less.
That's what level 2 means.
Number 3 means your mission is degraded.
So you can still do the mission.
But you're not going to hit all your targets.
4 is you lose the mission, but the asset is still recoverable.
So maybe you could try again in the future.
And then level 5 is a catastrophic failure that involves a loss of mission and/or loss of crew.
On the cost side, you have some thresholds for cost.
Obviously these numbers have to be adjusted for different programs.
A $10-million loss is a huge thing in some program and almost like pocket change in other programs.
And then schedule, so a level 1 milestone would, for example, be launch.
Good example of this was the Mars Science Laboratory, their Curiosity mission.
Originally it was supposed to launch in 2009.
They missed that deadline, mainly due to problems with cryogenic actuators.
They took a lot of the blames, the actuators.
But there were a lot of problems across the board.
They missed that launch window.
And they had to launch in 2011.
So that was considered, from a programmatic standpoint, a level 5 failure.
Because you missed your main launch window and you had to wait 26 months for the next one.
So that's good.
Because now when you assign a probability and impact, you can really look at these criteria.
And it's easier to do that in a repeatable fashion.
The other thing is if you look at the colors on the matrix, you can see it goes from 1, blue, which means low risk, to 12, which is the highest risk.
So there's 12 risk levels.
But when you look at the matrix, there's something peculiar about it.
So look closely at the colors.
And you'll see something special about this matrix.
Anybody notice what I'm talking about?
Let's see at EPFL, do you guys, when you look at those colors, at the matrix, do you notice something?
Go ahead
It goes from blue to red, which is a light spectrum, with the blue the lowest radio waves, waves, and red the highest ones
Right
[INAUDIBLE]
Go ahead
It's due to impact is more serious than probability
Right.
So it's asymmetric.
You see that?
It's asymmetric.
So the high-impact, low-probability corner is weighted more heavily than the low-impact, high-probability.
And that's intentional because it's been shown in the past that things that are not likely to happen but if they happen, they're really bad, in the past, people have sort of pushed that away and ignored those.
So the purpose of this asymmetry in this matrix is to elevate the low-probability, high-impact events to be higher in the risk level so that people pay more attention to it.
OK?
So most risk matrices don't have that asymmetry in them, but this one does.
[SNEEZES] And I like it because of that.
Bless you.
So the question then is, what do you do with this?
The idea is you do your risk management.
You identify your risks.
You place them on this matrix.
And then you track each of these risk items over time.
So here's your 12 risk levels, between 1 and 12.
That's the y-axis.
And then on your x-axis there's Time.
And for each of your risk items, people might disagree.
Some people on your team might say, hey, look, this is not a big deal.
We've seen this before.
The impact is not big.
We have a quick fix for this.
We know how to deal with this.
And other people disagree and say no, this is very, very serious.
You have to take it seriously.
So the idea is that for each risk item, you have this optimistic, expected, and pessimistic estimate of what is the true level of risk.
That's what these bars are.
And then you track it over time.
And depending on whether you're before PDR or CDR, you could have very substantial risks.
And that's, I guess, OK still, as long as you find ways to reduce the level of risk.
For example, by doing extra testing, by changing your design to put in extra power margins, bigger solar panels, redundancy, for example.
There's a lot of things you can do to affect both the probability and the impact, radiation, extra shielding.
And so the idea is that over time you're going to reduce these risks gradually below some threshold.
So this red line here is like the acceptable threshold of risks at that point in the program.
And then as you get closer to launch, things should be below the threshold.
And if it's below this watch domain, then you even don't track it.
You don't pay much attention to it.
If it's above this red line, you have a big problem.
You might have to stop the program or do a major redesign, or repeat a major milestone.
And some programs have been canceled because they just couldn't get these risks under control.
So the idea is that gradually you transition.
And you do this by actually doing risk mitigation around that risk management cycle.
Now, the last thing I will say here is that every mission that is worthwhile doing is still going to have some residual risk at the end.
The requirement is not that all the risks are at 0 in the lower-left corner.
You will launch, and you're going to have residual risks.
And you just have to accept those.
But you have to be cognisant of this.
And this is really no different in the automotive industry, for example.
When you're developing a new car or a medical device and you're going to launch it to the market, if your requirement is 0 risk, you will never sell anything.
You will never launch anything.
Because you will always think of something bad that could happen.
And there will always be people saying it's too risky.
We can't do it.
So knowing how much residual risk you should be willing to carry is a big part of being a leader, being a system engineer, really understanding things.
And in the automotive industry, there are people whose primary job they have is to do this work.
And they're called Quality Engineers or Warranty Engineers.
So the Warranty Engineers, their job is twofold.
Before you launch a vehicle to market, it's actually ranking on this particular vehicle or program, what are the top 10 things that could cause warranty planes and problems in the future?
We don't know that they will, but they might.
Right?
And then once a vehicle goes to market and reports are coming back from users and from the fleet, actually tracking what these issues are and then knowing when has it hit a threshold where you do need to do a recall, you do need to do a retrofit, this is a big part of it.
And it's really a big deal.
I mean, the amount of money that automotive companies spend on recalls every year is about the same as what their profits are.
So if you could eliminate recalls and warranty claims altogether, you'd basically double your profit.
And so depending on what industry you're in, whether it's automotive, medical, spacecraft, how much risk and safety is involved, this is more or less emphasized in the industry.
But it's a big part, I think, of system engineering job, is to understand this.
This is, again, a flow diagram for how to do this risk management properly.
You have a Risk Management Plan, your technical risk issues that are placed on the matrix, any measurements or data you have.
And then how do you report this?
And then out of it comes a mitigation plan, a set of actions, technical risk reports, and then any work product from the technical risk management.
And the idea is that you repeat this process on a regular basis.
And it's a big part of your milestone reviews as well.
OK, so I'd like to spend a few minutes on system safety.
I am not going to do this justice because there's a whole class here at MIT on this, taught by Professor Nancy Leveson.
By the way, who's taken that class?
Or who's been thinking about taking it?
About three, four of you.
So I'm just going to give you a very quick exposure to this.
So this is a book that Professor Leveson wrote several years ago.
And she basically distinguishes two kinds of failures.
Component failures, which most people think about, an axle broke or there was a battery caught fire.
And clearly, component failures are real and they happen, single or multiple component failures.
And usually there's some randomness to them.
So most of the classic accident investigation techniques and safety techniques focus on component failures.
But there's also component interaction accidents or failures which are trickier, in a sense, because you could have a system that has no single component that's failed.
And yet, you had a system failure.
And so it's the interactions among components.
And this could be related to interactive complexity in coupling, more and more computers and software, and then the role of humans and systems.
And this is really what a lot of this is about.
So the traditional safety thinking is that component failures, you need to worry about the component failures only.
So here's a classic example of a sequence of events.
This is for a tank failure.
So in this case, we have a tank.
And there's moisture that builds up in the tank.
And then corrosion, essentially, as a result.
The metal gets weakened.
And under the operating pressure of the tank, the tank itself has been weakened due to corrosion.
The operating pressure causes a tank rupture.
And the tank rupture then causes, essentially, an explosion.
And fragments or shrapnel from the tank will be projected and then cause equipment damage or personnel injury.
And so in this linear chain-of-events model, the way you think about safety is putting in barriers.
This is often also referred as the Swiss cheese model.
if you take layers of Swiss cheese-- and I guess it has to be Emmentaler, right?
You guys know the Emmentaler at EPFL?
Emmentaler is the one, it's got the big holes
Yes
So you take these slices of cheese.
And if you can look at the cheese, and there's actually a hole right through, then the accident can happen.
But if you put another barrier in between, you can't see through the cheese and the accident is prevented.
That's the classical thinking around system safety is chain of events.
And then put barriers between these.
And this is, I think, valid for a very particular kind of accidents, which are these component accidents.
What Professor Leveson says in her STAMP and STPA Framework is a little different.
This is based on essentially thinking about safety as a lack of control of a system, lack of control-ability of a system.
And so if you think about it this way, I'm showing you here a control loop.
In this case, you have the actual system, the actual process that you're executing.
The controlled process is here.
You're sensing things about that process, so temperature, pressure, proper alignment.
And so one problem could be your sensors are inadequate.
You're sensing the wrong information.
And then here's your controller model, how should you control the system?
So you have the wrong controller or the wrong process model.
And then here's your actuators.
Are you issuing commands at the right time?
Are you issuing the right commands?
Or are you not issuing commands when you should be to the system?
And that feeds back into the control process itself.
And so process inputs could be wrong or missing.
You could have disturbances into the process that are unidentified or out of range.
And then eventually, if this process goes unstable, then you have a failure or an accident.
So it's quite different.
It's essentially thinking of this as a control problem instead of a chain-of-events problem.
And the argument here is that for safety or failures that involve a combination of hardware, software and humans, often this model is able to be more complete, in terms of identifying hazards and potential mitigation actions.
So I think we're out of time.
But I want I want to give this to you as a homework for thinking about this.
This is an accident that happened earlier this year, I guess, in July.
And this is the Virgin Galactic crash that happened.
Virgin Galactic is one of the space tourism companies.
And during a test flight, the airplane crashed because the copilot unlocked the brake system.
So it has a kind of feathering mechanism.
And the pilot unlocked it too early during a high-speed flight phase.
So what I'd like you to do, the link is here.
Just read the story.
There's a more lengthy accident report that's come out.
I'd like you to just read this quickly and then think about how does this relate to risks?
How does it relate to this particular model of system safety?
So the System's Theoretic View of Safety is then that safety is an emergent system property.
Accidents arise from interactions among system components-- physical, human, social-- constraint violation.
Losses are the result of complex processes, not simple chain of events.
And that most accidents arise from-- you could have a system that's quite safe when you start operating it.
But over time it migrates to an unsafe state.
Because sensors fail.
People start bypassing safety procedures.
And gradually, it migrates to high risk.
OK.
So the last thing I want to talk about-- just for a minute or two-- is the FRR, the Flight Readiness Review, which is one of the later milestones.
And what happens at the flight FRR?
Essentially, this is your last chance to raise a red flag.
This is the last milestone before launch.
Have all the V&V activities been passed successfully?
Are there any waivers that need to be granted?
What are the residual risks we just talked about?
And then after the FRR has passed, you actually start the countdown-- T minus X days, Y hours, Z seconds-- to an actual launch, or a product launch, whatever it is.
And then here's from the handbook, the entrance and success criteria for FRR.
Everything should have been done at this point.
Your design, your integration, your testing, your operating procedure, your people should be trained.
This is your last chance to raise a red flag.
After the FRR, you're essentially go for launch.
So the stakes are high.
OK, so a quick summary.
Verification and validation are critical.
There's a distinction between the two.
Verification is against the requirements as written.
Validation is you go back to your customer and you test in a real environment.
Testing, many different kinds of testing.
It's a fundamentally Q&A activity, and it's really expensive, but it needs to be done right.
Risk management, we have different tools like the risk matrix, risk identification, mitigation.
And that's really where the rubber meets the road, in terms of the tension between cost, scope, schedule, and risk in projects.
System safety, think about not just the chain of events model, but this control's view as well.
STAMP/STPA is a particular framework for this.
And if you're interested, there's a whole class.
There's a whole set of things you can learn about just safety.
And then finally, FRR is your last chance to raise the red flag.
It's sort of the big milestone before you go live with your system.
OK?
So any last questions or comments?
EPFL?
Yes, please go ahead
So after the FRR you should have finished your tests, right?
It's the limit?
Yes, you should have finished your tests except for the ones that you're going to do, say, on orbit, right?
Because I was wondering, in the list there is the go, no-go test.
And I'm wondering if it's not related to the launch, actually?
Yeah, so the actual launch itself, of course, has the actual launch countdown.
And you can stop the launch.
So this is a review.
The FRR is more like a CDR.
So the countdown hasn't actually started yet.
But if you successfully pass the FRR, that's when you begin the countdown.
The official countdown starts.
And then you still have a possibility, of course, of stopping the launch.
But in terms of a formal programmatic review, this is your last chance
I think the point here is more about the what is the system boundary?
If you consider the system boundary being the whole mission, including the launch, then obviously the system will only be finished when the mission has finished phase
Right
And this FRR, Flight Readiness Review, is linked to the panel or to the whole satellite, specifically that you are allowed to go forward and now start the countdown issues.
And then all the other things.
Like, you'd extend the mission boundaries to the whole space program or to the whole colonization of Mars, it will be, obviously, later.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The engines, the aircraft engines, have a lot of sensors on them because the engines really need to work reliably.
So it tends to be more sort of thermal structural components.
And my sense is in cars, you monitor more electronics, how many--
Emissions
--certainly admissions.
It's slightly different what's being monitored, and the sensors are different.
Even the companies that provide these sensors and health-monitoring equipment tend to be different as well.
I think, there are similar trends, but they're somewhat parallel.
I don't think one came before the other
OK. Yeah
Hey, Volcker, do you want to mention something about health monitoring in general for operations?
I just sent you a link on your email because we've been for six years monitoring the health of [INAUDIBLE] that's up there.
And it's live link, so I don't know if you are able from your computer to open it.
I just sent you the link on your email.
Right?
So actually, into the first satellite that we both [INAUDIBLE] few years ago, we figured we want to know everything.
So the primary [?
dom ?]
system were all sent on every telemetry data the same [?
dom ?]
at every path and kept actual.
So maybe after break or whenever there's a chance, go to the link, and you can see the actual housekeeping data.
And you'll see that the battery, the voltage, is the only one that's [INAUDIBLE].. Likely, [INAUDIBLE] of the six years, which is not bad for a VC system
Yup, OK, I'll pull it up during the break.
Thanks for that.
That'll be interesting.
OK, let's move on.
So unfortunately, I would say the F-18, and there's a lot of systems that were designed with operational excellence in mind-- maintainability, high reliability.
That was a big thing.
I did mention this before, I think, but I just want to say that it's not always the case.
So this is one of the counter-examples, the space shuttle.
And I have a lot of respect.
Don't misunderstand me.
I think people that worked on the shuttle did an amazing job.
It was an amazing vehicle.
But things turned out quite differently than what was promised.
I think that everybody will agree with that.
So this is a paper that was actually a very short paper published believe it or not in nature in 2011 right after the retirement of the shuttle and by Pilcke et al.
And what they did is, they looked at the costs of the shuttle program.
And the cost is well known because this is all public money.
This is all money that was appropriated by Congress.
And then the light blue bars are the number of launches that happen in a particular year.
So you can see the maximum was nine-- nine launches in 1984-- not a launch a week that Congress had been told.
And so we have about 10 years of design, build, test until initial operating capability.
And then we have about 30 years of actually usage and operations.
And this is true for any system.
The operational phase is much longer than the design phase.
So in this case, the operational phase was 30 years right, three times longer than the actual design phase, which was about a decade.
The Vision was a partially reusable space vehicle, quick turnaround, high flight rate.
What we actually got is a complex and fragile vehicle with an average cost of about $1.5 billion for flight and a workforce of about 20,000 people to keep the shuttle flying.
And I think I've shown you this before, right?
Did I show you this before?
So this is an original illustration from the proposal to Congress.
It kind of looks like the F-18 that I just showed you.
It's a hangar, pristine, a few people, a couple of ground support equipment carts-- this is kind of like an airplane, like an airliner.
And then this is what we actually got.
This is a picture taken in the Orbiter Processing Facility at the Kennedy Space Center.
And you can't even see the shuttle, right?
It's hidden behind the scaffolding, and the main systems that required a lot of work between flights were the shuttle main engine and of course the TPS, the thermal protection system.
Now, you can say, well why such a big discrepancy between what was promised, the Vision, and what's actually delivered?
And people will have different opinions.
My opinion is, certainly over-optimism was a part of it.
But also Congress kept the research development costs for the Orbiter in particular to about 5.1, 5 billion.
There was actually an act of Congress that said those shall not spend more 5.1, 5 billion on development of the Orbiter.
So then, when it was clear, 24 metric tons to low-earth orbit.
If you can't achieve that, the system will not satisfy.
So performance is king, and of course, this was also a politically challenging because there were military requirements and civilian requirements that had to be met.
And then maintainability, like I said, just doesn't happen automatically.
You have to actually write requirements for maintainability.
How long should it take?
How many hours of work?
How many actions or procedures to do certain maintenance actions?
It has to be designed into the system, therefore the need for requirements.
And then no realistic lifecycle cost or value optimization.
So that's, I think, the counter-example.
But again, I am not blaming individuals for this.
I'm blaming the system for this.
Yes?
To what extent do you think that kind of problems within shuttle can be traced to effectively a failure to stop defining the requirements?
I feel like when you look at commercial launch systems kind of as a comparison point.
And I understand they're not reusable.
And they're not being designed to necessarily the same specifications as shuttle.
But I feel like because the commercial launch industry has had standards set by NASA and the Air Force that they needed to meet, that they had a very clear set of requirements from the beginning.
Whereas the development of shuttle, like the failure to design for maintainability, when I look at the development cycle, it looks like they continue to refine requirements as they went and kind of discovered new things.
And then said, OK, so we need to do it this way instead of saying from the beginning, these departments we know we have to meet.
Let's design to that
Right, though I--
It look like they kind of crept through the development phase to me, that there was no hard stop.
So I was just wondering if you could comment on that kind of--
It's definitely true that there was sort of reacting to-- and after the first few-- so an interesting history there is the shuttle main engines.
They were completely disassembled and inspected after the first couple of flights.
But that wasn't supposed to be done every flight.
It was supposed to be, you're going to launch them, I think, like five times before you actually do big inspections.
But they had already done inspections after every test flight.
And then they just kept doing it.
So something that was supposed to be only a maintenance action or inspection during test flights became or crept into becoming an operational requirement.
And of course, you have a big workforce and there's jobs and so forth.
So there's that too.
But it was never intended that the shuttle main engines would be disassembled and rebuilt after every flight
Why was that permitted?
Well, I guess, the sense was that it would be safer to do that, and that you really want to know what's the state of these engines.
And I will say this, I mean at the contractual, there's jobs there.
There's money there.
And so the more maintenance actions you can do, the more of a business this is.
But of course, that's not what the Vision was.
The Vision was a very lean, operations, few people.
So there's a socioeconomic things tied up with it as well
Do you think that because that's inherently a government project that's part of the reason?
Like whereas a commercial company is looking to cut as many employees as possible to increase profit margin?
I think that that's a part of it

Yeah, I do think that's a part of it.
Yeah, absolutely, it's a really interesting history.
There's a lot to be learned from this.
OK, so let me move on.
So this is kind of a list, not a checklist really, but a list of operational considerations that you should think about when you design a system.
So how will it be operated?
And of course, we've done the CONOPS a while ago, but this is sort of more detailed than the CONOPS.
How will you inspect the system?
How will you maintain it?
What insights do the operators need into the system status?
So when you are operating the system, how much about the internal workings do you really need to know?
This is the internal telemetry temperatures, pressures, in the avionics, the electrical.
I think the example of the electrical bus was good, the cryogenics system example, how much insight do you need?
Before turning over to the operators, what checks do you need to perform?
How might the system fail?
Think about failure, and you of course need to think about that as early as possible.
What are the options available to you in case you have failures?
And you will have failures.
What spares are needed to repair the system?
Will this system still perform even under partial failure?
So maybe something failed but not catastrophically.
It's a partial failure.
Can you keep going with the system?
How far can you push the system?
I think if you think about this is like five questions here, but I think, they are five of the most important questions for operations.
Now in terms of the NASA lifecycle, we're talking Phase E here, just to be clear.
So Phase E is called Operations and Sustainment.
And depending on what mission you're talking about, this could be short-- like the Apollo missions, like two weeks and they're back home-- ISS, six months rotations.
Or it could be something like Voyager.
Voyager was launched when in '76?
Voyager's been flying for 40 years, and we're still getting data and telemetry at very low data rates, like 100 bits per second or something like this.
But still, that's remarkable, right?
So Phase E, in that case, is very long.
So it's worth really thinking about this.
All right, let me talk about commissioning.
So commissioning is essentially the transition from Phase D to Phase E. So Phase D is System Assembly, Integration, Test, and Launch, and Transition to Use.
And Phase E is then Operations and Sustainment.
So to conduct the mission, meet the initially-identified need, maintain support, and then implement your mission operations plan.
So commissioning, essentially, is transitioning from Phase D to Phase E. And usually, the people that will operate the system day in, day out, tend to be different people than the people who design the system, who build the system, who launch the system.
So usually, you have some kind of a handoff or handover of the system from the designers, builders to the operators.
And that handover is very important that it be done well.
And that's what we call commissioning.
Or in this case, this is the-- do you remember this?
We haven't shown this for a while.
Remember what do we call this, this thing here?
The engine, right?
The system's engineering engine at every level.
So this is Step 9, Product Transitioning Process.
And then there has to be a flow chart, right?
Can't do without a flow chart.
It's not particularly fancy or anything, but the idea is that you have inputs, which are the end product, ready to be used, documentation.
So when I ask you about the cryogenic system-- you mentioned the cryogenic system-- I ask you, do you have a user manual for it?
So that would be here, in this box here, on the left side-- the documentation that goes with the end product.
And then any product transition-enabling products, so those would be things that you only use during the transition, like equipment or facilities that you just use for this transitioning process.
And then you don't use them during operations.
You can think of examples of that.
And then you go through this multiple steps.
There may be multiple sites that you have to prepare, multiple locations.
And at the end of it, you've delivered the end product.
It's operational, transition work products, and you're essentially operational.
So what it means in practice is deploying the system in the field, transitioning to the operators, physically and legally also-- I should point this out.
This is really important.
So usually, in this commissioning phase, the legal ownership of the product or the asset is transferred from one organization to the other.
So if it breaks now, it's your problem.
It's not my problem.
You've already taken ownership.
You've signed off on it.
And for insurance purposes, this is a very big deal.
So at what point does legal ownership of the asset transition?
You really have to know that.
And then, of course, the training.
Checkout-- checkout means turning on all your systems and subsystems, making sure everything works, making sure there's no emergent behaviors, weird behaviors, unexpected behaviors.
Comparing the predicted parameters against the actual behaviors.
Does the system behave as we had predicted, based on calculations?
These days, we usually build simulations.
We build a pretty realistic simulation of the system, and there's the concept of digital twin.
Who's heard this before?
Digital twin.
Who's heard this before?
Oh, nobody, OK.
So the idea of a digital twin is that here's your physical system, airplane, satellite, cube sat, whatever it is, medical device.
And there's a digital twin of it.
There's a in silico version, a simulated twin, of that system that exists somewhere.
And before you turn the system on and do operations on it, you do it on the digital twin to see what will happen and predict everything-- all the parameters, the position, velocity, accelerations, temperatures, pressures.
So if things look good in the digital twin, then you actually do it in the real system.
So for example, JPL with their Mars rovers, they will do that.
They have a digital twin, they have a digital version.
They actually have a physical twin too.
But they typically only use that if there's problems, like stuck in a sand dune or something like that.
So you actually simulate commands that you're going to send to the spacecraft on your digital twin, make sure that the command sequence is correct, that things will-- and then you do it on the real system.
That's what we mean here.
And during checkout, you do it initially.
But actually, if it's a very complex system, you might do this as a matter of routine.
And then sustainment is the third thing here.
So sustainment means maintenance, both preventative and corrective.
So preventative maintenance means you're taking actions before the system breaks.
Corrective action means you're taking maintenance actions after you have failures of different kind-- spare parts management, reconfiguring system during use for different purposes, upgrading system, and then retrofits.
So what is a retrofit?
Anybody know?
At EPFL, are you familiar with the term retrofit?
Have you heard this word before?
Volcker, do you want to explain what it means?
Absolutely.
So for example, let's take your [INAUDIBLE] system, the system volt's M109.
Ours says from USA.
Back in the '60s, '70s, then these things are aluminum, launch self-propelled.
And in the '80s, '90s, they decided to not throw them away.
They don't rust.
But to retrofit them, to put guidance navigation system, new cam.
So waiting through a retrofit program, meaning you sort of break down everything, down to the lowest elements, you decide what you can keep.
You throw away the obsolete stuff.
You buy new things, and you try to make it
Yup
And then you reconfigure
So retrofit typically means you physically go out in the field, and you physically change the system.
Like Volcker said, you remove things, you typically add new things.
So now you have like old generation and new generation stuff mixed up in the system.
Retrofitting is a huge deal, particularly in the military, but in other domains as well, like hospitals, infrastructure, train systems.
You go there, and you see a mix of old stuff from when the system was originally built and deployed, and then new stuff layered on top of it through retrofits.
And the key question, of course, when you do a retrofit is, you want the retrofit not to interfere.
You want it to actually be value added as opposed to causing more problems.
Yeah, go ahead
Is retrofit the same as medium-life up grade?
I would say it this way-- a medium-life or middle-of-the-life upgrade is a particular type of retrofit that you do roughly halfway through the nominal mission life.
Yes, and do you have experience with such upgrade?
Yes, actually we have both F-5 and AMX in Brazil.
We have the medium-life upgrade and both performed by Embraer

So it's interesting because the F-5, for example, is Northrop.
And the medium-life upgrade was performed by Embraer.
And we had lots of issues about it
So what was done to the airplane?
Completely change of the dashboard panels and new electronic warfare equipment.
But they kept the old hydraulics engine and-- actually, hydraulics and engine.
The electrical system was changed as well
Was changed as well.
So you keep the frame, the engine--
And you change the radar, the antennas, and lots of different stuff
That makes sense.
That makes sense
A new head-up display, a new helmet, stuff like that
The interesting discussion, which I think in the US for much equipment, and Switzerland same discussions, is by the time you start adding up the cost of a midlife upgrade or retrofit, you start adding all these things up, pretty quickly you come to the question, well, wow that's pretty expensive.
Is it really worth it, investing x millions or billions to squeeze another 10, 15 years out of this platform?
Or should we just buy a new one and retire this one?
And you'll have very vigorous debates about that.
Yeah?
Just how does the funding structure ever come into play when determining how sustainable, or how you're going to retrofit something?
I know sometimes, where you worked funding ends this time and something else starts up this time.
So sometimes, you kind of design a little bit around those things.
Is that ever taken into consideration?
Well, I think enlightened companies, enlightened organizations will actually build in retrofit and upgrade costs into their original budgets.
But does it happen often?
I think it's a minority of organizations who really are honest to think about the full lifecycle cost, including upgrades and retrofits.
And to be honest, sometimes, these retrofits, it's hard to know about them ahead of time.
In some cases you can predict them, but in many cases, it's more reactive.
In fact, I'll tell you a story about the F-18.
These retrofits often come in bundles or packages called ECPs, Engineering Change Proposals.
They're sort of packages of changes.
And it's almost blackmail, but it's like, here's a bundle of changes that is x million dollars.
And it's up to you whether you want to implement that or not.
But if you choose not to implement, then the warranty is void.
Or you're losing compatibility with any future upgrades.
Do you see what I'm saying?
So the configurations then start to diverge.
And then you have to make a tough decision.
Do you spend money on an upgrade that maybe you don't absolutely need, but some other customers wanted?
And if you say no, we don't want this.
We're going to freeze the configuration where we are today.
Then, you might lose the option for future upgrades.
And if you then choose to upgrade later, it could be much more expensive.
And that's systems engineering too.
And it gets into technical issue, financial issues, strategic issues.
So I hope you're getting a sense here that this is all happening during operations.
This is fascinating stuff.
This is not boring.
This is not just routine operations.
OK, so in operations, we have launch for spacecraft science operations, safehold, anomaly resolution, and so forth.
And I just want to show you a quick video here about the James Webb-- this is a one-minute video about the anticipated deployment of the James Webb Space Telescope.
I think, I mentioned to you.
I've worked on this as a Master's student.
This is supposed to happen in 2018, launch from Kourou on an Ariane rocket.
This is a NASA-ISA collaboration.
You can see right now, the spacecraft has been launched.
It's deploying its sunshield.
These are very thin membranes.
The purpose of the sunshield is to keep the optics cold.
The optics are on the top here.
And there's spreaders, and there shouldn't be any wrinkles.
And there's multiple layers.
You can see the layers.
Now, first you spread it out horizontally.
Now, the layers are being spread to be right at the right angles.
And I don't know what's happening right now.
But the last step that should happen-- I guess it didn't quite play until the end.
But the last thing that happens is the optics are then deployed.
And the optical deployment is also very, very involved and has to be very precise.
So let me just minimize this.
So and then after that, you have a commissioning.
I should also mention that the James Webb State, when I worked on it, it was a $500-million mission.
It's now become an $8-billion mission.
But the purpose of this instrument is to look back in time to what's known as the dark ages.
It's basically like 300,000 years to about 100 million years after the Big Bang.
The formation of the very first Pluto galaxies.
We can't observe that right now with Hubble or from the ground, mainly because these are so far away because of the expansion of the universe that the radiation is redshifted.
And this is all in the infrared.
So you to have a very quiet, very cold instrument to see these first proto-galaxies being formed.
So I know we can argue, is it worth spending 8 billion to see the very first galaxy being formed or not?
People will debate it.
But the fact is, it's happening.
James Webb will launch.
And then, it's going to go through a commissioning phase, like I just talked about.
So I want you to answer this question here.
How long do you think the commissioning phase of the James Webb Space Telescope will take?
Three days, a week-- in orbit that is.
Three days, a week, three weeks, a month, three months, six months, or you're not sure.
So answer that question, and then we'll take a short break, like five minutes, and look at the answer.
Yup?
Are they still on schedule with the telescope?
Or do you have any idea on whether that will slip some more?
When I worked on it, it was supposed to launch in 2008.
But I do think-- I mean, the spacecraft is being integrated right now at Northrop Grumman.
We saw it in January.
So they're not lying to us, I think.
They're actually putting things together.
And I think, it's going to launch in 20-- maybe slip by another six months, or maybe a year but not more.
I don't think so.
Basically, how long does it take from what you just saw in the video until PIs, Principal Investigator scientists-- [INTERPOSING VOICES]
Can start using it
--can start using it for real science.
How long will that take?
So who wants to respond?
Who thinks it's like a month, three weeks, a week?
A month or less?
Who responded to that, a month or less.
Go ahead.
What were you thinking?
No specific idea
What were you imagining would happen in a month?
I thought that it couldn't be very, very long because if the objective is to collect the data, you can't wait that much before you start using it.
But I had no specific ideas
No, I think that's a good answer.
You deploy the solar panels, the sunshield, the optics, the spacecraft's at the right place.
By the way, this is going to launch too in Earth-trailing orbit, like Earth-Sun L2, kind of trailing orbit.
So it's kind of away from the Earth, from albedo, all that.
So you deployed it, let's get on with it.
[LAUGHTER] Yeah, I agree.
And then I think, 37% of you, a third of you thought maybe three months, and then 40% about six months.
All right, so let's look at the current plans.
And I took some slides here-- I referenced it-- from a lady that works at the Space Telescope Science Institute that's located in Baltimore, the NASA's Goddard Space Center of Baltimore.
This is from last year.
So here's the plan to launch in October of 2018, and then do the deployment.
And the answer is right now six months.
Six months commissioning phase-- and then, there's different cycles of-- so GO stands for guest observer.
So you have primary PIs.
They usually get first dibs.
And then you have observing time for guest observers.
Here's another little bit more detail.
So full schedule of deployment and check-out activities, a limited set of science calibration ops possible, science observations highly unlikely during this six-months phase.
And then, you have this Guest Observer Program, and there's a budget, actually, Guaranteed Time Observation Program from April of 2019-- a total of 3,960 hours allocated in the first 30 months after commissioning, OK?
And so those 3,960 hours are-- people will fight over this.
And there's a very detailed process for how to allocate and compete for that observation time.
But the answer here is six months.
And it really surprised me too that it's going to take this long but the reason it takes so long is mainly calibration.
Calibration is a big thing.
You want to make sure that all the observations you're going to take are correct.
And in order to do a proper calibration, you have to essentially image things that have already been imaged before by other instruments that were also properly calibrated.
So that for a known set of targets, you know you're getting the same answer.
And that just takes time.
I think, in a nutshell, that's the main reason it takes so long
Pardon me?
Yup, go ahead
There's also the one aspect with this observatory is that you have to be stabilized.
And as you get to bring it up from Earth, even though it was pulled together in a clean, very high [INAUDIBLE],, it's going to have to deploy.
And then, just to get to the chemical stabilization, it's going to spend weeks before they even can start to calibrate the instruments.
So for you to get the outgassing, the whole volatile substances and get that they move far away, not from the spacecraft, that they dilute in space, get stabilization of temperature, and then you can only start checking instruments
Great point.
So thermal stability, outgassing-- so really everything is very stable.
Yeah, very good point.
OK, so let me talk briefly.
I'm going to go through two examples of research in operations.
And then, we'll talk about the post-flight review.
All right, so the first thing is, each of these two examples are based on papers.
So "Spare parts requirements for space missions with reconfigurability and commonality."
This is based on a paper in Journal of Spacecraft and Rockets, 2007.
So and this work was done during the Constellation Program which was, we're going to go back to the moon.
We will re-establish a human presence on the moon.
And we're going to bring a whole bunch of stuff with us, more than we did during Apollo to do this.
So the picture on the upper-right, you see a habitat on top of a Lander stage.
You see an Ascent Vehicle with a Lander stage.
You see a Rover that looks kind of similar to the Lunar Rover, but there's also a pressurized version of it.
So we're going to bring a whole bunch of stuff.
And the challenge is during operations, things will break.
So you need to bring spare parts to support all this.
And some recent research we've done in my group shows that for Mars, it's the same problem.
You're going to stay there a long time.
You don't know exactly what will break.
But if you want a high probability of being able to successfully operate, you do need to bring spares.
So the idea that was explored here is, what if those spares could be common or reconfigurable and you can do scavenging?
So the idea is that, instead of bringing a dedicated spare-- if you give these three systems that are shown here, The Habitat, the Ascent Vehicle, and the Rover, to three different companies to build, and they don't talk to each other, and you don't impose any commonality requirements, you're going to get very different solutions.
What's the classic example of this in spaceflight, human spaceflight?
Apollo 13.
What happened in Apollo 13?
The cartridges for the CO2 scrubbing, square cartridges versus round cartridges.
The two different contractors, they didn't talk to each other.
The government didn't say you have to make these common, so they weren't common.
So the idea here is, what is the effect of reconfigurable and common spares on system availability if you allow temporary scavenging and cannibalization of systems that aren't used?
And so one thing you need for that is an operational profile for each element.
And that's shown in the lower left.
So this is essentially a binary.
Zero means that particular element is dormant or not being used.
It's kind of in slumber mode.
And 1 means it's actively being used.
And so then we have at each of these time periods, T1, T2, T3-- our time periods where there is a change in the operational status of a particular element.
Either it goes from sleeping mode to active mode, or from active mode to inactive mode.
And so knowing these cycles' operational profiles is very important to do the analysis.
So this is a little bit of math here.
I'm not going to go through this in detail.
But basically, when you do classical sparing, and maintainability, and failure analysis, you assume that failures arrive according to a Poisson process.
So this is the equation for a Poisson distribution.
And then you can see the various variables that are used here.
So your lambda is your failure rate.
P of n is the probability they have exactly n failures.
And then, down here, we have the spares that are available to you.
And really spares can come from two different sources.
One is, you bring spares with you from Earth.
So this is a spares from repository, s sub i.
These are spares you brought with you as a pool of spares.
And then s sub e are spares that you take out of elements that are not being used.
So these are spares that are scavenged temporarily from inactive elements.
That's s sub e minus n sub f, which is the number of failures that have occurred up to that point.
So the total number of spares available at any given point in time is your initial spares pool plus spares you can scavenge from other inactive elements in the system, minus elements that have already failed.
And this is assuming no repair, so you can't repair.
And it assumes that you know ahead of time what the operational profile is.
So there's spares available from elements from scavenging is this equation here.
It's essentially the sum over all the elements, E. E is the number of elements in your architecture.
Q sub E is something known as quantity per application, QPA also.
So basically, if you have like a mission computer or in like UAVs, we have servos.
Like, this particular element has six servos, identical servos.
They're used on the vehicle.
So QPA, this QE would be 6.
So if that element isn't used, we could go in, take out a servo, put it in somewhere else.
And so we can treat that as a spare, at least during the period where that UAV isn't used.
Does that make sense?
OK, so the difference then is that in the kind of classic way of doing it, we have dedicated spares.
Each element, one element 1, element i, element E, has dedicated spares that only work on that vehicle.
There's no swapping, there's no scavenging, there's no communality.
And therefore, your spares repository, s sub i, is going to be pretty big because there's no commonality, no sparing.
In this new situation, you have reconfigurable or common parts.
So we still have a spares repository.
But now those spares can be deployed across all or a subset of the elements, plus we can treat-- and this is this dashed line here-- we can treat elements that are part of idle elements.
We can treat them temporarily as being part of the spares repository.
Does that make sense?
So that's a much more, in a sense, smarter way to do.
And the question is, what's the benefit of this?
So in order to calculate the benefits for this, we essentially-- we have some constraints.
So for example, the number of failures that you can have is between-- if you have zero failures, that's great.
That's your lower floor.
And then N is the total number of units you have in your architecture, including the ones in the inventory, in the initial inventory, and plus the ones that are built into all the vehicles.
And then, the key here is what's known as back-order level.
So back-order essentially is the number of spares that you would need but may not-- so when the back-order level becomes larger than zero, it essentially means that your number of failures have exceeded the number of spares that you have.
You don't have enough spares to satisfy all your operational needs.
So this is this conditional back-order at spares level s. And you sum that essentially then over all the possible failure states that you could see.
And why is that useful?
Because you can then calculate essentially your element availability.
So a is your availability of at time ti.
And then your probability of the whole system is the minimum of your system availability at any time and point during your mission horizon, your mission time, T. Do you see how that works?
So you have your spares.
You have failures of the spares, which are random.
And then you can calculate, are you going to have enough spares to operate every element that you need to operate when it's supposed to be operational?
And the back-order level, this number b here, is what you use.
You look at your back-order level across the whole mission timeline to see what your system availability will be.
And there's a closed form approximation of this.
And then for a larger number of elements and different QPAs and so forth, you typically then have to switch to simulation, like Monte Carlo simulations.
So what are the results here?
So this was applied to-- the example here is a co-located mission elements.
This is for, I think, it was a lunar mission.
And you define essentially the operational time profile, quantity per application, and so forth.
And the example that was used was an electronic control unit, and ECU, with a meantime to failure of 100,000 hours.
So failure rate is 1 over meantime to failure.
Lambda is the expected failure rate is 1 over meantime to failure.
So pretty reliable, 100,000 hours of operation meantime to failure is pretty reliable.
But of course, you're far from Earth.
This is a 600-day timeline, and you can see here the operational profiles for your various mission elements.
So this is actually kind of reminds you of The Martian, right?
Anybody seen the movie The Martian recently?
Who's seen The Martian?
Who's not seen The Martian?
You've got to go.
You're the only one in the room.
It's a great movie.
So he does scavenging and even things that weren't supposed to be scavenged.
And so we have four elements here.
We have the PR, which is the Pressurized Rover, the Habitat, the All-Terrain Vehicle, and then the ATV, which is the Ascent-Descent Vehicle.
In this case, it's the same vehicle for ascent and descent.
And so the Ascent-Descent Vehicle, you only need it when you land and when you depart.
The rest of the time it's dormant.
The Habitat is used while you're on the surface.
The ATV is used while you're on the surface.
And then, the Pressurized Rover is used-- the assumption here is you're going to operate for like 100 days very close to the base.
And only after 100 days are you going to start going further away with the Pressurized Rover.
So those are the operational profiles.
Yeah?
Is there an argument that you wouldn't want to be scavenging parts from your Ascent-Descent Vehicle at all?
That's a good point.
So if basically, your back-order level is too high, and you run out of your last spare the day before you're supposed to launch on the ADV, you're in trouble.
But if that's the case, then what you have to do is you have to exclude the minimum-- and there may be redundancy in that Ascent Vehicle.
And you may or may not be willing to sacrifice the redundancy.
I mean, this is all about risk, right?
And exclude those or keep out spares that you cannot touch because if you can't, then you can't get back home.
So you just don't count those in your spares pool if that's the case.
But you can still do the whole analysis.
OK, so what's the bottom line here?
What's the punch line here?
So the punch line is that if you have a-- the D case here is the dedicated case, OK?
So this is where there's no scavenging, there's no commonality of spares among these elements.
You have to have dedicated spares.
And let's assume you want a 90% availability, which is not that high.
That's a relatively modest requirement.
You want 90% availability.
What it means is you need to have in this case, you can see the line just crosses below here, this crossover.
You need four spares in your initial spares pool for this electronic control unit.
You have to have at least four spares, dedicated spares, for that electronic control unit for a 600-day mission for a unit that has 100,000 MTTF or MTBF to guarantee at least 90% availability.
And you say, oh, that's not a big deal.
That's just four spares.
Well, that's just one unit.
That's just one box, right?
You probably have dozens of boxes, or even hundreds across all these elements.
And so that could be a lot of spares.
You translate that to mass and volume, that's a big deal.
So using closed-form analytics to calculate the minimum number of spares you need, this is the R case, the reconfigurable case.
You can see that you can achieve that same requirement with two spares.
You see that?
That's this line here.
It's this line here, and this is a conservative model.
This is a conservative model.
So by doing reconfigurable and common spares and scavenging, you can cut your number of spares in half.
You can cut your number of spares in half and still achieve a 90% system availability.
And then you multiply that across all the elements in your architecture.
And that's a big deal.
And then this result here-- so this is a rigorous bound because it makes some conservative assumptions, and it uses closed-form equations to calculate this.
And the details are in the paper.
And then this curve here is the simulated.
So if you simulate this using essentially a discrete event simulation several times, and then you take averages, it's actually even a little better.
This suggests you could even get away with one spare.
But it's not as conservative as the closed-form solution.
OK, so the bottom line is reconfigurable parts allow for 33% to 50% reduction in the number of required spares for 90% availability level.
But if you think about what that means operationally, it means that you really have to know ahead of time what's going to be operational when, you have to have the crew trained to be able to actually go in and scavenge.
And the equipment, the vehicles, have to be designed such that you can actually remove this stuff and put it back in relatively easily.
So that would impose some accessibility, and replaceability, and maintainability requirements.
The other option is you say, we don't want spares.
Well, then you need to do a lot of redundancy.
And then, your vehicles are going to get heavy and more complex.
I mean, those are the real world trade-offs.
Yup, Sam?
Does this potentially increase the risk for the crew of the mission if there's some sort of inherent failure in the particular part that is being used everywhere?
So I don't know if you looked ahead or you just-- I know you're a smart guy, Sam.
So this is the answer to your question right here.
So this is great.
OK, we like the fact that we can cut down on the number of spares and still meet the same availability.
But you can ask the question in different ways.
So on the left side is a kind of sensitivity analysis that says, well, maybe we're not so mass constrained.
We have plenty of transportation capacity.
So we're not really mass constrained.
But we know that designing ultra-reliable electronic boxes or whatever components is very expensive, right?
So can we decrease the requirement on the manufacturer of these boxes?
And can we go from, say, 100,000 to 75,000 MTTF?
So we're decreasing the nominal reliability of the equipment by a fourth.
We're making the job easier for the supplier of that box.
Presumably, that should be cheaper.
That should be a cheaper box to design and build.
So what will happen if we drop the reliability requirement by 25%?
And then you can see the impact here.
So the blue curve is essentially what you get for the less reliable equipment.
And then the black was the original.
So now, Sam's question.
So this is basically-- so here's your failure rate.
OK, this is your failure rate, 10 to the minus 3.
And we're varying the failure rate over a large range.
And then we're looking at system availability.
This is for a fixed number of spares.
And what's really interesting-- so when your failure rate is low-- so we're on the left side-- then the blue curve is lower, the dedicated case.
So for a fixed number of spares, for relatively reliable equipment, you're better off going with the reconfigurable or common spares.
This is logarithmic, so it's a little tricky.
So system availability is higher when you have common and reconfigurable spares, relatively reliable equipment.
But there's a crossover point.
If your boxes, your elements, your components, are very unreliable, which is on the right side, there's actually a crossover point.
And the idea is that, well, now because it's common and reconfigurable across all the vehicles, you have this really bad box that breaks all the time.
Now, the problem is everywhere.
Every vehicle, everything is affected by it.
Whereas before, it was kind of more contained, right?
And so that's what that crossover is.
So if you're in a situation with very unreliable equipment, unreliable components, you're actually worse off making them common or reconfigurable.
And you can now calculate where that crossover point is.
That's pretty cool, don't you think?
I get excited about this stuff.
I don't know about you guys, but this is real.
This is what you care-- in operations, this is-- how many spares of each kind, can we guarantee that we can successfully do this campaign, whether it's a military campaign, or you go to Antarctica.
And we heard about the Octanus Rover.
And it's just one Rover, but what's all this stuff that you need to bring with you in terms of spares to make sure you can actually have a successful campaign and have a guarantee of that?
Yeah?
I'm wondering if the reconfigurability argument also has some sort of impact on the kind of finances and manufacturability-- in terms of if you're making reconfigurable parts for different things, they may be more similarly manufactured, and therefore may represent a decrease in cost on the manufacturing side?
If that's another motivation
That's a great point.
What I will say about that is that in order to capture that benefit, you need to lock that in contractually.
So either you give all this work to the same manufacturer and say, OK, you're now making 50 boxes instead of just five.
There has to be a discount on a per-unit basis for that.
Or if it's different manufacturers, then it gets hard.
So yes, but you have to have the kind of business arrangements that will allow you to capture that value.
EPFL-- were you able to follow this?
I know this is pretty detailed discussion here on these curves.
Is it-- Maxim, you're sort of shaking your head.
Is it clear?
[INAUDIBLE]-- I can give you a concrete practical about everyday life example that you probably know about in this particular view of [INAUDIBLE].
But imagine you go with your car on vacation for two weeks, and your children all need Pampers [INAUDIBLE].. And you can take them with you from your store.
Well, you know how they are.
They are more expensive but reliable.
But you don't know how many you're going to consume.
But you get a certain minimum amount per day, and you hope you don't go over the limit.
Now, it takes lots of volume.
So you have to trade off your space because you have fixed space in your car.
You cannot take more than the volume available.
And so this is exactly the kind of reliability curve you have.
You can take too little with you.
You have to buy the wrong stuff when you get there, wherever you are.
And then it might have leaks and failures and not worth it.
So you'll need more than what you thought initially and cost more as well.
So actually, this is extremely important in everyday life
OK, so I like your-- I'm past that stage with my kids several years
Me too
But I'm going to amend your example in the following way.
Basically, if you had two kids, and they're very different, like one is really tall, and their same diapers will not fit, then you have a problem, right?
But if they're twins, or if you buy stretchable diapers that have a huge range, then you can cut down, right?
So you have to have like different kids in the car.
Then it works.
I think then the example works.
OK, let's see.
We're kind of running short on time.
So one more example.
This is based on the doctoral thesis of Jeremy [?
Oktay.
?]
He's a flight test engineer.
And the question that he looked at was robustness of degraded airplanes.
So the idea here is that some systems are going to be used longer in long, ultra-endurance kind of systems where you do not have the option of repair.
You don't have the option to land, repair, and fix the system, and bring it back to its pristine everything-is-working state.
And on the right side here, you see some examples of-- these are real systems that people have worked on, or are working on, or are thinking about that have this situation.
The DARPA Vulture is a program where a UAV would stay aloft for five years with no landing and a repair allowed.
So you can obviously see it's got solar panels, and I think it has fuel cells as well, if I'm not mistaken.
So how do you achieve that?
This is an example from Antarctica flying several UAVs to map the ice sheets.
This was a gap solution while the iSat satellite between iSat I and iSat II.
And then here's a human colony maybe on the moon or looks like the moon.
So we have this dome here.
This crater has been covered with a dome, and we have a greenhouse inside.
So life support systems that have to be super reliable for a very long amount of time.
And so the question that Jeremy looked at is, how do we design and optimize systems that have ultra-long endurance, where you know ahead of time that failures will occur?
Failures will occur, and you can't repair the failures easily.
So you have to design the system apriori, such that in those partially-failed states, it will give you the maximum residual performance that it possibly could.
And it turns out, when that's your objective function, you design the systems differently than if you just optimize nominal performance and then worry about what happens if there's failures.
And this is the second paper that I uploaded for you.
So both of these papers "The Reconfigurable Spares" and then this one are uploaded both on Moodle and Stellar if you want to take a deeper look.
So the case study here is the C-12-- this is the King Air airplane.
This is the military version of it and typically flies out of Edwards Air Force base.
And the idea is that there can be different failures on this airplane.
So this diagram, this is a so-called Markov chain where n means nominal.
And you can see the table that goes with this.
Nominal means nothing has failed, and you just do turn control.
This is climbing, left-turn climbing.
You just turn control with your ailerons, which is just standard flying.
And then different failures could occur of different elements.
So the left engine, the rudder, and the aileron could fail.
Of course, a lot more can fail.
But in this case study, those are the elements that are allowed to fail.
And so 1 means the left engine has failed, 2 means the rudder has failed, and 3 means the ailerons have failed.
And then as you move to the right, it gets worse and worse.
So state 4 means you lost your left engine and the aileron, but you still have your rudder.
And then, the worst is state 7, where everything's failed.
The left engine's failed, the rudder, and the aileron has failed.
And what you worry about is what we call availability.
Expected availability is what's the probability or fraction of probability that the system will perform above some minimum threshold, which in this case we call WM.
And then expected performance is the probability of each of these failure states multiplied by the performance that you're still getting in that failure state, OK?
And what was done here is to say, OK, we know what the C-12 airplane looks like the way it exists today.
But what if that airplane had been designed slightly differently?
Low value and high value around the baseline, a little bit bigger wing, bigger tail-- how would it impact the performance, not just the nominal, but the off-nominal performance in these failed states?
So just to show you why this is interesting or relevant-- I picked out the most interesting failure states are the intermediate ones here, these partially-failed states.
And so on this picture, you can see in red what's failed on the aircraft, and then plotting bank angle versus specific excess power, which is essentially your ability to climb in feet per minute.
And this is all calculated through a simulator, a pretty accurate six degree of freedom simulator, where you can actually modify the airplane.
It's a very cool open-source simulator that Jeremy modified where you can actually fly the airplane simulated, and you can change the plane during flight.
Like you could grow the wings as the plane is flying-- like morphing wings and things like this.
And of course here, you fail parts of the airplane during flight.
And then, the flight dynamics are automatically the physics, automatically you have to adjust to that failure.
So this is not like spreadsheet-type analysis, just so you know.
So if you're inside the safe region here, then you can keep the bank angle between-- we defined it between 25 and 35 degrees-- and you have positive rate of climb.
Then you're still in the safe region.
And each of these points here is a slightly tweaked version of the baseline airplane.
So what's interesting here is in some of these states, you're outside of the safe region.
So a small difference in the design of the nominal airplane will mean the difference between losing the plane or still being able to fly.
So these points that are out here, these are the interesting points because they fall outside the safe region.
And the points inside are inside the safe region.
Everything is the same except the airplane geometry has been tweaked.
And then of course, state 7 everything has failed.
You lose the airplane.
There's no way to recover the airplane.
So in some cases, it's very clean.
It's very clear what will happen.
And these intermediate failure states are the interesting ones because small changes in the upfront design make a big difference in how the airplane will perform in a partially-degraded state or partially-failed state.
So the other thing you can do then is you can do a sensitivity analysis and say, well, how sensitive is, for example, expected performance to the various design variables in the airplane?
Like rudder cord, and I'll talk about vertical tail here, and the engine failure rate, and so forth.
And the difference between the yellow and the green is yellow is an eight-hour mission.
So this is a typical eight-hour mission.
You fly, you come back.
And if something's failed, you just repair it.
In the 20,000 mission, 20,000-hour mission, you can't come back and repair.
So what you see here is interesting.
It means the sensitivity of performance to these design decisions depends on how long you will operate the system.
And because the longer you operate it, the more likely is that it will see these partial failure modes, which will then influence the degraded or the residual performance of the airplane.
And the most interesting parameter here, if you just look at this diagram beside the engine failure rate-- so the engine failure rate matters a lot when you fly a 20,000-hour mission.
But if you look at the cross these parameters, which one of these is the most interesting?
Which one of these is the most interesting parameter?
Let's see at EPFL.
Can you see this diagram?
Which one of these parameters is the most interesting?
Go ahead
The vertical tail?
Why?
Because it has different behavior for the lifecycle and just for the single mission
Yes.
So if you're only going to fly short missions with it and then land and repair, you see the sensitivity is off to the left, slightly negative sensitivity.
So you could actually make it just a little bit smaller.
But if you're going to fly very long, you should actually make it bigger.
And this particular airplane has an undersized vertical tail.
It has a Dutch roll mode, where it has a yaw damper that was added later.
And so if you're going to fly for a long time period, you're going to see a failure mode that will then exasperate that particular failure mode.
So what it means is that if you're actually going to design this airplane for a 20,000-hour mission that you would design a much larger vertical tail which penalizes you in the nominal.
It penalizes you in nominal operations, but it will benefit you in these partially-failed states.
Does that make sense to you guys?
You guys were laughing, or smiling, or thinking about this
Yeah, it does
So have you flown this airplane?
Or do you have experience in it?
We did a flight test course, and we tested that troll
OK, so this brought up some fuzzy memories or--
Yeah
I mean, it's basically the wing tip does like a figure eight.
That's sort of classic
Right
That's the classic manifestation of Dutch roll
Yeah
But if the Dutch roll is too big, you can actually lose the airplane.
You can induce an instability.
So this is a big deal.
Yeah?
Would that mean that the vertical tail is sized properly then?
Because you can make it smaller and get a little better in this area, or make it bigger and get a little better in this area.
Because It seems like the vertical, like, other things--
It's fine with the yaw damper.
It's fine with the yaw damper.
But actually, there has been an airplane lost.
There has been an airplane lost in flight.
And the accident investigation showed that it was basically a Dutch roll mode that became unstable.
And so the point here is, if you have a failure, then you're in trouble with the small vertical tail.
But if you don't have a failure, then you're probably OK. And that's the whole point of me showing you this-- is to make the point that if you are going to deal with a system or operate a system that is going to have a very long operational life, long endurance, without the possibility to land, repair, and send it back out-- so it's going to be perfect and pristine on day one.
And then gradually, stuff will fail, and that's the case for infrastructure, bridges, spacecraft past Neptune, airplanes that are going to be aloft for five years.
You just have to design them differently.
You just have to design-- and this is hard for engineers to think about-- that I'm actually designing for failure.
I am designing a system expecting that it will partially fail, not completely, but partially fail.
And by taking that into account, the design looks different.
This is kind of non-conventional.
OK, final thing, post-flight review-- what happens at the post-flight or post-launch review?
PLR, PFR.
So essentially, what you do is you review the telemetry from flight.
You compare against your predictions.
You find repair any failures.
You secure the data for later use.
And then you initiate the detailed commissioning and hand-over to Operations.
And here's a detailed list of entrance and success criteria for post-flight review.
And I put this up here because those of you that will actually go to the CanSat competition, this is part of the package.
You are expected to launch your payload, do the flight, and if something goes wrong, you can maybe have a backup flight.
And then after the flight, you do a post-flight review.
And good post-flight reviews are planned ahead.
You already know what data you're going to analyze, if you already have your software ready to suck in the data after the flight and really get insights out of it.
Summary here-- this is just kind of a checklist for thinking about operations.
System check-out and the lab, hangar, field-- is everything working OK?
Bring sufficient consumables, spare parts, tools, support equipment, remote control, telemetry, cameras.
Train the operators and support personnel.
Checklists for nominal operations and routine emergency and contingencies.
Think about your transportation logistics and plan enough time for a ramp-up for commissioning before operations.
OK, so that was all I wanted to cover today.
Any questions about commissioning and operations?
My main point in this lecture today was to kind of get you excited about operations and to highlight that there's a lot here.
System engineers are not done.
After CDR or PDR, you say, oh, I did my job as a system engineer.
It's up to you guys, the operators.
No, no, this is territory for system engineering as well.
And in the end, you just have to get operational experience.
There's no substitute for actually being out in the field operating systems and getting that experience.
Any questions or comments about this?
OK, so a quick reminder, a quick reminder, we're going to have our PDR.
You saw the schedule.
Monday, Tuesday, Wednesday, please log in five minutes before.
We're going to be using my WebEx Personal Room.
The link is here.
Upload your slide deck.
30 minutes presentation, and then we have up to 30 minutes of Q&A.
All right?
Questions?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Let's talk about the V-Model, here.
We are-- interesting, we have a little bit of a lag here.
So the lifecycle management is our last topic for today and that sort of completes the V. And then next [AUDIO OUT] manufacturing.
So what I'd like to cover today is essentially a definition of [INAUDIBLE] mean by that, and then focus on the lifecycle [INAUDIBLE] also known as the [?
ilities.
?]
Discuss a little bit where they come from, how much we know about them.
The bulk of the discussion is going to be a case study called Reconfiguration of Communication Satellite Constellations.
And this sort of explains the idea that system, once you brought a system into operation, that's not the end of the story.
The system is going to continue to live and change and be modified over its life.
I'll try to summarize some of the key concepts over the whole semester.
And then I have one slide on career and study recommendations, if you want to go further into systems engineering.
So lifecycle management.
So this is my definition of what lifecycle management is.
It's essentially the active engagement of-- it's the active engagement of all the stakeholders with the system, between the time when it starts to operate-- so you've done the commissioning, you've done the operations-- until you decide to decommission and retire the system.
So it's during the whole operation life.
And what you want to do, is you want to maximize the value that you gain from the system's existence.
And so life cycle management starts from the very, very, very start.
And so I listed here some activities that are included in lifecycle management.
So obviously daily operations, you know monitoring, is the system working as expected?
Training and certification of the operators.
Sometimes, you know, in long live systems, the people who were the early operators, generation one, are not necessarily the people who are going to operate for the whole life.
So you have to have new people operate.
Maybe one of the most extreme examples of that is an airplane known as the b-52.
Have you heard of this?
This airplane might be the first plane that actually has a 100 year operational life.
And there are apparently Navy-- Air Force guys at MIT, you can confirm this, that there's actually like-- the grandfather was the first generation and there's like third generation pilots now flying on the b-52.
That's what I've been told.
Did you hear this as well?
Yeah, That's correct
Do some of these people?
No, I've never actually worked on that airframe
OK.
But anyway, so that's what I mean by [INAUDIBLE] not just the initial operators, but over multiple generations.
Then the third thing here is servicing.
And servicing comes in two flavors, of preventive maintenance, you know regular maintenance, and then corrective maintenance, which we also call repair.
Dealing with small and large failures, recalls, anomalies.
This is a big deal in the automotive industry, as we know.
Increasingly protecting the system from either random or targeted attacks.
Cyber, physical, right, physical security but also cyber security is becoming a huge topic.
Sharing and archiving the data that's being produced by the system.
More and more systems produce larger amounts of data.
Well, what do you do with it?
How do you store that data?
Upgrading and retrofitting the system as needed.
So retrofitting means we're physically changing the configuration of the system-- right?-- over its life to add new capabilities.
And upgrading; upgrading can be done sometimes without having to change the hardware.
So you're doing a software upgrade, or software update, software patching.
All of this would go under upgrades.
Cross strapping the system with other system in a federation of systems.
So what I mean by this is a system may have been designed just to operate on its own, but now you're connecting it with another system.
This happened, for example, the electrical grid.
Right?
The early electrical grids were local, regional power grids.
And now we've connected them, in the US we have three major grids, the eastern, the western, and then Texas has its own grid.
Texas does a lot of its own stuff.
It's called ERCCOT.
And then here in Europe, you know the European electrical grid is now continental-wide.
It didn't used to be.
And of course, that has big implications for operations.
Reducing the resource consumption and environmental burden of the system over time, and then finally, decommissioning the system when it's time to do so.
And in many systems, this is a big challenge as to finding when is the right time to decommission the system.
And that's often the case when, you know, the operating costs exceed the value that the system delivers.
So this is a pretty long list here.
And a lot of tasks, a lot of decisions to be made.
Any questions about this list?
I don't think it's complete, but this is a lot of things you need to take care of during operations.
Here's a more graphical view of this.
So this is my sketch of the systems engineering lifecycle.
So this is part one, right?
Conceptual design-- conception, design, and implementation.
You start the lifecycle, you do something like a system requirements review, understand the mission, the requirements, the constraints.
You do conceptual design, that's where you do creativity, architecting, trade studies.
As a PDR, you choose-- so in this case, you know, we're going to go through this triangle concept and then you design all the details within it.
That's where we do modeling, simulation, experiments, MDO.
And we iterate-- sometimes you have to iterate this [?
dashed ?]
duration means abandoning the concept you chose and go for a different concept.
That's usually undesirable.
After the CDR, we implement the system, we turn information to matter, and this all happens in a technological, economic, social context.
So then, the second part of it, which in many systems-- if this is 10 years, this could be 30 years or more-- is the actual operational space.
So the system has now been tested, validated, verified, and deployed.
And now, you know, we operate the system.
Things break, we need to service it, and then I have this-- you see, there's two versions of the system.
One is solid, and the other one is kind of faint.
That's the virtual version of the system.
So one of the big concepts that's being pushed now, certainly in the US by the DOD, but also other places.
Is the idea of a digital twin.
The idea of a digital twin is that there's a physical system-- right?-- that exists, and then there's a digital version, a digital twin of that system, somewhere on a computer that mirrors exactly what the real system is doing.
So if something breaks on the physical system, well that same component fails in the digital twin.
And then, before you actually do a repair or any actions, you execute those actions on the digital twin to see whether the system will operate properly.
And so this is kind of the latest thinking, is that for any system we should have a digital twin doing operations.
Upgrading the system.
So in this case, we're adding things to it, we're connecting it.
You know, and then at some point, the system does degrade.
Because, for example, you know, materials, age, they get brittle.
Technology becomes obsolete.
You know, it gets harder and harder to maintain spare parts or the suppliers of the original spare parts went out of business.
So these really old legacy systems can be very expensive to operate.
And then at some point you liquidate and it's the end of the life cycle.
So I've already told you about one example of this, which is the space shuttle.
And so I'm not going to belabor this again, but I think it's a great opportunity to learn.
The space shuttle had a 10 year design life, roughly from 1971 until '81, first flight.
And then we operated the shuttle for 30 years and we had a total of 135 launches during that time.
We spent $192 billion dollars on the shuttle program.
And if you average that, it's $1.5 billion per flight, which is a lot more than was planned.
But just yesterday here at the EPFL, we had the annual meeting of the Swiss Space Center.
And one of the astronauts here, Claude Niccolier and his colleague, Professor [?
Mehron ?]
gave a great talk about the Hubble and the servicing of the Hubble and the amazing things we've learned through it.
And you guys at MIT, Jeff Hoffman, was part of at least the first servicing mission as well.
So there's incredible things that were done by the shuttle.
And so I think we need to acknowledge that, not just the fact that it was very expensive.
It accomplished great things during its life.
But you know, if we had a chance to do it again, would we-- would we come up with the exact same system, would we make all the same design decisions?
You know, we lost two of them.
Probably not, we could probably do something-- something different.
So here's what we wanted, and then finally, we got a much more complex system in the end.
And particularly my sense is that a lot of the things that made the shuttle expensive to operate during its life were things related to maintainability.
Right?
Reliability and so forth.
So the lifecycle properties, which I want to talk about next.
So I'm just going to repeat the questions, so you guys can hear as well.
So the question was about reusability.
So the shuttle was-- the orbiter was fully reusable.
The external fuel tank was not reusable.
Right?
And then the solid rocket boosters were partially reusable because they had to be stripped down, you know, and rebuilt, essentially, for every launch.
The idea of Blue Origin and then Space-X Stage One, flying it back to the launch site, is absolutely the fact that if you can make it reusable, then you can amortize the capex in that element over multiple flights.
And it should drop, you know, the cost by a factor of five or 10 or more.
Now, the devil's in the details, right?
So if you start and restart an engine multiple times, you can have a big transient every time.
So was this system, in fact, designed to withstand these transients?
Can it handle multiple starts?
What does it do to the materials?
And so forth.
And so to really model this ahead of time, and then test it over the cycles is the key.
So in the shuttle, the big problems-- the two major subsystems in the shuttle, in the orbiter, in particular that made it expensive and the difference between the top picture and the bottom picture was a, the TPS, the thermal protection system.
You know, every tile has hit a different curvature in geometry, so really inspecting every tile, replacing tiles, making sure that the TPS-- because just one weakness in the TPS could basically be fatal on re-entry.
And so, TPS was very difficult and-- you know, as opposed to an ablative heat shield that you just sacrifice the heat shield completely.
And then the other was the main engine-- the shuttle main engine.
Originally, the idea was to only do very deep inspections and disassemble-- the disassembly of the shuttle main engine after the first few test flights, and then fly multiple times without having to really re-inspect or disassemble the engine.
So they did the disassembly of the engines after the test flights and then they just kept doing it for every flight, where the original intent was to only do it for the test flights.
And so the shuttle main engines were the second big cost driver in operations.
So re-usability is a great idea, but to the degree to which re-usability actually happens in reality, that's really about detailed design decisions you make.
OK.
So let me move on then.
When it comes to really understanding lifecycle, I do want to point you to the 15288 standard, which is the ISO 15288 standard, which has a fairly complete list of system lifecycle processes.
So this is, essentially the system and software engineering, system lifecycle process standard, that has a whole range of processes that are described in quite some detail.
And they're not just the technical processes shown here on the right side, we focused a lot on those technical processes in this class, the stakeholders, the requirements, the architectural design-- which is essentially the conceptual design-- but there's also the project processes, right?
Executing the project agreement processes.
So this would be negotiating contracts, you know supply contracts, acquisition contracts, and then all the organizational things you need to do to create the right organization to execute these projects.
So what's nice about the 15288 standard is that it's a pretty broad standard.
OK.
So let me talk about the ilities of the lifecycle properties in particular.
What they are and how we can describe them and how, especially how they relate to each other.
And so the paper that underlie this lecture that you need to get some more detail.
And this is one of them.
So this paper is called Investigating the Relationships and Semantic Sets Among Lifecycle Properties.
And we published this about three years ago in Delft at the CESUN Conference.
So the background on here is the following, is that as you've seen, complex engineering systems live for decades-- some of them even for centuries-- and the ilities-- by ilities, we mean properties of systems that are not the primary functional properties.
In software engineering they're often called nonfunctional properties.
And the thing that's tricky about the ilities is that you often only can observe them during operations.
So you know, you can test systems in the short term, you can see does it work?
Does it fulfill its function?
But whether certain ilities are present, it often only shows itself over time.
And so most of the research that's been done and sort of quantifying the ilities, has been looking at these properties one at a time.
So the questions we wanted to go here, wanted to answer is, which of these lifecycle properties are more prevalent than others?
You know, the top 20.
And then especially, what's the relationship among lifecycle properties?
Do they form what we call semantic sets?
And then how could you use this information?
And so here, we're going to do this using two different methods.
The first method is what I call prevalence analysis.
We're going to look in the literature and on the internet, how frequently these lifecycle properties show up, are mentioned, how much we know about them.
And then the second method is a cognitive method.
Where we ask people to give their understanding of the lifecycle properties and put them into a hierarchy, and then we'll compare the results.
So here are essentially the results from the prevalence analysis.
And this is ranked according to the number of journal articles.
This is scientific papers written, where this particular key word, by quality or reliability, shows up in the title or in the abstract of the paper.
And you can see this is in units of thousands.
OK?
So quality is number one.
Right?
Quality is number one.
The most scientific papers are written about quality.
And you can see, it's almost a million journal articles.
This is-- I'll show this to you over time, but since 1884 is the first year.
And the databases that we use for this are called it's Compendex and Inspect.
These are actually combined-- if you go to a website called engineeringvillage.com, this is sort of a master database for scientific papers and engineering.
That was the basis for this.
Number two, reliability.
Number three, safety.
Then flexibility, robustness, and so forth.
And the other bar, the gray bar, is if you Google essentially for this particular keyword and-- you know, keep in mind, this was done about five years ago-- this is the number-- the millions of hits that you will get.
And, so you know, there's a factor of 1,000 difference here between journal articles and hits on internet.
But still, you can sort of compare the two.
And so, in some sense, the black bar is the totality of scientific knowledge about this lifecycle property.
And then the gray bar would be the amount of information or uses-- usage of that knowledge, at least as far as it's represented on the internet.
And what's interesting is, in some cases the black bar is smaller than the gray bar, which means that common usage is [?
leading.
?]
So sustainability would be an example.
Everybody talks about sustainability, companies have-- they say, our system is sustainable because, you know, it uses less resources, it produces less emissions.
But the actual amount of scientific knowledge, the actual amount of research as to what really is sustainability and how do you design for it, is actually smaller than the usage.
And then there's areas where it's the opposite.
Like for example, modularity.
Right?
You see that in modularity.
So in the relative sense, academic interest is leading.
We see there's quite a lot of literature on modularity.
In mathematics, you know, modularity and software, modularity and design, but most people-- you know, modularity per se is not so interesting and so exciting to the general public.
So keep in mind that for some ilities, there's an imbalance essentially, between our understanding of-- the scientific knowledge, our understanding to be design, how to design for it, and you know, how frequently at least a keyword is used.
Now here, this is a little bit more-- this is a little bit harder to see.
This is essentially looking at these lifecycle properties over time.
So what this graph those you know is starting here in 1884, cumulatively the number of journal articles published about each lifecycle property.
And roughly what way you can think about this is that there's some lifecycle property, the top four-- five that we've been actually working on for a long time.
Even safety, you know, there's some interesting articles in the 1890s about safety for example, in mines.
You know, coal mines.
There's an article about how does the impact of lighting-- better lighting-- on safety and productivity in 1890.
And so they show that just providing better lighting, actually has a dual benefit; fewer accidents, fewer fatalities, and better production output.
And so that's-- and this was a scientific study that was done by comparing data in different mines and actually performing some experiments.
Then you have a group of life cycle properties that we only started really thinking about and publishing about I would say, around World War Two.
Such as usability, maintainability, and my interpretation of this is that especially during World War Two, a big difference made logistics.
How easy was it was it to use different weapons?
How easy was it to maintain equipment?
It became a huge determining factor in the outcome of the war.
You know, for example, in North Africa in the North African theater, you know tanks, trucks being exposed to the sand and so forth.
So people really started thinking about the military, started thinking heavily about the importance of maintainability in the design of these systems.
And after the war then, a lot of these concepts like usability and maintainability started spreading into general civilian life and products and so forth.
The third type of ilities are the newer ones that we've done research on just since the 70s, in the last 30 years.
So here in this category I would put things like sustainability, recyclability, evolvability, even interoperability-- which means the ability of systems to connect together and work across system boundaries.
Those are pretty recent lifecycle properties and we're still actively researching them.
So another way to show this is by essentially making a network of these lifecycle properties.
And so the way-- this is still all using this prevalence data.
Essentially what you see here is a network diagram that has these lifecycle properties in a network relationship.
Let me just explain, so the size of the nodes relates to how much knowledge we have.
Essentially the height of the bars that we saw earlier.
And then the strength the thickness of the line relates to the strength of-- really how closely are these two ilities related.
And the way this is calculated it is using the so-called 2-tuple correlation.
So you take, essentially, articles that have-- you look at articles that have, for example reliability and maintainability in the same article.
Right?
And then you divide that by the total number of articles on reliability and or maintainability.
And that's a ratio between 0 and 1.
And then this graph was produced with a cutoff strength of so.
1, right?
So if there's more than 10% of articles list these two properties together, then there will be a line here.
And the stronger the line, the thicker the line is, the more closely, the more often these concepts are mentioned in the same article or the same piece of work.
So that reflects the strength of relationship.
Now, what's interesting is when you first look at this, you don't see much.
But after you look at this for a while, you start to realize a few things.
First of all, in the center of the graph we have the classic ilities of engineering.
Quality, safety, reliability, and I would argue flexibility as well.
Those are the top four that we saw before.
And then around the periphery of the graph, we have lifecycle properties that are more recent.
We haven't really thought about them too much, we don't fully know how to design for them yet.
And if you look at them in groups, I will argue that there are three major groups here.
So the first group in the upper left is things like maintainability, durability, reliability, quality, and so forth.
So this is all about, is the system made well, with high quality, particularly early in its life.
Right?
Then we have a group here on the right, and this is about, is the system easy to change.
Is it easy to change the system configuration-- flexibility, extensibility, modularity and scalability, interoperability.
Those are all different sub-flavors, if you want, of being able to modify the system over its life.
And so that's a group of ilities that that goes together.
And then the third one is resilience, robustness, quality, safety.
And so this is really related to performance of the system under different types of uncertainty.
You know, either environmental variability or failures in the system, and the ability of the system to withstand, or at least perform-- you know, have good residual performance even in the face of failures.
So that's, I think, an important way to think about it.
And when you're writing requirements for systems-- like the system should be resilient.
Then this helps understand well, what are the other properties that are linked to it and that maybe support that?
OK, any questions here or at MIT about-- this is sort of method one, is get data about the lifecycle property and put them in relation to each other.
Any questions?
Johanna, any question there?
I have a question.
So is this chart that you have now actually used in the initial design and like conception [?
con-ops ?]
to like try to tease out the interdependencies, or is it too abstract and just more of an educational tool
It's really at this point-- you know this is fairly recent.
And this was just done in the last few years.
So I don't think this has fully penetrated system engineering practice yet.
But the point-- the point that I want to make here is that there's a huge-- there's a huge gap right now between when people in briefings you know, either at the Pentagon or at a corporate headquarters say, you know we want a sustainable product or we want-- we're a software company, we do optical networking and we want we want to have the most resilient design of the industry.
You know, they'll put that as a goal for the project.
The question then is, well OK, that's fine.
But what does that really mean?
What does it really mean, resilient?
You have to operationalize that definition such that you can derive from it lower level requirements that you can actually go and design to, that you can test for.
So what this helps you to do is understand what may be supporting-- what are supporting elements that will be related to resilience.
You know, what are supporting concepts, supporting lifecycle properties that this property that you're looking for is linked to?
Right?
So it's an evolution-- you know, we know how to design now for speed and energy efficiency, and you know, the sort of things that were really hard to do 20, 30, 50 years ago.
It's pretty standard practice now.
You know, how do you design for optimal interoperability?
We don't quite know yet, but we're finding our way.
And so what I believe is that especially the lifecycle properties on the outer periphery of this chart, those are the ones that need more work and those are the ones that we're really learning how to operationalize.
Does that make sense?
Yeah, thanks.
That makes sense
OK, good.
Let me talk about the [?
message ?]
too.
So this was basically trying to get to-- how do people, how do humans-- there's semantics, semantics means the meaning of words, right?
Semantics is the science of the meaning of words.
How do they interpret these lifecycle properties?
And so humans have a deep and possibly varied understanding of the semantics of these lifecycle properties.
So what was done here was that a list of 15 life cycle properties, many of them are overlapping with the ones you saw earlier, were presented and then the challenge was to-- the question that these four groups-- 12 participants, four groups-- had to say, is there a hierarchy here?
Are some higher level lifecycle properties are some of them lower level properties that support these.
So there was a round one, find the parent-child relationships, describe these.
Interviews, and then the second round.
So here's essentially what was the results of the first round.
So four different groups, so each of them basically came up with its own version of a hierarchy.
They didn't talk to each other, they were firewalled from each other.
And in all cases this notion of value robustness came out on top.
So value robustness means the system should deliver value, right, to the stakeholders.
Despite failures you know, environmental changes, so value delivery of the system is the top, the most important thing.
Then what's different here is how do you achieve that?
So for example, you see group one had robustness and changeability.
So robustness typically means even if you don't make any changes to the system, it should continue, you know, it should be survivable.
It should be versatile.
Changeable means you actually modify the system over its life.
And then we have lower level properties like modularity or reconfigurability.
And you know, there were differences between the groups but not as big as you might think.
So in the second round, as a result of the second round, then this so-called means to end hierarchy was constructed.
So means means, you know, these are the enabling lifecycle properties and ends means this is the final, sort of this is really what you want to achieve.
And so this was the result of that.
So again, at the top we have this notion of value robustness.
And this is achieved by a combination of survivability robustness and changeability.
And then these, in turn, are achieved by lower level lifecycle properties.
And then at the lowest level we have things like interoperability, modularity, reconfigurability, and so forth.
And the difference between the solid lines and the dashed lines here is that if it's a solid line and these are directed arrows, that means that three or four out of four groups, right?
So the majority of groups had this is a particular parent-child relationship.
And if it's a dashed line it means only two out of four.
And if it's only one out of 4, it's not shown here.
OK?
So this is kind of a combined result across the four groups.
So you know, what can we take from this?
So first of all, lifecycle properties are absolutely critical.
You'll find us in mission statements, you know, you'll find it in even in advertising.
Right?
A lot of companies say we have we have a robust solution, we have a sustainable system, we have resilient networks.
So it really is a huge, huge selling point.
It's very critical.
What I encourage you to do as system engineers is take a critical look at this and say, well how resilient is resilient?
You know, how many subsequent failures of the system can you tolerate?
Sustainability; what does that mean in terms of kilowatt hours per hour of usage?
You have to get down to the details to start quantifying what these lifecycle properties mean and compare them among systems.
Despite differences, the two methods that we just looked at-- so one is the prevalence analysis and the other one is the human semantic exercise-- despite differences, the high level conclusions were similar.
So some ilities are closely related to each other and form semantic sets, meaning they're tied together by both [?
synonymie ?]
or polysemi relationships.
[?
Synonymie ?]
means they essentially mean the same-- they're synonymous-- they essentially mean the same thing.
polysemi means-- it is one word, but it means you can have possible sub meanings.
Right?
So the idea of groups of semantic sets.
And those groups essentially are robustness-- so this is the ability of the system to perform its job despite either internal or exogenous disturbances.
Right?
That's robustness.
Flexible or changeable, which means that you can modify the system easily.
You know, if you operate the system for a while and then you realize, Ah, I need the system to do something else.
Or I need to adapt it, or make it bigger, or make it smaller, or add some function.
So you would modify the system and that's flexibility or changeability.
And then resilient and survivable is very specifically the ability of this system to continue performing despite failures or attacks.
Right?
That's what resilience really means.
And so those seem to be the big three semantic clusters that we see in life cycle properties.
And then the third point here is there appears to be a hierarchy of life cycle properties with two or three levels.
Where we have the lower level properties of systems like modularity, for example.
Really your customer probably does not care about modularity, right?
If you advertise the modularity of the product, some of the more educated, some of the more technically savvy customers, they may understand what that means.
But most of your customers really won't appreciate modularity because it's kind of a low level form-oriented technical property of the system.
But what they will appreciate is the ability of the system to be reconfigured or adaptive for different pieces, so interoperability, modularity, et cetera are low level lifecycle properties that act as enablers of a higher level lifecycle properties.
And so future work-- future work in this area is to both just to apply these methods to a broader set abilities, larger group of test subjects and more data, but also from a practical standpoint to operationalize better.
How do we write requirements?
How do we actually design for the lifecycle properties like resilience, flexibility, changeability.
How do you really design it?
So go from the keyword to real engineering by operationalizing the sub-attributes or factors in the system.
OK?
So that's a quick summary of life-- you know, I could go on for hours about this.
I'm very passionate about this topic and I will say that people who have dealt with large complex systems you know, whether you're operating the transportation system of a city or in airline operations, or you're running the IP infrastructure of a major corporation.
These words, these words are real.
These words are you know, real dollars, real challenges.
This is really where the action is in a lot of these large complex systems.
OK. Any comments or questions about lifecycle properties?
What they are, how they relate to each other, why they're important.
Voelker, did you want to maybe say something?
OK. All right.
So let me talk about-- try to make this a little more real.
Let me talk about a case study, a very specific case study about communications satellite constellations.
And this is the second paper that underlies today's lecture.
And so let me give you a little context for this first.
So this work here was originally published about a decade ago, in March 2004.
And that was a few years after the Iridium and Globalstar-- these the two well known communication satellite constellations-- had been launched.
And I have to give you a little context, when I first came to MIT in the mid 1990s, there was a huge interest in this area of satellite constellations.
You know commercially, scientifically, in fact, the impression was there were so many applications for satellite compilations filed, that we're going to have so many satellites up there you won't even see the sun anymore.
It was just like thousands and thousands of satellites.
And the Iridium and Globalstar were really the first two constellations that were fully developed, launched, and both of them failed commercially within a very short time, a couple of years.
And so after this happened the whole market and interest in satellite constellations collapsed for a long time, until about two years ago, two or three years ago.
Now people talk about these constellations again.
You know, constellations of [INAUDIBLE],, Iridium Next.
You know, there's sort of new enthusiasm , new wave of enthusiasm for constellations.
So this paper, and this case that I want to tell you about is really about the first wave.
And in terms of the ilities, the one that that I'd like to explain to you about here is flexibility and scalability.
Rather than thinking about a system as something that you design and build all at once, how do you design a system such that you can gradually deploy it?
We call that staged deployment approach.
And what's the benefit of that?
So here's some pictures you see on the right side, this is what the original iridium satellites look like.
They actually use phased array antennas-- and actually both use phased array antennas.
So you have individual elements here.
And by differentially phasing the signal, you can actually steer the beam.
This was a very new technology at time.
And then the Globalstar satellite shown at the lower picture.
Here is a little bit of that data.
So both of these were launched in the late 1990s.
Iridium has 66 satellites, Globalstar 48.
Iridium is a polar constellation, so the satellites go almost directly over the poles.
Globalstar is a walker constellation, so inclined.
Doesn't quite give you full global coverage, it's about plus or minus [?
78 ?]
degrees, so you can't use Globalstar at the poles.
The altitudes are a bit different too.
Iridium is at 780 kilometers, while Globalstar is at 1,400.
Which-- I know at least one or two of you are working on the Van Allen-- the Van Allen Belt Commission.
Right?
[?
The cube stats ?]
to measure the Van Allen Belts.
Somebody mentioned that today, who was that?
No?
Did I hear that wrong?
It was mentioned, right?
So who was that?
Arnold, and he's not here.
Ah, see.
So 1,400 kilometers, are you actually pushing, you're starting to push the lower edges of the Van Allen Belt.
So one of the big, you know, in some sense, it's easier to be higher because you need fewer satellites to cover the whole earth.
But the higher you go, the more exposed you are to the radiation environment.
You get closer to the Van Allen Belt, so that's the big trade-off there.
You know, the mass of the satellites, they're between 450 and 700 kilograms, transmitter power, around 400 watts.
You know, which is not that much, if you think about it, right?
That's four very strong light bulbs, the old style light bulbs.
And then what's very different, again, is the multi-access schemes.
So Iridium used time division multiplexing and Globalstar, which was supported by Qualcomm, used essentially CDMA.
So you don't chop your frequency band into separate channels, you use the whole frequency band and then you use a pseudo random access code that essentially de-convolves the signal for each channel.
The number of channels, about 72,000, about 120,000 duplex channels so that you can actually-- duplex mean you can carry on a two way conversation as opposed to just be asynchronous.
And then you can see the data rates, quite low, like per channel.
4.8 or kilobits per second, 2.4, 4.8, 9.6 for Globalstar it's enough for having a conversation.
And total system cost, Iridium was about $5.7 billion dollars and Globalstar about $3.3 million.
Not including the cost of the ground stations.
Both went bankrupt relatively quickly after they launched.
However, they've been operating really since then, right?
Since this time.
Iridium Next is currently under development and is scheduled to launch in 2017.
Globalstar is publicly traded and actually valued as a company at $1.9 billion dollars.
So they're actually both-- you could almost argue that they were a decade ahead of their time.
And I want to tell you a little bit about the story of, especially Iridium.
So here's a couple of press releases, so things that have been written in the press.
So look at this one, 26th of June, 1990.
Motorola unveiled a new concept for global personal communication.
[?
Bases ?]
a constellation of low earth orbit cellular satellites.
August 18th 1999, nine years later, last week Iridium LLC filed bankruptcy court protection last investments are estimated at $5 billion.
So the question is why did it happen?
The technology actually worked quite well, it was not a technological failure, it was a business failure-- but I would argue a system's failure.
To think about the problem differently, so the fundamental challenge is to properly size the capacity of a large system.
So if you're designing, you know, a car factory, a new power system and for a future uncertain demand, it's very difficult to do this because demand is uncertain.
So is it better to oversized the system or are you conservative and you make it smaller?
Market assumptions can change, right?
In a seven to eight years.
So essentially the v, right?
Like getting back to the v. Each of those two systems, Iridium and Globalstar, it took them essentially a decade, almost 10 years-- right?-- to go across the whole v. And when you make your requirements, your stakeholders, all the stuff in the upper left portion of the v, there's so many years that elapse between when you make those assumptions, you write those requirements, and when you actually go to market a lot of things can change.
And that's fundamentally the challenge here.
Just to illustrate this for you, showing some data, this is cellular this is not space space, this is on the ground.
Cellular subscribers for mobile phones.
I know this is hard for your generation to sort of understand this, but we actually didn't have mobile phones or there were these clunky bricks in your car.
You know, it's really remarkable what happened.
So look at this data, in 1991 which is when the system was just being developed, there were less than 10 million mobile phone users in the US.
It just wasn't-- it was very expensive, it was just not widely-- that technology hadn't been, the networks weren't there.
You know, so the green bars where the forecasts that was done in 1991, as to how quickly the mobile user market would evolve in the US.
So their prediction was that by 2000, a decade later, there'd be just shy of 40 million users.
OK?
Now the dark blue bar is the actual evolution.
So actually by 2000 you know the US now has 310 or so million inhabitants.
So by 2000 there were 120 million users.
So the forecast that was done 10 years earlier was off by a factor of three.
That grounds, the terrestrial mobile networks, developed three times faster than had been predicted.
Now that's great for the terrestrial people, right?
AT&T and Comcast-- not Comcast, Qualcomm, et cetera.
But the problem is, of course, that because you know ground based communications was so much easier, a lot of the market that had been anticipated for the satellite based communications was essentially eaten away by this competitor, by the ground based competitor
[INAUDIBLE]
Yeah, OK. Can you still hear it, MIT, are you still with us?
Yes
Yes, you are.
OK. Voelker wants to say something
Actually, the figures that you showed [INAUDIBLE] and for once, Americans did not look over the pond.
Because at that time in Europe, the GSM systems came really strong and very quickly.
In the early 90s, you could have already had your Sim card in your cell phone [INAUDIBLE] and in fact in America then, you could only buy the telephone with all the Sim card in-- you had to buy the whole system.
And so as the system was controlled by the large companies and not by the users, the large companies, Motorola, thought that they could impose their satellite system on the markets.
And in this case, the country to the-- video recorders, hi fi systems earlier, Europe went much faster than America.
And actually European GSM market-- cell phones, as you know them-- went much quicker.
And then Nokia, which is actually European, overtook and it took probably the next decade for Americans to catch up.
So finally here this was, a combination of people had the product and they just couldn't get in the US.
So they were limited to their own markets, and not to sell their products to the rest of the world as they had done in the past
Yeah, good point.
And in fact, you know one of the things that-- I think both Globalstar and Iridium had to do-- and this was a very late decision, just in the last couple of years here before launch, 97, 98, is to make their handsets dual use.
Like that if you were in an area where there was a cellular network, the phone would switch to that because that would be cheaper, and if you didn't have cellular network access it would automatically try to communicate to the satellite.
So there was a lot of-- there were a lot of issues that came up by essentially the markets, I think in Europe, your point, well for-- about the GSM, and in the US just developing quite differently than had been anticipated.
OK so I want to give you a little bit about sort of economics.
In the end a lot of this is driven by money, by economics.
And so this is satellite economics 101.
The key question here is how expensive is it-- what's the cost, and then what is the price that you can charge for one minute of service, right?
One minute, one unit of service.
And so look at this equation here.
This is CPF, stands for cost per function.
And so in the numerator we have the lifecycle cost, which is your initial investment.
And then we will essentially capitalize that with some interest rate k. Plus then, for each year of operation, you have the operational costs for that year you have to add.
So that's your initial-- that's your development costs, your manufacturing costs for the satellite, including the launch costs.
And then the ops costs would be operating your ground stations, your networks, any replenishment costs, they would be in your ops costs.
And then you divide this by what's called here the number of billable minutes.
That doesn't mean you're actually going to bill people, it means just they are potentially billable minutes.
So that's the capacity of the system, [?
see ?]
the best times you know, 365 days, times 24 hours, times 60 minutes, times what's the load factor.
So the load factor is essentially the capacity utilization of the system that you anticipate.
OK.
So that's the basic equation for calculating CPF, cost per function.
Plug-in some numbers here.
These are the numbers that have been assumed, you know a $3 billion investment, 5% interest rate, $300 million per year of ops cost, 15 year life cycle-- this is the capital T, over 15 years.
100,000 channels-- that's your capacity.
Number of users in this case, so the load factor is simply the number of users times the average activity per user.
In this case, it seems to be 1,200 minutes per year, about 100 minutes per month.
OK?
So that gets you-- that gets you a CPF of $0.20 per minute.
And based on that you can-- the business case was made based on these kind of numbers.
And so 3 million users, just 3 million subscribers, right?
Not 3 million users at the same time, that number would be much smaller.
That number can't be bigger than 100,000 because that's your capacity.
So for example, if you run-- if you run a fitness club, right?
At any given time in your fitness club you can have 50 or 100 people actually working out, but your number of customers, your subscribers, should be 1,000 or 2,000.
And if they all show up at the same time you're in big trouble, right?
So that's the difference between number of users or the number of subscribers and number of active users at any given time.
That's a big part of managing these kind of systems.
However, what actually happened is the number of actual users grew much slower.
So if you plug-in some different numbers, let's keep all the numbers the same, except for the subscriber base-- the number of users.
In this case we're going to assume 50,000 users instead of the three million.
This is closer to what they had after about a year of operation.
Now your CPF goes to $12 per minute, which is noncompetitive.
Right?
Except for some extremely, like, military applications or making some emergency phone calls on an oil rig, you know, on the ocean.
Most people at the time-- now and at that time-- would never pay that for one minute of service.
And so that was the fundamental problem, is that the user base did not materialize as fast as planned.
Therefore this cost per function was way higher and they did charge, you know, $3 to $5 per minute of usage of the system, which as you try to squeeze your existing users more, you're not going to get the ramp up and scale up in the system that you need.
That was the fundamental problem with the economics of the system.
So let me talk a little bit about the design decisions the conceptual design of what this design space looks like.
So fundamental-- oh, the other thing I should mention to you, what was interesting is after the bankruptcy, both of Iridium and Globalstar, both of the chief engineers for both of these systems took refuge at MIT.
Essentially, they came to like-- Joel Schindall, who was the chief engineer for Globalstar, great guy, both very competent people, came to MIT-- still there, still a professor there.
And then Ray Leopold was one of the three architects for Iridium, also came to MIT during the time.
So I had extensive discussions with them, and what you see here on slide 25 is one result of those discussions.
Which is, the key design decisions that they have to make, fundamentally, there is-- you won't be surprised to see this, this is the magic number seven, right?
Seven key design decisions.
When you design a satellite consultation for communication purposes, constellation type-- polar or walker, orbital altitude, minimum elevation angle above the horizon-- that you can communicate-- satellite transmitting power, the size of your primary antenna, your multi-access scheme.
So this is time division or code division multiplexing.
And then the last one is about the network architecture-- do you have inter-satellite links, yes or no?
Inter-satellite links means satellites can talk to each other.
And Iridium chose that, Globalstar did not.
So in Globalstar, the Globalstar satellites cannot talk to each other directly in space.
They can only talk-- it's a bent pipe system to a ground station.
OK?
So if you this is like the morphological matrix that you learned about, you just pick design decisions in this morphological matrix and you come up with an architecture.
In fact, a full factorial search of the space would reveal 1,440 different satellite architectures.
So that's on the input side, what's the output vector?
What do you care about in terms of output of the system?
Well first of all, performance.
So the performance-- this is for voice communications.
In a sense it applies to data communications as well.
It is your data rate per channel, right?
4.8 Kbps, your bit error rate, like what is the average number of bits that are wrong?
10 to the minus 3.
That's actually pretty-- not a very-- that's not a very stringent requirement.
And the reason is this is for voice communication, this is not for sending, you know, commands to a spacecraft going to Mars.
If you send commands, it would have to be 10 to the minus 10, or 10 to the minus 9.
Much, much better bit error rate.
But this is OK for voice.
And then the link fading margin, 16 DB.
This is the strength of the signal, which which will dictate whether or not you can use the phone under trees or in buildings.
So you want this number to be higher, but the higher it is, you know given the power you have on the satellite, the fewer channels you have.
So there's trade-offs.
So what was done here is to keep the performance foxed, and so that you can compare architectures, compare apples to apples.
Then we have capacity, which is the number of simultaneous duplex channels.
And then finally, the lifecycle cost of the system, which includes research development, test, and evaluation, satellite construction and test, launch and orbital insertion, and then the operations and replenishment.
So if the satellite fails in orbit, either you have to already have prepositioned to spare, or you're going to launch a replacement.
And this actually happened in both cases, both constellations.
So in order to then, connect the input to the output, you need to build a simulator or a model of this system.
And this is a high-level view of what that model looks like.
So we take our input, our design decisions, and certain constants that we assume-- a constant vector.
And cascade this information from the constellation module, takes the altitude and the minimum elevation angle, and produces-- t is the number of satellites and p is the number of orbital planes, so this is sort of orbital dynamics.
The spacecraft module calculates the satellite mass, the satellite network builds essentially the communication pathways.
The link budget will calculate the capacity of the system.
You know, given those performance constraints.
The launch module will determine how many launches do you need, from where.
And then the cost module basically calculates the total cost of the system, the lifecycle cost, and finally you get essentially a trade-off of lifecycle cost versus capacity of the system.
Let me just show you some, you know, you say, well that's tying together a lot of information into a multi-disciplinary model.
So what kind of information do you have?
You have a mix, really of physics based models-- so this is a very well known equation, the eB/N0, that this is the energy bit over noise ratio.
This is a closed form physics-based equation that tells you how much energy is there per bit-- what's the signal to noise ratio on a per bit basis.
And this is a function of transmitter power, receiver and transmission gains, various losses in the system.
And then some of the equations-- some of the information is empirical.
For example, the relationship between spacecraft [?
wet ?]
mass and payload power.
You know, if wanted to have a closed form or you wanted to have more detail, you would have to almost build a CAD model, right?
A separate model.
You'd have to-- for each of the 1,440 architectures, you'd have to manually construct an individual detailed design.
And that's not feasible, so what you do instead is you use some prior data-- and you can see here a scaling relationship.
It's not perfect, but we have error bars, we know how good it is.
Satellite mass, satellite wet mass is a function of transmitter power and, in this case, propellant mass.
So two kinds of equations.
Benchmarking.
So once you have this model, you need to ask the question, how can I trust this model?
Does it give me a reasonable answers?
And in this case, benchmarking the process of validating simulation by comparing the predictive response against reality.
So just quickly here, showing you four kinds of data.
So one is the simultaneous channels of the constellation, this is the prediction of capacity.
You can see it's pretty good.
In this case, the model is actually a little conservative.
So the blue bars are the actual planned capacities.
The red ones or magenta is simulated, and you can see that the simulation under-predicts, slightly, the true capacity of the system.
Life cycle costs, you saw that Iridium was just a bit more than $5 billion.
Globalstar was between $3 million and $4 million.
And so here we're in the right ballpark.
And then in terms of satellite mass and the number of satellites in the constellation required, we matched that very closely.
And the reason for this is, fundamentally this is just geometry.
Right?
If I can tell you the altitude and the minimum elevation angle, and I'm telling you I need I need global coverage and I want dual redundancy, so you can always see at least two satellites, it's just geometry to be able to figure out how many satellites and how many orbital planes you need.
And that's why the model, or the simulation, and reality match very, very closely.
So this gives you some confidence that this model is reasonable.
So what you can do with it is now what we call Trade Space exploration.
Which is, in a sense, what you did for the [INAUDIBLE] competition.
So this picture here, this graph shows you the life cycle costs over those 15 years, versus global capacity of a system.
Each of these blue dots represents one of those 1,440 architectures.
And you can even see on this, where Iridium actual versus simulated falls and where Globalstar actual versus simulated falls.
So both of them were actually off the Pareto frontier, which was interesting and led to quite some discussion.
And you can-- the Pareto frontier itself was, of course, very useful.
One of the reasons that Iridium is not on the Pareto frontier is that fundamentally, polar constellations are inefficient.
And, if you think about it, when the satellites are crossing over the poles, they're very close to each other.
And they're actually not crossing exactly over the poles, or you could actually collide, right?
So you offset them slightly to avoid collisions.
But some of the satellites actually get turned off when they cross the poles.
So you're not utilizing your assets super efficiently in the polar constellation.
Which is one of the reasons why they are not on Pareto frontier.
Now, the way you would use this in a traditional system engineering approach, which is traditional system engineering approach means give me the requirement, write down the requirements, and then find the minimum cost design for that requirement.
So a requirement could be we need a capacity of 50,000, for example, and then you find this intersection with the Pareto front, and that's the minimum lifecycle cost design that gives you that capacity.
Right?
And that's the system you pick, and that's what you go and build.
And that's essentially what they did.
The problem with this if there is high uncertainty is, the true requirement is kind of unknown.
The market will tell us in the future what the true capacity should be.
So how do you deal with this?
Well, you could be-- we're going to be on the safe side, right?
We're going to oversize-- so if the true demand, the true capacity that you would need, is higher than what you just had, your system is going to be under capacity, right?
Your system is going to be undersized for what it should be.
And so what happens in practice?
What do you think happens in an under capacity situation?
What would you say?
The system is too small relative to the market that you're in.
You're not capturing as much as you can, meaning a competitor will probably capture.
I mean, I guess you can change your price policy, but fundamentally, you're missing out on market opportunity.
The other situation, which is actually what happened, is that the demand is much less than we anticipated.
Your system is oversized and remember, once you launch a satellite constellation, it's not like a fleet of taxis, right?
Where you just park them in a parking lot.
You've already-- the fixed cost is very high.
So if demand is below capacity, you have all this investment here, lifecycle cost has been wasted because you oversized the system.
And the challenge is that the true requirement is kind of like this-- there's this probability, density function.
Right?
It's a probabilistic requirement, essentially.
And there's no set of experiments because of all the [INAUDIBLE] uncertainty.
It's not a systemic uncertainty.
There's no set of experiments you could do today that will help you refine the requirements.
Now they did do market studies, but again those market studies were years ahead of when the actual system was launched.
So they were unreliable, fundamentally.
So what I'm arguing here is in this kind of system where you have large uncertainties as to the true requirements, you shouldn't just guess and then put billions of dollars on a guess.
This is like playing in the casino, right?
Essentially.
What you should do is think about the problem differently.
And in this case, the answer is staged deployment, or one of the answers is staged deployments.
So you build a system such that it can adapt itself to the uncertain demand.
So you build, initially, a smaller, more affordable system.
But-- and then the system has already built into it the flexibility or scalability to expand if needed.
But only if needed.
And there are two major economic advantages to that.
One is that you don't need to spend all the capital up front.
As so you're deferring capital investment.
And you retain the decision freedom to reconfigure or not, and that's typically what we call the real option.
You've created a real option for the future.
So the question, then, is well how valuable is that?
It is worthwhile doing this staged deployment approach?
So there's probably different ways of doing this, and I just want to share with you-- and this is from the paper-- how this was done.
So step one is you partition your design vector.
So you basically decide which part of the system is flexible-- because you can't make everything flexible typically-- what part is flexible and what part is fixed, is the base, you can't change it?
So in this case usually the idea is that the satellites themselves, the design of the satellite, the transmitter power, that their protocol, and the network architecture should be fixed, right?
Those things are difficult to change.
We're going to allow the astrodynamics, the actual shape of the constellation, to be flexible.
So keep the satellites the same, only change the arrangement in space.
So what you do is you partition the design vector into the flexible and inflexible parts.
And when you do this, in step two, what's nice about this is you can then actually find families of designs, right?
You can find families of designs where the-- we have a little lag here-- that you can find families of design that share the constant parts.
That have the common-- So what's shown in this graph here-- this is again, our design space, kind of zoomed in a little bit more-- every one of these points that are connected with these lines uses the same types of satellite [INAUDIBLE]..
Same transmitter power, same antenna diameter, and all of them have inter-satellite links.
So effectively, you're partitioning the design say into subsets that share common features, and then we call this a family of designs.
And the idea is to start small, so on the left side, and then you grow the system over time right?
But only if needed.
So when you do this, you 1440- strong design space turns out to be decomposable into 40 different paths.
Now, which path would you choose here?
Well you want to be as close to the Pareto front possible.
So this is an example of all of a path.
In this path, if we started on the lower we would start with 24 satellites and then we would actually move them to a lower altitude.
And add 30 more in step two, and then you would gradually grow the constellation over time.
You see that?
This is actually not too different from how GPS was done, right?
The GPS constellation was deployed in phases, but it was not market driven, it was just sort of phasing the development for risk reduction, and spreading off the capital.
The other thing that's interesting here is that as you grow the constellation, it moves further away.
It moves further away from the front.
So when it's small, this particular [INAUDIBLE] is close to being optimal, and as you scale it up it becomes more sub-optimal.
And there's other paths that have the opposite behavior; where when small it's kind of suboptimal, and as you make it bigger, it gets closer to the optimality.
Pretty interesting.
So that's step two, find the paths.
Step three, you now have to model the actual uncertainty.
And in this case there are different ways to model uncertainty.
In this case, what we used was a GBM model-- geometric Brownian motion.
This is well-known in physics and applied to statistics, physics, this has been applied to the stock market, right?
And the idea here is that in this case, the man behaves like a particle in a fluid.
Right?
That's kind of moving in an unpredictable fashion.
So this is the basic equation for-- this is a discrete version of geometric Brownian motion, you have some trends nu, nu times delta-t, so this is the delta and demand, or the change in demand, divided by demand, so this is normalized change in demand, is nu times delta-t, so your trend times delta-t plus sigma times epsilon times square root of delta-t. Epsilon is a standard, normally distributed random variable between 0 and 1, and epsilon and sigma is your volatility, essentially.
So here's some examples, if you start with an initial demand of 50,000, which is what they actually saw as an initial demand early on, you have a growth of about 8% per year.
This is your trend.
Plus in this case, a volatility of 40% per year, which seems high but in this kind of very new technology, new system, is actually not that far fetched.
You get three different-- these are just three examples of how demand might evolve.
These are just three scenarios.
So demand can go up, it can go down, and-- GBM is very nice, but one of the downsides of GBM is there's infinitely many scenarios that you can generate because it's fundamentally-- even though it's discretized in time, it's not discretized and in demand.
So a simpler version of this, a more discrete version, is a so-called binomial lattice model.
So the way this works is again, you start with some initial demand here on the left and then you discretize this such that moving through time, you can look at different scenarios.
Right?
These scenarios could be-- the best scenario is things just keep growing.
Right?
Grow, grow, grow.
Here, time, the 15 years, have been sub-divided into five three-year periods.
So here's a sample scenario, demand goes up, goes up, and then it goes down twice, and then it goes up again in the period.
And so this is a discretized random walk.
And you can choose the numbers, so the sigma, the volatility, the amount of the up and down movement, and the probability, p, of going up, and then the probability of 1 minus p of going down, are chosen such that they're consistent with the GBM model.
So this model is the equivalent of the GBM model, it's just discretized.
So the beauty of this is that you can now simplify this to 32 different scenarios instead of infinitely many.
And each of those scenarios is not equally probable.
There is actually probability weighted scenarios, depending on what nu and sigma are that are underlined.
So now we have 40 paths, right?
We have 40 evolution paths of the system and we have 32 different future scenarios.
So what we do now in the next step, four, is to put the two together and calculate the cost of each path.
Calculate the cost of each path with respect to each of the demand scenarios, look at the weighted average of all the possible paths, and the one thing-- the one tricky thing about this is you need to build in a decision rule.
Such that, if the system-- if demand exceeds capacity the system, you're going to expand and move to the next stage, you're going to stage deploy the next phase.
And there's many different ways of doing this decision rule.
So the simplest one was chosen here, and of course costs are discounted.
So let me explain to you how this works.
So we start the simulation, so the first initial deployment.
This is our initial stage one-- right-- for the constellation.
And then we start operating for two years, and demand in this period goes up.
You see the ops cost in this case goes slightly down, this is due to the discounting.
This is a discounting effect.
We arrive at the end of our first three year period and then the question is here is our capacity of the system.
What would you do in this situation?
Let me ask somebody at MIT, make sure you guys are still with me.
So what would you do in this situation?
You're now at the end of year three.
Anybody?
We lost you for a second halfway through your statement, could you repeat?
Yes, so you've done your initial three years, you've deployed your initial constellation.
And you now have the situation shown on this chart.
What's the right decision, according to the decision rule?
What's the right decision?
OK, I'd keep it the same.
Because--
Yes, exactly
--it has not exceeded demand yet.
But prepare to change--
Exactly, you wait, right?
Because you have to lower your capacity
You keep it the same, you don't do anything.
You just keep operating.
Exactly, right.
So you have another three years, right?
During these three years, demand keeps growing.
And you now, at year six, arrive at this point here.
So what's the right decision now?
Can I ask a question about the decision to stay the same?
Would you say the same?
What's happening to the demand line?
Veronica, what's happening to the demand line?
Can I ask a quick question first, about the decision to stay the same?
Yes
I feel like by waiting until we've exceeded demand to choose to expand the constellation, we're introducing a lag between demand and capability that creates an inefficiency that may actually drive users away from the system.
And I see this kind of oscillatory effect, where you would expand, and then people would meet, and then the system would be insufficient and then people would move to a different system.
And you kind of have a yo-yo around the maximum carrying capacity.
And that doesn't seem efficient to me, so I'm wondering if you could speak to that
Yeah
So what you're essentially looking for is what I would [INAUDIBLE] decision rule.
So you would actually-- so the decision here is to deploy, right?
You've got to deploy your second stage because you've saturated the system.
What you're arguing for is before this occurs, you know like at 80% saturation, that's when you trigger the next stage, right?
In order to anticipate the saturation.
And so, absolutely, you could do this.
There are many different decision rules that you can try when you design a flexible, deployable, scalable system.
And that's part of the decision space.
So in this particular example, we just implemented the simplest possible decision rule, which was when saturation has occurred you do deploy the next stage, but not before
OK, thank you
Is that clear?
Yes, thank you
OK, so we deploy the second stage now, and you can see there's another spike here-- right-- of capital that's needed.
It's not as high as the first initial stage, but it is substantial.
So now as we deploy the next stage, our capacity went to the higher limit.
So we're now at capacity level two.
We're now operating the system.
Demand keeps growing, that's good.
Now we have year nine, you can see.
But we haven't yet saturated our new capacity.
So again, the optimal decision is-- or I shouldn't say the optimal decision, but the decision according to the rule is you just wait, right?
You keep operating.
Ah, now from year nine to year 12, demand starts to go down.
Right?
And this happens in some systems, right?
They are growing and they peak, and then things go down.
So in this case, we wait.
And then in this scenario it goes down again.
You see how that works?
So what's nice about this is you can now-- in this sense, all the 32 scenarios of possible futures, you can run them against the 40 different families of designs or evolution paths.
And out of that you can find the best path, right?
This is the path, the path that on average-- I should point out on average-- will satisfy your demand at minimum lifecycle costs, given the uncertainty models that you have composed.
Yes, [INAUDIBLE]
[INAUDIBLE]
Why did it go down?
Like, [INAUDIBLE]
This here?
[INAUDIBLE]
No, the capacity stays the same.
You don't go down in capacity.
The reason is-- this is a good question-- this particular system, a satellite constellation is dominated by fixed costs, right?
So if you wouldn't retire half your satellite, because then you would lose coverage also, potentially.
Because you've moved them through orbits that give you the right coverage, you just increased your capacity.
This is different from a system that is dominated by variable costs.
Like for example, if you operate a fleet of taxis and and for whatever reason, demand goes way down, you can go and park half your taxis and just not operate them.
You may still have to pay a lease on them, but essentially you can downward that the capacity of the system.
You can't do that here, it doesn't make sense.
Right?
So that's the big difference between systems that are fundamentally fixed cost dominated versus variable cost.
So in this case, we just don't use it as efficiently in the last six years.
The answer here is that there's an optimal path, and that's shown here, that will, for a given targeted capacity, you can compare essentially this blue path against the traditional fixed design.
And in this case, the traditional architecture-- fixed architecture-- would cost about $2 billion dollars to build, both and on average the red point is the average lifecycle cost of the evolvable system, is 1.36.
Right?
Lifecycle cost of the rigid design versus the expected lifecycle cost of the best deployment strategy or staged deployment strategy, which is-- essentially the way that's calculated is the probability weighted lifecycle cost of each of the scenarios, right?
Against this path.
And the difference between those two numbers is about a third, $650 million.
And that is the value of the real option.
That's the value of designing the system with scalability, with flexibility.
OK?
Yeah?
[INAUDIBLE]
Yes.
That's true.
So the lifetime-- the question was, is the life expectancy similar?
Of course-- so the 15 year life of the whole system is the same for both.
Of course, in the staged deployment strategy, some of the satellites are going to be younger at the end of the 15 years.
And so they may have longer residual life left, which is actually not included.
That's not even included in this real option value
[INAUDIBLE]
Right
[INAUDIBLE]
That-- that's right.
So you could sort of refine this model to include a more-- a staged decommissions, right?
Or staged transition to a next generation system.
So in this case, in this model, after 15 years-- boom-- you know everything, you just finish.
No more revenues, you just decommission.
A hard end, a hard stop after 15 years.
But you could actually build transition and decommissioning models as well.
Good point.
So that's essentially the case study that I wanted to show you.
And, you know, you say that's kind of like in the US we talk about Monday morning quarterbacking, you know?
You come in and all the football games that happened and the mistakes that the coaches made, and I would have done this or I would have done that.
So yes, you know, this is sort of like looking at this problem in hindsight, but the reality is that this really has had a big impact and a big dampening effect and that the new generation of systems-- I think-- are built much more intelligently with this kind of evolution in mind.
I will also mention to you that Iridium is actually the system that went bankrupt first.
And the reason for the bankruptcy-- well, one of the immediate reasons for the bankruptcy-- is because of the way that the project was financed.
So about one third of the funds-- the $5 billion-- one third of the funds for Iridium came from equity from Motorola.
So that was their own money that they lost.
About one third was an IPO, initial public offering, you know, shares sold to the public.
And about one third was bank loans.
And these bank loans were relatively expensive and the banks expect that they get paid back at a certain speed, depending on the market evolution.
And it's essentially the inability to pay for and service their loans, that caused the Iridium bankruptcy.
And it turns out that the loans were about one third of the total capitalization of the project.
So what I would argue-- I would argue that if they had done this more flexible, staged approach, they could have saved one third, which is about the value of this option and capitalization, and just build the system only with equity and with the money from the IPO.
And that would have given them a lot more time, you know they wouldn't have had to service those loans and it would have given them a lot more time to wait for the market to develop, which it eventually did.
So the whole financial architecture was poorly done as well.
It's a whole other-- that's a whole other question
Professor, quick question?
Yeah
Are there situations where [INAUDIBLE]
Yes.
So the question, I'm just going to repeat it for you guys at MIT, the question was are there cases where the flexible approach is not the best?
And the answer is yes.
So for example, if you're in-- I know that you're studying the energy market-- if you're in a situation where you have guarantees, a guaranteed customer, so the government or one of the big utilities is entering into a long-term agreement with you, that they will buy the power that you produce at a certain price for the next 20 years.
And so you're not subject to that market volatility, but you have a long term agreement, including clauses that say well if the government doesn't do this then they'll compensate you for any losses, this is a financial contraction.
If you basically have eliminated demand uncertainty, because of the particular contractual arrangements, then there's no reason you shouldn't build the system for a fixed capacity.
And then you can build something right on the pareto front, because that particular uncertainty has been eliminated for policy or contractual reasons.
So, good question.
OK, so we're almost there.
Let me just try to summarize some--
Can I ask you question on that last slide?
Please, go ahead
On the slide before, 37.
It seemed like the probability weighting was for a general capacity.
Like something that could be adapted over the life cycle but here we're pinning down a specific capacity.
So why is it the sum of probabilities, then?
Haven't, kind of, the probabilities been realized, don't you know exactly what your capacity is that you're operating at?
Yes, so just to be clear on this.
So the probability, those p's of i's, they come from the lattice model.
They are determined by the lattice model.
So what you do is for those 32 scenarios of the future, this is the pi, right?
The i is the index, is the-- for this particular future scenario, for each future scenario, you always start with a1, which is your initial configuration of the constellation.
You always start with a1.
And for each of those future scenarios, you then simulate that future and for those futures, where the demand doesn't ever really take off and materializes, you never actually trigger that next expansion stage.
And it's actually that asymmetry that gives you the advantage.
And it's essentially not deploying capacity when you really don't need it.
And so you do this for all of this n scenarios for the future, and the pi, the probability, weighting of each future scenario comes directly out of the lattice model.
That's not something that you have to manually determine
Right, no, I understand that.
But here, since we've kind of pinned down a capacity already, we know what endpoint we're at.
I'm just wondering how the probabilities come into that, or is it kind of working backwards now, of all the paths that go there on this specific plot?
I see what you're saying.
OK.
So the last point, the a1 point, so the a1 point is driven by-- you need to make some assumptions about initial capacity and then the endpoint, the a4, is driven by how large you made the trade space.
So you could make the trade space, you could make the design space even larger, right?
This is a discretized full factorial design space, you could make it even larger.
And then a4 would not be the end form.
This is just sort of a finite size effect, given to the size of the trade space
All right, thanks
All right, so the way I want to do the summary here is just go back to the learning objectives.
You know, I always do this in every class, you know, let's close the loop.
What did I promise you in lecture one that you would learn in this class?
And, you know, each of you, you're going to have to decide for yourself, you know, did I actually learn this?
Did we actually do this?
So this is sort of due diligence on the learning objectives.
And I have one slide on each of them.
So the first objective was to-- for you to really understand the system engineering standards and best practices, as well as more approaches.
Number two is understand the key steps in the system engineering process.
Number three was analyze and understand the role of humans in the system as beneficiaries, designers, operators, and so forth.
Number four was being honest and characterizing the limitations of system engineering as we practice it today.
And then objectified learning and objectifiable was applying these methods and tools to a real-- even if not so complex-- cyber electromechanical system.
So I see one, essentially, hopefully-- no, we didn't check the readings but I'm hoping that you did your readings, that you really feel like you understand the NASA System Engineering Handbook, which is our quasi-textbook for this class.
But there's other standards as well, such as the [INAUDIBLE] handbook, which is actually the basis for certification.
So several of you-- your [?
EPS ?]
fellow mentioned to me that you're interested in certification, so I encourage that.
Most of that is based on the number two.
I did mention the ISO standard 15288.
That's probably the best known standard; ISO is a very-- they're located right here in Geneva.
ISO is a very important organization, you know?
I know ISO standards are not the most exciting things to read, but they have huge impact and so 15288 is something that worldwide-known and across all industries really.
And then I want to mention also the European Systems Engineering Standard.
And Foster Voelker and I had a discussion about this earlier today.
This is issued by the European Space Agency.
And it is a little-- there are some subtle differences between ISO approach and the NASA approach.
And so for those of you on this side of the Atlantic, I do encourage you to take a look at that [?
ESES ?]
standard as well.
And then we augmented this with different papers, and readings, and so forth.
So I want to do a very quick-- this is our last concert question for the class.
So here's the link, [INAUDIBLE] and it's essentially I want to get some feedback what you think about the standard.
All outdated and surpassed by the digital revolution, one of you thinks that's true.
90% of you think they're useful codification and best practice.
14% think they can be dangerous if you adhere to them too closely.
About 60% of you think they're essential reading for any serious system engineer.
And 10% said that you would never use them as a daily reference book
OK, so that's good.
I think that's-- what do you think, Voelker, pretty reasonable outcomes?
OK.
But what is true is that they're going to be referenced, in contracts.
You know if-- if you're doing-- you know, it depends, again, on the industry that you're in.
But you're going to adhere to some of these things in contracts, and if things don't go well, people will actually check whether, did you do this step, did you do that step.
So it does have real consequences
OK, great.
Thank you very much.
That was that was good.
Step two, you know the key steps, v model, I think I'm not going to explain it again.
You know, hopefully this is something that you will not forget.
And just one point about the v model is it does not imply that system engineering is always sequential.
You know, it's possible to iterate between steps and across the whole v as well.
SE3, stakeholders and value of network.
So this is the role of humans.
We talked about this very early on, several of you talked about it during the oral exam here today.
You know, the hub and spoke versus the stakeholder value network.
The method underlying this is critical, but I do want to mention, very quickly-- and this is something we didn't spend a lot of time on-- is human factors.
Right?
How do you design systems so that humans can effectively and safely use them?
You know, interfaces, procedures, and so this is a couple of slides from one of my colleagues, Missy Cummings.
This is actually some-- in a nuclear plant-- Katya, you're going to like this-- lots of these dials, and this is actually a Russian nuclear plant.
So it has it has a very specific layout.
And so the important questions in human factors are, how do you best display status information?
What tasks do humans do?
What level of automation?
What are the training requirements?
And these are also very important questions.
And so you can apply the human systems engineering process, very much analogous to what we covered in class so far.
So you do mission and scenario analysis, function allocation.
And then in human factors, you talk more about task analysis.
But essentially, eventually, leads to system design.
Like what are the buttons that you push, what does the layout look like, or the user interface, and so forth.
So, you know, I'm not I'm not presenting human factors as a separate topic, but the human factors requirements should be built in right from the start.
And essentially the functional allocation is sort of a key question here.
How much is done by automation, how much is done by humans, and how do you split between the two?
And one of the most important things to consider here is this human performance curve.
And we know this pretty well now, that when humans are extremely highly loaded-- you know, when you have a huge workload-- your performance goes down.
But interestingly, if you're under challenge your performance goes down as well.
And you know, so for example, people who are in power stations or mission control centers where nothing is happening for days and weeks, you know their attention goes down, and they don't perform at a very high level.
Humans perform best at a moderate workload, and this is well-known.
And it should influence how you define the human interface and the split among automation and humans.
SE4, system complexity.
Last time I will mention the magic number seven, plus or minus two.
The real limitations come in and when we operate at levels, you know, three, four, five, and six.
And the main reason for this is because now a single human cannot understand, cannot remember, cannot deal with all the detailed components.
And you need to split the work among teams, among organizations, and creates extra interfaces and complexities.
OK?
And just to show you, Iridium, we talked about Iridium.
This is a very recent news story this is from October 29th.
The new Iridium Next has again been pushed back by four months.
This gentleman here is the CEO of Iridium, chief executive for Iridium Next, Mr. Desh.
And he's talking here about a particular [INAUDIBLE] and then Viasat is one of the contractors.
There's a particular component that's been giving trouble, and that's sort of in level three or four in the hierarchy, and that's essentially delaying the whole system.
So there's an example of how the large complexity of the system is causing issues
[INAUDIBLE]
But they're actually a supplier to [INAUDIBLE] through the whole chain, right?
Oh you're saying there's more going on than meets the eye.
Yeah, who knows?
Who knows?
OK, and then finally here I want to mention application to a case study.
You know, we used the CanSat 2016 competition as a quote, unquote safe case study.
And I have to say I'm really pleased with how this worked out.
You had the 47 requirements as a starter, you approved, them you group them, and my comments about the PDR that we had a week ago are all teams passed successfully, the PDR.
You know, if this was a real PDR, we would have issued a couple of [?
ridds, ?]
I think.
There was some-- a couple of teams were over budget, or needed to work out their aerodynamics in more detail.
But by and large I thought this was an excellent application of system engineering concepts.
Went beyond my expectations.
Several of you mentioned the importance of concept generation, you know hybrid use of structured and unstructured creativity techniques.
I know at least one team applied to actually go to the competition.
And this shows you a couple examples.
So we have one team here at the EPSL using a bio-inspired design.
This is an actual seed airfoil that actually occurs in nature.
And then here's an example from MIT, the Rogallo Wing Team 7.
So, I'm sorry I didn't mention all the teams here, but it was really great to see the application of this in the CanSat competition case.
So the last thing I want to do-- and I only have one minute left, actually I'm already slightly over time-- but I just want to give you some career and study recommendations.
So first of all, you know, I want to make sure that you-- and I said this in the first lecture-- there's a lot more to the system engineering and systems research than we were able to cover in this class.
So this class is really what I call a door opener to the world of system engineering.
If you want to go deeper, there are deeper subjects.
You know, model based system engineering is the big trend.
System safety.
And then in the spring I'm teaching a class called Multi-Disciplinary System Optimization.
And there is actually a WebEx access to it as well.
So it's not going to be officially offered as a [INAUDIBLE] class, but if individual students here are interested, please contact me.
Self-study, you know the System Engineering Journal, there's IEEE journals, there's also-- you know, for some of you this is maybe a little too soon-- but there's professional degrees in system engineering.
For example at MIT we have the SDN program, System Design and Management.
The average age of the students in that program is 33, but it is-- they're sort of coming back to get their masters in system design and management.
And there's quite a bit of SLOAN content as well.
So finance, you know, understanding the financial-- how to build a business case around systems as well.
Professional experience.
You know, getting experience in actual projects.
Like at MIT, for example Rexus.
You know, you've got the Clean Space One project, we heard about [?
Optaneous ?]
One, Solar Impulse, there are a lot of opportunities.
Or-- we also had a couple of people here mention that you're starting your own company, your own venture.
And you know, when you're doing, that you have to be the system engineer.
You have to understand all of these things, interfaces, suppliers, requirements, markets, all that has to be integrated.
Finally, you did hear about INCOSE.
We had a quick dial-in with at an earlier session.
So if you're interested, you can join either as a student or a professional member.
And also certification.
This class was not [INAUDIBLE] for that, but if you're interested in this site, [INAUDIBLE] number of years of experience at the [?
CCF ?]
level.
And then finally, please keep in touch.
And last but not least, I want to thank all of you, the students at MIT, at EPFL, our TAs, [INAUDIBLE] here at EPFL, Johana at MIT, the technical staff has been helping run the technology, Voelker, thank you.
And-- and that's it.
So I want to thank all of you.
Next week we do have a voluntary seminar on sort of future trends in manufacturing, but again, it's not mandatory.
It's going to be in the same place, but like I said, it's not part of the official class.
So with that, sorry for running a little bit over time but it's been a joy to teach this class in this kind of new format.
Welcome everyone to 16.885.
And I guess this is ESD.35 as well.
I think.
I cannot keep track of the number.
Formally the title is Aircraft Systems Engineering.
I never thought I would actually be teaching a course with aircraft in the title of it since I am not an airplane person, I am a space person.
But, as you all know, we have decided to devote the course this year to a study of the Space Shuttle.
Very briefly, what the course has been historically, it was taught for quite a few years by Professors Murman and Hansman in the Aero-Astro Department.
It is a course in systems engineering devoted to aircraft systems.
And what they typically would do would be to have a series of lectures about principles of systems engineering, and also about various systems - general systems in aviation.
And then, as a class assignment, the students would work in teams and choose an aircraft and do a systems analysis study of that aircraft.
Well, a couple of years ago we were just sort of sitting down and chatting.
And someone brought up the idea that well, you know, the Space Shuttle kind of is like an airplane.
Maybe it would be interesting to devote the course one year to a special study of the Space Shuttle.
And that was actually the origin of what we are doing this year.
Following that, Professor Aaron Cohen, who is sitting over here, and I will introduce him in more detail in a minute, visited MIT, I think it was early last year, and gave a few lectures on systems engineering and the development of the Space Shuttle.
And we started talking about putting together a course on the shuttle.
And we were very fortunate now that this fall Professor Cohen is here as a visiting professor.
And he will be here at MIT roughly about half the time.
And he and I have put this course together.
And the way we are going to organize it is that we're going to emphasize two main areas.
First of all, to get to an understanding of the Space Shuttle as a flying machine, as a spacecraft, and also to study systems engineering as an engineering discipline.
We have put together a list of speakers, and I am going to pass out here copies of the syllabus.
If you will pass these around.
I have also established a stellar class website and, once you are all enrolled in the course, you will all have access to the website.
If any of you are here as listeners or for some other reason don't get formally registered to get course access, contact me independently and we can set you up for a special access so that you can look on the website.
If you look through here you will see that most of the class periods are devoted to guest lecturers.
And, thanks in large part to Professor Cohen, we have actually been able to invite people who played pivotal roles in the very early stages of the design of the Space Shuttle.
And also people who played pivotal roles in the testing and eventual operation of the shuttle.
So we have people who are active in the design and also some people who were critical in Mission Control and in the test flights of the shuttle.
It is, I think, a really unique collection of speakers.
And because I don't think there has ever been a time where this sort of a collection of people has actually come together to share their experiences on the shuttle.
And because we thought that it would have historical significance, and people would be interested in being able to look at that in the future, we are having the lecture series taped for eventual inclusion on MIT's OpenCourseWare.
So that is why you see the television camera in the back.
No need for all of you to dress up for it, but just so that you know.
The way we plan to run the course is as follows.
The lectures will run roughly about an hour, we will take a little break at the end of an hour, and then there will be an opportunity basically for close interaction between you in the class and the lecturer.
It is kind of free form because we have lots of different lectures.
Of course, there will be some variability in the way lecturers go about presenting the material.
Professor Cohen and I will attempt to make sure that, in each case, we do emphasize some of the basic systems engineering aspects of the presentation.
We want you to understand the principles of how some of the different subsystems were designed, basically how they were influenced by external requirements to make sure that we understand what the requirements were, how the systems were constructed and tested, and how they were operated.
What we will be asking from you in terms of your deliverables, first of all, we would like you to take notes on the lectures.
We will post the lecture notes on the website.
If you look through the schedule that I have given you here, you will see that I list the deliverables, and I am going to have to adjust the timing on that.
Two times in the course we will ask to see the lecture journals.
What we are looking for is to make sure that we are presenting and that you are getting out of this the basic systems engineering principles for each of the subsystems that are presented.
And we will talk a little bit about journal notes in the future.
The main deliverable from you will be a little bit different from in the normal Aircraft Systems Engineering course where groups of students basically chose and airplane and then looked at the design of that particular airplane.
We are only dealing with one vehicle, the shuttle, so what we will be asking you to do is choose one of the subsystems on the shuttle.
And, as you look through the notes, you see we will have information presented on a lot of the different subsystems.
I have a lot of material available, and I will go over that a little bit later, both in digital form and in books which we will have on reserve at the library which go into great detail on all the different subsystems.
So we have lots of information, as well as experts that you can talk with.
We will ask you to form up into teams, roughly four people on a team, more or less.
Choose a subsystem.
And then, basically, write a paper on how you would design that subsystem, if you were doing it today, using 21st century technology.
It gives you an opportunity to understand the subsystem, as it was designed for the shuttle, and then to take a look at current technology in that subsystem.
I know there are some people here from Engineering Systems, as well as from Aero-Astro.
People may have some other ideas.
If you have some ideas about writing about systems from a systems engineering and integration point of view or any other particular personal ideas of what you would like to do as a project which is slightly different from specifically working with a subsystem, come see me and we will talk about it.
The main thing is to make sure that you have a chance to explore some aspect of the systems engineering of the shuttle in greater depth.
That is all I am going to say right now.
Go to the website.
I will be posting more information because I want to give as much time as possible to our two speakers.
I, by the way, will be posting bios of all the speakers on the website so you will be able to look ahead of time and get an idea of who it is who will be speaking to us and what their roles were in the development or testing or operation of the shuttle.
So I would like to present now Professor Aaron Cohen who was born in Texas and has had a very distinguished career at NASA.
Just very briefly, he was the project manager, I guess first in Apollo, he was the Command and Service Module Project Manager.
And then he was the Project Manager for the Space Shuttle and eventually became the Center Director of the Johnson Space Center in Houston.
When he retired in the early `90s he took a teaching position at Texas A&M in the Mechanical Engineering Department where he is now Professor Emeritus.
And we are extremely fortunate to have him here co-teaching the course with me.
So, Aaron, I will turn it over
Thank you very much.
And it is a pleasure to be here with you.
I have a few things I would like to talk to you about very briefly and then I will turn it over to our guest speaker today.
I would like to let you know what you can expect for the next several lectures.
I am going to provide you with the overall shuttle, how the shuttle works, the requirements of the shuttle, the design and the development of the subsystems to a certain extent to give you an overall view of the shuttle so you will have a background of information when you later start to hear the detailed lectures of the subsystems in some degree.
As you listen to these various technical lectures, you should be prepared to figure out what system you are really interested in and what system you would like to move forward with in your future talks.
And so you will see that.
So that is what I will do in the next two lectures, Tuesday and Thursday.
In trying to figure out how we start the course, I went back in my memory to figure out a man who was there at the very beginning in Washington.
And this man, Mr. Dale Myers, who is going to talk today, is what you might say is a true aerospace engineer.
He had a very distinguished career in both industry and government, in aircraft and space.
Dale, or Mr. Myers, who was Deputy Administrator at NASA from October 1986 to 1989, and that was when President Regan called Dale Myers back to be Deputy Administrator after the Challenger accident.
Mr. Myers was Corporate Vice President of Rockwell International, President of the North American Aircraft Group, where he was responsible for the B1 and various military and commercial aircraft.
In 1970 he has been an Associate with Rockwell International and Vice President and Manager for the Apollo Command and Service Module.
So he was Apollo and Shuttle.
The key thing that Mr. Myers is going to talk to you about today, he was the NASA Associate Administrator for Manned Space Flight in 1970 when the Space Shuttle began.
And Mr. Myers is going to talk to you about the beginning of the Space Shuttle and how the external environment helped create or generate requirements that really forced, you might say, the configuration of the shuttle.
And I think it is very important for you to understand that, because many times, when you get out and start to work, go to work, the requirements become generated by external environments.
So, without further ado, let me turn the speaker over to Dale Myers to give you his lecture
Well, thanks for the opportunity to talk to you guys about a very interesting historical element.
I was asked to talk about the origin of the shuttle.
And I was there in 1970.
And I think once we get this thing working where I can run the slides -- The joy of electronics.
When I first went to work for North American Aviation we used Marchant Calculators where you put in your numbers and pull a lever and actually move it to the next position.
And we did a lot of dynamic analysis.
And it took weeks to do a total dynamic flight analysis of an aircraft with a Marchant calculator.
Now we have the wonders of the computer.
The first airplane I worked on was a P51 Mustang, a fighter in World War II, and that had some interesting system engineering issues in it, too.
They used the inlet, it had a liquid cooled engine, and we had a radiator that was in a duct.
And the air came through the radiator, cooled the liquid and heated up the air that was then properly adjusted with a flap at the back of the radiator that gave it thrust out of the heat that was involved coming through the radiator.
That is the reason why the Mustang is about 15 miles an hour faster than the German's ME 109.
That is one of the prettiest pictures I have ever seen of the shuttle.
That is a picture of the additional photography that they brought into the system after the Columbia accident, and this was used on this last flight.
The first time I had ever seen the rearview of all the connections for the transfer of fuel from the tank into the shuttle, the connections to the tank, the forward bipod where the foam was that came off on the Columbia accident, just a terrific picture of all the tiles up under the wing that Aaron was so involved with.
But what I want to talk about, oh, here is another one that is really a first for me.
This is the mach 1 shock on the shuttle about 20,000 feet altitude, mach 1.1.
That big envelope of condensation, not as pretty as it looks on an F-18 or some really slick airplane, but it is there.
I am going to talk about what happened leading up to the shuttle.
And it is interesting because it involves the specific interests and personalities of the people who were involved.
Jim Webb was the administrator of NASA from 1961 to 1968, and he was a terrific interactor with the rest of the administration and with the President directly.
And did a great job of administrating NASA through the Apollo program up through 1968.
It turned out in 1968 things were kind of going sour for President Johnson.
Webb's big tie was with President Johnson, and he ended up leaving in 1968 after he suggested to President Johnson that he might want to leave some time soon.
I think he was thinking he was going to stay through the Lunar landing, but Johnson said why don't you leave now?
And I don't really understand the interaction that was involved there, but Webb did not want to make future plans.
He really never paid much attention to the work that was being done inside the system on new ideas, new things beyond Apollo.
Of course, the Apollo was an immense program, 400,000 people.
Well, 300,000 on the Apollo, another 100,000 on other activities in NASA like the aeronautics program and the science programs.
But Webb didn't want to talk about things that were going to happen beyond that time period.
In the meantime, in the back rooms a lot of people were doing a lot of thinking about where does NASA go after the Apollo program?
Apollo at that time, that Webb was there, had planned to go through Apollo 20, I think.
And so it was going to go on until 1973 or so.
And so I think Webb took the attitude that we don't need to think about the future yet.
Well, when he left in 1968, they brought in a fellow named Tom Paine who had been at General Electric Company doing advanced planning for General Electric, a very bright, very aggressively forward-thinking guy.
A guy that I always felt never saw a future plan he didn't like.
And NASA was doing a fantastic amount of future planning at that time because in the 1968 time period, we had just done the Apollo 7, gotten it back into flight again.
At the end of '68 we did the Apollo 8 which went around the moon, and NASA could do no wrong at that time.
They were just on a step.
And with Tom coming in, in early '69, all the work that was being done by NASA at that time, the idea was that the NASA programs were going to continue to grow and that you could really begin to do some expansive thinking about going out into space.
There had been work going on since 1964 on lifting bodies and on different configurations that people imagined could be used for traveling in space and returning to the ground in a more sophisticated manner than coming down on parachutes in the water.
Water landings were extremely expensive, having the whole Navy out there to support them, and so people were beginning to think about land landing.
And by 1969, enough pressure came on the administration that Nixon appointed his Vice President, Spiro Agnew, to run a program reviewing the future of NASA.
And he got a really good group of people together.
He got Bob Seamans, I am sure you all know.
Tom Paine, the Administrator of NASA, Lee DuBridge had been head of Caltech and was the Science Advisor to the President.
They had a guy that was the head of the Atomic Energy Commission and the Head of the Bureau of the Budget.
They did about a six month study supported by NASA.
And NASA's dreams were that there should be a space transportation system that would include the moon and finally Mars.
And it started with a 30 foot diameter, 12 man space station, two of them in earth orbit, possibly reaching 100 people in earth orbit.
Another space station the same size around the moon with 12 men.
A lunar base.
A nuclear stage to transfer resources from earth orbit space stations to the lunar orbit space station.
A two-stage fully recoverable shuttle with 100 to 150 flights a year because of this massive program that was being developed.
A SkyLab with five visits by the Command Module.
This, by the way, was a second SkyLab.
SkyLab was in our program, and a second SkyLab was under construction at that time in this time period.
And, of course, to do all this, we would continue the Saturn 1b and the Saturn V production.
Those were the big rockets involved in the Apollo program.
And space tug to go to higher than low earth orbit - to geosynchronous.
And, as I mentioned, the nuclear stage.
And a mars program by 1983.
That was all presented to this group, and they ended up setting up three different programs.
One was this massive all-inclusive program.
A second one was a program where the Space Shuttle would be built and the plans to go to Mars.
I never found the report so I don't know exactly how to describe that.
But, to go to mars, the NASA program said you have to have a space station to first develop medical information about man's long duration in space, in other words, how long a guy could last in space to go out to mars, and it became a fuel transfer operation, where low earth orbit rendezvous and docking and transfer of fuel would be made for a device to go onto mars.
So, that second case, the one of build the shuttle and go to mars was the one that Agnew and Paine recommended.
Paine, in his mind, said there has got to be a space station with that.
And so he left in place the studies that NASA had out with industry on building a space station.
And NASA then started a program on a fully recoverable two-stage shuttle.
There have been a lot of studies on how to do that, and we are going to get into some of the system dynamics that we were involved in that program as we go along here.
Meanwhile, the budget crashed.
NASA had a budget of about $6 billion in 1968.
And, by 1970, it was down to about $3.7 billion.
Actually, at the same time, the Manned Space Flight budget had gone from about $3 billion down to $1.7 billion.
So it got hit even harder than the rest of NASA during this time period.
And the reasons, well, we were in the middle of the Vietnam War, the budget deficit was going up dramatically, President Johnson who started The Great Society program for the poor in the country, and that was a big load on the budget.
And Nixon just wasn't a big supporter of the space program, and so the budget was going down.
The question was is there going to be a human space flight program at all?
And that was a really big question because in a time period about a year after this period, one of the senators put a bill before the senate to cancel the shuttle.
The Shuttle Phase B program was underway by that time.
We did studies.
Phase A, a small amount of money to industry.
Phase B was enough to get a definition to where you could decide that you were ready to go into detailed design.
And Phase CD was actually the design and development and testing.
So we were into Phase B on the shuttle at that time.
He brought forward to the senate let's cancel the shuttle, and the vote was 50/50.
So the Vice President had to go in and vote to keep the Shuttle program going.
That is how close to cancellation it was.
Now, a really important guy in all this was a guy named George Miller.
He had been head of the Manned Space Flight program.
He had been the stimulus within NASA for this broad systems study of going out to lunar bases and then to Mars.
And he left in late 1969.
He had done his job.
He got man to the moon and home safely, and he saw this cut in budgets going on.
I don't know whether that really influenced him to leave, but I think his general pattern had been that whatever he wanted to do, he wanted to complete it successfully and then he would move on.
And he did that with the Apollo program so he moved on.
George was a guy that really supported this tremendous set of future dreams for NASA.
Tom Paine left in late 1970.
And the reason he left was he kept pushing for a space station.
And the people in the administration had kind of seen the studies that had been done by this Space Task Group, where Agnew had said let's do the Shuttle and then let's go to Mars.
Paine, knowing in his few that you had to have a space station to be able to go to Mars, kept pushing the 12 man space station that would require a Saturn V for a launch so it was a big expense.
And it was a program that really called for NASA's budget to go up instead of down.
He accepted the idea that it had been pushed down to the $3.7 billion level, but he expected it to be $6 billion or $8 billion by 1974.
And nobody in the administration was buying that so he left in 1970.
And I think he was sort of asked to leave.
I don't know that for a fact, but all the evidence would seem to be that he wasn't really making it with the administration.
And I think this, by the way, turns out to be kind of an important part here because when Tom Paine left there was kind of a bad taste in the administration about NASA being too aggressive and wanting more and more big programs.
Paine left in 1970.
George Low became the acting administrator, and he became a very important part of the shuttle system background.
His background, by the way, started way early in NASA.
And he had been the program manager for the Command and Service Module for a period of time before he went up to NASA headquarters.
And I guess, Aaron, you became the program manager after George left.
And so I came in, in January of 1970.
I had been in charge of the Command and Service Module at Rockwell.
And George Low asked me to come back.
And I had worked so closely with George that I felt kind of a commitment to help in that area.
So I went back in 1970, Jim Fletcher came in, in April of 1971, and we saw where we stood as far as the budget was concerned.
So 1970, with this new cast of characters, we kind of accepted the idea that we were in trouble to the place where we could lose Manned Space Flight completely.
And our real strategy had to be to get something that would be important to the future of NASA with respect to the programs.
And our view was the most important part of the game was to build a shuttle that would reduce the cost of getting into orbit.
And that was the whole idea of the shuttle.
There was a general consensus that if you had a shuttle that would be recoverable and reusable it would reduce the cost of the operations.
As they used to say in those days, I think I have it on another chart, you wouldn't think of flying from San Diego to Boston on an airplane and then throwing away the airplane.
Which was, of course, what we were doing with the Ballistic systems.
But we thought if we could get a low-cost transportation system to a low earth orbit the rest of the systems would then follow naturally.
But, because of the budget picture and because of where we stood with the shuttle in Phase B recognizing it was going to be an expensive program, things started to fall out of the program.
They cancelled the Apollo 18 and 19.
I guess 20 had already been cancelled.
And they cancelled Saturn 1b and the Saturn V which were our big heavy lift capabilities.
Cancelled the second SkyLab that was already essentially complete.
That is the one that is in the Smithsonian Museum in Washington.
Cancelled the Command and Service Modules.
Cancelled the 30 foot diameter space stations.
And that was a big hit against the group because we were in phase B there getting ready to go into detailed design on a 30 foot diameter space station.
We didn't start the space tug, we didn't start the nuclear stage and we cancelled the Mars program.
No, excuse me, we deferred the Mars program.
Industry went down from 400,000 people working for all the NASA programs down to about 150,000.
I have seen numbers lower than that.
Now, concept for the shuttle, reusability equals low-cost.
That was fundamental.
Everybody believed that.
We had studies done by all sorts of outside groups.
IDA, the Aerospace Corporation and others did studies that essentially agreed with us that there would be a terrific reduction in the cost of getting stuff into orbit if we would build a recoverable vehicle.
Now, it was clear that since the R&D already T&E costs are higher, but you need a whole bunch of flights.
If you had a few flights, the extra R&D on the shuttle wouldn't pay off because you could build cheap Ballistic launch vehicles that would pay off before the shuttle.
So you need a lot of flights for a recoverable vehicle to be economical.
The lower the R&D the less flights needed to be better than the Ballistic launch vehicles.
And, if you got a lot of flights because the flight costs are so low, then a two-stage fully recoverable system would be the right way to go.
That was our concepts of what we were dealing with on the shuttle.
There has been a lot of technology.
I don't know how many of you know that the early lifting-body was done by a guy named Burnelli.
Not Bernoulli but Burnelli.
Burnelli was in Long Island and he built the first lifting-body.
That is an airfoil section and that is a broad piece of fuselage wide enough that the two engines could be involved in it.
Big windows for transport.
It was really quite an interesting beginning of a cargo airplane in 1921.
He built one of them and that was the end of that, but there were a lot of other things going on.
After Sputnik, the United States just kind of went wild with ideas for a while and then settled down on having NASA put in place, deciding that the military dinosaurs and mold programs wouldn't be done, that NASA would take over that kind of activity.
But we got some really interesting stuff going.
HL-10 lifting-body.
X-24A lifting-body.
X-15, although that was not considered a space device, by the time they put external tanks on it, it got up to mach 6 and 300,000 feet.
Really some terrific performance out of that airplane.
And then I added the Navaho, which I worked on for many years, because it had a parallel tank separation at mach 3.
The booster was under the vehicle.
The vehicle was a ramjet vehicle and it separated at mach 3 at about 40,000 feet.
It was a high dynamic pressure separation, but it showed us that parallel separation would work.
And that gets into this picture later.
Next step was in that '69 to '71 time period this guy named Max Faget who is really important, almost a genius in my mind in design that did the original Mercury and Gemini capsules, physically designed the shape of the Apollo Command Module and then came up with this first sort of practical view of a two-stage fully recoverable system.
It had straight wings like an X-15.
In fact, if you looked at the plan form, it looked quite a lot like an X-15.
But it had two of them.
It had pilots in each of the two stages.
It had internal fuel.
It had metal shingles, what I used to call unobtanium, but it was like molybdenum and Rene 41 and some really interesting materials which were really difficult to handle.
Stress, corrosion problems and all kinds of things that were tough to handle.
That is why I talked about unobtanium or some ablative.
And Max expected to have to use ablative on the leading edge of the wings.
And it had varying payloads.
The highest one I saw in any of the history was 20,000 pounds.
But 14,000, 20,000, that kind of thing.
A payload bay of about 12x40 and 400 miles max crossrange.
This gets important in requirements.
And, at that time, because we were going to have all these space stations and go to the moon and all that sort of stuff, we were going to have 100 to 150 flights a year.
And, if you have a lot of flights, it overcomes the base cost of the RDT&E.
And he was getting down into the $5 million a flight in his estimates.
But, meanwhile, because we had lost the space station, we had lost the lunar base, all this grand plan had disappeared, we needed more payloads.
We needed to get up to the 40 to 50 payloads a year to be able to make the shuttle look economic at the levels of cost-effectiveness that the Office of Management and Budget was demanding of us.
After this vote in the senate, George Low decided that we had to be responsive to the OMB so we had to get some more flights.
And this is where our requirements began to come into the picture.
We spent about a year working with the military where they finally agreed they would put all their payloads, and I mean all their payloads on the shuttle if we could meet the cost estimates that we had.
The commercial people were eager to get on the shuttle if our costs would be this low because they were beginning to see launch costs equal to or more than the cost of the satellites that we were putting up.
And the science people bought into the idea of space servicing.
This really got important because they agreed to design the Hubble Space Telescope so that it could be serviced in space.
And that turned out to be, of course, the key to the Hubble because of the mistake that was made in the mirror.
The re-servicing of the Hubble Space Telescope, is what - by bringing another optics in front of this distorted mirror - brought the Hubble back to the fantastic performance that it has today
In your opinion, was the science community really anxious to have this servicing or was it something that was really forced upon them to increase the constituency?
I think originally they thought they were being forced.
But, as they thought it through, they could see that they could later change sensors and add additional stuff.
At least I know that the head of the science group, I cannot think of his name now, enthusiastically supported that and got the system out there to support that idea.
And they worked very closely with us on how to design it so that you could get access to the thing in space.
With all the difficulty we had with gloves and so on, they worked to help us understand how to remove and replace systems
Was there at that time, if you recall, an opposition from the space science community to the size of the NASA budget that was being spent on the shuttle in the Manned program?
Oh, absolutely.
Yeah, there was.
But, on the other hand, some of the committees like the town committee supported the shuttle because they could see other science opportunities that were involved.
So they didn't gang on us.
They were shooting arrows at us here and there.
And let me go on with the requirements.
Because of the military requirements, we had to change our specifications.
And this became another one of the elements that drove the final design.
The military wanted a 60 foot long payload bay.
It had been 40 in the designs that we had been doing so far.
They wanted 40,000 pounds Polar, and that made our due east payload up to about 65,000.
That was a big change from 20,000 to 65,000.
And they needed 1,500 crossrange.
They wanted to be able to go around the earth, while the earth turned, and land at the same spot.
So they had to have 1,500 miles of crossrange.
Now, you can do it a lot of different ways.
You could carry turbojets when you came back in and fly it back the 1,500 miles, or you could do it without turbojets which means you have to have aerodynamic crossrange while you are coming in.
Payload bay increased to 15x60.
15 was an increase by NASA because they saw that they didn't have a Saturn V anymore to do space station, so the best they could do was increase the diameter of the shuttle to where they have a 15 foot diameter to where they could carry the sections of the space station that we now have in the program.
We thought anything less than that was just too cramped for the guys.
We decided that we would find a non-ablative reusable thermal protection system.
Technology had moved far enough by that time that we were beginning to see these ceramic tiles develop to the place where they looked feasible.
They had been able to find a hardening for the surface that made them less penetratable than they had been.
And carbon-carbon came in that we could use for the leading edge, for 3,000 degree leading-edge temperatures.
We followed the tradition that said if it is fully recoverable it is going to be cheaper so let's go for a two-stage fully recoverable system.
And, of course, all the other things that we were developing at that time to reduce the cost of operations with automatic checkout and so on.
Well, that's a bad picture.
I am not an expert at this stuff so I wasn't able to get that slide down far enough to show where the Saturn V was, but Saturn V is only about 50% longer than that upper stage there.
So this thing had gotten big.
And the booster was larger than a 747.
It had to operate up to about mach 6.
The orbiter was about the size of an MD-80, MD-90, twin engine small transport.
And so they were big.
And these things had 12 big high-pressure engines in them.
And we in management, and I think the guys in design, were getting pretty worried about whether an airplane that large at that mach number was going to be a practical thing in terms of a system.
And that's not a system engineering approach.
That's sort of a gut feeling that you get after being in the airplane business for a lot of years and watching the development problems that were involved on the X-15, for example.
How long it took the X-15 to get to where it could go to mach 6.
But that was the direction we were going in.
By the way, one of the companies had the wings on the orbiter turned up.
I don't know why, but they had them turned up.
And by the way, the reason you see two different configurations here is in our phase B studies to the industry we said design us one that only has 400 miles crossrange and design us one that has 1,500 miles crossrange.
The upper ones have 1,500 miles crossrange.
And the delta wing will do that, the straight wings won't.
So, as we developed these requirements, it became clear that we were not going to go with a straight wing system.
These phase B studies that we had showed that we were going to have a development cost of some place between $12 billion and $15 billion for R&D.
And about that time Nixon had a meeting with Fletcher and said you can build any kind of shuttle you want, as long as it only costs $5 billion.
Well, that was a big shock to the system.
And, having heard that, OMB said make it cost-effective.
And that was a real tremendous driver in the system because we had never been asked to do that before.
And we had a whole new set of requirements to try to deal with.
So we had had this phase B program, it was almost complete, we had all these big, beautiful configuration studies and we had to look again.
So we went out and said let's get imaginative, guys.
Let's see if there is any way that we can reduce the cost.
There had been enough going on where one of the companies had been looking at the possibility of putting external tanks like drop tanks on the top of the wing on each side, two of them, one on each side of the orbiter.
And it made the orbiter itself, of course, much smaller.
Remember, we were carrying hydrogen and oxygen, and we were doing it inside the vehicles.
It is in those studies you saw in phase B.
And so it grew the outside dimensions tremendously.
And by going externally with the fuels, it really shrank down the system to where the diameter of a payload compartment was essentially the diameter of the orbiter.
And so people began to look at other ways to do it.
We had guys coming in talking about single-staged orbit.
And that was one that I rejected without study because I knew that the mass fractions required for that were just out of the world of reality.
There was a thing called a "trinese" where the theory was you had three vehicles.
They were deltas and fit together in a nice little teepee.
And two of them would be used for boosters going up and the third one would go on into orbit.
And that was kind of a dumb idea.
Because it turns out boosters and orbiters have entirely different requirements.
And so they might look the same on paper but they would not be the same when you build them.
Lockheed came in with an X-24b which had tanks mounted up forward of the delta wing.
And the idea would be that they would peel off after fuel was fed to the main engines in the orbiter itself.
And then we began to see these external orbiter tank studies.
And when the tanks began to look good externally then the question is do you boost from under the tank or do you boost with two boosters parallel to the tank itself?
When the tanks went external, it finally ended up obvious that it would want one tank instead of two.
So the one tank went underneath the orbiter.
Then the question is do you boost through the tank or do you boost in parallel to the tank with two attached boosters?
So we had all these different studies.
And what was happening in the administration was they were kind of locked up with reducing Manned Space Flight budget down to like $1.5 to $1.7 billion.
And we needed to do this cost-effectiveness study for OMB.
We hired an outfit called Mathematica which had a senior well-known economist named Morgenstern and a bright young guy named Klaus Heiss who did a study for us on the cost-effectiveness of the shuttle.
And, to make a very long story short, the results were that the present configuration that we have for the shuttle today is the one that looked the best.
Present configuration being an orbiter with a tank under it with the hydrogen and oxygen in that tank fed separately into the orbiter and to the main engines which were so expensive that we wanted to recover them.
And boosters being solid rockets attached to the tank so that the total vehicle was a little shorter.
Oh, an important point.
At that time the solids were recoverable.
Their wall thickness was enough that you could use the solid rockets, drop them off by parachute into the ocean, pick them up, bring them back, clean them out and use them again.
And that was going to be a cost saving in the program.
And, by doing all this, we had liftoff thrust augmentation of the engines in the orbiter.
These engines, by the way, were 12,000 pounds per square inch internal pressure engines stage combustion.
The most advanced technology you could imagine.
And they were started a year before the shuttle was started to give them more time to develop.
They almost became the long pole of the tent, but I think maybe when it finally boiled down the thermal protection was the longest pole.
He was down there trying to fix them.
I think he was almost down there gluing them on.
Down at the Cape when we were getting ready to launch, we were still having trouble getting the thermal protection system working right.
Well, the result of all this, and there were a lot of other things that happened at the time, in 1971, I don't remember when, the Supersonic Transport was cancelled.
And that was a big technology blow to this country.
In other words, there was a major program that would have absorbed a lot of the high-tech engineers that were involved in the Apollo program.
And, I think, some of the administration thought the Supersonic Transport is a better place to have our technology capability than would be a shuttle.
But Supersonic Transport was cancelled by Boeing.
And I think that probably helped the atmosphere that was involved.
But the other thing that happened is that Congress and the administration finally got the idea that we really weren't going to build a space station immediately.
That we were just interested in getting the shuttle started.
And so we didn't have this massive budget increase that Tom Paine kept wanting.
And Fletcher kept working on trying to get budgets spread so that it wouldn't be a major peak in budgets close in.
And he did that by starting the main engine early, starting the tanks late, starting with solids late and putting an obvious emphasis on the orbiter itself.
So that spread the budget out and helped a lot in giving the administration the feeling that we weren't going to kill them with budget requirements.
So Nixon started the program January of 1972.
George Low and Jim Fletcher went over and had about a 40 minute talk with President Nixon, and he announced the same day we were going to start building the shuttle.
It was going to be a reusable orbiter, the engines in the orbiter, reusable solid cases, an extendable fuel tank, 40 to 50 flights a year, $10 million to $15 million a flight.
Our internal calculations were more like $10 million but we wanted to have some pad in it.
And $5.2 billion, plus a 20% reserve for the administrator for what we call unk-unks, unknown unknowns, things you get into trouble during a development program where you need some more money to do some more testing.
And Nixon agreed to that.
The Bureau of the Budget was in the meeting with him.
And, as soon as Nixon left office, the Office of Management Budget forgot the 20%.
So then it was now $5.2 billion of 1970 dollars.
To make it worse, the NASA Comptroller pressed, I'm sure by OMB, didn't agree that we would use 1970 as the base.
He took the $5.2 billion in 1972.
We lost two years of inflation.
Well, it may not sound like a lot to you but, boy, it sounded like a lot to the guys working on the program because it was clear that we were going to have a tough time meeting that budget.
Here I am explaining to the press, I think it was about two days after Nixon's announcement, one of the studies was to use the first stage of the Saturn V, the S1C.
This one apparently, I think this is the winged version, because all the engines at their back center gradually all the way back here some place.
And we were using, in this design, using the first stage of Saturn V boosting directly into the tank which was attached to the orbiter.
This one was parallel.
I guess that was liquids.
They look like they have bigger diameter.
And I didn't talk about that, but a lot of people were pushing a pressure-fed liquid engine so that you'd be sure you had the capability to cut them off.
And the idea was that because they were pressure-fed that the thickness of the walls was enough that they, too, could be recovered in the ocean and brought back.
And the final decision by the people that were in the propulsion business in NASA was that the technology looked tough.
It was new.
We didn't have a background of pressure-fed boosters.
And the solids, as we will get to later, looked like they were a better deal.
So the design issues, as I saw them as the head of Manned Space Flight, were that the delta wing was required for crossrange.
External tanks were much lighter, the system got to be about half the weight because of all the reduction in external configuration when you took all the fuel out and put it separately in the tank.
Thermal insulation, we bought off on the ceramic tile, the carbon-carbon and fiber blankets, the same as we have today, solid or liquid boosters.
The solids looked more reliable at that time.
There had been a history of solids on many of the large military boosters, and they looked better.
And, at that time, I thought we were going to have a way to terminate the thrust of the solids.
Engine location and type started on the ground for safer, better performance and the stage combustion for better performance.
We had, under our study, retractable turbojets.
Once you got into the atmosphere, you would pop these turbojets out and flew home.
And we decided we couldn't handle it.
Thank God we had had all this lifting body experience where the guys that landed these very low elevatory devices.
Actually, the orbiter had a little better elevatory than some of the HL-10s and X-24s, so we dropped the turbojets out of the system.
Series versus parallel boosters, series was heavier and had less performance with a lot more bending loads in the system.
When we go up, we go max Q.
And with crosswinds we get big loads on that wing.
And so this turned out to be a heavier way to do it.
Well, a little more on solids versus liquids.
I always like to tell this story about solids could be shipped by rail.
I can say it another way.
The diameter of the solids was set by rail shipment.
And there is a story which I am sure some of you have heard that the rails of the American rails were set first by the British who brought the British rail system to the United States.
The British rail system was set by the width of the wheels on the carts that they used to have.
And the cart's width was set by the Roman chariots that used to be on the roads because they made grooves in the tiles.
So that set the diameter of the shuttle.
And the Roman chariot's wheel width was set by two horses in front of them.
So there are some who say that the diameter of the shuttle rocket engines was designed by two horses' asses.
Those were the guys that wanted to use liquid.
They can be shipped by rail.
They had a better reliability record at the time.
Solids could be recovered.
Industry studied pressure fed to recover them, too, but we didn't buy that.
The designers thought they could turn off the solids but later found out they couldn't turn them off uniformly and that the thrust variance that would be involved between the two would be totally beyond the capability of the vehicle to sustain it.
So we dropped it.
Thermal insulation, we talked about it.
They were all new developments that had been experimentally tested, they looked like they were going to work, but we had a lot of work to do.
And the ceramic tiles really turned out to be one of the toughest new technologies that we got into.
High-pressure stage combustion engine, we knew that was a big new development.
And so, as I said, that was started early.
And design crew escape, well, the idea was we were going to be able to terminate the thrust on the rocket engines.
And we looked at these rockets to pull away the cabin, we looked at all that stuff, and none of them had a broad application.
You had the question of safety.
If you took it off at the launch site, if you took the whole front end off the vehicle at the launch site, it had big questions of the reliability of the system.
We went through a lot of studies to try to find a way to capture the crew in case of a problem and never found a system that fit into the program.
So we ended up with crew escape only with a complete structure.
And that, of course, was the problem in Challenger.
We ended up wrecking the structure to where we did not have a recovery capability, but we put in a system where if the vehicle were complete and structurally sound and was gliding the guys could get out.
But that was the only escape system that we had.
I wanted to touch operation costs a little bit.
We had built enormous confidence from the Apollo program.
In spite of the Apollo 13 problem, the rest of the vehicles had worked beautifully.
I used to say that every flight always had man in the loop some place during the flight that was important to the success of the program.
Many times it was a minor thing.
Like when Apollo 12 got struck by lightning when it was launched, the guys were able to reconfigure switches to get the power back on, get everything back to normal and had a nice flight to the moon and hit a few golf balls.
Every flight had man involved, but every flight was a tremendous success, except Apollo 13.
And, at the time, we were dealing here in the 1970 time period.
Well, April of 1970 was when we launched Apollo 13.
Anyway, we had tremendous confidence.
And we thought we had tremendous support from the industry and we thought we had tremendous support from the public, but we still were concerned about the operational costs so we hired American Airlines, for one, to work with us on what the costs would be and how would you design the system to give you the least operational costs?
The military, because they were committing to put their payloads on the shuttle, had studies done by the Aerospace Corporation about the operation costs.
There was a study done by IDA, Institute of Defense Analysis.
All three of them agreed that we were going to have tremendous reductions in the cost of operations.
They didn't quite come down to the same levels that NASA had estimated but they were close.
It was kind of interesting that we all thought it could be done.
We all thought there could be be enormous reductions in the cost of operations with the shuttle.
We thought we had enough space-based hardware that we could do quick turnarounds and handle it more like an airplane.
But NASA and these groups didn't really properly account for the costs associated with post-flight maintenance, the rocket engine.
When IDA and Aerospace did the studies, we told them the rocket engine was going to be reusable for at least 20 flights.
Well, it turned out it wasn't.
And it was such an enormous new development that in the early flights of the shuttle it took a lot of time and a lot of effort to replace engines and refurbish engines.
Assuring the safety of flight in a hostile environment, and space is hostile, and we are dealing here with what amounts to a short amount of R&D development testing when you get into these flights.
Difficult cutting-edge technologies.
The engine in the thermal program, the tiles worked but they often got chips outs of the tiles so we had to replace tiles between flights.
And this is fail operation, fail operation, fail safe.
The airplanes have fail operation, fail safe.
They took the attitude that if you had three computers that was plenty, and if one went bad during a checkout you launched anyway.
You don't know that but that is what is happening to you on commercial airplanes today.
We went one more.
We went fail operation, fail operation, fail safe.
We have four computers in the shuttle, but we cannot fly without all four in perfect condition.
So those things add cost when you do that.
And then cost tradeoffs between R&D and operations, people have argued with me many times that our decision to put the tank externally was a bad deal.
It turns out that it was certainly a bad deal on Columbia.
The foam on that tank came off and hit the carbon-carbon leading-edge of the wing and broke a hole in it and caused thermal excesses in the re-entry.
So you could argue, yeah, we should have had a two-stage fully recoverable system.
But those were the cost tradeoffs that were involved in getting a system that would be accepted and bought off on by the administration.
I want to talk a little bit about operation costs.
That has been the big miss that we made in this program.
As I said, cost of operations never got down to what it should have been.
Well, it never got down there for a couple of reasons.
We were never able to get up to a flight rate that would favor a reusable vehicle.
I think we got up to 26 flights in one year, but most of them were down around eight or ten.
And so we weren't up far enough to offset the cost of the research and development costs to get the operational costs down low.
But I have an interesting little summary.
It is not an exact thing at all but it gives you a little feeling for what I have seen out in this program.
In 1970, the $10 million flight price was based on the same accounting system that we used for the Apollo.
Now, when you go down to Cape Canaveral we had a lot of other things going on besides Apollo.
And so we had the common support activities like the medical department and the mail system and all that sort of stuff as a common separate accounting.
And all the costs for the Apollo were those that we called hands-on, things associated with buying parts, bringing in spares, putting on spares, checking out the vehicle and launching it.
So we had two different pieces of money involved in the Apollo program, and one of them never even was charged to the Apollo program.
So we used that same system, it seemed logical to go ahead with the same accounting system for the shuttle as we did for the Apollo.
Well, it turned out that these separate items were a pretty big chunk of money.
And I have assumed that it was about $400 million a year.
I am not sure that is right, but I wanted to do it just to give you an idea of what happens with inflation.
Remember we said we would do this job using 1970 dollars and we said that the costs would be $10 million of 1970 dollars?
With $400 million in overhead and inflation, according to the Consumer's Price Index, which I looked up on Google, by the way, they have a nice little calculator for inflation, 40 flights a year, no overhead, in other words, like Apollo, a $10 million price in 1970 would be $23 million by first flight and would be $50 million now.
Same 40 flights but including overhead would make the flight cost $20 million in 1970, $45 million in 1981 and $101 million per flight for 40 flights in 2005.
A huge increased price because of the inflation that occurred in the 1980 to 1982 time period.
Eight flights per year, including overhead, runs it up to $60 million in 1970 dollars.
And eight flights per year is sort of what we have been running here recently.
$135 million at time of first flight and $302 million a flight in 2005.
Now, the cost per flight on the shuttle, I don't know.
I know that it is up in that $400 million or $500 million price.
I use this only to give you kind of a rough feeling that, although we missed operational costs badly, we didn't really just be totally out of the ballpark on them.
Shuttle performance is great.
The shuttle has done everything it was designed to do, and probably a few more things we didn't think of at the time.
It has put military devices in orbit, commercial devices in orbit, scientific payloads all to LEO with solids we brought along, and it has taken stuff to GEO, geosynchronous orbits.
It has retrieved and replaced satellites.
It has retrieved satellites and brought them down to the ground and repaired them and brought them back into orbit.
It has repaired satellites in orbit and it has launched elements to the space station.
In the 1980s, the shuttle had only 4% of all the launches in the country but carried 41% of the mass launched.
Shuttle R&D was well within what Nixon and Fletcher agreed to, $5.2 billion plus 20% in 1970 dollars.
And I mean quite a lot of it.
Probably the 20%, only about 5% to 10% of that was actually used.
In that sense, they overran what the OMB said we were to develop it for where they didn't give us the 20% reserve.
We overran it by 5% or 10%.
Missed two key design issues.
Call them system engineering issues.
Cold O rings in the Challenger.
We had O rings in that vehicle which when they were cold they lost their flexibility.
And when they were cold in a design that was opened a little bit when the pressure came on internally, that was a disaster waiting to happen.
So that was a bad design of the way the O rings were designed into the vehicle.
The second one is the foam shedding.
We knew that we were going to have ice and/or foam on that tank, and we really pressed the industry to make sure that that foam was going to stay on.
We had foam enough that we didn't get a lot of icing, but we had foam that had to stay on.
Because we know that as it shed, we didn't think so much of the carbon-carbon as the tiles, these brittle tiles that we had on the bottom of the shuttle.
So foam shedding was known to be a problem all the way through this arm of the development but just has not been able to solved.
And after the Columbia accident, the Martin company, I assume, worked for two years trying to make that foam stick better.
And it did stick better but pieces still came off.
So the fleet has been grounded and they got to get that fixed.
And, of course, we missed the operational cost.
Two-stage reusable vehicle would have missed worse, I am sure of that, because of the size of that first-stage booster and the mach numbers it had to go to.
I guess I have concluded that spacecraft are not like airplanes.
Every flight is a structural dive demonstration.
You know, you develop an airplane and you fly it many times before you fly it to the corner of a VG diagram.
And I can only think of one exception.
A guy named Weese Welch, one of the greatest test pilots North America every had flew the first flight of the F-86 which is the fancy new jet that we brought in just before the Korean War.
And, on his first flight, it flew so well that he took it into a little dive and the mach meter went up to one.
And, actually, the ground data showed that he probably went to 1.04.
Some say they heard a sonic boom.
I am not sure of that.
But that was about a month before the X1 went supersonic.
In those days, test pilots were kind of innovative and they did things that they were told not to do.
And he did it.
But every flight is a structural dive demonstration.
We go right up to max Q every time we fly.
We go through wind shears and take it up to high Gs.
We go to high Gs on the way up, we go to high Gs on the way down, we go max thermal every flight.
So we are dealing with a tough set of activities when we do this.
No reusable space system ever gets the millions of hours of stressed operation that airplanes get.
Once an airplane gets through development, it starts getting millions of hours of test data or information where if you have a problem you fix it.
And you just don't get that in these systems.
The space system has decades of evolutionary model development.
In other words, the airplane business has been so dramatically economical that you could build new airplanes every ten years or so.
And each new airplane took advantage of all the things known from the past airplane and designed into it.
As I said, every reusable system is exposed to enormous environmental variations every time.
Thermal, vibration, pressure, mach number, all these things happen every time.
And so I look at the shuttle as being an amazing piece of machinery which is done extremely well in what I consider a continuing R&D environment.
We just don't have yet an operational system.
So my view of the next program, keep it simple.
It has sort of been a prime view that I have had of design every since I have been in the airplane business.
Don't stretch the technology.
Use really good margins of safety because we are dealing here with, as I said, maximum conditions on every flight.
You better play it safe.
Keep it small.
Carry as few passengers as possible.
Carry people or cargo but not both.
Keep the requirements to a minimum.
Use as many past components and systems as have been proven to be reliable.
Design for operations, very important.
Easy access.
One man can replace a black box.
You don't have to run a big pickup machine to take something out of it.
And keep the design reserved while you are designing it so that when operational issues come up you can design for the operational issue and keep reducing the cost of operations.
I don't know what all that means.
I think it means that if we had it to do over again it would have been great to be able to contain the requirements within NASA, probably build a much smaller system that you could get many more test flights at lower cost, but we didn't have that opportunity, as you now know.
So we have the shuttle disappearing into the distance.
Decision has been made that the shuttle be phased out in 2010.
And it is going to be a tough issue because now the shuttle is down in between flights because we lost foam on the last flight.
They grounded the fleet until we can figure out what to do about that foam problem.
And then Hurricane Katrina knocked off the top of the VAB, which isn't a big deal, but it really messed up some of the tank facilities and tank access.
And people's lives have been affected with losses of homes.
So there is a whole bunch of new issues involved in the shuttle that I read in the paper this morning which may mean another delay in the next launch of the shuttle.
And that means a compression of the time between now and 2010 when they are trying to use the shuttle to meet the commitments that we have with the Europeans and the Japanese about putting pieces of the space station up into space.
An interesting new problem for the shuttle.
OK?
[APPLAUSE]
Let's take a one-minute stretch break.
Stand up if you'd like, turn around in a few circles, and then we will have a half an hour for some questions, answers and general comments.
The external environment, the background, what went into the design of the shuttle.
And I think what Dale alluded to at the end is very much to the point, and we will be talking about this with the people when they talk about the individual systems.
In terms of the actual performance, which we have gotten out of the shuttle, despite the fact that we have had two catastrophic accidents which, by the way, involved not only the design of the shuttle but the way we operated the shuttle.
And that is an important thing that we will spend time talking about.
Had we not made the decision to launch Challenger on that cold day, who knows what would have happened?
And, similarly, we accepted the fact that foam was continually falling off of the tank, even though that was incompatible with the design specifications on the thermal insulation for the shuttle.
So we had basically two parts of the shuttle system and we had design incompatibilities, but we chose to keep on flying.
But, as a whole, the shuttle has been remarkably successful from a technical point of view in terms of what we have been able to do in near earth space, has been I think, compared to what you could do working out of an Apollo capsule, was absolutely phenomenal.
And, in fact, I think in terms of near earth operations, the shuttle will be sorely missed when we retire it and there will be a lot of capabilities that we will be giving up.
But, on the other hand, where we really got it wrong, by orders of magnitude, was in the cost and reusability of the shuttle.
Now, perhaps that goes back to the requirements because, again, we were trying to do an awful lot of things for the very first time and, yet, we were being told by the Office of Management and Budget that you said it had to be cost-effective.
I mean, in a sense, NASA was being asked to operate the shuttle almost as a commercial enterprise and to make money on it.
This is like you build a test vehicle for the first time and you are being asked, at the same time, to operate it at a profit.
And, as you know, one of the systems engineering principles is you have this triangle.
I am sure you have all seen this.
Performance, cost, schedule, you have all seen the triangle, right?
And you cannot specify all three.
They are not all independent parameters.
If you specify the performance and then you're limiting the cost, you cannot control the schedule.
Anyway, all three of those we, in a sense, got specified when we accepted the requirements to build the shuttle.
And something has to give.
And, in the end, it was the schedule and the cost.
And we will have more to say about this in the next couple of lectures, but now let's take advantage of the last 20 minutes and give you a chance to ask some questions about the content of the lecture.
And Dale can try to answer them.
I will say, by the way, in terms of the schedule of the class, when this was given as an aircraft systems course, there were two lectures of an hour and a half and then there was a laboratory period scheduled on Wednesdays.
And the idea, I think, was that by scheduling a lab everybody would have one period of time at the same time so that it would make it easier for you to work as teams on your project.
Because we have so many people coming from out of town, I thought that it would be better to make sure that they have full time to interact with you.
And so we didn't schedule a laboratory period for this course.
Instead, we have two two-hour lectures.
And I am assuming that all of you, once you form teams, will be able to work out some times when you can get together.
The only other thing I will mention, if you look ahead for Tuesday, September the 20th, that is the first deliverable regarding your term project.
And all we want you to do there is just to think about what subsystem you might be interested in studying and write a paragraph about what you think you might want to do with it.
And what we will do is we will be taking a spiral approach to this project.
So, if you look ahead, you will see there are various times when we ask for preliminary results.
And then we will work with you, give you guidance in how to deepen that so that then you can go back.
And the next time you hand something in it will be at a deeper level until the end when you're finished with the project.
OK, let's move into the question and answer period.
And there has been a lot of material presented.
And this is your chance to ask the guy who was here when it all happened.
Dale, why don't you take center stage now?
Thanks so much for coming.
I was kind of intrigued by your talking about the different phases of development.
You called it phase A, phase B and C/D.
And I was kind of wondering when, in those phases, you kind of developed the high-level requirements, when you developed kind of low-level requirements, how much industry was involved in different levels?
What percent was industry engineers and what percent was NASA engineers in those studies?
And I was just wondering if you could expand on that
Yeah.
Good question.
The theory is that you do phase A as conceptual activity and, when you have gotten your requirements nailed down, you then do a phase B.
And that's what we thought we had done.
It turned out that military requirements came in after phase B had been started, so we actually had to change the contracts with the industry to take into consideration those military requirements.
And they were a big change to the requirements.
So, in that sense, we had some inefficiency in phase B.
And it was after we began to get the results of phase B that we realized we didn't have a system that was going to meet the requirements.
So, instead of canceling phase B, we finished the phase B because we needed that basic understanding of all of the systems and all of the elements that made up the system.
We finished phase B but we started some additional phase As, conceptual activities to try to find a solution.
And I don't remember how we worked from the recognition of the new configuration back into the phase B guys.
Oh, we had a phase B extension to bring that new configuration.
NASA's decision that we would go to an external tank.
We modified the phase B studies again with an extension that allowed the industry then to catch up with what was going on.
And, by the time we finished that phase B extension, we all the requirements in place, we had all of the design understood well enough to start phase C/D.
I think it is pretty amazing that a device that was going to do what we wanted to do with the shuttle, go into orbit, come back in and land, the configuration really stayed the same from that point on.
It is just amazing that we did that well, I think, in definition so that when we really started the C and D phase, which is the detailed design, things stayed in place.
And that meant all the aerodynamic work that had been done which was, by the way, the most aerodynamic work, most wind tunnel testing ever done on a new system, I think, logically because we were working through the mach number range.
And all of the other elemental testing that had gone on all allowed us to keep the configuration identical from that point
I know there has been a lot of talk, but it seems obvious now in hindsight that the capsule, or the orbiter or what have you should be on the top of the launcher to clear it from the debris from the fuel tank or what else.
What I'm wondering is at the time in the early stages, was there ever any talk about safety issues in putting the orbiter so low on the hull on the side?
Yeah, there was a lot of talk with Martin about foam shedding at that time.
And, during the initial decision process for putting it on the side, we had done studies of stacking it in series.
And it was a weight problem.
It was literally an issue of the structural weight of the orbiter mounted vertically because of the terrific loads that you get separately on that system.
And so we recognized that the side mounted tank was going to be a much more economical system.
So we had to worry about ice and foam.
And so we had a lot of discussion with Martin Company at that time.
I think we did a lot of work down at Marshall, too
That's a very good question.
In fact, I will try to develop some of that thought process as we go along.
The key point to make is the following.
Should we have challenged the requirements?
Making the orbiter so big with the payload base so big, it was very difficult to put that on top.
If you made it smaller you could.
The real question, I think, and I was going to ask Dale this question which is a follow-up to yours, should NASA, once OMB and the White House gave a cost constraint and once we had the change in the Air Force requirements, should NASA had said no, we don't want to do it?
And that is really a very fundamental issue in terms of understanding your requirements, because the requirements drove the 14-day turnaround time, the fact that you wanted large payloads, you wanted to get to the payloads, put the orbiter where it was.
The fact that you needed a high-performance engine and a lot of payload in orbit said that you needed a liquid oxygen liquid hydrogen engine to get the highest specific impulse to get the engine at highest performance.
All that added together, the thermal protection system was basically a glass house which was incompatible with material coming off the tank.
So there was some, you might say, incompatibilities and requirements.
And the question is should we have challenged those requirements more strongly?
That's really the fundamental question.
I don't think we should of.
But let me ask Dale because it would say what you would have done with the orbiter
I did.
I challenged the requirements inside NASA.
I never challenged it with the military, but I challenged it inside NASA with George Low and with Jim Fletcher.
And their conclusion was that we would not have a Manned Space Flight program if we challenged the military requirements.
And then the rest of it followed
But your question is a very, very pertinent question
Yeah, it sure is
It is a very key question in today's environment.
Of course, you are infinitely smarter after it happens, but your point is very well taken
Do we have better systems engineering tools now than we did in the `70s?
And so, if you used today's tools to design the shuttle then it would be better?
I think.
Well, of course
Would you have avoided cost overrun and so on?
System engineering is better, yes.
Cost estimation, I'm not so sure.
We had the best guys in the country doing cost estimations on the shuttle, but we missed it probably as much as anything else by just not having those people understand the complexities of operating in space.
And I think a lot more is known generally now about the cost of operating in space.
I think that the next try at a reduction in cost for getting into space will be a much more significant activity, but I consider cost estimation a part of system engineering.
A lot of it is much better, some of it is not, I think
Well, just to follow-up a little bit.
When we designed the orbiter, we didn't have CAD/CAM systems.
If you look in the aft end of the orbiter, it is sort of like the hardest thing you have ever seen because we didn't have a computer-aided design.
If we had had that, we probably would have done a much easier job in the aft end of the orbiter and in the mid fuselage and in the cockpit.
That is systems engineering.
You today have much more valuable tools than we had during the Apollo program and during the Shuttle program, but there still is a lot of education you need in systems engineering.
And I think Dr. Hoffman explained the famous triangle, cost, schedule and performance.
And that is a continued work in systems engineering
And I always think of systems engineering as the people who work across the system with everybody in a real communication system.
And it is that kind of communication that does good system engineering.
Tools are not.
OK?
Anything else?
Could you talk a little bit about the Astronaut Office and what they thought during these conversations?
Were they in favor of the recoverable fully piloted booster and what were their input on the risk conversations?
Yeah.
They were aware of it.
We kept in touch all the way through the development program, including the decisions not to have a launch abort system.
And they all recognized there was risk in the program, no question about it.
Aaron, you were there.
What about it?
Well, you said it right.
I think that's right.
They were part of the design and development team and the requirements team, so they were very much in favor of it.
Of course the big issue, which we will talk more about, is escape systems.
And we will go into that a little bit.
Why don't we have an escape system?
And I'm sure when Chris Kraft comes you can ask him a lot of questions about that.
I'm sure he will have a lot to say about it, but a lot of us will talk about that.
I think the astronauts were very much a part of the design, the development, the requirements in this phase of the program.
So they were very much a part of it
They weren't too much in favor of an automatic landing system
That's right
That was my next question
[UNINTELLIGIBLE PHRASE] When does it appear clearly that the Space Shuttle was not a low-cost access to space, or was it already too late to change the program or requirements?
I think the problem was that we never got up to flight rate.
There were payloads waiting for us but we never got to flight rate.
And, if we had gotten to a higher flight rate, operational costs would have been lower.
Not enough lower because, no matter what we would do, we never would have met our original estimates on operational costs.
But, as you saw by that inflation story that I had, costs today would be enormously higher than that $10 million estimate that we had in 1970 just because of inflation.
But we never got flight rate so we didn't ever get to the lower cost.
And, in the early days, I wasn't there, but it appears to me there was a lot of pressure to get that flight rate up so that the cost per flight would come down.
And that pressure got to be instilled into the people at NASA and the industry to where the decision made on that cold day in January, or whatever it was, on the Challenger.
Even though there was evidence that those O links had leaked in previous flights, the decision was made to launch.
Now, that is a management policy issue associated with trying to reduce the cost of flight.
And so that was a bad decision.
Anything else?
I will say one other thing on cost per flight.
You have to realize when you're dealing with a reusable system it is hard to specify exactly what you even mean by the cost of the flight.
You can take the total amount of money you spend on the shuttle program every year and divide that by the number of flights.
Well, this year we only had one flight which came to a pretty high cost.
And last year the cost was infinite.
On the other hand, you can look at what is the cost of flying six flights a year versus what is the cost of flying seven flights a year.
And that is what you would call in economics the incremental cost of a flight.
Also, you have to realize that in the cost of the flight there are an awful lot of things that are wrapped up.
Not just the cost of the shuttle itself but all of the mission operations, the flight planning that has to be carried out.
There was one flight, a space lab flight I think back in the `80s where they launched the space lab mission, it was supposed to be a two-week mission, but they had fuel cell problem so they had to come back after four days.
And, in order to give the scientists the opportunity to get their flight data, they rescheduled the flight for a few months later.
So they had the same crew, they had the same flight plan so they didn't have all of the expenses, the paperwork expenses, the training, all of the re-planning and the experiments were the same.
It was the least expensive flight that we possibly could have run.
And, at the time, the estimates were that that actually cost NASA probably about $120 million.
That was kind of the barebones estimate of the incremental cost of a shuttle flight.
And then it can go from there all the way up to billions of dollars if you just take one flight a year like we have this year
Well, the other thought, too, I remember going up to see Dale Myers when he was Associate Administrator for Manned Space Flight and I was the Orbiter Project Manager.
As he pointed out, we had four computers.
The original thought is if one computer went out on the ground we would lift off with three computers.
And that's what we talked about.
Well, of course, that never happened.
I mean, not only that, we have five computers now.
We actually have a fifth computer which is a backup computer.
So things change, environments change, and we were going to do very routine payloads.
We were going to take it up, launch a payload and come back down, just very routine payloads.
Almost every payload today is different.
And it does take that large amount of infrastructure to get that
One of the cost elements in our cost-effectiveness study was a reduction in the cost of scientific payloads because we were going to have sort of a boilerplate bus.
A heavy, rugged bus that had power and communications.
And the scientists would bring their experiments to this bus, put it on this standard vehicle, take it into orbit, launch or keep it, depending on what the experiment was, then bring it back.
And we were going to have this standard bus that was going to be one of the big improvements in cost of the science payloads.
So we showed a reduction in the cost of scientific activity in our cost-effectiveness studies.
That never happened.
The science guys never could accept the idea of an independent bus
We could go on talking for a long time, but it is the end of the class.
Let's thank Dale Myers again.
[APPLAUSE]
Thank you.
I enjoyed it
So send me your bios and have a good weekend and see you next Tuesday.
The formal part of the class will start about five past nine, but I'll just start with some introductory remarks.
First of all, I have got bios from a large number of people.
I haven't done a count yet to see from whom I haven't gotten them, but if you haven't sent me a short bio please do.
It just helps us to get to know who you are.
Let's see.
Second thing.
The next deliverable, which is, I think, in a week or so, I don't have the syllabus with me right now, we would like a preliminary indication from you.
I am leaving it up to you to form your own teams.
Roughly, we've found about 4 people make up a good team.
That is a good way to spread the work around.
If any of you are having difficulty or whatever, let me know, we will try to help, but you are almost all graduate students.
You can organize yourself.
With a preliminary indication of what system you would like to do a study on.
Take a handout.
I am just wondering if you can provide a list of email addresses to everyone so that we can contact each other.
I will be happy to do that.
I will post that.
Let's see.
The second thing.
There are a fair number of listeners in the class, and quite a few of you have contacted me to make sure that it is OK to be here as listeners.
I have no problem with that.
The only thing that I would ask, right now we are more or less filled to capacity.
I expect that the class isn't going to grow after this, so I think we are OK. What I did tell people is that if we get into a seat crunch, obviously, the people who are taking the course for credit have priorities for seats.
So any of you who are listeners please keep that in mind.
Next thing.
I am going to be putting some things on reserve in the library.
This is courtesy of Professor Cohen.
This is the proceedings of a Space Shuttle Technical Conference.
Aaron, do you want to say something just about the background?
The background is that we decided, after the Shuttle had flown a couple of times, that we would have a typical NACA, going back to NACA, but this was NASA technical conference where we actually had technical papers written by the various people that designed the system, some you will hear talk, some you won't.
But these are technical papers of how the systems were developed.
So it is a pretty detailed understanding of the thermal protection system, the main propulsion system, so forth and so on.
So it is a pretty good documentary of how it was done.
This should be a good reference for your papers.
It is 1983.
Like I say, I will go down this afternoon and put this on reserve in the library.
I will also put on reserve a copy of a systems study that was done at Texas A&M for a similar course that Professor taught on systems engineering and the Space Shuttle just to give you kind of an idea of what someone else did.
This was a rather good paper.
I see you gave them a very high mark so presumably that means it is a good example.
Finally, you have all got a copy of this paper by Professor Cohen and Milt Silvera who was also a very responsible person in the Shuttle program.
It is a high level background on the Shuttle and its systems.
I am sure there are some of you here who are space enthusiasts and who probably have followed the Shuttle system pretty closely and to whom there won't be a whole lot new here.
But, on the other hand, we want everybody in the course to come up to a certain level of knowledge about the Shuttle, its systems and the systems engineering that went behind it.
So I think it's good to have something like this available.
It's a nice reference piece.
Today and Thursday Professor Cohen is going to give an introduction into the Shuttle, its systems and the system engineering that went into it.
Before we start that, are there any other questions either about the technicalities of the course or just anything?
Oh, I know one thing.
I have a few comments from people telling me that the pdf files that I posted on the website, some people could read them and some people could not read them.
I have Macs but have been posting pdf files for numerous courses.
I have never had any problem with people reading it before.
First of all, who has tried to look at the files?
OK. Of those people who have tried to look at the files, who has had difficulty?
This is strange.
Well, if anybody has any ideas about what the problem might be, I would be very curious.
I mean have you ever had this problem with other courses?
I will do a couple of experiments and post one or two more documents, both in pdf and in either Word or PowerPoint form.
I would suggest that everybody during the next two days go and try to open them.
Let's see if we can isolate where the problem might be.
Like I said, I've never had this problem with any other courses.
And, in principle, I mean the whole point of pdf files is it's supposed to be compatible across Macs, PCs and all that.
So I don't know what the problem is but we'll see if we can run it down.
And thanks to those of you who let me know that there was a difficulty.
That is all I have.
Any other questions, comments?
What are you kind of expecting with these journal deliverables so we can kind of be just ahead of that?
The idea is to basically put together the highlights of the lecture from the systems engineering point of view.
In other words, what we want -- When somebody is talking about a specific system, we want you to have a brief description of the system, the purpose, what it does, something about the design considerations, something about the operations, kind of the basic concerns from the systems engineering point of view.
And anything they talk about, particular interactions between that system and other systems.
Can you think of anything else in particular?
I think that about it.
OK.
I will think that through a little bit more, and maybe I can come up with a checklist that would help you put that together.
It's not meant to be a big burden.
It's just a way to organize everybody's thinking on how to get the most out of the lecturers, particularly because, as I mentioned, these lectures are all being given, by and large, by different people.
They will have different styles.
We have kind of explained to everybody the structure of the course, what we're looking for, but I cannot really guarantee in advance what the content of the lecture will be.
Professor Cohen and I will make any attempt, if the lectures don't cover some of those critical points from a systems engineering point of view, we will try to stimulate the discussion on that or fill in.
But you should do the same thing.
In other words, make sure that there is something that you think you would like to be getting out of the lecture and you haven't gotten, don't be embarrassed to ask questions.
Something you ought to think about, when these individual lectures come up, is it should be the description and function of the system.
If they don't bring that out then we ought to try to bring it out.
The requirements of the subsystem or the system, what are the requirements?
Because that is very fundamental.
What are the requirements?
The development of the subsystem.
What kind of problems did they have?
What kind of technologies did they have to overcome?
And, of course, the operation of the subsystems.
Those are, I think, key points.
Now, whether you're going to get that from everybody or not, I don't know.
But I think you need to look for that.
And, if you don't, then we will try to develop that.
Are you looking for the notes to be taken in class?
I don't follow you.
I mean I am not looking for a Xerox of what you're writing in the notebook, if that's what you mean.
No.
I mean I would like you to put your thoughts together into a concise form.
And I think that will be a good reference to you, hopefully, after the course on Shuttle systems.
OK?
Over to you.
You are going to have a very unique opportunity.
You are going to have people speak, lecture you on the various Shuttle subsystems in quite a bit of detail.
You will have some people that will be very, very positive about the Shuttle, that think it's a great design, a great operation.
You will have others that will not think it's so good.
And you will have some that will give you just a detailed technical approach of what happened.
What I would like to suggest to you, I think, for your good and for the good of the future, future students, future designers, is that you ought to come up with your own decision.
Was it the right design?
Was it the correct design?
Should it have been done differently?
And, if so, why and how would you do it?
And I think it will be good for, when you go to work on future projects.
And it would also be good, I think, for NASA, something we could turn over to NASA.
I think it's a very valuable thing to do.
So I would suggest you do that as you go through the course.
And I will be very happy to talk to you about any of your ideas that you have through the Internet, through emails or in person discussion.
And you might want to do this later on as the semester develops.
A little bit what I am going to say today really starts off where Dale Myers, your previous speaker, left off.
And it gets into a little bit more detail.
So, as this course develops, you're going to get more and more detail.
But let me start off again, just very much like Dale did, and talk about the Shuttle history.
In 1952, the fully reusable launch vehicles concept was discussed.
People were interested in that.
1962, fully reusable vehicles were seriously considered.
The Air Force study project Dynosaur was cancelled in 1969.
In 1969, NASA adopted the idea of a fully reusable spaceship.
I became the Orbiter Project Manager for NASA in August of 1972.
And, at that time, I also was manager of the Systems Engineering Organization for the first two years.
So I, you might say, the total system at that time and the Orbiter in 1972.
Yes?
Could you tell us a little bit about your background?
Yes.
Well, my background was I graduated from Texas A&M University in 1952 and went to the Army, went to Korea.
And then when I came back went to work for RCA.
And I worked on the microwave tubes, the microwave oven.
In fact, when I told my wife what I was working on, I said I'm working on a microwave oven.
That was in 1954.
That was a long time ago.
How many people have microwave ovens today?
Everybody does.
Well, in 1954-1955, when they came out, they were about $3,000 a piece.
I was working on one and told my wife.
And we'd been married a long time.
I told my wife that I was working on something called a microwave oven.
And you are going to be able to cook a roast in a couple of minutes and a potato in a couple of minutes.
And she looked at me and said that will never sell.
Anyway, that's what I worked on.
Then I worked for General Dynamics on the Atlas and Centaur.
And then, in 1962, I went to the Johnson Space Center and worked very closely on the Apollo Program, worked very closely with the MIT Instrumentation Lab, now the Draper Lab on the guidance and navigation control system.
I became head of System Engineering in Apollo.
Then manager of the Command and Service Module in Apollo.
Then, in August 1972, I became the manager of the Space Shuttle Orbiter.
Then I became Director of Research and Engineering at Johnson Space Center.
And then I became Director of the Johnson Space Center.
And then, for a while, I was the Acting Deputy Administrator in Washington.
And then I retired and went to Texas A&M to teach.
Then Jeff asked me to come to do this, and I am very happy to be here.
Actually, one thing you mentioned reminded me of something, not to divert the lecture today, but the fact that there was specifically a systems engineering group at the center which was separate from the project offices is something which is probably worth talking about at some point.
That's right.
Well, let me just make a mention of that.
When we were in Apollo we sat around the table for many days and months trying to figure out how you define systems engineering.
We didn't even know what systems engineering was.
In fact, today, I'm not sure you'll get a clear definition of what it is.
I want to give you some examples.
As we go through the lecture, I am going to give you some examples of what I think it is and how it was used.
So we will do that.
Let me say just one other thing.
Professor Cohen has these view graphs in electronic form down in Texas and, when he gets back, he will send them to me and I will post them on the website.
Well, I will give you this whole package because there are also some pictures.
I don't know what you want to do with the photos.
Something that is very key in any design and something you really need to pursue, whenever you do a design project, is understand the requirements.
Because, if you don't understand the requirements, you might get a very good product that is useless.
So you have to understand what your customer wants, the top level requirements.
One thing that was handed down to us is it was supposed to be fully reusable.
That was one requirement.
14-day turnaround time.
You were supposed to be able to turn the Shuttle around in 14 days.
Deploy and retrieve payloads.
You had to deploy a payload and you had to retrieve a payload.
Design, development and test is estimated to be $5.1 billion in 1971 dollars.
Dale Myers didn't tell you the whole story, but one reason Dale Myers and I are such good friends is he was Associate Administrator of Manned Space Flight, I was the Orbiter Project Manager and they, they being headquarters, headquarters was always there to help, actually took away two years of inflation.
If they had given those two years of inflation, we would have met the $5.1 billion in 1971 dollars.
And Dale fought for that but lost.
Now, here is where we missed it.
The original cost per flight for 65,000 pounds was $10.5 million per flight in 1971 dollars but for a flight rate of 60 flights per year.
When I told my wife I was doing that, and she's been around the space program a long time, she said you never agreed to that, did you?
Sixty flights per year is pretty hard to do, but that's what we came up with at the time.
Now, Dale mentioned the Air Force requirements, but here were the phase A studies.
The phase A studies were conducted to determine the basic requirements and their effects on design in 1969.
The principle issues were the size and weight of the payload, the cross-range of the Orbiter and what kind of heat material was going to be used.
You have to recognize are we going to use heat-resistant structure or reusable insulating material?
You have to recognize that our background was Mercury, Gemini and Apollo, and they all used an ablative material.
Ablative material cannot be reused because basically the surface changes.
You had to have some kind of insulated material so the surface did not change.
Let me now go through very quickly for you, just for the sake of completeness, Dale showed you a few of the studies.
I want to flash up a lot of phase A studies just to let you take a look at what people were doing.
And you don't have to read the words but you might just look at the diagram.
There was General Dynamic's phase A study.
There was North American Rockwell.
That is the one Dale was talking about because that was where he was.
And there was a phase B study for McDonnell Douglas.
So that is what they all looked like at the time.
Now, I am going to go through some just for the sake of completeness here.
There was, again, another McDonnell Douglas study, Martin Marietta.
So those were basically some of the studies.
You can see they all had some kind of a wing-type vehicle.
Already most of those were delta wings so the decision had already been made.
Interestingly enough, and I am going to quote some names to you, and you might go back and research it.
Max Faget was like the lead engineer.
He was sort of a man that was immortalized in terms of the space program.
He felt we ought to go with the straight wing.
He felt very strongly about a straight wing, but that eliminated the very large cost range the Air Force needed.
Straight wing was easier to build, not as high loads on it and so forth.
Here, interestingly enough, if you look at Chrysler.
Chrysler had a study.
And Chrysler actually had a capsule, but here were more vehicles.
And people were really thinking about this time of a fly back booster where you actually had a booster that returned and came back and landed.
And they also had a land vehicle that landed.
So that was what the real requirement was at the time.
Somebody sort of commented about Chrysler.
There was also Ford.
There were a lot of aerospace companies back in those days.
It has changed a lot, that's right.
And here is McDonnell Douglas, of course Grumman.
So a lot of these studies.
Then you're getting to see something already starting to look a little bit like the Shuttle.
And I am almost through.
And here are more.
These are, I think, getting maybe in the phase B studies.
But you can see some of them are starting to look like the existing Shuttle.
Lockheed.
Now, the principle issues in the Shuttle studies, now we're starting to get a little bit more technical and a little more detailed, is should the reaction control system, now, the reaction control system is basically a propulsion system that controls the vehicle about its center of gravity.
It's for alttitude control, basically.
Should the reaction control system be liquid oxygen/liquid hydrogen or should you use a hypergolic system where you know it is a storable type system.
That was a big issue.
Does everybody here know what hypergolic fuel is?
Let's talk about it a little bit.
Well, hypergolic fuel is a fuel like hydrazine.
It actually has an oxidizer and a propellant in the same fuel.
Now, another issue was a fly-by-wire flight control system.
That was a big issue.
Do we have a fly-by-wire flight control system?
And now everything has a fly-by-wire.
All the military jets have fly-by-wire.
But basically was it going to be a computer controlled system or are we going to have cables fly the machine?
That was a very big issue at the time.
Wind tunnel tests to determine wing size and configuration.
That is a very difficult thing to do.
This is starting to get into what you might say is systems engineering.
Air breathing engines were considered for fly back and later determined to be too heavy.
And some Shuttle studies that still had to be done was the entry techniques, landing speed, what type of landing speed were we going to have and the approach pattern?
So these were all some things that had to be understood during the studies.
The phase B studies were performed in the mid 1970s to determine the preliminary design.
The results showed a fully recoverable Orbiter.
Disposable fuel tank.
Parachute recoverable solid rocket boosters.
High performance hydrogen oxygen engines placed in the Orbiter to recover.
This was all systems engineering that lead up to the design.
So you have systems engineering in various phases of the program.
And usually systems engineering composes of an interdisciplinary team that has been given some assumptions, some constraints.
They have some top level requirements.
They do an iterative process with some tools such as computer tools for calculation of loads and flight mechanics.
And they come up with an iteration of what the design is going to be.
So that is basically how it was done.
Now, some of that was sort of like the ground rules.
Some of the things that came out of it, once we started putting the total system together, we showed that the fully reusable with a fly back booster was greater than $5.1 billion.
So that was thrown out.
Now, there is a question, and that's what I asked Dale, should we have said, hey, we need to money to really have a fly back booster?
But they gave us a constraint of $5.1 billion in 1971 dollars and it didn't make it.
I showed you, many configurations were studies.
And the turnaround time of 14 days dictated a landing with a winged vehicle on a runway.
You weren't going to be able to land this in the water or with parachutes.
You wouldn't have to land it on a runway so you could quickly turn it around in 14 days.
The payload, deployment and retrieval requirement determined the location of the Orbiter and the launch configuration.
If you look at that large payload bay, it would be very difficult to put that on top of the vehicle.
One of the requirements that NASA has now is the CEV should be on top of the stack.
Well, if you're going to have that large a vehicle, it is pretty hard to put it on top of the stack, particularly with wings.
This is what we had to come up.
These are some of the results.
And these are all systems studies.
This is systems engineering.
Now you're getting to look at what the design is starting to look like.
This is the agency commitment in March of 1972.
In May of 1972, you had the North American Proposal.
And then I became Orbiter Project Manager in August of 1972.
We did a study.
And PRR is the preliminary requirements review.
But that was the configuration.
We made some changes.
And the production commitment was made in May of 1973.
This chart, which was sort of the gee-whiz chart at one point in time, showed the 1971 dollars cost per flight for the Thor, the Atlas, the Titan 3c, the Saturn 1b and the Shuttle.
And that is payload to orbit.
So you can see that the thing that was really missed in the Shuttle was the $10.5 million cost per flight.
Yes, sir.
What is the notch on top of the second two that disappeared?
That up there?
That was their data probe.
It was taken off because it wasn't needed.
Good question.
Thank you.
And, by the way, do not hesitate to interrupt me and ask me questions any time.
And, if you don't understand what I'm saying, please stop me, tell me slow down or ask me any questions you would like to ask me.
There are data probes on the Shuttle.
Major configuration decisions.
And, by the way, you don't see them on the Shuttle because they are inside the thermal protection system.
And they are only sort of turned outwards to take data when you get down to about mach 3 and you're through the heating.
Otherwise, they would burn off.
In the approach and landing test they were out front.
When we separated from the 747, I am going to tell you that story in a minute.
Here are some of the major decisions.
We were going to go with a hydrogen/oxygen main engine.
That is one of the system problems you have to decide.
Once you decide what kind of engine you are going to use, that basically sizes the tankage.
And I will show you that on another chart.
As I said, this size, the liquid oxygen/hydrogen tanks are not reusable, and I will make that point to you a little later on, but once you decide what kind of engine you are going to use, that size of the tank because of using the equations of motion you can figure out how much propellants you need, you get the density of the propellant and now you know what size tank you are going to use.
Yes, sir.
A couple slides back you had said the main engines of the Orbiter had to be recovered.
Right.
[AUDIENCE QUESTION] It was a separate contract, but it was placed inside the Orbiter.
When the Orbiter came back the engines came back with it.
Was that your question?
Yes.
For example, the Johnson Space Center was responsible for the Orbiter.
And you're going to have the person talk that was responsible for the engine, J.R. Thompson.
He was at the Marshall Space Flight Center.
But the engines were installed in the Orbiter so the Orbiter brought them back.
Solid rocket boosters provided the additional propulsion required to get the Orbiter in orbit and the solid rocket boosters were designed to be recoverable and reused.
Those were some of the system studies that led to the configuration.
Yes, sir.
At that period, was there any discussion of the environmental impact of solids being used at 60 flights a year?
Yes there was.
There was quite a bit.
I don't recall the details, but I do know there was a lot of work on that many solids being used.
And I guess we basically put that to rest, but there was a lot.
I don't know the details of it.
In fact, that might be a good question.
If you are not here, we will ask J.R. Thompson because J.R. should know that.
I met with some of the Russians who worked on the Buran which was the Russian comparable to the Shuttle.
One of the big changes was that they said how could the Americans have used solids for that many flights?
It was studied and actually put to bed, or put to rest, should I say?
I don't know the details, but that is a very good question.
There are going to be various speakers, as I said, and J.R. should have that answer.
Solid rockets recovered and reused.
Well, some of the things I have said before, the Orbiter entry cross-range required delta wings.
To go 1100 nautical miles cross-range you needed delta wings.
Deletion of the air breathing engines for moving the Orbiter required the Boeing 747 to carry the Orbiter.
Let me tell you that story.
All of you are very familiar now that when we land at Edwards Air Force Space, we put the Orbiter on top of the 747 and we fly it back to Kennedy.
Well, I was the Orbiter Project Manager, I became Orbiter Project Manager August of 1972, and I was having all sorts of problems.
The first thing they did to me was cut my budget in half.
The OMB cut my budget in half.
That was the first thing that happened.
And then I just had a lot of problems.
But I had worked on the Apollo Program.
I had a lot of friends in the organization, although I was Orbiter Project Manager.
Three of my friends came into the office one afternoon, I forget, maybe two or three months after we started, and said Aaron, we have a great idea.
I said what's that?
They said we can put the Orbiter on top of a 747 or a DC10 and ferry it back to Kennedy from Edwards Air Force Base.
We may have to make one or two stops and ferry it back.
I look at him for a moment and said that is absolutely the dumbest idea I heard in my life, and I basically threw the people out of my office.
And these were my friends.
Well, these people will not take no for answer.
It happened to be they had another very good quality.
They were all world-class model airplane builders.
And these guys had won competition all over the world, three of them.
So they came back about ten days later.
And I don't know how many of you have seen the Johnson Space Center but we have a lot of acreage out there in Texas.
They said come out, we want to show you something.
They had built a radio controlled model of the 747 and an Orbiter and actually flew it for me and separated the Orbiter from the 747.
That is how it got started.
And so we eliminated the air breathing engines.
But I remember throwing them out of the office.
Fly-by-wire with a digital autopilot.
Yes, sir.
Are you saying that you had air breathing engines and the Orbiter itself would fly back to Edwards Air Force Base?
That's a good question.
I missed that point.
Let me explain it to you.
That is exactly right.
If you recall, when the Orbiter lands, the nose gear is very short.
What we had to do, he asked a very pertinent question, we had to actually replace the landing gear with a different landing gear that caused the Orbiter to have an attitude like this.
It wasn't like this.
It was like this.
We put air breathing engines on and took off and had to have five in-flight refuelings to get to California.
Strap on air breathing engines.
And, plus, a different landing gear.
And we took off horizontally.
And five in-flight refuelings to get from California to Florida.
And you brought up an important point that I left out, that's the reason why we changed it.
Thank you very much.
And, of course, we went with the fly-by-wire with a digital autopilot.
This was a very fundamental change.
The astronauts at that time did not like this very much.
Now, when you get all these new pilots in, they wouldn't know what you were talking about.
Why not have a fly-by-wire system?
They didn't like it at one time, but we went with a fly-by-wire with a digital autopilot.
Even though you are going to have a special briefing on the guidance navigation control system, I am going to talk a little bit more in detail about that because that happens to be my expertise.
So I am going to talk about that a little bit in more detail.
Yes, sir.
What does fly-by-wire mean?
It means that you actually have a computer which actually controls the surfaces.
Whereas, in past airplanes, your stick actually had cables that controlled the surfaces.
So you get a lot more performance.
It is a little bit confusing in the sense of a wire.
Don't think of it as a hard wire which is like the old type of airplanes where there was a cable.
When you pulled on the stick there was actually a cable that went back to the ailerons and the rudder and everything.
And, as professor Cohen says, everything now has a computer in the middle.
And what you're really doing is flying the computer.
And the computer then issues the commands to the hydraulic system.
But the Shuttle, I guess, was the first vehicle that really had that system.
There were no commercial planes flying with that system or military planes flying.
And the real concern was safety and reliability.
Suppose you have a computer problem, what are you going to do?
I will talk about that in detail.
That's a good question.
Any other questions.
Those are very good questions.
I appreciate them.
Yes, sir.
Can you just explain cross-range for a moment?
Cross-range.
Realize the Orbiter is a glider so-to-speak.
Not much of a glider but a glider.
And, actually, downrange would be going in this direction.
Cross-range would be out of plane direction, so you could actually maneuver out of plane.
We have a nice globe here.
I am not going to carry it up to the front and I don't have a piece of chalk either.
These are very good questions, by the way.
Thank you very much.
Here is the United State roughly.
This was designed for the military requirements.
They wanted to be able to, for reconnaissance satellites, basically, you want to be in polar orbit because you are going around, let's say this is an orbit here, then the earth turns underneath it, and so you basically fly over all parts of the earth.
That was the basic military requirement.
They wanted to be able to launch out of Vandenberg on the West Coast into a polar orbit.
And, because of security reasons, this sounds a little bit strange when we think back on it, but in a time of crisis, remember we were in the Cold War and everything, you wanted to be able to put a satellite up without necessarily giving the other side a chance to make all the radar measurements on the Shuttle and everything and figure out right away where the satellite is.
And also there might be hostilities.
The Shuttle was a strategic asset.
So basically they wanted the Shuttle to be able to land the very next orbit.
Well, all right, you take off from California, you fly over the pole and you deploy your satellite.
And, by the way, we have never, with all the satellites we have deployed, ever deployed a satellite on the first orbit.
That would be an incredible feat, but that was the requirement.
So you fly over the pole, you come back around.
Now you're ready to land in California.
But, during that time, the earth has turned by a thousand miles, 24,000 mile circumference in 24 hours.
Actually, 1500 miles because an orbit is 90 minutes, 1.5 hours.
Your orbit now would put you right over the Pacific Ocean.
If you just burn your engines, slow down and come down through the atmosphere, that is where you are going to land.
So instead, as you're flying through the atmosphere, you basically have to come down banked on your side.
So, essentially, you're generating a lift vector.
And, instead of turning your lift vector up, you turn your lift vector to the side and that pushes you over.
And delta wings can generate a higher lift vector than the straight wings, and that was the determining factor.
Very good.
[AUDIENCE QUESTION] Well, I think that's right.
Can you address that, Jeff?
My recollection at the time was that even coming in over the Pacific, yes, there were some places where you could land on land but it would mean landing in hostile territory, the Soviet Union or Eastern Europe, and we didn't want to do that.
So the cross-range initially was to be able to make Australia.
Well, I'm sure that's right.
Certainly, there were landing sites that we had all over the world.
It could have been a much smaller cross-range if we were willing [OVERLAPPING VOICES].
And, I will say, continue to remind us if we don't explain some basic things.
Because we've been dealing with this for so long that some of these things just seem like second-nature.
You weren't even born when the Shuttle started flying.
And so the level of backgrounds is going to be very different.
We want to bring everybody along with us.
So if there is something that we say the same thing, you know, the question about hypergolic fuel, if we use a term and you don't understand it, we will try not to use a lot of acronyms.
I cannot guaranty that the speakers won't use acronyms.
Don't be embarrassed to just stop and say what does that mean, what are you talking about?
But, as we go through the course, you are going to get more detail, more detail.
For example, you have a total lecture by a man named Henry Pole who is going to talk about the reaction control system.
He will give you all the details you want to know about hypergolics and storables, more than you ever wanted to know.
So you are going to get more details as you go through.
But don't hesitate to ask questions because it's good for us.
Let me tell you another satellite story.
My wife and I moved to College Station.
My wife was working at a junior college in the registrar's office.
She was working with a young lady and Apollo 13 came out.
Well, Apollo 13 happened to be my first mission when I was Manager of Command and Service Module so I know a little bit about it.
In fact, I am going to give a lecture in Dick Batten's course on the 26th on Apollo 13.
But my wife talked to this young lady and said I went to see Apollo 13 yesterday.
And she said it was so exciting.
I didn't know how it was going to end.
[LAUGHTER] So it is a frame of reference.
Well, we've talked about this.
The size of the payload bay is 60 feet long by 15 feet diameter.
Size of the crew cabin defined to be over 2600 cubic feet.
The payload, as you know, is 65,000 pounds at liftoff, 35 pounds at landing.
And what you need to understand is the Orbiter is a launch vehicle, it's a spacecraft and it's an aircraft.
And when you look at the two different systems, I worked on Apollo, as I said, very much and on the Shuttle, and there is no question that the Shuttle is much more complicated than Apollo.
On the other hand, the Apollo mission is much more complicated than the Shuttle mission.
But this is something that makes it very, very interesting.
This is a very old chart.
And, in fact, this is the original chart.
It comes out yellow.
This was the original cost estimate and how they totaled up the $5.15 billion.
We talked about that.
And that was the very original cost estimate done on a cost analyst chart.
There were no personal computers at the time.
In fact, I found that in my files.
That's a very old yellow chart.
It's hard to believe some of those numbers.
On the other hand, I had a gentleman who worked for me who got his PhD at the University of Colorado and his subject was the cost of the shuttle Orbiter.
And it turns out that if we had those two years of inflation we really would have made that cost on the $5.15 in 1971 dollars.
It only ran over by about 10%.
That's right.
Let's now talk a little bit about configuration.
I don't know if you can see this chart or not.
Now, you might say how did you go from that conception in August of 1972 to something with all these numbers on it?
Now, this is really a case in systems engineering.
And let me explain to you what I mean by that.
Certain assumptions had been made.
One assumption was that in the Orbiter it was going to weigh about 175,000 pounds without payload and you were going to have a 65,000 pound payload.
You had to make that assumption.
You also had to make an assumption that you were going to use a liquid oxygen liquid hydrogen engine.
And why did you select that?
Today I'm not quite sure it was a good decision, but we did.
Why?
Because of specific impulse, the ISP or the performance of the engine was the highest for the liquid oxygen liquid hydrogen chemical compulsion that we know today.
So we selected that.
Well, when you use the equations of motion, you integrate the equations of motion or any other techniques you may want to use, you then show how much propellant you're going to need to get that into orbit.
Then you know that liquid hydrogen has a density of about 4 pounds per cubic foot, very low density.
That means this big external tank is mostly hydrogen.
It's a very, very big volume.
And liquid oxygen is about 70 pounds per cubic foot.
That basically sizes your external tank.
It's a very simplified approach but that systems engineering.
You get much iteration because there is an old adage that the devil is in the details.
So you keep iterating on that with this expert team.
You then say, well, that's still not going to be the most efficient way to get you to orbit.
You say you need really a stage system, so you put the solid rocket boosters on.
You don't have what they call a single-stage-to-orbit, you really have more like a two-staged orbit and you size the solid rocket boosters and determine when they have to come off.
Basically, that is how you go about doing the systems approach using systems engineering to come up with a configuration.
Now, that is a very over-simplified way of doing it, the statement I made, but that's basically how you do it.
And these are some of the dimensions that then come out of the vehicle.
And I think they're in that handout we gave you also.
And it looks like I am going to take off.
I did know that this chart tries to lift off on me.
I don't know what the best way to do it is.
You can find those dimensions, I believe, in that paper that was handed out.
But this shows you all the dimensions of the Space Shuttle system, the solid rocket boosters, the external tank and the Orbiter.
Now, an interesting sideline in this, which I got to thinking about after the Columbia accident where the foam came off.
Of course, the reason why we put foam on was because with liquid oxygen, liquid hydrogen, at those temperatures you are going to have a lot of ice form.
And when ice forms and comes down and hits the Orbiter with the thermal protection system, I am going to talk about that in more detail, you are going to do some damage.
So one solution would be to put foam on the tank to eliminate the ice.
And, of course, you would assume that foam could stay on the tank.
Another solution would have been to not go with the liquid oxygen, liquid hydrogen engine and go with what we call storables.
Of course, they wouldn't have gotten as much performance and you probably couldn't put 65,000 pounds of payload into orbit.
But the point of bringing this to you, those are some of the decisions, as you go into various projects, you need to challenge the requirements because the requirements are really going to decide what kind of system you are going to have.
That's really the point I wanted to make.
Are there any questions?
Yes.
[AUDIENCE QUESTION] That's a good question.
One thought at one time was to put the liquid hydrogen tank in the Orbiter and that would basically isolate it.
You'd put it inside.
Or, put insulation on the inside, as you pointed out.
Those were thought about.
The decision, obviously, was not to do it.
I don't recall the reason why.
But, when J.R. Thomas comes, I will ask him that question.
That is a very good question.
Yes, it was two things.
Could you put some of this propellant inside the tank?
And the other is could you put insulation inside the tank itself?
And that was looked at.
And I guess it was complicated and very costly to do it that way, but it was looked at.
That's a very good question.
By the way, these are very good questions.
One thing, when you look at this external tank, only the very upper part is for oxygen because the density of hydrogen is so low.
I mean all of this part of the tank is for hydrogen.
So, to have put all the hydrogen inside the Orbiter, you would have a very different looking Orbiter.
You wouldn't have been able to do it.
But she really said could you put insulation inside the tank [OVERLAPPING VOICES].
Any other questions?
But that's basically how the system evolved.
Now, I oversimplified it to quite an extent but you have to realize there were a lot of iterations.
One other key thing, though, when you have new systems engineering, you do it in a team.
You usually have different capabilities on the team, arrows, mechanical engineers, electrical engineers, so you have different types of disciplines on the systems engineering team that can help you do that.
And then you have a systems engineer that sort of heads it up.
But we're going to concentrate more on the Orbiter.
And we have one of the speakers who is going to concentrate on the engine and so forth.
That's what the system looked like.
The other thing that is interesting, I think, is what the mission profile essentially looked like.
And this happens to be for STS-5, but this is the basic mission profile.
Now, before I start that, let me say this gentleman right here has flown that profile five times.
And so he's done that five times.
He can talk much more about it than I can.
One time, I will tell the story, I'm just about tired of hearing it, but when I was Director of the Johnson Space Center we had a lot of visitors.
One time I had Mr. James Baker, who at that time was Secretary of the State, and Eduard Shevardnadze from Russia who had the same position in Russia at the time.
And I took them over to Mission Control as a visit and put Mr. Shevardnadze down in the flight controller seat and was going to let him talk to the crew.
Not knowing that Jeff Hoffman was on duty at the time, Edward Shevardnadze spoke in Russian up to the crew.
And, before the interpreter could answer it, down comes this beautiful answer in Russian from Jeff Hoffman.
So that really floored both Mr. Baker and Edward Shevardnadze, and me.
That was an interesting story.
Here gives you a little profile of the Shuttle mission.
You lift off.
You've reached max dynamic pressure in about one minute.
About 3800 feet is where you reach dynamic pressure.
You have the SRB sep in about two minutes.
You remember in the Challenger action, I believe it happened at about 60 seconds.
But in two minutes you get SRB sep. And it lands the SRB by parachutes.
You have MECO, which is main engine cutoff.
The main engine cuts off.
And that, to me, is the biggest issue.
When that main engine has to burn for over eight minutes that is taking all the propellant out of that big tank, the liquid oxygen, liquid hydrogen firing the main engines.
What's the velocity of the main engine cutoff, do you know?
It's within a few hundred feet per second of [OVERLAPPING VOICES].
And then you get external tank separation.
The external tank comes down about 9000 miles downrange and lands in the Indian Ocean.
Then you use the orbital maneuvering system, the OMS engine, the pods on the back of the Orbiter.
You use that for a very short period of time to get into orbit.
Then you have your operations, whatever you're going to do, go out and service the Hubble or whatever.
You de-orbit with the orbital maneuvering system engine.
You have entry interface at 400,000 feet.
And why do you pick 400,000 feet?
Because that's where you start sensing gravity.
You get about 0.05 Gs at 400,000 feet.
And then you reenter and land either at Edwards or Kennedy.
[AUDIENCE QUESTION].
Yes?
[AUDIENCE QUESTION] Why does it turn?
Yes.
You mean liftoff?
Yes.
Well, that's a very good question.
Let's see if I can repeat that very accurately.
The fact is that we used the Apollo launch pad and the ditches for the flame bucket, where the engine goes.
So, in order to get it to the right alttitude, we had to make that maneuver.
You had to make that roll maneuver during liftoff.
Is that what you're talking about, the roll maneuver during liftoff?
Yes.
We had to make that roll maneuver because we weren't in the right orientation.
We had to make the roll maneuver because it wasn't oriented correctly on the pad.
Now, let me answer your question.
The Buron was a Russian vehicle.
Well, Dan Brandenstein who was an astronaut headed the Astronaut Office saw that the Russians made this roll maneuver.
And he asked the Russians why they made that roll maneuver?
I mean they didn't have to do it, he said, because you do.
[LAUGHTER] That's true.
Did you hear that story?
Did I say it correctly?
I think so.
He said because you do.
That is a good question.
There may be one other aspect to it, and that's the question of why does the Shuttle actually fly upside down on the way up?
And I believe the original decision was aerodynamics because the thrust is asymmetric.
You have the external tank, and the Shuttle is sitting on the external tank, so the thrust actually has to be through the center of mass of the whole system.
So it is actually flying not straight but is a little bit screwed.
And, for aerodynamic purposes, I guess they figured there was less stress.
Although, recently they've started partway through the launch doing another roll maneuver.
And I think that's for communications.
I'm not 100% sure when they started doing that again.
I'm not sure either.
But the early part of the launch, when you're riding the solids aerodynamically, you go through max Q you're in much better shape if you're upside down.
Yes?
[AUDIENCE QUESTION] Well, to use a big solid rocket booster, it's not very efficient.
Many people have studied single-stage-to-orbits.
Let me just ask a question.
I know we've got a mixture of aero-astro and TPP, ESD.
Are there people here who have not seen the rocket equation?
Everybody.
Well, OK. That's very important.
Let me just very briefly If you have a rocket and you want to increase the velocity of the rocket, I mean that's a critical thing, by a certain delta V, you can take the mass of the rocket when you start the burn, which we call M sub i, the initial mass, and the final mass.
And, of course, the initial mass equals the final mass plus the mass of the propellant.
And that equals an exponential to the negative power of the velocity decrement, which you're trying to put in, divided by the exhaust velocity of the propellant that's coming out.
Now, this is usually phrased in terms of what we call the specific impulse times gravity because it's done by engineers.
That's right.
So you will hear reference to the specific impulse and the units of that are seconds.
But, if you multiply that by the gravitational acceleration, it will actually give you the exhaust velocity.
Because this is in an exponential it is extremely sensitive to the exhaust velocity.
Whatever delta V you are trying to get out, if you can add a few more seconds to the specific impulse then the actual mass of propellant that you have to burn in order to create that delta V goes down substantially.
Now, for hydrogen/oxygen, the specific impulse is about 450 seconds.
For solid rocket motors, the specific impulse is on the order of about 250, something like that.
That's what I thought it was, about 250.
That's almost a factor of two.
And you put that into an exponent, if you try to get into orbit with purely a solid rocket motor, you need a lot more propellant.
And that means the payload mass you can take into orbit is reduced.
That's why, in addition to using hydrogen and oxygen, and we will have a lecture specifically on the main engine, they got every last ounce of performance out of it by running it.
You'll hear references to running it at 104% abraded thrust, 109% abraded thrust.
And all of these have engineering consequences because, in order to increase the exhaust velocity -- Which, of course, if you do that, that's in the denominator so that's going in the right direction, you have to increase the chamber pressure, the temperature and all of the design problems.
He is going to ask you the question why don't you use liquids all the way?
I knew what your question was.
[AUDIENCE QUESTION] He is asking why don't you use the liquid hydrogen engine all the way rather than using solid rocket boosters?
Well, first of all, why not just have a bigger fuel tank, one big fuel tank and just use the Shuttle engine?
That's what he's saying.
That would be essentially single-stage-to-orbit.
And I'm not going to spend a lot of time with the rocket equation, but the delta V that you need to get into orbit is, well, orbital velocity is just under 8 kilometers per second.
But to that you have to add the gravity loss [OVERLAPPING VOICES].
And so the overall delta V is effectively about 9 kilometers per second.
If you put that into the equation, you actually can get the ratio of the final to the initial masses.
And even for a hydrogen/oxygen engine you can basically almost 95% of the mass sitting on the launch pad has to be propellant.
Which means that all of your structure can only be about 5%.
Now, if we could make super strong, super light structures, like an eggshell, then, in principle, we could get to orbit without staging.
Now, the first thing you learn in Rockets 101 is why you cannot do single-stage-to-orbit.
It is because right now we do not have the technology to do that.
Well, see, many people have studied single-stage-to-orbit.
In fact, NASA tried to build one the X-23 and we failed.
What they say is really what you need is to be as smart or as high-tech as an eggshell, be able to make the structure as thin as an eggshell, hold all the fluid in it and have very low density.
Some day maybe we will do it.
[AUDIENCE QUESTION] Basically, by using the boosters or, in a more traditional rocket, a first stage, once you've exhausted the fuel then you drop all of that structural mass.
And so, for the second stage, this initial mass becomes much lower.
[OVERLAPPING VOICES] The other thing that's interesting is that if you go back to Apollo, during Apollo there were three ways to go to the moon.
One was direct.
Von Braun wanted to build a large vehicle called the Nova where you actually lifted off from the Cape and sent the whole vehicle right to the moon.
The other was to do earth orbital rendezvous.
And the other way was to do Lunar orbital rendezvous like we did.
It turns out that that Nova vehicle had to be so big that that was ruled out, so we wound up with earth orbital rendezvous.
Yes, sir.
How much did the role of geography play in the design considerations?
Because just looking at that, I mean, you have to have the external tank separate at a specific [OVERLAPPING VOICES] talking about launching in Vandenberg.
Originally we thought about launching out of New Mexico.
Actually, when you're doing flight design, depending on whether you're going into a due east launch to go into a 28 degree orbit or a high inclination orbit, the placement of the external tank reentry is a major factor in flight design.
We spent a lot of time on that.
And, in fact, often the trajectory has to be shaped to be not quite efficient as it otherwise might be because you have to control the landing of the external tank.
But your question is a good one.
I did some consulting for a company, after I retired, and we were looking at a commercial launch vehicle.
And we were trying to launch the vehicle out of various places in the United States.
And it's very difficult to get approval because of the concerns they had for types of failures or anything that you're going to do.
Yes, sir.
I'm not sure why I thought this, but I was under the impression that the external tank burned up in the atmosphere.
No.
It didn't hit down.
It breaks up in the atmosphere.
It didn't come down as one big glob.
Some of it burns up, but some of it does get back to the ground.
OK. Is there any talk about a need to clean up all that?
Well, they haven't yet.
I'm not an astronaut.
That's mainly aluminum.
Remember, the liquid hydrogen, liquid oxygen inside evaporate and are non-toxic.
They question that Dr. Young asked about the solid rocket booster environmental problem is a real problem that had to be solved.
The other problem I had, I worked on trying to get the Galileo cleared because it had a radio thermal nuclear generator on it.
To get that cleared going into orbit is a big job.
I worked on that and thought I would never finish it, but we finally got it cleared.
But it was a very tough job.
We had RTGs cleared to be launched.
I don't remember specifically all the noxious chemicals that come out of the solid rocket boosters, but it is pretty nasty stuff.
Yeah, it is.
I remember one of my launched, the families and guests are taken to a launch site viewing area about three miles inland from the launch pad.
And it just so happened that the wind was blowing onshore that day, it was an afternoon launch, and my brother is a real space nut who always liked to watch as the rocket went.
After about seven and a half minutes it just sort of disappears over the horizon.
But, after about five minutes, the solid rocket exhaust was actually approaching the spectator area because of the wind blowing in.
And they made everyone get on the buses and drove them away so that nobody got injured.
And my brother was really annoyed because he didn't want to leave.
If we have time, I would like to talk to you a little bit about the Challenger accident, talk to you about the O ring seal.
I would like to talk to you about that.
I don't want to sweep those things under the rug, but I would like to talk to you about that because I've got my own thoughts on the problem that I would like you to hear.
OK. Any other questions on the mission profile?
Shall we keep going?
Yeah.
Tell you what.
Let's take a two-minute break, stand up and turn around.
An hour is a long enough time to sit for a bit.
I am assuming that everybody is aware that the Russians, at one time, had a space Shuttle program.
And they manufactured a vehicle which looked almost identical.
Not exactly but almost identical.
And it was no accident because they used our plans.
They used our thermal protection system, too.
And they only ended up flying it once.
And they actually flew it unmanned.
And, despite a little bit of nail-biting during the landing, they did recover it successfully.
And then they discovered, just what we had discovered, that it was a lot more expensive to operate than they had anticipated.
And they had a lot less money than we did.
And so it basically never flew again.
They had crews of cosmonauts who had been training to fly the Buran, but there were actually two differences.
The first was that they put the engines on the external tank.
This did two things.
First of all, it did improve the performance.
And also the Russians were turning out engines on assembly lines basically.
They turn out a tremendous amount of rocket engines.
And I guess, from their point of view, and I don't know the details of the performance of their engine, how their main engine is compared to ours, I think they actually had four engines, if I remember.
[AUDIENCE QUESTION] The Orbiter was just a glider, and it was no difference.
It was an unpowered glider.
They did learn one thing from us that when you have a delta wing vehicle you are very, very sensitive to the center of mass of the system.
And it was a problem because the Orbiter, when it flies back through the atmosphere, it hits the top of the atmosphere at mach 25 and then flies all the way down to subsonic.
And the control characteristics and stability characteristics change throughout that flight envelope.
And we will have a lecture specifically on the aerodynamics of the Shuttle, but it turns out that the Shuttle is extremely sensitive to the forward CG.
And I think it was around the mach 3 flight regime, if you're just an inch or two forward of the critical area in the CG, you can lose control.
And so we actually, on many flights, they always do a weight and balance on the Shuttle before a launch, have had to be put lead ballast in the aft engine compartment of the Shuttle just to get the CG far enough back to get mach 3 stability.
I hate to tell you how many tons of lead we've launched into orbit over the course of the Shuttle program because of the CG.
And, if you look, the delta wing profile of Buran is slight different from the Orbiter because I guess they learned the lesson.
And so they were not as sensitive to the CG.
But, of course, once you build an Orbiter you cannot really change it.
And so we were sort of stuck with it.
To add on, one of the other concerns was having the Orbiter with the thrust behind the Orbiter and this large mass in the tank you could get a pogo-type activity.
And so one concept that Max Faget had early in the program was to have a swing engine, actually have the engines fire on the back of the tank and then, when you get ready to separate the tanks, swing the engines back into the Orbiter and bring them back into the Orbiter.
And we threw that out.
That was a little complicated.
You're going to have a discussion on mechanism in mechanical systems.
And, mechanisms in mechanical systems, everybody can design but everybody has a hard time making them work.
When you put up an electrical schematic everybody accepts it.
When you put up a mechanical drawing of a mechanical system everybody has an idea of how it is supposed to work.
Let's move on.
I've got some pictures now just to show you.
It is showing a profile.
And then, of course, you've seen the vehicle when it gets to roll out of the assembly building.
There it is stacked.
And then it's on a crawler.
This crawler basically was used for Apollo, and the crawler takes the vehicle out the pad.
Just one thing so that people notice the difference here.
Is anyone familiar enough with Apollo to be able to say what the fundamental difference is with the crawler mechanism here and the crawler in the pad?
Well, if you look at the picture of a Saturn rocket being rolled on the crawler, the whole launch tower was on the crawler.
And so they rolled the whole thing out.
With the Shuttle it's a little different.
I will be bringing in some pictures at some point to show you some of these details, but they actually cut off the top part of the Saturn launch tower because the shuttle stack isn't quite as tall.
And they added a movable what's called a payload change-out fixture.
And so once the shuttle actually gets on the pad, there are actually railroad tracks here.
Is this a better picture of it?
OK. Yeah.
This whole enclosure rolls over and forms a kind of hermetic seal around the shuttle so that you can open the cargo bay doors which, by the way, cannot support their own weight in one G. So you have to put an external strong back on them.
And, in the meantime, you put the payload that is going to be installed, this payload canister here, that is installed in the change-out room.
And then it is swung over, the doors are open, the payload is installed in the bay.
And then this forms a protective enclosure over the shuttle.
And it is not swung back usually until the day before launch.
See, now this is again another detail in systems engineering.
You go in for that schematic or that little plan form of the Orbiter, and you did iterations of the size and then you had to do iterations of the launch complex, how it was going to put payloads in.
And that's all systems engineering, certain levels of systems engineering.
I mean it's very easy to say, well, we will just put the payload in on the pad.
But the actual development of the mechanisms to do that is extremely complicated to say nothing of which, I mean what kind of amazes me is that it has to be done essentially in a clean-room environment.
You're up there crawling around the pad inside the payload change-out room in white coats, bunny suits and gloves on.
And outside the wind is blowing and it could be raining.
There is sand blowing by.
So the whole thing has to be done in a clean-room environment, but it's on the scale of a naval shipyard.
This is a huge vehicle.
So it's really a challenge.
Of course, that's one of the reasons, in all honesty why the cost per flight has gone up.
The original concept, when Dale Myers was Associate Administrator of Manned Space Flight, the man who spoke to you, I went up to him, seeing I was the Orbiter Project Manager.
And we concluded this will be very simple.
We were going to make very standard payloads.
You went, put the payload in, fixed it in, we were going to launch it, deploy it, come back, fail the computer on the pad, we'd go anyway, wouldn't replace the computer.
Of course, that all changed.
The missions became very complicated and we didn't do that.
That is another important point, though.
You better be sure, when you get to be a project manager or manager, you understand your requirements, understand your customer's need.
But, if the ground rules change, your performance is going to change.
So you need to understand that.
Any other point you want to make, Jeff?
Any other questions?
I've just got some other pictures.
Here is the liftoff, as you see, everything burning.
Have we had a case where we hit a bird on launch?
Yes, I believe we have.
Of course, in the Orbiter windows, one of the things we did, I hate to say this, is fired dead chickens into the window to see if you could break the window in the system.
I think we did an aircraft also.
And we actually did, I remember, one simulation where in the simulator, right at liftoff, the instructor came in and threw a rubber bird in the commander's lap and said you've just been incapacitated by a bird strike.
And so the copilot had to take over flying.
You should realize that Kennedy Space Center is actually a wildlife sanctuary, and so there are a lot of birds.
There's a National Wildlife Sanctuary, park rangers and a lot of eagles.
There are eagles and a lot of turkey buzzards.
There is a landing strip that is right in the middle of the bird sanctuary.
And so often, before the shuttle gets ready to come in, they will send planes to sort of buzz the runway to scare the birds away.
They also had a problem at one point with owls.
No, woodpeckers, excuse me.
Woodpeckers were decided that the external tank insulation was a good place to find bugs.
It sounds funny, but think of what that actually potentially could mean.
They had to go down and had loudspeakers and stuffed owls and everything, which they ended up putting around to scare away the woodpeckers.
These are things, again, which I think the original systems engineering never took into account.
We were worried about F equals MA.
And there is a typical deployment of [OBSCURED], if I'm not mistaken, to get the payload into orbit.
Here is a picture of the crew at their station with their CRTs.
And, of course, then you de-orbit.
And this is, of course, not a real picture.
Do I have that right?
This is right.
And then, of course, landing on the runway.
That's a typical profile that we've gone through in terms of showing.
Now, let me talk a little bit more about systems engineering.
When I took over in August of 1972, we had what we called Space Shuttle Orbiter Management Reviews.
That's North American.
This was one of September 1972.
And I think it might be interesting to look at the agenda in September of 1972 what we were talking about.
And here was the agenda.
I'm not going to go through all the paperwork but here was the agenda.
Basic resizing of the vehicle because of certain problems we had in the iteration.
Cabin configuration and docking location.
The Space Shuttle main engine interface control document.
It turns out interface control documents are probably one of the most systems integration tool because interface control document defines how you're going to put the main engine into the Orbiter.
What is interface?
An interface could be this plug.
If you had a plug in a socket, it would be bad if you couldn't plug that in.
But that's an interface.
When you change your tire, if those bolt hole patterns don't match you'd be very frustrated.
So that's an interface.
Interfaces can be as simple as that or they can be very, very complicated.
And so this was the SSME interface control document, how we actually get the main engine in terms of mechanical propulsion put into the Orbiter.
Integrated asset control, abort requirements.
And now we're talking about just procurement, RCS and OMS procurement.
The rocket motor's procurement package.
Avionics definition, leading edge TPS and emergency risk.
These were some of the problems we started talking about.
Now, these are lower level systems engineering problems.
But they are systems engineering to make the definition of the total launch configuration.
Here is an interesting chart.
This is dollars.
Why this is interesting is you see on this chart Rockwell's proposal.
These are $1 million.
$123 million the first year, but when I became Orbiter Project Manager that was Rockwell's proposal.
The first thing they did to me, they told me I only had $73 million that year, so I had to re-phase the whole program.
I had to rephrase the whole program to fit the funding curve that the Office of Management Budget gave me.
This was a major perturbation.
When you do that and you don't get your money as the proposal, it comes out to be a cost increase.
Right away we went a little bit cost increase to 4.6, but as you don't get the funding requirements you then essentially lose your momentum, you lose your schedule.
And once you lose schedule you start costing.
As Dr. Hoffman mentioned, you have the three-ledged stool, cost, schedule and performance.
And that is a continual tradeoff as you go through a systems engineering program.
And the things you have to look at, the things that cause project managers to lose their job is first you're going to recognize that you have a weight increase in the vehicle.
Whatever you build is going to cause to have your weight increase.
The next thing you're going to have is a schedule slip.
And then, when you put those together, you're going to have a cost increase.
And then you're going to have technical problems.
And those are disastrous cases for a project manager.
Those are reasons to firing somebody.
Luckily I made it.
I made it because of a man that you are going to hear talk, Chris Kraft, because he was my immediate boss and he ran interference for me.
And you're going to hear Dr. Kraft talk.
He's a good man to have on your side running interference for you.
You're going to hear him talk.
He's a great guy.
And he's going to tell you what he thinks.
And he'll tell you what he thinks.
But that's a little bit about cost.
Further systems engineering questions in the Orbiter Management Review is Orbiter maximum wing size.
ATP is our authority to proceed.
Wing size is very important.
You've got to get the aerodynamic model wing load squared away.
We said no major line change after program requirement is reviewed.
Controlled dry weight was 170,000 pounds including margin.
And I don't think we made that, did we?
No, we didn't.
More like 200,000.
No ejection seats on vertical flights.
Top located docking port.
Wing area for landing speeds of 150 knots.
Wing stiffness criteria.
Flutter margins.
Control effectiveness for entry and ferry.
Elevon design concepts.
[OBSCURED] crew cabin.
These were some of the systems engineering problems that had to be resolved before you could start manufacturing.
One of the interesting things, I forgot who was head of the astronaut office at the time.
I don't know if it was Brandon Stein.
I don't know who it was.
At which time?
Well, at about the time of the design of the Shuttle.
About this time.
[AUDIENCE QUESTION] I forgot who I asked, but we were trying to get the cockpit laid out.
It was very important to get the cockpit laid out.
And I wanted the crew to have their inputs on how they wanted the cockpit.
You want the crew to be a part of laying out the cockpit.
I called the head of the Astronaut Office over and said we need to lay out this cockpit and you need to come back to me with a decision of how you want the cockpit.
He said I cannot do it.
I've got 100 astronauts over there and they won't agree.
I said I'll tell you what you do.
You tell them that I will give two weeks and then I'm going to do it.
And that really scared them because I didn't know anything about laying out a cockpit.
And we had the cockpit laid out.
So that's sometimes how you have to use management techniques.
Now we're going to get more into the details of what you're going to hear in some detail.
And I'm going to start if off a little bit.
Here are basically the hardware subsystems.
You're going to hear a technical briefing on the thermal protection system and the structures.
You will see in your syllabus when that is, but you're going to hear a very detailed discussion of the thermal protection system, the structures by Tom Moser who actually did the work.
He was actually what I call my subsystem manager on the Shuttle in the early days of the program.
You are going to hear a technical briefing on the Space Shuttle main engines and a very detailed briefly by J.R. Thompson.
J.R. Thompson, a little background about him, he was Project Manager of the main engine at Marshall Space Flight Center.
Then he became Director of the Marshall Space Flight Center.
Then he was Deputy Administrator at NASA.
Now he is Vice President of Orbital Sciences.
You are going to have a detailed briefing on the hydraulic system, the auxiliary power system, fuel cells, orbital maneuvering system and the reaction control systems.
That is going to be by Henry Pohl.
Guidance, navigation and control is going to be by Phil Hattis from the Draper Laboratories.
Environment control and life support of the crew cabin, that is going to be a man who is currently working at the Johnson Space Center, Walt Guy who actually did this job for the Apollo.
Most of these people, by the way, did the same thing for the Apollo that they did for Shuttle.
Very similar jobs, so you can ask these people about the Apollo program.
Landing and mechanical systems by Al Louviere.
Communications and electrical power we're really not going to brief, although we will talk electrical power when we talk fuel cells, but we're not going to really have a discussion on communications and fuel cells.
And then we're going to have some analytical studies on aerodynamics and aerothermodynamics.
These will probably be detailed technical discussions when we get into aerodynamics and aerothermodynamics.
Again, these people did the same on Apollo as they did on Shuttle.
That is going to be your details of what you're going to see.
What I'd like to do now, with the few minutes we have remaining, is talk to you a little bit and give you a little bit of flavor of some of these subsystems, a little bit more detail than that we've previously given you, but not as much detail as you're going to see.
Let's start off a little bit with the Orbiter and just show you what the Orbiter looks like.
And talk a little bit about some of the subsystems in the Orbiter.
The forward reaction control systems is up in the far nose of the vehicle.
And then you have the reaction control system back in what is called the OMS pod, orbital maneuvering system pod.
You have reaction control systems here and here.
That's reaction control systems.
You have your Orbiter maneuvering system back in these OMS pods.
Of course, you have the main landing gear and the nose landing gear.
The wing, faring and the leading edge, which is the carbon material.
This wing edge is carbon material.
And the side hatch.
And the overhead windows.
And then the side windows and the far windows.
The payload bay doors are made out of graphite epoxy.
They were originally made out of aluminum.
We made it out of graphite epoxy because of the weight growth.
We could say quite a bit of weight if we went from aluminum to graphite epoxy.
That was the largest, probably at that time, composite material used in an airplane.
It was made out of graphite epoxy.
And then the main engines, of course, are in the aft end of the Orbiter.
And then you have the vertical tail stabilizer.
And, of course, you've got your elevons.
And you've got the body flap.
The body flap was actually put on originally to deflect some of the heat from the engine, but it actually turned out to be a control surface also.
It actually turned out to be both the focus of a control surface.
That's sort of the layout.
And you can visualize where you use these systems, these active systems.
The reaction control system is used primarily in orbit getting ready to de-orbit.
The OMS is used to help you get into orbit and get out of orbit or maneuvering you want to do.
The body flaps and the elevons are actually used primarily during entry, a little big during assent, during entry.
We do use the RCS system a little bit during entry.
In the high altitude of entry you use the RCS system, but eventually during entry the load just becomes so high the RCS system or reaction control system does not become very effective.
I guess that's about all I want to say.
Any questions on that?
Yes, sir.
I was wondering why the nose gear was so short.
Well, it actually saves weight making it shorter.
It was easier to put in and it saved weight.
And I think also when you land, by having the nose down you actually have an aerodynamic force pushing the Orbiter down on the runway which increases your stability.
And since you don't have to worry about eventually taking off.
Like Professor Cohen said, if they had wanted the Shuttle to be able to take off horizontally from the runway they would have had to raise the nose.
Yes, sir.
[AUDIENCE QUESTION] Well, when you say under-designed, I mean I would call it a minimum design.
I mean if it were under-designed it would break.
I mean there are landing limits.
As I said, for a planned nominal landing, and you have to realize there are different kinds, there are emergency limits and limits for nominal operations, you can take off with 65,000 pounds.
That means if you have a launch abort and you have to return and land with your payload, you will be landing with 65,000 pounds.
But you don't plan to do that.
That's a one off deal.
If you're planning to do it, it means you can do it over and over again and then the limit is 35,000 pounds.
They don't build it like a Navy carrier landing airplane, I will tell you that.
Not just landing gear, I found the tires are just really small.
The tires are small.
They have no treads either.
And the brakes were small.
Actually, the original brakes were under-designed and we kept having brake failures.
In fact, I took the drag shoots off and then we put them back on.
I took them off because of weight and put them back on.
Tires are very interesting.
We're going to have a very detailed discussion on tires, but tires are very, very complicated.
You think by putting more plies on the tires you make it stronger, but not necessarily because when you cycle them they got hot and the more plies tend to hold the heat in and weakens the tire.
Tires are not a very simple thing to design.
They are only used once then.
No.
I think we used these tires, I think, five times, or could use them up to five times.
The brakes are used five times.
I don't know about the tires.
I don't know.
Let me talk about weight and margin.
Let me go back to Apollo.
We had Apollo designed.
And we found that on the Lunar module taking off from the Lunar surface it was too heavy.
We couldn't lift off.
The engines were already built, the tanks were already sized and we couldn't get off the Lunar surface.
And that's not a very good deal.
The astronauts didn't like that too well.
George Low was the program manager at the time.
We were spending, in those year dollars, $24,000 a pound to get weight out of the Lunar module.
And they scraped, they did everything they could to get any [OVERLAPPING VOICES].
You're basically offering them $24,000 for every pound that they could scrub out because we couldn't resize the engine.
The engine was there.
They were literally in there with emery cloth shaving down the aluminum from the inside.
So weight can be very critical.
I don't think most people realize how close the margins were there.
I mean the Lunar module was only certified for, I think, five pressurizations.
And then it was beyond its structure limits.
My statement to many people is the next time we go to the moon we are going to find out how hard it really was because, to tell you the truth, we went to the moon with one computer in the service module and one computer in the Lunar module.
Excuse me.
We did have two computers in the Lunar module.
We had the MIT computer and then we had a strap-down system built by TRW.
That's an interesting point.
I think somebody asked the question last time.
Let me talk about the aft end of the Orbiter a little bit because we did not have CAD/CAM systems at the time.
And let me show you some pictures taken.
I don't know what vehicle it is, but that's the aft end of the Orbiter.
That's where the big engines go into.
This looks like it is going to take off.
If you crawled in the back of the aft end of the Orbiter there is plumbing, there is wiring, there is structure.
Now, had we had computer-aided design at that time, I am convinced we would have a much neater, better design than we have today.
We had to mark everything up.
We marked it up and we changed it.
People bumped their heads on it.
If you go to the aft end of the Orbiter you'd wonder how they can do anything, but that was done because we did not have computer-aided design.
Like airplanes now have virtual markups.
You can actually build a virtual markup right on your computer.
We didn't have any of that.
And this, again, is just to show you what the crew module looks like in a structural point of view.
That was the pressure vessel.
That's basically a pressure vessel going together.
And there were the windows.
But the thing that's interesting about that, it had very intricate welding techniques.
And these techniques were very good and actually proved to be a very, very satisfactory design in a pressure vessel.
I would like to now talk a little bit about a couple of subsystems that you're going to hear much more detail about.
But now venture into some systems and tell you a little bit about them.
It's probably going to be the thermal protection system and the guidance and navigation system.
I will talk to you a little bit about that.
Here is an old chart that shows, now, unfortunately I use English units, maximum heat rate and BTUs per foot second squared, integrated heat load and radiative equilibrium temperature.
And then you see the Apollo return as number four.
And the Apollo design was about 100 BTUs per foot squared integrated heat load, if I read the chart right, and a heat flux of about 300 BTUs per foot squared second with a radiative equilibrium temperature of about a little over 5000 degrees Fahrenheit.
On Apollo that was our baseline.
We used an ablative head shield.
Now, the ablative heat shield has a density of about 130 to 140 pounds per cubic foot and is not reusable.
On the Shuttle it was about 40,000 BTUs per foot squared integrated heat load and 50 BTUs per foot squared second for heat flux with a radiative equilibrium temperature of about 3000 degrees Fahrenheit.
Now, at that point of time we had to come down from our baseline knowledge of an ablative material to what we could use and make it reusable and light.
And there were several competitors at the time.
One was a metal insulation, Rene-41, I believe, shingle-type material, very thin, very high temperature, and it had pretty low density.
And then, of course, there was the ceramic material or the so-called tiles which was basically a ceramic material.
It is made out of silica.
In fact, interesting enough, it's made out of what we call gopher sand that comes from Minnesota.
Why is it gopher sand?
I'm not quite sure, but it's got a very high pure silica content.
And you make it into a mulch and then you actually bake it and it comes out in these tiles.
And, of course, tiles sometimes are hard to bond to the vehicle.
How many people knew the tiles were hard to stay on the vehicle?
Well, I was the Orbiter Project Manager and my home town is San Antonio, Texas.
We would go back to San Antonio to visit my family.
My little old aunts would say their nephew is in charge of the shuttle program and he was having trouble making the tiles stay on the vehicle.
They couldn't understand that because they had tiles in their kitchen and their bathroom and had no trouble at all having those tiles stay.
So they couldn't understand why I was having so much trouble having the tiles stay on the vehicle.
We had a problem.
What would we use?
There were two competitions on it.
One was the competition from two of the research centers.
The Langley Research Center wanted to go with the metallic version and the Ames Research Center wanted to go with the ceramic version.
And at JSC we're trying to figure out which one to go with, and we decided to go with the ceramic version.
Of course, Tom Moser is going to give us some detail because Tom was the guy really in charge of it.
But just to give you a little background, and he'll go into more details of it, the problem we had with the tiles is that when you bonded the tiles, of course, they were basically a ceramic material.
Forget about this layer called densified for a moment.
When you bonded the tiles to the room temperature vulcanizing material, which is basically a high temperature glue, to the strained isolator pad, you got stress risers set up and the tile became very weak.
It broke really at the bond line of the tile because it became very weak.
The stress risers were such that because it was ceramic it broke at the interface.
We had one very cleaver engineer that saved my day because tiles were falling off and I couldn't get the vehicle built.
He said if we densified below a quarter of an inch of the tiles, and tiles could be anywhere from two inches to four inches thick, and by that I mean put liquid glass inside that tile, the bottom became like really a hard surface, no stress riser, you could bond it like this and basically have the inherent strength of the tile.
It no longer broke at the bond line, it broke at the tile, which had a very high load.
So, after we did that, we really did solve the tile problem to a large extent.
That was the day we really solved the tile problem, but I remember this so distinctly that we were doing all these tests and all these analyses and all of a sudden they called me one day and said Aaron, we're just doing a very simple flat-wise tension test and tiles were just coming off the vehicle.
So that is what saved the day.
I think it's another good example of problems in system engineering if you just leave it there for a moment.
Sure.
We worked so hard figuring out how to make these tiles, and they are extraordinary.
You can take one of these tiles, put one side of it into a kiln, heat it red hot and you can hold the other side of it in your fingers.
I mean that's how good the insulation is.
And I guess everybody just sort of figured we know how to glue things on.
You brought up another very good point that I left out.
The problem is we spent so much time figuring out the efficiency of the tiles to do the thermal protection.
In other words, this is fantastic tile.
It weighs about like balsawood, so it is anywhere from 10 to 12 pounds per cubic foot compared to 130 pounds per cubic foot.
It can take the higher temperature ablative material and be reused.
So it answered all the questions we had about thermal protection.
What we didn't do, though, was figure out how we're going to attach it to the vehicle.
But everybody, I think, just assumed, I mean it's almost like your relatives, we glue tiles in the bathroom all the time.
Anybody can glue stuff onto something else.
But we really did a bad job.
In fact, that's an example I use in my course at A&M.
We come up with a function structure.
What functions does this have to perform?
Well, it doesn't actually have to thermally protect the vehicle.
It has got to stay on.
And we forgot about the staying on part.
We talked about the thermal protection part.
And this thing is so good, its thermal diffusivity is so good that, like Professor Hoffman says, you could get the surface temperature to 2000 degrees Fahrenheit and hold it in the palm of your hand.
And so it turned out it was a very, very good solution but we almost blew the day by not coming up with this design.
Why does it have to be individual tiles rather than big long panels?
Well, because of the coefficient expansion.
Now, that's the point of this strain isolation.
And it's another interface problem which is part of the system's engineering.
The structure of the shuttle is aluminum.
Thermally, it expands and contracts by several inches every time you go from night to day.
I mean the Orbiter, we've done these measurements.
One side of the Orbiter is pointed towards the sun.
The other side is cold.
The Orbiter actually bananas by a couple of degrees.
And they did thermal tests in some of the orbital flight tests where the payload bay, the clearances could change by a couple of inches depending on the thermal conditions and designing mechanisms to latch them closed with those tolerances.
So the orbiter is flexible and expands and contracts as a metal.
The tiles don't.
They are rigid.
And so, if you have too large a tile area, the aluminum is going to bend under it and the tiles will crack.
And even with the small tiles, between the aluminum and the tile there is this strain isolation pad.
This of it as sort of like a felt type material.
It kind of eases the interface between the two.
And, of course, if you look at the tile installation on the vehicle, in between the tiles, remember on the last mission we went EVA to fix the gap fillers between each one of those tiles mainly for problems during liftoff to keep the tiles from vibrating and damaging each other.
And those gap fillers, I'm not concerned about going EVA to fix it.
Well, on the other hand, once you see the problem you cannot let it go so you've got to do something about it.
They went EVA and pulled the gap fillers out, but those are the tile installations.
But your question is a very pertinent one.
Why are there so many?
And that's the reason, because of the coefficient expansion.
Then you put gap fillers in between those tiles to keep it from damaging during liftoff.
It does turn out that one of our biggest concerns is going to be how fragile the tiles were.
And they really turned out to be a pretty good system.
We were concerned about losing a tile and then getting heat and having an unzippering effect where you lose a bunch of tiles.
And we really alleviated that problem.
I will tell you another funny story.
This is probably not so funny but it was a story anyway.
I've forgotten what mission it was but these tiles are actually waterproof.
We actually put Scotchgard on them because the waterproofing is in the tile.
And after a certain mission the waterproofing boils out so we had to waterproof the tile.
STS-4 was the hailstorm.
That was the first one where they recognized the problem.
Well, anyway, Rockwell International was my contractor.
The head engineer at Rockwell was a brilliant guy, but he ran a test where he put a densified tile panel in a bucket of this Scotchgard or this rain repellant material.
And he called me about 11:00 at night.
And I was getting ready to go in because we were [OBSCURED].
We just ran this test and all the tiles came off.
I said what do you want me to do with that information?
We're getting ready to back the de-orbit burn.
I mean they were coming down.
I said what do you want me to do?
I said that was sort of a dumb test, wasn't it?
I said is that what happened on the vehicle?
Did you actually put the tiles in a bucket of water repellent material?
He said no.
I said that's a dumb test.
I took it upon myself, after talking to the man you're going to hear, Tom Moser, what do we do with it?
We didn't tell anybody.
Now, I took a big risk on that.
But what could you do about it?
There was absolutely nothing you could do about it on that test.
So that's an interesting case in discretionary project management.
But you will hear more.
Let me just mention the question of waterproofing because, again, that was something that I think at first slipped through the systems engineering.
And it was the fourth shuttle flight STS-4.
I was down at the Cape for some other activity.
The night before a launch they had rolled back the payload change-out room, the whole shuttle was exposed.
Well, a thunderstorm with hail came through, and it pelted the bottom of the shuttle.
There were pox marks all over it.
And, of course, it was raining.
And people realized that the tiles actually were absorbing material.
The outside of the tile is kind of impervious, but once you crack that the rain can get in.
And they actually brought a tile expert down from the Cape and did some calculations.
It was an early flight so we didn't have a maximum payload luckily because we ended up carrying, they figured, probably a few hundred, many over a thousand pounds of water into orbit.
And there was so much water in the tiles that the orbital dynamicists were able to measure the perturbation of the Shuttle's orbit because of all of the water that was evaporating.
And, of course, it was evaporating from the bottom so that it effectively was asymmetrical and had a propulsive force.
And after that was when they came up with the idea of Scotchgarding.
And, if you look closely at each of the tiles, there's a little hole where every tile before every flight they go through with a hypodermic syringe and just inject a little bit of Scotchgard.
And think how critical it is.
You've got over 30,000 tiles.
If you add one ounce to every tile, you do the math, that's a lot of extra weight.
So you want to put in as little as possible Scotchgard but enough to do the job.
This might be a very good subject for you to decide to redesign.
You might think about what are the technologies, what are the differences, how would you do it differently because their technology has changed.
That's why I brought this one up.
And you will hear a much more detailed discussion by Tom Moser, but you might think about this as one of the subsystems to look at.
I guess that's all for today.
OK. We will see you on Thursday.
The formal part of the class will start about five past nine, but I'll just start with some introductory remarks.
First of all, I have got bios from a large number of people.
I haven't done a count yet to see from whom I haven't gotten them, but if you haven't sent me a short bio please do.
It just helps us to get to know who you are.
Let's see.
Second thing.
The next deliverable, which is, I think, in a week or so, I don't have the syllabus with me right now, we would like a preliminary indication from you.
I am leaving it up to you to form your own teams.
Roughly, we've found about 4 people make up a good team.
That is a good way to spread the work around.
If any of you are having difficulty or whatever, let me know, we will try to help, but you are almost all graduate students.
You can organize yourself.
With a preliminary indication of what system you would like to do a study on.
Take a handout.
I am just wondering if you can provide a list of email addresses to everyone so that we can contact each other.
I will be happy to do that.
I will post that.
Let's see.
The second thing.
There are a fair number of listeners in the class, and quite a few of you have contacted me to make sure that it is OK to be here as listeners.
I have no problem with that.
The only thing that I would ask, right now we are more or less filled to capacity.
I expect that the class isn't going to grow after this, so I think we are OK. What I did tell people is that if we get into a seat crunch, obviously, the people who are taking the course for credit have priorities for seats.
So any of you who are listeners please keep that in mind.
Next thing.
I am going to be putting some things on reserve in the library.
This is courtesy of Professor Cohen.
This is the proceedings of a Space Shuttle Technical Conference.
Aaron, do you want to say something just about the background?
The background is that we decided, after the Shuttle had flown a couple of times, that we would have a typical NACA, going back to NACA, but this was NASA technical conference where we actually had technical papers written by the various people that designed the system, some you will hear talk, some you won't.
But these are technical papers of how the systems were developed.
So it is a pretty detailed understanding of the thermal protection system, the main propulsion system, so forth and so on.
So it is a pretty good documentary of how it was done.
This should be a good reference for your papers.
It is 1983.
Like I say, I will go down this afternoon and put this on reserve in the library.
I will also put on reserve a copy of a systems study that was done at Texas A&M for a similar course that Professor taught on systems engineering and the Space Shuttle just to give you kind of an idea of what someone else did.
This was a rather good paper.
I see you gave them a very high mark so presumably that means it is a good example.
Finally, you have all got a copy of this paper by Professor Cohen and Milt Silvera who was also a very responsible person in the Shuttle program.
It is a high level background on the Shuttle and its systems.
I am sure there are some of you here who are space enthusiasts and who probably have followed the Shuttle system pretty closely and to whom there won't be a whole lot new here.
But, on the other hand, we want everybody in the course to come up to a certain level of knowledge about the Shuttle, its systems and the systems engineering that went behind it.
So I think it's good to have something like this available.
It's a nice reference piece.
Today and Thursday Professor Cohen is going to give an introduction into the Shuttle, its systems and the system engineering that went into it.
Before we start that, are there any other questions either about the technicalities of the course or just anything?
Oh, I know one thing.
I have a few comments from people telling me that the pdf files that I posted on the website, some people could read them and some people could not read them.
I have Macs but have been posting pdf files for numerous courses.
I have never had any problem with people reading it before.
First of all, who has tried to look at the files?
OK. Of those people who have tried to look at the files, who has had difficulty?
This is strange.
Well, if anybody has any ideas about what the problem might be, I would be very curious.
I mean have you ever had this problem with other courses?
I will do a couple of experiments and post one or two more documents, both in pdf and in either Word or PowerPoint form.
I would suggest that everybody during the next two days go and try to open them.
Let's see if we can isolate where the problem might be.
Like I said, I've never had this problem with any other courses.
And, in principle, I mean the whole point of pdf files is it's supposed to be compatible across Macs, PCs and all that.
So I don't know what the problem is but we'll see if we can run it down.
And thanks to those of you who let me know that there was a difficulty.
That is all I have.
Any other questions, comments?
What are you kind of expecting with these journal deliverables so we can kind of be just ahead of that?
The idea is to basically put together the highlights of the lecture from the systems engineering point of view.
In other words, what we want -- When somebody is talking about a specific system, we want you to have a brief description of the system, the purpose, what it does, something about the design considerations, something about the operations, kind of the basic concerns from the systems engineering point of view.
And anything they talk about, particular interactions between that system and other systems.
Can you think of anything else in particular?
I think that about it.
OK.
I will think that through a little bit more, and maybe I can come up with a checklist that would help you put that together.
It's not meant to be a big burden.
It's just a way to organize everybody's thinking on how to get the most out of the lecturers, particularly because, as I mentioned, these lectures are all being given, by and large, by different people.
They will have different styles.
We have kind of explained to everybody the structure of the course, what we're looking for, but I cannot really guarantee in advance what the content of the lecture will be.
Professor Cohen and I will make any attempt, if the lectures don't cover some of those critical points from a systems engineering point of view, we will try to stimulate the discussion on that or fill in.
But you should do the same thing.
In other words, make sure that there is something that you think you would like to be getting out of the lecture and you haven't gotten, don't be embarrassed to ask questions.
Something you ought to think about, when these individual lectures come up, is it should be the description and function of the system.
If they don't bring that out then we ought to try to bring it out.
The requirements of the subsystem or the system, what are the requirements?
Because that is very fundamental.
What are the requirements?
The development of the subsystem.
What kind of problems did they have?
What kind of technologies did they have to overcome?
And, of course, the operation of the subsystems.
Those are, I think, key points.
Now, whether you're going to get that from everybody or not, I don't know.
But I think you need to look for that.
And, if you don't, then we will try to develop that.
Are you looking for the notes to be taken in class?
I don't follow you.
I mean I am not looking for a Xerox of what you're writing in the notebook, if that's what you mean.
No.
I mean I would like you to put your thoughts together into a concise form.
And I think that will be a good reference to you, hopefully, after the course on Shuttle systems.
OK?
Over to you.
You are going to have a very unique opportunity.
You are going to have people speak, lecture you on the various Shuttle subsystems in quite a bit of detail.
You will have some people that will be very, very positive about the Shuttle, that think it's a great design, a great operation.
You will have others that will not think it's so good.
And you will have some that will give you just a detailed technical approach of what happened.
What I would like to suggest to you, I think, for your good and for the good of the future, future students, future designers, is that you ought to come up with your own decision.
Was it the right design?
Was it the correct design?
Should it have been done differently?
And, if so, why and how would you do it?
And I think it will be good for, when you go to work on future projects.
And it would also be good, I think, for NASA, something we could turn over to NASA.
I think it's a very valuable thing to do.
So I would suggest you do that as you go through the course.
And I will be very happy to talk to you about any of your ideas that you have through the Internet, through emails or in person discussion.
And you might want to do this later on as the semester develops.
A little bit what I am going to say today really starts off where Dale Myers, your previous speaker, left off.
And it gets into a little bit more detail.
So, as this course develops, you're going to get more and more detail.
But let me start off again, just very much like Dale did, and talk about the Shuttle history.
In 1952, the fully reusable launch vehicles concept was discussed.
People were interested in that.
1962, fully reusable vehicles were seriously considered.
The Air Force study project Dynosaur was cancelled in 1969.
In 1969, NASA adopted the idea of a fully reusable spaceship.
I became the Orbiter Project Manager for NASA in August of 1972.
And, at that time, I also was manager of the Systems Engineering Organization for the first two years.
So I, you might say, the total system at that time and the Orbiter in 1972.
Yes?
Could you tell us a little bit about your background?
Yes.
Well, my background was I graduated from Texas A&M University in 1952 and went to the Army, went to Korea.
And then when I came back went to work for RCA.
And I worked on the microwave tubes, the microwave oven.
In fact, when I told my wife what I was working on, I said I'm working on a microwave oven.
That was in 1954.
That was a long time ago.
How many people have microwave ovens today?
Everybody does.
Well, in 1954-1955, when they came out, they were about $3,000 a piece.
I was working on one and told my wife.
And we'd been married a long time.
I told my wife that I was working on something called a microwave oven.
And you are going to be able to cook a roast in a couple of minutes and a potato in a couple of minutes.
And she looked at me and said that will never sell.
Anyway, that's what I worked on.
Then I worked for General Dynamics on the Atlas and Centaur.
And then, in 1962, I went to the Johnson Space Center and worked very closely on the Apollo Program, worked very closely with the MIT Instrumentation Lab, now the Draper Lab on the guidance and navigation control system.
I became head of System Engineering in Apollo.
Then manager of the Command and Service Module in Apollo.
Then, in August 1972, I became the manager of the Space Shuttle Orbiter.
Then I became Director of Research and Engineering at Johnson Space Center.
And then I became Director of the Johnson Space Center.
And then, for a while, I was the Acting Deputy Administrator in Washington.
And then I retired and went to Texas A&M to teach.
Then Jeff asked me to come to do this, and I am very happy to be here.
Actually, one thing you mentioned reminded me of something, not to divert the lecture today, but the fact that there was specifically a systems engineering group at the center which was separate from the project offices is something which is probably worth talking about at some point.
That's right.
Well, let me just make a mention of that.
When we were in Apollo we sat around the table for many days and months trying to figure out how you define systems engineering.
We didn't even know what systems engineering was.
In fact, today, I'm not sure you'll get a clear definition of what it is.
I want to give you some examples.
As we go through the lecture, I am going to give you some examples of what I think it is and how it was used.
So we will do that.
Let me say just one other thing.
Professor Cohen has these view graphs in electronic form down in Texas and, when he gets back, he will send them to me and I will post them on the website.
Well, I will give you this whole package because there are also some pictures.
I don't know what you want to do with the photos.
Something that is very key in any design and something you really need to pursue, whenever you do a design project, is understand the requirements.
Because, if you don't understand the requirements, you might get a very good product that is useless.
So you have to understand what your customer wants, the top level requirements.
One thing that was handed down to us is it was supposed to be fully reusable.
That was one requirement.
14-day turnaround time.
You were supposed to be able to turn the Shuttle around in 14 days.
Deploy and retrieve payloads.
You had to deploy a payload and you had to retrieve a payload.
Design, development and test is estimated to be $5.1 billion in 1971 dollars.
Dale Myers didn't tell you the whole story, but one reason Dale Myers and I are such good friends is he was Associate Administrator of Manned Space Flight, I was the Orbiter Project Manager and they, they being headquarters, headquarters was always there to help, actually took away two years of inflation.
If they had given those two years of inflation, we would have met the $5.1 billion in 1971 dollars.
And Dale fought for that but lost.
Now, here is where we missed it.
The original cost per flight for 65,000 pounds was $10.5 million per flight in 1971 dollars but for a flight rate of 60 flights per year.
When I told my wife I was doing that, and she's been around the space program a long time, she said you never agreed to that, did you?
Sixty flights per year is pretty hard to do, but that's what we came up with at the time.
Now, Dale mentioned the Air Force requirements, but here were the phase A studies.
The phase A studies were conducted to determine the basic requirements and their effects on design in 1969.
The principle issues were the size and weight of the payload, the cross-range of the Orbiter and what kind of heat material was going to be used.
You have to recognize are we going to use heat-resistant structure or reusable insulating material?
You have to recognize that our background was Mercury, Gemini and Apollo, and they all used an ablative material.
Ablative material cannot be reused because basically the surface changes.
You had to have some kind of insulated material so the surface did not change.
Let me now go through very quickly for you, just for the sake of completeness, Dale showed you a few of the studies.
I want to flash up a lot of phase A studies just to let you take a look at what people were doing.
And you don't have to read the words but you might just look at the diagram.
There was General Dynamic's phase A study.
There was North American Rockwell.
That is the one Dale was talking about because that was where he was.
And there was a phase B study for McDonnell Douglas.
So that is what they all looked like at the time.
Now, I am going to go through some just for the sake of completeness here.
There was, again, another McDonnell Douglas study, Martin Marietta.
So those were basically some of the studies.
You can see they all had some kind of a wing-type vehicle.
Already most of those were delta wings so the decision had already been made.
Interestingly enough, and I am going to quote some names to you, and you might go back and research it.
Max Faget was like the lead engineer.
He was sort of a man that was immortalized in terms of the space program.
He felt we ought to go with the straight wing.
He felt very strongly about a straight wing, but that eliminated the very large cost range the Air Force needed.
Straight wing was easier to build, not as high loads on it and so forth.
Here, interestingly enough, if you look at Chrysler.
Chrysler had a study.
And Chrysler actually had a capsule, but here were more vehicles.
And people were really thinking about this time of a fly back booster where you actually had a booster that returned and came back and landed.
And they also had a land vehicle that landed.
So that was what the real requirement was at the time.
Somebody sort of commented about Chrysler.
There was also Ford.
There were a lot of aerospace companies back in those days.
It has changed a lot, that's right.
And here is McDonnell Douglas, of course Grumman.
So a lot of these studies.
Then you're getting to see something already starting to look a little bit like the Shuttle.
And I am almost through.
And here are more.
These are, I think, getting maybe in the phase B studies.
But you can see some of them are starting to look like the existing Shuttle.
Lockheed.
Now, the principle issues in the Shuttle studies, now we're starting to get a little bit more technical and a little more detailed, is should the reaction control system, now, the reaction control system is basically a propulsion system that controls the vehicle about its center of gravity.
It's for alttitude control, basically.
Should the reaction control system be liquid oxygen/liquid hydrogen or should you use a hypergolic system where you know it is a storable type system.
That was a big issue.
Does everybody here know what hypergolic fuel is?
Let's talk about it a little bit.
Well, hypergolic fuel is a fuel like hydrazine.
It actually has an oxidizer and a propellant in the same fuel.
Now, another issue was a fly-by-wire flight control system.
That was a big issue.
Do we have a fly-by-wire flight control system?
And now everything has a fly-by-wire.
All the military jets have fly-by-wire.
But basically was it going to be a computer controlled system or are we going to have cables fly the machine?
That was a very big issue at the time.
Wind tunnel tests to determine wing size and configuration.
That is a very difficult thing to do.
This is starting to get into what you might say is systems engineering.
Air breathing engines were considered for fly back and later determined to be too heavy.
And some Shuttle studies that still had to be done was the entry techniques, landing speed, what type of landing speed were we going to have and the approach pattern?
So these were all some things that had to be understood during the studies.
The phase B studies were performed in the mid 1970s to determine the preliminary design.
The results showed a fully recoverable Orbiter.
Disposable fuel tank.
Parachute recoverable solid rocket boosters.
High performance hydrogen oxygen engines placed in the Orbiter to recover.
This was all systems engineering that lead up to the design.
So you have systems engineering in various phases of the program.
And usually systems engineering composes of an interdisciplinary team that has been given some assumptions, some constraints.
They have some top level requirements.
They do an iterative process with some tools such as computer tools for calculation of loads and flight mechanics.
And they come up with an iteration of what the design is going to be.
So that is basically how it was done.
Now, some of that was sort of like the ground rules.
Some of the things that came out of it, once we started putting the total system together, we showed that the fully reusable with a fly back booster was greater than $5.1 billion.
So that was thrown out.
Now, there is a question, and that's what I asked Dale, should we have said, hey, we need to money to really have a fly back booster?
But they gave us a constraint of $5.1 billion in 1971 dollars and it didn't make it.
I showed you, many configurations were studies.
And the turnaround time of 14 days dictated a landing with a winged vehicle on a runway.
You weren't going to be able to land this in the water or with parachutes.
You wouldn't have to land it on a runway so you could quickly turn it around in 14 days.
The payload, deployment and retrieval requirement determined the location of the Orbiter and the launch configuration.
If you look at that large payload bay, it would be very difficult to put that on top of the vehicle.
One of the requirements that NASA has now is the CEV should be on top of the stack.
Well, if you're going to have that large a vehicle, it is pretty hard to put it on top of the stack, particularly with wings.
This is what we had to come up.
These are some of the results.
And these are all systems studies.
This is systems engineering.
Now you're getting to look at what the design is starting to look like.
This is the agency commitment in March of 1972.
In May of 1972, you had the North American Proposal.
And then I became Orbiter Project Manager in August of 1972.
We did a study.
And PRR is the preliminary requirements review.
But that was the configuration.
We made some changes.
And the production commitment was made in May of 1973.
This chart, which was sort of the gee-whiz chart at one point in time, showed the 1971 dollars cost per flight for the Thor, the Atlas, the Titan 3c, the Saturn 1b and the Shuttle.
And that is payload to orbit.
So you can see that the thing that was really missed in the Shuttle was the $10.5 million cost per flight.
Yes, sir.
What is the notch on top of the second two that disappeared?
That up there?
That was their data probe.
It was taken off because it wasn't needed.
Good question.
Thank you.
And, by the way, do not hesitate to interrupt me and ask me questions any time.
And, if you don't understand what I'm saying, please stop me, tell me slow down or ask me any questions you would like to ask me.
There are data probes on the Shuttle.
Major configuration decisions.
And, by the way, you don't see them on the Shuttle because they are inside the thermal protection system.
And they are only sort of turned outwards to take data when you get down to about mach 3 and you're through the heating.
Otherwise, they would burn off.
In the approach and landing test they were out front.
When we separated from the 747, I am going to tell you that story in a minute.
Here are some of the major decisions.
We were going to go with a hydrogen/oxygen main engine.
That is one of the system problems you have to decide.
Once you decide what kind of engine you are going to use, that basically sizes the tankage.
And I will show you that on another chart.
As I said, this size, the liquid oxygen/hydrogen tanks are not reusable, and I will make that point to you a little later on, but once you decide what kind of engine you are going to use, that size of the tank because of using the equations of motion you can figure out how much propellants you need, you get the density of the propellant and now you know what size tank you are going to use.
Yes, sir.
A couple slides back you had said the main engines of the Orbiter had to be recovered.
Right.
[AUDIENCE QUESTION] It was a separate contract, but it was placed inside the Orbiter.
When the Orbiter came back the engines came back with it.
Was that your question?
Yes.
For example, the Johnson Space Center was responsible for the Orbiter.
And you're going to have the person talk that was responsible for the engine, J.R. Thompson.
He was at the Marshall Space Flight Center.
But the engines were installed in the Orbiter so the Orbiter brought them back.
Solid rocket boosters provided the additional propulsion required to get the Orbiter in orbit and the solid rocket boosters were designed to be recoverable and reused.
Those were some of the system studies that led to the configuration.
Yes, sir.
At that period, was there any discussion of the environmental impact of solids being used at 60 flights a year?
Yes there was.
There was quite a bit.
I don't recall the details, but I do know there was a lot of work on that many solids being used.
And I guess we basically put that to rest, but there was a lot.
I don't know the details of it.
In fact, that might be a good question.
If you are not here, we will ask J.R. Thompson because J.R. should know that.
I met with some of the Russians who worked on the Buran which was the Russian comparable to the Shuttle.
One of the big changes was that they said how could the Americans have used solids for that many flights?
It was studied and actually put to bed, or put to rest, should I say?
I don't know the details, but that is a very good question.
There are going to be various speakers, as I said, and J.R. should have that answer.
Solid rockets recovered and reused.
Well, some of the things I have said before, the Orbiter entry cross-range required delta wings.
To go 1100 nautical miles cross-range you needed delta wings.
Deletion of the air breathing engines for moving the Orbiter required the Boeing 747 to carry the Orbiter.
Let me tell you that story.
All of you are very familiar now that when we land at Edwards Air Force Space, we put the Orbiter on top of the 747 and we fly it back to Kennedy.
Well, I was the Orbiter Project Manager, I became Orbiter Project Manager August of 1972, and I was having all sorts of problems.
The first thing they did to me was cut my budget in half.
The OMB cut my budget in half.
That was the first thing that happened.
And then I just had a lot of problems.
But I had worked on the Apollo Program.
I had a lot of friends in the organization, although I was Orbiter Project Manager.
Three of my friends came into the office one afternoon, I forget, maybe two or three months after we started, and said Aaron, we have a great idea.
I said what's that?
They said we can put the Orbiter on top of a 747 or a DC10 and ferry it back to Kennedy from Edwards Air Force Base.
We may have to make one or two stops and ferry it back.
I look at him for a moment and said that is absolutely the dumbest idea I heard in my life, and I basically threw the people out of my office.
And these were my friends.
Well, these people will not take no for answer.
It happened to be they had another very good quality.
They were all world-class model airplane builders.
And these guys had won competition all over the world, three of them.
So they came back about ten days later.
And I don't know how many of you have seen the Johnson Space Center but we have a lot of acreage out there in Texas.
They said come out, we want to show you something.
They had built a radio controlled model of the 747 and an Orbiter and actually flew it for me and separated the Orbiter from the 747.
That is how it got started.
And so we eliminated the air breathing engines.
But I remember throwing them out of the office.
Fly-by-wire with a digital autopilot.
Yes, sir.
Are you saying that you had air breathing engines and the Orbiter itself would fly back to Edwards Air Force Base?
That's a good question.
I missed that point.
Let me explain it to you.
That is exactly right.
If you recall, when the Orbiter lands, the nose gear is very short.
What we had to do, he asked a very pertinent question, we had to actually replace the landing gear with a different landing gear that caused the Orbiter to have an attitude like this.
It wasn't like this.
It was like this.
We put air breathing engines on and took off and had to have five in-flight refuelings to get to California.
Strap on air breathing engines.
And, plus, a different landing gear.
And we took off horizontally.
And five in-flight refuelings to get from California to Florida.
And you brought up an important point that I left out, that's the reason why we changed it.
Thank you very much.
And, of course, we went with the fly-by-wire with a digital autopilot.
This was a very fundamental change.
The astronauts at that time did not like this very much.
Now, when you get all these new pilots in, they wouldn't know what you were talking about.
Why not have a fly-by-wire system?
They didn't like it at one time, but we went with a fly-by-wire with a digital autopilot.
Even though you are going to have a special briefing on the guidance navigation control system, I am going to talk a little bit more in detail about that because that happens to be my expertise.
So I am going to talk about that a little bit in more detail.
Yes, sir.
What does fly-by-wire mean?
It means that you actually have a computer which actually controls the surfaces.
Whereas, in past airplanes, your stick actually had cables that controlled the surfaces.
So you get a lot more performance.
It is a little bit confusing in the sense of a wire.
Don't think of it as a hard wire which is like the old type of airplanes where there was a cable.
When you pulled on the stick there was actually a cable that went back to the ailerons and the rudder and everything.
And, as professor Cohen says, everything now has a computer in the middle.
And what you're really doing is flying the computer.
And the computer then issues the commands to the hydraulic system.
But the Shuttle, I guess, was the first vehicle that really had that system.
There were no commercial planes flying with that system or military planes flying.
And the real concern was safety and reliability.
Suppose you have a computer problem, what are you going to do?
I will talk about that in detail.
That's a good question.
Any other questions.
Those are very good questions.
I appreciate them.
Yes, sir.
Can you just explain cross-range for a moment?
Cross-range.
Realize the Orbiter is a glider so-to-speak.
Not much of a glider but a glider.
And, actually, downrange would be going in this direction.
Cross-range would be out of plane direction, so you could actually maneuver out of plane.
We have a nice globe here.
I am not going to carry it up to the front and I don't have a piece of chalk either.
These are very good questions, by the way.
Thank you very much.
Here is the United State roughly.
This was designed for the military requirements.
They wanted to be able to, for reconnaissance satellites, basically, you want to be in polar orbit because you are going around, let's say this is an orbit here, then the earth turns underneath it, and so you basically fly over all parts of the earth.
That was the basic military requirement.
They wanted to be able to launch out of Vandenberg on the West Coast into a polar orbit.
And, because of security reasons, this sounds a little bit strange when we think back on it, but in a time of crisis, remember we were in the Cold War and everything, you wanted to be able to put a satellite up without necessarily giving the other side a chance to make all the radar measurements on the Shuttle and everything and figure out right away where the satellite is.
And also there might be hostilities.
The Shuttle was a strategic asset.
So basically they wanted the Shuttle to be able to land the very next orbit.
Well, all right, you take off from California, you fly over the pole and you deploy your satellite.
And, by the way, we have never, with all the satellites we have deployed, ever deployed a satellite on the first orbit.
That would be an incredible feat, but that was the requirement.
So you fly over the pole, you come back around.
Now you're ready to land in California.
But, during that time, the earth has turned by a thousand miles, 24,000 mile circumference in 24 hours.
Actually, 1500 miles because an orbit is 90 minutes, 1.5 hours.
Your orbit now would put you right over the Pacific Ocean.
If you just burn your engines, slow down and come down through the atmosphere, that is where you are going to land.
So instead, as you're flying through the atmosphere, you basically have to come down banked on your side.
So, essentially, you're generating a lift vector.
And, instead of turning your lift vector up, you turn your lift vector to the side and that pushes you over.
And delta wings can generate a higher lift vector than the straight wings, and that was the determining factor.
Very good.
[AUDIENCE QUESTION] Well, I think that's right.
Can you address that, Jeff?
My recollection at the time was that even coming in over the Pacific, yes, there were some places where you could land on land but it would mean landing in hostile territory, the Soviet Union or Eastern Europe, and we didn't want to do that.
So the cross-range initially was to be able to make Australia.
Well, I'm sure that's right.
Certainly, there were landing sites that we had all over the world.
It could have been a much smaller cross-range if we were willing [OVERLAPPING VOICES].
And, I will say, continue to remind us if we don't explain some basic things.
Because we've been dealing with this for so long that some of these things just seem like second-nature.
You weren't even born when the Shuttle started flying.
And so the level of backgrounds is going to be very different.
We want to bring everybody along with us.
So if there is something that we say the same thing, you know, the question about hypergolic fuel, if we use a term and you don't understand it, we will try not to use a lot of acronyms.
I cannot guaranty that the speakers won't use acronyms.
Don't be embarrassed to just stop and say what does that mean, what are you talking about?
But, as we go through the course, you are going to get more detail, more detail.
For example, you have a total lecture by a man named Henry Pole who is going to talk about the reaction control system.
He will give you all the details you want to know about hypergolics and storables, more than you ever wanted to know.
So you are going to get more details as you go through.
But don't hesitate to ask questions because it's good for us.
Let me tell you another satellite story.
My wife and I moved to College Station.
My wife was working at a junior college in the registrar's office.
She was working with a young lady and Apollo 13 came out.
Well, Apollo 13 happened to be my first mission when I was Manager of Command and Service Module so I know a little bit about it.
In fact, I am going to give a lecture in Dick Batten's course on the 26th on Apollo 13.
But my wife talked to this young lady and said I went to see Apollo 13 yesterday.
And she said it was so exciting.
I didn't know how it was going to end.
[LAUGHTER] So it is a frame of reference.
Well, we've talked about this.
The size of the payload bay is 60 feet long by 15 feet diameter.
Size of the crew cabin defined to be over 2600 cubic feet.
The payload, as you know, is 65,000 pounds at liftoff, 35 pounds at landing.
And what you need to understand is the Orbiter is a launch vehicle, it's a spacecraft and it's an aircraft.
And when you look at the two different systems, I worked on Apollo, as I said, very much and on the Shuttle, and there is no question that the Shuttle is much more complicated than Apollo.
On the other hand, the Apollo mission is much more complicated than the Shuttle mission.
But this is something that makes it very, very interesting.
This is a very old chart.
And, in fact, this is the original chart.
It comes out yellow.
This was the original cost estimate and how they totaled up the $5.15 billion.
We talked about that.
And that was the very original cost estimate done on a cost analyst chart.
There were no personal computers at the time.
In fact, I found that in my files.
That's a very old yellow chart.
It's hard to believe some of those numbers.
On the other hand, I had a gentleman who worked for me who got his PhD at the University of Colorado and his subject was the cost of the shuttle Orbiter.
And it turns out that if we had those two years of inflation we really would have made that cost on the $5.15 in 1971 dollars.
It only ran over by about 10%.
That's right.
Let's now talk a little bit about configuration.
I don't know if you can see this chart or not.
Now, you might say how did you go from that conception in August of 1972 to something with all these numbers on it?
Now, this is really a case in systems engineering.
And let me explain to you what I mean by that.
Certain assumptions had been made.
One assumption was that in the Orbiter it was going to weigh about 175,000 pounds without payload and you were going to have a 65,000 pound payload.
You had to make that assumption.
You also had to make an assumption that you were going to use a liquid oxygen liquid hydrogen engine.
And why did you select that?
Today I'm not quite sure it was a good decision, but we did.
Why?
Because of specific impulse, the ISP or the performance of the engine was the highest for the liquid oxygen liquid hydrogen chemical compulsion that we know today.
So we selected that.
Well, when you use the equations of motion, you integrate the equations of motion or any other techniques you may want to use, you then show how much propellant you're going to need to get that into orbit.
Then you know that liquid hydrogen has a density of about 4 pounds per cubic foot, very low density.
That means this big external tank is mostly hydrogen.
It's a very, very big volume.
And liquid oxygen is about 70 pounds per cubic foot.
That basically sizes your external tank.
It's a very simplified approach but that systems engineering.
You get much iteration because there is an old adage that the devil is in the details.
So you keep iterating on that with this expert team.
You then say, well, that's still not going to be the most efficient way to get you to orbit.
You say you need really a stage system, so you put the solid rocket boosters on.
You don't have what they call a single-stage-to-orbit, you really have more like a two-staged orbit and you size the solid rocket boosters and determine when they have to come off.
Basically, that is how you go about doing the systems approach using systems engineering to come up with a configuration.
Now, that is a very over-simplified way of doing it, the statement I made, but that's basically how you do it.
And these are some of the dimensions that then come out of the vehicle.
And I think they're in that handout we gave you also.
And it looks like I am going to take off.
I did know that this chart tries to lift off on me.
I don't know what the best way to do it is.
You can find those dimensions, I believe, in that paper that was handed out.
But this shows you all the dimensions of the Space Shuttle system, the solid rocket boosters, the external tank and the Orbiter.
Now, an interesting sideline in this, which I got to thinking about after the Columbia accident where the foam came off.
Of course, the reason why we put foam on was because with liquid oxygen, liquid hydrogen, at those temperatures you are going to have a lot of ice form.
And when ice forms and comes down and hits the Orbiter with the thermal protection system, I am going to talk about that in more detail, you are going to do some damage.
So one solution would be to put foam on the tank to eliminate the ice.
And, of course, you would assume that foam could stay on the tank.
Another solution would have been to not go with the liquid oxygen, liquid hydrogen engine and go with what we call storables.
Of course, they wouldn't have gotten as much performance and you probably couldn't put 65,000 pounds of payload into orbit.
But the point of bringing this to you, those are some of the decisions, as you go into various projects, you need to challenge the requirements because the requirements are really going to decide what kind of system you are going to have.
That's really the point I wanted to make.
Are there any questions?
Yes.
[AUDIENCE QUESTION] That's a good question.
One thought at one time was to put the liquid hydrogen tank in the Orbiter and that would basically isolate it.
You'd put it inside.
Or, put insulation on the inside, as you pointed out.
Those were thought about.
The decision, obviously, was not to do it.
I don't recall the reason why.
But, when J.R. Thomas comes, I will ask him that question.
That is a very good question.
Yes, it was two things.
Could you put some of this propellant inside the tank?
And the other is could you put insulation inside the tank itself?
And that was looked at.
And I guess it was complicated and very costly to do it that way, but it was looked at.
That's a very good question.
By the way, these are very good questions.
One thing, when you look at this external tank, only the very upper part is for oxygen because the density of hydrogen is so low.
I mean all of this part of the tank is for hydrogen.
So, to have put all the hydrogen inside the Orbiter, you would have a very different looking Orbiter.
You wouldn't have been able to do it.
But she really said could you put insulation inside the tank [OVERLAPPING VOICES].
Any other questions?
But that's basically how the system evolved.
Now, I oversimplified it to quite an extent but you have to realize there were a lot of iterations.
One other key thing, though, when you have new systems engineering, you do it in a team.
You usually have different capabilities on the team, arrows, mechanical engineers, electrical engineers, so you have different types of disciplines on the systems engineering team that can help you do that.
And then you have a systems engineer that sort of heads it up.
But we're going to concentrate more on the Orbiter.
And we have one of the speakers who is going to concentrate on the engine and so forth.
That's what the system looked like.
The other thing that is interesting, I think, is what the mission profile essentially looked like.
And this happens to be for STS-5, but this is the basic mission profile.
Now, before I start that, let me say this gentleman right here has flown that profile five times.
And so he's done that five times.
He can talk much more about it than I can.
One time, I will tell the story, I'm just about tired of hearing it, but when I was Director of the Johnson Space Center we had a lot of visitors.
One time I had Mr. James Baker, who at that time was Secretary of the State, and Eduard Shevardnadze from Russia who had the same position in Russia at the time.
And I took them over to Mission Control as a visit and put Mr. Shevardnadze down in the flight controller seat and was going to let him talk to the crew.
Not knowing that Jeff Hoffman was on duty at the time, Edward Shevardnadze spoke in Russian up to the crew.
And, before the interpreter could answer it, down comes this beautiful answer in Russian from Jeff Hoffman.
So that really floored both Mr. Baker and Edward Shevardnadze, and me.
That was an interesting story.
Here gives you a little profile of the Shuttle mission.
You lift off.
You've reached max dynamic pressure in about one minute.
About 3800 feet is where you reach dynamic pressure.
You have the SRB sep in about two minutes.
You remember in the Challenger action, I believe it happened at about 60 seconds.
But in two minutes you get SRB sep. And it lands the SRB by parachutes.
You have MECO, which is main engine cutoff.
The main engine cuts off.
And that, to me, is the biggest issue.
When that main engine has to burn for over eight minutes that is taking all the propellant out of that big tank, the liquid oxygen, liquid hydrogen firing the main engines.
What's the velocity of the main engine cutoff, do you know?
It's within a few hundred feet per second of [OVERLAPPING VOICES].
And then you get external tank separation.
The external tank comes down about 9000 miles downrange and lands in the Indian Ocean.
Then you use the orbital maneuvering system, the OMS engine, the pods on the back of the Orbiter.
You use that for a very short period of time to get into orbit.
Then you have your operations, whatever you're going to do, go out and service the Hubble or whatever.
You de-orbit with the orbital maneuvering system engine.
You have entry interface at 400,000 feet.
And why do you pick 400,000 feet?
Because that's where you start sensing gravity.
You get about 0.05 Gs at 400,000 feet.
And then you reenter and land either at Edwards or Kennedy.
[AUDIENCE QUESTION].
Yes?
[AUDIENCE QUESTION] Why does it turn?
Yes.
You mean liftoff?
Yes.
Well, that's a very good question.
Let's see if I can repeat that very accurately.
The fact is that we used the Apollo launch pad and the ditches for the flame bucket, where the engine goes.
So, in order to get it to the right alttitude, we had to make that maneuver.
You had to make that roll maneuver during liftoff.
Is that what you're talking about, the roll maneuver during liftoff?
Yes.
We had to make that roll maneuver because we weren't in the right orientation.
We had to make the roll maneuver because it wasn't oriented correctly on the pad.
Now, let me answer your question.
The Buron was a Russian vehicle.
Well, Dan Brandenstein who was an astronaut headed the Astronaut Office saw that the Russians made this roll maneuver.
And he asked the Russians why they made that roll maneuver?
I mean they didn't have to do it, he said, because you do.
[LAUGHTER] That's true.
Did you hear that story?
Did I say it correctly?
I think so.
He said because you do.
That is a good question.
There may be one other aspect to it, and that's the question of why does the Shuttle actually fly upside down on the way up?
And I believe the original decision was aerodynamics because the thrust is asymmetric.
You have the external tank, and the Shuttle is sitting on the external tank, so the thrust actually has to be through the center of mass of the whole system.
So it is actually flying not straight but is a little bit screwed.
And, for aerodynamic purposes, I guess they figured there was less stress.
Although, recently they've started partway through the launch doing another roll maneuver.
And I think that's for communications.
I'm not 100% sure when they started doing that again.
I'm not sure either.
But the early part of the launch, when you're riding the solids aerodynamically, you go through max Q you're in much better shape if you're upside down.
Yes?
[AUDIENCE QUESTION] Well, to use a big solid rocket booster, it's not very efficient.
Many people have studied single-stage-to-orbits.
Let me just ask a question.
I know we've got a mixture of aero-astro and TPP, ESD.
Are there people here who have not seen the rocket equation?
Everybody.
Well, OK. That's very important.
Let me just very briefly If you have a rocket and you want to increase the velocity of the rocket, I mean that's a critical thing, by a certain delta V, you can take the mass of the rocket when you start the burn, which we call M sub i, the initial mass, and the final mass.
And, of course, the initial mass equals the final mass plus the mass of the propellant.
And that equals an exponential to the negative power of the velocity decrement, which you're trying to put in, divided by the exhaust velocity of the propellant that's coming out.
Now, this is usually phrased in terms of what we call the specific impulse times gravity because it's done by engineers.
That's right.
So you will hear reference to the specific impulse and the units of that are seconds.
But, if you multiply that by the gravitational acceleration, it will actually give you the exhaust velocity.
Because this is in an exponential it is extremely sensitive to the exhaust velocity.
Whatever delta V you are trying to get out, if you can add a few more seconds to the specific impulse then the actual mass of propellant that you have to burn in order to create that delta V goes down substantially.
Now, for hydrogen/oxygen, the specific impulse is about 450 seconds.
For solid rocket motors, the specific impulse is on the order of about 250, something like that.
That's what I thought it was, about 250.
That's almost a factor of two.
And you put that into an exponent, if you try to get into orbit with purely a solid rocket motor, you need a lot more propellant.
And that means the payload mass you can take into orbit is reduced.
That's why, in addition to using hydrogen and oxygen, and we will have a lecture specifically on the main engine, they got every last ounce of performance out of it by running it.
You'll hear references to running it at 104% abraded thrust, 109% abraded thrust.
And all of these have engineering consequences because, in order to increase the exhaust velocity -- Which, of course, if you do that, that's in the denominator so that's going in the right direction, you have to increase the chamber pressure, the temperature and all of the design problems.
He is going to ask you the question why don't you use liquids all the way?
I knew what your question was.
[AUDIENCE QUESTION] He is asking why don't you use the liquid hydrogen engine all the way rather than using solid rocket boosters?
Well, first of all, why not just have a bigger fuel tank, one big fuel tank and just use the Shuttle engine?
That's what he's saying.
That would be essentially single-stage-to-orbit.
And I'm not going to spend a lot of time with the rocket equation, but the delta V that you need to get into orbit is, well, orbital velocity is just under 8 kilometers per second.
But to that you have to add the gravity loss [OVERLAPPING VOICES].
And so the overall delta V is effectively about 9 kilometers per second.
If you put that into the equation, you actually can get the ratio of the final to the initial masses.
And even for a hydrogen/oxygen engine you can basically almost 95% of the mass sitting on the launch pad has to be propellant.
Which means that all of your structure can only be about 5%.
Now, if we could make super strong, super light structures, like an eggshell, then, in principle, we could get to orbit without staging.
Now, the first thing you learn in Rockets 101 is why you cannot do single-stage-to-orbit.
It is because right now we do not have the technology to do that.
Well, see, many people have studied single-stage-to-orbit.
In fact, NASA tried to build one the X-23 and we failed.
What they say is really what you need is to be as smart or as high-tech as an eggshell, be able to make the structure as thin as an eggshell, hold all the fluid in it and have very low density.
Some day maybe we will do it.
[AUDIENCE QUESTION] Basically, by using the boosters or, in a more traditional rocket, a first stage, once you've exhausted the fuel then you drop all of that structural mass.
And so, for the second stage, this initial mass becomes much lower.
[OVERLAPPING VOICES] The other thing that's interesting is that if you go back to Apollo, during Apollo there were three ways to go to the moon.
One was direct.
Von Braun wanted to build a large vehicle called the Nova where you actually lifted off from the Cape and sent the whole vehicle right to the moon.
The other was to do earth orbital rendezvous.
And the other way was to do Lunar orbital rendezvous like we did.
It turns out that that Nova vehicle had to be so big that that was ruled out, so we wound up with earth orbital rendezvous.
Yes, sir.
How much did the role of geography play in the design considerations?
Because just looking at that, I mean, you have to have the external tank separate at a specific [OVERLAPPING VOICES] talking about launching in Vandenberg.
Originally we thought about launching out of New Mexico.
Actually, when you're doing flight design, depending on whether you're going into a due east launch to go into a 28 degree orbit or a high inclination orbit, the placement of the external tank reentry is a major factor in flight design.
We spent a lot of time on that.
And, in fact, often the trajectory has to be shaped to be not quite efficient as it otherwise might be because you have to control the landing of the external tank.
But your question is a good one.
I did some consulting for a company, after I retired, and we were looking at a commercial launch vehicle.
And we were trying to launch the vehicle out of various places in the United States.
And it's very difficult to get approval because of the concerns they had for types of failures or anything that you're going to do.
Yes, sir.
I'm not sure why I thought this, but I was under the impression that the external tank burned up in the atmosphere.
No.
It didn't hit down.
It breaks up in the atmosphere.
It didn't come down as one big glob.
Some of it burns up, but some of it does get back to the ground.
OK. Is there any talk about a need to clean up all that?
Well, they haven't yet.
I'm not an astronaut.
That's mainly aluminum.
Remember, the liquid hydrogen, liquid oxygen inside evaporate and are non-toxic.
They question that Dr. Young asked about the solid rocket booster environmental problem is a real problem that had to be solved.
The other problem I had, I worked on trying to get the Galileo cleared because it had a radio thermal nuclear generator on it.
To get that cleared going into orbit is a big job.
I worked on that and thought I would never finish it, but we finally got it cleared.
But it was a very tough job.
We had RTGs cleared to be launched.
I don't remember specifically all the noxious chemicals that come out of the solid rocket boosters, but it is pretty nasty stuff.
Yeah, it is.
I remember one of my launched, the families and guests are taken to a launch site viewing area about three miles inland from the launch pad.
And it just so happened that the wind was blowing onshore that day, it was an afternoon launch, and my brother is a real space nut who always liked to watch as the rocket went.
After about seven and a half minutes it just sort of disappears over the horizon.
But, after about five minutes, the solid rocket exhaust was actually approaching the spectator area because of the wind blowing in.
And they made everyone get on the buses and drove them away so that nobody got injured.
And my brother was really annoyed because he didn't want to leave.
If we have time, I would like to talk to you a little bit about the Challenger accident, talk to you about the O ring seal.
I would like to talk to you about that.
I don't want to sweep those things under the rug, but I would like to talk to you about that because I've got my own thoughts on the problem that I would like you to hear.
OK. Any other questions on the mission profile?
Shall we keep going?
Yeah.
Tell you what.
Let's take a two-minute break, stand up and turn around.
An hour is a long enough time to sit for a bit.
I am assuming that everybody is aware that the Russians, at one time, had a space Shuttle program.
And they manufactured a vehicle which looked almost identical.
Not exactly but almost identical.
And it was no accident because they used our plans.
They used our thermal protection system, too.
And they only ended up flying it once.
And they actually flew it unmanned.
And, despite a little bit of nail-biting during the landing, they did recover it successfully.
And then they discovered, just what we had discovered, that it was a lot more expensive to operate than they had anticipated.
And they had a lot less money than we did.
And so it basically never flew again.
They had crews of cosmonauts who had been training to fly the Buran, but there were actually two differences.
The first was that they put the engines on the external tank.
This did two things.
First of all, it did improve the performance.
And also the Russians were turning out engines on assembly lines basically.
They turn out a tremendous amount of rocket engines.
And I guess, from their point of view, and I don't know the details of the performance of their engine, how their main engine is compared to ours, I think they actually had four engines, if I remember.
[AUDIENCE QUESTION] The Orbiter was just a glider, and it was no difference.
It was an unpowered glider.
They did learn one thing from us that when you have a delta wing vehicle you are very, very sensitive to the center of mass of the system.
And it was a problem because the Orbiter, when it flies back through the atmosphere, it hits the top of the atmosphere at mach 25 and then flies all the way down to subsonic.
And the control characteristics and stability characteristics change throughout that flight envelope.
And we will have a lecture specifically on the aerodynamics of the Shuttle, but it turns out that the Shuttle is extremely sensitive to the forward CG.
And I think it was around the mach 3 flight regime, if you're just an inch or two forward of the critical area in the CG, you can lose control.
And so we actually, on many flights, they always do a weight and balance on the Shuttle before a launch, have had to be put lead ballast in the aft engine compartment of the Shuttle just to get the CG far enough back to get mach 3 stability.
I hate to tell you how many tons of lead we've launched into orbit over the course of the Shuttle program because of the CG.
And, if you look, the delta wing profile of Buran is slight different from the Orbiter because I guess they learned the lesson.
And so they were not as sensitive to the CG.
But, of course, once you build an Orbiter you cannot really change it.
And so we were sort of stuck with it.
To add on, one of the other concerns was having the Orbiter with the thrust behind the Orbiter and this large mass in the tank you could get a pogo-type activity.
And so one concept that Max Faget had early in the program was to have a swing engine, actually have the engines fire on the back of the tank and then, when you get ready to separate the tanks, swing the engines back into the Orbiter and bring them back into the Orbiter.
And we threw that out.
That was a little complicated.
You're going to have a discussion on mechanism in mechanical systems.
And, mechanisms in mechanical systems, everybody can design but everybody has a hard time making them work.
When you put up an electrical schematic everybody accepts it.
When you put up a mechanical drawing of a mechanical system everybody has an idea of how it is supposed to work.
Let's move on.
I've got some pictures now just to show you.
It is showing a profile.
And then, of course, you've seen the vehicle when it gets to roll out of the assembly building.
There it is stacked.
And then it's on a crawler.
This crawler basically was used for Apollo, and the crawler takes the vehicle out the pad.
Just one thing so that people notice the difference here.
Is anyone familiar enough with Apollo to be able to say what the fundamental difference is with the crawler mechanism here and the crawler in the pad?
Well, if you look at the picture of a Saturn rocket being rolled on the crawler, the whole launch tower was on the crawler.
And so they rolled the whole thing out.
With the Shuttle it's a little different.
I will be bringing in some pictures at some point to show you some of these details, but they actually cut off the top part of the Saturn launch tower because the shuttle stack isn't quite as tall.
And they added a movable what's called a payload change-out fixture.
And so once the shuttle actually gets on the pad, there are actually railroad tracks here.
Is this a better picture of it?
OK. Yeah.
This whole enclosure rolls over and forms a kind of hermetic seal around the shuttle so that you can open the cargo bay doors which, by the way, cannot support their own weight in one G. So you have to put an external strong back on them.
And, in the meantime, you put the payload that is going to be installed, this payload canister here, that is installed in the change-out room.
And then it is swung over, the doors are open, the payload is installed in the bay.
And then this forms a protective enclosure over the shuttle.
And it is not swung back usually until the day before launch.
See, now this is again another detail in systems engineering.
You go in for that schematic or that little plan form of the Orbiter, and you did iterations of the size and then you had to do iterations of the launch complex, how it was going to put payloads in.
And that's all systems engineering, certain levels of systems engineering.
I mean it's very easy to say, well, we will just put the payload in on the pad.
But the actual development of the mechanisms to do that is extremely complicated to say nothing of which, I mean what kind of amazes me is that it has to be done essentially in a clean-room environment.
You're up there crawling around the pad inside the payload change-out room in white coats, bunny suits and gloves on.
And outside the wind is blowing and it could be raining.
There is sand blowing by.
So the whole thing has to be done in a clean-room environment, but it's on the scale of a naval shipyard.
This is a huge vehicle.
So it's really a challenge.
Of course, that's one of the reasons, in all honesty why the cost per flight has gone up.
The original concept, when Dale Myers was Associate Administrator of Manned Space Flight, the man who spoke to you, I went up to him, seeing I was the Orbiter Project Manager.
And we concluded this will be very simple.
We were going to make very standard payloads.
You went, put the payload in, fixed it in, we were going to launch it, deploy it, come back, fail the computer on the pad, we'd go anyway, wouldn't replace the computer.
Of course, that all changed.
The missions became very complicated and we didn't do that.
That is another important point, though.
You better be sure, when you get to be a project manager or manager, you understand your requirements, understand your customer's need.
But, if the ground rules change, your performance is going to change.
So you need to understand that.
Any other point you want to make, Jeff?
Any other questions?
I've just got some other pictures.
Here is the liftoff, as you see, everything burning.
Have we had a case where we hit a bird on launch?
Yes, I believe we have.
Of course, in the Orbiter windows, one of the things we did, I hate to say this, is fired dead chickens into the window to see if you could break the window in the system.
I think we did an aircraft also.
And we actually did, I remember, one simulation where in the simulator, right at liftoff, the instructor came in and threw a rubber bird in the commander's lap and said you've just been incapacitated by a bird strike.
And so the copilot had to take over flying.
You should realize that Kennedy Space Center is actually a wildlife sanctuary, and so there are a lot of birds.
There's a National Wildlife Sanctuary, park rangers and a lot of eagles.
There are eagles and a lot of turkey buzzards.
There is a landing strip that is right in the middle of the bird sanctuary.
And so often, before the shuttle gets ready to come in, they will send planes to sort of buzz the runway to scare the birds away.
They also had a problem at one point with owls.
No, woodpeckers, excuse me.
Woodpeckers were decided that the external tank insulation was a good place to find bugs.
It sounds funny, but think of what that actually potentially could mean.
They had to go down and had loudspeakers and stuffed owls and everything, which they ended up putting around to scare away the woodpeckers.
These are things, again, which I think the original systems engineering never took into account.
We were worried about F equals MA.
And there is a typical deployment of [OBSCURED], if I'm not mistaken, to get the payload into orbit.
Here is a picture of the crew at their station with their CRTs.
And, of course, then you de-orbit.
And this is, of course, not a real picture.
Do I have that right?
This is right.
And then, of course, landing on the runway.
That's a typical profile that we've gone through in terms of showing.
Now, let me talk a little bit more about systems engineering.
When I took over in August of 1972, we had what we called Space Shuttle Orbiter Management Reviews.
That's North American.
This was one of September 1972.
And I think it might be interesting to look at the agenda in September of 1972 what we were talking about.
And here was the agenda.
I'm not going to go through all the paperwork but here was the agenda.
Basic resizing of the vehicle because of certain problems we had in the iteration.
Cabin configuration and docking location.
The Space Shuttle main engine interface control document.
It turns out interface control documents are probably one of the most systems integration tool because interface control document defines how you're going to put the main engine into the Orbiter.
What is interface?
An interface could be this plug.
If you had a plug in a socket, it would be bad if you couldn't plug that in.
But that's an interface.
When you change your tire, if those bolt hole patterns don't match you'd be very frustrated.
So that's an interface.
Interfaces can be as simple as that or they can be very, very complicated.
And so this was the SSME interface control document, how we actually get the main engine in terms of mechanical propulsion put into the Orbiter.
Integrated asset control, abort requirements.
And now we're talking about just procurement, RCS and OMS procurement.
The rocket motor's procurement package.
Avionics definition, leading edge TPS and emergency risk.
These were some of the problems we started talking about.
Now, these are lower level systems engineering problems.
But they are systems engineering to make the definition of the total launch configuration.
Here is an interesting chart.
This is dollars.
Why this is interesting is you see on this chart Rockwell's proposal.
These are $1 million.
$123 million the first year, but when I became Orbiter Project Manager that was Rockwell's proposal.
The first thing they did to me, they told me I only had $73 million that year, so I had to re-phase the whole program.
I had to rephrase the whole program to fit the funding curve that the Office of Management Budget gave me.
This was a major perturbation.
When you do that and you don't get your money as the proposal, it comes out to be a cost increase.
Right away we went a little bit cost increase to 4.6, but as you don't get the funding requirements you then essentially lose your momentum, you lose your schedule.
And once you lose schedule you start costing.
As Dr. Hoffman mentioned, you have the three-ledged stool, cost, schedule and performance.
And that is a continual tradeoff as you go through a systems engineering program.
And the things you have to look at, the things that cause project managers to lose their job is first you're going to recognize that you have a weight increase in the vehicle.
Whatever you build is going to cause to have your weight increase.
The next thing you're going to have is a schedule slip.
And then, when you put those together, you're going to have a cost increase.
And then you're going to have technical problems.
And those are disastrous cases for a project manager.
Those are reasons to firing somebody.
Luckily I made it.
I made it because of a man that you are going to hear talk, Chris Kraft, because he was my immediate boss and he ran interference for me.
And you're going to hear Dr. Kraft talk.
He's a good man to have on your side running interference for you.
You're going to hear him talk.
He's a great guy.
And he's going to tell you what he thinks.
And he'll tell you what he thinks.
But that's a little bit about cost.
Further systems engineering questions in the Orbiter Management Review is Orbiter maximum wing size.
ATP is our authority to proceed.
Wing size is very important.
You've got to get the aerodynamic model wing load squared away.
We said no major line change after program requirement is reviewed.
Controlled dry weight was 170,000 pounds including margin.
And I don't think we made that, did we?
No, we didn't.
More like 200,000.
No ejection seats on vertical flights.
Top located docking port.
Wing area for landing speeds of 150 knots.
Wing stiffness criteria.
Flutter margins.
Control effectiveness for entry and ferry.
Elevon design concepts.
[OBSCURED] crew cabin.
These were some of the systems engineering problems that had to be resolved before you could start manufacturing.
One of the interesting things, I forgot who was head of the astronaut office at the time.
I don't know if it was Brandon Stein.
I don't know who it was.
At which time?
Well, at about the time of the design of the Shuttle.
About this time.
[AUDIENCE QUESTION] I forgot who I asked, but we were trying to get the cockpit laid out.
It was very important to get the cockpit laid out.
And I wanted the crew to have their inputs on how they wanted the cockpit.
You want the crew to be a part of laying out the cockpit.
I called the head of the Astronaut Office over and said we need to lay out this cockpit and you need to come back to me with a decision of how you want the cockpit.
He said I cannot do it.
I've got 100 astronauts over there and they won't agree.
I said I'll tell you what you do.
You tell them that I will give two weeks and then I'm going to do it.
And that really scared them because I didn't know anything about laying out a cockpit.
And we had the cockpit laid out.
So that's sometimes how you have to use management techniques.
Now we're going to get more into the details of what you're going to hear in some detail.
And I'm going to start if off a little bit.
Here are basically the hardware subsystems.
You're going to hear a technical briefing on the thermal protection system and the structures.
You will see in your syllabus when that is, but you're going to hear a very detailed discussion of the thermal protection system, the structures by Tom Moser who actually did the work.
He was actually what I call my subsystem manager on the Shuttle in the early days of the program.
You are going to hear a technical briefing on the Space Shuttle main engines and a very detailed briefly by J.R. Thompson.
J.R. Thompson, a little background about him, he was Project Manager of the main engine at Marshall Space Flight Center.
Then he became Director of the Marshall Space Flight Center.
Then he was Deputy Administrator at NASA.
Now he is Vice President of Orbital Sciences.
You are going to have a detailed briefing on the hydraulic system, the auxiliary power system, fuel cells, orbital maneuvering system and the reaction control systems.
That is going to be by Henry Pohl.
Guidance, navigation and control is going to be by Phil Hattis from the Draper Laboratories.
Environment control and life support of the crew cabin, that is going to be a man who is currently working at the Johnson Space Center, Walt Guy who actually did this job for the Apollo.
Most of these people, by the way, did the same thing for the Apollo that they did for Shuttle.
Very similar jobs, so you can ask these people about the Apollo program.
Landing and mechanical systems by Al Louviere.
Communications and electrical power we're really not going to brief, although we will talk electrical power when we talk fuel cells, but we're not going to really have a discussion on communications and fuel cells.
And then we're going to have some analytical studies on aerodynamics and aerothermodynamics.
These will probably be detailed technical discussions when we get into aerodynamics and aerothermodynamics.
Again, these people did the same on Apollo as they did on Shuttle.
That is going to be your details of what you're going to see.
What I'd like to do now, with the few minutes we have remaining, is talk to you a little bit and give you a little bit of flavor of some of these subsystems, a little bit more detail than that we've previously given you, but not as much detail as you're going to see.
Let's start off a little bit with the Orbiter and just show you what the Orbiter looks like.
And talk a little bit about some of the subsystems in the Orbiter.
The forward reaction control systems is up in the far nose of the vehicle.
And then you have the reaction control system back in what is called the OMS pod, orbital maneuvering system pod.
You have reaction control systems here and here.
That's reaction control systems.
You have your Orbiter maneuvering system back in these OMS pods.
Of course, you have the main landing gear and the nose landing gear.
The wing, faring and the leading edge, which is the carbon material.
This wing edge is carbon material.
And the side hatch.
And the overhead windows.
And then the side windows and the far windows.
The payload bay doors are made out of graphite epoxy.
They were originally made out of aluminum.
We made it out of graphite epoxy because of the weight growth.
We could say quite a bit of weight if we went from aluminum to graphite epoxy.
That was the largest, probably at that time, composite material used in an airplane.
It was made out of graphite epoxy.
And then the main engines, of course, are in the aft end of the Orbiter.
And then you have the vertical tail stabilizer.
And, of course, you've got your elevons.
And you've got the body flap.
The body flap was actually put on originally to deflect some of the heat from the engine, but it actually turned out to be a control surface also.
It actually turned out to be both the focus of a control surface.
That's sort of the layout.
And you can visualize where you use these systems, these active systems.
The reaction control system is used primarily in orbit getting ready to de-orbit.
The OMS is used to help you get into orbit and get out of orbit or maneuvering you want to do.
The body flaps and the elevons are actually used primarily during entry, a little big during assent, during entry.
We do use the RCS system a little bit during entry.
In the high altitude of entry you use the RCS system, but eventually during entry the load just becomes so high the RCS system or reaction control system does not become very effective.
I guess that's about all I want to say.
Any questions on that?
Yes, sir.
I was wondering why the nose gear was so short.
Well, it actually saves weight making it shorter.
It was easier to put in and it saved weight.
And I think also when you land, by having the nose down you actually have an aerodynamic force pushing the Orbiter down on the runway which increases your stability.
And since you don't have to worry about eventually taking off.
Like Professor Cohen said, if they had wanted the Shuttle to be able to take off horizontally from the runway they would have had to raise the nose.
Yes, sir.
[AUDIENCE QUESTION] Well, when you say under-designed, I mean I would call it a minimum design.
I mean if it were under-designed it would break.
I mean there are landing limits.
As I said, for a planned nominal landing, and you have to realize there are different kinds, there are emergency limits and limits for nominal operations, you can take off with 65,000 pounds.
That means if you have a launch abort and you have to return and land with your payload, you will be landing with 65,000 pounds.
But you don't plan to do that.
That's a one off deal.
If you're planning to do it, it means you can do it over and over again and then the limit is 35,000 pounds.
They don't build it like a Navy carrier landing airplane, I will tell you that.
Not just landing gear, I found the tires are just really small.
The tires are small.
They have no treads either.
And the brakes were small.
Actually, the original brakes were under-designed and we kept having brake failures.
In fact, I took the drag shoots off and then we put them back on.
I took them off because of weight and put them back on.
Tires are very interesting.
We're going to have a very detailed discussion on tires, but tires are very, very complicated.
You think by putting more plies on the tires you make it stronger, but not necessarily because when you cycle them they got hot and the more plies tend to hold the heat in and weakens the tire.
Tires are not a very simple thing to design.
They are only used once then.
No.
I think we used these tires, I think, five times, or could use them up to five times.
The brakes are used five times.
I don't know about the tires.
I don't know.
Let me talk about weight and margin.
Let me go back to Apollo.
We had Apollo designed.
And we found that on the Lunar module taking off from the Lunar surface it was too heavy.
We couldn't lift off.
The engines were already built, the tanks were already sized and we couldn't get off the Lunar surface.
And that's not a very good deal.
The astronauts didn't like that too well.
George Low was the program manager at the time.
We were spending, in those year dollars, $24,000 a pound to get weight out of the Lunar module.
And they scraped, they did everything they could to get any [OVERLAPPING VOICES].
You're basically offering them $24,000 for every pound that they could scrub out because we couldn't resize the engine.
The engine was there.
They were literally in there with emery cloth shaving down the aluminum from the inside.
So weight can be very critical.
I don't think most people realize how close the margins were there.
I mean the Lunar module was only certified for, I think, five pressurizations.
And then it was beyond its structure limits.
My statement to many people is the next time we go to the moon we are going to find out how hard it really was because, to tell you the truth, we went to the moon with one computer in the service module and one computer in the Lunar module.
Excuse me.
We did have two computers in the Lunar module.
We had the MIT computer and then we had a strap-down system built by TRW.
That's an interesting point.
I think somebody asked the question last time.
Let me talk about the aft end of the Orbiter a little bit because we did not have CAD/CAM systems at the time.
And let me show you some pictures taken.
I don't know what vehicle it is, but that's the aft end of the Orbiter.
That's where the big engines go into.
This looks like it is going to take off.
If you crawled in the back of the aft end of the Orbiter there is plumbing, there is wiring, there is structure.
Now, had we had computer-aided design at that time, I am convinced we would have a much neater, better design than we have today.
We had to mark everything up.
We marked it up and we changed it.
People bumped their heads on it.
If you go to the aft end of the Orbiter you'd wonder how they can do anything, but that was done because we did not have computer-aided design.
Like airplanes now have virtual markups.
You can actually build a virtual markup right on your computer.
We didn't have any of that.
And this, again, is just to show you what the crew module looks like in a structural point of view.
That was the pressure vessel.
That's basically a pressure vessel going together.
And there were the windows.
But the thing that's interesting about that, it had very intricate welding techniques.
And these techniques were very good and actually proved to be a very, very satisfactory design in a pressure vessel.
I would like to now talk a little bit about a couple of subsystems that you're going to hear much more detail about.
But now venture into some systems and tell you a little bit about them.
It's probably going to be the thermal protection system and the guidance and navigation system.
I will talk to you a little bit about that.
Here is an old chart that shows, now, unfortunately I use English units, maximum heat rate and BTUs per foot second squared, integrated heat load and radiative equilibrium temperature.
And then you see the Apollo return as number four.
And the Apollo design was about 100 BTUs per foot squared integrated heat load, if I read the chart right, and a heat flux of about 300 BTUs per foot squared second with a radiative equilibrium temperature of about a little over 5000 degrees Fahrenheit.
On Apollo that was our baseline.
We used an ablative head shield.
Now, the ablative heat shield has a density of about 130 to 140 pounds per cubic foot and is not reusable.
On the Shuttle it was about 40,000 BTUs per foot squared integrated heat load and 50 BTUs per foot squared second for heat flux with a radiative equilibrium temperature of about 3000 degrees Fahrenheit.
Now, at that point of time we had to come down from our baseline knowledge of an ablative material to what we could use and make it reusable and light.
And there were several competitors at the time.
One was a metal insulation, Rene-41, I believe, shingle-type material, very thin, very high temperature, and it had pretty low density.
And then, of course, there was the ceramic material or the so-called tiles which was basically a ceramic material.
It is made out of silica.
In fact, interesting enough, it's made out of what we call gopher sand that comes from Minnesota.
Why is it gopher sand?
I'm not quite sure, but it's got a very high pure silica content.
And you make it into a mulch and then you actually bake it and it comes out in these tiles.
And, of course, tiles sometimes are hard to bond to the vehicle.
How many people knew the tiles were hard to stay on the vehicle?
Well, I was the Orbiter Project Manager and my home town is San Antonio, Texas.
We would go back to San Antonio to visit my family.
My little old aunts would say their nephew is in charge of the shuttle program and he was having trouble making the tiles stay on the vehicle.
They couldn't understand that because they had tiles in their kitchen and their bathroom and had no trouble at all having those tiles stay.
So they couldn't understand why I was having so much trouble having the tiles stay on the vehicle.
We had a problem.
What would we use?
There were two competitions on it.
One was the competition from two of the research centers.
The Langley Research Center wanted to go with the metallic version and the Ames Research Center wanted to go with the ceramic version.
And at JSC we're trying to figure out which one to go with, and we decided to go with the ceramic version.
Of course, Tom Moser is going to give us some detail because Tom was the guy really in charge of it.
But just to give you a little background, and he'll go into more details of it, the problem we had with the tiles is that when you bonded the tiles, of course, they were basically a ceramic material.
Forget about this layer called densified for a moment.
When you bonded the tiles to the room temperature vulcanizing material, which is basically a high temperature glue, to the strained isolator pad, you got stress risers set up and the tile became very weak.
It broke really at the bond line of the tile because it became very weak.
The stress risers were such that because it was ceramic it broke at the interface.
We had one very cleaver engineer that saved my day because tiles were falling off and I couldn't get the vehicle built.
He said if we densified below a quarter of an inch of the tiles, and tiles could be anywhere from two inches to four inches thick, and by that I mean put liquid glass inside that tile, the bottom became like really a hard surface, no stress riser, you could bond it like this and basically have the inherent strength of the tile.
It no longer broke at the bond line, it broke at the tile, which had a very high load.
So, after we did that, we really did solve the tile problem to a large extent.
That was the day we really solved the tile problem, but I remember this so distinctly that we were doing all these tests and all these analyses and all of a sudden they called me one day and said Aaron, we're just doing a very simple flat-wise tension test and tiles were just coming off the vehicle.
So that is what saved the day.
I think it's another good example of problems in system engineering if you just leave it there for a moment.
Sure.
We worked so hard figuring out how to make these tiles, and they are extraordinary.
You can take one of these tiles, put one side of it into a kiln, heat it red hot and you can hold the other side of it in your fingers.
I mean that's how good the insulation is.
And I guess everybody just sort of figured we know how to glue things on.
You brought up another very good point that I left out.
The problem is we spent so much time figuring out the efficiency of the tiles to do the thermal protection.
In other words, this is fantastic tile.
It weighs about like balsawood, so it is anywhere from 10 to 12 pounds per cubic foot compared to 130 pounds per cubic foot.
It can take the higher temperature ablative material and be reused.
So it answered all the questions we had about thermal protection.
What we didn't do, though, was figure out how we're going to attach it to the vehicle.
But everybody, I think, just assumed, I mean it's almost like your relatives, we glue tiles in the bathroom all the time.
Anybody can glue stuff onto something else.
But we really did a bad job.
In fact, that's an example I use in my course at A&M.
We come up with a function structure.
What functions does this have to perform?
Well, it doesn't actually have to thermally protect the vehicle.
It has got to stay on.
And we forgot about the staying on part.
We talked about the thermal protection part.
And this thing is so good, its thermal diffusivity is so good that, like Professor Hoffman says, you could get the surface temperature to 2000 degrees Fahrenheit and hold it in the palm of your hand.
And so it turned out it was a very, very good solution but we almost blew the day by not coming up with this design.
Why does it have to be individual tiles rather than big long panels?
Well, because of the coefficient expansion.
Now, that's the point of this strain isolation.
And it's another interface problem which is part of the system's engineering.
The structure of the shuttle is aluminum.
Thermally, it expands and contracts by several inches every time you go from night to day.
I mean the Orbiter, we've done these measurements.
One side of the Orbiter is pointed towards the sun.
The other side is cold.
The Orbiter actually bananas by a couple of degrees.
And they did thermal tests in some of the orbital flight tests where the payload bay, the clearances could change by a couple of inches depending on the thermal conditions and designing mechanisms to latch them closed with those tolerances.
So the orbiter is flexible and expands and contracts as a metal.
The tiles don't.
They are rigid.
And so, if you have too large a tile area, the aluminum is going to bend under it and the tiles will crack.
And even with the small tiles, between the aluminum and the tile there is this strain isolation pad.
This of it as sort of like a felt type material.
It kind of eases the interface between the two.
And, of course, if you look at the tile installation on the vehicle, in between the tiles, remember on the last mission we went EVA to fix the gap fillers between each one of those tiles mainly for problems during liftoff to keep the tiles from vibrating and damaging each other.
And those gap fillers, I'm not concerned about going EVA to fix it.
Well, on the other hand, once you see the problem you cannot let it go so you've got to do something about it.
They went EVA and pulled the gap fillers out, but those are the tile installations.
But your question is a very pertinent one.
Why are there so many?
And that's the reason, because of the coefficient expansion.
Then you put gap fillers in between those tiles to keep it from damaging during liftoff.
It does turn out that one of our biggest concerns is going to be how fragile the tiles were.
And they really turned out to be a pretty good system.
We were concerned about losing a tile and then getting heat and having an unzippering effect where you lose a bunch of tiles.
And we really alleviated that problem.
I will tell you another funny story.
This is probably not so funny but it was a story anyway.
I've forgotten what mission it was but these tiles are actually waterproof.
We actually put Scotchgard on them because the waterproofing is in the tile.
And after a certain mission the waterproofing boils out so we had to waterproof the tile.
STS-4 was the hailstorm.
That was the first one where they recognized the problem.
Well, anyway, Rockwell International was my contractor.
The head engineer at Rockwell was a brilliant guy, but he ran a test where he put a densified tile panel in a bucket of this Scotchgard or this rain repellant material.
And he called me about 11:00 at night.
And I was getting ready to go in because we were [OBSCURED].
We just ran this test and all the tiles came off.
I said what do you want me to do with that information?
We're getting ready to back the de-orbit burn.
I mean they were coming down.
I said what do you want me to do?
I said that was sort of a dumb test, wasn't it?
I said is that what happened on the vehicle?
Did you actually put the tiles in a bucket of water repellent material?
He said no.
I said that's a dumb test.
I took it upon myself, after talking to the man you're going to hear, Tom Moser, what do we do with it?
We didn't tell anybody.
Now, I took a big risk on that.
But what could you do about it?
There was absolutely nothing you could do about it on that test.
So that's an interesting case in discretionary project management.
But you will hear more.
Let me just mention the question of waterproofing because, again, that was something that I think at first slipped through the systems engineering.
And it was the fourth shuttle flight STS-4.
I was down at the Cape for some other activity.
The night before a launch they had rolled back the payload change-out room, the whole shuttle was exposed.
Well, a thunderstorm with hail came through, and it pelted the bottom of the shuttle.
There were pox marks all over it.
And, of course, it was raining.
And people realized that the tiles actually were absorbing material.
The outside of the tile is kind of impervious, but once you crack that the rain can get in.
And they actually brought a tile expert down from the Cape and did some calculations.
It was an early flight so we didn't have a maximum payload luckily because we ended up carrying, they figured, probably a few hundred, many over a thousand pounds of water into orbit.
And there was so much water in the tiles that the orbital dynamicists were able to measure the perturbation of the Shuttle's orbit because of all of the water that was evaporating.
And, of course, it was evaporating from the bottom so that it effectively was asymmetrical and had a propulsive force.
And after that was when they came up with the idea of Scotchgarding.
And, if you look closely at each of the tiles, there's a little hole where every tile before every flight they go through with a hypodermic syringe and just inject a little bit of Scotchgard.
And think how critical it is.
You've got over 30,000 tiles.
If you add one ounce to every tile, you do the math, that's a lot of extra weight.
So you want to put in as little as possible Scotchgard but enough to do the job.
This might be a very good subject for you to decide to redesign.
You might think about what are the technologies, what are the differences, how would you do it differently because their technology has changed.
That's why I brought this one up.
And you will hear a much more detailed discussion by Tom Moser, but you might think about this as one of the subsystems to look at.
I guess that's all for today.
OK. We will see you on Thursday.
Let's get started.
Again, just putting things in perspective, Professor Cohen talked about how in a big engineering design project you work through the various stages.
And the first stage is phase A where you come up with the basic concepts.
And I think the level of detail of the system and subsystem lectures that he is giving, last lecture and today, we could sort of look at as being on kind of the phase A level to show the basic feasibility and the overall structure.
Those of you from aero-astro have certainly been familiar with our approach to systems engineering which we call CDIO.
Someone from aero-astro can tell me the C is conceive, design, implement, which actually means manufacture and test, and operate.
So we're kind of dealing in these first few lectures.
And this could be looked at as phase A and then the detailed design, that's your phase B.
That ends with the critical design review after which, in principle, you're supposed to be ready to cut hardware.
And then you get into the phase CD where you actually build and test.
And then, of course, we operate.
My sort of background, the way I got into engineering, and I probably should have mentioned this at the beginning as sort of truth in advertising is I was never trained as an engineer, unlike Professor Cohen.
I was actually trained as an astrophysicist.
But, when I went to NASA, I spent so much time working with all the technical systems and interacting with the engineers and the people who use them that I learned certainly a lot about the way these systems are designed and particularly operated.
So my approach to a lot of these engineering situations is very much from an operator's point of view.
And I will try to emphasize that as we go along.
Very important, right from the beginning of the design, I think that you think about how you're going to operate the system.
We have had too many examples that I've come across and many people who have to actually operate systems where you build something without really thinking of how the system is going to be maintained and taken care of.
One of the things that we have been doing for the last few years is to take a group of undergraduates down to the Kennedy Space Center every January, an interim activities period, and have them spend a couple of weeks with the engineers and technicians who have to maintain and operate the shuttle system.
And they hear lots of stories from the engineers and particularly from the technicians who say, boy, I would like to have a chance to talk to the person who designed this little system.
And I have to get my hand all the way around.
And it takes five hours to turn the bolt or something more fundamental.
And we will probably discuss this when we talk about the main engines.
The fact that originally the main engines were supposed to be reusable without being taken out of the shuttle so that you could cycle them many times.
Well, it turns out that in order to get sufficient confidence that the engines are ready to fly, we really do have to take them out after every flight.
And they are extensively borescoped when you look inside.
But some of the engineers in the main engine shop pointed out that, for instance, if certain diagnostic test equipment had been built into the engine so that you could have taken data as the engines were shutting down over and above the data that we actually get, possibly we would have been able to reduce considerably the maintenance on those engines.
Again, the shuttle was the first time we had really tried to design reusability into a space vehicle and engines.
And we've learned an awful lot.
So I think it's very important when we discuss the systems in the course of the term that we don't just look at the detailed design but we also consider the operations.
That's very important.
In that spirit, Professor Cohen is going to continue his introduction to the shuttle systems.
And, Aaron, I will turn it over to you.
OK.
Thank you.
As Professor Hoffman said, I'm really giving you what I would call a technical management overview because, when we talk about structures, on the 22nd Mr. Moser is going to come and talk about structures and he's really going to go into the details, how the loads were calculated, how the stressors were calculated, how NASTRAN was used, how we came up with the basic structure.
But I'm just going to really give you sort of an overview on it.
And I think it's interesting to note that the structures may be a very good system to look at for your project because it weighs a lot.
We certainly didn't have all the tools for calculations that you have today to make it a more efficient structure.
The materials are a lot different.
But basically this structure, if you start at the front of it, the crew cabin, which I showed you a little bit about being made, a welded configuration, and the forward fuselage are basically an aluminum skin-stringer structure, very basic aluminum.
If you go back down to the mid-fuselage, which is the large part there, it's, again, a skinned structure.
Interestingly enough, the forward fuselage and the crew cabin where made by Rockwell International in Downey.
The mid-fuselage was made by General Dynamics in San Diego.
So we had various people putting the structure parts together.
The wings were made by Grumman.
So we had this vehicle being built all over the country coming to Palmdale, California for assembly.
And the aft thrust structure was built by Rockwell.
The vertical tail, I believe, was built by Fairchild on Long Island.
Some of these places don't exist anymore.
But then you go back to the vertical tail.
Again, it is machined skins with honeycomb.
The aft fuselage, as you can see, is very complicated.
It has all the plumbing, all the wiring, all the auxiliary power unit, so it is a maze of plumbing and wiring.
You could get lost in there and they'd never find you again.
But it is a maze of wiring and plumbing.
And it's made basically of aluminum.
But a lot of the support structure are boron/aluminum thrust structure panels with graphite epoxy skin panels.
And then the payload bay doors, which was our innovative material going into large composites, we use graphite epoxy, that was the first time that really a large composite structure was used in a vehicle.
Now it's used quite commonly in the Air Force and a lot of places, but graphite epoxy were the panels we used.
And that saved a lot of weight.
One of the problems we had, though, was when we built these panels we found we got moisture trapped in it.
And, of course, the moisture, when you heat it up coming back it would pop the panels off.
So, on the pad, we had to go in and drill little holes in these panels to be sure we could get the moisture to escape.
So those are things you learn when you do new technologies.
But, as I said, Mr. Moser is going to go through a very detailed explanation of this.
This may be something you want to consider because there could be a real innovative way of reducing the weight and making the vehicle more robust, so I suggest you think about that for the structure.
Now I'm going to talk a little bit about guidance, navigation and control.
And some people asked me yesterday where they could find some information on it.
Actually, in those orange or yellow books, I've forgotten what color they are, that we showed you that are on-hold in the library, have a very detailed analysis and discussion of how the guidance, navigation and control system was formulated.
And it is probably as good as you're going to find.
It was very early in the program.
It's a little bit different than it is today, but it's a very good description of the redundant computer set, the fail-operational, fail-safe system.
So I think you could really get a lot out of going through that book.
I should just say it turns out that we have two sets of those books in the library and they are on reserve so you can only use them for two hours at a time.
There really should not be any problem with everybody in the class having access to those books when you need to go and look at them.
That will probably be the best source, except when Phil Hattis from the Draper Labs comes and talks about the guidance system.
I will talk a little bit more about that.
This chart or this schematic or whatever you want to call it was originated by the Johnson Space Center and Rockwell International.
It took us many hours but it basically is the architecture, I am going to go through it in some detail, for the guidance, navigation and control system.
Now, this was the original one.
It is a little bit different today, and I will explain the differences that we have, but this was basically the guidance, navigation and control system main architecture.
And you can see on the left side, and the left side is what I would call sensors.
First of all, in a very simplistic term, some of your experts in guidance, navigation and control, but guidance, navigation and control, navigation is actually determining where you are, guidance is getting to where you want to go and control is controlling the vehicle and its stability and its characteristics around the center of gravity.
So that's basically control.
This is for the total guidance, navigation and control system.
Interestingly enough, the contractor for this was IBM Federal Systems Division.
We used their computer and they did the software.
Much of the hardware was designed and built by other contractors, and Rockwell International basically did the integration.
And they did the integration on a system called the Shuttle Avionics Integration Laboratory.
It was basically a vehicle that was in the laboratory that had all the electronics, the cockpit and everything in it.
It simulated the actuators.
The hydraulic system was simulated and the engines were simulated, but basically all the hardware and software was in the Shuttle Avionics Integration Laboratory.
I imagine you spent some time.
We actually had two.
We had one in Houston and there was one out at Rockwell.
And, when Professor Cohen says that it was a vehicle, you have to appreciate that it was actually laid out just like the shuttle so that they had the controllers, which were in the aft end, like in the engine compartment, were a hundred feet away from the crew cabin and the computers and all the lines, the data and power lines were laid out as closely as possible to physically duplicate the layout in the shuttle because there was concern about the timing of signals going back and forth.
And they wanted to run the simulation as accurately as possible.
And then, of course, you have to have a set of simulation computers to try to determine the environment that the shuttle would be flying in so that it could make the inputs to the rate gyros and the other parts of the measurement units to try to duplicate the flight regime.
And here is an explanation of the alphabet soup of all the systems, subsystems, components.
On this side you see really the sensors.
That's the information that you need to do your navigation part of it.
And then, of course, that information then is sent to the computer through a multiplexer demultiplexer, MDM, which basically is an analog to digital converter, digital to analog converter.
The computer does its computations and then sends it to the effectors which actually change, whether it's the RCS, reaction control system, whether it's the orbital maneuvering system, whether it's the aerosurfaces, the SRB actuators, the solid rocket booster actuators, the main propulsion system actuators.
So you've got sensors, computation and effectors.
Let me talk a little bit about one sensor which probably many of you can relate to, and that's the IMU or the inertial measurement unit.
And Draper Labs is famous for its inertial measurement units.
I worked with Draper Labs or MIT Instrumentation Lab on Apollo, and we had the original initial measurement developed by the MIT Instrumentation Lab in that vehicle.
The initial measurement unit has gyros on it to determine angles and it has accelerometers on it.
And it is aligned with the star tracker.
It's basically a stable platform.
It's aligned with the star tracker to get a reference system in inertial space.
And when you make a maneuver you get acceleration and you send it to the computer.
And you integrate it once, you get velocity and you integrate it twice and you get position.
And it's also used during powered flight and it's used during entry but that then goes to the computer.
So the inertial measurement unit is very critical.
Just to give you an example, here you have three of them and you have four computers.
I'm going to talk about the computers in a minute.
You have three of them.
You have to recognize when we went to the moon on Apollo we had one IMU and one computer in the Command and Service Module, and we had one IMU and one computer in the Lunar Module, except we did have a backup system called a strap-down system.
The primary system was built by Draper Labs, MIT, and the backup system was TRW.
So we had two different contractors.
Then you have rate gyros and accelerometer assemblies primarily for ascent, for the stability of the bending of the vehicle.
Here's the air data transducers that we talked about yesterday for entry.
Then you have the microwave landing system, the tactical navigation, radar altimeter, rendezvous radar.
These are all sensors you use during various phases of a mission, which again go through the MDMs or multiplexer demultiplexers to the computers and then to the effectors which change your position and velocity as need be.
Let me talk about the computers for a moment.
You see four computers.
That was the real test of this system.
We had four computers that were synchronized.
This vehicle, as we talked about yesterday, for aerodynamicists we decided to make it statically unstable.
We saved a lot of weight.
By that I mean a regular airplane, a normal airplane that's stable.
When you get a disturbance it will still come back.
Without any augmentation it will come back to its original position.
With this vehicle, if you get a disturbance it will diverge.
So you had to continually have augmentation.
So you had to have a fail-operational, fail-safe system.
And that's why you have four computers.
It's fail -operationa/ fail-safe You could lose one computer and your fail-operational will use another computer.
That means you have two left.
And you're fail-safe and you come home.
So that's what it is.
Now, the real concern about it is these computers are synchronized.
They essentially communicate with each other 440 times a second.
Now, I'm recalling a lot of this from memory, but 440 times a second.
And they actually vote on each other to simplistic form terminology.
And if one computer is out, it votes it out and another computer takes over.
Basically that's it.
But the concern we had was that in doing this we could have a generic failure and lose all computers or we could have what we call a two on two split.
And these would be sort of diabolical errors which would cause the vehicle to fail.
So we decided, after this was made, to put in a backup flight system, a backup computer, a fifth computer.
Now, it turns out there was an argument there.
Should we have the fifth computer a different computer and different people or should we have a same computer with different people?
And we argued long and hard on how to do it.
And we had a lot of experience with the Draper Labs.
The Draper Labs just did an outstanding job.
The MIT Instrumentation Lab just did an outstanding job on the Apollo vehicle, both on the Command and Service module and Lunar module.
We went to the MIT Instrumentation Lab, Draper Labs, and asked them to actually take the same computer but put it outside the loop.
It's not part of the redundant set.
It's outside the loop.
I didn't show the schematic right because it gets all the same information that the other computers get, but it then can take over if the primary system fails.
Now, Phil Hattis did this work for the Draper Labs, for NASA in this backup system.
He's going to give you a lecture.
He's very good to lecture on this total system.
He can tell you about the total system.
We thought we'd put the backup flight system in just for a short period of time to give us confidence in the primary system, but it turns out that we started putting things in it that now we needed so we could not take it out.
So, it's really part of the major system.
That is a very different part of the system today.
I don't think we ever really had a diabolical problem in flight.
I think we did have one in the Shuttle Avionics Integration Laboratory one time when we had a two on two split.
But when we did the approach and landing test where we had the Orbiter on top of the 747 and we separated the Orbiter and landed, I remember that first flight, I was sitting in the Control Station at Edwards Air Force Base, and at that time I smoked a pipe, and when we separated a big X came across the screen because we lost the first computer.
I guess the shock actually broke a solder joint in the computer.
The power shock from separating actually broke the first computer so we had to go to the second computer.
I bit my pipe in two.
But it did prove that the redundant set did work because it took over and landed.
We had a very successful in-flight test, not planned, but it turned out to be a very successful in-flight test.
Now, when you make the computations, you get the information on position that the computer computes, it sends it to the effectors.
The effectors, during the high part of entry, may be to the reaction control system, the forward RCS system or to the aft, or it could be to the OMS systems if you're trying to make a maneuver.
Or, when you get down lower in the atmosphere, it could be the aerosurface actuators.
And then, during ascent, it's to the solid rocket boosters or the main propulsion system.
So, those are the effectors.
The flight control system is a very interesting system for entry because, as you can envision early in the entry you don't have any aerodynamics so your aerosurfaces aren't of any value to you, you use the reaction control system.
But, as you get farther into the atmosphere, the loads become so high that the reaction control system becomes ineffective and then you have to use the aerosurfaces.
So it's a blended system.
And the system has to know when to handle the aerosurfaces versus the reaction control system.
And, of course, then you also have displays to the crew and actually information that the crew could use to make certain decisions.
So that basically is the guidance, navigation and control system.
There is one additional change, that I don't have on this chart, which they have incorporated the global positioning system in the shuttle.
The GPS is now part of the system.
It took a long time to do it because I remember it wasn't done when I left.
And were you still there when it was implemented?
Yeah, they did put it in.
And GPS you can essentially get position.
If you could get attitude then theoretically you could eliminate the inertial measurement.
And I don't think they do it that way.
But theoretically, the GPS, for you looking at the guidance system change, this again becomes a very interesting system to look at.
Would you do it different today?
For example, the computer that was used, I cannot remember all the characteristics of it, but it's basically the IBM 4 Pi computer, which is a very old computer.
It was probably made before many of you were using computers.
Or, weren't born.
[OVERLAPPING VOICES] which I think was a design from the early `70s.
Yeah, it is the early `70s.
This would be a very neat system to take a look at for your system.
Not only that, you could probably get a lot of support from Draper Labs and that type of thing.
So it would really be a good system to take a look at.
And I'm sure you could make a lot of improvements.
Those improvements could theoretically be used on the new CEV, the crew exploration vehicle.
I think it would land a lot of merit of you very smart people to take a look at the guidance, navigation and control system.
Yes, sir.
In the air data transducers, were there more than just torpedo tubes and static cords?
That's all there was, I believe.
I think that was.
We had a lot of problems with those.
I think it's just very old technology.
Yes.
I was going to ask, how you got position information before GPS.
Position?
Well, I don't know exactly.
But it's got 13 satellites going around.
Well, there were two ways.
I mean the desire for GPS, of course, was to give the shuttle autonomous navigation capability.
The shuttle with the star trackers and the IMUs, it has the autonomous ability to determine its attitude, but it doesn't know where it is because the inertial measurement units drift over time.
And so you need to be able to update the state vectors.
Originally, the only way you could do it was radar tracking from the ground.
You would track it.
You would get a state vector.
You would uplink the state vector and then the shuttle would know where it was.
And then periodically you would have to update it because of IMU drift.
Once we got the tracking and data relay satellite system installed, you do get navigational information by the Doppler navigation and from the signals coming back and forth through TDRSS, but it was generally calculated on the ground.
So it was not really until the advent of GPS that we got autonomous capability.
You need it for things like if you're going to deploy a satellite, for instance, you need to know your position pretty accurately.
Because the satellite is going to take that position and has to do its burn accordingly.
And, obviously, for your reentry burn or for rendezvous it's critical that you know your position.
When you're doing a rendezvous, the shuttle does have a rendezvous radar.
I don't know if we'll deal with that in detail.
Once you get close enough to the object that you're rendezvousing with such that you can get it either with your radar, or they actually use the star trackers to optically track the object, then you can start getting your relative position with respect to the object, Even though you may not know your absolute position.
And so, from that point of view, the shuttle gets a certain degree of autonomy.
Maybe I'll just say one other thing about the computers because people are amazed at how primitive the shuttle computers are.
But, like I say, the original idea and the concepts that we talked about, this was going to be an airplane-like vehicle.
And originally they wanted to use some off-the-shelf hardware.
And, of course, the AP-101s, as they finally used them, were not off the shelf.
But these original computers had 128k of memory.
And the memory they used, I don't know if any of you have read back in the history of computing where they actually had the little magnetic ring cores with the wires going through.
I mean this was really old stuff.
And they did finally replace that with a solid-state memory with a whopping 256k.
And, again, probably none of you have dealt with overlay technology but, back when I was a graduate student and we were using computers to do complex astrophysical calculations, sometimes you would have a program that was too big for the computer memory so you would have the whole program on a tape recorder and you'd segment it into what they would call overlays.
And then you would load it one part at a time and it would do its calculations.
And then you would stop and load the next batch of software.
Well, that's the way we have to run a mission.
The computers cannot hold enough software to do ascent, orbit and entry.
There are three segments of flight software.
You start the mission with the ascent software loaded.
Then, when you get up into orbit, you do what they call a major mode transition where you basically punch a button and everything goes blank.
And you sort of sit there saying I hope this is going to load properly.
And the tape recorder chugs away.
And then it loads the orbit part.
And the same thing when you're getting ready for entry.
Now, the backup system they decided that they didn't want to take that risk.
That if something went wrong in a major way they wanted the whole flight software on the backup system.
And that meant it had to be scrubbed.
The backup system is capable of flying the shuttle and getting it home safely, but there are a lot of capabilities which the main computer system can do which the backup system cannot do just because they're limited to the amount of software.
The one other thing I'll say is putting together this redundant set of four computers because, as Professor Cohen said, the shuttle will not fly without the computers.
I mean it's absolutely flight critical.
Hundreds of times a second, every computer is looking at the data from all the other computers.
And they are all voting.
And so you have this matrix, just to make sure you understand the way the system works.
If computer number one sees a problem with computer number three, it's likely that computer three is also going to see a problem with computer one because they're doing something different.
But if computer number two sees a problem with computer three and computer three sees a problem with computer two and computer four sees a problem with computer three, now you've got a three to one vote.
And so computer three will recognize that it is the problem and it will take itself out of the set.
And that's what happened, well, in that case, the computer actually shut down.
I remember that big X. I bit my pipe in two.
And, at the same time, all of this information has to go back and forth to the backup computer.
Because, if you're ever going to engage the backup computer, it has to be ready to go at a split second's notice.
And there are situations where you can get a two on two split.
When we run through the simulations to learn how to work this, actually learning the ins and outs of how to work the computer system is probably one of the biggest trainings.
The astronauts become more knowledgeable than the designers.
Yes?
If there is this danger of a two to two vote, was it ever thought about to have five computers then?
Well, that's why we put the backup system in.
Oh, the backup system.
Yeah, the backup system.
That was why we put the backup system in.
It did a little bit more, too.
It was programmed by different people.
Even though it used the same computer, it was actually programmed and formulated by different people.
So, that was one way of saying take away any, you might say, systematic errors that were in the redundant set.
In other words, they kept the same software requirements document but the actual coding was done by Draper rather than by IBM.
That was the other problem.
Suppose all four computers are doing the same thing but there was an error in the code which never turned out so that something starts to diverge, what's your protection against that?
There was a big argument in doing that.
The real argument said should you really have this or a different computer?
Should you really just say just do everything differently?
Just have a different computer?
A different everything?
And, of course, there are advantages to that and there are disadvantages to that.
So, we settled on this.
I think there was another question.
Was there a question back there?
Yes?
You said this was the first fly-by-wire system.
I was wondering what competing ideas there were besides having [four computer systems?].
Well, fly-by-wire means when you put an input into the control system, whether you do it manually or whether the computer does it, nevertheless, ultimately your command just goes into the computer.
You put the stick to the left saying I want to do a left bank, all you're doing is telling the computer you want to do a left bank.
And now the computer has to figure out what's my navigation state, where am I if I'm coming down through the atmosphere, what's my altitude, what's the air pressure, what's my mach number?
And then it's programmed with aerodynamic control laws.
This is one of the big challenges.
Because, as I mentioned the other day, you hit the top of the atmosphere at mach 25, you do your initial control with the RCS.
As the dynamic pressure starts to increase, first the ailerons, the roll becomes active at about, I think, ten pounds per square inch, something like that.
You blend the roll control into your aerosurfaces.
At about 20, I think it's pounds per square inch, you can do the pitch control.
But because the shuttle is coming in at a 40 degree angle, the vertical stabilizer is pretty much shadowed.
So for yaw control you actually have to keep using the RCS all the way down to about mach one when the shuttle finally pitches down and the vertical stabilizer now has some effectivity.
The computer calculates all this information and it comes to the effectors, and you do the control laws on that.
And the problem is that the flight control laws, which the computer has to use in order to control the air surfaces in the RCS, are not constant on your way down.
I mean the flight control laws at mach 15 are very different from at mach 5.
And there is part of the flight regime, for instance, where if you want do a left bank, you actually have to start by commanding a right yaw.
And then the cross-coupling terms in the aerodynamics actually causes you to go in the other direction.
It gets very complex.
And the flight control system is continually changing as the entry progresses.
And so the idea that maybe you should have a direct backup link between the pilot's inputs and the aerocontrol surfaces, although it might give you a warm, fuzzy feeling, you could not fly the shuttle directly.
Because to be able to take into account these change in control laws on the way down, you just cannot do it.
In fact, the shuttle is uniquely, I mean, unless the shuttle knows where it is, if the navigation state goes bad the shuttle cannot fly.
You could have the runway in sight, but if the shuttle's inertial nav state is wrong you will lose control and you cannot land.
So it's a very complex system from that point of view.
This is a very, very, if you can envision the systems engineering that went into this system, it takes aeordynamics, it takes flight dynamics, it takes electrical signals, it takes guidance laws, navigation laws, hardware, software.
This is probably the biggest integration job, or systems engineering job.
It requires everything.
It also has to take into consideration aerodynamic heating.
I'm going to show you that in a minute, because you've got to be sure that you don't fly outside the regime that you were designed for aerodynamically.
I think you had a question.
Just real quick, how big are those IMUs?
Because nowadays you can get a solid-state one not quite that big.
Well, I don't recall.
It's a box, sort of like a shoebox.
The Apollo IMU was about this big.
That was built for the Polaris vehicle.
It was one IMU and it was a three gimble platform.
And I remember it was a big issue.
A three gimble platform has a singularity, so you had to maneuver the Apollo vehicle a certain way so you didn't get the singularity in it.
And one of our astronauts, Jim McDivitt, quite a famous astronaut who was a good friend of mine, said he wanted a fourth gimble.
And there was no way we could get an IMU in that day in time with a fourth gimble.
So, what I did was gave him a fourth gimble to carry with him on his flight.
A little gimble system so he had a fourth gimble.
We didn't have the gimble lock problem on the shuttle because it was a four gimble platform.
But I remember we went through that with MIT Instrumentation many times because all you could get at that time, because the IMUs were so big, a fourth gimble was almost impossible.
And McDivitt was a very good astronaut but he wanted the four gimbles.
In fact, he wound up being my boss.
Before we go on, let me just mention one more thing about the backup system.
Sure.
Go ahead.
The backup system actually takes a lot of care and maintenance and there's a lot of money that goes into that.
And this is not a dead issue because in the design of the CEV, NASA is going to have to make a decision.
There were some guidelines which were produced a couple of years ago out of the Johnson Space Center with a lot of astronaut office input about what are the requirements in the future for human space vehicles?
And they put in that you should have a backup computer system because basically that's the way we did the shuttle.
And, actually, that was an afterthought.
And yet now it's being listed as a requirement.
And this is now being questioned for the CEV because it's a huge financial impact.
And so we're going to have to deal with these problems all over again.
It's an interesting thing to think about if some of you want to delve a little bit more deeply.
I think this could lead to a very interesting activity for you.
Yes, sir.
Do backup computers have their own sensor or are they reading the same sensors?
The same sensors.
I just didn't show this very accurately because I looked at the chart, when I had it, I said my God this is the wrong chart.
But this was the original chart before the backup system so I quickly penciled in the backup system so I would remember to talk about it.
But, no, it has the same sensors.
It has all the same information.
But, again, most of those sensors are redundant.
Yeah, they're redundant sensors.
Not all of them but most of them.
Any one of the four computers can read any of these sensors.
It's not like sensor one is dedicated [to this computer?].
The sensor input is put on a data bus, and they have multiple data buses.
Like four data buses in each of the computers can read all four data buses.
I mean there's just a tremendous amount of redundancy built in.
Yes, go ahead.
Did they ever use the star tracker to update your gyros?
The platform?
Yeah, they do that.
There is also a procedure if you totally lose attitude.
Then you have to go right back and manually take a star site and orient.
And that we've only had to do in a simulator.
To look at the failure history of this would be very interesting.
In my tenure, I know we lost one inertial measurement unit.
We lost that one computer during the approach and landing test.
I don't know what other failures we actually had in this system.
The MDMs, this box was made by Sperry at the time in Phoenix, Arizona.
And I've visited that many times because that is probably the most complicated electronics box of this whole system other than the computer.
It's very, very complicated.
Yes, sir.
Computers have developed a lot faster so why do you still think that it will be a major expense now?
I don't think it will be a major expense now.
As I recall, and you're absolutely right, NASA does not put a lot of money into development of this technology because other people have done it.
It's so much more sophisticated than we have now.
We don't need it.
It turns out, though, at one point in time structures was the most expensive component of a space craft.
Then, of course, there was propulsion.
And then there became avionics and software.
And avionics and software for the shuttle is very, very expensive.
It's probably one of the most expensive things.
I think today, in today's environment, with the technology we have in both software and hardware, I think it won't be that expensive anymore.
So, I think you're right.
That's why I'm saying in redoing it, you could use new technology and really show how much less it's going to weigh because the IMUs are probably going to be smaller.
You could reduce the weight, you could reduce the electric power and you could reduce the cost.
But you're going to have another detailed briefing on this.
And Phil Hattis, I know him very well, I think he will do an outstanding job in explaining it to you.
Let me just show you very briefly, we talked a little bit about this yesterday, but here are some of the profiles that the guidance system has to be able to do.
It certainly has to take care of launch.
The abort modes, which are interesting, there is an abort mode that is a return to launch site.
And I guess you would use return to launch site primarily for a main engine failure, I guess.
If the main engine fails during ascent, you can go to a return to launch site.
Or, a total loss of cabin pressurization, something like that.
And I guess that's never been tried.
And I guess that's probably one of the most difficult maneuvers, most biggest fears the astronauts have if they ever have to come back to return to launch site.
But, of course, the guidance system has got to be capable to take care of that.
Then you have abort once around.
Again, when you have main engine cutoff, you have separation and you abort to once around.
Again, I guess that's primarily a main engine problem.
And then you have, of course, an abort to orbit.
And those are the three abort regimes you have.
Yes, sir.
Transatlantic landing?
Transatlantic.
Yes, you have transatlantic abort.
That's right.
You do have transatlantic abort.
Yes, thank you.
That actually drives the launch window, because [if they don't have clear weather overseas they don't launch?]
.
That's right.
So, the guidance system has to take care of all those abort modes.
Again, I'm sure Phil will go through this with you.
But first stage guidance really consists of attitude and throttle schedules as a function of relative velocity.
It's not completely open-loop, it's almost an open-looped system, but you do control the thrust vectors of the engine and the solid rocket booster actuators.
And the key thing, though, where systems engineering comes into play, it has to get maximum performance.
But the angle tack history has got to be shaped to control aerodynamic loads.
Because some of the highest loads you have, and you will find out when Moser talks about it, some of the highest loads you have on the structure is during ascent.
Control of maximum dynamic pressure.
And you have to provide the flight angle at SRB staging to allow recovery of the spent boosters.
These are some of the constraints that you have.
And, again, this comes out to be a real systems engineering problem.
And for entry, entry really becomes a task in itself because one of the basic problems you have to control is thermal control.
The thermal protection system, I should have pointed out on that structural slide that the back face temperature of the tiles have to be at 350 degrees Fahrenheit.
While the surface temperature may be 15 to 2000 degrees Fahrenheit the back face has to be at 350 degrees Fahrenheit because that's the limit of aluminum, so you've got to control the guidance.
The thermal control for guidance is to keep the vehicle within the temperature constraints in the peak heating region.
So, that's all got to be married together so that the tile system you design keeps the back face temperature to within 350 degrees on the aluminum structure.
Then you go into equilibrium glide.
Constant bank angle, we were talking about, is modulated for drag control.
And then the transition guides the vehicle from the high out braking to the lower angle of attack.
And then you basically land.
All this has to be tied together for your guidance, navigation and control system.
I have one more system that I'd like to talk about briefly.
Now, you're going to have another very detailed briefing on this.
This is the hydraulic system.
This ties in together with the other systems we talked about, the flight control system, the guidance system, the Department of Flight Control, because the hydraulic system is three systems.
It's a 3000 pound per square inch system.
And what does it do?
It basically is used during ascent and entry for it to control the thrust vector control of the engines, for the body flap, for the elevons, for the rudder speed brake, the actuators for the main engine, the doors for the external tank separation, main landing gear, nose wheel steering and braking.
So, that's what the hydraulic system does.
If you lose your hydraulic system you had a bad day.
And there are three systems.
I do believe there is one place where there's a single point failure in the hydraulic system.
It's pretty hard to eliminate all single point phase.
I believe that.
When Henry Pohl talks, he's going to be doing a very detailed discussion on the hydraulic system, you might ask him that.
I don't recall, but he'll tell you whether there is a single point failure in the hydraulic system.
And this is a schematic of the hydraulic system.
I am not going to go through the details of it because I'm not sure I could explain it.
But basically this is the hydraulic system that shows how system one, system two and system three is tied into the right outboard elevon, the right inboard elevon and so forth and so on.
The main engines.
The external tank.
And we've never had a loss in the hydraulic system.
We did test the hydraulic system in what we call the flight control hydraulics laboratory at Downey where we had actually all the hydraulic systems tied together with the computer system.
And actually flew the vehicle, what we called this iron bird, with a hydraulic system.
And we did have a failure there one time.
Early in the program, we had a single point failure and lost all the hydraulic fluid.
We had to go back and make a major change to the actuator system, to the hydraulic system.
Yes, sir.
I'm just wondering how the hydraulic system on the shuttle compares to the hydraulic system on an aircraft?
An airplane.
Yeah.
I'm really not sure I can explain it.
I think there's not much difference.
Let me tell you the only difference.
I think airplanes have three hydraulic systems.
The shuttle started out with four hydraulic systems.
We actually started out with four, but it was so heavy and so complicated we decided to go to three.
And I think the airplanes have three hydraulic systems, if I'm not mistaken.
Does anybody know?
I at least know quite a few that have three.
I don't know if all airplanes have three.
The big difference in it is that the hydraulic system in the shuttle is powered by what we call an auxiliary power unit.
And that's what pressurizes the system.
And I think on an airplane it's powered by the engine itself.
But this system is pressurized by what we call an auxiliary power unit.
An auxiliary power unit is a box about this big.
It generates 135 horsepower.
And it's got a ten inch turbine wheel that goes about 10,000 to 20,000 RPM.
And actually it's fueled by hydrazine.
And it essentially pressurizes the system to get you up to 3000 PSI.
Again, if this auxiliary power unit doesn't work, we have three of them, it's a bad day.
And I think we did have some problems with one of the auxiliary power units at one time during ascent.
But that's the major difference, I think, because I think we tried to copy pretty much aircraft designs using the same type of hydraulic fluid and everything else.
So I think we copied their standard in using this, as I recall, except we're pressurized by the auxiliary power unit.
Again, not to belabor the point, you're going to have a very detailed briefing or lecture on the hydraulic system, the auxiliary power unit and the reaction control system, the OMS system by Henry Pohl.
So, you will have more details on this.
Again, the hydraulic system might be another interesting thing to take a look at.
I hope what I'm trying to get across to you is how these systems all fit together.
You cannot do the guidance, navigation and control without the hydraulic system.
You have the aerodynamics and you have the aerothermodynamics.
You've got the structures and everything that has to fit together.
And you can imagine the systems engineering problem associated with trying to put all that together.
Just to remind you of a few other things that you have to deal with.
When you're working with a space system that also makes it a little bit different from designing for the ground.
For instance, you have a fuel tank with the hydrazine inert.
This is a generic problem with any liquid tank.
Once you're in weightlessness, how do you get the liquid to flow out?
This is actually a fairly traditional old-fashioned design where they have a diaphragm in the tank.
And, on one side of the diaphragm, you pressurize it with either helium or nitrogen.
And that pushes the material.
It's basically sort of like squeezing it out of a bag.
The problem is that hydrazine is nasty stuff.
And in the orbiter maneuvering and reaction control systems they decided that, for reusability, they didn't want to use diaphragms.
And so we'll probably learn more about some of the details.
But there's a very elaborate screen mechanism which uses surface tension to collect the material.
And just getting liquids out of a tank into where you want to go is something you have to worry about in space.
The second problem is a thermal problem.
You're not flying through the atmosphere so you don't have air cooling.
As you see, you've got the gear box.
You're generating a lot of heat.
And, actually, in order to cool, they use water spray boilers.
And you essentially, we actually do this to get rid of heat from the orbiter as well before we open the payload bay doors which have radiators.
While those are closed, it has to be done with a water spray boiler.
And so you get rid of all your heat by putting it into a heat exchanger.
And you basically shoot liquid water.
And, of course, in a vacuum the water flash evaporates and that takes the heat away.
And so, although in essence the control system is similar to airplanes, even there, there are a lot of special design features that have to be put in because this is a system that has to work in space as well.
That's a very good point.
The cooling is one of the biggest differences.
And because this system is so complex and heavy and needs a lot of maintenance and uses hydrazine, which is very nasty stuff, there have been, on various occasions, studies of could this system be replaced by an electromechanical system?
And as motors become more powerful and battery and fuel cell systems are more efficient, I think just about two years ago we gave up on the last effort, but it always turned out to be too heavy.
To get an electromechanical system which had enough muscle to move these air surfaces around, it is just beyond what we can do.
But that might be another good system to take a look at today.
It might be another good system.
The other thing.
You see where it has a speed control and safety for the APU controller?
The interesting thing about this turbine wheel, I forget the exact RPM, but it's pretty high.
I think it's at least 10,000 to 20,000 RPM.
If this shaft should break off, it's a very hard to control it in the box.
I mean it will go right through the box it's in and share the vehicle.
We've tried everything we know.
We've put protection around it.
We've put a lot of margin into the turbine wheel, into the shaft, but that's a very critical thing.
And, as Professor Hoffman mentioned, there have been many exercises to replace the auxiliary power unit.
And I think again this would be another good challenge to take a look at seeing what you could do to come up with a different design for the auxiliary power unit.
But there will be other systems that you might want to look at.
But I thought I'd just give you a little discussion of some that I personally think are very pertinent to look at.
That's the thermal protection system, the structures, the guidance, navigation and control, the hydraulic system, including the APU.
And these, I think, would be a very good system.
Now, you might pick others, but these might be very good ones to look at.
Let me wind up my discussion by talking to you about what I think you ought to look for in the system.
This is just a very simplistic chart, but this is my way of thinking about what you do when you go about designing something.
You need to look at the functions.
As we pointed out yesterday, the functions are very important because, when we talked about a thermal protection system , we looked at the functions of protecting the vehicle, maintaining that back surface temperature to 350 degrees Fahrenheit.
We worked that to finite detail.
We knew exactly how thick the tiles had to be, what the characteristics had to be, but what did we?
We forgot, oddly enough, that it had to be attached to the vehicle, which is sort of dumb.
And you would think, my gosh, they ought to be able to do that.
But you're infinitely smarter after you find out your problem.
One thing you ought to do is really understand what functions have to be performed.
And you might say that becomes your functional requirements.
The whole thing is understanding your requirements.
Then you ought to understand what performance is required.
In other words, performance requirements.
You ought to have a first order of magnitude calculation of what kind of performance requirements you need.
A lot of times this is going to have come from assumptions, talking to people, getting experts involved.
But what kind of performance requirements?
It is going to have to be iterated, but what kind of performance are you thinking about in this system?
Whatever the system may be, hydraulic system, thermal protection system, whatever.
Then we talk about the three-legged stool, as Professor Hoffman said.
We talk about schedule, cost and output weight under performance.
Because, as I told you before, the first thing you're going to wind up, in designing a system, is that the weight is going to get too high.
Then you're going to find out that performance goes down.
You have schedule slips.
But the first thing that usually happens is you wake up in the morning and you find out, my gosh, my system weighs a lot more, my subsystem weighs a lot more than they told me I could have.
So, you've got to understand your weight.
And then you need to think about what is the available technology?
What is the technology available?
As somebody pointed out, today the avionics technology is probably pretty high up on the ladder.
And you could probably pick something today right off the shelf.
Although, in my experience of the 30 years I had in the space program, I never was able to find something that was off the shelf.
Space programs just don't usually allow you to take something off the shelf.
People told me to go do it.
And, once I went and did it, we changed everything.
But today, in the technology we have in the avionics, you may be able to get a lot off the shelf.
So, what technology is available?
And then one of the key things, one of the biggest things you have to know is what are the interfaces?
If you could see the multitude of interfaces that are required for the guidance, navigation and control system, it takes into consideration all the interfaces you could think about.
And so you need to think about interfaces, whether they be mechanical, whether they be electrical, whether they be functional.
So, you need to think about what interfaces you're talking about.
And, to me, those are some guidelines you need to think about.
Of course then we talk about cost and schedule.
Usually your cost is going to grow, unfortunately, and usually your schedule is going to slip.
And those are, what I would call, career limiting problems.
That's the best way to get fired, to have your costs go up and you schedule increase.
So, these are some of my guidelines for things you ought to look for when you try to design your system.
I would be happy to answer any questions for you that you may have.
Yes, sir.
About taking technology off the shelf, I'm wondering, and I don't think, from what I know, it was really thought of much during the shuttle, but the idea of maybe putting something on the shelf, so to speak, the idea of designing something, a subsystem or something so it would have the flexibility to be used in future systems.
Do you see where I'm going?
No, I don't think I quite follow what you're saying.
Try again.
Well, you said there wasn't really off the shelf technology available then but maybe there is now.
The idea I'm thinking about is the idea of designing something not so much just for the shuttle in this phase, but so that it could be used for other things.
Well, that's a very possible thing to do and would be a very good thing to do.
In my experience, and we tried something like that, usually this tends to start growing into it.
You start saying, well, what is it really going to cost you to do that?
I know, when I was project manager, people would come to me with something like that.
And I used to be a pretty mean guy.
I'm not quite as mean as I used to be.
But that's the first thing I'd ask them, what about the cost, because that's what happens.
When you start trying to make multiple uses out of something, you usually drive the cost up.
And that's what you have to be careful of.
Yes, sir.
[UNINTELLIGIBLE PHRASE] Well, let me add to that.
I feel very strongly that had it not been for the MIT Instrumentation Lab or the Draper Lab or the people they had there, the technology they had there, we wouldn't have gone to the moon when we said we were going to doing it.
I mean I worked very closely with them, and I feel very strongly that Draper Labs, I keep getting them mixed up, at that time the MIT Instrumentation Lab was really one of the prime movers of the whole system.
It was really fantastic.
So the people working here, and you can take advantage of that, would be very useful for you to do that.
Any other questions?
Well, I will be back on the 22nd.
And I look forward to working with you some more.
One more question.
Sorry.
Yes.
It has not as much to do with the subsystems but to the overall concept of the shuttle.
I was wondering why, if you know at all, the military thought it was important to have the ability to capture a satellite and bring it back?
Was it because satellites were so expensive at the time and maybe they aren't as much?
Because I don't think we've really done that.
Well, it really wasn't the military.
Several satellites we did bring back.
I cannot recall which ones.
Westar and Palapa.
I have pictures of that a little later.
And we retrieved those.
And it turns out, interestingly enough, they were insured by Lloyds of London.
And that was the biggest salvage operation Lloyds of London ever achieved when they returned those satellites, bigger than anything in the ocean they ever picked up.
In fact, they rang the bell.
When they have a big recovery, a big thing in Lloyds of London they ring a bell.
And they came down to serve somebody, whoever he was, I don't recall who he was, but he was head of Lloyds of London.
And we had a big reception at the Johnson Space Center after we retrieved Westar and Palapa.
I don't think the Air Force really had that much demand at the time, or it went away, whatever it was, for retrieving payloads, but commercials did.
What the military was interested in was the possibility of refueling satellites in orbit.
That's right.
We did do that.
Because when you have recognizant satellites, very often you have to change their orbit to get them over to the right place at the right time.
And orbital maneuvering fuel is a limiting commodity on satellites.
And these are very expensive satellites to build.
So if you have the possibility of refueling then, in principle, you could extend the life of these very high value assets.
And we did do a demonstration.
Of course, the fuel, in most cases, is hydrazine which is very nasty stuff.
If you get it on your spacesuit you cannot come inside until you bake it off.
It's fairly dangerous.
And so we actually did do a demonstration just in the shuttle's cargo bay, but we've never actually done a real refueling of a satellite in orbit.
But they're still working on it.
Now I think the military is still working on possibly robotic refuelling technologies.
For whatever the next version of the shuttle or crew vehicle would be, is it necessary or even a good idea to have this huge cargo bay?
Well, I think that's what they're eliminating.
I haven't been that close to it.
But the CEV is going to be really a passenger carrier.
And then any cargo will be on an unmanned vehicle, I believe.
So, that's what they're separating.
Is that right?
I mean I haven't really been that close to it.
This is one of the guidelines now for future space vehicles, is to the maximum extent possible separate humans and cargo.
If the original concept of the shuttle had been realizable that the shuttle would be capable of flying frequently enough that it could basically satisfy all of our launch needs then you can make the argument, well, all right, we're doing this and people are in it to fly it and we'll just do it that way.
But given the fact that we cannot fulfill that goal then you have to ask why put humans at risk just to take a satellite up in the cargo bay and launch it when we can launch satellites using unmanned vehicles?
And, in fact, that was the decision after the Challenger accident, was any payloads that are carried into space on the shuttle in general, I mean there were some exceptions for various reasons, but you ought to be using the shuttle to do things where you need people to do them.
And some of those things, for instance, to service in the Hubble or the assembly of the space station, although even that could be argued that, had we planned it differently, we might have been able to do it without making such extensive use of the shuttle.
But there we do make full use of the big cargo bay.
But future human space vehicles, the CEV will be basically a people carrier with a small amount of cargo.
And any time you want to launch large amounts of cargo you will do it without people.
Let's take a one or two-minute break.
I'm going to get my computer set up.
Let me put this down here.
I want to give you a short presentation just to bring everybody up to a certain level of familiarity with what the shuttle does and what it looks like.
And, again, I tend to look at this from an operational point of view.
This will be informal, you know, ask questions as we go through.
As we've said, the two critical phases of shuttle operations and what makes this such a unique vehicle is it launches vertically like a spaceship with a tremendous amount of power.
And it lands.
You look at this, and you have to remind yourself that it looks like an airplane landing and, yet, a half an hour ago this was a spaceship going in orbit around the earth.
So it really has been a spectacular technological achievement.
Let's go through some of the maintenance operations, what is actually done to the shuttle.
We've talked about it a little bit, but I think if you actually see some of the images it will help to give some reality to some of this.
After the shuttle lands, here it's landing on the runway at the Kennedy Space Center.
And, as we mentioned, all of this is a bird sanctuary so sometimes you have to chase the birds away.
And there's a problem from time to time with crosswinds.
There is only one runway so, if you get wind blowing across the runway at more than 15 knots, you cannot land.
And so that's also a constraint sometimes to launch because when you take off you always have to be able to turn around and come back and land here, just like you have to be able to land over in Europe or Africa.
And so there are a lot of launch constraints.
What they do is, this was actually hooked up back on the runway, but they hook up air conditioning units.
They have to purge.
There's an ammonia boiler.
Remember we talked about getting rid of heat as you're coming in using water spray boilers.
But, once you get below 100,000 feet, the pressure is actually above the triple point of water and you cannot flash evaporate anymore.
And so they switch from a water boiler to an ammonia boiler.
And so, after the shuttle lands, there are ammonia fumes all over the place.
If you've looked at pictures of shuttle servicing on the runway, sometimes if there isn't actually wind blowing they bring around a big fan to blow the ammonia way.
And everybody has to stay upwind of the ammonia.
And there is also the possibility, of course, that there might be a hydrazine leak or who knows.
Anyway, the orbiter has to be safe.
And you have the people come out in what looks like space suits.
Escape suits they call them.
Self-contained, I don't remember the acronym.
But, in any case, we're bringing the orbiter back to the hanger area where it will undergo a lot more than 14 days of servicing.
I remember, again, just to give you a sense of the state of mind before we were actually operating the shuttle, as a new astronaut back in 1978.
Remember the shuttle didn't fly until '81.
We were getting a series of lectures on all the shuttle systems.
And I remember the lecture they gave us on turnaround, which was supposed to take 14 days.
And we got a briefing from the people who were planning the turnaround.
And I remember they told us we've studied this really carefully, and we just don't think it's going to be possible to turn the shuttle around in 14 days.
We've cut out every unnecessary step and we think it's impossible to do it in less than 16 days.
[LAUGHTER] And that's kind of the way people were thinking.
We were talking about it's supposed to make 60 flights a year, and people were skeptical.
There is no way they can make more than 40 flights a year.
People just didn't have a concept of how complex it was going to be to operate this vehicle because it is such a complex vehicle.
To operate it safety is difficult.
Now sometimes it lands in California.
And, in that case, you put it on the 747.
This is the same bipod fitting, just to essentially duplicate the way that the orbiter is put on the external tank, and the tail pod is, of course, for aerodynamics.
And that was also the configuration for the approach and landing tests.
And, actually, later, I think, in the middle of October, Gordon Fullerton who was one of NASA's premier test pilots, he still flies, I mean I'll have to find out from him how many different types of airplanes he's flow, he's just an amazing guy, but he was actually in the shuttle during that first approach and landing test.
So he can tell us about what it was actually like test flying the shuttle, as well as being on the third orbital flight test.
Yes?
Were they carrying anything on the airplane?
You mean inside here?
Yeah.
No, this is not your opportunity for a transcontinental vacation trip.
[LAUGHTER] I mean there are a few people who ride along with it because you need the maintenance people.
They also have an airplane flying about 50 miles in front of this checking for turbulence.
And, in the early days, they had a chase airplane as well.
I told people they were crazy trying to get them to do this because I thought it was the dumbest thing I heard of.
[LAUGHTER] OK. Now a look at some of the details.
Remember we talked about how the engines are taken out?
You get a little bit of a view here inside the engine compartment.
I had the chance, on numerous occasions, to actually go inside the engine compartment.
I mean it's just amazing.
You get these huge big pipes.
And it really helps when you're going to be using a system like this.
We spent a lot of time in the simulators.
And you'll flip a switch, and that switch controls what they call the main fuel shutoff valve.
It's a big butterfly valve about 17 inches in diameter inside this huge pipe.
And you sit in the simulator and you flick the switch and the talk back shows that the thing is closed.
And then you actually go in the compartment and you look and there's this huge pipe and this area.
And you realize this huge thing that's moving around.
And it kind of gives a sense of realism.
And, in fact, that's one of the big safety concerns.
When the fuel is flowing, the butterfly valve is in a vertical position so the fuel flips by it.
Obviously, it's an unstable situation.
If it goes out just a little bit then the fuel flow can slam it closed and you've got an explosion on your hands.
So that was a major consideration.
This is the body flap.
So, you can see all of the plumbing.
And all these red things are removed before flight, stickers.
It's a very, very complex process.
And they pull the engines out after every flight now.
That's the thing, all of the platforms have to be designed so that they fold up and they fold down so that you can get access to all the places.
I mean it's a very complex system.
And a lot of the equipment, the launch platforms and everything, as we mentioned, were adapted from the Apollo program.
But the hangers, which we call the OPF, the orbiter processing facility was built specifically for the shuttle.
The tiles we talked about.
I mean that's just a huge amount of work replacing, maintaining and testing the tiles.
This is inside the cargo bay.
You've got the fuel cells.
You've got hydrogen and oxygen cryogenic tanks.
Helium pressurization tanks.
Nitrogen for your atmospheric system.
This is the bulkhead in front of which on one side you have the engine compartment.
On the other side you have the crew compartment.
And, like we mentioned, this has to be done essentially in clean-room conditions and yet it's on the scale of a battleship.
So, it's a real challenge.
This is the forward window.
On maybe one in every five flights we'll get a little ding on the windshield from a little piece of usually orbital debris.
In fact, more often than not, it turns out to be little paint chips.
Whenever they do that they remove the window, they replace it and do a chemical analysis to see whether it was a micrometeorite or a piece of space debris.
The windows, actually there are three panes.
There's a redundant pressure pane.
The two panes on the inside are capable of holding pressure and the outer is a thermal pane.
This is one of the orbital maneuvering system pods.
These are removable because, again, they contain nitrogen tetroxide and hydrazine.
And we mentioned the fact that these are called hypergolic fuels.
When you bring them together they ignite spontaneously, as opposed to hydrogen and oxygen you need a spark igniter.
So, in that one sense, it's a nice system because when you want to use them in your reaction control and you only want a very tiny tenth of a second pulse, you don't have to worry about an igniter which is a potential failure point.
You just squirt in the hydrazine and the nitrogen tetroxide.
But they're very nasty stuff.
They are extremely toxic and, of course, highly combustible and they are very corrosive.
So, these pods are serviced in a place far away from where the other activities take place for the orbiter so just in case there is a leak or an explosion it's not going to take down the rest of the critical facilities.
This is the orbiter maneuvering engine, and then they also have these smaller reaction control engines.
And there are two aft pods.
And all the fuel tanks with the hydrazine and nitrogen tetroxide are in here.
The aft system, the two pods, they actually have inner connects so that you can actually cross-feed from one system to the other.
And all of that stuff has to be hooked up.
The forward system is independent.
You cannot cross-feed to that.
The landing gears.
We talked a little bit the other day about the tires.
These things sit up in space for two weeks at a time.
You've got to worry about thermal control.
I mean can you imagine if you had a slow leak and you found out that your tire was flat before you started your reentry?
And then, of course, the problem of sealing this against the hot gas.
The tiles on the edge of the doors for the landing gear, that's a very critical section.
Anyway, after typically about a three month turnaround in the hanger they wheel the orbiter, actually, originally it was pulled over on its own wheels, but that required that they then stow the wheels in this big vertical assembly building.
This is the vertical assembly building.
Hopefully you've seen pictures of it.
It's huge.
It was actually built to assemble the Saturn rockets.
It's much taller than we need for the orbiter stack, but it has been adapted.
So, what goes on here is it's wheeled into the big vertical assembly building.
There it is actually on the inside.
You just don't get a sense of the size of this building from any picture that can be taken.
You have to be there and see it to really appreciate it.
In the meantime, you remember the solid rocket boosters are recovered after every flight.
And the Liberty Star and I think the Freedom Star, that's the NASA Navy.
These boats are positioned out offshore.
And they actually have divers who go down and put in plugs and floatation devices so that they actually float the boosters.
And they drag them back where they're disassembled and cleaned up.
The actual segments, which contain the fuel, are shipped back to Utah to Thiokol which is now ATK for cleaning, refilling.
And the other parts, the nozzle and the top, which contains the electronics and the parachutes, those are serviced in Florida.
And then they're brought in segments into the vehicle assembly building where they are stacked.
And remember those are the segment sizes.
Some of these are assembled in the factory.
Those are the so-called factory joints.
And then were these come together that's the so-called field joint, and that's the joint that failed in the Challenger accident.
This gives you a sense of the size of those solid rocket motors in the way the solid fuel is put in.
And they do all of these tests to look for the roundness of the motor and the flatness of the propellant.
Yeah?
Is that the fuel itself?
It is.
Although, what confuses me about this is I'm not sure which segment this is because most of the fuel is actually put in with a star pattern in it and they actually shape the way it's loaded so that you shape the thrust profile.
And I've not been able to get a good explanation of why that is not the case here.
In any case, this is now the process where they lift one segment, they put it down on top of another.
Of course, all of these are hazardous activities.
And nobody is allowed, except for the critical personnel, nobody's allowed in the VAB when they're doing this just in case there is a problem.
And then they actually put all of these bolts in to join up the field joint.
And now, of course, after Challenger we have a new and improved O ring configuration.
Now we have the two solid rocket boosters sitting on the mobile launch platform.
And, again, you can see you have to design all of these platforms so you have access.
Now, the external tank we talked about.
This is in [UNINTELLIGIBLE].
This has actually been damaged, not too heavily, but it did suffer damage from Katrina.
This is the oxygen tank upfront.
Notice how small this is compared to this is the big hydrogen tank.
And then this is actually the front side, but this is then turned around and the two are joined together inside the outer shell.
And, of course, the solids are joined to the external tank, the orbiter is joined to the external tank, so basically the external tank needs a strong back mechanism inside it because that's ultimately what's tying the whole stack together.
And the trust from the solids and the thrust from the main engines, ultimately they're linked together through the external tanks.
So the tanks themselves, the hydrogen and oxygen tanks don't have a lot of beefy structure.
But going through this is a very heavy structure.
Yeah?
The hydrogen tank there is about the diameter of the overall structure?
Yeah.
That gives another idea of the scale.
This is cleaning and polishing inside the big hydrogen tank.
I mean, again, the scale of all of this, it's important to get a sense of what's involved in taking care of these vehicles.
Do you know if they actually had to go in and scrub it by hand?
I guess.
I mean the scrubbing is only part of it.
The inspection is the really critical thing, to see if there is anything that's not quite right.
One thing that's interesting, the use of liquid oxygen, liquid hydrogen is wanted so much because of its higher performance.
But, on the other hand, if you would do something like liquid oxygen and kerosene you would get a much smaller tank or some other propellant.
And there is a tradeoff.
I think today people are realizing, especially from the Russians, the Russians don't really use liquid hydrogen.
You get a lower ISP but your tanks become so much smaller to deal with.
And hydrogen is very hard to deal with.
It's very hard to find the hydrogen level, and hydrogen is very hard to deal with.
My second flight, do you remember?
That was the one with the hydrogen leaks, remember that?
We were supposed to launch in May of 1990.
And we went down.
And before they fill the system they do a helium leak check.
And, of course, helium, for those of you who have worked with vacuum systems, helium will leak through anything.
And, if the system is tight against helium, they figure it is tight.
The problem is that that's done at ambient temperature.
And, when you fill it with the cryogenic hydrogen, everything contracts.
And so things which were vacuum-tight at ambient temperature are not always vacuum-tight.
But the problem is you don't know that until you fill it, which is only done a few hours before a launch.
So, we had gone down to the Cape.
We were in medical quarantine.
And we just got the message about six hours before launch there's a hydrogen leak, launch is scrubbed.
They did some checks.
They drained the tank.
They did another helium leak check.
It was fine.
We had gone back to Houston.
We came back.
The same thing happened.
How long did it take?
Well, in the end, we made six trips over the course of six months.
And we didn't launch until December.
And, actually, another one of the shuttles also had a hydrogen leak.
They actually alternated us.
They pulled us off the launch pad.
They put somebody else on.
I mean what had happened was they had somewhat changed the procedure of installing some O rings in these big hydrogen lines, and it required a process where the workers were actually working in an area where they couldn't see.
And the O rings were being installed slightly wrong.
But it was just terribly difficult to track that down.
So, yeah, there are a lot of problems using cryogenic fuel.
Now, the other thing is if you have a fully staged vehicle with a separate first and second stage, one of the things that you learn in rocket propulsion classes is that in the first stage the specific impulse is not nearly as important.
What's really important is to get a lot of thrust.
In your upper stages having a high ISP becomes much more important in terms of the payload that you can carry.
So, if you had a fully staged vehicle, and, in fact, that's the way Saturn worked.
The first stage of Saturn was kerosene and liquid oxygen and the upper stages were cryogenic with hydrogen and oxygen just for that reason.
OK, let's move on.
Now we have the external tank, again, suspended in the vertical assembly building.
That's lifted up and joined between the two SRBs.
This is the feed line where the oxygen comes down and this is where these two feeds lines, this is the hydrogen.
And then the orbiter has two corresponding feed ports where the hydrogen and oxygen comes into the engine.
And, actually, when you fill the tank on the pad, you actually put the hydrogen and oxygen into the orbiter.
And it flows from the orbiter back then into the external tank.
And it was on this oxygen fitting from the foam that they put around this, was what fell off in the recent Discovery flight where they did get a big piece of foam.
OK, so now we go back.
Remember we brought the orbiter over into the vertical assembly building?
Now, you put this strong back to hold the orbiter.
And this is really spectacular.
I mean you lift the whole orbiter up off the ground and tilt it up.
And you basically hang it and lower it.
And these are big pieces of equipment and yet you need millimeters of accuracy.
So, very skilled crane operators, to say the least.
Now we have the orbiter and the strong back laid down in position and all of the mating is done.
And we're on the pad.
Excuse me, we're on the mobile launch platform and they put the crawler transporter underneath it and then they roll it on these specially prepared pads which have very, I don't know how deep it goes into the ground, but this is heavy river gravel.
Most of the boulders are about the size of your fist, but after one or two trips of the orbiter it's crushed down to the size of pea gravel.
And then they have to replace it with new stuff.
And, of course, as you're going out to the pad, when you're rolling up to the launch pad, you're actually going up at an angle.
And so this whole system, you can see the crawlers down here, each one of these treads, each single piece of the tread weighs a ton.
Just to give you a sense, it goes about one or two miles per hour top speed.
It takes hours to get out there.
And it has to be capable of staying level to within a few degrees, even as you're climbing the ramp up to the launch platform.
What are those things supporting the [UNINTELLIGIBLE]?
The whole stack is basically sitting on the two SRBs.
The skirts of the SRBs is taking the weight.
These are the fuel inputs which are connected through the other side where the hydrogen and the oxygen run into the shuttle.
And, from there, the way I showed you before, it goes into the external tank.
Did they also keep it tilted back?
Did it have the tendency to [UNINTELLIGIBLE]?
Each of the SRBs has four bolts.
Actually, they give us, as a souvenir after a flight, they give us the big nuts that are explosive bolts.
There are eight, four on each of the SRBs.
And, essentially, that's what gives it its stability.
Again, this is a hill.
It's a little hard to see the perspective, but keeping the whole thing steady.
And we talked about how you can see this track here where the payload change-out room eventually could come around and cover the shuttle to give it protection.
This is actually going up the hill to get up.
And it's just a pretty picture that I like.
And here we are.
This is nice because you have both shuttles on the launch pad.
This is a big water tower.
In order to protect the launch pad against flame damage, and also to protect the shuttle against acoustic effects, they actually get a shockwave when you ignite the engines which can bounce back and do some damage.
Shortly, about 15 seconds before T zero, they open up the valves.
And all that water flows with little jets.
If you've seen pictures of a launch, sometimes they show that as part of the launch sequence.
In fact, in another class, I will show you some of the details of an actual launch picture.
That's the water deluge, and that goes for about 30 seconds.
And that cushions the acoustic load reflected back to the shuttle.
OK, that's enough pictures on the pad.
Oh, I know what I wanted to show.
This is called the white room.
And that's how the crew goes in.
You take an elevator up to here.
And then that actually joins up with the hatch.
So the hatch is sitting open.
And you put on your parachutes and the last few pieces of equipment to get in.
And this is a view inside the white room.
This is, I guess, from our last flight in 1996.
And, when you go out on launch day, the big thing that you notice that's different is there are very few people on the pad, only the essential personnel.
And the whole stack is creaking because it's now filled with the cryogens and everything is shrunk and it's sort of alive in a very strange way.
So, we talked about the mission profile.
I won't deal with this.
And I do want to talk about some of the shuttle aborts, but we will do that another time because I want to get finished with the slides.
I did want to show you, after Challenger they introduced a bailout.
We did mention that the basic survivability for the crew requires an intact orbiter.
It used to be that it needed an intact orbiter landing on the ground or ditching in the ocean, which was kind of an unlikely survivability because you're going to hit the ocean at 200 knots and probably break up.
So now there are circumstances where you can lose more than one engine during launch or you might have to do an emergency de-orbit where you basically can be flying along at 40,000 feet stably but with no place to land.
And so that's really the only situation which the bailout system protects you against, but there is a collapsible poll.
The reason for that is because the aerodynamic studies show that if you just jumped out of the open hatch with a parachute the airflow would carry you back on top of the wing and you'd hit the OMS pod, and that would not be a good deal.
So, you actually hook yourself to this escape poll.
And that takes you down below the wing, and then you can open the parachute.
And we actually practiced this where you go out into a swimming pool using the escape poll.
These are some actual tests conducted by Army parachutists in, I think, a 141 Transport.
The system has been tested in flight, although never with the shuttle.
And that's the test where they have a simulated hatch and you basically jump out into the swimming pool.
Once again, into the white room.
You can see the hatch out here.
And that's the last thing that they'll do, is they close the hatch.
Then they pressurize the shuttle by about 1.5 PSI just to make sure that it has pressure integrity before a launch.
And then we launch.
As you can see, you're burning hydrogen and oxygen in the main engines.
What's coming out there is just hot steam, not very visible.
Most of the smoke, the noise and everything come from the solid rocket boosters.
However, you do need these engines.
In fact, people have calculated that if you tried to take off without these main engines pushing up on the shuttle that the attachment between the shuttle and the external tank would probably fail.
Each of the main engines is about, well, it's a half a million pounds of thrust in vacuum.
It's slightly under 400,000 pounds at sea level.
You've got a little over a million pounds coming from the main engines, but these are putting out almost three million pounds a piece.
So, most of your early thrusts in the first two minutes are coming from the solid rocket boosters.
And it's a pretty rough ride.
I mean there's a lot of vibration.
When you go through mach 1, which is max Q, maximum dynamic pressure, the vibrations are even more pronounced.
The first time on my first flight, I really thought the wings were going to come off, there was so much vibration.
But, of course, they don't.
But I'd love to get a hold of it.
I actually saw a high-speed, well, a slow motion picture actually looking at the tail, as you go through max Q.
And you can actually see the tail fluttering back and forth like that.
I mean the aerodynamic loads on the vehicle during ascent are significant.
And you do actually have to move your elevons to what's called load relief in order to take the stress off that.
I really like this.
It gives a sense of the power that you're sitting on top of to get up there.
And that's just a nice picture of riding the fire.
OK, so you get up in orbit and you drop the external tank.
And, although now they're going out of their way to take even better close-ups of the tank to look at the foam shedding.
We've been doing that, in fact, throughout the history.
Now they especially time the launch so that you're guaranteed to drop the external tank with good lighting conditions.
That didn't used to be a constraint to launch but it is now.
You can also, by the way, see how the atmosphere quickly fades out into space.
And that's a telephoto so it actually makes the atmosphere look even thicker than it really looks.
OK, so there is the orbiter in space.
That was actually after it had delivered a payload.
This was taken from the space station.
And just a quick reminder of all the different things that we've used the orbiter for.
Launching satellites, you've seen that picture.
This is a payload assist module to take the satellite from the shuttle orbit up to geosynchronous transfer orbit.
We have used it extensively for satellite repair in orbit.
This is the Intelsat where it was put into orbit by an expendable rocket, but into a bad orbit because of underperformance.
And they managed to get it into a shuttle compatible orbit.
And I won't go through the whole history.
Why there happened to be three people out there is a whole story in itself.
This is one of the satellites.
Again, the satellites were put into orbit by the shuttle.
And the shuttle deployment was fine, but the payload assist module, which you saw before, did not perform properly.
And so the two satellites were stranded in a useless orbit.
We actually brought them back to the ground.
They were refurbished and re-launched again on expendable rockets.
The shuttle has also been used as a space station with the space lab on in the inside to carry out scientific experiments.
This is a large pressurized module which is put into the cargo bay.
And the original idea, as Professor Cohen mentioned, was you could just take your laboratory equipment off the shelf, plug it in here, 120 volts AC power, I guess, and it would be just like working on the ground.
Well, it never was.
But, having said that, I think the space lab program, as a whole, was extremely successful.
And then it's been used to service the Mir Space Station.
We certainly added several years of useful life to the Mir Station because we could carry up a lot more equipment than the Russians could themselves.
And that also gave us an opportunity to get some US astronauts on long duration space missions.
And then, as you all know, we're using the shuttle to construct the International Space Station.
And hopefully we'll get more of it built before too long.
The shuttle is also an excellent platform for performing EVAs, space walks.
And that's a picture of when we went up and repaired the Hubble Space Telescope.
I cannot overestimate the capability that the shuttle gives us as a work platform in orbit.
We can do this sort of complex EVA activity on the Space Station, but once we retire the shuttle we've essentially lost that capability.
To have the manipulator arm and to be able to use people and the robotic arm to move equipment around, it's just a very powerful work platform.
That's one of my favorite pictures floating up there.
This is a pretty picture of firing the orbiter maneuvering engines just to start your descent into the atmosphere.
Despite the fact that you're going at 18,000 miles an hour, you only have to slow down by a few hundred feet per second in order to lower your perigee down to essentially the surface of the earth so that half an orbit later you intersect with the atmosphere.
And, of course, that produces the aerodynamic heating which you can see on the outside.
This is looking out to the front.
That's kind of a dull glow.
It starts out as a deep red and then it gets orange and yellow and finally white on the outside.
The most spectacular thing, you know, you're a meteor.
This is a picture that was taken from Houston.
This is at about 250,000 feet at about mach 12 on the way to a landing at the Kennedy Space Center.
The shuttle is flying like this.
If you look up at the overhead windows, you can actually look back into the wake.
And this is really spectacular because you have these different colors, and it's sort of shimmering around.
And every once in a while, when I would be looking at this, you'd see a big bright light.
And I would think boy, I hope that was nothing important.
[LAUGHTER] What people said it probably was were little bits of gap filler.
You may have heard that on the last flight they discovered that some gap fillers were protruding.
And so Steve Robinson went around and actually pulled some of them out.
But they've probably been doing that the whole time and just have come off.
I mean this little point where you have the convergence of the shockwaves, this is about 10,000 degrees Fahrenheit, that's the surface temperature of the sun.
It's a spectacular visual view.
And then, again, at this point we're just going subsonic flying like a glider and ready for touchdown.
So, I hope that gives you some sense visually of what goes on in the course of a shuttle flight with an emphasis on the maintenance operations.
Without appreciating how complex it is to operate this system, I think it's hard to really make the link between the original concept and the difficulties we've had in getting the shuttle to perform in terms of the turnaround maintainability.
So, again, just wrapping up, that's kind of my farewell picture, the shuttle was an amazingly ambitious concept.
And, I think, what has been astounding is how well the shuttle has been able to perform and do all the things that it was designed to do in terms of the satellite launching and being used as a science platform, performing EVA, repairing satellites, building space stations.
It has given us experience and capability to learn how to do things in earth orbit which we never had before.
And, as I say, we may well miss them once we retire the shuttle.
But where we really did get it wrong, and this will be one of the things that we'll look at when we deal with the individual subsystems, is in the operations.
It turned out to be a lot more complex, expensive and delicate to operate than had been anticipated.
So hopefully there will come a time when we set out to design another reusable vehicle, possibly a reusable winged vehicle.
And I think a lot of what we've learned from the shuttle will be folded into that.
It still is a question of how re-usable the next crew exploration vehicle, the CEV will be.
Re-usability has been put in as a requirement, but that remains to be seen.
And certainly the experience that we've gotten from the shuttle is going to make people look really, really closely at what assumptions we're making about the reusability whenever we do this again.
OK. Next Tuesday will be the last in kind of the conceptual part of what we're doing.
Professor John Logsdon from George Washington University is a very well known space policy analyst.
He did a seminal study on the Apollo program and has also written a lot and done a lot of research on the origins of the shuttle.
And so he'll talk to us then.
Let's see.
I had promised to post everybody's emails on the Web, and I got diverted and didn't get that done.
But I will do that before tomorrow, you know, in terms of forming your teams.
Put together an idea of what system you would like to work at.
If you're not able to form up as part of a team between now and Tuesday, just give me what your own personal preference is.
And then we can look and see if different people are interested in the same system.
We'll let you all know and help you form up teams.
I'll make sure your email is there in the student view.
[UNINTELLIGIBLE PHRASE] If you've already formed a group then just turn in something as a group.
That's fine.
If not, if you haven't hooked up with somebody, just let us know what system you're interested in working at and any ideas you might have of what you're going to look at.
Really, this is just a very, very short write-up essentially to get you started.
And that way, if you have any questions, we can talk about it next Tuesday.
You probably noticed Professor Cohen is a little bit more formal and wears a tie.
And so, when he's here, I also wear a tie.
[LAUGHTER] But today we're a little bit less formal and so no ties.
But I'll probably have one on, on Thursday, again.
Just so that you knew what the code was.
Tom Moser is coming.
You know, that was the generation where everybody wore ties.
So, I'll wear a tie, too.
Anyway, despite the fact that we're a little bit more informal that doesn't imply any intellectual informality.
We are actually, I think, very fortunate that Professor John Logsdon, who is the Director of the Space Policy Institute at George Washington University which is part of their Elliott School of International Affairs -- And we have a recent graduate of that program.
He is going to talk to us today.
And this is really going to be the last in the looks at kind of the policy which led to the original requirements on the Space Shuttle.
And, as we've pointed out on numerous occasions, when you're looking at systems engineering of any scale project -- Well, anything really.
It is absolutely critical to get the requirements straight.
And we cannot really understand a lot of the technical issues with the Space Shuttle and the challenges that we had to face without understanding how it got to be that way.
Professor Logsdon has written numerous articles about the shuttle.
And do you have an electronic version of your science article?
You don't.
We'll have to dig that up.
We'll give you a version of that.
But before that, actually, I guess the work where he really achieved a national recognition was his book on the Apollo program, "The Decision To Go To The Moon," which was a history of the Apollo program.
Professor Logsdon is a recognized expert is space policy.
You will see numerous articles by him in Space News.
And he is often the first person who gets called by the New York Times or National Public Radio or one of the other media for comments on various developments in space.
And actually, depending on how the talk today goes on the shuttle in terms of time, if there is some time left over he's brought some information about the new exploration architecture which was just announced formally yesterday.
And, given that we're setting out on another large space project where a lot of the same issues that we had to deal with about the shuttle will also apply, I think it would be interesting for people in the class to start following what's going on in this new space system.
So, that's enough from me.
John, I'll turn it over to you.
And take it away.
Good morning.
What I'm going to do this morning is somewhat different from what you just announced in the sense that I'm not going to talk about the political history of the shuttle requirements as much as the political history of the shuttle and how the requirements interacted with that political history.
So, it may be the same thing.
But I'm not a technical person.
Although, you have a bachelor's degree in physics which is a well-hidden fact.
Yes.
Next Tuesday is the 100th anniversary of the publication of the equation E equals MC squared.
And my degree in physics is almost before that, not quite.
[LAUGHTER] One of the things that we've been doing at George Washington University, seemingly forever, and close to it, it started in 1990, is a project to collect the seminal documents that define the evolution of the US Space Program.
There are now six volumes of this size printed, we're working on seven and there is one more to follow called "Exploring the Unknown".
Jeff thinks that it's in your library.
I'm going to try to get an electronic version of what I'll just talk about in a moment to put on your class website.
Volume four, which Dr. Hoffman has at his home, not in his office, deals with access to space.
And one section of volume four deals with the shuttle.
And that's what I'm going to try to get an electronic copy of for you.
What I've done is built this talk around the original documents that trace the policy history of the shuttle.
And I'll use them kind of as backdrop.
As NASA approached the end of the Apollo program, its leaders, or at least some of them were thinking about what followed Apollo.
And, at that time, the head of I'll say Manned Spaceflight and apologize for the gender-specific language.
But that indeed was the Office of Manned Spaceflight in the late `60s.
Its head was a very creative character named George Miller who is still active.
He's one of the founders and moving spirits in a thing called Kistler Aerospace that wants to provide alternative commercial access to space.
Miller gave this talk, as you see, August of 1968.
As far as anyone can tell, it's the first use of the term "efficient earth to orbit space transportation system and economical space shuttle".
Miller's concept of the shuttle, which had a lot of influence in one strain of its development, was rather grandiose in character.
Especially the notion that the shuttle would operate in a mode similar to large commercial air transports and work in and out of major airports.
Landing would be completely automated with prime dependence on spacecraft guidance with ground control backup.
And this is a long talk.
It's in the book I just mentioned.
Then its basic design could be applied to point-to-point transport.
If the Space Shuttle were used as a global transport, safety and comfort standards could be comparable to those of a large transport jet.
It was probably not like business class in a 747.
[LAUGHTER] Maybe like the Concorde.
But that's certainly what they were talking about with the National Aerospace plane.
I mean if you can develop a plane that can take off from a runway and fly into orbit.
And you remember, when we talked about the rocket equation, we tried to make it clear why that is such a difficult thing to do.
But, if you could do it, then you don't have to go all the way to orbit.
You can just go halfway to orbit and land in Tokyo after you take off from London or New York.
Right.
And this kind of holy grail reduction in cost by two orders of magnitude, that was the mental set of the guy who I would call at least the policy father of the Space Shuttle, is that you could have an aircraft-like operations, two orders of magnitude of a level of safety and reliability and operability that it could even be used for commercial transport.
As you know, NASA was successful in carrying out its mission of getting humans to the moon in July of 1969.
When Nixon came into office in January of 1969, he had a transition taskforce on space.
And that taskforce told him that there was a need for some decisions on what to do after Apollo.
With the focus on getting Apollo done, the then head of NASA Jim Webb didn't like long-range planning.
He wanted the politicians to tell NASA what to do, rather than the other way around.
NASA was woefully unprepared for what it wanted to do after Apollo.
And the country hadn't discussed it at all.
Nixon appointed a so-called space task force, Space Task Group and asked it for definitive recommendations on the post-Apollo space program.
That task group was chaired by the vice president who traditionally has had the space portfolio in most administrations.
At that time was a well-known space expert named Spiro Agnew.
You're all too young for that to even be a joke.
He was later caught taking bribes in the White House and resigned in shame.
He was a typical Maryland politician, which means corrupt.
[LAUGHTER] That's my home state.
The Space Task Group was captured by NASA, by its then administrator Tom Paine who was a very bravado character.
He told NASA that they should be swashbuckling.
And by Miller, who had developed a long-range plan for NASA.
And, ultimately, by Wernher von Braun who was brought up to Washington to add the charisma to the plan.
The report that was submitted by the Space Task Group to the White House two months after Apollo 11 had these recommendations -- -- for what NASA should do.
This was what NASA really wanted to do, mars starting in 1981, a hundred man space base in the mid '80s.
The program that was recommended ultimately was Program 2 which had mars in '86.
And you see these comparative accomplishments.
And in here was an earth-to- orbit space shuttle for some time between '75 and '77.
That's where the shuttle entered into national policy, was in Nixon's reaction, or in the country's reaction to NASA's post-Apollo proposal.
Again, just to give you a sense of this kind of thinking at that point, I'm not sure where this came from, to be frank.
But you take the station, the space based and other stuff in this maximum program and you're talking total space shuttle flights a year peaking in '83 with 66 flights a year, including 34 to service a six-man base and a six-man orbiting station at the moon.
So, these truly grandiose ideas of what might be done.
I don't think it was ever clear why you had to go to the moon every other week in order to maintain a six-man base either.
[LAUGHTER] No, I think it was not every other week.
Thirty flights a year.
I think it was a three month rotation of a crew.
But they've got 30, 40 flights a year.
Maybe logistics flights, I don't know.
Logistics.
And you're doing two things.
You've got a six person station in orbit around the moon.
Why?
I don't know.
And a six person base.
The architecture, it was announced yesterday, culminates in the buildup of a four or more person lunar base with a passing mention that, oh yeah, we might go to mars.
By the way, if anybody has comments or questions, interrupt me.
Otherwise, I'll just drone on.
Yeah, Larry.
To what extent was the Space Station's existence importance for the shuttle back in the `70s?
It was the reason for its existence.
At this point, the reason to have a space shuttle was to take crew and supplies to the station, period, at least in the core NASA planner's ideas.
You had people like Miller who left the agency in September of '69 with these very grandiose ideas.
He was succeeded by Dale Myers, who I understand has already talked to you.
And Myers was very instrumental in the negotiations that led ultimately to the decision to go forward with the station.
I remember the first time I heard the word space shuttle.
As Jeff said, I've been at this a long time.
I finished the book "The Decision To Go To The Moon," published by MIT Press, but out of print, in late 1968.
We tried to market it as a paperback.
We were going to put a rocket and a girl on the cover, and the inside story of why Kennedy sent us to the moon.
It didn't work.
It had footnotes and all because it was a PhD dissertation.
[LAUGHTER] Talking to an audience like this, they understand what a PhD dissertation looks like.
I went down to attend my first launch, which was Apollo 11, and for some reason hadn't rented a car so I was figuring on hitching a ride from Orlando to Coco Beach.
And the person that I ended up driving to was a man named Leroy Day.
Did you know Roy Day?
Roy was, at that time, running the Phase A studies of the Space Shuttle.
And he told me what he was doing.
It was the first time I had heard of the concept.
See, I don't know where this came from but it's the kind of thinking of the need for this.
So, to go to your point, Larry.
When NASA first presented its post-Apollo plans to the Congress in the spring of 1970, the program was called Station Shuttle.
And they were coupled at the hip.
And so it was the integral justification from the NASA side.
Although, people like Dale Myers were already negotiating with their counterparts in the Department of Defense for potential military use of the system.
Although, almost clearly this is before any military involvement or really commercial involvement came in because, when you look, there are only two unmanned satellites per year.
This is the manifest to implement this.
And that was purely as NASA program.
There was not much consideration of other users of the shuttle at this point because this was enough to justify the investment in a new vehicle.
Again, the logic was that you could put these big things in space like space stations but the logistics costs would drive you crazy if you were doing it with expendables.
And so the only way to operate a permanent outpost in orbit or beyond was to have reusability in the supply system, in the transportation system.
That was, in essence, the number one requirement.
Unfortunately, well, I don't know whether it was unfortunate but in reality the Nixon administration was not having any of this.
Nixon made a statement in response to the Space Task Force report, as you'll see, March of 1970.
So, it took him six months to respond to the report.
Meanwhile, NASA's budget was getting chopped to pieces.
But it was essentially a fundamental 180 degree change in policy from Apollo.
Now, Apollo was separate leaps requiring a massive concentration of energy.
Space must take their proper place within a rigorous system of national priorities, must be planned in conjunction with all the other undertakings.
In other words, space has to be compared in its priority to all the other demands on the federal budget.
And at least for the Nixon administration, but in reality for every administration since, the answer has been essentially the same.
When Kennedy made his speech saying we should go to the moon in 1961, the NASA budget jumped 89% the first year, 101% the second year, 38% the third year.
And it's like a rollercoaster that gets to the top of the first hill.
And the program has been living on that momentum every since.
And you came down that hill very quickly.
You see by '73 or '74, this value was percent of the federal budget, so it's kind of a constant measure.
As the budget goes up, NASA gets essentially the same share of the federal budget.
About seven-tenths or eight-tenths of one percent.
And has gotten that share for 35 years.
And this, I would say, is the way the democratic political system makes policy choice, is through budget allocations.
And if you have the same budget allocation essentially for 35 years, I would say that's where space ranks in the scheme of national priorities according to the political leadership in the White House and Congress.
Just to flip ahead to 2004, one of the fundamental premises in the Bush Vision for Space Exploration is that NASA will stay at this level of expenditures.
And that everything you want to do, going back to the moon, eventually to mars, has to be within that budget envelope, which means you have to design to that.
And remember the fundamental systems engineering triad we talked about on several occasions, cost, schedule, performance.
That clearly demonstrates cost is a fixed parameter.
We don't have the freedom.
Either for the shuttle or in this future program, cost is going to go up by very much.
Just in case anybody asks what these two blips are, this one is the replacement of Challenger after the 1986 shuttle accident.
It's a one-time cost of building another orbiter.
And this was Bush 41, you may remember, or no, who announced a space exploration initiative on the 20th anniversary of Apollo.
And he provided an increase in budget resources to carry out that initiative which, when Bill Clinton was elected, quickly got undone.
And you see the result in the past few years.
In a sense, that decision that space had to be planned in the context of all other priorities has had multiple impacts over 35 years.
The first thing is NASA has never accepted it and has always tried to do more than it has resources.
And one of the things Jeff didn't say was that I was a member of the Columbia Accident Investigation Board after the last accident.
One of the things we said in our report was that NASA had, for too many years, been trying to do too much with too little.
And it created the kinds of stresses in the organization that led to some of the organizational sloppiness that was at least a contributing factor in the Columbia accident.
It told NASA that it could not pursue in the `70s a post-Apollo program that was anywhere near its ambitions.
And so NASA had to reinvent its program from what it had proposed in 1969.
And, by the end of 1970, this is how budgets get done.
This is a letter from the then head of NASA, Jim Fletcher, transmitting NASA's recommendations for the next year's budget.
This happened last Monday, September the 12th this year, NASA submitted its formal budget proposal to the White House.
Every year this starts the process.
Well, you can read all this rhetoric later.
NASA had decided that the key element in the program for the `70s was not the Space Station by now but the Space Shuttle.
It supports the last four of the presidents' six objectives, these four.
And reflecting that decision, NASA announced, "We have made a major decision to defer development of a space station to a later time and to orient the space station studies towards modular systems that can be launched, as well as serviced by the space shuttle."
Again, a fundamental change in plans.
The station that NASA was planning in 1969 would have been launched by the Saturn 5.
Would have been 33 feet across, have lots of habitable space, be big, a 12 person minimum building up to 50 person, maybe eventually 100 person outpost.
This represented a major shift that said, number one, the shuttle becomes our number one priority, not the station, and the shuttle has to be designed to launch space station modules.
That was the overriding NASA goal.
And so I would argue or suggest that this decision made in late 1970 only separated in time shuttle and station that the intimate link between the two programs remained.
It was just going to do them in sequence, rather than at the same time.
And here we are 35 years later.
And the major issue in getting started on exploration remains, what do you do with the shuttle, what do you do with the station?
They are now seen as mortgages that have to be paid or obstacles to the next systems or however you want to characterize.
What this also meant is that the traffic model that was justifying the shuttle of all these launches to space stations and lunar bases that you saw was no longer operative.
And so beginning at the start of 1970 and all the way through this two-year complex decision process, the Office of Management and Budget kept saying well, how do you justify this investment?
You're talking about a multi-billion dollar investment in the future.
What is the justification for it?
This was the first time, in the early '70s, that the White House, through its Office of Management and Budget, used cost-effectiveness analysis, cost-benefit analysis as a tool in budget allocations.
It had not been done, certainly not been done in the space program of the `60s.
But OMB insisted that NASA show an economic justification for this investment.
And in order to make it come out the way OMB wanted it to come out, which was that there was no justification, how much economics do any of you get in this environment?
I have never had an economics course, except at Jesuit undergraduate college called Christian economics, which may be a contradiction.
Never mind.
[LAUGHTER] But I'm going to say something, I don't have a clue of what it means, which is that OMB insisted that NASA use a 10% discount rate, which is the future value of current money.
And that's much higher than the discount rate applied to many other investments, because this was a long-term and risky investment.
And so that meant the economic justification for the shuttle had to be very strong.
And, throughout this process, there was this constant pressure on one hand to justify the shuttle economically.
The only way that could be done, absent a space station or an ambitious NASA program, was finding other users.
And this goes back to your comment earlier.
NASA became not just a kind of suitor of the military as a user of the shuttle, but the economic justification for going ahead with the shuttle became totally dependent on the military willingness to use the vehicle.
And military is a euphemism.
Many of the payloads that were being discussed there were intelligence payloads operated by the organization called the National Reconnaissance Office which, at that time, the existence of the National Reconnaissance Office itself was classified.
So, you could not say NRO satellites.
You can say it now.
NRO's existence was declassified in 1992.
But, at that point, was all called Air Force or DOD satellite, many of which, including the most demanding were intelligence satellites.
The primary determinant of the size of the shuttle's payload bay, the width was the ability to launch space station modules.
Professor Young may be able to comment.
If I understand it right, the kind of human factor studies at the time said that people would be unwilling to live in tubes less than 14 feet across for long durations.
And so the shuttle had to be able to accommodate a 14 foot wide module.
The length could be adjusted.
But the military payloads, I think Hubble pointed down rather than pointed up.
I think there's been enough discussion of it that I'm not revealing classified material.
The reconnaissance equivalent of Hubble was the next generation reconnaissance satellite.
And that was basically 55 feet long.
And so the decision was that you needed a payload bay 60 feet long in order to capture many military and reconnaissance payloads.
And that was a determinant of the size of the payload bay, which again drove the size of the shuttle.
The other military requirement was the desire, well, there were two.
One was a desire to be able to go into polar orbit which meant a west coast launch site.
You cannot launch into polar orbit from Cape Canaveral Air Force Station or Kennedy Space Center without flying over Boston.
Actually, I guess you'd launch south flying over Miami and Cuba which, for range safety, is not a great idea.
If you launch from Vandenberg Air Force Base out in California, you've got several thousands of miles of open ocean in front of you.
The Air Force was in a nice position here because it could make up any requirements it wanted.
We've actually talked about the cross-range.
OK, you've talked about it.
That's where the cross-range came from.
And so you've talked about cross-range leading to delta wings, leading to heavier orbiter because of more thermal protection, but all of that came from the requirement of getting the Department of Defense to say they would use the shuttle as a way of justifying to the economists the large upfront investment.
Have you looked at anything like this?
This is kind of from the outcome of the Phase A studies in the last `60s and early `70s.
As you see, Phase B proposals, a bunch of studies.
And then, in June of 1971, a rapid shift so that in six months the configuration evolved to what was finally built.
And I presume if you've talked about cross-range and that sort of thing you've talked about the difference between the preferred shuttle of Johnson Space Center and its chief designer Max Faget, which was a straight wing minimal cross-range shuttle, probably technically simpler to build and less expensive to build into a delta wing configuration that matched the Air Force cross-range requirement.
What happened in June of 1971 was critical to this whole process.
At this point, in its studies, NASA had concluded to build a two-stage fully reusable shuttle that would match the cross-range requirement and the big enough to launch space station modules it would cost in the order of $10 billion to $14 billion in investment cost with a peaks funding of $2 billion a year during the `70s.
OMB, in May of 1971, said that's fine, but you can only have $5 billion with a peak spending of $1 billion a year.
If you want a shuttle at all, it has to fit within that budget curve.
And I presume Aaron and others are going to talk about the kind of hectic trades that got from a fully reusable shuttle to first moving the liquid hydrogen tanks outside the orbiter air frame and throwing them away.
Then coming up with the idea that you could put both the external oxygen and hydrogen fuel tanks on the outside and throw them away to the notion that you could use strap-on solids to assist in takeoff and move the orbiter down to the bottom so its engines could be used as part of the take-off thrust to the final configuration.
At that point, June to December 1971, there were not zillions but hundreds of different variations of shuttle design being floated around and other designs to do something that was approximating but not totally -- What's the right word I want to say?
Totally meeting all of the payload requirements that had been laid out.
I'm sure you're going to be talking a lot about the engineering choices that were involved in this.
And I'm not capable of talking about them.
But as apprentice young system engineers, the notion that you could go from here, totally different concepts to here in six months and know what you're doing should make you a little nervous.
Why was the shuttle ultimately approved?
OMB, the Office of Management and Budget was on one spectrum of the participants in this debate.
It really didn't believe, its staff, in the value of human spaceflight.
Its staff was, and is, the guardian of the federal budget, believed it was under the policy guidance of the Nixon administration to cut federal expenditures dramatically across the board.
And so OMB, through this whole process, through a variety of interventions and changing demands on NASA and political interventions, getting leaked information from the aerospace industry and asking NASA nasty questions that it didn't want to answer -- The career staff of OMB, they say, in retrospect, went too far in trying to kill the shuttle.
So they were at one end of the spectrum.
NASA was at the other end, obviously, because by now, 1971, the shuttle was a survival project for NASA as it viewed itself as a large organization built around human spaceflight and developing new large scale systems.
Yeah, Larry?
[AUDIENCE QUESTION] OMB, not Congress, is what I'm talking about.
Well, ask your question.
Well, the fact that non-elected
staffers, I was thinking of congressional staffers?
Yeah, but OMB is the same thing
Have enormous influence over not only implementing but making policy.
And they stay long after their term [UNINTELLIGIBLE]
Any of you heard of Paul Shawcross?
I wouldn't think so.
Paul is an MIT graduate.
He's the Examiner for Human Spaceflight in OMB right now.
He did a TPP masters up here ten years ago or so.
And he is leading the fight to ground the shuttle.
Now, nobody knows his name unless you're inside the beltway.
One of the things I'll say, Larry, in reaction to where you were going is at least the career staff on the Hill are relatively accessible.
So, if you're an aerospace industry operative, you know who they are and you can talk to them.
Particularly back in this period 35 years ago, the OMB staff operated under a cloak of anonymity, weren't open to talking to industry people.
It's changed a lot over the years.
And were able to operate behind a wall of secrecy and push their agenda into national policy.
It is my belief, after starting my 40th year in Washington.
God, that's a long time.
Most people outside of Washington think that Congress matters, but almost all the decisions that matter are made in the Executive Branch and Congress just snips at the margins, 2% or 3%.
Yes, sir.
How much did industry lobbying affect the design of the shuttle?
I mean you look at it and every aerospace company had a piece of the shuttle, you know, they were getting money from it.
I assume they also were probably lobbying their senators for places like [OVERLAPPING VOICES].
I mean how much did industry [OVERLAPPING VOICES]?
Well, if you look at this, all of industry had study contracts.
This is Grumman, which was a separate company at the time that had built the Lunar Lander.
And Boeing, before it bought Rockwell, this was North American Rockwell that had built the Apollo Command Module.
This was McDonnell Douglas.
So, the major aerospace companies each had a concept.
And they were lobbying or contending for the adoption of their concept rather than what you see now which is work shares of a single concept.
But this was a decision totally inside the Executive Branch at this point.
Congress was more or less supportive with the exception of one senator, Fritz Mondale, Walter Mondale who kept asking some difficult questions.
And NASA had its preferred concept.
MSC is Manned Space Craft Center, what's now Johnson Space Center.
And it was Grumman and McDonnell Douglas that came up with the idea of putting rockets on the side, at that point they weren't necessarily solid rockets, to enable a cheaper configuration.
What you ended up with was the preferred orbiter of the Manned Space Craft Center and the NASA Orbiter after all these design requirements.
With the Grumman McDonnell Douglas concept of an expendable external tank and recoverable strap-ons.
I don't want to say solids.
What came out of this was an amalgamation of everybody's ideas.
And I said in passing, I will say again, one of these industry firms, and all evidence points to North American, had a relationship with the OMB that was feeding OMB questions that would embarrass their competitors.
Or, result in not doing the shuttle at all and continuing on with the existing systems where North American was building at least the Apollo Command Module.
Players in this included the economic analysis.
Here is a report that was given, as this debate heated up, to NASA in October of 1971 done by a company called Mathematica, which was founded by Oscar Morgenstern, an economist at the Institute for Advanced Studies in Princeton.
And his young colleague, Klaus Heiss, was and is an Austrian somewhat crazy economist.
Again, that may be the same thing, crazy and economist.
And they had the contract to do the external economic analysis for the shuttle.
And they came up, through their analysis, with the conclusion that a reusable system is economically feasible at the current level of activity.
And that a thrust-assisted, that's the strap-ons, shuttle is the economically preferred choice.
This is economists designing technical systems, another thing that would make me nervous.
And this goes back to your comment earlier, the demand for space transportation by NASA, the Department of Defense, but particularly by commercial and other users is the basis for economic justification.
The economic analysis had, as an input, a demand model that was totally unconstrained.
It's everybody's wish list of things that might be launched but weren't funded for the next 15 years.
And that's where the next round of shuttle launches, 50 or 60, which was part of the image at the time the decision was made, came from this demand model which was done by the Aerospace Corporation given to Mathematica to play with in its economic analysis.
It's not clear how influential this set of recommendations was in the final decision to proceed.
Klaus Heiss, who is still very active, claims it was very influential.
I tend to think, well, you'll see my explanation why the shuttle was chosen.
These are the kind of economic comparisons that were talked about.
The launch vehicle investment costs, nonrecurrent, were clearly much greater for the new shuttle system, but the recurring costs of operations were much less than using the current system.
What is that?
Almost $6 billion.
This is 514 space shuttle flights over a twelve year or eleven year period.
Eleven, I guess.
That is, what, about 48 or 49 flights a year, the model that was being used at this time.
It always interests me when people do modeling like that.
You notice they chose the number 514, not 513 or 515.
I mean it sort of gives you the impression that they know what they're talking about.
[AUDIENCE QUESTION] If they had just put approximately 500, that's really as much as anybody knew at the time.
But that's the number that will make it work, I suppose.
[LAUGHTER] Now, that would be rigging the analysis, wouldn't it?
Look at how round the numbers are at the bottom, too.
Well, one of the things to watch here is that a lot of the costs were payload savings.
There was this illusion at the time, proven to be an illusion, that because of the characteristics of the shuttle you could make the payloads much less expensive.
You didn't have to design them to space program standards if you want.
Here are the payloads.
Instead of costing $18 billion over this period, we're going to cost $12 billion.
That's a $6 billion savings in payload.
And it's that combination of operation cost and payload savings that give you the $7 billion advantage in the economic argument for going ahead with the shuttle.
Bush 41 later used the term, which I think is properly applied to this analysis, calling it voodoo economics.
And I think most of the people involved in this decision recognize that.
In a technical decision, the White House often, at this period in time, depended on its Office of Science and Technology, now called OSTP, Office of Science and Technology Policy, and its President Science Advisory Committee called PSAC.
What does PSAC mean?
President Council of Advisors on Science and Technology.
And so the science advisor who was actually an engineer, not a scientist, named Ed David, commissioned a PSAC, President Science Advisory Committee study to look at NASA's proposals as a basis for the position he would take in White House debates.
Head chair of that study was Alexander Flax who was President of the Institute for Defense Analyses, a think-tank in Washington.
And this was a kind of summary report that Flax sent in about the panel.
Doubt that a viable shuttle program can be undertaken without a degree of national commitment over the long-term analogous to that which sustained the Apollo program.
It may be attainable but is certainly not apparent at this time.
This is a long letter, and I'm just going to show you a couple of things.
In retrospect, I think this advice was sound advice that was provided.
Maintaining the program is large and risky with the long-term prospect of fixed budget ceilings does not bode well for the future of the program.
Some decisions had been taken which introduce additional hazards to the success of the program technically, operationally and economically in order to reduce projected peak-year funding requirements.
At that point, I think the strap-ons, firing the main engines at liftoff.
I can show you the analysis in the letter, but I think that's what he was talking about.
And basically what the PSAC panel recommended was postponing the decision for a year or more while some of the uncertainties were studied.
General view.
No significant role for manned spaceflight in military and civilian or science.
Didn't believe NASA's suggestion that the shuttle would allow experimenters to conduct their activities in spaceflight.
Evoke no enthusiasm from the scientists.
You can counter that obviously.
The shuttle was not a wonderful laboratory for most applications.
The scientific community in large doubts the potential benefits of the space shuttle.
Manned spaceflight should be considered contributions in terms of national prestige, international cooperation, exploration and unforeseen future needs.
Basically, the justification was really kind of arm-waving intangibles, some of which I think are very real like prestige and cooperation.
[Jump back a little bit to the?]
science enthusiasm or lack of it, because I think there was a clear division in the science community then between the "real space scientist" and [OVERLAPPING VOICES] which sort of came into its own with Skylab when it was realized [OVERLAPPING VOICES].
But Skylab was two years after this, Larry.
But at this point the life science community was better that interesting things were going to be happening.
But without any real data Skylab was the first long duration exposure.
And the life science community did not have the high step in the space science community at that point.
Space science was dominated by physicists.
And, in fact, even within the NASA hierarchy.
I think at that point it was still part of space medicine.
Right, crew medicine.
So, yes, that split was there.
It must be noted that new approaches have often not been recognized or appreciated by the putative users until after they've been demonstrated.
Yeah, Mark.
Didn't Hubble then conveniently make scientists excited about the shuttle?
Some.
But, again, only after.
We don't want to talk about why.
We'll let Hoffman talk about whether the tradeoff of putting Hubble in the shuttle orbit compared to it being serviced was a good tradeoff compared to where you want a telescope.
If you would have been designing this large space telescope in 1970, would you have made it shuttle launched?
Well, everything had to be shuttle launched then.
I mean given the history of Hubble, obviously, had it been put in an inaccessible orbit, we wouldn't have a space telescope now.
So what can you say?
Again, the shuttle cannot be justified on a purely economic basis for the unmanned portion of the program so it's a position directly opposite the thing I showed you before.
It must be justified on the basis of new capability, contribution to leadership and prestige, its unique value if we're going to have intensity of infrequent manned spaceflight.
And you have to postulate expanding rather than level space budgets over the next ten years.
And the Nixon administration said that wasn't going to happen.
Again, the somewhat bottom line of the PSAC position -- -- led to the conclusion that if you had to make a choice in 1971, you had two choices.
Either proceed with the shuttle program now or soon or drop manned spaceflight after Skylab.
And nobody likes binomial choices like that.
But, in the large degree, that was the consideration or the mental set as this debate came to a head towards the end of 1971.
Actually, that brings into sharp relief.
Remember the comment that Professor Cohen made when he was talking about what should a systems engineer do when presented with requirements that you're not really happy with and don't know if you can meet?
But, on the other hand, recognizing as they came to that basically if they didn't build the shuttle that was being specified they probably were going to end up with nothing at all.
And I think what John just showed was justification that that, in fact, was the political environment at the time.
It wasn't the shuttle or something else.
It was the shuttle or nothing.
Well, except at the end people like PSAC and OMB kept suggesting alternatives.
This was a chart drawn in November of '71 by George Low who was the Deputy NASA Administrator and kind of the technical strength in this thing, showing the investment costs versus, this is in billions, this is in millions, the cost of operations for various things.
The two-stage fully reusable, $10 billion investment, low operating cost.
The baseline 15 x 60 foot payload bay could be done, he's saying, for $8 billion.
Within three weeks it was $5 billion.
A phase development, develop a simpler one first and then a more complex orbiter later with the large payload bay and various rocket assists developing a smaller one, smaller payload, smaller bay, or developing a Titan 3 launched glider sort of thing.
And the argument was in this curve it made sense to pick something along this line, the knee and the curve on that basis.
NASA made its last best case in a memo to the White House.
This is dated November the 22nd.
I think it shows up at the top.
And look at these reasonings.
This is really NASA's best case.
Number one, the US has to stay in the human spaceflight business.
That's not subject to analysis.
That's a belief.
And NASA argued that this should be a policy premise that the United States had to have humans in space.
And the shuttle is the only meaningful new manned space program, the operative word being "new".
You could have kept launching Apollo capsules and Saturn 1bs or something.
Saturn 5 had been cancelled by then.
The shuttle is a necessary next step for science applications, military position in international competition and cooperation.
The cost and complexity is one-half of what it was six months ago.
Again, as engineers, that statement ought to be very nervous that in six months you can cut cost and complexity in half.
And starting the shuttle now will have a significant positive effect in aerospace employment.
Not starting will be a serious blow to both the morale and health of the aerospace industry.
Let me talk about that last one.
Those, I think, were NASA's five best reasons for going ahead.
Employment impact was one of them.
This is an undated memorandum from somebody within OMB.
Peter Flanagan was Nixon's top person right at the intersection of policy and politics who was overseeing the space program.
And Flanagan had asked for impact of the shuttle on the aerospace industry.
And this is what came back.
What the program is.
Here is the additional employment impact on the engine program space shuttle.
Main engine.
Not very much in early '70.
This is '71.
But in '72 fairly significant employment impacts in either California or Florida.
And on the airframe, depending on when the decision was made to go ahead with the shuttle, the impact in '72 not very big, but big enough.
Peak of 70,000 jobs might ultimately result.
The number of actual jobs by the end of 1972 would be relatively small.
Why do you think 1972?
You have to recreate the environment of the time.
This was 1971.
The supersonic transport had been cancelled.
Defense spending on Vietnam was ramping down.
NASA had no new program.
And, in doing the article that Jeff mentioned on the space shuttle decision, I ended up one afternoon in, of all places, Santa Fe, New Mexico talking to John Ehrlich, one of Nixon's top guys.
He said they sat down in the White House and mapped NASA jobs on key election states and said if we want to win the 1972 election, again this is political history well before your time.
At that point, the leading candidate was Ed Muskie of Maine who was viewed as a serious candidate.
It wasn't George McGovern who was, for better or for worse, not a serious opponent.
So that the political people were worried about winning places like California and Florida and saw in the Shuttle Program a way of providing the indication of future jobs in key electoral states.
Some of the people I have talked to over the years say that, at least for the top political levels of the White House, that was the major reason for going ahead with this program.
Was in order to have aerospace employment impacts for the '72 election.
You can judge whether that's a good reason or not.
The decision kept getting postponed until very late in the budget process.
OMB kept asking for more studies.
This was a letter to the Deputy Director of OMB, Caspar Weinberger, later Secretary of Defense, which NASA said we've concluded the full capability still represents a best buy.
But, in recognition of budget problems, we are recommending a smaller vehicle, 14 x 45, because that is the smallest that will still be useful for manned spaceflight, Reid Space Station.
It won't accommodate many DOD payloads and some planetary payloads.
And here are the numbers.
I don't know whether you've seen these numbers yet.
Attached to this letter, this is what NASA was telling the White House last business day of 1971 what the cost of various shuttle configurations would be.
You notice very little difference in the development cost of the configurations, eight-tenths of a billion dollars between a very small and less capable and the full size fully capable.
And the operating cost relatively low across the board.
Look at that number, $7.7 million a flight for a payload cost of $118 a pound.
I think one of the points of your course, if I understand it, is to understand maybe where these numbers came from and where they ever possible?
I shouldn't bias the answer.
Were they ever a possible realization?
I mean here are the heads of the leading technical organization in the US government presenting these figures to the White House.
Did NASA lose its technical integrity in this process, was there any foundation for these numbers or where these total salesmanship, are all, I think, valid questions.
You said you took a two-minute stretch break.
Yeah.
Let's do that, and then I'll come back with the answer of why they ultimately went ahead with the shuttle.
I'm going to argue that the decision to go ahead with the shuttle was made before all of this last six months or so of 1971 back and forth when it occurred.
And the basis for that is primarily this memorandum written through the Director of the OMB, George Shultz, by Cap Weinberger to the President in which he is talking about the staff proposals for reducing the NASA budget, which included eliminating the last two Apollo flights and eliminating Manned Spaceflight.
And Weinberger said in this memo to the President, I believe this would be a mistake.
The reason for reducing NASA is because we cut it because it's cutable, not because it's not doing a thing.
That the uncontrollable programs that offer no real hope for the future, this is remember a republican administration, are eating up the budget.
We do need to reduce the budget but we need to do it on a reasonable basis.
There is real merit in the future of NASA.
And, if you took NASA apart, it would be very hard to put it back together again.
And he says stopping Apollo and not starting new programs would be confirming a belief, I fear, is gaining credence at home and abroad.
Our best years are behind us.
We are turning inward, reducing our defense commitments, involuntarily starting to give up our superpower status and our desire to maintain our world's superiority.
[LAUGHTER] America should be able to afford something besides increased welfare.
Notice the underlining.
And this came back with a handwritten note, I agree with Cap.
That's Nixon.
My view, the decision was made with those four words.
Yeah?
Just out of ignorance, who is Weinberger?
Weinberger at that time was the number two person in the office of Management and Budget, long-time California associate of Nixon, became Secretary of Defense under Reagan.
Who he is, in a sense, irrelevant, except he was a political appointee and a trusted associate of the President.
And basically he was telling Nixon that the reason for continuing the space program was image.
I mean, again, read those words because they're interesting words.
Not having the strongest space program would confirm our lack of desire to maintain our world's superiority.
I should you this December the 29th memorandum where NASA went to the White House and said we would recommend the full-size orbiter usually but, with tight budget, will go with 14 x 45.
That was a Friday, the 29th of December, or maybe earlier in the week.
Anyway, over that New Year's weekend, '71, '72, somehow somewhere Nixon and his inner circle decided to approve the shuttle and approve the full-size shuttle.
And they decided if we're going to approve it, we might as well approve the one that NASA thinks is best.
And there was a meeting scheduled between NASA leadership and the President in the San Clemente on January the 5th.
This is written by George Low.
For a historian, Dr. Low was wonderful.
He dictated his notes every week on the events of the week and then backed it up with the documents.
That's like a treasure load for somebody that's trying to write the history of this.
Met for 40 minutes.
Here's what the President had to say.
We should not hesitate to mention the military applications.
Routine operations.
Quick reaction times.
Solar power satellites.
These kinds of things tend to happen more quickly than we expect.
Nuclear waste disposal.
He liked the fact that ordinary people would be able to fly in the shuttle.
Preserve the skills of the people in the aerospace industry.
In summary, we do not know of the things the shuttle will be able to do.
It will open up entirely new fields.
Did we think it was a good investment?
We, the top two leaders of NASA.
It's not a $7 billion toy.
But he indicated even if it were not a good investment, we would have to do it anyway because spaceflight is here to stay.
Men are flying in space now and will continue to fly in space, and we best be part of it, which was essentially what Weinberger had said six months earlier.
And, to me, that link in doing research in this area, I've talked with both Weinberger and Ehrlichman and others around that.
It's that link of human spaceflight to national image of the United States, plus the employment impacts in the '72 election that were the fundamental reasons for going ahead with the shuttle.
You may make a judgment that those aren't great reasons, but there they were.
Finally, the decision was made, say, January 3rd, we would develop a shuttle with the big shuttle.
And the only major open issue was whether to use a liquid or solid strap-on.
And that was studied for three months.
Trade-off between future benefits and earlier savings.
Liquid boosters have lower operating cost, solid boosters have lower development cost.
Conclusions here are heavily dependent on the mission model.
The basic concern was keeping within the development cost of the shuttle and somebody else worry later about operating costs.
All of that argument led to a decision in favor of the solid booster.
The rest of this is kind of irrelevant to that.
Basically with the OMB acceptance of this letter and the choice of the solids, the configuration was frozen.
There were some things in it that I'm sure you'll talk about later.
There was at that point it had abort capability on the solids.
I'm not quite sure how that would have worked.
And somewhere along the line, and it's not clear to me, at one point the shuttle was going to have jet engines so it could fly to a landing rather than glide to a landing.
And those were taken out.
And I think it was after this, but I'm not sure.
As I said at the start, the technical requirements of the shuttle, I want to say it a little differently.
The reason for approving the shuttle had very little to do with the specific technical characteristics of the system.
If my argument that the main reasons were national prestige, national image, aerospace employment rather than the actual performance characteristics of a particular configuration.
As long as the shuttle could be developed within a $5 billion a year peak funding profile and as long as the shuttle could do things for the Department of Defense that made it useful to both civilian and military users.
Those were the drivers of the shuttle decision.
And the technology was derivative of that rather than the other way around.
That presented challenges, as I'm sure Aaron Cohen or Jeff have talked about, of developing thermal protection, developing a main engine, developing a vehicle that could operate in multiple flight regimes.
But those were secondary to the policy decision that the country should go ahead with this capability.
Questions?
Comments?
Reactions?
Yes, sir.
One quick question.
I saw, in one of those earlier things you put out, that it was around '71, it talked about first flight was '77 and fully operational by '79.
It seems to me that you could really reduce things like heat cost and you could spread out your development cost if you just said we're not in a hurry, let's do it right but let's take our time.
Because there wasn't the race anymore.
I mean we had done the Apollo.
We had beaten the Russians.
And I was wondering what kind of time constraints played into this, why they were trying to finish it by the late `70s and why not say let's launch it mid `80s?
Well, it ties into the current situation rather nicely in the sense that there was then, and I think is now, a perception that an extended gap in US human spaceflight is not politically acceptable.
And, at that point, at the end of '71, the only human spaceflight missions on the books were three flights to the Skylab Space Station in 1973.
The thing that followed that, the Apollo-Soyuz Test Project, had not yet been agreed on.
That wasn't agreed until May of '72.
There would have been from '73 to whatever future date a gap in American's flying to space.
And I think the general sense was that that was not acceptable.
Also, you had a workforce issue of maintaining the workforce with something to do at Johnson, by then not yet Johnson, but Manned Spacecraft Center, Marshall Spaceflight Center and Kennedy.
And so you needed a relatively rapid development program so that you didn't either disassemble the teams and have to reassemble them later.
And the same for the capability inside the industry.
This was a program that was paste within a budget ceiling to make full use of the space industrial base in a reasonable timeframe.
And I think that's why I would say, I mean the dates were set on the basis, this is the earliest we can do it on this budget profile.
Yes, sir.
You said the decision to go for solids instead of liquids for the boosters was the development costs as opposed to the operating costs.
And now, in this new architecture, the plan is to use the solid rocket booster.
It's something like it's proven to be the most reliable launcher ever developed or something along those lines.
It's true.
You've launched 228 of them with one failure.
I agree.
But, at the same time, if the idea then, if they went for solid they could sort of reduce development costs and sacrificing operating costs.
Is sticking with solids in the same configuration now kind of repeating the same possible mistake?
Well, I don't know.
First you seem to assume that going with solids in the first place was a mistake.
That a liquid strap-on solution would have been a better solution.
I'm not necessarily assuming that.
Many have argued that.
It's not really clear that the operating cost of a liquid booster would have been less.
One of the big concerns was you have a liquid booster, you've got a real rocket engine on it, and what happens when that lands in the ocean?
I mean there were real concerns about could you clean up and reuse a rocket engine once it's been exposed to salt water?
And we don't know the answer to that.
Maybe it's the time to segue, if we want to do this, to a quick look at the new architecture as it was presented yesterday, which is being driven heavily, the choices are being driven by budget ceilings again.
An interesting question, Mark, is whether you would be making the same choices now if you weren't constrained by budget, once again.
I didn't mean to assume that the liquid would be better than solid.
Just the observation of the basis the decision was made on.
What this is, or at a certain level what it isn't, is the briefing that is on the NASA website which is a 10-page briefing.
This is the 23-page briefing.
There is a clear set of top level requirements in this new vision.
And, if you're space types at all, you should know this.
An interesting attempt to develop rationale for exploration which, as you see, is mainly intangible.
Curiosity and leadership, they are very much the same things that started the shuttle program.
This is about the only mention of mars in the whole presentation, even though the President's vision says moon as a way of getting to mars.
But here is why moon.
And you're developing technologies that you're going to use downstream.
And, in particular, this Saturn 5 class.
The Apollo 17 Saturn 5 launcher took 117 metric tons to low earth orbit.
This vehicle that's being planned is slightly larger than Saturn 5.
One of the few areas of technological innovation in this system is a new engine which uses liquid methane rather than liquid hydrogen as a fuel.
Why?
Maybe I'll ask the class, why would you be interested in liquid methane?
Yeah.
You could manufacture it on mars.
Yeah, precisely.
It's also a lot easier to store over the long-term, liquid hydrogen.
But the main reason is it is a potential resource that you could get in situ on mars and so you wouldn't have to carry it all the way out there.
And you could get oxygen on mars because there is clearly water.
Yeah.
This may be slightly off topic, but I wonder if you could comment on what you think the feasibility of [UNINTELLIGIBLE PHRASE].
Whether it's feasible or not, we're not going to do it.
Elements of Mars Direct are in the NASA planning for mars which does a fair amount of in situ resource utilization.
Maybe I should back up and say what is this?
Mike Griffin was sworn in as NASA administrator April the 14th.
He had been convinced for a number of months that NASA's planning for implanting the Bush vision was proceeding at much too slow a pace.
And distributing money much too widely, including to MIT graduate students.
Jeff will explain that if none of you were affected by it.
And so he ordered, on April the 29th, a so-called 60-day exploration architecture study to develop a specific architecture for getting humans onto the surface of the moon.
And that architecture was basically finished by the end of July.
And it's taken six weeks to get White House permission to release it.
And so it was formally released yesterday.
It was mainly because of two reasons, because the senior people in the White House were on vacation in August, as we all know.
And so the OMB staff could sit and snip at this and say, well, you can put all this stuff down, but where is the money to carry it out?
You have to show the business case that you can actually do this with the budget that is allocated.
And it takes a little prestidigitation, I think, to do that.
So, this is what NASA has now said is its architecture for the next step in fulfilling our destiny as explorers.
Safe accelerated.
Accelerated in the sense that when Griffin got there the schedule for the first crude flight of the CEV, crew exploration vehicle was 2014, and he wanted the shuttle hard date retired in 2010.
And he wanted to close that gap and thought that it might be possible to have the CEV as early as 2011.
It's turning out it is probably not going to happen.
Why is this just not Apollo over again sending people back to the moon?
Here are the arguments.
For all crew on the moon, you can go anywhere on the moon, not just in the equatorial regions.
You can begin the buildup for permanent human presence in a lunar base.
Do institute resources.
And it's more reliable and safer.
The argument is that, at least on ascent, you get almost a factor of ten improvement in the safety.
And how is that going to happen?
Well, there are specific charts on that later.
Oh, OK.
Although, the system is being designed from getting to the moon backwards.
It can also be used for the International Space Station, if we continue with the space station.
What are we going to do on the moon?
Learn to operate away from earth.
Do science.
Learn how to use local resources.
Develop one mission at a time.
A lunar base.
And develop techniques for the eventual human missions to mars.
So there is another mars mention.
Science group picked a bunch of places that were interesting.
I believe this was primarily a drill because going in NASA knew that its preferred site was the Shackleton crater at the south pole of the moon.
Why?
Because of the possibility of water ice which would be a very valuable resource for in situ utilization.
And each mission will go to the same place and begin to leave on the surface the elements of a long duration base.
How is this going to be done?
First you're going to have this heavy lifter, launch the earth departure stage and the Lunar Lander.
Then you're going to have the smaller rocket launch the crew exploration vehicle.
They are going to rendezvous in earth orbit.
Fire up the departure stage and head off to the moon.
Arrive in lunar orbit.
And then the lander will separate and come down to the moon.
This is the lunar orbit rendezvous method that was used for Apollo, plus an earth orbit rendezvous step.
Work on the moon.
Come back and rendezvous with the CEV.
Probably ablative heat shield for what is hoped to be a land landing in Western United States in Oregon, Nevada or California so you don't have to deploy the fleet.
The hope, but it's what the contractors will confirm, is that most of the CEV will be able to reused up to ten times by replacing just the heat shield.
Again, that's the hope.
That is not put in as a requirement, which is important.
It's a desire.
It's a desire, yeah, because they don't know they can do it.
The baseline design is for a crew to the moon.
The thing can actually carry six people, either six people to space station early on or six people to a mars transit spaceship downstream.
It can also be used, if you take the crew accommodations out, as either a pressurized cargo module going to space station and bringing stuff back from the space station.
Provides down mass capability which is missing after the shuttle goes away.
The Apollo capsule was 3.9 meters across.
This is 5.5 meters, 32 degree slope on the capsule.
So, it's a much larger capsule with hopefully better characteristics.
NASA is calling the bluff of the commercial industry.
It will issue shortly a request for a proposal with a half a billion dollars behind it that says you, you, the commercial sector, demonstrate the ability to have cargo or maybe even crew deliver to the space station and we'll buy those services.
We won't use CEV.
But, in case you cannot demonstrate it, the CEV will be able to be a space station transport and crew rescue vehicle.
I don't think anybody believes that, in the relevant timeframe, anybody in NASA believes in the relevant timeframe the private sector is going to develop crew transport to an acceptable level of reliability.
Maybe cargo.
The National Space Transportation Policy was issued last January that said that there should be full utilization of the evolved expendable launch vehicles, Delta 4 and Atlas 5.
And that caused a problem because NASA said well, for our purposes, we're going to build something else.
And part of the price of that is NASA agreement to use primarily Delta 4s and Atlas 5s for its robotic missions.
The problem with that is those things are expensive, a lot more expensive than the Delta 2s.
And where does that leave people like Elon Musk in space exploration in privately developed launch systems?
Griffin was party to a study commissioned last year by the Planetary Society that came out with the conclusion that a shuttle derived launch system was the best way to approach this.
This study examined that system, shuttle derived, and a number of possible alternatives and came up with this conclusion.
This is the so-called stick.
The first stage is the current solid rocket booster on the shuttle for segment solid rocket.
There's a new upper stage powered by some version one space shuttle main engine, liquid, hydrogen, oxygen fuel.
A capsule on top with a service module and an escape tower.
Now you can say why is it safer?
It's because the crew is above any debris.
And, if something bad happens in the first couple minutes with a solid, you've got an escape system to pull the crew away from it.
And so NASA's probabilistic risk assessments say that this is a much safer system.
You want to comment on that?
Well, the probabilistic risk assessment, I think it's fairly obvious, if your escape system has a 90% probability of working then you have cut down, whatever the reliability of the main rocket, you've just increased your survival probability by a factor of ten if the rocket blows up.
Now, if you listen to some of the stories that Professor Cohen has alluded to, and we'll probably talk more about that, there were some serious questions in Apollo about how well the ejection rocket would work throughout the flight regime.
And he said everybody always breathed a sigh of relief when the ejection rocket was jettison in the course of the launch.
But, nevertheless, we've never used it in the US Space Program.
But there was one example of a Russian Soyuz which they did have a pad abort and the crew was pulled off the pad by the ejection rocket.
At a very high G load, like 15 Gs or so, but they survived and went on to fly again.
You want to comment on the comparison of that to your level of confidence on return to launch site aborts on the shuttle?
Which was never done, thank heavens.
We'll actually talk about some of the abort schemes for the shuttle in more detail.
And I think I'll leave it for that.
But there's no question, the only way to survive in the shuttle is for the shuttle itself to survive.
We now have the capability either of returning the shuttle to the launch pad and landing or going across the ocean.
But if you cannot quite get back for a landing, we do now have the capability.
I showed you the pictures of the escape.
You can actually bail out of the shuttle now, but it has to be under controlled flight.
And there's definitely no system of just extracting you out of the shuttle.
I mean the whole logic is this is claimed to be an order of magnitude safer for the crew on ascent.
Well, I think the other point is that the solid booster by itself is more reliable.
Right now, for the shuttle to have a safe launch, you have to have both solid rocket boosters work, plus all of the three main engines.
So you've got a lot more failure points.
This is a simpler system so only one solid booster.
And then the second stage is your liquid rocket.
So, there are a lot fewer things to go wrong.
Plus the whole aerodynamics is much simpler because it's a simple stack formation.
I think, just from an aerospace design point of view, it is a simpler and safer system.
Apparently, there were some concerns because this was so tall of bending and that sort of thing.
Well, that's something they'll have to deal with.
But we've launched tall skinny rockets before.
And I suspect they'll be able to figure out how to do that.
The heavy lift is built around something derived from the space shuttle external tank with five versions of a throw-away version of the space shuttle main engine, plus two five segment solid rockets.
So, it's adding one more segment to the existing booster.
The upper stage will be powered by one or two derivatives of the J2 engine used for the upper stages of the Saturn 5.
So, this is a pretty retro system.
But it was a good engine.
And the other thing to mention, some people had suggested using five segment solids for the crew launch vehicle.
One of the other things, when you talk about reliability, as John said, we've had now 228 launches of the solid rocket boosters.
And one of the great things about the recovery is not just the economic impact of being able to recover and reuse the solid booster, but you get to examine how it performed.
And that makes a huge difference in terms of flying safely.
Because, if you look back at the history of the Challenger accident, we knew for many years that we had a problem with blow by around the O ring seal.
Unfortunately, for various reasons, management chose to ignore that and fly anyway.
But if you can recover your rocket after you use it and actually see how it performed and look and see if there are any critical failures which are suggesting that there are problems, that also improves your reliability.
So, we have a lot of experience with four-segment solid rocket boosters.
And, by choosing not to use this new and improved five segment booster for the human launches, we're basically saying we're going to go with what we have experience with and what we understand.
Well, besides that, we don't have it now.
It will be developed, but when we develop it and use it for this, some day, after we get a lot of experience with it, we may decide to -- Well, it says can be certified for [OVERLAPPING VOICES].
Now, this is the heavy lift thing.
Right, but you're going to want to fly it many times before we decide to [OVERLAPPING VOICES].
And the intent is not to human rate this from the start.
Without the upper stage, you can get 100 tons, 106 tons to low earth orbit just with the first stage and 55 metric tons to, well, you can read, I think.
That's a new development, obviously.
This is Marsha Ivins who presented this yesterday, one of Jeff's former colleagues.
This really isn't what it's going to look like, the Lunar Lander.
What's interesting is, in addition to carrying the crew down, the idea is that you can carry a fairly significant cargo load down to the lunar surface and leave it there.
And that enables the fairly early buildup of a lunar base capability.
And, again, this ascent stage will use a liquid methane propulsion.
Any sense of scale on the habitat?
Well, this is enough for four people so it's not super big.
I've seen dimensions on it, but this is just a nominal design anyway.
This design, with all the tanks down here, is not very good for carrying cargo down.
They added the cargo capability and didn't change the picture.
Here is what NASA says are the commercial opportunities in this initiative.
It's interesting.
There were some other ones that have gone away from earlier briefings.
Here are the international opportunities.
And they are focused in the longer run on lunar surface systems.
And the reality is that without international contributions you cannot do a lunar base because, on the budget available, you cannot afford to build this stuff.
I am looking at this hard for the first time.
I saw a version of this in July, and there have been some significant changes.
The July version said opportunities for non-US astronauts going to the moon, and it's not here in this final briefing.
Committed long-term lunar effort is needed.
You can show mars up here, but this is really a plan for getting back to the moon.
And to reach for mars we must first reach for the moon.
A Griffin quote, Mike and I have talked about this, he believes that the spread of the human species into the solar system is inevitable, and the United States should lead so we carry the principles and values of Western philosophy and culture.
You can make a judgment whether you think that's a good rationale for doing this or not, but he means it.
Great nations do great and ambitious things.
We must continue to be great.
Cue the music now.
[LAUGHTER] This is interesting.
This was a presentation that was given yesterday.
And it is different than the presentation that was presented to industry yesterday.
And the biggest difference is the industry presentation had a budget.
[LAUGHTER] And the budget shows that within the next five years all of this fits within the plan budget curve and then stops in terms of the affordability downstream.
And it also shows no mars research and technology until fiscal 1917.
And a fairly big wedge for lunar outpost.
This is a US-only scenario.
Any relief from this is going to come from international contributions.
And one of the things that changed over the past week or so is now the phrase is go as you can pay.
In the trade between performance requirements, cost and schedule, what you're going to trade is schedule.
And NASA is very nervous of announcing this thing in the middle of the Katrina recovery saying we're going to spend $100 billion going back to the moon.
Actually, that was the first question that Griffin got at the press conference.
Right.
I talked with some media yesterday, and this is the same sort of thing.
It's interesting that the budget chart is not in the presentation I was given.
The budget said that you cannot do it?
The budget says that they can get started.
[LAUGHTER] What the budget is going to support and the hardware that we will be building, over the next few years it's just the crew exploration vehicle.
And the launch vehicle.
The upper stage in the launch vehicle.
Right, this.
We will retire the shuttle.
We will use that to support the space station, if we're doing the space station at that point.
And not until the shuttle is retired will then the money that is now used to support shuttle flights can start going into building this.
That's the way I understand it.
And that's the schedule which means you cannot get to the moon until near the end of the next decade.
2018 is the target date.
And that doesn't support any extra equipment once you get to the surface of the moon.
So, we don't have a long-term habitat, we don't have rovers, we don't have NC2 resources.
All the stuff that we'd like to do on the moon, that's over and above this.
I think that's fair to say, isn't it?
I think it's fair to say except that, as I say, in this budget curve it's in the budget.
But the only way to do the rest of it is to get that green part, the lunar base buildup to be paid for by somebody else.
So, we will see.
This is your future, I think.
If you're going into aerospace engineering, this is at least the NASA project for the next 15 years.
And maybe 20 years from now there will be a class here talking about the systems engineering of the Lunar Exploration Program.
So, you're kids can attend that class.
[LAUGHTER] Then it was typewritten memos.
Now it's PowerPoint.
For you engineering junkies, there is a thousand page report coming next month that has all the information of the trade studies and everything underpinning all this.
This is the output of a study.
The study report is coming.
OK, just briefly, from most of you I've gotten an indication of what you want to do for your projects.
I'll have a look at those.
If you haven't sent them to me, please make sure I get them by Thursday.
One or two of you have said you want to come and talk with me about it, that's fine.
Let's see, that's on the reports.
The last thing, just to remind you, is this really is the last of the kind of introductory policy, how did the shuttle program get started.
For the next six weeks or so we'll be going deep into the nitty-gritty of some of the systems.
Tom Moser will be here, and he will be talking about shuttle structures and the thermal protection system on Thursday.
See you then.
OK. Good morning everyone.
As I mentioned at the last lecture, we're now moving into the nitty-gritty of the subsystems.
And we have quite an extraordinary list of speakers who actually worked on the original design of these systems.
And, as Professor Cohen mentioned, not only did they design these systems for the shuttle but, in most cases, they came from having designed essentially the same systems on the Apollo program.
So, in the question and answer period, if there are some things that you'd like to know about the comparison of Apollo and Shuttle, and also any questions about the future since some of you will be writing up possibilities for future development of some of these systems, please ask the questions.
And I spoke with Tom Moser, and he's fully prepared to handle questions during the talk.
Because there will be a lot of fairly detailed technical stuff, so if there's something that you're not familiar with, you don't understand or just have questions about, please ask while the lecture is going on.
I want to remind you, we have fairly extensive bios for all the speakers.
Rather than take the time in class to read through this, I've posted them all on the class website.
So my suggestion is go to the class website before each class and just see the background of the lecture.
Professor Cohen is going to make some personal remarks about, well, whatever it is that you want to say about our speaker.
Before we go into that, two things.
Tom Moser asked me if we had this book.
This actually is my personal copy.
I'm going to put it on reserve.
We may actually have it in the library as well.
I'll check, and I'll make sure that this gets on reserve.
This really is a rather complete book.
A lot the material that you've seen, the early design of the shuttle is in here, discussions of subsystems, and then detailed discussions of the individual flights all the way up through the first hundred shuttle flights.
It's a very nice resource, and we'll have it on reserve in the library.
The next thing.
I believe I have now indications from everybody about what you want to do your paper on.
We have two groups doing GN&C.
One group doing displays and controls.
We had two small groups interested in the propulsion system.
And I've asked Brian and other people, you've gotten together?
OK. Only one person, Dan, I think, indicated an interest in the thermal protection system.
I know we've got a lot people on GN&C.
One of you, I think, had actually indicated an interest in thermal protection.
I will leave that up to you.
If somebody would like to work on thermal protection with Dan, he's back there and is available, I guess.
We have one group which is going to do a little bit of a different sort of project on modularity and some of the design methods that went into the shuttle compared to modern techniques that are available now, so we'll be interested in seeing what you'll come up with.
And then we had a group that wants to look at the propulsion system, but from the point of view of fuel storage and what the system would have looked like if we had possibly used alternate fuels.
We've got a nice spectrum of things that people will look at.
Any time you'd like to come and talk with me, with Professor Cohen or take advantage of one of the speakers who happens to be talking about the area that you're interested in, please do that.
We do have time usually after class, if you're free.
The speakers will generally be available so take advantage of it.
OK. That's enough talking from me.
Aaron, I'll give this to you..
I just have a few personal comments about Mr. Moser.
He is going to talk to you today, lecture you today about the structures on the space shuttle.
But Tom, as Jeff mentioned, did the same work on the Apollo program.
I was the manager of the Command and Service Module on Apollo and on the Space Shuttle Orbiter, so Tom and I have been working tighter for many years.
And I relied very, very heavily on Tom throughout the years.
I told a little anecdote the other day, and just let me say it again because the person that helped me with it is Tom Moser.
During one of the shuttle missions, we were getting ready to come back, about 11:00 at night, I was getting ready to leave my house to go to the Control Center, which I live about ten minutes away, and we were getting ready to make the de-orbit burn, I got a call from Rockwell International, their head of the program there.
And he said, Aaron, we just did this test.
We took the panel of tiles and dumped it into a bucket of waterproofing agent and all the tiles came off.
I said what do you want me to do with that information?
I said they're getting ready to come back and there's not much I can do about that.
You're telling me all the tiles are going to come off?
He said no, they're not because that's not really what we did on the shuttle.
I said well, why did you do that test?
That's sort of a dumb test.
I had a decision to make, whether I'd call the people in the Control Center like Chris Kraft and tell him we did this test and all the tiles are going to come off.
But we've got to come back.
I decided what I would do is call my good colleague Tom Moser.
I said Tom, what should we do?
We talked for a while and talked for a while, and we decided to keep this information to ourselves until after the landing.
It turns out it all worked out fine.
There were no problems.
But my point to you is that how much I relied on Tom.
The other point is you may find yourself in that type of situation some day on a project maybe not associated with that.
Tom turned out to be afterward my Deputy Manager in the Orbiter Project Office.
Then he went on to become Head of Engineering at the Johnson Space Center and then went on to be Director of the Space Station in Washington, DC and now is a consultant.
So, with no further ado, let me turn it over to Tom.
One thing I learned to do is have my own mic because Aaron will keep it.
One of the advantages to having a gray beard, besides some of the folks in the back, is I'm not going to refer to him as Professor Cohen or as Professor Hoffman.
It's Jeff and Aaron.
It's not out of disrespect.
It's just the way I grew up so that's OK to do.
When Aaron asked me to do this, and Jeff, I thought that's easy, I'll just pull out some of my old notes and stuff like that and I will be up the next day.
Until Jeff sent me the syllabus and what it was to do, it made me start thinking about what I'd spent a good part of my life doing.
And that was designing and development of the shuttle with Aaron and some of the others, but from a systems engineering perspective which you guys are doing.
So, what I did is I went back and looked at everything from a systems engineering point of view from 1968 all the way until '81 when the shuttle flew.
But I looked at it through the knothole of a structure and thermal person.
So, that's what I'm going to present to you today.
And, as you go through each phase of the program it changes.
And sometimes it's down to the minute detail and other times it is the macro.
That's my objective, and I hope we accomplish that.
Ask me any questions any time that you want to.
Just a little bit of credit and recognition of some of the people that made this thing happen.
You're going to hear from a lot of these people.
You won't hear from John Yardley or Max Faget because they are deceased now, but these people up here made this program go.
And then you will have also some references that give you a lot more detail than I'm going to give you today, even though I'm going to cover a lot of it.
There is one unique thing about a program like this that is complex.
And that is the technical team and the management team have to work together and get together.
Everybody that is listed up there stayed on this program from day one, and that is key.
I don't think there has been a program since then that that has happened.
And I think that if people judge the Shuttle Program as being successful, I think that that's a major contribution to that success.
Beginning in '68 when the concept studies began, all the way down to operations from a structural perspective, what is important at each one of those phases in its weight and cost and produceability from day one.
But then it gets into the certification phase, the last part of the program, how are you going to certify this thing?
How are you going to prove that it's good for flight?
How are you going to prove that the crew is safe?
And you have to do it on the ground.
So, it's a completely different knothole you're looking through and a completely different set of parameters.
I didn't start on Apollo at the beginning like Aaron did.
I came out a little bit later.
But, in this program, I had the good fortune of working on it from sketchpad to launch pad.
And that's the way I characterized it.
And Aaron didn't know this, but at the end I would have worked for free to complete the program because I was going to see something from beginning to end.
And so if you ever had that opportunity to get on the program at the very beginning, I don't care what knothole you're looking at it through, stay on that program if you can because it's a completely different life at every step along the way.
So, from the knothole of Orbiter structure, I'm going to break this thing into two pieces, Orbiter structure and the thermal protection system.. On the concept studies that began in '68, what was the objective of those concept studies?
Conceiving, characterizing and characterizing by qualitative and quantitative concepts that would appear to work.
So, it was determining really the feasibility of the concepts.
When that began, the only requirement in the Shuttle Program was to have a reusable space transportation system.
Get something that goes from earth to low earth orbit and back reliably and reusable.
There wasn't a requirement on payload size, there wasn't a requirement on number of missions, there wasn't a requirement on any of those things.
That's what it was.
The variables that we all looked at, though, were budgets, yearly budgets, developments costs, operations cost.
And you'll see, as you go through some of this stuff, and probably some of the stuff that Dale Myers showed you, is development cost, you can spend a lot of money on development and reduce the operations cost or you can spend a little bit of money on development and have very high operational cost.
So, there's a trade there.
And when you're dealing with budgets that you really don't understand exactly what they are, that's a very important variable.
Payload mass and size is important from a structural and thermal standpoint because it has to do with mass and it has to do with energy that has to be dissipated during reentry.
The operational orbit is important.
Fully reusable flight systems or partially reusable, and that's a trade on cost and weight.
Turnaround time and cross-range.
Cross-range was critical to us at this point because NASA didn't have a requirement for cross-range.
And that's cross-range after you come back into the atmosphere to be able to deviate your normal ballistic trajectory coming back in.
But the Air Force thought that they had a requirement so we had to include that.
And then, as we got into the next phase of it, looking at what was important from a structural standpoint is the efficiency of the load path, the weight, the payload size, the aerodynamic surface loading, now we're beginning to get in and look at what are the wing loads, et cetera, and how does that manifest itself in weight and produceability?
We did that under two years of contracted effort.
Then within JSC, the Johnson Space Center, which was formally the Manned Space Craft Center, we conceptionally looked at designs in-house.
And we looked at 53 different designs.
You talk about a systems engineering thing, what we did was got a group of people about the size of this room and went away and locked ourselves up.
But we had expertise in every area, propulsion, guidance and control, aerodynamics, aero heating, that kind of stuff.
And we were almost doing a configuration a week when we did that.
So, when you look at 53 different configurations over basically a two-year period, we were not only just looking at it but we were quantifying then, what it meant in terms of all the parameters that I showed you a while ago.
How much did it cost?
What was the development cost?
What was the operations cost?
What was the weight?
What was the maximum temperature as we saw on the vehicle?
Was there a thermal protection system that could accommodate those types of temperatures?
That was the characterization in the systems engineering parameters that we used.
Tom, where these designs kind of refined?
Were you refining with each iteration or was it like we'll try this for a couple iterations and then maybe try something completely different?
Were you working towards a final [OVERLAPPING VOICES]?
Well, what we were doing was looking at the variables.
In other words, like here, when we looked at 53 different things, we looked at payloads ranging anywhere from 15,000 to 40,000 pounds.
We didn't know what the answer was going to be.
The driving thing there was reusability and cost profile.
We thought we knew what we could afford to do.
We looked at payloads from 8 feet in diameter to 15 feet in diameter, 30 feet long to 75 feet long.
We were looking at all of those things.
And, as we did that, we changed configuration from a straight wing, which is not good for cross-range, to a delta wing which is better for a cross-range, to a double delta wing which is even more structurally efficient.
Landing weights then was important.
Just like everybody saw the plane landing last night, it had to get that landing weight down to where it could control it.
Well, that was the same thing we were doing.
We were looking at landing weights, not only from a controllability but also from a produceability.
We looked at weights from 70,000 to 215,000 pounds, boosters from fully reusable to partially reusable, propulsion systems and various types of things.
We even looked at air breathers so that they didn't have to come back a dead stick like they do now when Jeff was in the vehicle.
And pressure fed and pump fed systems.
There was simplicity and complications in all of them.
Tankage, internal or external to the Orbiter.
Could you maybe say, for people's benefit here who have grown up in the computer age, something about the tools that you had?
I am going to do that in a minute.
You are, OK.
I'm going to give you a challenge, too.
So, we looked at all of those.
And, we not only looked at it, we quantified it to the extent that we could get the first order estimates on what the cost in all those things where.
Now then, I chose a couple of them here.
This was one of the early designs of the Orbiter.
Look where the locks and hydrogen are.
It's inside the Orbiter.
It has a straight wing so it was fairly lightweight, except for having to carry all that tankage.
The engines were on the Orbiter itself, but the payload was really small.
Very low cross-range, very low payload.
So that was probably the 8 foot diameter 40,000 pound payload.
We evolved that over a series of studies until we finally got down to about February of '72 where we said we're going to have to have a larger payload than this.
And the cost profile didn't fit.
What we did is we came up with a configuration that's almost like the shuttle that you see today.
Large payload, all the propulsion systems, the main propulsion systems are outside the Orbiter.
And it's a fly back, and part of it was throwaway.
In this case, we had a booster in line with the external tank rather than in parallel with it like the SRBs are today.
Basically that's what we started the detailed design and development with, but not exactly.
And I'll show you that.
Now then, some of us who like to worry about load pass and simplicity and low weight of the structure, we said ah-ha.
What we will do is put the engines on the external tank.
All the mass is down here.
Very little mass up here.
And for boost reduce the weight of the Orbiter, which is going to reduce development and operations costs a whole bunch and thermal protection system, but we have to swing these engines for reusability from the external tank back up the Orbiter and stow them for entry.
Well, our brother in mechanical engineers, they beat us pretty hard.
They beat us black and blue.
Anyway, that was a concept that we looked at very late in the program.
And that didn't go anywhere, even though Max Faget and I wanted to do it pretty badly.
As we continue now.
Now we're four years into the thing, we've done all these systems engineering analysis, so we end up with a final concept.
Here's what we want.
We want a 2.5 stage launch vehicle because it costs too much basically to fly back the booster.
We just didn't have the money so we said we want to have the most important part be fully reusable, so that was the Orbiter.
It was going to be a delta wing.
We figured that the mission life, we had a mission model of 500 total missions, 100 missions per vehicle, and there were five vehicles.
We had an ascent acceleration of 3 g's.
Why was that?
Really, the requirement was to keep down the inertia loads, but it was also to let people off the street flying the thing.
Go ahead.
What constitutes half a stage?
Half a stage means that the external tank is like a half a stage.
The SRB is a stage, the engine, the Orbiter is a stage, but the tank is a half a stage.
We kept the max dynamic pressure down because that was a major driver for control systems and for aerodynamic services.
You would love not to have the wings on the Orbiter going uphill.
That's a penalty that you pay.
So, one of the things to help reduce that is to keep that aerodynamic load down on the wings.
Then for atmospheric flights, and I'm going to talk a little bit more about that, we said this thing is going to come back like an airplane.
Let's fly it.
Let's design it like an airplane 2.5 g normal maneuver load factor and a negative 1 g. A crew of four for one week.
That becomes kind of important because that sized the crew module, that sized a lot of the environmental control systems you'll hear about later and other things in the life support systems.
So, without a lot of changes, to show you what the flexibility and capability of this vehicle is, the Orbiter is now flying seven people for two weeks.
So, it went from 28 man days to 63 man days basically.
And I don't think a lot of people understand what that has done.
And there is a lot more robustness in the Orbiter, in the shuttle system that was not designed into it but had some inherent capability.
And also some of it was a little bit of forethought in the thing.
Here comes the Air Force stuff.
65,000 pounds up.
40,000 pounds return.
That was the requirements.
15 x 60.
Another important thing is we didn't know what they'd be but maybe up to five payloads at a time.
And that's going to become a problem, I'll show you about in a minute, as we start peeling this systems engineering onion of getting down into the details.
And deployable payloads.
Cross-range.
A little less than 1300 nautical miles cross-range.
TPS material.
We didn't know what the hell it was going to be, but that's what we started with.
So, we began the contract for design, development, test and evaluation.
NASA does pretty good stuff in-house.
And I think with Dr. Mike Griffin at the helm now you're going to see a lot more of that coming into NASA where they're doing a lot of stuff in-house.
But when it gets down to doing the detail, design and manufacturing and cost-control, down to the detailed parts and manifesting everything around, that's where the contract is.
NASA doesn't have that capability in a large program.
So, this is where we gave a contract to Rockwell International.
They had the integration contract.
And another company, Martin, had the external tank.
[Backhaul?]
had the SRBs.
And who else am I missing?
And Rockwell had the integration and Orbiter contract, both.
This is what they started with.
That was their authority to proceed configuration, even though this is shown as in 1972.
But you see there is very little difference in the configuration then and what the configuration is today.
There was some minor mods which are not worthy of even talking about right now.
But let me give Aaron a lot of credit.
His favorite word, the whole time he was the project manager, was no.
And he had above his blackboard better is the enemy of good.
We've got something.
We know it will work.
We knew that damn configuration would work.
And Aaron got inundated with people coming back after we started the program.
Aaron, if you put reaction control jets out on the wing tips and here and up on the vertical stabilizer you get a lot more control authority.
And when you guys start looking at this guidance and control stuff in propulsion you're going to come up with that.
But it complicated the entry in the thermal protection system.
It complicated getting the fuel to those things, so Aaron said no, no, no.
And the astronauts would come in.
They'll always meet with Aaron at 7:00 in the morning because that's when they would get their word in.
And so they would have to go do flight training or something like that.
And, as they walked out the door, Aaron would say no.
They didn't hear him, but it was always no.
And that was critical in this thing.
And so the program came in at $5.1 billion.
It started $5.1 billion.
It came in at $5.1 billion.
It was only because of being able to say no.
But to say no you better do a good systems engineering job at the beginning.
And were there some faults and errors?
Yeah.
I'll confess and open my kimono here on the few of the things, but all in all it wasn't too bad.
And I will say one other thing about systems engineering.
It was interesting to watch four or five different large companies look at the various concepts.
I can say this now because none of these companies even exist in the form that they were then.
Grumman had a very large systems engineering organization.
Rockwell had a very small systems engineering organization, almost down to one or two people, but they were extremely good.
They were extremely good systems engineers.
NASA went with Rockwell for a number of reasons, but one of the things that probably benefited the program was having a very concentrated set of engineering requirements coming out of a systems engineer which almost turned out to be one guy, Ed Smith.
And he was very, very good at that.
The reason I bring that up to you, the thing that's most defficient, from my perspective now in the United States today, are good systems engineers.
Very good thermal engineers, good structural engineers, good propulsion engineers.
There are very few systems engineers that are good.
If you make a note of that and become one of those, you'd be highly sought after.
So, a little bit of this is a repeat.
As we went into these requirements, and from a government system's perspective, now the challenge was to give the contractor the requirements that they need but don't over-specify the requirements.
We were very careful to say here are the top level requirements, don't ask us what the internal loads on the wing are because we're not going to tell you what that is.
That is for you to decide.
And, if you want to change something within these constraints, you can change it.
But the burden is on you to make everything else right.
That is something that I think the Orbiter did probably better than the external tank.
I'll just say it like it is.
Go ahead.
I'm just wondering, sir, what happened to the canard configuration with the delta wing?
What happened to it?
Yeah.
It was not selected.
And I don't remember why it was not selected.
What was driving it, I don't know.
I don't have those notes anymore.
It could have been cross-range.
I don't remember that particular configuration, if it had the required cross-range for the payload mass.
I just don't remember.
I don't think it would have been cost.
I think it was a lot of complexity.
Well, it did have complexity so it had to add some cost.
Weight-wise, it probably was not too different just because of the control authority of having the canards there.
But that's a good question.
Tom, you might mention [UNINTELLIGIBLE PHRASE].
I'm going to touch on that, again, a little bit in a minute about what we did in surveying and determining the loads.
And let me hold that if I can.
Now we have begun with the design..
When you use the term top level requirements, what does the top level refer to as opposed to just requirements or performance requirements?
Well, the top level requirement changes as the phase changes.
The very beginning, in '68, the top level requirement was a transportation system.
We don't know what it's going to be, reusable, and we don't know how much payload it is going to have to carry.
So, that was the top level requirement.
Then it was to look at all of the combinations of things that you could create.
The solution was it is feasible and we think that this is about what it's going to cost.
Now, as we get into this point, these are the top level requirements.
So, the granularity of the requirements increases as the program advances.
Good question.. For the challenges, now I'm going to call it the challenges of beginning this thing, we know what the configuration is, you know what the design life is, et cetera, all that kind of stuff, but we still haven't decided on what the material of the airframe is going to be.
We estimated some stuff because it could be aluminum or it could be titanium and this is what it would be.
And it all fits within the right cost and performance envelopes, but let's optimize that from a systems standpoint a little bit and see what that is.
Some of the challenges in structural design, and I'm going to talk about each one of these things that's listed on here separately, should the cabin be an integral part of the fuselage or should it be a pressure vessel floating within the fuselage?
Trade to be made.
How are we going to account for thermal stress in this?
Well, what the hell is thermal stress?
Apollo, we didn't care a whole lot about thermal stress.
It really wasn't an issue, not to the extent that a vehicle like this is.
It was very sensitive for thermal stress.
Compartment venting.
We'll talk more about that.
Major structure concept trades to reduce weight basically.
And then how in the hell do we get the design loads on this thing?
From a structural design criteria we said, well, let's start with 1.5.
That's what all airplanes are designed for so we'll do that.
Even though some of the boosters were designed for 1.25.
Does everybody know what an ultimate factor safety is?
That's the allowable of the material that you've decided to use compared to the maximum expected load that you will ever want to see, three sigma kind of loads.
And then whatever the factor is about that, that's the factor of safety.
So, you simply take the maximum loads you can expect to find, multiply it by 1.4 and it better meet what the allowable is.
If there's margin in that allowable then that's called a design margin.
Ideally, you would like to have zero margin.
That still gives you a 40% factor of safety.
Everybody with me?
Yield is classically something that you decided on material.
Well, I want to also have a factor safety on yield.
And we sat around and asked ourselves why the hell do we care about that?
The only thing you don't want it to do is you don't want it to deform such that it won't operate so doors won't open, hinges won't work, et cetera.
We did not impose that on the program of the yield factor safety.
We didn't on the Orbiter.
The external tank did.
And they paid a weight for that because if you put a 1.2 factor safety on yield for some materials, that gives you a lower allowable than an ultimate factor safety when you're really only interested in ultimate strength.
And then we had said thermal stress is going to be important, but we don't want to be so conservative that we let the thermal stress add in such a way that it adds conservatism.
But, at the same time, we don't want it to count on it if it's relieving when we cannot really rely on it to be there.
So, it is decreasing from the stress point of view.
A scatter factor of four on life for a hundred missions.
What is scatter factor?
Scatter factor just means a factor of four.
If you have 10,000 cycles at 20,000 psi stress then you have to certify it for 40,000 cycles.
Typically, most airplanes you fly around on, they have a design life of about 20,000 flight hours.
I think that's about right.
They are fatigue tested to 80,000 flight hours to make sure that they have that kind of factor on life.
And then we said well, this thing is going to be used a lot, we've never looked at that before so we will arbitrarily say we'll use a 1.2 factor at the end of life or ultimate.
And then these are just classical engineering material allowables that everybody uses today.
A lot of people have mentioned lately, as we were thinking of the end of the life of the shuttle, that although it was designed for a hundred missions, I guess it was always assumed that those hundred missions would be flown over the course of just a few years.
And so, I think it's true, you more concerned with the long-term effects of stress than things like weathering or being exposed to salt, air over 20 years.
Is that correct?
Well, you hit a very key point.
And you're exactly right.
It turns out a hundred missions wasn't really designing anything.
I mean it could have.
Well, as a matter of fact, it didn't.
I don't know of anything.
A hundred missions design is for a cyclic stress, high cycle, low stress fatigue or low cycle high stress.
It didn't.
But the environment sitting around or the life of the material exposed has.
The leading edge turned out to be there is a degradation in the strength of the leading edge material, the carbon-carbon material because of being exposed to the conditions.
There was some corrosion found in the wings during an inspection.
Nothing was wrong with the low carrying capability, except it was beginning to corrode.
[UNINTELLIGIBLE PHRASE] You've got me mixed up with somebody else.
I'm not talking about avionics.
[LAUGHTER] No, but it's true.
There, I think, just to bring that up, the idea was they would not be good for that length of time so change them out, we're going to have to upgrade them anyway.
Aaron, I think, just left being program manager when they decided to change,no you still were there, they decided to change the general-purpose computers.
It took ten years to change the general-purpose computers because of all the certification.
Did the use and safety factors, like what numbers you were using, how much did the Apollo program [UNINTELLIGIBLE PHRASE] aircraft safety factors?
The Apollo program was like 1.2 factor of safety, as I recall, because it was a single use item.
And I'm going to take the opportunity to go back and verify that, but I think that's was right for some conditions.
And it could have been 1.5 on others.
I know on pressure alone it was 1.5 psi.
You say, well, why was it different for pressure than it was for others?
Because there was some historical data there that NASA had that said that was the right thing to use.
But that's a good question.
Now you've caught me.
I can remember on the boosters it was 1.25, but on the command module itself, let me retract that, it was probably 1.5.
It was 1.25 on the boosters and 1.5 on the crude part of the vehicle.
One of the challenges that we had here was to establish a criteria.
These were our objectives.
To assure that there was a realistic stress that we're putting in the vehicle and we weren't being overly conservative with it.
We were not reducing the stress because of thermal gradients.
And then we were incorporating the classical pressure induced stress.
And now what were the details of that?
Here were the details.
We came up with this algorithm that says we will use a factor of 1.4 on all external loads because that's aerodynamic loads, inertia loads and so forth.
We will use a factor of 1.4 on the thermally induced loads, if you will.
They are really thermally induced strains and stress.
We'll use a 1.4 factor on that if it's additive to the mechanical, but if it's subtractive we'll only use one because we probably won't reach the maximum thermal conditions so you cannot rely on that.
On pressure, we used 1.4.
Unless it was pressure alone, we used 1.5.
But the whole thing is we would never have less than 1.4 of the total combined load.
So, that's what we did.
And so you say why in the hell did you do that?
Why did you have to go to that kind of detail?
The reason being is because you probably had about 30,000 stress engineers working on the program, and they needed to know how to combine this stuff.
If you didn't, this guy is going to do one thing and this guy is going to do something different.
We got it down to that level to save weight in the vehicle and save complexity in the vehicle.
Well, we have our criteria set now.
Let me give one more story on marginal safety.
One of the people that I showed you at the top of the credits list was John Yardley.
And probably Aaron and Larry Young and other people would probably agree with this, John Yardley is probably the best engineer that I ever knew in my entire life.
He's probably one of the best managers I ever knew.
John Yardley had the job of being the Program Manager on McDonnell Douglas F4 aircraft.
And he was an old stress guy.
He knew that weight was going to be a critical parameter in the success of that program and he had to keep the weight out.
What he did, on that previous slide where I showed you the stress criteria and make sure that you have a zero margin of safety with a factor of 1.4, is he told all the stress engineers, because they all worked for him, design to a negative 10%.
Which means if you really do your job right this thing is not going to be able to reach ultimate load.
It's going to break.
But he also knew that they were probably conservative because he was one of those guys.
And he also had in his hip pocket, if he's wrong he would find out because he had the opportunity to do an ultimate load test on the airframe.
It turns out he did the ultimate load test, it passed the ultimate load capability and he saved a bunch of weight in airplane which made it a very successful airplane.
Sometimes from a systems perspective, it's what you learn in the details or in the trenches as you're coming up and being able to apply it the same way that Aaron did a lot of stuff as he was managing the program.
On the airframe we looked at a lot of different structural materials, we looked at a lot of different TPS materials, and some of the parameters that were coming out in there was not only strength of the material but how much heat sink there was because you were having to rely on that to keep the weights down.
And let me go up one slide and show you something here.
I don't know if you can read that, but on the left side what it was is it was all aluminum airframe.
It had an ablator thermal protection system on it.
And you said, well, I thought you said it was going to be fully reusable.
Well, we also had a cost constraint.
So we put that in there as a reference point.
And that was the lowest cost.
And here's weight up here.
It wasn't quite the lowest weight, it was pretty low, but it still was violating the objectives and requirements that we had.
And we said that will be our reference point.
Then we looked at different types of aluminum.
We looked a beryllium.
We looked at titanium.
We looked at the thermal protection system on a beryllium substrate.
Every kind of combination you could think of.
The interesting thing was, look where the whites were staying.
They were all staying within the 60,000 to 80,000 pound total weight envelope.
What was happening is we were decreasing the thermal protection system thickness.
If we were using titanium, which we could operate to 600 degrees, the TPS weight was going down.
Titanium was not as good a heat sink as aluminum, even though we're working it to a higher temperature.
And it turned out that the combination of TPS plus structure was pretty much a constant.
I'm oversimplifying it, but that's basically what it was.
And you can see where the cost was.
This is beryllium titanium.
The cost was way out of whack compared to everything else, so we said we're not going to do that.
And there were some other exotic materials over here for hot structures.
We said let's now decide what this airframe material is going to be.
It's going to either be aluminum or titanium.
And total weight, total cost doesn't make any difference, they're about the same.
We weren't' smart enough to decide so we went out to the skunk works.
Kelly Johnson skunk works, I don't know if you know what that means.
Kelly Johnson designed and built more airplanes in a short period of time.
And they were the ones, Larry, was it in the `50s, the SR-71 was designed, the Black Bird?
I think it was in the 1950s.
It was somewhere in there.
They had to develop titanium for that airplane.
It was a hot structure design.
We went out and spent some time with Kelly Johnson.
We went through all this stuff with him.
All right, Mr. Johnson, at the end of the day it's a mix for us.
Would you build it out of aluminum or titanium?
He said aluminum.
And the reason being is because titanium was so difficult to work to produce.
The manufacturing was difficult.
Today it's a lot less difficult than it was then.
We made the decision and went with aluminum, so we had to protect that structure to 350 degrees.
That was a lot of work and a lot of analysis to make that decision.
And we talked to Aaron and said that's what we want to do, and he said go for it.
The next thing we looked at, we got the airframe design, now we're starting to put together the fuselage.
We said we can put this crew cabin as part of an integral part of the fuselage or we can make it a separate pressure vessel inside.
Went through the trades on this.
And we came up with some of the discrete advantages.
It's a purely simple pressure vessel when you don't have any inertia loads, except the mass that is inside the crew module itself.
There is a discrete attachment between the crew module and the forward fuselage which means you could start the design and construction of these two things in parallel.
And simple interfaces are important.
Somebody mentioned modularity, and it's probably modularity in some analytical tools.
But simple interfaces where you're putting things together are extremely important.
So, this created a very simple interface for us.
Also, we didn't have a lot of heat transfer to the crew module.
It was easy to control the environment within the crew module.
And we designed it out of a material, a 2219 aluminum, which had an inherent advantage of if it gets a crack, the crack doesn't grow catastrophically under the operating stress before it starts leaking a lot.
Well, that's good because, from a crew safety standpoint, you know that if the seals are working and it's still leaking you've got a crack in that pressure vessel but it's not going to be a crack that is going to propagate to be a catastrophic failure.
We chose the floating design.
When I talked a while ago about the crew of seven in two weeks now compared to four for seven days, back in '72 we said we don't know what the requirements are going to be, but let's make this thing as robust as we can without penalizing ourselves weight-wise.
So, we went back to Apollo and said what were all the densities in the Command Module for the Crew Module?
We figured out a density and a volume.
We said that will be our baseline.
That's what we came up with.
Well, for another 50 pounds of weight, we could increase the carrying capability of the crew module by about 500 pounds.
That was a good trade at that time so we did it.
We just said we're going to design this thing.
Instead of for 25,000 pounds we'll design it for 30,000 pounds of payload carrying capability within the crew module.
It turned out to be a good decision.
We weren't smart enough to know operationally we'll need larger crews for a longer period of time, but that helped us out a lot.
You probably also want to mention, when you look at the structure of the crew cabin, in both of the shuttle disasters, the Challenger and Columbia, we have every indication that the crew cabin actually survived the breakup of the Orbiter.
So, it really was an excellent structure.
I'm glad you brought that up.
As a matter of fact, I have a note here to bring it up myself.
Jeff is exactly right that that's what it did, but it was never designed to be a crew escape module.
It had some inherent capabilities, probably could survive some things that it couldn't had it been part of the fuselage, different flight regime, so it had that inherent capability.
But it was never designed -- And I know there was a lot of talk after Challenger that it looked like it would have survived all the way, but it would have never made it all the way.
But it would have under a lot of conditions.
We said Apollo didn't have a lot of thermal stress issues, but we know this vehicle now we've skinnied down weight-wise as much as we can to the extent that the thermal gradients between any two different pieces of structure, where they were different materials or different masses is going to cause a thermal gradient.
And thermal gradient cause thermal stresses.
And you will see in a minute that was not only important to stress but is important to all these tiles we were sticking on the outside of this thing.
So we said we have to look at every one of these thermal gradients and we have to understand what that induced stress is because we had an indication it was going to probably contribute about 30% of the total stress in the vehicle.
It was going to be from thermal stressors at different flight regimes.
Kelly Johnson helped us out pretty well deciding on aluminum and titanium.
We said why don't we just fix this thing?
We'll design around all the thermal stress.
We'll put stress relief in it, like expansion joints along the sidewalk.
We'll do all these kinds of cute things and we'll simplify the heck out of this.
As a matter of fact, the SR-71, the Black Bird, it had huge thermal stress problems.
So the wing on an SR-71, normally the skin carries a lot of the wing bending.
Not true on an SR-71.
It's corrugated skin so it can expand and contract.
All the wing bending is carried in the spar caps, the frames that go out the entire wing, and the caps themselves.
They paid a penalty but avoided the thermal stress issue.
We said we'll be smart with that.
We'll go talk to the SR-71 guys.
And we'll go talk to the Concorde.
The Concorde was an airplane that had high thermal stress, even though it wasn't that high a temperature.
I think it was reaching 500 or 600 degrees outside, but it was moving fuel around all over the vehicle.
So when it moved this high mass of fuel from one part of the vehicle to the other, it was creating big thermal stresses.
They knew what to do.
They designed in stress relief at these high points.
It bit them.
Every time they did it they had fatigue failures.
Every time they did that.
They finally gave up and said just accommodate.
We said we'll do that.
Now then, we decided on that criteria.
We're going to account for thermal stress, but how can we do it?
We don't have a 3D model that we can apply mechanical loads to, I'll call aero loads mechanical loads, and temperature distribution.
The finite element model didn't exist.
We had a finite element model that had 50,000 degrees of freedom, but we didn't have a computing capability to combine thermal and mechanical loads on there at the same time to be able to decide how to size the structure.
We said now we've got a problem.
What the hell are we going to do about that?
What we did is looked at what were the conditions causing the thermal stress?
Going uphill thermal stress is not an issue.
It's all coming back in, in the entry phase.
We knew we had initial conditions that were going to primarily be the cause of it.
Coming back where the vehicle had been sitting in top sun for a long time, bottom sun for a long time, side sun for a long time, so we looked at those initial conditions as being the worse.
And we proved to ourselves that it was the worst.
Then what we cleverly did is we went around the vehicle where we had a detailed structural model of a lot of stuff.
And we created a hundred different thermal models of different types of structure.
This is one that was in the wing truss.
We had a wing skin panel, we had a truss member and a lower wing skin panel.
And we did a detailed thermal analysis of these hundred models.
We then applied that to a structural model simplified, which was giving us the internal loads and stresses that we needed, and then we hand extrapolated that over the entire vehicle.
There was no other way to do it.
So, as you look at your analytical capability probably on your damn laptop now, computing capability, you couldn't really do it on that, but think about that.
And I noticed nobody wants to talk about structures as one of these groups.
If one of you changes then I'm going to feel really good when I go back and fight the hurricane in Texas.
But that's the way we did that.
It was a necessity that we had, but we didn't have the capability so we invented a way to do it.
And, I'll show you in a minute, it worked.
Another issue that we had.
Normally you'd like to just vent everything through one area in the vehicle, but we couldn't do that because the payload bay had to be very clean.
It had to be contamination free.
And there was hydrogen in the backend of the vehicle.
There was a pressure vessel in the front-end and a bulkhead up there.
We said what we've got to do is we have dictated to ourselves that we have to design a venting system.
This turned out to be a major part of a lot of internal loads in the vehicle because of venting from one compartment to the other.
And, stop and think about it, we had vents all along the fuselage.
We had a different pressure coefficient at each one of these vents for our various attitudes during ascent, for our various attitudes during entry, so we now had a whole myriad of complicated internal pressures that we had to accommodate.
But that was pretty straightforward.
We just complicated our design with the venting system that we had, but we had to do it to meet the requirement.
I don't know that people are actually aware that there are all these vent doors because it's not something that you would normally pay attention to in the pictures.
That's right, you don't.
As a matter of fact, in this book that Jeff referenced, it shows where all the vent doors are.
I didn't show that detail to you today.
And they do open and close at different times during ascent and entry.
Venting just for carbon dioxide gases?
No.
When you start off you're at one atmosphere, right?
As you rapidly go uphill there's a delta p across internal bulkheads and internal compartments, internal and outside the vehicle.
And, depending on what the flight regime is and where the shockwave is and where the vent door is, it's changing the whole venting thing.
That's a whole lecture in itself on what we did.
The crew compartment is designed to have a delta pressure of one atmosphere, but inside the payload bay, that's not designed to be a pressurized environment.
And so you need to be sure that the air can get out of the payload bay fast enough that you don't over-pressurize, for instance, the payload bay doors or other parts of the structure.
There were some other trades that we had to make.
Let me skip forward so you can better understand this.
For reference purposes, the main engines are here, 1.5 million pounds of thrust coming into this part of the vehicle.
There is a Longeron that goes all the way along here.
A big mass up here with a crew module.
And the crew module, I showed you, had just discrete attachment points.
All of the ascent inertia loads are reacted right here, so all these loads go along this Longeron.
Now, all of a sudden we have a very good and efficient load path.
The wing right here, this is a primary load carrying member of the wing.
That's a spar.
And it ties into this big bulkhead which is right here.
When Jeff talks about the pressure differential, don't forget this thing is 15 feet in diameter here so you can imagine the total loads you have on that with a couple of psi delta p. It's big.
That was a significant part of the driving stresses in that.
Another point I want to make about this is -- Well, I talked about simple interfaces between the crew module and the forward fuselage with bolted attachments.
We had a simple interface between the wing and the mid fuselage.
A simple interface between the mid fuselage and the aft fuselage.
The same thing with the vertical stabilizer and aft fuselage and these orbital maneuvering propulsion system.
That was important, from a structural point of view, to be able to modularize and analyze these things.
But it was also important because four different contractors built all these parts so they had to have an interface that they could not only design and analyze to but that they could physically attach to.
Sometimes, when you have just a sketch on a piece of paper, you don't think about that.
And it does cause a little bit of complexity sometimes in a program.
The main thrust structure is carrying 1.5 million pounds of load from the engines.
How do we design that?
Well, we could have done a space frame or we could have done a truss configuration.
We decided to go with the space frame or the truss rather than a plate girder, the term I didn't use correctly.
And with that we saved 1700 pounds of weight in the vehicle.
We used titanium.
And this is a compression design.
We thought we needed to get a little bit more weight out of this thing.
What can we do to increase the compression modulus of titanium?
We put boron/epoxy, scabbed it on the axial load members of the thrust structure, and that's the way we got a lot of that weight out.
That was a manufacturing problem, I won't go into now, of how you build this thrust structure, but it works fine.
The aft wing carried through -- Excuse me.
Go ahead.
I just had a quick question.
Sure.
We heard earlier that there was a CG problem in that the CG was too far forward in the aft [but bled in the back?]
for a number a number of missions.
I was wondering how that weight that they had to add to the CG compared to the weight that you guys saved.
The weight of the payloads themselves, you're saying that there was a problem if heavy weights were too far aft or too far forward?
The Orbiter.
I guess the CG location was too far forward and they had to adjust it on certain missions by adding weight near the back.
Well, what that is, that is true and that's a problem.
It's not a problem.
It's something that has to be addressed on every mission, depending on what the payload is that you're carrying.
If you have a real heavy forward payload, yeah, you have to add some ballast to the aft end of the thing.
Now, normally the way they do it is they'll find some payload that can fit in the aft end to help ballast that.
But I think, in some of the early flights, we put some ballast in the back for CG control.
We've carried many tons of lead into orbit, so it was ballasting the thing for control purposes.
The same way yesterday in Jet Blue, a lot of people had to move to the back end of the airplane because they wanted that CG for a little bit different landing performance.
1307 bulkhead, we saved about 500 pounds there.
Now, see what we're doing?
We started out with the initial trades in the early concept phases.
What's the wing loading?
We didn't care about what the internal structure trades were.
Now we're getting down to trading all the stuff at a semi-macro level.
And I'll show you in a minute a micro-level that we had to get into, which was pretty interesting.
How are we doing time-wise?
We're OK.
So, we saved some weight there.
An interesting thing about the payload bay doors.
We decided that for this vehicle to be safe and to re-enter those doors had to close.
We had a lot of trouble in Gemini with things on orbit not working.
Mechanical systems quite often are problems.
Docking systems are problems.
There have been a lot of door problems on orbit in spacecraft.
And you design them and you put them in thermal vacuum chambers and they all work, but you get on orbit and sometimes they don't work.
Maybe it's thermal distortion that we're not accounting for.
So, we said that is critical.
We've got to close these payload bay doors.
What we did was say the way we're going to do that is the payload doors will carry only two types of load.
They will carry pressure loads, like Jeff talked about, and they'll carry torsion loads.
Because, if they are closed, we know that it is good for torsion in the vehicle, reacting torsional loads.
But we will not let them carry body bending loads or else we cannot make them flexible enough.
What we did is all of the body bending in this vehicle is carried along this Longeron through this section in the lower skin of the vehicle.
This is the modulus of the vehicle, if you will, at a cross-section here between this Longeron and the lower skin.
Payload bay doors don't come into being.
And the way the doors close on orbit is they start zipping along from here up to the top because they are flexible.
And the same way back here, they zip closed, zipping just a latch at a time.
They were sort of ratcheting themselves closed.
And then they close along the center line.
There has never been a problem with payload bay door closures on orbit.
It was something we decided to add to the design.
We could have made the vehicle lighter had we not done that, but we would have also complicated the safety of the vehicle if we had done it.
Sort of in line, but it's not the CG thing, we had a payload attachment issue.
And, when we were laying out all this stuff in the early `70s, we didn't know what the payloads were.
We knew what the total mass was.
We knew there may be five at a time.
We didn't know what they were.
Now, you can just go in and bolt a payload into the fuselage of the Orbiter.
And if you just do that randomly or without thinking about it, now all of a sudden you start analyzing the Orbiter and you start twisting it, what happens?
The payload becomes part of the load path.
Now, all of a sudden, you have impacted the design of the payload or maybe you have impacted the design of the Orbiter depending on what is happening in the payload.
We said ah-ha, what we'll do is make it so it's statically determinant.
If two things are attached statically determinately they cannot interact with one another as far as their stiffnesses are concerned.
We said that's what we're going to do.
We'll put attachments along the Longerons to carry the axial loads, some along the keel to carry some of the lateral loads.
Viola, we've got it.
So, the requirement became that we would design it that way.
Our next step was, and you can ask Al Louviere when he comes up about that because Al Louviere and Max Faget told me they would never fly with those Longerons that way.
We did it anyway.
What we had to do then was say now we understand what the attachment is.
How do we determine within the CG constraints for these different masses -- And we had to assume what they were not knowing exactly the definition of the payloads.
We did a Monte Carlo analysis.
Ten million cases of combined payloads.
CG locations, numbers of payloads, forward, aft, all that.
And we did that with a Monte Carlo analysis.
We said voila, that's where we're going to design the mid fuselage.
That's the way we did it.
Now getting into the detail design loads.
How do we determine all these things now?
Because now it's coming down to sizing the vehicle.
This part of the flight regime from liftoff through max Q, that's a critical loading condition, a critical design for the Orbiter.
When it gets into orbit pretty benign.
Nothing is really happening up there except some temperatures which are going to be important at this point over here, because now all of a sudden you start adding more heat into it.
Now you get into the regime where you're maneuvering in the atmosphere.
Now you're in a different loading regime which becomes critical.
So, critical as far as the airframe is concerned, is here and over here only.
Let me break that down into the constituent portions there.
Liftoff loads.
We had another statistical challenge here because what we had to do was needed to look at all the variables associated with the rocket engines.
They have start sequences.
We've got three main engines that are not going to start exactly the same time.
They're not going to come up with the same thrust profile as they start up.
The thrust vector misalignment is going to be different.
And we also had to look at ignition over pressure.
And I'll show you that was a surprise to us.
Also part of liftoff we had to look at winds and gusts, vortex shedding, proximity of aerodynamics to the other structures is on the gantry in the pad.
And then we had to do the pressurization and also look at shrinkage of these vehicles that components or elements at cryo-temperatures.
Here was a cross-section, in a generic sense, of what's happening during liftoff.
There is some combination of winds that you have to account for.
And I will show you what we did there.
The thrust profile from the SSMEs.
The SRBs come up to thrust after you get confirmation that the engines are operating at full performance.
And there is a lot of vibration acceleration going on.
Here is something that becomes pretty obvious after a while.
When you have a vehicle that is tied down here with a base moment and you put 1.5 million pounds of thrust on it, this vehicle is going to bend over this way.
There is a lot of strained energy in the solid rocket motors when that happens.
If you were to ignite the SRBs, when that vehicle is over here, the party is over because it releases all that strained energy and the SRBs couldn't take it.
When you see the Orbiter prior to liftoff, the SSMEs will come up, you will see the vehicle do like this.
And then, when it comes back over zero, viola, you kick off the SRBs.
But you have to wait until it gets back to that neutral point.
A little detail that is pretty important.
Ascent.
A really complicated part of the design of the vehicle, especially with the aerodynamic surfaces.
As I said before, you would like for these things not to be there during ascent.
They don't buy you anything.
You just need them coming back.
Some other complications here are the acoustic effects will become important, as I'll show you, in acoustic fatigue for the vehicle.
I showed you life.
Well, that's going to be a critical thing.
Another thing is the plumes of the vehicle are changing the pressure distribution as you're going up.
The pressure distribution along the Orbiter, especially the wing, is changing the whole times you're accelerating, not only through the various mach regimes but also because of the blockage from the plume expansion.
We said how in the world are going to do that?
We cannot analyze all those things.
If we do it deterministically, you won't get a capability that you need, we thought, operational.
We said what we'll do is one piece of data that we had, we had synthetic wind profiles for everything that existed at the Cape.
We said that's a given.
We'll use these synthetic wind profiles as a guideline to all of our winds aloft.
What we need to do is decide what the angle of attack and the side slip is going to be through this vehicle flying through these winds and a control system that's changing the attitude of the vehicle.
That's a factor that we have to consider and that's going to be important for the design of the elevon surfaces and all because they're getting loaded up pretty heavily at that time.
There are dispersions in the propulsion system that have to be added.
And then what we decided to do, after we looked at all that, we said what we're going to do is we're going to take all these parameters, and you won't find this in any textbook, but we created something called a squatcheloid.
And I forgot to look up what squatcheloid means.
It must be a Greek term that was really good.
But what we did was flew the vehicle through different mach numbers.
And we looked at the various conditions we could get at for combinations of dynamic pressure and angle of attack and dynamic pressure and side slip for all the mach numbers, and then we said is that realistic to do, not only from a control systems standpoint but from a standpoint of the propulsion system capability?
And we said it is feasible to do so we ought to design within those envelopes.
What we did was walked our way around all of these external points with pressure and inertial loads and everything else and we looked at the structural model that we had simplified.
And we said we found the points that were critical for the vertical stabilizer, for the outboard elevon, et cetera, and that's the way we designed the vehicle for ascent loads.
The thing that that enabled us to do, too, is it enabled us to do a rational combination that Aaron brought up a while ago.
What did you do with the SSMEs for engine out and that kind of stuff?
It let us do a rational combination of engine out and also engine vector control.
You could design the thrust structure so that the engines went out to the extremes, but it didn't make sense as far as the control system was concerned.
What we did is we saved a lot of weight in the vehicle by using a deterministic or realistic, I should say, SSME thrust profile and vector characteristics.
Well, I just mention that.
That's what the squatcheloid did for it.
Also what it did was then gave the guys who were designing ascent trajectories, now they had an envelope to design to, which was key for their operations.
Next problem.
Now we come back to descent loads.
When you said that you designed it for realistic cases instead of worst cases, does that mean the engines were never able to vector all the way out to maximum?
Right.
It just could not happen?
It could not happen.
That was a failure mode that it had enough redundancy and it couldn't happen, and there wasn't any need of creating a load condition like that.
It couldn't happen.
OK, so you looked for it.
We looked for the cases where you could be in the worst case winds, the worst case misalignment with the SRB, with the different throttling of the engines.
And we did all that and said to control this vehicle, because if you cannot control the vehicle there's no need to look at that load case.
We said to control it within those environments, what's the extreme of the engines?
We said that's a design case for the thrust structure.
Good question.
Descent loads.
This was pretty easy.
The structure guys said this is a piece of cake because this vehicle is coming back in, in a ballistic trajectory.
And, viola, there are not even loads on this vehicle.
Well, that didn't make a whole lot of sense because we knew that there would have to be more maneuvering capability than anybody was fessing up to.
Not that they weren't trying to hide it, but we weren't smart enough.
The Aaron Cohen guys weren't smart enough to know all the conditions we would have to fly it to.
What we did was we went through the classical Venn diagram where you're plotting for different mach numbers.
The normal load factor versus the velocity of the vehicle that it can fly in that flight regime.
We said if this thing is going to behave like an airplane and have to perform like an airplane, we are going to design it like an airplane.
We caught a lot of guff.
When Bass Redd comes in and talks in the future ask him about this because he fought us on this hand and foot.
He said you guys don't have to design it that way.
We said well, we're going to, but let him have capability in his aerodynamics and talk to the control system guys about that, too.
And I see the Draper guys left on that.
[LAUGHTER] We said we're going to build in that capability for this thing to maneuver like an airplane.
2.5 g's normal load factor.
Follow-up classical Venn diagram where we have equivalent airspeed of 375 psf.
We were determining what the equivalent air speed for those conditions was.
And then we did it.
Now we've gotten up to a point I'll call CDR, and a point I should have mentioned a while ago.
When I showed the authority to proceed, there aren't any drawings which exist, except some sketches.
I mean there are mol line drawings and that's about it.
And the mol line of the vehicle was what came out of all the other studies.
It said that's it, we've got to put everything within this mol line for this vehicle to perform like it is.
But there are no detailed drawings.
When you get to a preliminary design review in a classical design and development program, typically that's about where you reach 10% of your drawings that are released.
And what does it mean to release a drawing?
That means you sign it off, whatever your discipline is, and it say we can make this airplane to these drawings and it's given to the people to go make it.
So, once you release a drawing you don't ever want to bring it back and change it because it costs you a lot of money.
The PDR, which was somewhere like about '73, '74, somewhere like that, that was the state which we existed.
We hadn't defined everything on the structural low pass like I told you about a while ago, nor the materials.
Now we're at the detail design.
Now we're into CDR, that's the critical design review, and 90% of all your drawings are released.
And now it gets very expensive to go back and change.
You can just see how it would ripple through the entire operation if you change a drawing at that time.
You have to, to make it work sometimes, but you like not to change it.
What we have to do here, our challenge from a structural point of view is to complete the design, do some of the details which means get weight out of the vehicle where you can scrape it without really changing a lot of stuff.
And then also decide how you're going to certify the vehicle.
It means what you're going to do on the ground to say it's really safe for the crew to get in.
That's what that was.
Weight reduction is a major part of the program.
Here are some of the things that we took out about this timeframe.
We took out 900 pounds of weight in the payload bay doors.
We didn't change the configuration, they were still flexible, but they were aluminum honeycomb.
We said we can save 900 pounds if we go to graphite/epoxy.
Graphite/epoxy characterization of the material didn't even exist then.
Literally Aaron, myself and three or four other guys sat around.
We said OK, Aaron, this is what we have, this is what we know about it, here are the risks associated and here is the weight savings.
Go for it.
So, we went for it and it worked.
That was the largest graphite/epoxy structure ever flown.
We characterized it, passed that on to the industry and that helped a bunch.
Another little interesting thing -- I have to watch my time here.
I mentioned the spending profile.
Well, we got into the program about the time we were starting to make the payload bay doors, which is probably '75, '76.
And Aaron didn't get all the money that he needed.
We built the first set of payload bay doors, which is people process dependant.
It's like laying up fiberglass.
It was all hand layup with epoxy and bake it and all that kind of stuff.
We got the first set built and we fired everybody because we didn't have enough money to keep them on the payroll for the rest of that year.
We literally laid everybody off at Rockwell in Tulsa.
And they came back a year later, not all of them, but we had to start over again.
So, there is a risk associated with that.
Thrust structure.
We saved 1200 pounds by stiffening, like I told you about a while ago.
And then we used a bunch of other composites.
A lot of people said, especially after Challenger, let's get rid of this vehicle because it's antiquated and was designed too long ago.
That probably is still one of the most advanced composite structure vehicles flying today.
Except, where there is a lot of aluminum honeycomb, you probably could go with graphite/epoxy.
And I will give you a challenge with that later on.
But there are beryllium aluminum struts in here.
There are boron/epoxy scab on devices.
There is graphite/epoxy all over the vehicle.
Not all over the vehicle.
In a lot of places in the vehicle.
The vehicle has got a lot of composites.
And we stretched to be able to do that.
Certification.
Now we've got the thing pretty well designed.
What are we going to do to certify this thing?
This is where we deviated from the norm and were pretty innovative.
Classically, in the way this program started off, there was a dedicated structural test article and there was a dedicated fatigue test article.
There were a couple of problems with that.
If you don't need it, you might as well not build it, even though that was in the program.
The situation was, as I showed you a while ago, thermal stress was a major part of that.
You couldn't apply a mechanical and thermal load to this vehicle and still be practical.
Concorde did it.
The way that Concorde was designed and certified, they applied mechanical loads in an environment in which they could induce the temperature by convective heating on that vehicle.
And it took them like three years to test the vehicle.
And it was extremely expensive.
It was a big jobs program for Greg Britton and France and some of the others, but we decided we couldn't afford to do that so we are going to have to figure out how else to do it.
Besides that, Aaron had $100 million problem that year.
What can you guy help us do?
What we'll do, we think we can do this, is take an airframe and we will apply 110% of the limit mechanical loads only to it.
And that doesn't certify anything.
And then we will put strain gauges all over this vehicle.
And we will pre-predict what the strain response is going to be for 3,000 points on the vehicle.
If we can pre-predict what the strain response is for applying a bunch of different loads then we know how to analyze the vehicle.
That proves we know how to analyze it.
We can extrapolate to 140% to our ultimate load capability from mechanical, and we will add the thermal stress to it analytically.
So, we did that.
And then we refurbished the vehicle and that became Challenger.
The test article is going to cost $100 million.
We said we don't need it.
We'll use it for a flight airframe.
And Aaron gave us a little ceramic eagle for doing it.
Don't anybody expect As out of this deal, if Aaron has anything to do with it.
There it was.
We applied the million and a half pounds of load at the backend.
We concentrated through loading fixtures, loads on the wings of the fuselage, put pressure differential at various points on the vehicle, and we did exactly what I said.
Fatigue life.
Mechanical fatigue was not a problem.
Acoustic fatigue was an issue because we had some really lightweight structures and really high acoustic levels.
You cannot see it here.
It was like 165 DB around the base-end of the thing, lower levels on other portions, so we said how are we going to do that?
We came up with a different way to do it.
What we will do is we'll go around the vehicle and we will identify a characteristic structure, graphite/epoxy, aluminum, 7075-T6 aluminum fuselage up here, wing elevon, aluminum honeycomb, et cetera.
We identified 44 test articles.
We said that's what we'll do.
We went to Aaron and said it's going to take us 44 test articles.
We went through all the rationale with each one of them, and he said you can have 14.
And so we said but that won't work.
He said go figure it out.
We went back and scratched out heads.
Sure enough, we figured out how to do more extrapolation.
So, we had 14 test articles that you see.
And then what we did was say now we've got our test articles.
Our approach to this thing was we would test them to failure acoustically.
Now we have an acoustic allowable for that type of structure because it's a function of the details.
I mean it had to be the detail.
And, in addition to making sure we have a good test, if this was our acoustic fatigue test article, only the center third was a viable part of the test because the rest of it is boundary conditions that weren't right because they were clamped along the edges or whatever.
We said only the middle third of the test article is viable.
We tested that, we got a fatigue allowable from acoustics and then we degraded it analytically for combined mechanical and combined thermal.
We did that and said also we know that it's probably not going to fail on the first flight so we'll do some inspections.
Tom, let me just make a comment.
He made it sound a lot simpler than it was.
One thing that's important, whenever you do a project or you're in charge of something, you don't want to have yes people around you.
You want to have people that say you're not doing it right.
And it didn't go down that simple.
They were not yes people, believe me.
They were certainly not yes, sir, we're going to go do this.
That's very important.
It's a wonder we still speak to one another.
[LAUGHTER] I'm sorry.
I don't know what a test article is.
It's a test specimen.
If you wanted to put so much mass in here and you weren't sure whether this seam was going to work.
You just put that much mass in here.
You add a little bit more to it and you see if that seam breaks.
That's your test article.
That's what it is, just a test specimen, test article.
But that's a good point because, don't forget, we start off with two entire $100 million test articles for static test and for fatigue test.
Not counting acoustic fatigue.
That was just mechanical fatigue.
As we got into it we said we don't need that.
Did the Challenger's mission life drop because of the test?
No.
As a matter of fact, that's a very good question.
That was one of the questions that was asked by the investigation is what's different about this vehicle than any other vehicles?
Well, it was a static test article.
And we showed ad nauseam that had nothing to do with the failure of Challenger.
The leading edge was not -- Oh, no, excuse me.
I'm off on Columbia now.
No, it didn't have anything to do with it.
As a matter of fact, this is getting into detailed details now, when you apply above a limit load on a vehicle like we did on Challenger it puts a lot of the joints in residual compression.
There is compression at the joint just as you load it and you unload it.
And residual compression increases the fatigue life of the vehicle.
And the reason it does is because fatigue is the function of tensile cycles, not compressive cycles normally.
When McDonnell Douglas tested a DC-10 and they proof load a vehicle, it carries a premium on the selling price for that because it theoretically has a longer life than one that hasn't been loaded.
Good question.
Tom, one more question.
Yeah.
How much did an increase in computer processing speed and availability allow you to do these extrapolations and like do the thermal testing just analytically?
Was that a key factor?
You mean how much as it increased?
Well, how much did it increase between like, say, Apollo and designing the Shuttle.
Not a whole lot.
As far as the analytical capabilities of the finite element models and all, you could get more detail through simplification.
There are more elements available, let me put it that way.
But as far as the crunching capability, it increased somewhat but we were still, you know, if you have to go from here to here, we were about here.
That isn't a quantitative answer for you, but we're still a long way.
We basically used mainframes, and we did have the NASTRAN model.
Yeah, we had NASTRAN.
But NASTRAN didn't have, on a large model like that, the large capability for structural loads and thermal loads.
But you used to put your cards in one day and it would take a couple days before you got your answer.
To do a complete load cycle on this vehicle for internal loads it was something like three months it would take us to do that.
And so it was quite [OVERLAPPING VOICES].
That's another story.
Thermal protection system.
Change gears now.
Let's talk about the thing that protects the vehicle.
Now I'm going to switch back and go back to get ready to proceed toward preliminary design.
What were the requirements on the thermal protection system?
It had to protect the vehicle from max temperatures like about 2800 degrees on the surface, reusable 100 times, it had to be lightweight and had to be cost-effective.
Pretty simple high-level requirements.
And Dr. Bob Ried will talk to you about the things that they derived as far as the aerothermal in this and how it became that, but it was a given to us.
What did we know about this thing?
Well, we had some ablative TPS experience from Apollo.
Gemini had some metallic TPS on it primarily Rene 41 and some other exotic materials.
Mercury had an ablative TPS, but they weren't reusable.
They were hot structure designs which existed but that we didn't have materials that could carry the load at temperature such that it could be a fully hot structural design, and that was extremely complex.
Metallic TPS, we could get there theoretically with some fairly exotic materials called columbium and Rene 41 and some other things, but the devil is in the detail in that.
Let me tell you a story about that.
We were looking at some Haynes 188 panels.
And so we had a test article, a big panel about the size of the desk that you're sitting at, any one of you there, and it was corrugated so that it could expand and contract.
And it was good for 1800 degrees, so we tested it to 1800 degrees in the center of the panel multiple times.
And it had a frame around it where the panel could move around because it needed to, to be able to thermally expand.
And the center of the panel was 1800 degrees, the edge of the panel where the heat sink was, was 40 degrees lower temperature than the center.
And the panel floated within a gap maybe about like that, so it could move around pretty good.
But what it also did, with that temperature differential of 40 degrees, we got something called creep buckling.
The structure expanded and crept and deformed permanently.
Had we flown that vehicle, had we designed a vehicle like that, what that would have done is would have allowed plasma to flow through the shingles, if you will, and into the structure.
And that's exactly what happened in Columbia.
Letting the hot plasma gas get in the vehicle, you cannot stand it.
It's the details about that.
We said we're not going to go with a metallic design.
Plus, if you scratch it, it oxidizes.
And, if it oxidizes, then it can fail.
So we went with something that was just coming online, and that was a fused silica material.
And fused silica is pure sand, pure silica material.
That's all in the world it is.
The process is you make it into a fibrous structure, if you will.
Fibrous material is a better term.
A fibrous material that is mostly air.
It is about the same density as balsa wood.
It has an ultimate strength of about 12 psi, tensile load is about all that it can take, but it has a thermal performance that is fantastic.
As you will recall this chart here, when you look at LI-1500, is what that was with aluminum, and that's what we went with, we used this same characterization to decide what the material we were going to use was on the vehicle from a weight standpoint and a cost standpoint.
The way the tiles work thermally like this is the low strength brittle, almost like glassy material, it's a ceramic, is coated with literally a glass coating on the outside about 60-thousandths of an inch thick and it's black.
As the vehicle comes back in and dissipates its energy through drag and heating, 90% of the heat is radiated away from the vehicle, 10% of it goes into the vehicle.
But this is such a good thermal isolator, a poor thermal conductor, by the time the heat gets to the vehicle, the vehicle is back into an atmosphere where it's not heating anymore.
That dictated the thickness of these tiles, of this silica material.
The vehicle has got almost 21,000 tiles on it, and they vary anywhere from a half inch thick up to about three inches thick.
It's sculptured to stay within the mol line because the aerodynamicist told us what it has to be.
And so what we did was sculptured the TPS so that we didn't have any more than we needed.
Great job by all the thermal analysts.
I mean they did this fantastic.
I want to mention, when that says 9 pound or 22 pound tiles, that's not the weight of each individual tile, that's the density per cubic foot.
Per cubic foot, correct.
Thank you, Jeff.
What was the footnote about Columbia on that?
On the last chart, Tom, there is a footnote about the difference in the Columbia.
Let me come back to that in just one second.
What I want to talk to you about right now, this is something I had from some old material.
I was just trying to get a characterization.
What that said is on Columbia there was an infrared sensor as an experiment.
So, the number of tiles varied on that vehicle compared to all the other vehicles because of this infrared thing at the top of the vertical stabilizer.
I should have cut that off.
You're reading too close.
[LAUGHTER] We thought let's figure out how to take this fragile material and put it on this aluminum structure which has a high thermal coefficient of expansion.
The tiles have almost a zero thermal coefficient expansion.
We said we've got to isolate that.
We put a strain isolation pad [UNINTELLIGIBLE PHRASE].
Why do they vary in density?
Yeah.
Because primarily for strength, for one of the things.
And the LI-2200 was a little bit higher temperature capability.
And I don't remember [OVERLAPPING VOICES].
But it had a little higher peak temperature capability.
And that was dictated by what one of the failure modes of a tile was if you exceed the temperature too much it starts slumping.
I don't want to say melting but it begins to distort.
And a LI-2200 didn't distort to the same extent that a LI-900 tile did.
We said let's isolate the structure here from this material up here which is required to keep from failing all these tiles, so we put something between them.
It was just a very loosely woven felt material.
Now we proved that the aluminum can expand all it wants to, or within limits, we looked at it realistically, and that the tile was OK, except the tile couldn't be too large because of this relative expansion and contraction.
So, that dictated the size of the tile.
We literally pre-cracked them, if you will.
We put expansion joints is another way of looking at it.
A typical tile is 6 inches by 6 inches when it gets into the highly heated area.
Now all of a sudden we've got some room for the title to move relative to one another, structure to move underneath it because of the strain isolation pad.
And we said, viola, we've got this problem fixed.
We're good to go.. We said we've got 25,000 of these tiles that we now have to certify that they're good to go for the vehicle.
There were gaps between the tiles?
Correct.
But, when you re-enter, do the tiles expand so that there are no gaps?
No.
What we do is make sure that the gap can just close but not close such that you're loading one tile relative to the other.
That set the gap.
And, if during manufacturing which is not that precise, if the gap was too large we just stuck a gap filler in.
Not the one that was on the last vehicle.
It's a different kind of gap fill.
Just a piece of Nomex or SIP material strain isolation pad coated with rubber or with RTV, and we stuck it in between the tile just to reduce the flow if the gap was too large.
So there is no chance that you would have hot plasma coming in between them?
Yes, there is a chance you can have plasma going through there if the gap is too large.
We had a very specific requirement.
I think it was like 90-thousandths of an inch between tiles.
And if it were, say, 130-thousandths of an inch, as installed because of tolerance buildup and all that kind of stuff, then what we did was stuck a plasma flow preclude, if you will, a gap filler between those to stop the plasma flow from getting between the tiles.
And we had some experience where we lost some gap fillers, plasma flowed between the tiles, caused excessive heating on the tiles and started causing them to slump and melt a little bit, did a little bit of structural deformation but nothing significant.
Losing gap filler was a turnaround issue, not a safety of flight issue.
We made sure of that.
Excellent question.
This is more of a general question, but I'm wondering if since then anybody has done any work to find a material that could accomplish both tasks, the structure and the thermal protection rather than having to deal with these types of issues.
That was back to one of my trades were we said let's take that approach and see if we cannot design a vehicle that can take the temperature and the loads, too, both.
There was no material that existed.
I don't think a material exists today that I know of to be worked to a high enough stress that you can keep the weight down to have those kinds of temperatures and to be worked at a high temperature.
I think on the X-33 design they were talking once again of trying to make it out of a metallic structure.
Part of it was structure and part of it was tons.
But we never got to the point of being able to fly and test it.
But they also were using a thermal isolation system like tiles over part of the vehicle on the X-33.
The Buran, which was the Soviet copy of the Shuttle, they were very anxious to get away from the expense of the tiles.
And they ended up with a thermal protection system not very different than what you said.
You mean they had tiles?
Yeah.
As a matter of fact, I remember they were more like bigger blankets.
That's a good point, Larry.
I was about to skip over it.
I said they were typically 6 inches where they were really thick, but in other areas we made like 12 or 15 inch tiles that were very thin.
And the reason we could do that is we decided if they crack it doesn't matter because the gap wouldn't be large enough for the plasma flow to go through there and it would simply be a self-relieving.
So, we tried to reduce the cost of manufacturing by making some of these tiles larger.
And I will show you that in just a second.
I'm going to skip forward here and show you one thing.. That's 50-thousandths of an inch.
The allowable for the tile was 12 psi.
The ultimate stress was 12 psi.
The allowable stress was about 8 psi.
And that's with the total loads combined on the tile.
What do you have to consider in one of these 25,000 tiles that you have to assure is not going to come off the vehicle?
Remember I said when we looked in the early phase studies of the vehicle we were looking at wing loads and stuff like that?
Then we got into how is the wing going to carry the load into the fuselage to the spar and what was our design trade on that?
Those were sort of macro and semi-macro systems engineering studies.
Now we're into a micro.
Now we've got this little critical part, 6 inches by 6 inches.
Lose any one of about 10,000 and you lose the vehicle.
It's got an 8 psi allowable strength.
We said well, that's no big deal.
What are all the environments on this thing?
We started looking at pre-liftoff, liftoff, ascent, et cetera.
There is always something called mismatch.
When you take something that is not perfectly smooth and you bond something to it and they are not to the same flatness, if you will, there is going to be a stress induced into both parts of those.
Well, the structure doesn't care about that stress but this little 8 psi allowable tile does.
It says we cannot exceed about 19-thousandths of an inch under one of these tiles for this to happen.
There is ignition over pressure during ascent.
There is acoustic and vibration and that kind of stuff that has to be considered.
On ascent there are gradients because of shock moving across the tile.
There is internal pressure.
There is skin friction and drag on the tile, so you've got to consider that.
You have to consider the dynamics, the inertial loads induced in it.
And the out of plane deflection is now the vehicle flying.
The structure is deforming.
So, you've got to consider that.
Now you can see you've got 25,000 tiles with all these combinations of load conditions so how in the hell do you design that?
This is the characterization of it.
This is what happens with a structural deformation of like 10-thousandths of an inch deflection underneath a tile.
With that kind of formation it causes 1 psi.
And it linear almost.
So, 20-thousandths of an inch deflection is going to cause 2 psi.
Well, 2 psi is not a lot but it's a quarter of your allowable.
We said we've got to consider that.
We have to understand what is happening to this structure as you fly it, a condition.
This is a free body, if you will, of the pressure distribution on a tile.
It has internal pressure within the tile.
It has a pressure differential on the outside of the tile because of a shockwave moving across it because of the pressure as it varies around the vehicle.
So, you have to consider all of that.
We did all that and said we've got a problem, because what happened was all of a sudden our allowable on our tile was decreased by half.
And this was not good.
We screwed up, in the systems engineering point of view, when we did this.
Remember we put the strain isolation pad underneath the tile and we did all these other loads on it?
What we forgot, or what we didn't realize was this strain isolation pad had little stiff spots in it because of the way it was stitched to keep the strain isolation pad together.
And every stiff spot acted like a hard point.
Now, when you take a tile and you put external loads on it and there is a stress concentration on each one of those things, that stress concentration had an amplification of two.
All of a sudden our allowable were decreased by a factor of two.
And we thought we have a major problem.
This is when Aaron and I got to know one another better than we ever wanted to.
We met every day on this thing.
We had tiles bonded all over the vehicle.
Necessity was the mother of invention.
What we had to do was dissipate this stress concentration.
We said we'll put a plate underneath it, a graphite/epoxy plate or something like that.
That would have added a lot of weight to the vehicle.
Glen Ecord, a materials guy came up with the idea in the lab one weekend, he took silica powder in water and just painted it on the bottom of the tile.
What it did was fill all the pours in the tile for about three-sixteenths of an inch.
And just compacted themselves in there.
And that was an inherent capability of a tile with that powder packed in it, if you will, that doubled the strength of the tile.
What it did is it dissipated the load from being concentrated into the tile.
So, without any weight, some cost, we had to pull some tiles off the vehicle and densify them.
Now all tiles are densified.
I meant to bring one to show you today but I didn't.
Analytically, we went through 25,000 tiles.
Here was our factor of safety distribution over the entire vehicle.
We said we're good for that, but the thing we don't know is if the tile is really bonded on the vehicle like it should be.
We said we can fix that problem.
We will verify it.
We will pull on every tile and make sure that it has a capability of the maximum expected mechanical load that it is going to see.
And so we pulled on every tile.
And John Yardley, being an old stress guy, says let me ask you a question.
When you pull on every tile, how do you know you haven't induced more damage and decreased the allowable of the title because of your proof test?
Good question.
Easy answer.
We'll put a microphone on every tile and we will characterize the sound of the tile as we pull up on it, and we will get a sound allowable, if you will, for proof-testing the tile.
We did that on every tile that had to be proof-tested.
We had an acoustic emission device, we had a microphone on there, we characterized all these tiles and we did it.
Another little mother's necessity of invention.
And I'm not going to go through this.
You'll get a handout of this.
What this did is said you cannot proof load every tile, every tile is not densified, some tiles are thick, some tiles are thin, but prove under no uncertain conditions that you're safe to fly.
We went through this logic on everything.
Some of these things had gates that said go directly to fly.
Others we had to go through.
Others we had to pull off the vehicle and densify and proof load and do a bunch of other things to prove that they were OK. We got there.
Now we're into operations.
We're just about out of time.
I know.
I've got three minutes left.
Can I have all three?
OK.
Here is what we did.
We did some pretty innovative things on this thing, and I think I've shown you some of that stuff.
But what were the surprises we had on the first flight?
Not very many.
As we were going uphill, I told you about the plume effect from the engines blocking the flow, that causes the pressure distribution on the wing to be different than we designed for.
The center pressure was further aft and outboard.
That loaded up the wing more.
It could not fly to the design conditions that we needed to under worse case.
And I'll tell you what we did on that.
Well, let me just tell you right now.
What we did was almost day-of-launch wind analysis, loads analysis for the wings with load indicators in the wing to understand what a specific flight regime was going to do for that vehicle.
And we literally designed the trajectories for the first six or eight or ten shuttle flights so it stayed within a reduced capability of the vehicle.
Because we characterized it so well, we could do that.
The other thing was the overpressure of the vehicle, when the main engines let off there was an accumulation of hydrogen gas underneath the vehicle.
And as soon as the engines fired it ignited that hydrogen and sent a big shockwave up the vehicle.
It could have been pretty bad but it wasn't, so we fixed that by isolating the isolation and also burning it off prior to engine ignition.
And then we got some tile damage from external tank.
We fixed that.
After about the second or third flight on the external tank, we changed the foam process and controlled that.
And so those were our only surprises.
You remember the chart that I showed you of all the combined environments on a tile?
This is not politically correct what I'm going to say, but I'm going to say it anyway.
The shockwaves, the pressure, the out of plane deformation, all that, you didn't see debris anyplace on that chart.
Those tiles are not designed for debris, period.
You can rationalize it.
You can arm wave it.
You can statistically analyze it.
The vehicle is not designed to fly in debris.
It can take every other thing it's designed for, but it cannot take what it's not designed for.
The engineers today have two choices.
They can either eliminate the debris from the external tank or, in my judgment, they can go back and recertify the tiles for the expected debris.
And that's probably not too big a deal, but they cannot say we're ready to fly, going through a logic matrix like I did, until they do that.
OK, guys and gals.
Here are some challenges for you.
As you go through from the beginning of a program, I would like you to be looking at this crew exploration vehicle, what other parameters would you look at other than what I've shown you here and what tools would you use?
On the combined thermal mechanical, I think a lot can be done on that.
I don't know what all the computing capability is today, but I think that you could really simplify and probably decrease the weight of the vehicle by doing that.
Could they be made more rugged?
Yeah, they probably could.
But be careful.
This is a big systems engineering issue and it's a political issue.
And I'm going to get off the stage here in one second.
Should there be a dedicated crew escape system or should the reliability be built in the vehicle?
And I'll promise you that answer is not known.
It may be known from a political standpoint that you have to have a crew escape system, and that's OK.
If that's a requirement, that's a requirement, but in a total system crew safety reliability that's not obvious.
It depends on what the design is and what the reliability of the constituent elements are.
How would you fix the ET?
Put shrink wrap all over it.
Another thing that's not part of the curriculum here is political systems engineering.
You can have the best engineering design in the world, but if you don't have the political support in a program like this it doesn't matter.
How does political influence come into a systems engineering thing, kind of like the crew escape system?
Thank you.
[APPLAUSE] Well, that was just super.
This is hopefully the first of many lectures which will take us to a much greater depth [then when we did the shuttle system?
].
We talked about and gone through the history of the Shuttle a little bit.
We're going to go into it some more today.
But, to refresh your memory, there really were three subsystems on the shuttle that were pressing the state of the art.
One was the thermal protection system which we talked about, Tom Moser talked about.
The other was the avionic system with the computers, the computer synchronized.
The four computers synchronized because, as we explained, the Orbiter really needs a computer to fly because it is a fly-by-wire system, which is statically unstable.
And the other system that was pressing the state of the art was the Space Shuttle main engine.
It had a high pressure, high temperature and high performance.
And so, you're going to hear about that today.
Now, the person that's going to talk to you -- Many of you, I'm sure, heard the term rocket scientist but you probably don't really know what that means.
Well, today you're going to meet one.
You're going to meet a true rocket scientist.
J.R. Thompson was responsible for the design, development, test and operation of the space shuttle main engine.
During Apollo J.R. had a very similar function in the design and working on the launch vehicle for the Apollo vehicle.
He became director of the Marshall Space Flight Center.
Then he was deputy administrator at NASA in Washington.
And now he's president of Orbital Sciences.
You're in for a real treat.
Thanks, Aaron.
Aaron asked me to consider this talk, I don't know, several months ago.
And I was a little hesitant at first, but then the more I thought about it, it's given me a good opportunity to go back and kind of recount some of the highlights and the low points in the program.
It was some 30 years ago for me.
So, there is some of this that's going to be still a little fuzzy.
But there is a lot of it that it's just like it was yesterday.
Actually, the Shuttle main engine has its roots back in the technology programs that were funded and came out of Apollo.
As early as the mid-1960s, people at the Marshall Space Flight Center in the same propulsion group that I was in, although my attention was focused on Apollo at that time, were heavily involved with Pratt & Whitney, Rocketdyne and Aerojet in developing the high-pressure turbomachinery that would one day be envisioned to use in the Shuttle, if there ever were a Shuttle.
So, that's back when it started.
And there was a good bit of effort put in at that time.
I think, as you probably know, Rocketdyne of North America won the contract in July.
Actually, July 13th of 1971.
So, shortly after Apollo.
Apollo was still winding down when the Shuttle Program got its legs.
And the Shuttle main engine was one of the very early awards because, as Aaron indicated, it was early on envisioned that that would be what we called back then the long tent pole in the program.
It came on the heels of a one-year Phase B competition between Pratt, Rocketdyne and Aerojet.
During this Phase B, NASA funded some technology demonstrations, requirements definition and that kind of thing.
Actually, Rocketdyne, at that time, did a very bold demonstration of what they call power head.
And I will show you what that encompassed in just a minute.
But it was a demonstration of the heart of the engine.
It only operated for a short period of time but very high pressure.
It was risky.
They pulled it off.
It was actually driven by a fellow called Paul Castenholz who was a guy coming out of Apollo that basically solved or lead the team that solved the combustion instability on the F1 engine.
Paul was a very ambitious fellow, very aggressive and very bold in trying to capture this award for Rocketdyne.
At that time, though, NASA envisioned what they called a fly back booster.
The Shuttle main engine was envisioned to be a common engine for the fly back booster, as well as the Orbiter.
In that early concept, there would be 12 SSMEs on the backend of the fly back booster operating at 550,000 pounds of vacuum thrust.
And on the Orbiter configuration it was three engines operating at 632,000 pounds of thrust in vacuum.
Because of the money that was forecasted at that time and that had been appropriated to date, NASA was always behind the power curve in the early phases of the program.
And they never got everything they wanted.
And I'm sure that Dale Myers and others that have preceded me have told you the ins and outs of why then NASA scaled back from the fly back booster to the solid rocket motors.
And so, at that point, the Shuttle main engine was refocused at 470,000 pounds of thrust.
And that was what's called the rated power level.
At that time, I think there was a full power level and an emergency power level of 109% of the rated thrust that was in the program, but I'll show you a view graph in a minute that has the specific parameters of the engine as it finally settled out.
After the award, Pratt & Whitney protested.
It was a hard fought competition.
NASA had specified that they wanted an engine bell configuration, a nozzle bell as opposed to Rocketdyne's aerodynamic spike that they had promoted in the late `60s.
I'm not sure if everybody here knows what an aerospike is.
You may say one or two things about it.
Well, it's a truncated nozzle.
Actually, it's packaged and derives the nozzle from the expansion of the gases here where the nozzle wall is formed by additional gases that come out in the cooling system.
And it's a very high performing engine.
Its packaging is certainly an advantage where you don't need a big boat tail, say like you would with the engine.
You're saving about 10 or 12 feet there and the weight that goes along with it.
Pratt & Whitney had been focusing on a bell nozzle all the time.
And frankly those of us that were kind of on the periphery of this program and still involved in the Apollo kind of figured that Pratt had the advantage in this because Rocketdyne was, of course, awarded the propulsion systems for the Apollo Saturn Program.
And it was kind of viewed as Pratt & Whitney's turn.
It didn't turn out that way.
I attribute a lot of that to the good proposal, the boldness and the demonstration program that Rocketdyne accomplished during the competitive period.
Anyway, Pratt & Whitney protested.
It took about nine months for the protest to be settled.
It was settled in favor of Rocketdyne.
In the meantime, Rocketdyne was allowed to continue to work with the contractors on the vehicle side because they hadn't made the selection at that time there.
So they could continue to support their work.
Anyway, in 1972, it was all settled.
The contract was awarded cost plus.
And, as I recall, it was $200 million for the development which was called Phase A, and $200 million for the production which was called Phase B.
And the Phase B program included, I believe, 26 production engines.
I won't comment as to what the costs eventually grew to, but very substantially beyond that.
Let me kind of highlight for you the characteristics of the engine.
I think I mentioned the thrust levels.
The rated power level was 470,000 pounds.
It had the capability, if an engine was out early and you wanted to abort the orbit to throttle the engine up to 109% of the rated thrust that was called full power level.
Early on in the program, I think it was termed emergency power level.
But, at the rated conditions, it accommodated or required a little over 3,000 pounds per square inch chamber pressure in the combustion chamber.
The area ratio of the nozzle was 77:1.
It had a very good specific impulse.
About 453, 454.
That compares to J2 in Apollo of about 442, as I recall, somewhere in that range.
Weight was about 7,000 pounds.
Life 7.5 hours and 55 missions.
That's quite misleading, though, let me tell you.
And I'll comment more on this as we go through.
But it was a very tough development program.
It took from '72, as I mentioned, through first flight in 1981.
I joined the program after Apollo and after Skylab in May of 1974.
And it was torture from there until the first flight in April of '81.
Early on it was envisioned that, as I mentioned, the SSME would be used for both the fly back booster and the Orbiter configuration.
It would use basically the same power head.
You would just change out the nozzles to give you the two engine thrust levels.
And so it was rather simple in that concept.
Of course, that's not the way it worked out.
It went to one configuration to service the Orbiter.
And so everything was optimized and focused on that.
Now let me say a few words about the schematic itself and what the engine looked like.
In conceptual terms, it had two low pressure pumps which were required to give the proper inlet pressures to both high pressure pumps to avoid cavitation.
It had the two high pressure pumps.
It was all fed through a common power head to a thrust chamber assembly and then into the nozzle.
Starting on the fuel side, the fuel pump increased the pressure to about 300 pounds per square inch.
And then that went into the high pressure pump on the fuel side where the pressure was boosted to a little over 6,000 pounds per square inch.
About 80% of the fuel all went to the two pre-burners, the fuel and the oxidizer pre-burner contrasted to about 12% of the oxidizer.
That was to provide a very fuel rich power system to drive the two turbines, the high pressure fuel pump turbine and the LOX pump turbine.
The turbine temperatures were in the range of 1750 degrees air ranking.
Almost all of the housing structure on probably 80% of the engine was Inconel, very tough steel.
Able to take very high pressures.
As I mentioned, the combustion pressure in the chamber was about 3,000 pounds per square inch.
At the entry to the pre-burners, the pressure could get up to 8,000 pounds per square inch.
It was a very high pressure system up and down.
It was all in series.
In other words, this was the way they achieved the high efficiency.
All of the propellant came through the low pressure system, the high pressure system into the pre-burners, in through the cooling circuits, all in the hot gas manifold, all in the main chamber, all of it exited the nozzle.
None was dumped overboard to simplify the flow pad.
A good bit of the fuel at the exit of the high pressure pump went directly to feed the two pre-burners.
A good bit of it, about 20% of it went to cool the nozzle, then up through and cool the -- Rather drove the turbine because, having cooled the nozzle, it was converted to a warm gas which then drove the low pressure turbine, came back in and was captured and served to cool the shell of the hot gas manifold.
The same similar thing on the oxidizer side.
By the way, I'll mention the speed of the low pressure fuel pump was about 15,000 rpm and high pressure fuel pump about 35,000 rpm.
The speed of the low pressure LOX pump was 5,000 rpm and of the high pressure oxidizer pump about 30,000 rpm.
The discharge of the low pressure LOX pump was about 250 pounds per square inch entered into the main LOX pump where the pressure was elevated to 4500 pounds per square inch.
Then it went some to the pre-burners and the rest directly into the main combustion chamber.
Some part of the lock flow then was further boosted to about a little over 8,000 pounds per square inch, which fed the oxidizer into the two pre-burners.
It was a very efficient cycle, very high pressured cycle.
A couple other features that you don't see on this chart, there was a heat exchanger wrapped around the high pressure oxidizer turbo pump turbine which served to preheat gas to pressurize the oxidizer tank.
I think those are the major points that I would make on the cycle itself.
Most of the problems, I'll just point out here, had very few problems with the two low pressure pumps.
A lot of technology in the high pressure pumps, both fuel and oxidizer.
The main problem with the high pressure fuel pump was sub-synchronous whirl, and I'll say a little bit more about this in a moment.
It was a very traumatic time in the early period of developing the Shuttle main engine.
It caused a lot of delays.
A tough problem to solve.
And I'll mention what caused it and how we solved it.
Then the high pressure oxidizer turbopump bearing overloads, LOX fires, explosions.
That was probably the single toughest component to developing the program as I recall it.
I think those are the major points that I'll make there.
Early on it was planned, because they had such a head start in planning for the development of this program, to be done in a very methodical way.
And they had, when I say they I'm talking about the people at Marshall with the people at Rocketdyne, a very elaborate system of design verification systems where you couldn't progress to the next stage until you had passed certain testing on a valve or on a low pressure pump.
And all this was envisioned to be done at Santa Suzanna which is right out a short distance from Canova Park up in the mountain on some test facilities that both NASA and Rocketdyne owned up at that time and that they were carried over and upgraded from the Apollo program to be done on the component facilities up there.
But because of the high pressures involved it became a very expensive undertaking, a lot of money, too much time, huge facilities, valves that were probably as big as that side wall over here, to be able to handle these high pressures to assist in the development of this high pressure turbomachinery.
I think it became pretty clear, pretty early in the program, that that very methodical way of developing the engine, and that is, before we go to an engine system test, let's develop it at the component level so we eliminate those problems.
Although, theoretically it sounded very good.
It just didn't work.
The facilities were not available in time to do that.
And the money was going to be exorbitant Kind of a side story.
There was a contractor Bovey Crail as I recall was the name of the contractor out in Los Angeles.
And early in the program when they were having contract discussions and Bovey Crail was late and Rocketdyne, at the instance of NASA, was withholding money until they made certain progress.
And so this fellow who was the president of Rocketdyne, Bill Brennan, I was spending a lot of time out there, and he invited me to sit in this meeting that he was going to have with the head of Bovey Crail.
I guess he thought maybe it would impress him that somebody from NASA was watching this thing.
Anyway, so I joined him.
I think it was a Saturday morning.
And both Bill and I had on suits, coats and ties.
And this contractor or head guy from Bovey Crail came in with some white bucks.
He had on some yellow pants and a pink shirt.
And he plopped down on Bill's sofa there.
And, before Bill could open his mouth, he said all I want is my blankety blank money.
And so it was a short discussion.
[LAUGHTER] He was there to collect his money and Brennan couldn't pay him because of some of the restrictions that NASA had.
So, it was a messy deal trying to get those facilities built.
And it became clear that we were going to have to develop the components in parallel with the engine test.
It was the biggest systems engineering challenge that I think we had in the shuttle program.
Certainly, there were a lot of challenges in major systems integrating the shuttle engine into the Orbiter, for example.
But down at the engine level, the systems engineering, to develop those components, the two low pressure pumps, the two high pressure pumps, the pre-burners, the hot gas manifold, the main injector, all the control valves.
And then we also had on the engine, which I didn't show on the schematic, a computer, redundant computers that were cross-strapped so that the input or the output could be cross-strapped and be very tolerant to failure.
We went to what we call an integrated subsystem test bed, ISTB.
That became the bobtail engine of the program.
Bobtail was a 35:1 area ratio nozzle which allows us to not only start the engine but to operate it at the throttle condition 50% of the rated thrust and the nozzle would still flow full.
It would not separate.
And so then we could proceed with the development of the testing or the development of the engine program.
It was an efficient way to go about the program so we didn't stall.
It was tough because we had to solve the engine problems in parallel with the component problems all at the same time.
And sometimes it was hard to tell which was the problem.
We started our first test in May of 1975.
That is me there on the left, and Norm Reuel who replaced Paul Castenholz they're in the center, and Dom Sanchini who I view to this day as the strength of the Shuttle Program, or rather the main engine who eventually replaced Norm.
A few comments on the evolution of some of the people on the contractor side.
I mentioned Paul Castenholz who to me was the key to Rocketdyne winning the program.
At that time, and this is a personal view, I think Rocketdyne took a big sigh of relief after they won the contract, probably relaxed a little too long, got in trouble up at Santa Susanna in not developing that component facility.
The test control centers up there, Coke bottles were laying around, so it was not a well-disciplined operation.
They had gotten lax, I guess, was the best way to say it.
Norm Reuel came in to run the program at the request of NASA.
He had done the same job on a J2 and other programs within Rocketdyne for the Saturn Program and worked with me and the other fellows at Marshall.
And we appointed Dom to drive this ISTB to help us learn how to start the engine to how we could properly integrate a number of the components into the engine and proved in the long-run to be a very valuable tool.
This chart depicts the buildup of run time on the engine.
Test seconds are plotted on the right in thousands, and the number of tests plotted on the left and the year is shown down here.
We started, as I mentioned, in 1975.
We finally flew in April of 1981 on STS-1.
The title, someone asked me a little earlier, that's first manned orbital flight.
That's what we referred to our goal back at that time.
I've highlighted along the way, I believe, 14 major engine explosions all of which were very traumatic in themselves.
I'll show you some pictures shortly of what an engine looks like after it goes through that.
And then you can extrapolate that to envision what was in the boat tail of the Orbiter had it occurred in flight and what it would have done to the flight itself.
The test seconds curve is down here.
You can see the long plateau of about nine months where a combination of learning to properly ignite the engine without over-temping the turbine blades or other parts of the turbine combined with what I'll call this sub-synchronous whirl on the high pressure fuel pump.
Sub-synchronous whirl, there is a very exotic definition of it, but it's an orbiting of the shaft within the bearings themselves caused by a softening of that system.
And you can imagine the softening is attributed to overheating of the bearings.
You don't have the stiffness.
And so this allows the rotor to orbit there, vibrations get very high and we have to shut the engine down.
We couldn't get into the test, but about 2.35 seconds was the nominal time before we would encounter this very high vibration and then have to shut the engine down.
And then after that the engine and the turbomachinery were located down in Mississippi where we were doing the testing.
It wasn't like you could just try it again.
You had to bring it all back, replace the bearings, try to figure out what the problem was and put in some kind of a fix.
We went through a number of fixes trying to stiffen the system so we could be able to tolerate and drive through.
I thought at that time that if we could ever get through this period then we would be all right.
But, as it turned out [NOISE OBSCURES].
Well, it manifested itself in overheating of the bearings, spalling of the bearings in their raceways.
I think turbine speeds at that time, or the speed of the rotor was probably in the 15,000 rpm before you could catch the engine and shut it down operating for a second or a half a second with those high side loads, inadequate cooling, overheating of the bearings.
You'd get the bearings back and you'd put them in your hand and they were very much damaged.
I mean that was the manifestation of the problem.
We went into this for quite a period of time.
I think what I wanted to capture was the picture over here.
Ignore this.
This is more dramatic.
I will come back to that later.
But this is a high pressure fuel turbopump.
The sub-synchronous whirl and the overheating of the bearings occurred on the turbine side.
The coolant path is probably going to be hard for you to see, but you come through some labyrinth seals here.
You go down this passage, up through the center of the shaft and in through some provisions that were made to cool the bearings on a turbine end.
We brought in a lot of external consultants, had a lot of cooperation from people across the country, but after nine months it was one of the internal guys at Rocketdyne, Joe Stangeland, and some of the people in his turbomachinery group that came up with the idea that there was a vortex that was occurring in this cavity on a turbine end.
And what we had to do was to kill that vortex and then allow the coolant to go through.
And so he put a little what he called a paddle, which is this part you see right here, which is about the size of dime that was screwed into here to the rotor at that point.
And the first task, after we included the paddle right off the block where the engine had been stalled and couldn't get beyond about 2.35 seconds, went right up to the minimum power level, which is the power level we had set and planned all of the tests to accelerate to that level if we could make it.
And so it was a very dramatic solution, fix, and allowed the program to move on and see what was behind the next hurtle we were going to run into.
So this problem number one was just being able to understand and accommodate the start sequence.
You had to start the engine fuel rich.
Any excessive oxidizer would give you a cutoff because of all the sensors that we had.
The second problem and the one that causes the most time that I've mentioned is the high pressure fuel turbopump sub-synchronous whirl.
And then problem number three, which are duplicated here several times, are the LOX pump explosions.
They could be triggered by loss of a turbine blade on a turbine end which would unbalance the rotor, overload the bearings and then cause a LOX rich fire which very quickly consumed the whole engine.
There were numerous, well, I think three, rather four highlighted there.
We had some other problems.
I've noted a fuel pre-burner burn through, just a structural burn through of the [OVERLAPPING VOICES].
How did you eventually solve that, because the last failure you had was also the liquid oxygen explosion?
Well, I'm not sure I can differentiate between the LOX pump explosions.
But late in the program, probably at that time, we had been having for a long time problems with the turbine blades, very limited life.
Cracks would grow.
At that time, we didn't really understand how long we could run them before we had to replace them so we went to dampers on the turbine blades.
And this occurred fairly late in the program that eventually solved the vibration of the blades within the turbine wheel stack and allowed us to proceed.
Whether that was that one that caused the imbalance of the rotor or some earlier when they were primarily driven by bearing problems considered for some time.
We used ball bearings in the Rocketdyne turbomachinery.
Later in the program they had gone to roller bearings.
Pratt & Whitney has been contracted to develop that.
But the bearing problems and turbine blades were the major problem, as I recall, that caused the LOX pump problem.
As a matter of fact, yes.
Do you think you would have had similar problems if you had developed an aerospike instead?
The turbo machinery was going to have to be basically the same.
And so, no, I think you'd have the same problems.
The only problems that we had on the engine, we had a nozzle steer horn failure.
That was just a structural feed line that was in the shape of a steer horn at the aft end of the bell nozzle.
We had two of those structural failures that were very traumatic for the program.
No, they were both right here.
And, of course, as soon as you lose the coolant to the nozzle, you start shutting down a lot of the engine system LOX rich which causes a fire.
Certainly, you would have eliminated those.
You would have eliminated a number of I'll call them nuisance problems in terms of we had 1,080 tubes that made up the nozzle that we flowed the hydrogen through to keep the nozzle cool.
1,080 of them.
And we had a number of, again, I would call them nuisance cracks or splits in those tubes that we learned to live with.
We learned to go out and put a cap on them post-flight, braise over and just cap off the leak.
Certainly, you wouldn't have had those.
But all of the other problems I would have seen come in between the aerospike as well as the bell nozzle.
J.R., the shuttle main engine had the pressures that you mentioned, the 3,000 pounds per square inch up to 8,000.
Give us some feeling for just how much higher that was than the previous operational engine and whether the pressure itself was causing a lot of those problems.
Oh, yes, the J2, chamber pressure was about 700 pounds per square inch, as I recall it.
RL10, which was the first LOX/hydrogen engine in the country developed by Pratt & Whitney, chamber pressure was a couple hundred pounds per square inch.
This was a real push.
Was it worth it?
As you look back on the engine, you've now taken into orbit over 300 of these engines and 100 flights three at a time.
You have had one shut down because of a safety sensor that failed and shut one engine down in flight.
And we were far enough along in the flight so we aborted to orbit and the other two burned a little longer and went to orbit.
But it was a very costly program.
On some reflection, it was almost viewed in the late `60s as that's the challenge.
Not the shuttle engine but the technology.
I mean let's really drive the technology and make it very efficient, very high pressure.
Aaron and I were talking a little earlier.
The engine people, had the Orbiter flown a 100 times and had several major failures, people would have expected the Shuttle engine, I think, to have been the cause.
That's not the way it turned out.
And I'll comment on at least a contributor in just a minute.
But this is the aftermath.
When you're in the middle of a development program, you've got a lot of budget pressures, you've got the Shuttle's first flight date changed several times, you've got all that hanging over your head and then you're called down on Mississippi on a test and you look down on the engine and that's what it looks like, it's like a kick in the stomach.
You've got to start all over.
Well, that mental picture was in our minds 14 times in this program where you had to start over.
The recovery was typically it took about a month.
We formed a team, usually within Rocketdyne and one within NASA, to go solve the problem.
Probably half of the time it was fairly evident what the problem was quickly after the test.
Sometimes it took a couple of weeks to narrow it down it could have been this or that.
And then you fix both this or that not knowing exactly what it was.
And so it was a very tedious and time-consuming.
We were testing around the clock.
If it occurred during the holidays you just cancelled your holidays and jumped in, and we did what we had to do.
I will mention, as I look back, kind of one of the dumbest things that I did on this program was somewhat associated with the testing.
This happened to be, I think back early on in the sub-synchronous whirl days, we were changing out the turbo machinery after almost every test because we were doing the damage to the bearings that I described a little earlier.
And it took about three shifts to change out a turbopump down in Mississippi.
And then it took flight time to get the turbopump back to LA and then tear it down.
We had others in the meantime that we were bringing along, but it was very time-consuming.
And it was in the middle of the summer back in whenever it was, '75 or whenever, and a lot of rain showers.
And that was holding us up because when it was raining or it looked like it was going to rain, you couldn't open up the engine, drop the pump and take it back.
So, that slowed us down.
And so I had a brilliant idea of first of all, the engine, if any of you have ever been down to Mississippi on those test stands, it's on about the fifth level is the engine position, about the fifth level.
And so on about the seventh level I wanted to put here, what I want to call, a rain shield.
Put a tin roof a couple of levels higher than the engine so the workers wouldn't have to stop when it was raining.
Well, you can probably imagine what happened.
That was OK for a while.
And then we were back into a test and we had a hydrogen leak.
And we weren't using the igniters at that time at the end of the bell to burn off the leak.
So that leak just accumulated on those two stories up to that rain shield.
And then, when we went into the test and lit the engine off, the whole Mississippi sky, you know, there wasn't any fuel.
It burned all the wires.
It didn't damage the end canal on the engine structure.
I certainly got a lot of ribbing after that.
And, as I recall, the test was toward the evening.
I wasn't down there at the time to see it, but was listening to the test over the phone.
And the guys almost couldn't speak.
I mean the whole place went up.
These kinds of failures -- This was not evidence of a little fuel fire.
This was oxidizer and the fuel was the metal.
And so it was quite a problem for us.
I might mention at this time that just this last summer I attended a little event there in Huntsville where the Rocketdyne team came down.
And now the engine program had just passed its millionth test second mark.
And so you can see, for STS-1, the total test seconds were about 110,000.
A little over half of that, 65,000, almost 70,000 at the rated conditions, the conditions you flew it at.
And so it's gone up by about a factor of ten in the meantime.
Another key to the Shuttle engine program was the philosophy imparted by John Yardley.
I think he recognized early on that we had bitten off probably a little bit more than we could chew to have an engine that you could solve the problems of where you put the bearings in there.
The earlier view graph I had said 55 starts.
You ran them for five starts and they came out in pristine condition.
That wasn't going to happen.
And the turbine blades, one blade failure, I actually forget how many are in those two stages on that wheel, probably well over a hundred, but one would offset or offload the balance on that rotor.
Then you would overload the bearing going at the high speed of the 35,000 rpm's and would fail it.
The turbine blades, you just could not tolerate a failure.
But in evidence after testing, where you can tear the blades down and de-stack them and look at them under a microscope, you could see fatigue cracks.
And so all of us, I think, but John provided the leadership, recognized this fact.
And if the requirement were going to be that you wouldn't fly with turbine blade cracks, we weren't going to fly.
And so he encouraged us, in that instance, in bearings and probably other areas to test to failure, drive it to failure, know where the cliff was and then back off a sufficient amount.
And then conduct your certification with cracked blades, with spalled bearings so that it was clear to everybody, it was clear to the people that were running the program and others that had to fly that the problem was understood, we thought reasonably well understood, and it was tested to accommodate that condition.
And so that was the philosophy that was engrained in the Shuttle.
And probably not enough in some of the other areas.
The Shuttle Engine Program was a little bit blessed with this fear of failure.
It's the toughest technical problem so we got the most money.
I mean it's not all a downside.
So, we got the money to provide the test depth that provided the insight to the Shuttle managers that could make the call as to when we were ready to go.
The squeaky wheel gets oiled.
I contrast that with maybe the famous O rings on the booster.
That was a problem that was kind of observed, a few tests were run, but not enough, you know, not to failure.
It wasn't ever tested to failure on the ground.
It was tested so you could see that the O rings were split maybe but not to be staring at a picture like that as to what is going to happen when that O ring gives and very quickly can burn through.
And the other example I would make is a tank.
And this is the most frustrating to me because early on we all saw some of that foam come off, but not in the sized pieces that the Shuttle Program saw three flights before Columbia.
I mean that was a piece of foam about this size.
It came straight down the vehicle.
It made a small dent in the aft skirt of the solid rocket boosters that were subsequently recovered.
But that should have been an eye-opener that that piece of foam doesn't always have to go straight down the side of that vehicle.
It could get out in the slip stream and hit a wing or something.
That type of testing was never done.
Whereas, on the engine program, there were a number of flaws that were accommodated and we felt comfortable with.
Now, you go and read the Challenger report, you read the Columbia report, and they will almost tell you that NASA became comfortable with flaws.
And that lead to the problem.
But I take issue with that.
I viewed it different because that was almost the foundation of the Shuttle Engine Program.
It was built on flaws.
It was tested so wherever the soft spots were you knew it and you attacked it and tested it in the appropriate way.
John Yardley, I keep coming back to him, as a part of the certification program of which I think prior to the first Orbiter flight there were eight certifications completed.
One of those certifications was to be conducted.
And a certification, as I recall, was about 13 tests.
One was abort to orbit which was 623 seconds.
And a nominal shuttle test or mission is 520 seconds, I think.
And then an RTLS, return to launch site is 820 something seconds.
It had a mix of those in there.
The engine had some FPL, full power level.
It was tested at all of the limits, but in some of those certifications we had to go into the test with cracked turbine blades.
We started the test with a known crack.
And we knew, by analysis, what its growth rate was that we had judged from other tests.
And so we planted blades that did that.
We did the same thing with bearings.
One time both Dom Sanchini and I were trying to get the most on every test.
We put cracked blades and spalled balls and combined them all in a test.
Now, Yardley didn't mean that.
He didn't want to go that far.
But there was a lot of NASA leadership that bucked up the back of the program manager's, both myself and Sanchini.
I mean they knew the way to develop the confidence in this engine was to test it.
And that became the theme.
That isn't true today.
And that's what I worry about.
And I can come back and comment a little bit more on that in a minute.
I will say a few more words about Dom Sanchini.
He passed away about 15 years ago, probably in his early sixties, was the deputy program manager on the F1 engine to Paul Castenholz, a good engineer.
He was a lawyer by trade but had also accumulated a good background well steeped in engineering.
He was a hard driver.
He thrived on failures because he saw a failure as that's when you're on the steepest part of the learning curve.
You never learn more than in the aftermath of a failure where you're forced to go through and look at all of the data and postulate a lot of other different failure modes.
He thrived on it.
And he made sure that the whole Rocketdyne team viewed it in that light so he was a real strength to the program.
I want to allow a little bit of time for some discussion.
A few minutes ago I alluded to the fact, but that's not the case now.
The Shuttle engine, back in the hay-day or the time period when we were in the development program, we were building them around the clock out of Canova Park three shifts out on a manufacturing floor probably at a rate of about one a month, ten a year, that's about right.
So, a pretty high production right there.
I could probably ask for a show of hands, do you know how much a shuttle engine costs, you know, just to make one?
Do you have the foggiest idea?
My last count was about $40 million.
I imagine in today's money it's probably closer to $60 million a copy.
But today they build about three-quarters of an engine a year so the production rate is way down.
They test, aside from a lot of the terrible consequence of the two hurricanes down at Slidell, you know, they're not testing anything right now.
But, even prior to this time, the testing was very infrequent.
That's to save money.
You don't get that much out of it.
And that's the Achilles' heel, I view, of the Shuttle Program.
If you're not going to do it right, whether you cannot afford to do it right or you have other ambitions, you want to do something else in the Space Program or within NASA, you probably ought to stop it.
Because it's going to be the next failure if you don't treat it right.
Today you've got the Shuttle Program, back when the engine started.
It is a little over almost 3.5 decades old.
That's at a minimum two generational changes and a lot of small businesses that support the Shuttle Program.
And so you're going to lose a lot of the knowledge when you have the turnover of these generational changes.
And little things, I'll tell you, when you look back.
One of the major disappointments to me or traumatic times in the program was back in the late `70s when we were building the first three engines that would power Columbia on STS-1, when we were building those engines we had a mix-up of well wire in the Canoga plan.
And the mix-up was the well wire was, you know, one could take a heat treat and the other one couldn't take a heat treat.
I forget the application where you wanted the non-heat treatable well wire, but that was a mom and pop operation that came in, got the well wire, took it home and cleaned it and then delivered it back in baskets to Rocketdyne.
Well, they got it mixed up.
And so we built the first three flight engines with well wire that would not take a heat treat that we depended on.
You go out and look at those engines with inches and inches and rows and rows of wells that are not to the proper heat treat.
We had to go through and analyze every section and over test those engines at a little higher pressure than we normally would have to be able to show that having been built with the wrong well wire it was still good enough because the design was up to FPL and a little beyond.
That it could still take it and operate at the rated thrust condition.
It is mistakes like that, that without frequent manufacturing exposure, without frequent test use you're going to lose in a program and will become the next -- You know, the program has not been surprised by a problem yet.
I'm talking about all the shuttles.
The O ring failures, they winked early on, To Thiokol and then to Marshall before Challenger.
And the foam problem has been winking all the time.
And it got worse later.
I suppose it has something to do with the change they made in the insulation later on.
But, as you know and you read, the agency is at a very critical time today.
They've got to make some decisions in terms of where their priorities are and what they want to try to do.
And they don't want to spend money on the shuttle because they seem to be committed to replacing it.
And if that's what you're going to do, that's what you're going to do.
But between now and 2010, if that's the year they choose to retire the Shuttle, they are not going to be motivated to test.
The program is going to be at more risk over the next four years than it was in the first four years because of that reason.
You don't know where the ledges are, where the cliffs are.
And testing once a month or once every two months is just not going to do it.
You said something that I really never recognized.
You said there could have been a point of time in the program where you could have gone back and reduced the pressure levels and that type of thing to make the engine a little simpler to certify.
What effect would that have had?
Well, an easy way to do it, we looked at probably several years before the first flight.
We had seen enough, at that time, to know that even with the philosophy that we had adopted, we were on fairly shaky ground.
I would never have guessed that the Shuttle would have flown over a hundred times and never had a failure of the Shuttle main engine or some problem.
I would have never guessed that having looked at what I looked at.
We were starting to think.
And an easy way to do it is open up the throat on a main combustion chamber.
If you open up the throat you relax the pressures up and down the turbine system, the pre-burners, the pump requirements.
You wouldn't have to open it up much.
You'd lose a little specific impulse.
You'd gain a little in thrust.
And so you would get some offset there.
You may want to adjust the mixture ratio overall you operate at.
Instead of 6.0 maybe 6.2.
You could have done it with a modest performance here.
You're not going to get it for free, but you could have relaxed the pressures throughout that whole system by a change in one component.
And we were contemplating that and the monies never came.
It was a configuration change.
There is a lot of the -- What don't you know about that change is what would bother some people.
I don't want to oversell it as simple, but conceptually it was a simple change.
That would be one.
I think looking back and knowing the capability of the shuttle and, to be honest, being somewhat troubled, quite a bit troubled by what I hear and read about NASA wanting to retire it.
Although, I understand their reasoning.
Some view it as a flawed design.
I don't view it as a flawed design.
I think you need to relook at how you operate it.
I hadn't been particularly pleased with the operation of the Shuttle.
Watching all this foam fall off and not even raising your hand.
But then, after all that occurred, nobody stopped and stripped all that foam off and maybe replaced it with cork that's heavier, that's got some tensile strength that you can anchor it on there with glue or whatever you want to do.
And it's going to cost you some performance.
NASA has been biased too much.
And they certainly started this in the Shuttle main engine in the direction of performance where if you back off a little bit the system will be a lot more robust, serve you a lot better over the long haul, and maybe you don't push too much in some of these directions that have got us in trouble.
Do you think that we had a fictitious requirement with the performance we were trying to achieve with the Shuttle?
I don't think we needed it all.
Tell me the missions that we needed all that performance on.
It's not there.
You'd have to reset the manifest in some cases.
Well, speaking for the one element that I know the most about, the Shuttle engine.
If you folks were to undertake the job, OK, what would I do different?
They're talking about using a Shuttle engine on this heavy lift launch vehicle for a cryogenic upper stage engine.
I would open that throat up and look hard at that.
Rocketdyne knows how to do that.
They've done it.
We just haven't incorporated it.
Although, men aren't envisioned to be on that.
You're still going to be a lot better off.
I'll tell you a problem you're going to have with the vacuum start or at upper stage start is that start sequence.
This start sequence is very, very sensitive, and you're probably going to have to go to Tullahoma to get in some kind of a vacuum system to demonstrate that.
That's going to be expensive in itself.
But, yes, Aaron, I think probably across the board, we set the bar higher than we really needed to.
And it cost us in money.
And it has also cost us in margin that we don't have that we wish we had.
J.R., usually around this time we take about a two-minute break.
OK. Give your voice a rest.
Everybody stand up, turn around and just stretch a bit.
Two hours is a long time to go without a stretch.
I'll just give one statistic that I remember really impressed me during our initial astronaut training when they were talking about the main engine.
And you talked a lot about the high pressure turbopump.
This is a device that is about the size of a typical automobile engine, right?
That's right.
It produces 50,000 horsepower just to pump the liquid oxygen at high pressure.
I mean that really gets your attention.
When you talk about pushing the state of the art and trying to get a lot of power out of a small volume that just really amazed me.
As was indicated, I think I would like to just make a couple more remarks and then maybe open it up for some discussion in terms of points that I haven't covered that are on your mind or other questions that you'd like to ask.
Just kind of in summary, if I look back on the program, I think there were two main keys that were paramount to the success that the Shuttle engine enjoyed, and is enjoying through its track record today.
One was a decision to use this ISTB, to get away from the serial component test and then the systems test to try to combine it and do a systems engineering job on a thing from day one.
In other words, that was key to me.
It might have been a little extra pain but saved a lot of time and a lot of money.
And I'm not sure we could have done it the other way anyway.
And the other one which may be a little bit more important, but just as important, was the philosophy in the program of test to failure.
Know where the failures are.
And certainly, if you look at that earlier chart, we had plenty of data points.
I mean we had encountered a number of them, some of them three or four times.
Maybe in some cases we were slow learners, but there wasn't just one problem with that high pressure LOX turbopump.
I think I'd be remiss if I didn't acknowledge some of the people as viewed from the engine that made major contributions to the Shuttle.
Not just the engine now but to Shuttle.
And in Rocketdyne, I mentioned Dom Sanchini a couple of time.
Bob Biggs, who has been there since day one and has done all of the test planning and is key.
He is the systems engineer on the SSME.
Byron Wood, who is now the president at Rocketdyne.
Joe Stangeland who was in charge of turbomachinery who designed this little paddle and solved this vortex problem that uncoupled us from this terrible time that we were at a standstill.
Matt Eck who was in charge of turbomachinery at Rocketdyne before he passed on several years ago.
And then, within NASA, I had mentioned John Yardley.
He has now passed on as well.
I don't think there would be a Shuttle without John's leadership.
Bob Thompson who you might get an opportunity to hear a little bit later, I think he was a driving force in the Shuttle.
Bob Lindstrom, who was my boss for some time.
Chris Kraft, who I understand you will hear or have heard.
Certainly Aaron, we had to interface with the Orbiter and he had an equal challenge.
Arnie Aldridge who I've always thought a lot of, as well as Dick Coors.
And then I'll mention Bill Lucas who was the director at Marshall and George Hardy, both of whom got caught up in a later controversy on Challenger.
Not necessarily fair, but that's life.
I mean they were both superb engineers and meant a lot.
And then also Gene Covert from MIT who, how shall I say this, came in periodically and chastised us appropriately and I think made a major contribution to the program.
At the time Gene was also head of the aero-astro department here at MIT.
I'm not sure who triggered bringing Gene in, but I know John Yardley was a key driver in that.
And so I'm not sure I've touched on the kind of things you'd be interested in, but I'll certainly be glad now to try to answer areas that perhaps I haven't hit or maybe make a few broader comments.
I understand the only element, other than the Orbiter and maybe the overall system, but this is the only propulsion element that is talked about, I think the tank, I've already mentioned there the insulation was their tough nut.
And I think having had some data there are certainly some things that we could have done differently.
The solid rocket boosters, there's quite a history there.
Thiokol has now been acquired by ATK in consolidation of the solid rocket motor industry, but they've also made major improvements in the way they manufacture that.
Those solid rocket motors are also made with glue.
That's put on by hand.
I remember, in visiting up there, after the Challenger accident back when I was a director at Marshall, a number of changes were made to automate a lot of that.
I think probably the solid rocket boosters, when properly used and appropriately tested and backed up by the right analysis, is an excellent propulsion system.
ISP is down some but they are more than adequate to do the job.
J.R., something that really strikes me as complicated and didn't really seem, or at least I don't think we seemed to have that much problem was integrating the engine to the aft end of the Orbiter.
That was a complicated system, but it did go pretty well.
Was it the main propulsion test article that did that?
Well, yes.
It was earlier when we went to school a lot on the integration of the J2 into the S4B stage and the S2 where the conditioning of the engine, prior to engine start, was important and was a big interface with the stage itself.
And certainly that was true with the boat tail of the Orbiter.
And the main propulsion test article, which Aaron mentioned, I guess we had a dozen tests done there, somewhere between a half dozen and a dozen.
As a matter of fact, we had a structural failure of a main fuel valve crack down in Mississippi in that test article.
And it caused some consternation along the way.
But the integration, as Aaron mentioned, went quite well.
I don't think in flight we have ever had any problems with over-pressurizing that boat tail or any problems with it.
There are a number of tests that we didn't conduct, or a few that we didn't conduct on the engine that I would have liked to, A, for curiosity and then, B, to fill a square so we knew exactly what would happen if we ever got in that condition.
And that was with a LOX depletion.
There is some thought that a LOX depletion is going to allow you to imbalance the rotor of the LOX pumps and they're going to rub, cause a fire and all that.
On the other hand, it is, by definition, going to be a fuel rich shutdown, I would tend to think.
But it was one test that we debated quite a bit about in the Shuttle Program and decided that the probability of getting in that situation probably didn't merit the test.
I will add that there was a lot of good tension within the Shuttle Program up and down at all levels.
I thought the whole management team was relatively cohesive, but there was good tension.
I mean we got the best out of everybody and arrived at the best answer by the balance that we had between the institutional managers.
I mentioned the contribution I think that Chris Kraft made and Bill Lucas.
I'd also add early on Rocco Petrone.
And then the program managers.
The John Yardleys and Bob Thompsons and Arnie Aldridges.
And then the so-called level three, which Aaron and I were.
I guess you were two.
I think it was a well-balanced management structure.
And I'm not sure you have that today.
Today, for whatever reason, the program has gotten into an operational mindset.
Center directors are more viewed to keep the grass cut at the centers and that kind of thing.
You don't have that tension.
Who is holding the program people in check?
I see that missing.
I always felt that there was another institutional side that had my hands cuffed at the right time.
And so I think, as NASA goes forward, and maybe some of you folks that are going to have careers in the industry, you ought to make sure that, A, you're surrounded by good people and that you're surrounded by a good system, I mean a good system that pulls you up at the right time or calls you in-check at the right time whatever you do.
Whether it's going back to the moon or mars or, maybe in retrospect, flying a shuttle a little bit more.
Here, let me slow down and stop a minute now and see if there are any other areas that you all would like to cover.
Yes.
We've talked in the past a lot about the slow kind of turnaround on the Shuttle, and part of that has been attributed to having to remove the engines and I'm not sure if it's overhauling them but basically taking them partly apart and looking at them and examining them.
And I was wondering if that was part of the original plan or what kind of changed the efficiency of that turnaround time, what caused it to change and slow down in the original plan.
Well, the original plan was for 55 missions on an engine.
But that just didn't materialize at all once we saw what we had.
I don't know what today's life on a set of bearings are, but it's a handful of missions.
You've got to tear down both of the high pressure pumps and replace a bearing.
The same on turbine blades.
You want to replace that stack after a few missions that are defined in a current certification program.
It's certainly less than a half a dozen.
And, yes, that has built in the time.
But you could overcome that by just dropping the whole engine and replacing it with another one and doing it all in parallel.
There were a lot of people on this program that I never envisioned you needed.
I think NASA did the right thing by turning it over to a contractor or a team of contractors, but they stayed too much involved so it was all done by a committee.
I mean I don't want to pick on anybody, but who was in charge?
Who felt they were really responsible for that accident?
I couldn't see it and I followed it and could offer my opinion, but I didn't sense that the program had somebody in charge.
There was another one.
Yes.
I was wondering if you could talk about any examples that you know where specific technologies that came out of the Shuttle main engines have either been used or avoided in newer launch vehicles today.
No, I think a lot of the materials will certainly be carried forward.
Coatings, seal technology, that's probably one of the areas in the turbomachinery that I didn't talk about enough.
There are a number of seals on those shafts that are very critical to the operation.
And there was a lot of technology advancement made in the development of the SSME on seals.
Those would be several examples.
Yes.
I was wondering if there was much discussion before the project started on the risks in going from 700 psi operating [NOISE OBSCURES] whether there was some option to go to an intermediate pressure.
I am sure there probably was.
Again, I came in the program about a year after it started so I was on a periphery of some of the early stuff.
But I think the risk at that time more focused on NASA, particularly at Marshall.
I think they wanted a liquid booster.
Eberhard Greff, the German felt very comfortable with liquid boosters because you could shut them off, as opposed to the solids.
But cost and other things, I think they eventually became comfortable that the solids were fine.
I think the risk tradeoff was more on the booster.
You shut a liquid off.
When you light those solids you're going somewhere.
On the engine, I know they knew it was going to be harder.
I'm not sure they appreciated how hard it actually got to end up with those 8,000 psi pressures at the head of the pre-burners and what that did to the rest of the system.
And what it did to materials and what it did to crack growth rates and everything from then on.
You wouldn't fly today without it.
You understood the mechanism, you applied fraction mechanics, and then you applied several factors, and then you certified that and that's where you went.
Does the aircraft industry do the same thing because they have turbine blades, too, that crack, don't they?
I'm really not sure.
I don't want to fly on an airplane that has got a cracked blade.
J.R., you talked about the lack of testing nowadays compared to at the beginning of the program.
To what extent does the fact that, I mean after every flight they take the engines out and they borescope them and look at that.
I mean does that, to some extent, constitute continual testing?
No.
What's the difference between that and what you get out of the tests?
Well, I think probably -- Today's teams, they go to the log book and they read what was written down by the last generation and that's what they inspect.
That's fine and that's very thorough.
But the testing, the introduction of some kind of a small change by a mom and pop operation, that I alluded to earlier, that's what a test program will catch.
Some new problem that creeps in or will get caught that's not done there.
No.
Let me say it another way.
I believe it is very dangerous to execute this program in the way that I understand is planned to be executed between now and when they retire the Shuttle.
I think that's the most dangerous period in the life of the Shuttle because of that.
I mean the people that are in the program now have not been part of developing it.
I don't know how many pictures they've seen of a LOX pump that has burnt up.
You just think about things differently once you've been exposed to that first-hand, that's my view of it.
Now, how you transition out of a shuttle program onto something new, I don't have that answer.
I mean I think you've got to do both.
Like the wing-walker, I wouldn't let loose of a shuttle before I had something else in hand.
And today they're going to let loose this and hope this other thing comes along.
And I think there's some risk to that.
Along that line with the O ring issue and the foam issue, do you think that those were never solved, were never looked at because of lack of money or just confidence in the technology at the time?
I mean the O ring was kind of carried over a little bit.
The foam, I don't know how much of that was brand-new.
And nowadays are we just relying on the confidence that these things have worked for a hundred mission or so or is it just lack of money?
No, I cannot say there was a lack of money back at the time of the O ring.
Had the culture been and if they were inquisitive enough to pursue it, I think they could have gotten the money to do it.
On the Shuttle engine, out of necessity the culture was there.
I mean it was driven by people like Yardley.
I think another thing at NASA and aerospace, you know, you can do wonderful things with computers today, but too little of all this analysis is anchored by a good failure.
You've got the analysis, you've got a lot of programs, you can do a heck of a job on analysis today, but it's not necessarily anchored in the remnants of a good failure, one that has adequate data.
If I'm building a test bed now, how do I kind of intelligently justify it to my sponsors that I want to have a budget [NOISE OBSCURES]?
Well, I don't think you're going to sell it that way.
[LAUGHTER] Right.
Well, I think your sponsor or whoever has got to have a good appreciation of how far you're going to be pushing the technology.
If you oversell it then you're not going to get the money to be able to stand those.
If you push too hard in that direction your sponsor is probably going to get disinterested.
That's going to be a fine line.
Pick a current example.
Pick at least, as I understand it, what NASA is trying to do with the Lunar project or the moon project.
I, for one, think that their yardstick is going to be tough.
Apollo didn't have any failures, why are you going to have any failure?
I think they are going to have to be very careful how they sell that program.
Just because they're using or will use shuttle-derived elements, an external tank, maybe a five or an additional segment to the solid motors, the SSME, that's not going to be a freebee.
I mean, I've already told you, I would do some things to the shuttle main engine in building a different program.
The tank, you know, they're not going to keep the foam from coming off on the configuration that they have.
And I also wouldn't assume that I don't have a problem now because I don't have an orbiter on the side of it.
The foam is not supposed to come off.
I would degrade the insulation and put on something that would stay, as an example.
Yes.
How much of the original engine, the current engine right now, is it exactly the same as the original one or what has changed.
Materials?
Is the computer the same?
Is everything exactly the same?
I think it is the same.
I'm sure there are, through engineering change proposals or fairly low-level change traffic, some things that have been upgraded.
But Inconel is still the basic material of the housing.
The turbine blades are still Mar-M 246 which is a high strength copper, I think.
That's the same thing.
They still use ball bearings when they fly.
And I don't know the change point on this, but Pratt & Whitney makes the high pressure turbo pumps now and they used roller bearings.
And that would be a big change, the two high pressure pumps that have now been incorporated in the Shuttle engine.
But other areas -- [OVERLAPPING VOICES] the high pressure turbo pumps?
Yeah.
At some point a couple of years ago, I think, they were incorporated and brought in.
Those would be big part number changes.
But a lot of other areas of the engine, I think it's the same.
And I think that's one of the reasons that they feel they don't need to test much.
They've got all that.
So, we will have to see how the next four or five years play out.
Anything else?
Well, since they're talking about using a lot of these with the next generation vehicle and they won't be reusable, what can be done?
I mean what's the way to go about kind of de-rating the Shuttle?
You talked about opening the throat, but all the other things that make the engine reusable and presumably make it more expensive.
Will it really be the same engine when they get finished?
I mean what has to be done?
I don't know.
I mean some may argue and suppose because the requirement for reusable engine isn't there will eliminate some of the things.
I don't know what they'd be.
But, having paid all that development, I would be more inclined to really minimize the change for change sake and do some things like open up the throat.
I mentioned that as an example that would reduce the operating pressures and, I think, give them more margin across the board.
On a tank, I would go to an installation system that is going to be a little less efficient.
You're going to have more boil off, weigh a lot more, but something that doesn't give you other problems.
J.R., let me ask a question that was asked of me and I couldn't answer earlier.
Did we ever look at putting the insulation internal to the external tank rather than the outside?
I'm not aware that it was looked at.
Somebody along the way, I suspect, could have.
No, I'm not aware of what it was or what it would have been.
Anything else?
Well, I've enjoyed it.
I hope you don't repeat the same mistakes that we made this time around.
And good luck in your projects.
Thanks, J.R. [APPLAUSE] I'm just going to take the last three minutes.
J.R. referred to the fact that we only had one engine shut down in the whole history of the program.
And that was due to a sensor.
Just to give you a sense of what goes into the operation of this system.
It was always recognized that you have sensors looking at the temperatures.
And the idea is that if anything starts to go wrong with the engine, you want to shut it down now before the thing blows up.
Because we can lose one engine and have an intact abort, but if the engine blows up and takes the whole boat tail with it you're not going to get back.
Generally, there's a little switch in the cockpit which enables the ability of the sensors to shut down the engines and you fly with that on.
But the problem is if you lose one engine and you get into an abort mode now, at least for the first part of that abort mode, if you lose a second engine now, with only one engine remaining, you cannot complete an intact abort.
What the crew does, if you lose an engine, you take that switch and you disable the automatic shutdown.
Well, what happened was a few minutes into the launch the sensor started to go off scale.
The engine was shutdown.
But they were far enough into the launch that they can actually do an abort to orbit.
They took the engine switch to disable, normal procedure.
When they got a little further along so that they were in what they would be able to, a little further on, I forget the exact details, but they took the switch back to enable.
But the main engine flight controller on the ground noticed that the sensor in one of the other engines was also starting to go off scale.
And, if that was allowed, it would basically take down a second engine.
That would have put them in a single engine, what we would have called an intact transatlantic abort.
But, by that time, they were too far to land in the normal abort site, and they would have ended up landing somewhere in Africa, I think, in the dark.
I mean it would have been a really bad situation.
And so the flight controller was sharp enough to tell the flight director we've got another bad sensor, tell the crew to take the switch to inhibit.
Luckily, the flight director's background was in propulsion systems.
He knew exactly what the flight controller was talking about.
They called it up to the crew, they took it, and so the second engine did not shut down.
And, in fact, they made it into orbit and they managed to complete the mission.
But these are decisions which have to be made in split seconds.
In fact, the flight controller who made that call was given a special award from NASA.
You really have to know these systems inside and out.
And that is, of course, why we have so many simulations where they run those sorts of failure cases so that people are able to make these decisions very quickly in real-time.
And that basically saved the Shuttle and the crew from potentially a really bad situation.
Is the orbit burn done by the main engines?
No, they're done by the OMS.
Once the main engines shut down, you've dropped the tank so you don't have any more propellant.
And there have been a few, I think four, engine shutdowns on the pad.
You start the engines about six seconds before T zero.
That gives them enough time to come up to full operating performance so that you can check out that they are OK.
Remember we talked about the twang because of the asymmetric thrust that gives you enough time for the Orbiter to tilt forward and then come back?
And then when you're vertical that is when you fire the engines.
I've got some pictures of this.
At one class I will show you some slow motion pictures of the launch.
You can see all that.
We've had four pad shutdowns.
Of those two were due to real problems with the engine and two were due to instrumentation problems.
It's always a problem of how much instrumentation do you put in and how much do you trust it?
And I know Aaron has made the point, on several occasions, of if you put an abort system in for the crew, is that going to be triggered automatically or does it have to be manual?
You certainly don't want to get shot off the end of a good working rocket just because your sensors have told you that something wrong is happening.
There are a lot of interesting engineering decisions to be made with that.
End of class.
We'll see you on Thursday.
And thanks again to J.R.
A couple things.
First of all, this is for those of you who are working on GN&C for your projects.
I have spoken with Phil Hattis from Draper Lab, who is going to be giving the lecture on GN&C, but he is not giving the lecture until the third of November.
Now, that is kind of late.
And so I asked him if he would be willing to meet with all of you who are working on GN&C just so you could talk with him and ask questions.
If you want him to give you a short version of his presentation that he's going to give in November, he can do that.
But he is our in-house resident expert.
And I figured it's better if you get a chance to talk to him early.
He has some time free tomorrow and he has some time next week.
He is out of town through today.
I know we have two teams working on GN&C.
If you guys want to get together and find out if there is some time when you could all meet with Phil, I will give you his contact information.
He says he is happy to do it.
It would be much more convenient, from his point of view, if you could all see him at the same time and talk about it together.
Let me know what you come up with.
But, as I say, he is happy to do it.
Second thing.
There was an interview printed in US Today, which I have posted on the class website with Mike Griffin, the administrator.
And it says NASA administrator says space shuttle was a mistake.
And so the question was what do we intend to do about that in this class?
Well, let me actually read the specific words that Griffin uses because I think that's important.
Griffin said my opinion is that the Shuttle was a design which was extremely aggressive and just barely possible.
And that's actually, I think, a slightly different way of looking at this.
I mean I would also say that Apollo was a project that was extremely aggressive and just barely possible.
And I think that it is actually NASA's business to do things which are aggressive and just barely possible.
Now, whether it was the right decision for the country to abandon the Exploration Program after Apollo and do the Shuttle and then later on the Space Station that's more, I think, in the political realm.
And, as I think you've heard both from Dale Myers and John Logsdon, that, in fact, there were political determining factors which didn't really give NASA a lot of choice.
Just to remind you, the basic point of this course is not the political and economic history of the shuttle.
It's a technical exploration of how we did the shuttle.
I think we would all agree that the concept of the Shuttle was extremely aggressive and the technology was being stretched and it was just barely possible.
And, yet, we did it.
And the idea of this course is to understand how we did it and how the systems work.
I think it will be interesting, when we get towards the end of the course, maybe we'll sit down and relax and instead of having, I think there are two kinds of open sessions towards the end that I've been thinking about what we want to do there.
And in one of them I think we may just talk a little bit and reflect on what we've learned, and that will give us an opportunity to discuss issues like this.
That's all I'm going to say about this.
I don't propose that we discuss this further today because we really are fortunate to have Bass Redd.
And Aaron Cohen is going to introduce him formally.
I will just say that when I was at Johnston Space Center last spring and I met one of the older managers there and I was telling him about our plans for this course and how Aaron was going to be a visiting professor here and we were inviting some of the experts in the early design of the Shuttle, he said, well, there is one guy you absolutely have to get, and that's Bass Redd.
Well, we've got him.
We're very fortunate.
And, Aaron, I will turn it over to you and let you explain a little bit more why we're so fortunate.
Well, first I have to tell you how I found Bass.
I had a hard time finding Bass.
He lives in a placed called Smithville, Texas.
And, when people ask him where Smithville, Texas is, he said it's about a few minutes after you pass resume speed.
So, it's not a very well known place.
But I finally found Bass.
And he is truly an outstanding aerodynamicist.
And the thing that is so important is that aerodynamics, to me, from my vantage point, is really the linchpin, you might say, of putting a system together.
Aerodynamics makes the system come alive.
And as you could hear from the various speakers, from Tom Moser, and we talked a little bit about the guidance, navigation and control, it's important for the structures.
Aerodynamics is important for the structures, it's important for the flight control system, it's important for the hydraulic system and it's important for the heating system.
So, you have to understand the aerodynamics before you can make the vehicle actually come alive.
And Bass Redd is the man to do it.
He is absolutely one of the great aerodynamicists of our times.
And he started out before there was computational fluid dynamics and the old wind tunnel testing.
And now there is computational fluid dynamics, so you're going to be in for a real treat today to hear how Bass Redd put both the Apollo together, he did the Apollo aerodynamics, and he also did the Shuttle aerodynamics.
And he is also now consulting to help NASA do their future work.
Let me now turn it over to Bass.
Thank you, Aaron.
It's a great honor to be with you this morning.
And you sure do look young.
[LAUGHTER] We will try to cover basically -- [STATIC OBSCURES] And with that overview then of the Orbiter is how the program requirements drove the aerodynamic configuration, probably the biggest thing in what the aerodynamic characteristics are driven by the program requirements.
And that will make a little more sense after we get into the briefing.
Then the aerodynamic considerations, those that aren't due to program requirements but just due to the basic fundamentals of aerodynamics and the configuration that is being evaluated.
We will do that for both the Orbiter and the Integrated Vehicle.
Then I'll try to tell you how the aerodynamic design was accomplished through each of those phases, a lot of ways to work problems.
Early on having powerful tools or sometimes a curse to you because you can produce more data than the system can consume.
And I remember when we first got started on the Shuttle design, the people from structures had just finished the NASTRAN on the Apollo and they immediately wanted enough data to put into the NASTRAN for every configuration that was being conceived.
That's not really a smart thing to do when you have literally hundreds of configurations to be considered.
We will try to talk to you about phases, phase A, B and C, and defining the problem areas.
And then defining the aerodynamic characteristics and do just a quick summary of the wind tunnel test program.
And with some comments on how CFD might have driven that were it available.
Through the `60s and `80s these design studies were accomplished in phases.
Phase A is primarily in-house civil service design.
I don't know how well you understand the NASA part, what the contractor part, what the civil service part is, but phase A is primarily in-house civil service.
Whenever the contractors began to feel that this might be a real program, they began to do their in-house studies and they began to visit and give us recommendations and ideas on their own to drive that.
None of that is paid for.
That's a gratuitous kind of a thing.
And contractors have a bit of change set aside to do those kinds of studies, but it's a freebee and really a super way to do things.
And that's still going on today.
Phase B is then you give contracts for generic studies.
Scopes out the concept feasibilities and the program requirements and does those trades, does projected cost and timelines.
By the way, before I forget it, Humboldt Mandell on costing would be excellent.
We worked quite close hand-in-hand on trying to get those requirements and cost run together along with the weight.
NASA then, after they get through this phase B, they produce a detailed statement of work that is required for the phase C and D. And then phase C and D is the design and development.
The Air Force separated those two.
NASA always sort of put those together.
So the design, development and construction.
And the contractors do the detail work.
NASA manages and checks it.
Usually the way we operate, at least in our area, the aerodynamic area, we had a crack team that would go back in and define the problem, a potential solution to that problem and then suggest to the contractor they might look at this way or that way.
But they really put the muscle to it.
They put the final studies.
They have the design documentation.
And we try to steer that train as those things happen.
In the Shuttle now, phase A, we started in 1969, just shortly after the Lunar landing, they put together a small team.
And this is at Johnson Space Craft Center.
And that's through my knothole and the way that I look at things.
But we did end up being the lead center in the shuttle.
But we started with a small E&D group that was headed by a fellow by the name of Jim Chamberlain.
Jim Chamberlain was the one that was responsible for the Gemini Space Craft, a very unique individual gone to heaven to be with the Lord now, reviewed by Max Faget who was head of E&D and Bob Gilruth who was center director, but that was a real privilege to work with him.
What came out of that initial phase A study is what the vehicle should be, is a small personnel vehicle with a four man crew, 10k payload landing on the runway with about 200 nautical miles cross-range.
Quite different from the shuttle that we have today, but that was during the phase A.
Strong design modules in this time were developed.
And, showing how the other program requirements go with the design, the size and the program cost, there were individuals who had in their databank, which consisted of books that they kept on their tables, were all of the aircraft, all of the spacecraft, all of the missiles, what the weight of every individual subsystem was based upon plots with ECLS's on how much heat they took out, on the avionics as to how many computations they made.
Well, not to chase that rabbit too far, but basically they were down to the subsystem kind of level.
And then with the primary structure, secondary structure, how that was affected by landed weights or entry weights, the thermal protection system.
And it was amazing what a good job that kind of approach would do.
And then we had point designs that were being looked at both in-house at Johnson Space Craft Center, at Marshall and by all the contractors.
And then all that data was fed into the system.
So, that was a designed database.
Some of those have been automated now and pretty good.
And they don't give you the right answers, but the answers they give you aren't very far wrong either.
They get into the ballpark for you.
Phase B started in 1971.
Something happened.
The requirements were not dictated by NASA.
The requirements were dictated by the Air Force, at least the driving requirements.
If we had our own, a small vehicle it would have satisfied that.
The Air Force requirements were 55,000 pound payload, 60 foot long payload bay, 15 foot in diameter, it had to be returned with a 1200 nautical mile cross-range.
At least 32,000 pounds of it had to be returned.
It was classified at that time.
It is no longer classified.
You can use your imagination to figure out what kind of payload that you would want to go up into a high inclination orbit and bring back in one pass.
Where that would take you and why you'd have to get back that quickly.
But those were the requirements that were fed into phase B.
Maximum reusability did not come from the Air Force.
That came from the NASA side as that was part of technology.
NASA has two basic jobs to do.
One is the design of the vehicle, the delivering of the hardware, accomplishing the mission, but the other requirement for NASA is that it increased the technology.
Going where no man has gone before, yes, but doing things that man has not done before in a better, faster way.
And so maximum reusability was a goal that the engineers were handed.
Soft ride.
They wanted to be able to a congressman or a senator up in it.
[LAUGHTER] A neat thing.
Multi-center participation.
If you ever get Hum up here, that cost you.
Any time you break a job down and give it to more than one center or more than one contractor, it's like Humpty-Dumpty, you break him apart, you've got to put him back together again.
And it takes a lot of eyes and elbows on both sides to accomplish that.
And basically, once you break something down, you've increased your cost by 30% to 40% to put it back together again.
Now, NASA has to have, under congressional mandate, contracts in every state in the union.
Multi-center participation drives a great number of things so I felt it needed to be on the chart.
But the big thing that you need to know as engineers, sometimes you cannot set your own program requirements.
They are dictated to you for strange reasons.
And you have to play the hand that you're dealt.
Phase C and D, we had four contractors that bid on phase C and D. The interesting thing is Rockwell was the only contractor that bid within the allocated funds.
They also had the best proposal, but that was very interesting.
What they did was use the aerodynamic database that was developed in phase A and in phase B for their design analysis.
They augmented that, they did some in-house testing, and Rockwell said we can do it for that amount of money but only if you don't change anything, NASA.
And we changed things.
They had a smaller orbiter, and we rejected that very early on.
It had no capability for weight growth.
And we showed that every program that has ever existed grew in weight by at least 5%, 10%, 15%.
And so the Orbiter was sized up just a little bit larger than the one that they had proposed with their agreement.
Aaron probably had a contract change authorization because of that.
But, anyway, we did that right at the beginning.
What we did also aerodynamically during that phase B is we developed a complete database in the nation's best facility to be used for what we call verification analysis.
And that database is very well documented, and we will talk about that a little bit more.
Program requirements.
May I ask, before you go onto program requirements, could you say something about kind of, you talked about the aerodynamic database, but nowadays we've got a lot of information about hypersonic aerodynamics?
Can you say a little bit about the state of the art?
What did people really know?
Yeah.
As a matter of fact, I'm going to do that when we go through it by phases specifically.
I'm kind of telling it to you like a mystery story.
[LAUGHTER] The program requirements dictated the basic orbiter configuration, payload size, crew size and returning the payload bay, the engine.
Now, if you just visualize in your mind, and it's kind of a two-chart thing, let me just jump over to that since I know how to go back now.
If you've got a payload that's got to be that long and you've got a crew size and the crew takes at least one telephone booth's worth of volume per crew takes at least one telephone booths worth of volume per crew, Even Superman needed something that big.
[LAUGHTER] You've got the volume of this.
Engines in the back.
You want to return that because engines are very, very expensive.
The avionics, of course, you want to bring those back.
They are very, very expensive.
And you've got to have it pointy end first.
You looked at that and said, boy, you didn't have much choice as far as the fuselage was concerned.
And that's right, you really don't.
Now, that's pretty well set.
Now, the wing size is required to augment the body lift.
Now, if you just had the body on there, it would have to land about 300 knots.
We didn't have wheel tires and breaks to land at that those speeds so we had to have some more lift.
And that was accommodated with a wing.
Figure out what CL max is.
They're all about the same.
You like a delta wing because it's a little more nice from a heat transfer standpoint.
Sure a whole lot nicer if you've got to go through supersonic and transonic conditions at low angles of attack.
So, you've got a delta wing that you want to have on there.
You don't have much choice on the size of that.
Airflow by the way 12% thick.
That makes it nice for the structures people.
Still does a super job aerodynamically, too.
And so that's pretty well dictated.
Then the entry angle of attack is dictated by the hypersonic L/D required for the cross-range.
And you can go through your equations, Newtonian aerodynamics sign or the exposed area, and the then L/D, the tangents.
And so you've got the hypersonic aerodynamics from that angle of attack required for the cross-range.
The wing shape to balance the configuration for the aft CG and acceptable wing leading edge heating.
In other words, we can position that wing at the right place for the center of gravity that we have which, by the way, even in phase A is known pretty well didn't change much.
The control services was the magic thing.
With the payload in and out 1.5% CG changed.
There has only been one other aerodynamic vehicle that has ever been built that has a CG of 1.5% of body length.
That was a B-36.
And this vehicle had to have the 1.5% change in the center of gravity because of payload in/ payload out.
And you can try to stack it together in many different ways.
That's just as good as you can do.
That dictated the size of the control services to accommodate that.
They are very big.
They are barn doors.
It makes it nice for controlling the vehicle.
Oh, and the vertical tail, we'll talk about that.
That's for crosswind landings.
But the way, most all the vehicles that you've seen from overseas that have been proposed cannot land in a crosswind.
I don't know what they are going to do.
They don't have enough control service to land in the crosswind.
But that dictated the vertical tail size.
Now, as you kind of go around the loop on those things, some potential items that we probably want to point out here is the body shape.
We've already basically talked about it.
Nose shape, that's pretty straight forward.
The basic Orbiter shape didn't really change from phase A through phase C and D. We blended the OMS pot a little bit because we didn't want it to interfere with the payload bay door which was built by a different set of groups.
The delta wing, we put a little crank in it, a double delta, and got a little more subsonic lift.
And we didn't have the body flap moving to start with, but we found that it was a very powerful control of which assisted the center of gravity movement.
The full-spanned elevons were needed to control the center of pressure for hypersonics.
And there are other little things there, and you can read those through at your convenience, but those are just basic things that you have to do.
And we will comment on some of those a little bit later.
The Integrated Vehicle, the launch vehicle, if you please, what do the program requirements do to it?
Well, the Orbiter was sized for that payload bay weight.
It had to land.
It had to enter.
We knew the weights pretty well.
The 150k pound Orbiter, the 65k pound payload dictated the requirements of the Orbiter injected weight.
You add them up.
You've got to get it into orbit.
Once around at the high inclination and return to the launch site set the Orbiter performance requirements.
We had to have a certain amount of energy available from the propulsion system.
Maximum hardware reusability mixed with a cost consideration yielded a stage and a half vehicle.
And I will talk to you about that in just a minute.
But the cost constraints, there was a great move afoot to have a return to the launch site booster, manned booster, if you please, but the cost and constraints and the size of the vehicle got pretty big.
The wing vehicle loads were significantly reduced by the parallel burn.
You've got the orbiter tucked in between the solid rocket boosters and the external tank versus an orbiter that would be on the end.
It makes a great difference in the structural loads that the aerodynamics produces.
Solid rocket booster mismatch decreased or dictated an increased beta dispersion.
Typically on an end-to-end vehicle you've got everything kind of tucked in pretty tightly.
On this one the SRBs are spread, any mismatch at burnout and we had some big beta dispersions that still haunt us a little bit today.
Soft ride requirement.
Yes, I'm sorry.
I just don't know what a beta dispersion is.
You don't know what a beta dispersion is.
All right.
[LAUGHTER] Angle of attack.
Angle the size of it, beta.
Alpha, beta, standard aerodynamic definition.
Which, by the way, brings me to a comment.
We've been asked to chase a rabbit with you here.
I would suggest very highly that you get yourself a NASA dictionary where standard terms that your bosses will use because they've been using them for years.
[LAUGHTER] That you do that.
And I'm not fussing at you at all.
You can get a little gray book that's called the NASA Dictionary.
And you will know what alpha and betas are.
CMQ plus CM alpha dots.
All those standards terms are defined.
Flutter.
Buffet.
And you will really look sharp when you know what he's saying.
[LAUGHTER] All right.
Go ahead.
I know there are some people here who didn't go through aero-astro, and so may have actually not had a full course in aerodynamics.
Hopefully most of you have had it.
But any time there are expressions or terms you don't understand just do ask and we'll bring you up to speed.
And that NASA dictionary, by the way, covers not just aerodynamics.
It covers all kinds of other vocabulary kinds of things that your peers will be using and your supervisors will be using and you will know what they're talking about.
Soft ride requirements, that caused us to go to a real low dynamic pressure which is nice.
Most vehicles are up around 1200 pounds per square foot dynamic pressure.
This one is down around 800.
And the reason that came about is lesser G loads early on.
SRB flare angle, no big deal there.
Plume effects have really ate up aerodynamics.
To this day that is the only thing aerodynamically that we still don't know how to predict or the base pressure due to plume effects through the subsonic and supersonic speed regime.
CFD gets it right every now and then.
Sometimes it doesn't get it right.
And the wind tunnel test still hadn't got it right with solid plumes.
All the scaling parameters that you use still don't produce, to me, what a real valid answer is.
The flight data is used, and we update that.
Yes, sir.
When you say flare angle, do you mean like in the stack away from the Orbiter?
Yes, sir.
This flare angle right back here.
We found that if you set that right, it pretty well sets the flow and the plumes behave themselves.
If you set that too small the plumes will change greatly with mach numbers, sometimes in an unpredictable fashion, so that SRB skirt size was primarily from that consideration.
This is real precious to Aaron's heart.
We changed the tank shape early in phase C and D. And remember I told you that we had developed a database during phase B that would suffice and we could go immediately into the design and development of the total stack?
By the way, I like the ogive.
It's a lot prettier than the cone that we had on the front end.
But basically what that did, that changed the aerodynamics of the whole stack.
And the data that was delivered to structure for their structural analysis, the data that was delivered to the heating people for their heat transfer analysis had to be done all over again.
That's a year and a half cycle from the time you say let's go build a model, let's get the data, let's analyze it.
The other thing that happened when we changed the tank, everybody took note of that, and there are all kinds of other small changes that jumped on.
[LAUGHTER] On the Orbiter a little bit, on the external tank, on the SRBs and on the inclination that we had between the Orbiter and the tank.
Let me just hit some things I think that you want to take note of on this one.
I guess the nose shape, we mentioned that one already, reduces the drag.
It also reduces the loads.
We have to worry about the punch loads between the SRB and the tank.
And those aerodynamics are driven quite a bit by that nose shape.
The SRB locations fore and aft, there is some adjustment that you can have in that, and those are all optimized that whenever we do the aerodynamics that puts a minimum amount of load into the system.
Orbiter instance angle, that's the angle between the Orbiter and the external tank.
It drives the stability of the configuration to some degree.
But, more than that, it drove the loads quite a bit into the program.
And that was optimized.
Full span elevators, we use those to adjust the pitching moments so we get minimum change required to trim the vehicle from the SRBs, solid rocket boosters, and the SSMEs, the main shuttle engines.
SSME, plume effects and the plume effects from the SRB, that's the only thing that we got a bad grade on as far as predicting aerodynamics on either the Orbiter or the Integrated Vehicle.
All the rest of the stuff came out just about as we had predicted.
Now, getting back to where you asked the question.
Phase A.
Go back now.
What is a phase A?
That is the conceptual design.
We had all kinds of configurations because we had all kinds of requirements.
We really didn't know what those were, and so that analysis had to be done.
Many, many vehicles.
A lot of considerations.
The analytical techniques we used to identify problem areas.
Hypersonics, quite frankly in phase A. Newtonian vehicle shape was approximated with flat plates.
Works great on a configuration.
[LAUGHTER] That's pretty much like a vehicle that is kind of trimmed out of a piece of a sphere, if you please.
And those aerodynamic characteristics were very good.
As a matter of fact, the thing is laid out on graph paper and you counted the squares and saw where the CG was and what things had to be.
And it worked.
It worked.
Now, we didn't know that it was working at that time.
But we found out later that it did.
But that's what we used.
[LAUGHTER] NASA documentation.
NASA had pretty good TNs on vehicles that were somewhat similar.
And the characteristics of those, we made the adjustments because of the difference in configuration.
The Air Force had a document called Dataman.
It was unpublished at that time, but we got a preliminary draft that we worked through by hand.
By the way, this is a published thing.
I would suggest very highly if aerodynamics is going to be your forte is that you learn how to use that program.
It's Dataman Aerodynamics Handbook.
Navel Ordinance had a handbook out.
They've never computerized theirs, but it's the Handbook of Supersonic Aerodynamics and good-old Horner's fluid dynamics lift and drag.
It is still around today, still use it, but those were the aerodynamic methods that we used during phase day.
And we also did small scale wind tunnel tests and those things that were the problem areas.
And the problem areas were primarily in the transition.
Yes, sir.
I can understand how you can do wind tunnel testing.
What about plume effects?
You said you didn't understand those.
How do you test that experimentally?
[UNINTELLIGIBLE PHRASE] No, that's done in a wind tunnel.
Some people said you need to match momentum on a scale vehicle.
Certain other characteristics.
What you'd like to do is be able to simulate it with a solid plume, so we did solid plume testing.
We had method of characteristics back then.
It took it about three weeks to run, to get the plume sizes, and they did a pretty good job.
But when you have the plume sizes you still after the base effects change in the pitching moment.
And so we did the solid plumes.
And we did gaseous plumes at Ames Research Center.
We did cold gas and hot gas.
But none of those met all the scaling parameters.
When you rolled all that up and you did the best you could, we still missed base pressure.
Base pressure was missed on Saturn 5.
Base pressure was missed on Mercury.
Base pressure was missed on Little Joe II and Little Joe I.
Every launch vehicle that I could find we missed base pressure on.
We missed base pressure on the Shuttle also.
Not a big effect, but it is embarrassing that you cannot get it right.
But this was done primarily more in phase C and D than in the phase A time period.
We just kind of said plumes are going to be about that much based upon data from other vehicles.
And then making sure that it was not a design driver or design stopper, if you please, that it wasn't going to be a problem that was so big that it changed things so greatly that your performance wasn't adequate.
And we had margins and everything.
Anyway, we did some small scale testing.
Phase B.
What did we do on the Shuttle on phase B?
We went into phase B with four major configurations.
Three externally identical configurations with swing wings.
It was called the Triamese.
And I think General Dynamics was the one that proposed that one.
They have all bought on another, and it's hard to find out who is who any longer.
But, anyway, that was one of the concepts.
Very, very expensive.
The swing wings, never really solved those problems.
Lockheed had a large single stage blended body, kind of an HL-10 looking thing with a flat bottom, and then with a large external tank around.
And I couldn't find pictures of all this to show you, but that was one that came in.
And then the two-stage fully reusable with both the first and the second stages being runway landers, a very favorite of Marshall.
And then, the one that we called the stage and a half where you're literally burning the first stage boosters and the second stage engines at the same time with the external tank, we referred to that as a stage and a half with parallel burn which is the current shuttle.
But those are the things that we started phase B in and did aerodynamic designs on all those.
How did we do that?
Well, we had gotten a little smarter.
A lot of the Newtonian programs had now been automated.
It was still pretty much a flat plate kind of a program, though, were you divided the vehicle up into a number of flat plates.
And then there was some contractor in-house wind tunnel test.
NASA did some limited tests on potential problem areas post phase B and pre phase C and D. That was a gap between the time we put out an RFP, request for proposal, and the time that the contractors came back with a proposal.
We had narrowed it down that it would be a stage and a half.
That would be what we would write the RFP for.
We developed a large wind tunnel test program, developed a full aerodynamic database and heating database for that shuttle that was there at that time that was delivered to the contractors that were going to bid on that.
Integrated Vehicle.
We used the USF stability and control datcom.
Some contractor wind tunnel tests were done and limited NASA testing and analysis for potential problem areas.
And also we did a post phase B, full aerodynamic database for that vehicle.
Now, with that, what we're trying to do is eliminate configuration changes if at all possible.
And we accomplished that to some degree, but the change to the tank and other changes came back in that caused us some delay.
But it was minimized.
Phase C and D, configuration was baseline.
A big word here, make work changes only.
Nothing has changed because it looks better, works better.
You've got to change it or it will crash.
That's what we said.
That's not what we did.
We made some changes because they were nice.
Theoretical aerodynamic techniques were employed to ensure you never get to test the right Reynolds number.
Henceforth, your boundary layer is a little bit wrong.
Plume effects, they've got to be corrected.
So there was a lot of analytical work that was done during that phase C and D. Let me say this.
That area of our technology was really improved.
A lot of dollars that went into that, Aaron's courtesy, that the program probably didn't need to fly the shuttle.
Now I tell him.
I didn't tell him back then, you're right.
[LAUGHTER] But, for the benefit of technology, we did that.
We pumped great money into CFD, great money into method of characteristics.
A lot of money into other techniques that were there.
And once more NASA goal was not only to do that mission but to help technology.
And the reason you can do CFD today was because the dollars that the shuttle program pumped into that.
Had they not pumped that in you would still be adding up flat plates.
[LAUGHTER] Aerodynamic variations and uncertainty were developed for all aerodynamic coefficients.
No other vehicle had really done that.
You sort of did the best that you could and said these are my aerodynamics.
This is my normal force versus angle of attack.
This is my side force coefficients, yawing moment coefficients versus beta.
And that said you would analyze that, give that to the flight control system, give that to the structures people and they would put their margins of safety in and look at it.
We were quite concerned on this vehicle, especially when it starts out at a 45 degree angle of attack.
And sometimes during the supersonic and transonic regime it transists down to a low angle of attack.
Flow is separated.
Some place in there it reattaches.
It cannot always make up its mind.
You've got to be able to control the vehicle, even though we weren't intentionally maneuvering at that time.
And so we developed these variations.
If you go back in the tunnel three times you get what?
Four different answers.
[LAUGHTER] And they're all very, very close.
If you change facilities you get slightly different answers.
We had some Reynolds level boundary layer kinds of problems between the wind tunnel and the full scale.
And so these uncertainties were developed for all the aerodynamic coefficients.
Then this data was delivered to the GN&C people where they make for sure that they can fly their system with the uncertainties.
The CN betas, yawing moment coefficient with angle of side slip, the control moment coefficients, the delta yawing moments and rolling moments due to elevon maneuvers, where the errors on those, that the control system wouldn't go berserk with any combination of those.
The same with structures and with flight dynamics.
Now, we're not living with those today.
We did flight tests to reduce the variations.
The astronauts put in PTI, pilot test inputs, where they would do it a certain way and we would pre-predict those characteristics.
Then we'd look at them post-flight.
We would pull the aerodynamic characteristics out with regression analysis techniques to say this is the aerodynamic coefficients that had to be there for that maneuver to look like it looked.
That, within itself, has some plus or minus stuff because you can change stuff a little bit and it still looks pretty good.
Those two then were combined and we had a set of reduced aerodynamics.
But today when you go to the aerodynamic data book, which we hope to be able to get you copies of, it will have not only the coefficients for all mach numbers, angles of attack, angle to slide slip, it will tell you what the uncertainty as a function of mach number angle of attack is for that.
And your system should be able to work with all those.
I want to mention this aerodynamic design substantiation report.
You won't be able to get a copy of that any place that I know of but we're going to try to get a copy to Jeff for you, and it should be available in your files.
But what that did, that goes back in, and for every coefficient, every angle of attack, how we arrived at that specific value and those specific variations using both wind tunnel and flight test.
A super document.
Not pretty but an excellent technical document.
And that was done during the flight test program and afterwards.
I thought we would stick up there the Wind Tunnel Testing Program.
That's primarily the backbone of the aerodynamic database of the shuttle both for the Orbiter and the Integrated Vehicle.
149 entry tests on the entry vehicle.
79 heating.
40 structural.
You can add them up.
Hours of wind tunnel test time 17,000.
Aaron asked me how that compared with the Apollo.
As best I remember, we did about 4,000 to 5,000 hours on the entry vehicle on Apollo.
However, Apollo didn't have all of those moveable surfaces.
Moveable surfaces are nice, do a lot of things for you, but you've got to test and make sure they're there.
And those were through various wind tunnels throughout the country?
Yes, sir.
These were the best facilities in the country.
Do they still exist?
Far up and away, yes, sir, they still exist.
The ones at Ames are still usable.
The ones at AEDC are still usable.
Unfortunately, those at Langley, even though they're there and they work, because of funding constraints, Langley has a hard time bringing them up and using them.
And that's unfortunate because they have some excellent facilities there.
And we surely used them.
In this Wind Tunnel Test Program, you can find out in a special wind tunnel testing program summary.
By the way, you can get this document pretty straightforwardly.
Ascent vehicle.
A few less hours, and that's because you don't move as many surfaces primarily.
Heating.
That's something to work on.
If you want a good project to work on, how do you take heating data in a wind tunnel on a small scale vehicle and make full scale predictions?
The reason that's near and dear to my heart, they ran a test not too long ago and they still cannot match those two.
A good PhD thesis.
Maybe you can figure out how to do that and how to do it right.
A lot of scaling parameters that are used, kind of like plumes.
You're not sure which scaling parameter really works, but there is slide test data, calorimeters, and there is fly test data and wind tunnel data.
It doesn't bother us that much because we've got enough conservatism in the system and uncertainties that we know we're safe to fly, but it does hurt your pride when you said hey, that heating rate should be 9 BTU's per foot squared per second and the fly test data said it's not but 2.
You might look good one way, you look bad another way, but if I was looking for something to get my plow into and I was interested in theoretical stuff, that's probably what I'd pick.
Total Wind Tunnel Test Program, Aaron paid for that out of his budget, by the way.
It was less than $100 million.
[LAUGHTER] In those year dollars.
But it was an excellent program.
Every test that was conducted was documented, analyzed, and we can make that available.
That concludes the things that I had to tell you.
And we hit that hour, just about right.
Do you have any questions that you'd like to ask?
If not, I'll give you a test.
Yes, sir.
[UNINTELLIGIBLE PHRASE] How do we affect the trim?
[UNINTELLIGIBLE PHRASE] Hypersonics, and when I say hypersonic, let me take it down to mach 5, was probably the best at prediction that we had.
A flat plate which the belly of the orbiter has a little bit of surface bending both in the pitch plane and in the yaw plane for stability.
If you put a big sphere or segment of a sphere, you could place the Orbiter on that and you'd say, hey.
And I know a sphere or segment of a sphere has stability.
Now, the uncertainties turn out not to be in normal forces or in even side forces.
Pitching moment is the one that you tend to miss the most.
And sometimes you don't know if you have a wrong pitching moment or a wrong center of gravity.
They both precipitate a delta pitching moment coefficient change.
The big barn doors, we have that into the system where we control not only a rate control system but an attitude control system.
And so it takes, with a 70 square foot, it just takes a small amount of change to correct any error that you might have in either CGR pitching moment.
Your control mechanism is with the big elevators and also the body flap a little bit, if you want to use that.
The reaction control system is a rate system.
And I didn't mention that.
And we intentionally put the reaction control system in the wake of the vehicle, even though, for orbital maneuvering, we have jets that are the forward firing end.
But so much uncertainty in how that changes the aerodynamics in a surface that is not a hard surface.
And that interaction, you can change things in such a way that you amplify it or reduce it.
And so we elected not to try to grapple with that problem and use just the aft jets, but it's just for a rate control system.
End pitch, that's the distance that separated that, and yaw on both sides, and in roll.
[UNINTELLIGIBLE PHRASE] X-15 was a low angle of attack airplane type configuration from mach 25 to mach 5.
You're flying a spacecraft kind of configuration where you basically roll about the velocity vector.
You want to maintain that pitch trim and roll about the velocity vector for cross-range in either direction.
Now, when you begin transition because you've got to be down here when you're landing, you pick the best place to do that.
And the best place to do that is back in the mach below 5 and, of course, above the subsonic regime.
And we try not to have any maneuvers that are required during that time period.
Any time you're going from a totally separated flow on the back of the vehicle to one that is now attached, separated flow cannot make its mind sometimes.
Does it want to be separated?
Does it want to be attached?
Not only the whole vehicle but even on small pieces of the vehicle.
And so there are uncertainties that come in there.
But that's the area.
But you've got to be sure.
And that's where we did a lot of our PTIs, by the way, as we became more and more brave to get those coefficients.
And the wind tunnel did a pretty good job on them.
Yes.
[UNINTELLIGIBLE PHRASE] What you're doing is changing the Newtonian flow now on my flat plate 70 square foot per surface.
You mean the rear-end?
The rear-end, the big end, yeah.
Now I've got a normal force coefficient sine squared to the angle.
Then I just change the angle up and reduce the forces on that, and that gives me a nose up.
I bring it down.
That gives me a nose down.
It works beautifully.
This is the body flap we're talking about.
Body flap and elevators, yeah, both.
Body flap and aft control surfaces.
Yes, sir.
You talked about the flight test portion of the aerodynamics.
What sort of planes were you using to do that?
The flight test was the shuttle itself.
After we safely entered and everything then we designed these pilot test inputs to extract aerodynamic characteristics.
The same technique that Edwards developed.
And they did develop that, by the way, on the X-15 and on other vehicles, too.
And we used that same technique.
We sent one of our men out and he spent about almost a half a year learning all their techniques, brought that back, developed that and worked with the astronauts very, very closely.
They trained for those maneuvers.
We extracted data for those maneuvers.
We would change the aerodynamic characteristics and not tell them what they are in the simulators.
And made sure that we didn't get any difficulty and made sure that the aero people could extract that data right and tell us how we changed it.
They all got good As.
And it worked very, very well.
[UNINTELLIGIBLE PHRASE] No, sir.
We did that one.
That was for the approach and landing test.
But for the entry vehicle, no, those were supersonic.
Yes, and we did hypersonic.
We're talking about tests that were done during the flights.
The first four flights were designated orbital flight tests.
And I think the PTIs actually continued well after that.
Yes.
Actually, what this has given us, in terms of a hypersonic database for a winged vehicle, is incredible.
At first they were called pilot test inputs because the pilots were doing them automatically.
And then I remember some of the pilots were saying you've got to do it very precisely and so we really have got to be concentrating on that.
And there are so many other things we have to concentrate on.
Could we possibly computerize this?
And so I don't remember which flight they started doing that, but at that point the PTIs became known as the program test inputs rather than the pilot test inputs.
You basically put in a little pitch up, pitch down and see how it affected the roll and the yaw and all these cross-coupling things at lots of different mach numbers.
And, over the course of a bunch of different flights and a lot of different PTIs, it really built up the aerodynamic coefficients at all these different mach numbers.
You might say a little bit about how we collected hypersonic test data in wind tunnels.
I mean it's not like you can just have a fan going and produce a mach 20 airflow for 20 minutes and you can play with your little thing.
Hypersonic testing was done primarily at the Ames Research Center three and a half foot facility.
We also did some up at Calspan, also used some of the Langley facilities, but the real workhorse was this three and a half foot hypersonic tunnel.
And basically you would have the air coming through the tunnel at the required conditions.
You would put the model of the vehicle, which is usually a 1.5% to 3% model in the facility.
You have what is called a six component balance that fits into the model.
That is a strain gauge relationship.
It gives you force and moments in both directions, or forces in both directions and some difference between the gauges so you can pull the pitching moments and the yawing moments and the rolling moments out of that.
You reduce it based primarily upon dynamic pressure or you say CN is equal to the normal force divided by QS, the dynamic pressure times a reference area.
Standard aerodynamics, but once more a language kind of thing where if you're in gas dynamics and not aerodynamics sometimes you don't get back over to those coefficients.
By the way, in that same document all of those terms are defined pretty well.
But you measure those with a balance.
Then the question happens, well, I wasn't at full scale Reynolds number.
And then that's where you come up with your theoretical techniques.
And you come back in and you look at it and you see that the boundary layer and the Reynolds number corrections theoretically make a very little bit of difference.
And then you go through today the CFD programs.
You say it gets about the same answer.
You go back to the flight test data and it gets about the same answer.
Basically, what you conclude is wind tunnels do a super job hypersonically for this kind of a configuration.
Now, it did good on the Apollo, it did good on the Gemini, it did good on the Mercury and it did good on the Shuttle.
Give me that configuration and I'll tell you what the hypersonic aerodynamic characteristics are.
I can tell you in phase A, and it's not going to be too much different at the end of phase B.
And maybe you don't want to hear that because maybe you were going to do great things in the hypersonic area.
But on a blunt body, which is what we're talking about, it does a super job now.
X-15, low angles of attack flying like an airplane, a different story completely.
[UNINTELLIGIBLE] configuration.
No, I wouldn't want to use just Newtonian to predict the aerodynamic characteristics hypersonically of that vehicle.
On the other hand, I wouldn't want to fly it there either.
It would burn up.
Blunt body is good.
98% of the energy goes into the shockwave.
98% of the energy goes into your vehicle when you're flying low.
A manned entry vehicle enters the earth's atmosphere.
You're going to be flying something like a segment of a sphere or at worst a right elliptical cone.
And Newtonian works good on that one, too, by the way.
We wind tunnel tested those for different configurations.
That gets your L/D on up to a 1.75 rather than 1.5.
Yes, sir.
Why couldn't any sort of straight winged configuration make the cross-range requirement?
The cross-range requirement is primarily angle of attack oriented.
The relationship between the CG payload in and payload out is a pretty good change.
And, if you had a straight wing, your straight wing cannot be placed in such a way that it trims subsonically and trims hypersonically.
Now, it can,at higher angles of attack with less L/D, you can make it work.
But down at 45 degrees you end up with a delta wing configuration.
Does that answer your question?
And that's a good exercise, though.
But you can go through it on a couple of sheets of paper and say ah-ha.
Because hypersonically you've got that CG, you've got your CP, you know where it has got to be placed, you get down to low angles of attack, you know where your CP is for that body, you know your CP is up at the quarter card and you know you've got 4.5 CGs.
It doesn't quite work.
It takes a delta to do that.
So it was more CG related than heating?
Yes, sir.
Although, the one that we mentioned, the 200 nautical mile cross-range with a straight wing, Langley did some tests on that, even at high angle of attack.
And right in where that straight wing ties back into the fuselage had super high heating rates.
But we could put little fillets in there and overcome that heating particular thing.
But, if you're going to go real fast, delta wings look better.
[LAUGHTER] All right.
Yes, ma'am.
When you were doing all this aerodynamic analysis, did you consider a perfect surface, or did you also do tests or analysis on what if the TPS fails?
Good question.
First of all, we have an external insulator.
And Bob Ried will talk to you about that, and Tom Moser probably has too.
Which meant that our main aluminum surface, which we want to keep quite cool, is different than the one that we initially designed.
We went back in, made models that simulated those differences, made calculations that simulated those differences.
They were very small corrections that needed to be made.
Then the other question that you had was suppose you lose a piece of TPS?
Which we did.
We've lost some chunks of TPS early in the program.
And you cannot do much testing on that because it's so little bitty and it's like a greased pencil kind of a thing.
But you go back in.
The first approach is we take some maximum things.
The pressure cannot be greater than total and it cannot be less than zero.
And so let's put those deltas into that area and into a nice area around there and see if it makes any difference to the aerodynamic coefficients.
And they're quite small and so we feel quite comfortable with that.
Now, from a heating, a different answer.
But, from an aerodynamic standpoint, does it affect the aerodynamics?
We scope it in that way.
It's a pretty small area.
Yes, sir.
Where was the greatest risk on the aerodynamic design?
During the first flight, was there anything you were particularly concerned about?
Oh, yeah.
[LAUGHTER] Getting it right.
I guess my biggest pucker factor is in the transitional phase.
I'd just seen so many things go wrong transonically that transonic wind tunnels miss.
It's a dynamic situation where you're moving.
And that was, personally, my greatest concern, did we get that right?
We did everything we could.
And it turned out that we did a good job, but I worried about that more than anything else.
Launch vehicle.
Primarily did we get the structural loads right and did we put enough margin in?
We did not use uncertainties in the pressure distributions.
Couldn't really figure out how to do that.
We had to have balance loads.
And if you said my CP can be higher or lower, do you put it higher over the whole vehicle or higher here and lower there?
And then you've got unbalanced loads.
But we did measure the wing bending moments, wing shears, wing torsion and also the attached loads in those.
And so we put some uncertainties on those.
And Tom has already been here.
I don't know if he mentioned how they tried to handle those.
But at least they had some of those things.
But we never figured out how to put uncertainties and pressure distributions and still be able to get all the balance loads for NASTRAN.
Maybe one of you all are sharp enough to figure out how to do something like that.
But that was the one on the ascent, had we adequately covered the uncertainties in pressure distributions with the way that we accomplished that by the uncertainties in the bending moments, torsion and shear?
Let's take a couple minute break and then we can resume with questions.
I am going to go down to my office and get one or two things which I think will be interesting that we can also talk about.
Take a minute or two.
Instead of one configuration, you've got one configuration times 5 degrees, 10 degrees, minus 5 degrees.
These split down the middle.
The same way here, body flap.
You've got one, two, three, four, five times five.
You've got 25 different configurations versus one.
And so it means more testing, but you're using the same facility.
It just means more hours of wind tunnel time, more hours of plotting and looking.
Say a few words about how you did the systems engineering or the systems integration between the aerodynamics and the flight control system [UNINTELLIGIBLE PHRASE].
All right.
Let's take flight control system.
We, in our division, were responsible for aerodynamics and flight dynamics.
And flight dynamics wasn't very well defined.
The Avionics Division was responsible for the GN&C system control system and also flight dynamics.
So, you've got to work together.
And you're going to get in one another's bailiwick.
That's part of good systems engineering.
You're going to do what the other fellow is going to redo, and he's going to do what you're going to redo.
But our responsibility was two-fold.
One is we had to have the right aerodynamic characteristics.
But, before we had the right aerodynamic characteristics, we had to have a configuration that should be able to live with a GN&C system.
The way we accomplished that is Ken Cox, is Ken going to be here with you?
No, OK. Ken Cox was head of the Avionics and GN&C System.
He gave us a functional GN&C system.
It didn't have all the strings in it, didn't have all of the relays and all the stuff there, but if we gave it a command to change that we knew how much it would change and how long it would take it to change.
An elevator position as a for instance.
And so that was called a functional GN&C system.
It did the right functions but it didn't do it the same method that the real GN&C system did.
We would take our aerodynamics and see that it would work.
And we would be able to make the maneuvers that were required.
And we would analyze it.
Ad we would be pretty satisfied that with those aerodynamics and with the variations that we were OK.
But that didn't say the system was OK. That just said that we had a pretty high probability of making it work.
Then we sent that aerodynamic database.
And all this is happening not ship it, throw it over the fence, but over a period of time we sent them the aerodynamic characteristics with the variations.
And then they beat it bloody flying everything they can think of with their GN&C system.
Fortunately, that worked out pretty well.
He would pick up a few things, but not anything major.
And, by the way, this is the only vehicle I know that happened that way on the Orbiter.
We didn't have any major configuration changes that we had to make once we got well into the program.
It stayed the same.
[UNINTELLIGIBLE PHRASE] That's right, yes.
And that was true.
That whole thing, not only the GN&C system but the hydraulics is when you give it the command, when you say I want a delta theta so much or delta E of so much, that it takes it about a quarter of a second to get there.
And that's a long time on a lag time when you pile all those things end to end up.
Now, structure is much the same way.
They gave us load indicators where they said we can take so much wing shear, so much wing torsion, so much wing bending, so much punch loads into the tank from the attach point in the front, so much punch loads into the tank on the backend, SRB and we have all the gimbling.
That's where we're doing the flight performance calculations, through the wind shears and the wind gust.
And we assure ourselves that we don't violate any of the load indicators.
But that does not say the vehicle is safe.
Then GN&C flies that one through that and gives all those conditions back over to the structures people, and they analyze the hound out of it.
They look at all the cracks and crevices.
And, if you have people that don't have a cooperative spirit, it will not work.
It just doesn't work.
So, you've got to learn how to get along with your buddies over on the other side.
Don't build your fiefdom up that your wall is so big and you control your own destiny because you don't.
And we had a super group of people to work with.
And, to me, that's what really made it work.
The same thing on heating, by the way.
They would give us heating indicators and we would fly it because that was a little more straightforward where they would give us the Q dot locals to Q dot references.
And we would look through all the Q dot references.
And we'd give them all the final set of trajectories and they would analyze it in detail.
But I look at it as here is a problem and I can work it like a piece of pie and go on this problem, go all the way down to the center, then go over and eat this piece and this piece.
It doesn't work that way.
You start around the edges and you cycle in on it.
You eat all the pie, but you accomplished it around the edges first and work out your bigger problems and then get down to the smaller problems.
Bass, I think the important point, too, is that no matter what system you work on, whether you work on the Apollo or you work on the Shuttle or whether you work on the new CEV, you're going to basically have to go through the same process.
It may be simpler or harder but you're going to have to go through the same process.
Absolutely.
Yes, sir.
You said there was no major configuration changes like around the program, but is there anything that you felt you were kind of stuck with like I wish we could have changed this?
Like not having the OMS pods stick out in the back like that?
What that a problem for aerodynamics?
No.
We lost the increased drag a little bit, but not enough to hurt anything.
And aerodynamics, it didn't bother me any that my L/D went down by 0.001.
But now the fellows that were flying it, they got in there, they said I can still go around the hack and I can still do all the things I want to.
But all that was done.
And once more, systems integration, you've got to include the crew.
GN&C system, it doesn't bother them in aero, it doesn't bother them in structures, but you've still got to fly.
And Dr.Gilvarry said you don't want the pilots talking bad about your airplanes in the bars so keep your crew happy.
[LAUGHTER] But, to answer your question, no, not really.
To meet the requirements, it's going to look pretty much the same.
If I designed it today, OK, let's go back, the same requirements now.
If you give me a different set of requirements, it's not going to look the same.
Bass, some people wanted to put some canards [UNINTELLIGIBLE PHRASE].
canards, that's not a joke.
That was done, but it was a big argument.
A canard is an aerodynamic surface part of the center of gravity.
And the beneficial part, let me tell you the good parts about canards.
canards, subsonically, I can give a pitch up moment and increase my lift.
If it's on the aft of the vehicle when I give a pitch up moment, I decrease my lift.
So, canards are wonderful.
But what canards do is it fouls up the flow over all of the rest of your vehicle.
And so, back to here, I've got one configuration.
Unsteady aerodynamics.
Heating.
Fixed configuration.
I put a canard out there.
Now I've got two more configurations, three more configurations, four more, five more, six more.
And it does a lot of grief with vortexes and stuff like that.
Now, if you had to have it, you have to have it.
Concorde had to have it, but that didn't mean that Shuttle had to have it.
It operated fine without canards.
I sure wouldn't put canards on there, if we had another shot at it.
Yes, ma'am.
Would it be easier, faster or cheaper to design the Shuttle, the Orbiter [UNINTELLIGIBLE PHRASE]?
You ask good questions, young lady.
[LAUGHTER] Yes and no.
With the tools we have and the foreknowledge that we have, having been there before, absolutely yes.
And I don't want to be negative, but you don't have the cooperative spirit going on that you had then.
I don't know why either.
The young engineers are sharper, no ifs, and or buts.
You all know a whole lot more than I ever knew, but you don't get along as well.
[LAUGHTER] This is mine.
I don't need you here.
I don't want you here.
Get out of here.
I'm going to do it my way.
And that's just an observation I made.
It might be that my knothole is too restricted.
But it goes up to pretty high management, too.
Not just at the working level.
A second thing is they asked me to come in and consult with them.
And they had lost technical capability.
Didn't use any engineers much after a certain time on the Shuttle.
And then, after the last accident, they had to bring the engineers in.
And they said I've never looked at that and I'm on the payroll.
The capability was greatly lacking, and so we developed some ways to bring them to speed in about a year and a half period of time which happened.
Observation.
By the way, I was pleased that you all got to class on time.
I told management to send them to ethics school.
Now, if there is a meeting and you tell somebody you will be there at that time, you be there at that time.
If you tell somebody you will make a deliverable at that time, you make a deliverable at that time.
Be a man or woman of your word.
And I found that lacking.
I don't know that ethics school helped too much that we sent them all to.
Well, it did, too, because the second day of class there were, I think, ten of them late.
And he sent them back and wouldn't let them in the class.
[LAUGHTER] But learn to get along with folks.
They are your friends.
They're your buddies.
They are going to be there for a lifetime.
And, yeah, you're going to have some grief.
And, yeah, he will give you some headaches or she will give you some headaches.
But you're all on the same team, and work hard towards that.
I know you've got to grit your teeth and spit sometimes, but that's major.
And that didn't cost you anything.
About the wind tunnel testing, firstly, could you have reduced that now with CFD?
And, secondly, you showed there were several hundred tests.
Yes.
And several tens of thousands of hours.
Yes.
Does that mean each test ran for several hundred hours?
Some tests ran for hundreds of hours.
That was the total number of hours for all those tests, not a per hour basis.
But, yeah, there were some hundred hour tests that went on.
To answer your question, does CFD reduce the number of wind tunnel hours?
Three areas that we're working in right now.
One reason is to bring the engineers up to technical speed where they can stand up and be counted.
And you can ask them a technical question, they can give you a technical answer and say this is my analysis and this is how I got that.
To do that we recommended back to Johnson Spacecraft Center that you better get your engineers knowing how to do wind tunnel tests and you've got to get them to know how to make the calculations to run these programs.
You've got to be able to do all that.
That's your integration contractor.
They need to have that capability.
And they've done that and they really came up to speed.
In the midst of that, they've also increased their CFD capabilities.
And I'm very envious of what you can do with CFD.
However, it hasn't decreased the amount of runs that they're requesting.
It's increasing the amount of runs that they're requesting because they will do this little bitty analysis and here is the cable tray going down that, and they'll say does it go sonic under the cable tray?
Well, I don't know.
What does the wind tunnel test say?
Well, we just have a data point here and a data point here and our CFD shows that it can go sonic under there, we need to test that to make sure that's there.
[LAUGHTER] And so it could but I don't think it will.
I think the better your tools get you want to make sure they're right.
Now, one day I think yes, that's going to happen, but right now there is still a lot of question, even in the CFD-er's mind, am I really right?
And they've always been accused of they can tell you what the right answer is if you give it to them.
[LAUGHTER] And that's really not true any longer.
That was for a number of years.
But that CFD is so great.
And they used overflow, by the way, is the one that's primary to the big workhorse right now, and it does a super job.
Was there a question?
All right.
Any other question?
I would like to address the question that was asked about [UNINTELLIGIBLE PHRASE].
Let me not talk necessarily about the Shuttle, but let me talk about Apollo for a moment.
And I mentioned it before.
In Apollo, we went to the moon with a single string avionic system.
We had one IMU, one computer on the Command and Service Module.
We had one service propulsion system to get you out of lunar orbit or to get you into lunar orbit.
And I think today it would be very difficult to do that.
I think you're probably going to have to have multiple strings.
You're going to have an escape system.
I think, in that context, it may be more difficult.
I do think the tools you have today will make it easier, but I do think what we know today, just like Bass said on understanding a little bit about the aerodynamics, you want more.
I think in some respects -- My comment is that the next time we go to the moon we're going to find out how hard it really was because I do think it's going to be hard to do.
At least that's my personal feeling.
And I think a lot of people agree with me.
One of the toughest decisions we ever made in the Space Program, and Bass and I were talking about it at breakfast, is when we decided to do Apollo 8.
Apollo 8 was probably the most fantastic mission that we ever came up with.
That was the first time we left the influence of earth's gravity, we went into orbit around the moon and we got out of orbit of the moon.
Now, we didn't land on the moon but did everything we had to do for the first time.
And I think that was probably the boldest decision that NASA and the government ever made.
Again, I look back, and I'm not sure how we would make that decision today.
It was pretty tough.
And, of course, one man led that attack, George Low who was Manager of the Apollo Program Office at Johnson Space Center, then became Deputy Administrator and then became President of Rensselaer Polytechnic Institute.
But that was really a fantastic project.
I will comment on the programs that I worked on.
Aerodynamics on Dyna-Soar, which was a defunked Air Force manned spacecraft program.
I worked on Mercury, Gemini, Apollo and Blue Gemini.
And each time I would go onto the next program and learn more, I would say we sure were lucky.
[LAUGHTER] When I was telling you about these new young engineers now that are up to speed where they didn't want to say anything.
Now that they're crunching numbers, they got a lot of criticism for the way we did it.
[LAUGHTER] And rightfully so.
A lot of capability that is there.
And you're going to have more capability today than you had yesterday, more capability on the next program than you had on the last program.
You're going to find some things to do and be able to do it better than the last program was done.
That's both good news and bad news.
Bad news is it will cost you more to do it and the good news is it will be done a whole lot better.
All right.
Any other question?
I thought it might be interesting, just to make sure you really have an understanding of what the Shuttle has to do, to sort of take them through the entry.
And maybe, Bass, you'll make some comments about the aerodynamics and I'll talk a little bit about the operations.
You're in orbit a couple hundred miles above the ground, and the first thing that you do is basically put the Orbiter, after you get everything stowed and ready for entry, so you're essentially going backwards, face to the ground.
And you burn the OMS engines.
And just, in terms of redundancy, we've got two engines.
You can cross-feed the fuel that is inside the OMS pod.
You can cross-feed from one side to the other.
So, even if you lose one OMS engine, you can still do a de-orbit burn just on the other engine.
Of course, you will have to turn a little bit so that you can burn through the CG.
If you lose both OMS engines, you can cross-feed the propellant into the reaction control system aft jets and you can come back using those.
And if you're having a real bad day and you run out, well, you know, you might have a fuel leak, for instance, and so you don't have enough propellant back there.
Actually, there was a procedure where you then would flip over and burn the remainder of your forward RCS propellant to just slow yourself down as much as you could.
We never had to do that.
In fact, I don't think we ever lost an OMS engine.
But we practice all these things.
Anyway, the point is, one way or the other, you slow yourself down.
Here is the earth and here is your orbit.
This is grossly out of scale.
You realize that on this scale your orbit is actually about like that.
In any case, you do your burn just enough to lower your perigee just about two ground level.
And now you're essentially in free-fall.
I mean you're in free-fall in orbit, but now you're gradually getting closer to the earth.
And so what you do is you now flip over and you put yourself in a position where when you actually hit the top upper reaches of the atmosphere -- And about 400,000 feet is what we call entry interface.
And, actually, the flight control system changes at that point and the computer actually goes into a different mode.
So, at this point, you're now at a 40 degree angle.
And, as Bass said, that gives you your blunt surfaces approaching.
And basically the sensors feel the deceleration.
And that's really how you know where you are in the entry.
You sense the deceleration.
That tells you what the dynamic pressure is.
During the early part of the entry, all of your control is by the reaction control system because you don't have enough dynamic pressure on the aerodynamic surfaces to be effective.
As you get down further and further into the atmosphere -- Let's see.
I think the number that comes to mind is ten.
I assume that's ten pounds per square foot dynamic pressure.
Yes.
Is when the ailerons become effective.
And so you have what is called a blended control system.
The first thing that you can do is actually control your roll, and so you disable the roll jets and you enable the ailerons.
But the pitch and the yaw is still controlled by the RCS.
And the critical thing, like Bass was saying, is you've got to keep your angle of attack constant with respect to your velocity vector because your aerodynamic heating depends on that.
If you get down to too low an angle now you're going to heat the surfaces, the upper part of the Orbiter and you're going to burn up.
You're controlling your angle around the velocity vector.
And now you can move about that.
You have a lift vector.
And now the whole name of the game, because, remember, this is a glider.
And so energy control is what is absolutely essential to make sure that you're going to end up at the threshold of the runway at about -- You said we were aiming for 150 knots.
We never actually made it.
Actually, about 200 knots was the touchdown speed.
But pretty good anyway.
We got close.
At every phase of your entry, knowing your energy and controlling it is critical.
For instance, for whatever reason, if your burn didn't go right and you think you're going to be short of the runway, we notched down the angle of attack just a little bit.
I think about 38 degrees was about the lowest you could safely go.
And you keep your lift vector.
You basically would fly the entire entry with your lift vector going away from the earth, and that maximizes your range.
But you actually target the entry so that you have more energy than you need because it's a lot easier to bleed off energy than somehow to get it back, especially when you don't have any engines.
What you normally do then is you will basically do a roll maneuver.
And you'll be rolling so that your angle of attack, with respect to your velocity vector, is always 40 degrees.
But essentially you're rolling around the velocity vector so you can now point your lift over to the side.
And, for instance, if you didn't have enough propellant when you were burning, and so now you actually have too much energy, what you're going to do -- Or, for instance, if you have a lot of cross-range that you have to make up, you could spend your entire entry over on your side like this.
And, in fact, there was even a case which we would simulate if you have an under-burn, now you have too much energy and you have to get really get rid of a lot of energy, you could actually roll 180 degrees.
And that would be pretty sporty.
[LAUGHTER] Again, nobody has ever had to do that.
But we did simulate it.
And I don't know what you would make of the aerodynamics coming in, but the point is you have a basic design that you're working with and then you try to develop operational procedures to make the system work in as many contingencies as possible.
Anyway, that's the deal.
You basically are rolling around the velocity vector to control both your energy and your cross-range.
Let's see.
Then at 20 psf the elevons actually get pitch affectivity, and so you disable the pitch jets.
And now both the pitch and the roll are controlled by the elevons.
The vertical stabilizer, as you can see, is in the shadow.
And so, actually, you don't get yaw affectivity out of the vertical stabilizer until just about transonic and subsonic, when you actually do the final pitch-over to get ready for going into the heading alignment circle.
And that's why sometimes, if you've seen pictures of the Shuttle as it approaches the landing at Kennedy Space Center, you will see little plumes come out the side.
And that's the yaw jets and the RCS which are still firing, because it's still supersonic when it gets over the Space Center.
And, if you're on the ground, you can actually hear this double sonic boom which is pretty impressive.
And I guess you get one from the front and one from the back of the vehicle, so it's a boom boom and then it goes around.
And then, of course, you hit the atmosphere at mach 25.
And so you're going through this entire aerodynamic regime down through hypersonic, supersonic, transonic, subsonic.
And I remember, Max Faget gave us a little talk about how the aerodynamic coefficients, I mean you were talking about them, but they change at different mach numbers, right?
Oh, yes.
And we mentioned this before, you know, why you really need a fly-by-wire system on this vehicle, because the control laws that you're using to do a roll maneuver at mach 15 are different from doing a roll maneuver at mach 8.
These program test inputs, you can hardly see them.
I mean you really didn't feel them.
They were just a couple of degrees that you would move the stick, you know, just little things.
But that collected enough data to get a much better determination of these coefficients.
You're very sensitive to things like beta angle.
Probably about two degrees.
If you go off more than that.
Actually, when you read the detailed description of the loss of Columbia, when they had a penetration of the forward edge of the wing, that disturbed the airflow.
And so the control system was fighting a really valiant fight to keep Columbia on target.
And they could see that on the ground.
There was more aerodynamic surface activity and the RCS jets kicked in.
In the end, of course, you run out of control authority and the vehicle eventually diverged and it actually broke up aerodynamically.
It was not an explosion.
It just broke up aerodynamically.
The hot gases in the wing were disrupting the airflow and eventually the wing was slumping, and so things were just getting worse and worse.
And I don't know exactly what went first.
Pieces were falling off along the way.
The same thing if you read the analysis of the Challenger accident on a second-by-second basis, you'll see the main engines were adjusting their thrust and trying to keep it -- The system was really working hard just doing what it was supposed to do.
Essentially the flight control system was trying to keep it within the control boundaries, and eventually it ran out of muscle because of the other failures.
And once you go to a bad beta angle or angle of attack the vehicle breaks up aerodynamically.
Bass, you might say a word about the aerodynamic coefficients and which ones were the hardest to determine and how big [were the errors?].
OK. Basically, aerodynamic coefficients are normal forces, side forces, rolling moments, yawing moments, pitching moments and those that change that like elevators, body flaps, rudders and speed breaks.
The coefficients, the biggest variations are in the low supersonic to transonic speed regimes primarily in the control variables.
The biggest ones were just before transition or any time that the elevators were in an up position where the flow couldn't quite decide whether to be attached or separated.
And so you want to avoid that.
Digging down, not very much variations in those at all, but coming in the up position pretty big variations in the rolling moments and yaw moments due to those.
And we tried to avoid those, of course, and henceforth with the CG location.
But those were the ones that probably were the toughest and we tried to stay away from.
Overall, hypersonics right on, subsonics second best, transonic the most difficult, low supersonic kind of in between there.
The coefficients, it's hard to talk about them on a percentage basis because CM is zero and the delta CM is an infinite percentage so you cannot really talk about them on a percentage basis.
But, when you look at the plots like CM versus alpha, very small variation after we got through with the flight test program.
Before we got through the flight test program, we probably cut the variations in half on things like pitching moments, yawing moments and rolling moments.
We did better than that on the pitching moments, yawing moments and rolling moments due to elevator positions when they were local angles of attack.
And those were cut quite significantly with the PTIs.
But still, even today, you don't just have one coefficient at one angle of attack, one angle of side slippage.
You evaluate that one plus the variation that's about it.
And I remember one of the big activities in the simulators before the first flight.
They would have the test flight crews, who were going to make these flights, do flying entries.
And they would actually vary the aerodynamic coefficients.
And, if you put in a big enough variation, three sigma and all of the parameters and they went in the wrong direction, you could always make it impossible to have a successful entry.
And so it was a real question where the managers, and Aaron, maybe you have something to say about that, you and Chris Kraft had to really make the decision when you knew enough and when you could fly safely.
You're going to hear Chris Kraft talk shortly.
But I remember after we landed the first mission there were a lot of people saying that the Shuttle wasn't going to work, all the tiles were going to come off.
The flight control system was a big issue because of the aerodynamics.
The flight control system, the aerodynamics wasn't going to work.
I don't know if you've ever been to the Johnson Space Center and to the Control Room, but I was sitting at the consul with Chris Kraft and he got on the Net and said -- Because there was a lot of criticism.
A lot of people wrote letters that the Shuttle would crash and wasn't going to make it.
And he got on the Net and said we're infinitely smarter today.
And I guess that summarized it, we were infinitely smarter and we did design a good system.
And I think that about summed it up, that we were infinitely smarter.
Any more questions?
Yeah.
Actually, it's an operational question.
OK.
I'm just wondering if the entire crew is trained on the whole landing procedure.
No.
We have two pilots just like in any airplane.
Some of the people who flew center seat as flight engineers were trained pilots.
And I would have been quite comfortable if they had landed the Shuttle.
But how many failures do you prepare for?
We have two people onboard who are trained to land.
I always thought to myself, we also have this auto land software which nobody has ever used, but if somehow all the other six people were incapacitated and it was just me, I would push the auto land sequence.
[LAUGHTER] And all you have to do is put the gear down manually, and I know how to do that.
But landing the Shuttle is really very complicated.
And they train hundreds and hundreds of hours in the simulator.
Some day we could spend a whole session talking about the Shuttle Training Aircraft.
It is one thing to land in the simulator, but nobody has ever died in a simulator, number one.
And, number two, you're looking at a TV screen.
You're not actually looking out at the runway.
You don't feel the bumpy aerodynamics.
And so they developed a Shuttle Training Aircraft, which is the Gulfstream II very heavily modified.
It's actually a flying simulator.
Your final approach, because your L/D on this vehicle, I don't know what it is exactly, is about five maybe.
I think five at best, yeah.
You're coming in at about a 20 degree approach angle, which may not seem like a lot.
But a commercial airplane comes in at about three degrees and the problem is, you take a regular airplane, and you point it down at a 20 degree angle,and your airspeed is going to go above your redline.
So you've got to make enough drag to simulate the Shuttle.
There's computers inside this Shuttle training aircraft.
The left seat is set up to be just like the Shuttle cockpit, and when the pilot makes the maneuvers, it basically goes back into the computers and they work the flight control system of the gulf stream, to make the gulf stream fly like the Shuttle.
The only way that they can produce enough drag to make it go at a constant speed at a 20 degree angle, is you have full speed brakes your gear is down, you have thrust reverses you run your engines in reverse, after your on the runway to slow youself down.
On the gulf stream, they run their engines at 90% reverse thrust while they're flying in the air.
It's an extraodinary vehicle.
I've flown in the center seat just to sit in it and you may be flying on this angle but it looks like your flying like this [LAUGHTER] It's really extraodinary.
Just one thing to remind you,a week from today we've asked for the initial report outline plus some reference material.
What we want to get a sense of what is your plans, what's the scope of what you're going to be looking at, what are you going to deal with and I hope by this time that you've located some reference materials you're going to be using.
We'll be sort of spiralling in on this so the idea is we'll take a look at what your plans are and then we can work together if you have some questions or having difficulties this gives us some opportunities to talk about it to make sure you're on the right track so you'll be doing something that you can be proud of, that you'll actually learn something from, so that's due a week from today.
It would be useful for me to have those electronically so that I can send them to Professor Cohen for him to look at.
I am happy to accept hard copy as well but I want them electronically Any questions?
Have a good weekend.
Allen Louviere started life as an airplane designer, went to Lamar University in Texas and then worked at General Dynamics in Fort Worth.
You told me about the YB-57.
I don't know what other planes you worked on up there, but -- Oh, B-58s, B-36s, F16s, F111s and B-57Fs.
Then they wanted to send him all around the world and had a young family.
And that was about the time that Johnson Space Center, which at that time was called the Manned Space Flight Center, was getting started in Houston in 1962.
So, Allen came to work.
And his specialty is mechanical systems.
We had a little discussion last night about some of the things that he has designed, but for me one of the most amazing is the landing gear on the Lunar Module.
I mean when you think that you can actually design the first piece of the hardware that touches down on another planet that's pretty exciting.
And, like most of the people who have been lecturing, he went onto work in similar systems for the Shuttle.
So, he also worked on the landing gear.
You can see a piece of tire here.
We'll talk about that landing gear, payload bay doors, and robotic arms.
And so he's going to share with us some of the experiences, both from the subsystem design point of view and then from the bigger systems engineering picture.
So, Allen, it is all yours.
Thank you.
The Lunar landing was sort of interesting.
The big problem was we didn't know what the lunar surface was like so you didn't know how to size the pad, which is always interesting.
The landing gear, we are going to talk about today, was much easier and it will do.
We're just going to talk about just mechanical systems.
It's not as romantic or ingratiating, but you've got to have them.
So, some of the avionics and other systems that you will see.
Actually, I should say one thing.
From the point of view of space engineering in general.
When you're building satellites or other space systems, the one thing that you try to avoid at all costs, I think this is far to say, is any mechanisms.
Because you send a mechanism into space and they fail.
They get too cold, they get too hot, they get vacuum welded.
Now, sometimes when you have people on a mission they can fix it.
I mean if you remember Apollo 16, the Lunar Rover, one of the fenders broke off.
And so they actually had to make another fender out of their checklist so that the dust didn't get all over them.
But basically that is the fear that all mechanical designers live in is that you're designing these things to work in an incredible environment.
And I'm sure you will talk about the landing gear.
That is one thing.
We know how to build landing gear for airplanes, but how do you keep them alive for two weeks in space in a rather unfriendly environment?
Well, the professor is right.
The problem you have is when you get these mechanical systems, if you have a lot of linkages, you're afraid they're going to weld themselves together.
And we've never had that happen but it certainly could.
The division I was in had all of these pieces.
The landing gear.
We had all the doors on Orbiter.
I think there were 25 or 35 doors on Orbiter that had to have mechanisms that had to open and close.
We built the repair tools.
We had a set of tools for these fellows to us.
And if that payload door didn't close that was really a bad day.
We put some tools onboard, a little wrench and a lot of cutting tools, and you will see some of that.
The manipulator, these are pretty well things that had been done before.
We had never done one of these before.
That's a whole different animal.
The separation system was for the separation system for the external tank.
There were three big separation systems.
It was not only a mechanical system but it was a pyro-system, it was an explosive-type system.
Believe or not, we had the ejection seats on about the first four or five flights.
We used the SR-71 Blackbird seats.
And we never had to use them, thank goodness.
We also had the docking system.
We had the Apollo system.
We had the Apollo-Soyuz.
And, if you look at the Orbiter, it is very close to Apollo-Soyuz with a little bit of modification to it.
That's the whole system.
But, fear not, we're not going to talk about all of those because that's just too many to talk about.
We're going to talk about two.
We're going to talk about the landing gear, wheels, brakes and tires.
And we've been building these for a long time.
It's more empirical design than I ever realized when we got into it.
We had never built a manipulator before.
And we started this as a technology study, and it ended up being the main cargo handling piece on the Orbiter.
Also, we did a lot of coordination and integration.
All mechanical systems depend a lot on other people.
He tells me that he wants to emphasize systems engineering, and I would like to second that because it's very important that you know that.
When you look at the systems engineering -- By the way, you don't have to be silent.
If you want to speak, now is the time.
I'm just here for a day.
I'm going to talk about this.
All engineers love the design part.
I did.
I found out that there was a little more to it than I first suspected.
If you look at requirements, schedules, cost and weight, most of the engineering work you think would be here in the requirements.
Once they are set, sometimes they're very difficult to change.
On Orbiter they were very difficult to change.
And we may have gotten some requirements that maybe we could have done a little better with had we changed them early.
And you will see the results of that, particularly on the landing gear.
That is probably the number one thing that we worry about at Johnson.
We like to keep these guys where they come back.
Now, these pieces right here are really engineering sort of things.
They relate to physics and maybe even Newtonian mechanics.
These don't.
Those are really estimates that somebody makes early.
And what they end being is a refinement process.
Most of these equations these fellows use from here are really empirically developed over many years.
They say here is the way this goes.
But all of these end up being estimates.
You think you're going to work there, but these three right here, schedule, cost and particularly for us in the aerospace weight turns out to be a big, big driver.
In fact, it will tell you what you're going to have to do in many cases.
Well, we're going to talk a lot about that.
We'll talk a little bit about test.
I presume you all, in your labs, do a little testing of some sort.
But we do a lot.
We do development testing early on.
For the manipulator, we did a little bit more than that.
We do qualification testing which is I'm going to go test this thing and see if it meets all the specifications that somebody has designed for it or the requirements.
And last but not least, after all of that is done, you do what is called a verification and certification test.
And it should be ready for flight.
I don't know what industries you're going to work on, but even in avionics, if all of your avionics people, there's a mechanical guy.
Even in avionics you'll have some sort of validation and certification test.
And these will drive you, too.
After all this is done, we did a flight test program.
And the initial test that came back from there made us do some changes on this.
And then you always get this.
Engineers are never satisfied with just making it good.
Better is what they want.
And program managers, where is Aaron?
He is not here today so I don't have to worry about him.
Program managers don't like that because what happens is this goes up and this goes up and that slows down, and that's not what they want to do so we have to go back and do that.
Enough of the intro.
These were essentially the first requirements that I could conjure up after all these years.
You've got to remember, this was 30 years ago.
Some of these may be a little off but they are about right.
It was going to be a freefall gear.
We're just going to get it started, open the doors and let the aerodynamics and just the weight of it drop that gear down and lock it in place.
The gear extension time was nominally ten seconds.
That was the max that we ever wanted it.
It would probably go down at about six or seven.
And we normally said the landing velocity, we don't want to drop that gear.
Once the gear is down, we don't want to go more than 225 knots.
Remember, this is in the early `70s and we're still trying to figure out what we're going to do.
But you've got to have some requirements to start with and these were the ones you started with.
We had a runway.
And we said the runway is going to be 12,500 feet.
We would like 15,000 feet, but because of the flight that the Orbiter took there were landing sites all over the world so we had to be sure that we'd get the landing sites with the 12,500 feet at least.
Cross-winds.
When you're coming in for a landing, if you have a cross-wind, 15 knots was about it.
We'll chat a little bit about that.
The deceleration, that was just for the brakes.
We had three hydraulic sources that we could use.
There were some auxiliary power units in the back that powered up the hydraulic system, and we had about three sources.
Most air craft just have two and a pneumonic system.
We did not have a pneumatic system.
But that worked very well, by the way.
The steering is nose gear steering.
And the hand controllers you could steer the front nose gear.
And it would go plus or minus nine degrees.
The turnover angle, if you just took two wheels and turned it over, the max that you could ever get out of that was about 63 degrees.
This set of requirements is about what we started with.
And, not to stop you, here are some more.
These turn out to be really pretty critical.
We will call them critical conditions that the Orbiter has to meet.
And this is landing weights.
And that is just how fast you're coming in when you hit the runway.
At 214,000 pounds, we were going to say we can take a sink rate of 9.6 feet per second, which is pretty fast.
And this is nominal.
If you go to Orbit and you come back and you make a nominal landing.
If it's 230,000 pounds, it's about six feet per second.
I came in yesterday on an 880.
I asked the pilot what he landed at, and it was 130 knots.
And I think the weight of that thing is about 150,000 to 170,000 pounds.
We're coming in with no engines so we're coming in pretty fast to be able to do that and set down.
This is an abort condition.
If we're on a mission and the controller is calling abort, you take off, turn around and come back and come back to the Cape.
In that case, we have a payload onboard so the weight is going to be a little heavier.
Now, the pilot has to be a little better here because he's got to have a little lower sink rate because we don't want to take all that load in the gear.
In fact, the transatlantic landing, when was working, there were only two sites.
One was at Dakar across the Atlantic and the other was at Rota, Spain.
I think Newfoundland now is one for the high inclination ones.
And AOA was once around.
You'd go up to orbit and you'd come right back down the next time around so you'd get back to the Cape.
That was essentially what we had.
This particular one was a driver also because, in the early days, the Air Force wanted to have a mission like this where you'd go up once, you deploy a satellite and you come back around.
And they were only one single orbit and you'd come back around.
Now, getting those payload bay doors open and getting the satellite out was, to say the least, an interesting design problem to do that.
You don't have a lot of time.
Anyway, these were the other drivers that you had to do.
These were the ones that really drive you because they'll tell you what the loads are in the system.
And this just tells you that all NASA guys have these acronyms.
This is a set of circumstances that says we want fail operational, fail operation, fail safe.
So, if it fails first you can still operate it, if it fails again at least you're fail safe and you at least preserve the crew.
We usually use fail op, fail op, fail safe.
In this case, in the landing gear, we only had two.
But we had a whole lot of ways to get this gear down.
And I'll show you.
It's kind of complicated to take a look at but if you release the gear it starts to fall just by gravity, but then we have a little hydraulic assist and then we have a pyro backup.
Whether you want that pyro to fire or not, it's going to fire every time you drop the gear.
So when the gear comes out, it comes out backwards.
And the airflow is this way which drops the gear down and locks it into place.
That gear is not a light gear.
I mean it's pretty heavy.
You're talking about 400 or 500 pounds coming down.
And that's just the nose.
The main gears are larger.
And then the mains operate about the same way.
And I'll show you where all this stuff is.
I have a little drawing here that will get you going.
The professor and I talked a little bit about design.
Yes.
You only drop the landing gear a few seconds before landing.
If one fails you still have time to do all the others?
I'm sorry?
You only drop the landing gear a few seconds before touchdown.
Yeah.
If one method fails, did you still have enough time to try one of the others?
We decided that all three of these were going to be in play every time.
We made sure that we got that gear down.
That's truly a bad day if that gear is not down.
All you do from the cockpit is just put down the gear handle, and then everything works at once.
Everything works at once.
You don't have to decide has the gear gone down and, if not, do something else.
It just all works.
I think you'll find that we tend, at Johnson Space Center, to overdesign on the side of safety.
And it really didn't matter to us anyway.
We really want that gear to go down and lock in place, and that's what this is all about.
The other things that we worried about, and probably where the requirements on this particular vehicle were the weakest, we said we could turn this thing around in 30 to 60 days at the Cape after every flight.
Well, that really didn't work out that way.
When you have that many systems onboard and there is so much to take a look at after you've gone to a flight, just the post flight operations is a long time.
And I'm not sure we ever made 30 days ever.
We may have made 60 once, I think, but I'm not sure.
And the other thing was the pad stay time.
First flight, as I recall, was on the pad for about four to six months.
And now you've got to make all your systems work.
It's like having shelf-life on this thing.
You've got to make this thing work.
It's got to function.
That was the other one and that was a big problem for us, particularly from tires.
They leak.
You sure don't want to have those tires leaking in about four to six months.
We had no landing propulsion system.
Once those guys came in, it was all energy management from an altitude all the way.
And the landing velocities were rather high, the weights were rather high.
Probably some of you fly.
Let's see.
A 737 weighs about, I'd say, 160,000 to 170,000 pounds.
And they come in just right at about a maximum of 175 miles an hour, but we're coming in with no engines at about 200.
The runway touchdown point, I've got a little drawing of a runway and I'll show you what that is, that turned out to be a real key issue.
When we started out that 12,500 feet I showed you was actually 10,000.
And after a while we decided without engines and then without reverse thrusters we really couldn't do that.
While that not only affected the Cape design but it affected where in the world we could land this thing, I think Guam was the other place.
We could go to Anderson Air Force Base on Guam, right?
Yeah.
And here is what frightens all mechanical engineers, limited testing, where we're only going to have about three or four flights.
We were going to have five.
It worked so well that they decided to go with four.
And everybody gritted their teeth on that one because we really wanted one more flight.
These are some more requirements of sorts, but these were outside requirements that you have to take what you get.
And so, when you do design work or whatever field you're going to work in, you're going to find you're not always your own boss on this.
You have to go with what you've got.
And I think that's part of the system of engineering.
We were told that the idea of moving the Orbiter with the 747 wasn't brought up immediately and that it was going to tried to be move with like strap-on engines.
So this part about no end-flight retraction [OVERLAPPING VOICES]?
The first design of the Orbiter, actually, the early designs had jet engines, but the weight -- Remember we talked about the weight earlier?
The weight just drove that out.
We couldn't do that.
Plus, when you reentered with those engines, you've got another problem with all the heating.
And you probably have the fuel that you have to worry about so you've got a lot of problems.
[UNINTELLIGIBLE PHRASE] Would that have been a real big problem [UNINTELLIGIBLE PHRASE]?
Yeah, it would cost you money and weight.
Remember those two things I told you up there?
It would cost you money and weight.
Remember now, Congress is a finicky bunch.
You don't go back to them unless you really have to, to get the money.
If Aaron is here, he could probably give you a lecture on that.
He used to lecture me on that, as a matter of fact.
The Carrier Aircraft or ferry, one of the guys in our division, a gentleman who just passed away recently, named John Kyker had been a B-24 pilot.
Actually, he and another fellow named Owen Morris built a little model of the 747 and a model of the Orbiter.
And they actually flew it.
The wingspan was about like that.
And they actually flew it to see if it would be all right to put that Orbiter on top of that model.
And, by golly, surprise, it worked like a charm.
Also, when they flew that model, they found that they had, on entry, a different setting on their elevon than what was being projected.
And, low and behold, they went back and they found, yes, they did and they changed it.
We don't have any in-flight retraction for what you were talking about.
We would get the gear stored on the pad and ready to go.
And, as I said, we want to be sure it comes down when we tell it to come down.
Actually, in the hanger the Orbiter is up on jacks.
And one of the last things they do before they wheel in a carrier mechanism that will take the Orbiter out over to the vehicle assembly building, you have to push the gear up manually.
And, actually, the final stage because it has to actually go in and lock inside the locking mechanism, someone is up there basically with a big pole and just push it up until you get it to click into place.
And that's terrible.
There is no hydraulic system to reconnect it.
Well, we get a little bit of hydraulic help but it's not enough.
You have to get manual labor.
But it's all because of the weight and cost that we were talking about.
And weight was really one of the big issues.
When you go to work, you're going to find out that all of those empirical things, not the design problems, not the Newtonian physics, is what really gets you.
Did they have any sort of like emergency barricades at landing sites if a tire went?
Yes, they did.
At Dakar we put up a net at the end of it.
And I think Dakar was one of the shorter runways.
I think we were worried that when we came in at Dakar with the kind of peddles we were carrying that the Orbiter was going to be very heavy.
It would be this 256,000 pound thing.
And that's a big thing with no engines.
And we'll talk about the brakes here a little bit.
The brakes were not really qualified for a short stop on a heavy vehicle.
Remember, in the early days we had no drag shoot?
No drag shoot.
Well, we're going to talk about that, too.
I'm going to try to talk about all this stuff until you guys go to sleep or something.
[LAUGHTER] Well, enough of that.
One last thing.
You will see that I'm worrying you to death on requirements.
Whether you like it or not, when you go into engineering you're going to find that if you get the requirements right you'll probably get the design right.
If they are not right you're going to have lots of trouble.
We didn't get them all right.
We didn't get them all right but it was good engineering work.
The tire pressure, we had to start somewhere so we said here is what we're going to do.
We're going to do some sizing on these tires.
And, on the landing gear, we did this before we even had a contractor, our first pressure was about 340.
I think the tires on your car are around 35 psi or something like that, if you're lucky on a good day and they're not leaking.
And the upgrade now is about 380.
I think they want to go to 410.
The wheel is aluminum.
The axel is aluminum.
We've actually bent the axel before.
We've landed that thing.
The first axel we had, we bent the axel.
It deflected a little bit and took the brakes out.
That was another one of those things so we had to up that.
It was sealed with two O rings.
That was a good one.
That was good to start with.
The initial design, as a matter of fact, was great because that four months we had to stay on the pad for the first flight we got no leakage, which was good.
The initial brake pads were beryllium.
Now, beryllium has one great feature that everybody liked.
The Langley Research Center had been working on brakes and they were using beryllium.
Goodrich had been working with the beryllium.
And it turns out that it has a great heat transfer capability, and we thought that was really going to be good.
And what do you say in your jargon?
Wrong.
It was wrong.
It did not work out right.
We had four rotors to start with.
We have five now.
If you're going to look at your brakes, I've got a picture up here, just like the front wheel brakes on your car, they have a caliper and you just put a pad on it.
But we have more pads than you do.
You will see some more about this.
This really got us.
A negative angle of attack, if you ever looked at the Orbiter when it was sitting on the runway, it sits nose down.
And when you come in and you drop that nose down it's going to load up that main gear.
And you will see what that does.
That might be all of the requirements that I have to start with.
Here is what the wheel looks like.
This particular drawing or graph is really what we started with.
And these are the summary things that we began with.
And this is the nose gear.
There are probably two or three things over here that are really important that we had to worry about.
One was a static load and the other was the weight for this thing.
This is all aluminum.
And we didn't want those going up.
At the time, inflation pressure for the nose was 300.
I think it's still a little more than that.
But the rest of this is just spec sort of things.
Now, when we did this, we had to start with something.
We did not have all this data.
What we did is we had a chart similar to this, in conjunction with Rockwell, and we worked back and forth.
That's another one of the interfaces that you had.
You always have a contractor.
And, of course, Rockwell had several contractors, I think, ten or twelve on this gear.
Getting this particular chart straight was very, very important to us.
Maybe some of the features, there is the inflation valve, the diameter here is about, I think this one was about 32 inches, 34.
And one of the design problems you get is how big a tire can I put in the wing or how big a tire can I put in the nose?
Now, the Orbiter wing is pretty good.
It's kind of deep.
The nose is not that roomy, but the nose gear is a smaller wheel.
It doesn't take the load.
That is low right now.
I think that's a little higher.
Here is the tire that goes with that.
The same sort of thing.
We start out with a sheet similar to this, not this neat and not this accurate, but now you'll notice that the tire weight is about 50 pounds.
Now we're about 100 pounds of tire just for one wheel and tire on the nose gear.
And there are two so now we're at 200 pounds.
I will tell you for sure, program managers will ask you what those weights are.
If Aaron were here he'd be fussing.
He'd say, yeah, you let that get away from you.
But that was the other one.
This is not terribly dramatic.
But I think the other thing is, see this thing, the design tire life was just for two landings.
I'm not sure we've ever done two landings on a nose gear.
I think after every flight we will take them off.
And, because of the way the runway is built, I will show you a little bit about the runway.
Here is a tire.
This is ALT-1 which is approach and landing test one.
If you can lift it.
I am going to pass it around for you.
This is ALT-1.
Here you will see that we've got 20 plies which is all this stuff here.
And that is what we used.
I think it is 28 now.
And you will see that at the top here is real rubber which is great, but synthetic rubber is better.
The tire we have now is synthetic rubber and is much better than what we have.
And you will see we have these beads in here.
Now, when you go back to the other piece here, we've got to realize that if you've ever tried to change a tire and you put the tire, all of you have bicycles.
Everybody in this town has got a bicycle, I think.
[LAUGHTER] Even him.
Anyway, when you try to change a tire, you can put one side in.
But getting this on would be really tough.
See, if you try to change this tire out it would be pretty hard.
You have to split the wheel so you can get that thing, and you put it in and then you put the wheel around the tire instead of putting the tire around the wheel.
And, when you see this thing, you will understand why.
I will pass this around for you and you can take a look.
It is heavy.
And those beads are really interesting.
They mean business.
They are tough.
They are steel.
Here is a nose gear.
This is STS-95.
Where you on 95?
Oh, I've got yours in here.
I've got yours.
Well, I'm going to let you tell it.
This hit pretty hard.
And if you take a look, you can barely see it right there, but a chunk of rubber came off.
And it's that good rubber that I was telling you about.
It's the natural rubber.
The ones we have now are better.
And when these tires hit, I think this particular one, they dropped the nose gear on rotation early.
And I think, when they did, they hit the tough part of the concrete at the Cape.
And you just get rubber everywhere.
Smoke and rubber and all kinds of stuff.
I think these are 32 or 34.
These are things that you really have to go and handle with care.
They are big.
You'll note there are no brakes on these front tires.
We will get that in a little bit.
Just to comment.
Last time we talked a little bit about what goes on when you actually land, but what Allen was saying is you flare out and then the pilot has to stay on the rear gear and hold the nose up for just the right amount of time.
If you hold it up for too long then you're going to lose the lift.
And, in the end, the nose gear is going to fall down too fast and you run the risk of busting your nose tire.
On the other hand, when you come down because of the negative angle, you actually, as Allen said, when you get down below horizontal, now you're putting more weight on the rear wheels.
And so, if you come down too early, now you run the risk of blowing your rear tires.
And so it's really a critical maneuver.
You have to hold it up just the right amount of time and then bring it down real slowly.
I think Lousma was a Marine, wasn't he?
Yeah.
Astronaut Lousma came in.
We didn't like him to fly too much because I think he did carrier landings or something.
I'm not sure what the guy did.
He'd come in and drop that gear down pretty hard and we'd all cringe thinking those tires were going to go.
Now, we saw the nose gear.
This is a repeat performance, except now we've got to make this wheel so we can put the brakes on it.
We've got to put the wheels on.
And now the same thing.
Now I'm going to tell you a little bit about integration with other people.
In the landing gear world, we were at the mercy of the structure's people.
You really wanted to be friends with them because they told you what the loads were.
And coordination with those people was really critical.
And I'm sure you're going to find that.
If you're in any kind of construction whatsoever, all the structures folks, you want to get them early on.
And the other people you want in there early are the operations people.
Those two people are the ones you want very, very early.
Here is the right load now.
If you just multiply that by four, you'd get about 250,000 pounds, 270,000 pounds that we could take.
And that is about what you're going to get on that back gear.
When you come down, you will get a little less than that.
We have about 20% to 30% margin empirically designed.
And we tested it at Wright-Patterson.
We tested this wheel.
This is a single O ring wheel.
The O ring is about right there.
And we tested this at Wright-Patterson with a side load on it, like on the flight he was on.
And I had to tell Aaron Cohen that every piece of the wheel came apart, which is not what you want to do with the program manager, but we had to start over.
So, we did.
And we beefed up the wheel, we beefed up the bolts.
We found that the two O rings we liked because they were good for leakage, but we also found that when you put them on that the wheel failed because of the O rings.
Also they failed because we were putting the wrong lubricant on those bolts, and that took a while.
And that particular failure, I just talked to you about, had both those problems.
How did the lubricant cause the wheel to come apart?
They couldn't get the right torque on the bolts.
I think they put too much torque on the bolts.
We never really found out, but we changed lubricants and that went away.
I told you now this is not Newtonian physics we're talking about here.
We're talking about empirical design.
And you had to go test this stuff.
And we did.
Let's see, what was the other point?
Here is where the axel fits in here.
I will show you where the brakes go here in just a minute.
I don't have a main gear tire cut like that one, but it's much larger.
It's about 40.
I think this one was 42 inches.
They tell me that they want to go to 44 inches for a little bigger wheel.
In the beginning, again, weight and cost stopped us from going any farther than that.
Mainly weight in that case.
Again, this was the critical wheel that we had to work with the structures people.
Our structures group worked it and Rockwell structures group worked it, so we had two structures groups that were working together and we had to wait until they gave us a load.
But when you have a design you cannot really wait until all the loads are in.
You have to start with something, so we started with a design that we thought was conservative.
It was not.
But we started with one that we thought was conservative and it did not do it.
You can see that the max load was 142,000 pounds per wheel which gave us a nice margin.
Of course, those are the aft wheels, the main gear.
And we liked that margin on there.
And the inflation pressure is about 370 on that.
This thing is like a stick of dynamite.
If it blows you have really got some problems.
I mean you have shrapnel everywhere.
But when you do the test it is always behind the cage or something where you can do it.
Again, safety was our problem.
When I saw that last orbiter come in, I thought for sure that something had happened to the wheel well and that the explosion was in the wheel well.
It terrified me.
Partially it was not the wheels that went.
They went eventually, but if you blow four of those wheels, two on each side, that's not a good day at all.
The Orbiter is about the same size as a 737.
Everybody flies a Boeing 737 about the same size, but they land a little softer and not quite as heavy.
But the geometry, a planned form looks about the same.
They don't have the payload bay that we have, but it is pretty close.
And here are the big guys.
Now you will see the hydraulic systems.
There are all the hydraulic systems that go in there.
And trying to move one of those tires by yourself is a chore.
It will get you in shape.
Rolling that thing around at the Cape will get you in shape.
And it's the same thing.
It's a split wheel that we have.
This one has the same problem.
You cannot see it but it is right there.
There is a big piece gone from right there.
And, again, it's the runway.
If you hit the runway on the high coarseness this is what you get.
I'm going to show you the runway here in just a minute, or a little sketch of it and you'll be able to see it, but you can see that that is a fairly big strut.
And we needed it.
How much travel does the strut have?
You mean after it's down and locked?
Oh, I would say probably six inches to a foot, just depending on which ones that Rockwell picked.
I'm not sure which one they have now.
We had a pretty stiff strut to start with.
I think they've done a little work on that.
That's a good question, by the way.
That does affect it a whole lot because you don't want to drop that gear too low because those doors are open.
If you drop that gear too low and that strut goes up, you're going to get those doors, and there goes that turnaround time we were talking about.
There goes the turnaround time.
I've got another one of my goodies here.
This is not the beryllium.
The beryllium had one big problem.
We thought that the heat transfer was great but the homogeneity of the heating was not good.
You would get spots.
Once you did that it would warp the brake.
When it warped the break then it would do all sorts of bad things.
When you look at this, here are the stators and here are the rotors.
This was a stator.
And what this is is they had two sets of stators on each of these.
And then, of course, the rotor went all the way around.
But here is a brake.
And you will see that it's only a piece of a brake.
And you are going to find out it is very, very heavy.
And there is really not much I could tell you about these, except that the one good thing we did was we insisted that Rockwell leave enough room in here for five stators and rotors.
We started with four and right now we have five.
And so we did it partially right anyway to start with.
Here is the axel, and this will fit on all of these.
And we thought we did about 25 stops with this thing, but I think we probably end up somewhere between four and ten.
Again, that's just money.
And, again, that's the turnaround time we were talking about.
So, all of the things we thought we could do, systems engineering things, did not work out quite right.
So the stators had to pass?
Yeah.
The caliber just pushes against it?
Just pushes, yeah.
It just shoves it.
There is the caliber so you just close that thing up and you just squeeze those guys together and you watch the smoke come out.
Did the caliber float at all like on a modern one?
We found that we had to do an orphus hydraulic system.
We found out the problem you're talking about, they had a funny name for it, but it was the one you're getting at.
And what was happening is that the brake was cycling because it was not orphused.
Once we orphused that thing it worked very, very smooth.
And, just like your front wheels, it just doesn't look much different than your front wheels, except they are a lot bigger.
And how would you like to have that pad on your car?
You would never have to have them replaced.
[LAUGHTER] Here you go.
This is a mechanical engineers' dream.
Here are more linkages than you could ever want.
You can have all sorts of fun with this kind of linkage.
This is the forward part of the Orbiter.
The nose comes around kind of like that.
And you can see that we've got that gear stuffed in there pretty good.
And some of the things that are on the door are sliced out, so when you look at the door in there you have just a whole lot of things in there.
And what happens is, remember I said the first thing is we have a little hydraulic actuator that starts this thing?
And, whether you want to or not, it opens those doors.
And we will fire this bungee right here.
This is a pyrotechnic thing.
And no matter what, we are going to fire that thing to be sure that if the hydraulic system isn't going to that, we are going to drop that thing with a pyro.
So, it opens the doors.
Once it opens and you start that gear down, remember, you've got about 200 something knots of wind that is going to drop.
So, this is going to help you.
It comes down, it folds down that way.
Our only comment to Rockwell was that we didn't like all this.
The leakages were even more than that when we started, and we had to reduce those down.
There is a little fitting right up here that you can get a little bit of help to get the thing back up.
But you do need the guys with their polls shoving it back up.
There is the wheel.
And once it's down, there is no brake on that one.
This is the forward one.
This will give you a little better idea.
This doesn't look too much different.
Forward is this way.
If you go that way, you see the gear has come down and is now locked into place.
In the other picture you couldn't see these little yokes, but when the strut comes up those yokes come up and grab it.
There is the extender strut.
Essentially, the drag shut, it's a solid piece.
I'm not sure what you could say other than the fact that that right there is the booster actuator that I was talking about, the pyro actuator.
And it is going to fire every time no matter what.
The same thing for the mains, except they are bigger.
One problem we had with this particular one was these doors.
Those are big doors.
They are bigger than probably two of these tables.
Maybe like that and wider so they are big.
And the problem is sealing them.
When you're coming in and you're going to re-enter, you've got to get a seal on there that is going to work.
Most of these were bolt seals.
You just put the Freon and they squeezed closed.
There is no seal sticking out.
Of course, it has got the re-entry material on it.
But, again, these were part of our problem.
And those seals were probably the trickiest things that we had.
And here professor is at extend/retract actuator, but it still won't do it all.
You could hook up a hydraulic system to get this gear back up.
Forward is that way.
But you need a lot of help.
Again, it was a weight problem.
I am going to show you this one, but I had to go to the Space Center Houston because I couldn't get a picture of the Orbiter.
So, this is a little mockup.
And here is the gear.
And what I really want to show you is, see that little guy right there, that's the pyro system that kicked that thing out.
They call it a bungee.
It's a Rockwell term, but it's not like a bungee.
It's really actually like a pyro.
And when that strut comes back up here are those yokes I was telling you about.
It just clamps around that strut and it hold it up.
That's probably the only thing you can see that is worthwhile on this picture.
Now we're going to talk a little bit about integration, an interesting problem.
Rockwell was our prime integrator.
They integrated everything.
We helped them, but they had all these people as contractors.
Not only did Rockwell have to deal with us.
Rockwell had to deal with all these contractors that they had.
The wheels were made by BF Goodrich.
Not Goodyear, Goodrich, which is now Michelin, I think.
And they flew it on the F-14 and C-5a which is a pretty big airplane.
We didn't have a four-wheel truck like the big aircraft did like a 747.
We thought that would be a great idea, but we didn't have the room in there to put the whole thing in.
The brakes were BF Goodrich, the same ones.
They were the ones that really wanted to go with a beryllium.
It turned out that they passed all of the initial tests for the beryllium, but the beryllium had no life, so we never really had a test failure.
But we had a test of things we really didn't like the way it was going so we changed.
They changed, I think, in 1990 or 1985 to the carbon brake that I was showing you just a minute ago.
The nose gear steering is a bi-steeror.
Nose gear steering is after you land and you get the nose gear down you can steer.
It turns out that the astronauts did not like to use the nose gear steering, so what they would do is get on the brakes.
They would do the brakes like this.
And they'd steer that way.
When you do that you heat those brakes up pretty good.
You heat the brakes up.
And, besides that, there was a single point failure in that nose gear steering.
If that thing failed, and it failed in a bad position, it might be nine degrees off, and so you're going to run the Orbiter off there.
I had to agree with the astronauts that probably the right thing to do was just get on those brakes and steer with those guys.
Menasco, they built struts for almost everybody.
HydroAir, they had all kinds of actuators.
And Rockwell did all the bar linkages that I was showing you about.
The antiskid system was another one of those important things.
What is it on your cars these days?
The ABS system.
This is our ABS system right here.
And we had one.
And we had nose gear steering.
We had all that good stuff.
You've got to remember now this is 30 years ago so it's a little different.
There we go.
I had to put that in there.
I like this picture.
There is nothing technical about it but I like it.
Isn't that neat?
Look at those little skinny wheels on that big vehicle.
Isn't that something?
[LAUGHTER] It looks like somebody made a mistake.
Anyway, it's coming in there.
You will note there are no drag shoots.
The initial design actually had a drag shoot.
One of the guys in our division decided that we didn't need a drag shoot so he put in an improvement which reduced the cost.
That was a mistake again, so we took the drag shoot off.
And I eventually had to put it back on anyway.
All engineers are not exactly right, but this thing flew well.
The gears really worked out real well.
We were tickled pink whenever we would see that thing fly.
I may have already told you.
I had to put that in there.
I like that picture.
This was a dilemma.
The tires never really failed to the spec that we gave them.
We didn't like it because we didn't think we would get the life that we wanted out of it.
And so we had running discussions.
Maybe not battles, but certainly running discussions with Goodrich on that tire.
We got excessive testing tire damage with a dynamometer test where you do all the testing and we put the side loads on.
And we just didn't like what was happening.
Even though it passed this we went to, say, 20% over.
We were really getting failures that we really didn't like.
And we knew, from the very beginning after the test, that we were not going to get the five or six landings that we thought with every tire set.
On the mains you'd get one landing and that's it.
And you can see why you would get these big hunks.
I think they are better with the new runway or with the way they fix the runway.
I talked about the wheel.
The wheel cracked.
We had a wheel crack with a load.
The bolt lubricant did not work.
We couldn't get the right torque.
The two O rings caused a failure in the split wheel.
The bolts broke and we failed the 1,000 mile test.
That doesn't sound great, does it?
But it really was because this was an empirical design.
It worked fine because it met the spec that we had.
And what drives you, as engineers you might not like it, but program managers have their problems, too.
They have costs they have to worry about, they've got schedules they have got to worry about, and they get you to do things that will work every time.
And then later on you can make the improvements.
The beryllium brakes, we had cracked hot spots on the pad, we had thermal expansion and some melting.
Goodrich kept telling us that those cracks did not matter for flight, but we didn't like it and we fussed with them a lot.
And we fussed with Rockwell a lot.
And after maybe ten years they finally changed them.
The entry displacement was lower than expected.
We thought sure that the beryllium would be a great way to do it, and it didn't work out.
And I talked to you a little bit about the hydraulic system causing that wobble.
That was the test results on our outline.
Let's talk a little bit about the runway.
I have more goodies for you.
The runway was about 15,000 feet at the Cape.
Before the flight test, how do you actual simulate flight conditions, especially with the wings to simulate heat dissipation?
You mean dropping the gear?
Yeah.
I don't think we ever really did.
I don't think we ever had a wind load on it.
But we tested that gear.
One time we did it with a lower pressure just to see if the hydraulic pressure would work.
And that may give us a little bit of something, but that's all we ever did.
We had high confidence in that pyro.
If you ever heard it go off.
But you rely on the wing load to dissipate the heat, right?
One of the things, yeah.
And gravity, of course.
You're getting a gravity down, too, on the thing.
So, you're getting wind, you're getting gravity, you're getting pyro, you're getting hydraulic.
No, to dissipate the heat in the brakes.
Not the gear, but to dissipate the heat from the brakes you rely on the wing load.
Oh, sure.
But you couldn't test that on the ground?
I don't think we ever put it in a wind tunnel or anything.
If we did, I don't remember.
But we did check a lot to see what the heating was.
And we did it on a dynamometer which is like you do to your car tires.
No, I don't think we ever did.
That's a good question.
I wonder why we didn't do that.
I had to think about that one.
Here is the runway.
A lot of people in NASA have built runways and looked at them.
And what they did is they put these grooves in there to get the water off, just like they do on the highways now.
And you will see that is the grooved runways right there.
It sits in there about like that.
And I will pass this around and let you see it.
And you will see that it is really rough.
And you can see hitting that surface at those speeds is not really good.
What they have done, after I left by the way, which is a great thing to do, is they shortened the high friction part of the runway and they pinned it with like a ball pinned hammer.
They pinned it like they do highways.
That got our friction factor down.
We missed the friction factor, by the way.
And they put a thing called a corduroy on the front.
And here is the corduroy.
You want to take a look at that.
It's like that.
And so now the runway looks like this.
I think if we were going to fly the Orbiter some more that probably with this runway now at the Cape we would probably get a little better use out of the tires.
It turns out that trying to determine the friction factor here was not as easy as we thought.
And you will see when you see that the thing is not a smooth surface.
It's a pitted surface and it has little things that stick up.
And so if you pin it down like you do like a highway and get that it's much better.
We've gone through the requirements.
We've gone through some of the testing.
Now let's talk about operational findings that we found on this thing.
Remember, if you're going to be an engineer, which you are I hope, you're going to find out that all three of these things will be a problem for you.
You'll have to look at the requirements, you will have to look at the testing, and you will have to look at the operations of the thing.
If you work for a company they will put you way ahead if you tell them we ought to go find out how this thing is operating with our customers.
Here is what we found.
Our deployment velocity, when we dropped the gear on landing, was 312 knots operational.
Good.
We were right on.
Landing velocity was 225.
We actually came in once at 232 or a little higher than that.
And that was the Marine landing.
I'm not quite sure he was wanting to do those carrier landings.
The operational weight was higher.
We designed for 214 to 230.
We ended up with about 225 to 232 which is not bad because we had enough margin in those tires and wheels to take it.
The sink rates were not bad.
6.7 is about what they would always land at, which was really pretty good.
Now you can see that our design was not that far off.
The negative angle of attack I want to talk about here.
Let me show you it.
For some reason the initial design, which I guess we were all part of, had a negative angle of attack if you would take this route.
It was about four degrees down like that.
Now, if you take a look at this, when you come in here you will see that if you're going to get a load in your landing, so the elevons are going to be up a little bit, what you're going to do is all those loads are going to dump right straight into that back gear.
And that's really not what you want.
You ask yourself, well, why don't you just raise this gear?
And the answer is money [LAUGHTER] and weight.
But the initial thought was we ought to have it to look like that and do that and we will take the load off the brakes.
I'm not sure that was one of our better moves but that wasn't bad.
We talked a little bit about this [the negative angle of attack?].
The nose gear steering was never really used.
I'm not sure they do it now.
Oh, no, they fixed it.
Have they fixed it?
Yeah, they use it.
In fact, it was after our flight.
That is when they finally decided to fix it.
We're going to talk about his flight in just a minute here.
I went through great lengths to get his.
We had one tire failure on 51D.
And, after our operations, we did find that we could get John Young, who was chief astronaut at the time, to say, yes, we're going to change how we do these elevons on landing so we will take some of the load off that gear.
Let's review it.
We had the requirements, we had the design, we had the test results and now we've got the operational results.
The whole systems engineering scheme is to go from start to finish.
And this is what we did.
I may have made this sound more negative than it was.
It was great fun and it worked really well.
I mean it worked well.
As an example, and what the professor was talking about a minute ago, we did one of these.
This is loads here, but we did one for heating early on.
And we had a little computer, which was one of the few on the center at the time.
It was a Wang desktop.
They've never seen that.
I know that.
[LAUGHTER] It's a foreign word.
It's about this big.
And I think it had a max of about 256k and that was it.
We, to keep up with Rockwell, instead of loads, did heating because we were more worried about the brakes than we were the tires.
And, if you look at this curve, this one sets that nose gear down a little early.
You come in on the mains and you drop that nose gear down.
And then all of a sudden you get this spike.
Well, you can imagine if you get this spike here, the heating is going to be higher from there on.
If you keep that nose up, like he was talking about a minute ago, and you just wait a while and you get way down the runway or down the runway some place, you get a little lower load and it's a little easier on those tires and on the brakes.
It gives you a little longer rollout, but that's not bad because we've got plenty of room.
Now, some of these fellows had a problem with this one.
This is 51D.
This is the infamous flight for the professor.
See that tire right there?
It is blown.
It is the only tire that we blew in the whole of the flight test.
And what are we at 210 or 214 along flights, and it is the only tire.
I'm sorry, 114.
It's the only tire that we blew.
And I tried to ask him last night what they did.
I'll tell the story.
Well, I was going to tell you, he had a tremendous side load.
He had a tremendous 15 knot crosswind that they got.
And the gentleman who was flying was a fellow named Bobko.
He's kind of fun to talk to.
He did a good job getting that thing in.
And they got pretty far down the runway.
I think it was only 200 feet, our guys told me, before that tire blew.
And you were almost at a stop.
You were just going a little slow.
But the brakes were heating up because he was on those brakes fighting that side load.
More operational findings.
We shot-blasted the center strip at the runway.
We improved the groves, like I was showing you, and we put the corduroy in.
And that all happened after I was there, but I still deal with the guys out in the center and they helped me out to get this.
The design problem was the wing volume that we had.
We started 42.
They tell me they are going to go to 44.
I don't know if they have.
The tire pressure was 370.
We could go to 410.
I think the new tire will have 410.
We talked a little bit about lengthening the nose gear which would relieve some of that problem with the nose down.
It turned out the proposal was prepared.
It was presented.
And they decided that they really didn't want to do it because it would cost too much money and was too much weight.
The Orbiter, at one time, had a CG problem.
And I don't think they wanted to weight up there with that much of a moment arm on it, was one of the reasons.
The carbon brakes were added in 1990.
They worked great.
The new synthetic tire in 1992, which is different from that tire right there, worked very, very well.
The drag shoot.
We put the drag shoot back on.
And this is nothing more but a [UNINTELLIGIBLE] picture, but there is that drag shoot right there.
We have that drag shoot on now.
And they tell me that helps a lot.
If you put that drag shoot on, you save the brakes and you save those tires, too.
And it also tends to stabilize you to keep it going straight down the runway.
Even with the crosswind?
Even with the crosswind, yeah.
And you have to realize that this has to be designed not just for a landing at Kennedy Space Center or at Edwards where you have a three mile long runway.
Like Allen said, you've got contingency runways around the world where you might be landing with just two miles.
And that is when this stuff is really going to save your bacon.
We were proponents of the drag shoot for a long time mainly to save those tires and the brakes and get the cost back in there.
It was added in 1992.
And so with all of that.
After I told you all of that, I was a little bit negative, just to let you know, that everything doesn't go smooth.
If it does not go smoothly when you do engineering, I would suggest to you that you're probably going to have the same kind of situations.
Just concluding comments.
The tire design on development had margins to meet all the initial requirements, and they did fine.
It is still flying and flies well, a good vehicle.
We only had one tire failure, and the professor was a witness there, he was there.
We could have had a longer life on all that, but the clear issue was really weights and money.
Again, it is one of those systems engineering things where you have to trade that off.
The wheels performed a design.
Even better because we didn't have any leakage in six months on the pad which was great for us.
The beryllium brakes functioned but were sporadically marginal.
The KSC runway had friction and rough surfaces but that is fixed so everything is pretty good now.
The initial design of the requirements and improvements have continued, and I am sure that they will continue until the Orbiter doesn't fly anymore.
And I am going to stop right there.
You've had enough of me for a little bit.
Any questions on this stuff?
What did you think about that concrete?
Did you pass that stuff around?
Isn't that something?
It fools you, doesn't it?
I had never been to the Cape.
It is a lot rougher than you think.
And you see that stuff and say what were we thinking?
That's no good for a big tire like that.
OK, gang.
I will just finish the story of this 51D flight that Allen mentioned.
It was the sixteenth flight of the Shuttle.
It was my first flight.
And actually there were seven people onboard.
Six of us were rookies, so we had never been in space before.
And the commander had only been in space once and he was in charge.
And it was quite an exciting flight because we had a malfunctioning satellite, we had to do an unplanned spacewalk.
We did an unplanned rendezvous with a satellite.
And so lots of stuff had happened.
And then we came in re-entry.
And I just remember the deal was, as Allen said, the nose wheel steering was built in, but it was a single string system.
There was only one hydraulic loop in the nose gear.
And if that went hard over then you will leave the runway.
And so Gastros decided that until they made that redundant we weren't going to use the nose wheel steering.
In order to steer down the runway, you put even more or less pressure on the right or the left brake.
It turned out that we landed with the max crosswind.
We had a crosswind from the right side, I believe, and so wanted to blow the Orbiter over to the left side of the runway.
And so the commander had to put extra brakes on the right wheel.
And so that was what was heating up.
And I just remember, because I was sitting right behind him up on the flight deck, we touched down, everything went fine down the runway, the nose wheel came down and we could feel the deceleration.
I thought we were just about stopped.
And I remember thinking to myself it's all over, nothing can go wrong now.
Boom, the whole Orbiter shook.
I mean I really thought that one of the fuel tanks had blown up or something.
It was really impressive.
And, like Allen said, these things put out a lot of shrapnel.
In fact, after most flights, sometimes you will see the crew get out and walk around and kind of kick the tires and just sort of pat the Orbiter and say thank you for a nice flight and all that.
But they made everybody get out of there because the other tires, obviously, were overheated as well and they were worried that they were break.
But, as Yogi Bear said, it ain't over till it's over.
And it was exciting.
But after that they fixed the nose wheel steering, they got rid of the beryllium brakes and replaced them with carbon, they put in the drag shoot, and we really haven't had any problems since.
And I think it's a really good example of the whole engineering process where you build in margin.
And that is absolutely critical when you're designing a system.
And then going through the testing and then the operation and then closing the loop.
Once you find out how these systems really operate then, if you have the ability to make changes, you can go in and improve them.
Let's take our two-minute break, stand up, turn around and then we'll take up again with the manipulator systems.
Start up again with a different mechanical system.
In this case the manipulator system which is the space crane.
That's a good term for it.
It is a space crane.
This is a whole different way we did this.
When the Orbiter was built there were several sized payload bays.
One was 30 feet and one was 60 feet.
And it turned out that we did not really have a system to handle the payload where you were going to carry the payload, but we really didn't have them at the time.
Altitude and inclination varied.
Here was the significant capability we had to do.
We had to do 65,000 pounds, 15 foot in diameter and about 60 feet long.
That is about the size of a greyhound bus.
And we really did not have any system to handle it.
The thoughts were don't worry, we will spring it out, we'll throw it out, but we had no retrievalment methods.
Besides that, another systems engineering problem, most of the resources in the program were on the Orbiter, or just to get the shuttle flying.
And there was really not a lot of attention paid to the payloads in the early days.
In addition to that, when you look back, the ongoing programs were Skylab.
We had Skylab still going.
We had Apollo-Soyez just about ready to fly.
And then the prime objective, the Orbiter.
All these things contributed to the fact that it was a little later in getting this started than perhaps it should have been.
All the NASA centers had a concept.
Marshall had a concept.
We did not have a concept at Johnson at the time.
We were doing some off-line work on manipulators, but the initial part was not for the Shuttle.
We had some systems that were pretty good for deployment, but we had no system for retrieval at the time.
That was a problem.
The manipulator became a crane.
It essentially became a contender once we understood a little bit more about the requirements.
And the one that came up that said we have to go capture a satellite.
That is the one that was the real driver for the manipulator.
We had some studies going on.
I guess you guys call it robotics.
And we had found that we could get from General Electric some of the Atomic Energy Commission which they used to handle the very critical elements.
And we were just working on it.
And this was a side issue.
We were doing this technology.
It had nothing to do with the Orbiter at the time.
And we decided, after a while, that we liked the way it worked and what we would do is just make an extension of that, so we did.
And we decided that we would try to go to handling system.
And we did.
I'm not sure.
Is master slave a good term for robotics these days?
Let me tell you what a master slave is.
The hand controllers or whatever you're using looks just like the manipulator.
You have a little bitty one over here.
You've got a big one over here.
We were using a master slave kind of system.
And we found out right off that if you've made an arm that long and you try to put it in the crew cabin, it took up a lot of volume in the crew cabin.
And so we didn't have that much room to start with.
That became an issue right off.
The other issue was that we found out that when we were working with the AEC kind of things that dexterity and feedback was very important because they didn't want to drop this thing.
They wanted to be sure they could handle it.
But we really didn't need that.
We decided that we had to have a larger system than that.
And we also found that during these tests the fellow would hold his arms up and would get tired.
We decided if we got a barber he would be the guy that could operate this thing better than any of us.
[LAUGHTER] That's a 1G problem.
Even there, for training, he would get tired so easily.
But we never found a barber that could do it.
And we kept the master slave concept for future testing.
Just so we understand what a master slave is, let's take a quick look at this.
Here is the master.
You can go over and grab the hand controller right up here.
And, as you move that hand controller, this piece over here moves exactly like you move the hand controller and the arm.
Now, that is interesting because if this is one foot and that is 50 feet, you get a radio of about 1:50.
Instead of being able to move it like we wanted, you now have to ratchet the thing every time you did it.
It took more space.
So, we decided that really was not a good idea.
In addition to that, we found that the manipulator that we had was really too small for what we really wanted to do.
And the other problem was that the GE hydraulic manipulator was probably not what we were going to fly in space.
We did not want to fly a hydraulic system that would leak.
Now we had a problem between hydraulics and electrics, which now we had to do some modeling on.
Of all things, we had some money from the NASA Medical Group.
We told them what we wanted to do, they gave us some money and we went and bought a manipulator from GE.
They got it from Pittsburg Plate and Glass.
It was about 25 to 27 feet long and it had a suction end effector on it.
And what they would do is pick up these big pieces of glass and move them.
We bought their manipulator for about $50,000, I think, but it was medical money of all things.
And it had a master slave and it had a vacuum end detector.
We decided that there were several things that we had to analyze just to get this thing started.
One was the master slave, and we decided we wanted an alternate concept.
That one didn't even work.
The feedback versus fixed hand controllers was the other one.
A lot of people wanted feedback.
Do you do robotics with feedback, too?
Any of you in robotics?
I can tell you anything, can't it?
[LAUGHTER] Well, we didn't think we need it.
The viewing, the line of site turned out to be another issue.
When you look at it, in the Orbiter, we had it at the aft station.
And looking back, at the time, there were no windows facing out.
What we did for the first time was put TVs on the Orbiter to use it.
So, the initial test was to see if we could do things with the TVs.
We decided that was an issue.
The end effector configuration was really never solved until about the end, and that's the thing that is the grabber that you have to grab this thing with.
And the power source we got fairly early.
We did not want to use hydraulics.
The other thing was the tip speed, and that is the speed at which the tip of the manipulator has to move so you can do things with it.
The problem we had is if we are going to capture a satellite and we had a relative velocity between the Orbiter and a satellite, since the guy was going to have to move this thing, but we didn't have a requirement so we had to go work on that.
Payload and cargo handling was the other.
We knew that we were going to handle 64,000 pounds.
And the manipulators we were working on weren't going to do it.
And, of course, the satellite capturing and retrieval was what I've already talked about.
We did a little modeling and converted some things at the center.
We decided that we had to have a high bay, a big room, so we got a room that was Building 13 and we built a plastic floor that was very smooth and put payloads on air bearings.
And then we mounted our manipulator right next to it.
Then what we would do is, on the air bearing floor, everybody would push this 5,000 or 6,000 pound payload across the floor.
And we had two strips where we could get the velocity of what was going.
And we decided that we would try to find out what the tip speed had to be to capture at certain velocities, so we did that.
And we just had very, very simple-minded effectors, but it worked fairly good.
We also worked on stationary.
These payloads right here, after using this floor for probably a year and a half, we came up with some requirements.
We couldn't use a master slave.
The feedback didn't really help us.
It was just too big.
Tell you what, if you ever look at the linemen these days that go up and fix these high lines, their manipulator, their system is similar to what we have.
We have a little bit more sophistication in the avionics, but it is very, very similar.
Mechanically they are very, very similar.
The line of site would be required.
We had an astronaut come over and use the manipulator.
We gave him a TV and then put what it is called an eye mark recorder on his head.
And what this eye mark record was is it looked into your eyes, and when you looked at something out here on a TV over here, you could see where he was looking.
Then we ran a test to see, while he was operating to capture these satellites, whether he was looking at the satellite -- I'm sorry, the TV, or while he was going line of sight.
It turned out, it was very interesting, it was 60% he was looking line of sight and 40% on the TV.
At the time, there was no aft window, so we went to Rockwell and they decided to put the aft windows.
Again, here is one of those problems.
The aft window is about that thick.
It pure quartz and is very, very heavy.
And nobody wanted to put them in.
I think we had some overhead windows in to start with.
I'm not real sure about that.
But we have overhead windows in there and aft windows.
So, we got those put in.
We also defined the end effector speed, the tip speed.
We found out that about foot per second was about all he needed.
Other than that, probably the satellite was moving so fast and it probably wasn't safe to do it so we settled on that.
We decided that we could capture satellites.
That was done.
And we again said we could not see using hydraulics on this system at all.
So, we moved to a bigger building.
Now we built a 60 x 80 foot floor and ran satellites across it, a very smooth floor.
The smoothness of the floor, we had a little hole in the middle that was 0.009 inches deep, and we shot the whole thing with a laser and got the floor built.
We also needed an Orbiter mockup so we built an Orbiter mockup to about where it would be on the Orbiter.
We got the floor built.
That should be 80.
The manipulator was now 50 feet long, so we had to make it 50 feet long.
But the problem was, since we were going to use the electrical system instead of the hydraulic system, now we had to make that hydraulic system think it was operating like an electrical system.
What we did was, on the electrical system, if you put the arm straight out and you lift it, we had a first cut at 100 pounds.
Now in orbit it is inertia that is going to get you, but on the ground we said we will hold this thing out at 100 pounds for the electrical system.
So, we modeled the hydraulic system to work just like that.
Also, we needed a lower scale payload.
We did not have any.
Remember what I was telling you a minute age?
60 feet by 15 feet is about the size of a greyhound bus.
Was what we were trying to work on.
And we also went to a fixed hand controller which drove us into the hydraulics business.
I'm sorry, into the avionics business.
Before we did that we had to figure out a way to handle payloads.
And so what we did was we had looked up some of these people who were loggers up in the Northeast, and they were using essentially a dirigible to take the logs down.
And they were big, about like what we wanted.
We got that company, Sheldahl, I think was the company, to build this thing and put helium in it.
And it turned out something this big has a tremendous inertia, which was great because that was what we wanted.
We didn't want it to be heavy but we wanted the inertia.
Plus, we got a little resistance from the air when we moved it around.
We had to take that one and show that we could grab it and put it in the payload bay.
And that took a while because when you get it all set around, when you start putting it down near the payload bay, say there is the aft station of the payload bay, you find out that you cannot see anything.
Now you really are on the TVs and you really have to get that manipulator where it knows where it is.
That was the other thing.
We found that once you started drop some of these payloads you will want to automate a whole bunch of this stuff.
In addition to that, when you look at the requirements, just putting it in the payload bay is kind of interesting, but once you get it in there you've got to hold it.
What we did is -- There is the trunnion right there.
You can see right there is where the trunnion goes in there.
This thing comes back, it opens up and it's like a big latch.
If you can drop that thing in there, you can just put it in there.
Now, that is not so easy so I will show you.
We built some guides.
But the other thing was none of the payloads were all the same size.
What we had to do is, this is the longeron right there.
There is that longeron in there.
And so we had these bridge fittings that you could move along and move all along the payload bay.
Now anybody that came along with any kind of size of payload we could now accommodate those fellows.
And it worked fairly well.
And we had done that.
The other part was once you got the trunnion latch you had to get this thing in there.
What we did was we tried a lot of schemes, but this was one that worked.
Let me start with this side.
There is the trunnion.
That is essentially these guys right here.
Those are the trunnions.
When you look at the trunnion it sticks out of there and it just drops down into the payload bay and it goes into that latch.
But getting it into that latch is not that easy because you really cannot see sometimes.
What we did was built a thing called a V Guide.
This is the V Guide right here.
And the V Guide over here is right there.
And then we have a scuff plate so that once you drop it in the V Guide this scuff plate kept the payload from hitting up against that latch.
And they are still using the same concept.
Here is what we really went with.
Not master slave anymore.
I'm sure in robotics labs or something you probably go to hand controllers like this.
We went out and looked at a lot of the cranes that people work with, and we found out that these three motors right here all were parallel, their rotation was parallel.
What we did was put sort of the auxiliary placement things outside those three things, just the way the linemen do out there now.
They do not have a hand controller.
They have a switch box or they have little handles that they use.
That is essentially the configuration that we started with on there.
And, if you look at it, this is a little more complex.
And about this time we had had enough work with the manipulator that we had that we had built in-house a 50 foot model that we could use and prove to the Program Office that we could handle the payloads.
We had the big inflatables, we had the air-bearing floor, and so the Shuttle Program Manager said we're going to go with it.
And the Air Force wants it so we're going to use it.
At about that time, the Canadians decided they wanted to be in the space and they are the ones who came and asked us could they build it.
And people in Washington said yes, so our division kind of backed off for a while and they formed a program office with the Canadians.
Actually, that is a Canadian arm called the Canada Arm.
Well, you know, you've got to have everything.
[LAUGHTER] This is the longeron in the Orbiter.
This right here is the longeron in the Orbiter.
And these are just little sort of wrists, and they will latch it in.
And you can see that we got the TVs here.
I think that probably this gives you a little better feel for the rotation.
We had one rotation that the first thing we had to do was, because when the payload bay doors closed, they closed on top of this thing so what you had to do was roll the manipulator out because we had to get it out of the way so there would be no interference with the payload bay doors.
There is a degree of freedom that is really just repositions the thing.
And then after that there is five degrees of freedom on there at least.
And if you start from the back here that first joint is a yaw.
And then all three of these center joints are translation.
And then there is a roll up here and there is a pitch up here.
We could go to any place with the end effector if we had the tip speed and we had enough joints.
The problem was that when we did this and we tried to build some gear trains and we found out that you didn't want to back drive this arm.
You caught something and it back drives.
We did that a lot with the inflatables.
If you use spur gears you would strip the gears out, so we went to roller gears or essentially the planetary type gears.
I think the Canadians now have all these interior angles or all those roller gears.
In the interest of time, since I am going on, here is the manipulator and this is the aft station.
And I have a little better one of that.
But about the time that we were just about to get this thing proven, we found out that people in the world were very interested in this.
We had a Russian general want to come see us, so we said OK and we entertained the Russian general.
We wanted to show him that even though the arm was about 40 feet long that we were very dexterous so we were going to pour him a glass of water.
And we did, except when we got the general there and I walked in, they had the glass, but the water wasn't in what I thought it was in.
It was in a vodka bottle.
[LAUGHTER] And there is no liquor allowed on a federal installation.
And I'm sitting there and the guide is looking at me and I'm saying there goes that job.
[LAUGHTER] I told the general, his interpreter, hold your glass there so that you got the bottle.
Real nice.
The guy was really good at the manipulator.
He poured it in there.
Fortunately it was water.
[LAUGHTER] But we actually poured a glass of water out of vodka bottle for the general.
But I thought my time at NASA had come to an end.
You guys wouldn't do that, would you?
No.
It would have been vodka.
[LAUGHTER] Yeah, they'd use the vodka.
Nowadays, they might use the vodka.
I hadn't thought about that.
Well, let me go on here.
I was going to point out one thing here that the TVs turn out to be a very important part of this whole thing.
Once you get a payload and you sort of are blind, now you have lots of TVs you can use.
There are four on Orbiter.
There are two up front, two on the back bulked and there is one on the end effector so that you can sort of see where you're going for the last engagement.
And you can see here there is the aft station, there are the windows.
Now, he's got a pretty good line of site if there is nothing in there.
If there is a docking tunnel in there that makes it a little more difficult.
Did you have a docketing tunnel?
You just had a place where you could set it.
My favorite thing, for this gentleman, is that fixing the Hubble to me was the greatest thing that we have ever done.
And I hope they do it again some time.
And the manipulator always plays a part in that.
The only other thing that we did that was interesting, early on we built an end effector where the astronaut could stand on it.
Our first cut at this was to do it like the guys that do the high lines, and we put all the controls out on the end effector.
The Canadians did not like that.
It complicated things.
Remember, we only had about 120 kilobytes total for the computer for this thing and that was it.
They didn't want to waste any more and they didn't want all the weight.
I guess the astronauts or someone said we'll go with just a foot stand and then we will control it from the aft station here.
And I think that's the way you do it now, isn't it?
Yeah.
It was a tough battle, but weight and cost won out again.
It's tough to beat them.
These aren't really that great, but you can see that this thing takes up a lot of control panel in the aft station of the Orbiter.
It really does.
And here are the windows right there.
And there is an overhead window.
We thought at one time that what we would have to do, the arm is here, this aft, there was a payload that wanted the arm to come back this way.
If it did that you had to have that window.
A very strange thing happened, because when you turn your arm back that way, right is left and left is right.
We had to practice the reverse of everything or get a mirror or something.
We tried a lot of it.
The Program Office wisely decided that was not a good idea so they were going to handle that payload another way.
I wish I could remember exactly what it was.
There are two hand controller here, and here they are right here.
This was a lot more fun than wheels, brakes and tires.
This was a fun project.
And this is the translation controller.
You've operated this one.
And there is the rotational hand controller.
This one on the left essentially operates the three center joints where it goes like that.
And then all the others are done with this hand controller.
I think those are pretty normal flight controllers, aren't they?
Those are the ones you used.
At least we thought it was.
The biggest argument we had with the Canadians was this, the end effector.
That is the hand, the thing that grabs it.
We wanted one that was more dextrose.
They argued, and rightfully so, that what we really wanted was one that would make a good sturdy latch.
And this one does.
The way this works is that there are three little wires in here, three cables that come in, so once this end effector is placed over this grapple pin here these wires close.
And, as they close, they also move back and they grab on the end of that pin right there and pull this end effector into this shoulder right here.
Now, in the meantime, there is a docking target that he looks at that he can see.
Now, once it sits on this shoulder that really is a good structural latch.
You've got a great structural latch.
And if this indexes you can also make an electrical connection across there, which some payload is willing to do.
Now you make a capture latch which is essentially the way you do docking a lot of times.
You make a capture latch with these wires that actually just close around this thing.
Then it pulls it back, sets it on these shoulders and now you have a nice sturdy thing.
And the Canadians did a good job there.
I have to tell them that.
The problem with that is the one we had on the nose of your steering.
We had ten single point failures in there which makes you nervous, particularly if you're hauling a big payload or an expensive payload.
However, they did a good job.
Reliability has never failed that I know of, so that was a good thing that they did.
This was a lot more fun than the wheels, brakes and tires guys.
Then the only other thing is we had a backup mode.
We were afraid.
At the time we weren't that enthused with computers so we wanted a backup mode.
So, we got everybody to go to a single joint mode.
What we did was hardwired all the joints and put a box on the control panel so you could move each joint singularly.
And that was the backup mode.
You can do it with a single joint or you can do it computer supported.
You can do it either way.
And we also had to have a jettison mode.
If this thing stuck out there, you would have to throw it overboard because you couldn't close the doors.
We had to get rid of it.
The Canadians thought that was not a good idea.
OK, last bit.
This was a different project from the other one I was talking to you about.
This one did not start with any history at all.
We really had to start this one from scratch.
It is probably one of those things that came along just at the right time.
And we were studying some technology.
And we had enough sense to convert, but it essentially is just a big crane.
I think that the thing that surprised us the most was this one right here.
The crew took to this very, very easily.
It took very little for them to do it.
As a matter of fact, our simulator in Building 9A became a trainer forum.
We were still doing development work.
And they said, no, we're going to make this a trainer.
And those guys were pretty good.
They came in, in very, very little time.
We had to do this.
This was that inflatable testing.
Remember on our other project I said you've got to do the testing, you've got to prove the requirements?
Here was one where we did a little bit of innovation with that big inflatable to show that we could meet the requirements.
And we did.
And surprisingly and happily working with the Canadians was great fun.
It was the first time I had ever worked with any of the internationals, but it was great fun.
And they did a great job on this thing.
They are still upgrading it.
It is still flying.
And I'm taking up all your time I'm going to take up.
Just one or two things in finishing up.
Allen talked a lot about the importance of testing.
And this was a challenge with the arm because, remember, all the stuff that we did with the inflatables, that was all done with a hydraulic arm.
The actual arm with the electric motors was incapable of lifting its own weight in the earth's gravity.
I remember going once up to the factory in Canada, and the only testing that they could really do, they had a big air bearing floor up there.
And they could use the electrical arm, but only in two-dimensions.
And so it didn't get its full three-dimensional testing until STS-3, the third orbital flight test where they had the arm on.
And that was the first real use where they could put it through all the paces.
But it really has worked well.
And, in fact, you've all seen the picture of the astronaut Bruce McCandless and the manned maneuvering unit flying off by himself.
That has become like an icon.
The reason that was originally built and fielded was because people didn't think that the manipulator arm would actually have the capability of grabbing onto a moving or rotating satellite.
And so the idea was that the astronauts would have to go and catch it and then fly it or stabilize it so that you could come pick it up with the manipulator.
But, on the solar maximum repair, actually, it turned out that they couldn't catch it with the manipulator because there was a configuration problem and they couldn't latch it.
And so they actually had to go after it with the manipulator arm, while it was spinning, and they were able to get it.
And, in fact, a lot of work that we did in simulators showed that the manipulator had a lot better capability than we originally thought it would.
That was Joe Allen, right?
Well, it was Terry Hart who actually caught it.
Terry Hart.
There was Pinky Nelson who went out.
Joe and Dale went out to get [OVERLAPPING VOICES].
These were not big astronauts.
They were a couple of the smaller guys.
Anyway, just to end, the moral of this is there is a lot of discussion often about humans and robots and who should be doing what.
And I think a lot of people don't realize that, from the very beginning, we have actually had a human robotic interaction on the Shuttle.
And particularly for EVA activities it has allowed -- And I will be talking about this later in the term.
We get humans and robotic systems working together.
Now, granted these are teleoperated robotic systems.
The real hardcore roboticists don't call it robotics unless it's autonomous.
But, nevertheless, it is a telerobotic system that we use in combination with humans.
And it has given us very great capabilities.
OK, we will see you on Thursday.
Let's see.
Oh, there is a lot of stuff.
OMS, RCS, fuel cells, auxiliary power unit and hydraulic systems.
And also we will be looking for the outline, the first cut on your reports.
Henry will reveal you with all of that.
And let's thank Allen.
[APPLAUSE] Thank you.
Henry has been a longtime colleague of mine working in the early days of Apollo and Shuttle.
He is an old country boy but he has created more innovative designs on technical problems than anybody I know.
And we will try to bring some of those out because he has really created some very innovative solutions to technical problems.
And I am going to set Henry up a little bit so let me set the stage for you a little bit.
I was manager of the Space Shuttle Orbiter, and the engineers came to me and said, Aaron, the waste management system on the Shuttle is not working very well.
Now, the waste management system, in simple terms, is the toilet.
They said the toilet is not working very well, and we really need to go off and design a new toilet.
I thought for a moment, well, that's probably a pretty important thing to do, so we did.
Well, it turns out the original contract, and I am quoting a little bit from memory, was about $10 million.
Now, $10 million for a toilet when you can go down to any place and get a toilet for a couple hundred dollars is a pretty high number.
But that wasn't the worst of it.
The worst of it, about six months later they came to me and said, Aaron, guess what?
It's not $10 million.
It's $20 million.
Congress got wind of this and wanted me to come up and testify.
And you can see what kind of fun they are going to have.
Waste management, they're really going to have fun with this subject.
And you may know some people in congress, but the name was Mr. Sensenbrenner from Wisconsin.
I remember very clearly.
He is still in congress now.
He was head of the Science Committee, and that is who we had to go testify for.
Well, there was a reporter for The Washington Post.
Her name was Kathy Sawyer.
Now, Kathy was the science reporter for The Washington Post.
Very, very good.
Kathy was very smart, very fair, but she really bore in on things.
Kathy got wind of this so she wanted to publish an article on it.
She called me and said would you please talk to me about the waste management system?
I said, boy, she is going to get me in trouble.
I said, I will tell you what, I've got somebody that could really tell you about the waste management system, and I will have Henry Pohl call you.
I talked to Henry about it.
Now, that is going to be Henry's introduction to this class of what he told Kathy Sawyer because it is a classic.
Henry, it is all yours.
Aaron almost forgot about Kathy Sawyer.
Kathy called me up and said can you explain to me how I might explain to my neighbor's 14 year old daughter why NASA has spent $10 million building a toilet?
And I said I will try.
And I thought for a little bit on how am I going to start out on this?
I said, you know, if you take the commode that you've got in your bathroom and you bolt it upside down on the ceiling, now try using it.
[LAUGHTER] I said that is really a little better situation than we have on the Orbiter because at least we know which way the gravity field is on that.
In orbit you don't know.
Most of the time you don't have any, but it might be to the right or left or up or down.
You know, that was the only reporter that I ever talked to it that put everything down verbatim of what I said, the questions she asked and what I said and printed that article in the newspaper.
But that kind of sets a stage for operating in the absence of gravity.
I explained to her that the Space Shuttle volume that the people had to live in is about the same volume as you have in a modern bathroom.
If you look at a bathroom with a shower and a room just outside with a sink and everything, we have just about that same volume in the Shuttle.
Now, you seal that all up, you've got a certain amount of air that you can use in there and you only have the air to replace gravity.
You've got to have a flow of air to direct whatever you want to direct in a certain direction.
You use that air.
You run it through.
You've got to deodorize it.
You've got to clean it.
You've got to run it back into the cabin almost instantly.
Otherwise, you pull the vacuum in the cabin.
And so whenever you start putting all of that equipment in there and all that stuff in there on something that has never been done before, $10 million or $20 million is kind of cheap when you get right down to it.
You look at the number of people that that will hire for a year working on it is cheap.
But to try to get that message across is not easy.
Go ahead and give me my first slide.
Let me just tell you another one since Aaron got me off on a tangent here.
The Chicago Tribune called me one day when we had a problem on Shuttle, and she was badmouthing the Shuttle.
And all she wanted to do was get a comment out of me that I agreed with something that she said since she could quote it.
And I said I don't agree with you.
I said you know that Orbiter is really a pretty good vehicle when you look at what it has to do.
I said you realize when it's in orbit it is going eight times faster than a bullet when it leaves the muzzle of a .30-06?
Silence on the other end.
What's a .30-06?
Well, I knew that wasn't a good example.
[LAUGHTER] But I told her that is a military rifle that was used in the Second World War.
And I used to fire that thing in the desert.
And I could pull the trigger and see that bullet hit the ground out of that 300 yards almost instantly.
Now, that shuttle is going eight times faster than that.
And I could see the headlines in the Chicago Tribune, the shuttle flies eight times faster than a bullet.
Long silence.
She said maybe I ought to find something else to write about.
I said why?
She said I don't think I have a story here.
Well, it wasn't a story because it wasn't negative.
Kathy Sawyer was the only time I ever got a good story from the press.
I want you all to feel free to ask questions.
I'm here more to talk about what you're interested in talking about than my thoughts, so feel free to ask questions any time.
I would like to talk about a half an hour.
Somebody is going to have to tell me when a half an hour is up.
And then I would like to have some time for some questions.
My first involvement in the Space Shuttle was Max Faget kind of came up with this idea of a winged vehicle going into space.
And we kicked it around for a while.
And he formed a little committee, very small, about 20 people as I recall.
Put the people in a room.
He wanted to keep it quiet.
He wanted to keep it secret until he got enough data so that he could know whether it was feasible or not.
Well, they wanted me to send the best engineer I had over there.
And he appointed a guy by the name of Jim Chamberlain to head up that group.
I sent a guy over there.
They closed the doors, had a guard in front of the doors so nobody could come in, no information could get out.
On the third day, Jim Chamberlain called me up and said I need to have a talk with my engineer.
I said why?
He said he balked.
I said what do you mean he balked?
He says, Henry, you know what an old mule does when they balk, don't you?
I said, yeah, they won't go.
He said, well, that's your Mr. Kindrick, he won't go.
I called up my engineer and said what's going on over there, girl?
He said, oh, they don't know what they want.
He said they gave me some requirements.
I designed them an RCS system for it.
They gave me another set of requirements.
I designed another RCS system.
They gave me some different requirements.
I designed them a different RCS system.
And then they came out with another set of requirements.
I'm just going to wait until they decide what they want and then I'll design them an RCS system for it.
Well, what he didn't understand was that's the way you go about setting the requirements.
You get a small group of people together.
You set down a set of requirements.
You try to put a vehicle together and see where it punches out.
Usually weight, CG, something.
So you change the requirements up a little bit and you design another system.
You do that very quickly and you look at where that one punches out.
And then you change your requirements again and you look at those requirements and where it punched out.
The first requirements, where it punched out.
You change the requirements a little bit and see where it punches out.
Very, very quickly you run through many configurations.
That got the Shuttle started.
After it got started we kept changing the requirements as we went through the program.
I would suggest that you understand your requirements very, very clearly, and the impact of your requirements very clearly.
Things like fail op fail safe, or fail safe fail safe fail op are very, very good buzz words.
But unless you really understand the impact of those, they can actually make a system less safe.
I can make a very good case that a two engine airplane is safer than a four engine airplane simply by the way that you set your requirements.
FAA has a requirement that an airplane has to take off with one engine out.
If you've got a two engine airplane then most of the time you've got 100% more power than you need.
If you have a four engine airplane you only have 25% more power than you need.
Most airplane crashes are caused from lack of power at the right time.
If you've got the power when you need it then you can get out of most bad situations.
Our original requirements on the Shuttle, after we got that started, was to keep the operations cost low.
We wanted to keep the maintenance cost and the operations cost low.
We initially were looking at oxygen/hydrogen systems.
In fact, all of the early orbiters had the hydrogen/oxygen onboard.
In place of their payload, we had hydrogen tanks and oxygen tanks in there and carried the hydrogen/oxygen onboard.
And we actually looked at using the hydrogen out of those tanks.
You get it free to power the OMS and the RCS.
And we looked at all kinds of ways of getting the pressure up high enough to be able to use the residuals to power the vehicle.
One of the students asked the question the other day and we didn't give a very good answer that we did at one time look at putting the hydrogen in the payload bay of the Shuttle.
All of the very early configurations of the Shuttle had the hydrogen and the oxygen in the Shuttle in place of payload bay.
That was the very early configuration.
Later on, as DOD got involved and other people wanted big spaces and carrying greyhound buses up there and things like that we had to use that space and that's when we came up with the external tank.
And that's when we came up with a side-by-side configuration on it.
That's the way I recall it.
But we looked at oxygen/hydrogen.
Then we went to methane because we were looking at bulk densities so that we had smaller tanks, smaller volumes.
Then we looked at oxygen/alcohol.
And I really thought we had a good system with oxygen/alcohol.
I had a lot of confidence in that system.
We used it on Redstone.
My first job was working on Redstone.
I really liked alcohol.
I thought we had developed a good ignition system for the RCS so we didn't have to worry about the thousand starts.
But obviously that was going to cost us money and cost a lot of development money.
And, as time went on, we changed from having low operational cost to low development cost.
And when we went to low development cost then we went to bipropellant on the OMS and initially to monopropellant on the RCS.
When I say bipropellant, what I mean are storables, monomethyl hydrazine and nitrogen tetroxide.
Both are storable propellants.
They are self-igniting so you don't have to have an ignition system for it.
That simplifies the design a whole lot.
We had a lot of experience with those propellants on Titan II, on Agena and on Apollo, all of the Apollo stages used either aerozine 50 and nitrogen tetroxide or monomethyl hydrogen.
So, we went to that.
On the RCS, we initially went to hydrazine as a monopropellant because it was the simplest system, the cheapest system, but very quickly the weight got out of hand and we had to go to something that had a little bit more performance than a hydrazine because all of that weight was in the backend of the vehicle.
And the backend of the vehicle was getting too heavy.
If you only remember two things out of this exercise and what I have to say today, two things I want you to remember is that it is just not natural to think in terms of the absence of gravity and it is not natural to think in terms of the absence of pressure.
When you think about it, you live in it around here on the ground, it is so natural that it's hard to think in terms of designing systems and the impact that the absence of gravity has or the absence of pressure has on the design of the systems.
The other thing I would like for you to remember is that there is no substitute for a good well thought out test program.
Let me give you the OMS as an example.
We did not have a vibration test program on the propulsion systems, on the OMS or the RCS in the program because, by that time, we had good structural programs, we could analyze the effect of vibration and stress on the vehicle.
But we had a good propulsion test program and we built an OMS pod, just like the ones we had on the vehicle for qualification.
And when we got through with all the propulsion tests on it, and we went through that program in a breeze, we had no problems at all with it, so we finished it early.
We took that pod down to JSC, we put it in a vibro-acoustic facility, and while we were running the QVVT the low level vibration test on that pod just to get the response of how things responded to everything, the helium bottle fell out.
Five minutes after that helium bottle fell out every structural guy in the world could tell you it should have fell out, it would fall out, but nobody thought about it in advance.
What happened, we had taken a high pressure helium line coming out of the bottom of the tank and going five feet or eight feet over to the structure and bolting it to the structure over there.
Now, when you started shaking it, that was acting as a torquing element on that tank and torquing that tank this way.
And the struts they had coming down holding the tank, it was fatiguing where they were put onto the tank.
It fatigued them right in there and broke them off very quick, but we didn't pick it up.
The OMS was very, very straightforward from a development standpoint.
We came out with a platelet injector that gave us very good combustion.
You have to explain what a platelet injector is.
OK. A platelet injector is one where we used photographic techniques to etch holes in the injector.
You could get a very, very fine pattern.
You made it in very thin sheets and you coated the sheet with a coating where you could put them in a press and heat them and glue them together.
You came up with an injector design that was made out of many, many thousands of very thin plates that set up the manifolding on it.
Normal injectors you have to do a lot of very integrated drilling.
Normal injectors, you drill out a manifold in the back, and then you take the face plate and you have to drill holes in it.
And the problem we get into when you start doing that is with these very small drills they tend to walk or bend.
And you don't get a straight hole.
You don't get a good spray pattern.
With a platelet injector we could get a very close match.
We could just kind of dribble a field in there and oxidizer in there where it always matched real well.
We got very high performance out of it.
We're talking about the injector up here where you have the fuel and oxygen, and then you have to have a whole bunch of holes in a pattern such that the fuel and the oxidizer are properly mixed when they ignite inside.
You need to get the oxidizer to impinge on a fuel very precisely so that with an aero-storable propellant, a hypergolic propellant as we call it, that is self-igniting.
And basically what you have is when the two fuels come together it is like having a very strong acid and a very strong base coming together.
The first reaction that you get is a heating reaction.
That chemical reaction that you get will take the propellant temperature up to 800, 1000, 1200 degrees.
Anyway, it gets it up to the combustion temperature and then you start the combustion process.
So you don't have to use any other ignition source.
All you do is bring those two propellants together and you get that chemical reaction that provides the energy, the heat to heat the propellant to the combustion temperature.
Like all high performance rocket engines, we had a stability problem with the OMS.
And we had to put big acoustic cavities.
38% of the injector on the OMS was cavities designed and sized to damp out the stability, so we had a very good stable engine.
The RCS, on the other hand, is a totally different design.
Now, we put the propellant tanks on the RCS up in front of the OMS tanks for aerodynamic purposes.
That gave us a 30 foot propellant line down to the RCS.
Then we would manifold three, four, five engines on the end of that pipe that is pulsing in and out of phase or steady state.
Some of them were running at 40 millisecond or 80 millisecond pulses.
Some of them might be running at four times that on a pulse width.
And some of them may be running at steady state.
Anyway, the dynamics, the pressure in that line that the engine sees varies somewhere between vapor pressure and about 1000 psi.
And, since one propellant is 1.6 times the density of the other propellant, when that valve opens down there, that pressure wave has got to go back up to the tank and come back down to the injector before you get basically any flow out of it.
Just the compressibility that you have in the pipe will come out.
The pressure essentially drops to vapor pressure.
And we use helium to pressurize those tanks.
When we use helium to pressurize those tanks, the helium is dissolved in the propellant.
When it gets down to the engine it comes out as solution.
The engine has to be able to stay together at any mixture ratio and almost any pressure.
And you are looking at hundreds of thousands of starts over the life of one of those programs on one of those engines.
And we talk about pulse width.
Pulse width is the minimum ohm time that you can have on a rocket engine.
Now, there is a volume between the face of the valve and the face of the injector that we call dribble volume.
The larger that volume is the larger the pulse width you need on it.
You would like to have the seat of the valve to be the face of the injector so you have zero dribble volume in there.
That way you would get closer to a square wave pulse.
And your friends in Guidance Control, they always want to square away pulse.
And they want it to have an infinite thrust level and zero width so that you put an instantaneous pulse.
It makes it easier for them to calculate where the vehicle is going to go than if you get one that builds up kind of slow and goes down kind of slow.
The dribble volume, the volume that you have between the valve seat and the injector phase when the engine shuts down, when the valve closes that flow stops and that amount of propellant stays in there.
Now, in a vacuum, when it gets down to the vapor pressure starts boiling out and goes out through the chamber.
And you have a refrigeration effect from that boiling of the propellant.
If the pulse width is too short then the refrigeration effect will overcome the heating effect from that short pulse and you keep reducing the temperature of the hardware.
The colder the propellant gets the more heat you have to put in the chemical reaction, when it first comes together, to get it to ignite.
You get an ignition delay in the thing and then you start getting into hard starts if it gets too cold.
So, we set our pulse width on the Shuttle at 40 milliseconds.
That was the neutral point at which the refrigeration effect equaled the heating effect from the pulse, and so you would maintain the temperature.
We later on went to 80 milliseconds on it because we added the Veniers.
That gave us a little bit of heat input.
It's a pretty tough design in itself.
The valve that we used for a 1000 pound rocket on the Orbiter weighed less than the valves that we used on the Apollo program for 100 pound thruster.
And what we did is we went to a pilot operated valve.
That was kind of my concept that I borrowed from Sears Roebuck.
If you ever take a washing machine apart on a Sears Roebuck and you look at how the valve is designed, it is a pilot operated valve.
And they use the water pressure to open the valve.
Well, I thought that was good way to go.
You put a very small coil in there, a quick acting coil and replace that rubber bellows that used on a Sears Roebuck washing machine with a stainless steel diaphragm and you've got it made.
Well, unfortunately the water pressure in a house doesn't vary as much as the water hammer does on the Space Shuttle.
So, before we got through with the design of that valve, it got to be a very complex valve.
And it was a very expensive valve but it worked.
Probably the other thing we ought to say about designing any system that is going to work with these hypergolic fuels, I mean they just eat away at things like O ring seals.
I mean it's just nasty stuff.
And so, in addition to the mechanical problems that Henry is talking about, you've got all the materials problems which just makes your life harder.
We really wanted to stay away from hypergols, from the very strong acids and the very strong bases for the simple reason that there are very few materials that are compatible with them.
As a matter of fact, the main reason we went to a hydrazine RCS in the beginning was to avoid having an expulsion system in the oxidizer because we had no material available that you could put in there as a diaphragm to push the propellants out that was compatible with N2O4 that you could make cycles with.
We could have used metal bellows, but they weren't very reliable.
They were extremely heavy.
And so we had to come up with a better system than that.
And then we kind of got to looking into putting in screens in the system.
And that was really a hard sell.
I mean I spent lots of time with Aaron.
I was convinced that system would work because if you look at an automobile gas tank, every fuel tank on every automobile has got a sock in it.
That sock is about an inch and a half in diameter, and it will suck every drop of fuel out of that gas tank.
They putit in a little sump in there, and it will just draw that tank dry before it will break down and let air go through it.
I was convinced that we could build one of those systems and make it work.
But if you look at an automobile, you know it goes over rough roads, it bounces, you've got G fields, you've got all kinds of forces on that thing, and yet it works very good.
The problem we had in trying to sell that concept to the program, though, was how do you prove it?
The only technique we had to prove it was through analysis.
And you're trying to prove that something like that will work purely from analysis.
And qualified by analysis was a hard, hard sell.
We, later on in a program, came up with techniques where we could wrap tape around most of the screens in there so that we close up most of the area.
And so we would draw a propellant in, in very small areas with aerospaces on top and prove that you could get down to a certain delta P and cross it before it would break down to qualify our analysis.
We still were not satisfied with that so we made up a system with a glass tank, we put it on our vomit comet and flew it.
And that was the only time where I ever road that thing.
If we had made one more time it would have been all over for me.
[LAUGHTER] But you get up 38,000 feet and you make a dive on that thing.
And you can maintain zero G in it through that 30 second or 38 second period of time.
And by expelling the right amount at the right levels we could prove that it would work.
And it has worked very, very well on all the flights.
In the future design of the spacecraft, what is going to be the difference in the RCS system?
Because every system is going to have to have some kind of reaction control system.
What do you think they are going to go with in the future systems?
I really believe that we ought to go with either methane and LOX or alcohol and LOX.
I think it's a much cleaner system.
We developed that little piece of electric igniter for the Shuttle Program.
We never used it after we went away from those systems.
But you buy these little latch torches now that you push a button and pull a trigger on it.
And they light off most of the time, although the last one I got didn't light too well.
But we have good, reliable ignition systems now.
And I think that you can design a system now that has the reliability that you need for a space system.
Otherwise, we've done nothing in this country since the Shuttle Program on developing [OVERLAPPING VOICES].
The things you have got to remember is that the thermal characteristics are very, very different in a vacuum than it is on earth.
It is very different in the absence of gravity.
You wouldn't think gravity would make any difference in it but it does.
You've got some fluid in the tank.
You put some heat on the bottom of the tank.
Where does heat go?
It goes to the top of the tank.
The top of the tank is always the hottest part.
When you've got a liquid in one G, you put it in zero G, you put heat on the bottom of the tank, a bubble forms down there, it pushes the liquid away from it and nothing comes out of the tank.
Let me have some questions.
I'm slightly confused on the fuel that the system uses.
Both RCS and the OMS use hydrazine or hypergolic?
I'm confused.
Hydrazine is N2H4.
We use that as a monopropellant.
A bipropellant earth storable.
A monopropellant system you normally run it over a catalytic bed and it decomposes.
Yes.
There are two monopropellants out there that work very good.
One is hydrogen peroxide.
And if you can get it above 90% it gets a little more stable.
And the other one is hydrazine.
Both of those you run through a catalyst.
When it hits a catalyst you get the reaction out of it and it breaks down.
The hydrogen peroxide breaks down to steam and oxygen, and hydrazine breaks down to hydrogen and ammonia, NH3.
But both of those give you hot gases then for propulsion.
You mix that with an acid and you get combustion.
And we used aerozine 50 which I mentioned.
Aerozine 50 was 50% hydrazine and 50% unsymmetrical dimethylhydrazine.
Monomethylhydrazine is just that, monomethylhydrazine.
It is a lot more stable than is hydrazine.
It has a wider freezing /boiling point than hydrazine.
Hydrazine has a lot of the characteristics of water as far as density, freezing and boiling.
It will freeze at about the same temperature as water, it will boil at about the same temperature as water and it has almost the same density as water.
Monomethylhydrazine has a little bit slightly less density, it gives you a slightly better performance but it freezes at like minus 63 degrees or something like that.
Monomethylhydrazine and nitrogen tetroxide.
The same thing for the OMS.
Now, one of the things we do is we put about four-tenths of a percent of nitric oxide in the nitrogen tetroxide to keep the tanks from breaking.
We have titanium tanks.
And on the Apollo Program we got into a very serious problem because a tank started exploding.
It turned out with aerozine 50 or monopropellant hydrazine, we had a little bit of water in there and that gave off a little bit of free hydrogen.
And that free hydrogen, titanium just does not like hydrogen.
How many people have taken fraction mechanics?
Well, fraction mechanics really started from that.
That's when we started working with a gentleman called Dr. Tiffany at Boeing.
We worked with the whole world on that problem.
That's how fraction mechanics really got started.
You were going to ask something.
While we're talking about fractions, one of the other problems of dealing with these hypergolic fuels is the freezing problem.
Maybe we will talk a little bit about that.
Because, unlike water, when the hydrazine freezes it shrinks.
And so if you imagine that you've got hydrazine liquid in a line under pressure and now it gets too cold so it freezes.
And now that leaves a little bit of free volume so you get more liquid that comes in there and that will freeze.
Until finally the entire line is clogged up with solid hydrazine.
Now, when it warms up it expands and you crack the line.
And now you've got a hypergolic leak which is really bad news.
So, in fact, maintaining thermal control of the OMS and RCS propellant lines becomes a very critical issue.
And there are heating coils all over the place and thermostats.
And so, again, you don't just have a propulsion system in the systems engineering that we talked about.
It affects the thermal, the electrical system, because all these things are inner-related.
If you lose an electrical system so that you cannot run the heaters then you run the risk of losing your propellant system.
And so then you have to have redundant heaters and so on.
And it gets more and more complicated.
That was a major, major problem we had on Apollo 13.
We didn't have enough power.
We had to turn everything off quick.
And trying to get those people to let that temperature go down out of limits was not easy.
And I remember sitting down and spending an entire night running out the thermal calculations and calculating out when we could turn the heaters off or we could keep the service module off of that vehicle and not freeze the propellant and the RCS on the command module.
And then I got really, really worried.
I gave myself four degrees above freezing on that system, and I missed it two degrees.
It got two degrees colder than we had planned.
The design of the RCS system was very, very complicated.
How many RCS specialists do we have on the Shuttle, 40?
Thirty-eight primary and four Vernier.
And so it's a very complicated system in designing it.
It had a lot of requirements placed on it.
On the other hand, the OMS engine, as I recall, we have never had a failure on the OMS engine.
The OMS engine has been very, very successful.
It is a very straightforward program.
By getting the combustion close to the injector on the OMS then we could make the chamber small on the OMS.
And, by making the chamber very small on the OMS, then you could cool it.
You could put cooling channels down the chamber and run the fuel down there and keep the chamber cool so it wouldn't burn out.
Now, we also did the same thing on the RCS a little bit different.
The RCS is a buried installation.
We have to run that engine for 500 seconds or more if we want to use it for de-orbit in case the OMS plays out sometimes.
That engine has to stay together on these very, very short burns.
It also has to stay together on these very, very long burns with fluctuating inlet pressures and fluctuating mixture ratios.
We found that if we make a very, very short chamber -- And that's why when you look at the RCS it is a big chamber, it is very short.
We could put enough fuel down the walls of that chamber to effectively cool the chamber so it would only go up steady states to a certain safe temperature and still have good performance out of it.
So, we put a lot of fuel down the walls on it.
Looking in this direction, if this is the injector plate, the holes around the outer most layer, those are all fuel.
So, you basically get a fuel bath coming along that actually comes in contact.
And if you have a fuel-rich mixture, the same as in a car, the fuel mixture burns cooler.
The thing that you're really afraid of, in fact, you have to deal with a lot of contingencies on this system, what happens if you get a clog in one of your feed lines?
The worst situation you can get in is that you get a clog in your fuel inlet line, and that gives you an oxidizer-rich mixture which burns hot.
And you can then actually get a melt through.
And so you've got to shut your engine down in a hurry if that happens.
These are very, very small orifices, and a lot of them around the outside of it.
And if you get a couple of these orifices plugged then that becomes a hot spot on the side of the chamber.
And we used columbium, which has very poor heat transfer.
We used molybdenum on Apollo which has good heat transfer.
To return to your point about the possibility of using the RCS as a backup for reentry, what prevents one from using a series of relatively short burns [carried on the mass?]
of the Orbiter to smooth out the velocity chain?
Why do you have to have a single file of second burn?
Before you start to burn, to come back in, how much time you can put in between those and how short, a 1,000 pound thruster for deorbit takes a fairly long burn.
And the engine goes to steady state fairly quick, you know, in a matter of 20 seconds.
[UNINTELLIGIBLE PHRASE] We ran RCS engines for the Shuttle up to an hour and a half on a single burn.
[UNINTELLIGIBLE PHRASE] Even though you put it in multiple burns you still have to have fairly long pulse widths on each one of them.
And that impacts the guidance very, very badly.
Let me help with a little orbital mechanics.
We talked about this before.
You're in orbit.
When you do a burn, you can do a retrograde burn which lowers the other side of your orbit so that now becomes the perigee.
The thing is, from orbital mechanics, the most efficient way to do a burn is to have, like Henry said, an infinite thrust with a zero pulse width.
You do what they call an impulsive burn.
But, in the real world, that never happens.
Now, it is 45 minutes from here to here, it's a 90 minute orbit.
Now, when you have two OMS engines, a typical deorbit burn lasts two to two and a half minutes.
You're burning over this segment.
It's pretty close to an impulsive burn.
You lose one OMS engine and now you have to double it.
Now you're up to a five minute burn.
But, if you lose your second OMS engine and now you have to do an RCS deorbit, now you're getting into a 10, 15 minute burn.
And so you're actually really far away from optimum.
And the burn gets much less efficient.
And so if you have plenty of propellant you're fine, but if you're low on propellant you need to worry about the efficiency of your burn.
If you were to do a little bit of a burn and then shut it down to let your engines cool off and then you try to complete the burn around here, now it has gotten really, really inefficient and you might run out of propellant.
We spent an enormous amount of time with our orbital mechanics people and our guidance people trying to get the OMS strut down as low as we could get it.
Because the lower you can get it the smaller the hardware, the lighter the weight of the hardware and the performance is essentially the same.
The ISP for a 3,000 pound engine is essentially the same as it is for a 20,000 or maybe even a little bit higher for 3,000.
We had 5,000 pound thrust in there.
We had 4,000 pound thrust in there.
I think we finally wound up with 3,500.
Been left up to me, I think we could have done it with a 1,500 pound thrust engine.
But we compromised.
Yes.
Professor Cohen once mentioned that some people suggested putting some RCS thrusters on the edge of the wing tips, but that would be very difficult after the design was finalized.
All of our early configurations that we had, we had the RCS pods out on the wing tips.
It made the RCS very efficient.
Our RCS was the OMS at one time, and we had it all out on the wing tips.
It didn't make our structures friends too happy at first because that puts a lot of mass out into the wings and they didn't like it too much, but after they got looking at it that turned out that really wasn't much of a driver.
It made the RCS very efficient.
The thing that kind of changed that is when we put the big payload bay on the inside of the vehicle and then we had to put a big OMS pod on there.
And then to put the RCS pods out on a wing tip of a delta system out there just didn't weigh out good.
It was more efficient from a weight standpoint to put the OMS and RCS together because we actually had those two interconnected.
If we have to do an OMS burn with the RCS, we can take the propellant out of the OMS tank to feed the RCS.
And so it just made a more efficient system to bring it all in.
We had another requirement.
When we went to the acid-based propellants, we had a requirement that we had to have removable pods.
You had to be able to take the propulsion system off of the orbiter and take it over to another facility to rework it because we knew we were going to have a lot of work and rework on those systems, at least early on.
You had a question right here.
To what extent can you change orbits due to a higher or lower orbit [UNINTELLIGIBLE]?
That is very limited, but we do have the capability to go up and down in orbit.
I don't know, maybe 50 miles.
I don't know.
I don't have a good answer.
I don't remember that.
That has been too long ago.
With a full load you have a few hundred feet per second, I would have to go and look up, you cannot make the plane change.
No, plane changes are out.
But we certainly have gone from, let's see, like 180 nautical mile orbits down to a 105 mile.
I remember one flight where we did that.
To go up, you know, what you really want to do is, of course, use your main propulsion system to get into the right orbit that you want to go to.
Then, if you're going to do a rendezvous, you have to change the height a little bit.
But you typically try to get orbit insertion within about 20 nautical miles, so about 40 kilometers radius of your ultimate intended orbit.
For some special missions, like I was on one mission where we were up at about 180 nautical miles doing various operations, but then the last couple of days they wanted to look at the interaction of the Orbiter with the atomic oxygen and nitrogen which causes this orbiter glow phenomena, which maybe some of you have heard about.
We had to go down to about 105 nautical miles, which is about as low as you can go and stay in orbit for more than a day or so.
And so, yeah, that was OK because that was just you're doing part of your deorbit burn but then you stop.
What you wouldn't want to do is go to a 105 mile orbit and then go up, and then you would have to come all the way down again.
I think we will take one more question and go to break.
It is awfully easy to decrease the orbit because you're using part of the energy that you have to use to come back home anyway.
When you start trying to go up is when you're adding.
But we have a lot of contingency propellant onboard.
We've got contingent propellant in case the main engines shut off early.
Then you have to use the RCS.
If you've got a docking mission we've got propellant in there for three or four attempts at docking and things like that.
There is extra propellant that you can use to give you a little bit of a boost.
You had a question here.
Going back to the comment that was made about the [UNINTELLIGIBLE PHRASE] and presenting a hazard that way, what kind of feedback did the crew have when something like that was happening?
Were there sensors in that system [UNINTELLIGIBLE PHRASE]?
I don't hear too well.
The question is what sort of sensors do we have for the operation of the engines?
There are pressure sensors.
There are temperature sensors.
Again, remember when we were talking earlier, though, sensors don't always tell the truth.
Sometimes you can get sensor failures.
And so we spent a lot of time practicing.
And you work out all the different scenarios.
This is what you will see if you have a real failure.
This is what you will see if you have a sensor failure.
And, before you shut down a good working engine, you would like to confirm the fact that it is really an engine problem, not a sensor problem.
On the other hand, you don't want to take a chance that you're going to get into an oxidizer-rich situation and blow up your engine.
You practice and after a while get pretty good at diagnosing the problems fairly quickly.
The computer doesn't normally shut down the OMS engine on its own.
The computer will not shut down any of the propulsion systems on its own.
It will shut down the main engine, but none of these will it shut down on its own.
Somebody has got to take some action to shut it down.
We get an awful lot of data on the ground, too, that you don't have in the cockpit that you can look at different instruments and backup things and try to understand where there is a sensor failure.
Any other questions on OMS/RCS?
During entry, when you're coming down using the RCS for like trajectory guidance before your aerosurfaces are usable, how much propellant would you typically use?
Is there like a certain percentage that would be average?
That's a pretty low percentage, and I don't recall.
That has been a long time now, and I don't recall the numbers.
But I would say it's less than 10% of the propellant.
What did you say?
About 10%.
Well, at least that ought to be a pretty good number.
[LAUGHTER] It's not a lot of propellant come back in on it.
The vehicle is a fairly stable vehicle.
Now, on Columbia, I guess they either ran out of propellant or just about ran out of propellant on it trying to hold it.
RCS was doing all it could to try to hold that vehicle on course after it started picking up drag on one side.
And we actually did have procedures in place for what we called a no RCS entry if you had run out of propellant entirely.
I don't know if it would have worked.
You tried to get it into a stable aerodynamic situation as possible and just hold it there.
I never had to actually do it.
Anymore OMS/RCS questions?
Otherwise, why don't we take a little break and then we will come back for APU/hydraulics.
Two minute break.
Thank you all.
Appreciate it.
You're not finished.
APU/hydraulics.
And I guess I need to point out some acronyms.
Calling the APU an APU was the biggest mistake we ever made.
That gave me more grief than any other system that I had to deal with, the acronym.
APU stands for auxiliary power unit.
And the Aerospace Safety Advisory Committee -- I will tell you, Henry, my kids growing up used to think it was a three letter bad word.
I mean they thought APU was a bad word.
[LAUGHTER] Because every time I got a call there was something wrong with it.
If we would have called it primary power unit then we would have had a lot less grief in trying to defend it to the outside world.
APU/hydraulics, like the OMS and RCS, all of our first activity was directed toward keeping the operational cost down low, keeping the maintenance cost down low.
And we looked at those kinds of systems that gave you clean propellants, easy propellants to deal with and you didn't have to worry about the strong bases and strong acids to work with.
Later on, as time went on, we got more into looking at what we could do cheap from a development standpoint and let the operational cost float, because it is obvious that the money was not going to be there to put into a big development program.
The control of the vehicle in the atmosphere was a major, major activity to try to find out what you could get by with versus what you would like to have.
You try to move these huge barn doors fast and a whole lot, there are huge forces on those things.
It takes lots and lots of power to move them and move them quick.
And our aero people and our guidance people would like to have, like we were talking a while ago, an instantaneous impulse into that vehicle.
When it starts to drift a little bit and they want to move it some other way they like to put an impulse in there and just kick it right back where it is supposed to be instantly.
But that becomes impractical when you look at these kinds of systems.
We started out with dual tandem actuators.
Now, dual tandem actuators is something that fail op fail safe criteria.
In other words, you had two pistons and one actuator driven by two different hydraulic systems and you would move them together.
When you start looking at all of the failure modes associated with dual tandem actuators, even though it meets the fail op fail safe criteria that makes the safety people happy and the people that are looking at the buzz words, it actually was making the system less safe.
Was that because you had a single passenger [using up?]
all the hydraulic fluid?
No.
What happened in a dual tandem actuator, in order to get them to work well, if you busted an actuator then that became a big sponge?
You didn't have any way to take that space out.
Trying to have that actuator work when the other half went out made it very, very difficult, from a design standpoint, to come up with a system that you could lock it in place.
And we even had designs where if one hydraulic system failed that that shaft would actually lock itself.
With the absence of pressure it would lock itself.
Well, that becomes a failure in itself because if that mechanism fails then you've got a locked actuator.
One thing you cannot stand on an aeroplane is for an actuator not to move.
We were finally able to convince the people.
And Aaron was having lots and lots problems about that time because all of the weight was in the backend of the vehicle.
The dual tandem actuators were extremely heavy and the backend of the vehicle was getting too heavy, so we actually went to a single actuator.
And we put switching valves in there where we could switch any one of the four APU systems into that actuator.
And we had four APUs at that time.
Weight was still a problem so we were able to convince the community that we could live with three APUs and have one APU fail and come home normal mode with two.
And if you had two failures you could still land the vehicle with one actuator.
You lost a whole bunch of systems but you could still land it with one APU.
And we almost did that one time, because if someone would have told me that we would ever have two failures of the same type on the same flight of the nature that we had I would have never believed it.
But we came home on one flight with two APUs burning.
We had a fire in both of the APUs.
And the reason for that, after the fact again, is very, very simple.
We went through a very good quality program.
We did everything that we needed to on those things only to find out that we had not planned on landing the vehicle in California and piggy backing it back down to the Cape over and over again.
What happens when you do that is that you land the vehicle, there is a little bit of residual APU hydrazine in those tubes between the valves and the injector.
You go back up into a vacuum or low pressure.
All the air goes out of the system.
And then we come and land in Florida.
It is always humid in Florida.
There is a lot of moisture there.
And as that moist air started feeding back in the engine, it went up past the [cat?]
bed and got in those tubes between the valve and the injector and set up a very, very corrosive environment of residual hydrazine and water.
And that gave off free hydrogen.
And you had hydrogen or intergranular corrosion in those tubes.
And we had two of them break on the same flight, but it worked.
We spent an awful lot of time looking at power sources to power the hydraulic systems.
And we even looked at going with all electrical systems.
DOD had some very good fuel cells out there that had a lot of promise.
It put an awful lot of power at that time.
Let me just digress a little bit and talk about shuttle fuel cells.
I don't think anybody can cover that.
On Gemini the fuel cells never did work.
We always came back with half voltage or partial voltage and most of the fuel cells out.
On Apollo, it only took 14 PhDs to start them and shut them down.
And then you couldn't start them again.
On the Space Shuttle, those fuel cells, it is just a very, very good battery with the chemicals stored external to the battery.
They would make an outstanding DC welder.
I mean you could put electrodes in that thing and put an electrical rod in there and strike and arc and weld with them and break the arc, strike and arc and weld and break the arc.
And they just do it repeatedly.
You can throw a switch, they are on, you throw a switch and they are off.
And so they are very, very simple.
I really, really wanted us to go with an all electrical system using electrical motors and power hinges and electrical mechanical actuators to drive the systems because I was absolutely convinced that one of these days we were going to have a leak in the hydraulic system.
And when we have a leak in the hydraulic system we are going to have a fire.
All airplanes have hydraulic leaks.
Now, you don't have to worry too much about hydraulic leaks on an airplane.
You have enough pressure where you seldom have to worry about a fire.
We went from 5606 hydraulic fluid to 83282 hydraulic fluid simply because it was advertised as being more flame-resistant.
It is really not more flame-resistant.
The 83282 hydraulic fluid has a much, much lower vapor pressure than does 5606.
And a way that they test it is they have got a Bunsen burner out here.
They take a pipe cleaner and dip it in the fluid, clamp it in this device that rotates past that Bunsen burner and they count the number of times it will go across it before the fire starts.
Well, with 5606, it will usually start on a third burn.
With 83282 it takes 10, 11, 12 passes before it will ignite and start burning simply because it has a lower vapor pressure and you cannot burn any liquid in a liquid stage.
You have to get it warm enough so that it will gasify.
And then you have to heat it enough in a gas phase to get it up to the combustion temperature or put a spark in it after it is in the gas phase.
But it won't ignite as a liquid.
Well, with 6506, if you have a leak in the hydraulic system going uphill you probably don't have to worry about it coming home because it's gone.
It all boils off and is gone.
With 83282 all it does is soak out into the structure and in the insulation and all over the place.
And then when you come back home, if it gets in a hot spot and you start heating it up, it forms a gas and you're going to get an explosion in the back of the vehicle.
So, that was a bad decision, Aaron.
We went in a bad direction, but it gave everybody a warm feeling.
We would have went with an electric mechanical system but, by that time, we had a lot of hydraulic people working on the program.
And we would lay those people all off and we had our whole bunch of electrical mechanical people, and a lot of people were concerned about the immaturity of those systems.
And again it boiled down to development cost.
Did we really have the energy source that could handle that?
Oh, I'm convinced that we could have put six or eight of those fuel cells that we've got on there right now and could have handled it.
Broke it down.
And the fuel cells that DOD had under development at that time were very, very good fuel cells.
If I was upgrading the shuttle today that is one of the things that I absolutely would go to.
I would go to something to get rid of the hydraulic fluid.
It creates a lot of other kinds of problems that you have to deal with.
Actually, before the decision was made that the shuttle was going to be retired in five years, NASA was working on upgrades if we were going to fly the shuttle for another 20 or 25 years.
And one of the upgrades, which actually got to a fairly advanced stage in the design, was just what you were saying.
Get rid of the APU/hydraulics, use the latest technology now available for electromechanical actuators.
But the problem, I don't know the details of it, but the costs just kept going up and up and up.
And by the time it went up above about $300 million they just said we cannot afford to do this.
The problem we always had in trying to upgrade the propulsion, we could have gone to an alcohol/LOX space RCS/OMS on it.
We could have upgraded to an electromechanical system, but it seemed to me like that the displays and those kinds of upgrades took a lot of computers.
Those systems took a lot of priority in where the monies went in trying to upgrade the Orbiter.
We did go from four to three systems.
We did go from 4 APUs to 3 APUs and still having the ability to land the vehicle with one APU.
We look at hydrogen/oxygen APUs.
We looked at bipropellant APUs.
We looked at monopropellant APUs.
We looked at pulse modulated versus pressure modulated.
If you don't know what I mean by that, pulse modulated is when you have a valve that goes open and closed and you get steady state pressure in your gas generator while it is on.
Pressure modulated you have a throttle valve where you throttle the flow down going to the gas generator to give you just the power that you need for the load that you are trying to pull at the time.
And that was the easiest system to come up with simply because we did not know at the time if we could build a valve that was capable of millions and millions of cycles and not a leak.
But the problem with a pressure modulated system was that most of the time you're operating at very, very low power levels.
The power just peaks up every once in a while to high power levels.
But when you're running very low pressures in a gas turbine, the turbine becomes less efficient.
And the less efficient the turbine becomes the more propellant you have to burn and, again, the weight goes up.
So we kind of bit the bullet on that and went to a pressure modulated system where we varied the ohm time.
In valve development we were very successful.
We had very little trouble getting a valve that would work, except for contamination.
We always had a little contamination.
We had a lot of problems down the Cape with early valves leaking.
We finally put good filters in the system and that eliminated that.
Yes.
Was the reason to go from four to three APUs just a weight issue?
It's again a weight issue.
All of that weight was in the back of the vehicle, and the back of the vehicle was always too heavy.
And every time we could take a pound out of the back of the vehicle, we could take a pound of ballast out of the front of the vehicle, so there were two pounds you didn't have to carry.
So, it was strictly weight.
I'm sorry, the APUs are turbines?
I mean you're combusting [UNINTELLIGIBLE]?
The APUs we settled on was going with a monopropellant hydrazine, feed that through a catalyst bed and then through a turbine.
We looked at dual stage, triple stage turbines.
We finally wound up with a single stage turbine running at about 38,000 to 45,000, I don't remember the exact number now, RPM.
About ten inches in diameter [UNINTELLIGIBLE].
The gas went in and then made a pass and went back through, so we got maybe 5%, 10% more power out of it the second time through, a little bit more power out of it the second time through.
When we contracted for the APU, the contractor did not have altitude facilities.
And so they proposed that we did not need to test the APU in a vacuum because that was all structure and all they had to do was pull a vacuum on the exhaust and could get the performance out of that.
So we did not have it in our quality program to test the APU in a vacuum.
Fortunately, Dr. Cohen gave us enough money to buy an APU offline of the Shuttle Program.
We brought it down to JSC.
We put it in our vacuum facilities there.
And we programmed the vacuum facility to go down in pressure.
The Orbiter was going up in altitude.
And we ran the ascent profile and shut the APU off just like we would on a first flight.
And five minutes after we shut it off it exploded, it detonated.
I mean talk about things coming unglued.
Everything came unglued.
Aaron came unglued.
We got another APU.
We brought it down there.
We brought all of the contractor people down, everybody down there.
We repeated the test.
It exploded again.
[LAUGHTER] By that time we discovered what the problem was.
We have a requirement in the program that you can only have hot surfaces up to about 500 degrees.
I don't remember the exact temperature, but some temperature like 500 degrees.
If it is higher than that you have to shield it.
The gas generator is obviously running over 500 degrees, probably up around 800, 900 degrees, so we put a heat shield around it.
We covered that whole thing up and had about a half-inch standoff in the insulation on that thing.
When you run it in one atmosphere, if you shut it off it acts like a chimney.
The air heats up between the insulation and the hot surface and goes out the top.
It draws cold air in.
It goes out the top.
It draws more cold air in so it cools it off.
So the heat never did soak out into the valves and get the valves too hot.
When you put it in a vacuum there is no place for that heat to go.
And so all of that heat that was in the gas generator and all of the catalyst and all of that soaked back out through the structure and through the tubes back up to the valves and got the valves up to the temperature at which hydrazine will detonate.
And it did.
It is so simple.
It is so easy to understand after you know about it.
But to think about it, I remember sitting around there one night to 8:00, 9:00 with all of my troops discussing the pros and cons of an altitude facility.
Do we have to have it?
Do we have to direct the contractor to put one in?
And we could not come up with a good reason to put an altitude test in the program so we didn't put it in there.
Fortunately, just like on ohms with the helium bottle falling out, we had a backup because some of the people weren't confident in flying the vehicle without that test.
And we found something that we didn't expect.
The message that Henry is giving you on testing is the same message that J.R. Thompson gave you on testing on the main engine.
And, in contrast, we didn't really do that type of testing on the solid rocket booster, the fair we had and on the foam.
And that is the difference, we really tested.
When we saw a problem we tested it.
And we came out very fortunate, but we did do it.
Another thing with the development of the APU, we flew those things on airplanes all the time in case they lost main engine, our fighter aircraft and then some of the other airplanes including that supersonic plane that we had.
It had APUs in there in case they lost engine power to provide hydraulics to be able to land the vehicle.
And that got a lot of the flack because the Aerospace Safety Committee told me that those systems never did work because they're called auxiliary power units.
They never worked when you needed them.
But they had those systems.
And they were good systems.
But when we tried to use that same kind of design on the Shuttle the oil wouldn't go back to the sump in the absence of gravity.
You sling it out and it just coats the walls on an automobile.
You put an oil pan down the bottom, you put a pump right at the bottom, you put a little filter in there and pump the oil out and pump it through the engine and use it over and over and over.
But when you have no bottom, you have no place for that oil to go.
So we had to come up with a technique where we could use the gears as oil pumps.
And we came up with very, very close tolerances between the gear and the case, and let the gear sling the oil out or pump the oil out into a cavity and pipe it off to where we had a sump to where we could pick oil up with an oil pump and pump it back through the system.
And we had some problems with some hydrazine getting in there and gelling it one time.
And Fram came out with this deal where you could pay me now or you can pay me later kind of thing because they still aren't changing the oil filter.
Hydraulics was mostly off the shelf at the time, the pump and that stuff.
We did go to titanium propellant lines or hydraulic lines.
We did develop a special fitting to put titanium lines together.
It turned out to be very, very good.
We did have to add a water boiler to cool the hydraulic fluid because on airplanes just put a little radiator in there and use the atmosphere to cool it, but that didn't work in space.
The first one they came up with was a bucket and it had a core of tubes in that bucket with a pipe that went out with some baffles in there to keep the liquid from going out.
But there, again, they did not understand the absence of gravity.
When in zero G and you put heat in that water, it just pushes the water out away from the tube.
And all the water would go out the exhaust pipe and you would have nothing but gas in the tank.
We had to change that design and go into what we call a water spray barge.
To pulse the water in there as a spray is more efficient.
But complicated that system very much.
It still does.
It is very, very difficult to handle water in a vacuum.
Getting back to the RCS one moment, the first rain we had down there at the Cape, the RCS jets filled up with water.
And they wanted to launch that system assuming that the water would boil out when they started going uphill.
But, no, what happens is when you put water in a vacuum, about 60% of it will go to a gas and 30% will freeze.
When it freezes then it is a solid, you cannot get it out, it just has to sublime which takes weeks and months.
Now, trying to get us a water boiler, we had valves in there that were supposed to open and close when the pressure built up.
Well, when you get past that valve, when that valve would open up that steam would instantly freeze on the outside and it keeps growing back.
Finally, the valve would be stopped up and you would build the pressure up way high and would blow the ice out.
The pressure would come back down and start working again.
So, we went to an orifice.
Trying to size that orifice to the right size where it maintains a little pressure in there, and you've got a variable load on the system and you're putting variable energy into it, that's difficult to design that, too.
The water boiler still freezes.
We still have a freezing problem on it.
It is not an easy solution to it.
Questions?
Do you lose that water?
Yeah, we just dump it overboard.
We have a whole bunch of tubes going through this water, and we spray the water in, in a spray.
And it makes one pass through those tubes and then it goes out the exhaust.
If you wanted to come up with some kind of regenerative system where you would use the water over and over and over then you've got to go through some kind of system to cool the water or you have to carry too much water aboard.
Well, if you want to cool the water, the only way you really have got to cool it is by using more water or supplementing some of that water.
In a vacuum, you cannot just put it out on a surface some place.
I guess the temperature varies maybe.
Henry, could you say a word about the flight control hydraulics lab, the tests we ran with the hydraulic systems.
The hydraulic system was probably one of the most thoroughly tested systems that we had as far as mechanical systems were concerned.
We built a whole hydraulic system integrated with the avionics, and we had it designed so you could put loads on it and react the loads.
And you could program in what you thought was typical emissions where you could vary everything.
The only difference is instead of driving the hydraulic pumps with APUs, we drove the hydraulic pumps with electric motors because we had ground-based electric motors to drive them.
And one of the things we ground ruled out was the use of flex hoses -belts- simply because they, at that time, were notorious for fatiguing and breaking and not working very good.
So we put trombone tubes in.
Take a long tube and bring it back this way and tie it in.
And now, as the actuator moved, it could move.
And you had enough spring or enough give in those tubes.
On the very first time they fired that thing up, standing up in there in the control room looking at it, those trombone tubes just vanished.
They would be there and then all at once you wouldn't see them.
And I mean you couldn't see them.
You're standing up there in the control room looking and there are no tubes in there.
They were shaking so bad that you couldn't see them.
So we went through a big effort to figure out ways to damp those things and isolate them so that they didn't vibrate.
We had a major failure.
It was a hard, hard sell to convince the people to go to a single actuary, from a dual tandem single actuator.
Well, no more than we got our first single actuator built, the initial design had too much slack in a tube that went from one side to the other side because you had to have an expansion joint in there.
And we had an O ring in there.
And they had too much slack in there and it blew one of the O rings out.
Well, of course, when you blew an O ring out you lost all the hydraulics not only from that system but all three systems because it was downstream of the switching valves.
And when one system ran out of fluid it would switch over to the other one and you would run out of fluid.
Had we not had a very, very strong guy in the Program Office at that time that never gave up on anything that would have probably finished us off as far as single actuators are concerned.
But we were able to convince the program that if we tightened the tolerance up on it, paid very close attention to the tolerance on it that it could not happen.
And it has not happened again.
Another question?
Yes.
You talked a little bit before about maybe substituting an electromechanical system for hydraulic.
Would there be any other changes you would make in the design if you had to go back and do it again?
Oh, if I had to do it again, I definitely would go with electromechanical systems.
Now, that takes on many varieties.
You could have small electric motors driving a hydraulic system right at the unit.
Like you put an electric motor on an actuator.
And that electric motor drives a small hydraulic pump that would move the actuator if you did not have confidence in worm gears and screw jacks and those kind of things to provide the mechanical force, power hinges and things like that.
But there is absolutely no question in my mind that one of the safest things you could do for the Orbiter right now would be to replace the APUs in the hydraulic system with an electrical system.
We have the motor technology, we have, I think, the gear and the ball screw technology to be able to do that.
I think we could do it cheap.
I think that would really change the operational cost.
It would reduce the operational of cost immensely.
It would move the CG further forward on the vehicle so you probably could take out a little bit more of the ballast that we usually fly on the front-end.
I guess we're still flying it.
We always did.
I have another question.
You talked earlier and you had it on the slide, but you didn't really talk about it, how you were shifting weight between budgets, when it came to like the controls people versus the hydraulics people and the APUs.
And I was wondering if you could kind of just expand on that of how, in an actual development program, it goes back and forth.
I mean does it usually come down from high you're going to do this and you're going to do that, or is it usually they let you kind of work it out between the groups or what?
The control people have no weight budget.
I mean their budget is the electrons that flow back and forth and the requirements.
What they do is they start out a requirement that they would like to have.
And it is usually two, three, four times what they absolutely have to have.
And then you have to sit down and start negotiating with them and explaining to them what it is costing.
And often it gets down to the fact that the vehicle won't fly.
I mean it just flat won't fly.
And when they are convinced then that it cannot fly then they are willing to concede that they can get by with a little more.
And it still won't fly and so they can get by with a little bit more.
And that's kind of the way you get it done.
That's what you do.
You have a work breakdown structure, and you parcel that work breakdown structure out to your subsystem managers.
Now, there was an issue in the Shuttle and the Apollo Program, since I was a project manager, you give the subsystem managers the requirements and the authority technically.
You tell them what their constraint is for dollars, you give them the weight bogies or the weight requirements, the function requirements, performance requirements, the schedule requirements, but you keep the dollars.
Now, that's been a very long argument.
Do you allow the subsystem managers to actually have control over the dollars?
We decided not to.
That was a very big argument of whether you should let the other people have the dollars.
I kept the dollars.
They had to come to me if they wanted to make a big change outside their work breakdown structure because I had the problem of trading off the dollars between the propulsion system and the structure system or the aerodynamic system.
And you can argue that was the right or wrong thing to do but that was how we did it.
And I am not sure if aircraft companies like Boeing now or Lockheed Martin how they do it, but they all start off with a work breakdown structure of some type.
Yes.
At that stage,do you also give them target weights?
You can target weights also, yeah.
Negotiated target weights and negotiated schedules.
And they develop the performance.
You basically use the target weights, from the preliminary designs that you've done, to look at the concept that you're looking at to make sure that it is feasible.
And when it looks like that is a feasible design, then you break up the weights and you give all of the subsystem people their target weights to stay within of which almost immediately the weights start growing.
That's right.
And, when you're a project manager like I was, it is career limiting.
The first thing is your weight starts to go up.
The next thing is you start to have schedule slips because you're finding technology problems.
And then the cost starts to go up.
You really come with a lot of career limiting problems in project managing, but you have to have good people working for you.
I think, Henry, another interesting thing you might talk about is, they've seen Bass Redd talk about the aerodynamics, they are going to have Phil Hattis talk about the guidance, navigation and control, you might give them your perspective with the hydraulic system of tying those systems together.
You mentioned it a little bit but it is probably a little bit harder than you think in putting the requirements on the hydraulic system.
I probably spent more personal time dealing with the avionics people, the guidance people and the aero people than I did working with the subsystem designs early on just trying to get some reasonableness in the requirements.
Trying to get the size of the OMS engines down, trying to get the size of the RCS down, trying to get the pulse width a little bit wider.
On the hydraulics, that was a major, major issue because those were really big weight items.
On the OMS and RCS it was not a big weight impact, it was a performance impact of how much propellants you had to carry onboard.
But on the hydraulics, on the design of that system, how fast you had to move one of those flaps on that thing was a big big issue.
And I spent lots of time going over and looking at their data, looking at what they were coming up with, looking at their what-ifs.
That is what drives a system, is what if this happens or what if that happens?
Another problem I remember we had was the gimbal rate of the main engines.
The requirement they put on the gimbal rate.
That was a compounded problem, too, because you were kind of dealing between centers.
But, yeah, the gimbal weight.
And the need for a gimbal system on the solid rocket motors, if I had my day in court, we would take the gimbal system off of the solid rocket boosters.
That would save six or seven or eight tons of weight back there.
If you would take that system out, it would make the solid rocket boosters much more reliable because they've got those great big old boots in there and those big springs.
They horse those things back and forth with huge actuators on them.
And the only reason we could not eliminate the gimbal system on the SRBs is that we had one SRB that burned out with the max burn time.
And the other one on the other side burned out with minimum burn time.
And we lost the top engine on the SSME on the same flight.
Now, you know the probability of that happening is once in a billion.
But, yet, that is what drives the requirement for having the gimbal system.
We would have to gimbal the main engines another two degrees to make that work.
But that is what drove the design, that requirement that we have on the solid rocket motors.
So, if I can leave you with one message, it is understand your requirements extremely well and understand the impact that that requirement is having on your system and on the flyability of the vehicle.
I told you that Henry had a lot of innovative design.
Talk a little bit more about the RCS.
I thought you were going to mention that before.
When they got water and how you solved that problem.
This is Henry Pohl's solution to a very complex problem on the RCS system, on the pad.
When we got the RCS engines filled up with water we needed to come up with a design to keep the water out of the engines in case of rain because it always rains at the Cape.
So that requirement fell on the contractor to design a system to keep water from getting into the RCS engines.
They came up with a plug with the throat and another big seal to go around those big scarf nozzles out there with an O ring on that.
And that was all tied together with cables.
And they stuck a big pole upside the vehicle.
And just before liftoff they were going to pop that pole over and jerk all of that stuff out.
And they were going to jerk it out fast enough so that those big tanks would not fall down and hit the tile and knock all the tile off.
Well, that looked kind of complex and complicated to me.
We had a grocery store across the street by the name of Wine Gardens, and I went down and got a roll of that wrapping paper that you use to wrap meats in that has wax on one side and then paper on the other side.
I call it butcher paper, but butcher paper is actually a little bit different.
If you go down and buy something it is freezer paper, but I bought a roll of that.
Then I got back up into the meeting by the time it was over and said this is the way to fix it.
And I started tearing off sheets off and passed it around to the managers to look at.
We are going to glue that on the surface with the wax side out and glue it on with RTV.
And then, when you lift it off, it would come off.
It was light.
It wouldn't hurt the tile.
If it impacted anything it wasn't going to hurt anything.
If it didn't come off when you fired the RCS you would blow it off or burn it off.
And that is what we did.
And we glued that on all of them.
[LAUGHTER] It sounds funny but it is true.
You look at the Orbiter sitting on the pad and all the RCS jets are covered up with paper.
But after I left they changed that.
They changed from butcher paper to a special paper developed special for that purpose that is made out of Teflon.
You have to tell them one more story about the SSME on the launch pad and the hydrogen.
This was JR's.
When we did the first firing on the pad -- Just a test firing on the launch pad to make sure that everything worked right, we blew the back end out of the shuttle.
When those main engines light off, each one of those engines dump out about 125 pounds of hydrogen before it ignites.
That hydrogen is cold.
It is fairly dense in liquid form.
It is 4.5 pounds per cubic foot or something.
By that time it is a little bit lighter.
It goes down in the flame pit.
It mixes with a lot of air in there.
And then the flame hits it and detonates.
And it blew the backend out of the vehicle.
We needed a quick fix because we're going to launch that thing in three or four days.
I looked at the problem and said we will just put Roman candles under it.
You put a bunch of Roman candles under there that fire those balls out across there and the hydrogen comes out.
That will ignite the hydrogen.
It will burn and won't accumulate behind the pit.
And that is what we did.
Those sparklers you see coming on before the main engines light off, those are my Roman candles.
And as long as we're talking about blowing the back end of the Orbiter out, maybe you can talk about the pressure wave as well.
The pressure wave that came back and the water suppression.
We had another problem, more of an acoustical problem from the pressure waves coming back up under the Orbiter, and we needed some kind of a solution to do that.
Back while I was in Huntsville as a test engineer in a test lab there in Huntsville, we blew the windows out seven miles away when we started firing the Saturn 5 vehicle.
We spent a lot of time on sound suppression techniques to reduce the pressure effect.
And one of the things we came up with is we found water is very good at damping that.
When we had that problem down the Cape what we did is put those sausages, the hammocks across down there and fill them with water to damp out the pressure wave that came back up to the back of the vehicle.
There is another one of them.
Essentially like big water balloons.
We called them sausage.
It was a fabric across the bottom, filled it up with water and it was strong enough to hold the water up.
And then that dampened the sound waves, the shockwaves that came back up through the vehicle.
Again, just remember that it is not natural for earth-bound humans to think in terms of the absence of gravity and the absence of pressure.
I cannot stress that too much.
You take a pot of water, you put it on the stove and it gets hot on top.
You put it on a stove at zero G, it does not get hot on top.
It pushes all the water out.
Heat transfer.
You don't realize how much heat is transferred out through the atmosphere.
If you have no atmosphere you don't get any of that heat transfer out.
The other thing to remember is that there is absolutely no substitute for a very well thought out test program.
It is not easy to test in the absence of gravity.
It is not easy to test in the absence of pressure when you are reducing large quantities of gas or chemicals.
But that is very, very important.
And that is the one thing that really separates a space program from anything that we do here on earth, is that you have no atmosphere and you have no gravity to help you in your design.
Yet, all of our thought process is based on having gravity and having an atmosphere.
And some of the designs of various experiments that I've worked with on Spacelab flights and other flights, numerous failures.
And over half of the equipment that failed, this is stuff generally that we use inside the shuttle, it was because of thermal problems that they would overheat just because people didn't get the calculations right.
You have no convection.
And getting the thermal design to work is just really difficult.
Convection is one of the major problems that is overlooked because when you put heat in one place on the ground it goes up.
If you take the gravity away it will not go up.
And that changes the whole complex, the thermal matrix.
Henry talked a little bit about the screen system that was developed for the OMS and RCS tanks, but just to make sure you appreciate the complexity of this, you've probably seen pictures of astronauts playing with food and liquid, you blow these big liquid bubbles and they just sort of float there.
Well, the liquid does the same thing inside a fuel tank.
And when you push the button because you want to turn your engine to turn on, how do you get the fuel to flow into the engine?
And, as Henry said, in the old-days you had kind of a diaphragm.
Everything was in like a little balloon and you would pressurize the outside of the balloon and that would force the fuel out, but that balloon would get eaten away.
I mean remember that was a system that is only used for one flight.
Now we have a system that has to be reusable.
And if you didn't want to replace those diaphragms or balloons every flight you needed something that was a different system.
And that is where they came up with the idea of the screens which would have surface tension so enough of the propellant would stick to the screen.
And then you would get the helium on the other side that would push it out of the screen.
With the OMS, of course, once you got the system started then you're producing an acceleration.
And it is always in the same direction so that the OMS tanks are designed so that they only have to feed enough propellant through the screening system to get the engine started.
And then everything, I won't call it gravity-fed, but is fed by the acceleration.
When both engines are burning, it is about a tenth of a G. But the RCS, that can push you in any direction.
And so you have to have a system which will continuously feed, no matter which direction your acceleration vector is.
It is a really complex and cleaver system.
And the design is based strictly on surface tension.
Why are there so many RCS engines?
For example, in the nose, there are 16.
Is that for redundancy?
Part of it is for redundancy and part of it is to make sure that you've got one to cover every direction.
I guess we've got six going down in front.
Now, we need two of those for certain maneuvers.
So you can lose four of them.
And we have four going out.
You can lose one on either side.
And remember you need a couple pair.
You would like to be able to do a pure rotation.
If you just fire one engine in the aft you're going to get a rotation, but you will also get a translation.
So, you need a couple pairs.
And then for redundancy we have three sets.
If you add up the x-axis, the y-axis, the z-axis, you've got to be able to translate in three axies, you've got to be able to rotate about three axies, and then you need dual redundancy for each of that axis.
You add it up and you end up with 38 primary jets.
And then, on top of that, those are 850 pound jets.
They use a lot of propellant.
When you are in orbit generally and all you worry about is your attitude and you're not doing a propulsive burn for rendezvous or anything, we shut down the primary RCS system and we just use six little 25 pound Vernier thrusters, all of which are pointed down.
When you use those for attitude control they do give you a little bit of propulsive impulse as well.
But it is a small enough thrust that it doesn't change your orbit by very much.
Let me tell you another story about the expulsion devices on the Shuttle.
Back in my young days, I grew up in the country, and we had a Ford tractor.
The Ford tractor had a screen in the fuel tank and it stood up about that tall.
And it had a little stand pipe in there.
And the valve was such that when you got down you could set the valve one way and it would leave about a gallon of fuel in the tank.
You turn it the other way at the end and it would use that gallon of fuel out, kind of like a gas gauge on the early Volkswagen.
Well, what happened is that I had that out one day and it had a tenth of an inch diameter hole punched in that screen about halfway up on the screen.
I said that screen is not doing any good if it has a hole in it.
I got me a ball of solder and I stopped up that screen.
You know, after that, you could have it set where the fuel are supposed to remain in the tank.
And when it quit it was empty.
You'd switch it over and it still wouldn't run.
You couldn't get home.
You would have to walk home.
Well, it turned out that they had put that hole in there as a vacuum breaker to break the vacuum on it.
And that is the way that the Shuttle system works.
The screens that we have in there, it is fine enough so that the surface tension across those small pores is strong enough so that it will keep the air from punching through, unless you put a big hole in them.
We had to make sure.
And that was another one of the things that convinced me that we could design a screen system that would work in the absence of gravity.
It goes back to a Ford tractor.
It goes all the way back to 1947 and a Ford tractor.
Henry, we've come to the end.
Let's thank Henry.
[APPLAUSE]
I'm going to do that.
I encourage questions.
I may not be able to answer all your questions either because, A, I don't know or, B, I cannot say.
I am going to leave it to you to figure out which one is which.
[LAUGHTER] I would like to say a few things before I start.
One is I'm not a historian so I'm not here to give a history of the military and the Shuttle.
I am here to give you my experiences with it which is a very select data slice, I think, as you will see.
Second, the things I say here, although we're going to tape them, I have to [NOISE OBSCURES] very carefully.
And, third, I really speak from a different era.
I speak from an area of the '70s and early '80s when the political situation was a lot different than it is today.
When a Cold War was hot.
When there was a real threat of a massive land war, plus worse, in Northern Europe.
When there were real threats of nuclear exchanges between the super powers.
Things which are not true today, thank goodness.
We have now been overcome by other threats and challenges, but I speak really from an era of the '70s and '80s.
So, with that, I would like to start.
Again, I encourage questions.
Here is what I would like to do today.
I would like to give you a little bit of intro and background.
Talk about DoD and Air Force space missions.
There is a difference which I shall highlight.
I will talk about military system requirements and impacts on the Space Shuttle as a system.
I will talk about facility impacts because that is easy to do, but there are impacts that are far more far reaching than that.
I will close a little bit on national space policy and then I will show you a ten minute video.
For the video I would like Tom to turn off the recording and we will just watch it kind of in silence.
OK. Any questions?
My background is I served 29 years in the Air Force.
As you folks know, you can retire up to 20, so I couldn't count.
I stayed far longer than I should have.
The reason I stayed was because the Air Force gave me assignments I could not turn down.
I have had lots of experience from a program office payload user perspective on Shuttle, Titans off both coasts, MX, Pegasus XL.
Last January I gave a talk, which I will repeat this year, on my activities on two accident investigations.
One on Titan IV and one on Pegasus.
I do that at one time.
It takes about an hour and a half.
If any of you are on campus in January, please come by.
It's an interesting story.
I served in multiple program offices in programs A and B.
Programs A and B are subsets of the NRO.
I won't say any more about that, but Program A is the Air Force element.
Program A not only built Air Force satellites for the intelligence community, they also provided all the launch services for the intelligence community.
I have been director of two major programs.
The first one Space-Based Infrared Low.
The second one was the DoD Space Test Program.
I have also been the NRO Director of Safety.
In general, Air Force and DoD space missions fall into this laundry list.
The first one is Navigation which you know from GPS Communication which has many satellites of which Nelstar and DSCS are the ones you recognize.
DSCS was launched on the Space Shuttle I believe twice, I'm not sure.
Meteorology, the Air Force launches the DMSP which is the polar orbiting weather satellite now replaced now by the NPOESS system.
For missile launch detection, the intelligence community and the Air Force have either operated the DSP satellite system or else proposed SVRS which is suffering tremendous programmatic problems currently.
For arms free verification, it is terribly important to have complied at one time with [UNINTELLIGIBLE] limitations.
That is an interesting slice of history where at one time the total number of warheads, total number of bombers, submarines, ships, this sort of thing was limited by international agreement but it was the responsibility of the other side to verify compliance.
There is also nuclear weapons test detection on Vela.
And then a whole handful of what I call intelligence community or IC programs.
Bob Siemens, a retired professor emeritus is planning to drop by today for the Q&A.
His position, his role in this story is really very central.
His influence is woven throughout the whole story of the DoD and the Space Shuttle, and I hope you will get a chance to talk to him.
An extremely interesting and very pivotal guy.
I would like to concentrate on Vandenberg Air Force Base right up on the Eastern Range for the Space Shuttle because the Vandenberg Air Force Base launch site was really built up for sun synchronous polar orbits, for really one payload, and that was the fellow here depicted in the late 1970s, launching out of the Western Range.
This is a Titan IIID.
I have helped launched, I think, eight of these off the West Coast.
This is a heavy lift machine.
It is roughly 30,000 pounds in low earth orbit, 27,000 to 30,000.
It has two solid strap-ons.
I saw the first one go in 1970 and subsequently was able to transition into the Program Office to launch several times out of there.
As some of you are aware because I have talked to you out of class, I served for many years not in what we call the regular Air Force but what is called the Office of Special Projects.
It is called an Air Force Element.
We did not report for the usual chain of command through the Air Force chains.
We went straight up to the secdef of the JCS.
We were given the responsibility to launch things quickly, quietly, with great national pressure.
And we did that.
And the machines we used to launch, payloads included, are Titans of various kinds, Atlases, as well as the Space Shuttle.
The story I am going to tell you is really the special projects utilization of the Shuttle.
More so than what I call the regular Air Force missions.
In any case, when I joined SAFSP in the late 1970s, life was really exciting.
We were in the midst of phasing out all the expendables for the Space Shuttle.
And when I went in there, I found within the first week they said the Space Shuttle is designed for our program, because we were flying on a regular basis out of the Western Range.
So, the payload that sits inside here is a form, fit and function drop-in in the Space Shuttle bay.
At the time, there were no Space Shuttle payloads of that size yet on the drawing boards.
The intent was to have a seamless transition from Titans into Space Shuttle, first for our program and then for others.
We drove the payload bay size, as you've heard from Professor Hoffman and Aaron Cohen, 15 x 60.
The cross-range requirement dictated the configuration of the vehicle plus the wing size because the Air Force and the DoD was very, very insistent on having a return to launch base capability.
First orbit deploy, come back cross-range, land back at Vandenberg, recycle, be ready to go hopefully in a matter of days.
It is interesting to note the evolution of the Titan space launch family.
In the 1970s we pick it up with the Titan IIID we see here.
The 3E was a Transtage version used for basically out-of-space missions.
This is a commercial version.
But the DoD use went from the 34B, 3C, 34D through this family here, and now we see the 4A and the 4B which are the heavy lift guys.
These guys basically were the follow-on to Shuttle.
They lifted the Shuttle payloads that the Shuttle could not carry.
And I will explain that in a few minutes.
Very large, expensive, complex machines, complex programs.
At Vandenberg, it turns out that back in the '60s there was a program called the Manned Orbiting Laboratory, MOL.
Many of the original Space Shuttle Mercury astronauts were part of that.
They built up a space lodge complex up there called Slick-6 to launch the MOL.
It was going to be a space capsule with military officers onboard to do recognizance and surveillance flying off a rather large Titan IIIC.
To do that, again, back in the '60s, they built a launch pad.
I'm sorry.
As we got ready for the Shuttle, they took the MOL things and did all these things to it to make it shuttle-ized.
They operated a launch pad.
They put in a Mobile Service Tower.
As you know, at the Cape, what they do is kind embrace the Shuttle and its stack with a gantry.
And they pull back a little bit and then they launch.
The Mobile Service Tower is a large, almost like a skyscraper complex that completely encloses the rocket during processing and then rolls back several hundred yards to expose the vehicle for launch, a totally different OPS concept.
This facility here, the OMCF is similar to the KSC Orbiter Processing Facility.
Then they have other facilities here.
The payload preparation room where the payloads are received from transportation put together and tested.
The PCRs were the payloads actually put into the shuttle bay.
The Vandenberg runway was almost doubled in length.
And then all this infrastructure is put in here, including crew quarters for the astronauts.
And they were able to reuse the MOL facilities.
A lot of [UNINTELLIGIBLE].
But what you see here is the buildup of a complex very similar to Cape Kennedy only on a California coast.
Very, very elaborate, very, very complicated, very expensive, and taking a tremendous amount of budget to go do it.
Back in the '70s by law or by policy, which is perhaps even more powerful, the Space Shuttle was going to be the only launch system for all military, civil and commercial payloads.
This is somewhat of a forgotten fact, but at one time the cry went out from the advocates that said by having a reusable spacecraft that can launch 25, 30 times a year from both coasts, so that is times two, we can do great things.
And certainly, coming from the folks that gave us Apollo, this did not seem that farfetched.
The idea of launching a spacecraft like an airliner, as we know, did not happen in our generation.
I think in your generation, if you can make that happen, that would be extremely wonderful.
But at one time there was going to be a phase out, a shutdown of all the launch vehicles.
All the launch vehicle folks were told fly out what you have, close up business, break up your tool and dies, send your people home, we are going to go with Shuttle.
And to do that there was extensive redesign of payloads to fit the payload bay, the width, the length, to fit the safety requirements.
All these things changed.
So, it was not just a matter of just taking your existing payloads for these guys and dropping them into the shuttle payload bay.
It was a huge engineering effort.
And we thought at one time this is the way it is going to be forever.
The things that are unique about this is, first of all, liftoff and flight loads are much different because of not only the main events, everything from the SRBs, the staging, but also being able to survive safely, things like slap down on landing if you had to come back home with a payload.
Many of the heavy payloads require, as you know, upper stages.
The PAMs are the small payload assist module, small [UNINTELLIGIBLE] upper stages.
The IUSs, as you probably heard, are much, much more elaborate solid fuel devices.
The Shuttle Center, Cryo Center was going to be a cryogenic upper stage that was going to take our biggest payloads up to GEO.
That never got through the safety process but they worked it hard.
The safety requirements, to make sure that the crew was not endangered at any time by anything, either accidental or inadvertent or a mishap, were huge.
That is a whole separate discussion in itself, which I will be glad to have with you, but basically everything was looked at in terms of being dual fault tolerant so that in no case could any two events link up to cause hazard to the crew, even including accommodation of hardware and software errors.
This was huge and caused, in many cases, extensive retests and redesign of payload systems.
National Space Policy, back in the '70s and '80s.
First of all, in '78 there was "strong endorsement for the Space Shuttle to be the prime mover for national security and civil missions".
What is missing from this lineup here?
Commercial.
There were very few, almost no commercial missions at the time.
Civil means NASA.
The launch business of that time was dominated by the government, NASA and the DoD.
1982 the position was strengthened.
And, by the way, I suggest if you haven't already to look up these documents.
National Space Policy is voiced in these documents.
It is extremely powerful.
It is the basis for programs.
It is the basis for legislation.
It is the basis for budgets.
It is huge.
Every word in these documents is worked over at great length by all the competing stakeholders, and it is big.
But in '82, SCS was going to be the primary launch for national security in civil missions.
What that really meant was anybody else with a rocket was told you're in the wrong list, don't come back.
Then we had Challenger.
And in 1988 there was now a directive.
A 1988 PD that said we shall have a mix between man and unmanned launch systems.
This is in contradiction to this I'm sure as you notice.
An NSPD4 in 1981 now gave us the words "assured access to space."
Code words.
Anybody here care to tell me what that really means in plain English, anybody?
Yes.
Even if the Space Shuttle won't fly they want to have some alternatives.
That's right.
It says don't put all your eggs in one basket.
Have two ways to get to space for your heavy payloads.
This is authorization for what we now know as the EELV.
But also at the time shortly after Challenger, Pete Aldridge who was Secretary of the Air Force at the time, now an executive of the Aerospace Corporation, previously the director of the NRO, prior to Challenger in the '84, '85 time period started getting a little bit antsy about access to space because of all the delays in the Shuttle Launch Manifest.
And so he authorized, on his own authority, a start-up of the Titan fleet again.
This led to huge problems with NASA because what they saw that as, correctly, was a loss of support for the Space Shuttle Program.
But he did that.
And when the Challenger disaster occurred in '86, the Air Force was about 18 to 24 months ahead of where they would have been.
Here is Dr. Siemens.
Hi, Bob?
Hi, Pete.
You want to sit over here?
I would like to have you meet Dr. Bob Siemens.
And he will be here for the Q&A afterwards.
Bob, I'm just going over some of the National Space Policy.
Write-ups that kind of led to where we are today.
This was NSC37.
This was NSC42, the primary launch.
This was the post-Challenger mix between the manned and unmanned systems.
And then this was the assured access to space presidential declaration.
From a user perspective, the security requirements to fly on Shuttle were huge.
What was required to be protected was any information on the mission type and details.
LEO, GEO, HEO, you name it, that could not leak out.
Any information on the spacecraft, who made it, where it was supposed to go, of course what the payload did had to be protected.
Also protected was the Program Office and the prime contractors because often you can tell from that who does what.
And during operations the deployment time location and the final payload orbit had to be protected.
This is huge.
As you know, thousands of people participate in getting a mission together for the Shuttle.
It starts with a mission planning template that is roughly two to three years prior to flight.
It goes all the way through preparation of the flight plans, the mission rules, the launch constraint documents, the processing, and finally the on orbit ops.
All that had to be protected.
And that was big.
It was a huge and expensive philosophy.
It was called the Control Mode Security System.
It took a lot of effort from all parts and was almost airtight.
Yes, a question.
Was this for every shuttle mission or just DoD payloads?
No, this is for what I call the intelligence community DoD missions.
If you go back and look on the Shuttle Manifest, and I am not going to do it for you but you can do it if you want, you will see a number of missions where they will say DoD mission payload type, not unknown, but not available, NA.
Also, the DoD/Air Force missions were all military astronauts.
That is felt to be a safe thing to do.
For implementation of encrypted voice data and commanding links starting, really, with the testing at the launch sites but also going through the flight ops.
I mention all these things having been involved in the safety certifications.
I will say that was long and arduous.
The safety review process has to be extremely, extremely thorough starting early on.
It is not just a whitewash of what the hardware is that comes with the pad.
In many cases, it changed the design of the spacecraft, the design of its deployment systems and things like that.
To protect classified information there also had to be need-to-know access to program details.
This was a major, major set of impacts to the Space Shuttle system and payload mission.
All these areas, which I can talk to you about at great length afterwards.
I will say, thought, that when I do get back to JSC or when I talk to my NASA friends, they all say they were proud to have been on those missions.
They were all hooked together on missions of great national importance.
To the best of my knowledge, security has held tight over all those years, despite all the people who were cleared of the programs.
And the programs were extremely successful as a result.
Yes.
You said all military crews.
Does that mean active military or people who have been transitioned to the regular astronaut corp. from the military?
These are military folks who transitioned to the astronaut corp., yes.
There were also impacts of the Air Force Satellite Communication Networks.
As you know, these are the worldwide set of tracking stations that were going to be replaced by TDRS.
That is an interesting story.
The selling point for TDRS was they would shut down the AFSCNs which were getting old and were very labor intensive.
Well, we still have them both today.
Worldwide network scattered around the world, but to provide secure Shuttle communication there had to be extensive upgrades because working with the Shuttle with a complicated payload in orbit, getting it initialized, tested and deployed was something new for the SCN.
The SCN ordinarily just did space-to-ground data transfers through orbiting satellites, so to be working with a human crew and human system, to have all the contingencies in place was big.
The SCNs were involved in the all the prelaunch testing and commanding, health and status checks, as well as all the deployment and post-deployment payload communications.
I wish I could show you the contingencies, all the workarounds, the plan As, the plans Bs required for a complicated mission.
It is immense.
It takes a long time to do it right.
The point is that when you're ready to fly, you should be ready for all know contingencies, even up to loss of communication, failure of critical computer systems and that sort of thing.
You never want to fly and have to scratch your head and say what should we do next?
Pre first flight issue was an STS1.
Another MIT professor, Gene Covert who is also here and someone you should really talk to at some point in time, was part of our panel that reviewed Space Shuttle main engine quality testing.
And they were the folks that really came to grips with this new and emerging issue called turbine micro-cracks.
The turbine blades in the SSME were found to be developing very small cycle dependent micro-cracks.
The question came up what is the ultimate failure mode?
How many cycles can they stand?
How good are they good for?
The original intent, as you know, was to be able to re-fly the SSMEs with zero maintenance between flights many times.
That hasn't happened.
But even the question of would it be good even for one mission was a big, big question.
And Gene, who is an extremely thorough manager and engineer, a lot of common sense, a lot of smarts, worked that very, very hard.
There were also issues, of course, about thermal tile loss.
There were questions about whether those would fall off.
This is not ice now.
This is thermal tiles.
And, at the same time, there were huge costs growing in the development of the IUS, the Shuttle Centaur upper stage and also Vandenberg facilities.
Everything was either late and/or over cost.
Many folks thought that the IUS would never fly because the development problems were so huge.
The fact that it has flown as many times as it has, on a variety of launch platforms, is truly amazing from my sight.
And we also saw, prior to first flight, several delays, two and a half years is about right.
And then more delays ensued.
And so, from a user perspective, all these delays really added up to lost opportunities, extensive cost growth and things like that, really a heartbreaking story.
This is a picture of a Defense Satellite Program, DSP.
A machine lifting out of the shuttle bay.
I think you have seen this.
This is the IUS turntable that rotates the vehicle up and pushes it out with springs.
This is a two-stage solid misconfiguration that takes the beast out to geosynch.
Post-Challenger, the space policy was changed.
The Shuttle was not to be used anymore for lifting all satellites but only for mission where human presence was required.
But there was a problem.
Because of the backup of missions, the SCS had to fly out the existing DoD payloads.
We looked very hard at converting them back to the Titans.
Since we were taking steps to shut down the Titan line, the Titans weren't ready at that time.
But what we saw afterwards was increased use of, a restart, really, of these efforts, modernizations.
And that was interesting because there was always the chance the Shuttle could come back and launch, hopefully, 15, 20 times a year.
And we hoped it would happen but it never did.
Steven Dorfman from Hughes was here as a visiting professor several years ago.
He was extremely bitter about the fact that, from a satellite manufacturing point of view, he had to eat the cost to switch back to extendables from Shuttle.
First he was told by National Space Policy, "Thou shalt fly on Space Shuttle."
Then he was told, "You are now bared from flying that Space Shuttle."
A huge lost to him.
And I think Hughes is still contesting it.
I don't think it will ever win that in court.
And kind of most importantly, from a Central California point of view, the Vandenberg shuttle facilities were shut down completely.
My DoD SCS mission involvement personally was I worked on the very first DoD sponsored mission STS4.
It was Columbia launched in late June of '84 flown by Ken Mattingly and Hank Hartsfield.
This was a pathfinder for the DoD.
It was a test not only of the security procedures but also the ability to do launch operations, orbital operations in a classified way.
It came out truly successfully.
We learned a lot.
It was not easy.
Getting any complicated mission to fly on the Shuttle is a very, very difficult process.
To do it under the cloak of security makes it even harder.
I was a primary DoD launch integrator, the interface between the Program Office and the Air Force, for two primary missions.
I shadowed three others.
I was also member of the NASA DoD Safety Review Team.
This was interesting.
The Safety Review Team or SRT was a very small, streamlined, fast-moving team that basically reviewed and bought off on the payload.
As opposed to having the figurative casts of thousands, we had a cast of just a very small number of people both from the DoD and the NASA side.
And we moved as quickly as we could to get these missions approved and bought off.
And, lastly, in the mid '90s, I was a program manager for the DoD space test program.
STP is in existence today.
Their charter is to be the sponsor for space experiments with military relevant.
They flew, among other things, the first atomic clocks that led to GPS as we know it today.
They continue to fly numerous secondary and piggyback missions not just on Shuttle but also on Ariens, Russian vehicles, Korean vehicles, Indian vehicles.
When I was the manager of the program it was great.
It was like you were running a big used car lot and you were offering rides to everybody.
It was terrific.
The emphasis here is to use excess launch capacity to fly space experiments that you would screen every year.
And then, on rare occasions, you would build a dedicated satellite and launch vehicle about every four years to launch the bigger satellites.
My summary is that the DoD had requirements that really went back to the first years, the first days of the Shuttle Program.
They were extremely demanding and dramatically affected the architecture of the Space Shuttle system.
The heartbreak, of course, as we know, is that we never were able to achieve the launch rates and the low recurring costs that were promised.
That really when we realized it was slipping away, those were truly sad days.
I don't think anyone ever anticipated the one-time expenses and effort required to redesign payloads.
Not just for mechanical form-fit function but also for safety.
No one realized the extensive work required to certify the payloads for manned operations.
The mission integration at the Cape was very complicated, and security, as you can tell, permeated everything.
And, lastly, the sad thing is that MOL and SCS at Vandenberg all suffered from the curse of Slick-6, which was that both programs were cancelled despite millions of dollars put into refurbishing the facilities.
I have here a list of references.
Heppenheimer's book "Development of the Space Shuttle" I found was quite good.
It talks a lot about the programmatics of the Space Shuttle.
I hope you've had a chance to read it.
I took some information from here.
The Federation of America Scientists, the FAS site is interesting.
I think it is still current on the Space Shuttle.
And there are these books here which I point you to give you a little bit more insight into the missions, the roles of the intelligence community doing things such as flying and operating satellites in space.
If there are any questions, I will be glad to take them now and then I will show a short ten-minute video.
How am I doing on time?
I'm moving fast.
Yes, question.
How many times was the [UNINTELLIGIBLE PHRASE]?
Was it even used to launch the Shuttle?
Slick-6 never saw anything launched out of there until they launched a Lockheed low-cost machine called the Athena in about 1996.
Presently, the Delta 4, remember the big machine that flew out of the Cape about nine months ago, is stacked there at Slick-6 ready to go with an operational payload.
They are having big concerns, though, about fuselage so their launch date, which was going to be last week, has been rolled off to the right several months.
They had a problem on the first flight of basically fuel cavitatation.
And they are not happy with it yet.
It is ready to go.
When the Delta 4 launches out of Slick-6, it will be the first large payload system to go out of there for the first time.
Question.
Yes.
I don't know if it is really fair, but in hindsight, was it worth it for the military to get onboard with the Shuttle from the military's perspective?
Well, if the promises had been kept it certainly would have been worthwhile.
With the way things have turned out, was it worth it?
We were kind of driven into it.
Can you say was it worth it from a cost point of view, from a schedule point of view?
The answer is no.
I think from the perspective of having people do extremely difficult tasks under very pressing circumstances, it showed us what NASA and the Air Force could do, although at a great cost.
It was certainly one of the more, I will say, exciting times.
It was one of the more exciting times for NASA.
Questions?
Yes.
When you talked about safety certifying these payloads, was it mainly an issue of the fuel supplies for the payloads?
Well, I will just give you one example.
To get a payload into the bay and withstand launch loads, you have to have very large structural members, steel rods, for example, that had to take all of the launch loads and not have the satellite break away inside the payload bay in the worst case either during liftoff or in the worst case on landing.
You could have as many as 18 to 24 steel bolts that all had to be fractured simultaneously within seconds by pyrotechnics to get them out.
You had to prove by analysis and by test that you could have these simultaneous events for which you could not have any redundancy.
You just couldn't do it.
These were must-work devices.
As probably Professor Hoffman has told you, on every Shuttle launch there is a large number of must-work category-one issues and items.
Like the bolts that free the external tank from the vehicle, if those don't work you've got a big problem.
From a safety certification point of view on the Shuttle, we had to make sure that in no case could the payloads pose any hazard to the crew under any circumstances, including the ones that are the most farfetched such as landing overseas in Spain or Morocco.
But the things we were worried about was having a payload hung up during deployment so the Shuttle couldn't return or a payload damaging the Shuttle at any time and causing obviously big problems there and things like that.
Plus things such as propulsion systems accidentally activating.
Liquid solids, it didn't matter, all extremely hazardous.
And we found out a lot, because it is one thing to design a payload for launching an expendable.
There are certain things that you can or should not do.
But when you go to a Shuttle context, we have a lot of commanding going into the vehicle.
Pre-flight, during assent and ops, it becomes a much more tough a problem.
For example, I will just give one example.
If you have a very complicated payload which consists of a spacecraft bus, all the housekeeping and the payload front-end which may have all kinds of hazards associated with it like antenna booms that come out on command, how do you verify final configuration before you liftoff?
How do you know every system is in the state that you think it is?
How do you verify that?
That is very hard.
Because you can go in there with test equipment, but test equipment has subtleties all its own.
OK.
Questions?
Do you think it would be possible to do the one orbit and return to base type mission?
It sounds very difficult to do.
It is not that hard.
First orbit deploys were not easy but they were done.
Coming back, I think the big problem is that it really forces the timelines.
Because, as you know, to do a de-orbit burn and return to earth, it requires quite a bit of mission planning, real-time updates to verify you're on the right track, plus making sure that you have everything right so when you do the de-boost you're going to be within your corridors.
To the best of my knowledge, NASA has never done a first orbit return.
And I don't think they would want to.
It's very stressing because in 90 minutes you have to not only get the spacecraft out, but then the vehicle re-safed and buttoned up and planned out to land back either at the primary or an alternate landing site.
That's a lot of work to do in 90 minutes.
Any other questions?
Yes.
Is there any way to give a ballpark figure of how many DoD payloads can be put in space by the Shuttle?
Ten or hundreds?
Oh, not, it's not hundreds.
The NASA Manifest is published.
From what they call a regular Air Force site, they launched a whole series of communication satellites, DSCS, DSP.
In fact, the very last DSP satellites are still Shuttle compatible and could be launched if there was a problem with the Titans.
They are up there in storage in Southern California.
And then there are a fairly large number of what are called Intelligence Community Missions that are flown.
And all the manifest will say is that these were just DoD missions.
Let's see.
What I would like to do is ask Tom to shut off the videotape and I will give you a little introduction to my video which I hope will run.
We are ready to start her, folks.
Bob Siemens is really a very pivotal person, not only in the Space Shuttle story, but also in the Apollo Program, which he can probably talk to you at a separate time.
But I think he will like to spend ten or fifteen minutes talking about his role in the Space Shuttle story.
OK.
Here we go.
I'm just wondering where to go to get comfortable.
Can we turn that off?
We cannot.
All right.
I would just like to get comfortable here.
You know, as Pete said -- First of all, I thought that was a great summary of actually a very difficult subject to discuss.
Let me just take you back, first of all, to the early '60s.
I know that is going awful far back, but that was when I joined NASA as a general manager.
And, before we knew it, we had gone from a billion dollar a year operation with the Mercury Program to a situation where we were given the assignment of putting men on the moon.
And that was a gigantic shift in our responsibility.
And in all of the planning and the discussions and so on, and we did work very closely with the Department of Defense, the Department of Defense had all kinds of assets that were going to be required in the Space Program.
The Navy, for example, to pick up the astronauts.
One example, a major example most people never heard of, was we had to construct very large facilities.
The most famous and obvious is the Vertical Assembly Building and the large facility down at the Cape.
You just saw the Shuttle taking off from that facility.
And, in operating down there at the Cape, there already was Cape Canaveral.
And there was not room for what we were going to do to fit on Cape Canaveral.
We looked at seven different world sites and finally decided to camp on Merritt Island which is just across the river from Cape Canaveral because, again, the Department of Defense had all kinds of facilities down there with a tracking range and so on.
But the biggest support that we got was in the building of those facilities.
There was absolutely no confidence in NASA to build the largest structure in the world, the VAB, or many other facilities for assembly and for tests and so on.
The Corp of Engineers was a major part of the operation.
I am bringing this out because we had to figure out how far we were going to go with our launch vehicles on a shared basis.
This is before anybody thought of a manned shuttle that would carry stuff up and stick it into orbit.
But we put together a planning organization that actually operated for over a year's time.
A fellow named Golivan for NASA, a fellow named Cavanaugh for the Department of Defense put this together.
One of the big emphases was the use of the Titan.
The Titan was coming along to the point where we thought that the earliest version would be suitable for the Mercury Program.
And, at that time, in 1961, we were thinking that Mercury only weighed 3,000 pounds.
You could put one man in it.
You couldn't do much with it.
But if we could, in effect, just enlarge it and have a more powerful vehicle, an Atlas, namely the Titan, we could really have a vehicle that would have some capability to run through a lot of the orbital operation and so on that we're going to ultimately be required for going to the moon.
And out of that came the planning for the Titan III, the Titan IV and vehicles that proved right off the bat, and over time, they still are very important to our defense capability.
Well, I don't think that is quite true.
I don't think we are using any of those assets today within NASA.
And if you then go forward in time, in 1968, I have been down there in NASA for 7.5 years.
I planned to go down for two.
I came back.
MIT was nice enough to invite me back, and I came back right here as a Hunsaker professor.
To my very great amazement, I got a call one day from somebody I had never met, a man named Mel Laird.
I just barely knew that he had been designated by Nixon to be the next Secretary of Defense.
He said are you going to be down here in Washington the next day or two?
As a matter of fact, I was because I was going to go down the next day to get on a plan and fly down and see the launch of Apollo 8 which was going to fly around the moon.
And he said come on for lunch.
That is when he asked if I would be the willing to be the Secretary of the Air Force.
I told him that was absolutely impossible.
We just moved our family back here.
My wife is in the hospital, which she was.
But he was very persistent.
And after about ten days I agreed to go back down in the government, but that is a long story you don't want to hear the details of getting my wife well and getting a house and all that stuff.
One thing that I inherited right off the bat was the Manned Orbital Laboratory.
And, even then, it was clearly in some jeopardy.
When programs start getting a ceiling built in where they say we're going to keep it going but it is going to be kept going at a level of, and I forget what the level was for MOL, something like $500 million or something like that.
Because any large program over time there is always opposition to it and there is more time for it to be shot down and ultimately eliminated.
I realize that it was in deep trouble when I was over in the Bureau of the Budget talking to a junior member of the Bureau of the Budget and he said I hope you realize that the Shuttle is in deep trouble from a standpoint of support here in the White House.
And so I went back to Mel Laird and said, look, one thing I want to do is to have one shot with the President to be sure he fully understands what the capability of the MOL will be.
It was far along.
Lots of expense had gone into it, including major facility construction out of Vandenberg.
I had my day in court.
It was a sunny afternoon.
I got General Stewart, who was very involved in the MOL, to join me and Mel Laird.
And we went over there.
It was just Kissinger and the President.
And I had a few simple-minded charts that showed what we were going to do from a resolution standpoint if we kept going with the MOL and what that would mean in terms of understanding more clearly what was going on in a given situation.
I had my half-hour in court.
I could see a band outside getting ready to play so I knew that the President was about to be rushed out for some kind of a ceremony.
That was a Saturday.
Monday morning I got a call from Kissinger.
And he said Bob, and I cannot really imitate his German accent, that was a very, very fine presentation.
And a day later I found out that MOL was cancelled.
That was with President Nixon.
He sat there the whole time.
He had a yellow foolscap and took prodigious notes of everything I was saying, which was sort of nerve-racking, the President of the United States bothering with what I am saying.
Anyway, that was sort of where I came from, from the MOL.
The next step along the way was after Apollo, what was going to happen in space?
There were to be eight launches of the Apollo Lunar Program.
Nixon cut that back to two sort of arbitrarily.
And there were no plans for using those assets.
Jim Webb was my boss in NASA.
And I used to see him.
He was very, very ill the latter part of his life.
And I would drop by.
And he would ask me strange questions like what do you plan to do with your life before you kick the bucket kind of questions like that.
And he said I haven't got long to live.
And I would say you're doing fine, Jim.
And I am just going to work along and see what I can do to help out here and there.
Anyway, his big thrust was we felt we were building a major capability for the country and now it is all being washed away.
And what is going to replace it?
The Shuttle came up as an option.
Actually, to say on the surface, it makes an awful lot of sense to recover something.
We could easily visualize a transportation system for the country where every time a 747 went across the country with a payload everybody jumped out and then you threw it in the ocean.
It didn't seem to make much sense.
You just knew it had to be more efficient to reuse something.
Although, with Gemini, we had looked into that possibility.
We were recovering the Gemini's, why didn't we use them again?
And we found that we are going to have to put probably 75% of the original cost into reactivating the Gemini's.
So, actually, I had a hunch that it wasn't going to be quite as simple as landing an airplane and then taking off again.
The Air Force was going to be one of the prime users of the Shuttle.
And the question was how large did the bomb bay -- Not the bomb bay.
[LAUGHTER] How large did the experimental bay have to be for the missions that were going to be carried out?
There was no thought, let me just quickly say, of putting armament aboard the Shuttle.
I misspoke.
And then one of the questions was how rapidly did you have to recover it if something happened and you wanted to bring something back in a hurry?
And to bring it back you had to make up a co-planer change, and that was going to take a certain amount of energy, to put it mildly.
Anyway, those are many of the issues.
And I think, as I remember it, my friends in NASA thought the Air Force was being pretty tough on them.
But if we were going to accept the use of a vehicle, in all seriousness, as you've just heard, it had to have ability to carry out the missions.
And so that was sort of the next step along the way.
And then, after I got through in the government, I ended up some years later out at Aerospace and ended up as the chairman of the corporation.
And they worked closely with that element in the Air Force that was so much involved in these type programs that you've just been hearing about.
And we were beginning to go into shock, at least those in the nucleus who were sort of running the aerospace.
[UNINTELLIGIBLE] and myself.
More and more it appeared that we were going to not be allowed to put anything in space except through the Shuttle.
And that seemed to be imminently wrong.
To think that you're going to have to risk the life of astronauts every time you wanted to put any kind of a satellite in orbit.
From the standpoint of a paperclip individual who is looking at cost, to just have the one vehicle and use it and use it and use it and thereby supposedly cutting the cost, had a lot of charm.
And it reached the point where he and I decided we had to try to do something about it.
And we made an appointment with the then Secretary of the Air Force and said you've just going to be stashing away Titan somewhere so that if we run into trouble with the Shuttle we are going to be able to move over and put these very important payloads into orbit.
And then they had the Challenger accident.
At that point this, I think, major fallacy in policy was changed back where it should have been.
You just shouldn't rely on a single vehicle.
And that is my introduction.
Yes.
You wanted reusability, but some people also didn't want to risk the lives of astronauts.
Why not make a reusable unmanned vehicle?
That's a very good question.
Why couldn't we do it?
That might be a good design challenge for you guys here in this class to investigate that possibility.
Unmanned.
We've gone partway in that direction by, say, recovering the Shuttle casings, obviously the Shuttle itself.
But, trying to recover at least elements of the unmanned launch vehicles, I'm not prepared to really give you a definitive answer on that.
That's a good question and we will talk about it later.
Yes.
Before the Shuttle concept was finalized and military requirements were finalized for the Shuttle, the military had been launching their satellites and their missions using Titan's and Atlas's, correct?
So why this requirement of the Shuttle having to be able to bring back a satellite using really a military satellite when that wasn't being done or required beforehand?
Like why was that created?
What was so special about it?
We worked very hard on proposals to bring back satellites that were out of fuel or needed refurbishment, but when I was referring to landing with the Shuttle with payloads in the bay that is for a failed mission.
And for some reason the payload could not get ejected.
You still had to be able to land back what was now a very heavy spacecraft and glide on it and land safely.
That turns out to be a really hard thing to do, especially if you're carrying solids and liquids onboard that were close to being ready to ignite, so to speak.
But there is talk of retrieving satellites.
And, as you know, NASA did bring back some small satellites, the Palapa one from Hughes.
And also did a lot of very innovative repairs in space.
But when we looked at it from a customer point of view, it turns out there is a lot that wears out in satellites, not just using a propellant.
But the processors degrade due to radiation, solar panels degrade just due to micro dust and things like that.
And so bringing back a satellite for reuse was never felt, at the time we looked at it, to be a worthwhile thing to do.
And that ties in directly with what I was saying about recovering a Gemini.
And, of course, on top of that with a Gemini is the fact you are landing them in the water so they got a good dousing of salt.
And so the degradation of the Gemini's were such that we just didn't think it made any sense to try to refurbish them.
Yes.
The military was only really looking to capture and service in space, then release, not necessarily capture and bring back?
They never seriously went after the capture and refurbish and release.
They really went after launch and deployment.
But I would say that in that theme the most successful story actually of refurbishment is the Hubble Space Telescope.
And not for the reasons that you think of.
When you think about stories of the Hubble Space Telescope you hear about Jeff Hoffman going up there and changing processors and fixing the optics.
Also on those missions they replaced solar panels that were causing huge problems due to thermal warpage and shrinkage and what they call oil canning due to thermal stresses.
And so the replacement of those solar panels, which turned out to be not very good as initially designed, is one of the really true success stories of man in space.
And there is actually a very nice video which maybe you have seen of the lady astronaut who did the replacement of the solar panels on Hubble pushing out and releasing the solar panels.
Things fly away in the sunshine looking like giant butterfly wings reflecting all the colors and then finally fall into the atmosphere.
It is almost poetic, I tell you.
It's amazing.
On the Hubble, one of the considerations right from the beginning was that it should be a serviceable satellite.
It was designed so that you could get inside of it relatively easily and so on.
Frankly, I don't know of any other satellite that was really designed quite that way.
Maybe I'm wrong.
I believe you're right.
Yes.
On the solar panel, it wasn't designed to replace the panel.
But you can go back to SkyLab, which was the next to last launching of the Apollo.
On that one, I guess, we decided to take what was the third stage of a Saturn and just gut it, not have any propulsion in it and fit it out to be a spacecraft, namely a space station.
And, when they got up there, the solar panels got fouled up.
And one of the real tricks when the astronauts first got to it because it was not launched manned was to unravel some wires that got caught on it so actually the panels could spread out and it could start to operate again.
Colonial Siemens, I don't know if either of you would know, but what are the duties involved in like the heavy payload aspect of the new CEV project?
Delta 4 going out of Vandenberg was carrying classified payload.
It's not a dummy either.
It is real.
They have standing requirements for these types of things.
The national needs go on.
The international stage, as we know, is filled with new actors and new threats.
We have what is called unsymmetrical warfare, things that are hard to counter by conventional means.
Back in the '70s and '80s we had symmetrical warfare, submarines versus submarines, missiles versus missiles.
Now, as we know, it is much harder, much tougher, but the needs are still there.
But is the collaboration between NASA and the military still pretty active?
It is not in terms of payloads.
It is there in exchange of technologies, things like development of more efficient sensors, processors, that sort of thing.
It is everybody's benefit, for example, to have more powerful, faster, rad-hard space processors.
Everybody wins in that type of situation.
Back to your question on reusability.
You will notice that one of the real strong pluses or the calling cards of the Space Shuttle Program was the fact that it was a reusable launch vehicle.
In this current environment of the CEVs and new launch vehicles, what do you hear today about reusable launch vehicles?
Nothing.
Isn't it amazing how things have changed just in a matter of 25 years?
One thing I just sort of forgot to mention with regard to the Manned Orbital Laboratory.
It was thought, at that time, that by having men there and their ability to inspect a fairly wide swath that they would have time in flying over that swath to do some searching.
And could do a better job of detecting possible items of tremendous interest from a military standpoint than trying to do it all automatically.
And, of course, what finally washed out that argument was the ability to have satellites that had almost instantaneous transmission of information back home.
What was the design for the MOL for crew transportation?
They were going to take off on a Titan, they were going to aboard a Gemini, and they were going to have a laboratory of sorts which would have the necessary reconnaissance and other equipment aboard.
So it was similar to SkyLab in that you had a non-reusable capsule as the crew transfer?
It had a Gemini type return vehicle, but it was definitely going to be very small and cramped compared to Shuttle.
There was no thought of having another Gemini come up and make use of it.
Yes.
Why did Gemini, Apollo and Mercury all land on water, because that is very expensive, and the Russians landed on land?
Well, very simple.
If you take a look at where the launch sites were for the Soviets, if they aborted they had to have the capability of coming down on land, any kind of abort mission.
Similarly, operating out of the Cape Canaveral area, if you abort, you have to come down on the water.
The question was then was it worthwhile to have the capability of doing both?
Well, that just added weight.
And so we stuck with the water and they stuck with the land.
Do any of the new defense payloads require human intervention in their deployment or are they all completely [OVERLAPPING VOICES]?
They are all Titan IV EELV compatible.
Anything else?
I think we wore everybody out.
Well, you know, Bob and I will stick around, but we're not going to hold you down here if there are no questions.
I guess we will call the class.
And we will stick around if anybody wants to ask any more detailed questions or have follow-ups.
My phone number and email is up there.
I will be glad to talk to any of you one-on-one about any of this.
There is quite a lot more detail we can provide, either Bob or myself.
And, if I have to, I will write Bob a note asking for further information.
OK?
Well, thanks a lot.
[APPLAUSE]
Let's see.
The original schedule today had Colonel Gordon Fullerton going to come and talk to us about both cockpit design, but primarily about test flying the Shuttle.
Unfortunately, he is in charge of Flight Operations.
Well, not unfortunately.
He is in charge of Flight Operations at Dryden Flight Research Center for NASA.
Unfortunately for us, since the time that we had agreed on this date, they scheduled, at the beginning of next week, a major base inspection.
And if any of you are connected with the military you know what that involves.
And, since he is head of flight operations, he really has to be there to make sure that they are ready for it.
He has rescheduled.
He will be here in November.
And I will make adjustments to the syllabus.
On Thursday, Bob Ried is going to talk about aerothermodynamics.
And what I thought I would do today is kind of a potpourri maybe doing a little bit of recap on where we've gotten to so far.
And also I brought a few videos just to give you a visual reminder of some of the things that we've been talking about with the Shuttle.
Any sort of general questions about what I handed back to you?
I am happy to meet with teams individually to discuss any specific questions.
OK. Let's see.
One thing that I -- Hi.
I was just telling the class Gordon Fullerton was supposed to be here, but he has had to postpone to November because they're having a base inspection of flight operations.
So, he is not here.
I had an email from Professor Cohen who will be here.
And let's see if this is -- He had an idea that now that we've kind of started going into details on various subsystems that it might be interesting to work our way through a Shuttle flight and just remind ourselves of the different subsystems that are brought into play at various times during a Shuttle mission.
This is kind of part of the systems integration process.
And remember that we talked at the beginning how in the systems engineering process we always want to keep in mind the eventual operational use of the systems that are being designed.
So, the idea, for instance, would be start at the very beginning.
You're sitting on the pad.
You could even go further than that because remember we talked about how systems had to be developed, for instance, for loading the payload into the cargo bay.
They have that big payload changeout room.
They had to design the hypergolic reaction control and OMS motors in such a way that the pods were actually removable from the Shuttle because they were using hazardous hypergolic fuel and they cannot be serviced in the main hanger.
They have to be taken over to a remote section of the Kennedy Space Center.
And then, of course, the servicing of the tiles.
There is a lot of servicing and maintenance work that has to be done on all the subsystems.
And, ideally, it should be designed in from the beginning.
Of course, in many cases, we got a lot smarter after we started to use the Shuttle.
And, if we were doing it again, which is something that you will be thinking about, we would, in fact, have built certain things into the subsystems which would have made it easier for them to be maintained and repaired.
What I am going to do, just as an experiment here, let's think briefly about these subsystems.
And I've got a little video taking us through the launch and the entry stages.
And then I have one or two other things to show you, which will just illustrate certain other aspects of the Shuttle Mission.
Right at the moment of liftoff, just kind of to set the stage, the final countdown starts at T minus nine minutes.
They typically will have holds built into the count so that particularly when you're going for a very tight launch window, like if you're on a rendezvous mission to the Space Station, you have to launch when the Space Station basically is pretty much going overhead.
And, since the Space Station is in a 51-degree orbit, it takes more energy to get into a 51-degree orbit.
Because normally if you launch from Cape Canaveral, hopefully you understand the orbital dynamics, if you launch due east you get the full speed of the earth's rotation.
And since the latitude of Cape Canaveral is 28 degrees that puts you into an orbit which is inclined 28 degrees to the equator.
That is what we did when we did a rendezvous with the Hubble Telescope which is in 28 degrees.
In fact, I remember my brother was telling me that we launched about 4:00 AM so it was night.
And just about 20 minutes before we launched they saw Hubble fly over and everybody thought that is a good sign, they seem to know what they are doing.
[LAUGHTER] The thing is that when you have an object that is in orbit, and the orbit you can think of as being roughly fixed in inertial space, obviously it precesses a few degrees every day.
But if you think of it as fixed in inertial space on the time scale of a day or so then the earth is turning underneath it.
And you have to launch basically when you are in that orbital plane.
Now, if you have a little bit of extra energy to burn you can launch a little bit earlier or a little bit later and use that extra delta V to change your velocity to get into that plane.
If you're launching to rendezvous with Hubble, say, which is in a 28-degree orbit, you have a bigger launch window than you do if you launch into the Space Station at a 51-degree orbit.
Because, going into a 51-degree orbit, you are using a lot of extra delta V just to change from a 28 to a 51-degree inclination.
So the launch window to rendezvous with the Space Station is very short.
It is only about five minutes.
So in order to make sure that they have the maximum chance of making the launch window, they actually build in some extra large holes, even more so than for a normal launch.
But you pick up the final count at nine minutes.
The Shuttle, at that point, is on external power.
At about seven minutes, the pilot, that is the person sitting on the right-hand side who should be called the copilot, and I've talked about this with a few of you.
None of the astronaut test pilots, you know, these are kind of big ego people.
Nobody wants to be called a copilot.
The person sitting in the right seat is the pilot.
The person sitting in the left seat, who is really the pilot, is called the commander.
The pilot does not fly the Shuttle, except for usually if you have a nice commander he will let the pilot -- when you're pulling away from the Space Station it is kind of a tradition that you give the pilot the stick and let the pilot do a fly around.
And, in the old days, some of the commanders actually gave the pilots a little bit of stick time before landing.
But that generally doesn't happen now.
Anyway, the pilot is in charge of a lot of the subsystems, the main engines, the electrical power system, the auxiliary power units.
It is the pilot who has to throw a switch to put the Shuttle on internal power.
At about eight minutes before a launch you go onto fuel cell power.
Then at about 5.5 minutes before a launch you turn on the auxiliary power unit.
Remember we talked about that subsystem.
That generates the hydraulic pressure to move the flight control surfaces and the main engine bells.
And you will see, if you watch a launch sequence, starting at about three minutes after the auxiliary power unit comes on -- at that point you can sort of feel a little bit of humming in the vehicle when you're sitting in it -- they then move the engine bells back and forth through their entire range just to make sure that the hydraulics are working.
And actually the whole Shuttle shakes when that happens so you can sort of feel that when you're sitting there.
And then, of course, one of the challenges with a cryogenic engine -- and this going to be an interesting development project for the future when they want to have a restartable engine.
Remember these are cryogenic engines and so you have to do a cool down.
And so you start to flow the liquid oxygen and liquid hydrogen through the piping.
Again, inside a minute you can feel kind of a rumbling down below as valves are opening and things are starting to flow.
At 31 seconds the launch control center gives control over to the Shuttle computers.
And the last 31 seconds are controlled entirely by the Shuttle computers.
They look at thousands of data points to make sure that everything is working properly.
At T minus six seconds the engines go through an ignite sequence.
The engine starts are staggered.
And you will see that when you look at the slow motion of the video that I am going to show you.
That reduces the shock on the system.
It only takes a couple of seconds for the engines to come up to full power.
You certainly don't want to commit yourself to lighting the solid boosters unless you are 100% sure that your main engines are firing.
Because if even one main engine is not operating at 100% efficiency you cannot make it to orbit.
In fact, if you have one engine out on the pad, you have to do a return to launch site abort.
We will talk about the details of that whole process at some point.
AUDIENCE QUESTION You can abort launch right up to when the solid -- Once the boosters are lit you are going.
Just hope you're pointed in the right direction because there is no thrust termination on the solids.
And it has happened four times that we have had pad shutdowns.
The engines are monitored.
On two of those occasions it turned out that the problems were instrumentation.
On two occasions it turned out that there were, in fact, real engine problems.
And I would say, for a 50% success rate, you're doing well.
I mean it was the right thing to shut down the engines all those times.
Although, the very first time when they had an engine shutdown on the pad, well first of all, just an acronym is MECO, main engine cutoff, which normally happens about 8.5 minutes after launch.
The comment by Steve Hawley, who was the flight engineer in the center seat when they had -- this was on the fourteenth Shuttle flight -- when they had a pad abort, his comment was, "Somehow I thought we would be a little bit higher at MECO."
And it turned out, they didn't realize it at the time, but they actually had a hydrogen fire on the pad.
Remember we talked about how there was a lot of hydrogen.
And, of course, before the engine ignition, we have those sparklers to get rid of the excess hydrogen.
But, in the process of shutting down the engines, there was also a lot of excess hydrogen which got into the engine [UNINTELLIGIBLE].
And the problem is that a hydrogen fire, as you know, is pretty much invisible.
And they didn't have any infrared cameras at the time.
And so basically everybody went around, the crew took their time and everybody went out.
Once they realized it, they put on a water douse.
Now we have infrared cameras.
And, whenever you have an engine shutdown, you have to be very careful that you don't, in fact, have a hydrogen fire.
Coming back to the subsystem, the last stage of the launch, the main engines have gone on.
You have turned on the hydraulic system both for the main engine and the boosters have their own hydraulic systems and the boosters have their own auxiliary power unit.
Guidance navigation and control, of course, is absolutely critical.
Now, first stage is what they call basically open loop guidance.
There is a specific projectory that you are supposed to follow.
Once the solid boosters are off you are following a closed loop guidance which is taking you to a specific point and velocity in time and space and the trajectory is actively adjusted to get there.
Because the solid boosters have so much thrust you don't want to do a lot of sporty maneuvering.
In fact, the biggest challenge from a maneuvering point of view is when you're going through max Q and maximum dynamic pressure and you have to relieve wind shear loads on the aero surfaces.
Basically, when the boosters are firing, from the crew's point of view, it is hands-off.
There is now way that you could take manual control over the boosters.
There is just too much power.
And just a little bit of deviation from your course and you will break up aerodynamically.
It is hands-off.
The fuel cells are generating all of the power.
The guidance, navigation and control is being done inside the Shuttle and the results are sent to the solid boosters.
The gimbals can move.
So the roll maneuver, for instance, is done by the solid engines.
How do the solids gimbal?
The solids can gimbal, yeah.
How do they gimbal?
The same way as the main engines.
I mean they are on a mounting with hydraulic actuators.
And so the actuators expand and contract.
And you can imagine the forces that are involved there because these are 2.5 million pound thrust engines.
It is pretty amazing.
Going through maximum dynamic pressure, SRB separation, main engine cutoff, basically the same systems are working.
The critical things are, of course, the main engine has to be throttleable.
This was a big challenge, as we heard, in making the main engine.
It is throttleable for two reasons.
Going through a maximum dynamic pressure.
And at that point, the solids are also, you know, the way the solid propellant is loaded, the actual cross-sectional profile is such that you don't have a constant thrust profile.
The thrust profile is very high right at liftoff where you need a lot of thrust.
It falls off as you're going through maximum dynamic pressure.
And then it builds up as you burn through the last two minutes of SRB firing.
The Shuttle is pulling about 2.5 Gs at SRB cutoff.
The aerodynamic separation of the solids got a lot of attention.
To have a re-contact would be a disaster, so you have four separation motors, two at the top and two at the bottom which basically push the solid boosters away from the Shuttle.
And, of course, those motors are pointed right at the Shuttle in order to move the boosters away.
It is pretty spectacular when you're inside, especially at night.
They are little solid rocket motors and they are firing right at you so visually it's spectacular.
But it also, unfortunately, leaves kind of a nasty coating on the front windshield most of which luckily gets burned off during reentry.
But the pilots were a little bit concerned that that might hurt their visibility in landing, particularly if you had a bad sun angle.
They always try to consider the sun angle for landing, as well as winds.
After SRBs come off, now you're back to about one G. You have second-stage guidance, which is closed loop.
You're sensing acceleration and orientation.
Nowadays they often fire the OMS engines for part of liftoff as well just to get a little extra oomph when they're going up to the Space Station, so that's another system which is brought into play during ascent.
Then you get into, after MECO, 8.5, 8.75 minutes.
Now you have to get rid of the external tank and you've got a lot to do basically to power down the main engines.
Remember, you have been burning liquid oxygen, liquid hydrogen.
You've got a tremendous amount of ice which forms in the engine.
I have some pictures to show you of what that looks like when you get into orbit.
Also, you've got thousands of pounds of hydrogen and oxygen left inside the pipes.
And if you just close your valves, eventually that will all vaporize, build up pressure and actually could cause an explosion.
One of the first things that you have to do, when you get into orbit, is to vent the main propulsion system.
Let's see.
Actually, Professor Cohen did not put on RCS here, which is absolutely critical, because you have explosive bolts to separate you from the external tank.
But basically the external tank has no propulsion on its own so you, on the Shuttle, have to fly away from it.
So you use your plus Z.
That is the downward thrusters.
Remember we talked about how the Shuttle has two RCS systems.
It has the primary systems which have about 850 pounds of thrust for each thruster.
And then once you're in orbit and you don't have to do a lot of propulsive maneuvering, then we shut down the primaries and we use the verniers which only have 25 pounds.
That takes a lot less fuel.
But those primary thrusters we use not only the ones in the aft but in the forward so that you have balanced thrust.
If you're trying to move away from the external tank you want to fire the thrusters both in the nose and in the tail so that you move straight up.
Well, we're sitting up in the nose.
And when those thrusters go off, I mean I've never been in a war or a battle, but it strikes me that the sound is what you would get if mortar shells were exploding around you.
I mean it is just a boom.
And right after riding through launch, the first thing that happens is you go weightless.
But then the next thing is you're hearing these explosions all around you.
And it can be quite exciting.
Actually, I will just share a story with you.
On my first flight, they decided to do a special kind of a test.
There are fill and drain valves for the main engine on the side of the Shuttle.
Usually we vent the propulsion out through the engines themselves, but they wanted to look for a quicker way to do it.
Let's see.
This was the sixteenth Shuttle flight, which was my first flight, and they gave us what they call a detailed test objective where right after external tank separation we opened the side fill and drain valve.
And I remember the commander asked some of the guidance and control people is the Shuttle RCS going to be able to compensate for the extra thrust that you get?
They said, yeah, no problem, we've done the calculations and it looks fine.
And this is the reason why you do tests because, in fact, what happened was we came off the tank, opened the fill and drain valve and the Shuttle rolled over 90 degrees and the RCS jets started going like crazy to try to correct us.
They were in a force fight.
After about 15 seconds the pilot said to the commander, do you think I should shut the valve, because the noise was incredible.
Like I say, it was like you were in the middle of a battlefield.
And at that point the commander said, well, it's too late now because we had gotten through most of the disturbance.
And we did recover and got to normal attitude, but it was pretty exciting.
The point is that at that point you are in a transitional state.
Your computers are still in the launch configuration.
You have your primary RCS.
And what you have to do is transform the Shuttle now into an on-orbit configuration.
The Shuttle computers are so ancient.
Has anyone here ever, never mind used an overlay technique, but, even heard of computer overlays?
No, I didn't think so.
Back in the old days when computers had rather small memory and you had to run large programs, what you would do is you would have your entire program, well, originally it was on punch cards and then eventually on magnetic tape.
But you would segment your program and then you would load the first part of your program as much as you could fit into memory.
Run that, save the results, load the second part, transfer the results, run on that and so on.
Well, believe it or not, that's the way the Shuttle computers still operate.
It is an overlay system.
They cannot hold all the software for the entire flight so we have a launch overlay, an ascent, an on-orbit and an entry.
Now, the backup flight system is stripped down.
And so the backup computers hold the software for the entire mission, but there are a lot of things that they don't do that the main computers do.
What you actually have to do is perform what they call a major mode transition, which basically means you punch a few buttons, tell the computers to go into orbit mode and then all the screens go blank.
I mean it is a very kind of a strange feeling.
And you just have to sort of hold your hands and not touch anything for about a minute or so while the tape recorders are spinning down in the guts of the Shuttle and they're loading the stuff onto the Shuttle computers.
Now, I think they upgraded the tape recorders to solid state, I believe.
That's correct, isn't it?
Do you know that?
I don't think they use tape recorders in the Shuttle anymore, but they did, in any case, at the beginning.
I mean just maintaining this computer system has been a big challenge.
And then you get into the orbit configuration.
In the ascent configuration your vernier RCS isn't even enabled.
And inside the ascent configuration, of course, you have not just your nominal ascent but you have all of the launch abort modes or return to launch site, transatlantic abort and abort to orbit or abort once around.
All that gets flushed, and now you are on-orbit.
Your on-orbit software does not give you the ability to do entry and landing.
You are going to have to re-do another overlay or what we call major mode transition before you do that.
Yeah?
[AUDIENCE QUESTION] There are several different landings spots, right?
Right.
[AUDIENCE QUESTION] There is always one picked as a prime, but you have the ability to re-designate depending on where in the ascent you run into a problem.
[AUDIENCE QUESTION] All options are available, but there always is a prime transatlantic abort site which is chosen.
And usually what happens is, I mean there are several transatlantic launch sites, sometimes one or more of them is down because of weather.
And, in fact, if all of them are down, even if the weather is perfect in Florida, you scrub because you need to have an abort site.
OK.
I am going to show you a video now.
OK.
I will try to walk you through some of these.
Remember that is the water deluge to stop the shockwave coming back.
Now you have the stage combustion.
These are the OMS engines, the RCS and the main engines.
Remember we talked about the off-angle thrust.
Those are the bolts which hold the solid boosters down.
All eight of them have to fire at exactly the moment of SRB ignition.
If you watch closely, when you see it go up, you can actually see it sort of moving that way, again, because of the off-angle thrust of the main engine.
This is the roll maneuver which puts you -- You go over on your back, headed towards the proper orbit.
And then, as you go through max dynamic pressure and break mach one, you often get those condensation trails.
This is a pilot's eye view now of a Shuttle launch.
This is speeded up by a factor of two.
That was the roll maneuver and the Florida coastline.
And I always thought one of the amazing things was actually how fast the blue sky turns black as you are going up.
SRB separation, you're up at an altitude of about 20 miles.
And you can see up there that very quickly the blue sky of earth is turning into the black of space.
This launch was a high inclination launch so they were actually going into a 58 degree orbit, so they flew literally right up the Eastern seaboard.
AUDIENCE QUESTION It was a SpaceLab mission, I believe, and one of the things they were going to do was observe the earth so they wanted to get into as high an inclination as possible.
Remember when Bass Redd was talking about the aerodynamics of the Shuttle he said that one of the things that they never were able to get quite right was the plume.
And he talked a lot about the angle of the skirt and how that affects the aerodynamics of the plume.
And you can watch how the plume is continually changing as you are getting higher in the atmosphere and you have less and less pressure.
You also notice some of the flames are getting sucked back.
In between the Shuttle and the external tank, of course that is where we had the problems with the foam shutoff, you've got supersonic shockwaves bouncing around.
It is a very, very nasty environment in there.
And you see, when the external tank comes off, often there is a lot of area here which is charred and burned out just because of the shocks.
You can see now that the plumes are tremendously expanded.
The other thing that you can -- Well, I will wait until SRB separation.
As I say, you are up to about 2.5 Gs at this point, and now you can see that the solids are starting to tail off.
You can see a qualitative difference in the flame pattern.
And now you can definitely feel this inside.
You see the separation motors.
The boosters fall off.
And, as you know, they were recovered.
Making sure that the thrust tail-off matches, as we heard, was critical.
Now here is a picture from a belly camera of the boosters falling off.
Now the main engines are firing.
When this happens -- I don't know if you have seen pictures of Saturn launches.
They actually have some of the most beautiful patterns of the reentering plumes.
But the plumes, as you get up into a vacuum, the exhaust plume expands.
And it actually folds back around the vehicle which is not a danger to the vehicle, but visually can be quite spectacular.
On the first night launch of the Shuttle -- During the day the plume is so faint that you really cannot see it from outside the windows.
From the ground you can see the bright points when you're looking right at the tail of the Shuttle but you don't actually see the plumes.
But the second flight engineer who has a good view out the overhead window, about five minutes into launch he asked the pilot, who is in charge of looking after the main propulsion system, is everything OK with the engines?
Yeah, everything is OK.
In about six minutes, everything OK with the engines?
Yeah.
And after they were on-orbit the pilot asked him why did you keep asking me about the engines?
Everything was fine.
Well, apparently the guy had been looking out the overhead window and had seen flames starting to come out which is essentially the expanding plume.
Of course it was the first night launch so nobody had ever seen that on the Shuttle before.
And so you're always looking for something off nominal.
But, as it turned out, it is just normal plume behavior.
Now we've got a few minutes riding on the main engine to build up to orbital velocity.
Essentially, we are flying up the Eastern seaboard.
And it will give you some sense of the acceleration.
Hopefully, you will recognize some of the geography.
When they talk about taking the Shuttle from Washington to Boston or back, this is the real thing.
It takes about 8 minutes, 8.5 minutes, to get from Cape Canaveral to Cape Cod.
There is Long Island coming up.
I heard Plattsburgh, New York was an alternate landing site?
Yeah.
We are really moving at this point.
You're going about five miles a second orbital velocity.
There is Cape Cod.
And now you will see a little bit of a pitch as we separate from the tank.
And here, again, is the belly camera as the tank comes off.
And you can see these little charred patterns in here.
That is where the big piece of foam came off which destroyed the Columbia.
We've talked about, for its orbital operation, it was all the different things that the Shuttle was supposed to do that really drove the design.
Speaking of which, how did things go last Thursday?
Colonel Young gave the lecture on the military.
I haven't had a chance to talk with him yet but hopefully maybe he had some new insights.
I know we talked a lot already about military requirements.
But, in addition to launching large satellites, of course we have repaired satellites, we have done science experiments, we have done a lot of educational activities.
And I am not going to deal with those subsystems so much now.
Remember that the computers now really don't deal with the main propulsion system anymore.
The only thing that you have to be concerned about generally in orbit you still have to maintain thermal control over all of the parts of the Shuttle, so the thermal subsystem is very critical.
We talked about the problems with hydrazine.
If hydrazine freezes and then it expands when it thaws it can break the lines.
Hydrazine leaks are very nasty.
We had one instance on a Shuttle flight where one of the thrusters did develop a leak, and you could just see the exhaust coming out of it until they shut off the isolation valve.
There are several sets of RCS thrusters, each of which is connected to the tanks by a set of isolation valves.
You have essentially triple redundancy in your reaction control system.
And if any one of the systems develops a leak you can isolate individual thrusters to tell them not to fire if it's a leak which only occurs when you fire it, or you can shut down an isolation valve for an entire set of jets.
I won't go into great details but we talked about the importance of redundancy.
And that is one of the ways that it is built in.
Let's see.
What else do we want to say about subsystems during orbit?
What are some of the other subsystems which get called into play for orbital operations?
Environmental subsystems.
Environmental.
Is that controlled by the computer or is that kind of done separate and automated?
A little bit of both.
For instance, pressure valves are mechanical so they will open at a specific pressure mechanically.
If you have an overpressure or you set your, we have different settings.
The normal setting is for sea 0:43:20.526 level atmosphere.
And, of course, you have to balance the oxygen and the nitrogen.
And so there is a fairly complex algorithm.
Well, it is fairly simple, I suppose.
But there is an algorithm which senses both the overall pressure and the oxygen partial pressure so that if your pressure goes down a little bit, the system has to know whether to open up the oxygen valve or the nitrogen valve.
Also, if we are going to be doing extended space walks, and I will be giving a talk on the airlock and the spacesuit and the other systems that get involved in space walks, so I won't go into that in detail here, but we actually have to bring the cabin down to two-thirds of an atmosphere, 10.2 pounds per square inch.
We keep the partial pressure of oxygen at sea level so that raises the oxygen concentration to about 30%.
And the reason we do that is to purge the nitrogen out of our bodies so that you don't get the bends when you go down to the 4 psi, at pure oxygen, which is what we run our suits at.
At that point, we actually have to trick the system because they didn't build in a 10.2 controller.
They have a 14.7 psi controller.
But, in any case, that we control manually.
And we basically bring the cabin down to 10.2 psi.
Humidity control, we have the water system.
Of course, we make a lot of water with our fuel cells for the electrical power system.
There are five different water tanks.
One of the tanks is sterilized for drinking.
In the early days they put too much iodine in the water.
In fact, when I came back from my first flight my teeth were totally brown.
The dentist had a fit, and they actually have cut back on that.
And the water is actually reasonably palatable at this point.
You couldn't drink it early on unless you mixed it with Kool-Aid or Tang or something, which is even worse, but that's another story.
The rest of the water tanks, there is just a lot more water than anybody can drink, and so those are called supply water tanks.
And we use that water for cooling.
Remember that is another part of the environmental control system.
We have talked about all of these a little bit.
This is just kind of a review.
Remember that during ascent, once you get above 100,000 feet and during the early time on-orbit, we get rid of excess heat by water boiler.
Now, it is an interesting system, a multi-component system.
Out in the cargo bay we use Freon to bring the heat through the radiators.
First it goes through the radiators, and so once the payload bay doors are open, particularly if you're pointed away from the sun, you can radiate a lot of heat.
After the Freon runs through the radiators, if it is still hot, you can activate the water boilers.
And so if the payload bay doors are closed or if there is some problem with the radiators and you need excess cooling capacity you activate the water spray boilers.
Now the water boilers, if that is your only cooling system, they use water at a rate faster than the fuel cells produce it.
If you cannot get your payload bay doors open -- and that is part of the whole mechanisms, remember, which Henry Pohl talked about -- if you cannot get your payload doors open within a few hours, you have to turn around and come home because you will run out of water.
Now, on the other hand, how do you get the heat out of the crew cabin?
You don't want Freon running around the crew cabin because it is nasty stuff if you develop a leak.
There are actually two ways.
First of all, you circulate the cabin air which gets heated up both by the people and the electronics.
And you run the cabin air through a heat exchanger where it dumps its heat into a cold water loop.
And the cold water loop takes cabin heat, and then the cold water loop is also circulated by some of the electronics.
Some of the electronics are air-cooled.
And, of course, there is no density driven thermal convection in weightlessness.
You have to have forced-air convection, so there are fans all over the place blowing air at the electronics and at the crew as well.
And these are all small fans running at very high RPM and so they tend to be very noisy.
In fact, the typical environment in a space cabin is dominated by high-pitched fan noise.
In fact, it gets so bad that in the Russian Mir, we found this once US astronauts starting to fly up there, it was actually causing permanent hearing damage in a lot of crew members.
And people were having to wear earplugs or headphones while they were working near some of the equipment.
From a human factors point of view, to have earplugs in and then somebody tries to call to you from the next module because there is an emergency, that is not a good deal.
Yeah.
[AUDIENCE QUESTION] No.
There are a lot of electrical signals up there because the crew uses wireless headsets, but everything is checked for electromagnetic compatibility.
And, in fact, if you try to fly a payload on the Shuttle or the Station, you have to pass a lot of EMC compatibility testing.
Anyway, the water from the cabin, which is taking the heat both from the air and the electronics, is then taken out of the cabin and passed through a water Freon heat exchanger.
And then the Freon takes the heat further aft where it is dumped into the radiator or the water spray boiler.
Cabin air revitalization.
We re-breathe the air.
It's not like in scuba diving where you take a breath of air and then you just blow it all away.
We cannot afford that so the air is re-breathed, which means you have to scrub out the carbon dioxide.
The Shuttle is not designed for long duration, and so we use lithium hydroxide scrubbers, lithium hydroxide plus carbon dioxide makes lithium carbonate plus water plus heat.
And so those are canisters which have to be replaced twice a day.
And, in fact, the preparation for replacing the lithium hydroxide scrubbers is you have to turn the cabin fans off so you don't get a lot of air going through the cylinders where you put the scrubbers.
For that one minute, you can actually appreciate the peace and quiet of being in orbit.
And then you turn the fans on again and you realize how noisy it is.
Because, after a while, you get used to it and you don't hear the noise until you turn it off.
So, that's air revitalization system.
Humidity separation.
You're pumping out a lot of humidity in your sweat, and that is put through a cooler and essentially a centrifugal, they call it a slinger which gets rid of the water.
And that goes into the wastewater tank, along with urine from the waste collection system.
And that is also dumped overboard.
There are separate dumps for the supply water and the waste water.
And, when the water is dumped overboard, of course up in orbit the pressure is below the triple point of water so liquid water cannot exist.
You either have gaseous water vapor or ice.
And so, as soon as the water comes out the wastewater dump valve, it solidifies.
And you get essentially a snowstorm blowing away from you.
And, if the sun is lighting it up in just the right way, it can be quite spectacular.
If I have time, I will show you a picture of that.
Let's see.
Other on-orbit systems?
Of course, we have the payload systems.
We talked about the remote manipulator, the robotic arm.
Anything else?
[AUDIENCE QUESTION] The crew uses lots of checklists and cue cards.
Typically, the center of the front cockpit has mostly the airplane-type controls which you use for flying the Shuttle.
The systems controls tend to be around the sides.
And then the controls for doing rendezvous and operating payloads are in the aft part of the cockpit.
The forward center cockpit is not used very much when you're on-orbit.
Overhead panels is mostly OMS and RCS.
And then electrical stuff is over the right.
And environmental control is over on the left.
[AUDIENCE QUESTION] Yes.
During the early phases of the rendezvous, when you cannot see anything, typically the commander and the pilot will sit in the front seats.
But once you get close enough to have visual contact, you typically don't approach straight on, you typically approach with your payload bay pointing towards the object you're rendezvousing with.
That way the rendezvous radar can pick it up.
There is a rendezvous radar system, KU band radar, which is on the starboard side.
And there are also star trackers which can pick it up when it is farther away.
And then visually you just look out the overhead windows.
And so you control the final phases of the rendezvous, which we call proximity operations, from the aft flight deck.
And there is a controller stick there.
And, actually, it is interesting.
This is part of the human factors.
Both for controlling the manipulator arm and the Shuttle, there is a switch.
For instance, there is a translational hand controller and a rotational hand controller.
The translational is to make the Orbiter go in plus or minus X, Z or Y.
Normally, you think of the Shuttle, when you push the stick back, you want the Shuttle to go in the direction of the stick.
And so that would be a plus or minus X, which is along the long axis of the Shuttle.
And that is the way the forward controller works.
And you can set the back controller up so that it is just the inverse of the forward controller so, when you push on the stick, the Shuttle goes backwards.
But, when you are actually looking out the window and you see the object -- the Space Station, or Hubble or whatever you're rendezvousing with -- is ahead of you and you want to go towards it, you would like to be able to push the stick.
And so what they've done is, here is the upper window, they have the stick actually at an angle.
Depending on how you set the switch, you can make it so that when you push on that stick it gives you a minus X or you can put it in the Z mode so when you push the stick it gives you a minus Z.
And that is a nice human factor.
They have a similar situation for the manipulator arm.
On the end of the end effector, when you're coming into a grapple fixture, there are actually three wires inside the end effector.
And those actually are moved around like so, so that they tighten up over the little knob.
But there is a camera up here.
And then above the grapple fixture there is kind of a bull's-eye with a little stick on it.
The stick is like this.
And if you're off to the side a little bit you can see by the angle on the stick both how you are in translation and in the attitude of the end effector.
The thing is when you're actually flying the (UNINTELLIGIBLE) and you're looking out the window, I might want to be able to push the translational hand controller and the end effector will move away from me in the cargo bay.
That is called a Shuttle Coordinate Mode.
And they have a nice setup.
Again, this was from a lot of human factor studies.
You can then go into a different mode when you're actually bringing the end effector right up to the docking fixture where you switch over into what they call end-effector mode.
Now, I am actually looking in the television picture and I see this image.
And I want to correct it and I want to move to the side a little bit and go in, but I want to do that in the frame of reference of the end effector.
And so by going into end-effector mode, when I'm working the translational and rotational hand controller, it actually works around that coordinate system.
And there are actually two other coordinate systems for different types of payload operations.
They have done a lot of work in these systems essentially to use computer-assisted modes to make it easier to operate both the robotic and also the Shuttle rendezvous and docking system.
What else do we want to say about on-orbit?
That will probably do it.
Let's take a two-minute break and then we will move ahead to entry.
Enough of a stretch.
Yeah?
Before we leave the on-orbit part, how much of the crews' time was taken up with changing the carbon dioxide scrubbers, dumping wastewater, just the general like you've got to do it to survive stuff versus [UNINTELLIGIBLE]?
Typically one person can take care of most of the on-orbit just maintenance activities.
You have to do things like periodically check your filters.
Every one or two days you have to go through with a vacuum cleaner and clear your filters.
You have to change the lithium hydroxide.
There you want somebody to help to turn off the cabin fans while the person downstairs is changing them out.
Periodically clean the toilet.
Prepare food.
I would say more or less one person around the clock can take care of most of the things that have to get done, if nothing goes wrong.
You kind of alternate between crew members?
Yeah.
Typically, you can have either a one shift or a two shift flight, depending on whether your payload requires 24-hour operations.
And typically the pilot and the commander are considered to be the Orbiter crew, and so they will take care of Orbiter-related things.
And other people concentrate more on the payload or doing space walks or whatever the mission is about.
Everybody is more or less trained to perform maintenance activities so you can share the load, but typically with a crew of six or seven you have the luxury of specialization.
Let's see.
I spoke at length about what goes on during entry so I am not going to go through all that again.
I will just run the video here to remind you.
But do remember, again, before you are ready to do your de-orbit burn once again you have to reconfigure your computer system.
Again, everybody gets in their seats, hands-off because you don't want to throw any switches, and you definitely don't want to touch the computer while it is going through this major mode reconfiguration.
It is sort of like when your computer is booting up, you don't want to mess with anything around.
You just keep your hands off.
Once you are in the de-orbit and entry major mode, that is actually then subdivided into sub-modes where one of the software loads controls your de-orbit burn and then another one is in effect while you are essentially in free fall before you hit the atmosphere.
And then, once you hit the atmosphere, you go through a series of sub-modes depending on your altitude.
And remember we spoke about how the flight control system continually has to upgrade its flight control laws depending on the atmospheric condition, so you need to know your nav.
state.
And then finally it switches into the final approach and landing mode when you go around the heading alignment circle.
That is the final step, which I don't think we talked about.
Remember I pointed out to you that since the Shuttle is a glider that energy management is absolutely critical.
And so you always come in with a little bit more energy than you are going to need because it is easier to bleed the energy off through a series of S turns.
And then they make the transition when you're essentially mach 2 or mach 3 and you are over the landing field, either in Florida or in California.
If this is your landing runway, a regular airplane will typically do a downwind, a base and then come in for a landing.
With the Shuttle, we actually want a much larger pattern to give you more opportunity for energy control.
At about 50,000 feet the Shuttle should be essentially overhead.
And then it makes a big spiral.
And, since you're in this big spiral, you have the option of either making the spiral slightly bigger or slightly smaller in order to control your energy.
If you're a little bit too hot, have too much energy, then you make your spiral a little bit bigger.
If you are a little low on energy you make your spiral smaller.
But by having every landing end in this spiral which you can adjust the radius, but the basic procedure and navigation is not going to change from flight to flight, so that makes the whole training process much easier.
You go into the so-called heading alignment circle and the pilot does have energy displays available so you get predictors.
It tells you where you will be 30 seconds, one minute and 90 seconds in the future.
You get a vertical display showing your energy, a horizontal display.
Unfortunately, and this is something for the people who are looking at displays, the horizontal and the vertical displays are not well integrated.
In the talk that some of you heard on the cockpit electronics upgrade, they had some plans to upgrade that and have a unified landing display.
But at the moment we don't have that.
There is active guidance so that there are guidance needles which you can fly to.
And, of course, the pilots are well enough trained that they are using their out-the-window cues as well to make sure that they get the right visual picture just in case something should be wrong with guidance.
Yeah.
This is a ways back.
How noticeable are the big changes in acceleration during like SRB separation or de-tail?
SRB separation, as I said, right before SRB tail-off, F equals MA and the force is pretty much constant at that point, but the mass is decelerating because you're burning a ton of fuel, over a ton per second.
So the acceleration is increasing.
And you've built up to about 2.5 Gs when the SRBs finally tail-off.
Then you drop down to about one G. You're sitting in your seat.
This is one G into your chest.
Remember at this point the Shuttle is kind of upside down.
And, since you haven't achieved orbital velocity yet, you still have some weight pulling you out of the seat.
But you are strapped into the seat.
And mostly you feel yourself pulled back into the seat.
And, of course, as your velocity increases and you get closer and closer to orbital velocity then the downward acceleration to the earth essentially matches more and more the acceleration of the Shuttle itself until, by the time you're in orbit, you are in freefall.
MECO is different.
The same thing happens.
The main engines are working at maximum, well, 104% thrust.
We don't take them up to the 109% which is emergency only.
They are working at 104% thrust and your acceleration now starts to pick up.
After about seven minutes you've gotten up to 3 Gs.
And the Shuttle was designed for a 3 G maximum.
They wanted to have smooth, easy right for the payloads.
And they also wanted to make it possible for nonprofessional astronauts to fly on the Shuttle as well, the payload specialist scientists.
The engines start to throttle down in order to hold your acceleration to 3 Gs.
And they throttle down eventually to about 65% right at MECO.
So you are sitting in your seat pulling 3 Gs, eyeballs-in, for the last minute and a half or so.
3 Gs is not particularly comfortable but, on the other hand, any healthy person can tolerate 3 Gs for a minute or two.
It is not a real big deal.
There have been concerns.
If the pilots have to reach some of the overhead switches to reconfigure the OMS or the RCS at 3 Gs, could they actually reach it?
And some of the shorter pilots decided, with the Gs, they probably couldn't.
And so they actually made a special tool, we called it a swizzle stick.
It is an aluminum rod about, I don't know, 80 centimeters long, that has an interface so you can actually use that.
And you have a little mirror if you need it as well so that you can flick the overhead switches when you're pulling Gs.
You cannot ignore human factors in these things when you're designing it.
If you've really got to flip those switches.
And it is not just the problem of being able to reach it.
It's also you have a helmet on.
And, when you're pulling 3 Gs, your helmet which may have weighed 15 pounds now weighs 45 pounds.
And so you really cannot move your head around very much.
And so just being able to pull yourself away from the seat and look up, you cannot do at 3 Gs.
So you do need some assistance there.
In principle, for future designs, I would think taking advantage of the ability to send commands through computers would make a lot of that easier, if your computer is within easy reach.
This was the early days of fly-by-wire and people felt that they really wanted to have a hard switch that they could throw.
But, in fact, in most cases, not all, but in most cases the switches go through the computer system.
And it is possible, in fact, to go into the guts of the computer.
And all the switches tend to be multiple pole.
I am getting into some of the details now, maybe more than ever wanted to know, but just to give you an idea of the levels of redundancy that they have.
Every switch has multiple poles, and it is possible that you can get a little solder ball or a little piece of wire or just one of the pole units that stops working.
You can actually call up that switch on the computer and disable one or more of the poles or you can disable the entire switch and take over computer control of the switch.
Like I say, most of the switches work like that.
Not all of them.
But it is very much a computer-controlled system.
Yeah?
How does the acceleration compare to a de-orbit burn?
De-orbit burn is interesting.
When two OMS engines fire they only produce about a tenth of G acceleration.
However, you have been weightless for a week, a month, six months depending what your mission has been.
And I will tell you, the Shuttle is accelerating forward, although you're actually going backwards.
But internally, the Shuttle is accelerating forward, so everything wants to go aft.
And, in fact, it is kind of fun.
We have done just experiments and taking pictures of it where you can actually pour water at a tenth of a G and watch the water slowly go through an arc into a cup.
And then, of course, you've got to get it closed up before the burn stops.
And you can take a little pencil and push it in the y-axis and watch it gradually curve as you're accelerating.
It is only a tenth of a G but, I will tell you, it feels like you're pressed against the wall if you're not sitting in your seat, just because you haven't been experiencing any acceleration for all that time.
And then it gets a lot worse, as you hit the atmosphere.
By that time hopefully you are sitting in your seat.
It is kind of interesting.
What I tended to like to do is keep a pencil sort of floating in front of me.
And, after a while, you see very slowly the pencil is starting to come down.
And then you would pick it up and then it is coming a little faster.
One of the early things that you realize, you know, if you've seen pictures of people in orbit, you're hands are always floating up here.
The same thing as if you go into a swimming pool, just hold your breath and float under water.
Your natural total relaxed body position is sort of bent over a little bit and your hands are out front.
Try it some time.
It is quite relaxed, quite comfortable.
And it was a strange feeling.
Again, I got used to it after my first entry.
But the first time, when I realized that I was sitting in my seat and my hands were actually in my lap, and that hadn't happened.
I had realized all the time before that your hands were sort of floating out there.
And then I reached for the camera to take a picture out the window of the shockwaves behind the Shuttle, and all of a sudden it has got weight.
It is amazing how the human mind can get used to a very bizarre situation, you know, like weightlessness, such that when you get back it feels strange.
It doesn't stay strange for long.
I mean you grew up in one G and your body has a pretty good memory, so it doesn't take long to get back.
But there are potential vestibular problems.
We have people in the Manned Vehicle Lab who have spent their lifetimes studying some of these phenomena.
A lot of your orientations for up or down come from your inner ear, which have gravity receptors.
Well, those basically don't work very well.
They don't work at all, because the little bone particles inside there are basically floating around.
You don't get the proper feedback, but you still get visual feedback and you get rotational feedback from your inner ear.
And so your body has to essentially reprogram the feedback loops so that you can maintain your sense of orientation.
And that leads to space sickness.
And about roughly two-thirds of people who go up experience some form of space sickness.
It is sort of like seasickness.
Anywhere from mild -- [LOST VIDEO AND AUDIO] [LOST VIDEO AND AUDIO] -- the feeling ofweightlessness.
But launch is really an overwhelming experience with all the vibration and the power and the acceleration and everything.
And I remember thinking to myself at MECO when I started floating, I thought now I'm in an environment that I understand.
Because I had had so many 135 parabolas that the feeling of being weightless was not as overwhelmingly new as it otherwise might have been.
Although, what I realized after about a minute, I was sort of holding onto the seat waiting for the pullout.
That is when I realized, no, wait a minute, no pullout, you're really in orbit.
And that is when I got really excited and sort of floated over to the window.
And I basically couldn't wipe the smile off my face for about seven days.
Yeah?
What about the internal clock?
Do you still have 24-hour days?
Yeah, basically.
The sun rises and sets every 90 minutes.
What we try to do, this wasn't done in the early Shuttle flights, but often you have to launch at odd hours, if you're doing a rendezvous particularly.
For instance, when we launched to repair Hubble, the launch was 4:00 AM because that's when Hubble happened to be flying over.
Now 4:00 AM, typically your body is at about its lowest point in the 24-hour cycle.
You know, your basal metabolism, your temperature.
Not the condition you want to be in when you go through a launch in case there is any problem.
What you want to do is adjust your sleep cycle.
It is very hard to adjust your sleep cycle when you have the sun rising and setting every 90 minutes because you don't get any light feedback, but what we do is, starting a week before a launch, the crew goes into a medical quarantine so that you don't get exposed to any cold germs because you don't want to get sick on-orbit.
And so at that time what they have done is use some medical research over the last 20 years or so which indicates that bright lights, at the right time of day, can actually help your body change to a new time zone.
That's why they say when you fly over to Europe, you fly overnight and you don't usually sleep too much in the airplane.
You land early in the morning.
And a lot of people then want to go to their hotel and go to sleep for a while, but the doctors tell you that is the worst thing to do as far as getting used to the new time zone.
What you really want to do is go stay out in the bright sunlight at a time when your body thinks it should be dark, get the sunlight into your brain.
And that is what we do.
In the quarantine quarters everything is white.
The tables have white butcher paper.
And the walls and the ceiling are white.
And there is essentially wall-to-wall fluorescent lights on the ceiling.
And so it is really bright.
I mean you have to wear sunglasses when you first go in the room.
And it is too bright to watch television or even to use your computer, but it works.
And after about two days you are pretty much accommodated to the new time.
And then, if you want to go out during the day when the sun is out but it is a time when the sun shouldn't be out for the time you are trying to switch to, you have to wear really, really dark sunglasses so you don't get deprogrammed.
The problem is sometimes when you're on-orbit, because of the procession of orbits, you tend to have to re-enter, on a short shuttle flight -- this is different for the Space Station -- but on a short Shuttle flight you typically reenter a few hours earlier than your equivalent launch time.
And that means that you have to often do a sleep shift in orbit.
And that is much more difficult, particularly if you try to shift earlier.
It is relatively easy to stay awake for an extra hour.
I mean everybody likes to lie in bed for an extra hour.
That is easy.
But to go to bed an hour or two hours early it is hard to get to sleep.
And then you have to wake up two hours early, that is difficult.
But that typically is what you have to do.
The medical people have actually imposed maximum shifts both per day, per week and per month for how much the crew can be asked to shift their sleep schedule.
The Russians have another problem when they do space walks because they don't have a full equivalent of a TDRS system, tracking data relay system, like we do.
They like to be doing their space walks when they're making passes over Russia so that they can get ground tracking.
And so often when a Russian crew is preparing for a space walk, they have to shift sometimes by eight hours at a time.
And it can be really tough on a crew to have to do that.
If any of you have worked third shift.
It's not like pulling an all-nighter where the next day you go back onto your normal shift.
To actually stay on a different shift for three or four days at a time can be tough when you don't have any light cues to accommodate you.
Yeah?
Just an observation that all those measures to shift the sleep cycle are obsolete now with the no night launches, right?
As long as there are no night launches that is probably true.
And what that means, of course, is that they can only launch to the Space Station at certain times of the year.
That is the price you pay.
Yeah?
On the launch pad, what is supporting the entire weight of the Orbiter?
And, secondly, if you are on the mid-deck, are there any windows?
And, if not, can you see?
I should bring in the launch bolts.
I have a couple in my office.
The entire weight of the Shuttle stack is supported on the two solid rocket boosters, so the bottom skirts of the boosters are actually sitting on the launch pad.
And so on each of the boosters there are four big nuts and bolts, four on each booster.
They are explosive nuts.
And at the moment that the solid rocket boosters ignite, all eight of those nuts are split in half so that the boosters can lift off.
Now, I asked the question what would happen if one of them didn't go.
And the answer was, well, we think that the booster is strong enough that it would just rip it out, but nobody wants to find out.
Obviously, these are redundant explosive bolts or nuts.
They did have, on one flight, they found that one or two of the redundant firing units did not fire.
And that got people's attention.
And I don't know what the final resolution was.
But basically those bolts, you know, if there is wind blowing and the Shuttle is rocking back and forth, all of that load is taken through those bolts.
The structure of the solid boosters has to be able to accommodate that.
You asked me another question as well.
Oh, the mid-deck.
The hatch, where the crew gets in, has a window in it.
Before Challenger, one seat in the mid-deck was right next to the hatch.
My first flight, I actually sat next to the hatch during ascent downstairs but had a great view out the window.
Now, after Challenger, we have an ejection pole, a telescoping pole which is suspended from the ceiling.
I think we talked about this before.
If the Shuttle is in a situation where it is in stable flight but you cannot make it to a runway, at about 40,000 feet you blow the hatch.
The telescoping pole, you release the pin, the pole will extend, and then one by one you clip onto the pole and you jump out.
And the pole takes you below the wing so that you won't hit the Shuttle structure.
And then eventually your parachute will open.
That now takes up the area over by the window and the three seats no longer have a view.
If you're sitting on the mid-deck how during a launch, how do you know what is going on...?
[UNINTELLIGIBLE] You hear the comm.
loop.
And if you have a nice commander he will tell you.
For instance, if you have a rookie astronaut onboard, they want to know when they go through 100 kilometers because that means you have officially been in space and you can get your astronaut wings.
So, if the commander is nice, he will call 100 clicks on the way up and congratulate the new rookies.
But basically that is about it.
And the same for entry, you don't have a view.
There are still plenty of physiological things to keep your attention, so it is not like it is a boring ride, but you don't have the view.
OK. Let me run the entry picture here.
OK. OMS engine ignition is when you see that big flame pattern.
During the burn you don't see anything.
Now, looking out the front, you see the plasma sheath.
It goes from a dull red gradually to orange and then eventually white hot.
You can only see this at night.
It is fairly faint, but typically a lot of flights you land during the early part of the day on the ground.
And, of course, you do your burn halfway around the world so the early part of your entry is very often at night and then you fly into the day.
And then I think there is another shot looking out the aft window where you actually see the plasma wake in the back of the Shuttle.
And you can see this is the old-fashioned computer screen.
That is the wake looking out the overhead window.
And, where these converge, that little point there they say is about 10,000 degrees Fahrenheit, which is the surface temperature of the sun.
It is remarkably stable, although every once in a while you get those instabilities.
And every once in a while you get big bright things go flying past, which I assume are gap fillers coming out.
And I remember I would always think to myself I hope that is nothing important.
You're basically riding on a meteorite because it is supposed to make it all the way down to the earth.
But that is essentially what you are seeing, that trail behind you.
And I am sure you have all seen these landings.
Remember all the talk about the landing gear and the brakes.
I looked for a video from my first flight to show you the blowout.
I couldn't find one, but if I find one I will bring it in and show you another time.
This is a landing out at Edwards.
Have you ever landed at night?
I have landed at night.
I was going to show you one other.
It doesn't have a landing light, does it?
No.
Do they light up the runway?
Yeah.
They have two banks of xenon lights which light up the runway for you.
Actually, this next video is from my last flight where we deployed the tethered satellite.
I am not going to show you all that stuff, but I wanted to show you the launch sequence because we have a few pictures showing the opening of the payload bay doors.
And then you can actually see back to the aft end of the Shuttle where you have the main engine.
And remember I talked about how the hydrogen and oxygen have now combined into water and that forms ice.
And so for the first day or so on-orbit there are just constant ice particles coming out of the engines, and you are surrounded by this cloud of ice.
Unfortunately the pictures are black and white because it is low-light level TV, but visually it is spectacular because these little ice particles glint in the sunlight.
And they are rotating so they go through all the different colors of the spectrum and you are surrounded by this.
You will also see one of the RCS jets firing.
Water deluge.
Main engine sequence.
Again, you see the whole tilt.
When it gets back to vertical we take off.
This is a much shorter sequence here for the launch than the last one.
How does it roll?
The SRB boosters go like that.
Is it the entire stack or just the lower...?
No, it is just the nozzle at the bottom.
This was Columbia.
You can always tell Columbia because, if you look at the vertical stabilizer up here, that is an infrared pod.
I will talk about that in a minute.
The payload bay doors open right away.
And so now you can see, when we've done the engine vent, you can see all the ice particles coming out.
And you will see a primary RCS thruster go in there.
It can be pretty spectacular.
And that just keeps going for about a day and then you've gotten rid of most of the ice particles.
OK. That is enough of that.
Yeah?
[AUDIENCE QUESTION] That was a primary RCS thruster.
[AUDIENCE QUESTION] Yeah, absolutely.
In fact, all of the vernier thrusters are downward thrusting only.
Remember we talked about with the primary thrusters, they were designed and sized for rendezvous where you actually have to have controlled propulsion.
And if you want a Z propulsion of the Shuttle, you want to be able to get a pure Z so you fire thrusters on each end.
And the same for roll and yaw.
And you don't want a lot of cross-coupling because you're targeting your burn to fractions of a foot per second in order to get your rendezvous to come out right.
And so that is when you're using the primary thrusters.
Other than that, the primaries use a lot of propellant.
And so we disable the primaries and use the verniers.
With the verniers there are two verniers in the nose which are sort of canted down at about 45 degrees.
And, in the rear, there are two pointing straight down and two pointing straight out.
Typically, for a given maneuver, you will fire only one or two.
If you want to do a roll, for instance, you will fire the 45-degree down forward thruster and the downward pointing aft thruster.
But they are not completely balanced so you will get a little bit of pitch and a little bit of yaw.
And you don't use the verniers for propulsion at all.
Although, because all of the verniers are essentially downward firing, if you are, for instance, station keeping, if here is the Space Station and you're in the same orbit but, say, half a mile in front of it, now you're station keeping using your vernier thrusters, typically that does give you, after a while, a little bit of propulsive maneuver which pushes you in.
So, you have to compensate for that.
The verniers are single string.
There is no redundancy.
They do have access-to-access cross-coupling.
They are designed for attitude control only; 25 pound thrusters are reasonably efficient.
I thought at the end we could explore a little bit together some of the ideas of improvements to the Shuttle which you are going to be thinking about.
And one of the things that I thought would be interesting to think about, you know we talked about the fact that if you're designing the Shuttle to the same set of requirements that were originally put on it, your ability to change some of the major physical characteristics of the Shuttle are very limited.
We can do a lot with electronics and some of the other systems.
The new vehicle, the crew exploration vehicle, is being designed not just to go to earth orbit but they want a vehicle that can actually return to the earth from the moon or from mars.
That means instead of an orbital velocity of five miles per second you have a reentry velocity, a hyperbolic velocity, of about eight miles per second.
Does anybody remember the relationship between low orbital velocity and escape velocity?
If your lower velocity is V, what is your escape velocity?
Square root of 2V.
That is a nice thing to remember.
It is an easy way to keep those things straight.
First of all, you don't need wings to go to the moon.
They don't do you much good when you get there.
To have a winged vehicle that you can thermally protect against a hyperbolic reentry velocity just doesn't make sense.
It would be too heavy.
We don't think we can do that.
And, in fact, the Russians are now talking about building this new clipper vehicle which possibly will have wings on it or be a lifting body.
And that actually probably means that they will only use it for low earth orbit.
But it does bring up the interesting question, if you are designing a vehicle that is only designed to go to earth orbit as a passenger carrying vehicle, one of the first things that we ought to ask ourselves, if we're doing a new design, is should we have wings?
Think a little bit about the advantages and disadvantages of wings.
If you're sitting down now with a blank drawing board going through a design process, first of all, what are the advantages of wings?
You've heard quite a bit about Shuttle aerodynamics.
What advantages do you get from using wings during a reentry?
And, obviously, it's only during entry that the wings are going to help you because they are a real nuisance during ascent.
Greater cross-range?
Cross-range helps with the wings.
The glide?
Yeah, it allows you to land on a runway.
In terms of having to design against the shock, you know, the Crew Exploration Vehicle is going to land with parachutes.
And they still don't know are they going to use airbags or retrorockets or crushable structure or whatever.
They don't know.
But it is going to take a much harder hit.
That may potentially affect reusability, maintainability.
Yeah?
I was going to say just purely control while coming back in ... [UNINTELLIGIBLE PHRASE].
Well, we had the cross-range.
But targeting I guess, too?
Targeting.
If you were to land, say, in a certain part of the desert in New Mexico, you would want to have a lot more accuracy?
Right.
For the CEV, they are just basically saying they are going to land out in the Western desert somewhere.
Now, the Apollo shots were getting pretty accurate.
The last couple of Apollos landed within visual range of the aircraft carrier that went to pick them up, so they were getting pretty good.
But that's within a couple of miles, which is fine when you're landing in the ocean.
And maybe it's OK if you're landing in the Western desert.
It is certainly not OK if you're going to land on a runway.
Hardly an engineering consideration, but how much thought in the design was put into making it a winged vehicle for it to be a comfortable and familiar look to people, especially if the idea was to have nonprofessional astronauts fly?
It seems more comfortable to get in something that looks like an airplane?
I don't know about comfort from that point of view, but there is no question that the whole aviation metaphor dominated discussion in the early days.
And I think you have gotten a sense of this.
The idea was we want a reusable vehicle.
That means we have to have airline-type operations.
Well, if you're going to have airline-type operations then you want a vehicle that lands on the runway.
You take it into the hangar.
You turn it around.
You launch it again.
I mean it's a very powerful metaphor.
What are some of the disadvantages of wings?
What are some of the things that argue against wings?
Again, we're going right back to the conceptual design exercise here.
And this is not what you're doing for your papers because we're totally changing the concept of the Shuttle.
But, for the purpose of the class, let's think it through.
What are the disadvantages of wings?
I think a lot of what we talked about is initial design cost versus, you know, and no wings obviously has a lot less initial design cost because you don't have to design the structure to support the wings and it is a lot simpler design, I think.
So, that probably saves you money in your initial development.
[AUDIENCE COMMENT] Now, on the other hand, I don't know how much any of you have studied the theory of reentry bodies, but the reentry of a ballistic capsule typically gives you much higher Gs than you can get if you have a lifting body.
And, of course, with wings you get much more lift.
You can actually decrease the G loading so that with the Shuttle we can have a reentry pulling no more than about 1.5 Gs.
Whereas, you come back in a capsule even from low earth orbit and you're going to be pulling 4, 5, 6 Gs.
And that does relate back to the idea that we wanted the Shuttle to be able to take much more delicate payloads and hopefully take people who were not necessarily trained astronauts.
So, again, it's an advantage but a disadvantage of wings.
The area under the wings is huge, so that puts huge demands on your thermal protection system.
I don't know what they're going to end up using on the CEV, but it is going to be a much smaller area that has to be protected.
Another interesting thing, when you think about a winged vehicle like the Shuttle, now I know that Buran, the Russian Shuttle, made its one flight unmanned.
They had some tense moments at the end, but they did manage to land it successfully.
But the way we fly our Shuttle, we have to have two trained pilots who spend a tremendous amount of time doing nothing but learning how to land the Shuttle.
Sometimes we don't think about the crew requirements on the design, but training is not an insignificant part of the cost of operating the Shuttle.
There is a huge enterprise involved in training astronauts.
And training pilots particularly is very expensive.
You have to keep them qualified as pilots so NASA maintains a fleet of T38 supersonic jets, which are great fun.
I mean I love to fly backseat in them.
But the only justification for having them is if somebody is going to be a pilot they have got to stay current as a pilot which means you have got to fly.
And we have the Shuttle Training Aircraft which I have spoken to you about.
And we have all the simulator time.
And then we have the problem that people said -- there was a time when they were talking about having the Shuttle spend extended periods of time up at the Space Station, you know, a month, maybe two months, make some adjustments to the Shuttle System so you can do that.
Then the problem is, well, who is going to land the Shuttle, because I talked to you about the spatial disorientation and everything that you feel during reentry.
Well, you don't want a pilot to do that.
And, if a pilot has been up in the Space Station for one or two months, has not had any stick time, is going to have potentially serious neurovestibular problems during entry.
And so the real question was what's the maximum time that it is OK for a pilot to spend in space?
We will never actually have to answer that because 17 or 18 days was the maximum Shuttle Mission.
That turned out to be OK. And we are never going to do long duration missions with the Shuttle and the Space Station.
Yeah?
[AUDIENCE QUESTION] No.
They learn a lot of the malfunction procedures, but you don't actually fly the ballistic capsule.
I mean I am not in any sense denigrating the training that the Apollo astronauts needed to land their capsule, but it did not compare in the least to the amount of training, for instance, to land on the moon where you were actually piloting the vehicle.
OK. Well, I hope just that very brief exploration will make you think about designing new vehicles from a slightly different point of view.
In other words, when we designed the Shuttle, it was just assumed as gospel essentially that if you want a reusable vehicle put wings on it.
I don't know what the right answer is.
It probably depends on some of the details of the mission, but you really have to go back and question your most basic assumptions.
When you're doing your concepts and you're setting out your requirements, you don't want to write your requirements in such a way that they presuppose the technical solution.
If you write your requirements for a winged vehicle you have already shut down a whole part of designed space.
And, as designers, you don't want to do that.
Yeah?
Last question.
[AUDIENCE QUESTION] No.
The Russians have actually taken, my understanding is, they have taken some of the electronics out of capsules and reused internal parts.
But the exterior shell of the capsules I don't believe have ever been reused.
OK. We will see you Thursday.
Bob Ried is going to give a talk on aerothermodynamics.
And I will look forward to seeing your journals.
Any questions on the reports, the outlines that I returned, email me, come to see me.
I am happy to talk with you about it.
Today Dr. Robert Ried, Bob Ried, is going to talk to us about aerothermodynamics.
He got a bachelor's in Mechanical Engineering here at MIT a few years ago, and then went on to get a doctorate at Rice University.
And he has done a lot of things.
You all have the biographies on the website so I hope you've looked at that.
And, like I say, it is better for you to read it rather than me to take five minutes at the beginning just going through it.
But he has played an important role in developing both the theory and the practice of atmospheric reentry both for Apollo and the Shuttle.
And, in fact, in many ways the experience gained through Apollo allowed us to design the systems for the Shuttle.
And so he is actually going to talk about both Apollo and the Shuttle, comparing and contrasting.
And I think it is going to be a very interesting opportunity for all of us.
And I am going to let Professor Cohen say one or two more words.
What I want to say is that Dr. Ried has done a fantastic job in understanding aerothermodynamics.
If you look back on Apollo that was one of the technologies we really didn't understand.
Coming in at 36,000 feet per second from the moon in various atmospheres, what was going to happen?
And, in that era, we had to do a lot of research and development to understand the methodology of doing aerodynamic heating.
The systems engineering comes in as you heard Bass Redd talk about the aerodynamics, you heard Tom Moser talk about the tile system or the thermal protection system, and Dr. Ried sort of tied that together.
Because he actually came up with based on the trajectory you were flying, based on the atmosphere, based on the materials that you had, what the surface temperatures were going to be and how you designed the thermal protection system to withstand those temperatures and still maintain the back-face temperature at a suitable level.
I have a couple of displays for you, and if you can start looking at them.
The first is a specimen that came out of Apollo 10.
That was a Lunar orbital mission, and it is a core sample out of an ablative heat shield.
It shows you the thickness of a thermal protection system was about two inches, and the depth of the char layer is about an inch.
And we always used to give Dr. Ried a hard time, why couldn't we take some weight out of the heat shield?
But he always said he never flew the Design Reference Mission.
And he will talk about that a little bit.
So, that is one specimen.
The other specimen is a tile that flew on the 102 mission four times, I believe, that it got to 2800 degrees Fahrenheit.
The ablator got to 42,000 degrees Fahrenheit.
These will give you some examples of what the results of Dr. Ried's work was in terms of designing the ablator for the Apollo vehicle and the tiles for the Shuttle vehicle.
That is really a systems engineering job.
I hope you can appreciate understanding the aerodynamics, understanding the guidance, navigation and control trajectory that you fly and understanding the materials that you're going to use.
That is really a systems engineering problem.
Let me now turn it over to Dr. Ried to put all the details together for you.
Thank you.
Good morning.
We are going to try to cover an awful lot of ground here in short order.
First, I am going to try to go through aerothermodynamics in terms of the discipline and give you an understanding.
I am going to try to relate to what you've had in your basic heat transfer and fluid mechanics courses to actual application and where the technology is today.
I am also going to, as Aaron said, go back to Apollo which is really where the rubber hit the road in terms of being able to enter things at those types of velocities.
And then I will get into how that affected the Shuttle.
And the Shuttle is a revolutionary system in many ways, but particularly in terms of the technology associated with aerothermodynamics, computational fluid dynamics in particular.
And I am really going to talk about three levels of aerothermodynamics which I will explain as I get into it.
And then I will try to get into the systems engineering, as Aaron mentioned.
And, in fact, one of the major differences between Apollo and Shuttle, which we had to accomplish, is a significant accomplishment in systems engineering which I hope to get into.
First, I don't think people generally appreciate the perimeters of what we're working with.
Traveling at orbital velocity due east 25,000 feet per second, that is about five miles a second.
And the Orbiter is about a quarter of a million pounds.
And, I am sorry, I am going to use a lot of English units.
A quarter of a million pounds is a pretty good sized system.
Picture that at Logan Airport and one second later in this classroom.
That is five miles a second.
Another way of looking at it is that is about 125 tons.
If you take a supertanker, about a half a million ton, and just convert the energy, that supertanker would be traveling at 250 knots.
That is the energy that we have to dissipate.
Norm Augustine introduced the concept of rocket scientist.
Our job was to dissipate that energy.
If I was coming back from the moon that supertanker, which is about a billion pounds, a half million ton, would be traveling about 400 knots.
Almost twice the amount of energy.
Actually, the whole problem was relatively simple.
Basic parameters are first free stream density of the air that you're flowing into, the velocity that you're moving at and unit area that the mass flux is just density times velocity.
The momentum that that air has relative to you per unit mass is the velocity, so this is the momentum flux which basically is the pressure to a very high accuracy.
The energy that the gas has relative to you is one-half V squared.
And so we're looking at an energy flux coming to the vehicle one-half rho VQ.
Those are very important parameters.
Now, I am going to focus on accuracy and not on precision.
People have gotten lost in details in terms of getting parameters.
If you've got density and velocity, you basically have the gas parameters with some major exceptions which I will get into.
And that is relative to the equation of state, if you want, the constitutive relations.
The other thing is the geometry which is the major difference between the Apollo and the Shuttle.
All right.
Now, let's go back to the Apollo system, a nice, simple system flying at angle-of-attack.
One of the first problems that I was introduced to was a newspaper publication saying this system is going to burn up.
It is going to burn up from the thermal radiation from the gas cap.
Now, let's consider coming back initially at 36,000 feet per second.
Peak heating is around 33,000 feet per second, 10 kilometers per second if you want.
We've got a shock free stream density.
At some point, we get to ideally equilibrium air.
Out here you're basically at ambient temperature of ten to the minus four, ten to the minus three atmosphere density.
Here, once you get to equilibrium, you're operating at a temperature of about ten to the fourth degrees ranking.
The effective black-body radiation from the sun is 10,000 degrees ranking.
The distribution in terms of the spectrum.
That is the temperature of the gas at equilibrium.
At the wall of the vehicle perhaps 6,000 degrees ranking.
And we will get into that a little bit.
But what happened was there were experiments done in shock tubes indicating some very intense radiation.
This gas is hot enough to radiate.
And, in fact, at the peak heating about half of the heating is from the gas radiation.
The gas is ten centimeters thick, four inches, and yet it is hot enough and radiates enough that the heat transfer from that radiation is almost comparable to the convective heating.
The load is another matter.
But what happened is out here you've got molecular oxygen and nitrogen just sitting there at ambient temperature bouncing along.
All of a sudden it is swept up by this snowplow.
The degrees of freedom basically two rotation, three translation, five degrees of freedom, the gamma of 1.4.
Go through a shock, the density ratio is only a factor of six.
Here the density is 15 to 20 times free strain density.
And the temperature goes to 100,000 degrees ranking if it just stayed as a molecular gas.
Now, in fact, that temperature has no meaning other than if all the energy went into translation and rotation that would be the temperature.
Let me just plot temperature as a function of distance.
And these were monitored in shock tubes.
We worked predominantly at AFCO with an electric wired discharge in helium.
And we could get to ten kilometers per second or 33,000 feet per second.
Conceptually, what happens is if you plot temperate as a function of distance here, ambient temperature is negligible.
Initially, theoretically, you've got this ten to the fifth, so your translational temperature comes down to ten to the fourth.
And I will talk about the dimensions here after a while.
But the initial collisions, I mean basically the molecules are piling up into other molecules.
Quickly, you have collisions that give rise to vibration, ionize the electronic energy levels in the molecule.
Once it is vibrating it will also dissociate, so then you've got ionized atoms as well.
And we have characterized these as basically temperatures, a vibrational temperature.
Again, this is a very non-equilibrium situation so the concept of temperature really goes away, but it is sort of a measure of energy to equate, for example, vibrational and electronic.
Now, what happened was I mentioned how significant the radiation was from the equilibrium.
In the shock tube, the radiation in this region where the energy has not distributed itself into an equilibrium level was two orders of magnitude higher.
That led to the publication you cannot bring people back from the moon.
That radiation is just going to vaporize the capsule.
Now, things were with us.
The time to relax and the measurements of radiation, if you want, were extremely intense and then they relaxed to equilibrium levels.
It turned out that we used what was termed a binary scaling.
This relaxation distance depended on how many collisions you had.
Collisions depended on the pressure.
The pressure depended on the density.
The velocity is fixed coming in from lunar conditions.
It gives you the pressure.
In the shock tubes, we couldn't quite get to the low pressures that we had on Apollo.
And I will show that in my first chart, actually.
But what happened is at the pressures that we experienced, this distance came to be very small.
And that increased the rate so that even though we had two orders of magnitude higher radiation intensity, it was two orders of magnitude smaller in radiating volume.
And so the non-equilibrium radiation actually turned out to be a little less important than the equilibrium radiation.
Now, the point is that before we went to the moon we didn't have the foggiest idea of what was going on.
We knew that you couldn't use ideal gas, we knew that you had to go to equilibrium, and everybody was worried about trying to calculate equilibrium accurately.
All sorts of charts and tables.
We didn't have the computer capability that we have today.
Everybody was focused on equilibrium, but then initial results in one of the major facilities for aerothermodynamics which is basically a shock tube.
Does everybody understand a shock tube and how it functions?
Basically, it is a one-dimensional situation, just like I've described here, but the way it is generated is you have a driver section and a driven section.
In the driver, you try to get extremely high pressure with a high speed of sound.
Here you have a diaphragm.
Here you have, if you want, rho infinity.
This is the density that you're trying to test in.
What we had was an electric wire that exploded, got into the helium, went to tremendous pressures.
Basically, these were all battleship guns, if you want.
And we had to go to a two-inch diameter tube in order to get close enough to the Apollo.
Basically, you start with a pressure distribution that looks like this.
This is pressure as a function of distance.
Once you discharge there is a diaphragm here.
The diaphragm cannot take that pressure.
The diaphragm blasts and you get an expansion wave coming back here and a shockwave traveling in that direction.
The reason we had to go to two feet is you get a boundary layer building up.
And if you want some test gas -- I've kind of go this reversed here.
In here you've got the shock, you've got a boundary layer building up and, if you have two small tubes, the boundary layer will suck up the gas and you don't really have a test condition.
But AFCO worked everything out very nicely and we had a very nice test condition.
We basically had this condition going by, a little higher pressure than we experienced on Apollo, and we could measure the radiation.
The challenge at that time was the instrumentation and a quick response capability to measure all that.
We learned an awful lot.
We went from ideal gas like you get in Shapiro to real gas equilibrium.
In fact, James Fay, in course two, had done some real pioneering work looking at stagnation point heating and a gas dynamics.
I am getting kind of lost in that detail, but basically this was the phenomenological aspect that we had to understand in order to work with Apollo.
This was not why we had twice as much ablator as we needed, and I will get to that later.
Second area, what I'm talking about here basically is fluid convection and some chemistry or physical chemistry.
The next significant item is diffusion.
The simplest case, obviously, is thermal diffusion.
The first thing you learn in unsteady heat transfer, you have a one-dimensional situation, temperature initially, and you have a heat input.
You end up with a distribution of temperature which diffuses out.
The functional form of that, if you recall, T is proportional to an exponential, let's call this X squared over four thermal diffusivity times time, and there is the square root.
Think of it as a one-dimensional normal distribution.
Except instead of two variant squared here you've got four times the diffusivity time.
The significant thing is when we talk about ablators, when we talk about tiles or thermal protection systems, this is the significant parameter.
We have an extremely high temperature at the surface.
The job is to keep that from the structure.
Thermal diffusivity is prime parameter.
It is also, not thermal diffusivity, but if you now consider simplest fluid mechanics, flat plate, the first thing you learn about in terms of viscous flow.
You've got some velocity coming along here.
You've got a boundary layer that builds up.
The boundary layer compared to X varies as one over square root of Reynolds number.
Where does that come from?
The same diffusion.
It is a diffusion of the shear of this wall against the undisturbed flow, whether it be the thermal diffusion associated with heat transfer, the diffusion associated with the shear, the same phenomena.
When I first found this in fluid mechanics, I was told that was empirical.
And it basically is, but it really goes back to the fact that if you approximate the Navier-Stokes Equations in this one-dimensional situation you basically have a convection in this direction and a diffusion in the [UNINTELLIGIBLE] [Unthougtable]direction.
This is extremely important, not just in terms of boundary layer but also in terms of the Stanton number which is approximately heat transfer divided by one-half rho infinity V infinity cubed for the case of a sphere.
Now, this is a one-dimensional, obviously, flat plat.
A sphere is also one-dimensional.
If you look at a stagnation point on a sphere, this is a singularity.
If you go to spherical coordinates you've got basically the same phenomena.
The diffusion from the wall gives you characteristic dimensions.
It is very important because heat transfer is basically inversely proportional to the square root of the Reynolds number.
We have square roots showing up in the thermal diffusion.
We have square roots showing up in the heat transfer.
Now, when you put in the effects of real viscosity and everything else there is a little variation.
But first order is the behavior that we are looking for.
And that I will illustrate now.
If we go into a wind tunnel and we measure, well, let's just basically go to fly.
Let me do local heating over the energy flux.
I am going to do a logarithm of that as a function of log of what I'm going to call a normal shock Reynolds number.
All I do, as I go across the shock, the density of velocity -- The mass flux is conserved, obviously.
The viscosity is what changes.
I am going to use the normal shock equilibrium air viscosity because that is more characteristic of everything going on in what the Russians call a protective shock layer.
If I looked at either the wind tunnel data or flight data in the region of prime concern, I've got that minus one-half slope from that square root Reynolds number.
I will also have a bound up here that my heat transfer cannot exceed the energy flux.
And, obviously, if you go all the way to orbit you're basically one.
The molecules are coming in at orbital velocity, going into the material, releasing all their energy and then eventually coming off.
We actually do have that limit, although we never really worry about it too much.
The significant heating is in this laminar regime.
I feel like, because I understand Navier-Stokes Equations, because I understand the diffusion process and that limit of the Navier-Stokes, I sort of understand that flow.
However, we also have turbulent heating which incompressible flow has its slope on the order of [minus fifth?].
I don't really understand stand that, and do not claim to understand that.
It is a combination of the diffusion and the convection, all sorts of things going on.
And then we have transition from one to the other.
Theoretically, this would come out and come out lower.
Now, this is extremely important in the shuttle design, as I will get to when I get the charts.
But basically we go into a wind tunnel and can measure this on a particular configuration.
We got to flight.
And let me mention heat transfer and aerothermodynamics has been a field of study for many years.
People have built models, have done wind tunnel tests, shock tube tests, flown vehicles.
And there was a lot of money spent trying to fly vehicles to verify our understanding.
The difficulty, certainly with ablators, is measuring the environment.
Did you really have the right heat transfer or did the material just start to degrade?
What is really going on?
When we flew the Shuttle, we had a reusable thermal protection system which is the tile that is being passed around with a real thin coating and very low thermal diffusivity.
What better instrument could you have to measure heat transfer than that?
And the data from the Shuttle configuration is absolutely fantastic from a technological standpoint, from an aerothermodynamics standpoint.
Unbelievable.
So much better than anything that was done in the past with vehicles that were dedicated to try to get that information.
Let me say a couple words about, well, first let me address the facilities.
I mentioned the shock tube and went through that briefly.
And the value of the shock tube was to look at phenomena just like this.
You also can, which is where Professor Fay got his information, put a model in the shock tube.
Right here.
And all of a sudden you've got the flow at the right enthalpy and closer conditions to flight than anywhere else.
Getting to these energy levels is extremely difficult.
There are three facilities you can get to.
One is in a shock tube.
Second is a ballistic facility where you fire a projectile that looks like what you're going to fly.
And that does a good simulation job, except it is very small.
And getting any measurements is very difficult.
If you go into a wind tunnel you can get great measurements but you cannot get the energy.
The shock tube and ballistic facility, I will just put this down here, it has the difficulty of measuring things.
The other place you can get the energy is in an arc jet.
Now, an arc jet is a continuous electrical discharge, sort of like the shock tube, where you're discharging an arc along a tube about the length of these tables, a very high current continuous for a significant period of time into nitrogen.
And then you're expanding this and you're introducing a supersonic flow.
And you test materials like tiles or the ablator in as close to the environment as you can get on the ground.
That is for the times that are required.
Entry times are like 20 minutes.
You need something that has the energy and runs for 20 minutes.
Now, this is a continuous flow of electrons.
The thermodynamics here is not well-characterized.
People have studied it like mad.
This is very non-equilibrium.
You've got a very high density of electrons.
The temperature is immense in here.
It is not at all in equilibrium.
But, as you expand through a jet, collisions drop because the pressure goes down and it pretty much freezes.
But this is where we do research on ablators and research on service catalysis which I will talk about in a little bit which really can affect the heating, particularly on the Shuttle.
And I will show you some results there.
But, in any event, the arc jet is the other major facility from a TPS materials standpoint.
TPS being thermal protection system.
Wind tunnels are classic.
That's where NASA came from, NAC came from.
Aerothermodynamics is my field, spelling is another area.
Wind tunnels you're all familiar with.
And, on Apollo, we tested in every facility you could think of because we didn't know what we were doing.
And whatever the different facilities told us we tried to understand.
What we learned was energy is extremely important, the enthalpy.
And, frankly, going beyond mach 8, which is what everybody tried to do, get to the higher mach number, really doesn't work too well in a ground facility when you're talking about heating distribution.
It does give you mach number effects.
But the gas is not at all like what you have in flight.
And so on the shuttle we didn't really do much aerothermodynamic testing beyond mach eight.
That gave us the right normal shock Reynolds number region, it was closer to, on the Shuttle, the actual flight environment than if we had gone to a mach 12, a mach 16, a mach 18, a mach 20 facility.
Because the way they get the mach number there is to get the temperature very, very low.
It is very valuable information, but you really need to understand what you're doing with that.
You cannot just take mach number as the major parameter.
It is a significant parameter but you need to get the Reynolds number.
And, fundamentally, you need to get the enthalpy of the gas.
And there is not really a good dimensionalist number for the enthalpy of the gas.
Too much going on for that.
The other significant source of information is flight test.
In the early days with the capsules, and certainly with Apollo, we did a lot more testing, invested a lot more on testing than we do now.
For example, on the Apollo, I talked about this radiation, we had very good test results in a shock tube.
We thought we really knew what was going on.
NASA Langley went and built a small scale Apollo about so big to measure the radiation in flight because the shock tubes are operating at little higher pressures than we were going to fly the Apollo.
It basically had three brilliant heat shields with a quartz window in it.
It comes in zero angle-of-attack and gets to a given point where the window basically starts to melt.
You cannot see through it, you cannot make a measurement, it sheds that.
And ideally the second heat shield comes about right at peak heating simulating Apollo entry.
And then a third.
Well, we were really excited about that because the shock tube was great.
And we thought we understood everything.
Well, we flew Fire 1.
It was called Fire.
I don't remember what the acronym stood for, but it was basically a small Apollo coming in.
We got the data and it wasn't anything like what we expected.
What had happened?
Well, it turned out that in order to get the high speed, we launched the fire vehicle, and then we fired a rocket down back into the atmosphere to simulate lunar return.
Everything went well.
There was a spring between the capsule and the boost stage and they separated by design.
Fine.
But what happened was the booster lined up with the wake of the incoming test vehicle and just road the wake because there was a lot less resistance where the air has been sucked along, clobbered the back of the vehicle and it mutated.
And so we weren't looking at the stagnation point.
We were looking off at 30 degrees.
Not only that but the radiation was doing this.
Figured out the problem, sorted it out and had the second flight test which worked beautifully.
It turned out at altitude the radiation did not agree with the binary scaling that I mentioned, and basically the radiation profile here, as I said, was like two hours magnitude higher characteristically.
But at high altitudes it just wasn't there.
At peak heating it was.
But, prior to that, which was a significant heat load, it was not.
There were not adequate collisions to excite the radiators before the gas came to equilibrium.
Now, as I mentioned before, at peak heating conditions on Apollo the shock, I cannot draw it very accurately to scale, was on the order of about ten centimeters or four inches here at peak heating.
[Q]: Is that the thickness of the shock or the standoff distance?
The standoff distance, thank you.
Standoff distance.
The shock is relatively thin, a few mean free paths.
And some people might call this effective shock thickness, but I term the transition to higher temperature of the shock thickness and then a relaxation distance.
Any altitudes above that the flow could be completely out of equilibrium.
It hasn't gotten back to equilibrium because this characteristic time is directly proportional pressure.
And I erased it.
It is directly proportional to free strain density.
So at higher altitudes it out of equilibrium.
As you will see shortly, Shuttle is much higher altitudes than Apollo.
And I will explain why.
At lower altitudes this compresses and is predominantly at equilibrium radiation.
And that we were able to characterize.
OK. Back to ways of getting information.
Numerical simulation.
This is a significant revolution that you're probably more familiar with than I am, but it really occurred right through the development of the Shuttle.
And frankly -- Well, I will show this in charts in terms of the design approach to getting the aerothermodynamic environment at flight, used the same technology that we used in Apollo.
And that was we take a scale model, go into a wind tunnel.
Now, this is a blunt vehicle, and so if you look at it as an inviscid flow, the close to normal shock entropy is basically the entropy of the entire body inviscidly.
And so the normal shock or strong shock gas dominates the flow around the vehicle.
As you go low angle-of-attack you get into an entirely different situation.
In any event, what happens in terms of the wind tunnel, the heating here compared to different distributions, is all pretty much proportionate.
And that is fundamentally what we used.
And that included even the wake region back here.
And I will talk more about that when we get into the charts.
On the Shuttle we get into far more complicated geometries.
And the major challenge on Shuttle was that three-dimensional geometry.
And we did a bunch of numerical simulation development.
We had focused on the subsonic region on Apollo which basically is most of the front side here, is basically subsonic.
About the middle of the vehicle down is supersonic.
And then, of course, you get into the wake here.
This was our flow field interest on Apollo primarily because of this radiation.
We had to understand the flow field in order to be able to compute the radiation.
About half of that radiation is optically thick from the ultraviolet deionization radiation and also line radiation from atoms.
Optically thick.
The other half, the radiation is basically optically thin, the molecular band radiation and other things.
We needed to understand the flow field.
We did some pretty crude engineering calculations in order to get that flow field, but we were focused on getting to a numerical simulation of the subsonic region in particular.
Now, in those days the computer was our real limitation.
And, of course, it was sort of embryonic.
We have collectively just become orders of magnitude beyond where we are today.
However, the design approach that I would recommend now is basically what we would like to have used on the Shuttle.
And, to an extent, we did.
Before we actually flew the Shuttle we had confidence as a result of numerical simulation.
It is a combination of these facilities and numerical simulation.
This problem of non-equilibrium is kind of beyond numerical simulation, in my opinion.
Now, you can correlate things, but I would like to point out the limitations.
You don't just go get a computer program and you calculate everything because there are some very fundamental things missing.
The way one gets reaction rates, for example, dissociation rates or recombination rates is to go into a shock tube.
As you approach equilibrium, the concept of temperature, the equilibrium constant makes sense.
So you can relate forward and backward rates.
And you do diagnostics spectroscopically on a concentration and you get rates.
And it gets to be kind of complicated.
There was a lot of stuff back in the early Apollo days said in terms of coupling between the different modes of vibration, the electronic excitation.
It is a pretty complex problem.
But, coming to equilibrium, is where we get our reaction rates for high temperature gases.
Coming back here, we're not close to equilibrium.
Temperature doesn't work.
Unless you start with an end body problem and Schroedinger's equation, you're not going to really be able to compute this.
Now, what is used in the field right now is the basic data and a jump condition in terms of what matches the relaxation condition.
I am not knocking it.
All I am saying is there are limitations to numerical simulation, whether it be in physical chemistry or just about anything you work with.
At the same time, it is invaluable in terms of relating what happens in a wind tunnel and what happens in flight or what happens in an arc jet and the phenomena.
I will say a little bit more about that as we get into the charts.
Before we flew, in regions of the vehicle, I made statements which I still would make, we could compute the heating on a wind tunnel model about as accurately as you could measure it in regions.
Not over the entire region.
Not over the separated region.
Not over conditions where the flow was turbulent.
But, in the laminar environment, we could compute it as well as you could measure it.
That is outstanding because now if I say I've got a program, I can tell you what the heating is, I can tell you all about the aerodynamics, I can tell you everything.
Good.
Run it on a wind tunnel test.
I will take the data, you run it on the wind tunnel test and we will see if we agree.
You've got a validation of numerical simulation in terms of the geometry in wind tunnel, in terms of the chemistry in shock tubes or ballistic facilities for that matter and certainly in terms of flight test.
And you've got Shuttle data and a lot of code validation for that Shuttle data.
Now, this is just from the heating standpoint.
I haven't talked very much yet about the thermal protection system or the structure and the rest of the things.
Let me say just a few things about configuration and then I will say a few things about the thermal protection system and then we will go to some charts that has some real information on it instead of just a hand-waving that I'm doing right now.
Configuration.
I mentioned dissipating this energy.
George Truhall [SP?]
was in charge of the thermal protection system on Apollo.
And I used to come to him and say, wow, we take care of 98% of that energy by putting it into the air.
We compress the air, heat the air.
You only have 2% to deal with, a small percentage.
And he used to come right back to me and say I do the same thing.
I get rid of 98% of it and only 2% gets to the structure.
And, in fact, that it is true.
By design, if you look at a meteor coming in, the surface will vaporize.
And, depending on the angle of momentum, if it rotates it will vaporize all the way around on the outside.
And some of it will come in.
In fact, Professor Fay wrote a wonderful paper about meteors, what become meteorites and what are meteors.
An excellent paper.
Put it in a real perspective.
Broad range, much broader than what I'm talking about here.
The way George was able to get rid of this heat was by re-radiating.
And this has been kind of unique to manned vehicles.
Fundamentally, you've got a vehicle coming in, you've got a surface with the capsules far exceeding the material capability.
The heat flux coming in here gets you to temperatures that are far above any material capability.
And so what happens is the material degrades and is called an ablator.
And the initial concept and application for ablators is, as the material vaporizes, it basically pushes the air away to reduce the heat.
And this is sort of what happens in meteors if you want.
So, you lose some of the initial material.
And that sample that Aaron passed around, that black material, well, it was a little thicker when it started than the sample that he has.
But you see this what we call char layer where the material is decomposed?
That generates a gas which at high pressures and high heating rates basically tends to absorb the energy that is coming in through the chemical reactions primarily, and also due to the blowing.
But a fair more effective way of getting rid of that heat, for us in Manned Spaceflight, at the higher altitudes was to re-radiate it.
How do you draw re-radiation?
I guess sort of a wavy line.
Radiation is always a wave, right?
And so first order, 98% of this energy goes into heating the air which flows around the vehicle.
The other 2% that gets to the vehicle, George re-radiated 98% away with a high-emissivity, high-temperature surface.
Now, in the ablator, that char, think of it as a charcoal briquette.
You start with some material.
It degrades and all sorts of chemical reactions go.
If you've got a good ablator for our application what happens is you're left with a residual char which is black and very high-emissivity.
It is pure carbon, if you want.
That is the challenge, is to get a carbon surface or a high-emissivity, high-temperature surface that can re-radiate.
Just to remind the class, what is the most efficient radiator?
Black-body, there you go.
Thank you.
High-emissivity, black-body radiation, get rid of all that energy before it ever has to diffuse through the tile or through the ablator to get to the structure.
That is fundamentally what we use to protect the vehicle.
Now, while I'm on that, the ablators were developed for capsules.
And we kept getting lower and lower density because we wanted to minimize the diffusion of energy from the surface.
There needed to be a substant enough surface to hang together.
I mean if you just have charcoal and blow on it, it's not going to stay there.
With the early capsule, with the Apollo, we had a fiberglass structure in there that basically retarded the ablator from flowing away.
Now, everybody tells us we were twice too heavy on the ablator on a capsule.
And I will explain that in a bit.
It's a systems engineering problem, we got away from it on Shuttle, so pay attention when I get to that problem.
In any event, we were a factor too high over most of the vehicle but not in this region.
On that region, we were right on.
Not by knowing what we were doing, just by luck.
When you say right on, what you're saying is that the char layer went essentially all the way into -- We hit our temperature requirement on the structure.
If I blow that up a little bit, I've got a surface here of ablator which is [paralysizing?]
[he actually said that!
there's no reference in aerothermodynamics that refers to that term] and charing.
And I've got a flow over the vehicle.
I've got one heck of a pressure grid.
And I just described this char.
It's a nice porous carbon like a charcoal briquette.
Ever blow on a charcoal briquette?
You can blow your air right through it and it flames up.
That's exactly what happened here.
We had flow through.
The honeycomb wasn't as a high a temperature as the carbon.
That was just there to hold it as the system degraded.
So, the combination of the two-dimensional flow which we really hadn't simulated on the ground, all we could do, in the arc jet, was basically little pucks to test the ablator.
You're dealing with an awful lot of energy, and if you spread that energy over a large surface the enthalpy basically goes down.
The heating on the surface goes down.
We were testing six inch specimens characteristically.
We really didn't get the flow through.
In fact, we eliminated the flow through because we had one-dimensional models and we wanted to understand the process of one-dimensional.
But, in flight, fortunately we were a factor or two over just in that region.
Over this region, and I will show you a chart, we were way over.
And, again, I will try to address that.
And, again, the Shuttle has really helped us in understanding the leeside flow.
Configuration.
Everybody has got their own requirements for configurations.
From a heating standpoint, I want to put all that energy into the air so I want a maximum drag configuration.
I just want a flat plat normal to the flow.
Well, that's wonderful.
It's high drag but it's not stable at all.
Let's go to a sphere.
Now, the center pressure, no matter where the pressure is on the vehicle, no matter what the pressure is, it goes right to the center.
And that is probably also about the center of gravity if I flew a sphere.
Certainly if it was solid, but if I build a spherical spacecraft it would be neutrally stable.
Well, that would be great.
I could spin it and distribute the heating over the entire thing.
As they enter, that happens to some meteors.
Well, it's not too comfortable and I don't have control and I also generate a little left.
There are all kinds of problems with that.
One other problem, if I use just a section of a flat plate, if I look at the heating distribution, if I plot the heat transfer in this direction from ground test, not from flight, from ground test, you have some level of heating.
But then, when you get to the corner, it really goes up because of pressure gradient.
The same thing that we experienced on Apollo in terms of the ablator.
So, you want to give it some curvature, not only from the standpoint of stability but also from the standpoint of the heating distribution.
And, indeed, if you look at the Apollo at zero angle of attack it pretty much has almost a uniform heating distribution.
That is very efficient.
I can design my thermal protection system to be so thick and I manufacture it, produce it over the entire surface exactly the same.
That is wonderful, except I still have the corner problem.
And so I round the corners a bit.
This is only from an aerothermodynamic standpoint now.
And so I don't have this corner edge heating problem nearly as much.
Well, that's fine, except a zero L/D vehicle gets kind of high Gs coming in and you don't have much control.
And I will address a little bit of that, and you've probably already heard about the flight mechanics control.
In any event, I need some L/D so I got to angle-of-attack.
In order to go to angle-of-attack, I have to take the center pressure, which is basically the point about which the aerodynamic forces, if I put a string and pulled on that, that's how the aerodynamic forces are acting on that.
And I want the center of gravity ahead of that.
And now I've got a nice stable configuration.
Not too stable because the control people want to do things with it but stable from the standpoint of not having to fire RCS jets or have control surfaces and all that type of thing.
That is how we designed Apollo.
It worked very well.
And, in fact, if you look at the Shuttle at angle-of-attack with some liberties -- Son of a gun.
In two-dimensions.
In three-dimensions it is significantly different.
I think I'm about ready to go to the charts now.
OK. Why don't we take a two-minute break.
We usually break about 10:00.
Great.
And I will get the charts set up.
Good.
Quick stretch.
Turn around.
I didn't ask if there were any questions.
I expected everybody to just raise their hands and start asking questions.
But, since there are no 8:00 classes, this is the first class of the day, maybe that's why we're not getting any questions.
That was my hand-waving.
I am going to get into specifics here after the break.
I have a series of charts.
Most of these came from a technical conference that Professor Cohen had after the Shuttle test flights.
We had five test flights, just for information.
We had 17 flights in Mercury before we put a man in the vehicle.
On Apollo, we never put a man on the vehicle or did anything before we had a test of the systems in the vehicle, before we'd put a man in it, with one exception, and that was the actual Lunar landing.
We couldn't really simulate that very well.
We couldn't test that.
On the Shuttle, we did as much testing as we could, but the design constraints forced us to go with men in the system initially.
And that was the least riskiest thing to do considering everything.
That's a whole different subject.
All right.
This first chart is altitude in thousands of feet and velocity in feet per second.
What I've shown here is, first of all, this is one atmosphere total pressure.
This is a tenth of an atmosphere total pressure.
And first I would like to focus on the Apollo orbital return, and I am going to show data from actually one of these flights.
These bands are to show the flight regime.
And this is the Lunar return coming back 36,000 feet per second and then essentially going through an orbital entry.
You can see there is an order of magnitude difference here.
I'm sorry.
Here is the Shuttle.
This was the design coming from a polar orbit at 265,000 feet per second.
And these are the orbital flight tests, the five flight tests.
They are all laid together here, and you cannot really see much of a difference.
Now, this is the heating boundary.
If we went beyond that the tiles degrade.
Now, I mentioned three levels of aerothermodynamics.
The design level, well, I'll get into the design level a little bit later.
The first level was basically the Apollo technology.
That was we went to the wind tunnel, we correlated the distribution of heating on the Shuttle relative to a reference, and we used the one foot sphere as a reference.
And I gave that to the flight mechanics people and said do not violate these constraints.
So they developed the flight mechanics control and everything else to fly right along here.
Now, what isn't shown is we anticipated transition.
And I will show that in time histories.
There is another boundary for heating that actually comes along here.
These trajectories are very tight relative to having a re-usable thermal protection system and not having to completely refurbish the vehicle.
That was the target.
If we wanted a reusable system, we could turn around and fly again.
You've seen the ablator from Apollo and you've seen tile that really had some severe environment from the Shuttle.
But basically here we have a simple configuration.
Here we had a much more complex configuration.
We had capability to fly it and we had capability to avoid excessive heating.
It takes about 20 minutes to enter.
Ten minutes along this heating boundary where basically your heating rate comes up and you are on a plateau.
And you will see that in just a minute.
That contrasts the Apollo and the Shuttle.
And I mentioned on Apollo we did a lot of shock tube testing trying to get into this environment, about 33,000 feet per second for peak heating, when this thing comes down to also about max pressure on this chart anyway.
But Shuttle was significantly higher altitude, significantly more out of equilibrium, which is the bad news.
The good news is the radiation was not that significant.
In fact, when we calculated what the astronauts would see coming in -- On Apollo, when they were looking out the windows on the wake, it was extremely bright.
This is just in the wake which is orders of magnitude down from the radiation on the front side.
On the Shuttle, we suggested a 100 watt light bulb behind a table.
It worked beautifully in the simulator.
Significant change in environment from Apollo at 33,000 to 36,000 feet per second to orbital environment.
The heating basically increases.
In this direction, I don't show [UNINTELLIGIBLE PHRASE].
[question about mars, that i miss to catch due to the cough some other person enters] Mars return is like 45,000 feet per second.
The convective heating goes up as velocity cubed.
The radiation probably goes up by two orders of magnitude.
The radiation is very, very sensitive.
Some of the people at Ames did correlations with velocity and with temperature.
And there were numbers like a temperature to the eighth power, a temperature to the twelfth power because it is not limited by black-body over a lot of the spectrum.
And, as the temperature goes up, there are more and more radiated degrees of freedom.
It gets much closer to a black-body.
Radiation becomes dominant on a Mars return.
On Lunar return it is a problem.
However, if you look at the radiative heating, it is first order proportional to two-dimension.
Whereas, a convective heating is just the opposite, it is inversely proportional square root.
As you go up in dimension, your convective heating is less important, as we did with the Shuttle.
And I will try to address that.
So we don't really have an ablator type of material that would work for Mars return?
I think we could develop an ablator.
But we would have to do it in an environment that gets as close as we can to simulating the radiation.
And these days there is a lot more capability there than there was in Apollo days.
But that is a real challenge.
Yes.
I have a couple questions.
Sure.
[all/among the lines Why do they come backup] That was the basic flight mechanics that dissipate the energy.
And, actually, I will get into this a little bit more on the next chart and then go through an entry.
You wanted to not exceed the deceleration.
Actually, from an ablator standpoint, one of the most efficient ways to come in is hot and heavy.
Go to max heating rate but keep that time down.
You remember I mentioned the square root of time on the diffusion of the energy in.
You want to keep that time down.
And you're better off taking your lumps on heating rate.
Heat load is really what designs.
Did that answer your question?
[UNINTELLIGIBLE PHRASE] [i'm sorry enviroment noise plus air condition] This is the flight mechanics, and I'm not an expert on that, but basically this portion was designed to capture.
If you don't capture you're gone for another two weeks.
And so you didn't want to get beyond 20 Gs because nothing would take beyond 20 Gs obviously.
You didn't want to skip out.
This is a linear plot.
You're looking at an atmosphere and you're coming in at high speed.
I want to make sure you capture it.
Not too steep or you're blown apart, but if you miss you're gone for another two weeks.
First thing is capture.
Dissipate that energy.
Then we worry about getting down to a landing point.
In terms of corridor, they term that corridor, the Apollo vehicle had about a 27 mile corridor at 400,000 feet.
If you were too shallow then you would skip out.
If you were too steep in that corridor you would go in too deep and exceed the G level.
Coming back from the moon, 240,000 miles away, you basically had to a corridor about 27 miles in the earth's atmosphere.
And that was really what the guidance navigation system did for us.
And with the mid-course correction, of course you've got a lot of leverage, but it is a very tight corridor that you've got to hit.
Well, I can talk about it now.
No, I will wait until the next chart in terms of the systems engineering and everything.
When I was here, the computers were in the electrical engineering department and occupied an entire room.
Mechanical engineers like me, I mean we didn't understand all that stuff.
Generations.
Design and flight test environments, a lot of points I want to make here.
This is a log maximum heating rate and this is the maximum integrated heat load at design conditions.
Now, the numbers are not so important.
Apollo is triangles.
Shuttle is circles.
Filled is design.
Open is actual.
Let's start with the Apollo.
Our design is up here coming back from the moon.
Actually, there were two design points.
This was the maximum load which was the thickness of the ablator.
That is the weight.
We picked the material.
We got the best material we could.
Now the question is how thick do we make it?
And that's the weight.
Here is the design condition.
The heat load I mentioned before relative to the question.
We didn't have the trajectories of flight mechanics when we started the design.
We had to start to design the capsule, the ablator and everything else in parallel with the development of the computer capability and the flight mechanics and being able to actually fly this thing.
What happened?
We knew we couldn't exceed 20 Gs.
And so give me a trajectory that comes in and hits 20 Gs.
I've got to be able to handle that heating rate.
We also needed to capture.
If we skipped out and went another two weeks, that was kind of tough on the crew, so give me a trajectory that just stays in.
They are miles apart.
That point is up here on that chart.
For clarity and trying to illustrate other things, it is way up here in heating rate.
And there is sort of a range of entry of Apollo in terms of -- That's the maximum you could get.
All sorts of things in between.
This is an actual flight.
This is log paper.
A significant difference.
A factor of two.
Square root of that, 40% of that ablator that we didn't need is in the trajectory difference.
But now here is the systems engineering point.
When we started the Shuttle, we looked at Apollo and said how do we get this heat shield down?
Well, if you look at the aerospace industry or NASA you have specialists.
Everybody does their own little thing.
I am aerothermodynamicist.
George Truhall was the thermal protection system guy.
Tom Moser was a structures guy.
Then there was a materials guy.
All these different people, they all do their thing and work together as a team.
We're designing a system to go to the moon and come back.
Boy, I sure don't want to put too low a heating rate in.
I mean I don't want to be the cause of a failure, so I think the heating rate is going to be right here or whatever.
Well, I don't have that level of confidence, I've got some uncertainty, so I will put 10% or 20% at least in on my heating.
The ablator guy, he is testing on arc jets, and he does exactly the same thing.
The structures guy says, well, we want to do this and that.
Well, on the Shuttle, for example, our guideline was 100 missions, a structure that would take 100 cycles.
Well, if I don't exceed this temperature, you know, how well do I know that, what is the stress level, all the different variables, you know, this is what I expect but I better put a little pad in.
There is nothing wrong with that.
What was wrong is we didn't have communication with all these people.
Somebody gives me a trajectory and I calculate heat and I say, boy, this is what I think it's going to be.
And I'm going to show you some of that, in all honesty, which you won't see in journal articles.
But I better put a little bit of pad on that because I don't know this that well.
They compound.
We call it compound conservatism.
Everybody puts their 10% in and you get your factor of two.
Now, you just lost significant, if not half the payload on the Shuttle by doing that.
We did not do that on the Shuttle.
Now, let me very honest.
In the early days, when we recognized that, because we were getting beat on.
You don't need all that TPS.
When we realized where it really came from, we would go back to the management and say this is what you need to do.
You need a system where you have all this communication.
Well, that's going to be expensive.
We did it, but we did it informally.
We did it by communicating informally everybody understanding.
And we actually did a statistical assessment before we flew the Shuttle to give us confidence that things would work.
Extremely important.
I mean when you talk about systems engineering it is communication, different disciplines, different requirements, different everything.
Communication between people is very important.
All right.
Where was I?
Here is the orbital entry on Apollo, the two flight tests that we did, 201 and 202.
And, I'm sorry, I don't remember why we called them one, three, four.
This was a design, there's another design up here.
That was a conservative.
All right.
Here are the five OFT flights on Shuttle, heat rate and heat load, and there is the design.
We do not have a whole lot of margin.
But, as you saw from the previous chart, we can fly it.
We're obviously able to do it as long as the TPS and the structure are intact, obviously.
Any questions?
Yes.
How much does your ability to control the trajectory play into that?
I mean it seems like you'd be able to control the trajectory of the Shuttle a lot more precise than you would Apollo.
Yes.
Absolutely.
You've got a body flap on there, in particular, to trim angle-of-attack.
We also have RCS engines which we'd use if we would have to.
And, in some cases, we have to.
And you've got aileron settings which primarily are used later.
But, yes, crucial.
In terms of getting this level of precision you need control.
But isn't the most important factor the flight path angle, the angle between the velocity vector and the local horizontal at [400,000?]
feet?
Initially yes.
On Shuttle you could modulate that.
Yes.
And, actually, you see some of these wiggles, you know, we have highs and lows.
We have hurricanes and high pressure areas, but the atmosphere is an exponential decay.
And any waves, any ripples, by the time you get to the tail end of the whip it can get very significant.
And so there are density variations that you really don't know about until you hit them.
And being able to control is sort of crucial there.
Coming back from the moon right on, that initial angle is crucial, and the de-orbit from orbit obviously the same thing.
Any other questions on this before we go to the next chart?
OK. All right.
I mentioned three levels of aerothermodynamic methodology.
One is to correlate in the wind tunnel and relate everything to reference heating or stagnation point heating, which is what we did on the capsules which are good blunt vehicles.
The real technology challenge on the Shuttle was the geometry.
It was a lot more complicated than just the shock and a stagnation point and flow around a blunt vehicle.
You name it, you've got it, in terms of flow on this thing.
Now, the way this was modeled in terms of -- When we started, we didn't have computational fluid dynamics.
The design methodology that was used to actually design the system was to model the flow.
Obviously, up front here looks kind of like a sphere.
And so I calculate the heating on a sphere.
I go into a wind tunnel and look at the actual data and relate that and say, well, that's kind of like a sphere of two foot radius.
So that's my heating there.
I look at flow down the center line -- And, I'm sorry, this wedge isn't supposed to be wedge or a flat platted angle-of-attack.
This flat surface down here looks kind of like a wedge.
Well, I can calculate [a boundary?]
on a wedge given the pressure which Newtonian would work fine.
Certainly, in the blunt regions and on a wedge, I can do that flow.
I can do a swept cylinder for the leading edge and I can do a comb for the boundary layer.
I can take diagnostics in a wind tunnel, look at the boundary layer path and say, boy, this thing is spreading kind of like a cone.
This is the design methodology.
These are all one-dimensional flows for the boundary layer.
These are geometric flow models where the boundary layer is basically one-dimensional.
And even though, for example, on a cone, the boundary layer is spreading, it's still a one-dimensional flow.
So I calculated in a wind tunnel what the heating would be at this particular condition.
I calculate, for example, flat plate.
This heating rate has a function of distance from the nose to the tail if you want.
This is what I calculate, here is what I measure, a little factor in there.
But I assume whatever I don't know in the wind tunnel, I don't know in flight, too.
But I take this analysis which can include the chemistry, all the nice things that go on in flight.
So I calibrate it to wind tunnel which is basically an empirical flow field now and I take that to flight.
It works pretty well.
It really does.
Better than my normal shock Reynolds number that I gave to the trajectory guys.
Actually, there are areas where there is significant disagreement.
But mostly they are pretty consistent because they're reflecting the basic diffusion of the boundary layer and the basic physics that I tried to talk about in the beginning here.
It is very important to normalize what you're doing to something fundamental.
If you cannot do it on the back of the envelope, have a question about it.
If you cannot back it up with the back of the envelope, have a question about it.
So this is the design methodology.
I mentioned the Apollo methodology which was used for quick numbers and for trajectories.
The third level is a computational fluid dynamics.
I'm only going to show results from the technology that we had at the time we flew the Shuttle.
Since then computers have done so much, we can do so much more, but I want to compare what we expected and what we actually got with the technology we had at the time.
Boundary layer transition.
I talked about the boundary between the turbulent and laminar heating.
Characteristically, and in the Shuttle level, your heating goes up by about a factor of three.
If you go to higher Reynolds numbers it can get significantly higher, so you want to avoid that factor of three.
Our initial estimate on the effect of roughness, I have to say, was not as good as it should have been, but the technology wasn't really there.
This is the logic.
And I won't go through in intimate detail.
Just to point out the complexity to recognize the level of effort that goes into getting a basic database to design vehicles.
This is all in that document in that conference report.
And there are two copies, I think.
I brought a copy, and I think there is a copy in the library that Dr. Hoffman has on reserve, but this is all documented in there.
Yes.
And I won't spend much time other than to say we first looked at smooth body transition.
This is heating rate versus distance.
Smooth body.
And then we put roughness in.
We actually went to cryogenic models with simulated tiles to get that boundary layer to suck down to be a little better simulation of flight.
The alternative was to put bowling balls on the surface of the thing to try to trip the flow, and that didn't have anything to do with physical reality.
So a lot of work there.
We related and took the simulations for the transition and came up with an effective roughness relative to smooth body transition.
Again, I don't claim to understand turbulence, I certainly don't claim to understand transition and certainly on a complex configuration.
So then we correlated that and we had predictions.
And you will see some of that in a few minutes.
On the tile problem on the last mission when those little gap fillers came out, do you think that it tripped the boundary layer?
No, I don't.
On the other hand, some of the people that had been correlating the various missions felt that we would.
And the problem is it was the first time we had a picture.
Yeah, I understand.
We had tile gaps after we landed and we said oh, gee, this is the relationship.
There was a lot of correlation, but I'm not sure it was that valid.
I really don't think so.
Now, this is the surface catalysis.
I haven't really talked about that.
I talked about the gas going from molecular to dissociated, ionized to weekly ionized.
When you get back to the surface conditions, you're back to a molecule again.
Now, I am simplifying things here.
But I think the way I look at it is you start out with a molecule, you blast it apart, you have atoms, you get back to the surface of the vehicle.
And at equilibrium now you're back to a microstate.
Yes.
In the whole design process, was the shape of the wings and the underbelly designed first and then you calculated properties?
Or, did you have to go back and say no, this redesign is impossible?
Because of the heating do it this way.
Was there a [cyclic process?
(check)] or was it one way?
It was primarily one way.
Here is the aerodynamic requirement in order to come in.
The biggest exception of that was on boundary layer transition.
That classic if the structures guys design an airplane or if the electronics guys design an airplane or the aerodynamic guys, they all come out to be different airplanes.
The prime thing was the aerodynamics, to be able to control it and bring it in.
We did alter it relative to where the thermal protection system was.
The other thing we did is don't fool with Mother Nature.
The structures guys like to make things nice and flat or cylindrical.
And the flow is a continuous radius curvature type phenomenon.
And that difference is very, very important, particularly for boundary layer transition.
So we did fair the geometry to keep the boundary layer transition essentially two orders of magnitude lower than it might have been.
But overall I would say predominantly it is aerodynamics.
And we tried to calculate the heating to the configuration that we had.
[UNINTELLIGIBLE PHRASE] [would you change?
< 3 or 4 of this type of question ] The only thing I would have preferred to have done is fly higher angle-of-attack which could have reconfigured the thermal protection system.
The tiles are a very efficient system.
They are fragile but are very efficient from [an entry?]
standpoint.
The carbon nose and leading edge are heavy and they don't insulate worth a darn.
I mean that's a layer of carbon.
When it gets hot it radiates out, but it also radiates in.
And so you have to have an insulation behind that.
And, in fact, the way we're able to reconcile the environment with the temperature capability of the carbon is it's sort of like an oven.
And it's easier to illustrate on a leading edge.
Just picture a two-dimensional air foil at angle-of-attack.
And a carbon section, if you want, covers from here to here.
Now, if I look at the heating distribution, here the heating is quite high, still high over here, still high over here, still high over here.
Boy, it drops off very quickly over here.
Well, if I can radiate this energy over here, this basically becomes a uniform temperature oven first order.
And insulation is back here.
I don't know how you illustrate insulation.
Insulation is back here to keep the lower temperature structure from getting too hot.
I don't think a lot of people realize that, but behind the carbon-carbon, on the leading edge of the wing and the nose, there are actually tiles in there just like on the outside of the rest of the Shuttle.
So this is sort of a staged thermal protection system.
And if we didn't do that, if we put the tiles right up here then this temperature could exceed the carbon capability.
By flying at high angle of attack, I could reduce the amount of carbon.
What was the problem going to a higher angle of attack?
You said there was a conflict.
Yeah.
The problem was one of the requirements was long cross-range, and you don't get that at high angle of attack.
I mean high angle of attack just comes in ballistically, if you want.
You roll around the velocity vector so you get a little L/D.
But, if you want to go range, you need more lift so you need to drop your angle of attack.
A very significant design parameter for Shuttle.
And if you recall the trajectories, it was primarily needed from a polar orbit where you're trying to get to a particular runway.
You don't have as much capability coming from polar orbit as you do from equatorial or lower inclinational orbits.
You remember we were talking about energy management, and I talked about how if you end up low on energy you actually have to decrease your angle-of-attack even more and no S turns or anything, just straight in.
But there is a thermal boundary.
I don't remember whether it was 38 degrees or 37, but 40 degrees was nominal.
And you really didn't have a whole lot to play with.
If you drop your angle-of-attack in order to increase your lift so that you can stretch your trajectory to make the runway, at some point you're going to violate thermal constraints and start melting your thermal protection.
There were years of concepts and vehicles that were developed and flown, test vehicles with different emphasis.
And there were some aerothermodynamic vehicles nice and smooth.
I mean it just looked beautiful.
It looked like something an architect would come up with, if you want.
And, from a heating standpoint, they worked well.
But the structures guys, it was very heavy from a structures standpoint to get all these compound curvatures, this, that and the other thing.
There were also vehicles that were designed specifically from a structures standpoint.
And I'm not knocking the structure people.
I do a lot of structure work, too.
But there was one that had discontinuous two flat surfaces to be able to handle all kinds of good stuff.
And that was a terrible configuration from an aerothermodynamic standpoint.
So there actually was a heritage of all kinds of attempts.
The big challenge was how do you land some of these things?
A nice hypersonic aerothermodynamic configuration and aerodynamic configuration, OK, now you try to put it down the runway and it is hot.
A lot of test pilots had some problems with some of these vehicles.
There are all kinds of heritage of people pursuing this, that and the other thing.
And one of them was aerothermodynamic design.
And it didn't have these big wings that we have on Shuttle, but it was hot as can be coming in.
I think the Shuttle frankly, I mean I don't look at it just from an aerothermodynamic standpoint.
If I look at it overall, we needed all the lift we could get on landing.
And, as it was, we had to build a pretty special runway to do that or go to the dessert.
So, from an aerodynamic standpoint, we would like a lot higher lift.
In fact, that is a nice area for innovation.
Some of the earlier concepts for vehicles, you're probably with familiar with, capsules with rotogyros on them that power up as you come down.
Boosters, if you wanted a cylindrical configuration with a straight wing on it that swings out.
Now you've got to lift an airplane when it comes in to land, but there are problems with all of them.
In that case, the weight estimate for all the hardware and all that kind of good stuff was excessive.
Anyway, that is a real challenge for design.
I understand you folks are going to come up with a better design, some real innovative thoughts in terms of how to marry all these different requirements.
In my opinion, the mindset for hypersonic vehicles was [UNINTELLIGIBLE] [fibercups] have straight wings.
Supersonics like this and, boy, hypersonic ought to be like this.
I mean that's just kind of the mindset.
And it kind of works.
But an entry vehicle is a whole different vehicle.
Capsules work fine for their requirements.
Shuttle has been a fantastic marriage of different requirements.
The challenge today, and I think the reason for the class, is how about some new and better ideas.
It's not going to be easy but there's a big need.
OK.
Surface catalysis.
First order, start with molecules in a free strain.
Atoms.
Some ionized.
And then I get to a surface back here at equilibrium, I am back to molecules again.
They're pretty hot but they are still molecules.
Well, Professor Fay did a wonderful job in looking at a stagnation point on a sphere and saying, well, we've got limits.
If we're at equilibrium, that is the chemistry is fast enough that you're always at equilibrium, all the way through to the boundary layer, you get this amount of heating.
And that amount of heating included not just the conduction but the chemistry changes going from dissociated atoms, if you want, and putting that energy back into translation and rotation into molecules.
Or the other limit is completely out of equilibrium where all you get is the translational energy.
And what happens there, however, is if the surface is catalytic -- That is I have atoms coming into the surface and they are, if you want for discussion, absorbed on the surface.
And then another atom comes in, recombines and forms a molecule that releases that energy.
That is a catalytic surface.
So you have sort of two limits, a completely catalytic surface and a non-catalytic surface in a non-equilibrium environment.
And that is very significant on Shuttle, and you will see that in some data I am going to show here in a short while.
This is just the concept which, again, as Aaron mentioned is in the report.
We had to go to arc jets and spectroscopic diagnostics and flow models of what is going on in the arc jet to understand the surface catalysis of tiles and carbon.
We had to model the flow and model the chemistry to come up with an efficiency.
Then we go to flight with similar analysis and predict, and I will show some predictions of this.
This is a significant phenomenon at the high altitudes, low density for the Shuttle.
Any questions on surface catalysis?
I always found it amazing that you don't think of designing a space vehicle that you have to worry about chemistry.
There are lots of things you have to worry about, but chemistry I had never thought about.
But when I first heard about the surface catalysis problem, there are just so many things you have to take into account.
This is the flow field.
At the time of this conference and the OFT flights, again, the design was the methodology that I explained where you use simple configurations, model it and extrapolate the flight.
This is sort of the level of the technology, and this is at 20% of the vehicle length.
If the vehicle length is 1.0, it is 0.2, 0.4, 0.5 cross-sections.
And it's just the cross flow speed contours.
The only point I'm trying to make here is this is kind of the level of CFD that we had at the time.
Since then, boy, we've gone gangbusters.
But this kind of the level.
We did not have the finite rate chemistry in this.
That is a lot more computer than we could handle.
At this point, we were trying to develop algorithms and grids.
And, as I said, we could, in certain regions of the body, compute the heat transfer as accurately as we could measure it in the wind tunnel.
But that's not at flight.
That is just sort of a picture of where we were, and that is where I'm focusing in terms of comparing what we expected from what we actually got.
Now I'm going to show results in terms of temperature as a function of time.
This is the entry time from about 400 to 1,000 seconds.
You notice this plateau.
You're talking about 20 minutes entry and ten minutes of flying it so you don't exceed the thermal protection system.
I am going to show three locations.
Right up here at the nose, just behind the carbon, mid body and aft body.
Now, the flow field technology, we had a heck of a time getting from our starting solution, subsonic flow, which used one particular algorithm [to match?]
supersonically down the vehicle.
We succeeded in doing that prior to flying.
And we were able to compute up to the point where the shocks from the wing intersect the shock from the fuselage.
And then we had another subsonic region and we weren't able to handle it.
Now we can do that.
But at the time we flew, we could only compute up to about here.
Now, this is not what you'd see in a journal.
I will show you one you'd see in a journal here in the middle.
This is right behind the carbon.
Now, what we did is we computed at different times through the trajectory and then there were correlations to get the history.
This is JSC prediction right here.
Now, we didn't worry about exceeding Stanton number one.
That is we didn't worry about the heat transfer being higher than the energy flux to the vehicle up here because it doesn't, I mean it is, if you look at these numbers, we should have accounted for that, but it's not that important so don't worry about that portion of the curve.
Up here is very important.
Now, this is temperature.
And I just mentioned epsilon sigma T to the fourth.
That means the heat transfer is four times a bad.
We did a terrible job up here, and I will get back to that.
Now over here what happens is I've tried to stay laminar.
I come down.
And this was really a stretch.
We never got turbulence heating up on the nose.
Way beyond wind tunnel, way beyond our experience base, but the model just went up there.
So we really didn't expect that.
That was a pretty conservative boundary layer transition.
And then, son of a gun, the actual temperature is higher than turbulent even though it would appear to be laminar.
What is really going on here?
This is right behind a carbon.
The carbon is an oven.
As the service temperature of the tile drops, which drops very quickly because it re-radiates.
Not because it diffuses.
We control that.
It re-radiates so, as soon as the heating drops, the temperature drops.
I mean this is almost identical to reflecting what the heating is.
It is dropping like mad.
And, indeed, we're agreeing quite well.
We're kind of out of the real bad chemistry and surface catalysis which is part of the problem here.
And here, all of a sudden, the laminar is higher than the turbulent.
The carbon is this oven.
It doesn't cool down.
Even sitting on a runway it is as hot as can be.
If you think about it, it's got this oven.
It can only radiate out of surface and inside it is still radiating.
Just like when you turn your oven off at home, it takes a while for it to cool down.
The flow is going across the nose and actually heating the air.
At least changing the boundary layer profile so that the heating is higher.
How do I know that?
[Here is your tank?
yup].
Over here there can be a little bit of that effect also, and it takes a while to preheat the oven, if you want, if you're doing any cooking.
There is a little bit of that.
But, predominantly, this is the chemistry.
See, we come much closer together here in time.
This is predominantly the chemistry.
At this time, this non-equilibrium relaxation distance is about six inches.
Now, we're not into radiating gas but we're certainly not into equilibrium.
That is also a factor and that is not well-included, even today in my opinion, other than through correlations of the shock in what's going on here.
You won't see that on paper because we did a terrible job there.
We did not exceed design material requirements.
And I don't think I put anything in there about -- Well, when I get to the thermal protection system, I will go through the philosophy of having confidence to fly.
And, obviously, with our predictions, we would not exceed the top capabilities so we could fly.
But we were off here badly.
But at least you were off in the conservative direction.
Well, that's the way we tried to be.
This I will get into.
This is really what we expected, to the best of our understanding.
Yes, I was surprised at this also.
We did consider service catalysis, which I will show here after a while, but we did not really include the nonequilibrium flow in the inviscid, what I've been talking here about Apollo.
That was beyond our capability to do finite rate chemistry in the inviscid flow field.
Why don't we go to mid body now?
All right.
Here is something you might see in a journal.
Right where we wanted it.
And that's where we had the most confidence, too.
We got away from the subsonic, supersonic condition.
We were into a marching solution.
That is as good as we could do it those days.
And we knew that transition was going to be later than what we predicted because wind tunnels, you've got walls.
You've got all kinds of reflected noise.
And wind tunnels are notoriously conservative relative to boundary layer transition.
But, amazingly, the turbulent heating correlated quite well.
And that even works with normal shock Reynolds numbers.
I mean when you go back and look at all of the Shuttle data with a crude back of the envelope normal shock local heating to reference heating, including turbulence, it is amazing.
It really is.
First order physics seems to hang in there, in spite of all the complications of chemistry and all the kinds of things that can happen.
You have to be careful, but overall things usually work for you.
Again, that is academic there.
If we put in a constraint it would look a lot better.
That is good stuff, but we can do even better now.
And, as you can see, when I said we could do a wind tunnel as accurately as we could measure, in flight basically we could do the same thing.
Mid body.
Everything is just right for the CFD folks.
Now we'll go to the rear where we go past CFD and more to the wind tunnel correlations, and we're not doing as well.
Certainly on transition we knew we'd be very conservative.
This is ideally how we design the vehicle.
Fly to temperature capability of the tile.
And on this STS-3 that's what we were trying to do.
Now, if we exceeded this that's all right, but we don't want to exceed the material capability.
And we obviously don't want to exceed the structural capability.
Let's see.
I want to talk about surface catalysis.
Then we will get into the thermal protection system.
I've got one chart I am going to spend a lot of time on.
This is distance down the vehicle from the nose.
I mean it is linear distance.
There is 50%.
This is surface heat flux at one particular time in the entry.
The circles are flight data.
These are catalysis experiments.
This is a fully catalytic or equilibrium heat transfer.
This is a zero catalysis non-equilibrium, boundary layer non-equilibrium.
The reactions are not occurring in the boundary layer.
This is the flight data that opens circles.
I don't remember what the measurement problem here was.
And here is this point way up front you can see again.
Now, this is a viscous flow field where we could do the chemistry but we couldn't do the fluid mechanics real accurately.
This is still a mix and match.
Look at the difference between the measurement and the prediction.
The same thing as before only I eased it by showing temperature instead of heat flux.
The surface catalysis is a predominant factor.
Now, there is some rambling in the data here, depending on particular location, et cetera.
But our friends at Ames developed a catalytic coding, and they coded tiles.
And they said we're going to demonstrate service catalysis, so they put these tiles at these particular locations.
If the whole vehicle was like that, the heating would have been up here.
But since there was just the tile, the boundary layer comes along here and all of a sudden, boom, it gets hit with a different boundary condition.
We've got all those atoms and they see this catalytic surface, the heating goes way up and then relaxes.
This is the computation and there is the measurement.
Indeed, here is a demonstration that chemistry can be important in this flight regime.
If you come down in altitude where everything starts going to equilibrium, it's not that important.
You're going to get this because the chemistry is going to, the gas is going to react.
But where we are, up in altitude trying to maintain conditions so we can be reusable, that chemistry is significant.
Now, I think the next one is probably the thermal protection system.
No, one more measurement.
This is the leeside.
This is normal shock versus film heat transfer coefficient.
Basically heat transfer.
Log.
Log.
This is the heating shown here as a function of normal shock Reynolds number.
Now, remember I said we designed the trajectories so we went up here and spent 10 minutes and then came down as normal a normal shock Reynolds number.
And here are three trajectories with a heating on one location of the leeside.
By the way, Max Faget told me to burn a hole some place on the leeside.
He didn't want too much tile there.
The only thing I was able to do was we exceeded heating here because we didn't simulate the flow very well in the wind tunnel.
We had to put tiles there after first flight.
So I told him that was as close as we came.
In any event, this is heating on three trajectories through normal shock history.
It's sort of like time.
Three different vehicles.
This is to reference heat.
This is our old Apollo level technology.
It works quite well all the way through to peak heating, and then it starts to change as the Reynolds number picks up and the weight characteristic changes.
In ballistic facilities, if you have a vehicle traveling at high speed, eventually your weight goes transitional and turbulent.
As you increase the Reynolds number that moves forward.
As that moves forward, you get more mixing.
As you get more mixing the gas in the wake gets hotter.
And I believe that is fundamentally what's going on.
It is very repeatable.
We cannot get this in the wind tunnel.
Now we'll go to TPS.
No, one more on the leeside heating.
On the Shuttle we had thermal couples located here, there, as many places as the aerothermodynamics could put them, the TPS guys, but we couldn't put them anywhere.
This is the 201 vehicle and it's on its side.
I apologize because of my chartmanship.
The vehicle is coming in angle-of-attack on here.
This is the windward side where there was charring.
This is white paint down here.
And, I'm sorry, it is a terrible photograph.
Because of some of the protrusions we had on the first vehicle, we also got some charring on this side.
This is not charring.
This is some charring.
And it is blown up here in the way the chart was made.
One point.
This is the first entry heating.
And the leeside was extremely important.
How well did we do?
How hot is it?
We got the data and it was all over the place.
It was down where we thought it probably should be.
And it was like an order of magnitude higher.
We thought that data is no good.
Well, then we went back and realized what happened is when the control system fired jets, a lot more dynamic pressure than the airflow.
A lot cooler.
So, when I fired a jet, all of a sudden it is like putting a big wing out there.
Tremendous disturbance.
And, indeed, we were able to correlate firing RCS jets with when the heating was way up here.
And flow times you're talking milliseconds for reaction time back then when we weren't firing a jet.
Unbelievable.
We went to learning something.
Again, interaction of systems.
Now, finally, this is my one and only chart on the thermal protection system which is the real hardware stuff.
This is temperature versus time at a mid body location.
Now, this is from STS-1.
We didn't have data until this time on that flight, which is why I haven't shown much aerothermodynamic today.
We only had late time data.
This is the design trajectory coming in from polar orbit.
Our design condition was we do not exceed 350 degrees Fahrenheit if we want to have a hundred flight life with this aluminum.
Aluminum is great stuff because it is a good conductor but it is low temperature and it expands.
[Bond line?]
that means at the [hahahahahaha] bottom, just so that everybody understands.
Yes, thank you.
Those temperatures are much lower than you see on the surface of the tiles.
That's after the heat has diffused.
Right.
I think Tom probably talked about the SIP, the strain isolation pad.
I'm sure he did.
This is after you diffuse through the clouds and through the RTV bonds through the SIP.
This is the actual aluminum temperature.
Now I'm talking predictions.
This is what the vehicle was designed to experience.
Now, the design prediction was right here.
That was purposefully a little conservative.
The guy who is building it doesn't want to lose the vehicle.
He is not going to get paid, plus much bigger problems, but this is what we expected.
This is our best estimate.
And, again, if you recall, I showed you where we really knew where the heating was.
This is really a TPS test right now.
If you read the TPS section in that report, those guys with the heating, they were just way too hot.
Right here we were right on.
This is a purely TPS test, except it turns out it was my fault here.
In any event, here comes the data.
This is the prediction for this particular flight STS-1 right here.
This is what we predicted for STS-1.
This is what we would have predicted for design.
And this is the design technology prediction which has a little conservatism in it, obviously, and on purpose.
This is yes you can fly the vehicle with confidence.
We're flying this STS-1 no problem.
And we're pretty sure we can fly a design but things have to go right for us.
We did a pretty good job along here.
And this structure is intricate and everything.
The thermal modeling has to take into account all kinds of geometry, materials, et cetera.
And all of a sudden right here, boom, what happened?
I admit to overlooking this.
What happened is we opened the vents, we let some air in.
I mean you're up in a vacuum.
You do not want to let air in when it's hot.
The best way of burning a hole right through the vehicle is open a front and back door.
The absolutely worst thing that could happen to you.
You cannot radiate the energy.
You don't get rid of that 98%.
It was just like a blowtorch.
But, once you get down, if you don't do something, you've got this vacuum vehicle and you're getting atmospheric pressure coming up on you.
And all of a sudden you're going to be crushed.
At some point, OK, it's all right.
We open the door.
We open a vent.
This particular location, not all locations, could experience that.
And, not only that, but the air is expanding so it is chilled as it comes in from outside.
Still cold compared to what we've been working with.
I overlooked that.
Now, in fairness, there are a lot of areas where there is insulation, you don't get that cold air and so you cannot use it in many places.
But the point is, looking at the overall system and the kinds of things that could happen, you could take advantage of those in designs if you're cleaver, if you're innovative.
And so there was, not 100 degrees, a significant difference there.
You could take advantage of that if you design a vehicle that you pop it at the right time and get some cold air in there because that's what we're trying to control, this temperature, from a temperature standpoint.
The other thing in the Shuttle, which I'm sure Tom discussed, was the thermal stress.
I mean if the belly gets heated and the wings don't, the whole wings will pop up.
That is one big integrated problem which gave us a lot of difficulty.
Again, with the simulation and computational capability we have today, I think we can do a much better job of that.
And also just understanding the importance of it.
I think those are all my charts.
No, I'm sorry.
I have one more which is not a Shuttle chart.
We were looking at an experimental orbital transfer vehicle, a [low WOCDA?]
vehicle running from geosynchronous to low earth orbit, and then we use the Shuttle to go from low earth orbit to the ground.
We were trying to make that reusable because you don't want to go up and put an ablator on a vehicle in orbit on a space station.
And we thought we could do that.
Now, this is not coming back from the moon.
This is coming back from geosynchronous.
It's not quite as bad.
We had a design.
And, to prove the concept, we had a little model we were going to build and fly.
This came from a paper I gave discussing that.
Basically, this is the analysis and testing from research to an actual real hardware system, from the fundamental equations down to numerical simulation.
Back of the envelope or perspective fundamentals, that's kind of the fun part that I enjoy.
Then modeling as we do, for example, in the design approach on Apollo.
Correlations of data, numerical computation and then numerical simulation where you're trying to do as good as we understand with the equations.
Fundamental research.
The technology development which is where NASA's major emphasis is.
Component development of systems.
And finally subsystem, model and system testing.
You need all that stuff.
You would like to go right down the matrix here, have a good, firm foundation so you really understand what you're doing.
Therefore, you have confidence in doing it.
And, therefore, you develop capability.
There are no shortcuts.
Shuttle has done a fantastic job in both these areas all the way down to computational fluid dynamics.
It is not limited just to NASA Shuttle people or aerospace people.
And certainly in this area, in Apollo, but all the human space flight.
We've got a lot of experience.
That needs to be taken advantage of for our future systems.
It is not just it looks like this or it looks like that, it's going to use this system, it's going to use that system.
It's an overall integrated take advantage of the experience in what we've learned.
Yes.
I just have a question about the chart.
I'm just wondering if the inner section points on those lines correspond to a particular path or something like that?
I used this from the standpoint of we were trying to do a flight test model.
We were going to predict what happened to an aero braking vehicle coming in, and so I was focused on this and also on the numerical aspect.
Marrying those two, at this point, which was the reason for this flight test.
But I just thought that is applicable today in terms of where we're going in general.
I didn't get very many questions.
It must be because it's the first class of the day.
Well, you were going at a mile a minute.
I think we got a few in there.
But, yeah, the content of what we've been exposed to today has been tremendous.
We really want to thank you.
If they want to do some simple calculations, what reference would you give them?
Are there any simple calculations they could do?
[Faye and Rodell?
yes] has the boundary layer activity.
That is a crucial reference.
In the paper that they have, the Shuttle Technology Conference, I have a list of references that were to date in the various areas, whether it be service catalysis, TPS, you name it.
And those were the best references at the time.
And the individuals named, many of them have gone on and done much better work.
Plus, there are lots of younger people, too.
I think that would be the best source.
OK, Bob.
Thank you very much.
Thank you.
[APPLAUSE]
Walt Guy's biography is on the website.
I hope you've had a chance to look at it.
I will just say briefly that I guess, of all the lecturers coming, that Walt is the only one who is still actually working actively for NASA.
He is head of the Automation, Robotics and Simulation Division, division chief at the Johnson Space Center.
But that is not what he's talking about there.
He's talking about the job that he used to do for about 20 years which was head of environmental thermal life support and crew systems.
And, of course, the crew system includes all the space suits and the good stuff that I was fortunate enough to get to use.
So I've got a personal debt of thanks to Walt.
Aaron, I think you wanted to say a couple of things so I will pass the microphone to you.
Yes, I would like to say a few words about Walt.
We've worked together for many years.
And Walt Guy has the very unique capability of being not only an outstanding engineer, a technical person, but he also has a very unique capability of being a great or outstanding project manager, program manager.
No matter what task you gave Walt, he could deliver the product on performance, on schedule and on time.
He has that unique capability.
And he's done that from the Apollo Program through the Shuttle Program.
And I think you're really in for a treat to hear Walt talk about the environmental control systems.
He has a lot of information to offer you so, without further ado, let me turn it over to Walt.
Thank you, Aaron.
As Aaron said, we worked together over several programs over a long period of time.
I was noticing the picture.
I'm sure that wasn't put there for me.
But my current responsibilities are the robotic arm you see there.
That is a shuttle arm.
Of course there are station arms also.
Grapple fixture over here on the left.
And the little jet pack on the tumbling crewman there is part of my current responsibilities.
The 20 years that Jeff mentioned, I was responsible for the development of the Shuttle spacesuit.
So it was like it was meant for me anyway.
What I'd like to do is talk about the design.
I know in the syllabus it said environmental control systems.
So where did all the rest of the stuff come from?
If you go back in perspective, the Mercury, Gemini, Apollo Program had the classical thing that aerospace calls environmental control systems.
It was basically a cabin system.
It took care of the living quarters for the crew.
But in the Shuttle era the responsibility of that system expanded to include more than just the cabin and, thus, the very long hard to say name.
These are the subsystem elements of the environmental thermal control and life support system.
The first area I am going to -- These are the areas, the subsystems.
The first area is the cabin atmospheric revitalization system.
This is a long way of saying to take care of the atmosphere in the cabin.
Humans are pollutants and this is the system that takes care of that.
The cabin atmospheric pressure and composition, this is obviously to maintain the pressure and enough oxygen for breathing.
The third is the cabin thermal control, and that is the classical environmental control system.
But, for Shuttle, we've added some functions.
One is the water and waste management subsystem.
There are some hygiene functions required, there are some recharge of the spacesuit's backpack, management of the fuel cell water system, the fuel cells of the electrical power supply and provide a lot of extra water.
Also, the vehicle at this point, has a dedicated active thermal control system that takes care of all of the systems in the vehicle, not just the cabin systems.
And then last is the airlock support subsystem.
This is the EVA crewman's portal to space.
And so the environmental thermal control and life support system covers all these areas.
What I will do is go through each one of those.
The first is the cabin atmospheric revitalization system.
Its functions are CO2 and trace gas removal, humidity control, environmental cooling and atmospheric circulation and ventilation.
Again, I will talk each about these.
But I do want to mention that there is a uniqueness about space and that there is no convection so circulation is important.
Also, because of the fact that there is no convection, getting rid of the body heat, the latent heat requires a ventilation of the crew.
So each of these areas ended up as part of the system design.
This is an overview silhouette of the Orbiter.
I will use this throughout the presentation.
As you can see, part of the system we're talking about now, the atmospheric revitalization system is located upfront where the crew is.
It is beneath the floor.
And we will be talking about the systems in the remainder of the Orbiter as we go along.
This is a simplified schematic.
I would like to start at the cabin fans.
These are redundant.
Each fan will do the job.
The next in line is the CO2 scrubber followed by the cooler, the cabin heat exchanger.
There is a bypass around that for control of the amount of cooling that is needed.
As you are aware, the Shuttle can fly with few or many crewmen.
It turns out, as time has gone on, we fly pretty much a full compliment.
But the original design was to fly a small number of crewmen.
Also, this does give you temperature control.
The same heat exchanger provides the condensation of the water from the atmosphere.
Therefore, you get humidity control.
And then that is collected.
The gas continues on.
It is now chilled and is provided out to the flight deck and the middeck to the crew.
Then the gas from the cabin is then drawn back in through some flight deck avionics and back through the fans.
So this is the basic systems schematic.
I might pause here a moment to sort of explain the role of the government in the development of the Shuttle.
As you know, the prime was Rockwell.
It was their job to build the vehicle.
But the government philosophy, management philosophy was to have ownership of the technical aspects of the design.
So we had basically no excuse.
If it didn't work it wasn't because the contractor did something wrong.
We had to have ownership.
In fact, Aaron, as the project manager, would probably look to us first as the reason for a problem as opposed to Rockwell because we were the conscience of the program so we should have made sure they didn't do it wrong and provided whatever technical insight to Aaron to make sure he made the programmatic decisions to make sure things were not done wrong.
Very early in the Shuttle Program, I was responsible for an engineering organization.
And that engineering organization wanted to own the design of the environmental thermal control and life support system.
We acquired a chamber.
This is a near atmospheric chamber.
It is a square so it is obviously not shaped like the Orbiter cabin, but the volume was appropriate.
And it did allow a sealed environment for testing.
It would take a couple of PSI delta Ps.
That's about all it would take.
We developed a -- [COMMENT FROM HOFFMAN], No, that was a different one.
OK. We developed an environmental control system or an atmospheric revitalization control system and placed it inside to do early testing.
As you can see here, this is the fan package.
The CO2 scrubbers are here.
Heat exchange package over here.
And you can tell we're not current in terms of calendar-wise.
The CO2 and trace gas removal, the requirements were to maintain less than 7.6 millimeters of mercury.
This is certainly to provide a habitable environment for the crew.
This is higher than sea level but the medics agreed that that was a completely adequate level for CO2.
The CO2 absorption came from the humidified cabin gas so the system had to accommodate that as a feed stream.
The absorbent selected was lithium hydroxide.
The equation was a lithium hydroxide reacted with the CO2 to produce lithium carbonate, water and heat.
And this was a single use system.
It turns out that the lithium hydroxide was an old solution.
The previous spacecraft had used the same solution, but pretty much their solution was unique purchased production of lithium hydroxide.
When we entered the Shuttle era, we were hoping for a long duration.
And the idea was to use the commercial products to get away from special production.
And so we looked around.
And the Navy was producing a screened special production run for their submarines.
So there was a lot of use there.
And so the production was reasonably upo.
We decided to use the Navy grade.
And so we brought it in and put it in the test bed that I just showed you and did some tests, and it didn't work worth a darn.
You notice the loops here show that our use of the canister was much more rapid that we had hoped for.
The nice beautiful deep loops here are a process that we developed to screen the Navy LiOH so that we didn't have to create a new production capability but were able to test the LiOH that came off their production line and get a LiOH that was much more humidity tolerant.
The key here turned out to be not the chemical itself but its ability to produce good performance over a wide humidity range.
So we were able to screen that and make that a viable non-special production run LiOH.
The lithium hydroxide, I said, was an old solution.
All previous spacecraft have been pretty short mission.
And even the Shuttle was a short mission.
But the specification for Shuttle said that it should have a capability of going 28 days, and 28 days is not a short mission, at least not in Shuttle parlance.
The engineering part of the organization said, well, why don't we go with a regenerative system?
SkyLab had just been successfully flown with a regenerative system, something that wouldn't have all this expendables because 28 days worth of lithium hydroxide for seven crewmen fills everything up with lithium hydroxide.
There is no room for anything else.
So we began to look at a regenerative system that could be used.
Actually, it was proposed as part of the original Shuttle development.
But, at that point, the 28-day mission was considered a special case and the design case was the shorter missions.
And, in fact, there were even projected to be a lot of short missions.
The idea that the Shuttle was a space truck.
It would take stuff up, dump it out and come home.
And so the lithium hydroxide was a lighter system than a regenerative system.
The lithium hydroxide was placed into the vehicle as a basic design.
But, when the Space Lab era came along, we needed longer missions.
We flew about four or five times in this 13 to 16 day range, and those missions did require a lot of lithium hydroxide.
And so it was proposed to use an absorbent that was regenerable, and the one that was selected was a solid amine.
The absorbent is a polymerized ethyleneimine.
Its designation is RNH, and I don't really know why.
The absorption actually comes in, in two steps.
First there is a reaction with water to create the Freon.
And then that is then reactive with the CO2 to give you another free ion, but the CO2 is now captured.
The desorption then with the heat and vacuum gives you back your original amine.
So if you pack this in a bed, and what we used was an expanded metal foam heat exchanger that you could fill all the holes with the solid amine and put the passages adjacent to each other so that you could use the heat of absorption to do your desorption on your other side.
And all you would need to do is expose it to vacuum.
It was a very simple system.
It worked fine for the three or four missions.
I'm not sure how many missions it was used.
But it was an adjunct.
It was added in, put onto the floor, and then it was taken back out when it was no longer needed because it was just wasted load.
Yes.
For the lithium hydroxide, the product LiCO3, is that a solid form or a gas form?
No, it's a solid lithium carbonate.
And that goes in the canisters that you remove?
Right.
When you remove the canister it's a big lump.
OK. Finishing up the CO2 and trace removal, we go to the trace removal and use the simple system of activated charcoal.
And, in fact, it is packed into the same canister that the lithium hydroxide is packed into.
It is a single use.
There is a configuration, I believe, now operationally where the entire canister is activated charcoal so it can be used for cabin gas cleanup.
So, if you had an issue that you needed to clean the cabin up, you could do that with a canister just of charcoal.
The next area is the environmental cooling and humidity control.
I mentioned on the schematic that we have a cabin heat exchanger.
The front part of the heat exchanger basically is your reduced cabin gas temperature.
The backend of the heat exchanger is where the condensing occurs because we get the gas below the dew point and, therefore, the water drops out.
The heat exchanger has the characteristic that, as the water collects at the outlet of the heat exchanger, there were some holes drilled, actually, in the thin passages so the water can be sucked out.
And it has the nickname "slurper."
So it slurps the water out.
It takes about 2% of the flow.
So, as the cabin fan produces the flow through the heat exchanger, about 2% is siphoned off in the slurper.
And that carries the water to the centripetal water gas separator.
This is a device that basically spins the airstream.
It's an airstream with water and trained.
It spins the airstream.
The water goes to the outside.
It is collected in a trough and there is a pitot tube in the trough that the water impinges on, and we get about a 40 PSI delta head with that.
It's just a pump basically, a pitot pump.
The atmospheric circulation and ventilation is the next part of the system.
As you know, we're still in atmospheric revitalization.
There are the redundant fans that I mentioned.
Each fan will do the job.
There is also flight deck and mid deck duct system.
It looks very complicated but there are only several points I want to make here.
This is the atmospheric revitalization system I mentioned before.
As you see, it is below the floor.
This is the mid deck here.
This is the flight deck.
So it's below the floor of the mid deck.
The gas exits the heat exchanger.
It goes up.
There is some aft mid deck ventilation exhaust.
It goes on up to the flight deck and exhausts on the flight deck.
Also there is a section that goes on the forward side of the cabin through the forward control console and blows in the face of the crew.
The return is back through some avionic areas to provide cooling, and then back down to the system.
This is a complicated picture, but really it is a very simple flow system.
This becomes important later in our discussion because good mixing turned out to be a really important parameter.
Not just cooling.
OK.
The second.
Remember we had six?
This is the second element.
This is the atmospheric pressure and composition control.
The first job is the 14.7.
The Shuttle selected sea level, the idea being we could not have a lot of special designed equipment.
Shuttle was intended to be a less expensive spacecraft than the previous ones.
And so a sea level atmosphere gave us no questions about human survival and gave us a lot of opportunity for using more standard equipment and not specially designed equipment.
There was a requirement, though, for a PSIA total control, but this was only for an emergency de-orbit.
The problem statement is that you got a hole in your cabin.
Don't know how you got it but you have it.
And the idea is the system has to be able to sustain the crew for a de-orbit.
And I don't know who made up the number but 169 minutes is the number.
We have an emergency return, 169 minutes.
The system has to be able to accommodate that keeping the total pressure at or above 8 PSI.
So there has to be enough flow capability, enough consumables to do that.
The half-inch hole was a programmatic decision.
I cannot speak for where the number exactly came from, but it probably had to do with meteoroid debris kind of analysis that said what type of hole you could expect.
Anyway, that was the design for the system.
The next aspect or element of the system is the O2/N2 partial pressure control.
At the total pressure of 14.7, we wanted sea level which is about 3.2 PSI oxygen.
At the 8 PSI, we wanted it to be survivable but the oxygen could be less.
Obviously, people live in Denver so that works.
There are also crew breathing masks which can be used in conjunction with low oxygen partial pressure.
The cabin had to have both positive and negative relief.
It had to store the oxygen and nitrogen, and it had to provide pressurization for the water management system.
Remember earlier I talked about the more pervasive requirements?
The water and waste management system now is placing requirements on the atmospheric pressure and composition control.
This is a silhouette again where basically here you see the oxygen tank, which is 3300 PSI oxygen tank.
There are nitrogen tanks on both sides.
These are high pressure tanks all 3300 PSI.
The relief valves are shown here.
And there is an O2/N2 supply panel which has all of the valves and regulators and everything on it.
This system, however, though, gets most of its oxygen from the cryogenic tanks.
So that is only just shown as an inlet.
These tanks are basically part of the power control system.
Electrical power is hydrogen and oxygen fuel cell.
We use a lot less oxygen than they do so the storage was integrated.
And then we use the oxygen for most of the uses.
It comes from the main tanks.
This is a schematic.
I will start right there.
This is the cryogenic system.
This is the main cryogenic system.
This is not part of the tankage of the pressure and composition control system.
It goes through a restrictor.
This is stored in a super critical state so the delivery is somewhat limited.
And so this goes through a restrictor so that we don't decrease the pressure too fast in the cryogenic tank.
It comes on down through a regulator that regulates down to 100 PSI.
And that comes on down to two regulators.
One is the normal regulator, the 14.7, and the other is the 8 PSI regulator, both of which are available depending on which cabin mode you're in.
If you go over this way you will see the nitrogen.
The nitrogen goes through its own regulator, and it is regulated at 200 PSI.
And that's important because that is contrasted to the 100 here.
Next the nitrogen goes through an on/off valve, a solenoid that is connected to partial pressure sensors of oxygen.
And, if you're making up cabin gas, you have to make a choice as to whether you want nitrogen or oxygen.
And that choice is made by this system and a simple on/off valve.
There is no sophisticated control system.
This is a simple on/off valve.
If the cabin needs pressure but doesn't need oxygen then this valve is open and the 200 PSI basically backs down the oxygen system.
There is a check valve here, no oxygen flow because you get nitrogen.
At the point that you need oxygen, this valve closes, then there is no 200 PSI anymore.
Therefore, the 100 PSI provides oxygen to the regulators.
It is a very simple design.
Off the top of your head, what kind of failures did you have during qual test that really bothered you?
The regulators were produced by Carlton, and the regulators were derivatives of regulators they had been building for years.
So the regulator functions really were pretty good.
But the early problem we had was the pressure tolerance wandered and didn't have as good a pressure tolerance.
And they did make some mods to correct that.
But the more significant issues got into flow issues.
Their test capability was pretty limited.
And when you're talking about very high flow cases you have to do something with the gas to make the regulator think that it is in an operational environment.
And so we did a lot of testing, and I will show you that in just a second, for the high flow cases.
Let's see.
There is a redundant system.
It is not shown in this schematic but there are two of these, just like this.
This is the pressurization for the water and waste management system you see here.
I would say there is a crossover to the redundant system so that you can do mix and match.
If you have a problem in both systems, you can mix and match the capabilities.
Also, the oxygen system provides the emergency breathing system.
It also provides oxygen to the air lock for recharge of the backpack.
We will talk about that later.
And I believe that's it.
So, for normal pressure, we have 14.7.
And you have the automatic pressure regulator, basically on/off valve and the regulator.
The partial pressure is controlled by the 3.2 PSI which is the solenoid that controls that.
It is a very simple system.
The oxygen partial pressure does have another set point at the PSIA goes down to 2.45 PSI.
And I will show you that in a minute.
For cabin pressure relief, the problem statement really is why do you need relief?
Well, what if one of those lines breaks?
You've got 3300 PSI tankage.
It is all plumed together.
The crew, God forbid, step on a line and break it, and now you've got all this nitrogen rushing in.
You don't want the cabin to explode, right?
So you need a relief valve.
And the relief valve was set at 16.2.
And, again, that is one of the high flow cases that we ended up testing.
The system was designed with three relief valves.
Any tool will do the job.
That is just a redundancy philosophy.
Sometimes you will see two items where either one does the job.
In this case, this is a very critical issue so there is a double redundancy here.
For the negative pressure protection, there is an 8 PSI delta relief valve.
In fact, there are three of those.
You don't need the two to operate, but there are three.
This is for the return that I just showed you.
If you come back with a hole in the cabin, the cabin is at 8 PSI, you cannot reverse flow through that hole.
That is not going to work.
You don't want the cabin imploding on the crew so you've got to make up, as you come in the lower altitude, obviously the higher pressure, you've got to get that gas back in the cabin.
So that's what that is for.
OK.
This is the other pressure vessel that was used as a test site.
This was called the Environmental Test Article.
Rockwell gave it that name.
What they planned to do with this device, when they built it, was to do all the wiring runs and the plumbing runs in there.
It was going to be a mockup.
It turned out they built a rather beefy tank.
In fact, maybe they got one very inexpensively.
I'm not sure how they ended up with it, but they got it.
They did do all the secondary structure, so it had the right shape inside.
And so it turned out they didn't end up using it.
They went onto other methods.
And so it ended up leftover in the program.
We convinced Aaron that he ought to give it to us, and he did.
This is what it looks like in cutaway.
You see here all the secondary structure, it was in boxes but we put it all in.
Foamed it in so that the volume was right, so the cabin volume was essentially perfect.
It had the flight deck.
It had the mid deck.
It had the lower deck for the environmental control system.
But we also asked Aaron to give us the cert hardware, which he did.
So we had the cert hardware underneath the floor.
And the pressure control system we're talking about, actually, is on this back wall.
That is a picture of it.
What we did is faithfully run the lines.
In fact, we even use the same line material, same fittings, same everything.
So we faithfully duplicated inside this test article the volume and the geometry and the system.
All the ventilation ducts you saw, all those were in place.
We will come back to it later, but you'll notice on the back wall of this test facility there is an airlock hatch.
And later on I will tell you where that goes.
But this is the control panel and the nitrogen/oxygen system.
This is test data.
We did some testing to see how the system worked.
This was testing of basically the on/off oxygen control system.
Let me see if I can walk through this simply.
We're working total pressure here, oxygen partial pressure here.
During this first interval the oxygen is on.
What we're doing here is we're breathing it down.
And, when we hit the lower limit so that the regulator wants to do something, then the oxygen partial pressure starts up because the regulator comes on.
Now, the total pressure is maintained because that is what the regulator is supposed to do, maintain total pressure.
But as it maintains the total pressure it is doing it with oxygen and, therefore, the partial pressure is going up.
When it is satisfied, that is enough partial pressure of oxygen, then the solenoid valve opens and you see the nitrogen come on.
Now, you remember the nitrogen feed was at 300 PSI versus 200 PSI.
So there is a little jump.
That is just a mechanical artifact of the regulator.
If you change the inlet pressure it regulates slightly differently because it is a balance system.
You see a little jump there.
And then, during this period, the pressure is maintained.
But we're breathing off the oxygen because we've got the oxygen fenced off now.
And then when the pressure is satisfied and the solenoid goes back the other way then we again start the oxygen again.
So the system worked quite well.
If you notice the tolerance band spec-wise was pretty wide, the system operated on a very narrow brand so we were very pleased with that.
Normally that is important.
Not only because it shows you that you do have a good sensitive system, but you cannot -- Mechanical hardware, electrical hardware, either one never operates down the middle always.
There is drift.
You need a little tolerance in your band here to have a decent system design.
Yes.
Does this change a lot when you have the crew performing high metabolic activity?
The cabin is so large that it really doesn't know.
The cycles would be different, obviously, because you consume more oxygen, you would need more oxygen.
So the on/off cycle would favor longer oxygen cycles.
But, in the big picture, there is really not a lot of change.
What was this time period?
It looks like this was about three hours maybe.
That might go to two hours for a cycle.
I mean not really very much.
Sorry.
Does microgravity have much effect on circulation?
There is no circulation in microgravity.
It just lays there.
You push it or it doesn't go.
Diffusion is a driving force, so if you're really, really patient, even at zero G you can wait, but you cannot stay alive that way.
But, yes, you do have to have ventilation.
Still in the atmospheric pressure and composition control.
This is the emergency breathing equipment.
We used plug-in face masks.
There are some uses for the mask that require a carry on bottle to give portability, but the vehicle does have plug-in capability.
The masks were purge-type masks.
You're probably not familiar much with masks, but there is a mask called a rebreather which basically your lung power does all the work and you only purge enough to keep the CO2 down and you keep rebreathing.
And then the purge-type is every breath is a fresh breath, and you then exhale and the gas goes out in the cabin.
And that's the type that this is because the cabin needs the oxygen.
I mean it's not going anywhere bad.
It is OK that it's not completely spent.
It's good.
That's like a scuba mask, right?
I don't scuba.
Sorry.
But I think the answer is yes.
With scuba, you take in the air and then you blow it all out.
I mean there are scuba rebreathing systems, but the typical scuba gear is not.
Rather than raising your hand, if you make a noise of some kind.
Sorry.
I don't mean to ignore you.
The mask can be used in a contaminated environment and the mask can be used if the concentration of O2 is low to give you a better oxygen level.
This is the PSIA test that we did.
Actually, a series of tests.
But this is data from one.
As you see, we started up in the 14.7 range.
We simulated the hole.
The pressure immediately starts down.
The partial pressure of oxygen we let degrade to about the 2.2 range.
This is a different set point.
Remember I told you on the oxygen sensors we have two set points?
This is the second set point.
And so we have the lower oxygen partial pressure.
The system came down.
The PSIA regulator, which is always online, captured the pressure decay, caught it, and the solenoid valve popped back and forth to keep the oxygen OK because we're losing now.
We're going out this half-inch hole.
And the system worked just fine.
This did have the crew on the masks because that is livable, you can live there, but the mask is a better environment.
This is a mask case.
This was a requirement that came on later.
The design is the design now.
The vehicle exists so we have to now operate the vehicle with accommodations for operationally what we want to do with it.
And a new requirement developed and was imposed on the vehicle.
And that requirement had to do with the space suit for the EVA operation.
The Shuttle spacesuits are basically 100% O2.
And the classical way of going EVA from a sea level environment was a four hour prebreathe.
Actually, as much as six hours depending on how conservative the medics were at the time, that number varied around some, to prevent bends.
But it was determined that if you accommodized at 9 PSI for about 12 hours then you only required a really short prebreathe.
And the suit-up time, which gave you 30 to 40 minutes, actually gave you sort of a free tune-up to your prebreathe as part of your suit-up operation.
And so that was viewed as a much better way to do EVAs than the previous way.
The masks are useful but are not very comfortable.
So, if you have to keep them on for a long period of time, they're not very comfortable at all.
Also the medics were very concerned about mask leakage, that if there was any nitrogen inflow that that could break the prebreathe.
But our system isn't designed for 9.
We have a 14.7 and an eight, but we don't have a 9, so we needed a manual procedure.
But before that could be accepted there were some issues that had to be evaluated.
And we had to do testing on each of these issues.
The first is the flow rate acceptability at 9 PSI.
Obviously, the less dense gas you're not going to get the same mask flow rate.
And we had to make sure that that ventilation was going to be adequate.
And testing proved that it was.
The next is a thermal acceptability.
There are really two aspects to this.
One is the fan itself is cooled by the environment, so if the environment doesn't provide as much cooling to fan itself it could overheat.
Also, the electronic equipment, some of which is air cooled, the less dense gas could be a problem there.
It turned out that could be controlled with load control.
That could off load power and make that acceptable.
The CO2 performance at 9 PSI, we needed to make sure that was OK.
It turned out that, of course, the 9 PSI case, we don't care about the lithium hydroxide.
The purge flow is so great there is no CO2 problem.
But at 9 PSI for 12 hours you have to get performance out of the lithium hydroxide.
So we checked that and that worked out fine.
And the last item was the ventilation adequacy.
We do have a press and depress.
You've got to come down to 9 and you've got to go back.
And both of those adjust the gas mixture.
And so we wanted to prove that that was OK. And so we went to the facility that we were talking about and ran this profile.
Now, this profile was created by our-- yes?
[AUDIENCE QUESTION] Well, that's what I just said.
Yes, it does.
The air-cooled equipment is going to run hot.
If you want to cool that equipment, it will become a little clearer later, but the equipment is in equipment bays.
And so, with the less dense gas, you need to have some of that equipment turned off so that the cooling that is available is adequate.
So you do load control, get rid of some of the electronics.
And, therefore, the air that's left, the atmosphere that is left can cool the equipment.
And then when do you increase the pressure back to normal after the EVA?
After the 12 hours.
No, actually it's before the EVA.
Well, I'm not sure what they do now.
When you're all finished with your EVAs then you bring the cabin back.
Because, typically, you do multiple EVAs.
And so, from one day to the next, you just keep the cabin at, actually, we use 10.2 now rather than 9, but I think you have to certify it for 9, right?
This was an early certification for 9.
Yes.
But wouldn't it have been easier to simply design the whole system for 9 from the beginning if you had [OVERLAPPING VOICES]?
Well, this was not a Shuttle requirement.
This was an operational requirement that occurred after the Shuttle specifications were set.
You're correct.
If we knew it ahead of time it would have been much easier to do it all for 9 PSIs, exactly.
It would have just been a third condition.
But, as Jeff pointed out, we changed our mind.
The first was 9 and now we're at 10.2, so we're not consistent.
As we learn, we change our mind.
And the 10.2 was selected later.
Our mission operations people that do the crew time lining and all, they created this procedure.
And so we tested it.
This has a simple depress cycle.
You come from 14.7.
You hesitate at about 11 and replenish the oxygen.
You do that twice.
You then do manual control down at the 9 psi level.
You notice the oxygen partial pressure here is always maintained by these adjustments, and also during the manual period you select what you want.
And then when you come back, the procedure was to increase the nitrogen and then finish the top off with oxygen.
That was thought to be completely acceptable because the oxygen level, at this point, is completely viable.
And so doing the nitrogen first and topping it off with oxygen was thought to be an acceptable way to operate.
So we did the test and this is what happened.
We were all nice and comfortable with our oxygen partial pressure.
And you see the area next to where we were repressing.
We depleted the oxygen substantially.
And the mid deck and even the flight deck, we depleted the oxygen.
And I've only shown you half the data.
If you look at the rest of the data, you see where the oxygen went.
These equipment bays that I mentioned earlier, the avionics bays and other equipment bays are basically isolated volumes from the cabin.
So, as the pressure comes up, what you're doing is you're pressurizing these areas with cabin gas, which is oxygen and nitrogen, you're pressurizing these areas with nitrogen only.
Basically, you are depleting the partial pressure here.
You're increasing the partial pressure here.
You remember partial pressure.
You just count the molecules.
That's all you have to do.
Basically, we're putting more oxygen molecules here and we're taking them out of these areas.
And, therefore, they are going down.
This is just a picture of one of the bays.
You see all the empty space around the equipment.
That is where the gas is.
We came up with an alternate procedure which is to simultaneously put the oxygen and nitrogen in together.
And, of course, there is the 9 back to 14.7.
And you notice that solved the problem.
There is still some dip at the supply area because the mixture is lean.
We need from 9 to 14.7, 5.7 PSI total pressure increase.
But we only need 2.5 up to 3.2 oxygen.
This is a mixture, but it's a lean mixture.
So there is still some slight loss, but the only area that came under an issue is near the inlet where the O2/N2 supply panel is.
And, of course, you can just avoid that area.
Well, I should say that that area happens to be right around the bathroom.
And so, before we repressurized the Shuttle, the commander would say anybody who has to use the bathroom use it now because for about ten, fifteen minutes, while we're depressurizing, it's off limits.
OK. Next item.
This is the water and waste management area.
This is one of the six items.
We have potable and waste water inventory management.
We're continually producing potable water, as you know, and waste water.
We have to dump them, we have to use them, so that is part of the requirement of this system.
We have to store water for drinking, food prep and waste water storage.
It has to be dumped to space.
Commode and urinal for human waste collection.
And we provide water for an evaporative heat sink, which we will talk about later, called the flash evaporator.
Yes.
This is an outstanding example of systems engineers, this total system that Walt is talking about.
We can really see the interaction between all the systems and the spacecraft.
This really is a systems engineering problem.
Potable water, we take all the fuel cell byproduct water and use it.
So it is basically in a fluent to the power system.
We do have a launch storage capability for water and we do provide sterilization.
The sterilization is iodine.
We started out designing a silver ion system.
It turned out that we had manufacturing problems and the company ended up going out of business, so we reverted to iodine.
The Lunar Module used iodine.
Waste water, the condensate from the cabin humidity control and urine and the urine pretreat.
Urine has to be pretreated to bind the urea.
If you don't, you get a lot of ammonia.
It uses a chemical called Oxzone.
It's an acid.
I'm not really sure what it is.
This is where the system is located.
This is one of the simpler schematics.
The water tanks are here.
The water comes from the fuel cell.
As I said, by the way, the delivery is about 150 degrees and 60 psi.
So there is a lot of hydrogen dissolved in the water.
The water is collected in the fuel cells on the hydrogen side, so there is a lot of hydrogen.
There is a hydrogen separator here.
Otherwise, your tanks would end up full of hydrogen which is what you don't want.
You will get water in the tanks.
The hydrogen separator is a silver palladium metal.
It is catalyzed.
It is basically porous to hydrogen but not to water.
If you expose the backside to the vacuum it pulls the hydrogen off and the water goes through just fine.
I will say that there still is a rather high gas content in the water that you drink.
And that gas, of course, evolves once it gets inside your stomach with predictable results.
[LAUGHTER] Yes.
In fact, all sorts of systems that Jeff could explain.
They have slinger systems and all sorts of systems to get separation because it is not really possible to get all the gas out of the water.
OK.
There is a dump system.
There is a potable waste dump to vacuum.
There is also a vacuum vent, which I will show you what that is all about.
And then the flash evaporator that I mentioned, and I will mention more later, is in the back of the vehicle and it has a redundant water feed system both sides.
This is a schematic.
Start at the same place.
Water from the fuel cells goes through the hydrogen separator and then into this bank of tanks.
The tanks are pressurized.
They are yellow bellows tanks.
On the back side of the bellows is nitrogen.
On the forward side, of course, is the potable water.
There is a hydrophobic filter on the downside of each tank.
If a bellows were to leak, you don't want water into your nitrogen system.
So this is a filter to prevent that.
I mentioned the twin flash evaporator feeds.
There they are.
This is the dump nozzle here.
The potable water system provides water to the crew so there is a regular water interface and a chilled water interface here.
I believe that is it.
I remember there were more problems with Apollo than there were with Shuttle.
49:30 I don't have those sorted mentally.
I'm sorry.
I don't remember.
I'm not sure.
[AUDIENCE QUESTION] Maybe that was on Apollo.
[AUDIENCE QUESTION] Basically that means you put your water into a plastic bag.
And now you can actually swing it around like so.
And so the water will go to the outside and the gas is in the inside.
So that actually separates them and you can squeeze the gas out through a valve.
On-orbit trash storage, there is an overboard bleed for odor control.
It's a very small overboard bleed to vacuum.
And the human solid waste collection and storage is also vacuum dried and stabilized.
This is a waste management picture.
The subject we were just talking about is the vacuum vent.
That vacuum vent does vent the commode area where the feces are.
That vent also provides a vent for the water separator.
It also provides a vent for airlock depress for EVA.
So this is the vacuum system.
The urine is collected and trained in a liquid.
Well, let me say that different.
The urine, after it has been separated from the gas stream, is stored in a waste tank, as is the humidity from the cabin atmospheric revitalization system.
And then that is pressurized from the pressured composition control system and is dumped overboard.
This waster is not used for anything.
It is basically dumped just to do inventory management.
The next picture explains a little better about the commode and the urinals themselves.
The urinal originally had a unisex interface cup.
It turned out that didn't work very well, and so later flights there was a custom male/female.
And later there were even some more customizing for the crew, I believe, but I don't have the details on that.
The urinal itself, obviously for urination, there needs to be some way to make the urine go where you want it to go.
And those were done in fan separators.
So there is gas pulled in the cup.
And in training the urine, the urine then comes down and goes through the separators.
This is also a centrifugal-type separator.
And then that is sent to the head that is created here.
Then puts that waste water urine in the waste system.
The EMU spacesuit, it also has a condensate drain after each EVA, and it also goes through this same system and into the waste tank.
The commode.
Basically there is a seat.
The seat has a small opening.
The commode itself is basically a cylinder or a tub.
And there is a slinger in the bottom.
So the slinger basically distributes the paper and the fecal matter on the outside wall in a thin layer.
The vacuum then can dry that so that it deactivates it so it does not end up either any odor or creating any kind of bacteriological issues.
But the entrainment of the fecal material so that it, in fact, hits the slinger, is done with an air flow.
The air flow comes in around the seat itself.
That requires good contact with the seat.
It also requires positioning.
Positioning turned out to be an early issue, or reliable positioning, and so a trainer was put together that had a camera here.
And the trainer, I might add, was discretely located in a non-observable place so that it could be used in privacy, the comings and goings, and also the use of the facility itself.
And they told us it was not hooked up to a tape recorder.
[LAUGHTER] Just to elaborate on that.
Basically, the problem is it really is a rather small opening.
And it just doesn't feel natural to know where to put yourself down.
And it actually did lead to significant messes on orbit.
I won't go into any more detail.
But the idea is that you would position yourself the way you thought it should be, and then you would turn on the lights.
And there is a TV screen in front of you.
I'm not making this up.
[LAUGHTER] So that you could see how close you are to the bull's-eye.
Enough said.
The cabin thermal control is another one of the subsystem elements.
It is a circulating liquid cooling system.
It provides the heat sink for the atmosphere itself.
It also provides the ability to reject that heat to the spacecraft cooling system because you've got to get rid of it out of the cabin.
It has some avionics cold plates, some air cool avionics, support the crew in the airlock and a portable water chiller.
This is the location.
You will notice the avionics bays are listed here with their little circulation systems.
The liquid system then services each of those, the chill or the condensing heat exchanger.
And this is the place that the cabin system connects to the spacecraft system, on the aft bulkhead right here.
We will see the other half of this system in a little bit.
This is a schematic that is very complicated looking but it is fairly simple.
Let me show you the air piece first.
Look at the black lines.
Those are twin fans.
It goes through the air-cooled avionics through the heat exchanger to get rid of the heat.
There are some cold plated avionics in each bay, so you take care of the cold plated and the air-cooled in this bay and in this bay and in this bay.
Now you've got all of the electronics taken care of.
I should have started this up at the pump.
Sorry about that.
Here is the pump and we will come down.
It also takes care of the forward and overhead window mounts around the window for thermal control.
And the hatch is just a way to keep those thermally stabilized.
Back in the early flights, we had something called DFI, development flight instrumentation.
It was a special instrumentation package.
It was also liquid cooled.
And then this is the heat exchanger to the spacecraft system.
It is a Freon system.
This is where the main heat is rejected right here.
On the other side of this you've got cool liquid.
It turns out this is water.
We go through the liquid cooling garment which is the spacesuit cooling garment, the water chiller, go through the cabin heat exchanger to cool the cabin and condense the water out and then back to the pump.
I failed to mention the IMU.
It has triply redundant fans.
It pulls air through it.
And that is also cooled by this gas stream.
Anyway, this is the cabin thermal control system.
These functions, I just said each one of them so I won't repeat them.
Cabins circulating liquid cooling loop.
As I mentioned, water is the coolant.
Water is actually a very good coolant.
And, if you have any requirements that push you that way in your designing thermal control systems, you like water.
Water is completely nontoxic so it can be around the crew.
Leaks don't make any difference.
It has a very high CP.
It's a really good fluid.
The problem with water is freezing.
You don't want to freeze it and you don't want to get it so hot that it turns into steam.
But if you can stay within the boundaries it is a good fluid.
And that is what this is.
Redundant pumps, I showed you that on the schematic.
The liquid gas heat exchanger, I showed you that before.
Cold plates for electronic cooling and the window hatch mount.
The circulating gases that I mentioned for the avionics, the cabin gas was used as a coolant.
You saw the redundant fans and the doubly redundant fans.
I mentioned earlier, and someone commented on it, that the promise early on was the Shuttle could be a less expensive development activity if we used off the shelf avionics, the kind that aircraft use.
And that's a good promise.
It turned out that that's not a very good, not as good a solution for thermal control as liquid cooled.
Liquid cooled is a much more efficient way of providing a heat sink to electrical equipment.
When you blow the cabin gas through it then you end up with the problems of if you change the pressure in the cabin, which we talked about.
That can effect cooling.
Also you have to put out a lot more power.
You have to blow a lot more gas than you do power to move the liquid around, so it's not a very good thermal solution.
And actually the promise of less expensive avionics I don't think was actually fulfilled, although that's somebody else's topic and not mine, because I think everything was pretty well custom.
It did potentially complicate the thermal control system for a good goal.
And how well that goal was met, somebody else will have to speak for it.
Another one of the subsystem elements of the spacecraft.
Well, we usually take a two minute break when we're about halfway through.
Certainly.
Why don't we take our two minutes now and then we will pick it right back up?
[AUDIENCE QUESTION] From an engineering standpoint, it is much better to cool with liquid.
If you use cold plates, it is much more efficient.
You get a good reliable heat sink.
And you can then design your thermal conduction paths so that out of the avionics box you make sure the heat goes to the cold plate.
And you use a lot less power.
You can pump water a lot more efficiently for the cooling capability than you pump gas.
Is that what happens on the Shuttle now or do you have a mixture?
[AUDIENCE QUESTION] This is the air-cooled part of the loop and here are the cold plates.
Let's get the class back.
Thank you very much.
I am sorry.
Did you see what I was pointing at here?
Yeah, you put both.
Yeah, and this is one bay.
And they have both cold plates and the air-cooled here, here and here.
There they are.
Of course, when you go outside the vehicle there is no more air so all the external equipment is cold plated.
Was there another question?
What kind of design changes would you have to make in order to go strictly to completely liquid cooled electronics?
Well, if you're trying to use off-the-shelf and it is air-cooled already, there is a cabinet right there full of it.
If it is air-cooled already then you have to stay with that concept.
If you're custom designing then, from the very beginning, you say I'm custom designing this to be liquid cooled.
And then the internal design is such that there are conduction pads to the base plate so that it can be cooled.
OK.
I guess previously to that all of our spacecraft were liquid cooled.
Apollo was, wasn't it?
Yes.
Because you had to be able to take those spacecraft down the back?
Right.
We evacuated the cabin both in the Command Module and the Lunar Module.
And Gemini, too.
This is the spacecraft active thermal control system.
This is the vehicle level.
So now, even though we're still under this ETCLSS, we are not talking the classical environmental control anymore.
This is a bigger picture.
Also the reason the word active is stuck in here was a left over from previous programs, too.
For example, on Apollo we had several heat rejection systems.
Not a single one.
And we also did a lot of passive thermal control on Apollo.
We had a mission mode called "barbeque" which is basically you roll the vehicle very slowly so you normalize the environment.
If you leave the same side toward the sun all the way to the moon that side gets really hot.
This is the active system to get away from heaters and/or vehicle level control.
There is an on-orbit.
We have a radiative heat sink.
We also have an evaporative heat sink on orbit.
During ascent and entry we have two heat sinks, both of which are evaporative.
We have a circulating liquid system.
We take heat from the cabin, from the fuel cells, hydraulics, all the cold plate avionics and electronics, and payloads, we do cooling for payloads.
All of these are the functions of the active thermal control system.
This, as you notice, is a lot busier than some of the others have been.
You will notice the radiators here on the side wall.
This is the doors.
When closed up there are no radiators.
When opened the underside of the payload doors are your radiators.
And this is on both side, obviously.
That same heat exchanger that I showed you before that interfaces with the cabin is right here.
The cabin load is dumped there.
Fuel cell load is dumped here.
The fluid is a Freon 21.
There is a Freon 21 pump package.
The payload heat exchanger is here, I believe.
Mid body cold plates.
Aft cold plates.
Each radiator side has its own flow control system.
The evaporative heat sinks, one is the flash evaporator that I've shown you a couple times before.
I will describe it a little better in the minute.
It is here.
It uses water.
There is an ammonia boiler here that is also an evaporative heat sink.
And when we're on the ground there is a GSE connector for the GSE heat sink.
[AUDIENCE QUESTION] More than you would ever know.
In fact, this is the most complicated, integrated thermal test ever run at JSC.
And I will show you a picture of it in a minute.
We actually tested all of that together.
And it all had to work together.
And it gets worse as we go along.
I will show you all the integration factors as we go along, but it changes from mission mode to mission mode.
It changes within mission modes as to how the system works.
And I will try to describe that.
OK, so this sort of gives you the feel now.
Keep in mind, geometry is going to play an effect here as to how you plum the system up.
So Freon is running through all of those lines there to the radiators?
It will be a little clearer in a minute.
This is a schematic.
Let's start at the radiators.
On a good day, this is doing the job for you.
All the radiators are getting you nice cool liquid.
The radiator uses a bypass system, so you will see the bypass around the radiators.
It is good to remind yourselves that heat load can be tremendously different.
And the high level mission phases, a lot of electronics, a lot of crew activity, the vehicle is alive and pumping out the power and, therefore, pumping out the heat.
If you ever did one of those 28 day missions we talked about, which was a spec requirement, you can imagine this vehicle is almost quiescent.
It is doing very little.
You're trying to link them the mission.
So you're doing hopefully experiments or something in SpaceLab.
You're doing something else.
The vehicle is not working for you very hard.
Just as an example, we have five computers.
And we use all five during ascent and entry, the active phases for safety, but once you're on orbit, when you have a long mission like that, you actually power down all but two of the computers.
That is just it.
And there are a lot of other systems which you power down.
And that way you're limited in missions by the electrical energy which is limited by the amount of hydrogen and oxygen you can carry for fuel cells.
So, by powering down, you extend your mission.
And, of course, you will also reduce the heat load.
And, when you reduce the heat load, you've got to find some way to make that same radiator work for you at a very low heat load.
So it has the capability of getting rid of a lot of heat.
If you don't want it to get rid of that much heat, because you don't have it, then you have to go to bypass system.
Anyway, the next thing in the loop is the GSE heat exchanger.
On the ground you don't have radiators working so you use the GSE heat exchanger.
Then there is the ammonia boiler.
When you reenter the ammonia boiler gives you the ability to operate with an evaporative heat sink prior to the connection of the GSE, and so it is part of the thermal control system.
The next is the flash evaporator.
And you'll notice there are two flash chambers here.
One is called the high load and one is called the topper.
And I will talk about that a little more later.
Then the nice cool liquid, however it was cooled, all these cool it.
However it was cooled is then available for the vehicle.
And there is one more little heat load, and that is the cryogenic oxygen that we're bringing in.
We also warm it up with this coolant loop.
So the flow then goes to the aft cold plates.
And then the flow also goes to the cabin heat exchanger and the payload heat exchanger.
You notice these are either or.
And that's because sometimes you don't even have a payload heat exchanger use so you don't use it.
Sometimes you have a lot of use for it.
And maybe, for example, SpaceLab and Space Station have kind of an arrangement.
You wouldn't be in the cabin as much so you can divert that.
It allows you to use the capacity in either of two places here.
Then you come on down and come through your pumping package, there are redundant pumps.
Then you go the fuel cell heat exchanger because now you've used the nice cold fluid up.
And so the fuel cell can stand the warmer temperatures.
Also some cold plates.
Now, it might have occurred to you, it occurred to me, why did we waste our nice cold fluid on these cold plates?
Why didn't we put those cold plates down here so we could have had more cold fluid for our cabin heat exchangers?
And the answer is geometry.
Those are way in the back.
The cabin is in the front.
After you've come forward from the flash evaporator, which is in the back, to the cabin, you don't want to go back to pick this up.
You wouldn't want to have to go back to the back of the vehicle and then back forward again, so this is a geometry problem.
We have cold plates here.
We also have cold plates here.
They come forward.
The hydraulics heat exchanger is the last thing on the loop and then back to the radiators.
We will talk about that a little more.
The functions of the active thermal control system.
Collect the waste energy from all the systems.
We just talked about that.
And rejected radiatively to space.
That is our first choice.
Our first choice is to use the radiators.
It turns out that the radiators in some heat load environments don't do the job.
You have a high load and a bad environment, you cannot get rid of all the heat.
Also, we are producing a lot of water, so something constructive has to be done with this water.
The fuel cells are pumping it out because every time you make a kilowatt you've got more water to use.
So we decided to augment the space radiators with the evaporative heat sink at these high load cases.
This gives us an ability to accept a radiator system that cannot really do the job by itself but make it completely adequate for some of the low heat load cases, but in the high heat load cases then it is augmented by water.
Now, when you want to get rid of water and you don't have some of this bad environment you can actually throttle the radiators down.
And, if you throttle them down, then you force the evaporative heat sink to come online and do the job for you.
In other words, you change the set point on the radiator.
Normally we use 40 degrees set point.
If you change it to 56 degrees and let the water take you from 56 back down to 40 then you artificially get rid of water.
It is a good water dump system.
During ascent, over 100,000 feet we use the water boiler.
The flash evaporator.
Not a water boiler.
That's a Freudian slip, I guess.
The Apollo had what's called a water boiler.
It was a nightmare.
We said never will we do that again, so we ended up with a different design for the Shuttle.
The reentry system below 100,000 feet is ammonia.
It will carry you all the way to the ground and on the ground until you get the GSE.
Water won't carry you because water won't evaporate less than 100,000 feet, but the ammonia will so that's the reason it is there.
Let's talk about the radiators themselves.
The two payload bay doors are a mirror image of each other.
They are radiators on each side that are exactly mirror image.
I don't know what happened there.
Anyway, this is supposed to say that the radiators on these two sides are mounted and are plumbed in two separate cooling loops.
The reason for that is there is always a redundant loop.
Obviously, you have to be able to take a failure and still have a good system.
If you run both loops through both radiator sides and you took some sort of collision hit then you could wipe out both of your cooling loops in one hit.
And so that is unacceptable.
By plumbing the systems up with mirror image radiator systems on different loops then you can still use the capacity together.
But if you lose one side you still have half your capacity left.
And plus you have your evaporative heat sinks to make up the difference.
So we can survive with a door out.
I mentioned the dual set points before.
I also mentioned the bypass thermal control.
I mentioned the dual set points, 40 and 56.
40 is our normal.
If you're trying to burn up some water you go to 56 and you use the difference in water.
The radiators, we have two different kinds of radiators.
We have the single-sided that are on the back of the vehicle and some two-sided ones on the front of the vehicle.
The reason for that is when you open the door you expose the underside of the door.
If you have radiators on the underside of the door then you've exposed your radiators.
But if you lift the radiator panel off the door, because the doors go way down, then you can see the underside.
It's not as good, you don't get a full view of the heavens, but it does get rid of heat.
And so we needed that extra capacity.
On the front four panels they are the two-sided and on the back four they are single-sided.
The radiators themselves are made out of a honeycomb aluminum structure.
The tubes were imbedded in the structure so that the surface was smooth, and that was because we wanted to use a silver Teflon tape for our thermal control coding.
For previous vehicles we always painted the radiators, painted them white.
If you look at the alpha epsilon white paint, it is pretty good.
I think Apollo was 0.18 alpha and epsilon about 0.92.
That's pretty good.
The market had moved along.
Technology had advanced and there was not the silver Teflon out there.
And what the silver Teflon does is the silver gives you the reflection which is good and the Teflon gives you the radiation which is good, so you get the best of both worlds.
Silver Teflon was about 0.05 epsilon and about 0.88, 0.9, something like that.
You lost a slight amount in epsilon and gained a lot in alpha, so basically you could ignore the sun which is what you'd like to do, is ignore the sun.
It turned out the silver Teflon had some issues.
We tested the radiators.
The radiator was not a slam dunk with eight panels, four of which are two-sided.
Effectively you have 12 panels.
You have fluid going through those in parallel tubes.
You're trying to guaranty that the fluid all goes where it's supposed to go.
And, therefore, the viscosity of Freon, which is very low, analytically said it was all right.
But we wanted some testing to prove it so we did test the entire radiator system.
And, in that test, all the silver Teflon fell off.
It is a really good thermal coating but it doesn't seem to be very practical.
It turned out that the problem was adhesive.
We did a little adhesive work and the materials people came up with a Permacel that worked perfectly.
They have never come off again.
In fact, we get 10, 12 years out of the radiators, lots of time.
The problem that the Permacel couldn't solve was beneath the tape you could get very small little out-gassing of the materials.
Almost all materials outgas some.
Basically, we would put bubbles underneath the tape.
We went to a perforated.
There are millions of little holes in this tape.
And it is placed on the vehicle and it stayed on just fine.
It wasn't very long, though, that people began to worry about the focusing of this silver, that the doors are curved upward.
And so there is a focal point there that is a very, very hot point.
And there was concern that that's really not a good thing to have.
If we could make the surface less specular and more diffuse without losing the alpha and epsilon we would have a better design.
And so they dimpled it.
Now we have Permacel dimpled, perforated silver Teflon tape.
It works great.
The flash evaporator.
Water is the flash evaporator's evaporant.
This was a brand-new design.
The water boiler on Apollo was no good.
It was a problem.
The flash evaporator is a very simple concept.
You have a chamber.
You run your coolant around the chamber that you want to be cooled.
And you spray water on the walls of the chamber.
And, if the pressures are all right, the water will flash.
You won't get ice.
It will go straight to steam.
Steam will go overboard and it will all be wonderful.
And if you can convince a program manager of that good luck.
Anyway, the way to convince a program manager is to show him good test data.
During the development, we developed this process of spraying the walls and not getting water carryover which will obviously freeze up your steam duct.
Which you don't want.
But, as the design evolved, we had to get more sophisticated because we needed this flash evaporator to burn up the extra water when we didn't need the fuel cell water.
And we did that with that 40 to 56 set point.
And so we needed a small evaporator that could handle that.
It was called the "topper."
It tops off the radiator.
But when the doors are closed there is nothing else to get rid of your heat, now you need a loss of capacity.
And so the second chamber was the hallowed chamber.
Together they do the whole job.
When the radiators are working only topper, you can come home with this system functional and make it home.
It turns out we never did freeze up the steam duct so that part was good.
It turns out that we had some freezing in the chamber.
And, in fact, we had to develop a process that acknowledged that that could occur.
And how would we flush it?
It is called "flushing."
And so we did develop a process.
And it has never failed.
The flushing process always works.
And we have had on several occasions, I probably cannot count them for you, but four of five times maybe over the Shuttle flights we've had some freezing inside the cavities.
There is a non-propulsive overboard vent.
That was the one going out both sides for the topper because it could be used for very long times during the mission and you might be pointing or whatever you might be doing.
But when you're coming home the high load is just a big out the side of the vehicle type duct.
The ammonia boiler, its heat sink is ammonia.
As I mentioned before, it evaporates all the way to the ground where the water won't.
The redundant boiler is here.
Up here there are no redundant boilers.
These two chambers have redundant everything.
They have redundant feed water systems and redundant control systems, redundant spray nozzles, redundant everything, but the chamber is a chamber.
You get what you get.
They are redundant boilers is the way the ammonia boiler gets its redundancy.
It was designed to be utilized during entry and on the runway until the cooling is available.
It turns out that we don't do that anymore.
It turns out operationally we learned that if we cold soak the radiators before we shut the doors and then bypass them and come home on the flash evaporator that there is a lot of heat capacity still left in those radiators.
There is a lot of fluid out there and it is cold.
And so we can actually go all the way to wheel stop on the radiator.
And so at 175,000 feet we start flowing the radiator.
And at 100,000 feet the flash evaporator is off and we cold soak all the way home.
And after wheel stop the ammonia boiler cranks up.
I mentioned the Freon 21, the low viscosity.
We got an excellent performance out of that.
The low viscosity is really, really important, as much parallelism as we have.
I didn't really give you the numbers.
In the single panels we have 26 tubes.
In the two-sided panels, I believe, there are 66 tubes.
Lots of tubes.
Lots of parallelism.
It is awfully easy to get flow instability in a thermal system like that but the Freon loop does great.
We have redundant pumps, all liquid heat exchangers and cold plates.
This is that chamber that I told you.
If you can look right down here, this is one person.
In fact, there is a radiator panel.
This was during buildup.
Twice we tested the flash evaporator in this environment once as part of the integrated system with all the radiators and once by itself.
And twice we tested the radiators, one as part of the integrated system and once by themselves.
So we did essentially three large tests.
And the one that had the entire system in it, had all the cold plates, had everything in it, as I said, is the most sophisticated thermal test we've ever done at JSC.
And it proved that the system could work.
It also proved that we had to tune the flash evaporator nozzle because it was wrong.
But we did get the data, we did prove it was wrong and we got it re-machined.
A machine out of columbium, somebody else will have to answer why.
And so it was a problem for Aaron to go get us another one, but it fixed it and it was successful.
[AUDIENCE QUESTION] You can have a command module and a service module and a stack configuration in side that chamber.
And it had a solar and full vacuum.
[AUDIENCE QUESTION] It was a manned chamber then.
It is not now.
After Apollo-Soyuz, it hasn't been manned since then.
It's still available for testing but is not manned.
It has a sister chamber 100 feet that way where all of the EVA testing is done.
A good example, I think, of how the Shuttle Program built on a lot of Apollo-- The same thing at the Cape where a lot of the launch hardware was adapted is the Shuttle Program, you know, you heard about the budgetary constraints.
The Shuttle Program never could have afforded to build something like that.
But luckily a lot of this equipment was built during Apollo when the money was much more available.
If you have been keeping count, we're getting toward the end here.
I did want to mention, before we left the main orbiter systems, that there was some rotating equipment life testing that we did for the program.
The equipment we're talking about here is the cabin fan, the water gas separators, the avionics bay fans, cabin coolant and the vehicle coolant pumps.
The life requirement was 100 missions, and that was calculated by somebody to be 20,000 hours.
If you could show 20,000 hours of successful operation, you had the design life of the hardware.
We set up a laboratory to run the equipment.
It was obviously a low-maintenance lab.
We collected data and checked in the room occasionally to see if everything was all right.
It turned out that all the equipment passed the 20,000 hour requirement.
We did learn something, though, that some of that stuff was so noisy.
Oh, it was noisy.
There was some work done on some muffling, but the hardware itself was all good.
We are at the end.
Yes.
Were there noise requirements [AUDIENCE QUESTION]?
Yes, there was a cabin noise spec.
And, yes, it had to be met.
And I don't remember how it was met.
We put mufflers on some of the equipment.
Maybe some insulation inside of the ducting, I'm not sure.
If we have time,
I will tell a little anecdote here.
I chaired the Change Control Board.
[NOISE OBSCURES] The medical organization came to see me and said that the cabin was very, very noisy and it was going to really do a detriment to the astronauts.
And it was very expensive in terms of dollars and weight to really get all the cabin fans and cabin environment down to the level they wanted.
I thought for a moment.
I go out to California a lot.
I go out to Rockwell and stay in motels that are on the Los Angeles freeway.
And I said you don't see me worried about my ears in staying in those freeways and not being able to sleep and having all that noise.
And the doctor looked at me for a moment and said, well, you're already deaf.
And I said what did you say?
But we did have to make some modifications of it to actually reduce noise.
Let me interject one item.
I think somebody asked the issue about the IMU.
We decided to pick the IMU off the shelves, and it was air-cooled, to really do it right.
If we would have done it right, we would have picked an IMU which was compatible with liquid cooling rather than redesign it.
But we decided to stick with the IMU that was air-cooled.
Whether that was the right decision to make, I don't know.
It seemed like it was because I think we've only had one problem with an IMU.
I don't think we've had many problems with the IMU.
But that is a little anecdote that I thought of.
That's a program manager's trait.
Probably the air-cooled IMU was a less expensive option.
It might have cost a little in terms of thermal efficiency, but that is only energy power.
And power was available so it was a good tradeoff.
This is the last of the subsystem areas.
This is the EVA airlock support.
The Environmental Thermal Control and Life Support System does support the airlock.
It maintains the cabin pressure and composition during the airlock repress and depress.
This is really a requirement back on the ARS and the pressure and composition control system because airlock use basically dumps cabin gas and then you repress the airlock.
It is a requirement that you cannot upset the cabin while you're doing the EVA function.
We also provide the service and cooling umbilical.
If you have any kind of mental picture of being in the airlock, the crew is connected with cooling and oxygen through an umbilical so they don't use the consumables in their spacesuit.
And so that is provided by the ETCLSS.
Heat rejection, the crew wears a liquid cooling garment.
And the cooling for that is done by the vehicle until you go EVA, of course.
Then the recharge.
We supply the backpack O2 recharge from the 900 PSI cryo.
We supply the water to recharge for the sublimator.
NASA is overrun with acronyms.
EMU is extra vehicular mobility unit, which means nothing but spacesuit and backpack.
So the spacesuit and backpack uses an evaporative heat sink and it uses a sublimator.
And the idea there is it's a device that exposes a layer of ice to space and then it sublimates and you get your cooling from that as opposed to putting a liquid water on a surface and flashing it.
Anyway, that has to be serviced.
That water is provided from the system.
And then the condensate that the crew brings back, EVA, that's a sweaty job.
That condensate goes back into the system.
So the airlock support is another part of this system.
In fact, that is the sixth part.
So we are through.
This is a picture of the airlock.
Not really the airlock.
This is the test chamber that we built to put the equipment in.
You'll notice it has the right hatch in it.
And, if you can recall back an hour and a half ago, I showed you a picture where there was this hatch basically on the backside of the test chamber.
We hook those together.
Because, as I said, as you depress and repress the airlock it affects the cabin.
So we needed a test environment that had both those simultaneously.
But the difference is this has to go all the way to vacuum.
So we connected this to our vacuum system.
And, when the crew is inside this, it is a manned vacuum test.
And when the crew is inside the other test chamber it is basically sea level or, at most, 8 PSI would be the lowest it would ever be.
And I think the last picture is a spacesuit inside the airlock.
And this is just a little cutaway showing the airlock.
This was mounted on the back of that chamber you saw earlier.
If you bring that picture back up, I will talk to that for just a minute and then we will have some questions.
Before every flight, one of the things that we do is that every EVA crewmember takes his or her spacesuit into that EVA test chamber, into the airlock test chamber so you actually get a chance to take your own suit down to vacuum.
And particularly, before you do your very first EVA, it is really a confidence builder because the physical sensations that you get in a suit -- And remember you're at about a 4 PSI pure oxygen environment.
Your body, the inside of your mouth feels a little different.
There are funny sounds with the fans inside the suit.
The sorts of things that you don't want to experience the first time when you actually go outside into space.
It is much better to do it here in a controlled environment where if there is a problem -- Actually, you are supported because of the weight of the suit.
It doesn't look like there is anybody in this suit.
But, when you're in this suit, you're actually supported by a cable to the ceiling of the chamber which is on quick release bolts.
You're suspended up here.
And, if you ever have a problem, they can release this, repressurize the chamber within about 30 seconds.
I think that's the limit for a manned rated chamber.
You're going to be able to repress in about 30 seconds and then just lift you out, suit and all from the top, and get you out of your suit and get you to the medics.
This is just an inverted bell jar.
The hatch is not sealed.
When you depress it just like pulls it out and seals due to the pressure difference.
You just lift the top off.
Yet another example of the extensive testing that we go through before every mission.
I might add one more thing, which I forgot actually to add earlier.
The manned rating is a term used in manned spacecraft which means that you've specifically tested it and said that its design is worthy of supporting a human.
The airlock had to provide that function.
So Aaron was faced with man rating the airlock.
And manned test capability is very scarce.
In fact, JSC may be the only place anymore that can do it.
Anyway, we volunteered to do that as part of our test program as a programmatic requirement.
Not as testing or watching or confirming anything, but as a prime part of the test program was the manned rating of the airlock.
And that was done in Houston.
That is an interesting thought.
If my memory serves me correctly, one of the problems we had in the early part of the Manned Space Program was doing some chamber runs McDonnell-Douglas with a two-gas system where we actually lost the heat [NOISE OBSCURES] in the chamber and almost lost a test subject.
And that was when Apollo decided to go to 100% oxygen.
And that is what led up to the fire on the pad.
It went to 100% oxygen at 16 pounds per square inch with an invert over the hatch for another reason.
And that is really if you can realize that 100% oxygen at 16 PSI everything burns.
Stainless steel is not self-extinguishing at 16 PSI.
And, of course, that's what we were up when we had the Apollo I fire on the pad.
It was a function of doing some test work with the combined atmosphere.
So you have got to be careful that sometimes you don't over-react the thing.
Then we went back to a different type of atmosphere.
Did you have a question?
The systems you've been describing today seem to take up a lot of space in the Orbiter like other systems.
How did you negotiate and manage other subsystems for the space and the volume in the way that you wanted?
That is a good question but you're asking the wrong person.
We are the government.
We hired a contractor to design the vehicle.
Now, the design aspects of it, the philosophy of it, the functionality of it we were very, very integral with.
We watched, we audited, we said that looks smart, but we didn't go and say put that cold plate here and put that heat exchanger there.
The vehicle integrated design was Rockwell's.
Let me tell you how it was done.
Actually, in those years we didn't have CAD/CAM systems.
What we did is used everything on mockups.
We did mockups.
The system was laid out of what to do and the functions of what they had to do and the plumbing.
And this was all laid out on a mockup.
And basically that was how it was negotiated in terms of what you need.
In today's environment you wouldn't do that.
You would use electronic capability.
You would use your CAD/CAM systems.
You would actually draw it up on your computer.
In fact, the 777 was built that way.
The 777 at Boeing, they didn't use hard mockups.
They used electronic mockups.
And that is what we would do today.
And it would be much easier.
You would probably call all the engineers together.
You would sit down and show how the equipment was laid out, get a satisfactory buyoff on it and do it much more rapidly and much more quickly.
In the future, what you will be doing is actually use a CAD/CAM system or CAD system to actually negotiate the space you need for it.
That is a good question.
And it was based on purely that you would get drawings, mockups, go out and lay it out in a soft mockup or hard mockup.
You would get the engineers to come out, the manufacturing people to come out and say this is how it is going to look and then get a buyoff on it.
Today you wouldn't do it that way.
You would do it all electronically so it would change.
Well, let me ask you a question.
Of all the work you've done in the Shuttle what was your biggest concern in getting ready to fly or during the flight?
What did you have the biggest concern about?
I was young and confident.
I don't know that I had a specific area that I was really concerned about.
Like these people right here, right?
Young and confident.
I did recognize, though, but it did result in the integrated tests that I showed.
I did recognize that we had the most highly integrated thermal control system ever conceived.
And it had, as I said, modes within modes.
It changed modes with respect to the vehicle phases.
It changed modes with respect to operation.
We had 9 PSI that went to 10.2 PSI.
We had 8 PSI.
We had thermal control loops, ammonia boilers, flash evaporators, so I did recognize the sophistication.
In fact, we proposed early on to use that large chamber in an integrated vehicle test.
Not we, me, but the big we looked at an integrated test like the Command Service Module was tested.
And that was viewed to be impractical.
It was cut it off at the bulkhead.
You could do the forward cabin part and then do the aft section, but that was viewed to be impractical.
But it still was a very integrated design.
All the systems engineering that was there created that inner-dependency.
And I think I was smart enough to know that there was certainly some potential risks there.
Remember in the very beginning of this I talked about ownership of all the technical aspects?
We had ownership.
We believed that what we had done was prudent, appropriate, and we believed it would work.
It wasn't like you bought something and you hope it works.
Can you say a word of what you think, this is looking in your crystal ball, about the CEV?
Do you think it's going to have a very similar type system or have you thought about that?
Well, we hope so.
Our new administrator has sort of back to the grassroots philosophy, or at least right now.
And so we hope that the CEV will be developed with a lot of government involvement, that the government will have ownership again of the technical aspects of the design.
If that happens then I think a lot of the experience that is around and a lot of the free-thinking and the independence that the government can bring to that, there is another school of thought that says keep the government out of the way and let the contractor go do their thing.
And, obviously, you wouldn't get that position from me but there are people that believe that's the right way to do it.
Do you think the system will look pretty much like this or similar to it?
Well, similar, yes, I do think so.
I believe that the very tight schedule for the CEV is going to put a lot of pressure on the program manager to go with the tried and true.
So I don't think there is going to be innovation that has risk with it.
Maybe there will be innovation that is good engineering, but I think the innovation with risk will be very much minimized.
Because taking Shuttle offline in 2010 there is going be a lot of pressure to get this vehicle finished.
I think that is going to breed some conservatism on the design side.
We talk about considering operation while you're doing the design, but there were always things once the design was made.
Well, like you mention, once we came up with a requirement for a 10.2 cabin we had to develop workarounds.
And one of the other things that after the system is designed, we then start to figure out how can it fail and work on malfunction procedures.
And I know that the malfunction procedure for loss of two coolant loops that was probably the worst simulation case that we ever had to deal with.
I am curious to get your opinion.
Walt described how we have two independent cooling loops so that in principle one failure won't take them both out.
Now, I don't remember what it was, but they did discover a single point failure somewhere in there which I think was corrected but where you could have lost both of the cooling loops with a single failure.
The only failure I remember is one that John Young used to worry about.
It had to do with if you did lose a radiator that that could bleed your accumulator down.
And so they did add an isolation valve so they can isolate the radiator side.
But the idea was to obviously not lose your fluid with a single hit.
That even though you have a nice completely functional thermal control system it is useless because it has no fluid.
We did have an emergency de-orbit procedure for loss of two Freon cooling loops.
And, of all the things that we practices, that was the hairiest because it was absolutely time critical.
At this point, you have no way of getting heat out of the cabin so you cannot ultimately cool your electronics.
The cabin gets hot, your electronics get hot so you have to do an immediate de-orbit.
Hopefully you're within range of a landing site.
And then you turn off all the equipment that you can possibly turn off, including most of your computers.
And, when you're riding down through entry, you basically watch the temperature of the computer.
And, as each computer would get up to its failure temperature, you would then switch on the next computer, transfer the data, you know, turn off the computer which has just about failed, and hopefully you could make it to the ground.
And I know the instructors that would lead us through this exercise said that the information they had gotten from the thermal analyst was that nobody was really sure whether we could make it.
I don't know what your opinion was or not.
We practiced it.
But obviously, I think I've told you, nobody has ever died in a simulator.
But you never know whether the thermal model that they use in the simulator is really going to duplicate the way the Shuttle is going to behave in an absolute emergency situation like that.
We survived in the simulator but I don't know what your feeling is.
Well, we do address the thermal model accuracy.
We do specific testing to validate the thermal model so we do have pretty good confidence in those.
But the real issue in overheat is that when does a piece of equipment stop working in an overheat situation?
I mean there is no magic number.
It is not like at 143 degrees it is OK and 144 it is not OK. And it varies with age, it varies with equipment and it varies with a lot of things.
So there wasn't really a precise answer as to how long you could keep everything functional and come home.
Yes.
You talked earlier a little about oxygen and nitrogen mixing in the cabin.
I was wondering if there is a way to really study that on the ground since you're not in a DoD environment.
How do you know there are not pockets of nitrogen sitting around that an astronaut can stick his heat into?
Very good.
I just forgot to mention that earlier.
The testing that we ran, we had to set it up very carefully.
If you notice, there is a flight deck and a mid-deck, so obviously buoyancy is going to mix that stuff for you.
What we did is set up a thermal condition where less than one degree thermal gradient everywhere in there had to exist before we ran the test.
And that was to guaranty that we had no help from the naturally induced circulation.
But you are absolutely right.
That test would have been a useless test without neutralizing the gravity effects.
And actually the first time that I did the spacesuit test in there we actually used the 10.2 protocol.
So I actually had to go in the day before, take the cabin to 10.2, spend the night and then we could do just a 40 minute prebreathe.
For most of the other tests, when you go in, in order to avoid the inconvenience and expense of staying through the night, you go in early in the morning.
And then you're coming right from a sea level environment.
So, before you go down to 4 PSI in the suit, you actually have to prebreathe pure oxygen for four hours.
As Walt said, that was the alternative.
You basically just get in your suit.
You can either bring a book and read your book or they will pipe music into you if you want to give them a CD, or I guess there was an audiocassette in those days, or they have a little black and white TV monitor.
And you just sort of sit there for four hours.
And the problem is you're not allowed to go to sleep.
That would be the easiest thing, but they want you actually to move around because when you're moving your muscles you're actually then doing a better job of de-nitrogenating.
So you have to stay awake.
And if you don't say anything for more than about 10 minutes they will pipe something in and say, Jeff, are you awake?
But this was a very, very useful facility.
I mean they used it for all different sorts of things.
And I guess that was the one new facility which was built for the Shuttle specifically.
Yes.
Jeff mentioned several times the 10.2 and I talked about the 9.
The 9 was the original design, but the issues with the 9 PSI had to do with the oxygen percentage got too high.
And so there was much concern about flammability.
The idea was if you could raise the pressure up some to 10.2 then that would be less of a flammability issue.
But then the problem became at 10.2 that the suit at 4 PSI was too much of a drop.
If you're in the bends while diving, somebody mentioned diving earlier, scuba, you could usually use a two to one.
If you don't reduce pressure more than half you're probably OK. You're probably not going to get the bends, but half of 10.2 is not 4.
And the suit is at 4, so there was that issue again.
They raised and bumped the suit up a little bit and got it up to 4.3 and went to 10.2 and developed the prebreathe protocols that made that acceptable.
That is sort of how it balanced out.
That was another operational move later.
And, as Walt said, you're spending enough time, once you get in the suit and you purge all the nitrogen out, and so you're sitting in a pure oxygen environment, you've got to do a bunch of tests and check out communication tests and cooling tests.
You basically are not losing much time by having to spend 40 minutes prebreathing.
The whole thing worked out pretty well.
And it will be an interesting question on how they designed the CEV because the CEV has to be compatible with docking to the Space Station which works at a sea level environment.
But, on the other hand, if you're going to take the CEV into a situation where you want to do a lot of space walks, you probably want to be able to operate it at more like 8.5 or 9 PSI so that you can go right into a spacesuit without an extensive prebreathe.
And, of course, people are always trying to figure out could we design the spacesuit to work at a higher pressure?
But then you give up maneuverability.
We will talk about that in another lecture that I will be giving on EVA systems.
Walt, thank you very much.
I enjoyed it.
[APPLAUSE]
We're quite lucky today to have Bob Sieck talk to us about launch operations.
Bob joined NASA early in his career after a spell with the Air Force.
This was in the early '60s.
And he worked on prelaunch checkout and servicing of Gemini and Apollo.
Then went into the Shuttle Program.
He worked on ground operations for the Approach and Landing Tests.
We will hear more about that testing when Gordon Fullerton comes here at the beginning of December.
And then he worked as Launch Director for a lot of the early Shuttle missions, including my first flight.
And then went to be the Shuttle Operations Director at Kennedy Space Center.
But then after the Challenger accident he came back and worked on another, I don't know, 40 or so launches.
I think my last launch was in '96.
You had already graduated, right?
Yeah.
But Bob was the Launch Director of the first four of my flights.
He is the head of the whole team of really thousands of people who are responsible for getting the Shuttle ready to fly and for certifying that, in fact, it is ready to fly.
It is a very complicated, critical and from the safety point of view absolutely one of the most important things we do, is to figure out when this incredibly complex vehicle is really ready to go.
Bob is going to talk to us both about some of the technical aspects of the launch operations.
But also I hope, from a systems engineering point of view, we will have a chance towards the end to talk about some of the planning that was going on while the Shuttle was being designed and some of the compromises which had to be made, some of which we've already addressed, between the efficiency of turning around and maintaining a vehicle like the Shuttle vis-à-vis the actually initial upfront cost of it.
With that as an introduction, Bob, I will turn it over to you.
Thanks, Jeff.
And good morning.
As Jeff indicated, I got into the business of launching people and payloads into space a long time ago.
I was privileged to be part of all of the human space flight programs, so I did a lot of that.
What I didn't do, I saw most of.
And I have an opinion on all of it, so please ask questions.
If I don't have the answer, I am sure to have an opinion.
I am going to spend a little bit of time, we talk about launch operations, but that's the final three to four days of a campaign that begins three to four months before you actually start the countdown clocks and put the astronauts in the vehicle and go fly.
So I am going to spend some time on the background of preparing the Shuttle, the ground systems, the flight systems prior to the launch count process particularly to talk about the engineering involvement and the responsibilities in those processes.
But, in order to talk about launching, I have to talk some about the operations of putting the final pieces together and doing the tests and inspections prior to launch.
And then I want to get into the real meat of it which is the human factors, the role of the engineer, the role of the managers in the process of testing, inspecting, checking the flight systems, the ground systems and certifying that it is OK to put the crew in and go fly.
And that's where we're going to spend most of our time, but I've got to go through some background first.
I don't know how many of you have been to the Kennedy Space Center.
Anybody been down there?
OK, so you've seen it.
It is a city where we've launched everything since the beginning programs, Mercury, Gemini, Apollo.
The most prominent fixture is this Vehicle Assembly Building which was used for the Apollo Program to put together the 300 foot tall Saturn-Apollo rocket.
Shuttle is about half that size but we use this building to do the assembly.
And I will talk a little bit about that later.
But it is a city.
We have our own power systems.
We have facilities to check out the orbiter to process the solid rocket motors.
We have administrative homes for the people.
And, of course, we have the highly visible launch pads that are out on the ocean.
And we also have a runway.
When the weather cooperates and we can bring the shuttles back to Florida, we can take care of the landing and recovery there.
The process.
We'll talk a little bit about the process.
When an orbiter returns from its previous mission, it goes through essentially a disassembly, inspection and then build it back up and test it to make sure that it is still within the certification base that the engineers set up for it that you've already heard from previous to this lecture.
You would like to think that if the Orbiter flies and the rest of the element is a successful mission you can just do it like an airplane, take down the down cargo, whatever, get it configured for the next payload that is going to go up, clean the windshield, so to speak, and the crew cabin and go fly it again.
But the rules don't allow you to do that.
The rules that the other engineers who have talked to you have set up are such that you have to assume that in some way, even though during the mission everything seemed to work OK, that that hardware was compromised through the process of reentering, landing, refurbishing those systems that have to be refurbished.
Something may have been invalidated.
So you have to test all that stuff that worked just fine in the previous mission, plus all of the systems that were not used in the previous mission, particularly the backup system, the things that you would have to rely on in the event that the primary system doesn't cooperate during the next mission.
And, because of the way the Orbiter was built with its miles of wiring and thousands of components that are active components, you have to disassemble a lot of that stuff in order to meet the requirements that the program is levied on.
After the Orbiter lands and you get it safe and the crew is out and you take out the payload, we spend a minimum of two to three months, roughly a quarter of a million labor hours of touch labor on the Orbiter to do this disassembly, test, inspection, refurbishment process before we move it from the hanger, take it to that Vehicle Assembly Building, integrate it with the tank and the solid rocket boosters and take it on out to the launch pad to go fly again.
And that's where the lion's share of the work is.
The work on the tank and the payloads and the solid rocket boosters is all peripheral to the processing of the Orbiter.
Yes, sir.
[AUDIENCE QUESTION] Do you find a lot of failures when you go through?
We find very few.
And you could do the tradeoff of, well, is this effort worth it?
But, on the other hand, the few that you find, you're glad that you found them because if not you may not have discovered them until you got out to the launch pad and serviced your hazardous systems.
Or, worse yet, you find out when you get in the mission.
And then the impact of that could be not only a safety impact but could be a higher consequence than spending the month to two months in an orbiter processing facility doing that work.
Yes?
But is there a learning curve?
I mean after each mission do you know which system you have to look at specifically?
We look at it and say this system is performing just fine so we don't have to do this intense invasive inspection and test every flight.
Maybe we can just do it once a calendar year or every two to three missions, whatever the designer thinks is the criteria for checking on the health of this system.
And we've done that over the years.
And after we've had a couple of years of missions where everything is working just fine, we start relaxing these requirements.
And then something like Challenger happens or something like Columbia happens and the pendulum of conservatism swings the other direction.
And we do more tests and more inspections to verify the hardware because people want to be able to say, and we want to be able to say, well, we've done everything possible to make sure that that's a 100% functional, operating, safe orbiter.
And so getting this process down to where it meets the original advertising brochure which was we're going to fly these things every two to three weeks, you start in that direction.
And then events occur either during the mission or you have the really bad incidents like Challenger and Columbia.
And the conservatism swings the other way and you end up doing more work.
And, in addition to that, engineers, because we are what we are, I was one, we have all this hardware available, we like to modify it.
We change it.
We upgrade it because there is new technology available in computers or in composite materials.
So, when the orbiter gets to this processing facility, regardless of how well the mission flew, the designers, with their good intensions, say we want this orbiter to last another five years or ten years so we want you to do this upgrade to the avionics system or we want to change out all of the actuators that hold the payload in the payload bay and this and that.
So you modify.
And that adds more time.
It makes more complexity to the process which invites more tests and inspections.
You're constantly in a mode where unfortunately, if the mission flies perfectly, by the time you plan your campaign through the processing facility -- You plan your campaign with schedules to last two months maybe, roughly a third of the work you do is work that you hadn't planned when you initially brought that orbiter into the orbiter processing facility.
It's nonstandard work.
It is stuff that you uncover in the process of doing your test or inspection or its new requirements or new changes that come from the designers that have to be implemented.
And, from an efficiency standpoint, if you're in the manufacturing or production business, that is not efficient.
If a third of your work that you laid out for the next two months, at the end of that two month period, a third of it was stuff that you had not planned on doing which means you don't have the parts ready, you don't have the engineering, you don't have the instructions.
Yes?
Do you build in time to cover that now?
Contingency time, yes.
Let me go back to that.
We set up our team and operations to run five days a week, two shifts a day leaving the weekends free to catch up for this nonstandard stuff.
We essentially end up running a six day work week if we're in a standard template of flying three missions a year, which we're not now because we're still recovering from the Columbia accident.
We end up working six days a week and roughly every three or four days we will work a continuous around the clock 24 hour operation.
We try to maintain that original schedule but we only make it roughly 50% of the time due to this, again, nonstandard work, the extra stuff that works its way in.
And that's why we have a big team.
Our team is made up of the United Space Alliance as a contractor.
They do the hands-on work.
They have the technicians.
They have the engineers that staff the consoles.
And they are roughly a 6,000 population at Kennedy Space Center.
They take care of the orbiter, the tank, the boosters.
They are responsible for maintaining all of our equipment that hooks into the shuttle that is required, the process of servicing it.
And they have ties to the contractors, the original equipment manufacturers that built the hardware.
Technically, they can gain access to all the original manufacturing records for the parts that are in the shuttle.
NASA's responsibility -- NASA owns the requirements for the Shuttle.
We delegated all the operational work to the contractors, but it's a national resource.
Taxpayers paid for it.
The government has a responsibility.
They cannot divest themselves from that.
NASA owns the requirements for saying what you test, what you inspect and what the specifications are for acceptable performance of that.
That is controlled by NASA.
The management of the contractor at Kennedy is done by NASA engineers in an organization that has roughly 500 people out of the 2000 population at Kennedy Space Center of civil service people.
500 are directly associated with the Shuttle.
The rest do payload work.
The Space Station, of course, is a big work item now at the Kennedy Space Center.
And they are responsible for the requirements.
If somebody wants to change the requirement, a NASA person has to approve that change.
And we do insight and everything the contractor does.
We have at each one of the milestones, when we leave the processing facility, when we leave the Vehicle Assembly Building and when we get ready to launch, there is a review conducted by NASA to review the work that was done by the contractor during the preceding weeks or months.
And NASA approves moving the vehicle onto the next stage of its assembly prior to launch.
And the launch director specifically is responsible not only for those last few days of the campaign, so to speak, leading up to launch, but also insuring the government contractor team has all the tools available to them to be successful.
Is the work being scheduled in an orderly fashion?
Is the ever-tightening budget constraining their ability to get their work done?
Is there too much scheduled pressure to meet a milestone?
That is the launch director's job.
The launch director doesn't just come in the last three days of the campaign with a tie and a suit and orchestrate the countdown.
They have a responsibility from the beginning to the end of the campaign.
Engineering.
I have already addressed this, but the NASA engineers manage the requirements, the changes to the requirements.
They are the ones in addition to the contractor, this is a check and balance, that look at all the data, the engineering results of the tests and the inspections.
And they approve that they met the requirements.
Also, they participate in critical operations.
A critical operation is a test on a specific system.
Servicing of the vehicle for launch count or any of the non-standard work that comes up during the process.
They audit what the contractor does.
And, of course, because we're the customer and they are the contractor, we have a lot of criteria to grade them on for their award fee performance.
The operations people manage the scheduling.
As I indicated, we like to work five days a week, two shifts a day.
We really end up working six days a week and sometimes around the clock.
The scheduling activity is dynamic.
And NASA approves all the scheduling.
We have to approve all the overtime.
We have the authority to say stop or go not only in launch count but anywhere in the campaign of processing the hardware, as well as the big picture schedule, you know, how many times a year you're going to fly and which orbiter is assigned to which payload in those missions.
That is still a NASA responsibility.
Orbiter processing facility, like I mentioned before, it is like a hanger for an airplane.
A big part of the work effort used to be the tile.
I think you already had a presentation on tile.
There are 25,000 to 30,000 of these little bricks glued to the vehicle.
We classically damage about a hundred of them on each mission due to ice and things coming off the tank that have to be replaced, but we also pull another hundred or so off just to inspect the integrity of the glue that is holding the tile to the structure of the vehicle.
Because a lot of those little bricks have been glued on there for 25 years and have flown 25 to 30 missions.
Although we have offline test programs that try to duplicate the environment that the tile sees either from a calendar standpoint or the thermal cycles of the mission, you would like to go to the real hardware and see what is happening.
We do that invasive type of work of actually destroying tile, pulling it off the vehicle just to make sure that the aging process is not causing any degradation of the system.
We pull the engines out and do the maintenance on the engines offline because the aft fuselage of the orbiter is a terrible place to work.
Any of the hardware we can get out of there and do it on the bench, it is much better to do that than to try to do it in the vehicle with people crawling all over wire bundles and structure and that sort of thing.
They are concerned about the collateral damage to do the other work in that compartment of the vehicle where the engines are.
Of course, you've got to take the accommodations from the previous mission out of the payload bay and put in the new stuff for the next mission.
I already mentioned modifications.
Unfortunately, we change more stuff than we probably should but in order to keep up with the aging of the vehicle and the new technology that goes with the territory of flying a system that was designed over 30 years ago.
And we prepared for the next milestone which was the vehicle assembly building operations.
This is where we put the big pieces together.
The solid rocket boosters are stacked.
There are four segments in each booster.
That is a very critical process but only takes less than a month to do.
The external tank comes in essentially ready to fly from Mississippi.
Except, as you know now, because of the Columbia accident, we are still looking at the ramifications of these lines and other fittings on the tank.
The acreage foam on the tank, just to digress a little bit, is sprayed on.
And that stuff is tough.
Early in the program we had problems with some of that coming off, but we solved that years ago.
What we haven't solved is where you have to do a manual application of foam over a fitting that is there because that is where the crane is attached to it or because there is a line there that has to be installed at the Kennedy Space Center.
You cannot do it back at the factory.
Putting that thermal protection system foam on there is a labor-intensive process.
It is a critical technique process.
And, obviously, there are still flaws in the process that the engineers haven't sorted out yet.
And that is why that stuff keeps coming off the tank in those places.
But the solid rocket motors are attached to this mobile launch platform with four bolts the diameter of my wrist.
The structure engineers really figured this out good.
There are four on each one of those that are attached to the launch platform.
The tank is attached to the boosters with four attach points, and those are bolts, again, that are the diameter of my wrist.
And then the orbiter is hung on the tank in three places with the same sized bolts.
You've got six million pounds of weight that is attached to the mobile launch platform that is held up by these four bolts, well, eight total on the solid rocket motors.
And they work out the dynamics of when the engines start because the vehicles sways like this because the center of gravity is not right over where those eight bolts are attached.
And that system works fine.
It always amazed me that you could hang that much weight on those few bolts that are only the diameter of this, but it works.
The orbiter is brought over after its campaign, it's hung on the tank and we test all those connections between the solid rocket booster, the platform, the tank and the orbiter.
And that only takes about a week to do that.
And, if that looks good, we have a review of all of that work.
And if the review says we didn't leave anything undone, that we shouldn't take out the launch pad then we go out to the launch pad.
[AUDIENCE QUESTION] Sure.
Say a little bit about this difference from just bringing your car into have a 50,000 mile checkup, that is the clean room aspects, the following of procedures, of having a checker.
Sure.
Well, a 50,000 mile checkup on a car is usually an external inspection.
You change the oil and the filter and you check all the belts and the radiator hoses and you change all the fluids.
Well, you do that with the orbiter also.
But, in the case of the orbiter, you want to know whether you are this far away from having to do a valve job or the rings in the engine have now got so many thousand miles on them.
We have the equivalent.
You take the heads off the engine and you give it a valve job.
And you take the pistons out of the block and you check for scoring in the block.
You take the transmission out and you pull the gears out and make sure that the synchronizers are still OK. And you go through the rear-end.
And you take all the avionics out.
Or, you take your carry-on test equipment in and you hook it up to these connectors on the airflow meter and the computer in your vehicle and you run all those diagnostics to make sure all that stuff, even though it worked fine after you turned that ignition key off the last time, that all the redundancies and all the capability is still within that hardware and all the margins are still there.
And then you button it back up and you make sure you button it back up properly, all of which is pre-established quite rigid flowcharts, checklists.
A checker checking the checker.
Oh, yes, and you have inspectors where there are critical things.
If you're torquing the connecting rods onto the crankshaft, it says that you've got to do those bolts at 50 something foot pounds.
And the technician will do that.
And there will be an inspector there verifying that he set the torque wrench at 50 foot pounds and it really clicked there, and they stamp the procedure that it was done that way.
I wanted to just add to that, I mean, even more emphasis.
You don't do anything to the vehicle or to the payload unless you have a written procedure.
What is absolutely not allowed is oh, there is a little problem, something didn't work the way it was supposed to in the test.
Let's just fiddle around.
Let's throw a few switches and see what happens.
Well, I mean, that is normally the way you work at things if you're in a laboratory or working on your car.
But the thing is you need absolute traceability.
Because if something later on turns up out of spec or there is an anomaly during flight, you need to be able to recreate everything that was done to the orbiter.
That is why we keep such good records on parts.
Well, you can tell you that story about the tires that you just mentioned.
People keep track of where all the parts of the orbiter came from, the lot numbers.
And so, if any problem turns up, you can actually then trace on the orbiter and see if it affects there.
All the work is done for approved work authorization documents, every step, every touch labor item that is done on the vehicle.
And if you deviate from that procedure, which is approved by engineering, you have to get engineering approval to deviate from it.
The assembly process, the standard work process of putting the vehicle together after a mission, there is over two million verifiable work items for the standard flow.
And that doesn't include modifications or the nonstandard work that comes up, the proverbial glitch that occurs when you're checking out your computer system or when you pressurize this fluid system and the leak rate is higher than the specification allows.
Well, engineering comes in and says OK, here is the trouble procedure.
Here is what you're going to pursue, but you don't do any of that until the engineers write the procedure and give it to the technicians or the console operators that say OK, here is what you're going to do to try to find out the source of this leak.
As Jeff indicated, you don't just start throwing switches and turning valves and say well, let's see if we can find out what's going on here.
It is all pre-approved.
And the reason for that is when you have these reviews that culminate into final review before you get ready to fly, you want to be able to say that this vehicle is assembled per print, that the designer's requirements, the people that you've already heard from that certified that this system will work just fine in flight if it looks like this when we launch it, you're trying to prove that this is what it looks like when you launch it and it is within the certification base and it is per the engineering drawings that we were given and it met all these requirements.
Otherwise, you are guessing that it is OK to go fly the machine.
And you cannot do that in this business.
[AUDIENCE QUESTION] Yes.
Does that seem to create any sort of tension on the engineers feeling like their hands are tied to go out and do stuff?
Well, no, because they are the ones that ultimately say what goes in those two million steps.
And if they want to add more tests or checks or inspections it is within their purview to propose that.
The program, there is a group of senior engineers that are on these what we call control boards that approve the requirements that are implemented at the Kennedy Space Center.
And they can say yeah, that's a good idea or, no, we don't want to change this, go back and give us some more rationale for what you think is a good idea to do to this system.
In that respect, it is probably somewhat frustrating for the engineers because they cannot do anything different from last time unless they can justify the rationale for it.
But that's the way it ought to be.
The launch pad.
The facilities at the launch pad are pretty simple.
These are the same launch pads we flew from in Apollo.
We've got storage facilities for the liquid hydrogen, liquid oxygen.
We try to spend a minimum amount of time at the launch pad because Florida is a nice place to be on spring break but it's a bad place to have exotic hardware sitting outdoors with all that salt, humidity and all the other elements that mother nature throws at you like lightening.
And, I think you can see down here, we have a good lightening protection system.
There is a single mast with two wires that go out about a half a mile from the launch pad.
It makes a good Faraday shield.
And over the years we've been whacked a lot of times with lightening.
In fact, it's a good attraction for lightening.
And we have never gotten energy inside of that Faraday shield to damage any hardware.
However, when we get lightening down there, we tell the workers to stand down and, of course, seek safe haven.
But we have instrumented it.
And it has taken some big hits, but we haven't gotten anything major inside that would cause us a problem from electric.
Yes?
After you take the Shuttle out onto the launch pad, is it very expensive to roll it back when you find a problem and then bring it back out again?
Yes, it is.
And that's why we have these reviews before we commit it to go to the launch pad to make sure that we're not taking any unknowns or any work out to the launch pad that should rightfully be done either in the Vehicle Assembly Building or further back in the Orbiter Processing Facility.
It is a lengthy process because it takes about a week to hook everything up at the pad and check it before you go into the process of servicing the propellants and doing the hazardous work which has to all be undone if you roll back to the Vehicle Assembly Building.
You've got to make sure you're really ready to go before you head on out there.
Yes?
Approximately how many times has that happened?
Going back?
Well, we probably have done it for technical reasons maybe 20% of the time.
For weather reasons like we're worried about a hurricane or something like that the same, about 20% of the time.
One out of every five times we've gone to the launch pad, we've had to come back for either a weather problem or a technical problem that either occurred at the launch pad or it's something that happened with a different vehicle in the fleet that may still be in the Orbiter Processing Facility or a components that is similar to one that is on the Shuttle that is at the pad that they were doing testing in the laboratory or doing what we call fleet leader testing at one of the NASA facilities.
And they said, you know, there is an inherent flaw in this auxiliary power unit and we have to change the ones that are in the orbiter because they were manufactured at the same time or they have the same component on them and you cannot do that work out at the launch pad so you've got to roll it back.
So the hardware at the pad may be just fine from the standpoint that you followed your processes and they checked out great in the previous part of the campaign.
But an offline issue can cause you to say no, we've got to roll back and go change out this hardware or fix it or further test it before we can commit it to go fly.
We have a big water tower out there also which I will talk about later.
That is our sound suppression system.
But basically a month is the maximum time you like to spend out there.
We used to put a lot of payloads in the Orbiter Processing Facility when we were flying laboratories and that sort of thing.
Space Station hardware, we install all that out at the launch pad now.
And we test all the connections at the new facility.
And we have a simulated launch count with the astronauts.
And this is a tradition that goes back to the Mercury program.
They bring the astronauts down to the Cape.
They go through a day, two days of training, emergency egress, familiarization with the hardware, And then we have a simulated launch count without the propellants and all the hazardous stuff just to, if nothing else, remind the launch team that this is more than just a machine.
We are going to fly people in this thing.
And this occurs roughly two weeks before launch.
And it is a good readiness test for the whole team.
It's serious.
We're a couple weeks before flight.
The crew is in town.
This vehicle is going to look the best it ever has.
And we do a simulated launch countdown to within a couple seconds of liftoff.
And then we do a simulated abort with the safeing of the vehicle and the crew gets out.
And I will just add something.
That is always something the crew looked forward to, first of all, because you actually get to get in the vehicle which, despite the fact that we have pretty good simulators in Houston, there is nothing like actually crawling inside the shuttle that you're going to fly in.
And there is a lot of safety training that you do down at the Cape which is really kind of fun.
You don't have any pictures of the launch escape, the pad slide wire basket?
No, I don't think so.
There is a requirement that you need to be able to get off the pad quickly if there is a launch emergency.
And, of course, on the real launch day, unlike the simulated countdown where the pad is crawling with people, on launch day there are very few people, even when you're getting into the vehicle because it is fueled.
And then everybody else leaves.
So they have this big slide wire where if you have to get out of the orbiter in a hurray you go out and you jump in the bag, hit a little guillotine and it cuts you loose and you slide all the way down.
Actually, in the early days, the astronauts kept saying we need to try this out.
And they said no, you cannot do it because it hasn't been man rated.
We said wait a minute.
Well, it's rated for emergency use only so you cannot try it out.
And then after Challenger they insisted that somebody actually ride down in it.
Not every crew.
Just one person tried it out.
But the other thing that you do, once you get down to the bottom there is an underground bunker that you run into.
And either you stay there and wait for help or they have these armored personnel carriers sitting around there with little tanks which you get in.
And you've got to be able to drive your tank.
They have a breakout section of the fence so you drive the tank through the fence out to a helicopter pickup point.
So we all have to go out and learn how to drive the tank.
And we drive over the sand dunes and everything.
And then also they have a big pool which they light on fire.
And they give you a big fireman's hose and they show you how to make your way through a fire by squirting a water path in front of you.
All the little things you dream of as a kid, to be a fireman, drive a tank.
So it's good fun.
But it is part of the extensive safety procedures that they have.
And all these things periodically they have to exercise.
They do go through, every once in a while, a disaster drill down at the Cape where they simulate an emergency where they have to pick up the crew either outside the launch pad or either out in the ocean.
And you've got to keep people at full operational readiness.
That's one of the aspects of training.
We train for the anomalies, the nonstandard things.
The standard work, they do so much of that anyway you don't have to train for that.
You don't have to train to put a payload in the payload bay.
You don't have to train to power up your system and run your system checkout because that is a repetitive thing.
But the nonstandard stuff, when things go wrong, that's where the emphasis of the training is.
And, like Jeff said, it's a lot of fun.
It's fun for the launch team, too.
We have a control center at the Cape, and it is called the Launch Control Center.
You would think this is where you launch it from.
Well, you do that.
But all of the other work that is done in the preceding three months is controlled from the Launch Control Center by test conductors and engineers who wrote the procedures and the software that implement all the requirements.
And they control the activity on the orbiter or the tank or the boosters when it is still back in the offline facilities or the Vehicle Assembly Building or out to the launch pad.
They do it from the Launch Control Center.
We automate what we can from there.
But for the manual activity -- And a lot of it is manual because regrettably you cannot do all this refurbishment on the shuttle with automated systems.
It wasn't built that way.
It is all managed from out Launch Control Center with old computers but with a great team of people.
The software we use in the computers, I am not going to go into a lot of detail because it is really old stuff, but it works.
It is like the software program that the FAA uses to track airplanes.
It is hacker-proof.
There are no external interfaces outside of the Launch Control Center firewall.
It works just fine.
And that is where we do the management of our day-to-day operations from our Launch Control Room.
And we call it the LPS Launch Processing System software.
We've tried to change it a couple of times over the years.
Buy new hardware.
We have supplemented it with laptops so that we can do more human engineering displays, but the computers are the same ones that we bought back in the late '70s.
But then so are the computers that are on the orbiter that fly the machine.
So it works.
This is a bit repetitive, but I want to go through it again to make sure to emphasize what the role of the engineering is.
The designers that you've heard from, they certified their design.
And they did extensive testing, just like Detroit does on a car on the door latch.
And they've tested that door latch and said, well, this thing out to be good for 250,000 miles or 20 years or so many gazillion opening and closings of the door before it wears out and you've got to replace it.
Well, they did all that.
And to make sure that that hardware is still within the certification base, they developed requirements, sent them down to the Cape.
We put them in the procedures, either manual procedures or software to implement those requirements.
And the team, United Space Alliance the contractor and the NASA engineers will certify at each of the milestone reviews and the process of moving the hardware out to the pad they've met those requirements.
And they did that either by participating in the activity or by reviewing the test data.
And that allows us to sign a certificate of flight readiness a few days before launch that all the requirements have been met.
When we get into launch count, we take in a subset of those requirements that when everything is powered up and ready to go fly all of these systems that are active ought to look like this.
The voltages should be between this upper limit and this lower limit.
The pressure should be between here and here.
These indicators should show that they are open and these indicators should say that this latch is closed or the valve is closed or whatever.
And we put that in what is called launch commit criteria.
And that is the acceptable limit for the performance of that hardware when you're at that point which is nine minutes before launch when everybody commits that they are ready to go fly.
And all those requirements, again, are implemented by procedures in software.
And the three days of launch count have roughly, these are numbers, but about 500 requirements.
There are thousands of measurements associated with those requirements.
And most of the looking at those measurements is done by computers, obviously.
But you have the engineer involved because if it is not within limits you turn to the engineer for what is wrong.
Is there any chance to go fix this or do we have to change hardware?
Do we have to do trouble shooting?
So it's not a hands-off operation, obviously.
The structure of the launch team, the launch director is shown in the middle, but that doesn't mean he or she is the only one with a go or a no-go button.
The people the implement the requirements, there are roughly 150 of them in our Control Center, and they are the ones that provide the go, no-go for all the subsystems both flight and ground that everything is operating within the limits.
They report that to a team of test conductors, NASA and contractor, and that's reported to the launch director.
We are not along, obviously.
We have an engineering support area, which are all the hidden senior engineers that you don't see on television.
And there is a room full of about 150 to 200 of those.
And they are looking at the performance of these systems as they are activated in launch count.
And they've got all the trend data, the previous history of it.
And they're looking at things like, well, all right, everything is in limits, but is it within family?
The last time the system performed the voltage with this or the pressure was this.
Well, it's still in limits.
But it's not the same as it was before so we go look at it.
Well, that's for this group of engineers over here.
These guys and gals are running through their procedures.
And, if everything was within limits, we just turn to the next page and go onto the next one.
And they also have the emergency procedures.
If something bad happens these are the people that will implement the emergency procedures.
This offline support, they're out there.
They're the senior people, they've been there before, they've been out there who knows how many years and they're asking questions like that didn't look right or we missed that step, go back and check it again.
That's what they are for.
The mission management team over here, these are the people that do the certificate of flight readiness.
These are the folks that stand up in the readiness reviews and say I gave the Cape folks a good set of requirements.
There is nothing that happened with the hardware that they're responsible for.
And all those offline facilities are on previous missions.
That shouldn't have any cloud over this mission.
They are responsible for certifying that.
And they do that through a structure of the mission management team.
You've got, of course, the flight director with flight rules and the flight team that they look at during the launch count process.
And you will hear more about that from Wayne Hale.
And they, obviously, have a go, no-go input to the launch director in the decision process also.
We have an integration activity.
They system engineers are each responsible for, you know, they have boundary conditions on what they are responsible for.
It is these pieces of hardware.
It is this wiring.
It is these actuators or latches or whatever.
But a lot of that stuff fits together.
Integration is a technical term for making sure that this person talks to this person and that they are hooked in with this one.
That activity is done by a console in the Control Room, again, with senior engineers who actually manage all the automated software that runs the last couple hours of launch count that really launches the vehicle.
And they are tied in with that.
You also have the more subjective stuff.
We launch on a public range.
Public safety is obviously an issue, so the range has to make sure that it is safe to fly from a public standpoint, there are no boats in the launch danger area, no airplanes and that the weather meets the criteria of being able to track the vehicle should it go off course.
And, of course, the payload has to certify, particularly if there is an active payload, that all of their launch commit criteria is met.
And last, but certainly not least, you have safety oversight of all of that activity.
And what I don't show on there, of course, is the flight crew, but they are obviously in communication with all of these people.
So that is the network.
But everybody out here has no-go authority.
Anybody can come on the Net or press their switch and say I've got a problem, we're not going anywhere.
And it is the launch director and the launch team's responsibility to make sure that problem is addressed, hold at the convenient, graceful to stop all the activity hold point until the issue is resolved.
The real job, and I'll get into this later, of the launch director is to say no when everybody else wants to go because, as you get further into the launch count, the "launch fever" process sets in.
It is a natural thing, it seems to be.
People want to go.
I want to ask you about that a little bit, I know we'll get into it later on, but flight rules and the launch decision, there are flight rules that say you must scrub if these conditions are not met.
Right.
And, yet, there is a tendency to allow judgment to come in.
Let's take an example of a limit on crosswind at you landing to the return to launch site.
You see it go above limits, but the rules strictly says no, you cannot go.
Talk about that a little bit.
That is what we use the mission management team for.
And you bring up the weather example.
Most of this stuff is pretty straightforward.
It is either the voltage pressure temperature is in limits or out of limits.
A person can question that and say I don't like the trend, and you've got to stop and clear the air if they say I want more discussion on this, I want more data review and that sort of thing.
If it is a discussion like that involves activity outside of the Control Room beyond the purview of the console operators that are running through the procedure, we rely on the mission management team to manage that activity so the console operators can concentrate on their launch commit criteria and their procedures and their software.
Something like crosswind limits, if there is a debate with the flight director and the weather people, we hold the clock and say mission management team, that is yours.
You manage it, and whatever you have to do to resolve the decision, we'll scrub if need-be, we'll hold if need-be, or if the community can get comfortable that these crosswinds that are peaking occasionally out of spec, that it's OK to go fly anyway, then we will wait to hear from the flight director that that's OK. We don't insulate the Control Room from that, but we don't want to burden the control center with work and what is an offline issue.
And it is up to the mission management team to disposition that.
And you bring up weather because weather is one of the few things that there is really a lot of judgment still involved.
Is the weather good enough to go fly?
And it is probably the only thing that is in that category because everybody has an opinion on the weather.
And Florida weather is dynamic.
And usually, unless it is wintertime, there are clouds.
And there is always a tendency, if you've held for weather or the weather is marginal, to hold longer to see if the weather gets better.
And the real issue for the launch director is to try to convince people, if the weather is good enough, that this is good enough.
And, yeah, you could wait longer for it to get better but it's good enough.
We're running out of window time.
All the time we sit here on the ground with all these systems humming along, the probability of some glitch or something coming up to scrub you is increased.
And just as a personal note, when I was I the Air Force, I was a weather forecaster for Missile Operations.
And I felt that that was a waste of an engineer's time because I was a graduate engineer with my diploma sticking out of my pocket.
I wanted to go launch missiles and rockets, and the Air Force said Lieutenant Sieck is going to be a weather forecaster supporting Missile Operations.
I endured that for a couple of years and put all of that experience in my hip pocket.
And I didn't need it until I got into this launch director job.
And I spent many hours talking to the weather people who provide the forecast for the launch going over all of the data and the technical aspects of the weather situation to see if there it was prudent to sit here any longer and wait for the weather to improve or whether it was better to scrub, get these guys out of their uncomfortable suits and send them back to crew quarters and recycle for the next day.
And they won't necessarily scrub, but you have to disposition their concern.
And you will hold until their concern is taken care of.
Yes?
This is unrelated, but you know how the Russians, in going out to the launch pad, use a rail track and they have the rocket flat?
Can you do that with the shuttle?
Instead of stacking it like this and then rolling it out, would it be better, easier?
Well, the problem with that are the solid rocket boosters.
They weight approximately, when assembled, a little less than three million pounds a piece.
And the joints that caused the Challenger accident are critical.
And I don't know if they went through the technical aspects of the design, but these things are 14 feet in diameter.
Each segment weighs 300,000 pounds and they have these two cleavaces that come together with O rings in there.
And what caused the Challenger accident was there was rotation in these joints to the point where the O rings, because of cold weather, weren't making a good seal and caused a gas path.
And 1200 degree temperature.
Exhaust plume just cut through the metal once it found a path.
So you want to maintain the integrity of these joints.
If you stacked them horizontally, and that could be done, and then you lift this thing vertically, the joints aren't certified to maintain their integrity doing that.
Plus, you'd need some kind of huge crane to pull this 3 million pounds from horizontal to vertical.
I think it's something that could have been done, had it been built into the design at the beginning.
And the Russians have a tradition of doing this.
And so that is the way they design their rockets.
For whatever reason, we never started that way.
From the very beginning, our rockets got stacked vertically.
And that's the way all of the American rockets have been designed.
Yes?
[AUDIENCE QUESTION] The launch director, whether it is flight worthy, actually, that mission management team is responsible for the flight worthiness because they encompass not only watching what you did in launch count, but the certification of all this hardware before it even got to the launch pad.
And, ultimately, the launch director is responsible for conducting an orderly launch and making sure that this mission management team doesn't have any issues, that that engineering team doesn't have any issues, that the console operators have really completed their procedures.
That is their responsibility.
And, if there is any fuzz on that, so to speak, even though console operators say they're go, and the mission management team says they're go and flight says they're go, if there is any concern about that, it's the launch director's job to say no, we're not going to go fly today.
We are going to give you another day or two days to do more homework, look at more data, have more discussions.
We're going to scrub today, even though everybody may say we're go.
And that has happened before.
And I always wondered, after not too many times, when I've said no, when everybody wants to go, whether I'll get the phone call saying well, I'm glad you made it safe today, Sieck, but you don't have this job tomorrow.
Well, I never got that call.
Never did.
The process, not to bore you with it, but since it is an operation, normally we try to launch in the middle of a week, Wednesday or Thursday.
We usually take the weekend off before that to clean up everything and give the launch team a rest, so we power up everything.
And that's the way we start the process.
And that's a couple of shifts worth of work to bring up the ground systems, the flight systems, make sure that all the avionics hardware really comes to life.
And we put in a no work hold at the end of that to take care of any problems.
If that all goes well then we get into the more critical activity that has time constraints associated with it.
We load the fuel cell.
That is the power plants and the orbiter reactants, liquid hydrogen and liquid oxygen, because they have limited life.
Even if you don't have the fuel cells activated that stuff boils off and you lose it and that could count against your mission capability.
And that is a hazardous operation also.
And then we put in another no work hold at the end of that.
And then we have a period of time where we bring up the final systems, put in the time critical storage, particularly when we're flying laboratories and you put in plants or critters or whatever it may be either in the crew module or the payload bay.
And the last 12 hours down here is when you really have got to make sure this is a real decision point right here.
Are we going to load the external tank, put a cycle on the tank?
And we know that putting liquid hydrogen and liquid oxygen, and there is roughly half a million gallons in a tank, puts stresses into the hardware.
You really don't want to do that unless you're pretty sure you're going to launch that day or that night.
We have a meeting before that to make sure the weather looks pretty good and that the mission management team is not working any offline issues.
And that has happened before.
A war story for you.
In the early '80s, in one of our missions, we assembled the mission management team to decide whether or not we ought to go load the external tank and go fly.
Everything looked pretty good.
And the orbiter project manager said well, I need to make you aware of this.
I just got a telegram from the manufacturers of the tires.
And the manufacturer said they were inspecting a lot, a lot of the tires of which we have two of them from the same lot on the Orbiter, and they found some blems on these tires.
And they don't know whether it's a manufacturing flaw or an aging defect.
And we don't know whether it affects the integrity of the tires.
But since we knew you were launching tomorrow, we thought we would share this with you.
Warm regards, BF Goodrich, I think it was at the time.
Well, the mission management team threw on that on the table and said hey, we've got to go work this.
These people, we don't know, they haven't said they're not certifying the tires.
They just said they've got a problem.
Well, that's fine.
The mission management team, go work that and let's talk about whether or not it's prudent, under these circumstances, to go load the tank with some probability that we get all the way down to T minus nine minutes and you folks are going to say hey, we need to look at more test data on these tires or have more discussions with BF Goodrich on whether or not we go fly today.
Well, you don't want to load the tank and go through all that if that's what you think the situation is going to be eight hours away from that.
And you've put a cycle on the tank, you've used up a lot of cryos that you cannot save because of the heat leak.
Plus, your launch team has put a cycle on, depending on the time of day, they've been up all day, all night or whatever.
Those are the kinds of issues that you need to make sure get worked.
And the mission management team has to handle that, but the launch director has to decide just go work that for another day or two.
We're not going to load the tank.
Or, what's the promise of this coming to fruition, the possibility so that it makes sense to go try to fly?
What did you do with the tires?
In that case, we went ahead and tanked.
The subsystem manager for that, which you may have already heard from, went back to a lot of the test data that they had run on the tires, that they had been fairly recent.
Langley, I think, was involved in it.
And they said we don't have to worry about those blems being a problem for the tires that are on the Orbiter which, by the way, you don't have any access to at the launch pad.
If we were going to change the tires, you would have to roll back to the Orbiter Processing Facility.
And that would be a couple of month's worth of impact to that flight.
We ended up not doing anything with the tires, but it had the mission management team really busy for about eight hours.
And this process takes about three days.
And we have refined those procedures, as you might expect over the years, to make this as an efficient and repetitive a process as we can.
In addition to minimizing the hazards to the people and the equipment.
Before we got to terminal count, usually we take like a two minute break.
This is a good time to do that.
Terminal count phase.
After the tank is loaded, half a million gallons of liquid hydrogen, liquid oxygen, and there are no leaks, and we have leak detectors all over the vehicle, and we've scrubbed a lot of launches because of leaks, we give the crew a go to come on out.
And a crew of seven or eight people, it takes an hour and a half to get them all in and get them connected, check their communications and that sort of thing.
Roughly three hours before launch we bring them out, we get them all connected, we close the hatch and we verify the integrity of the crew module.
And clockwise we're down to T minus 20 minutes, but if it's a rendezvous window, and most of them are now.
The launch window is only five to ten minutes long so we have to be in phase with the same plane that the Space Station is in and we've got to get in phase with being able to catch the Space Station or have it catch us.
The launch window, we don't have enough energy in the Shuttle to steer to any orbit we'd like to.
The launch window is only about ten minutes long, so we put in a long hold at T minus nine minutes for those kinds of missions.
It is the equivalent of the two minute timeout in a professional football game.
Everybody has a chance to review their information, discuss their strategies, if it's a long launch window, if it was just a lab mission then we only wait for ten minutes at this T minus nine mark.
But, if it's a short window, we set up the clock such that we hold there for almost an hour.
And then, as you saw on that previous chart, we do a poll of all those people and make sure that they are all still go and they're not working any problems or issues.
And then we start this automated software program that looks at, in addition to the systems engineers and their consoles looking at all their information, we have an automated program called the Ground Launch Sequencer that looks at all the measurements, all the parameters to make sure that they are within limits.
And it issues all the commands to the vehicle and the ground support equipment.
And we did that so we could manage the repeatability of that process.
And you had to take the human factor button pressing out of the process as much as possible.
And there is not a lot of work that goes on in there.
And there is very little manual work.
You get five minutes.
The orbiter access arm, their ability o get out of the vehicle in a big hurry is retracted.
It takes two to three minutes to pull that back, but we can get it back up to the vehicle in less than 30 seconds which is about the amount of time it takes them to unstrap and get out of there anyway if there's a fire or something bad like that, that they have to do an emergency egress.
They start the propulsion units on the Orbiter that pressurize the hydraulic system.
Just like on an airplane, there is an automated test of all the aero surfaces like they do on the end of a runway.
But this is done by the computer and it wiggles the elevons and the rudder and the main engines to make sure all of that works.
And we also start the conditioning of the propellant at the engine interface.
And they already had the discussion about the main engines.
You have to make sure that the quality of the fluid in the external tank and the quality of it that's right there at the injector of the engine is within the temperature and density parameters that the engine was certified at.
Because we have to drain the propellant out of the fill lines.
And, once we've started doing that, that propellant starts heating up right there on those valves that are going to go into the injector and start the main engines on the Orbiter.
That limits our ability to stop and hold after that.
We are good for four to five minutes after that starts.
And that process starts at T minus five minutes.
After that point in time, we are either going to launch or scrub in the next ten minutes.
And then we pressurize the oxygen tank and we pressurize the hydrogen tank, but anywhere in this period of time we can stop.
If an engineer or the automated sequencer says something is out of limits, we will stop at those milestones and disposition that problem if we can.
But, per our rules, we won't waive any requirements at that point.
Even though an engineer may have a perfectly good explanation as to why this temperature, pressure or whatever, our limit is not being met, if the launch commit criteria, the launch rules says no, it has to be like this, then we're not going to go launch that day.
We won't.
After 31 seconds, the solid rocket boosters come to life.
The propellant doesn't ignite, but their hydraulic system is powered up.
Their avionics systems are powered up.
Their engine nozzles are checked again in an automatic sequence.
And at 10 seconds, if everything looks good and all these measurements that our Ground Launch Sequencer has been looking at, we send up the computer data bus a go to the onboard computers that said all of our stuff has been satisfied.
Now, we're still looking at a few things.
We're looking at those bolts that are holding the solid rocket motors to the launch pad.
And the ability to blow the nuts that are holding them, should that system fail, we would send the cutoff to the onboard computers.
Regardless of where the engines were in their startup sequence, the three main engines, everything will stop.
And hopefully it will gracefully come to a stop and we will safe all the systems and go from there.
But that is the final handshake that is preplanned between flight and ground is that ten seconds when we send a command up to the computer saying we are all go.
We can still shut you off, and we can shut you off manually, too.
If a console operator sees something happening that they think could compromise the safety of the launch, they can call for a cutoff.
And we have a switch that says cutoff.
It sends a command to the onboard computers, and it will stop everything within 10 to 20 milliseconds.
But you don't like to put humans in a position to have to make that call.
And that's why all of the critical stuff is done by the computers.
And I mentioned the Ground Launch Sequencer.
That's a console and operators in the back of the Control Room.
And they are the ones, by the way, that determine nobody presses a button for launch.
A woman in the back of the Control Room types the liftoff time into the computer when the launch director tells her, hey, look, it looks like we're going to come out of our T minus nine minute hold at this point in time.
So here is the liftoff time.
Put that in the computer.
And after that it's a hands-off operation.
Can I ask, are these commands being physically sent, I mean they're hard line cable?
It's what we call a launch data vest.
It's a cable that is connected from the computers in the Control Room through the liftoff umbilical into the Orbiter computers.
It's a data train.
And when is that connection broken?
At liftoff.
The onboard computers will check the health of the main engines, and if the three main engines are within all of their operating parameters, the turbines and the pumps and the temperatures then it sends a command to those eight bolts that hold the solid rocket boosters to the mobile launch platform and the bolts that hold the external tank vent arm to the tank.
And the two Orbiter umbilicals that says fire the nuts, we're going to go.
And that is orchestrated by the onboard computers.
And when that happens you're flying, you're going.
The command to ignite the igniters on the solid rocket motors is sent at that time also by the onboard computers within a couple of milliseconds.
So all that happens at once.
And it's an onboard automated thing.
I talked about the human factor that repeatability is important.
And I mentioned I gave the engineers, you know, I always threw rocks at them because they always want to change things.
It is the launch director's responsibility to manage.
They're the owner, so to speak, of the launch count procedures, the 5,000 pages of documentation and the 500 or so software programs that are executed in the launch count process.
And we discouraged changes to that for obvious reasons.
Unless there is a modification to the flight hardware or the ground hardware or we found something in our previous launch attempt that said we need to fix this or there's an opportunity to increase the margins, we don't change it.
We try to maintain the procedure and the hardware the way it always has been.
And it essentially is.
We do training, obviously, but we train for the nonstandard work.
Just like in the flight team, we have a simulation supervisor that throws diabolical failures out there and we have computer programs and into the control center we bring the launch team, power up the consoles.
And their displays look like the shuttle is at the pad and it is getting ready to launch and we throw failures at them to test their reaction and to find bugs in our safing procedures.
And we've done that over the years.
I think the next generation of vehicles will probably have more automation.
And that will probably be a better thing but you will never take the console operator out of the Control Center either, the Launch Control Center or the Flight Control Center.
You need the person there to disposition the problems, but predictability is what we're always after.
And stable procedures, be they flight or ground, give predictable results.
And that is why we automate.
And we control change, too.
Responsibility.
I talked about the engineers.
I didn't mention, I don't think, technicians and inspectors.
But when a technician stamps a procedure that I torque that bolt and the inspector stamps that same procedure and said I saw him torque that bolt, that is their warranty that that event really happened.
And that paper trail and that warranty is very important.
When an engineer signs a procedure that tested the Flight Control System that said I reviewed all the data, didn't see any glitches, everything met specifications, that signature is his or her warranty that the procedure ran correctly and that they understood the requirements that were implemented by that procedure.
And the requirements were met.
And that is important.
Our whole concept of launch readiness is based on people being responsible for their system, be it the designer that signs it that says there is nothing going on offline, there are no faxes out there that say there is a cloud over the tires.
Or, my last test of the main engines at Stennis, there was nothing there that says these three engines that are on this orbiter shouldn't work just fine.
And they do that in a flight readiness process.
That is their warranty and responsibility.
And we hammer this home.
And it probably comes from, I had this quote in my office for years and I am going to bore you with it, and maybe some of you can guess where this came from.
Responsibility is a unique concept.
It can only reside in here in a single individual.
You may share it with others, but your portion is not diminished.
You may delegate it but it is still with you.
You may disclaim it but you cannot divest yourself of it.
Even if you do not recognize it or admit its presence, you cannot escape it.
If responsibility is rightfully yours, no evasion or ignorance or passing the blame can shift the burden to someone else.
And this is what we push home.
And, by the way, this quote was Admiral Rickover who was testifying before Congress after the first nuclear submarine accident.
And he was saying I am responsible.
It is my program, it is my submarine, I am responsible.
Even though somebody else was commanding it, it doesn't matter, I am responsible.
And we impress that on the managers, the engineers, the technicians, the inspectors, you know, you're important.
And the work you do is important.
And when you sign or stamp that procedure or give a go on the Net that is your warranty that everything is working and you understand the requirements.
Very important in the business that we're in.
Decision making.
On the chart.
This structure has been in place since the Mercury program.
There is nothing new here.
And the only issue comes in when there is a gray area.
Who makes the call and how much judgment is involved in the call?
And I gave you the example of weather.
Everybody has an opinion on it.
It can be judgmental.
Well, the weather decision is if it is launch weather related it is the launch director's responsibility to determine if it's good enough.
If the flight director has an issue with the return to launch site winds, that ultimately is his or her decision.
The people may get involved in it and they may have offline discussions with the flight director, but it is still ultimately his decision.
And that is the way it has to be because they are responsible.
This mission management team is not responsible for the go, no-go decision on weather on launch day.
It's the flight director for the flight stuff.
It is the launch director for the launch-related stuff.
And it is set up that way so there is no fuzz on it.
Other people can have an opinion, other people can have input, but unless this person up here says launch director we're no go, it is the launch director's decision in that case.
Communication.
Real important.
And I will give you an example of that.
Apollo 13.
Most of you probably don't remember Apollo 13, but Apollo 13 was like Challenger and the Columbia except we didn't lose the flight crew.
But it was just as disastrous an event.
And by all rights the crew shouldn't have made it safety back to earth, but thanks to the heroics of the flight crew and the flight team and the timing of when that event occurred they were able to get back home.
But that was all caused by activity that was done on the ground well before the launch.
And, specifically since I was involved in that, I remember this like it was yesterday.
Of course, part of that is an old age thing.
Stuff that happened 40 years ago comes in loud and clear.
Stuff that happened four days ago, I have a hard time remembering.
But we had a tank in the spacecraft that we ran our simulated launch count.
And back then we put propellants in the vehicle, all the hazardous stuff and detanks.
And we couldn't detank the liquid oxygen out of this tank.
They had the fuel cells on the spacecraft.
And just a little more history.
The tank had been dropped when it was installed back in California, but engineers and managers did some tests and some rationale on saying well, it's still OK to install it.
We got it down the Cape.
We put the liquid oxygen in it and all the rest of the system.
We couldn't get it out of this tank.
And after reviewing it they found, well, the stand pipe in there that is used to drain it on the ground was probably bent or damaged and that's why we cannot get it out.
So the managers and engineers got together for a few days and said turn the heaters on in the tank and vent the liquid oxygen off.
Heat it up and it will boil off as oxygen gas through another loop that takes it through the fuel cells and out vent on the vehicle.
Use the heaters to go do that.
So they developed the procedure and we went on station at night a week after we had the anomaly and turned everything on and turned on the heaters.
And it is fine.
You can see the liquid is turning to gas and it is coming out.
And the console operator in our Control Room said hey, Sieck, I cannot monitor the temperature in this tank anymore.
Well, why is that?
Well, the range on the temperature is from ambient down to minus 300 degrees because it's measured in liquid oxygen.
And it is upper limits.
It started heating up, and now I cannot tell you whether it is 50 degrees or 350 degrees.
I said OK, stop.
So we stopped the test and got all these managers involved, including the people that built the tank and say hey, we've lost visibility.
We're putting energy into this tank that has liquid oxygen in it.
We cannot monitor the temperature anymore.
The pressure seems to be fine.
They said don't worry about it.
There is a thermostat in there that is heat sensitive, and it will open up the power to the heaters if it gets to some limit in there.
Just keep your eye on the pressure.
Make sure the pressure doesn't get too high.
Well, of course the pressure isn't going to get too high because the vent is open.
So we said fine.
We turned the ground power supplies on and let the thing cook for 10 to 12 hours.
And what we didn't know was that the thermostat wasn't certified to the voltage we were using from our ground power supplies because we didn't tell them, although our requirements allowed it, that we cranked the power supplies up to 50 volts.
And why did we do that?
Well, the more energy you put into the tank the higher the heaters get and the quicker the liquid oxygen boils off.
You know, high school physics.
But we didn't tell them that.
And the people that built the tank and built that thermostat had only certified it to the voltage of fuel cell supply which is half that value.
So the thermostat welded together, the points did, and we had continuous power going on in there for eight to ten hours.
It got to 800, 900 degrees in the tank.
We burned all the insulation off the wires to the fans and the heaters.
And, from that point on, we were just waiting for something to happen to make those wires come together.
And if you didn't turn the power on you're going to get a spark in an oxygen tank.
Not a good thing.
Well, that is what happened a day into the mission when they were heading to the moon.
And they turned the heaters and the fans on to stir the cryogenics because they were worried about stratification and zero gravity.
And they got the spark and it blew up the service module.
The point is that communication, we wrote down in our procedures what we did, but we didn't communicate that on a real-time basis.
And, although we reviewed the procedures afterward, as the rest of the team did, the designer who was 1500 miles away who had approved what we did over an intercom system didn't review our paperwork.
You hit that thing with 50 volts, it's only certified for 25 volts, even though our requirements allowed us to use 50 volts.
It is a case where requirements didn't preclude us from launching hardware then in hindsight was flawed.
But, on the other hand, communication, we didn't tell them in real-time.
And, as a result, Apollo 13 happened.
So I learned that lesson early on.
I mean even if you're boring them with what appears to be minutia, you're going to tell them everything and document everything.
And, thankfully, they went through our procedures and said well, there it is.
And I thought we'd get fired.
And actually we got patted on the back for documenting what we had done so it made it easier to zero in on the root cause of the problem.
And, as a result, we fixed that and we were able to fly again in a few months as opposed to having to go through a Challenger or Columbia type of activity.
Very important.
Launch fever.
The human factor.
It is hard to describe what the environment is in the spacecraft a few minutes before launch or in the Control Room.
But there is a lot of tension and there is a lot of energy.
And everybody in there, for the most part, has been awake for the last 10 to 12 hours.
And they came to work that day or that night with all hopes of safely launching the Space Shuttle.
And they really don't want anything to go wrong with their system or somebody else's system.
I mean that's the mood.
Because, if something goes wrong, you're going to come back tomorrow or the next day or the next week.
So they really want to hear this everybody, you know, when they do the polls they go, go, go, go.
And the hardest thing is saying no when everything sounds like it's go.
And there are a lot of times when we've had problems with ground support equipment.
It is old.
I wish we had all new stuff for Shuttle.
A lot of this stuff is the same we used on Apollo.
And, in spite of our efforts to maintain it, it is all out in the corrosive atmosphere at the launch pad.
And that stuff constantly gave us problems.
And we would lose some of our redundant systems and some of our primary systems.
And more than once in launch count, when we'd have a problem out there, the team would come to the launch director through that organization chart you saw and say we've got an idea.
We think if we patch around this and then hook this up to this and we throw these switches and we get this result that it will be OK and we can go fly without this power supply being active or this purge system operating normally and all these other things that we've got out there.
And what you have to do as launch director is say well, just sit back and analyze whether or not -- I mean you rely on your engineers to use their Yankee ingenuity to fix a problem, but you have to assess whether or not they're literally shot-gunning it and coming up with the proverbial quick fix because they want to go home in an hour or so and celebrate a good launch.
Just like these guys want to get out of those uncomfortable suits they have been sitting in for the last couple hours and either get out of the spacecraft or get up there in zero G. As a launch director, you have to assess well, OK, but is this really the right thing to do or are they working this too hard?
If you gave them another 24 hours to look at the data and think about their approach to a fix would the answer be any different?
And there have been a few times when they said well, that sounds pretty good right now but I am going to give you another 24 hours to think about it so we're going to scrub for the day.
And you can hear the big groan in the Control Room.
And you can just see it going through their heads, well, this guy has lost confidence in us.
I mean we all came to work, we get paid to solve problems and get things done, and he is telling us that ain't a good answer, that ain't a good fix.
So you have to balance that when you're in charge, but the real job of the launch director is to say no when everybody else wants to go.
That is really what you are there for.
Because these guys get tunnel vision.
They want to go fly.
The console operators, things are on automatic in that last nine minutes, and they just hope they don't get that little red or yellow light on their screen that says hey, you better go look at this, it isn't working right.
The last people you want to ask is it OK to launch is the crew.
They'll say yes, no matter what is happening.
And we do ask them, by the way, but it is a formality because we know what the answer is.
Unless he's got a fire extinguisher up there putting out something in the spacecraft, he is going to say we're go.
And that is also why we put in a rule that says after five minutes, for these critical launch commit criteria items, the 500 that you absolutely have to look at and certify, you don't proposition us with a change to that after five minutes because we're not going to entertain it.
If they're out of limits we're not going today or tonight.
It is going to be at least 24 hours.
A long time.
But the team that does it, I mean they are good, they are professionals.
And we talk a lot about the Apollo program and how great things were back then, but I just gave you the example of Apollo 13.
The Shuttle team versus the Apollo team has a much higher degree of difficulty situation to deal with.
There are fewer people on Shuttle, yet it's a more complex vehicle than the whole Apollo system.
The Saturn 5 rocket and the spacecraft were simple compared to the Shuttle system.
And we had 25,000 people at the Cape in the Apollo program flying two to three times a year.
And in the Shuttle program back in the early '90s we flew seven times a year with 6,000 people with a vehicle that is more complex, older.
In the Apollo program our ground equipment was new and all the flight hardware was new.
It wasn't reused.
It was fresh out of the box factory pristine stuff.
And requirements were easy, you know, it had to act and look brand-new or you replaced it.
The Shuttle, because it is 20, 30 years old, depending on which orbiter and the ground equipment which is 30 to 40 years old, there are requirements that allow fair wear and tear.
The engineers are constantly pushed with is this good enough?
These wire bundles are frayed and there are dings in this line and that weld is looking like it has corrosion on it, is it good enough?
Well, they are constantly propositioning where back in the Apollo program you didn't have to mess with that gray area stuff.
It either looked or acted brand-new or you replaced it.
And replacing it was easy because money was unlimited.
As a journeyman engineer on Apollo I was told whatever you need you can have.
System engineers seek more test equipment, you need more technicians to be trained on this over here, whatever you need just ask for it.
Money is not an object.
As a manager on the Shuttle program when I finally got promoted out of the job I liked to launch director, I had to tell the journeymen engineers this is all the money you get for next year.
And be real efficient with this and frugal because next year you're probably going to get less.
And your strategy and approach to issues is much different if you have those two different environments to deal with.
And finally the other thing that Shuttle deals with today versus Apollo is the acceptance of risk.
I mean I already mentioned Apollo 13.
Back then if you made a mistake you were patted on the back and told hey, nice try.
Now, is there anything that can be done to better enhance your probability of success?
Whereas, in the Shuttle program, risk aversion in this country, I don't want to get started on that tangent, is becoming more and more a lifestyle.
We don't want to do things we might lose or we might not win or we may not be successful.
And risk, you know, there is less tolerance to it.
And, when you have a problem like Columbia or Challenger, you get half a dozen boarding parties coming into NASA saying here is what you need to do differently or this is wrong with your agency and this and this and you need to go fix that.
And, again, back in Apollo, it was what do you need to be successful?
Much different.
And it makes it difficult to operate.
Bob, could you comment upon risk aversion and the launch director and whether there were more aggressive and more conservative people who have had your job?
Well, I would say, before I was launch director, there was one.
And he was more aggressive than I was.
He was the console MANOPS person.
And he would push, and he would push until you would say uncle, I surrender or whatever, and then he would make a judgment call.
I was more conservative than that.
I trained the one immediately after me.
And he was pretty much the same but he got the highly visible job, management say that this guy was good, we'll take him away from here and go put him over here managing this large organization and worrying about budgets and contracts.
That is the same thing that happened to me.
And that's unfortunate, but NASA's strongest suit is not succession planning, unfortunately.
It just isn't.
Yes?
Are you saying the Shuttle program is too risk averse?
Because you also said that, for example, it is your job as launch director to be conservative and to say no when everyone else is saying yes.
No, I wouldn't say that the Shuttle program is risk adverse.
I would say our society is risk adverse.
And you see some of that permeating into the Shuttle program, but not to the point where I would say it is detrimental.
I mean asking questions, digging in to understand what your risks are is an important thing to do.
And early in the Shuttle program we didn't do that.
We had a lot of confidence in hardware after the first few missions.
It's just like a new car.
You put 5000 miles on it, you get the bugs out of it and you fully expect it to last another so many hundred thousand miles.
Well, that was the same expectation in the Shuttle program.
You fly a few flights and you learn that the tiles really stayed glued to the Orbiter and the thermal control system really responds as it should on orbit and managing the temperature and the coolant leaks and that sort of thing.
And you build up confidence and you say it should work now.
We got the bugs out of it and we can go fly this thing as often as those guys at the Cape can turn it around.
But what they didn't consider and what all these engineers, with all due respect that you heard from that certify their systems, they didn't totally capture the environment that the Shuttle sees over the long period of time in their initial certification.
They didn't capture it.
And that environment includes not only what happens in space or the calendar exposure to just time on some of these systems that have soft goods in them, for instance, O rings and that sort of thing, but it is the environment of the Cape with people constantly removing hardware, disconnecting things, moving wire bundles out of the way to get access to this or that other component to implement the requirements that were levied on the Cape to do the tests and inspections.
And they are very invasive, and collateral damage is a way of life when you're crawling around in the Orbiter.
To put in policy that tries to compensate for the fact that the certification didn't capture that environment that this hardware has seen for the last 20 to 30 years, to reduce the unknowns involved in that, that is the right thing to do.
You're trying to reduce the risk of reflying a very complex vehicle.
So that is fine, but you take that to a limit.
I mean you can talk yourself into never flying which would be pretty easy to go do.
And then this recovery after the Columbia accident, that was starting to happen.
The managers, because they had this investigation board report that says NASA, you've become complacent and overconfident.
And, by the way, in my opinion that is somewhat of a bum rap to say that NASA in general, and particularly the Shuttle program, had fallen into that mode.
But, to have the pendulum swing the other way to understand what your risks are and then you can have knowledge on whether you decide to accept it or not, that is a good thing.
What didn't happen after Columbia, but we did do after Challenger, is after we understood the root cause of the accident and some of the other factors like communication that we're part of that, reviewed all the systems and all the requirements, the managers said look, we're willing to accept some risk.
You people who are responsible for these systems and these processes at the Cape, you need to quantify that for us and bring it to us.
And we will either say OK, we won't accept that, you go back and change your system or change your limits or do more tests or whatever, or we will accept it.
We will say we'll accept that risk, management will accept some risk.
That happened after Challenger, and we were able to fly again roughly two years later after we recertified the design to the solid rocket motors.
That didn't happen after Columbia.
And you can speculate whether or not we could have flown six months ago or a year ago or we should have waited longer to fly because we still had a piece of foam come off the tank.
You could debate that.
But management never acknowledged that they were willing to accept some risk after Columbia.
They did say tell us what the risk is but we really want you to crank it down to zero.
We don't want to take any risk, unlike Challenger.
Now, is that bad?
See, what really happened with Columbia was the mission management team's discipline had eroded.
There were communication issues.
And, if you're a student of that investigation, they didn't grade out very well.
But the investigation said there is complacency and overconfidence in the system.
Well, having been part of the system prior to my retirement five years ago, you never found any complacency or overconfidence with the engineers, technicians, inspectors or the managers that I knew.
Now, they weren't as smart as they thought they were or, in some cases, as smart as they needed to be.
But that is not the same as complacency or overconfidence.
Because the latter to me implies an attitude problem that I've got it, I know it all, I am smart as I need to be.
And, for the most part, you know, there were some people I didn't like because I thought he or she was arrogant or something like that, but I never sensed an attitude problem with the Shuttle team.
From the technicians, many of which I knew and grew up with, to the top managers in the Shuttle program, I never sensed that.
But were we not as smart as we thought we were or as smart as we needed to be?
Were there signs that there was something going on here that you ought to go fix?
Yes.
But, again, they didn't just blow off, so to speak, these problems we were having with foam on the tank and these other systems which, to me, would have been complacency or overconfidence.
Anything else?
We're just about out of time.
I've got two announcements.
I've been working with the Stellar people.
As you know, some of the pdf files that I've tried to load didn't load.
They cannot really figure out why, but I have sent them the files, they've loaded most of them on so hopefully almost everything is there.
We're continuing to work at it, but hopefully all the things that you need access to you can find.
I said there were two things, but that's the only one I remember.
Well, let me close with one note then.
I am glad you're here.
Thanks for your choice to pursue an engineering career.
Doing things that are hard, I have found that in my travels that unfortunately there are fewer people deciding to pursue the tough curriculums and the occupations that require a lot of work and have some risk associated with them.
But it is encouraging to see people in graduate school that want to do what I did.
I am not sure how much engineering I did in my 40 year career.
I did a lot of managing and a lot of other assignments, but I am sure you can accomplish a lot.
I had a grandmother who was born at the end of the 19th century.
And, when she was your age, there weren't any computers, airplanes, cars, TV and that sort of thing.
Yet, in her lifetime, she got to see her favorite grandson, Bobby, help put a man on the moon.
And I don't know what can be done in your lifetime, but you can make great things happen with the career you have chosen.
So thanks for choosing, really.
Bob, thanks very much.
A pleasure.
[APPLAUSE]
Professor Sheila Widnall.
We are really lucky that she is able to come and talk to us today.
She has been here at MIT since 1964 as a professor and then as provost, but she didn't serve very long as provost because she was assigned to go to Washington by the President to be the Secretary of the Air Force, which she did for almost five years, I guess.
Four years.
Then came back here.
And she is now an institute professor which basically means you can do anything you want, right, including come and talk to our class.
In particular, Professor Widnall was on the Columbia Accident Investigation Board.
And that specifically is what she's going to talk to us about today, along with just general comments about how this relates back to systems engineering, the design, operation of the Shuttle, and basically anything else that you would like to say because it is your time.
So thanks for being with us, Sheila.
OK. Well, thank you, Jeff.
As I understand the schedule.
It is two hours.
And if I don't finish, we still take a break.
And then we come back and finish, is that right?
Generally, the way we've run the classes is that we encourage students to ask questions while you're talking.
Absolutely.
That will probably slow me down a little bit.
Up to you.
Anyway, two hours, we'll just take it as it comes.
OK. Well, I am going to focus on the Columbia Accident Investigation.
I must say, and I'm sure I will say many times as I go through it, that it was an incredible experience to be a member of that board.
02:00 I get chills up my back when I see that.
The reason I show this is for two reasons.
First of all, I mean it was, in fact, an incredible event.
I am sure that all of you can remember where you were that morning when that happened.
I also show it because of the expression on the face of the principal mission controller, Leroy Cain, when he realized that the Shuttle was lost.
And if the Shuttle doesn't make it to Cape Kennedy, more or less on schedule, there is no engine at that point, you've got a problem.
And so he was in a windowless box in Houston.
And so he had no kind of external, although all the citizens of Texas knew what had happened, the guys in the windowless box did not.
So it was really an incredible emotional experience for all of NASA as an agency.
But then what happens after that?
Well, fortunately the Challenger Commission had anticipated the need for a method to call an accident board into being if there should be another Shuttle accident.
And so there were individuals who were more or less ex-officio by rank or title waiting to be called.
So, the fact of the matter is, that on Saturday, when this all happened, the Accident Investigation Board was pretty much called into being at that point.
Now, who were the Accident Investigation Board at that point?
Well, they were primarily government employees.
Not so much NASA employees.
That clearly would have been unacceptable.
But they were military.
Three Air Force generals, the head of Air Force Safety, a general from Space Command and a general from Material Command who had actually served as an executive staff to the Challenger Commission.
That would be General Barry.
So, they had some very good experience.
The board was chaired by Admiral Hal Gehman who had been the Vice Chief of Naval Operations.
He had also been the Chairman of the Cole Investigation.
I don't remember how many of you remember the Cole Investigation, but there was a big explosion.
It is in the Middle East.
I don't remember.
Where was it?
Yemen.
It was probably Yemen killing 19 people.
And there were other government employees.
Somebody from the National Transportation Safety Board.
Jim Hallock who is across the street here at the Department of Transportation, the head of NASA Ames and the head of Navy Safety, Admiral Turcotte.
It was a smaller board than it ultimately ended up being.
05:30 And I'm blanking on names because I am at that sort of age, but we had a Nobel Prize physicist from Stanford.
You have to have a Nobel Prize physicist on a Shuttle investigation.
Also, somebody who had been looking into issues of policy and history with respect to the Shuttle Program.
And I am not getting any of these names right so I'm not even going to try.
Pardon?
The class has met John Logsdon.
John Logsdon, right.
Thank you, John Logsdon.
Doug Osheroff was the Stanford physicist.
He was kind of a kick because at the beginning he was very media shy.
He just wouldn't talk to the media, but then he really got into it and he went way outside the box, which was kind of interesting to see how that all happened.
In any case, we were all pulled together and we were chartered to uncover the facts, as well as the actual or probable cause of the Shuttle mishap and recommend preventative and other appropriate actions to preclude the recurrence of a similar mishap.
Yes?
Professor Widnall, before you and Sally Ride came on there were no females?
That's right, no women.
[AUDIENCE QUESTION] I'm not sure.
Maybe a little bit.
But, actually, the day of the accident, of course everybody was saying what happened?
That's not what I was saying.
I was saying I wonder who is going to be on the board.
I actually had a strong sense that I, in fact, would be on the board.
Think about it.
Gene Covert had been on the Challenger.
I think, like you have to have Nobel Prize physicist from Stanford, you have to have an aeronautical engineer from MIT.
I think the other thing was, as Secretary of the Air Force, I was really in charge of accident investigation.
All accident investigations reported to me.
Well, you guys don't know a lot about accident investigations in the military.
But there are two accident investigations in the military.
One called a safety investigation, which is privileged, and you take testimony and you seal it.
And I will talk about that later because we tried to replicate some of that on the Columbia investigation.
You have that one and then you have the public investigation which you take testimony, people can be court-martialed.
There is an Accident Investigation procedure that goes on in the Air Force, and I had really been in charge of that.
In addition to be an aerodynamicist, which is always good, I was the aerodynamicist on the commission, I was a faculty member at MIT, I had been Secretary of the Air Force, I had been in charge of accidents.
Also, I had spent my last five years really being interested in space, safety and mission assurance in space.
I mean all these things kind of came together.
Sally Ride was obvious.
She was an astronaut.
And she had been a member of the Challenger investigation board.
She brought that historic knowledge.
I don't think that was the primary thing, but it is probably good to have a balanced group.
See, I actually did come on board pretty much February 18th.
That was when I came.
I think that one of the early issues that we faced was our relationship with NASA.
I think at the very beginning NASA had this image that we would work for them.
That was one thing.
And the second thing was that we would find the widget that failed and then we would leave leaving them to carry on.
Not with this group of people you don't do that, and not in this situation.
We very clearly and fairly early established our independence.
We removed a number of senior NASA people from inline responsibility for the investigation like Ralph Rowe.
Again, I'm blanking on these names, but the people who had been in charge of the program.
Of course, some of them quit even before we got there.
Linda Ham was right in the middle of things.
And we just said look, we're not questioning your integrity but you just cannot investigate yourself so we need to move you aside.
Go back to your normal responsibilities and bring in independent people who had not been so deeply involved.
And they will be the primary interface between the Accident Investigation Board and NASA because we really needed to use NASA.
I basically had very strong interactions with the NASA aerodynamics division.
And you will see the reasons why as we proceed.
But at the end of my interaction with them, which occurred at the end of June, and they presented their 450 page report to me with all the calculations and studies and things that they had done at my direction, I basically looked to the group and said well, I think I could probably give 22 PhDs at this point to this group.
Because we had really pressed the envelope in terms of improving the methodology that we used to analyze some of the details of kind of what I'm going to talk about.
But, in any case, the issue of independence was extremely important.
And I recall a lot of meetings with a lot of banging on the table and yelling and screaming before we finally got this all straightened out.
Admiral Gehman was an incredible leader.
He really just did such good work.
He had a sense of where to press and where to stay on pad and how to interact.
He really was a marvelous chairman.
Incidentally, he recently served on the BRAC Commission so he is somebody who is deeply involved in public service.
And I know, from my conversations with him, that he is a person who refuses to serve on any board of directors that has anything to do with the Department of Defense because he does not want to give the appearance of a conflict of interest.
And he is just a man of incredible integrity.
Obviously, I am a big fan.
In any case, we proceeded then to wrap our group together and begin our interactions with NASA.
12:30 And I will show that a couple of times.
And this happened in about 81 seconds, two seconds after launch.
I don't remember the mach number, but I think it was something like 2, 2.5, something like that.
This is the video.
And you see the piece coming off and smashing.
And I will show it again.
That video was pretty much available the weekend of the launch.
There was an understanding among senior program managers that there could be a problem.
Now, what did we know about it?
Well, there were video cameras at the site.
There was a video camera located here, a video camera located here.
A rather shallow angle to try to do any triangulation.
But, obviously, you use what you have.
And so, as a result of triangulated these two video cameras, we were able to make an estimate.
It's probably also worth mentioning that there were other video cameras but they were out of order.
Right.
And I was going to talk about that with respect to our recommendations.
One of our recommendations was that working video cameras were an absolute necessity for launch.
In other words, you do not launch if you don't have a reasonable set of video cameras.
And, by the way, you don't launch at night because then, obviously, the video cameras would not be any good.
That was one of our recommendations, what I would call a near term recommendation, return to flight recommendation.
Yes?
[AUDIENCE QUESTION] I don't know the answer to that.
Maybe you know the answer to that.
It is very interesting because I have been doing a lot of mission assurance and space accident stuff.
You almost always have a picture of a space accident mostly because they occur on launch.
So, in the case of the Challenger, we had excellent video.
We could see the little hole in the Challenger.
We could see the stuff coming out.
In this case, we had reasonable video of potential impact.
You can launch with an 8000 foot ceiling.
In which case we would not have seen the Challenger.
Yeah, at that point.
I don't think it's true now.
And I don't know how they followed up on that, but there is also a possibility of using aircraft if they cannot get the video.
In any case, it turns out to be an important issue.
And we made recommendations.
What we knew about the impact was we figured -- Of course you guys all know exactly what this is.
This is the left wing underside.
This is where the curvature changes, and so it was kind of right in here some place, what we refer to as panel seven, panel eight, panel nine of the underside of the composite material that surrounds the leading edge.
Now, I don't know if you guys have studied that composite material.
It is a truly marvelous material.
And I may say a little bit more about it.
So, we sort of knew where the impact had occurred.
Obviously, we were hampered by the shallow angle of the video but this was our best estimate.
The velocity was estimated to be roughly 800 feet per second.
And that was obtained from several different methods.
One was a computational fluid dynamics calculation of how would the airstream accelerate a piece of foam of that particular shape weighing two pounds and having it hit?
And the other thing was from the video itself.
We had pretty good correlation between the estimates based on the video and the estimates based on calculations using computational fluid dynamics, so we felt pretty comfortable about this velocity.
What did we know?
When I joined the board what did we know?
What kind of data did we have?
Well, this particular vehicle had some telemetry of some sensors in the wing and in other places.
At this point in the investigation, we had not recovered the flight data recorder.
The other thing that is interesting is that this is the only vehicle with a flight data recorder.
This is the vehicle that was considered to be a research and development vehicle.
All the other vehicles were operational.
They did not equip them with flight data recorders.
Now, I will get back to this point when I talk about the criticisms that we had of the agency.
But we took strong exception to the notion that the Shuttle had be declared to be operational.
And, therefore, there was no need to continue to study it as a research and development vehicle.
It was just like a piper cub or a 747.
That takeoff was a routine event and was not, in fact, a major event.
And even in this system, if a sensor failed in the flight data recorder it was not replaced.
So what we had on the Columbia was a wasting asset, but at least we had an asset.
We had a flight data recorder.
At this point we had not found it.
Where were these sensors located?
Well, there were a lot of sensors in the wheel well.
There were some temperature sensors in this region of the leading edge.
There were some temperature sensors back in here and different places.
Now, one thing that is significant about this is that these temperature sensors were connected to two different boxes.
Actually, it looks like three but what I'm going to focus on are these wires.
Some of the temperature sensors were connected to wires that were basically in this region and running along the wheel well.
19:30 That's one of the reasons why that was important and kind of how we began to hypothesize what had gone wrong.
Now, you see the time was sort of 8:44.
That is Eastern Standard, I think.
I was shopping, actually, at the time.
Between 8:44 and 9:00 is when all of this took place, so we're talking about 15 minutes of problems.
At 8:44 everything was fine and at 9:00 the vehicle had crashes.
20:30 Why did these sensors fail first?
It was because the wire that controls these sensors is located here in this sort of critical area, so that was the wire cutting.
Yes?
You said the other orbiters don't have flight data recorders.
Right.
[AUDIENCE QUESTION] Commercial, that's a whole different mindset.
Yeah.
I mean those carry people and it is an FAA regulation.
21:30 That the wires were being cut.
And so we could follow the failure of these sensors right up to the point when the vehicle, we had loss of signal which was at about 8:59.
This whole thing happened fairly rapidly, of course, and this was the trajectory of the vehicle.
I will show another picture, but the whole problem sort of started off Hawaii and then the vehicle began to break up here in Texas.
This was kind of what we knew on roughly February 18th.
In fact, that's all we knew really.
I mean the temperature increases were not particularly dramatic, you know, 50 degrees, 30 degrees.
That's not really interesting.
What was interesting is the wires were cut.
That was more interesting.
OK. We began to look for the debris.
We had people essentially marching over the State of Texas about that far apart.
It was an extremely expensive thing, but we recovered about 40% of the material that came from Columbia.
And, of course, it is all these little pieces of stuff that fell on the ground.
Fortunately, it fell in a fairly rural area.
Nobody was hurt.
I don't think there was any damage to structures or anything like that.
And we were able to recover a great deal of it.
A lot of it, because I actually looked at the weather radar, was very small particles that actually floated into the Gulf of Mexico.
I mean you could see it on the weather radar, the NEXRAD system.
You guys all know what the weather radar is.
But you could see the particles streaming into the Gulf of Mexico.
I think a lot of it was just sort of vaporized.
And there is a footprint of the debris.
And we did a lot of searching in that area.
We tried to search upstream.
We were really trying to find the earliest pieces of debris, but these fell down onesies and twosies in really strange places like in mountains in Utah and stuff like that.
So we were never really successful, although some of these had been tracked on radar by the FAA.
And there is the debris collection, a hanger down at Kennedy.
This is pretty standard stuff for a commercial aircraft, I think you all know that.
When you have an accident, what you try to do is recover the debris and reconstruct the vehicle.
This debris was particularly interesting.
And I don't know if I have a graph later, but let me talk a little bit about the temperatures involved.
We're talking about reentry of a damaged vehicle at mach 25.
Now, of course, that is the course that I tried to get out of when I was a graduate student, was that high temperature stuff where the gas begins to ionize and dissociate and, you know, all that enthalpy stuff.
I really tried to avoid that, so I got paid back.
Anyway, that is the subject of high-speed gas dynamics.
And so I had to take a refresher course.
And Judd Barren, one of our faculty members, was extremely helpful.
I used to go over to his house on Saturdays and he would give me references.
But, roughly speaking, if the gas didn't dissociate, the temperatures on reentry at the leading edge of this vehicle would give off 50,000 degrees Fahrenheit, which is really pretty hot.
The fact that the gas does dissociate -- Ionization at this mach number is not that important.
It interferes with the radio signals but does not, in fact, absorb a lot of energy.
The gas is slightly ionized 1%, something like that, 2%, but not enough to be a big energy issue.
But the dissociation of oxygen, dissociation of nitrogen is an extremely important phenomenon.
And, because of that, the gas temperature is roughly 10,000 degrees Fahrenheit which is a lot better than 50,000 degrees Fahrenheit.
So that makes a big difference.
Let's talk about the leading edge material.
The leading edge material is, in fact, an incredible material.
It is a carbon-carbon composite material.
It is about a quarter of an inch thick and it can withstand a temperature of 3200 degrees Fahrenheit.
And the Shuttle was conceived in, what are we talking about here, '70?
Yeah, '69, '70.
We don't have much better materials today.
We might be able to go to 3400 degrees Fahrenheit instead of 3200 degrees Fahrenheit, so we've made some progress, but this is a marvelous material.
Now, how in the world, with the gas temperature of 10,000 degrees Fahrenheit, can you use a material of 3200 degrees Fahrenheit?
Well, that's the other part that I tried to avoid as a graduate student.
It is called radiation.
And what is it?
The Stefen-Boltzmann Law of Radiation, T to the fourth.
And you could write it down on the back of an envelope and you can calculate it.
And it turns out that the gas coming in does heat the leading edge up, but then the leading edge radiates out the right amount of energy to basically have an equilibrium temperature of the order of magnitude of, say, 2800 degrees or maybe 3000 degrees.
It is comfortable below its maximum temperature, but this is obviously a very dangerous situation.
In the early days of the Shuttle Program they made some tests of this material, its ability to withstand impact.
They shot BBs at it and made quarter inch holes in it.
And then I'm a little fuzzy on this part.
They may have tested those quarter inch holes in an arc jet to see would they ablate, how would they behave and would they grow?
And I think at that point they felt comfortable that they could withstand the impact of a BB and not be destroyed.
So that is an important set of issues.
I guess the other thing I want to point out is if you have a good solid continuous edge you can support a temperature of 3200 degrees because of radiation.
But if you get a hole in that leading edge then the gas that goes into the cavity behind the leading edge is your old friend 10,000 degrees.
But there is no radiation balance to bring you back down to 3200 degrees so you have a 7000 to 10,000 degree arc jet coming through any major size hole in the leading edge of the shuttle.
And that is, of course, exactly what happened.
And, to remind people, 10,000 degrees is the surface temperature of the sun.
Oh, it's very hot.
Yes.
There is no material that will withstand this.
And this happens routinely in shuttle operations is that the shuttle reenters at mach 25 and the temperatures of the gas surrounding the shuttle are about 10,000 degrees.
And we've successfully gone through how many shuttle flights in total?
114.
It's a sporty course.
Now, I want to talk about something else we knew before we found the flight data recorder.
And this was of great interest to me for lots of different reasons, but one of the things we were able to do is infer the aerodynamics of the damaged vehicle by looking at what the Flight Control System had to do in order to keep the Shuttle perfectly on its pre-assigned flight path.
In other words, until very late in the flight the Shuttle was doing exactly what it was programmed to do.
But it was working a lot harder and in very off nominal ways.
And we had to back this out.
Now, this is almost like a zero over zero calculation so it's a little sensitive, a little hard to do, but this was something the aerodynamics group did.
And I worked really closely with these guys.
I have great admiration for them.
They were able to back out the off nominal roll moments and the off nominal yaw moments of the Shuttle during this final reentry.
And I made this little schematic of kind of where were they when some of these things happened in terms of the Coast of California and their flight over New Mexico and down to loss of control over Texas.
Before I do that, let me just add to my previous remark that you almost always have a picture of a space accident.
Now, normally that's because space accidents occur on launch.
This is a remarkable photo of a space accident.
This photo was taken by modestly my guys at Kirtland Air Force Base.
And basically they had been working with optics.
That is basically their business.
And so Saturday morning IBM personal computer, fooling around with some little telescope that they had been developing for tracking things.
32:00 Yeah, this is interesting.
This is a bulge at the leading edge showing some distortion of the shock shape due to vehicle damage.
And I obviously consider this photo to be extremely probative.
And also there is a small amount of debris that has got some optical signature to it that is sort of streaming out of that general area indicating -- I mean there are all sorts of things going on in this vehicle, but one of the things that was going on was the melting of aluminum, the vaporization of aluminum and the combustion of aluminum because aluminum will burn.
I don't know how many of you know it but aluminum is something you add to rocket engines in order to get higher temperatures.
It burns when it gets hot enough.
It can be a very dangerous material.
In any case, we suspect there was a lot of aluminum combustion going on while this thing was happening.
Had there been, for reference, any similar photographs?
No.
So we didn't have a nominal baseline.
No, as far as I know.
I actually don't know the answer to that question.
I know, because I know the guys who took the photo and I know their boss, that they were doing it for a very specific reason that had nothing to do with anything they had done in the past.
They were trying to develop a little telescope to track things because that's what they do at Kirtland, is they work with optics.
But I don't know whether anybody has tried to do it.
I suspect their level of precision is a little greater than the average citizen trying to take a picture of the Shuttle coming in.
But this was not a really high powered telescope.
I mean they had some really high powered telescope at Kirtland.
This was just a small one.
Anyway, I don't know the answer to that.
Well, I think what we're looking at here is the glowing of this 10,000 degree envelope that surrounds the Shuttle.
That is what we're looking at, is we're looking at the radiation, basically, from the gas surrounding the Shuttle.
It was normal.
But, of course, this was 8:50 in the morning Boston time.
This is February so it probably was dark in New Mexico.
That is kind of the picture that we go, which I think is really neat.
This is the trajectory, but this is the backing out of the off nominal moments.
Now, let me talk about the yaw moments just to give you a feeling for what does all this mean.
I contend that off nominal aerodynamic forces are an indication of external damage to the vehicle.
That is basically what it is.
The vehicle was flying because its reaction jets were moving and doing all sorts of things.
The aerodynamic surfaces were virtually useless at these dynamic pressures.
This was a reaction jet controlled vehicle, but we knew how much force the reaction jets were putting out so we could do the vehicle aerodynamics.
Let me talk about the yaw moment.
What we have here is that as the accident began the yaw moment became negative, kind of flattened out for a little while and then became very strongly negative.
What is yaw moment?
Yaw moment is the force turning the vehicle this way.
If I have a damaged left wing, what is my yaw moment going to be and why?
Well, I'm going to have increased drag on this wing because I've got a big chunk out of the leading edge, I've got a disturbed shock, I've got all sorts of stagnation pressure in this region where in normal circumstances it would be smoothly flowing.
I've got increased drag on this wing, and so that's going to drive me this way which is exactly what you see.
That is off nominal yaw moment.
Roll moment is a little stranger.
Basically, as the drag increases, and you probably lose lift so your roll moment begins to go like that, you have decreasing roll moment.
This was a bit of a mystery because basically what it said is you lose roll moment, but then all of a sudden your roll moment increases.
That is a problem that only an aeroelastician could solve.
I happened to be an aeroelastician.
Do you know what I'm talking about?
If you stick your hand out of the car window and you turn it slightly what happens?
It goes up.
And it may also turn some more.
In other words, if you lose the leading edge spar of a wing, you lose its resistance to torsion.
Now, we don't have all the data we need to prove that that's what was going on.
But I believe a strong hypothesis that we lost the strength in the leading edge spar and the leading edge spar tipped up.
And obviously the lift would increase because the angle-of-attack was increasing.
But this was a very, very puzzling and very interesting phenomenon.
But, in any case, what happened is that both the roll moments and the yaw moments got too strong for the control system of the vehicle which, again, was just the reaction control jets that are on the area around the vertical tail.
Then, ultimately, the thing went into a spin, lost control and then, of course, the vehicle was lost, began to break apart.
This was a very interesting set of data.
And, as I say, when we started, this was pretty much all we had.
There was a pivotal moment, and it occurred sort of at the end of March, about March 27th.
Remember, the accident occurred February 1st.
We found the flight data recorder.
And we found it by these guys walking across the field looking for stuff.
And the flight data recorder was found.
And it was in pretty good condition considering it had done a reentry at mach 25.
We were able to recover the data from the flight data recorder.
I think we pretty much got it all.
At that point we had hundreds and hundreds of data points.
And I would say, at this point, NASA's attitude changed.
I think up until this point there had been a certain level of denial about the accident, about what caused it.
Maybe it was a micro meteorite.
Maybe it was some unexplained system failure.
They were not really sure that we were pursuing the right road until the flight data recorder was found.
And I think maybe that's to NASA's credit.
NASA is data driven.
And when they began to see the data and they began to see how well the data correlated with the hypothesis and the models then they hopped right onboard and did a tremendous amount of work to try to help us unscramble all of this and come to a hypothesis.
What kind of data did we have?
Well, this is a picture of the leading edge spar, this big thing here.
And this is a picture of the hypothesis sort of about a piece of foam taking out a fairly large piece of this carbon-carbon fiber and allowing hot gas to enter into the region behind the leading edge.
This leading edge region is, as you can see by the picture, sort of hollow.
There isn't anything back there.
There is a layer of insulation covering this, but that is because of the radiation.
In other words, because this, in fact, is 3200 degrees.
It radiates in both directions.
And so there is a layer of insulation here to protect against radiation.
But that insulation will not protect against the equivalent of an arc jet.
That is just, again, burrowing and blasting it.
It was fairly quickly that this gas just blew a hole through the leading edge spar.
How did we know that?
Well, it's kind of interesting.
Let's look at one of these.
We had all these wires back here.
And we had these temperature sensors.
Let's just pick one temperature sensor.
I didn't even know where it was located, but it was not located in this region.
It was located back someplace else.
So we looked at the temperature sensor and compared it with earlier flights where the earlier flights would show this behavior and this temperature sensor kind of did this.
It began to go up and then all of a sudden we lost it.
What happened?
42:00 The picture begins to emerge of wire bundles being cut, of gas entering here, of this wire bundle being cut to the sensor of this, in fact, entire region beginning to sort of fill up with extremely hot gas, 7000 degrees roughly, eventually entering the wheel well, although, that was later in the flight.
We did have some indication of the gas in the wheel well because of the temperatures and the loss of sensors.
Because of this the focus turned to the question of this what we call RCC, which is this composite material that surrounds the leading edge.
And where did we find it?
And we found a lot of it.
I mean we found it all along in Texas.
The other thing we were able to look at, with respect to the RCC, is its pattern of erosion.
I mean this RCC was kind of flopping out in the breeze here at roughly 7000 to 10,000 degrees.
And so it was a material that was being effectively subject to an arc jet.
And, if you took a material like that and stuck it in an arc jet, it probably would ablate and it would maybe sharpen up and you'd have maybe little pointy edges where it had ablated.
And so we looked at the part that was particularly eroded.
And that would be the yellow stuff.
And we found that upstream of the stuff that was more put together, the rest of the RCC from the wing.
And, from the right wing, we found the right wing downstream of the left wing which means that the left wing came off first.
Now, the image that emerges from this is hole in the leading edge, material coming off, but gas flow going down this what some people called the "chunnel."
I never liked that term, but it's a big open cavity.
Gas flowing down this open cavity and simply destroying all the fittings that held the RCC onto the wing.
I mean the screws and the brackets and all of that.
And, again, gas is coming in.
And the wing just unzips.
All this stuff just kind of unzips and falls off.
And, of course, that is where we find it, just lying on the ground.
It all unzipped and fell off.
What did we actually find?
45:30 What we had to do, however, is to construct analysis using a whole lot of different disciplines.
And we had to be able to analyze, according to the rules of that discipline, and line them all up for consistency.
And I would say that we never found a theoretical or experimental result that was in conflict with our basic hypothesis.
And that everything sort of lined up.
I was more or less in charge of the aerodynamic analysis, as well as my favorite subject thermodynamic analysis.
I never liked that part either, but the whole question of heat transfer and how fast these materials would melt, I was pretty much involved in that.
I may have another picture later, but let me just show you some pictures that we had.
There is a hypersonic wind tunnel down at Langley.
And we did a lot of tests in that wind tunnel.
One of the things we did was we had a score of wind tunnel models.
They were about this big.
But we took chunks out of their leading edge.
We have our wind tunnel model which was the nominal shuttle.
And then we'd have a nominal shuttle with panel six missing and then a nominal shuttle with panel seven missing and a nominal shuttle with panel eight missing.
47:30 And then you see the green.
That is probably a little intermediate temperature.
And then you see the blue.
That is probably lower.
But, if you look at this, being an aerodynamicist, I can tell you where that vehicle gets hot.
It gets hot near stagnation points and near leading edges.
And especially at the stagnation point on the vehicle.
You can put a wind tunnel model like that in a hypersonic tunnel painted with this special paint, and you can take pictures of it and you can get that temperature distribution.
Here is my little figure of aerodynamic forces.
We had all sorts of data from both the flight data recorder and the telemetry that gave us a pretty good indication of the timeline.
If any of you ever want to come into my office, I have this enormous poster.
It's like ten feet long which basically shows the timeline of the Shuttle reentry with all kinds of the individual data bits, the temperature distributions and all the things that kind of we could fit on this enormous poster.
That is kind of what we were trying to do.
We looked at the debris.
We looked at forensic analysis of the debris.
Where was it ablated?
What was its chemical composition?
Did we see melted Incanol?
I mean we saw all these things and were able to figure it out.
Well, we knew what temperature Incanol melts at so, therefore, the temperature on this particular piece was, I don't know what the temperature of Incanol is, but let me say 1200 degrees.
It would be that kind of material.
49:30 And you don't replace this material.
This material is the same material that was used on the original shuttles.
Every time it flies it loses a little bit of stuff through vaporization.
And so then they kind of tried to paint it again, but I think our hypothesis is it just gets weaker and weaker as it is being used.
And the problem is it is hardly being made anymore.
It is material that was all made at the beginning of the program.
It is not being made anymore.
It is very expensive.
It is not a material that is easy to get.
And, in fact, that became an issue for us because one of the things I will show is we wanted to do a full scale mockup of the leading edge and we wanted to test it with a piece of foam going 800 feet per second.
Now, why would we want to do that?
Well, that is exactly the test conditions that we wanted to simulate.
Well, that is not only expensive but this material just doesn't exist.
I don't remember where we got the material.
We probably took it off of Endeavor or something like that.
But, I mean, this was a big deal that we should run this test because the material is very precious.
But, anyway, we banged on the table and said that we were going to do this and that they had to support us.
We did the wind tunnel test.
I have a question about that test.
Yes?
My reaction, as an outsider, when they published the pictures of the holes on that test was wow.
Wow.
That was my reaction, too.
Yeah, my reaction was wow.
51:30 Anyway, we did the tests and we studied timelines, we conducted arc jet tests, we did forensic analysis of debris, both chemical and sort of physical.
We did a lot of different in-depth analysis.
The test itself, in addition to being expensive and working with material that was really hard to pull together enough of this material to run the test was also emotional.
It was also an emotional event for the people involved.
And just very late in the game NASA said we are prepared to accept that the foam put a hole in the leading edge so don't run the test.
And we said well, you may be prepared to accept his but somewhere down in your organization is someone who doesn't believe that, and this test is for him.
I actually met that guy.
I mean there are people in NASA who do not believe this.
So maybe what makes sense is let me show the movies and then we will have our break.
Here is the wow.
Can I hear it from the class?
Wow.
And this really was.
I mean the guy who developed this material, and, as I say, I have great respect for that material.
53:30 Let me go inside.
This is inside.
They had a camera mounted inside.
That is kind of gee whiz thing because you can see the deformation before it breaks.
Was this done in the wind tunnel?
No, this was a full scale mockup that was done at Southwest Research Institute.
We had a big mockup of the vehicle.
It was done outside.
Southwest Research has been involved in the Shuttle Program for quite a while.
They have a foam gun that they have used to shoot foam primarily at the underbody tiles.
The concern has always been on the strength of the tiles.
Do you guys know what the tiles are?
The sort of foam stuff that is underneath the vehicle.
They have done a lot of tests on foam.
Not big pieces of foam but sort of small pieces of foam.
Never tested the leading edge because it is damn expensive to test this leading edge material.
And the only tests I am aware of were these BBs that were shot.
This was really the first time this test was done.
Now, let me make a profound comment about space systems.
There is a fundamental principle in the development of space systems, test-as-you-fly.
You all know that.
I hope you all know that.
This test should have been done at the beginning of the Shuttle Program.
It was not done.
And the reason it wasn't done is because there was a requirement in the Shuttle Program that foam shouldn't come off the orbiter or off, whatever that thing is, the tank.
It was a requirement that foam shouldn't come off the tank so, therefore, why do the test if it's a requirement that foam not come off?
Well, the fact of the matter is foam has come off on every single shuttle flight, as far as we know.
So this was a violation of the test-as-you-fly principle.
That is, I would say, the number one principle in the development of space systems is test-as-you-fly.
It is the most important thing.
Anyway, we can take our break now.
And remember JR Thompson's remarks about testing to failure, which they did on the main engine, didn't do on the SRBs, and they clearly didn't do here.
OK, a quick two minute, you know the drill.
What you see is sort of an aluminum frame.
I mean what you have here is really what would be called a full scale mockup.
It is an aluminum frame that holds the tile in the right position.
And I suppose holds it with fittings of the right strength.
In other words, what you're trying to do is replicate with what would be called a full scale mockup the structural conditions of the leading edge.
You would have fittings holding the panels and then you would have it angled properly to the foam gun, which is over here some place, and then you fire the foam at 800 feet per second.
So that's basically the test.
And, as I say, it was done outdoors.
It was quite spectacular.
A big audience, you know, there was the media and NASA.
A big audience.
It was quite an event.
As a result of this, the committee felt pretty confident that we had identified the technical cause of the accident.
When we started this, we didn't know how sure we would be so we had all these words like probably, more likely than not, conceivably.
Because we needed to think about describing our certainty whether we knew, in fact, what had happened.
But by the time we finished all of this, because everything had lined up so well, we felt pretty confident that we knew, in fact, what had happened.
And so we all sat down around the table and argued about these words.
This was a committee consensus.
We voted on every word.
Well, not every word.
Some of the words are obvious.
But we basically came to consensus on every single important word in this statement of technical cause.
I will just sort of let you read it because it took us a long time to write.
I should mention that one of the things.
And I think this was from the flight data recorder.
I really have not given you as much data about the flight data recorder.
But one of the things we had from the flight data recorder is an indication of conditions on assent relative to earlier flights.
And if you look at the conditions on assent, you can make a pretty good case that there was a hole in the leading edge on assent after the foam had hit.
Because you see increased temperatures on the sensor right behind that region of the leading edge.
And so you can make a pretty good case that the hole was there on assent after mach 2.5 or whatever the mach number was.
But that was data from the data recorder?
I believe it was from the flight data recorder.
They didn't get that on telemetry?
They did not have that from telemetry.
I'm pretty sure about that but I'm not absolutely sure.
Yes?
[AUDIENCE QUESTION] Oh, they knew that weekend.
Oh, in fact, I'm going to talk about that because that's part of the organizational and cultural issues.
Finishing the technical cause, this was our final statement about the cause of the accident.
But we didn't stop there.
Not this group of people.
Yes?
I was going to ask how soon did you arrive at the hypothesis that the hole led to the gases that led to the [OVERLAPPING VOICES]?
I think pretty quickly?
Yeah, I would say it was pretty quickly.
Now, there was a little pushback from NASA on that.
But I described my appearance on February 18th on the board and the fact that we had the wire cutting sensor information, that we had the video of the foam hitting the Shuttle and exploding.
In other words, there was a shattering of the foam.
And we had the aerodynamic forces which indicated damage to the vehicle which were consistent with the wing leading edge.
We have a picture from Kirtland of some bulging around the wing leading edge.
I would think by sort of the third week in February that's where we were headed.
Now, you don't want to get blindsided.
Sure, we don't want to get blindsided.
We got lots of inputs from the public, as well as from the scientific community.
And there was an input we felt we had to take seriously.
And I can sort of say what it was.
We got an input from sort of solar physicists who said that on that day there was a violent sun episode that would have sent a shower of solar radiation to the earth on that particular day.
And that it was possible that this could have created some kind of shockwave in the vicinity of the Shuttle and done some damage.
I mean it was a credible hypothesis.
I called the Air Force.
Out at Hanscom we have these guys who do basically radiation physics, radiation weather, space weather.
And Jim Hallock from DOT kind of shepherded that part of the project, but Jim is a physicist.
He has a PhD from MIT and works at the Department of Transportation.
And so the two of us go together and said we have to take this seriously.
We have to put an expert team together and completely examine this hypothesis.
So we did that.
And we had data from all over the world of this radiation coming in, and so it was very exhaustive.
It turned out that this radiation didn't reach the earth until 3:00 PM on that day, but we took it seriously.
I think that was probably one of the most credible things.
We obviously got a lot of junk stuff that it was an Israeli plot or something like that.
I cannot even remember those things.
And then I think there was always this suggestion that it was just a micro meteorite, but I think because of the magnitude of the damage and the fact that, I think this is general knowledge, the Air Force watches space junk.
They have a catalog of everything that is lying up there, and there wasn't really a piece up there large enough to do that kind of damage so that was eliminated.
There were a number of hypotheses.
But, again, when we did the data, we didn't go in with a hypothesis.
We went in to see what the data told us.
That was a bit of independence with respect to the data and the analysis.
Yes?
[AUDIENCE QUESTION] No, it is melting.
No, the temperatures.
See, it's aluminum.
[AUDIENCE QUESTION] Eventually.
Well, I think heat was the main culprit.
The dynamic pressures, and I am sorry I didn't bring the graph, in this regime were not that large.
I am thinking they were less than 100 pounds per square foot because you're really up at very high altitude.
And, even though you're going very fast, the dynamic pressures are not that high.
I actually have a graph of dynamic pressures.
In fact, come into my office and you will see the dynamic pressure as a function.
I would think the way to think about it is that the important parts melted.
See, we know from observations, and I didn't show all of these, but people were recording, I don't remember what we called them, but big pieces of the Shuttle were coming off.
You saw the video with all these pieces flying in, but we have pieces coming off as early as California.
And I think we hypothesized at one point that the upper wing came off.
In other words, the upper surface of the wing just lifted off.
And the reason it did is not so much from pressure is that all this gas is in there.
Aluminum melts at 700 degrees Fahrenheit or some ridiculously low number.
We're talking 7000 degrees Fahrenheit so we're talking about destruction primarily by arc jet.
You have to think of the atmosphere as a big arc jet.
And eventually the vehicle became uncontrollable, unflyable because the aerodynamics were so off nominal and the vehicle couldn't be controlled.
But, again, I think the dynamic pressures didn't get larger than 200 pounds per square foot.
I don't think.
Which is about what I am doing right here.
That is about 200 pounds per square foot.
Yes?
What happened to the crew?
[AUDIENCE QUESTION] I probably don't want to talk about that.
We have some information.
Well, let me say what I can say.
The vehicle itself stayed intact through a large part of the reentry and got into what would be called the fighter pilot's regime, you know, 50,000 feet.
The actual cause of the death of the crew was what any fighter pilot would experience if he lost his vehicle at 50,000 feet.
It's blunt force trauma.
That is what it was.
And we found a lot of the cockpit.
Reentry is really interesting.
There is something called ballistic coefficient.
Do you know about ballistic coefficient?
We found briefcases, pillows.
Ballistic coefficient, people talk about it in two different ways, and one is the reciprocal of the other.
I always get confused about that.
But if you have a low ballistic coefficient and you drop a pillow at mach 25 at 400,000 feet, it would probably make it to the earth because it will slow down and then it will just gradually float down to earth.
We found a lot of the crew compartment, a lot of pillows, chairs, all the stuff that was really light just reentered without burning up because of ballistic coefficient.
There was a lot there.
69:00 And so that gets into some of your questions having to do with when did they know.
And, of course, the big why question.
Let me go to that.
Now I'm going to go off into the second part of our investigation.
And we consider this to be as important, if not more important than our technical investigation.
We talked a little bit about the history of the Shuttle Program.
And I am sure that is something that you have been through in terms of the budgets, the margins, what we believe is a mischaracterization of the vehicle as a mature operational system.
And we think that there is a story here to be told, and so we looked into this.
Now, I have to say that I was not as much a part of this as some other members because I was busy on the aerodynamics part of it.
But you had John Logsdon, we had quite a number of the Air Force people, and Admiral Gehman were more deeply involved in pulling together the sort of history of how this happened, the cultural issues.
I am sort of reporting on behalf of my fellow board members what our conclusions were and the data that we gathered together to put this together.
The things that were pointed out are that there was this fundamental uncertainty in the Shuttle Program and that led to a sort of fluctuating attitude towards investing in upgrades and infrastructure.
The nation didn't really know where it was going.
And I hope you won't take offense at this.
I worked for NASA.
You worked for NASA.
We found them to be an extremely insular organization.
And I think, actually, they still are.
They don't take advice from the outside.
And I will be prepared to admit that they are the only organization that does human spaceflight, at least in our nation, but they are not the only organization that handles risky technologies.
And they have a lot to learn from other organizations.
And they have not been a part of that conversation with the type of people like the nuclear navy and some of the commercial industries that use extremely risky technologies.
They have not been a part of that dialogue.
And that is where we have criticized them.
As I say, they are insular.
They think that they cannot learn anything from anybody else.
That they know how to do all these things and that they don't need to accept any external suggestions.
That is a very dangerous line of reasoning for an organization, really, no matter how good you are.
So we felt that the people at NASA, that leadership clearly believed these things and the people in the engineering workforce were under tremendous pressure to go along, in some sense, with these basic attitudes.
And we saw that in our investigation of kind of the what went on during the flight.
I mentioned before that there is this history of foam.
And let me introduce the word anomaly.
Have you talked about anomaly?
Not directly.
OK.
Anomaly is one of the most important words in space systems.
Anomaly means that something happened that shouldn't happen.
Anomaly should bring your organization to a standstill while you figure out what happened and why it happened.
Anomaly is a violation of requirements.
Sometimes I think it's a euphemism for failure.
Well, yeah.
Hopefully you catch it before failure.
Anomaly is a clue that something may even worse happen down the road.
It is something to be taken extremely seriously.
Foam shedding off the tank was an anomaly.
It didn't destroy the vehicle but it is a violation of requirements.
And, in the early days, it was really put on a list of problems to be worked.
I mean NASA didn't completely ignore it, but they put it on a list of the other 5000 problems that needed to be worked.
And they just kept flying until foam shedding really became a normal expectation that all the flights would have foam shedding and that nothing serious would ever happen.
It was treated as a maintenance turnaround problem, you know, when the vehicle gets back we can fix it.
And that was true in this case, too.
They said something like well, if foam hits the Shuttle, we'll fix it when it gets back.
I mean that was the stance.
Now, I think what's interesting about this, if you think about this, is that the anomaly of foam shedding without hurting the Shuttle was treated as if it were a planned engineering test.
In other words, it was a test that was planned to see if foam could hurt the Shuttle.
And the fact that foam didn't hurt the Shuttle was treated as confirmation that foam couldn't hurt the Shuttle.
Now, I think that's kind of a fundamental.
I like that particular phrase.
There was enormous schedule pressure on NASA.
They had already slipped a couple of times because of other what I would call strong signals.
We use the word strong signals.
When something really dramatic is identified NASA shuts down.
And I don't remember the issues.
There was something about a connector that didn't seem to work properly and it just grounded the Shuttle fleet for three months or something.
And there was some other problem that was a strong signal and they simply shut it down.
The foam we put in the category of a weak signal.
It is something that while you're fighting all the strong signals it is hard to justify shutting the organization down for what you perceive as a weak signal.
So they had used up almost all of their margin to complete the Space Station.
And they were under tremendous pressure.
Sean O'Keefe was obviously sent to NASA to put the house in order, put the schedule and the budget in shape and get this Space Station finished by February 19th.
Ooh, we've already passed that one.
We will talk a little bit about what happened on the ground while the vehicle was in the air.
I mentioned before that in the military we do two investigations.
We do a safety investigation where the testimony is privileged and we do an investigation which is public.
And our group was kind of doing both.
In other words, we were obviously doing a public investigation.
We had press conferences every week and the media would come and we'd share everything with them.
But, because we did have so many military people who understood the importance of accident investigation, we also had, I don't know, an annex, I guess you'd call it, a way to take testimony and keep it privileged.
And we did that.
We took testimony from individual people at NASA.
It was privileged testimony.
And they felt free to share with us their participation and what happened and who said what when and all of that.
And we had the damnedest trouble keeping this from Congress.
I mean it really was.
And anything that's written down is accessible to Congress, at least that's the way they look at it, but we really held very firm and we would not allow this privileged testimony to be accessed.
And I think the compromise we made was that a staffer could come over to where we were located, he could read the testimony and take no notes.
Well, they soon tired of that so the fact of the matter is they didn't come.
But we had a lot of privileged testimony that allowed us to put this picture together with employees not being too concerned about the implications of sharing with us.
So, the weekend that the flight went up, NASA knew that there was a potential problem.
And they put in place something called the "debris assessment team" which is supposed to look at all the data that they had and decide whether or not there was a problem.
Now, I've chaired a lot of committees, not only here at MIT but other places.
And I understand how important it is for a committee to have a clear charter and a clear line of reporting and to have a certain level of independence from their home organization.
For example, if I were chairing a committee here at MIT on an important research issue or important policy issue and members of my committee were getting pressure from their department heads, I would blow the whistle.
I would say that is not the way a committee works.
A committee has a charter.
It has independence.
It is not subject to outside influence from the people to whom these people report in their normal way of life.
So, with respect to the debris assessment team, I would make two comments.
First of all, they had a very unclear charter.
It wasn't clear kind of who they reported to or what their authority was.
And they, as individuals, were getting pressure from their home organization, both NASA and the companies, the USA, Lockheed, Boeing, because those guys were getting pressure from NASA.
It was just an enormous amount of pressure being applied to various people in what I considered to be a totally inappropriate way.
In any case, this team, I admire them.
I mean I think they were doing a good job.
They wanted to get on-orbit pictures to see whether or not using national assets we could take pictures of the Shuttle.
And so, who do you contact when you want something like that?
You talk to the Air Force.
And the Air Force is, of course, like a great collie dog just waiting to show what they can do.
You know?
I mean oh, great, we've got this wonderful opportunity to take these pictures.
It's really going to be fun.
We're going to show what we can do.
They are very enthusiastic about responding to that kind of request, but NASA didn't want the pictures.
I mean the senior managers of NASA didn't want the pictures.
The debris assessment team did want the pictures.
And they tried to make these requests of different parts of the organization.
And every time they would make a request it would get slapped down by senior managers.
Well, I guess we have it.
This is a highly classified area, but let me just say there were eight separate opportunities where we could have gotten some significant information about the state of the Shuttle and we didn't do it.
We didn't do it because every time there was a flurry of activity around there NASA said they didn't need it.
They'd fix the Shuttle when it got back.
If the damage had been discovered what do you think would have been done?
Well, I can talk to that issue.
I mean this is as good a time as any.
We actually asked NASA to do some studies about what could have been done.
This is a nation that will save a whale that is trapped in the Arctic or Baby Jessica in the well.
I mean if we find that there is a crisis we will mobilize.
There is no question about it.
It would be a very chancy situation.
We know the vehicle is destroyed.
The vehicle is lost.
There is no question about that the vehicle cannot reenter safely with the people.
So what was the best option?
I mean we looked at the possibility of stuffing the hole with things, you know, old sleeping bags and suitcases and stuff.
There wasn't anything onboard that could have withstood the temperatures.
And, of course, one of our recommendations was that there should be some capability for in-flight repair.
That has been a big issue.
But the only other thing you could do is to sort of go into life support mode.
Stop doing the science.
Jettison the module, I guess, which would make the vehicle lighter and then maybe the reentry would be a little more successful.
Conserve electricity.
Conserve water.
Conserve food.
Try to stay in orbit for an additional, I don't remember the number, 15, 16 days, something like that so that you could send up a second shuttle.
I think it was Atlantis.
They could get Atlantis ready and send it up.
It was sort of by chance, but it just happened that in this mission they did have the possibility of doing a rescue.
Right.
It's a gutsy move, but I have a feeling that if we actually had that information that we could have put that together.
It is interesting that when you read some of the stuff that is being talked about, like the Hubble and things like that, we did not recommend that a second shuttle be standing by.
But that seems to have been internalized at NASA as a potential safety measure.
That they will always want to have a second shuttle standing by if they send the shuttle up.
The problem with that is it's a bit of a "going out of business" strategy because there aren't that many left.
Yes?
[AUDIENCE QUESTION] We never looked at that option.
We simply didn't.
This particular vehicle was not in an orbit that could have reached the Space Station.
And that is another big issue, is whether you want to have the Space Station available for a safe house if you have an accident.
In this particular case, this vehicle could not have reached the Space Station.
Soyuz could not have reached this because Soyuz launches from 51 degrees.
This was in 28 degrees.
Also remember there were seven people aboard so you would have needed three Soyuzs.
The Russians just don't have them available.
It was either Atlantis or nothing.
Larry, you had a question.
Yeah, going back to the on-orbit photo.
It was alleged at the time that the NASA management thought that the quality of the photos was what it had been several years earlier and, therefore, would not have helped them.
Let me speak to that because I was going to mention that.
In the early days of the Shuttle Program there was, in fact, a good interaction between NASA and the national security apparatus that does this sort of work.
And people at NASA were cleared, to an appropriate level, to have this interaction.
That has atrophied.
The current people at NASA basically have no experience with that world, are not cleared to an appropriate level, have just not had an interaction and see it as a bigger deal than it really is.
It is not a big deal.
We do it all the time with my collie dog friend.
I mean it's just not a big deal.
So our recommendation was that there should be a better interaction between NASA and the national security apparatus.
And that the senior people at NASA should be cleared to an appropriate level to have that interaction.
And that this should be, in fact, routine that you always take pictures when the shuttle is in orbit.
And so I think that was basically one of our recommendations.
In any case, we were not pleased with the decision or the failure to take photos onboard.
86:00 It was called the "crater model".
Its fundamental purpose was to figure out whether foam would hurt the tiles.
It was developed as a result of the experiments done at Southwest Research Institute of shooting small pieces of foam at the tiles.
Not the leading edge but the tiles.
And it was sort of all things kind of rolling together.
The people who developed this model were in Huntington Beach, California when they transferred the operation to the Cape or to Houston, I don't remember.
The senior people in that group refused to move so they turned the tool over to junior people.
There was a loss of corporate memory, a loss of understanding, and so it just got worse and worse and worse.
And so finally the people who were applying these tools were not the same people who were in the development of the tool and really lacked understanding of what this tool was all about.
The tool predicted that you had a serious problem.
But they said, oh, well, the tool always over-predicts.
They ignored it and said there was no problem.
I mean it was just cascading of kind of bad things happening.
We talked a little bit about the organization of NASA.
And I don't know whether you talked to Aaron, but this would be a good question to ask Aaron Cohen.
I am setting him up here.
About an independent safety organization at NASA.
Yes?
[AUDIENCE QUESTION] As far as we can tell nothing until about one or two minutes before.
Actually, Mission Control had arranged for a press conference with the crew in orbit.
Right.
Are you going to mention that?
I will talk about that.
Because the crew new that some foam had come off, but they were told that it was no problem.
Yeah, but that's different than the crew knowing.
Right.
No, I think that is true.
That the crew was notified that there had been an incident on liftoff but there was no problem.
And the only reason they told them was because somebody from the press might ask them.
Yeah.
And I have had media training so I know exactly what they were thinking about.
The Shuttle lands, the astronauts get out, the microphone gets into the mouth and they say do you know?
And the astronauts would say of course, we knew, and we understood it was no problem.
I mean it was a pre-media sort of skull session for the astronauts, as opposed to a more responsible concern.
What we do know about the astronauts is we have the voice recording and stuff, and I think there was something from Mission Control to the astronauts saying we see a temperature anomaly in your wheel well.
The temperature was 50 degrees higher than would have normally been.
And this was literally just like I don't think any more than five minutes before the vehicle lost control.
Maybe even less than that.
I mean it's all in the timeline.
And that is probably all I want to say about it.
I think the evidence we have seems to indicate that the astronauts knew.
Obviously, when the vehicle went out of control they knew there was a problem, when it began spinning and stuff like that.
But, as far as we know, there was no indication prior to that.
Everything else was sort of nominal.
Yes?
After both shuttle accidents, you have investigation boards, but what happened after Apollo 1, for example?
[AUDIENCE QUESTION] I don't know the answer to that.
Well, actually, that's what Aaron did talk about, safety organization with respect to that and Apollo 13 and the fact that when it was set up, they did start out with a very strong independent safety organization which did not exist before Apollo 1.
90:30 Now, why did we say that?
Well, you're all engineers.
The idea behind an independent safety organization is that you should have engineers fighting.
How do engineers fight, in an ideal world?
They fight with data and analysis and hypothesis and testing.
In other words, if the program wants to waive a requirement, change a requirement then they have to present a case.
And if the safety organization feels that there is crossing the line then they present a case.
And the two organizations basically try to bring together the best analytical and experimental framework they can in order to determine what should be done.
And so that's the notion of an independent safety organization.
I guess the thing that I was somewhat gratified by, we will see how it all works out, is that we recommended, as a return to flight thing, that NASA present a plan for an independent safety organization with the idea that in the midterm they would actually begin such an organization.
Sean O'Keefe took the bit in his teeth, I guess is the way to say it, and he moved out much more strongly on that than I would have anticipated.
We met with him last December.
I guess it was December.
These years go by very quickly.
But he had actually put in place a structure, to get ready for the flight, that had outside experts from within NASA to sort of come together as an independent safety board to look at anomaly resolution.
I guess they didn't do too good on the foam.
But, in any case, as far as the structure goes, it was a stronger safety structure than had existed before.
That was something that we recommended.
It was very important that there be an independence in the safety organization and that this is, in fact, a characteristic of an organization that effectively manages high-risk.
And we would claim that there are several such organizations in our country.
And there is a lot to be learned and a lot to be shared.
We also looked at the comparisons between this accident and the Challenger accident.
And, in fact, found more similarity than you might have expected given that Challenger occurred on liftoff and this occurred, well, in reentry, but did occur on liftoff.
In terms of the role of engineers and technical managers, this notion of normalizing deviance.
Deviance is an anomaly.
It is a departure from the requirements.
And, if you normalize it, it means you accept it.
You accept it and, in fact, you try to profit from it by saying that it's proven that there is no danger.
So this phrase normalization of deviance is an important sort of technical phrase that our group used.
We had with us, actually, a faculty member from Boston College who had written a book on the Challenger investigation.
She is a social scientist, and she worked very closely with us to try to put, what I would call, the social science and organizational effects together.
That is Diane Vaughn.
You might bring her over some time.
She is a fascinating speaker.
But the thing that was interesting about Diane, not only was she a great colleague, she published her first book on the Challenger accident, is there's an in-depth analysis of the cultural flaws in the organization that led to the Challenger accident.
She published this book.
And she said well, you know, I got a lot of reaction from the book.
She said the Navy called, my old boyfriend called, the various industries called to have her come and talk about safety culture.
NASA never called.
NASA never indicated any interest in her analysis of the cultural flaws within the organization that led to the Challenger accident, which I think is another indication of the insularity.
That was, I think, an important input to our look at the organizational issues.
We got together as a result of all of this.
And, as I said, I wasn't deeply involved in, say, some of the privileged interviews having to do with the individuals.
But certainly as a board, we came to this organizational cause which we considered to be as important as the technical cause.
And the various words rooted in the Space Shuttle Program.
History and culture.
Original compromises that were required to gain approval.
The years of resource constraints.
The fluctuating priorities.
The scheduled pressures.
This mischaracterization of the Shuttle as operational rather than developmental.
Lack of agreed national vision for human space flight.
The cultural traits and organizational practices that were detrimental to safety that we're allowed to develop including reliance on past success as a substitute for sound engineering practices such as testing to understand why systems were not performing in accordance with requirements.
Organizational barriers that prevented effective communication of safety information and stifled professional differences of opinion.
Lack of integrated management across program elements and the evolution of an informal chain of command and decision-making process that operated outside the organization rules.
A bit of an old-boy network in some sense.
And we should mention the lack of an agreed national vision for human space flight has been the jumping off point for everything that has happened since with the new vision, which is presumably going to determine what we do in the next ten years, so I think it is great that all of you called attention to that.
Let me talk about our recommendations.
We made recommendations in basically three piles.
One of them are the near term recommendations which are the return to flight recommendations.
And these were monitored by a committee of 28 people.
It was enormous.
In any case, we asked them to present a plan for an independent safety organization.
I think they went a little further.
We asked that they develop a method to do onboard repair.
Now, there was a big discussion about onboard repair.
It is very clear that it is much easier to do onboard repair for all sorts of different reasons, including crew safety if you are going to the Space Station.
Because then you've got a safe house and all sorts of supplies.
It is like your recreation room.
If you're not going to the Space Station, it is considerably more dangerous, and that is the fundamental issue with respect to a Hubble mission.
Because a Hubble mission is incompatible with going to the Space Station.
If you take this recommendation seriously, you either have to develop an autonomous onboard repair, which is much more difficult than a Space Station onboard repair, or you cannot have any Hubble mission.
So, this is a bit of turmoil in the political issues surrounding missions.
The other impacts, as I mentioned, not our recommendation, but this notion of having a second shuttle standing by seems to have been internalized.
There were a number of other recommendations.
Our report is on the Web.
I mean more video cameras.
More onboard video that gets telemetered to the ground during the flight so you don't have to wait until the vehicle gets back to see the video.
External video cameras launched during day.
Keep your sensors up to date.
I mean there was just a whole lot of I think 25 recommendations of a return to flight.
And the Return to Flight Committee was in charge of monitoring all of those.
And they have recently issued a report.
Incidentally, I got a copy of the Return to Flight report and donated it to the AeroLibrary.
It should be in the AeroLibrary.
And there was a very interesting minority report that was submitted by like about five people which was just damming with respect to their observations about how NASA works the safety issues.
Take it as the opinion of these five people who were on the Return to Flight Committee.
It makes good reading.
But that is all in the AeroLibrary.
Our midterm recommendations.
In some sense, we made these recommendations assuming the Shuttle Program would continue.
And I think we had this recommendation for an independent safety organization.
And what I thought was one of our most important recommendations was that if you intend to fly the Shuttle past 2010, you should recertify it.
Now, certification is a very technical process where you have to go through every piece of equipment and, in some sense, revalidate that it will satisfy the requirements.
101:05 What better project for the Shuttle Program Office and the Independent Safety Organization than to recertify the Shuttle?
At least that was my vision.
That would really be an extremely useful thing to do, to work together, to recertify the Shuttle.
Now, unfortunately, evidently the Shuttle is not going to operate past 2010 so we will see.
I mean they won't recertify it.
Now, I think what's going to happen is they're going to get up to 2009 and are going to say oh, my gosh, we don't have any way to get to space.
There is going to be enormous pressure to keep the Shuttle operating past 2010.
And, of course, they will be late to need in terms of recertifying.
It is going to be a bit of a jump-all at that point.
It should be very interesting to watch.
Now, the long-term recommendation, of course, we made was that we need an agreed upon vision for further manned space flight.
And so, from my point of view, we skip the midterm and we went immediately to the long-term because with the President's new program that we have basically set this vision.
And, unfortunately, we have skipped some of the intermediate steps that would have strengthened NASA as an organization and its ability to carry out a new mission.
I stand back watching, as all of this develops, having had this incredible background of being a member of this Accident Investigation Board.
It looks like I landed the plane right on time.
Yes.
Incidentally, I brought this Halloween candy.
And I would really appreciate it if you guys would come and take it because it's not something that my husband and I need.
And if it doesn't get eaten, would somebody please take it down and put it with the graduate students downstairs?
Good.
Anyway, you had a question.
I am wondering if you think there might have been any effect on that if it was explicitly stated that this is why we want the recertification to happen.
I don't know.
You know, it is hard to micromanage a big organization.
I personally was disappointed because I believed very strongly in this.
And I felt that it would be a way to bring the organization together around this set of ideas.
So, that won't happen.
And we will see how they work with this "independent safety board" that they have put in place.
All of what we're doing is self-education.
It is organizational education.
It is self-education.
It is process education.
So, we will see how they will bring all that together.
The reason I asked that was because actually, in terms of reading this recommendation, I read it a little differently.
I thought it was kind of a push to move past the Shuttle to say that recertification is going to be too complicated.
Well, there might have been a little bit of that.
I mean it really was up to NASA to decide whether a process that was so resource-intensive was worth doing, I mean given the fact that the Shuttle is in the sort of sunset of its career.
I guess I felt that we should leave that to them but that they could not continue to operate the way they were operating.
I think that was really the message.
They had to make a decision.
Sheila, the board's recommendation of the number of 2010 has had enormous impact.
I know.
Of course it has.
Very negative from my point of view.
Say something about how you picked that number and how flexible it is?
Well, recall we did this in 2004.
We wanted to give NASA enough time to get back to flying.
And, of course, the whole definition of what recertification means is a bit unclear.
In other words, I would think that NASA could have gone in.
I think what we saw was we needed to get a couple of flights under our belt, but we needed to go back.
I think what we meant by recertification is a relook at the whole mission.
What is that thing called?
Mission rules?
No, not mission rules.
When they get together and decide whether they are going to fly.
Mission readiness review.
Flight readiness review.
We felt that the flight readiness review was broken and that part of the reason it was broken is that there were too many requirements that were silly like the Shuttle had only two wings instead of four because you'd need redundancy.
Wings are important, and if you really thought they were important you would have four instead of two.
And every time they did a flight readiness review you would have to go through these, what I would call silly requirements.
One issue about recertification was to try to narrow down to what were the key systems that needed reexamination in the light of technology and what we know about the Shuttle?
That, in and of itself, would have been a very useful process that would have helped this.
And so, I think what we saw is that there is a set of processes that need to be carried out after a couple of successful flights that would really position the organization to do its job.
107:00 They wasted a lot of time.
But suppose someone had come back to you, or members of the board and said well, life would be easier for us if we called it 2013?
Well, I would think if you're going to do that, that NASA should come forward with a plan at the minimum of how they're going to operate the Shuttle safely through 2013.
They have not even thought about doing that.
They are carrying on with "business as usual".
Flight, flight, flight.
And they are not thinking about this more fundamental issue which would, I think, be very useful for them as an organization of what are the critical issues if we wanted to operate the Shuttle to 2013?
They just kind of got off the hook by saying 2010, we can do that.
You know, pretty shallow.
Yes?
And this is going to have to be the last question, unfortunately.
Anyway, you're all invited to my office to see my big poster on the flight.
Your recommendation for not flying unless you have a safe house in orbit.
No, we didn't recommend that.
Oh, safe house in orbit.
OK.
If you would have said that at the beginning of the Shuttle Program, that would have cut out dozens of flights.
Sure.
But, you see, at the beginning of the program, NASA was committed to a system that met its requirements.
Once they backed off of a system that met it requirements, they left itself open to operating a risky system.
And so then you come in, after having two accidents, and say OK, how can we operate this system safety?
You've already demonstrated that you're willing to back off on requirements so how can we identify what some key issues are so that you get back to operating safely?
What we fundamentally recommended was that you should develop onboard repair capability.
And that's really all we said.
And we sort of left open how they would do that.
And so they investigated a lot of things and decided that it would be a heck of a lot easier if they could use the Space Station for a repair.
And so they themselves said OK, Space Station only.
That really was not our recommendation.
Although, we understood that it would be a lot easier to do Space Station repair.
OK.
Thank you very much.
[APPLAUSE]
Good morning.
I am filling in for Professor Hoffman who is off addressing the French Parliament at this moment, or maybe it was yesterday afternoon.
Tough choices of whether to stay here or go there.
Well, that's right, Noah.
These days the issue of knowing where you are, navigation is really taken for granted.
We have GPS in our cell phones.
GPS in handheld devices.
GPS tells the taxi driver whether or not he should be turning up that one way street.
When we began with human space travel, particularly as we entered the Apollo era, the question of navigation, along with guidance and control, was still a major issue.
In fact, it was rather uncertain whether or not, in the Apollo mission, we were going to be able to, with assurance, do all of the navigation required for going to lunar trajectory and then doing the precise navigation to land on the moon.
The history of leading guidance navigation and control has now, for a period of over half a century, been located just adjacent to, at one time part of MIT, the Instrumentation Laboratory, now Draper Laboratory.
And the tradition carried on through Apollo to the Space Shuttle Program.
And then, as I think as you are aware, beyond that into NASA's current plans.
We're privileged today to have discuss with us the guidance, navigation and control issues on the Shuttle, Dr. Phil Hattis.
Dr. Hattis, a graduate of Northwestern and Caltech with his PhD from the Aero-Astro Department at MIT, has been at Draper since 1974.
He is a member of the Laboratory Technical Staff, which is the highest technical position available there.
He serves as the Technical Lead for the Crew Exploration Vehicle Development Program in GN&C at Draper Laboratory.
Phil has been very active in AIAA.
He is a fellow.
He has been head of the New England region.
Has received the Draper Lab Distinguished Performance Awards and various NASA recognition awards for his contributions to STS-1 and STS-8 missions.
Then we will hear about Draper's contributions and the overall issue of GN&C.
Thanks a lot, Larry.
I should just point out, when I came up here as a graduate student to pursue my doctorate, I was a Draper fellow.
I started working on the Shuttle as a Draper fellow, so much of the work I will be talking about here, I was probably only a year or two older than any of you when I was doing this.
And that's turned out to be fairly useful to NASA because with the Shuttle still flying from time to time issues come up and they stick pick my brain about what we did in 1974, '75 and '76, which wouldn't had been so easy if I had been at my current age then.
But it is also a little bit alarming because it means the people that are working on the system now don't really understand why it was designed the way it is.
Now, the other thing they could have asked me for was the report on which this material was based, which I wrote in 1983 to educate the rest of the people at Draper who were going to work on the program subsequently about this system.
Now, the other thing I just want to say is feel free to interrupt me at any point to ask questions.
I am going to be covering a lot of ground, and I may not get back to the area that you're interested in asking me about if you wait.
So just raise your hand or speak up or whatever you feel like.
And the other things I want to point out are two things.
One, what you're going to be looking is largely what was done for the first Shuttle flight except where I mention specific upgrades.
Now, I am not going to be comprehensively covering all the upgrades that have been done to the Shuttle since.
What you will also note is this presentation will be largely monochrome.
And why?
Because it is drawn from a presentation of 1983 when there was no such thing as a laptop or PowerPoint and color was a real pain to get.
You had to go to the artist and then get it lithographically reproduced which was an incredibly cost.
What color you see, I've added now and it's limited, except for a couple of pictures at the end.
I do have some before and after cockpit pictures at the end.
You will periodically see a chart like this, topics of discussion, we'll go from section to section and we're going to delineate the areas I'm going to cover.
There are going to be a whole bunch of sub-bullets for each of these areas as I go through.
This will be not real deep, unless you ask me the questions and I will go as deep as you want with the questions but covering a lot of ground.
Some of these pictures you may have seen in one form or another, but I should just highlight certain points.
The systems related to the flight control were placed all over the Shuttle.
In the forward area, which was the only pressurized portion of the Shuttle, below the livable areas was the avionics bay.
And, in there where the computers, the inertial measurement unit, and I want to say something about that in a moment, what they refer to as multiplexers-demultiplexers.
You've got a lot of analog systems or you had to convert back and forth between digital and analog.
And the electronic boxes that drove the commands for the reaction control system thrusters.
And then you have hand controllers and displays and indicators in the cockpit.
In the back you have pods which have many of the reaction control system jets, the orbital maneuvering system thrusters.
And I will be talking quite a bit more about them later.
And there was also in the aft avionics bay that had specific subsystems for which it was deemed unacceptable to have them forward.
Some of them were local analog digital conversion boxes, but also rate gyros which were used during assent and entry where they wanted them closer to the center of mass by being in the back and in the front avoiding some of the flexure issues associated with the long distance to the front.
This particular configuration you're looking at is before the external tank separated after the solid rocket boosters had come off.
This configuration is where the story begins because what I am going to be talking about is the part that Draper did which is the exoatmospheric flight control system.
And that begins at main engine shutdown and ends when you hit 400,000 feet on the way back.
There are different phases that we will be talking about.
The first is what we refer to as insertion which is from the time the main engines cut off to the time you do initial orbit circularization.
And that includes a brief but design challenge phase while you're attached to the external tank.
It includes the separation maneuver from that.
And, in the original flight profile for the Shuttle two burns of the orbital maneuvering system, the original orbit insertion strategy for the Shuttle put it in an orbit that typically had an apogee of about 60 nautical miles, a perigee of just a few nautical miles.
What you would do is the first burn would raise that perigee up to the 100 plus nautical mile target altitude and then the second burn halfway around the earth would put you into a circular orbit and then you would begin your mission there.
Later in the program for overall efficiency, in order to improve the payload margins, that strategy changed.
And they tended to do more what they call direct insertion which had a substantial higher apogee.
The perigee wasn't much higher but you would end up somewhere between those two.
And then you would do one OMS burn and you would get a net gain of maybe one or two thousand pounds which became very important.
During this insertion phase all the applicable sensors were on.
Now, I said I was going to say something about the IMU.
And most of you don't even think probably these days about having rate gyro separate from an inertial measurement unit.
You get these combined packages called inertial navigation systems now.
They're actually navigators that have the software.
They do all the processing for you.
They even have built in GPS receivers.
Of course, we didn't have GPS then.
The inertial measurement unit was a pretty clunky gimbaled device at the time the Shuttle first flew.
It subsequently got upgraded to ring laser gyro systems.
But that inertial measurement unit was simply outputting angles that were significant throughput issues.
Because you had to do a lot of data crunching to get rates from that.
Computers are really slow.
I will talk a little bit more about that in a few minutes.
Having rate gyros separate was a way to get data that these days would be all built into one box.
General purpose computers, all of them were on during ascent.
I will talk about the partisan of them, but there were actually five of them.
And the Vernier reaction control system, which is a small group of jets, and I will point those out later, were not active.
The larger thrusters were.
And then, after the second OMS burn, you transition to another flight phase.
And why all these flight phases, I will get to in a couple minutes also.
Then orbit phase began after the second burn.
You quickly open the doors so that you could dump waste heat.
The radiators are on the inside of the doors.
And, when the doors are closed, the heat just radiates back into the vehicle.
All payload operations are during this phase.
And you did a lot of power down to save.
You're working off of fuel cells.
It limits your mission life both because of the limits on the weight of the reactants and on the places you can put any extra tanks.
So you turn off the rate gyros, you don't need those anymore, and I will explain why.
Two of the five general purpose computers were shut down.
This also being a late `70s computer design, you're talking maybe on the order of a couple hundred watts per computer which, by the way, I will explain more about, but was a 104,000 word memory capacity.
That was actually an improvement from the approach and landing test when it was 64,000 words.
And then you turned off two of the three redundant inertial measurement units, except for critical phases.
Also the safe power, the feeling was that if you lost your navigation reference you would have a relatively benign environment to bring one of the others up.
And the Vernier thrusters were made available because they were used for flying control.
Then you have the deorbit phase.
You close the doors again.
You do the deorbit burn.
You dump any residual propellant from the forward tanks by simultaneously burning opposing thrusters in order to get an acceptable center of mass for entry.
Getting the acceptable location of the center of mass for attitude and thermal control during entry is very critical.
You reactive all the sensors.
You go back to all the computers being up.
Then you turn the Vernier jets back off and you fly the vehicle using this mode until you're at 400,000 feet which is about where you pick up 0.05 Gs.
And that is where the entry phase takes over.
And this is just a summary of the profile.
Now, this is where I wanted to take the opportunity to talk a little bit about computers and profiles and everything else.
When we started this program with a 64,000 word computer, I talk in terms of words instead of bytes.
The architecture of this computer didn't have bytes.
You had words and half words.
Each word was equivalent to about four bytes in terms of number of characters you could insert into it.
But you only could break it down into pieces of two.
So we had 208,000 - well, a thousand times 24 pieces of memory that we could work with on this computer.
For the approach and landing test, which was very limited, you flew it off of a 747 for a couple of minutes, 64,000 words worked just fine.
And they were chugging along with the program saying we're going to get all this orbital mission stuff into that computer.
And, of course, we discovered probably a year after we really began the job, we began the job seriously in '75, and by '76 it was obvious 64,000 words wasn't going to work.
They upped it to 104,000 words.
It was probably obvious four months later that 104,000 words wasn't going to work.
So the solution, in addition to descoping as much as possible, what you had to have was to separate the computer loads that you had for up and down which you did when you were doing your orbital mission.
And then you had something called the mass memory device which is basically a tape drive which when you went from this phase to this phase would reload some of the computers.
And then you went from this phase to this phase we'd reload them again.
And I said there were five of these computers.
Four of them, of what I'll be talking about, were the primary computer set.
Quad redundant so that they would vote all data going in and out to decide whether or not there was an inconsistency between one computer and the other.
And it would automatically deselect the bad computer, the implications of which I will talk about probably about three-quarters of the way through the presentation.
When you went up and down, all four computers were operating the same software.
The fifth computer was called the backup flight control system.
And the reason it was there, these four primary computers, a chunk of that 104,000 words, probably 30,000 to 40,000 words was used to assure the computer set operated successfully redundantly.
There was always a concern that there would be a generic software error that would show up at some bizarre time and that you could pull down the whole computer set.
It was an independently coded similar architecture from the standpoint of algorithm content, but independently coded software called the backup flight control system that on the hand controller there was a button the crew could hit, a panic button, if they needed to.
And the system would revert from the primary system backup system.
It never happened in the history of the program, it has never been used, but it is still there.
And so those five computers are all operating on the way up and on the way down.
Now, when you go to the onboard phase, and only three computers are up, you freeze dry, as they say, to the computers.
One computer remains the backup flight computer turned off ready to turn on from emergency entry to backup.
The other one is a primary software load ready to start.
The remaining three computers, two of them were redundant set for the on-orbit functions.
And one of them, again, because of memory problems, everything payload related was in a system management and monitoring other non-flight control related functions with one of the other computers.
They did this spread of functions across computers, in addition to adding the tape drive in order to accommodate the memory constraints because the Shuttle Mission was so much more complex than what the computers were originally designed to accommodate.
So you say it never happened.
Have there been instances in which any of the backup computers have been brought online?
There have been instances in which primary computers have failed.
There has never been an instance in which they've reverted to the backup system.
Now, when the primary systems fail, and I will elaborate this in more detail, while each computer computes the functionality for everything, they only control more or less a quarter of the subsystems.
And there is a distribution.
And some of the charts towards the end will talk about how this was done.
And there are implications associated with what you lose when the computer goes down.
But, if you have time in your noncritical flight phase, you can restring those things to the remaining healthy computers and recover accessible systems even though that computer has gone down.
Now, on STS-9, that was incidentally our MIT department's first Shuttle flight.
And it had, Byron Lichtenberg, one of our people.
And what turned out happened to be floating solder balls and an early version of these computers that caused intermittent shorts.
And one computer went down before entry.
They recovered the string.
Another computer went down during entry.
And, because they had reconfigured the string, they had three of the four strings left but only two computers.
And another one failed on touchdown.
And had that one failed before touchdown they probably would have reverted to the backup system.
That's the closest they ever came to a backup system.
Now, the question that comes to my mind there, since the generic failure was not software but floating solder balls, which all the computers were susceptible to, what would have happened if they had gone to the backup system?
Because it could have gone down, too, and then they would have had nothing.
Because, once they've gone to the backup, it is not easy to revert in critical flight phase to the primary system.
Lichtenberg, as I mentioned, was a crew member from our lab.
Later I asked him how did he feel when the first computer went down and then the second computer went down?
Byron has an aero-astro PhD and pretty savvy.
He said he was pretty worried until he looked at the commander, who was John Young, and John said well, we might as well go to sleep because we're not going to reenter today.
And when John went to sleep he said he might as well go to sleep, too, and it will be all right.
And it was.
Yes?
[AUDIENCE QUESTION] The computer was the same.
The software was not.
The computers themselves are all interchangeable AP101 computers.
Subsequently, they were changed to AP101S computers, which is modified version that was used on the B-1 Bomber.
And they went to 256,000 words of memory.
And that is the current state-of-the-art.
You have to understand several things.
One, it is very expensive to upgrade systems that are already flying.
But, independent of that, you never fly in space something that is close to the state-of-the-art because you have to go through all the qualifications, which takes a lot of time, and it has to be radiation hardened when you're in space.
And when it's a human vehicle, it has also got to go through human qualification.
The Space Station architecture is IBM 386 processor quality, so it is basically like maybe about 1986, '87 early laptop generation computers.
[AUDIENCE QUESTION] Oh, yes.
There were quite a few missions.
I don't think there have ever been any missions where they lost two.
The IMU failures don't seem to have been anything systemic but just sort of a random problem.
And I'm not sure since they have gone to the ring laser gyro systems that they have had any failures.
I think it was the mechanical ones from the early days that they had some problems.
Now, major requirements for the systems.
We had two different autopilots for three phase transmission which incorporated both the insertion and the orbit features and on-orbit, so we only had two different software loads on that tape drive for the primary flight control system.
The rules for Shuttle were that after any subsystem failure you have the capability to remain operational.
No critical functions were lost.
After two failures, safe operation, all critical things necessary to terminate the mission and bring them home would be possible.
But some of the mission objectives may not be achievable.
The system was mainly aimed at controlling rigid body characteristics and velocity changes within specification.
Only when we started to look at docking with the mirror in the Space Station did we start worrying about flexible body effects.
And it was because of what they were attaching to and not because of the Shuttle, all those appendages and those things.
If Shuttle does control Space Station and did control Mir when it was docked.
And so there have been some modes, which I won't really be talking about today, but were created to facilitate that control without causing unacceptable loads on those flexible appendages of the stations.
And 80 millisecond app cycle, two reasons that happened.
We originally planned to do 40 milliseconds.
The approach and landing test program was.
First there is less process of burden if you do it half as often.
And the other, when it came to the reaction control system, as we discovered in about early 1980, a little more than a year before the first flight that there was a water hammer effect in the propellant lines on the RCS jets were the opening of the valves caused an expansion wave which reflected as a compression wave.
The closing of the valves caused a compression wave.
The inner section compression waves, if the valves were opened and closed too quickly, could cause a catastrophically large compression wave that could burst the line.
So they deemed it better rather than redesign the entire feed system to limit us to never firing faster than 80 millisecond cycles.
Now, that differed from Apollo.
Apollo had 100 millisecond cycle time, but they had the ability to interrupt the cycle to turn out enough jets if they wanted a short firing.
We couldn't do that because of the water hammer effect.
And also because the propellant feed system involved a liquid in the tank with zero G acquisition devices around the surface of the tank directly exposed to the pressured helium, which means some of the helium dissolved into the fluid.
If the fluid was drawn too fast the helium could bubble out causing a gap in those zero G acquisition devices around the tank preventing flow.
If you get unbalanced flow of hypergolic propulsion systems you can also get an explosion.
So the solution to that was limiting how many jets you could fire at one time off of one tank.
Does everybody understand what the zero G acquisition problem is?
It's a surface tension base.
You have various types of rings and shapes on there that captured fluid by surface tension to there, which would begin to draw the fluid.
And once you began to get some flow because of firing things it would pull the blobs of fluid into the tanks and into the feed lines from the tanks.
As you recall, the origin of the problem is that the fluid would not be at the bottom of the tank so you risk drawing a bubble.
The surface tension holds enough there to start the firing.
When you fire you get some force.
The force draws the blob to the same parts of the tanks where you can acquire the fluid.
And, as long as you don't draw too fast, the communication between that part of the tank and the fluid remains.
This was a very complicated qualification program.
A lot of C135 parabolic trajectory time got used to test out various perturbating of the inside of the tanks.
And I am not going to talk about that in detail but I'm sure there are a lot of papers out in the literature about how they qualified these things.
And I don't think there were too many systems before the Shuttle that actually did it this way.
One used systems that tended to use membranes where you had the pressure on one side and the fluid on the other side, and the membrane would just force the fluid to stay in contact.
But the problem is the membranes would degrade over time on exposure to these hypergolic propellants.
Hydrazine and nitrogen tetroxide are very chemical reactive materials.
For instance, it was supposed to be used over many years up to a hundred times.
The idea that you would have periodically keep opening up these tanks to change a membrane was not attractive when you consider the tanks are deep inside of the structure.
So this may have been a unique issue because of a reusable system, but it also may be relevant to systems that even if they aren't reusable have to have a very long life in space.
Modes and submodes I will quickly go through, but you have rotation and translation modes.
These modes could be used simultaneously for the RCS system and then separately for the OMS system.
And you had special modes which to get extra oomph out of the RCS jets, if you had an abort, that would sometimes be done if the OMS engineers weren't available.
The OMS engines, there were two of them which were up on the back right and left pods from that picture I showed a few minutes ago.
And you could use one or two depending on what you were doing.
And in the RCS rotation modes, you had various ways you could use it.
Proportional meant you moved the stick.
The response you get is in proportion to what you do, how long the jets fire.
Discrete means you move the stick and you get a specific amount of rate change.
Pulse means you just get a single little pulse out.
And acceleration means as long as you're holding the stick out it keeps firing them.
And those are all.
There were push button displays that the crew could adjust.
There were some modes for each of those which affected how many jets fired, whether or not you wanted to force it to use something that approximated couples or you were in propellant conservation mode and you were willing to get accept getting a rotation rate.
With coupled in the translation you would care about that effect.
And in the translation there were various submodes as well.
And, in particular, you would use more jets when you were separating from the external tank to get away as fast as possible.
Yes?
On the previous slide, are there different modes for the RCS jets?
Was there one that they happened to use the most or was like supposed to be the primary usage or are they all part of the normal operating?
I think they would rarely use this or this.
I think these two were commonly used.
This is very fuel inefficient.
It would only be used as kind of an emergency measure.
That's also very difficult from the point of view of the pilot to have direct acceleration control.
Right.
I mean this might be something you would call upon if actually, for some reason, the vehicle started to spin up unexpectedly and you would have to neutralize that.
Does that adequately answer your question for now?
There will be an opportunity to collect a little more detail on that later.
But, again, you wanted to get off the tank quickly so you would use all the jets you had to get off of it.
You wouldn't do that later.
You would have more jets that you would use for roll control while you're on the tank because you had a much higher roll inertia.
You want to only spend a few seconds on the tank after the main engines shut down.
You could often be left with residual rates you've got to quickly kill.
There was an inhibit on the separation if you had more than half a degree per second on each of the axes.
There was actually a phenomenon on the first Shuttle flight which almost got us in trouble.
One of the first things that happens after the main engines shut down is you slew the engines back to stow position where you want the engines for entry.
The reason is the auxiliary power units are needed to move the main engines.
You want to shut those down and save the hydrazine for those until you get back to it just before entry.
On the first Shuttle flight they kicked those engines at about one hertz.
It turned out the first fundamental mode, the rock mode of the orbiter on the external tank had a subharmonic of about almost exactly one-fourth of that slew rate on the main engines.
The slewing of the engines then causes the rocky mode to be excited.
We were seeing oscillations very close to the inhibit for the separation.
The crew was getting a little worried but we just got it in bounds in the automatic mode.
And if they hadn't separated in time it would have gotten pretty complicated to do it manually.
We made quite a few changes after that.
There were a lot of things we learned on the first flight, and I will point a few of them out as we go along.
There was also a launch pad phenomenology.
It is not part of my talk.
Has this come up in the class about the shockwave from the SRB ignition?
Yes.
OK.
They didn't have those waterbeds in there on the first flight.
What was relevant here, one of the things that you may or may not know is that the struts that held the forward RCS tanks on the first flight were buckled almost to failure.
That wasn't realized until they got home, but had they failed they probably would have burst and blown up the vehicle.
The on-orbit modes, we have both primary and Vernier jets which were only used separately, except under special circumstances which were designed substantially after the first flight.
We have local, vertical and inertial frame of reference control capability with respect to the discrete rate mode.
And, otherwise, it is pretty similar to what you saw in the previous picture.
And submodes, some features were added because of rendezvous.
You could fire a lot of jets on the fort side.
If you wanted to do a rapid breaking, you could inhibit all the jets to fire in that direction.
If you wanted to limit the use of plumes, it turns out that you didn't lose completely your translation control authority.
Because the jets, in the front and back, that were in the x-axis coupled about 20% of the trust into the z-axis.
So if you fired them simultaneously in both directions you could avoid pluming an object in front of you and still get enough translation to control that axis.
And, just to show how everything was hooked up, the inertial measurement units and rate gyros and hand controllers and panel controls all went in through these multiplexer devices feeding the signals then into the computer and displays.
And then you had outputs going through these similar types of electronic boxes specific to the reaction control jets that generated the commands that were needed by the solenoid valves that actually opened and closed the hypergolic feed lines.
Looking at the whole top level architecture then of the GN&C system is all those boxes feeding into what was inside of the computer.
Inside of the computer you have subsystem operation software managing each of the subsystems doing the redundancy management.
The specific guidance navigation control algorithms.
There was a moding and sequencing function which was based on both manual and automatic scripts.
Then the actual driving of the displays of controls is interactive with the flight control system both ways providing feedback to the crew members and accepting their inputs.
And then what was left was the system management function on this computer that didn't go into the separate computer.
By the way, anything related to robotic arm operations were operated through that separate computer.
One of the issues that came along later in the program is the flight control computer never knew what was happening with the arm.
If you take a space telescope sized payload and put it 40 feet out there, it drastically changes the mass properties.
And one of the features you will see a little later that we stuck in was the ability for the crew, by pushbutton, to select different tables about expected accelerations of the jets because we didn't know when we needed to respond to that.
There were also flexure issues with the arm which came up as they went along, too, and established constraints on how we operated.
But was there any thought given to having an adaptive system which would identify the current parameters?
That would have gone way beyond the capacity of the computers that we had.
It would be relatively easy to do with the programs we had but it would never fit in a 104k memory computer.
But there has been lots of work that some of my graduate students have done over the years of how we should have done that.
Brent Appleby, who is now a division leader at the Lab, actually did some work back in the `80s, I think that may have been his master's thesis, on some of those issues.
[AUDIENCE QUESTION] But, in reality, we're approaching very large memories which would allow you to do it.
Well, when we go into CEV, we're probably going to assume that 100 megabytes is no big deal.
[AUDIENCE QUESTION] unfortunately was cancelled, but we were doing exactly that.
We knew exactly the position [NOISE OBSCURES] all the movement when we were grappling.
And we were changing all the tables based on the current angles [NOISE OBSCURES].
It is a real challenge to maintain stability on a system where you have no insight into that.
But you would never build a spacecraft that way today so I'm not sure.
The challenges we have are very interesting but probably no longer relevant.
The challenges that remain are the flexural dynamic interaction problems.
I just wanted to indicate that within the control laws you have a steering processor, which I will talk a little bit about more, an RCS jet processor, a state estimator and an OMS processor.
The state estimator is unique to the on-orbit flight.
And there is more that represents that in a minute.
So I am going to now, having entered the overview, go into each of the subsystems in more detail by pulling up this picture to talk about it a little more, these subsystems in the context.
The forward RCS system had 14 primary 870 pound jets and two Vernier 24 pound jets.
There were 24 and 4 respectively.
Those systems in back evenly divided left and right.
Each of the pods in the back had one OMS engine.
The forward RCS system had its own self-contained hypergolic tanks.
The aft system had RCS tanks and OMS tanks which could be interconnected from within the pod or could be cross-fed across the pods.
Now, the consequence to the flight control system, there were different constraints and simultaneous jet firings.
And how you counted, whether it was only left or right or both, depending on which mode you were in, if you had a mission and needed a lot of RCS propellant and you had spare space in the OMS tanks, it allowed benefiting from that.
Why are there so many thrusters pointing in the same direction?
For redundancy?
Yes, redundancy and maximum control authority.
You want to find control authority normally with redundancy but for external tank separation you wanted high acceleration in one direction.
For high rendezvous breaking you wanted high acceleration in the other direction plus or minus Z.
And for backup to the OMS engine you wanted to have higher acceleration in the plus X direction.
And then, when you were doing entry, which I'm not talking about today, you had an on-demand RCS control authority in the upper atmosphere during hypersonic flight.
And you would turn on one, two, three or four yaw thrusters in particular as needed.
I think one, two, and occasionally three thrusters had been turned on during disturbances.
One of the things we've learned from the telemetry of the Columbia accident is as this vehicle was falling apart for probably 20 or 30 seconds, the vehicle was controlling very nicely because they kept turning on more and more jets.
They were getting major torque imbalances from missing pieces of the vehicle.
But it was still controlling the attitude until the damage got so severe that was no longer possible.
Again, I already talked about the number of the thrusters.
The primary thrusters, for the reasons we talked about, there were many jets.
And also because translation and rotation control was accommodated by these Verniers, a feature I will mention briefly later.
It was only a rotation control system and fundamentally does not have redundancy.
I already talked about the on-time.
These are just typical propellant loads and thrust levels and specific impulse numbers.
You notice that large maneuvers are always a little more efficiently done with the primary jets and even more efficiently still with the OMS as you will see in another chart.
Life for duty cycles and on-time are relevant for a vehicle that is going to fly a lot of missions.
Flying control almost always will be done with Vernier jets, not just because of propellant, which it is much more efficient, but also because you will get a lot more mission life out of it.
This is a stick drawing of those.
There is a numbering system associated with it.
FRL for which pot it's in.
Up, down, forward, right, left.
The last character for which direction it fires.
And then the middle number is a manifold.
If you see a five that is the Vernier jets.
One, two, three or four, any pod that has got the same middle number is on one manifold.
If a failure shut that manifold or a string took down that manifold, all of those jets were lost.
When you lose a string, you would have one manifold per pod that you would lose.
That means under some circumstance, because the Vernier system is redundant, a single failure could take that out.
But that wasn't critical to carrying out most mission objectives or to safety.
That is probably enough said on that.
The OMS, typical propellant loads mentioned here.
On-time, never less than two seconds.
Never less than, with one engine, 12,000 pound second impulse with a 6,000 pound thrust not suitable for fine maneuvers.
The RCS jets would always be used to trim out any large OMS burn errors.
Had a significantly higher specific impulse.
For large maneuvers you're clearly better off from a propellant weight perspective using that system.
Each of the engines had redundant gimbal control, redundant by having one mechanical screw system, but you could drive the nut or you could drive the screw.
And there were different electrical systems that did that.
This was the maximum authority.
And the two axes at each engine could move a portion of which was used to tract the center of mass as the vehicle consumed propellant or delivered payloads and a portion of which was for actual thrust vector control management.
We also had to subtract a little bit.
You never want to go too close to the hard stops because you risk mechanical failure by doing that.
And you always have a little bit of mechanical uncertainty of exactly where you are anyway.
A portion of that was a mechanical uncertainty and a portion of that was just a mechanical safety margin.
This is a drawing in the two different planes of the rotation of the engines showing the span between the center, say about 15 feet apart, no surprise given the general cross-section of the Shuttle.
An important thing to notice is that the engine, while I can point through the CG, is not pointing anywhere near the body access of the vehicle.
And, by the way, the vehicle body axes weren't -- There was a significant offset between the principle axis of the vehicle and the body axis component of inertia, and the body axis was quite large.
Three units, even with the replacements they have gone through over the years, there remains three units for the INSs now, but the IMU where mechanical systems with quanta for knowledge of state which were not all that small.
And the RGAs were even worse, the rate gyros.
These quanta were quite significant.
The reason was we had a half-word in that multiplexity multiplexer for translating the analog signal to a digital signal which determined, based on the maximum range, the maximum range being dictated by maneuver rates where possible during ascent and entry and not on orbit.
But we were stuck with that.
It was hardwired into the cards.
And so these quanta then, we discovered there was a one sigma probability of an every third cycle one quanta noise spike on the data that came across the MDMs.
And that was pretty significant when we were trying to do fine control during the transition phase.
Were the noise levels determined by this quantization, in effect, the [NOISE OBSCURES]?
The noise phenomenology was related to the electronics of the card, but it was directly related to the least significant byte.
But the mechanical sensors themselves were superior to that?
Yes, they were.
It was the MDM card that introduced the noise.
If you had used one word from the MDMs that would have [NOISE OBSCURES]?
Given the state-of-the-art to have processed that much information across the MDMs would have made it too slow.
We're talking orders of magnitude slower electronics from the late `70s than you have today, so the half-word was dictated by the data rates that we required of these boxes.
Now I'm going to go into specific features on the software side.
Yeah?
With the gyros, it seems on Hubble and Station, for example, we always hear bad news on the gyros having to be replaced every so often.
Well, first of all, what I'm talking about here, at the time the Shuttle first flew, are mechanical gyros.
Hubble may have started mechanical gyros.
They have been changed out at least once or twice.
And are probably fiber optic systems.
Hubble has the problem that it's in a fairly high orbit, 300 plus nautical miles high for the Shuttle anyway.
It has a significant radiation exposure, particularly when it goes through the South Atlantic Anomaly of the radiation belts which are 300 nautical miles.
Much more time they spend on that than at 100 nautical miles.
Cumulative radiation damage to the electronics in those gyros is probably a contributing factor to the failure rates that they're seeing on Hubble.
I would say they must spend a few percentage of their time in the South Atlantic Anomaly at that altitude.
Are the gyro replacements we're talking about measurement gyros or attitude control gyros?
Well, the fine guidance gyros are the ones that have been the big problem on the Hubble.
We're talking about quantitatively a different regime we're operating here.
The quantization, take away the noise effects, was something we could live with for operating the flight control system.
In Hubble, you want to be able to measure rates two or three orders of magnitude lower than what we're talking about here.
The actual design of the sensors is substantially different because they're trying to get very, very tiny little rates out.
These are trying to maintain the image lock when you're using the full magnification capability of the telescope and whatever target that it has.
Nevertheless, I would imagine that if the Shuttle gyros are staying at 300 nautical miles for five to ten years, they would fail, too.
Radiation is one of the fundamental drivers for all missions.
And it often that Atlantic Anomaly is one of the big drivers for low inclination.
Space telescopes are 28.5 degree inclinations.
So they don't have to deal with the magnetic fields coming in toward the poles which a polar mission has to, but there is this big dip in the Van Allen Belts off the coast of South America which poses a problem for everything that isn't low orbit.
The functionality we have in the autopilot, we use the rate gyros and the inertial measurement unit on the transitioned app to get states direction giving us altitude, giving us rate.
On orbit we have the gyro shutdown to conserve power, view data only.
That then dictates that we are going to have a state estimator on orbit.
We have to put some special features to overcome the rate gyro noise in the transition phase, which is irrelevant when the rate gyros aren't operating in the on-orbit phase.
We have the Vernier jets and the associated algorithm logic for on-orbit which isn't in the transition phase.
We worry a lot more about every detail propellant deficiency on-orbit because we spend so much more time there.
We have a lot of features we've added to minimize propellant there.
The OMS capability in the two phases is actually identical with both of them having a capability to wrap around the RCS jets to the thrust vector control should the thrust vector control not behave properly during that OMS burn.
And this just delineates the various features we have for steering.
OMS and RCS.
Where we add lots more features because there are a lot more things you are attempting to track when you're doing your mission on orbit than when you're just trying to get to and from orbit.
Notice the rates that we're talking about.
Typically, you look at an INS box these days and you see hundreds of hertz data rates.
What restricted us here was how fast could we -- For instance, none of the software was in the sensor package.
It was in our computer.
And we had a severe throughput problem.
And the solution, since we had the rate gyros for rates and we only needed the IMUs for attitudes, and rates could be used to extrapolate attitudes for a reasonable period of time, was to greatly reduce the processing rate of the IMU down to on the order of one hertz, submultiple of 25 hertz which is why it is 1.04.
On orbit, since it was our only source of information, we had to eat a larger processing burden, operating on a 6.25 hertz, but we still were extrapolating in between the state estimate of one time constant.
We'll talk a little bit about what we're doing with that, but everything down here is operating at 12.5 and we're getting the data in at 6.25.
There are a lot of things you would do differently simply because you don't have these low rate constraints due to throughput limits.
The architecture then for the autopilot that is being representative on orbit is you would have a maneuver module where all the features for steering the vehicle would be, kind of an adjunct to guidance.
You would have these modes controlled by the crew and the push button display of what the stick deflections would do.
You would have the phase plane which would be tracking attitude error, rate error and whether or not you should fire jets as a function of those errors separate per axis.
Are there people here that don't know about phase planes?
OK. Well, the concept of the phase plane, you go to optimal control theory and you look into a situation where you have a control effector which is on or off by directional, which is what thrusters are.
And you look at what it takes, given an error in a plane which is attitude error and rate error, and you want to get minimum time to neutralize that error to zero.
51:35 But you have lots of dead zones, which I will talk about more, to assure that you don't inefficiently use the jets so that you are not constantly trying to fire the jets to get exactly to the origin which is never possible.
The state estimator, which is a form of a Kalman filter.
And then the jet selection logic which is essentially for the primary jet lookup tables.
But it turned out to probably be the first use of real based intelligence.
We didn't think about these things in that time.
And, when I talk about the stringing later, I think we were also using an early version of failure tree analysis.
But, in the 1970s, none of these things were named.
And then you have various loads for the parameters that determine these dead zones and tables and all that.
And the crew would select these from push button displays.
For the OMS, you could either have hand controller inputs or cross-product steering and based guidance inputs which would then go into roll and pitch and yaw.
Thrust vector processing channels were roll and pitch coupled.
Roll is only possible, of course, when you're firing two engines by differentially pitching the gimbals on the two engines.
This would actually be an RCS loop automatically with one engine, but you had to have the two pads coupled when you were trying to do both of them.
But the yaw axis was separate.
Just keeping in mind the time.
Go through one or two more charts before the break?
That's fine.
I will go through the state estimator and then we can take a short break.
The state estimator, again, we were only getting attitude information at 6.25 hertz from the primary thrusters trying to maintain an estimate of the vehicle rotation rates.
We also wanted to know what disturbances on orbit you can have out-gassing the vehicle.
You have gravity gradients.
You have aero torques which have a diurnal variation depending on where you are on earth orbit relative to where the sun is which are tending to torque the vehicle in a particular direction.
Having knowledge of how that torque is behaving in a certain time enables you to manipulate your phase plane switching lines to more efficiently use the jets.
We were trying to estimate that.
Given that we only had IMU data with noise and quantization effects, we also had flexure we weren't accounting for in doing that.
So we had a low rate filter which incorporated the measurements directly and a higher rate filter which was also taking in feed forward information we're going to fire the jets.
We expect these velocity changes, rotational and translational, to occur as a result of the jet firing.
And you can build that into the estimate to anticipate that effect.
And then, given all that information, basically use that in the form of a common filter.
We had different gains associated with primary and Vernier jet usage because, given the factor of 30 difference in the rotational acceleration authority of these jets, they were fundamentally different bandwidth systems on the basis of the actuators.
We accommodated those different bandwidths in the software.
That, by the way, affected us when we started worrying about flexure on the arm because we found that some of the modes with heavy payloads in the arm actually fell within this bandwidth.
And, in the case of the Vernier jets, were falling right near the roll off point, which was the worst possible place to have a flexure mode.
And there were some significant design issues that were addressed later in the program as a result of that.
And then this disturbance acceleration -- Question.
At the time that you were designing these, was there sufficient knowledge of the structural modes or the bending modes?
For the RMS payload operations, not at all.
We first learned about that when we started looking at use of the arm to deploy the SPAS one payload in STS-7.
Nobody understood at the time we were designing for the first Shuttle flight the kind of coupling effects you would get from payloads on the arm.
It hadn't been modeled yet.
I would say probably about the time the first flights were occurring is when we started to look at that stuff, but the Shuttle software for the first flight was 95% frozen by '78, even though the flight didn't occur until '81.
You then discovered them in simulations or from flight data?
No, it was from high-fidelity simulations with the arm dynamics included in that.
We understood it pretty well.
We refined it after the flights.
We did even do some flight tests on STS-8 with an object called a payload flight test article.
The original payload on that flight couldn't fly in time so they put this 8,000 pound dumbbell on there and the arm was able to manipulate.
Went through all kinds of exercises and pretty much validated it with the simulations.
Were towing us at that point in time.
But it was a real lot of work that went into those high-fidelity robotic arm simulations and coupling that to the flight control system to get those numbers.
And a lot of work into evaluating what it all meant in terms of restrictions on the use of the control system.
And then the last thing I will talk about before the break, the disturbance acceleration estimator had a 56 second time constant.
Mostly the disturbances we are talking about were either orbital or semi-orbit, half-orbit type of periodicity.
You wanted to allow adequate time to integrate and determine their effect, but not so long that you weren't able to properly respond to it.
Somewhere in the one-minute range seemed to be about right for something that would have 45 minute periodicity.
And I think the next topic will be RCS processor.
And that is a good point to take the break.
Before we break, let me ask one historical question.
The timeframe for the design of this was mid to late '70s?
We began the work in '75.
There was a famous phase of '76 which is the Hay Scrub which is where we realized that not everything can go onto one computer.
The computer memory had to increase.
The real design architecture of what was going to be the first flight started to gel in '76.
The original expected launch date of the Shuttle was '78.
It kind of stayed ahead of us a certain amount of time, but we had probably 90% or 95% of the design done by '78.
We're going through detailed flight verification and crew training with Jeff being one of the crew members that was assigned to us.
And we would go out to Downey to do that in the '78 to '80 period.
In the design, now, you refer to the state estimator and optimal control.
By that time, had that methodology been completely accepted as a substitute for the classical control design?
Oh, yeah, I think so.
I think the phase plane concept first appeared, actually, in Apollo.
And there was sort of a rudimentary application of optimal control theory that was quite successful.
By that time I think the aircraft zoom maneuver [NOISE OBSCURES], people were happy with that.
The Kalman filter work, an early form of that also made it into Apollo.
I don't think we were actually pushing the envelope that much in using these things.
We actually carried over.
But Apollo, applying any of these technologies in the mid to late `60s, was very groundbreaking.
Any other questions before the break?
OK. Let's take five minutes.
Am I missing anybody?
I know Larry hasn't come back.
Close enough.
OK. Well, I'm going to go into the RCS processor now.
We've talked a lot about the presence of the phase plane.
We're going to go into some of the details in the jet selection.
Important consideration of the jet selection is we had to accommodate failures, maintaining control authority for any type of single failure, thruster, manifold, string, which you will see those later.
And you also had to be able to maintain adequate authority for safety with two of those combined failures.
We also had to limit plume restrictions, accommodate the tank constraints.
We talked about having couples, not having couples, the balance propellant in tanks to limit fuel usage when you didn't worry about translational coupling and minor orbital perturbations and all those other special maneuvers that we pointed out before they required higher authority.
You had your manual modes going in and then you had all these steering modes which without orbit, in addition to the ones we talked about, discrete rate and pulse and acceleration and all that kind of stuff, you had various landmark tracking modes, orbital object tracking modes which were a guidance function providing inputs to the control loop, that all had to be managed through a proper way of manipulating the phase plans.
They were tracking the errors of matter with respect to the mode that you were in and then sending, based on the errors being detected in the phase plane commands, they would be processed by the jet selection logic.
The principles of the phase plane, we figured you weren't important enough to get started.
[LAUGHTER] You had a phase plane per axis.
Roll.
Pitch.
Yaw.
You had switch lines which were shaped based on the expected torques.
You always had a parabolic feature, but much more complicated than just that.
You added dead bands because you didn't want to fire.
When you didn't know exactly where you were, you wanted to be able to get in the general vicinity far enough from a firing zone that it would stay in the general vicinity for a while and then only fire when you had to when you were diverging from that.
You had to deal in the transition phase with that rate gyro noise phenomenology.
And you had to get the disturbance acceleration on orbit.
What we did about all those maneuver modes is each phase plane had an origin, the origin being zero with respect to sun frame and attitude and position.
You could move that origin at a rate or you could increment its attitude position if you were commanding the vehicle to do something such as a tracking maneuver or a discrete rate.
And that way the errors of the phase plane we're looking at with respect to where you wanted to be rather than in absolute sense.
And you would always create [NOISE OBSCURES] in here so that if you had a big error you could spend a little bit of fuel to get to somewhere which would cause you to go in the right direction without continuously firing the jet.
Whereas, if you truly followed the parabolic switch curve you would just keep firing the jet until you were back to the origin which can be quite costly.
So I am going to show you this picture and then the on-orbit one.
These are the residual parabolic switch lines which, in the optimal control theory, would be going through the origin.
And then if you're out over in these region out here you will always fire a jet meaning a plus direction or the minus direction.
You're inside here.
You will fire until you hit another switch line on the other side.
The expectation is that you're coming in this way or you're coming in this way.
You want to get to a point where you're not likely to fire again for a while.
But then you also would be concerned because you don't automatically want to fire back right away because the rate gyro noise phenomenology could cause, where you think you are with respect to that line, to move back and forth causing you to fire too soon, causing the fire again to reverse that and get in trouble.
So what we actually did is if you actually hit this line and began to fire, each time you fired up to two or three times it would move the line out temporarily so that the quantum noise from that MDM would not make you think, even though you were moving this way, that you were going back out.
Because, if you fired that jet again, not only would you be spinning the propellant to go faster, you'd hit the other line a lot faster still.
For every time you double the size of the impulse that you use to reverse your rate, you're quadrupling the total propellant time.
Because, if you double the propellant each time, you hit a surface twice as fast before you get to the next one.
Now, on-orbit we didn't have that noise problem so we don't have that moving switch line there.
But we have this moving switch line.
If you're coming in you're looking to hit the zero line and cut off, but here you're not going to cut off until you hit the disturbance acceleration line.
It is expecting, when you hit that, that the predicted acceleration is going to say you're going to go up this way.
That means it will be longer before you hit another surface than if you start doing that over here.
So this one is a trick to lower the frequency of the jet firings based on other knowledge that we had about the acceleration.
Jet selection.
We had entirely different laws for the primary and the Vernier.
The Verniers had a very complicated configuration, had to be able to be fault tolerant, be able to handle simultaneous translation rotation commands.
It turned out, because of throughput processing limits, the correct approach was something like a table lookup.
But we had all kinds of rules we went through to say which kind of table did we want to go to?
What are the consequences of failures?
Do we want to start modifying the commands we've received because of what we basically know about which jets have failed and what we no longer can accomplish?
We actually had a Bullion implementation of those tables that actually got implemented as tables because we developed algorithms and IBM converted the software.
And they didn't necessarily do exactly what we told them.
Subsequent to that there was an experiment on the Shuttle called a phase space autopilot which is based on looking for velocity changes and optimal combinations of jets.
You would go through a linear optimal search and find the combinations.
That was flown a couple times for a few hours on Shuttle.
The experiment was quite successful but never converted into a basic shuttle capability.
But certainly with the processing capability that you have now would be a very good option as an alternative to what we did.
The Vernier jets, which were only used on orbit, only had six jets.
We weren't trying to deal with redundancies so an entirely different kind of scheme was done there.
We were looking to find the jets with respect to a three-axis command which best contributed to producing rates in that direction.
We could find a first jet that would be the best jet.
And then we could see, given how good that one is, is there another one which is half as good?
If so, we would pick it.
And, if we found one half as good, we might say if there is another one which is half again as good we might pick that and end up with an aggregate number of jets we would turn on to start producing the command.
This would be based not just on whether or not the phase plane said you should have a jet fire in an axis, but also on how big the error was but not yet to the point of hitting the phase plane line in the other axis.
And so the composite vector you were trying to neutralize would be a combination of the command and error reduction on the other axes.
And you would find the jets which would then respond your command and reduce the errors on the other axes.
Now, we would not re-compute that every cycle.
If we found that the ones or minus ones in the three axes for the commands were not changing, even though the fractional values and the uncommand of axes were changing for up to five cycles we would not re-compute the jets.
And that would minimize the duty cycles which was a life issue on the jets.
And then there was the other phenomenology I mentioned which is you start having large payloads or you're attached to Mir or something like that.
Mass properties are very different.
We don't know about that unless the crew tells us, but we could put a discrete number of alternative configurations in the tables for what we expected accelerations of the jets would be.
The crew could tell us which one applies, and then it would do this selection based on that.
The Vernier algorithm looks something like this flow chart.
Whereas, the primary algorithm would probably fill a 50 page flowchart.
The Vernier algorithm was very simple.
You go through and have a command vector which was ones and minus ones or fractional values, depending on whether or not phase planes have commanded an axis or just had a bias because of an error in that axis.
You would do a dot product of the six jets.
You would look for the maximum value of that dot product of the acceleration and the command.
And then, based on that value of that dot product, you would see if there was one that was half as good.
And, if there was one as half as good, you would see if there was one that was a quarter as good.
And then, if we already selected, we would be doing this counter of up to five times where we wouldn't recomputed.
Now, I was just talking during the break about one of the constraints on here.
When we were using these computers, the dot product of a three vector by three vector took about one millisecond.
We had an 80 millisecond cycle.
And, in that 80 millisecond cycle, seven to ten milliseconds could be allocated to control some of the guidance and some of the other functions.
Six milliseconds was being used to do these dot products.
That is why we never could have considered doing something like that for the primary jets.
The TVC processor, we have instead of discrete control with thrusters, we have nearly continuous control because we're doing gimbal steering up to the nonlinear effect of a gimbal limit.
Remind them what TVC is.
Thrust vector control.
Moving the thrust of the OMS engine by steering the gimbals on which it is attached, which you cannot do with the RCS thrusters.
What we would do then is cross-product steering.
You have an error vector and a thrust vector, and the cross-product between the two could tell you the direction you needed to steer the vehicle to turn into the desired direction you wanted to thrust.
That steering command was then used to determine the commands we sent to the gimbals for moving the engines.
You had manual and automatic modes for doing that.
Generally, we were limited to two degrees per second because the gimbals were fairly slow.
We didn't want to get ourselves in a situation where we over-steered and had to correct back which would take a lot of time and maybe cost us some propellant.
But, if we got into trouble, the reaction control system could wrap around.
And that could happen automatically if the errors got big enough or the crew could induce that by hitting the hand controller.
And you have the two pads going into the manual auto mode.
And manual always overrides auto from any of these operations.
We had a lot of things we had to tolerate in determining our filter gains.
We didn't always know the thrust direction engines all that way because of mechanical misalignment.
When you build this thing an additional misalignment occurs after you launch the thing causing changes in the gimbal drive connections a little bit.
There are errors.
You rotate to a desired burn direction with the RCS jets before you light the engines.
You're never exactly there when you start up.
You can have failures during burns.
An engine can shut down.
An actuator can stop driving.
If it is the OMS-1 burn in the early Shuttle days, during that burn they were dumping the residual oxygen and hydrogen in the feed lines for the main engines during the burn causing torque disturbance on the vehicle that we had to overcome.
The OMS burn actually helped drive that fluid out of those lines.
And then, of course, we had steering noise and bias.
And then we had to have margins for things that all liquid propellant vehicles have.
Slosh, flexure, actuator nonlinearities and sample rate effects.
Now we had a design and then we had to change it.
And the reason we had to change it is we flew for a little while, and we discovered that these brushless electric motors being used for the OMS actuators were turning on and off at a high enough rate during the burns that they were overheating making a hundred mission life maybe two or three missions.
And they could take the pods off, but they were fairly hard to get and expensive to maintain.
So they asked us to change the bandwidth.
And that's what we did beginning with the twelfth shuttle flight making these changes here.
And that caused a little bit more sloppy behavior at the beginning and the end of the burn but didn't make very appreciable difference on the performance and made a big difference on the actuator life.
There was an outer loop just showing where the cross-product steering comes in and the fact that there are various filters and digital compensation effects that are going into this to deal in an adequate manner with all those effects that I listed a few minutes ago.
The wraparound was there because we have the means to cover for large perturbations.
Why not implement it?
I mean the situation you always have with human space flight, if there was a capability you can take advantage of in a contingency scenario put it in.
This also changed, though, between the first flight and the twelfth flight.
And the reason we found is that you put this contingency capability in, but when you really study it you realize you can actually take these two fundamentally different control laws and cause them to interact adversely.
We could perturb it and then induce the jets to fire in a way that would cause it to counteract the effect that the thrust vector control system was doing.
So, when we lowered the bandwidth, we also changed some of the parameters in there to, in combination, preclude that first flight.
This is a case where we continued to evaluate the baseline system after we were flying and discovered additional potential unanticipated deficiencies that we should fix, which is saying you're never done analyzing the system even after it starts flying.
The maneuver and track modes, I mentioned that they were there, but we had this universal pointing display that the crew could manipulate.
Yes?
On that last general comment, on the last subject of discovering issues and potential problems after the system is flying.
You're doing the analysis.
You're doing your research.
What, in general, were the reactions of your NASA counterparts when you bring that to their attention?
Would they call for an immediate fix before the next launch?
It depended on the nature of the problem.
And let me mention two or three of them at one time.
I mentioned one about that phenomenology for the excitation or the rocking motion on the tank.
And then another one with the Vernier jets which I will explain in a moment.
The external tank one had to be fixed before a second shuttle flight.
There was one that potentially created a dangerous situation.
Tank separation violated safety of flight rules.
This one was a potential wraparound interaction.
It could only occur if you already had a significant contingency under fairly complex conditions.
And, if the crew knew about it, there was a procedural way to temporarily inhibit the reaction control system interactions stopping the effect.
Because there was a crew workaround, they knew about the problem, it was deemed acceptable to go some number of missions for an already scheduled software update to insert it.
Now, the third problem discovered as a result of analysis of STS-1 and 3 was a plume and pendulum phenomenology of the down firing aft jets.
The body flap sticks out as kind of a random position on orbit, and the down firing thrusters, part of their plume hits it.
They had evaluated that effect for the primary jets because they knew in the direction they pointed and their lower expansion ratio in the Vernier jets there was going to be a problem.
And that was properly dealt with in the flight control system.
They never modeled for the Vernier jets, but only discovered that the state estimator was not converging as well as expected on the first Shuttle flight when using the Vernier jets.
And it turned out there was probably a 20% or 30% net reduction in thrust and a 15 or 20 degree effective change in the direction of the thrusts, those two down firing Verniers.
And the feed forward estimation from the RCS jets, as a result, was giving the wrong data in the state estimator causing a jump in the value and a long time constant to settle out.
And that was causing a few percent increase in the propellant consumption but probably a factor of ten increase in the duty cycles of the jets which was unacceptable from the life of the jets.
That was compounded by a problem in STS-3 which is the first time they used the arm.
And they actually put a few hundred pound payload and tried controlling the attitude of the vehicle in moving that payload.
The combination of those things they went back and, by STS-5, had to make serious changes to the phase plane estimation logic and tables for the accelerations for those Vernier jets because they didn't want to burn those jets.
I think, after STS-2, they actually changed some of the jets.
And then they didn't want to have to change them again.
The maneuver mode.
You want to do various types of [NOISE OBSCURES] maneuvers with respect to various frames of reference which could be landmark tracking, local vertical tracking, second spacecraft if you're doing a rendezvous tracking.
Guidance to provide that.
That would be broken down into components by axis.
And then there was an additional level of hysteresis about what you were telling the face plane to do.
And I talked about adjusting the origin.
This would decide whether or not you would adjust the origin.
If you're error in a given axis with respect to what you wanted to do with the maneuver was less than twice the dead band, you would not adjust the origin.
And if it was more than twice you would.
And, again, it is a case of why force it to do things it doesn't have to do if it's going to get there eventually anyway?
And that is actually an adjustable parameter which, for certain payload missions, they might change that value from something else.
There were these various direct manual translation rotation modes and the auto modes that I think bit by bit I have talked about.
The reason you have these different references, inertial and local vertical, if you're doing an earth observation mission, almost everything is going to be in a local vertical frame.
You're going to have your payload bay pointing down.
You want to keep it pointed at a certain spot.
If you're doing a solar telescope mission, everything is going to be in an inertial frame.
You're going to want to keep it pointing at the sun or within a few degrees where maybe the telescope will have a limited travel of its own.
If you're doing rendezvous then it's going to have to be you're accounting for the orb rates of the two vehicles.
You're going to want to keep your rendezvous radar pointed in the right direction.
By the way, the rendezvous radar, there is a deployable dish antenna that goes out of the payload bay, the side of the payload bay of the Shuttle.
That is a dual-purpose antenna.
It can be used to point at the TDRS satellite to send data at high rates to the earth and track the satellite.
Or it can be used to point a radar beam at another spacecraft.
It was not used for TDRS in the beginning of the program.
It was only used for radar because the first TDRS was launched by STS-6.
Electronic string, and I think we're about at the right point of time on this, too, to allowing me to get into this in a little bit of detail.
This was something that kind of crept up on us in importance.
It was thought it would make the systems redundant, have separate strings, don't put them all on the same computer.
All those things were recognized, but I don't think until about a year, a year and half before the first flight actually occurred that we realized how difficult it is to be sure that all the interactions of these strings, the power string, the pluming string and the electronic string and the possible combinations of failures, how difficult it is to assure that you meet those high level failure tolerance requirements.
And that is what I think I did in 1980, '81, probably was the early form of fault tree analysis.
And I have in a handbook I developed for the first flight, a series what if tables with all these different breakdowns of what could happen.
And actually a person that didn't know about probabilities and all looked at it.
It was a pretty scary table when you looked at all the possible bizarre two failure combinations.
But first you've got to understand what the strings are.
You would have one forward, one aft left, one aft right, so three boxes, three half boxes and multiplexers-demultiplexers on a string.
And there were two MDM boxes in each pod, forward left right aft.
And the electronics were broken apart into each half box.
So you actually had, in effect, four sets of electronics in each pod, even though you only had two boxes.
And one card from each of those boxes would be dedicated to string, but each string would have a card per pod.
So three cards would be tied electronically to one computer anomaly.
Now, you couldn't take a string that could be commanded by more than one computer.
The computer saw the data from all four strings so they could vote the data to decide whether or not each of the computers was healthy, but they could only command one string.
I should say each string could only be commanded by one computer, but you could, if you started losing computers, latch up more than one string to one computer.
Even though one string could only take commands from one computer, more than one string could take commands from the same computer if that became necessary.
If you lost a computer and were in a benign environment and you wanted to recover all those systems, that happened with the first computer failure in STS-9, then you would manually re-latch those strings to a healthy computer.
Of course, if that computer went down then it would take two strings down.
There was a unique latch up of these string components to power systems, but it wasn't necessarily one-to-one.
And that is where things got very complicated, when you start crossing the subsystems where one power system could cause some subsystems to fail on more than one string.
Then you take a string down.
That would be a very different failure scenario than taking two strings down.
This is showing how the thrusters laid out.
You can see there are a lot of thrusters on each string.
If you go back and look at the vehicle carefully and you look at the down firing thrusters, you will see that in the front of the tank there are two thrusters that point down to the left and two thrusters that point down to the right candid out about 40 degrees.
That means that if you lose two thrusters, two strings or two manifolds it was possible to lose both of them on one side.
That is very important because that's one of the phenomena we had to protect against for separating from the external tank.
And it becomes a highly coupled separation maneuver.
And there were some special jet select tables created for that and some special control logic loops that were created just for that one scenario.
You go through these things and certain critical events where things cannot fail even if your subsystems have failed twice.
And the subsystems may not have been designed to make it very convenient, but you still have to do it.
That happens to be one of the scenarios we looked at a lot.
And that we see forward down left down left or forward down right down right.
So somehow you lose the thrusters, these strings or whatever.
That became a special design case.
And there were a lot of those kinds of things.
I would say 70% or 80% of our design time goes into learning those special cases.
With the thrusters the manifolds are basically the valve lines, so there is a commonality between the electronic stringing of manifolds in those valves.
With the OMS that's not really true.
The way the OMS engines work, each engine [NOISE OBSCURES] is actually an oxidizer and fuel tank going into two loops of lines each with two valves.
At least one valve in each path had to open to feed fuel.
At least both valves in one loop had to close to stop fuel from flowing.
Any combination which would leave two open after a burn started or would leave two closed before the burn started was a problem.
Then you have pressure sensors.
And there was only one per engine.
You lacked insight into whether or not the engine started based on pressure if one of those failed.
The stringing of this was not the same as the electronic string, so we had to start looking at the possibility of individual valve failures and electronic string failures.
Taking down other valves.
Did that happen before an engine started, after an engine started?
How did that correlate with then influence on the same engine's actuator control?
We could get situations where we could start the engine but we couldn't steer it, engines that we could steer but couldn't start it, and so we ended up with this maze of tables looking at those situations coming up with contingency designs.
Sometimes reverting to using the XRCS jets under severe multiple failure scenarios that actually do deorbit burns.
We talked about what could go down.
All of these things could be lost with a single failure, though not all strings have an IMU because there are only three.
And only two of the chamber pressure.
Some strings were more important than others with respect to a subset of the systems.
And since there are only two thrusters on one manifold per pod of the Verniers, there are also only three strings that affected those.
All this got set up so that any one failure was quite manageable, but the moment you talked about any possible two failures was when it became much more complicated.
Generally, where we ended up, after lots and lots of special design accommodation was we tolerated loss of translation control in the axes that were not necessary for managing deorbit.
We tolerated degradation and rotation control but assured that we retained it, at least in a time average sense, in all three axes.
The biggest complexity here was assuring we still had adequate time averaged translation and rotation control for tank separation.
So we would pull away without recontacting the tank.
Remember, recontacting the tank meant tiles hitting the tank.
Those tiles are really fragile.
They can survive thousands of degrees but not much impact.
We accommodated the scenario where we could lose all thrust vector control on the OMS engines and still do a deorbit burn usually with RCS wraparound properly designed.
And we had to accommodate the situations where MDMs did not reset, so everything tied to an MDM would be lost.
And that is the kind of situation where you start having valve here, a valve here, a valve there being shut down along with a bunch of jets.
And the real pain was understanding what things could collectively go down together.
You had to lay out all the faults, all the things that could happen with each of the faults and then overlaying the combinations of those.
And then you would find it wasn't everything you were worried about.
It was usually a small fraction, but those few fractions drove the design details and effort to a great degree.
Now, on orbit, because you would often being using one IMU and you only had two computers, one of those strings going down meant you lost your navigation base.
But because you're in a relatively benign environment, restringing and bringing up another system wasn't a problem.
If you were doing terminal rendezvous with the Space Station, you would not be in a one IMU mode.
You would be in a three IMU mode on two computers, but you would not lose your navigation base.
If you lost a computer at a critical point in that, you would probably abort the rendezvous, reconfigure again and resume your operations.
One of the ground rules for all rendezvous operations with the Shuttle, and I think will be true of the CVE is an rendezvous operation planned to do has to be able to be repeated at least once.
The way I want to end this, and then we could have questions, is two pictures, before and after cockpit upgrade on the Shuttle.
This was the way the Shuttle was for many flights.
Those are CRT displays, green monochrome where no images could be drawn except little sticks, dots and lines and rudimentary.
These are analog tape measures, analog eight ball with analog needles for attitude and rate information.
And really very 1970s.
Comment on that.
One of the things that we did as payload specialists is we got some training in the cockpit, and that was the version that was available in the early `90s when I was down there.
And you would go from your desk at the Johnson Space Center when you were using an IMB Think Pad or something, and you felt as though you had gone through a time warp when you went back to the simulator on what was the world's most expensive vehicle.
It was extraordinary.
And, of course, you could explain why this was kept on so long.
Well, it was this multi seven digit figure for its upgrade, as well as the issue about recertification.
And let me mention both of those in a moment, point out a couple of things and get back to that before I show you the current configuration.
[AUDIENCE QUESTION] The amount in the Shuttle versus the amount to support the Shuttle?
In the Shuttle because [NOISE OBSCURES].
Well, you only have a 104k memory computer.
You have the actual number of lines that are [NOISE OBSCURES].
Yeah, you had the 104k for the backup system, the 104k for ascent and entry, the 104k for on-orbit, the 104k for system management.
And then there have been other things where their own software capabilities have been added in subsequent years.
So it is a lot of software now, but human validation means if you make small change everything has to be reassessed for possible interaction to a degree that you would never do for a mission that doesn't put human safety at risk.
That means the cost of doing that each time is probably an order of magnitude higher than it would be for an unmanned system.
You look at what it was going to take to put a cockpit upgrade in there.
We're talking hundreds of millions of dollars to do it for the fleet.
And when I say hundreds of millions, a couple hundred million.
It did eventually get done, as I will show you in the next picture, but the main reason was obsolescence rather than wanting to be contemporary.
You cannot buy these pieces to replace them anymore.
The companies that actually made some of these systems may not exist.
Or, if they do, they have no economic incentive for maintaining the base for a customer that may buy another five of them.
And so, one of the things you have to deal with when a system is going to do this many years, another example of that is the B-52 or the Triton missiles or things that stay in operation for a long time, is you have to plan, as part of your operation cost, for periodic upgrades to mitigate obsolescence.
[AUDIENCE QUESTION] In some program they had some support.
It was Digital microcomputers.
We had a room over in the Hill building that had about 100 of them stacked up as backups.
I don't think the boxes ever got opened.
I think the program did the buy, but the program never lasted as long as it was supposed to.
But you are right.
You can try to buy it.
But, even so, even unused systems had a shelf life.
And you don't know how good they are going to be 30 years later.
If you had asked me in 1978 how long the Shuttle was going to fly before it was replaced, I would have said ten years, maybe 15.
Before the Columbia accident they were talking about another 20.
It is only the safety issues now that are going to make them stop it soon.
Anyway, the other thing I want to point out here, this is the crew interaction mechanism, a push button display.
No such thing as GUI.
That is unheard of in the Shuttle.
Can you talk a bit about the hand controllers?
Are they the same ones used for landings that are also used for docking?
I mean was it the same hand controller for everything?
It is the same hand controller for everything, except there is another station in the back, I didn't bring a picture of it, which is used for docking.
The cockpit, you're looking out the forward windows, you turn around and there is another set of cockpit instrumentation, hand controllers.
And there are two windows above you and two windows looking on the payload bay.
And there are also controllers there for the arm.
Everything related to the arm is done looking out toward the payload bay.
The arm is attached there.
When you are doing rendezvous you're looking out at the overhead windows of the vehicle you are approaching.
So, in effect, you have a couple of these CRT displays and identical hand controller and the equivalent then or the updated equivalent now of these displays on a back station.
To avoid any issues of changing controllers and changing viewpoints, you might note that normally the controls that are done out of the back window with that separate controller and separate displays are done by different crew members.
That is typically a mission specialist's job, not a pilot or commander job, and so they are separately trained.
Right.
That's an important point.
For rendezvous and docking, it is the pilot or commander who uses the aft controls to control [NOISE OBSCURES].
Do you know?
I believe that everything related to the RMS that you said is correct.
But for a rendezvous the commander is always involved.
But the point was that in that case that is done out the rear window.
It is still done at the rear window, so he has got to mentally reverse his frame of reference while he is doing that.
That is correct.
[NOISE OBSCURES] And that is something that I think will not be done with the CEV.
The same frame of reference will be used on the CEV because I think that has always been a little bit of a point of contention in the problems of maintaining simultaneous proficiency.
For remote manipulation with the arm that has been an issue.
Yeah.
And I think that has always been done by uniquely trained mission specialists.
That's right.
Today's cockpit looks a lot more like what you will see on a commercial airliner vintage mid `80s, the late `80s perhaps, but still more familiar where you've got glass multicolored displays.
The eight ball still exists in image form because that is what a lot of the astronauts used to learn how to fly these vehicles.
They still fly with respect to it, except it is a digital representation, but you're still not with the gooey environment.
You still won't have touch pads or a mouse.
It is all pushbutton displays.
And there is a case to be made that touch displays or touch screens don't work very well in a vehicle where you G environments keep changing.
It is very difficult to get a touch display that works with a light touch and a heavy touch when you're in zero G or when you're pulling several Gs.
Even if the technology was space qualified, it is not clear that they would use it, except on a vehicle that stays in a constant environment.
At this point, let me introduce Dr. Hayashi.
Miwa Hayashi is, in fact, at NASA Ames Research Center now, was here at MIT and worked on the design of the next generation of the Shuttle cockpit upgrade.
In fact, that is what you will be talking about in ten minutes across the hall in the 16.400 class.
It is essentially what would have been done if Shuttle life had been extended.
I might add that, although the room is very crowded, if a few of you would like to sort of hear where this story would have gone, you're welcome to come across the hall to 419 and hear Dr. Hayashi.
And you are also giving a lab meeting lecture at 1:00.
Could you tell us that subject?
That is about the astronaut scanning behavior.
Our team had a model about the astronaut's scanning behavior in the cockpit, this upgraded cockpit.
Anyone interested in this kind of topic you are welcome.
It is from 1:00 PM 33206.
Thank you.
Go ahead, Phil.
Well, at this point, I think I'm open to a few more questions.
We have a couple of minutes for further questions, any topic we've covered or that's related that we didn't cover.
Yeah?
You talked a lot about the constraints of the invented computers.
[AUDIENCE QUESTION] Well, we had very large facilities for that purpose that evolved over time.
The first major facility was the Flight Simulation Laboratory in Downing, California, which was then Rockwell.
You had a room with a couple cockpits, you had another room which had the digital interfaces of the cockpit, and a half a floor with the analog computers that provided a lot of the information generation at a rate that was not achievable with digital systems.
You had this hybrid system for driving what were man-in-the-loop simulations.
And we spent 24 hours a day, 7 days a week using those labs for several years.
I would often go out to California and be on the 5:00 PM to 5:00 AM shift often spanning weekends.
By the way, I was doing that, I think, when I was still a graduate student, which was kind of an interesting experience.
But then NASA built the Shuttle Avionics Integration Laboratory in Houston which eventually superseded the laboratory at Rockwell.
That became an all-digital system.
The hybrid systems went away.
The high capacity computers, they could fit in one good size room everything they needed, and were able to get much more digital displays for the crews.
Sometimes what they used to do for the imagery of the crew in the early days is they would actually drive a camera across a simulated scene because you couldn't generate the scene.
By the time they got the sail facility developed they were able to do scene generation with fairly high computers.
Now everything could almost be tabletop.
I mean it is just so dramatic as how this evolved over the years.
The one thing they had at Rockwell that never got replaced, though, is they also were able to put into the loop actual hydraulic systems.
When they were doing entry they could turn on aero surfaces and hydraulics with simulated loads.
And you always knew when they were doing it because the high pitch scream of the APUs, when they were doing it, you could hear two blocks away.
OK. Last question.
Sir, I was wondering if you could give kind of a concept of the cost of developing the software either in man hours or in comparison of what they spent on the hardware.
Well, the initial development of the system probably involved the equivalent of 15 or 20 full time people for a few years.
And this was to develop the algorithms and support the validation.
The actual flight software was actually produced by IBM separately.
And they had a small team of people that would take the detail design specifications and create the software.
It is a big effort but a very small part of the cost of developing a Shuttle.
I mean you measure the Shuttle in billions.
You measure the flight software development in millions.
One thing that I remember when the software was developed and we were working and validating it, IBM wanted every line of code change in million dollars.
A million dollars for one line of code change because they had to verify the whole software.
That is why you would never, unless there was a flight critical thing, do that.
NASA wanted generally, whenever possible, to aggregate changes for a year or maybe two years sometimes.
And then put them in there just for that reason because there was this huge cost of revalidating.
And that was the IBM cost.
It would be a cost to bringing us back in to certify that, too.
Well, it worked and you should be very proud of it, you and your colleagues.
Thank you very much, Phil.
Thank you.
[APPLAUSE]
Welcome everybody.
We are extremely fortunate and proud to have with us today Christopher Kraft.
Actually, Christopher Kraft, Jr.
He told us last night that his father, Christopher Kraft, was born within about a block of Columbus Circle and that is how he picked up the name Christopher Columbus Kraft, which was passed onto Chris Kraft, Jr.
I don't really need to say very much by means of introduction because Chris Kraft is a name that has been associated with America's Space Program since the very beginning.
And, actually, from the very beginning of his career, right after he graduated from a university in Virginia, he went to work for the old NACA, the National Advisory Commission for Aeronautics.
01:00 And eventually the director of the Johnson Space Center which is the home of Human Space Flight in this country.
He had that position through 1982, which was the end of the original orbital flight test phase of the orbiter.
So, he really was in charge of the Space Center when the Space Shuttle was being developed.
And, of course, this course is of course in the systems engineering of the Space Shuttle.
I just want, in public here, to acknowledge that there have been expenses involved in bringing all of the special lecturers that we have had who have participated in this course.
And would not have been possible without the support of the Draper Laboratory.
And we thank Dr. Eli Gai who has provided that support.
We couldn't have done it without you, and we really appreciate it.
That is enough for me.
We all came here to hear Chris Kraft talk about the invention and development of Mission Control and the systems engineering and development of the Space Shuttle.
I will tell you that Chris Kraft is somebody who is not afraid to express his opinions.
And we are looking forward to hearing them.
Chris.
Good morning.
It is not hard but sometimes difficult to return to MIT where I have a lot of friends who came this morning.
They sort of overwhelmed me then, and I am sure they would overwhelm me now if we got into some deep technical subject about which I knew very little at the time.
And I will say a little bit more about that.
What I did want to say, though, is that the people that preceded me in lecturing to you, you've been very fortunate.
Because they indeed are the stars of the Space Shuttle.
They did a fantastic job.
And I hope you got that sense from them as they spoke to you.
They are the best.
And if I were going to say two things about management that I have learned in my lifetime, the first would be you are absolutely no better than the people around you.
Without a lot of great brains around you you're not very good, no matter if you're the best person in the world.
And too many people have learned that the hard way and not recognized that fact.
The second thing we talked about last night around dinner, and that was the second thing you learn as an engineering manager is that every day is a compromise.
Everything you do you have this idealistic view of doing it the best way possible, doing it better every day, doing it without worrying too much about the cost, too much about the budget, too much about schedule.
You go in with that idea.
But those things you have to face every day, and so managers become great compromisers.
If the systems that you end up with are not what you really wanted, but if you're smart they do the job.
If I were going to add two things to your education at MIT, that is where I would come from.
The third thing I would say is that, whether you like it or not, you people sitting here, no you old heads, but you people sitting here are the people that are going to do the next Space Program.
You are the ones that are going to take us back to the Moon, if and when we get there.
It is going to be up to you to do the job.
In 1968, the average age of my organization, and I think I was 44, was 26.
We had an awful lot of young people who did the job and did it extremely well.
The guy sitting there on Apollo 11 screaming into his headset that it was still go I think was 25 years old at the time, and he was a veteran.
And if he hadn't been a veteran, he sure in hell was a few minutes after he kept yelling into that microphone that it was go.
And I hope you've seen that on television.
If you haven't it's a really great moment in Apollo.
Let me start from the beginning.
In Project Mercury, we started with a space task group of 35 people, eight of which were secretaries.
And those of us that came out of the NACA, the National Advisory Committee for Aeronautics, were smart guys.
We were very capable people, but we didn't know a damn thing about how to fly in space, believe me.
If you would have asked us at the time how do you get fluid out of a tank at zero G, I don't believe you would get the right answer from more than two guys.
Maybe Max Faget would have said well, you've got to put a bladder in there and put pressure behind it and squeeze the stuff out because it is going to be floating.
You don't know where it is in the tank.
And, secondly, how much have you got left in a tank?
Kind of an interesting project if you don't know what zero G is all about.
When we started in Project Mercury we didn't know much about systems design for space.
We certainly had high questions about man's capability to perform a task in space.
And I would say 98% to 99% of the medical community in the United States thought that the astronaut, when he got there, would be a blithering idiot, that he would probably swallow his tongue, that he couldn't see because his eyeballs were bulging out or that because of the worry he was going through he would have a 24 hour ulcer sitting on the pad.
And they would suddenly have to be at his side, the medical community thought.
That is where we were coming from, so we decided we're going to put man in space.
It was a daunting task but one which most of us realized from the get-go that we were in the middle of probably man's greatest adventure.
Believe me.
We did know that.
I felt it and I think everybody felt it.
It was sort of a euphoria.
And we were in what you might call engineering euphoria like Ed White was on Gemini 4.
When he was outside the spacecraft, I am absolutely certain he was euphoric.
The press said he must have been euphoric.
And I said oh, no, he was worried about doing the right things and doing all the right things at the right time.
He was euphoric.
And you don't recall but they said something finally at the end of that.
They said well, what does the flight director have to say?
And I said get him back in the spacecraft as loud as I could say it.
I think that is only one of the few times I have ever spoken on the air to ground.
We were faced with putting somebody into this new environment for the first time.
And how do you do that?
What are the problems you are faced with?
When we began to think about how we did flight tests on airplanes, you would sit it on the ground and you would write a flight test requirement, a set of things you wanted them to do on a flight.
You would sit on the ground, hold a microphone and talk to him.
And he would say I just did so and so.
And, if he was one of the best test pilots, he probably didn't tell you damn thing.
They just kept quiet like Neil Armstrong did most of the time.
He just kept quiet.
And you kept having to prompt them to tell you what they were doing.
That is where we were coming from.
We had instrumentation.
We had been developing telemetry from the bomb drop tests that we had made at Langley.
So, we knew quite a bit about telemetry.
We knew very little about air to ground.
We knew that we would like to talk this guy about 15 minutes or so.
That is what you did when you went across the country as an airplane pilot.
If you're going around the world, we would like to talk to them about every 15 minutes.
And so we got out the geography books and said well, we're going to fly around this thing and here is what this Chinese finger puzzle looks like as it goes around the Earth.
And we are going to be over this part of the Earth and this is where insertion is going to take place and this is where we're going to do orbit determination and this is where we're going to do retrofire.
And we are going to be up there going around this particular section of the Earth.
And you looked at the geography and said well, we've got a tracking range in Cape Canaveral.
We've got tracking range on the West Coast.
We have a few radars in Australia.
But, if we're going to speak to them every 15 minutes, this is where we would like to be.
And so we end up saying well, there are the Canary Islands, there is Kano, there is Zanzibar, there is Muchea and Australia and so on.
And immediately said well, if we're going to have to talk to them, we're going to have reception there so we're going to have to build a station at each one of those locations.
And then we're going to have to tie them all together.
And, Lord have mercy, here we are with a whole requirement to build a worldwide network and nobody to do it with, 35 people to do it with.
And so we immediately got a group together.
And we got Western Electric and Bell Labs and a bunch of people like that and started building the worldwide network.
That was a heck of a project to do at that point in time.
And just the diplomatic requirements in all the states that we had to deal with around the world was a project in itself.
Having done that we just said how many times around the Earth do you think we would like to go or need to go on the first flight?
And what do you think would determine that?
Well, in 1959, if you put a spacecraft up from Cape Canaveral or from Vandenberg Air Force Base and you asked is it in orbit, of the then flight director, he would say I don't know.
I will tell you when it comes up over Kodiak, Alaska 45 minutes from now.
And I am saying to myself well, if this thing isn't in orbit and I want to bring it down in the water before it hits the coast of Africa, I have got to know when to turn that spacecraft around and fire the retrorockets.
I have to know immediately, or at least within two or three minutes to turn the spacecraft around and fire the retrorockets, what the orbit is.
Because, if I don't, I don't know where it is coming down and I don't know where to send the ships to pick that young astronaut up.
Realize that in 1959 nobody knew what a short arc solution was from a C-ban radar in 30 seconds of data.
Furthermore, they didn't have a computer to do it with.
We were slide rule people, Marchant computers, crank computers.
And you were suddenly faced with the fact that you've got to build a computer system to take radar data from Cape Canaveral and Bermuda, massage that data and within 30 seconds of the short arc solution tell the people that have got to turn that spacecraft around and fire the retrorockets in two minutes.
Today that sounds unbelievable.
When you talk about air to ground communications or ground to ground communications, in Africa the best you had was 20 words of teletype per minute.
Now, you've got to know what is going on in the spacecraft or what the astronaut said as he flew over Kano, Nigeria.
You were going to get it back in 20 words of teletype.
How do you make real-time decisions under those circumstances?
What is a real-time decision?
Where are you going to make a decision?
Do you need some central facility which invented Mission Control?
And suddenly then, if we're going to do this job and we're going to have people looking at this data, we have got to train a group of people to go to all these locations around the world.
And if you're going to make decisions in a central location then you've got to have some means of getting that data back to them, of massaging that data, letting people know outside the limits of that control facility what is going on so they can interrelate with each other.
Nobody had ever done that before.
And the first time we cranked ourselves up in a bunch of small cubby holes in an old wind tunnel building in Langley Field, Virginia and started doing what we would call the initial simulations, we found that we didn't even know how to talk with each other.
And everybody was talking at once.
And so we had to invent a whole new language and had to have negative reporting and things like that which people had never heard of before.
And rapidly then we began to realize that we had a big task in front of us.
If you're going to recover this gentleman at the end of the flight, that is not too hard.
We can send a few ships out there.
And we probably ought to have a helicopter there to pick them up.
And maybe we could have one of these light carriers.
But if the doctors are right, we might have to come down anywhere in the 360 degrees on those three revolutions.
Now, who are we talking to?
We're talking about talking to the search and rescue people.
We're talking to destroyer captains.
We're talking to people that have got to fish this thing out of the sea.
And how do they do that and how do they not get burned with the fuel that might be running out of the spacecraft.
And suddenly we have to train probably 10,000 people on how to recover this machine.
If the spacecraft is sitting in the water, we've got to train several hundred frogmen how to jump out of an airplane with tools to get to the astronaut.
It was a tremendous task for a group of people who had never done much but do wind tunnel tests or flight tests out of Langley Field, Virginia.
The early days we had to come up with orbit determination.
How to look at the astronaut's health.
How do we get something down that we can look at?
Can we get an EKG down?
Can we get his breath rate down to each one of these stations so we know what the man's health is?
An interesting story.
We did eventually build a simulator to train the astronauts, and we had no way of getting data to each one of these sites around the world that would allow us to run a full-fledged worldwide simulation in real-time.
We would put it on tape.
Cut it up in sections.
Send out a script of what the astronaut was going to say and do.
And play this six or eight minutes of a tape as that's what they would see as the spacecraft appeared over their station.
And when we said we've got to train a bunch of doctors, an interesting story was we went down to the Veteran's Administration in Houston and said we'd like to put some instruments that we were developing on people that are sick here.
And so, as they come in, various types of diseases.
And, fortunately, one day we had a guy instrumented and he had a heart attack.
We were able to record all of these things that were going on in this gentleman, his temperature, his EKG, his breath rate, what his blood pressure was and so on, and put all that on the tape.
And we sent that out to the remote sites.
And then at each of the sites, as this occurred, had the doctors diagnose what was wrong with the astronaut.
I don't believe in any one of the 17 stations we had anybody diagnosed it as a heart attack.
They all said he had an appendicitis or he was having some kind of shock take place to him because he was frightened to death.
Anything but a heart attack.
So, that was sort of classical of the things we did and improvised in order to get ourselves capable of running a worldwide operation which allowed us to make decisions in real-time.
Now, the other thing that we invented at that time, I say invented, it just came about by evolution, was a book called Mission Rules.
And that was probably the smartest thing we ever did.
As we began to look at the spacecraft systems, we started asking questions.
If this system is failing, what are the measurements that we're going to have there?
And, if it is failing and it isn't operating at the right temperature or the right pressure and it is off nominal, what will the system do?
And how do we measure that on the ground?
How do we detect it?
Where is the instrument located on the system because it might be effected by the position it is located in the spacecraft.
It might be hot.
It might be cold.
It might be suffering different kinds of pressures than it was measured on the ground.
And, as we began to ask those questions of the system engineer, the "system" engineer.
Not "systems" engineers because I don't think we had any at the time.
And they would say why the hell do you want to know that?
The system is either working or it ain't working.
And we said yes, that's a good answer except that now we've got this system in space.
And if we want to continue this flight and not have a contingency operation, we would like to know how long the system is going to last if it isn't operating under normal conditions.
That prompted us then to start thinking about how the system failed and what we were going to do about it.
If the thermal system that kept the astronaut from getting hot or getting too cold wasn't functioning properly, what could we do about it?
How long could he stand being at a temperature of 85 degrees inside his space suit?
And then that said well, if it stays there and we can only go X number of minutes, what are we going to do about it?
What is the rule of the game that says we should re-enter or not re-enter or go to the next primary recovery area, et cetera?
And it allowed us then to write down, for every system, and the man what we would do under certain circumstances.
Called those a set of mission rules.
And that prompted us to develop a bunch of malfunction criteria.
What malfunction procedures are you going to go through?
And then that prompted us to ask the contractor and the manufacturer of the systems.
And that developed a whole new set of schematics that they hadn't been used to.
It cost us a lot of money to do that, and they didn't want to do it.
They didn't know why we wanted to do it.
But as they began to see the mission rules -- We got those out in front of them and said we're going to do this with your spacecraft and your system.
Then they began to realize they better start thinking about those things.
And that is what brought, in my mind, a group of people together in "systems" engineering because you began to find out how the systems reacted with each other.
And that was a question that most engineers didn't think about.
If the thermal control system is not functioning properly, what does it do to the reaction control system?
Or, as we had on one of our first orbital flights, the seats on the small thrusters they were using for attitude control were not seating properly.
And the experts said it is freezing.
It is getting slush in the system and is causing the valves to stay open and they're not getting the proper fluid to it.
So we put a thermostat on the next spacecraft and it wasn't freezing, it was getting hot because the feedback from the thruster was getting on the lines and causing the seats to warp.
And it was sitting there dribbling out and causing the attitude to be sloppy and jump around.
And, as a matter of fact, on one flight we had to reenter early with the first chimpanzee flight because the machine was running out of propellant.
It began to have everybody start thinking about how does my system fit with everybody else's system?
How does that fit with the game plan that we're trying to come up with?
And, at the same time, the organization then was able to look at all of these things that we said we were going to do and became a heck of a management tool.
I remember James Webb used to come down, the Administrator of NASA at the time, and I would show him, in the Control Center, how we ran an operation and how we made decisions.
And he was absolutely livid about that because he said that's what I want in Washington.
I want to be able to have those kinds of things put in front of me so I've got all these things so I can make a decision.
I need you in Washington.
I want you to come up here and tell me how to build a system like that to do management.
I must say that I have never been able to do that, but he was very emphatic about wanting to do that.
Jump to the conclusion of Mercury.
I think we learned an awful lot form that program.
We learned that man could do a job.
He could do it just as well at zero gravity.
Particularly, in Mercury where he couldn't move around and he didn't get sick, fortunately, as he did eventually in some of our spacecraft.
But certainly man could do the job in space as well up there as he could in a fighter airplane on the Earth.
But it was child's play.
Mercury was child's play.
We put it up there, we fired the retrorockets and it landed, and then we picked them up.
A hell of a job at that time, but it was child's play.
And so Mr. Kennedy, in his great wisdom, in April of 1961, when he saw the reaction of it, we were all down in Sheppard's first flight, asked NASA what can we do to ace the Russians?
And NASA, in its great wisdom said well, probably in about ten years we can go around the Moon.
George Low and others at Washington had been doing some work on a lunar spacecraft.
And in the great wisdom of whoever made such a decision, the president asked why can't you land on the Moon?
Now, I want you to know that that was 1961 and Chris Kraft did not know how to determine orbital mechanics from 30 seconds of radar at Cape Canaveral.
And this man, in 1961, says we're going to the Moon in this decade.
And I thought he was a little daft.
I must say, I thought he was a little daft.
The second day I thought a little bit better of it.
And then about three months later, when he came to make that famous speech in Rice Stadium, I was called back from Cape Canaveral to tell him how we were going to go to the Moon.
And, I am telling you, I did not know a damn thing about how to go to the Moon.
If you had said free return trajectory to me, god, I'd a thought it was a pass to the Astro's baseball game.
But here I was faced with the fact I've got to stand up in front of the President of the United States in a room, much like this one, only with about ten or 12 people in it and tell that gentleman how you're going to go to the Moon.
And that was a quick learn, I'm telling, a really quick learn from people like these guys, John Mayer and Bill Tindall taught me in a few hours how to do the orbital mechanics to go to the Moon.
Not how to do it but what took place.
Here we were at the end of Mercury.
And we are going to then have to go to the Moon.
And how are we going to get there?
And how are we going to train ourselves?
What are the systems we need to do the job?
What new control center do we need?
What kind of operation do we need to think about?
What kind of trajectory analysis do we need?
And what kind of computers do we need?
And what kind of communications do we need?
Suddenly, we've got a whole new set of problems.
If we're going to do rendezvous at the Moon, we've got to teach ourselves how to do rendezvous at the Earth.
If we're going to send something around the Moon, we better have a heck of a system to determine whether we are truly aiming at the Moon or whether we're going to hit the Moon.
And, in fact, on Apollo 8, I wasn't sure that George Miller, who was the head of Manned Space Flight, was sure we weren't going to hit the Moon when we told him that we wanted to do the trajectory as we were going to do it when we landed.
Then it ended up being 60 miles above the lunar surface as you entered orbit around the Moon.
And can you really tell me 270,000 miles away whether the spacecraft is going to hit the Moon when you are ten hours away or is it going to go around the Moon?
So, we were faced with all those new problems.
That is what got us to the Gemini program.
We wanted to be able to build a spaceship that would allow us to do maneuvering in orbit.
That would allow us to stay up there 14 days, which is how long the spacecraft flight to the Moon and back would be, that would allow us to do reentry guidance using the L/D of a blunt body, enough to skip it out as you came back to Earth and then go back up and then reenter at a much lower velocity so you wouldn't burn up the spacecraft.
Those are the kind of things we were suddenly thinking about as we built the Gemini spacecraft.
We needed an onboard computer.
Unheard of in that time period.
The Air Force had been putting some on airplanes but never had we had on onboard a spacecraft.
So Gemini was designed to be a maneuvering capability in space to rendezvous and dock with a target, to determine the capability of man to survive for 14 days, to do a heat shield which was much more flexible and reusable.
And to build a maneuvering system.
And, finally, to do guidance and control for landing point and control and develop a footprint on the Earth for Gemini which is what we were going to have to do on Apollo.
Gemini was a very successful program.
Without it we could never have gone to the Moon.
We learned how to operate in space, how to maneuver in space.
We learned how to do EVA, which was a total disaster as we flew in Gemini.
I don't think we even, by doing it five or six times on the final flight of Gemini 12, Buzz Aldrin was able to do a reasonable job in extra vehicular activity.
We had to build a suit that was flexible to be able to walk on the Moon.
We had to build a backpack which was, in truth, another spacecraft to do Apollo.
We had to build a new control center because we had a computer which we actually doubled the storage capacity on Mercury and then gave it 64,000 words.
Today, you have that in some kid's thing that he carries on his airplane and one touch of his stroke.
But 64,000 words was all that we had then.
When we flew Gemini, we had a million words.
When we flew Apollo, we had 5.5 million words, so that computer complex was changing on us continuously.
When we did Mercury, we used a grease pencil to write down the numbers as they came back from Kano, Nigeria and 20 words of teletype.
We had to build a new display system, a digital display system with a computer.
And the first digital display system was not graphics at all.
What we did was build a slide that was the background for the display that you wanted.
We had a set of 4.5 inch lantern slides, a bank of 100 for each station in the Control Center, and then the computer filled in the numbers.
Now, that was in 1964, '65 and '66.
It wasn't until we got to the latter stages of Apollo that we had computer graphics.
I don't know whether you can realize that or not.
Computer graphics today is, golly, you have football games on computer graphics.
But then we didn't have it so we had to redesign a control center and continuously redesign a control center.
We had to have a computer controlled communication system.
All of those things were built.
We had to utilize and build in NASA the first communication satellite from which came the revolution in the world, in my opinion.
I want to go through Apollo and things like translunar trajectories and free return trajectories and what might happen if you were off by a few feet per second or a few tenths of a degree when you fired the orbital maneuvering system of Apollo on the backside of the Moon, which is where you had to do it for optimum performance characteristics.
When the thing showed up as it came out on view on the front side of the Moon, and it was not in the right trajectory, what the hell are you going to do about it?
Where is it going?
Is it going around the sun?
Is it coming back to Earth?
Is it going to hit the Moon?
And what am I going to do about it if it is on one of those paths and my maneuvering engine, which has 10,000 pounds of thrust, is not working or not working properly or it wasn't pointed in the right direction?
You had to be prepared to think about those problems and make a real-time decision as to what to do.
I hope I am impressing you with that because that is what you guys are faced with in going back to the Moon.
It isn't just a simple problem of orbital mechanics.
It is a problem of what are you going to do if it isn't correct, if it isn't on the right path?
If the system isn't working properly can you land?
Those things have to be thought out and thought out carefully before the fact, not in real-time.
You can make all of the decisions in the compute which you would have made after you had thought about it in real-time, but think of the orbital mechanics problems associated with that in real-time and the background then of the math and the thought processes that have to go into making those decisions.
You're descending to the Moon.
And as one of these gentlemen sitting here, you start to do the descent to the Moon, and, low and behold, you bring up the system on the LEM and the abort light is on.
What the hell does that mean?
Well, it means if you start the engine right now it is going to start doing a rendezvous back with the command module.
It is not going to land on the lunar surface.
And I've got a computer program that is hardwired to do that job with.
I don't have the capability of reprogramming it like you would have by just sending up a whole new set of software.
How am I going to figure that damn thing out?
I've got a thousand words of pad in this computer.
Is it possible in real-time to obviate that abort signal and still land on the Moon?
And this gentlemen sitting over here figured that out.
He figured out how to tell the computer to ignore that signal by going into the certain places in that hardwired software and saying don't listen to the abort signal for a while.
Don't listen to it, but if I need to listen to it on the way down then listen to it.
You've got to do that with a thousand words.
That is a pretty tough problem in real-time, one which nobody had thought about before until it happened.
Or, as I said, this 25 year old young man on Apollo 11 and the vehicle is descending to the Moon and he is getting all these signals back that says the computer is overloaded and is doing so many tasks and stopping.
Why is it doing that?
We've done it on Apollo 10 when we started down to the Moon and it worked fine, and we did a rendezvous from it, but we had the radar on.
And going down to the Moon, we didn't need the radar on it.
That radar was going into the computer, it was flooding the computer with data, but these guys didn't know that.
They had to figure out how to get around that signal.
If I sound like that is a big problem, it is a big problem, and it is going to get bigger.
With a spacecraft you're going to get more complex with each passing day and you're going to have to figure out how to do that stuff in real-time.
And that is what you, the flight operations people and the designers of tomorrow are going to be, and that is what you're going to be faced with.
I know you're working on the Space Shuttle trying to make it better.
That is what your task is in this class.
We should have had you around for the last 25 years because it needs to be made better.
And it is a travesty, I will use that word again, that we haven't been making it better and making it less costly to fly.
We should have been doing that.
Let me start into the Space Shuttle a little.
One of the questions that Jeff asked me that these people will be interested in hearing is how did you decide to do it manned as opposed to unmanned on the first flight?
Sort of out of necessity I guess you would say.
The more we looked at the systems, the more we looked at the Space Shuttle the more we realized that the man could furnish us a certain amount of reliability in space operations and in space systems.
And in choosing systems the more reliable the machine would become.
But we had to convince ourselves that that was a rational thing to do.
Now, let's go back and give you some thought process about the Space Shuttle design.
As we did the initial design, we wanted an escape system.
We wanted to build a pod into the cockpit to allow the astronauts to escape if we had problems.
And building the Space Shuttle main engine was very difficult.
Do these people know about sub-synchronous whirl?
They heard about it from J.R. We couldn't find any bearings in the world that would withstand that load.
They were failing.
And you didn't know when they were going to fail, so we built an automatic shutdown system into the engine.
And Aaron and I were talking about that this morning with Professor Cohen.
That was well, we'll just figure out what all the parameters are that tell you when the engine is malfunctioning and shut it down because we don't want it to blow up.
We are going to look at RPM of the pumps and we're going to look at temperature in the prompts and we're going to look at the pressure in the engine head and we're going to look at the fuel flow rates, et cetera.
That sounds like we can do that, but how do you know it is right?
How do you know you're not shutting down a good engine?
And the point I am making there is that reliability of the instrument becomes more important than the engine.
Think about that.
You can say I will have an automatic shutdown.
I will have the automatic abort and get the astronaut away, but somebody has got to make that decision.
Nobody on the ground can do it fast enough if the engine is going to blow up.
The astronaut cannot do it.
His reaction time is about 1.5 seconds, no matter how much data you give them or how good the data is.
So, it has got to be automatic.
And that is a very difficult task for an engineer.
He can make this system work but he cannot tell you very rapidly when it is going to fail.
And rocket engines have a bad, nasty habit of going like that and it is gone.
You've got to figure out when that is going to happen, so designing instrumentation to do that.
We then started thinking about that in application to the Space Shuttle.
And our experience with redundancy, reliability numbers thinking about that we said we want a system in the Shuttle that is going to be fail operational, fail safe.
That is quad redundancy, so every critical system has quad redundancy in the Space Shuttle.
It sounds like a good idea, doesn't it?
We will come back to that in a minute.
We said we would like to have an escape pod but the escape pod has got to have a stability and control system.
It has got to have a control system because it is going to be a spacecraft.
When is it going to use the pod?
Is the pod going to be used at 100 feet off the pad, 100,000 feet off the pad, 500,000 feet off the pad?
And if this thing is descending from any one of those altitudes, how are you going to control it?
Very rapidly that is another spacecraft.
It is probably a bigger job in building that spacecraft than building the Space Shuttle itself.
So you can see why we didn't do it.
It is too tough, too big a job.
We said we would like to have a go around capability on this machine.
It is going to have an L/D of about four to five at best and the descent angle that you come down to land is 23 degrees.
Let me tell you something.
I have flown in a Gulfstream II on a 23 degree descent trajectory and that is scary as hell.
You're flying a brick.
That machine is coming down like that and you're hanging on your straps.
We would like to be able to have a go-around capability.
OK, we'll put some jet engines on it.
It has got to have fuel.
It has got to have a tank.
It has got to have lines on it.
It has got to have a certain amount of redundancy with it and a certain amount of power.
It has got to be absorbed into the thermal protection system and come out and extend.
What I want to do is go around as I'm reentering from space so I can go once around the runway and land.
It didn't take us long to figure out if we built that system we couldn't carry a damn pound into orbit.
There went our payload.
And we very rapidly started figuring out how to do dead-stick landings and how to preserve the energy and how to get this machine lined up on the runway at way high altitudes, et cetera.
So we did away with the go-around capability.
We've done away with the pod.
We've done away with the go-around capability.
Now we don't have an escape system.
What is your escape system on the Shuttle?
Well, on Mercury and Apollo the escape system is a solid rocket.
Solid rockets have close to 100% reliability as you can get.
Usually, if it lights it burns and it goes.
God, we've got two solid rockets on the Shuttle.
Why aren't they our escape system?
Because, if we can get this thing to 200,000 feet, it will fly.
We can return to the launch site and land from 200,000 feet because you aren't very far down range at 200,000 feet.
That is a good idea, a great idea, but the rockets have to be 100% reliable.
It wasn't 100% reliable in the Challenger accident.
Unfortunately, I cannot account for the fallacies of man.
And I will come back to that, if you want me to talk about the Challenger accident.
But we built solid rockets to have 100% reliability, and they were our escape rocket.
We've got quad redundancy, we've got engines that will perform to the best of our ability and shut down if they are going to explode, we've got solid rockets as an escape system, and we will teach the pilots, with the best control system we can come up with, to land this machine dead-stick.
Now we've got to convince ourselves and the management that the best way to fly this thing is unmanned or manned on the first flight.
And we looked at flying it unmanned.
We could have done that.
We could have put an automatic control system in to take the place of man.
And these pilots that tell you they do manual control during reentry in the Shuttle, hogwash.
Pure hogwash.
The Shuttle will not fly without the automatic control system.
The pilots are flying the outer loop.
I could fly it.
I have flown it in the simulator.
You know how I do it?
I don't touch the damn thing.
You set the damn end of the runway into the computer, right down.
And that's the way 95% of the airliners land today, on automatic control.
And, frankly, that is how the astronauts ought to do it.
They haven't done it yet.
They keep telling me what are we going to do if the system fails?
And I say to them you better wind your damn clock because if that system fails the thing is gone.
It is an unstable machine.
From mach 25 to touchdown it is an unstable machine.
If it diverges it is gone.
If pilots tell you they are doing manual entry on the Space Shuttle, I repeat that statement, hogwash.
It cannot be done.
Now, I can teach them, however, by building a Gulfstream II into a space shuttle like vehicle.
I can put the control system that they've got in there.
I can repeat that.
I can make it come down.
I can put reverse thrust, if you believe, on both engines in a GII.
And you're descending and you've got both those damn engines back there going like that as it does its reverse thrust to match the drag of the shuttle during entry.
But I can teach them pretty well how to do dead-stick landings.
And so, I put all that together.
And Professor Cohen and myself and a few others go to Washington to convince the powers that be that we can fly this machine manned on the first flight because it is the most reliable way to fly the machine.
And we convince them.
It took a little doing, a lot of fancy talking, I guess, but that is what we decided to do.
And I think it was a good decision.
In retrospect, it was a lousy decision.
Why?
Because if we had had unmanned flying capability on the orbiter and we had the Challenger accident, we could have flown it again the next day with the unmanned control system and proved that it was OK. Well, we didn't have that capability.
And, when you get into the politics of flying men in space in this country, rational thinking does not carry the day.
Political thinking carries the day, even in NASA.
You will find that out as young engineers also, that you not only have to be an engineer, you do have to be somewhat of a politician in order to sell your programs.
We convinced ourselves that it was the best thing to do at that time to fly man on the first flight.
Because it is in my head and it comes out, I want to say a little bit about the problems that they face today.
Or the way in which they use the Space Shuttle today is frankly not how we intended the machine to fly.
We have quad redundancy.
And the way NASA uses it makes it less reliable than if we had just had a damn single string system.
Why?
Because they have to have all quad redundant systems working at liftoff.
So all four systems have to be operating at liftoff.
That is hard.
It is hard enough to have one operating properly.
Now you have to have all four in all the places where you have quad redundancy.
That isn't how we intended it to work.
What we intended it to do was, you would get to the pad and get ready to launch and one system fails, you would keep right on going.
If you have two fail you keep right on going.
That is the way we intended it to be because we wanted it to be a reusable system with quick turnaround.
Two weeks we want to be able to turn that machine around.
And you could do that today because they've flown in 100 and something times.
They know that machine.
They know it well.
They know how the systems perform.
You can look at the telemetry when it lands and say all three of the four were working and I've got a thing over here.
I can fix that in ten minutes.
I am ready to go.
They put more time on the systems on the ground than they do in flight testing the machine.
Is that expensive?
My God, that's the reason it cost $4 billion a year to keep all those running standing Army at Cape Canaveral.
If you're going to build redundancy in the future, you guys, if you're going to put redundancy in this machine, don't tell them about it.
Now, that may sound funny.
But that is how your computer is designed.
You don't have BITE in your computer anymore, built-in test equipment.
Do you know why?
Because damn engineers kept using it.
Now the computer builders build in the test equipment but they don't tell you it is there.
The redundancy works automatically.
Probably got three or four in there just like we do in the Shuttle, but your computer keeps working.
It might be a little slower, sometimes it can get you as mad as hell, but they don't tell you about it.
My advice is don't tell the people about it the next time.
Don't tell the people at the Cape this thing has quad redundancy.
Now, that is probably somewhat foolishness but I guaranty that if I had anything to do about it the next time, I might very well do it that way because it is not the right way to think about this machine.
The machine is a beautiful, wonderful piece of hardware.
The orbiter system, the most complex system ever built by man to fly, I was talking to some coops a few days ago, I've said that word 14 times, it has never had a failure.
The orbiter has never had a failure that would have prevented that machine from landing safely.
NASA is now going to use, however, all of the components that have failed.
They are going to use the solid rocket.
They are going to put a command-like service module on top of it with an escape rocket and fly it.
That system has failed.
They are going to use the tank.
Engineers, this day and time, cannot figure out how to put insulation on the tank.
Ridiculous.
Not only ridiculous.
Ludicrous.
You mean to tell me there is not a design engineer sitting in this room that cannot tell you how to keep the foam insulation on that tank?
Hell, my son could do it.
In fact, my grandson could probably do it.
The Space Shuttle main engines, now that is a damn tough piece of hardware.
That is what we're going to use.
That is what we're going to put in orbit and use as a propulsion system to send you to the Moon.
Now they say we're going to make some changes to it and make it better.
I will probably be dead and gone, but I would like to see it.
That thing is designed right up to the teeth.
The maximum power you can get out of hydrogen and oxygen has an ISP of about 460, and the engine on the Shuttle is 458.
I don't know how you could get much more efficient than that, but that really is a tough engine.
You're continuously changing components on it.
But those are the three components.
NASA says I am going to use those to go to the Moon, but I will throw the orbiter away.
Never had a failure on the orbiter.
We talked a little bit about RTLS, return to launch site.
That sounds like something easy to do, but think about the software involved in that little dude in figuring out how to do it, how to take that machine back to land at Cape Canaveral if you do have an abort.
Think about the software involved in that.
Think about the possibilities.
Think about the cutoff conditions.
Think about the mach number range you're going to have to fly.
Return to launch site was part of our philosophy of not having an escape system per se on the Shuttle.
I know we have some navigation and guidance people in this audience.
How then did we decide?
Jeff asked me to fly the first time.
That was tough, very difficult.
We had some of the best experts in the world come review what we were doing and look at all the systems.
The one thing, two things, really, that they had the most trouble with in making sure we knew what we were doing.
First was the thermal protection system.
And we never really convinced the experts that we did have a thermal protection system that would work.
As a matter of fact, the man who invented lunar orbit rendezvous, he didn't invent it but he sold the day in using lunar orbit rendezvous.
And the chief structures engineer in the United States, that is probably a stretch but one of them at Stanford wrote me a cosigned letter on T minus one month after we had reviewed this system, we had done everything they asked us to do.
We ran combined loads test, we ran worst loads test, we ran combined worst loads test to prove to ourselves that the thermal protection system wouldn't fail.
I got this letter at T minus one month.
It is now in my effects at Virginia Tech.
As a matter of fact, Virginia Tech said are you sure you want this letter in your files?
I said oh, yes, I want that letter in my files.
It is probably the most important letter you will ever have in my files.
But these two gentlemen said we implore you not to fly the orbiter because the tiles are going to come off.
It may not come off while you're at the max heating pulse, but by the time you get ready to land NASA is going to be totally embarrassed because all the tiles are going to fall off.
And we would ask you to put a steel net around this vehicle so that they won't fall off.
I am still aghast at that letter because I don't know what else we could have done to satisfy them that we had done everything they asked us.
They were concerned that the strain isolation pad, which is what the tile sits on, and you've had that explanation, they were concerned that the combined vibrations and aerodynamic loads would cause a failure in the SIP or at least at the glue joint.
And that, indeed, all the major tiles that had gone through the large heat pulse would fall off.
Professor Cohen and I met them on the Queen Mary at an AIEE meeting after the first flight had taken space.
It was satisfactory.
It flew well.
It flew beautifully.
Those two gentlemen came running out and asked Aaron if we could come and sit down and have a drink with them.
And we said hell, no, and walked away from them.
That is what I thought of that letter.
Say that again.
We didn't get the last part.
Now, the other thing I want to talk about is the control system of the Shuttle.
We had several gentlemen [NOISE OBSCURES] and said we are infinitely smarter today.
Yes, I did.
The public affairs officer said we need a comment from you, after the machine had landed, on what you think of this flight.
I said we have just become infinitely smarter.
Not so much about the tile were we infinitely smarter but the story I'm about to tell.
The automatic control system, in terms of the aerodynamic parameters that it has to deal with, there are roughly 35 variables in the aerodynamics of the machine all the way from products of inertia to cnBeta.
Does everybody know cnBeta?
CnBeta is the capability of the rudder to stabilize the machine, the yawing moment due to side slip.
And I still do remember a little bit of my aerodynamics.
Anyway, there are no facilities in the world to measure those 35 parameters, very accurately anyway, that would tell you what to design the automatic control system to.
It is not so difficult at high mach numbers above, say, ten.
You can use, what do you call it, Newtonian flow?
And, as you well know, you use the aerodynamic controls when you can.
You use the thrusters when you can, and you combine the two as you transfer from one to the other in the range of flight regimes.
But, actually, it is a guess.
We looked at all of the possibilities that had been done on the X-15, had been done on the Dyna-Soar program, had been done in wind tunnel tests of almost every configuration for hypersonic flow.
And you just couldn't nail down what those 35 parameters were.
What we did was said here is what we think it is, here is a variation on top of that, and here is a deviation on top of that and here are those 35 parameters throughout the total mach range.
And we are going to do a Monte Carlo analysis at every tenth of a mach number from 25 to touchdown and have no failures.
And, if we have a failure, we have to change the gains and go back and run it again.
I don't know about you.
That sure convinced me that that machine would fly.
But I still said we just became infinitely smarter.
I said, before we flew, we certainly have to change the gains because there will be regions where the machine is oscillatory coming down and might frighten somebody, even if it were going to damp.
That was post-flight.
I said I don't ever want to change the gains.
The damn thing worked.
So, those two problems were the fundamentally hardest problems to be able to test and assure yourself that you were ready to fly.
But, in the end, how did we decide to fly?
Frankly, I didn't know what else to do.
In my mind, Professor Cohen's mind, John Yardley's mind who was head of Manned Space Flight, these gentlemen sitting in front of me that worked on the automatic control system, we didn't know what else to do.
We had done everything we could think of.
We suspected that there were some unknown unknowns.
A good old NASA term, "unknown unknowns".
But we didn't know what they were.
We didn't know how to test it so we've got to go light the torch and do it.
I don't think it took a lot of guts or nerve.
71:00 And the responsibility of the program because it may very well be something that just causes the Space Program to collapse in the United States.
And that is a tough task.
You don't learn that overnight.
You just sort of have to learn to live with it.
And, frankly, you have to like it.
You have to like being in that position.
I had one of my flight directors who became very much affected by it mentally.
So affected that he quit and went and became a psychiatrist.
Did not ever perform as a psychiatrist but got his degree in psychiatry.
Phil Shaffer, if you know him, one of my very closest friends.
He is now making me kind of rich because we own a lot of oil wells in Oklahoma together.
It is a tough job but it is a marvelous job.
There is nothing I can think of, including pitching in the World Series or winning the US Open that comes close to being in this business.
Every day is a challenge, every day is hell and everyday is the best feeling you've ever had in your life.
That is how you feel every day.
You go through those transients almost every day.
I highly recommend it to you.
I don't think you can get rich until you retire and become a board member like some of us have, but before that you're not going to get rich but you're going to be happy every day.
And you are going to feel like you contributed every day.
And I don't know, at least in our business, anything better than that.
And these gentlemen all sitting here in front of me will attest to that.
It is a great life.
With that I will stop and take your questions.
[APPLAUSE] Chris, well, I'll start off.
Aaron, can you stand up?
Yes.
What would be the systems that you would think could use most improvement on the Shuttle today?
With today's technology and then as you see things, what would be the systems that you think could use the most improvement in terms of performance margin?
Well, again my first answer would be political.
I would approve the thermal protection system.
I don't think it has to be improved very much, but it would sure get a lot of people off our backs if we improved the thermal protection system.
And there are ways of doing that today.
There are some advancements in the state-of-the-art of the materials probably in the way you attach to the plow to the machine, probably the way you proof test it.
All of those things, I think, would be where I would go first.
I would put an electric system in secondly.
What about hydraulics?
And get rid of the hydraulics because I think that is a maintenance problem.
And I think that the APUs are always going to be a problem because that is something that is turning up at 400,000 RPM, a rotor that is about that big turning up 400,000 RPM.
And, because at 400,000 RPM, actually, instead of being this diameter is now an eight to a quarter of an inch bigger because the metal is stretching under those conditions.
The valves we've had trouble with, as you well know.
The material where this thing is pulsing.
And these are nothing new.
There are designs, there are electric motors, there are power systems that would do it much better than a hydraulic system.
Those two systems, I think, are primary.
Now, the other thing is you have to keep up with the state-of-the-art.
And one of the biggest problems you have with the Shuttle today is nobody builds the parts anymore.
You go to the manufacturers and they say oh, we stopped building that system ten years ago.
All the circuit boards and everything is passé.
And that sounds like, well, we'll just change the circuit boards.
Well, that is a tough problem because the process specs have got to be looked at.
The sneak circuitry, the sneak paths have got to be looked at because that was the thing that always got to us in space flight, everything works fine and then some day somebody turned something off and the thing glitches and it fired the retrorocket or something like that.
And so you have to be sure that when you do redesign the circuitry into the modern world that it is properly done.
A lot of things you don't need on Space Shuttle probably, you don't need the backup flight control system which costs a lot of money.
And then I think the biggest thing I would do is force the system to use automated checkout.
That is where all the money is, I think.
And maintaining the machine, nobody is willing to use automatic checkout.
It is there and you can do it very easily but the people at Cape Canaveral, they need to be flight controllers.
What I mean by that is the people at Cape Canaveral who prepare and maintain the machine have a totally different approach to the space machine than the flight controllers do.
The people at the Cape want it to be perfect when it is launched.
And so, when they do the checkout and it doesn't work right, they go plug a new board in or they go put a new system in or change out the fuel cell.
A flight controller does not have that prerogative.
He has got to figure out how do I live with what I've got and make the best of it?
I think that is the best attitude.
And they need more of that in the maintenance side.
I think that is where the biggest savings and the biggest improvement could be made in the Space Shuttle.
The other thing I think about is the engine, SSME.
I think if you derated the SSME that the turnaround time on the engine and the reliability of the engine would go up significantly.
If you derated the engine, i.e., instead of asking it to put out 108% every time you go to the Space Station because you're going up to the higher inclinations.
You just put a bigger head in it or whatever it takes to derate the engine.
Make it run at 95% power.
Gee whiz, the thing would last forever if you did that.
Chris, you mentioned in your talk about going back to the Moon.
And I was going to ask you a couple of parts of the question.
One is what for?
Two is it to go to the Moon or is it set in stone to do something else?
And then perhaps you could comment on what your prediction is for the Space Program for the US in the future.
You're going to get graded on it.
Why go back to the Moon?
Well, I think that there are enough resources on the Moon to make it economically viable, number one.
Number two, I think you could develop enough electrical power on the Moon to provide enough power for the people that were going to live there and be stationed there permanently.
And you would do it on the backside, by the way, not on the front side.
That is where you would want your permanent base on the Moon to be on the backside because it would be shielded from all the electronic signals from the Earth.
It is the best place in the world to look at the rest of the universe.
I would build my space base on the backside of the Moon.
And then I believe there are possibilities, although it is not technically sound yet.
But I believe there are possibilities of providing enough electrical power to the Earth from the Moon that you could shutdown every power plant on the Earth.
You get that much electrical energy from the Moon.
And that would certainly offload the power requirements on the Earth which are almost logarithmic, aren't they?
China is going to end up using more electrical power than we do shortly, so we need electrical power.
And I think you could get that from the Moon.
Plus, I think that the geophysicists and the geologists, astrophysicists all can give you a thousand reasons why you should go back to the Moon because of what Professor Yuri said.
It is the Rosetta Stone of the universe.
There is more to be known from the Moon about the Earth than you can ever get out of the Earth.
It is still very useful to go back there from a scientific point of view.
Plus, the engineering and economics of it.
I didn't say it because I wasn't sure you wanted to have such a discussion.
But I will.
I think it is a travesty that we aren't doing it with the Space Shuttle and the Space Station to go back to the Moon or to go to Mars.
Why do I think that?
Because, in either case, I don't understand why you would want to build an Earth entering vehicle to go to the Moon.
What you want is an interplanetary spacecraft.
Why does it have to have a heat shield?
Why does it have to have parachutes or some kind of landing capability?
Why not just go to and from Earth orbit?
And the Space Shuttle is the greatest machine for carrying things too and from orbit that has ever been thought of.
And the one they are going to build, even if it's an unmanned vehicle, is still going to have an awful lot of complexity to it.
And I think that if you've made the Space Shuttle economically viable, which nobody thinks you can.
And maybe Chris Kraft is the only one that thinks you can.
If that is true then that is true.
I don't think that way.
I think the Space Shuttle is an economically viable machine, and so it is a travesty to me to throw it away.
And the Space Station could be used as the place where you assemble all this stuff.
And everybody says well, it's at the wrong inclination.
Yes, it is at the wrong inclination.
You were foolish to put it there in the first place, but that is where it is.
Every time I had a little bit of fuel left over, I'd inch it down a little bit, and the first thing you know we'd be down to 28.5 degrees [NOISE OBSCURES].
Today it costs you 15,000 pounds of payload to go to the Space Station, and that is a travesty, too.
You wouldn't want to do that continuously.
I would use the Space Station as my assembly point and I would use the Space Shuttle as the machine to go there.
And then I would build myself a bunch of interplanetary spacecraft to go to and from the Moon and Mars.
And when I did that I would be smart enough to build all these newfangled structures, inflatable structures, et cetera, which you could use not only as the interplanetary spacecraft.
You would use that where you would live on the Moon.
I think it has a better approach from an engineering point of view.
Whether that is the political way to sell the program, Mr. Griffin has got to do that.
I mean that is what his job is.
Fortunately, it is not mine.
I was asked to be the Deputy Administrator and the Administrator of NASA several times.
And you can hear the way I talk here that I would have lasted about six days to six months.
What do I think about the possibilities of today's plans?
I hate to say this but I think it is going to fail.
I don't think it will work.
I don't think that the program as stipulated today is the way to do it.
And I don't think that political climate is such that the budgetary support will be provided.
And I hope I am wrong.
I don't think I am.
Do you think the Chinese will be there?
Beg your pardon.
Does that mean the Chinese will be there?
No.
Hell, the Chinese are 50 years from going to the Moon.
They cannot buy it from Russia.
That is what they're doing today.
They're buying all their technology to put man in space from Russia.
You know, that makes me think the Russians are still using the same spacecraft, with slight variation, that they put Gagarin up in 1961.
Literally, it's pretty close to it.
I've sat in it.
Have you sat in it, Fred?
You sit in it like this.
But they used it over and over and over and over again.
The B-52 has been used over and over again.
It has a new wing.
It has new electronics.
It had new bombs.
It has new everything on it.
The only thing that is the same is the configuration.
It had new engines probably five times in its lifetime.
That is what we ought to be doing with the Shuttle, isn't it?
We seem to have this great propensity in this country for building something wonderful and great and high performance and then throwing it away.
We put up the SkyLab.
Wonderful.
Throw it away and don't build anymore.
Build the Saturn 5.
Gee whiz, it will put 200,000 pounds to the Moon.
It is rotting away at Johnson Space Center.
The Trekkies in the country got so mad at the Johnson Space Center that they made them build a hanger to put it in so it wouldn't rot anymore.
We built a Space Station and we're throwing it away.
We built a Space Shuttle and we're throwing it away.
Golly, my mother would have gone bananas.
We had leftovers almost every other meal.
I know that's trite, but don't you think that is foolishness to do things that way?
I think it is.
We did learn from space flight that everybody seemed to learn from everybody else's experiences.
That was an amazing factor to me, in the early days particularly.
The astronauts, each one that flew was so much better than the one before.
And knew how to do the job better than the one before.
All of those things, you could see yourself advancing, and they just built on this experience of each man.
And I don't know how they transmitted that to each other.
They didn't, obviously, because they didn't have a brain connection.
But that is the way we ought to be doing it here.
We've got all these things that we build and then throw away and don't take advantage of.
That doesn't make a lot of sense to me.
Now, I know there are politics involved.
And that is a reason I said you had to take into account the politics.
When can you get the money?
How do you convince people that you need to do things?
What does it do to the job situation?
How does it affect each senator's and each congressman's state in terms of the money that you bring into that area, et cetera.
When we built the Space Shuttle we had a contract in every state in the union except Alaska.
And we did that on purpose.
Aaron and I flew to every one of those 75 major subcontractors in the Shuttle.
They were all over the country, and we would go visit them at least three or four times n a period of a couple of years.
The politics has to be there.
We were lucky in Apollo.
I call it the conjunction of the stars and the conjunction of the politics.
That is a story I meant to tell.
By 1978, '77 maybe, we were really behind a power curve on the budget for the Shuttle.
We had been pushing a wave of about 10% less funds than we needed each year.
At about that time period we needed, if we were going to have any semblance of making a 1980 or '81 first flight, we needed a $600 million supplement and about that much per year more than we were getting.
And NASA had this big meeting down at the Johnson Space Center.
And we all talked about what the problems were and how we were going to meet that.
And everybody, absolutely all the politics said you cannot go ask for a supplement, you cannot go ask the Congress for any more, you cannot go to the White House and say we need more money.
So we may just have to turn this thing into an X-15 project, a research project.
The first Shuttle flight will be whenever we can build it and it will just be a test vehicle.
We were all very downhearted about that.
Dr.
Frost, who was then the administrator, went up to the White House about three days later.
And Carter, who was the President, called him in and said I want to tell you how wonderful that Space Shuttle is.
And Frost said he could feel himself tighten up.
And he said, you know, I just had this meeting with the Russians at the Salt Talks, whatever SALT meant.
And I was pointing out to them that we were building this marvelous new space machine and we were going to be able to do all kinds of things with it.
We were going to be able to fly over Russia and look at the world.
And he said it carried the day.
And Frost said, oh, my God, what is NASA going to do about that?
He came back, thought about it for a few days and went back to see the President.
He said, Mr. President, I have to tell you, this Shuttle is in trouble.
We don't have the money.
We're not going to make the launch dates.
It is questionable whether we can even build the machine at this point in time because we haven't built the thermal protection system.
We didn't even have the factory built yet to build the tiles in.
Mr. Carter said how much do you need?
And he said, well, I think we need about $600 million this year and I think we will need about $400 million a year.
That was a WAG on Dr.
Frost's part.
And the President said you will get it.
That was how close we were to the Shuttle failing from a political point of view.
I don't know what the politics of tomorrow is that might change our mind.
It would be wonderful if the Chinese would land on the Moon tomorrow because that might get the Congress back in action.
But can you see the Congress, in the face of what has happened with Katrina and Rita and a few other things that happened in the United States, and the Iraqi War and the budgetary problems they face -- I cannot see them giving NASA the money they would need to do the program.
And NASA says they can go back to the Moon between now and 2018 for $106 billion.
Mr. Webb doubled the price.
All us great cost estimators, we estimated the cost of the Shuttle, and Jim Webb got it and multiplied it by two.
Today I would say you would have to multiply that by ten if you think you're going back to the Moon.
I cannot imagine.
Aaron and I worked on a program in 1998.
When did we do the Mars study?
In '89.
We estimated the cost in 1989 dollars to go to Mars at $400 billion, and I think we were low.
I keep have to catch myself.
I know it's the politics that you have to be concerned about.
If you told them that it was going to cost $400 billion dollars then for sure it would be cancelled, right?
You've got to tell them something that is rational, but I don't think they will get supported even with the amount of money they say it is going to cost.
I hope I am a pessimist.
Yes, sir.
I have two separate questions.
One is the improvements you said that were made in computing capability from Mercury through the Shuttle, I was wondering if there was any thought into making the computer systems on the Shuttle more upgradable?
And, if there was, would there have been any value in using modern systems?
And then the second question is related to the comment you made about if the Shuttle was designed to fly automatically after the Challenger accident they could have just flown again the next day.
I wasn't really sure what you meant by that so if you could elaborate on it a bit.
Let me go to your first question.
It isn't the hardware.
It is the software.
And it is the software checkout and it is the software validation that you have to worry about on the orbiter.
When you have four systems that are operating on a 40 millisecond time cycle and checking with each other at the end of that cycle that they are all in lock-step -- John is looking at me saying my God, how did we ever do that?
And I don't know how we ever did it, but we did it.
It is the software and not the hardware.
When you have an updatable computer, which you should have and will have, it is the software that is the problem, not the hardware.
That has always been the case.
Trying to make the software fit into the new computer, the way the hooks are in system is just totally different.
And, therefore, the guaranty that the system doesn't have a bunch of glitches in it that are going to get to you, you know, when we flew to the moon on Apollo 11, we had a book about that thick with computer anomalies in it.
We understood them all, but that is how many software fixes we needed to make.
We just didn't make them because we didn't want to nor had the time to do them.
What I meant by flying an unmanned shuttle was that you couldn't convince the politicians or the powers that be, whoever they are, that you could fly the shuttle the next day manned.
But, if you didn't have a man in it, who would have cared?
And so you could have flown it the next day and it would have worked perfectly because it was warmer at the Cape.
Do you know what the condition on the pad was the morning they launched Challenger?
There were icicles hanging off the gantry that long and that big around at Cape Canaveral.
Let me tell you something.
You know why they got there?
They did what we did in 1920.
They turned the water on and let it run all night because they were afraid that the fire suppression system on the pad was going to freeze and not be able to be turned on when they launched so they let it drip all night.
And I am sitting there saying that solid rocket has not been qualified for temperatures below 47 degrees Fahrenheit.
And they're convincing themselves that the core temperature of the solids is much higher than that because it has been sitting in the sun for the last two months and the temperature is so and so, but they didn't think about the seals.
If you remember that professor sticking the seal in the ice.
Well, that's what I meant.
The next day the temperature would have been warmer, put new rockets on it.
Max Faget said the next day, why don't you just put a heater around the joint and put a belly band over the top of it and fly?
Damn good idea.
Yes, sir.
You mentioned that one of the reasons the Shuttle is so expensive is because you have so many redundant systems onboard.
Now, what would be your solution?
Not to have them onboard or just not to have them all upgrading at launch?
No, you got the wrong implication there.
It is expensive because they insist on having them operating at the time of launch.
It is not the redundancy that makes it more expensive.
It is the checkout and the testing and proof that it's there and the replacement of the systems and the use of the systems that makes it more expensive.
But what makes the Shuttle so expensive are the numbers of people involved in preparing it.
There are roughly 10,000 people involved in that operation who make X number of dollars per hour or day.
And it costs, in today's money, about $5 billion a year to fly the Shuttle seven times.
It probably cost $5 billion a year to fly it 30 times.
Not much difference.
Maybe $5.5 billion.
So, it is the people.
You've got to get rid of the people.
And people have been trying to do that ever since Mercury, get rid of the amount of people.
And they try and they put in the automaticity and the automatic checkout and then don't use it.
Check it out.
And, as I said, they put more hours on it in the hanger than they do in space.
It wears out the system checking it.
And it takes people to do that, so it's people.
So what's your solution to that?
I think you have to have some hardnosed SOB that says I'm going to get rid of the people, and you do it with automaticity, automatic checkout, automatic everything.
In today's world, why would you do it any other way?
I will give you another example.
When we built the caution and warning system on the Space Shuttle, the safety and reliability people said it has to be hardwired.
Can you imagine that?
Why wouldn't you use bits instead of hardwire?
Oh, but it's a lot safer and a lot more reliable.
I will use my same word, hogwash.
It is a lot safer and more reliable with bits.
But that's the system.
You've got to change it.
And two of us have tried.
You can see us flying down in flames almost everywhere.
Yes, sir.
For the young engineer, a lot of us hear about the Apollo Mission and talk about the good old days where we wish we were in the days of engineering by the seat of your pants.
Is that type of job and that type of excitement of engineering possible in NASA's environment today or should we look elsewhere like towards private enterprises?
Could we find the type of job we hear about in NASA these days?
I have to be careful how I answer that.
I want you in NASA.
It is important that you guys be in NASA.
Or at least in the Space Program working in the industry.
Because you are tomorrow's opportunity.
Just because you hear Chris Kraft say things that doesn't mean a damn thing tomorrow.
You have to do it yourself.
And you have to want to do it yourself.
And you have to bring the ideas to the program.
And you have to be willing to do that.
And this willingness to take on the things that must be taken on in order to get the job done.
Your question about seat of the pants, et cetera, sure, we did a lot of things when we first started by seat of the pants because we didn't know any other way to do it.
We did it by feel.
We did it by past experience.
A lot of us had been in an airplane flight test world.
So, we did it by having seen the past doing things the right way.
That was our seat of the pants.
Our seat of the pants wasn't just a scarf around our neck, so to speak.
It was an educated seat of the pants.
And that is what you have to provide.
In the end it will take a lot of the seat of the pants.
I think the way I know Aaron and I have done it was to believe in the people you had.
You have to learn how to find out who the guys are that know what they're talking about and trust them.
You have to put them in the job, give them the responsibility and authority to do it and then trust them.
And you have to build them.
The biggest problem that you have today, that NASA has today and the aerospace industry has today, the biggest problem, I cannot say that too strongly, is that they have not built anything in 25 years.
And so they've forgotten what it takes to do it.
You don't know how to do it.
But if I put you on the job and gave you the authority to do it, you could do it in three or four years.
You need my help and guys like me to tell you where the bumps in the road are.
But you're going to do it a hell of a lot better than I did given the opportunity to do it.
Did that answer your question?
[AUDIENCE QUESTION] I think it is.
You might have to help it.
You have to vote.
In the `60s, I was hoping that all these space cadets, all these Trekkies, all those guys would now be in the Congress and that they would vote for the Space Program.
Boy was I ever dead wrong.
There aren't enough Grateful Dead fans around.
Yes, sir.
Do you think the success of the Apollo Project was in some way linked to the ferment that was taking place in the `60s?
Oh, absolutely.
I think that we had an enthusiasm at the time that is probably unparalleled in engineering circles.
Now, we had the Manhattan Project to build a nuclear bomb.
Draper had the project to build a Polaris submarine, but it was clustered.
What is that word?
And it wasn't seen as we brought it to the fore.
It wasn't a national program.
It wasn't a national priority.
It wasn't national pride.
Nobody knew about it.
Everybody knew about Apollo.
I was damn proud to walk into any room where I ever went to say I worked on Apollo.
That reminds me of a horrible cartoon, to make my point.
I was in Austin, Texas following the Challenger accident.
And here I am one of the proudest people to ever be in NASA.
The cartoon in the paper when I got up that morning was the following.
It showed these kids playing with a Frisbee.
The first one throws it over here to this guy.
The second one throws it back.
The third one the damn thing explodes in his face.
The fourth was says his father works for NASA.
Boy, that brings it home to you pretty damn fast, doesn't it?
Nobody in 1967, '68, '69 would have dared put that cartoon in the paper.
By having new people in a space program there is always this lack of experience which you have to train just to have.
How do you transmit that experience to the new people?
You intermingle the young with the old.
You bring the elderly engineers into the system and make you responsible for designing the system but have me in your hip pocket.
After six months, two years, you won't need me in your hip pocket anymore because you would have learned all those things And learned them better and done them better.
You have to mix them.
The ingredients for the pie and the cake have to be there.
And that is part of the ingredient.
And why hasn't that taken place?
Because of two reasons.
NASA hasn't done anything in terms of building new hardware, other than the Space Station which really wasn't testing the state-of-the-art.
It was a great program but didn't test the state-of-the-art.
The industry has been doing the same thing.
The industry has been building airplanes, but they haven't been building any spaceships so they've lost that capability also.
That's the travesty, too.
You've got to rebuild them both at the same time.
But it only happens by experience.
I cannot take what Professor Cohen knows and what I know and put it in your head.
You have to fail a few times.
It is only by the failures that you're going to learn.
I once heard a sermon that said when you're a young Christian or Hebrew walking down the aisle all your doors are open.
I am walking down the aisle and I'm pretty close to the end and all the ones behind me are shut.
You're willing to go in all those doors.
I am frightened to death to go in those damn doors.
We need you and you need me, but you don't need me very long until you get up to speed.
Don't be frightened of it.
Go do it.
And don't be afraid to fail.
You learn much more from our failures than you ever learn from our successes.
Back there.
Given all these problems with NASA that we've seen just growing over the years, what do you think about the commercial efforts to access space?
Not necessarily what is going on now but also in the future?
If we had that ability to sort of bypass NASA's framework with all the [UNINTELLIGIBLE] bureaucracy issues, is that a possibly way for us to continue?
Well, I think it is very possible but the problem is investment.
The investment specialists say if you cannot give me a return on investment in three years, maximum five years, I won't invest in it.
And the aerospace industry is worse than that.
They don't have any money.
They aren't willing to make an investment in the future.
The investors who have the money want a return.
The guys that have the capability to do it don't have the money to invest.
Until it gets to the point where it is a little more realistic from the return on investment, it is not going to get to the point where commercial ventures are willing to do it.
Because it is just too expensive.
And probably if failure is so high in our business that investors shy away from it.
I think it will come to that, but I don't think it is going to come very rapidly as it did, for instance, in the airplane business.
I mean even today we wouldn't be flying the transports we have or the supersonic airplanes that we have without government having made that investment at the time.
The 707, the first big really good jet transport, was totally dependent upon the B-47 airplane, which is built by the same people, right?
And so it was a government investment.
I said, well, we did it in an airplane business, to a certain extent, but nowhere near as much as people think.
All of the technology was done by the government investment in the airplane.
And the next step, the supersonic transport, which we ought to have.
That is another travesty, that we don't have a supersonic transport.
That hasn't been done by Boeing or Lockheed because they won't invest that kind of money on their own.
They take the investment of the government in supersonic aerodynamics and engines and structure and use it in a supersonic airplane.
I think it is a ways off.
A lot of people have tried it, the most notable being Kistler recently, and have failed.
I think it can be done and done better.
You can do it better outside of the government because you don't have all those regulations to contend with and all the GAOs on top of you.
It is just going to take a while.
I hesitate to say this but I will.
Programs like SpaceShipOne, that is trickery, that is child's play what he is doing.
Wait until he tries to go to orbit.
That is his next step.
Tell me when that is going to happen.
He is kidding the world at the moment.
Chris Kraft says that.
I want you to remember that because I could be dead wrong, but that is my opinion.
Just child's play.
We did the X1 in 1946.
And we didn't have any buckling in the structure either, which he had.
Where do you think that buckling came from?
What do you think happened there?
Where did it come from?
But if his plane renews interest in space, I mean if it inspires a ten year old to end up working for NASA 15 years from now, isn't that a good thing?
That is great.
Absolutely.
But don't kid yourself that the next step is flying a machine to orbit.
I am at fault for being so negative about it because, you're right, I think it does inspire the young to do it.
As much as I hate to cut us off, it is 11:00.
Class is over.
Students again remember to pick up your papers.
Chris, that has been just an extraordinary opportunity and an experience for all of us.
And, once more, we are very appreciative.
We know you don't give very many public lectures these days, and that you chose to come here and talk to everybody at MIT, we truly appreciate it and we would like to thank you again.
[APPLAUSE]
It was quite an experience for all of us to get to hear him.
And it is equally very fortunate now to have Wayne Hale who has had a very illustrious career at NASA.
Wayne came to NASA the same year I did in 1978, worked his way up as an engineer through the propulsion system, the integrated communication systems.
Became a flight director in 1988 and served there for 15 years.
He was flight director both for the orbit phase and then the most really kind of intense flight control phase as ascent and entry.
And Wayne was ascent and entry flight director and then lead flight director for many flights, including-- we were trying to figure out before.
But I think he actually served for all of my last four flights.
So we've known each other for a long time.
In 2003, he was sent to the Kennedy Space Center to be head of launch operations.
But he didn't get to spend very much time in the Florida sunshine because they asked him to come back to Houston the next year as deputy program manager for the Space Shuttle.
And then, when Bill Gerstenmeyer left to move up to NASA headquarters, Wayne became the program manager.
So he is able to talk to us both about Mission Control operations and the current and future state of affairs of the Shuttle.
And he is very happy to have you ask questions while he is talking.
And I am not going to talk anymore because they came to hear you, Wayne
Thank you, Jeff.
I cannot tell you how much I've been looking forward to coming here to talk to you folks.
This is my favorite subject.
Space, number one.
The Shuttle, number two.
And Mission Control, number three.
Maybe not exactly in that order, but they are all three topics that have been-- [NOISE OBSCURES] Oh, that was you.
I thought that was me.
I thought that's a sound I haven't heard before and I'm in big trouble.
OK.
But, as Jeff said, please feel free to ask any questions.
I will tell you all kinds of interesting stories today, I think.
Some of them might even be about Dr. Hoffman.
I should tell you my background as an engineer.
I got my bachelor's and master's degree in mechanical engineering.
Did not decide to pursue a doctorate degree for a variety of reasons.
You need to understand that the organization of NASA is very much around the centers.
It was organized as a group largely of independent centers with a very thin veneer of integration at NASA headquarters.
And it remains that way to this day.
And every center is quite different from every other center.
We have some centers that are largely involved in research.
And, if you want to work in the research center, Langley, Glenn, Ames then you need to clearly pursue a PhD.
If you're more interested in an ops center, Kennedy, Marshall, Johnson, then you need to kind of not go after the PhD.
Because people with PhDs clearly do not have a real world practical understanding of problems.
And it is only the guys with bachelor's degrees that get out and know how to turn wrenches in their spare time required there.
So it is very interesting, the different cultures.
I can say that with a grin because there are different cultures within NASA.
And working between the centers is very interesting.
And since I have moved in the management ranks, for what reason I don't understand, I have fouled up in my career.
[OVERLAPPING VOICES] make you a manager when you don't get to do the things that you're good at anymore.
Yes, promote you something that you were never trained for.
It has been very interesting for me to try to work between the various centers to help them accomplish the goals that the agency has.
One of the things I should say is that I regret my academic career as I didn't take more psychology, more accounting.
Because, frankly, when I was in graduate school, I can remember one of the courses I had was systems of nonlinear partial differential equations.
Anybody taken a class that has a title vaguely similar to that?
I never used that class.
I have never touched a differential equation in my professional career.
Now, why that is, is kind of interesting.
But what I really did need was trigonometry and accounting.
And you will see a little bit of that.
Today my class is not going to be highly technical.
It is going to be a series of illustrations about Mission Control.
And, as I said, I will be happy to talk to you about it.
I go to a lot of conferences these days, and I frequently sit in advanced space systems sessions.
I sat in a very interesting future reusable launch vehicle session the last AIAA conference I went to.
And I walked away totally distressed because they have no concept of what the real world is like in many of these academic seminars.
And I want to inject a little bit of real world realism into what you guys think about as engineers.
One of the favorite topics of people these days that are designing advanced space systems include what they call and "austere" launch pad.
In other words, you cannot fix anything on the rocket when it is on the launch pad.
And the reason they side is look how much money all these programs spend fixing their rocket out on the launch pad.
Well, the fact of the matter is rockets are very finicky, rockets have very little margin in many of their components and things break.
They can break and you don't even know it until you plug them all together and turn them on.
And if you have a launch pad that is austere and you can do no work on it, well, what was one of those colorful phrases that Chris Kraft would use?
You have really bought the farm.
I won't use his colorful phrases.
You wind up spending a lot more money because you've got to take the rocket back to the barn and take it all apart rather than maybe get access to fix it.
If there was anything that I wish we had was more capability to fix things on the launch pad.
We spend a lot of time and money working around problems that we cannot fix at the launch pad, even though we have a huge amount of capability.
You don't save money there.
It is a false economy.
Similarly, there is a notion en vogue that you don't need a big Mission Control.
That is just a lot of people that cost a lot of money that you don't really need.
I think it reached the height of the anti Mission Control forces during the DCX program when they were very proud to proclaim their Mission Control was a trailer house with three people in it.
And they thought that was all any space vehicle needed to have to launch.
Of course, they only reached an altitude to 5000 feet and they crashed their only prototype vehicle, not because of Mission Control but perhaps because they didn't pay enough attention to operational situations.
In my business now, I can tell you that in the Space Shuttle program Mission Control cost me less than 2% of the total budget for the Space Shuttle program.
What we get for that 2% is quite amazing.
And every time the budget folks come down from Washington to review our budget, they want to help us drive down the ops cost, they take one look, you have the budget breakdown of the Shuttle program and they say you guys don't spend very much on ops.
This is not where we need to look if we want to save some money.
If you want to save some money, you need to make a reusable vehicle.
Reusable and not so maintenance intensive to turn it around.
And that may be a theme that you hear today.
I've got a couple of things I want to pass out to get started, a couple of glossy brochures.
I think there are enough of these for everyone about Mission Control.
People in the class should get first dibs at this, but hopefully there are enough to go around.
I think there are plenty of these.
And the little one is similar to the big one.
And I'm not going to spend a lot of time.
If you want to know about how Mission Control Room is laid out and who the people are that do things, it's all in this brochure.
I don't intend to go over that.
I think Chris Kraft probably gave you a good background about why do we have a Mission Control.
Why did he invent a Mission Control?
And this is kind of the nuts and bolts of Mission Control as it exists today.
Public Affairs Office is really good about putting those things together, and I don't intend to cover them.
Also because I had a whole bunch of copies of my last two papers sitting around that I needed to get rid of, I lugged them up here.
I want to give you copies of my last two papers which have to do with operational issues that you can read at your leisure.
And I particularly want you to look at the considerations in rendezvous launch windows.
Those are two different papers.
One discussing entry operational considerations and hypersonic entry that we've learned with the Space Shuttle.
The other about rendezvous launch windows.
And it will give you a little flavor for what a real operation needs.
Many people can talk to you about the physics, kind of the things that God says you must do if you are going to rendezvous in space, for example.
But there are many other considerations that you run into in the real world.
So I am going to talk today mostly about Mission Control.
Perhaps at the end we can talk a little bit about the Shuttle program.
And these I am short of so, please, people in the class only as my handout for the slides here today.
And the slides will be posted on the Web as well.
Yes.
Dr. Hoffman has got them off my electronics here.
OK, so the things I am going to talk about today basically are outlines on this page.
We have a Mission Control.
Here are some of the reasons we are going to talk about trajectory.
Planning and monitoring flight planning.
Systems engineering.
And then I've got a couple of stories from the trenches.
And then I promise you no differential equations today.
Question.
Why should you have a Mission Control?
Why is it necessary to have a Mission Control?
I have four basic reasons to have a Mission Control.
The first one is safety.
We have seven or eight astronauts flying arguably the most complex flight vehicle that has ever been invented.
They don't have enough time or enough resources to watch everything to the degree that is necessary.
We have a large group of folks on the ground in Mission Control, about 800 folks in Mission Control that are also monitoring at a very detailed level with more information than the astronauts get to make sure that everything is operating as it should be.
We provide a level of safety in Mission Control.
Secondly, we provide flexibility.
We start planning a Space Shuttle Mission typically about 18 months in advance.
The flight operations people are the flight planning people, so the Space Shuttle program, NASA headquarters will come down and say you need to fly a Space Shuttle flight that does X, services the Hubble Space Telescope, deploys the Chandra X-Ray Telescope, goes to the International Space Station and adds a module or carries supplies or what have you.
Then it is up to the planners who will become the operators to put that plan together.
How are we going to accomplish the big objectives on this flight?
It's not just enough to take this big module up and plug it into the Space Station.
You've got to wire it up.
You've got know when to do which wire first and what switch to throw in what order or you will fry the circuits.
It's a very complicated business.
It is really complicated when you have to deal with scientists.
So, when we fly scientific payloads, we deal with folks that are not operationally minded.
They may have been working on their experiment for, oh, how long did it take to get a doctorate degree, 15, 20 years in some cases.
They have been working on this experiment for many years.
They are going to get the maximum amount of data they can, but they are not aware of the fact that there are four other payloads onboard and four other competing folks that want to get their science data in.
We always have a discussion about how many person hours of astronaut time can we devote to an experiment?
How many kilowatt hours of power can we devote to an experiment?
On and on and on.
And it makes these researchers do some really tough sole-searching.
And the saddest thing that I have ever seen is an experiment, and there has been more than one of them, where they flipped the switch, it blew the fuse, it was down for the count and lost all objectives with the first switch throw.
And then the scientists would come running and in and say if you'll just let the astronauts take the back panel off and rewire this circuit it will work.
And we don't have time to do that on a spaceflight.
Or, if we did have time to do it, we would take that time away from experimenter number two or experimenter number three or all of the above.
In operations, there is a lot of work to set priorities, set timelines, make sure you do things in the right order, and then train the operators and the crew.
Now, once you do that planning for 18 months you understand the priorities, the pros and cons, the logic behind all of those trades.
And so, when you go into flight and something happens, as it always does, we can re-plan it on the fly.
And we do, almost every flight or the vast majority of flights, take the flight plan that we published pre-flight, put it gently in the recycle bin and develop a new flight plan that can be drastically different.
And we're able to achieve a great degree of success in our operation of the objectives that we set out to do because we have the people that can, in real-time while we're flying the flight, in this short period of less than two weeks overnight re-plan and come back with a new plan to accomplish the objectives.
Yes sir.
Wayne, I was one of those scientists who didn't understand about the realities of spaceflight.
And I liked your choice of the words "you would discuss with us".
Dictate would be the key word.
Indeed.
The question I wanted to address to you is this business of taking the flight pad, putting it in the wastebasket and then generating a new one.
In the seven Shuttle flights that I would have investigator status on, they all had, as their basic philosophy, a return to baseline.
If something went wrong, get back to your baseline plan as quickly as you could, even if you knew that it was non-optimal.
Could you comment on that?
I would say, from my perspective, quite the opposite.
Yes, there is a baseline plan.
And there were very good reasons for that baseline plan.
But the flight control team is always looking for the optimum solution.
So I would say my perspective may be a little bit different from yours, but that has been my experience.
We are always trying to achieve the maximum number of objectives.
So flexibility from Mission Control is the second big advantage.
The third big advantage is let the crew focus on those things that people can only do in space.
They have access to vacuum.
They've got zero gravity.
They've got the high vantage point to do earth observations, what have you.
You cannot do those things on earth.
That is why we put people in space.
And why would we make them calculate the gas budget for deorbit burn which can be done by some bachelor of science engineer with an accounting minor on the ground?
So we let the crews focus on those things that you can only do in space and we do every job that we can on the ground.
Somebody calculated for me one time, and I almost hate to give the statistic, that it is $30,000 a minute for people in space.
You can hire a lot of engineers to offload that amount of time.
And, finally, there are some jobs that you can only do on the ground.
The NOAA weather satellites do not directly uplink to the Shuttle so the astronauts don't know anything about weather, other than the little part of the world they can see out their window.
If you want to forecast where you land, that's a job that you can only do on the ground.
Obviously, radar tracking there are hundreds of things that you can only do on the ground.
And I would say interact with management and review the priorities is one of those things that can only be done on the ground.
Here is my first cartoon for the day that compares airline travel to space travel.
And I think this is a telling cartoon because it is entirely true.
I flew in last night on probably a 12 or 15 year old 737.
It was a nice plane but a little shabby.
And we got bumped around pretty good coming into Boston.
And I am thinking all the time about I wonder if they have maintained this aircraft well.
You can pick up the papers.
And they do a very good job in the airlines, but the fact of the matter is much of the airline industry is involved with passengers and customer relations and baggage handling and all those good things.
And the maintainers and the safety people are few by comparison.
In comparison with the Shuttle, we don't have very many passengers to deal with.
Generally, they are cooperative.
Sometimes they get surly.
But we don't have very many to deal with.
And everybody in the program is a safety worker.
And that is something we continuously exercise.
To turn a Space Shuttle orbiter around from one flight to the next takes 500,000 man hours of maintenance.
There is a gem for you.
People have been working for 30 years to decrease the amount of time that it takes to turn the Space Shuttle orbiter around.
It is a reusable vehicle, but it is a reusable vehicle built with very small margins with a lot of complicated technology.
And it takes a lot of maintenance.
Some people think we would have been better launching expendable rockets.
The same amount of time, actually, a little more is involved in launching Titan IVs which have the same payload throw weight as the orbiter does to low earth orbit.
The new launch vehicles Atlas V, Delta IV and its variance are supposed to be launched with a whole lot less.
However, they do not have the payload capacity to orbit.
And, in fact, they have not achieved their goals of man hours to a launch that they have set out to yet.
Although, they are early yet.
We've got to give them time to work on it.
But, the fact of the matter, getting into space, whether you doing with an Ariane, a Soyuz, an H2 or an Atlas V, Delta IV or the Space Shuttle remains a very expensive process which I think the public doesn't quite understand.
I would like to talk about some of the things that Mission Control does to get ready for a flight and then how they execute it in-flight.
And first among those is trajectory control.
Who in here is an orbital mechanics wizard?
Good.
OK. We will have some equations for you in just a minute.
Here is the first one.
Would you like to step up and explain?
OK.
This is all involved in orbital mechanics.
This is how do you get to orbit?
We have the flight performance reserve of main propulsion system propellant.
We load a half a million gallons of propellant.
That is over 2.5 million pounds of liquid hydrogen and liquid oxygen in the external tank for the Shuttle.
How much do we have for a normally planned mission at the end of when you achieve orbital insertion?
Anybody want to guess?
Remember, every pound of propellant that you leave in the tank and throw away into the ocean is a pound of payload that you could have carried to orbit.
So this is not a trivial process.
Our goal is about 900 pounds.
And you are going to see how we do.
But this is a little plot that we've developed based on mixture ratio of the engines.
Now, the Space Shuttle main engines have what we call an overboard mixture ratio of 6.039 nominally.
We test fire them at the Stennis Space Center, every engine for every flight, to check how they operate mixture ratio because mixture ratio and ISP are critical.
When we updated the Space Shuttle main engines to the block two engines they are vastly safer than the original engines in the Shuttle because they operate at lower pressures, lower temperatures and their rotating turbo machinery operates considerably slower RPM.
However, we gave us a second and a half of ISP.
And that is huge in this business, a second and a half of ISP.
This plot shows, as a function of mixture ratio, if you have any shift in mixture ratio in flight, which we have, I'm going to talk a little bit about that, you will change the amount of residual remaining.
Here the little blue dots are the normal flight performance reserve.
That is how much gas you've got left in the tank when you get to orbit.
And you can see our goal here is about 2500 pounds.
Of that about 900 pounds is fuel residual based on this curve right here.
Now, you may ask why do you want to have an extra amount of hydrogen fuel in the tank over oxygen, what is the difference?
The engines are designed, the cutoff hydrogen rich.
We don't operate at the stoichiometric chemical ratio that you would combine hydrogen and oxygen to make water.
We operated at about a mixture ratio of six instead of a mixture ratio that would be 18, I think.
Did I do my chemistry right?
It's been a while.
We operate considerably off the stoichiometric ratio.
As you get mixture ratios that approach the stoichiometric mixture ratio your fire burns a lot hotter.
You get a lot more heat out of that fire.
Metallurgically, the engines cannot stand those higher temperatures.
We already operate at 6000 degrees Fahrenheit.
That is tough.
You run out of fuel and cut off oxygen rich, those temperatures will go out of site in the engine turbo pumps.
So we want to cut off fuel rich.
We biased ourselves so we cut off fuel rich, but it does not take very much of a mixture ratio shift to get you to cut of LOX rich.
In fact, as mixture ratio crosses the knee of this curve, you wind up leaving actually a lot of LOX in the system cutting off on the fuel side.
So we have to carefully monitor how the engines come in.
And, when we talk about overboard mixture ratio, that is not just what the engines operate at, but it's whatever losses you have in the system.
As you bleed hot hydrogen or hot oxygen off the engines to repressurize the tank, that becomes a loss and that affects mixture ratio.
One of the real problems we're dealing with is, as we develop what we call engine tags down at the Stennis Space Center, we have found an interesting phenomenon.
In flight, we pressurize the tanks with their native constituent, the oxygen tanks pressurized with oxygen and the hydrogen tanks pressurized with hydrogen.
At Stennis, where we do the engine tests and develop the characteristics of each engine, and every engine is just a little bit different, they pressurize their tanks with nitrogen for safety reasons.
The nitrogen can diffuse into your hydrogen, but more importantly it can diffuse into your oxygen system.
And, therefore, you have a less pure propellant and it drives our mixture ratio off as a function of having impurities in the propellant.
So we get a tag of what we think the mixture ratio is of the engine.
That is different than what we get when we go fly.
And, in fact, the last flights, one of the last meetings I had before I came down here yesterday was a meeting with our flight analyst who have an unexplained missing 300 pounds of hydrogen every flight for the last 10 or 12 flights.
And we are beginning to converge on the idea that it is this nitrogen impurity in Stennis that gives us a false mixture ratio which we design our flights around.
Let's talk a little bit about it.
Here is a little schematic.
The external tanks really have two tanks, the oxygen tank on top and the hydrogen tank on bottom.
When we make it to orbit, how much is remaining in each tank?
Well, the answer to that question is none.
What we are running on at orbit insertion is what is left in the pipe.
And it is really not orbit insertion but close enough to talk about it.
So we have 17 inch diameter pipe coming down the side of the hydrogen tank from the oxygen tank up above.
It goes across into the engine compartment of the orbiter.
And this shows you where several of the early flights were at main engine cutoff, guided cutoff where we wanted to be from an orbital mechanics standpoint, altitude velocity, flight path angle and so forth.
That is where we were.
That is how much was remaining in the tank.
None in the tank.
That is how much is remaining in the line.
That is how close you have to cut it for spaceflight.
I have another presentation, and if we have time I might show you a few charts out of that, that talks about the difference between commercial aviation and spaceflight.
We operate on much smaller margins.
And every pound you see in that far part of the table that talks about the residual remaining, those number of pounds, and, by the way, we're antiques.
We use English tradition units in the Space Shuttle program.
But every pound residual that you have in that far column is a pound that you could have taken to orbit of payload but did not.
You threw that away.
Yes.
How do you measure that mass?
Very good question.
They look at pressure hit.
We have pressure transducers down in the lines as they approach the engine and you're accelerating at 3G.
We have an inertial measurement unit that measures velocity, and we control the engine throttle setting to 3Gs as we approach MECO.
And so you look at the pressure and that determines, based on a very simple head calculation, where you are in that standby coming down the outside.
The fuel side is a little bit more complicated.
They have a 5% level sensor in the tank that tells you when you're at 5% in the tank.
And then we have fuel flow meters on the hydrogen side, and they calculate flow rate minus from the 5% level down to when a main engine is cut off.
So those are the ways we can do residual.
And that is not a simple exercise.
It is vastly more complicated than the simple explanation I just gave you because the engines are throttling continuously.
Now, I wanted to talk a little bit, if I can move from normal mission planning to abort mission planning.
Normal mission planning is something we do upfront and then we monitor during flight.
And one of the stories at the very end of this presentation is about a day when things did not go right in the engine world and what we did about it.
This is going to talk about abort modes.
Now, the Shuttle is very interesting.
Human spaceflight is different than expendable spaceflight because we would really like to get the people back.
And expendable spaceflight, your basic principle is if one of the rocket engines quits early you are done.
The payload goes in the water.
Or, if you launch from [NOISE OBSCURES] it may go into Mongolia.
But you don't get it back.
There are no overs with very few exceptions.
If you have a problem in your main propulsion system on an expendable rocket it is time to talk to your insurance company because you are done.
Even very minor problems can strand satellites in lower orbits than they have useful life.
It is very important that the propulsion system work well.
But the Shuttle was designed with the thought that one of the three main engines could shut down early.
We have safety monitoring on those engines to make sure that they would shut down if something goes wrong in a contained, that is to say they don't blow up, manner.
And then you have to have the capability to get the crew back, much like an airliner.
If you fly on an airliner or any kind of multi-engine aircraft, one of the things that they must be certified for is engine out operations.
Lose an engine, pass the commitment point on takeoff and still be able to takeoff safely and return to the airport.
That is a fundamental principle in aviation safety for a multi-engine aircraft.
One of the Shuttle design goals is very similar.
Lose one of the rocket engines, have it shut down prematurely and still be able to land safely somewhere.
Notice I said one engine.
Not two engines.
Not three engines.
One engine.
That is a huge technological leap over expendable rockets.
A huge technological leap.
And we pay the price in terms of performance.
Here is a little bit of the different abort modes.
If you have an engine out somewhere between launch and about four minutes into flight, you have to return to the launch site which involves turning around and flying backwards at mach 6 through your own rocket plume, which is everybody's favorite thing to want to avoid doing.
A little bit later than that we pick up the capability to do transatlantic abort.
Sometimes we call the transoceanic aborts, but the acronym really is transatlantic abort, TAL.
And land somewhere in Europe or Africa.
And somewhat further along you can abort to orbit.
You can actually dump your secondary propellant and wind up in a lower orbit.
And then you may orbit once around or you may come back on the third orbit.
Or you may be able, in fact, to fly nearly a normal mission depending on how lucky you got for the time that the engine was out and how you were doing with reserves.
Here is a little bit of different graphic, a cartoon of the same thing.
I don't like it too much because RTLS actually turns the other way.
But, as you can see, we would try to return to the Kennedy Space Center which has possibly the worst weather in the entire western hemisphere or continue on around or up on end to orbit.
And you have to always be concerned about external tank disposal.
This was the abort regions chart generated pre-flight for our last flight, STS-114.
We have three TAL sites, transatlantic intact TAL sites, one at the Zaragoza Air Base in North Spain, one at Moron Air Base in South Spain near Seville and one at Le Tube which is the French Air Force Flight Test Facility near Marseilles.
And then we have what we call "Press to ATO".
Press to abort which involves a dump, involves changing your inclination of your orbit to a lower inclination if you invoke this.
You cannot, for example, ever rendezvous with the Space Station because you won't wind up in the right orbital inclination, even though you may make the altitude.
And then we have what we call "Press to MECO" which involves just kind of closing your eyes and riding it out and see what happens as you get close to your guided MECO.
So we build these charts in advance.
And then we have a very sophisticated computer program in Mission Control that monitors the engines the entire time during launch called the "abort region determinator" that can emulate a single engine out, two engines out, three engines out or a vacuum impact point.
And a flight dynamics officer is responsible for making those calls.
The crew does not make these calls.
The crew has a crew card very similar to this onboard so that if they lose communication with the ground and something bad happens they might have a chance at pulling it off.
But it is based on everything more or less performing as planned.
If anything is performing off normal than these charts are no good.
That is hosted in Mission Control.
And until some day we put a supercomputer onboard the space vehicle it will continue to be hosted in Mission Control.
Do you have a question?
Yes.
That is a critical real-time call that is made in a hurry.
It is not always decision aid.
Can you comment a little bit on the reliance upon the automatic decision aid computer that is grinding this out and the judgment of flight control?
Well, first of all, you have made a false distinction because the people that wrote the computer code are the flight controllers.
They understand the logic that went into it with practice probably three days a week in Mission Control exercising simulated missions that have engine problems so you practice that.
It's not an either or.
It's a symbiotic relationship.
The abort region determinator doesn't work without operators.
Operators only have the cue cards to go on if the abort region determinator software quits.
You cannot get by without the other.
There is some judgment involved.
And a good flight dynamics officer with these charts, with no [UNCLEAR] can make some judgment calls based on trends and performance.
But we need both.
You cannot have one or the other.
It is very sophisticated.
It is very subtle.
And very small changes can make this software, this predictor work.
It is probably the best example of an expert system that I know of, but it takes a great deal of care and feeding.
And it is very, very sensitive to the inputs.
So it does take a lot of judgment and experience to interpret.
Along with that, we need to talk about where you land.
Because here are all the places.
We kind of closed down Banjul and the Gambia because we don't fly that inclination anymore.
We do fly at the International Space Station Missions, and we have the TAL sites I talked about.
We could fly as high as 57 degrees inclination, which we have done in the past, but that is outside the range of what we need to do for the Space Station.
The interesting thing is we get into the weather story and the weather forecasting, and deciding which landing site you can go to is pretty complicated.
I never studied meteorology in college.
It is another subject I wish I had.
That is not all.
We talked about intact abort landings if one engine quits.
But being good flight planners and controllers you always take it to the next level.
What if two engines quit?
What if all three engines quit?
And so we have made agreements all the way up the East Coast of the United States with the Canadians to have places to land.
And, again, we have agreements with all of these landing sites.
We send people to train them about what would happen if the Shuttle would land there.
We check the weather.
Across the Atlantic we have landing sites.
Again, Shannon, Ireland was probably the first dry patch of land that you can come to on the far side of the Atlantic Ocean.
Typical weather conditions there are 500 feet overcast and rain and fog so we don't really probably want to use that.
England is a little better.
Cologne Bonn is a little better.
And then we have all of these landing sites that we can look at.
And that keeps the State Department busy for us because we have to have international agreements to use any of these places.
It took us two years to negotiate with the French government for emergency use only of their landing site.
We typically don't have people at these landing sites other than our intact TAL sites, Moron, Zaragoza and Istres-Le Tube.
But they are there.
We would not hold the launch up if one of the landing sites for two or three engines out was not available, but we would if we did not have an intact landing site which is the one engine out requirement.
So we have requirements difference.
One of the really fun things is when we get into the summer months we frequently cannot use some of these airports because they have air shows going on.
And they would really like us to land the Shuttle there during an air show, but we don't.
One of the things that we have to watch very carefully is where do we get rid of the external tank?
Now, we can carry the external tank to orbit.
There have been some very colorful viewgraph presentations put together of using the external tank as parts for a Space Station.
Totally science fiction.
Operationally it would be extremely difficult to do.
Carrying the tank to orbit would be very easy, but we stopped just short of orbital velocity so that we can control the disposal of the tank.
We now have international treaties that say you shouldn't put things in orbit without being able to put them in a place where they don't hurt people.
We don't like to have satellites reenter and drop radioactive garbage or even just big hunks of metal on top of people, so we have the external tank that we separate and drop just short of orbital velocity and use the small orbital maneuvering engines on the Shuttle to provide the last couple hundred feet a second to get to orbit.
And we have to plan this very carefully.
We've done a lot of studies to find out when that tank will rupture, how big the pieces are, how far they've come apart.
And we actually have flight rules talking about ET disposal.
This is a serious subject.
Everybody that flies an expendable rocket has to know where their stages are going to fall, except maybe the Russians who just drop them in Kazak or used to.
I grew up in New Mexico in the 1970s.
We were very excited about the thought that White Sands might be the launch site for the Shuttle when it was under early design phase.
When they picked solid rocket boosters that fell off just a couple hundred miles downrange, we knew that White Sands was out of the running because that would put people at-risk in the continental United States.
And those kinds of decisions make for launch site decisions.
Yes sir.
[AUDIENCE QUESTION] We're talking a little less than an hour.
Probably about 45 minutes from managing cutoff to the rupture at about, let me see, let me make sure I say that right, 122 kilometers.
I don't operate in kilometers.
I have to do the math, but it's not long.
It's less than one orbit.
One orbit taken 90 minutes.
It's about halfway to two-thirds of the way around the world that we get rid of the tank.
And this is a very important subject.
In fact, US laws and the Eastern Range that we have to work with to get launch clearance issues what we call "Notice to Airmen" or "ET disposal" about 48 hours prior to the Shuttle launch to clear the ocean area and the air space.
And this is all based on what we call a normal main engine cutoff.
We made our guided cutoff to orbit.
And here is a little picture of a problem we had earlier I want to talk about.
Here are the ET disposal lines for the different inclinations.
You can see that some of these get kind of close to the West Coast of Mexico, but most of them fall out in the Pacific Ocean.
And, in particular, here are the Space Station lines for disposal of the external tank.
And, as we go through the five minute launch window, the inclination of the orbit -- Not the inclination but the steering the Shuttle has got to do to reach the Internal Space Station changes the place where these tanks are disposed of.
I think I have a better picture.
And, in particular, we had a problem with this little island right here that we were infringing on the internationally recognized 200 nautical mile limit for pieces.
Now, if we do a normal insertion, you will see that we have these typical nominal footprints based on where we were in the launch window.
That is where all the pieces will fall.
But we have dispersions based on trajectory and other dispersions that say we've got to clear to a 99.7% probability this whole area, or this is the area that results from all the dispersions at that level.
So we don't want to drop pieces on people.
Once upon a time, the United States Air Force thought that Pitt Island down here was a bird sanctuary and nobody lived there.
And they sent out the Notice to Airmen to clear the area, and we found out there are a couple of hundred people that live on this little island and the Australian government got kind of bent out of shape over that.
We had to negotiate infringing by just a few miles on the 200 nautical mile circle around these guys just at the toe of the footprint, and that went back to the French government.
And that took us two years to negotiate.
So they don't teach you everything you need to know in engineering school like negotiating with the French
[BEGINS SPEAKING WITHOUT MICROPHONE] ...disposal probability to your average diplomat, like you say, we don't get training for that.
Way beyond.
Now, here is my funny story number one.
Dick Richards, who was program manager before me, was asked to go on speaking engagement to American Samoa.
Now, I've been asked to speak a lot of places and they are always like Boston or, you know, OK, this is a nice place to come, but think about a speaking engagement in American Samoa.
So he had to go to American Samoa to speak.
And, as they got ready to leave, the airline pilot came on and said we cannot take off because they are going to launch the Shuttle today.
He had seen these tanks come in.
He was not about to violate that airspace that they have been notified.
And the launch time would put him past the point of no return where they had to continue onto Hawaii and could not turn back to American Samoa, so he wasn't going to take off until he got word that the Shuttle had either launched or scrubbed for the day.
And there was a delay.
And so Dick Richards got on his cell phone and called to Houston and found out that we were sitting on the ground in Florida based on bad weather at the TAL site at Banjul and the Gambia.
The passengers started talking about this.
And they said you mean here we are in American Samoa waiting on a Shuttle launch from Florida that has been delayed because of bad weather in West Africa?
The answer is yes.
So, what you do has an impact all around the world.
OK, enough about trajectory.
And I have quite a lot I could talk about.
Let's talk about flight planning.
One of the interesting things that we do onboard the Space Shuttle is we carry a lot of laptop.
At last count, I think we carried about 11 laptops on the last flight.
But we don't rely on those laptops because they keep breaking.
What we rely on is paper.
Spacelab flights are worse.
I'd say the typical Space Shuttle flight that launches carries 80 pounds of paper checklists.
Every one of those pages lovingly crafted.
We got maps and charts and we got all kinds of little cue cards that stick up on Velcro all around the cockpit, but by and large they come in books, and here are some of the books.
And Mission Control folks build those books, understand those books, can modify those books in real-time, and frequently do.
One of the things we have to watch is printer paper onboard because we cannot run out of printer paper when we update checklists.
And so we have to monitor how much paper we are using as we print out changes to the checklists onboard.
Yes sir.
[AUDIENCE QUESTION] In so far as paper can be made low flammable.
It is but it is paper and it will burn.
The Shuttle cockpit is full of flammable stuff.
But it is tried to kept under control.
It is tried to be protected.
And, of course, we don't operate in a pure oxygen atmosphere as Apollo did.
It is very much like earth normal atmosphere.
So, no, it is not fire proof.
I wish it was.
When we start a flight, we start out with what we call a flight requirement.
Here is the flight definition requirement document.
Here is the flight we just flew and the basic parameters that we've got.
Which orbiter are we going to fly?
The next flight up we're going to fly Atlantis.
Which external tank is TBD?
Because we're still arguing about that.
Which solid rocket booster pair, so on and so forth?
Which main engine by serial number?
Which flight software release?
How many cryogenic tanks do we have the arm onboard and all this stuff?
What is the manifest?
Well, it is a station manifest for the utilization flight 1.1 which has got an MPLM.
It used to be called the Miniature Payload Logistics Module.
Now it is the Multi-Purpose.
That transmigration of acronyms over time have become more politically correct as a subject for a doctoral thesis, I think.
Which launch pad we're going to operate out of?
And how many days are we going to fly?
How many people?
We're going to carry seven up, six down, land at KSC and a bunch of remarks.
This is kind of the basic definition.
This is the starting point for a flight.
This is what we get to start.
And then we have to flush those out.
We build a word document and we talk about the requirements for a flight.
This is done at the program level.
What is the launch window?
What is the launch period?
And many of these things are just flushing out the basic requirements you saw in the chart.
Here is an interesting one, part of that flight requirements document that talks about EVA spacewalks and talks about what we're going to do.
And you will notice, in great bureaucratic language, the purposes of the EVAs are defined in another document that you have to go off and look at.
So, they're not any flight requirements documents but at least you got a reference where to go.
And here are some of the things we are going to be able to do on this flight.
And, in fact, we're going to have some contingency or emergency EVAs that can do these emergency things, which we hope we don't have to do but we are going to be prepared to do.
From that we develop flight rules and flight plans.
Here is a page out of the flight rules book that the ops people built.
This is discussing how we take pictures of the external tank.
You saw those beautiful pictures just a few minutes ago of the tank taken by the Shuttle.
And here are the guidelines.
We're going to have to do a thrust from the plus X jets, but we won't do it if we've got problems in the propellant system, if we are way under speed, if we're going to be dark because we don't have a flash that big, and so on and so forth.
And then that's the automated umbilical well camera.
And then we've got the handheld photography.
And we've got a whole bunch of reasons why we wouldn't do that.
If we had to go to a backup software, we don't want to do that because it makes it too sporty, just all kinds of reasons.
Bob's people are thinking about this all the time.
And, in fact, it's not enough to have the rule.
Here is the rationale behind that rule.
Why is it we decided you would or would not take those pictures which are very important to us for post-flight analysis on a very detailed level?
Anything you see in italics font is what we call flight rule rationale which is the subject of interminable meetings and lots of haranguing and wrangling.
Here is a little bit of the inspect rule.
We all know that the thermal protection system on the Shuttle is not as robust as we would like it to be.
We are going to inspect it every time.
Here are the priorities that are planned before flight for thermal inspection system.
And we start with the most important and we work to the least important.
Your goal is to do them all but your plan is to have a plan ready if you run out of time.
And then, again, the backup of why did we say that.
And, again, it is more inspection priorities.
I am not going to go over all of these.
You can look at them.
I want to come back to EVAs for a minute.
Here are some of the EVA task objectives.
This is something that they did just the other day onboard the International Space Station.
This happens to be an International Space Station rule, but it is closely related to the Shuttle rules.
They put in a new camera group.
All this is perfectly clear to you guys, all these acronyms.
It takes about six weeks to learn the language when you come to work at NASA.
And then we have something on the Station called the floating point potential measurement device which is broken and became junk and they threw it overboard just a couple of days ago.
Well, you get through all the rules and you come to a flight plan.
And here is the flight plan.
This is from the last flight.
Eileen Collins, here is what she is doing this particular day.
This is flight day number three.
This is what the pilot is doing.
Some time I will have to tell you the difference between pilot astronauts and mission specialists.
Dr. Hoffman has his PhD.
He well versed in many things.
And the pilot, Jim Kelly, one of my favorite guys, I like him a lot, flies jet planes so he gets post-sleep exercise.
And so we have all the guys.
And what they are doing by the hour of the day, and this is the overview, by the way, but these guys are going out on a spacewalk.
And so here is their EVA preparation, purging their suits, getting setup, going out the door.
Here is the EVA which are going to be outside for 6.5 hours.
And this particular day they're doing the thermal protection system repair detailed test objective which you saw in the rules as one of the priorities.
So, we build this level of flight plan.
Now, this is not it.
We go to the minute-by-minute level, detailed checklist, this is the overview.
This is kind of OK, you've got to get up and brush your teeth and make breakfast and then go do some work kind of thing.
And then we have all of our other procedure and checklist to make sure everything goes as perfectly as it possibly can.
And this is just A day.
And then we get all kinds of good little information down here.
One of which is when we're going to be able to communicate with the astronauts.
There was a revolution in flight control when we launched the Tracking and Data Relay Satellites.
Before we had Tracking and Data Relay Satellites and you just had ground stations to communicate with the crew, we could talk to the crew about 15% of the time.
85% of the time they were out of communications with the earth.
Now, if you were going to the Moon and you used a Deep Space Network, you pretty much had continuous comm.
But if you were doing Shuttle or any other low-earth orbit kind of thing you could only talk to the crew for about 15% of the time.
And sometimes it lined up that you would talk to them three or four minutes every hour and a half.
The Russians have a huge problem in that they lost all of their tracking stations outside of geographical Russia.
And so they talk to their crew and plan their crew day on those parts of the orbit where they come over geographical Russia.
And they are a slave to that.
And we launched the Tracking and Data Relay Satellite.
Now we can talk to the crew all the time.
There is virtually no time you cannot talk to the crew.
The crew gets really tired of that.
They want to shut the radio off because Mission Control is always calling up and yammering at them.
But it is a revolution because now we can get continuous data, communications and command with the crew from the ground.
And the bottom line on that is we're much more efficient.
We don't have to wait for critical activities to happen until we get that communication link.
Yes sir.
[AUDIENCE QUESTION] Mars there are going to be four to 20 minute time delays in communications one way.
Yes.
What are the preliminary plans for dealing with that?
I don't know.
No, I'm sorry.
I'm serious.
A lot of people are really worried about that.
It will be a different way to operate.
And the folks that are working on those programs have got a real challenge.
Two quick questions.
One is if you time the checklist, I mean do you know how long page three is going to take?
Absolutely.
And the other thing is do you have backup flight plans for every contingency pretty much where you know if this experiment fails, per se, we will pull out this flight plan and it reschedules everybody?
No.
In fact, what we do is we have some emergency checklists.
If you develop a leak in a cabin, OK, here is the emergency drill checklist to get you back on the ground as soon as possible.
But the flexibility that I talked about in Mission Control is no, we don't build complete different timelines.
Say if experiment A doesn't work we have a different timeline.
What we do is we have the smart people in Mission Control that can take the existing timeline and build a new one.
And, in fact, they do almost every night.
I call it night.
That is when the crew is asleep.
It could be daytime in Houston but it is when the crew is asleep.
The last thing I want to talk about in a flight preparation.
Yes.
If the crew fall asleep at the same time, isn't that not a bad idea in case something goes wrong?
Shouldn't someone be keeping watch?
Mission Control is watching, A, number one.
And, B, number two, there are enough automatic sensing.
The computer is watching the really vital things so that if you've got a link in the cabin, if you've got fire, something like that happens, the alarms will go off and wake the crew off.
One of the case studies I am going to show is when Mission Control didn't get to watch, so look for that in a minute.
The post-sleep period?
Ask Jeff.
The post-sleep period is what do you do after your alarm clock goes off?
There is cleanup, shave, get dressed and make breakfast, which is a huge deal.
This is not like taking something out of the freezer, throw it in the microwave and eat it.
It's a huge deal to cook and cleanup onboard.
Personal hygiene time.
It's also time that most crews spend reading the morning mail.
They don't get a newspaper but they get reams and reams of paper off the printer from Mission Control.
You thought you were going to do an EVA today, but here is what we're really going to do.
It's a really important time, getting ready for the day, so we allow the crews about an hour and a half of post-sleep activities.
And that is everything to get to work.
Yes.
How do you decide what to tell the crew and who makes that decision?
Well, that's an interesting discussion.
The flight director, of course, is in overall charge.
I should say that there are three shifts of flight controllers in Mission Control, who are not ironmen, but that work about 9 hours.
There is an hour handover period, and so three shifts a day cover the day.
A flight director is in charge of the team.
The flight planners, big flight activities officer we call them, obviously have a big part of this.
On a day like this when it's an EVA day the EVA officers have a big part of this.
The CAPCOM, capsule communicator which is held over from Mercury days, is an astronaut.
And hopefully a flown astronaut.
Not always but typically a flown astronaut that is the crew's representative in Mission Control.
And the CAPCOM is a very valuable resource in saying here are things the crew would want to know, here are some things the crew already knows so we don't have to tell them.
Here you've built a plan that is going to overwork the crew or here you've built a plan that has got people sitting around with a lot of white space not doing anything getting bored.
So, it becomes a team effort.
Finally, the flight director approves all uplink messages.
Yes.
[AUDIENCE] ...when do you tell the crew about that and who decides?
Well, again, it is a team so it's not this military hierarchy that orders come from the top.
There is a team.
The team will develop here is what is happening, here is what we think we're going to do.
The stories come together enough and here is the judgment factor.
The flight director will tell the CAPCOM to tell the crew in basic terms what the crew should know.
Now, we have a lot of ways to communicate with the crew.
And the Shuttle we are very fixed, obsessed with doing things on an open air to ground because in the old days they did some things kind of pine that got came out in press conferences that were bad.
The station crew has got this IP phone, Internet Protocol phone and they can call you.
Like right now if they had your phone number.
They can just call you and talk on the phone, and nobody knows what's going on.
We have email with the crew.
And a lot of things go up on email, but all of those communication paths are utilized to get information back and forth to the crew.
I probably ought to tell you one little short story.
This is my sports analogy story so you will have to put up with that.
Working in Mission Control is like working in different sports.
If you are the ascent entry shuttle team it is like playing basketball full court press all the time.
You are always running short time but very intense.
If you are a shuttle on orbit it is kind of like American football.
You huddle up, figure out what play you're going to do, go out, execute the play, it worked or it didn't work, you come back, huddle up.
It is very episodic but it can happen fast when it is happening, when you're executing it.
The Space Station is baseball.
It's a very much different game.
And one of the hard things as you move as a flight control from Shuttle ascent to Shuttle orbit to Space Station, you have to accommodate the different rhythms of the game.
The Russians have a phrase for when the Shuttle comes to visit the Space Station.
They say the hurricane came through.
Because they have this nice orderly regime and these folks come and fill the place up and they're doing stuff and they're throwing things here and there and then they leave.
And then we have to sort it all out and go back to our normal kind of placid existence.
The last thing about flight planning is we talk about decision making.
Here is the simplified diagram for the Chandra X-Ray Telescope launch decision.
AXAF, as you might know, is the Advanced X-Ray Astrophysics Facility which got renamed Chandra Telescope.
And we have the ascent flight direction, we have a mission director, we have the ops director, we have the launch director, we've got the KSC people and all these people.
Then the prelaunch timeframe are all watching their parts of this.
They're watching the telescope.
They're watching the upper stage.
They're watching the Shuttle.
They're watching a launch complex.
Somebody has got to decide and they all have got to talk to each other.
And they are all, by the way, geographically in different places.
And you've got to know who is responsible for what and who has authority to give information for what.
Because you don't want somebody in the AXAF control center saying to the NASA test director your shuttle doesn't look right on TV.
He doesn't know that his TV is out of adjustment or something.
The plan to get all these people together, what numbers they call, what they loops they talk on has got to all be worked through.
That is a real operational nightmare when you come to decision-making and communications.
We have literally thousands of people involved on a launch decision, a payload deploy decision and looking at their piece part.
It has to be clearly defined what you're looking at, what you're going to do if it doesn't look right, and so you have to spend a conservable amount of time planning that, training for it and then executing.
And I think we ought to break here if we're going to take a two-minute break.
Thanks, Jeff.
I hope this is helpful to you guys.
It is not exactly the standard academic fare.
I want to talk a little bit about systems engineering because one of the things we pride ourselves on in an operations community is that we are systems engineers.
We're not mechanical, aero, what have you.
You're a systems engineer and you've got to know a little bit about a lot.
One of my favorite authors is Robert Heinlein.
You may have heard of him.
And his quotation about what a human being is I think is particularly appropriate to us.
And somehow this didn't come off on the page, but the bottom line is he comes out with specialization is for insects.
I'm not going to go over the Shuttle.
You guys kind of know what it looks like and why it got there.
I was looking at the syllabus of the class thinking I should have been in this class.
Structurally, it is a complex vehicle.
Inordinately complex in my way of thinking.
But that is because of requiring wings for the cross range, large aero surfaces for the aerodynamics and, of course, being able to transform from a rocket to an on-orbit spacecraft to a hypersonic glider is not easy.
The main landing gear and the tires is a subject that I could talk about for a long time.
And, in the interest of time, I took it out of my presentation.
But if I ever get a chance to come back -- I think you probably had a discussion of that, but operationally they are a bear.
Al Louviere actually gave a nice talk.
Good.
We are the only folks that operate tires in that regime, other than the Concorde.
And you guys know what happened to the Concorde.
And we have real constraints on our tires and fret about them all the time.
Here is a list of the Space Shuttle systems that we divide up into.
One of my favorites is right here in the middle.
Food.
But it goes everywhere from the main propulsion system onto the waste management system.
Isn't that a great NASA acronym?
This, I think, is an erroneously named chart because it says Space Shuttle Systems.
This ought to be Orbiter Systems.
There are many other systems in the main engines and so forth.
One I would like to talk just a little bit about is the environmental life control system.
I think you probably had a little talk about that before.
Have you had an introduction to the environmental system before?
I think it is a wonderful system.
Again, it is wonderfully complex.
We have water in the crew compartment so that if it leaks it is not hazardous.
We have Freon out in a payload bay so it won't freeze when it gets very cold.
And we have this ammonia stuff for the last part of entry which is a pain in the butt but it is what it takes to get there.
As you know the story, basically we take oxygen and hydrogen in the fuel cells and make electricity.
And, as a happy byproduct, we make water.
My case study here is going to be what do we do with the water?
These days, most of the time, we give the water to the International Space Station where they can electrolyze it using electrical energy from the solar rays and turn it back into oxygen, dump the hydrogen overboard and breath the oxygen.
We probably made this stuff by electrolyzing seawater so it is going this back and forth, water to electricity to components.
I wanted to talk a little bit about what we do with this water.
Some of the water is used as coolant through the flash evaporator system.
It is very important to us but we typically run in excess, and so we've got to get rid of it which becomes a waste management problem.
If the crew doesn't drink it, we have to dump it overboard.
As I say, we've tied into that system.
And typically we've used the very pure water and give it to the International Space Station these days.
I am sorry the print is kind of small.
But right here is the dump valve.
And one of these is the waste and one of them is the supply water dump valve where we dump that water out.
Now, you'll notice something very interesting about that dump valve.
We have a window in a hatch and we've got windows up here and we've got a TV camera here and a TV camera here and nobody can see what is going on.
Nobody can see what is going on coming out of that dump valve.
The only way to see is to take the arm out and put it in a very sensitive position to look at what happened.
In the old days they used to teach drafting.
I always liked to look at these.
The computer did not do this drawing.
This is a work of art.
This is the nozzle.
The Shuttle was developed in the 1970s.
We don't have a CAD model of the Shuttle.
Jeff asked me could you send us the CAD model of the Shuttle?
We had a little arrow CAD model of the outer mold line that the aero people use.
There is no CAD model of the Shuttle.
It exists on 800,000 paper drawings.
And, when I came to work, one of the things I got in the Program Office was a recommendation from the independent people to go off and computerize all of this.
And so we went off and tried to develop an estimate to computerize it.
They said it would take us eight years and cost about $40 million to convert these drawings from paper to the latest -- What's the CAD model that everybody uses?
CATIA.
I cannot remember.
We are not going to do it because we don't have the money, so we're still working with the paper products.
Anyway, here is the nozzle.
The water comes in.
There is an orifice and a set of heaters because you've got space on this side.
And if you just push water out there it will freeze.
You want to keep it from freezing so you've got the heaters in there.
And this is a very tricky small orifice to design, install and maintain.
Now, here is what Mission Control sees.
This is a plot of temperature, well, it's actually three or four things versus time.
And what we have here is the first thing that happens is you turn on the heaters on the nozzle and the nozzle temperature warms up.
Then you open the dump valve and the water supply quantity goes down.
And, as the water goes out, the nozzle temperatures jitter a little bit, but they basically stay in this nice, warm 150 degree temperature range.
Close the dump valve.
The water quits flowing, the nozzle bakes out, turn the heaters off and it turns off.
That is a normal water dump, what Mission Control looks at all the time.
The crew has got a timer.
Or, better yet, Mission Control calls and says start the dump now, stop the dump now.
Sometimes they'll set a timer.
Here is a little bit of an abnormal signature.
Turn the heaters on, the temperature comes up, it plateaued out and it kind of does this kind of thing, and people start scratching their heads what is going on?
There is another interesting thing.
Here is one that warmed up, didn't get quite as warm as it should have, sputtered out and quit.
What is going on?
We actually took the arm out and looked at this nozzle, and there was an icicle that had grown on the outside.
And, if you go back to this little drawing, it turns out there is an offset here.
And this nozzle has got a little offset to it.
And they had rotated the offset 180 degrees so that the heat wasn't being applied properly and we were building an icicle.
And we built probably a 16 foot long icicle off the side of the Shuttle on more than one occasion.
And actually went out on one flight with the arm and knocked it off.
Actually knocked it off like you would knock an icicle off your eaves.
The thing that clued us into this, in addition to these temperature plots, was the fact that we came back with ice stuck to the top of the payload bay doors.
There was actually a lump of ice that survived reentry.
At the Kennedy Space Center there was a lump of ice on top of the payload bay doors.
How would you get a lump of ice on top of the payload bay door?
The reason is when this door is open, right here, it hangs over.
And this icicle had grown all the way from the gap from the nozzle to the open door and had actually stuck on the open door.
And when we closed the door the icicle broke off, people theorize, at the root and then carried this long stick of ice up and reentered that way.
And when we landed most of it was gone but about a two or three pound ball of ice on the doors had survived reentry.
Now, you might say, well, what is the big deal about that?
We also found a big hunk out of the insulating tile on the OMS pod where some of that had broken off and had traveled back during entry and struck the OMS pod.
This is not a good thing because you are depending, in early phases of atmospheric flight, on these thrusters for attitude control.
And their propellant tank is right there inside that pod.
Not a good plan.
There is an example of a mystery in Mission Control that we had to work our way through over the course of, really, several flights.
And they redesigned the nozzle.
Here is the simplified drawing of the electrical system.
I think you guys have seen some of these systems things before.
We've got three fuel cells connected to three main buses which branch out into all these sub-buses.
If you are a flight controller you need to understand how you get electrical power because everything on the Shuttle works on electricity.
I am old enough now that I went to work in Mission Control before the simulator had programmed in the electrical system.
And I was a propulsion guy, OMS RCS guy, and we practiced all this stuff about the propulsion system and what would you do if different things happened.
One day they released a new drop of the software and all of a sudden the trainers could cause electrical power buses to fail.
We didn't know what the heck was going on but we learned in a big hurray because that is a big important part of our job.
Here is a simplified picture of the communication systems onboard the Shuttle.
This is the simplified.
Did you catch that?
Have you guys been through the communication system?
We have S-band FM, S-band PM and Ku, and it all comes in here and goes out there.
And it is all cross-strap so that if any one of these little black boxes doesn't work the integrated communications officer can send commands and change it all around.
Unless it was the communications boxes didn't work, in which case we've got to call the crew on the other radio and tell them to go throw some switches, which is very complicated.
And if you don't talk to the crew from Mission Control you don't do anything.
If you don't get telemetry from the vehicle, from Mission Control, you don't do anything.
If you cannot command the vehicle, from Mission Control, you don't do anything.
You might as well go out and get a cup of coffee because there is nothing you can do.
If the comm systems folks are having a bad day -- And I will just add that's one of the most serious malfunctions that we would practice all the time so you had it cold.
I mean if most things, if they break, you can talk to the ground, get some help, they're looking over your shoulder, but if you cannot communicate with the ground at that point onboard your success in getting home depends on being able to analyze what is the malfunction and reestablish communication.
So, we take that very seriously.
And, I've got to tell you, this is a serious design flaw in the Shuttle.
And here is a principle you need to remember.
When the systems were categorized [NOISE OBSCURES] they were categorized on the basis of the severity of their failure mode.
Obviously, the main propulsion system is what we call a criticality one system.
If bad things go wrong in the main propulsion system really bad things could happen, so they spent serious design effort making sure that nothing bad would happen.
The communication system is a crit-3 system.
That means that people did not spend the time to make a robust communication system.
It is not as reliable as some of the other systems.
And that was a huge mistake.
And we have to fuss with this all the time.
That is a huge headache.
When you design something people say, well, communications isn't critical.
Absolutely wrong.
Communications is absolutely vital.
You cannot do anything without communication.
A flight rule on the books says if we lose all communication between a station and a shuttle the crew must land within 24 hours.
And we keep updated with just enough information so that they know where to land without communication.
That is part of the news you get every morning is an update of if you lose communication these are the landing sites.
That's right.
Here is where you want to go.
Here is the weather and here are the times.
This is the simplified drawing of the data processing system.
This is the hardware of the data process system.
And you've got this general purpose computer that goes out through all these data buses to all these multiplexers and demultiplexers to talk to all these different pieces of gear.
And we've got different buses to get into each of the black boxes.
And you can switch ports and it is very reconfigurable and is a big pain in the butt.
And if you want to operate as a systems flight controller you have to understand it cold.
This is the software, very simplified high level view of the flight software onboard the computer.
You guys talk about the onboard computers on the Shuttle?
OK. How much memory does a general purpose computer have?
512K memory.
Not megs, not gigs, K. Of course, it works in ascent and vibrations and radiation and all these other things.
Very sophisticated software in a very small place.
What I grew up on is the reaction control system.
This has got helium pressurization going through a series of regulators and valves down to propellant tanks which get manifolded out to the various thrusters.
You've got oxidizer and fuel.
They come together and they make fire.
And so I started life as a mechanical engineer specializing in fluids so I'm supposed to understand things like pressure and temperature and combustion.
But I had to learn about valves which are mechanical and electrically operated that are operated through the computer system so all of those data buses on this page right here became very important to me.
They are sometimes operated automatically by the software so I have to understand all the software.
And then we get to what we call we redundancy management.
We have this basic principle that we want to have more than one of a critical thing.
And so the question is how do you monitor and manage that redundancy.
And the reaction control system has got the most sophisticated redundancy management program in the Shuttle.
And it depends on instrumentation coming back through those multiplexers and signal conditioners going to the computer that when it commands through the digital autopilot those jets to fire they either do or they don't.
And it tells it all about it.
And then the crew gets notified back over here.
And it is very complicated, redundancy management.
Redundancy is a way to provide reliability.
Redundancy is not a means to an end.
I noticed Chris Kraft said that the Shuttle is quad redundant.
That is not correct, Chris.
The Shuttle is not quad redundant.
We have four computers.
They are not quad redundant.
Now, if you go to a lot of systems, you only have three of.
If you go to some systems that you have four of, one cannot do the job.
For example, flight control.
We have four flight control hydraulic channels that operate the elevons.
One channel cannot operate that.
It takes two.
So you've got four but it takes two.
That means I can lose two.
So, I am at best, three redundant.
I can lose two and fly with two.
The trouble is if you lose two and they get in a fight with the other two it becomes very difficult to manage.
One of my favorite systems is the inertial measurement system.
The main requirement on Shuttle avionics is that it is fail operational/fail safe.
You've got to be able to take the first failure and you can keep flying the mission as planned.
You take the second failure and you're still safe to land because you've got one left.
Three inertial measurement units, clearly we are fail operational/fail safe.
Well, no.
If the first inertial measurement unit fails, good to go, we keep on doing what we're doing.
But those last two, if they disagree, how do you know which one is telling you the truth?
That is tough.
We developed this mathematical scheme using matrices and the navigation quaternion that would give us the best chance.
We have about a 99.5% chance of determining which IMU is telling us the truth versus lying to us, but it isn't foolproof.
It takes people to manage it.
We have opened ourselves, through redundancy, to a system in some cases where we don't have quad redundancy.
We have four ways to fail the system.
The reaction control system is one of my favorites because we have four jets on each side.
For example, the yaw jets in the aft that you need.
So, you would say quad redundancy.
No, I have four ways to fail.
What I'd really like to have is just one jet that would do the job and was highly reliable, but I've got four jets that are leaky that get clogged up, that lie to you on the instrumentation that you've got to watch all the time because I need at least two of them to do the job during entry.
So, be careful when you say things about redundancy.
What you're really after is reliability, not redundancy.
Redundancy is a way to reliability.
And you build these incredibly complicated schemes to deal with redundancy to provide the reliability of the system level you need.
Simple is better, let me tell you.
Complicated is not.
I've got two stories from the trenches and then I'm going to quit.
You guys have not been asking too many questions so that either means I am a brilliant lecturer or I'm putting you to sleep.
This is not your standard academic fair.
Is this good for you guys?
OK.
Yes sir.
I have a question on the redundancy.
Chris Kraft talked earlier this week about how because you have four strings that you should launch if one string is broken and not worry about it since you're still failsafe with what is left and that that would increase the turnaround time and you would be able to launch more often.
One IMU has failed, we'll launch with the two because we know that 99.5% of the time that is good enough for us.
Do you have any comments on that?
Chris and I have had this discussion before.
I have a technical response to this discussion.
Bullshit.
We are not reliable enough to launch with anything down.
This vehicle is barely reliable enough to make the mission as planned when we launch full up.
If you want to build a spacecraft that needs two of or three of or four of.
And you want to be able to launch with one broken on the launch pad like sometimes your airplane takes off with something broken that you, the passenger, don't know, but the pilot does and say it is OK. We're not at that stage.
That is a nice idea.
That is a great goal.
I think people, when they thought about designing the Shuttle, thought that we ought to do it that way.
It doesn't work with the design we've got.
If you were going to build a new Shuttle, yeah, I would put five IMUs on it.
Well, shoot, I would throw out the IMUs and I would put GPS on it or something like that because it is more reliable.
But we are not at a stage where we can launch with less than the normal stuff.
Our flight history is that we have terminated three Shuttle flights because we lost redundant gear to the point where the flight rules said you needed to come home.
Now, if you had launched with just enough gear so that the next failure puts you into shortened mission, you would have terminated more flights early.
The whole theory about the Shuttle, if you go back to the very beginning, we are going to fly a flight a week.
What?
64 flights a year originally.
That didn't happen for a variety of reasons.
If you flew 64 flights a year, the theory was if you got up there and something broke and you had to bring the payload back, OK, we would just role it into the one next week and we would have enough flights.
It hasn't happened that way.
Spaceflight remains difficult because these flights are rare.
The best we've ever done, I think, is ten flights in a year in 1985.
And typically we're talking four or five flights in a year.
These flights are rare.
The pressure is on to get the maximum advantage out of every flight.
And I think spaceflights are going to remain rare with the technology we've got into the future.
That is probably a discussion for a future date, but the fact of the matter is that Shuttle does not have the reliability in its piece parts to launch with one of things down
Wayne, let me ask you a question to speak to, you know, just here at a classroom, not as the Shuttle manager and not for attribution.
This is a danger, when people [OVERLAPPING VOICES] .
Sheila Widnall was here and told us about CARB and their reputation.
What is your feeling about the wisdom of ending the Shuttle flights at the end of the decade?
I have a couple of thoughts.
First of all, I am a Shuttle hugger.
I grew up with Shuttle.
Shuttle is an amazing vehicle.
It is a huge technological leap.
I am very proud of what it has done.
On the other hand, we need a replacement.
It has got some serious shortcomings.
And if you look at the history of aviation and the first 30 years from the Wright Brothers to, say, the DC3, that was about 35 years, a little less.
DC3 was the first economically practical airliner, right?
Everybody wanted to compare the Shuttle to the DC3.
The problem is between the Wright Flyer in 1903 and a DC3 in 1935 or thereabouts, they went through probably 10,000 designs.
They had trial and error.
We tried things out.
We found out what worked, we found out what didn't.
They junked the bad designs.
They took the good designs, and they took the good parts of the good designs and built the next designs even better.
They probably went through 10,000 variations on aircraft to get to that point.
Now, we've been flying in space for about 35 years.
Count them all.
Chinese, Russian, American.
How many space vehicles have there been?
Human space vehicles.
Less than ten.
Soyuz, Vostok, Voskhod, Mercury, Gemini, Apollo, Skylab, Shuttle, Shenzhou, what am I missing?
That's about it.
How can you possibly advance that technology?
It is ludicrous to think that you are going to advance the technology without doing the iteration that we saw in early aviation.
We should have replaced the Shuttle 20 years ago as a nation with a more advanced version that fixed some of the shortcomings that made it more economical to operate.
We should have done a lot of things but, for national reasons, we didn't.
So, I am torn.
I love the Shuttle.
It is a great machine.
I spent my whole career with it.
It gives us capabilities that we are going to give up, frankly, when we go to the CEV.
It's going to be a different kind of machine that does different kinds of things.
And we are going to miss the Shuttle, I am convinced.
But should we long ago have built a new one?
Absolutely.
Are we behind where we should be?
Absolutely.
We need to invent the next generation of spacecraft and be ready to go on and invent the next one after that.
The Shuttle was designed for a ten year life.
We should have been working on Shuttle II the day that Columbia launched the first flight.
OK. That is my perspective.
Anybody else?
Yes.
This might be outside of the realm of the discussion, but those 10,000 different designs that were done for aircraft were done largely by private sector, right?
There was a fair amount of government.
And, remember, it wasn't all American.
There was a large amount of government subsidy.
And it was, frankly, a cheaper technology to develop.
Rocket technology is difficult to develop.
I will go back to Heinlein my favorite author.
He says when you're in earth orbit you are halfway to anywhere in the universe.
Getting the first hundred miles off the planet is very hard, but once you get in earth orbit or thereabouts you are halfway to anywhere in the universe.
And we still have not cracked that nut.
I really like the space elevator guys.
It is science fiction but the idea is a good one.
There ought to be a different technology other than rockets to get to space.
Somebody did a calculation that said if we had an elevator to the Moon, we could get to the Moon for about $10 worth of electricity.
Of course, there is a big "if" that goes in front of that rolling that elevator.
So, rockets are an exceedingly difficult technology.
I want one of you guys to invent a new technology.
Aaron Cohen, he may have already told you this, tells one of the great stories of all times about spaceflight.
He talks about when he was in the management of the Space Shuttle program and the fact that the main engines were causing just awful problems getting them developed.
Stop me if you heard the story.
And one day he woke up and said wouldn't it be great if somebody just invented an antigravity device and we could get away from rockets?
Wouldn't that just be great?
And he thought about it a little while longer and said no, it would still have braised welds and electronic parts and all the things that are causing us problems on the engines would cause us problems with the antigravity machine.
Get him to tell you the story.
He tells it better than I do.
But we need a better technology, quite frankly.
We need something that makes the transition from propellers to jet engines.
We need something like that.
The Space Shuttle main engines in terms of the rocket cycle thermodynamically are about 99% of the maximum theoretical efficiency for a hydrogen/oxygen rocket engine.
You're not going to do any better.
We need a new technology.
You might make them cheaper, you might make them more reliable but you're not going to lift any more pounds to orbit.
So, we need that revolution.
OK. Well, I'm passionate about it.
Anybody else?
Yes.
[AUDIENCE QUESTION] You're talking to the wrong guy.
Of course it's a help.
If you want to talk about whether it's a hindrance, ask him after I'm gone.
Some of our astronauts friends which we'd shut up.
But, no, seriously, I think everybody would say that Mission Control is actually a vital part of the process.
You've got to plan the missions.
You've got to execute the missions.
There are only so many people onboard the vehicle.
These are not autonomous vehicles.
That is another word that really sets my teeth on edge when people say space vehicles ought to be autonomous like commercial aircraft.
It just sets my teeth on edge.
Have you ever seen what it takes to plan a commercial aircraft flight?
There are more people on the ground than there are in the cockpit by a lot.
And I'm not talking about the baggage handlers and I'm not even really talking about the mechanics that keep it flying.
Everybody has got to plan the routes, got to make sure that they've got the manifesting right, make sure that they've got the fuel right, all that planning process, you've got to have people that do that.
Saying you're going to get by without that shows a total ignorance of how the world really works.
Now I'm beginning to sound like Chris Kraft.
I want to share with you a couple of stories.
This is something you ought to past on your wall.
The last law of robotics.
The only real errors are human errors.
Mother nature does not make mistakes.
If you flew your airplane into a thunderstorm and it crashes, was it mother nature's fault?
No, you were stupid and flew your airplane into a place that it wasn't designed to handle.
Perhaps the weather forecaster gave you a bad forecast.
Perhaps your weather radar was insufficient and didn't pick up that nimbus cumulous cloud on its radar, but it wasn't mother nature's fault.
It was a human error.
They used to talk about in aircraft accidents there were really three causes for aircraft accidents.
There is pilot error, which we all understand.
The pilot turned left when he should have gone right, you know, something much more sophisticated than that.
Pilot error.
There is mechanical failure.
Mechanical failure can come for two reasons.
Number one, the aircraft was not maintained properly.
I remember that Alaska airline jet that went down because it had the mechanism in the tail that had the long spiral grooved shaft and they didn't lubricate it properly and it wore off.
And finally they had no elevator control and the plane crashed.
It wasn't maintained properly or it wasn't designed properly.
It wasn't design properly to handle the environment that it flew in.
So, mechanically.
Or weather.
Well, I submit that weather is not a cause of an accident.
Weather is a human failure because you need to understand what you're capable of operating your vehicle in, and you don't operate it in environments that you're not capable of handling.
The only real errors are human errors.
It's either the pilot, the engineer that designed it, the guys that didn't maintain it properly or maybe the guys that didn't forecast the weather right.
Those are human errors.
They are not acts of god.
You need to understand the environment you're going to operate your spacecraft in.
Make sure you design it robustly so it doesn't come apart.
Makes sure you design it so that it can be maintained and you make sure the instructions for the maintainers is done properly.
And, finally, you've got to train your crews so they can pilot it properly.
OK. One of the things that the Shuttle doesn't do well is navigate on its own.
The Shuttle has an inertial navigation system.
We're trying to upgrade it to GPS.
We've been trying to upgrade it to GPS for ten years.
Maybe we will get the next vehicle Endeavor out of its maintenance depot period with GPS and fly it with GPS, but right now we fly it with inertial measurement systems.
Those inertial measurement systems, developed right here at the Charles Stark Draper Lab, have some drift in them.
After about a day their knowledge of where the Shuttle is creeps off.
It creeps off enough so that you could not reenter safely because the error in the onboard knowledge of where the Shuttle is is different from where the Shuttle actually is.
In addition to that, the integration over time doesn't give you a good state vector so we track the Shuttle from the ground with radar and update what we call the state vector position, velocity and direction, six components at least once a day.
On STS-32, Mission Control screwed it up.
There is a long flight.
The ninth day of the flight the INCO officer sent a bad command that caused the Shuttle orbiter to lose attitude control.
And if the propulsion system had been configured differently they would have run the little jets.
If they had been on the big jets we might have used up enough gas so that the crew could not have reentered safety.
This is a serious error.
It also happened at about 3:00 in the morning.
I would offer to you that you ought not do critical things in the wee hours of the morning.
Writing term papers, running somewhat hazardous experiments are not things you want to do at 3:00 in the morning.
That error was recognized and corrective actions were taken immediately.
But, due to some other circumstances, it was a near thing.
We were out of control and out of communications for about ten minutes.
And this was in the middle of crew sleep.
What they did basically was to uplink a state vector that told the computer that the position of the orbiter was somewhere outside the Milky Way Galaxy.
I mean it was that kind of thing.
OK.
Here is the story for the night.
This is 17 days, 23 hours to 18 days GMT.
This was in the early part of the year.
This is a one hour time period.
The crew is awakened in the middle of crew sleep because an onboard smoke alarm goes off.
There was no fire.
It was just an erroneous alarm, but it woke the crew up.
Now, when you are the flight control team and you are working when the crew is asleep, your number one goal is to keep the crew asleep.
Don't let them wake up.
So, this flight control team has already failed.
They allowed an erroneous alarm to wake the crew.
A little bit later the flight dynamics officer says we need to reinitialize the state vector, which is something that we normally do, about once a day.
It is interesting that this is in the middle of crew sleep.
Normally it is done when the crew is awake, but the flight dynamics officer says we need to uplink a new position and velocity sort of vectors.
The flight director says did you do a good job, Fido?
Fido says of course we did.
Flight say OK, you have a go to uplink that vector.
The integrated communication officer gets the word from the flight dynamics officer, I want you to go to the computer and get vector number umpty-ump and uplink it to the crew.
And the integrated communications officer uplinks the vector to the onboard system.
Now, there is a check in the onboard system that goes into a buffer in the computer.
And that buffer gets sent telemetry back to the ground, and the ground computer compares what is in the onboard versus what is sent and they should be the same.
We send about 5000 commands in the course of a two week flight, and normally they always compare.
This particular time there was a problem, some radio noise or something and the data got scrambled.
And it came back to the ground and the computer put out the little words "data reject".
In other words, the command that you sent is not what is onboard, the integrated communications officer.
The backroom is doing other things.
The guy in the front room, integrated communications officer checks the display and says, for whatever reasons at 3:00 in the morning, OK, punches the button to send the execute.
In other words, move the data from the buffer into the navigation software.
It is wrong.
But he just makes a human error and sends it.
His backroom guy, because we always work in teams, is doing something else and didn't check his work.
Normally there is a check and balance.
Before you send buffer execute you say to somebody else does this look OK to you, too?
They missed that check.
What happens?
They send this command.
The Shuttle thinks it is in orbit around, I was going to say Alpha Centauri, but it was a lot farther away than that doing what we call local vertical/local horizontal hold.
Well, now it's doing LV/LH around the star in the Andromeda Galaxy, I guess, and it goes out of control.
Not fast.
Now, this doesn't tumble end over end.
It reaches three degrees a minute rate, which is not a high rate but you're moving out of your attitude.
Well, what happens?
When you move out of your attitude the antennas are no longer pointing at each other.
The Shuttle antenna is no longer pointing at the tracking and data relay satellite so command, data, voice, go away.
Loss of signal.
The worst thing the flight director can hear is loss of signal with the crew.
OK. We got lucky because about ten minutes later it just happened to be acquired back through the satellite.
It just happened to be acquired back.
We got lucky and they called the crew.
The crew switched to a manual autopilot, turns on the big jets, restores the attitude and life goes back to normal.
The crew now has been awakened twice, by the way.
They are going to be grumpy the whole next day.
If the big jets, which use a lot of gas, had been on in that ten minutes, we could have used the entire entry allowance of propellant.
OK. As anything there is always a chain of events.
The flight dynamics officer was unable to do this navigation state vector prior to crew sleep because of the vehicle activity.
In other words, they had been doing maneuvers and they had to get the radars to track to build a solution.
Plans were made to uplink the state vector during sleep, which is not terribly unusual but not the typical situation.
During a sleep period, and I don't know why they put during a sleep period, we typically can have telemetry dropouts and radio frequency interference and conditions which cause telemetry dropouts.
They were predicted because of the orbiter attitude.
The antennas don't always point in the best part of the antenna pattern.
That we're going to have that.
We had the onboard smoke alarm that woke the crew up.
We let them go back to sleep.
This is the same thing I went through.
Twelve seconds after he sent the back command the backroom guy this did trivial recorder command and we saw that they were miscompared.
And this is really the key.
Seventeen seconds after calling up the display the backroom attempted to question the decision but too late, the button had been pushed by seventeen seconds.
OK.
The flight dynamic officer is looking to see if we get a good state vector on the board.
He didn't see on the board.
INCO said I sent it.
What's going on?
The data processing system officer reports that the computers, both of them, the guidance and navigation computer and the system management computer are clocking internal errors.
They have a term for this.
It's called divide by zero.
Computers don't like to do that arithmetically.
They send an alarm.
The propulsion system officer reports continuous jet firing.
The guidance officer reports that they are huge autopilot errors and high vehicle rates.
When I said three degrees, I meant three degrees a second, quite a lot.
CAPCOM says wake up.
CAPCOM, we need to tell the crew something is going on.
Wake the crew up.
Voice link is normally disabled during crew sleep.
Because every once in a while somebody pushes the button and wakes the crew up during crew sleep so we configure it so that we cannot do it.
So we had to reconfigure the ground voice system to allow the CAPCOM to communicate.
We lost the one satellite.
We had the wrong antenna selecting.
And then ten minutes everybody thought they were dead.
We woke the crew up, they put the vehicle on manual mode and life returned to normal after the new state vector onboard.
That never made the press, I don't think.
It was directly caused by operator error.
He clearly did things outside of what he was trained to.
And these are all nice little bureaucratic words saying that everything worked like it was supposed to except for the guy.
And here is what we did in our great bureaucratic mode.
Procedures were updated.
Software was updated.
Rules were updated.
Consol handbook procedures were updated.
Work guidelines of making people work ten or twelve days in a row on nine or ten or twelve hour shifts, particularly on a nice shift were revised so we let people off.
And, basically, what we did was we added more checks and balances to the system.
Now, is that the kind of thing that you do when you are designing a spacecraft?
Why would you design a spacecraft where you had to update the state vector every day?
Why would you design a spacecraft that would crash into the surface of Mars when it was supposed to go into orbit around Mars?
You've got to be careful when you design your system of the unintended consequences of your operation.
And if you don't think very clearly about what you're putting on the operators you'll force them into positions like this, so you've got to think about the operation.
Not just is the wing going to fall off because the wind gust is going to exceed the structural capability?
You have to think about the operations.
I've got one more.
Do we have time for one more?
If you can do it in two minutes.
OK.
This main engine combustion chamber, the main engines have a computer that looks at sensors that controlled our mixture ratio and things.
One of these sensors plugged up.
And they give you the 30 second version.
You can read it all.
The ground had been using a pressure check with a little pressure meter that had a Neoprene rubber O ring.
And, when they pulled the pressure gauge off, it left the Neoprene rubber there and stopped up the sensor.
And, because of that, the engine nearly shut down in flight.
And, if Mission Control hadn't been paying attention and disabled that sensor during real-time, we would have done our first return to launch site abort on that engine.
So, little things count for a lot.
Small instrumentation things count for a lot.
I hope this has been helpful to you.
Anybody got any other questions before I sit down?
[APPLAUSE] One question from the back.
[AUDIENCE QUESTION] Well, my thoughts are: I'm excited that NASA's got the goal of going back to the Moon and Mars, I wish we'd never gotten away from that.
I'm working really hard in the Space Shuttle program to free up money so that exploration can do what it needs to do.
I'm not in a position to know whether they've got enough to do their job.
It does look a little tight - I don't know, time will tell.
But I'm just excited to have the opportunity to head in that direction.
I'm going to give you the diplomatic answer.
Chris is retired; I still have to go to work.
This will be Professor Cohen's kind of last complete lecture, although, on the last day of classes the plan is we're going to take some of the time just to talk over what we've done and review things.
And it will be really oriented towards general aspects of systems engineering.
And, Aaron, I will turn it over to you.
Thank you.
Today, I'm going to lecture primarily on systems engineering but management as it relates to systems engineering.
I have several checks and balances.
And, of course, today I've got people from the Draper Laboratories and people from the predecessor to Draper Laboratories, MIT Instrumentation Laboratory are going to check me to see if I say the right thing.
First of all, I would like to say to you as a class that I read your last submission reports and really thought they were outstanding.
They show a great deal of interest, a great deal of understanding and a great deal of initiative.
And so, I really compliment you.
And I'm sure your final reports will be better, but they really were very, very, very good.
You've heard a lot of people talk previously, starting with, in my way of thinking, Dale Myers who was sort of the person who started the program in Washington at the time.
You didn't hear from another very important man, George Miller, who was really Dale's boss at the time.
But George is still alive, is still doing very well.
And you didn't hear from a lot of other people.
You did hear from Chris Kraft which sort of set the tone, you might say, for a very interesting and understanding man who really knows what he's doing and says what he thinks.
But there are several other people that I would like to talk about that had a great bearing on my career in terms of systems engineering.
There were two people, both who have passed away.
One is Joe Shea.
And Joe was a very remarkable engineer.
He was program manager of the Apollo Program at the Johnson Space Center, then left and became Vice President of Raytheon here and then became a professor at MIT in the Aero-Astro Department.
And I thought he maybe taught the forerunner to this course.
He taught systems engineering.
And Joe Shea was actually the best systems engineer I've ever met, so I am going to talk a little bit and give you some understanding of my interaction with Joe.
The best job I ever had, you might say, really, is I've had a lot of different jobs, but my first job was really working with the MIT Instrumentation Laboratory where I met a great number of people.
I'm not going to mention everybody because it would fill up the time.
They taught me systems engineering, but had it not been for the MIT Instrumentation Laboratory we wouldn't have gone to the Moon on the schedule we went on.
They really did a fantastic job, all these people.
I mean they were just you might say giants.
There were great people at Rockwell, at North American Aviation such as George Jeffs and great people at Grumman such as Joe Gavin and other places, but I have to say that the MIT Instrumentation Lab really carried the ball on understanding how to do guidance, navigation and control.
And if you read some of the comments by George Low, who is the other person I am going to talk about, George Low is another giant.
He says the most complicated system on the Apollo spacecraft was the guidance navigation system, if you read what he said.
And could you really hit a target 240,000 miles away and bring it back?
But let me get back to Joe Shea.
I said he was a great systems engineer.
And Joe Shea taught me a couple of things.
One, he taught me not to be afraid to make a decision.
He said the fact that you are in charge, the fact that you have looked at all the information, the fact that you know the details, you are the best one to make a decision.
And if you could make a timely decision and your battering average is a little over 60% you're doing very well.
But the fact you made it timely and the fact that you understand it, you can make a change.
You can change if you are wrong.
And that always stuck with me, not to be afraid to make a decision.
Because many times as you go through your role in systems engineering you're not going to have all the information to make a decision on a timely basis.
It is just not going to be there, especially if you do a program that pertains to advanced technology, advanced research and development.
That is one thing that is important.
Another thing that Joe Shea brought to the table.
During the Apollo Program, before we had Joe Shea, we would sit around a conference table in the early `60s trying to figure out what systems engineering was, but we couldn't really explain what we had to do.
We didn't know what we really had to do.
But when Joe Shea came, he brought with him the insight of what you needed to do to do systems engineering.
Now, this is the Apollo stack as you see in on the pad you might say.
The launch vehicles.
The launch vehicle.
The first stage, second stage.
Then you get to the lunar module, the command and service module, launch escape tower.
That is what it looks like on the pad.
Well, Joe Shea gave me probably the toughest job and one of the most important jobs that I had in my career.
He said, Aaron, you are going to be in charge of resolving all the ICDs, the interface control documents.
That is all the interfaces between all the stages, between the command and service module and the lunar module, between the command and service module, lunar module and the stack, between the command and service module, lunar module and the launch complex.
There were about a thousand ICDs involved in that.
And that was a formidable task because an interface control document you cannot design without the interfaces being defined, and you cannot define the interfaces without the design.
So it is really a chicken and the egg type of thing.
And it turns out that was a turning point in my career because we did do that.
In fact, a little sideline, a little interesting story.
He anointed me to do this job but didn't just leave me there.
He took me to the contractors, he took me to the MIT Instrumentation Lab, he took me to Rockwell, he took me to Grumman and he took me to the Marshall Space Flight Center and Kennedy Space Center.
And the funny part about it, he took me in to see von Braun, which I didn't know about.
I didn't know von Braun.
And he said, Wernher, this is Aaron Cohen.
We were in his big office at Marshall.
He said he is going to resolve all the ICDs, all the interface control documents.
And Wernher von Braun said that is wonderful.
What is an ICD?
So I knew I had a problem there right away that the head charge of Marshall didn't know what an ICD was.
But that was a little bit of a sideline on it.
I did have the job to actually resolve all the interface control documents.
And that turned out to be one of my most interesting, most challenging jobs as a young engineer.
It really taught me systems engineering, because one of the important parts of system engineering is understanding where the interfaces are and how you get the interfaces resolved.
And you could imagine trying to get Grumman at that time at North American Aviation and MIT Instrumental Labs to agree on an interface document.
I mean it wasn't the easiest thing in the world to do.
Yes.
Did you have a question?
I was just wondering, aside from technically making sure that the interfaces are working and everything, what is involved in resolving an ICD?
That's a good question.
Well, first of all, I am going to talk a little bit more about ICDs in general.
ICDs you can define in several categories.
One is mechanical or physical ICDs, bolt hold patterns, just how things bolt together.
The other, you might say, is electrical ICDs, what kind of electrical signals are sent and electrical interfaces.
And the other are functional ICDs.
One of the big ICDs we had to resolve was a term called Guidance Reference Release.
Marshal had an inertial measurement unit in their instrument unit, and we had to have that synchronized with our system or with the system in the command and service module.
What is involved in it is you sit down and you talk.
You have a draft ICD.
And we had these big meetings.
We had them at the Kennedy Space Center.
I don't know if you've ever been to the Kennedy Space Center, but we actually fill the firing room with people.
And each group had a certain set of draft ICDs, and they would sit down in working groups and decide what had to be done to get the interface compatible in terms of the three phases Physical, electrical and functional.
And you just work out the details.
Aaron, probably now in the 21st century, we ought to add data ICDs.
Data ICDs, yeah.
Well, now things have changed probably a lot since I've been involved in it.
I mean remember Mars Polar Lander.
We had English units coming over.
Yeah, that's right.
And the people thought they were getting metric units.
So the data ICDs are also critical.
You know, and a lot of things are very simple and mundane.
I could tell you a story with the crew.
When we said you're going to have in-flight maintenance with the crew.
And we gave the crew to change out a component, but instead of taking the screw out very easily, we had actually put Loctite in the screw so we couldn't undo the screw.
There are very simple interfaces that have to be resolved.
Some are mundane, some are very sophisticated.
I don't know if that answers your question.
The problem is there is no F=ma.
There is no closed form solution that tells you exactly how to do it.
You just have to sit down and work it because the design hasn't been complete but you have ideas.
You float ideas, you negotiate, you arbitrate and then you come up with a decision.
The main thing is you've got to make a decision, and somebody has to be there to make a decision.
That was my job, to make a decision.
That was what Joe Shea left me with.
And people that know him will agree that he was an outstanding engineer but also probably one of the best systems engineers.
Larry, do you know what course Joe taught when he was here?
Yes, systems engineering.
He taught this course?
Yes.
I thought he taught this course because I know I came up and gave a lecture for him.
Until he was taken ill. Let me talk about another person that was very prominent in informing my career.
His name was George Low.
After Joe retired, George Low became the Apollo Program Manager at the Johnson Space Center.
And George was very much a disciplinarian.
He always had time to see you.
But there were several criteria in seeing you.
You had to tell him what your subject was, how long he wanted and why you wanted to see him.
So when you met with him, if you asked for 15 minutes, you had 15 minutes.
And no matter how important the job was, if you didn't finish was you had to say in 15 minutes you were out the door.
And you only did that once.
But you didn't ask for too much time either because you got penalized if you said you needed an hour and you only took 30 minutes.
So, he was very much a disciplinarian.
He actually, after he retired from NASA, after he retired from the Apollo Program became Deputy Administrator at NASA and was very instrumental in the Shuttle Program and then retired and became president of Rensselaer Polytechnic Institute.
So, George had a very illustrious career.
He passed away several years ago.
One incident might be of interest to you, and I recall it very vividly.
I don't know if everybody else does, but we were getting ready to go to the Moon on Apollo 11.
This is on Apollo.
And you monitor all the systems very carefully.
And we were monitoring the inertial measurement unit in the lunar module which is right here.
And for some reason or other the drift rate changed fairly drastically.
Now, it was still in spec.
The drift rate was still in spec but it changed.
And to get to the lunar module and change out an inertial measurement unit on the pad is pretty hard to do.
First of all, the lunar module has very little structure to it.
It is made out of Reynolds Wrap really.
I mean it doesn't have very much structure to it.
And so we reviewed it with all the people involved, reviewed the data and we concluded very clearly that there was nothing wrong with it.
And so we went to George Low and we told him a unified story, the MIT people, the MIT Instrumentation people, the Rockwell people, everybody, the JSC people.
And George Low listened very attentively and then said can you explain why it changed?
And we said no, we're going to change it out.
That is the thing that always stuck in my mind.
If there is something that you don't understand then you need to take some action to fix it.
And to me that was a very important part of my learning process.
And so, those were the people you might say that really had an effect on my systems engineering thought process.
The other thing I want to talk about with you today is you heard a lot of people talk that I would class as subsystem managers.
You heard Tom Moser talk about structures.
I don't know if I can recall them all.
Al Louviere talk about the mechanical systems.
You heard Bob Ried talk about aerothermodynamics, Bass Redd about aerodynamics, Henry Pohl about reaction control systems, auxiliary power units and hydraulics, Walt Guy about environmental control systems.
You heard of these people.
These people are what I call subsystem managers that actually, in some regards, work for me and, in some regards, didn't work for me.
And that is what you call a matrix management system.
Now, I'm going to talk about matrix management.
Matrix management was actually a very popular management system several years ago.
I don't know if it is used today or not, but matrix management is a type of management basically used by large organizations.
Primarily what it is are large projects organized with teams that work on a functional rather than a project basis.
In other words, these people I just talked about actually had two bosses.
They had a boss in their engineering organization, a very famous man named Max Faget.
And then they had another boss, which was so famous, me.
And I was their boss that controlled the project's cost schedule and performance.
And they actually reported to me on that.
But they actually had two bosses.
And so, you do that for several reasons.
You do that in order to conserve resources.
And many large companies do that.
The other way of doing it, actually, would be to put all that engineering talent in the project office.
And you could see the advantages and disadvantages.
I will show you more explicitly the advantages and disadvantages on that.
The advantages are that you have a dedicated organization.
The disadvantages are that when the project is over there is no place for these people to go.
And I don't know what you use.
You have a matrix at the Draper Labs today.
I'm not that close to industry.
Do most companies use matrix today?
Yes, most aerospace companies.
There are too many projects that change all the time and somebody has to be responsible for the people.
See, that's an interesting part.
We, at the Johnson Space Center, didn't have that many projects.
We only had really maybe one or two projects, but we still used matrix management because there is a check and balance on matrix management which you don't get on centralized or project management.
There is advantage to it.
There are some disadvantages to it.
His rotation lab was project-oriented at the time when we did Apollo.
You were project-oriented because you had one project and you were just project-oriented all NASA project.
So, there are advantages and disadvantages.
I had some dealings with Ford Motor Company, and I think Ford Motor Company actually uses matrix management.
I think most large companies use matrix management today.
Anyway, that's really the advantages and disadvantages of matrix management.
And a lot of things I have already said.
Under matrix management, all the people who do one type of work are in a pool.
For example, all the engineers in one engineering department report to an engineering manager.
That is what I said.
They all report to an engineering manager.
In this case, the name was Max Faget.
And if you do history in the Space Program, Max Faget becomes a very prominent name.
He was really, you might say, the original designer of the Mercury, Gemini and a lot to the Apollo spacecraft configuration.
So, you will run into his name time and time again.
But, the interesting part about it is that Max and I used to go at it tooth and nail.
I mean his people would want to do something like a Walt Guy.
Walt at that time was a very stubborn person.
And he would want to do something and I wouldn't want to do it.
Well, he would go to Max.
And then Max and I and Walt would have a meeting at the Summit with Chris Kraft.
It was a common point.
We all went to see Chris Kraft.
Sometimes I won and sometimes I lost, but I learned actually to love it.
Even though it frustrated me, I learned to love it because there was that check and balance on me that I could not do something that was going to actually be wrong.
Somebody else was going to catch me if I did something wrong.
Now, I honestly do not know today what NASA uses, whether they use matrix management on the Shuttle program or on the Space Station program or how they are going to do it.
Do you have any idea?
The Shuttle was certainly much more project-oriented.
We don't have the same, I mean there is an engineering division at JSC, but during the Shuttle that was -- It started out matrix management.
I think it really decreased, and there was no more Max Faget running things in that same way.
And so that, in my mind, at least in my way of thinking, is probably one reason why problems occurs, because you don't have that independent check and balance.
There is a very good reason why matrix management, I think, is important, even though you don't have a number of projects that would cause you to do matrix management.
Well, this says these same engineers may be assigned to different projects and report to a project manager while working on that project.
Therefore, each engineer may have to work under several managers to get his or her job done.
And that is what I was saying.
It just turns out that you do have that type of problem in doing that.
Proponents suggest there are two advantages to matrix management.
First, it allows team members to share information more readily across task boundaries.
Second, it allows for a specialization that can increase the depth of knowledge.
You can see how, if you have a matrix manager, you can call on more resources to go back to your home organization facilities in terms of testing.
I have worked under, and when I was in industry, total project organizations.
And it has a lot of advantages and disadvantages also.
The disadvantage of matrix management is that employees can be confused due to conflicting loyalties.
I mean was Walt Guy loyal to Max Faget or was he loyal to Aaron Cohen?
Now, Max actually gave him in raises but I controlled the dollars for the project.
But, you're right, that is what counts.
They call that pink ticket.
He was actually pink ticketed to Max and actually worked for me.
And I had to use all my persuasion to get him to do the type of job.
Properly management cooperative environment, however, can neutralize these disadvantages.
In Apollo there was one program.
Right.
[AUDIENCE QUESTION] For example, we had Air Force, Navy and NASA.
Right.
And we were doing accuracy evaluation for each one.
And we were developing the whole software from scratch for each one.
It was different language and different computers and you couldn't compare results.
Right.
And when you have one engineering organization that does the same at least you can compare.
You have the check and balance.
Right.
That's a very good point.
Yes, Jerry.
[AUDIENCE QUESTION] Whether you a project organization, you had your thumb on the people who had to produce.
In the matrix organization, off times the engineering manager would shuffle his man right out from under you to another project.
Oh, that's right.
That's what I had to contend with.
On the other hand, my strong personality and my good will kept it from going that way.
And there was a certain amount of esprit de corps that they wanted to get the job done.
In the end it is the people.
It's the people.
It doesn't matter what organization.
That's right.
If the people are wrong, no organization is going to work.
Well, that's absolutely right.
You have to have the right people.
Well, that's why I brought some of these people to talk to these students because these people were the people I worked with.
Even though we argued at times.
There was nobody that argued more than Henry Pohl and I did, but he is usually always right.
But you can learn to live with it.
Matrix management can put some difficult on the project managers because they must work closely with other managers and workers in order to complete the project.
The functional mangers may have different goals and objectives.
I was saying this as I present these charts, but I think you can visualize that.
And don't hesitate to ask questions because you've got a lot of people here in the room today that can help answer these questions.
Yes.
The point about checking and balance.
You have several managers.
Yes.
From my understanding, you have maybe a functional manager and then you have [OVERLAPPING VOICES].
Well, let me tell you what I mean by check and balance.
Let me give you a specific example.
Let's say Henry Pohl on the reaction control system wanted to do more testing or wanted to do more stability testing.
And I looked at my budget and I looked at where we stood in the schedule, and I said we just cannot afford that, we're just not going to do that.
I made an arbitrary decision.
Well, Henry would be very upset about that and he would go to Max, his functional manager.
And Max would say boy, you're right, Aaron is crazy.
He would then call a meeting with Chris Kraft between Henry, Max and myself to decide what we were going to do.
And then Chris was the final word, the man you met.
He's a pretty strong guy.
And if he agreed with Max, if Max had a convincing story, he would turn me around and then we would do the testing.
And I would just have to find the money some place.
But if I could convince Chris that Max was overdoing it then I would win.
And normally I lost, but that is OK. Do the engineers in any of these organizations out of matrix management feel that they have too many managers?
Well, it is frustrating to them a little bit.
Yes, they did.
It's frustrating because they had to feel that they were working for two different people.
And it is a little bit confusing to them.
I mean they had two loyalties.
That's what I pointed out.
They essentially had two different loyalties.
One to their functional manager who paid their salaries and the other to me who really had the money for the project and the schedule.
So, yes, it is a very tough thing to do.
But, as you could tell, by talking to these people that experienced it both in Apollo and in the early days of Shuttle, they were very satisfied with the results.
That is why it is very frustrating to some of us to see the way things are going because we carried over most of these people you talked to or are here in the room today worked on Apollo and then went and worked on Shuttle.
We had the same process.
We had the same Joe Shea, George Low process that we did.
Yes.
[AUDIENCE QUESTION] Well, that's absolutely true.
Your decision-making process in matrix management turns out to be a little bit more cumbersome than if you have direct control.
In other words, if you have your thumb on everybody you can get a decision pretty quickly.
On the other hand, you can violate some good checks and balances so you give up the process of a rapid decision-making process versus having some good checks and balances.
Does that answer your thoughts?
OK.
I am going to leave that subject, unless there are any other questions on it.
Can I just bring up something?
Sure.
And I welcome your opinion on this, Elliot.
And that is there has been a big change in NASA recently which is affecting management.
And that is because NASA has gone over to what they call full cost accounting which means that during all of this time the engineering organization maintained their engineers.
That's true.
They paid the salary and then they would be assigned to projects as required.
The system, as it's now working, is it's supposed to be much more like in industry where the time of every engineer has to be billed to a specific project.
And if you don't have a project to bill your time to, ultimately you become redundant.
And so, those of you who read Aviation Week and Space News, I hope all of you do, are aware that NASA is doing a lot of downsizing.
It puts a lot of constraint on management.
Now, in industry, obviously people have to be paid for.
But are there always enough projects to bill people to?
Well, there is never a situation that you have the exact number of people.
You're either short of many people, which is now the situtation at Draper, or you don't have enough money.
What you do, in a place like Draper, we have our own money.
It's called internal research.
And we use some money [NOISE OBSCURES] and we keep those people.
But those have to be people the company believes can be used in the future.
There is no point in keeping people that nobody wants.
Actually, the matrix system, and I talked about it with Jerry because we are doing the evaluation, that is why I have to leave, is really sorting the people out.
Because when it was a big project, it was a big family.
The guy owned them.
He knows them.
That's true.
Now, you work at the function organization and we send you to work, he wants the best people for his project.
He will say well, I don't want this guy.
Now, if it is a mediocre guy, we will say look, you cannot have the best.
But if it is somebody who is very weak we will say for one time, OK, maybe he didn't get along with the program so put him in another program.
But if we get the feedback, it doesn't work there, we know there is a problem.
In that way we weeded out a lot of people that, if you have the old system, probably will stay and make us less efficient.
NASA will be more efficient now, I think.
I think that is true.
On the other hand, it causes a lot of pain.
It is very difficult to let people go but that is part of management.
It's not just going to parties.
[UNINTELLIGIBLE PHRASE] That's right.
Yes, there is a question.
I guess another angle on the "very tough to let people go" I've noticed that it is extremely difficult from a bureaucratic standpoint to let somebody go in government.
And I'm just wondering with full cost accounting, OK, you've decided it is time for this person to go, but there is just a lot of politics and bureaucracy paperwork to go through to get somebody out of the organization.
There are civil service laws which have to be complied with.
It is difficult.
And, I have to say, I agree with Elliot.
In the long-run I think NASA will become more efficient.
Well, to answer your question, there are several ways to let people go.
One is if they've really done something bad.
You have a board and you go through it and you release them.
The other is if you NASA has a downsizing, which is called a RIF, reduce in force.
Then you can arbitrarily let people go.
I didn't mean to interrupt you, Jeff, but those are two ways.
I will just add the third way, which NASA is also doing, is you can offer what they call buyouts.
They offer up to $25,000 if you will take early retirement, so a lot of people do that.
And companies, not government, have a much easier time.
I don't know how Draper is, but companies have a much easier time letting people go than other places.
But that was about all I was going to say on management.
Now I'm going to get into, you might say, the interesting land of systems engineering and talk about systems engineering a little bit.
My charts are sort of yellow because they go way back.
It turns out systems engineering has been studied a long time a lot.
And there are many aspects of systems engineering that some people in the room may or may not agree with.
I've had time to reflect a little bit from my days at NASA in both Apollo and Shuttle and in Space Station.
And I was, as I say, Director of the Johnson Space Center, I was Deputy Administrator, Acting Deputy Administration under Goldin for a year, and then I taught at A&M.
And all that time I have had time to reflect and I have had time to do some, you might say, research of what I thought systems engineering was and how you explain it.
I want to try to do that today.
People in the room may or may not agree with it but it is my thought process.
And it turns out that much of this information was gathered from studies done by the military, mil specs on systems engineering.
They have spent an awful lot of money studying systems engineering.
And many of these charts are plagiarized, you might say, from those studies and they look sort of like they are yellow.
And they are.
But let's start off with systems engineering heritage.
Systems engineering, as we pointed out, is not new.
The pyramids, you know, that was a real systems engineering problem.
I don't know how they did it but it was a real systems engineering problem.
I don't know if even Jim Nevins could do that.
That was a tough job.
Broadcasting service and standards.
Now, most of the time when you talk to people about systems engineering, in fact, I've done a lot of research on that, they talk about systems engineering as operations research.
Well, I don't think that can be further from the truth of systems engineering.
I think it is a tool maybe used in systems engineering, but it is not really systems engineering.
The RAND Corporation developed the systems analysis.
The Bell Labs telephone systems.
That was, of course, where Joe Shea came from.
Joe Shea came from the Bell Labs at the time.
NASA military systems, and these are mil standards.
And the Army.
And then systems engineering management guide for Belvoir Defense Systems Management College.
There were a lot of studies done for systems engineering.
And we, in Apollo, were trying to do systems engineering in the 1960s for Apollo, is when we tried to start to do it.
By the way, these charts are not on the Web but I will give these to Jeff and they will be on the Web.
Trans-highlighting the need for increased systems engineering, there is a need to manage the total picture in the area of increasing system size and cost, but even a small project has systems engineering.
If you can think of how you design a bicycle, how you design an automobile, I mean these are very complex problems.
They are not simple.
They are really systems engineering problems.
But the increasing technological growth and specialization, increasing systems complexity, you know, the PC is a fantastic systems engineering problem.
And I guess Bill Gates saw the need for a true systems engineering job and made a lot of money on it in terms of what he did understanding systems engineering.
Increasing sensitivity to environmental factors and increasing costs of life cycle.
All those things make the trend in systems engineering important.
And, as I look at your reports, wherever you got it, I think you have a pretty good knowledge of what systems engineering is.
So, I am not sure this lecture is going to help you that much because I think you do look at it from a systems engineering standpoint.
By the way, my last topic is going to be cost analysis.
I'm sure you're waiting to hear that because cost analysis is very hard for you to come to grips with.
And I am going to talk a little bit about how you would go about doing a cost analysis which may not dot all the Is and cross all the Ts for you.
But I am going to try to do that.
We're going to take a break at about 11:00 and then I will have some time to go through cost analysis.
Well, what is systems engineering?
I don't know.
You may not agree with this, but systems engineering is the application of scientific and engineering to transfer an operational need into a description of system performance parameters and of systems configuration through an iterative process.
Now, let me give you a needs statement.
When President Kennedy said we are going to send men to the Moon and return them safety within a decade that is a perfect needs statement.
It doesn't tell you how to do it, he didn't put a cost constraint on it, he did put a time constraint on it, and he told you what you had to do but didn't tell you anything about how to do it.
That is a very good needs statement.
I don't know if he did that for that reason, but he gave the public and NASA a very good needs statement.
Now, our job was to transform an operational need into a description of system performance parameters and the system configuration through an iterative process.
And that is an exercise left to the student.
That is not easy to do, but that is really what you need to do.
Now, to integrate technical parameters and insure capability of all interfaces, that is what I talked about.
You talked about physical, functional and program interfaces.
They call them different things but basically interfaces become a very important part of the systems engineering process.
And the other factors into the total engineering are to meet cost, schedule and technical performance.
What is that?
I think Professor Hoffman talked about that.
That is the three-legged stool.
It is schedule, performance and cost.
It is a continuous tradeoff because you are going to have a criteria.
Whereas, President Kennedy didn't give us a cost, he did give us a schedule and, to a certain extent, performance.
But he didn't give us a cost.
Now, as it turns out, cost always becomes a factor.
It always becomes a factor.
But those were the things that really comprised, you might say, in simple-minded terms systems engineering.
I cannot emphasize those enough.
Now, I don't know if that is how you visualize systems engineering or how everybody in the room visualizes systems engineering but that and the design process is really what systems engineering and design is all about.
Now, whether we utilize that always in design of our projects I cannot say.
I think we could have done a better upfront job in systems engineering on the Shuttle.
I really think we could have.
I see things we could have done differently on the Shuttle today if we really practiced a true systems engineering process.
It is very difficult to do because you can stay in this systems engineering loop a long time and not get anything done.
There is an old adage, if you never get out of this loop, you could do something, do something and do something and never get anything done, so you've got to be very careful with that.
What is the role of systems engineering?
Technical customer interface requirements definition.
Requirements definition sound very, very simple, but requirements definition is probably one of the hardest things you have to do, is to find out what the true requirements are.
And, unfortunately, requirements do change as a function of time as you go through the phase of the program.
But requirements definition is very, very difficult to do and very important to come to grips with.
As JR Thompson talked to you, I don't know if you recall JR's review of the Shuttle main engine, SSME.
He said had they widened the throat a little bit they would have reduced the complexity of the design and development of the main engine.
Made it much more reliable but you would give up some performance.
Now, why didn't we do that?
Because there was a requirement to advance technology.
And I think JR made a very, very important point there that those are things you need to look at early in the program.
So, requirements definition are very important, extremely important.
And it is very difficult to get to the bottom of it.
Requirements definition is a task in itself.
And then, of course, there is a requirements management analysis and flow down audit.
And then, again, you get interface management.
Risk management is becoming a big item today.
You heard Bob Siemens talk, and I didn't mention Bob Siemens.
He is another very, very important man in this whole era of Apollo and down to Shuttle.
His role cannot be stated too highly.
He did a fantastic job.
His risk management was very interesting.
I heard him talk not too long ago.
And he said that the risk management in terms of cost and in terms of human life for the Apollo program was accepted, not only by the executive branch, the legislative branch but the public.
So, risk management needs to be accepted.
And that is one thing that is very, very important.
Understand what the risk management is in terms of life, in terms of cost and in terms of what it is going to do for you.
So, risk management is very important.
And it is the function of the systems engineer to bring that to bear.
Performance management, design process, so forth and so on.
Technology need identifier.
I had the reputation of not wanting to advance technology mainly because of budget constraints and I wanted to go with given technology.
Is that right?
I don't know if that is right or not, but the point being is that technology can drive a program.
The fact that you have to develop the technology can drive the program.
These are what the role of a systems engineering needs to do to figure out how they are going to handle technology, if they are going to advance technology or not.
What is a system?
A system is a complete solution to a defined need in its full environment over the prescribed life.
The system includes hardware, software, documentation, human resources, nonhuman resources, esoteric factors.
It includes almost everything you can think about.
Software turns out to be a big driver today.
Software in the system turns out to be one of the largest drivers.
Some of you asked me, and I hope it helped, what was the cost of the Shuttle Guidance and Navigation System.
I don't know who asked me for that.
Somebody asked me.
Was that helpful to you, the number we gave you?
The Shuttle.
They wanted to know the Shuttle.
But the fact is it didn't separate the cost of the software and the hardware.
And the hardware came on pretty much on cost but the software was a lot higher.
It was exponentially growing.
Software turns out to be, I don't know, Jerry, do you want to say a word on software?
Well, the problems of software were the issues of validation, verifying tests and retests.
And then every once in a while somebody would come along and say well, but we really ought to add this.
And adding on was a killer.
Well, as Bill Tenel used to say, it is sort of the garbage dump for things you couldn't do in the hardware.
The number that I remember is at some point, when you wanted to change one line of code, IBM wanted $1 million.
Each time for one line of code because they had to change the whole software, so that is why they have the software control mode [NOISE OBSCURES].
That's right.
I remember when I retired, Dave Leestma, who was head of the Astronaut Office at the time, said that he thought he could get a change through.
I forgot.
It was something to do with return to launch site.
I don't remember what it was but it was some kind of program he wanted to put in.
He talked to everybody and they all approved it, and it came to me and I turned it down.
And he said at my retirement party, now Aaron is leaving, maybe we can get it approved.
But software changes do increase the cost of the program.
A simple system example.
Space Shuttle.
Your house.
Electronic calculator.
Freeway.
The Golden Gate Bridge.
Rapid transit.
Computers.
A system really involves everything.
Almost everything you can think of is a system.
The other thing that is interesting is this chart right here.
You may have seen this charge before, you've seen it different ways, but the importance of systems engineering, the systems engineering is doing the right thing right.
As system increase in complexity and value, early system decisions become increasingly important.
It shows that actually in the impact of program funding, when 10% of the program funding is accomplished you have actually committed 90% of your dollars, systems engineering systems have actually committed 90% of your dollars, so that means early decision-making on systems engineering.
And whether you were going to make the engine throat wide or not actually impacts the program very heavily earlier.
That is why systems engineering is so important.
Systems engineering is requirements in solution management.
Understanding analysis, allocations, so forth and so on, which really affect performance, cost and schedule.
And there is the three-legged stool again.
This really says that if you remember certain parameters, the continued tradeoff between cost, schedule and performance is really one of the most important things you can understand that is really a systems engineering job.
One thing I would like to talk about is the attributes of a systems engineering.
Systems engineers are very hard to come by.
They really are very hard to come by, and it is almost hard to tell you how to become one.
I mean there are courses you can take, but one thing I would suggest you do, if you really want to be a systems engineer is you specialize in one technical area.
Whether it be structures, whether it be thermodynamics, heat transfer, guidance and control, control systems, specialize in one area.
That is the first step.
You need to be competent and show your competency in one area, but then you need to broaden out a little bit and look at how people actually operate.
Good judgment of technical competence in others.
Ability to build trust of team customer management.
And this is what this course is about, to work as a team and to have relationships with each other.
I don't know if it was true at MIT, but many years ago in universities it was very bad to work as a team because you tended to rely on each other.
And they wanted to have independent research, independent development, so students were not really encouraged to work as teams.
I don't know if that is true at MIT but it was certainly true at some universities I knew.
I don't know.
Was it true?
Well, we've gone through a major shift.
And I think the rest of the country is as well from individual to team effort.
Yeah, team effort.
Particularly in engineering.
That's right.
When I first went to A&M, they thought it was terrible.
Some of the professors thought it was terrible working as teams.
The difficult that all the top universities have is you guys are all admitted on the basis of your high school or I guess in graduate school your college grads, so those are individual efforts that got you here.
Now we're saying put the emphasis on team activity.
The Japanese have traditionally done that far better than we have.
That's right.
I just wanted to strengthen what you said because a lot of people think that a systems engineer should be a "Jack of all trades" and should know a little bit about everything.
Absolutely not true.
You have to be very good in one specific area, and that is your baseline.
That's right.
The most difficult thing in engineering as a manager is to make a decision when the guy who comes to you knows about the subject more than you because he is the expert.
And the only way you can do it is by knowing how an expert behaves.
And, in your area of expertise, you can make that judgment.
Some people are confused on this, but I think that I 100% agree with you.
You have to be an expert in one field, otherwise it won't work.
I should say there have been a lot of discussions among the faculty in the Aero-Astro Department and perhaps in other departments as well as to when systems engineering should be taught.
Should it be taught at the undergraduate level?
And right now the guiding philosophy in the Aero-Astro Department is to devote undergraduate time really to building up expertise in the individual subject matter, as Elliot said.
The idea being that wait until you really have a good grounding as an engineer in one or more specific subjects, and then, at the graduate level, that is time enough to actually look at systems engineering as a discipline so that you can draw on the expertise that you have built up.
That is not a universally held decision.
And there are people who make a good pedagogical case that you can actually start teaching principles of systems engineering right from the freshman level.
And, in the whole CDIO philosophy that our department tries to follow, we try to develop these principles which we think will then be useful as you get into more technical aspects of systems engineering levels.
We always try to balance the technical expertise with the systems level thinking.
Yes, Jerry.
I would like to point out that in the organizing teams, a key important feature of that is to have that systems engineer who is organizing the team and setting up the meetings to know why and who should be attending.
I attended a number of Space Station meetings when I had these product teams where every company in the world was there.
And you had 50 people sitting around and there were only four who were real contributors, and the others I could swear were falling asleep.
You fundamentally have to have good management in team setups.
Well, you're absolutely right.
Of course, one of the problems with Space Station, in all honesty, if you had tried to devise a management system that was complex the Space Station system would have won because it is probably the most complex management system.
In fact, I tried to explain it to the Bob Gilruth one time, who is another man I should have mentioned.
He was the Director of the Johnson Space Center.
And he came to see me and his eyes just rolled back in his head.
He couldn't figure out what I was talking about.
So, you're right.
But, you know, the other person that agrees with you, Elliot, is Chris Kraft.
Chris Kraft feels very strongly about becoming an expert in one field before you tackle something.
Self-motivated.
Able to motivate others.
In matrix management, that becomes a good systems engineer, if you're able to motivate others.
Methodical.
Analytical.
Intuitive.
Questioning.
Open-minded.
I think that is one of the key points, being open-minded.
If you make a mistake be ready to accept the mistake and correct it and make a change to it.
That is very important in systems engineering, especially if you have to make some important decisions.
Confident.
A good communicator.
High level of integrity.
So, that is really some of my background on systems engineering.
Now, let's talk a little bit about more how you go about doing it.
Thank you, Elliot.
A system is generally considered a conglomeration of objects that perform a specific function.
And you can think of those.
I mean you can think of all those in the laptop computers you have, many things like that which we talked about.
And I mentioned this before.
This is what I say.
It is a postulate or whatever you want to call it.
What is important for the whole system is nearly identical to what would be the best in the long-run for each of its components.
However, what is best for each individual constituent may not be the best for the whole system.
And so that really is something you need to keep in mind.
That is really the postulate of systems engineering.
It may be the right thing for the system but it may not be the right thing for the total configuration.
And the systems engineering process is the customer/user need analysis, mission requirements, functional analysis, system concept, development, tradeoffs, design.
Now, this is systems engineering as it pertains to design.
Now, there is systems engineering as it pertains to operations, but this is systems engineering as it pertains to design.
Design optimization, requirements flow down, telecommunication, design insurance, verification audit and then you go through this iteration process.
In fact, this is what you do in your mind.
I am sure you do that.
When I taught this, when I spoke about this to several professors, my friends at Texas A&M, they say well, there is nothing in this.
They said it is all commonsense.
And, to a greater extent, it is commonsense.
But the problem is do you really practice it?
Do you really have a process in place and do you really practice it?
And we find that we don't really practice it too well.
And we say this over and over again.
But the three factors of systems engineering will be the cost, schedule and performance.
And that is the famous three-legged stool that we talked about many times.
But that really is the essence.
And, you as an engineer going to work and getting paid, that is what you're going to be paid for, getting a product out the door that essentially meets the cost schedule and performance.
You look quizzical.
Excuse me.
No, go ahead.
The statistic that you gave.
After 10% of the project funding then you have 90% of your requirements defined, do you think that function there might change today now that we have faster prototyping tools?
It could change, yes.
This was done some time ago.
Technology is changing.
Rapid prototype is changing.
CAD design.
A lot of things are changing.
It might be not quite as dramatic, you might say, but it is still going to be there.
A lot of your system engineering decisions are going to affect your end product very early in the game.
And one reason why that 90% gets locked in is because of the difficulty of making changes once you've got all the plans drawn.
That's right.
And so, to the extent that CAD systems make it easier to change drawings and propagate changes throughout the system, it may be that making certain changes are less expensive than they might have been.
As Jerry said about software, you start changing your software around, you've got to do re-verification and so on.
Certainly, to the degree that modern techniques might make things more flexible, that number might get pushed down a little bit.
But generally, once you've gone through preliminary design review, you have basically set the parameters of your project and things have a momentum and it really does get harder to change.
And, of course, once you've gotten to the critical design review -- That's right.
At that point you're starting to cut metal.
And then, once you've actually started to build something, the price of changes goes up astronomically.
When we did the Apollo Program we started in '61.
It is my guess that by '64, and not too far into '64, I remember June of '64 we had the implementation meetings with Apollo.
By that time all the hardware definition was in place.
That's right.
The program went on after that.
And the only thing that kept growing or changing was software, but the basic hardware picture was pretty much the nail was in, in 1964, three years into the program.
Well, as Jeff said, once you start cutting chips and start cutting hardware it is pretty hard to make changes.
If I could clarify.
What exactly is it that is being locked in?
Well, usually your requirements.
Primarily what your requirements are is the big driver.
And then, once your requirements are defined, you can actually start your iterative process of design.
Once you get your functional performance requirements defined, you then start your design iteration and you start locking into things and, pretty quick, you start cutting hardware and software.
Hopefully you have gotten a sense, from all of the lectures on the subsystems of the Shuttle, the degree to which they interact with one another.
That's a good point.
I mean imagine if halfway down the road there had been a major change to the thermal protection system which had bumped the weight up by, say, 20%.
Imagine the ripples that that would have affected.
Or, if somebody wanted to change the requirements for cross-range, after you had done the basic design of the system.
Let me give you a very real example in that.
We had the wing basically designed.
And you know that the landing gear comes up in the wing box.
Well, we came to the conclusion that if you blew one tire on touchdown you could blow all the tires.
And so, what they wanted to do is increase the number of tires that went into the wheel well.
And that was pretty late in the program.
I mean we had already decided, from the systems engineering point of view, we were going to have the number of wheels we had and go up into the wheel well.
But then came along a study that showed well, if you blew one tire you're liable to lose the vehicle on touchdown.
They wanted to go put more tires into that wheel well.
Well, can you imagine what that would do?
That would be redesigning the whole wing when the wing was built, so we lived with that.
We lived with that with some risk, but you didn't make the change to it.
That's the type of thing that gets into it.
And I'm sure there are a number of items I could talk about that would cause that problem to occur.
Sometimes if you come up with a problem that really would be fatal, you've got to change it.
You've got to stop, yeah.
And one of the examples being the O ring seal on the solid booster.
In retrospect, people have said we should have changed that at the beginning.
Now, who knows?
If on one of the Shuttle flights a tire had blown and we had lost the entire vehicle, there would have been an accident investigation board.
And they would have gone back and said we were flying with an untenable system.
And we should have made that change.
And that is where the engineer who has become a manager is really faced with a tough potentially life or death decision.
And there is always an element of risk, but if you try to take out all the elements of risk you will never fly.
Well, that's absolutely right.
And so how do you make that judgment?
As I said, that is why they pay you such a big salary.
Well, the interesting part about that is after the Challenger accident my deputy, who became a very good friend of yours, Paul Whites, who was an astronaut, a Navy fighter pilot.
And our job was to get the Shuttle flying again after the Challenger accident.
I thought I brought it with me.
We found this picture of a ship on a very ominous sea.
It said a ship in the harbor is safe, but that is not what ships are built for.
You could make it so safe that you could never fly or you could never fly cross-country or you could never take a train.
You've got to use some judgment.
And, of course, that risk today, that risk level today is not very well accepted.
A point I was trying to make is that during the Apollo Program it was accepted.
We knew we had to take risk.
Tell me when you want to break, Jeff.
Do you want to break now?
This is probably a good time.
OK. We will take about a five minute break.
I want to get to cost, but I have got a few more things to say.
Let me talk about process.
We had a contract with Ford Motor Company, and I showed this to Ford Motor Company of a design process I would go through.
And they said that is very interesting, we studied for a long time and this is exactly what we use to understand the need.
And the need was not the Ford Edsel.
That was not a good needs statement.
But you need a need, you need a need analysis [OVERLAPPING VOICES].
Well, I tell you.
You need to talk to Henry Pohl.
Henry Pohl, who heard lecture, still has an Edsel.
Henry is probably one of the few people that still has an Edsel in his garage.
Henry has a lot of things like that, but Henry still has an Edsel.
But you need to do a need analysis, understand what the need means.
You need a function structure.
And, actually, in some of your reports I saw very good examples of all these.
Now, maybe not in all the reports, but I saw somebody talking about the needs, somebody talking about needs analysis, somebody talking about function structure requirements.
The hardest thing for engineers to do, especially young engineers to do, is to make some assumptions and constraints and do some preliminary calculations.
It is very difficult for them to do that.
They like to see F=ma.
And I once mentioned that to Joe Shea, and he said I am glad you were still on your formula when I talked about F=ma.
You need some calculations and then you need to do some conceptual designs.
And that really is, you might say, the design process.
And you can use various forms of it, but that is probably the best form of it you can think about.
Systems engineering.
Interdisciplinary activities.
Customer need.
Functional identification requirements.
Iterative design process with reviews at various stages.
Conceptual, preliminary and final designs.
And that is what Professor Hoffman was talking about.
As you go downstream very quickly on that it becomes harder and harder and harder to make changes.
Let me now stop with systems engineering, not that I couldn't go longer.
But let me stop with systems engineering and talk to you about a subject you're probably having more problems with than anything, and that is how you do a cost analysis.
And, unfortunately, I am not going to give you a closed form solution for it.
There is none.
Not only that, I did the 90-day study when President Bush was president.
I stood on the steps of the Smithsonian.
Dick Truly appointed me to do it.
And that study was really paned by everything.
Because they said give us a cost analysis.
And I really thought they wanted to know what it was going to cost so I told them.
Well, they didn't like it.
And, in all honesty, I really wasn't that high because it was a 30-year project.
I gave them the cost for a 30-year project actually building launch vehicles, technology.
But, nevertheless, I had my day in the barrel on cost analysis.
Let me talk a little bit about cost estimation techniques.
It turns out some of you were trying to use it.
Here are the cost estimating techniques.
Expert interview.
You talk to an expert in the field, if you can find one.
Parametric analysis.
I am going to spend most of my time talking about parametric cost analysis because I think that is the one that pertains to an engineering design course.
Parametrics.
Because it is in terms that you understand, weight, performance, that type of thing.
I am going to talk about that.
Analogies.
And then the engineering which is really, you might say, the grassroots or the buildup of what it is going to cost for engineering hours, manufacturing hours and that type of thing, which is pretty hard to do.
I am going to go through those.
Now, some of you have tried to do some of these.
Some of you have tried to call contractors to find out what a particular thing was going to cost.
Some of you have tried to find out what the Shuttle Guidance and Navigation System cost.
You were trying to do it, but it's not easy to come by.
You call a vendor and he says well, he will quote you, but he wants to know what you're going to use it for.
It is hard to do.
I know it is very, very hard to do.
And they want to know how many hundred you are going to buy.
I don't minimize the job you have in trying to do a cost estimate, but you've got to recognize that, as an engineer and you go out and start to work, they are going to want to know what this thing is going to cost you and what allies you have to make.
Now, these are the cost estimating techniques over the project cycles.
You can see, as the project goes along, you have this so-called parametric method, analogies, judgments, system level CERs, cost estimate relationships.
And that is a very key thing, cost estimate relationships.
I am going to talk more about that, but let me tell you what a cost estimate relationship in a very simple-minded approach is.
Let's say you're going to go buy a house.
What is the first thing you want to know about buying a house?
How many square feet do you have, right?
You take how many square feet you're going to have and you usually know about what it cost per square foot in the area you want to buy a house in.
You multiply that and you get a rough estimate of the house.
Now, there is another factor that goes into that.
There is the culture.
Where you are going to build this house is going to affect the cost per square foot.
If you build it in a very expensive neighborhood -- I always give the relationship in Houston because I know Houston.
If you're going to build it in Clear Lake where NASA is, it is one culture.
If you build it in River Oaks where the very wealth oil people, it is another culture.
The cost per square foot in Clear Lake is much lower than the cost per square foot in Clear Lake.
You can get a rough estimate, but that is a very simple-minded CER.
It is a cost estimate relationship.
They become very, very complicated when you talk about past history, when you talk about weight of spacecraft and so forth and so on.
I will develop a little bit of that.
General subsystem CERs and calibrated system CERs.
But these are the parametric methods.
And you can see where they are used, analogies and judgment.
And they are used for a certain period of time.
Then, when you get to the phase B or CD, you talk about component buildup estimates, detailed estimates and vendor quotes.
That is sort of the schedule.
And this was provided by the cost technology over the project cycles by the Lunar and Mars Exploration Office.
That goes way back.
It has abolished and now restored again.
So, that is the cycle you go through.
And, of course, you're in really the phase A so it is going to be pretty hard for you to do the direct methods.
Even though you might try, it is going to be pretty hard for you to do that.
You almost have to use some type of a parametrics.
And I am going to talk about how you do a parametric.
I am going to talk to you about how you do both of them.
And this can get you into more trouble.
Cost estimates get you in a lot of trouble.
How many in here are familiar with what we call a work breakdown structure?
OK, good.
Work breakdown structure is really the management tool that is needed in doing a project.
Whenever you get a project you form a work breakdown structure which is a hierarchical breakdown, the work necessary to complete a project.
Work breakdown structure elements should be identified by title, by numbering system that performs the following functions.
It identifies the level of the breakdown structure element, it identifies the higher level to which the breakdown structure would be integrated and shows the cost.
That is the key thing.
It shows the cost account number of the element and how much it costs, how much labor hours is going to take.
That is a very, very important tool.
In fact, when you go out and work in industry, actually, if you become a member of a work breakdown structure, you're graded on how well you perform under this work breakdown structure.
That is the tool they grade you on.
We will put these charts on the Internet.
Roll of the work breakdown structure.
Project and technical planning and scheduling.
Cost estimating and budget formulation.
Project status reporting and plans such as specifications and drawings.
Now, those that raised their hands, where did you learn to do work breakdown structure?
Can anybody answer me?
Where did you learn to do that, those that said you were familiar with it?
Yes, sir.
I learned it here at MIT.
Did you?
Very good.
They taught you.
That's very good.
What course was it?
It is not a course.
It is my research.
Oh, your research.
Did you find it hard to do?
I found it initially difficult, but once I realized how to work it, I definitely saw the benefits of that approach.
Very good.
Well, it is an important tool that you need to do.
And that is why it is probably a little hard for you to do cost estimation because you are not used to some of these things you need to do to get the cost done.
Well, some of the basic elements of cost estimation is understand the product, develop a detailed work breakdown structure.
Did you do that for this project?
No.
That is OK.
I didn't mean to put you under the gun.
Understand the development of the culture.
Understand the culture.
It turns out the NASA culture is pretty high level because it does involve risk.
And I will give you a little story about.
Somebody asked von Braun one time why NASA had to gold plate everything.
He said because you made it out of pure gold it would cost too much.
It is not that they gold plate everything, but it is that you really have a life concern in a very hostile environment.
You have to understand the development culture.
Gather data from close analogs, develop the cost estimate relationships.
There are textbooks that show you how to develop CERs.
There is a large society called parametric cost analysis.
And in that you can get how to develop CERs.
I am not suggesting you necessarily do that because it is very, very time consuming, but just know that they do exist.
There are, by the way, very sophisticated computer programs, which I want to talk about, that develop the CERs for you based on inputs you give them.
These programs are very expensive.
One is called PRICE.
I will show you what the other one is.
And it was actually developed by RCA.
It now is owned by the Martin Company.
Or was owned by the Martin Company.
Group CERs according to the work breakdown structure.
And you need to quantify the risk and the method.
Estimate the cost, spread the cost and do a reality check.
Those are some of the things you need to do.
Now, understanding the costing and the analysis process, as I said, you need to understand the project, participate in the design team activities.
Scoping, inheritance and complexity.
Identification of content, schedule and goals.
Collecting cost, estimating inputs by weight statements, technical characteristics, preparing the estimates, CERs, translating labor hours and dollar hours.
And this is a lot of details that you are not going to have time to do, but you do need to understand the complexity that doing cost analysis entails.
It is very, very difficult and very hard to do.
And you have groups set up to do that.
But, on the other hand, engineers working on a project are going to be called upon to do it because you are the best source of information that they have to do it.
Here is the basic building block of the parametric cost estimating, is the relationship of the cost estimating relationship.
And the dependent value variable is cost.
The independent variable is something that you can understand.
It is weight, power, ISP and so forth.
They are specific engineering terms.
And there is some relationship between weight, certainly for structures, there is certainly a value based on historical data points of weight of the structure, the type of materials and so forth on the CER, the cost relationship.
So, that is how it is developed.
A lot of students had a very difficult time using this program because they didn't know how the CERs were developed.
It frustrated them because here was this magic program that calculated the CERs for them.
Right, Josh?
Josh is one of my students.
And they had a very difficult time trying to understand how to use the program because they couldn't understand how to do the CERs.
But there is a methodology in it.
And there are many books that explain it.
The other thing that is important, as I pointed out, is the cultural variable.
The cultural variable is this.
See, this is one, unit dry weight.
And it can show us the total cost, but it can show that for aircraft you're on one culture plane, on an unmanned spacecraft you're on another cultural plane, on manned spacecraft you're on another cultural plane and on a planetary spacecraft you're on another cultural plane.
That is a very big issue.
One of the issues with NASA they say, to get the cost down for NASA, is you need to change the cultural plane.
Of course, if you change the cultural plane you probably have to give up something.
You probably would give up some risk in terms of safety or margin.
So, you have got to recognize that the cultural variable is a very strong indicator.
Are you happy to note that the aircraft is so low and everybody flies on an airplane, but that is the cultural variable that it shows.
For a structure, you can see how the structure varies based on the cultural plane.
When you were talking about like using square footage as a rough estimate for a house, and in the space business we do use weight as kind of a substitute.
You can translate from weight into cost.
And yet people always point out, well, yeah, but a kilogram of aluminum structure costs a lot less than a kilogram of computer chips.
But, nevertheless, when you put everything down when you're designing a planetary spacecraft, they do have these estimated relationships.
If you're going to have a project which lands 150 kilograms on Mars compared to a ton on Mars, we do have these relationships.
Because, in general, we figure that the ratio of hardware and software and computer chips to aluminum and all that is roughly similar.
But if you are going to try to build something that is fundamentally new, which involves a lot of new technology, then of course you can go way off of these rough estimates.
To get a good understanding, you might say a very quick understanding, you might go to the NASA website and get parametric cost analysis.
It will give you a very good understanding of what you can get.
And you can get some of these programs off the Internet today.
They are not very sophisticated but they do have some CER relationships, as Jeff was talking about.
It might be a little bit more difficult for some of the items.
Structures would be an easy item to do but, unfortunately, I don't think anybody is doing that.
But it would even be good for the avionics system, I think, in life support.
Yes.
Are these software packages basically just a large database?
They are basically a large database that you put in the data.
In fact, they are so sophisticated now, you can put in the software, the type of software you are using, the number of lines of code and, based on the CER for that particular subject, it can tell you how much it is going to cost per line of code.
It is very, very sophisticated.
And NASA uses it quite a bit.
Yes?
With this software, you must have to update it very, very frequently because of technology changes.
They update it very frequently.
That's right.
And that is why they are so expensive.
They are very expensive, but they update them.
For example, I started using it at A&M, and they didn't have even a software package in there for composites.
Now they have one for composites.
And there are a lot of these which are proprietary and you cannot even get access to them.
I mean the Aerospace Corporation has compiled a huge database of costs of a lot of components historical to how much it was estimated, how much it actually cost.
And so NASA and the DOD often go to Aerospace to do cost analysis for them.
And, of course, Aerospace charges.
And they don't release their database because it is proprietary.
Knowledge is power and money in this situation.
They put a lot of work into putting the database together.
Yes.
[AUDIENCE QUESTION] Were you able to try to use it?
Did you try to check it out?
[AUDIENCE QUESTION] And, of course, it may not have the update as this gentleman was talking about, but it has a certain amount.
You could do a pretty quick analysis.
I would suggest you try to do that because I know that I had looked at it, and there are ways to do it.
Now, you've got to be careful of the results because sometimes the results become pretty high.
But you've got to be careful of the results and look at it pretty carefully.
This is another development of cost scaling law.
This happens to be weight versus -- You can see the two different cultural planes, known cost and weight, unknown cost from a given weight.
This is basically how they generate the CERs.
It is a process.
It isn't an analytical process, but there also is a lot of empirical data that goes with it.
And, as Jeff said, it is based on a large database.
Now, cost estimation, I am telling you hard it is to do and you probably already know it.
But, for a cost estimation, now, this is after you get over the errors of parametric cost analysis.
Now we're talking about how you do a cost analysis when you're really trying to figure how you go forward to ask for a budget.
You need fairly detailed information.
It is unrealistic for designers to give cost information for conceptualist designs, but designers must be able to make rough cost estimates.
That is what I am saying, that you do need to be able to make rough cost estimates.
Now, if you are trying to make a detailed cost estimate, and that goes back to the work breakdown structure, you need the number of labor hours and number of people on the project.
I mean people is probably going to be the most expensive.
You need the number of labor hours, the number of people on the project.
During the Shuttle Program, Orbiter Project Manager, I used to call my counterpart at Rockwell every Saturday morning.
And we would go through how many people we had on each work breakdown structure for each contractor.
We would talk about it to be sure we had the budget to cover it.
And then, of course, we had to be sure we got the product out the door.
But the number of labor hours, the number of people on the project, the length of time these people are going to be on it, the overhead rates, office space, computer, benefits, cost per hour of a person basically is what you are looking for, that is how you really do a cost estimate.
And, of course, it is going to vary, depending on the locale of the contractor.
What are the cost drivers?
Now, here is where the cost comes out, some of the things that Dr. Hoffman was talking about.
The errors in the design.
Change in requirements.
Core interface definition.
Technology development.
Poor communication.
Poor teamwork.
These are all elements of systems engineering that have gone awry, that have not gone in the right direction.
But this is what caused the cost to go up.
In my experience, the biggest problem in cost estimates have been the changes in requirements.
That is really the thing that has caused most of the problems.
That is why you need really good definition requirements and not change it.
Technology development, that can cost you a lot of money and sometimes you have to go forward with it.
Now, this is old data.
This is 1993 data, but I got this from a contractor a long time ago.
First of all what you need to do, and of course it is a lot different, I don't know if it is higher or lower today with CAD systems, this was before they really used CAD systems, but this the average cost of an engineering drawing.
And it goes through how you do it.
It is probably the labor rates are higher today.
The average hours per drawing are probably lower.
But this says in 1993, the rate is an estimate for 1993 government year of about $3,000 in average drawing, so you have to figure out how many drawings you need.
I mean that is a good way to understand the cost estimate of what it is going to be.
You need to know how many drawings you need and you need an understanding of the cost settlement.
The elements of cost during manufacturing then are the number and complexity of the drawings, cost of a drawing, number and complexity of machine and fabricated parts and what the parts are made out of.
Aluminum, steel, titanium, nickel, inconel and so forth, because some are more complicated to machine than others.
But some are used in the Space Program, some are not.
Titanium was becoming more and more important.
Other things of manufacturing are tolerance of parts.
It turns out engineers tend to, you might say, put extra tolerances on parts that they don't need.
And that can cause parts to be very, very expensive.
The finish of parts, the quality of parts, assembly, complexity and time qualification of checking a test.
I remember my first day at work, when I worked for RCA, I was designing a microwave tube, magnetron.
I was a terrible draftsman, as a matter of fact.
That was a long time ago.
I would probably be better with CAD systems.
But I sent out this part to the shop.
And this guy coming in showing this stogy out in the machine shop.
At that time we sat in these big bullpens, each person at a desk, and I had my name on my desk.
He comes in this big room and says where the hell is Cohen?
That is me.
He said this is the dumbest drawing I have ever seen in my life.
How do you expect anybody to make this part?
So, you have to be sure you understand what you are doing.
It turns out I learned to work with that guy and became good friends.
That never happened again, but he wanted to know where the hell was this Cohen who put out this dumb drawing?
He said there is no way I can make this part.
And that was a small part, but you need to understand the complexity of the part and what you're putting out to the shop.
But today they have checks and balances.
And, with the CAD systems, I am not even sure this is accurate anymore.
This about brings me to the close of what I wanted to talk to you about, but let me just give you one final chart.
And then I would like you to ask some questions about anything I talked about or anything anybody else covered that you heard about.
I would stress that for you today in your course is to try to use parametric cost analysis.
That is what I would think you should do.
At least that is what I think you ought to do.
And I think you can get information off the Internet that will allow you to do something.
Now, it may not pertain to all projects.
I am not saying you have to do it, but it might be the thing to do.
It is going to be very difficult.
Doing the right thing by creating a work breakdown structure in trying to do that, it is going to be very difficult for you to do it in this course.
But the parametric approach to cost, hardware costs are not produced from parts less than labor tables, so you don't need that.
They are produced from general measures of the impact of such items costs and in preparing inexpensive and more realistic and manageable costs based on cost estimating relationships.
You might want to do some reading on cost estimating relationships, and there are a lot of references to that, just to be sure you understand what I am talking about and what they are so it is not a complete mystery to you so you get a better feel for what it is.
A lot of textbooks have cost estimating relationships.
CERs make use of characteristics that can be readily quantifiable such as weight, size and to estimate variables that are difficult to quantify such as cost and production schedules.
As I said, parametric models can range from a simple arithmetical relationship to a sophisticated computerized model so they can be very, very complicated or very simple.
And I suggest you try to do that.
Let me just show you what the programs are, not that you will be able to get them, but when you get out of industry, if I can find them here.
Well, it may not happen.
There is certainly a lot of information.
Here is the price model.
That is price H. And you can actually do structural weight material types, tolerances.
You can actually do electronics, digital, analog technology.
So, you do have quite a bit of latitude of what you do.
Now, you're probably not going to get that program here because it is pretty expensive and pretty hard to do.
But when you go out in industry or government there is a very high probability you will be using some of these programs.
And Johnson Space Center uses this one and the Marshall Space Flight Center uses this one, CRH.
There are two programs that are used by NASA, I know, and they are becoming more and more important as we go ahead.
Aaron, one thing you haven't discussed in all this discussion of cost is margin.
And, as a manager, you have to deal with that, so maybe you can make a few comments on that.
Sure.
Well, let me start off this way.
What Professor Hoffman is alluding to is that when you create a budget you want to have some type of reserve or margin over and above your cost estimate.
And that takes care of several things.
It takes care of changes, it takes care of I forgots and it takes care of inefficiencies.
Studies have found that in a high technology program for the first time you are about 30% inefficient, so you are only about 70% efficient.
Now, this was data that was taken some time ago.
You are 70% efficient, so you need to allow some type of margin in that for your inefficiencies.
If you do that and you talk about changes and you talk about I forgots, your reserve becomes very high.
And normally your boss, the guy that is over you is not going to let you have that much money.
One of the investigations we had on me during the Shuttle Program, because my cost was growing, a very famous man came in to be head of the program named General Abrahamson.
Many of you may have heard of General Abrahamson.
While he did the investigation, he said, Aaron, you need a large reserve.
He didn't know he was going to become manager of the program.
When he became manager of the program, I had this large reserve in there, he was reviewing and said you cannot have that much reserve so he wacked it down.
But you do need a reserve because you're not going to make it without a reserve.
And, I don't know if you noticed or not but in the papers I gave you on the cost of the Shuttle system, it had about a 30% reserve in there.
When I did the study for the 90-day study, I had a 50% reserve in there because I felt that was a pretty far out program and you needed a reserve.
You need to fight for your reserve, but I guaranty you're not going to get everything you think you need.
Yes, sir.
For something like aerospace, every time you do a cost estimate, they talk about it, look at it and say, well, let's increase that by 50% or double it.
And often, even once you've done that, you still don't come under budget.
I mean look at the ISS or the Airbus 380 that is going out.
Why?
Well, again, it is really because -- First of all, it is primarily for the inefficiencies.
You just cannot do a high-tech program.
Look at what it cost to do the Big Dig, to drive it home.
It is very difficult to do a good cost estimate, but that is the problem that Congress sees with government programs today, both the Department of Defense and NASA, that they cannot do a good cost estimate.
The current program says they are going to do the CEVing and go to the Moon for $104 billion, which is 55% of what it cost to do the Apollo Program in current year dollars.
And that is going to be interesting to see if they can do it.
It is tough to do.
But really I think the biggest problem is your inefficiencies.
Of course there are changes and I forgots.
Interesting enough, though, you may or may not believe this, and you probably won't but I will say it anyway, it turns out that the studies done for the Shuttle Program, for the Orbiter actually, in real year dollars, met the cost if we would have gotten the inflation rate.
We lost the inflation for two years, which Dale Myers fought for vigorously with OMB.
If we would have gotten the inflation for those two years, we would have met cost for the development cost.
Not for the operational cost, but for the development cost we would have come pretty close to it.
We didn't really miss it that far.
In Apollo, I'm not sure, you would need to ask Dr. Siemens, but the story goes is that NASA came in with an estimate.
And at that time the administrator, Webb, looked at it and said that is fine and he doubled it, and so Apollo did very well.
I think Bob Siemens has told that.
The story we heard was it was on their way over to the White House that he said our final estimate, we've put all the factors and everything in, is $10 billion.
And Webb said, OK, we will tell the President $20 billion.
I am sure Bob Siemens would know.
Yes, sir.
Aaron, I wanted to ask you about the location of the reserves.
Not just dollar reserves, but my experience as a payload [NOISE OBSCURES].
The model I have been faced with is that the mission manager has a reserve for crew time or for weight of something but, down on the project level, you would really like to go and have your own.
And so there was a tendency for everybody but the lowest level, from the individual experiment up to say, well, I am going to put in a little bit of pad rather than count on the fact that I can go up to the mission manager and ask him for some reserve later.
Well, that is the danger with that.
You are absolutely right.
Everybody would like to put their own reserve in.
Of course, being a project manager, I thought the reserve ought to be at my level.
But actually, John Yardley was my boss in Washington, and he thought he should handle the reserve.
And the subsystems managers wanted their reserve, right?
Wanted their reserve.
And John Yardley was another person I should have mentioned.
John actually basically pulled the reserve up and handled it in Washington.
There is a certain efficiency, I assume, in having all the reserves centralized so that there is not a lot of waste down to lower levels, but you have to have the confidence that you are going to be able to draw on it.
Right.
And with that John would partial out some.
What he would do is be sure that you weren't double reserving him or triple reserving him.
And then he would allocate it to me.
I know he had a rule of thumb.
When JR Thompson went up, you heard JR talk, he always asked for more than he needed, so John would give him less.
When I went up, I always asked for less than I needed, so he would give me more.
He had a good feel for his managers.
That is what he told JR one day.
Normally, you wind up handling it.
If you have a good program manager at the top you normally wind up handling it there and he parcels it out based on the fact there is no double bookkeeping.
Yes.
My experience at Draper is typically there are two different ways of estimating.
One was called top down and the other was bottoms up.
And traditionally bottoms up would always stick their reserve in some way and top down would always say I want to know what your reserves are because I want to bring it up to the top.
And that was a constant argument.
Then when you got to submitting your proposal often the customer didn't want to know about this extra money and then you would have to shove it back in and hide it, in a sense, by padding things.
Handling costing in reserves is, to some extent, at least in my experience, a psychological thing dealing with customers and your management.
And it has always been one where you shuffle that reserve around.
But this gentleman asked the question about why does it appear that NASA never can stay on their budget?
That is really a tough question to answer.
I don't know.
I mean I have a personal feeling on it, but I think the basic problem you have is that it is a high technology program.
It is one of a kind and it is a lot of inefficiencies.
When you come to work every day, you don't know exactly what you are going to do.
You have a technology development that costs.
You have I forgots.
You have changing requirements.
Changing requirements are something that systems engineering ought to be able to control.
I forgots are something systems engineering ought to be able to control.
Inefficiencies, to a certain extent, systems engineering ought to be able to control.
That is one of the purposes, in all honesty, that are concentrating on system engineering at such places at MIT and other schools to allow the future generation of designers and developers in working that they can have a budget and then stay on the budget.
Maybe in your generation you will be smarter and able to control it, but it is not going to be an easy job.
That is one thing I would say.
Let me just make one other comment.
We cannot estimate too much.
That you've got to always keep in mind this triangle of cost, performance, schedule.
If you are working on a project where the schedule is fixed like Apollo then that better ring the bells to say that I need more reserves.
And maybe that is what was going on in Jim Webb's mind when he said if we really have to get this done by the end of the decade, I better make sure that I am not going to be constrained by cost and so I will double it.
The other thing that you have to take into account with the performance is how much new technology is this program going to require?
Because the new technology, that is the hardest thing to estimate the cost.
That's right.
And so, if you have a new technology program then your margins better be big enough to accommodate that.
That is why, for the CEV, they are really going out of their way to saying, to the maximum extent possible, we don't want to develop new technology for the CEV.
We want to use Shuttle parts wherever we can, Apollo heritage.
We want to build it out of things that we know and we understand so that we hopefully can keep the cost as low as possible.
And we will see how it all works out.
Yes, sir.
It seems to me that you often times have the pressure to give a cost estimate on the low end of what you believe it to actual be just so the project might be more appealing [NOISE OBSCURES].
That is certainly another part of the whole political thing.
And not just from the political point of view of presenting your estimate when NASA presents the estimate to Congress, but then you have the contractors who want to get the contract from NASA.
And you have certainly had experience with this.
How do you deal with the contractor who low-balls and then assumes that they will be able to make it up later on?
Low-balling really should not be tolerated.
And they try to do that, not tolerating it, but you are absolutely right.
When I did this 90-day study, I really felt they wanted to know what it was going to cost, so I had a 55% reserve in there because I really felt that there were so many unknowns with it.
And, of course, that didn't go over very well.
They said my study actually killed the projects.
But I told them what it was going to cost.
And then this is a subject for another lecture entirely, which I am not sure that we are going to do in this course, but there are many different ways to let out contracts, fixed price contracts, cost plus contracts, award contracts.
And all of those can have an impact on the ultimate cost of your project.
And they all have their utility depending on how well the technology is known and what is the time pressure.
Again, cost, performance, schedule.
And you asked about Airbus.
I don't know the specifics of Airbus contracting, but I am familiar with ESA.
And, because of some of the peculiarities of the European system, they tend to use fixed price contracts a lot more than we do here.
And sometimes it works but sometimes you get really burned on it because contractors are not going to go into bankruptcy because they cannot deliver on their fixed price contracts.
You try to go fixed price on the CEV and it would cost you a fortune.
Those are very good questions.
Unfortunately, we don't give you very firm answers on cost.
But at least you are thinking and I appreciate your thought process.
It is very encouraging to hear you ask those types of questions.
Have a good weekend.
We will see you on Tuesday.
There were a couple of areas that have been discussed a little bit, which I wanted to elaborate on because I think it is significant.
One of them relates to intact aborts.
The question of a crew escape system and whether that should be automated, whether the crew should have control and what is required if you actually have the crew take control.
I think Chris Kraft gave an interesting perspective with the idea that because of the difficulties, or given the requirements, really, the impossibility of incorporating a capsule escape system for the Shuttle, the project basically took the point of view that the Shuttle itself was the escape pod, which means that you have to recover the Shuttle intact regardless of any engine failures.
Now that, of course, requires 100% reliability of the solid rocket boosters which we never did achieve with Challenger.
Although, of course, had we operated it within its spec limits it might be another story.
But, in any case, I want to go through some of the details of a return to launch site abort and then I will describe an incident which occurred on our last flight to give you an idea of what is involved in the actual operation of abort modes.
The idea is you have the two solid rocket boosters which have to light.
And they are going to perform no matter what.
If one of those boosters doesn't light and the other does it is a bad day.
What can you say?
And, obviously, if one of them fails during flight it is a bad day.
And, which was more of a design problem, if they don't tail off at just the identical rate so that you get asymmetric thrust greater than a certain capability of the Shuttle to control, you have had a bad day.
And because of the requirement to have symmetric thrust, I don't think if this was discussed before, when the solid rocket boosters are poured, they mix a batch of propellant sort of like you mix bread dough.
And then they pour the propellant into both the left and the right booster segments at the same time so each booster segment has the identical batch of propellant in it.
It may not be the same from the bottom to the top, but it is the same from the left to the right.
And that is critical.
And then they always reserve a certain proportion of the propellant.
And they do tests on that to make sure that there are no process control problems.
OK. You still need the three main engines to have enough thrust.
The boosters each give you about 2.5 million pounds thrust, so that is about five million pounds to get off the ground.
These give you about a half a million pound thrust at vacuum, four hundred and some odd thousand pounds thrust, so still over a million pounds thrust.
And, of course, the problem you have during first stage is you have big gravity losses.
The Shuttle on the pad weighs about 5.5 million pounds with all the propellant.
If you don't fire your main engines you are barely going to lift off the ground.
Actually, I guess it is a little bit less than five million pounds.
But, in any case, without the three main engines firing you don't have enough thrust to get into orbit.
Now, remember we talked about for deorbit burn, normally we fire both OMS engines.
If one of the OMS engines is out you can fire one engine for twice as long.
Or, if they are both out, you can fire the four RCS thrusters for twice as long again.
Because you are in orbit, you are weightless, you don't have gravity losses, so firing half as many engines for twice as long is completely equivalent.
When you are taking off against gravity that is not true.
I mean if you only have five million pounds of thrust for a five million pound payload you are just going to sit on the pad and burn all your fuel.
OK. You need your three engines.
And, in fact, another interesting point, the attachment between the external tank and the Shuttle, that is stressed to assume that you have at least one engine burning.
They tell me that if the Shuttle took off with just the solid boosters and you didn't have any thrust, sort of keeping the Shuttle up with the external tank, that the structural attachment points would fail.
Anyway, you need your engines.
What happens if you lose an engine early on?
If you are far enough into the launch, you are over the main gravity loss segment usually starting at about 3.5 to 4 minutes into launch.
If you lose one engine you can make it over the ocean, And that is called a Trans-Atlantic Abort.
And, when you do that, you are basically flying upside down.
You will do a roll maneuver, drop your external tank and come in and land.
And you are basically going in the same direction which is nice when you are flying a rocket.
These things are hard to turn around.
But if you are going to do a return to launch site abort that is exactly what you have to do.
So, this is a procedure that has been analyzed to death in many, many computer simulations.
The basic procedure is you take off and you lose an engine.
Now, it doesn't matter when you lose an engine in the first 2.5 minutes.
You don't do anything while the solid rocket boosters are firing.
They basically have open loop guidance.
There is a certain projector programmed in and you cannot readjust that trajectory with a closed loop guidance solution to accommodate for any sort of a malfunction.
So, you don't even declare an RTLS abort.
And that, by the way, is done by the crew punching a button which essentially is a protected button.
You punch that.
And you can also do it via a computer input if something goes wrong with the button.
You declare an RTLS abort after SRB separation.
Immediately what happens is your trajectory increases because you want to gain altitude, and you will see why in a minute.
But what happens now is you've got a pretty hefty downrange velocity already and you have got to turn around and come back.
You have also got to worry about the disposal of your external tank.
If you turn around and you come back, you are still burning your engines, you are still riding your tank, and you don't want the tank coming down on Melbourne or Disney World or anything like that.
The basic procedure is as follows.
You are going out, you loft your trajectory a bit, once you get far enough out now what you do is you do a powered pitch around.
A very sporty maneuver.
The external tank is here.
Now you are flying backwards into your plume and everybody has speculated on what that would actually look like.
Now you are flying backwards decelerating.
And, as you decelerate, of course, gravity is starting to push you down so you've got to increase your pitch velocity until finally you basically have no more horizontal velocity and are just sitting statically on top of your exhaust.
And now you gradually pitch forward and you start to fly back.
Again, you have got to plan your trajectory so that when you drop the tank it is going to come down in the ocean.
And then, if all that goes well, now you enter the heading alignment circle.
And, of course, you are a heavy weight vehicle because you also, in the process of the RTLS, dump your OMS tanks and your RCS propellant so that you want to make sure you are as light as possible.
Plus, in case you have any problem with landing you don't want your hypergolic propellant tanks to rupture.
If all that goes well, now you enter the heading alignment circle and you have hopefully a nominal landing.
As I say, this has been analyzed in high-fidelity dynamic simulations.
The crew practices it over and over again.
Everybody knows the procedures.
Nobody wants to be the first crew to try this out.
Before STS-1, you remember we talked about how there were lots of problems with the tiles falling off.
I remember one of the people very high up in NASA, I don't remember if it was the administrator or one of the deputies, suggested that maybe the safest thing to do would be to do a planned RTLS because an RTLS, of course, you don't get up to those high velocities.
And so you don't have any thermal problems.
And so if a whole bunch of tiles had fallen off on the launch pad you could safely do an RTLS.
Well, obviously, they did not appreciate the difficulties of doing an RTLS and that suggestion was not taken up.
We basically decided we were not going to fly until we had confidence that the tiles were going to stay on.
Cut forward to February of 1996, which was my fifth and final Shuttle flight.
I was the second flight engineer so I was sitting right behind the pilot who is responsible for the main engine performance.
Now, I should tell you, we have had, in the course of the 114 Shuttle flights we have had four pad shutdowns, pad aborts.
And I think we talked about that briefly.
The main engines turn on six seconds before the solids.
Just reviewing that does two things.
It gives the engines time to come up to full thrust and then you can perform checks on them to make sure that they are operating properly before you commit yourself to flight.
And, of course, it gives time, because of the asymmetric thrust, for what we call the "twang" that the whole Shuttle stack goes forward and then back.
And then, when you get to the vertical, you light the boosters.
In fact, when they had the first test firing on the pad before STS-1, one of the reasons that they wanted to do that, in addition to just confirming that everything was working right, is they really wanted precise timing on the twang.
People had calculated it, but that is not something you want to get wrong.
So, they wanted an experimental verification.
Of the four pad shutdowns, two of them were caused by instrumentation and two of them were real engine problems.
I remember the first one was the last flight before what would have been my first flight back in 1984.
And the main engine cutoff is called MECO.
So, 8.5 minutes into the flight.
After they had a pad abort the flight engineer called to Mission Control we have MECO, somehow I thought we would be a little bit higher.
And, of course, when that happens now you've got to pull all the engines out, take them back to the shop, bring in new engines and it is about a two or three week stand down.
And the crew has to go back to Houston.
And, of course, all of your friends and family are there and they have to go home.
It is no fun.
I mean nobody complains if it is safe.
We're sitting on the pad, we're counting down and everything was on schedule.
In fact, the first of my five flights it had zero problems, zero delays.
We were going on the day that was predicted on the time.
Countdown to six seconds.
You can feel the engines.
Everything starts to rumble.
The engines start up.
There is what we call a steam gage, a vertical bar which shows the engine power.
Center and right engines come up to 100%.
The left engine comes up to 40%.
The pilot calls out left engine at 40%.
In times like this, your mind is working pretty fast.
I remember thinking to myself oh, damn, we are going to have a pad shutdown, we have got to go back to Houston and back to the simulators.
I mean stupid things to be thinking about at that time, but that is sort of the way it works.
And all of a sudden kaboom.
I feel this big kick in my back.
The solids have lit.
What in the hell is going on?
We were supposed to have a pad abort.
Now we are going with an engine.
And, actually, when you are going through maximum dynamic pressure, max q, the engines throttle down to about 65%.
And there is a malfunction if one of your engines gets stuck it is called getting stuck in the thrust bucket.
If it doesn't come back to full power you are RTLS abort.
Well, we were taking off with one engine at 40%.
It sort of dawned on me that we were going to be the first crew to do an RTLS, so at that point I start going through my checklists.
And the pilot calls to the ground we have ignition, left engine at 40%.
And so, for about the first 15 seconds or so, I just had barely time to get out the RTLS checklist and start getting ready to read the procedures.
And then it turns out that the ground has more insight than the crew does.
And it turns out that there are several sensors, one of which feeds that gauge.
But, in fact, that sensor had gone bad.
And, in fact, the left engine was performing nominally and the ground was able to confirm that both by the other sensors and by looking at the acceleration.
And so, everything was OK, I put away the checklist and enjoyed the ride.
But, first of all, it was kind of a scary situation.
But it also shows the difficulty.
Now, suppose we, as the crew, had been responsible for declaring an abort.
If you are going to build in a system where this is the crew's responsibility then you better design your instrumentation so that the crew has all the instrumentation required to make that decision correctly.
Similarly, if this were to be an automatic abort, you've got to have failure detection, identification, reconfiguration.
The last thing you want is to go into an abort situation when you don't have to because there is no way that an abort is going to be anywhere near as safe as a nominal mission.
Obviously, it's just a good story now.
But I think it does have something to inform about how you are going to have to design abort systems.
And, as both Aaron Cohen and Chris Kraft said, although the astronauts always felt comfortable with their abort system but I am not sure that all the people in Mission Control were quite so comfortable knowing the things that might go wrong with it.
And, in fact, as he said, they always breathe a sigh of relief after first stage when the abort rocket would detach and fire itself away.
Yeah.
If one of the engines is not working or not working properly, I guess the other two engines try to do the work [NOISE OBSCURES].
Well, they certainly will throttle up to 109% if they need to, but they cannot make up for the first engine when you're in the first stage and you have lots of gravity losses.
In fact, when you get, like I said, to about 3.5, 4 minutes then you have enough altitude and downrange velocity to make it across.
And then at some point you get to the situation where you basically gained your altitude.
And, although you are not at orbital velocity yet so you still have some gravity losses, the gravity losses are small enough that if you have a loss of an engine you can still make it to a lower orbit.
That is called abort to orbit, ATO.
And you will hear the call press to ATO means that you can now make it to a lower orbit which is the safest thing to do with two engines.
And then a little bit later on you will hear like single engine TAL which means if you lose two engines with only one engine you can make it across the Atlantic.
And then later on you will hear single engine ATO and then finally single engine.
Then you will also hear press to MECO which means at this point if you lose an engine you can make it through your nominal orbit.
And usually about the last call a single engine press to MECO which means even if you lose two engines you can still make it to MECO.
What happens then if there is an instrumentation problem [NOISE OBSCURES]?
The crew will never declare an abort on their own because it is just realized that the ground has so much more information.
The only exception is if you lose comm.
If we had lost comm.
and we saw our engine at 40%, I mean there were a few other displays which the pilot was starting to call up.
And it could be that he would have figured it out perhaps, perhaps not, but that is the only circumstance under which the crew would make that call.
There was abort at some stage.
What happened to that?
Did they manage to make it [OVERLAPPING VOICES]?
I was trying to remember.
Did I talk about that in class?
Well, all right, I will run through that because it was, again, a very interesting story.
The things that you are afraid about is that if things go back in a hurry in an engine that an engine can actually blow up.
And it is one thing if an engine gets shutdown.
Now you are just flying on two engines.
But if an engine blows up and takes out your whole rear section you have had a bad day.
The instrumentation is biased towards an early shutdown.
If you sense that something has gone wrong shut it down.
We have got intact aborts.
Better to err on the side of safety, even if that might potentially put you into an abort situation.
And I am trying to exactly reconstruct it.
The thing is that if you lose one engine now you are abort ATO.
Now, if you lose a second engine you might be in a situation where certainly you cannot make it to orbit.
And, because you have changed your trajectory, you might not be able to make it to a good Trans-Atlantic landing site.
So, at that point, if you have lost one engine there is a switch on the center console where you can override the engine shutdown command.
What you are now saying is we are in an unsurvivable situation if we only have one engine.
And, therefore, we will take the extra chance of getting an instrumentation caused engine shutdown.
We are not going to take that chance so we are going to disable the ability of instrumentation to shut down the engine and we will just take our chances.
The ground will continue the calculation.
And, as soon as you get to the point where you have the capability of doing a single engine TAL or a single engine abort to orbit, you re-enable the instrumentation.
What happened is the Shuttle launched.
It must have been about 4 or 5 minutes into launch because they lost an engine, it shut down automatically, they are now abort to orbit lower orbit, but they were abort to orbit.
Take the switch, put it to inhibit.
Now, the main propulsion flight controller was very good.
She realized right away that it was an instrumentation problem because sometimes you can see certain trends or you see things change in a way that tells you that this is not the way a real engine behaves, that this is more typical of instrumentation.
She suspected she had an instrumentation problem.
Now they get a little further along in the launch where they could have potentially done -- They were too far to land in the normal TAL sites which were in the daytime on the West Coast of Africa, but they could have made a nighttime landing at some emergency airport in the Congo, I think it was, if they had lost another engine.
But, by the flight rules, they take the switch back to enable so that now, because you can "safely" lose a second engine you want to protect yourself against an explosive situation.
So, now they are enabled.
Now the flight controller sees that the instrumentation on one of the other two engines is starting to go back the way the first one was.
And she realized that in about 15 seconds that engine was going to shut down so she immediately called the flight director and said flight, take the switch back to inhibit.
And this is where the discipline becomes very important, luckily flight didn't get into a long involved discussion about why do you want to do that, what is going on, but just said CAPCOM engine switch to inhibit.
They took it.
And sure enough the instrumentation went bad but the engine was OK.
They got to orbit.
Then they used some of their OMS propellant to raise their orbit a little bit.
That was a Spacelab mission.
And they had a good mission.
But the flight controller actually won a big award for that because she really saved the day.
So, yeah, that is the closest we ever came to a real intact abort.
Yeah.
Two questions.
The first one is going back to the return to launch site.
Do all of the main engine propellant get used up before the tank gets dropped or do you not burn some of the propellant?
You know, I don't know the answer to that.
I don't.
And the other one is on the pad, with your flight, wouldn't the sequencing computer shut down the other engines if it detected that one of them had failed?
And so it must have used the good sensor.
Yeah, that is right.
And, luckily, our little gauge was just fed by one.
And, in retrospect, you can say probably that's not a good way to design it but that is the way it was designed.
You're right.
If the engine controller senses with a majority or however they program it, I don't know the details of the guts of the software there.
But if it is convinced that one of the engines is not working right it shuts down all three engines, which is what happened on four occasions.
Yeah.
When you talked about the forced pitch over in the return to launch site, was that just the gambling of the main engines that gives you that?
Yeah, because you are off the solids at that point.
And that is the thing.
I mean with the solids there is such an enormous thrust and the aerodynamic loads are huge.
You don't want to deviate from your planned trajectory because one of the things that they do is program in load relief for the aerodynamic surfaces.
They take the day of winds launch.
And, if you have wind shears or other atmospheric high jet streams that you are going to fly through, they actually program the elevons and the body flap to move in such a way as to relieve the aerodynamic loading on the wings.
And that is preprogrammed in.
It is not a closed loop.
It is totally open loop so you don't want to change the trajectory once you've put all that load relief in.
How long does that take?
It's pretty quick.
It's about ten seconds, if I remember from the simulator.
It is a sporty maneuver, I will tell you.
No RCS that goes into that?
Oh, the RCS is trivial compared to what you're getting out.
You've got 500,000 pounds coming out of each engine.
The primary RCS is 750 pounds.
But, like I said, they do then open up the RCS as you are flying back.
And what they do is they open up all the engines so that you have symmetrical thrusts so that you are firing out of both sides.
The idea is just to deplete the tanks as much as possible.
The second technical area that I had been hoping that Al Louviere would get into and didn't have a chance to was the payload bay doors which are a fascinating mechanism.
And it is also going to be a segue into EVA because it was a strange situation.
It is kind of relate back to that now when you look at how many successful EVAs have been done and the Hubble repair and the building of Space Station and all the other things, but management at Johnson Space Center was not particularly friendly to EVA back in the `70s.
I asked Chris Kraft about that.
At first he denied it.
He said well, I always liked EVA.
He said well, maybe it was the management of the Astronaut Office that didn't like EVA.
He did admit that Bob Gilruth who was the Director of Johnson Space Center at the time, this was when Chris Kraft was head of flight operations, definitely did not like EVA.
He was afraid of it.
He didn't think it was safe, despite the fact that back in 1973 EVA had saved the entire Skylab project.
Hopefully most of you know that story.
I know it is ancient history.
And clearly we had done a lot of safe EVAs on the surface of the Moon, but basically EVA was looked at it was expensive, it was potentially hazardous.
Given the things that the Shuttle was supposed to do in the original planning, mainly launch satellites, launch pieces of the Space Station, that went away, Launch Defense Department satellites, commercial satellites, EVA was not really part of the big picture in planning.
And, although they did design an airlock for the Shuttle, there were real questions of how much preparation do we have to do for EVA?
Do we really even need it?
Maybe we could save some weight by not having space suits aboard.
That is where we finally ended up.
That is the Hubble Telescope.
This is the side of the payload bay.
I mean, obviously, EVA became a very successful activity.
These are the payload bay doors.
This is in the orbiter processing facility.
And one of the things, well, I have a picture of closing the door.
But the mechanism of these doors is extraordinary.
There are two motors, one at the front and one at the back of each.
Each of the motors has two drive units and they work through a differential so that if one of the motors fails the other one can still drive although at half the speed, just similar to the way a car works.
There is a long torsion rod which runs the length of the Shuttle.
And then that torsion rod is attached to these bars over here.
And I am sorry the picture was scanned in and is not terribly sharp.
As the torsion bar twists then these rods lift up the payload bay door and bring it to a close.
The mechanism, this is the best picture I could find, you notice there is a strong back because, as was mentioned, the doors cannot support their own weight in 1G.
But the system of latches here, you have to remember that the payload bay can expand and contract by several inches due to thermal environment.
And so you have to design a latch system that can accommodate this thermal behavior.
It was an incredible mechanical challenge.
And the complexity of this system, I mean I am always amazed when I look at how the thing works.
I have a schematic of them later, which you can look at, but basically you close the doors.
And then the first thing you do is you want to close the latches on either end.
And then there is also center line latches.
And I think Al Louviere did discuss the fact that these payload bay doors are part of the structural element of the Shuttle during reentry.
You cannot reenter if the doors are not just closed by latched.
You need them latched for structural strength.
There is a series of bollards here, little protrusions which these latches grab on.
And they have designed the rigging of the rods such that the latches that are close to the hinge line actually close before the latches that are further away.
It is a sequential closing of the latches all with a single mechanism so that basically you pull the part of the door that is closest to the torque rod, you latch that down first, and then it is kind of a zipper effect where then you pull it down so that even if the door is a little bit warped thermally you will get the door to close properly.
You wouldn't want to close this latch first and then have the door in a situation where it was buckled out.
And then, of course, the center line latch, because of the thermal expansion, you can have the doors coming together like that or like that.
And you have got to have centerline latches that can accommodate for that.
It is really quite an incredible design.
And Al Louviere had said that he was going to talk about it, but he didn't have time to.
There is plenty of redundancy in here because there are these double motors, but because this is so critical people started to look at suppose you have some degree stuck in the mechanism, it doesn't matter how many motors, you cannot get it closed.
Suppose the motors jam on one side.
Now the other motor is not going to be able to drive it.
So, various things were done to allow for EVA intervention.
And, in the early Shuttle flights, this was the only thing that the crew trained for with EVA, it was just payload bay door contingencies.
For instance, the PDU, the drive unit, this is the payload bay door drive unit or PLBDU.
That is what we call a nested acronym.
That is part of the game there.
You have to go out and actually peel away the liners of the payload bay.
And inside here there is a little mechanism where you can insert a special tool and twist it.
And that actually disconnects that motor drive unit from the torque so that it can turn freely.
And I hope, when you look at these things, you will get some sense of the complexity of these mechanisms.
And so what we do for training -- Of course they build mockups of this in the water and we also look at the real Shuttle down in Florida to actually see these units.
And, for the people who are on the EVA crew, you spend a lot of time crawling around the payload bay getting intimately familiar with a lot of the actual mechanisms just in case you actually have to go out and do something with it.
Then suppose the problem is not with the drive unit but that one of these latches get jammed up.
This is actually a tube cutter.
It is a little hard to see.
I don't know if you are familiar with the things.
They are on a ratchet.
And, as it gradually cuts, you push the blade in bit by bit.
And then eventually you cut away.
And this gives you an idea that they just gave us a lot of tools, a pin extractor, a crowbar, vice grips, you name it.
How much did that crowbar cost?
Well, the crowbar probably cost about $10 at Sears.
And then they had to put on all of the Velcro and the tethers to go in the tool thing, and that probably cost a few hundred dollars.
And then they had to make a drawing of it, and you know how it goes.
EVA is very expensive because everything has to be custom made.
I mean that is one of the things that, because of the constraints of the suits and the gloves, you cannot, in most case, just use regular tools.
And, as I say, everything is tethered.
And I will show you pictures later.
We have a little mini workstation.
They built little receptacles on the side of the suit because it is a hard upper torso which can take a certain amount of load.
And so you plug basically a tool caddy into your suit.
And then you hang all of these tools on the tool caddy.
And you will see some pictures later of how that works.
If the motors don't work and you have now freed the doors so that the door can move but you have no motor to drive it, now we have a mechanical winch and a rope on it.
And I'm not making this up.
You actually run the rope up.
These are the little bollards that the latches catch onto.
You run the rope up over that and then you have to crawl out on the payload bay door, which would be a lot of fun, and attach it here.
And then you actually wind it closed.
And we do this in the water.
I mean this is all fully mocked up.
Yeah.
What do you do if you have trouble opening the doors just after launch?
Well, if you cannot open the doors then you come home.
You wouldn't want to.
I mean if there was something seriously wrong with the doors you better come home.
Yeah, we can do this to get them closed.
But you don't want to purposely put yourself in that situation if you know you have a problem beforehand.
Yeah.
[NOISE OBSCURES] We think it could be done with one person.
Normally, we only go out with two.
But in the orbital flight test, the first four flights only had two crew members.
Was of them was EVA trained.
And then what happens if you get the doors closed by the latches don't latch?
They actually made latches that we could put on by hand.
These are the latches for the side.
And this is the schematic that sort of show the latches going on the orders in that order one, two, three and four.
And, again, the way they designed this so that they actually close in sequence but you only have one drive unit is really just a beautiful mechanical system.
But if they don't close then you take this, and I won't go into the details, but you have to spring it around at least two of those.
And that will give you the structural strength to come home.
And then if the center line latches don't close, again, you can look at the mechanism here.
And they have yet a different tool.
Now, this is actually shown on its side, but this is actually on the top.
And this is a bitch in the water because, yeah, your suit is weightless but the tool is not.
And so you are holding this tool.
When you're holding the tool you want to sink to the bottom, and so you are trying to hold on with one hand.
And you have a tether so you try to tie yourself to the top there so that you're kind of hanging on the tether.
But then, depending on how your weigh out is, if you move this big tool back and forth your body is sort of rocking back and forth.
And you are working above your head.
I mean everybody has to do it.
If you are going to be an EVA crew member and you cannot do this job you are disqualified.
But nobody likes it.
It is a real pain.
That is basically how EVA got sold to the Shuttle System.
I will show you a few pictures of general EVA activities to give you an idea of how the system works.
And then I want to go into the question of the interaction between EVA and the pressurization and the environmental control system of the rest of the Shuttle because, again, this is a systems engineering problem.
And you cannot, in most cases, do one thing without it having an impact on something else.
I talked about doing all this training.
Now we are stepping ten years into the future so that is training for the Hubble Telescope.
And, actually, the telescope is so tall that we couldn't get the whole thing in.
The pool wasn't deep enough so we had to actually cut it in half, and we would work on one half and then work on the other half.
Here is an example.
This is me installing the Wide Field/Planetary Camera.
Now, the training, people normally see the water tank.
You're not weightless.
You all understand that.
If you go upside down the blood rushes to your head.
If you're holding onto tools, the tools will fall down.
And of course you don't have these guys when you're out in space.
Angels if you saw them out in space.
If you're moving around large objects, they have resistance from the water, so you need to get a sense of how easy it is to push things around so you don't get things moving too fast.
We also train on air bearing floors.
This is a mass model of the Wiff-Pick.
The same mass, moments of inertia.
And I am up here on a weight relief system.
And you can really get a sense of what it feels like.
And so you are trying to build a kind of muscle memory so that when you get in space you are not trying to push things as hard as you push them in the water.
And you also remember that if you want to stop something you also have to be able to exert a force on it.
Whereas, in the water, all you have to do is stop pushing.
Now we go into space and we are ready for the EVA.
I will sort of take you through what is involved to give you a sense of the complexity of this system.
First of all are tools.
That is what I looked like before all the cosmic rays made my hair fall out.
It is just one of the hazards of spaceflight.
Here you see the kind of tool caddies.
This is for a ratchet wrench.
And we had about 250 different tools.
About a hundred planned to use and the rest were for contingencies.
And we had to take them all out.
We call this our fish stringer.
Sort of like a Portuguese fishermen.
The fisher wives hang all the fish on the string and then hang them out.
And then we take this out on the fish stringer.
This is the airlock at launch.
And it is totally packed.
And, in fact, we had four spacesuits rather than the normal two so there wasn't a whole lot of room.
And the first thing is you go and you start checking the various systems.
And then you have to unpack everything and move it into the flight deck.
That is Story.
We went out together.
This is the hard upper torso.
This is looking down into the inside of the spacesuit.
The back of the spacesuit there is battery and lithium hydroxide CO2 scrubber.
The CO2 scrubber has to be replaced after every EVA.
It is good for about 12 hours.
And normally if you're only going out once you don't worry about recharging the batteries, but since we were doing successive every day EVAs we would have to take the batteries out every night and charge them.
Take the ones that had been charged and put them into the suits that are going to go out the next day.
And just keeping track of which cartridges have been used and which batteries have been charged and how much, I mean there is a tremendous amount of overhead effort just in managing all of this equipment.
And we had practiced doing this a lot on the ground, as you can imagine, before we went up.
The morning of the EVA.
You get up early and eat as big a breakfast as you can because you are not going to get much to eat for the next eight hours.
These are radiation monitors.
For Hubble we were actually up at about 600 kilometers, which is as high as the Shuttle ever goes, so we got really the highest radiation dose that you can get on the Shuttle on a normal mission.
It was about between one and two REMs which is no big health problem.
But typically the dose on a Shuttle flight at a more typical 300 kilometers is more like tens or maybe a little over 100 milliREMs.
They did want us to take the radiation monitor.
Now we are in the mid-deck.
And when you see these pictures you get a sense of when you're doing an EVA the entire middeck is just full of stuff.
I mean there is so much equipment.
You've got the pants units floating over there, plus the two extra suits and all of the auxiliary equipment.
You try to keep things organized.
Then for thermal control, the original Apollo spacesuits were designed for daytime use on the Moon when it is very hot.
And, in addition, people are walking so they are putting a lot of metabolic heat into the suit.
And so they were very concerned about being able to have sufficient cooling.
They developed a very, very efficient sublimator cooling unit.
You have a little layer of ice that is continually fed from the bottom and it is sublimates.
That basically is how you get rid of your heat.
And then you run your cooling water underneath that ice and it cools off.
And then it is run through what they call LCVG, liquid cooling and ventilation garment which has lots of Tygon tubing that goes through it so that it carries the cold water next to your skin.
And also the oxygen that comes into the suit comes through the helmet over the back of your head so it flows over your face so you breathe the fresh oxygen.
But you would like to get ventilation through the entire suit so you don't build up lots of humidity in your arms and legs.
What you do is the return valve actually comes through these air ducts.
And so the return is picked up at the bottom of your arms and at the bottom of your legs.
That insures that the air does get circulated through your arms and legs.
But when we started doing Shuttle EVAs metabolically you are not nearly as stressed.
Plus, you spend almost half your time in the dark and it gets really, really cold.
And this has been a constant problem.
And here is an interesting systems problem to show how everything relates to one another.
We were very concerned about when you open up the doors of the Hubble Telescope, if direct sunlight comes in, it will cause an out-gassing of the internal material which could pollute the ultraviolet mirrors.
The ultraviolet optics are very, very sensitive to organic contamination, so under no circumstances could we have sunlight penetrating the interior of the telescope.
But we were going to be spending a lot of time with the doors open, so how do you insure that you will never get the sun coming in?
Well, the Hubble Telescope is sitting up here like this.
If that is the sun, just make sure that the belly of the orbiter is always pointed towards the sun.
Then you never have to worry about sun coming into the telescope.
But what you do have to worry about is that, well, during the dayside of your orbit, if the sun is over here and the earth is over here, during the day your payload bay is going to be pointed towards the earth.
And so, you basically have a radiative heat exchange with something that is at about 20 degrees C. No big deal.
If you go around the other side now, you at nighttime, you're pointed at deep space.
And so you're radiating towards absolute zero or three degrees Kelvin.
And that is a concern.
And you get very cold.
They were predicting temperatures of as much as 150 below zero Celsius.
We were very concerned not only were our hands going to get cold but some of our tools might not work.
We pushed very hard for a test in a thermal vacuum chamber.
Our tools are stored in a toolbox outside in the cargo bay.
And we wanted to go through going out and taking the tools out.
And, of course, they are held in there because they have launch locks so there are little pit pins and latching mechanisms that you have to remove so that you can get the tools out.
I went in to do the first test.
And it is a vacuum chamber.
And the walls are painted black.
And they can run liquid nitrogen through the vacuum chamber.
And they could cool it down to the lowest temperatures that we were expecting to experience.
I opened up the toolbox and about 75% of the tools the launch release mechanisms were just frozen shut.
I couldn't get them out.
So, that started to get people's attention.
They sent the engineers in to redo some of the tolerance.
They played around with the lubrication a little bit.
Then Story went in a week later to do his run and he was able to get most of the tools out.
I think he got all the tools out, actually, because I was only in for a few hours.
Because once we realized that it wasn't working I came out.
He was in there.
I remember he said at one point just holding onto these tools my hands are really, really cold.
I don't know.
About an hour or so later someone asked him how his hands were.
He said oh, they are fine, they've warmed up.
Well, Story is from the South.
Those of you with experience in winter know that if your hands are really cold and after a while you don't feel them anymore, it doesn't necessarily mean that they have warmed up.
When they took him out of the chamber his hands were deep purplish black and he had severe frostbite to the point where they sent him up to Alaska to some frostbite specialists.
In the end, he was very fortunate.
He got off with no major permanent damage and was able to fly.
But, as you can imagine, that got serious management attention.
So, gloves now have electrical heating units in them.
But we didn't have that available.
What we had to do was to change the attitude profile so that we were basically pointing towards the earth.
Now, that is generally OK.
If you are pointing towards the earth and the sun is coming from your belly you are not going to hurt the Shuttle, but a quarter way around the earth you do have the possibility now that the sun is going to hit the telescope.
What they had to do was to do two attitude maneuvers every orbit in order to basically keep us in a more benign thermal attitude for the EVA.
But to prevent the sun from shinning on the telescope.
And we didn't the reaction control system propellant budget to support that.
Remember we were going into a high orbit which meant we needed to use a lot of OMS fuel to get up there?
And we needed a lot of propellant for the deorbit because the higher you are the more you have to burn to get down.
And so the flight control community, together with the pilots got together and developed a special way that we could do maneuvers using only half the number of jets that you normally use.
And so this is an interconnection between the thermal environment of EVA and the reaction control propellant budget which nobody in the early days of flight planning would ever have believed that there is this sort of connection.
But it is just another good example of how these systems all play with one another and you cannot make changes in one without another.
I am going to finish this unit and then we will take a little break and then I will do the last part.
The process of getting in the suit.
First you get in LTA, lower torso assembly, basically the pants.
And, of course, this is one time when you can put your pants on both legs at a time.
You cannot do that too often.
But, again, look at the middeck here.
It is full.
This, by the way, is the escape pole.
We talked about this earlier.
During launch and entry it is actually bolted into position so that it will take you out the door, but once you are in orbit you just sort of float it up and out of the way and it just kind of Velcroed to the ceiling.
And, of course, it has no weight up there so it is perfectly happy.
You get in the pants.
Now, the upper torso of the suit is attached to the wall inside the airlock.
Now you have to float into the airlock.
And that is what you're looking at when you're getting into the suit.
There are a lot of connections you have to make.
This is the water bag.
This is the comm.
connection for your Snoopy cap.
This is the water and air connection which you have to hook into your LCBG.
And these are the arms that you have to get into.
And the problem in the design here is your shoulders are wider than your chest.
When you are actually inside the suit you want the distance between these two arms to be conformal with your armpits.
Because, if it is out here that cuts way down on your mobility.
But if it is too narrow you cannot get into it.
That is the basic design problem in this sort of a suit.
Now, the Russians have a rear entry suit that doesn't have that problem.
It is always a challenge and you really have to push and struggle to get in.
Eventually they wanted to design this suit so that one person could get in by themselves, but it cannot be done.
I mean you can get in like this but you just cannot pull up the waist ring and attach it by yourself.
Nobody has been able to do that.
It is a great feeling of accomplishment when you finally get inside.
Then you put your helmet on.
Now you're starting to see what we are dealing with because we have to take all this stuff outside in addition to the tools that are already outside.
These are the plugs which we put the mini work station.
We haven't put it on quite yet but it is just a whole bunch of equipment which you have to manage.
At this point we purge the suit with an oxygen flow.
Now, remember, we are at an atmospheric pressure, or almost an atmospheric pressure, I will get into that later, of pure oxygen, so that also puts very serious flammability constraints.
Although, when you go outside, you are only at 4 psi.
When you are inside here you are at the full cabin pressure.
Actually, in order to do your leap check you have to go to 4 psi above cabin pressure.
You are actually working at above one atmosphere of pressure of pure oxygen, so that is a very serious flammability design constraint.
And we have to sit and breathe pure oxygen then for about 40 minutes to get the nitrogen out of our blood.
And, again, I will be discussing this whole atmospheric and bends problem.
But, if you don't, when you drop down to 4 psi, the nitrogen in your blood bubbles out and you get the bends, just like a diver who comes up too fast from a dive.
Just one or two pictures of what went on outside.
There is Hubble.
I was changing some fuels there.
It was kind of neat working underneath the solar panel.
Those were the old ones which we took off and replaced the next day.
But you have to be really careful because you don't want to bang against it.
And, of course, you cannot see above your head, so we always have other people looking at us.
This is the kind of situation where if you had a little heads up display where you could get this sort of a view to show what you're really doing it would really improve your situational awareness.
But, at the moment, we don't have that.
It takes not just the people outside but the people inside paying full attention, reading procedures.
Historically, we carried all of our procedures on a cuff checklist, like so.
But the procedures for Hubble and some of the other flights are so complex that you just cannot do it.
So, nowadays, these are just emergency procedures.
I will pass this around.
I would like it back.
Let's see.
After putting in the new Wiff-Pick, ground had to do some tests on it, so I got Claude to fly me out over the arm.
And we had one of the old Lunar Hasselblad cameras which they let us carry.
Normally, we just use Nikons on the Shuttle, but they let us take a Hasselblad.
That gives a view of what the payload bay looks like EVA.
Here is Story over here working on one of the other things.
The Earth is not flat, don't worry about the picture.
I went around it many times.
And, I mean, some parts of EVA, I really should share it because it just gets really spectacular.
This is about 50 feet, I don't know whether to say above or below or whatever of the Shuttle, but you're just out there in the middle of nowhere.
And, at this point, I was the free-floater.
One of the people is always attached to the end of the arm, which is good when you have to move around big pieces of equipment because now you can react the forces with your feet.
When you are a free-floater either you're in a foot restraint, which limits your mobility, or one hand has to actually be holding you so that you can react forces.
Of course, we are attached by a waste tether, which is then attached to a long real of stainless steel.
But you can set it so that the springiness is taken out.
And so I could basically, and I did from time to time, must sort of let go.
And it is a really neat feeling because once you convince yourself that you're not going to fall down, which I'm enough of a physicist to understand the orbital mechanics, but, nevertheless, the first time I let go it was an interesting feeling.
The thing is when you're holding onto something, whether by your feet or your hands, of course the Shuttle is so much more massive that you feel yourself physically related.
You are controlled by the Shuttle.
And so the Shuttle is your point of reference.
Maybe the Shuttle is in orbit but you are attached to the Shuttle.
As soon as I let go, the physics totally changes.
And it was really this transformation into being a human satellite.
I mean I really felt like I was a satellite now.
I wasn't attached to the Shuttle.
And sometimes, if I could turn around before I let go so I didn't see the Shuttle, it was really kind of neat, especially at night, just sort of floating there and all the stars and everything.
Yeah.
Do they build in time for you to be able to kind of do that kind of thing?
No, they don't build in the time.
But, like in this case, two days before on Story's and my last EVA we had replaced -- Not replaced.
These are the magnetometers up at the top.
The gross maneuvering of Hubble is done by reaction wheels, but the reaction wheels, when they get spinning too fast, you have to be able to desaturate them so they have magnetic torquers, these long torque rods which create a magnetic field.
There are no jets on Hubble because it would cause pollution of the optics, so you need magnetometers up at the top to sense the magnetic field in order to operate the torquers.
The magnetometers were never designed to be replaced.
They weren't supposed to fail.
Well, both of them failed in the first couple of years.
We couldn't take them off so they designed two new magnetometers that we could actually insert and bolt on, on top of the old ones.
And then we took the electrical and data connections and hooked them up.
But I noticed on that day -- We were very concerned with quality control.
First of all, that we didn't break anything that wasn't already broken but also to look for any signs of damage because, you know, this was a telescope that was supposed to be maintained.
We saw a little bit of paint chipping off the outside of the old magnetometers.
They were concerned that they might float around and get into the optics.
We went into the payload bay the next day, the other EVA team, and they cut off some of this gold insulation material from one of the thermal enclosures that some of the equipment was in.
And I'm not supposed to stand in front.
I keep forgetting that.
We made those covers, and then we had to go up and install them.
But, actually, after we installed them then the ground wanted photographic documentation.
We took pictures.
And then the crew inside got their telephoto lenses and they took a lot of pictures.
And then they got the television camera on the end of the arm to take pictures.
And so, during all that time, you do have a little bit of free time just to sort of enjoy the environment.
It would be a shame not to because it is such a spectacular place.
Yeah.
How do you work when there is not light when you're on the dark side.
Let me turn the laser pointer on here.
Oh, no, what did I do?
That is a good time for a break, right?
Take a two-minute break.
And then, of course, when the sun rises you know it so you can turn your lights off.
OK.
I talked about the necessity of doing a nitrogen purge and pre-breathing oxygen to get the nitrogen out of your blood.
This turns out to be an interesting systems problem because it doesn't just affect the suit, it affects the spacecraft.
Those of you who were here at the beginning saw I was blowing up a balloon which unfortunately popped.
It was only to make the point that you can look at a spacesuit, the arms and legs, cylindrical.
It is like one of those balloons.
And when you try to bend the balloon it doesn't like to stay bent because you are compressing the gas and you're doing elastic work on the material of the balloon and it wants to snap back.
And spacesuits basically work the same way.
The old-fashioned pressure suits that test pilots used to wear, which were the genesis of the original Mercury suits, they basically stiffen you.
And, in fact, the launch entry suits that we use on the Shuttle are not designed for joint mobility.
They are just pressurized.
And we do a pressure check before the mission when we're getting suited up.
And they blow it up and you just go like that.
You can move a little bit, you know, enough so that if you're sitting in your seat with a seatbelt holding you in form so that your waist is bent, you can move your arms enough to get to the controls.
But I certainly wouldn't want to try and go out and do any useful work in that.
It has really been an extraordinary design process that people have been able to develop spacesuits with articulating joints.
I mean it is certainly not like walking around and doing things just with your body, but it is amazing how flexible spacesuits are.
And it is a continual challenge to develop, particularly for the gloves.
And, when we get back to the moon, to have legs that can actually walk in so that you don't have to hop around all the time.
Although, there are times when that is good, too.
But, in any case, the stiffness of the suit, because it turns out that most of the stiffness comes from compressing the gas, you want to try, first of all, to build articulated joints that don't change their volume.
And, actually, people have designed metallic suits that look like Robby the Robot.
So, they truly are zero delta volume suits.
The problem is they are very heavy and there are other problems with metallic suits.
And I don't have time to go into that.
Given that you're going to have some volume change, the amount of work that you do is thermodynamically pressure times volume.
You are going to have a delta V. You want to reduce the pressure.
The lower your suit pressure, the easier it is to bend.
Basically, you don't want to fill your suit with any gas that you don't really need.
We certainly don't need nitrogen to stay alive, not on a short-term basis, so we fill the suit with oxygen.
And we run it at about 4.3 psi for the Shuttle.
This is just historically Mercury, Gemini and Apollo, the suit pressure was a little bit lower.
Doctors felt, after doing calculations, what you're really interested in is the partial pressure of oxygen in your blood.
And they wanted a little bit more margin.
Since the Shuttle was supposed to be operational this and operational that, they wanted a little bit more margin.
And so we bumped up the suit pressure a little bit.
Mercury, Gemini and Apollo cabin was 100% oxygen at the same pressure so that you didn't have to worry about pre-breathing.
In Skylab, it was not pure oxygen but it was a high enough oxygen that, again, they didn't have to worry about pre-breathe.
It wasn't until we went to the Shuttle, which typically the Shuttle is at 14.7 or 100 kilopascals normal atmospheric pressure.
If we know we are going to be doing a lot of EVAs, and I will talk about this later, what we do is we drop the overall cabin pressure, but we keep the partial pressure of oxygen the same so you end up bumping up the oxygen concentration.
We don't do that with the ISS.
The decision was made by the life scientists that since they wanted to study biology on the Space Station and all of our database is at one atmosphere, that if they took the Space Station down to 10.2 psi basically it would invalidate all of their life science research.
So, the Station was only designed for one atmosphere.
The operational people, when they started to talk about how much EVA was going to be involved in building and maintaining the Space Station, the astronauts said this is crazy.
And you will see the impact of this later.
This is just looking in the future.
These are decisions that are going have to be made for future space systems, and they are looking at a lower working pressure.
And you will see why as I go through this presentation.
I think I'm not going to go through those charts.
Again, this is a system.
So, you are affecting a great many different things.
Let's see.
Which is the laser here?
There we go.
One of the interesting things here with EVA, just like in any ECLS system, is that the human body becomes one of your subsystems that you have to take care of.
And, in many cases, we don't have a whole lot of design flexibility with our bodies because they come predesigned.
So, we have to deal with that.
We have all of the physiology, the bends.
Materials.
I talked about flammability in a pure oxygen environment.
I talked about the constraints of microgravity and partial gravity physiology studies in the ISS.
Of course, NASA has decided that we don't need to do that anymore.
And, I guess, our international partners are still going to want to do this.
In any case, the Station is designed.
The cooling we talked about, I think, in the ECLA section lecture, the fact that if you drop your cabin pressure now you've got to pump more air for all the air cooled things.
And if you really want a design for a variable cabin pressure then that flows over into the cooling requirements, more water cooling, different fan requirements and so on.
And it goes on.
If you are going to do multiple EVAs, what you're really interested in, when you're looking at efficiency, is what we call EVA Work Efficiency Index.
It is the total amount of time involved in preparing for and carrying out an EVA, the ratio of that to the actual useful working time you get outside.
And the more time you have to spend breathing pure oxygen to denitrogenate your blood, that is taking away from your overall work efficiency.
When we do tests on the ground and we start out at normal sea level pressure and we want to go down to 4.3 psi pure oxygen, we actually have to do a four hour nitrogen purge.
We have to just sit in the suit or stand in the suit for four hours, this is in the EVA test chambers, before we actually can go down to pressure.
If you want to get an eight hour work day of EVA, that is just unacceptable.
And that is why they made the decision to lower the Shuttle's cabin pressure.
Then you only have to do about a 40 minute EVA.
And I have some of the specific numbers on that later.
As I say, just the review here is that we do have this capability on the Shuttle.
We don't have it on the Station.
They have figured out a few ways to make the denitrogenation process a little bit more efficient on the Station, but it is still a big overhead hit.
And we are very concerned for the future because EVA is not just an afterthought when you're going to be exploring on the Moon or Mars.
You are going there basically to do EVA.
Otherwise, why bother?
We have got to deal with this.
Here is the basic physiology that we are dealing with.
Oxygen percentage on the abscissa, total pressure on the ordinate and all different units.
Millimeters of mercury, psi, and I think now people use kilopascals, which I don't really relate to.
But I know about 100 kilopascals is about one atmosphere, so luckily that makes it easy.
Normally, we are at one atmosphere 20% or 21% oxygen, we are up here, and this is the normal sea level equivalent.
You start dropping the pressure and you have to increase the percentage of oxygen.
There is a maximum level of breathing oxygen at pressure oxygen toxicity because oxygen is almost totally absorbed through your alveoli.
And normally our alveoli stay inflated because we have 80% nitrogen which is only very reluctantly absorbed.
I mean it does get through, that is how it gets into our blood and so on, but it is a much slower transport.
But oxygen gets right through.
And if you breathe pure oxygen for a long time you can really hurt your lungs and it can be lethal at a certain point.
We actually don't have any good physiological data.
I was down in Houston last week at a big EVA conference.
That is why we didn't have class on Tuesday.
They were talking about what about long-term planetary EVAs, even at 4 psi pure oxygen, is that going to be harmful to health?
We honestly don't know.
I mean that is an active area which needs research.
On the other side, of course you get into hypoxia which you have all heard of.
You get it when you go up on Mount Everest and so on.
So, you've got a boundary that you have to work in.
Now, this is sort of where things have fallen.
The blue line is the hypoxic boundary.
The green line is normal oxygen.
When we were using pure oxygen spacecraft environments, we were well above that.
You would like to stay as close as possible.
With Shuttle EVA, we moved a little bit closer to hypoxic, but we're still in a physiologically perfectly reasonable environment.
Decompression sickness, just a quick review.
Any scuba divers here?
You are all familiar with this.
There are various levels of decompression sickness anywhere from just a mild tingling of the skin to joint pain to phase three DCS where you get central nervous system impairment which can actually cause death, so you don't want to mess around with it.
People refer to this famous R value.
You run into this all the time.
What you are really interested in is what is the ratio between the actual partial pressure of nitrogen in your blood compared to your suit pressure?
And this over here shows this R value.
If you are down at one the nitrogen isn't going to bubble out at all so you have no incidence of bends.
VGE, venous gas emboli.
And then, as the R value increases, the dotted line gas emboli, that just means you have bubbles forming.
DCS means actual symptoms of bends.
And then grade three DCS which is very serious indeed.
The way they make these measurements, they have these large physiological studies where they actually get people to volunteer to go through these pumped down protocols.
And then they actually put sensitive microphones and ultrasound things that can actually hear the bubbles as they move around your veins.
I have never been able to figure out why anybody would volunteer for these experiments.
I take my hat off to them.
Because the only way you build up this data is that some people actually do get the bends from these experiments.
And, of course, they have hyperbaric chambers which, as soon as there are any symptoms, they put them right into the chamber and pump them to pressure and it goes away.
And I don't think that they have lost any volunteers.
Like I say, that is not something I am going to volunteer for.
Other factors, this is something that becomes important.
It turns out that the amount of time you spend at reduced pressure is important.
Also exercise.
It turns out that the more you exercise, maybe it makes sense, you're moving your joints around, but the bubbles come out.
Actually, what they have started to do on the Space Station, since we cannot go to a reduced cabin, that while they are breathing oxygen for the first hour or so they do very exhaustive exercise, both upper and lower body on an exercise bike with arm exercise as well.
To try to drive away as much of the oxygen early on in the decompression preparation as you can.
OK. Where does that put us?
As I said, the more nitrogen you have in your blood to start out with, the longer a pre-breathe you are going to have to do before you can go out.
These are the normoxic and hypoxic lines, which I showed you before.
And then these are the contours of the amount of time you have to pre-breathe.
Remember, I told you if you are at one atmosphere, 20% oxygen, your pre-breathe time is 240 minutes, four hours.
This is to get yourself down to an R value of 1.65.
This is not 100% safe because, remember, at 1.65, there still is a 25% incidence of DCS.
Now, the interesting thing, these are the statistics that come from the laboratory trials, we have never had a reported case of bends during EVAs.
And people are not quite sure why.
Some people suspect that maybe even if an astronaut is getting joint pain they are not going to report it because maybe that would prevent you from doing another EVA.
I mean there are enough other pains that you undergo in just using a spacesuit that maybe you don't even notice it.
You feel a problem in your joints or your fingers and you say oh, damn, my gloves don't fit right or something like that.
I have another slide mentioning this later.
There is some suspicion that weightlessness may have an impact, which means that the bends susceptibility on the Moon or on Mars may actually be greater than it is in weightlessness.
We just don't know.
Here is where we have gotten to on the Shuttle EVA.
Again, this is no pre-breathe, this is one hour pre-breathe, so we are at about 40 minutes on the Shuttle.
That is where we are.
Now, this is a design problem.
How are we going to design the CEV?
How are we going to design the equipment, the habitats that we are going to use on the Moon and Mars?
Now we're talking about surface exploration.
And, as I said, there is a lot of uncertainty about the affect of gravity.
But we suspect that it may be more bends inducing than weightlessness.
And so what they are doing for safety and conservatism is instead of using the R value of 1.65, which we use on the Shuttle, they are taking it down to about 1.3, 1.4.
Now, what does that do?
Here is 1.4, 1.3.
Where does that put the Shuttle EVA we now would have to get down to that R value?
We have got a two hour pre-breathe.
To get down to 1.3, we have got a 2.5 hour pre-breathe.
And this is with the normal spacesuit at 4.3 psi.
We are heading an operational problem.
If you want to go out on your geology traverse on the Moon, you have got to depressurize, denitrogenate for 2, 2.5 hours.
I don't think so.
But what are we going to do?
Well, if you increase the suit pressure now the R value, at a given level of nitrogen, goes down.
And so at 6 psi, even for an R of 1.3, you are in the zero pre-breathe range, which would be great.
People have suggested, well, maybe we should build a variable pressure suit so that you could go out at 6 psi, sort of get things set up.
And then all that time counts towards your denitrogenation so maybe then after two hours into your EVA now you can drop your pressure.
And now, if you have any things where you need more dexterity, you will be OK.
But the pressure control system in a spacesuit is a very complex undertaking.
The Russians do have a dual pressure suit.
They never used the lower pressure, to my knowledge.
And it is a big hit to try to design this into the suit.
And, of course, to design a suit to operate at a higher pressure.
The Russians work at about 5 psi, but it means your suit is less flexible, heavier and is just going the opposite direction from the desire to accomplish useful work, which is why you are putting on the spacesuit in the first place.
This is what we are working against.
Again, we have got a systems problem.
It is not just the pressure.
It is the oxygen.
In Skylab, in Apollo, because you were working in a pure oxygen environment, now, of course, we're talking about what is the pressure going to be in the cabin.
You want to keep the cabin at a higher oxygen environment.
Your material selection is highly limited.
As it says, they tended to use a lot of metallic materials.
Well, what do we know about metals?
Particularly aluminum.
First of all, aluminum will burn at 100% oxygen.
But your cabin, even if it is not 100% oxygen, you are going to have a lot of metallic material.
Well, metal, when you're dealing with radiation, you hit metal, aluminum or even higher atomic number metals with primary cosmic rays and you get spallation.
You produce a lot of secondary particles.
And you end up with actually more radiation than you would have gotten if you had just gotten hit by the incident cosmic ray.
What you really prefer is to have low atomic number.
Hydrogen is best.
Water, hydrogen, oxygen and so forth.
If you are forced to increase the metallic content of your spacecraft because of the oxygen flammability problems now you are going in the wrong direction for radiation protection which becomes an issue when you're dealing with long duration stays on the Moon, interplanetary transport and so on.
So, it is all interrelated.
You cannot change one thing without changing the other.
Flammability.
Well, we have been talking about that.
I won't stay on that.
But, again, it is very dependent on oxygen.
Oxygen restricts the use of non-metallic materials.
But nonmetallic materials are what we would like to use from a radiation point of view.
You would like to have a lot of hydrocarbons, you know, polyethylene plastic in your spacecraft because they absorb cosmic rays.
But, in any case, it is pretty well agreed that we never want to go above 30% oxygen.
It just becomes too restrictive on the types of materials we can use.
That is going to be a design constraint.
And, remember, we are aiming towards a systems level design of our long duration space habitats which are EVA compatible.
30% oxygen.
Now we are back to this.
Now we can draw a red line.
And we have got to stay on the left of that line.
We have got to stay above the blue hypoxic line.
Now, also remember that we have got these pre-breathe lines.
Let's say we are going to limit pre-breathe to no more than one hour.
Well, we are cutting down on our design space here.
We have got the oxygen flammability which is cutting off to the right.
We have got the hypoxic limit which is cutting off in this direction.
And we have got the pre-breathe limit which is cutting off there.
We are kind of in a, I won't call it a box, a rather small triangle.
Now, if we go to a 6 psi spacesuit, as I said, we open up the design space a lot.
But at a price of flexibility, maneuverability and the ability to do the things that we want to do in the first place.
Where are we going to go with this?
As I say, it is an active area of research.
I cannot tell you what the answer is going to be but it looks like they are going to probably be going for 8 to 9 psi with an oxygen concentration approaching 30%.
And that the CEV, from what I have been told, is most likely going to be designed with a variable pressure capability because it also has to be able to dock with the Space Station which means it has to be able to take one atmosphere.
And also, because the CEV by itself is not going to have an airlock, if you would have to do an emergency EVA out of the CEV, for whatever reason, you are going to have to depressurize like we did back in the Gemini days, or Apollo.
And, of course, that is going to affect the design of all the other systems on the CEV because they will have to be able to operate from a vacuum all the way up to one atmosphere.
Just in the end, I hope what you have gotten out of this is, again, we are trying to look at things from a systems engineering point of view.
And this is one big system where EVA cannot be considered just on its own.
I gave an example of how it affects the RCS system with Hubble.
Here it affects the environmental control life support.
Flammability, radiation protection, it is all linked together.
This was the recommendation that we came out with, slightly below 9 psi pushing 30%.
But, as I said, there is a lot of research that has to be done to understand the way bends behave at partial gravity.
How we are going to do that?
I have no idea.
As I said, for the International Space Station, we have to be able to go up to regular atmosphere.
We were making this presentation as part of the study we were doing last year on CEV, but it was so relevant to what we did here that I didn't see any point in modifying it for this class.
But we had to make these decisions on the Shuttle based on similar calculations looking at the design space.
The difference was with the Shuttle we were using an R value of 1.6, 1.7.
And so we were able to get down to a 40 minute pre-breathe.
What we do with the Shuttle is about the day before you are going to do your first EVA you actually drop the cabin pressure.
And when we had the lecture on environmental control, remember there is a cabin pressure controller at 14.7 and a pressure controller at 8 psi.
But when they designed the Shuttle, nobody was thinking about EVA or about this pre-breathe time.
And so, there is no 10.2 controller.
We have to do that manually.
You drop it and then you add a little bit of nitrogen or a little bit of oxygen, depending on what the instrumentation tells you you need.
And then there is a periodic maintenance which you have to perform in order to keep the proper gas concentrations.
And then we leave it down there for the duration of all the EVAs.
And then, when the last EVA is finished, then you turn the 14.7 controller back on.
Now, of course, since you have the proper oxygen partial pressure, most of the gas that comes into the cabin is nitrogen.
And, actually, I think we may have mentioned this before, it comes in the bathroom.
And so, for the time when you are re-pressurizing the cabin, the bathroom is off limits.
And I think that is basically it.
The timing has been pretty good.
Questions?
Comments?
Yeah.
Is anybody looking at using a fully one atmospheric spacesuit?
Actually, it turns out there is a principle called holdings principle.
Let me get this up because I have some pictures of what we were doing.
If you change by less than a factor of two you don't get the bends.
This is an empirical finding.
And so, the idea was if you could have a suit that worked at about 8 psi you would have a zero pre-breathe suit.
And I participated in a bunch of tests.
These are some other robotic things.
This was the hard suit designed by Ames.
Because it is a constant volume suit it is not sensitive to the pressure the way that a soft suit is.
And we did a lot of tests, actually.
We always came unstuck on the gloves.
Nobody has been able to design an 8 psi glove that really gives you sufficient mobility.
I mean it is a dream.
People would love to be able to do it.
If we could figure it out that would solve all these problems.
Although, if you are going to Mars maybe you don't want full pressure because if you have one atmosphere compared to 8 or 9 psi your structure has to be that much thicker and heavier.
They went through some of those calculations.
This is now on the hand with the Space Station.
And they figured that for all the meteorite shielding and everything they had to put on the outside of the Space Station that actually the change in thickness that would be involved in changing the pressure from one atmosphere to another really wouldn't make that much of a difference.
All of these things, again, they are interrelated.
But it is a good point.
That is why I showed those designs for 6 psi because people are still thinking suppose we could design a spacesuit to work at a higher pressure.
But right now we don't know how to do that.
I hope everybody has a very happy Thanksgiving.
We will see you a week from today.
And if there are any questions, again, about either your oral presentations or the written presentations, I will be here today.
And then I am gone for the rest of the week, but I will be looking at emails so you can send me emails and we can discuss things.
OK.
Today we have an old colleague and friend of mine.
Tony Lavoie actually got his engineering degree in the Aero-Astro Department here at MIT, so this is coming back home for him.
And he actually grew up in Massachusetts.
But, for the last 23 years, he has been in Huntsville, Alabama at the Marshall Space Flight Center and actually has been an engineer on quite a few of the projects that I have flown with, including the Astro Observatory and the Tethered Satellite.
And Tony has continued to rise up in the ranks of the NASA engineering community.
He is now, I guess, the Project Manager for the Robotic Lunar Exploration Program.
If you have been following the space news, Marshall Space Flight Center is going to be developing the first robotic lunar lander that we have ever sent to the Moon since Surveyor.
It will be like 40 years, I guess, we haven't done that.
And we are going to have to figure out how to do it.
Tony was also the Chief Engineer on the Chandra X-Ray Observatory which was launched on the Shuttle.
And so, I thought it would be interesting to hear, since we've been looking at many other aspects of Space Shuttle operations, to hear something about what it was like getting a payload ready to fly on the Shuttle.
But also we were discussing, before class began, that he may have some comments on the Robotic Lunar Exploration Program, which is now in the pre-phase A studies.
And it will sort of be interesting.
We talked a lot about what it was like in the pre-phase A and the early days of coming up with the requirements for the Shuttle.
And clearly, in the systems engineering, getting the requirements right is one of the key goals for success.
Tony may have some comments on that.
I have talked enough.
Tony, you've got it.
Hi there.
We are going to start talking about Chandra.
And, as Jeff pointed out, I am also going to talk about my new challenge, my new assignment as Project Manager and in pre-phase A what that means.
And I think it is a lot different, being a student, where you have a single closed form solution and you go and you work a problem and it is done.
In the real world you seldom, if ever, work a problem one time and have it finish.
And we will get into that as we go along.
I am going to talk about Chandra X-Ray Observatory.
And these are just a few images that Chandra has produced.
It is an outstanding x-ray imager, and it has been flying since July of '99.
And you are going to get all the gory details and all the lessons learned that we had in working that project.
I will do an overview Chandra history and then the challenges we faced and lessons learned associated with that, and we will cover those topics.
In terms of how we operate, feel free to ask questions as we go along.
It is kind of lose and easy.
Chandra, or back then it was AXAF, Advanced X-Ray Astrophysics Facility.
NASA, by the way, loves acronyms.
I mean, if you are interested in working with NASA or associated with NASA, they are just acronym crazy.
AXAF was one of the four great observatories.
The four great observatories were meant to cover as close to the entire spectrum as NASA could.
And so, the visible great observatory is Hubble.
And Ultraviolet, that's true, little Ultraviolet.
The X-Ray Observatory is AXAF, which was later named Chandra.
The Gamma Ray Observatory is the Compton Gamma Ray Observatory.
And the last one was Infrared.
And that originally started off as SIRTF.
And now it is Spitzer.
And that was launched a few years ago.
In each of the four great observatories, they were higher class than most of the other science missions, even the science telescopes.
And NASA did invest quite a bit of money on each of those.
And so, you will see that the performance from the four great observatories are, in terms of quality, probably the best in the world and the best NASA has ever done.
For the x-ray region, Chandra or AXAF was built.
And its objective, you start with program objectives and you work down with science objectives.
The science objectives were really to understand the nature of the universe and determine the nature of celestial objects, in particular those that are hot.
And hot objects will produce x-rays.
And, of course, Jeff knows all about that.
If you have any questions related to that you can ask Jeff.
But, from an engineer perspective, what happens is you are given a set of objects and maybe a program objective, and now you have to craft the mission and craft the project from those.
And one of the problems that you face is that sometimes they are not crisp and sometimes they are flexible based on cost.
And also sometimes they are not very specific.
Like determine the nature of celestial objects from stars to quasars.
Now what?
But this is the kind of thing that typically you start with.
And you have to iterate with, in this case, the science community.
If you are building part of the Shuttle, you have to iterate with, for instance, the users, the astronauts, the payload developers that would use the Shuttle, the operators, et cetera.
But the process is the same.
You start with the top level objectives and you work your way down.
Now, again, because of the nature of what we are talking about, it is usually a very iterative process that can last for several months.
And even several years if there is a technical challenge that causes you to stretch out.
And I will mention when it started.
In terms of AXAF, kind of the key parameters are the percent in circled energy, i.e., how sharp is the image?
Registration meaning where is the target in the sky?
And you have got to remember that x-ray astronomy is a relatively new branch of astronomy.
Because of the atmosphere it absorbs x-rays.
So we, humans, only discovered x-ray astronomy in the last 40 years.
We need to go outside the atmosphere to get information on x-ray astronomy.
One of the things that we did know is if there are some x-ray sources there they may or may not be the same sources that also emit invisible light.
And, if they are not, then you have a question of you kind of know there is a source out there, but you don't know where in the sky it is.
And so, one of the challenges that AXAF had was even if I have no stars around this x-ray source, I have got to be able to pinpoint where that x-ray source is.
That was a particular challenge that we had, trying to figure out how to do that.
And that is what registration means.
And then effective area is related to how many photons you can collect to make a good image.
And recognize that if you are looking at the sun, the sun is very close so you can get a lot of photons, but if you are looking at something that is ten billion light-years away you are not collecting very many photons so you have to wait a long time to get those photons, unless you have a big collecting area.
Those are kind of the key performance requirements that you have to address when you are building an x-ray telescope.
From those there are key derived requirements that have to be derived basically to be able to maximize the scientific performance requirements.
And so, from these pieces of information, you derive mirror size and design, you derive focal length.
Pointing and control requirements, how accurately do you need to point?
Thermal stability because that plays a role in orbit, a very big role.
Instrument sensitivity and the fiducial transfer system, which I will talk about later, which is the thing that allows you to do the registration, even the absence of local visible stars around it.
Once you have the bulk of the science performance requirements and the derived requirements from these performance requirements then you fill in kind of the rest of the pie, if you will.
And those can be safety requirements in the case of a manned program.
In fact, that is pretty significant.
And it is a pretty significant cost driver for things that fly in space.
Unfortunately, they also cost a lot to make sure they are not a hazard to the crew that flies.
There are also design and construction standards, which also turns out to be a big deal.
For instance, if you are building a satellite and you have circuit cards, you are going to have requirements on soldering, you are going to have requirements on integrated circuits, you are going to have requirements on glazing, you are going to have requirements on bonding.
You are going to have requirements on how to do things across the whole gamut of the spacecraft.
Not just the spacecraft but also the Shuttle and anything that flies up there.
For a given mission, there are probably on the order of 100, 150 separate documents that describe various things, design and construction standards that you have to go through for a particular piece of building the vehicle.
And so, those things are always a challenge for a project manager because the engineers want to get the latest standard that applies, and in excruciating detail as engineers love to do.
And yet, from the project manger's standpoint, we are trying to maintain cost.
There is always a tradeoff between how good is good enough?
From a program manager's perspective better is the enemy of good.
Yet, from an engineer, good is always better, or better is always better.
There is always a healthy tension between the project manager and the engineers.
And I would say that a successful project knows how to tradeoff.
The project manager and the engineers know how to balance that tension to be able to get the best product.
And we have done that before, and we have also imbalanced it and had some spectacular failures.
Once requirements are set then you can start kind of designing and starting to put the drawings on developing the system, if you will.
But that is kind of the general flow of information for a science payload or a science mission.
You start with the top level requirements for performance, the scientific performance that you are looking for.
From those you derive the direct performance requirements, like what you see here, and then everything else cascades below that.
And, once you have the requirements set, then you can start developing concepts.
Now, this was a preliminary design of AXAF.
I am not going to talk about it, but I am just going to show the next slide as to what it looks like now.
And so, you can see that there is a major difference between what it started out as and what it ended up as.
And this is typical.
It is not atypical.
And the reason is cost.
A lot of times when you are first starting out with requirements, you kind of don't know what the cost of the mission is going to be.
And so, you come up with an idea that has a lot of capability.
This was orbit serviceable, it was in low earth orbit, and it had four focal plane instruments that you could select.
And it would actually move, much like Hubble.
And it was a 15 year mission with reservicing with the shuttle astronauts.
But, when you start putting that on paper and you start crafting the requirements, one of the things you do, in the early phase of a program, is you also cost, or you try and cost what those requirements result in, in terms of design.
Even though you don't do the final design or you don't start working on the design in detail until after you've got the requirements set, in practical terms you have to still put together a concept design so that you can cost it.
And that is exactly what happened on AXAF.
We put together a design in parallel with working the requirements and we costed the design.
And it turned out that that was more money than NASA could afford.
And so, as is typical, the iteration process begins with headquarters to change the parameters of the mission.
To, in a sense, compromise the scientific objectives a little bit in terms of implementing a requirement set.
And the result is finally you get something that you can pay for and that still meets the intent of the scientific objectives.
That is a key lesson to learn.
When you are first starting out, when you are first crafting what you want to do, except it to change.
And the key there is it is usually cost driven as to what you can afford.
Not only that, NASA, just by the nature of what it is doing, since it has never built an AXAF before, it is really hard to get a good, accurate cost of what it costs.
So, a lot of times, the cost is derived from a parametric usually weight-based system such that, for instance, if the mirrors or if the solar rays, you want them this big to provide this much power.
That big means they have to weigh or they mass so many kilograms.
And so that is about X amount of dollars based on weight.
Now, there are a lot of additional factors that you put in like complexity, interfaces, et cetera, technology readiness, whether you are looking for state-of-the-art or something that has already flown.
But generally it is what is called a parametric cost model.
And it is not based on taking a look at all the design drawings and saying this cost this much from the vendor, this cost this much, and you put it all together.
That is called a bottoms-up assessment.
And it is more accurate, but it also relies on having an accurate design picture which obviously you don't have early on in a program.
I mean it is almost like a black art to try and take a requirement set and craft a design early on or at least bound the design and bound the cost.
Because, again, early on what you are trying to do is get into the cost envelope and still meet the scientific objectives or the objectives that you are trying to do.
Yes.
In this final design, how are its capabilities versus the original?
I mean because you didn't put as much money into it.
Correct.
To make it on-orbit replaceable means that all of the boxes that you see in the spacecraft bus, or a lot of them would have had to have been, on this picture, replaceable.
They were all replaceable.
You have to spend some money making sure that the drawers slide in, slide out, there are no sharp edges and it is accessible, et cetera.
So, packaging is important.
That is one area where you can reduce cost and not really have an effect on performance, per se.
However, one of the big areas that we did take a little bit of a hit on performances is on the mirrors.
Now, x-rays, you cannot just have a regular shaped lens.
What we use is grazing incidence mirrors.
And I will explain this later.
Because the x-rays are so highly energetic, you have to graze them gradually to a focus.
And the original concept had six mirrors nested.
They were basically hyperbola parabola shaped.
The final concept had four.
We had to remove, not the inside or the outside, but two of the middle mirrors.
So, the mirror set is four.
This orbit that this flies in is a highly elliptical orbit.
It is about 140,000 kilometers by 10,000 kilometers.
The original orbit was in low earth orbit which is about 350 kilometers.
And so that affects the viewing time.
But, in all of the other registration and encircled energy, I think that we still were able to meet the original objectives of the scientific mission.
There are ways to compromise that don't really affect directly what the scientists want to do, but it is a tradeoff.
It is usually not their first desire but they signed up to it, they are comfortable with the final answer.
Explaining a little bit of how it is put together.
This is a rather simplified version.
Here are the solar rays.
That is where we get our power.
This is the spacecraft bus.
This is the HRMA which is the mirrors, the nested cylinders that I was talking about earlier.
This is a low energy grading and a high energy grading.
For spectroscopy, what you want to do is you want to bend all of the photons in a particular energy much like a prism does for visible light.
And the way you do that is with these facets.
There are probably a thousand facets on this.
And I have a picture of that later.
And you flip it into the beam when you want to get some spectroscopy data.
And you remove the grading when you want to get an x-ray image of what you are looking at.
And that is a function of what the scientists want to do at a particular target.
Some scientists want to get an image because that conveys more information for them than spectroscopy.
Others want to see a spectrogram of the image because that tells them what the photons are and what the energy of the photons are.
And that can tell you, for instance, what elements are present and what temperatures for a given gas cloud or neutron star or what have you.
And then we have an optical bench.
And all that means is it is a piece of structure that is very stiff.
Because obviously you can tell if you flex a lot that sure doesn't help your camera imaging the performance.
It has got to be very stiff, and that is why they call it an optical bench.
And the ISM is the Integrated Science Module.
That is where the instruments sit in Chandra.
And here is a picture, as I mentioned before, of how the x-rays graze off of the cylinders in the HRMA down to a focal point about ten meters away.
Jeff had mentioned that sometimes it takes quite a while for science to come to fruition in a mission.
And this is probably a good case.
It started in 1978 with some concepts.
We did get approval for new start in 1988, so you can tell there was ten years time where this was just an idea in a scientist's mind.
And typically the course of events is that the scientist lobbies and submits papers, submits proposals to NASA, lobbies Congress, lobbies the National Science Foundation to say this is a good idea, we should do this and here is the concept.
And, of course, that concept changes over time as you get new technology, et cetera.
And so, that is exactly what happened here.
We did get approval ten years from when the original concept started.
Authority to proceed for the prime contract was in January 1989.
What this means is that is when we hired a contractor to build the spacecraft.
And the start of that process is called ATP.
We had two separate ATPs.
Sometimes you can say I can let out one contract to a company and let that company buy the science instruments or I can compete the science instruments separately.
And there are various pros and cons, but in this case NASA decided to compete the instruments separately.
From an engineer's point of view, one of the things that should peak your interest or note that when you have separate contracts you always have a question of integration.
If you are going to compete separately, you are going to have two separate pieces.
Who is going to integrate them?
The integration job will be harder when you compete them separately.
Now, your performance is probably better when you have direct insight into the instruments and direct insight into the spacecraft contractor.
But the penalty is now, when you integrate the two, you have to make sure that works.
And so, as a project manager, as a systems engineer you have to make sure you apply enough resources to make sure they work together and continue to work together as they are defining the interfaces, for instance, between the two elements.
This was interesting in that in order to get it funded, Congress said we will let you build an AXAF but, along the way, you have to do some testing to show that you can meet the performance that you say you can meet.
And so, we were required by law to have a test of the mirrors in June of '91.
And we also had another test later on, the same thing, to verify that we could get the performance that we showed on paper with actual hardware before we committed to fly and before NASA committed to spend the rest of the money to fly.
And so, sometimes that happens, sometimes if you have a smaller program it doesn't go to Congress so you don't get into those things.
But, when you have a great observatory and NASA is spending a lot of money, a lot of times Congress will get in and say yes, you're constrained, but you have to show me along the way that you can do this.
I wouldn't be surprised if Congress does that for the CEV, the new vehicle coming up, or the CLV.
In fact, for the CLV, the launch vehicle, there is a push to have a demonstration test early just to verify that we can do it.
Now, during this time, of course, we are still formulating requirements.
And we rack up the cost.
And, low and behold, NASA cannot afford it.
We went through a program restructure at that time where we dropped a couple of the mirrors, we changed its orbit, we removed crew servicing.
And you could tell that the whole shape of the spacecraft changed.
Again, that is unfortunately not atypical.
It happens quite often.
Yes.
What was the reason for the orbit change?
Good question.
In orbit there are Van Allen Radiation Belts.
And we really cannot observe too well within those belts.
Now, in the first mission, the low earth orbit mission we were below the belts so we could operate.
But the problem with low earth orbit is you have this big earth, you are close to earth, and so you need power from the sun.
And, guess what, the sun goes behind the earth.
And so, you are eclipsed for a good percentage of the orbit time.
When we had a 15 year mission in low earth orbit we said since we don't have servicing, the lifetime cuts down to five years.
Now, if I am five years in low earth orbit, I have just lost two-thirds of my mission time.
So let's see if we can crank up the orbit, get as far outside the radiation belts as we can.
And then we have basically 100% visibility, the whole orbit, if we can get completely outside.
The problem with that is that costs propellant because the radiation belts end, they breathe, but roughly around 60,000 kilometers.
We could raise apogee to 140,000, but we didn't have enough money to raise perigee.
It gets down to money again.
So, perigee stayed at about 10,000 kilometers.
The resulting orbit time, if you looked at the integral of time outside the radiation belts, was probably about 70%.
It was a compromise.
We did end up losing some overall time, but about 70% of the orbit is usable for viewing.
That is the strategy and that is why we went to a different orbit.
As you can tell, there are a lot of things that are traded off during that time to fit within the constraints that you have got.
Again, that is typical.
The three reviews that are kind of standard reviews for all programs and projects are you start with a requirements review.
And that is pretty basic.
It is held early on.
Requirements have started before that review.
For science missions, you start with science objectives.
And you are percolating science requirements.
And the system requirements review is really a review that baselines, if you will, and sets the requirements as firm as you can.
And it is usually after you have gone through the cost gyrations, but it is done prior to doing any of the design reviews.
Once you have a requirements review done, it was in December of '92, almost two years later we had what is called a preliminary design review in November of '94.
The third review is called a critical design review in February of '96.
What makes a preliminary design review and critical design review has to do with the maturity level of the design.
For critical design review, supposedly you have on the order of 90% of your drawings complete and on the order of 10% of your hardware built.
And I think for PDR it is about 10% of your drawings built, the final drawings.
That kind of gages what you are talking about there.
And it is a typical milestone that NASA uses for all of its programs.
Usually these are set early on in the mission, in the program milestones.
And, for political and for programmatic reasons, you tend not to deviate.
You try and meet those milestones.
Even if the project is not as mature as you would like, usually that is one that you try and keep hold of.
Maybe you don't have 10%, maybe you have 5% or maybe you don't have 90% you have 70%.
Sometimes you do go and you hold the milestone, you hold the review, but when you are not mature enough sometimes you have to hold a delta review to catch up.
And that is, again, something that NASA does for various reasons.
If the situation is right, sometimes you can slip and allow a single review at the right time.
Most often it is driven by nontechnical things that require you, either by your customer headquarters, they don't want you to slip so you tend to hold it where he said he wanted it.
And then we had the mirror delivery to the calibration facility in November.
It shipped the whole thing to the Cape for launch in February and we launched in July.
You can see we spent about five or six months down at the Cape integrating into the Shuttle testing, and then we flew in July of '99.
Those are the orbit parameters.
We achieved the final orbit August 7th, so you can see about two weeks.
And the way we did that is the Shuttle only puts stuff to low earth orbit, so you need some additional propulsion.
We had an upper-stage that got us part of the way there, used most of the energy.
However, it wasn't at the final orbit.
So, integral to the spacecraft, we had a propulsion system that had to do five additional burns to get us to the final altitude and the final orbit parameters.
That is why it took us about two weeks.
And, again, that is not atypical.
When you are not operating in low earth orbit, it takes you while to get your final orbit.
And these are a couple of interesting parameters.
Safe mode, we will talk a little bit about that.
Spacecraft are designed such that if -- Yeah.
I am just wondering if the inclination affects the requirements at all or does it not really matter so it just stays [NOISE OBSCURES]?
The 28.5 is driven largely by the launch site at Kennedy Space Center.
Right.
I am just wondering if the science requirements could drive the inclination [OVERLAPPING VOICES].
For some missions, yes.
We were fairly incentive to that, as long as we took advantage of launching at KSC at the optimum inclination.
But there are missions like Space Station.
Because you have a launch from KSC and a launch from Russia, to optimize the performance of both launch centers then the orbit is 58.5.
Now, what that means to KSC launch is that you pay a significant penalty.
You pay about a 30% weight penalty for launch at KSC to the Space Station.
Sometimes the answer is yes, you can optimize inclination, and sometimes, when you have a large program and multiple launch sites it is a compromise.
So, it is not efficient for each site.
I was mentioning spacecraft typically have their avionics systems.
And typically they do not contract the ground.
They are not in communication with the ground all the time.
And so, for Chandra, that is the case.
We may have "contact" with Chandra probably about 5% to 10% of the time per day.
In the meantime, figure that 90% of the time it is out of communication range.
Now, this is something, by the way, that is somewhat different from a manned mission.
The manned missions tend to optimize and maximize communication coverage.
When Jeff is flying in the Shuttle we probably have 90%, 95% coverage over an orbit and over a day for communication.
But, in science missions, rarely is that the case.
And so what that means is the whole philosophy of operating a science mission is based on stored commands.
And so, we send up a stored command load.
And that load is good for a day.
And basically it steps through and automatically executes based on time.
What that also means is you have to design a spacecraft to recognize when it has a problem.
If a hardware box has a failure, it has got to be able to recognize that and go into a safe mode so that the ground can then recover and configure the systems to continue to operate.
That is one part of flying a spacecraft, is building a robust safe mode to be able to do that.
And our first problem occurred August 17, 1999.
Now, note it was a ground error.
As usually the case, spacecraft are pretty complicated.
And we probably had 400 or 500 separate procedures and commands on the ground for doing certain things.
And, even though we try and test them out individually, the combination of those procedures and in the execution, sometimes you put the spacecraft in the wrong configuration with one sequence, and then you follow with another sequence.
Low and behold, the software that you designed for safe mode says I am in the wrong configuration.
Sorry, I am going to safe mode.
And so, that happens.
We have had only probably three safe mode events for six years, so that is not too bad at all.
Hubble, for instance, has had probably an order of magnitude more than that.
I think we have gotten smarter based on our design of Hubble to design this.
But, again, typically spacecraft do have safe mode designs in them.
Given our orbit, we still have a small eclipse season.
Twice a year in the fall and in the spring we have eclipses.
And so our first eclipse season was then.
This is the translation mechanics for the science instruments.
This is where the telescope is.
I won't go through that.
This is kind of where we purchases or we contracted for the various instruments.
This is a CCD imaging spectrometer.
In fact, it was both Penn State and MIT that produced this one.
And that is one of the focal plate instruments and the other one was a high resolution camera using micro-channel plate technology instead of a CCD.
And that one was from SAO just down the street in Cambridge.
The high energy transmission grating, as I had mentioned, was also from MIT.
And the low energy grating was from the Netherlands.
And this is a good point to make also, is that a lot of times for science instruments or for science missions, NASA tries to lower the cost of the mission.
And the way you can do that is you can reduce requirements or you can get a partner and have the partner pay for an instrument that you want to fly.
And so, in this case, as is typically the case for science missions, NASA tries to go and get international partners that will bring to the table an element or a piece of hardware or some software that reduces the overall cost of the mission but, at the same time, maintains its capability.
That is what we did with the low energy transmission grating.
We got the Netherlands to basically design and build that and fly that in exchange for that kind of arrangement.
They get a percentage of viewing time that they can say for 5% of the viewing time we get to point the telescope to wherever we want.
And so, typically that is kind of the handoff between our international partners and NASA in trying to reduce the overall cost to NASA for a given mission.
And these are the gratings, as I had mentioned.
And there are a bunch of facets along these.
Now, notice these are the four mirror shells where they co-align with the four mirror shells, the inner most and the outer most.
And this is kind of how it works.
The gratings get flipped in.
And this is the array of CCDs.
Each one of these is a CCD.
And the pattern of light or the pattern of x-rays follows along these lines when the x-rays actually come in.
That is what actually you see.
Now, Chandra does have a ground system architecture.
We communicate with the deep space network.
That is how we communicate because it is not in low earth orbit so we cannot use the satellites that are around the earth.
We get that signal from that satellite and it is beamed to the control center, here in Cambridge, in fact.
And we do various things on the data.
We process it out and send it to the general observers.
Now, the way this operates is that every year this control center or the science folks operating the control center solicit objectives.
OK, science community come and request time and give me a proposal for viewing time for what you want to do, what target you want to look at, what is the configuration of the instrument, which instrument, et cetera.
And so, the Chandra folks go through, and they probably collect about 800 proposals a year from all around the world.
They will select roughly about 200.
And it is not based on numbers, it is based on time.
You have a certain amount of seconds in a year that is available.
And so, some of these proposals have large times, some have small times, but roughly it is about a four to one.
About 200 are selected, 800 are proposed, so that's good, you're oversubscribed, you have a lot of interest.
NASA does fund those that are selected.
So if you are a PI and you say I want to go look at Crab Nebula or I want to go look at a Quasar somewhere, and here are the reasons, here is what that will tell me.
And it gets selected.
NASA will fund you some money to be able to do that observation and process the data and publish.
Now, NASA won't fund any that were selected from foreign countries.
So the foreign countries, they have to get their funding, but we will allow them viewing time.
And that is also typically the case.
And, by the way, all of these scientists across the world never have to come to Cambridge.
It is all done electronically.
They don't get their data real-time.
They get it probably a few weeks after the observation.
And the reason for that is there is a lot of processing.
You don't actually send, or you can, but usually you don't send the raw data out to the user.
It is heavily processed, and the products of that processing are what is actually sent out.
Now, there are tools that the Control Center also provides, but generally it is the process data that is sent.
Yes?
What sort of data is this?
Images or numerical data?
It is data that can be used for images.
It can be used for registering each photon, what energy it is.
There are a lot of different choices of data.
And some of the data is overlapping, but it does give you information on energy, location, number count, where in the field.
It can be used to then process an image or process a spectrogram.
And you do that by using the tools that are also provided.
It is like a viewer or it is like a piece of software that allows you to view what is in that data.
But the data is just the raw data.
It is just a series of tables.
And, as we said, it began in 1978.
I picked the prime and was given a new start.
I am not going to go through these.
It was restructured.
We went to four mirror pairs.
We dropped two focal plane instruments, dropped the servicing requirements.
I mentioned that.
Interestingly enough, one of the instruments we wanted to create a separate spacecraft just for that instrument.
And, believe it or not, the addition of this spacecraft and this one for the spectroscopy mission was going to be cheaper than the original concept.
But again, due to cost, we continued this spectroscopy spacecraft for about a year to two years.
And then that was cut due to cost.
So we ended up flying just this one spacecraft.
And the result is the Chandra Observatory that we have now.
Note that sometimes it is not within your control as to whether your program is cancelled or not.
We were doing quite well.
It was within our cost envelope that they had given us.
But, at the time, there were other priorities at NASA and there were other overruns at NASA.
And they have to weigh.
Sometimes they say, well, you can have this much money for these many years.
And every year they re-evaluate.
And that is because you don't know how much things are going to cost because NASA typically builds one of a kind things, and you don't know really how much it costs until it is already done.
So that re-evaluation process continues now.
And it will probably continue for as long as NASA flies.
So imaging and spectroscopy is what we do.
Why is imaging so important?
Well, clearly this image was a Rosat image with the highest technology at the time.
And here is the Chandra image.
And you can see the neutron star right there.
And there is no way you can see a neutron star in there.
And they can learn a lot more from this image than they can from this, so that is why scientists push to get good imaging resolution as well as good spectroscopy.
The biggest challenge for us on Chandra was the mirrors.
Now, recognize that due to the analysis that we performed early on, yes, this was all theoretically possible, but we had to be able to polish the surfaces of those mirrors, those grazing incidence mirrors to an accuracy of on the order of angstroms.
And that is pretty small.
Can we measure that?
The answer is no.
At the time, we couldn't even measure how accurately we needed to polish the mirrors.
So that tells you that, boy, we've got our work cut out.
We had to work with the National Institute for Standards to figure out how to first measure how accurately we can polish it and then ended up polishing the mirrors.
And polishing, as a process, getting down to that smoothness clearly is a huge challenge and takes a long time.
And so, that was probably our biggest challenge, is to polish the mirrors.
Metrology is the science of working mirror technology and getting the surface figure of the mirrors correct.
So, really, the key to success is developing three different types of metrology measurements that are independent, that allow you to cross-check.
Because you are talking about things you have never built before and you are pushing the state of the art and you don't know how to measure it.
The key there for an engineer should say, OK, I need not just one way of checking my work, but I need at least two ways, and preferably three independent ways of making sure that I am doing the right thing.
So when you're pushing the state of the art, a good lesson is try and get into a position where you have got more than one independent check of your analysis to make sure it is correct.
So that was a very big challenge for us.
And, again, that is kind of how it goes down.
This is the general shape of the mirrors.
And here are the mirror pairs being assembled.
And, of course, one thing about the mirrors is when you are talking about that surface finish, you cannot just assemble it in your garage with dust around.
You have to be extremely clean.
And, believe it or not, one of the dirtiest things in a laboratory is a human.
And so, a human has to be almost completely covered to prevent any kind of accumulation on the mirrors themselves.
In fact, we had to limit the exposure time of humans to the unpolished and polished mirrors during this timeframe because contamination is an incredible problem when you are talking about the atomic scales of polishing the mirrors.
Again, I mention that the important thing is to make sure that you are doing cross-checks in the metrology as you are going along.
Because, again, you don't know exactly what you are doing.
I mean on paper, yes, you've got an analytical solution that tells you the right answer, but you have never done it before.
You have never built it before.
You don't know how to measure it.
So you have got to work on building confidence that your analysis is correct.
How do I know that analysis is correct?
And so, you have got to think about a sanity check, if you will, to make sure that those are correct.
And, of course, the best way for proving that it is correct is test them.
And that is what we did.
Mandated by Congress, we tested the mirrors.
Now, the thing about that is so I test the mirrors.
I am testing the mirrors in a 1G environment so, even after the test, how do I know that those results are correct?
Because the 1G deflection, even though you think it is small, it does affect the mirror performance on the ground versus in orbit.
And another thing is the finite source distance.
You cannot get a source that is infinitely far away on the ground just because of the nature of putting a source out there.
Our source was about, I think, a quarter mile away.
And we had to make analytical corrections to compensate for how we expected the spot size to change because of that finite source distance.
We had to analytically correct the image on the ground to compensate for the 1G affects of the mirrors.
And so, even a test you would think, OK, go test it.
Well, it is not straightforward because, even in that test, that is not a true representation of how it would work on orbit.
So it is a lot more complicated than one would think.
What do you use your x-rays for?
Various things.
We had some monochromatic x-rays at various wavelengths.
We had registration x-rays.
Iron.
A certain wavelength of iron is a big one to use.
You don't have a picture of the big tunnel, do you?
I think I do in here.
I have got a picture of it and I can talk about.
But that is another thing, is when you are testing, how well do I know my source?
And that is all factored into verifying that you have got the performance correct, because if your source is all over the place and you haven't characterized your source well enough then how in the world can you say that the mirrors are that good?
So we did end-to-end testing.
And, again, you don't just look at the raw data and say, yeah, I am there.
Even in the ground test, a lot of times you have to massage the data and compensate analytically for factors that you cannot control.
And the key to success there is good systems engineering.
And a lot of that is because of multiple separate parties, academia, nonprofits and foreign groups all involved in the workings of AXAF.
You have got to make sure that you have got a team that works together.
Lesson learned, as I had mentioned, perform multiple cross-checks either via test or analysis with a different tool as possible.
Another lesson learned is let more than one group perform the review.
A lot of times, for Chandra, we had the Marshall engineers, but we also contracted SAO down the street to kind of independently assess, using their own software tools, where we were.
And that was a good idea because sometimes we had good ideas that we could incorporate and sometimes they had good ideas, but it gave us a sanity check when we both matches and we both felt good.
There is also no substitute for direct test or measurement.
Analytically, we have wonderful tools nowadays, but they will never take the place of testing.
Another thing that was very important for us because of the iteration cycle with headquarters on funding is make sure that you keep the science or the scientists that are part of the mission informed and part of the decision-making process.
One would think, and one would like to think that this is a technical project, all things are technical.
No, there are personalities involved and there are human responses involved.
And so, to keep the team together you need make sure that the science is informed and supporting your decision-making process.
A lot of times that is overlooked, but it is, nonetheless, very important as you are building a project.
And the final lesson learned is, as we are going through and making those mirrors, you construct what is called an error budget.
You assume perfect performance and then you kind of step back and say I am going to allow imperfect performance in this region.
And I am going to allocate that imperfect performance to thermal design.
I am going to allocate imperfect performance to the inaccurate knowledge of knowing the source positions in the sky.
I am going to allocate some error budget to flexing of the optical bench.
I am going to allocate some error sources to the difference between how long the focal length is and how long I think it is.
And so, you take each of the error terms.
And for Chandra we probably had a hundred or more.
And you go through and you verify that your assumption in that error budget term was correct.
And usually, for the critical ones, you did it in more than one way, more than one method so that you were sure that you had those error terms defined.
And the end result was we had mirrors that performed better than expected.
And, as a result, the payoff is just wonderful.
This is a time-lapse series of images.
And, supposedly, this wave front was moving out from the source pulsar at significant fractions of the speed of light on the order of 20% to 30% of the speed of light.
That was pretty cool.
And if you need more information just talk to Jeff.
But that was a pretty big payoff.
The next challenge that we had was the programmatic challenge.
Again, as I had mentioned, the normal process of going through a program in a project is you are first trying to size the mission.
And, really, you are sizing it for cost.
And that is very hard to do when you are building something new.
And that was a very big challenge.
And so, we changed a lot of things.
But, eventually, the key or the critical thing that you are trying to do is I compromise on this but I want to still try and maintain performance.
We maintained our imaging resolution performance and our registration performance.
And we compromised on a few other things but we maintained that performance.
Finish this slide and then we will take a quick break.
Note down here that we also needed to reduce the weight by a factor of two from our original designs.
Our original designs were pretty simple aluminum structures.
Our final designs were the best composites that we knew how to make at lowest weight.
And so, that optical bench, that tube was all composite.
And so, that is the kind of weight reduction scheme we had to go through.
Now, also all of the fittings, that means all of the brackets that we use were also composite.
And another challenge that we had is analyzing the stresses in a complicated composite fitting.
Very hard to do.
Didn't have techniques to do it when we started.
As part of the process of building this telescope, we had to go through and learn that and develop those techniques.
Before the break, I just want to point out I hope you all are catching the incredible parallels between this project, a complex technological process and the sorts of things that we've heard about the systems engineering of the Shuttle.
Right from the weight problems and composites, new materials, new ways of analysis right into the intervention of Congress into the engineering process, I mean it is all there.
And I think whenever you get involved in a big space project you can expect that those things are going to happen, and they are probably going to happen in the Exploration Program as well.
Two minute break and then we will start up again.
Start up again.
Let's see.
We were talking about for us the next biggest challenge was programmatic in that it you have to work that early on.
And typically, especially the more expensive missions do go through this as a typical part of the lifecycle of a project.
I have a question on that last slide.
[NOISE OBSCURES] The budget for systems engineering was cut which translates directly to the number of folks at the contractor site.
Working systems engineering was cut so we were challenged.
And what we did at Marshall was we offset with NASA's civil servants doing functions and tasks that the contractor would normally do to compensate for that.
So that is what that meant.
Typically, systems engineering, a very important thing if you talk to folks in the trade, almost everyone has a different idea of what all is encompassed in systems engineering.
But the point is there probably are a number of things that almost everybody would agree are part of systems engineering.
And they are very key to executing the program and the project.
They are really the glue that holds all the subsystems together.
And I will probably talk more about it, but for good systems engineers you have to be able to communicate well with all the other subsystems and all the other stakeholders, including the science when you have a science mission, including the users when you have a manned mission, including safety and everybody to make sure they are all connected.
And be smart enough to be able to challenge the subsystems when they really have their own ideas on where this needs to go and it is not really in concert with everybody else.
Weight reduction did have some impacts, as I mentioned, using light-weight composites.
And that was a challenge in and of itself.
We also dropped two of the SIs.
Another challenge was the science instrument module which was all composite.
And it had the capability of adjusting focus and translating, so it had two degrees of freedom.
But it needed to be reproducible in terms of motion on the micron scale over the whole orbit environment.
That was quite a challenge in and of itself to be able to do that.
I mean just kind of a flavor for the things that you have to deal with is not everybody had the same analysis tools so universities like MIT and like SAO had different structural analysis tools for their hardware than the contractors did.
Now you do a couple loads analysis, which is you pull together all of your stress, all of your structure analysis models together in one place and you go through a loads assessment all through the whole element.
Well, to do that I need a model from you, I need a model from you, I need a model from you.
But you are in California, you are a university, you are another company and you have to make sure they all play together.
And a lot of times there is parochialness with company A, I am not going to use anything but my model.
And the university says, well, I cannot do anything but what we have been doing.
And university C says, well, this is the only thing that will work and you guys are crazy.
It is not that bad but that is the kind of thing that you have to deal with as a systems engineer.
You have to make sure everybody plays together.
And sometimes you have to go through and work the best solution that may not be the optimum solution for each piece.
But it is the optimum solution for the overall answer.
Again, getting the programmatic challenge behind us, a key thing is also setting allocations.
I talk about the error budgets.
Another good thing that the systems engineer does is allocates various important resources.
And one that every spacecraft does is weight.
Weight, power and data are usually three of the important resources that are allocated.
For scientific telescope missions another one is error budgets for scientific performance, but the key to that is good weight allocation and good iteration, which with each of the elements that you allocated the weight to, to keep up with them to make sure they can meet that or it looks like they are going to exceed.
So I have got to go borrow some of that allocation from somebody else.
And I have got to take away from Peter, if you will, to pay Paul.
So systems engineering and going through that making the initial allocations and keeping up with that is a pretty critical factor to making sure you get to the final answer.
And, if you do that well, then usually you have less of a problem getting to the final solution.
To compensate for the loss of systems engineering, we established technical oversight panels which were NASA employees at our center.
We controlled, at a project level, the internal ICDs.
Now, what is an ICD?
It is an interface control drawing.
Because when you have company A developing an element and university A, where they come together is an interface.
And you have to document that interface, not just mechanically but thermally.
What is the heat flow across the interface, the data across the interface, the power, the signal characteristics across the interface?
The data flow.
What data is going across the interface?
And, as you can realize, as both of those things change, you have to change the interface.
So you have to keep up with that.
And a lot of times good systems engineers will recognize the further away in location it is the harder it is to interface with the two parties.
When you are in California it is one thing when they are in the same town, it is another thing when they are in the same country and it is another thing where they are across the water when you are integrating an international partner.
So, as systems engineers, the factor is the closer you are the better it is because I can just go in a car and go across town to get out the drawings and talk about the drawings versus in California that is a plane ride.
But, in Europe, the time difference allows you to only be able to talk to them for a few hours a day because that is when your work shifts overlap.
So it is a lot more complicated interacting with folks in Europe, interacting with folks in Japan, in India because of the time difference and because it is very far away when you have to sit down.
Those are kind of the challenges as systems engineers that you should think about.
Lessons learned.
Set your resource allocations early and continue to monitor them and work with each of the owners of the elements of those allocations to make sure that they are meeting their allocation.
As you recognize, it is a zero sum gain.
So when somebody goes over their allocation somebody has got to be reduced.
And it is very difficult sometimes to negotiate that reduction because engineers typically want to hold their margin, they don't want to release it.
They want it because they are not yet confident in their final solution.
But to make everything play sometimes you have to take away from them and reduce their margin to be able to let somebody else survive if you will.
It is not an easy thing, but attention to that detail is important.
Maintaining strong systems engineering group is important for that reason.
If I have a lot of pretty dominant subsystem managers that say I am building a power system and I need this much weight and I need this much thermal capability that's good, and they may be very good, but what they are doing is if you have a lot of dominant subsystems and no central systems engineering, as traffic cop there is no check and balance.
And so that dominant thing may work very well at the expense of the integrated performance of everybody else.
That is why systems engineering is so important, that dominance and to balance weakness where you have weakness.
And for Chandra hard work pays off when you get it right.
Technology.
A lot of times NASA will build a science spacecraft.
And the spacecraft itself usually isn't pushing a state of the art.
For Chandra it was a little different.
We pushed the mirrors, and there are a few other things we pushed.
But typically it is the case where NASA pushes the state of the art on the instruments.
Better sensors, better CCDs, better processing electronics.
And Chandra was no different.
Usually it is an embellishment from something that has flown before on a different spacecraft, only now it is better.
We are getting it bigger.
We are getting more resolution.
It takes less power.
There is usually a steppingstone.
And for ASIS, the one that was built at MIT, it charged a couple of devices in an array size that they have never been built before.
And so that was a challenge.
Also a challenge was in order for the CCDs to operate they needed to be cooled to minus 100 degrees Celsius, 120.
And, in addition, they developed a new way of clocking out the data from the rest of the CCDs that have been built.
That had better spectroscopy performance and a very low noise signal chain.
There are always going and they are always looking at, well, on spacecraft X this CCD was flown.
It may have been a 512 x 512.
Well, you know, it is pretty easy.
I can stretch it a little bit and work the substrate in 24 x 24.
Well, usually it is the case where it is not as simple as you think.
And there are always problems.
In particular, technology should be a warning flag to systems engineers to say typically it is not as simple as you think so make sure you are paying attention to these new developments or these stretching of new applications of existing technology.
Because, for instance, one of the things that you get bit by, and we got bit by is radiation susceptibility.
Thermal extremes for multilayer circuit cards that have to operate at minus 120 C on one end and room temperature on another end.
Low yields for those new types of CCDs.
I can just stretch it and make it 1024 x 1024, but where I was using 90% of the batch, now I am only using 10% just because I cannot get one that big to work out and to be smooth and homogenous.
ESD sensitivity.
Because they are getting bigger and bigger and thinner and thinner it is much more susceptible to electrostatic discharge.
And simple things like my yield in the wintertime, because the humidity in the air is lower, I have less of a yield in winter than I do in summer.
And so that is what we found out.
And there are also mitigating effects that you can come and put humidifiers in and compensate for that.
But those are the types of things that you have to worry about that can come and bite you with new technology because you are doing something new.
Because of going down to 120 to optimize the performance and to get the very low noise, we said we need a radiator or a sunshade.
And, low and behold, flying on the Shuttle in certain attitudes that you have to verify, because in some off nominal events you can, instead of being deployed on day one of a mission you get deployed on day three, but you have to go across the sun terminator frequently and you get a little glint of direct sunlight on these sunshades.
And, therefore, you have to analyze what the temperature extreme and what the effect is going to be.
And, low and behold, we see that we have delamination on that case.
But here is a case where it is an off nominal event in the Shuttle that we have to design for and we have to compensate for, even though chances are we will never see that design case.
But a lot of times the programmer or project manager has to say I am going to suck it up and I am going to make the change for that small eventuality.
Or I am going to risk it based on cost and we are going to try and preclude that eventuality from happening.
A lot of times it is not cut and dry.
It is a guessing game.
It is a balance that is based on judgment, past performance and a lot of other factors to say, boy, that is so low probability of occurrence that I am not going to worry about it.
And that threshold is never defined.
You have to define it as a project manager.
Just kind of a picture of what the instruments look like.
There are the CCDs.
This is the long stretched away.
And, for the spectroscopy measurements, that is where the lower energies -- Actually, the higher energies in the middle and the lower energies go along this wing and along this wing because you get the energy spread.
And there are the imaging devices that are used when you just want a good image.
On HRC, there is the detector right there in this diamond-shaped thing.
And it is operating off a completely new and older technology than the CCDs.
The instrument of choice has been this for probably 70%, 75% of the observations just because of the capability that this allows you not just an image, but the CCDs also give you some rudimentary information on energy.
And so, that is why this is preferable over this one.
On HRC, we talked about ASIS.
We had, again, a low noise signal chain.
On HRC, we never had three micro channel plates tied together and linked together.
And we had that challenge.
We also had very accurate event timing.
So, when we have a pulsar, I know exactly in registered time when those pulses are occurring.
We had spacecraft charging protection for new technology problems and spurious noise susceptibility.
Spacecraft charging.
To keep the mirrors nice and warm at plus or minus half a degree C across the entire mirror surface all around it required thermal blankets on the outside.
Well, the thermal blankets were susceptible to charging up and discharge with popping.
And so, the question is, is that going to affect my instrument performance by incorrectly registering a photon event every time these blankets discharged?
So we had to go through a test campaign to verify that, no, this discharging of those blanket electrostatically would not affect the imaging performance at the detector level.
That is something we didn't even think about before.
But, low and behold, we came up with it.
Yes, these things are going to discharge, they are going to pop so is it going to affect our measurements.
Those are the kinds of things that you will run into.
Again, good systems engineering and sound communications are the key to getting through those.
Insuring participating.
Keeping parties informed.
Encouraging teamwork to be part of the team and to help contribute to its success.
Lesson learned is establish standing interface working groups with mandatory participation from the SIs.
The SIs kind of have a different paradigm.
Science instruments are usually built by universities, and the universities are typically staffed by grad students and some full time engineers.
But usually the paradigm and the thinking process is a university atmosphere.
When you have companies it is a completely different interface, very regimented, very by the book, we have done it before and this is how you do it.
And so different cultures, you have to make sure they blend together to produce the final product.
And a lot of times that is not technical.
That is human interaction and making sure they all work together.
When problems occur recognize that you need to go outside of the project office to get help.
And this is typically done at NASA.
If Marshall is managing a project and we have got a problem with CCDs, the key to success is recognize, well, Ames Research Center in California has an expert in CCDs.
Call him up on the phone or bring him down and let's talk to him to get his expertise in on this project.
NASA does good on that.
It does go to where the expertise is.
Not just at another NASA center, but it may call experts across the industry or across academia to come help with a problem.
The foam problem on the Shuttle is a pretty big problem, and we have called experts from around the country and around the world to try and help understand what was going on with the foam.
That is something that NASA does well, is it brings in experts and it doesn't think that it knows everything locally.
And that is something to keep in mind.
If you don't know an answer, don't just try and think of it yourself.
Go get the answers from somebody who might have experienced that before.
And encourage teamwork.
A lot of times programs suffer from the culture clash between different paradigms and different cultures.
And that is especially the case between academia and industry.
It is also evident between international partners and industry.
And, of course, the payoff is this is a spectrogram, the counts per bin, number of photons per energy level.
And, as you can tell, very sharp and very well-defined peaks for where those lines are.
Next challenge was integrated testing.
Integration is always a challenge.
How in the world are we going to integrate this whole thing?
The logistics of who does what when, we must choreograph it.
Here we have instrument A coming in.
We have instrument B.
We have the gratings coming in.
We have the ISIM built at Ball Brothers.
We have California building the optical bench.
Where are we all going to put this all together and test it out?
That is always a challenge because, again, you have different players involved in different pieces.
And you have got to make sure it all plays together.
And the key there is to create a working group to make sure you address every aspect of integration.
You plan very much ahead of time.
You choreograph everybody coming in to make sure you have got everything covered.
And in integration always, always it takes longer than you think.
You always plan assuming nothing is going to go wrong.
This test takes this long and this test takes this long and this test takes this long.
When, in fact, as is always the case, you always have problems in testing.
Any time you are matching two boxes together, especially electronically, a lot of times you have problems.
So testing is always something that you should make sure you have enough time to do troubleshooting.
And plan for failures and not plan a success-oriented schedule.
Even the best laid plans, TRW planned the integrated testing in a vacuum chamber.
And during observatory integration TRW couldn't hold the schedule because, guess what, the first thing that goes when you are in a money crunch is testing at the end.
And you say why do that?
Because they say we had six months of margin in the schedule and you have eroded that margin.
But that margin you haven't really defined anything to go into that margin.
So, hypothetically speaking, if everything goes OK, I don't need that.
Well, so you take it off the table and it gets eaten away.
And that is what happened.
It got eaten away and couldn't hold schedule caused, in this case, by antiquated EGSE and software.
But it is typical of a paradigm that says I create schedule, you have got to try and preserve this schedule and fight tooth and nail to keep that reserve in every step of the way because in integration test is where usually you get behind.
When we got behind the recovery is daily focus on schedule.
Every day we were on the phone going over what was to occur that day.
24 hour operations one year before launch continuing all the way around the clock.
We changed the I&T integration and test lead out at TRW.
We brought Marshall NASA engineers into help TRW do it.
And we had a technical presence help out at the test site.
We have a contractor, we hire a contractor and the contractor is supposed to do the job.
And, low and behold, the contractor is not doing the job.
So we could take a step back and say whack them on the head, you are not doing the job.
Or, you can go in and say we have got to get this job done.
We are going to go and we are going to send folks over there that are technical folks that will help your guys work these problems and work the issues and work the integration and get this thing done.
And so, that is what we did.
That is kind of a typical schedule.
We have a Comprehensive Acceptance Test.
That is just a functional test going through all of the paces.
EMI is your kind of electrical susceptibility to noise.
When you have a palm pilot or a cell phone and it is near a computer or near some device you hear this chattering.
That is what that test is all about to look at interference patterns and look at whether you are susceptible to noise.
We have an end-to-end test.
That is where we get a compatibility van.
And we make sure that we can communicate with the deep space network dishes in Goldstone and Madrid and Canberra.
That is our pointing at control systems polarity test.
Did we mount the gyro correctly or is it upside down?
Did we mount the reaction wheel correctly?
Do I have the spin vector pointed this way or pointed that way?
You think that is pretty simple but, in fact, on the box it doesn't say arrow pointing up.
You have to actually test it out.
And NASA has screwed it up before.
We do an acoustic test and then a pyroshock test.
Acoustic is you blast it with acoustic noise and the noise couples and creates vibrations, especially you are worried about thin films and large surface areas that are exposed to the noise pattern, the wave front.
So we painted the whole spacecraft with a huge woofer.
I mean it was enormous.
Just to see the vibration and what would happen to make sure that it was OK. And we have the solar ray full motion test and the low gain antenna release test.
We have mechanisms which are typically things that fail a lot or don't operate the way you thought.
We want to test those hopefully in the environment that they would see in orbit.
That means if it is cold, if it is exposed to a plasma we want to try and duplicate that on the ground.
Typically, that is what we do with mechanisms.
Yeah.
I was wondering if you had like a separate satellite that was forecasting and they kind of knew that [NOISE OBSCURES].
Good question.
Typically, in the old days, when we had a lot of money, we would build what is called a qualification article which is basically a test article for the whole thing.
And the qualification article would match one for one the flight unit.
And we would test the heck out of that qualification article.
Nowadays, with money problems, typically we don't do that.
For selected components where we are worried about the design, we will develop a qualification article at the element level.
But typically we don't do it with whole spacecraft, not anymore.
Although, the one exception to that is the structural test article.
Usually spacecraft do build a structural test article that does look like the final flight design from the overall structure.
And you get your mode shapes and your frequencies when you ring it and see how it performs.
And you fold that back into your analysis.
But that is probably the only case where you build a full test article for structural purposes.
It would be nice if we could do that.
The DOD does that when they have a production of a thousand units, but NASA nowadays cannot afford to build a whole complex ground test article.
It would be nice but sometimes we just cannot do it.
We will do it on a selected element level.
Hardly ever testing to failure.
We talked about that with the Shuttle because JR Thompson was pointing out, like with the main engines, testing to failure was really critical for success.
But, remember, that is a very different operating situation than a satellite where normally if the satellite can survive launch and the vibration, and the vibration is tested, the structural, you know, if that survives then the operating environment is a lot more benign.
Now, what do, though, is we compensate for not testing to failure by testing with two our expected flight environment plus margin.
Analytically, or based on previous data, we determine what the environment is going to be.
And we will add margin to that and we will test to those margins.
Most times we do pretty well.
Sometimes we don't, but that is probably few and far between.
And we have learned, after 40 years, of how to apply margin to our best analysis or our best tools that define what the environment is going to be.
So that kind of shows you starting in October '87, ending April '98, that was the plan.
We actually took about nine months more than what you see there because of all the problems that we had.
A key point is when you are pressed for time do not cut corners.
We did review all the testing and we did not delete any testing that we considered was mandatory.
And, in fact, we added systems testing to verify that we had total system performance that we knew what this thing was going to do.
We also added some end-to-end testing in thermal vac and added much more end-to-end testing with the control center that will eventually operate.
A lot of times you have, in the integration and test time, a different set of engineers testing this than will actually operate it in flight.
Probably not a good idea.
What you really want is to get your operators in flight operating and testing early enough that they can go through all of the gyrations of the testing environment and all of the idiosyncrasies of the hardware before flight so that they are trained on the hardware and they know how the hardware performs.
And so, if you can, you want to try and get the operators to do the testing prior to launch.
And a lot of times that isn't the case, unfortunately.
Lesson learned.
Try to keep one integrated database along that theme of the Control Center for testing and operations because what you don't want to do is to have a database for testing that then you discard and you create a separate database for operations.
It is much better to have one database so that you know what the parameters are.
And, once you have verified them in test, they are the same parameters that you are using in flight.
Also, to define an explicit test and integration lead, review the approach, encourage end-to-end testing participations of the operations group early and often and, if you can, get the operations group at the Control Center to run the tests.
And give adequate time for box level testing and data system integrated testing.
A lot of times what will happen is you're running late on schedule and at the box level I have a multiplexer.
And I had scheduled, at the box level, probably 300 hours worth of testing.
Well, you know what, I am running late.
They are wanting me at the spacecraft level.
Well, let me jip it up now.
It has only had 50 hours but now I am going to the spacecraft.
Low and behold, now what you have just done is you have moved the problem further on downstream when it is much more expensive and it costs you a lot more money.
When I have a problem in integrated testing, guess what?
The whole spacecraft stops until I fix this box.
If you can, try and do it so that you have got enough margin that you can test at the small box level in parallel with all of the other boxes before you get to integrated testing.
Management.
There were plenty of challenges in the management.
And a term used is unknown unknowns.
We didn't know at the time that money was going to be a problem with program restructuring.
You don't know that you are going to have an integration delay.
There are a lot of things that you really don't know.
But, nonetheless, you have to expect for and you have to plan for.
That is why, at the beginning of a project, you create what is called reserve resources.
In terms of cost, depending on the nature of the complexity, the technology push, you may have 20% to 30% of cost held in reserve.
Not applied anywhere because you don't know where the problems are going to occur.
And you try, as a project manager, to hold onto that by tooth and nail and not release it if you can at all and only grudgingly allow reserves to be used.
Approach to success in that kind of environment is experience, getting top management that has background and experience, working closely with the science community.
Again, focusing on teamwork.
You will see that as a recurring theme.
Hold enough reserves so that when you are scoping out and you've got a lot of unknowns that you have enough money and time to do that.
Set it as high priority for the company or the organization that you are working for and set the schedule early and balance the need for maintaining schedule with the need to slow down and understand what your problems are rather than racing to the finish line.
Again, these are some of the good lessons learned.
In fact, I just covered that one.
This is what it ended up being, the AXAF Schedule starting around calendar year '92 with our SRR.
There is our PDR and there is our CDR.
And the launch is even off the chart.
But that is typically the kind of things that we go through, including in parallel your Control Center development back down here.
There is a mirror.
All of this will be posted.
I mean these are really valuable lessons.
In fact, any of you who eventually go into systems engineering and project management take some of these lessons learned with you.
Print them out and review them periodically.
And remember them.
They are lessons of hard knocks.
What I am doing is now is I am now the project manager for a project or a mission called RLEP 2, which is Robotic Lunar Exploration Program.
And that mission will be the first mission that NASA has, in a long time, to land on the Moon.
We are going to land the Lander.
And we are going to have some type of mobility device that will either drive into a crater or hop into a crater or crawl into a crater or walk into a crater and look for lunar ice that we think might be there at the South Pole.
Now, this is a great example of how things are done.
We are given a couple of requirements to say we want to verify our precision landing capability in an unmanned sense, and we are going to use that technique and those algorithms for the future human lander.
And we are also going to need you to go look at the ice.
Characterize it and see whether or not ice is there.
Now, why is ice so important?
Ice is important because of the oxygen that you can get from the ice.
And the oxygen is not so much to breathe but it is for fuel or for oxidizer for propulsion systems.
Instead of getting it at the earth's gravity, well, you get it at the gravity well of the Moon.
That is our challenge.
Now, when they asked that, NASA didn't exactly know what it would cost or what the cost profile would be.
They originally designed it for a $400 million mission.
They said, well, why don't you look at a $750 million solution?
They don't know how many objectives we can meet.
This RLEP is supposed to be robotic lunar exploration.
It is really a precursor to the human missions, so they are not even sure how to use this program to help bring humans there.
And so, this is typical.
NASA doesn't exactly know what it wants to do, but it kind of puts out feelers to say try this.
It says go start with this, go touch the lunar surface, characterize precision landing and go see if there is ice there.
Well, go see if there is ice there.
I can do that with one hop into the crater, one sample and hopefully, if the ice is there I can find it.
Or, on the other extreme, I can go with a Rover, spend a year in the crater, take a thousand measurements and characterize the ice.
Obviously, one cost a lot more than the other.
So how in the world do you answer that question?
Which one should you do and whether it should be a mix?
Those are the very things facing the project that I am working on now.
Not only that, one of the things is we are trying to get prepared for a human mission and we want to reduce the overall cost and reduce the risk.
We would like to bring risk forward from the manned lunar mission into the RLEP program and retire it in the robotics program.
What that means is if I have a technology challenge for a cryogenic engine, that means a liquid oxygen, liquid hydrogen engine for the lunar descent, I want to use that same engine, if I can, on this mission to work out the kinks.
But how much is that worth to NASA?
If that cost an extra $100 million for RLEP is it worth it?
There is no set answer.
And so, what we are doing right now is what is called a pre-phase A.
Much like I showed you on Chandra where you had that initial design and a final design, we propose to headquarters with one design.
It happened to have a Rover.
It was nuclear powered.
We went into the crater.
And we stayed there for quite a while.
And we definitely characterized what was there.
But it was expensive.
Is that the right answer?
We don't know.
We are right now going through a pre-phase A that we are looking at other options.
We are looking at kind of three classes of sizes.
Different solutions to get into the crater.
Different solutions on the size of the lander.
Its extensibility to the human mission.
Like, for instance, right now we use Russian progress vehicles to resupply Space Station.
We hope to design a robotic lander that would do the same thing during human missions.
It is not man rated, it won't fly men but it can fly supplies to the Moon with one design.
But that cost money because now I am not optimizing design for a point solution mission.
So during this timeframe there are a lot of unknown questions.
And the answers are not straightforward.
So what we are doing is we are giving a series of options and rationale and capabilities for those options to headquarters.
And we will make a recommendation as to what we think is the best answer for NASA to go forward with this design and here are the reasons.
But it could very well be that they choose something different because there is no definitive solution to this problem or this challenge, if you will.
So it is very challenging.
It is going to look probably a lot different now than it will when we actually launch it in 2010 or 2011, but it is something that we face now.
And it is more defined by how much money you can use versus what the requirements are.
Sometimes the requirements do set pretty much the basic tenets of the mission, but other times it is driven by cost.
And so, right now we are in the middle of working that interesting and challenge problem for headquarters to develop what we think that mission should look like.
And, as you can envision, you don't really have requirements.
You have a few requirements.
You have a desirability to reduce the overall cost and reduce the overall risk of a human mission.
But that desirability comes with a cost.
The more extensible you get to the human mission, yes, you can make a big lander that gives you 2,000 kilograms of payload, but it is going to cost you more.
I can make 1,000 kilograms of payload or 500 kilograms of payload and get the definition of what is in the ice.
But then later on I will have to design a new lander for getting other things accomplished that I want to get to on the Moon.
And is that the right answer?
You can tell that there are a lot of challenges with that?
Any questions?
We talked about the cost, performance and schedule triangle.
Do you have a sense, inside NASA, of how much they are pushing schedule at this point/ Where is your flexibility?
In their solicitation to the centers, they allowed schedule flexibility.
They allowed cost flexibility.
Other programs I will say are generally more bounded than this one.
This one is a matter of NASA not knowing exactly what the reason and what the purpose of the program is for.
You've got this general idea of going back to the Moon, using robots first to prepare for humans, but exactly what does that mean and exactly how do you prepare for humans and how much is it going to cost and when is it going to launch and how many missions are you going to have?
They are all undefined.
Now, what we would do is we have already gone to the program that will fund the human missions to say what do you guys plan to do and what do you want to do when you get there?
So we can take their mission design and say how can we help?
We can help here.
We can help here.
We can pull this technology forward and fly it.
The problem there is that they haven't thought that far ahead.
They don't know exactly what they are going to do.
And a lot of that is, is there ice there or not?
If there is ice there, we will want to process it.
We will want to extract oxygen from it.
As part of our solution, we do know that we have to answer that question.
We also do know that we do have to demonstrate precision landing.
But, above that, they have given us a target date of 2010 to 2011.
But they said schedule is not fixed.
They have given us a cost generally between $400 and $750 million.
And, by the way, if you can leverage areas of the budget to help your mission, do that, in particular technology.
There is a separate technology budget.
If I can get technology to pay for a new technology development at no cost to the program then I will do that.
And, again, if we come in and say we have $1 billion but it buys you $3 billion when the crew is ready to come, we think that is worth the investment.
And then headquarters has to scratch their head and figure out whether it is really worth minimizing the long-term cost versus the yearly constraint that they are given by Congress and the pressures of Shuttle and the pressures of Stations on the overall cost of the program to NASA.
It is real challenging.
It is a lot of fun.
We are going to have a blast.
The next one is going back to the Moon, and you will probably hear more about it in the next couple of years.
OK.
Thank you.
[APPLAUSE] This was really the last of the lectures on systems engineering per se.
Thursday is the last external lecture.
Gordon Fullerton will be talking about test flying the Shuttle.
And hopefully by then I will have the schedule for the presentations next Tuesday.
In that case, I will move on to introduce Colonel Gordon Fullerton who got is bachelor's and master's degree from Caltech in mechanical engineering, did ROTC and went on to the Air Force where, well, I guess he has flown, in the course of his career about 135 different types of airplanes.
16,000 flying hours.
He flew F86 interceptors, B47 Bombers before going to Air Force test pilot school in the mid `60s.
In '66, he was chosen by the Air Force for the Manned Orbiting Laboratory which was going to be the Air Force's Space Station.
That program was cancelled in 1969 and the MOL, well, some of the MOL astronauts came over to NASA and became NASA astronauts.
That was in the heyday of Apollo.
And Gordon was support crew on the last four Apollo missions.
Then went on to be the test pilot on three of the first five approach and landing tests.
If you remember, that is when the Shuttle Enterprise, which was not designed for orbital flight but was designed to test basically the last 50,000 feet down to the ground, was dropped from the 747.
Went onto pilot the third Shuttle flight, which was one of the orbital flight tests.
That was the flight which landed in New Mexico.
And then commanded STS-51F which was the first flight of the instrument pointing system.
That was also the flight, I am trying to think, where you lost the engine, which we talked about.
That was the one flight where we had an engine shutdown.
And we went over that in some detail.
I think they should hear about it from what it seemed like inside the cockpit, now that they have heard kind of the technical details.
In any case, after leaving the Astronautic Office, Gordon has moved into pilot's heaven which is Edwards Air Force Base, the Dryden Test Flight Center where he basically spends all his time.
Well, not all of his time because there is management stuff as well.
Unfortunately, that is the price you pay.
But basically he gets to fly airplanes for a living.
And lots of different kinds, including the career aircraft and the B52 which launches various test aircraft and a whole bunch of other airplanes.
If you remember in the whole systems engineering approach to things, we talk about the conceived design, manufacture, test and operation.
So this is basically the final lecture and this is the test and operation of the Space Shuttle from a pilot's point of view.
So, with that, I will give it to you, Gordon.
Thanks.
I was sitting at your place back in 1958 when I was a grad student at Caltech for a year before heading off to the Air Force.
Wow, almost 50 years ago.
But the real blessing of my life is to be able to do what I love to do for as long as I have done it, beyond all reason.
Anyway, I hope to share some of that with you this morning.
This is my first time through this pitch.
Feel free to holler out any questions as we go and we will stumble through it together.
I planned to talk about three different phases of the orbiter test program.
The first were called ALT, the Approach and Landing Test, that consisted of really 13 flights, five captive inert flights where the enterprise was flown on top of the 747 but nobody in the cockpit.
The controls were locked.
That was to clear the envelope for the kind of unlikely looking combination of a 747 with an orbiter on top.
And then there were three "captive active" flights where we were manned up in the orbiter with the systems running, the electronics, the hydraulics were active.
The controls were free to move a little bit, not full throw.
And we also determined how high the combination could get.
Because altitude was not 50,000 feet, it was more like 28,000 feet max even with the 747 engines over boosted.
Followed by three "free flights" were we actually pushed the button, munched off the top and glided to a landing.
The orbital flight test, prior to the Shuttle Program, OFT consisted of four flights.
STS 1 through 4 of varying durations.
STS-1 was planned for just a couple days to get up and get down and have a look at how things looked after it flew.
STS-2 was supposed to be five days.
They had a fuel cell problem that cut their flight short, unfortunately for the crew.
Then I was on STS-3.
We were scheduled for seven days.
And, because of a raging dust storm at the planned landing site, we got an extra day.
A real blessing.
With no tests to do, we had a free day on orbit.
And ended up with eight days.
STS-4 planned and flew seven days, landing on the fourth of July with President Reagan out there to watch.
And then we got into operational flights of which there had been a lot.
Something over a hundred.
The last flight to fly, the return to flight was STS-114, but the number system went through a couple of phases.
There have not been 114 flights.
I don't know how many it has been, but there had been a bunch.
I will talk about three, well, more than three flights, but the ALT flights.
If you have to have a crew patch, that is a major hurdle that you have to overcome.
When you are assigned to a crew, get the patch designed and then get it approved by upper management.
Part of the OFT, STS-3 in 1982 and finally then an operational flight 51F in 1985.
Prior to any of these flights, I worked into a dream job for really systems engineering, I guess, and for a pilot.
I had been to test pilot school.
I was intrigued with all the airplanes I flew with a cockpit design and was very aware of the flaws of cockpit design.
It seems like sometimes engineers lie awake nights trying to make it hard to operate subsystems.
I got in at the beginning of the orbiter crew interface and became, before it was over, sort of the Czar who was responsible to sign off all the drawings, lots of reviews.
It was a great responsibility because I was immediately and early into subsystem design because you have to know what the subsystem does to make any intelligent choice of what the controls and displays should be.
This is a view of, not the Enterprise, but probably, I don't know whether it is a simulator or the real airplane, they look alike, but one of the orbital capable shuttles.
This is like the little old ladies always say when you're out on display for Armed Forces Day and look at all the things, how do you remember what they all do?
Well, that really is a challenge to make things straightforward.
And I became painfully aware, especially later on when we had a lot of experiments coming in, that engineers in their little ivory tower building a system in the lab are so totally, personally familiar with how their stuff works that it doesn't bother them that the switches are unlabeled or labeled illogically.
And they bring that all in and want to put it in the orbiter.
And when the crew says wait a minute, I have to remember, before I turn on switch A, switch C and D have got to be turned on in this order or the whole thing will blow up?
And they say oh, yeah, that is the way it is.
And they know it so well personally.
And, yet, when you are overwhelmed with a whole lot of experiments, I am sure you ran into it on your flight, that all the stuff that is just totally familiar to the guy that invented it becomes a challenge when you are faced with a whole lot of them and a limited amount of time to learn it.
And so, standardization of what the scheme, especially in the software.
While there are a lot of gadgets, in fact, if you count all the switches, gauges, circuit breakers, upstairs, downstairs and the mid deck, there are about 2,100 total in the orbiter cockpit.
But the real challenge is what shows up on the screens here.
So the software was a bigger thing in making them all play together.
Here is a higher view in the overhead panel above your head.
And the forward cockpit has a lot of other switches.
An example of what I am talking about, this array of switches right here controls the reaction control system valving and then the orbital maneuvering system.
Back in Apollo, which I worked on when I first came to Johnson Space Center, I learned, while lying many hours in the Apollo crew simulator, that that had the worst cockpit every designed by man.
As an example, it had a helium pressurization system to push the propellants for the reaction control jets out of the tanks and then the valves that went to the various arrays of thrusters all around the command module.
And one would think logically, if you have something to turn on the helium, you label that helium on/off.
And then the manifold, the next branch in the Christmas tree manifold would be system A versus system B or something like that.
And, in fact, it had names like that.
But they put the switches in a long room and labeled them A through H, right there on the panel.
No clue as to what you were throwing, so every move you made in the cockpit, some of these things were critical, required you to either be a metal giant or open a checklist and say to pressurize system B, I have got to throw B on and then A next and then H last or something like that.
Obviously, unacceptable.
It became my career and mania to try to improve that.
The first orbiter simulator built was called the OAS, Orbiter Aeroflight Simulator.
It was on a moving based platform like the standard airline simulators you see today.
This is Fred Hayes and I sitting in the cockpit.
This one was built originally to be the Enterprise.
And the Enterprise was OV-101 in the scheme of things, a numbering of orbiters as they were built.
101 had only the systems it needed to fly the approach and landing test.
The plan was to retrofit it later to make it into orbit capable.
But before the ALT program it had just what you needed and only that to fly in the atmosphere very short flights.
But this is our first chance to look at what we had and what we were getting into as far as pilot concerns.
Now, I guess you have had all the subsystems, right?
You were all completely briefed and experts on subsystems by the people that had preceded me here in the course.
I will give you, though, a quick review here, the avionics system.
The Shuttle is build with inside the outer mole line as the pressure vessel that is roughly back to the forward bulkhead of the payload bay and is imbedded inside basically aluminum structure with a thermal protection system applied to the outside.
And most of the avionics are in racks right up here, right in there with you in the cockpit on the middeck.
The stowage lockers are aft of the racks full of black boxes and lots of wires.
And the other place there were avionics is back here just after the aft bulkhead of the payload bay.
And there are some areas where they put items like rate gyros and accelerometers.
A lot way back.
From a systems engineering standpoint, Fred Hayes and I were out at Palmdale where they built all the orbiters.
It seems like the testing always happens after 2:00 AM, tests would run around the clock.
And we were out there way after midnight in the cockpit participating in the first time they were going to close the loop between the rate gyros and the flight control system which is contained back up here in the nose and the general purpose computers.
The whole orbiter was suspended on some big airfield bag so it had compliance to move.
And the hydraulic system was on powered by a facility hydraulic source.
The elevons were powered and the rudder speed brake was powered and the process let us down for the very first time that we were going to close the loop between the software up here and the rate gyros particularly and accelerometers mounted in the back.
And it was a milestone.
And we had people stationed around the back to see if anything spit out hydraulic fluid or whatever.
Anyway, we went up to the glass shield and pushed the button to go into CSS, Control Stick Steering, which hooked the stick up now to the flight control system.
I will never forget.
We pushed the button and a rumble started.
And the rumble built very rapidly to a violent rumble.
I mean we were being bounced around the cockpit.
This is 150,000 pounds of airplane dancing on these rubber bags.
And it didn't take us long to say this doesn't feel good.
We pulled the button that downloads back to direct control so there is no feedback.
And the upshot of it was that these rate gyros are mounted on this flexible bulkhead.
And that had been taken into account properly.
And so there was a structural resonance that was just diversion.
In spite of all the Grounds tests down at Downy and the Iron Bird mockups and everything, when we put it in the real airplane it tried to jump off the supports.
And Fred and I were in the cockpit, our eyes were big, and we called the control room which were people back in an adjacent room.
It seems like most spacecraft tests, traditionally, are done with engineers all locked up in windowless room, no view of the real hardware.
Down at the Cape they are miles away when the spacecraft is either out on the pad or in the main checkout building.
And so they said no, the procedure says this is OK and we want you to re-engage.
We tried one more time.
And this time the guy stationed in back that were watching said this doesn't look good.
They started hollering on the loop so we pitched it off.
And we went into a raging argument.
We need to complete this test, the engineers were saying.
And we were saying we don't want to break something.
And they got people out of bed down in California, controls engineers.
And so, it was a memorable night.
The resolution was finally a redesign of the mounting area back there and reanalysis of the vibrational modes of the aft bulkhead sorted all out.
Here is what we had for a crew interface.
You saw in the earlier picture, upfront three CRTs, monochrome, green, it was either green or nothing on the screen, and we had two keyboards, three CRTs.
There was always this little concern, when you are punching on a keyboard, you want to be sure it is selected to the CRT you think you are doing something on.
Another marginal design, but that is what we ended up with.
You notice we have a hexadecimal keyboard here.
We have some strange keys called item and electric and ops which had to do with this truly unique way you cause things to happen through software.
And lots of things were critical through software.
And then this rather dim green screen.
This was way before the days, I guess, Bill Gates was still here at MIT or maybe even hadn't registered yet.
I don't know.
It was way before Windows 1.0.
And so it seems almost comical now when you compare it to even modern airliner crew interface with glass cockpits, but we were breaking new ground here.
And it, even at the time, seemed antiquated because that is all they were willing to embrace in the way of computer control and design.
And it was unique.
And it causes lots of headaches and delays.
The big thing about the orbiter with its data process system, the decision was made that everything is going to be done with these all powerful general purpose computers in which there were five in an array that worked together.
Flight control, system management, navigation, inertial subsystem control of everything.
And so, the result was the software loads, as they were built, had lots of flaws when they came down, yet we couldn't fix them.
We had cases as dumb as a display that showed the Freon loop A pump on.
And, when it got through the entire software belt, it was backwards.
It would say off when it was really on.
Seems like a simple fix, right?
Go back to the programmer and rewrite the code.
But we couldn't do it.
We had to live with it for a long period of time through training until another whole load was built.
Because the load that would fix this dumb little bi-level mistake was also the one that steered you during ascent.
So everything was frozen and there was great fear of changing anything that would ricochet through to something truly critical.
And so, even when we got to STS-3, we flew with a book about this thick of program notes that told you, line after line, what was wrong, what was backwards, what was a trap.
And really a big challenge to the crew to embrace and live with because there just wasn't time in the program to build another load, check it out through all the different labs that had to certify it.
That is load for modular software where you can change something here and be assured you are not bollixing up something else in the same machine.
OMS, RCS.
These are the two pods.
On either side of the vertical tail are a couple big OMS, propellant tanks, oxidizer and fuel hypergolic.
We didn't need any ignition system.
As soon as these came together in a thruster you had thrust.
And there were, I think, 44 main thrusters.
Here are four yaw jets firing out to the side.
S&B pitch jets.
The same thing repeated on the other side.
And up in the nose are the forward RCS.
They had jets that fired up and out to the side and down.
And I mentioned the controls in the cockpit, which we did.
One of my great successes was getting the orbiter panels to be arranged in a plumbing layout so that you could see what you were pressurizing when you turned on the helium for a given let and that sort of thing.
Helium tanks are what pressurized in a really cleaver system of OH control.
That is when you are in zero G and you have propellant floating around randomly.
And you turn on the helium to push it toward the outlet.
If the outlet is uncovered, the helium is going to squirt right out.
But they had an elaborate system of surface tension kind of baffles that kept and trapped fuel near the outlet of the tank so that when the thrust came on and all the propellant went down where it should, it sort of recharged this chamber at the bottom of the tanks to keep it flowing smoothly and not lose all the pressurization.
I remember my first impression of what reaction control jets would be like was based on science fiction movies where you would see the spacecraft out there and there were little squirts of jets that nudge you around.
And we went out to the Ground test facility at White Sands on a fieldtrip out there where they actually had the forward RCS setup, and they were going to do real firings of the main jets out there.
And we were probably a couple hundred feet away from the test setup.
We were out there waiting for this sequence of jet firing to start on a test.
And the first firing of one of the main jets, which are right up around 900 pounds thrust, was just this short pulse.
And everybody went about three feet off the ground.
I mean it was like you fired a howitzer.
It was just stunning.
Wham, wham, and it just blew you away.
And the realization of we are going to be in the cockpit here, and these things are firing right outside the windshield.
They did.
In space, the ignition transient ricochets through the structure.
You can hear this kind of bombing effect.
But once they are on, if it is a long duration firing, it is silent.
And the same way visually, you can see a flash when the fuel is led a little bit in front of the oxidizer to make sure you don't overheat something.
And so, there is incomplete combustion at the ignition.
You can see a flash go out there.
But what is on during an OMS burn, for instance, when these OMS engines are firing and you are looking back at the aft, you see the flash and they are on and they are pushing and you can feel it.
But otherwise invisible is the way I remember.
On the other hand, there are vernier jets, little guys.
I don't know if they are shown here.
But there are little 25 pound thrusters or something used 98% of the time on orbit.
You really are doing that kind of thing with the vernier jets.
And you cannot hear those or see anything when they are going off.
In fact, we got everybody quiet and kind of held onto the walls just trying to see if we could barely detect when a vernier jet fired physically in a cockpit.
Interesting.
The orbiter is one giant heat transfer machine.
Lots of calories of heat energy pumped around from one end to the other.
And so, there are water loops within the cabin that transfer the heat to Freon loops.
The fluid is pumped through the radiators to reject heat.
In spite of all these heat rejection systems then there are heaters everywhere, whole banks of heaters to keep stuff from freezing.
Then for the landing and takeoff phases, there are flash evaporators that flash water and/or ammonia.
When you are down in the atmosphere you use ammonia to cool the loops.
And, of course, the payload bay doors are closed.
And it turns out STS-3, a major test of it was test all these loops and how the structure responded to long durations of given attitudes with the sun shining on one side and cold space on the other side and how the structure would distort or hopefully not distort and how all that system would work.
As it turned out, if you like lots of redundancy, this is probably the weak point I thought in the whole orbiter design, we've got three engines.
We proved that you can get there with only two.
But the Freon loops, there were two redundant through the radiators.
And if one Freon loop pumped quick you are in a severe emergency.
That means stop everything and power down severely.
Turn off all the computers except one, get in a proper attitude and go into a panic reconfiguration to come back and land.
Originally, the design had three Freon loops.
It could use one and go on with no concern.
But, as it turned out, the heat loads were so high and the weight criticality determined that all we had electrical power to run were two loops.
And so we ended up with a single Freon loop which fortunately, as far as I know, never failed.
But if it had it would have been a crash deorbit landing under severely powered down conditions.
While it is kind of mundane and people don't think about it as a primary system, the thermal control really was critical.
Payload bay doors, this is a busy chart but indicates these are really complicated.
60 feet long and something like 120 little motor driven latches to latch them closed.
And you don't just go chunk close like the bomb bay on a B52 and it is closed.
To close the payload bay doors you get them close and then the latches along the bulked, all in a zipper fashion, have to sequentially close.
And then you go down the center section to latch the two left and right doors together.
All software controlled.
And there was a lot of concern that we address on STS-3 particularly about whether this is all going to work.
If you have been floating around with these pointed to dark space, would they distort to where the zipper scheme would work?
We worried about it enough that part of our emergency training, contingency training, a lot of time was spent underwater in the neutral buoyancy tank looking at a system going out EVA with a series of come-alongs, you know, the kind of things you use to pull the motor out a car, that kind of thing, that we could go down there and hookup and latch the doors shut to actually manually get them shut and then latch them shut with these.
Fortunately, that has never been used.
And this complex system has worked good, but it was scary the first time we flew it on STS-3.
That was a big concern.
What I have got here is hydraulic system.
Hydraulics are hydrazine powered turbines that run at some ridiculous RPM, like 200,000 RPM to drive hydraulic pumps back here in the backend.
And then they are crucial during launch because they have got to hold the elevons in position.
But they also have to provide the muscle to gimbal the main engines and keep you steering straight during launch.
During entry they are what you use to fly when you get into a dynamic pressure situation.
Control the rudder speed brake and the body flap.
The body flap is a big surface that sticks out underneath the main engine bells and is a slow moving trem device.
The body flat is always moved.
You go through this mach 25 to mach zero envelope.
And the trem conditions necessary with these big elevons varies a lot.
So, to desaturate the elevons and keep them basically faired with the wing, the body flap is your trim device.
And it is like this row of tables long and this wide.
I remember asking to see the motor.
It is driven by a whole lot of mechanical advantage by a hydraulic motor.
A guy took me around.
I couldn't find the actuator for the body flap.
He showed me.
The motor was this big around and this long, a little hydraulic motor.
So you can do anything with mechanical advantage.
But it didn't have to move very fast.
Finally, on the system pictures here, the RMS.
Hopefully, you have heard about this.
It was built by Spara up in Canada.
The Canadians main contribution to the orbiter program.
A pretty magical device.
It can go on either side, but for our flights mounted on the left-hand side or the right if you're looking out the back windows at it.
And it had joints logically called the shoulder joint, elbow, wrist joint.
All electric motor driven.
But the interesting thing was a control.
You had hand controllers mounted in the aft part on the right console in the back where you looked out.
And you could work in the mode where you actually drive in the end left, right, up, down, in, out without regard to whatever joint was done.
That is all resolved by the software.
And worked well.
The end effector was a cleaver kind of a hallow cylindrical device.
It would rotate inside and three cables that would be curved around the periphery inside this cavity when activated.
I put this over a post on whatever you wanted to grab onto with a knob on the end.
You go over the post and then rotate the inner barrel and the cables would wrap up and grab onto the post.
And then they would retract in to rigidize.
And that was the grabber on the RMS.
And we flew this on STS-3.
I will show you later.
Anyway, that is the subsystem review.
Let me show you how things went during approach and landing tests.
Now, let's see.
I have never figured out how to imbed video and PowerPoint so I am going to be doing it manually.
Any questions while we are waiting on this to come up?
Well, this is the desert around where I live and a Gila monster and a Joshua tree to set the scene.
And here you have OV-101.
The numbering system, as it turned out, is kind of interesting.
They built OV-99 as the structural test article.
It was strictly built to be in the lab and do all the structural testing on it.
101 then was the first orbiter build, the Enterprise, being hauled from Palmdale across the backgrounds and the desert roads to get over to Edwards in these scenes.
And 101 was the ALT airplane.
You can see kind of phony looking RCS.
It has, as I mentioned, only the systems that you needed to fly atmospheric flight.
This was a big thing getting it across the country.
It had a Pitot boom on it.
It has just simulated thermal protection.
Styrofoam painted to look kind of like they thought the space vehicles would look but wasn't real.
It was put in the mating and docking device which is at NASA Dryden where I work now.
Hoisted it up.
And then the 747 and 905, the first Shuttle carrier aircraft, was pulled under and they mated them up.
I remember flying out to Edwards and seeing this unlikely combination.
And the first time I saw the two together, I thought they cannot be serious.
But then I realized I was going to be in it.
There were 13 total flights.
This is a taxi test.
When you have an aerodynamic test you can sneak up, and so you do baby steps.
This was just a high speed down the runway to see if it would rotate and if anything would shake alarmingly.
And that will look good.
Preparations proceeded for the first of the captive inert flights.
This is Fitz Fulton and Tom McMurtry, both real good friends, and they were the crew on the 747 for this first milestone flight.
A lot of people were out watching and wondering if it would fly, which it did very well.
And off it went.
You notice a longer front attached structure there.
And the orbiter is purposely on there at angle of attack to produce lift.
I flew a 747 for ferry flights now and to haul orbiters back to the Cape when they land at Edwards, as the last one did.
And the nose, that front bipod is much shorter so that the orbiter is down for less drag for the combination.
But this setup here was so that we could get the orbiter off the top and not the tail.
There is a rumble.
The crew noticed a shaking.
It is still the case when you fly.
These vertical fins were added onto the horizontal stabs to increase directional stability because the orbiter blanks a lot of flow over the normal vertical.
In fact, they overdid it a little bit.
There is too much structural stability which has resulted in the weird situation.
Your cross-wind limit, when you have an orbiter onboard, is greater than when it is not because you have so much directional stability and a crosswind.
There is a strong weathervaning and you run out of normal rudder without an orbiter, so you are good only to 15 knots crosswind with the orbiter on.
The limit is 20 knots.
And it is a good limit.
I have flown them both ways in cross winds.
And, indeed, at those numbers you run out of rudder as you land and are rolling out.
The strong weathervane intensity is leading you out this side of the runway.
Anyway, that looked good.
And so I came back and we had a big post-flight party.
ALT was great for parties.
We had 13 post-flight parties.
And, at the same time we were doing the ALT flights, we are developing a Shuttle training aircraft.
This is Gulfstream II with big side force generators on the belly.
That is an airborne simulator.
The left seat is set up with orbiter instruments.
And the right seat was the standard gulfstream cockpit that had flown thousands of dives at the ground in the SGA preparing for the whole idea of first time for sure unpowered landing that you have to do right the first time.
And back in the simulator here for lots more runs.
And it was a great time because everything was brand new, very interesting.
This is a camera and an actual model.
This is before the days of computer generated video.
It flew the little camera down to landing.
And we learned a lot about [NOISE OBSCURES] characteristics.
One that we really wondered about, when you made a lateral input to roll the orbiter in the simulator, it would bang you.
It was this lateral lurch, we call it.
It would jolt you sideways.
And we thought this cannot be real.
It doesn't make any sense.
No airplane we ever flew does that.
We squawked it.
They came back and said the equations are all worked out so we live with it and thought it was just an artifact that snuck in there and didn't worry about it.
Now, the captive-active started where we actually got in and cranked things up.
On the second flight, I was chasing in a T-38.
And Engle and Truly were in the cockpit.
And I noticed a shiny look on the side.
That big X on the CRT is what happens when the computer that is driving that CRT quits.
That is sort of the panic symbol when the X shows up.
Anyway, I noticed a leak, as it turned out to be.
We had that severe hydrazine leak.
Hydrazine is bad stuff to have leaking.
And one of the APU supply tanks let go.
Plus, they had an over temp.
We found things that were wrong that made these captive flights worthwhile.
If we had just gone and launched off on the first one, it could have been more exciting than it needed to be.
We had five of those.
And Fred Hayes and I got the odd numbered flights, one, three and five.
Five more post-flight parties.
And then we decided we were ready to go.
Well, this is the last of the captive-active flights.
The scheme by which we plan to get off of here and not take off the tail of the 747 was that the 747 would go as high as it could go.
It could get up to around 27,000 feet with engines over-boosted.
Pratt & Whitney said you could go this long with extra power.
And then nose over and speed up to 240 knots.
It was climbing maybe to 190, 200 knots.
With a little downhill push, we got up to 240.
At 240 the SEA pilots would go to idle power and air brakes up to put drag on the carrier.
At that condition of 240 knots and the angle of attack of the orbiter, it was actually lifting more than its own weight.
So the result is we really dropped the carrier, as it turns out.
There is three-quarters of a G difference between the two.
And so, we got a lot of use.
For some reason Fred's left shoulder, he is in the left seat and he mashed the separation button after Fitz on the carrier aircraft hollered "carrier ready," that is they were back at idle with the speed brakes up, pushed the go button and bang, all seven explosive bolts would blow.
Control Room was in Houston, strange enough.
We are talking to people back in the old JSC Control Room when we are out at Edwards flying these flights.
It was just like they were there.
And we were coming up on a condition.
And this was in August of '77.
All the ALT flights happened in 1977.
You notice there is a tail cone on here.
The tail cone was flown on the first three flights.
That makes a big difference.
It smoothes the flow s that the buffet on the tail, the carrier aircraft was less, and it just about doubled the L/D of the orbiter.
We are going to arm.
And we had two chases, one off to the right and in trail separation.
Just like the engineer said and the load cells that we had on there, testing exactly which was the thing tugging before we did it, said we could do straight up.
This is slow motion.
Nothing is leaking.
The vortex is out at the wing tips causing condensation and away we went.
And simultaneous with this bang that shutters through the airplane with all these explosive volts a big X on the CRT in front of me.
We had cue cards and we had practiced this.
My job, while Fred flew, he actually got a vertical clear from this chase pilot and then rolled into a bank and a lateral clear from the guy that was chasing from behind and knew that it was OK to push over.
Because we are only now at 200 knots and we wanted to get to 300.
And it took a considerable pushover to get up to speed.
Meanwhile, I am pulling circuit breakers on rate gyros and turning off accelerometers at our preplanned procedure to accommodate flaws in the redundancy management.
When you lose one computer you also take out part of the sensors in the flight control system.
And, to be sure that a subsequent failure wouldn't put you in worse shape, it took about a minute's worth of manual reconfiguration on my part.
And I kind of missed the whole first part of the flight.
When I turned around I realized hey, this thing is flying good, it is smooth and is on speed.
And Fred was flying.
We flew a rectangular pattern out of a downwind leg here as we nosed over.
And I got my chance to roll into a left 90 degree turn and fly base leg.
And, as I put the stick over to roll, we got this big lateral arch just like the simulator said.
It just stunned me.
Son of a gun, they were right after all.
It was just the nature of something flying at high alpha and rotating about a stability axis and the cockpit gets slid sideways.
It has big elevons so it flies like a fighter.
It really does.
And you are doing it all with a little stick that doesn't move much.
Anyway, we came down final at 290, I think it was, flared out.
My job was to put the gear down, a critical function.
And we are landing on lakebed 1-7 on the dry lake at Edwards which is something like ten miles long, so we have pretty good margins there.
And Fred took the airplane back from ALT and I got to fly base leg.
But we touched down at about 185, I think it was, nice and smooth.
It was beautiful.
The speed brake comes out to help cushion the de-rotation, nose gear down and he raised a little dust as he rolled to a stop.
So it was a good day.
Now, we had a really good post-flight party after this one.
And see the rudder speed brake flared there and the elevons fold up.
And so, now Fred truly did it again.
A couple weeks later or thereabouts they got their turn and they did much the same flight, repeated it.
Oh, one thing.
You see the Pitot boom out there.
The Pitot boom is strictly on the Enterprise, put out there to get good air data out in front of this big blivit of an airplane which is hard to get good measurements.
Let me get the next one going here, if I can figure out how to get out of this.
The excitement came later.
On STS-1, I looked out there when were on final, I had to look out and look at the Pitot boom, and it is going through an arch of about three feet.
It was built with the perfect resonance of everything else with the airplane, and it was just a blur out there going back and forth.
And I really thought it was going to break off.
Was that your only Pitot source or did you have the normal ones [OVERLAPPING VOICES]?
Good question.
Somehow we had at least inertially derived ones.
Everything looked good.
And, actually, it still put out good airspeed, even though it was a blur.
And I think the beta averaged out.
Anyway, it wasn't a problem we noticed other than visually it looked like it was going to break off.
They changed and stiffened it up so it wasn't a problem on subsequent flights.
Now, you can see here we are at free flight four now which was the other crew's turn.
The tail cone is off and our final pressure glide slope with the tail cone on was 11 degrees.
With tail cone off it was 23 degrees, halves your L/D.
So you really are in a steep dive.
Normal ILS into Logan out here would be 2.5 degrees so you are coming down gangbusters.
And it is even worse in ALT than it was in the orbital missions which were coming down with the same configuration but much heavier.
Heavier you've got more gravity working for you and you can fly shallower.
It doesn't seem to figure, but that is the way it is.
So 19 degrees is a normal return from orbit.
Final pressure glide slope we were up in the 20s.
Kind of catching up a little.
One of our jobs on the flight preceding this was to try the braking.
We just let it roll out on the first couple flights.
They wanted to do brake tests.
And Fred jumped on the brakes after we touched down and another resonance problem, interaction between the structure and the anti-skid, and I thought it was going to break the airplane.
It was just violent shuttering, shaking, everything rattling in the cockpit.
And I hollered to Fred, get off the brakes, we are going to brake something.
He said no, the card says we have got to do max braking.
I was pleading with him to get off the brakes, which he did.
Easy fix.
When they flew the next flight here they did hard braking, and it was smooth.
They went in to change the resister and the feedback electrons, and that did it.
I mean it was instant fix.
So there still is reason to do flight tests because surprises happen.
Now free flight five, it turns out there was a little brake here.
We are back at Houston and Prince Charles, who wasn't married yet at that time, came by on a visit.
And we gave him a ride in the simulator.
He had some flying time.
And he got into a horrendous PIO.
I mean he was out of control and crashed in the simulator.
And felt bad.
And we said that is because you're not used to the stick and everything.
And that was his tour.
Now, for this flight, he was actually out along the runway.
They had him out there so he could watch.
The objective here was to land on the concrete runway, put it right on the line, there is a white line 5,000 feet down, and Fred was really working at it, putting in a lot of inputs.
Look at the elevons going there.
You see them?
And he is making roll commands.
And so we logged a couple landings.
And down and rolled to a stop.
Afterwards, we called out and he came up to say hello.
He said oh, you made me feel good.
You guys did as bad as I did.
There he is right there with his binoculars.
Well, that scared everybody to death.
It looked worse from outside than it did in the cockpit.
And the flight control flaw that we found, though, was serious.
You've got elevons that do both pitch and roll.
So, in the software, when you are asking for both pitch and roll at the same time, it has got to decide which one do I move?
This is just a supposition of what would come.
The software had to give priority.
And they decided to give priority to pitch.
And roll was taken second priority when you had to limit the commands to the elevons to within the capability of the hydraulic system.
The result was a lag in roll response when you are working pitch hard.
And the lag was up around two-tenths to three-tenths of a second.
You may be aware if you studied flight control systems.
Any kind of system that has a quarter second delay between input and response is asking for a PIO, Pilot Induced Oscillation.
And we had a big time in roll.
Fred was clanking back and forth to stop and roll trying to level the wings out there that first skip.
I heard the controller banging over there.
And I suggested if you get off maybe it will damp.
We're doing this and, sure enough, that is what he did.
He stopped the inputs, the airplane damped right away and then it went OK.
But it was something we had to fix, obviously.
Here is a picture just a micro-second after liftoff on one of the last two.
Well, there was a decision with a lot of debate about do we go back and fix the flight control and fly some more ALT?
But this is a huge production and it was costing time and we weren't getting into orbit.
A decision was made now we will do it in simulators and in-flight simulators and fix the problem, but press on.
That was the end of ALT, at the end of which, when you do flight tests, you talk about opening the envelope of an airplane.
Well, the envelope of the Space Shuttle is 400 nautical miles altitude and mach 25.
And in ALT we got the four miles and 0.3 mach.
It is really down in the lower left corner, but worth doing.
Test results.
I mentioned as we went we had that APU fuel leak.
Fixed it.
Didn't come back.
The buffet lifter drag was right on what we expected, no surprises there.
Handling qualities were good, even with the lurch.
It was not a problem.
I mentioned the Pitot boom, the anti-skid.
SCA did the job, although it struggled to get up.
With the tail coming off, it could only get to about 18,000 feet.
So this was not a good program to build up your flying time.
It was about a total, in three flights, of about 12 minutes of flying time.
And the biggie I mentioned was the handling qualities and the PIO tendency that we fixed with lots of studies later and changes in the way the control equations were written.
And, as a final thing, they did some more flights with the nose lowered down and determine what the ferry flight performance of the combination is.
I, later after flying in space, checked out on the 747 and still fly it on the ferry flights.
And it does the job.
It burns lots of gas.
I figured out that the combination across the country, the mileage is about 300 feet per gallon, so a football field per gallon of fuel.
40,000 pounds an hour is the field flow.
It goes about 1.5 times its own length when we get on the gas, but it gets there.
I don't have to buy the fuel.
A lot of people said wasn't this scary, ALT?
Actually, from a risk standpoint, we had ejection seats.
If anything had gone wrong, anywhere along the line, we could have pulled the handle and jumped out.
The scariest part of it was when we got in, and during the captive flights, we would just walk up a stairway and crawl through the hatch.
And the same way getting out.
But after the captive active flights, we would land still on the orbiter.
And they had this cherry picker to get us out.
Now, the situation here is it's a round hatch hedged at the bottom.
They only wanted on person at a time on the end of this thing.
You open a round hatch and then they bring this up, the guy on the ground.
And he doesn't want to get too close and ding so he leaves about this much of a gap from this round hatch which is 60 feet above the ground that you have got to crawl out on your knees, then stand up and then make the leap across the thing onto this wobbly cage up there.
That is how our captive actives went.
Where you wearing flight suits for all these tests?
Yeah, we were wearing flight suits and helmets like you would in a 518 or any fighter.
Not pressure suits.
We didn't have a severe cabin pressure risk at 25,000 feet.
Let me get into the full screen mode here.
I put in some thoughts about what is different when you test an aircraft versus a spacecraft.
With an airplane you can do little baby steps, incremental tests.
You can do static runs of the propulsion up to the full power in the airplane itself.
Then you taxi it slow and then faster and faster down the runway.
You can rotate the nose a little bit.
The first flight you will fly slow and leave the gear down and then land it soon and check everything.
And you could hold off for a perfect day to do it.
And then you go faster and faster to expand the envelope.
If anything goes wrong you can peel off and get back on the ground quick.
And, as I mentioned, you can always pull the handle and bail out if things go really bad.
When you have a spacecraft it is all or nothing.
You have got to use the propulsion at its full power.
All the critical systems and subsystems have got to work.
You cannot make a quick abort.
Long delays like days are sometimes required to get back to the conditions where you can make an appropriate landing.
And the weather could turn sour at the time.
When you are going to flight test the spacecraft you have got to have a lot of redundancy in the design to allow for the bad day that has to carry it through in a considerable amount of time.
You necessarily have got to test everything to the limit, wind tunnels, CVD, thermal vacuum because the environment is lots more stressful.
You have got to have a lot of instrumentation so that you can tell what did go wrong or be sure that everything went right.
You are talking about an army of controllers in a control room to pull it off with a complex system like the Shuttle.
And lots of simulation to get everybody trained.
A biggie is to verify the software.
I mentioned the software is guaranteed to be wrong on the first release and several subsequent.
It is just the nature of building software.
And so getting that checked out is a major one.
Procedures.
When we flew the orbital flight, the stack of checklists just laid on top of each other was about this high, for 51F, and it weighed 210 pounds.
It was called the Flight Data File.
And then lots of time and expense getting everybody really ready to go.
And being sure that when you're going into an unknown part of the envelope that the margins on the flight control are adequate.
So the surprises and the aeroderivatives that hit you can be accommodating.
I have rambled on a long time here.
We can take the five-minute break, I can answer questions, however you want to do it.
Maybe it's time for a stretch.
Let's do a two-minute break.
These people are tough.
All right.
Break is over.
The next assignment for me was being put on an orbital flight test, and specifically STS-3.
A two man crew, Jack Lousma, a Marine, and myself from the Air Force, we were in pressure suits, you will notice there, and on ejection seats in the Columbia.
This is OV-102.
101 was the Enterprise.
102 was the Columbia.
And our countdown went smoothly.
This was in March of '82.
And this is a signature moment in your life when the SRBs light.
I mean it looks smooth and nice as we are flying through a cloud here on launch, logging weather time I might add, but it is rough and noisy and rattley.
And the acceleration is right away, and it builds up to 3Gs.
When the main engines are throttled back, the whole 3Gs.
And so, it is no doubt you are on your way somewhere really fast wherever the solids want to go.
Everything worked good right up to MECO.
And the external tank, as you saw, drifted away.
And then here come the first order of business, opening the payload bay doors so we can start rejecting heat with the radiators that are mounted on the inside.
We are tailed down.
And as you watch here, we are going right over Los Angeles at the time.
This is just north of LA.
You can see LA.
Sorry, this is Santa Monica, LAX and Palos Verdes right there.
Anyway, the first thing we had to do is get out of Theodolite, that is a sighting device that surveyors use, and take a daunting number of readings on little targets that were pasted around inside the payload bay and on the doors to measure where they were.
And we did that through the flight all the way along.
It wasn't fast enough.
Out in front of the nose there were some dark patches.
That is where tile were, white tile, and fell off during launch.
They actually found some washed up on the beach.
Fortunately, none of the black tile on the bottom fell off but we did lose some on the top.
Zero G is really one of the delights.
There are two really good things about flying in orbit, weightlessness and the view out the window.
And that is like nothing that you experience, even flying high-performance airplanes.
We had the RMS cranked up.
That was my job as RMS operator.
And we had a package in the back, the plasma diagnostic package to grab and move out there.
There are some things that show you the housekeeping area.
In the corner there is the john.
And long duration spaceflight exercise is important.
It is really not a big deal on a seven-day flight.
Jack is a Marine, as I mentioned.
He has got to have his daily ration of pushups, so here we go.
How about a one-handed pushup, Jack?
How about a no-handed pushup?
This is a quick summary.
We went our seven days.
And the lakebed at Edwards, they had a really wet spring, was unusable, so we went to White Sands, New Mexico.
The first day, right as we are about to light the OMS engines to start deorbit, they said hold off because this terrible dust storm is going on.
And then we waited a delightful day with nothing in the flight plane much to enjoy ourselves, and then came down the next day.
Now we find another gotcha in handling quantities here.
Right here Jack thought the nose might be coming down too fast and tried to stop it, and ended up doing a little wheelie.
Subsequent analysis showed that the whole response of the vehicle changes when you are down on the main gear only.
And nobody thought of that.
It would seem like, well, they are worried about pitch in the air but not while you are rolling on the main gear trying to de-rotate.
So it resulted in more software changes.
Another thing we learned, they were pushing for auto-land.
We flew the whole turn onto final and down final hands off in auto-land.
The program wanted to take this out of the hands of the crew, so here on the third flight they had us sitting there watching the airplane fly down to pre-flare.
And then the first chance Jack got to grab the stick was at the last minute to do the flare and landing.
And that was dumb when you look back, really dumb.
And so, now all flight commanders, if it has been auto down to 40,000 feet where you go subsonic, we will fly manually all the way down to get a feel for the airplane.
It makes a lot more sense.
That is the way flights are done now.
Here is where we started from orbit.
This is how Cape Canaveral looks.
Right here is the shuttle landing strip that is down there.
You can see it.
And the launch pads are these two dots.
And so, that is where usually a shuttle flight starts and ends.
Here is a picture of our payload bay on STS-3.
And we did have a payload.
Previous crews had pretty much empty payload bay, but we had an environmental measurement thing, a lot of instrumentation.
Then this PDP, Plasma Diagnostic Package that we actually had something to grab with the manipulator arm, as you see here.
I looked around and found, I don't know if you can recognize this, but that is Cape Cod back there.
A picture of MIT.
Not a good picture.
There have got to be lots of better ones.
And here is where we ended up, the White Sands area near Alamogordo, New Mexico.
What we learned, this is a pretty cryptic summary.
Training was a real problem.
The first two crews that flew STS-1 and STS-2 had all the attention of the training people in the simulators.
They had priority, properly so.
We, on the third flight, had low priority and really struggled.
And what we found, when we got in the simulators, is when they didn't work right nobody could explain why.
Many times we try on ascent and crash and burn.
And you always were scratching your head is this real?
Is this the way the orbiter flies or is it some flaw in the simulator?
Early on you are developing not only the orbiter but the training facilities.
And the people didn't understand the system.
We had more exposure from ALT to subsystems than the people that were supposed to be the experts.
It was a group learning process and a long way from what happens now when crews go in.
And they get the straight word right from the beginning on how things are.
The tiles fell off.
Not good.
They changed the bonding scheme, improved the quality control and, fortunately for the program, no critical tiles fell off.
But that was alarming.
We looked out there, we reported it, took pictures of it looking at these gaps in the system right out in front of the windshield.
But Ground said don't worry, that is a cool area during entry.
Entry is at 40 degrees angle of attack so it is cool on top.
They were right.
Everything was fine.
Another tile thing was after the flight, actually, delay that.
It was really more after the next flight.
I mentioned how the ride is.
The first two minutes were rough and exciting.
The next six or seven are unbelievably smooth when you get on the main engines, and then more vibration that you are getting sitting in the chairs now.
And it is just a steady relentless push.
It is just a spectacular right.
But you are back in the seat.
Had an APU oil overheat during that 3G part.
Actually, about two minutes before MECO, Ground called and said we needed to shut down one APU.
We were up past the point where we could live without all three APUs running.
I had a chance to test what I worked on years previously with the cockpit design.
You are concerned about visibility when you are squashed in the seat at 3Gs and access.
And the APU switches, we had to put them down here.
And, I remember worrying specifically, if we have got to get to the APU switches, the guy on the right seat under 3Gs, can he lift the helmet and look over there and find the correct switch?
And you do not want to shut down the wrong one, one of the good ones.
And it turned out, I guess with the adrenaline lifting this heavy helmet at 3Gs, I did it without even thinking about it and went right to the switch.
So it worked.
General habitability was good in the orbiter.
The first time anybody lived there for eight days.
A good way to think about living in the orbiter, think of it as a camping trip.
You go on a camping trip you don't worry about hot water, showers and all the niceties of home.
You get in that frame of mind and it is fine.
The food is adequate, the facilities are OK, although in our case the toilet ground to a halt.
It is a drum that rotates to centrifugally sling all fecal material out and contain it.
And it ground to a halt.
We were in a real camping trip for a while on that, although it worked out.
The arm worked great, RMS.
The main jets I mentioned earlier.
Of course, you don't use the main jets much.
It would be hard to sleep with them booming away, but you use the Verniers most of the time.
Verniers are critical to smooth operation over long periods of time.
We spent a lot of time, for instance, tail sun.
For like three days we put the tail right at the sun and stayed in that inertial attitude.
And so it cooked the backend while the front end of the orbiter is cold.
We took all the Theodolite measurements, opened and closed the doors.
All of that worked fine.
The systems handled it.
A lot of it was ground analysis.
And how the temperatures were, were extensively instrumented.
And we did notice a strange thing.
Not strange but it became apparent.
When you are in orbit, when you are on the sunny side of the earth, as soon as the sun comes up it is high noon bright, but there is no atmosphere around to diffuse the light.
When we were at tail sun, it is hidden behind perfectly eclipsed by the aft structure.
And so, there is no sun ball to be seen anywhere, even though it is bright sunny from just the reflections of the structure you can see out the window.
I noticed, looking at the radiators, which are very, very smooth concave surfaces out there and shiny silver colored.
It looked like there were bright diamonds all along the surface when we were tail sun.
And I finally figured out it was just little specks of dust on the radiators, which you wouldn't even notice normally.
But exactly tail sun, the sun's rays are coming right down exactly tangent to that surface.
And the least little thing on it would flare.
And interesting effect which took me a while to figure out what all these bright spots on the radiators are.
Entry on STS-3 was done to an about 1:00 in the morning landing at White Sands, New Mexico.
We started the entry.
Halfway around the world it is dark, and most of the entry was in the dark.
The plasma light show that happens, as you get down in the atmosphere, the ionization that happens produces a spectacular scene out the windshield.
I mean it also had us thinking about the missing tiles that are going.
But a night entry is really a show.
The next flight was a day entry which was not nearly as spectacular.
I mentioned about grabbing the stick too late, unfair to Jack, and then the de-rotation.
Now onto part three.
How am I doing?
I might make it yet.
STS-51F was a whole different situation than the earlier test flights.
One big difference is there were seven people instead of two.
With seven we had to add a little bit on the patch to put the extra two names.
This is the ninth flight.
There are nineteen stars.
Primary experiments.
This was a science flight.
It was a space lab but without the laboratory.
All equipment in the payload bay full stem to stern with telescopes mostly.
And all the telescopes mounted on the pointing system we had were solar telescopes.
So we had the sun in the accurate part of the sky.
Here is Orion and Leo, the backwards question mark.
Nineteen stars.
A lot of symbology here.
A short entertaining video.
We had seven people, including three mission specialists, which Dr. Hoffman was when he was down in Houston.
And we had two payload specialists, guys who were not full time astronauts.
They were solar physicists.
And they were the experts on the solar telescopes that we had and went along to fly.
First try here at main engine ignition ended three seconds after ignition.
We felt the spacecraft rock around, the noise and then nothing.
And we had a pad shutdown because the left engine, the automatic system detected a failure, a slow-acting valve and it shut us off.
So we are there wondering if we are on fire and whether we ought to race out and so forth.
It was tense for a while but there wasn't a fire, just this automatic shutdown.
We crawled out.
We were told there would be a two-week delay, so we took the kids to Disneyland and had a good time, went back home and started the whole training cycle again.
And came back two weeks later and got a good start this time.
But not after lying on our back on the pads strapped in for five hours because of some software glitches that had to be fixed and computers reloaded and tested.
And one of our reports was five hours is the limit.
I mean you are lying on your back with your feet up in a pressure suit strapped in.
That is more than is reasonable to ask prior to launch.
On the way up, well, I will mention that later.
Here is the instrument system.
This baby was built in Germany.
It is this gimbal device that held these four solar telescopes and the star trackers.
Here is a physics experiment.
The OMS engines are lit and produces one-sixteenth of a G. The thrust of the orbital maneuvering engines.
If you are trying to sleep, that is what happens when the OMS engines come on.
And you saw the water going across.
It is déjà vu.
We had the same old PDP package on this, but with a difference this time.
We took it out of the payload bay and brought it up here.
That is my right ear you are looking at in profile out the window.
It is crowded up there with seven people up on the flight deck all trying to get a view here, but we let go of it.
And it had a momentum wheel in there to cause it to spin up slowly.
And we backed away from it and flew a couple of very challenging loops around it.
Here is a sequel to the hairbrush.
This is not me.
It is Story Musgrave who has the same hairdo as I do.
And he is getting spiffed up for the TV show.
That is all we did is horse around for seven days.
It got extended a day to eight days again.
And this landing was back at Edwards, runway 23 on August 6th when it was 105 degrees, lots of heat waves.
This landing was by far the most forward CG because we still had all that equipment in the payload bay.
And we were almost the heaviest orbiter landing, so getting the nose down before it fell down, crashed and broke the nose gear was one of my primary concerns on this one.
More about the actual launch itself.
There is the customary view.
They put a camera right in close and got this really close-up view, but it was about 5.5 minutes after this picture.
We had delayed five hours.
It is late afternoon so very quickly heading out over the Atlantic we were in darkness.
And then, bang.
Remember the left engine got us on the pad abort?
Well, the center engine shut down.
We were at 3Gs and all of a sudden we are at 2Gs or something like that.
Instant square wave cutoff and acceleration.
Looking there and confirming the center engine shut down.
What happened was there are a number of sensors on the main engines.
The particular one that got us was the fuel high pressure turbo pump.
A piece of machinery running at 75,000 RPM high pressures.
The high-tech gear that makes the main engine so efficient, IISP, but really run at the limits of metallurgy and technology.
The sensor that senses the output temperature was a series of four platinum wires in the cavity in the output of that turbo pump.
They had some tendency for the wires to burn through.
When they burn through, they showed over-temp was the electrical result of a burn through, one of these sensors.
Well, we had two burn throughs.
And the software said that must mean there are two sensors that say you are too hot, I will shut the motor off.
And it did.
With no reaction we just kept going straight and smooth, but it now put us into what is called an abort.
Well, here is the crew.
There are seven of us.
I need to practice this more.
But, anyway, it was a two shift operation.
We were going to work around the clock, so we had a red team and a blue team.
And I was the commander so I had a striped shirt.
Abort to orbit.
We had to call from the ground, confirm what we knew already, which means turning that rotary knob to ATO, abort to orbit, kind of an oxymoron in a way.
It is not really an abort.
It is pressing on and pushing the button behind it.
It loads the software in to do a couple of things, lower the insertion target.
We had tried to get as high orbit as performance would permit, but now we are going lower.
Engine down costs you total performance.
And we also started dumping OMS fuel.
We had to turn on the OMS engines, and we ended up dumping 4500 pounds of OMS fuel, a little bit of extra thrust but a significant loss of weight to allow the remaining performance to get us to an acceptable orbit.
And, interestingly, the last sim we did in training in Houston, before we went to the Cape to fly about two days before, was an integrated sim where we are tied into all the controllers and everybody is working as a team.
And the very last run we had was an ATO, abort to orbit.
And, son of a gun, that is what we did.
We always had trouble starting a watch.
And you try to do everything internally on the crew so that if you lose communication with the ground you can complete the emergency properly on your own.
On ATOs before, we had to start a stopwatch at the time we started the OMS dump.
And we had a chart to go in to find out how many minutes of OMS dump we needed to do given at what velocity we lost the engine.
And Story's job was to do the watch.
In the excitement of all the other stuff we had to do to get the OMS on, we would always forget the watch.
But this time he got the watch going so we actually did it right.
And we knew that when we got to where we burned out all the available fuel, which we did, we went to a fuel depletion cutoff, that we would be really close to having to do right away another OMS burn to get an acceptable orbit.
We are right on the boundary, as it turned out.
We dumped the right amount of OMS, every last drop of main propellant gone, we were just on the boundary of not having to do the first OMS burn and we could relax 45 minutes later halfway around the world to do the final circularization.
So it worked out good, except we were in a lot lower orbit than we had planned.
And everything about this space lab flight depended on the orbit, so we were into a giant re-plan.
Here is a nice picture of the IPS.
Gordon, I also explain the situation of the inhibit switch on and off.
I don't know how much you were aware of what was going on there in the cockpit.
There is a switch in the cockpit.
You have a lot of parameters that are monitored in the software on the main engines which, if they go out of tolerance, will shut the engine off.
In fact, that is exactly why we lost the center engine.
But, right after that engine shutdown, we were in a state where we really don't want to lose another engine.
Because if, with only one left, we did not have the performance to get to orbit, we would have had to have gone to Zaragoza, Spain and land in the middle of the night where it was a 10,000 foot overcast and raining.
I mean if we pulled that off we would have deserved some kind of metal.
And so, as a preplanned procedure, when you lose one engine before, the point where you can stagger into orbit on one remaining engine you inhibit the limits.
You throw a switch that says you're telling the engines to keep running regardless of over-temps or anything because you take the chance, which is a big one if something let go, to not be susceptible that got us for the first engine, that is a sensor failure.
And so we did that.
We inhibited the limits until we got to a point in the energy profile where we were "press to MECO."
That is the call.
Press to MECO means that you are now at a velocity where you can make it into acceptable low orbit on one engine if you should lose another one.
Press to MECO then the plan was we will re-enable the limits so we are not hanging it out on a possible second engine failure.
And we would accept a second engine failure because we knew we could make it on one engine.
So we did that, the call came up, we went to enable.
And then, before we got there, we got another call from the ground that said go back to inhibit.
Well, it wasn't time to get a full explanation.
We dutifully did it.
What they saw on the ground is more failures on the right engine.
Remember the left got us on the pad abort?
The center engine shut down on the way up.
And, to make everything equal, the right engine, the same sensors they can see on the ground, it is not visible in the cockpit, each of those four platinum wires they saw starting to go.
Clearly a design problem here.
And they could tell those were going to go and get us that second engine, so that is why we went back to inhibit for the rest of the way.
It made a good war story, but we got there.
We started operating the IPS.
A platform built in Germany with all the telescopes.
These are start trackers to align where it is.
And this baby could point.
The spec was one arch second of accuracy.
An arch second is the size of a dime.
If you are standing at the capital and looking at dime at the Lincoln Memorial, that is how much.
But, when we cranked it up, it didn't work worth a hoot.
They just couldn't get it to stabilize or point where it was supposed to.
And, for three days, the payload specialists and the MSs were in constant communication with the ground.
And they patched the software in a major way and kept sending up new software loads to get this thing to work.
And viola, on the third day they finally got it right, then it worked well and got a lot of good solar science.
Just an aside, when you are looking at pictures from space, you tend to think you walk around out here and the atmosphere is this big ocean of air above you, right?
Lots of air way up there.
This little bitty blue line right there, barely perceptible, there always appears on any sunlit picture showing the limb of the earth, that is the same blue as the scattering that causes the sky to be blue.
And that is how thick it is.
It is just this little thin shell of atmosphere around.
At 18,000 feet three miles up you have lost half the atmospheric pressure.
And that is laying on an 8,000 mile diameter globe.
And so that is where all of human history has happened, in that little shell over there laying around the globe.
A philosophical point for you engineers.
What is flying on orbit like?
Well, it is one of the 215 pounds of checklist.
Here I am checking propellant usage against what we've got and where we are because we dumped 4,500 pounds of crucial fuel that we needed to make the planned most maneuvers that ever been done in the orbit on our mission.
We didn't want an extra blip out of an RCS thruster, and it worked very well.
We got through seven days and even had enough to have another day of science on the eighth day.
Flying is mostly watching the clock.
Everything happens according to time.
And, since you have 16 sunrises and sunsets every 24 hours, you never know what time it is without looking at a watch.
And you are looking at a crew activity plan with everything laid out.
And we even had an alarm system you could set in the CRT to make a tone to wake you up.
It is a lot of clock watching to pull off a complex mission like this.
It is just the way life is on orbit.
Now, here is one of our really serious high-level scientific objectives.
This was the official name, the Carbonated Beverage Test.
I don't know if it was an MIT guy or not.
He was a real serious academic type.
An Indian I think.
Anyway, he built a Coke can that had a lot of pressure regulators in there and laminar flow annular things.
So many that this Coke can only held four ounces of Coke.
But he talked to the White House.
Reagan was president.
And somebody on the staff approved we are going to fly these Coke cans and see if they had a system to provide pop in orbit with the regular kind of carbonation and dispense OK. We got the briefing and we had the four stowed in a locker.
And I got them out at the start of the test.
But 30 days before launch, all of a sudden this big thing came down.
Pepsi got wind of this experiment and went to the White House staff and they got Pepsi put on.
Pepsi went into a crash program to fly more cans of Pepsi.
Here is Carl trying the Pepsi.
You notice that looks a lot like a shaving cream can.
Well, that's how the Pepsi came out, just like shaving cream, this frothy ball of stuff.
I mean theirs was the last minute deal.
It didn't have any pressure regulators in it.
It just went out there.
Now we worried about having to make a judgment, which tasted better.
Well, the problem was we didn't have any kind of refrigerator so they are both warm.
Warm pop is not my favorite.
And so, they were not anything anybody fought over, either kind.
If you let the Pepsi float around it would coalesce a little CO2 in random fashion, but the physics of it, Carl found by blowing very carefully on the edge you could get it spinning up.
And spinning Pepsi would have the froth at the poles and the liquid around the equator.
And we always had a tall guy standing by in case it drifted into something.
This is the end of the experiment here.
He got it into his mouth without touching any of it.
We had the same PDP but, as I mentioned and as you saw in the movie, we let go of it.
We actually maneuvered all around the orbit and looked at the wake through the plasma, let go of it.
The proximity ops were very extensive and challenging and many, many maneuvers.
That is in the Caribbean.
The tongue of the ocean.
A spectacular coral reef around very deep water.
Here is the Gibraltar.
The rock, I believe, is out on the point.
The Mediterranean here and the Atlantic here.
You've heard you cannot see borders in space.
Not true.
This is Israel this way and Egypt this way and the Gaza strip along here, which has been in the news as of late.
And in the same area the Dead Sea, the Sea of Galilee.
Here is most of Israel in one picture.
A little closer to home.
This is most of New Jersey.
Long Island.
There is Manhattan.
Hudson River and East River.
There are about 10,000 copies of this picture.
This is Southern California.
This is San Diego right here.
Los Angeles.
And, in high rise print, you can see water here which is San Francisco Bay.
And the center of it all is the Antelope Valley right here where I live and where Edwards is right here.
This is the San Andreas Fault right through here.
Closer up of LA which, at the end of 51F, we came right by Catalina Island, right up the Harbor Freeway into Edwards.
And we went by Harbor Freeway at about mach 3.
And closer up of the dry lake at Edwards.
And the landing.
Flight test results most of which I have mentioned already.
And, to finish up, I have reports on a memory stick here, the pilot reports we wrote as crews which you can take at your leisure.
If you are interested, copy it off.
They are available.
Not great works of art, but the specific things that we learned on doing the flight test.
And these are just thoughts of transitioning into operation from test mode, which I got to see the whole gamut from beginning to full operational.
I apologize for running over.
I don't think anybody minded.
As you see, everybody is listening very intently.
Thank you very much for taking the time.
[APPLAUSE]
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Professor: So, again welcome to 18.01.
We're getting started today with what we're calling Unit One, a highly imaginative title.
And it's differentiation.
So, let me first tell you, briefly, what's in store in the next couple of weeks.
The main topic today is what is a derivative.
And, we're going to look at this from several different points of view, and the first one is the geometric interpretation.
That's what we'll spend most of today on.
And then, we'll also talk about a physical interpretation of what a derivative is.
And then there's going to be something else which I guess is maybe the reason why Calculus is so fundamental, and why we always start with it in most science and engineering schools, which is the importance of derivatives, of this, to all measurements.
So that means pretty much every place.
That means in science, in engineering, in economics, in political science, etc.
Polling, lots of commercial applications, just about everything.
Now, that's what we'll be getting started with, and then there's another thing that we're gonna do in this unit, which is we're going to explain how to differentiate anything.
So, how to differentiate any function you know.
And that's kind of a tall order, but let me just give you an example.
If you want to take the derivative - this we'll see today is the notation for the derivative of something - of some messy function like e ^ x arctanx.
We'll work this out by the end of this unit.
All right?
Anything you can think of, anything you can write down, we can differentiate it.
All right, so that's what we're gonna do, and today as I said, we're gonna spend most of our time on this geometric interpretation.
So let's begin with that.
So here we go with the geometric interpretation of derivatives.
And, what we're going to do is just ask the geometric problem of finding the tangent line to some graph of some function at some point.
Which is to say (x0, y0).
So that's the problem that we're addressing here.
Alright, so here's our problem, and now let me show you the solution.
So, well, let's graph the function.
Here's it's graph.
Here's some point.
All right, maybe I should draw it just a bit lower.
So here's a point P. Maybe it's above the point x0.
x0, by the way, this was supposed to be an x0.
That was some fixed place on the x-axis.
And now, in order to perform this mighty feat, I will use another color of chalk.
How about red?
OK.
So here it is.
There's the tangent line, Well, not quite straight.
Close enough.
All right?
I did it.
That's the geometric problem.
I achieved what I wanted to do, and it's kind of an interesting question, which unfortunately I can't solve for you in this class, which is, how did I do that?
That is, how physically did I manage to know what to do to draw this tangent line?
But that's what geometric problems are like.
We visualize it.
We can figure it out somewhere in our brains.
It happens.
And the task that we have now is to figure out how to do it analytically, to do it in a way that a machine could just as well as I did in drawing this tangent line.
So, what did we learn in high school about what a tangent line is?
Well, a tangent line has an equation, and any line through a point has the equation y - y0 is equal to m the slope, times x - x0.
So here's the equation for that line, and now there are two pieces of information that we're going to need to work out what the line is.
The first one is the point.
That's that point P there.
And to specify P, given x, we need to know the level of y, which is of course just f(x0).
That's not a calculus problem, but anyway that's a very important part of the process.
So that's the first thing we need to know.
And the second thing we need to know is the slope.
And that's this number m. And in calculus we have another name for it.
We call it f prime of x0.
Namely, the derivative of f. So that's the calculus part.
That's the tricky part, and that's the part that we have to discuss now.
So just to make that explicit here, I'm going to make a definition, which is that f '(x0) , which is known as the derivative, of f, at x0, is the slope of the tangent line to y = f (x) at the point, let's just call it p. All right?
So, that's what it is, but still I haven't made any progress in figuring out any better how I drew that line.
So I have to say something that's more concrete, because I want to be able to cook up what these numbers are.
I have to figure out what this number m is.
And one way of thinking about that, let me just try this, so I certainly am taking for granted that in sort of non-calculus part that I know what a line through a point is.
So I know this equation.
But another possibility might be, this line here, how do I know - well, unfortunately, I didn't draw it quite straight, but there it is - how do I know that this orange line is not a tangent line, but this other line is a tangent line?
Well, it's actually not so obvious, but I'm gonna describe it a little bit.
It's not really the fact, this thing crosses at some other place, which is this point Q.
But it's not really the fact that the thing crosses at two place, because the line could be wiggly, the curve could be wiggly, and it could cross back and forth a number of times.
That's not what distinguishes the tangent line.
So I'm gonna have to somehow grasp this, and I'll first do it in language.
And it's the following idea: it's that if you take this orange line, which is called a secant line, and you think of the point Q as getting closer and closer to P, then the slope of that line will get closer and closer to the slope of the red line.
And if we draw it close enough, then that's gonna be the correct line.
So that's really what I did, sort of in my brain when I drew that first line.
And so that's the way I'm going to articulate it first.
Now, so the tangent line is equal to the limit of so called secant lines PQ, as Q tends to P. And here we're thinking of P as being fixed and Q as variable.
All right?
Again, this is still the geometric discussion, but now we're gonna be able to put symbols and formulas to this computation.
And we'll be able to work out formulas in any example.
So let's do that.
So first of all, I'm gonna write out these points P and Q again.
So maybe we'll put P here and Q here.
And I'm thinking of this line through them.
I guess it was orange, so we'll leave it as orange.
All right.
And now I want to compute its slope.
So this, gradually, we'll do this in two steps.
And these steps will introduce us to the basic notations which are used throughout calculus, including multi-variable calculus, across the board.
So the first notation that's used is you imagine here's the x-axis underneath, and here's the x0, the location directly below the point P. And we're traveling here a horizontal distance which is denoted by delta x.
So that's delta x, so called.
And we could also call it the change in x.
So that's one thing we want to measure in order to get the slope of this line PQ.
And the other thing is this height.
So that's this distance here, which we denote delta f, which is the change in f. And then, the slope is just the ratio, delta f / delta x.
So this is the slope of the secant.
And the process I just described over here with this limit applies not just to the whole line itself, but also in particular to its slope.
And the way we write that is the limit as delta x goes to 0.
And that's going to be our slope.
So this is slope of the tangent line.
OK. Now, This is still a little general, and I want to work out a more usable form here, a better formula for this.
And in order to do that, I'm gonna write delta f, the numerator more explicitly here.
The change in f, so remember that the point P is the point (x0, f(x0)).
All right, that's what we got for the formula for the point.
And in order to compute these distances and in particular the vertical distance here, I'm gonna have to get a formula for Q as well.
So if this horizontal distance is delta x, then this location is (x0 delta x).
And so the point above that point has a formula, which is (x0 delta x, f(x0 and this is a mouthful, delta x)).
All right, so there's the formula for the point Q.
Here's the formula for the point P. And now I can write a different formula for the derivative, which is the following: so this f'(x0) , which is the same as m, is going to be the limit as delta x goes to 0 of the change in f, well the change in f is the value of f at the upper point here, which is (x0 delta x), and minus its value at the lower point P, which is f(x0), divided by delta x.
All right, so this is the formula.
I'm going to put this in a little box, because this is by far the most important formula today, which we use to derive pretty much everything else.
And this is the way that we're going to be able to compute these numbers.
So let's do an example.
This example, we'll call this example one.
We'll take the function f(x) , which is 1/x .
That's sufficiently complicated to have an interesting answer, and sufficiently straightforward that we can compute the derivative fairly quickly.
So what is it that we're gonna do here?
All we're going to do is we're going to plug in this formula here for that function.
That's all we're going to do, and visually what we're accomplishing is somehow to take the hyperbola, and take a point on the hyperbola, and figure out some tangent line.
That's what we're accomplishing when we do that.
So we're accomplishing this geometrically but we'll be doing it algebraically.
So first, we consider this difference delta f / delta x and write out its formula.
So I have to have a place.
So I'm gonna make it again above this point x0, which is the general point.
We'll make the general calculation.
So the value of f at the top, when we move to the right by f(x), so I just read off from this, read off from here.
The formula, the first thing I get here is 1 / x0 delta x.
That's the left hand term.
Minus 1 / x0, that's the right hand term.
And then I have to divide that by delta x. OK, so here's our expression.
And by the way this has a name.
This thing is called a difference quotient.
It's pretty complicated, because there's always a difference in the numerator.
And in disguise, the denominator is a difference, because it's the difference between the value on the right side and the value on the left side here.
OK, so now we're going to simplify it by some algebra.
So let's just take a look.
So this is equal to, let's continue on the next level here.
This is equal to 1 / delta x times... All I'm going to do is put it over a common denominator.
So the common denominator is (x0 delta x)x0.
And so in the numerator for the first expressions I have x0, and for the second expression I have x0 delta x.
So this is the same thing as I had in the numerator before, factoring out this denominator.
And here I put that numerator into this more amenable form.
And now there are two basic cancellations.
The first one is that x0 and x0 cancel, so we have this.
And then the second step is that these two expressions cancel, the numerator and the denominator.
Now we have a cancellation that we can make use of.
So we'll write that under here.
And this is equals -1 / (x0 delta x)x0.
And then the very last step is to take the limit as delta x tends to 0, and now we can do it.
Before we couldn't do it.
Why?
Because the numerator and the denominator gave us 0 / 0.
But now that I've made this cancellation, I can pass to the limit.
And all that happens is I set this delta x to 0, and I get -1/x0^2.
So that's the answer.
All right, so in other words what I've shown - let me put it up here- is that f'(x0) = -1/x0^2.
Now, let's look at the graph just a little bit to check this for plausibility, all right?
What's happening here, is first of all it's negative.
It's less than 0, which is a good thing.
You see that slope there is negative.
That's the simplest check that you could make.
And the second thing that I would just like to point out is that as x goes to infinity, that as we go farther to the right, it gets less and less steep.
So as x0 goes to infinity, less and less steep.
So that's also consistent here, when x0 is very large, this is a smaller and smaller number in magnitude, although it's always negative.
It's always sloping down.
All right, so I've managed to fill the boards.
So maybe I should stop for a question or two.
Yes?
Student: [INAUDIBLE] Professor: So the question is to explain again this limiting process.
So the formula here is we have basically two numbers.
So in other words, why is it that this expression, when delta x tends to 0, is equal to -1/x0^2 ?
Let me illustrate it by sticking in a number for x0 to make it more explicit.
All right, so for instance, let me stick in here for x0 the number 3.
Then it's -1 / (3 delta x)3.
That's the situation that we've got.
And now the question is what happens as this number gets smaller and smaller and smaller, and gets to be practically 0?
Well, literally what we can do is just plug in 0 there, and you get (3 0)3 in the denominator.
Minus one in the numerator.
So this tends to -1/9 (over 3^2).
And that's what I'm saying in general with this extra number here.
Other questions?
Yes.
Student: [INAUDIBLE] Professor: So the question is what happened between this step and this step, right?
Explain this step here.
Alright, so there were two parts to that.
The first is this delta x which is sitting in the denominator, I factored all the way out front.
And so what's in the parentheses is supposed to be the same as what's in the numerator of this other expression.
And then, at the same time as doing that, I put that expression, which is the difference of two fractions, I expressed it with a common denominator.
So in the denominator here, you see the product of the denominators of the two fractions.
And then I just figured out what the numerator had to be without really... Other questions?
OK.
So I claim that on the whole, calculus gets a bad rap, that it's actually easier than most things.
But there's a perception that it's harder.
And so I really have a duty to give you the calculus made harder story here.
So we have to make things harder, because that's our job.
And this is actually what most people do in calculus, and it's the reason why calculus has a bad reputation.
So the secret is that when people ask problems in calculus, they generally ask them in context.
And there are many, many other things going on.
And so the little piece of the problem which is calculus is actually fairly routine and has to be isolated and gotten through.
But all the rest of it, relies on everything else you learned in mathematics up to this stage, from grade school through high school.
So that's the complication.
So now we're going to do a little bit of calculus made hard.
By talking about a word problem.
We only have one sort of word problem that we can pose, because all we've talked about is this geometry point of view.
So far those are the only kinds of word problems we can pose.
So what we're gonna do is just pose such a problem.
So find the areas of triangles, enclosed by the axes and the tangent to y = 1/x.
OK, so that's a geometry problem.
And let me draw a picture of it.
It's practically the same as the picture for example one.
We only consider the first quadrant.
Here's our shape.
All right, it's the hyperbola.
And here's maybe one of our tangent lines, which is coming in like this.
And then we're trying to find this area here.
Right, so there's our problem.
So why does it have to do with calculus?
It has to do with calculus because there's a tangent line in it, so we're gonna need to do some calculus to answer this question.
But as you'll see, the calculus is the easy part.
So let's get started with this problem.
First of all, I'm gonna label a few things.
And one important thing to remember of course, is that the curve is y = 1/x.
That's perfectly reasonable to do.
And also, we're gonna calculate the areas of the triangles, and you could ask yourself, in terms of what?
Well, we're gonna have to pick a point and give it a name.
And since we need a number, we're gonna have to do more than geometry.
We're gonna have to do some of this analysis just as we've done before.
So I'm gonna pick a point and, consistent with the labeling we've done before, I'm gonna to call it (x0, y0).
So that's almost half the battle, having notations, x and y for the variables, and x0 and y0, for the specific point.
Now, once you see that you have these labellings, I hope it's reasonable to do the following.
So first of all, this is the point x0, and over here is the point y0.
That's something that we're used to in graphs.
And in order to figure out the area of this triangle, it's pretty clear that we should find the base, which is that we should find this location here.
And we should find the height, so we need to find that value there.
Let's go ahead and do it.
So how are we going to do this?
Well, so let's just take a look.
So what is it that we need to do?
I claim that there's only one calculus step, and I'm gonna put a star here for this tangent line.
I have to understand what the tangent line is.
Once I've figured out what the tangent line is, the rest of the problem is no longer calculus.
It's just that slope that we need.
So what's the formula for the tangent line?
Put that over here.
it's going to be y - y0 is equal to, and here's the magic number, we already calculated it.
It's in the box over there.
It's -1/x0^2 ( x - x0).
So this is the only bit of calculus in this problem.
But now we're not done.
We have to finish it.
We have to figure out all the rest of these quantities so we can figure out the area.
All right.
So how do we do that?
Well, to find this point, this has a name.
We're gonna find the so called x-intercept.
That's the first thing we're going to do.
So to do that, what we need to do is to find where this horizontal line meets that diagonal line.
And the equation for the x-intercept is y = 0.
So we plug in y = 0, that's this horizontal line, and we find this point.
So let's do that into star.
We get 0 minus, oh one other thing we need to know.
We know that y0 is f(x0) , and f(x) is 1/x , so this thing is 1/x0.
And that's equal to -1/x0^2.
And here's x, and here's x0.
All right, so in order to find this x value, I have to plug in one equation into the other.
So this simplifies a bit.
This is -x/x0^2.
And this is plus 1/x0 because the x0 and x0^2 cancel somewhat.
And so if I put this on the other side, I get x / x0^2 is equal to 2 / x0.
And if I then multiply through - so that's what this implies - and if I multiply through by x0^2 I get x = 2x0.
OK, so I claim that this point weve just calculated it's 2x0.
Now, I'm almost done.
I need to get the other one.
I need to get this one up here.
Now I'm gonna use a very big shortcut to do that.
So the shortcut to the y-intercept is to use symmetry.
All right, I claim I can stare at this and I can look at that, and I know the formula for the y-intercept.
It's equal to 2y0.
All right.
That's what that one is.
So this one is 2y0.
And the reason I know this is the following: so here's the symmetry of the situation, which is not completely direct.
It's a kind of mirror symmetry around the diagonal.
It involves the exchange of (x, y) with (y, x); so trading the roles of x and y.
So the symmetry that I'm using is that any formula I get that involves x's and y's, if I trade all the x's and replace them by y's and trade all the y's and replace them by x's, then I'll have a correct formula on the other ways.
So if everywhere I see a y I make it an x, and everywhere I see an x I make it a y, the switch will take place.
So why is that?
That's just an accident of this equation.
That's because, so the symmetry explained... is that the equation is y= 1 / x.
But that's the same thing as xy = 1, if I multiply through by x, which is the same thing as x = 1/y.
So here's where the x and the y get reversed.
OK now if you don't trust this explanation, you can also get the y-intercept by plugging x = 0 into the equation star.
OK?
We plugged y = 0 in and we got the x value.
And you can do the same thing analogously the other way.
All right so I'm almost done with the geometry problem, and let's finish it off now.
Well, let me hold off for one second before I finish it off.
What I'd like to say is just make one more tiny remark.
And this is the hardest part of calculus in my opinion.
So the hardest part of calculus is that we call it one variable calculus, but we're perfectly happy to deal with four variables at a time or five, or any number.
In this problem, I had an x, a y, an x0 and a y0.
That's already four different things that have various relationships between them.
Of course the manipulations we do with them are algebraic, and when we're doing the derivatives we just consider what's known as one variable calculus.
But really there are millions of variable floating around potentially.
So that's what makes things complicated, and that's something that you have to get used to.
Now there's something else which is more subtle, and that I think many people who teach the subject or use the subject aren't aware, because they've already entered into the language and they're so comfortable with it that they don't even notice this confusion.
There's something deliberately sloppy about the way we deal with these variables.
The reason is very simple.
There are already four variables here.
I don't wanna create six names for variables or eight names for variables.
But really in this problem there were about eight.
I just slipped them by you.
So why is that?
Well notice that the first time that I got a formula for y0 here, it was this point.
And so the formula for y0, which I plugged in right here, was from the equation of the curve.
y0 = 1 / x0.
The second time I did it, I did not use y = 1 / x. I used this equation here, so this is not y = 1/x.
That's the wrong thing to do.
It's an easy mistake to make if the formulas are all a blur to you and you're not paying attention to where they are on the diagram.
You see that x-intercept calculation there involved where this horizontal line met this diagonal line, and y = 0 represented this line here.
So the sloppines is that y means two different things.
And we do this constantly because it's way, way more complicated not to do it.
It's much more convenient for us to allow ourselves the flexibility to change the role that this letter plays in the middle of a computation.
And similarly, later on, if I had done this by this more straightforward method, for the y-intercept, I would have set x equal to 0.
That would have been this vertical line, which is x = 0.
But I didn't change the letter x when I did that, because that would be a waste for us.
So this is one of the main confusions that happens.
If you can keep yourself straight, you're a lot better off, and as I say this is one of the complexities.
All right, so now let's finish off the problem.
Let me finally get this area here.
So, actually I'll just finish it off right here.
So the area of the triangle is, well it's the base times the height.
The base is 2x0 the height is 2y0, and a half of that.
So it's 1/2( 2x0)(2y0) , which is (2x0)(y0), which is, lo and behold, 2.
So the amusing thing in this case is that it actually didn't matter what x0 and y0 are.
We get the same answer every time.
That's just an accident of the function 1 / x.
It happens to be the function with that property.
All right, so we have some more business today, some serious business.
So let me continue.
So, first of all, I want to give you a few more notations.
And these are just other notations that people use to refer to derivatives.
And the first one is the following: we already wrote y = f(x).
And so when we write delta y, that means the same thing as delta f. That's a typical notation.
And previously we wrote f' for the derivative, so this is Newton's notation for the derivative.
But there are other notations.
And one of them is df/dx, and another one is dy/ dx, meaning exactly the same thing.
And sometimes we let the function slip down below so that becomes d / dx (f) and d/ dx(y) .
So these are all notations that are used for the derivative, and these were initiated by Leibniz.
And these notations are used interchangeably, sometimes practically together.
They both turn out to be extremely useful.
This one omits - notice that this thing omits- the underlying base point, x0.
That's one of the nuisances.
It doesn't give you all the information.
But there are lots of situations like that where people leave out some of the important information, and you have to fill it in from context.
So that's another couple of notations.
So now I have one more calculation for you today.
I carried out this calculation of the derivative of the function 1 / x. I wanna take care of some other powers.
So let's do that.
So Example 2 is going to be the function f(x) = x^n.
n = 1, 2, 3; one of these guys.
And now what we're trying to figure out is the derivative with respect to x of x^n in our new notation, what this is equal to.
So again, we're going to form this expression, delta f / delta x.
And we're going to make some algebraic simplification.
So what we plug in for delta f is ((x delta x)^n - x^n)/delta x.
Now before, let me just stick this in then I'm gonna erase it.
Before, I wrote x0 here and x0 there.
But now I'm going to get rid of it, because in this particular calculation, it's a nuisance.
I don't have an x floating around, which means something different from the x0.
And I just don't wanna have to keep on writing all those symbols.
It's a waste of blackboard energy.
There's a total amount of energy, and I've already filled up so many blackboards that, there's just a limited amount.
Plus, I'm trying to conserve chalk.
Anyway, no 0's.
So think of x as fixed.
In this case, delta x moves and x is fixed in this calculation.
All right now, in order to simplify this, in order to understand algebraically what's going on, I need to understand what the nth power of a sum is.
And that's a famous formula.
We only need a little tiny bit of it, called the binomial theorem.
So, the binomial theorem which is in your text and explained in an appendix, says that if you take the sum of two guys and you take them to the nth power, that of course is (x delta x) multiplied by itself n times.
And so the first term is x^n, that's when all of the n factors come in.
And then, you could have this factor of delta x and all the rest x's.
So at least one term of the form (x^(n-1))delta x.
And how many times does that happen?
Well, it happens when there's a factor from here, from the next factor, and so on, and so on, and so on.
There's a total of n possible times that that happens.
And now the great thing is that, with this alone, all the rest of the terms are junk that we won't have to worry about.
So to be more specific, there's a very careful notation for the junk.
The junk is what's called big O((delta x)^2).
What that means is that these are terms of order, so with (delta x)^2, (delta x)^3 or higher.
All right, that's how.
Very exciting, higher order terms.
OK, so this is the only algebra that we need to do, and now we just need to combine it together to get our result.
So, now I'm going to just carry out the cancellations that we need.
So here we go.
We have delta f / delta x, which remember was 1 / delta x times this, which is this times, now this is (x^n nx^(n-1) delta x this junk term) - x^n.
So that's what we have so far based on our previous calculations.
Now, I'm going to do the main cancellation, which is this.
All right.
So, that's 1/delta x( nx^(n-1) delta x this term here).
And now I can divide in by delta x.
So I get nx^(n-1) now it's O(delta x).
There's at least one factor of delta x not two factors of delta x, because I have to cancel one of them.
And now I can just take the limit.
And the limit this term is gonna be 0.
That's why I called it junk originally, because it disappears.
And in math, junk is something that goes away.
So this tends to, as delta x goes to 0, nx ^ (n-1).
And so what I've shown you is that d/dx of x to the n minus - sorry -n, is equal to nx^(n-1).
So now this is gonna be super important to you right on your problem set in every possible way, and I want to tell you one thing, one way in which it's very important.
One way that extends it immediately.
So this thing extends to polynomials.
We get quite a lot out of this one calculation.
Namely, if I take d / dx of something like (x^3 5x^10) that's gonna be equal to 3x^2, that's applying this rule to x^3.
And then here, I'll get 5*10 so 50x^9.
So this is the type of thing that we get out of it, and we're gonna make more hay with that next time.
Question.
Yes.
I turned myself off.
Yes?
Student: [INAUDIBLE] Professor: The question was the binomial theorem only works when delta x goes to 0.
No, the binomial theorem is a general formula which also specifies exactly what the junk is.
It's very much more detailed.
But we only needed this part.
We didn't care what all these crazy terms were.
It's junk for our purposes now, because we don't happen to need any more than those first two terms.
Yes, because delta x goes to 0.
OK, see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu.
Okay so I'd like to begin the second lecture by reminding you what we did last time.
So last time, we defined the derivative as the slope of a tangent line.
So that was our geometric point of view and we also did a couple of computations.
We worked out that the derivative of 1 / x was -1 / x^2.
And we also computed the derivative of x ^ nth power for n = 1, 2, etc., and that turned out to be x, I'm sorry, nx^(n-1).
So that's what we did last time, and today I want to finish up with other points of view on what a derivative is.
So this is extremely important, it's almost the most important thing I'll be saying in the class.
But you'll have to think about it again when you start over and start using calculus in the real world.
So again we're talking about what is a derivative and this is just a continuation of last time.
So, as I said last time, we talked about geometric interpretations, and today what we're gonna talk about is rate of change as an interpretation of the derivative.
So remember we drew graphs of functions, y = f(x) and we kept track of the change in x and here the change in y, let's say.
And then from this new point of view a rate of change, keeping track of the rate of change of x and the rate of change of y, it's the relative rate of change we're interested in, and that's delta y / delta x and that has another interpretation.
This is the average change.
Usually we would think of that, if x were measuring time and so the average and that's when this becomes a rate, and the average is over the time interval delta x.
And then the limiting value is denoted dy/dx and so this one is the average rate of change and this one is the instantaneous rate.
Okay, so that's the point of view that I'd like to discuss now and give you just a couple of examples.
So, let's see.
Well, first of all, maybe some examples from physics here.
So q is usually the name for a charge, and then dq/dt is what's known as current.
So that's one physical example.
A second example, which is probably the most tangible one, is we could denote know the letter s by distance and then the rate of change is that what we call speed.
So those are the two typical examples and I just want to illustrate the second example in a little bit more detail because I think it's important to have some visceral sense of this notion of instantaneous speed.
And I get to use the example of this very building to do that.
Probably you know, or maybe you don't, that on Halloween there's an event that takes place in this building or really from the top of this building which is called the pumpkin drop.
So let's illustrates this idea of rate of change with the pumpkin drop.
So what happens is this building, well let's see here's the building, and here's the dot, that's the beautiful grass out on this side of the building, and then there's some people up here and very small objects, well they're not that small when you're close to them, that get dumped over the side there.
And they fall down.
You know everything at MIT or a lot of things at MIT are physics experiments.
That's the pumpkin drop.
So roughly speaking, the building is about 300 feet high, we're down here on the first usable floor.
And so we're going to use instead of 300 feet, just for convenience purposes we'll use 80 meters because that makes the numbers come out simply.
So we have the height which starts out at 80 meters at time 0 and then the acceleration due to gravity gives you this formula for h, this is the height.
So at time t = 0, we're up at the top, h is 80 meters, the units here are meters.
And at time t = 4 you notice, (5 * 4^2) is 80.
I picked these numbers conveniently so that we're down at the bottom.
Okay, so this notion of average change here, so the average change, or the average speed here, maybe we'll call it the average speed since that's, over this time that it takes for the pumpkin to drop is going to be the change in h / the change in t. Which starts out at, what does it start out as?
It starts out as 80, right?
And it ends at 0.
So actually we have to do it backwards.
We have to take 0 - 80 because the first value is the final position and the second value is the initial position.
And that's divided by 4 - 0; times 4 seconds minus times 0 seconds.
And so that of course is -20 meters per second.
So the average speed of this guy is 20 meters a second.
Now, so why did I pick this example?
Because, of course, the average, although interesting, is not really what anybody cares about who actually goes to the event.
All we really care about is the instantaneous speed when it hits the pavement and so that's can be calculated at the bottom.
So what's the instantaneous speed?
That's the derivative, or maybe to be consistent with the notation I've been using so far, that's d/dt of h. All right?
So that's d/dt of h. Now remember we have formulas for these things.
We can differentiate this function now.
We did that yesterday.
So we're gonna take the rate of change and if you take a look at it, it's just the rate of change of 80 is 0, minus the rate change for this -5t^2, that's minus 10t.
So that's using the fact that d/dt of 80 is equal to 0 and d/dt of t^2 is equal to 2t.
The special case... Well I'm cheating here, but there's a special case that's obvious.
I didn't throw it in over here.
The case n = 2 is that second case there.
But the case n = 0 also works.
Because that's constants.
The derivative of a constant is 0.
And then the factor n there's 0 and that's consistent.
And actually if you look at the formula above it you'll see that it's the case of n = -1.
So we'll get a larger pattern soon enough with the powers.
Okay anyway.
Back over here we have our rate of change and this is what it is.
And at the bottom, at that point of impact, we have t = 4 and so h' which is the derivative is equal to -40 meters per second.
So twice as fast as the average speed here, and if you need to convert that, that's about 90 miles an hour.
Which is why the police are there at midnight on Halloween to make sure you're all safe and also why when you come you have to be prepared to clean up afterwards.
So anyway that's what happens, it's 90 miles an hour.
It's actually the buildings a little taller, there's air resistance and I'm sure you can do a much more thorough study of this example.
All right so now I want to give you a couple of more examples because time and these kinds of parameters and variables are not the only ones that are important for calculus.
If it were only this kind of physics that was involved, then this would be a much more specialized subject than it.
Is And so I want to give you a couple of examples that don't involved time as a variable.
So the third example I'll give here is.
The letter t often denotes temperature, and then dt/dx would be what is known as the temperature gradient.
Which we really care about a lot when we're predicting the weather because it's that temperature difference that causes air flows and causes things to change.
And then there's another theme which is throughout the sciences and engineering which I'm going to talk about under the heading of sensitivity of measurements.
So let me explain this.
I don't want to belabor it because I just am doing this in order to introduce you to the ideas on your problem set which are the first case of this.
So on problem set one you have an example which is based on a simplified model of GPS, sort of the Flat Earth Model.
And in that situation, well, if the Earth is flat it's just a horizontal line like this.
And then you have a satellite, which is over here, preferably above the earth, and the satellite or the system knows exactly where the point directly below the satellite is.
So this point is treated as known.
And I'm sitting here with my little GPS device and I want to know where I am.
And the way I locate where I am is I communicate with this satellite by radio signals and I can measure this distance here which is called h. And then system will computer this horizontal distance which is L. So in other words what is measured, so h measured by radios, radio waves and a clock, or various clocks.
And then L is deduced from h. And what's critical in all of these systems is that you don't know h exactly.
There's an error in h which will denote delta h. There's some degree of uncertainty.
The main uncertainty in GPS is from the ionosphere.
But there are lots of corrections that are made of all kinds.
And also if you're inside a building it's a problem to measure it.
But it's an extremely important issue, as I'll explain in a second.
So the idea is we then get at delta L is estimated by considering this ratio delta L/delta h which is going to be approximately the same as the derivative of L with respect to h. So this is the thing that's easy because of course it's calculus.
Calculus is the easy part and that allows us to deduce something about the real world that's close by over here.
So the reason why you should care about this quite a bit is that it's used all the time to land airplanes.
So you really do care that they actually know to within a few feet or even closer where your plane is and how high up it is and so forth.
All right.
So that's it for the general introduction of what a derivative is.
I'm sure you'll be getting used to this in a lot of different contexts throughout the course.
And now we have to get back down to some rigorous details.
Ok, everybody happy with what we've got so far?
Yeah?
Student: How did you get the equation for height?
Professor: Ah good question question.
The question was how did I get this equation for height?
I just made it up because it's the formula from physics that you will learn when you take 8.01 and, in fact, it has to do with the fact that this is the speed if you differentiate another time you get acceleration and acceleration due to gravity is 10 meters per second.
Which happens to be the second derivative of this.
But anyway I just pulled it out of a hat from your physics class.
So you can just say see 8.01 .
All right, other questions?
All right, so let's go on now.
Now I have to be a little bit more systematic about limits.
So let's do that now.
So now what I'd like to talk about is limits and continuity.
And this is a warm up for deriving all the rest of the formulas, all the rest of the formulas that I'm going to need to differentiate every function you know.
Remember, that's our goal and we only have about a week left so we'd better get started.
So first of all there is what I will call easy limits.
So what's an easy limit?
An easy limit is something like the limit as x goes to 4 of (x 3 / x^2 1).
And with this kind of limit all I have to do to evaluate it is to plug in x = 4 because, so what I get here is 4 + 3 / (4^2 1).
And that's just 7 / 17.
And that's the end of it.
So those are the easy limits.
The second kind of limit, well so this isn't the only second kind of limit but I just want to point this out, it's very important is that: derivatives are are always harder than this.
You can't get away with nothing here.
So, why is that?
Well, when you take a derivative, you're taking the limit as x goes to x0 of f(x), well we'll write it all out in all its glory.
Here's the formula for the derivative.
Now notice that if you plug in x = x0, always gives 0 / 0.
So it just basically never works.
So we always are going to need some cancellation to make sense out of the limit.
Now in order to make things a little easier for myself to explain what's going on with limits I need to introduce just one more piece of notation.
What I'm gonna introduce here is what's known as a left-hand and a right limit.
If I take the limit as x tends to x0 with a sign here of some function, this is what's known as the right hand limit.
And I can display it visually.
So what does this mean?
It means practically the same thing as x tends to x0 except there is one more restriction which has to do with this sign, which is we're going from the plus side of x0.
That means x is bigger than x0.
And I say right-hand, so there should be a hyphen here, right-hand limit because on the number line, if x zero is over here the x is to the right.
All right?
So that's the right-hand limit.
And then this being the left side of the board, I'll put on the right side of the board the left limit, just to make things confusing.
So that one has the minus sign here.
I'm just a little dyslexic and I hope you're not.
So I may have gotten that wrong.
So this is the left-hand limit, and I'll draw it.
So of course that just means x goes to x0 but x is to the left of x0 .
And again, on the number line, here's the x0 and the x is on the other side of it.
Okay, so those two notations are going to help us to clarify a bunch of things.
It's much more convenient to have this extra bit of description of limits than to just consider limits from both sides.
Okay so I want to give an example of this.
And also an example of how you're going to think about these sorts of problems.
So I'll take a function which has two different definitions.
Say it's x + 1, when x &gt; 0 and -x 2, when x &lt; 0.
So maybe put commas there.
So when x &gt; 0, it's x + 1.
Now I can draw a picture of this.
It's gonna be kind of little small because I'm gonna try to fit it down in here, but maybe I'll put the axis down below.
So at height 1, I have to the right something of slope 1 so it goes up like this.
All right?
And then to the left of 0 I have something which has slope -1, but it hits the axis at 2 so it's up here.
So I had this sort of strange antennae figure here is my graph.
Maybe I should draw these in another color to depict that.
And then if I calculate these two limits here, what I see is that the limit as x goes to 0 from above of f(x), that's the same as the limit as x goes to 0 of the formula here, x + 1.
Which turns out to be 1.
And if I take the limit, so that's the left-hand limit.
Sorry, I told you I was dyslexic.
This is the right, so it's that right-hand.
Here we go.
So now I'm going from the left, and it's f(x) again, but now because I'm on that side the thing I need to plug is the other formula, -x + 2, and that's gonna give us 2.
Now, notice that the left and right limits, and this is one little tiny subtlety and it's almost the only thing that I need you to really pay attention to a little bit right now, is that this, we did not need x = 0 value.
In fact I never even told you what f(0) was here.
If we stick it in we could stick it in.
Okay let's say we stick it in on this side.
Let's make it be that it's on this side.
So that means that this point is in and this point is out.
So that's a typical notation: this little open circle and this closed dot for when you include the.
So in that case the value of f(x) happens to be the same as its right hand limit, namely the value is 1 here and not 2.
Okay, so that's the first kind of example.
Questions?
Okay, so now our next job is to introduce the definition of continuity.
So that was the other topic here.
So we're going to define.
So f is continuous at x0 means that the limit of f(x) as x tends to x0 = f(x0) .
Right?
So the reason why I spend all this time paying attention to the left and the right and so on and so forth and focusing is that I want you to pay attention for one moment to what the content of this definition is.
What it's saying is the following: continuous at x0 has various ingredients here.
So the first one is that this limit exists.
And what that means is that there's an honest limiting value both from the left and right.
And they also have to be the same.
All right, so that's what's going on here.
And the second property is that f(x0) is defined.
So I can't be in one of these situations where I haven't even specified what f(x0) is and they're equal.
Okay, so that's the situation.
Now again let me emphasize a tricky part of the definition of a limit.
This side, the left-hand side is completely independent, is evaluated by a procedure which does not involve the right-hand side.
These are separate things.
This one is, to evaluate it, you always avoid the limit point.
So that's if you like a paradox, because it's exactly the question: is it true that if you plug in x0 you get the same answer as if you move in the limit?
That's the issue that we're considering here.
We have to make that distinction in order to say that these are two, otherwise this is just tautalogical.
It doesn't have any meaning.
But in fact it does have a meaning because one thing is evaluated separately with reference to all the other points and the other is evaluated right at the point in question.
And indeed what these things are, are exactly the easy limits.
That's exactly what we're talking about here.
They're the ones you can evaluate this way.
So we have to make the distinction.
And these other ones are gonna be the ones which we can't evaluate that way.
So these are the nice ones and that's why we care about them why we have a whole definitions associated with them.
All right?
So now what's next?
Well, I need to give you a a little tour, very brief tour, of the zoo of what are known as discontinuous functions.
So sort of everything else that's not continuous.
So, the first example here, let me just write it down here.
It's jump discontinuities.
So what would a jump discontinuity be?
Well we've actually already seen it.
The jump discontinuity is the example that we had right there.
This is when the limit from the left and right exist, but are not equal.
Okay, so that's as in the example.
Right?
In this example, the two limits, one of them was 1 and of them was 2.
So that's a jump discontinuity.
And this kind of issue, of whether something is continuous or not, may seem a little bit technical but it is true that people have worried about it a lot.
Bob Merton, who was a professor at MIT when he did his work for the Nobel prize in economics, was interested in this very issue of whether stock prices of various kinds are continuous from the left or right in a certain model.
And that was a very serious issue in developing the model that priced things that our hedge funds use all the time now.
So left and right can really mean something very different.
In this case left is the past and right is the future and it makes a big difference whether things are continuous from the left or continuous from the right.
Right, is it true that the point is here, here, somewhere in the middle, somewhere else.
That's a serious issue.
So the next example that I want to give you is a little bit more subtle.
It's what's known as a removable discontinuity.
And so what this means is that the limit from left and right are equal.
So a picture of that would be, you have a function which is coming along like this and there's a hole maybe where, who knows either the function is undefined or maybe it's defined up here, and then it just continues on.
All right?
So the two limits are the same.
And then of course the function in begging to be redefined so that we remove that hole.
And that's why it's called a removable discontinuity.
Now let me give you an example of this, or actually a couple of examples.
So these are quite important examples which you will be working with in a few minutes.
So the first is the function g(x), which is sin x / x, and the second will be the function h(x), which is 1 - cos x / x.
So we have a problem at g(0) , g(0) is undefined.
On the other hand it turns out this function has what's called a removable singularity.
Namely the limit as x goes to 0 of sin x / x does exist.
In fact it's equal to 1.
So that's a very important limit that we will work out either at the end of this lecture or the beginning of next lecture.
And similarly, the limit of 1 - cos x / x, as x goes to 0 is 0.
Maybe I'll put that a little farther away so you can read it.
Okay, so these are very useful facts that we're going to need later on.
And what they say is that these things have removable singularities, sorry removable discontinuity at x = 0.
All right so as I say, we'll get to that in a few minutes.
Okay so are there any questions before I move on?
Yeah?
Student: [INAUDIBLE] Professor: The question is: why is this true?
Is that what your question is?
The answer is it's very, very unobvious, I haven't shown it to you yet, and if you were not surprised by it then that would be very strange indeed.
So we haven't done it yet.
You have to stay tuned until we do.
Okay?
We haven't shown it yet.
And actually even this other statement, which maybe seems stranger still, is also not yet explained.
Okay, so we are going to get there, as I said, either at the end of this lecture or at the beginning of next.
Other questions?
All right, so let me just continue my tour of the zoo of discontinuities.
And, I guess, I want to illustrate something with the convenience of right and left hand limits so I'll save this board about right and left-hand limits.
So a third type of discontinuity is what's known as an infinite discontinuity.
And we've already encountered one of these.
I'm going to draw them over here.
Remember the function y is 1 / x.
That's this function here.
But now I'd like to draw also the other branch of the hyperbola down here and allow myself to consider negative values of x.
So here's the graph of 1 / x.
And the convenience here of distinguishing the left and the right hand limits is very important because here I can write down that the limit as x goes to 0 of 1 / x.
Well that's coming from the right and it's going up.
So this limit is infinity.
Whereas, the limit in the other direction, from the left, that one is going down.
And so it's quite different, it's minus infinity.
Now some people say that these limits are undefined but actually they're going in some very definite direction.
So you should, whenever possible, specify what these limits are.
On the other hand, the statement that the limit as x goes to 0 of 1 / x is infinity is simply wrong.
Okay, it's not that people don't write this.
It's just that it's wrong.
It's not that they don't write it down.
In fact you'll probably see it.
It's because people are just thinking of the right hand branch.
It's not that they're making a mistake necessarily, but anyway, it's sloppy.
And there's some sloppiness that we'll endure and others that we'll try to avoid.
So here, you want to say this, and it does make a difference.
You know, plus infinity is an infinite number of dollars and minus infinity is and infinite amount of debt.
They're actually different.
They're not the same.
So, you know, this is sloppy and this is actually more correct.
Okay, so now in addition, I just want to point out one more thing.
Remember, we calculated the derivative, and that was -1/x^2.
But, I want to draw the graph of that and make a few comments about it.
So I'm going to draw the graph directly underneath the graph of the function.
And notice what this graphs is.
It goes like this, it's always negative, and it points down.
So now this may look a little strange, that the derivative of this thing is this guy, but that's because of something very important.
And you should always remember this about derivatives.
The derivative function looks nothing like the function, necessarily.
So you should just forget about that as being an idea.
Some people feel like if one thing goes down, the other thing has to go down.
Just forget that intuition.
It's wrong.
What we're dealing with here, if you remember, is the slope.
So if you have a slope here, that corresponds to just a place over here and as the slope gets a little bit less steep, that's why we're approaching the horizontal axis.
The number is getting a little smaller as we close in.
Now over here, the slope is also negative.
It is going down and as we get down here it's getting more and more negative.
As we go here the slope, this function is going up, but its slope is going down.
All right, so the slope is down on both sides and the notation that we use for that is well suited to this left and right business.
Namely, the limit as x goes to 0 of -1 / x^2, that's going to be equal to minus infinity.
And that applies to x going to 0 and x going to 0-.
So both have this property.
Finally let me just make one last comment about these two graphs.
This function here is an odd function and when you take the derivative of an odd function you always get an even function.
That's closely related to the fact that this 1 / x is an odd power and x^1 is an odd power and x^2 is an even power.
So all of this your intuition should be reinforcing the fact that these pictures look right.
Okay, now there's one last kind of discontinuity that I want to mention briefly, which I will call other ugly discontinuities.
And there are lots and lots of them.
So one example would be the function y = sin 1 / x, as x goes to 0.
And that looks a little bit like this.
Back and forth and back and forth.
It oscillates infinitely often as we tend to 0.
There's no left or right limit in this case.
So there is a very large quantity of things like that.
Fortunately we're not gonna deal with them in this course.
A lot of times in real life there are things that oscillate as time goes to infinity, but we're not going to worry about that right now.
Okay, so that's our final mention of a discontinuity, and now I need to do just one more piece of groundwork for our formulas next time.
Namely, I want to check for you one basic fact, one limiting tool.
So this is going to be a theorem.
Fortunately it's a very short theorem and has a very short proof.
So the theorem goes under the name differentiable implies continuous.
And what it says is the following: it says that if f is differentiable, in other words its the derivative exists at x0, then f is continuous at x0.
So, we're gonna need this is as a tool, it's a key step in the product and quotient rules.
So I'd like to prove it right now for you.
So here is the proof.
Fortunately the proof is just one line.
So first of all, I want to write in just the right way what it is that we have to check.
So what we have to check is that the limit, as x goes to x0 of f(x) - f(x0) = 0.
So this is what we want to know.
We don't know it yet, but we're trying to check whether this is true or not.
So that's the same as the statement that the function is continuous because the limit of f(x) is supposed to be f(x0) and so this difference should have limit 0.
And now, the way this is proved is just by rewriting it by multiplying and dividing by (x - x0).
So I'll rewrite the limit as x goes to x0 of (f(x) - f(x0) / by x - x0) (x - x0).
Okay, so I wrote down the same expression that I had here.
This is just the same limit, but I multiplied and divided by (x - x0).
And now when I take the limit what happens is the limit of the first factor is f'(x0).
That's the thing we know exists by our assumption.
And the limit of the second factor is 0 because the limit as x goes to x0 of (x - x0) is clearly 0 .
So that's it.
The answer is 0, which is what we wanted.
So that's the proof.
Now there's something exceedingly fishy looking about this proof and let me just point to it before we proceed.
Namely, you're used in limits to setting x equal to 0.
And this looks like we're multiplying, dividing by 0, exactly the thing which makes all proofs wrong in all kinds of algebraic situations and so on and so forth.
You've been taught that that never works.
Right?
But somehow these limiting tricks have found a way around this and let me just make explicit what it is.
In this limit we never are using x = x0.
That's exactly the one value of x that we don't consider in this limit.
That's how limits are cooked up.
And that's sort of been the themes so far today, is that we don't have to consider that and so this multiplication and division by this number is legal.
It may be small, this number, but it's always non-zero.
So this really works, and it's really true, and we just checked that a differentiable function is continuous.
So I'm gonna have to carry out these limits, which are very interesting 0 / 0 limits next time.
But let's hang on for one second to see if there any questions before we stop.
Yeah, there is a question.
Student: [INAUDIBLE] Professor: Repeat this proof right here?
Just say again.
Student: [INAUDIBLE] Professor: Okay, so there are two steps to the proof and the step that you're asking about is the first step.
Right?
And what I'm saying is if you have a number, and you multiply it by 10 / 10 it's the same number.
If you multiply it by 3 / 3 it's the same number.
2 / 2, 1 / 1, and so on.
So it is okay to change this to this, it's exactly the same thing.
That's the first step.
Yes?
Student: [INAUDIBLE] Professor: Shhhh...
The question was how does the proof, how does this line, yeah where the question mark is.
So what I checked was that this number which is on the left hand side is equal to this very long complicated number which is equal to this number which is equal to this number.
And so I've checked that this number is equal to 0 because the last thing is 0.
This is equal to that is equal to that is equal to 0.
And that's the proof.
Yes?
Student: [INAUDIBLE] Professor: So that's a different question.
Okay, so the hypothesis of differentiability I use because this limit is equal to this number.
That that limit exits.
That's how I use the hypothesis of the theorem.
The conclusion of the theorem is the same as this because being continuous is the same as limit as x goes to x0 of f(x) = f(x0).
That's the definition of continuity.
And I subtracted f(x0) from both sides to get this as being the same thing.
So this claim is continuity and it's the same as this question here.
Last question.
Student: How did you get the 0 [INAUDIBLE] Professor: How did we get the 0 from this?
So the claim that is being made, so the claim is why is this tending to that.
So for example, I'm going to have to erase something to explain that.
So the claim is that the limit as x goes to x0 of x - x0 = 0.
That's what I'm claiming.
Okay, does that answer your question?
Okay.
All right.
Ask me other stuff after lecture.
The following is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Professor: In today's lecture I want to develop several more formulas that will allow us to reach our goal of differentiating everything.
So these are derivative formulas, and they come in two flavors.
The first kind is specific, so some specific function we're giving the derivative of.
And that would be, for example, x^n or (1/x) .
Those are the ones that we did a couple of lectures ago.
And then there are general formulas, and the general ones don't actually give you a formula for a specific function but tell you something like, if you take two functions and add them together, their derivative is the sum of the derivatives.
Or if you multiply by a constant, for example, so (cu), the derivative of that is (cu)' where c is constant.
All right, so these kinds of formulas are very useful, both the specific and the general kind.
For example, we need both kinds for polynomials.
And more generally, pretty much any set of forumulas that we give you, will give you a few functions to start out with and then you'll be able to generate lots more by these general formulas.
So today, we wanna concentrate on the trig functions, and so we'll start out with some specific formulas.
And they're going to be the formulas for the derivative of the sine function and the cosine function.
So that's what we'll spend the first part of the lecture on, and at the same time I hope to get you very used to dealing with trig functions, although that's something that you should think of as a gradual process.
Alright, so in order to calculate these, I'm gonna start over here and just start the calculation.
So here we go.
Let's check what happens with the sine function.
So, I take sin (x delta x), I subtract sin x and I divide by delta x.
Right, so this is the difference quotient and eventually I'm gonna have to take the limit as delta x goes to 0.
And there's really only one thing we can do with this to simplify or change it, and that is to use the sum formula for the sine function.
So, that's this.
That's sin x co delta x plus Oh, that's not what it is?
OK, so what is it?
Sin x sin delta x. OK, good.
Plus cosine.
No?
Oh, OK.
So which is it?
OK. Alright, let's take a vote.
Is it sine, sine, or is it sine, cosine?
Audience: [INAUDIBLE] Professor: OK, so is this going to be... cosine.
All right, you better remember these formulas, alright?
OK, turns out that it's sine, cosine.
All right.
Cosine, sine.
So here we go, no gotta do x here, sin (delta x).
Alright, so now there's lots of places to get confused here, and you're gonna need to make sure you get it right.
Alright, so we're gonna put those in parentheses here.
Sin (a + b) is sin a (cos b) cos a (sin b).
All right, now that's what I did over here, except the letter x was a, and the letter b was delta x.
Now that's just the first part.
That's just this part of the expression.
I still have to remember the - sin x.
That comes at the end.
Minus sin x.
And then, I have to remember the denominator, which is delta x. OK?
Alright, so now...
The next thing we're gonna do is we're gonna try to group the terms.
And the difficulty with all such arguments is the following one: any tricky limit is basically 0 / 0 when you set delta x equal to 0.
If I set delta x equal to 0, this is sin x - sin x.
So it's a 0 / 0 term.
Here we have various things which are 0 and various things which are non-zero.
We must group the terms so that a 0 stays over a 0.
Otherwise, we're gonna have no hope.
If we get some 1 / 0 term, we'll get something meaningless in the limit.
So I claim that the right thing to do here is to notice, and I'll just point out this one thing.
When delta x goes to 0, this cosine of 0 is 1.
So it doesn't cancel unless we throw in this extra sine term here.
So I'm going to use this common factor, and combine those terms.
So this is really the only thing you're gonna have to check in this particular calculation.
So we have the common factor of sin x, and that multiplies something that will cancel, which is (cos delta x - 1) / delta x.
That's the first term, and now what's left, well there's a cos x that factors out, and then the other factor is (sin delta x) / (delta x).
OK, now does anyone remember from last time what this thing goes to?
How many people say 1?
How many people say 0?
All right, it's 0.
That's my favorite number, alright?
0.
It's the easiest number to deal with.
So this goes 0, and that's what happens as delta x tends to 0.
How about this one?
This one goes to 1, my second favorite number, almost as easy to deal with as 0.
And these things are picked for a reason.
They're the simplest numbers to deal with.
So altogether, this thing as delta x goes to 0 goes to what?
I want a single person to answer, a brave volunteer.
Alright, back there.
Student: Cosine Professor: Cosine, because this factor is 0.
It cancels and this factor has a 1, so it's cosine.
So it's cos x.
So our conclusion over here - and I'll put it in orange - is that the derivative of the sine is the cosine.
OK, now I still wanna label these very important limit facts here.
This one we'll call A, and this one we're going to call B, because we haven't checked them yet.
I promised you I would do that, and I'll have to do that this time.
So we're relying on those things being true.
Now I'm gonna do the same thing with the cosine function, except in order to do it I'm gonna have to remember the sum rule for cosine.
So we're gonna do almost the same calculation here.
We're gonna see that that will work out, but now you have to remember that cos (a + b) = cos cos, no it's not cosine^2, because there are two different quantities here.
It's (cos a cos b) - (sin a sin b).
All right, so you'll have to be willing to call those forth at will right now.
So let's do the cosine now.
So that's cosine ((x + delta x) - cos x) / delta x. OK, there's the difference quotient for the cosine function.
And now I'm gonna do the same thing I did before except I'm going to apply the second rule, that is the sum rule for cosine.
And that's gonna give me (cos x cos delta x) - (sin x sin delta x).
And I have to remember again to subtract the cosine divided by this delta x.
And now I'm going to regroup just the way I did before, and I get the common factor of cosine multiplying ((cosine delta x - 1) / delta x).
And here I get the sin x but actually it's - sin x.
And then I have (sin delta x) / delta x.
All right?
The only difference is this minus sign which I stuck inside there.
Well that's not the only difference, but it's a crucial difference.
OK, again by A we get that this is 0 as delta x tends to 0.
And this is 1.
Those are the properties I called A and B.
And so the result here as delta x tends to 0 is that we get negative sine x.
That's the factor.
So this guy is negative sine x. I'll put a little box around that too.
Alright, now these formulas take a little bit of getting used to, but before I do that I'm gonna explain to you the proofs of A and B.
So we'll get ourselves started by mentioning that.
Maybe before I do that though, I want to show you how A and B fit into the proofs of these theorems.
So, let me just make some remarks here.
So this is just a remark but it's meant to help you to frame how these proofs worked.
So, first of all, I want to point out that if you take the rate of change of sin x, no let's start with cosine because a little bit less obvious.
If I take the rate of change of cos x, so in other words this derivative at x = 0, then by definition this is a certain limit as delta x goes to 0.
So which one is it?
Well I have to evaluate cosine at 0 delta x, but that's just delta x.
And I have to subtract cosine at 0.
That's the base point, but that's just 1.
And then I have to divide by delta x.
And lo and behold you can see that this is exactly the limit that we had over there.
This is the one that we know is 0 by what we call property A.
And similarly, if I take the derivative of (sin x) at x= 0, then that's going to be the limit as delta x goes to 0 of sine delta x / delta x.
And that's because I should be subtracting sine of 0 but sine of 0 is 0.
Right?
So this is going to be 1 by our property B.
And so the remark that I want to make, in addition to this, is something about the structure of these two proofs.
Which is the derivatives of sine and cosine at x = 0 give all values of d/dx sin x, d/dx cos x.
So that's really what this argument is showing us, is that we just need one rate of change at one place and then we work out all the rest of them.
So that's really the substance of this proof.
That of course really then shows that it boils down to showing what this rate of change is in these two cases.
So now there's enough suspense that we want to make sure that we know that those answers are correct.
OK, so let's demonstrate both of them.
I'll start with B. I need to figure out property B.
Now, we only have one alternative as to a type of proof that we can give of this kind of result, and that's because we only have one way of describing sine and cosine functions, that is geometrically.
So we have to give a geometric proof.
And to write down a geometric proof we are going to have to draw a picture.
And the first step in the proof, really, is to replace this variable delta x which is going to 0 with another name which is suggestive of what we're gonna do which is the letter theta for an angle.
OK, so let's draw a picture of what it is that we're going to do.
Here is the circle.
And here is the origin.
And here's some little angle, well I'll draw it a little larger so it's visible.
Here's theta, alright?
And this is the unit circle.
I won't write that down on here but that's the unit circle.
And now sin theta is this vertical distance here.
Maybe, I'll draw it in a different color so that we can see it all.
OK so here's this distance.
This distance is sin theta.
OK?
Now almost the only other thing we have to write down in this picture to have it work out is that we have to recognize that when theta is the angle, that's also the arc length of this piece of the circle when measured in radians.
So this length here is also arc length theta.
That little piece in there.
So maybe I'll use a different color for that to indicate it.
So that's orange and that's this little chunk there.
So those are the two pieces.
Now in order to persuade you now that the limit is what it's supposed to be, I'm going to extend the picture just a little bit.
I'm going to double it, just for my own linguistic sake and so that I can tell you a story.
Alright, so that you'll remember this.
So I'm going to take a theta angle below and I'll have another copy of sin theta down here.
And now the total picture is really like a bow and its bow string there.
Alright?
So what we have here is a length of 2 sin theta.
So maybe I'll write it this way, 2 sin theta.
I just doubled it.
And here I have underneath, whoops, I got it backwards.
Sorry about that.
Trying to be fancy with the colored chalk and I have it reversed here.
So this is not 2 sin theta.
2 sin theta is the vertical.
That's the green.
So let's try that again.
This is 2 sin theta, alright?
And then in the denominator I have the arc length which is theta is the first half and so double it is 2 theta.
Alright?
So if you like, this is the bow and up here we have the bow string.
And of course we can cancel the 2's.
That's equal to sin theta / theta.
And so now why does this tend to 1 as theta goes to 0?
Well, it's because as the angle theta gets very small, this curved piece looks more and more like a straight one.
Alright?
And if you get very, very close here the green segment and the orange segment would just merge.
They would be practically on top of each other.
And they have closer and closer and closer to the same length.
So that's why this is true.
I guess I'll articulate that by saying that short curves are nearly straight.
Alright, so that's the principle that we're using.
Or short pieces of curves, if you like, are nearly straight.
So if you like, this is the principle.
So short pieces of curves.
Alright?
So now I also need to give you a proof of A.
And that has to do with this cosine function here.
This is the property A.
So I'm going to do this by flipping it around, because it turns out that this numerator is a negative number.
If I want to interpret it as a length, I'm gonna want a positive quantity.
So I'm gonna write down (1 - cos theta) here and then I'm gonna divide by theta there.
Again I'm gonna make some kind of interpretation.
Now this time I'm going to draw the same sort of bow and arrow arrangement, but maybe I'll exaggerate it a little bit.
So here's the vertex of the sector, but we'll maybe make it a little longer.
Alright, so here it is, and here was that middle line which was the unit... Whoops.
OK, I think I'm going to have to tilt it up.
OK, let's try from here.
Alright, well you know on your pencil and paper it will look better than it does on my blackboard.
OK, so here we are.
Here's this shape.
Now this angle is supposed to be theta and this angle is another theta.
So here we have a length which is again theta and another length which is theta over here.
That's the same as in the other picture, except we've exaggerated a bit here.
And now we have this vertical line, which again I'm gonna draw in green, the bow string.
But notice that as the vertex gets farther and farther away, the curved line gets closer and closer to being a vertical line.
That's sort of the flip side, by expansion, of the zoom in principle.
The principle that curves are nearly straight when you zoom in.
If you zoom out that would mean sending this vertex way, way out somewhere.
The curved line, the piece of the circle, gets more and more straight.
And now let me show you where this numerator (1 - cos theta) is on this picture.
So where is it?
Well, this whole distance is 1.
But the distance from the vertex to the green is cosine of theta.
Right, because this is theta, so dropping down the perpendicular this distance back to the origin is cos theta.
So this little tiny, bitty segment here is basically the gap between the curve and the vertical segment.
So the gap = 1 - cos theta.
So now you can see that as this point gets farther away, if this got sent off to the Stata Center, you would hardly be able to tell the difference.
The bow string would coincide with the bow and this little gap between the bow string and the bow would be tending to 0.
And that's the statement that this tends to 0 as theta tends to 0.
The scaled version of that.
Yeah, question down here.
Student: Doesn't the denominator also tend to 0 though?
Professor: Ah, the question is "doesn't the denominator also tend to 0?"
And the answer is yes.
In my strange analogy with zooming in, what I did was I zoomed out the picture.
So in other words, if you imagine you're taking this and you're putting it under a microscope over here and you're looking at something where theta is getting smaller and smaller and smaller and smaller.
Alright?
But now because I want my picture, I expanded my picture.
So the ratio is the thing that's preserved.
So if I make it so that this gap is tiny... Let me say this one more time.
I'm afraid I've made life complicated for myself.
If I simply let this theta tend in to 0, that would be the same effect as making this closer and closer in and then the vertical would approach.
But I want to keep on blowing up the picture so that I can see the difference between the vertical and the curve.
So that's very much like if you are on a video screen and you zoom in, zoom in, zoom in, and zoom in.
So the question is what would that look like?
That has the same effect as sending this point out farther and farther in that direction, to the left.
And so I'm just trying to visualize it for you by leaving the theta at this scale, but actually the scale of the picture is then changing when I do that.
So theta is going to 0, but I I'm rescaling so that it's of a size that we can look at it, And then imagine what's happening to it.
OK, does that answer your question?
Student: My question then is that seems to prove that that limit is equal to 0/0.
Professor: It proves more than it is equal to 0 / 0.
It's the ratio of this little short thing to this longer thing.
And this is getting much, much shorter than this total length.
You're absolutely right that we're comparing two quantities which are going to 0, but one of them is much smaller than the other.
In the other case we compared two quantities which were both going to 0 and they both end up being about equal in length.
Here the previous one was this green one.
Here it's this little tiny bit here and it's way shorter than the 2 theta distance.
Yeah, another question.
Student: (Cos theta -1) / (cos theta) is the same as (1- cos theta) / theta?
Professor: Cos theta - 1 over... Student: [INAUDIBLE] Professor: So here, what I wrote is (cos delta x - 1) / delta x, OK, and I claimed that it goes to 0.
Here, I wrote minus that, that is I replaced delta x by theta.
But then I wrote this thing.
So (cos theta - 1) - 1 is the negative of this.
Alright?
And if I show that this goes to 0, it's the same as showing the other one goes to 0.
Another question?
Student: [INAUDIBLE] Professor: So the question is, what about this business about arc length.
So the word arc length, that orange shape is an arc.
And we're just talking about the length of that arc, and so we're calling it arc length.
That's what the word are length means, it just means the length of the arc.
Student: [INAUDIBLE] Professor: Why is this length theta?
Ah, ok so this is a very important point, and in fact it's the very next point that I wanted to make.
Namely, notice that in this calculation it was very important that we used length.
And that means that the way that we're measuring theta, is in what are known as radians.
Right, so that applies to both B and A, it's a scale change in A and doesn't really matter but in B it's very important.
The only way that this orange length is comparable to this green length, the vertical is comparable to the arc, is if we measure them in terms of the same notion of length.
If we measure them in degrees, for example, it would be completely wrong.
We divide up the angles into 360 , and that's wrong unit of measure.
The correct measures is the length along the unit circle, which is what radians are.
And so this is only true if we use radians.
So again, a little warning here, that this is in radians.
Now here x is in radians.
The formulas are just wrong if you use other units.
Ah yeah?
Student: [INAUDIBLE].
Professor: OK so the second question is why is this crazy length here 1.
And the reason is that the relationship between this picture up here and this picture down here, is that I'm drawing a different shape.
Namely, what I'm really imagining here is a much, much smaller theta.
OK?
And then I'm blowing that up in scale.
So this scale of this picture down here is very different from the scale of the picture up there.
And if the angle is very, very, very small then one has to be very, very long in order for me to finish the circle.
So, in other words, this length is 1 because that's what I'm insisting on.
So, I'm claiming that that's how I define this circle, to be of unit radius.
Another question?
Student: [INAUDIBLE] the ratio between 1 - theta and theta and theta will get closer and closer to 1.
I don't understand [INAUDIBLE].
Professor: OK, so the question is it's hard to visualize this fact here.
So let me let me take you through a couple of steps, because I think probably other people are also having trouble with this visualization.
The first part of the visualization I'm gonna try to demonstrate on this picture up here.
The first part of the visualization is that I should think of a beak of a bird closing down, getting narrower and narrower.
So in other words, the angle theta has to be getting smaller and smaller and smaller.
OK, that's the first step.
So that's the process that we're talking about.
Now, in order to draw that, once theta gets incredibly narrow, in order to depict that I have to blow the whole picture back up in order be able to see it.
Otherwise it just disappears on me.
In fact in the limit theta = 0, it's meaningless.
It's just a flat line.
That's the whole problem with these tricky limits.
They're meaningless right at the (0, 0) level.
It's only just a little away that they're actually useful, that you get useful geometric information out of them.
So we're just a little away.
So that's what this picture down below in part A is meant to be.
It's supposed to be that theta is open a tiny crack, just a little bit.
And the smallest I can draw it on the board for you to visualize it is using the whole length of the blackboard here for that.
So I've opened a little tiny bit and by the time we get to the other end of the blackboard, of course it's fairly wide.
But this angle theta is a very small angle.
Alright?
So I'm trying to imagine what happens as this collapses.
Now, when I imagine that I have to imagine a geometric interpretation of both the numerator and the denominator of this quantity here.
And just see what happens.
Now I claimed the numerator is this little tiny bit over here and the denominator is actually half of this whole length here.
But the factor of 2 doesn't matter when you're seeing whether something tends to 0 or not.
Alright?
And I claimed that if you stare at this, it's clear that this is much shorter than that vertical curve there.
And I'm claiming, so this is what you have to imagine, is this as it gets smaller and smaller and smaller still that has the same effect of this thing going way, way way, farther away and this vertical curve getting closer and closer and closer to the green.
And so that the gap between them gets tiny and goes to 0.
Alright?
So not only does it go to 0, that's not enough for us, but it also goes to 0 faster than this theta goes to 0.
And I hope the evidence is pretty strong here because it's so tiny already at this stage.
Alright.
We are going to move forward and you'll have to ponder these things some other time.
So I'm gonna give you an even harder thing to visualize now so be prepared.
OK, so now, the next thing that i'd like to do is to give you a second proof.
Because it really is important I think to understand this particular fact more thoroughly and also to get a lot of practice with sines and cosines.
So I'm gonna give you a geometric proof of the formula for sine here, for the derivative of sine.
So here we go.
This is a geometric proof of this fact.
This is for all theta.
So far we only did it for theta = 0 and now we're going to do it for all theta.
So this is a different proof, but it uses exactly the same principles.
Right?
So, I want do this by drawing another picture, and the picture is going to describe Y, which is sin theta, which is if you like the vertical position of some circular motion.
So I'm imagining that something is going around in a circle.
Some particle is going around in a circle.
And so here's the circle, here the origin.
This is the unit distance.
And right now it happens to be at this location P. Maybe we'll put P a little over here.
And here's the angle theta.
And now we're going to move it.
We're going to vary theta and we're interested in the rate of change of Y.
So Y is the height P he but we're gonna move it to another location.
We'll move it along the circle to Q.
Right?
So here it is.
Here's the thing.
So how far did we move it?
Well we moved it by an angle delta theta.
So we started theta, theta is going to be fixed in this argument, and we're going to move a little bit delta theta.
And now we're just gonna try to figure out how far the thing moved.
Well, in order to do that we've got to keep track of the the height, the vertical displacement here.
So we're going to draw this right angle here, this is the position R. And then this distance here is the change in Y. Alright?
So the picture is we have something moving around a unit circle.
A point moving around a unit circle.
It starts at P it moves to Q.
It moves from angle theta to angle theta delta theta.
And the issue is how much does Y move?
And the formula for Y is sin theta.
So that's telling us the rate of change of sin theta.
Alright, well so let's just try to think a little bit about what this is.
So, first of all, I've already said this and I'm going to repeat it here.
Delta Y is PR.
It's going from P and going straight up to R. That's how far Y moves.
That's the change in Y.
That's what I said up in the right hand corner there.
Oops.
I said PR but I wrote PQ.
Alright, that's not a good idea.
Alright.
So delta Y is PR.
And now I want to draw the diagram again one time.
So here's Q, here's R, and here's P, and here's my triangle.
And now what i'd like to do is draw this curve here which is a piece of the arc of the circle.
But really what I want to keep in mind is something that I did also in all these other arguments.
Which is, maybe I should have called this orange, that I'm gonna think of the straight line between.
So it's the straight line approximation to the curve that we're always interested in.
So the straight line is much simpler, because then we just have a triangle here.
And in fact it's a right triangle.
Right, so we have the geometry of a right triangle which is going to now let us do all of our calculations.
OK, so now the key step is this same principle that we already used which is that short pieces of curves are nearly straight.
So that means that this piece of the circular arc here from P to Q is practically the same as the straight segment from P to Q.
So, that's this principal.
Well, let's put it over here.
Is that PQ is practically the same as the straight segment from P to Q.
So how are we going to use that?
We want to use that quantitatively in the following way.
What we want to notice is that the distance from P to Q is approximately delta theta.
Right?
Because the arc length along that curve, the length of the curve is delta theta.
So the length of the green which is PQ is almost delta theta.
So this is essentially delta theta, this distance here.
Now the second step, which is a little trickier, is that we have to work out what this angle is.
So our goal, and I'm gonna put it one step below because I'm gonna put the geometric reasoning in between, is I need to figure out what the angle QPR is.
If I can figure out what this angle is, then I'll be able to figure out what this vertical distance is because I'll know the hypotenuse and I'll know the angle so I'll be able to figure out what the side of the triangle is.
So now let me show you why that's possible to do.
So in order to do that first of all I'm gonna trade the boards and show you where the line PQ is.
So the line PQ is here.
That's the whole thing.
And the key point about this line that I need you to realize is that it's practically perpendicular, it's almost perpendicular, to this ray here.
Alright?
It's not quite because the distance between P to Q is non-zero.
So it isn't quite, but in the limit it's going to be perpendicular.
Exactly perpendicular.
The tangent line to the circle.
So the key thing that I'm going to use is that PQ is almost perpendicular to OP.
Alright?
The ray from the origin is basically perpendicular to that green line.
And then the second thing I'm going to use is something that's obvious which is that PR is vertical.
OK?
So those are the two pieces of geometry that I need to see.
And now notice what's happening upstairs on the picture here in the upper right.
What I have is the angle theta is the angle between the horizontal and OP.
That's angle theta.
If I rotate it by ninety degree, the horizontal becomes vertical.
It becomes PR and the other thing rotated by 90 degrees becomes the green line.
So the angle that I'm talking about I get by taking this guy and rotating it by 90 degrees.
It's the same angle.
So that means that this angle here is essentially theta.
That's what this angle is.
Let me repeat that one more time.
We started out with an angle that looks like this, which is the horizontal that's the origin straight out horizontally.
That's the thing labeled 1.
That distance there.
That's my right arm which is down here.
My left arm is pointing up and it's going from the origin to the point P. So here's the horizontal and the angle between them is theta.
And now, what I claim is is that if I rotate by 90 degrees up, like this, without changing anything - so that was what I did - the horizontal will become a vertical.
That's PR.
That's going up, PR.
And if I rotate OP 90 degrees, that's exactly PQ.
OK?
So let me draw it on there one time.
Let's do it with some arrows here.
So I started out with this and then, we'll label this as orange, OK so red to orange, and then I rotate by 90 degrees and the red becomes this starting from P and the orange rotates around 90 degrees and becomes this thing here.
Alright?
So this angle here is the same as the other one which I've just drawn.
Different vertices for the angles.
OK?
Well I didn't say that all arguments were supposed to be easy.
Alright, so I claim that the conclusion is that this angle is approximately theta.
And now we can finish our calculation, because we have something with the hypotenuse being delta theta and the angle being theta and so this segment here PR is approximately the hypotenuse length times the cosine of the angle.
And that is exactly what we wanted.
If we divide, we divide by delta theta, we get (delta Y) / (delta theta) is approximately cos theta.
And that's the same thing as...
So what we want in the limit is exactly the delta theta going to 0 of (delta y) / (delta theta) = cos theta.
So we get an approximation on a scale that we can visualize and in the limit the formula is exact.
OK, so that is a geometric argument for the same result.
Namely that the derivative of sine is cosine.
Yeah?
Student: [INAUDIBLE].
Professor: You will have to do some kind of geometric proofs sometimes.
When you'll really need this is probably in 18.02.
So you'll need to make reasoning like this.
This is, for example, the way that you actually develop the theory of arc length.
Dealing with delta x's and delta y's is a common tool.
Alright, I have one more thing that I want to talk about today, which is some general rules.
We took a little bit more time than I expected with this.
So what I'm gonna do is just tell you the rules and we'll discuss them in a few days.
So let me tell you the general rules.
So these were the specific ones and here are some general ones.
So the first one is called the product rule.
And what it says is that if you take the product of two functions and differentiate them, you get the derivative of one times the other plus the other times the derivative of the one.
Now the way that you should remember this, and the way that I'll carry out the proof, is that you should think of it is you change one at a time.
And this is a very useful way of thinking about differentiation when you have things which depend on more than one function.
So this is a general procedure.
The second formula that I wanted to mention is called the quotient rule and that says the following.
That u / v' has a formula as well.
And the formula is ((u'v - uv' ) / v^2).
So this is our second formula.
Let me just mention, both of them are extremely valuable and you'll use them all the time.
This one of course only works when v is not 0.
Alright, so because we're of time we're not gonna prove these today but we'll prove these next time and you're definitely going to be responsible for these kinds of proofs.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu.
Professor: I am Haynes Miller, I am substituting for David Jerison today.
So you have a substitute teacher today.
So I haven't been here in this class with you so I'm not completely sure where you are.
I think just been talking about differentiation and you've got some examples of differentiation like these basic examples: the derivative of x^n is nx^(x-1).
But I think maybe you've spent some time computing the derivative of the sine function as well, recently.
And I think you have some rules for extending these calculations as well.
For instance, I think you know that if you differentiate a constant times a function, what do you get?
Student: [INAUDIBLE].
Professor: The constant comes outside like this.
Or I could write (cu)' = cu'.
That's this rule, multiplying by a constant, and I think you also know about differentiating a sum.
Or I could write this as (u v)' = u' + v'.
So I'm going to be using those but today I'll talk about a collection of other rules about how to deal with a product of functions, a quotient of functions, and, best of all, composition of functions.
And then at the end, I'll have something to say about higher derivatives.
So that's the story for today.
That's the program.
So let's begin by talking about the product rule.
So the product rule tells you how to differentiate a product of functions, and I'll just give you the rule, first of all.
The rule is it's u'v + uv'.
It's a little bit funny.
Differentiating a product gives you a sum.
But let's see how that works out in a particular example.
For example, suppose that I wanted to differentiate the product.
Well, the product of these two basic examples that we just talked about.
I'm going to use the same variable in both cases instead of different ones like I did here.
So the derivative of (x^n)sin x.
So this is a new thing.
We couldn't do this without using the product rule.
So the first function is x^n and the second one is sin x.
And we're going to apply this rule.
So u is x^n.
u' is, according to the rule, nx^(n - 1).
And then I take v and write it down the way it is, sine of x.
And then I do it the other way.
I take u the way it is, that's x^n, and multiply it by the derivative of v, v'.
We just saw v' is cosine of x.
So that's it.
Obviously, you can differentiate longer products, products of more things by doing it one at a time.
Let's see why this is true.
I want to try to show you why the product rule holds.
So you have a standard way of trying to understand this, and it involves looking at the change in the function that you're interested in differentiating.
So I should look at how much the product uv changes when x changes a little bit.
Well, so how do compute the change?
Well, I write down the value of the function at some new value of x, (x delta x).
Well, I better write down the whole new value of the function, and the function is uv.
So the whole new value looks like this.
It's u (x delta x)v(x + delta x).
That's the new value.
But what's the change in the product?
Well, I better subtract off what the old value was, which is u(x) v(x).
Okay, according to the rule we're trying to prove, I have to get u' involved.
So I want to involve the change in u alone, by itself.
Let's just try that.
I see part of the formula for the change in u right there.
Let's see if we can get the rest of it in place.
So the change in x = ((u(x) + delta x) - u(x)).
That's the change in x [Correction:___c hange___in___u].
This part of it occurs up here, multiplied by v (x delta x), so let's put that in too.
Now this equality sign isn't very good right now.
I've got this product here so far, but I've introduced something I don't like.
I've introduced u (v ( x delta x)), right?
Minus that.
So the next thing I'm gonna do is correct that little defect by adding in u(x) (v (x + delta x)).
Okay, now I cancelled off what was wrong with this line.
But I'm still not quite there, because I haven't put this in yet.
So I better subtract off uv, and then I'll be home.
But I'm going to do that in a clever way, because I noticed that I already have a u here.
So I'm gonna take this factor of u and make it the same as this factor.
So I get u(x) times this, minus u(x) times that.
That's the same thing as u times the difference.
So that was a little bit strange, but when you stand back and look at it, you can see multiplied out, the middle terms cancel.
And you get the right answer.
Well I like that because it's involved the change in u and the change in v. So this is equal to (delta u) v (x delta x) - u(x) times the change in v. Well, I'm almost there.
The next step in computing the derivative is take difference quotient, divide this by delta x.
So, (delta (uv)) / (delta x) is well, I'll say (delta u / delta x) v ( x delta x).
Have I made a mistake here?
This plus magically became a minus on the way down here, so I better fix that.
Plus u(delta v / (delta x).
This u is this u over here.
So I've just divided this formula by delta x, and now I can take the limit as goes to 0, so this is as delta x goes to 0.
This becomes the definition of the derivative, and on this side, I get du/dx times... now what happens to this quantity when delta x goes to 0?
So this quantity is getting closer and closer to x.
So what happens to the value of v?
It becomes equal to x(v).
That uses continuity of v. So, v (x delta x) goes to v(x) by continuity.
So this gives me times v, and then I have u(n delta v/ v delta x) gives me dv/dx.
And that's the formula.
That's the formula as I wrote it down at the beginning over here.
The derivative of a product is given by this sum.
Yeah?
Student: How did you get from the first line to the second of the long equation?
Professor: From here to here?
Student: Yes.
Professor: So maybe it's easiest to work backwards and verify that what I wrote down is correct here.
So, if you look there's a u(v (x delta x)) there.
And there's also one here.
And they occur with opposite signs.
So they cancel.
What's left is u ( x delta x) v (x delta x) - (uv).
And that's just what I started with.
Student: They cancel right?
Professor: I cancelled out this term and this term, and what's left is the ends.
Any other questions?
Student: [INAUDIBLE].
Professor: Well, I just calculated what delta uv is, and now I'm gonna divide that by delta x on my way to computing the derivative.
And so I copied down the right hand side and divided delta x. I just decided to divide the delta u by delta x and delta v by delta x.
Good.
Anything else?
So we have the product rule here.
The rule for differentiating a product of two functions.
This is making us stronger.
There are many more functions you can find derivatives of now.
How about quotients?
Let's find out how to differentiate a quotient of two functions.
Well again, I'll write down what the answer is and then we'll try to verify it.
So there's a quotient.
Let me write this down.
There's a quotient of two functions.
And here's the rule for it.
I always have to think about this and hope that I get it right.
u'v - uv' / v^2.
This may be the craziest rule you'll see in this course, but there it is.
And I'll try to show you why that's true and see an example.
Yeah there was a hand?
Student: [INAUDIBLE] Professor: What letters look the same?
u and v look the same?
I'll try to make them look more different.
The v's have points on the bottom.
u's have little round things on the bottom.
What's the new value of u?
The value of u at (x delta x) is (u delta u), right?
That's what delta u is.
It's the change in u when x gets replaced by (x delta x).
And the change in v, the new value v, is (v delta v).
So this is the new value of u divided by the new value of v. That's the beginning.
And then I subtract off the old values, which are - u/v.
This'll be easier to work out when I write it out this way.
So now, we'll cross multiply, as I said.
So I get (uv (delta u)v) minus, now I cross multiply this way, you get (uv - u(delta v)).
And I divide all this by (v delta v)u [Correction:___( v___+___delta___v)v].
Okay, now the reason I like to do it this way is that you see the cancellation happening here.
uv and uv occur twice and so I can cancel them.
And I will, and I'll answer these questions in a minute.
Audience: [INAUDIBLE].
Professor: Ooo, that's a v. All right.
Good, anything else?
That's what all hands were.
Good.
All right, so I cancel these and what I'm left with then is (delta u)v - u(delta v) and all this is over (v delta v)v. Ok, there's the difference.
There's the change in the quotient.
The change in this function is given by this formula.
And now to compute the derivative, I want to divide by delta x, and take the limit.
So let's write that down, delta(u/v)/delta x is this formula here divided by delta x.
And again, I'm going to put the delta x under these delta u and delta v. Okay?
I'm gonna put delta x in the denominator, but I can think of that as dividing into this factor and this factor.
So this is ((delta u/ delta x)v) - u(delta v/delta x)).
And all that is divided by the same denominator, (v delta v)v. Right?
Put the delta x up in the numerator there.
Next up, take the limit as delta x goes to 0.
I get, by definition, the derivative of (u/v).
And on the right hand side, well, this is the derivative u(du/ dx) right?
Times v. See and then u times, and here it's the derivative (dv/ dx).
Now what about the denominator?
So when delta x goes to 0, v stays the same, v stays the same.
What happens to this delta v?
It goes to 0, again, because v is continuous.
So again, delta v goes to 0 with delta x because they're continuous and you just get (v*v).
I think that's the formula I wrote down over there.
(du/dx)v - u(dv/dx).
And all divided by the square of the old denominator.
Well, that's it.
That's the quotient rule.
Weird formula.
Let's see an application.
Let's see an example.
So the example I'm going to give is pretty simple.
I'm going to take the numerator to be just 1.
So I'm gonna take u = 1.
So now I'm differentiating 1 / v, the reciprocal of a function; 1 over a function.
Here's a copy of my rule.
What's du/ dx in that case?
u is a constant, so that term is 0 in this rule.
I don't have to worry about this.
I get a minus.
And then u = 1, and dv/ dx.
Well, v is whatever v is.
I'll write dv/dx as v'.
And then I get a v^2 in the denominator.
So that's the rule.
I could write it as (v^-2)v'.
(-v'/v^2).
That's the derivative of 1 / v. How about sub-example of that?
I'm going to take the special case where u = 1 again.
And v = x^n.
And I'm gonna use the rule that we developed earlier about the derivative of x^n.
So what do I get here?
d / dx (1/x^n) is, I'm plugging into this formula here with v = x^n.
So I get minus, uh, v^-2.
If v = x^n, v^-2 is, by the rule of exponents, x^-2n.
And then v' is the derivative of x^n, which is (nx)^(n-1).
Ok, so let's put these together.
There's several powers of x here.
I can put them together.
I get -nx ^ ((- 2n) (n - 1)).
One of these n's cancels.
And what I'm left with is ((-n) - 1).
So we've computed the derivative of 1 / x^n, which I could also write as x^-n, right?
So I've computed the derivative of negative powers of x.
And this is the formula that I get.
If you think of this - n as a unit, as a thing to itself, it occurs here in the exponent.
It occurs here, and it occurs here.
So how does that compare with the formula that we had up here?
The derivative of a power of x is that power times x to one less than that power.
That's exactly the same as the rule that I wrote down here.
But the power here happens to be a negative number, and the same negative number shows up as a coefficient and there in the exponent.
Yeah?
Student: [INAUDIBLE].
Professor: How did I do this?
Student: [INAUDIBLE].
Professor: Where did that x^-2n come from?
So I'm applying this rule.
So the denominator in the quotient rule is v^2.
And v was x^n, so the denominator is x^2n.
And I decided to write it as x^-2n.
So the green comments there... What they say is that I can enlarge this rule.
This exact same rule is true for negative values of n, as well as positive values of n. So there's something new in your list of rules that you can apply, of values of the derivative.
That standard rule is true for negative as well as positive exponents.
And that comes out of a quotient rule.
Okay, so we've done two rules.
I've talked about the product rule and the quotient rule.
What's next?
Let's see the chain rule.
So this is a composition rule.
So the kind of thing that I have in mind, composition of functions is about substitution.
So the kind of function that I have in mind is, for instance, y = (sin t)^10.
That's a new one.
We haven't seen how to differentiate that before, I think.
This kind of power of a trig function happens very often.
You've seen them happen, as well, I'm sure, already.
And there's a little notational switch that people use.
They'll write sin^10(t).
But remember that when you write sin^10(t), what you mean is take the sine of t, and then take the tenth power of that.
It's the meaning of sin^10(t).
So the method of dealing with this kind of composition of functions is to use new variable names.
What I mean is, I can think of this (sin t)^10.
I can think of it it as a two step process.
First of all, I compute the sine of t. And let's call the result x.
There's the new variable name.
And then, I express y in terms of x.
So y says take this and raise it to the tenth power.
In other words, y = x^10.
And then you plug x = sine of t into that, and you get the formula for what y is in terms of t. So it's good practice to introduce new letters when they're convenient, and this is one place where it's very convenient.
So let's find a rule for differentiating a composition, a function that can be expressed by doing one function and then applying another function.
And here's the rule.
Well, maybe I'll actually derive this rule first, and then you'll see what it is.
In fact, the rule is very simple to derive.
So this is a proof first, and then we'll write down the rule.
I'm interested in delta y / delta t. y is a function of x. x is a function of t. And I'm interested in how y changes with respect to t, with respect to the original variable t. Well, because of that intermediate variable, I can write this as ((delta y / delta x) (delta x/ delta t)).
It cancels, right?
The delta x cancels.
The change in that immediate variable cancels out.
This is just basic algebra.
But what happens when I let delta t get small?
Well this give me dy /dt.
On the right hand side, I get (dy/dx) (dx/dt).
So students will often remember this rule.
This is the rule, by saying that you can cancel out for the dx's.
And that's not so far from the truth.
That's a good way to think of it.
In other words, this is the so-called chain rule.
And it says that differentiation of a composition is a product.
It's just the product of the two derivatives.
So that's how you differentiate a composite of two functions.
And let's just do an example.
Let's do this example.
Let's see how that comes out.
So let's differentiate, what did I say?
(sin t)^10.
Okay, there's an inside function and an outside function.
The inside function is x as a function of t. This is the inside function, and this is the outside function.
So the rule says, first of all let's differentiate the outside function.
Take dy/ dx.
Differentiate it with respect to that variable x.
The outside function is the 10th power.
What's it's derivative?
So I get 10x^9.
In this account, I'm using this newly introduced variable named x.
So the derivative of the outside function is 10x^9.
And then here's the inside function, and the next thing I want to do is differentiate it.
So what's dx /dt, d/dt (sine t), the derivative of sine t?
All right, that's cosine t. That's what the chain gives you.
This is correct, but since we were the ones to introduce this notation x here, that wasn't given to us in the original problem here.
The last step in this process should be to put back, to substitute back in what x is in terms of t. So x = sin t. So that tells me that I get 10(sin(t))^9, that's x^9, times the cos(t).
Or the same thing is sin^9(t)cos(t).
So there's an application of the chain rule.
You know, people often wonder where the name chain rule comes from.
I was just wondering about that myself.
So is it because it chains you down?
Is it like a chain fence?
I decided what it is.
It's because by using it, you burst the chains of differentiation, and you can differentiate many more functions using it.
So when you want to think of the chain rule, just think of that chain there.
It lets you burst free.
Let me give you another application of the chain rule.
Ready for this one?
So I'd like to differentiate the sin(10t).
Again, this is the composite of two functions.
What's the inside function?
Okay, so I think I'll introduce this new notation.
x = 10t, and the outside function is the sine.
So y = sin x.
So now the chain rule says dy/ dt is...
Okay, let's see.
I take the derivative of the outside function, and what's that?
Sine' and we can substitute because we know what sine' is.
So I get cosine of whatever, x, and then times what?
Now I differentiate the inside function, which is just 10.
So I could write this as 10cos of what?
10t, x = 10t.
Now, once you get used to this, this middle variable, you don't have to give a name for it.
You can just to think about it in your mind without actually writing it down, d/dt (sin(10t)).
I'll just do it again without introducing this middle variable explicitly.
Think about it.
I first of all differentiate the outside function, and I get cosine.
But I don't change the thing that I'm plugging into it.
It's still x that I'm plugging into it.
x is 10t.
So let's just write 10t and not worry about the name of that extra variable.
If it confuses you, introduce the new variable.
And do it carefully and slowly like this.
But, quite quickly, I think you'll get to be able to keep that step in your mind.
I'm not quite done yet.
I haven't differentiated the inside function, the derivative of 10t = 10.
So you get, again, the same result.
A little short cut that you'll get used to.
Really and truly, once you have the chain rule, the world is yours to conquer.
It puts you in a very, very powerful position.
Okay, well let's see.
What have I covered today?
I've talked about product rule, quotient rule, composition.
I should tell you something about higher derivatives, as well.
So let's do that.
This is a simple story.
Higher is kind of a strange word.
It just means differentiate over and over again.
All right, so let's see.
If we have a function u or u(x), please allow me to just write it as briefly as u.
Well, this is a sort of notational thing.
I can differentiate it and get u'.
That's a new function.
Like if you started with the sine, that's gonna be the cosine.
A new function, so I can differentiate it again.
And the notation for the differentiating of it again, is u''.
So u'' is just u' differentiated again.
For example, if u = sin x, so u' = cos(x).
Has Professor Gerison talked about what the derivative of cosine is?
What is it?
Ha, ok so u'' = - sin x.
Let me go on.
What do you suppose u''' means?
I guess it's the derivative of u''.
It's called the third derivative.
And u'' is called the second derivative.
And it's (u')' differentiated again.
So to compute u''' in this example, what do I do?
I differentiate that again.
There's a constant term, - 1, constant factor.
That comes out.
The derivative of sine is what?
Okay, so u''' = - cos x.
Let's do it again.
Now after a while, you get tired of writing these things.
And so maybe I'll use the notation u^(4).
That's the fourth derivative.
That's u''''.
Or it's (u''')' the fourth derivative.
And what is that in this example?
Okay, the cosine has derivative -sin , like you told me.
And that -sin cancels with that sine, and all together, I get sin x.
That's pretty bizarre.
When I differentiate the function sine of x four times, I get back to the sine of x again.
That's the way it is.
Now this notation, prime prime prime prime, and things like that.
There are different variants of that notation.
For example, that's another notation.
Well, you've used the notation du/ dx before.
u' could also be denoted du/ dx.
I think we've already here, today, used this way of rewriting du/ dx.
I think when I was talking about d/dt(uv) and so on, I pulled that d / dt outside and put whatever function you're differentiating over to the right.
So that's just a notational switch.
It looks good.
It looks like good algebra doesn't it?
But what it's doing is regarding this notation as an operator.
It's something you apply to a function to get a new function.
I apply it to the sine function, and I get the cosine function.
I apply it to x^2, and I get 2x.
This thing here, that symbol, represents an operator, which you apply to a function.
And the operator says, take the function and differentiate it.
So further notation that people often use, is they give a different name to that operator.
And they'll write capital D for it.
So this is just using capital D for the symbol d/dx.
So in terms of that notation, let's see.
Let's write down what higher derivatives look like.
So let's see.
That's what u' is.
How about u''?
Let's write that in terms of the d/dx notation.
Well I'm supposed to differentiate u' right?
So that's d/dx applied to the function du/ dx.
Differentiate the derivative.
That's what I've done.
Or I could write that as d/dx applied to d/dx applied to u.
Just pulling that u outside.
So I'm doing d/dx twice.
I'm doing that operator twice.
I could write that as (d/dx)^2 applied to u. Differentiate twice, and do it to the function u.
Or, I can write it as, now this is a strange one.
I could also write as like that.
It's getting stranger and stranger, isn't it?
This is definitely just a kind of abuse of notation.
But people will go even further and write (d^2)u/dx^2.
So this is the strangest one.
This identity quality is the strangest one, because you may think that you're taking d of the quantity x^2.
But that's not what's intended.
This is not d(x^2).
What's intended is the quantity dx^2.
In this notation, which is very common, what's intended by the denominator is the quantity dx^2.
It's part of this second differentiation operator.
So I've written a bunch of equalities down here, and the only content to them is that these are all different notations for the same thing.
You'll see this notation very commonly.
So for instance the third derivative is (d^3)u/dx^3, and so on.
Sorry?
Student: [INAUDIBLE].
Professor: Yes, absolutely.
Or an equally good notation is to write the operator (D^3)u.
Absolutely.
So I guess I should also write over here when I was talking about d^2, the second derivative, another notation is do the operator capital D twice.
Let's see an example of how this can be applied.
I'll answer this question.
Student: [INAUDIBLE].
Professor: Yeah, so the question is whether the fourth derivative always gives you the original function back, like what happened here.
No.
That's very, very special to sines and cosines.
All right?
And, in fact, let's see an example of that.
I'll do a calculation.
Let's calculate the nth derivative of x^n.
Okay, n is a number, like 1, 2, 3, 4.
Here we go.
Let's do this.
So, let's do this bit by bit.
What's the first derivative of x^n?
So everybody knows this.
I'm just using a new notation, this capital D notation.
So it's n x ^ (n -1).
Now you know know, by the way, n could be a negative number for that, but for now, for this application, I wanna take n to be 1, 2, 3, and so on; one of those numbers.
Ok, we did one derivative.
Let's compute the second derivative of x ^ n. Well there's this n constant that comes out, and then the exponent comes down, and it gets reduced by 1.
All right?
Should I do one more?
D^3 (x^n) = n(n-1).
That's the constant from here.
Times that exponent, (n - 2), times 1 less, (n - 3) is the new exponent.
Well, I keep on going until I come to a new blackboard.
Now, I think I'm going to stop when I get to the n minus first derivative, so we can see what's likely to happen.
So when I took the third derivative, I had the n minus third power of x.
And when I took the second derivative, I had the second power of x.
So, I think what'll happen when I have the n minus first derivative is I'll have the first power of x left over.
The powers of x keep coming down.
And what I've done it n - 1 times, I get the first power.
And then I get a big constant out in front here times more and more and more of these smaller and smaller integers that come down.
What's the last integer that came down before I got x^1 here?
Well, let's see.
It's just 2, because this x^1 occurred as the derivative of x^2.
And the coefficient in front of that is 2.
So that's what you get.
The numbers n( n - 1)...2)x^1.
And now we can differentiate one more time and calculate what (D^n)(x^n) is.
So I get the same number, n(n-1)... and so on and so on, times 2.
And then I guess I'll say times 1.
Times, what's the derivative of x ^ 1?
1, so times 1.
Time 1, times 1.
Where this one means the constant function 1.
Does anyone know what this number is called?
That has a name.
It's called n factorial.
And it's written n!
And we just used an example of mathematical induction.
So the end result is (D^n) (x^n) = n!
constant.
Okay that's a neat fact.
Final question for the lecture is what's D^n 1 applied to x ^ n?
Ha.
Excellent.
It's the derivative of a constant.
So it's 0.
Okay.
Thank you.
The following content was created under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Professor: So, we're ready to begin the fifth lecture.
I'm glad to be back.
Thank you for entertaining my colleague, Haynes Miller.
So, today we're going to continue where he started, namely what he talked about was the chain rule, which is probably the most powerful technique for extending the kinds of functions that you can differentiate.
And we're going to use the chain rule in some rather clever algebraic ways today.
So the topic for today is what's known as implicit differentiation.
So implicit differentiation is a technique that allows you to differentiate a lot of functions you didn't even know how to find before.
And it's a technique - let's wait for a few people to sit down here.
Physics, huh?
Ok, more Physics.
Let's take a break.
You can get those after class.
All right, so we're talking about implicit differentiation, and I'm going to illustrate it by several examples.
So this is one of the most important and basic formulas that we've already covered part way.
Namely, the derivative of x to a power is a x ^ a - 1.
Now, what we've got so far is the exponents, 0 plus or minus 1 plus or minus 2, etc.
You did the positive integer powers in the first lecture, and then yesterday Professor Miller told you about the negative powers.
So what we're going to do right now, today, is we're going to consider the exponents which are rational numbers, ratios of integers.
So a = m / n. m and n are integers.
All right, so that's our goal for right now, and we're going to use this method of implicit differentiation.
In particular, it's important to realize that this covers the case m = 1.
And those are the nth roots.
So when we take the one over n power, we're going to cover that right now, along with many other examples.
so this is our first example.
So how do we get started?
Well we just write down a formula for the function.
The function is y = x^ m / n. That's what we're trying to deal with.
And now there's really only two steps.
The first step is to take this equation to the nth power, so write it y ^ n = x ^m.
Alright, so that's just the same equation re-written.
And now, what we're going to do is we're going to differentiate.
So we're going to apply d / dx to the equation.
Now why is that we can apply it to the second equation, not the first equation?
So maybe I should call these equation 1 and equation 2.
So, the point is, we can apply it to equation 2.
Now, the reason is that we don't know how to differentiate x^ m / n. That's something we just don't know yet.
But we do know how to differentiate integer powers.
Those are the things that we took care of before.
So now we're in shape to be able to do the differentiation.
So I'm going to write it out explicitly over here, without carrying it out just yet.
That's d / dx of y^ n = d / dx of x ^m.
And now you see this expression here requires us to do something we couldn't do before yesterday.
Namely, this y is a function of x.
So we have to apply the chain rule here.
So this is the same as - this is by the chain rule now - (d/dy y ^ n) dy / dx.
And then, on the right hand side, we can just carry it out.
We know the formula.
It's mx ^ m - 1.
Right, now this is our scheme.
And you'll see in a minute why we win with this.
So, first of all, there are two factors here.
One of them is unknown.
In fact, it's what we're looking for.
But the other one is going to be a known quantity, because we know how to differentiate y to the n with respect to y.
That's the same formula, although the letter has been changed.
And so this is the same as - I'll write it underneath here - (n y^ n - 1) dy / dx = m x^m - 1.
Okay, now comes, if you like, the non-calculus part of the problem.
Remember the non-calculus part of the problem is always the messier part of the problem.
So we want to figure out this formula.
This formula, the answer over here, which maybe I'll put in a box now, has this expressed much more simply, only in terms of x.
And what we have to do now is just solve for dy / dx using algebra, and then solve all the way in terms of x.
So, first of all, we solve for dy/ dx.
So I do that by dividing the factor on the left hand side.
So I get here m x ^m - 1 / n y ^ n - 1.
And now I'm going to plug in -- so I'll write this as m / n. This is x^ m - 1.
Now over here I'm going to put in for y, (x ^ m / n)( n - 1).
So now we're almost done, but unfortunately we have this mess of exponents that we have to work out.
I'm going to write it one more time.
So I already recognize the factor a out front.
That's not going to be a problem for me, and that's what I'm aiming for here.
But now I have to encode all of these powers, so let's just write it.
It's m - 1, and then it's minus the quantity (n - 1) t m / n. All right, so that's the laws exponents applied to this ratio here.
And then we'll do the arithmetic over here on the next board.
So we have here m - 1 - (n - 1) m / n = m - 1.
And if I multiply n by this, I get - m. And if the second factor is minus minus, that's a plus.
And that's + m /n.
Altogether the two m's cancel.
I have here - 1 + m / n. And lo and behold that's the same thing as a - 1, just what we wanted.
All right, so this equals a x^n - 1.
Okay, again just a bunch of arithmetic.
From this point forward, from this substitution on, it's just the arithmetic of exponents.
All right, so we've done our first example here.
I want to give you a couple more examples, so let's just continue.
The next example I'll keep relatively simple.
So we have example two, which is going to be the function x^2 + y^2 = 1.
Well, that's not really a function.
It's a way of defining y as a function of x implicitly.
There's the idea that I could solve for y if I wanted to.
And indeed let's do that.
So if you solve for y here, what happens is you get y^2 = 1 - x ^2, and y = plus or minus the square root of 1 - x ^2.
So this, if you like, is the implicit definition.
And here is the explicit function y, which is a function of x.
And now just for my own convenience, I'm just going to take the positive branch.
This is the function.
It's just really a circle in disguise.
And I'm just going to take the top part of the circle, so we'll take that top hump here.
All right, so that means I'm erasing this minus sign.
I'm just taking the positive branch, just for my convenience.
I could do it just as well with the negative branch.
Alright, so now I've taken the solution, and I can differentiate with this.
So rather than using the dy / dx notation over here, I'm going to switch notations over here, because it's less writing.
I'm going to write y ' and change notations.
Okay, so I want to take the derivative of this.
Well this is a somewhat complicated function here.
It's the square root of 1 - x^2, and the right way always to look at functions like this is to rewrite them using the fractional power notation.
That's the first step in computing a derivative of a square root.
And then the second step here is what?
Does somebody want to tell me?
Chain rule, right.
That's it.
So we have two things.
We start with one, and then we do something else to it.
So whenever we do two things to something, we need to apply the chain rule.
So 1 - x^2, square root.
All right, so how do we do that?
Well, the first factor I claim is the derivative of this thing.
So this is 1/2 ^ - 1/2.
So I'm doing this kind of by the advanced method now, because we've already graduated.
You already did the chain rule last time.
So what does this mean?
This is an abbreviation for the derivative with respect to blah of blah ^ 1/2, whatever it is.
All right, so that's the first factor that we're going to use.
Rather than actually write out a variable for it and pass through as I did previously with this y and x variable here, I'm just going to skip that step and let you imagine it as being a placeholder folder that variable here.
So this variable is now parenthesis.
And then I have to multiply that by the rate of change of what's inside with respect to x.
And that is going to be - 2x.
The derivative of 1 - x^2 = - 2x.
And now again, we couldn't have done this example two before example one, because we needed to know that the power rule worked not just for a integer but also for a = 1/2.
We're using the case a = 1/2 right here.
It's 1/2 times, and this - 1/2 here is a - 1.
So this is the case a = 1/2.
a - 1 happens to be -1/2.
Okay, so I'm putting all those things together.
And you know within a week you have to be doing this very automatically.
So we're going to do it at this speed now.
You want to do it even faster, ultimately.
Yes?
Student: [INAUDIBLE] Professor: The question is could I have done it implicitly without the square roots.
And the answer is yes.
That's what I'm about to do.
So this is an illustration of what's called the explicit solution.
So this guy is what's called explicit.
And I want to contrast it with the method that we're going to now use today.
So it involves a lot of complications.
It involves the chain rule.
And as we'll see it can get messier and messier.
And then there's the implicit method, which I claim is easier.
So let's see what happens if you do it implicitly The implicit method involves, instead of writing the function in this relatively complicated way, with the square root, it involves leaving it alone.
Don't do anything to it.
In this previous case, we were left with something which was complicated, say x ^ 1/3 or x ^ 1/2 or something complicated.
We had to simplify it.
We had an equation one, which was more complicated.
We simplified it then differentiated it.
And so that was a simpler case.
Well here, the simplest thing us to differentiate is the one we started with, because squares are practically the easiest thing after first powers, or maybe zeroeth powers to differentiate.
So we're leaving it alone.
This is the simplest possible form for it, and now we're going to differentiate.
So what happens?
So again what's the method?
Let me remind you.
You're applying d / dx to the equation.
So you have to differentiate the left side of the equation, and differentiate the right side of the equation.
So it's this, and what you get is 2 x + 2 y y ' = to what?
0.
The derivative of 1 = 0.
So this is the chain rule again.
I did it a different way.
I'm trying to get you used to many different notations at once.
Well really just two.
Just the prime notation and the dy / dx notation.
And this is what I get.
So now all I have to do is solve for y '.
So that y ', if I put the 2 x on the other side is - 2 x, and then divide by 2y, which is -x /y.
So let's compare our solutions, and I'll apologize, I'm going to have to erase something to do that.
So let's compare our two solutions.
I'm going to put this underneath and simplify.
So what was our solution over here?
It was 1/2 (1 - x ^2) ^ -1/2 ( -2 x).
That was what we got over here.
And that is the same thing, if I cancel the 2's, and I change it back to looking like a square root, that's the same thing as - x / square root of 1 - x ^2.
So this is the formula for the derivative when I do it the explicit way.
And I'll just compare them, these two expressions here.
And notice they are the same.
They're the same, because y = square root of 1 - x^2.
Yeah?
Question?
Student: [INAUDIBLE] Professor: The question is why did the implicit method not give the bottom half of the circle?
Very good question.
The answer to that is that it did.
I just didn't mention it.
Wait, I'll explain.
So suppose I stuck in a minus sign here.
I would have gotten this with the difference, so with an extra minus sign.
But then when I compared it to what was over there, I would have had to have another different minus sign over here.
So actually both places would get an extra minus sign.
And they would still coincide.
So actually the implicit method is a little better.
It doesn't even notice the difference between the branches.
It does the job on both the top and bottom half.
Another way of saying that is that you're calculating the slopes here.
So let's look at this picture.
Here's a slope.
Let's just take a look at a positive value of x and just check the sign to see what's happening.
If you take a positive value of x over here, x is positive.
This denominator is positive.
The slope is negative.
You can see that it's tilting down.
So it's ok. Now on the bottom side, it's going to be tilting up.
And similarly what's happening up here is that both x and y are positive, and this x and this y are positive.
And the slope is negative.
On the other hand, on the bottom side, x is still positive, but y is negative.
And it's tilting up because the denominator is negative.
The numerator is positive, and this minus sign has a positive slope.
So it matches perfectly in every category.
This complicated, however, and it's easier just to keep track of one branch at a time, even in advanced math.
Okay, so we only do it one branch at a time.
Other questions?
Okay, so now I want to give you a slightly more complicated example here.
And indeed some of the, so here's a little more complicated example.
It's not going to be the most complicated example, but you know it'll be a little tricky.
So this example, I'm going to give you a fourth order equation.
So y ^ 4 + x y ^2 - 2 = 0.
Now it just so happens that there's a trick to solving this equation, so actually you can do both the explicit method and the non-explicit method.
So the explicit method would say okay well, I want to solve for this.
So I'm going to use the quadratic formula, but on y ^2.
This is quadratic in y ^2, because there's a fourth power and a second power, and the first and third powers are missing.
So this is y ^2 = - x plus or minus the square root of x ^2 - 4 (-2 ) / 2.
And so this x is the b.
This -2 is the c, and a = 1 in the quadratic formula.
And so the formula for y is plus or minus the square root of - x plus or minus the square root x ^2 + 8 /2.
So now you can see this problem of branches, this happens actually in a lot of cases, coming up in an elaborate way.
You have two choices for the sign here.
You have two choices for the sign here.
Conceivably as many as four roots for this equation, because it's a fourth degree equation.
It's quite a mess.
You should have to check each branch separately.
And this really is that level of complexity, and in general it's very difficult to figure out the formulas for quartic equations.
But fortunately we're never going to use them.
That is, we're never going to need those formulas.
So the implicit method is far easier.
The implicit method just says okay I'll leave the equation in its simplest form.
And now differentiate.
So when I differentiate, I get 4 y ^3 y ' +... now here I have to apply the product rule.
So I differentiate the x and the y ^2 separately.
First I differentiate with respect to x, so I get y ^2.
Then I differentiate with respect to the other factor, the y ^2 factors.
And I get x (2 y y ').
And then the 0 gives me 0.
So - 0 = 0.
So there's the implicit differentiation step.
And now I just want to solve for y '.
So I'm going to factor out 4 y ^3 + 2 x y.
That's the factor on y '.
And I'm going to put the y ^2 on the other side.
- y ^2 over here.
And so the formula for y ' is - y ^2 / 4 y ^3 + 2 x y.
So that's the formula for the solution.
For the slope.
You have a question?
Student: [INAUDIBLE] Professor: So the question is for the y would we have to put in what solved for in the explicit equation.
And the answer is absolutely yes.
That's exactly the point.
So this is not a complete solution to a problem.
We started with an implicit equation.
We differentiated.
And we got in the end, also an implicit equation.
It doesn't tell us what y is as a function of x.
You have to go back to this formula to get the formula for x.
So for example, let me give you an example here.
So this hides a degree of complexity of the problem.
But it's a degree of complexity that we must live with.
So for example, at x = 1, you can see that y = 1 solves.
That happens to solve y ^ 4 + x y ^2 - 2= 0.
That's why I picked the 2 actually, so it would be 1 + 1 - 2 = 0.
I just wanted to have a convenient solution there to pull out of my hat at this point.
So I did that.
And so we now know that when x = 1, y = 1.
So at (1, 1) along the curve, the slope is equal to what?
Well, I have to plug in here, - 1 ^2 / 4 * 1^3 + 2 * 1 * 1.
That's just plugging in that formula over there, which turns out to be - 1/6.
So I can get it.
On the other hand, at say x = 2, we're stuck using this formula star here to find y.
Now, so let me just make two points about this, which are just philosophical points for you right now.
The first is, when I promised you at the beginning of this class that we were going to be able to differentiate any function you know, I meant it very literally.
What I meant is if you know the function, we'll be able give a formula for the derivative.
If you don't know how to find a function, you'll have a lot of trouble finding the derivative.
So we didn't make any promises that if you can't find the function you will be able to find the derivative by some magic.
That will never happen.
And however complex the function is, a root of a fourth degree polynomial can be pretty complicated function of the coefficients, we're stuck with this degree of complexity in the problem.
But the big advantage of his method, notice, is that although we've had to find star, we had to find this formula star, and there are many other ways of doing these things numerically, by the way, which we'll learn later, so there's a good method for doing it numerically.
Although we had to find star, we never had to differentiate it.
We had a fast way of getting the slope.
So we had to know what x and y were.
But y ' we got by an algebraic formula, in terms of the values here.
So this is very fast, forgetting the slope, once you know the point.
yes?
Student: What's in the parentheses?
Professor: Sorry, this is... Well let's see if I can manage this.
Is this the parentheses you're talking about?
Well, so maybe I should put commas around it.
But it was "s a y", comma comma, okay?
Well here was at x = 1.
I'm just throwing out a value.
Any other value.
Actually there is one value, my favorite value.
Well this is easy to evaluate right?
x = 0, I can do it there.
That's maybe the only one.
The others are a nuisance.
All right, other questions?
Now we have to do something more here.
So I claimed to you that we could differentiate all the functions we know.
But really we can learn a tremendous about functions which are really hard to get at.
So this implicit differentiation method has one very, very important application to finding inverse functions, or finding derivatives of inverse functions.
So let's talk about that next.
So first, maybe we'll just illustrate by an example.
If you have the function y = to square root x, for x positive, then of course this idea is that we should simplify this equation and we should square it so we get this somewhat simpler equation here.
And then we have a notation for this.
If we call f (x) = the square root of x, and g ( y) = x, this is the reversal of this.
Then the formula for g(y) is that it should be y ^2.
And in general, if we start with any old y = f(x), and we just write down, this is the defining relationship for a function g, the property that we're saying is that g ( f(x)) has got to bring us back to x.
And we write that in a couple of different ways.
We call g the inverse of f. And also we call f the inverse of g, although I'm going to be silent about which variable I want to use, because people mix them up a little bit, as we'll be doing when we draw some pictures of this.
So let's see.
Let's draw pictures of both f and f inverse on the same graph.
So first of all, I'm going to draw the graph of f(x) = square root of x.
That's some shape like this.
And now, in order to understand what g ( y) is, so let's do the analysis in general, but then we'll draw it in this particular case.
If you have g (y) = x, that's really just the same equation right?
This is the equation g (y) = x, that's y ^2 = x.
This is y = square root of x, those are the same equations, it's the same curve.
But suppose now that we wanted to write down what g( x) is.
In other words, we wanted to switch the variables, so draw them as I said on the same graph with the same x, and the same y axes.
Then that would be, in effect, trading the roles of x and y.
We have to rename every point on the graph which is of the ordered pair (x, y), and trade it for the opposite one.
And when you exchange x and y, so to do this, exchange x and y, and when you do that, graphically what that looks like is the following: suppose you have a place here, and this is the x and this is the y, then you want to trade them.
So you want the y here right?
And the x up there.
It's sort of the opposite place over there.
And that is the place which is directly opposite this point across the diagonal line x = y.
So you reflect across this or you flip across that.
You get this other shape that looks like that.
Maybe I'll draw it with a colored piece of chalk here.
So this guy here is y = f^(-1)(x).
And indeed, if you look at these graphs, this one is the square roots.
This one happens to be y = x^2.
If you take this one, and you turn it, you reverse the roles of the x axis and the y axis, and tilt it on its side.
So that's the picture of what an inverse function is, and now I want to show you that the method of implicit differentiation allows us to compute the derivatives of inverse functions.
So let me just say it in general, and then I'll carry it out in particular.
So implicit differentiation allows us to find the derivative of any inverse function, provided we know the derivative of the function.
So let's do that for what is an example, which is truly complicated and a little subtle here.
It has a very pretty answer.
So we'll carry out an example here, which is the function y = tan^(-1).
So again, for the inverse tangent all of the things that we're going to do are going to be based on simplifying this equation by taking the tangent of both sides.
So, us let me remind you by the way, the inverse tangent is what's also known as arctangent.
That's just another notation for the same thing.
And we're going to use to describe this function is the equation tan y = x.
That's what happens when you take the tangent of this function.
This is how we're going to figure out what the function looks like.
So first of all, I want to draw it, and then we'll do the computation.
So let's make the diagram first.
So I want to do something which is analogous to what I did over here with the square root function.
So first of all, I remind you that the tangent function is defined find two values here, which are pi / 2 and - pi/2.
And it starts out at minus infinity and curves up like this.
So that's the function tan x.
And so the one that we have to sketch is this one which we get by reflecting this across the axis.
Well not the axis, the diagonal.
This slope by the way, should be less - a little lower here so that we can have it going down and up.
So let me show you what it looks like.
On the front, it's going to look a lot like this one.
So this one had curved down, and so the reflection across the diagonal curved up.
Here this is curving up, so the reflection is going to curve down.
It's going to look like this.
Maybe I should, sorry, let's use a different color, because it's reversed from before.
I'll just call it green.
Now, the original curve in the first quadrant eventually had an asymptote which was straight up.
So this one is going to have an asymptote which is horizontal.
And that level is what?
What's the highest?
It is just pi / 2.
Now similarly, the other way, we're going to do this: and this bottom level is going to be - pi/2.
So there's the picture of this function.
It's defined for all x.
So this green guy is y = arctan x.
And it's defined all the way from minus infinity to infinity.
And to use a notation that we had from limit notation as x goes to infinity, let's say, arctan x = pi/2.
That's an example of one value that's of interest in addition to the finite values.
Okay, so now the first ingredient that we're going to need, is we're going to need the derivative of the tangent function.
So I'm going to recall for you, and maybe you haven't worked this out yet, but I hope that many of you have, that if you take the derivative with respect to y of tan y .
So this you do by the quotient rule.
So this is of the form u / v, right?
You use the quotient rule.
So I'm going to get this.
But what you get in the end is some marvelous simplification that comes out to cos ^2y.
1 / cos squared.
You can recognize the cosine squared from the fact that you should get v ^2 in the denominator, and somehow the numerators all cancel and simplifies to 1.
This is also known as sec^2y.
So that something that if you haven't done yet, you're going to have to do this as an exercise.
So we need that ingredient, and now we're just going to differentiate our equation.
And what do we get?
We get, again, (d /dy tan y ) dy / dx = 1.
Or, if you like, 1 / cos ^2 y times in the other notation, y' = 1.
So I've just used the formulas that I just wrote down there.
Now all I have to do is solve for y '.
It's cos ^2y.
Unfortunately, this is not the form that we ever want to leave these things in.
This is the same problem we had with that ugly square root expression, or with any of the others.
We want to rewrite in terms of x.
Our original question was what is d / dx of arctan x.
Now so far we have the following answer to that question: it's cos ^2 ( arctan x).
Now this is a correct answer, but way too complicated.
Now that doesn't mean that if you took a random collection of functions, you wouldn't end up with something this complicated.
But these particular functions, these beautiful circular functions involved with trigonometry all have very nice formulas associated with them.
And this simplifies tremendously.
So one of the skills that you need to develop when you're dealing with trig functions is to simplify this.
And so let's see now that expressions like this all simplify.
So here we go.
There's only one formula, one ingredient that we need to use to do this, and then we're going to draw a diagram.
So the ingredient again, is the original defining relationship that tan y = x.
So tan y = x can be encoded in a right triangle in the following way: here's the right triangle and tan y means that y should be represented as an angle.
And then, its tangent is the ratio of this vertical to this horizontal side.
So I'm just going to pick two values that work, namely x and 1.
Those are the simplest ones.
So I've encoded this equation in this picture.
And now all I have to do is figure out what the cos y is in this right trying here.
In order to do that, I need to figure out what the hypotenuse is, but that's just square root of 1 + x ^2.
And now I can read off what the cos y is.
So the cos y is one divided by the hypotenuse.
So it's 1 / square root, whoops, yeah, 1 + x^2.
And so cos ^2 is just 1 / 1 + x^2.
And so our answer over here, the preferred answer which is way simpler than what I wrote up there, is that d/dx of arctan x = 1 / 1 + x^2.
Maybe I'll stop here for one more question.
I have one more calculation which I can do even in less than a minute.
So we have a whole minute for questions.
Yeah?
Student: [INAUDIBLE] Professor: What happens to the inverse tangent?
The inverse tangent this... Ok this inverse tangent is the same as this y here.
Those are the same thing.
So what I did was I skipped this step here entirely.
I never wrote that down.
But the inverse tangent was that y.
The issue was what's a good formula for cos y in terms of x?
So I am evaluating that, but I'm doing it using the letter y.
So in other words, what happened to the inverse tangent is that I called it y, which is what it's been all along.
Okay, so now I'm going to do the case of the sine, the inverse sine.
And I'll show you how easy this is if I don't fuss with... because this one has an easy trig identity associated with it.
So if y = arcsin x, and sin y = x, and now watch how simple it is when I do the differentiation.
I just differentiate.
I get (cos y) y ' = 1.
And then, y ', so that implies that y ' = 1 / cos y, and now to rewrite that in terms of x, I have to just recognize that this is the same as this, which is the same as 1 / square root of 1 - x^2.
So all told, the derivative with respect to x of the arcsine function is 1 / square root of 1 - x^2.
So these implicit differentiations are very convenient.
However, I warn you that you do have to be careful about the range of applicability of these things.
You have to draw a picture like this one to make sure you know where this makes sense.
In other words, you have to pick a branch for the sine function to work that out, and there's something like that on your problem set.
And it's also discussed in your text.
So we'll stop here.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so let's begin Lecture Six.
We're talking today about exponentials and logarithms.
And these are the last functions that I need to introduce, the last standard functions that we need to connect with Calculus, that you've learned about.
And they're certainly as fundamental, if not more so, than trigonometric functions.
So first of all, we'll start out with a number, a, which is positive, which is usually called a base.
And then we have these properties that a to the power 0 is always 1.
That's how we get started.
And a^1 is a.
And of course a^2 , not surprisingly, is a times a, etc.
And the general rule is that a^(X1 X2) is a^X1 times a^X2 .
So this is the basic rule of exponents, and with these two initial properties, that defines the exponential function.
And then there's an additional property, which is deduced from these, which is the composition of exponential functions, which is that you take a to the X1 power, to the X2 power.
Then that turns out to be a to the X1 times X2.
So that's an additional property that we'll take for granted, which you learned in high school.
Now, in order to understand what all the values of a^x are, we need to first remember that if you're taking a rational power that it's the ratio of two integers power of a.
That's going to be a ^ m, and then we're want to have to take the nth root of that.
So that's the definition.
And then, when you're defining a ^ x, so a^x is defined for all x by filling in.
So I'm gonna use that expression in quotation marks, "filling in" by continuity.
This is really what your calculator does when it gives you a to the power x, because you can't even punch in the square root of x.
It doesn't really exist on your calculator.
There's some decimal expansion.
So it takes the decimal expansion to a certain length and spits out a number which is pretty close to the correct answer.
But indeed, in theory, there is an a to the power, square root of 2, even though the square root of 2 is irrational.
And there's a to the pi and so forth.
All right, so that's the exponential function, and let's draw a picture of one.
So we'll try, say y = 2^X here.
And I'm not going to draw such a careful graph, but let's just plot the most important point, which is the point (0,1).
That's 2^0, which is 1.
And then maybe we'll go back up here to - 1 here.
And 2 to the - 1 is this point here.
This is (-1, 0.5) .
The reciprocal.
And over here, we have 1, and so that goes all the way up to 2.
And then exponentials are remarkably fast.
So it's off the board what happens next out at 2.
It's already above my range here, but the graph looks something like this.
All right.
Now I've just visually, at least, graphically filled in all the rest of the points.
You have to imagine all these rational numbers, and so forth.
So this point here would have been (1, 2).
And so forth.
All right?
So that's not too far along.
So now what's our goal?
Well, obviously we want to do calculus here.
So our goal, here, for now - and it's gonna take a while.
We have to think about it pretty hard.
We have to calculate what this derivative is.
All right, so we'll get started.
And the way we get started is simply by plugging in the definition of the derivative.
The derivative is the limit as delta x goes to 0 of a to the x plus delta x, minus a to the x, divided by delta x.
So that's what it is.
And now, the only step that we can really perform here to make this is into something a little bit simpler is to use this very first rule that we have here.
That the exponential of the sum is the product of the exponentials.
So we have here, a^x .
So what I want to use is just the property that a^x delta x = (a^x) (a^delta x).
And if I do that, I see that I can factor out a common factor in the numerator, which is a^x.
So we'll write this as the limit as delta x goes to 0, of a to the x times this ratio, now a to the delta x, minus 1, divided by delta x.
So far, so good?
We're actually almost to some serious progress here.
So there's one other important conceptual step which we need to understand.
And this is a relatively simple one.
We actually did this before, by the way.
We did this with sines and cosines.
The next thing I want to point out to you is that you're used to thinking of x as being the variable.
And indeed, already we were discussing x as being the variable and a as being fixed.
But for the purposes of this limit, there's a different variable that's moving.
x is fixed and delta x is the thing that's moving.
So that means that this factor here, which is a common factor, is constant.
And we can just factor it out of the limit.
It doesn't affect the limit at all.
A constant times a limit is the same as whether we multiply before or after we take the limit.
So I'm just going to factor that out.
So that's my next step here.
a^x, and then I have the limit.
Delta x goes to 0 of a to the delta x minus 1, divided by delta x.
All right?
And so what I have here, so this is by definition the derivative.
So here is d/ dx of a^x, and it's equal to this expression here.
Now, I want to stare at this expression, and see what it's telling us, because it's telling us as much as we can get so far, without some...
So first let's just look at what this says.
So what it's saying is that the derivative of a^x is a^x times something that we don't yet know.
And I'm going to call this something, this mystery number, M(a) .
So I'm gonna make the label, M(a) is equal to the limit as delta x goes to 0 of a to the delta x minus 1 divided by delta x.
All right?
So this is a definition.
So this mystery number M(a) has a geometric interpretation, as well.
So let me describe that.
It has a geometric interpretation, and it's a very, very significant number.
So let's work out what that is.
So first of all, let's rewrite the expression in the box, using the shorthand for this number.
So if I just rewrite it, it says d/dx of a^x is equal to this factor, which is M(a), times a^x .
So the derivative of the exponential is this mystery number times a^x.
So we've almost solved the problem of finding the derivative of a^x.
We just have to figure out this one number, M(a), and we get the rest.
So let me point out two more things about this number, M(a).
So first of all, if I plug in x = 0, that's going to be d / dx of a^x , at x = 0.
According to this formula, that's M(a) times a^0 , which of course M(a).
So what is M(a) ?
M(a) is the derivative of this function at 0.
So M(a) is the slope of a^x at x = 0, of the graph.
The graph of a^x at 0.
So again over here, if you looked at the picture.
I'll draw the one tangent line in here, which is this one here.
And this thing has slope, what we're calling M(2).
So, if I graph the function y = 2 ^x, I'll get a certain slope here.
If I graph it with a different base, I might get another slope.
And what we got so far is the following phenomenon: if we know this one number, if we know the slope at this one place, we will be able to figure out the formula for the slope everywhere else.
Now, that's actually exactly the same thing that we did for sines and cosines.
We knew the slope of the sine and the cosine function at x = 0.
The sine function had slope 1.
The cosine function had slope 0.
And then from the sum formulas, well that's exactly this kind of thing here, from the sum formulas.
This sum formula, in fact is easier than the ones for sines and cosines.
From the sum formulas, we worked out what the slope was everywhere.
So we're following the same procedure that we did before.
But at this point we're stuck.
We're stuck, because that time using radians, this very clever idea of radians in geometry, we were able to actually figure out what the slope is.
Whereas here, we're not so sure, what M(2) is, for instance.
We just don't know yet.
So, the basic question that we have to deal with right now is what is M(a)?
That's what we're left with.
And, the curious fact is that the clever thing to do is to beg the question.
So we're going to go through a very circular route here.
That is circuitous, not circular.
Circular is a bad word in math.
That means that one thing depends on another, and that depends on it, and maybe both are wrong.
Circuitous means, we're going to be taking a roundabout route.
And we're going to discover that even though we refuse to answer this question right now, we'll succeed in answering it eventually.
All right?
So how are we going to beg the question?
What we're going to say instead is we're going to define a mystery base, or number e, as the unique number, so that M(e) = 1.
That's the trick that we're going to use.
We don't yet know what e is, but we're just going to suppose that we have it.
Now, I'm going to show you a bunch of consequences of this, and also I have to persuade you that it actually does exist.
So first, let me explain what the first consequence is.
First of all, if M(e) is 1, then if you look at this formula over here and you write it down for e, you have something which is a very usable formula.
d / dx of e^x is just e^x.
All right, so that's an incredibly important formula which is the fundamental one.
It's the only one you have to remember from what we've done.
So maybe I should have highlighted it in several colors here.
That's a big deal.
Very happy.
And again, let me just emphasize, also that this is the one which at x = 0 has slope 1.
That's the way we defined it, alright?
So if you plug in x = 0 here on the right hand side, you got 1.
Slope 1 at x = 0.
So that's great.
Except of course, since we don't know what e is, this is a little bit dicey.
So, next even before explaining what e is...
In fact, we won't get to what e really is until the very end of this lecture.
But I have to persuade you why e exists.
We have to have some explanation for why we know there is such a number.
Ok, so first of all, let me start with the one that we supposedly know, which is the function 2^x .
We'll call it f(x) is 2^x.
All right?
So that's the first thing.
And remember, that the property that it had, was that f '(0) was M(2) .
That was the derivative of this function, the slope of x = 0. of the graph.
Of the tangent line, that is.
So now, what we're going to consider is any kind of stretching.
We're going to stretch this function by a factor k. Any number k. So what we're going to consider is f(kx).
If you do that, that's the same as 2^kx.
Right?
But now if I use the second law of exponents that I have over there, that's the same thing as 2 to the k to the power x, which is the same thing as some base b ^ x, where b is equal to... Let's write that down over here.
b is 2^k .
Right.
So whatever it is, if I have a different base which is expressed in terms of 2, by being of the form 2^k , then that new function is described by this function f (kx) , the stretch.
So what happens when you stretch a function?
That's the same thing as shrinking the x axis.
So when k gets larger, this corresponding point over here would be over here, and so this corresponding point would be over here.
So you shrink this picture, and the slope here tilts up.
So, as we increase k, the slope gets steeper and steeper.
Let's see that explicitly, numerically here.
Explicitly, numerically, if I take the derivative here...
So the derivative with respect to x of b^ x, that's the chain rule, right?
That's the derivative with respect to x of f(kx), which is what?
It's k times f '(kx) .
And so if we do it at 0, we're just getting k times f '(0) , which is k times this M(2).
So how is it exactly that we cook up the right base b?
So b = e when k = 1 over this number.
In other words, we can pick all possible slopes that we want.
This just has the effect of multiplying the slope by a factor.
And we can shift the slope at 0 however we want, and we're going to do it so that the slope exactly matches 1, the one that we want.
We still don't know what k is.
We still don't know what e is.
But at least we know that it's there somewhere.
Yes?
Student: How do you know it's f(kx)?
How do I know?
Well, f(x) is 2 ^ x.
If f(x) is 2^x, then the formula for f(kx) is this.
I've decided what f(x) is, so therefore there's a formula for f(kx).
And furthermore, by the chain rule, there's a formula for the derivative.
And it's k times the derivative of f. So again, scaling does this.
By the way, we did exactly the same thing with the sine and cosine function.
If you think of the sine function here, let me just remind you here, what happens with the chain rule, you get k times cosine k t here.
So the fact that we set things up beautifully with radians that this thing is, but we could change the scale to anything, such as degrees, by the appropriate factor k. And then there would be this scale factor shift of the derivative formulas.
Of course, the one with radians is the easy one, because the factor is 1.
The one with degrees is horrible, because the factor is some crazy number like 180 over pi, or something like that.
Okay, so there's something going on here which is exactly the same as that kind of re-scaling.
So, so far we've got only one formula which is a keeper here.
This one.
We have a preliminary formula that we still haven't completely explained which has a little wavy line there.
And we have to fit all these things together.
Okay, so now to fit them together, I need to introduce the natural log.
So the natural log is denoted this way, ln(x).
So maybe I'll call it a new letter name, we'll call it w = ln x here.
But if we were reversing things, if we started out with a function y = e^x , the property that it would have is that it's the inverse function of e^x .
So it has the property that the lny = x.
Right?
So this defines the log.
Now the logarithm has a bunch of properties and they come from the exponential properties in principle.
You remember these.
And I'm just going to remind you of them.
So the main one that I just want to remind you of is that the ln (X1*X2) = ln X1 ln X2.
And maybe a few more are worth reminding you of.
One is that the logarithm of 1 is 0.
A second is that the logarithm of e is 1.
Alright?
So these correspond to the inverse relationships here.
If I plug in here, x = 0 and x = 1.
If I plug in x = 0 and x = 1, I get the corresponding numbers here: y = 1 and y = e. And maybe it would be worth it to plot the picture once to reinforce this.
So here I'll put them on the same chart.
If you have here e to the x over here.
It looks like this.
Then the logarithm which I'll maybe put in a different color.
So this crosses at this all important point here, (0,1).
And now in order to figure out what the inverse function is, I have to take the flip across the diagonal, x = y.
So that's this shape here, going down like this.
And here's the point (1, 0).
So (1, 0) corresponds to this identity here.
But the log of 1 is 0.
And notice, so this is ln x, the graph of ln x.
And notice it's only defined for x positive, which corresponds to the fact that e to the x is always positive.
So in other words, this white curve is only above this axis, and the orange one is to the right here.
It's only defined for x positive.
Oh, one other thing I should mention is the slope here is 1.
And so the slope there is also going to be 1.
Now, what we're allowed to do relatively easily, because we have the tools to do it, is to compute the derivative of the logarithm.
So to find the derivative of a log, we're going to use implicit differentiation.
This is how we find the derivative of any inverse function.
So remember the way that works is if you know the derivative of the function, you can find the derivative of the inverse function.
And the mechanism is the following: you write down here w = ln x.
Here's the function.
We're trying to find the derivative of w. But now we don't know how to differentiate this equation, but if we exponentiate it, so that's the same thing as e^w = x.
Because let's just stick this in here.
e^ln x = x.
Now we can differentiate this.
So let's do the differentiation here.
We have d/dx e ^ w = d / dx of x, which is 1.
And then this, by the chain rule, is d / dw of e^w times dw /dx.
The product of these two factors.
That's equal to 1.
And now this guy, the one little guy that we actually know and can use, that's this guy here.
So this is e^w times dw/ dx, which is 1.
And so finally, dw/ dx = 1 / e^w .
But what is that?
It's x.
So this is 1 / x.
So what we discovered is, and now I get to put another green guy around here, is that this is equal to 1/x.
So alright, now we have two companion formulas here.
The rate of change of ln x is 1 / x.
And the rate of change of e^x is itself, is e^x.
And it's time to return to the problem that we were having a little bit of trouble with, which is somewhat not explicit, which is this M(a) times x.
We want to now differentiate a to the x in general; not just e^x .
So let's work that out, and I want to explain it in a couple of ways, so you're want to have to remember this, because I'm going to erase it.
But what I'd like you to do is, so now I want to teach you how to differentiate basically any exponential.
So now to differentiate any exponential.
There are two methods.
They're practically the same method.
They have the same amount of arithmetic.
You'll see both of them, and they're equally valuable.
So we're going to just describes them.
Method one I'm going to illustrate on the function a^x .
So we're interested in differentiating this thing, exactly this problem that I still didn't solve yet.
Ok?
So here it is.
And here's the procedure.
The procedure is to write, so the method is to use base e, or convert to base e. So how do you convert to base e?
Well, you write a^x as e to some power.
So what power is it?
It's e to the power ln a, to the power x.
And that is just e ^ x ln a.
So we've made our conversion now to base e. The exponential of something.
So now I'm going to carry out the differentiation.
So d / dx of a ^x = d / dx of e ^ x ln a.
And now, this is a step which causes great confusion when you first see it.
And you must get used to it, because it's easy, not hard.
Okay?
The rate of change of this with respect to x is, let me do it by analogy here.
Because say I had e ^ 3X and I were differentiating it.
The chain rule would say that this is just 3, the rate of change of 3X with respect to x times e ^ 3X.
The rate of change of e to the u with respect to u.
So this is the ordinary chain rule.
And what we're doing up here is exactly the same thing, because ln a, as frightening as it looks, with all three letters there, is just a fixed number.
It's not moving.
It's a constant.
So the constant just accelerates the rate of change by that factor, which is what the chain rule is doing.
So this is equal to ln a times e ^ x ln a.
Same business here with ln a replacing 3.
So this is something you've got to get used to in time for the exam, for instance, because you're going to be doing a million of these.
So do get used to it.
So here's the formula.
On the other hand, this expression here was the same as a ^ x.
So another way of writing this, and I'll put this into a box, but actually I never remember this particularly.
I just re-derive it every time, is that the derivative of a^x = (ln a) a^x .
Now I'm going to get rid of what's underneath it.
So this is another formula.
So there's the formula I've essentially finished here.
And notice, what is the magic number?
The magic number is the natural log of a.
That's what it was.
We didn't know what it was in advance.
This is what it is.
It's the natural log of a.
Let me emphasize to you again, something about what's going on here, which has to do with scale change.
So, for example, the derivative with respect to x of 2^x is (ln 2) 2 ^ x.
The derivative with respect to x, these are the two most obvious bases that you might want to use, is log 10 times 10^x .
So one of the things that's natural about the natural logarithm is that even if we insisted that we must use base 2, or that we must use base 10, we'd still be stuck with natural logarithms.
They come up naturally.
They're the ones which are independent of our human construct of base 2 and base 10.
The natural logarithm is the one that comes up without reference.
And we'll be mentioning a few other ways in which it's natural later.
So I told you about this first method, now I want to tell you about a second method here.
So the second is called logarithmic differentiation.
So how does this work?
Well, sometimes you're having trouble differentiating a function, and it's easier to differentiate its logarithm.
That may seem peculiar, but actually we'll give several examples where this is clearly the case, that the logarithm is easier to differentiate than the function.
So it could be that this is an easier quantity to understand.
So we want to relate it back to the function u.
So I'm going to write it a slightly different way.
Let's write it in terms of primes here.
So the basic identity is the chain rule again, and the derivative of the logarithm, well maybe I'll write it out this way first.
So this would be d of ln u/ du, times du / dx .
These are the two factors.
And that's the same thing, so remember what the derivative of the logarithm is.
This is 1 / u.
So here I have a 1 / u, and here I have a du / dx.
So I'm going to encode this on the next board here, which is sort of the main formula you always need to remember, which is that (ln u)' = u' / u.
That's the one to remember here
[INAUDIBLE]
The question is how did I get this step here?
So this is the chain rule.
The rate of change of ln u with respect to x is the rate of change of ln u with respect u, times the rate of change of u with respect to x.
That's the chain rule.
So now I've worked out this identity here, and now let's show how it handles this case, d / dx of a^x.
Let's do this one.
So in order to get that one, I would take u = a^x .
And now let's just take a look at what ln u is.
Ln u = x ln a.
Now I claimed that this is pretty easy to differentiate.
Again, it may seem hard, but it's actually quite easy.
So maybe somebody can hazard a guess.
What's the derivative of x ln a?
It's just log a.
So this is the same thing that I was talking about before, which is if you've got 3X, and you're taking its derivative with respect to x here, that's just 3.
That's the kind of thing you have.
Again, don't be put off by this massive piece of junk here.
It's a constant.
So again, keep that in mind.
It comes up regularly in this kind of question.
So there's is our formula, that the logarithmic derivative is this.
But let's just rewrite that.
That's the same thing as u' / u, which is log u prime is equal to ln a, right?
So this is our differentiation formula.
So here we have u'.
u' is equal to u times ln a, if I just multiply through by u.
And that's what we wanted.
That's d / dx of a^x = ln a (I'll reverse the order of the two, which is customary) times a^x.
So this is the way that logarithmic differentiation works.
It's the same arithmetic as the previous method, but we don't have to convert to base e. We're just keeping track of the exponents and doing differentiation on the exponents, and multiplying through at the end.
Okay, so I'm going to do two trickier examples, which illustrate logarithmic differentiation.
Again, these could be done equally well by using base e, but I won't do it.
Method one and method two always both work.
So here's a second example: again this is a problem when you have moving exponents.
But this time, we're going to complicate matters by having both a moving exponent and a moving base.
So we have a function u, which is, well maybe I'll call it v, since we already had a function u, which is x ^ x.
A really complicated looking function here.
So again you can handle this by converting to base e, method one.
But we'll do the logarithmic differentiation version, alright?
So I take the logs of both sides.
And now I differentiate it.
And now when I differentiate this here, I have to use the product rule.
This time, instead of having ln a, a constant, I have a variable here.
So I have two factors.
I have ln x when I differentiate with respect to x.
When I differentiate with respect to this factor here, I get that x times the derivative of that, which is 1/x.
So, here's my formula.
Almost finished.
So I have here v' / v. I'm going to multiply these two things together.
I'll put it on the other side, because I don't want to get it mixed up with ln x plus one, the quantity.
And now I'm almost done.
I have v' = v (1 ln x), and that's just d / dx( x ^x) = x^x (1 + lnx).
That's it.
So these two methods always work for moving exponents.
So the next thing that I'd like to do is another fairly tricky example.
And this one is not strictly speaking within Calculus.
Although we're going to use the tools that we just described to carry it out, in fact it will use some Calculus in the very end.
And what I'm going to do is I'm going to evaluate the limit as n goes to infinity of one plus one over n to the power n. So now, the reason why I want to discuss this is, is it turns out to have a very interesting answer.
And it's a problem that you can approach exactly by this method.
And the reason is that it has a moving exponent.
The exponent n here is changing.
And so if you want to keep track of that, a good way to do that is to use logarithms.
So in order to figure out this limit, we're going to take the log of it and figure out what the limit of the log is, instead of the log of the limit.
Those will be the same thing.
So we're going to take the natural log of this quantity here, and that's n ln (1 (1 / n)).
And now I'm going to rewrite this in a form which will make it more recognizable, so what I'd like to do is I'm going to write n or maybe I should say it this way: delta x = 1 / n. So if n is going to infinity, then this delta x is going to be going to 0.
So this is more familiar territory for us in this class, anyway.
So let's rewrite it.
So here, we have one over delta x.
And then that is multiplied by ln 1 delta x.
So n is the reciprocal of delta x.
Now I want to change this in a very, very minor way.
I'm going to subtract 0 from it.
So that's the same thing.
So what I'm going to do is I'm going to subtract ln 1 from it.
That's just equal to 0.
So this is not a problem, and I'll put some parentheses around it.
Now you're supposed to recognize, all of a sudden, what pattern this fits into.
This is the thing which we need to calculate in order to calculate the derivative of the log function.
So this is in the limit as delta x goes to 0 equal to the derivative of ln x.
Where?
Well the base point is x=1.
That's where we're evaluating it.
That's the X0.
That's the base value.
So this is the difference quotient.
That's exactly what it is.
And so this by definition tends to the limit here.
But we know what the derivative of the log function is.
The derivative of the log function is 1 / x.
So this limit is 1.
So we got it.
We got the limit.
And now we just have to work backwards to figure out what this limit that we've got over here is.
So let's do that.
So let's see here the log approached 1.
So the limit as n goes to infinity of 1 plus (1 over n) to the n. So sorry, the log of this.
Yeah, let's write it this way.
It's the same thing, as well, the thing that we know is the log of this.
1 plus 1 over n to the n. And goes to infinity.
That's the one that we just figured out.
But now this thing is the exponential of that.
So it's really e to this power here.
So this guy is the same as the limit of the log of the limit of the thing, which is the same as log of the limit.
The limit of the log and the log of the limit are the same.
Ok, so I take the logarithm, then I'm going to take the exponential.
That just undoes what I did before.
And so this limit is just 1, is e ^ 1.
And so the limit that we want here is equal to e. So I claim that with this step, we've actually closed the loop, finally.
Because we have an honest numerical way to calculate e. The first.
There are many such.
But this one is a perfectly honest numerical way to calculate e. We had this thing.
We didn't know exactly what it was.
It was this M(e), there was M(a), the logarithm, and so on.
We have all that stuff.
But we really didn't need to nail down what this number e is.
And this is telling us, if you take for example 1 plus 1 over 100 to the 100th power, that's going to be a very good, perfectly decent anyway, approximation to e. So this is a numerical approximation, which is all we can ever do with just this kind of irrational number.
And so that closes the loop, and we now have a coherent family of functions, which are actually well defined and for which we have practical methods to calculate.
Okay see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Right now, we're finishing up with the first unit, and I'd like to continue in this lecture, lecture seven, with some final remarks about exponents.
So what I'd like to do is just review something that I did quickly last time, and make a few philosophical remarks about it.
I think that the steps involved were maybe a little tricky, and so I'd like to go through it one more time.
Remember, that we were talking about this number ak, which is (1 1/k)^k.
And what we showed was that the limit as k goes to infinity of ak was e. So the first thing that I'd like to do is just explain the proof a little bit more clearly than I did last time with fewer symbols, or at least with this abbreviation of the symbol here, to show you what we actually did.
So I'll just remind you of what we did last time, and the first observation was to check, rather than the limit of this function, but to take the ln first.
And this is typically what's done when you have an exponential, when you have an exponent.
And what we found was that the limit here was 1 as k goes to infinity.
So last time, this is what we did.
And I just wanted to be careful and show you exactly what the next step is.
If you exponentiate this fact; you take e to this power, that's going to tend to e ^ 1, which is just e. And then, we just observe that this is the same as ak.
So the basic ingredient here is that e ^ ln a = a.
That's because the ln function is the inverse of the exponential function.
Yes, question?
[INAUDIBLE]
So the question was, wouldn't the log of this be 0 because ak is tending to 1.
But ak isn't tending to 1. Who said it was?
If you take the logarithm, which is what we did last time, logarithm of ak is indeed k(ln(1 + 1/k)).
That does not tend to 0.
This part of it tends to 0, and this part tends to infinity.
And they balance each other, 0 times infinity.
We don't really know yet from this expression, in fact we did some cleverness with limits and derivatives, to figure out this limit.
It was a very subtle thing.
It turned out to be 1.
All right?
Now, the thing that I'd like to say - I'm sorry I'm going to erase this aside here - but you need to go back to your notes and remember that this is what we did last time.
Because I want to have room for the next comment that I want to make on this little blackboard here.
What we just derived was this property here, but I made a claim yesterday, and I just want to emphasize it again so that we realized what it is that we're doing.
I looked at this backwards.
One way you can think of this is we're evaluating this limit and getting an answer.
But all equalities can be read both directions.
And we can write it the other way: e = the limit, as k goes to infinity, of this expression here.
So that's just the same thing.
And if we read it backwards, what we're saying is that this limit is a formula for e. So this is very typical of mathematics.
You want to always reverse your perspective all the time.
Equations work both ways, and in this case, we have two different things here.
This e was what we defined as the base, which when you graph e ^ x, you get slope 1 at 0.
And then it turns out to be equal to this limit, which we can calculate numerically.
If you do this on your calculators, you, of course, will have a way of programming in this number and evaluating it for each k. And you'll have another button available to evaluate this one.
So another way of saying it is it that there's a relationship between these two things.
And all of Calculus is a matter of getting these relationships.
So we can look at these things in several different ways.
And indeed, that's what we're going to be doing at least at the end of today in talking about derivatives.
A lot of times when we talk about derivative, we're trying to look at them from several perspectives at once.
Okay, so I have to keep on going with exponents, because I have one loose end.
One loose end that I did not cover yet.
There's one very important formula that's left, and it's the derivative of the powers.
We actually didn't do this - well we did it for rational numbers r. So this is the formula here.
But now we're going to check this for all real numbers, r. So including all the irrational ones as well.
This is also good practice for using base e and using logarithmic differentiation.
So let me do this by our two methods that we can use to handle exponential type problems.
So method one was base e. So if I just rewrite this base e again, that's just this formula over here.
x ^ r = (e^ln x)^r, which is e ^ r ln x.
Okay, so now I can differentiate this.
So I get that d / dx (x^r), now I'm going to use prime notation, because I don't want to keep on writing that d / dx here; e ^ (r ln x)'.
And now, what I can do is I can use the chain rule.
The chain rule says that it's the derivative of a this times the derivative of the function.
So the derivative of the exponential is just itself.
And the derivative of this guy here, well I'll write it out once, is (r ln x)'.
So what's that equal to?
Well, each of the r ln x's is just x ^ r. And this derivative here is... Well the derivative of r is 0.
This is a constant.
It just factors out.
And ln x now the derivative... What's the derivative of ln x?
1 / x, so this is going to be times r / x.
And now, we rewrite it in the customary form, which is r, we put the r in front, x ^ (r - 1).
Ok?
So I just derived the formula for you.
And it didn't matter whether r was rational or irrational, it's the same proof.
Okay so now I have to show you how method two works as well.
So let's do method two, which we called logarithmic differentiation.
And so here I'll use a symbol, say u for x ^ r, and I'll take its logarithm.
That's r ln x.
And now I differentiate it.
I'll leave that in the middle, because I want to remember the key property of logarithmic differention.
But first I'll differentiate it.
Later on, what I'm going to use is that this is the same as u'/u.
This is one way of evaluating a logarithmic derivative.
And then the other is to differentiate the explicit function that we have over here.
And that is just, as we said, r /x.
So now, I multiply through, and I get u' = u(r/x), which is just x^r(r/x), which is just what we did before.
It's r x ^ (r - 1).
Again, you can now see by comparing these two pieces of arithmetic that they're basically the same.
Pretty much every time you convert to base c or you do logarithmic differentiation, it'll amount to the same thing, provided you don't get mixed up.
You generally have to introduce a new symbol here.
On the other hand, you're dealing with exponents there.
It's worth it to know both points of view.
All right, so now I want to make one last remark before we finish with exponents.
And, I'll try to sell this to you in a lot of ways as the court goes on, but one thing that I want to try to emphasize is that the natural logarithm really is natural.
So, I claim that the natural log is natural.
And the example that we're going to use for this illustration is economics.
Okay?
So let me explain to why the natural log is the one that's natural for economics.
If you are imagining the price of a stock that you own goes down by a dollar, that's a totally meaningless statement.
It depends on a lot of things.
In particular, it depends on whether the original price was a dollar or 100 dollars.
So there's not much meaning to these absolute numbers.
It's always the ratios that matter.
So, for example, I just looked up an hour ago, the London Exchange close, and it was down 27.9, which as I said, is pretty meaningless unless you know what the actual total of this index is.
It turns out it was 6,432.
So the change in the price, divided by the price, which in this case is 27.9 / 6,432, is what matters.
And, in this case, it happens to be 0.43%.
All right?
That's what happened today.
And similarly, if you take the infinitessimal of this, people think of days as being relatively small increments when you're investing in a stock, you would be interested in the infinitessimal sense, p'/p.
The derivative of p / p. That's just (ln p)'.
So this is the - let me put a little box around it - the formula of logarithmic differentiation.
But let me just emphasize that it has an actual significance, and it's the one that's used by economists and people who are modeling prices of things all the time.
They never use absolute prices when there are large swings.
They always use the log of the price.
And there's no point in using log base 10, or log base 2.
Those give you junk.
They give you an extra factor of log 2.
It's the natural log that's the obvious one to use.
It's completely straightforward that this is a simpler expression than using log base 10 and having a factor of natural log of 10 there, which would just mess everything up.
All right, so this is just one illustration.
Anything that has to do with ratios is going to encounter logarithms.
All right, so that's pretty much it.
That's all I want to say for now anyway.
There's lots more to say, but we'll be saying it when we do applications of derivatives in the second unit.
So now, what I'd like to do is to start a review.
I'm just going to run through what we did in this unit.
I'll tell you approximately what I expect from you on the test that's coming up tomorrow.
And, well, so let's get started with that.
So this is a review of Unit One.
And I'm just going to put on the board all of the things that you need to think about, anyway, keep in your head.
And there what are called general formulas for derivatives.
And then there are the specific ones.
And let me just remind you what the general formulas are.
There's what you do to differentiate a sum, a multiple of a function, the product rule, the quotient rule.
Those are several general formulas.
And then there's one more, which is the chain rule, which I'm going to say just a little bit more about.
So the derivative of a function of a function is the derivative of the function times the derivative of the other function.
So here, I've abbreviated u = u(x).
Right, so this is one of two ways of writing it.
The other way is also one that you can keep in mind and you might find easier to remember.
It's probably a good idea to remember both formulas.
And then the last type of general formula that we did was implicit differentiation.
Okay?
So when you do implicit differentiation, you have an equation and you don't try to solve for the unknown function.
You just put it in its simplest form and you differentiate.
So, we actually did this, in particular, for inverses.
That was a very, very key method for calculating the inverses of functions.
And it's also true that logarithmic differentiation is of this type.
This is a transformation.
We're differentiating something else.
We're transforming the equation by taking its logarithm and then differentiating.
Ok, so there are a number of different ways this is applied.
It can also be applied, anyway, these are two of them.
So maybe in parenthesis.
These are just examples.
All right.
I'll try to give examples of at least a few of these rules later.
So now, the specific functions that you know how to differentiate: well you know how to differentiate now x ^ r thanks to what I just did.
We have the sine and the cosine functions, which you're responsible for knowing what their derivatives are.
And then other trig functions like tan and secant.
We generally don't bother with cosecants and cotangents, because everything can be expressed in terms of these anyway.
Actually, you can really express everything in terms of sines and cosines.
But what you'll find is that it's much more convenient to remember the derivatives of these as well.
So memorize all of these.
All right, and then we had e^x and ln x.
And we had the inverses of the trig functions.
These were the two that we did: the arctangent and the arcsin.
So those are the ones you're responsible for.
You should have enough time, anyway, to work out anything else, if you know these.
All right, so basically the idea is you have a bunch of special formulas.
You have a bunch of general formulas.
You put them together, and you can generate pretty much anything.
Okay, so let's do a few examples before going on with the review.
Okay, so I do want to do a few examples in sort of increasing level of difficulty and how you would combine these things together.
So first of all, you should remember that if you differentiate the secant function, that's just - oh I just realized that I wanted to say something else before - so forget that.
We'll do that in a second.
I wanted to make some general remarks.
So there's one rule that you discussed in my absence, which is the chain rule.
And I do want to make just a couple of remarks about the chain rule now to remind you of what it is, and also to present some consequences.
So, a little bit of extra on the chain rule.
The first thing that I want say is that we didn't really fully explain why it's true.
And I do want to just explain it by example, okay?
So imagine that you have a function which is, say, 10x b.
All right?
So y = 10x b.
Then obviously, y is changing 10 times as fast as b, right?
The issue is this number here, dy / dx = 10.
And now if x is a function of something, say t, shifted by some other constant here, then dx/dt = 5.
Now all the chain rule is saying is that if y is going 10 times as fast as t, I'm sorry as x, and x is going 5 times as fast as t, then y is going 50 times as fast as t. And algebraically, all this means is if I plug in and substitute, which is what the composition of the two functions amounts to, 10(5t + a) + b and I multiply it out, I get 50t 10a b.
Now these terms don't matter.
The constant terms don't matter.
The rate is 50.
And so the consequence, if we put them together, is that dy/dt = 10*5, which is 50.
All right, so this is in a nutshell why the chain rule works.
And why these rates multiply.
The second thing that I wanted to say about the chain rule is that it has a few consequences that make some of the other rules a little easier to remember or possibly to avoid.
The messiest rule in my humble opinion is the quotient rule, which is kind of a nuisance to remember.
So let me just remind you, if you take just the reciprocal of a function, and you differentiate it, there's another way of looking at this.
And it's actually the way that I use, so I want to encourage you to think about it this way too.
This is the same as (v^-1)' power .
And now instead of using the quotient rule, which we could've used, we can use the chain rule here with the power -1, which works by the power law.
So what is this equal to?
This is equal to (-v^-2)(v').
So here, I've applied the chain rule rather than the quotient rule.
And similarly, suppose I wanted to derive the full quotient rule.
Well, now this may or may not be easier.
But this is one way of remembering what's going on.
If you convert it to (uv^-1) and you differentiate that, now I can use the product rule on this.
Of course, I have to use the chain rule and this rule as well.
So what do I get?
I get u' , the inverse, u, and then I have to differentiate the v inverse.
That's the formula right up here.
That's (-v^-2)(v').
So that's one way of doing it.
This actually explains the funny minus sign when you differentiate v in the formula.
The other formula, the other way that we did it, was by putting this over a common denominator.
The common denominator was v^2.
This comes from this v ^ -2.
And then the second term is - u v'.
And the first term, we have to multiply by an extra factor of v, because we have a v^2 in the denominator.
So it's u'v.
All right, so this is the quotient rule as we wrote it down in lecture, and this is just another way of remembering it or deriving it without remembering it, if you just remember the chain rule and the product rule.
Okay, so you'll find that in many contexts, it's easier to do one or the other.
Okay, so now i'm ready to differentiate the secant and a few such functions.
So we'll do some examples here here.
So here's the secant function, and I want to use that formula up there for the reciprocal.
This is the way I think of it.
This is the cos x ^ -1.
And so, the formula here is just what?
It's just - (cos x) ^ -2 (-sin x).
So now this is usually written in a different fashion, so that's why I'm doing this for a reason actually.
Which is although there are several formulas for things, with trig functions, there are usually five ways of writing something.
So I'm writing this one down so that you know what the standard way of presenting it is.
So what happens here is that we have two minus signs cancelling.
And we get sin x / cos^2 x.
That's a perfectly acceptable answer, but there's a customary way in which is written.
It's written (1 / cos x) (sin x / cos x).
And then we get rid of the denominators by rewriting it in terms of secant and tangent, so sec x tan x.
So this is the form that's generally used when you see these formulas written in textbooks.
And so you know, you need to watch out, because if you ever want to use this kind of Calculus, you'll have not be put off by all the secants and tangents.
All right, so getting slightly more complicated, how about if we differentiate ln(sec x)?
If you differentiate the natural log, that's just going to be (sec x)' / sec x.
And plugging in the formula that we had before, that's (sec x) (tan x) / sec x, which is tan x.
So this one also has a very nice form.
And you might say that this is kind of an ugly function, but the strange thing is that the natural log was invented before the exponential by a guy named Napier, exactly in order to evaluate functions like this.
These are the functions that people cared about a lot, because they were used in navigation.
You wanted to multiply sines and cosines together to do navigation.
And the multiplication he encoded using a logarithm.
So these were invented long before people even knew about exponents.
And it was a surprise, actually, that it was connected to exponents.
So the natural log was invented before the log base 10 and everything else, exactly for this kind of purpose.
Anyway, so this is a nice function, which was very important, so that your ships wouldn't crash into the reef.
Okay, let's continue here.
So there's another kind of function that I want to discuss with you.
And these are the kinds in which there's a choice as to which of these rules to apply.
And I'll just give a couple of examples of that.
There usually is a better and a worse way, so let me illustrate that.
Okay, yet another example.
I hope you've seen some of these before.
Say (x ^ 10 + 8x) ^ 6.
So it's a little bit more complicated than what we had before, because there were several more symbols here.
So what should we do at this point?
There's one choice which I claim is a bad idea, and that is to expand this out to the 6th power.
That's a bad idea, because it's very long.
And then your answer will also be very long.
It will fill the entire exam paper, for instance.
Yeah?
Can you use the chain rule?
Chain rule.
That's it.
We use the chain rule.
So fortunately, this is relatively easy using the chain rule.
We just think of this box as being the function.
And we take 6 times this guy to the 5th, times the derivative of this guy, which is 10x ^ 9 8.
And this is, feeling this in, it's x^10 + 8x.
And that's it.
That's all you need to do differentiate things like this.
The chain rule is very effective
[INAUDIBLE]
That's a good question.
So I'm not really willing to answer too many questions about what's going to be on the exam.
But the question that was just asked is exactly the kind of question I'm very happy to answer.
Ok the question was, in what form is an acceptable answer?
Now in real life, that is a really serious question.
When you ask a computer a question and it gives you 500 million sheets of printout, it's useless.
And you really care what form answers are in, and indeed, somebody might really care what this thing to the 6th power is, and then you would be forced to discuss things in terms of that other functional form.
For the purposes of this exam, this is okay form.
And, in fact, any correct form is an okay form.
I recommend strongly that you not try to simplify things unless we tell you to.
Sometimes it will be to your advantage to simplify things.
Sometimes we'll say simplify.
It takes a good deal of experience to know when it's really worth it to simplify expressions.
Yes?
[INAUDIBLE]
Right, so turning to this example.
The question is what is this derivative?
And here's an answer.
That's the end of the problem.
This is a more customary form.
But this is answer is ok.
Same issue.
That's exactly the point.
Yes?
[INAUDIBLE]
The question is, do you have to show the work?
Do you have to show the work?
Well if I ask you what is d/dx of sec x, then if you wrote down this answer or you wrote down this answer showing no work, that would be acceptable.
If the question was derive the formula for this from the formula for the derivative of the cosine or something like that, then it would not be acceptable.
You'd have to carry out this arithmetic.
So, in other words, typically this will come up, for instance, in various contexts.
You just basically have to follow directions.
Yes?
[INAUDIBLE]
The next question is, are you expected to be able to prove what the derivative of the sine function is?
The short answer to that is yes.
But I will be getting to that when I discuss the rest of the material here.
We're almost there.
Okay, so let me just finish these examples with one last one.
And then we'll talk about this question of things like the derivative of the sine function, and deriving it.
So the last example that I'd like to write down is the one that I promised you in the first lecture, namely to differentiate e ^ x arctan x.
Basically you're supposed to be able to differentiate any function.
So this is the one that we mentioned at the beginning.
So here it is.
Let's do it.
So what is it?
Well, it's just equal to - I have to differentiate.
I have to use the chain rule - it's equal to the exponential times the derivative of this expression here.
That's the chain rule.
That's the first step.
And now I have to apply the product rule here.
So I have e ^ x arctan x.
And I differentiate the first factor, so I get arctan x.
Add to it what happens when I differentiate the second factor, leaving alone the x.
So that's x / 1 x^2.
And that's it.
That's the end of the problem.
It wasn't that hard.
Of course, it requires you to remember all of the rules, and a lot of formulas underlying them.
So that's consistent with what I just told you.
I told you that you wanted to know this.
I told you that you needed to know this product rule, and that you needed to know the chain rule.
And I guess there was one more thing, the derivative of e to the x came into play there.
So of these formulas, we used four of them in this one calculation.
Okay, so now what other things did we talk about in Unit One?
So the main other thing that we talked about was the definition of a derivative.
And also there was sort of a goal which was to get to the meaning of the derivative.
So these are things - so we had a couple of ways of looking at it, or at least a couple that I'm going to emphasize right now.
But first, let me remind you what the formula is.
The derivative is the limit as delta x goes to 0 of (f(x delta x) - f(x)) / delta x.
So that's it, and this is certainly going to be a central focus here.
And you want to be able to recognize this formula in a number of ways.
So, how do we use this?
Well one thing we did was we calculated a bunch of these rates of change.
In fact, more or less, they're the ones which are written right over here.
This list of functions here.
Now, which ones did we start out with just straight from the definition here?
Which of these things?
There were a whole bunch of them.
So we started out with a function 1 / x.
We did x^n.
We did sine x.
We did cosine x.
Now there was a little bit of subtlety with sine x and cosine x.
We got them using something else.
We didn't quite get them all the way.
We got them using the case x = 0.
We got them from the derivative at x = 0, we got the formulas for the derivatives of sine and cosine.
But that was an argument which involved plug in sine x delta x, and running through.
So that's one example.
We also did a ^ x.
And that may be it.
Oh yeah, I think that's about it.
That may be about it.
No.
It isn't.
Ok, so let me make a connection here which you probably haven't yet made, which is that we did it for (u v)'.
And we also did it for (u / v)'.
So sorry, I shouldn't write primes, because that's not consistent.
I differentiated the product; I differentiated the quotient using the same delta x notation.
I guess I forgot that because I wasn't there when it happened.
So look, these are the ones that you do by this.
And, of course, you might have to reduce them to other things.
These involve using something else.
This one involves using the slope of this function at 0, just the way the sine and the cosine did.
This one involves the slopes of the individual functions, u and v. And this one also involves the individual.
So, in other words, it doesn't get you all the way through to the end, but it's expressed in terms of something simpler in each of these cases.
And I could go on.
We didn't do these in class, but you're certainly... e ^ x is a perfectly okay one on one of the exams.
We ask you for 1 / x^2.
In other words, I'm not claiming that it's going to be one on this list, but it certainly can be any one of these.
But we're not going to ask you to go all the way through to the beginning in these formulas.
There are also some fundamental limits that I certainly want you to know about.
And these you can derive in reverse.
So I will describe that now.
So let me also emphasize the following thing: I want to read this backwards now.
This is the theme from the very beginning of this lecture.
Namely, if you're given the function f, you can figure out its derivative by its formula here.
That is the formula for this in terms of what's on the right hand side.
On the other hand, you can also use the formula in that direction, and if you know the slope of something, you can figure out what the limit is.
For example, I'll use the letter x here, even though it's cheating.
Maybe I'll call it delta x so it's clearer to you.
Maybe I'll call it u.
Suppose you look at this limit here.
Well, I claim that you should recognize that is the derivative with respect to u of the function e^u at u = 0, which of course we know to be 1.
So this is reading this formula in reverse.
It's recognizing that one of these limits - let me rewrite this again here - one of these so-called difference quotient limits is a derivative.
And since we know a formula for that derivative, we can evaluate it.
And lastly, there's one other type of thing which I think you should know.
These are the ones you do with difference quotients.
There are also other formulas that you want to be able to drive.
You want to be able to derive formulas by implicit differentiation.
In other words, the basic idea is to take whatever equation you've got and simplify it as much as possible, without insisting that you solve for y.
That's not necessarily the most appropriate way to get the rate of change.
The much simpler formula is sin y = x.
And that one is easier to differentiate implicitly.
So I should say, do this kind of thing.
So that's, if you like, a typical derivation that you might see.
And then there's one last type of problem that you'll face, and it's the other thing that I claim we discussed.
And it goes all the way back to the first lecture.
So the last thing that we'll be talking about is tangent lines.
All right?
The geometric point of view of a derivative.
And we'll be doing more of this in next the unit.
So first of all, you'll be expected to be able to compute the tangent line.
That's often fairly straightforward.
And the second thing is to graph y' , the derivative of a function.
And the third thing, which I'm going to throw in here, because I regard it in a sort of geometric vein, although it's got an analytical aspect to it.
So this is a picture.
This is a computation.
And if you combine the two together, you get something else.
And so this is to recognize differentiable functions.
Alright, so how do you do this?
Well, we really only have one way of doing it.
We're going to check the left and right tangents.
They must be equal.
So again, this is a property that you should be familiar with from some of your exercises.
And the idea is simply, that if the tangent line exists, it's the same from the right and from the left.
Ok, now I'm going to just do one example here from this sort of qualitative sketching skill to give you an example here.
And what I'm going to do is I'm going to draw a graph of a function like this.
And what I want to do underneath is draw the graph of the derivative.
So this is the function y = f(x), and here I'm going to draw the graph of the function y = f'(x) right underneath it.
So now, let's think about what it's supposed to look like.
And the one step that you need to make in order to do this, is to draw a few tangent lines.
I'm just going to draw one down here.
And I'm going to draw one up here.
Now, the tangent lines here - noticed that the slope of these tangent lines are all positive.
So everything I draw down here is going to be above the x-axis.
Furthermore,, as I go further to the left, they get steeper and steeper.
So they're getting higher and higher.
So the function is coming down like this.
It starts up there.
Maybe I'll draw it in green to illustrate the graph here.
So that's this function here.
As we get farther out, it's getting flatter and flatter.
So it's leveling off, but above the axis like that.
So one of the things to emphasize is, you should not expect the derivative to look like the function.
It's totally different.
It's keeping track at each point of its tangent line.
On the other hand, you should get some kind of physical feel for it, and we'll be practicing this more in the next unit.
So let me give you an example of a function which does exactly this.
And it's the function y = ln x.
If you differentiate it, you get y' = 1 / x.
And this plot above is, roughly speaking, the logarithm.
And this plot underneath is the function 1 / x.
We still have time for one question.
And so, fire away.
Yes?
[INAUDIBLE]
The question is, can you show how you derive the inverse tangent of x.
So that's in a lecture.
I'm happy to do it right now, but it's going to make me a whole two minutes.
So, here's how you do it.
y = arctan x.
And now this is hopeless to differentiate, so I rewrite it as tan y = x.
And now I have to differentiate that.
So when I differentiate it, I get the derivative of tan y with respect to x, so with respect to y.
That's (1 / 1 y^2 )y'.
So this is a hard step.
That's the chain rule.
And on the left side I get 1.
So I'm doing this super fast because we have thirty seconds left.
But this is the hard step right here.
And it needs for you to know that d/dy (tan y) = sec^2 y.
So here's the identity.
So you need have known this in advance.
And that's the input into this equation.
So now, what we have is that y' = 1 / sec^2 y, which is the same thing as cos^2 y.
Now, the last bit of the problem is to rewrite this in terms of x.
And that you have to do with a right triangle.
If this is x and this is 1, then the angle is y, because the tangent of y is x.
So this expresses the fact that the tangent of y is x.
And then the hypoteneuse is the square root of 1 x^2.
And so the cosine is 1 divided by that.
So this thing is 1 divided by the square root of 1 x^2, the quantity squared.
So, and then the last little bit here, since I'm racing along, is that it's 1 / 1 + x^2, squared, which I incorrectly wrote over here.
OK, so good luck on the text.
See you tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCorseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're starting a new unit today.
And, so this is Unit 2, and it's called Applications of Differentiation.
OK.
So, the first application, and we're going to do two today, is what are known as linear approximations.
Whoops, that should have two p's in it.
Approximations.
So, that can be summarized with one formula, but it's going to take us at least half an hour to explain how this formula is used.
So here's the formula.
It's f(x) is approximately equal to f(x0) f'(x)( x - x0).
Right?
So this is the main formula.
For right now.
Put it in a box.
And let me just describe what it means, first.
And then I'll describe what it means again, and several other times.
So, first of all, what it means is that if you have a curve, which is y = f(x), it's approximately the same as its tangent line.
So this other side is the equation of the tangent line.
So let's give an example.
I'm going to take the function f(x), which is ln x, and then its derivative is 1 / x.
And, so let's take the base point x0 = 1.
That's pretty much the only place where we know the logarithm for sure.
And so, what we plug in here now, are the values.
So f (1) is the ln of 0.
Or, sorry, the ln of 1, which is 0.
And f'(1), well, that's 1/1, which is 1.
So now we have an approximation formula which, if I copy down what's right up here, it's going to be ln x is approximately, so f(0) is 0, right?
1 (x - 1).
So I plugged in here, for x0, three places.
I evaluated the coefficients and this is the dependent variable.
So, all told, if you like, what I have here is that the logarithm of x is approximately x - 1.
And let me draw a picture of this.
So here's the graph of ln x.
And then, I'll draw in the tangent line at the place that we're considering, which is x = 1.
So here's the tangent line.
And I've separated a little bit, but really I probably should have drawn it a little closer there, to show you the whole point is that these two are nearby.
But they're not nearby everywhere.
So this is the line y = x - 1.
Right, that's the tangent line.
They're nearby only when x is near 1.
So say in this little realm here.
So when x is approximately 1, this is true.
Once you get a little farther away, this straight line, this straight green line will separate from the graph.
But near this place they're close together.
So the idea, again, is that the curve, the curved line, is approximately the tangent line.
And this is one example of it.
All right, so I want to explain this in one more way.
And then we want to discuss it systematically.
So the second way that I want to describe this requires me to remind you what the definition of the derivative is.
So, the definition of a derivative is that it's the limit, as delta x goes to 0, of delta f / delta x, that's one way of writing it, all right?
And this is the way we defined it.
And one of the things that we did in the first unit was we looked at this backwards.
We used the derivative knowing the derivatives of functions to evaluate some limits.
So you were supposed to do that on your.
In our test, there were some examples there, at least one example, where that was the easiest way to do the problem.
So in other words, you can read this equation both ways.
This is really, of course, the same equation written twice.
Now, what's new about what we're going to do now is that we're going to take this expression here, delta f / delta x, and we're going to say well, when delta x is fairly near 0, this expression is going to be fairly close to the limiting value.
So this is approximately f'(x0).
So that, I claim, is the same as what's in the box in pink that I have over here.
So this approximation formula here is the same as this one.
This is an average rate of change, and this is an infinitesimal rate of change.
And they're nearly the same.
That's the claim.
So you'll have various exercises in which this approximation is the useful one to use.
And I will, as I said, I'll be illustrating this a little bit today.
Now, let me just explain why those two formulas in the boxes are the same.
So let's just start over here and explain that.
So the smaller box is the same thing if I multiply through by delta x, as delta f is approximately f'( x0 ) (delta x).
And now if I just write out what this is, it's f ( x ) right, - f ( x0), I'm going to write it this way.
Which is approximately f' ( x0 ), and this is x minus x0.
So here I'm using the notations delta x is x - x0.
And so this is the change in f, this is just rewriting what delta x is.
And now the last step is just to put the constant on the other side.
So f ( x ) is approximately f ( x0 ) f'(x0)( x - x0).
So this is exactly what I had just to begin with, right?
So these two are just algebraically the same statement.
That's one another way of looking at it.
All right, so now, I want to go through some systematic discussion here of several linear approximations, which you're going to be wanting to memorize.
And rather than it's being hard to memorize these, it's supposed to remind you.
So that you'll have a lot of extra reinforcement in remembering derivatives of all kinds.
So, when we carry out these systematic discussions, we want to make things absolutely as simple as possible.
And so one of the things that we do is we always use the base point to be x0.
So I'm always going to have x0 = 0 in this standard list of formulas that I'm going to use.
And if I put x0 = 0, then this formula becomes f(x), a little bit simpler to read.
It becomes f ( x ) = f ( 0 ) f' ( 0 )x.
So this is probably the form that you'll want to remember most.
That's again, just the linear approximation.
But one always has to remember, and this is a very important thing, this one only worked near x is 1.
This approximation here really only works when x is near x0.
So that's a little addition that you need to throw in.
So this one works when x is near 0.
You can't expect it to be true far away.
The curve can go anywhere it wants, when it's far away from the point of tangency.
So, OK, so let's work this out.
Let's do it for the sine function, for the cosine function, and for e ^ x, to begin with.
Yeah.
Question
[INAUDIBLE]
Yeah.
When does this one work.
Well, so the question was, when does this one work.
Again, this is when x is approximately x0.
Because it's actually the same as this one over here.
OK. And indeed, that's what's going on when we take this limiting value.
Delta x going to 0 is the same.
Delta x small.
So another way of saying it is, the delta x is small.
Now, exactly what we mean by small will also be explained.
But it is a matter to some extent of intuition as to how much, how good it is.
In practical cases, people will really care about how small it is before the approximation is useful.
And that's a serious issue.
All right, so let me carry out these approximations for x.
Again, this is always for x near 0.
So all of these are going to be for x near 0.
So in order to make this computation, I have to evaluate the function.
I need to plug in two numbers here.
In order to get this expression.
I need to know what f( 0 ) and I need to know what f' ( 0 ) is.
If this is the function f ( x ), then I'm going to make a little table over to the right here with f' and then I'm going to evaluate f ( 0 ), and then I'm going to evaluate f' ( 0 ), and then read off what the answers are.
Right, so first of all if the function is sine x, the derivative is cosine x.
The value of f ( 0 ), that's sine of 0, is 0.
The derivative is cosine.
Cosine of 0 is 1.
So there we go.
So now we have the coefficients 0 and 1.
So this number is 0.
And this number is 1.
So what we get here is 0 1x, so this is approximately x.
There's the linear approximation to sine x.
Similarly, so now this is a routine matter to just read this off for this table.
We'll do it for the cosine function.
If you differentiate the cosine, what you get is - sine x.
The value at 0 is 1, so that's cosine of 0 at 1.
The value of this = sine at 0 is 0.
So this is going back over here, 1 0x, so this is approximately 1.
This linear function happens to be constant.
And finally, if I do need e ^ x, its derivative is again e ^ x, and its value at 0 is 1, the value of the derivative at 0 is also 1.
So both of the terms here, f ( 0 ) and f' ( 0 ), they're both 1 and we get 1 x.
So these are the linear approximations.
You can memorize these.
You'll probably remember them either this way or that way.
This collection of information here encodes the same collection of information as we have over here.
For the values of the function and the values of their derivatives at 0.
So let me just emphasize again the geometric point of view by drawing pictures of these results.
So first of all, for the sine function, here's the sine - well, close enough.
So that's - boy, now that is quite some sine, isn't it?
I should try to make the two bumps be the same height, roughly speaking.
Anyway the tangent line we're talking about is here.
And this is y = x.
And this is the function sine x.
And near 0, those things coincide pretty closely.
The cosine function, I'll put that underneath, I guess.
I think I can fit it.
Make it a little smaller here.
So for the cosine function, we're up here.
It's y = 1.
Well, no wonder the tangent line is constant.
It's horizontal.
The tangent line is horizontal, so the function corresponding is constant.
So this is y = cosine x.
And finally, if I draw y = e^x, that's coming down like this.
And the tangent line is here.
And it's y = 1 x.
The value is 1 and the slope is 1.
So this is how to remember it graphically if you like.
This analytic picture is extremely important and will help you to deal with sines, cosines and exponentials.
Yes, question
[INAUDIBLE]
The question is what do you normally use linear approximations for.
Good question.
We're getting there.
First, we're getting a little library of them and I'll give you a few examples.
OK, so now, I need to finish the catalog with two more examples which are just a little bit, slightly more challenging.
And a little bit less obvious.
So, the next couple that we're going to do are ln (1 x) and (1 x)^r.
OK, these are the last two that we're going to write down.
And that you need to think about.
Now, the procedure is the same as over here.
Namely, I have to write down f' and I have to write down f ( 0 ) and I have to right down f' ( 0 ).
And then I'll have the coefficients to be able to fill in what the approximation is.
So f' = 1 / 1 x, in the case of the logarithm.
And f ( 0 ), if I plug in, that's ln of 1, which is 0.
And f' if I plug in 0 here, I get 1.
And similarly if I do it for this one, I get r (1 x) ^ r - 1.
And when I plug in f ( 0 ), I get 1 ^ r, which is 1.
And here I get r ( 1 ) ^ r - 1, which is r. So the corresponding statement here is that ln 1 x is approximately x.
And (1 x) ^ r is approximately 1 rx.
That's 0 1x and here we have 1 r x.
And now, I do want to make a connection, explain to you what's going on here and the connection with the first example.
We already did the logarithm once.
And let's just point out that these two computations are the same, or practically the same.
Here I use the base point 1, but because of my, sort of, convenient form, which will end up, I claim, being much more convenient for pretty much every purpose, we want to do these things near x is approximately 0.
You cannot expand the logarithm and understand a tangent line for it at x equals 0, because it goes down to minus infinity.
Similarly, if you try to graph (1 x) ^ r, or x ^ r without the 1 here, you'll discover that sometimes the slope is infinite, and so forth.
So this is a bad choice of point.
1 is a much better choice of a place to expand around.
And then we shift things so that it looks like it's x = 0, by shifting by the 1.
So the connection with the previous example is that the, what we wrote before I could write as the ln u = u - 1.
Right, that's just recopying what I have over here.
Except with the letter u rather than the letter x.
And then I plug in, u = 1 x.
And then that, if I copy it down, you see that I have a u in place of 1 x, that's the same as this.
And if I write out u - 1, if I subtract 1 from u, that means that it's x.
So that's what's on the right-hand side there.
So these are the same computation, I've just changed the variable.
So now I want to try to address the question that was asked about how this is used.
And what the importance is.
And what I'm going to do is just give you one example here.
And then try to emphasize.
The first way in which this is a useful idea.
So, or maybe this is the second example.
If you like.
So we'll call this Example 2, maybe.
So let's just take the logarithm of 1.1.
Just a second.
Let's take the logarithm of 1.1.
So I claim that, according to our rules, I can glance at this and I can immediately see that it's approximately 1/10.
So what did I use here?
I used that the ln (1 x) is approximately x, and the value of x that I used was 1/10.
Right?
So that is the formula, so I should put a box around these two formulas too.
That's this formula here, applied with x = 1/10.
And I'm claiming that 1/10 is a sufficiently small number, sufficiently close to 0 this is an OK statement.
So the first question that I want to ask you is, which do you think is a more complicated thing.
The left-hand side or the right-hand side.
I claim that this is a more complicated thing, you'd have to go to a calculator to punch out and figure out what this thing is.
This is easy.
You know what a tenth is.
So the distinction that I want to make is that this half, this part, this is hard.
And this is easy.
Now, that may look contradictory, but I want to just do it right above as well.
This is hard.
And this is easy.
OK.
This looks uglier, but actually this is the hard one.
And this is giving us information about it.
Now, let me show you why that's true.
Look down this column here.
These are the hard ones, hard functions.
These are the easy functions.
What's easier than this?
Nothing.
OK. Well, yeah, 0.
That's easier.
Over here it gets even worse.
These are the hard functions and these are the easy ones.
So that's the main advantage of linear approximation is you get something much simpler to deal with.
And if you've made a valid approximation you can make much progress on problems.
OK, we'll be doing some more examples, but I saw some more questions before I made that point.
Yeah
[INAUDIBLE]
Is this is ln of 1.1 or what?
[INAUDIBLE]
This is a parens there.
It's ln of 1.1, it's the digital number, right.
I guess I've never used that before a decimal point, have I?
I don't know.
Other questions
[INAUDIBLE]
OK.
So let's continue here.
Let me give you some more examples, where it becomes even more vivid if you like.
That this approximation is giving us something a little simpler to deal with.
So here's Example 3.
I want to, I'll find the linear approximation near x = 0.
I also, when I write this expression near x = 0, that's the same thing as this.
That's the same thing as saying x is approximately 0.
Of the function (e ^ - 3x) / the square root of 1 x.
So here's a function.
OK. Now, what I claim I want to use for the purposes of this approximation, are just the sum of the approximation formulas that we've already derived.
And just to combine them algebraically.
So I'm not going to do any calculus, I'm just going to remember.
So with either the - 3x, it's pretty clear that I should be using this formula for e ^ x.
For the other one, it may be slightly less obvious but we have powers of 1 x over here.
So let's plug those again.
I'll put this up so that you can remember it.
And we're going to carry out this approximation.
So, first of all, I'm going to write this so that it's slightly more suggestive.
Namely, I'm going to write it as a product.
And there you can now see the exponent.
In this case, r = 1/2.
- 1/2 that we're going to use.
OK.
So now I have e ^ - 3x (1 x) ^ -1/2, and that's going to be approximately, well I'm going to use this formula.
I have to use it correctly.
x is replaced by - 3x, so this is 1 - 3x And then over here, I can just copy verbatim the other approximation formula.
With r = - 1/2.
So this is times 1 - 1/2x.
And now I'm going to carry out the multiplication.
So this is 1 - 3x - 1/2x 3/2x^2.
So now, here's our formula.
So now this isn't where things stop.
And indeed, in this kind of arithmetic that I'm describing now, things are easier than they are in ordinary algebra, in arithmetic.
The reason is that there's another step, which I'm now going to perform.
Which is that I'm going to throw away this term here.
I'm going to ignore it.
In fact, I didn't even have to work it out.
Because I'm going to throw it away.
So the reason is that already, when I passed from this expression to this one, that is from this type of thing to this thing, I was already throwing away quadratic and higher-ordered terms.
So this isn't the only quadratic term.
There are tons of them.
I have to ignore all of them if I'm going to ignore some of them.
And in fact, I only want to be left with the linear stuff.
Because that's all I'm really getting a valid computation for.
So, this is approximately 1 minus, so let's see.
It's a total of 7/2x.
And this is the answer.
This is the linear part.
So the x^2 term is negligible.
So we drop x^2 term.
Terms, and higher.
All of those terms should be lower-order.
If you imagine x is 1/10, or maybe 1/100, then these terms will end up being much smaller.
So we have a rather crude approach.
And that's really the simplicity, and that's the savings.
So now, since this unit is called Applications, and these are indeed applications to math, I also wanted to give you a real-life application.
Or a place where linear approximations come up in real life.
So maybe we'll call this example 4.
This is supposedly a real-life example.
I'll try to persuade you that it is.
So I like this example because it's got a lot of math, as well as physics in it.
So here I am, on the surface of the earth.
And here is a satellite going this way.
At some velocity, v. And this satellite has a clock on it because this is a GPS satellite.
And it has a time, t, OK?
But I have a watch, in fact it's right here.
And I have a time which I keep.
Which is t'.
And there's an interesting relationship between t and t', which is called time dilation.
And this is from special relativity.
And it's the following formula.
t' = t / the square root of 1 - (v^2 / C^2), where v is the velocity of the satellite, and C is the speed of light.
So now I'd like to get a rough idea of how different my watch is from the clock on the satellite.
So I'm going to use this same approximation, we've already used it once.
I'm going to write t. But now let me just remind you.
The situation here is, we have something of the form (1 - u) ^ - 1/2.
That's what's happening when I multiply through here.
So with u = v^2 / C^2.
So in real life, of course, the expression that you're going to use the linear approximation on isn't necessarily itself linear.
It can be any physical quantity.
So in this case it's v squared over C squared.
And now the approximation formula says that if this is approximately equal to, well again it's the same rule.
There's an r and then x is - u, so this is - - 1/2, so it's 1 1/2 u.
So this is approximately, by the same rule, this is t, t' is approximately t ( 1 1/2 v^2 / C^2).
Now, I promised you that this would be a real life problem.
So the question is when people were designing these GPS systems, they run clocks in the satellites.
You're down there, you're making your measurements, you're talking to the satellite by, or you're receiving its signals from its radio.
The question is, is this going to cause problems in the transmission.
And there are dozens of such problems that you have to check for.
So in this case, what actually happened is that v is about 4 kilometers per second.
That's how fast the GPS satellites actually go.
In fact, they had to decide to put them at a certain altitude and they could've tweaked this if they had put them at different places.
Anyway, the speed of light is 3 ( 10^5) kilometers per second.
So this number, v^2 / C^2 is approximately 10 ^ - 10.
Now, if you actually keep track of how much of an error that would make in a GPS location, what you would find is maybe it's a millimeter or something like that.
So in fact it doesn't matter.
So that's nice.
But in fact the engineers who were designing these systems actually did use this very computation.
Exactly this.
And the way that they used it was, they decided that because the clocks were different, when the satellite broadcasts its radio frequency, that frequency would be shifted.
Would be offset.
And they decided that the fidelity was so important that they would send the satellites off with this kind of, exactly this, offset.
To compensate for the way the signal is.
So from the point of view of good reception on your little GPS device, they changed the frequency at which the transmitter in the satellites, according to exactly this rule.
And incidentally, the reason why they didn't, they ignored higher-order terms, the sort of quadratic terms, is that if you take u^2 that's a size 10 ^ - 20.
And that really is totally negligible.
That doesn't matter to any measurement at all.
That's on the order of nanometers, and it's not important for any of the uses to which GPS is put.
OK, so that's a real example of a use of linear approximations.
So.
let's take a little pause here.
I'm going to switch gears and talk about quadratic approximations.
But before I do that, let's have some more questions.
Yeah
[INAUDIBLE]
OK, so the question was asked, suppose I did this by different method.
Suppose I applied the original formula here.
Namely, I define the function f (x), which was this function here.
And then I plugged in its value at x = 0 and the value of its derivative at x = 0.
So the answer is, yes, it's also true that if I call this function f ( x ), then it must be true that the linear approximation is f (x0 ) f' of - I'm sorry, it's at 0, so it's f ( 0 ) f' ( 0 )x.
So that should be true.
That's the formula that we're using.
It's up there in the pink also.
So this is the formula.
So now, what about f ( 0 )?
Well, if I plug in 0 here, I get 1 * 1.
So this thing is 1.
So that's no surprise.
And that's what I got.
If I computed f', by the product rule it would be an annoying, somewhat long, computation.
And because of what we just done, we know what it has to be.
It has to be negative 7/2.
Because this is a shortcut for doing it.
This is faster than doing that.
But of course, that's a legal way of doing it.
When you get to second derivatives, you'll quickly discover that this method that I've just described is complicated, but far superior to differentiating this expression twice
[INAUDIBLE]
Would you have to throw away an x^2 term if you differentiated?
No.
And in fact, we didn't really have to do that here.
If you differentiate and then plug in x = 0.
So if you differentiate this and you plug in x = 0, you get - 7/2.
You differentiate this and you plug in x = 0, this term still drops out because it's just a 3x when you differentiate.
And then you plug in x = 0, it's gone to.
And similarly, if you're up here, it goes away and similarly over here it goes away.
So the higher-order terms never influence this computation here.
This just captures the linear features of the function.
So now I want to go on to quadratic approximation.
And now we're going to elaborate on this formula.
So, linear approximation.
Well, that should have been linear approximation.
Liner.
That's interesting.
OK, so that was wrong.
But now we're going to change it to quadratic.
So, suppose we talk about a quadratic approximation here.
Now, the quadratic approximation is going to be just an elaboration, one more step of detail.
From the linear.
In other words, it's an extension of the linear approximation.
And so we're adding one more term here.
And the extra term turns out to be related to the second derivative.
But there's a factor of 2.
So this is the formula for the quadratic approximation.
And this chunk of it, of course, is the linear part.
This time I'll spell 'linear' correctly.
So the linear part is the first piece.
And the quadratic part is the second piece.
I want to develop this same catalog of functions as I had before.
In other words, I want to extend our formulas to the higher-order terms.
And if you do that for this example here, maybe I'll even illustrate with this example before I go on, if you do it with this example here, just to give you a flavor for what goes on, what turns out to be the case.
So this is the linear version.
And now I'm going to compare it to the quadratic version.
So the quadratic version turns out to be this.
That's what turns out to be the quadratic approximation.
And when I use this example here, so this is 1.1, which is the same as ln of 1 1/10, right?
So that's approximately 1/10 - 1/2 (1/10)^2.
So 1/200.
So that turns out, instead of being 1/10, that's point, what is it, .095 or something like that.
It's a little bit less.
It's not 0.1, but it's pretty close.
So if you like, the correction is lower in the decimal expansion.
Now let me actually check a few of these.
I'll carry them out.
And what I'm going to probably save for next time is explaining to you, so this is y, this factor of 1/2, and we're going to do this later.
Do this next time.
You can certainly do well to stick with this presentation for one more lecture.
So we can see this reinforced.
So now I'm going to work out these derivatives of the higher-order terms.
And let me do it for the x approximately 0 case.
So first of all, I want to add in the extra term here.
Here's the extra term.
For the quadratic part.
And now in order to figure out what's going on, I'm going to need to compute, also, second derivatives.
So here I need a second derivative.
And I need to throw in the value of that second derivative at 0.
So this is what I'm going to need to compute.
So if I do it, for example, for the sine function, I already have the linear part.
I need this last bit.
So I differentiate the sine function twice and I get, I claim minus the sine function.
The first derivative is the cosine and the cosine derivative is minus the sine.
And when I evaluate it at 0, I get, lo and behold, 0.
Sine of 0 is 0.
So actually the quadratic approximation is the same.
0x^2.
There's no x^2 term here.
So that's why this is such a terrific approximation.
It's also the quadratic approximation.
For the cosine function, if you differentiate twice, you get the derivative is -sin and derivative of that is - cos.
So that's f'' And now if I evaluate that at 0, I get - 1.
And so the term that I have to plug in here, this - 1 is the coefficient that appears right here.
So I need a - 1/2 x^2 extra.
And if you do it for the e ^ x, you get an e ^ x, and you got a 1 and so you get 1/2 x^2 here.
I'm going to finish these two in just a second, but I first want to tell you about the geometric significance of this quadratic term.
So here we go.
Geometric significance (of the quadratic term).
So the geometric significance is best to describe just by drawing a picture here.
And I'm going to draw the picture of the cosine function.
And remember we already had the tangent line.
So the tangent line was this horizontal here.
And that was y = 1.
But you can see intuitively, that doesn't even tell you whether this function is above or below 1 there.
Doesn't tell you much.
It's sort of begging for there to be a little more information to tell us what the function is doing nearby.
And indeed, that's what this second expression does for us.
It's some kind of parabola underneath here.
So this is y = 1 - 1/2 x^2.
Which is a much better fit to the curve than the horizontal line.
And this is, if you like, this is the best fit parabola.
So it's going to be the closest parabola to the curve.
And that's more or less the significance.
It's much, much closer.
All right, I want to give you, well, I think we'll save these other derivations for next time because I think we're out of time now.
So we'll do these next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, we're ready to begin Lecture 10, and what I'm going to begin with is by finishing up some things from last time.
We'll talk about approximations, and I want to fill in a number of comments and get you a little bit more oriented in the point of view that I'm trying to express about approximations.
So, first of all, I want to remind you of the actual applied example that I wrote down last time.
So that was this business here.
There was something from special relativity.
And the approximation that we used was the linear approximation, with a - 1/2 power that comes out to be t( 1 1/2 v^2 / C^2).
I want to reiterate why this is a useful way of thinking of things.
And why this is that this comes up in real life.
Why this is maybe more important than everything that I've taught you about technically so far.
So, first of all, what this is telling us is the change in t / t, if you do the arithmetic here and subtract t that's using the change in t is t' - t here.
If you work that out, this is approximately the same as 1/2 (v^2 / C^2).
So what is this saying?
This is saying that if you have this satellite, which is going at speed v, and little c is the speed of light, then the change in the watch down here on earth, relative to the time on the satellite, is going to be proportional to this ratio here.
So, physically, this makes sense.
This is time divided by time.
And this is velocity squared divided by velocity squared.
So, in each case, the units divide out.
So this is a dimensionless quantity.
And this is a dimensionless quantity.
And the only point here that we're trying to make is just this notion of proportionality.
So I want to write this down.
Just, in summary.
So the error fraction, if you like, which is sort of the number of significant digits that we have in our measurement, is proportional, in this case, to this quantity.
It happens to be proportional to this quantity here.
And the factor is, happens to be, 1/2.
So these proportionality factors are what we're looking for.
Their rates of change.
Their rates of change of something with respect to something else.
Now, on your homework, you have something rather similar to this.
So in Problem, on Part 2b, Part II, Problem 1, there's the speed of a pitch, right?
And the speed of the pitch is changing depending on how high the mound is.
And the point here is that that's approximately proportional to the change in the height of the mound.
In that problem, we had this delta h, that was the x variable in that problem.
And what you're trying to figure out is what the constant of proportionality is.
That's what you're aiming for in this problem.
So there's a linear relationship, approximately, to all intents and purposes this is an equality.
Because the lower order terms are unimportant for the problem.
Just as over here, this function is a little bit complicated.
This function is a little more simple.
For the purposes of this problem, they are the same.
Because the errors are negligible for the particular problem that we're working on.
So we might as well work with the simpler relationship.
And similarly, over here, so you could do this with, in this case with square roots, it's not so hard here with reciprocals of square roots.
It's also not terribly hard to do it numerically.
And the reason why we're not doing it numerically is that, as I say, this is something that happens all across engineering.
People are looking for these linear relationships between the change in some input and the change in the output.
And if you don't make these simplifications, then when you get, say, a dozen of them together, you can't figure out what's going on.
In this case the design of the satellite, it's very important.
The speed actually isn't just one speed.
Because it's the relative speed of u to the satellite.
And you might be, it depends on your angle of sight with the satellite what the speed is.
So it varies quite a bit.
So you really need this rule of thumb.
Then there are all kinds of other considerations in this question.
Like, for example, there's the fact that we're sitting on Earth and so we're rotating around on what's called a non-inertial frame.
So there's the question of that acceleration.
There's the question that the gravity that I experience here on Earth is not the same as up at the satellite.
And that also creates a difference in time, as a result of general relativity.
So all of these considerations come down to formulas which are this complicated or maybe a tiny bit more.
Not really that much.
And then people simplify them enormously to these very simple-minded rules.
And they don't keep track of what's going on.
So in order to design the system, you must make these simplifications, otherwise you can't even think about what's going on.
This comes up in everything.
In weather forecasting, economic forecasting.
Figuring out whether there's going to be an asteroid that's going to hit the Earth.
Every single one of these things involves dozens of these considerations.
OK, there was a question that I saw, here.
Yes
[INAUDIBLE]
Yeah.
Basically, any problem where you have a derivative, the rate of change also depends upon what the base point is.
That's the question.
You're saying, doesn't this delta v also depend, I had a base point in that problem.
I happened to decide that pitchers pitch on average about 90 miles an hour.
Whereas, in fact, some pitchers pitch at 100 miles an hour, some pitch at 80 miles an hour, and of course they vary the speed of the pitch.
And so, this varies a little bit.
In fact, that's sort of a second order effect.
It does change the constant of proportionality.
It's a rate of change at a different base point.
Which we're considering fixed.
In fact, that's sort of a second order effect.
When you actually do the computations, what you discover is that it doesn't make that much difference.
To the a.
And that's something that you get from experience.
That it turns out, which things matter and which things don't.
And yet again, that's exactly the same sort of consideration but at the next order of what I'm talking about here is.
You have to have enough experience with numbers to know that if you take, if you vary something a little bit it's not going to change the answer that you're looking for very much.
And that's exactly the point that I'm making.
So I can't make them all at once, all such points.
So that's my pitch for understanding things from this point of view.
Now, we're going to go on, now, to quadratic approximations, which are a little more complicated.
So, we talked a little bit about this last time but I didn't finish.
So I want to finish this up.
And the first thing that I should say is that you use the when the linear approximation is not enough.
OK, so, that's something that you really need to get a little experience with.
In economics, I told you they use logarithms.
So sometimes they use log linear functions.
Sometimes they use log quadratic functions when the log linear ones don't work.
So most modeling in economics is with log quadratic functions.
And if you've made it any more complicated than that, it's useless.
And it's a mess.
And people don't do it.
So they stick with the quadratic ones, typically.
So the basic formula here, and I'm going to take the base point to be 0, is that f ( x ) is approximately f ( 0 ) f' ( 0 )x.
That's the linear part.
Plus this extra term.
Which is f'' ( 0 ) / 2x^2.
And this is supposed to work for x near 0.
So it shows in the base point as simply as possible.
So here's more or less where we left off last time.
And one thing that I said I was going to explain, which I will now, is why it's (1/2) f'' ( 0 ).
So we need to know that.
So let's work that out here first of all.
So I'm just going to do it by example.
So if you like, the answer is just, well, what happens when you have a parabola?
A parabola's a quadratic.
It had better, its quadratic approximation had better be itself.
It's got to be the best one.
So it's got to be itself.
So this formula, if it's going to work, has to work on the nose.
For quadratic functions.
So, let's take a look.
If I differentiate, I get b 2cx.
If I differentiate a second time, I get 2c.
And now let's plug it in.
Well, we can recover, what is it that we want to recover?
We want to recover these numbers a, b and c using the derivatives evaluated at 0.
So let's see.
It's pretty easy, actually.
f ( 0 ) = a.
That's on the nose.
If you plug in x = 0 here, these terms drop out and you get a.
And now, f' ( 0 ), whoops that was wrong.
So I wrote f' but what I meant was f. So f ( 0 ) is a.
Let's back up.
f ( 0 ) is a, so if I plug in x = 0 I get a.
Now, f' ( 0 ), that's this next formula here, f' ( 0 ), I plug in 0 here, and I get b.
That's also good.
And that's exactly what the linear approximation is supposed to be.
But now you notice, f'' is 2c.
So to recover c, I better take half of it.
And that's it.
That's the reason.
There's no chance that any other formula could work.
And this one does.
So that's the explanation for the formula.
And now I remind you that I had a collection of basic formulas written on the board.
And I want to just make sure we know all of them again.
So, first of all, there was sine x is approximately x. Cosine x is approximately 1 - 1/2 x^2.
And e ^ x is approximately 1 x 1/2 x^2.
So those were three that I mentioned last time.
And, again, this is all for x near 0.
All for x near 0 only.
These are wildly wrong, Far away, but near 0 they're nice, good, quadratic approximations.
Now, the other two approximations that I want to mention are the logarithm and we use the base point shifted.
So we can put it at x near 0.
And this one - sorry, this is an approximately equals sign there.
Turns out to be x - 1/2 x^2.
And the last one is one (1 x) ^ r, which turns out to be 1 rx r ( r - 1) / 2x^2.
Now, eventually, your mind will converge on all of these and you'll find them relatively easy to memorize.
But it'll take some getting used to.
And I'm not claiming that you should recognize them and understand them all now.
But I'm going to put a giant box around this
[INAUDIBLE]
Yes.
So the question was, you get all of these if you use that equation there.
That's exactly what are you going to do.
so I already did it actually for these three, last time.
But I didn't do it yet for these two.
But I will do it in about two minutes.
Well, maybe five minutes.
But first I want to explain just a few things about these.
They all follow from the basic formula.
In fact, that one deserves a pink box too, doesn't it.
That one's pretty important.
Alright.
Yeah.
Maybe even some little sparkles.
Alright.
OK.
So that's pretty important.
Almost as important as the more basic one without this term here.
So now, let me just tell you a little bit more about the significance.
Again, this is just to reinforce something that we've already done.
But it's closely related to what you're doing on your problem set.
So it's worth your while to recall this.
So, there's this expression that we were dealing with.
And we talked about it in lecture.
And we showed that this tends to e as k goes to infinity.
So that's what we showed in lecture.
And the way that we did that was, we took the logarithm and we wrote it as k times, sorry, the ln of 1 (1 / k).
And then we evaluated the limit of this.
And I want to do this limit again, using linear approximation.
To show you how easy it is.
If you just remember the linear approximation.
And then we'll explain where the quadratic approximation comes in.
So I claim that this is approximately equal to k ( 1 / k).
Now, why is that?
Well, that's just this linear approximation.
So what did I use here?
I used ln of 1 x is approximately x.
For x = 1 / k. That's what I used in this approximation here.
And that's the linear approximation to the natural logarithm.
And this number is relatively easy to evaluate.
I know how to do it.
It's equal to 1.
That's the same, well, so where does this work?
This works where this thing is near 0.
Which is when k is going to infinity.
This thing is working only when k is going to infinity.
So what it's really saying, this approximation formula, it's really saying that as we go to infinity, in k, this thing is going to 1.
As k goes to infinity.
So that's what it's saying.
That's the substance there.
And that's how we want to use it, in many instances.
Just to evaluate limits.
We also want to realize that it's nearby when k is pretty large, like 100 or something like that.
Now, so that's the idea of the linear approximation.
Now, if you want to get the rate of convergence here, so the rate of what's called convergence.
So convergence means how fast this is going towards that.
What I have to do is take the difference.
I have to take ln ak, and I have to subtract 1 from it.
And I know that this is going to 0, and the question is how big is this.
We want it to be very small.
And the answer we're going to get, so the answer just uses the quadratic approximation.
So if I just have a little bit more detail, then this expression here, in other words, I have the next higher order term.
This is like 1 / k, this is like 1 / k^2.
Then I can understand how big the difference is between the expression that I've got and its limit.
And so that's what's on your homework.
This is on your problem set.
OK, so that is more or less an explanation for one of the things that quadratic approximations are good for.
And I'm going to give you one more illustration.
One more illustration.
And then we'll actually check these formulas.
Yeah, another question
[INAUDIBLE]
That's a very good question here.
When they, which in this case means maybe, me, when I give you a question, does one specify whether you want to use a linear or a quadratic approximation.
The answer is, in real life when you're faced with a problem like this, where some satellite is orbiting and you want to know the effects of gravity or something like that, nobody is going to tell you anything.
They're not even going to tell you whether a linear approximation is relevant, or a quadratic or anything.
So you're on your own.
When I give you a question, at least for right now, I'm always going to tell you.
But as time goes on I'd like you to get used to when it's enough to get away with a linear approximation.
And you should only use a quadratic approximation if somebody forces you to.
You should always start trying with a linear one.
Because the quadratic ones are much more complicated as you'll see in this next example.
OK, so the example that I want to use is, you're going to be stuck with it because I'm asking for the quadratic.
So we're going to find the quadratic approximation near, for x near 0.
To what?
Well, this is the same function that we used in the last lecture.
I think this was it.
e ^ - 3x (1 x) ^ - 1/2.
OK.
So, unfortunately, I stuck it in the wrong place to be able to fit this very long formula here.
So I'm going to switch it.
I'm just going to write it here.
And we're going to just do the approximation.
So we're going to say quadratic, in parentheses.
And we'll say x near 0.
So that's what I want.
So now, here's what I have to do.
Well, I have to write in the quadratic approximation for e ^ - 3x, and I'm going to use this formula right here.
And so that's (1 (- 3x) (- 3x)^2 / 2).
And the other factor, I'm going to have to use this formula down here.
because r is - 1/2.
And so that's (1 - 1/2 x 1/2 ( - 1/2)( - 3/2)x^2).
So this is the r term, and this is the r - 1 term.
And now I'm going to do something which is the only good thing about quadratic approximations.
They're messy, they're long, there's nothing particularly good about them.
But there is one good thing about them.
Which is that you always get to ignore the higher order terms.
So even though this looks like a very ugly multiplication, I can do it in my head.
Just watching it.
Because I get a 1 * 1, I'm forced with that term here.
And then I get the cross terms which are linear, which is - 3x - 1/2 x.
We already did that when we calculated the linear approximation, so that's this times the 1 and this times that 1.
And now I have three cross-terms which are quadratic.
So one of them is these two linear terms are multiplying.
So that's plus 3/2 x^2.
That's (- 3)( - 1/2).
And then there's this term, multiplying the 1, that's plus 9/2 x^2.
And then there's one last term, which is this monster here.
Multiplying 1, and that is - 3/8.
So the great thing is, we drop x^3, x ^ 4, etc., terms.
Yeah?
[INAUDIBLE]
OK, well so copy it down.
And you work it out as I'm doing it now.
So what I did is, I multiplied 1 by 1.
I'm using the distributive law here.
That was this one.
I multiplied this 3x by this one, that was that term.
I multiplied this by this, that's that term.
And then I multiplied this by this.
In other words, 2 x terms that gave me an x^2 and a (- 3)( - 1/2).
And I'm going to stop at that point.
Because the point is it's just all the rest of the terms that come up.
Now, the reason, the only reason why it's easy is that I only have to go up to x squared term.
I don't have to do the higher ones.
Another question, way back here.
Yeah, right there
[INAUDIBLE]
OK.
So somebody can check my arithmetic, too.
Good
[INAUDIBLE]
Why do I get to drop all the higher-order terms.
So, that's because the situation where I'm going to apply this is the situation in which x is, say, 1/100.
So if here's about 1/100.
Here's something which is on the order of 100.
This is on the order of 1/100^2.
1/100^2, all of these terms.
Now, these cubic and quartic terms are of the order of 1/100^3.
That's 10 ^ - 6.
And the point is that I'm not claiming that I have an exact answer.
And I'm going to drop things of that order of magnitude.
So I'm saving everything up to 4 decimal places.
I'm throwing away things which are 6 decimal places out.
Does that answer your question?
[INAUDIBLE]
So.
That's the situation, and now you can combine the terms.
I mean, it's not very impressive here.
This is equal to 1 - 7/2 x, maybe 51/8 x^2.
If I've made that, if those minus signs hadn't canceled, I would have gotten the wrong answer here.
Anyway.
So, this is a 2 here, sorry.
7/2.
This is the linear approximation we got last time and here's the extra information that we got from this calculation.
Which is this 51/8 term.
Right, you have to accept that there's a certain degree of complexity to this problem and the answer is sufficiently complicated so it can't be less arithmetic because we get this peculiar 51/8 there, right.
So one of the things to realize is that these kinds of problems, because they involve many, many terms are always going to involve a little bit of complicated arithmetic.
Last little bit, I did promise you that I was going to derive these two relations, as I said.
Did the ones in the left column.
So let's carry that out.
And as someone just pointed out, it all comes from this formula here.
So let's just check it.
So we'll start with the ln function.
This is the function, f, and then f' is 1/1 x.
And f'', so this is f', this is f'', is - 1/1 x^2.
And now I have to plug in x = 0.
So at x = 0 this is ln 1, which is 0.
So this is at x = 0.
I'm getting 0 here, I plug in 0 and I get 1.
And here, I plug in 0 and I get - 1.
So now I go and I look up at that formula, which is way in that upper corner there.
And I see that the coefficient on the constant is 0.
The coefficient on x is 1.
And then the other coefficient, the very last one, is - 1/2.
So this is the - 1 here.
And then in the formula, there's a 2 in the denominator.
So it's half of whatever I get for this second derivative, at 0.
So this is the approximation formula, which is way up in that corner there.
Similarly, if I do it for (1 x) ^ r, I have to differentiate that I get r( 1 x) ^ r - 1, and then r ( r - 1( x 1) ^ r - 2.
So here are the derivatives.
And so if I evaluate them at x = 0, I get 1.
That's 1 ^ r = 1.
And here I get r. (1 ^ r - 1)r. And here, I plug in x = 0 and I get r ( r - 1).
So again, the pattern is right above it here.
The 1 is there, the r is there.
And then instead of r ( r - 1), I have half that.
For the coefficient.
So these are just examples.
Obviously if we had a more complicated functional, we might carry this out.
But as a practical matter, we try to stick with the ones in the pink box and just use algebra to get other formulas.
So I want to shift gears now and treat the subject that was supposed to be this lecture.
And we're not quite caught up, but we will try to do our best to do as much as we can today.
So the next topic is curve sketching.
And so let's get started with that.
So now, happily in this subject, there are more pictures and it's a little bit more geometric.
And there's relatively little computation.
So let's hope we can do this.
So I want to, so here we go, we'll start with curve sketching.
And the goal here
[INAUDIBLE]
So that's like, liner, the last time.
That's kind of sketchy spelling, isn't it?
Yeah, there are certain kinds of things which I can't spell.
But, alright.
Sketching.
Alright.
So here's our goal.
Our goal is to draw the graph of f, using f' and f''.
Whether they're positive or negative.
So that's it.
This is the goal here.
However, there is a big warning that I want to give you.
And this is one that unfortunately I now have to make you unlearn, especially those that you that have actually had a little bit of calculus before, I want to make you unlearn some of your instincts that you developed.
So this will be harder for those of you who have actually done this before.
But for the rest of you, it will be relatively easy.
Which is, don't abandon your precalculus skills.
And common sense.
So there's a great deal of common sense in this.
And it actually trumps some of the calculus.
The calculus just fills in what you didn't quite know yet.
So I will try to illustrate this.
And because we're running a bit late, I won't get to the some of the main punchlines until next lecture.
But I want you to do it.
So for now, I'm just going to tell you about the general principles.
And in the process I'm going to introduce the terminology.
Just, the words that we need to use to describe what is that we're doing.
And there's also a certain amount of carelessness with that in many of the treatments that you'll see.
And a lot of hastiness.
So just be a little patient and we will do this.
So, the first principle is the following.
If f' is positive, then f is increasing.
That's a straightforward idea, and it's closely related to this tangent line approximation or the linear approximation that I just did.
You can just imagine.
Here's the tangent line, here's the function.
And if the tangent line is pointing up, then the function is also going up, too.
So that's all that's going on here.
Similarly, if f' is negative, then f is decreasing.
And that's the basic idea.
Now, the second step is also fairly straightforward.
It's just a second order effect of the same type.
If you have f'' as positive, then that means that f' is increasing.
That's the same principle applied one step up.
Right?
Because if f'' is positive, that means it's the derivative of f'.
So it's the same principle just repeated.
And now I just want to draw a picture of this.
Here's a picture of it, I claim.
And it looks like something's going down.
And I did that on purpose.
But there is something that's increasing here.
Which is, the slope is very steep negative here.
And it's less steep negative over here.
So we have the slope which is some negative number, say, - 4.
And here it's - 3.
So it's increasing.
It's getting less negative, and maybe eventually it'll curve up the other way.
And this is a picture of what I'm talking about here.
That's what it means to say that f' is increasing.
The slope is getting larger.
And the way to describe a curve like this is that it's concave.
So f is concave up.
And similarly, f'' negative is going to be the same thing as f concave, or implies f concave down.
So those are the ways in which derivatives will help us qualitatively to draw graphs.
But as I said before, we still have to use a little bit of common sense when we draw the graphs.
These are just the additional bits of help that we have from calculus.
In drawing pictures.
So I'm going to go through one example to introduce all the notations.
And then eventually, so probably at the beginning of next time, I'll give you a systematic strategy that's going to work when what I'm describing now goes wrong, or a little bit wrong.
So let's begin with a straightforward example.
So, the first example that I'll give you is the function f (x) = 3x - x^3.
Just, as I said, to be able to introduce all the notations.
Now, if you differentiate it, you get 3 - 3x^2.
And I can factor that.
This is 3 ( 1 - x)( 1 x).
OK?
And so, I can decide whether the derivative is positive or negative.
Easily enough.
Namely, just staring at this, I can see that when -1 < x < 1, in that range there, both these numbers, both these factors, are positive.
1 - x is a positive number and 1 x is a positive number.
So, in this range, f' ( x ) is positive.
So this thing is, so f is increasing.
And similarly, in the other ranges, if x is very, very large, this becomes, if it crosses 1, in fact, this becomes, this factor becomes negative and this one stays positive.
So when x > 1, we have that f ' (x) is negative.
And so f is decreasing.
And the same thing goes for the other side.
When it's less than - 1, that also works this way.
Because when it's less than - 1, this number factors positive.
But the other one is negative.
So in both of these cases, we get that it's decreasing.
So now, here's the schematic picture of this function.
So here's - 1, here's 1.
It's going to go down, up, down.
That's what it's doing.
Maybe I'll just leave it alone like this.
That's what it looks like.
So, this is the kind of information we can get right off the bat.
And you notice immediately that it's very important, from the features of the function, the sort of key features of the function that we see here, are these two places.
Maybe I'll even mark them in a, like this.
And these things are turning points.
So what are they?
Well, they're just the points where the derivative changes sign.
Where it's negative here and it's positive there, so there it must be 0.
So we have a definition, and this is the most important definition in this subject, which is that is if f' (x0) = 0, we call x0 a critical point.
The word 'turning point' is not used just because, in fact, it doesn't have to turn around at those points.
But certainly, if it turns around then this will happen.
And we also have another notation, which is the number y0 which is f ( x0 ) is called a critical value.
So these are the key numbers that we're going to have to work out in order to understand what the function looks like.
So what I'm going to do is just plot them.
We're going to plot the critical points and the values.
Well, we found the critical points relatively easily.
I didn't write it down here but it's pretty obvious.
If you set f(x) = 0, that implies that (1 - x)( 1 x) = 0, which implies that x is or - 1.
So those are known as the critical points.
And now, in order to get the critical values here, I have to plug in f (1), for instance, the function is 3x - x^2, so there's this 3 * 1 - 1^3, which is 2.
And f ( - 1), which is 3 ( - 1) - (- 1)^3, which is - 2.
And so I can plot the function here.
So here's the point - 1 and here's, up here, is 2.
So this is - whoops, which one is it?
Yeah.
This is - 1, so it's down here.
So it's (- 1, - 2).
And then over here, I have the point (1, 2).
Alright, now, what information do I get from - so I've now plotted two, I claim, very interesting points.
What information do I get from this?
The answer is, I know something very nearby.
Because I've already checked that the thing is coming down from the left, and coming back up.
And so it must be shaped like this.
Over here.
The tangent line is 0, it's going to be level there.
And similarly over here, it's going to do that.
So this is what we know so far, based on what we've computed.
Question
[INAUDIBLE]
The question is, what happens if there's a sharp corner.
The answer is, calculus, it's not called a critical point.
It's a something else.
And it's a very important point, too.
And we will be discussing those kinds of points.
There are much more dramatic instances of that.
That's part of what we're going to say.
But I just want to save that, alright.
We will be discussing.
Yeah.
Question
[INAUDIBLE]
The question that was asked was, how did I know at the critical point that it's concave down over here and concave up over here.
The answer is that I actually did not use the second derivative yet.
What I used is another piece of information.
I used the information that I derived over here.
That f' is positive, where f' is positive and where it's negative.
So what I know is that the graph is going down to the left of - 1.
It's going up to the right, here.
It's going up here and it's going down there.
I did not use the second derivative.
I used the first derivative.
OK, but I didn't just use the fact that there was a turning point here.
So, actually, I was using the fact that it was a turning point.
I wasn't using the fact that it had the second derivative, though.
OK. For now.
You can also see it by the second derivative as well.
So now, the next thing that I'd like to do, I need to finish off this graph.
And I just want to do it a little bit carefully here.
In the order that is reasonable.
Now, you might happen to notice, and there's nothing wrong with this, so let's even fill in a guess.
In order to fill in a guess, though, and have it be even vaguely right, I do have to notice that this thing crosses, this function crosses the origin.
The function f(x) = 3x - x^3 happens also have the property that f ( 0 ) = 0.
Again, common sense.
You're allowed to use your common sense.
You're allowed to notice a value of the function and put it in.
So there's nothing wrong with that.
If you happen to have such a value.
So, now we can guess what our function is going to look like.
It's going to maybe come down like this.
Come up like this.
And come down like this.
That could be what it looks like.
But, you know, another possibility is it sort of comes along here and goes out that way.
Comes along here and goes out that way, who knows?
It happens, by the way, that it's an odd function.
Right?
Those are all odd powers.
So, actually, it's symmetric on the right half and the left half.
And crosses at 0.
So everything that we do on the right is going to be the same as what happens on the left.
That's another piece of common sense.
You want to make use of that as much as possible, whenever you're drawing anything.
Don't want to throw out information.
So this function happens to be odd.
Odd, and f ( 0 ) = 0.
I'm considering those to be kinds of precalculus skills that I want you to use as much as you can.
So now, here's the first feature which is unfortunately ignored in most discussions of functions.
And it's strange, because nowadays we have graphing things.
And it's really the only part of the exercise that you couldn't do, at least on this relatively simpleminded level, with a graphing calculator.
And that is what I would call the ends of the problem.
So what happens off the screen, is the question.
And that basically is the theoretical part of the problem that you have to address.
You can program this.
You can draw all the pictures that you want.
But what you won't see is what's off the screen.
You need to know something to figure out what's off the screen.
So, in this case, I'm talking about what's off the screen going to the right, or going to the left.
So let's check the ends.
So here, let's just take a look.
We have the function f(x), which is, sorry, 3x - x^3.
Again this is a precalculus sort of thing.
And we're imagining now, let's just do x goes to plus infinity.
So what happens here.
When x is gigantic, this term is completely negligible.
And it just behaves like - x^3, which goes to minus infinity as x goes to plus infinity.
And similarly, f (x) goes to plus infinity if x goes to minus infinity.
Now let me pull down this picture again, and show you what piece of the information we've got.
We now know that it is heading up this way.
It doesn't go like this, it goes up like that.
And I'm going to put an arrow for it, And it's going down like this.
Heading down to minus infinity as x goes out farther to the right.
And going out to plus infinity as x goes farther to the left.
So now there's hardly anything left of this function to describe.
There's really nothing left except maybe decoration.
And we kind of like that decoration, so we will pay attention to it.
And to do that, we'll have to check the second derivative.
So if we differentiate a second time, the first derivative was, remember, 3 - 3x^2.
So the second derivative is - 6x.
So now we notice that f'' (x) is negative if x is positive.
And f'' ( x) is positive if x is negative.
And so in this part it's concave down.
And in this part it's concave up.
And now I'm going to switch the boards so that you'll, and draw it.
And you see that it was begging to be this way.
So we'll fill in the rest of it here.
Maybe in a nice color here.
So this is the whole graph and this is the correct graph.
It comes down in one swoop down here, and comes up here.
And then it changes to concave down right at the origin.
So this point is of interest, not only because it's the place where it crosses the axis, but it's also what's called an inflection point.
Inflection point, that's a point where because f'' at that place is equal to 0.
So it's a place where the second derivative is 0.
We also consider those to be interesting points.
Now, so let me just making one closing remark here.
Which is that all of this information fits together.
And we're going to have much, much harder examples of this where you'll actually have to think about what's going on.
But there's a lot of stuff protecting you.
And functions will behave themselves and turn around appropriately.
Anyway, we'll talk about it next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, we're ready to start the eleventh lecture.
We're still in the middle of sketching.
And, indeed, one of the reasons why we did not talk about hyperbolic functions is this that we're running just a little bit behind.
And we'll catch up a tiny bit today.
And I hope all the way on Tuesday of next week.
So let me pick up where we left off, with sketching.
So this is a continuation.
I want to give you one more example of how to sketch things.
And then we'll go through it systematically.
So the second example that we did as one example last time, is this.
The function is x + + 1 / x + + 2.
And I'm going to save you the time right now.
This is very typical of me, especially if you're in a hurry on an exam, I'll just tell you what the derivative is.
So in this case, it's 1 / (x + 2)^2.
Now, the reason why I'm bringing this example up, even though it'll turn out to be a relatively simple one to sketch, is that it's easy to fall into a black hole with this problem.
So let me just show you.
This is not equal to 0.
It's never equal to 0.
So that means there are no critical points.
At this point, students, many students who have been trained like monkeys to do exactly what they've been told, suddenly freeze and give up.
Because there's nothing to do.
So this is the one thing that I have to train out of you.
You can't just give up at this point.
So what would you suggest?
Can anybody get us out of this jam?
Yeah
[INAUDIBLE]
Right.
So the suggestion was to find the x values where f (x) is undefined.
In fact, so now that's a fairly sophisticated way of putting the point that I want to make, which is that what we want to do is go back to our precalculus skills.
And just plot points.
So instead, you go back to precalculus and you just plot some points.
It's a perfectly reasonable thing.
Now, it turns out that the most important point to plot is the one that's not there.
Namely, the value of x = - 2.
Which is just what was suggested.
Namely, we plot the points where the function is not defined.
So how do we do that?
Well, you have to think about it for a second and I'll introduce some new notation when I do it.
If I evaluate 2 at this place, actually I can't do it.
I have to do it from the left and the right.
So if I plug in - 2 on the positive side, from the right, that's going to be equal to - 2 + 1 / - 2, a little bit more than - 2, +2.
Which is - 1 divided by - now, this denominator is - 2, a little more than that, +2.
So it's a little more than 0.
And that is, well we'll fill that in in a second.
Everybody's puzzled.
Yes
[INAUDIBLE]
No, that's the function.
I'm plotting points, I'm not differentiating.
I've already differentiated it.
I've already got something that's a little puzzling.
Now I'm focusing on the weird spot.
Yes, another question
Wouldn't it be a little less than 0?
Wouldn't it be a little less than 0?
OK, that's a very good point and this is a matter of notation here.
And a matter of parentheses.
So wouldn't this be a little less than 2.
Well, if the parentheses were this way; that is, 2+ , with a - after I did the 2+ , then it would be less.
But it's this way.
OK.
So the notation is, you have a number and you take the part of it.
That's the part which is a little bit bigger than it.
And so this is what I mean.
And if you like, here I can put in those parentheses too.
Yeah, another question
[INAUDIBLE]
Why doesn't the top one have a plus?
The only reason why the top one doesn't have a plus is that I don't need it to evaluate this.
And when I take the limit, I can just plug in the value.
Whereas here, I'm still uncertain.
Because it's going to be 0.
And I want to know which side of 0 it's on.
Whether it's on the positive side or the negative side.
So this one, I could have written here a parentheses 2+, but then it would have just simplified to - 1.
In the limit.
So now, I've got a negative number divided by a tiny positive number.
And so, somebody want to tell me what that is?
Negative infinity.
So, we just evaluated this function from one side.
And if you follow through the other side, so this one here, you get something very similar, except that this should be -- whoops, what did I do wrong?
I meant this.
I want it -2 the same base point, but I want to go from the left.
So that's going to be - 2 + 1, same numerator.
And then this - 2 on the left + 2, and that's going to come out to be - 1 / 0 -, which is plus infinity.
Or just plain infinity, we don't have to put the plus sign.
So this is the first part of the problem.
And the second piece, to get ourselves started, you could evaluate this function at any point.
This is just the most interesting point, alright?
This is just the most interesting place to evaluate it.
Now, the next thing that I'd like to do is to pay attention to the ends.
And I haven't really said what the ends are.
So the ends are just all the way to the left and all the way to the right.
So that means x going to plus or minus infinity.
So that's the second thing I want to pay attention to.
Again, this is a little bit like a video screen here.
And we're about to discover something that's really off the screen, in both cases.
We're taking care of what's happening way to the left, way to the right, here.
And up above, we just took care what happens way up and way down.
So on these ends, I need to do some more analysis.
Which is related to a precalculus skill which is evaluating limits.
And here, the way to do it is to divide by x the numerator and denominator.
Write it as (1 + 1 / x)/ (1 + 2 /x).
And then you can see what happens as x goes to plus or minus infinity.
It just goes to 1.
So, no matter whether x is positive or negative.
When it gets huge, these two extra numbers here go to 0.
And so, this tends to 1.
So if you like, you could abbreviate this as f (+ or - infinity) = 1.
So now, I get to draw this.
And we draw this using asymptotes.
So there's a level which is y = 1.
And then there's another line to draw.
Which is x = - 2.
And now, what information do I have so far?
Well, the information that I have so far is that when we're coming in from the right, that's to - 2, it plunges down to minus infinity.
So that's down like this.
And I also know that it goes up to infinity on the other side of the asymptote.
And over here, I know it's going out to the level 1.
And here it's also going to the level 1.
Now, there's an issue.
I can almost finish this graph now.
I almost have enough information to finish it.
But there's one thing which is making me hesitate a little bit.
And that is, I don't know, for instance, over here, whether it's going to maybe dip below and come back up.
Or not.
So what does it do here?
Can anybody see?
Yeah
[INAUDIBLE]
It can't dip below because there are no critical points.
What a precisely correct answer.
So that's exactly right.
The point here is that because f' is not 0, it can't double back on itself.
Because there can't be any of these horizontal tangents.
It can't double back, so it can't backtrack.
So sorry, if f' is not 0, f can't backtrack.
And so that means that it doesn't look like this.
It just goes like this.
So that's basically it.
And it's practically the end of the problem.
Goes like this.
Now you can decorate your thing, right?
You may notice that maybe it crosses here, the axes, you can actually evaluate these places.
And so forth.
We're looking right now for qualitative behavior.
In fact, you can see where these places hit.
And it's actually a little higher up than I drew.
Maybe I'll draw it accurately.
As we'll see in a second.
So that's what happens to this function.
Now, let's just take a look in a little bit more detail, by double checking.
So we're just going to double check what happens to the sign of the derivative.
And in the meantime, I'm going to explain to you what the derivative is and also talk about the second derivative.
So first of all, the trick for evaluating the derivative is an algebraic one.
I mean, obviously you can do this by the quotient rule.
But I just point out that this is the same thing as this.
And now it has, whoops, that should be a 2 in the denominator.
And so, now this has the form 1 - (1 / x + 2).
So this makes it easy to see what the derivative is.
Because the derivative of a constant is 0, right?
So this is, derivative, is just going to be, switch the sign.
This is what I wrote before.
And that explains it.
But incidentally, it also shows you that that this is a hyperbola.
These are just two curves of a hyperbola.
So now, let's check the sign.
It's already totally obvious to us that this is just a double check.
We didn't actually even have to pay any attention to this.
It had better be true.
This is just going to check our arithmetic.
Namely, it's increasing here.
It's increasing there.
That's got to be true.
And, sure enough, this is positive, as you can see it's 1 over a square.
So it is increasing.
So we checked it.
But now, there's one more thing that I want to just have you watch out about.
So this means that f is increasing.
On the interval minus infinity < x < - 2.
And also from - 2 all the way out to infinity.
So I just want to warn you, you cannot say, don't say f is increasing on minus infinity < infinity, for all x. OK, this is just not true.
I've written it on the board, but it's wrong.
I'd better get rid of it.
There it is.
Get rid of it.
And the reason is, so first of all it's totally obvious.
It's going up here.
But then it went zooming back down there.
And here this was true, but only if x is not - 2.
So there's a break.
And you've got to pay attention to the break.
So basically, the moral here is that if you ignore this place, it's like ignoring Mount Everest, or the Grand Canyon.
You're ignoring the most important feature of this function here.
If you're going to be figuring out where things are going up and down, which is basically all we're doing, you'd better pay attention to these kinds of places.
So don't ignore them.
So that's the first remark.
And now there's just a little bit of decoration as well.
Which is the role of the second derivative.
So we've written down the first derivative here.
The second derivative is now - 2 / (x + 2)^3, right?
So I get that from differentiating this formula up here for the first derivative.
And now, of course, that's also, only works for x not equal to - 2.
And now, we can see that this is going to be negative, let's see, where is it negative?
When this is a positive quantity, so when -2 < x < infinity.
It's negative.
And this is where this thing is concave.
Let's see.
Did I say that right?
Negative, right?
This is concave down.
Right.
And similarly, if I look at this expression, the numerator is always negative but the denominator becomes negative as well when x < -2.
So this becomes positive.
So this case, it was negative over positive.
In this case it was negative divided by negative.
So here, this is in the range - infinity < x < -2.
And here it's concave up.
Now, again, this is just consistent with what we're already guessing.
Of course we already know it in this case if we know that this is a hyperbola.
That it's going to be concave down to the right of the vertical line, dotted vertical line.
And concave up to the left.
So what extra piece of information is it that this is giving us?
Did I say this backwards?
No.
That's OK.
So what extra piece of information is this giving us?
It looks like it's giving us hardly anything.
And it really is giving us hardly anything.
But it is giving us something that's a little aesthetic.
It's ruling out the possibility of a wiggle.
There isn't anything like that in the curve.
It can't shift from curving this way to curving that way to curving this way.
That doesn't happen.
So these properties say there's no wiggle in the graph of that.
Alright.
So.
Question
Do we define the increasing and decreasing base purely on the derivative, or the sort of more general definition of picking any two points and seeing.
Because sometimes there can be an inconsistency between the two definitions
OK, so the question is, in this course are we going to define positive derivative as being the same thing as increasing.
And the answer is no.
We'll try to use these terms separately.
What's always true is that if f' is positive, then f is increasing.
But the reverse is not necessarily true.
It could be very flat, the derivative can be 0 and still the function can be increasing.
OK, the derivative can be 0 at a few places.
For instance, like some cubics.
Other questions?
So that's as much as I need to say in general.
I mean, in a specific case.
But I want to get you a general scheme and I want to go through a more complicated example that gets all the features of this kind of thing.
So let's talk about a general strategy for sketching.
So the first part of this strategy, if you like, let's see.
I have it all plotted out here.
So I'm going to make sure I get it exactly the way I wanted you to see.
So I have, its plotting.
The plot thickens.
Here we go.
So plot, what is it that you should plot first?
Before you even think about derivatives, you should plot discontinuities.
Especially the infinite ones.
That's the first thing you should do.
And then, you should plot end points, for ends.
For x going to plus or minus infinity if there don't happen to be any finite ends to the problem.
And the third thing you can do is plot any easy points.
This is optional.
At your discretion.
You might, for instance, on this example, plot the places where the graph crosses the axis.
If you want to.
So that's the first part.
And again, this is all precalculus.
So now, in the second part we're going to solve this equation and we're going to plot the critical points and values.
In the problem which we just discussed, there weren't any.
So this part was empty.
So the third step is to decide whether f', sorry, whether, f' is positive or negative on each interval.
Between critical points, discontinuities.
The direction of the sign, in this case it doesn't change.
It goes up here and it also goes up here.
But it could go up here and then come back down.
So the direction can change at every critical point.
It can change at every discontinuity.
And you don't know.
However, this particular step has to be consistent with 1 and 2, with steps 1 and 2.
In fact, it will never, if you can succeed in doing steps 1 and 2, you'll never need step 3.
All it's doing is double-checking.
So if you made an arithmetic mistake somewhere, you'll be able to see it.
So that's maybe the most important thing.
And it's actually the most frustrating thing for me when I see people working on problems, is they start step 3, they get it wrong, and then they start trying to draw the graph and it doesn't work.
Because it's inconsistent.
And the reason is some arithmetic error with the derivative or something like that or some other misinterpretation.
And then there's a total mess.
If you start with these two steps, then you're going to know when you get to this step that you're making mistakes.
People don't generally make as many mistakes in the first two steps.
Anyway, in fact you can skip this step if you want.
But that's at risk of not double-checking your work.
So what's the fourth step?
Well, we take a look at whether f'' is positive or negative.
And so we're deciding on things like whether it's concave up or down.
And we have these points, f'' ( x ) = 0, which are called inflection points.
And the last step is just to combine everything.
So this is this the scheme, the general scheme.
And let's just carry it out in a particular case.
So here's the function that I'm going to use as an example.
I'll use f ( x ) = x / ln x.
And because the logarithm - yeah, question.
Yeah
[INAUDIBLE]
The question is, is this optional.
So that's a good question.
Is this optional
[INAUDIBLE]
OK, the question is is this optional; this kind of question.
And the answer is, it's more than just -- so, in many instances, I'm not going to ask you to.
I strongly recommend that if I don't ask you to do it, that you not try.
Because it's usually awful to find the second derivative.
Any time you can get away without computing a second derivative, you're better off.
So in many, many instances.
On the other hand, if I ask you to do it it's because I want you to have the, work to do it.
But basically, if nobody forces you to, I would say never do step 4.
Other questions.
Alright.
So we're going to force ourselves to do step 4, however, in this instance.
But maybe this will be one of the few times.
So here we go, just for illustrative purposes.
OK, now.
So here's the function that I want to discuss.
And the range has to be x positive, because the logarithm is not defined for negative values.
So the first thing that I'm going to do is, I'd like to follow the scheme here.
Because if I don't follow the scheme, I'm going to get a little mixed up.
So the first part is to find the singularities.
That is, the places where f is infinite.
And that's when the logarithm, the denominator, vanishes.
So that's f ( 1 +), if you like.
So that's 1 / ln 1 +, which is 1 / 0, with a little bit of positiveness to it.
Which is infinity.
And second, we do it the other way.
And not surprisingly, this comes out to be negative infinity.
Now, the next thing I want to do is the ends.
So I call these the ends.
And there are two of them.
One of them is f ( 0 ) from the right.
f ( 0+).
So that is 0 + / ln 0 +, which is 0 plus divided by, well, ln 0 + is actually minus infinity.
That's what happens to the logarithm, goes to minus infinity.
So this is 0 over infinity, which is definitely 0, there's no problem.
about what happens.
The other side, so this is the end, this is the first end.
The range is this.
And I just did the left endpoint.
And so now I have to do the right endpoint, I have to let x go to infinity.
So if I let x go to infinity, I'm just going to have to think about it a little bit by plugging in a very large number.
I'll plug in 10 ^ 10, just to see what happens.
So if I plug in 10 ^ 10 into x ln x, I get 10 ^ 10 / ln 10^10.
Which is 10 ^ 10 / 10 ( log 10).
So the denominator, this number here, is about 2.something.
2.3 or something.
So this is maybe 230 in the denominator, and this is a number with ten 0's after it.
So it's very, very large.
I claim it's big.
And that gives us the clue that what's happening is that this thing is infinite.
So, in other words, our conclusion is that f of infinity is infinity.
So what do we have so far for our function?
We're just trying to build the scaffolding of the function.
And we're doing it by taking the most important points.
And from a mathematician's point of view, the most important points are the ones which are sort of infinitely obvious.
For the ends of the problem.
So that's where we're heading.
We have a vertical asymptote, which is at x = 1.
So this gives us x = 1.
And we have a value which is that it's 0 here.
And we also know that when we come in from the - sorry, so we come in from the left, that's f, the one from the left, we get negative infinity.
So it's diving down.
It's going down like this.
And, furthermore, on the other side we know it's climbing up.
So it's going up like this.
Just start a little higher.
Right, so.
So far, this is what we know.
Oh, and there's one other thing that we know.
When we go to plus infinity, it's going back up.
So, so far we have this.
Now, already it should be pretty obvious what's going to happen to this function.
So there shouldn't be many surprises.
It's going to come down like this.
Go like this, it's going to turn around and go back up.
That's what we expect.
So we don't know that yet, but we're pretty sure.
So at this point, we can start looking at the critical points.
We can do our step 2 here -- we need a little bit more room here -- and see what's happening with this function.
So I have to differentiate it.
And it's, this is the quotient rule.
So remember the function is up here, x / ln x.
So I have a ln x^2 in the denominator.
And I get here the derivative of x is 1, so we get 1 ( ln x) - x ( the derivative of ln x, which is 1 /x).
So all told, that's (ln x - 1) / ln x^2.
So here's our derivative.
And now, if I set this equal to 0, at least in the numerator, the numerator is 0 when x = e. The ln e = 1.
So here's our critical point.
And we have a critical value, which is f(e).
And that's going to be e / ln e. Which is e, again.
Because ln e = 1.
So now I can also plot the critical point, which is down here.
And there's only one of them, and it's at (e e).
That's kind of not to scale here, because my blackboard isn't quite tall enough.
It should be over here and then, it's slope 1.
But I dipped it down.
So this is not to scale, and indeed that's one of the things that we're not going to attempt to do with these pictures, is to make them to scale.
So the scale's a little squashed.
So, so far I have this critical point.
And, in fact, I'm going to label it with a c. Whenever I have a critical point I'll just make sure that I remember that that's what it is.
And since there's only one, the rest of this picture is now correct.
That's the same mechanism that we used for the hyperbola.
Namely, we know there's only one place where the derivative is 0.
So that means there no more horizontals, so there's no more backtracking.
It has to come down to here.
Get to there.
This is the only place it can turn around.
Goes back up.
It has to start here and it has to go down to there.
It can't go above 0.
Do not pass go, do not get positive.
It has to head down here.
So that's great.
That means that this picture is almost completely correct now.
And the rest is more or less decoration.
We're pretty much done with the way it looks, at least schematically.
However, I am going to punish you, because I warned you.
We are going to go over here and do this step 4 and fix up the concavity.
And we're also going to do a little bit of that double-checking.
So now, let's again, just, I want to emphasize.
We're going to do a double-check.
This is part 3.
But in advance, I already have, based on this picture I already know what has to be true.
That f is decreasing on 0 to 1. f is also decreasing on 1 to e. And f is increasing on e to infinity.
So, already, because we've plot a bunch of points and we know that there aren't any places where the derivative vanishes, we already know it goes down, down, up.
That's what it's got to do.
Now, we'll just make sure that we didn't make any arithmetic mistakes, now.
By actually computing the derivative, or staring at it, anyway.
And making sure that it's correct.
So first of all, we just take a look at the numerator.
So f,' remember, was (ln x - 1) / ln x^2, the quantity squared.
So the denominator is positive.
So let's just take a look at the three ranges.
So we have 0 < x < 1.
And on that range, the logarithm is negative, so this is negative divided by positive, which is negative.
That's decreasing, that's good.
And in fact, that also works on the next range.
1 l< x < e, it's negative divided by positive.
And the only reason why we skipped 1, again, is that it's undefined there.
And there's something dramatic happening there.
And then, at the last range, when x is bigger than e, that means the logarithm is already bigger than 1.
So the numerator is now positive, and the denominator's still positive, so it's increasing.
So we've just double-checked something that we already knew.
Alright, so that's pretty much all there is to say about step 3.
So this is checking the positivity and negativity.
And now, step 4.
There is one small point which I want to make before we go on.
Which is that sometimes, you can't evaluate the function or its derivative particularly well.
So sometimes you can't plot the points very well.
And if you can't plot the points very well, then you might have to do 3 first, to figure out what's going on a little bit.
You might have to skip.
So now we're going to go on the second derivative.
But first, I want to use an algebraic trick to rearrange the terms.
And I want to notice one more little point.
Which I, as I say, this is decoration for the graph.
So I want to rewrite the formula.
Maybe I'll do it right over here.
Another way of writing this is (1 / ln x) - (1 / (ln x)^2).
So that's another way of writing the derivative.
And that allows me to notice something that I missed, before.
When I solved the equation ln x - 1 - this is equal to 0 here, this equation here.
I missed a possibility.
I missed the possibility that the denominator could be infinity.
So actually, if the denominator's infinity, as you can see from the other expression there, it actually is true that the derivative is 0.
So also when x = 0 +, the slope is going to be 0.
Let me just emphasize that again.
If you evaluate using this other formula over here, this is (1 / ln 0+) - (1 / (ln 0+)^2).
That's 1 / - infinity - 1 / infinity, if you like, squared.
Anyway, it's 0.
So this is 0.
The slope is 0 there.
That is a little piece of decoration on our graph.
It's telling us, going back to our graph here, it's telling us this is coming in with slope horizontal.
So we're starting out this way.
That's just a little start here to the graph.
It's a horizontal slope.
So there really were two places where the slope was horizontal.
Now, with the help of this second formula I can also differentiate a second time.
So it's a little bit easier to do that if I differentiate 1 / ln, that's -( ln x) ^ - 2 ( 1 / x) + 2 (ln x) ^ -3 (1/x).
And that, if I put it over a common denominator, is x ln x^3 times, let's see here.
I guess I'll have to take the 2 - ln x.
So I've now rewritten the formula for the second derivative as a ratio.
Now, to decide the sign, you see there are two places where the sign flips.
The numerator crosses when the logarithm is 2, that's going to be when x = e ^2.
And the denominator flips when x = 1, that's when the log flips from positive to negative.
So we have a couple of ranges here.
So, first of all, we have the range from 0 to 1.
And then we have the range from 1 to e^2.
And then we have the range from e ^2 all the way out to infinity.
So between 0 and 1, the numerator is, well this is a negative number in this, so minus a negative number is positive, so the numerator is positive.
And the denominator is negative, because the ln is negative it's taken to the third power.
So this is a negative numbers, so it's positive divided by a negative number, which is less than 0.
That means it's concave down.
So this is concave down plot.
And that's a good thing, because over here this is concave down.
So there are no wiggles.
It goes straight down, like this.
And then the other two pieces are f'' is equal to, well it's going to switch here.
The denominator becomes positive.
So it's positive over positive.
So this is concave up.
And that's going over here.
But notice that it's not the bottom where it turns around, it's somewhere else.
So there's another transition here.
This is e ^2.
This is e. So what happens at the end is, again, the sign flips again.
Because the numerator, now, when x > e ^2, becomes negative.
And this is negative divided by positive, which is negative.
And part is concave down.
And so we didn't quite draw the graph right.
There's an inflection point right here, which I'll label with i.
Makes a turn the other way at that point.
So there was a wiggle.
There's the wiggle.
Still going up, still going to infinity.
But kind of the slope of the mountain, right?
It's going the other way.
This point happens to be (e^2, e ^2 / 2).
So that's as detailed as we'll ever get.
And indeed, the next game is going to be avoid being, is to avoid being this detailed.
So let me introduce the next subject.
Which is maxima and minima.
OK, now, maxima and minima, maximum and minimum problems can be described graphically in the following ways.
Suppose you have a function, right, here it is.
OK?
Now, find the maximum.
And find the minimum.
OK.
So this problem is done.
The point being, that it is easy to find max and the min with the sketch.
It's very easy.
The goal, the problem, is that the sketch is a lot of work.
We just spent 20 minutes sketching something.
We would not like to spend all that time every single time we want to find a maximum and minimum.
So the goal is to do it with, so our goal is to use shortcuts.
And, indeed, as I said earlier, we certainly never want to use the second derivative if we can avoid it.
And we don't want to decorate the graph and do all of these elaborate, subtle, things which make the graph look nicer and really, or aesthetically appropriate.
But are totally unnecessary to see whether the graph is up or down.
So essentially, this whole business is out, which is a good thing.
And, unfortunately, those early parts are the parts that people tend to ignore.
Which are typically, often, very important.
So let me first tell you the main point here.
So the key idea.
Key to finding maximum.
So the key point is, we only need to look at critical points.
Well, that's actually what it seems like to, in many calculus classes.
But that's not true.
This is not the end of the sentence.
And, end points, and points of discontinuity.
So you must watch out for those.
If you look at the example that I just drew here, which is the one that I carried out, you can see that there are actually five extreme points on this picture.
So let's switch.
So we'll take a look.
There are five places where the max or the min might be.
There's this important point.
This is, as I say, the scaffolding of the function.
There's this point, there down at minus infinity.
There's this, there's this, and there's this.
Only one out of five is a critical point.
So there's more that you have to pay attention to on the function.
And you always have to keep the schema, the picture of the function, in the back of your head.
Even though this may be the most interesting point, and the one that you're going to be looking at.
So we'll do a few examples of that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
In the twelfth lecture, we're going to talk about maxima and minima.
Let's finish up what we did last time.
We really only just started with maxima and minima.
And then we're going to talk about related rates.
So, right now I want to give you some examples of max-min problems.
And we're going to start with a fairly basic one.
So what's the thing about max-min problems?
The main thing is that we're asking you to do a little bit more of the interpretation of word problems.
So many of the problems are expressed in terms of words.
And so, in this case, we have a wire which is length 1.
Cut into two pieces.
And then each piece encloses a square.
Sorry, encloses a square.
And the problem - so this is the setup.
And the problem is to find the largest area enclosed.
So here's the problem.
Now, in all of these cases, in all these cases, there's a bunch of words.
And your job is typically to draw a diagram.
So the first thing you want to do is to draw a diagram.
In this case, it can be fairly schematic.
Here's your unit length.
And when you draw the diagram, you're going to have to pick variables.
So those are really the two main tasks.
To set up the problem.
So you're drawing a diagram.
This is like word problems of old, in grade school through high school.
Draw a diagram and name the variables.
So we'll be doing a lot of that today.
So here's my unit length.
And I'm going to choose the variable x to be the length of one of the pieces of wire.
And that makes the other piece 1 - x.
And that's pretty much the whole diagram, except that there's something that we did with the wire after we cut it in half.
Namely, we built two little boxes out of it.
Like this, these are our squares.
And their side lengths are x / 4 and (1 - x) / 4.
So, so far, so good.
And now we have to think, well, we want to find the largest area.
So I need a formula for area in terms of variables that I've described.
And so that's the last thing.
I'll give the letter a as the label for the area.
And then the area is just the square of x/4 + the square of 1 - x. Whoops, that strange 2 got in here.
Over 4.
So far, so good.
Now, The instinct that you'll have, and I'm going to yield to that instinct, is we should charge ahead and just differentiate.
Alright?
That's alright.
We'll find the critical points.
So we know that those are important points.
So we're going to find the critical points.
In other words, we take the derivative we set, the derivative of a with respect to x = 0.
So if I do that differentiation, I get the, well, so the first one, x^2 / 16, that's 8.
Sorry.
That's x / 8, right?
That's a derivative of this.
And if I differentiate this, I get well, the derivative of 1 - x ^2 is 2 ( 1 - x)( a - 1).
So it's - (1 - x) / 8.
So there are two minus signs in there, I'll let you ponder that differentiation, which I did by the chain rule.
Hang on a sec, OK?
Just wait until we're done.
So here's the derivative.
Is there a problem?
[INAUDIBLE]
Right, so there's a 1/16 here.
This is x^2 / 16.
And so it's 2x / 8, over 16, sorry.
Which has an 8.
That's OK. Alright, so now, This is equal to 0 if and only if x = 1 - x.
That's 2x = = 1, or in other words x = 1/2.
Alright?
So there's our critical point.
So x = 1/2 is the critical point.
And the critical value, which is what you get when you evaluate a at 1/2, is (1/2) / 4, that's 1/8.
So that's (1/8)^2 + (1/8)^2 which = 1/32.
So, so far, so good.
But we're not done yet.
We're not done.
So why aren't we done?
Because we haven't checked the end points.
So let's check the end points.
Now, in this problem, the end points are really sort of excluded.
The ends are between 0 and 1 here.
That's the possible lengths of the cut.
And so what we should really be doing is evaluating in the limit, so that would be the right-hand limit as x goes to 0 of a.
And if you plug in x = 0, what you get here is 0 + (1/4)^2.
Which is 1/16.
And similarly, at the other end, that's 1 - 1 from the left we get (1/4)^2 + 0, which is also 1/16.
So, what you see is that the schematic picture of this function, and isn't even so far off from being the right picture here, is that it's level here as 1/16 and then it dips down and goes up.
Right?
This is 1/2, this is 1, and this level here is a half that.
This is 1/32.
So we did not find, when we found the critical point we did not find the largest area enclosed.
We found the least area enclosed.
So if you don't pay attention to what the function looks like, not only will you about half the time get the wrong answer, you'll get the absolute worst answer.
You'll get the one which is the polar opposite from what you want.
So you have to pay a little bit of attention to the function that you've got.
And in this case it's just very schematic.
It dips down and goes up, and that's true of pretty much most functions.
They're fairly simple.
They maybe only have one critical point.
They only turn around once.
But then, maybe the critical point is the maximum or maybe it's the minimum.
Or maybe it's neither, in fact.
So we'll be discussing that maybe some other time.
So what we find here is that we have the least area enclosed.
Enclosed is 1/32.
And this is true when x = 1/2.
So these are equal squares.
And most when there's only one square.
Which is more or less the limiting situation.
If one of the pieces disappears.
Now, so that's the first kind of example.
And I just want to make one more comment about terminology before we go on.
And I will introduce it with the following question.
What is the minimum?
So, what is the minimum?
Yeah
[INAUDIBLE]
Right.
The lowest value of the function.
So the answer to that question is 1/32.
Now, the problem with this question and you will, so that refers to the minimum value.
But then there's this other question which is where is the minimum.
And the answer to that is x = 1/2.
So one of them is the minimum point, and the other one is the minimum value.
So they're two separate things.
Now, the problem is that people are sloppy.
And especially since you usually find the critical point first, and the value that is plugging in for a second, people will stop short and they'll give the wrong answer to the question, for instance.
Now, both questions are important to answer.
You just need to have a word to put there.
So this is a little bit careless.
When we say what is the minimum, some people will say 1/2.
And that's literally wrong.
They know what they mean.
But it's just wrong.
And when people ask this question, they're being sloppy.
Anyway.
They should maybe be a little clearer and say what's the minimum value.
Or, where is the value achieved.
It's achieved at, or where is the minimum value achieved.
"Where is min achieved?
", would be a better way of phrasing this second question.
So that it has an unambiguous answer.
And when people ask you for the minimum point, they're also - so why is it that we call it the minimum point?
We have this word, critical point, which is what x = 1/2 is in critical value.
And so I'm making those same distinctions here.
But there's another notion of a minimum point, and this is an alternative if you like.
The minimum point is the point (1/2, 1/32).
Right, that's a point on the graph.
It's the point - well, so that graph is way up there, but I'll just put it on there.
That's this point.
And you might say min there.
And you might point to this point, and you might say max.
And similarly, this one might be a max.
So in other words, what this means is simply that people are a little sloppy.
And sometimes they mean one thing and sometimes they mean another.
And you're just stuck with this, because there'll be some authors who will say one thing and some people will mean another and you just have to live with this little bit of annoying ambiguity.
Yeah?
[INAUDIBLE]
OK, so that's a good - very good.
So here we go, find the largest area enclosed.
So that's sort of a trick question, isn't it?
So there are various - that's a good thing to ask.
That's sort of a trick question, why?
Because according to the rules, we're trapped between the two maxima at something which is strictly below.
So in other words, one answer to this question would be, and this is the answer that I would probably give, is 1/16.
But that's not really true.
Because that's only in the limit.
As x goes to 0, or as x goes to 1 -.
And if you like, the most is when you've only got one square.
Which breaks the rules of the problem.
So, essentially, it's a trick question.
But I would answer it this way.
Because that's the most interesting part of the answer, which is that it's 1/16 and it occurs really when one of the squares disappears to nothing.
So now, let's do another example here.
And I just want to illustrate the second style, or the second type of question.
Yeah
[INAUDIBLE]
The question is, since the question was, what was the largest area, why did we find the least area.
The reason is that when we go about our procedure of looking for the least, or the most, we'll automatically find both.
Because we don't know which one is which until we compare values.
And actually, it's much more to your advantage to figure out both the maximum and minimum whenever you answer such a question.
Because otherwise you won't understand the behavior of the function very well.
So, the question.
We started out with one question, we answered both.
We answered two questions.
We answered the question of what the largest and the smallest value was
Also, I'm wondering if you can check both the minimum [INAUDIBLE] approaches [INAUDIBLE]
Yes.
One can also use, the question is, can we use the second derivative test.
And the answer is, yes we can.
In fact, you can actually also stare at this and see that it's a sum of squares.
So it's always curving up.
It's a parabola with a positive second coefficient.
So you can differentiate this twice.
If you do you'll get 1/8 + another 1/8 and you'll get 1/16.
So the second derivative is 1/16.
Is 1/4.
And that's an acceptable way of figuring it out.
I'll mention the second derivative test again, in this second example.
So let me talk about a second example.
So again, this is going to be another question
[INAUDIBLE]
The question is, when I say minimum or maximum point which will I mean
[INAUDIBLE]
So I just repeated the question.
So the question is, when I say minimum point, what will I mean?
OK?
And the answer is that for the purposes of this class I will probably avoid saying that.
But I will say, probably, where is the minimum achieved.
Just in order to avoid that.
If I actually sat at I often am referring to the graph, and I mean this.
And in fact, when you get your little review for the second exam, I'll say exactly that on the review sheet.
And I'll make this very clear when we were doing this.
However, I just want to prepare you for the fact that in real life, and even me when I'm talking colloquially, when I say what's the minimum point of something, I might actually be mixing it up with this other notion here.
So let's do another example.
So this is an example to get us used to the notion of constraints.
So we have, so consider a box without a top.
Or, if you like, we're going to find the box without a top.
With least surface area for a fixed volume.
Find the box without a top with least surface area for a fixed volume.
The procedure for working this out is the following.
You make this diagram.
And you set up the variables.
In this case, we're going to have four names of variables.
We have four letters that we have to choose.
And we'll choose them in a kind of a standard way, alright?
So first I have to tell you one more thing.
Which is something that we could calculate separately but I'm just going to give it to you in advance.
Which is that it turns out that the best box has a square bottom.
And that's going to get rid of one of our variables for us.
So it's got a square bottom, and so let's draw a picture of it.
So here's our box.
Well, that goes down like this, almost.
Maybe I should get it a little farther down.
So here's our box.
Let's correct that just a bit.
So now, what about the dimensions of this box?
Well, this is going to be x, and this is very foreshortened, but it's also x.
The bottom is x by x, it's the same dimensions.
And then the vertical dimension is y.
So far, so good.
Now, I promised you two more letter names.
I want to compute the volume.
The volume is, the base is x ^2, and the height is y.
So there's the volume.
And then the area, the area is the area of the bottom, which is x ^2, that's the bottom.
And then there are the four sides.
And the four sides are rectangles of dimensions xy.
So it's 4xy.
So these are the sides.
And remember, there's no top.
So that's our setup.
So now, the difference between this problem and the last problem is that there are two variables floating around, namely x and y, which are not determined.
But there's what's called a constraint here.
Namely, we've fixed the relationship between x and y.
And so, that means that we can solve for y in terms of x.
So y = v / x ^2.
And then, we can plug that into the formula for a.
So here we have a which is x ^2 + 4x ( v / x^2).
Question
[INAUDIBLE]
The question is, will you need to know this intuitively?
No.
That's something that I would have to give to you.
I mean, it's actually true that a lot of things, the correct answer is something symmetric.
In this last problem, the minimum turned out to be exactly halfway in between because there were sort of equal demands from the two sides.
And similarly, here, what happens is if you elongate one side, you get less - it actually is involved with a two variable problem.
Namely, if you have a rectangle and you have a certain amount of length associated with it.
What's the optimal thing you can do with that.
But I won't, in other words, the optimal rectangle, the least perimeter rectangle, turns out to be a square.
That's the little sub-problem that leads you to this square bottom.
But so that would have been a separate max-min problem.
Which I'm skipping, because I what to do this slightly more interesting one.
So now, here's our formula for a, and now I want to follow the same procedure as before.
Namely, we look for the critical point.
Or points.
So let's take a look.
So again, a is (x ^2 + 4v) / x.
And A' = 2x - (4v / x^2).
So if we set that equal to 0, we get 2x = 2v / x ^2.
So 2x^3.
How did that happen to change into 2?
Interesting, guess that's wrong.
OK.
So this is x ^ 3 = 2v.
And so x = (2 ^ 1/3)( v ^1/3).
So this is the critical point.
So we are not done.
Right?
We're not done, because we don't even know whether this is going to give us the worst box or the best box, from this point of view.
The one that uses the most surface area or the least.
So let's check the ends, right away.
To see what's happening.
So can somebody tell me what the ends, what the end values of x are?
Where does x range from?
[INAUDIBLE]
What's the smallest x can be, yeah
[INAUDIBLE]
OK, the claim was that the largest x could be root a, because somehow there's this x ^2 here and you can't get any further past than that.
But there's a key feature here of this problem.
Which is that a is variable.
The only thing that's fixed in the problem is v. So if v is fixed, what do you know about x?
[INAUDIBLE]
x > 0, yeah.
The lower end point, that's safe.
Because that has to do geometrically with the fact that we don't have any boxes with negative dimensions.
That would be refused by the Post Office, definitely.
Over and above the empty top, which they wouldn't accept either
[INAUDIBLE]
It's true that x < square root of v / y.
So that's using this relationship.
But notice that y = v / x ^2.
So 0 to infinity, I just got a guess there over here, that's right.
Here's the upper limit.
So this is really important to realize.
This is most problems.
Most problems, the variable if it doesn't have a limitation, usually just goes out to infinity.
And infinity is a very important end for the problem.
It's usually an easy end to the problem, too.
So there's a possibility that if we push all the way down to x = 0, we'll get a better box.
It would be very strange box.
A little bit like our vanishing enclosure.
And maybe an infinitely long box, also very inconvenient one.
Might be the best box.
We'll have to see.
So let's just take a look at what happens.
So we're looking at a, at 0 +.
And that's x^2 + 4v / x with x at 0 +.
So what happens to that?
Notice right here, this is going to infinity.
So this is infinite.
So that turns out to be a bad box.
Let's take a look at the other end.
So this is x ^2 + 4v / x, x going to infinity.
And again, this term here means that this thing is infinite.
So the shape of this thing, I'll draw this tiny little schematic diagram over here.
The shape of this thing is like this, right?
And so, when we find that one turnaround point, which happened to be at this strange point 2/3 , (2 ^ 1/3)( v ^ 1/3), that is going to be the minimum.
So we've just discovered that it's the minimum.
Which is just what we were hoping for.
This is going to be the optimal box.
Now, since you asked earlier and since it's worth checking this as well, let's also check an alternative justification.
So an alternative to checking ends is the second derivative test.
I do not recommend the second derivative test.
I try my best, when I give you problems, to make it really hard to apply the second derivative test.
But in this example, the function is simple enough so that it's perfectly OK.
If you take the derivative here, remember, this was whatever it was, 2x - (4v / x ^2).
If I take the second derivative, it's 2 + (8v / x ^3).
And that's positive.
So this thing is concave up.
And that's consistent with its being, the critical point is a min.
Is a minimum point.
See how I almost said, is a min, as opposed to minimum point.
So watch out.
Yes
[INAUDIBLE]
You're one step ahead of me.
The question is, is this the answer to the question or would we have to give y and a and so on and so forth.
So, again, this is something that I want to emphasize and take my time with right now.
Because it depends, what kind of real life problem you're answering, what kind of answer is appropriate.
So, so far we've found the critical point.
We haven't found the critical value.
We haven't found the dimensions of the box.
So we're going to spend a little bit more time on this, exactly in order to address these questions.
So, first of all.
The value of y.
So, so far we have x = (2 ^ 1/3)( v ^ 1/3).
And certainly if you're going to build the box, you also want to know what the y value is.
The y value is going to be, let's see.
Well, it's v / x ^2, so that's v / ((2 ^ 1/3)( v ^ 1/3) ^2, which comes out to be (2 ^ - 2/3)( v ^ 1/3).
So there's the y value.
On top of that, we could figure out the value of a.
So that's also a perfectly reasonable part of the answer.
Depending on what one is interested in, you might care how much money it's going to cost you to build this box.
This optimal box.
And so you plug in the value of a.
So a, let's see, is up here.
It's x ^2 + 4v / x.
So that's going to be ((2 ^ 1/3)( v ^ 1/3) ^2 + (4v / (2 ^ 1/3)( v ^ 1/3)).
And if you work that all out, what you get turns out to be 3 ( 2 ^ 1/3)( v ^ 2/3).
So if you like, one way of answering this question is these three things.
That would be the minimum point corresponding to the graph.
That would be the answer to this question.
But the reason why I'm carrying it out in such detail is I want to show you that there are much more meaningful ways of answering this question.
So let me explain that.
So let me go through some more meaningful answers here.
The first more meaningful answer is the following idea simply, what are known as dimensionless variables.
So the first thing that you notice is the scaling law.
That a / v ^ 2/3 is the thing that's a dimensionless quantity.
That happens to be 3 ( 2 ^ 1/3).
So that's one thing.
If you want to expand the volume, you'll have to expand the area by the 2/3 power of the volume.
And if you think of the area as being in, say, square inches, and the volume of the box as being in cubic inches, then you can see that this is a dimensionless quantity and you have a dimensionless number here, which is a characteristic independent of what a and v were.
The other dimensionless quantity is the y:x.
So x : y.
So, again, that's inches divided by inches.
And it's (2 ^ 1/3)( v ^ 1/3) / ( 2 ^ - 2/3)( v ^ 1/3), which happens to be 2.
So this is actually the best answer to the question.
And it shows you that the box is a 2:1 box.
If this is 2 and this is 1, that's the good box.
And that is just the shape, if you like, and it's the optimal shape.
And certainly that, aesthetically, that's the cleanest answer to the question.
There was a question right here.
Yes
[INAUDIBLE]
Could you repeat that, I couldn't hear
I'm wondering if you'd be able to get that answer if you [INAUDIBLE] square
The question is, could we have gotten the answer if we weren't told that the bottom was square.
The answer is, yes in 18.02 with multivariable.
You would have to have three letters here, an x, a y, and a z, if you like.
And then you'd have to work with all three of them.
So I separated out into one, there's a separate one variable problem that you can do for the base.
And then this is a second one variable problem for this other thing.
And it's just two consecutive one variable problems that solve the multivariable problem.
Or, as I say in multivariable calculus, you can just do it all in one step.
Yeah?
[INAUDIBLE]
Why did I divide x by y, rather than y by x, or in any?
So, again, what I was aiming for was dimensionless quantities.
So x and y are measured in the same units.
And also the proportions of the box.
So that's another word for this is proportions.
Are something that's universal, independent of the volume v. It's something you can say about any box, at any scale.
Whether it be, you know, something by Cristo in the Common.
Maybe we'll get in here to do some fancy --
[INAUDIBLE]
The proportions is with geometric problems typically, when there's a scaling to the problem.
Where the answer is the same at small scales and at large scales.
This is capturing that.
So that's why, the ratios are what's capturing that.
And that's why it's aesthetically the nicest thing to ask
So, what exactly does the ratio of the area to the volume ratio [INAUDIBLE] tell us?
Unfortunately, this number is a really obscure number.
So the question is what does this tell us.
The only thing that I want to emphasize is what's on the left-hand side here.
Which is, it's the area to the 2/3 power of the volume, so it's a dimensionless quantity that happens to be this.
If you do this, for example, in general with planar diagrams, circumferenced area is a bad ratio to take.
What you want to take is the square of circumference to area.
Because the square of circumference has the same dimensions; that is, say, inches squared to area.
Which is in square inches.
So, again, it's these dimensionless quantities that you want to cook up.
And those are the ones that will have universal properties.
The most famous of these is the circle that encloses the most area for its circumference.
And, again, that's only true if you take the square of the circumference.
You do the units correctly.
Anyway.
So we're here, we've got a shape.
We've got an answer to this question.
And I now want to do this problem.
Well, let's put it this way.
I wanted to do this problem by a different method.
I think I'll take the time to do it.
So I want to do this problem by a slightly different method here.
So, here's Example 2 by implicit differentiation.
So the same example, but now I'm going to do it by implicit differentiation.
Well, I'll tell you the advantages and disadvantages to this method here.
So the situation is, you have to start the same way.
So here is the starting place of the problem.
And the goal was the minimum of a with v constant.
So this was the situation that we were in.
And now, what I want to do is just differentiate.
The function y is implicitly a function of x, so I can differentiate the first expression.
And that yields 0 = 2xy + (x ^2 )( y').
So this is giving me my implicit formula for y', So y' = - 2xy / x ^2.
Or in other words, - 2y / x.
And then I also have the dA/dx.
Now, you may notice I'm not using primes quite as much.
Because all of the variables are varying, and so here I'm emphasizing that it's a differentiation with respect to the variable x.
And this becomes 2x + 4y + 4xy'.
So again, this is using the product rule.
And now I can plug in for what y' is, which is right above it.
So this is 2x + 4y + 4x ( - 2y / x).
And that's equal to 0.
And so let's gather that together.
So what do we have?
We have 2x + 4y, and then, altogether, this is 8 - 8y = 0.
So that's the same thing as 2x = 4y.
The - 4y goes to the other side.
And so, x / y = 2.
So this, I claim, so you have to decide for yourself.
But I claim that this is faster.
It's faster, and also it gets to the heart of the matter, which is this scale in variant proportions.
Which is basically also nicer.
So it gets to the nicer answer, also.
So those are the advantages that this has.
So it's faster, and it gets to this, I'm going to call it nicer.
And the disadvantage is it did not check.
Whether this critical point is a max, min, or neither.
So we didn't quite finish the problem.
But we got to the answer very fast.
Yeah, question
[INAUDIBLE]
How would you check it?
[INAUDIBLE]
Well, so it gives you a candidate.
The answer is - so the question is, how would you check it?
The answer is that for this particular problem, the only way to do it is to do something like this.
So in other words, it doesn't save you that much time.
But with many, many, examples, you actually can tell immediately that if the two ends, the thing is, say, 0, and inside it's positive.
Things like that.
So in many, many, cases this is just as good.
So now I'm going to change subjects here.
But the subject that I'm going to talk about next is almost, is very, very closely linked.
Namely, I talked about implicit differentiation.
Now, we're going to just talk about dealing with lots of variables.
And rates of change.
So, essentially, we're going to talk about the same type of thing.
So, I'm going to tell you about a subject which is called related rates.
Which is really just another excuse for getting used to setting up variables and equations.
So, here we go.
Related rates.
And i'm going to illustrate this with one example today, one tomorrow.
So here's my example for today.
So, again, this is going to be a police problem.
But this is going to be a word problem and - sorry, I'm don't want to scare you, no police.
Well, there are police in the story but they're not present.
So, but I'm going to draw it immediately with the diagram because I'm going to save us the trouble.
Although, you know, the point here is to get from the words to the diagram.
So you have the police, and they're 30 feet from the road.
And here's the road.
And you're coming along, here, in your, let's see, in your car going in this direction here.
And the police have radar.
Which is bouncing off of your car.
And what they read off is that you're 50 feet away.
They also know that you're approaching along the line of the radar at a rate of 80 feet per second.
Now, the question is, are you speeding.
That's the question.
So when you're speeding, by the way, up 95 feet per second is about 65 miles per hour.
So that's the threshold here.
So what I want to do now is show you how you set up a problem like this.
This distance is 50.
This is 30, and because it's the distance to a straight line you know that this is a right angle.
So we know that this is a right triangle.
And this is set out to be a right triangle, which is an easy one, a 3, 4, 5 right triangle just so that we can do the computations easily.
So now, the question is, how do we put the letters in to make his problem work, to figure out what the rate of change is.
So now, let me explain that right now.
And we will actually do the computation next time.
So the first thing is, you have to understand what's changing and what's not.
And we're going to use t for time, in seconds.
And now, an important distance here is the distance to this foot of this perpendicular.
So I'm going to name that x. I'm going to give that letter x.
Now, x is varying.
The reason why I need a letter for it as opposed to this 40 is that it's going to have a rate of change with respect to t. And, in fact, it's related to, the question is whether dx / dt is faster or slower than 95.
So that's the thing that's varying.
Now, there's something else that's varying.
This distance here is also varying.
So we need a letter for that.
We do not need a letter for this.
Because it's never changing.
We're assuming the police are parked.
They're not ready to roar out and catch you just yet, and they're certainly not in motion when they've got the radar guns aimed at you.
So you need to know something about the sociology and style of police.
So you need to know things about the real world.
Now, the last bit is, what about this 80 here.
So this is how fast you're approaching.
Now, that's measured along the radar gun.
I claim that that's d by dt of this quantity here.
So this is d is also changing.
That's why we needed a letter for it, too.
So, next time, we'll just put that all together and compute dx / dt.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to keep on going with related rates.
And you may recall that last time we were in the middle of a problem with this geometry.
There was a right triangle.
There was a road.
Which was going this way, from right to left.
And the police were up here, monitoring the situation.
30 feet from the road.
And you're here.
And you're heading this way.
Maybe it's a two lane highway, but anyway it's only going this direction.
And this distance was 50 feet.
So, because you're moving, this distance is varying and so we gave it a letter.
And, similarly, your distance to the foot of the perpendicular with the road is also varying.
At this instant it's 40, because this is a 3, 4, 5 right triangle.
So this was the situation that we were in last time.
And we're going to pick up where we left off.
The question is, are you speeding if the rate of change of D with respect to t is 80 feet per second.
Now, technically that would be - 80, because you're going towards the policemen.
Alright, so D is shrinking at a rate of - 80 feet per second.
And I remind you that 95 feet per second is approximately the speed limit.
Which is 65 miles per hour.
So, again, this is where we were last time.
And, got a little question mark there.
And so let's solve this problem.
So, this is the setup.
There's a right triangle.
So there's a relationship between these lengths.
And the relationship is that x ^2 + 30 ^2 = D ^2.
So that's the first relationship that we have.
And the second relationship that we have, we've already written down.
Which is dx / dt - oops, sorry.
dD / dt = minus 80.
Now, the idea here is relatively straightforward.
We just want to use differentiation.
Now, you could solve for x. Alright, x is the square root of D^2 - 30 squared.
That's one possibility.
But this is basically a waste of time.
It's a waste of your time.
So it's easier, or easiest, to follow this method of implicit differentiation, which I want to encourage you to get used to.
Namely, we just differentiate this equation with respect to time.
Now, when you do that, you have to remember that you are not allowed to plug in a constant.
Namely 40, for t. You have to keep in mind what's really going on in this problem which is that x is moving, it's changing.
And D is also changing.
So you have to differentiate first before you plug in the values.
So the easiest thing is to use, in this case, implicit differentiation.
And if I do that, I get 2x dx / dt is equal to 2D (dD / dt).
No more DDT left.
We hope.
Except in this blackboard.
So there's our situation.
Now, if I just plug in, now I can plug in.
In values.
so this is after taking the derivative.
And, indeed, we have here 2 times the value for x which is 40 at this instant.
And then we have dx / dt.
And that's equal to 2 (D), which is 50.
And then dD / dt is - 80.
So the 80's cancel and we see that dv / dt = - 100 feet per second.
And so the answer to the question is yes.
Although you probably wouldn't be pulled over for this much of a violation.
So that's, right, it's more than 65 miles an hour, by a little bit.
So that's the end of this question.
And usually in these rate of change or related rates questions, this is considered to be the answer to the question, if you like.
So that's one example.
I'm going to give one more example before we go on to some other applications implicit differentiation.
So my second example is going to be, you have a it is a conical tank.
With top of radius 4 feet, let's say.
And depth 10 feet.
So that's our situation.
And then it's being filled with water.
So, is being filled with water.
At 2 cubic feet per minute.
So there is our situation.
And then the question is, how fast is the water rising when it is at depth 5 feet?
So if this thing is half-full in the sense, well not half-full in terms of total volume.
But half-full in terms of height, what's the speed at which the water is rising.
So, how do we set up problems like this?
Well, we talked about this last time.
The first step is to set up a diagram and variables.
And I'm just going to draw the picture.
And I'm actually going to draw the picture twice.
So here's the conical tank.
And we have this radius, which is 4.
And we have this height, which is 10.
So that's just to allow me to think about this problem.
Now, it turns out because we have a varying depth and so on, and this is just the top.
That I'd better depict also the level at which the water actually is at present.
And furthermore, it's better to do this schematically, as you'll see.
So the key point here is to draw this triangle here.
Which shows me the 10 and shows me the 4, over here.
And then imagine that I'm filling it partway.
So maybe we'll put that water level in in another color here.
So here's the water level.
And the water level, I'm going to depict that horizontal distance here, as r. That's going to be my variable.
That's the radius of the top of the water.
So I'm taking a cross-section of this, because that geometrically the only thing I have to keep track of.
At least initially.
So this is our water level.
And it's really rotated around.
But I'm depicting just this one half-slice of the picture.
And then similarly, I have the height.
Which is this dimension there.
Or, if you like, the depth of the water.
So the water has filled us up, up to this point here.
So I set it up this way so that it's clear that we have two triangles here.
And that one piece of information we can get from the geometry is the similar triangles information.
Namely, that r / h = 4 / 10.
So this is by far the most typical geometric fact that you'll have to glean from a picture.
So that's one piece of information that we get from this picture.
Now, the second thing we have to do is set up formulas for the volume of water, and then figure out what's going on here.
So the volume of water is, of a cone.
So again, you have to know something about geometry to do many of these problems.
So you have to know that the volume of a cone is 1/3 base times height.
Now, this one is upside down.
The base is on the top and it's going down but it works the same way.
That doesn't affect volume.
So it's 1/3 and the base is pi r^2, that's the base.
And times h, which is the height.
So this is the setup for this problem.
And now, having our relationship, we have one relationship left that we have to remember.
Because we have one more piece of information in this problem.
Namely, how fast the volume is changing.
It's going at 2 cubic feet per minute.
It's increasing, so that means that dv / dt = 2.
So I've now gotten rid of all the words and I have only formulas left.
I started here with the words, I drew some pictures, and I derived some formulas.
Actually, there's one thing missing.
What's missing?
Yeah
[INAUDIBLE]
Exactly.
What you want to find.
What I left out is the question.
So the question is, what is dh / dt when h = 5?
So that's the question here.
Now, we've got the whole problem and we never have to look at it again if you like.
We just have to pay attention to this piece here.
So let's carry it out.
So what happens here.
So look, you could do this by implicit differentiation.
But it's so easy to express r as a function of h that that seems kind of foolish.
So let's write r as 2/5 h. That's coming from this first equation here.
And then let's substitute that in.
That means that v = 1/3 pi ( 2/5 h^2) h. And now I have to differentiate that.
So now I will use implicit differentiation.
It's very foolish at this point to take cube roots to solve for h. You just get yourself into a bunch of junk.
So there is a stage at which we're still using implicit differentiation here.
I'm not going to try to solve for h as a function of v. Instead I'm just differentiating straight out from this formula, which is relatively easy to differentiate.
So this is dv / dt, which of course = 2 and if I differentiate it I just get this constant, pi / 3, this other constant, (2/5)^2.
And then I have to differentiate h^3.
(h^2)h. So that's 3h^2 ( dh / dt).
That's the chain rule.
So now let's plug in all of our numbers.
Again, the other theme is, you don't plug in numbers, fixed numbers, until everything has stopped moving.
At this point, we've already calculated our rates of change.
So now I can put in the numbers.
So, 2 = pi 3 ( 2/5)^2( 3), and then h was 5, so this is 5^2.
And then I have dh / dt.
There's only one unknown thing left in this problem, which is dh / dt.
Everything else is a number.
And if you do all the cancellations, you see that this cancels this.
One of the 2's cancels - well, this cancels this.
And then one of the 2's cancels that.
So all-told what we have here is that dh / dt = 1 / 2 pi.
And so that happens to be feet per second.
This is the whole story.
Questions, way back there
[INAUDIBLE]
Where did I get -- the question is where did I get r = 2/5 h from.
The answer was, it came from similar triangles, which is over here.
I did this similar triangle thing.
And I got this relationship here.
r / h = 4/10, but then I canceled it, got 2/5 and brought the h over.
Another question, over here
[INAUDIBLE]
The question was the following.
Suppose you're at this stage, can you write from here dv / dh.
So, suppose you're here.
And work out what this is.
It's going to end up being some constant times h^2.
And then also use dv / dt = dv / dh ( dh / dt).
Which the chain rule.
And the answer is yes.
We can do that, and indeed that is what my next sentence is.
That's exactly what I'm saying.
So when I said this - sorry, when you said this, I did that.
That's exactly what I did.
This chunk is exactly dv / dh.
That's just what I'm doing.
OK.
So, definitely, that's what I had in mind.
Yeah, another question
[INAUDIBLE]
You're asking me whether my arithmetic is right or not?
[INAUDIBLE]
Pi - per second.
Oh.
This should - no, OK, right.
I guess it's per minute.
Since the other one is per minute.
Thank you.
Yes.
Was there another question?
Probably also fixing my seconds to minutes.
Way back there
I don't understand, why did you have to do all that.
Isn't the speed 2 cubic feet per minute?
The speed at which you're filling it is 2 cubic feet, but the water level is rising at a different rate, depending on whether you're low down or high up.
It depends on how wide the pond, the surface, is.
So in fact it's not.
In fact, the answer wasn't 2 cubic - it wasn't.
There's a rate there.
That is, that's how much volume is being added.
But then there's another number that we're keeping track of, which is the height.
Or, if you like, the depth of the water.
OK.
So this is the whole point about related rates.
Is you have one variable, which is v, which is volume of something.
You have another variable, which is h, which is the height of the cone of water there.
And you're keeping track of one variable in terms of the other.
And the relationship will always be a chain rule type of relationship.
So, therefore, you'll be able to - if you know one you'll be able to figure out the other.
Provided you get all of the related rates.
These are what are called related rates.
This is a rate of something with respect to something, etc.
etc.
So it's really all about the chain rule.
And just fitting it to word problems.
So there's a couple of examples.
And I'll let you work out some more.
So now, the next thing that I want to do is to give you one more max-min problem.
Which has to do with this device, which I brought with me.
So this is a ring.
Happens to be a napkin ring, and this is some parachute string, which I use when I go backpacking.
And the question is if you have a - you think of this is a weight, it's flexible.
It's allowed to move along here.
And the question is, if I fix these two ends where my two hands are, where my right hand is here and where my left hand is over there.
And the question is, where does this ring settle down.
Now, it's obvious, or should be maybe obvious, is that if my two hands are at equal heights, it should settle in the middle.
The question that we're trying to resolve is what if one hand is a little higher than the other.
What happens, or if the other way.
Where does it settle down?
So in order to depict this problem properly, I need two volunteers to help me out.
Can I have some help?
OK.
So I need one of you to hold the right side, and one of you to hold the left side.
OK. And just hold it against the blackboard.
We're going to trace.
So stick it about here, in the middle somewhere.
And now we want to make sure that this one is a little higher, alright?
So we'll have to - yeah, let's get a little higher up.
That's probably good enough.
So the experiment has been done.
We now see where this thing is.
But, so now hold on tight.
This thing stretches.
So we want to get it stretched so that we can see what it is properly.
So this thing isn't heavy enough for this demonstration.
I should've had a ten-ton brick attached there.
But if I did that, than I would tax my, right, I would tax your abilities to.
Right, so we're going to try to trace out what the possibilities are here.
So this is, roughly speaking, where this thing - and so now where does it settle.
Well, it settles about here.
So we're going to put that as x marks the spot.
OK, thank you very much.
Got to remember where those dots -- OK, all right.
Sorry, I forgot to mark the spots.
OK, so here's the situation.
We have a problem.
And we've hung a string.
And it went down like this and then it went like that.
And we discovered where it settled.
Physically.
So we want to explain this mathematically, and see what's going on with this problem.
So this is a real-life problem.
It honestly is the problem you have to solve if you want to build a bridge.
Like, a suspension bridge.
This, among many problems.
It's a very serious and important problem.
And this is the simplest one of this type.
So we've got our shape here.
This should be a straight line, maybe not quite as angled as that.
The first, we've already drawn the diagram and we've more or less visualized what's going on here.
But the first step after the diagram is to give letters.
Is to label things appropriately.
And I do not expect you to be able to do this, at this stage.
Because it requires a lot of experience.
But I'm going to do it for you.
We're going to just do that.
So the simplest thing to do is to use the coordinates of the plane.
And if you do that, it's also easiest to use the origin.
My favorite number is 0 and it should be yours, too.
So we're going to make this point be (0, 0).
Now, there's another fixed point in this problem.
And it's this point over here.
And we don't know what its coordinates are.
So we're just going to give them letters, a and b.
But those letters are going to be fixed numbers in this problem.
And we want to solve it for all possible a's and b's.
Now, the interesting thing, remember, is what happens when b Is not 0.
If b = 0, we already have a clue that the point should be the center point.
It should be exactly that x, the middle point, which I'm going to label in a second, is halfway in between.
So now the variable point that I'm going to use is down here.
I'm going to call this point (x, y).
So here's my setup.
I've now given labels to all the things on the diagram so far.
Most of the things on the diagram.
So now, what else do I have to do?
Well, I have to explain to you that this is a minimization problem.
What happens, actually, physically is that the weight settles to the lowest point.
That's the thing that has the lowest potential energy.
So we're minimizing a function.
And it's this curve here.
The constraint is that we're restricted to this curve.
So this is a constraint curve.
And we want the lowest point of this curve.
So now, we need a little bit more language in order to describe what it is that we've got.
And the constraint curve, we got it in a particular way.
Namely, we strung some string from here to there.
And what happens at all these points is that the total length of the string is the same.
So one way of expressing the constraint is that the length of the string is constant.
And so in order to figure out what the constraint is, what this curve is, I have to describe that analytically.
And I'm going to do that by drawing in some helping lines.
Namely, some right triangles to figure out what this length is.
And what the other length is.
So this length is pretty easy if I draw a right triangle here.
Because we go over x and we go down y.
So this length is the square root of x^2 + y^2.
That's the Pythagorean theorem.
Similarly, over here, I'm going to get another length.
Which is a little bit of a mess.
It's the vertical.
So I'm just going to label one half of it, so that you see.
So this horizontal distance is x.
And this horizontal distance from this top point with this right angle, over there.
It starts at x and ends at a.
The right-most point is a in the x coordinate.
So the whole distance is a - x.
So that's this leg of this right triangle.
And, similarly, the vertical distance will be b - y.
And so, the formula here, which is a little complicated for this length, is the square root of (a - x)^2 + (b - y)^2.
So here are the two formulas that are going to allow me to set up my problem now.
So, my goal is to set it up the way I did here, just with formulas.
And not with diagrams and not with names.
So here's what I'd like to do.
I claim that what's constrained, if I'm along that curve, is that the total length is constant.
So that's this statement here.
The square root of x^2 + y^2 + this other square root.
These are the two lengths of string.
Is equal to some number, L, which is constant.
And this, as I said, is what I'm calling my constraint.
Yeah
[INAUDIBLE]
So the question is, shouldn't it be b plus y.
No, and the reason is that y is a negative number.
It's below 0.
So it's actually the sum, - y is a positive number.
Alright, so here's the formula.
And then, we want to find the minimum of something.
So what is it that we're finding the minimum of?
This is actually the hardest part of the problem, conceptually.
I tried to prepare it, but it's very hard to figure this out.
We're finding the least what?
It's y.
We got a name for that.
So we want to find the lowest y.
Now, the reason why it seems a little weird is you might think of y as just being a variable.
But really, y is a function.
It's really y = y ( x ) is defined implicitly by the constraint equation.
That's what that curve is.
And notice the bottom point is exactly the place where the tangent line will be horizontal.
Which is just what we want.
So from the diagram, the bottom point is where y' = 0.
So this is the critical point.
Yeah?
[INAUDIBLE]
Exactly.
So I'm deriving for you, so the question is, could I have just tried to find y' = 0 to begin with.
The answer is yes, absolutely.
And in fact I'm leading in that direction.
I'm just showing you, so I'm trying to make the following, very subtle, point.
Which is in maximum-minimum problems, we always have to keep track of two things.
Often the interesting point is the critical point.
And that indeed turns out to be the case here.
But we always have to check the ends.
And so there are several ways of checking the ends.
One is, we did this physical problem.
We can see that it's coming up here.
We can see that it's coming up here.
Therefore the bottom must be at this critical point.
So that's OK, so that's one way of checking it.
Another way of checking it is the reasoning that I just gave.
But it's really the same reasoning.
I'm pointing to this thing and I'm showing you that the bottom is somewhere in the middle.
So, therefore, it is a place of horizontal tangency.
That's the reasoning that I'm using.
So, again, this is to avoid having to evaluate a limit of an end or to use the second derivative test, which is a total catastrophe in this case.
OK, now.
There's one other thing that you might know about this if you've seen this geometric construction before.
With a string and chalk.
Which is that this curve is an eclipse.
It turns out, this is a piece of an eclipse.
It's a huge ellipse.
These two points turn out to be the so-called foci of the ellipse.
However, that geometric fact is totally useless for solving this problem.
Completely useless.
If you actually write out the formulas for the ellipse, you'll discover that you have a much harder problem on your hands.
And it will take you twice or ten times as long, so, it's true that it's an ellipse, but it doesn't help.
OK, so what we're going to do instead is much simpler.
We're going to leave this expression along and we're just going to differentiate implicitly.
So again, we use implicit differentiation on the constraint equation.
So that's the equation which is directly above me there, at the top.
And I have to differentiate it with respect to x.
So that's a little ugly, but we've done this a few times before.
When you differentiate a square root, the square root goes into the denominator.
And there's a factor of 1/2, so there's a 2x which cancels.
So I claim it's this.
Now, because y depends on x, there's also a y y' here.
So technically speaking, it's twice this with a half.
2 / 2 times that.
So that's the differentiation of the first piece of this guy.
Now I'm going to do this second one, and it's also the chain rule.
But you're just going to have to let me do it for you.
Because it's just a little bit too long for you to pay attention to.
It turns out there's a minus sign that comes out, because there's a - x and a - y there.
And then the numerator, the denominator is the same massive square root.
So it's (a - x)^2 ( b - y)^2.
And the numerator is a - x, which is what replaces the x over here.
And then another term, which is (b - y)y'.
I claim that that's analogous to what I did in the first term.
And you'll just have to check this on your own.
Because I did it too fast for you to be able to check yourself.
Now, that's going to be equal to what on the right-hand side.
What's the derivative of L with respect to x?
It's 0.
It's not changing in the problem.
Although my parachute stuff stretches.
We tried to stretch it to its fullest extent.
So that we kept it fixed, that was the goal here.
So now, this looks like a total mess.
But, it's not.
And let me show you why.
It simplifies a great deal.
And let me show you exactly how.
So, first of all, the whole point is we're looking for the place where y' = 0.
So that means that these terms go away.
y' = 0.
So they're gone.
And now what we have is the following equation.
It's x / square root of x^2 + y^2 squared is equal to, if I put it on the other side the minus sign is changed to a plus sign.
a - x divided by this other massive object, (a - x)^2 + (b - y)^2.
So this is what it's simplifies to.
Now again, that also might look complicated to you.
But I claim that this is something, this is a kind of equilibrium equation that can be interpreted geometrically, in a way that is very meaningful and important.
So first of all, let me observe for you that this x is something on our picture.
And the square root of x^2 + y^2 is something on our picture.
Namely, if I go over to the picture, here was x and this was a right triangle.
And this hypotenuse was square root of x^2 + y^2.
So, if I call this angle alpha, then this is the sine of alpha.
Right?
It's a right triangle, that's the opposite leg.
So this guy is the sine of alpha.
Similarly, the other side has an interpretation for the other right triangle.
If this angle is beta, then the opposite side is a - x, and the hypotenuse is what was in the denominator over there.
So this side is sine of beta.
And so what this condition is telling us is that alpha = beta.
Which is the hidden symmetry in the problem.
I don't know if you can actually see it when I show you this thing.
But, no matter how I tilt it, actually the two angles from the horizontal are the same.
In the middle it's kind of obvious that that should be the case.
But on the sides it's not obvious that that's what's happening.
Now, this has even, so that's a symmetry, if you like, of the situation.
These two angles are equal.
But there's something more to be said.
If you do a forced diagram for this, what you'll discover is that the tension on the two lines is the same.
Which means that when you build something which is hanging like this, it will involve the least stress.
If you hang something very heavy, and one side carries twice as much load as the other, then you have twice as much chance of its falling and breaking.
If they each hold the same strength, then you've distributed the load in a much more balanced way.
So this is a kind of a balance condition, and it's very typical of minimization problems.
And fortunately, there are nice solutions which distribute the weight reasonably well.
That's certainly the principal of suspension bridges.
So yeah, one more question
[INAUDIBLE]
OK, so the question where it hangs, that is, what the formula for x is and what the formula for y is.
And other things, like the equation for the ellipse and lots of other stuff like that.
Those are things that I will answer for you in a set of notes which I will hand out.
And they're just a mess.
You see they're, just as in that other problem that we did last time, there's some illuminating things you can say about the problem and then there's some messy formulas.
You know, you want to try to pick out the simple things that you can say.
In fact, that's a property of math, you want to focus on the more comprehensible things.
On the other hand, it can be done.
It just takes a little bit of computation.
So I didn't answer the question of what the lowest y was.
But I'll do that for you.
Maybe I'll just mention one more thing about this problem.
I'm using problems from a completely different point of view.
If you sort of roll the ellipse around, you get the same phenomenon from each place here.
So it doesn't matter where a and 0 are.
You'll get the same phenomenon.
That is, the tangent line.
So, this angle and that angle will be equal.
So you can also read that as being the angle over here and the angle over here are equal.
That is, beta, that is the complementary angles are also equal.
And if you interpret this as a ray of light, and this as a mirror, then this would be saying that if you start at one focus, every ray of light will bounce and go to the other focus.
So that's a property that an ellipse has.
More precisely, a property that this kind of curve has.
And in fact, a few years ago there was this great piece of art at something called the DeCordova Museum, which I recommended very highly to you go sometime to visit in your four years here.
There was a collection of miniature golf holes.
So they had a bunch of mini golf pieces of art.
And every one was completely different from the other.
And one of them was called hole in one.
And the tee was at one focus of the ellipse.
And the hole was at the other focus of the ellipse.
So, no matter how you hit the golf ball, it always goes into the hole.
No matter where it bounces, it just, one bounce and it's in the hole.
So that's actually a consequence of the computation that we just did.
Time to go on.
We're going to now talk about something else.
So our next topic is Newton's method.
Which is one of the greatest applications of calculus.
And I'm going to describe it here for you.
And we'll illustrate it on an example, which is solving the equation x^2 = 5.
We're going to find the square root of 5.
Now, you can actually solve any equation this way.
Any equation that you understand, you can solve this way, essentially.
So even though I'm doing it for the square root of 5, which is something that you can figure out on your calculator, in fact this is really at the heart of many of the ways in which calculators work.
So, the first thing is to make this problem a little bit more abstract.
We're going to set f ( x ) = x^2 - 5.
And then we're going to solve f (x) = 0.
So this is the sort of standard form for solving such a.
So you take some either complicated or simple function of x, linear functions are boring, quadratic functions are already interesting.
And cubic functions, as I've said a few times, you don't even have formulas for solving.
So this would be the only method you have for solving them numerically.
So here's how it works.
So the idea, I'll plot this function.
Here's the function, it's a parabola, y = x^2 - 5.
It dips below 0, sorry, it should be centered, but anyway.
And now the idea here is to start with a guess.
And square root of 5 is pretty close to the square root of 4, which is 2.
So my first guess is going to be 2, here.
So start with initial guess.
So that's our first guess.
And now, what we're going to do is we're going to pretend that the function is linear.
That's all we're going to do.
And then if the function were linear, we're going to try to find where the 0 is.
So if the function is linear, what we'll use is we'll plot the point where 2 is on, that is, the point (2, f ( 2 )), and then we're going to draw the tangent line here.
And this is going to be our new guess.
x = x, which I'll call x1.
So the idea here is that this point may be somewhat far from where it crosses, but this point will be a little closer.
And now we're going to do this over and over again.
And see how fast it gets to the place we're aiming for.
So we have to work out what the formulas are.
And that's the strategy.
So now, the first step here is, we have our guess, we have our tangent line.
Which has the formula y - y0 = m(x - x0).
So that's the general form for a tangent line.
And now, I have to tell you what x1 is.
In terms of this tangent line.
x1 is the x intercept.
The tangent line, if you look over here at the diagram, the tangent line is the orange line.
Where that crosses the axis, that's where I want to put x1.
So that's the x intercept.
Now, how do you find the x intercept?
You find it by setting y = 0.
That horizontal line is y = 0.
So I set y = 0, and I get 0 - y0 = m (x1 - x0).
So I changed two things in this equation.
I plugged in 0 here, for y.
And I said that the place where that happens is going to be where x is x1.
So now let's solve.
And what we get here is - y divided by - sorry, -y0 / m = x1 - x0.
And now I can get a formula for x1.
So x1 = x0 - (y0 / m).
I now need to tell you what this formula means, in terms of the function f. So first of all, x0 is x0, whatever x0 is.
And y0, I claim, is f (x0).
And m is the slope at that same place.
So it's f' (x0).
And this is the whole story.
This is the formula which will enable us to calculate basically any root.
I'm going to repeat this formula, so I'm going to tell you again what Newton's method is, and put a little more colorful box around it.
So Newton's method involves the following formula.
In order to get the n + 1st point, that's our better and better guess, I'm going to take the nth one and then I'm going to plug in this formula.
So f ( xn ) / f' ( xn ).
So this is the basic formula, and if you like, this is the idea of just repeating what I had before.
Now, we've gone from geometry, from just pictures, to an honest to goodness formula which is completely implementable and very easy to implement in any case.
So let's see how it works in the case that we've got.
Which is x0 = 2. f(x) = x^2 - 5.
Let's see how to implement this formula.
So first of all, I have to calculate for you, f' (x).
That's 2x.
And so, x1 = x0 -, so f' , sorry, f ( x ) would be x0^2 - 5.
That's what's in the numerator.
And in the denominator I have the derivative, so that's 2 x0.
And so all told, well, let's work it out in two steps here.
This is - 1/2 x0 for the first term, and then + (5/2 / x0) for the second term.
And these two terms combine, so we have here 1/2 x0 + 5/2 with an x0 in the denominator.
So here's the formula for x1.
in this case.
Now I'd like to show you how well this works.
So first of all, we have x1 = 1/2 ( 2), if I plug in x1, + 5/4, which is 9/4.
And x2, I have 1/2 ( 9/4) + 5/2 ( 4/9).
That's the next one.
And if you work this out, it's 161/72.
And then x3 is kind of long.
But I will just write down what it is, so that you can see it's 1/2 ( 161/72) + 5/2 and then I do the reciprocal of that.
So 72/161.
So let's see how good these are.
I carefully calculated how far off they are.
Somewhere on my notes.
So I'll just take a look and see what I said.
Oh yeah, I did do it.
So, what's the square root of 5 minus - so here's n, here's the square root of 5 - xn, if you like.
Or the other way around.
The size of this.
You'll have to decide on your homework whether it comes out positive or negative, to the right or to the left of the answer.
But let's do this.
So when n = 0, the guess was 2.
And we're off by about 2 ( 10 ^ - 1).
And if n = 1, so that's this 9/4, that's off by about 10 ^ - 2.
And then n = 2, that's this number here, right?
And that's off by about 4 ( 10 ^ - 5).
That's already as good an approximation to the square root of 5 as you'll ever need in your life.
If you do 3, this number here turns out to be accurate to 10 ^ - 10 or so.
This goes right off to the edge of my calculator, this one here.
So already, with the third iterate.
you're of way past the accuracy that you need for most things.
Yep, question
[INAUDIBLE]
How come the x0 disappears?
[INAUDIBLE]
So, from here to here [INAUDIBLE]
[INAUDIBLE]
Hang on, folks, we have a question.
Let's just answer it.
So here we have an x0, and here we have - 1/2, there's an x0^2 and an x which cancel.
And here we have a minus, and a - 5/2 x0.
So I combine the 1 - 1/2, I got + 1/2, that's all.
OK?
Alright, thanks.
We'll have to ask other questions after class.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
What we're going to talk about today is a continuation of last time.
I want to review Newton's method because I want to talk to you about its accuracy.
So if you remember, the way Newton's method works is this.
If you have a curve and you want to know whether it crosses the axis.
And you don't know where this point is, this point which I'll call x here, what you do is you take a guess.
Maybe you take a point x0 here.
And then you go down to this point on the graph.
and you draw the tangent line.
I'll draw these in a couple of different colors so that you can see the difference between them.
So here's a tangent line.
It's coming out like that.
And that one is going to get a little closer to our target point.
But now the trick is, and this is rather hard to see because the scale gets small incredibly fast, is that if you go right up from that, and you do this same trick over again.
That is, this is your second guess, x1, and now you draw the second tangent line.
Which is going to come down this way.
That's really close.
You can see here on the chalkboard, it's practically the same as the dot of x.
So that's the next guess.
Which is x2.
And I want to analyze, now, how close it gets.
And just describe to you how it works.
So let me just remind you of the formulas, too.
It's worth having them in your head.
So the formula for the next one is this.
And then the idea is just to repeat this process.
Which has a fancy name, which is in algorithms, which is to iterate, if you like.
So we repeat the process.
And that means, for example, we generate x2 from x1 by the same formula.
And we did this last time.
And, more generally, the n + 1st is generated from the nth guess, by this formula here.
So what I'd like to do is just draw the picture of one step a little bit more closely.
So I want to blow up the picture, which is above me there.
That's a little too high.
Where are my erasers?
Got to get it a little lower than that, since I'm going to depict everything above the line here.
So here's my curve coming down.
And suppose that x1 is here, so this is directly above it is this point here.
And then as I drew it, this green tangent coming down like that.
It's a little bit closer, and this was the place, x2, and then here was x, our target, which is where the curve crosses as opposed to the straight tangent line crossing.
So that's the picture that I want you to keep in mind.
And now, we're just going to do a very qualitative kind of error analysis.
So here's our error analysis.
And we're starting out, the distance between x1 and x is what we want to measure.
In other words, how close we are to where we're heading.
And so I've called that, I'm going to call that Error 1.
That's x - x1.
In absolute value.
And then, E2 would be x - x2, in absolute value.
And so forth.
And, last time, when I was estimating the size of this, so En would be whatever it was.
Last time, remember, we did it for a specific case.
So last time, I actually wrote down the numbers.
And they were these numbers, maybe you could call them En, which was the absolute value of square root of 5 - xn.
These are the sizes that I was writing down last time.
And I just want to talk about in general what to expect.
That worked amazingly well, and I want to show you that that's true pretty much in general.
So the first distance, again, is E1, is this distance here.
That's the E1.
And then this shorter distance, here, this little bit, which I'll mark maybe in green, is E2.
So how much shorter is E1 than E2?
Well, the idea is pretty simple.
It's that if this distance in this vertical, distances are probably about the same as the perpendicular distance.
And this is basically the situation of a curve touching a tangent line.
Then the separation is going to be quadratic.
And that's basically what's going to happen.
So, in other words the distance E2 is going to be about the square of the distance E1.
And that's really the only feature of this that I want to point out.
So, approximately, this is the situation that we're going to get.
And so what that means is, and maybe thinking from last time, what we had was something roughly like this.
You have an E0, you have an E1, you have an E2, you have an E3, and so forth.
Maybe I'll write down E4 here.
And last time, this was about 10 ^ - 1.
So the expectation based on this rule is that the next error's the square of the previous one.
So that's 10 ^ - 2.
The next one is the square of the previous one.
So that's 10 ^ - 4.
And the next one is the square of that, that's 10 ^ - 8.
And this one is 10 ^ - 16.
So the thing that's impressive about this list of numbers is you can see that the number of digits of accuracy doubles at each stage.
Accuracy doubles at each step.
The number of digits of accuracy doubles at each step.
So, very, very quickly you get past the accuracy of your calculator, as you saw on your problem set.
And this thing works beautifully.
So, let me just summarize by saying that Newton's method works very well.
By which I mean this kind of rate.
And I want to be just slightly specific.
If, there's really two conditions disguised in this, that are going on.
One is that f' has to be, not to be.
To be not small.
And f'' has to be not too big.
That's roughly speaking what's going on.
i'll explain these in just a second.
And x0 starts nearby.
Nearby the target value x.
So that's really what's going on here.
So let me just illustrate to you.
So I'm not going to explain this, except to say the reason why this second derivative gets involved is that it's how curved the curve is, that how far away you get.
If the second derivative were 0, that would be the best possible case.
Then we would get it on the nose.
If the second derivative is not too big, that means the quadratic part is not too big.
So we don't get away very far from the green line to the curve itself.
The other thing to say is, as I said, that x0 needs to start nearby.
So I'll explain that by explaining what maybe could go wrong.
So the ways the method can fail, and one example which actually would have happened in our example from last time, which was y = x^2 - 5, is suppose we'd started x0 over here.
Then this thing would have gone off to the left, and we would have landed on not the square root of 5 but the other root.
So if it's too far away, then we get the wrong root.
So that's an example, explaining that the x0 needs to start near the root that we're talking about.
Otherwise, the method doesn't know which root you're asking for.
It only knows where you started.
So it may go off to the wrong place.
OK, it can't read your mind.
Yes, question
[INAUDIBLE]
Oh, good question.
So the question was, what if the first error is larger than 1.
Are you in trouble?
And the answer is, absolutely, yes.
If you have quadratic behavior, you can see.
If you have a quadratic nearby, it's pretty close to the straight line.
But far away, a parabola is miles from a straight line.
It's way, way, way far away.
So if you're foolish enough to start over here, you may have some trouble making progress.
Actually, it isn't - when I, that little wiggle there just meant proportional to.
In fact, in the particular case of a parabola, it manages to get back.
It saves itself.
But there's no guarantee of that sort of thing.
You really do want to start reasonably close.
Yep
[INAUDIBLE]
What you have to do is you have to watch out.
That is, it's hard to know what assumptions to make about x0.
You plug it into the machine and you see what you get.
And either it works or it doesn't.
You can tell that it's marching toward a specific place, and you can tell that that place probably is a 0, usually.
But maybe it's not the one you were looking for.
So in other words, you have to use your head.
You run the program and then you see what it does.
And if you're lucky - the problem is, if you have no idea where the 0 is, you may just wander around forever.
As we'll see in a second.
So the next example here is the following.
I said that f' has to be not too small.
There's a real catastrophe hiding just inside this picture.
Which is the transition between when you find the positive root and when you find the negative root here.
Which is, if you're right down here.
If you were foolish enough to get 0, then what's going to happen is your tangent line is horizontal.
It doesn't even meet the axis.
So in the formula, you can see that's a catastrophe.
Because there's an f' in the denominator.
So that's 0.
That's undefined.
It's not surprising, it's consistent that the parallel line doesn't meet the axis.
And you have no x1.
So you had, so if you like, another point here is that f' = 0 is a disaster.
A disaster for the method.
Because the next, so say, if f (x0) = 0, then x1 is undefined.
And finally, there's one other weird thing that can happen.
Which is, which I'll just draw a picture of schematically.
Which you can get from a wiggle.
So this wiggle has three roots.
The way I've drawn it.
And it can be that you can start over here with your x0.
And draw your tangent line and go over here to an x1.
And then that tangent line will take you right back to the x0.
I didn't draw it quite right, but that's about right.
So it goes over like this.
So let me draw the two tangent lines, so that you can see it properly.
Sorry, I messed it up.
So here are the two tangent lines.
This guy and this guy.
And it just goes back and forth.
x0 cycles to x1, and x1 goes back to x0.
We have a cycle.
And it never goes anywhere.
This is, the grass is always greener.
It's over here, it thinks, oh, I really would prefer to go to this 0 and then it thinks oh, I want to go back.
And it goes back and forth, and back and forth.
Grass is always greener on the other side of the fence.
Never, never gets anywhere.
So those are the sorts of things that can go wrong with Newton's method.
Nevertheless, it's fantastic.
It works very well, in a lot of situations.
And solves basically any equation that you can imagine, numerically.
Next we're going to move on.
We're going to move on to something which is a little theoretical.
Which is the mean value theorem.
And that will allow us in just a day or so to launch into the ideas of integration, which is the whole second half of the course.
So let's get started with that.
The mean value theorem will henceforth be abbreviated MVT.
So I don't have to write quite as much every time I refer to it.
The mean values theorem, colloquially, says the following.
If you go from Boston to LA, which I think a lot of Red Sox fans are going to want to do soon, so that's 3,000 miles.
In 6 hours, then at some time you are going at a certain speed.
The average of this speed.
Average, so speed, which in this case is what?
So we're going at the average speed.
That's 3,000 miles times 6 hours, so that's 500 miles per hour.
Exactly.
So sometime on your journey, of course, some of the time you're going more than 500 miles an hour, sometimes you are going less.
And some time you must've been going 500 miles an hour exactly.
That's the mean value theorem.
The reason why it's called mean value theorem is that word mean is the same as the word average.
So now I'm going to state it in math symbols, the same theorem.
And it's a formula.
It says that the difference quotient, so this is the distance traveled divided by the time elapsed.
That's the average speed, is equal to the infinitesimal speed for some time in between.
So some c, which is in between a and b. I'm not quite done.
It's a real theorem, it has hypotheses.
I've told you the conclusion first, but there are some hypotheses, they're straightforward hypotheses.
Provided f is differentiable; that is, it has a derivative in the interval a < x < b.
And continuous in a < or = x, less than or equal to.
There has to be a sense that you can make out of the speed, or the rate of change of f at each intermediate point.
And in order for the values at the ends to make sense, it has to be continuous.
There has to be a link between the values at the ends and what's going on in between.
If it were discontinuous, there would be no relation between the left and right values and the rest of the function.
So here's the theorem, conclusion and its hypothesis.
And it means what I said more colloquially up above.
Now, I'm going to prove this theorem immediately.
At least, give a geometric intuitive argument, which is not very different from the one that's given in a very systematic treatment.
So here's the proof of the mean value theorem.
It's really just a picture.
So here's a place, and here's another place on the graph.
And the graph is going along like this, let's say.
And this line here is the secant line.
So this is (a, f ( a )) down here.
And this is (b, f ( b )) up there.
And this segment is the secant, its slope is the slope that we're aiming for.
The slope of that line is the left-hand side of this formula here.
So we need to find something with that slope.
And what we need to find is a tangent line with that slope, because what's on the right-hand side is the slip of a tangent line.
So here's how we construct it.
We take a parallel line, down here.
And then we just translate it up, leaving it parallel, we move it up.
Towards this one.
Until it touches.
And where it touches, at this point of tangency, down there, I've just found my value of c. And you can see that if the tangent line is parallel to this line, that's exactly the equation we want.
So this thing has slope f' (c).
And this other one has slope equal to this complicated expression, f ( b) - f (a) / (b - a).
That is almost the end of the proof.
There's one problem.
So, again, we move a parallel line up.
Move up the parallel line until it touches.
There's a little subtlety here, which I just want to emphasize.
Which is that that dotted line keeps on going here.
But when we bring it up, we're going to ignore what's happening outside of a.
And beyond b, alright?
So we're just going to ignore the rest of the graph.
But there is one thing that can go wrong.
So if it does not touch, then the picture looks likes this.
Here are the same two points.
And the graph is all above.
And we brought up our thing.
And it went like that.
So we didn't construct a tangent line.
If this happens.
So we're in trouble, in that point.
In this situation, sorry.
But there's a trick, which is a straightforward trick.
Then bring the tangent lines down from the top.
So parallel lines, sorry, not tangent lines.
Parallel lines.
From above.
So, that's the whole story.
That's how we cook up this point c, with the right properties.
I want to point out just one more theoretical thing.
And then the rest, we're going to be drawing conclusions.
So there's one more theoretical remark about the proof, which is something that is fairly important to understand.
When you understand a proof, you should always be thinking about why the hypotheses are necessary.
Where do I use the hypothesis.
And I want to give you an example where the proof doesn't work to show you that the hypothesis is an important one.
So the example is the following.
I'll just take a function which is two straight lines like this.
And if you try to perform this trick with these things, then it's going to come up and it's going to touch here.
But the problem is that the tangent line is not defined here.
There are lots of tangents, and there's no derivative at this point.
So the derivative doesn't exist here.
So this is the claim that one bad point ruins the proof.
We need f' to exist at all so, f' ( x ) to exist at all x in between.
Can't get away even with one defective point.
Now it's time to draw some consequences.
And the main consequence is going to have to do with applications to graphing.
But we'll see tomorrow and for the rest of the course that this is even more significance.
It's significant to all the rest of Calculus.
I'm going to list three consequences which you're quite familiar with already.
So, the first one is if f' is positive, then f is increasing.
And the second one is if f' is negative, then f is decreasing.
And the last one seems like the simplest.
But even this one alone is the key to everything.
If f' = 0, then f is constant.
These are three consequences, now, of the mean value theorem.
And let me show you how they're proved.
I just told you that they were true, maybe a while ago.
And certainly I mentioned the first two.
The last one was so simple that we maybe just swept it under the rug.
You did use it on a problem set, once or twice.
But it turns out that this actually requires proof, and we're going to give the proof right now.
The way that the proof goes is simply to write down, to rewrite star.
Rewrite our formula.
Which says that f (b) - f (a) / (b - a) = f' (c).
And you see I've written it from left to right here to say that the right-hand side information about the derivative is going to be giving the information about the function.
That's the way I'm going to read it.
In order to express this, though, I'm going to just rewrite it a couple of times here.
So here's f ( a ), multiplying through by the denominator.
And now I'm going to write it in another customary form for the mean value theorem.
Which is f ( b ) = f ( a ) + f' (c) ( b - a).
So here's another version.
I should probably have put this one in the box to begin with anyway.
And, just changing it around algebraically, it's this fact here.
They're the same thing.
And now with the formula written in this form, I claim that I can check these three facts.
Let's start with the first one.
I'm going to set things up always so that a < b.
And that's the setup of the theorem.
And so that means that b - a is positive.
Which means that this factor over here is a positive number.
If f' is positive, which is what happens in the first case, that's the assumption that we're making, then this is a positive number.
And so f( b ) > f( a ).
Which means that it's increasing.
It goes up as the value goes up.
Similarly, if f' (c) is negative, then this is a positive times a negative number, this is negative.
f ( b ) < f(a).
So it goes the other way.
Maybe I'll write this way.
And finally, if f' (c) = 0, then f ( b ) = f(a).
Which if you apply it to all possible ends means if you can do it for every interval, which you can't, then that means that f is constant.
It never gets to change values.
Well you might have believed these facts already.
But I just want to emphasize to you that this turns out to be the one key link between infinitesimals, between limits and these actual differences.
Before, we were saying that the difference quotient was approximately equal to the derivative.
Now we're saying that it's exactly equal to a derivative.
Although we don't know exactly which point to use.
It's some point in between.
I'm going to be deducing some other consequences in a second, but let me stop for second to make sure that everybody's on board.
Especially since I've finished the blackboards here.
Before we, everybody happy?
One question
[INAUDIBLE]
I'm just going to repeat your question first.
I'm a little bit confused, you said, about what guarantees that there's a point of tangency.
That's what you said.
So do you want to elaborate, or do you want to want to stop with what you just send?
What is it that confuses you?
[INAUDIBLE]
Yeah
[INAUDIBLE]
So I'm not claiming that there's only one point.
This could wiggle a lot of times and it maybe touches at ten places.
In other words, it's OK with me if it touches more than once.
Then I just have more, the more the merrier.
In other words, I don't want there necessarily only to be one.
It could come down like this.
And touch a second time.
Is that what was concerning you?
So in mathematics, when we claim that this is true for some point, we don't necessarily mean that it doesn't work for others.
In fact, if the function is constant, this is 0 and in fact this equation is true for every c. That satisfies your question?
The fact that this point exists actually is a touchy point.
I just convinced you of it visually.
It's a geometric issue, whether you're allowed to do this.
Indeed, it has to do with the existence of tangent lines and more analysis then we can do in this class.
Yeah.
Another question
[INAUDIBLE]
Pardon me
[INAUDIBLE]
The question is, what's the difference between this and the linear approximation.
And I think, let me see if I can describe that.
I'll leave the theorem on the board.
I'm going to get rid of the colloquial version of the theorem.
And I'll try to describe to you the difference between this and the linear approximation.
I was planning to do that in a while, but we'll do it right now since that's what you're asking.
That's fine.
So here's the situation.
The linear approximation, so let's say comparison with linear approximation.
They're very closely related.
The linear approximation says the change in f over the change in x, that's the left-hand side of this thing, is approximately f' (a).
For b near a, and b - a = delta x.
This statement, which is in the box, which is sitting right up there, is the statement that this change in f is actually equal to something.
Not approximately equal to it.
It's equal to f' of some c. And the problem here is that we don't know exactly which c. This is for some c. Between a and b.
Right, so.
That's the difference between the two.
And let me elaborate a little bit.
If you're trying to understand what f ( b ) - f ( a ) / (b - a) is, the mean value theorem is telling you for sure that it's equal to this f' (c).
So that means it's less than or equal to the largest possible value on the largest value you can get, for sure.
And this is on the whole interval.
And I'm going to include the ends, because when you take a max it's sometimes achieved at the ends.
And similarly, because it's f' (c), it's definitely bigger than the min on this same interval here.
This is all you can say based on the mean value theorem.
All you know is this.
And colloquially, what that means is that the average speed is between the maximum and the minimum.
Not very surprising.
The mean value theorem is supposed to be very intuitively obvious.
It's saying the average speed is trapped between the maximum speed and the minimum speed.
For sure, that's something, that's why, incidentally this wasn't really proved when Newton and Leibniz were around.
But, let's write this so that you can read it.
Average speed is between the max and the min.
But nobody had any trouble, they didn't disbelieve it because it's a very natural thing.
Now if, for example, I take any kind of linear approximation; say, for instance, e ^ x is approximately 1 + x.
Then I'm making the guess, now, don't want to say this yet.
That's not going to explain it to you well enough.
What we're saying, so this is the mean value here.
This is what the mean value theorem says.
And here's the linear approximation.
The linear approximation is saying that the average speed is approximately the initial speed, or possibly the final speed.
So if a is the left endpoint, then it's the initial speed.
If it happens to be the right endpoint, if the value of x is to the left then it's the final speed.
So those are the - so you can see it's approximately right.
Because the speed, when you're on a short interval, shouldn't be varying very much.
The max and the min should be pretty close together.
So that's why the linear approximation is reasonable.
And this is telling you absolutely, it's no less than the min and no more than the max.
Yeah
[INAUDIBLE]
The little kink?
[INAUDIBLE]
If you approach from the top.
So if it's still under here I can show you it again.
Oh yeah, it's still there.
Good
[INAUDIBLE]
Oh, the one with the wiggle on top?
Yeah, this one you can't.
Because there's nothing to touch and it also fails from the bottom because there's this bad point.
From the top, it could work.
It can certainly work both ways.
So, for example.
See if you're a machine, you maybe don't have a way of doing this.
But if you're a human being you can spot all the places.
There are a bunch of spots where the slope is right.
And it's perfectly OK. All of them work
[INAUDIBLE]
It's not that the c is the same.
It's just we've now found one, two, three, four, five c's for which it works
[INAUDIBLE]
If you're asked to find a c, so first of all that's kind of a phony question.
There are some questions on your problem set which ask you to find a c. That actually is struggling to get you to understand what the statement of the mean value theorem is, but you should not pay a lot of attention to those questions.
They're not very impressive.
But, of course, you would have to find all the - if it asked you to find one, you find one.
If you can find some more, fine.
You can pick whichever one you want.
Mean value theorem just doesn't care.
The mean value theorem doesn't care because actually, the mean value theorem is never used except in real life, except in this context here.
You can never nail down which c it is, so the only thing you can say is that you're going slower than the maximum speed and faster than the minimum speed.
Sorry, say that again?
[INAUDIBLE]
If you're asked for a specific c, you have to find a specific c. And it has to be in the range.
In between, it has to be in here.
So now I want to tell you about another kind of application, which is really just a consequence of what I've described here.
I should emphasize, by the way, this, probably, should be doing this.
I guess we've never used this color here.
This popular.
This is pink.
So this one is so good.
So since we're going to do this.
So the reason why the exclamation points are temporary, this is such an obvious fact.
But this is the way that you're going to want to use the mean value theorem, and this is the only way you need to understand the mean value theorem.
On your test, or ever in your whole life.
So this is the way it will be used.
As I will make very clear when we review for the exam.
In practice what happens is you even forget about the mean value theorem, and what you remember is these three properties here.
Which are themselves consequences of the mean value theorem.
So these are the ones that I want to illustrate now.
In my next discussion here.
I'm just going to talk about inequalities.
inequalities are relationships between functions.
And I'm going to prove a couple of them using the properties over there, the properties that functions with positive derivatives are increasing.
Here's an example.
e ^ x > 1 + x, where x > 0.
The proof is the following.
I consider, so here's a proof.
I consider the function f ( x ), which is the difference.
e ^ x - (1 + x).
I observe that it starts at f ( 0 ) equal to, well, that's e ^ 0 - (1 + 0), which is 0.
And, it keeps on going.
f' (x) = e ^ x.
If I differentiate here, the 1 goes away.
I get - 1.
That's the derivative of the function.
And this function, because e ^ x > 1, for x positive is positive.
As x gets bigger and bigger, this rate of increase is positive.
And therefore, three dots, that's therefore, f ( x ) is bigger than its starting place.
For x > 0.
If it's increasing, then that's, in particular, it's increasing starting from 0.
So this is true.
Now, all I have to do is read what this inequality says.
And what it says is that e ^ x, just plug in for f ( x ), which is right here.
-( 1 + x) is greater than the starting value, which was 0.
Now, I put the thing that's negative on the other side.
So that's the same thing as e^x > 1 + x.
That's a typical inequality.
And now, we'll use this principle again.
Oh gee, I erased the wrong thing.
I erased the statement and not the proof.
Well, hide the proof.
The next thing I want to prove to you is that e ^ x > 1 + x + (x^2 / 2).
So, how do I do that?
I introduce a function g (x), which is e^x minus this.
And now, I'm just going to do exactly the same thing I did before.
Which is, I get started with g ( 0 ).
Which is 1 - 1.
Which is 0.
And g' ( x ) is e ^ x minus - now, look at what happens when I differentiate this.
The 1 goes away.
The x gives me a 1, and the x^2 / 2 gives me a + x.
And this one is positive for x > 0, because of step 1.
Because of the previous one that I did.
So this one is increasing.
g is increasing.
Which says that g ( x ) > g ( 0 ).
And if you just read that off, it's exactly the same as our inequality here.
e^x > 1 + x + (x^2 / 2).
Now, you can keep on going with this essentially forever.
And let me just write down what you get.
You get e ^ x > 1 + x + (x^2 / 2).
The next one turns out to be (x^3 / 3 * 2) + (x^4 / 4 * 3 * 2).
And you can do whatever you want.
You can do others.
And this is like the tortoise and the hare.
This is the tortoise, and this is the hare, it's always ahead.
But eventually, if you go infinitely far, it catches up.
So this turns out to be exactly equal to e ^ x in the limit.
And we'll talk about that maybe at the end of the course.
The following content is provided under a Creative Commons license.
your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're moving on from theoretical things from the mean value theorem to the introduction to what's going to occupy us for the whole rest of the course, which is integration.
So, in order to introduce that subject, I need to introduce for you a new notation, which is called differentials.
I'm going to tell you what a differential is, and we'll get used to using it over time.
If you have a function which is y = f ( x), then the differential of y is going to be denoted dy, and it's by definition f' ( x ) dx.
So here's the notation.
And because y is really equal to f, sometimes we also call it the differential of f. It's also called the differential of f. That's the notation, and it's the same thing as what happens if you formally just take this dx, act like it's a number and divide it into dy.
So it means the same thing as this statement here.
And this is more or less the Leibniz, not Leibniz, interpretation of derivatives.
Of a derivative as a ratio of these so called differentials.
It's a ratio of what are known as infinitesimals.
Now, this is kind of a vague notion, this little bit here being an infinitesimal.
It's sort of like an infinitely small quantity.
And Leibniz perfected the idea of dealing with these intuitively.
And subsequently, mathematicians use them all the time.
They're way more effective than the notation that Newton used.
You might think that notations are a small matter, but they allow you to think much faster, sometimes.
When you have the right names and the right symbols for everything.
And in this case it made it very big difference.
Leibniz's notation was adopted on the Continent and Newton dominated in Britain and, as a result, the British fell behind by one or two hundred years in the development of calculus.
It was really a serious matter.
So it's really well worth your while to get used to this idea of ratios.
And it comes up all over the place, both in this class and also in multivariable calculus.
It's used in many contexts.
So first of all, just to go a little bit easy.
We'll illustrate it by its use in linear approximations, which we've already done.
The picture here, which we've drawn a number of times, is that you have some function.
And here's a value of the function.
And it's coming up like that.
So here's our function.
And we go forward a little increment to a place which is dx further along.
The idea of this notation is that dx is going to replace the symbol delta x, which is the change in x.
And we won't think too hard about - well, this is a small quantity, this is a small quantity, we're not going to think too hard about what that means.
Now, similarly, if you see how much we've gone up - well, this is kind of low, so it's a small bit here.
So this distance here is, previously we called it delta y.
But now we're just going to call it dy.
So dy replaces delta y.
So this is the change in level of the function.
And we'll represent it symbolically this way.
Very frequently, this just saves a little bit of notation.
For the purposes of this, we'll be doing the same things we did with delta x and delta y, but this is the way that Leibniz thought of it.
And he would just have drawn it with this.
So this distance here is dx and this distance here is dy.
So for an example of linear approximation, we'll say what's 64.1, say, to the 1/3 power, approximately equal to?
Now, I'm going to carry this out in this new notation here.
The function involved is x ^ 1/3.
And then it's a differential, dy.
Now, I want to use this rule to get used to it.
Because this is what we're going to be doing all of today is, we're differentiatating, or taking the differential of y.
So that is going to be just the derivative.
That's 1/3 x ^ - 2/3 dx.
And now I'm just going to fill in exactly what this is.
At x = 64, which is the natural place close by where it's easy to do the evaluations, we have y = 64 ^ 1/3, which is just 4.
And how about dy?
Well, so this is a little bit more complicated.
Put it over here.
So dy = 1/3 ( 64) ^ - 2/3 dx.
And that is (1/3 ) 1/16 dx, which is 1/48 dx.
And now I'm going to work out what 64 to the, whatever it is here, this strange fraction.
I just want to be very careful to explain to you one more thing.
Which is that we're using x = 64, and so we're thinking of x + dx is going to be 64.1.
So that means that dx is going to be 1/10.
So that's the increment that we're interested in.
And now I can carry out the approximation.
The approximation says that 64.1 ^ 1/3 is, well, it's approximately what I'm going to call y + dy.
Because really, the dy that I'm determining here is determined by this linear relation.
dy = 1/48 dx.
And so this is only approximately true.
Because what's really true is that this = y + delta y.
In our previous notation.
So this is in disguise.
What this is equal to.
And that's the only approximately equal to what the linear approximation would give you.
So, really, even though I wrote dy is this increment here, what it really is if dx is exactly that, is it's the amount it would go up if you went straight up the tangent line.
So I'm not going to do that because that's not what people write.
And that's not even what they think.
They're really thinking of both dx and dy as being infinitesimally small.
And here we're going to the finite level and doing it.
So this is just something you have to live with, is a little ambiguity in this notation.
This is the approximation.
And now I can just calculate these numbers here.
y at this value is 4.
And dy, as I said, is 1/48 dx.
And that turns out to be 4 + 1/480, because dx is 1/10.
So that's approximately 4.002.
And that's our approximation.
Now, let's just compare it to our previous notation.
This will serve as a review of, if you like, of linear approximation.
But what I want to emphasize is that these things are supposed to be the same.
Just that it's really the same thing.
It's just a different notation for the same thing.
I remind you the basic formula for linear approximation is that f ( x ) is approximately f ( a) + f' ( a )( x - a).
And we're applying it in the situation that a = 64 and f(x) = x ^ 1/3.
And so f ( a ), which is f ( 64 ) is of course 4.
And f' ( a ), which is a, (1/3)a ^ - 2/3, is in our case 1/16.
No, 1/48.
OK, that's the same calculation as before.
And then our relationship becomes x ^ 1/3 is approximately equal to 4 + 1/48 ( x - a ), which is 64.
So look, every single number that I've written over here has a corresponding number for this other method.
And now if I plug in the value we happen to want, which is the 64.1, this would be 4 + 1/48 ( 1/10 ), which is just the same thing we had before.
So again, same answer.
Same method, new notation.
Well, now I get to use this notation in a novel way.
So again, here's the notation.
This notation of differential.
The way I'm going to use it is in discussing something called antiderivative Again, this is a new notation now.
But it's also a new idea.
It's one that we haven't discussed yet.
Namely, the notation that I want to describe here is what's called the integral of g ( x ) dx.
And I'll denote that by a function capital G (x).
So it's, you start with a function g ( x ) and you produce a function capital G ( x ), which is called the antiderivative of G. Notice there's a differential sitting in here.
This symbol, this guy here, is called an integral sign.
Or an integral, or this whole thing is called an integral.
And another name for the antiderivative of g is the indefinite integral of g. And I'll explain to you why it's indefinite in just, very shortly here.
Well, so let's carry out some examples.
Basically what I'd like to do is as many examples along the lines of all the derivatives that we derived at the beginning of the course.
In other words, in principle you want to be able to integrate as many things as possible.
We're going to start out with the integral of sine x dx.
That's a function whose derivative is sine x.
So what function would that be?
Cosine x, minus, right.
It's - cos x.
So - cos x differentiated gives you sine x.
So that is an antiderivative of sine.
And it satisfies this property.
So this function, capital G ( x ) = - cos x, has the property that its derivative is sine x.
On the other hand, if you differentiate a constant, you get 0.
So this answer is what's called indefinite.
Because you can also add any constant here.
And the same thing will be true.
So, c is constant.
And as I said, the integral is called indefinite.
So that's an explanation for this modifier in front of the integral.
It's indefinite because we actually didn't specify a single function.
We don't get a single answer.
Whenever you take the antiderivative of something it's ambiguous up to a constant.
Next, let's do some other standard functions from our repertoire.
We have an integral of (x ^ a)dx.
Some power, the integral of a power.
And if you think about it, what you should be differentiating is one power larger than that.
But then you have to divide by 1 / a + 1, in order that the differentiatiation be correct.
So this just is the fact that d / dx of x ^ a + 1, or maybe I should even say it this way.
Maybe I'll do it in differential notation.
d ( x ^ a + 1) = (a + 1) (x ^ a) dx.
So if I divide that through by a + 1, then I get the relation above.
And because this is ambiguous up to a constant, it could be any additional constant added to that function.
Now, the identity that I wrote down below is correct.
But this one is not always correct What's the exception?
Yeah.
a equals
0
Negative 1.
So this one is OK for all a.
But this one fails because we've divided by 0 when a = - 1.
So this is only true when a is not equal to - 1.
And in fact, of course, what's happening when a = 0, you're getting 0 when you differentiate the constant.
So there's a third case that we have to carry out.
Which is the exceptional case, namely the integral of dx/x.
And this time, if we just think back to what are - so what we're doing is thinking backwards here, which a very important thing to do in math at all stages.
We got all of our formulas, now we're reading them backwards.
And so this one, you may remember, is ln x.
The reason why I want to do this carefully and slowly now, is right now I also want to write the more standard form which is presented.
So first of all, first we have to add a constant.
And please don't put the parentheses here.
The parentheses go there.
But there's another formula hiding in the woodwork here behind this one.
Which is that you can also get the correct formula when x is negative.
And that turns out to be this one here.
So I'm treating the case, x positive, as being something that you know.
But let's check the case, x negative.
In order to check the case x negative, I have to differentiate the logarithm of the absolute value of x in that case.
And that's the same thing, again, for x negative as the derivative of the lograrithm of negative x.
That's the formula, when x is negative.
And if you carry that out, what you get, maybe I'll put this over here, is, well, it's the chain rule.
It's (1 / -x) d/dx(-x).
So see that there are two minus signs.
There's a - x in the denominator and then there's the derivative of - x in the numerator.
That's just - 1.
This part is - 1.
So this negative 1 over negative x, which is 1 / x.
So the negative signs cancel.
If you just keep track of this in terms of ln negative x and its graph, that's a function that looks like this.
For x negative.
And its derivative is 1 / x, I claim.
And if you just look at it a little bit carefully, you see that the slope is always negative.
Right?
So here the slope is negative.
So it's going to be below the axis.
And, in fact, it's getting steeper and steeper negative as we go down.
And it's getting less and less negative as we go horizontally.
So it's going like this, which is indeed the graph of this function, for x negative.
Again, x negative.
So that's one other standard formula.
And very quickly, very often, we won't put the absolute value signs.
We'll only consider the case x positive here.
But I just want you to have the tools to do it in case we want to use, we want to handle, both positive and negative x.
Now, let's do two more examples.
The integral of sec^2 x dx.
These are supposed to get you to remember all of your differentiatation formulas, the standard ones.
So this one, integral of sec^2 dx is what?
Tan x.
And here we have + c, alright?
And then the last one of, a couple of, this type would be, let's see.
I should do at least this one here, square root of 1 - x ^2.
This is another notation, by the way, which is perfectly acceptable.
Notice I've put the dx in the numerator and the function in the denominator here.
So this one turns out to be sin-1 x.
And, finally, let's see.
About the integral of dx / 1 + x ^2.
That one is tan -1 x.
For a little while, because you're reading these things backwards and forwards, you'll find this happens to you on exams.
It gets slightly worse for a little while.
You will antidifferentiate when you meant to differentiate.
And you'll differentiate when you're meant to antidifferentiate.
Don't get too frustrated by that.
But eventually, you'll get them squared away.
And it actually helps to do a lot of practice with antidifferentiations, or integrations, as they're sometimes called.
Because that will solidify your remembering all of the differentiation formulas.
So, last bit of information that I want to emphasize before we go on some more complicated examples is this It's obvious because the derivative of a constant is 0.
That the antiderivative is ambiguous up to a constant.
But it's very important to realize that this is the only ambiguity that there is.
So the last thing that I want to tell you about is uniqueness of antiderivatives up to a constant.
The theorem is the following.
The theorem is if F' = F', then F = G. So F ( x ) = G ( x) + c. But that means, not only that these are antiderivatives, all these things with these + c's are antiderivatives.
But these are the only ones.
Which is very reassuring.
And that's a kind of uniqueness, although its uniqueness up to a constant, it's acceptable to us.
Now, the proof of this is very quick.
But this is a fundamental fact.
The proof is the following.
If F' = G', then if you take the difference between the two functions, its derivative, which of course is F' - G' = 0.
Hence, F( x) - G (x) is a constant.
Now, this is a key fact.
Very important fact.
We deduced it last time from the mean value theorem.
It's not a small matter.
It's a very, very important thing.
It's the basis for calculus.
It's the reason why calculus make sense.
If we didn't have the fact that the derivative is 0 implied that the function was constant, we would be done.
We would have, calculus would be just useless for us.
The point is, the rate of change is supposed to determine the function up to this starting value.
So this conclusion is very important.
And we already checked it last time, this conclusion.
And now just by algebra, I can rearrange this to say that f ( x) = G ( x) + c. Now, maybe I should leave differentials up here.
Because I want to illustrate.
So let's go on to some trickier, slightly trickier, integrals.
Here's an example.
The integral of, say, x^3 ( x ^ 4 + 2) ^ 5 dx.
This is a function which you actually do know how to integrate, because we already have a formula for all powers.
Namely, the integral of x ^ a is equal to this.
And even if it were a negative power, we could do it.
So it's OK. On the other hand, to expand the 5th power here is quite a mess.
And this is just a very, very bad idea.
There's another trick for doing this that evaluates this much more efficiently.
And it's the only device that we're going to learn now for integrating.
Integration actually is much harder than differentiation.
Symbolically.
It's quite difficult.
And occasionally impossible.
And so we have to go about it gently.
But for the purposes of this unit, we're only going to use one method.
Which is very good.
That means whenever you see an integral, either you'll be able to divine immediately what the answer is, or you'll use this method.
So this is it.
The trick is called the method of substitution.
And it is tailor-made for notion of differentials.
So tailor-made.
for differential notation.
The idea is the following.
I'm going to to define a new function.
And it's the messiest function that I see here.
It's u = x ^ 4 + 2.
And then, I'm going to take its differential and what I discover, if I look at its formula, is and the rule for differentials, which is right here.
Its formula is what?
4x^3 dx.
Now, lo and behold with these two quantities, I can substitute, I can plug in to this integral.
And I will simplify it considerably.
So how does that happen?
Well, this integral is the same thing as, well, really I should combine it the other way.
So let me move this over.
So there are two pieces here.
And this one is u ^ 5.
And this one is 1/4 du.
Now, that makes it the integral of (u ^ 5 du) / 4.
And that's relatively easy to integrate.
That is just a power.
So let's see.
It's just 1/20 u to the - not 1/20.
The antiderivative of u ^ 5 is u ^ 6.
With the 1/6, so it's 1/24 u ^ 6 + c. Now, that's not the answer to the question.
It's almost the answer to the question.
Why isn't it the answer?
It isn't the answer because now the answer's expressed in terms of u.
Whereas the problem was posed in terms of this variable x.
So we must change back to our variable here.
And we do that just by writing it in.
So it's 1/24 (x ^ 4 + 2) ^ 6 + c. And this is the end of the problem.
Yeah, question
[INAUDIBLE]
The question is, can you see it directly?
Yeah.
And we're going to talk about that in just one second.
OK. Now, I'm going to do one more example and illustrate this method.
Here's another example.
The integral of x dx / squre root of 1 + x ^2.
Now, here's another example.
Now, the method of substitution leads us to the idea u = 1 + x ^2.
du = 2x dx, etc.
It takes about as long as this other problem did.
To figure out what's going on.
It's a very similar sort of thing.
You end up integrating u ^ - 1/2.
It needs to the integral of u ^ - 1/2 du.
Is everybody seeing where this..?
However, there is a slightly better method.
So recommended method.
And I call this method advanced guessing.
What advanced guessing means is that you've done enough of these problems that you can see two steps ahead.
And you know what's going to happen.
So the advanced guessing leads you to believe that here you had a power - 1/2, here you have the differential of the thing.
So it's going to work out somehow.
And the advanced guessing allows you to guess that the answer should be something like this.
(1 + x ^2) ^ 1/2.
So this is your advanced guess.
And now you just differentiate it, and see whether it works.
Well, here it is.
It's 1/2 (1 + x ^2) ^ - 1/2( 2x), that's the chain rule here.
Which, sure enough, gives you x / square root of 1 + x ^2.
So we're done.
And so the answer is square root of (1 + x^2) + c. Let me illustrate this further with another example.
I strongly recommend that you do this, but you have to get used to it.
So here's another example.
e ^ 6x dx.
My advanced guess is e ^ 6x.
And if I check, when I differentiate it, I get 6e ^ 6x.
That's the derivative.
And so I know that the answer, so now I know what the answer is.
It's 1/6 e ^ 6x + c. Now, OK, you could, it's also OK, but slow, to use a substitution, to use u = 6x.
Then you're going to get du = 6dx ...
It's going to work, it's just a waste of time.
Well, I'm going to give you a couple more examples.
So how about this one.
x ( e^ - x^2) dx.
What's the guess?
Anybody have a guess?
Well, you could also correct.
So I don't want you to bother - yeah, go ahead
[INAUDIBLE]
Yeah, so you're already one step ahead of me.
Because this is too easy.
When they get more complicated, you just want to make this guess here.
So various people have said 1/2, and they understand that there's 1/2 going here.
But let me just show you what happens, OK?
If you make this guess and you differentiate it, what you get here is e^ - x ^2 times the derivative of negative 2x, so that's - 2x.
- x^2, so it's - 2x.
So now you see that you're off by a factor of not 2, but - 2.
So a number of you were saying that.
So the answer is - 1/2 e^ - x ^2 + c. And I can guarantee you, having watched this on various problems, that people who don't write this out make arithmetic mistakes.
In other words, there is a limit to how much people can think ahead and guess correctly.
Another way of doing it, by the way, is simply to write this thing in and then fix the coefficient by doing the differentiation here.
That's perfectly OK as well.
Alright, one more example.
We're going to integrate sin x cos x dx.
So what's a good guess for this one?
[INAUDIBLE]
Someone suggesting sine ^2 x.
So let's try that.
Over 2 - well, we'll get the coefficient in just a second.
So sine ^2 x, if I differentiate I get 2 sine x cosine x.
So that's off by a factor of 2.
So the answer is 1/2 sine ^2 x.
But now I want to point out to you that there's another way of doing this problem.
It's also true that if you differentiate cosine ^2 x, you get 2 cos x ( - sine x).
So another answer is that the integral of sin x cos x dx = - 1/2 cos^2 x + c. So what is going on here?
What's the problem with this?
[INAUDIBLE]
Pardon me?
[INAUDIBLE]
Integrals aren't unique.
That's part of the - but somehow these two answers still have to be the same
[INAUDIBLE]
OK. What do you think?
If you add them together, you just get c
If you add them together you get c. Well, actually, that's almost right.
That's not what you want to do, though.
You don't want to add them.
You want to subtract them.
So let's see what happens when you subtract them.
I'm going to ignore the c, for the time being.
I get sin^2 x, 1/2 sin^2 x - (-1/2 cos^2 x).
So the difference between them, we hope to be 0.
But actually of course it's not 0.
What it is, is it's 1/2 (sin^2 + cos^2) which is 1/2.
It's not 0, it's a constant.
So what's really going on here is that these two formulas are the same.
But you have to understand how to interpret them.
The two constants, here's a constant up here.
There's a constant, c1 associated to this one.
There's a different constant, c2 associated to this one.
And this family of functions for all possible c1s and all possible c2s, is the same family of functions.
Now, what's the relationship between c1 and c2?
Well, if you do the subtraction, c1 - c2 has to be equal to 1/2.
They're both constants, but they differ by 1/2.
So this explains, when you're dealing with families of things, they don't have to look the same.
And there are lots of trig functions which look a little different.
So there can be several formulas that actually are the same.
And it's hard to check that they're actually the same.
You need some trig identities to do it.
Let's do one more example here.
Here's another one.
Now, you may be thinking, and a lot of people are, thinking ugh, it's got a ln in it.
If you're experienced, you actually can read off the answer just the way there were several people who were shouting out the answers when we were doing the rest of these problems.
But, you do need to relax.
Because in this case, now this is definitely not true in general when we do integrals.
But, for now, when we do integrals, they'll all be manageable.
And there's only one method.
Which is substitution.
And in the substitution method, you want to go for the trickiest part.
And substitute for that.
So the substitution that I proposed to you is that this should be, you should be ln x.
And the advantage that that has is that its differential is simpler then itself.
So du = dx /x.
Remember, we use that in logarithmic differentiation, too.
So now we can express this using this substitution.
And what we get is, the integral of, so I'll divide the two parts here.
It's 1 / ln x, and then it's dx / x.
And this part is 1 / u, and this part is du.
So it's the integral of du / u.
And that is ln u + c. Which altogether, if I put back in what u is, is ln (ln x) + c. And now we see some uglier things.
In fact, technically speaking, we could take the absolute value here.
And then this would be absolute values there.
So this is the type of example where I really would recommend that you actually use the substitution, at least for now.
Alright, tomorrow we're going to be doing differential equations.
And we're going to review for the test.
I'm going to give you a handout telling you just exactly what's going to be on the test.
So, see you tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And this last little bit is something which is not yet on the Web.
But, anyway, when I was walking out of the room last time, I noticed that I'd written down the wrong formula for c1 - c1.
There's a misprint, there's a minus sign that's wrong.
I claimed last time that c1 - c2 was + 1/2.
But, actually, it's - 1/2.
If you go through the calculation that we did with the antiderivative of sine x cosine x, we get these two possible answers.
And if they're to be equal, then if we just subtract them we get c1 - c2 + 1/2 = 0.
So c1 - c2 = 1/2.
So, those are all of the correction.
Again, everything here will be on the Web.
But just wanted to make it all clear to you.
So here we are.
This is our last day of the second unit, Applications of Differentiation.
And I have one of the most fun topics to introduce to you.
Which is differential equations.
Now, we have a whole course on differential equations, which is called 18.03.
And so we're only going to do just a little bit.
But I'm going to teach you one technique.
Which fits in precisely with what we've been doing already.
Which is differentials.
The first and simplest kind of differential equation dy/dx = some function, f (x).
Now, that's a perfectly good differential equation.
And we already discussed last time that the solution; that is, the function y, is going to be the antiderivative, or the integral, of x.
Now, for the purposes of today, we're going to consider this problem to be solved.
That is, you can always do this.
You can always take antiderivatives.
And for our purposes now, that is for now, we only have one technique to find antiderivatives.
And that's called substitution.
It has a very small variant, which we called advanced guessing.
And that works just as well.
And that's basically all that you'll ever need to do.
As a practical matter, these are the ones you'll face for now.
Ones that you can actually see what the answer is, or you'll have to make a substitution.
Now, the first tricky example, or the first maybe interesting example of a differential equation, which I'll call Example 2, is going to be the following.
(d / dx + x)y = 0.
So that's our first differential equation that were going to try to solve.
Apart from this standard antiderivative approach.
This operation here has a name.
This actually has a name, it's called the annihilation operator.
And it's called that in quantum mechanics.
And there's a corresponding creation operator where you change the sign from plus to minus.
And this is one of the simplest differential equations.
The reason why it's studied in quantum mechanics all it that it has very simple solutions that you can just write out.
So we're going to solve this equation.
It's the one that governs the ground state of the harmonic oscillator.
So it has a lot of fancy words associated with it, but it's a fairly simple differential equation and it works perfectly by the method that we're going to propose.
So the first step in this solution is just to rewrite the equation by putting one of the terms on the right-hand side.
So this is dy / dx = - x y.
Now, here is where you see the difference between this type of equation and the previous type.
In the previous equation, we just had a function of x on the right-hand side.
But here, the rate of change depends on both x and y.
So it's not clear at all that we can solve this kind of equation.
But there is a remarkable trick which works very well in this case.
Which is to use multiplication.
To use this idea of differential that we talked about last time.
Namely, we divide by y and multiply by dx.
So now we've separated the equation.
We've separated out the differentials.
And what's going to be important for us is that the left-hand side is expressed solely in terms of y and the right-hand side is expressed solely in terms of x.
And we'll go through this in careful detail.
So now, the idea is if you've set up the equation in terms of differentials as opposed to ratios of differentials, or rates of change, now I can use Leibniz's notation and integrate these differentials.
Take their antiderivatives.
And we know what each of these is.
Namely, the left-hand side is just - ah.
well, that's tough.
OK.
I had an au pair who actually did a lot of Tae Kwan Do.
She could definitely defeat any of you in any encounter, I promise.
OK.
Anyway.
So, let's go back.
We want to take the antiderivative of this.
So remember, this is the function whose derivative is 1 / y.
And now there's a slight novelty here.
Here we're differentiating the variables x, and here we're differentiating the variable as y.
So the antiderivative here is ln y.
And the antiderivative on the other side is - x^2 / 2.
And they differ by a constant.
So we have this relationship here.
Now, that's almost the end of the story.
We have to exponentiate to express y in terms of x.
So, e ^ ln y = e ^ -x^2 / 2 + c. And now I can rewrite that as y = I'll write as A e ^ - x^2 / 2, where a = e ^c.
And incidentally, we're just taking the case y positive here.
We'll talk about what happens when y is negative in a few minutes.
So here's the answer to the question, almost, except for this fact that I picked out y positive.
Really, the solution is y = ae ^ - x^2 / 2.
Any constant, a; a positive, negative, or 0.
Any constant will do.
And we should double-check that to make sure.
If you take dy / d x right, that's going to be a d/ dx ( e^ -x ^2 / 2).
And now by the chain rule, you can see that this is a times the factor of - x, that's the derivative of the exponent, with respect to x times the exponential.
And now you just rearrange that.
That's - x y.
So it does check.
These are solutions to the question.
The a didn't matter.
It didn't matter whether it was positive or negative.
This function is known as the normal distribution, so it fits beautifully with a lot of probability and probabilistic interpretation of quantum mechanics.
This is sort of the, where the particle is.
So next, what I'd like to do is just go through the method in general and point out when it works.
And then I'll make a few comments just to make sure that you understand the technicalities of dealing with constants and so forth.
So, first of all, the general method of separation of variables.
And here's when it works.
It works when you're faced with a differential equation of the form f (x) g( y).
That's the situation that we had.
And I'll just illustrate that.
Just to remind you here.
Here's our equation.
It's in that form.
And the function f (x) = - x, and the function g( y) is just y.
And now, the way the method works is, this separation step.
From here to here, this is the key step.
This is the only conceptually remarkable step, which all has to do with the fact that Leibniz fixed his notations up so that this works perfectly.
And so that involves taking the y, so dividing by g( y), and multiplying by dx, it's comfortable because it feels like ordinary arithmetic, even though these are differentials.
And then, we just antidifferentiate.
So we have a function, h, which is the integral of dy/ g( y), and we have another function which is F. Note they are functions of completely different variables here.
Integral of f ( x) dx.
Now, in our example we did that.
We carried out this antidifferentiation, and this function turned out to be ln y, and this function turned out to be - x^2 / 2.
And then we write the relationship.
Which is that if these are both antiderivatives of the same thing, then they have to differ by a constant.
Or, in other words, H ( y) has to equal to F(x) + c. Where c is constant.
Now, notice that this kind of equation is what we call an implicit equation.
It's not quite a formula for y, directly.
It defines y implicitly.
That's that top line up here.
That's the implicit equation.
In order to make it an explicit equation, which is what is underneath, what I have to do is take the inverse.
So I write it as y = = H -1( F ( x) + c).
Now, in real life the calculus part is often pretty easy.
And it can be quite messy to do the inverse operation.
So sometimes we just leave it alone in the implicit form.
But it's also satisfying, sometimes, to write it in the final form here.
Now I've got to give you a few little pieces of commentary before.
For those of you walked in a little bit late, this will all be on the Web.
So just a few pieces of commentary.
So if you like, some remarks.
The first remark is that I could have written ln | y| = - x ^2 / 2 + c. We learned last time that the antiderivative works also for the negative values.
So this would work for y not equal to 0.
Both for positive and negative values.
And you can see that that would have captured most of the rest of the solution.
Namely, | y| would be = A e ^ - x^2 / 2, by the same reasoning as before.
And then that would mean that y = plus or minus A e^ - x^2 / 2, which is really just what we got.
Because, in fact, I didn't bother with this.
Because actually in most - and the reason why going through this, by the way, carefully this time, is that you're going to be faced with this very frequently.
The exponential function comes up all the time.
And so, therefore, you want to be completely comfortable dealing with it.
So this time I had the positive a, while the negative a fits in either this way, or I can throw it in.
Because I know that that's going to work that way.
But of course, I double-checked to be confident.
Now, this still leaves out one value.
So, this still leaves out.
So, if you like, what I have here now is a is equal to plus or minus capital A.
The capital A1 being positive 1.
But this still leaves out one case.
Which is y = 0.
Which is an extremely boring solution, but nevertheless a solution to this problem.
If you plug in 0 here for y, you get 0.
If you plug in 0 here for y, you get that these two sides are equal.
0 = 0.
Not a very interesting answer to the question.
But it's still an answer.
And so y = 0 is left out.. Well, that's not so surprising that we missed that solution.
Because in the process of carrying out these operations, I divided by y. I did that right here.
So, that's what happens.
If you're going to do various non-linear operations; in particular, if you're going to divide by something, if it happens to be 0 you're going to miss that solution.
You might have problems with that solution.
But we have to live with that because we want to get ahead.
And we want to get the formulas for various solutions.
So that's the first remark that I wanted to make.
And now, the second one is almost related to what I was just discussing right here.
That I'm erasing.
And that's the following.
I could have also written ln y + c1 = - x^2 / 2 + c2.
Where c1 and c2 are different constants.
When I'm faced with this antidifferentiation, I just taught you last time, that you want to have an arbitrary constant.
Here and there, in both slots.
So I perfectly well could have written this down.
But notice that I can rewrite this as ln y = - x ^2 / 2 + c2 - c1.
I can subtract.
And then, if I just combine these two guys together and name them c, I have a different constant.
In other words, it's superfluous and redundant to have two arbitrary constants here, because they can always be combined into one.
So two constants are superfluous.
Can always be combined.
So we just never do it this first way.
It's just extra writing, it's a waste of time.
There's one other subtle remark, which you won't actually appreciate until you've done several problems in this direction.
Which is that the constant appears additive here, in this first solution to the problem.
But when I do this nonlinear operation of exponentiation, it now becomes multiplicative constant.
And so, in general, there's a free constant somewhere in the problem.
But it's not always an additive constant.
It's only an additive constant right at the first step when you take the antiderivative.
And then after that, when you do all your other nonlinear operations, it can turn into anything at all.
So you should always expect it to be something slightly more interesting than an additive constant.
Although occasionally it stays an additive constant.
The last little bit of commentary that I want to make just goes back to the original problem here.
Which is right here.
The example 1.
And I want to solve it, even though this is simpleminded.
But example 1 via separation.
So that you see our variables.
So that you see what it does.
The situation is this.
And the separation just means you put the dx on the other side.
So this is dy = = f(x) dx.
And then we integrate.
And the antiderivative of dy is just y.
So this is the solution to the problem.
And it's just what we wrote before; it's just a funny notation.
And it comes to the same thing as the antiderivative.
OK, so now we're going to go on to a trickier problem.
A trickier example.
We need one or two more just to get some practice with this method.
Everybody happy so far?
Question
[INAUDIBLE]
So, the question is, how do we deal with this ambiguity.
I'm summarizing very, very, briefly what I heard.
Well, you know, sometimes a > 0, sometimes a < 0, sometimes it's not.
So there's a name for this guy.
Which is that this is what's called the general solution.
In other words, the whole family of solutions is the answer to the question.
Now, it could be that you're given extra information.
If you're given extra information, that might be, and this is very typical in such problems, you have the rate of change of the function, which is what we've given.
But you might also have the place where it starts.
Which would be, say, it starts at 3.
Now, if you have that extra piece of information, then you can nail down exactly which function it is.
If you do that, if you plug in 3, you see that ae ^ - 0^2 / 2 = 3.
So a = 3.
And the answer is y = 3e ^ -x^2 / 2.
And similarly, if it's negative, if it starts out negative, it'll stay negative.
For instance.
If it starts out 0, it'll stay 0 this particular function here.
So the answer to your question is how you deal with the ambiguity.
The answer is that you simply say what the solution is.
And the solution is not one function, it's a family of functions.
It's a list and you have to have what's known as a parameter.
And that parameter gets nailed down if you tell me more information about the function.
Not the rate of change, but something about the values of the function
[INAUDIBLE]
The general solution is this solution
[INAUDIBLE]
And I'm showing you here that you could get to most of the general solution.
There's one thing that's left out, namely the case a = 0.
So, in other words, I would not go through this method.
I would only use this, which is simpler.
But then I have to understand that I haven't gotten all of the solutions this way.
I'm going to need to throw in all the rest of the solutions.
So in the back of your head, you always have to have something like this in mind.
So that you can generate all the solutions.
This is very suggestive, right?
The restriction, it turns that the restriction A > 0 is superfluous, is unnecessary.
But that, we only get by further thought and by checking.
Another question?
Over here
[INAUDIBLE]
The aim of differential equations is to solve them.
Just as with algebraic equations.
Usually, differential equations are telling you something about the balance between an acceleration and a velocity.
If you have a falling object, it might have a resistance.
It's telling you something.
So, actually, sometimes in applied problems, formulating what differential equation describe this situation is very important.
In order to see that that's the right thing, you have to have solved it to see that it fits the data that you're getting
[INAUDIBLE]
The question is, can you solve for x instead of y.
The answer is, sure.
That's the same thing as - so that would be the inverse function of the function that we're officially looking for.
But yeah, it's legal.
In other words, oftentimes we're stuck with just the implicit, some implicit formula and sometimes we're stuck with a formula x is a function of y versus y as a function of x.
The way in which the function is specified is something that can be complicated.
As you'll see in the next example, it's not necessarily the best thing to think about a function, y as a function of x.
Well, in the fourth example.
Alright, we're going to go on and do our next example here.
So the third example is going to be taken as a kind of geometry problem.
I'll draw a picture of it.
Suppose you have a curve with the following property.
If you take a point on the curve, and you take the ray, you take the ray from the origin to the curve, well, that's not going to be one that I want.
I think I'm going to want something which is steeper.
Because what I'm going to insist is that the tangent line be twice as steep as the ray from the origin.
So, in other words, slope of tangent line equals twice slope of ray from origin.
So the slope of this orange line is twice the slope of the pink line.
Now, these kinds of geometric problems can be written very succinctly with differential equations.
Namely, it's just the following.
dy / dx, that's the slope of the tangent line, is equal to, well remember what the slope of this ray is, if this point, I need a notation.
At this point is (x, y) which is a point on the curve.
So the slope of this pink line is what?
[INAUDIBLE]
y / x.
So if it's twice it, there's the equation.
OK, now, we only have one method for solving these equations.
So let's use it.
It says to separate variables.
So I write dy / y here = 2 dx / x.
That's the basic separation.
That's the procedure that we're always going to use.
And now if I integrate that, I find that on the right-hand side I have the logarithm of y.
And - sorry, on the left-hand side I have the logarithm of y.
On the right-hand side, I have twice the logarithm of x, plus a constant.
So let's see what happens to this example.
This is an implicit equation, and of course we have the problems of the plus or minus signs, which I'm not going to worry about until later.
So let's exponentiate and see what happens.
We get e ^ ln y = e ^ 2 ln x + c. So, again, this is y on the left-hand side.
And on the right-hand side, if you think about it for a second, it's (e ^ ln x)^2.
Which is x^2.
So this is x ^2, and then there's an e^ c. So that's another one of these A factors here.
A = e^ c. So the answer is, well, I'll draw the picture.
And I'm going to cheat as I did before.
We skipped the case y negative.
We really only did the case y positive, so far.
But if you think about it for a second, and we'll check it in a second, you're going to get all of these parabolas here.
So the solution is this family of functions.
And they can be bending down.
As well as up.
So these are the solutions to this equation.
Every single one of these curves has the property that if you pick a point on it, the tangent line has twice the slope of the ray to the origin.
And the formula, if you like, of the general solution is y = ax^2, a is any constant.
Question?
[INAUDIBLE]
Yeah.
So again - so first of all, so there are two approaches to this.
One is to check it, and make sure that it's right.
When a formula works for some family of values, sometimes it works for others.
But another one is to realize that these things will usually work out this way.
Because in this argument here, I allow the absolute value.
And that would have been a perfectly legal thing for me to do.
I could have put in absolute values here.
In which case, I would've gotten that the absolute value of this was equal to that.
And now you see I've covered the plus and minus cases.
So it's that same idea.
This implies that y = either + A x^2 or - Ax^, depending on which sign you pick.
So that allows me for the curves above and curves below.
Because it's really true that the antiderivative here is the function.
It's defined for y negative.
So let's just double-check.
In this case, what's happening, we have y = ax^2 and we want to compute dy / dx to make sure that it satisfies the equation that I started out with.
And what I see here is that this is 2ax.
And now I'm going to write this in a suggestive way.
I'm going to write it as 2ax^2 / x.
And, sure enough, this is 2y / x.
It does not matter whether a, it works for a positive, a negative, a = 0.
It's OK. Again, we didn't pick up by this method the a = 0 case.
And that's not surprising because we divided by y.
There's another thing to watch out about, about this example.
So there's another warning.
Which I have to give you.
And this is a subtlety which you definitely won't get to in any detail until you get to a higher level ordinary differential equations course, but I do want to warn you about it right now.
Which is that if you look at the equation, you need to watch out that it's undefined at x = 0.
It's undefined at x = 0.
We also divided by x, and x is also a problem.
Now, that actually has an important consequence.
Which is that, strangely, knowing the value here and knowing the rate of change doesn't specify this function.
This is bad.
And it violates one of our pieces of intuition.
And what's going wrong is that the rate of change was not specified.
It's undefined at x = 0.
So there's a problem here, and in fact if you think carefully about what this function is doing, it could come in on one branch and leave on a completely different branch.
It doesn't really have to obey any rule across x = 0.
So you should really be thinking of these things as rays emanating from the origin.
The origin was a special point in the whole geometric problem.
Rather than as being complete parabolas.
But that's a very subtle point.
I don't expect you to be able to say anything about it.
But I just want to warn you that it really is true that when x = 0 there's a problem for this differential equation.
So now, let me say our next problem.
Next example.
Just another geometry question.
So here's Example 4.
I'm just going to use the example that we've already got.
Because there's only so much time left here.
The fourth example is to take the curves perpendicular to the parabolas.
This is another geometry problem.
And by specifying that the the curves are perpendicular to these parabolas, I'm telling you what their slope is.
So let's think about that.
What's the new equation?
The new diff.
eq.
is the following.
It's that the slope is equal to the negative reciprocal of the slope of the tangent line.
Of tangent to the parabola.
So that's the equation.
That's actually fairly easy to write down, because it's - 1 / 2(y / x).
That's the slope of the parabola.
2y / x.
So let's rewrite that.
Now, this is the x goes in the numerator, so it's - x / 2y.
And now I want to solve this one.
Well, again, there's only one technique.
Which is we're going to separate variables.
And we separate the differentials here, so we get 2y dy = - x dx.
That's just looking at the equation that I have, which is right over here.
dy / dx = - x / 2y, and cross-multiplying to get this.
And now I can take the antiderivative.
This is y^2.
And the antiderivative over here is - x^2 / 2 + c. And so, the solutions are x^2 / 2 + y^2 = some c. Some constant c. Now, this time, things don't work the same.
And you can't expect them always to work the same.
I claimed that this must be true.
But unfortunately I cannot insist that every c will work.
As you can see here, only the positive numbers c can work here.
So the picture is that something slightly different happened here.
And you have to live with this.
Is that sometimes not all the constants will work.
Because there's more to the problem than just the antidifferentiation.
And here there are fewer answers rather than more answers.
In the other case we had to add in some answers, here we have to take them away.
Some of them don't make any sense.
And the only ones we can get are the ones which are of this form, where this is, say, some radius squared.
Well maybe I shouldn't call it a radius.
I'll just call it a parameter, a^2.
And these are of course ellipses.
And you can see that the ellipses, the length here is the square root of 2a and the semi-axis, vertical semi-axis, is a.
So this is the kind of ellipse that we've got.
And I draw it on the previous diagram, I think it's somewhat suggestive here.
There, ellipses are kind of eggs.
They're a little bit longer than they are high.
And they go like this.
And if I drew them pretty much right, they should be making right angles.
At all of these places.
OK, last little bit here.
Again, you've got to be very careful with these solutions.
And so there's a warning here too.
So let's take a look at, this is the implicit solution to the equation.
And this is the one that tells us what the shape is.
But we can also have the explicit solution.
And if I solve for the explicit solution, it's either y = + square root of a^2 - x^2 / 2, or y = - square root of a^2 - x^2 / 2.
These are the explicit solutions.
And now, we notice something that we should have noticed before.
Which is that an ellipse is not a function.
It's only the top half, if you like, that's giving you a solution to this equation.
Or maybe the bottom half that's giving it the solution to the equation.
So the one over here, this one is the top halves.
And this one over here is the bottom halves.
And there's something else that's interesting.
Which is that we have a problem at y = 0. y = 0 is where x = square root of 2a.
That's when we get to this end here.
And what happens is the solution comes around and it stops.
It has a vertical slope.
Vertical slope.
And the solution stops.
But really, that's not so unreasonable.
After all, look at the formula.
There was a y in the denominator here.
When y = 0, the slope should be infinite.
And so this equation is just giving us what it geometrically and intuitively should be giving us.
At that stage.
So that is the introduction to ordinary differential equations.
Again, there's only one technique which is - we're not done yet, we have a whole four minutes left and we're going to review.
Now, so fortunately, this review is very short.
Fortunately for you, I handed out to you exactly what you're going to be covering on the test.
And it's what's printed here but there's a whole two pages of discussion.
And I want to give you very, very clear-cut instructions here.
This is usually the hardest test of this course.
People usually do terribly on it.
And I'm going to try to stop that by making it a little bit easier.
And now here's what we're going to do.
I'm telling you exactly what type of problems are going to be on the test.
These are these six.
It's also written on your sheet, your handout.
It's also just what was asked on last year's test.
You should go and you should look at last year's test and see what types of problems they are.
I really, really, am going to ask the same questions, or the same type of questions.
Not the same questions.
So that's what's going to happen on the test.
And let me just tell you, say one thing, which is the main theme of the class.
And I will open up.
We'll have time for one question after that.
The main theme of this unit is simply the following.
That information about derivative and sometimes maybe the second derivative, tells us information about f itself.
And that's just what were doing here with ordinary differential equations.
And that was what we were doing way at the beginning when we did approximations.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going on to the third unit here.
So we're getting started with Unit 3.
And this is our intro to integration.
It's basically the second half of calculus after differentiation.
Today what I'll talk about is what are known as definite integrals.
Actually, it looks like, are we missing a bunch of overhead lights?
Is there a reason for that?
Hmm.
Let's see.
Ahh.
Alright.
OK, that's a little brighter now.
Alright.
So the idea of definite integrals can be presented in a number of ways.
But I will be consistent with the rest of the presentation in the course.
We're going to start with the geometric point of view.
And the geometric point of view is, the problem we want to solve us to find the area under a curve.
The other point of view that one can take, and we'll mention that at the end of this lecture, is the idea of a cumulative sum.
So keep that in mind that there's a lot going on here.
And there are many different interpretations of what the integral is.
Now, so let's draw a picture here.
I'll start at a place a and end at a place b.
And I have some curve here.
And what I have in mind is to find this area here.
And, of course, in order to do that, I need more information than just where we start and where we end.
I also need the bottom and the top.
By convention, the bottom is the x axis and the top is the curve that we've specified, which is y = f(x).
And we have a notation for this, which is the notation using calculus for this as opposed to some geometric notation.
And that's the following expression.
It's called an integral, but now it's going to have what are known as limits on it.
It will start at a and end at b.
And we write in the function f(x) dx.
So this is what's known as a definite integral.
And it's interpreted geometrically as the area under the curve.
The only difference between this collection of symbols and what we had before with indefinite integrals is that before we didn't specify where it started and where it ended.
Now, in order to understand what to do with this guy, I'm going to just describe very abstractly what we do.
And then carry out one example in detail.
So, to compute this area, we're going to follow initially three steps.
First of all, we're going to divide into rectangles.
And unfortunately, because it's impossible to divide a curvy region into rectangles, we're going to cheat.
So they're only quote-unquote rectangles.
They're almost rectangles.
And the second thing we're going to do is to add up the areas.
And the third thing we're going to do is to rectify this problem that we didn't actually hit the answer on the nose.
That we were missing some pieces or were choosing some extra bits.
And the way we'll rectify that is by taking the limit as the rectangles get thin.
Infinitesimally thin, very thin.
Pictorially, again, that looks like this.
We have a and our b, and we have our guy here, this is our curve.
And I'm going to chop it up.
First I'm going to chop up the x axis into little increments.
And then I'm going to chop things up here.
And I'll decide on some rectangle, maybe some staircase pattern here.
Like this.
Now, I don't care so much.
In some cases the rectangles overshoot; in some cases they're underneath.
So the new area that I'm adding up is off.
It's not quite the same as the area under the curve.
It's this region here.
But it includes these extra bits here.
And then it's missing this little guy here.
This little bit there is missing.
And, as I say, these little pieces up here, this a little bit up here is extra.
So that's why we're not really dividing up the region into rectangles.
We're just taking rectangles.
And then the idea is that as these get thinner and thinner, the little itty bitty amounts that we miss by are going to 0.
And they're going to be negligible.
Already, you can see it's kind of a thin piece of area, so we're not missing by much.
And as these get thinner and thinner, the problem goes away and we get the answer on the nose in the limit.
So here's our first example.
I'll take the first interesting curve, which is f ( x) = x^2.
I don't want to do anything more complicated than one example, because this is a real labor here, what we're going to go through.
And to make things easier for myself, I'm going to start at a = 0.
But in order to see what the pattern is, I'm going to allow b to be arbitrary.
Let's draw the graph and start breaking things up.
So here's the parabola, and there's this piece that we want, which is going to stop at this place, b, here.
And the first step is to divide into n pieces.
That means, well, graphically, I'll just mark the first three.
And maybe there are going to be many of them.
And then I'll draw some rectangles here, and I'm going to choose to make the rectangles all the way from the right.
That is, I'll make us this staircase pattern here, like this.
That's my choice.
I get to choose whatever level I want, and I'm going to choose the right ends as the shape of the staircase.
So I'm overshooting with each rectangle.
And now I have to write down formulas for what these areas are.
Now, there's one big advantage that rectangles have.
And this is the starting place.
Which is that it's easy to find their areas.
All you need to know is the base and the height, and you multiply, and you get the area.
That's the reason why we can get started with rectangles.
And in this case, these distances, I'm assuming that they're all equal, equally spaced, intervals.
And I'll always be doing that.
And so the spacing, the bases, the base length, is always b / n. All equal intervals.
So that's the base length.
And next, I need the heights.
And in order to keep track of the heights, I'm going to draw a little table here, with x and f ( x), and plug in a few values just to see what the pattern is.
The first place here, after 0, is b / n. So here's b / n, that's an x value.
And the f ( x) value is the height there.
And that's just, I value it f(x), f)x) = x^2.
And that's (b / n)^2.
And similarly, the next one is 2b / n. And the value here is (2b / n^2.
That's this.
This height here is 2b / n. That's the second rectangle.
And I'll write down one more.
3b / n, that's the third one.
And the height is (3b / n^2.
And so forth.
Well, my next job is to add up these areas.
And I've already prepared that by finding out what the base and the height is.
So the total area, or the sum of the areas, let's say, of these rectangles, is - well, the first one is (b / n) ( b / n)^2.
The second one is 2b / n - I'm sorry, is (b / n)( 2b / n)^2.
And it just keeps on going.
And the last one is (b / n)( nb / n)^2.
So it's very important to figure out what the general formula is.
And here we have a base.
And here we have a height, and here we have the same kind of base, but we have a new height.
And so forth.
And the pattern is that the coefficient here is 1, then 2, then 3, all the way up to n. The rectangles are getting taller and taller, and this one, the last one is the biggest.
OK, this is a very complicated gadget.
and the first thing I want to do is simplify it and then I'm actually going to evaluate it.
But actually I'm not going to evaluate it exactly.
I'm just going to evaluate the limit.
Turns out, limits are always easier.
The point about calculus here is that these rectangles are hard.
But the limiting value is an easy value.
So what we're heading for is the simple formula, as opposed to the complicated one.
Alright, so the first thing I'm going to do is factor out all these b / n factors.
There's a b / n here, and there's a (b / n)^2.
So all told, we have a (b / n)^3.
As a common factor.
And then the first term is 1, and the second term, what's left over, is 2^2.
2^2.
And then the third term would be 3^2, although I haven't written it.
In the last term, there's an extra factor of n^2.
In the numerator.
OK, is everybody with me here?
Now, what I'd like to do is to eventually take the limit as n goes to infinity here.
And the quantity that's hard to understand is this massive quantity here.
And there's one change that I'd like to make, but it's a very modest one.
Extremely minuscule.
Which is that I'm going to write 1, just to see that there's a general pattern here.
Going to write 1 as 1^2.
And let's put in the 3 here, why not.
And now I want to use a trick.
This trick is not completely recommended, but I will say a lot more about that when we get through to the end.
I want to understand how big this quantity is.
So I'm going to use a geometric trick to draw a picture of this quantity.
Namely, I'm going to build a pyramid.
And the base of the pyramid is going to be n by n blocks.
So imagine we're in Egypt and we're building a pyramid.
And the next layer is going to be n - 1 by n - 1.
So this next layer in is n minus 1 by n minus 1.
So the total number of blocks on the bottom is n squared.
That's this rightmost term here.
But the next term, which I didn't write in but maybe I should, the next to the last term was this one.
And that's the second layer that I've put on.
Now, this is, if you like, the top view.
But perhaps we should also think in terms of a side view.
So here's the same picture, we're starting at n and we build up this layer here.
And now we're going to put a layer on top of it, which is a little shorter.
So the first layer is of length n. And the second layers is of length n - 1, and then on top of that we have something of length n - 2, and so forth.
And we're going to pile them up.
So we pile them up.
All the way to the top, which is just one giant block of stone.
And that's this last one, 1^2.
So we're going backwards in the sum.
And so I have to build this whole thing up.
And I get all the way up in this staircase pattern to this top block, up there.
So here's the trick that you can use to estimate the size of this, and it's sufficient in the limit as n goes to infinity.
The trick is that I can imagine the solid thing underneath the staircase, like this.
That's an ordinary pyramid, not a staircase pyramid.
Which is inside.
And this one is inside.
And so, but it's an ordinary pyramid as opposed to a staircase pyramid.
And so, we know the formula for the volume of that.
Because we know the formula for volumes of cones.
And the formula for the volume of this guy, of the inside, is 1/3 base times height.
And in that case, the base here - so that's 1/3, and the base is n by n, right?
So the base is n^2.
That's the base.
And the height, it goes all the way to the top point.
So the height is n. And what we've discovered here is that this whole sum is bigger than 1/3 n^3.
Now, I claimed that - this line, by the way has slope 2.
So you go 1/2 over each time you go up 1.
And that's why you get to the top.
On the other hand, I can trap it on the outside, too, by drawing a parallel line out here.
And this will go down 1/2 more on this side and 1/2 more on the other side.
So the base will be (n + 1) by (n + 1) of this bigger pyramid.
And it'll go up 1 higher.
So on the other end, we get that this is less than 1/3 (n + 1)^3.
Again, (n + 1)^2 ( n + 1) again this is a base times a height.
Of this bigger pyramid.
Yes, question
[INAUDIBLE] and then equating it to volume
The question is, it seems as if I'm adding up areas and equating it to volume.
But I'm actually creating volumes by making these honest increments here.
That is, the base is n but the height is 1.
Thank you for pointing that out.
Each one of these little staircases here has exactly height 1.
So I'm honestly sticking blocks there.
They're sort of square blocks, and I'm lining them up.
And I'm thinking of n by n cubeds, if you like.
Honest cubes, there.
And the height is 1.
And the base is n^2.
Alright, so I claim that I've trapped this guy in between two quantities here.
And now I'm ready to take the limit.
If you look at what our goal is, we want to have an expression like this.
And I'm going to - this was the massive expression that we had.
And actually, I'm going to write it differently.
I'll write it as b^3( 1^2 + 2^2 + n^2 / n^3).
I'm going to combine all the n's together.
Alright, so the right thing to do is to divide what I had up there.
Divide by n^3 in this set of inequalities there.
And what I get here is 1/3 < (1 + 2^2 + 3^2 + n^2 / n^3) < 1/3 ( n + 1)^3 / n^3.
And that's 1/3( 1 + (1 / n))^3.
And now, I claim we're done.
Because this is 1/3, and the limit, as n goes to infinity, of this quantity here, is easily seen to be, this is, as n goes to infinity, this goes to 0.
So this also goes to 1/3.
And so our total here, so our total area, under x^2, which we sometimes might write the integral from 0 to b x^2 / dx, is going to be equal to - well, it's this 1/3 that I've got.
But then there was also a b^3 there.
So there's this extra b cubed here.
So it's 1/3 b^3.
That's the result from this whole computation.
Yes, question
[INAUDIBLE]
So that was a very good question.
The question is, why did we leave the b / n^3 out, for this step.
And a part of the answer is malice aforethought.
In other words, we know what we're heading for.
We know, we understand, this quantity.
It's all one thing.
This thing is a sum, which is growing larger and larger.
It's not what's called a closed form.
So, the thing that's not known, or not well understood, is how big is this quantity here.
1^2 + 2^2.
The sum of the squares.
Whereas, this is something that's quite easy to understand.
So we factor it out.
And we analyze carefully the piece which we don't know yet, how big it is.
And we discovered that it's very, very similar to n^3.
But it's more similar to 1/3 n^3.
It's almost identical to 1/3 n^3.
This extra piece here.
So that's what's going on.
And then we match that.
Since this thing is very similar to 1/3 n^3 we cancel the n^3's and we have our result.
Let me just mention that although this may seem odd, in fact this is what you always do if you analyze these kinds of sum.
You always factor out whatever you can.
And then you still are faced with a sum like this.
So this will happen systematically, every time you're faced with such a sum.
OK, now I want to say one more word about notation.
Which is that this notation is an extreme nuisance here.
And it's really sort of too large for us to deal with.
And so, mathematicians have a shorthand for it.
Unfortunately, when you actually do a computation, you're going to end up with this collection of stuff anyway.
But I want to just show you this summation notation in order to compress it a little bit.
The idea of summation notation is the following.
So this thing tends, the ideas are following.
I'll illustrate it with an example first.
So, the general notation is the sum of ai, i = 1 to n = a1 + a2 + ... plus an.
So this is the abbreviation.
And this is a capital Sigma.
And so, this quantity here, for instance, is (1 / n^3) the sum i^2, i = 1 to n. So that's what this thing is equal to.
And what we just showed is that that tends to 1/3 as n goes to infinity.
So this is the way the summation notation is used.
There's a formula for each of these coefficients, each of these entries here, or summands.
And then this is just an abbreviation for what the sum is.
And this is the reason why I stuck in that 1^2 at the beginning, so that you could see that the pattern worked all the way down to i = 1.
It isn't an exception to the rule.
It's the same as all of the others.
Now, over here, in this board, we also had one of these extremely long sums.
And this one can be written in the following way.
And I hope you agree, this is rather hard to scan.
But one way of writing it is, it's the sum from i = 1 to n of, now I have to write down the formula for the general term.
Which is (b / n)( ib / n)^2.
So that's a way of abbreviating this massive formula into one which is just a lot shorter.
And now, the manipulation that I performed with it, which is to factor out this (b / n)^3, is something that I'm perfectly well allowed to do also over here.
This is the distributive law.
This, if I factor out b^3 / n^3, I'm left with the sum i = 1 to n of i^2, right?
So these notations make it a little bit more compact.
What we're dealing with.
The conceptual phenomenon is still the same.
And the mess is really still just hiding under the rug.
But the notation is at least fits with fewer symbols, anyway.
So let's continue here.
I've giving you one calculation.
And now I want to fit it into a pattern.
And here's the thing that I'd like to calculate.
So, first of all let's try the case, so I'm going to do two more examples.
I'll do two more examples, but they're going to be much, much easier.
And then things are going to get much easier from now on.
So, the second example is going to be the function f(x) = x.
If I draw that, that's this function here, that's the line with slope 1.
And here's b.
And so this area here is the same as the area of the triangle with base b and height b.
So the area is equal to 1/2 b * b, so this is the base.
And this is the height.
We also know how to find the area of triangles.
And so, the formula is 1/2 b^2.
And the third example, notice, by the way, I didn't have to do this elaborate summing to do that, because we happen to know this area.
The third example is going to be even easier.
f(x) = 1.
By far the most important example, remarkably, when you get to 18.02 and multivariable calculus, you will forget this calculation.
Somehow.
And I don't know why, but it happens to everybody.
So, the function is just horizontal, like this.
Right?
It's the constant 1.
And if we stop it at b, then the area we're interested in is just this, from 0 to b.
And we know that this is height 1, so this is area, is base, which is b * 1.
So it's b.
Let's look now at the pattern.
We're going to look at the pattern of the function, and it's the area under the curve, which is this, has this elaborate formula in terms of, so this is just the area under the curve.
Between 0 and b.
And we have x^2, which turned out to be b^3 / 3.
And we have x, which turned out to be - well, let me write them over just a bit more to give myself some room.
x, which turns out to be b^2/ 2.
And then we have 1, which turned out to be b.
So this, I claim, is suggestive.
If you can figure out the pattern, one way of making it a little clearer is to see that x = x^ 1.
And 1 = x ^ 0 .
So this is the case, 0, 1 and 2.
And b = b ^ 1 / 1.
So, if you want to guess what happens when f(x) = x^3, well if it's 0, you do b ^ 1 / 1.
If it's 1, you do b ^ 2 / 2.
If it's 2, you do b ^ 3 / 3.
So it's reasonable to guess that this should be b ^ 4 / 4.
That's a reasonable guess, I would say.
Now, the strange thing is that in history, Archimedes figured out the area under a parabola.
So that was a long time ago.
It was after the pyramids.
And he used, actually, a much more complicated method than I just described here.
And his method, which is just fantastically amazing, was so brilliant that it may have set back mathematics by 2,000 years.
Because people were so, it was so difficult that people couldn't see this pattern.
And couldn't see that, actually, these kinds of calculations are easy.
So they couldn't get to the cubic.
And even when they got to the cubic, they were struggling with everything else.
And it wasn't until calculus fit everything together that people were able to make serious progress on calculating these areas.
Even though he was the expert on calculating areas and volumes, for his time.
So this is really a great thing that we now can have easy methods of doing it.
And the main thing that I want to tell you is that's we will not have to labor to build pyramids to calculate all of these quantities.
We will have a way faster way of doing it.
This is the slow, laborious way.
And we will be able to do it so easily that it will happen as fast as you differentiate.
So that's coming up tomorrow.
But I want you to know that it's going to be.
However, we're going to go through just a little pain before we do it.
And I'll just tell you one more piece of notation here.
So you need to have a little practice just to recognize how much savings we're going to make.
But never again will you have to face elaborate geometric arguments like this.
So let me just add a little bit of notation for definite integrals.
And this goes under the name of Riemann sums.
Named after a mathematician from the 1800s.
So this is the general procedure for definite integrals.
We divide it up into pieces.
And how do we do that?
Well, so here's our a and here's our b.
And what we're going to do is break it up into little pieces.
And we're going to give a name to the increment.
And we're going to call that delta x.
So we divide up into these.
So how many pieces are there?
If there are n pieces, then the general formula is always the delta x is 1 / n times the total length.
So it has to be b - a / n. We will always use these equal increments, although you don't absolutely have to do it.
We will, for these Riemann sums.
And now there's only one bit of flexibility that we will allow ourselves.
Which is this.
We're going to pick any height of f between.
In the interval, in each interval.
So what that means is, let me just show it to you on the picture here.
Is, I just pick any value in between, I'll call it ci, which is in there.
And then I go up here.
And I have the level, which is f( ci).
And that's the rectangle that I choose.
In the case that we did, we always chose the right-hand, which turned out to be the largest one.
But I could've chosen some level in between.
Or even the left-hand end.
Which would have meant that the staircase would've been quite a bit lower.
So any of these staircases will work perfectly well.
So that means were picking f ( ci), and that's a height.
And now we're just going to add them all up.
And this is the sum of the areas of the rectangles, because this is the height.
And this is the base.
This notation is supposed to be, now, very suggestive of the notation that Leibniz used.
Which is that in the limit, this becomes an integral from a to b of f(x) dx.
And notice that the delta x gets replaced by a dx.
So this is what happens in the limit.
As the rectangles get thin.
So that's as delta x goes to 0.
And these gadgets are called Riemann sums.
This is called a Riemann sum.
And we already worked out an example.
This very complicated guy was an example of a Riemann sum.
So that's a notation.
And we'll give you a chance to get used to it a little more when we do some numerical work at the end.
Now, the last thing for today is, I promised you an example which was not an area example.
I want to be able to show you that integrals can be interpreted as cumulative sums.
Integrals as cumulative sums.
So this is just an example.
And, so here's the way it goes.
So we're going to consider a function f, we're going to consider a variable t, which is time.
In years.
And we'll consider a function f( t), which is in dollars per year.
Right, this is a financial example here.
That's the unit here, dollars per year.
And this is going to be a borrowing rate.
Now, the reason why I want to put units in here is to show you that there's a good reason for this strange dx, which we append on these integrals.
This notation.
It allows us to change variables, it allows this to be consistent with units.
And allows us to develop meaningful formulas, which are consistent across the board.
And so I want to emphasize the units in this when I set up this modeling problem here.
Now, you're borrowing money.
Let's say, every day.
So that means delta t = 1/365.
That's almost 1 / infinity, from the point of view of various purposes.
So this is how much you're borrowing.
In each time increment you're borrowing.
And let's say that you borrow, your rate varies over the year.
I mean, sometimes you need more money sometimes you need less.
Certainly any business would be that way.
And so here you are, you've got your money.
And you're borrowing but the rate is varying.
And so how much did you borrow?
Well, in Day 45, which is 45/365, you borrowed the following amount.
Here was your borrowing rate times this quantity.
So, dollars per year.
And so this is, if you like, I want to emphasize the scaling that comes about here.
You have dollars per year.
And this is this number of years.
So that comes out to be in dollars.
This final amount.
This is the amount that you actually borrow.
So you borrow this amount.
And now, if I want to add up how much you get, you've borrowed in the entire year.
That's this sum.
i = 1 to 365 of f of, well, it's (i / 365) delta t. Which I'll just leave as delta t here.
This is total amount borrowed.
This is kind of a messy sum.
In fact, your bank probably will keep track of it and they know how to do that.
But when we're modeling things with strategies, you know, trading strategies of course, you're really some kind of financial engineer and you want to cleverly optimize how much you borrow.
And how much you spend, and how much you invest.
This is going to be very, very similar to the integral from 0 to 1 of f (t) dt.
At the scale of 1/35, it's probably, 365, it's probably enough for many purposes.
Now, however, there's another thing that you would want to model.
Which is equally important.
This is how much you borrowed, but there's also how much you owe the back at the end of the year.
And the amount that you owe the bank at the end of the year, I'm going to do it in a fancy way.
It's, the interest, we'll say, is compounded continuously.
So the interest rate, if you start out with P as your principal, then after time t, you owe, so borrow P, after time t, you owe P e ^ rt, where r is your interest rate.
Say, 0.05 per year.
That would be an example of an interest rate.
And so, if you want to understand how much money you actually owe at the end of the year, at the end of the year what you owe is, well, you borrowed these amounts here.
But now you owe more at the end of the year.
You owe e ^ r times the amount of time left in the year.
So the amount of time left in the year is 1 - (i / 365).
Or 365 - i days left.
So this is (1 - i / 365).
And this is what you have to add up, to see how much you owe.
And that is essentially the integral from 0 to 1.
The delta t comes out.
And you have here e ^ r (1 - t), so the t is replacing this i / 365, f (t) dt.
And so when you start computing and thinking about what's the right strategy, you're faced with integrals of this type.
So that's just an example.
And see you next time.
Remember to think about questions that you'll ask next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to continue with integration.
And we get to do the probably the most important thing of this entire course.
Which is appropriately named.
It's called the fundamental theorem of calculus.
And we'll be abbreviating it FTC and occasionally I'll put in a 1 here, because there will be two versions of it.
But this is the one that you'll be using the most in this class.
The fundamental theorem of calculus says the following.
It says that if F' = f, so F' ( x) = little f ( x), there's a capital F and a little f, then the integral from a to b of f ( x) = F ( b) - F (a).
That's it.
That's the whole theorem.
And you may recognize it.
Before, we had the notation that f was the antiderivative, that is, capital F was the integral of f(x).
We wrote it this way.
This is this indefinite integral.
And now we're putting in definite values.
And we have a connection between the two uses of the integral sign.
But with the definite values, we get real numbers out instead of a function.
Or a function up to a constant.
So this is it.
This is the formula.
And it's usually also written with another notation.
So I want to introduce that notation to you as well.
So there's a new notation here.
Which you'll find very convenient.
Because we don't always have to give a letter f to the functions involved.
So it's an abbreviation.
For right now there'll be a lot of f's, but anyway.
So here's the abbreviation.
Whenever I have a difference between a function at two values, I also can write this as F ( x) with an a down here and a b up there.
So that's the notation that we use.
And you can also, for emphasis, and this sometimes turns out to be important, when there's more than one variable floating around in the problem.
To specify that the variable is x.
So this is the same thing as x = a.
And x = b.
It indicates where you want to plug in, what you want to plug in.
And now you take the top value minus the bottom value.
So F ( b) - F(a).
So this is just a notation, and in that notation, of course, the theorem can be written with this set of symbols here.
Equally well.
So let's just give a couple of examples.
The first example is the one that we did last time very laboriously.
If you take the function capital F(x), which happens to be x^3 / 3, then if you differentiate it, you get, well, the the factor of 3 cancels.
So you get x^2, that's the derivative.
And so by the fundamental theorem, so this implies by the fundamental theorem, that the integral from say, a to b of x^3 over - sorry, x^2 dx, that's the derivative here.
This is the function we're going to use as f ( x) here = this function here.
F ( b) - F ( a), that's here.
This function here.
So that's write F(b) - F( a), and that's equal to b^3 / 3 - a^3 / 3.
Now, in this new notation, we usually don't have all of these letters.
All we write is the following.
We write the integral from a to be, and I'm going to do the case 0 to b, because that was the one that we actually did last time.
So I'm going to set a = 0 here.
And then, the problem we were faced last time as this.
And as I said we did it very laboriously.
But now you can see that we can do it in ten seconds, let's say.
Well, the antiderivative of this is x^3 / 3.
I'm going to evaluate it at 0 and at b and subtract.
So that's going to be b^3 / 3 - 0^3 / 3.
Which of course is b^3 / 3.
And that's the end, that's the answer.
So this is a lot faster than yesterday.
I hope you'll agree.
And we can dispense with those elaborate computations.
Although there's a conceptual reason, a very important one, for understanding the procedure that we went through.
Because eventually you're going to be using integrals and these quick ways of doing things, to solve problems like finding the volumes of pyramids.
In other words, we're going to reverse the process.
And so we need to understand the connection between the two.
I'm going to give a couple more examples.
And then we'll go on.
So the second example would be one that would be quite difficult to do by this Riemann sum technique that we described yesterday.
Although it is possible.
It uses much higher mathematics to do it.
And that is the area under one hump of the sine curve, sine x.
Let me just draw a picture of that.
The curve goes like this, and we're talking about this area here.
It starts out at 0, it goes to pi.
That's one hump.
And so the answer is, it's the integral from 0 to pi of sin x dx.
And so I need to take the antiderivative of that.
And that's - cos x.
That's the thing whose derivative is sin x.
Evaluating it at 0 and pi.
Now, let's do this one carefully.
Because this is where I see a lot of arithmetic mistakes.
Even though this is the easy part of the problem.
It's hard to pay attention and plug in the right numbers.
And so, let's just pay very close attention.
I'm plugging in pi.
That's - cos pi.
That's the first term.
And then I'm subtracting the value at the bottom, which is - cos 0.
There are already five opportunities for you to make a transcription error or an arithmetic mistake in what I just did.
And I've seen all five of them.
So the next one is that this is - (- 1).
Minus negative 1, if you like.
And then this is minus, and here's another - 1.
So altogether we have 2.
So that's it.
That's the area.
This area, which is hard to guess, this is area 2.
The third example is maybe superfluous but I'm going to say it anyway.
We can take the integral, say, from 0 to 1, of x ^ 100.
Any power, now, is within our power.
So let's do it.
So here we have the antiderivative is x ^ 101 / 101.
Evaluate it at 0 and 1.
And that is just 1 / 101.
That's that.
So that's the fundamental theorem.
Now this, as I say, harnesses a lot of what we've already learned, all about antiderivatives.
Now, I want to give you an intuitive interpretation.
So let's try that.
We'll talk about a proof of the fundamental theorem a little bit later.
It's not actually that hard.
But we'll give an intuitive reason, interpretation, if you like.
Of the fundamental theorem.
So this is going to be one which is not related to area, but rather to time and distance.
So we'll consider x (t) is your position at time t. And then x' (t), which is dx/dt, is going to be what we know as your speed.
And then what the theorem is telling us, is the following.
It's telling us the integral from a to b of v ( t) dt.
So, reading the relationship is equal to x (b) - x ( a).
And so this is some kind of cumulative sum of your velocities.
So let's interpret the right-hand side first.
This is the distance traveled.
And it's also what you would read on your odometer.
Right, from the beginning to the end of the trip.
That's what you would read on your odometer.
Whereas this is what you would read on your speedometer.
So this is the interpretation.
Now, I want to just go one step further into this interpretation, to make the connection with the Riemann sums that we had yesterday.
Because those are very complicated to understand.
And I want you to understand them viscerally on several different levels.
Because that's how you'll understand integration better.
The first thing that I want to imagine, so we're going to do a thought experiment now, which is that you are extremely obsessive.
And you're driving your car from time a to time b, place q to place r, whatever.
And you check your speedometer every second.
OK, so you've read your speedometer in the i'th second, and you've read that you're going at this speed.
Now, how far do you go in that second?
Well, the answer is you go this speed times the time interval, which in this case we're imagining as 1 second.
Alright?
So this is how far you went.
But this is the time interval.
And this is the distance traveled.
in that a second number, i, in the i'th second.
The distance traveled in the i'th second, that's a total distance you traveled.
Now, what happens if you go the whole distance?
Well, you travel the sum of all these distances.
So it's some massive sum, where n is some ridiculous number of seconds.
3600 seconds or something like that.
Whatever it is.
And that's going to turn out to be very similar to what you would read on your odometer.
Because during that second, you didn't change velocity very much.
So the approximation that the speed at one time that you spotted it is very similar to the speed during the whole second.
It doesn't change that much.
So this is a pretty good approximation to how far you traveled.
And so the sum is a very realistic approximation to the entire integral.
Which is denoted this way.
Which, by the fundamental theorem, is exactly how far you traveled.
So this is x ( b) - x (a).
Exactly.
The other one is approximate.
OK, again this is called a Riemann sum.
Alright so that's the intro to the fundamental theorem.
And now what I need to do is extend it just a bit.
And the way I'm going to extend it is the following.
I'm going to do it on this example first.
And then we'll do it more formally.
So here's this example where we went someplace.
But now I just want to draw you an additional picture here.
Imagine I start here and I go over to there and then I come back.
And maybe even I do a round trip.
I come back to the same place.
Well, if I come back to the same place, then the position is unchanged from the beginning to the end.
In other words, the difference is 0.
And the velocity, technically rather than the speed.
It's the speed to the right and the the speed to the left maybe are the same, but one of them is going in the positive direction and one of them is going in the negative direction, and they cancel each other.
So if you have this kind of situation, we want that to be reflected.
We like that interpretation and we want to preserve it even when in the case when the function v is negative.
And so I'm going to now extend our notion of integration.
So we'll extend integration to the case f negative.
Or positive.
In other words, it could be any sign.
Actually, there's no change.
The formulas are all the same.
We just, if this v is going to be positive, we write in a positive number.
If it's going to be negative, we write in a negative number.
And we just leave it alone.
And the real, so here's, let me carry out an example and show you how it works.
I'll carry out the example on this blackboard up here.
Of the sine function.
But we're going to try two humps.
We're going to try the first hump and the one that goes underneath.
There.
So our example here is going to be the integral from 0 to 2pi of sin x dx.
And now, because the fundamental theorem is so important, and so useful, and so convenient, we just assume that it be true in this case as well.
So we insist that this is going to be - cos x.
Evaluate it at 0 and 2pi.
With the difference.
Now, when we carry out that difference, what we get here is - cos 2pi.
- (- cos 0).
Which is - 1 - (- 1), which is 0.
And the interpretation of this is the following.
Here's our double hump, here's pi and here's 2pi.
And all that's happening is that the geometric interpretation that we had before of the area under the curve has to be taken with a grain of salt.
In other words, I lied to you before when I said that the definite integral was the area under the curve.
It's not.
The definite integral is the area under the curve when it's above the curve, and it counts negatively when it's below the curve.
So yesterday, my geometric interpretation was incomplete.
And really just a plain lie.
So the true geometric interpretation of the definite integral is plus the area above the axis, above the x axis, minus the area below the x axis.
As in the picture.
I'm just writing it down in words, but you should think of it visually also.
So that's the setup here.
And now we have the complete definition of integrals.
And I need to list for you a bunch of their properties and how we deal with integrals.
So are there any questions before we go on?
Yeah
[INAUDIBLE]
Right.
So the question was, wouldn't the absolute value of the velocity function be involved?
The answer is yes.
That is, that's one question that you could ask.
One question you could ask is what's the total distance traveled.
And in that case, you would keep track of the absolute value of the velocity as you said.
Whether it's positive or negative.
And then you would get the total length of this curve here.
That's, however, not what the definite integral measures.
It measures the net distance traveled.
So it's another thing.
In other words, we can do that.
We now have the tools to do both.
We could also, so if you like, the total distance is equal to the integral of this.
From a to b.
But the net distance is the one without the absolute value signs.
So that's correct.
Other questions?
Alright.
So now, let's talk about properties of integrals.
So the properties of integrals that I want to mention to you are these.
The first one doesn't bear too much comment.
If you take the cumulative integral of a sum, you're just trying to get the sum of the separate integrals here.
And I won't say much about that.
That's because sums come out, the because the integral is a sum.
Incidentally, you know this strange symbol here, there's actually a reason for it historically.
If you go back to old books, you'll see that it actually looks a little bit more like an s. This capital Sigma is a sum.
S for sum, because everybody in those days knew Latin and Greek.
And this one is also an s, but gradually it was such an important s that they made a bigger.
And then they stretched it out and made it a little thinner, because it didn't fit into one typesetting space.
And so just for typesetting reasons it got stretched.
And got a little bit skinny.
Anyway, so it's really an s. And in fact, in French they call it sum.
Even though we call it integral.
So it's a sum.
So it's consistent with sums in this way.
And similarly, similarly we can factor constants out of sums.
So if you have an integral like this, the constant factors out.
But definitely don't try to get a function out of this.
That won't happen.
OK, in other words, c has to be a constant.
Doesn't depend on x.
The third property.
What do I want to call the third property here?
I have sort of a preliminary property, yes, here.
Which is the following.
And I'll draw a picture of it.
I suppose you have three points along a line.
So then I'm going to draw a picture that.
And I'm going to use the interpretation above the curve, even though that's not the whole thing.
So here's a, here's b and here's c. And you can see that the area of this piece, of the first two pieces here, when added together, gives you the area of the whole.
And that's the rule that I'd like to tell you.
So if you integrate from a to b, and you add to that the integral from b to c, you'll get the integral from a to c. This is going to be just a little preliminary, because the rule is a little better than this.
But I will explain that in a minute.
The fourth rule is a very simple one.
Which is that the integral from a to a of f ( x ) dx = 0.
Now, that you can see very obviously because there's no area.
No horizontal movement there.
The rectangle is infinitely thin, and there's nothing there.
So this is the case.
You can also interpret it a F ( a) - F ( a).
So that's also consistent with our interpretation.
In terms of the fundamental theorem of calculus.
And it's perfectly reasonable that this is the case.
Now, the fifth property is a definition.
It's not really a property.
But it's very important.
The integral from a to b of f( x) dx = minus the integral from b to a, of f( x) dx.
Now, really, the right-hand side here is an undefined quantity so far.
We never said you could ever do this where the a < b.
Because this is working backwards here.
But we just have a convention that that's the definition.
Whenever we write down this number, it's the same as minus what that number is.
And the reason for all of these is again that we want them to be consistent with the fundamental theorem of calculus.
Which is the thing that makes all of this work.
So if you notice the left-hand side here is F ( b) - F ( a), capital F. The antiderivative of little f. On the other hand, the other side is minus, and if we just ignore that, we say these are letters, if we were a machine, we didn't know which one was bigger than which, we just plugged them in, we would get here F( a) - F( b), over here.
And to make these two things equal, what we want is to put that minus sign in.
Now it's consistent.
So again, these rules are set up so that everything is consistent.
And now I want to improve on rule 3 here.
And point out to you.
So let me just go back to rule 3 for a second.
That now that we can evaluate integrals regardless of the order, we don't have to have a < b, b < c in order to make sense out of this.
We actually have the possibility of considering integrals where the a's and the b's and the c's are in any order you want.
And in fact, with this definition, with this definition 5, 3 works no matter what the numbers are.
So this is much more convenient.
We don't, this is not necessary.
Not necessary.
It just works using convention 5.
OK, with 5.
Again, before I go on, let me emphasize we really want to respect the sign of this velocity.
We really want the net change in the position.
And we don't want this absolute value here.
Because otherwise, all of our formulas are going to mess up.
We won't always be able to check.
Sometimes you have letters rather than actual numbers here, and you won't know whether a is bigger than b.
So you'll want to know that these formulas work and are consistent in all situations.
OK, I'm going to trade these again.
In order to preserve the ordering 1 through 5.
And now I have a sixth property that I want to talk about.
This one is called estimation.
And it says the following.
if f(x) <= g ( )x, then the integral from a to b of f(x) dx <= the integral from a to b of g (x) dx.
Now, this one says that if I'm going more slowly than you, then you go farther than I do.
OK. That's all it's saying.
For this one, you'd better have a < b.
You need it.
Because we flip the signs when we flip the order of a and b.
So this one, it's essential that the lower limit be smaller than the upper limit.
But let me just emphasize, because we're dealing with the generalities of this.
Actually if one of these is negative and the other one is negative, then it also works.
This one ends up being, if f is more negative than g, then this added up thing is more negative than that one.
Again, under the assumption that a < b.
So as I wrote it's in full generality.
Let's illustrate this one.
And then we have one more property to learn after that.
So let me give you an example of estimation.
The example is the same as one that I already gave you.
But this time, because we have the tool of integration, we can just follow our noses and it works.
I start with the inequality, so I'm trying to illustrate estimation, so I want to start with an inequality which is what the hypothesis is here.
And I'm going to integrate the inequality to get this conclusion.
And see what conclusion it is.
The inequality that I want to take is that e ^ x >= 1, for x >= 0.
That's going to be our starting place.
And now I'm going to integrate it.
That is, I'm going to use estimation to see what that gives.
Well, I'm going to integrate, say, from 0 to b. I can't integrate below 0 because it's only true above 0.
This is e ^ x dx >= the integral from 0 to b of 1 dx.
Alright, let's work out what each of these is.
The first one, e ^ x dx, is, the antiderivative is e ^ x, evaluated at 0 and b.
So that's e ^ b - e ^ 0.
Which is e ^ b - 1.
The other one, you're supposed to be able to get by the rectangle law.
This is one rectangle of base b and height 1.
So the answer is b.
Or you can do it by antiderivatives, but it's b.
That means that our inequality says if I just combine these two things together, that e ^ b - 1 >= b.
And that's the same thing as e ^ b >= 1 + b.
Again, this only works for b >= 0.
Notice that if b were negative, this would be a well defined quantity.
But this estimation would be false.
We need that the b > 0 in order for this to make sense.
So this was used.
And that's a good thing, because this inequality is suspect.
Actually, it turns out to be true when b is negative.
But we certainly didn't prove it.
I'm going to just repeat this process.
So let's repeat it.
Starting from the inequality, the conclusion, which is sitting right here.
But I'll write it in a form e ^ x >= 1 + x, for x >= 0.
And now, if I integrate this one, I get the integral from 0 to b, e ^ x dx >= the integral from 0 to b, (1 + x) dx and I remind you that we've already calculated this one.
This is e ^ b - 1.
And the other one is not hard to calculate.
The antiderivative is x + x^2 / 2.
We're evaluating that at 0 and b.
So that comes out to be b + b^2 / 2.
And so our conclusion is that the left side, which is e ^ b - 1 >= b + b^2 / 2.
And this is for b >= 0.
And that's the same thing as e ^ b >= 1 + b + b^2 / 2.
This one actually is false for b negative, so that's something that you have to be careful with the b positives here.
So you can keep on going with this, and you didn't have to think.
And you'll produce a very interesting polynomial, which is a good approximation to e^ b.
So so that's it for the basic properties.
Now there's one tricky property that I need to tell you about.
It's not that tricky, but it's a little tricky.
And this is change of variables.
Change of variables in integration, we've actually already done.
We called that, the last time we talked about it, we called it substitution.
And the idea here, if you may remember, was that if you're faced with an integral like this, you can change it to, if you put in u = u (x) and you have a du, which is equal to u' ( x) du, dx, sorry.
Then you can change the integral as follows.
This is the same as g (u( x)) u' (x) dx.
This was the general procedure for substitution.
What's new today is that we're going to put in the limits.
If you have a limit here, u1, and a limit here, u2, you want to know what the relationship is between the limits here and the limits when you change variables to the new variables.
And it's the simplest possible thing.
Namely the two limits over here are in the same relationship as u ( x) is to this symbol u here.
In other words, u1 = u ( x1), and u2 = u(x2).
That's what works.
Now there's only one danger here, there's subtlety which is, this only works if u' does not change sign.
I've been worrying a little bit about going backwards and forwards, and I allowed myself to reverse and do all kinds of stuff, right, with these integrals.
So we're sort of free to do it.
Well, this is one case where you want to avoid it, OK?
Just don't do it.
It is possible, actually, to make sense out of it, but it's also possible to get yourself infinitely confused.
So just make sure that -- now it's OK is u' is always negative, or always going one way, so OK if u' is always positive, you're always going the other way, but if you mix them up you'll get yourself mixed up.
Let me give you an example.
The example will be maybe close to what we did last time.
When we first did substitution.
So the integral from 1 to 2, this time I'll put in definite limits, of x^2 plus -- sorry, maybe I call this x^3.
x^3 + 2, let's say, I don't know, to the 5th power, x^2 dx.
So this is an example of an integral that we would have tried to handle by substitution before.
And the substitution we would have used is u = x^3 + 2.
And that's exactly what we're going to do here.
But we're just going to also take into account the limits.
The first step as in any substitution or change of variables, is this.
And so we can fill in the things that we would have done previously.
Which is that this is the integral and this is u ^ 5.
And then because this is 3x^2, we see that this is 3.
Sorry, let's write it the other way.
1/3 du = x^2 dx.
So that's what I'm going to plug in for this factor here.
So here's 1/3 du, which replaces that.
But now there's the extra feature.
The extra feature is the limits.
So here, really in disguise, because, and now this is incredibly important.
This is one of the reasons why we use this notation dx and du.
We want to remind ourselves which variable is involved in the integration.
And especially if you're the one naming the variables, you may get mixed up in this respect.
So you must know which variable is varying between 1 and 2.
And the answer is, it's x is the one that's varying between 1 and 2.
So in disguise, even though I didn't write it, it was contained in this little symbol here.
This reminded us which variable.
You'll find this amazingly important when you get to multivariable calculus.
When there are many variables floating around.
So this is an incredibly important distinction to make.
So now, over here we have a limit.
But of course it's supposed to be with respect to u, now.
So we need to calculate what those corresponding limits are.
And indeed it's just, I plug in here u1 is going to be able to what I plug in for x = 1, that's going to be 1 ^3 + 2, which is 3.
And then u2 is 2^3 + 2, which = 10, right?
8 + 2 = 10.
So this is the integral from 3 to 10, of u ^ 5 (1/3 du).
And now I can finish the problem.
This is 1/18 u ^ 6, from 3 to 10.
And this is where the most common mistake occurs in substitutions of this type.
Which is that if you ignore this, and you plug in these 1 and 2 here, you think, oh I should just be putting it at 1 and 2.
But actually, it should be, the u value that we're interested in, and the lower limit is u = 3 and u = 10 as the upper limit.
So those are suppressed here.
But those are the ones that we want.
And so, here we go.
It's 1/18 times some ridiculous number which I won't calculate.
10 ^ 6 - 3 ^ 6.
Yes, question
[INAUDIBLE]
So, if you want to do things with where you're worrying about the sign change, the right strategy is, what you suggested works.
And in fact I'm going to do an example right now on this subject.
But, the right strategy is to break it up into pieces.
Where u' has one sign or the other, OK?
Let me show you an example.
Where things go wrong.
And I'll tell you how to handle it roughly.
So here's our warning.
Suppose you're integrating for - 1 to 1, x^2 dx.
Here's an example.
And you have the temptation to plug in u = x^2.
Now, of course, we know how to integrate this.
But let's just pretend we were stubborn and wanted to use substitution.
Then we have du = 2x dx.
And now if I try to make the correspondence, notice that the limits are u1 = (- 1)^2, that's the bottom limit.
And u2 is the upper limit.
That's 1 ^2, that's also equal to 1.
Both limits are 1.
So this is going from 1 to 1.
And no matter what it is, we know it's going to be 0.
But we know this is not 0.
This is the integral of a positive quantity.
And the area under a curve is going to be a positive area.
So this is a positive quantity.
It can't be 0.
If you actually plug it in, it looks equally strange.
You put in here this u and then, so that would be for the u^2.
And then to plug in for dx, you would write dx = (1 / 2x) du.
And then you might write that as this.
And so what I should put in here is this quantity here.
Which is a perfectly OK integral.
And it has a value, I mean, it's what it is.
It's 0.
So of course this is not true.
And the reason is that u = x^2, and u' ( x ) = 2x, which was positive for x positive, and negative for x negative.
And this was the sign change which causes us trouble.
If we break it off into its two halves, then it'll be OK and you'll be able to use this.
Now, there was a mistake.
And this was essentially what you were saying.
That is, it's possible to see this happening as you're doing it if you're very careful.
There's a mistake in this process, and the mistake is in the transition.
This is a mistake here.
Maybe I haven't used any red yet today, so I get to use some red here.
Oh boy.
This is not true, here.
This step here.
So why isn't it true?
It's not true for the standard reason.
Which is that really, x = plus or minus square root of u.
And if you stick to one side or the other, you'll have a coherent formula for it.
One of them will be the plus and one of them will be the minus and it will work out when you separate it into its pieces.
So you could do that.
But this is a can of worms.
So I avoid this.
And just do it in a place where the inverse is well defined.
And where the function either moves steadily up or steadily down.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
To begin today I want to remind you, I need to write it down on the board at least twice, of the fundamental theorem of calculus.
We called it FTC 1 because it's the first version of the fundamental theorem.
We'll be talking about another version, called the second version, today.
And what it says is this: If F' = f, then the integral from a to b of f (x) dx = F ( b) - F ( a).
So that's the fundamental theorem of calculus.
And the way we used it last time was, this was used to evaluate integrals.
Not surprisingly, that's how we used it.
But today, I want to reverse that point of view.
We're going to read the equation backwards, and we're going to write it this way.
And we're going to use f to understand capital F. Or in other words, the derivative.
To understand the function.
So that's the reversal of point of view that I'd like to make.
And we'll make this point in various ways.
So information about F, about F', gives us information about F. Now, since there were questions about the mean value theorem, I'm going to illustrate this first by making a comparison between the fundamental theorem of calculus and the mean value theorem.
So we're going to compare this fundamental theorem of calculus with what we call the mean value theorem.
And in order to do that, I'm going to introduce a couple of notations.
I'll write delta F as F ( b ) - F ( a).
And another highly imaginative notation, delta x = b - a.
So here's the change in f, there's the change in x.
And then, this fundamental theorem can be written, of course, right up above there is the formula.
And it's the formula for delta F. So this is what we call the fundamental theorem of calculus.
I'm going to divide by delta x, now.
And If I divide by delta x, that's the same thing as 1 / b - a times the integral from a to b of f ( x) dx.
So I've just rewritten the formula here.
And this expression here, on the right-hand side, is a fairly important one.
This is the average of f. That's the average value of f. Now, so this is going to permit me to make the comparison between the mean value theorem, which we don't have stated yet here.
And the fundamental theorem.
And I'll do it in the form of inequalities.
So right in the middle here, I'm going to put the fundamental theorem.
It says that delta F in this notation is equal to, well if I multiply by delta x again, I can write it as the average of F. So I'm going to write it as the average of F' here.
Times delta x.
So we have this factor here, which is the average of F', or the average of little f, it's the same thing.
And then I multiplied through again.
So I put the thing in the red box.
Here.
Isn't what the average of big F. So the question is, why is this the average of little f rather than the average of big F. So the average of a function is the typical value.
If, for example, little f were constant, little f were constant, then this integral would be, so the question is why is this the average.
And I'll take a little second to explain that.
But I think I'll explain it over here.
Because I'm going to erase it.
So the idea of an average is the following.
For example, imagine that a = 0 and b = n, let's say for example.
And so we might sum the function from 1 to n. Now, that would be the sum of the values from 1 to n. But the average is, we divide by n here.
So this is the average.
And this is a kind of Riemann sum, representing the integral from 0 to n, of f (x) dx.
Where the increment, delta x, is 1.
So this is the notion of an average value here.
But in the continuum setting as opposed to the discrete setting.
Whereas what's on the left-hand side is the change in f. The capital F. And this is the average of the little f. So an average is a sum.
And it's like an integral.
So, in other words what I have here is that the change in f is the average of its infinitesimal change times the amount of time elapsed, if you like.
So this is the statement of the fundamental theorem.
Just rewritten.
Exactly what I wrote there.
But I multiplied back by delta x.
Now, let me compare this with the mean value theorem.
The mean values theorem also is an equation.
The mean value theorem says that this is equal to F' (c) delta x.
Now, I pulled a fast one on you.
I used capital F's here to make the analogy clear.
But the role of the letter is important to make the transition to this comparison.
We're talking about the function capital F here.
And its derivative.
Now, this is true.
So now I claim that this thing is fairly specific.
Whereas this, unfortunately, is a little bit vague.
And the reason why it's vague is that c is just somewhere in the interval.
So some c - sorry, this is some c, in between a and b.
So really, since we don't know where this thing is, we don't know which of the values it is, we can't say what it is.
All we can do is say well for sure it's less than the largest value.
Say, (the maximum of F' ) delta x.
And the only thing we can say for sure on the other end is that it's less than or equal to - sorry, it's greater than or equal to, (the minimum of F' ) delta x.
Over that same interval.
This is over 0 less than - sorry, a < x < b.
So that means that the fundamental theorem of calculus is a much more specific thing.
And indeed it gives the same conclusion.
It's much stronger than the mean value theorem.
It's way better than the mean value theorem.
In fact, as soon as we have integrals, we can abandon the mean value theorem.
We don't want it.
It's too simple-minded.
And what we have is something much more sophisticated, which we can use.
Which is this.
So it's obvious that if this is the average, the average is less than the maximum.
So it's obvious that it works just as well to draw this conclusion.
And similarly over here with the minimum.
OK, the average is always bigger than the minimum and smaller than the max.
So this is the connection, if you like.
And I'm going to elaborate just one step further by talking about the problem that you had on the exam.
So there was an Exam 2 problem.
And I'll show you how it works using the mean value theorem and how it works using integrals.
But I'm going to have to use this notation capital F. So capital F', as opposed to the little f, which was what was the notation that was on your exam.
So we had this situation here.
These were the givens of the problem.
And then the question was, the mean value theorem says, or implies, if you like, it doesn't say it, but it implies it.
Implies a < capital F ( 4) < b, for which a and b?
So let's take a look at what it says.
Well, the mean value theorem says that F( 4) - F (0) = F' ( c)( 4 - 0).
This is this (F') delta x, this is the change in x.
And that's the same thing as (1 / 1 + c )( 4).
And so the range of values of this number here is from (1 / 1 plus 0)( 4), that's 4.
To, that's the largest value, to the smallest that it gets, which is (1 / 1 + 4)( 4).
That's the range.
And so the conclusion is that F (4) - f ( 0 ) is between, well, let's see.
It's between 4 and 4/5.
Which are those two numbers down there.
And if you remember that F ( 0) was 1, this is the same as F ( 4), is between 5 and 9/5.
So that's the way that you were supposed to solve the problem on the exam.
On the other hand, let's compare to what you would do with the fundamental theorem of calculus.
With the fundamentals theorem of calculus, we have the following formula.
F( 4) - F ( 0) = the integral from 0 to 4 of dx / 1 + x.
That's what the fundamental theorem says.
And now I claim that we can get these same types of results by a very elementary observation.
It's really the same observation that I made up here, that the average is less than or equal to the maximum.
Which is that the biggest this can ever be is, let's see.
The biggest it is when x is 0, that's 1.
So the biggest it ever gets is this.
And that's equal to 4.
Right?
On the other hand, the smallest it ever gets to be, it's equal to this.
The smallest it ever gets to be is the integral from 0 to 4 of 1/5 dx.
Because that's the lowest value that the integrand takes.
When x = 4, it's 1/5.
And that's equal to 4/5.
Now, there's a little tiny detail which is that really we know that this is the area of some rectangle and this is strictly smaller.
And we know that these inequalities are actually strict.
But that's a minor point.
And certainly not one that we'll pay close attention to.
But now, let me show you what this looks like geometrically.
So geometrically, we interpret this as the area under a curve.
Here's a piece of the curve y = 1/1 + x.
And it's going up to 4 and starting at 0 here.
And the first estimate that we made; that is, the upper bound, was by trapping this in this big rectangle here.
We compared it to the constant function, which was 1 all the way across.
This is y = 1.
And then we also trapped it from underneath by the function which was at the bottom.
And this was y = 1/5.
And so what this really is is, these things are the simplest possible Riemann sum.
Sort of a silly Riemann sum.
This is a Riemann sum with one rectangle.
This is the simplest possible one.
And so this is a very, very crude estimate.
You can see it misses by a mile.
The larger and the smaller values are off by a factor of 5.
But this one is called the, this one is the lower Riemann sum.
And that one is less than our actual integral.
Which is less than the upper Riemann sum.
And you should, by now, have looked at those upper and lower sums on your homework.
So it's just the rectangles underneath and the rectangles on top.
So at this point, we can literally abandon the mean value theorem.
Because we have a much better way of getting at things.
If we chop things up into more rectangles, we'll get much better numerical approximations.
And if we use simpleminded expressions with integrals, we'll be able to figure out any bound we could get using the mean value theorem.
So that's not the relevance of the mean value theorem.
I'll explain to you why we talked about it, even, in a few minutes.
OK, are there any questions before we go on?
Yeah
[INAUDIBLE]
I knew that the range of c was from 0 to 4, I should have said that right here.
This is true for this theorem.
The mean value theorem comes with an extra statement, which I missed.
Which is that this is for some c between 0 and 4.
So I know the range is between 0 and 4.
The reason why it's between 0 and 4 is that 's part of the mean value theorem.
We started at 0, we ended at 4.
So the c has to be somewhere in between.
That's part of the mean value theorem
[INAUDIBLE]
The question is, do you exclude any values that are above 4 and below 0.
Yes, absolutely.
The point is that in order to figure out how F changes, capital F changes, between 0 and 4, you need only pay attention to the values in between.
You don't have to pay any attention to what the function is doing below 0 or above 4.
Those things are strictly irrelevant
[INAUDIBLE]
Yeah, I mean it's strictly in between these two numbers.
I have to understand what the lowest and the highest one is
[INAUDIBLE]
It's approaching that, so.
OK.
So now, the next thing that we're going to talk about is, since I've got that 1 up there, that Fundamental Theorem of Calculus 1, I need to talk about version 2.
So here is the Fundamental Theorem of Calculus version 2.
I'm going to start out with a function little f, and I'm going to assume that it's continuous.
And then I'm going to define a new function, which is defined as a definite integral.
G ( x ) is the integral from a to x of f ( t ) dt.
Now, I want to emphasize here because it's the first time that I'm writing something like this, that this is a fairly complicated gadget.
It plays a very basic and very fundamental, but simple role but it nevertheless is a little complicated.
What's happening here is that the upper limit I've now called x, and the variable t is ranging between a and x, and that the a and the x are fixed when I calculate the integral.
And the t is what's called the dummy variable.
It's the variable of integration.
You'll see a lot of people who will mix this x with this t. And if you do that, you will get confused, potentially hopelessly confused, in this class.
In 18.02 you will be completely lost if you do that.
So don't do it.
Don't mix these two guys up.
It's actually done by many people in textbooks, and it's fairly careless.
Especially in old-fashioned textbooks.
But don't do it.
So here we have this G ( x).
Now, remember, this G ( x) really does make sense.
If you give me an a, and you give me an x, I can figure out what this is, because I can figure out the Riemann sum.
So of course I need to know what the function is, too.
But anyway, we have a numerical procedure for figuring out what the function g is.
Now, as is suggested by this mysterious letter x being in the place where it is, I'm actually going to vary x.
So the conclusion is that if this is true, and this is just a parenthesism not part of the theorem.
It's just an indication of what the notation means.
Then G' = f. Let me first explain what the significance of this theorem is, from the point of view of differential equations.
G ( x) solves the differential equation y' = f ( x).
So y' = f, you put the x in if I got it here, with the condition y ( a) = 0.
So it solves this pair of conditions here.
The rate of change, and the initial position is specified here.
Because when you integrate from a to a, you get 0 always.
And what this theorem says is you can always solve that equation.
When we did differential equations, I said that already.
I said we'll treat these as always solved.
Well, here's the reason.
We have a numerical procedure for computing things like this.
We could always solve this equation.
And the formula is a fairly complicated gadget, but so far just associated with Riemann sums.
Alright, now.
Let's just do one example.
Unfortunately, not a complicated example and maybe not persuasive as to why you would care about this just yet.
But nevertheless very important.
Because this is the quiz question which everybody gets wrong until they practice it.
So the integral from, say 1 to x of dt / t ^2.
Let's try this one here.
So here's an example of this theorem, I claim.
Now, this is a question which challenges your ability to understand what the question means.
Because it's got a lot of symbols.
It's got the integration and it's got the differentiation.
However, what it really is an exercise in recopying.
You look at it and you write down the answer.
And the reason is that, by definition, this function in here is a function of the form G ( x) of the theorem over here.
So this is the G ( x).
And by definition, we said that G' ( x) = f(x).
Well, what's the f ( x)?
Look inside here.
It's what's called the integrand.
This is the integral from 0 to x of f ( t ) dt, right?
Where f ( t) = 1 / t^2.
So your ability is challenged.
You have to take that 1 / t ^2 and you have to plug in the letter x, instead of t for it.
And then write it down.
As I say, this is an exercise in recopying what's there.
So this is quite easy to do, right?
I mean, you just look and you write it down.
But nevertheless, it looks like a long, elaborate object here.
Pardon me?
[INAUDIBLE]
So the question was, why did I integrate
[INAUDIBLE]
Why did I not integrate?
Ah.
Very good question.
Why did I not integrate.
The reason why I didn't integrate is I didn't need to.
Just as when you take the - sorry, the derivative of something, you take the antiderivative, you get back to the thing.
So, in this case, we're taking the antiderivative of something and we're differentiating.
So we end back in the same place where we started.
We started with f ( t), we're ending with f. Little f. So you integrate, and then differentiate.
And you get back to the same place.
Now, the only difference between this and the other version is, in this case when you differentiate and integrate you could be off by a constant.
That's what that shift, why there are two pieces to this one.
But there's never an extra piece here.
There's no + c here.
When you integrate and differentiate, you kill whatever the constant is.
Because the derivative of a constant is 0.
So no matter what the constant is hiding inside of g, you're getting the same result.
So this is the basic idea.
Now, I just want to double-check it, using the Fundamental Theorem of Calculus 1 here.
So let's actually evaluate the integral.
So now I'm going to do what you've suggested, which is I'm just going to check whether it's true.
No, no I am because I'm going just double-check that it's consistent.
It certainly is slower this way, and we're not going to want to do this all the time, but we might as well check one.
So this is our integral.
And we know how to do it.
No, I need to do it.
And this is - t ^ - 1, evaluated at 1 and x.
Again, there's something subliminally here for you to think about.
Which is that, remember, its t is ranging between 1 and t = x.
And this is one of the big reasons why this letter t has to be different from x.
Because here it's 1 and there it's x.
It's not x.
So you can't put an x here.
Again, this is t = 1 and this is t = x over there.
And now if I plug that in, I get what?
I get - 1 / x, and then I get - ( - 1).
So this is, let me get rid of those little t's there.
This is a little easier to read.
And so now let's check it.
It's d / dx.
So here's what G ( x) is.
G ( x) = 1 - 1 / x.
That's what G( x) is.
And if I differentiate that, I get + 1 / x ^2.
That's it.
You see the constant washed away.
So now, here's my job.
My job is to prove these theorems.
I never did prove them for you.
So, I'm going to prove the Fundamental Theorem of Calculus.
But I'm going to do 2 first.
And then I'm going to do 1.
And it's just going to take me just one blackboard.
It's not that hard.
The proof is by picture.
And, using the interpretation as area under the curve.
So if here's the value of a, and this is the graph of the function y equals f of x.
Then I want to draw three vertical lines.
One of them is going to be at x.
And one of them is going to be at x + delta x.
So here I have the interval from 0 to x, and next I have the interval from x to delta x more than that.
And now the pieces that I've got are the area of this part.
So this has area which has a name.
It's called G ( x ).
By definition, G( x ), which is sitting right over here in the fundamental theorem, is the integral from a to x of this function.
So it's the area under the curve.
So that area is G ( x ).
Now this other chunk here, I claim that this is delta G. This is the change in G. It's the value of G ( x ) that is the area of the whole business all the way up to x + delta x - the first part, G ( x ).
So it's what's left over.
It's the incremental amount of area there.
And now I am going to carry out a pretty standard estimation here.
This is practically a rectangle.
And it's got a base of delta x, and so we need to figure out what its height is.
This is delta G, and it's approximately its base times its height.
But what is the height?
Well, the height is maybe either this segment or this segment or something in between.
But they're all about the same.
So I'm just going to put in the value at the first point.
That's the left end there.
So that's this height here, is f ( x ).
So this is f (x), and so really I approximate it by that rectangle there.
And now if I divide and take the limit, as delta x goes to 0 of delta G / delta x, it's going to equal f ( x ).
And this is where I'm using the fact that f is continuous.
Because I need the values nearby to be similar to the value in the limit.
OK, that's the end.
This the end of the proof, so I'll put a nice little q.e.d.
here.
So we've done Fundamental Theorem of Calculus 2, and now we're ready for Fundamental Theorem of Calculus 1.
So now I still have it on the blackboard to remind you.
It says that the integral of the derivative is the function, at least the difference between the values of the function at two places.
So the place where we start is with this property that F' = f. That's the starting, that's the hypothesis.
Now, unfortunately, I'm going to have to assume something extra in order to use the Fundamental Theorem of Calculus 2, which is I'm going to assume that f is continuous.
That's not really necessary, but that's just a very minor technical point, which I'm just going to ignore.
So we're going to start with F' = f. And then I'm going to go somewhere else.
I'm going to define a new function, G (x), which is the integral from a to x of f ( t ) dt.
This is where we needed all of the labor of Riemann's sums.
Because otherwise we don't have a way of making sense out of what this even means.
So hiding behind this one sentence is the fact that we actually have a number.
We have a formula for such functions.
So there is a function g (x) which, once you've produced a little f for me, I can cook up a function capital G for you.
Now, we're going to apply this Fundamental Theorem of Calculus 2, the one that we've already checked.
So what does it say?
It says that G' = f. And so now we're in the following situation.
We know that F' ( x ) = G' ( x).
That's what we've got so far.
And now we have one last step to get a good connection between F and G. Which is that we can conclude that F ( x ) = G ( x) + c. Now, this little step may seem innocuous but I remind you that this is the spot that requires the mean value theorem.
So in order not too lie to you, we actually tell you what the underpinnings of all of calculus are.
And they're this: the fact, if you like, that if two functions have the same derivative, they differ by a constant.
Or that if a function has derivative 0, it's a constant itself.
Now, that is the fundamental step that's needed, the underlying step that's needed.
And, unfortunately, there aren't any proofs of it that are less complicated than using the mean value theorem.
And so that's why we talk a little bit about the mean value theorem, because we don't want to lie to you about what's really going on.
Yes
[INAUDIBLE]
The question is how did I get from here, to here.
And the answer is that if G' is little f, and we also know that F' is little f, then F' is G'.
OK. Other questions?
Alright, so we're almost done.
I just have to work out the arithmetic here.
So I start with F(b) - F ( a).
And that's equal to (G ( b) + c) - (G (a) + c).
And then I cancel the c's.
So I have here G(b) - G(a).
And now I just have to check what each of these is.
So Remember the definition of G here.
G ( b) is just what we want.
The integral from a to b of f(x) dx.
Well I called it f ( t) dt, that's the same as f(x) dx now, because I have the limit being b and I'm allowed to use x as the dummy variable.
Now the other one, I claim, is 0.
Because it's the integral from a to a.
This one is the integral from a to a.
Which gives us 0.
So this is just this - 0, and that's the end.
That's it.
I started with F( b) - F( a), I got to the integral.
Question?
[INAUDIBLE]
How did I get from F ( b) - F (a), is (G ( b) + c) - (G( a) + c), that's the question
[INAUDIBLE]
Oh, sorry this is an equals sign.
Sorry, the second line didn't draw.
OK, equals.
Because we're plugging in for f (x) the formula for it.
Yes
[INAUDIBLE]
This step here?
Or this one?
there's
[INAUDIBLE]
Right.
So that was a good question.
But the answer is that that's the statement that we're aiming for.
That's the Fundamental Theorem of Calculus 1, which we don't know yet.
So we're trying to prove it, and that's why we haven't, we can't assume it.
OK, so let me just notice that in the example that we had, before we go on to something else here.
In the example above, what we had was the following thing.
We had, say, F ( x ) = - 1 / x.
So F' (x) = 1 / x ^2.
And, say, G ( x) = 1 - (1 / x).
And you can see that either way you do that, if you integrate from 1 to 2, let's say, which is what we had over there, dt / t ^2, you're going to get either - 1 / t, 1 to 2 or, if you like, 1 - (1 / t), 1 to 2.
So this is the F version, this is the G version.
And that's what plays itself out here, in this general proof.
Alright.
So now I want to go back to the theme for today, which is using little f to understand capital F. In other words, using the derivative of f to understand capital F. And I want to illustrate it by some more complicated examples.
So I guess I just erased it, but we just took the antiderivative of 1 / t ^2.
And there's all of the powers work easily.
But one, and the tricky one is the power 1 / x.
So let's consider the differential equation L' ( x) = 1 / x.
And say, with the initial value L (1) = 0.
The solution, so the Fundamental Theorem of Calculus 2 tells us the solution is this function here.
L( x) equals the integral from 1 to x, dt / t. That's how we solve all such equations.
We just integrate, take the definite integral.
And I'm starting at 1 because I insisted that L( 1 ) be 0.
So that's the solution to the problem.
And now the thing that's interesting here is that we started from a polynomial.
Or we started from a rational, a ratio of polynomials; that is, 1 / t or 1 / x.
And we get to a function which is actually what's known as a transcendental function.
It's not an algebraic function.
Yeah, question
[INAUDIBLE]
The question is why is this equal to that.
And the answer is, it's for the same reason that this is equal to that.
It's the same reason as this.
It's that the 1's cancel.
We're taken the value of something at 2 minus the value at 1.
The value at 2 minus the value at 1.
And you'll get a 1 in the one case, and you get a 1 in the other case.
And you subtract them and they will cancel.
They'll give you 0.
These two things really are equal.
This is not a function evaluated at one place, it's the difference between the function evaluated at 2 and the value at 1.
And whenever you subtract two things like that, constants drop out
[INAUDIBLE]
That's right.
If I put 2, here if I put c here, it would have been the same.
It would just have dropped out.
It's not there.
And that's exactly this arithmetic right here.
It doesn't matter which antiderivative you take.
When you take the differences, the c's will cancel.
You always get the same answer in the end.
That's exactly why I wrote this down, so that you would see that.
It doesn't matter which one you do.
So, we still have a couple of minutes left here.
This is actually, so let me go back.
So here's the antiderivative of 1 / x, with value 1 at 0.
Now, in disguise, we know what this function is.
We know this function is the logarithm function.
But this is actually a better way of deriving all of the formulas for the logarithm.
This is a much quicker and more efficient way of doing it.
We had to do it by very laborious processes.
This will allow us to do it very easily.
And so, I'm going to do that next time.
But rather than do that now, I'm going to point out to you that we can also get truly new functions.
OK, so there are all kinds of new functions.
So this is the first example of this kind would be, for example, to solve the equation y' = e ^ - x^2 with y( 0 ) = 0, let's say.
Now, the solution to that is a function which again I can write down by the fundamental theorem.
It's the integral from 0 to x of e ^ - t^2 dt.
This is a very famous function.
This shape here is known as the bell curve.
And it's the thing that comes up in probability all the time.
This shape e ^ - x^2.
And our function is geometrically just the area under the curve here.
This is F (x).
If this place is x.
So I have a geometric definition, I have a way of constructing what it is by Riemann's sums.
And I have a function here.
But the curious thing about F ( x ) is that F ( x ) cannot be expressed in terms of any function you've seen previously.
So logs, exponentials, trig functions, cannot be.
It's a totally new function.
Nevertheless, we'll be able to get any possible piece of information we would want to, out of this function.
It's perfectly acceptable function, it will work just great for us.
Just like any other function.
Just like the ln.
And what this is analogous to is the following kind of thing.
If you take the circle, the ancient Greeks, if you like, already understood that if you have a circle of radius 1, then its area is pi.
So that's a geometric construction of what you could call a new number.
Which is outside of the realm of what you might expect.
And the weird thing about this number pi is that it is not the root of an algebraic equation with rational coefficients.
It's what's called transcendental.
Meaning, it's just completely outside of the realm of algebra.
And, indeed, the logarithm function is called a transcendental function, because it's completely out of the realm of algebra.
It's only in calculus that you come up with this kind of thing.
So these kinds of functions will have access to a huge class of new functions here, all of which are important tools in science and engineering.
So, see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
One correction from last time.
Sorry to say, I forgot a very important factor when I was telling you what an average value is.
If you don't put in that factor, it's only half off on the exam problem that will be given on this.
So I would have gotten half off for missing out on this factor, too.
So remember you have to divide by n here, certainly when you're integrating over 0 to n, the Riemann sum is the numerator here.
And if I divide by n on that side, I've got to divide by n on the other side.
This was meant to illustrate this idea that we're dividing by the total here.
And we are going to be talking about average value in more detail.
Not today, though.
So this has to do with average value.
And we'll discuss it in considerable detail in a couple of days, I guess.
Now, today I want to continue.
I didn't have time to finish my discussion of the Fundamental Theorem of Calculus 2.
And anyway it's very important to write it down on the board twice, because you want to see it at least twice.
And many more times as well.
So let's just remind you, the second version of the Fundamental Theorem of Calculus says the following.
It says that the derivative of an integral gives you the function back again.
So here's the theorem.
And the way I'd like to use it today, I started this discussion last time.
But we didn't get into it.
And this is something that's on your problem set along with several other examples.
Is that we can use this to solve differential equations.
And in particular, for example, we can solve the equation y' = 1 / x with this formula.
Namely, using an integral.
L ( x ) is the integral from 1 to x of dt / t. The function f ( t ) is just 1 / t. Now, that formula can be taken to be the starting place for the derivation of all the properties of the logarithm function.
So what we're going to do right now is we're going to take this to be the definition of the logarithm.
And if we do that, then I claim that we can read off the properties of the logarithm just about as easily as we could before.
And so I'll illustrate that now.
And there are a few other examples of this where somewhat more unfamiliar functions come up.
This one is one that in theory we know something about.
The first property of this function is the one that's already given.
Namely, its derivative is 1 / x.
And we get a lot of information just out of the fact that its derivative is 1 / x.
The other thing that we need in order to nail down the function, besides its derivative, is one value of the function.
Because it's really not specified by this equation, only specified up to a constant by this equation.
But we nail down that constant when we evaluate it at this one place, L ( 1).
And there we're getting the integral from 1 to 1 of dt / t, which is 0.
And that's the case with all these definite integrals.
If you evaluate them at their starting places, the value will be 0.
And together these two properties specify this function L (x), uniquely.
Now, the next step is to try to think about what its properties are.
And the first approach to that, and this is the approach that we always take, is to maybe graph the function, to get a feeling for it.
And so I'm going to take the second derivative.
Now, notice that when you have a function which is given as an integral, its first derivative is really easy to compute.
And then its second derivative, well, you have to differentiate whatever you get.
So it may or may not be easy.
But anyway, it's a lot harder in the case when I start with a function to get to the second derivative.
Here it's relatively easy.
And these are the properties that I'm going to use.
I won't really use very much more about it than that.
And qualitatively, the conclusions that we can draw from this are, first of all, from this, for example we see that this thing is concave down every place.
And then to get started with the graph, since I see I have a value here, which is L( 1) = 0, I'm going to throw in the value of the slope.
So L' ( 1), which I know is 1 of 1, that's reading off from this equation here, so that's 1.
And now I'm ready to sketch at least a part of the curve.
So here's a sketch of the graph.
Here's the point (1, 0); that is, x = 1, y = 0.
And the tangent line, I know, has slope 1.
And the curve is concave down.
So it's going to look something like this.
Incidentally, it's also increasing.
And that's an important property, it's strictly increasing.
That's because L' ( x) is positive.
And so, we can get from this the following important definition.
Which, again, is working backwards from this definition.
We can get to where we started with a log in our previous discussion.
Namely, if I take the level here, which is y = 1, then that crosses the axis someplace.
And his point is what we're going to define as e. So the definition of e is that it's the value such that L ( e) = 1.
And again, the fact that there's exactly one such place just comes from the fact that this L' is positive, so that L is increasing.
No, there's just one other feature of this graph that I'm going to emphasize to you.
There's one other thing which I'm not going to check, which you would ordinarily do with graphs.
Once it's increasing there are no critical points, so the only other interesting thing is the n's.
And it turns out that the limit as you go down to 0 is minus infinity.
As you go over to the right here it's plus infinity.
It does get arbitrarily high; it doesn't level off.
But I'm not going to discuss that here.
Instead, I'm going to just remark on one qualitative feature of the graph, which is this remark that the part which is to the left of 1 is below 0.
So I just want to remark, why is L (x) negative for x < 1.
Maybe I don't have room for that, so I'll just put in here x < 1.
I want to give you two reasons.
Again, we're only working from very first principles here.
Just that the property that L' = 1 / x, and L (1) = 0.
So our first reason is that, well, I just said it.
L ( 1) = 0.
And L is increasing.
And if you read that backwards, if it gets up to 0 here, it must have been negative before 0.
So this is one way of seeing that L ( x) is negative.
There's a second way of seeing it, which is equally important.
And it has to do with just manipulation of integrals.
Here I'm going to start out with L (x), and its definition.
Which is the integral from 1 to x dt / t. And now I'm going to reverse the order of integration.
This is the same, by our definition of our properties of integrals, as the integral from x to 1 with a minus sign dt / t. Now, I can tell that this quantity is negative.
And the reason that I can tell is that this chunk of it here, this piece of it, is a positive number.
This part is positive.
And this part is positive because x < 1.
So the lower limit is less than the upper limit, and so this is interpreted.
The thing in the green box is interpreted as an area.
It's an area.
And so negative a positive quantity is negative, minus a positive quantity's negative.
So both of these work perfectly well as interpretations.
And it's just to illustrate what we can do.
Now, there's one more manipulation of integrals that gives us the fanciest property of the ln.
And that's the last one that I'm going to do.
And you have a similar thing on your homework.
So I'm going to prove that this is, as I say, the fanciest property of the log.
On your homework, by the way, you're going to check that L( 1 / x) = - L ( x).
But we'll do this one.
The idea is just to plug in the formula and see what it gives.
On the left-hand side, I have 1 to ab, dt / t. That's L ( ab).
And then that's certainly equal to the left-hand side.
And then I'm going to now split this into two pieces.
Again, this is a property of integrals.
That if you have an integral from one place to another, you can break it up into pieces.
So I'm going to start at 1 but then go to a.
And then I'm going to continue from a to ab.
So this is the question that we have.
We haven't proved this.
Well, this one is actually true.
If we want this to be true, we know by definition L ( ab) is this.
We know, we can see it, that L(a) is this.
So the question that this boils down to is, we want to know that these two things are equal.
We want to know that L ( b) is that other integral there.
So let's check it.
I'm going to rewrite the integral.
It's the integral from - sorry, from lower limit a to upper limit ab of dt / t. And now, again, to illustrate properties of integrals, the key property here that we're going to have to use is change of variables.
This is a kind of a scaled integral where everything is multiplied by a factor of a from what we want to get to this L ( b) quantity.
And so this suggests that we write down t = a u.
That's going to be our trick.
And if I use that new variable u, then the change in t, dt, is adu.
And as a result, I can write this as equal to an integral from, let's see, dt = adu.
And t = au.
So I've now substituted in for the integrand.
But on top of this, with definite integrals, we also have to check the limits.
And the limits work out as follows.
When t = a, that's the lower limit.
Let's just take a look.
t = a u.
So that means that u is equal to, what?
It's 1.
Because a * 1 = a.
So if t = a, this is if and only if.
So this lower limit, which really in disguise was where t = a, becomes where u = 1.
And similarly, when t = ab, u = b.
So the upper limit here is b.
And now, if you notice, we're just going to cancel these two factors here.
And now we recognize that this is just the same as the definition of L ( b).
Because L ( x) is over here in the box.
And the fact that I use the letter t there is irrelevant; it works equally well with the letter u.
So this is just L (b).
Which is what we wanted to show.
So that's an example, and you have one in your homework, which is a little similar.
Now, the last example, that I'm going to discuss of this type, I already mentioned last time.
Which is the function F ( x), which is the integral from 0 to x of e ^ -t^2 dt.
This one is even more exotic because unlike the logarithm it's a new function.
It really is not any function that you can express in terms of the functions that we know already.
And the approach, always, to these new functions is to think of what their properties are.
And the way we think of functions in order to understand them is to maybe sketch them.
And so I'm going to do exactly the same thing I did over here.
So, what is it that I can get out of this?
Well, immediately I can figure out what the derivative is.
I read it off from the fundamental theorem.
It's this.
I also can figure out the value at the starting place.
In this case, the starting place is 0.
And the value is 0.
And I should check the second derivative, which is also not so difficult to compute.
The second derivative is - 2x e ^ x^2.
And so now I can see that this function is increasing, because this derivative is positive, it's always increasing.
And it's going to be concave down when x is positive and concave up when x is negative.
Because there's a minus sign here, so the sign is negative.
This is less than 0 when x is positive and greater than 0 when x is negative.
And maybe to get started I'll remind you F( 0), 0.
It's also true that F' ( 0), that just comes right out of this, F' ( 0) = e ^ - 0^2 which is 1.
That means the tangent line again has slope 1.
We do this a lot with functions.
We normalize them so that their slopes of their tangent lines are 1 at convenient spots.
So here's the tangent line of slope 1.
We know this thing is concave down to the right and concave up to the left.
And so it's going to look something like this.
With an inflection point.
Right?
Now, I want to say one more, make one more remark about this function, or maybe two more remarks about this function, before we go on.
Really, you want to know this graph as well as possible.
And so there are just a couple more features.
And one is enormously helpful because it cuts in half all of the work that you have.
and that is the property that turns out that this function is i. Namely, F( - x) = - F(x).
That's what's known as an odd function.
Now, the reason why it's odd is that it's the antiderivative of something that's even.
This function in here is even.
And we nailed it down so that it was 0 at 0.
Another way of interpreting that, and let me show it to you underneath, is the following.
When we look at its derivative, its derivative, course, is the function e ^ x.
Sorry, e ^ - x ^2.
So that's this shape here.
And you can see the slope is 0, but fairly close to 0, but positive along here.
It's getting, this is its steepest point.
This is the highest point here.
And then it's leveling off again; the slope is going down, always positive.
This is the graph of F' = e ^ -x^2.
Now, the interpretation of the function that's up above is that the value here is the area from 0 to x.
So this is area F(x).
Maybe I'll color it in, decorate it a little bit.
So this area here is F ( x).
Now, I want to show you this odd property, by using this symmetry.
The graph here is even, so in other words, what's back here is exactly the same as what's forward.
But now there's a reversal.
Because we're keeping track of the area starting from 0 going forward.
That's positive.
If we go backwards, it's counted negatively.
So if we went backwards to - x, we'd get exactly the same as that green patch over there.
We'd get a red patch over here.
But it would be counted negatively.
And that's the property that it's odd.
You can also check this by properties of integrals directly.
That would be just like this process here.
So it's completely analogous to checking this formula over there.
So that's one of the comments I wanted to make about this.
And why does this save us a lot of time, if we know this is odd?
Well, we know that the shape of this branch is exactly the reverse, or the reflection, if you like, of the shape of this one.
What we want to do is flip it under the axis and then reflect it over that way.
And that's the symmetry property of the graph of F(x).
Now, the last property that I want to mention is what's happening with the ends.
And at the end there's an asymptote, there's a limit here.
So this is an asymptote.
And the same thing down here, which will be exactly because of the odd feature, this'll be exactly negative.
The opposite value over here.
And you might ask yourself, what level is this, exactly.
Now, that level turns out to be a very important quantity.
It's interpreted down here as the area under this whole infinite stretch.
It's all the way out to infinity.
So, let's see.
What do you think it is?
You're all clueless.
Well, maybe not all of you, you're just afraid to say.
So it's obvious.
It's the square root of pi/2.
That was right on the tip of your tongue, wasn't it?
Ah, yes
Right, so this is actually very un-obvious, but it's a very important quantity.
And it's an amazing fact that this thing approaches this number.
And it's something that people worried about for many years before actually nailing down.
And so what I just claimed here is that the limit as x approaches infinity of F ( x) = the square root of pi / 2.
And similarly, if you do it to minus infinity, you'll get minus square root of pi/2.
And for this reason, people introduce a new function because they like the number 1.
This function is erf, short for error function.
And it's 2 / the square root of pi times the integral from 0 to x, e ^ - t ^2 dt.
In other words, it's just our original, our previous function multiplied by 2 / the square root of pi.
And that's the function which gets tabulated quite a lot.
You'll see it on the internet everywhere, and it's a very important function.
There are other normalizations that are used, and the discussions of the other normalizations are in your problems.
This is one of them, and another one is in your exercises.
The standard normal distribution.
There are tons of functions like this, which are new functions that we can get at once we have the tool of integrals.
And I'll write down just one or two more, just so that you'll see the variety.
Here's one which is called a Fresnel integral.
On your problem set next week, we'll do the other Fresnel integral, we'll look at this one.
These functions cannot be expressed in elementary terms.
The one on your homework for this week was this one.
This one comes up in Fourier analysis.
And I'm going to just tell you maybe one more such function.
There's a function which is called Li ( x), logarithmic integral of x, which is this guy.
The reciprocal of the logarithm, the natural log.
And the significance of this one is that Li ( x) is approximately equal to the number of primes < x.
And, in fact, if you can make this as precise as possible, you'll be famous for millennia, because this is known as the Riemann hypothesis.
Exactly how closely this approximation occurs.
But it's a hard problem, and already a century ago the prime number theorem, which established this connection was extremely important to progress in math.
Yeah, question
[INAUDIBLE]
Is this stuff you're supposed to understand.
That's a good question.
I love that question.
The answer is, this is, so we launched off into something here.
And let me just explain it to you.
I'm going to be talking a fair amount more about this particular function, because it's associated to the normal distribution.
And I'm going to let you get familiar with it.
What I'm doing here is purely cultural.
Well, after this panel, what I'm doing is purely cultural.
Just saying there's a lot of other beasts out there in the world.
And one of them is called C of [INAUDIBLE].
So we'll have a just a very passing familiarity with one or two of these functions.
But there are literally dozens and dozens of them.
The only thing that you'll need to do with such functions is things like understanding the derivative, the second derivative, and tracking what the function does.
Sketching the same way you did with any other tool.
So we're going to do this type of thing with these functions.
And I'll have to lead you through.
If I wanted to ask you a question about one of these functions, I have to tell you exactly what I'm aiming for.
Yeah, another question
[INAUDIBLE]
Yeah, I did.
I called these guys Fresnel integrals.
The guy's name is Fresnel.
It's just named after a person.
But, and this one, Li's logarithmic integral, it's not named after a person.
Logarithm is not somebody's name.
So look, in fact this will be mentioned also on a problem set, but I don't expect you to remember these names.
In particular, that you definitely don't want to try to remember.
Yes, another question
[INAUDIBLE]
The question is, will we prove this limit.
And the answer is yes, if we have time.
It'll be in about a week or so.
We're not going to do it now.
It takes us quite a bit of work to do it.
OK.
I'm going to change years now, I'm going to shift gears.
And we're going to go back to a more standard thing which has to do with just setting up integrals.
And this has to do with understanding where integrals play a role, and they play a role in cumulative sums, in evaluating things.
This is much more closely associated with the first Fundamental Theorem.
That is, we'll take, today we were talking about how integrals are formulas for functions.
Or solutions to differential equations.
We're going to go back and talk about integrals as being the answer to a question as opposed to.
what we've done now.
So in other words, and the first example, or most of the examples for now, are going to be taken from geometry.
Later on we'll get to probability.
And the first topic is just areas between curves.
Here's the idea.
If you have a couple of curves that look like this and maybe like this, and you want to start at a place a and you want to end at a place b, then you can chop it up the same way we did with Riemann sums.
And take a chunk that looks like this.
And I'm going to write the thickness of that chunk.
Well, let's give these things names.
Let's say the top curve is f(x), and the bottom curve is g ( x).
And then this thickness is going to be dx.
That's the thickness.
And what is the height?
Well, the height is the difference between the top value and the bottom value.
So here we have (f ( x) - g ( x)) dx.
This is, if you like, base times - whoops, backwards.
This is height, and this is the base of the rectangle.
And these are approximately correct.
But of course, only in limit when this is an infinitesimal, is it exactly right.
In order to get the whole area, I have add these guys up.
So I'm going to integrate from a to b.
That's summing them, that's adding them up.
And that's going to be my area.
So that's the story here.
Now, let me just say two things about this.
First of all, on a very abstract level before we get started with details of more complicated problems.
The first one is that every problem that I'm going to be talking about from now on for, several days, involves the following collection of, the following goals.
I want to identify something to integrate.
That's called an integrand.
And I want to identify what are known as the limits.
The whole game is simply to figure out what a, b, and this quantity is here.
Whatever it is.
And the minute we have that, we can calculate the integral if we like.
We have numerical procedures or maybe we have analytic procedures, but anyway we can get at the integral.
The goal here is to set them up.
And in order to set them up, you must know these three things.
The lower limit, the upper limit, and what we're integrating.
If you leave one of these out, it's like the following thing.
I ask you what the area of this region is.
If I left out this end, how could I possibly know?
I don't even know where it starts, so how can I figure out what this area is if I haven't identified what the left side is.
I can't leave out the bottom.
It's sitting here, in this formula.
Because I need to know where it is.
And I need to know the top and I need to know this side.
Those are the four sides of the figure.
If I don't incorporate them into the information, I'll never get anything out.
So I need to know everything.
And I need to know exactly those things, in order to have a formula for the area.
Now, when this gets carried out in practice, as we will do now in our first example, it's more complicated than it looks.
So here's our first example: Find the area between x = y ^2 and y = x - 2.
This is our first example.
Let me make sure that I chose the example that I wanted to.
Yeah.
Now, there's a first step in figuring these things out.
And this is that you must draw a picture.
If you don't draw a picture you'll never figure out what this area is, because you'll never figure out what's what between these curves.
The first curve, y = x ^2, is a parabola.
But x is a function of y.
It's pointing this way.
So it's this parabola here.
That's y = x^2.
Whoops, x = y ^2.
The second curve is a line, a straight line of slope 1, starting at (x = 2, y = 0).
It goes through this place here, which is 2 over and has slope 1, so it does this.
And this shape in here is what we mean by the area between the curves.
Now that we see what it is, we have a better idea of what our goal is.
If you haven't drawn it, you have no hope.
Now, I'm going to describe two ways of getting at this area here.
And the first one is motivated by the shape that I just described right here.
Namely, I'm going to use it in a straightforward way.
I'm going to chop things up into these vertical pieces just as I did right there.
Now, here's the difficulty with that.
The difficulty is that the upper curve here has one formula but the lower curve shifts from being a part of the parabola to being a part of the straight line.
That means that there are two different formulas for the lower function.
And the only way to accommodate that is to separate this up into two halves.
Separate it out into two halves.
I'm going to have to divide it right here.
So we must break it into two pieces and find the integral of one half and the other half.
Question?
[INAUDIBLE]
So, you're one step ahead of me.
We'll also have to be sure to distinguish between the top branch and the bottom branch of the parabola, which we're about to do.
Now, in order to distinguish what's going on I actually have to use multi colors here.
And so we will do that.
First there's the top part, which is orange.
That's the top part.
I'll call it top.
And then there's the bottom part, which has two halves.
They are pink, and I guess this is blue.
Alright, so now let's see what's happening.
The most important two points that I have to figure out in order to get started here.
Well, really I'm going to have to figure out three points, I claim.
I'm going to have to figure out where this point is.
Where this point is, and where that point is.
If I know where these three points are, then I have a chance of knowing where to start, where to end, and so forth.
Another question
[INAUDIBLE]
Could you speak up?
[INAUDIBLE]
The question is, why do we need to split up the area.
And I think in order to answer that question further, I'm going to have to go into the details of the method, and then you'll see where it's necessary.
So the first step is that I'm going to figure out what these three points are.
This one is kind of easy; it's the point (0, 0).
This point down here and this point up here are intersections of the two curves.
I can identify them by the following equation.
I need to see where these curves intersect.
At what, well, if I plug in x = y^2, I get y = y^2 - 2.
And then I can solve this quadratic equation.
y ^2 - y - 2 = 0.
So (y - 2)( y + 1) = 0.
And this is telling me that y = 2 or y = - 1.
So I've found y = - 1.
That means this point down here has second entry - 1.
Its first entry, its x value, I can get from this formula here or the other formula.
I have to square, this - 1 ^2 = 1.
So that's the formula for this point.
And the other point has second entry 2.
And, again, with his formula y = x ^2, I have to square y to get x, so this is 4.
Now, I claim I have enough data to get started.
But maybe I'll identify one more thing.
I need the top, the bottom left, and the bottom right.
The top is the formula for this branch of x = y ^2, which is in the positive y region.
And that is y = the square root of x.
The bottom curve, part of the parabola, so this is the bottom left, is y = minus square root x.
That's the other branch of the square root.
And this is exactly what you were asking before.
And this is, we have to distinguish between these two.
And the point is, these formulas really are different.
They're not the same.
Now, the last bit is the bottom right chunk here, which is this pink part.
Bottom right.
And that one is the formula for the line.
And that's y = x - 2.
Now I'm ready to find the area.
It's going to be in two chunks.
This is the left part, plus the right part.
And the left part, and I want to set it up as an integral, I want there to be a dx and here I want to set up an integral and I want it to be dx.
I need to figure out what the range of x is.
So, first I'm going to - well, let's leave ourselves a little more room than that.
Just to be safe.
OK, here's the right.
So here we have our dx.
Now, I need to figure out the starting place and the ending place.
So the starting place is the leftmost place.
The leftmost place is over here.
And x = 0 there.
So we're going to travel from this vertical line to the green line.
Over here.
And that's from 0 to 1.
And the difference between the orange curve and the blue curve is what I call top and bottom left, over there.
So that is square root of x minus minus square root of x.
Again, this is what I call top, and this was bottom.
But only the left.
I claim that's giving me the left half of this, the left section of this diagram.
Now I'm going to do the right section of the diagram.
I start at 1.
The lower limit is 1.
And I go all the way to this point here.
Which is the last bit.
And that's going to be x = 4.
The upper limit here is 4.
And now I have to take the difference between the top and the bottom again.
The top is square root of x all over again.
But the bottom has changed.
The bottom is now the quantity (x - 2).
Please don't forget your parenthesis.
There's going to be minus signs and cancellations.
Now, this is almost the end of the problem.
The rest of it is routine.
We would just have to evaluate these integrals.
And, fortunately, I'm going to spare you that.
We're not going to bother to do it.
That's the easy part.
We're not going to do it.
But I'm going to show you that there's a much quicker way with this integral.
And with this area calculation.
Right now.
The quicker way is what you see when you see how long this is.
And you see that there's another device that you can use that looks similar in principle to this, but reverses the roles of x and y.
And the other device, which I'll draw over here, schematically.
No, maybe I'll draw it on this blackboard here.
So, Method 2, if you like, this was Method 1, and we should call it the hard way.
Method 2, which is better in this case, is to use horizontal slices.
Let me draw the picture, at least schematically.
Here's our picture that we had before.
And now instead of slicing it vertically, I'm going to slice it horizontally.
Like this.
Now, the dimensions have different names.
But the principle is similar.
The width, we now call dy.
Because it's the change in y.
And this distance here, from the left end to the right end, we have to figure out what the formulas for those things are.
So on the left, maybe I'll draw them color coded again.
So here's a left.
And, whoops, orange is right, I guess.
So here we go.
So we have the left, which is this green.
Is x = y ^2.
And the right, which is orange, is y = x - 2.
And now in order to use this, it's going to turn out that we want to write x as, we want to reverse roles.
So we want to write this as x is a function of y.
So we'll use it in this form.
And now I want to set up the integral for you.
This time, the area is equal to an integral in the dy variable.
And its starting place is down here.
And it's ending place is up there.
This is the lowest value of y, and this is the top value of y.
And we've already computed those things.
The lowest level of y is - 1.
So this is y = - 1.
And this top value is y = 2.
So this goes from - 1 to 2.
And now the difference is this distance here, the distance between the rightmost point and the leftmost point.
Those are the two dimensions.
So again, it's a rectangle but its horizontal is long and its vertical is very short.
And what are they?
It's the difference between the right and the left.
The right-hand is (y + 2), and the right-hand is y ^2.
So this is the formula
[INAUDIBLE]
What was the question?
Why is it right minus left?
That's very important.
Why is it right minus left?
And that's actually the point that I was about to make.
Which is this.
That y + 2, which is the right, is bigger than y ^2, which is the left.
So that means that y + 2 - y ^2 is positive.
If you do it backwards, you'll always get a negative number and you'll always get the wrong answer.
So this is the right-hand end minus the left-hand end gives you a positive number.
And it's not obvious, actually, where you are.
There's another double-check, by the way.
When you look at this quantity, you see that the ends pinch.
And that's exactly the crossover points.
That is, when y = - 1, y + 2 - y ^2 = 0.
And when y = 2, y + 2 - y ^2 = 0.
And that's not an accident, that's exactly the geometry of the shape that we picked out there.
So this is the technique.
Now, this is a much more routine integral.
I'm not going to carry it out, I'll just do one last step.
Which is that this is (y ^2 / 2) + 2y - (y ^3 / 3), evaluated at - 1 and 2.
Which, if you work it out, is 9/2.
So we're done for today.
And tomorrow we'll do more volumes, more things including three dimensions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, I'm going to continue the idea of setting up integrals.
And what we'll deal with is volumes by slices.
By slicing.
And it's lucky that this is after lunch.
Maybe it's after breakfast for some of you, because there's the typical way of introducing this subject is with a food analogy.
There's a lot of ways of slicing up food.
And we'll give a few more examples than just this one.
But, suppose you have, well, suppose you have a loaf of bread here.
So here's our loaf of bread, and I hope that looks a little bit like a loaf of bread.
It's supposed to be sitting on the kitchen counter ready to be eaten.
And in order to figure out how much bread there is there, one way of doing it is to cut it into slices.
Now, you probably know that bread is often sliced like this.
There are even machines to do it.
And with this setup here, I'll draw the slice with a little bit of a more colorful decoration.
So here's our red slice of bread.
It's coming around like this.
And it comes back down behind.
So here's our bread slice.
And what I'd like to figure out is its volume.
So first of all, there's the thickness of the bread.
Which is this dimension, the thickness is this dimension dx here.
And the only other dimension that I'm going to give, because this is a very qualitative analysis for now, is what I'll call the area.
And that's the area on the face of the slice.
And so the area of one slice, which I'll denote by delta V, that's a chunk of volume, is approximately the area times the change in x.
And in the limit, that's going to be something like this.
And maybe the areas of the slices vary.
There might be a little hole in the middle of the bread somewhere.
Maybe it gets a little small on one side.
So it might change as x changes.
And the whole volume you get by adding up.
So if you like, this is one slice.
And this is the sum.
And you should think of it in a sort of intuitive way as being analogous to the Riemann sum, where you would take each slice individually.
And that would look like this.
Alright, so that's just a superficial and intuitive way of looking at it.
Now, we're only going to talk about one kind of systematic slice.
It's already on your problem set, you had an example of a slice of some region.
But we're only going to talk systematically about something called solids of revolution.
The idea here is this.
Suppose you have some shape, some graph, which maybe looks like this.
And now I'm going to revolve it.
This is the x axis and this is the y axis.
In this case, I'm going to revolve it around the x axis.
If you do that, then the shape that you get is maybe like this.
If I can draw it a little bit.
It's maybe a football.
So that's the shape that you get if you take this piece of disk and you revolve it around.
If you had made this copy underneath, it still would have been the same region.
So we only pay attention to what's above the axis here.
So that's the basic idea.
Now, I'm going to apply the method of slices to figure out the volume of such regions and give you a general formula.
And then apply it in a specific case.
I want to take one little slice of this football, maybe a football wouldn't work too well, maybe we should go back to a loaf of bread.
Anyway, the key point is that you never really have to draw a 3-D picture.
And 3-D pictures are awful.
They're very hard to deal with.
And it's hard to visualize with them.
And one of the reasons why we're dealing with solids of revolution is that we don't have as many visualization problems.
So we're only going to deal with this.
And then you have to imagine from the two-dimensional cross-section what the three-dimensional picture looks like.
So we'll do a little bit of an exercise with the three-dimensional picture.
But ultimately, you should be used to, getting used to, drawing 2-D diagrams always.
To depict the three-dimensional situation.
Since it's much harder to draw.
The first step is to consider what this slice is over here.
And again it's going to have width dx.
And we're going to consider what it looks like over on the 3-D picture.
So it starts out being more or less like this.
But then we're going to sweep it around.
We're revolving around the x axis, so it's spinning around this way.
And if you take this and think of it as being on a hinge, which is down on the x axis, it's going to swing down and swoop around and come back.
Swing around.
And that traces out something over here.
Which I'm going to draw this way.
It traces out a disk.
So it's hard for me to draw, and I'm not going to try too hard.
Maybe drew it more like a wheel, looking like a wheel.
But anyway, it's this little flat disk here.
And so the method that I'm describing for figuring out the volume is called the method of disks.
This is going to be our first method.
Now I'm going to apply the reasoning that I have up on the previous blackboard here.
Namely, I need to get the volume of this chunk.
And the way I'm going to get the volume of this chunk is by figuring out its thickness and its area, its cross-sectional area.
And that's not too difficult to do.
If this height, so this height is what we usually call y.
And y is usually a function of x, it's varying.
And this particular distance is y.
Then the area of the face is easy to calculate.
Because it's a circle, or a disk if you like, with this radius.
So its area is pi y^2.
So that's, if you like, one of the dimensions.
And then the thickness is dx.
So the incremental volume is this.
So this is the method of disks.
And this is the integrand.
Now, there's one peculiar thing about this formula.
And there are more peculiar things about this formula.
But there's one peculiar thing that you should notice immediately.
Which is that I'm integrating with respect to x.
And I haven't yet told you what y is.
Well, that will depend on what function y = f(x) I use.
So we have to plug that in eventually.
If we're actually going to calculate something, we're going to have to figure that out.
There's another very important point which is that in order to get a definite integral, something I haven't mentioned, we're going to have to figure out where we're starting and we're ending the picture.
Which is something we dealt with last time in 2-D pictures.
So let's deal with an example.
And we'll switch over from a football to a soccer ball.
I'm going to take a circle, and we'll say it has radius a.
So this is 0 and this is a.
And I'm putting it in this particular spot for a reason.
You can do this in lots of different ways, but I'm picking this one to make a certain exercise on your homework easier for you.
Because I'm doing half of it for you right now.
Appreciate it, yeah.
I'm sure especially today it's appreciated.
So again, the formula has to do with keeping track of these slices here.
And we're sweeping things around.
So the full region that we're talking about is the volume of the ball of radius a.
That's what our goal is, to figure out what the volume of the ball of radius a is.
Alright, again as I say, the thing sweeps around.
Coming out of the blackboard, spinning around on this x axis.
So the setup is the following.
It's always the same.
Here's our formula.
And we need to figure out what's going on with that formula.
And so we need to solve for y as a function of x.
And in order to do that, what we're going to do is just write down the equation for the circle.
This is the circle.
It's centered at (a, 0), so that's its formula.
And now there's one nice thing, which is that we really didn't need it to find the formula for y, we only needed to find the formula for y squared.
So let's just solve for y^2 and we won't have to solve a quadratic or anything.
Take a square root, that's nice.
This is a^2 - (x^2 - 2ax + a^2).
The a^2's cancel.
These two terms cancel.
And so the formula here is 2ax - x^2.
Alright now, that is what's known as the integrant.
Well, except for this factor of pi here.
And so the answer for the volume is going to be the integral of pi times this integrant, (2ax - x^2) dx.
And that's just the same thing as this.
But now there's also the issue of the limits.
Which is a completely separate problem, which we also have to solve.
The range of x is, from this leftmost point to the rightmost point.
So x varies, starts at 0, and it goes all the way up to what, what's the top value here.
2a.
And so now I have a completely specified integral.
Again, and this was the theme last time, the whole goal is to get ourselves to a complete formula for something with an integrant and limits.
And then we'll be able to calculate.
Now, we have clear sailing to the end of the problem.
So let's just finish it off.
We have the volume is, if I take the antiderivative of that, that's pi ax^2, whose derivative is 2ax, - x^3 / 3.
That's the thing whose derivative is - x ^2.
Evaluated at 0 and 2a.
And that is equal to pi times, let's see.
2^2 = 4.
So this is 4 a^3 - 8a^3 / 3, right?
And so all told, that is, let's see, 12/3 - 8/3 pi a^3.
Which is maybe a familiar formula, 4/3 pi a^3.
So it worked, we got it right.
Let me just point out a couple of other things about this formula.
The first one is that from this point of view, we've actually accomplished more then just finding the volume of the ball.
We've also found the volume of a bunch of intermediate regions, which I can draw schematically this way.
If I chop this thing, and this portion is x here, then the antiderivative here, this region here, which maybe I'll fill in with this region here.
Which I'm going to call V (x).
Is the volume of the portion of the sphere.
Volume of portion of width x of the ball.
And, well, the formula for it is that it's the volume = pi (ax ^2 - x ^3 / 3).
That's it.
So we've got something which is actually a lot more information.
For instance, if you plug in x = a, not surprisingly, and this is a good idea to do because it checks that we've actually got a correct formula here.
So if you like you can call this a double-check.
If you check V ( a), this should be the volume of a half-ball.
That's halfway.
If I go over here and I only go up to a, that's exactly half of the ball.
That had better be half, so let's just see.
V ( a) in this case is pi.
And then I have (a^3 - a ^3 / 3).
And that turns out to be pi times a total of 2/3 a ^3, which is indeed half.
Now, on your problem set, what you're going to want to look at is this full formula here.
Of this chunk.
And what it's going to be good for is a real life problem.
That is, a problem that really came up over the summer.
And this fall at a couple of universities near here, where people were trying to figure out a phenomenon which is well-known.
Namely, if you have a bunch of particles in a fluid, and maybe the size of these things is 1 micron.
That is, the radius is 1 micron.
And then you have a bunch of other little particles, which are a lot smaller.
Maybe 10 nanometers.
Then what happens is that the particles, the big particles, like to hug each other.
They like to clump together, they're very nice.
Friendly characters.
So what's the explanation for this?
The explanation is that actually they are not quite as friendly as they might seem.
What's really happening is that the little guys are shoving them around.
And pushing them together.
And they have sharp elbows, the little ones and they're pushing them.
Don't like them to be around and they're pushing them together.
But there's actually another possibility.
Which is that they also will stick to the sides of the container.
So there are two things that actually happen here.
And if you want to get a quantitative handle on how much of this happens, it has to do with how much space these things take up.
And so the issue is some kind of overlap between a band around one sphere and a band around the other sphere.
And this overlap region is what you have to calculate.
You have to calculate what's in here.
And you can do that using this formula here.
It's not even difficult.
So this is, if you cut it in half, turns out to be two of these guys.
And then you're on your way to figuring out this problem.
And the question is, which do they prefer.
Do they prefer to touch each other, or do they prefer to touch the wall.
Do they all cluster to the wall.
So you can actually see this in solutions, what they do.
And the question is, to what extent do they prefer one configuration to the other.
So that's a real live problem, which really comes up.
Came up just this year.
And frequently comes up.
Which is solved by our first calculation.
So now I have, I called this Method 1.
For solids of revolution, which is called the method of disks.
And now I need to tell you about the other standard method.
Which is called the method of shells.
So this is our second method.
I'm going to illustrate this one with a holiday themed example here.
This is supposed to be a witches' cauldron.
Whoops, witch, witches, well.
Maybe more than one witch will have this cauldron here.
So here's this shape.
And we're going to figure out how much liquid is in here.
I'm going to plot this.
Maybe I'll put this down just a bit lower here.
And I'm going to make it a parabola.
This is y = x^2.
And I'm going to make the top height be y = a.
So here's my situation.
And I want to figure out how much liquid is in here.
Now, the reason why I presented the problem in this form, of course, is really to get you used to these things.
And the first new thing that I want you to get used to is the idea that now we can also revolve around the y axis, not just the x axis.
So this one is going to be revolved around the y axis.
And that's what happens.
If you spin the parabola around, you get this kind of shape, this kind of solid shape here.
Now, I'm going to use the same kind of slicing that I did before.
But it's going to look totally different.
Namely, I'll draw it in red again.
I'm going to take a little slice over here.
And now I want to imagine what happens if it gets revolved around the y axis.
This time it's not a disk.
Actually, if I revolve this around the other way, it would have had a hole in it.
Which is also possible to do.
That's practically the same as the method of disks.
You'll maybe discuss that in recitation.
Anyway, we're going to revolve it around this way.
So again, I need to sweep it around, swing it around like this.
And I'll draw the shape.
It's going to sweep around in a circle and maybe it'll have a little bit of thickness to it.
And this is the thing that people call a shell.
This is the so-called shell of the method.
I would maybe call it a cylinder, and another way of thinking about it is that you can maybe wrap up a piece of paper.
Like this.
There it is, there's a cylinder you can see.
Very thin, right?
Very thin.
Now, the reason why I used the piece of paper as an example of this is that I'm going to have to figure out the volume of this thing.
Its thickness, as usual, is equal to dx.
Its height is what?
Well, actually I have to use the diagram to see that.
The top value is a, and the bottom value is what we call y.
So the height is equal to, I'm sorry, a - y.
Now, again this is an incredibly risky thing here.
And I've done this before I pointed this out on the very first day of lecture.
The letter y represents a lot of different things.
And in disguise, when I call this y, I mean y = x^2.
In other words, the interesting curve.
I don't mean y = a, which is the other part.
In general, you might think of it as being equal to y top - y bottom.
So there are, of course, two y's involved.
In disguise.
But we have a shorthand, and the sort of uninteresting one we call by its, we just evaluate immediately, and the interesting one we leave as the symbol y.
Now, the last bit that I have to do, I claim, in order to figure this out, is the circumference.
And the reason for that is that if I think of this thing as like this tube, or piece of paper here.
In order to figure out how much stuff there is here, all I have to do is unfold it.
Its size, it's the whole quantity of paper here is the same whether it's rolled up like this or whether it's stretched out.
So if I unwrap it, it looks like what?
Well, it looks like kind of a slab?
Right, it looks like just a slab like this.
And again, the thickness is dx.
The height is a - y.
And now we can see that the length here is all the way around.
It's the circumference.
So this is going to be the circumference when I unwrap it.
And in order to figure out the circumference, I need to figure out the radius.
So the radius is, on this diagram, is right over there.
This is the radius.
And that distance is x.
So this length here is x.
And so this circumference is 2 pi x.
And this height is still ay.
a - y, sorry.
And the thickness is still dx.
So in total, we're just going to multiply these numbers together to get the total volume.
We have, in other words, dV = the product of the (2 pi x) dimension, the (8 - y) dimension, and the dx dimension.
Incidentally, dimensional analysis is very useful and important in these problems.
You can see that there are three lengths being multiplied together.
So we'll get a volume in end.
Something cubic.
And we will be coming back to that, because it's a quite subtle issue sometimes.
So here's the formula, and let's simplify it a little bit.
We have 2 pi x times, remember, first I have to substitute, otherwise I'm never going to be able to integrate.
And then I rewrite that as 2 pi (ax - x^2, whoops, x^3) dx.
Better not get that wrong.
And now the last little bit here, that I had better be careful about in order to figure out what the total volume is, is the limits.
So the volume is going to be the integral of this quantity 2 pi (ax - x ^3) dx.
And now I have to pay attention to what the limits are.
Now, here you have to be careful.
x is possibly the, you have to always go back to the 2-D diagram.
I went to it immediately, but that's the whole point.
Is that everything gets read off from this diagram here.
When you take this guy and you sweep it around, you take care of everything that's to the left.
So we only have to count what's to the right.
We don't have to count anything over here.
Because it's taken care of when we sweep around.
So the starting place is going to be x = 0.
That's where we start.
And where we end is the farthest, rightmost spot for x.
Which is down here.
You've got to watch out about where that is.
In the y variable, it's up at y = a.
But in the x variable, we can see that it's what?
It's the square root of a.
So these limits, this is where you're going to focus all your attention on the integrand and getting it just right.
And then you're going to lose your steam and not pay attention to the limits.
They're equally important.
You've no hope of getting the right answer without getting the limits right.
So this is the integral from 0 to square root of a.
Again, that's just because y = a and y = x^2 implies x = square root a.
That's that upper limit there.
And now, we're ready to carry out this, to evaluate this integral.
So we get 2 - sorry, we get 2 pi ax, that's pi ax^2.
Maybe I'll leave the 2's in there.
2 pi (ax^2 is the antiderivative of this ax^2 / 2, and then here x ^4 / 4), evaluated at 0 and square root of a.
And finally, let's see, what is that.
That's 2 pi ( a ^2 / 2 - a ^2 / 4).
Which is a total of 1/4, right, 2 pi ( a ^2 / 4).
Which is pi/ 2 a^2.
Yes, question
[INAUDIBLE]
Right.
So the question is, why did I integrate only from the middle to this end, instead of all the way from over here in minus square root of a, all the way to the plus end.
And the reason is that you have to look at what's happening with the rotation.
This red guy, when I swept it around, I counted the stuff to the right and the left.
So in other words, if I just rotate the right half of this, I'm covering the left half.
So if I counted the stuff from minus square root of 8, I would be doing it twice.
I would be doubling what I need.
So it's too much.
Another way of saying it is if I wanted to take the whole region, if I rotate it around only 180 degrees, only pi, that would fill up the whole region.
If I did both halves.
And then instead of a circumference, instead of a 2 pi x, I could use a pi x.
But then I would have double what I had.
So there are two ways of looking at it.
The same goes, actually, for the football case.
When I have that football, I didn't count the bottom part.
Because when I swung it around the x-axis, the top part sufficed and I could ignore the bottom half.
Yeah, another question
[INAUDIBLE]
Ooh, good question.
The question is, when do you know, how do you know when to take the rectangle to be vertical or horizontal.
So far we've only done vertical rectangles.
And I'm going to do a horizontal example in a second.
And the answer to the question of when you do it is this.
You can always set it up both ways.
One way may be a difficult calculation and one way may be an easier calculation.
Yesterday, we did it - or, sorry, the last time.
Yeah, I guess it was yesterday.
We did it with the horizontal and the vertical were quite different in character.
One of them was really a mess, and one of them was a little easier.
So very often, one will be easier than the other.
Every once in a while, one of them is impossible and the other one is possible.
In other words, the difference in difficulty can be extreme.
So you don't know that in advance.
Yeah, another question
[INAUDIBLE]
The question is, did we just find the volume when you rotate this green region around.
Or, did we find the volume when we rotate this whole region.
In other words, just the right half or the right and the left half.
The answer is both.
The region that you get is the same.
You always get this cauldron, whether you take this right half when you rotate it around or you take both and you rotate it around.
So the answer to both of those questions is the same and it's this.
Yes
[INAUDIBLE]
So that if you rotated them both around and you only wanted to cover things once, you would rotate halfway around.
Only by 180 degrees.
That's true.
But you can rotate around as many times as you want.
You're still covering the same thing.
Over and over and over and over again.
So let's go on.
I have a very subtle point that I need to discuss with you in order to go on to the next application.
So here's my, and I do want to get to the point of horizontal cross-sections as well.
So let's continue here.
So the first thing that I want to point out to you now is, I want you to beware of units.
There's something a little fishy in this problem.
And it can be summarized in the character of the answer, which is just a little bit not clear.
Namely, it looks like it's a^2, right?
And we know that a is in units and we should've gotten cubic units.
So there's something a little bit tricky about this question.
And so I want to illustrate the paradox right now.
So, suppose that a = 100 centimeters.
And suppose the units are centimeters.
Then the formula for the volume is pi / 2 ( 100)^2.
and the units we must take are centimeters cubed.
Despite the fact that you kind of want to square the centimeters.
But that's not what this problem says.
OK, so this is the situation that we've got.
Now, if you work out what this is, to figure out what the volume of this cauldron is, what you find is that it's pi / 2 times, well, 10^ 4 is 10 * 1,000 centimeters cubed.
And those are otherwise known as liters.
So this is approximately 10 pi / 2 liters, which is about 16 liters.
And so that's how much was in the cauldron under this choice of units.
Now, I'm going to make another choice of units now.
And we're going to make a comparison.
Suppose that the units are 1 meter.
Looks like it should be the same, but if I calculate the volume, it's going to be pi / 2, and 1^2 * m^3.
That's what the formula tells us to do.
And if you calculate that out, that's pi / 2 (100 cm)^3.
And with this unit notation we really do want to cube the centimeters and cube the 100.
So we get pi / 2.
And here we get 10 ^ 6 cm^3.
And that comes out to pi / 2 * 1000 liters.
Or, in other words, about 1600 liters.
So I'd like to ask you first to contemplate this.
And this is a paradox.
And this is a serious paradox.
If you really want to apply problems, you actually have to understand what your answers mean.
So what do you think is going on here?
Yeah
[INAUDIBLE]
Yes
[INAUDIBLE]
Right.
So the question is, how could either of these make sense.
How am I dealing with the units in either case.
So now I'm going to explain to you the answer.
Because this is really quite puzzling.
But it has a resolution.
The answer to this question is that both answers are correct.
This is correct reasoning in both cases.
What's the matter is that you have to interpret the equation y = x ^2 in two different ways.
Two ways.
So let me explain what they are.
For instance, you can take y = x^2 in centimeters.
So y = x ^2 in centimeters.
In which case, the picture looks like the following.
a = 100 centimeters.
And this distance here, which is the x value, this is 10.
This is 10 centimeters.
And that's what the relationship means.
So the top of the cauldron, if you like, this distance here is 20 centimeters.
This is actually very badly drawn to scale.
It's actually very, very, deep, this thing.
It's a rather skinny, deep one.
So this is very much not to scale, this picture.
The other picture, the other picture is interpreting y = x ^2 in meters.
And that's more like what I had in mind, actually.
I had in mind this big vat here.
And this distance here is 1 meter, and then the square root of 1 is 1.
So this distance here is also 1 meter.
And the top is 2 meters.
Now, it's not that crazy.
And in fact it's easy to check that, it's pretty reasonable in terms of scale.
That this thing has 16 liters in it.
And this guy has 1600 liters in it.
So you actually have to know what your symbols mean when you're dealing with these kinds of applied problems.
And if you're ever really going to do some real consequences, you have to know what the units mean.
And the problem with the equation y = x^2 is that it's the one that violated scaling rules.
Yeah
[INAUDIBLE]
Yeah
[INAUDIBLE]
No
[INAUDIBLE]
OK, so the question is whether the formula V = pi / 2a ^2.
This is the correct answer to the problem.
But it is not consistent in units.
If you plug in a equals some number of centimeters, some number of millimeters, some number of inches and so on, every single time you'll get a different answer.
And they're all inconsistent.
In other words, this formula violates scaling
[INAUDIBLE]
If you study each step correctly, you will discover that these are the consistent and correct statements, what I'm writing on the blackboard.
And this makes sense in a unit-less sense.
But then if you actually stick units on them, one of them, they both are correct.
And one of them describes this situation and one of them describes this situation.
And it's a mistake to think of this as being 1 meter and cubing the meters.
That will be an error that will cause you problems.
This 1 is just unit-less, and then the meters cubed got converted.
So I encourage you to study this on your own.
So now I'm going to introduce the next problem.
We'll have to solve it next time.
But the reason why I spent all the time on units is that otherwise it would be impossible for you to believe me when I do this next calculation.
Because we're trying to get a real answer out of a real question.
And I'm going to make conversions between centimeters and meters back and forth.
And we have to get it consistent in order to have the right answer.
So there was a reason for illustrating this pitfall.
So this second, the next thing that I'd like to do, is I'd like to boil the water in the witches' cauldron.
This is definitely seasonally appropriate, since we're approaching Halloween.
And we'll work it out fully next time.
Now, I'm going to introduce another feature into the problem.
And this is the one that I want you to understand now.
We'll set it all up tomorrow, but right now I need you to understand what the new main idea that we're going to get.
There is the new physical feature that I'm going to add to this problem is that if when you're boiling, when the witches are boiling this water, the temperature of the water is not the same at each level in the kettle.
At the bottom of the kettle, where you're heating it up, it's at its highest temperature.
So at the bottom it's going to be, say, 100 degrees.
That is, it's going to be totally boiling.
100 degrees Celsius.
And at the top, it's going to be, say, 70 degrees.
Right, it's very cold outside.
In fact, it's 0 degrees outside.
Which is the temperature at which all witches operate, I think.
Anyway, so they're boiling their stuff.
And the question that we're going to ask is how much heat, how much heat, do they need to do it.
Now, the thing starts out at 0 degrees Celsius.
And we're going to heat it up to this temperature configuration here.
But it's rising from 100 down here to 70 up here.
So the temperature is varying in height.
And for simplicity I'm going to make the formula for the temperature be 70 at the top and 100 at the bottom.
So it's going to be 100 - 30y.
We'll let the level be 1, here.
Sorry, 30.
I said 3, I wrote 3, but I meant 30.
So this is the situation that we have.
Now, the point about this problem is we're going to figure out the total temperature, the total amount of heat that you need to add in order to heat this thing up.
That's going to be temperature times volume.
But some places will count more than others.
These will be hotter, but there's less water down here.
This is wider up here, so there's more water up here.
So there are various things that are varying in this problem.
Now, the only way to set up so that it works is to chop things up horizontally instead of vertically.
Because it's on the horizontal levels that the temperature is constant.
So we'll have an easy calculation for how much it takes to heat up a layer, a horizontal layer.
When we rotate this guy around the y axis, that is which kind of shape.
It's a disk.
So actually this one's going to be an easier problem.
It's going to be a disk problem, not a shell problem.
But we're going to have to work things out with respect to the dy variation.
In other words, the integral will be with respect to dy.
So we will do that next time.
We'll figure out how much heat it takes to boil the kettle.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to hold off just a little bit on boiling water.
And talk about another application of integrals, and we'll get to the witches' cauldron in the middle.
The that I'd like to start with today is average value.
This is something that I mentioned a little bit earlier, and there was a misprint on the board, so I want to make sure that we have the definitions straight.
And also the reasoning straight.
This is one of the most important applications of integrals, one of the most important examples.
If you take the average of a bunch of numbers, that looks like this.
And we can view this as sampling a function.
As we would with Riemann's sum.
And what I said last week was that this tends to this expression here, which is called the continuous average.
So this guy is the continuous average.
Or just the average of f. And I want to explain that, just to make sure that we're all on the same page.
In general, if you have a function and you want to interpret the integral, our first interpretation was that it's something like the area under the curve.
But average value is another reasonable interpretation.
Namely, if you take equally spaced points here, starting with x0, x1, x2, all the way up to xn, which is the left point b, and then we have values y1, which = f ( x1), y2, which = f ( x2), all the way up to yn, which = f ( xn).
And again, the spacing here that we're talking about is b - a / n. So remember that spacing, that's going to be the connection that we'll draw.
Then the Riemann sum is y1 through yn, the sum of (y1 ... yn) delta x.
And that's what tends, as delta x goes to 0, to the integral.
.
The only change in point of view if I want to write this limiting property, which is right above here, the only change between here and here is that I want to divide by the length of the interval.
b - a.
So I will divide by b - a here.
And divide by b - a over here.
And then I'll just check what this thing actually is.
Delta x / b - a, what is that factor?
Well, if we look over here to what delta x is, if you divide by b - a, it's 1 / n. So the factor delta x / b - a = 1 / n. That's what I put over here, the sum of y1 through yn / n. And as this tends to 0, it's the same as n going to infinity.
Those are the same things.
The average value and the integral are very closely related.
There's only this difference that we're dividing by the length of the interval.
I want to give an example which is an incredibly simpleminded one, but it'll come into play later on.
So let's take the example of a constant.
And this is, I hope, will make you slightly less confused about what I just wrote.
As well as making you think that this is as simpleminded and reasonable as it should be.
If I check what the average value of this constant is, it's given by this relatively complicated formula here.
That is, I have to integrate the function c. Well, it's just the constant c. And however you do this, as an antiderivative I was thinking of it as a rectangle, the answer that you're going to get is c here.
So work that out.
The answer is c. And so the fact that the average of c = c, which had better be the case for averages, explains the denominator.
Explains the 1 / b - a there.
That's cooked up exactly so that the average of a constant is what it's supposed to be.
Otherwise we have the wrong normalizing factor.
We've clearly got a piece of nonsense on our hands.
And incidentally, it also explains the 1 / n in the very first formula that I wrote down.
The reason why this is called the average, or one reason why it's the right thing, is that if you took the same constant c, for y all the way across there n times, if you divide it by n, you get back c. That's what we mean by average value and that's why the n is there.
So that was an easy example.
Now none of the examples that we are going to give are going to be all that complicated.
But they will get sort of steadily more sophisticated.
The second example is going to be the average height of a point on a semicircle.
And maybe I'll draw a picture of the semicircle first here.
And we'll just make it the standard circle, the unit circle.
So maybe I should have called it a unit semicircle.
This is the point negative 1, this is the point 1.
And we're picking a point over here and we're going to take the typical, or the average, height here.
Integrating with respect to dx.
So sort of continuously with respect to dx.
Well, what is that?
Well, according to the rule, it's 1 / b - a times - sorry, it's up here in the box.
1 / b - a, the interval from a to b, f ( x) dx.
That's 1 / + 1 - (- 1).
The integral from - 1 to 1, square root of 1 - x ^2 dx.
Right, because the height is y = , this is y = the square root of 1 - x^2.
And to evaluate this is not as difficult as it seems.
This is 1/2 times this quantity here, which we can interpret as an area.
It's the area of the semicircle.
So this is the area of the semicircle, which we know to be half the area of the circle.
So it's pi / 2.
And so the answer, here the average height, is pi/ 4.
Now, later in the class and actually not in this unit, we'll actually be able to calculate the antiderivative of this.
So in other words, we'll be able to calculate this analytically.
For right now we just have the geometric reason why the value of this is pi / 2.
And we'll do that in the fourth unit when we do a lot of techniques of integration.
So here's an example.
Turns out, the average height of this is pi / 4.
Now, the next example that I want to give introduces a little bit of confusion.
And I'm not going to resolve this confusion completely, but I'm going to try to get you used to it.
I'm going to take the average height again.
But now, with respect to arc length.
Which is usually denoted theta.
Now, this brings up an extremely important feature of averages.
Which is that you have to specify the variable with respect to which the average is taking place.
And the answer will be different depending on the variable.
So it's not going to be the same.
Wow, can't spell the word length here.
Just like the plural of witches the last time.
We'll work on that.
We'll fix all of our, that's an ancient Gaelic word, I think.
Lengh.
So now, let me show you that it's not quite the same here.
It's especially exaggerated if maybe I shift this little interval dx over to the right-hand end.
And you can see that the little portion that corresponds to it, which is the d theta piece, has a different length from the dx piece.
And indeed, as you come down here, these very short portions of dx length have much longer portions of theta length.
So that the average that we're taking when we do it with respect to theta, is going to emphasize the low values more.
They're going to be more exaggerated.
And the average should be lower than the average that we got here.
So we should expect a different number.
And it's not going to be pi / 4, it's going to be something else.
Whatever it is, it should be smaller than pi / 4.
Now, let's set up the integral.
The integral follows the same rule.
It's just 1 over the length of the interval times the integral over the interval of the function.
That's the integral, but now where does theta range?
This time, theta goes from 0 to pi.
So the integral is from 0 to pi.
And the thing we divide by is pi.
And the integration requires us to know the formula for the height.
Which is sin theta.
In terms of theta, of course.
It's the same as square root of - x^2, but it's expressed in terms of theta.
So it's this.
And here's our average.
I'll put this up here.
So that's the formula for the height.
So let's work it out.
This one, we have the advantage of being able to work out because we know the antiderivative of sin theta.
It happens with this factor of pi, it's - cos theta.
And so, that's - 1 / pi ( cos pi), sorry.
(cos pi - cos) 0.
Which is - 1 / pi ( - 2), which is 2 / pi.
And sure enough, if you check it, you'll see that 2 / pi 8.
Yeah, question
[INAUDIBLE]
The question is how do I get sin theta.
And the answer is, on this diagram, if theta is over here then this height is this, and this is the angle theta, then the height is the sine.
OK. Another question
[INAUDIBLE]
The question is, what is the first one, the first one is an average of height, of a point on a semicircle and this one is with respect to x.
So what this reveals is that it's ambiguous to say what the average value of something is, unless you've explained what the underlying averaging variable is
[INAUDIBLE]
The next question is how should you interpret this value.
That is, what came out of this calculation?
And the answer is only sort of embedded in this calculation itself.
So here's a way of thinking of it which is anticipating our next subject.
Which is probability.
Which is, suppose you picked a number at random in this interval.
With equal likelihood, one place and another.
And then you saw what height was above that.
That would be the interpretation of this first average value.
And the second one is, I picked something at random on this circle.
And equally likely, any possible point on this circle according to its length.
And then I ask what the height of that point is.
And those are just different things.
Another question
[INAUDIBLE]
Cos pi, shouldn't it be 0?
No.
cos of, it's - 1.
Cos pi = - 1.
Cosine, sorry.
No, cos 0 = 1. cos pi = - 1.
And so they cancel.
That is, they don't cancel.
It's - 1 - 1, which = - 2.
Key point.
Yeah
[INAUDIBLE]
All right, let me repeat.
So the question was to repeat the reasoning by which I guessed in advance that probably this was going to be the relationship between the average value with respect to arc length versus the average value with respect to this horizontal distance.
And it had to do with the previous way this diagram was drawn.
Which is comparing an interval in dx with an interval in theta.
A little section in theta.
And when you're near the top, they're nearly this same.
That is, it's more or less balanced.
It's a little curved here, a little different.
But here it becomes very exaggerated.
The d theta lengths are much longer than the dx lengths.
Which means that importance given by the theta of variable to these parts of the circle is larger, relative to these parts.
Whereas if you look at this section versus this section for the dx, they give equal weights to these two equal lengths.
But here, with respect to theta, this is relatively short and this is much larger.
So, as I say, the theta variable's emphasizing the lower parts of the semicircle more.
That's because this length is shorter and this length is longer.
Whereas these two are the same.
It's a balancing act of the relative weights.
I'm going to say that again in a different way, and maybe this will.
The lower part is more important for theta
[INAUDIBLE]
So the question is, but shouldn't it have a bigger value because it's a longer length.
Never with averages.
Whatever the length is, we're always dividing.
We're always compensating by the total.
We have the integral from 0 to pi, but we're dividing by pi.
Here we had the integral from - 1 to 1, but we're dividing by 2.
So we divide by something different each time.
And this is very, very important.
It's that the average of a constant is that same constant regardless of which one we did.
So if it were a constant, we would always compensate for the length.
So the length never matters.
If it's the integral from 0 to 1,000,000, or 100, let's say, 1/100 cdx, it's just the same.
It's always that, it doesn't matter how long it is.
Because we compensate.
That's really the difference between an integral and an average, is that we're dividing by the total.
Now I want to introduce another notion, which is actually what's underlying these two examples that I just wrote down.
And this is by far the one which you should emphasize the most in your thoughts.
Because it is much more flexible, and is much more typical of real life problems.
So the idea of a weighted average is the following.
You take the integral, say from a to b, of some function.
But now you multiply by a weight.
And you have to divide by the total.
And what's the total going to be?
It's the integral from a to b of this total weighting that we have.
Now, why is this the correct notion?
I'm going to explain it to you in two ways.
The first is this very simpleminded thing that I wrote on the board there, with the constants.
What we want is the average value of c to be c. Otherwise this makes no sense as an average.
Now, let's just look at this definition here.
And see that that's correct.
If you integrate c, from a to b, w ( x) dx, and you divide by the integral from a to b, w ( x) dx, not surprisingly, the c factors out.
It's a constant.
So this is c times the integral a to b, w (x), dx, divided by the same thing.
And that's why we picked it.
We picked it so that these things would cancel.
And this would give c. So in the previous case, this property explains the denominator.
And again over here, it explains the denominator.
And let me just give you one more explanation.
Which is maybe a real life pretend real life example.
You have a stock which you bought for $10 one year.
And then six months later you brought some more for $20.
And then you bought some more for $30.
Now, what's the average price of your stock?
Well, it depends on how many shares you bought.
If you bought this many shares the first time, and this many shares the second time, and this many shares the third time, this is the total amount that you spent.
And the average price is the total price divided by the total number of shares.
And this is the discrete analog of this continuous averaging process here.
The function f now, so I use w for weight, the function f now is the function whose values are 10, 20 and 30.
And the weightings are the relative importance of the different purchases.
So again, these wi's are weights.
There was another question.
Out in the audience, at some point.
Over here, yes
[INAUDIBLE]
Very, very good point.
So in this numerator here, the statement is, in this example, we factored out c. But here we cannot factor out f ( x).
That's extremely important and that is the whole point.
So, in other words, the weighted average is very interesting you have to do two different integrals to figure it out in general.
When it happens that this is c, it's an extremely boring integral.
Which in fact because, it's an average, you don't even have to calculate at all.
Factor it out and cancel these things and never bother to calculate these two numbers.
So these massive numbers just cancel.
So it's a very special property of a constant, that it factors out.
That was our first discussion, and now with this example I'm going to go back to the heating up of the witches' cauldron and we'll use average value to illustrate the integral that we get in that context as well.
I remind you, let's see.
The situation with the witches' cauldron was this.
The first important thing is that there were, so this is the big cauldron here.
This is the one whose height is 1 meter and whose width is 2 meters.
And it's a parabola of revolution here.
And it had about approximately 1600 liters in it.
And this curve was y = x^2.
And the situation that I described at the end of last time was that the initial temperature was T = 0 degrees Celsius.
And the final temperature, instead of being a constant temperature, we were heating this guy up from the bottom.
And it was hotter on the bottom than on the top.
And the final temperature was given by the formula T = 100 - 30 times the height y.
So at y = 0, at the bottom, it's 100.
And at the top, T = 70 degrees.
OK, so this is the final configuration for the temperature.
And the question was how much energy do we need.
So, the first observation here, and this is the reason for giving this example, is that it's important to realize that you want to use the method of disks in this case.
The reason, so it doesn't have to do with, you shouldn't think of the disks first.
But what you should think of is the horizontal.
We must use horizontals because T is constant on horizontals.
It's not constant on verticals.
If we set things up with shells, as we did last time, to compute the volume of this, then T will vary along the shell.
And we will still have an averaging problem, an integral problem along the vertical portion.
But if we do it this way, T is constant on this whole level here.
And so there's no more calculus involved in calculating what the contribution is of any given level.
So t is constant on horizontals.
Actually, in disguise, this is that same trick that we have here.
We can factor constants out of integrals.
You could view it as an integral, but the point is that it's more elementary than that.
Now I have to set it up for you.
And in order to do that, I need to remember what the equation is.
Which is y = x ^2.
And the formula for the total amount of energy is going to be volume times the number of degrees.
That's going to be equal to the energy that we need here.
And so let's add it up.
It's the integral from 0 to 1, and this is with respect to y.
So the y level goes from 0 to 1.
This top level's y = 1, this bottom level's y = 0.
And the disk that we get, this is the point (x, y) here, is rotated around.
And its radius is x.
So the thickness is dy, and the area of the disk is pi x^2.
And the thing that we're averaging is T. Well, we're not yet averaging, we're just integrating it.
We're just adding up the total.
Now I'm just going to plug in the various values for this.
And what I'm going to get is T, again, = 100 - 30y.
And this radius is measured up to this very end.
So x^2 = y.
So this is pi y dy.
And this is the integral that we'll be able to evaluate.
Yeah, question
[INAUDIBLE]
All right.
Well, let's carry this out.
Let's finish off the calculation here.
Let's see.
This is equal to, what does it equal to.
Well, I'll put it over here.
It's equal to 50 pi y ^2 - right, because this is 100 pi y, and then there's a 30, this is 100 pi y - 30 pi y ^2, and I have to take the antiderivative of that.
So I get 50 pi y ^2, and I get 10 pi y ^3.
Evaluate it at 0 and 1.
And that is 40 pi.
Now, I spent a tremendous amount of time last time focusing on units.
Because I want to tell you how to get a realistic number out of this.
And there's a subtle point here that I pointed out last time that had to do with changing meters to centimeters.
I claim that I've treated those correctly.
So, what we have here is that the answer is in degrees, that is Celsius, times cubic meters.
These are the correct units.
And now, I can translate this into Celsius is spelled with a C. That's interesting.
Celsius.
I can translate this into units that you're more familiar with.
So let's try 40 pi deg * m ^3, and then do the conversion factors.
First of all there's one calorie per degree times a milliliter.
That's one conversion.
And then let's see.
I'm going to have to translate from centimeters so I have here (100 cm / m)^ 3.
So these are the two conversion factors that I need.
And so, I get 40 pi ( 10 ^ 6), that's 100^3.
And this is in calories.
So how much is this?
Well, it's a little better, maybe, to do it in 40 pi * 1,000 kilocalories, because these are the ones that you actually see on your nutrition labels of foods.
And so this number is around 125 or so.
Let's see, is that about right?
Let's make sure I've got these numbers right.
Yeah, this is about 125.
40 pi.
And so one candy bar, this is a Halloween example, so.
One candy bar is about 250 kilocalories.
So this is half a candy bar.
So the answer to our question is that it takes 500 candy bars to heat up this thing.
OK, so that's our example.
Now, yeah.
Question
[INAUDIBLE]
What does the integral give us?
This integral is, the integral represents the following things.
So the question is, what does this integral give us.
So here's the integral.
Here it is, rewritten so that it can be calculated.
And what this integral is giving us is the following thing.
You have to imagine the following idea.
You've got a little chunk of water in here.
And you're going to raise is from 0 degrees all the way up to whatever the target temperature is.
And so that little milliliter of water, if you like, has to be raised from 0 to some number which is a function of the height.
It's something between 70 and 100 degrees.
And the one right above it also has to be raised to a temperature, although a slightly different temperature.
And what we're doing with the integral is we're adding up all of those degrees and the calorie represents how much it takes, one calorie represents how much it takes to raise by 1 degree 1 milliliter of water.
One cubic centimeter of water.
That's the definition of a calorie.
And we're adding it up.
So in other words, each of these cubes is one thing.
And now we have to add it up over this massive thing, which is 1600 liters.
And we have a lot of different little cubes.
And that's what we did.
When we glommed them all together.
That's what the integral is doing for us.
Other questions.
Now I want to connect this with weighted averages before we go on.
Because that was the reason why I did weighted averages first.
I'm going to compute also the average final temperature.
So, final because this is the interesting one, the average starting temperature's very boring, it's 0.
The average final temperature is, individually the temperatures are different.
And the answer here is it's the integral from 0 to 1 of T pi y dy divided by the integral from 0 to 1 of pi y dy.
So this is the total temperature, weighted appropriately to the volume of water that's involved at that temperature, divided by the total volume of water.
And we computed these two numbers.
The number in the numerator is what we call 40 pi.
And the number in the denominator, actually this is easier than what we did last time with shells you can just look at this and see that it's the area under a triangle.
It's pi / 2.
And so the answer here is 80 degrees.
This is the average temperature.
Note that this is a weighted average.
The weighting here is different according to the height.
The weighting factor is pi y.
That's the weighting factor.
And that's not surprising.
When y is small, there's less volume down here.
Up above, those are more important volumes, because there's more water up at the top of the cauldron than there is down at the bottom of the cauldron.
If you compare this to the ordinary average, if you take the maximum temperature plus the minimum temperature, divided by 2, that would be 100 + 70 / 2.
You would get 85 degrees.
And that's bigger.
Why?
Because the cooler water is on top.
And the actual average, the correct weighted average, is lower than this fake average.
Which is not the true average in this context.
All right so the weighting is that the thing is getting fatter near the top.
So now I'm going to do another example of weighted average.
And this example is also very much worth your while.
It's the other incredibly important one in interpreting integrals.
And it's a very, very simple example of a function, f. The weightings will be different, but the functions, f, will be of a very particular kind.
Namely, the function f will be practically a constant.
But not quite.
It's going to be a constant on one interval, and then 0 on the rest.
So we'll do those weighted averages now.
And this subject is called probability.
In probability, what we do, so I'm just going to give some examples here.
I'm going to pick a point to in quotation marks at random.
In the region y < x < 1 - x ^2.
That's this shape here.
Well, let's draw it right down here.
For now.
So, somewhere in here.
Some point, (x, y).
And then I need to tell you, according to what this random really means.
This is proportional to area, if you like.
So area inside of this section.
And then the question that we're going to answer right now is, what is the chance that, it's usually called probability, that x > 1/2.
Let me show you what's going on here.
And this is always the case with things in probability.
So, first of all, we have a name for this.
This is called P ( x > 1/2).
And so that's what it's called in our notation here.
And what it is, is the probability is always equal to the part / the whole.
It's a ratio just like the one over there.
And which is the part and which is the whole.
Well, in this picture, the whole is the whole parabola.
And the part is the section x > 1/2.
And it's just the ratio of those two areas.
Let's write that down.
That's the integral from 1/2 to 1 of (1 - x ^2) dx, divided by the integral from - 1 to 1, (1 - x ^2 ) dx.
And again, the weighting factor here is 1 - x^2.
And to be a little bit more specific here, the starting point a = - 1 and the endpoint b = + 1.
So this is P(x < 1/2).
And if you work it out, it turns out to be 5/18, we won't do it.
Yeah
[INAUDIBLE]
What we're trying to do with probability.
So I can't repeat your question.
But I can try say, because it was a little bit too complicated.
But it was not correct, OK. What we're taking is, we have two possible things that could happen.
Either, let's put it this way.
Let's make it a gamble.
Somebody picks a point in here at random.
And we're trying to figure out what your chances are of winning.
In other words, the chances the person picks something in here versus something in there.
And the interesting thing is, so what percent of the time do you win.
The answer is it's some fraction of 1.
And in order to figure that out, I have to figure out the total area here.
Versus the total of the entire, all the way from - 1 to 1, the beginning to the end.
So in the numerator, I put success, and in the denominator I put all possibilities.
So that, right?
[INAUDIBLE]
And that's the interpretation of this.
So maybe I didn't understand your question
[INAUDIBLE]
Ah, why is 1 - x ^2 the weighting factor.
That has to do with how you compute areas under curves.
The curve here is y = 1 - x ^2.
And so, in order to calculate how much area is between 1/2 and 1, I have to integrate.
That's the interpretation of this.
This is the area under that curve.
This integral.
And the denominator's the area under the whole thing.
OK, yeah.
Another question
[INAUDIBLE]
Ah.
Yikes.
It was supposed to be the same question as over here.
Thank you
[INAUDIBLE]
This has something to do with weighting factors.
Here's the weight factor.
Well, it's the relative importance from the point of view of this probability of these places versus those.
That is, so this is a weighting factor because it's telling me that in some sense this number 5/18, actually that makes me think that this number is probably wrong.
Well, I'll let you calculate it out.
It looks like it should be less than 1/4 here, because this is 1/4 of the total distance and there's a little less in here than there is in the middle.
So in fact it probably should be less than 1/4, the answer
[INAUDIBLE]
The equation of the curve is 1 - x^2.
The reason why it's the weighting factor is that we're interpreting, the question has to do with the area under that curve.
And so, this is showing us how much is relatively important versus how much is not.
This is, these parts are relatively important, these parts are less important.
According to area.
Because we've said that area is the way we're making the choice.
So I don't have quite enough time to tell you about my next example.
Instead, I'm just going to tell you what the general formula is.
And we'll do our example next time.
I'll tell you what it's going to be.
So here's the general formula for probability here.
We're going to imagine that we have a total range which is maybe going from a to b, and we have some intermediate values x1 and x2, and then we're going to try to compute the probability that some variable that we picked at random occurs between x1 and x2.
And by definition, we're saying that it's an integral.
It's the integral from x1 to x2 of the weight dx, divided by the integral all the way from a to b.
Of the weight.
So, again, this is the part divided by the whole.
And the relationship between this and the weighted average that we had earlier was that the function f ( x) is kind of a strange function.
It's 0 and 1.
It's just the picture, if you like, is that you have this weighting factor.
And it's going from a to b.
But then in between there, we have the part that we're interested in.
Which is between x1 and x2.
And it's the ratio of this inner part to the whole thing that we're interested in.
Tomorrow I'm going to try to do a realistic example.
And I'm going to tell you what it is, but we'll take it up tomorrow.
I told you it was going to be tomorrow, but we still have a whole minute, so I'm going to tell you what the problem is.
So this is going to be a target practice problem.
You have a target here and you're throwing darts at this target.
And so you're throwing darts at this target.
And somebody is standing next to the dartboard.
Your little brother is standing next to the dartboard here.
And the question is, how likely you are to hit your little brother.
So this will, let's see.
You'll see whether you like that or not.
Actually, I was the little brother.
So, I don't know which way you want to go.
We'll go either way.
We'll find out next time.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I want to get started by correcting a mistake that I made last time.
And this was mistaken terminology.
I said that what we were computing, when we computed in this candy bar example was energy and not heat.
But it's both.
They're the same thing.
And in fact, energy, heat and work are all the same thing in physics.
I was foolishly considering the much more, what am I trying to say, the intuitive feeling of heat is just being the same as temperature.
But in physics, usually heat is measured in calories and energy can be in lots of things.
Maybe kilowatt-hours or ergs, these are various of the units.
And work would be in things like foot-pounds.
That is, lifting some weight some distance.
And the amount of force you have to apply.
And these all have conversions between them.
They're all the same quantity, in different units.
OK, so these are the same quantity.
Different units.
So that's about as much physics as we'll do for today.
And sorry about that.
Now, the example that I was starting to discuss last time and then I'm going to carry out today was this dartboard example.
We have a dartboard, which is some kind of target.
And we have a person, your little brother, who's standing over there.
And somebody is throwing darts.
And the question is, how likely is he to be hit.
So I want to describe to you how we're going to make this problem into a math problem.
Yep
[INAUDIBLE]
What topic is this that we're going over.
We're going over an example.
Which is a dartboard example.
And it has to do with probability.
OK.
So what is the probability that this guy, your little brother, gets hit by a dart.
Now, we have to put some assumptions into this problem in order to make it a math problem.
And I'm really going to try to make them pretty realistic assumptions.
So the first assumption is that the number of hits is proportional to ce ^ - r ^2.
So that's actually a kind of a normal distribution.
That's the bell curve.
But as a function of the radius.
So this is the assumption that I'm going to make in this problem.
And a problem in probability is a problem of the ratio of the part to the whole.
So the part is where this little guy is standing.
And the whole is all the possible places where the dart might hit.
Which is maybe the whole blackboard or extending beyond, depending on how good an aim you think that this older child has.
So, if you like, the part is, we'll start simply.
I mean, this doesn't sweep all the way around.
But we're going to talk about some section.
Like this.
Where this is some radius r1, and this other radius is some longer radius, r2.
And the part that we'll first keep track of is everything around there.
That's not very well-centered, but it's supposed to be two concentric circles.
Maybe I should fix a bit so that it's a little bit easier to read here.
So here we go.
So here I have radius r1, and here I have radius r2.
And then the region in between is what we're going to try to keep track of.
So I claim that we'll be able to get, so this is what I'm calling a part, to start out with.
And then we'll take a section of it to get the place where the person is standing.
Now, I want to take a side view of e ^ - r ^2.
The function that we're talking about.
Again, that's the bell curve.
And it sort of looks like this.
This is the top value, this is now the r axis.
And this is up, or at least, so you should think of this in terms of the fact that the horizontal here is the plane of the dartboard.
And the vertical is measuring how likely it is that there will be darts piling up here.
So if they were balls tumbling down or something else falling on, many of them would pile up here.
Many fewer of them would be piling up farther away.
And the chunk that we're keeping track of is the chunk between r1 and r2.
That's the corresponding region here.
And in order to calculate this part, we have to calculate this volume of revolution.
Sweeping around.
Because really, in disguise, this is a ring.
This is a side view, it's really a ring.
Because we're rotating it around this axis here.
So we're trying to figure out what the total volume of that ring is.
And that's going to be our weighting, our likelihood for whether the hits are occurring in this section or in this ring versus the rest.
To set this up, I remind you we're going to use the method of shells.
That's really the only one that's going to work here.
And we want to integrate between r1 and r2.
And the range here is that because this is, if you like, a solid of revolutions.
So the variable r is the same as what we used to call x.
And it's ranging between r1 and r2, and then we're sweeping it around.
And the circumference of a little piece, so at a fixed distance r, here, the circumference is going to be 2 pi r. So 2 pi r is the circumference.
And then the height is the height of that green stem there.
That's e ^ - r ^2.
And then we're multiplying by the thickness, which is dr.
So the thickness of the green is dr.
The height of this little green guy is e ^ - r^2, and the circumference is e pi times the radius, when we sweep the circle around.
Question
[INAUDIBLE]
So the question is, why is we're interested in not this pink area.
And the reason is an interpretation of what I meant by this.
What I meant by this is that if you wanted to add up what the likelihood is that this thing will be here versus here, I want it to be, really, the proportions are the number of hits times the, if you like, I wanted it d area.
That's really what I meant here.
The number of hits in a little chunk.
So little, maybe I'll call it delta a.
A little chuck.
Is proportional to the chunk times that.
So there's already an area factor.
And there's a height.
So there are a total of three dimensions involved.
There's the area and then this height.
So it's a matter of the, what I was given, what I intended to say the problem is.
Yeah, another question
[INAUDIBLE]
Yeah, yeah.
Exactly.
The height is c times that, whatever this c is.
In fact, we don't know what the c is, but because we're going to have a part and a whole, we'll divide, the c will always cancel.
So I'm throwing the c out.
I don't know what it is, and in the end it won't matter.
That's a very important question.
Yes
[INAUDIBLE]
Say it again?
So what I mean is the number of hits in some chunk.
That is, suppose you imagine, the question is, what does this left-hand side mean.
That right?
Is that the question that's you're asking?
When I try to understand what the distribution of dartboard hits is, I should imagine my dartboard.
And very often there'll be a whole bunch of holes in some places.
And fewer holes else.
I'm trying to figure out what the whole distribution of those marks is.
And so some places will have more hits on them and some places will have fewer hits on them.
And so what I want to measure is, on average, the number of hits.
So this would really be some, this constant of proportionality is ambiguous because it depends on how many times you try.
If you throw a thousand times, it'll be much more densely packed.
And if you have only a hundred times it'll be fewer.
So that's where this constant comes in.
But given that you have a certain number of times that you tried, say, a thousand times, there will be a whole bunch more piled in the middle.
And fewer and fewer as you get farther away.
Assuming that the person's aim is reasonable.
So that's what we're saying.
So we're thinking in terms of the person's always aiming for the center.
So it's most likely that the person will hit the center.
But on the other hand, it's a fallible person, so the person may miss.
And so it's less and less likely as you go farther out.
And we're just counting how many times this gets hit, how many time this and so on.
In proportion to the area
[INAUDIBLE]
Yeah.
r1 and r2 are arbitrary.
We're going to make this calculation in general.
We're going to calculate what the likelihood is that we hit any possible band.
And I want to leave those as just letters for now.
The r1 and the r2.
Because I want to be able to try various different possibilities
[INAUDIBLE]
Say it again., why do we have to take volume.
So this is what we were addressing before.
It's a volume because it's number of hit per unit area.
So there's a height, that is, number of hits.
And then there's an area and the product of those is, so this is, if you like, a histogram of the number of hits.
But this should be measured per area.
Not per length of r. Because on the real diagram, it's going all the way around.
There's a lot more area to this red band than just the distance implies.
OK.
So, having discussed the setup, this is a pretty standard setup -- oh, one more question.
Yes
[INAUDIBLE]
Yeah.
So the question is, why is this a realistic.
Why is this choice of function here e to the minus r squared a realistic choice of function for the darts.
So I can answer this with an analogy.
When people were asking themselves where the V-2 rockets from Germany hit London, they used this model.
It turned out to be the one which was the most accurate.
So that gives you an idea that this is actually real.
The question, this makes it look like people are masters.
That is, that they'll all hit in the center more often than elsewhere.
But that's actually somewhat deceptive.
There's a difference between the mode, that is, the most likely spot, and what happens on average.
So in other words, the single most likely spot is the center.
But there's rather little area in here, and in fact the likelihood of hitting that is some little tiny bit.
In here.
In fact, you're much more likely to be out here.
So if you take the total of the volume, you'll see that much of the volume is contributed from out here.
And in fact, the person hits rather rarely near the center.
So this is not a ridiculous thing to do.
If you think of it in terms of somebody's aiming at the center but there's some random thing which is throwing the person off, then there is likely to be to left or to the right, or they might even get lucky and all those errors cancel themselves and they happen to hit pretty much near the center.
Yeah, another question
[INAUDIBLE]
How does the little brother come into play?
The little brother is going to come into play as follows.
I'll tell you in advance.
So the thing is, the little brother was not so stupid as to stand in front of the target.
I know.
He stood about twice the radius of the target away.
And so, we're going to approximate the location by some sector here.
Which is just going to be some chunk of one of these things.
We'll just break off a piece of it.
And that's how we're going to capture.
So the point is, the target is here.
But there is the possibility that the seven-year-old who's throwing the darts actually missed the target.
That actually happens a lot.
So, does that answer your question?
Alright.
are we ready now to to do?
One more question
[INAUDIBLE]
I'm giving you this property here.
I'm telling you that this is what's called a mathematical model, when you give somebody something like this.
In fact, that requires further justification.
It's an interesting issue.
Yeah
[INAUDIBLE]
I'm giving it to you for now.
And it's something which really has to be justified.
In certain circumstances it is justified.
But, OK.
So anyway, here's our part.
This is going to be our chunk, for now, that we're going to estimate the the importance.
The relative importance of.
And now, this is something whose antiderivative we can just do by substitution or by guessing.
It's just - pi e ^ - r^2.
If you differentiate that, you get a - 2r, which cancels the minus sign here.
So you get 2pi re ^ - r ^2.
So that's the antiderivative.
And we're evaluating it at r1 and r2.
So with the minus sign that's going to get reversed.
The answer is going to be pi ( e ^ - r1^2, that's the bottom one.
- e ^ - r2 ^2), that's the top.
So this is what our part gives us.
And, more technically, if you wanted to multiply through by c, it would be c times this.
I'll say that in just a second.
OK, now I want to work.
So, if you like, the part is equal to maybe even I'll call it c pi (e^ -r1 ^2 - e ^ - r2 ^2).
That's what it really is if I put in this factor.
So now there's no prejudice as to how many attempts we make.
Whether it was a thousand attempts or a million attempts at the target.
Now, the most important second feature here of these kinds of modeling problems is, there is always some kind of idealization.
And the next thing that I want to discuss with you is the interpretation of the whole.
That is, what's the family of all possibilities.
And in this case, what I'm going to claim is that the reasonable way to think of the whole is it's that r can range all the way from 0 to infinity.
Now, you may not like this.
But these are maybe my first and third favorite number, my second favorite number being 1.
So infinity is a really useful concept.
Of course, it's nonsense in the context of the darts.
Because if you think of the basement wall where the kid might miss a target, he'd probably hit the wall.
He's probably not going to hit one of the walls to the right, and anyway he's certainly not going to hit over there.
So there's something artificial about sending the possibility of hits all the way out to infinity.
On the other hand, the shape of this curve is such that the real tail ends here, because of this exponential decrease, are tiny.
And that's negligible.
And the point is that actually the value, if you go all the way out to infinity, is the easiest value to calculate.
So by doing this, I'm idealizing the problem but I'm actually making the numbers come out much more cleanly.
And this is just always done in mathematics.
That's what we did when we went from differences to differentials, to differentiation and infinitesimals.
So we like that.
Because it makes things easier, not because it makes things harder.
So we're just going to pretend the whole is from 0 to infinity.
And now let's just see what it is.
It's c pi times the starting place is e^ - 0 ^2.
That's r1, right, this is the role that r1 plays is this, and the r2 is this value.
Minus e ^ - infinity ^2.
Which is that negligibly small number, 0.
So this is just c pi.
Because this number is 1, and this other number is 0.
This is just (1 - 0), in the parentheses there.
And now I can tell you from these two numbers what the probability is.
The probability that we landed on the target in a radius between r1 and r2, so that's this annulus here, is the ratio of the part to the whole.
Which in this case just cancels the c pi.
So it's (e ^ - r1 ^2 - e ^ - r2^2).
There's the formula for the probability.
So the c canceled and the pi canceled.
It's all gone.
Now, again, let me just emphasize the way this formula is supposed to work.
The total probability of every possibility here is supposed to be set up to be 1.
This is some fraction of 1.
If you like, it's a percent.
Yes
[INAUDIBLE]
This one is giving, the question is, doesn't this just give the probability of the ring?
This gives you the probability of the ring, but this is a very, very wide ring.
This is a ring starting with 0, nothing, on the inside.
And then going all the way out.
So that's everything.
So this corresponds to everything.
This corresponds to a ring.
So now, let's see.
Where do I want to go from here.
So in order to make progress here, I still have to give you one more piece of information.
And this is, again, supposed to be realistic.
When I was three years old and my brother's friend Ralph was seven, I watched him throwing darts a lot.
And I would say that for Ralph, so for Ralph, at age seven, anyway, later on he got a little better at it.
But Ralph at age seven, the probability that he hit the target was about 1/2.
Right?
So he hit the target about half the time.
And the other times, there was cement on the walls of the basement, it wasn't that bad.
Just bounced off.
That also meant that the points got a little blunter as time went on.
So it was a little less dangerous when they hit.
Alright.
So now, so here's the extra assumption that I want to make.
So a is going to be the radius of the target.
Now, the other realistic assumption that I want to make is where this little kid would be standing.
And now, here, I want to get very specific and just do the competition in one case.
We're going to imagine the target is here.
And the kid is standing, say, between, so we'll just do a section of this.
This is between 3 o'clock and 5 o'clock.
There's more of him, but it's lower down and maybe negligible here.
So this section is the part, the chunk that we want to see about.
And this is a, and then this distance here is 2a.
And then the longest distance here is 3a.
So the longest distance is 3a.
So in other words, what I'm saying is that the probability, if you're standing too close, the chance Ralph hits younger brother is about 1/6, right?
Because 2/12 = 1/6.
1/6 of the probability that we're between a and 3a.
That's the number that we're looking for.
Another question
[INAUDIBLE]
The 2/12 came from the fact that we assumed.
So we made a very, very bold assumption here.
We assumed that this human being, who is actually standing, the floor is about down here.
Maybe he wasn't that tall.
But anyway, so he's really a little bigger than this.
That the part of him that was close to the target covered about this section here, between radius 2a and 3a.
As you'll see, actually from the computation, because the likelihood drops off pretty quickly, whatever of him was standing outside there wouldn't have mattered anyway.
So we're just worried about the part that's closest to the target here
[INAUDIBLE]
Why is it out of 12?
Because I made it a clock.
And I made it from 3 o'clock to 5 o'clock, so it's 2 of the 12 hours of a clock.
It's just a way of me making a section that you can visibly see.
So now, so here's what we're trying to calculate.
And in order to figure this out, I need one more item.
here.
So maybe I'll leave myself a little bit of room.
I have to figure out something about what our information gave us.
Which is that the probability, sorry, this probability was 1/2.
So let's remember what this is.
This is going to be e ^ - 0 ^2 - e ^ - a ^2.
That's what's this probability is.
And that's equal to 1/2.
So that means that 1 - e ^ - a ^2 = 1/2.
Which means that e ^ - a ^2 = 1/2.
I'm not going to calculate a, this is the part about the information about a that I want to use.
That's what I'll use.
And now I'm going to calculate this other probability here.
So the probability right up there is this.
And that's going to be the same as e ^ - (2a)^2 - e ^ - (3a)^2.
That's what we calculated.
And now I want to use some of the arithmetic of exponents.
This is (e ^ - a ^2)^4.
Because it's really (2a)^2 = 4.
The quantity is 4a^2, and then I bring that exponent out.
- (e ^ - a ^2) ^ 9 that's 3 ^2.
And so this comes out to be (1/2) ^ 4 - (1/2) ^9.
Which is approximately 1/16.
This is negligible.
This part here.
And this is actually why these tails, as you go out to infinity, don't really matter that much.
So this is a much smaller number.
So the probability of the whole band is 1/16.
And now I can answer the question up here.
This is approximately 1/6 * 1/16.
Which is about 1/100, or about 1%.
So if I stood there for 100 attempts, then my chances of getting hit were pretty high.
So that's the computation.
That's a typical example of a problem in probability.
And let me just make one more connection with what we did before.
This is connected to weighted averages or integrals over weights.
But the weight that's involved in this problem was w ( r) =, so let's just look at what happened in all of those integrals.
What happened in all the integrals was, we had this factor here.
2 pi r. And if I include this c, it was really 2 pi c r e ^ - r ^2.
This was the weight that we were using.
The relative importance of things.
Now, this is not the same as e ^ - r ^2 that we started out with.
Because this is the one-dimensional histogram.
And that has to do with the method of shells that gave us that extra factor of r here.
So that also connects with the question at the beginning.
Which had to do with this paradox, and it looks like these places in the middle are the most likely.
But that's the plot of e ^ - r ^2.
If you actually look at this plot here, you see that as r goes to 0, it's going to 0.
This is a different graph here.
And actually, so this is what's happening really.
In terms of how likely it is that you'll get within a certain distance of the center of the target.
Again, it's not the area under that curve that we're doing.
It's that volume of revolution.
We're going to change subjects now.
OK, one more question.
Yes
[INAUDIBLE]
Yeah, that's supposed to be the graph of w (r)
[INAUDIBLE]
Well, so, the question is, wouldn't the importance of the center be greatest?
It's a question of which variable you're using.
According according to pure radius, it's not.
It turns out that there are some bands in radius which are more important, more likely for hits than others.
It really has to do with the fact that the center, or the core, of the target is really tiny.
So it's harder to hit.
Whereas a whole band around the outside has a lot more area.
Many, many ways to hit that band.
So it's a much larger area.
So there's a competition there between those two things.
So we're going to move on.
And I want to talk now about a different kind of weighted average.
These weighted averages are going to be much simpler.
And they come up in what's called numerical integration.
There are many, many methods of integrating numerically.
And they're important because many, many integrals don't have formulas.
And so you have to compute them with a calculator or a machine.
So the first type that we've already done are Riemann sums.
They turned out to be incredibly inefficient.
They're lousy.
The next rule that I'm going to describe is a little improvement.
It's called the trapezoidal rule.
And this one is much more reasonable then the Riemann sum.
Unfortunately, it's actually also pretty lousy.
There's another rule which is just a slightly trickier rule.
And it's actually amazing that it exists.
And it's called Simpson's Rule.
And this one is actually pretty good.
It's clever.
So let's get started with these.
And the way I'll get started is by reminding you what the Riemann sum is.
So this is a good review, because you need to know all three of these.
And you're going to want to see them all laid out in parallel.
So here's the setup.
OK, so here we go.
We have our graph, we have our function it starts out at a, it ends up at b.
Maybe goes on, but we're only paying attention to this interval.
This is a function y = f(x).
And we split it up when we do Riemann's sum.
So this is 1, this is Riemann sums.
We start out with a, which is a point we call x0, and then we go a certain distance.
We go all the way over to b.
And we subdivide this thing with these delta x's.
Which are the step sizes.
So we have these little steps of size delta x.
Corresponding to these x values, we have y values.
y0 = f(x0), that's the point above a.
Then y1 = f ( x1), that's the point above x1.
And so forth, all the way up to yn, which = f (xn).
In order to figure out the area, you really need to know something about the function.
You need to be able to evaluate it.
So that's what we've done.
We've evaluated here at n + 1 points.
Enumerated 0 through n. And those are the numbers out of which we're going to get all of our approximations to the integral.
So somehow we want average these numbers.
So here's our goal.
Our goal is to average, or add, I'm using average very loosely here.
But I was going to say add up these numbers to get an approximation.
To average or add the y's.
To get an approximation to the integral.
Which we know is the area under the curve.
So here's what the Riemann sum is.
It's the following thing.
You take (y0 + y1 + ... up to yn - 1) And you multiply by delta x.
That's it.
Now, this one is the one with left endpoints.
So the left-hand sum.
There's also a right one.
Which is, if you start at the right-hand ends.
And that will go from y1 to yn.
OK, so this one is the right-hand.
Right Riemann sum.
Those are the two that we did before.
Now I'm going to describe to you the next two.
They have a similar pattern to them.
And the one with trapezoids requires a picture.
Here's a shape.
And here's a bunch of values.
And we're trying to estimate the size of these chunks.
And now, instead of doing something stupid, which is to draw horizontal lines in rectangles, we're going to do something slightly more clever.
Which is to draw straight lines that are diagonal.
You see that many of them actually coincide probably pretty closely with what I drew there.
Although if they're curved, they miss by a little bit.
So this is called the trapezoidal rule.
Because if you pick one of these shapes, say, this is y2 and this is y3, if you pick one of these shapes, this height here is y2.
And this height is y3, and this base is delta x.
This is a trapezoid.
So this being a trapezoid, I can figure out its area.
And what do I get?
I get the base times the average height.
If you think about if, you work out what happens when you do something with a straight line on top, like that, you'll get this average.
So this is the average height of the trapezoid.
And now I want to add up.
I want to add them all up to get my formula for the trapezoidal rule.
So what do I do?
I have delta x times the first one.
Which is y0 + y1 / 2.
That's the first trapezoid.
The next one is y1 + y2 / 2.
And this keeps on going.
And at the end, I have yn - 2 + yn - 1 / 2.
And then last of all, I have yn - 1 + yn / 2.
That's a very long formula here.
We're going to simplify it quite a bit in just a second.
What's this equal to?
Well, notice that I get y0 / 2 to start out with.
And now, y1 got mentioned twice.
Each time with a factor of 1/2.
So we get a whole y1 in here.
And the same thing is going to be true of all the middle terms.
You're going to get y2 and all the way up to yn - 1.
But then, the last one is unmatched.
yn is only 1/2, only counts 1/2.
So here is what's known as the trapezoidal rule.
Now, I'd like to compare it for you to the Riemann sums, which are sitting just to the left here.
Here's the left one, and here's the right one.
If you take the average of the left and the right, that is, 1/2 of this + 1/2 of that, there's an overlap.
The y1 through yn things are listed in both.
But the y0 only gets counted 1/2 and the yn only gets counted 1/2.
So what this is, is this is the symmetric compromise between the two Riemann sums.
This is actually equal to the left Riemann sum + the right Riemann sum / 2.
It's the average of them.
Now, this would be great and it does look like it's closer.
But actually it's not as impressive as it looks.
If you actually do it in practice, it's not very efficient.
Although it's way better than a Riemann sum, it's still not good enough.
So now I need to describe to you the fancier rule.
Which is known as Simpson's Rule.
And so, this is, if you like, 3, Method 3.
The idea is again to divide things into chunks.
But now it always needs n to be even.
In other words, we're going to deal not with just one box, we're going to deal with pairs of boxes.
Here's delta x, and here's delta x again.
And we're going to study the area of this piece here.
So let me focus just on that part.
Let's reproduce it over here.
And here's the delta x, here's delta x.
And of course there are various heights.
This starts out at y0, this is y2, and this middle segment is y1.
Now, the approximating curve that we're going to use is a parabola.
That is, we're going to fit a parabola through these three points.
And then we're going to use that as the approximating area.
Now, it doesn't look like, this looks like it misses.
But actually, most functions mostly wiggle either one way or the other.
They don't switch.
They don't have inflection points.
So, this is a lousy, at this scale.
But when we get to a smaller scale, this becomes really fantastic.
As an approximation.
Now, I need to tell you what the arithmetic is.
And in order to save time, it's on your problem set what the actual formula is.
But I'm going to tell you how to think about it.
I want you to think about it as follows.
So the area under the parabola is going to be a base times some kind of average height.
And the base here, you can already see.
It's 2 delta x.
The base is 2 delta x.
Now, the average height is weird.
You have to work out what it is for a parabola, depending on those three numbers.
y0, y1, and y2.
And it turns out to be the following formula.
It has to be an average, but it's an interesting weighted average.
So this was the punchline, if you like.
Is that there were such things as interesting weighted averages.
This one's very simple, it just involves three numbers.
But it's still interesting.
It's the following.
It turns out to be y0 + 4 y1 + y2 / 6.
Why divided by 6?
Well, it's supposed to be an average.
So the total is 1 + 4 + 1 of these things.
And 6 is in the denominator.
So it emphasizes the middle more than the sides.
And that's what happens with a parabola.
So this is a computation which is on your homework.
And now we can put this together for the full Simpson's Rule formula.
Which I'll put up over here.
We have here 2 delta x, and we divide by 6.
And then we have y0 + 4 y1 + y2 / 6 +... that's the first chunk.
Now, the second chunk, maybe I'll just put it in here, starts this is x2.
This is x0.
And it goes all the way to x4.
So x2, x3, x4.
So the next one involves the indices 2, 3 and 4.
So this is y2 + 4 y3 + y4 / 6.
Oh, oh, oh, oh, no.
I think I'll get rid of these 6's.
I have too many 6's.
Alright.
Let's get rid of them here.
Let's take them out.
Put them out here.
Thank you.
All the way to the end, which is y(n - 2) + 2 y(n - 1) - sorry, + 4 y(n - 1) + yn.
I was about to divide by 6, but you saved me.
So here are all the chunks.
Now, what does this pattern come out to be?
This comes out to be the following.
This is 1, 4, 1, added to 1, 4, 1 added to 1, 4, 1.
You add them up.
You 1, 4, and then there's a repeat, so you get a 2 and a 4, and a 2 and a 4 and a 1.
So the pattern is that it starts out with 1's on the far ends.
And then 4's next in.
And then it alternates 2's and 4's in between.
So the full pattern of Simpson's Rule is delta x / 3, I have now succeeded in canceling this 2 with this 6 and getting out that factor of 2.
And then here I have y0 + 4 y1 + 2 y2 + 4 y3 + ... it keeps on going and keeps on going and keeps on going.
And in the end it's 2 y(n - 2) + 4 y(n - 1) + yn.
So again, 1 and a 4 to start.
1 and a 4 and a 1 to end.
And then alternating 2's and 4's in the middle And this weird weighted average is way better.
As I will show you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now, today I need to get started by finishing up what I did last time.
Namely, talking about numerical methods.
And I want to just carry out one example.
And then I want to fill in one loose end.
And then we'll talk about the unit overall.
We were talking, last time, about numerical integration.
I'm going to illustrate this just with the simplest example that I can.
We're going to look at the integral from 1 to 2 of dx / x.
Which we know perfectly well already is the log of x evaluated between 1 and 2, which is ln 2 - ln 1.
Which is just ln 2.
Now, if you punch that into your calculator, you're going to get something like this.
I hope I saved it here.
Yeah.
It's about 0.693147.
That's more digits than we're going to get in our discussion here.
Anyway, that's about how big this number is.
And the numerical integration methods will give you about as much accuracy as you can get on the function itself.
And, of course, some functions we may have more trouble approximating.
But the function 1 / x, we know pretty well how to do, because we know how to divide.
So since the function that we're integrating here is 1 / x, it's going to be not too difficult to get some arithmetic.
Nevertheless, I'm going to do this in the simplest possible case.
Namely, just with two intervals.
Now, you really can't expect things to work so well with two intervals.
That's a pretty ridiculous approximation to your function.
When you have two intervals, that means you're looking at the graph of this hyperbola.
And you have 1 here, and you have 2 here and you have 3/2.
And you're really only keeping track of the values at these three spots.
So the idea that you can approximate the area just by knowing the values of three places is a little bit of a stretch of the imagination.
But we're going to try it anyway.
Now, the trapezoidal rule is the following formula.
It's delta x ( 1/2 the first value + the second value + 1/2 the third value).
In this case, the pattern is 1/2, 1, 1, 1, 1, 1, 1/2.
And in this case, delta x = 1/2 because this interval's of length 1.
The b - a, right.
Let's just point that out here.
Here, b = 2. a = 1. b - a = 1.
And the number n = 2.
And so, delta x, which is b - a / n = 1/2.
So here's what we get.
And let's just see what this number is.
It's 1/2 of the value at here.
Well, so let's just check what these values are.
This value is 1, this value over here is 2/3, and the last value is 1/2.
Because the function, of course, was y = 1 / x.
And those were the three values that we have.
So y0, this one is y0, this one is y1, and this one is y2.
Now, here we have (1/2* 1 + 2/3 + 1/2 * 1/2).
Now, on an exam, I don't expect you to add up long messes of numbers like this.
When you have two numbers, I expect you to add them up if they're reasonable, or subtract them.
Just as we do when we take antiderivatives.
Like, for example, I don't want you to leave the answer to an integration like this in this form.
I want you to simplify it at least down to here.
And I of course don't expect you to know the numerical approximation.
But I certainly expect you to be able to do that.
On the other hand, when the arithmetic gets a little bit long, you can relax a little bit.
But I did carry this out on my calculator.
Unless I'm mistaken, it's about 0.96.
It's pretty far off.
So remember what it was.
It's what you get when you get these straight lines.
And there are these little extra pieces of junk there.
Now, don't trust that too much, but the point is that it's far off.
So now, let's take a look at Simpson's Rule.
And I claim that Simpson's Rule is surprisingly accurate.
In this case, really, even a little more than it deserves to be.
The formula is (delta x / 3) ( y0 + 4 y1 + y2).
So the pattern is 1, 4, 1, or 1, 4 and then it alternates 2's and 4's until 4, 1 at the very end.
And if I just plug in the numbers now, what I get is 1/6, because delta x = 1/2 again.
And the value for y0 = 1.
And the value for y1 = 2/3.
And the value for y2 = 1/2.
So here's the estimate in this case.
And this one I did carry out carefully.
And it came out to 0.69444.
Which is actually pretty impressive, if you think about it.
Given what the logarithm is.
Now, what's going on with Simpson's Rule in general is this.
If you -- Simpson's minus the exact answer.
In absolute value, is approximately of the size of (delta x)^ 4.
That's really the way it behaves.
Which means that if delta x is about 1/10, so if we had divided this up into 10, intervals which we didn't, but if we'd divided it up into 10 intervals, then you could expect that delta x, the error would be about 10 ^ - 4.
In other words, four digits of accuracy here for this thing.
But the exact analysis of this, a more careful analysis of this, is in your textbook.
And I'm not going to do.
But I just want to point out that it is an effective method.
It really does give you nice four-digit with manageable, you could even really do it by hand.
It's so convenient.
The Simpson's Rule.
Whereas the other rules aren't really that impressive as far as giving fairly accurate answers.
The last little remark to make is that the reason is that Simpson's Rule is matching a parabola.
And somehow the parabola follows this curve better.
It's giving the exact answer.
So I'll mention this.
Simpson's Rule is derived using the exact answer for all degree 2 polynomials.
In other words, parabolas.
All parabolas.
But even all the ones of lower degree.
So straight lines would work, and constants would work as well.
Whereas the other ones only work for, say, straight lines.
The trapezoidal rule only works for straight lines.
But ther isn't a weird accident.
It turns out that it also works for cubics.
Once you get the formulas, it works for cubics.
So it's also exact for cubics.
And that's what explains the fourth order validity.
The last thing that I want to point out is that this is extremely vague, that I said there.
And you should be a little bit cautious about it.
You need to watch out for 1 / x for x near 0.
All bets are off if the function is singular.
And there's a lot of area under there.
And it's also true that if the derivative messes up, you're in trouble too.
You really need for the function to be nice and smooth in order for Simpson's Rule to work.
This is woth out.
That's a real woth out, but try to.
Watch out.
Watch out for whenever x near 0.
Then this thing doesn't work.
This thing really depends on bounds on derivatives.
But I'm going to be relatively vague about that.
I'm not attempting to give you an error analysis here.
OK, so if you were doing this on an exam, how do you remember this strange pattern of numbers?
The one thing that I want to recommend to you is, as a way of remembering it, so the one mnemonic device, we'll call it a mnemonic device here, for remembering what it is that you're doing, is to remind yourself of what happens for the simplest possible case.
Which is f ( x) = 1.
It seems very modest, but if it doesn't give you the exact answer for f (x) = 1, you've got the wrong weightings.
And here, if you check out what happens in the first formula here, y0 / 2 + y1 +..., well, we'll go all the way to yn - 1 + yn / 2.
If you check that formula out here, this is the trapezoidal rule.
If you check it out for this case, then what you get is that this is equal to delta x times what?
Well, all of these are 1's.
And how many are there in the middle?
There are n - 1 of them in the middle.
So it's 1/2 + n - 1 + 1/2.
At the tail end.
So all told it's (delta x)( n).
And I remind you that delta x = b - a / n. So, delta x, this thing, is equal to b - a.
And that's just as it should be.
What we just calculated is an approximation to this integral here.
Which is just the area of the rectangle of base b - a and height 1.
Which of course is b - a.
So this is the check that you got your weighted average correct here.
You've put the correct weightings on everything.
And you can do this same thing with Simpson's Rule.
And match up those quantities.
There was a question in the room at some point.
No, OK.
So now, the next thing I want to do for you is the loose end which I left hanging.
Namely, I want to compute that mysterious constant square root of pi / 2.
This is really one of the most famous computations in calculus.
And it's a very, very clever trick.
I certainly don't expect you to come up with this trick.
I certainly wouldn't have myself.
But it's an important thing to calculate.
And it's just very useful.
So I'm going to tell you about it.
And it's just on the subject that we're dealing with in this unit; namely, slicing.
Or adding up.
So the first step, which is just something that we already did, was that we found the volume under this curve.
This bell-shaped curve, e ^ - r ^2.
But rotated around an axis.
Rotated around this axis.
Around this way.
So we figured that out.
And that was a relatively short computation.
I'm just going to remind you, it goes by shells.
We integrate the whole range from 0 to infinity.
And we have 2 pi r e^ - r ^2 dr.
So this again is the circumference of the shell.
This is the height of the shell, and this is the thickness of the shell.
Circumference, height, thickness.
So we're just taking a little piece here and sweeping it around.
And then adding up.
And then this antiderivative is pi, - pi e^ - r ^2, evaluated at 0 and infinity.
And we worked this out last time.
This is pi.
It's pi (1 - 0).
Which is pi.
So the conclusion is that V = pi.
We already know that.
Now, the problem that we want to deal with now is the problem not of a volume, but an area.
And this looks quite different.
And of course the answer is going to be different.
But let's do it.
So this is this question mark here.
And I'm going to do the one from minus infinity to infinity.
And I'll relate it to what we talked about earlier in this unit, in just a couple of minutes when I show you the procedure that we're going to follow.
So here's the quantity and now, what this is interpreted as is the area under this bell curve.
This time, Q is really an area.
Now, what's going to turn out to happen, is this.
This is the trick.
We're going to compute V in a different way.
And you'll see it laid out in just a second.
We will compute V by slices.
We're going to slice it like a piece of bread here.
We're going to solve for that same thing here.
And then, amazingly, what's going to happen is that we will discover that V = Q ^2.
That's going to be what's going to come out.
And that's the end of the computation that we want.
Because actually we already know what V is.
We don't want to read this equation forward.
We want to read it the other way.
We want to say Q^2 = V, which we already know is pi.
And so Q = the square root of pi.
I haven't shown this yet, this is the weird part.
And I'm going to put it in a little box so that we know that this is what we need to check.
We need to check this fact here.
We haven't done that yet.
Now, let me connect this with what we did a few days ago.
With what I called one of the important functions of mathematics besides the ones you already know.
And so the function that we were faced with, and that we discussed, was this one.
And then, we were interested in the value at infinity.
We were interested in this.
Which, if you draw a picture of it, and you draw the same bell curve, that's the area under half.
of it.
That's the area starting from 0 and going to infinity.
That's the area under half.
So this chunk is F of infinity.
And now I hope that this part of the connection is not meant to be fancy.
The idea here is that Q = 2 F(infinity).
This number here.
And so F of infinity = to the square root of pi / 2, if we believe what we said on the last panel.
And that was the thing that I drew a picture of on the board.
Namely, the graph of F looked like this.
And there was this asymptote, which was the limit F (x) tends to square root of pi / 2.
As x goes to infinity.
That was that limiting value.
Which is F of infinity.
So this is the asymptote.
And now I've explained the connection between what we claimed to be 4, which was quite mysterious, and what we're actually going to be able to check now.
Concretely, by making this computation.
So how in the world can you get something like this.
What's in that orange box there, that V = Q ^2.
Again, the technique is to use slices here.
And I'm going to have to draw you a 3-D picture to visualize the slice.
Let's do that.
I'm going to draw three axes now, because we're now going to be in three-dimensional space, and I want you to imagine the x axis as coming out of the blackboard, the y axis is horizontal, and there's a new axis, which I'll call the z axis which is going up.
So what's happening here is that I'm thinking of this, this is, if you like, some kind of side view.
And this is a view where I've tilted things a little bit up to the top.
Now, the distribution, or you could think of this target in the plane, where the most likely places to hit were in the middle and it died off.
As we went down.
Now, I want to draw a picture of this graph.
I'm going to draw a picture of e^ - r squared.
And it's basically a hump.
So I'm going to take the first, the slice along y = 0.
The y = 0 slice.
And I claim that that goes up like this.
And then comes back down.
Let me shade this in, so that you can see what kind of a slice this is.
This is supposed to be along this vertical plane here.
Which is coming out of the blackboard and coming towards you.
And that's a slice.
Now, I'm going to draw one more slice so that you can see what's happening.
I'm going to draw a slice at another place.
Along here.
This will be y = b.
Some other level.
And now I'm going to show you what happens.
What happens is that the hump dies down a little bit.
So the bump is just a little bit lower.
And it's going to look a little bit the same way.
But it's just going to be a bit smaller.
So there's another slice here.
Like that.
And I want to give a name to these slices.
I'm going to call this A ( b).
That is, the area of the b slice.
Under the surface.
OK Yes, question
[INAUDIBLE]
Yeah, the solid.
Yeah.
We're trying to figure out this volume here, which is the one we started out with, by slices.
So first I have to think of, I'm going to visualize.
So here I didn't even visualize.
I took a cross section and I thought about how to spin it around without actually doing that in three-dimensional space.
But now I'm going to take a different kind of slice.
I'm going to take that same bump, which is a three-dimensional object.
I'm going to lay it down on a plane.
Which looks like this.
And then it's a bump here.
It's a hump.
And now I'm going to try to slice it by various planes
[INAUDIBLE]
So one way of defining the bump, as you just suggested, is you take this curve and you rotate it around this z-axis.
So in other words, you make this the axis of rotation, you spin it around.
That's correct.
So that shows you that the peaks as you go down here are going to descend the same way.
But I don't want to draw those lines.
I want to imagine what the parallel slices are.
Because I don't want to get cross slices.
I want all slices parallel to the same thing
[INAUDIBLE]
OK.
This is not particularly easy to visualize.
Now, here's the formula for volume by slices.
The formula for volume by slices is that you add up the areas of the slices.
That's how you do it.
You take each slice.
You add the cross-sectional area, and then you take a little thickness, dy, and then you add all of them up.
Because this is extending over the whole plane, we're going to have to go all the way from minus infinity to plus infinity.
And this is the formula for volumes by slicing.
And now our goal, in order to do this calculation, we're going to just fix y = some b.
We're just going to fix one of these slices.
And we're going to calculate A ( b).
That's what we need to do in order to make this procedure succeed.
This is the only place where this method works.
But it's an important one.
In order to make it work, I'm going to have to again draw the plot from a different point of view.
I'm going to do the top view.
So I want to look down on this x-y plane here.
This is the x direction, and here's the y direction.
And then again I want to draw my slice.
My slice is here.
At y = b.
So we're just right on top of it.
And it's coming up at some kind of bump.
Here, with a little higher in the middle and going down on the sides.
Now, the formula for the height is this.
If I take a distance r here, the formula for the height of the bump is e ^ - r ^2.
I'll store that over here.
e ^ - r ^2 is the height at this place.
If this distance to the origin is r. That's true all the way around.
And in terms of b and x, we can figure out that by this right triangle.
This height is b, and this distance is x.
So r ^2 = b ^2 + x ^2.
Question
[INAUDIBLE]
The question is, is that the x-y plane.
So the answer is that over here I cleverly used the letter r. I avoided using y's and z's or anything.
And over here, this is the distance r. And you like this, is z, going up.
That's the way to think of it.
So that all of the letters are consistent.
So I just avoided giving it a name.
That's good, that's exactly the point.
Alright.
So now, I claim I have a formula for r ^2.
And so I can write this down.
This is e ^ - (b ^2 + x ^2).
But now I'm going to use the rule of exponents.
Which is that this is the same as (e ^ - b ^2) ( e^ - x^2).
And that's going to be the main way in which we use the particular function that we're dealing with here.
That's really the main step, amazingly.
So now I get to compute what A ( b) is.
A( b) is the area under a curve.
So it's going to be, let me write it over here, A(b) is the area under this curve here.
Which is some constant times, so if you imagine, call this thing the name c. Under some curve, c e ^ - x ^2.
Where the c = e^ - b ^2.
That's what our slice is.
In fact, it looks like one of those.
It looks like one of those bumps.
Here's its formula again.
It's the integral from minus infinity to infinity of (e^ - b ^2)( e ^ - x ^2) dx.
We just recopied what I had up there.
And this is the height at each value of x, with b fixed.
And now, so we have a lot of steps here.
But each of them is very elementary.
The first one was just that law of exponents.
That we could split the two into products.
Now I'm going to make that splitting even further.
This is a constant.
It's not varying with x.
So I'm going to factor it out of the integral.
This is e ^ - b ^2 times the integral from minus infinity to infinity of e^ -x^2 dx.
So this might look frightening, but actually it's just the property of an integral.
All integrals have this kind of property.
You can always factor out a constant.
And now here comes the remarkable thing.
This is e ^ - b^2 times a number which is now familiar to us.
What is this number?
This is what we're looking for.
This is our unknown, Q.
So I've computed A(b), and now I'm ready to finish the problem off.
A (b) = ( e^ - b^2) Q. Q is that strange number which we don't know yet.
What it is.
So now I'm going to compute the whole volume.
The whole volume, remember, it's over there, it's minus infinity to infinity, A ( y) dy.
And now I'm just going to plug in the formula that we've found for a.
Now I'm doing this for each b, so I'm doing it varying over all b's.
So I have the integral from minus infinity to infinity.
And here I have e^ - y ^2.
I've replaced b by y.
And now I have Q.
And I have dy.
I just recopied what I had over there into the formula for slicing.
And now, I'm going to do this trick of factoring out the constant a second time.
This is a constant.
It doesn't depend on y.
It's the same for all y, it just will factor out.
So this is the same as Q times the integral from minus infinity to infinity, e ^ - y ^2 dy.
And now, lo and behold, this expression here.
Of course, notice how I defined Q.
Let's go back carefully to where Q is defined.
Here's Q.
This t is a dummy variable.
It doesn't matter what I call it.
I can call it x, I can call it u, I can call it v. In this case, I've given it two different names.
At this stage, I called it x.
And at this stage I'm calling it y.
But it's the same variable.
And so this little chunk is Q and altogether I have two of them, for Q ^2 being the total.
And that's the end of the argument.
It's a real miracle
[INAUDIBLE]
Great question.
The question is, wait a minute.
As y changes, doesn't x change.
And so then this wouldn't be a constant.
So that's the way in which we've used the letters x and y in this whole course.
When you get to 18.02, you'll almost never do that.
Always y and x will be different variables.
And they won't have to depend on each other.
Now, let me show you where on this picture the x and the y are.
We've got a whole x-y plain, and here I'm fixing y = b, y isn't varying.
Whereas x is changing.
So, in other words, I don't have a relationship between x and y, unless I fix it.
In this case I've decided that y is going to be constant.
For all x.
Over here, I made a computation.
And I have a Q, which is just a single number.
No matter which b I took, it didn't matter which.
No matter which y = b.
Of course, I changed the name to b so it wouldn't be so jarring to you.
But in fact this b was y all along.
It's just that the x varied completely independently of the y. I could fix the y and vary the x, I could fix the x and vary the y.
So it's a different use of the letters.
From what you're used to.
It happens that y is not a function of x.
In this case.
Yes
[INAUDIBLE]
Yes
[INAUDIBLE]
The question is, because I'm rotating around the z axis, doesn't x change exactly as much as y does.
What happens is that x and y are symmetric variables.
They can be treated equally.
But if I decide to take slices with respect to y being fixed and x varying, then of course they're now separated, and I have a separate role for the x and a separate role for the y.
Or if I'd sliced it the other way, I would have gotten the same answer.
I just would have reversed the roles of x and y.
So what's happening is that x and y are on equal footing with each other.
In this picture, and I could've sliced the other way.
It would have gotten the same answer.
That's more or less the answer to your question.
OK. Now I have given you a review sheet, and I want to run through, briefly, what's going to be on the exam.
And this list of exam questions is what's going to be on the exam.
There are, sorry this is not displayed correctly.
So, exam questions, but now I'm just going to show you what they are.
There are five questions on the exam.
They are completely parallel to what you got last year.
So you should look at that test.
It's worth looking at.
And you'll see in the descriptions on this sheet that what I'm describing is what's on that test.
So what's going to happen is, and this is also posted on the Web, is that you'll be expected to calculate some definite integrals using the fundamental theorem of calculus.
Do a numerical approximation.
There'll be a Riemann, a trapezoidal rule and a Simpson's Rule.
Calculate areas and volumes.
And then some other cumulative sum.
Either an average value or probability or perhaps work.
And sketch a function which is given in this form as an integral.
So those are the questions, and you'll see by the example of last year's exam exactly the style.
They're really going to be very similar.
Yes, question
[INAUDIBLE]
OK, good question.
So the question is, for Riemann sums, what's the difference between upper and lower, and right and left?
So here we have a Riemann sum.
And I'm going to give you a picture which is, maybe this function y = 1 / x, which was the one that we were discussing earlier.
If you take the function y = 1 / x and you break it up into pieces here, however it doesn't matter how many pieces, let's just say there are four of them.
Then the lower Riemann sum is the staircase which fits underneath.
So this one is a picture of the lower sum.
It's always less.
And in the case of a decreasing function, it's going to be, so since if you like, since 1 / x decreases, the lower sum equals the right sum.
You can see that visually on this picture.
The values you're going to select are going to be the right ends of the rectangles.
The upper sum is the left one.
Now, if the function wiggles up and down, then you have to pick whichever side is appropriate.
Or maybe it'll be a point in the middle, if the maximum is achieved in the middle.
Yeah, another question
[INAUDIBLE]
Correct.
If the function is increasing, then the lower sum is the left sum.
So it just exactly reverses what's here.
So this is decreasing, lower sum is right-hand sum.
Increasing, lower sum is left-hand sum
[INAUDIBLE]
Yes
[INAUDIBLE]
Good question.
Suppose you're faced with a function like this in this last problem.
Which, generally, these are the trickiest problems.
And the question is, how are you ever going to be able to decide on an asymptote, even whether there is an asymptote.
And the answer is, you're not.
It's going to be pretty tricky to get keep track of what's happening as it goes to infinity.
We had an example on the homework where is was oscillating and it's very unclear what's going on.
You have to do a very long analysis for that.
So in fact, just don't worry about that now.
At the very end of the class, we'll talk a little bit about these asymptotes.
And really, the first issue is whether they exist or not.
And that's even something.
That's a serious question which we'll address at the very end of this course
[INAUDIBLE]
That's right.
It's not going to be anything that complicated.
Other questions?
We we still have a five whole minutes, and I have an example to give, if nobody has a question.
Yeah
[INAUDIBLE]
The question, uh, will I tell you which one of what to use?
[INAUDIBLE]
When I tell you the numeric approximation is, you'll see on the exam.
The practice exam that you have.
I will ask you for all three.
I will ask you for the Riemann sum, the trapezoidal rule, and the Simpson's rule.
I'm guaranteeing you they'll all three be on the exam.
I'm guaranteeing that every single thing which is on that piece of paper is on the exam.
And you'll see it on the exam that you've got.
It's exactly parallel to what's there
[INAUDIBLE]
So with areas and volume, the question is will I tell you which method to use.
So let's discuss that.
So with areas and volumes, there's basically, so this is always true with areas.
And it's true with volumes of revolution.
By the way you should read this sheet.
Not everything that's on here is, have I said.
But you should read it.
Because it's all relevant.
So with volumes of revolution, you always work your way back to some 2-D diagram.
So there's some 2-D diagram which is always two-dimensional diagram, which is always connected with these problems.
I mean, something this hard is really just too hard to do on an exam, right?
I mean, I'm not going to ask you something this complicated on the exam.
Because this involves a three-dimensional visualization.
But once you're down to 2-D, you're supposed to be able to handle it.
Now, what's the main issue after you've got your 2-D diagram?
The main issue is, do you want to integrate with respect to dx or dy?
And the answer is that it will depend.
And if there's one that's going to cause you incredible difficulty, and I feel that you're not able to dodge it, then I might give you a hint and say you'd better use shells, or you'd better use disks or washers or something like that.
But if I feel that you're grown up enough to figure out which one it is, because one of them is so ridiculous you say forget it, immediately.
After thinking about it.
Then I won't tell you which one.
Because I figure, in other words, I don't want you to waste your time.
But I'm willing to waste a minute or two of your time on a wild goose chase.
So let me give you an example of this.
Suppose you're looking at the curve y < 0 < - x^3.
So this is some kind of lump.
Like this.
It goes from 0 to 1, because the right-hand side is 0 at 0 and 1 here.
It's some kind of thing.
And there are these two possibilities.
One of them is to do shells.
And then, so this is supposed to be rotated around the y axis.
In this case.
And the same would apply, actually, to the area problem.
So I'm doing a slightly more complicated problem.
But you could ask for the area underneath this.
So forth.
OK.
So we can integrate this dx, or we can integrate this dy.
This indicates that I'm deciding that this is going to be of thickness dx, and I'm integrating dx.
So that's a choice that I'm making.
Now, the minute I made that choice I know that these are shells.
Because they sweep around this way and that makes them shells.
Cylindrical shells.
And if I do that, the setup is this.
It's 2 pi x ( x - x ^3) dx.
Now, I claim that when you get to this point, you already know you've won.
Because this is an easy integral to calculate.
So you're done here.
You're happy.
Now, if you happened to say, oh gee, I hate to do this.
I want to do something clever, you could try to do it with cutting this way.
Let's do this.
And this would be the dy thickness.
And then when you sweep this around, you get what we call a washer.
Which is really just the difference of two disks.
So the shape here is this thing swung around this axis.
And it looks like this.
So it's going to be the difference of radii.
So what's the formula for this?
It's some integral of pi times the right end, which I'll call x2, and here the left end, which I'll call x1.
So this is pi (x2 ^2 - x1 ^2) dy.
Now, already at this stage, you think to yourself this is more complicated than the other method.
So you've already abandoned it.
But I'm just going to go one step further into this one to see what it is that's happening.
If you try to figure out what these values x1 and x2 are, that corresponds to solving for x1 and x2 in terms of y.
So that's the following equation.
x1 and x2 solve the equation that the curve, x - x^3 = y.
Now, look at this equation.
That's the equation x ^3 - sorry, x ^3 - x + y, I guess.
Let's see.
Yeah, that's right, is equal to 0.
This is a cubic equation.
Although there is a formula for this.
You've never been taught the formula for this equation.
So therefore, you will never, ever be able to get a formula for x2 and x1 as a function of y.
And you'll never be able to compute this one.
This is more than just a dead end, it's like crash, burn, and, you know self-destruct.
So there may be such a thing, so do the other way.
Good luck, folks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, because our subject today is trig integrals substitutions, Professor Jerison called in his substitute teacher for today.
That's me.
Professor Miller.
And I'm going to try to tell you about trig substitutions and trig integrals.
And I'll be here tomorrow to do more of the same, as well.
So, this is about trigonometry, and maybe first thing I'll do is remind you of some basic things about trigonometry.
So, if I have a circle, trigonometry is all based on the circle of radius 1, and centered at the origin.
And so if this is an angle of theta, up from the x-axis, then the coordinates of this point are cosine theta and sine theta.
And so that leads right away to some trig identities, which you know very well.
But I'm going to put them up here because we'll use them over and over again today.
Remember the convention sin^2 theta secretly means (sin theta)^2.
It would be more sensible to write a parenthesis around the sign of theta and then say you square that.
But everybody in the world puts the 2 up there over the sin, and so I'll do that too.
So that follows just because the circle has radius 1.
But then there are some other identities too, which I think you remember.
I'll write them down here.
Cos 2 theta, there's this double angle formula that says cos 2 theta = cos ^2 theta - sin ^2 theta.
And there's also the double angle formula for the sin 2 theta.
Remember what that says?
2 sin theta cos theta.
I'm going to use these trig identities and I'm going to use them in a slightly different way.
And so I'd like to pay a little more attention to this one and get a different way of writing this one out.
So this is actually the half angle formula.
And that says, I'm going to try to express the cos theta in terms of the cos 2 theta.
So if I know the cos 2 theta, I want to try to express the cos theta in terms of it.
Well, I'll start with a cos 2 theta and play with that.
OK. Well, we know what this is, it's cos ^2 theta - sin ^2 theta.
But we also know what the sin square root of theta is in terms of the cosine.
So I can eliminate the sin^2 from this picture.
So this is equal to the cosine ^2 theta - (the quantity 1 - cos ^2 theta).
I put in what sin^2 is in terms of cos^2.
And so that's 2 cos ^2 of theta - 1.
There's this cos^2, which gets a plus sign.
Because of these two minus signs.
And there's the one that was there before, so altogether there are two of them.
I want to isolate what cosine is.
Or rather, what cos^2 is.
So let's solve for that.
So I'll put the 1 on the other side.
And I get 1 + cos 2 theta.
And then, I want to divide by this 2, and so that puts a 2 in this denominator here.
So some people call that the half angle formula.
What it really is for us is it's a way of eliminating powers from sines and cosines.
I've gotten rid of this square at the expense of putting in a 2 theta here.
We'll use that.
And, similarly, same calculation shows that the sin ^2 theta = 1 cos 2 theta / 2.
Same cosine, in that formula also, but it has a minus sign.
For the sin^2.
OK. so that's my little review of trig identities that we'll make use of as this lecture goes on.
I want to talk about trig identity.
Trig integrals, and you know some trig integrals, I'm sure, already.
Like, well, let me write the differential form first.
You know that d sin theta, or maybe I'll say d sin x, is, let's see, that's the derivative of sin x dx, right.
The derivative of sin x = cos x dx.
And so if I integrate both sides here, the integral form of this is the integral of the cos x dx.
Is the sin x + a constant.
And in the same way, d cos x = - sin x dx.
Right, the derivative of the cosine is - sine.
And when I integrate that, I find the integral of the sin x dx = - cos x + c. So that's our starting point.
And the game today, for the first half of the lecture, is to use that basic, just those basic integration formulas, together with clever use of trig identities in order to compute more complicated formulas involving trig functions.
So the first thing, the first topic, is to think about integrals of the form sin^n (x) cos^n(x).
Where here I have in mind m and n. Are non-negative integers.
So let's try to integrate these.
I'll show you some applications of these pretty soon.
Looking down the road a little bit, integrals like this show up in Fourier series and many other subjects in mathematics.
It turns out they're quite important to be able to do.
So that's why we're doing them now.
Well, so there are two cases to think about here.
When you're integrating things like this.
There's the easy case, and let's do that one first.
The easy case is when at least one exponent is odd.
That's the easy case.
So, for example, suppose that I wanted to integrate, well, let's take the case m = 1.
So I'm integrating sin^n (x) cos x dx.
I'm taking -- oh.
I could do that one.
Let's see if that's what I want to take.
Yeah.
My confusion is that I meant to have this a different power.
You were thinking that.
So let's do this case when m = 1.
So the integral I'm trying to do is any power of sin cos. Well, here's the trick.
Recognize, use this formula up at the top there to see cos x dx as something that we already have on the blackboard.
So, the way to exploit that is to make a substitution.
And substitution is going to be u = sin x.
And here's why.
Then this integral that I'm trying to do is the integral of u ^ n. That's already a simplification.
And then there's that cos x dx.
When you make a substitution, you've got to go all the way and replace everything in the expression by things involving this new variable that I've introduced.
So I'd better get rid of the cosine of x dx and rewrite it in terms of du or in terms of u.
And I can do that because du, according to that formula, is the cos x dx.
Let me put a box around that.
That's our substitution.
When you make a substitution, you also want to compute the differential of the variable that you substitute in.
So the cos x dx that appears here is just, exactly, du.
And I've replaced this trig integral with something that doesn't involve trig functions at all.
This is a lot easier.
We can just plug into what we know here.
This is (u ^ (n + 1) / n + 1) + a constant, and I've done the integral.
But I'm not quite done with the problem yet.
Because to be nice to your reader and to yourself, you should go back at this point, probably, go back and get rid of this new variable that you introduced.
You're the one who introduced this variable, you.
Nobody except you, really, knows what it is.
But the rest of the world knows what they asked for the first place that involved x.
So I have to go back and get rid of this.
And that's not hard to do in this case, because u = sin x.
And so I make this back substitution.
And that's what you get.
So there's the answer.
OK, so the game was, I use this odd power of the cosine here, and I could see it appearing as the differential of the sine.
So that's what made this substitution work.
Let's do another example to see how that works out in a slightly different case.
So here's another example.
Now I do have an odd power.
One of the exponents is odd, so I'm in the easy case.
But it's not 1.
The game now is to use this trig identity to get rid of the largest even power that you can, from this odd power here.
So use sin^2 x = 1 - cos^2 x, to eliminate a lot of powers from that odd power.
Watch what happens.
So this is not really a substitution or anything, this is just a trig identity.
This sin^3 = sin^2 sin.
And sin^2 = (1 - cos^2 x) And then I have the remaining sin x.
And then I have cos^2 x dx.
So let me rewrite that a little bit to see how this works out.
This is the integral of (cos^2 (x) -, and then there's the product of these two.
That's cos^ 4 x) sin of x dx.
So now I'm really exactly in the situation that I was in over here.
I've got a single power of a sine or cosine.
It happens that it's a sine here.
But that's not going to cause any trouble, we can go ahead and play the same game that I did there.
So, so far I've just been using trig identities.
But now I'll use a trig substitution.
And I think I want to write these as powers of a variable.
And then this is going to be the differential of that variable.
So I'll take u to be cosine of x, and that means that du = - sin x, dx.
There's the substitution.
So when I make that substitution, what do we get.
Cos^2 becomes u^2.
Cos^ 4 becomes u ^ 4, and sin x dx becomes not quite du, watch for the signum, watch for this minus sign here.
It becomes - du.
But that's OK.
The minus sign comes outside.
And I can integrate both of these powers, so I get - u ^3 / 3.
And then this 4th power gives me a 5th power, when I integrate.
And don't forget the constant.
Am I done?
Not quite done.
I have to back substitute and get rid of my choice of variable, u, and replace it with yours.
Questions?
[INAUDIBLE]
There should indeed.
I forgot this minus sign when I came down here.
So these two gang up to give me a plus.
Was that what the other question was about, too?
Thanks.
So let's back substitute.
And I'm going to put that over here.
And the result is, well, I just replace the u by cosine of x.
So this is - cos^3 x / 3 +, thank you, cos^5 x / 5 + c. And there's the answer.
By the way, you can remember one of the nice things about doing an integral is it's fairly easy to check your answer.
You can always differentiate the thing you get, and see whether you get the right thing when you go back.
It's not too hard to use the power rules and the differentiation rule for the cosine to get back to this if you want to check the work.
Let's do one more example, just to handle an example of this easy case, which you might have thought of at first.
Suppose I just want to integrate a cube.
Sin^3 x.
No cosine in sight.
But I do have an odd power of a trig function, of a sine or cosine.
So I'm in the easy case.
And the procedure that I was suggesting says I want to take out the largest even power that I can, from the sin^3.
So I'll take that out, that's a sin^2 and write it as 1 - cos^2.
Well, now I'm very happy.
Because it's just like the situation we had somewhere on the board here.
It's just like the situation we had up here.
I've got a power of a cos sin x dx.
So exactly the same substitution steps in.
You get, and maybe you can see what happens without doing the work.
Shall I do the work here?
I make the same substitution.
And so this is (1 - u ^2 )( - du).
Which is u - u ^3 / 3.
But then I want to put this minus sign in place, and so that gives me - u + u ^3 / 3 + a constant.
And then I back substitute and get cos x + cos^3 x / 3.
So this is the easy case.
If you have some odd power to play with, then you can make use of it and it's pretty straightforward.
OK the harder case is when you don't have an odd power.
So what's the program?
I'm going to do the harder case, and then I'm going to show you an example of how to integrate square roots.
And do an application, using these ideas from trigonometry.
So I want to keep this blackboard.
Maybe I'll come back and start here again.
So the harder case is when they're only even exponents.
I'm still trying to integrate the same form.
But now all the exponents are even.
So we have to do some game.
And here the game is use the half angle formula.
Which I just erased, very sadly, on the board here.
Maybe I'll rewrite them over here so we have them on the board.
I think I remember what they were.
So the game is I'm going to use that half angle formula to start getting rid of those even powers.
Half angle formula written like this, exactly, talks about, it rewrites, even powers of sines and cosines.
So let's see how that works out in an example.
How about just the cos^2 for a start.
What to do?
I can't pull anything out.
I could rewrite this as 1 - sin^2, but then I'd be faced with integrating the sin^2, which is exactly as hard.
So instead, let's use this formula here.
This is really the same as 1 + cos 2 theta / 2.
And now, this is easy.
It's got two parts to it.
Integrating one half gives me theta over -- oh.
Miraculously, the x turned into a theta.
Let's put it back as x. I get x / 2 by integrating 1/2.
So, notice that something non-trigonometric occurs here when I do these even integrals.
x / 2 appears.
And then the other one, OK, so this takes a little thought.
The integral of the cosine is the sine, or is it minus the sine.
- sine.
Shall we take a vote?
I think it's positive.
And so you get the sin 2x, but is that right?
Over 2.
If I differentiate the sin 2x, this 2 comes out.
And would give me an extra 2 here.
So there's an extra 2 that I have to put in here when I integrate it.
And there's the answer.
This is not a substitution.
I just played with trig identities here.
And then did a simple trig integral, getting your help to get the sign right.
And thinking about what this 2 is going to do.
It produces a 2 in the denominator.
But it's not applying any complicated thing.
It's just using this identity.
Let's do another example that's a little bit harder.
This time, sin^2 cos^2.
Again, no odd powers.
I've got to work a little bit harder.
And what I'm going to do is apply those identities up there.
Now, what I recommend doing in this situation is going over to the side somewhere.
And do some side work.
Because it's all just playing with trig functions.
It's not actually doing any integrals for a while.
So, I guess one way to get rid of the sin^2 and the cos^2 is to use those identities and so let's do that.
So the sin = 1 - cos 2x / 2.
And the cos = 1 + cos 2x / 2.
So I just substitute them in.
And now I can multiply that out.
And what I have is a difference times a sum.
So you know a formula for that.
Taking the product of these two things, well there'll be a 4 in the denominator.
And then in the numerator, I get the square of this minus the square of this.
(a - b)( a + b) = a ^2 - b^2.
So I get that.
Well, I'm a little bit happier, because at least I don't have 4.
I don't have 2 different squares.
I still have a square, and want to integrate this.
I'm still not in the easy case.
I got myself back to an easier hard case.
But we do know what to do about this.
Because I just did it up there.
And I could play into this formula that we got.
But I think it's just as easy to continue to calculate here.
Use the half angle formula again for this, and continue on your way.
So I get a 1/4 from this bit.
And then - 1/4 of the cos^2 2x.
And when I plug in 2x in for theta, there in the top board, I'm going to get a 4x on the right-hand side.
So it comes out like that.
And I guess I could simplify that a little bit more.
This is a 1/4.
Oh, but then there's a 2 here.
It's half that.
So then I can simplify a little more.
It's 1/4 - 1/8, which is 1/8.
And then I have 1/8 cos 4x.
OK, that's my side work.
I just did some trig identities over here.
And rewrote sine squared times cosine squared as something which involves just no powers of trig, just cosine by itself.
And a constant.
So I can take that and substitute it in here.
And now the integration is pretty easy.
(1/8 - cos 4x / 8) dx, which is, OK the 1/8 is going to give me x / 8.
The integral or cosine is plus or minus the sine.
The derivative of the sine is plus the cosine.
So it's going to be plus the, only there's a minus here.
So it's going to be the sin - the sin 4x / 8, but then I have an additional factor in the denominator.
And what's it going to be?
I have to put a 4 there.
So we've done that calculation, too.
So any of these, if you keep doing this kind of process, these two kinds of procedures, you can now integrate any expression that has a power of a sine times a power of a cosine in it, by using these ideas.
Now, let's see.
Oh, let me give you an alternate method for this last one here.
I know what I'll do.
Let me give an alternate method for doing, really doing the side work over there.
i'm trying to deal with sin^2 cos^2.
Well that's the square of the sin x cos x.
And the sin x cos x shows up right here.
In another trig identity.
So we can make use of that, too.
That reduces the number of factors of sines and cosines by 1.
So it's going in the right direction.
This is equal to 1/2 sine 2x, squared.
Sin cos = 1/2, say this right.
It's sin 2x / 2, and then I want to square that.
So what I get is the sin^2 2x / 4.
Which is, well, I'm not too happy yet, because I still have an even power.
Remember I'm trying to integrate this thing in the end, even powers are bad.
I try to get rid of them.
By using that formula, the half angle formula, so I can apply that to the sin x here again.
I get 1/4 of 1 - cos of 4x / 2.
That's what the half angle formula says for the sin^2 2x.
And that's exactly the same as the expression that I got up here, as well.
It's the same expression that I have there.
So it's the same expression as I have here.
So this is just an alternate way to play this game of using the half angle formula.
OK, let's do a little application of these things and change the topic a little bit.
So here's the problem.
So this is an application and example of a real trig substitution.
So here's the problem I want to look at.
OK, so I have a circle whose radius is a.
And I cut out from it a sort of tab, here.
This tab here.
And the height of this thing is b.
So this length is a number b.
And what I want to do is compute the area of that little tab.
That's the problem.
So there's an arc over here.
And I want to find the area of this, for a and b, in terms of a and b.
So the area, well, I guess one way to compete the area would be to take the integral of y dx.
You've seen the idea of splitting this up into vertical strips whose height is given by a function, y (x).
And then you integrate that.
That's an interpretation for the integral.
The area is given by y dx.
But that's a little bit awkward, because my formula for y is going to be a little strange.
Its constant value of b along here, and then at this point it becomes this arc, of the circle.
So working this out, I could do it but it's a little awkward because expressing y as a function of x, the top edge of this shape, it's a little awkward and takes two different regions to express.
So, a different way to say it is to say x dy.
Maybe that'll work a little bit better.
Or maybe it won't, but it's worth trying.
I could just as well split this region up into horizontal strips.
Whose width is dy, and whose length is x.
Now I'm thinking of this as a function of y.
This is the graph of a function of y.
And that's much better, because the function of y is, well, it's the square root of a^2 - y ^2, isn't it.
That's x ^2 + y ^2 = a ^2.
So that's what x is.
And that's what I'm asked to integrate, then.
Square root of (a ^2 - y ^2) dy.
And I can even put in limits of integration.
Maybe I should do that, because this is supposed to be an actual number.
I guess I'm integrating it from y = 0, that's here.
To y = b, dy.
So this is what I want to find.
This is a integral formula for the area of that region.
And this is a new form.
I don't think that you've thought about integrating expressions like this in this class before.
So, it's a new form and I want to show you how to do it, how it's related to trigonometry.
It's related to trigonometry through that exact picture that we have on the blackboard.
After all, this a^2 - y ^2 is the formula for this arc.
And so, what I propose is that we try to exploit the connection with the circle and introduce polar coordinates.
So, here if I measure this angle then there are various things you can say.
Like the coordinates of this point here are (a, cosine theta, a.
Well, I'm sorry, it's not.
That's an angle, but I want to call it theta 0.
And, in general you know that the coordinates of this point are (a, cosine theta, a, sine theta).
If the radius is a, then the angle here is theta.
So x is a cosine theta, and y is a sine theta, just from looking at the geometry of the circle.
So let's make that substitution.
y = a sine theta.
I'm using the picture to suggest that maybe making the substitution is a good thing to do.
Let's follow along and see what happens.
If that's true, what we're interested in is integrating a^2 - y ^2.
Which is a ^2, we're interested in integrating the square root of (a ^2 - y^2).
Which is the square root of a ^2 minus this.
a ^2 sin ^2 theta.
And, well, that's = a cos theta.
That's just sin^2 + cos^2 = 1, all over again.
It's also x.
This is x.
And this was x.
So there are a lot of different ways to think of this.
But no matter how you say it, the thing we're trying to integrate, a squared, a ^2 - y ^2 is, under this substitution it is a cos theta.
So I'm interested in integrating the square root of (a ^2 - y ^2) dy.
And I'm going to make this substitution y = a sin theta.
And so under that substitution, I've decided that the square root of a ^2 - y ^2 = a cos theta.
That's this.
What about the dy?
Well, I'd better compute the dy.
So dy just differentiating this expression, is a cos theta d theta.
So let's put that in.
dy = a cos theta d theta.
OK. Making that trig substitution, y = a sin theta has replaced this integral that has a square root in it.
And no trig functions.
With an integral that involves no square roots and only trig functions.
In fact, it's not too hard to integrate this now, because of the work that we've done.
The a ^2 comes out.
This is cos^2 theta.
d theta.
And maybe we've done that example already today.
I think we have.
Maybe we can think it through, but maybe the easiest thing is to look back at notes and see what we got before.
That was the first example in the hard case that I did.
And what it came out to, I used x instead of theta at the time.
So this is a good step forward.
I started with this integral that I really didn't know how to do by any means that we've had so far.
And I've replaced it by a trig integral that we do know how to do.
And now I've done that trig integral.
But we're still not quite done, because of the problem of back substituting.
I'd like to go back and rewrite this in terms of the original variable, y.
Or, I'd like to go back and rewrite it in terms of the original limits of integration that we had in the original problem.
In doing that, it's going to be useful to rewrite this expression and get rid of the sin 2 theta.
After all, the original y was expressed in terms of the sin theta, not the sin 2 theta.
So let me just do that here, and say that this, in turn, is equal to a^2 theta / 2 +, well, the sin 2 theta = 2 sin theta cos theta.
And so, when there's a 4 in the denominator, what I'll get is sin theta cos theta / 2.
I did that because I'm getting closer to the original terms that the problem started with.
Which was sin theta.
So let me write down the integral that we have now.
The square root of a ^2 - y ^2 dy is, so far, what we know is a ^2 (theta / 2 + sin theta cos theta / 2) + c. But I want to go back and rewrite this in terms of the original value.
The original variable, y.
Well, what is theta in terms of y?
Let's see.
y in terms of theta was given like this.
So what is theta in terms of y?
Ah.
So here the fearsome arc sin rears its head, right?
Theta is the angle so that y = a sin theta.
So that means that theta is the arc sign, or sine inverse, of y / a.
So that's the first thing that shows up here.
Arc sin (y / a).
All over 2.
That's this term.
Theta is the arc sin ( y /a) / 2.
What about the other side, here?
Well sine and cosine, we knew what they were in terms of y and in terms of x, if you like.
Maybe I'll put the a ^2 inside here.
That makes it a little bit nicer.
Plus, and the other term is a ^2 (sin theta cos theta).
So the a sin theta is just y.
Maybe I'll write this (a sin theta)( a cos theta) / 2 + c. And so I get the same thing.
And now here a sin theta, that's y.
And what's the a cos theta?
It's x, or, if you like, it's the square root of a ^2 - y ^2.
And so there I've rewritten everything, back in terms of the original variable, y.
And there's an answer.
So I've done this indefinite integration of a form of this quadratic, this square root of something which is a constant - y ^2.
Whenever you see that, the thing to think of is trigonometry.
That's going to play into the sin^ 2 + cos^2 identity.
And the way to exploit it is to make the substitution y = a sin theta.
You could also make a substitution y = a cos theta, if you wanted to.
And the result would come out to exactly the same in the end.
I'm still not quite done with the original problem that I had, because the original problem asked for a definite integral.
So let's just go back and finish that as well.
So the area was the integral from 0 to b of this square root.
So I just want to evaluate the right-hand side here.
The answer that we came up with, this indefinite integral.
I want to evaluate it at 0 and at b.
Well, let's see.
When I evaluate it at b, I get a ^2 ( arc sin (b / a) / 2 + y, which is b times the square root of a ^2 - b ^2, putting y = b, divided by 2.
So I've plugged in y = b into that formula, this is what I get.
Then when I plug in y = 0, well the, sine of 0 is 0, so the arc sine of 0 is 0.
So this term goes away.
And when y = 0, this term is 0 also.
And so I don't get any subtracted terms at all.
So there's an expression for this.
Notice that this arc sin ( b / a), that's exactly this angle.
The arc sin ( b / a), it's the angle that you get when y = b.
So this theta is the arc sin (b / a).
Put this over here.
That is theta 0.
That is the angle that the corner makes.
So I could rewrite this as a ^2 theta 0 / 2 + b times the square root of a ^2 - b ^2 / 2.
Let's just think about this for a minute.
I have these two terms in the sum, is that reasonable?
The first term is a ^2.
That's the radius squared times this angle, times 1/2.
Well, I think that is exactly the area of this sector.
a ^2 theta / 2 is the formula for the area of the sector.
And this one, this is the vertical elevation.
This is the horizontal.
a ^2 - b ^2 is this distance.
Square root of a ^2 - b ^2.
So the right-hand term is b times the square root of a ^2 - b ^2 / 2, that's the area of that triangle.
So using a little bit of geometry gives you the same answer as all of this elaborate calculus.
Maybe that's enough cause for celebration for us to quit for today.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're going to continue to talk about trig integrals and trig substitutions.
This is maybe the most technical part of this course, which maybe is why professor Jerison decided to just take a leave, go AWOL just now and let me take over for him.
But I'll do my best to help you learn this technique and it'll be useful for you.
So we've talked about trig integrals involving sines and cosines yesterday.
There's another whole world out there that involves these other trig polynomials.
Trig functions.
Secant and tangent.
Let me just make a little table to remind you what they are.
Because I have trouble remembering myself, so I enjoy the opportunity to go back to remind myself of this stuff.
Let's see.
The secant is one over one of those things, which one is it?
It's weird, it's 1 / cos. And the cosecant = 1 / sin.
Of course the tangent, we know.
It's the sin / cos and the cotangent is the other way around.
So when you put a co in front of it, it exchanges sine and cosine.
Well, I have a few identities involving tangent and secant up there, in that little prepared blackboard up above.
Maybe I'll just go through and check that out to make sure that we're all on the same page with them.
So I'm going to claim that there's this trig identity at the top.
Sec^2 = 1 + tangent.
So let's just check that out.
So the sec = 1 / cos, so sec ^2 = 1 / cos ^2.
And then whenever you see a 1 in trigonometry, you'll always have the option of writing as cos ^2 + sin^2.
And if I do that, then I can divide the cos ^2 into that first term.
And I get 1 + sin ^2 / cos ^2.
Which is the tan ^2.
So there you go.
That checks the first one.
That's the main trig identity that's going to be behind what I talk about today.
That's the trigonometry identity part.
How about this piece of calculus.
Can we calculate what the derivative of the tan x is.
Actually, I'm going to do that on this board.
So the tangent of x = sin x / cos x.
So I think I was with you when we learned about the quotient rule.
Computing the derivative of a quotient.
And the rule is, you take the numerator and you -- sorry, you take the derivative of the numerator, which is cosine.
And you multiply it by the denominator, so that gives you cos ^2.
And then you take the numerator, take minus the numerator, and multiply that by the derivative of the denominator, which is - sin x.
And you put all that over the square of the denominator.
And now I look at that and before my eyes I see the same trig identity, cosine ^2 + sine ^2 = 1, appearing there.
This is 1 / cosine ^2 x, which is secant ^2.
And, good.
So that's what the claim was.
The derivative of the tangent is the secant ^2.
That immediately gives you an integral.
Namely, the integral of secant ^2 is the tangent.
That's the fundamental theorem of calculus.
So we verified the first integral there.
Well, let's just do the second one as well.
So if I want to differentiate the secant, derivative of the secant.
So that's d/dx of 1 / cosine.
And again, I have a quotient.
This one's a little easier because the numerator's so simple.
So I take the derivative of the numerator, which is 0.
And then I take the numerator, I take minus the numerator your times the derivative of the denominator.
Which is - sin x, and put all that over the square of the same denominator.
So 1 - sin x came from the quotient rule, and the other one came because that's the derivative of the cosine.
But they cancel, and so I get sin / cos ^2, which is (sin / cos) ( 1 / cos) and so that's the secant, that's (1 / cos), times tan x.
So, not hard.
That verifies that the c derivative of secant is secant tangent.
And it tells you that the integral of that weird thing in case you ever want to know, the integral of the secant tangent is the secant.
Well, there are a couple more integrals that I want to do for you.
Where I can't sort of work backwards like that.
Let's calculate the integral of the tangent.
Just do this straight out.
So the tangent is the sine / cosine.
And now there's a habit of mine, that I hope you get into.
When you see the cosine and you're calculating an integral like this, it's useful to remember what the derivative of the cosine is.
Because maybe it shows up somewhere else in the integral.
And that happens here.
So that suggests we make a substitution.
u = cos x.
Which means du = - sin x dx.
That's the numerator, except for the minus sign.
And so I can rewrite this as, under the substitution, I can rewrite this as - du, that's the numerator, sin x dx is - du / by u.
Well, I know how to do that integral too.
That gives me the natural log, doesn't it.
So this is - ln u + a constant.
I'm not quite done.
I have to back-substitute and replace this new variable that I've made up, called u, with what it is.
And what you get is - ln cos x.
So the integral of the tangent is - ln cos. Now, you find these tables of integrals in the back of the book.
Things like that.
I'm not sure how much memorization Professor Jerison is going to ask of you, but there is a certain amount of memorization that goes on in calculus.
And this is one of the kinds of things that you probably want to know.
Let me do one more integral.
I think I'm making my way through a prepared board here, let's see.
Good.
So the integral of the tangent is - ln cos.
I'd also like to know what the integral of the secant of x is.
And I don't know a way to kind of go straight at this, but let me show you a way to think your way through to it.
If I take these two facts, tan' is what it is, and sec' is what it is, and add them together, I get this fact.
That the derivative of the sec x + tan x is, well, it's the sum of these two things.
Secant ^2 + secant tangent.
And there's a secant that occurs in both of those terms.
So I'll factor it out.
And that gives me, I'll put it over here.
There's the secant of x that occurs in both terms.
And then in one term, there's another secant.
And in the other term, there's a tangent.
So that's interesting somehow, because this same term appears on both sides of this equation.
Let's write u, for that secant x + tangent of x.
And so the equation that I get is u' = u (sec x).
I've just made a direct substitution.
Just decide that I'm going to write u for that single thing that occurs on both sides of the equation.
So u' is on the left, and u * sec x is on the right.
Well, there's my secant.
That I was trying to integrate.
And what it tells you is that the secant of x = u' / by u.
Just divide both sides by u, and I get this equation.
u' / u, that has a name.
Not sure that professor Jerison's used this in this class, but u' / u, we've actually used something like that.
It's on the board right now.
It's a logarithmic derivative.
It is the derivative of the national logarithm of u.
Maybe it's easier to read this from right to left, if I want to calculate the derivative of the logarithm, well, the chain rule says I get the derivative of u times the derivative of the ln function, which is 1 / u.
So often u' / u is called the logarithmic derivative.
But it's done what I wanted.
Because it's expressed the secant as a derivative.
And I guess I should put in what u is.
It's the secant + the tangent.
And so that implies that the integral, integrate both sides.
That says that the integral of the sec x dx, is ln( sec x + tan x).
So that's the last line in this little memo that I created.
That we can use now for the rest of the class.
Any questions about that trick?
It's a trick, I have nothing more to say about it.
OK.
So, the next thing I -- oh yes, so now I want to make the point that using these rules and some thought, you can now integrate most trigonometric polynomials.
Most things that involve powers of sines and cosines and tangents and secants and everything else.
For example, let's try to integrate the integral of sec^ 4 (x).
Big power of the secant function.
Well, there are too many secants there for me.
So let's take some away.
And I can take them away by using that trig identity, secant ^2 = 1 + tan^2.
So I'm going to replace two of those secants by 1 + tangent ^2.
That leaves me with two left over.
Now there was method to my madness.
Because I've got a secant ^2 left over there.
And secant ^2 is the derivative of tangent.
So that suggests a substitution.
Namely, let's say, let's let u = tangent of x, so that du = secant ^2 x dx.
And I have both terms that occur in my integral sitting there very nicely.
So this is the possibility of making this substitution and seeing a secant squared up here as part of the differential here.
That's why it was a good idea for me to take two of the secants and write them as 1 + tangent ^2.
So now I can continue this.
Under that substitution, I get 1.
Oh yeah, and I should add the other fact, that, well I guess it's obvious that tangent ^2 = u ^2.
So I get 1 + u ^2.
And then du sec^2 (x) dx, that is du.
Well that's pretty easy to integrate.
So I get u + u ^3 / 3.
Plus a constant.
And then I just have to back-substitute.
Put things back in terms of the original variables.
And that gives me tan x + tan ^3 / 3.
And there's the answer.
So we could spend a lot more time doing more examples of this kind of polynomial trig thing.
It's probably best for you to do some practice on your own.
Because I want to talk about other things, also.
And what I want to talk about is the use of these trig identities in making really trig substitution integration.
So we did a little bit of this yesterday, and I'll show you some more examples today.
Let's start with a pretty hard example right off the bat.
So this is going to be the integral of dx / x ^2 ( the square root of 1 + x ^2).
It's a pretty bad looking integral.
So how can we approach this?
Well, the square root is the ugliest part of the integral, I think.
What we should try to do is write this square root in some nicer way.
That is, figure out a way to write 1 + x ^2 as a square.
That'll get rid of the square root.
So there is an example of a way to write 1 plus something squared in a different way.
And it's right up there.
Secant ^2 = 1 + tangent ^2.
So I want to use that idea.
And when I see this form, that suggests that we make a trig substitution and write x as the tangent of some new variable.
Which you might as well call theta, to because it's like an angle.
Then 1 + x ^2 is the secant ^2.
According to that trig identity.
And so the square root of 1 + x ^2 = sec theta.
right?
So this identity is the reason that the substitution is going to help us.
Because it gets rid of the square root and replaces it by some other trig function.
I'd better be able to get rid of the dx, too.
That's part of the substitution process.
But we can do that, because I know what the derivative of the tangent is.
It's secant ^2.
So dx d theta is secant ^2 theta.
So dx is the secant ^2 theta ( d theta).
So let's just substitute all of that stuff in, and rewrite the entire integral in terms of our new variable, theta.
So dx is in the numerator.
That's secant ^2 theta d theta.
And then the denominator, well, it has an x ^2.
That's tangent ^2 theta.
And then there's this square root.
And we you know what that is in terms of theta.
It's secant of theta.
OK, now.
we've done the trig substitution.
I've gotten rid of the square root, I've got everything in terms of trig functions of the new variable.
Pretty complicated trig function.
This often happens, you wind up with a complete scattering of different trig functions in the numerator and denominator and everything.
A systematic thing to do here is to put everything in terms of sines and cosines.
Unless you can see right away, how it's going to simplify, the systematic thing to do is to rewrite in terms of sines and cosines.
So let's do that.
So let's see.
The secant ^2, secant is 1 / cosine.
So I'm going to put a cosine ^2 in the denominator.
Oh, I guess the first thing I can do is cancel.
Let's do that.
That's clever.
You were all thinking that too.
Cancel those.
So now I just get one cosine denominator from the secant there in the numerator.
It's still pretty complicated, secant / tangent ^2, who knows.
Well, we'll find out.
Because the tangent is sine / cosine.
So I should put a sine ^2 where the tangent was, and a cosine ^2 up there.
And I still have d theta.
And now you see some more cancellation occurs.
That's the virtue of writing things out in this way.
So now, the square here cancels with this cosine.
And I'm left with cosine theta d theta, / sine square root of theta.
That's a little simpler.
And it puts me in a position to use the same idea I just used.
I see the sine here.
I might look around in this integral to see if its derivative occurs anywhere.
The differential of the sine is the cosine.
And so I'm very much inclined to make another substitution.
Say, u, direct substitution this time.
And say u as the cosine of theta.
Because then du -- oh, I'm sorry.
Say, u as the sine of theta.
Because then du is cosine of theta d theta.
And then this integral becomes, well, the numerator just is du.
The denominator is u ^2.
And I think we can break out the champagne, because we can integrate that one.
Finally get rid of the integral sign.
Yes sir
[INAUDIBLE]
OK, how do I know to make u = sine rather than cosine.
Because I want to see du appear up here.
If I'd had a sine up here, that would be a signal to me that maybe I should say let u be the cosine.
OK?
Also, because this thing in the denominator is something I want to get rid of.
It's in the denominator.
So I'll get rid of it by wishful thinking and just call it something else.
It works pretty well in this case.
Wishful thinking doesn't always work so well.
So I integrate u ^ - 2 du, and I get - 1 / u + a constant, and I'm done with the calculus part of this problem.
I've done the integral now.
Gotten rid of the integral sign.
But I'm not quite done with the problem yet, because I have to work my way back through two substitutions.
First, this one.
And then this one.
So this first substitution isn't so bad to get rid of.
To undo.
To back-substitute.
Because u is just the sine of theta.
And so 1 / u is, I guess a fancy way to write it is the cosecant of theta.
1 / sin = cosecant.
So I get - cosecant of theta + a constant.
Is there a question in the back?
Yes sir?
[INAUDIBLE]
I'm sorry, my hearing is so bad
[INAUDIBLE]
How did I know substitution in the first place.
It's because of the 1 + x ^2.
And I want to make use of the trig identity in the upper left-hand corner.
I'll make you a table in a few minutes that will put all this in a bigger context.
And I think it'll help you then.
OK, I'll promise.
So, what I want to try to talk about right now is how to rewrite a term like this.
A trig term like this, back in terms of x.
So I want to undo this trick substitution.
This is a trig sub.
And what I want to do now is try to undo that trig sub.
And I'll show you a general method for undoing trig substitutions.
This happens quite often.
I don't know what the cosecant of theta is.
But I do know what the tangent of theta is.
So I want to make a relation between them.
OK, so undoing.
Trig subs.
So let's go back to where trigonometry always comes from, this right angled triangle.
The theta in the corner, and then these three sides.
This one's called the hypotenuse.
This one is called the adjacent side, and that one's called the opposite side.
And now, let's find out where x lies in this triangle.
Let's try to write the sides of this triangle in terms of x.
And what I know is, x is the tangent of theta.
So the tangent of theta, tangent of this angle, is opposite / adjacent.
Did you learn SOH CAH TOA.
OK, so it's opposite / adjacent.
Is the tangent.
So there are different ways to do that, but why not just do it in the simplest way and suppose that the adjacent is 1, and the opposite is x.
This is correct now, isn't it?
I get the correct value for the tangent of theta by saying that the lengths of those are 1 and x.
And that means that the hypotenuse has length 1 + x ^2.
Well, here's a triangle.
I'm interested in computing the cosecant of theta.
Where's that appear in the triangle?
Well, let's see.
The cosecant of theta = 1 / sin.
And the sine is opposite / hypotenuse.
So the cosecant is hypotenuse / opposite.
And the hypotenuse is the square root of 1 + x ^2, and the opposite is x.
And so I've done it.
I've undone the trig substitution.
I've figured out what this cosecant of theta is, in terms of x.
And so the final answer is - square root of 1 + x ^2 / x + a constant, and there's an answer to the original problem.
This took two boards to go through this.
I illustrated several things.
Actually, this three half boards.
I illustrated this use of trig substitution, and I'll come back to that in a second.
I illustrated patience.
I illustrated rewriting things in terms of sines and cosines, and then making a direct substitution to evaluate an integral like this.
And then there's this undoing all of those substitutions.
And it culminated with undoing the trig sub.
So let's play a game here.
Why don't we play the game where you give me.
So, there's a step in here that I should have done.
I should've said this is - csc ( arc tan of theta) + a constant.
The most straightforward thing you can do is to say since x is the tangent of theta, that means that -- sorry, if x, that means that theta is the arc tangent of x.
And so let's just put in theta as the arc tangent of x, and that's what you get.
So really, what I just did for you was to show you a way to compute some trig function applied to the inverse of another trig function.
I computed cosecant of the arc tangent by this trick.
So now, let's play the game where you give me a trig function and an inverse trig function, and I try to compute what the composite is.
OK.
So who can give me a trig function.
Has to be one of these standard ones
Tan
Tangent.
Alright.
How about another one?
Sine
Sine.
Do we have agreement on sine
[INAUDIBLE]
Secant?
[INAUDIBLE]
Right, csc has the best cheer.
So that's the game.
We have to compute, try to compute, that composite.
Something wrong with this?
[INAUDIBLE]
What does acceptable mean?
Don't you think, that so the question is, isn't this a perfectly acceptable final answer.
It's a correct final answer.
But this is much more insightful.
And after all the original thing was involving square roots and things, this is the kind of thing you might hope for is an answer.
This is just a nicer answer for sure.
And likely to be more useful to you when you go on and use that answer for something else.
OK, so let's try to do this this.
Undo a trig substitution that involved a cosecant.
And I manipulate around, and I find myself trying to find out what's the tangent of theta.
So here's how we go about it.
I draw this triangle.
Theta is the angle here.
This is the adjacent, opposite, hypotenuse.
So, the first thing is how can I make the cosecant appear here.
csc x.
What dimensions should I give to the sides in order for the cosecant of x, sorry, in order for theta to be the cosecant of x.
This thing is theta.
So, that means that the cosecant of x, that means the cosecant of theta should be x. Theta is the arc cosecant, so x is the cosecant of theta.
So, what'll I take the sides to be, to get the cosecant?
The cosecant is 1 / sine.
And the sine is the opposite / hypotenuse.
So I get hypotenuse / opposite.
And that's supposed to be what x is.
So I could make the opposite anything I want, but the simplest thing is to make it 1.
Let's do that.
And then what does that mean about the rest of the sides?
Hypotenuse had better be x.
And then I've recovered this.
So here's a triangle that exhibits the correct angle.
This remaining side is going to be useful to us.
And it is the square root of x^2 - 1.
So I've got a triangle of the correct angle theta, and now I want to compute the tangent of that angle.
Well, that's easy.
That's opposite / adjacent.
So I get 1 / square root of x ^2 - 1.
Very flexible tool that'll be useful to you in many different times.
Whenever you have to undo a trig substitution, this is likely to be useful.
OK, that was a good game.
No winners in this game.
We're all winners.
No losers, we're all winners.
OK.
So, good.
So let me make this table of the different trig substitutions, and how they can be useful.
Summary of trig substitutions.
So over here, we have, if you see, so if your integrand contains, make a substitution to get.
So if your integrand contains, I'll write these things out as square roots.
if it contains the square root of a ^2 - x ^2, this is what we talked about on Thursday.
When I was trying to find the area of that piece of a circle.
There, I suggested that we should make the substitution x = a cos theta.
Or, x = a sin theta.
Either one works just as well.
And there's no way to prefer one over the other.
And when you make the substitution, x = a cos theta, you get a ^2 - a ^2 cos ^2 theta.
1 - cos ^2 is the sin^2.
So you get a sine of theta.
So this expression becomes equal to this expression under that substitution.
And then you go on.
Then you've gotten rid of the square root, and you've got a trigonometric integral that you have to try to do.
If you made the substitution a sin theta, you'd get a ^2 - a ^2 sin^2, which = a cos theta.
And then you can go ahead as well.
We just saw another example.
Namely, if you have a ^2 + x ^2.
That's like the example we had up here.
a = 1 in this example.
What did we do?
We tried the substitution x = a tangent of theta.
And the reason is that I can plug into the trig identity up here in the upper left.
And replace a ^2 + x ^2 by a sec theta.
Square root of the sec^2.
There's one more thing in this table.
Sort of, the only remaining sum or difference of terms like this.
And that's what happens if you have x ^2 - a ^2.
So there, I think we can make a substitution a sec theta.
Because, after all, sec^2 theta, so x ^2 - a ^2.
Let's see what happens when I make that substitution.
x ^2 - a ^2 = a ^2 sec^2 theta - a ^2.
Under this substitution.
That's sec ^2 - 1.
Well, put the 1 on the other side.
And you find tan ^2, coming out.
So this is a ^2 ( tan ^2 of theta).
And so that's what you get.
a tan theta.
After I take the square root, I get a tan theta.
So these are the three basic trig substitution forms.
Where trig substitutions are useful to get rid of expressions like this, and replace them by trigonometric expressions.
And then you use this trick, you do the integral if you can and then you use this trick to get rid of the theta at the end.
So now, the last thing I want to talk about today is called completing the square.
And that comes in because unfortunately, not every square root of a quadratic has such a simple form.
You will often encounter things that are not just the square root of something simple.
Like one of these forms.
Like there might be a middle term in there.
I don't actually have time to show you an example of how this comes out in a sort of practical example.
But it does happen quite frequently.
And so I want to show you how to deal with things like the following example.
Let's try to integrate dx / x ^2 + 4x, the square root of x ^2 + 4x.
So there's a square root of some square, some quadratic.
It's very much like this business.
But it isn't of any of these forms.
And so what I want to do is show you how to rewrite it in one of those forms using substitution, again.
All this is about substitution.
So the game is to rewrite quadratic as something like x + something or other.
+ some other constant.
So write it, try to write it, in the form of a square + or - another constant.
And then we'll go on from there.
So let's do that in this case.
x ^2, x ^2 + 4x.
Well, if you square this form out, then the middle term is going to be 2ax.
So that, since I have a middle term here, I pretty much know what a has to be.
The only choice in order to get something like x ^2 + 4x out of this, is to take a to be 2.
Because then, this is what you get.
This isn't quite right yet, but let's compute what I have here.
x ^2 + 4x, so far so good.
+ 4, and I don't have a + 4 here.
So I have to fix that by subtracting 4.
So that's what I mean.
I've completed the square.
The word for this process of eliminating the middle term by using the square of an expression like that.
That's called completing the square.
And we can use that process to compute this integral.
So let's do that.
So I can rewrite this integral, rewrite this denominator like this.
And then I'm going to try to use one of these forms over here.
So in order to get a single variable there, instead of something complicated like x + 2, I'm inclined to come up with another variable name and write it equal, write x + 2 as that other variable name.
So here's another little direct substitution.
u = x + 2.
Figure out what du is.
That's pretty easy.
And then rewrite the integral in those terms.
So dx = du.
And then in the denominator I have, well, I have the square root of that.
Oh yeah, so I think as part of this I'll write out what x ^2 + 4x is.
The point is, it's equal to u ^2 - 4.
4. x ^2 + 4x = u ^2 - 4.
There's the data box containing the substitution data.
And so now I can put that in.
I have x ^2 + 4x there.
In terms of u, that's u ^2 - 4.
Well, now I'm a happier position because I can look for u ^2 - 4 for something like that in my table here.
And it actually sits down here.
So except for the use of the letter x here instead of u over there.
That tells me what I want.
So to handle this, what I should use is a trig substitution.
And the trig substitution that's suggested is, according to the bottom line with a = 2, so a ^2 = 4.
The suggestion is, I should take x but I'd better not use the letter x any more.
But I don't have a letter x, I have the letter u. I should take u = 2 secant.
And then some letter I haven't used before.
And theta is available.
This is a look-up table process.
I see the square root of u ^2 - 4, I see that that's of this form.
I'm instructed to make this substitution.
And that's what I just did.
Let's see how it works out.
So that means the du = 2, OK. What's the derivative of the secant?
Secant tangent.
So du = 2 secant theta tangent theta.
And u ^2 - 4 is, here's the payoff.
I'm supposed to be able to rewrite that in terms of the tangent.
According to this.
u ^2 - 4 = 4 secant ^2 - 4.
And secant ^2 - 1 = tangent ^2.
So this is 4 tangent ^2.
Of theta.
Right, yeah?
[INAUDIBLE]
But I squared it.
And now I'll square root it.
And I'll get a 2 and this tangent will go away.
So there's my data box for this substitution.
And let's go on to another board.
So where I'm at is the integral of du / square root of u ^2 - 4.
And I have all the data I need here to rewrite that in terms of theta.
So du = 2 sec theta tan theta d theta.
And the denominator is 2 tan theta.
Ha.
Well, so some very nice simplification happens here.
The 2's cancel.
And the tangents cancel.
And I'm left with trying to work with the integral, the secant theta d theta.
And luckily enough at the very beginning of the hour, I worked out how to compute the integral of the secant of theta.
And there it is.
So this is ln ( sec theta + tan theta) + a constant.
And we're done with the calculus part.
There's no more integral there.
But I still am not quite done with the problem, because again I have these two substitutions to try to undo.
So let's undo them one by one.
Let's see.
I have this trig substitution here.
And I could use my triangle trick, if I need to.
But maybe I don't need to.
Let's see, do I know what the secant of theta is in terms of u?
Well, I do.
So I get ln ( u / 2).
Do I know what the tangent is in terms of u?
Well, I do.
It's here.
So I lucked out, in this case.
And I don't have to go through and use that triangle trick.
So the tangent of theta is the square root of u ^2 - 4 / 2.
Good.
So I've undone this trig substitution.
I'm not quite done yet because my answer is involved with u.
And what I wanted originally was x.
But this direct substitution that I started with is really easy to deal with.
I can just put x + 2 every time I see a u.
So this is ln ( x + 2 / 2 + the square root... What's going to happen when I put x + 2 in place for u here?
You know what you get.
You get exactly what we started with.
Right?
I put x + 2 in place of the u here.
I get x ^2 + 4x.
So I've gotten back to a function purely in terms of x OK, that's a good place to quit.
Have a great little one-day break.
I guess, this class doesn't meet on Monday anyway.
Bye.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to continue our discussion of methods of integration.
The method that I'm going to describe today handles a whole class of functions of the following form.
You take P (x) / Q (x) and this is known as a rational function.
And all that means is that it's a ratio off two polynomials, which are these functions P ( x) and Q ( x).
We'll handle all such functions by a method which is known as partial fractions.
And what this does is, it splits P / Q into what you could call easier pieces.
So that's going to be some kind of algebra.
And that's what we're going to spend most of our time doing today.
I'll start with an example.
And all of my examples will be illustrating more general methods.
The example is to integrate the function 1 / x - 1 +, say, 3 / x + 2 dx.
That's easy to do.
It's just, we already know the answer.
It's ln x - 1 + 3 ln x + 3.
Plus a constant.
So that's done.
So now, here's the difficulty that is going to arise.
The difficulty is that I can start with this function, which is perfectly manageable.
And than I can add these two functions together.
The way I add fractions.
So that's getting a common denominator.
And so that gives me x + 2 here + 3 ( x - 1).
And now if I combine together all of these terms, then altogether I have 4x + 2 - 3, that's - 1.
And if I multiply out the denominator that's x ^2 + that 2 turned into a 3, that's interesting.
Hope there aren't too many more of those transformations.
Is there another one here?
[INAUDIBLE]
Oh, it happened earlier on.
Wow that's an interesting vibration there.
OK.
Thank you.
So, I guess my 3's were speaking to my 2's.
Somewhere in my past.
OK, anyway, I think this is now correct.
So the problem is the following.
This is the problem with this.
This integral was easy.
I'm calling it easy, we already know how to do it.
Over here.
But now over here, it's disguised.
It's the same function, but it's no longer clear how to integrate it.
If you're faced with this one, you say what am I supposed to do.
And we have to get around that difficulty.
And so what we're going to do is we're going to unwind this disguise.
So we have the algebra problem that we have.
Oh, wow.
There must be something in the water.
Impressive.
Wow.
OK, let's see.
Is 2/3 = 3/2?
Holy cow.
Well that's good.
Well, I'll keep you awake today with several other transpositions here.
So our algebra problem is to detect the easy pieces which are inside.
And the method that we're going to use, the one that we'll emphasize anyway, is one algebraic trick which is a shortcut, which is called the cover-up method.
But we're going to talk about even more general things than that.
But anyway, this is where we're headed.
Is something called the cover-up method.
Alright.
So that's our intro.
And I'll just have to remember that 2 is not 3.
I'll keep on repeating that.
So now here I'm going to describe to you how we unwind this disguise.
The first step is, we write down the function we want to integrate.
Which was this.
And now we have to undo the first damage that we did.
So the first step is to factor the denominator.
And that factors, we happen to know the factors, so I'm not going to carry this out.
But this can be a rather difficult step.
But we're going to assume that it's done.
For the purposes of illustration here.
So I factor the denominator.
And now, the second thing that I'm going to do is what I'm going to call the setup here.
How I'm going to set things up.
And I'll tell you what these things are more systematically in a second.
And the setup is that I want to somehow detect what I did before.
And I'm going to write some unknowns here.
What I expect is that this will break up into two pieces.
One with the denominator x - 1, and the other with the denominator x + 2.
So now, my third step is going to be to solve for A and B.
And then I'm done, if I do that.
That's the complete unwinding of this disguise.
And this is where the cover-up method comes in handy.
This is this method that I'm about to describe.
Now, you can do the algebra in a clumsy way, or you can do it in a quick way.
And we'd like to get efficient about the algebra involved.
And so let me show you what the first step in the trick is.
We're going to solve for A by multiplying by (x - 1).
Now, notice if you multiply by (x - 1) in that equation 2, what you get is this.
You got 4x - 2 / the x - 1's cancel.
You get this on the left-hand side.
And on the right-hand side you get A.
The x - 1's cancel again.
And then we get this extra term.
Which is B/ ( x + 2)( x - 1).
Now, the trick here, and we're going to get even better trick in just a second.
The trick here is that I didn't try to clear the denominators completely.
I was very efficient about the way I did it.
It just cleared one factor.
And the result here is very useful.
Namely, if I plug in now x = 1, this term drops out too.
So what I'm going to do now is I'm going to plug in x = 1.
And what I get on the left-hand side here is 4 - 1 and 1 + 2, and on the left-hand side I get A.
That's the end.
This is my formula for A.
A happens to be equal to 1.
And that's, of course, what I expect.
A had better be 1, because the thing broke up into 1 / x - 1 + 3 / x + 2.
So this is the correct answer.
There was a question out here, which I missed
Aren't polynomials defined as functions with whole powers, or could they be square roots?
Are polynomials defined as functions with whole powers, or can they be square roots?
That's the question.
The answer is, they only have whole powers.
So for instance here I only have the power 1 and 0.
Here I have the powers 2, 1 and 0 in the denominator.
Square roots are no good for this method.
Another question
[INAUDIBLE]
Why did I say x = 1?
The reason why I said x = 1 was that it works really fast.
You can't know this in advance, that's part of the method.
It just turns out to be the best thing to do.
The fastest way of getting at the coefficient A.
Now the curious thing, let me just pause for a second before I do it.
If I had plugged x = 1 into the original equation, I would have gotten nonsense.
Because I would've gotten 0 in the denominator.
And that seems like the most horrible thing to do.
The worst possible thing to do, is to set x = 1.
On the other hand, what we did is a trick.
We multiplied by x - 1.
And that turned the equation into this.
So now, in disguise, I multiplied by 0.
But that turns out to be legitimate.
Because really this equation is true for all x except 1.
And then instead of taking x = 1, I can really take x tends to 1.
That's really what I need.
The limit is x goes to one.
The equation is still valid then.
So I'm using the worst case, the case that looks like it's dividing by 0.
And it's helping me because it's cancelling out all the information in terms of B.
So the advantage here is this cancellation that occurs in this part.
So that's the method.
We're going to shorten it much, much more in a second.
But let me carry it out for the other coefficient as well.
So the other coefficient I'm going to solve for B, I'm going to multiply by x + 2.
And when I do that, I get 4x - 1 / x - 1, that's the left-hand side, the very top expression there.
And then down below I get A/ ( x - 1)( x + 2).
And then again the x + 2's cancel.
So I get B sitting alone.
And now I'm going to do the same trick.
I'm going to set x = - 2.
That's the value which is going to knock out this A term here.
So that cancels this term completely.
And what we get here all told is - 8 - 1 / - 2 - 1 = B.
In other words, B = 3, which was also what it was supposed to be.
B was this number 3, right here.
Which I'm now not going to change to 2.
Because I know that it's not 2.
There was a question
[INAUDIBLE]
All right.
Now, this is the method which is called cover-up.
But it's really carried out much, much faster than this.
So I'm going to review the method and I'm going to show you what it is in general.
So the first step is to factor the denominator, Q.
That's what I labeled 1 over there.
That was the factorization of the denominator up top.
The second step is what I'm going to call the setup.
That's step 2.
And that's where I knew what I was aiming for in advance.
And I'm going to have to explain to you in every instance exactly what this setup should be.
That is, what the unknowns should be and what target, simplified expression, we're aiming for.
So that's the setup.
And then the third step is what I'll now call cover-up.
Which is just a very fast way of doing what I did on this last board, which is solving for the unknown coefficients.
So now, let me perform it for you again.
Over here.
So it's 4x - 1 divided by, so this is to eliminate writing here.
Handwriting it makes it much faster.
So this part just factoring the denominator, that was 1, that was step 1.
And then step 2, again, is the setup, which is setting it up like this.
Alright, that's the setup.
And now I claim that without writing very much, I can figure out what A and B are.
Just by staring at this.
So now what I'm going to do is I'm just going to think what I did over there.
And I'm just going to do it directly.
So let me show you what the method consists of visually.
I'm going to cover up, that is, knock out this factor, and focus on this number here.
And I'm going to plug in the thing that makes the 0, which is x = 1.
So I'm plugging in x = 1.
To this left-hand side.
And what I get is 4 - 1 / 1 + 2 = A.
Now, that's the same thing I did over there.
I just did it by skipping the intermediate algebra step, which is a lot of writing.
So the cover-up method really amounts to the following thing.
I'm thinking of multiplying this over here.
It cancels this and it gets rid of everything else.
And it just leaves me with A on the right-hand side.
And I have to get rid of it on this side.
So in other words, by eliminating this, I'm isolating a on the right-hand side.
So the cover-up is that I'm covering this and getting A out of it.
Similarly, I can do the same thing with B.
It's focused on the value x = - 2.
And b is what I'm getting on the right-hand side.
And then I have to cover up this.
So if I cover up that, then what's left over with x = - 2 is again, - 8 - 1 / - 2 - 1.
So this is the way the method gets carried out in practice.
Writing, essentially, the least you can.
Now, when you get to several variables, it becomes just way more convenient to do this.
So now, let me just review when cover-up works.
So this cover-up method works if Q ( x) has distinct linear factors.
And, so you need two things here.
It has to factor completely, the denominator has to factor completely.
And the degree of the numerator has to be strictly less than the degree of the denominator.
I'm going to give you an example here.
So, for instance, and this tells you the general pattern of the setup also.
Say you had x ^2 + 3x + 8, let's say.
Over (x - 1) ( x - 2)( x + 5).
So here I'm going to tell you the setup.
The setup is going to be A / (x - 1) + B / (x - 2) + c / x + 5.
And it will always break up into something.
So however many factors you have, you'll have to put in a term for each of those.
And then you can find each number here by this cover-up method.
Now we're done with that.
And now we have to go on to the algebraic complications.
So would the first typical algebraic complication be.
It would be repeated roots or repeated factors.
Let me get one that doesn't come out to be extremely ugly here.
So this is what we'll call Example 2.
And this is going to work when the degree, you always need that the degree of the numerator is less than the degree of the denominator.
And Q has now repeated linear factors.
So let's see which example I wanted to show you.
So let's just give this here.
I'll just repeat the denominator.
With an extra factor like this.
Now, the main thing you need to know, since I've already performed the factorization for you.
Already performed Step 1.
This is Step 1 here.
You have to factor things all the way, and that's already been done for you.
And here's what this setup is.
The setup is that it's of the form A / (x - 1) + B / (x - 1) ^2.
We need another term for the square here.
+ C / (x + 2).
In general, if you have more powers you just need to keep on putting in those powers.
You need one for each of the powers.
Why does it have to be squared?
OK. Good question.
So why in the world am I doing this?
Let me just give you one hint as to why I'm doing this.
It's very, very much like the decimal expansion of a number or, say, the base 2 expansion of a number.
So, for, example the number 7/16 is 0 / 2 + 1 / 2 ^2 + 1/2 ^3 +, is that right?
So it's 4/16 + 1/2 ^4.
It's this sort of thing.
And I'm getting this power and this power.
If I have higher powers, I'm going to have to have more and more.
So this is what happens when I have a 2 ^ 4.
I have to represent things like this.
That's what's coming out of this piece with the repetitious here.
Of the powers.
This is just an analogy.
Of what we're doing.
Yeah, another question over here
[INAUDIBLE]
Yes.
So this is an example, but it's meant to represent the general case and I will also give you a general picture.
For sure, once you have the second power here, you'll need both the first and the second power mentioned over here.
And since there's only a first power over here I only have to mention a first power over there.
If this were a 3 here, there would be one more term which would be the one for x - 1 ^2 in the denominator.
That's what you just said.
OK, now, what's different about this setup is that the cover-up method, although it works, it doesn't work so well.
It doesn't work quite as well.
The cover-up works for the coefficients B and C, not A.
We'll have a quick method for the numbers B and C. To figure out what they are.
But it will be a little slower to get to A, which we will do last.
Let me show you how it works.
First of all, I'm going to do the ordinary cover-up with C. So for C, I just want to do the same old thing that I did before.
I cover up this, and that's going to get rid of all the junk except for the C term.
So I have to plug x = - 2.
And I get x -- sorry, I get (-2 ) ^2 + 2 in the numerator.
And I get (- 2 - 1)^2 in the denominator.
Remember I'm covering this up.
So that's all there is on the left-hand side.
And on the right-hand side all there is C. Everything else got killed off, because it was x - 2 times that.
That's 0 times all that other stuff.
And the x - 2 over here canceled.
This is the shortcut that I just described, and this is much faster than doing all that arithmetic.
And algebra.
So all together this is a 6/9, right?
So it's C = 6/9, which is 2/3.
Now, the other one which is easy to do, I'm going to do by the slow method first.
But you omit a term.
The idea is to cover up the other bad factor.
Cover ups, I'll do it both the way and the slow way.
I'll do it the fast way first, and then I'll show you the slow way.
The fast way is to cover this up.
And then I have to cover up everything else.
That gets eliminated.
And that includes everything but B.
So I get B on this side.
And I get 1 on that side.
So that's 1 ^2 + 2 / 1 + 2.
So in other words, B = 1.
That was pretty fast, so let me show you what arithmetic was hiding behind that.
What algebra was hiding behind it.
What I was really doing is this.
In multiplying through by x - 1 ^2, so I got this.
So this canceled here, so this C just stands alone.
And then I have here C /x + 2 (x - 1) ^2.
Notice again, I cleared out that 1, this term from the denominator and sent it over to the other side.
Now, what's happening is that when I set x = 1 here, this term is dying.
This term is going away, because there's more powers in the numerator than in the denominator.
This is still 0.
And this one is gone also.
So all that's left is B.
Now, I cannot pull that off with a single power of x - 1.
I can't expose the A term.
It's the B term that I can expose.
Because I can multiply through by this thing squared.
If I multiply through by just x - 1, what'll happen here is I won't have canceled this (x - 1 )^2.
It's useless.
I still have a 0 in the denominator.
I'll have B / 0 when I plug in x = 1.
Which I can't use.
Again, the cover-up method is giving us B and C, not A.
Now, for the last term, for A, I'm going to just have to be straightforward about it.
And so I'll just suggest for A, plug in your favorite number.
So plug in my favorite number.
Which is x = 0.
And you won't be able to plug in x = 0 if you've already used it.
Here the two numbers we've already used are x = 1 and x = - 2.
But we haven't used x = 0 yet, so that's good.
I'm going to plug in now x = 0 into the equation.
What do I get?
I get 0 ^2 + 2 / (- 1) ^2 * 2 is equal to, let's see.
A is the thing that I don't know.
So it's A / - 1 +, B / x - 1 ^2 so B = 1, so that's 1 / (- 1) ^2.
And then C was 2/3.
2/3 / x + 2.
So that's 0 + 2.
Don't give up at this point.
This is a lot of algebra.
You really have to plug in all these numbers.
You make one arithmetic mistake and you're always going to get the wrong answer.
This is very arithmetically intensive.
However, it does simplify at this point.
We have 2 / 2, that's 1.
Is equal to - A + 1 + 1/3.
So let's see.
A on the other side, this becomes A = 1/3.
And that's it.
This is the end.
We've we've simplified our function.
And now it's easy to integrate.
Question.
Another question
[INAUDIBLE]
So the question is, if x = 0 has already been used, what do I do?
And the answer is, pick something else.
And you said pick a random number.
And that's right, except that if you really picked a random number it would be 4.12567843, which would be difficult.
What you want to pick is the easiest possible number you can think of.
Yeah
[INAUDIBLE]
If you had, as in this sort of situation here.
More powers.
Wouldn't you have more variables.
Very good question.
That's absolutely right.
This was a 3 by 3 system in disguise, for these three unknowns, A, B and C. What we started with in the previous problem was two variables.
It's over here, the variables A and B.
And as the degree of the denominator goes up, the number of variables goes up.
It gets more and more and more complicated.
More and more arithmetically intensive
[INAUDIBLE]
Well, so.
The question is, how would you solve it if you have two unknowns.
That's exactly the point here.
This is a system of simultaneous equations for unknowns.
And we have little tricks for isolating single variables.
Otherwise we're stuck with solving the whole system.
And you'd have to solve the whole system by elimination, various other tricks.
I'll say a little more about that later.
Now, I have to get one step more complicated with my next example.
My next example is going to have a quadratic factor.
So still I'm sticking to the degree of the polynomial and the numerator is less than the degree of the polynomial in the denominator.
And I'm going to take the case where Q has a quadratic factor.
Let me just again illustrate this by example.
I have here (x - 1) (x^2 + 1).
I'll make it about as easy as they come.
Now, the setup will be slightly different here.
Here's the setup.
It's already factored.
I've already done as much as I can do.
I can't factor this x^2 + 1 into linear factors unless you know about complex numbers.
If you know about complex numbers this method becomes much easier.
And it comes back to the cover-up method.
Which is the way that the cover-up method was originally conceived by heavy side.
But you won't get to that until 18.03.
So we'll wait.
This, by the way, is a method which is used for integration.
But it was invented to do something with Laplace transforms and inversion of certain kinds of differential equations.
By heavy side.
And so it came much later than integration.
But anyway, it's a very convenient method.
So here's the set up with this one.
Again, we want a term for this (x - 1) factor.
And now we're going to also have a term with the denominator x squared plus 1.
But this is the difference.
It's now going to be a first degree polynomial.
One degree down from the quadratic here.
So this is what I keep on calling the setup, this is number 2.
You have to know that in advance based on the pattern that you see on the left-hand side.
Yes
[INAUDIBLE]
The question is, if the degree of the numerator.
So in this case, if this were cubed, and this is matching with the denominator, which is total of degree 3.
The answer is that this does not work
[INAUDIBLE]
It definitely doesn't work.
And we're going to have to do something totally different to handle it.
Which turns out, fortunately, to be much easier than this.
But we'll deal with that at the end.
Keep this in mind.
This is an easy way to make a mistake if you start with a higher degree numerator.
You'll never get the right answer.
So now, so I have my setup now.
And now what can I do?
Well, I claim that I can still do cover-up for A.
It's the same idea.
I cover this guy up.
And if I really multiply by it it would knock everything out but A.
So I cover this up and I plug in x = 1.
So I get here 1 ^2 / 1 ^2 + 1 = A.
In other words, A = 1/2.
Again cover-up is pretty fast, as you can see.
It's not too bad.
Now, at this next stage, I want to find B and C. And the best idea is the slow way.
Here, it's not too terrible.
But it's just what we're going to do.
Which is to clear the denominators completely.
So for B and C, just clear the denominator.
That means multiply through by that whole business.
Now, when you do that on the left-hand side you're going to get x ^2.
Because I got rid of the whole denominator.
On the right-hand side when I bring this up, the x - 1 will cancel with this.
So the a term will be A ( x ^2 + 1).
And the Bx + C term will have a remaining factor of x - 1.
Because the x ^2 + 1 will cancel.
Again, the arithmetic here is not too terrible.
Now I'm going to do the following.
I'm going to look at the x ^2 term.
On the left-hand side and the right-hand side.
And that will give me one equation for B and C. And then I'm going to do the same thing with another term.
The x^2 term on the left-hand side, the coefficient is 1.
It's 1 ( x ^2).
On the other side, it's A. remember I actually have A.
So I'm going to put it in, it's 1/2.
So this is the A term.
And so I get 1/2 ( x ^2).
And then the only other x^2 is when this Bx multiplies this x.
So Bx * x is Bx ^2, so this is the other coefficient on x ^2 is B.
And that forces B to be 1/2.
And last of all, I'm going to do the x^ 0 power term.
Or, otherwise known as the constant term.
And on the left-hand side, the constant term is 0.
There is no constant term.
On the right-hand side there's a constant term, 1/2 * 1.
That's 1/2 here.
And then there's another constant term, which is this constant times this - 1.
Which is - C. And so the conclusion here is that C = 1/2.
Another question.
Yeah
[INAUDIBLE]
There's also an x^ 0 power hidden in here.
Sorry, an x ^ 1 , that's what you were asking about, sorry.
There's also an x ^ 1 .
The only reason why I didn't go to the x^ 1 is that it turns out with these two I didn't need it.
The other thing is that by experience, I know that the extreme ends of the multiplication are the easiest ends.
And the middle terms have tons of cross terms.
And so I don't like the middle term as much because it always involves more arithmetic.
So I stick to the lowest and the highest terms if I can.
So that was really a sneaky thing.
I did that without saying anything.
Yes
[INAUDIBLE]
Another good question.
Could I just set equals 0?
Absolutely.
In fact, that's equivalent to picking out the x^ 0 term.
And you could plug in numbers.
If you wanted.
That's another way of doing this besides doing that.
So you can also plug in numbers.
Can plug in numbers.
x = 0.
Actually, not x = 1, right?
- 1, 2, etc.
Not 1 just because we've already used it.
We won't get interesting information out.
Yes
[INAUDIBLE]
So the question is, could I have done it this other way.
With the polynomial, with this other one.
Yes, absolutely.
So in other words what I've taught you now is two choices which are equally reasonable.
The one that I picked was the one that was the fastest for this guy and the one that was fastest for this one, but I could've done the other way around.
There are a lot of ways of solving simultaneous equations.
Yeah, another question
[INAUDIBLE]
The question is the following.
So now everybody can understand the question.
If this, instead of being x ^2 + 1, this were x^3 + 1.
So that's an important case to understand.
That's a case in which this denominator is not fully factored.
So it's an x^3 + 1, you would have to factor out an x + 1.
So that would be a situation like this, you have an x^3 + 1, but that's (x + 1)( x^2 + x + 1), this kind of thing.
If that's the right, there must be a minus sign in here maybe.
OK, something like this.
Right?
Isn't that what it is?
[INAUDIBLE]
I think it's right.
But anyway, the point is that you have to factor it.
And then you have a linear and a quadratic.
So you're always going to be faced eventually with linear factors and quadratic factors.
If you have a cubic, that means you haven't factored sufficiently.
So you're still back in Step 1
[INAUDIBLE]
In the x^3 + 1 case?
[INAUDIBLE]
In the x^3 + 1 case, we are out of luck until we've completed the factorization.
Once we've completed the factorization, we're going to have to deal with these two factors as denominators.
So it'll be this plus something over x + 1 + a Bx + C type of thing over this thing here.
That's what's eventually going to happen.
But hold on to that idea.
Let me carry out one more example here.
So I've figured out what all the values are.
But I think it's also worth it to remember now that we also have to carry out the integration.
What I've just shown you is that the integral of x ^2 dx over (x - 1)( x ^2 + 1) is equal to, and I've split up into these pieces.
So what are the pieces?
The pieces are, 1/2, x - 1 + 1/2 x / x ^2 + 1.
This is the A term.
This is the B term.
And then there's the C term.
So we'd better remember that we know how to antidifferentiate these things.
In other words, I want to finish the problem.
The others were pretty easy, so I didn't bother to finish my sentence, but here I want to be careful and have you realize that there's something a little more to do.
First of all we have the, the first one is no problem.
That's this.
The second one actually is not too bad either.
This is, by the advanced guessing method, my favorite method, something like the logarithm, because that's what's going to appear in the denominator.
And then, if you differentiate this, you're going to get 2x over this.
But here we have 1/2.
So altogether it's 1/4 of this.
So I fixed the coefficient here.
And then the last one, you have to think back to some level of memorization here and remember that this is 1/2 the arc tangent
[INAUDIBLE]
Why did I go to 1/4?
Because in disguise, for this guy, I was thinking d / dx of ln (x ^2 + 1) = 2x / x^2 + 1.
Because it's the derivative of this divided by itself.
This is the derivative of ln u is u ' / u. Ln u' = u' / u.
That was what I applied.
And what I had was 1/2, so I need a total of 1/4 to cancel.
So 2/4 is 1/2.
Now I've got to get you out of one more deep hole.
And I'm going to save the general pattern for next time.
But I do want to clarify one thing.
So let's handle this thing.
What if the degree of P is bigger than or equal to the degree of Q.
That's the thing that I claimed was easier.
And I'm going to describe to you how it's done.
Now, in analogy, with numbers you might call this an improper fraction.
That's the thing that should echo in your mind when you're thinking about this.
And I'm just going to do it by example here.
Let's see., I cooked up an example so that I don't make an arithmetistic mistake along the way.
So there are two or three steps that I need to explain.
So here's an example.
The denominator's degree 2, the numerator is degree 3.
It well exceeds, so there's a problem here.
Our method is not going to work.
And the first step that I want to carry out is to reverse Step 1.
That is, I don't want the factorization for what I'm going to do next.
I want it multiplied out.
That means I have to multiply through, so I get x ^2 + x - 2.
I'm back to the starting place here.
And now, the next thing that I'm going to do is, I'm going to use long division.
That's how you convert an improper fraction to a proper fraction.
This is something you were supposed to learn in, I don't know, Grade 4, I know.
Grade 3, Grade 4, Grade 5, Grade 6, etc.
So here's how it works in the case of polynomials.
It's kind of amusing.
So we're dividing this polynomial into that one.
And so the quotient to first order here is x.
That is, that's going to match the top terms.
So I get x^3 + x ^2 - 2x.
That's the product.
And now I subtract.
And it cancels.
So we get here - x ^2 + 2x.
That's the difference.
And now I need to divide this next term in.
And I need a - 1.
So - 1 times this is - x ^2 - x + 2.
And I subtract.
And the x^2's cancel.
And here I get + 3x - 2.
Now, this thing has a name.
This is called the quotient.
And this thing also has a name.
This is called the remainder.
And now I'll show you how it works by sticking it into the answer here.
The quotient is x - 1.
And the remainder is, let's get down there.
3x - 2 / x ^2 + x - 2.
So the punchline here is that this thing is easy to integrate.
This is easy.
And this one, you can use, now you can use cover-up, The method that we had before.
Because the degree of the numerator is now below the degree of the denominator.
It's now first degree and this is second degree.
What you can't do is use cover-up to start out with here.
That will give you the wrong answer.
So that's the end for today, and see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now, to start out today we're going to finish up what we did last time.
Which has to do with partial fractions.
I told you how to do partial fractions in several special cases and everybody was trying to figure out what the general picture was.
But I'd like to lay that out.
I'll still only do it for an example.
But it will be somehow a bigger example so that you can see what the general pattern is.
Partial fractions, remember, is a method for breaking up so-called rational functions.
Which are ratios of polynomials.
And it shows you that you can always integrate them.
That's really the theme here.
And this is what's reassuring is that it always works.
That's really the bottom line.
And that's good because there are a lot of integrals that don't have formulas and these do.
It always works.
But, maybe with lots of help.
So maybe slowly.
Now, there's a little bit of bad news, and I have to be totally honest and tell you what all the bad news is.
Along with the good news.
The first step, which maybe I should be calling Step 0, I had a Step 1, 2 and 3 last time, is long division.
That's the step where you take your polynomial divided by your other polynomial, and you find the quotient plus some remainder.
And you do that by long division.
And the quotient is easy to take the antiderivative of up because it's just a polynomial.
And the key extra property here is that the degree of the numerator now over here, this remainder, is strictly less than the degree of the denominator.
So that you can do the next step.
Now, the next step which I called Step 1 last time, that's great imagination, it's right after Step 0, Step 1 was to factor the denominator.
And I'm going to illustrate by example what the setup is here.
I don't know maybe, we'll do this.
Some polynomial here, maybe cube this one.
So here I've factored the denominator.
That's what I called Step 1 last time.
Now, here's the first piece of bad news.
In reality, if somebody gave you a multiplied out degree, whatever polynomial here, you would be very hard pressed to factor it.
A lot of them are extremely difficult to factor.
And so that's something you would have to give to a machine to do.
And it's just basically a hard problem.
So obviously, we're only going to give you ones that you can do by hand.
So very low degree examples.
And that's just the way it is.
So this is really a hard step in disguise, in real life.
Anyway, we're just going to take it as given.
And we have this numerator, which is of degree less than the denominator.
So let's count up what its degree has to be.
This is 4 + 2 + 6.
So this is degree 4 + 2 + 6.
I added that up because this is degree 4, this is degree 2 and (x ^2) ^3 is the 6th power.
So all together it's this, which is 12.
And so this thing is of degree <= 11.
All the way up to degree 11, that's the possibilities for the numerator here.
Now, the extra information that I want to impart right now, is just this setup.
Which I called Step 2 last time.
And the setup is this.
Now, it's going to take us a while to do this.
We have this factor here.
We have another factor.
We have another term, with the square.
We have another term with the cube.
We have another term with the fourth power.
So this is what's going to happen whenever you have linear factors.
You'll have a collection of terms like this.
So you have four constants to take care of.
Now, with a quadratic in the denominator, you need a linear function in the numerator.
So that's, if you like, B0 x + C0 divided by this quadratic term here.
And what I didn't show you last time was how you deal with higher powers of quadratic terms.
So when you have a quadratic term, what's going to happen is you're going to take that first factor here.
Just the way you did in this case.
But then you're going to have to do the same thing with the next power.
Now notice, just as in the case of this top row, I have just a constant here.
And even though I increased the degree of the denominator I'm not increasing the numerator.
It's staying just a constant.
It's not linear up here.
It's better than that.
It's just a constant.
And here it stayed a constant.
And here I stayed a constant.
Similarly here, even though I'm increasing the degree of the denominator, I'm leaving the numerator, the form of the numerator, alone.
It's just a linear factor and a linear factor.
So that's the key to this pattern.
I don't have quite as jazzy a song on mine.
So this is so long that it runs off the blackboard here.
So let's continue it on the next.
We've got this B2 x + C2 - sorry, B3 x + C3 / (x ^2 + 4) ^3.
I guess I have room for it over here.
I'm going to talk about this in just a second.
Alright, so here's the pattern.
Now, let me just do a count of the number of unknowns we have here.
The number of unknowns that we have here is 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.
That 12 is no coincidence.
That's the degree of the polynomial.
And it's the number of unknowns that we have.
And it's the number of degrees of freedom in a polynomial of degree 11.
If you have all these free coefficients here, you have the coefficient x^ 0, x ^ 1, all the way up to x^ 11.
And 0 through 11 is 12 different coefficients.
And so this is a very complicated system of equations for unknowns.
This is twelve equations for twelve unknowns.
So I'll get rid of this for a second.
So we have twelve equations, twelve unknowns.
So that's the other bad news.
Machines handle this very well, but human beings have a little trouble with 12.
Now, the cover-up method works very neatly and picks out this term here.
But that's it.
So it reduces it to an 11 by 11.
You'll be able to evaluate this in no time.
But that's it.
That's the only simplification of your previous method.
We don't have a method for this.
So I'm just showing what the whole method looks like but really you'd have to have a machine to implement this once it gets to be any size at all.
Yeah, question
[INAUDIBLE]
It's one big equation, but it's a polynomial equation.
So there's an equation, there's this function r ( x) = a11 x^2 11 + a10 x^ 10... and these things are known.
This is a known expression here.
And then when you cross-multiply on the other side, what you have is, well, it's a1 times, if you cancel this denominator with that, you're going to get (x + 2) ^3 ( x ^2 + 2x + 3)( x^2 + 4)^3 + the term for a2, etc.
It's a monster equation.
And then to separate it out into separate equations, you take the coefficient on x^ 11th, x ^ 10, ... all the way down to x^ 0.
And all told, that means there are a total of 12 equations.
11 through 0 is 12 equations.
Yeah, another question
[INAUDIBLE]
Should I write down rest of this?
[INAUDIBLE]
Should you write down all this stuff?
Well, that's a good question.
So you notice I didn't write it down.
Why didn't I write it down?
Because it's incredibly long.
In fact, you probably, so how many pages of writing would this take?
This is about a page of writing.
So just think of your machine, how much time you want to spend on this.
So the answer is that you have to be realistic.
You're a human being, not a machine.
And so there's certain things that you can write down and other things you should attempt to write down.
So do not do this at home.
So that's the first down-side to this method.
It gets more and more complicated as time goes on.
The second down-side, I want to point out to you, is what's happening with the pieces.
So the pieces still need to be integrated.
We simplified this problem, but we didn't get rid of it.
We still have the problem of integrating the pieces.
Now, some of the pieces are very easy.
This top row here, the antiderivatives of these, you can just write down.
By advanced guessing.
I'm going to skip over to the most complicated one over here.
For one second here.
And what is it that you'd have to deal with for that one.
You'd have to deal with, for example, so e.g., for example, I need to deal with the this guy.
I've got to get this antiderivative here.
Now, this one you're supposed to be able to know.
So this is why I'm mentioning this.
Because this kind of ingredient is something you already covered.
And what is it?
Well, you do this one by advanced guessing, although you it as the method of substitution.
You realize that it's going to be of the form (x^2 + 4) ^ - 2, roughly speaking.
And now we're going to fix that.
Because if you differentiate it you get 2x (-2), that's - 4 (x)(x) times this.
There's an x in the numerator here.
So it's - 1/4 of that will fix the factor.
And here's the answer for that one.
So that's one you can do.
The second piece is this guy.
This is the other piece.
Now, this was the piece that came from B3.
This is the one that came from B3.
And this is the one that's coming from C3.
This is coming from C3.
We still need to get this one out there.
So C3 times that will be the correct answer, once we've found these numbers.
So how do we do this?
How's this one integrated?
Trig substitution?
Trig substitution.
So the trig substitution here is x = 2 tan u.
Or 2 tan theta.
And when you do that, there are a couple of simplifications.
Well, I wouldn't call this a simplification.
This is just the differentiation formula.
dx = 2 sec^2 u du.
And then you have to plug in, and you're using the fact that when you plug in the tan^2, 4 tan ^2 + 4, you'll get a sec ^2.
So altogether, this thing is, 2 sec^2 u du.
And then there's a (4 sec^2 u) ^3, in the denominator.
So that's what happens when you change variables here.
And now look, this keeps on going.
This is not the end of the problem.
Because what does that simplify to?
That is, let's see, it's 2/64, the integral of sec^6 and sec^2.
That's the same as cos^4.
And now, you did a trig substitution but you still have a trig integral.
The trig integral now, there's a method for this.
The method for this is when it's an even power, you have to use the double angle formula.
So that's this guy here.
And you're still not done.
You have to square this thing out.
And then you'll still get a cos^2 2u.
And it keeps on going.
So this thing goes on for a long time.
But I'm not even going to finish this, but I just want to show you.
The point is, we're not showing you how to do any complicated problem.
We're just showing you all the little ingredients.
And you have to string them together a long, long, long process to get to the final answer of one of these questions.
So it always works, but maybe slowly.
By the way, there's even another horrible thing that happens.
Which is, if you handle this guy here, what's the technique.
This is another technique that you learned, supposedly within the last few days.
Completing the square.
So this, it turns out, you have to rewrite it this way.
And then the evaluation is going to be expressed in terms of, I'm going to jump to the end.
It's going to turn out to be expressed in terms of this.
That's what will eventually show up in the formula.
And not only that, but if you deal with ones involving x as well, you'll also need to deal with something like ln of this denominator here.
So all of these things will be involved.
So now, the last message that I have for you is just this.
This thing is very complicated.
We're certainly never going to ask you to do it.
But you should just be aware that this level of complexity, we are absolutely stuck with in this problem.
And the reason why we're stuck with it is that this is what the formulas look like in the end.
If the answers look like this, the formulas have to be this complicated.
If you differentiate this, you get your polynomial, your ratio of polynomials.
If you differentiate this, you get some ratio of polynomials.
These are the things that come up when you take antiderivatives of those rational functions.
So we're just stuck with these guys.
And so don't let it get to you too much.
I mean, it's not so bad.
In fact, there are computer programs that will do this for you anytime you want.
And you just have to be not intimidated by them.
They're like other functions.
OK, that's it for the general comments on partial fractions.
Now we're going to change subjects to our last technique.
This is one more technical thing to get you familiar with functions.
And this technique is called integration by parts.
Please, just because its name sort of sounds like partial fractions, don't think it's the same thing.
It's totally different.
It's not the same.
So this one is called integration by parts.
Now, unlike the previous case, where I couldn't actually justify to you that the linear algebra always works.
I claimed it worked, but I wasn't able to prove it.
That's a complicated theorem which I'm not able to do in this class.
Here I can explain to you what's going on with integration by parts.
It's just the fundamental theorem of calculus, if you like, coupled with the product formula.
Sort of unwound and read in reverse.
And here's how that works.
If you take the product of two functions and you differentiate them, then we know that the product rule says that this is u' v + uv'.
And now I'm just going to rearrange in the following way.
I'm going to solve for uv'.
That is, this term here.
So what is this term?
It's this other term, (uv)'.
Minus the other piece.
So I just rewrote this equation.
And now I'm going to integrate it.
So here's the formula.
The integral of the left-hand side is equal to the integral of the right-hand side.
Well when I integrate a derivative, of I get back the function itself.
That's the fundamental theorem.
So this it.
Sorry, I missed the dx, which is important.
I apologize.
Let's put that in there.
So this is the integration by parts formula.
I'm going to write it one more time with the limits stuck in.
It's also written this way, when you have a definite integral.
Just the same formula, written twice.
Alright, now I'm going to show you how it works on a few examples.
And I have to give you a flavor for how it works.
But it'll grow as we get more and more experience.
The first example that I'm going to take is one that looks intractable on the face of it.
Which is the integral of ln x dx.
Now, it looks like there's sort of nothing we can do with this.
And we don't know what the solution is.
However, I claim that if we fit it into this form, we can figure out what the integral is relatively easily.
By some little magic of cancellation it happens.
The idea is the following.
If I consider this function to be u, then what's going to appear on the other side in the integrated form is the function u', which is -- so, if you like, u = ln x.
So u' = 1 / x.
Now, 1 / x is a more manageable function than ln x.
What we're using is that when we differentiate the function, it's getting nicer.
It's getting more tractable for us.
In order for this to fit into this pattern, however, I need a function v. So what in the world am I going to put here for v?
The answer is, well, dx is almost the right answer.
The answer turns out to be x.
And the reason is that that makes v' = 1.
It makes v' = 1.
So that means that this is u, but it's also uv'.
Which was what I had on the left-hand side.
So it's both u and uv'.
So this is the setup.
And now all I'm going to do is read off what the formula says.
What it says is, this is equal to u v. So u is this and v is that.
So it's x ln x, minus, so that again, this is uv.
Except in the other order, vu.
And then I'm integrating, and what do I have to integrate?
u ' v. So look up there.
u' v with a minus sign here.
u' = 1 / x, and v = x.
So it's 1 / x, that's u'.
And here is x, that's v, dx.
Now, that one is easy to integrate.
Because (1/x) x = 1.
And the integral of 1 dx is x + c, if you like.
So the antiderivative of 1 is x.
And so here's our answer.
Our answer is that this is x ln x - x + c. I'm going to do two more slightly more complicated examples.
And then really, the main thing is to get yourself used to this method.
And there's no one way of doing that.
Just practice makes perfect.
And so we'll just do a few more examples.
And illustrate them.
The second example that I'm going to use is the integral of (ln x) ^2 dx.
And this is just slightly more recalcitrant.
Namely, I'm going to let u be (ln x)^2.
And again, v = to x.
So that matches up here.
That is, v' = 1.
So this is u v'.
So this thing is u v'.
And then we'll just see what happens.
Now, the game that we get is that when I differentiate the logarithm squared, I'm going to to get something simpler.
It's not going to win us the whole battle, but it will get us started.
So here we get u'.
And that's 2 ln x ( 1 / x).
Applying the chain rule.
And so the formula is that this is x (ln x)^2, minus the integral of, well it's u' v, right, that's what I have to put over here.
So u' = 2 ln x ( 1 / x), and v = x.
And so now, you notice something interesting happening here.
So let me just demarcate this a little bit.
And let you see what it is that I'm doing here.
So notice, this is the same integral.
So here we have x (ln x) ^2.
We've already solve that part.
But now know notice that the 1 / x and the x cancel.
So we're back to the previous case.
We didn't win all the way, but actually we reduced ourselves to this integral.
To the integral of ln x, which we already know.
So here, I can copy that down.
That's - 2 (x ln x - x), and then I have to throw in a constant, c. And that's the end of the problem here.
That's it.
So this piece, I got from Example 1.
Now, this illustrates a principle which is a little bit more complicated than just the one of integration by parts.
Which is a sort of a general principle which I'll call my Example 3, which is something which is called a reduction formula.
A reduction formula is a case where we apply some rule and we figure out one of these integrals in terms of something else.
Which is a little bit simpler.
And eventually we'll get down to the end, but it may take us n steps from the beginning.
So the example is l(n x^ n) dx.
.
And the claim is that if I do what I did in Example 2, to this case, I'll get a simpler one which will involve the n - 1st power.
And that way I can get all the way back down to the final answer.
So here's what happens.
We take u as ln x^ n. This is the same discussion as before, v = x.
And then u' is n l(n x) ^ n - 1( 1 / x).
And v' is 1.
And so the setup is similar.
We have here x ( ln x)^ n minus the integral.
And there's n times, it turns out to be (ln x)^ n - 1.
And then there's a 1 / x and an x, which cancel.
So I'm going to explain this also abstractly a little bit just to show you what's happening here.
If you use the notation Fn (x) is the integral of (ln x)^n dx, and we're going to forget the constant here.
Then the relationship that we have here is that Fn (x) = n ln, I'm sorry, x (ln x)^ n. That's the first term over here.
Minus n times the preceding one.
This one here.
And the idea is that eventually we can get down.
If we start with the nth one, we have a formula that includes, so the reduction is to the n - 1st.
Then we can reduce to the n - 2nd, and so on.
Until we reduce to the 1, the first 1.
And then in fact we can even go down to the 0th one.
So this is the idea of a reduction formula.
And let me illustrate it exactly in the context of Examples 1 and 2.
So the first step would be to evaluate the first one.
Which is, if you like, (ln x)^ 0 dx.
That's very easy, that's x.
And then F1 ( x) = x ln x - F0 (x).
Now, that's applying this rule.
So let me just put it in a box here.
This is the method of induction.
Here's the rule.
And I'm applying it for n = 1.
I plugged in n = 1 here.
So here, I have x ln x ^ 1 - 1 ( F0 ( x).
And that's what I put right here, on the right-hand side.
And that's going to generate for me the formula that I want, which is x ln x - x.
That's the answer to this problem over here.
This was Example 1.
Notice I dropped the constants because I can add them in at the end.
So I'll put in parentheses here, (+ c).
That's what would happen at the end of the problem.
The next step, so that was Example 1, and now Example 2 works more or less the same way.
I'm just summarizing what I did on that blackboard right up here.
The same thing, but in much more compact notation.
If I take F2 ( x), that's going to be equal to x (ln x)^2 - 2 F1 ( x).
Again, this is box for n = 2.
And if I plug it in, what I'm getting here is x (ln x) ^2 - twice this stuff here.
Which is right here.
x ln x - x.
If you like, + c. So I'll leave the c off.
So this is how reduction formulas work in general.
I'm going to give you one more example of a reduction formula.
So I guess we have to call this Example 4.
Let's be fancy, let's make it the sine.
No no, no, let's be fancier still.
Let's make it e^ x.
So this would also work for cosine x and sine x.
The same sort of thing.
And I should mention that on your homework, you have to do it for cosine of x. I decided to change my mind on the spur of the moment.
I'm not going to do it for cosine because you have to work it out on your homework for cosine.
In a later homework you'll even do this case.
So it's fine.
You need the practice.
OK, so how am I going to do it this time.
This is again, a reduction formula.
And the trick here is to pick u to be this function here.
And the reason is the following.
So it's very important to pick which function is the u and which function is the v. That's the only decision you have to make if you're going to apply integration by parts.
When I pick this function as the u, the advantage that I have is that u' is simpler.
How is it simpler?
It's simpler because it's one degree down.
So that's making progress for us.
On the other hand, this function here is going to be what I'll use for v. And if I differentiated that, if I did it the other way around and I differentiated that, I would just get the same level of complexity.
Differentiating e^x just gives you back e ^ x.
So that's boring.
It doesn't make any progress in this process.
And so I'm going to instead let v = e ^ x and, sorry this is v '.
Make it v ' = e ^ x.
And then v = e ^x.
At least it isn't any worse when I went backwards like that.
So now, I have u and v ', and now I get (x ^ n)( e ^ x).
This again is u, and this is v. So it happens that v = t v ' so it's a little confusing here.
But this is the one we're calling v '.
And here's v. And now minus the integral and I have here nx ^ n - 1.
And I have here e ^ x.
So this is u ' and this is v dx.
So this recurrence is a new recurrence.
And let me summarize it here.
It's saying that Gn ( x) should be the integral of (x ^ n)( e ^ x) dx.
Again, I'm dropping the c. And then the reduction formula is that Gn (x) = this expression here, (x ^ n)( e ^ x) - n Gn - 1 (x).
So here's our reduction formula.
And to illustrate this, if I take G0 (x), if you think about it for a second that's just, there's nothing here.
The antiderivative of e ^ x, that's going to be e ^ x, that getting started at the real basement here.
Again, as always, 0 is my favorite number.
Not 1.
I always start with the easiest one, if possible.
And now G1, applying this formula, is going to be equal to x e ^ x - G0 ( x).
Which is just right, because n is 1 and n - 1 is 0.
And so that's just x e ^ x - e^ x.
So this is a very, very fancy way of saying the following fact.
I'll put it over on this other board.
Which is that the integral of x e^ x dx = x e^ x - x + c. Yeah, question
[INAUDIBLE]
The question is, why is this true.
Why is this statement true.
Why is G 0 = e^ x. I did that in my head.
What I did was, I first wrote down the formula for G0.
Which was G0 is equal to the integral of e^ x dx.
Because there's an x to the 0 power in there, which is just 1.
And then I know the antiderivative of e ^ x.
It's e ^x
[INAUDIBLE]
How do you know when this method will work?
The answer is only by experience.
You must get practice doing this.
If you look in your textbook, you'll see hints as to what to do.
The other hint that I want to say is that if you find that you have one factor in your expression which when you differentiate it, it gets easier.
And when you antidifferentiate the other half, it doesn't get any worse, then that's when this method has a chance of helping.
And there is there's no general thing.
The thing is, though, if you it with x^ n (e^ x), x ^ n cosine x, especially on sine x, those are examples where it works.
This power of the ln.
I'll give you er one more example here.
So this was G1 ( x), right.
I'll give you one more example in a second.
Yeah
[INAUDIBLE]
Thank you.
There's a mistake here.
That's bad.
I was thinking in the back of my head of the following formula.
Which is another one which we've just done.
So these are the types of formulas that you can get out of integration by parts.
There's also another way of getting these, which I'm not going to say anything about.
Which is called advance guessing.
You guess in advance what the form is, you differentiate it and you check.
That does work too, with many of these cases.
I want to give you an illustration.
Just because you these formulas are somewhat dry.
So I want to give you just at least one application.
We're almost done with the idea of these formulas.
And we're going to get back now to being able to handle lots more integrals than we could before.
And what's satisfying is that now we can get numbers out instead of being stuck and hamstrung with only a few techniques.
Now we have all of the techniques of integration that anybody has.
And so we can do pretty much anything we want that's possible to do.
So here's, if you like, an application that illustrates how integration by parts can be helpful.
And we're going to find the volume of an exponential wine glass here.
Again, don't try this at home, but.
So let's see.
It's going to be this beautiful guy here.
I think.
OK, so what's it going to be.
This graph is going to be y = e^ x.
Then we're going to rotate it around the y axis.
And this level here is the height y = 1.
And the top, let's say, is y = e. So that the horizontal here, coming down, is x = 1.
Now, there are two ways to set up this problem.
And so there are two methods.
And this is also a good review because, of course, we did this in the last unit.
The two methods are horizontal and vertical slices.
Those are the two ways we can do this.
Now, if we do it with, so let's start out with the horizontal ones.
That's this shape here.
And we're going like that.
And the horizontal slices mean that this little bit here is a thickness dy.
And then we're going to wrap that around.
So this is going to become a disk.
This is the method of disks.
And what's this distance here?
Well, this place is x.
And so the disk has area pi x ^2.
And we're going to add up the thickness of the disks and we're going to integrate from 1 to e. So here's our volume.
And now we have one last little item of business before we can evaluate this integral.
And that is that we need to know the relationship here on the curve, that y = e ^ x.
So that means x = ln y.
And in order to evaluate this integral, we have to evaluate x correctly as a function of y.
So that's the integral from 1 to e of (ln y)^2, times pi, dy.
So now you see that this is an integral that we did calculate already.
And in fact, it's sitting right here.
Except with the variable x instead of the variable y.
So the answer, which we already had, is this F2 ( y) here.
So maybe I'll write it that way.
So this is F2 (y) between 1 and e. And now let's figure out what it is.
It's written over there.
It's y (ln y) ^2 - 2(y ln y - y).
The whole thing evaluated at 1e.
And that is, if I plug in e here, I get e. Except there's a factor of pi there, sorry.
Missed the pi factor.
So there's an e here.
And then I subtract off, well, at 1 this is e - e. So it cancels.
There's nothing left.
And then at 1, I get ln 1 is 0, ln 1 is 0, there's only one term left, which is 2.
So it's - 2.
That's the answer.
Now we get to compare that with what happens if we do it the other way.
So what's the vertical?
So by vertical slicing, we get shells.
And that starts, that's in the x variable.
It starts at 0 and ends at 1 and it's dx.
And what are the shells?
Well, the shells are, if I can draw the picture again, they start, the top value is e. And the bottom value is, I need a little bit of room for this.
The bottom value is y.
And then we have 2 pi x is the circumference, as we sweep it around dx.
So here's our new volume.
Expressed in this different way.
So now I'm going to plug in what this is.
It's the integral from 0 to 1 of e - e ^ x.
That's the formula for y.
2 pi x dx.
And what you see is that you get the integral from 0 to 1 of 2 pi e x dx.
That's easy, right?
That's just 2 pi e ( 1/2).
This one is just the area of a triangle.
If I factor out the 2 pi e. And then the other piece is the integral of 2 pi x e^ x dx.
From 0 to 1
[INAUDIBLE]
Are you asking me whether I need an x ^2 here?
I just evaluated the integral.
I just did it geometrically.
I said, this is the area of a triangle.
I didn't antidifferentiate and evaluate it, I just told you the number.
Because it's a definite integral.
So now, this one here, I can read off from right up here.
Above it, this is G1.
So this is equal to, let's check it out here.
So this is pi e, right, - 2 pi G1 ( x), evaluated at 0 and 1.
So let's make sure that it's the same as what we had before.
It's pi e - 2 pi times here's g1.
So it's x e ^ x - e^ x.
So at x = 1, that cancels.
But at the bottom end, it's e^ 0.
So it's - 1 here.
Is that right?
Yep.
So it's pi e - 2.
It's the same.
Question
[INAUDIBLE]
From here to here, is that the question?
[INAUDIBLE]
So the step here is just the distributive law.
This is e 2 pi x, that's this term.
And the other terms, the minus sign is outside.
The 2 pi I factored out.
And the x and the e ^x stayed inside the integral sign.
Thank you.
The correction is that there was a missing minus sign, last time.
When I integrated from 0 to 1 x e^ x dx, I had a x e^ x - e^x.
Evaluated at 0 and 1.
And that's equal to + 1.
I was missing this minus sign.
The place where it came in was in this wineglass example.
We had the integral of 2 pi x e - e ^x dx.
And that was 2 pi e integral of x dx, from 0 to 1, - 2 pi, integral from 0 to 1 of x e^x dx.
And then I worked this out and it was pi e. And then this one was - 2 pi, and what I wrote down was - 1.
But there should have been an extra minus sign there.
So it's this.
The final answer was correct, but this minus sign was missing.
Right there.
So just, right there.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Now, today we get to move on from integral formulas and methods of integration back to some geometry.
And this is more or less going to lead into the kinds of tools you'll be using in multivariable calculus.
The first thing that we're going to do today is discuss arc length.
Like all of the cumulative sums that we've worked on, this one has a storyline and the picture associated to it, which involves dividing things up.
If you have a roadway, if you like, and you have mileage markers along the road, like this, all the way up to, say, sn here, then the length along the road is described by this parameter, s. Which is arc length.
And if we look at a graph of this sort of thing, if this is the last point b, and this is the first point a, then you can think in terms of having points above x1, x2, x3, etc.
The same as we did with Riemann sums.
And then the way that we're going to approximate this is by taking the straight lines between each of these points.
As things get smaller and smaller, the straight line is going to be fairly close to the curve.
And that's the main idea.
So let me just depict one little chunk of this.
Which is like this.
One straight line, and here's the curved surface there.
And the distance along the curved surface is what I'm calling delta s, the change in the length between, so this would be s2 - s1 if I depicted that one.
So this would be delta s is, say s. si - si - 1, some increment there.
And then I can figure out what the length of the orange segment is.
Because the horizontal distance is delta x.
And the vertical distance is delta y.
And so the formula is that the hypotenuse is delta x ^2 + delta y ^2.
Square root.
And delta s is approximately that.
So what we're saying is that delta s ^2 is approximately this.
So this is the hypotenuse.
Squared.
And it's very close to the length of the curve.
And the whole idea of calculus is in the infinitesimal, this is exactly correct.
So that's what's going to happen in the limit.
And that is the basis for calculating arc length.
I'm going to rewrite that formula on the next board.
But I'm going to write it in the more customary fashion.
We've done this before, a certain amount.
But I just want to emphasize it here because this handwriting is a little bit peculiar.
This ds is really all one thing.
What I really mean is to put the parenthesis around it.
It's one thing.
It's not d * s, it's ds.
It's one thing.
And we square it.
But for whatever reason people have gotten into the habit of omitting the parentheses.
So you're just going to have to live with that.
And realize that this is not d of s ^2 or anything like that.
And similarly, this is a single number, and this is a single number.
Infinitesimal.
So that's just the way that this idea here gets written in our notation.
And this is the first time we're dealing with squares of infinitesimals.
So it's just a little different.
But immediately the first thing we're going to do is take the square root.
If I take the square root, that's the square root of dx ^2 + dy ^2.
And this is the form in which I always remember this formula.
Let's put it in some brightly decorated form.
But there are about 4, 5, 6 other forms that you'll derive from this, which all mean the same thing.
So this is, as I say, the way I remember it.
But there are other ways of thinking of it.
And let's just write a couple of them down.
The first one is that you can factor out the dx.
So that looks like this.
1 + (dy / dx)^2.
And then I factored out the dx.
So this is a variant.
And this is the one which actually we'll be using in practice right now on our examples.
So the conclusion is that the arc length, which if you like is this total sn - s0, if you like, is going to be equal to the integral from a to b of the square root of 1 + (dy / dx)^2 dx.
In practice, it's also very often written informally as this.
The integral ds.
So the change in this little variable s, and this is what you'll see notationally in many textbooks.
So that's one way of writing it, and of course the second way of writing it which is practically the same thing is square root of 1 + f ' ( x^2) dx.
Mixing in a little bit of Newton's notation.
And this is with y = f (x).
So this is the formula for arc length.
And as I say, I remember it this way.
But you're going to have to derive various variants of it.
And you'll have to use some arithmetic to get to various formulas.
And there will be more later.
Yeah, question
[INAUDIBLE]
OK, the question is, is f ' (x)^2 = f '' (x).
And the answer is no.
And let's just see what it is.
So, for example, if f ( x) = x ^2, which is an example which will come up in a few minutes, then f ' (x) = 2x and f ' ( x )^2 = = (2x), which is 4x ^2.
Whereas f '' ( x) is equal to, if I differentiate this another time, it's equal to 2.
So they don't mean the same thing.
The same thing over here.
You can see this dy / dx, this is the rate of change of y with respect to x.
The quantity squared.
So in other words, this thing is supposed to mean the same as that.
Yeah.
Another question
[INAUDIBLE]
So the question is, you got a little nervous because I left out these limits.
And indeed, I did that on purpose because I didn't want to specify what was going on.
Really, if you wrote it in terms of ds, you'd have to write it as starting at s0 and ending at sn to be consistent with the variable s. But of course, if you write it in terms of another variable, you put that variable in.
So this is what we do when we change variables, right?
We have many different choices for these limits.
And this is the clue as to which variable we use
[INAUDIBLE]
Correct.
s0 and sn are not the same thing as a and b.
In fact, this is xn.
And this x0, over here.
That's what a and b are.
But s0 and sn are mileage markers on the road.
They're not the same thing as keeping track of what's happening on the x axis.
So when we measure arc length, remember it's mileage along the curved path.
So now, I need to give you some examples.
And my first example is going to be really basic.
But I hope that it helps to give some perspective here.
So I'm going to take the example y = m x, which is a linear function, a straight line.
And then y ' would = m, and so ds is going to be the square root of 1 + (y ') ^2 dx.
Which is the square root of 1 + m ^2 dx.
And now, the length, say, if we go from, I don't know, let's say 0 to 10, let's say.
Of the graph is going to be the integral from 0 to 10 of the square root of 1 + m ^2 dx.
Which of course is just 10 square root of 1 + m ^2.
Not very surprising.
This is a constant.
It just factors out and the integral from 0 to 10 of dx is 10.
Let's just draw a picture of this.
This is something which has slope m here.
And it's going to 10.
So this horizontal is 10.
And the vertical is 10 m. Those are the dimensions of this.
And the Pythagorean theorem says that the hypotenuse, not surprisingly, let's draw it in here in orange to remind ourselves that it was the same type of orange that we had over there, this length here is the square root of 10 ^2 + (10m)^2.
That's the formula for the hypotenuse.
And that's exactly the same as this.
Maybe you're saying duh, this is obvious.
But the point that I'm trying to make is this.
If you can figure out these formulas for linear functions, calculus tells you how to do it for every function.
The idea of calculus is that this easy calculation here, which you can do without any calculus at all, all of the tools, the notations of differentials and limits and integrals, is going to make you be able to do it for any curve.
Because we can break things up into these little infinitesimal bits.
This is the whole idea of all of the methods that we had to set up integrals here.
This is the main point of these integrals.
Now, so let's do something slightly more interesting.
Our next example is going to be the circle, so y = square root of 1 - x ^2.
If you like, that's the graph of a semicircle.
And maybe we'll set it up here this way.
So that the semicircle goes around like this.
And well start it here at x = 0.
And we'll go over to a.
And we'll take this little piece of the circle.
So down to here.
If you like.
So here's the portion of the circle that I'm going to measure the length of.
Now, we know that length.
It's called arc length.
And I'm going to give it a name, I'm going to call it alpha here.
So alpha's the arc length along the circle.
Now, let's figure out what it is.
First, in order to do this, I have to figure out what y ' is.
Or, if you like, dy / dx.
Now, that's a calculation that we've done a number of times.
And I'm going to do it slightly faster.
But you remember it gives you a square root in the denominator.
And then you have the derivative of what's inside the square root.
Which is - 2x.
But then there's also 1/2, because in disguise it's really (1 - x ^2)^2 1/2.
So we've done this calculation enough times that I'm not going to carry it out completely.
I want you to think about what it is.
It turns out to - x up here, because the 1/2 and the 2 cancel.
And now the thing that we have to integrate is this arc length element, as it's called.
ds.
And that's going to be the square root of 1 + (y ') ^2 dx.
And so I'm going to have to carry out the calculation, some messy calculation here.
Which is that this is 1 + ( - x / square root of 1 - x ^2) ^2.
So I have to figure out what's under the square root sign over here in order to carry out this calculation.
Now let's do that.
This is 1 + x ^2 / 1 - x ^2.
That's what this simplifies to.
And then that's equal to, over a common denominator, (1 - x ^2).
1 - x ^2 + x^2.
And there is a little bit of simplification now.
Because the 2x ^2's cancel.
And we get 1 / 1 - x^2.
So now I get to finish off the calculation by actually figuring out what the arc length is.
And what is it?
Well, this alpha is equal to the integral from 0 to a of ds.
Well, it's going to be the square root of what I have here.
This was a square, this is just what was underneath the square root sign.
This is 1 + (y ') ^2.
Have to take the square root of that.
So what I get here is dx / the square root of 1 - x ^2.
And now, we recognize this.
The antiderivative of this is something that we know.
This is the inverse sine.
Evaluated at 0 and a.
Which is just giving us the inverse sine of a, because the inverse sine of 0 = 0.
So alpha = the inverse sine of a.
That's a very fancy way of saying that sine alpha = a.
That's the equivalent statement here.
And what's going on here is something that's just a little deeper than it looks.
Which is this.
We've just figured out a geometric interpretation of what's going on here.
That is, that we went a distance alpha along this arc.
And now remember that the radius here is 1.
And this horizontal distance here is a.
This distance here is a.
And so the geometric interpretation of this is that this angle is alpha radians.
And sine alpha = a.
So this is consistent with our definition previously, our previous geometric definition of radians.
But this is really your first true definition of radians.
We never actually, people told you that radians were the arc length along this curve.
This is the first time you're deriving it.
This is the first time you're seeing it correctly done.
And furthermore, this is the first time you're seeing a correct definition of the sine function.
Remember we had this crazy way, we we defined the exponential function, then we had another way of defining the ln function as an integral.
Then we defined the exponential in terms of it.
Well, this is the same sort of thing.
What's really happening here is that if you want to know what radians are, you have to calculate this number.
If you've calculated this number then by definition if sine is the thing whose alpha radian amount gives you a, then it must be that this is sine inverse of a.
And so the first thing that gets defined is the arc sine.
And the next thing that gets defined is the sine afterwards.
This is the way the foundational approach actually works when you start from first principles.
This arc length being one of the first principles.
So now we have a solid foundation for trig functions.
And this is giving that connection.
Of course, it's consistent with everything you already knew, so you don't have to make any transitional thinking here.
It's just that this is the first time it's being done rigorously.
Because you only now have arc length.
So these are examples, as I say, that maybe you already know.
And maybe we'll do one that we don't know quite as well.
Let's find the length of a parabola.
This is Example 3.
Now, that was what I was suggesting we were going to do earlier.
So this is the function y = x ^2.
y ' = 2x.
And so ds = the square root of 1 + (2x) ^2 dx.
And now I can figure out what a piece of a parabola is.
So I'll draw the piece of parabola up to a, let's say, starting from 0.
So that's the chunk.
And then its arc length, between 0 and a of this curve, is the integral from 0 to a of square root of 1 + 4x ^2 dx.
OK, now if you like, this is the answer to the question.
But people hate looking at answers when they're integrals if they can be evaluated.
So one of the reasons why we went through all this rigamarole of calculating these things is to show you that we can actually evaluate a bunch of these functions here more explicitly.
It doesn't help a lot, but there is an explicit calculation of this.
So remember how you would do this.
So this is just a little bit of review.
What we did in techniques of integration.
The first step is what?
A substitution.
It's a trig substitution.
And what is it?
[INAUDIBLE]
So x equals something tan theta.
I claim that it's 1/2 tan, and I'm going to call it u.
Because I'm going to use theta for something else in a couple of days.
OK?
So this is the substitution.
And then of course dx = 1/2 sec ^2 u du , etc.
So what happens if you do this?
I'll write down the answer, but I'm not going to carry this out.
Because every one of these is horrendous.
But I think I worked it out.
Let's see if I'm lucky.
Oh yeah.
I think this is what it is.
It's a 1/4 ln 2x + square root of 1 + 4x ^2 + 1/2 x ( square root of 1 + 4x ^2).
Evaluated at a and 0.
So yick.
I mean, you know
[INAUDIBLE]
Why I did I make it 1/2?
Because it turns out that when you differentiate.
So the question is, why there 1/2 there?
If you differentiate it without the 1/2, you get this term and it looks like it's going to be just right.
But then if you differentiate this one you get another thing.
And it all mixes together.
And it turns out that there's more.
So it turns out that it's 1/2.
Differentiate it and check.
So this just an incredibly long calculation.
It would take fifteen minutes or something like that.
But the point is, you do know in principle how to do these things
[INAUDIBLE]
Oh, he was talking about this 1/2, not this crazy 1/2 here.
Sorry
[INAUDIBLE]
Yeah, OK.
So sorry about that.
Thank you for helping.
This factor of 1/2 here comes about because when you square x, you don't get tan ^2.
When you square 2x, you get 4x ^2 and that matches perfectly with this thing.
And that's why you need this factor here.
Yeah.
Another question, way in the back
[INAUDIBLE]
The question is, when you do this substitution, doesn't the limit from 0 to a change.
And the answer is, absolutely yes.
The limits in terms of u are not the same as the limits in terms of a.
But if I then translate back to the x variables, which I've done here in this bottom formula, of x = 0 and x = a, it goes back to those in the original variables.
So if I write things in the original variables, I have the original limits.
If I use the u variables, I would have to change limits.
But I'm not carrying out the integration, because I don't want to.
So I brought it back to the x formula.
Other questions.
OK, so now we're ready to launch into three-space a little bit here.
We're going to talk about surface area.
You're going to be doing a lot with surface area in multivariable calculus.
It's one of the really fun things.
And just remember, when it gets complicated, that the simplest things are the most important.
And the simple things are, if you can handle things for linear functions, you know all the rest.
So there's going to be some complicated stuff but it'll really only involve what's happening on planes.
So let's start with surface area.
And the example that I'd like to give, this is the only type of example that we'll have, is the surface of rotation.
And as long as we have our parabola there, we'll use that one.
So we have y = x ^2, rotated around the x axis.
So let's take a look at what this looks like.
It's the parabola, which is going like that.
And then it's being spun around the x axis.
So some kind of shape like this with little circles.
It's some kind of trumpet shape, right?
And that's the shape that we're.
Now, again, it's the surface.
It's just the metal of the trumpet, not the insides.
Now, the principle for figuring out what the formula for area is, is not that different from what we did for surfaces of revolution.
But it just requires a little bit of thought and imagination.
We have a little chunk of arc length along here.
And we're going to spin that around this axis.
Now, if this were a horizontal piece of arc length, then it would spin around just like a shell.
It would just be a surface.
But if it's tilted, if it's tilted, then there's more surface area proportional to the amount that it's tilted.
So it's proportional to the length of the segment that you spin around.
So the total is going to be ds, that's one of the factors here.
Maybe I'll write that second.
That's one of the dimensions.
And then the other dimension is the circumference.
Which is 2 pi, in this case y.
So that's the end of the calculation.
This is the area element of surface area.
Now, when you get to 18.02, and maybe even before that, you'll also see some people referring to this area element when it's a curvy surface like this with a notation d S. That's a little confusing because we have a lower case s here.
We're not going to use it right now.
But the lower case s is usually arc length.
The upper case s is usually surface area.
So.
Also used for dA.
The area element.
Because this is a curved area element.
So let's figure out this example.
So in the example, is equal to x ^2 then the situation is, we have the surface area is equal to the integral from, I don't know, 0 to a if those are the limits that we wanted to choose.
Of 2 pi x ^2, right?
Because y = x ^2 ( the square root of 1 + 4x ^2) dx.
Remember we had this from our previous example.
This was ds from previous.
And this, of course, is 2 pi y.
Now again, the calculation of this integral is kind of long.
And I'm going to omit it.
But let me just point out that it follows from the same substitution.
Namely, x = 1/2 tan u.
Is going to work for this integral.
It's kind of a mess.
There's a tan ^2 here and the sec ^2.
There's another secant and so on.
So it's one of these trig integrals that then takes a while to do.
So that just is going to trail off into nothing.
And the reason is that what's important here is more the method.
And the setup of the integrals.
The actual computation, in fact, you could go to a program and you could plug in something like this and you would spit out an answer immediately.
So really what we just want is for you to have enough control to see that it's an integral that's a manageable one.
And also to know that if you plugged it in, you would get an answer.
When I actually do carry out a calculation, though, what I want to do is to do something that has an answer that you can remember.
And that's a nice answer.
So that turns out to be the example of the surface area of a sphere.
So it's analogous to this 2 here.
And maybe I should remember this result here.
Which was that the arc length element was given by this.
So we'll save that for a second.
So we're going to do this surface area now.
So if you like, this is another example.
The surface area of a sphere.
This is a good example, and one, as I say, that has a really nice answer.
So it's worth doing.
So first of all, I'm not going to set it up quite the way I did in Example 2.
Instead, I'm going to take the general sphere, because I'd like to watch the dependence on the radius.
So here this is going to be the radius.
It's going to be radius a.
And now, if I carry out the same calculations as before, if you think about it for a second, you're going to get this result.
And then, the rest of the arithmetic, which is sitting up there in the case, a = 1, will give us that ds = what?
Well, maybe I'll just carry it out.
Because that's always nice.
So we have 1 + x ^2 / a ^2 - x ^2.
That's 1 + (y ') ^2.
And now I put this over a common denominator.
And I get a ^2 - x ^2.
And I have in the numerator a ^2 - x ^2 + x^2.
So the same cancellation occurs.
But now we get an a ^2 in the numerator.
So now I can set up the ds.
And so here's what happens.
The area of a section of the sphere, so let's see.
We're going to start at some starting place x1, and end at some place x2.
So what does that look like?
Here's the sphere.
And we're starting at a place x1.
And we're ending at a place x2.
And we're taking more or less the slice here, if you like.
The section of this sphere.
So the area's going to equal this.
And what is it going to be?
Well, so I have here 2 pi y. I'll write it out, just leave it as y for now.
And then I have ds.
So that's always what the formula is when you're revolving around the x axis.
And then I'll plug in for those things.
So 2 pi, the formula for y is square root a ^2 - x ^2.
And the formula for ds, well, it's the square root of this times dx.
So it's the square root of a ^2 / a ^2 - x ^2 dx.
So this part is ds.
And this part is y.
And now, I claim we have a nice cancellation that takes place.
Square root of a ^2 = a.
And then there's another good cancellation.
As you can see.
Now, what we get here is the integral from x1 to x2, of 2 pi a dx, which is about the easiest integral you can imagine.
It's just the integral of a constant.
So it's 2 pi a ( x2 - x1).
Let's check this in a couple of examples.
And then see what it's saying geometrically, a little bit.
So what this is saying, so special cases that you should always check when you have a nice formula like this, at least.
But really with anything in order to make sure that you've got the right answer.
If you take, for example, the hemisphere.
So you take 1/2 of this sphere.
So that would be starting at 0, sorry.
And ending at a.
So that's the integral from 0 to a.
So this is the case x1 = 0. x2 = a.
And what you're going to get is a hemisphere.
And the area is (2 pi a ) a.
Or in other words, 2 pi a ^2.
And if you take the whole sphere, that's starting at x1 = - a, and x2 = a, you're getting (2 pi a) ( a - (- a)).
Which is 4 pi a ^2.
That's the whole thing.
Yeah, question
[INAUDIBLE]
The question is, would it be possible to rotate around the y axis?
And the answer is yes.
It's legal to rotate around the y axis.
And there is, if you use vertical slices as we did here, that is, well they're sort of tips of slices, it's a different idea.
But anyway, it's using dx as the integral of the variable of integration.
So we're checking each little piece, each little strip of that type.
If we use dx here, we get this.
If you did the same thing rotated the other way, and use dy as the variable, you get exactly the same answer.
And it would be the same calculation.
Because they're parallel.
So you're, yep
[INAUDIBLE]
Can you do service area with shells?
Well, ah shell shape.
The short answer is not quite.
The shell shape is a vertical shell which is itself already three-dimensional, and it has a thickness.
So this is just a matter of terminology, though.
This thickness is this dx, when we do this rotation here.
And then there are two other dimensions.
If we have a curved surface, there's no other dimension left to form a shell.
But basically, you can chop things up into any bits that you can actually measure.
That you can figure out what the area is.
That's the main point.
Now, I said we were going to, we've just launched into three-dimensional space.
And I want to now move on to other space-like phenomena.
But we're going to do this.
So this is also a preparation for 18.02, where you'll be doing this a tremendous amount.
We're going to talk now about parametric equations.
Really just parametric curves.
So you're going to see this now and we're going to interpret it a couple of times, and we're going to think about polar coordinates.
These are all preparation for thinking in more variables, and thinking in a different way than you've been thinking before.
So I want you to prepare your brain to make a transition here.
This is the beginning of the transition to multivariable thinking.
We're going to consider curves like this.
Which are described with x being a function of t and y being a function of t. And this letter t is called the parameter.
In this case you should think of it, the easiest way to think of it is as time.
And what you have is what's called a trajectory.
So this is also called a trajectory.
And its location, let's say, at time 0, is this location here.
Of (0, y ( 0)), that's a point in the plane.
And then over here, for instance, maybe it's (x ( 1), y (1)).
And I drew arrows along here to indicate that we're going from this place over to that place.
These are later times.
t = 1 is a later time than t = 0.
So that's just a very casual, it's just the way we use these notations.
Now let me give you the first example, which is x = a cos t, y = a sin t. And the first thing to figure out is what kind of curve this is.
And to do that, we want to figure out what equation it satisfies in rectangular coordinates.
So to figure out what curve this is, we recognize that if we square and add.
So we add x ^2 to y^2, we're going to get something very nice and clean.
We're going to get a^2 cos ^2 t + a ^2 sin ^2 t. Yeah that's right, OK.
Which is just a ^2 (cos^2 + sin ^2), or in other words a^2.
So lo and behold, what we have is a circle.
And then we know what shape this is now.
And the other thing I'd like to keep track of is which direction we're going on the circle.
Because there's more to this parameter then just the shape.
There's also where we are at what time.
This would be, think of it like the trajectory of a planet.
So here, I have to do this by plotting the picture and figuring out what happens.
So at t = 0, we have (x, y) is equal to, plug in here (a cos 0, a sin 0).
Which is just a * 1 + a * 0, so a0.
And that's here.
That's the point (a, 0).
We know that it's the circle of radius a.
So we know that the curve is going to go around like this somehow.
So let's see what happens at t = pi / 2.
So at that point, we have (x,y) = ( a cos pi / 2, a sin pi / 2).
Which is (0, a), because sine of pi / 2 = 1.
So that's up here.
So this is what happens at t = 0.
This is what happens at t = pi / 2.
And the trajectory clearly goes this way.
In fact, this turns out to be t = pi, etc.
And it repeats at t = 2 pi.
So the other feature that we have, which is qualitative feature, is that it's counterclockwise.
No the last little bit is going to be the arc length.
Keeping track of the arc length.
And we'll do that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today we're going to continue our discussion of parametric curves.
I have to tell you about arc length.
And let me remind me where we left off last time.
This is parametric curves, continued.
Last time, we talked about the parametric representation for the circle.
Or one of the parametric representations for the circle.
Which was this one here.
And first we noted that this does parameterize, as we say, the circle.
That satisfies the equation for the circle.
And it's traced counterclockwise.
The picture looks like this.
Here's the circle.
And it starts out here at t = 0 and it gets up to here at time t = pi / 2.
So now I have to talk to you about arc length.
In this parametric form.
And the results should be the same as arc length around this circle ordinarily.
And we start out with this basic differential relationship.
dx ^2 is dx ^2 + dy ^2.
And then I'm going to take the square root, divide by dt, so the rate of change with respect to t of s is going to be the square root.
Well, maybe I'll write it without dividing.
Just write it as ds.
So this would be (dx / dt)^2 + (dy / dt)^2 dt.
So this is what you get formally from this equation.
If you take its square roots and you divide by dt squared in the inside, the square root and you multiply by dt outside.
So that those cancel.
And this is the formal connection between the two.
We'll be saying just a few more words in a few minutes about how to make sense of that rigorously.
Alright so that's the set of formulas for the infinitesimal, the differential of arc length.
And so to figure it out, I have to differentiate x with respect to t. And remember x is up here.
It's defined by a cos t, so its derivative is - a sin t. And similarly, dy / dt = a cos t. And so I can plug this in.
And I get the arc length element, which is the square root of )- a sin t) ^2 (+ a cos t) ^2 dt.
Which just becomes the square root of a ^2 dt, or a dt.
Now, I was about to divide by t. Let me do that now.
We can also write the rate of change of arc length with respect to t. And that's a, in this case.
And this gets interpreted as the speed of the particle going around.
So not only, let me trade these two guys, not only do we have the direction is counterclockwise, but we also have that the speed is, if you like, it's uniform.
It's constant speed.
And the rate is a.
So that's ds / dt.
Travelling around.
And that means that we can play around with the speed.
And I just want to point out.
So the standard thing, what you'll have to get used to, and this is a standard presentation, you'll see this everywhere.
In your physics classes and your other math classes, if you want to change the speed, so a new speed going around this would be, if I set up the equations this way.
Now I'm tracing around the same circle.
But the speed is going to turn out to be, if you figure it out, there'll be an extra factor of k. So it'll be a k. That's what we'll work out to be the speed.
Provided k is positive and a is positive.
So we're making these conventions.
The constants that we're using are positive.
Now, that's the first and most basic example.
The one that comes up constantly.
Now, let me just make those comments about notation that I wanted to make.
And we've been treating these squared differentials here for a little while and I just want to pay attention a little bit more carefully to these manipulations.
And what's allowed and what's not.
And what's justified and what's not.
So the basis for this was this approximate calculation that we had, that delta s ^2 was delta x ^2 + delta y ^2.
This is how we justified the arc length formula before.
And let me just show you that the formula that I have up here, this basic formula for arc length in the parametric form, follows just as the other one did.
And now I'm going to do it slightly more rigorously.
I do the division really in disguise before I take the limit of the infinitesimal.
So all I'm really doing is I'm doing this.
Dividing through by this, and sorry this is still approximately equal.
So I'm not dividing by something that's 0 or infinitesimal.
I'm dividing by something non-0.
And here I have (delta x/ delta t) ^2 + (delta y / delta t) ^2.
And then in the limit, I have ds / dt = to the square root of this guy.
Or, if you like, the square of it, so.
So it's legal to divide by something that's almost 0 and then take the limit as we go to 0.
This is really what derivatives are all about.
That we get a limit here.
As the denominator goes to 0.
Because the numerator's going to 0 too.
So that's the notation.
And now I want to warn you, maybe just a little bit, about misuses, if you like, of the notation.
We don't do absolutely everything this way.
This expression that came up with the squares, you should never write it as this.
This, put it on the board but very quickly, never.
OK. Don't do that.
We use these square differentials, but we don't do it with these ratios here.
But there was another place which is slightly confusing.
It looks very similar, where we did use the square of the differential in a denominator.
And I just want to point out to you that it's different.
It's not the same.
And it is OK. And that was this one.
This thing here.
This is a second derivative, it's something else.
And it's got a dt squared in the denominator.
So it looks rather similar.
But what this represents is the quantity d / dt ^2.
And you can see the squares came in.
And squared the two expressions.
And then there's also an x over here.
So that's legal.
Those are notations that we do use.
And we can even calculate this.
It has a perfectly good meaning.
It's the same as the derivative with respect to t of the derivative of x, which we already know was - sine.
Sorry, a sine t, I guess.
Not this example, but the previous one.
Up here.
So the derivative is this and so I can differentiate a second time.
And I guess - a cosine t. So that's a perfectly legal operation.
Everything in there makes sense.
Just don't use that.
There's another really unfortunate thing, right which is that the 2 creeps in funny places with signs.
You have sin^2.
It would be out here, it comes up here for some strange reason.
This is just because typographers are lazy or somebody somewhere in the history of mathematical typography decided to let the 2 migrate.
It would be like putting the 2 over here.
There's inconsistency in mathematics right.
We're not perfect and people just develop these notations.
So we have to live with them.
The ones that people accept as conventions.
The next example that I want to give you is just slightly different.
It'll be a non-constant speed parameterization.
Here x = 2 sine t. And y = say, cosine t. And let's keep track of what this one does.
Now, this is a skill which I'm going to ask you about quite a bit.
And it's one of several skills.
You'll have to connect this with some kind of rectangular equation.
An equation for x and y.
And we'll be doing a certain amount of this today.
In another context.
Right here, to see the pattern, we know that the relationship we're going to want to use is that sin^2 + cos^2 = 1.
So in fact the right thing to do here is to take 1/4 x ^2 + y ^2.
And that's going to turn out to be sin ^2 t + cos ^2 t. Which is 1.
So there's the equation.
Here's the rectangular equation for this parametric curve.
And this describes an ellipse.
That's not the only information that we can get here.
The other information that we can get is this qualitative information of where we start, where we're going, the direction.
It starts out, I claim, at t = 0.
That's when t = = 0, this is (2 sine 0, cosine 0), right?
(2 sine 0, cosine 0) = the point (0, 1).
So it starts up, up here.
At (0, 1).
And then the next little place, so this is one thing that certainly you want to do.
t = pi / 2 is maybe the next easy point to plot.
And that's going to be (2 sine pi / 2, cosine pi / 2).
And that's just (2, 0).
And so that's over here somewhere.
This is (2, 0).
And we know it travels along the ellipse.
And we know the minor axis is 1, and the major axis is 2, so it's doing this.
So this is what happens at t = 0.
This is where we are at t = pi / 2.
And it continues all the way around, etc.
To the rest of the ellipse.
This is the direction.
So this one happens to be clockwise.
Alright, now let's keep track of its speed.
Let's keep track of the speed, and also the arc length.
So the speed is the square root of the derivatives here.
That would be (2 cosine t) ^2 + (sine t) ^2.
And the arc length is what?
Well, if we want to go all the way around, we need to know that that takes a total of 2 pi.
So 0 to 2 pi.
And then we have to integrate ds, which is this expression.
Or ds/ dt, dt.
So that's the square root of 4 cosine^2 t + sine ^2 t dt.
The bad news, if you like, is that this is not an elementary integral.
In other words, no matter how long you try to figure out how to antidifferentiate this expression, no matter how many substitutions you try, you will fail.
That's the bad news.
The good news is this is not an elementary integral.
It's not an elementary integral.
Which means that this is the answer to a question.
Not something that you have to work on.
So if somebody asks you for this arc length, you stop here.
That's the answer, so it's actually better than it looks.
And we'll try to -- I mean, I don't expect you to know already what all of the integrals are that are impossible.
And which ones are hard and which ones are easy.
So we'll try to coach you through when you face these things.
It's not so easy to decide.
I'll give you a few clues, but.
OK.
So this is the arc length.
Now, I want to move on to the last thing that we did.
Last type of thing that we did last time.
Which is the surface area.
And yeah, question
[INAUDIBLE]
The question, this is a good question.
The question is, when you draw the ellipse, do you not take into account what t is.
The answer is that this is in disguise.
What's going on here is we have a trouble with plotting in the plane what's really happening.
So in other words, it's kind of in trouble.
So the point is that we have two functions of t, not one.
x ( t) and y ( t).
So one thing that I can do if I plot things in the plane.
In other words, the main point to make here is that we're not talking about the situation.
y is a function of x.
We're out of that realm now.
We're somewhere in a different part of the universe in our thought.
And you should drop this point of view.
So this depiction is not y as a function of x.
Well, that's obvious because there are two values here, as opposed to one.
So we're in trouble with that.
And we have that background parameter, and that's exactly why we're using it.
This parameter t. So that we can depict the entire curve.
And deal with it as one thing.
So since I can't really draw it, and since t is nowhere on the map, you should sort of imagine it as time, and there's some kind of trajectory which is travelling around.
And then I just labelled a couple of the places.
If somebody asked you to draw a picture of this, well, I'll tell you exactly where you need the picture in just one second, alright.
It's going to come up right now in surface area.
But otherwise, if nobody asks you to, you don't even have to put down t = 0 and t = pi / 2 here.
Because nobody demanded it of you.
Another question
[INAUDIBLE]
So, another very good question which is exactly connected to this picture.
So how is it that we're going to use the picture, and how is it we're going to use the notion of the t. The question was, why is this from t = 0 to t = 2 pi?
That does use the t information on this diagram.
the point is, we do know that t starts here.
This is pi / 2, this is pi, this is 3 pi / 2, and this is 2 pi.
When you go all the way around once, it's going to come back to itself.
These are periodic functions of period 2 pi.
And they come back to themselves exactly at 2 pi.
And so that's why we know in order to get around once, we need to go from 0 to 2 pi.
And the same thing is going to come up with surface area right now.
That's going to be the issue, is what range of t we're going to need when we compute the surface area
[INAUDIBLE]
In a question, what you might be asked is what's the rectangular equation for a parametric curve?
So that would be 1/4 x^2 + y ^2 = 1.
And then you might be asked, plot it.
Well, that would be a picture of the ellipse.
OK, those are types of questions that are legal questions
[INAUDIBLE]
The question is, do I need to know any specific formulas?
Any formulas that you know and remember will help you.
They may be of limited use.
I'm not going to ask you to memorize anything except, I guarantee you that the circle is going to come up.
Not the ellipse, the circle will come up everywhere in your life.
So at least at MIT, your life at MIT.
We're very round here.
Yeah, another question
I'm just a tiny bit confused back to the basics.
This is more a question from yesterday, I guess.
But when you have your original ds^2= dx^2 + dy ^2, and then you integrate that to get arc length, how are you, the integral has dx's and dy's.
So how are you just integrating with respect to dx?
OK, the question is how are we just integrating with respect to x?
So this is a question which goes back to last time.
And what is it with arc length.
so.
I'm going to have to answer that question in connection with what we did today.
So this is a subtle question.
But I want you to realize that this is actually an important conceptual step here.
So shhh, everybody, listen.
If you're representing one-dimensional objects, which are curves, maybe, in space.
Or in two dimensions.
When you're keeping track of arc length, you're going to have to have an integral which is with respect to some variable.
But that variable, you get to pick.
And we're launching now into this variety of choices of variables with respect to which you can represent something.
Now, there are some disadvantages on the circle to representing things with respect to the variable x.
Because there are two points on the circle here.
On the other hand, you actually can succeed with half the circle.
So you can figure out the arc length that way.
And then you can set it up as an integral dx.
But you can also set it up as an integral with respect to any parameter you want.
And the uniform parameter is perhaps the easiest one.
This one is perhaps the easiest one.
And so now the thing that's strange about this perspective, and I'm going to make this point later in the lecture as well.
Is that the letters x and y, as I say, you should drop this notion that y is a function of x.
This is what we're throwing away at this point.
What we're thinking of is, you can describe things in terms of any coordinate you want.
You just have to say what each one is in terms of the others.
And these x and y over here are where we are in the Cartesian coordinate system.
They're not, and in this case they're functions of some other variable.
Some other variable.
So they're each functions.
So the letters x and y just changed on you.
They mean something different.
x is no longer the variable.
It's the function.
Right?
You're going to have to get used to that.
That's because we run out of letters.
And we kind of want to use all of them the way we want.
I'll say some more about that later.
So now I want to do this surface area example.
I'm going to just take the surface area of the ellipsoid.
The surface of the ellipsoid formed by revolving this previous example, which was Example 2.
Around the y axis.
So we want to set up that surface area integral here for you.
Now, I remind you that the area element looks like this.
If you're revolving around the y axis, that means you're going around this way and you have some curve.
In this case it's this piece of an ellipse.
If you sweep it around you're going to get what's called an ellipsoid.
And there's a little chunk here, that you're wrapping around.
And the important thing you need besides this ds, this arc length piece over here, is the distance to the axis.
So that's this horizontal distance here.
I'll draw it in another color.
And that horizontal distance now has a name.
And this is, again, the virtue of this coordinate system.
The t is something else.
This has a name.
This distance has a name.
This distance is called x.
And it even has a formula.
Its formula is 2 sin t. In terms of t. So the full formula up for the integral here is, I have to take the circumference when I spin this thing around.
And this little arc length element.
So I have here 2 pi ( 2 sin t).
That's the x variable here.
And then I have here ds, which is kind of a mess.
So unfortunately I don't quite have room for it.
Plan ahead.
Square root of 4 cos^2 t + sin^2 t, is that what it was, dt?
Alright, I guess I squeezed it in there.
So that was the arc length, which I re-copied from this board above.
That was the ds piece.
It's this whole thing including the dt.
That's the answer except for one thing.
What else do we need?
We don't just need the integrand, this is half of setting up an integral.
The other half of setting up an integral is the limits.
We need specific limits here.
Otherwise we don't have a number that we can get out.
So we now have to think about what the limits are.
And maybe somebody can see.
It has something to do with this diagram of the ellipse over here.
Can somebody guess what it is?
0 to pi.
Well, that was quick.
That's it.
Because we go from the top to the bottom, but we don't want to continue around.
We don't want to go from 0 to 2 pi, because that would be duplicating what we're going to get when we spin around.
And we know that we start at 0.
It's interesting because it descends when you change variables to think of it in terms of the y variable it's going the opposite way.
But anyway, just one piece of this is what we want.
So that's this setup.
And now I claim that this is actually a doable integral.
However, it's long.
I'm going to spare you, I'll just tell you how you would get started.
You would use the substitution u = cos t. And then the du is going to be - sin t dt.
But then, unfortunately, there's a lot more.
There's another trig substitution with some other multiple of the cosine and so forth.
So it goes on and on.
If you want to check it yourself, you can.
There's an inverse trig substitution which isn't compatible with this one.
But it can be done.
Calculated.
In elementary terms.
Yeah, another question
[INAUDIBLE]
So, if you get this on an exam, I'm going to have to coach you through it.
Either I'm going to have to tell you don't evaluate it or, you're going to have to work really hard.
Or here's the first step, and then the next step is, keep on going.
Or something.
I'll have to give you some cues.
Because it's quite long.
This is way too long for an exam, this particular one.
OK.
It's not too long for a problem set.
This is where I would leave you off if I were giving it to you on a problem set.
Just to give you an idea of the order of magnitude.
Whereas one of the ones that I did yesterday, I wouldn't even give you on a problem set, it was so long.
So now, our next job is to move on to polar coordinates.
Now, polar coordinate involve the geometry of circles.
As I said, we really love circles here.
We're very around.
Just as I love 0, the rest of the Institute loves circles.
So we're going to do that right now.
What we're going to talk about now is polar coordinates.
Which are set up in the following way.
It's a way of describing the points in the plane.
Here is a point in a plane, and here's what we think of as the usual x-y axes.
And now this point is going to be described by a different pair of coordinates, different pair of numbers.
Namely, the distance to the origin.
And the second parameter here, second number here, is this angle theta.
Which is the angle of ray from origin with the horizontal axis.
So that's what it is in language.
And you should put this in quotation marks, because it's not a perfect match.
This is geometrically what you should always think of, but the technical details involve dealing directly with formulas.
The first formula is the formula for x.
And this is the fundamental, these two are the fundamental ones.
Namely, x = r cos theta.
The second formula is the formula for y, which is r sin theta.
So these are the unambiguous definitions of polar coordinates.
This is it.
And this is the thing from which all other almost correct statements almost follow.
But this is the one you should trust always.
This is the un ambiguous statement.
So let me give you an example something that's close to being a good formula and is certainly useful in its way.
Namely, you can think of r as being the square root of x ^2 + y ^2.
That's easy enough to derive, it's the distance to the origin.
That's pretty obvious.
And the formula for theta, which you can also derive, which is that it's the inverse tangent of y / x.
However, let me just warn you that these formulas are slightly ambiguous.
So somewhat ambiguous.
In other words, you can't just apply them blindly.
You actually have to look at a picture in order to get them right.
In particular, r could be plus or minus here.
And when you take the inverse tangent, there's an ambiguity between, it's the same as the inverse tangent of - y / - x.
So these minus signs are a plague on your existence.
And you're not going to get a completely unambiguous answer out of these formulas without paying attention to the diagram.
On the other hand, the formula up in the box there always works.
So when people mean polar coordinates, they always mean that.
And then they have conventions, which sometimes match things up with the formulas over on this next board.
Let me give you various examples here first.
But maybe first I should I should draw the two coordinate systems.
So the coordinate system that we're used to is the rectangular coordinate system.
And maybe I'll draw it in orange and green here.
So these are the coordinate lines y = 0, y = 1, y = 2.
That's how the coordinate system works.
And over here we have the rest of the coordinate system.
And this is the way we're thinking of x and y now.
We're no longer thinking of y as a function of x and x as a function of y, we're thinking of x as a label of a place in a plane.
And y as a label of a place in a plane.
So here we have x = 0, x = 1, x = 2, etc.
Here's x = - 1.
So forth.
So that's what the rectangular coordinate system looks like.
And now I should draw the other coordinate system that we have.
Which is this guy here.
Well, close enough.
And these guys here.
Kind of this bulls-eye or target operation.
And this one is, say, theta = pi / 2.
This is theta = 0.
This is theta = - pi / 4.
For instance, so I've just labeled for you three of the rays on this diagram.
It's kind of like a radar screen.
And then in pink, this is maybe r = 2, the radius 2.
And inside is r = 1.
So it's a different coordinate system for the plane.
And again, the letter r represents measuring how far we are from the origin.
The theta represents something about the angle, which ray we're on.
And they're just two different variables.
And this is a very different kind of coordinate system.
OK so, our main job is just to get used to this.
For now.
You will be using this a lot in 18.02.
It's very useful in physics.
And our job is just to get started with it.
And so, let's try a few examples here.
Tons of examples.
We'll start out very slow.
If you have (x, y) = (1, - 1), that's a point in the plane.
I can draw that point.
It's down here, right?
This is - 1 and this is 1, and here's my point, (1, - 1).
I can figure out what the representative is of this in polar coordinates.
So in polar coordinates, there are actually a bunch of choices here.
First of all, I'll tell you one choice.
If I start with the angle horizontally, I wrap all the way around.
That would be to this ray here, let's do it in green again.
Alright, I labeled it actually as - pi / 4, but another way of looking at over here it is that it's this angle here.
So that would be r = square root of 2.
Theta = 7 pi / 4.
So that's one possibility of the angle and the distance.
I know the distance is a square root of 2, that's not hard.
Another way of looking at it is the way which was suggested when I labeled this with a negative angle.
And that would be r = square root of 2, theta = - pi / 4.
And these are both legal.
These are perfectly legal representatives.
And that's what I meant by saying that these representations over here are somewhat ambiguous.
There's more than one answer to this question, of what the polar representation is.
A third possibility, which is even more dicey but also legal, is r = - square root of 2.
Theta = 3 pi / 4.
Now, what that corresponds to doing is going around to here.
We're pointing out 3/4 pi, direction.
But then going negative square root of 2, distance.
We're going backwards.
So we're landing in the same place.
So this is also legal.
Yeah
[INAUDIBLE]
The question is, don't the radiuses have to be positive because they represent a distance to the origin?
The answer is I lied to you here.
All of these things that I said are wrong, except for this.
Which is the rule for what polar coordinates mean.
So it's maybe plus or minus the distance, is what it is always.
I try not to lie to you too much, but I do succeed.
Now, let's do a little bit more practice here.
There are some easy examples, which I will run through very quickly.
r = a, we already know this is a circle.
And the 3 theta = a constant is a ray.
However, this involves an implicit assumption, which I want to point out to you.
So this is Example 3.
Theta's equal to a constant as a ray.
But this implicitly assumes 0 <= r < infinity.
If you really wanted to allow minus infinity < r < infinity in this example, you would get a line.
Gives the whole line.
It gives everything behind.
So you go out on some ray, you go backwards on that ray and you get the whole line through the origin, both ways.
If you allow r going to minus infinity as well.
So the typical conventions, so here are the typical conventions.
And you will see people assume this without even telling you.
So you need to watch out for it.
The typical conventions are certainly this one, which is a nice thing to do.
Pretty much all the time, although not all the time.
Most of the time.
And then you might have theta ranging from minus pi to pi, so in other words symmmetric around 0.
Or, another very popular choice is this one.
Theta's >= 0 and strictly less than 2 pi.
So these are the two typical ranges in which all of these variables are chosen.
But not always.
You'll find that it's not consistent.
As I said, our job is to get used to this.
And I need to work up to some slightly more complicated examples.
Some of which I'll give you on next Tuesday.
But let's do a few more.
So, I guess this is Example 4.
Example 4, I'm going to take y = 1.
That's awfully simple in rectangular coordinates.
But interestingly, you might conceivably want to deal with it in polar coordinates.
If you do, so here's how you make the translation.
But this translation is not so terrible.
What you do is, you plug in y = r sin theta.
That's all you have to do.
And so that's going to be equal to 1.
And that's going to give us our polar equation.
The polar equation is r = 1 / sin theta.
There it is.
And let's draw a picture of it.
So here's a picture of the line y = 1.
And now we see that if we take our rays going out from here, they collide with the line at various lengths.
So if you take an angle, theta, here there'll be a distance r corresponding to that and you'll hit this in exactly one spot.
For each theta you'll have a different radius.
And it's a variable radius.
It's given by this formula here.
And so to trace this line out, you actually have to realize that there's one more thing involved.
Which is the possible range of theta.
Again, when you're doing integrations you're going to need to know those limits of integration.
So you're going to need to know this.
The range here goes from theta = 0, that's sort of when it's out at infinity.
That's when the denominator is 0 here.
And it goes all the way to pi.
Swing around just one half-turn.
So the range here is 0 < theta < pi.
Yeah, question
[INAUDIBLE]
The question is, is it typical to express r as a function of theta, or vice versa, or does it matter?
The answer is that for the purposes of this course, we're almost always going to be writing things in this form.
r as a function of theta.
And you can do whatever you want.
This turns out to be what we'll be doing in this course, exclusively.
As you'll see when we get to other examples, it's the traditional sort of thing to do when you're thinking about observing a planet or something like that.
You see the angle, and then you guess far away it is.
But it's not necessary.
The formulas are often easier this way.
For the examples that we have.
Because it's usually a trig function of theta.
Whereas the other way, it would be an inverse trig function.
So it's an uglier expression.
As you can see.
The real reason is that we choose this thing that's easier to deal with.
So now let me give you a slightly more complicated example of the same type.
Where we use a shortcut.
This is a standard example.
And it comes up a lot.
And so this is an off-center circle.
A circle is really easy to describe, but not necessarily if the center is on the rim of the circle.
So that's a different problem.
And let's do this with a circle of radius a.
So this is the point (a, 0) and this is (2a, 0).
And actually, if you know these two numbers, you'll be able to remember the result of this calculation.
Which you'll do about five or six times and then finally you'll memorize it during 18.02 when you will need it a lot.
So this is a standard calculation here.
So the starting place is the rectangular equation.
And we're going to pass to the polar representation.
The rectangular representation is (x - a) ^2 + y ^2 = a ^2.
So this is a circle centered at (a, 0) of radius a.
And now, if you like, the slow way of doing this would be to plug in x = r cos theta, y = r sin theta.
The way I did in this first step.
And that works perfectly well.
But I'm going to do it more quickly than that.
Because I can sort of see in advance how it's going to work.
I'm just going to expand this out.
And now I see the a ^2's cancel.
And not only that, but x^2 + y &2 = r ^2.
So this becomes r ^2.
That's x ^2 + y ^2 - 2ax = 0.
The r came from the fact that r ^2 = x ^2 + y ^2.
So I'm doing this the rapid way.
You can do it by plugging in, as I said.
r equals.
So now that I've simplified it, I am going to use that procedure.
I'm going to plug in.
So here I have r ^2 - 2a r cos theta = 0.
I just plugged in for x.
As I said, I could have done that at the beginning.
I just simplified first.
And now, this is the same thing as r ^2 = 2ar cos theta.
And we're almost done.
There's a boring part of this equation, which is r = 0.
And then there's, if I divide by r, there's the interesting part of the equation.
Which is this.
So this is or r = 0.
Which is already included in that equation anyway.
So I'm allowed to divide by r because in the case of r = 0, this is represented anyway.
Question
[INAUDIBLE]
r = 0 is just one case.
That is, it's the union of these two.
It's both.
Both are possible.
So r = 0 is one point on it.
And this is all of it.
So we can just ignore this.
So now I want to say one more important thing.
You need to understand the range of this.
So wait a second and we're going to figure out the range here.
The range is very important, because otherwise you'll never be able to integrate using this representation here.
So this is the representation.
But notice when theta = 0, we're out here at 2a.
That's consistent, and that's actually how you remember this factor 2a here.
Because if you remember this picture and where you land when theta = 0.
So that's the theta = 0 part.
But now as I tip up like this, you see that when we get to vertical, we're done.
With the circle.
It's gotten shorter and shorter and shorter, and at theta = pi / 2, we're down at 0.
Because that's cos pi / 2 = 0.
So it swings up like this.
And it gets up to pi / 2.
Similarly, we swing down like this.
And then we're done.
So the range is - pi / 2 < theta < pi / 2.
Or, if you want to throw in the r = 0 case, you can throw in this, this is repeating, if you like, at the ends.
So this is the range of this circle.
And let's see.
Next time we'll figure out area in polar coordinates.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So again, welcome back.
And today's topic is a continuation of what we did last time.
We still have a little bit of work and thinking to do concerning polar coordinates.
So we're going to talk about polar coordinates.
And my first job today is to talk a little bit about area.
That's something we didn't mention last time.
And since we're all back from Thanksgiving, we can certainly talk about it in terms of a pie.
Which is the basic idea for area in polar coordinates.
Here's our pie, and here's a slice of the pie.
The slice has a piece of arc length on it, which I'm going to call delta theta.
And the area of that shaded-in slice, I'm going to call delta A.
And let's suppose that the radius is a.
Little a.
So this is a pie of radius a.
That's our picture.
Now, it's pretty easy to figure out what the area that slice of pie is.
The total area is, of course, pi a ^2.
We know that.
And to get this fraction, delta A, all we have to do is take the percentage of the arc of the total circumference.
That's delta theta / 2 pi.
This is the fraction of area -- sorry, fraction of the total circumference, the total length around the rim.
And then we multiply that by pi a ^2.
And that's giving us the total area.
And if you work that out, that's delta A is equal to, the pi's cancel and we have 1/2 a ^2 delta theta.
So here's the basic formula.
And now what we need to do is to talk about a variable pie here.
That would be a pie with a kind of a wavy crust.
Which is coming around like this.
So r = r (theta).
The distance from the center is varying with the place where we are, the angle where we're shooting out.
And now I want to subdivide that into little chunks here.
Now, the idea for adding up the area, the total area of this piece that's swept out, is to break it up into little slices whose areas are almost easy to calculate.
Namely, what we're going to do is to take, and I'm going to label it this way.
I'm going to take these little circular arcs, which go.
So I'm going to extend past where this goes.
And then I'm going to take each circular arc here.
So here's a circular arc.
And then here's another circular arc.
And here's another circular arc.
It's just right on the nose in that case.
Now, in these two cases, so basically the picture that I'm trying to draw for you is this.
I have some sector.
And then I have some circular arc.
And maybe it takes a little extra.
There's a little extra area, I'm making an error in the area.
This is a little extra area.
And maybe to draw it the other way.
I'm a little short on this one.
And let's say on this one I'm right on the nose.
I have the same arc as the curve of the surface.
Now this is a little bit like the step functions that we used in Riemann sums.
It's practically the same.
Eventually, this little band of stuff that we're missing by, if we take very, very narrow little slices here, is going to be negligible.
It'll get closer and closer to the curve itself.
So that area will tend to 0 in the limit.
So we don't have to worry about it.
And the approximate relationship is sitting here.
Where this distance now is r. So this radius is r. And this is this delta theta.
And so in the approximate case, what we have is that delta A is approximately 1/2 r^2 delta theta.
Which is practically the same thing we had here.
Except that that r is replacing the constant there.
And it's approximately true, because r is varying.
And then in the limit, we have the exact formula for the differential.
Which is this one.
So this is the main formula for area.
And if you like, the total area then is going to be the integral from some starting place to some end place of 1/2 r ^2 d theta.
Now, this is only useful in the situation that we're in.
Namely, so this is the other important formula.
And this is only useful when r is a function of theta.
When this is the way in which the region is presented to us.
So that's the setup.
And that's our main formula.
Let's do what example.
The example that I'm going to take is the one that we did at the end of last time, or near the end of last time.
Which was this formula here.
r = 2a cos theta.
Remember, that was the same as (x - a) ^2 + y ^2 = a ^2.
So this is what we did last time.
We connected this rectangular representation to that polar representation.
And the picture is of a circle.
Where this is the point (2a, 0).
So let's figure out what the area is.
Well, first of all, we have to figure out when we sweep out the area, we have to realize that we only go from - pi / 2 to pi / 2.
So that's something we can get from the picture.
You can also get it directly from this formula if you realize that cosine is positive in this range here.
And at the ends, it's 0.
So the thing encloses a region at these ends.
So at the ends, cosine of plus or minus pi / 2 = 0.
That's what synchs this up like a little sack, if you like.
So the area is now going to be the integral from - pi / 2 to pi / 2 of 1/2 times the square of r, that's (2a cos theta)^2 d theta.
Question
[INAUDIBLE]
How do I know from looking at the picture that I'm going from - pi / 2 to pi / 2, is the question.
I do it with my whole body.
I say, here I am pointing down.
That's - pi / 2.
I sweep up, that's 0.
And I get all the way up to here.
that's pi / 2.
So that's the way I do it.
That's really the way I do it, I'm being honest.
Now if you're a machine, you can't actually look.
And you don't have a body, so you can't point your arms.
Then you would have to go by the formulas.
And you'd have to actually use something like this formula here.
The fact that this is where the loop syncs up.
This is where the radius comes into 0.
At pi / 2.
So you need to know that in order to understand the range.
Another question
[INAUDIBLE]
So when we're doing each, and we just guess that it's going to be a loop.
I'm probably going to give you some clues as to what's going on.
Because it's very hard to figure these things out.
Sometimes it'll be bounded by one curb and another curb, and I'll say it's the thing in between those two curbs.
That's the kind of thing that I could do.
Here, you really should know this one in advance.
This is by far the most, or this is one of the typical cases, anyway.
I'm going to give you a couple more examples.
Don't get too worked up over this.
You will somehow be able to visualize it.
I'll give you some examples to help you out with it later.
So here's the situation.
Here's my integral.
And now we're faced with a trig integral.
Which we have to remember how to do.
Now, the trig integral here -- so first let me factor out the constants.
This is 4a ^2 / 2, so it's 2a ^2 integral from - pi / 2 to pi / 2 of cos ^2 theta d theta.
And now you have to remember what you're supposed to do at this point.
So think, if you haven't done it yet, this is practice you need to do.
This trig integral is handled by a double angle formula.
As it happens, I'm going to be giving you these formulas on the review sheet.
You'll see they're written on the review sheet.
At least in some form.
So for example, there's a formula, and this will be on the exam, too.
So this is the correct formula to use here.
Is that this is 1 + cos 2 theta / 2 d theta.
So that's the substitution that you use for the cosine ^2 in order to integrate it.
That serves as a little review of trig integrals.
And now, this is quite easy.
This integral now is easy.
Why is it easy?
Well, because it's the antiderivative of a constant, and cos 2 theta its antiderivative you're supposed to be able to write down.
So the antiderivative of 1 is theta.
And the antiderivative of the cos is 1/2 sin when it's 2 theta.
And that is a ^2 (pi /2 - (- pi) / 2).
And the signs go away because they're both 0.
So all told we get pi a ^2, which is certainly what we would like it to be.
It's the area of the circle.
Another question?
[INAUDIBLE]
The question, so I'm not sure which question you're asking.
I pivoted my arm around (0, 0).
This point, this is the point we're talking about, (0, 0), is a key point.
It's where I guess you could say I stuck my elbow there.
Now, the reason is that it's the place where r = 0.
So it's more or less the center of the universe from the point of view of this problem.
So it's the reference point and if you like, when you're doing this, it's a little bit like a radar screen.
Everything is centered at the origin and you're taking rays coming out from it.
And seeing where they're going to go.
So for example, this is the theta = 0 ray, this is the theta = pi / 4 ray.
This the theta = pi / 2 ray.
And indeed, if my elbow is right at this center here, I'm pointing in those various directions.
So that's what I had in mind when I did that.
You can always get these formulas, by the way, from the original business, x = r cos theta, y = r sin theta.
But it's useful to have the geometric picture as well.
In other words, if you were a machine you'd have to rely on these formulas.
And plot things using these.
Always.
Now, in terms of plotting I want to expand your brain a little bit.
So we need just a little bit more practice with plotting.
In polar coordinates.
And so, the first question that I want to ask you is, what happens outside of this range of theta?
In other words, what happens if theta's beyond pi / 2?
Can somebody see what's happening to the formulas in that case?
So what I'm looking at now, let's go back to it.
What I'm looking at is this formula here.
But to use the elbow analogy here, I'm swept around like this.
But now I'm going to point this way.
I'm going to point out over there.
My hand is up here in the northwest direction.
So what's going to happen?
Somebody want to tell me?
[INAUDIBLE]
It goes around itself.
That's right.
What happens is that when r crosses this vertical, r = 0, when it crosses over here it goes negative.
So although my theta is pointing me this way, the thing is going to go backwards.
And there's another clue.
Which is very important.
How far backwards is it going?
Well, you don't actually need to know anything but this equation here, to understand that it has to be on the same circle.
So when I'm pointing this way, the things points backwards to this point over there.
So what happens is, it goes around once.
And then when I point out this way, it sweeps around a second time.
It just keeps on going around the same circle.
So over here it's empty.
Because it's pointing the other way and it's sweeping around the same curve.
A second time.
Now, if you were foolish enough to integrate, say, from 0 to 2 pi or some wider range, what would happen is you would just double the area.
Because you would have swept it out twice.
So that's the mistake that you'll make.
Sometimes you'll count things as negative and positive.
But because there's a square here, it's always a positive quantity.
And you'll always over-count if you go too far.
So that's what happens.
Again, it sweeps out the same region.
That's because these two equations really are equivalent to each other.
It's just that this one sweeps it out twice.
And this one doesn't say how it's sweeping it out.
Yeah, another question
Doesn't this equation also work if you just go from 0 to pi?
Does the integration work if you just go from 0 to pi?
The answer is yes.
That's a very weird object, though.
Let me just show you what that is.
If you started from 0 to 2 pi.
So I'll illustrate it on here.
The first thing that you swept out between 0 and pi over 2 is this part here.
That was swept out.
And then, when you're going around this next quadrant here, you're actually sweeping out this underside here.
So actually, you're getting it because you're getting half of it on one half, and getting the other half on the other quadrant.
So it's actually giving you the right answer.
That turns out to be OK.
It's a little weird way to chop up a circle.
But it's legal.
But of course, that's an accident of this particular figure.
You can't count on that happening.
It's much better to line it up exactly with what the figure does.
So don't do that too often.
You might run into troubles.
So I'm going to give you a couple more examples of practice with these pictures.
And maybe I'm going to get rid of this one up here.
So here's another favorite.
Here's another favorite.
So this, if you like, is Example 2.
I guess we had an Example 1 up there.
And now we're really not going to try to do any more area examples.
The area examples are actually straightforward.
It's really just figuring out what the picture looks like.
So this is examples of drawings.
So this one is one that's kind of fun to do.
This is r = sin 2 theta.
Something like this is on your homework.
And so what happens here is the following.
What happens here is that at theta = 0, that's the first place.
So let's just plot a few places here.
I'm not going to plot very many.
Theta's = 0, I get r as 1.
Whoops, I get r is 0.
Sorry.
And then pi / 4, that's where I get sin pi / 2, I get 1 here.
For this.
And then again, at pi / 2, I get sin pi, which is back at 0 again.
So and the other thing to say is in between here it's positive.
In between.
So what it does is, it starts out at 0 and it goes out to the radius 1 over here.
And then it comes back.
So it does something like this.
It goes out, and it comes back.
Now because of the symmetries of the sine function, this is pretty much all you need to know.
It does something similar in all of the quadrants.
But in order to see what it's doing, it's useful for you to watch me draw it.
Because the order is very important for understanding what it's doing.
It's similar to this weird business with the circle here.
So watch me draw this guy.
I'll draw it in red because it usually has a name.
So here it is.
It does this things.
And then it does this.
And then it does this.
And then it does that.
So it's called a four-leaf rose.
I drew it in pink because it's kind of a rose here.
So it started out over here.
This is Step 1.
And this is the range 0 < theta < pi / 4.
It did this part here.
And then it went to here.
So I should draw these in white, because they're harder to read in red.
But now look at what it did.
It did not make a right angle turn.
It was nice and smooth.
It went around here and then it went down here.
This is 3.
Back here, that's 4.
And then over here, that's 5.
Back up here, that's 6.
And then around here, that's 7.
And down here, that's 8.
And then back where it started and goes around again.
And this is because actually it's switching sign when it crosses the origin.
When it was over in this quadrant the first time, it actually was tracing what's directly behind it.
So this is kind of amusing.
From this little tiny formula you get this pretty diagram here.
Anyway that's, as I say, an old favorite.
And here if you want to do the area of one leaf, you've got to make sure you understand that it's a small piece of the whole.
OK, now I have one last drawing example that I want to discuss with you.
And it involves another skill that we haven't quite gotten enough practice with.
So I'm going to do that one.
And it's also preparation for an exercise.
But one that we're going to do after the test.
So here's my last example.
We're going to discuss what happens with this function here.
Sorry, that's not legible, is it.
That's a cosine.
r = 1 / 1 + 2 cos theta.
Now, the first thing I want to do is just take our time a little bit and plot a few points.
So here's the values of theta and here are the values of r, and we'll see what happens.
And we'll try to figure out what it's doing.
When theta = 0, cos = 1.
So r = 1/3.
The denominator is 1 + 2, so it's 1/3.
If theta, I'm going to make it easy, we're not going to do so many.
I'm going to do pi / 2, that's an easy value of the cosine.
That's cos pi / 2 = 0.
So that value of r = 1.
And now I'm going to back up and do - pi / 2.
- pi / 2, again, cosine = 0.
And r = 1.
So now I'd like to just plot those points anyway, and see what's going on with this expression here.
The first one is a rectangular.
I'm going to write the rectangular coordinates here, not the polar coordinates.
The rectangular coordinates here 1/3 out at the horizontal, so it's (1/3, 0).
The polar coordinates is (1/3, 0), but the rectangular coordinate is also that.
And over here, at pi / 2, the distance is 1.
So this is the point (0, 1) in x-y coordinates.
And then down here at, - pi / 2, it's (0, - 1).
Let me just emphasize.
You should be able to think of this visually if you can crank your arm around and think it.
Or if you're right-handed you'll bend that way now.
Anyway.
Or you'll have to use.
But this also works using this formulas x = r cos theta, y = r sin theta.
Notice that in this case, r was 1 but the cosine was 0.
So you plug in theta = - pi / 2.
And r = 1.
And lo and behold, you get 0 here.
And here you get - 1 here you get 1.
So this is - 1.
So this is an example.
I did it purely visually or sort of organically.
But you can also do it by plugging in the numbers.
Now in between, the denominator is positive.
And it's something in between.
It's going to sweep around something like this.
That's what happens in between.
As theta increases from - pi / 2 to pi / 2.
And now something interesting happens with this particular function, which is that we notice that the denominator is 0 at a certain place.
Namely, if I solve 2 cos theta = - 1, then the denominator is going to be 0 there.
That's cos theta = - 1/2, so theta is equal to, it turns out, plus or minus 2 pi / 3.
Those are the values here.
So when we're out here somewhere, in these directions, there's nothing.
It's going infinitely far out.
Those ways.
OK that's about as much as we'll be able to figure out of this diagram without doing some analytic work.
And that's the other little piece that I want to explain.
Namely, going backwards from polar coordinates to rectangular coordinates.
Which is one thing that we haven't done.
So let's do that.
So what is the rectangular equation?
That means the (x, y) equation for this r = 1 / 1 + 2 cos theta.
And let's see what it is.
Well, first I'm going to clear the denominator here.
This is r + 2r cos theta = 1.
And now I'm going to rewrite it as r = 1 - 2r cos theta.
And the reason for that is that in a minute I'll explain to you why.
This is 1 - 2x.
And this guy, I'm going to square now.
I'm going to make this r ^2 = (1 - 2x) ^2.
And now, with an r^2, I can plug in x ^2 + y ^2.
So this is a standard thing to do.
And it's basically what you're going to do any time you're faced with an equation like this.
Is try to work it out.
And, in these situations where you have 1 / a + b cos theta, or sin theta, you'll always come out with some quadratic expression like this.
Now, I'm going to combine terms.
So here I have - 3x ^2 + y ^2, and put everything on the the left side.
So that's this.
And we recognize, well you're supposed to recognize, that this is what's known as a hyperbola.
If the signs are the same, it's an ellipse.
If the the signs are opposite it's a hyperbola.
And in between, if one of the coefficients on the quadratic is 0, it's a parabola.
So now we see that the picture that we drew there is actually, turns out it's going to have asymptotes, it's going to be a hyperbola.
So now, let me ask you the last little mind-bending question that I want to ask.
Which is, what happens.
So now I'm using my right arm, I guess.
But my elbow's at the origin here.
What happens if I pass outside, to the range where this denominator is negative.
It crossed 0 and it went to negative.
It's sweeping out something over here.
Is it sweeping out the same curve?
Anybody have any idea what it's doing?
Yeah
[INAUDIBLE]
Yeah, exactly.
Good answer.
It's the other branch of the hyperbola.
So what's actually happening is in disguise, there's another branch of the hyperbola which is being swept up by the other piece of this thing.
Now, that is consistent with these algebraic equations.
The algebraic equation that I got here doesn't say which branch of the hyperbola I've got.
It's actually got two branches.
And the curve really was, in disguise, capturing both of them.
I want to make the connection now with the basic formula for area here.
Because this is a really beautiful connection.
And I want to make that connection in connection also with this example.
The hyperbolas, as you probably know, are the trajectories of comets.
And ellipses, which is what you would get if maybe you put 1/2 here instead of a 2, would be the trajectories of planets or asteroids.
But there's actually something much more important, physically that goes on.
That's special about this particular representation of the hyperbola.
And what happens when you get the ellipses as well.
Which is that in this case, r = 0 is the focus of the hyperbola.
And what that means is that it's actually the place where the sun is.
So this is the right representation, if you want the center of gravity in the center of your picture.
And pretty much any other.
I mean, you can't tell that at all from the algebraic equations here.
So this hyperbola is going to be the trajectory of some comet going by here.
And this formula here is actually a rather central formula in astronomy.
Namely, there's something called Kepler's Law.
Which says that the rate of change of area which is swept out is constant.
The rate of change of area relative to the center of mass, relative to the sun.
So in equal areas, this is amount of area.
So this tells you now that when a comet goes around the sun like this, its speed varies.
And it's speed varies according to a very specific rule.
Namely, this one here.
And this rule was observed by Kepler.
But if you have this connection here, we also have something else.
We also know that dA / dt = 1/2 r ^2 d theta / dt.
So that's this formula here, formally dividing by t. That's the rate of change with respect to this time parameter, which is the honest to goodness time.
Real physical time.
And that means, this quantity here is constant.
And this is one of the key insights that physicists had, long after Kepler made his physical observations, they realized that he had managed to get the best physics experiment of all, because it's a frictionless setup.
Outer space, there's no air.
Nothing is going on.
This is what's known nowadays as conservation of angular momentum.
This is the expression for angular momentum.
And what Kepler was observing, it turns out, is what we see all the time in real life.
Which is when you start something spinning around it continues to spin at roughly the same rate.
Or, if you're an ice skater and you get yourself scrunched together a little bit more, you can spin faster.
And there's an exact quantitative rule that does that.
And it's exactly this polar formula here.
So that's a neat thing.
And we will do a little exercise on this rate of change after the exam.
So that's it for generalities and a little pep talk on what's coming up to you when you learn a little more physics.
Right now we need to talk about the exam.
So first of all, let me tell you what the topics are.
They're the same as last year's test.
Which you can take a look at.
And let's see.
So what did we do?
One of the main topics of this unit were techniques of integration.
And there are three, which we will test.
One is trig substitution.
One is integration by parts.
And one is partial fractions.
So that's more than half of the exam, right there.
The other half of the exam is parametric curves.
Arc length.
These are all interrelated.
And area of surfaces of revolution.
Those are the only kind that we can handle.
Just as we did with volume of surfaces of revolution.
And then there's a final topic, which is polar coordinates.
And area in polar coordinates, including area.
That's it.
That's what's on the test, there are six problems.
They're very similar.
Well, they're not actually that similar.
But they're somewhat similar to last year's.
I'd say the test is similar.
Maybe a tiny bit more difficult.
We'll see.
We'll see.
Yeah
[INAUDIBLE]
The question was, we didn't do arc length in polar coordinates, did we?
And the answer is no, we did not.
We did not do arc length in polar coordinates.
When I give you an exercise, I'm going to ask you about, if you know the speed of a comet here, what's the speed of the comet there.
And we'll have to know about arc length for that.
But we're not doing it on this exam.
Other questions
[INAUDIBLE]
The question is, will I expect you to know r equals, so let's see if I can formulate this question.
It's related to this four-leaf rose here.
So the question is, suppose I gave you something that looked like this.
Would I expect you to be able to know what it is.
I think the answer, the fair answer to give you, is if it's this complicated, I only have two possibilities.
I can give you a long time to sketch this out.
And think about what it does.
Or I can tell you that it happens to be a three-leaf rose.
And then you have some clue as to what it's doing.
It doesn't have six.
Because of some weird thing, having to do with repetitions.
But the odds and evens work differently.
So, in fact I would have to tell you what the picture looks like, if it's going to be this complicated.
Similarly, so this is an important point to make, when we come to techniques of integration, any integral that you have, I'm not going to tell you which of these three techniques to use on the ones which are straightforward integrals.
But if it's an integral that I think you're going to get stuck on, either I'm going to give you a hint, tell you how to do it.
Or I'm going to tell you, don't do it.
If I tell you don't do it, don't try to do it.
It may be impossible.
And even if it's possible, it's going to be very long.
Like an hour.
So don't do it unless I tell you to.
On the other hand, all of these setups in this second half of this unit, they involve somehow setting something up.
And they're basically three issues.
One is what the integrand is.
One is what the lower limit is, what is the upper limit.
They're just three things, three inputs, to setting up an integral.
All integrals, this is going to be the setup for all of them.
And then the second step is evaluating.
Which really is what we did in the first half here.
And, unfortunately, we don't have infinitely many techniques and indeed there's some integrals that can't be evaluated and some that are too long.
So we'll just try to avoid those.
I'm not trying to give you ones which are hopelessly long.
Alright, other questions.
Yes
[INAUDIBLE]
The question is, will the percentages be the same.
And the answer is, no.
I'll tell you exactly.
This is 55 points.
Unless I change the point values.
This is 55, and this is 45.
That's what it came out to be.
You are going to want to know about all of the things that I've written down here.
You're definitely going to want to know, for example, surfaces of revolution.
How to set those up.
Yes.
there was another question I saw.
Yes
[INAUDIBLE]
So if you have a partial fraction with something like (x + 2)^2 and maybe an x and maybe an x + 1, and you're interested in what happens with this denominator here?
So what's going to happen is, you're going to need a coefficient for each degree of this.
So altogether, the setup is going to be this.
Plus one for x.
And one for x + 1.
This is the setup.
So you need --
[INAUDIBLE]
So if I change this to being a 3 here, then I need, I guess I'll have to call it E, (x + 2) ^3.
I need that.
Now, it gets harder and harder.
The more repeated roots there are, the more repeated factors there are, the harder it is.
Because the ones you can pick off by the cover-up method are, is just the top one here.
And these two.
So C, D, and E you can get.
But B and A you're going to have to do by either plugging in or some other, more elaborate, algebra.
So the more of these lower terms there are, the worse off you are
[INAUDIBLE]
The question is, does this x ^3 + 21 affect this setup.
And the answer is almost no.
That is, not at all.
It's the same setup exactly.
But, there's one thing.
If the degree gets too big, then you've got to use long division first to knock it down.
I'll give you an example of this type of practice.
Unless there are more question.
Yes
[INAUDIBLE]
Are you going to have to know how to do reduction formulas?
Anything that's a little out of the ordinary like a reduction formula, I will have to coach you to do.
So, in other words, what you'll have to be able to do in that situation is follow directions.
If I tell you OK, you're faced with this, then do an integration by parts.
And do that, then get the reduction formula
[INAUDIBLE]
Yeah.
OK, so the question had do with the partial fractions method.
And what happens if you have a quadratic.
So, for instance, if it were this, this one's too disgusting.
I'm going to just do it with two of them.
So the parts with x + x + 1 are the same.
But now you have linear factors here.
Ax + B / x^2 + 2.
And A, maybe I'll call them 1, and A2 x + B2 / (x^2 + 2) ^2 + C / x + D / x + 1.
This is the way it works.
OK, I'm going to give you one more quick example of an integration technique just to liven things up.
Let's see.
So here's a somewhat tricky example.
This is just a little trickier than I would give you on a test.
But it's the same principle, and I may do this on a final exam.
So suppose you're faced with this integral.
What are you going to do?
Integration by parts, great.
That's right, that's because this guy is begging to be differentiated, to be made simpler.
So that means that I want this one to be u, and I want this one to be v '.
And I want to use integration by parts.
And then u ' = 1 / 1 + x ^2, and v = x ^2 / 2.
So the answer is now, x^2 / 2 tan inverse x - the integral of this guy.
Which is going to be x ^2 / 2.
And then I have 1 / 1 + x ^2 dx.
Now, you are not done at this point.
You're still in slightly hot water.
You're in tepid water, anyway.
So what is it that you have to do here?
You're faced with this integral, which I'll put on the next board.
It's a lot simpler than the other one, but as I say you're not quite out of the woods.
You're faced with the integral of 1/2, - 1/2 x ^2 / 1 + x ^2 dx
[INAUDIBLE]
Trig substitution actually, interestingly, will work.
But that wasn't what I wanted you to do.
I wanted you to, yeah, go ahead
[INAUDIBLE]
Add and subtract 1 to the numerator.
So now, that's the correct answer.
This is the case where the numerator and the denominator are tied.
And so you have to use long division.
But a shortcut is just to observe that the result of long division is the same thing as doing this.
And then noticing that this is 1 - 1 / 1 + x ^2.
So this is the same as long division, in this case.
Because when you divide in, it goes in with a quotient of 1.
And so this guy turns out to be - 1/2 the integral of 1 - 1 / 1 + x ^2 dx.
Which is 1/2 x - 1/2 tan inverse x + c. So this is one extra step that you may be faced with someday in your life.
And just keep that in mind.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we're through with techniques of integration, which is really the most technical thing that we're going to be doing.
And now we're just clearing up a few loose ends about calculus.
And the one we're going to talk about today will allow us to deal with infinity.
And it's what's known as L'Hopital's Rule.
Here's L'Hopital's Rule.
And that's what we're going to do today.
L'Hopital's Rule it's also known as L'Hospital's Rule.
That's the same name, since the circumflex is what you put in French to omit the s. So it's the same thing, and it's still pronounced L'Hopital, even if it's got an s in it.
Alright, so that's the first thing you need to know about it.
And what this method does is, it's a convenient way to calculate limits including some new ones.
So it'll be convenient for the old ones.
There are going to be some new ones and, as an example, you can calculate x ln x as x goes to infinity.
You could, whoops, that's not a very interesting one, let's try x goes to 0 from the positive side.
And you can calculate, for example, x e^ - x, as x goes to infinity.
And, well, maybe I should include a few others.
Maybe something like ln x / x as x goes to infinity.
So these are some examples of things which, in fact, if you plug into your calculator, you can see what's happening with these.
But if you want to understand them systematically, it's much better to have this tool of L'Hopital's Rule.
And certainly there isn't a proof just based on a calculation in a calculator.
So now here's the idea.
I'll illustrate the idea first with an example.
And then we'll make it systematic.
And then we're going to generalize it.
We'll make it much more, so when it includes these new limits, there are some little pieces of trickiness that you have to understand.
So, let's just take an example that you could have done in the very first unit of this class.
The limit as x goes to 1 of x ^ 10 - 1 / x ^2 - 1.
So that's a limit that we could've handled.
And the thing that's interesting, I mean, if you like this is in this category that we mentioned at the beginning of the course of interesting limits.
What's interesting about it is that if you do this silly thing, which is just plug in x = 1, at x = 1 you're going to get 0 / 0.
And that's what we call an indeterminate form.
It's just unclear what it is.
From that plugging, in you just can't get it.
Now, on the other hand, there's a trick for doing this.
And this is the trick that we did at the beginning of the class.
And the idea is I can divide in the numerator and denominator by x - 1.
So this limit is unchanged, if I try to cancel the hidden factor x - 1 in the numerator and denominator.
Now, we can actually carry out these ratios of polynomials and calculate them by long division in algebra.
That's very, very long.
We want to do this with calculus.
And we already have.
We already know that this ratio is what's called a difference quotient.
And then in the limit, it tends to the derivative of this function.
So the idea is that this is actually equal to, in the limit, now let's just study one piece of it.
So if I have a function f ( x), which is x ^ 10 - 1, and the value at 1 happens to be equal to 0, then this expression that we have, which is in disguise, this is in disguise the difference quotient, tends to, as x goes to 1, the derivative, which is f' ( 1).
That's what it is.
So we know what the numerator goes to, and similarly we'll know what the denominator goes to.
But what is that?
Well, f ' (x) = 10x ^ 9.
So we know what the answer is.
In the numerator it's 10x ^ 9.
In the denominator, it's going to be 2x, that's the derivative of x ^2 - 1.
And then were going to have to evaluate that at x = 1.
And so it's going to be 10/2, which is 5.
So the answer is 5.
And it's pretty easy to get from our techniques and knowledge of derivatives, using this rather clever algebraic trick.
This business of dividing by x - 1.
What I want to do now is just carry this method out systematically.
And that's going to give us the approach to what's known as L'Hopital's Rule.
What my main subject for today.
So here's the idea.
Suppose we're considering, in general, a limit as x goes to some number a of f ( x) / g ( x).
And suppose it's the bad case where we can't decide.
So it's in determinate.
f ( a) = g ( a) = 0.
So it would be 0 / 0.
Now we're just going to do exactly the same thing we did over here.
Namely, we're going to divide a numerator and denominator, and we're going to repeat that argument.
So we have here f ( x) / x - a.
And g (x) / x - a also.
I haven't changed anything yet.
And now I'm going to write it in this suggestive form.
Namely, I'm going to take separately the limit in the numerator and the denominator.
And I'm going to make one more shift.
So I'm going to take the limit, as x goes to a in the numerator, but I'm going to write it as f ( x) - f ( a) / x - a.
So that's the way I'm going to write the numerator, and I've got to draw a much longer line here.
So why am I allowed to do that?
That's because f (a) = 0.
So I didn't change this numerator of the numerator any by subtracting.
f ( a) = 0.
And I'll do the same thing to the denominator.
Again, g ( a) = 0, so this is OK. And lo and behold, I know what these limits are.
This is f ' (a) / g '(a).
So that's it.
That's the technique and this evaluates the limit.
And it's not so difficult.
The formula's pretty straightforward here.
And it works, provided that g ' (a) is not 0.
Yeah, question
[INAUDIBLE]
The question is, is there a more intuitive way of understanding this procedure.
And I think the short answer is that there are other, similar, ways.
I don't consider them to be more intuitive.
I will be mentioning one of them, which is the idea of linearization, which goes back to what we did in Unit 2.
I think it's very important to understand all of these, more or less, at once.
But I wouldn't claim that any of these methods is a more intuitive one than the other.
But basically what's happening is, we're looking at the linear approximation to f, at a.
And the linear approximation to g at a.
That's what underlies this.
So now I get to formulate for you L'Hopital's Rule at least in what I would call the easy version or, if you like, Version 1.
So here's L'Hopital's Rule.
Version 1.
It's not going to be quite the same as what we just did.
It's going to be much, much better.
And more useful.
And what is going to take care of is this problem that the denominator is not 0.
So now here's what we're going to do.
We're going to say that it turns out that the limit a x goes to a of f(x) / g ( x) = the limit as x goes to a to a f '(x) / g' (x).
Now, that looks practically the same as what we said before.
And I have to make sure that you understand when it works.
So it works provided this is one of these undefined expressions.
In other words, f ( a) = g ( a) = 0.
So we have a 0 / 0 expression, indeterminant.
And, also, we need one more assumption.
And the right-hand side, the right-hand limit exists.
Now, this is practically the same thing as what I said over here.
Namely, I took the ratio of these functions, x ^ 10 - 1 and x ^2 - 1.
I took their derivatives, which is what I did right here, right.
I just differentiated them and I took the ratio.
This is way easier than the quotient rule, and is nothing like the quotient rule.
Don't think quotient rule.
Don't think quotient rule.
So we differentiate the numerator and denominator separately.
And then I take the limit as x goes to 1 and I get 5.
So that's what I'm claiming over here.
I take these functions, I replace them with this ratio of derivatives, and then I take the limit instead, over here.
And it turned out that the functions got much simpler when I differentiated them.
I started with this messy object and I got this much easier object that I could easily evaluate.
So that's the big game that's happening here.
It works, if this limit makes sense and this limit exists.
Now, notice I didn't claim that g, that the denominator, had to be non-0.
So that's what's going to help us a little bit in a few examples.
So let me give you a couple of examples and then we'll go further.
Now, this is only Version 1.
But first we have to understand how this one works.
So here's another example.
Take the limit as x goes to 0, of sin 5x / sin 2x.
This is another kind of example of a limit that we discussed in the first part of the course.
Unfortunately, now we're reviewing stuff.
So this should reinforce what you did there.
This will be an easier way of thinking about it.
So by L'Hopital's Rule, so here's the step.
We're going to take one of these steps.
This is the limit, as x goes to 1, of the derivatives here.
So that's 5 cos 5x / 2 cos 2x.
The limit was 1 over there, but now it's 0. a is 0 in this case.
This is the number a.
Thank you.
So the limit as x goes to 0 is the same as the limit of the derivatives.
And that's easy to evaluate.
Cosine of 0 = 1, right.
This is equal to 5 (cos (5* 0).
And that's a multiplication sign.
Maybe I should just write this as 0.
Divided by 2 cos 0.
But you know that that's 5/2.
So this is how L'Hopital's method works.
It's pretty painless.
I'm going to give you another example, which shows that it works a little better than the method that I started out with.
Here's what happens if we consider the function cos x - 1 / x ^2.
That was a little harder to deal with.
And again, this is one of these 0 / 0 things near x = 0.
As x tends to 0, this goes to an indeterminate form here.
Now, according to our method, this is equivalent to, now I'm going to use this little wiggle because I don't want to write limit, limit, limit, limit a million times.
So I'm going to use a little wiggle here.
So as x goes to 0, this is going to behave the same way as differentiating numerator and denominator.
So again this is going to be - sin x in the numerator.
In the denominator, it's going to be 2x.
Now, notice that we still haven't won yet.
Because this is still of 0 / 0 type.
When you plug in x = 0 you still get 0.
But that doesn't damage the method.
That doesn't make the method fail.
This 0 / 0, we can apply L'Hopital's Rule a second time.
And as x goes to 0 this is the same thing as, again, differentiating the numerator and denominator.
So here I get - cos x in the numerator, and I get 2 in the denominator.
Again this is way easier than differentiating ratios of functions.
We're only differentiating the numerator and the denominator separately.
And now this is the end.
As x goes to 0, this is - cos 0 / 2, which is - 1/2.
Now, the justification for this comes only when you win in the end and get the limit.
Because what the theorem says is that if one of these limits exists, then the preceding one exists.
And once the preceding one exists, then the one before it exists.
So once we know that this one exists, that works backwards.
It applies to the preceding limit, which then applies to the very first one.
And the logical structure here is a little subtle, which is that if the right side exists, then the left side will also exist.
Yeah, question
[INAUDIBLE]
Why does the right-hand limit have to exist, isn't it just the derivative that has to exist?
No.
The derivative of the numerator has to exist.
The derivative of the denominator has to exist.
And this limit has to exist.
What doesn't have to exist, by the way, I never said that f prime of a has to exist.
In fact, it's much, much more subtle.
I'm not claiming that f ' (a) exists, because in order to evaluate this limit, f ' (a) need not exist.
What has to happen is that nearby, for x not equal to a, these things exist.
And then the limit has to exist.
So there's no requirements that the limits exist.
In fact, that's exactly going to be the point when we evaluate these limits here.
Is we don't have to evaluate it right at the end
[INAUDIBLE]
So the question that you're asking is, why is this the hypothesis of the theorem?
In other words, why does this work?
Well, the answer is that this is a theorem that's true.
If you drop this hypothesis, it's totally false.
And if you don't have this hypothesis, you can't use the theorem and you will get the wrong answer.
I mean, it's hard to express it any further than that.
So look, in many cases we tell you formulas.
And in many cases it's so obvious when they're true that we don't have to worry about what we say.
And indeed, there's something implicit here.
I'm saying well, you know, if I wrote this symbol down, it must mean that the thing exists.
So that's a subtle point.
But what I'm emphasizing is that you don't need to know in advance that this one exists.
You do need to know in advance that that one exists.
Essentially, yeah.
So that's the direction that it goes.
You can't get away with not having this exist and still have the statement be true.
Alright, another question.
Thank you
[INAUDIBLE]
So I'm getting a little ahead of myself, but let me just say.
In these situations here, when x is going to 0 and x is going to infinity.
For instance, here when x goes to 0, the logarithm is undefined at x = 0.
Nevertheless, this theorem applies.
And we'll be able to use it.
Over here, as x goes to infinity, neither of these -- well, actually, come to think of it, e^ -x, if you like, it's equal to 0 at infinity.
If you want to say that it has a value.
But in fact, these expressions don't necessarily have values.
At the ends.
And nevertheless, the theorem applies.
I mean, it can exist.
It's perfectly OK for it to exist.
It's no problem.
It just doesn't need to exists.
It isn't forced to exist.
So here's a calculation which we just did.
And we evaluated this.
Now, I want to make a comparison with the method of approximation.
In the method of approximations, this Example 2, which was the example with the sine function, we would use the following property.
We would use sin u is approximately u.
We would use that linear approximation.
And then what we would have here is that sin 5x / sin 2x is approximately 5x / 2x, which is of course 5/2.
And this is true when u is approximately 0, and this is true certainly as x goes to 0, it's going to be a valid limit.
So that's very similar to Example 2.
In Example 3, we managed to look at this expression cos x - 1 / x ^2.
And for this one, you have to remember the approximation near x = 0 to the cosine function.
And that's 1 - x ^2 / 2.
So that was the approximation, the quadratic approximation to the cosine function.
And now, sure enough, this simplifies.
This becomes - x ^2 / 2 / x ^2, which is - 1/2.
So we get the same answer, which is a good thing.
Because both of these methods are valid.
They're consistent.
You can see that neither of them is particularly a lot longer.
You may have trouble remembering this property.
But in fact it's something that you can easily derive.
And, indeed, it's related to the second derivative of the cosine, as is this calculation here.
They're almost the same amount of numerical content to them.
So now what I'd like to do is explain to you why L'Hopital's Rule works better in some cases.
And the real value that it has is in handling these other more exotic limits.
So now we're going to do L'Hopital's Rule over again.
And I'll handle these functions.
But I'll have to rewrite them, but we'll just do that.
So here's the property.
That the limit as x goes to a of f ( x) / g (x) is equal to the limit as x goes to a of f ' ( x) / g '(x).
That's the property.
And this is what we'll always be using.
Very convenient thing.
And remember it was true provided that f ( a) = g (a) = 0.
And that the right-hand side exists.
But I claim that it works better, and I'll get rid of these.
But I'll write them again to show you that it works for these.
So there are other cases.
And the other cases that are allowed are this.
First of all, as indicated by what I just erased, you can allow a to be equal to plus or minus infinity.
It's also OK.
So you can take the limit going to the far ends of the universe.
Both left and right.
And then the other thing that you can do is, you can allow f ( a) and g (a) to be plus or minus infinity.
Is OK.
So now, the point is that we can handle not just the 0 / 0 case, but also the infinity / infinity case.
That's a very powerful tool, and quite different from the other cases.
And the third thing is that the right-hand side doesn't really quite have to exist, in the ordinary sense.
Or, it could be plus or minus infinity.
That's also OK. That's still information.
So if we can see where it goes, then we're still good.
If it goes to plus infinity, if it goes to 0, if it goes to a finite number, if it goes to minus infinity, all of that will be OK.
It just if it oscillates wildly that we'll be lost.
And those calculations we'll never encounter.
So this basically handles everything that you could possibly hope for.
And it's a very convenient process.
So let me carry out a few examples.
And, let's see, I guess the first one that I wanted to do was x ln x.
So what example are we up to.
Example 3, so Example 4 is coming up.
Example 4, this is one of the ones that I wrote at the beginning of the lecture, x ln x.
This one was on our homework problem.
In the limits of some calculation.
But so this one, you have to look at it first to think about what it's doing.
It's an indeterminate form, but it sort of looks like it's the wrong type.
So why is it in an indeterminate form.
This one goes to 0, and this one goes to minus infinity.
So, excuse me, this is a product.
It's 0 times minus infinity.
So that's an indeterminate form, because we don't know whether the 0 wins or the infinity this could keep getting smaller and smaller and smaller, and this could be getting bigger and bigger bigger.
The product could be anything in between.
We just don't know.
So the first step is to write this as a ratio of things, rather than a product of things.
And it turns out that the way to do that is to use the logarithm in the numerator, and the 1 / x in the denominator.
So this is a choice that I'm making here.
Now, I've just converted it to a limit of the type minus infinity divided by infinity.
Because the numerator is going to minus infinity as x goes to 0 plus and the denominator 1 / x is going to plus infinity.
Again, there's a competitions, but now it's one of the forms to which L'Hopital's Rule applies.
Now I'm just going to apply L'Hopital's Rule.
And what it says is that I differentiate here.
So I just differentiate a numerator and denominator.
Applying L'Hopital's Rule is a breeze.
You just differentiate, differentiate.
And now it just simplifies and we're done.
This is the limit as x goes to 0 plus of, well, the x^2's cancel.
This is the same as just - x. x factors cancel.
And so that's 0.
The answer is that it's 0.
So x goes to 0 faster then ln n goes to minus infinity.
This 0 was the winner.
Something you can't necessarily predict in advance.
So let's do the other two examples that I wrote down.
I'm going to do them in slightly more generality, because they're the most fundamental rate properties that you're going to need to know for the next section.
Which is improper integrals.
And also they're just very important for physical math, and any other kind of thing, basically.
So here, let's just do these.
So let's see, which one do I want to do first.
So I wrote down the limit of x e^2 - x, but I'm going to make it even more general.
I'm going to make it any negative power here, where p is some positive constant.
Now again, this is a product of functions, not a quotient, a ratio, of functions.
And so I need to rewrite it.
I'm going to write it as x / e ^ p x.
And now I'm going to apply, well, so it's of this form infinity / infinity.
And now that's the same as the limit as x goes to infinity of 1 / p e^ px.
So where does that go?
As x goes to infinity.
Now we can decide.
The 1 stays where it is.
And this, as x goes to infinity, goes to infinity.
So the answer is 0.
And the conclusion is that x grows more slowly then e ^ px.
As x goes to infinity.
Remember, p is positive here, of course.
It's the increasing exponentials.
Not the decreasing ones.
Let's do a variant of this.
I'll do it the opposite way.
So I'm going to call this example 5 '.
It really doesn't give us any more information, but it gives you just a little bit more practice.
So suppose I look at things the other way.
e^ px divided by, say, x^ 100.
Now, this is an infinity / infinity example, again.
And you can work out what it's doing.
But there are two ways of thinking about this.
There's the slow way and the fast way.
The slow way is to differentiate this 100 times.
That is, right?
Apply L'Hopital's Rule over and over and over and over again.
All the way.
It's clear that you could do it, but it's kind of a nuisance.
So there's a much cleverer trick here.
Which is to change this to (the limit as x goes to infinity of the e ^ px / 100 / x) ^ 100.
So if you do that, then we just have one L'Hopital's Rule step here.
And that one is that this is the same as (x goes to infinity of, well it's p / 100 e^ p x / 100 / 1) ^ 100.
That's our L'Hopital's step.
And of course, that's (infinity / 1 ) ^ 100.
Which is infinity.
Now, again I did this in a slightly different way to show you that it works with infinity as well.
So that was this other case.
The right-hand side can exist, or it can be plus or minus infinity.
And that applies to this limit.
And therefore, to the original limit.
And the conclusion here is that e ^ px, p > 0, grows faster than any power of x. I picked x ^ 100, but obviously it didn't matter what power I picked.
The exponents beat all the powers.
So we have one more of the ones that I gave at the beginning to take care of.
And that one is the logarithm.
And its behavior at infinity.
So I'll do a slightly variant on that one, too.
So we have Example 6, which is ln x, and instead of dividing by x, I'm going to divide by x^ 1/3.
I could divide by any positive power of x, we'll just do this example here.
So now this, as x goes to infinity, is of the form infinity / infinity.
And so it's equivalent to what happens when I differentiate numerator and denominator separately.
And that's 1 / x, and here I have 1/3 x ^ - 2/3.
1 / x, and then 1/3 x^ - 2/3.
Now, when the dust settles here and you get your exponents right, we have an x^ - 1, and this is an x ^ + 2/3, and that's a 1/3 becomes a 3.
So this is what it is.
And that's equal to 3x ^ - 1/3.
Which we can decide.
It goes to 0.
As x goes to infinity.
And so the conclusion is that ln x grows more slowly as x goes to infinity, than x ^ 1/3 or any positive power of x.
So any x ^ p, p positive, will work.
So ln is really slow, going to infinity.
It's very, very gradual.
Yeah, question
[INAUDIBLE]
The question is, how many hypotheses do you need here?
So I said that, and I think what you were asking is, if I have this hypothesis, can I also have this hypothesis.
That's OK.
I can have this hypothesis combined with this one.
I need something about f (a) and g ( a).
I can't assume nothing about f(a) and g(a).
So in other words, I have to be faced with either an infinity / infinity, or a 0 / 0 situation.
So let's see.
A rule applies in the 0 / 0, or infinity / infinity case.
These are the only two cases that it applies in.
And a can be anything.
Including infinity.
Plus or minus infinity.
The rule applies in these two cases.
So in other words, this is what f ( a) / g ( a) is.
Either one of these.
And in fact, it can be plus or minus
[INAUDIBLE]
And the right-hand side has to be something.
It has to be either finite or plus or minus infinity.
So you need something.
You need a specific value of a, you need to decide whether it's an indeterminate form.
And you need the right-hand limit to exist.
It's not hard to impose this.
Because when you look at the right-hand side, you'll want to be calculating it.
So you want to know what it is.
So you'll never have problems confirming this hypothesis.
Alright.
Let me give you one more example here.
Which is just slightly trickier.
Which involves, so here's another indeterminate form.
That's going to be 0 ^ 0.
So there are lots of these things where you just don't know what to do.
And they come out in various different ways.
The simplest example of this is the limit as x goes to 0 from above of x ^ x.
In order to work out what's happening with this one, we have to use a trick.
And the trick is this is a moving exponent.
And so it's appropriate to use base e. This is something that we did way back in the first unit.
So, since we have a moving exponent, we're going to use base e. That's the good base to use whenever you have a moving exponent.
And so rewrite this as x^ x = e ^ x ln x.
And now, in order to figure out what's happening, we really only have to know what's going on with the exponent.
So remember, actually we already did this.
But I'm going to do it once more for you.
This is ln x / (1 / x).
And that's equivalent, as x goes to 0, to using L'Hopital's Rule to 1 / x, and this is - 1 / x ^2, which is - x, which goes to 0.
As x goes to 0.
And so what we have here is that this one is going to be equivalent to, well, it's going to tend to what we got over here.
It's e ^ 0.
That exponent is what we want.
As x goes to 0.
So that's the answer This limit happens to be 1.
That's actually relatively easy to do, given all of the power that we have at our hands.
Now, let me give you one more example.
Suppose you're trying to understand the limit of sin x / x ^2.
If you apply L'Hopital's Rule, as x goes to 0, you're going to get cos x / 2x.
And if you apply L'Hopital's Rule again, as x goes to 0, you're going to get the - sin x / 2.
And this, as x goes to 0, goes to 0.
On the other hand, if you look at the linear approximation method, linear approximation says that sin x is approximately x near 0.
So that should be x / x ^2.
Which is 1 / x, which goes to infinity.
As x goes to 0, at least from one side, minus infinity to the other side.
So there's something fishy going on here, right?
So this is fishy.
Or maybe this is fishy, I don't know.
So, tell me what's wrong here.
Yeah
[INAUDIBLE]
OK.
So the claim is that the second application of L'Hopital's Rule, this one, is wrong.
And that's correct.
And this is where you have to watch out, with L'Hopital's Rule.
This is exactly where you have to watch out.
You have to apply the test.
Here it's an indeterminate form.
It's 0 / 0 before I applied the rule.
But in order to apply the rule the second time, it still has to be 0 / 0.
But this one isn't.
This one is 1 / 0.
It's no longer an indeterminate form.
It's actually infinite.
Either plus or minus, depending on the sign of the denominator.
Which is just what this answer is.
So the linear approximation is safe.
And we just applied L'Hopital's Rule wrong.
So the moral of the story here is look before you L'Hop.
Alright.
Now, let me say one more thing.
I need to pile it on just a little bit, sorry.
So don't use it as a crutch.
We don't want to just get ourselves so weak, after being in the hospital for all this time, that we can't use, I'm sorry.
So remember that you shouldn't have lost your senses.
If you have something like this, so we'll do this one here.
Suppose you're trying to understand what this does as x goes to infinity.
Now, you could L'Hopital's Rule five times, or four times.
And get the answer here.
But really, you should realize that the main terms are sitting there right in front of you.
And that there's some algebra that you can do to simplify this.
Namely, it's the same as 1 + 2 / x + 1 / x^ 5.
And then in the denominator, well, let's see.
It's x.
So this would be dividing by 1 / x^ 5 in both numerator and denominator.
And here you have 1 / x + 2 over, sorry I overshot.
But that's OK. 2 / x^ 5 here.
So these are the main term, if you like.
And it's the same as 1 / 1 / x, which is the same as x, and it goes to infinity.
As x goes to infinity.
Or, if you like, much more simply, just x ^ 5 / x^ 4 is the main term.
Which is x.
Which goes to infinity.
So don't forget your basic algebra when you're doing this kind of stuff.
Use these things and don't use L'Hopital's Rule.
OK, see you next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Now, today we are continuing with this last unit.
Unit 5, continued.
The informal title of this unit is Dealing With Infinity.
That's really the extra little piece that we're putting in to our discussions of things like limits and integrals.
To start out with today, I'd like to recall for you, L'Hopital's Rule.
And in keeping with the spirit here, we're just going to do the infinity / infinity case.
I stated this a little differently last time, and I want to state it again today.
Just to make clear what the hypotheses are and what the conclusion is.
We start out with, really, three hypotheses.
Two of them are kind of obvious.
The three hypotheses are that f (x) tends to infinity, g ( x) tends to infinity, that's what it means to be in this infinity / infinity case.
And then the last assumption is that f ' ( x) / g ' (x), tends to a limit, L. And this is all as x tends to some a.
Some limit a.
And then the conclusion is that f( x) / g ( x) also tends to L, as x goes to a.
Now, so that's the way it is.
So it's three limits.
But presumably these are obvious, and this one is exactly what we were going to check anyway.
Gives us this one limit.
So that's the statement.
And then the other little interesting point here, which is consistent with this idea of dealing with infinity, is that a equals plus or minus infinity and L equals plus or minus infinity are OK. That is, the numbers capital L, the limit capital L and the number a can also be infinite.
Now in recitation yesterday, you should have discussed something about rates of growth, which follow from what I said in lecture last time and also maybe from some more detailed discussions that you had in recitation.
And I'm going to introduce a notation to compare functions.
Namely, we say that f ( x) is a lot less than g ( x).
So this means that the limit, as it goes to infinity, this tends to 0.
As x goes to infinity, this would be.
So this is a notation, a new notation for us.
f is a lot less than g. And it's meant to be read only asymptotically.
It's only in the limit as x goes to infinity that this happens.
And implicitly here, I'm always assuming that these are positive quantities.
f and g are positive.
What you saw in recitation was that you can make a systematic comparison of all the standard functions that we know about.
For example, the ln function goes to infinity.
But a lot more slowly than x to a power.
A lot more slowly then e^ x.
A lot more slowly than, say, e ^ x ^2.
So this one is slow.
This one is moderate.
And this one is fast.
And this one is very fast.
Going to infinity.
Tends to infinity, and this is of course as x goes to infinity.
All of them go to infinity, but at quite different rates.
And, analogous to this, and today we're going to be doing this, needing to do this quite a bit, is rates of decay, which are more or less the opposite of rates of growth.
So rates of decay are rates at which things tend to 0.
So the rate of decay, and for that I'm just going to take reciprocals of these numbers.
So 1 / ln x tends to 0.
But rather slowly.
It's much bigger than 1 / x ^ p. Oh, I didn't mention that this exponent p is meant to be positive.
That's a convention that I'm using without saying.
I should've told you that.
So think x ^ 1/2, x ^ 1, x ^2, they're all in this sort of moderate intermediate range.
And then that, in turn, goes to 0 but much more slowly then 1 / e ^ x, also known as e^ - x.
And that, in turn, this guy here goes to 0 incredibly fast.
e ^ - x ^2 vanishes really, really fast.
So this is a review of the L'Hopital's Rule.
What we said last time, and the application of it, which is to rates of growth and tells us what these rates of growth are.
Today, I want to talk about improper integrals.
And improper integrals, we've already really seen one or two of them on your exercises.
And we mention them a little bit, briefly.
I'm just going to go through them more carefully and more systematically now.
And we want to get just exactly what's going on with these rates of decay and their relationship with improper integrals.
So I need for you to understand on the spectrum of the range of functions like this, which ones are suitable for integration as x goes to infinity.
Well, let's start out with the definition.
The integral from a to infinity of f(x) dx is, by definition the limit as n goes to infinity of the ordinary definite integral up to some fixed, finite level.
That's the definition.
And there's a word that we use here, which is that we say the integral, so this is terminology for it, converges if the limit exists.
And diverges if not.
Well, these are the key words for today.
So here's the issue that we're going to be addressing.
Which is whether the limit exists or not.
In other words, whether the integral converges or diverges.
These notions have a geometric analog, which you should always be thinking of at the same time in the back of your head.
I'll draw a picture of the function Here it's starting out at a.
And maybe it's going down like this.
And it's interpreting it geometrically.
This would only work if f is positive.
Then the convergent case is the case where the area is finite.
So the total area is finite under this curve.
And the other case is the total area is infinite.
I claim that both of these things are possible.
Although this thing goes on forever, if you stop it at one stage, n, then of course it's a finite number.
But as you go further and further and further, there's more and more and more area.
And there are two possibilities.
Either as you go all the way out here to infinity, the total that you get adds up to a finite total.
Or else, maybe there's infinitely much.
For instance, if it's a straight line going across, there's clearly infinitely much area underneath.
So we need to do a bunch of examples.
And that's really our main job for the day, and to make sure that we know exactly what to expect in all cases.
The first example is the integral from 0 to infinity of e^ - kx dx.
Where k is going to be some positive number.
Some positive constant.
This is the most fundamental, by far, of the definite integrals.
Improper integrals.
And in order to handle this, the thing that I need to do is to check the integral from 0 up to n. e ^ - kx dx.
And since this is an easy integral to evaluate, we're going to do it.
It's - 1 / k e ^ - kx, that's the antiderivative.
Evaluated at 0 and n. And that, if I plug in these values, is - 1 / k, e^ - k N. Minus, and if I evaluate it at 0, I get a (- 1 / k) e^ 0.
So there's the answer.
And now we have to think about what happens as n goes to infinity.
So as n goes to infinity, what's happening is the second term here stays unchanged.
But the first term is e to some negative power.
And the exponent is getting larger and larger.
That's because k is positive here.
You've definitely got to pay attention.
Even though I'm doing this with general variables here, you've got to pay attention to signs of things.
Because otherwise you'll always get the wrong answer.
So you have to pay very close attention here.
So this is, if you like, e ^ minus infinity in the limit, which is 0.
And so in the limit, this thing tends to 0.
And this thing is just equal to 1 / k. And so all told, the answer is 1 / k And that's it.
Now we're going to abbreviate this a little bit.
This thought process, you're going to have to go through every single time you do this.
But after a while you also get good enough at it that you can make it a little bit less cluttered.
So let me show you a shorthand for this same calculation.
Namely, I write 0 to infinity e ^ - kx dx.
And that's equal to - 1 / k e ^ - kx 0 to infinity.
That was cute.
Not small enough, however.
So, here we are.
We have the same calculation as we had before.
But now we're thinking, really, in our minds that this infinity is some very, very enormous number.
And we're going to plug it in.
And you can either do this in your head or not.
You say - 1 / k e^ - infinity.
Here's where I've used the fact that k is positive.
Because e ^ - k times a large number is minus infinity.
And then here + 1 / k - (- 1 / k).
Let me write it the same way I did before.
And that's just equal to 0 + 1 / k, which is what we want.
So this is the same calculation, just slightly abbreviated.
Yeah.
Question
[INAUDIBLE]
Good question.
The question is, what about the case when the limit is infinity?
I'm distinguishing between something existing and its limit being infinity here.
Whenever I make a discussion of limits, I say a finite limit, or in this case, it works for infinite limits.
So in other words, when I say exists, I mean exists and is finite.
So here, when I say that it converges and I say the limit exists, what I mean is that it's a finite number.
And so that's indeed what I said here.
The total area is finite.
And, similarly, over here.
I might add, however, that there is another part of this subject.
Which I'm skipping entirely.
Which is a little bit subtle.
Which is the following.
If f changes sign, there can be some cancellation and oscillation.
And then sometimes the limit exists, but the total area, if you counted it all positively, is actually still infinite.
And we're going to avoid that case.
We're we're just going to treat these positive cases.
So don't worry about that for now.
That's the next layer of complexity which we're not addressing in this class.
Another question
[INAUDIBLE]
The question is, would this be OK on tests.
The answer is, absolutely yes.
I want to encourage you to do this.
If you can think about it correctly.
The subtle point is just, you have to plug in infinity correctly.
Namely, you have to realize that this only works if k is positive.
This is the step where you're plugging in infinity.
And I'm letting you put this infinity up here as an endpoint value.
So in fact that's exactly the theme.
The theme is dealing with infinity here.
And I want you to be able to deal with it.
That's my goal
[INAUDIBLE]
OK, so another question is, so let's be sure here when the limit exists, I say it has to be finite.
That means it's finite, not infinite.
The limit can be 0.
It can also be - 1.
It can be anything.
Doesn't have to be a positive number.
Other questions.
So we've had our first example.
And now I just want to add one physical interpretation here.
This is Example 1, if you like.
And this is something that was on your problem set, remember.
That we talked about the probability, or the number, if you like, the number of particles on average that decay in some radioactive substance.
Say, in time between 0 and some capital T. And then that would be this integral, 0 to capital T, some total quantity times this integral here.
This is the typical kind of radioactive decay number that one gets.
Now, in the limit, so this is some number of particles.
If the substance is radioactive, then in the limit, we have this.
Which is equal to the total number of particles.
And that's something that's going to be important for normalizing and understanding.
How much does the whole substance, how many moles do we have of this stuff.
What is it.
And so this is a number that is going to come up.
Now, I emphasize that this notion of T going to infinity is just an idealization.
We don't really believe that we're going to wait forever for this substance to decay.
Nevertheless, as theorists, we write down this quantity.
And we use it.
All the time.
Furthermore, there's other good reasons for using it, and why physicists accept it immediately.
Even though it's not really completely physically realistic ever to let time go very, very far into the future.
And the reason is, if you notice this answer here, look at how much simpler this number is, 1 / k, than the numbers that I got in the intermediate stages here.
These are all ugly, the limits are simple.
And this is a theme that I've been trying to emphasize all semester.
Namely, that the infinitesimal, the things that you get when you do differentiation, are the easier formulas.
The algebraic ones, the things in the process of getting to the limit, are the ugly ones.
These are the easy ones, these are the hard ones.
So in fact, infinity is basically easier than any finite number.
And a lot of appealing formulas come from those kinds of calculations.
Another question
[INAUDIBLE]
The question is, shouldn't the answer be a?
Well, the answer turns out to be a / k. Which means that when you set up your arithmetic, and you model this to a collection of particles.
So you said it should be a.
But that's because you made an assumption.
Which was that a was the total number of particles.
But that's just false, right?
This is the total number of particles.
So therefore, if you want to set it up, you want set up so that this number's the total number of particles.
And that's how you set up a model is, you do all the calculations and you see what it's coming out to be.
And that's why you need to do this kind of calculation.
OK, so.
The main thing is, you shouldn't make assumptions about models.
You have to follow what the calculations tell you.
They're not lying.
OK, so now.
We carried this out.
There's one other example which we talked about earlier in the class.
And I just wanted to mention it again.
It's probably the most famous after this one.
Namely, the integral from minus infinity to infinity of e^ - x ^2 dx.
Which turns out, amazingly, to be able to be evaluated.
It turns out to be the square root of pi.
So this one is also great.
This is the constant which allows you to compute all kinds of things in probability.
So this is a key number in probability.
It basically is the key to understanding things like standard deviation and basically any other thing in the subject of probability.
It's also what's driving these polls that tell you within 4% accuracy we know that people are going to vote this way or that.
So in order to interpret all of those kinds of things, you need to know this number.
And this number was only calculated numerically starting in the 1700s or so by people who, actually, by one guy whose name was de Moivre, who was selling his services to various royalty who were running lotteries.
In those days they ran lotteries, too.
And he was able to tell them what the chances were of the various games.
And he worked out this number.
He realized that this was the pattern.
Although he didn't know that it was the square root of pi, he knew it to sufficient accuracy that he could tell them the correct answer to how much money their lotteries would make.
And of course we do this nowadays, too.
In all kinds of ways.
Including slightly more legit businesses like insurance.
So now, I I'm going to give you some more examples.
And and the other examples are much more close to the edge between infinite and finite.
This distinction between convergence and divergence.
And let me just, maybe I'll say one more word about why we care about this very gross issue of whether something is finite or infinite.
When you're talking about something like this normal curve here, there's an issue of how far out you have to go before you can ignore the rest.
So we're going to ignore what's called the tail here.
Somehow you want to know that this is negligible.
And you want to know how negligible it is.
And this is the job of a mathematician, is to know what finite region you have to consider and which one you're going to carefully calculate numerically.
And then the rest, you're going to have to take care of by some theoretical reasoning.
You're going to have to know that these tails are small enough that they don't matter in your finite calculation.
And so, we care very much about the tails.
Because they're the only thing that the machine won't tell us.
So that's the part that we have to know.
And these tails are also something which are discussed all the time in financial mathematics.
They're very worried about fat tails.
That is, unlikely events that nevertheless happen sometimes.
And they get burned fairly regularly with them.
As they have recently, with the mortgage scandal.
So, these things are pretty serious and they really are spending a lot of time on them.
Of course, there are lots of other practical issues besides just the mathematics.
But you've got to get the math right, too.
So we're going to now talk about some borderline cases for these fat tails.
Just how fat do they have to be before they become infinite and overwhelm the central bump.
So we'll save this for just a second.
And what I'm saving up here is the borderline case, which I'm going to concentrate on, which is this moderate rate, which is x to powers.
Here's our next example.
I guess we'll call this Example 3.
It's the integral from 1 to infinity dx / x.
That's the power p = 1.
And this turns out to be a borderline case.
So it's worth carrying out carefully.
Now, again I'm going to do it by the slower method.
Rather than the shorthand method.
But ultimately, you can do it by the short method if you'd like.
I break it up into an integral that goes up to some large number, n. I see that its logarithm function is the antiderivative.
And so what I get is ln n - ln 1, which is just 0.
So this is just log n. In any case, it tends to infinity as n goes to infinity.
So the conclusion is, since the limit is infinite, that this thing diverges.
Now, I'm going to do this systematically now with all powers p, to see what happens.
I'll look at the integral.
Sorry, I'm going to have to start at 1 here.
From 1 to infinity, dx / x ^p, and see what happens with these.
And you'll see that p = 1 is a borderline when I do this calculation.
This time I'm going to do the calculation the hard way.
But now you're going to have to think and pay attention to see what it is that I'm doing.
First of all, I'm going to take the antiderivative.
And this is x ^ - p, so it's - p + 1 / - p + 1.
That's the antiderivative of the function 1 / x ^ - p. And then I have to evaluate that at 1 and infinity.
So now, I'll write this down.
But I'm going to be particularly careful here.
I'll write it down.
It's infinity to the - p + 1 / - p + 1 -, so I plug in 1 here.
So I get 1 / - p + 1.
So this is what I'm getting.
Again, what you should be thinking here is this is a very large number to this power.
Now, there are two cases.
There are two cases.
And they exactly split at p = 1.
When p = 1, this number is 0.
This exponent is 0, and in fact this expression doesn't make any sense because the denominator is also 0.
But for all of the other values, the denominator makes sense.
But what's going on is that this is infinite when this exponent is infinity to a positive power.
And it's 0 when it's infinity to a negative power.
So I'm going to say it here, and you must check this at home.
Because this is exactly what I'm going to ask you about on the exam.
This is it.
This type of thing, maybe with a specific value of p here.
When p < 1, this thing is infinite.
On the other hand, when p > 1, this thing is 0.
So when p > 1, this thing is 0.
It's just equal to 0.
And so the answer is 1 / p - 1.
Because that's this number.
Minus the quantity 1 / - p + 1.
This is a finite number here.
Notice that the answer would be weird if this thing went away in the p < 1 case.
Then it would be a negative number.
It would be a very strange answer to this question.
So, in fact that's not what happens.
What happens is that the answer doesn't make sense.
It's infinite.
So let me just write this down again, under here.
This is a test in a particular case.
And here's the conclusion.
Ah.
No, I'm sorry.
I think I was going to write it over on this board here.
So the conclusion is that the integral from 1 to infinity dx / x^p diverges if p <= 1.
And converges if p > 1.
And in fact, we can actually evaluate it.
It's equal to 1 / p - 1.
It's got a nice, clean formula even.
Alright, now let me remind you.
So I didn't spell the word diverges right, did I?
Oh no, that's an r. I guess that's right.
Diverges if p <= 1.
So really, I needed both of these arguments, which are sitting above it in order to do it.
Because the second argument didn't work at all when p = 1 because the formula for the antiderivative is wrong.
The formula for the antiderivative is given by the ln function when p = 1.
So I had to do this calculation too.
This is the borderline case, between p > 1 and p < 1.
When p > 1, we got convergence.
We could calculate the integral.
When p < 1, when we got divergence and we calculated the integral over there.
And here in the borderline case, we got a logarithm.
and we also got divergence.
So it failed at the edge.
Now, this takes care of all the powers.
Now, there are a number of different things that one can deduce from this.
And let me carry them out.
So this is more or less the second thing that you'll want to do.
And I'm going to emphasize maybe one aspect of it.
I guess we'll get rid of this.
But it's still the issue that we're discussing here.
Is whether this area is fat or thin.
I'll remind you of that.
So here's the next idea.
Something called limit comparison.
Limit comparison is what you're going to use when, instead of being able actually to calculate the number, you don't yet know what the number is.
But you can make a comparison to something whose convergence properties you already understand.
Now, here's the statement.
If a function, f, is similar to a function, asymptotically the same as a function, g, as x goes to infinity, I'll remind you what that means in a second.
Then the integral starting at some point out to infinity of f(x) dx, and the other one, converge and diverge at the same time.
So both, either, either -- sorry, let's try it the other way.
Either, both.
Either both converge, or both diverge.
They behave exactly the same way.
In terms of whether they're infinite or not.
And, let me remind you what this tilde means.
This thing means that f( x) / g ( x) tends to 1.
So if you have a couple of functions like that, then their behavior is the same.
This is more or less obvious.
It's just because far enough out, this is for large a, if you like.
We're not paying any attention to what happens.
It just has to do with the tail, and after a while f ( x) and g(x) are comparable to each other.
So their integrals are comparable to each other.
So let's just do a couple of examples here.
If you take the integral from 0 to infinity dx / the square root of x ^2 + 10, then I claim that the square root of x^2 + 10 resembles the square root of x ^2, which is just x.
So this thing is going to be like.
So now I'm going to have to do one thing to you here.
Which is, I'm going to change this to 1.
To infinity.
dx /x And the reason is that this x = 0 is extraneous.
Doesn't have anything to do with what's going on with this problem.
This guy here, the piece of it from, so we're going to ignore the part integral from 0 to 1 dx / square root of x ^2 + 10, which is finite anyway.
And unimportant.
Whereas, unfortunately, the integral of dx will have a singularity at x = 0.
So we can't make the comparison there.
Anyway, this one is infinite.
So this is divergence.
Using what I knew from before.
Yeah
[INAUDIBLE]
The question is, why did we switch from 0 to 1?
So I'm going to say a little bit more about that later.
But let me just make it a warning here.
Which is that this guy here is infinite for other reasons.
Unrelated reasons.
The comparison that we are trying to make is with the tail as x goes to infinity.
So another way of saying this is that I should stick an a here and an a here and stay away from 0.
So, say a = 1.
If I make these both 1, that would be OK.
If I make them both 2, that would be OK.
If I make them both 100, that would be OK.
So let's leave it as 100 right now.
And it's acceptable.
I want you to stay away from the origin here.
Because that's another bad point.
And just talk about what's happening with the tail.
So this is a tail, and I also had a different name for it up top.
Which is emphasizing this.
Which is limit comparison.
It's only what's happening at the very end of the picture that we're interested in.
So again, this is as x goes to infinity.
That's the limit we're talking about, the limiting behavior.
And we're trying not to pay attention to what's happening for small values of x.
So to be consistent, if I'm going to do it up to 100 I'm ignoring what's happening up to the first 100 values.
In any case, this guy diverged.
And let me give you another example.
This one, you could have computed.
This one you could have computed, right?
Because it's a square root of quadratic, so there's a trig substitution that evaluates this one.
The advantage of this limit comparison method is, it makes no difference whether you can compute the thing or not.
You can still decide whether it's finite or infinite, fairly easily.
So let me give you an example of that.
So here we have another example.
We'll take the integral dx / square root of x^3 + 3.
Let's say, for the sake of argument.
From 0 to infinity.
Let's leave off, let's make it 10 to infinity, whatever.
Now this one is problematic for you.
You're not going to be able to evaluate it, I promise.
So on the other hand 1 / the square root of x^3 + 3 is similar to 1 / the square root of x ^3, which is 1 / x ^ 3/2.
So this thing is going to resemble this integral here.
Which is convergent.
According to our rule.
So those are the, more or less the main ingredients.
Let me just mention one other integral, which was the one that we had over here.
This one here.
If you look at this integral, of course we can compute it so we know the area is finite.
But the way that you would actually carry this out, if you didn't know the number and you wanted to check that this integral were finite, then you would make the following comparison.
This one is not so difficult.
First of all, you would write it as twice the integral from 0 to infinity of e^ - x ^2 dx.
This is a new example here, and we're just checking for convergence only.
Not evaluation.
And now, I'm going to make a comparison here, Rather than a limit, comparison I'm actually just going to make an ordinary comparison.
That's because this thing vanishes so fast.
It's so favorable that we can only put something on top of it, we can't get something underneath it that exactly balances with it.
In other words, this wiggle was something which had the same growth rate as the function involved.
This thing just vanishes incredibly fast.
It's great.
It's too good for us, for this comparison.
So instead what I'm going to make is the following comparison.
e ^ - x ^2 < = e ^ - x.
At least for x >= 1.
When x > = 1, then x ^2 >= x, and so - x ^2 < - x.
And so e^ - x^2 is less than this.
So this is the reasoning involved.
And so what we have here is two pieces.
We have 2, the integral from 0 to 1, of e^ - x ^2.
That's just a finite part.
And then we have this other part, which I'm going to replace with the e ^ - x here.
2 times 1 to infinity e ^ -x dx.
So this is, if you like, this is ordinary comparison of integrals.
It's something that we did way at the beginning of the class.
Or much earlier on, when we were dealing with integrals.
Which is that if you have a larger integrand, then the integral gets larger.
So we've replaced the integral.
We've got the same integrand on 0 to 1.
And we have a larger integrand on, so this one is larger integrand.
And this one we know is finite.
This one is a convergent integral.
So the whole business is convergent.
But of course we replaced it by a much larger thing.
So we're not getting the right number out of this.
We're just showing that it converges.
So these are the main ingredients.
As I say, once the thing gets really, really fast decaying, it's relatively straightforward.
There's lots of room to show that it converges.
Now, there's one last item of business here which I have to promise you.
Which I promised you, which had to do with dealing with this bottom piece here.
So I have to deal with what happens when there's a singularity.
This is known as an improper integral of the second type.
And the idea of these examples is the following.
You might have something like this.
Something like this.
Or something like this.
These are typical sorts of examples.
And before actually describing what happens, I just want to mention.
So first of all, the key point here is you can just calculate these things.
And plug in 0 and it works and you'll get the right answer.
So you'll determine, you'll figure out, that it turns out that this one will converge, this one will diverge, and this one will diverge.
That's what will turn out to happen.
However, I want to warn you that you can fool yourself.
And so let me give you a slightly different example.
Let's consider this integral here.
The integral from - 1 to 1 dx / x ^2.
If you carry out this integral without thinking, what will happen is, you'll get the antiderivative, which is - x ^ -1, evaluated at - 1 and 1.
And you plug it in.
And what do you get?
You get - 1 (1 ^ - 1) -, uh-oh.
(- ( -1) ^ - 1).
There's a lot of - 1's in this problem.
OK, so that's - 1.
And this one, if you work it all out, as I sometimes don't get the signs right, but this time I really paid attention.
It's - 1, I'm telling you that's what it is.
So that comes out to be - 2.
Now, this is ridiculous.
This function here looks like this.
It's positive, right?
1 / x ^2 is positive.
How exactly is it that the area between - 1 and 1 came out to be a negative number?
That can't be.
There was clearly something wrong with this.
And this is the kind of thing that you'll get regularly if you don't pay attention to convergence of integrals.
So what's going on here is actually that this area in here is infinite.
And this calculation that I made is nonsense.
So it doesn't work.
This is wrong.
Because it's divergent.
Actually, when you get to imaginary numbers, it'll turn out that there's a way of rescuing it.
But, still, it means something totally different when that integral is thought to be at - 2.
So.
What I want you to do here, so I think we'll have to finish this up very briefly next time.
We'll do these three calculations and you'll see that these two guys are divergent and this one converges.
And we'll do that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Today we are continuing with improper integrals.
I still have a little bit more to tell you about them.
What we were discussing at the very end of last time was improper integrals.
Now and these are going to be improper integrals of the second kind.
By second kind I mean that they have a singularity at a finite place.
That would be something like this.
So here's the definition if you like.
Same sort of thing as we did when the singularity was at infinity.
So if you have the integral from 0 to 1 of f of x.
This is going to be the same thing as the limit.
As a goes to 0 from above.
The integral from a to 1 of f of x d x.
And the idea here is the same one that we had at infinity.
Let me draw a picture of it.
You have, imagine a function which is coming down like this and here's the point 1.
And we don't know whether the area enclosed is going to be infinite or finite and so we cut it off at some place a.
And we let a go to 0 from above.
So really it's 0 plus.
So we're coming in from the right here.
And we're counting up the area in this chunk.
And we're seeing as it expands whether it goes to infinity or whether it tends to some finite limit.
Right, so this is the example and this is the definition.
And just as we did for the other kind of improper integral, we say that this converges-- so that's the key word here --if the limit is finite, exists maybe I should just say and diverges if not.
Let's just take care of the basic examples.
First of all I wrote this one down last time.
We're going to evaluate this one.
The integral from 0 to 1 of 1 over the square root of x.
And this just, you almost don't notice the fact that it goes to infinity.
This goes to infinity as x goes to 0.
But if you evaluate it -- first of all we always write this as a power.
Right?
To get the evaluation.
And then I'm not even going to replace the 0 by an a. I'm just going to leave it as 0.
The antiderivative here is x to the 1/2 times 2.
And then I evaluate that at 0 and 1.
And I get 2.
2 minus 0, which is 2.
All right so this one is convergent.
And not only is it convergent but we can evaluate it.
The second example being not systematic but really giving you the principal examples that we'll be thinking about.
Is this one here d x over x And this one gives you the antiderivative as the logarithm.
Evaluated at 0 and 1.
And now again you have to have this thought process in your mind that you're really taking the limit.
But this is going to be the log of 1 minus the log of 0.
Really the log of 0 from above.
There is no such thing as the log of 0 from below.
And this is minus infinity.
So it's 0 minus minus infinity, which is plus infinity.
And so this one diverges.
All right so what's the general?
So more or less in general, let's just, for powers anyway, if you work out this thing for d x over x to the p from 0 to 1.
What you're going to find is that it's 1 over 1 minus p. When p is less than one.
And it diverges for p bigger than or equal to 1.
Now that's the final result.
If you carry out this integration it's not difficult.
All right so now I just want to try to help you to remember this.
And to think about how you should think about it.
So I'm going to say it in a few more ways.
All right just repeat what I've said already but try to get it to percolate and absorb itself.
And in order to do that I have to make the contrast between the kind of improper integral that I was dealing with before.
Which was not as x goes to 0 here but as x goes to infinity, the other side.
Let's make this contrast.
First of all, if I look at the angle that we have been paying attention to right now.
We've just considered things like this.
1 over x to the 1 half.
Which is a lot smaller than one over x.
Which is a lot smaller than say 1 over x squared.
Which would be another example.
This is as x goes to 0.
So this one's the smallest one.
This one's the next smallest one.
And this one is very large.
On the other hand it goes the other way at infinity.
As x tends to infinity.
All right so try to keep that in mind.
And now I'm going to put a little box around the bad guys here.
This one is divergent.
And this one is divergent.
And this one is divergent.
And this one is divergent.
When the cross over point is 1 over x.
When we get smaller than that, we get to things which are convergent.
When we get smaller than it on this other scale, it's convergent.
All right so these guys are divergent.
So they associated with divergent integrals.
The functions themselves are just tending towards-- well these tend to infinity, and these tend toward 0.
So I'm not talking about the functions themselves but the integrals.
Now I want to draw this again here, not small enough.
I want to draw this again.
And, so I'm just going to draw a picture of what it is that I have here.
But I'm going to combine these two pictures.
So here's the picture for example of y equals 1 over x.
All right.
That's y equals 1 over x.
And that picture is very balanced.
It's symmetric on the two ends.
If I cut it in half then what I get here is 2 halves.
And this one has an infinite area.
That corresponds to the integral from 1 to infinity, d x over x being infinite.
And the other piece-- which this one we calculated last time.
This is the one that we just calculated over here at example 2 has the same property.
It's infinite.
And that's the fact that the integral from 0 to 1 of dx over x is infinite.
Right, so both, we lose on both ends.
On the other hand if I take something like-- I'm drawing it the same way but it's really not the same --y equals 1 over the square root of x. y equals 1 over x to the 1 half.
And if I cut that in half here then the x to the 1 half is actually bigger than this guy.
So this piece is infinite.
And this part over here actually is going to give us an honest number.
In fact this one is finite.
And we just checked what the number is.
It actually happens to have area 2.
And what's happening here is if you would superimpose this graph on the other graph what you would see is that they cross.
And this one sits on top.
So if I drew this one in let's have another color here --orange let's say.
If this were orange if I set it on top here it would go this way.
OK and underneath the orange is still infinite.
So both of these are infinite.
On here on the other hand underneath the orange is infinite but underneath where the green is finite.
That's a smaller quantity.
Infinity is a lot bigger than 2.
2 is a lot less than infinity.
All right so that's reflected in these comparisons here.
Now if you like if I want to do these in green.
This guy is good and this guy is good.
Well let me just repeat that idea over here in this sort of reversed picture with y equals 1 over x squared.
If I chop that in half then the good part is this end here.
This is finite.
And the bad part is this part of here which is way more singular.
And it's infinite.
All right so again what I've just tried to do is to give you some geometric sense and also some visceral sense.
This guy it's tail as it goes out to infinity is much lower.
It's much smaller than 1 over x.
And these guys trapped an infinite amount of area.
This one traps only a finite amount of area.
All right so now I'm just going to give one last example which combines these two types of pictures.
It's really practically the same as what I've said before but I-- oh have to erase this one too.
So here's another example if you're in so let's take the following example.
This is somewhat related to the first one that I gave last time.
If you take a function y equals 1 over x minus 3 squared.
And you think about it's integral.
So let's think about the integral from 0 to infinity, d x over x minus 3 squared.
And suppose you were faced with this integral.
In order to understand what it's doing you have to pay attention to two places where it can go wrong.
We're going to split into two pieces.
I'm going say break it up into this one here up to 5, for the sake of argument.
And say from 5 to infinity.
All right.
So these are the two chunks.
Now why did I break it up into those two pieces?
Because what's happening with this function is that it's going up like this at 3.
And so if I look at the two halves here.
I'm going to draw them again and I'm going to illustrate them with the colors we've chosen-- which are I guess red and green --what you'll discover is that this one here, which corresponds to this piece here is infinite.
And it's infinite because there's a square in the denominator.
And as x goes to 3 this is very much like if we shifted the 3 to 0.
Very much like this 1 over x squared here.
But not in this context.
In the other context where it's going to infinity.
This is the same as at the picture directly above with the infinite part in red.
All right.
And this part here, this part is finite.
All right.
So since we have an infinite part plus a finite part the conclusion is that this thing, well this guy converges.
And this one diverges.
But the total unfortunately diverges.
Right because it's got 1 infinity in it.
So this thing diverges.
And that's what happened last time when we got a crazy number.
If you integrated this you would get some negative number.
If you wrote down the formulas carelessly.
And the reason is that the calculation actually is nonsense.
So you've gotta be aware if you encounter a singularity in the middle not to ignore it.
Yeah.
Question
[INAUDIBLE PHRASE]
Why do we say that the whole thing diverges?
The reason why we say that is the area under the whole curve is infinite.
It's the sum of this piece plus this piece.
And so the total is infinite
[INAUDIBLE PHRASE]
We're stuck.
This is an ill defined integral.
It's one where you're red flashing warning sign should be on.
Because you're not going to get the right answer by computing it.
You'll never get an answer.
Similarly you'll never get an answer with this.
And you will get an answer with that.
OK?
Yeah another question
[INAUDIBLE PHRASE]
So the question is if you have a little glance at an integral, how are you going to decide where you should be heading?
So I'm going to answer that orally.
Although you know, but I'll say one little hint here.
So you always have to check x going to infinity and x going to minus infinity, if they're in there.
And you also have to check any singularity, like x going to 3 for sure in this case.
You have to pay attention all the places where the thing is infinite.
And then you want to focus in on each one separately.
And decide what's going on it at that particular place.
When it's a negative power.
So remember d x over x as x goes to 0 is bad.
And d x over x squared is bad.
D x over x cubed is bad.
All of them are even worse.
So anything of this form is bad and equals 1, 2, 3.
These are the red box kinds.
All right.
That means that any of the integrals that we did in partial fractions which had a root, which had a factor of something, the denominator.
Those are all divergent integrals if you cross the singularly.
Not a single one of them makes sense across the singularity.
Right?
If you have square roots and things like that then you can repair things like that.
And there's some interesting examples of that.
Such as with the arc sign function and so forth.
Where you have an improper integral which is really OK. All right.
So that's the best I can do.
It's obviously something you get experience with.
All right.
Now I'm going to move on and this is more or less our last topic.
Yay, but not quite.
Well, so I should say it's our penultimate topic.
Right because we have one more lecture.
All right.
So that our next topic is series.
Now we'll do it in a sort of a concrete way today.
And then we'll do what are known as power series tomorrow.
So let me tell you about series.
Remember we're talking about infinity and dealing with infinity.
So we're not just talking about any old series.
We're talking about infinite series.
There is one infinite series which is probably, which is without question the most important and useful series.
And that's the geometric series but I'm going to introduce it concretely first in a particular case.
If I draw a picture of this sum.
Which in principle goes on forever.
You can see that it goes someplace fairly easily by marking out what's happening on the number line.
The first step takes us to 1 from 0.
And then if I add this half, I get to 3 halves.
Right, so the first step was 1 and the second step was a half.
Now if I add this quarter i, which is the next piece then I get some place here.
But what I want to observe is that I got, I can look at it from the other point of view.
I got, when I move this quarter I got half way to 2 here.
I'm putting 2 in green because I want you to think of it as being the good kind.
Right.
The kind that has a number.
And not one of the red kinds.
We're getting there and we're almost there.
So the next stage we get half way again.
That's the eighth and so forth.
And eventually we get to 2.
So this sum we write equals two.
All right that's kind of a paradox because we never get to 2.
This is the paradox that Zeno fussed with.
And his conclusion, you know, with the rabbit and the hare.
Oh no the rabbit and the tortoise.
Sorry hare chasing-- anyway the rabbit chasing the tortoise.
His conclusion-- you know, I don't know if you're aware of this --but he understood this paradox.
And he said you know it doesn't look like it ever gets there because they're infinitely many times between the time-- you know that the tortoise is always behind, always behind, always behind, always behind.
So therefore it's impossible that the tortoise catches up right.
So do you know what his conclusion was?
Time does not exist.
That was actually literally his conclusion.
Because he didn't understand the possibility of a continuum of time.
Because there were infinitely many things that happened before the tortoise caught up.
So that was the reasoning.
I mean it's a long time ago but you know people didn't, he didn't believe in continuum.
All right.
So anyway that's a small point.
Now the general case here of a geometric series is where I put in a number a instead of a half here.
So what we had before.
So that's 1 plus a plus a squared.
Isn't quite the most general but anyway I'll write this down.
And you're certainly going to want to remember that the formula for this in the limit is 1 over 1 minus a.
And I remind you that this only works when the absolute value is strictly less than 1.
In other words when minus 1 is strictly less than a is less than 1.
And that's really the issue that we're going to want to worry about now.
What we're worrying about is this notion of convergence.
And what goes wrong when there isn't convergence, when there's a divergence.
So let me illustrate the divergences before going on.
And this is what we have to avoid if we're going to understand series.
So here's an example when a is equal to 1.
You get 1 plus 1 plus 1 plus et cetera.
And that's equal to 1 over 1 minus 1.
Which is 1 over 0.
So this is not bad.
It's almost right.
Right?
It's sort of infinity equals infinity.
At the edge here we managed to get something which is sort of almost right.
But you know, it's, we don't consider this to be logically to make complete sense.
So it's a little dangerous.
And so we just say that it diverges.
And we get rid of this.
So we're still putting it in red.
All right.
The bad guy here.
So this one diverges.
Similarly if I take a equals minus 1, I get 1 minus 1 plus 1 minus one plus 1 because the odd and the even powers in that formula alternate sign.
And this bounces back and forth.
It never settles down.
It starts at 1.
And then it gets down to 0 and then it goes back up to 1, down to 0, back up to 1.
It doesn't settle down.
It bounces back and forth.
It oscillates.
On the other hand if you compare the right hand side.
What's the right hand side?
It's 1 over 1 minus minus 1. which is a half.
All right.
So if you just paid attention to the formula.
Which is what we were doing when we integrated without thinking too hard about this.
You get a number here but in fact that's wrong.
Actually it's kind of an interesting number.
It's halfway between the two between 0 1.
So again there's some sort of vague sense in which this is trying to be this answer.
All right.
It's not so bad but we're still going to put this in a red box.
All right.
because this is what we called divergence.
So both of these cases are divergent.
It only really works when alpha, when a is less than 1.
I'm going to add one more case just to see that mathematicians are slightly curious about what goes on in other cases.
So this is 1 plus 2 plus 2 squared plus 2 to cube plus etc.. And that should be equal to-- according to this formula --1 over 1 minus 2.
Which is negative 1.
All right.
Now this one it clearly wrong, right?
This one is totally wrong.
It certainly diverges.
The left hand side is obviously infinite.
The right hand side is way off.
It's negative 1.
On the other hand it turns out actually that mathematicians have ways of making sense out of these.
In number theory there's a strange system where this is actually true.
And what happens in that system is that what you have to throw out is the idea that 0 is less than 1.
There is no such thing as negative numbers.
So this number exists.
And it's the additive inverse of 1.
It has this arithmetic property but the statement that this is, that 1 is bigger than 0 does not make sense.
So you have your choice either this diverges or you have to throw out something like this.
So that's a very curious thing in higher mathematics.
Which if you get to number theory there's fun stuff there.
All right.
OK but for our purposes these things are all out.
All right.
They're gone.
We're not considering them.
Only a between negative 1 and 1.
All right.
Now I want to do something systematic.
And it's more or less on the lines of the powers that I'm erasing right now.
I want to tell you about series.
Which are kind of borderline convergent.
And then next time when we talk about powers series we'll come back to this very important series which is the most important one.
So now let's talk about some series general notations.
And this will help you with the last bit.
This is going to be pretty much the same as what we did for improper integrals.
Namely, first of all I'm going to have capital S N which is the sum of a n, n equals 0 to capital N. And this is what we're calling a partial sum.
And then the full limit which is capital S, if you like, and N equals 0 to infinity is just the limit as N goes to infinity of the Sn's.
And then we have the same kind of notation that we had before.
Which is there are these two choices which is that if the limit exists.
That's the green choice.
And we say it converges.
So we say the series converges.
And then the other case which is the limit does not exist.
And we can say the series diverges.
Question
[INAUDIBLE PHRASE]
The question was how did I get to this?
And I will do that next time but in fact of course you've seen in high school.
Right this a-- Yeah.
Yeah.
We'll do that next time.
The question was how did we arrive-- sorry I didn't tell you the question --the question was how do we arrive at this formula on the right hand side here.
But we'll talk about that next time.
All right.
So here's the basic definition and what we're going to recognize about series.
And I'm going to give you a few examples and then we'll do something systematic.
So the first example-- well the first example is the geometric series.
But the first example that I'm going to discuss now and in a little bit of detail is this sum 1 over n squared n equals 1 to infinity.
It turns out that this series is very analogous-- and we'll develop this analogy carefully --the integral from 1 to x d x over x squared.
And we're going to develop this analogy in detail later in this lecture.
And this one is one of the ones-- so now you have to go back and actually remember, this is one of the ones you really want to memorize.
And you should especially pay attention to the ones with an infinity in them.
This one is convergent.
And this series is convergent.
Now it turns out that evaluating this is very easy.
This is 1.
It's easy to calculate.
Evaluating this is very tricky.
And Euler did it.
And the answer is pi squared over 6.
That's an amazing calculation.
And it was done very early in the history of mathematics.
If you look at another example-- so maybe example two here -if you look at 1 over n cubed.
n equals-- well you can't start here at 0 by the way.
I get to start wherever I want in these series.
Here I start with 0.
Here I started with 1.
And notice the reason why I I started, it was a bad idea to start with 0 was that 1 over 0 is undefined.
Right?
So I'm just starting where it's convenient for me.
And since I'm interested mostly in the tale behavior it doesn't matter to me so much where I start.
Although if I want an exact answer I need to start exactly at n equals 1.
All right.
This one is similar to this integral here.
All right.
Which is convergent again.
So there's a number that you get.
And let's see what is it something like two thirds or something like that, all right, for this for this number.
Or a third.
What is it?
No a half.
I guess it's a half.
This one is a half.
You check that I'm not positive, but anyway just doing it in my head quickly it seems to be a half.
Anyway it's an easy number to calculate.
This one over here stumped mathematicians basically for all time.
It doesn't have any kind of elementary form like this.
And it was only very recently proved to be rational.
People couldn't even couldn't even decide whether this was a rational number or not.
But anyway that's been resolved it is an irrational number which is what people suspected.
Yeah question
[INAUDIBLE PHRASE]
Yeah sorry.
OK.
I violated a rule of mathematics-- you said why is this similar?
I thought that similar was something else.
And you're absolutely right.
And I violated a rule of mathematics.
Which is that I used this symbol for two different things.
I should have written this symbol here.
All right.
I'll create a new symbol here.
The question of whether this converges or this converges.
These are the the same type of question.
And we'll see why they're the same question it in a few minutes.
But in fact the wiggle I used similar I used for the connection between functions.
The things that are really similar are that 1 over n resembles 1 over x squared.
So I apologize I didn't--
[INAUDIBLE PHRASE]
Oh you thought that this was the definition of that.
That's actually the reason why these things correspond so closely.
That is that the Riemann sum is close to this.
But that doesn't mean they're equal.
The Riemann sum only works when the delta x goes to 0.
The way that we're going to get a connection between these two, as we will just a second, is with a Riemann sum with.
What we're going to use is a Riemann's sum with delta x equals 1.
All right and then that will be the connection between.
All right that's absolutely right.
All right.
So in order to illustrate exactly this idea that you've just come up with.
And in fact that we're going to use.
We'll do the same thing but we're going to do it on the example sum 1 over n. So here's example 3 and it's going to be sum 1 over n, n equals 1 to infinity.
And what we're now going to see is that it corresponds to this integral here.
And we're going to show therefore that this thing diverges.
But we're going to do this more carefully.
We're going to do this in some detail so that you see what it is, that the correspondence is between these quantities.
And the same sort of reasoning applies to these other examples.
So here we go.
I'm going to take the integral and draw the picture of the Riemann's sum.
So here's the level one and here's the function y equals 1 over x.
And I'm going to take the Riemann's sum.
With delta x equals 1.
And that's going to be closely connected to the series that I have.
But now I have to decide whether I want a lower Riemann's sum or an upper Riemann's sum.
And actually I'm going to check both of them because both of them are illuminating.
First we'll do the upper Riemann's sum.
Now that's this staircase here.
So we'll call this the upper Riemann's sum.
And let's check what it's levels are.
This is not to scale.
This level should be a half.
So if this is 1 and this is 2 and that level was supposed to be a half and this next level should be a third.
That's how the Riemann's sums are working out.
And now I have the following phenomenon.
Let's cut it off at the nth stage.
So that means that I'm going, the integral is from 1 to n d x over x.
And the Riemann's sum is something that's bigger than it.
Because the areas are enclosing the area of the curved region.
And that's going to be the area of the first box which is 1, plus the area of the second box which is a half plus the area of the third box which is a third.
All the way up the last one but the last one starts at n minus 1.
So it has 1 over n minus 1.
There are not n boxes here.
They're only n minus 1 boxes.
Because the distance between 1 and n is n minus one.
Right so this is n minus 1 terms.
However, if I use the notation for partial sum.
Which is 1 plus 1 over 2 plus all the way up to 1 over n minus 1 plus 1 over n. In other words I go out to the nth one which is what I would ordinarily do.
Then this sum that I have here certainly is less than Sn.
Because there's one more term there.
And so here I have an integral which is underneath this sum Sn.
Now this is going to allow us to prove conclusively that the-- so I'm just going to rewrite this-- prove conclusively that the sum diverges.
Why is that?
Because this term here we can calculate.
This is the log x evaluated at 1 and n. Which is the same thing as log n minus 0.
All right, the quantity log n minus log 1 which is 0.
And so what we have here is that log n is less than S n. All right and clearly this goes to infinity right.
As n goes to infinity this thing goes to infinity.
So we're done.
All right we've shown divergence.
Now the way I'm going to use the lower Riemann's sum is to recognize that we've captured the rate appropriately.
That is not only do I have a lower bound like this but I have an upper bound which is very similar.
So if I use the upper-- Riemann's oh sorry --the lower Riemann's sum again with delta x equals 1.
Then I have that the integral from 1 to n of d x over x is bigger than-- Well what are the terms going to be if fit sit them underneath?
If I fit them underneath I'm missing the first term.
That is the box is going to be half height.
It's going to be this lower piece.
So I'm missing this first term.
So it'll be a half plus a third plus all right, it will keep on going.
But now the last one instead of being 1 over n minus 1.
It's going to be 1 over n. This is again a total of the n minus 1 terms.
This is the lower Riemann's sum.
And now we can recognize that this is exactly equal to-- well so I'll put it over here-- this is exactly equal to S n minus 1 minus the first term.
So we missed the first term but we got all the rest of them.
So if I put this to the other side remember this is log n. All right.
If I put this to the other side I have the other side of this bound.
I have that S n is less than if I reverse it log n plus 1.
And so I've trapped it on the other side.
And here I have the lower bound.
So I'm going to combine those together.
So all hold I have this correspondence here.
It is the size of a log n is trapped between the-- sorry-- the size of S n which is relatively hard to calculate and understand exactly is trapped between log n and log n plus 1.
Yeah question
[INAUDIBLE PHRASE]
This step here is the step that you're concerned about.
So this step is a geometric argument which is analogous to this step.
All right it's the same type of argument.
And in this case it's that the rectangle's are on top and so the area represented on the right hand side is less than the area represented on this side.
And this is the same type of thing except that the rectangle's are underneath.
So the sum of the areas of the rectangle is less than the area under the curve.
All right.
So I've now trapped this quantity.
And I'm now going to state the sort of general results.
So here's what's known as integral comparison.
It's this double arrow correspondence in the general case, for a very general case.
There are actually even more cases where it works.
But this is a good case and convenient.
Now this is called integral comparison.
And it comes with hypotheses but it follows the same argument that I just gave.
If f of x is decreasing and it's positive then the sum f n, n equals 1 to infinity minus the integral from 1 to infinity of f of x d x is less than f of 1.
That's basically what we showed.
We showed that the difference between s n and log n was at most 1.
All right.
Now if both of them are, and the sum and the integral converge or diverge together.
That is they either both converge or both diverge.
This is the type of test that we like because then we can just convert the question of convergence over here to this question of convergence over on the other side.
Now I remind you that it's incredibly hard to calculate these numbers.
Whereas these numbers are easier to calculate.
Our goal is to reduce things to simpler things.
And in this case sums, infinite sums are much harder than infinite integrals.
All right so that's the integral comparison.
And now I have one last bit on comparisons that I need to tell you about.
And this is very much like what we did with integrals.
Which is a so called limit comparison.
The limit comparison says the following if f of n is similar to g of n. You will recall that means f of n over g of n tends to 1 as n goes to infinity.
And we're in the positive case.
So let's just say g n is positive.
Then-- that doesn't even, well-- then sum f n sum g n either both, same thing as above, either both converge or both diverge.
All right.
This is just saying that if they behave the same way in the tail, which is all we really care about, then they have similar behavior, similar convergence properties.
And let me give you a couple examples.
So here's one example if you take the sum 1 over n squared plus 1 square root.
This is going to be replaced by something simpler.
Which is the main term here.
Which is 1 over square root of n squared which we recognize as sum 1 over n which diverges.
So this guy is one of the red guys.
On the red team.
Now we have another example.
Which is let's say the square root of n, I don't know, to the fifth minus n squared.
Now if you have something where it's negative in the denominator you kind of do have to watch out that denominator makes sense.
It isn't 0.
So we're going to be careful and start this at n equals 2.
In which case, the first term, I don't like 1 over 0 as a term in my series.
So I'm just going to be a little careful about how-- as I said I was kind of lazy here.
I could have started this one at 0 for instance.
All right.
So here's the picture.
Now this I just replace by it's main term which is 1 over n to the fifth square root.
Which is sum 1 over n to the five halves which converges.
All right.
The power is bigger than 1.
1 is the divider for these things and it just misses.
This one converges.
All right so these are the typical ways in which these convergence processes are used.
All right.
So I have one more thing for you.
Which is an advertisement for next time.
And I have this demo here which I will grab.
But you will see this next time.
So here's a question for you to think about overnight but don't ask friends you have to think about it yourself.
So here's the problem.
Here are some blocks which I acquired when my kids left home.
Anyway yeah that'll happen to you too in about four years.
So now here you are, these are blocks.
So now here's the question that we're going to deal with next time.
I'm going to build it, maybe I'll put it over here because I want to have some room to head this way.
I want to stack them up so that-- oh didn't work --going to stack them up in the following way.
I want to do it so that the top one is completely to the right of the bottom one.
That's the question can I do that?
Can I get?
Can I build this up?
So what's let's see here.
I just seem to be missing-- but anyway what I'm going to do is I'm going to try to build this and we're going to see how far we can get with this next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials for hundreds of MIT courses, visit mit opencourseware at ocw.mit.edu
Last time we left off with a question having to do with playing with blocks.
And this is supposed to give us a visceral feel for something anyway, having to do with series.
And the question was whether I could stack these blocks, build up a stack so that I'm going to try here up.
I'm already off balance here, see.
The question is can I build this so that the-- let's draw a picture of it so that the first block is like this.
The next block is like this.
And maybe the next block is like this.
And notice there is no visible means of support for this block.
It's completely to the left of the first block.
And the question is, will this fall down?
Or at least, or more precisely, eventually we'll ask you know, how far can we go?
Now before you answer this question, the claim is it that this is a kind of a natural, physical question, which involves some important answer.
No matter whether the answer is, you can do it or you can't.
So this is a good kind of math question where no matter what the answer is, when you figure out the answer, you're going to get something interesting out of it.
Because they're two possibilities.
Either there is a limit to how far to the left we can go-- in which case that's a very interesting number-- or else there is no limit.
You can go arbitrarily far.
And that's also interesting and curious.
And that's the difference between convergence and divergence, the thing that we were talking about up to now concerning series.
So my first question is, do you think that I can get it so that this thing doesn't fall down with, well you see I have about eight blocks here or so.
So you can vote now.
How many in favor that I can succeed in doing this sort of thing with maybe more than three blocks.
How many in favor?
All right somebody is voting twice.
That's good.
I like that.
How about opposed?
So that was really close to a tie.
All right.
But I think the there was slightly more opposed.
I don't know.
You guys who are in the back maybe could tell.
Anyway it was pretty close.
All right.
So now I'm going-- because this is a real life thing-- I'm going to try to do it.
All right?
All right.
So now I'm going to tell you what the trick is.
The trick is to do it backwards.
When most people are playing with blocks, they decide to build it from the bottom up.
Right?
But we're going to build it from the top down, from the top down.
And that's going to make it possible for us to do the optimal thing at each stage.
So when I build it from the top down, the best I can do is well, it'll fall off.
I need to have it you know, halfway across.
That's the best I can do.
So the top one I'm going to build like that.
I'm going to take it as far to the left as I can.
And then I'm going to put the next one down as far to the left as I can.
And then the next one as far to the left as I can.
That was a little too far.
And then I'm going to do the next one as far to the left as I can.
And then I'm going to do the next one-- well let's line it up first-- as far to the left as I can.
OK?
And then the next one as far to the left as I can.
All right.
Now those of you who are in this line can see, all right, I succeeded.
All right, that's over the edge.
All right?
So it can be done.
All right.
All right.
So now we know that we can get farther than you know, we can make it overflow.
So the question now is, how far can I get?
OK. Do you think I can get to here?
Can I get to the end over here?
So how many people think I can get this far over to here?
How many people think I can get this far?
Well you know, remember.
I'm going to have to use more than just this one more block that I've got.
I don't, right?
Obviously I'm thinking, actually I do have some more blocks at home.
But, OK. We're not going to.
But anyway, do you think I can get over to here?
How many people say yes?
And how many people say no?
More people said no then yes.
All right.
So maybe the stopping place is some mysterious number in between here.
All right?
Well OK.
So now we're going to do the arithmetic.
And we're going to figure out what happens with this problem.
OK?
So let's do it.
All right, so now again the idea is, the idea is we're going to start with the top, the top block.
We'll call that block number one.
And then the farthest if you like to the right, that you can put a block underneath it, is exactly halfway.
All right well, that's the best job I can do.
Now in order to make my units work out easily, I'm going to decide to call the length of the block 2.
All right?
And that means if I start at location 0, then the first place where I am is supposed to be halfway.
And that will be 1.
OK so the first step in the process is 1 more to the right.
Or if you like, if I were building up-- which is what you would actually have to do in real life-- it would be 1 to the left.
OK now the next one.
Now here is the way that you start figuring out the arithmetic.
The next one is based on a physical principle.
Which is that the farthest I can stick this next block underneath is what's called the center of mass of these two, which is exactly halfway here.
That is there's 1/4 of this guy, and a 1/4 of that guy balancing each other.
Right?
So that's as far as I can go.
If I go farther than that, it'll fall over.
So that's the absolute farthest I can do.
So the next block is going to be over here.
And 1/4 of 2 is 1/2.
So this is 3/2 here.
All right so we went to 1.
We went to it 3/2 here.
And then I'm going to keep on going with this eventually.
All right so we're going to figure out what happens with this stack.
Question?
How do you know that this is the best way to optimize?
The question is how do I know that this is the best way to optimize?
I can't answer that question.
But I can tell you that it's the best way if I start with a top like this, and the next one like this.
Right, because I'm doing the farthest possible at each stage.
That actually has a name in computer science that's called the greedy algorithm.
I'm trying to do the best possible at each stage.
The greedy algorithm starting from the bottom, is an extremely bad strategy.
Because when you do that, you stack it this way, and it almost falls over.
And then the next time you can't do anything.
So the greedy algorithm is terrible from the bottom.
This is the greedy algorithm starting from the top, and it turns out to do much better then the greedy algorithm starting from the bottom.
But of course I'm not addressing whether there might not be some other incredibly clever strategy where I wiggle around and make it go up.
I'm not addressing that question.
All right?
It turns out this is the best you can do.
But that's not clear.
All right so now, here we have this thing.
And now I have to figure out what the arithmetic pattern is, so that I can figure out what I was doing with those shapes.
So let's figure out a thought experiment here.
All right?
Now the thought experiment I want to imagine for you is, you've got a stack of a bunch of blocks, and this is the first N blocks.
All right?
And now we're going to put one underneath it.
And what we're going to figure out is the center of mass of those N blocks, which I'm going to call C sub N. OK. And that's the place where I'm going to put this very next block.
I'll put it in a different color here.
Here's the new, the next block over.
And the next block over is the N plus 1st block.
And now I want you to think about what's going on here.
If the center of mass of the first N blocks is this number, this new one, it's of length 2.
And its center of mass is 1 further to the right than the center of mass that we had before.
So in other words, I've added to this configuration of N blocks, 1 more block, which is shifted.
Whose center mass is not lined up with the center of mass of this, but actually over farther to the right.
All right so the new center of mass of this new block-- and this is the extra piece of information that I want to observe-- is that this thing has a center of mass at C N plus 1.
It's 1 unit over because this total length is 2.
So right in the middle there is 1 over, according to my units.
All right now this is going to make it possible for me to figure out what the new center of mass is.
So C N plus 1 is the center of mass of N plus 1 blocks.
Now this is really only in the horizontal variable, right?
I'm not keeping track of the center of mass.
Actually this thing is hard to build because the center of mass is also rising.
It's getting higher and higher.
But I'm only keeping track of its left and right characteristic.
So this is the x-coordinate of it.
All right so now here's the idea.
I'm combining the white ones, the N blocks, with the pink one, which is the one on the bottom.
And there are N of the white ones.
And there's 1 of the pink one.
And so in order to get the center of mass of the whole, I have to take the weighted average of the two.
That's N times C N plus 1, times the center of mass of the pink one, which is C N plus 1.
And then I have to divide if it's the weighted average of the total of N plus 1 blocks, by N plus 1.
This is going to give me the new center of mass of my configuration at the N plus 1st stage.
And now I can just do the arithmetic and figure out what this is.
And the two C Ns combine.
I get N plus 1, times C N plus 1, divided by N plus 1.
And if I combine these two things and do the cancellation, that gives me this recurrence formula, C N plus 1 is equal to C N plus, there's a little extra.
These two cancel.
That gives me the C N. But then I also have 1 over N plus 1.
Well that's how much gain I can get in the center of mass by adding one more block.
That's how much I can shift things over, depending on how we're thinking of things to the left or the right, depending on which direction we're building them.
All right, so now I'm going to work out the formulas.
First of all C 1, that was the center of the first block.
I put its left-end at 0, the center of the first block is at 1.
That means that C 1 is 1.
OK?
C 2, according to this formula-- and actually I've worked it out, we'll check it in a-- C 2 is C 1 plus 1 over 2.
All right, so that's the case, N equals 1.
So this is 1 plus 1/2.
That's what we already did.
That's the 3/2 number.
Now the next one is C sub 2 plus 1/3.
That's the formula again.
And so that comes out to be 1 plus 1/2 plus 1/3.
And now you can see what the pattern is.
C N, if you just keep on going here, C N is going to be 1 plus 1/2 plus 1/3 plus 1/4 plus 1/N.
So now I would like you to vote again.
Do you think I can-- now that we have the formula-- do you think I can get over to here?
How many people think I can get over to here?
How many people think I can't get over to here?
There's still a lot of people who do.
So it's still almost 50/50.
That's amazing.
Well so we'll address that in a few minutes.
So now let me tell you what's going on.
This C N of course, is the same as what we called last time S N. And remember that we actually estimated the size of this guy.
This is related to what's called the harmonic series.
And what we showed was that log N is less than S N, which is less than S N plus 1.
All right?
Now I'm going to call your attention to the red part, which is the divergence part of this estimate, which is this one for the time being, all right.
Just saying that this thing is growing.
And what this is saying is that as N goes to infinity, log N goes to infinity, So that means that S N goes to infinity, because of this inequality here.
It's bigger than log N. And so if N is big enough, we can get as far as we like.
All right?
So I can get to here.
And at least half of you, at least the ones who voted, that was I don't know.
We have a quorum here, but I'm not sure.
We certainly didn't have a majority on either side.
Anyway this thing does go to infinity.
So in principle, if I had enough blocks, I could get it over to here.
All right, and that's the meaning of divergence in this case.
On the other hand, I want to discuss with you, and the reason why I use this example, is I want to discuss with you also what's going on with this other inequality here, and what its significance is.
Which is that it's going to take us a lot of numbers N, a lot of blocks, to get up to a certain level.
In other words, I can't do it with just eight blocks or nine blocks.
In order to get over here, I'd have to use quite a few of them.
So let's just see how many it is.
So I worked this out carefully.
And let's see what I got.
So to get across the lab tables, all right.
This distance here, I already did this secretly.
But I don't actually even have enough of these to show you.
But, well 1, 2, 3, 4, 5, 6, and 1/2.
I guess that's enough.
So it's 6 and 1/2.
So it's two lab tables is 13 of these blocks.
All right.
So there are 13 blocks, which is equal to 26 units.
OK, that's how far to get across I need.
And the first one is already 2.
So it's really 26 minus 2, which is 24.
Which that's what I need.
OK.
So I need log N to be equal to 24, roughly speaking, in order to get that far.
So let's just see how big that is.
All right.
I think I worked this out.
So let's see.
That means that N is equal to e to the 24th-- and if you realize that these blocks are 3 centimeters high-- OK let's see how many that we would need here.
That's kind of a lot.
Let's see, it's 3 centimeters times e to the 24th, which is about 8 times 10 to the 8th meters.
OK. And that is twice the distance to the moon.
So OK, so I could do it maybe.
But I would need a lot of blocks.
Right?
So that's not very plausible here, all right.
So those of you who voted against this were actually sort of half right.
And in fact, if you wanted to get it to the wall over there, which is over 30 feet, the height would be about the diameter of the observable universe.
That's kind of a long way.
There's one other thing that I wanted to point out to you about this shape here.
Which is that if you lean to the left, right?
If you put your head like this-- of course you have to be on your side to look at it-- this curve is the shape of a logarithmic curve.
So in other words, if you think of the vertical as the x-axis, and the horizontal that way, is the vertical, is the up direction, then this thing is growing very, very, very, very slowly.
If you send the x-axis all the way up to the moon, the graph still hasn't gotten across the lab tables here.
It's only partway there.
If you go twice the distance to the moon up that way, it's gotten finally to that end.
All right so that's how slowly the logarithm grows.
It grows very, very slowly.
And if you look at it another way, if you stand on your head, you can see an exponential curve.
So you get some sense as to the growth properties of these functions.
And fortunately these are protecting us from all kinds of stuff that would happen if there weren't exponentially small tails in the world.
Like you know, I could walk through this wall which I wouldn't like doing.
OK, now so this is our last example.
And the important number, unfortunately we didn't discover another important number.
There wasn't an amazing number place where this stopped.
All we discovered again is some property of infinity.
So infinity is still a nice number.
And the theme here is just that infinity isn't just one thing, it has a character which is a rate of growth.
And you shouldn't just think of there being one order of infinity.
There are lots of different orders.
And some of them have different meaning from others.
All right so that's the theme I wanted to do, and just have a visceral example of infinity.
Now, we're going to move on now to some other kinds of techniques.
And this is going to be our last subject.
What we're going to talk about is what are known as power series.
And we've already seen our first power series.
And I'm going to remind you of that.
Here we are with power series.
Our first series was this one.
And we mentioned last time that it was equal to 1 over 1 minus x, for x less than 1.
Well this one is known as the geometric series.
You didn't use the letter x last time, I used the letter a.
But this is known as the geometric series.
Now I'm going to show you one reason why this is true, why the formula holds.
And it's just the kind of manipulation that was done when these things were first introduced.
And here's the idea of a proof.
So suppose that this sum is equal to some number S, which is the sum of all of these numbers here.
The first thing that I'm going to do is I'm going to multiply by x. OK, so if I multiply by x-- let's think about that-- I multiply by x on both the left and the right-hand side.
Then on the left side, I get x plus x-squared plus x-cubed plus, and so forth.
And on the right side, I get S times x.
And now I'm going to subtract the two equations, one from the other.
And there's a very, very substantial cancellation.
This whole tail here gets canceled off.
And the only thing that's left is the 1.
So when I subtract, I get 1 on the left-hand side.
And on the right-hand side, I get S minus S times x.
All right?
And now that can be rewritten as S times 1 minus x.
And so I've got my formula here.
This is 1 over 1 minus x is equal to S. All right.
Now this reasoning has one flaw.
It's not complete.
And this reasoning is basically correct.
But it's incomplete because it requires that S exists.
For example, it doesn't make any sense in the case x equals 1.
So for example in the case x equals 1, we have 1 plus 1 plus 1 plus et cetera, equals whatever we call S. And then when we multiply through by 1, we get 1 plus 1 plus 1 plus, equals S times 1.
And now you see that the subtraction gives us infinity minus infinity is equal to infinity minus infinity.
That's what's really going on in the argument in this context.
So it's just nonsense.
I mean it doesn't give us anything meaningful.
So this argument, it's great.
And it gives us the right answer, but not always.
And the times when it gives us the answer, the correct answer, is when the series is convergent.
And that's why we care about convergence.
Because we want manipulations like this to be allowed.
So the good case, this is the red case that we were describing last time, that's the bad case.
But what we want is the good case, the convergent case.
And that is the case when x is less than 1.
So this is the convergent case.
Yep.
OK, so they're much more detailed things to check exactly what's going on.
But I'm going to just say general words about how you recognize convergence.
And then we're not going to worry about so much about convergence, because it works very, very well.
And it's always easy to diagnose when there's convergence with a power series.
All right so here's the general setup.
The general setup is that we have not just the coefficients 1 all the time, but any numbers here, dot, dot, dot.
And we abbreviate that with the summation notation.
This is the sum a n x to the n, n equals 0 to infinity.
And that's what's known as a power series.
Fortunately there is a very simple rule about how power series converge.
And it's the following.
There's a magic number R which depends on these numbers here such that-- and this thing is known as a radius of convergence-- and the problem that we had, it's this number 1 here.
This thing works for x less than 1.
In our case, it's maybe x less R. So that's some symmetric interval, right?
That's the same as minus R, less then x, less than R, and so where there's convergence.
OK, where the series converges.
Converges.
And then there's the region where every computation that you give will give you nonsense.
So x greater than R is the sum a sub n x to the n, diverges.
And x equals R is very delicate, borderline, and will not be used by us.
OK, we're going to stick inside the radius of convergence.
Now the way you'll be able to recognize this, is the following.
What always happens is that these numbers tend to 0 exponentially fast, fast for x in R, and doesn't even tend to 0 at all for x greater than R. All right so it'll be totally obvious.
When you look at this series here, what's happening when x less than R is that the numbers are getting smaller and smaller, less than 1.
When x is bigger than 1, the numbers are getting bigger and bigger.
There's no chance that the series converges.
So that's going to be the case with all power series.
There's going to be a cut off.
And it'll be one particular number.
And below that it'll be obvious that you have convergence, and you'll be able to do computations.
And above that every formula will be wrong and won't make sense.
So it's a very clean thing.
There is this very subtle borderline, but we're not going to discuss that in this class.
And it's actually not used in direct studies of power series
How can you tell when the numbers are declining exponentially fast, whereas just, in other words 1 over x [INAUDIBLE]?
OK so, the question is why was I able to tell you this word here?
Why was I able to tell you not only is it going to 0, but it's going exponentially fast?
I'm telling you extra information.
I'm telling you it always goes exponentially fast.
You can identify it.
In other words, you'll see it.
And it will happen every single time.
I'm just promising you that it works that way.
And it's really for the same reason that it works that way here, that these are powers.
And what's going on over here is there are, it's close to powers with this a n's.
All right?
There's a long discussion of radius of convergence in many textbooks.
But really it's not necessary, all right, for this purpose?
Yeah?
How do you find R?
The question was how do you find R?
Yes, so I just said, there's a long discussion for how you find the radius of convergence in textbooks.
But we will not be discussing that here.
And it won't be necessary for you.
Because it will be obvious in any given series what the R is.
It will always either 1 or infinity.
It will always work for all x, or maybe it'll stop at some point.
But it'll be very clear where it stops, as it is for the geometric series.
All right?
OK, so now I need to give you the basic facts, and give you a few examples.
So why are we looking at these series?
Well the answer is we're looking at these series because the role that they play is exactly the reverse of this equation here.
That is, and this is a theme which I have tried to emphasize throughout this course, you can read equalities in two directions.
Both are interesting, typically.
You can think, I don't know what the value of this is.
Here's a way of evaluating.
And in other words, the right side is a formula for the left side.
Or you can think of the left side as being a formula for the right side.
And the idea of series is that they're flexible enough to represent all of the functions that we've encountered in this course.
This is the tool which is very much like the decimal expansion which allows you to represent numbers like the square root of 2.
Now we're going to be representing all the numbers, all the functions that we know, e to the x, arctangent, sine, cosine.
All of those functions become completely flexible, and completely available to us, and computationally available to us directly.
So that's what this is a tool for.
And it's just like decimal expansions giving you handle on all real numbers.
So here's how it works.
The rules for convergent power series are just like polynomials.
All of the manipulations that you do for power series are essentially the same as for polynomials.
So what kinds of things do we do with polynomials?
We add them.
We multiply them together.
We do substitutions.
Right?
We take one function of another function.
We divide them.
OK. And these are all really not very surprising operations.
And we will be able to do them with power series too.
The ones that are interesting, really interesting for calculus, are the last two.
We differentiate them, and we integrate them.
And all of these operations we'll be able to do for power series as well.
So now let's explain the high points of this.
Which is mainly just the differentiation and the integration part.
So if I take a series like this and so forth, the formula for it's derivative is just like polynomials.
That's what I just said, it's just like polynomials.
So the derivative of the constant is 0.
The derivative of this term is a 1.
This one is plus 2 a 2 x.
This one is 3a 3 x-squared, et cetera.
That's the formula.
Similarly if I integrate, well there's an unknown constant which I'm going to put first rather than last.
Which corresponds sort of to the a 0 term which, is going to get wiped out.
That a 0 term suddenly becomes a 0 x.
And the anti-derivative of this next term is a sub 1 x-squared over 2.
And the next term is a 2 x-cubed over 3, and so forth.
Yeah, question?
Is that a series or a polynomial?
Is this a series or a polynomial?
Good question.
It's a polynomial if it ends.
If it goes on infinitely far, then it's a series.
They look practically the same, polynomials and series.
There's this little dot, dot, dot here.
Is this a series or a polynomial?
It's the same rule.
If it stops at a finite stage, this one stops at a finite stage.
If it goes on forever, it goes on forever
So I thought that the series add up finite numbers.
You can add up terms of x in series?
So an interesting question.
So the question that was just asked is I thought that a series added up finite numbers.
You could add up x?
That was what you said, right?
OK now notice that I pulled that off on you by changing the letter a to the letter x at the very beginning of this commentary here.
This is a series.
For each individual value of x, it's a number.
So in other words, it I plug in here x equals 1/2, I'm going to add 1 plus 1/2 plus 1/4 plus 1/8; and I'll get a number which is 2.
And I'll plug in a number over here, and I'll get a number.
On the other hand, I can do this for each value of x.
So the interpretation of this is that it's a function of x.
And similarly this is a function of x.
It works when you plug in the possible values x between minus 1 and 1.
So there's really no distinction there, it's just I slipped it passed you.
These are functions of x.
And the notion of a power series is this idea that you put coefficients on a series, but then you allow yourself the flexibility to stick powers here.
And that's exactly what we're doing.
OK there are other kinds of series where you stick other interesting functions in here like sines and cosines.
There are lots of other series that people study.
And these are the simplest ones.
And all those examples are extremely helpful for representing functions.
But we're only going to do this example here.
All right, so here are the two rules.
And now there's only one other complication here which I have to explain to you before giving you a bunch of examples to show you that this works extremely well.
And the last thing that I have to do for you is explain to you something called Taylor's formula.
Taylor's formula is the way you get from the representations that we're used to a functions, to a representation in the form of these coefficients.
When I gave you the function e to the x, it didn't look like a polynomial.
And we have to figure out which of these guys it is, if it's going to fall into our category here.
And here's the formula.
I'll explain to you how it works in a second.
So the formula is f of x, turns out there's a formula in terms of the derivatives of f. Namely, you differentiate n times, and you evaluate it at 0, and you divide by n factorial, and multiply by x to the n. So here's Taylor's formula.
This tells you what the Taylor series is.
Now about half of our job for the next few minutes is going to be to give examples of this.
But let me just explain to you why this has to be.
If you pick out this number here, this is the a n, the magic number a n here.
So let's just illustrate it.
If f of x happens to be a zero plus a 1 x plus a 2 x-squared plus a 3 x-cubed plus dot, dot, dot.
And now I differentiate it, right?
I get a 1 plus 2 a 2x plus 3 a 3 x.
If I differentiate it another time, I get 2 a 2, plus 3--sorry-- 3 times 2 a 3 x plus dot, dot, dot.
And now a third time, I get 3 times 2 a 3 plus et cetera.
So this next term is really in disguise, 4 times 3 times 2 x-- a sorry-- a 4x.
That's what really comes down if I kept track of the fourth term there.
So now here is my function.
But now you see if I plug in x equals 0, I can pick off the third term.
f triple prime of 0 is equal to 3 times 2 times a 3.
Right, because all the rest of those terms when I plug in 0 are just 0.
Here's the formula.
And so the pattern here is this.
And what's really going on here is this is really 3 times 2 times 1, a 3.
And in general a n is equal to f nth derivative divided by n factorial.
And of course, n factorial, I remind you, is n times n minus 1 times n minus 2, all the way down to 1.
Now there's one more crazy convention which is always used.
Which is that there's something very strange here down at 0, which is that 0 factorials turns out, has to be set equal to 1.
All right, so that's what you do in order to make this formula work out.
And that's one of the reasons for this convention.
All right.
So my next goal is to give you some examples.
And let's do a couple.
So here's, well you know, I'm going to have to let you see a few of them next time.
But let me just tell you this one, which is by far the most impressive.
So what happens with e to the x, if the function of f of x is e to the x, is that it's derivative is also e to the x.
And its second derivative is also e to the x.
And it just keeps on going that way.
They're all the same.
So that means that these numbers in Taylor's formula, in the numerator, the nth derivative is very easy to evaluate.
It's just e to the x.
And if I evaluated at x equals 0, I just get 1.
So all of those numerators are 1.
So the formula here, is the sum n equals 0 to infinity, of 1 divided by n factorial x to the n. In particular, we now have an honest formula for e to the first power.
Which is just e. Which if I plug it in, x equals 1, I get 1.
This is the n equals 0 term plus 1.
This is the n equals 1 term plus 1 over 2 factorial plus 1 over 3 factorial plus 1 over 4 factorial.
Right?
So this is our first honest formula for e. And also, this is how you compute the exponential function.
Finally if you take a function like sine x, what you'll discover is that we can complete the sort of strange business that we did at the beginning of the course-- or cosine x-- where we took the linear and quadratic approximations.
Now we're going to get complete formulas for these functions.
Sine x turns out to be equal to x minus x-cubed over 3 factorial plus x to the 5th over 5 factorial minus x to the 7th over 7 factorial, et cetera.
And cosine x is equal to 1, minus x-squared over 2 factorial-- that's the same as this 2 here-- plus x to the 4th over 4 factorial minus x to the 6th over 6 factorial, plus et cetera.
Now these may feel like they're hard to memorize because I've just pulled them out of a hat.
I do expect you to know them.
They're actually extremely similar formulas.
The exponential here just has this collection of factorials.
The sine is all the odd powers with alternating sines.
And the cosine is all the even powers with alternating sines.
So all three of them form part of the same family.
So this will actually make it easier for you to remember, rather than harder.
And so with that, I'll leave the practice on differentiation for next time.
And good luck, everybody.
I'll talk to you individually.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, Professor Jerison is relaxing in sunny London, Ontario today and sent me in as his substitute again.
I'm glad to the here and see you all again.
So our agenda today -- he said that he'd already talked about power series and Taylor's formula I gues on last week right, on Friday?
So I'm going to go a little further with that and show you some examples, show you some applications, and then I have this course evaluation survey that I'll hand out in the last 10 minutes or so of the class.
I also have this handout that he made that says 18.01 end of term, 2007.
If you didn't pick this up coming in, grab it going out.
People tend not to pick it up when they walk in, I see.
So grab this when you're going out.
There's some things missing from it.
He has not decided when his office hours will be at the end of term.
He will have them, just hasn't decided when.
So, check the website for that information.
And we're looking forward to the final exam, which is uh -- aren't we?
Any questions about this technical stuff?
All right, let's talk about power series for a little bit.
So I thought I should review for you what the story with power series is.
OK, could I have your attention please?
So, power series is a way of writing a function as a sum of integral powers of x.
These a0 , a1, and so on, are numbers.
An example of a power series is a polynomial.
Not to be forgotten, one type of power series is one which goes on for a finite number of terms and then ends, so that all of the other, all the higher a sub i's are all 0.
This is a perfectly good example of a power series it's a very special kind of power series.
And part of what I want to tell you today is that power series behave, almost exactly like, polynomials.
There's just one thing that you have to be careful about when you're using power series that isn't a concern for polynomials, and I'll show you what that is in a minute.
So, you should think of them as generalized polynomials.
The one thing that you have to be careful about is that there iss a number, so one caution.
There's a number which I'll call R, where r can be between 0 and it can also be infinity.
It's a number between 0 and infinity, inclusive, so that when the absolute value of x is less than R. So when x is smaller than R in size, the sum converges.
This sum -- that sum converges to a finite value.
And when x is bigger than R in absolute value, the sum diverges.
This R is called the radius of convergence.
So we'll see some examples of what the radius of convergence is in various powers series as well, and how you find it also.
But, let me go on and give you a few more of the properties about power series which I think that professor Jerison talked about earlier.
So one of them is there's a radius of convergence.
Here's another one.
If you're inside of the radius convergence, then the function has all it's derivatives, has all it's derivatives, just like a polynomial does.
You can differentiate it over and over again.
And in terms of those derivatives, the number a sub n in the power series can be expressed in terms of the value of the derivative at 0.
And this is called Taylor's formula.
So I'm saying that inside of this radius of convergence, the function that we're looking at, this f of x, can be written as the value of the function at 0, that's a sub 0, plus the value of the derivative.
This bracket n means you take the derivative n times.
So, when n is one, you take the derivative once at 0 divided by one factorial, which is one, and multiply it by x.
That's the linear term in the power series.
And then the qaudratic term is you take the second derivative.
Remember to divide by 2 factorial which is 2.
Multiply that by x squared and so on out.
So, in terms, so the coefficients in the power series just record the values of the derivatives of the function at x equals 0.
They can be computed that way also.
Let's see.
I think that's a end of my summary of things that he talked about.
I think he did one example, and I'll repeat that example of a power series.
This example wasn't due to David Jerison; it was due to Leonard Euler.
It's the example of where the function is the exponential function e to the x.
So, let's see.
Let's compute what -- I will just repeat for you the computation of the power series for e to the x, just because it's such an important thing to do.
So, in order to do that, I have to know what the derivative of e to the x is, and what the second derivative of e to the x is, and so on, because that comes into the Taylor formula for the coefficients.
But we know what the derivative of e to the x is, it's just e to the x again, and it's that way all the way down.
All the derivatives are e to the x over and over again.
So when I evaluate this at x equal 0, well, the value of e to the x is one, the value of e to the x is one and x equals 0.
You get a value of one all the way down.
So all these derivatives at 0 have the value one.
And now, when I plug into this formula, I find e to the x is 1, plus 1 times x, plus 1 over 2 factorial times x squared plus 1 over 3 factorial times x cubed plus and so on.
So all of these numbers are one, and all you wind up with is the factorials and the denominators.
That's the power series for e to the x.
This was a discovery of Leonhard Euler in 1740 or something.
Yes Ma'am
When your writing out the power series, how far do you have to write it out?
How far do you have to write the power series before it becomes well defined?
Before its a satisfactory solution to an exam problem, I suppose, is another way to phrase the question.
Until you can see what the pattern is.
I can see what the pattern is.
Is there anyone who's in doubt about what the next term might be?
Some people would tell you that you have to write the summation convention thing.
Don't believe them.
If you right out enough terms to make it clear, that's good enough.
OK?
Is that an answer for you?
Yes, Thank you
OK, so that's a basic example.
Let's do another basic example of a powers series.
Oh yes, and by the way, whenever you write out a power series, you should say what the readius of convergence is.
And for now, I will just to tell you that the radius of convergence of this power series is infiinity; that is, this sum always convergence for any value of x. I'll say a little more about that in a few minutes.
Yeah?
So which functions can be written as power series?
Which functions can be written as power series?
That's an excellent question.
Any function that has a reasonable expression can be written as a power series.
I'm not giving you a very good answer because the true answer is a little bit complicated.
But any of the functions that occur in calculus like sines, cosines, tangents, they all have power series expansions, OK?
We'll see more examples.
Let's do another example.
Here's another example.
I guess this was example one.
So, this example, I think, was due to Newton, not Euler.
Let's find the power series expansion of this function, 1 over 1 plus x.
Well, I think that somewhere along the line, you learned about the geometric series which tells you that, which tells you what the answer to this is, and I'll just write it out.
The geometric series tells you that this function can be written as an alternating sum of powers of x.
You may wonder where these minuses came from.
Well, if you really think about the geometric series, as you probably remembered, there was a minus sign here, and that gets replaced by these minus signs.
I think maybe Jerison talked about this also.
Anyway, here's another basic example.
Remember what the graph of this function looks like when x is equal to minus one.
Then there's a little problem here because the denominator becomes 0, so the graph has a poll there.
It goes up to infinity at x equals minus one, and that's an indication that the radius of convergence is not infinity.
Because if you try to converge to this infinite number by putting in x equals minus one here, you'll have a big problem.
In fact, you see when you put in x equals minus one, you keep getting one in every term, and it gets bigger and bigger and does not converge.
In this example, the radius of convergence is one.
OK, so, let's do a new example now.
Oh, and by the way, I should say you can calculate these numbers using Taylor's formula.
If you haven't seen it, check it out.
Calculate the iterated derivatives of this function and plug in x equals 0 and see that you get plus one, minus one, plus one, minus one, and so on.
Yes sir
For the radius of convergence as stated, if you do minus one it'll fall out.
If you put in one though, seems like it would be fine
The questions is I can see that there's a problem at x equals minus one, why is there also a problem at x equals one where the graph is perfectly smooth and innocuous and finite.
That's another excellent question.
The problem is, that if you go off to a radius of one in any direction and there's a problem, that's it.
That's what the radius of convergence is.
Here, what does happen if I put an x equals plus one?
So, let' look at the partial sums.
Do x equals plus one in your mind here.
So I'll get a partial sum one, then 0, and then one, and then 0, and then one.
So even though it doesn't go up to infinity, it still does not converge
And anything in between?
Any of these other things will also fail to converge in this example.
Well, that's the only two real numbers at the edge.
Right?
OK, let's do a different example now.
How about a trig function?
The sine of x. I'm going to compute the power series expansion for the sine of x, and I'm going to do it using Taylor's formula.
So Taylor's formula says that I have to start computing derivatives of the sine of x.
Sounds like it's going to be a lot of work.
Let's see, the derivative of the sine is the cosine.
and the derivative of the cosine, that's the second derivative of the sine, is what?
Remember the minus, it's minus sine of x. OK, now I want to take the third derivative of the sine, which is the derivative of sine, prime, prime, so it's the derivative of this.
And we just decided the derivative of sine is cosine, so I get cosine, but I have this minus sign in front.
And now I want to differentiate again, so the cosine becomes a minus sine, and that minus sine cancels with this minus sine to give me sine of x.
You follow that?
It's a lot of minus one's canceling out there.
So, all of a sudden, I'm right back where I started; these two are the same and the pattern will now repeat forever and ever.
Higher and higher derivatives of sines are just plus or minus sines and cosines.
Now Taylor's formula says I should now substitute x equals 0 into this and see what happens, so let's do that.
When x is equals to 0, the sine is 0 and the cosine is one.
The sine is 0, so minus 0 is also 0.
The cosine is one, but now there's a minus one, and now I'm back where I started, and so the pattern will repeat.
OK, so the values of the derivatives are all zeros and plus and minus ones and they go through that pattern, four-fold periodicity, over and over again.
And si we can write out what the sine of x is using Taylor's formula, using this formula.
So I put in the value at 0 which is 0, then I put in the derivative which is 1 multiplied by x.
Then, I have the second derivative divided by 2 factorial, but the second derivatve at 0 is 0.
So I'm going to drop that term out.
Now I have the third derivative which is minus one.
And remember the 3 factorial in the denominator.
That's the coefficient of x cubed.
What's the fourth derivative?
Well, here we are, it's on the board, it's 0.
So I drop that term out go up to the fifth term, the fifth power of x.
It's derivative is now one.
We've gone through the pattern, we're back at plus one as the value of the iterated derivative, so now I get 1/5 factorial times x to the fifth.
Now, you tell me, have we done enough terms to see what the pattern is?
I guess the next term will be a minus 1/7 factorial x to the seventh and so on.
Let me write this out again just so we have it.
X cubed over 3 factorial, so it's x minus x cubed over 3 factorial plus x to the fifth over 5 factorial.
You guessed it, and so on.
That's the power series expansion for the sine of x, OK?
And so, the sines alternate, and these denominators get very big, don't they?
Exponentials grow very fast.
Let me make a remark.
R is infinity here.
The radius of convergence of this power series again is infinity, and let me just say why.
The reason is that the general term is going to be like x to the 2n plus 1 divided by 2n plus one factorial, an odd number I can write as 2n plus 1.
And what I want to say is that the size of this, what happens to the size of this as n goes to infinity.
So let's just think about this.
For a fixed x, let's fix the number of x.
Look at powers of x and think about the size of this expression when n gets to be large.
So let's just do that for a second.
So, for x to the 2n plus 1 over 2n plus 1 factorial, I can write out like this.
It's x over one times x over 2 times-- sorry-- times x over 3, times x over 2n plus 1.
I've multiplied x by itself 2n plus 1 times in the numerator, and I've multiplied the numbers 1, 2, 3, 4, and so on, by each other in the denominator, and that gives me the factorial.
So I've just written this out like this.
Now x is fixed, so maybe it's a million, OK?
It's big, but fixed.
What happens to these numbers?
Well at first, they're pretty big.
This is a million over 2, this is a million over 3.
But when n gets to be maybe if n is a million, then this is about one half.
If n is a billion, then this is about 1/2,000, right?
The denominators keep getting bigger and bigger, but the numerators stay the same; they're always x.
So when I take the product, if I go far enough out, I'm going to be multiplying, by very, very small numbers and more and more of them.
And so no matter what x is, these numbers will converge to 0.
They'll get smaller and smaller as x gets to be bigger.
That's the sign that x is inside of the radius of convergence.
This is the sign for you that this series converges for that value of x.
And because I could do this for any x, this works.
This convergence to 0 for any fixed x.
That's what tells you that you can take, that the raidus of convergence is infinity because in the formula-- in the fact the radius of convergence talks about.
If R is equal infinity, this is no condition on x.
Every number is less than infinity in absolute value.
So if this covergence to 0 of the general term works for every x, then radius of convergence is infinity.
Well that was kind of fast, but I think that you've heard something about that earlier as well.
Anyway, so we've got the sine function, a new function with its own power series.
It's a way of computing sine of x.
If you take enough terms you'll get a good evaluation of the sine of x for any x.
This tells you a lot about the function sine of x but not everything at all.
For example, from this formula, it's very hard to see that the sine of x is periodic.
It's not obvious at all.
Somewhere hidden away in this expression is the number pi, the half of the period.
But that's not clear from the power series at all.
So the power series are very good for some things, but they hide other properties of functions.
Well, so I want to spend a few minutes telling you about what you can do with a power series once you have one to get new power series, and new power series from old.
And this is also called operations on power series.
So what are the things that we can do to a power series?
Well one of the things you can do is multiply.
So, for example, what if I want to compute a power series for x times the sine of x.
Well I have a power series for the sine of x, I just did it.
How about a power series for x?
Actually, I did that here too.
The function x is a very simple polynomial.
It's a polynomial where, about 0, a1 is 1, and all the other coefficients are 0.
So x itself is a power series, a very simple one.
The sine of x is a powers series.
And what I want to encourage you to do is treat power series just like polynomials and multiply them together.
We'll see other operations too.
So, to compute the power series for x times the sine of x, I just take this one and multiply it by x.
So let's see if I can do that right.
It distributes through x squared minus x to the fourth over 3 factorial plus x to the sixth over 5 factorial, and so on.
And again, the radius of convergence is going to be the smaller of the two radii of convergence here.
So it's R equals infinity in this case.
OK, you can multiply power series together.
It can be a pain if the power series are very long, but if one of them is x, it's pretty simple.
OK, that's one thing I can do.
Notice something by the way.
You know that even an odd functions?
So, sine is an odd function, x is an odd function, the product of two large functions is an even function.
And that's reflected in the fact that all the powers that occur in the power series are even.
For an odd function, like the sine, all the powers that occur are odd powers of x.
That's always true.
OK, we can multiply.
I can also differentiate.
so let's just do a case of that and use the process of differentiation to find out what the power series for the cosine of x is by writing the cosine of x as the derivative of the sine and differentiating term by term.
So, I'll take this expression for the power series of the sine and differentiate it term by term, and I'll get the power series for cosine.
So, let's see.
The derivative of x is one.
Now, the derivative of x cubed is 3x squared, and then there's a 3 factorial in the denominator.
And the derivative of x to the fifth is 5x to the fourth, and there's a 5 factorial in the denominator, and so on and so on.
And now some cancellation happens.
So this is one minus, well, the 3 cancels with the last factor in this 3 factorial and leaves you with 2 factorial.
And the 5 cancels with the last factor in the 5 factorial and leaves you with a 4 factorial in the denominator.
and so there you go, there's the power series expansion for the cosine.
It's got all even powers of x.
They alternate, and you have factorials in the denominator.
And of course, you could derive that expression by using Taylor's formula by the same kind of calculation you did here, taking higher and higher derivatives of the cosine.
You get the same periodic pattern of derivatives and values of derivatives at x equals 0.
But here's a cleaner way to do it, simpler way to do it because we already knew the derivative of the sine.
When you differentiate, you keep the same radius of convergence.
OK, so we can multiply, I can add too and multiply that constant, things like that.
How about integrating?
That's what half of this course was about isn't it?
So, let's integrate something.
So, the integration I'm going to do is this one: the integral from 0 to x of dt over one plus x.
What is that integral as a function?
So, when I find the anti-derivative of this, I get the natural log of 1 plus t, and then when I evaluate that at t equals x, I get the natural log of 1 plus x.
And when I evaluate the natural log at 0, I get the natural log of 1, which is 0, so this is what you get, OK?
This is really valid, by the way, for x bigger than minus 1.
But you don't want to think about this quite like this when x is smaller than that.
Now, I'm going to try to apply power series methods here and find, use this integral to find a power series for the natural log, and I'll do it by plugging in to this expression what the power series for 1 over 1 plus t was.
And I know what that is because I wrote it down on the board up here.
Change the variable from x to t there, and so 1 over 1 plus t is 1 minus t, plus t squared, minust t cubed, and so on.
So that's the thing in the in the inside of the integral, and now it's legal to integrate that term by term, so let's do that.
I'm going to get something which I will then evaluate at x and at 0.
So, when I integrate one I get x, and when they integrate t, I get t. I'm sorry.
When I intergrate t, I get t squared over 2, and t squared gives me t cubed over 3, and so on and so on.
And then, when I put in t equals x, while I just replace all the t's by x's, and when I put in t equals 0, I get 0.
So this equals x.
So, I've discovered that the natural log of 1 plus x is x minus x squared over 2, plus x cubed over 3 minus x to the fourth over 4, and so on and so on.
There's the power series expansion for the natural log of 1 plus x.
And because I began with a power series who's radius of convergence was just 1, I began with this power series, the radius of convergence of this is also going to be 1.
Also, because this function, as I just pointed out, this function goes bad when x becomes less than minus 1, so some problem happens, and that's reflected in the radius of convergence.
Cool.
So, you can integrate.
That is the correct power series expansion for the natural log of 1 plus x, and another victory of Euler's was to use this kind of power series expansion to calculate natural logarithms in a much more efficient way than people had done before.
OK, one more property, I think.
What are we at here, 3?
4.
Substitute.
Very appropriate for me as a substitute teacher to tell you about substitution.
So I'm going to try to find the power series expansion of e to the minus t squared.
OK?
And the way I'll do that is by taking the power series expansion for e to the x, which we have up there, and make the substitution x equals minus t squared in the expansion for e to the x.
Did you have a question?
Well, it's just concerning the radius of convergence.
You can't define x so that is always positive, and if so, it wouldn't have a radius of convergence, right?
Like I say, again the worry is this natural log of 1 plus x function is perfectly well behaved for large x.
Why does the power series fail to converge for large x?
Well suppose that x is bigger than one, then here you get bigger and bigger powers of x, which will grow to infinity, and they grow large faster than the numbers 2, 3, 4, 5, 6.
They grow exponentially, and these just grow linearly.
So, again, the general term, when x is bigger than one, the general term will go off to infinity, even though the function that you're talking about, log of net of 1 plus x is perfectly good.
So the power series is not good outside of the radius of convergence.
It's just a fact of life.
Yes?
[INAUDIBLE]
I'd rather-- talk to me after class.
The question is why is it the smaller of the two radii of convergence?
The basic answer is, well, you can't expect it to be bigger than that smaller one, because the power series only gives you information inside of that range about the function, so
[INAUDIBLE]
Well, in this case, both of the radii of convergence are infinity.
x has radius of convergence infinity for sure, and sine of x does too.
So you get infinity in that case, OK?
OK, let's just do this, and then I'm going to integrate this and that'll be the end of what I have time for today.
So what's the power series expansion for this?
The power series expansion of this is going to be a function of t, right, because the variable here is t. I get it by taking my expansion for e to the x and putting in what x is in terms of t. And so on and so on.
I just put in minus t squared in place of x there in the series expansion for e to the x. I can work this out a little bit better.
minus t squared is what it is.
This is going to give me a t to the fourth and the minus squared is going to give me a plus, so I get t to the fourth over 2 factorial.
Then I get minus t quantity cubed, so there'll be a minus sign and a t to the sixth and the denominator 3 factorial.
So the signs are going to alternate, the powers are all even, and the denominators are these factorials.
Several times as this course has gone on, the error function has made an appearance.
The error function was, I guess it gets normalized by putting a 2 over the square root of pi in front, and it's the integral of e to the minus t squared d t from 0 to x.
And this normalization is here because as x gets to be large the value becomes one.
So this error function is very important in the theory of probability.
And I think you calculated this fact at some point in the course.
So the standard definition of the error function, you put a 2 over the square root of pi in front.
Let's calculate it's power series expansion.
So there's a 2 over the square root of pi that hurts nobody here in the front.
And now I want to integrate either the minus t squared, and I'm going to use this power series expansion for that to see what you get.
So I'm just going to write this out I think.
I did it out carefully in another example over there so I'll do it a little quicker now.
intergrate this terms by term, you're just integrating powers of t so it's pretty simple, so I get-- and then I'm evaluating at and then 0.
So I get x minus x cubed over 3, plus x to the fifth over 5 times 2 factorial, 5 from integrating the t's of the fourth, and the 2 factorial from this denominator that we already had.
And then there's an a minus x to the seventh over 7 times 3 factorial, and plus, and so on, and you can imagine how they go on from there.
I guess to get this exactly in the form that we began talking about, I should multiply through.
So the coeifficient of x is 2 over the square root of pi, and the coefficient of x cubed is minus 2/3 times the square root of pi, and so on.
But this is a perfectly good way to write this power series expansion as well.
And, this is a very good way to compute the value of the error function.
It's a new function in our experience.
Your calculator probably calculates it, and your calculator probably does it by this method.
OK, so that's my sermon on examples of things you can do with power series.
So, we're going to do the CEG thing in just a minute.
Professor Jerison wanted me to make an ad for 18.02.
Just in case you were thinking of not taking it next term, you really should take it.
It will put a lot of things in this course into context, for one thing.
It's about vector calculus and so on.
So you'll learn about vectors and things like that.
But it comes back and explains some things in this course that might have been a little bit strange, like these strange formulas for the product rule and the quotient rule and the sort of random formulas.
Well, one of the things you learn in 18.02 is that they're all special cases of the chain rule.
And just to drive that point home, he wanted me to show you this poem of his that really drives the points home forcefully, I think.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So let's start right away with stuff that we will need to see before we can go on to more advanced things.
So, hopefully yesterday in recitation, you heard a bit about vectors.
How many of you actually knew about vectors before that?
OK, that's the vast majority.
If you are not one of those people, well, hopefully you'll learn about vectors right now.
I'm sorry that the learning curve will be a bit steeper for the first week.
But hopefully, you'll adjust fine.
If you have trouble with vectors, do go to your recitation instructor's office hours for extra practice if you feel the need to.
You will see it's pretty easy.
So, just to remind you, a vector is a quantity that has both a direction and a magnitude of length.
So -- So, concretely the way you draw a vector is by some arrow, like that, OK?
And so, it has a length, and it's pointing in some direction.
And, so, now, the way that we compute things with vectors, typically, as we introduce a coordinate system.
So, if we are in the plane, x-y-axis, if we are in space, x-y-z axis.
So, usually I will try to draw my x-y-z axis consistently to look like this.
And then, I can represent my vector in terms of its components along the coordinate axis.
So, that means when I have this row, I can ask, how much does it go in the x direction?
How much does it go in the y direction?
How much does it go in the z direction?
And, so, let's call this a vector A.
So, it's more convention.
When we have a vector quantity, we put an arrow on top to remind us that it's a vector.
If it's in the textbook, then sometimes it's in bold because it's easier to typeset.
If you've tried in your favorite word processor, bold is easy and vectors are not easy.
So, the vector you can try to decompose terms of unit vectors directed along the coordinate axis.
So, the convention is there is a vector that we call ***amp***lt;i***amp***gt; hat that points along the x axis and has length one.
There's a vector called ***amp***lt;j***amp***gt; hat that does the same along the y axis, and the ***amp***lt;k***amp***gt; hat that does the same along the z axis.
And, so, we can express any vector in terms of its components.
So, the other notation is ***amp***lt;a1, a2, a3 ***amp***gt; between these square brackets.
Well, in angular brackets.
So, the length of a vector we denote by, if you want, it's the same notation as the absolute value.
So, that's going to be a number, as we say, now, a scalar quantity.
OK, so, a scalar quantity is a usual numerical quantity as opposed to a vector quantity.
And, its direction is sometimes called dir A, and that can be obtained just by scaling the vector down to unit length, for example, by dividing it by its length.
So -- Well, there's a lot of notation to be learned.
So, for example, if I have two points, P and Q, then I can draw a vector from P to Q.
And, that vector is called vector PQ, OK?
So, maybe we'll call it A.
But, a vector doesn't really have, necessarily, a starting point and an ending point.
OK, so if I decide to start here and I go by the same distance in the same direction, this is also vector A.
It's the same thing.
So, a lot of vectors we'll draw starting at the origin, but we don't have to.
So, let's just check and see how things went in recitation.
So, let's say that I give you the vector ***amp***lt;3,2,1***amp***gt;.
And so, what do you think about the length of this vector?
OK, I see an answer forming.
So, a lot of you are answering the same thing.
Maybe it shouldn't spoil it for those who haven't given it yet.
OK, I think the overwhelming vote is in favor of answer number two.
I see some sixes, I don't know.
That's a perfectly good answer, too, but hopefully in a few minutes it won't be I don't know anymore.
So, let's see.
How do we find -- -- the length of a vector three, two, one?
Well, so, this vector, A, it comes towards us along the x axis by three units.
It goes to the right along the y axis by two units, and then it goes up by one unit along the z axis.
OK, so, it's pointing towards here.
That's pretty hard to draw.
So, how do we get its length?
Well, maybe we can start with something easier, the length of the vector in the plane.
So, observe that A is obtained from a vector, B, in the plane.
Say, B equals three (i) hat plus two (j) hat.
And then, we just have to, still, go up by one unit, OK?
So, let me try to draw a picture in this vertical plane that contains A and B.
If I draw it in the vertical plane, so, that's the Z axis, that's not any particular axis, then my vector B will go here, and my vector A will go above it.
And here, that's one unit.
And, here I have a right angle.
So, I can use the Pythagorean theorem to find that length A^2 equals length B^2 plus one.
Now, we are reduced to finding the length of B.
The length of B, we can again find using the Pythagorean theorem in the XY plane because here we have the right angle.
Here we have three units, and here we have two units.
OK, so, if you do the calculations, you will see that, well, length of B is square root of (3^2 2^2), that's 13.
So, the square root of 13 -- -- and length of A is square root of length B^2 plus one (square it if you want) which is going to be square root of 13 plus one is the square root of 14, hence, answer number two which almost all of you gave.
OK, so the general formula, if you follow it with it, in general if we have a vector with components a1, a2, a3, then the length of A is the square root of a1^2 plus a2^2 plus a3^2.
OK, any questions about that?
Yes?
Yes.
So, in general, we indeed can consider vectors in abstract spaces that have any number of coordinates.
And that you have more components.
In this class, we'll mostly see vectors with two or three components because they are easier to draw, and because a lot of the math that we'll see works exactly the same way whether you have three variables or a million variables.
If we had a factor with more components, then we would have a lot of trouble drawing it.
But we could still define its length in the same way, by summing the squares of the components.
So, I'm sorry to say that here, multi-variable, multi will mean mostly two or three.
But, be assured that it works just the same way if you have 10,000 variables.
Just, calculations are longer.
OK, more questions?
So, what else can we do with vectors?
Well, another thing that I'm sure you know how to do with vectors is to add them to scale them.
So, vector addition, so, if you have two vectors, A and B, then you can form, their sum, A plus B.
How do we do that?
Well, first, I should tell you, vectors, they have this double life.
They are, at the same time, geometric objects that we can draw like this in pictures, and there are also computational objects that we can represent by numbers.
So, every question about vectors will have two answers, one geometric, and one numerical.
OK, so let's start with the geometric.
So, let's say that I have two vectors, A and B, given to me.
And, let's say that I thought of drawing them at the same place to start with.
Well, to take the sum, what I should do is actually move B so that it starts at the end of A, at the head of A. OK, so this is, again, vector B.
So, observe, this actually forms, now, a parallelogram, right?
So, this side is, again, vector A.
And now, if we take the diagonal of that parallelogram, this is what we call A plus B, OK, so, the idea being that to move along A plus B, it's the same as to move first along A and then along B, or, along B, then along A.
A plus B equals B plus A. OK, now, if we do it numerically, then all you do is you just add the first component of A with the first component of B, the second with the second, and the third with the third.
OK, say that A was ***amp***lt;a1, a2, a3***amp***gt; B was ***amp***lt;b1, b2, b3***amp***gt;, then you just add this way.
OK, so it's pretty straightforward.
So, for example, I said that my vector over there, its components are three, two, one.
But, I also wrote it as 3i 2j k. What does that mean?
OK, so I need to tell you first about multiplying by a scalar.
So, this is about addition.
So, multiplication by a scalar, it's very easy.
If you have a vector, A, then you can form a vector 2A just by making it go twice as far in the same direction.
Or, we can make half A more modestly.
We can even make minus A, and so on.
So now, you see, if I do the calculation, 3i 2j k, well, what does it mean?
3i is just going to go along the x axis, but by distance of three instead of one.
And then, 2j goes two units along the y axis, and k goes up by one unit.
Well, if you add these together, you will go from the origin, then along the x axis, then parallel to the y axis, and then up.
And, you will end up, indeed, at the endpoint of a vector.
OK, any questions at this point?
Yes?
Exactly.
To add vectors geometrically, you just put the head of the first vector and the tail of the second vector in the same place.
And then, it's head to tail addition.
Any other questions?
Yes?
That's correct.
If you subtract two vectors, that just means you add the opposite of a vector.
So, for example, if I wanted to do A minus B, I would first go along A and then along minus B, which would take me somewhere over there, OK?
So, A minus B, if you want, would go from here to here.
OK, so hopefully you've kind of seen that stuff either before in your lives, or at least yesterday.
So, I'm going to use that as an excuse to move quickly forward.
So, now we are going to learn a few more operations about vectors.
And, these operations will be useful to us when we start trying to do a bit of geometry.
So, of course, you've all done some geometry.
But, we are going to see that geometry can be done using vectors.
And, in many ways, it's the right language for that, and in particular when we learn about functions we really will want to use vectors more than, maybe, the other kind of geometry that you've seen before.
I mean, of course, it's just a language in a way.
I mean, we are just reformulating things that you have seen, you already know since childhood.
But, you will see that notation somehow helps to make it more straightforward.
So, what is dot product?
Well, dot product as a way of multiplying two vectors to get a number, a scalar.
And, well, let me start by giving you a definition in terms of components.
What we do, let's say that we have a vector, A, with components a1, a2, a3, vector B with components b1, b2, b3.
Well, we multiply the first components by the first components, the second by the second, the third by the third.
If you have N components, you keep going.
And, you sum all of these together.
OK, and important: this is a scalar.
OK, you do not get a vector.
You get a number.
I know it sounds completely obvious from the definition here, but in the middle of the action when you're going to do complicated problems, it's sometimes easy to forget.
So, that's the definition.
What is it good for?
Why would we ever want to do that?
That's kind of a strange operation.
So, probably to see what it's good for, I should first tell you what it is geometrically.
OK, so what does it do geometrically?
Well, what you do when you multiply two vectors in this way, I claim the answer is equal to the length of A times the length of B times the cosine of the angle between them.
So, I have my vector, A, and if I have my vector, B, and I have some angle between them, I multiply the length of A times the length of B times the cosine of that angle.
So, that looks like a very artificial operation.
I mean, why would want to do that complicated multiplication?
Well, the basic answer is it tells us at the same time about lengths and about angles.
And, the extra bonus thing is that it's very easy to compute if you have components, see, that formula is actually pretty easy.
So, OK, maybe I should first tell you, how do we get this from that?
Because, you know, in math, one tries to justify everything to prove theorems.
So, if you want, that's the theorem.
That's the first theorem in 18.02.
So, how do we prove the theorem?
How do we check that this is, indeed, correct using this definition?
So, in more common language, what does this geometric definition mean?
Well, the first thing it means, before we multiply two vectors, let's start multiplying a vector with itself.
That's probably easier.
So, if we multiply a vector, A, with itself, using this dot product, so, by the way, I should point out, we put this dot here.
That's why it's called dot product.
So, what this tells us is we should get the same thing as multiplying the length of A with itself, so, squared, times the cosine of the angle.
But now, the cosine of an angle, of zero, cosine of zero you all know is one.
OK, so that's going to be length A^2.
Well, doesn't stand a chance of being true?
Well, let's see.
If we do AdotA using this formula, we will get a1^2 a2^2 a3^2.
That is, indeed, the square of the length.
So, check.
That works.
OK, now, what about two different vectors?
Can we understand what this says, and how it relates to that?
So, let's say that I have two different vectors, A and B, and I want to try to understand what's going on.
So, my claim is that we are going to be able to understand the relation between this and that in terms of the law of cosines.
So, the law of cosines is something that tells you about the length of the third side in the triangle like this in terms of these two sides, and the angle here.
OK, so the law of cosines, which hopefully you have seen before, says that, so let me give a name to this side.
Let's call this side C, and as a vector, C is A minus B.
It's minus B plus A.
So, it's getting a bit cluttered here.
So, the law of cosines says that the length of the third side in this triangle is equal to length A2 plus length B2.
Well, if I stopped here, that would be Pythagoras, but I don't have a right angle.
So, I have a third term which is twice length A, length B, cosine theta, OK?
Has everyone seen this formula sometime?
I hear some yeah's.
I hear some no's.
Well, it's a fact about, I mean, you probably haven't seen it with vectors, but it's a fact about the side lengths in a triangle.
And, well, let's say, if you haven't seen it before, then this is going to be a proof of the law of cosines if you believe this.
Otherwise, it's the other way around.
So, let's try to see how this relates to what I'm saying about the dot product.
So, I've been saying that length C^2, that's the same thing as CdotC, OK?
That, we have checked.
Now, CdotC, well, C is A minus B.
So, it's A minus B, dot product, A minus B.
Now, what do we want to do in a situation like that?
Well, we want to expand this into a sum of four terms.
Are we allowed to do that?
Well, we have this dot product that's a mysterious new operation.
We don't really know.
Well, the answer is yes, we can do it.
You can check from this definition that it behaves in the usual way in terms of expanding, vectoring, and so on.
So, I can write that as AdotA minus AdotB minus BdotA plus BdotB.
So, AdotA is length A^2.
Let me jump ahead to the last term.
BdotB is length B^2, and then these two terms, well, they're the same.
You can check from the definition that AdotB and BdotA are the same thing.
Well, you see that this term, I mean, this is the only difference between these two formulas for the length of C. So, if you believe in the law of cosines, then it tells you that, yes, this a proof that AdotB equals length A length B cosine theta.
Or, vice versa, if you've never seen the law of cosines, you are willing to believe this.
Then, this is the proof of the law of cosines.
So, the law of cosines, or this interpretation, are equivalent to each other.
OK, any questions?
Yes?
So, in the second one there isn't a cosine theta because I'm just expanding a dot product.
OK, so I'm just writing C equals A minus B, and then I'm expanding this algebraically.
And then, I get to an answer that has an A.B.
So then, if I wanted to express that without a dot product, then I would have to introduce a cosine.
And, I would get the same as that, OK?
So, yeah, if you want, the next step to recall the law of cosines would be plug in this formula for AdotB.
And then you would have a cosine.
OK, let's keep going.
OK, so what is this good for?
Now that we have a definition, we should figure out what we can do with it.
So, what are the applications of dot product?
Well, will this discover new applications of dot product throughout the entire semester,but let me tell you at least about those that are readily visible.
So, one is to compute lengths and angles, especially angles.
So, let's do an example.
Let's say that, for example, I have in space, I have a point, P, which is at (1,0,0).
I have a point, Q, which is at (0,1,0).
So, it's at distance one here, one here.
And, I have a third point, R at (0,0,2), so it's at height two.
And, let's say that I'm curious, and I'm wondering what is the angle here?
So, here I have a triangle in space connect P, Q, and R, and I'm wondering, what is this angle here?
OK, so, of course, one solution is to build a model and then go and measure the angle.
But, we can do better than that.
We can just find the angle using dot product.
So, how would we do that?
Well, so, if we look at this formula, we see, so, let's say that we want to find the angle here.
Well, let's look at the formula for PQdotPR.
Well, we said it should be length PQ times length PR times the cosine of the angle, OK?
Now, what do we know, and what do we not know?
Well, certainly at this point we don't know the cosine of the angle.
That's what we would like to find.
The lengths, certainly we can compute.
We know how to find these lengths.
And, this dot product we know how to compute because we have an easy formula here.
OK, so we can compute everything else and then find theta.
So, I'll tell you what we will do is we will find theta -- -- in this way.
We'll take the dot product of PQ with PR, and then we'll divide by the lengths.
OK, so let's see.
So, we said cosine theta is PQdotPR over length PQ length PR.
So, let's try to figure out what this vector, PQ, well, to go from P to Q, I should go minus one unit along the x direction plus one unit along the y direction.
And, I'm not moving in the z direction.
So, to go from P to Q, I have to move by ***amp***lt;-1,1,0***amp***gt;.
To go from P to R, I go -1 along the x axis and 2 along the z axis.
So, PR, I claim, is this.
OK, then, the lengths of these vectors, well,(-1)^2 (1)^2 (0)^2, square root, and then same thing with the other one.
OK, so, the denominator will become the square root of 2, and there's a square root of 5.
What about the numerator?
Well, so, remember, to do the dot product, we multiply this by this, and that by that, that by that.
And, we add.
Minus 1 times minus 1 makes 1 plus 1 times 0, that's 0.
Zero times 2 is 0 again.
So, we will get 1 over square root of 10.
That's the cosine of the angle.
And, of course if we want the actual angle, well, we have to take a calculator, find the inverse cosine, and you'll find it's about 71.5°.
Actually, we'll be using mostly radians, but for today, that's certainly more speaking.
OK, any questions about that?
No?
OK, so in particular, I should point out one thing that's really neat about the answer.
I mean, we got this number.
We don't really know what it means exactly because it mixes together the lengths and the angle.
But, one thing that's interesting here, it's the sign of the answer, the fact that we got a positive number.
So, if you think about it, the lengths are always positive.
So, the sign of a dot product is the same as a sign of cosine theta.
So, in fact, the sign of AdotB is going to be positive if the angle is less than 90°.
So, that means geometrically, my two vectors are going more or less in the same direction.
They make an acute angle.
It's going to be zero if the angle is exactly 90°, OK, because that's when the cosine will be zero.
And, it will be negative if the angle is more than 90°.
So, that means they go, however, in opposite directions.
So, that's basically one way to think about what dot product measures.
It measures how much the two vectors are going along each other.
OK, and that actually leads us to the next application.
So, let's see, did I have a number one there?
Yes.
So, if I had a number one, I must have number two.
The second application is to detect orthogonality.
It's to figure out when two things are perpendicular.
OK, so orthogonality is just a complicated word from Greek to say things are perpendicular.
So, let's just take an example.
Let's say I give you the equation x 2y 3z = 0.
OK, so that defines a certain set of points in space, and what do you think the set of solutions look like if I give you this equation?
So far I see one, two, three answers, OK.
So, I see various competing answers, but, yeah, I see a lot of people voting for answer number four.
I see also some I don't knows, and some other things.
But, the majority vote seems to be a plane.
And, indeed that's the correct answer.
So, how do we see that it's a plane?
So, I should say, this is the equation of a plane.
So, there's many ways to see that, and I'm not going to give you all of them.
But, here's one way to think about it.
So, let's think geometrically about how to express this condition in terms of vectors.
So, let's take the origin O, by convention is the point (0,0,0).
And, let's take a point, P, that will satisfy this equation on it, so, at coordinates x, y, z.
So, what does this condition here mean?
Well, it means the following thing.
So, let's take the vector, OP.
OK, so vector OP, of course, has components x, y, z.
Now, we can think of this as actually a dot product between OP and a mysterious vector that won't remain mysterious for very long, namely, the vector one, two, three.
OK, so, this condition is the same as OP.A equals zero, right?
If I take the dot product OPdotA I get x times one plus y times two plus z times three.
But now, what does it mean that the dot product between OP and A is zero?
Well, it means that OP and A are perpendicular.
OK, so I have this vector, A. I'm not going to be able to draw it realistically.
Let's say it goes this way.
Then, a point, P, solves this equation exactly when the vector from O to P is perpendicular to A.
And, I claim that defines a plane.
For example, if it helps you to see it, take a vertical vector.
What does it mean to be perpendicular to the vertical vector?
It means you are horizontal.
It's the horizontal plane.
Here, it's a plane that passes through the origin and is perpendicular to this vector, A. OK, so what we get is a plane through the origin perpendicular to A.
And, in general, what you should remember is that two vectors have a dot product equal to zero if and only if that's equivalent to the cosine of the angle between them is zero.
That means the angle is 90°.
That means A and B are perpendicular.
So, we have a very fast way of checking whether two vectors are perpendicular.
So, one additional application I think we'll see actually tomorrow is to find the components of a vector along a certain direction.
So, I claim we can use this intuition I gave about dot product telling us how much to vectors go in the same direction to actually give a precise meaning to the notion of component for vector, not just along the x, y, or z axis, but along any direction in space.
So, I think I should probably stop here.
But, I will see you tomorrow at 2:00 here, and we'll learn more about that and about cross products.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.mit.edu.
Thank you.
Let's continue with vectors and operations of them.
Remember we saw the topic yesterday was dot product.
And remember the definition of dot product, well, the dot product of two vectors is obtained by multiplying the first component with the first component, the second with the second and so on and summing these and you get the scalar.
And the geometric interpretation of that is that you can also take the length of A, take the length of B multiply them and multiply that by the cosine of the angle between the two vectors.
We have seen several applications of that.
One application is to find lengths and angles.
For example, you can use this relation to give you the cosine of the angle between two vectors is the dot product divided by the product of the lengths.
Another application that we have is to detect whether two vectors are perpendicular.
To decide if two vectors are perpendicular to each other, all we have to do is compute our dot product and see if we get zero.
And one further application that we did not have time to discuss yesterday that I will mention very quickly is to find components of, let's say, a vector A along a direction u.
So some unit vector.
Let me explain.
Let's say that I have some direction.
For example, the horizontal axis on this blackboard.
But it could be any direction in space.
And, to describe this direction, maybe I have a unit vector along this axis.
Let's say that I have any of a vector A and I want to find out what is the component of A along u.
That means what is the length of this projection of A to the given direction?
This thing here is the component of A along u.
Well, how do we find that?
Well, we know that here we have a right angle.
So this component is just length A times cosine of the angle between A and u.
But now that means I should be able to compute it very easily because that's the same as length A times length u times cosine theta because u is a unit vector.
It is a unit vector.
That means this is equal to one.
And so that's the same as the dot product between A and u.
That is very easy.
And, of course, the most of just cases of that is say, for example, we want just to find the component along i hat, the unit vector along the x axis.
Then you do the dot product with i hat, which is 100.
What you get is the first component.
And that is, indeed, the x component of a vector.
Similarly, say you want the z component you do the dot product with k that gives you the last component of your vector.
But the same works with a unit vector in any direction.
So what is an application of that?
Well, for example, in physics maybe you have seen situations where you have a pendulum that swings.
You have maybe some mass at the end of the string and that mass swings back and forth on a circle.
And to analyze this mechanically you want to use, of course, Newton's Laws of Mechanics and you want to use forces and so on, but I claim that components of vectors are useful here to understand what happens geometrically.
What are the forces exerted on this pendulum?
Well, there is its weight, which usually points downwards, and there is the tension of the string.
And these two forces together are what explains how this pendulum is going to move back and forth.
Now, you could try to understand the equations of motion using x, y coordinates or x, z or whatever you want to call them, let's say x, y.
But really what causes the pendulum to swing back and forth and also to somehow stay a constant distance are phenomenal relative to this circular trajectory.
For example, maybe instead of taking components along the x and y axis, we want to look at two other unit vectors.
We can look at a vector, let's call it T, that is tangent to the trajectory.
Sorry.
Can you read that?
It's not very readable.
T is tangent to the trajectory.
And, on the other hand, we can introduce another vector.
Let's call that N. And that one is normal, perpendicular to the trajectory.
And so now if you think about it you can look at the components of the weight along the tangent direction and along the normal direction.
And so the component of F along the tangent direction is what causes acceleration in the direction along the trajectory.
It is what causes the pendulum to swing back and forth.
And the component along N, on the other hand.
That is the part of the weight that tends to pull our mass away from this point.
It is what is going to be responsible for the tension of the string.
It is why the string is taut and not actually slack and with things moving all over the place.
That one is responsible for the tension of a string.
And now, of course, if you want to compute things, well, maybe you will call this angle theta and then you will express things explicitly using sines and cosines and you will solve for the equations of motion.
That would be a very interesting physics problem.
But, to save time, we are not going to do it.
I'm sure you've seen that in 8.01 or similar classes.
And so to find these components we will just do dot products.
Any questions?
No.
OK. Let's move onto our next topic.
Here we have found things about lengths, angles and stuff like that.
One important concept that we have not understood yet in terms of vectors is area.
Let's say that we want to find the area of this pentagon.
Well, how do we compute that using vectors?
Can we do it using vectors?
Yes we can.
And that is going to be the goal.
The first thing we should do is probably simplify the problem.
We don't actually need to bother with pentagons.
All we need to know are triangles because, for example, you can cut that in three triangles and then sum the areas of the triangles.
Perhaps easier, what is the area of a triangle?
Let's start with a triangle in the plane.
Well, then we need two vectors to describe it, say A and B here.
How do we find the area of a triangle?
Well, we all know base times height over two.
What is the base?
What is the height?
The area of this triangle is going to be one-half of the base, which is going to be the length of A.
And the height, well, if you call theta this angle, then this is length B sine theta.
Now, that looks a lot like the formula we had there, except for one little catch.
This is a sine instead of a cosine.
How do we deal with that?
Well, what we could do is first find the cosine of the angle.
We know how to find the cosine of the angle using dot products.
Then solve for sine using sine square plus cosine square equals one.
And then plug that back into here.
Well, that works but it is kind of a very complicated way of doing it.
So there is an easier way.
And that is going to be determinants, but let me explain how we get to that maybe still doing elementary geometry and dot products first.
Let's see.
What we can do is instead of finding the sine of theta, well, we're not good at finding sines of angles but we are very good now at finding cosines of angles.
Maybe we can find another angle whose cosine is the same as the sine of theta.
Well, you have already heard about complimentary angles and how I take my vector A, my vector B here and I have an angle theta.
Well, let's say that I rotate my vector A by 90 degrees to get a new vector A prime.
A prime is just A rotated by 90 degrees.
Then the angle between these two guys, let's say theta prime, well, theta prime is 90 degrees or pi over two gradients minus theta.
So, in particular, cosine of theta prime is equal to sine of theta.
In particular, that means that length A, length B, sine theta, which is what we would need to know in order to find the area of this triangle is equal to, well, A and A prime have the same length so let me replace that by length of A prime.
I am not changing anything, length B, cosine theta prime.
And now we have something that is much easier for us.
Because that is just A prime dot B.
That looks like a very good plan.
There is only one small thing which is we don't know yet how to find this A prime.
Well, I think it is not very hard.
Let's see.
Actually, why don't you guys do the hard work?
Let's say that I have a plane vector A with two components a1, a2.
And I want to rotate it counterclockwise by 90 degrees.
It looks like maybe we should change some signs somewhere.
Maybe we should do something with the components.
Can you come up with an idea of what it might be?
I see a lot of people answering three.
I see some other answers, but the majority vote seems to be number three.
Minus a2 and a1.
I think I agree, so let's see.
Let's say that we have this vector A with components a1.
So a1 is here.
And a2.
So a2 is here.
Let's rotate this box by 90 degrees counterclockwise.
This box ends up there.
It's the same box just flipped on its side.
This thing here becomes a1 and this thing here becomes a2.
And that means our new vector A prime is going to be -- Well, the first component looks like an a2 but it is pointing to the left when a2 is positive.
So, actually, it is minus a2.
And the y component is going to be the same as this guy, so it's going to be a1.
If you wanted instead to rotate clockwise then you would do the opposite.
You would do a2 minus a1.
Is that reasonably clear for everyone?
OK. Let's continue the calculation there.
A prime, we have decided, is minus a2, a1 dot product with let's call b1 and b2, the components of B.
Then that will be minus a2, b1 plus a1, b2 plus a1, b2.
Let me write that the other way around, a1, b2 minus a2, b1.
And that is a quantity that you may already know under the name of determinant of vectors A and B, which we write symbolically using this notation.
We put A and B next to each other inside a two-by-two table and we put these verticals bars.
And that means the determinant of these numbers, this guy times this guy minus this guy times this guy.
That is called the determinant.
And geometrically what it measures is the area, well, not of a triangle because we did not divide by two, but of a parallelogram formed by A and B.
It measures the area of the parallelogram with sides A and B.
And, of course, if you want the triangle then you will just divide by two.
The triangle is half the parallelogram.
There is one small catch.
The area usually is something that is going to be positive.
This guy here has no reason to be positive or negative because, in fact, well, if you compute things you will see that where it is supposed to go negative it depends on whether A and B are clockwise or counterclockwise from each other.
I mean the issue that we have -- Well, when we say the area is one-half length A, length B, sine theta that was assuming that theta is positive, that its sine is positive.
Otherwise, if theta is negative maybe we need to take the absolute value of this.
Just to be more truthful, I will say the determinant is either plus or minus the area.
Any questions about this?
Yes.
Sorry.
That is not a dot product.
That is the usual multiplication.
That is length A times length B times sine theta.
What does that equal?
And so that is equal to the area of a parallelogram.
Sorry.
Let me explain that again.
If I have two vectors A and B, I can form a parallelogram with them or I can form a triangle.
And so the area of a parallelogram is equal to length A, length B, sine theta, is equal to the determinant of A and B.
While the area of a triangle is one-half of that.
And, again, to be truthful, I should say these things can be positive or negative.
Depending on whether you count the angle positively or negatively, you will get either the area or minus the area.
The area is actually the absolute value of these quantities.
Is that clear?
OK.
Yes.
If you want to compute the area, you will just take the absolute value of the determinant.
I should say the area of a parallelogram so that it is completely clear.
Sorry.
Do you have a question?
Explain again, sorry, was the question how a determinant equals the area of a parallelogram?
OK.
The area of a parallelogram is going to be the base times the height.
Let's take this guy to be the base.
The length of a base will be length of A and the height will be obtained by taking B but only looking at the vertical part.
That will be length of B times the sine of theta.
That is how I got the area of a parallelogram as length A, length B, sine theta.
And then I did this manipulation and this trick of rotating to find a nice formula.
Yes.
You are asking ahead of what I am going to do in a few minutes.
You are asking about magnitude of A cross B.
We are going to learn about cross products in a few minutes.
And the answer is yes, but cross product is for vectors in space.
Here I was simplifying things by doing things just in the plane.
Just bear with me for five more minutes and we will do things in space.
Yes.
That is correct.
The way you compute this in practice is you just do this.
That is how you compute the determinant.
Yes.
What about three dimensions?
Three dimensions we are going to do now.
More questions?
Should we move on?
OK. Let's move to space.
There are two things we can do in space.
And you can look for the volume of solids or you can look for the area of surfaces.
Let me start with the easier of the two.
Let me start with volumes of solids.
And we will go back to area, I promise.
I claim that there is also a notion of determinants in space.
And that is going to tell us how to find volumes.
Let's say that we have three vectors A, B and C. And then the definition of their determinants going to be, the notation for that in terms of the components is the same as over there.
We put the components of A, the components of B and the components of C inside verticals bars.
And, of course, I have to give meaning to this.
This will be a number.
And what is that number?
Well, the definition I will take is that this is a1 times the determinant of what I get by looking in this lower right corner.
The two-by-two determinant b2, b3, c2, c3.
Then I will subtract a2 times the determinant of b1, b3, c1, c3.
And then I will add a3 times the determinant b1, b2, c1, c2.
And each of these guys means, again, you take b2 times c3 minus c2 times b3 and this times that minus this time that and so on.
In fact, there is a total of six terms in here.
And maybe some of you have already seen a different formula for three-by-three determinants where you directly have the six terms.
It is the same definition.
How to remember the structure of this formula?
Well, this is called an expansion according to the first row.
So we are going to take the entries in the first row, a1, a2, a3 And for each of them we get the term.
Namely we multiply it by a two-by-two determinant that we get by deleting the first row and the column where we are.
Here the coefficient next to a1, when we delete this column and this row, we are left with b2, b3, c2, c3.
The next one we take a2, we delete the row that is in it and the column that it is in.
And we are left with b1, b3, c1, c3.
And, similarly, with a3, we take what remains, which is b1, b2, c1, c2.
Finally, last but not least, there is a minus sign here for the second guy.
It looks like a weird formula.
I mean it is a little bit weird.
But it is a formula that you should learn because it is really, really useful for a lot of things.
I should say if this looks very artificial to you and you would like to know more there is more in the notes, so read the notes.
They will tell you a bit more about what this means, where it comes from and so on.
If you want to know a lot more then some day you should take 18.06, Linear Algebra where you will learn a lot more about determinants in N dimensional space with N vectors.
And there is a generalization of this in arbitrary dimensions.
In this class, we will only deal with two or three dimensions.
Yes.
Why is the negative there?
Well, that is a very good question.
It has to be there so that this will actually equal, well, what I am going to say right now is that this will give us the volume of [a box?]
with sides A, B, C. And the formula just doesn't work if you don't put the negative.
There is a more fundamental reason which has to do with orientation of space and the fact that if you switch two coordinates in space then basically you change what is called the handedness of the coordinates.
If you look at your right hand and your left hand, they are not actually the same.
They are mirror images.
And, if you squared two coordinate axes, that is what you get.
That is the fundamental reason for the minus.
Again, we don't need to think too much about that.
All we need in this class is the formula.
Why do we care about this formula?
It is because of the theorem that says that geometrically the determinant of the three vectors A, B, C is, again, plus or minus.
This determinant could be positive or negative.
See those minuses and all sorts of stuff.
Plus or minus the volume of the parallelepiped.
That is just a fancy name for a box with parallelogram sides, in case you wonder, with sides A, B and C. You take the three vectors A, B and C and you form a box whose sides are all parallelograms.
And when its volume is going to be the determinant.
Other questions?
I'm sorry.
I cannot quite hear you.
Yes.
We are going to see how to do it geometrically without a determinant, but then you will see that you actually need a determinant to compute it no matter what.
We are going to go back to this and see another formula for volume, but you will see that really I am cheating.
I mean somehow computationally the only way to compute it is really to use a determinant.
That is correct.
In general, I mean, actually, I could say if you look at the two-by-two determinant, see, you can also explain it in terms of this extension.
If you take a1 and multiply by this one-by-one determinant b2, then you take a2 and you multiply it by this one-by-one determinant b1 but you put a minus sign.
And in general, indeed, when you expand, you would stop putting plus, minus, plus, minus alternating.
More about that in 18.06.
Yes.
There is a way to do it based on other rows as well, but then you have to be very careful with the sign vectors.
I will refer you to the notes for that.
I mean you could also do it with a column, by the way.
I mean be careful about the sign rules.
Given how little we will use determinants in this class, I mean we will use them in a way that is fundamental, but we won't compute much.
Let's say this is going to be enough for us for now.
After determinants now I can tell you about cross product.
And cross product is going to be the answer to your question about area.
OK. Let me move onto cross product.
Cross product is something that you can apply to two vectors in space.
And by that I mean really in three-dimensional space.
This is something that is specific to three dimensions.
The definition A cross B -- It is important to really do your multiplication symbol well so that you don't mistake it with a dot product.
Well, that is going to be a vector.
That is another reason not to confuse it with dot product.
Dot product gives you a number.
Cross product gives you a vector.
They are really completely different operations.
They are both called product because someone could not come up with a better name, but they are completely different operations.
What do we do to do the cross product of A and B?
Well, we do something very strange.
Just as I have told you that a determinant is something where we put numbers and we get a number, I am going to violate my own rule.
I am going to put together a determinant in which -- Well, the last two rows are the components of the vectors A and B but the first row strangely consists for unit vectors i, j, k. What does that mean?
Well, that is not a determinant in the usual sense.
If you try to put that into your calculator, it will tell you there is an error.
I don't know how to put vectors in there.
I want numbers.
What is means is it is symbolic notation that helps you remember what the formula is.
The actual formula is, well, you use this definition.
And, if you use that definition, you see that it is i hat times some number.
Let me write it as determinant of a2, a3, b2, b3 times i hat minus determinant a1, a3, b1, b3, j hat plus a1, a2, b1, b2, k hat.
And so that is the actual definition in a way that makes complete sense, but to remember this formula without too much trouble it is much easier to think about it in these terms here.
That is the definition and it gives you a vector.
Now, as usual with definitions, the question is what is it good for?
What is the geometric meaning of this very strange operation?
Why do we bother to do that?
Here is what it does geometrically.
Remember a vector has two different things.
It has a length and it has a direction.
Let's start with the length.
A length of a cross product is the area of the parallelogram in space formed by the vectors A and B.
Now, if you have a parallelogram in space, you can find its area just by doing this calculation when you know the coordinates of the points.
You do this calculation and then you take the length.
You take this squared plus that squared plus that squared, square root.
It looks like a very complicated formula but it works and, actually, it is the simplest way to do it.
This time we don't actually need to put plus or minus because the length of a vector is always positive.
We don't have to worry about that.
And what is even more magical is that not only is the length remarkable but the direction is also remarkable.
The direction of A cross B is perpendicular to the plane of a parallelogram.
Our two vectors A and B together in a plane.
What I am telling you is that for vector A cross B will point, will stick straight out of that plane perpendicularly to it.
In fact, I would have to be more precise.
There are two ways that you can be perpendicular to this plane.
You can be perpendicular pointing up or pointing down.
How do I decide which?
Well, there is something called the right-hand rule.
What does the right-hand rule say?
Well, there are various versions for right-hand rule depending on which country you learn about it.
In France, given the culture, you even learn about it in terms of a cork screw and a wine bottle.
I will just use the usual version here.
You take your right hand.
If you are left-handed, remember to take your right hand and not the left one.
The other right, OK?
Then place your hand to point in the direction of A.
Let's say my right hand is going in that direction.
Now, curl your fingers so that they point towards B.
Here that would be kind of into the blackboard.
Don't snap any bones.
If it doesn't quite work then rotate your arms so that you can actually physically do it.
Then get your thumb to stick straight out.
Well, here my thumb is going to go up.
And that tells me that A cross B will go up.
Let me write that down while you experiment with it.
Again, try not to enjoy yourselves.
First, your right hand points parallel to vector A.
Then your fingers point in the direction of B.
Then your thumb, when you stick it out, is going to point in the direction of A cross B.
Let's do a quick example.
Where is my quick example?
Here.
Let's take i cross j. I see most of you going in the right direction.
If you have it pointing in the wrong direction, it might mean that you are using your left hand, for example.
Example, I claim that i cross j equals k. Let's see.
I points towards us.
J point to our right.
I guess this is your right.
I think.
And then your thumb is going to point up.
That tells us it is roughly pointing up.
And, of course, the length should be one because if you take the unit square in the x, y plane, its area is one.
And the direction should be vertical.
Because it should be perpendicular to the x, y plane.
It looks like i cross j will be k. Well, let's check with the definition i, j, k. What is i?
I is one, zero, zero.
J is zero, one, zero.
The coefficient of i will be zero times zero minus zero times one.
That is zero.
The coefficient of j will be one time zero minus zero times zero, that is a zero, minus zero j.
It doesn't matter.
And the coefficient of k will be one times one, that is one, minus zero times zero, so one k. So we do get i cross j equals k both ways.
In this case, it is easier to do it geometrically.
If I give you no complicated vectors, probably you will actually want to do the calculation.
Any questions?
Yes.
The coefficient of k, remember I delete the first row and the last column so I get this two-by-two determinant.
And that two-by-two determinant is one times one minus zero times zero so that gives me a one.
That is what you do with two-by-two determinants.
Similarly for [UNINTELLIGIBLE], but [UNINTELLIGIBLE] turn out to be zero.
More questions?
Yes.
Let me repeat how I got the one in front of k. Remember the definition of a determinant I expand according to the entries in the first row.
When I get to k what I do is delete the first row and I delete the last column, the column that contains k. I delete these guys and these guys and I am left with this two-by-two determinant.
Now, a two-by-two determinant, you multiply according to this downward diagonal and then minus this times that.
One times one, let me see here, I got one k because that is one times one minus zero times zero equals one.
Sorry.
That is really hard to read.
Maybe it will be easier that way.
Yes.
Let's try.
If I do the same for i, I think I will also get zero.
Let's do the same for i. I take i, I delete the first row, I delete the first column, I get this two-by-two determinant here and I get zero times zero, that is zero, minus zero times one.
That is the other trick question.
Zero times one is zero as well.
So that zero minus zero is zero.
I hope on Monday you should get more practice in recitation about how to compute determinants.
Hopefully, it will become very easy for you all to compute this next.
I know the first time it is kind of a shock because there are a lot of numbers and a lot of things to do.
Let me return to the question that you asked a bit earlier about how do you find actually volume if I don't want to know about determinants?
Well, let's have another look at the volume.
Let's say that I have three vectors.
Let me put them this way, A, B and C. And let's try to see how else I could think about the volume of this box.
Probably you know that the volume of a parallelepiped is the area of a base times the height.
Sorry.
The volume is the area of a base times the height.
How do we do that in practice?
Well, what is the area of a base?
The base is a parallelogram in space with sides B and C. How do we find the area of the parallelogram in space?
Well, we just discovered that.
We can do it by taking that cross product.
The area of a base, well, we take the cross product of B and C. That is not quite it because this is a vector.
We would like a number while we take its length.
That is pretty good.
What about the height?
Well, the height is going to be the component of A in the direction that is perpendicular to the base.
Let's take a direction that is perpendicular to the base.
Let's call that N, a unit vector in that direction.
Then we can get the height by taking A dot n. That is what we saw at the beginning of class that A dot n will tell me how much A goes in the direction of n. Are you still with me?
OK. Let's keep going.
Let's think about this vector n. How do I get it?
Well, I can get it by actually using cross product as well.
Because I said the direction perpendicular to two vectors I can get by taking that cross product and looking at that direction.
This is still B cross C length.
And this one is, so I claim, n can be obtained by taking D cross C. Well, that comes in the right direction but it is not a unit vector.
How do I get a unit vector?
I divide by the length.
Thanks.
I take B cross C and I divide by length B cross C. Well, now I can probably simplify between these two guys.
And so what I will get -- What I get out of this is that my volume equals A dot product with vector B cross C. But, of course, I have to be careful in which order I do it.
If I do it the other way around, A dot B, I get a number.
I cannot cross that.
I really have to do the cross product first.
I get the new vector.
Then my dot product.
The fact is that the determinant of A, B, C is equal to this so-called triple product.
Well, that looks good geometrically.
Let's try to check whether it makes sense with the formulas, just one small thing.
We saw the determinant is a1 times determinant b2, b3, c2, c3 minus a2 times something plus a3 times something.
I will let you fill in the numbers.
That is this guy.
What about this guy?
Well, dot product, we take the first component of A, that is a1, we multiply by the first component of B cross C. What is the first component of B cross C?
Well, it is this determinant b2, b3, c2, c3.
If you put B and C instead of A and B into there you will get the i component is this guy plus a2 times the second component which is minus some determinant plus a3 times the third component which is, again, a determinant.
And you can check.
You get exactly the same expression, so everything is fine.
There is no contradiction in math just yet.
On Tuesday we will continue with this and we will start going into matrices, equations of planes and so on.
Meanwhile, have a good weekend and please start working on your Problem Sets so that you can ask lots of questions to your TAs on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.mit.edu.
Thank you.
Let's continue with vectors and operations of them.
Remember we saw the topic yesterday was dot product.
And remember the definition of dot product, well, the dot product of two vectors is obtained by multiplying the first component with the first component, the second with the second and so on and summing these and you get the scalar.
And the geometric interpretation of that is that you can also take the length of A, take the length of B multiply them and multiply that by the cosine of the angle between the two vectors.
We have seen several applications of that.
One application is to find lengths and angles.
For example, you can use this relation to give you the cosine of the angle between two vectors is the dot product divided by the product of the lengths.
Another application that we have is to detect whether two vectors are perpendicular.
To decide if two vectors are perpendicular to each other, all we have to do is compute our dot product and see if we get zero.
And one further application that we did not have time to discuss yesterday that I will mention very quickly is to find components of, let's say, a vector A along a direction u.
So some unit vector.
Let me explain.
Let's say that I have some direction.
For example, the horizontal axis on this blackboard.
But it could be any direction in space.
And, to describe this direction, maybe I have a unit vector along this axis.
Let's say that I have any of a vector A and I want to find out what is the component of A along u.
That means what is the length of this projection of A to the given direction?
This thing here is the component of A along u.
Well, how do we find that?
Well, we know that here we have a right angle.
So this component is just length A times cosine of the angle between A and u.
But now that means I should be able to compute it very easily because that's the same as length A times length u times cosine theta because u is a unit vector.
It is a unit vector.
That means this is equal to one.
And so that's the same as the dot product between A and u.
That is very easy.
And, of course, the most of just cases of that is say, for example, we want just to find the component along i hat, the unit vector along the x axis.
Then you do the dot product with i hat, which is 100.
What you get is the first component.
And that is, indeed, the x component of a vector.
Similarly, say you want the z component you do the dot product with k that gives you the last component of your vector.
But the same works with a unit vector in any direction.
So what is an application of that?
Well, for example, in physics maybe you have seen situations where you have a pendulum that swings.
You have maybe some mass at the end of the string and that mass swings back and forth on a circle.
And to analyze this mechanically you want to use, of course, Newton's Laws of Mechanics and you want to use forces and so on, but I claim that components of vectors are useful here to understand what happens geometrically.
What are the forces exerted on this pendulum?
Well, there is its weight, which usually points downwards, and there is the tension of the string.
And these two forces together are what explains how this pendulum is going to move back and forth.
Now, you could try to understand the equations of motion using x, y coordinates or x, z or whatever you want to call them, let's say x, y.
But really what causes the pendulum to swing back and forth and also to somehow stay a constant distance are phenomenal relative to this circular trajectory.
For example, maybe instead of taking components along the x and y axis, we want to look at two other unit vectors.
We can look at a vector, let's call it T, that is tangent to the trajectory.
Sorry.
Can you read that?
It's not very readable.
T is tangent to the trajectory.
And, on the other hand, we can introduce another vector.
Let's call that N. And that one is normal, perpendicular to the trajectory.
And so now if you think about it you can look at the components of the weight along the tangent direction and along the normal direction.
And so the component of F along the tangent direction is what causes acceleration in the direction along the trajectory.
It is what causes the pendulum to swing back and forth.
And the component along N, on the other hand.
That is the part of the weight that tends to pull our mass away from this point.
It is what is going to be responsible for the tension of the string.
It is why the string is taut and not actually slack and with things moving all over the place.
That one is responsible for the tension of a string.
And now, of course, if you want to compute things, well, maybe you will call this angle theta and then you will express things explicitly using sines and cosines and you will solve for the equations of motion.
That would be a very interesting physics problem.
But, to save time, we are not going to do it.
I'm sure you've seen that in 8.01 or similar classes.
And so to find these components we will just do dot products.
Any questions?
No.
OK. Let's move onto our next topic.
Here we have found things about lengths, angles and stuff like that.
One important concept that we have not understood yet in terms of vectors is area.
Let's say that we want to find the area of this pentagon.
Well, how do we compute that using vectors?
Can we do it using vectors?
Yes we can.
And that is going to be the goal.
The first thing we should do is probably simplify the problem.
We don't actually need to bother with pentagons.
All we need to know are triangles because, for example, you can cut that in three triangles and then sum the areas of the triangles.
Perhaps easier, what is the area of a triangle?
Let's start with a triangle in the plane.
Well, then we need two vectors to describe it, say A and B here.
How do we find the area of a triangle?
Well, we all know base times height over two.
What is the base?
What is the height?
The area of this triangle is going to be one-half of the base, which is going to be the length of A.
And the height, well, if you call theta this angle, then this is length B sine theta.
Now, that looks a lot like the formula we had there, except for one little catch.
This is a sine instead of a cosine.
How do we deal with that?
Well, what we could do is first find the cosine of the angle.
We know how to find the cosine of the angle using dot products.
Then solve for sine using sine square plus cosine square equals one.
And then plug that back into here.
Well, that works but it is kind of a very complicated way of doing it.
So there is an easier way.
And that is going to be determinants, but let me explain how we get to that maybe still doing elementary geometry and dot products first.
Let's see.
What we can do is instead of finding the sine of theta, well, we're not good at finding sines of angles but we are very good now at finding cosines of angles.
Maybe we can find another angle whose cosine is the same as the sine of theta.
Well, you have already heard about complimentary angles and how I take my vector A, my vector B here and I have an angle theta.
Well, let's say that I rotate my vector A by 90 degrees to get a new vector A prime.
A prime is just A rotated by 90 degrees.
Then the angle between these two guys, let's say theta prime, well, theta prime is 90 degrees or pi over two gradients minus theta.
So, in particular, cosine of theta prime is equal to sine of theta.
In particular, that means that length A, length B, sine theta, which is what we would need to know in order to find the area of this triangle is equal to, well, A and A prime have the same length so let me replace that by length of A prime.
I am not changing anything, length B, cosine theta prime.
And now we have something that is much easier for us.
Because that is just A prime dot B.
That looks like a very good plan.
There is only one small thing which is we don't know yet how to find this A prime.
Well, I think it is not very hard.
Let's see.
Actually, why don't you guys do the hard work?
Let's say that I have a plane vector A with two components a1, a2.
And I want to rotate it counterclockwise by 90 degrees.
It looks like maybe we should change some signs somewhere.
Maybe we should do something with the components.
Can you come up with an idea of what it might be?
I see a lot of people answering three.
I see some other answers, but the majority vote seems to be number three.
Minus a2 and a1.
I think I agree, so let's see.
Let's say that we have this vector A with components a1.
So a1 is here.
And a2.
So a2 is here.
Let's rotate this box by 90 degrees counterclockwise.
This box ends up there.
It's the same box just flipped on its side.
This thing here becomes a1 and this thing here becomes a2.
And that means our new vector A prime is going to be -- Well, the first component looks like an a2 but it is pointing to the left when a2 is positive.
So, actually, it is minus a2.
And the y component is going to be the same as this guy, so it's going to be a1.
If you wanted instead to rotate clockwise then you would do the opposite.
You would do a2 minus a1.
Is that reasonably clear for everyone?
OK. Let's continue the calculation there.
A prime, we have decided, is minus a2, a1 dot product with let's call b1 and b2, the components of B.
Then that will be minus a2, b1 plus a1, b2 plus a1, b2.
Let me write that the other way around, a1, b2 minus a2, b1.
And that is a quantity that you may already know under the name of determinant of vectors A and B, which we write symbolically using this notation.
We put A and B next to each other inside a two-by-two table and we put these verticals bars.
And that means the determinant of these numbers, this guy times this guy minus this guy times this guy.
That is called the determinant.
And geometrically what it measures is the area, well, not of a triangle because we did not divide by two, but of a parallelogram formed by A and B.
It measures the area of the parallelogram with sides A and B.
And, of course, if you want the triangle then you will just divide by two.
The triangle is half the parallelogram.
There is one small catch.
The area usually is something that is going to be positive.
This guy here has no reason to be positive or negative because, in fact, well, if you compute things you will see that where it is supposed to go negative it depends on whether A and B are clockwise or counterclockwise from each other.
I mean the issue that we have -- Well, when we say the area is one-half length A, length B, sine theta that was assuming that theta is positive, that its sine is positive.
Otherwise, if theta is negative maybe we need to take the absolute value of this.
Just to be more truthful, I will say the determinant is either plus or minus the area.
Any questions about this?
Yes.
Sorry.
That is not a dot product.
That is the usual multiplication.
That is length A times length B times sine theta.
What does that equal?
And so that is equal to the area of a parallelogram.
Sorry.
Let me explain that again.
If I have two vectors A and B, I can form a parallelogram with them or I can form a triangle.
And so the area of a parallelogram is equal to length A, length B, sine theta, is equal to the determinant of A and B.
While the area of a triangle is one-half of that.
And, again, to be truthful, I should say these things can be positive or negative.
Depending on whether you count the angle positively or negatively, you will get either the area or minus the area.
The area is actually the absolute value of these quantities.
Is that clear?
OK.
Yes.
If you want to compute the area, you will just take the absolute value of the determinant.
I should say the area of a parallelogram so that it is completely clear.
Sorry.
Do you have a question?
Explain again, sorry, was the question how a determinant equals the area of a parallelogram?
OK.
The area of a parallelogram is going to be the base times the height.
Let's take this guy to be the base.
The length of a base will be length of A and the height will be obtained by taking B but only looking at the vertical part.
That will be length of B times the sine of theta.
That is how I got the area of a parallelogram as length A, length B, sine theta.
And then I did this manipulation and this trick of rotating to find a nice formula.
Yes.
You are asking ahead of what I am going to do in a few minutes.
You are asking about magnitude of A cross B.
We are going to learn about cross products in a few minutes.
And the answer is yes, but cross product is for vectors in space.
Here I was simplifying things by doing things just in the plane.
Just bear with me for five more minutes and we will do things in space.
Yes.
That is correct.
The way you compute this in practice is you just do this.
That is how you compute the determinant.
Yes.
What about three dimensions?
Three dimensions we are going to do now.
More questions?
Should we move on?
OK. Let's move to space.
There are two things we can do in space.
And you can look for the volume of solids or you can look for the area of surfaces.
Let me start with the easier of the two.
Let me start with volumes of solids.
And we will go back to area, I promise.
I claim that there is also a notion of determinants in space.
And that is going to tell us how to find volumes.
Let's say that we have three vectors A, B and C. And then the definition of their determinants going to be, the notation for that in terms of the components is the same as over there.
We put the components of A, the components of B and the components of C inside verticals bars.
And, of course, I have to give meaning to this.
This will be a number.
And what is that number?
Well, the definition I will take is that this is a1 times the determinant of what I get by looking in this lower right corner.
The two-by-two determinant b2, b3, c2, c3.
Then I will subtract a2 times the determinant of b1, b3, c1, c3.
And then I will add a3 times the determinant b1, b2, c1, c2.
And each of these guys means, again, you take b2 times c3 minus c2 times b3 and this times that minus this time that and so on.
In fact, there is a total of six terms in here.
And maybe some of you have already seen a different formula for three-by-three determinants where you directly have the six terms.
It is the same definition.
How to remember the structure of this formula?
Well, this is called an expansion according to the first row.
So we are going to take the entries in the first row, a1, a2, a3 And for each of them we get the term.
Namely we multiply it by a two-by-two determinant that we get by deleting the first row and the column where we are.
Here the coefficient next to a1, when we delete this column and this row, we are left with b2, b3, c2, c3.
The next one we take a2, we delete the row that is in it and the column that it is in.
And we are left with b1, b3, c1, c3.
And, similarly, with a3, we take what remains, which is b1, b2, c1, c2.
Finally, last but not least, there is a minus sign here for the second guy.
It looks like a weird formula.
I mean it is a little bit weird.
But it is a formula that you should learn because it is really, really useful for a lot of things.
I should say if this looks very artificial to you and you would like to know more there is more in the notes, so read the notes.
They will tell you a bit more about what this means, where it comes from and so on.
If you want to know a lot more then some day you should take 18.06, Linear Algebra where you will learn a lot more about determinants in N dimensional space with N vectors.
And there is a generalization of this in arbitrary dimensions.
In this class, we will only deal with two or three dimensions.
Yes.
Why is the negative there?
Well, that is a very good question.
It has to be there so that this will actually equal, well, what I am going to say right now is that this will give us the volume of [a box?]
with sides A, B, C. And the formula just doesn't work if you don't put the negative.
There is a more fundamental reason which has to do with orientation of space and the fact that if you switch two coordinates in space then basically you change what is called the handedness of the coordinates.
If you look at your right hand and your left hand, they are not actually the same.
They are mirror images.
And, if you squared two coordinate axes, that is what you get.
That is the fundamental reason for the minus.
Again, we don't need to think too much about that.
All we need in this class is the formula.
Why do we care about this formula?
It is because of the theorem that says that geometrically the determinant of the three vectors A, B, C is, again, plus or minus.
This determinant could be positive or negative.
See those minuses and all sorts of stuff.
Plus or minus the volume of the parallelepiped.
That is just a fancy name for a box with parallelogram sides, in case you wonder, with sides A, B and C. You take the three vectors A, B and C and you form a box whose sides are all parallelograms.
And when its volume is going to be the determinant.
Other questions?
I'm sorry.
I cannot quite hear you.
Yes.
We are going to see how to do it geometrically without a determinant, but then you will see that you actually need a determinant to compute it no matter what.
We are going to go back to this and see another formula for volume, but you will see that really I am cheating.
I mean somehow computationally the only way to compute it is really to use a determinant.
That is correct.
In general, I mean, actually, I could say if you look at the two-by-two determinant, see, you can also explain it in terms of this extension.
If you take a1 and multiply by this one-by-one determinant b2, then you take a2 and you multiply it by this one-by-one determinant b1 but you put a minus sign.
And in general, indeed, when you expand, you would stop putting plus, minus, plus, minus alternating.
More about that in 18.06.
Yes.
There is a way to do it based on other rows as well, but then you have to be very careful with the sign vectors.
I will refer you to the notes for that.
I mean you could also do it with a column, by the way.
I mean be careful about the sign rules.
Given how little we will use determinants in this class, I mean we will use them in a way that is fundamental, but we won't compute much.
Let's say this is going to be enough for us for now.
After determinants now I can tell you about cross product.
And cross product is going to be the answer to your question about area.
OK. Let me move onto cross product.
Cross product is something that you can apply to two vectors in space.
And by that I mean really in three-dimensional space.
This is something that is specific to three dimensions.
The definition A cross B -- It is important to really do your multiplication symbol well so that you don't mistake it with a dot product.
Well, that is going to be a vector.
That is another reason not to confuse it with dot product.
Dot product gives you a number.
Cross product gives you a vector.
They are really completely different operations.
They are both called product because someone could not come up with a better name, but they are completely different operations.
What do we do to do the cross product of A and B?
Well, we do something very strange.
Just as I have told you that a determinant is something where we put numbers and we get a number, I am going to violate my own rule.
I am going to put together a determinant in which -- Well, the last two rows are the components of the vectors A and B but the first row strangely consists for unit vectors i, j, k. What does that mean?
Well, that is not a determinant in the usual sense.
If you try to put that into your calculator, it will tell you there is an error.
I don't know how to put vectors in there.
I want numbers.
What is means is it is symbolic notation that helps you remember what the formula is.
The actual formula is, well, you use this definition.
And, if you use that definition, you see that it is i hat times some number.
Let me write it as determinant of a2, a3, b2, b3 times i hat minus determinant a1, a3, b1, b3, j hat plus a1, a2, b1, b2, k hat.
And so that is the actual definition in a way that makes complete sense, but to remember this formula without too much trouble it is much easier to think about it in these terms here.
That is the definition and it gives you a vector.
Now, as usual with definitions, the question is what is it good for?
What is the geometric meaning of this very strange operation?
Why do we bother to do that?
Here is what it does geometrically.
Remember a vector has two different things.
It has a length and it has a direction.
Let's start with the length.
A length of a cross product is the area of the parallelogram in space formed by the vectors A and B.
Now, if you have a parallelogram in space, you can find its area just by doing this calculation when you know the coordinates of the points.
You do this calculation and then you take the length.
You take this squared plus that squared plus that squared, square root.
It looks like a very complicated formula but it works and, actually, it is the simplest way to do it.
This time we don't actually need to put plus or minus because the length of a vector is always positive.
We don't have to worry about that.
And what is even more magical is that not only is the length remarkable but the direction is also remarkable.
The direction of A cross B is perpendicular to the plane of a parallelogram.
Our two vectors A and B together in a plane.
What I am telling you is that for vector A cross B will point, will stick straight out of that plane perpendicularly to it.
In fact, I would have to be more precise.
There are two ways that you can be perpendicular to this plane.
You can be perpendicular pointing up or pointing down.
How do I decide which?
Well, there is something called the right-hand rule.
What does the right-hand rule say?
Well, there are various versions for right-hand rule depending on which country you learn about it.
In France, given the culture, you even learn about it in terms of a cork screw and a wine bottle.
I will just use the usual version here.
You take your right hand.
If you are left-handed, remember to take your right hand and not the left one.
The other right, OK?
Then place your hand to point in the direction of A.
Let's say my right hand is going in that direction.
Now, curl your fingers so that they point towards B.
Here that would be kind of into the blackboard.
Don't snap any bones.
If it doesn't quite work then rotate your arms so that you can actually physically do it.
Then get your thumb to stick straight out.
Well, here my thumb is going to go up.
And that tells me that A cross B will go up.
Let me write that down while you experiment with it.
Again, try not to enjoy yourselves.
First, your right hand points parallel to vector A.
Then your fingers point in the direction of B.
Then your thumb, when you stick it out, is going to point in the direction of A cross B.
Let's do a quick example.
Where is my quick example?
Here.
Let's take i cross j. I see most of you going in the right direction.
If you have it pointing in the wrong direction, it might mean that you are using your left hand, for example.
Example, I claim that i cross j equals k. Let's see.
I points towards us.
J point to our right.
I guess this is your right.
I think.
And then your thumb is going to point up.
That tells us it is roughly pointing up.
And, of course, the length should be one because if you take the unit square in the x, y plane, its area is one.
And the direction should be vertical.
Because it should be perpendicular to the x, y plane.
It looks like i cross j will be k. Well, let's check with the definition i, j, k. What is i?
I is one, zero, zero.
J is zero, one, zero.
The coefficient of i will be zero times zero minus zero times one.
That is zero.
The coefficient of j will be one time zero minus zero times zero, that is a zero, minus zero j.
It doesn't matter.
And the coefficient of k will be one times one, that is one, minus zero times zero, so one k. So we do get i cross j equals k both ways.
In this case, it is easier to do it geometrically.
If I give you no complicated vectors, probably you will actually want to do the calculation.
Any questions?
Yes.
The coefficient of k, remember I delete the first row and the last column so I get this two-by-two determinant.
And that two-by-two determinant is one times one minus zero times zero so that gives me a one.
That is what you do with two-by-two determinants.
Similarly for [UNINTELLIGIBLE], but [UNINTELLIGIBLE] turn out to be zero.
More questions?
Yes.
Let me repeat how I got the one in front of k. Remember the definition of a determinant I expand according to the entries in the first row.
When I get to k what I do is delete the first row and I delete the last column, the column that contains k. I delete these guys and these guys and I am left with this two-by-two determinant.
Now, a two-by-two determinant, you multiply according to this downward diagonal and then minus this times that.
One times one, let me see here, I got one k because that is one times one minus zero times zero equals one.
Sorry.
That is really hard to read.
Maybe it will be easier that way.
Yes.
Let's try.
If I do the same for i, I think I will also get zero.
Let's do the same for i. I take i, I delete the first row, I delete the first column, I get this two-by-two determinant here and I get zero times zero, that is zero, minus zero times one.
That is the other trick question.
Zero times one is zero as well.
So that zero minus zero is zero.
I hope on Monday you should get more practice in recitation about how to compute determinants.
Hopefully, it will become very easy for you all to compute this next.
I know the first time it is kind of a shock because there are a lot of numbers and a lot of things to do.
Let me return to the question that you asked a bit earlier about how do you find actually volume if I don't want to know about determinants?
Well, let's have another look at the volume.
Let's say that I have three vectors.
Let me put them this way, A, B and C. And let's try to see how else I could think about the volume of this box.
Probably you know that the volume of a parallelepiped is the area of a base times the height.
Sorry.
The volume is the area of a base times the height.
How do we do that in practice?
Well, what is the area of a base?
The base is a parallelogram in space with sides B and C. How do we find the area of the parallelogram in space?
Well, we just discovered that.
We can do it by taking that cross product.
The area of a base, well, we take the cross product of B and C. That is not quite it because this is a vector.
We would like a number while we take its length.
That is pretty good.
What about the height?
Well, the height is going to be the component of A in the direction that is perpendicular to the base.
Let's take a direction that is perpendicular to the base.
Let's call that N, a unit vector in that direction.
Then we can get the height by taking A dot n. That is what we saw at the beginning of class that A dot n will tell me how much A goes in the direction of n. Are you still with me?
OK. Let's keep going.
Let's think about this vector n. How do I get it?
Well, I can get it by actually using cross product as well.
Because I said the direction perpendicular to two vectors I can get by taking that cross product and looking at that direction.
This is still B cross C length.
And this one is, so I claim, n can be obtained by taking D cross C. Well, that comes in the right direction but it is not a unit vector.
How do I get a unit vector?
I divide by the length.
Thanks.
I take B cross C and I divide by length B cross C. Well, now I can probably simplify between these two guys.
And so what I will get -- What I get out of this is that my volume equals A dot product with vector B cross C. But, of course, I have to be careful in which order I do it.
If I do it the other way around, A dot B, I get a number.
I cannot cross that.
I really have to do the cross product first.
I get the new vector.
Then my dot product.
The fact is that the determinant of A, B, C is equal to this so-called triple product.
Well, that looks good geometrically.
Let's try to check whether it makes sense with the formulas, just one small thing.
We saw the determinant is a1 times determinant b2, b3, c2, c3 minus a2 times something plus a3 times something.
I will let you fill in the numbers.
That is this guy.
What about this guy?
Well, dot product, we take the first component of A, that is a1, we multiply by the first component of B cross C. What is the first component of B cross C?
Well, it is this determinant b2, b3, c2, c3.
If you put B and C instead of A and B into there you will get the i component is this guy plus a2 times the second component which is minus some determinant plus a3 times the third component which is, again, a determinant.
And you can check.
You get exactly the same expression, so everything is fine.
There is no contradiction in math just yet.
On Tuesday we will continue with this and we will start going into matrices, equations of planes and so on.
Meanwhile, have a good weekend and please start working on your Problem Sets so that you can ask lots of questions to your TAs on Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Remember last time -- -- we learned about the cross product of vectors in space.
Remember the definition of cross product is in terms of this determinant det| i hat, j hat, k hat, and then the components of A, a1, a2, a3, and then the components of B| This is not an actual determinant because these are not numbers.
But it's a symbolic notation, to remember what the actual formula is.
The actual formula is obtained by expanding the determinant.
So, we actually get the determinant of a2, a3, b2, b3 times i hat, minus the determinant of a1, a3, b1, b3 times j hat plus the determinant of a1, a2, b1, b2, times k hat.
And we also saw a more geometric definition of the cross product.
We have learned that the length of the cross product is equal to the area of the parallelogram with sides A and B.
We have also learned that the direction of the cross product is given by taking the direction that's perpendicular to A and B.
If I draw A and B in a plane (they determine a plane), then the cross product should go in the direction that's perpendicular to that plane.
Now, there's two different possible directions that are perpendicular to a plane.
And, to decide which one it is, we use the right-hand rule, which says if you extend your right hand in the direction of the vector A, then curve your fingers in the direction of B, then your thumb will go in the direction of the cross product.
One thing I didn't quite get to say last time is that there are some funny manipulation rules.
What are we allowed to do or not do with cross products?
So, let me tell you right away the most surprising one if you've never seen it before: A cross B and B cross A are not the same thing.
Why are they not the same thing?
Well, one way to see it is to think geometrically.
The parallelogram still has the same area, and it's still in the same plane.
So, the cross product is still perpendicular to the same plane.
But, what happens is that, if you try to apply the right-hand rule but exchange the roles of A and B, then you will either injure yourself, or your thumb will end up pointing in the opposite direction.
So, in fact, B cross A and A cross B are opposite of each other.
And you can check that in the formula because, for example, the i component is a2 b3 minus a3 b2.
If you swap the roles of A and B, you will also have to change the signs.
That's a slightly surprising thing, but you will see one easily adjusts to it.
It just means one must resist the temptation to write AxB equals BxA.
Whenever you do that, put a minus sign.
Now, in particular, what happens if I do A cross A?
Well, I will get zero.
And, there's many ways to see that.
One is to use the formula.
Also, you can see that the parallelogram formed by A and A is completely flat, and it has area zero.
So, we get the zero vector.
Hopefully you got practice with cross products, and computing them, in recitation yesterday.
Let me just point out one important application of cross product that maybe you haven't seen yet.
Let's say that I'm given three points in space, and I want to find the equation of the plane that contains them.
So, say I have P1, P2, P3, three points in space.
They determine a plane, at least if they are not aligned, and we would like to find the equation of the plane that they determine.
That means, let's say that we have a point, P, in space with coordinates x, y, z.
Well, to find the equation of the plane -- -- the plane containing P1, P2, and P3, we need to find a condition on the coordinates x, y, z, telling us whether P is in the plane or not.
We have several ways of doing that.
For example, one thing we could do.
Let me just backtrack to determinants that we saw last time.
One way to think about it is to consider these vectors, P1P2, P1P3, and P1P.
The question of whether they are all in the same plane is the same as asking ourselves whether the parallelepiped that they form is actually completely flattened.
So, if I try to form a parallelepiped with these three sides, and P is not in the plane, then it will have some volume.
But, if P is in the plane, then it's actually completely squished.
So,one possible answer, one possible way to think of the equation of a plane is that the determinant of these vectors should be zero.
Take the determinant of (vector P1P,vector P1P2,vector P1P3) equals 0 (if you do it in a different order it doesn't really matter).
One possible way to express the condition that P is in the plane is to say that the determinant of these three vectors has to be zero.
And, if I am given coordinates for these points -- I'm not giving you numbers, but if I gave you numbers, then you would be able to plug those numbers in.
So, you could compute these two vectors P1P2 and P1P3 explicitly.
But, of course, P1P would depend on x, y, and z.
So, when you compute the determinant, you get a formula that involves x, y, and z.
And you'll find that this condition on x, y, z is the equation of a plane.
We're going to see more about that pretty soon.
Now, let me tell you a slightly faster way of doing it.
Actually, it's not much faster, It's pretty much the same calculation, but it's maybe more enlightening.
Let me actually show you a nice color picture that I prepared for this.
One thing that's on this picture that I haven't drawn before is the normal vector to the plane.
Why is that?
Well, let's say that we know how to find a vector that's perpendicular to our plane.
Then, what does it mean for the point, P, to be in the plane?
It means that the direction from P1 to P has to be perpendicular to this vector N. So here's another solution: P is in the plane exactly when the vector P1P is perpendicular to N, where N is some vector that's perpendicular to the plane.
N is called a normal vector.
How do we rephrase this condition?
Well, we've learned how to detect whether two vectors are perpendicular to each other using dot product (that was the first lecture).
These two vectors are perpendicular exactly when their dot product is zero.
So, concretely, if we have a point P1 given to us, and say we have been able to compute the vector N, then when we actually compute what happens, here we will have the coordinates x, y, z, of a point P, and we will get some condition on x, y, z.
That will be the equation of a plane.
Now, why are these things the same?
Well, before I can tell you that, I should tell you how to find a normal vector.
Maybe you are already starting to see what the method should be, because we know how to find a vector perpendicular to two given vectors.
We know two vectors in that plane, for example, P1P2, and P1P3.
Actually, I could have used another permutation of these points, but, let's use this.
So, if I want to find a vector that's perpendicular to both P1P2 and P1P3 at the same time, all I have to do is take their cross product.
So, how do we find a vector that's perpendicular to the plane?
The answer is just the cross product P1P2 cross P1P3.
Say you actually took the points in a different order, and you took P1P3 x P1P2.
You would get, of course, the opposite vector.
That is fine.
Any plane actually has infinitely many normal vectors.
You can just multiply a normal vector by any constant, you will still get a normal vector.
So, that's going to be one of the main uses of dot product.
When we know two vectors in a plane, it lets us find the normal vector to the plane, and that is what we need to find the equation.
Now, why is that the same as our first answer over there?
Well, the condition that we have is that P1P dot N should be 0.
And we said N is actually P1P2 cross P1P3.
So, this is what we want to be zero.
Now, if you remember, a long time ago (that was Friday) we've introduced this thing and called it the triple product.
And what we've seen is that the triple product is the same thing as the determinant.
So, in fact, these two ways of thinking, one saying that the box formed by these three vectors should be flat and have volume zero, and the other one saying that we can find a normal vector and then express the condition that a vector is in the plane if it's perpendicular to the normal vector, are actually giving us the same formula in the end.
OK, any quick questions before we move on?
are those two equal only when P is in the plane, or no matter where it is?
So, these two quantities, P1P dot the cross product, or the determinant of the three vectors, are always equal to each other.
They are always the same.
And now, if a point is not in the plane, then their numerical value will be nonzero.
If P is in the plane, it will be zero.
OK, let's move on and talk a bit about matrices.
Probably some of you have learnt about matrices a little bit in high school, but certainly not all of you.
So let me just introduce you to a little bit about matrices -- just enough for what we will need later on in this class.
If you want to know everything about the secret life of matrices, then you should take 18.06 someday.
OK, what's going to be our motivation for matrices?
Well, in life, a lot of things are related by linear formulas.
And, even if they are not, maybe sometimes you can approximate them by linear formulas.
So, often, we have linear relations between variables -- for example, if we do a change of coordinate systems.
For example, say that we are in space, and we have a point.
Its coordinates might be, let me call them x1, x2, x3 in my initial coordinate system.
But then, maybe I need to actually switch to different coordinates to better solve the problem because they're more adapted to other things that we'll do in the problem.
And so I have other coordinates axes, and in these new coordinates, P will have different coordinates -- let me call them, say, u1, u2, u3.
And then, the relation between the old and the new coordinates is going to be given by linear formulas -- at least if I choose the same origin.
Otherwise, there might be constant terms, which I will not insist on.
Let me just give an example.
For example, maybe, let's say u1 could be 2 x1 3 x2 3 x3.
u2 might be 2 x1 4 x2 5 x3.
u3 might be x1 x2 2 x3.
Do not ask me where these numbers come from.
I just made them up, that's just an example of what might happen.
You can put here your favorite numbers if you want.
Now, in order to express this kind of linear relation, we can use matrices.
A matrix is just a table with numbers in it.
And we can reformulate this in terms of matrix multiplication or matrix product.
So, instead of writing this, I will write that the matrix |2,3, 3; 2,4, 5; 1,1, 2| times the vector ***amp***lt;x1, x2, x3> is equal to ***amp***lt;u1, u2, u3>.
Hopefully you see that there is the same information content on both sides.
I just need to explain to you what this way of multiplying tables of numbers means.
Well, what it means is really that we'll have exactly these same quantities.
Let me just say that more symbolically: so maybe this matrix could be called A, and this we could call X, and this one we could call U.
Then we'll say A times X equals U, which is a lot shorter than that.
Of course, I need to tell you what A, X, and U are in terms of their entries for you to get the formula.
But it's a convenient notation.
So, what does it mean to do a matrix product?
The entries in the matrix product are obtained by taking dot products.
Let's say we are doing the product AX.
We do a dot products between the rows of A and the columns of X.
Here, A is a 3x3 matrix -- that just means there's three rows and three columns.
And X is a column vector, which we can think of as a 3x1 matrix.
It has three rows and only one column.
Now, what can we do?
Well, I said we are going to do a dot product between a row of
2,3, 3, and a column of
x1, x2, x3.
That dot product will be two times x1 plus three times x2 plus three times x3.
OK, it's exactly what we want to set u1 equal to.
Let's do the second one.
I take the second row of
2,4, 5, and I do the dot product with x1, x2, x3.
I will get two times x1 plus four times x2 plus five times x3, which is u2.
And, same thing with the third one: one times x1 plus one times x2 plus two times x3.
So that's matrix multiplication.
Let me restate things more generally.
If I want to find the entries of a product of two matrices, A and B -- I'm saying matrices, but of course they could be vectors.
Vectors are now a special case of matrices, just by taking a matrix of width one.
So, if I have my matrix A, and I have my matrix B, then I will get the product, AB.
Let's say for example -- this works in any size -- let's say that A is a 3x4 matrix.
So, it has three rows, four columns.
And, here, I'm not going to give you all the values because I'm not going to compute everything.
It would take the rest of the lecture.
And let's say that B is maybe size 4x2.
So, it has two columns and four rows.
And, let's say, for example, that we have the second column: 0,3, 0,2.
So, in A times B, the entries should be the dot products between these rows and these columns.
Here, we have two columns.
Here, we have three rows.
So, we should get three times two different possibilities.
And so the answer will have size 3x2.
We cannot compute most of them, because I did not give you numbers, but one of them we can compute.
We can compute the value that goes here, namely, this one in the second column.
So, I select the second column of B, and I take the first row of A, and I find: 1 times 0: 0.
2 times 3: 6, plus 0, plus 8, should make 14.
So, this entry right here is 14.
In fact, let me tell you about another way to set it up so that you can remember more easily what goes where.
One way that you can set it up is you can put A here.
You can put B up here, and then you will get the answer here.
And, if you want to find what goes in a given slot here, then you just look to its left and you look above it, and you do the dot product between these guys.
That's an easy way to remember.
First of all, it tells you what the size of the answer will be.
The size will be what fits nicely in this box: it should have the same width as B and the same height as A.
And second, it tells you which dot product to compute for each position.
You just look at what's to the left, and what's above the given position.
Now, there's a catch.
Can we multiply anything by anything?
Well, no.
I wouldn't ask the question otherwise.
But anyway, to be able to do this dot product, we need to have the same number of entries here and here.
Otherwise, we can't say "take this times that, plus this times that, and so on" if we run out of space on one of them before the other.
So, the condition -- and it's important, so let me write it in red -- is that the width of A must equal the height of B.
(OK, it's a bit cluttered, but hopefully you can still see what I'm writing.)
OK, now we know how to multiply matrices.
So, what does it mean to multiply matrices?
Of course, we've seen in this example that we can use a matrix to tell us how to transform from x's to u's.
And, that's an example of multiplication.
But now, let's see that we have two matrices like that telling us how to transform from something to something else.
What does it mean to multiply them?
I claim that the product AB represents doing first the transformation B, then transformation A.
That's a slightly counterintuitive thing, because we are used to writing things from left to right.
Unfortunately, with matrices, you multiply things from right to left.
If you think about it, say you have two functions, f and g, and you write f(g(x)), it really means you apply first g then f. It works the same way as that.
OK, so why is this?
Well, if I write AB times X where X is some vector that I want to transform, it's the same as A times BX.
This property is called associativity.
And, it's a good property of well-behaved products -- not of cross product, by the way.
Matrix product is associative.
That means we can actually think of a product ABX and multiply them in whichever order we want.
We can start with BX or we can start with AB.
So, now, BX means we apply the transformation B to X.
And then, multiplying by A means we apply the transformation A.
So, we first apply B, then we apply A.
That's the same as applying AB all at once.
Another thing -- a warning: AB and BA are not the same thing at all.
You can probably see that already from this interpretation.
It's not the same thing to convert oranges to bananas and then to carrots, or vice versa.
Actually, even worse: this thing might not even be well defined.
If the width of A equals the height of B, we can do this product.
But it's not clear that the width of B will equal the height of A, which is what we would need for that one.
So, the size condition, to be able to do the product, might not make sense -- maybe one of the products doesn't make sense.
Even if they both make sense, they are usually completely different things.
The next thing I need to tell you about is something called the identity matrix.
The identity matrix is the matrix that does nothing.
What does it mean to do nothing?
I don't mean the matrix is zero.
The matrix zero would take X and would always give you back zero.
That's not a very interesting transformation.
What I mean is the guy that takes X and gives you X again.
It's called I, and it has the property that IX equals X for all X.
So, it's the transformation from something to itself.
It's the obvious transformation -- called the identity transformation.
So, how do we write that as a matrix?
Well, actually there's an identity for each size because, depending on whether X has two entries or ten entries, the matrix I needs to have a different size.
For example, the identity matrix of size 3x3 has entries one, one, one on the diagonal, and zero everywhere else.
OK, let's check.
If we multiply this with a vector -- start thinking about it.
What happens when multiply this with the vector X?
OK, so let's say I multiply the matrix I with a vector x1, x2, x3.
What will the first entry be?
It will be the dot product between ***amp***lt;1,0,0> and ***amp***lt;x1 x2 x3>.
This vector is i hat.
If you do the dot product with i hat, you will get the first component -- that will be x1.
One times x1 plus zero, zero.
Similarly here, if I do the dot product, I get zero plus x2 plus zero.
I get x2, and here I get x3.
OK, it works.
Same thing if I put here a matrix: I will get back the same matrix.
In general, the identity matrix in size n x n is an n x n matrix with ones on the diagonal, and zeroes everywhere else.
You just put 1 at every diagonal position and 0 elsewhere.
And then, you can see that if you multiply that by a vector, you'll get the same vector back.
OK, let me give you another example of a matrix.
Let's say that in the plane we look at the transformation that does rotation by 90°, let's say, counterclockwise.
I claim that this is given by the matrix: |0,1; - 1,0|.
Let's try to see why that is the case.
Well, if I do R times i hat -- if I apply that to the first vector, i hat: i hat will be ***amp***lt;1,0> so in this product, first you will get 0, and then you will get 1.
You get j hat.
OK, so this thing sends i hat to j hat.
What about j hat?
Well, you get negative one.
And then you get 0.
So, that's minus i hat.
So, j is sent towards here.
And, in general, if you apply it to a vector with components x,y, then you will get back -y,x, which is the formula we've seen for rotating a vector by 90°.
So, it seems to do what we want.
By the way, the columns in this matrix represent what happens to each basis vector, to the vectors i and j.
This guy here is exactly what we get when we multiply R by i.
And, when we multiply R by j, we get this guy here.
So, what's interesting about this matrix?
Well, we can do computations with matrices in ways that are easier than writing coordinate change formulas.
For example, if you compute R squared, so if you multiply R with itself: I'll let you do it as an exercise, but you will find that you get |-1,0;0,-1|.
So, that's minus the identity matrix.
Why is that?
Well, if I rotate something by 90° and then I rotate by 90° again, then I will rotate by 180�.
That means I will actually just go to the opposite point around the origin.
So, I will take (x,y) to (-x,-y).
And if I applied R four times, R^4 would be identity.
OK, questions?
when you said R equals that matrix, is that the definition of R?
How did I come up with this R?
Well, secretly, I worked pretty hard to find the entries that would tell me how to rotate something by 90° counterclockwise.
So, remember: what we saw last time or in the first lecture is that, to rotate a vector by 90°, we should change (x, y) to (-y, x).
And now I'm trying to express this transformation as a matrix.
So, maybe you can call these guys u and v, and then you write that u equals 0x-1y, and that v equals 1x 0y.
So that's how I would find it.
Here, I just gave it to you already made, so you didn't really see how I found it.
You will see more about rotations on the problem set.
OK, next I need to tell you how to invert matrices.
So, what's the point of matrices?
It's that it gives us a nice way to think about changes of variables.
And, in particular, if we know how to express U in terms of X, maybe we'd like to know how to express X in terms of U.
Well, we can do that, because we've learned how to solve linear systems like this.
So in principle, we could start working, substituting and so on, to find formulas for x1, x2, x3 as functions of u1, u2, u3.
And the relation will be, again, a linear relation.
It will, again, be given by a matrix.
Well, what's that matrix?
It's the inverse transformation.
It's the inverse of the matrix A.
So, we need to learn how to find the inverse of a matrix directly.
The inverse of A, by definition, is a matrix M, with the property that if I multiply A by M, then I get identity.
And, if I multiply M by A, I also get identity.
The two properties are equivalent.
That means, if I apply first the transformation A, then the transformation M, actually I undo the transformation A, and vice versa.
These two transformations are the opposite of each other, or I should say the inverse of each other.
For this to make sense, we need A to be a square matrix.
It must have size n by n. It can be any size, but it must have the same number of rows as columns.
It's a general fact that you will see more in detail in linear algebra if you take it.
Let's just admit it.
The matrix M will be denoted by A inverse.
Then, what is it good for?
Well, for example, finding the solution to a linear system.
What's a linear system in our new language?
It's: a matrix times some unknown vector, X, equals some known vector, B.
How do we solve that?
We just compute: X equals A inverse B.
Why does that work?
How do I get from here to here?
Let's be careful.
(I'm going to reuse this matrix, but I'm going to erase it nonetheless and I'll just rewrite it).
If AX=B, then let's multiply both sides by A inverse.
A inverse times AX is A inverse B.
And then, A inverse times A is identity, so I get: X equals A inverse B.
That's how I solved my system of equations.
So, if you have a calculator that can invert matrices, then you can solve linear systems very quickly.
Now, we should still learn how to compute these things.
Yes?
[Student Questions:]"How do you know that A inverse will be on the left of B and not after it " Well, it's exactly this derivation.
So, if you are not sure, then just reproduce this calculation.
To get from here to here, what I did is I multiplied things on the left by A inverse, and then this guy simplify.
If I had put A inverse on the right, I would have AX A inverse, which might not make sense, and even if it makes sense, it doesn't simplify.
So, the basic rule is that you have to multiply by A inverse on the left so that it cancels with this A that's on the left
"And if you put it on the left on this side then it will be on the left with B as well?"
That's correct, in our usual way of dealing with matrices, where the vectors are column vectors.
It's just something to remember: if you have a square matrix times a column vector, the product that makes sense is with the matrix on the left, and the vector on the right.
The other one just doesn't work.
You cannot take X times A if A is a square matrix and X is a column vector.
This product AX makes sense.
The other one XA doesn't make sense.
It's not the right size.
OK. What we need to do is to learn how to invert a matrix.
It's a useful thing to know, first for your general knowledge, and second because it's actually useful for things we'll see later in this class.
In particular, on the exam, you will need to know how to invert a matrix by hand.
This formula is actually good for small matrices, 3x3,4x4.
It's not good at all if you have a matrix of size 1,000x1,000.
So, in computer software, actually for small matrices they do this, but for larger matrices, they use other algorithms.
Let's just see how we do it.
First of all I will give you the final answer.
And of course I will need to explain what the answer means.
We will have to compute something called the adjoint matrix.
I will tell you how to do that.
And then, we will divide by the determinant of A.
How do we get to the adjoint matrix?
Let's go through the steps on a 3x3 example -- the steps are the same no matter what the size is, but let's do 3x3.
So, let's say that I'm giving you the matrix A -- let's say it's the same as the one that I erased earlier.
That was the one relating our X's and our U's.
The first thing I want to do is find something called the minors.
What's a minor?
It's a slightly smaller determinant.
We've already seen them without calling them that way.
The matrix of minors will have again the same size.
Let's say we want this entry.
Then, we just delete this row and this column, and we are left with a 2x2 determinant.
So, here, we'll put the determinant 4,5, 1,2, which is 4 times 2: 8 -- minus 5: 3.
Let's do the next one.
So, for this entry, I'll delete this row and this column.
I'm left with 2,5, 1,2.
The determinant will be 2 times 2 minus 5, which is negative 1.
Then minus 2, then I get to the second row, so I get to this entry.
To find the minor here, I will delete this row and this column.
And I'm left with 3,3, 1,2.
3 times 2 minus 3 is 3.
Let me just cheat and give you the others -- I think I've shown you that I can do them.
Let's just explain the last one again.
The last one is 2.
To find the minor here, I delete this column and this row, and I take this determinant: 2 times 4 minus 2 times 3.
So it's the same kind of manipulation that we've seen when we've taken determinants and cross products.
Step two: we go to another matrix that's called cofactors.
So, the cofactors are pretty much the same thing as the minors except the signs are slightly different.
What we do is that we flip signs according to a checkerboard diagram.
You start with a plus in the upper left corner, and you alternate pluses and minuses.
The rule is: if there is a plus somewhere, then there's a minus next to it and below it.
And then, below a minus or to the right of a minus, there's a plus.
So that's how it looks in size 3x3.
What do I mean by that?
I don't mean, make this positive, make this negative, and so on.
That's not what I mean.
What I mean is: if there's a plus, that means leave it alone -- we don't do anything to it.
If there's a minus, that means we flip the sign.
So, here, we'd get: 3, then 1, -2, -3,1, 1... 3,-4, and 2.
OK, that step is pretty easy.
The only hard step in terms of calculations is the first one because you have to compute all of these 2x2 determinants.
By the way, this minus sign here is actually related to the way in which, when we do a cross product, we have a minus sign for the second entry.
OK, we're almost done.
The third step is to transpose.
What does it mean to transpose?
It means: you read the rows of your matrix and write them as columns, or vice versa.
So we switch rows and columns.
What do we get?
Well, let's just read the matrix horizontally and write it vertically.
We read 3,1, - 2: 3,1, - 2.
Then we read -3 3,1, 1: - 3,1, 1.
Then, 3, - 4,2: 3, - 4,2.
That's pretty easy.
We're almost done.
What we get here is this is the adjoint matrix.
So, the fourth and last step is to divide by the determinant of A.
We have to compute the determinant -- the determinant of A, not the determinant of this guy.
So: 2,3, 3,2, 4,5, 1,1, 2.
I'll let you check my computation.
I found that it's equal to 3.
So the final answer is that A inverse is one third of the matrix we got there: |3, - 3,3, 1,1, - 4, - 2,1, 2|.
Now, remember, A told us how to find the u's in terms of the x's.
This tells us how to find x-s in terms of u-s: if you multiply x1,x2,x3 by this you get u1,u2,u3.
It also tells you how to solve a linear system: A times X equals something.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
The topic for today is going to be equations of planes, and how they relate to linear systems and matrices as we have seen during Tuesday's lecture.
So, let's start again with equations of planes.
Remember, we've seen briefly that an equation for a plane is of the form ax by cz = d, where a, b, c, and d are just numbers.
This expresses the condition for a point at coordinates x, y, z, to be in the plane.
An equation of this form defines a plane.
Let's see how that works, again.
Let's start with an example.
Let's say that we want to find the equation of a plane through the origin with normal vector -- -- let's say vector N equals the vector <1,5, 10>.
How do we find an equation of this plane?
Remember that we can get an equation by thinking geometrically.
So, what's our thinking going to be?
Well, we have the x, y, z axes.
And, we have this vector
.
It's supposed to be perpendicular to our plane.
And, our plane passes through the origin here.
So, we want to think of the plane that's perpendicular to this vector.
Well, when is a point in that plane?
Let's say we have a point, P -- -- at coordinates x, y, z.
Well, the condition for P to be in the plane should be that we have a right angle here.
OK, so P is in the plane whenever OP dot N is 0.
And, if we write that explicitly, the vector OP has components x, y, z; N has components 1,5, 10.
So that will give us x 5y 10z = 0.
That's the equation of our plane.
Now, let's think about a slightly different problem.
So, let's do another problem.
Let's try to find the equation of the plane through the point P0 with coordinates, say, (2,1,-1), with normal vector, again, the same N = <1,5, 10>.
How do we find an equation of this thing?
Well, we're going to use the same method.
In fact, let's think for a second.
I said we have our normal vector, N, and it's going to be perpendicular to both planes at the same time.
So, in fact, our two planes will be parallel to each other.
The difference is, well, before, we had a plane that was perpendicular to N, and passing through the origin.
And now, we have a new plane that's going to pass not through the origin but through this point, P0.
I don't really know where it is, but let's say, for example, that P0 is here.
Then, I will just have to shift my plane so that, instead of passing through the origin, it passes through this new point.
How am I going to do that?
Well, now, for a point P to be in our new plane, we need the vector no longer OP but P0P to be perpendicular to N. So P is in this new plane if the vector P0P is perpendicular to N. And now, let's think, what's the vector P0P?
Well, we take the coordinates of P, and we subtract those of P0.
So, that should be x-2, y-1, and z 1, dot product with <1,5, 10> equals 0.
Let's expand this.
We get (x-2) 5(y-1) 10(z 1) = 0.
Let's put the constants on the other side.
We get: x 5y 10z equals -- here minus two becomes two, minus five becomes five, ten becomes minus ten.
I think we end up with negative three.
So, the only thing that changes between these two equations is the constant term on the right-hand side, the thing that I called d. The other common feature is that the coefficients of x, y, and z: one, five, and ten, correspond exactly to the normal vector.
That's something you should remember about planes.
These coefficients here correspond exactly to a normal vector and, well, this constant term here roughly measures how far you move from...
I f you have a plane through the origin, the right-hand side will be zero.
And, if you move to a parallel plane, then this number will become something else.
Actually, how could we have found that -3 more quickly?
Well, we know that the first part of the equation is like this.
And we know something else.
We know that the point P0 is in the plane.
So, if we plug the coordinates of P0 into this, well, x is 2 5 times 1 10 times -1.
We get -3.
So, in fact, the number we should have here should be minus three so that P0 is a solution.
Let me point out -- (I'll put a 1 here again) -- these three numbers: 1,5, 10, are exactly the normal vector.
And one way that we can get this number here is by computing the value of the left-hand side at the point P0.
We plug in the point P0 into the left hand side.
OK, any questions about that?
By the way, of course, a plane doesn't have just one equation.
It has infinitely many equations because if instead, say, I multiply everything by two, 2x 10y 20z = -6 is also an equation for this plane.
That's because we have normal vectors of all sizes -- we can choose how big we make it.
Again, the single most important thing here: in the equation ax by cz = d, the coefficients, a, b, c, give us a normal vector to the plane.
So, that's why, in fact, what matters to us the most is finding the normal vector.
In particular, if you remember, last time I explained something about how we can find a normal vector to a plane if we know points in the plane.
Namely, we can take the cross product of two vectors contained in the plane.
Let's just do an example to see if we completely understand what's going on.
Let's say that I give you the vector with components , and I give you the plane x y 3z = 5.
So, do you think that this vector is parallel to the plane, perpendicular to it, neither?
I'm starting to see a few votes.
OK, I see that most of you are answering number two: this vector is perpendicular to the plane.
There are some other answers too.
Well, let's try to figure it out.
Let's do the example.
Say v is <1,2, -1> and the plane is x y 3z = 5.
Let's just draw that plane anywhere -- it doesn't really matter.
Let's first get a normal vector out of it.
Well, to get a normal vector to the plane, what I will do is take the coefficients of x, y, and z.
So, that's .
So is perpendicular to the plane.
How do we get all the other vectors that are perpendicular to the plane?
Well, all the perpendicular vectors are parallel to each other.
That means that they are just obtained by multiplying this guy by some number.
for example, would still be perpendicular to the plane.
is also perpendicular to the plane.
But now, see, these guys are not proportional to each other.
So, V is not perpendicular to the plane.
So it's not perpendicular to the plane.
Being perpendicular to the plane is the same as being parallel to its normal vector.
Now, what about testing if v is, instead, parallel to the plane?
Well, it's parallel to the plane if it's perpendicular to N. Let's check.
So, let's try to see if v is perpendicular to N. Well, let's do v.N.
That's <1,2, - 1> dot <1,1, 3>.
You get 1 2 - 3=0.
So, yes.
If it's perpendicular to N, it means -- It's actually going to be parallel to the plane.
OK, any questions?
Yes?
[
] When you plug the vector into the plane equation, you get zero.
What does that mean?
Let's see.
If I plug the vector into the plane equation: 1 2-3, well, the left hand side becomes zero.
So, it's not a solution of the plane equation.
There's two different things here.
One is that the point with coordinates (1,2,- 1) is not in the plane.
What that tells us is that, if I put my vector V at the origin, then its head is not going to be in the plane.
On the other hand, you're right, the left hand side evaluates to zero.
What that means is that, if instead I had taken the plane x y 3z = 0, then it would be inside.
The plane is x y 3z = 5, so x y 3z = 0 would be a plane parallel to it, but through the origin.
So, that would be another way to see that the vector is parallel to the plane.
If we move the plane to a parallel plane through the origin, then the endpoint of the vector is in the plane.
OK, that's another way to convince ourselves.
Any other questions?
OK, let's move on.
So, last time we learned about matrices and linear systems.
So, let's try to think, now, about linear systems in terms of equations of planes and intersections of planes.
Remember that a linear system is a bunch of equations -- say, a 3x3 linear system is three different equations.
Each of them is the equation of a plane.
So, in fact, if we try to solve a system of equations, that means actually we are trying to find a point that is on several planes at the same time.
So... Let's say that we have a 3x3 linear system.
Just to take an example -- it doesn't really matter what I give you, but let's say I give you x z = 1, x y = 2, x 2y 3z = 3.
What does it mean to solve this?
It means we want to find x, y, z which satisfy all of these conditions.
Let's just look at the first equation, first.
Well, the first equation says our point should be on the plane which has this equation.
Then, the second equation says that our point should also be on that plane.
So, if you just look at the first two equations, you have two planes.
And the solutions -- these two equations determine for you two planes, and two planes intersect in a line.
Now, what happens with the third equation?
That's actually going to be a third plane.
So, if we want to solve the first two equations, we have to be on this line.
And if we want to solve the third one, we also need to be on another plane.
And, in general, the three planes intersect in a point because this line of intersection... Three planes intersect in a point, and one way to think about it is that the line where the first two planes intersect meets the third plane in a point.
And, that point is the solution to the linear system.
The line -- this is mathematical notation for the intersection between the first two planes -- intersects the third plane in a point, which is going to be the solution.
So, how do we find the solution?
One way is to draw pictures and try to figure out where the solution is, but that's not how we do it in practice if we are given the equations.
Let me use matrix notation.
Remember, we saw on Tuesday that the solution to AX = B is given by X = A inverse B.
We got from here to there by multiplying on the left by A inverse.
A inverse AX simplifies to X equals A inverse B.
And, once again, it's A inverse B and not BA inverse.
If you try to set up the multiplication, BA inverse doesn't work.
The sizes are not compatible, you can't multiply the other way around.
OK, that's pretty good -- unless it doesn't work that way.
What could go wrong?
Well, let's say that our first two planes do intersect nicely in a line, but let's think about the third plane.
Maybe the third plane does not intersect that line nicely in a point.
Maybe it's actually parallel to that line.
Let's try to think about this question for a second.
Let's say that the set of solutions to a 3x3 linear system is not just one point.
So, we don't have a unique solution that we can get this way.
What do you think could happen?
OK, I see that answers number three and five seem to be dominating.
There's also a bit of answer number one.
In fact, these are pretty good answers.
I see that some of you figured out that you can answer one and three at the same time, or three and five at the same time.
I yet have to see somebody with three hands answer all three numbers at the same time.
OK.
Indeed, we'll see very soon that we could have either no solution, a line, or a plane.
The other answers: "two points" (two solutions), we will see, is actually not a possibility because if you have two different solutions, then the entire line through these two points is also going to be made of solutions.
"A tetrahedron" is just there to amuse you, it's actually not a good answer to the question.
It's not very likely that you will get a tetrahedron out of intersecting planes.
"A plane" is indeed possible, and "I don't know" is still OK for a few more minutes, but we're going to get to the bottom of this, and then we will know.
OK, let's try to figure out what can happen.
Let me go back to my picture.
I had my first two planes; they determine a line.
And now I have my third plane.
Maybe my third plane is actually parallel to the line but doesn't pass through it.
Well, then, there's no solutions because, to solve the system of equations, I need to be in the first two planes.
So, that means I need to be in that vertical line.
(That line was supposed to be red, but I guess it doesn't really show up as red).
And it also needs to be in the third plane.
But the line and the plane are parallel to each other.
There's just no place where they intersect.
So there's no way to solve all the equations.
On the other hand, the other thing that could happen is that actually the line is contained in the plane.
And then, any point on that line will automatically solve the third equation.
So if you try solving a system that looks like this by hand, if you do substitutions, eliminations, and so on, what you will notice is that, after you have dealt with two of the equations, the third one would actually turn out to be the same as what you got out of the first two.
It doesn't give you any additional information.
It's as if you had only two equations.
The previous case would be when actually the third equation contradicts something that you can get out of the first two.
For example, maybe out of the first two, you got that x plus z equals one, and the third equation is x plus z equals two.
Well, it can't be one and two at the same time.
Another way to say it is that this picture is one where you can get out of the equations that a number equals a different number.
That's impossible.
And, that picture is one where out of the equations you get zero equals zero, which is certainly true, but isn't a very useful equation.
So, you can't actually finish solving.
OK, let me write that down.
unless the third plane is parallel to the line where P1 and P2 intersect.
Then there's two subcases.
If the line of intersections of P1 and P2 is actually contained in P3 (the third plane), then we have infinitely many solutions.
Namely, any point on the line will automatically solve the third equation.
The other subcase is if the line of the intersection of P1 and P2 is parallel to P3 and not contained in it.
Then we get no solutions.
Just to show you the pictures once again: when we have the first two planes, they give us a line.
And now, depending on what happens to that line in relation to the third plane, various situations can happen.
If the line hits the third plane in a point, then that's going to be our solution.
If that line, instead, is parallel to the third plane, well, if it's parallel and outside of it, then we have no solution.
If it's parallel and contained in it, then we have infinitely many solutions.
So, going back to our list of possibilities, let's see what can happen.
No solution: we've seen that it happens when the line where the first two planes intersect is parallel to the third one.
Two points: well, that didn't come up.
As I said, the problem is that, if the line of intersections of the first two planes has two points that are in the third plane, then that means the entire line must actually be in the third plane.
So, if you have two solutions, then you have more than two.
In fact, you have infinitely many, and we've seen that can happen.
A tetrahedron: still doesn't look very promising.
What about a plane?
Well, that's a case that I didn't explain because I've been assuming that P1 and P2 are different planes and they intersect in a line.
But, in fact, they could be parallel, in which case we already have no solution to the first two equations; or they could be the same plane.
And now, if the third plane is also the same plane -- if all three planes are the same plane, then you have a plane of solutions.
If I give you three times the same equation, that is a linear system.
It's not a very interesting one, but it's a linear system.
And "I don't know" is no longer a solution either.
OK, any questions?
[
] What's the geometric significance of the plane x y z equals 1, as opposed to 2, or 3?
That's a very good question.
The question is, what is the geometric significance of an equation like x y z equals to 1,2, 3, or something else?
Well, if the equation is x y z equals zero, it means that our plane is passing through the origin.
And then, if we change the constant, it means we move to a parallel plane.
So, the first guess that you might have is that this number on the right-hand side is the distance between the origin and the plane.
It tells us how far from the origin we are.
That is not quite true.
In fact, that would be true if the coefficients here formed a unit vector.
Then this would just be the distance to the origin.
Otherwise, you have to actually scale by the length of this normal vector.
And, I think there's a problem in the Notes that will show you exactly how this works.
You should think of it roughly as how much we have moved the plane away from the origin.
That's the meaning of the last term, D, in the right-hand side of the equation.
So, let's try to think about what exactly these cases are -- how do we detect in which situation we are?
It's all very nice in the picture, but it's difficult to draw planes.
In fact, when I draw these pictures, I'm always very careful not to actually pretend to draw an actual plane given by an equation.
When I do, then it's blatantly false -- it's difficult to draw a plane correctly.
So, instead, let's try to think about it in terms of matrices.
In particular, what's wrong with this?
Why can't we always say the solution is X = A inverse B?
Well, the point is that, actually, you cannot always invert a matrix.
Recall we've seen this formula: A inverse is one over determinant of A times the adjoint matrix.
And we've learned how to compute this thing: remember, we had to take minors, then flip some signs, and then transpose.
That step we can always do.
We can always do these calculations.
But then, at the end, we have to divide by the determinant.
That's fine if the determinant is not zero.
But, if the determinant is zero, then certainly we cannot do that.
What I didn't mention last time is that the matrix is invertible -- that means it has an inverse -- exactly when its determinant is not zero.
That's something we should remember.
So, if the determinant is not zero, then we can use our method to find the inverse.
And then we can solve using this method.
If not, then not.
Yes?
[
] Sorry, can you reexplain that?
You can invert A if the determinant of A is not equal to zero?
That's correct.
We can invert the matrix A if the determinant is not zero.
If you look again at the method that we saw last time: first we had to compute the adjoint matrix.
And, these are operations we can always do.
If we are given a 3x3 matrix, we can always compute the adjoint.
And then, the last step to find the inverse was to divide by the determinant.
And that we can only do if the determinant is not zero.
So, if we have a matrix whose determinant is not zero, then we know how to find the inverse.
If the determinant is zero, then of course this method doesn't work.
I'm actually saying even more: there isn't an inverse at all.
It's not just that our method fails.
I cannot take the inverse of a matrix with determinant zero.
Geometrically, the situation where the determinant is not zero is exactly this nice usual situation where the three planes intersect in a point, while the situation where the determinant is zero is this situation here where the line determined by the first two planes is parallel to the third plane.
Let me emphasize this again, and let's see again what happens.
Let's start with an easier case.
It's called the case of a homogeneous system.
It's called homogeneous because it's the situation where the equations are invariant under scaling.
So, a homogeneous system is one where the right hand side is zero -- there's no B.
If you want, the constant terms here are all zero: 0,0, 0.
OK, so this one is not homogenous.
So, let's see what happens there.
Let's take an example.
Instead of this system, we could take x z = 0, x y = 0, and x 2y 3z also equals zero.
Can we solve these equations?
I think actually you already know a very simple solution to these equations.
Yeah, you can just take x, y, and z all to be zero.
So, there's always an obvious solution -- -- namely, (0,0, 0).
And, in mathematical jargon, this is called the trivial solution.
There's always this trivial solution.
What's the geometric interpretation?
Well, having zeros here means that all three planes pass through the origin.
So, certainly the origin is always a solution.
The origin is always a solution because the three planes -- -- pass through the origin.
Now there's two subcases.
One case is if the determinant of the matrix A is nonzero.
That means that we can invert A.
So, if we can invert A, then we can solve the system by multiplying by A inverse.
If we multiply by A inverse, we'll get X equals A inverse times zero, which is zero.
That's the only solution because, if AX is zero, then let's multiply by A inverse: we get that A inverse AX, which is X, equals A inverse zero, which is zero.
We get that X equals zero.
We've solved it, there's no other solution.
To go back to these pictures that we all enjoy, it's this case.
Now the other case, if the determinant of A equals zero, then this method doesn't quite work.
What does it mean that the determinant of A is zero?
Remember, the entries in A are the coefficients in the equations.
But now, the coefficients in the equations are exactly the normal vectors to the planes.
So, that's the same thing as saying that the determinant of the three normal vectors to our three planes is 0.
That means that N1, N2, and N3 are actually in a same plane -- they're coplanar.
These three vectors are coplanar.
So, let's see what happens.
I claim it will correspond to this situation here.
Let's draw the normal vectors to these three planes.
(Well, it's not very easy to see, but I've tried to draw the normal vectors to my planes.)
They are all in the direction that's perpendicular to the line of intersection.
They are all in the same plane.
So, if I try to form a parallelepiped with these three normal vectors, well, I will get something that's completely flat, and has no volume, has volume zero.
So the parallelepiped -- -- has volume 0.
And the fact that the normal vectors are coplanar tells us that, in fact -- (well, let me start a new blackboard).
Let's say that our normal vectors, N1, N2, N3, are all in the same plane.
And let's think about the direction that's perpendicular to N1, N2, and N3 at the same time.
I claim that it will be the line of intersection.
So, let me try to draw that picture again.
We have three planes -- (now you see why I prepared a picture in advance.
It's easier to draw it beforehand).
And I said their normal vectors are all in the same plane.
What else do I know?
I know that all these planes pass through the origin.
So the origin is somewhere in the intersection of the three planes.
Now, I said that the normal vectors to my three planes are all actually coplanar.
So N1, N2, N3 determine a plane.
Now, if I look at the line through the origin that's perpendicular to N1, N2, and N3, so, perpendicular to this red plane here, it's supposed to be in all the planes.
(You can see that better on the side screens).
And why is that?
Well, that's because my line is perpendicular to the normal vectors, so it's parallel to the planes.
It's parallel to all the planes.
Now, why is it in the planes instead of parallel to them?
Well, that's because my line goes through the origin, and the origin is on the planes.
So, certainly my line has to be contained in the planes, not parallel to them.
So the line through the origin and perpendicular to the plane of N1, N2, N3 -- -- is parallel to all three planes.
And, because the planes go through the origin, it's contained in them.
So what happens here is I have, in fact, infinitely many solutions.
How do I find these solutions?
Well, if I want to find something that's perpendicular to N1, N2, and N3 -- if I just want to be perpendicular to N1 and N2, I can take their cross product.
So, for example, N1 cross N2 is perpendicular to N1 and to N2, and also to N3, because N3 is in the same plane as N1 and N2, so, if you're perpendicular to N1 and N2, you are also perpendicular to N3.
It's automatic.
So, it's a nontrivial solution.
This vector goes along the line of intersections.
OK, that's the case of homogeneous systems.
And then, let's finish with the other case, the general case.
If we look at a system, AX = B, with B now anything, there's two cases.
If the determinant of A is not zero, then there is a unique solution -- -- namely, X equals A inverse B.
If the determinant of A is zero, then it means we have the situation with planes that are all parallel to a same line, and then we have either no solution or infinitely many solutions.
It cannot be a single solution.
Now, whether you have no solutions or infinitely many solutions, we haven't actually developed the tools to answer that.
But, if you try solving the system by hand, by elimination, you will see that you end up maybe with something that says zero equals zero, and you have infinitely many solutions.
Actually, if you can find one solution, then you know that there's infinitely many.
On the other hand, if you end up with something that's a contradiction, like one equals two, then you know there's no solutions.
That's the end for today.
Tomorrow, we will learn about parametric equations for lines and curves.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So -- So, yesterday we learned about the questions of planes and how to think of 3x3 linear systems in terms of intersections of planes and how to think about them geometrically.
And, that in particular led us to see which cases actually we don't have a unique solution to the system, but maybe we have no solutions or infinitely many solutions because maybe the line at intersection of two of the planes happens to be parallel to the other plane.
So, today, we'll start by looking at the equations of lines.
And, so in a way it seems like something which we've already seen last time because we have seen that we can think of a line as the intersection of two planes.
And, we know what equations of planes look like.
So, we could describe a line by two equations telling us about the two planes that intersect on the line.
But that's not the most convenient way to think about the line usually, though, because when you have these two questions, have you solve them?
Well, OK, you can, but it takes a bit of effort.
So, instead, there is another representation of a line.
So, if you have a line in space, well, you can imagine may be that you have a point on it.
And, that point is moving in time.
And, the line is the trajectory of a point as time varies.
So, think of a line as the trajectory of a moving point.
And, so when we think of the trajectory of the moving point, that's called a parametric equation.
OK, so we are going to learn about parametric equations of lines.
So, let's say for example that we are looking at the line.
So, to specify a line in space, I can do that by giving you two points on the line or by giving you a point and a vector parallel to the line.
For example, so let's say I give you two points on the line: (-1,2,2), and the other point will be (1,3,-1).
So, OK, it's pretty good because we have two points in that line.
Now, how do we find all the other points?
Well, the other points in between these guys and also on either side.
Let's imagine that we have a point that's moving on the line, and at time zero, it's here at Q0.
And, in a unit time, I'm not telling you what the unit is.
It could be a second.
It could be an hour.
It could be a year.
At t=1, it's going to be at Q1.
And, it moves at a constant speed.
So, maybe at time one half, it's going to be here.
Times two, it would be over there.
And, in fact, that point didn't start here.
Maybe it's always been moving on that line.
At time minus two, it was down there.
So, let's say Q(t) is a moving point, and at t=0 it's at Q0.
And, let's say that it moves.
Well, we couldn't make it move in any way we want.
But, probably the easiest to find, so our role is going to find formulas for a position of this moving point in terms of t. And, we'll use that to say, well, any point on the line is of this form where you have to plug in the current value of t depending on when it's hit by the moving point.
So, perhaps it's easiest to do it if we make it move at a constant speed on the line, and that speed is chosen so that at time one, it's at Q1.
So, the question we want to answer is, what is the position at time t, so, the point Q(t)?
Well, to answer that we have an easy observation, which is that the vector from Q0 to Q of t is proportional to the vector from Q0 to Q1.
And, what's the proportionality factor here?
Yeah, it's exactly t. At time one, Q0 Q is exactly the same.
Maybe I should draw another picture again.
I have Q0.
I have Q1, and after time t, I'm here at Q of t where this vector from Q0 Q(t) is actually going to be t times the vector Q0 Q1.
So, when t increases, it gets longer and longer.
So, does everybody see this now?
Is that OK?
Any questions about that?
Yes?
OK, so I will try to avoid using blue.
Thanks for, that's fine.
So, OK, I will not use blue anymore.
OK, well, first let me just make everything white just for now.
This is the vector from Q0 to Q(t).
This is the point Q(t).
OK, is it kind of visible now?
OK, thanks for pointing it out.
I will switch to brighter colors.
So, OK, so apart from that, I claim now we can find the position of its moving point because, well, this vector, Q0Q1 we can find from the coordinates of Q0 and Q1.
So, we just subtract the coordinates of Q0 from those of Q1 will get that vector Q0 Q1 is OK, so, if I look at it, well, so let's call x(t), y(t), and z(t) the coordinates of the point that's moving on the line.
Then we get x of t minus, well, actually plus one equals t times two.
I'm writing the components of Q0Q(t).
And here, I'm writing t times Q0Q1.
y(t) minus two equals t, and z(t) minus two equals -3t.
So, in other terms, the more familiar way that we used to write these equations, let me do it that way instead, minus one plus 2t, y(t) = 2 t, z(t) = 2 - 3t.
And, if you prefer, I can just say Q(t) is Q0 plus t times vector Q0Q1.
OK, so that's our first parametric equation of a line in this class.
And, I hope you see it's not extremely hard.
In fact, parametric equations of lines always look like that.
x, y, and z are functions of t but are of the form a constant plus a constant times t. The coefficients of t tell us about a vector along the line.
Here, we have a vector, Q0Q1, which is .
And, the constant terms tell us about where we are at t=0.
If I plug t=0 these guys go away, I get minus 1,2, 2.
That's my starting point.
OK, so, any questions about that?
No?
OK, so let's see, now, what we can do with these parametric equations.
So, one application is to think about the relative position of a line and a plane with respect to each other.
So, let's say that we take still the same line up there, and let's consider the plane with the equation x 2y 4z = 7.
OK, so I'm giving you this plane.
And, the questions that we are going to ask ourselves are, well, does the line intersect the plane?
And, where does it intersect the plane?
So, let's start with the first primary question that maybe we should try to understand.
We have these points.
We have this plane, and we have these points, Q0 and Q1.
I'm going to draw them in completely random places.
Well, are Q0 and Q1 on the same side of a plane or on different sides, on opposite sides of the planes?
Could it be that maybe one of the points is in the plane?
So, I think I'm going to let you vote on that.
So, is that readable?
Is it too small?
OK, so anyway, the question says, relative to the plane, x 2y 4z = 7.
This point, Q0 and Q1, are they on the same side, on opposite sides, is one of them on the plane, or we can't decide?
OK, that should be better.
So, I see relatively few answers.
OK, it looks like also a lot of you have forgotten the cards and, so I see people raising two fingers, I see people raising three fingers.
And, I see people raising four fingers.
I don't see anyone answering number one.
So, the general idea seems to be that either they are on opposite sides.
Maybe one of them is on the plane.
Well, let's try to see.
Is one of them on the plane?
Well, let's check.
OK, so let's look at the point, sorry.
I have one blackboard to use here.
So, I take the point Q0, which is at (-1,2,2).
Well, if I plug that into the plane equation, so, x 2y 4z will equal minus one plus two times two plus four times two.
That's, well, four plus eight, 12 minus one, 11.
That, I think, is bigger than seven.
OK, so Q0 is not in the plane.
Let's try again with Q1.
(1,3, - 1) well, if we plug that into x 2y 4z, we'll have one plus two times three makes seven.
But, we add four times negative one.
We add up with three less than seven.
Well, that one is not in the plane, either.
So, I don't think, actually, that the answer should be number three.
So, let's get rid of answer number three.
OK, let's see, in light of this, are you willing to reconsider your answer?
OK, so I think now everyone seems to be interested in answering number two.
And, I would agree with that answer.
So, let's think about it.
These points are not in the plane, but they are not in the plane in different ways.
One of them somehow overshoots; we get 11.
The other one we only get 3.
That's less than seven.
If you think about how a plan splits space into two half spaces on either side, well, one of them is going to be the point where x 2y 4z is less than seven.
And, the other one will be, so, that's somehow this side.
And, that's where Q1 is.
And, the other side is where x 2y 4z is actually bigger than seven.
And, to go from one to the other, well, x 2y 4z needs to go through the value seven.
If you're moving along any path from Q0 to Q1, this thing will change continuously from 11 to 3.
At some time, it has to go through 7.
Does that make sense?
So, to go from Q0 to Q1 we need to cross P at some place.
So, they're on opposite sides.
OK, now that doesn't quite finish answering the question that we had, which was, where does the line intersect the plane?
But, why can't we do the same thing?
Now, we know not only the points Q0 and Q1, we know actually any point on the line because we have a parametric equation up there telling us, where is the point that's moving on the line at time t?
So, what about the moving point, Q(t)?
Well, let's plug its coordinates into the plane equation.
So, we'll take x(t) 2y(t) 4z(t).
OK, that's equal to, well, (-1 2t) 2( 2 t) 4( 2 - 3t).
So, if you simplify this a bit, you get 2t 2t -12t.
That should be -8t.
And, the constant term is minus one plus four plus eight is 11.
OK, and we have to compare that with seven.
OK, the question is, is this ever equal to seven?
Well, so, Q(t) is in the plane exactly when -8t 11 equals seven.
And, that' the same.
If you manipulate this, you will get t equals one half.
In fact, that's not very surprising.
If you look at these values, 11 and three, you see that seven is actually right in between.
It's the average of these two numbers.
So, it would make sense that it's halfway in between Q0 and Q1, but we will get seven.
OK, and that at that time, Q at time one half, well, let's plug the values.
So, minus one plus 2t will be zero.
Two plus t will be two and a half of five halves, and two minus three halves will be one half, OK?
So, this is where the line intersects the plane.
So, you see that's actually a pretty easy way of finding where a line on the plane intersects each other.
If we can find a parametric equation of a line and an equation of a plane, but we basically just plug one into the other, and see at what time the moving point hits the plane so that we know where this.
OK, other questions about this?
Yes?
Sorry, can you say that?
Yes, so what if we don't get a solution?
What happens?
So, indeed our line could have been parallel to the plane or maybe even contained in the plane.
Well, if the line is parallel to the plane then maybe what happens is that what we plug in the positions of the moving point, we actually get something that never equals seven because maybe we get actually a constant.
Say that we had gotten, I don't know, 13 all the time.
Well, when is 13 equal to seven?
The answer is never.
OK, so that's what would tell you that the line is actually parallel to the plane.
You would not find a solution to the equation that you get at the end.
Yes?
So, if there's no solution at all to the equation that you get, it means that at no time is the traveling point going to be in the plane.
That means the line really does not have the plane ever.
So, it has to be parallel outside of it.
On the other hand, if a line is inside the plane, then that means that no matter what time you choose, you always get seven.
OK, that's what would happen if a line is in the plane.
You always get seven.
So, maybe I should write this down.
So, if a line is in the plane then plugging x(t), y(t), z(t) into the equation, we always get, well, here in this case seven or whatever the value should be for the plane, If the line is parallel to the plane -- -- in fact, we, well, get, let's see, another constant.
So, in fact, you know, when you plug in these things, normally you get a quantity that's of a form, something times t plus a constant because that's what you plug into the equation of a plane.
And so, in general, you have an equation of the form, something times t plus something equals something.
And, that usually has a single solution.
And, the special case is if this coefficient of t turns out to be zero in the end, and that's actually going to happen, exactly when the line is either parallel or in the plane.
In fact, if you think this through carefully, the coefficient of t that you get here, see, it's one times two plus two times one plus four times minus three.
It's the dot product between the normal vector of a plane and the vector along the line.
So, see, this coefficient becomes zero exactly when the line is perpendicular to the normal vector.
That means it's parallel to the plane.
So, everything makes sense.
OK, if you're confused about what I just said, you can ignore it.
OK, more questions?
No?
OK, so if not, let's move on to linear parametric equations.
So, I hope you've seen here that parametric equations are a great way to think about lines.
There are also a great way to think about actually any curve, any trajectory that can be traced by a moving point.
So -- -- more generally, we can use parametric equations -- -- for arbitrary motions -- -- in the plane or in space.
So, let's look at an example.
Let's take, so, it's a famous curve called a cycloid.
A cycloid is something that you can actually see sometimes at night when people are biking If you have something that reflects light on the wheel.
So, let me explain what's the definition of a cycloid.
So, I should say, I've seen a lecture where, actually, the professor had a volunteer on a unicycle to demonstrate how that works.
But, I didn't arrange for that, so instead I will explain it to you using more conventional means.
So, let's say that we have a wheel that's rolling on a horizontal ground.
And, as it's rolling of course it's going to turn.
So, it's going to move forward to a new position.
And, now, let's mention that we have a point that's been painted red on the circumference of the wheel.
And, initially, that point is here.
So, as the wheel stops rotating, well, of course, it moves forward, and so it turns on itself.
So, that point starts falling back behind the point of contact because the wheel is rotating at the same time as it's moving forward.
And so, the cycloid is the trajectory of this moving point.
OK, so the cycloid is obtained by considering, so we have a wheel, let's say, of radius a.
So, this height here is (a) rolling on the floor which is the x axis.
And, let's -- And, we have a point, P, that's painted on the wheel.
Initially, it's at the origin.
But, of course, as time goes by, it moves on the wheel.
P is a point on the rim of the wheel, and it starts at the origin.
So, the question is, what happens?
In particular, can we find the position of this point, x(t), y(t), as a function of time?
So, that's the reason why I have this computer.
So, I'm not sure it will be very easy to visualize, but so we have a wheel, well, I hope you can vaguely see that there's a circle that's moving.
The wheel is green here.
And, there's a radius that's been painted blue in it.
And, that radius rotates around the wheel as the wheel is moving forward.
So, now, let's try to paint, actually, the trajectory of a point.
[LAUGHTER] OK, so that's what the cycloid looks like.
[APPLAUSE] OK, so -- So the cycloid, well, I guess it doesn't quite look like what I've drawn.
It looks like it goes a bit higher up, which will be the trajectory of this red point.
And, see, it hits the bottom once in a while.
It forms these arches because when the wheel has rotated by a full turn, then you're basically back at the same situation, except a bit further along the route.
So, if we do it once more, you see the point now is at the top, and now it's at the bottom.
And then we start again.
It's at the top, and then again at the bottom.
OK. No.
[LAUGHTER] OK, so the question that we want to answer is what is the position x(t), y(t), of the point P?
OK, so actually, I'm writing x(t), y(t).
That means that I have, maybe I'm expressing the position in terms of time.
Let's see, is time going to be a good thing to do?
Well, suddenly, the position changes over time.
But doesn't actually matter how fast the wheel is rolling?
No, because I can just play the motion fast-forward.
The wheel will be going faster, but the trajectory is still the same.
So, in fact, time is not the most relevant thing here.
What matters to us now is how far the wheel has gone.
So, we could use as a parameter, for example, the distance by which the wheel has moved.
We can do even better because we see that, really, the most complicated thing that happens here is really the rotation.
So, maybe we can actually use the angle by which the wheel has turned to parameterize the motion.
So, there's various choices.
You can choose whichever one you prefer.
But, I think here, we will get the simplest answer if we parameterize things by the angle.
So, in fact, instead of t I will be using what's called theta as a function of the angle, theta, by which the wheel has rotated.
So, how are we going to do that?
Well, because we are going to try to use our new knowledge, let's try to do it using vectors in a smart way.
So, let me draw a picture of the wheel after things have rotated by a certain amount.
So, maybe my point, P, now, is here.
And, so the wheel has rotated by this angle here.
And, I want to find the position of my point, P, OK?
So, the position of this point, P, is going to be the same as knowing the vector OP from the origin to this moving point.
So, I haven't really simplify the problem yet because we don't really know about vector OP.
But, maybe we know about simpler vectors where some will be OP.
So, let's see, let's give names to a few of our points.
For example, let's say that this will be point A.
A is the point where the wheel is touching the road.
And, B will be the center of the wheel.
Then, it looks like maybe I have actually a chance of understanding vectors like maybe OA doesn't look quite so scary, or AB doesn't look too bad.
BP doesn't look too bad.
And, if I sum them together, I will obtain OP.
So, let's do that.
So, now we've greatly simplified the problem.
We had to find one vector that we didn't know.
Now we have to find three vectors which we don't know.
But, you will see each of them as fairly easy to think about.
So, let's see.
Should we start with vector OA, maybe?
So, OA has two components.
One of them should be very easy.
Well, the y component is just going to be zero, OK?
It's directed along the x axis.
What about the x component?
So, OA is the distance by which the wheel has traveled to get to its current position.
Yeah.
I hear a lot of people saying R theta.
Let me actually say a(theta) because I've called a the radius of the wheel.
So, this distance is a(theta).
Why is it a(theta)?
Well, that's because the wheel, well, there's an assumption which is that the wheel is rolling on something normal like a road, and not on, maybe, ice, or something like that.
S So, it's rolling without slipping.
So, that means that this distance on the road is actually equal to the distance here on the circumference of the wheel.
This point, P, was there, and the amount by which the things have moved can be measured either here or here.
These are the same distances.
OK, so, that makes it a(theta), and maybe I should justify by saying amount by which the wheel has rolled, has moved, is equal to the, so, the distance from O to A is equal to the arc length on the circumference of the circle from A to P. And, you know that if you have a sector corresponding to an angle, theta, then its length is a times theta, provided that, of course, you express the angel in radians.
That's the reason why we always used radians in math.
Now, let's think about vector AB and vector BP.
OK, so AB is pretty easy, right, because it's pointing straight up, and its length is a.
So, it's just zero, a.
Now, the most serious one we've kept for the end.
What about vector BP?
So, vector BP, we know two things about it.
We know actually its length, so, the magnitude of BP -- -- a.
And, we know it makes an angle, theta, with the vertical.
So, that should let us find its components.
Let's draw a closer picture.
Now, in the picture I'm going to center things at B.
So, I have my point P. Here I have theta.
This length is A.
Well, what are the components of BP?
Well, the X component is going to be?
Almost.
I hear people saying things about a, but I agree with a. I hear some cosines.
I hear some sines.
I think it's actually the sine.
Yes.
It's a(sin(theta)), except it's going to the left.
So, actually it will have a negative a(sin(theta)).
And, the vertical component, well, it will be a(cos(theta)), but also negative because we are going downwards.
So, it's negative a(cos(theta)).
So, now we can answer the initial question because vector OP, well, we just add up OA, AB, and BP.
So, the X component will be a(theta) - a(sin(theta)).
And, a-a(cos(theta)).
OK.
So, any questions about that?
OK, so, what's the answer?
Because this thing here is the x coordinate as a function of theta, and that one is the y coordinate as a function of theta.
So, now, just to show you that we can do a lot of things when we have a parametric equation, here is a small mystery.
So, what happens exactly near the bottom point?
What does the curve look like?
The computer tells us, well, it looks like it has some sort of pointy thing, but isn't that something of a display?
Is it actually what happens?
So, what do you think happens near the bottom point?
Remember, we had that picture.
Let me show you once more, where you have these corner-like things at the bottom.
Well, actually, is it indeed a corner with some angle between the two directions?
Does it make an angle?
Or, is it actually a smooth curve without any corner, but we don't see it because it's too small to be visible on the computer screen?
Does it actually make a loop?
Does it actually come down and then back up without going to the left or to the right and without making an angle?
So, yeah, I see the majority votes for answers number two or four.
And, well, at this point, we can't quite tell.
So, let's try to figure it out from these formulas.
The way to answer that for sure is to actually look at the formulas.
OK, so question that we are trying to answer now is what happens near the bottom point?
OK, so how do we answer that?
Well, we should probably try to find simpler formulas for these things.
Well, to simplify, let's divide everything by a.
Let's rescale everything by a.
If you want, let's say that we take the unit of length to be the radius of our wheel.
So, instead of measuring things in feet or meters, we'll just measure them in radius.
So, take the length unit to be equal to the radius.
So, that means we'll have a=1.
Then, our formulas are slightly simpler.
We get x(theta) is theta - sin(theta), and y equals 1 - cos (theta).
OK, so, if we want to understand what these things look like, maybe we should try to take some approximation.
OK, so what about approximations?
Well, probably you know that if I take the sine of a very small angle, it's close to the actual angle itself if theta is very small.
And, you know that the cosine of an angle that's very small is close to one.
Well, that's pretty good.
If we use that, we will get theta minus theta, one minus one, it looks like it's not precise enough.
We just get zero and zero.
That's not telling us much about what happens.
OK, so we need actually better approximations than that.
So -- So, hopefully you have seen in one variable calculus something called Taylor expansion.
That's [GROANS].
I see that -- OK, so if you have not seen Taylor expansion, or somehow it was so traumatic that you've blocked it out of your memory, let me just remind you that Taylor expansion is a way to get a better approximation than just looking at the function, its derivative.
So -- And, here's an example of where it actually comes in handy in real life.
So, Taylor approximation says that if t is small, then the value of the function, f(t), is approximately equal to, well, our first guess, of course, would be f(0).
That's our first approximation.
If we want to be a bit more precise, we know that when we change by t, well, t times the derivative comes in, that's for linear approximation to how the function changes.
Now, if we want to be even more precise, there's another term, which is t^2 over two times the second derivative.
And, if we want to be even more precise, you will have t^3 over six times the third derivative at zero.
OK, and you can continue, and so on.
But, we won't need more.
So, if you use this here, it tells you that the sine of a smaller angle, theta, well, yeah, it looks like theta.
But, if we want to be more precise, then we should add minus theta cubed over six.
And, cosine of theta, well, it's not quite one.
It's close to one minus theta squared over two.
OK, so these are slightly better approximations of sine and cosine for small angles.
So, now, if we try to figure out, again, what happens to our x of theta, well, it would be, sorry, theta minus theta cubed over six.
That's theta cubed over six.
And y, on the other hand, is going to be one minus that.
That's about theta squared over two.
So, now, which one of them is bigger when theta is small?
Yeah, y is much larger.
OK, if you take the cube of a very small number, it becomes very, very, very small.
So, in fact, we can look at that.
So, x, an absolute value, is much smaller than y.
And, in fact, what we can do is we can look at the ratio between y and x.
That tells us the slope with which we approach the origin.
So, y over x is, well, let's take the ratio of this, too.
That gives us three divided by theta.
That tends to infinity when theta approaches zero.
So, that means that the slope of our curve, the origin is actually infinite.
And so, the curve picture is really something like this.
So, the instantaneous motion, if you had to describe what happens very, very close to the origin is that your point is actually not moving to the left or to the right along with the wheel.
It's moving down and up.
I mean, at the same time it is actually moving a little bit forward at the same time.
But, the dominant motion, near the origin is really where it goes down and back up, so answer number four, you have vertical tangent.
OK, I think I'm at the end of time.
So, have a nice weekend.
And, I'll see you on Tuesday.
So, on Tuesday I will have practice exams for next week's test.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, if you remember last time, we looked at parametric equations -- -- as a way of describing the motion of a point that moves in the plane or in space as a function of time of your favorite parameter that will tell you how far the motion has progressed.
And, I think we did it in detail the example of the cycloid, which is the curve traced by a point on a wheel that's rolling on a flat surface.
So, we have this example where we have this wheel that's rolling on the x-axis, and we have this point on the wheel.
And, as it moves around, it traces a trajectory that moves more or less like this.
OK, so I'm trying a new color.
Is this visible from the back?
So, no more blue.
OK, so remember, in general, we are trying to find the position, so, x of t, y of t, maybe z of t if we are in space -- -- of a moving point along a trajectory.
And, one way to think about this is in terms of the position vector.
So, position vector is just the vector whose components are coordinates of a point, OK, so if you prefer, that's the same thing as a vector from the origin to the moving point.
So, maybe our point is here, P. So, this vector here -- This vector here is vector OP.
And, that's also the position vector r of t. So, just to give you, again, that example -- -- if I take the cycloid for a wheel of radius 1, and let's say that we are going at unit speed so that the angle that we used as a parameter of time is the same thing as time when the position vector, in this case, we found to be, just to make sure that they have it right, .
OK, that's a formula that you should have in your notes from last time, except we had theta instead of t because we were using the angle.
But now I'm saying, we are moving at unit speed, so time and angle are the same thing.
So, now, what's interesting about this is we can analyze the motion in more detail.
OK, so, now that we know the position of the point as a function of time, we can try to study how it varies in particular things like the speed and acceleration.
OK, so let's start with speed.
Well, in fact we can do better than speed.
Let's not start with speed.
So, speed is a number.
It tells you how fast you are going along your trajectory.
I mean, if you're driving in a car, then it tells you how fast you are going.
But, unless you have one of these fancy cars with a GPS, it doesn't tell you which direction you're going.
And, that's useful information, too, if you're trying to figure out what your trajectory is.
So, in fact, there's two aspects to it.
One is how fast you are going, and the other is in what direction you're going.
That means actually we should use a vector maybe to think about this.
And so, that's called the velocity vector.
And, the way we can get it, so, it's called usually V, so, V here stands for velocity more than for vector.
And, you just get it by taking the derivative of a position vector with respect to time.
Now, it's our first time writing this kind of thing with a vector.
So, the basic rule is you can take the derivative of a vector quantity just by taking the derivatives of each component.
OK, so that's just dx/dt, dy/dt, and if you have z component, dz/dt.
So, let me -- OK, so -- OK, so let's see what it is for the cycloid.
So, an example of a cycloid, well, so what do we get when we take the derivatives of this formula there?
Well, so, the derivative of t is 1- cos(t).
The derivative of 1 is 0.
The derivative of -cos(t) is sin(t).
Very good.
OK, that's at least one thing you should remember from single variable calculus.
Hopefully you remember even more than that.
OK, so that's the velocity vector.
It tells us at any time how fast we are going, and in what direction.
So, for example, observe.
Remember last time at the end of class we were trying to figure out what exactly happens near the bottom point, when we have this motion that seems to stop and go backwards.
And, we answered that one way.
But, let's try to understand it in terms of velocity.
What if I plug t equals 0 in here?
Then, 1- cos(t) is 0, sin(t) is 0.
The velocity is 0.
So, at the time,at that particular time, our point is actually not moving.
Of course, it's been moving just before, and it starts moving just afterwards.
It's just the instant, at that particular instant, the speed is zero.
So, that's especially maybe a counterintuitive thing, but something is moving.
And at that time, it's actually stopped.
Now, let's see, so that's the vector.
And, it's useful.
But, if you want just the usual speed as a number, then, what will you do?
Well, you will just take exactly the magnitude of this vector.
So, speed, which is the scalar quantity is going to be just the magnitude of the vector, V. OK, so, in this case, while it would be square root of (1- cost)^2 sin^2(t), and if you expand that, you will get, let me take a bit more space, it's going to be square root of 1 - 2cos(t) cos^2(t) sin^2(t).
It seems to simplify a little bit because we have cos^2 plus sin^2.
That's 1.
So, it's going to be the square root of 2 - 2cos(t).
So, at this point, if I was going to ask you, when is the speed the smallest or the largest?
You could answer based on that.
See, at t equals 0, well, that turns out to be zero.
The point is not moving.
At t equals pi, that ends up being the square root of 2 plus 2, which is 4.
So, that's 2.
And, that's when you're truly at the top of the arch, and that's when the point is moving the fastest.
In fact, they are spending twice as fast as the wheel because the wheel is moving to the right at unit speed, and the wheel is also rotating.
So, it's moving to the right and unit speed relative to the center so that the two effects add up, and give you a speed of 2.
Anyway, that's a formula we can get.
OK, now, what about acceleration?
So, here I should warn you that there is a serious discrepancy between the usual intuitive notion of acceleration, the one that you are aware of when you drive a car and the one that we will be using.
So, you might think acceleration is just the directive of speed.
If my car goes 55 miles an hour on the highway and it's going a constant speed, it's not accelerating.
But, let's say that I'm taking a really tight turn.
Then, I'm going to feel something.
There is some force being exerted.
And, in fact, there is a sideways acceleration at that point even though the speed is not changing.
So, the definition will take effect.
The acceleration is, as a vector, and the acceleration vector is just the derivative of a velocity vector.
So, even if the speed is constant, that means, even if a length of the velocity vector stays the same, the velocity vector can still rotate.
And, as it rotates, it uses acceleration.
OK, and so this is the notion of acceleration that's relevant to physics when you find F=ma; that's the (a) that you have in mind here.
It's a vector.
Of course, if you are moving in a straight line, then the two notions are the same.
I mean, acceleration is also going to be along the line, and it's going to has to do with the derivative of speed.
But, in general, that's not quite the same.
So, for example, let's look at the cycloid.
If we take the example of the cycloid, well, what's the derivative of one minus cos(t)?
It's sin(t).
And, what's the derivative of sin(t)?
cos(t), OK.
So, the acceleration vector is .
So, in particular, let's look at what happens at time t equals zero when the point is not moving.
Well, the acceleration vector there will be zero from one.
So, what that means is that if I look at my trajectory at this point, that the acceleration vector is pointing in that direction.
It's the unit vector in the vertical direction.
So, my point is not moving at that particular time.
But, it's accelerating up.
So, that means that actually as it comes down, first it's slowing down.
Then it stops here, and then it reverses going back up.
OK, so that's another way to understand what we were saying last time that the trajectory at that point has a vertical tendency because that's the direction in which the motion is going to occur just before and just after time zero.
OK, any questions about that?
No.
OK, so I should insist maybe on one thing, which is that, so, we can differentiate vectors just component by component, OK, and we can differentiate vector expressions according to certain rules that we'll see in a moment.
One thing that we cannot do, it's not true that the length of dr dt, which is the speed, is equal to the length of dt.
OK, this is completely false.
And, they are really not the same.
So, if you have to differentiate the length of a vector, but basically you are in trouble.
If you really, really want to do it, well, the length of the vector is the square root of the sums of the squares of the components, and from that you can use the formula for the derivative of the square root, and the chain rule, and various other things.
And, you can get there.
But, it will not be a very nice expression.
There is no simple formula for this kind of thing.
Fortunately, we almost never have to compute this kind of thing because, after all, it's not a very relevant quantity.
What's more relevant might be this one.
This is actually the speed.
This one, I don't know what it means.
OK.
So, let's continue our exploration.
So, the next concept that I want to define is that of arc length.
So, arc length is just the distance that you have traveled along the curve, OK?
So, if you are in a car, you know, it has mileage counter that tells you how far you've gone, how much fuel you've used if it's a fancy car.
And, what it does is it actually integrates the speed of the time to give you the arc length along the trajectory of the car.
So, the usual notation that we will have is (s) for arc length.
I'm not quite sure how you get an (s) out of this, but it's the usual notation.
OK, so, (s) is for distance traveled along the trajectory.
And, so that makes sense, of course, we need to fix a reference point.
Maybe on the cycloid, we'd say it's a distance starting on the origin.
In general, maybe you would say you start at time, t equals zero.
But, it's a convention.
If you knew in advance, you could have, actually, your car's mileage counter to count backwards from the point where the car will die and start walking.
I mean, that would be sneaky-freaky, but you could have a negative arc length that gets closer and closer to zero, and gets to zero at the end of a trajectory, or anything you want.
I mean, arc length could be positive or negative.
Typically it's negative what you are before the reference point, and positive afterwards.
So, now, how does it relate to the things we've seen there?
Well, so in particular, how do you relate arc length and time?
Well, so, there's a simple relation, which is that the rate of change of arc length versus time, well, that's going to be the speed at which you are moving, OK, because the speed as a scalar quantity tells you how much distance you're covering per unit time.
OK, and in fact, to be completely honest, I should put an absolute value here because there is examples of curves maybe where your motion is going back and forth along the same curve.
And then, you don't want to keep counting arc length all the time.
Actually, maybe you want to say that the arc length increases and then decreases along the curve.
I mean, you get to choose how you count it.
But, in this case, if you are moving back and forth, it would make more sense to have the arc length first increase, then decrease, increase again, and so on.
So -- So if you want to know really what the arc length is, then basically the only way to do it is to integrate speed versus time.
So, if you wanted to know how long an arch of cycloid is, you have this nice-looking curve; how long is it?
Well, you'd have to basically integrate this quantity from t equals zero to 2 pi.
And, to say the truth, I don't really know how to integrate that.
So, we don't actually have a formula for the length at this point.
However, we'll see one later using a cool trick, and multi-variable calculus.
So, for now, we'll just leave the formula like that, and we don't know how long it is.
Well, you can put that into your calculator and get the numerical value.
But, that's the best I can offer.
Now, another useful notion is the unit vector to the trajectory.
So, the usual notation is T hat.
It has a hat because it's a unit vector, and T because it's tangent.
Now, how do we get this unit vector?
So, maybe I should have pointed out before that if you're moving along some trajectory, say you're going in that direction, then when you're at this point, the velocity vector is going to be tangential to the trajectory.
It tells you the direction of motion in particular.
So, if you want a unit vector that goes in the same direction, all you have to do is rescale it, so, at its length becomes one.
So, it's v divided by a magnitude of v. So, it seems like now we have a lot of different things that should be related in some way.
So, let's see what we can say.
Well, we can say that dr by dt, so, that's the velocity vector, that's the same thing as if I use the chain rule dr/ds times ds/dt.
OK, so, let's think about this things.
So, this guy here we've just seen.
That's the same as the speed, OK?
So, this one here should be v divided by its length.
So, that means this actually should be the unit vector.
OK, so, let me rewrite that.
It's T ds/dt.
So, maybe if I actually stated directly that way, see, I'm just saying the velocity vector has a length and a direction.
The length is the speed.
The direction is tangent to the trajectory.
So, the speed is ds/dt, and the vector is T hat.
And, that's how we get this.
So, let's try just to see why dr/ds should be T. Well, let's think of dr/ds.
dr/ds means position vector r means you have the origin, which is somewhere out there, and the vector r is here.
So, dr/ds means we move by a small amount, delta s along the trajectory a certain distance delta s. And, we look at how the position vector changes.
Well, we'll have a small change.
Let me call that vector delta r corresponding to the size, corresponding to the length delta s. And now, delta r should be essentially roughly equal to, well, its direction will be tangent to the trajectory.
If I take a small enough interval, then the direction will be almost tensioned to the trajectory times the length of it will be delta s, the distance that I have traveled.
OK, sorry, maybe I should explain that on a separate board.
OK, so, let's say that we have that amount of time, delta t. So, let's zoom into that curve.
So, we have r at time t. We have r at time t plus delta t. This vector here I will call delta r. The length of this vector is delta s. And, the direction is essentially that of a vector.
OK, so, delta s over delta t, that's the distance traveled divided by the time.
That's going to be close to the speed.
And, delta r is approximately T times delta s. So, now if I divide both sides by delta t, I get this.
And, if I take the limit as delta t turns to zero, then I get the same formula with the derivatives and with an equality.
It's an approximation.
The approximation becomes better and better if I go to smaller intervals.
OK, are there any questions about this?
Yes?
Yes, that's correct.
OK, so let's be more careful, actually.
So, you're asking about whether the delta r is actually strictly tangent to the curve.
Is that -- That's correct.
Actually, delta r is not strictly tangent to anything.
So, maybe I should draw another picture.
If I'm going from here to here, then delta r is going to be this arc inside the curve while the vector will be going in this direction, OK?
So, they are not strictly parallel to each other.
That's why it's only approximately equal.
Similarly, this distance, the length of delta r is not exactly the length along the curve.
It's actually a bit shorter.
But, if we imagine a smaller and smaller portion of the curve, then this effect of the curve being a curve and not a straight line becomes more and more negligible.
If you zoom into the curve sufficiently, then it looks more and more like a straight line.
And then, what I said becomes true in the limit.
OK?
Any other questions?
No?
OK.
So, what happens next?
OK, so let me show you a nice example of why we might want to use vectors to study parametric curves because, after all, a lot of what's here you can just do in coordinates.
And, we don't really need vectors.
Well, and truly, vectors being a language, you never strictly need it, but it's useful to have a notion of vectors.
So, I want to tell you a bit about Kepler's second law of celestial mechanics.
So, that goes back to 1609.
So, that's not exactly recent news, OK?
But, still I think it's a very interesting example of why you might want to use vector methods to analyze motions.
So, what happened back then was Kepler was trying to observe the motion of planets in the sky, and trying to come up with general explanations of how they move.
Before him, people were saying, well, they cannot move in a circle.
But maybe it's more complicated than that.
We need to add smaller circular motions on top of each other, and so on.
They have more and more complicated theories.
And then Kepler came with these laws that said basically that planets move in an ellipse around the sun, and that they move in a very specific way along that ellipse.
So, there's actually three laws, but let me just tell you about the second one that has a very nice vector interpretation.
So, what Kepler's second law says is that the motion of planets is, first of all, they move in a plane.
And second, the area swept out by the line from the sun to the planet is swept at constant time.
Sorry, is swept at constant rate.
From the sun to the planet, it is swept out by the line at a constant rate.
OK, so that's an interesting law because it tells you, once you know what the orbit of the planet looks like, it tells you how fast it's going to move on that orbit.
OK, so let me explain again.
So, this law says maybe the sun, let's put the sun here at the origin, and let's have a planet.
Well, the planet orbits around the sun -- -- in some trajectory.
So, this is supposed to be light blue.
Can you see that it's different from white?
No?
OK, me neither.
[LAUGHTER] OK, it doesn't really matter.
So, the planet moves on its orbit.
And, if you wait for a certain time, then a bit later it would be here, and then here, and so on.
Then, you can look at the amount of area inside this triangular wedge.
And, the claim is that the amount of area in here is proportional to the time elapsed.
So, in particular, if a planet is closer to the sun, then it has to go faster.
And, if it's farther away from the sun, then it has to go slower so that the area remains proportional to time.
So, it's a very sophisticated prediction.
And, I think the way he came to it was really just by using a lot of observations, and trying to measure what was true that wasn't true.
But, let's try to see how we can understand that in terms of all we know today about mechanics.
So, in fact, what happens is that Newton, so Newton was quite a bit later.
That was the late 17th century instead of the beginning of the 17th century.
So, he was able to explain this using his laws for gravitational attraction.
And, you'll see that if we reformulate Kepler's Law in terms of vectors, and if we work a bit with these vectors, we are going to end up with something that's actually completely obvious to us now.
At the time, it was very far from obvious, but to us now to completely obvious.
So, let's try to see, what does Kepler's law say in terms of vectors?
OK, so, let's think of what kinds of vectors we might want to have in here.
Well, it might be good to think of, maybe, the position vector, and maybe its variation.
So, if we wait a certain amount of time, we'll have a vector, delta r, which is the change in position vector a various interval of time.
OK, so let's start with the first step.
What's the most complicated thing in here?
It's this area swept out by the line.
How do we express that area in terms of vectors?
Well, I've almost given the answer by drawing this picture, right?
If I take a sufficiently small amount of time, this shaded part looks like a triangle.
So, we have to find the area of the triangle.
Well, we know how to do that now.
So, the area is approximately equal to one half of the area of a parallelogram that I could form from these vectors.
And, the area of a parallelogram is given by the magnitude of a cross product.
OK, so, I should say, this is the area swept in time delta t. You should think of delta t as relatively small.
I mean, the scale of a planet that might still be a few days, but small compared to the other old trajectory.
So, let's remember that the amount by which we moved, delta r, is approximately equal to v times delta t, OK, and just using the definition of a velocity vector.
So, let's use that.
Sorry, so it's approximately equal to r cross v magnitude times delta t. I can take out the delta t, which is scalar.
So, now, what does it mean to say that area is swept at a constant rate?
It means this thing is proportional to delta t. So, that means, so, the law says, in fact, that the length of this cross product r cross v equals a constant.
OK, r cross v has constant length.
Any questions about that?
No?
Yes?
Yes, let me try to explain that again.
So, what I'm claiming is that the length of the cross products r cross v measures the rate at which area is swept by the position vector.
I should say, with a vector of one half of this length is the rate at which area is swept.
How do we see that?
Well, let's take a small time interval, delta t. In time, delta t, our planet moves by v delta t, OK?
So, if it moves by v delta t, it means that this triangle up there has two sides.
One is the position vector, r. The other one is v delta t. So, its area is given by one half of the magnitude of a cross product.
That's the formula we've seen for the area of a triangle in space.
So, the area is one half of the cross product, r, and v delta t, magnitude of the cross product.
So, to say that the rate at which area is swept is constant means that these two are proportional.
Area divided by delta t is constant at our time.
And so, this is constant.
OK, now, what about the other half of the law?
Well, it says that the motion is in a plane, and so we have a plane in which the motion takes place.
And, it contains, also, the sun.
And, it contains the trajectory.
So, let's think about that plane.
Well, I claim that the position vector is in the plane.
OK, that's what we are saying.
But, there is another vector that I know it is in the plane.
You could say the position vector at another time, or at any time, but in fact, what's also true is that the velocity vector is in the plane.
OK, if I'm moving in the plane, then position and velocity are in there.
So, the plane of motion contains r and v. So, what's the direction of the cross product r cross v?
Well, it's the direction that's perpendicular to this plane.
So, it's normal to the plane of motion.
And, that means, now, that actually we've put the two statements in there into a single form because we are saying r cross v has constant length and constant direction.
In fact, in general, maybe I should say something about this.
So, if you just look at the position vector, and the velocity vector for any motion at any given time, then together, they determine some plane.
And, that's the plane that contains the origin, the point, and the velocity vector.
If you want, it's the plane in which the motion seems to be going at the given time.
Now, of course, if your motion is not in a plane, then that plane will change.
It's, however, instant, if a plane in which the motion is taking place at a given time.
And, to say that the motion actually stays in that plane forever means that this guy will not change direction.
OK, so -- [LAUGHTER] [APPLAUSE] OK, so, Kepler's second law is actually equivalent to saying that r cross v equals a constant vector, OK?
That's what the law says.
So, in terms of derivatives, it means d by dt of r cross v is the zero vector.
OK, now, so there's an interesting thing to note, which is that we can use the usual product rule for derivatives with vector expressions, with dot products or cross products.
There's only one catch, which is that when we differentiate a cross product, we have to be careful that the guy on the left stays on the left.
The guy on the right stays on the right.
OK, so, if you know that uv prime equals u prime v plus uv prime, then you are safe.
If you know it as u prime v cross v prime u, then you are not safe.
OK, so it's the only thing to watch for.
So, product rule is OK for taking the derivative of a dot product.
There, you don't actually even need to be very careful about all the things or the derivative of a cross product.
There you just need to be a little bit more careful.
OK, so, now that we know that, we can write this as dr/dt cross v plus r cross dv/dt, OK?
Well, let's reformulate things slightly.
So, dr dt already has a name.
In fact, that's v. OK, that's what we call the velocity vector.
So, this is v cross v plus r cross, what is dv/dt?
That's the acceleration, a, equals zero.
OK, so now what's the next step?
Well, we know what v cross v is because, remember, a vector cross itself is always zero, OK?
So, this is the same r cross a equals zero, and that's the same as saying that the cross product of two vectors is zero exactly when the parallelogram of the form has no area.
And, the way in which that happens is if they are actually parallel to each other.
So, that means the acceleration is parallel to the position.
OK, so, in fact, what Kepler's second law says is that the acceleration is parallel to the position vector.
And, since we know that acceleration is caused by a force that's equivalent to the fact that the gravitational force -- -- is parallel to the position vector, that means, well, if you have the sun here at the origin, and if you have your planets, well, the gravitational force caused by the sun should go along this line.
In fact, the law doesn't even say whether it's going towards the sun or away from the sun.
Well, what we know now is that, of course, the attraction is towards the sun.
But, Kepler's law would also be true, actually, if things were going away.
So, in particular, say, electric force also has this property of being towards the central charge.
So, actually, if you look at motion of charged particles in an electric field caused by a point charged particle, it also satisfies Kepler's law, satisfies the same law.
OK, that's the end for today, thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Let me just tell you first about the list of topics.
Basically, the list of topics is simple.
It is everything.
I mean, everything we have seen so far is on the exam.
But let me just remind you of the main topics that we have seen.
First of all, we learned about vectors, how to use them, and dot-product.
At this point, you probably should know that the dot-product of two vectors is obtained by summing products of components.
And geometrically it is the length of A times the length of B times the cosine of the angle between them.
And, in particular, we can use dot-product to measure angles by solving for cosine theta in this equality.
And most importantly to detect whether two vectors are perpendicular to each other.
Two vectors are perpendicular when their dot-product is zero.
Any questions about that?
No.
Is everyone reasonably happy with dot-product by now?
I see a stunned silence.
Nobody happy with dot-product so far?
OK.
If you want to look at Practice 1A, a good example of a typical problem with dot-product would be problem 1.
Let's see.
We are going to go over the practice exam when I am done writing this list of topics.
I think probably we actually will skip this problem because I think most of you know how to do it.
And if not then you should run for help from me or from your recitation instructor to figure out how to do it.
The second topic that we saw was cross-product.
When you have two vectors in space, you can just form that cross-product by computing its determinant.
So, implicitly, we should also know about determinants.
By that I mean two by two and three by three.
Don't worry about larger ones, even if you are interested, they won't be on the test.
And applications of cross-product, for example, finding the area of a triangle or a parallelogram in space.
If you have a triangle in space with sides A and B then its area is one-half of the length of A cross B.
Because the length of A cross B is length A, length B sine theta, which is the same as the area of the parallelogram formed by these two vectors.
And the other application of cross-product is to find a vector that's perpendicular to two given vectors.
In particular, to find the vector that is normal to a plane and then find the equation of a plane.
Another application is finding the normal vector to a plane and using that finding the equation of a plane.
Basically, remember, to find the equation of a plane, ax by cz = d, what you need is the normal vector to the plane.
And the components of the normal vector are exactly the coefficients that go into this.
And we have seen an argument for why that happens to be the case.
To find a normal vector to a plane typically what we will do is take two vectors that lie in the plane and will take their cross-product.
And the cross-product will automatically be perpendicular to both of them.
We are going to see an example of that when we look at problem 5 in practice 1A.
I think we will try to do that one.
Another application, well, we will just mention it as a topic that goes along with this one.
We have seen also about equations of lines and how to find where a line intersects a plane.
Just to refresh your memories, the equation of a line, well, we will be looking at parametric equations.
To know the parametric equation of a line, we need to know a point on the line and we need to know a vector that is parallel to the line.
And, if we know a point on the line and a vector along the line, then we can express the parametric equations for the motion of a point that is moving on the line.
Actually, starting at point, at time zero, and moving with velocity v. To put things in symbolic form, you will get a position of that point by starting with a position of time zero and adding t times the vector v. It gives you x, y and z in terms of t. And that is how we represent lines.
We will look at problem 5 in a bit, but any general questions about these topics?
No.
Do you have a question?
Do we have to know Taylor series?
That is a good question.
No, not on the exam.
[APPLAUSE] Taylor series are something you should be aware of, generally speaking.
It will be useful for you in real life, probably not when you go to the supermarket, but if you solve engineering problems you will need Taylor series.
It would be good not to forget them entirely, but on the 18.02 exams they probably won't be there.
Let me continue with more topics.
And then we can see if you can think of other topics that should or should not be on the exam.
Third topics would be matrices, linear systems, inverting matrices.
I know that most of you that have calculators that can invert matrices, but still you are expected at this point to know how to do it by hand.
If you have looked at the practice tests, both of them have a problem that asks you to invert a matrix or at least do part of it.
And so it is very likely that tomorrow there will be a problem like that as well.
In general, when a kind of problem is on both practice tests it's a good indication that it might be there also on the actual exam.
Unfortunately, not with the same matrix so you cannot learn the answer by heart.
Another thing that we have learned about, well, I should say this is going to be problem 3 on the test and will on the practice test.
On the actual test, too, I think, actually.
Anyway, we will come back to it later.
A couple of things that you should remember.
If you have a system of the form AX equals B then there are two cases.
If a determinant of A is not zero then that means you can compute the inverse matrix and you can just solve by taking A inverse times B.
And the other case is when the determinant of A is zero, and there is either no solution or there is infinitely many solutions.
In particular, if you know that there is a solution, for example, if B is zero there always is an obvious solution, X equals zero, then you will actually have infinitely many.
In general, we don't really know how to tell whether it is no solution or infinitely many.
Questions about that?
Yes?
Will we have to know how to rotate vectors and so on?
Not in general, but you might still want to remember how to rotate a vector in a plane by 90 degrees because that has been useful when we have done problems about parametric equations, which is what I am coming to next.
What we have seen about rotation matrices, that was the homework part B problem, you are not supposed to remember by heart everything that was in part B of your homework.
It is a good idea to have some vague knowledge because it is useful culture, I would say, useful background for later in your lives, but I won't ask you by heart to know what is the formation for a rotation matrix.
And then we come to, last by not least, the problem of finding parametric equations.
And, in particular, possibly by decomposing the position vector into a sum of simpler vectors.
You have seen quite an evil exam of that on the last problem set with this picture that maybe by now you have had some nightmares about.
Anyway, the one on the exam will certainly be easier than that.
But, as you have seen -- I mean, you should know, basically, how to analyze a motion that is being described to you and express it in terms of vectors and then figure out what the parametric equation will be.
Now, again, it won't be as complicated on the exam as the one in the problem set.
But there are a couple of those on the practice exam, so that gives you an idea of what is realistically expected of you.
And now once we have parametric equations for motion, so that means when we know how to find the position vector as a function of a parameter maybe of time, then we have seen also about velocity and acceleration, which the vector is obtained by taking the first and second derivatives of a position vector.
And so one topic that I will add in there as well is somehow how to prove things about motions by differentiating vector identities.
One example of that, for example, is when we try to look at Kepler's law in class last time.
We look at Kepler's second law of planetary motion, and we reduced it to a calculation about a derivative of the cross-product R cross v. Now, on the exam you don't need to know the details of Kepler's law, but you need to be able to manipulate vector quantities a bit in the way that we did.
And so on practice exam 1A, you actually have a variety of problems on this topics because you have problems two, four and six, all about parametric motions.
Probably tomorrow there will not be three distinct problems about parametric motions, but maybe a couple of them.
I think that is basically the list of topics.
Anybody spot something that I have forgotten to put on the exam or questions about something that should or should not be there?
You go first.
Yeah?
How about parametrizing weird trigonometric functions?
I am not sure what you mean by that.
Well, parametric curves, you need to know how to parameterize motions, and that involves a little bit of trigonometrics.
When we have seen these problems about rotating wheels, say the cycloid, for example, and so on there is a bit of cosine and sine and so on.
I think not much more on that.
You won't need obscure trigonometric identities.
You're next.
Any proofs on the exam or just like problems?
Well, a problem can ask you to show things.
It is not going to be a complicated proof.
The proofs are going to be fairly easy.
If you look at practice 1A, the last problem does have a little bit of proof.
6B says that show that blah, blah, blah.
But, as you will see, it is not a difficult kind of proof.
So, about the same.
Yes?
Are there equations of 3D shapes that we should know at this point?
We should know definitely a lot about the equations of planes on lines.
And you should probably know that a sphere centered at the origin is the set of points where distance to the center is equal to the radius of the sphere.
We don't need more at this point.
As the semester goes on, we will start seeing cones and things like that.
But at this point planes, lines.
And maybe you need to know about circles and spheres, but nothing beyond that.
More questions?
Yes?
If there is a formula that you have proved on the homework then, yes, you can assume it on the test.
Maybe you want to write on your test that this is a formula you have seen in homework just so that we know that you remember it from homework and not from looking over your neighbor's shoulder or whatever.
Yes, it is OK to use things that you know general-speaking.
That being said, for example, probably there will be a linear system to solve.
It will say on the exam you are supposed to solve that using matrices, not by elimination.
There are things like that.
If a problem says solve by using vector methods, things like that, then try to use at least a vector somewhere.
But, in general, you are allowed to use things that you know.
Yes?
Will we need to go from parametric equations to xy equations?
Well, let's say only if it is very easy.
If I give you a parametric curve, sin t, sin t, then you should be able to observe that it is on the line y equals x, not beyond that.
Yes?
Do we have to use -- Yes.
I don't know if you will have to use it, but certainly you should know a little bit about the unit tangent vector.
Just remember the main thing to know that the unit tangent vector is velocity divided by the speed.
I mean there is not much more to it when you think about it.
Yes?
Kepler's law, well, you are allowed to use it if it helps you, if you find a way to squeeze it in.
You don't have to know Kepler's law in detail.
You just have to know how to reproduce the general steps.
If I tell you R cross v is constant, you might be expected to know what to do with that.
I would say -- Basically, you don't need to know Kepler's law.
You need to know the kind of stuff that we saw when we derived it such as how to take the derivative of a dot-product or a cross-product.
That is basically the answer.
I don't see any questions anymore.
Oh, you are raising your hand.
Yes.
How to calculate the distance between two lines and the distance between two planes?
Well, you have seen, probably recently, that it is quite painful to do in general.
And, no, I don't think that will be on the exam by itself.
You need to know how to compute the distance between two points.
That certainly you need to know.
And also maybe how to find the compliment of a vector in a certain direction.
And that is about it, I would say.
I mean the more you know about things the better.
Things that come up on part Bs of the problem sets are interesting things, but they are usually not needed on the exams.
If you have more questions then you are not raising your hand high enough for me to see it.
OK. Let's try to do a bit of this practice exam 1A.
Hopefully, everybody has it.
If you don't have it, hopefully your neighbor has it.
If you don't have it and your neighbor doesn't have it then please raise your hand.
I have a couple.
If you neighbor has it then just follow with them for now.
I think there are a few people behind you over there.
I will stop handing them out now.
If you really need one, it is on the website, it will be here at the end of class.
Let's see.
Well, I think we are going to just skip problems 1 and 2 because they are pretty straightforward and I hope that you know how to do them.
I mean I don't know.
Let's see.
How many of you have no problem with problem 1?
How many of you have trouble with problem 1?
OK. How many of you haven't raised your hands?
OK. How many of you have trouble with problem 2?
OK. Well, if you have questions about those, maybe you should just come see me at the end because that is probably more efficient that way.
I am going to start right away with problem 3, actually.
Problem 3 says we have a matrix given to us |1 3 2; 2 0 - 1; 1 1 0|.
And it tells us determinant of A is 2 and inverse equals something, but we are missing two values A and B and we are supposed to find them.
That means we need to do the steps of the algorithm to find the inverse of A.
We are told that A inverse is one-half of |1 ... ...; - 1 - 2 5; 2 2 - 6|.
And here there are two unknown values.
Remember, to invert a matrix, first we compute the minors.
Then we flip some signs to get the cofactors.
Then we transpose.
And, finally, we divide by the determinant.
Let's try to be smart about this.
Do we need to compute all nine minors?
No.
We only need to compute two of them, right?
Which minors do we need to compute?
Here and here or here and here?
Yeah, that looks better.
Because, remember, we need to transpose things so these two guys will end up here.
I claim we should compute these two minors.
And we will see if that is good enough.
If you start doing others and you find that they don't end up in the right place then just do more, but you don't need to spend your time computing all nine of them.
If you are worried about not doing it right then, of course, you can maybe compute one or two more to just double-check your answers.
But let us just do those that we think are needed.
The matrix of minors.
The one that goes in the middle position is obtained by deleting this row and that column, and we are left with a determinant |3 2;1 0|, 3 times 0 minus 1 times 2 should be - 2 should be - 2.
Then the one in the lower left corner, we delete the last row and the first column, we are left with |3 2; 0 - 1|.
3 times (- 1) is negative 3 minus 0.
We are still left with negative three.
Is that step clear for everyone?
Then we need to go to cofactors.
That means we need to change signs.
The rule is -- We change signs in basically these four places.
That means we will be left with positive 2 and negative 3.
Then we take the transpose.
That means the first column will copy into the first row, so this guy we still don't know, but here we will have two and here we will have minus three.
Finally, we have to divide by the determinant of A.
And here we are actually told that the determinant of A is two.
So we will divide by two.
But there is only one-half here so actually it is done for us.
The values that we will put up there are going to be 2 and negative 3.
Now let's see how we use that to solve a linear system.
If we have to solve a linear system, Ax equals B, well, if the matrix is invertible, its determinant is not zero, so we can certainly write x equals A inverse B.
So we have to multiply, that is one-half | 1 2 - 3; - 1 - 2 5; 2 2 - 6|.
Times B [ 1, - 2,1].
Remember, to do a matrix multiplication you take the rows in here, the columns in here and you do dot-products.
The first entry will be one times one plus two times minus two plus minus three times one, one minus four minus three should be negative six, except I still have, of course, a one-half in front.
Then minus one plus four plus five should be 8.
Two minus four minus six should be -8.
That will simplify to [- 3,4, - 5].
Any questions about that?
OK. Now we come to part C which is the harder part of this problem.
It says let's take this matrix A and let's replace the two in the upper right corner by some other number C. That means we will look at 1 3 C; 2 0 - 1; 1 1 0|.
And let's call that M. And it first asks you to find the value of C for which this matrix is not invertible.
M is not invertible exactly when the determinant of M is zero.
Let's compute the determinant.
Well, we should do one times that smaller determinant, which is zero minus negative one, which is 1 times 1 minus three times that determinant, which is zero plus one is 1.
And then we have plus C times the lower left determinant which is two times one minus zero is 2.
That gives us one minus three is - 2 2C.
That is zero when C equals 1.
For C equals 1, this matrix is not invertible.
For other values it is invertible.
It goes on to say let's look at this value of C and let's look at the system Mx equals zero.
I am going to put value one in there.
Now, if we look at Mx equals zero, well, this has either no solution or infinitely many solutions.
But here there is an obvious solution.
Namely x equals zero is a solution.
Maybe let me rewrite it more geometrically.
X 3 y z = 0.
2x - z = 0.
And x y = 0.
You see we have an obvious solution, (0,0, 0).
But we have more solutions.
How do we find more solutions?
Well, (x, y, z) is a solution if it is in all three of these planes.
That is a way to think about it.
Probably we are actually in this situation where, in fact, we have three planes that are all passing through the origin and all parallel to the same line.
And so that would be the line of solutions.
To find it actually we can think of this as follows.
The first observation is that actually in this situation we don't need all three equations.
The fact that the system has infinitely many solutions means that actually one of the equations is redundant.
If you look at it long enough you will see, for example, if you multiply three times this equation and you subtract that one then you will get the first equation.
Three times (x y) - (2x - z) will be x 3y z.
Now, we don't actually need to see that to solve a problem.
I am just showing you that is what happens when you have a matrix with determinant zero.
One of the equations is somehow a duplicate of the others.
We don't actually need to figure out how exactly.
What that means is really we want to solve, let's say start with two of the equations.
To find the solution we can observe that the first equation says actually that <x, y, z> dot-product with =0.
And the second equation says dot-product with <2,0,- 1> is zero.
And the third equation, if we really want to keep it, says we should be also having this.
Now, these equations now written like this, they are just saying we want an x, y, z that is perpendicular to these vectors.
Let's forget this one and let's just look at these two.
They are saying we want a vector that is perpendicular to these two given vectors.
How do we find that?
We do the cross-product.
To find x, y, z perpendicular to <1,3, 1> and <2,0, - 1>, we take the cross-product.
And that will give us something.
Well, let me just give you the answer.
I am sure you know how to do cross-products by now.
I don't have the answer here, so I guess I have to do it.
That should be <- 3, probably positive 3, and then - 6>.
That is the solution.
And any multiple of that is a solution.
If you like to neatly simplify them you could say negative one, one, negative two.
If you like larger numbers you can multiply that by a million.
That is also a solution.
Any questions about that?
Yes?
That is correct.
If you pick these two guys instead, you will get the same solution.
Well, up to a multiple.
It could be if you do the cross-product of these two guys you actually get something that is a multiple -- Actually, I think if you do the cross-product of the first and third one you will get actually minus one, one, minus two, the smaller one.
But it doesn't matter.
I mean it is really in the same direction.
This is all because a plane has actually normal vectors of all sizes.
Yes?
I don't think so because -- An important thing to remember about cross-product is we compute for minors, but then we put a minus sign on the second component.
The coefficient of j in here, the second component, you do one times minus two times one.
That is negative three indeed.
But then you actually change that to a positive three.
Yes?
Well, we don't have parametric equations here.
Oh, solving by elimination.
Well, if it says that you have to use vector methods then you should use vector methods.
If it says you should use vectors and matrices then you are expected to do it that way.
Yes?
It depends what the problem is asking.
The question is, is it enough to find the components of a vector or do we have to find the equation of a line?
Here it says find one solution using vector operations.
We have found one solution.
If you wanted to find the line then it would all the things that are proportional to this.
It would be maybe minus 3t, 3t minus 6t, all the multiples of that vector.
We do because (0,0, 0) is an obvious solution.
Maybe I should write that on the board.
You had another question?
Not quite.
Let me re-explain first how we get all the solutions and why I did that cross-product.
First of all, why did I take that cross-product again?
I took that cross-product because I looked at my three equations and I observed that my three equations can be reformulated in terms of these dot-products saying that x, y, z is actually perpendicular these guys and these guys have normal vectors to the planes.
Remember, to be in all three planes it has to be perpendicular to the normal vectors.
That is how we got here.
And now, if we want something that is perpendicular to a bunch of given vectors, well, to be perpendicular to two vectors, an easy way to find one is to take that cross-product.
And, if you take any two of them, you will get something that is the same up to scaling.
Now, what it means geometrically is that when we have our three planes and they all actually contain the same line -- And we know that is actually the smae case because they all pass through the origin.
They pass through the origin because the constant terms are just zero.
What happens is that the normal vectors to these planes are, in fact, all perpendicular to that line.
The normal vectors -- Say this line is vertical.
The normal vectors are all going to be horizontal.
Well, it is kind of hard to draw.
By taking the cross-product between two normal vectors we found this direction.
Now, to find actually all the solutions.
What we know so far is that we have this direction <-3 3 - 6>.
That is going to be parallel to the line of intersections.
Let me do it here, for example, .
Now we have one particular solution.
0,0, 0.
Actually, we have found another one, too, which is <- 3,3, - 6>.
Anyway, if a line of solutions -- -- has parametric equation x = - 3t, y = 3t, z = - 6t, anything proportional to that.
That is how we would find all the solutions if we wanted them.
It is almost time.
I think I need to jump ahead to other problems.
Let's see.
I think problem 4 you can probably find for yourselves.
It is a reasonably straightforward parametric equation problem.
You just have to find the coordinates of point P. And for that it is a very simple trick.
Problem 5.
Find the area of a spaced triangle.
It sounds like a cross-product.
Find the equation of a plane also sounds like a cross-product.
And find the intersection of this plane with a line means we find first the parametric equation of the line and then we plug that into the equation of the plane to get where they intersect.
Does that sound reasonable?
Who is disparate about problem 5?
OK. Let me repeat problem 5.
First part we need to find the area of a triangle.
And the way to do that is to just do one-half the length of a cross-product.
If we have three points, P0, P1, P2 then maybe we can form vectors P0P1 and P0P2.
And, if we take that cross-product and take the length of that and divide by two, that will give us the area of a triangle.
Here it turns out that this guy is , if I look at the solutions, so you will end up with square root of 6 over 2.
The second is asking you for the equation of a plane containing these three points.
Well, first of all, we know that a normal vector to the plane is going to be given by this cross-product again.
That means that the equation of plane will be of a form x plus y plus 2z equals something.
If a coefficient is here it comes from the normal vector.
And to find what goes in the right-hand side, we just plug in any of the points.
If you plug in P0, which is (2,1, 0) then two plus one seems like it is 3.
And, if you want to double-check your answer, you can take P1 and P2 and check that you also get three.
It is a good way to check your answer.
Then the third part.
We have a line parallel to the vector v equals one, one, one through the point S, which is (- 1,0, 0).
That means you can find its parametric equation.
X will start at - 1, increases at rate 1.
Y starts at zero, increases at rate one.
Z starts at zero, increases at rate one.
You plug these into the plane equation, and that will tell you where they intersect.
Is that clear?
And now, in the last one minute, on that side I have one minute, let me just say very quickly -- Well, do you want to hear about problem 6 anyway very quickly?
Yeah.
OK.
Problem 6 is one of these like vector calculations.
It says we have a position vector R. And it asks you how do we find the derivative of R dot R?
Well, remember we have a product rule for taking the derivative.
UV prime is U prime V plus UV prime.
It also applies for dot-product.
That is dR by dt dot R plus R dot dR by dt.
And these are both the same thing.
You get two R dot dR/dt, but dR/dt is v for velocity vector.
Hopefully you have seen things like that.
Now, it says show that if R has constant length then they are perpendicular.
All you need to write basically is we assume length R is constant.
That is what it says, R has constant length.
Well, how do we get to, say, something we probably want to reduce to that?
Well, if R is constant in length then R dot R is also constant.
And so that means d by dt of R dot R is zero.
That is what it means to be constant.
And so that means R dot v is zero.
That means R is perpendicular to v. That is a proof.
It is not a scary proof.
And then the last question of the exam says let's continue to assume that R has constant length, and let's try to find R dot v. If there is acceleration then probably we should bring it in somewhere, maybe by taking a derivative of something.
If we know that R dot v equals zero, let's take the derivative of that.
That is still zero.
But now, using the product rule, dR/dt is v dot v plus R dot dv/dt is going to be zero.
That means that you are asked about R dot A.
Well, that is equal to minus V dot V. And that is it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, so far, we've seen things about vectors, equation of planes, motions in space, and so on.
Basically we've done geometry in space.
But, calculus, really, is about studying functions.
Now, we're going to actually move on to studying functions of several variables.
So, this new unit, what we'll do over the next three weeks or so will be about functions of several variables and their derivatives.
OK, so first of all, we should try to figure out how we are going to think about functions.
So, remember, if you have a function of one variable, that means you have a quantity that depends on one parameter.
Maybe f depends on the variable x.
And, for example, a function that you all know is f of x equals sin(x).
And, the way we would represent that is maybe just by plotting the graph of the function.
So, the graph of a function, we plot y = f(x).
And, the graph of a sine function that looks like this.
OK, so now, let's say that we had, actually, a function of two variables.
So, that means that the value of F depends actually on two different parameters, say, if the variables are x and y, or they can have any names you want.
So, given values of the two parameters, x and y, the function will give us a number that we'll call f(x, y).
That depends on x and y according to some formula, OK, not very surprising so far.
So, for example, I can give you the function f(x, y) = x^2 y^2.
And, of course, as with functions of one variable, we don't need things to be defined everywhere.
Sometimes there is the domain of definition.
So, this one is defined all the time.
But, if I tell you, say, f of x, y equals square root of y, well, this is only defined if y is nonnegative.
If I tell you f(x, y) equals one over x y, that's only defined if x y is not zero, and so on.
Now, so these are mathematical examples given by explicit formulas.
And, of course, there's physical examples.
For example, so examples coming from real life, so for example, you can look at the temperature at the certain point on the surface of the earth.
So, you use maybe longitude and latitude; that's x and y.
And then you have f(x, y) equals the temperature at that point.
In fact, because temperature depends also may be on how high up you are.
It depends on elevation.
So, it's actually a function of maybe x, y, z.
And, it also depends on time.
So, in fact, maybe it's a function of t in x y z coordinates in space.
So, you see that real-world functions can depends on a lot of variables.
So, our goal will be to understand how to deal with that.
OK, so now you will see very soon, but actually it's already tricky enough to picture a function of two variables.
So, we are going to focus on the case of functions of two variables.
And then, we'll see that if we have more than two variables, then it's harder to plot the function.
We cannot draw with the graph looks like anymore.
But, the tools are the same, the notion of partial derivatives, grade and vector, and so on, all the tools that we will develop work exactly the same way no matter how many variables you have.
So, I'll say, for simplicity -- -- we'll focus mostly on two or sometimes three variables.
But, it works the same in any number of variables.
OK, so the first question is how to visualize a function of two variables.
So, the first thing we can do is try to draw the graph of f. So, maybe I should say f -- which is a function of two variables.
So, the first answer will be, we can try to look at it's graph.
And, the idea is the same as with one variable, namely, we look at all the possible values of the parameters, x and y, and for each of them, we plot a point whose height is the value of a function at these parameters.
So, we'll plot, let's say, z equals f(x, y).
And, now that will become, actually, a surface in space.
OK, so for each value of x and y, yeah, we have x, y in the x, y plane, then we'll plot the point in space at position x, y.
And, z equals f of x, y. OK, and if we take all of these points together, they will give us some surface that sits in space.
Yes?
Oh, a function of two variables, shorthand.
Well, let's say how to visualize a function of two variables.
OK, so, how do we do that concretely?
Say that I give you a formula for f. How do we try to represent it?
So, let's do our first example.
Let's say I give you a function f(x, y) = -y. OK, so it looks a little bit silly because it doesn't depend on x.
But, that's not the problem.
It's still a valid function of x and y.
It just happens to be constant with respect to x.
So, to draw the graph we look at the surface in space defined by z equals y.
What kind of surface is that?
It's a plane, OK?
And, if we want to draw it, z equals minus y will look, well, let's put y axis.
Let's put x axis.
Let's put z axis.
If I look at what happens in the y, z plane in the plane of a blackboard, it will just look like a line that goes downward with slope one.
OK, so it will be this.
And, what happens if I change x?
Well, if I change x, nothing happens because x doesn't appear in this equation.
So, in fact, if instead of setting x equal to zero I set x equal to one, I'm in front of the blackboard, or minus one at the back.
Well, it still looks exactly the same.
So, I have this plane that actually contains the x axis and slopes downwards with slope one.
It's kind of hard to draw.
Now, you see immediately what the big problem with graphs will be.
But, these pictures are hard to read.
But that's our first graph.
OK, a question so far?
OK, so let's say that we have a slightly more complicated function.
How do we see it?
So, let's draw another example.
Let's say I give you f(x, y) = 1 - x^2-y^2.
So, we should try to picture what the surface z=1- x^2-y^2 looks like.
So, how do we do that?
Well, maybe you are very fast and figured out what it looks like.
But, if not, then we need to work piece by piece.
So, maybe it will help if we understand first what it does in the plane of the blackboard.
So, if we look at it in the y, z plane, that means we set x equal to zero.
And then, z becomes 1 - y^2.
What is that?
It's a parabola pointing downwards, and starting at one.
So, we should draw maybe this downward parabola.
It starts at one and it cuts the y axis at one.
When y is one, that gives us zero.
So, we might have an idea of what it might look like, or maybe not.
Let's get more slices.
Let's see what it does in the x, z plane, this other vertical plane that's coming towards us.
So, in the x, z plane, if we set y equal to zero, we get z equals one minus x^2.
It's, again, a parabola going downwards.
OK, so I'm going to try to draw a parabola that goes downward, but now to the front and to the back.
So, we are starting to have a slightly better idea but we still don't know whether the cross section of this thing might be round, square, something else.
So, it wants more confirmation.
We might want to also figure out where the surface intersects the x, y plane.
So, we hit the x, y plane when z equals zero.
That means 1-x^2-y^2 should be 0, that becomes x^2 y^2 = 1.
That is a circle of radius 1.
That's the unit size.
So, that means that here, we actually have the unit circle.
And now, you should imagine that you have this thing that when you slice it by a vertical plane, looks like a downwards parabola.
And, it's actually a surface of revolution.
You can rotate it around the z axis, OK?
Now, if you stare long enough at that equation, you'll actually see that, yes, we know that it had to be like that.
But, see, so these are useful ways of trying to guess what the graph looks like.
Of course, the other way is to just ask your computer to do it.
And then, you know, you will get that kind of formula.
OK, well, I can leave it on if you want.
No, because I plotted a different function that I will show you later.
So, it goes this way.
I mean, if you want, it's really going downward.
Yes, I agree that the sheet is upside down.
That's because I plotted something else.
OK, so, in fact, so I plotted in my computer was actually x^2 y^2 that looks like that.
See, it's the same with a parabola going upwards.
If you want to see more examples, I have various examples to show, well, here's the graph, y^2-x^2.
See, so that one is kind of interesting.
It looks like a saddle.
If you look at it in the y, z plane, then it's a parabola going up, z = y^2.
And, that's what we see to the left and to the right.
But, if you put it in the x, z plane, then that's a parabola going downwards, z = - x^2.
So, we have a parabola going downwards in one direction, upwards in the other one.
And together, they form this surface.
And of course, you can plot much more complicated functions.
So, this one, well, if you can read very small things, you can see the formula.
It doesn't matter, just to show you that you can put a formula into a computer: it will show you a picture.
OK, so that's pretty good.
I mean, you can see that it can get a bit cluttered because maybe those features that are hidden behind, or maybe we have trouble seeing if we don't have a computer, that looks very readable.
But, this is kind of hard to visualize sometimes.
So, there is another way to plot the functions of two variables.
And, let's call it the contour plot.
So, the contour plot is a very elegant solution to the problem that it's difficult to draw to space pictures on a sheet of paper or on a blackboard.
So, instead, let's try to represent the function of two variables by just the map, you know, the same way that when you walk around, you have actually geographical maps that fit on a piece of paper that tell you about what the real world looks like.
So, what contour plot looks something like this?
So, it's an x, y plot.
And, that, you have a bunch of curves.
And, what the curves represent are the elevations on the graph.
So, for example, this curve might correspond to all the points where f(x, y) = 1.
And, that curve might be all the points where f=2 and f=3 and so on, OK?
So, when you see you this kind of plot, you're supposed to think that the graph of the function sits somewhere in space above that.
It's like a map telling you how high things are.
And, what you would want to do with the function, really, is be able to tell quickly what's the value at a given point?
Well, let's say I want to look at that point.
I know that f is somewhere between 1 and 2.
Actually, it's much faster to read than the graph.
On the graph I might have to look carefully, and then measure things, and so on.
Here, I can just raise the value of f by comparing with the nearby lines.
OK, so let me try to make that more precise.
So, it shows all the points -- -- where f(x, y) equals some fixed values, some fixed constants.
And, these constants typically are chosen at regular intervals.
For example, here I chose one, two, three, and they could continue with zero minus one, and so on.
So, one way to think about it, how does this relate to the graph?
Well, that's the same thing as cutting, I mean, we slice the graph by horizontal planes.
So, horizontal planes have equations of a form z equals some constant, z equals zero, z equals one, z equals two, and so on.
So, maybe the graph of my function will be some sort of plot out there.
And, if I slice it by the plane z equals one, then I will get the level curve, which is the point where f(x, y) = 1, and so, that's called a level curve of f. OK, and so we repeat the process with maybe z equals two, and we get another level curve, and so on.
And, then we squish all of them up, and that's how we get the contour plot.
OK, so each of these lines, imagine that this is like some mountain or something that you are hiking on.
Each of these lines tells you how you could move to stay at a constant height if you want to get to the other side of the mountain but without ever going up or down.
You would just walk along that path, and it will get you there without effort.
So, in fact, if you have been talking about hiking on mountains, well, that's exactly what a topographical map is about.
So, I need to zoom a bit.
So, a topographic map, this one from the US geological survey shows you, basically, all the level curves of an altitude function on a piece of land.
So, you know that if you walk right along these curves, you will stay along the same height.
And you know that if you walk towards, these don't have numbers.
Let me find a place with numbers.
Here, there is a 500 in the middle.
So, you know that if you walk on the line that says 500, you stay always at 500 meters in elevation.
If you walk towards the mountain that I think is below it, then you will go up, and so on.
So, you can see, for example, here there's a peak, and here there is a valley with the river in it, and the altitudes go down, and then back up again on the other side.
OK, so that's an example of a contour plot of a function.
Of course, we don't have a formula for that function, but we have a contour plot, and that's what we need actually to understand what's going on there.
OK, any questions?
No?
OK, so another example of contour plots, well, you've probably seen various versions of these temperature maps.
So, that's supposed to be how warm it is right now.
So, this one is color-coded.
Instead of having curves, it has all these colors.
But, the effect is the same.
If you look at the separations between consecutive colors, these are the level curves of a function that tells you the temperature at a given point.
OK, so these are examples of contour plots in real life.
OK, no questions?
No?
OK, so basically, one of the goals that one should try to achieve at this point is becoming familiar with the contour plot, the graph, and how to view and deal with functions.
[APPLAUSE] OK, so -- Let's do an example.
Well, let's do a couple of examples.
So, let's start with f(x,y) = - y.
And, I'm going to take the same two examples as there to start with so that we see the relation between the graph and the contour plots.
So, let's try to plot it.
So, we are asked for the level curve, f equals 0 for this one?
Well, f is zero when y is zero.
So, that's the x axis.
OK, so that's the level, zero.
Where's the level one?
Well, f is one when negative y is one.
That means when y is negative one.
So, you go to minus one, and that will be where my level one is, and so on.
f is two when y is negative two.
F is negative one when y is one, and so on.
Is that convincing?
Do you see how we got that?
OK, let me do it again.
I don't see anybody nodding, so that's kind of bad news.
So, if I want to know, where is the level curve, say, one, I try to set f equals to one.
Let's do this one.
f equals one means that negative y is one means that y is minus one, and y equals minus one is this horizontal line on this chart.
OK, and same with the others.
So, you can see on the map that the value of a function doesn't depend on x.
If you move parallel to the x axis, nothing happens.
If you move in the y direction, it decreases at a constant rate.
That's why the contours are evenly spaced.
How spaced out they are tells you, actually, how steep things are.
So, that corresponds exactly to that picture, except that here we draw x coming to the front, and y to the right.
So, you have to rotate the map by 90� to get to that.
It's an unfortunate consequence of the usual way of plotting things in space.
OK, so these horizontal lines here correspond actually to horizontal lines here.
OK, second example.
Let's do 1-x^2-y^2.
OK, or maybe I will write it as 1 - (x^2 y^2).
It's really the same thing.
So, x, y, let's see, where is this function equal to zero?
Well, we said f is zero in the unit circle.
OK, so, the zero level, well, let's say that this is my unit.
That's where it's at zero.
What about f equals one?
Well, that's when x^2 y^2 = 0.
Well, that's only going to be here.
So, that's just a single point.
What about f equals minus one?
That's when x^2 y^2 =2.
That's a circle of radius square root of two, which is about 1.4.
So, it's somewhere here.
Then, minus two, similarly, will be x^2 y^2 = 3.
Square root of three is about 1.7.
And then, minus three will be of radius two, and so on.
So, what I want to show here is that they are getting closer and closer apart, OK?
So, first it's concentric circles that tells us that we have a shape that's a solid of the graph is going to be a surface of revolution.
Things don't change if I rotate.
And second, the level curves are getting closer and closer to each other.
That means it's getting steeper and steeper because I have to travel a shorter distance if I want my altitude to change by one.
OK, so, this top here is kind of flat.
And then it gets steeper and steeper.
And, that's what we observe on that picture there.
OK, so just to show you a few more, where did I put my, so, for x^2 y^2, the contour plot looks like this.
Maybe actually I'll make it.
OK, so it looks exactly the same as this one.
But, the difference is if you can see the numbers which are not there, so you can see them, then you would know that instead of decreasing as we move out, this one is increasing as we go out.
OK, so that's where we use, actually, the labels on the level curves that tell us whether things are going up or down.
But, the contour plots look exactly the same.
For the next one I had, I think, was y^2-x^2.
So, the contour plot, well, let me actually zoom out.
So, the contour plot looks like that.
So, the level curve corresponding to zero is actually two diagonal lines.
And, if you look on the plot, say that you started at the saddle point in the middle and you try to stay at the same level.
So, it looks like a mountain pass, right?
Well, there's actually four directions from that point that you can go staying at the same height.
And actually, on the map, they look exactly like this, too, these crossing lines.
OK, so, these are things that go on the side of the two mountains that are to the left and right, and stay at the same height as the mountain pass.
On the other hand, if you go along the y direction, to the left or to the right, then you go towards positive values.
And, if you go along the x axis, then you get towards negative values.
OK, the equation for, the function was y^2-x^2.
So, you can try to plot them by hand and confirmed that it does look like that.
But, I trust my computer.
And, finally, this one, well, so the contour plot looks a bit complicated.
But, you can see two things.
In the middle, you can see these two origins with these concentric circles which are not really circles, but, you know, these closed curves that are concentric.
And, they correspond to the two mountains.
And then, at some point in the middle, we have a mountain pass.
And there, we see the two crossing lines again, like, on the plot of y^2-x^2.
And so, at this saddle point here, if we go north or south, then we go down on either side to the Valley.
And, if we go east or west, then we go towards the mountains.
We'll go up.
OK, does that make sense a little bit?
OK, so, reading plots is not easy, but hopefully we'll get used to it very soon.
OK, so actually let's say, well, OK, so, I want to point out one thing.
The contour plot tells us, actually, what happens when we move, when we change x and y.
So, if I change the value of x and y, that means I'm moving east-west or north-south on the map.
And then, I can ask myself, is the value of the function increase or decrease in each of these situations?
Well, that's the kind of thing that the contour plot can tell me very quickly.
So -- OK, so say, for example, that I have a piece of contour plot.
That looks, you know, like that.
Maybe this is f equals one, and this is f equals two.
And here, this is f equals zero.
And, let's say that I start at the point, say, at this point.
OK, so here I have (x0, y0).
And, the question I might ask myself is if I change x or y, how does f change?
Well, the contour plot tells me that if x increases, and I keep y constant, then what happens to f(x, y)?
Well, it will increase because if I move to the right, then I go from one to a value bigger than one.
I don't know exactly how much, but I know that somewhere between one and two, it's more than one.
If x decreases, then f decreases.
And, similarly, I can tell that if y increases, then f, well, it looks like if I increase y, then f will also increase.
And, if y decreases, then f decreases.
And, that's the kind of qualitative analysis that we can do easily from the contour plot.
But, maybe we'd like to actually be more precise in that, and tell how fast f changes if I change x or y. OK, so to find the rate of change, that's exactly where we use derivatives.
So -- So, we are going to have to deal with partial derivatives.
So, I will explain to you soon why partial.
So, let me just remind you first, if you have a function of one variable, then so let's say f of x, then you have a derivative, f prime of x is also called df/dx.
And, it's defined as a limit when delta x goes to zero of the change in f. Sorry, it's not going to fit.
I have to go to the next line.
It's going to be the limit as delta x goes to zero of the rate of change.
So, the change in f between x and x plus delta x divided by delta x.
Sometimes you write delta f for the change in f divided by delta x.
And then, you take the limit of this rate of increase as delta x goes to zero.
Now, of course, if you have a formula for f, then you know, at least you should know, I suspect most of you know how to actually take the derivative of a function from its formula.
So -- Now, how do we do that?
Sorry, and I should also tell you what this means on the graph.
So, if I plot the graph of a function, and to have my point, x, and here I have f of x, how do I see the derivative?
Well, I look at the tangent line to the graph, and the slope of the tangent line is f prime of x, OK?
And, not every function has a derivative.
You have functions that are not regular enough to actually have a derivative.
So, in this class, we are not going to actually worry too much about differentiability.
But, it's good, at least, to be aware that you can't always take the derivative.
So, yes, and what else do I want to remind you of?
Well, they also have an approximation formula -- -- which says that, you know, if we have the value of f at some point, x0, and that we want to find the value at a nearby point, x close to x0, then our best guess is that it's f of x0 plus the derivative f prime at x0 times delta x, or if you want, x minus x0, OK?
Is this kind of familiar to you?
Yeah, I mean, maybe with different notations.
Maybe you called that delta x or something.
Maybe you called that x0 plus h or something.
But, it's the usual approximation formula using the derivative.
If you put more terms, then you get the dreaded Taylor approximation that I know you guys don't like.
So, the question is how do we do the same for a function of two variables, f(x, y)?
So, the difficulty there is we can change x, or we can change y, or we can change both.
And, presumably, the manner in which f changes will be different depending on whether we change x or y.
So, that's why we have several different notions of derivative.
So, OK, we have a notation.
OK, so this is a curly d, and it is not a straight d, and it is not a delta.
It's a d that kind of curves backwards like that.
And, this symbol is partial.
OK, so it's a special notation for partial derivatives.
And, essentially what it means is we are going to do a derivative where we care about only one variable at a time.
That's why it's partial.
It's missing the other variables.
So, a function of several variables doesn't have the usual derivative.
It has only partial derivatives for each variable.
So, the partial derivative, the partial f partial x at (x0, y0) is defined to be the limit when I take a small change in x, delta x, of the change in f -- -- divided by delta x. OK, so here I'm actually not changing y at all.
I'm just changing x and looking at the rate of change with respect to x.
And, I have the same with respect to y.
Partial f partial y is the limit, so I should say, at a point x0 y0 is the limit as delta y turns to zero.
So, this time I keep x the same, but I change y. OK, so that's the definition of a partial derivative.
And, we say that a function is differentiable if these things exist.
OK, so most of the functions we'll see are differentiable.
And, we'll actually learn how to compute their partial derivatives without having to do this because we'll just have the usual methods for computing derivatives.
So, in fact, I claim you already know how to take partial derivatives.
So, let's see what it means geometrically.
So, geometrically, my function can be represented by this graph, and I fix some point, (x0, y0).
And then, I'm going to ask myself what happens if I change the value of, well, x, keeping y constant.
So, if I keep y constant and change x, it means that I'm moving forwards or backwards parallel to the x axis.
So, that determines for me the vertical plane parallel to the x, z plane when I fix y equals constant.
And now, if I slice the graph by that, I will get some curve that goes, it's a slice of the graph of f. And now, what I want to find is how f depends on x if I keep y constant.
That means it's the rate of change if I move along this curve.
So, in fact, if I look at the slope of this thing.
So, if I draw the tangent line to this slice, then the slope will be partial f of partial x. I think I have a better picture of that somewhere.
Yes, here it is.
OK, that's the same picture, just with different colors.
So, I take the graph.
I slice it by a vertical plane.
I get the curve, and now I take the slope of that curve, and that gives me the partial derivative.
And, to finish, let me just tell you how, and I should say, partial f partial y is the same thing but slicing now by your plane that goes in the y, z directions.
OK, so I fix x equals constant.
That means that I slice by a plane that's parallel to the blackboard.
I get a curve, and I looked at the slope of that curve.
OK, so it's just a matter of formatting one variable, setting it constant, and looking at the other one.
So, how to compute these things, well, we actually, to find, well, there's a piece of notation I haven't told you yet.
So, another notation you will see, I think this is what one uses a lot in physics.
And, this is what one uses a lot in applied math, which is the same thing as physics but with different notations.
OK, so it just too different notations: partial f partial x, or f subscript x.
And, they are the same thing.
Well, we just treat y as a constant, and x as a variable.
And, vice versa if we want to find partial with aspect to y.
So, let me just finish with one quick example.
Let's say that they give you f of x, y equals x^3y y^2, then partial f partial x.
Well, let's take the derivative.
So, here it's x^3 times a constant.
Derivative of x^3 is 3x^2 times the constant plus what's the derivative of y^2?
Zero, because it's a constant.
If you do, instead, partial f partial y, then this is actually a constant times y.
The derivative of y is one.
So, that's just x^3.
And, the derivative of y^2 is 2y.
OK, so it's fairly easy.
You just have to keep remembering which one is a variable, and which one isn't.
OK, so more about this next time, and we will also learn about maxima and minima in several variables.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Today we are going to see how to use what we saw last time about partial derivatives to handle minimization or maximization problems involving functions of several variables.
Remember last time we said that when we have a function, say, of two variables, x and y, then we have actually two different derivatives, partial f, partial x, also called f sub x, the derivative with respect to x keeping y constant.
And we have partial f, partial y, also called f sub y, where we vary y and we keep x as a constant.
And now, one thing I didn't have time to tell you about but hopefully you thought about in recitation yesterday, is the approximation formula that tells you what happens if you vary both x and y. f sub x tells us what happens if we change x a little bit, by some small amount delta x. f sub y tells us how f changes, if you change y by a small amount delta y.
If we do both at the same time then the two effects will add up with each other, because you can imagine that first you will change x and then you will change y.
Or the other way around.
It doesn't really matter.
If we change x by a certain amount delta x, and if we change y by the amount delta y, and let's say that we have z= f(x, y) then that changes by an amount which is approximately f sub x times delta x plus f sub y times delta y.
And that is one of the most important formulas about partial derivatives.
The intuition for this, again, is just the two effects of if I change x by a small amount and then I change y.
Well, first changing x will modify f, how much does it modify f?
The answer is the rate change is f sub x.
And if I change y then the rate of change of f when I change y is f sub y.
So all together I get this change as a value of f. And, of course, that is only an approximation formula.
Actually, there would be higher order terms involving second and third derivatives and so on.
One way to justify this -- Sorry.
I was distracted by the microphone.
OK. How do we justify this formula?
Well, one way to think about it is in terms of tangent plane approximation.
Let's think about the tangent plane with regard to a function f. We have some pictures to show you.
It will be easier if I show you pictures.
Remember, partial f, partial x was obtained by looking at the situation where y is held constant.
That means I am slicing the graph of f by a plane that is parallel to the x, z plane.
And when I change x, z changes, and the slope of that is going to be the derivative with respect to x.
Now, if I do the same in the other direction then I will have similarly the slope in a slice now parallel to the y, z plane that will be partial f, partial y.
In fact, in each case, I have a line.
And that line is tangent to the surface.
Now, if I have two lines tangent to the surface, well, then together they determine for me the tangent plane to the surface.
Let's try to see how that works.
We know that f sub x and f sub y are the slopes of two tangent lines to this plane, two tangent lines to the graph.
And let's write down the equations of these lines.
I am not going to write parametric equations.
I am going to write them in terms of x, y, z coordinates.
Let's say that partial f of a partial x at the given point is equal to a.
That means that we have a line given by the following conditions.
I am going to keep y constant equal to y0.
And I am going to change x.
And, as I change x, z will change at the rate that is equal to a.
That would be z = 0 a(x - x0).
That is how you would describe a line that, I guess, the one that is plotted in green here, been dissected with the slice parallel to the x, z plane.
I hold y constant equal to y0.
And z is a function of x that varies with a rate of a.
And now if I look similarly at the other slice, let's say that the partial with respect to y is equal to b, then I get another line which is obtained by the fact that z now will depend on y.
And the rate of change with respect to y will be b.
While x is held constant equal to x0.
These two lines are both going to be in the tangent plane to the surface.
They are both tangent to the graph of f and together they determine the plane.
And that plane is just given by the formula z = z0 a( x - x0) b ( y - y0).
If you look at what happens -- This is the equation of a plane.
z equals constant times x plus constant times y plus constant.
And if you look at what happens if I hold y constant and vary x, I will get the first line.
If I hold x constant and vary y, I get the second line.
Another way to do it, of course, would provide actually parametric equations of these lines, get vectors along them and then take the cross-product to get the normal vector to the plane.
And then get this equation for the plane using the normal vector.
That also works and it gives you the same formula.
If you are curious of the exercise, do it again using parametrics and using cross-product to get the plane equation.
That is how we get the tangent plane.
And now what this approximation formula here says is that, in fact, the graph of a function is close to the tangent plane.
If we were moving on the tangent plane, this would be an actual equality.
Delta z would be a linear function of delta x and delta y.
And the graph of a function is near the tangent plane, but is not quite the same, so it is only an approximation for small delta x and small delta y.
The approximation formula says the graph of f is close to its tangent plane.
And we can use that formula over here now to estimate how the value of f changes if I change x and y at the same time.
Questions about that?
Now that we have caught up with what we were supposed to see on Tuesday, I can tell you now about max and min problems.
That is going to be an application of partial derivatives to look at optimization problems.
Maybe ten years from now, when you have a real job, your job might be to actually minimize the cost of something or maximize the profit of something or whatever.
But typically the function that you will have to strive to minimize or maximize will depend on several variables.
If you have a function of one variable, you know that to find its minimum or its maximum you look at the derivative and set that equal to zero.
And you try to then look at what happens to the function.
Here it is going to be kind of similar, except, of course, we have several derivatives.
For today we will think about a function of two variables, but it works exactly the same if you have three variables, ten variables, a million variables.
The first observation is that if we have a local minimum or a local maximum then both partial derivatives, so partial f partial x and partial f partial y, are both zero at the same time.
Why is that?
Well, let's say that f of x is zero.
That means when I vary x to first order the function doesn't change.
Maybe that is because it is going through...
If I look only at the slice parallel to the x-axis then maybe I am going through the minimum.
But if partial f, partial y is not 0 then actually, by changing y, I could still make a value larger or smaller.
That wouldn't be an actual maximum or minimum.
It would only be a maximum or minimum if I stay in the slice.
But if I allow myself to change y that doesn't work.
I need actually to know that if I change y the value will not change either to first order.
That is why you also need partial f, partial y to be zero.
Now, let's say that they are both zero.
Well, why is that enough?
It is essentially enough because of this formula telling me that if both of these guys are zero then to first order the function doesn't change.
Then, of course, there will be maybe quadratic terms that will actually turn that, you know, this won't really say that your function is actually constant.
It will just tell you that maybe it will actually be quadratic or higher order in delta x and delta y.
That is what you expect to have at a maximum or a minimum.
The condition is the same thing as saying that the tangent plane to the graph is actually going to be horizontal.
And that is what you want to have.
Say you have a minimum, well, the tangent plane at this point, at the bottom of the graph is going to be horizontal.
And you can see that on this equation of a tangent plane, when both these coefficients are 0 that is when the equation becomes z equals constant: the horizontal plane.
Does that make sense?
We will have a name for this kind of point because, actually, what we will see very soon is that these conditions are necessary but are not sufficient.
There are actually other kinds of points where the partial derivatives are zero.
Let's give a name to this.
We say the definition is (x0, y0) is a critical point of f -- -- if the partial derivative, with respect to x, and partial derivative with respect to y are both zero.
Generally, you would want all the partial derivatives, no matter how many variables you have, to be zero at the same time.
Let's see an example.
Let's say I give you the function f(x;y)= x^2 - 2xy 3y^2 2x - 2y.
And let's try to figure out whether we can minimize or maximize this.
What we would start doing immediately is taking the partial derivatives.
What is f sub x?
It starts with 2x - 2y 0 2.
Remember that y is a constant so this differentiates to zero.
Now, if we do f sub y, that is going to be 0-2x 6y-2.
And what we want to do is set these things to zero.
And we want to solve these two equations at the same time.
An important thing to remember, and maybe I should have told you a couple of weeks ago already, if you have two equations to solve, well, it is very good to try to simplify them by adding them together or whatever, but you must keep two equations.
If you have two equations, you shouldn't end up with just one equation out of nowhere.
For example here, we can certainly simplify things by summing them together.
If we add them together, well, the x's cancel and the constants cancel.
In fact, we are just left with 4y for zero.
That is pretty good.
That tells us y should be zero.
But then we should, of course, go back to these and see what else we know.
Well, now it tells us, if you put y = 0 it tells you 2x 2 = 0.
That tells you x = - 1.
We have one critical point that is (x, y) = (- 1; 0).
Any questions so far?
No.
Well, you should have a question.
The question should be how do we know if it is a maximum or a minimum?
Yeah.
If we had a function of one variable, we would decide things based on the second derivative.
And, in fact, we will see tomorrow how to do things based on the second derivative.
But that is kind of tricky because there are a lot of second derivatives.
I mean we already have two first derivatives.
You can imagine that if you keep taking partials you may end up with more and more, so we will have to figure out carefully what the condition should be.
We will do that tomorrow.
For now, let's just try to look a bit at how do we understand these things by hand?
In fact, let me point out to you immediately that there is more than maxima and minima.
Remember, we saw the example of x^2 y^2.
That has a critical point.
That critical point is obviously a minimum.
And, of course, it could be a local minimum because it could be that if you have a more complicated function there is indeed a minimum here, but then elsewhere the function drops to a lower value.
We call that just a local minimum to say that it is a minimum if you stick two values that are close enough to that point.
Of course, you also have local maximum, which I didn't plot, but it is easy to plot.
That is a local maximum.
But there is a third example of critical point, and that is a saddle point.
The saddle point, it is a new phenomena that you don't really see in single variable calculus.
It is a critical point that is neither a minimum nor a maximum because, depending on which direction you look in, it's either one or the other.
See the point in the middle, at the origin, is a saddle point.
If you look at the tangent plane to this graph, you will see that it is actually horizontal at the origin.
You have this mountain pass where the ground is horizontal.
But, depending on which direction you go, you go up or down.
So, we say that a point is a saddle point if it is neither a minimum or a maximum.
Possibilities could be a local min, a local max or a saddle.
Tomorrow we will see how to decide which one it is, in general, using second derivatives.
For this time, let's just try to do it by hand.
I just want to observe, in fact, I can try to, you know, these examples that I have here, they are x^2 y^2, y^2 - x^2, they are sums or differences of squares.
And, if we know that we can put things as sum of squares for example, we will be done.
Let's try to express this maybe as the square of something.
The main problem is this 2xy.
Observe we know something that starts with x^2 - 2xy but is actually a square of something else.
It would be x^2 - 2xy y^2, not plus 3y2.
Let's try that.
So, we are going to complete the square.
I am going to say it is x minus y squared, so it gives me the first two terms and also the y2.
Well, I still need to add two more y^2, and I also need to add, of course, the 2x and - 2y.
It is still not simple enough for my taste.
I can actually do better.
These guys look like a sum of squares, but here I have this extra stuff, 2x - 2y.
Well, that is 2 (x - y).
It looks like maybe we can modify this and make this into another square.
So, in fact, I can simplify this further to (x - y 1)^2.
That would be (x - y)^2 2( x - y), and then there is a plus one.
Well, we don't have a plus one so let's remove it by subtracting one.
And I still have my 2y^2.
Do you see why this is the same function?
Yeah.
Again, if I expand x minus y plus one squared, I get (x - y)^2 2 (x - y) 1.
But I will have minus one that will cancel out and then I have a plus 2y^2.
Now, what I know is a sum of two squared minus one.
And this critical point, (x,y) = (-1;0), that is actually when this is zero and that is zero, so that is the smallest value.
This is always greater or equal to zero, the same with that one, so that is always at least minus one.
And minus one happens to be the value at the critical point.
So, it is a minimum.
Now, of course here I was very lucky.
I mean, generally, I couldn't expect things to simplify that much.
In fact, I cheated.
I started from that, I expanded, and then that is how I got my example.
The general method will be a bit different, but you will see it will actually also involve completing squares.
Just there is more to it than what we have seen.
We will come back to this tomorrow.
Sorry?
How do I know that this equals -- How do I know that the whole function is greater or equal to negative one?
Well, I wrote f of x, y as something squared plus 2y^2 - 1.
This squared is always a positive number and not a negative.
It is a square.
The square of something is always non-negative.
Similarly, y^2 is also always non-negative.
So if you add something that is at least zero plus something that is at least zero and you subtract one, you get always at least minus one.
And, in fact, the only way you can get minus one is if both of these guys are zero at the same time.
That is how I get my minimum.
More about this tomorrow.
In fact, what I would like to tell you about now instead is a nice application of min, max problems that maybe you don't think of as a min, max problem that you will see.
I mean you will think of it that way because probably your calculator can do it for you or, if not, your computer can do it for you.
But it is actually something where the theory is based on minimization in two variables.
Very often in experimental sciences you have to do something called least-squares intercalation.
And what is that about?
Well, it is the idea that maybe you do some experiments and you record some data.
You have some data x and some data y.
And, I don't know, maybe, for example, x is -- Maybe your measuring frogs and you're trying to measure how bit the frog leg is compared to the eyes of the frog, or you're trying to measure something.
And if you are doing chemistry then it could be how much you put of some reactant and how much of the output product that you wanted to synthesize generated.
All sorts of things.
Make up your own example.
You measure basically, for various values of x, what the value of y ends up being.
And then you like to claim these points are kind of aligned.
And, of course, to a mathematician they are not aligned.
But, to an experimental scientist, that is evidence that there is a relation between the two.
And so you want to claim -- And in your paper you will actually draw a nice little line like that.
The functions depend linearly on each of them.
The question is how do we come up with that nice line that passes smack in the middle of the points?
The question is, given experimental data xi, yi -- Maybe I should actually be more precise.
You are given some experimental data.
You have data points x1, y1, x2, y2 and so on, xn, yn, the question would be find the "best fit" line of a form y equals ax b that somehow approximates very well this data.
You can also use that right away to predict various things.
For example, if you look at your new homework, actually the first problem asks you to predict how many iPods will be on this planet in ten years looking at past sales and how they behave.
One thing, right away, before you lose all the money that you don't have yet, you cannot use that to predict the stock market.
So, don't try to use that to make money.
It doesn't work.
One tricky thing here that I want to draw your attention to is what are the unknowns here?
The natural answer would be to say that the unknowns are x and y.
That is not actually the case.
We are not going to solve for some x and y. I mean we have some values given to us.
And, when we are looking for that line, we don't really care about the perfect value of x.
What we care about is actually these coefficients a and b that will tell us what the relation is between x and y.
In fact, we are trying to solve for a and b that will give us the nicest possible line for these points.
The unknowns, in our equations, will have to be a and b, not x and y.
The question really is find the "best" a and b.
And, of course, we have to decide what we mean by best.
Best will mean that we minimize some function of a and b that measures the total errors that we are making when we are choosing this line compared to the experimental data.
Maybe, roughly speaking, it should measure how far these points are from the line.
But now there are various ways to do it.
And a lot of them are valid they give you different answers.
You have to decide what it is that you prefer.
For example, you could measure the distance to the line by projecting perpendicularly.
Or you could measure instead, for a given value of x, the difference between the experimental value of y and the predicted one.
And that is often more relevant because these guys actually may be expressed in different units.
They are not the same type of quantity.
You cannot actually combine them arbitrarily.
Anyway, the convention is usually we measure distance in this way.
Next, you could try to minimize the largest distance.
Say we look at who has the largest error and we make that the smallest possible.
The drawback of doing that is experimentally very often you have one data point that is not good because maybe you fell asleep in front of the experiment.
And so you didn't measure the right thing.
You tend to want to not give too much importance to some data point that is far away from the others.
Maybe instead you want to measure the average distance or maybe you want to actually give more weight to things that are further away.
And then you don't want to do the distance with a square of the distance.
There are various possible answers, but one of them gives us actually a particularly nice formula for a and b.
And so that is why it is the universally used one.
Here it says list squares.
That's because we will measure, actually, the sum of the squares of the errors.
And why do we do that?
Well, part of it is because it looks good.
When you see this plot in scientific papers they really look like the line is indeed the ideal line.
And the second reason is because actually the minimization problem that we will get is particularly simple, well-posed and easy to solve.
So we will have a nice formula for the best a and the best b.
If you have a method that is simple and gives you a good answer then that is probably good.
We have to define best.
Here it is in the sense of minimizing the total square error.
Or maybe I should say total square deviation instead.
What do I mean by this?
The deviation for each data point is the difference between what you have measured and what you are predicting by your model.
That is the difference between y1 and axi plus b.
Now, what we will do is try to minimize the function capital D, which is just the sum for all the data points of the square of a deviation.
Let me go over this again.
This is a function of a and b.
Of course there are a lot of letters in here, but xi and yi in real life there will be numbers given to you.
There will be numbers that you have measured.
You have measured all of this data.
They are just going to be numbers.
You put them in there and you get a function of a and b.
Any questions?
How do we minimize this function of a and b?
Well, let's use your knowledge.
Let's actually look for a critical point.
We want to solve for partial d over partial a= 0, partial d over partial b = 0.
That is how we look for critical points.
Let's take the derivative of this with respect to a.
Well, the derivative of a sum is sum of the derivatives.
And now we have to take the derivative of this quantity squared.
Remember, we take the derivative of the square.
We take twice this quantity times the derivative of what we are squaring.
We will get 2(yi - axi) b times the derivative of this with respect to a.
What is the derivative of this with respect to a?
Negative xi, exactly.
And so we will want this to be 0.
And partial d over partial b, we do the same thing, but different shading with respect to b instead of with respect to a.
Again, the sum of squares twice yi minus axi equals b times the derivative of this with respect to b is, I think, negative one.
Those are the equations we have to solve.
Well, let's reorganize this a little bit.
The first equation.
See, there are a's and there are b's in these equations.
I am going to just look at the coefficients of a and b.
If you have good eyes, you can see probably that these are actually linear equations in a and b.
There is a lot of clutter with all these x's and y's all over the place.
Let's actually try to expand things and make that more apparent.
The first thing I will do is actually get rid of these factors of two.
They are just not very important.
I can simplify things.
Next, I am going to look at the coefficient of a. I will get basically a times xi squared.
Let me just do it and should be clear.
I claim when we simplify this we get xi squared times a plus xi times b minus xiyi.
And we set this equal to zero.
Do you agree that this is what we get when we expand that product?
Yeah.
Kind of?
OK. Let's do the other one.
We just multiply by minus one, so we take the opposite of that which would be axi plus b. I will write that as xia plus b minus yi.
Sorry.
I forgot the n here.
And let me just reorganize that by actually putting all the a's together.
That means I will have sum of all the xi2 times a plus sum of xib minus sum of xiyi equal to zero.
If I rewrite this, it becomes sum of xi2 times a plus sum of the xi's time b, and let me move the other guys to the other side, equals sum of xiyi.
And that one becomes sum of xi times a.
Plus how many b's do I get on this one?
I get one for each data point.
When I sum them together, I will get n. Very good.
N times b equals sum of yi.
Now, this quantities look scary, but they are actually just numbers.
For example, this one, you look at all your data points.
For each of them you take the value of x and you just sum all these numbers together.
What you get, actually, is a linear system in a and b, a two by two linear system.
And so now we can solve this for a and b.
In practice, of course, first you plug in the numbers for xi and yi and then you solve the system that you get.
And we know how to solve two by two linear systems, I hope.
That's how we find the best fit line.
Now, why is that going to be the best one instead of the worst one?
We just solved for a critical point.
That could actually be a maximum of this error function D. We will have the answer to that next time, but trust me.
If you really want to go over the second derivative test that we will see tomorrow and apply it in this case, it is quite hard to check, but you can see it is actually a minimum.
I will just say -- -- we can show that it is a minimum.
Now, the event with the linear case is the one that we are the most familiar with.
Least-squares interpolation actually works in much more general settings.
Because instead of fitting for the best line, if you think it has a different kind of relation then maybe you can fit in using a different kind of formula.
Let me actually illustrate that with an example.
I don't know if you are familiar with Moore's law.
It is something that is supposed to tell you how quickly basically computer chips become smarter faster and faster all the time.
It's a law that says things about the number of transistors that you can fit onto a computer chip.
Here I have some data about -- Here is data about the number of transistors on a standard PC processor as a function of time.
And if you try to do a best-line fit, well, it doesn't seem to follow a linear trend.
On the other hand, if you plug the diagram in the log scale, the log of a number of transitions as a function of time, then you get a much better line.
And so, in fact, that means that you had an exponential relation between the number of transistors and time.
And so, actually that's what Moore's law says.
It says that the number of transistors in the chip doubles every 18 months or every two years.
They keep changing the statement.
How do we find the best exponential fit?
Well, an exponential fit would be something of a form y equals a constant times exponential of a times x.
That is what we want to look at.
Well, we could try to minimize a square error like we did before.
That doesn't work well at all.
The equations that you get are very complicated.
You cannot solve them.
But remember what I showed you on this log plot.
If you plot the log of y as a function of x then suddenly it becomes a linear relation.
Observe, this is the same as ln of y equals ln of c plus ax.
And that is the linear best fit.
What you do is you just look for the best straight line fit for the log of y.
That is something we already know.
But you can also do, for example, let's say that we have something more complicated.
Let's say that we have actually a quadratic law.
For example, y is of the form ax^2 bx c. And, of course, you are trying to find somehow the best.
That would mean here fitting the best parabola for your data points.
Well, to do that, you would need to find a, b and c. And now you will have actually a function of a, b and c, which would be the sum of the old data points of the square deviation.
And, if you try to solve for critical points, now you will have three equations involving a, b and c, in fact, you will find a three by three linear system.
And it works the same way.
Just you have a little bit more data.
Basically, you see that this best fit problems are an example of a minimization problem that maybe you didn't expect to see minimization problems come in.
But that is really the way to handle these questions.
Tomorrow we will go back to the question of how do we decide whether it is a minimum or a maximum.
And we will continue exploring in terms of several variables.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, today we are going to continue looking at critical points, and we'll learn how to actually decide whether a typical point is a minimum, maximum, or a saddle point.
So, that's the main topic for today.
So, remember yesterday, we looked at critical points of functions of several variables.
And, so a critical point functions, we have two values, x and y.
That's a point where the partial derivatives are both zero.
And, we've seen that there's various kinds of critical points.
There's local minima.
So, maybe I should show the function on this contour plot,there is local maxima, which are like that.
And, there's saddle points which are neither minima nor maxima.
And, of course, if you have a real function, then it would be more complicated.
It will have several critical points.
So, this example here, well, you see on the plot that there is two maxima.
And, there is in the middle, between them, a saddle point.
And, actually, you can see them on the contour plot.
On the contour plot, you see the maxima because the level curves become circles that now down and shrink to the maximum.
And, you can see the saddle point because here you have this level curve that makes a figure eight.
It crosses itself.
And, if you move up or down here, so along the y direction, the values of the function will decrease.
Along the x direction, the values will increase.
So, you can see usually quite easily where are the critical points just by looking either at the graph or at the contour plots.
So, the only thing with the contour plots is you need to read the values to tell a minimum from a maximum because the contour plots look the same.
Just, of course, in one case, the values increase, and in another one they decrease.
So, the question -- -- is, how do we decide -- -- between the various possibilities?
So, local minimum, local maximum, or saddle point.
So, and, in fact, why do we care?
Well, the other question is how do we find the global minimum/maximum of a function?
So, here what I should point out, well, first of all, to decide where the function is the largest, in general you'll have actually to compare the values.
For example, here, if you want to know, what is the maximum of this function?
Well, we have two obvious candidates.
We have this local maximum and that local maximum.
And, the question is, which one is the higher of the two?
Well, in this case, actually, there is actually a tie for maximum.
But, in general, you would have to compute the function at both points, and compare the values if you know that it's three at one of them and four at the other.
Well, four wins.
The other thing that you see here is if you are looking for the minimum of this function, well, the minimum is not going to be at any of the critical points.
So, where's the minimum?
Well, it looks like the minimum is actually out there on the boundary or at infinity.
So, that's another feature.
The global minimum or maximum doesn't have to be at a critical point.
It could also be, somehow, on the side in some limiting situation where one variable stops being in the allowed rang of values or goes to infinity.
So, we have to actually check the boundary and the infinity behavior of our function to know where, actually, the minimum and maximum will be.
So, in general, I should point out, these should occur either at the critical point or on the boundary or at infinity.
So, by that, I mean on the boundary of a domain of definition that we are considering.
And so, we have to try both.
OK, but so we'll get back to that.
For now, let's try to focus on the question of, you know, what's the type of the critical point?
So, we'll use something that's known as the second derivative test.
And, in principle, well, the idea is kind of similar to what you do with the function of one variable, namely, the function of one variable.
If the derivative is zero, then you know that you should look at the second derivative.
And, that will tell you whether it's curving up or down whether you have a local max and the local min.
And, the main problem here is, of course, we have more possible situations, and we have several derivatives.
So, we have to think a bit harder about how we'll decide.
But, it will again involve the second derivative.
OK, so let's start with just an easy example that will be useful to us because actually it will provide the basis for the general method.
OK, so we are first going to consider a case where we have a function that's actually just quadratic.
So, let's say I have a function, W of (x,y) that's of the form ax^2 bxy cy^2.
OK, so this guy has a critical point at the origin because if you take the derivative with respect to x, well, and if you plug x equals y equals zero, you'll get zero, and same with respect to y.
You can also see, if you try to do a linear approximation of this, well, all these guys are much smaller than x and y when x and y are small.
So, the linear approximation, the tangent plane to the graph is really just w=0.
OK, so, how do we do it?
Well, yesterday we actually did an example.
It was a bit more complicated than that, but let me do it, so remember, we were looking at something that started with x^2 2xy 3y^2.
And, there were other terms.
But, let's forget them now.
And, what we did is we said, well, we can rewrite this as (x y)^2 2y^2.
And now, this is a sum of two squares.
So, each of these guys has to be nonnegative.
And so, the origin will be a minimum.
Well, it turns out we can do something similar in general no matter what the values of a, b, and c are.
We'll just try to first complete things to a square.
OK, so let's do that.
So, in general, well, let me be slightly less general, and let me assume that a is not zero because otherwise I can't do what I'm going to do.
So, I'm going to write this as a times x^2 plus b over axy.
And then I have my cy^2.
And now this looks like the beginning of the square of something, OK, just like what we did over there.
So, what is it the square of?
Well, you'd start with x plus I claim if I put b over 2a times y and I square it, then see the cross term two times x times b over 2a y will become b over axy.
Of course, now I also get some y squares out of this.
How many y squares do I get?
Well, I get b^2 over 4a^2 times a.
So, I get b2 over 4a y^2.
So, and I want, in fact, c times y^2.
So, the number of y^2 that I should add is c minus b^2 over 4a.
OK, let's see that again.
If I expand this thing, I will get ax^2 plus a times b over 2a times 2xy.
That's going to be my bxy.
But, I also get b^2 over 4a^2 y^2 times a.
That's b^2 over 4ay^2.
And, that cancels out with this guy here.
And then, I will be left with cy^2.
OK, do you see it kind of?
OK, if not, well, try expanding this square again.
OK, maybe I'll do it just to convince you.
But, so if I expand this, I will get A times, let me put that in a different color because you shouldn't write that down.
It's just to convince you again.
So, if you don't see it yet, let's expend this thing.
We'll get a times x^2 plus a times 2xb over 2ay.
Well, the two A's cancel out.
We get bxy plus a times the square of that's going to be b^2 over 4a^2 y^2 plus cy^2 minus b^2 over 4ay^2.
Here, the a and the a simplifies, and now these two terms simplify and give me just cy^2 in the end.
OK, and that's kind of unreadable after I've canceled everything, but if you follow it, you see that basically I've just rewritten my initial function.
OK, is that kind of OK?
I mean, otherwise there's just no substitute.
You'll have to do it yourself, I'm afraid.
OK, so, let me continue to play with this.
So, I'm just going to put this in a slightly different form just to clear the denominators.
OK, so, I will instead write this as one over 4a times the big thing.
So, I'm going to just put 4a^2 times x plus b over 2ay squared.
OK, so far I have the same thing as here.
I just introduced the 4a that cancels out, plus for the other one, I'm just clearing the denominator.
I end up with (4ac-b^2)y^2.
OK, so that's a lot of terms.
But, what does it look like?
Well, it looks like, so we have some constant factors, and here we have a square, and here we have a square.
So, basically, we've written this as a sum of two squares, well, a sum or a difference of two squares.
And, maybe that's what we need to figure out to know what kind of point it is because, see, if you take a sum of two squares, that you will know that each square takes nonnegative values.
And you will have, the function will always take nonnegative values.
So, the origin will be a minimum.
Well, if you have a difference of two squares that typically you'll have a saddle point because depending on whether one or the other is larger, you will have a positive or a negative quantity.
OK, so I claim there's various cases to look at.
So, let's see.
So, in fact, I claim there will be three cases.
And, that's good news for us because after all, we want to distinguish between three possibilities.
So, let's first do away with the most complicated one.
What if 4ac minus b^2 is negative?
Well, if it's negative, then it means what I have between the brackets is, so the first guy is obviously a positive quantity, while the second one will be something negative times y2.
So, it will be a negative quantity.
OK, so one term is positive.
The other is negative.
That tells us we actually have a saddle point.
We have, in fact, written our function as a difference of two squares.
OK, is that convincing?
So, if you want, what I could do is actually I could change my coordinates, have new coordinates called u equals x b over 2ay, and v, actually, well, I could keep y, and that it would look like the difference of squares directly.
OK, so that's the first case.
The second case is where 4ac-b^2 = 0.
Well, what happens if that's zero?
Then it means that this term over there goes away.
So, what we have is just one square.
OK, so what that means is actually that our function depends only on one direction of things.
In the other direction, it's going to actually be degenerate.
So, for example, forget all the clutter in there.
Say I give you just the function of two variables, w equals just x^2.
So, that means it doesn't depend on y at all.
And, if I try to plot the graph, it will look like, well, x is here.
So, it will depend on x in that way, but it doesn't depend on y at all.
So, what the graph looks like is something like that.
OK, basically it's a valley whose bottom is completely flat.
So, that means, actually, we have a degenerate critical point.
It's called degenerate because there is a direction in which nothing happens.
And, in fact, you have critical points everywhere along the y axis.
Now, whether the square that we have is x or something else, namely, x plus b over 2a y, it doesn't matter.
I mean, it will still get this degenerate behavior.
But there's a direction in which nothing happens because we just have the square of one quantity.
I'm sure that 300 students means 300 different ring tones, but I'm not eager to hear all of them, thanks.
[LAUGHTER] OK, so, this is what's called a degenerate critical point, and [LAUGHTER].
OK, so basically we'll leave it here.
We won't actually try to figure out further what happens, and the reason for that is that when you have an actual function, a general function, not just one that's quadratic like this, then there will actually be other terms maybe involving higher powers, maybe x^3 or y^3 or things like that.
And then, they will mess up what happens in this valley.
And, it's a situation where we won't be able, actually, to tell automatically just by looking at second derivatives what happens.
See, for example, in a function of one variable, if you have just a function of one variable, say, f of x equals x to the five, well, if you try to decide what type of point the origin is, you're going to take the second derivative.
It will be zero, and then you can conclude.
Those things depend on higher order derivatives.
So, we just won't like that case.
We just won't try to figure out what's going on here.
Now, the last situation is if 4ac-b^2 is positive.
So, then, that means that actually we've written things.
The big bracket up there is a sum of two squares.
So, that means that we've written w as one over 4a times plus something squared plus something else squared, OK?
So, these guys have the same sign, and that means that this term here will always be greater than or equal to zero.
And that means that we should either have a maximum or minimum.
How we find out which one it is?
Well, we look at the sign of a, exactly.
OK?
So, there's two sub-cases.
One is if a is positive, then, this quantity overall will always be nonnegative.
And that means we have a minimum, OK?
And, if a is negative on the other hand, so that means that we multiply this positive quantity by a negative number, we get something that's always negative.
So, zero is actually the maximum.
OK, is that clear for everyone?
Yes?
Sorry, yeah, so I said in the example w equals x^2, it doesn't depend on y.
So, the more general situation is w equals some constant.
Well, I guess it's a times (x b over 2a times y)^2.
So, it does depend on x and y, but it only depends on this combination.
OK, so if I choose to move in some other perpendicular direction, in the direction where this remains constant, so maybe if I set x equals minus b over 2a y, then this remains zero all the time.
So, there's a degenerate direction in which I stay at the minimum or maximum, or whatever it is that I have.
OK, so that's why it's called degenerate.
There is a direction in which nothing happens.
OK, yes?
Yes, yeah, so that's a very good question.
So, there's going to be the second derivative test.
Why do not have derivatives yet?
Well, that's because I've been looking at this special example where we have a function like this.
And, so I don't actually need to take derivatives yet.
But, secretly, that's because a, b, and c will be the second derivatives of the function, actually, 2a, b, and 2c.
So now, we are going to go to general function.
And there, instead of having these coefficients a, b, and c given to us, we'll have to compute them as second derivatives.
OK, so here, I'm basically setting the stage for what will be the actual criterion we'll use using second derivatives.
Yes?
So, yeah, so what you have a degenerate critical point, it could be a degenerate minimum, or a degenerate maximum depending on the sign of a.
But, in general, once you start having functions, you don't really know what will happen anymore.
It could also be a degenerate saddle, and so on.
So, we won't really be able to tell.
Yes?
It is possible to have a degenerate saddle point.
For example, if I gave you x^3 y^3, you can convince yourself that if you take x and y to be negative, it will be negative.
If x and y are positive, it's positive.
And, it has a very degenerate critical point at the origin.
So, that's a degenerate saddle point.
We don't see it here because that doesn't happen if you have only quadratic terms like that.
You need to have higher-order terms to see it happen.
OK. OK, so let's continue.
Before we continue, but see, I wanted to point out one small thing.
So, here, we have the magic quantity, 4ac minus b^2.
You've probably seen that before in your life.
Yet, it looks like the quadratic formula, except that one involves b^2-4ac.
But that's really the same thing.
OK, so let's see, where does the quadratic formula come in here?
Well, let me write things differently.
OK, so we've manipulated things, and got into a conclusion.
But, let me just do a different manipulation, and write this now instead as y^2 times a times x over y squared plus b(x over y) plus c. OK, see, that's the same thing that I had before.
Well, so now this quantity here is always nonnegative.
What about this one?
Well, of course, this one depends on x over y.
It means it depends on which direction you're going to move away from the origin, which ratio between x and y you will consider.
But, I claim there's two situations.
One is, so, let's try to reformulate things.
So, if a discriminate here is positive, then it means that these have roots and these have solutions.
And, that means that this quantity can be both positive and negative.
This quantity takes positive and negative values.
One way to convince yourself is just to, you know, plot at^2 bt c. You know that there's two roots.
So, it might look like this, or might look like that depending on the sign of a.
But, in either case, it will take values of both signs.
So, that means that your function will take values of both signs.
The value takes both positive and negative values.
And, so that means we have a saddle point, while the other situation, when b^2-4ac is negative -- -- means that this equation is quadratic never takes the value, zero.
So, it's always positive or it's always negative, depending on the sign of a.
So, the other case is if b^2-4ac is negative, then the quadratic doesn't have a solution.
And it could look like this or like that depending on whether a is positive or a is negative.
So, in particular, that means that ax over y2 plus bx over y plus c is always positive or always negative depending on the sign of a.
And then, that tells us that our function, w, will be always positive or always negative.
And then we'll get a minimum or maximum.
OK, we'll have a min or a max depending on which situation we are in.
OK, so that's another way to derive the same answer.
And now, you see here why the discriminate plays a role.
That's because it exactly tells you whether this quadratic quantity has always the same sign, or whether it can actually cross the value, zero, when you have the root of a quadratic.
OK, so hopefully at this stage you are happy with one of the two explanations, at least.
And now, you are willing to believe, I hope, that we have basically a way of deciding what type of critical point we have in the special case of a quadratic function.
OK, so, now what do we do with the general function?
Well, so in general, we want to look at second derivatives.
OK, so now we are getting to the real stuff.
So, how many second derivatives do we have?
That's maybe the first thing we should figure out.
Well, we can take the derivative first with respect to x, and then again with respect to x. OK, that gives us something we denote by partial square f over partial x squared or fxx.
Then, there's another one which is fxy, which means you take the derivative with respect to x, and then with respect to y.
Another thing you can do, is do first derivative respect to y, and then with respect to x.
That would be fyx.
Well, good news.
These are actually always equal to each other.
OK, so it's the fact that we will admit, it's actually not very hard to check.
So these are always the same.
We don't need to worry about which one we do.
That's one computation that we won't need to do.
We can save a bit of effort.
And then, we have the last one, namely, the second partial with respect to y and y fyy.
OK, so we have three of them.
So, what does the second derivative test say?
It says, say that you have a critical point (x0, y0) of a function of two variables, f, and then let's compute the partial derivatives.
So, let's call capital A the second derivative with respect to x.
Let's call capital B the second derivative with respect to x and y.
And C equals fyy at this point, OK?
So, these are just numbers because we first compute the second derivative, and then we plug in the values of x and y at the critical point.
So, these will just be numbers.
And now, what we do is we look at the quantity AC-B^2.
I am not forgetting the four.
You will see why there isn't one.
So, if AC-B^2 is positive, then there's two sub-cases.
If A is positive, then it's local minimum.
The second case, so, still, if AC-B^2 is positive, but A is negative, then it's going to be a local maximum.
And, if AC-B^2 is negative, then it's a saddle point, and finally, if AC-B^2 is zero, then we actually cannot compute.
We don't know whether it's going to be a minimum, a maximum, or a saddle.
We know it's degenerate in some way, but we don't know what type of point it is.
OK, so that's actually what you need to remember.
If you are formula oriented, that's all you need to remember about today.
But, let's try to understand why, how this comes out of what we had there.
OK, so, I think maybe I actually want to keep, so maybe I want to keep this middle board because it actually has, you know, the recipe that we found before the quadratic function.
So, let me move directly over there and try to relate our old recipe with the new.
OK, you are easily amused.
OK, so first, let's check that these two things say the same thing in the special case that we are looking at.
OK, so let's verify in the special case where the function was ax^2 bxy cy^2.
So -- Well, what is the second derivative with respect to x and x?
If I take the second derivative with respect to x and x, so first I want to take maybe the derivative with respect to x.
But first, let's take the first partial, Wx.
That will be 2ax by, right?
So, Wxx will be, well, let's take a partial with respect to x again.
That's 2a.
Wxy, I take the partial respect to y, and we'll get b. OK, now we need, also, the partial with respect to y.
So, Wy is bx 2cy.
In case you don't believe what I told you about the mixed partials, Wyx, well, you can check.
And it's, again, b.
So, they are, indeed, the same thing.
And, Wyy will be 2c.
So, if we now look at these quantities, that tells us, well, big A is two little a, big B is little b, big C is two little c. So, AC-B^2 is what we used to call four little ac minus b2.
OK, ooh.
[LAUGHTER] So, now you can compare the cases.
They are not listed in the same order just to make it harder.
So, we said first, so the saddle case is when AC-B^2 in big letters is negative, that's the same as 4ac-b2 in lower case is negative.
The case where capital AC-B2 is positive, local min and local max corresponds to this one.
And, the case where we can't conclude was what used to be the degenerate one.
OK, so at least we don't seem to have messed up when copying the formula.
Now, why does that work more generally than that?
Well, the answer that is, again, Taylor approximation.
Aww.
OK, so let me just do here quadratic approximation.
So, quadratic approximation tells me the following thing.
It tells me, if I have a function, f of xy, and I want to understand the change in f when I change x and y a little bit.
Well, there's the first-order terms.
There is the linear terms that by now you should know and be comfortable with.
That's fx times the change in x.
And then, there's fy times the change in y. OK, that's the starting point.
But now, of course, if x and y, sorry, if we are at the critical point, then that's going to be zero at the critical point.
So, that term actually goes away, and that's also zero at the critical point.
So, that term also goes away.
OK, so linear approximation is really no good.
We need more terms.
So, what are the next terms?
Well, the next terms are quadratic terms, and so I mean, if you remember the Taylor formula for a function of a single variable, there was the derivative times x minus x0 plus one half of a second derivative times x-x0^2.
And see, this side here is really Taylor approximation in one variable looking only at x.
But of course, we also have terms involving y, and terms involving simultaneously x and y.
And, these terms are fxy times change in x times change in y plus one half of fyy(y-y0)^2.
There's no one half in the middle because, in fact, you would have two terms, one for xy, one for yx, but they are the same.
And then, if you want to continue, there is actually cubic terms involving the third derivatives, and so on, but we are not actually looking at them.
And so, now, when we do this approximation, well, the type of critical point remains the same when we replace the function by this approximation.
And so, we can apply the argument that we used to deduce things in the quadratic case.
In fact, it still works in the general case using this approximation formula.
So -- The general case reduces to the quadratic case.
And now, you see actually why, well, here you see, again, how this coefficient which we used to call little a is also one half of capital A.
And same here: this coefficient is what we call capital B or little b, and this coefficient here is what we called little c or one half of capital C. And then, when you replace these into the various cases that we had here, you end up with the second derivative test.
So, what about the degenerate case?
Why can't we just say, well, it's going to be a degenerate critical point?
So, the reason is that this approximation formula is reasonable only if the higher order terms are negligible.
OK, so in fact, secretly, there's more terms.
This is only an approximation.
There would be terms involving third derivatives, and maybe even beyond that.
And, so it is not to generate case, they don't actually matter because the shape of the function, the shape of the graph, is actually determined by the quadratic terms.
But, in the degenerate case, see, if I start with this and I add something even very, very small along the y axis, then that can be enough to bend this very slightly up or slightly down, and turn my degenerate point in to either a minimum or a saddle point.
And, I won't be able to tell until I go further in the list of derivatives.
So, in the degenerate case, what actually happens depends on the higher order derivatives.
So, we will need to analyze things more carefully.
Well, we're not going to bother with that in this class.
So, we'll just say, well, we cannot compute, OK?
I mean, you have to realize that in real life, you have to be extremely unlucky for this quantity to end up being exactly 0.
[LAUGHTER] Well, if that happens, then what you should do is maybe try by inspection.
See if there's a good reason why the function should always be positive or always be negative, or something.
Or, you know, plot it on a computer and see what happens.
But, otherwise we can't compute.
OK, so let's do an example.
So, probably I should leave this on so that we still have the test with us.
And, instead, OK, so I'll do my example here.
OK, so just an example.
Let's look at f of (x, y) = x y 1/xy, where x and y are positive.
So, I'm looking only at the first quadrant.
OK, I mean, I'm doing this because I don't want the denominator to become zero.
So, I'm just looking at the situation.
So, let's look first for, so, the question will be, what are the minimum and maximum of this function?
So, the first thing we should do to answer this question is look for critical points, OK?
So, for that, we have to compute the first derivatives.
OK, so fx is one minus one over x^2y, OK?
Take the derivative of one over x, that's negative one over x^2.
And, we'll want to set that equal to zero.
And fy is one minus one over xy^2.
And, we want to set that equal to zero.
So, what are the equations we have to solve?
Well, I guess x^2y equals one, I mean, if I move this guy over here I get one over x^2y equals one.
That's x^2y equals one, and xy^2 equals one.
What do you get by comparing these two?
Well, x and y should both be, OK, so yeah, I agree with you that one and one is a solution.
Why is it the only one?
So, first, if I divide this one by that one, I get x over y equals one.
So, it tells me x equals y.
And then, if x equals y, then if I put that into here, it will give me y^3 equals one, which tells me y equals one, and therefore, x equals one as well.
OK, so, there's only one solution.
There's only one critical point, which is going to be (1,1).
OK, so, now here's where you do a bit of work.
What do you think of that critical point?
OK, I see some valid votes.
I see some, OK, I see a lot of people answering four.
[LAUGHTER] that seems to suggest that maybe you haven't completed the second derivative yet.
Yes, I see someone giving the correct answer.
I see some people not giving quite the correct answer.
I see more and more correct answers.
OK, so let's see.
To figure out what type of point is, we should compute the second partial derivatives.
So, fxx is, what do we get what we take the derivative of this with respect to x?
Two over x^3y, OK?
So, at our point, a will be 2.
Fxy will be one over x^2y^2.
So, B will be one.
And, Fyy is going to be two over xy^3.
So, C will be two.
And so that tells us, well, AC-B^2 is four minus one.
Sorry, I should probably use a different blackboard for that.
AC-B2 is two times two minus 1^2 is three.
It's positive.
That tells us we are either a local minimum or local maximum.
And, A is positive.
So, it's a local minimum.
And, in fact, you can check it's the global minimum.
What about the maximum?
Well, if a maximum is not actually at a critical point, it's on the boundary, or at infinity.
See, so we have actually to check what happens when x and y go to zero or to infinity.
Well, if that happens, if x or y goes to infinity, then the function goes to infinity.
Also, if x or y goes to zero, then one over xy goes to infinity.
So, the maximum, well, the function goes to infinity when x goes to infinity or y goes to infinity, or x and y go to zero.
So, it's not at a critical point.
OK, so, in general, we have to check both the critical points and the boundaries to decide what happens.
OK, the end.
Have a nice weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So far we have learned about partial derivatives and how to use them to find minima and maxima of functions of two variables or several variables.
And now we are going to try to study, in more detail, how functions of several variables behave, how to compete their variations.
How to estimate the variation in arbitrary directions.
And so for that we are going to need some more tools actually to study this things.
More tools to study functions.
Today's topic is going to be differentials.
And, just to motivate that, let me remind you about one trick that you probably know from single variable calculus, namely implicit differentiation.
Let's say that you have a function y equals f of x then you would sometimes write dy equals f prime of x times dx.
And then maybe you would -- We use implicit differentiation to actually relate infinitesimal changes in y with infinitesimal changes in x.
And one thing we can do with that, for example, is actually figure out the rate of change dy by dx, but also the reciprocal dx by dy.
And so, for example, let's say that we have y equals inverse sin(x).
Then we can write x equals sin(y).
And, from there, we can actually find out what is the derivative of this function if we didn't know the answer already by writing dx equals cosine y dy.
That tells us that dy over dx is going to be one over cosine y.
And now cosine for relation to sine is basically one over square root of one minus x^2.
And that is how you find the formula for the derivative of the inverse sine function.
A formula that you probably already knew, but that is one way to derive it.
Now we are going to use also these kinds of notations, dx, dy and so on, but use them for functions of several variables.
And, of course, we will have to learn what the rules of manipulation are and what we can do with them.
The actual name of that is the total differential, as opposed to the partial derivatives.
The total differential includes all of the various causes that can change -- Sorry.
All the contributions that can cause the value of your function f to change.
Namely, let's say that you have a function maybe of three variables, x, y, z, then you would write df equals f sub x dx plus f sub y dy plus f sub z dz.
Maybe, just to remind you of the other notation, partial f over partial x dx plus partial f over partial y dy plus partial f over partial z dz.
Now, what is this object?
What are the things on either side of this equality?
Well, they are called differentials.
And they are not numbers, they are not vectors, they are not matrices, they are a different kind of object.
These things have their own rules of manipulations, and we have to learn what we can do with them.
So how do we think about them?
First of all, how do we not think about them?
Here is an important thing to know.
Important.
df is not the same thing as delta f. That is meant to be a number.
It is going to be a number once you have a small variation of x, a small variation of y, a small variation of z.
These are numbers.
Delta x, delta y and delta z are actual numbers, and this becomes a number.
This guy actually is not a number.
You cannot give it a particular value.
All you can do with a differential is express it in terms of other differentials.
In fact, this dx, dy and dz, well, they are mostly symbols out there.
But if you want to think about them, they are the differentials of x, y and z.
In fact, you can think of these differentials as placeholders where you will put other things.
Of course, they represent, you know, there is this idea of changes in x, y, z and f. One way that one could explain it, and I don't really like it, is to say they represent infinitesimal changes.
Another way to say it, and I think that is probably closer to the truth, is that these things are somehow placeholders to put values and get a tangent approximation.
For example, if I do replace these symbols by delta x, delta y and delta z numbers then I will actually get a numerical quantity.
And that will be an approximation formula for delta.
It will be the linear approximation, a tangent plane approximation.
What we can do -- Well, let me start first with maybe something even before that.
The first thing that it does is it can encode how changes in x, y, z affect the value of f. I would say that is the most general answer to what is this formula, what are these differentials.
It is a relation between x, y, z and f. And this is a placeholder for small variations, delta x, delta y and delta z to get an approximation formula.
Which is delta f is approximately equal to fx delta x fy delta y fz delta z.
It is getting cramped, but I am sure you know what is going on here.
And observe how this one is actually equal while that one is approximately equal.
So they are really not the same.
Another thing that the notation suggests we can do, and they claim we can do, is divide everything by some variable that everybody depends on.
Say, for example, that x, y and z actually depend on some parameter t then they will vary, at a certain rate, dx over dt, dy over dt, dz over dt.
And what the differential will tell us then is the rate of change of f as a function of t, when you plug in these values of x, y, z, you will get df over dt by dividing everything by dt in here.
The first thing we can do is divide by something like dt to get infinitesimal rate of change.
Well, let me just say rate of change.
df over dt equals f sub x dx over dt plus f sub y dy over dt plus f sub z dz over dt.
And that corresponds to the situation where x is a function of t, y is a function of t and z is a function of t. That means you can plug in these values into f to get, well, the value of f will depend on t, and then you can find the rate of change with t of a value of f. These are the basic rules.
And this is known as the chain rule.
It is one instance of a chain rule, which tells you when you have a function that depends on something, and that something in turn depends on something else, how to find the rate of change of a function on the new variable in terms of the derivatives of a function and also the dependence between the various variables.
Any questions so far?
No.
OK. A word of warming, in particular, about what I said up here.
It is kind of unfortunate, but the textbook actually has a serious mistake on that.
I mean they do have a couple of formulas where they mix a d with a delta, and I warn you not to do that, please.
I mean there are d's and there are delta's, and basically they don't live in the same world.
They don't see each other.
The textbook is lying to you.
Let's see.
The first and the second claims, I don't really need to justify because the first one is just stating some general principle, but I am not making a precise mathematical claim.
The second one, well, we know the approximation formula already, so I don't need to justify it for you.
But, on the other hand, this formula here, I mean, you probably have a right to expect some reason for why this works.
Why is this valid?
After all, I first told you we have these new mysterious objects.
And then I am telling you we can do that, but I kind of pulled it out of my hat.
I mean I don't have a hat.
Why is this valid?
How can I get to this?
Here is a first attempt of justifying how to get there.
Let's see.
Well, we said df is f sub x dx plus f sub y dy plus f sub z dz.
But we know if x is a function of t then dx is x prime of t dt, dy is y prime of t dt, dz is z prime of t dt.
If we plug these into that formula, we will get that df is f sub x times x prime t dt plus f sub y y prime of t dt plus f sub z z prime of t dt.
And now I have a relation between df and dt.
See, I got df equals sometimes times dt.
That means the rate of change of f with respect to t should be that coefficient.
If I divide by dt then I get the chain rule.
That kind of works, but that shouldn't be completely satisfactory.
Let's say that you are a true skeptic and you don't believe in differentials yet then it is maybe not very good that I actually used more of these differential notations in deriving the answer.
That is actually not how it is proved.
The way in which you prove the chain rule is not this way because we shouldn't have too much trust in differentials just yet.
I mean at the end of today's lecture, yes, probably we should believe in them, but so far we should be a little bit reluctant to believe these kind of strange objects telling us weird things.
Here is a better way to think about it.
One thing that we have trust in so far are approximation formulas.
We should have trust in them.
We should believe that if we change x a little bit, if we change y a little bit then we are actually going to get a change in f that is approximately given by these guys.
And this is true for any changes in x, y, z, but in particular let's look at the changes that we get if we just take these formulas as function of time and change time a little bit by delta t. We will actually use the changes in x, y, z in a small time delta t. Let's divide everybody by delta t. Here I am just dividing numbers so I am not actually playing any tricks on you.
I mean we don't really know what it means to divide differentials, but dividing numbers is something we know.
And now, if I take delta t very small, this guy tends to the derivative, df over dt.
Remember, the definition of df over dt is the limit of this ratio when the time interval delta t tends to zero.
That means if I choose smaller and smaller values of delta t then these ratios of numbers will actually tend to some value, and that value is the derivative.
Similarly, here delta x over delta t, when delta t is really small, will tend to the derivative dx/dt.
And similarly for the others.
That means, in particular, we take the limit as delta t tends to zero and we get df over dt on one side and on the other side we get f sub x dx over dt plus f sub y dy over dt plus f sub z dz over dt.
And the approximation becomes better and better.
Remember when we write approximately equal that means it is not quite the same, but if we take smaller variations then actually we will end up with values that are closer and closer.
When we take the limit, as delta t tends to zero, eventually we get an equality.
I mean mathematicians have more complicated words to justify this statement.
I will spare them for now, and you will see them when you take analysis if you go in that direction.
Any questions so far?
No.
OK. Let's check this with an example.
Let's say that we really don't have any faith in these things so let's try to do it.
Let's say I give you a function that is x ^2 y z.
And let's say that maybe x will be t, y will be e^t and z will be sin(t).
What does the chain rule say?
Well, the chain rule tells us that dw/dt is, we start with partial w over partial x, well, what is that?
That is 2xy, and maybe I should point out that this is w sub x, times dx over dt plus -- Well, w sub y is x squared times dy over dt plus w sub z, which is going to be just one, dz over dt.
And so now let's plug in the actual values of these things.
x is t and y is e^t, so that will be 2t e to the t, dx over dt is one plus x squared is t squared, dy over dt is e over t, plus dz over dt is cosine t. At the end of calculation we get 2t e to the t plus t squared e to the t plus cosine t. That is what the chain rule tells us.
How else could we find that?
Well, we could just plug in values of x, y and z, x plus w is a function of t, and take its derivative.
Let's do that just for verification.
It should be exactly the same answer.
And, in fact, in this case, the two calculations are roughly equal in complication.
But say that your function of x, y, z was much more complicated than that, or maybe you actually didn't know a formula for it, you only knew its partial derivatives, then you would need to use the chain rule.
So, sometimes plugging in values is easier but not always.
Let's just check quickly.
The other method would be to substitute.
W as a function of t. Remember w was x^2y z. x was t, so you get t squared, y is e to the t, plus z was sine t. dw over dt, we know how to take the derivative using single variable calculus.
Well, we should know.
If we don't know then we should take a look at 18.01 again.
The product rule that will be derivative of t squared is 2t times e to the t plus t squared time the derivative of e to the t is e to the t plus cosine t. And that is the same answer as over there.
I ended up writing, you know, maybe I wrote slightly more here, but actually the amount of calculations really was pretty much the same.
Any questions about that?
Yes?
What kind of object is w?
Well, you can think of w as just another variable that is given as a function of x, y and z, for example.
You would have a function of x, y, z defined by this formula, and I call it w. I call its value w so that I can substitute t instead of x, y, z.
Well, let's think of w as a function of three variables.
And then, when I plug in the dependents of these three variables on t, then it becomes just a function of t. I mean, really, my w here is pretty much what I called f before.
There is no major difference between the two.
Any other questions?
No.
OK. Let's see.
Here is an application of what we have seen.
Let's say that you want to understand actually all these rules about taking derivatives in single variable calculus.
What I showed you at the beginning, and then erased, basically justifies how to take the derivative of a reciprocal function.
And for that you didn't need multivariable calculus.
But let's try to justify the product rule, for example, for the derivative.
An application of this actually is to justify the product and quotient rules.
Let's think, for example, of a function of two variables, u and v, that is just the product uv.
And let's say that u and v are actually functions of one variable t. Then, well, d of uv over dt is given by the chain rule applied to f. This is df over dt.
So df over dt should be f sub q du over dt plus f sub v plus dv over dt.
But now what is the partial of f with respect to u?
It is v. That is v du over dt.
And partial of f with respect to v is going to be just u, dv over dt.
So you get back the usual product rule.
That is a slightly complicated way of deriving it, but that is a valid way of understanding how to take the derivative of a product by thinking of the product first as a function of variables, which are u and v. And then say, oh, but u and v were actually functions of a variable t. And then you do the differentiation in two stages using the chain rule.
Similarly, you can do the quotient rule just for practice.
If I give you the function g equals u of v. Right now I am thinking of it as a function of two variables, u and v. U and v themselves are actually going to be functions of t. Then, well, dg over dt is going to be partial g, partial u.
How much is that?
How much is partial g, partial u?
One over v times du over dt plus -- Well, next we need to have partial g over partial v. Well, what is the derivative of this with respect to v?
Here we need to know how to differentiate the inverse.
It is minus u over v squared times dv over dt.
And that is actually the usual quotient rule just written in a slightly different way.
I mean, just in case you really want to see it, if you clear denominators for v squared then you will see basically u prime times v minus v prime times u.
Now let's go to something even more crazy.
I claim we can do chain rules with more variables.
Let's say that I have a quantity.
Let's call it w for now.
Let's say I have quantity w as a function of say variables x and y.
And so in the previous setup x and y depended on some parameters t. But, actually, let's now look at the case where x and y themselves are functions of several variables.
Let's say of two more variables.
Let's call them u and v. I am going to stay with these abstract letters, but if it bothers you, if it sounds completely unmotivated think about it maybe in terms of something you might now.
Say, polar coordinates.
Let's say that I have a function but is defined in terms of the polar coordinate variables on theta.
And then I know I want to switch to usual coordinates x and y.
Or, the other way around, I have a function of x and y and I want to express it in terms of the polar coordinates r and theta.
Then I would want to know maybe how the derivatives, with respect to the various sets of variables, related to each other.
One way I could do it is, of course, to say now if I plug the formula for x and the formula for y into the formula for f then w becomes a function of u and v, and it can try to take partial derivatives.
If I have explicit formulas, well, that could work.
But maybe the formulas are complicated.
Typically, if I switch between rectangular and polar coordinates, there might be inverse trig, there might be maybe arctangent to express the polar angle in terms of x and y.
And when I don't really want to actually substitute arctangents everywhere, maybe I would rather deal with the derivatives.
How do I do that?
The question is what are partial w over partial u and partial w over partial v in terms of, let's see, what do we need to know to understand that?
Well, probably we should know how w depends on x and y.
If we don't know that then we are probably toast.
Partial w over partial x, partial w over partial y should be required.
What else should we know?
Well, it would probably help to know how x and y depend on u and v. If we don't know that then we don't really know how to do it.
We need also x sub u, x sub v, y sub u, y sub v. We have a lot of partials in there.
Well, let's see how we can do that.
Let's start by writing dw.
We know that dw is partial f, well, I don't know why I have two names, w and f. I mean w and f are really the same thing here, but let's say f sub x dx plus f sub y dy.
So far that is our new friend, the differential.
Now what do we want to do with it?
Well, we would like to get rid of dx and dy because we like to express things in terms of, you know, the question we are asking ourselves is let's say that I change u a little bit, how does w change?
Of course, what happens, if I change u a little bit, is y and y will change.
How do they change?
Well, that is given to me by the differential.
dx is going to be, well, I can use the differential again.
Well, x is a function of u and v. That will be x sub u times du plus x sub v times dv.
That is, again, taking the differential of a function of two variables.
Does that make sense?
And then we have the other guy, f sub y times, what is dy?
Well, similarly dy is y sub u du plus y sub v dv.
And now we have a relation between dw and du and dv.
We are expressing how w reacts to changes in u and v, which was our goal.
Now, let's actually collect terms so that we see it a bit better.
It is going to be f sub x times x sub u times f sub y times y sub u du plus f sub x, x sub v plus f sub y y sub v dv.
Now we have dw equals something du plus something dv.
Well, the coefficient here has to be partial f over partial u.
What else could it be?
That's the rate of change of w with respect to u if I forget what happens when I change v. That is the definition of a partial.
Similarly, this one has to be partial f over partial v. That is because it is the rate of change with respect to v, if I keep u constant, so that these guys are completely ignored.
Now you see how the total differential accounts for, somehow, all the partial derivatives that come as coefficients of the individual variables in these expressions.
Let me maybe rewrite these formulas in a more visible way and then re-explain them to you.
Here is the chain rule for this situation, with two intermediate variables and two variables that you express these in terms of.
In our setting, we get partial f over partial u equals partial f over partial x time partial x over partial u plus partial f over partial y times partial y over partial u.
And the other one, the same thing with v instead of u, partial f over partial x times partial x over partial v plus partial f over partial u partial y over partial v. I have to explain various things about these formulas because they look complicated.
And, actually, they are not that complicated.
A couple of things to know.
The first thing, how do we remember a formula like that?
Well, that is easy.
We want to know how f depends on u.
Well, what does f depend on?
It depends on x and y.
So we will put partial f over partial x and partial f over partial y.
Now, x and y, why are they here?
Well, they are here because they actually depend on u as well.
How does x depend on u?
Well, the answer is partial x over partial u.
How does y depend on u?
The answer is partial y over partial u.
See, the structure of this formula is simple.
To find the partial of f with respect to some new variable you use the partials with respect to the variables that f was initially defined in terms of x and y.
And you multiply them by the partials of x and y in terms of the new variable that you want to look at, v here, and you sum these things together.
That is the structure of the formula.
Why does it work?
Well, let me explain it to you in a slightly different language.
This asks us how does f change if I change u a little bit?
Well, why would f change if u changes a little bit?
Well, it would change because f actually depends on x and y and x and y depend on u.
If I change u, how quickly does x change?
Well, the answer is partial x over partial u.
And now, if I change x at this rate, how does that have to change?
Well, the answer is partial f over partial x times this guy.
Well, at the same time, y is also changing.
How fast is y changing if I change u?
Well, at the rate of partial y over partial u.
But now if I change this how does f change?
Well, the rate of change is partial f over partial y.
The product is the effect of how you change it, changing u, and therefore changing f. Now, what happens in real life, if I change u a little bit?
Well, both x and y change at the same time.
So how does f change?
Well, it is the sum of the two effects.
Does that make sense?
Good.
Of course, if f depends on more variables then you just have more terms in here.
OK.
Here is another thing that may be a little bit confusing.
What is tempting?
Well, what is tempting here would be to simplify these formulas by removing these partial x's.
Let's simplify by partial x.
Let's simplify by partial y.
We get partial f over partial u equals partial f over partial u plus partial f over partial u.
Something is not working properly.
Why doesn't it work?
The answer is precisely because these are partial derivatives.
These are not total derivatives.
And so you cannot simplify them in that way.
And that is actually the reason why we use this curly d rather than a straight d. It is to remind us, beware, there are these simplifications that we can do with straight d's that are not legal here.
Somehow, when you have a partial derivative, you must resist the urge of simplifying things.
No simplifications in here.
That is the simplest formula you can get.
Any questions at this point?
No.
Yes?
When would you use this and what does it describe?
Well, it is basically when you have a function given in terms of a certain set of variables because maybe there is a simply expression in terms of those variables.
But ultimately what you care about is not those variables, z and y, but another set of variables, here u and v. So x and y are giving you a nice formula for f, but actually the relevant variables for your problem are u and v. And you know x and y are related to u and v. So, of course, what you could do is plug the formulas the way that we did substituting.
But maybe that will give you very complicated expressions.
And maybe it is actually easier to just work with the derivates.
The important claim here is basically we don't need to know the actual formulas.
All we need to know are the rate of changes.
If we know all these rates of change then we know how to take these derivatives without actually having to plug in values.
Yes?
Yes, you could certain do the same things in terms of t. If x and y were functions of t instead of being functions of u and v then it would be the same thing.
And you would have the same formulas that I had, well, over there I still have it.
Why does that one have straight d's?
Well, the answer is I could put curly d's if I wanted, but I end up with a function of a single variable.
If you have a single variable then the partial, with respect to that variable, is the same thing as the usual derivative.
We don't actually need to worry about curly in that case.
But that one is indeed special case of this one where instead of x and y depending on two variables, u and v, they depend on a single variable t. Now, of course, you can call variables any name you want.
It doesn't matter.
This is just a slight generalization of that.
Well, not quite because here I also had a z.
See, I am trying to just confuse you by giving you functions that depend on various numbers of variables.
If you have a function of 30 variables, things work the same way, just longer, and you are going to run out of letters in the alphabet before the end.
Any other questions?
No.
What?
Yes?
If u and v themselves depended on another variable then you would continue with your chain rules.
Maybe you would know to express partial x over partial u in terms using that chain rule.
Sorry.
If u and v are dependent on yet another variable then you could get the derivative with respect to that using first the chain rule to pass from u v to that new variable, and then you would plug in these formulas for partials of f with respect to u and v. In fact, if you have several substitutions to do, you can always arrange to use one chain rule at a time.
You just have to do them in sequence.
That's why we don't actually learn that, but you can just do it be repeating the process.
I mean, probably at that stage, the easiest to not get confused actually is to manipulate differentials because that is probably easier.
Yes?
Curly f does not exist.
That's easy.
Curly f makes no sense by itself.
It doesn't exist alone.
What exists is only curly df over curly d some variable.
And then that accounts only for the rate of change with respect to that variable leaving the others fixed, while straight df is somehow a total variation of f. It accounts for all of the partial derivatives and their combined effects.
OK. Any more questions?
No.
Let me just finish up very quickly by telling you again one example where completely you might want to do this.
You have a function that you want to switch between rectangular and polar coordinates.
To make things a little bit concrete.
If you have polar coordinates that means in the plane, instead of using x and y, you will use coordinates r, distance to the origin, and theta, the angles from the x-axis.
The change of variables for that is x equals r cosine theta and y equals r sine theta.
And so that means if you have a function f that depends on x and y, in fact, you can plug these in as a function of r and theta.
Then you can ask yourself, well, what is partial f over partial r?
And that is going to be, well, you want to take partial f over partial x times partial x partial r plus partial f over partial y times partial y over partial r. That will end up being actually f sub x times cosine theta plus f sub y times sine theta.
And you can do the same thing to find partial f, partial theta.
And so you can express derivatives either in terms of x, y or in terms of r and theta with simple relations between them.
And the one last thing I should say.
On Thursday we will learn about more tricks we can play with variations of functions.
And one that is important, because you need to know it actually to do the p-set, is the gradient vector.
The gradient vector is simply a vector.
You use this downward pointing triangle as the notation for the gradient.
It is simply is a vector whose components are the partial derivatives of a function.
I mean, in a way, you can think of a differential as a way to package partial derivatives together into some weird object.
Well, the gradient is also a way to package partials together.
We will see on Thursday what it is good for, but some of the problems on the p-set use it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
so -- OK, so remember last time, on Tuesday we learned about the chain rule, and so for example we saw that if we have a function that depends, sorry, on three variables, x,y,z, that x,y,z themselves depend on some variable, t, then you can find a formula for df/dt by writing down wx/dx dt wy dy/dt wz dz/dt.
And, the meaning of that formula is that while the change in w is caused by changes in x, y, and z, x, y, and z change at rates dx/dt, dy/dt, dz/dt.
And, this causes a function to change accordingly using, well, the partial derivatives tell you how sensitive w is to changes in each variable.
OK, so, we are going to just rewrite this in a new notation.
So, I'm going to rewrite this in a more concise form as gradient of w dot product with velocity vector dr/dt.
So, the gradient of w is a vector formed by putting together all of the partial derivatives.
OK, so it's the vector whose components are the partials.
And, of course, it's a vector that depends on x, y, and z, right?
These guys depend on x, y, z.
So, it's actually one vector for each point, x, y, z.
You can talk about the gradient of w at some point, x, y, z.
So, at each point, it gives you a vector.
That actually is what we will call later a vector field.
We'll get back to that later.
And, dr/dt is just the velocity vector dx/dt, dy/dt, dz/dt.
OK, so the new definition for today is the definition of the gradient vector.
And, our goal will be to understand a bit better, what does this vector mean?
What does it measure?
And, what can we do with it?
But, you see that in terms of information content, it's really the same information that's already in the partial derivatives, or in the differential.
So, yes, and I should say, of course you can also use the gradient and other things like approximation formulas and so on.
And so far, it's just notation.
It's a way to rewrite things.
But, so here's the first cool property of the gradient.
So, I claim that the gradient vector is perpendicular to the level surface corresponding to setting the function, w, equal to a constant.
OK, so if I draw a contour plot of my function, so, actually forget about z because I want to draw a two variable contour plot.
So, say I have a function of two variables, x and y, then maybe it has some contour plot.
And, I'm saying if I take the gradient of a function at this point, (x,y).
So, I will have a vector.
Well, if I draw that vector on top of a contour plot, it's going to end up being perpendicular to the level curve.
Same thing if I have a function of three variables.
Then, I can try to draw its contour plot.
Of course, I can't really do it because the contour plot would be living in space with x, y, and z.
But, it would be a bunch of level faces, and the gradient vector would be a vector in space.
That vector is perpendicular to the level faces.
So, let's try to see that on a couple of examples.
So, let's do a first example.
What's the easiest case?
Let's take a linear function of x, y, and z.
So, I will take w equals a1 times x plus a2 times y plus a3 times z.
Well, so, what's the gradient of this function?
Well, the first component will be a1.
That's partial w partial x.
Then, a2, that's partial w partial y, and a3, partial w partial z.
Now, what is the levels of this?
Well, if I set w equal to some constant, c, that means I look at the points where a1x a2y a3z equals c. What kind of service is that?
It's a plane.
And, we know how to find a normal vector to this plane just by looking at the coefficients.
So, it's a plane with a normal vector exactly this gradient.
And, in fact, in a way, this is the only case you need to check because of linear approximations.
If you replace a function by its linear approximation, that means you will replace the level surfaces by their tension planes.
And then, you'll actually end up in this situation.
But maybe that's not very convincing.
So, let's do another example.
So, let's do a second example.
Let's say we look at the function x^2 y^2.
OK, so now it's a function of just two variables because that way we'll be able to actually draw a picture for you.
OK, so what are the level sets of this function?
Well, they're going to be circles, right?
w equals c is a circle, x^2 y^2 = c. So, I should say, maybe, sorry, the level curve is a circle.
So, the contour plot looks something like that.
Now, what's the gradient vector?
Well, the gradient of this function, so, partial w partial x is 2x.
And partial w partial y is 2y.
So, let's say I take a point, x comma y, and I try to draw my gradient vector.
So, here at x, y, so, I have to draw the vector, <2x, 2y>.
What does it look like?
Well, it's going in that direction.
It's parallel to the position vector for this point.
It's actually twice the position vector.
So, I guess it goes more or less like this.
What's interesting, too, is it is perpendicular to this circle.
OK, so it's a general feature.
Actually, let me show you more examples, oops, not the one I want.
So, I don't know if you can see it so well.
Well, hopefully you can.
So, here I have a contour plot of a function, and I have a blue vector.
That's the gradient vector at the pink point on the plot.
So, you can see, I can move the pink point, and the gradient vector, of course, changes because the gradient depends on x and y.
But, what doesn't change is that it's always perpendicular to the level curves.
Anywhere I am, my gradient stays perpendicular to the level curve.
OK, is that convincing?
Is that visible for people who can't see blue?
OK, so, OK, so we have a lot of evidence, but let's try to prove the theorem because it will be interesting.
So, first of all, sorry, any questions about the statement, the example, anything, yes?
Ah, very good question.
Does the gradient vector, why is the gradient vector perpendicular in one direction rather than the other?
So, we'll see the answer to that in a few minutes.
But let me just tell you immediately, to the side, which side it's pointing to, it's always pointing towards higher values of a function.
OK, and we'll see in that maybe about half an hour.
So, well, let me say actually points towards higher values of w. OK, any other questions?
I don't see any questions.
OK, so let's try to prove this theorem, at least this part of the theorem.
We're not going to prove that just yet.
That will come in a while.
So, well, maybe we want to understand first what happens if we move inside the level curve, OK?
So, let's imagine that we are taking a moving point that stays on the level curve or on the level surface.
And then, we know, well, what happens is that the function stays constant.
But, we can also know how quickly the function changes using the chain rule up there.
So, maybe the chain rule will actually be the key to understanding how the gradient vector and the motion on the level service relate.
So, let's take a curve, r equals r of t, that stays inside, well, maybe I should say on the level surface, w equals c. So, let's think about what that means.
So, just to get you used to this idea, I'm going to draw a level surface of a function of three variables.
OK, so it's a surface given by the equation w of x, y, z equals some constant, c. And, so now I'm going to have a point on that, and it's going to move on that surface.
So, I will have some parametric curve that lives on this surface.
So, the question is, what's going to happen at any given time?
Well, the first observation is that the velocity vector, what can I say about the velocity vector of this motion?
It's going to be tangent to the level surface, right?
If I move on a surface, then at any point, my velocity is tangent to the curve.
But, if it's tangent to the curve, then it's also tangent to the surface because the curve is inside the surface.
So, OK, it's getting a bit cluttered.
Maybe I should draw a bigger picture.
Let me do that right away here.
So, I have my level surface, w equals c. I have a curve on that, and at some point, I'm going to have a certain velocity.
So, the claim is that the velocity, v, equals dr/dt is tangent -- -- to the level, w equals c because it's tangent to the curve, and the curve is inside the level, OK?
Now, what else can we say?
Well, we have, the chain rule will tell us how the value of w changes.
So, by the chain rule, we have dw/dt.
So, the rate of change of the value of w as I move along this curve is given by the dot product between the gradient and the velocity vector.
And, so, well, maybe I can rewrite it as w dot v, and that should be, well, what should it be?
What happens to the value of w as t changes?
Well, it stays constant because we are moving on a curve.
That curve might be complicated, but it stays always on the level, w equals c. So, it's zero because w of t equals c, which is a constant.
OK, is that convincing?
OK, so now if we have a dot product that's zero, that tells us that these two guys are perpendicular.
So -- So if the gradient vector is perpendicular to v, OK, that's a good start.
We know that the gradient is perpendicular to this vector tangent that's tangent to the level surface.
What about other vectors tangent to the level surface?
Well, in fact, I could use any curve drawn on the level of w equals c. So, I could move, really, any way I wanted on that surface.
In particular, I claim that I could have chosen my velocity vector to be any vector tangent to the surface.
OK, so let's write this.
So this is true for any curve, or, I'll say for any motion on the level surface, w equals c. So that means v can be any vector tangent to the surface tangent to the level.
See, for example, OK, let me draw one more picture.
OK, so I have my level surface.
So, I'm drawing more and more levels, and they never quite look the same.
But I have a point.
And, at this point, I have the tangent plane to the level surface.
OK, so this is tangent plane to the level.
Then, if I choose any vector in that tangent plane.
Let's say I choose the one that goes in that direction.
Then, I can actually find a curve that goes in that direction, and stays on the level.
So, here, that would be a curve that somehow goes from the right to the left, and of course it has to end up going up or something like that.
OK, so given any vector tangent -- -- let's call that vector v tangent to the level, we get that the gradient is perpendicular to v. So, if the gradient is perpendicular to this vector tangent to this curve, but also to any vector, I can draw that tangent to my surface.
So, what does that mean?
Well, that means the gradient is actually perpendicular to the tangent plane or to the surface at this point.
So, the gradient is perpendicular.
And, well, here, I've illustrated things with a three-dimensional example, but really it works the same if you have only two variables.
Then you have a level curve that has a tangent line, and the gradient is perpendicular to that line.
OK, any questions?
No?
OK, so, let's see.
That's actually pretty neat because there is a nice application of this, which is to try to figure out, now we know, actually, how to find the tangent plane to anything, pretty much.
OK, so let's see.
So, let's say that, for example, I want to find -- -- the tangent plane -- -- to the surface with equation, let's say, x^2 y^2-z^2 = 4 at the point (2,1, 1).
Let me write that.
So, how do we do that?
Well, one way that we already know, if we solve this for z, so we can write z equals a function of x and y, then we know tangent plane approximation for the graph of a function, z equals some function of x and y.
But, that doesn't look like it's the best way to do it.
OK, the best way to it, now that we have the gradient vector, is actually to directly say, oh, we know the normal vector to this plane.
The normal vector will just be the gradient.
Oh, I think I have a cool picture to show.
OK, so that's what it looks like.
OK, so here you have the surface x2 y2-z2 equals four.
That's called a hyperboloid because it looks like when you get when you spin a hyperbola around an axis.
And, here's a tangent plane at the given point.
So, it doesn't look very tangent because it crosses the surface.
But, it's really, if you think about it, you will see it's really the plane that's approximating the surface in the best way that you can at this given point.
It is really the tangent plane.
So, how do we find this plane?
Well, you can plot it on a computer.
That's not exactly how you would look for it in the first place.
So, the way to do it is that we compute the gradient.
So, a gradient of what?
Well, a gradient of this function.
OK, so I should say, this is the level set, w equals four, where w equals x^2 y^2 - z^2.
And so, we know that the gradient of this, well, what is it?
2x, then 2y, and then negative 2z.
So, at this given point, I guess we are at x equals two.
So, that's four.
And then, y and z are one.
So, two, negative two.
OK, and that's going to be the normal vector to the surface or to the tangent plane.
That's one way to define the tangent plane.
All right, it has the same normal vector as the surface.
That's one way to define the normal vector to the surface, if you prefer.
Being perpendicular to the surface means that you are perpendicular to its tangent plane.
OK, so the equation is, well, 4x 2y-2z equals something, where something is, well, we should just plug in that point.
We'll get eight plus two minus two looks like we'll get eight.
And, of course, we could simplify dividing everything by two, but it's not very important here.
OK, so now if you have a surface given by an evil equation, and a point on the surface, well, you know how to find the tangent plane to the surface at that point.
OK, any questions?
No.
OK, let me give just another reason why, another way that we could have seen this.
So, I claim, in fact, we could have done this without the gradient, or using the gradient in a somehow disguised way.
So, here's another way.
So, the other way to do it would be to start with a differential, OK?
dw, while it's pretty much the same content, but let me write it as a differential, dw is 2xdx 2ydy-2zdz.
So, at a given point, at (2,1, 1), this is 4dx 2dy-2dz.
Now, if we want to change this into an approximation formula, we can.
We know that the change in w is approximately equal to 4 delta x 2 delta y - 2 delta z. OK, so when do we stay on the level surface?
Well, we stay on the level surface when w doesn't change, so, when this becomes zero, OK?
Now, what does this approximation sign mean?
Well, it means for small changes in x, y, z, this guy will be close to that guy.
It also means something else.
Remember, these approximation formulas, they are linear approximations.
They mean that we replace the function, actually, by some closest linear formula that will be nearby.
And so, in particular, if we set this equal to zero instead of approximately zero, it means we'll actually be moving on the tangent plane to the level set.
If you want strict equalities in approximations means that we replace the function by its tangent approximation.
So -- [APPLAUSE] OK, so the level corresponds to delta w equals zero, and its tangent plane corresponds to four delta x plus two delta y minus two delta z equals zero.
That's what I'm trying to say, basically.
And, what's delta x?
Well, that means it's the change in x.
So, what's the change in x here?
That means, well, we started with x equals two, and we moved to some other value, x.
So, that's actually x- 2, right?
That's how much x has changed compared to 2.
And, two times (y - 1) minus two times z - 1 = 0.
That's the equation of a tangent plane.
It's the same equation as the one over there.
These are just two different methods to get it.
OK, so this one explains to you what's going on in terms of approximation formulas.
This one goes right away, by using the gradient factor.
So, in a way, with this one, you don't have to think nearly as much.
But, you can use either one.
OK, questions?
No?
OK, so let's move on to new topic, which is another application of a gradient vector, and that is directional derivatives.
OK, so let's say that we have a function of two variables, x and y.
Well, we know how to compute partial w over partial x or partial w over partial y, which measure how w changes if I move in the direction of the x axis or in the direction of the y axis.
So, what about moving in other directions?
Well, of course, we've seen other approximation formulas and so on.
But, we can still ask, is there a derivative in every direction?
And that's basically, yes, that's the directional derivative.
OK, so these are derivatives in the direction of I hat or j hat, the vectors that go along the x or the y axis.
So, what if we move in another direction, let's say, the direction of some unit vector, let's call it u .
OK, so if I give you a unit vector, you can ask yourself, if I move in the direction, how quickly will my function change?
So -- So, let's look at the straight trajectory.
What this should mean is I start at some value, x, y, and there I have my vector u.
And, I'm going to move in a straight line in the direction of u.
And, I have the graph of my function -- -- and I'm asking myself how quickly does the value change when I move on the graph in that direction?
OK, so let's look at a straight line trajectory So, we have a position vector, r, that will depend on some parameter which I will call s. You'll see why very soon, in such a way that the derivative is this given unit vector u hat.
So, why do I use s for my parameter rather than t. Well, it's a convention.
I'm moving at unit speed along this line.
So that means that actually, I'm parameterizing things by the distance that I've traveled along a curve, sorry, along this line.
So, here it's called s in the sense of arc length.
Actually, it's not really an arc because it's a straight line, so it's the distance along the line.
OK, so because we are parameterizing by distance, we are just using s as a convention just to distinguish it from other situations.
And, so, now, the question will be, what is dw/ds?
What's the rate of change of w when I move like that?
Well, of course we know the answer because that's a special case of the chain rule.
So, that's how we will actually compute it.
But, in terms of what it means, it really means we are asking ourselves, we start at a point and we change the variables in a certain direction, which is not necessarily the x or the y direction, but really any direction.
And then, what's the derivative in that direction?
OK, does that make sense as a concept?
Kind of?
I see some faces that are not completely convinced.
So, maybe you should show more pictures.
Well, let me first write down a bit more and show you something.
So I just want to give you the actual definition.
Sorry, first of all in case you wonder what this is all about, so let's say the components of our unit vector are two numbers, a and b.
Then, it means we'll move along the line x of s equals some initial value, the point where we are actually at the directional derivative plus s times a, or I meant to say plus a times s. And, y of s equals y0 bs.
And then, we plug that into w. And then we take the derivative.
So, we have a notation for that which is going to be dw/ds with a subscript in the direction of u to indicate in which direction we are actually going to move.
And, that's called the directional derivative -- -- in the direction of u. OK, so, let's see what it means geometrically.
So, remember, we've seen things about partial derivatives, and we see that the partial derivatives are the slopes of slices of the graph by vertical planes that are parallel to the x or the y directions.
OK, so, if I have a point, at any point, I can slice the graph of my function by two planes, one that's going along the x, one along the y direction.
And then, I can look at the slices of the graph.
Let me see if I can use that thing.
So, we can look at the slices of the graph that are drawn here.
In fact, we look at the tangent lines to the slices, and we look at the slope and that gives us the partial derivatives in case you are on that side and want to see also the pointer that was here.
So, now, similarly, the directional derivative means, actually, we'll be slicing our graph by the vertical plane.
It's not really colorful, something more colorful.
We'll be slicing things by a plane that is now in the direction of this vector, u, and we'll be looking at the slope of the slice of the graph.
So, what that looks like here, so that's the same applet the way that you've used on your problem set in case you are wondering.
So, now, I'm picking a point on the contour plot.
And, at that point, I slice the graph.
So, here I'm starting by slicing in the direction of the x axis.
So, in fact, what I'm measuring here by the slope of the slice is the partial in the x direction.
It's really partial f partial x, which is also the directional derivative in the direction of i.
And now, if I rotate the slice, then I have all of these planes.
So, you see at the bottom left, I have the direction in which I'm going.
There's this, like, rotating line that tells you in which direction I'm going to be moving.
And for each direction, I have a plane.
And, when I slice by that plane, I will get, so I have this direction here going maybe to the southwest.
So, that gives me a slice of my graph by a vertical plane, and the slice has a certain slope.
And, the slope is going to be the directional derivative in that direction.
OK, I think that's as graphic as I can get.
OK, any questions about that?
No?
OK, so let's see how we compute that guy.
So, let me just write again just in case you want to, in case you didn't hear me it's the slope of the slice of the graph by a vertical plane -- -- that contains the given direction, that's parallel to the direction, u.
So, how do we compute it?
Well, we can use the chain rule.
The chain rule implies that dw/ds is actually the gradient of w dot product with the velocity vector dr/ds.
But, remember we say that we are going to be moving at unit speed in the direction of u.
So, in fact, that's just gradient w dot product with the unit vector u. OK, so the formula that we remember is really dw/ds in the direction of u is gradient w dot product of u.
And, maybe I should also say in words, this is the component of the gradient in the direction of u.
And, maybe that makes more sense.
So, for example, the directional derivative in the direction of I hat is the component along the x axes.
That's the same as, indeed, the partial derivatives in the x direction.
Things make sense.
dw/ds in the direction of I hat is, sorry, gradient w dot I hat, which is wx,maybe I should write, partial w of partial x. OK, now, so that's basically what we need to know to compute these guys.
So now, let's go back to the gradient and see what this tells us about the gradient.
[APPLAUSE] I see you guys are having fun.
OK, OK, let's do a little bit of geometry here.
That should calm you down.
So, we said dw/ds in the direction of u is gradient w dot u.
That's the same as the length of gradient w times the length of u.
Well, that happens to be one because we are taking the unit vector times the cosine of the angle between the gradient and the given unit vector, u, so, have this angle, theta.
OK, that's another way of saying we are taking the component of a gradient in the direction of u.
But now, what does that tell us?
Well, let's try to figure out in which directions w changes the fastest, in which direction it increases the most or decreases the most, or doesn't actually change.
So, when is this going to be the largest?
If I fix a point, if I set a point, then the gradient vector at that point is given to me.
But, the question is, in which direction does it change the most quickly?
Well, what I can change is the direction, and this will be the largest when the cosine is one.
So, this is largest when the cosine of the angle is one.
That means the angle is zero.
That means u is actually in the direction of the gradient.
OK, so that's a new way to think about the direction of a gradient.
The gradient is the direction in which the function increases the most quickly at that point.
So, the direction of gradient w is the direction of fastest increase of w at the given point.
And, what is the magnitude of w?
Well, it's actually the directional derivative in that direction.
OK, so if I go in that direction, which gives me the fastest increase, then the corresponding slope will be the length of the gradient.
And, with the direction of the fastest decrease?
It's going in the opposite direction, right?
I mean, if you are on a mountain, and you know that you are facing the mountain, that's the direction of fastest increase.
The direction of fastest decrease is behind you straight down.
OK, so, the minimal value of dw/ds is achieved when cosine of theta is minus one.
That means theta equals 180�.
That means u is in the direction of minus the gradient.
It points opposite to the gradient.
And, finally, when do we have dw/ds equals zero?
So, in which direction does the function not change?
Well, we have two answers to that.
One is to just use the formula.
So, that's one cosine theta equals zero.
That means theta equals 90 degrees.
That means that u is perpendicular to the gradient.
The other way to think about it, the direction in which the value doesn't change is a direction that's tangent to the level surface.
If we are not changing a, it means we are moving along the level.
And, that's the same thing -- -- as being tangent to the level.
So, let me just show that on the picture here.
So, if actually show you the gradient, you can't really see it here.
I need to move it a bit.
So, the gradient here is pointing straight up at the point that I have chosen.
Now, if I choose a slice that's perpendicular, and a direction that's perpendicular to the gradient, so that's actually tangent to the level curve, then you see that my slice is flat.
I don't actually have any slop.
The directional derivative in a direction that's perpendicular to the gradient is basically zero.
Now, if I rotate, then the slope sort of increases, increases, increases, and it becomes the largest when I'm going in the direction of a gradient.
So, here, I have, actually, a pretty big slope.
And now, if I keep rotating, then the slope will decrease again.
Then it becomes zero when I perpendicular, and then it becomes negative.
It's the most negative when I pointing away from the gradient and then becomes zero again when I'm back perpendicular.
OK, so for example, if I give you a contour plot, and I ask you to draw the direction of the gradient vector, well, at this point, for example, you would look at the picture.
The gradient vector would be going perpendicular to the level.
And, it would be going towards higher values of a function.
I don't know if you can see the labels, but the thing in the middle is a minimum.
So, it will actually be pointing in this kind of direction.
OK, so that's it for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Last time we saw things about gradients and directional derivatives.
Before that we studied how to look for minima and maxima of functions of several variables.
And today we are going to look again at min/max problems but in a different setting, namely, one for variables that are not independent.
And so what we will see is you may have heard of Lagrange multipliers.
And this is the one point in the term when I can shine with my French accent and say Lagrange's name properly.
OK. What are Lagrange multipliers about?
Well, the goal is to minimize or maximize a function of several variables.
Let's say, for example, f of x, y, z, but where these variables are no longer independent.
They are not independent.
That means that there is a relation between them.
The relation is maybe some equation of the form g of x, y, z equals some constant.
You take the relation between x, y, z, you call that g and that gives you the constraint.
And your goal is to minimize f only of those values of x, y, z that satisfy the constraint.
What is one way to do that?
Well, one to do that, if the constraint is very simple, we can maybe solve for one of the variables.
Maybe we can solve this equation for one of the variables, plug it back into f, and then we have a usual min/max problem that we have seen how to do.
The problem is sometimes you cannot actually solve for x, y, z in here because this condition is too complicated and then we need a new method.
That is what we are going to do.
Why would we care about that?
Well, one example is actually in physics.
Maybe you have seen in thermodynamics that you study quantities about gases, and those quantities that involve pressure, volume and temperature.
And pressure, volume and temperature are not independent of each other.
I mean you know probably the equation PV = NRT.
And, of course, there you could actually solve to express things in terms of one or the other.
But sometimes it is more convenient to keep all three variables but treat them as constrained.
It is just an example of a situation where you might want to do this.
Anyway, we will look mostly at particular examples, but just to point out that this is useful when you study guesses in physics.
The first observation is we cannot use our usual method of looking for critical points of f. Because critical points of f typically will not satisfy this condition and so won't be good solutions.
We need something else.
Let's look at an example, and we will see how that leads us to the method.
For example, let's say that I want to find the point closest to the origin -- -- on the hyperbola xy equals 3 in the plane.
That means I have this hyperbola, and I am asking myself what is the point on it that is the closest to the origin?
I mean we can solve this by elementary geometry, we don't need actually Lagrange multipliers, but we are going to do it with Lagrange multipliers because it is a pretty good example.
What does it mean?
Well, it means that we want to minimize distance to the origin.
What is the distance to the origin?
If I have a point, at coordinates (x, y) and then the distance to the origin is square root of x squared plus y squared.
Well, do we really want to minimize that or can we minimize something easier?
Yeah.
Maybe we can minimize the square of a distance.
Let's forget this guy and instead -- Actually, we will minimize f of x, y equals x squared plus y squared, that looks better, subject to the constraint xy = 3.
And so we will call this thing g of x, y to illustrate the general method.
Let's look at a picture.
Here you can see in yellow the hyperbola xy equals three.
And we are going to look for the points that are the closest to the origin.
What can we do?
Well, for example, we can plot the function x squared plus y squared, function f. That is the contour plot of f with a hyperbola on top of it.
Now let's see what we can do with that.
Well, let's ask ourselves, for example, if I look at points where f equals 20 now.
I think I am at 20 but you cannot really see it.
That is a circle with a point whose distant square is 20.
Well, can I find a solution if I am on the hyperbola?
Yes, there are four points at this distance.
Can I do better?
Well, let's decrease for distance.
Yes, we can still find points on the hyperbola and so on.
Except if we go too low then there are no points on this circle anymore in the hyperbola.
If we decrease the value of f that we want to look at that will somehow limit value beyond which we cannot go, and that is the minimum of f. We are trying to look for the smallest value of f that will actually be realized on the hyperbola.
When does that happen?
Well, I have to backtrack a little bit.
It seems like the limiting case is basically here.
It is when the circle is tangent to the hyperbola.
That is the smallest circle that will hit the hyperbola.
If I take a larger value of f, I will have solutions.
If I take a smaller value of f, I will not have any solutions anymore.
So, that is the situation that we want to solve for.
How do we find that minimum?
Well, a key observation that is valid on this picture, and that actually remain true in the completely general case, is that when we have a minimum the level curve of f is actually tangent to our hyperbola.
It is tangent to the set of points where x, y equals three, to the hyperbola.
Let's write that down.
We observe that at the minimum the level curve of f is tangent to the hyperbola.
Remember, the hyperbola is given by the equal g equals three, so it is a level curve of g. We have a level curve of f and a level curve of g that are tangent to each other.
And I claim that is going to be the general situation that we are interested in.
How do we try to solve for points where this happens?
How do we find x, y where the level curves of f and g are tangent to each other?
Let's think for a second.
If the two level curves are tangent to each other that means they have the same tangent line.
That means that the normal vectors should be parallel.
Let me maybe draw a picture here.
This is the level curve maybe f equals something.
And this is the level curve g equals constant.
Here my constant is three.
Well, if I look for gradient vectors, the gradient of f will be perpendicular to the level curve of f. The gradient of g will be perpendicular to the level curve of g. They don't have any reason to be of the same size, but they have to be parallel to each other.
Of course, they could also be parallel pointing in opposite directions.
But the key point is that when this happens the gradient of f is parallel to the gradient of g. Well, let's check that.
Here is a point.
And I can plot the gradient of f in blue.
The gradient of g in yellow.
And you see, in most of these places, somehow the two gradients are not really parallel.
Actually, I should not be looking at random points.
I should be looking only on the hyperbola.
I want points on the hyperbola where the two gradients are parallel.
Well, when does that happen?
Well, it looks like it will happen here.
When I am at a minimum, the two gradient vectors are parallel.
It is not really proof.
It is an example that seems to be convincing.
So far things work pretty well.
How do we decide if two vectors are parallel?
Well, they are parallel when they are proportional to each other.
You can write one of them as a constant times the other one, and that constant usually one uses the Greek letter lambda.
I don't know if you have seen it before.
It is the Greek letter for L. And probably, I am sure, it is somebody's idea of paying tribute to Lagrange by putting an L in there.
Lambda is just a constant.
And we are looking for a scalar lambda and points x and y where this holds.
In fact, what we are doing is replacing min/max problems in two variables with a constraint between them by a set of equations involving, you will see, three variables.
We had min/max with two variables x, y, but no independent.
We had a constraint g of x, y equals constant.
And that becomes something new.
That becomes a system of equations where we have to solve, well, let's write down what it means for gradient f to be proportional to gradient g. That means that f sub x should be lambda times g sub x, and f sub y should be lambda times g sub y.
Because the gradient vectors here are f sub x, f sub y and g sub x, g sub y.
If you have a third variable z then you have also an equation f sub z equals lambda g sub z.
Now, let's see.
How many unknowns do we have in these equations?
Well, there is x, there is y and there is lambda.
We have three unknowns and have only two equations.
Something is missing.
Well, I mean x and y are not actually independent.
They are related by the equation g of x, y equals c, so we need to add the constraint g equals c. And now we have three equations involving three variables.
Let's see how that works.
Here remember we have f equals x squared y squared and g = xy.
What is f sub x?
It is going to be 2x equals lambda times, what is g sub x, y.
Maybe I should write here f sub x equals lambda g sub x just to remind you.
Then we have f sub y equals lambda g sub y. F sub y is 2y equals lambda times g sub y is x.
And then our third equation g equals c becomes xy equals three.
So, that is what you would have to solve.
Any questions at this point?
No.
Yes?
How do I know the direction of a gradient?
Do you mean how do I know that it is perpendicular to a level curve?
Oh, how do I know if it points in that direction on the opposite one?
Well, that depends.
I mean we'd seen in last time, but the gradient is perpendicular to the level and points towards higher values of a function.
So it could be -- Wait.
What did I have?
It could be that my gradient vectors up there actually point in opposite directions.
It doesn't matter to me because it will still look the same in terms of the equation, just lambda will be positive or negative, depending on the case.
I can handle both situations.
It's not a problem.
I can allow lambda to be positive or negative.
Well, in this example, it looks like lambda will be positive.
If you look at the picture on the plot.
Yes?
Well, because actually they are not equal to each other.
If you look at this point where the hyperbola and the circle touch each other, first of all, I don't know which circle I am going to look at.
I am trying to solve, actually, for the radius of the circle.
I am trying to find what the minimum value of f is.
And, second, at that point, the value of f and the value of g are not equal.
g is equal to three because I want the hyperbola x equals three.
The value of f will be the square of a distance, whatever that is.
I think it will end up being 6, but we will see.
So, you cannot really set them equal because you don't know what f is equal to in advance.
Yes?
Not quite.
Actually, here I am just using this idea of finding a point closest to the origin to illustrate an example of a min/max problem.
The general problem we are trying to solve is minimize f subject to g equals constant.
And what we are going to do for that is we are really going to say instead let's look at places where gradient f and gradient g are parallel to each other and solve for equations of that.
I think we completely lose the notion of closest point if we just look at these equations.
We don't really say anything about closest points anymore.
Of course, that is what they mean in the end.
But, in the general setting, there is no closest point involved anymore.
OK.
Yes?
Yes.
It is always going to be the case that, at the minimum, or at the maximum of a function subject to a constraint, the level curves of f and the level curves of g will be tangent to each other.
That is the basis for this method.
I am going to justify that soon.
It could be minimum or maximum.
In three-dimensions it could even be a saddle point.
And, in fact, I should say in advance, this method will not tell us whether it is a minimum or a maximum.
We do not have any way of knowing, except for testing values.
We cannot use second derivative tests or anything like that.
I will get back to that.
Yes?
Yes.
Here you can set y equals to favor x.
Then you can minimize x squared plus nine over x squared.
In general, if I am trying to solve a more complicated problem, I might not be able to solve.
I am doing an example where, indeed, here you could solve and remove one variable, but you cannot always do that.
And this method will still work.
The other one won't.
OK.
I don't see any other questions.
Are there any other questions?
No.
OK.
I see a lot of students stretching and so on, so it is very confusing for me.
How do we solve these equations?
Well, the answer is in general we might be in deep trouble.
There is no general method for solving the equations that you get from this method.
You just have to think about them.
Sometimes it will be very easy.
Sometimes it will be so hard that you cannot actually do it without the computer.
Sometimes it will be just hard enough to be on Part B of this week's problem set.
I claim in this case we can actually do it without so much trouble, because actually we can think of this as a two by two linear system in x and y.
Well, let me do something.
Let me rewrite the first two equations as 2x - lambda y = 0.
And lambda x - 2y = 0.
And xy = 3.
That is what we want to solve.
Well, I can put this into matrix form.
Two minus lambda, lambda minus two times x, y equals 0,0.
Now, how do I solve a linear system matrix times x, y equals zero?
Well, I always have an obvious solution.
X and y both equal to zero.
Is that a good solution?
No, because zero times zero is not three.
We want another solution, the trivial solution.
0,0 does not solve the constraint equation xy equals three, so we want another solution.
When do we have another solution?
Well, when the determinant of a matrix is zero.
We have other solutions that exist only if determinant of a matrix is zero.
M is this guy.
Let's compute the determinant.
Well, that seems to be negative four plus lambda squared.
That is zero exactly when lambda squared equals four, which is lambda is plus or minus two.
Already you see here it is a the level of difficulty that is a little bit much for an exam but perfectly fine for a problem set or for a beautiful lecture like this one.
How do we deal with -- Well, we have two cases to look at.
Lambda equals two or lambda equals minus two.
Let's start with lambda equals two.
If I set lambda equals two, what does this equation become?
Well, it becomes x equals y.
This one becomes y equals x.
Well, they seem to be the same.
x equals y.
And then the equation xy equals three becomes, well, x squared equals three.
I have two solutions.
One is x equals root three and, therefore, y equals root three as well, or negative root three and negative root three.
Let's look at the other case.
If I set lambda equal to negative two then I get 2x equals negative 2y.
That means x equals negative y.
The second one, 2y equals negative 2x.
That is y equals negative x.
Well, that is the same thing.
And xy equals three becomes negative x squared equals three.
Can we solve that?
No.
There are no solutions here.
Now we have two candidate points which are these two points, root three, root three or negative root three, negative root three.
OK. Let's actually look at what we have here.
Maybe you cannot read the coordinates, but the point that I have here is indeed root three, root three.
How do we see that lambda equals two?
Well, if you look at this picture, the gradient of f, that is the blue vector, is indeed twice the yellow vector, gradient g. That is where you read the value of lambda.
And we have the other solution which is somewhere here.
Negative root three, negative root there.
And there, again, lambda equals two.
The two vectors are proportional by a factor of two.
Yes?
No, solutions are not quite guaranteed to be absolute minima or maxima.
They are guaranteed to be somehow critical points end of a constraint.
That means if you were able to solve and eliminate the variable that would be a critical point.
When you have the same problem, as we have critical points, are they maxima or minima?
And the answer is, well, we won't know until we check.
More questions?
No.
Yes?
What is a Lagrange multiplier?
Well, it is this number lambda that is called the multiplier here.
It is a multiplier because it is what you have to multiply gradient of g by to get gradient of f. It multiplies.
Let's try to see why is this method valid?
Because so far I have shown you pictures and have said see they are tangent.
But why is it that they have to be tangent in general?
Let's think about it.
Let's say that we are at constrained min or max.
What that means is that if I move on the level g equals constant then the value of f should only increase or only decrease.
But it means, in particular, to first order it will not change.
At an unconstrained min or max, partial derivatives are zero.
In this case, derivatives are zero only in the allowed directions.
And the allowed directions are those that stay on the levels of this g equals constant.
In any direction along the level set g = c the rate of change of f must be zero.
That is what happens at minima or maxima.
Except here, of course, we look only at the allowed directions.
Let's say the same thing in terms of directional derivatives.
That means for any direction that is tangent to the constraint level g equal c, we must have df over ds in the direction of u equals zero.
I will draw a picture.
Let's say now I am in three variables just to give you different examples.
Here I have a level surface g equals c. I am at my point.
And if I move in any direction that is on the level surface, so I move in the direction u tangent to the level surface, then the rate of change of f in that direction should be zero.
Now, remember what the formula is for this guy.
Well, we have seen that this guy is actually radiant f dot u.
That means any such vector u must be perpendicular to the gradient of f. That means that the gradient of f should be perpendicular to anything that is tangent to this level.
That means the gradient of f should be perpendicular to the level set.
That is what we have shown.
But we know another vector that is also perpendicular to the level set of g. That is the gradient of g. We conclude that the gradient of f must be parallel to the gradient of g because both are perpendicular to the level set of g. I see confused faces, so let me try to tell you again where that comes from.
We said if we had a constrained minimum or maximum, if we move in the level set of g, f doesn't change.
Well, it doesn't change to first order.
It is the same idea as when you are looking for a minimum you set the derivative equal to zero.
So the derivative in any direction, tangent to g equals c, should be the directional derivative of f, in any such direction, should be zero.
That is what we mean by critical point of f. And so that means that any vector u, any unit vector tangent to the level set of g is going to be perpendicular to the gradient of f. That means that the gradient of f is perpendicular to the level set of g. If you want, that means the level sets of f and g are tangent to each other.
That is justifying what we have observed in the picture that the two level sets have to be tangent to each other at the prime minimum or maximum.
Does that make a little bit of sense?
Kind of.
I see at least a few faces nodding so I take that to be a positive answer.
Since I have been asked by several of you, how do I know if it is a maximum or a minimum?
Well, warning, the method doesn't tell whether a solution is a minimum or a maximum.
How do we do it?
Well, more bad news.
We cannot use the second derivative test.
And the reason for that is that we care actually only about these specific directions that are tangent to variable of g. And we don't want to bother to try to define directional second derivatives.
Not to mention that actually it wouldn't work.
There is a criterion but it is much more complicated than that.
Basically, the answer for us is that we don't have a second derivative test in this situation.
What are we left with?
Well, we are just left with comparing values.
Say that in this problem you found a point where f equals three, a point where f equals nine, a point where f equals 15.
Well, then probably the minimum is the point where f equals three and the maximum is 15.
Actually, in this case, where we found minima, these two points are tied for minimum.
What about the maximum?
What is the maximum of f on the hyperbola?
Well, it is infinity because the point can go as far as you want from the origin.
But the general idea is if we have a good reason to believe that there should be a minimum, and it's not like at infinity or something weird like that, then the minimum will be a solution of the Lagrange multiplier equations.
We just look for all the solutions and then we choose the one that gives us the lowest value.
Is that good enough?
Let me actually write that down.
To find the minimum or the maximum, we compare values of f at the various solutions -- -- to Lagrange multiplier equations.
I should say also that sometimes you can just conclude by thinking geometrically.
In this case, when it is asking you which point is closest to the origin you can just see that your answer is the correct one.
Let's do an advanced example.
Advanced means that -- Well, this one I didn't actually dare to put on top of the other problem sets.
Instead, I am going to do it.
What is this going to be about?
We are going to look for a surface minimizing pyramid.
Let's say that we want to build a pyramid with a given triangular base -- -- and a given volume.
Say that I have maybe in the x, y plane I am giving you some triangle.
And I am going to try to build a pyramid.
Of course, I can choose where to put the top of a pyramid.
This guy will end up being behind now.
And the constraint and the goal is to minimize the total surface area.
The first time I taught this class, it was a few years ago, was just before they built the Stata Center.
And then I used to motivate this problem by saying Frank Gehry has gone crazy and has been given a triangular plot of land he wants to put a pyramid.
There needs to be the right amount of volume so that you can put all the offices in there.
And he wants it to be, actually, covered in solid gold.
And because that is expensive, the administration wants him to cut the costs a bit.
And so you have to minimize the total size so that it doesn't cost too much.
We will see if MIT comes up with a triangular pyramid building.
Hopefully not.
It could be our next dorm, you never know.
Anyway, it is a fine geometry problem.
Let's try to think about how we can do this.
The natural way to think about it would be -- Well, what do we have to look for first?
We have to look for the position of that top point.
Remember we know that the volume of a pyramid is one-third the area of base times height.
In fact, fixing the volume, knowing that we have fixed the area of a base, means that we are fixing the height of the pyramid.
The height is completely fixed.
What we have to choose just is where do we put that top point?
Do we put it smack in the middle of a triangle or to a side or even anywhere we want?
Its z coordinate is fixed.
Let's call h the height.
What we could do is something like this.
We say we have three points of a base.
Let's call them p1 at (x1, y1,0); p2 at (x2, y2,0); p3 at (x3, y3,0).
This point p is the unknown point at (x, y, h).
We know the height.
And then we want to minimize the sum of the areas of these three triangles.
One here, one here and one at the back.
And areas of triangles we know how to express by using length of cross-product.
It becomes a function of x and y.
And you can try to minimize it.
Actually, it doesn't quite work.
The formulas are just too complicated.
You will never get there.
What happens is actually maybe we need better coordinates.
Why do we need better coordinates?
That is because the geometry is kind of difficult to do if you use x, y coordinates.
I mean formula for cross-product is fine, but then the length of the vector will be annoying and just doesn't look good.
Instead, let's think about it differently.
I claim if we do it this way and we express the area as a function of x, y, well, actually we can't solve for a minimum.
Here is another way to do it.
Well, what has worked pretty well for us so far is this geometric idea of base times height.
So let's think in terms of the heights of side triangles.
I am going to use the height of these things.
And I am going to say that the area will be the sum of three terms, which are three bases times three heights.
Let's give names to these quantities.
Actually, for that it is going to be good to have the point in the xy plane that lives directly below p. Let's call it q. P is the point that coordinates x, y, h. And let's call q the point that is just below it and so it' coordinates are x, y, 0.
Let's see.
Let me draw a map of this thing.
p1, p2, p3 and I have my point q in the middle.
Let's see.
To know these areas, I need to know the base.
Well, the base I can decide that I know it because it is part of my given data.
I know the sides of this triangle.
Let me call the lengths a1, a2, a3.
I also need to know the height, so I need to know these lengths.
How do I know these lengths?
Well, its distance in space, but it is a little bit annoying.
But maybe I can reduce it to a distance in the plane by looking instead at this distance here.
Let me give names to the distances from q to the sides.
Let's call u1, u2, u3 the distances from q to the sides.
Well, now I can claim I can find, actually, sorry.
I need to draw one more thing.
I claim I have a nice formula for the area, because this is vertical and this is horizontal so this length here is u3, this length here is h. So what is this length here?
It is the square root of u3 squared plus h squared.
And similarly for these other guys.
They are square roots of a u squared plus h squared.
The heights of the faces are square root of u1 squared times h squared.
And similarly with u2 and u3.
So the total side area is going to be the area of the first faces, one-half of base times height, plus one-half of a base times a height plus one-half of the third one.
It doesn't look so much better.
But, trust me, it will get better.
Now, that is a function of three variables, u1, u2, u3.
And how do we relate u1, u2, u3 to each other?
They are probably not independent.
Well, let's cut this triangle here into three pieces like that.
Then each piece has side -- Well, let's look at it the piece of the bottom.
It has base a3, height u3.
Cutting base into three tells you that the area of a base is one-half of a1, u1 plus one-half of a2, u2 plus one-half of a3, u3.
And that is our constraint.
My three variables, u1, u2, u3, are constrained in this way.
The sum of this figure must be the area of a base.
And I want to minimize that guy.
So that is my g and that guy here is my f. Now we try to apply our Lagrange multiplier equations.
Well, partial f of a partial u1 is -- Well, if you do the calculation, you will see it is one-half a1, u1 over square root of u1^2 plus h^2 equals lambda, what is partial g, partial a1?
That one you can do, I am sure.
It is one-half a1.
Oh, these guys simplify.
If you do the same with the second one -- -- things simplify again.
And the same with the third one.
Well, you will get, after simplifying, u3 over square root of u3 squared plus h squared equals lambda.
Now, that means this guy equals this guy equals this guy.
They are all equal to lambda.
And, if you think about it, that means that u1 = u2 = u3.
See, it looked like scary equations but the solution is very simple.
What does it mean?
It means that our point q should be equidistant from all three sides.
That is called the incenter.
Q should be in the incenter.
The next time you have to build a golden pyramid and don't want to go broke, well, you know where to put the top.
If that was a bit fast, sorry.
Anyway, it is not completely crucial.
But go over it and you will see it works.
Have a nice weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so we're going to continue looking at what happens when we have non-independent variables.
So, I'm afraid we don't take deliveries during class time, sorry.
Please take a seat, thanks.
[LAUGHTER] [APPLAUSE] OK, so Jason, you please claim your package at the end of lecture.
OK, so last time we saw how to use Lagrange multipliers to find the minimum or maximum of a function of several variables when the variables are not independent.
And, today we're going to try to figure out more about relations between the variables, and how to handle functions that depend on several variables when they're related.
So, just to give you an example, in physics, very often, you have functions that depend on pressure, volume, and temperature where pressure, volume, and temperature are actually not independent.
But they are related, say, by PV=nRT.
So, of course, then you can substitute and expressed a function in terms of two of them only, but very often it's convenient to keep all three.
But then we have to figure out, what are the rates of change with respect to t, with respect to each other, the rate of change of f with respect to these variables, and so on.
So, we have to figure out what we mean by partial derivatives again.
So, OK, more generally, let's say just for the sake of notation, I'm going to think of a function of three variables, x, y, z, where the variables are related by some equation, but I will put in the form g of x, y, z equals some constant.
OK, so that's the same kind of setup as we had last time, except now we are not just looking for minima and maxima.
We are trying to understand partial derivatives.
So, the first observation is that if x, y, and z are related, then that means, in principle, we could solve for one of them, and express it as a function of the two others.
So, in particular, can we understand even without solving?
Maybe we can not solve.
Can we understand how the variables are related to each other?
So, for example, z, you can think of z as a function of x and y.
So, we can ask ourselves, what are the rates of change of z with respect to x, keeping y constant, or with respect to y keeping x constant?
And, of course, if we can solve, that we know the formula for this.
And then we can compute these guys.
But, what if we can't solve?
So, how do we find these things without solving?
Well, so let's do an example.
Let's say that my relation is x^2 yz z^3=8.
And, let's say that I'm looking near the point (x, y, z) equals (2,3, 1).
So, let me check 2^2 plus three times one plus 1^3 is indeed eight.
OK, but now, if I change x and y a little bit, how does z change?
Well, of course I could solve for z in here.
It's a cubic equation.
There is actually a formula.
But that formula is quite complicated.
We actually don't want to do that.
There's an easier way.
So, how can we do it?
Well, let's look at the differential -- -- of this constraint quantity.
OK, so if we called this g, let's look at dg.
So, what's the differential of this?
So, the differential of x^2 is 2x dx plus, I think there's a zdy.
There's a ydz, and there's also a 3z^2 dz.
OK, you can get this either by implicit differentiation and the product rule, or you could get this just by putting here, here, and here the partial derivatives of this with respect to x, y, and z. OK, any questions about how I got this?
No?
OK.
So, now, what do I do with this?
Well, this represents, somehow, variations of g. But, well, I've set this thing equal to eight.
And, eight is a constant.
So, it doesn't change.
So, in fact, well, we can set this to zero because, well, they call this g. Then, g equals eight is constant.
That means we set dg equal to zero.
OK, so, now let's just plug in some values at this point.
That tells us, well, so if x equals two, that's 4dx plus z is one.
So, dy plus y 3z^2 should be 6dz equals zero.
And now, this equation, here, tells us about a relation between the changes in x, y, and z near that point.
It tells us how you change x and y, well, how z will change.
Or, it tells you actually anything you might want to know about the relations between these variables so, for example, you can move dz to that side, and then express dz in terms of dx and dy.
Or, you can move dy to that side and express dy in terms of dx and dz, and so on.
It tells you at the level of the derivatives how each of the variables depends on the two others.
OK, so, just to clarify this: if we want to view z as a function of x and y, then what we will do is we will just move the dz's to the other side, and it will tell us dz equals minus one over six times 4dx plus dy.
And, so that should tell you that partial z over partial x is minus four over six.
Well, that's minus two thirds, and partial z over partial y is going to be minus one sixth.
OK, another way to think about this: when we compute partial z over partial x, that means that actually we keep y constant.
OK, let me actually add some subtitles here.
So, here that means we keep y constant.
And so, if we keep y constant, another way to think about it is we set dy to zero.
We set dy equals zero.
So if we do that, we get dx equals negative four sixths dx.
That tells us the rate of change of z with respect to x.
Here, we set x constant.
So, that means we set dx equal to zero.
And, if we set dx equal to zero, then we have dz equals negative one sixth of dy.
That tells us the rate of change of z with respect to y. OK, any questions about that?
No?
What, yes?
Yes, OK, let me explain that again.
So we found an expression for dz in terms of dx and dy.
That means that this thing, the differential, is the total differential of z viewed as a function of x and y. OK, and so the coefficients of dx and dy are the partials.
Or, another way to think about it, if you want to know partial z partial x, it means you set y to be constant.
Setting y to be constant means that you will put zero in the place of dy.
So, you will be left with dz equals minus four sixths dx.
And, that will give you the rate of change of z with respect to x when you keep y constant, OK?
So, there are various ways to think about this, but hopefully it makes sense.
OK, so how do we think about this in general?
Well, if we know that g of x, y, z equals a constant, then dg, which is gxdx gydy gzdz should be set equal to zero.
OK, and now we can solve for whichever variable we want to express in terms of the others.
So, for example, if we care about z as a function of x and y -- -- we'll get that dz is negative gx over gz dx minus gy over gz dy.
And, so if we want partial z over partial x, well, so one way is just to say that's going to be the coefficient of dx in here, or just to write down the other way.
We are setting y equals constant.
So, that means we set dy equal to zero.
And then, we get dz equals negative gx over gz dx.
So, that means partial z over partial x is minus gx over gz.
And, see, that's a very counterintuitive formula because you have this minus sign that you somehow probably couldn't have seen come if you hadn't actually derived things this way.
I mean, it's pretty surprising to see that minus sign come out of nowhere the first time you see it.
OK, so now we know how to find the rate of change of constrained variables with respect to each other.
You can apply the same to find, if you want partial x, partial y, or any of them, you can do it.
Any questions so far?
No?
OK, so, before we proceed further, I should probably expose some problem with the notations that we have so far.
So, let me try to get you a bit confused, OK?
So, let's take a very simple example.
Let's say I have a function, f of x, y equals x y. OK, so far it doesn't sound very confusing.
And then, I can write partial f over partial x.
And, I think you all know how to compute it.
It's going to be just one.
OK, so far we are pretty happy.
Now let's do a change of variables.
Let's set x=u and y=u v. It's not very complicated change of variables.
But let's do it.
Then, f in terms of u and v, well, so f, remember f was x y becomes u plus u plus v. That's twice u plus v. What's partial f over partial u?
It's two.
So, x and u are the same thing.
Partial f over partial x, and partial f over partial u, well, unless you believe that one equals two, they are really not the same thing, OK?
So, that's an interesting, slightly strange phenomenon.
x equals u, but partial f partial x is not the same as partial f partial u.
So, how do we get rid of this contradiction?
Well, we have to think a bit more about what these notations mean, OK?
So, when we write partial f over partial x, it means that we are varying x, keeping y constant.
When we write partial f over partial u, it means we are varying u, keeping v constant.
So, varying u or varying x is the same thing.
But, keeping v constant, or keeping y constant are not the same thing.
If I keep y constant, then when I change x, so when I change u, then v will also have to change so that their sum stays the same.
Or, if you prefer the other way around, when I do this one I keep v constant.
If I keep v constant and I change u, then y will change.
It won't be constant.
So, that means, well, life looked quite nice and easy with these notations.
But, what's dangerous about them is they are not making explicit what it is exactly that we are keeping constant.
OK, so just to write things, so here we change u and x that are the same thing.
But we keep y constant, while here we change u, which is still the same thing as x.
But, what we keep constant is v, or in terms of x and y, that's y minus x constant.
And, that's why they are not the same.
So, whenever there's any risk of confusion, OK, so not in the cases that we had before because what we've done until now, we didn't really have a problem.
But, in a situation like this, to clarify things, we'll actually say explicitly what it is that we want to keep constant.
OK, so what's going to be our new notation?
Well, so it's not particularly pleasant because it uses, now, a subscript not to indicate what you are differentiating, but rather what you were holding constant.
So, that's quite a conflict of notation with what we had before.
I think I can safely blame it on physicists or chemists.
OK, so this one means we keep y constant, and partial f over partial u with v held constant, similarly.
OK, so now what happens is we no longer have any contradiction.
We have partial f over partial x with y constant is different from partial f over partial x with v constant, which is the same as partial f over partial u with v constant.
OK, so this guy is one.
And these guys are two.
So, now we can safely use the fact that x equals u if we are keeping track of what is actually held constant, OK?
So now, that's going to be particularly important when we have variables that are related because, let's say now that I have a function that depends on x, y, and z.
But, x, y, and z are related.
Then, it means that I look at, say, x and y as my independent variables, and z as a function of x and y.
Then, it means that when I do partials, say, with respect to x, I will hold y constant.
But, I will let z vary as a function of x and y.
Or, I could do it the other way around.
I could vary x, keep z constant, and let y be a function of x and z.
And so, I will need to use this kind of notation to indicate which one I mean.
OK, any questions?
No?
All right, so let's try to do an example where we have a function that depends on variables that are related.
OK, so I don't want to do one with PV=nRT because probably, I mean, if you've seen it, then you've seen too much of it.
And, if you haven't seen it, then maybe it's not the best example.
So, let's do a geometric example.
So, let's look at the area of the triangle.
So, let's say I have a triangle, and my variables will be the sides a and b.
And the angle here, theta.
OK, so what's the area of this triangle?
Well, its base times height over two.
So, it's one half of the base is a, and the height is b sine theta.
OK, so that's a function of a, b, and theta.
Now, let's say, actually, there is a relation between a, b, and theta that I didn't tell you about, namely, actually, I want to assume that it's a right triangle, OK?
So, let's now assume it's a right triangle with, let's say, the hypotenuse is b.
So, we have the right angle here, actually.
So, a is here.
b is here.
Theta is here.
So, saying it's a right triangle is the same thing as saying that b equals sine theta, OK?
So that's our constraint.
That's the relation between a, b, and theta.
And, this is a function of a, b, and theta.
And, let's say that we want to understand how the area depends on theta.
OK, what's the rate of change of the area of this triangle with respect to theta?
So, I claim there's various answers.
I can think of at least three possible answers.
So, what can we possibly mean by the rate of change of A with respect to theta?
So, these are all things that we might want to call partial A partial theta.
But of course, we'll have to actually use different notations to distinguish them.
So, the first way that we actually already know about is if we just forget about the fact that the variables are related, OK?
So, if we just think of little a, b, and theta as independent variables, and we just change theta, keeping a and b constant -- So, that's exactly what we meant by partial A, partial theta, right?
I'm not putting any constraints.
So, just to use some new notation, that would be the rate of change of A with respect to theta, keeping a and b fixed at the same time.
Of course, if we are keeping a and b fixed, and we are changing theta, it means we completely ignore this property of being a right triangle.
So, in fact, it corresponds to changing the area by changing the angle, keeping these lengths fixed.
And, of course, we lose the right angle.
When we rotate this side here, but the angle doesn't stay at a right angle.
And that one, we know how to compute, right, because it's the one we've been computing all along.
So, that means we keep a and b fixed.
And then, so let's see, what's the derivatives of A with respect to theta?
It's one half ab cosine theta.
OK, now that one we know.
Any questions?
No?
OK, the two other guys will be more interesting.
So far, I'm not really doing anything with my constraint.
Let's say that actually I do want to keep the right angle.
Then, when I change theta, there's two options.
One is I keep a constant, and then of course b will have to change because if this width stays the same, then when I change theta, the height increases, and then this side length increases.
The other option is to change the angle, keeping b constant.
So, actually, this side stays the same length.
But then, a has to become a bit shorter.
And, of course, the area will change in different ways depending on what I do.
So, that's why I said we have three different answers.
So, the next one is keep, I forgot which one I said first.
Let's say keep a constant.
And, that means that b will change.
b is going to be some function of a and theta.
Well, in fact here, we know what the function is because we can solve the constraint, namely, b is a over cosine theta.
But we don't actually need to know that so that the triangle, so that the right angle, so that we keep a right angle.
And, so the name we will have for this is partial a over partial theta with a held constant, OK?
And, the fact that I'm not putting b in my subscript there means that actually b will be a dependent variable.
It changes in whatever way it has to change so that when theta changes, a stays the same while b changes so that we keep a right triangle.
And, the third guy is the one where we actually keep b constant, and now a, we think a as a function of b and theta, and it changes so that we keep the right angle.
So actually as a function of b and theta, it's given over there.
A equals b cosine theta.
And so, this guy is called partial a over partial theta with b held constant.
OK, so we've just defined them.
We don't know yet how to compute these things.
That's what we're going to do now.
That is the definition, and what these things mean.
Is that clear to everyone?
Yes, OK.
Yes?
OK, so the second answer, again, so one way to ask ourselves, how does the area depend on theta, is to say, well, actually look at the area of the right triangle as a function of a and theta only by solving for b.
And then, we'll change theta, keep a constant, and ask, how does the area change?
So, when we do that, when we change theta and keep a the same, then b has to change so that it stays a right triangle, right, so that this relation still holds.
That requires us to change b.
So, when we write partial a over partial theta with a constant, it means that, actually, b will be the dependent variable.
It depends on a and theta.
And so, the area depends on theta, not only because theta is in the formula, but also because b changes, and b is in the formula.
Yes?
No, no, we don't keep theta constant.
We vary theta, right?
The goal is to see how things change when I change theta by a little bit.
OK, so if I change theta a little bit in this one, if I change theta a little bit and I keep a the same, then b has to change also in some way.
There's a right triangle.
And then, because theta and b change, that causes the area to change.
OK, so maybe I should re-explain that again.
So, theta changes.
A is constant.
But, we have the constraint, a equals be plus sine theta.
That means that b changes.
And then, the question is, how does A change?
Well, it will change in part because theta changes, and in part because b changes.
But, we want to know how it depends on theta in this situation.
Yes?
Ah, that's a very good question.
So, what about, I don't keep a and b constant?
Well, then there's too many choices.
So I have to decide actually how I'm going to change things.
See, if I just say I have this relation, that means I have two independent variables left, whichever two of the three I want.
But, I still have to specify two of them to say exactly which triangle I mean.
So, I cannot ask myself just how will it change if I change theta and do random things with a and b?
It depends what I do with a and b.
Of course, I could choose to change them simultaneously, but then I have to specify how exactly I'm going to do that.
Ah, yes, if you wanted to, indeed, we could also change things in such a way that the third side remains constant.
And that would be, yet, a different way to attack the problem.
I mean, we don't have good notation for this, here, because we didn't give it a name.
But, yeah, I mean, we could.
We could call this guy c, and then we'd have a different formula, and so on.
So, I mean, I'm not looking at it for simplicity.
But, you could have many more.
I mean, in general, you will want, once you have a set of nice, natural variables, you will want to look mostly at situations where one of the variables changes.
Some of them are held fixed, and then some dependent variable does whatever it must so that the constraint keeps holding.
OK, so let's try to compute one of them.
Let's say I decide that we will compute this one.
OK, let's see how we can compute partial a, partial theta with a held fixed.
[APPLAUSE] OK, so let's try to compute partial A, partial theta with a held constant.
So, let's see three different ways of doing that.
So, let me start with method zero.
OK, it's not a real method.
That's why I'm not getting a positive number.
So, that one is just, we solve for b, and we remove b from the formulas.
OK, so here it works well because we know how to solve for b.
But I'm not considering this to be a real method because in general we don't know how to do that.
I mean, in the beginning I had this relation that was an equation of degree three.
You don't really want to solve your equation for the dependent variable usually.
Here, we can.
So, solve for b and substitute.
So, how do we do that?
Well, the constraint is a=b cosine theta.
That means b is a over cosine theta.
Some of you know that as a secan theta.
That's the same.
And now, if we express the area in terms of a and theta only, A is one half of ab cosine, sorry, ab sine theta is now one half of a^2 sine theta over cosine theta.
Or, if you prefer, one half of a^2 tangent theta.
Well, now that it's only a function of a and theta, I know what it means to take the partial derivative with respect to theta, keeping a constant.
I know how to do it.
So, partial A over partial theta, a held constant, well, if a is a constant, then I get this one half a^2 coming out times, what's the derivative of tangent?
Secan squared, very good.
If you're European and you've never heard of secan, that's one over cosine.
And, if you know the derivative as one plus tangent squared, that's the same thing.
And, it's also correct.
OK, so, that's one way of doing it.
But, as I've already said, it doesn't get us very far if we don't know how to solve for b.
We really used the fact that we could solve for b and get rid of it.
So, there's two systematic methods, and let's say the basic rule is that you should give both of them a chance.
You should see which one you prefer, and you should be able to use one or the other on the exam.
OK, most likely you'll actually have a choice between one or the other.
It will be up to you to decide which one you want to use.
But, you cannot use solving in substitution.
That's not fair.
OK, so the first one is to use differentials.
By the way, in the notes they are called also method one and method two.
I'm not promising that I have the same one, am I?
I mean, I might have one and two switched.
It doesn't really matter.
So, how do we do things using differentials?
Well, first, we know that we want to keep a fixed, and that means that we'll set da equal to zero, OK?
The second thing that we want to do is we want to look at the constraint.
The constraint is a equals b cosine theta.
And, we want to differentiate that.
Well, differentiate the left-hand side.
You get da.
And, differentiate the right-hand side as a function of b and theta.
You should get, well, how many db's?
Well, that's the rate of change with respect to b.
That's cosine theta db minus b sine theta d theta.
That's a product rule applied to b times cosine theta.
So -- Well, now, if we have a constraint that's relating da, db, and d theta, OK, so that's actually what we did, right, that's the same sort of thing as what we did at the beginning when we related dx, dy, and dz.
That's really the same thing, except now are variables are a, b, and theta.
Now, we know that also we are keeping a fixed.
So actually, we set this equal to zero.
So, we have zero equals da equals cosine theta db minus b sine theta d theta.
That means that actually we know how to solve for db.
OK, so cosine theta db equals b sine theta d theta or db is b tangent theta d theta.
OK, so in fact, what we found, if you want, is the rate of change of b with respect to theta.
Why do we care?
Well, we care because let's look, now, at dA, the function that we want to look at.
OK, so the function is A equals one half ab sine theta.
Well, then, dA, so we had to use the product rule carefully, or we use the partials.
So, the coefficient of d little a will be partial with respect to little a.
That's one half b sine theta da plus coefficient of db will be one half a sine theta db plus coefficient of d theta will be one half ab cosine theta d theta.
But now, what do I do with that?
Well, first I said a is constant.
So, da is zero.
Second, well, actually we don't like b at all, right?
We want to view a as a function of theta.
So, well, maybe we actually want to use this formula for db that we found in here.
OK, and then we'll be left only with d thetas, which is what we want.
So, if we plug this one into that one, we get da equals one half a sine theta times b tangent theta d theta plus one half ab cosine theta d theta.
And, if we collect these things together, we get one half of ab times sine theta times tangent theta plus cosine theta d theta.
And, if you know your trig, but you'll see that this is sine squared over cosine plus cosine squared over cosine.
That's the same as secan theta.
So, now you have expressed da as something times d theta.
Well, that coefficient is the rate of change of A with respect to theta with the understanding that we are keeping a fixed, and letting b vary as a dependent variable.
Not enough space: sorry.
OK, in case it's clearer for you, let's think about it backwards.
So, we wanted to find how A changes.
To find how A changes, we write da.
But now, this tells us how A depends on little a, little b, and theta.
Well, we know actually we want to keep little a constant.
So, we set this to be zero.
Theta, well, we are very happy because we want to express things in terms of theta.
Db we want to get rid of.
How do we get rid of db?
Well, we do that by figuring out how b depends on theta when a is fixed.
And, we do that by differentiating the constraint equation, and setting da equal to zero.
OK, so -- I guess to summarize the method, we wrote dA in terms of da, db, d theta.
Then, we say that a is constant means we set da equals zero.
And, the third thing is that because, well, we differentiate the constraint.
And, we can solve for db in terms of d theta.
And then, we plug into dA, and we get the answer.
OK, oops.
So, here's another method to do the same thing differently is to use the chain rule.
So, we can use the chain rule with dependent variables, OK?
So, what does the chain rule tell us?
The chain rule tells us, so we will want to differentiate -- -- the formula for a with respect to theta holding a constant.
So, I claim, well, what does the chain rule tell us?
It tells us that, well, when we change things, a changes because of the changes in the variables.
So, part of it is that A depends on theta and theta changes.
How fast does theta change?
Well, you could call that the rate of change of theta with respect to theta with a constant.
But of course, how fast does theta depend to itself?
The answer is one.
So, that's pretty easy.
Plus, then we have the partial derivative, formal partial derivative, of A with respect to little a times the rate of change of a in our situation.
Well, how does little a change if a is constant?
Well, it doesn't change.
And then, there is Ab, the formal partial derivative times, sorry, the rate of change of b. OK, and how do we find this one?
Well, here we have to use the constraint.
OK, and we can find this one from the constraint as we've seen at the beginning either by differentiating the constraint, or by using the chain rule on the constraint.
So, of course the calculations are exactly the same.
See, this is the same formula as the one over there, just dividing everything by partial theta and with subscripts little a.
But, if it's easier to think about it this way, then that's also valid.
OK, so tomorrow we are going to review for the test, so I'm going to tell you a bit more about this also as we go over one practice problem on that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Let me start by basically listing the main things we have learned over the past three weeks or so.
And I will add a few complements of information about that because there are a few small details that I didn't quite clarify and that I should probably make a bit clearer, especially what happened at the very end of yesterday's class.
Here is a list of things that should be on your review sheet for the exam.
The first thing we learned about, the main topic of this unit is about functions of several variables.
We have learned how to think of functions of two or three variables in terms of plotting them.
In particular, well, not only the graph but also the contour plot and how to read a contour plot.
And we have learned how to study variations of these functions using partial derivatives.
Remember, we have defined the partial of f with respect to some variable, say, x to be the rate of change with respect to x when we hold all the other variables constant.
If you have a function of x and y, this symbol means you differentiate with respect to x treating y as a constant.
And we have learned how to package partial derivatives into a vector,the gradient vector.
For example, if we have a function of three variables, the vector whose components are the partial derivatives.
And we have seen how to use the gradient vector or the partial derivatives to derive various things such as approximation formulas.
The change in f, when we change x, y, z slightly, is approximately equal to, well, there are several terms.
And I can rewrite this in vector form as the gradient dot product the amount by which the position vector has changed.
Basically, what causes f to change is that I am changing x, y and z by small amounts and how sensitive f is to each variable is precisely what the partial derivatives measure.
And, in particular, this approximation is called the tangent plane approximation because it tells us, in fact, it amounts to identifying the graph of the function with its tangent plane.
It means that we assume that the function depends more or less linearly on x, y and z.
And, if we set these things equal, what we get is actually, we are replacing the function by its linear approximation.
We are replacing the graph by its tangent plane.
Except, of course, we haven't see the graph of a function of three variables because that would live in 4-dimensional space.
So, when we think of a graph, really, it is a function of two variables.
That also tells us how to find tangent planes to level surfaces.
Recall that the tangent plane to a surface, given by the equation f of x, y, z equals z, at a given point can be found by looking first for its normal vector.
And we know that the normal vector is actually, well, one normal vector is given by the gradient of a function because we know that the gradient is actually pointing perpendicularly to the level sets towards higher values of a function.
And it gives us the direction of fastest increase of a function.
OK. Any questions about these topics?
No.
OK. Let me add, actually, a cultural note to what we have seen so far about partial derivatives and how to use them, which is maybe something I should have mentioned a couple of weeks ago.
Why do we like partial derivatives?
Well, one obvious reason is we can do all these things.
But another reason is that, really, you need partial derivatives to do physics and to understand much of the world that is around you because a lot of things actually are governed by what is called partial differentiation equations.
So if you want a cultural remark about what this is good for.
A partial differential equation is an equation that involves the partial derivatives of a function.
So you have some function that is unknown that depends on a bunch of variables.
And a partial differential equation is some relation between its partial derivatives.
Let me see.
These are equations involving the partial derivatives -- -- of an unknown function.
Let me give you an example to see how that works.
For example, the heat equation is one example of a partial differential equation.
It is the equation -- Well, let me write for you the space version of it.
It is the equation partial f over partial t equals some constant times the sum of the second partials with respect to x, y and z.
So this is an equation where we are trying to solve for a function f that depends, actually, on four variables, x, y, z, t. And what should you have in mind?
Well, this equation governs temperature.
If you think that f of x, y, z, t will be the temperature at a point in space at position x, y, z and at time t, then this tells you how temperature changes over time.
It tells you that at any given point, the rate of change of temperature over time is given by this complicated expression in the partial derivatives in terms of the space coordinates x, y, z.
If you know, for example, the initial distribution of temperature in this room, and if you assume that nothing is generating heat or taking heat away, so if you don't have any air conditioning or heating going on, then it will tell you how the temperature will change over time and eventually stabilize to some final value.
Yes?
Why do we take the partial derivative twice?
Well, that is a question, I would say, for a physics person.
But in a few weeks we will actually see a derivation of where this equation comes from and try to justify it.
But, really, that is something you will see in a physics class.
The reason for that is basically physics of how heat is transported between particles in fluid, or actually any medium.
This constant k actually is called the heat conductivity.
It tells you how well the heat flows through the material that you are looking at.
Anyway, I am giving it to you just to show you an example of a real life problem where, in fact, you have to solve one of these things.
Now, how to solve partial differential equations is not a topic for this class.
It is not even a topic for 18.03 which is called Differential Equations, without partial, which means there actually you will learn tools to study and solve these equations but when there is only one variable involved.
And you will see it is already quite hard.
And, if you want more on that one, we have many fine classes about partial differential equations.
But one thing at a time.
I wanted to point out to you that very often functions that you see in real life satisfy many nice relations between the partial derivatives.
That was in case you were wondering why on the syllabus for today it said partial differential equations.
Now we have officially covered the topic.
That is basically all we need to know about it.
But we will come back to that a bit later.
You will see.
OK.
If there are no further questions, let me continue and go back to my list of topics.
Oh, sorry.
I should have written down that this equation is solved by temperature for point x, y, z at time t. OK. And there are, actually, many other interesting partial differential equations you will maybe sometimes learn about the wave equation that governs how waves propagate in space, about the diffusion equation, when you have maybe a mixture of two fluids, how they somehow mix over time and so on.
Basically, to every problem you might want to consider there is a partial differential equation to solve.
OK.
Anyway.
Sorry.
Back to my list of topics.
One important application we have seen of partial derivatives is to try to optimize things, try to solve minimum/maximum problems.
Remember that we have introduced the notion of critical points of a function.
A critical point is when all the partial derivatives are zero.
And then there are various kinds of critical points.
There is maxima and there is minimum, but there is also saddle points.
And we have seen a method using second derivatives -- -- to decide which kind of critical point we have.
I should say that is for a function of two variables to try to decide whether a given critical point is a minimum, a maximum or a saddle point.
And we have also seen that actually that is not enough to find the minimum of a maximum of a function because the minimum of a maximum could occur on the boundary.
Just to give you a small reminder, when you have a function of one variables, if you are trying to find the minimum and the maximum of a function whose graph looks like this, well, you are going to tell me, quite obviously, that the maximum is this point up here.
And that is a point where the first derivative is zero.
That is a critical point.
And we used the second derivative to see that this critical point is a local maximum.
But then, when we are looking for the minimum of a function, well, it is not at a critical point.
It is actually here at the boundary of the domain, you know, the range of values that we are going to consider.
Here the minimum is at the boundary.
And the maximum is at a critical point.
Similarly, when you have a function of several variables, say of two variables, for example, then the minimum and the maximum will be achieved either at a critical point.
And then we can use these methods to find where they are.
Or, somewhere on the boundary of a set of values that are allowed.
It could be that we actually achieve a minimum by making x and y as small as possible.
Maybe letting them go to zero if they had to be positive or maybe by making them go to infinity.
So, we have to keep our minds open and look at various possibilities.
We are going to do a problem like that.
We are going to go over a practice problem from the practice test to clarify this.
Another important cultural application of minimum/maximum problems in two variables that we have seen in class is the least squared method to find the best fit line, or the best fit anything, really, to find when you have a set of data points what is the best linear approximately for these data points.
And here I have some good news for you.
While you should definitely know what this is about, it will not be on the test.
[APPLAUSE] That doesn't mean that you should forget everything we have seen about it, OK?
Now what is next on my list of topics?
We have seen differentials.
Remember the differential of f, by definition, would be this kind of quantity.
At first it looks just like a new way to package partial derivatives together into some new kind of object.
Now, what is this good for?
Well, it is a good way to remember approximation formulas.
It is a good way to also study how variations in x, y, z relate to variations in f. In particular, we can divide this by variations, actually, by dx or by dy or by dz in any situation that we want, or by d of some other variable to get chain rules.
The chain rule says, for example, there are many situations.
But, for example, if x, y and z depend on some other variable, say of variables maybe even u and v, then that means that f becomes a function of u and v. And then we can ask ourselves, how sensitive is f to a value of u?
Well, we can answer that.
The chain rule is something like this.
And let me explain to you again where this comes from.
Basically, what this quantity means is if we change u and keep v constant, what happens to the value of f?
Well, why would the value of f change in the first place when f is just a function of x, y, z and not directly of you?
Well, it changes because x, y and z depend on u.
First we have to figure out how quickly x, y and z change when we change u.
Well, how quickly they do that is precisely partial x over partial u, partial y over partial u, partial z over partial u.
These are the rates of change of x, y, z when we change u.
And now, when we change x, y and z, that causes f to change.
How much does f change?
Well, partial f over partial x tells us how quickly f changes if I just change x. I get this.
That is the change in f caused just by the fact that x changes when u changes.
But then y also changes.
y changes at this rate.
And that causes f to change at that rate.
And z changes as well, and that causes f to change at that rate.
And the effects add up together.
Does that make sense?
OK. And so, in particular, we can use the chain rule to do changes of variables.
If we have, say, a function in terms of polar coordinates on theta and we like to switch it to rectangular coordinates x and y then we can use chain rules to relate the partial derivatives.
And finally, last but not least, we have seen how to deal with non-independent variables.
When our variables say x, y, z related by some equation.
One way we can deal with this is to solve for one of the variables and go back to two independent variables, but we cannot always do that.
Of course, on the exam, you can be sure that I will make sure that you cannot solve for a variable you want to remove because that would be too easy.
Then when we have to look at all of them, we will have to take into account this relation, we have seen two useful methods.
One of them is to find the minimum of a maximum of a function when the variables are not independent, and that is the method of Lagrange multipliers.
Remember, to find the minimum or the maximum of the function f, subject to the constraint g equals constant, well, we write down equations that say that the gradient of f is actually proportional to the gradient of g. There is a new variable here, lambda, the multiplier.
And so, for example, well, I guess here I had functions of three variables, so this becomes three equations.
f sub x equals lambda g sub x, f sub y equals lambda g sub y, and f sub z equals lambda g sub z.
And, when we plug in the formulas for f and g, well, we are left with three equations involving the four variables, x, y, z and lambda.
What is wrong?
Well, we don't have actually four independent variables.
We also have this relation, whatever the constraint was relating x, y and z together.
Then we can try to solve this.
And, depending on the situation, it is sometimes easy.
And it sometimes it is very hard or even impossible.
But on the test, I haven't decided yet, but it could well be that the problem about Lagrange multipliers just asks you to write the equations and not to solve them.
[APPLAUSE] Well, I don't know yet.
I am not promising anything.
But, before you start solving, check whether the problem asks you to solve them or not.
If it doesn't then probably you shouldn't.
Another topic that we solved just yesterday is constrained partial derivatives.
And I guess I have to re-explain a little bit because my guess is that things were not extremely clear at the end of class yesterday.
Now we are in the same situation.
We have a function, let's say, f of x, y, z where variables x, y and z are not independent but are constrained by some relation of this form.
Some quantity involving x, y and z is equal to maybe zero or some other constant.
And then, what we want to know, is what is the rate of change of f with respect to one of the variables, say, x, y or z when I keep the others constant?
Well, I cannot keep all the other constant because that would not be compatible with this condition.
I mean that would be the usual or so-called formal partial derivative of f ignoring the constraint.
To take this into account means that if we vary one variable while keeping another one fixed then the third one, since it depends on them, must also change somehow.
And we must take that into account.
Let's say, for example, we want to find -- I am going to do a different example from yesterday.
So, if you really didn't like that one, you don't have to see it again.
Let's say that we want to find the partial derivative of f with respect to z keeping y constant.
What does that mean?
That means y is constant, z varies and x somehow is mysteriously a function of y and z for this equation.
And then, of course because it depends on y, that means x will vary.
Sorry, depends on y and z and z varies.
Now we are asking ourselves what is the rate of change of f with respect to z in this situation?
And so we have two methods to do that.
Let me start with the one with differentials that hopefully you kind of understood yesterday, but if not here is a second chance.
Using differentials means that we will try to express df in terms of dz in this particular situation.
What do we know about df in general?
Well, we know that df is f sub x dx plus f sub y dy plus f sub z dz.
That is the general statement.
But, of course, we are in a special case.
We are in a special case where first y is constant.
y is constant means that we can set dy to be zero.
This goes away and becomes zero.
The second thing is actually we don't care about x.
We would like to get rid of x because it is this dependent variable.
What we really want to do is express df only in terms of dz.
What we need is to relate dx with dz.
Well, to do that, we need to look at how the variables are related so we need to look at the constraint g. Well, how do we do that?
We look at the differential g. So dg is g sub x dx plus g sub y dy plus g sub z dz.
And that is zero because we are setting g to always stay constant.
So, g doesn't change.
If g doesn't change then we have a relation between dx, dy and dz.
Well, in fact, we say we are going to look only at the case where y is constant.
y doesn't change and this becomes zero.
Well, now we have a relation between dx and dz.
We know how x depends on z.
And when we know how x depends on z, we can plug that into here and get how f depends on z.
Let's do that.
Again, saying that g cannot change and keeping y constant tells us g sub x dx plus g sub z dz is zero and we would like to solve for dx in terms of dz.
That tells us dx should be minus g sub z dz divided by g sub x.
If you want, this is the rate of change of x with respect to z when we keep y constant.
In our new terminology this is partial x over partial z with y held constant.
This is the rate of change of x with respect to z.
Now, when we know that, we are going to plug that into this equation.
And that will tell us that df is f sub x times dx.
Well, what is dx?
dx is now minus g sub z over g sub x dz plus f sub z dz.
So that will be minus fx g sub z over g sub x plus f sub z times dz.
And so this coefficient here is the rate of change of f with respect to z in the situation we are considering.
This quantity is what we call partial f over partial z with y held constant.
That is what we wanted to find.
Now, let's see another way to do the same calculation and then you can choose which one you prefer.
The other method is using the chain rule.
We use the chain rule to understand how f depends on z when y is held constant.
Let me first try the chain rule brutally and then we will try to analyze what is going on.
You can just use the version that I have up there as a template to see what is going on, but I am going to explain it more carefully again.
That is the most mechanical and mindless way of writing down the chain rule.
I am just saying here that I am varying z, keeping y constant, and I want to know how f changes.
Well, f might change because x might change, y might change and z might change.
Now, how quickly does x change?
Well, the rate of change of x in this situation is partial x, partial z with y held constant.
If I change x at this rate then f will change at that rate.
Now, y might change, so the rate of change of y would be the rate of change of y with respect to z holding y constant.
Wait a second.
If y is held constant then y doesn't change.
So, actually, this guy is zero and you didn't really have to write that term.
But I wrote it just to be systematic.
If y had been somehow able to change at a certain rate then that would have caused f to change at that rate.
And, of course, if y is held constant then nothing happens here.
Finally, while z is changing at a certain rate, this rate is this one and that causes f to change at that rate.
And then we add the effects together.
See, it is nothing but the good-old chain rule.
Just I have put these extra subscripts to tell us what is held constant and what isn't.
Now, of course we can simplify it a little bit more.
Because, here, how quickly does z change if I am changing z?
Well, the rate of change of z, with respect to itself, is just one.
In fact, the really mysterious part of this is the one here, which is the rate of change of x with respect to z.
And, to find that, we have to understand the constraint.
How can we find the rate of change of x with respect to z?
Well, we could use differentials, like we did here, but we can also keep using the chain rule.
How can I do that?
Well, I can just look at how g would change with respect to z when y is held constant.
I just do the same calculation with g instead of f. But, before I do it, let's ask ourselves first what is this equal to.
Well, if g is held constant then, when we vary z keeping y constant and changing x, well, g still doesn't change.
It is held constant.
In fact, that should be zero.
But, if we just say that, we are not going to get to that.
Let's see how we can compute that using the chain rule.
Well, the chain rule tells us g changes because x, y and z change.
How does it change because of x?
Well, partial g over partial x times the rate of change of x.
How does it change because of y?
Well, partial g over partial y times the rate of change of y.
But, of course, if you are smarter than me then you don't need to actually write this one because y is held constant.
And then there is the rate of change because z changes.
And how quickly z changes here, of course, is one.
Out of this you get, well, I am tired of writing partial g over partial x.
We can just write g sub x times partial x over partial z y constant plus g sub z.
And now we found how x depends on z.
Partial x over partial z with y held constant is negative g sub z over g sub x.
Now we plug that into that and we get our answer.
It goes all the way up here.
And then we get the answer.
I am not going to, well, I guess I can write it again.
There was partial f over partial x times this guy, minus g sub z over g sub x, plus partial f over partial z.
And you can observe that this is exactly the same formula that we had over here.
In fact, let's compare this to make it side by side.
I claim we did exactly the same thing, just with different notations.
If you take the differential of f and you divide it by dz in this situation where y is held constant and so on, you get exactly this chain rule up there.
That chain rule up there is this guy, df, divided by dz with y held constant.
And the term involving dy was replaced by zero on both sides because we knew, actually, that y is held constant.
Now, the real difficulty in both cases comes from dx.
And what we do about dx is we use the constant.
Here we use it by writing dg equals zero.
Here we write the chain rule for g, which is the same thing, just divided by dz with y held constant.
This formula or that formula are the same, just divided by dz with y held constant.
And then, in both cases, we used that to solve for dx.
And then we plugged into the formula of df to express df over dz, or partial f, partial z with y held constant.
So, the two methods are pretty much the same.
Quick poll.
Who prefers this one?
Who prefers that one?
OK.
Majority vote seems to be for differentials, but it doesn't mean that it is better.
Both are fine.
You can use whichever one you want.
But you should give both a try.
OK. Any questions?
Yes?
Yes.
Thank you.
I forgot to mention it.
Where did that go?
I think I erased that part.
We need to know -- -- directional derivatives.
Pretty much the only thing to remember about them is that df over ds, in the direction of some unit vector u, is just the gradient f dot product with u.
That is pretty much all we know about them.
Any other topics that I forgot to list?
No.
Yes?
Can I erase three boards at a time?
No, I would need three hands to do that.
I think what we should do now is look quickly at the practice test.
I mean, given the time, you will mostly have to think about it yourselves.
Hopefully you have a copy of the practice exam.
The first problem is a simple problem.
Find the gradient.
Find an approximation formula.
Hopefully you know how to do that.
The second problem is one about writing a contour plot.
And so, before I let you go for the weekend, I want to make sure that you actually know how to read a contour plot.
One thing I should mention is this problem asks you to estimate partial derivatives by writing a contour plot.
We have not done that, so that will not actually be on the test.
We will be doing qualitative questions like what is the sine of a partial derivative.
Is it zero, less than zero or more than zero?
You don't need to bring a ruler to estimate partial derivatives the way that this problem asks you to.
[APPLAUSE] Let's look at problem 2B.
Problem 2B is asking you to find the point at which h equals 2200, partial h over partial x equals zero and partial h over partial y is less than zero.
Let's try and see what is going on here.
A point where f equals 2200, well, that should be probably on the level curve that says 2200.
We can actually zoom in.
Here is the level 2200.
Now I want partial h over partial x to be zero.
That means if I change x, keeping y constant, the value of h doesn't change.
Which points on the level curve satisfy that property?
It is the top and the bottom.
If you are here, for example, and you move in the x direction, well, you see, as you get to there from the left, the height first increases and then decreases.
It goes for a maximum at that point.
So, at that point, the partial derivative is zero with respect to x.
And the same here.
Now, let's find partial h over partial y less than zero.
That means if we go north we should go down.
Well, which one is it, top or bottom?
Top.
Yes.
Here, if you go north, then you go from 2200 down to 2100.
This is where the point is.
Now, the problem here was also asking you to estimate partial h over partial y.
And if you were curious how you would do that, well, you would try to figure out how long it takes before you reach the next level curve.
To go from here to here, to go from Q to this new point, say Q prime, the change in y, well, you would have to read the scale, which was down here, would be about something like 300.
What is the change in height when you go from Q to Q prime?
Well, you go down from 2200 to 2100.
That is actually minus 100 exactly.
OK?
And so delta h over delta y is about minus one-third, well, minus 100 over 300 which is minus one-third.
And that is an approximation for partial derivative.
So, that is how you would do it.
Now, let me go back to other things.
If you look at this practice exam, basically there is a bit of everything and it is kind of fairly representative of what might happen on Tuesday.
There will be a mix of easy problems and of harder problems.
Expect something about computing gradients, approximations, rate of change.
Expect a problem about reading a contour plot.
Expect one about a min/max problem, something about Lagrange multipliers, something about the chain rule and something about constrained partial derivatives.
I mean pretty much all the topics are going to be there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, basically the last few weeks, we've been doing derivatives.
Now, we're going to integrals.
So -- OK, so more precisely, we are going to be talking about double integrals.
OK, so just to motivate the notion, let me just remind you that when you have a function of one variable -- -- say, f of x, and you take its integrals from, say, a to b of f of x dx, well, that corresponds to the area below the graph of f over the interval from a to b. OK, so the picture is something like you have a; you have b.
You have the graph of f, and then what the integral measures is the area of this region.
And, when we say the area of this region, of course, if f is positive, that's what happens.
If f is negative, then we count negatively the area below the x axis.
OK, so, now, when you have a function of two variables, then you can try to do the same thing.
Namely, you can plot its graph.
Its graph will be a surface in space.
And then, we can try to look for the volume below the graph.
And that's what we will call the double integral of the function over a certain region.
OK, so let's say that we have a function of two variables, x and y.
Then, we'll look at the volume that's below the graph z equals f of xy.
OK, so, let's draw a picture for what this means.
I have a function of x and y. I can draw its graph.
The graph will be the surface with equation z equals f of x and y.
And, well, I have to decide where I will integrate the function.
So, for that, I will choose some region in the xy plane.
And, I will integrate the function on that region.
So, it's over a region, R, in the xy plane.
So, I have this region R and I look at the piece of the graph that is above this region.
And, we'll try to compute the volume of this solid here.
OK, that's what the double integral will measure.
So, we'll call that the double integral of our region, R, of f of xy dA and I will have to explain what the notation means.
So, dA here stands for a piece of area.
A stands for area.
And, well, it's a double integral.
So, that's why we have two integral signs.
And, we'll have to indicate somehow the region over which we are integrating.
OK, we'll come up with more concrete notations when we see how to actually compute these things.
That's the basic definition.
OK, so actually, how do we define it, that's not really much of a definition yet.
How do we actually define this rigorously?
Well, remember, the integral in one variable, you probably saw a definition where you take your integral from a to b, and you cut it into little pieces.
And then, for each little piece, you take the value of a function, and you multiply by the width of a piece.
That gives you a rectangular slice, and then you sum all of these rectangular slices together.
So, here we'll do the same thing.
So, well, let me put a picture up and explain what it does.
So, we're going to cut our origin into little pieces, say, little rectangles or actually anything we want.
And then, for each piece, with the small area, delta A, we'll take the area delta a times the value of a function in there that will give us the volume of a small box that sits under the graph.
And then, we'll add all these boxes together.
That gives us an estimate of a volume.
And then, to get actually the integral, the integral will be defined as a limit as we subdivide into smaller and smaller boxes, and we sum more and more pieces, OK?
So, actually, what we do, oh, I still have a board here.
So, the actual definition involves cutting R into small pieces of area that's called delta A or maybe delta Ai, the area of the i'th piece.
And then, OK, so maybe in the xy plane, we have our region, and we'll cut it may be using some grade.
OK, and then we'll have each small piece.
Each small piece will have area delta Ai and it will be at some point, let's call it xi, yi ... yi, xi.
And then, we'll consider the sum over all the pieces of f at that point, xi, yi times the area of a small piece.
So, what that corresponds to in the three-dimensional picture is just I sum the volumes of all of these little columns that sit under the graph.
OK, and then, so what I do is actually I take the limit as the size of the pieces tends to zero.
So, I have more and more smaller and smaller pieces.
And, that gives me the double integral.
OK, so that's not a very good sentence, but whatever.
So, OK, so that's the definition.
Of course, we will have to see how to compute it.
We don't actually compute it.
When you compute an integral in single variable calculus, you don't do that.
You don't cut into little pieces and sum the pieces together.
You've learned how to integrate functions using various formulas, and similarly here, we'll learn how to actually compute these things without doing that cutting into small pieces.
OK, any questions first about the concept, or what the definition is?
Yes?
Well, so we'll have to learn which tricks work, and how exactly.
But, so what we'll do actually is we'll reduce the calculation of a double integral to two calculations of single integrals.
And so, for V, certainly, all the tricks you've learned in single variable calculus will come in handy.
OK, so, yeah that's a strong suggestion that if you've forgotten everything about single variable calculus, now would be a good time to actually brush up on integrals.
The usual integrals, and the usual substitution tricks and easy trig in particular, these would be very useful.
OK, so, yeah, how do we compute these things?
That's what we would have to come up with.
And, well, going back to what we did with derivatives, to understand variations of functions and derivatives, what we did was really we took slices parallel to an axis or another one.
So, in fact, here, the key is also the same.
So, what we are going to do is instead of cutting into a lot of small boxes like that and summing completely at random, we will actually somehow scan through our region by parallel planes, OK?
So, let me put up, actually, a slightly different picture up here.
So, what I'm going to do is I'm going to take planes, say in this picture, parallel to the yz plane.
I'll take a moving plane that scans from the back to the front or from the front to the back.
So, that means I set the value of x, and I look at the slice, x equals x0, and then I will do that for all values of x0.
So, now in each slice, well, I get what looks a lot like a single variable integral.
OK, and that integral will tell me, what is the area in this?
Well, I guess it's supposed to be green, but it all comes as black, so, let's say the black shaded slice.
And then, when I add all of these areas together, as the value of x changes, I will get the volume.
OK, let me try to explain that again.
So, to compute this integral, what we do is actually we take slices.
So, let's consider, let's call s of x the area of a slice, well, by a plane parallel to the yz plane.
OK, so on the picture, s of x is just the area of this thing in the vertical wall.
Now, if you sum all of these, well, why does that work?
So, if you take the origin between two parallel slices that are very close to each other, what's the volume in these two things?
Well, it's essentially s of x times the thickness of this very thin slice, and the thickness would be delta x0 dx if you take a limit with more and more slices.
OK, so the volume will be the integral of s of x dx from, well, what should be the range for x?
Well, we would have to start at the very lowest value of x that ever happens in our origin, and we'd have to go all the way to the very largest value of x, from the very far back to the very far front.
So, in this picture, we probably start over here at the back, and we'd end over here at the front.
So, let me just say from the minimum, x, to the maximum x.
And now, how do we find S of x?
Well, S of x will be actually again an integral.
But now, it's an integral of the variable, y, because when we look at this slice, what changes from left to right is y.
So, well let me actually write that down.
For a given, x, the area S of x you can compute as an integral of f of x, y dy.
OK, well, now x is a constant, and y will be the variable of integration.
What's the range for y?
Well, it's from the leftmost point here to the rightmost point here on the given slice.
So, there is a big catch here.
That's a very important thing to remember.
What is the range of integration?
The range of integration for y depends actually on x.
See, if I take the slice that's pictured on that diagram, then the range for y goes all the way from the very left to the very right.
But, if I take a slice that, say, near the very front, then in fact, only a very small segment of it will be in my region.
So, the range of values for y will be much less.
Let me actually draw a 2D picture for that.
So, remember, we fix x, so, sorry, so we fix a value of x. OK, and for a given value of x, what we will do is we'll slice our graph by this plane parallel to the yz plane.
So, now we mention the graph is sitting above that.
OK, that's the region R. We have the region, R, and I have the graph of a function above this region, R. And, I'm trying to find the area between this segment and the graph above it in this vertical plane.
Well, to do that, I have to integrate from y going from here to here.
I want the area of a piece that sits above this red segment.
And, so in particular, the endpoints, the extreme values for y depend on x because, see, if I slice here instead, well, my bounds for y will be smaller.
OK, so now, if I put the two things together, what I will get -- -- is actually a formula where I have to integrate -- -- over x -- -- an integral over y. OK, and so this is called an iterated integral because we iterate twice the process of taking an integral.
OK, so again, what's important to realize here, I mean, I'm going to say that several times over the next few days but that's because it's the single most important thing to remember about double integrals, the bounds here are just going to be numbers, OK, because the question I'm asking myself here is, what is the first value of x by which I might want to slice, and what is the last value of x?
Which range of x do I want to look at to take my red slices?
And, the answer is I would go all the way from here, that's my first slice, to somewhere here.
That's my last slice.
For any value in between these, I will have some red segment, and I will want to integrate over that that.
On the other hand here, the bounds will depend on the outer variable, x, because at a fixed value of x, what the values of y will be depends on x in general.
OK, so I think we should do lots of examples to convince ourselves and see how it works.
Yeah, it's called an iterated integral because first we integrated over y, and then we integrate again over x, OK?
So, we can do that, well, I mean, y depends on x or x depends, no, actually x and y vary independently of each other inside here.
What is more complicated is how the bounds on y depend on x.
But actually, you could also do the other way around: first integrate over x, and then over y, and then the bounds for x will depend on y.
We'll see that on an example.
Yes?
So, for y, I'm using the range of values for y that corresponds to the given value of x, OK?
Remember, this is just like a plot in the xy plane.
Above that, we have the graph.
Maybe I should draw a picture here instead.
For a given value of x, so that's a given slice, I have a range of values for y, that is, from this picture at the leftmost point on that slice to the rightmost point on that slice.
So, where start and where I stop depends on the value of x.
Does that make sense?
OK. OK, no more questions?
OK, so let's do our first example.
So, let's say that we want to integrate the function 1-x^2-y^2 over the region defined by x between 0 and 1, and y between 0 and 1.
So, what does that mean geometrically?
Well, z = 1-x^2-y^2, and it's a variation on, actually I think we plotted that one, right?
That was our first example of a function of two variables possibly.
And, so, we saw that the graph is this paraboloid pointing downwards.
OK, it's what you get by taking a parabola and rotating it.
And now, what we are asking is, what is the volume between the paraboloid and the xy plane over the square of side one in the xy plane over the square of side one in the xy plane, x and y between zero and one.
OK, so, what we'll do is we'll, so, see, here I try to represent the square.
And, we'll just sum the areas of the slices as, say, x varies from zero to one.
And here, of course, setting up the bounds will be easy because no matter what x I take, y still goes from zero to one.
See, it's easiest to do double integrals what the region is just a rectangle on the xy plane because then you don't have to worry too much about what are the ranges.
OK, so let's do it.
Well, that would be the integral from zero to one of the integral from zero to one of 1-x^2-y^2 dy dx.
So, I'm dropping the parentheses.
But, if you still want to see them, I'm going to put that in very thin so that you see what it means.
But, actually, the convention is we won't put this parentheses in there anymore.
OK, so what this means is first I will integrate 1-x^2-y^2 over y, ranging from zero to one with x held fixed.
So, what that represents is the area in this slice.
So, see here, I've drawn, well, what happens is actually the function takes positive and negative values.
So, in fact, I will be counting positively this part of the area.
And, I will be counting negatively this part of the area, I mean, as usual when I do an integral.
OK, so what I will do to evaluate this, I will first do what's called the inner integral.
So, to do the inner integral, well, it's pretty easy.
How do I integrate this?
Well, it becomes, so, what's the integral of one?
It's y.
Just anything to remember is we are integrating this with respect to y, not to x.
The integral of x^2 is x^2 times y.
And, the integral of y^2 is y^3 over 3.
OK, and that we plug in the bounds, which are zero and one in this case.
And so, when you plug y equals one, you will get one minus x^2 minus one third minus, well, for y equals zero you get 0,0, 0, so nothing changes.
OK, so you are left with two thirds minus x^2.
OK, and that's a function of x only.
Here, you shouldn't see any y's anymore because y was your integration variable.
But, you still have x.
You still have x because the area of this shaded slice depends, of course, on the value of x.
And, so now, the second thing to do is to do the outer integral.
So, now we integrate from zero to one what we got, which is two thirds minus x^2 dx.
OK, and we know how to compute that because that integrates to two thirds x minus one third x^3 between zero and one.
And, I'll let you do the computation.
You will find it's one third.
OK, so that's the final answer.
So, that's the general pattern.
When we have a double integral to compute, first we want to set it up carefully.
We want to find, what will be the bounds in x and y?
And here, that was actually pretty easy because our equation was very simple.
Then, we want to compute the inner integral, and then we compute the outer integral.
And, that's it.
OK, any questions at this point?
No?
OK, so, by the way, we started with dA in the notation, right?
Here we had dA.
And, that somehow became a dy dx.
OK, so, dA became dy dx because when we do the iterated integral this way, what we're actually doing is that we are slicing our origin into small rectangles.
OK, that was the area of this small rectangle here?
Well, it's the product of its width times its height.
So, that's delta x times delta y. OK, so, delta a equals delta x delta y becomes...
So actually, it's not just becomes, it's really equal.
So, the small rectangles for.
Now, it became dy dx and not dx dy.
Well, that's a question of, in which order we do the iterated integral?
It's up to us to decide whether we want to integrate x first, then y, or y first, then x.
But, as we'll see very soon, that is an important decision when it comes to setting up the bounds of integration.
Here, it doesn't matter, but in general we have to be very careful about in which order we will do things.
Yes?
Well, in principle it always works both ways.
Sometimes it will be that because the region has a strange shape, you can actually set it up more easily one way or the other.
Sometimes it will also be that the function here, you actually know how to integrate in one way, but not the other.
So, the theory is that it should work both ways.
In practice, one of the two calculations may be much harder.
OK. Let's do another example.
Let's say that what I wanted to know was not actually what I computed, namely, the volume below the paraboloid, but also the negative of some part that's now in the corner towards me.
But let's say really what I wanted was just the volume between the paraboloid and the xy plane, so looking only at the part of it that sits above the xy plane.
So, that means, instead of integrating over the entire square of size one, I should just integrate over the quarter disk.
I should stop integrating where my paraboloid hits the xy plane.
So, let me draw another picture.
So, let's say I wanted to integrate, actually -- So, let's call this example two.
So, we are going to do the same function but over a different region.
And, the region will just be, now, this quarter disk here.
OK, so maybe I should draw a picture on the xy plane.
That's your region, R. OK, so in principle, it will be the same integral.
But what changes is the bounds.
Why do the bounds change?
Well, the bounds change because now if I set, if I fixed some value of x, then I want to integrate this part of the slice that's above the xy plane and I don't want to take this part that's actually outside of my disk.
So, I should stop integrating over y when y reaches this value here.
OK, on that picture here, on this picture, it tells me for a fixed value of x, the range of values for y should go only from here to here.
So, that's from here to less than one.
OK, so for a given, x, the range of y is, well, so what's the lowest value of y that we want to look at?
It's still zero.
From y equals zero to,what's the value of y here?
Well, I have to solve in the equation of a circle, OK?
So, if I'm here, this is x^2 y^2 equals one.
That means y is square root of one minus x^2.
OK, so I will integrate from y equals zero to y equals square root of one minus x^2.
And, now you see how the bound from y will depend on the value of x. OK, so while I erase, I will let you think about, what is the bound for x now?
It's a trick question.
OK, so I claim that what we will do -- We write this as an iterated integral first dy then dx.
And, we said for a fixed value of x, the range for y is from zero to square root of one minus x^2.
What about the range for x?
Well, the range for x should just be numbers.
OK, remember, the question I have to ask now is if I look at all of these yellow slices, which one is the first one that I will consider?
Which one is the last one that I want to consider?
So, the smallest value of x that I want to consider is zero again.
And then, I will have actually a pretty big slice.
And I will get smaller, and smaller, and smaller slices.
And, it stops.
I have to stop when x equals one.
Afterwards, there's nothing else to integrate.
So, x goes from zero to one.
OK, and now, see how in the inner integral, the bounds depend on in the inner integral, the bounds depend on x.
In the outer one, you just get numbers because the questions that you have to ask to set up this one and set up that one are different.
Here, the question is, if I fix a given, x, if I look at a given slice, what's the range for y?
Here, the question is, what's the first slice?
What is the last slice?
Does that make sense?
Everyone happy with that?
OK, very good.
So, now, how do we compute that?
Well, we do the inner integral.
So, that's an integral from zero to square root of one minus x^2 of one minus x^2 minus y^2 dy.
And, well, that integrates to y-x^2y-y^3 over three from zero to square root of one minus x^2.
And then, that becomes, well, the root of one minus x^2 minus x^2 root of one minus x^2 minus y minus x^2 to the three halves over three.
And actually, if you look at it for long enough, see, this says one minus x^2 times square root of one minus x^2.
So, actually, that's also, so, in fact, that simplifies to two thirds of one minus x^2 to the three halves.
OK, let me redo that, maybe, slightly differently.
This was one minus x^2 times y.
So -- -- one minus x^2 times y becomes square root of one minus x^2 minus y^3 over three.
And then, when I take y equals zero, I get zero.
So, I don't subtract anything.
OK, so now you see this is one minus x^2 to the three halves minus a third of it.
So, you're left with two thirds.
OK, so, that's the integral.
The outer integral is the integral from zero to one of two thirds of one minus x^2 to the three halves dx.
And, well, I let you see if you remember single variable integrals by trying to figure out what this actually comes out to be is it pi over two, or pi over eight, actually?
I think it's pi over eight.
OK, well I guess we have to do it then.
I wrote something on my notes, but it's not very clear, OK?
So, how do we compute this thing?
Well, we have to do trig substitution.
That's the only way I know to compute an integral like that, OK?
So, we'll set x equal sine theta, and then square root of one minus x^2 will be cosine theta.
We are using sine squared plus cosine squared equals one.
And, so that will become -- -- so, two thirds remains two thirds.
One minus x^2 to the three halves becomes cosine cubed theta.
dx, well, if x is sine theta, then dx is cosine theta d theta.
So, that's cosine theta d theta.
And, well, if you do things with substitution, which is the way I do them, then you should worry about the bounds for theta which will be zero to pi over two.
Or, you can also just plug in the bounds at the end.
So, now you have the two thirds times the integral from zero to pi over two of cosine to the fourth theta d theta.
And, how do you integrate that?
Well, you have to use double angle formulas.
OK, so cosine to the fourth, remember, cosine squared theta is one plus cosine two theta over two.
And, we want the square of that.
And, so that will give us -- -- of, well, we'll have, it's actually one quarter plus one half cosine to theta plus one quarter cosine square to theta d theta.
And, how will you handle this guy?
Well, using, again, the double angle formula.
OK, so it's getting slightly nasty.
So, but I don't know any simpler solution except for one simpler solution, which is you have a table of integrals of this form inside the notes.
Yes?
No, I don't think so because if you take one half times cosine half times two, you will still have half, OK?
So, if you do, again, the double angle formula, I think I'm not going to bother to do it.
I claim you will get, at the end, pi over eight because I say so.
OK, so exercise, continue calculating and get pi over eight.
OK, now what does the show us?
Well, this shows us, actually, that this is probably not the right way to do this.
OK, the right way to do this will be to integrate it in polar coordinates.
And, that's what we will learn how to do tomorrow.
So, we will actually see how to do it with much less trig.
So, that will be easier in polar coordinates.
So, we will see that tomorrow.
OK, so we are almost there.
I mean, here you just use a double angle again and then you can get it.
And, it's pretty straightforward.
OK, so one thing that's kind of interesting to know is we can exchange the order of integration.
Say we have an integral given to us in the order dy dx, we can switch it to dx dy.
But, we have to be extremely careful with the bounds.
So, you certainly cannot just swap the bounds of the inner and outer because there you would end up having this square root of one minus x^2 on the outside, and you would never get a number out of that.
So, that cannot work.
It's more complicated than that.
OK, so, well, here's a first baby example.
Certainly, if I do integral from zero to one, integral from zero to two dx dy, there, I can certainly switch the bounds without thinking too much.
What's the reason for that?
Well, the reason for that is this corresponds in both cases to integrating x from zero to two, and y from zero to one.
It's a rectangle.
So, if I slice it this way, you see that y goes from zero to one for any x between zero and two.
It's this guy.
If I slice it that way, then x goes from zero to two for any value of y between zero and one.
And, it's this one.
So, here it works.
But in general, I have to draw picture of my region, and see how the slices look like both ways.
OK, so let's do a more interesting one.
Let's say that I want to compute an integral from zero to one of integral from x to square root of x of e^y over y dy dx.
So, why did I choose this guy?
Which is the guy because as far as I can tell, there's no way to integrate e^y over y.
So, this is an integral that you cannot compute this way.
So, it's a good example for why this can be useful.
So, if you do it this way, you are stuck immediately.
So, instead, we will try to switch the order.
But, to switch the order, we have to understand, what do these bounds mean?
OK, so let's draw a picture of the region.
Well what I am saying is y equals x to y equals square root of x.
Well, let's draw y equals x, and y equals square root of x.
Well, maybe I should actually put this here, y equals x to y equals square root of x. OK, and so I will go, for each value of x I will go from y equals xo to y equals square root of x.
And then, we'll do that for values of x that go from x equals zero to x equals one, which happens to be exactly where these things intersect.
So, my region will consist of all this, OK?
So now, if I want to do it the other way around, I have to decompose my region.
The other way around, I have to, so my goal, now, is to rewrite this as an integral.
Well, it's still the same function.
It's still e to the y over y.
But now, I want to integrate dx dy.
So, how do I integrate over x?
Well, I fix a value of y.
And, for that value of y, what's the range of x?
Well, the range for x is from here to here.
OK, what's the value of x here?
Let's start with an easy one.
This is x equals y.
What about this one?
It's x equals y^2.
OK, so, x goes from y2 to y, and then what about y?
Well, I have to start at the bottom of my region.
That's y equals zero to the top, which is at y equals one.
So, y goes from zero to one.
So, switching the bounds is not completely obvious.
That took a little bit of work.
But now that we've done that, well, just to see how it goes, it's actually going to be much easier to integrate because the inner integral, well, what's the integral of e^y over y with respect to x?
It's just that times x, right, from x equals y^2 to y.
So, that will be, well, if I plug x equals y, I will get e to the y minus, if I plug x equals y^2, I will get e to the y over y times y^2 into the y times y, OK?
So, now, if I do the outer integral, I will have the integral from zero to one of e to the y minus y^e to the y dy.
And, that one actually is a little bit easier.
So, we know how to integrate e^y.
We don't quite know how to integrate ye^y.
But, let's try.
So, let's see, what's the derivative of ye^y?
Well, there's a product rule that's one times e^y plus y times the derivative of e^y is ye^y.
So, if we do, OK, let's put a minus sign in front.
Well, that's almost what we want, except we have a minus e^y instead of a plus e^y.
So, we need to add 2e^y.
And, I claim that's the antiderivative.
OK, if you got lost, you can also integrate by integrating by parts, by taking the derivative of y and integrating these guys.
Or, but, you know, that works.
Just, your first guess would be, maybe, let's try minus y^e to the y.
Take the derivative of that, compare, see what you need to do to fix.
And so, if you take that between zero and one, you'll actually get e minus two.
OK, so, tomorrow we are going to see how to do double integrals in polar coordinates, and also applications of double integrals, how to use them for interesting things.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Yesterday we saw how to define double integrals and how to start computing them in terms of x and y coordinates.
We have defined the double integral over a region R and plane of a function f of x, y dA.
You cannot hear me?
Is the sound working?
Can you hear me in the back now?
Can we make the sound louder?
Does this work?
People are not hearing me in the back.
Is it better?
People are still saying make it louder.
Is it better?
OK. Great.
Thanks.
That's not a reason to start chatting with your friends.
Thanks.
When we have a region in the x, y plane and we have a function of x and y, we are defining the double integral of f over this region by taking basically the sum of the values of a function everywhere in here times the area element.
And the definition, actually, is we split the region into lots of tiny little pieces, we multiply the value of a function at the point times the area of a little piece and we sum that everywhere.
And we have seen, actually, how to compute these things as iterated integrals.
First, integrating over dy and then over dx, or the other way around.
One example that we did, in particular, was to compute the double integral of a quarter of a unit disk.
That was the region where x squared plus y squared is less than one and x and y are positive, of one minus x squared minus y squared dA.
Well, hopefully, I kind of convinced you that we can do it using enough trig and substitutions and so on, but it is not very pleasant.
And the reason for that is that using x and y coordinates here does not seem very appropriate.
In fact, we can use polar coordinates instead to compute this double integral.
Remember that polar coordinates are about replacing x and y as coordinates for a point on a plane by instead r, which is the distance from the origin to a point, and theta, which is the angle measured counterclockwise from the positive x-axis.
In terms of r and theta, you have x equals r cosine theta, y equals r sine theta.
The claim is we are able, actually, to do double integrals in polar coordinates.
We just have to learn how to.
Just to draw a quick picture -- When we were integrating in x, y coordinates, in rectangular coordinates, we were slicing our region by gridlines that were either horizontal or vertical.
And we used that to set up the iterated integral.
And we said dA became dx dy or dy dx.
Now we are going to actually integrate, in terms of the polar coordinates, r and theta.
Let's say we will integrate in the order with r first and then theta.
That is the order that makes the most sense usually when you do polar coordinates.
What does that mean?
It means that we will first focus on a slice where we fix the value of theta and we will let r vary.
That means we fix a direction, we fix a ray out from the origin in a certain direction.
And we will travel along this ray and see which part of it, which values of r are in our region.
Here it will be actually pretty easy because r will just start at zero, and you will have to stop when you exit this quarter disk.
Well, what is the equation of this circle in polar coordinates?
It is just r equals one.
So, we will stop when r reaches one.
But what about theta?
Well, the first ray that we might want to consider is the one that goes along the x-axis.
That is when theta equals zero.
And we will stop when theta reaches pi over two because we don't care about the rest of the disk.
We only care about the first quadrant.
We will stop at pi over two.
Now, there is a catch, though, which is that dA is not dr d theta.
Let me explain to you why.
Let's say that we are slicing.
What it means is we are cutting our region into little pieces that are the elementary, you know, what corresponds to a small rectangle in the x, y coordinate system, here would be actually a little piece of circle between a given radius r and r plus delta r. And given between an angle theta and theta plus delta theta.
I need to draw, actually, a bigger picture of that because it makes it really hard to read.
Let's say that I fix an angle theta and a slightly different one where I have added delta theta to it.
And let's say that I have a radius r and I add delta r to it.
Then I will have a little piece of x, y plane that is in here.
And I have to figure out what is its area?
What is delta A for this guy?
Well, let's see.
This guy actually, you know, if my delta r and delta theta are small enough, it will almost look like a rectangle.
It is rotated, but it is basically a rectangle.
I mean these sides, of course, are curvy, but they are short enough and it is almost straight.
The area here should be this length times that length.
Well, what is this length?
That one is easy.
It is delta r. What about that length?
Well, it is not delta theta.
It is something slightly different.
It is a piece of a circle of radius r corresponding to angle delta theta, so it is r delta theta.
So, times r delta theta.
That means now, even if we shrink things and take smaller and smaller regions, dA is going to be r dr d theta.
That is an important thing to remember.
When you integrate in polar coordinates, you just set up your bounds in terms of r and theta, but you replace dA by r dr d theta, not just dr d theta.
And then, of course, we have some function that we are integrating.
Let's say that I call that thing f then it is the same f that I put up here.
Concretely, how do I do it here?
Well, my function f was given as one minus x squared minus y squared.
And I would like to switch that to polar coordinates.
I want to put r and theta in there.
Well, I have formulas for x and y in polar coordinates so I could just replace x squared by r squared cosine squared theta, y squared by r squared sine squared theta.
And that works just fine.
But maybe you can observe that this is x squared plus y squared.
It is just the square of a distance from the origin, so that is just r squared.
That is a useful thing.
You don't strictly need it, but it is much faster if you see this right away.
It saves you writing down a sine and a cosine.
Now we just end up with the integral from zero to pi over two, integral from zero to one of one minus r squared r dr d theta.
Now, if I want to compute this integral, so let's first do the inner integral.
If I integrate r minus r cubed, I will get r squared over two minus r squared over four between zero and one.
And then I will integrate d theta.
What is this equal to?
Well, for r equals one you get one-half minus one-quarter, which is going to be just one-quarter.
And when you plug in zero you get zero.
So, it is the integral from zero to pi over two of one-quarter d theta.
And that just integrates to one-quarter times pi over two, which is pi over eight.
That is a lot easier than the way we did it yesterday.
Well, here we were lucky.
I mean usually you will switch to polar coordinates either because the region is easier to set up.
Here it is indeed easier to set up because the bounds became very simple.
We don't have that square root of one minus x squared anymore.
Or because the integrant becomes much simpler.
Here our function, well, it is not very complicated in x, y coordinates, but it is even simpler in r theta coordinates.
Here we were very lucky.
In general, there is maybe a trade off.
Maybe it will be easier to set up bounds but maybe the function will become harder because it will have all these sines and cosines in it.
If our function had been just x, x is very easy in x, y coordinates.
Here it becomes r cosine theta.
That means you will have a little bit of trig to do in the integral.
Not a very big one, not a very complicated integral, but imagine it could get potentially much harder.
Anyway, that is double integrals in polar coordinates.
And the way you set up the bounds in general, well, in 99% of the cases you will integrate over r first.
What you will do is you will look for a given theta what are the bounds of r to be in the region.
What is the portion of my ray that is in the given region?
And then you will put bounds for theta.
But conceptually it is the same as before.
Instead of slicing horizontally or vertically, we slice radially.
We will do more examples in a bit.
Any questions about this or the general method?
Yes?
That is a very good question.
Why do I measure the length inside instead of outside?
Which one do I want?
This one.
Here I said this side is r delta theta.
I could have said, actually, r delta theta is the length here.
Here it is slightly more, r plus delta r times delta theta.
But, if delta r is very small compared to r, then that is almost the same thing.
And this is an approximation anyway.
I took this one because it gives me the simpler formula.
If you take the limit as delta r turns to zero then the two things become the same anyway.
The length, whether you put r or r plus delta r in here, doesn't matter anymore.
If you imagine that this guy is infinitely small then, really, the lengths become the same.
We will also see another proof of this formula, using changes of variables, next week.
But, I mean, hopefully this is at least slightly convincing.
More questions?
No.
OK. Let's see.
We have seen how to compute double integrals.
I have to tell you what they are good for as well.
The definition we saw yesterday and the motivation was in terms of finding volumes, but that is not going to be our main preoccupation.
Because finding volumes is fun but that is not all there is to life.
I mean, you are doing single integrals.
When you do single integrals it is usually not to find the area of some region of a plane.
It is for something else usually.
The way we actually think of the double integral is really as summing the values of a function all around this region.
We can use that to get information about maybe the region or about the average value of a function in that region and so on.
Let's think about various uses of double integrals.
The first one that I will mention is actually something you thought maybe you could do with a single integral, but it is useful very often to do it as a double integral.
It is to find the area of a given region r. I give you some region in the plane and you want to know just its area.
In various cases, you could set this up as a single integral, but often it could be useful to set it up as a double integral.
How do you express the area as a double integral?
Well, the area of this region is the sum of the areas of all the little pieces.
It means you want to sum one dA of the entire region.
The area R is the double integral over R of a function one.
One way to think about it, if you are really still attached to the idea of double integral as a volume, what this measures is the volume below the graph of a function one.
The graph of a function one is just a horizontal plane at height one.
What you would be measuring is the volume of a prism with base r and height one.
And the volume of that would be, of course, base times height.
It would just be the area of r again.
But we don't actually need to think about it that way.
Really, what we are doing is summing dA over the entire region.
A related thing we can do, imagine that, actually, this is some physical object.
I mean, it has to be a flat object because we are just dealing with things in the plane so far.
But you have a flat metal plate or something and you would like to know its mass.
Well, its mass is the sum of the masses of every single little piece.
You would get that by integrating the density.
The density for a flat object would be the mass per unit area.
So, you can get the mass of a flat object with density.
Let's use delta for density, which is the mass per unit area.
Each little piece of your object will have a mass, which will be just the density, times its area for each small piece.
And you will get the total mass by summing these things.
The mass will be the double integral of the density times the area element.
Now, if it has constant density, if it is always the same material then, of course, you could just take the density out and you will get density times the total area if you know that it is always the same material.
But if, actually, it has varying density maybe because it is some metallic thing with various metals or with varying thickness or something then you can still get the mass by integrating the density.
Of course, looking at flat objects might be a little bit strange.
That is because we are only doing double integrals so far.
In a few weeks, we will be triple integrals.
And then we will be able to do solids in space, but one thing at a time.
Another useful application is to find the average value of some quantity in a region.
What does it mean to take the average value of some function f in this region r?
Well, you know what the average of a finite set of data is.
For example, if I asked you to compute your average score on 18.02 problem sets, you would just take the scores, add them and divide by the number of problem sets.
What if there are infinitely many things?
Say I ask you to find the average temperature in this room.
Well, you would have to measure the temperature everywhere.
And then add all of these together and divide by the number of data points.
But, depending on how careful you are, actually, there are potentially infinitely many points to look at.
The mathematical way to define the average of a continuous set of data is that you actually integrate the function over the entire set of data, and then you divide by the size of the sample, which is just the area of the region.
In fact, the average of f, the notation we will use usually for that is f with a bar on top to tell us it is the average f. We say we will take the integral of f and we will divide by the area of the region.
You can really think of it as the sum of the values of f everywhere divided by the number of points everywhere.
And so that is an average where everything is, actually, equally likely.
That is a uniform average where all the points on the region, all the little points of the region are equally likely.
But maybe if want to do, say, an average of some solid with variable density or if you want to somehow give more importance to certain parts than to others then you can actually do a weighted average.
What is a weighted average?
Well, in the case of taking the average your problem sets, if I tell you problem set one is worth twice as much as the others, then you would count twice that score in the sum and then you would count it as two, of course, when you divide.
The weighted average is the sum of the values, but each weighted by a certain coefficient.
And then you will divide by the sum of the weight.
It is a bit the same idea as when we replace area by some mass that tells you how important a given piece.
We will actually have a density.
Let's call it delta again.
We will see what we divide by, but what we will take is the integral of a function times the density times the area element.
Because this would correspond to the mass element telling us how to weight the various points of our region.
And then we would divide by the total weight, which is the mass of a region, as defined up there.
If a density is uniform then, of course, the density gets out and you can simplify and reduce to that if all the points are equally likely.
Why is that important?
Well, that is important for various applications.
But one that you might have seen in physics, we care about maybe where is the center of mass of a given object?
The center of mass is basically a point that you would say is right in the middle of the object.
But, of course, if the object has a very strange shape or if somehow part of it is heavier than the rest then that takes a very different meaning.
Strictly speaking, the center of mass of a solid is the point where you would have to concentrate all the mass if you wanted it to behave equivalently from a point of view of mechanics, if you are trying to do translations of that object.
If you are going to push that object that would be really where the equivalent point mass would lie.
The other way to think about it, if I had a flat object then the center of mass would basically be the point where I would need to hold it so it is perfectly balanced.
And, of course, I cannot do this.
Well, you get the idea.
And the center of mass of this eraser is somewhere in the middle.
And so, in principle, that is where I would have to put my finger for it to stay.
Well, it doesn't work.
But that is where the center of mass should be.
I think it should be in the middle.
Maybe I shouldn't call this three.
I should call this 2a, because it is really a special case of the average value.
How do we find the center of mass of a flat object with density delta.
If you have your object in the x, y plane then its center of mass will be at positions that are actually just the coordinates of a center of mass, will just be weighted averages of x and y on the solid.
So, the center of mass will be a position that I will call x bar, y bar.
And these are really just the averages, the average values of x and of y in the solid.
Just to give you the formulas again, x bar would be one over the mass times the double integral of x times density dA.
And the same thing with y. y bar is the weighted average of a y coordinate in your region.
You see, if you take a region that is symmetric and has uniform density that will just give you the center of the region.
But if the region has a strange shape or if a density is not homogeneous, if parts of it are heavier then you will get whatever the weighted average will be.
And that will be the point where this thing would be balanced if you were trying to balance it on a pole or on your finger.
Any questions so far?
Yes.
No.
Here I didn't set this up as a iterated integral yet.
The function that I am integrating is x times delta where density will be given to me maybe as a function of x and y.
And then I will integrate this dA.
And dA could mean dx over dy, it could mean dy over dx, it could be mean r dr d theta.
I will choose how to set it up depending maybe on the shape of the region.
If my solid is actually just going to be round then I might want to use polar coordinates.
If it is a square, I might want to use x, y coordinates.
If it is more complicated, well, I will choose depending on how I feel about it.
Yes?
Delta is the density.
In general, it is a function of x and y.
If you imagine that your solid is not homogenous then its density will depend on which piece of it you are looking at.
Of course, to compute this, you need to know the density.
If you have a problem asking you to find the center of mass of something and you have no information about the density, assume it is uniform.
Take the density to be a constant.
Even take it to be a one.
That is even easier.
I mean it is a general fact of math.
We don't care about units.
If density is constant, we might as well take it to be one.
That just means our mass unit becomes the area unit.
Yes?
That is a good question.
No, I don't think we could actually find the center of mass in polar coordinates by finding the average of R or the average of theta.
For example, take a disk center at the origin, well, the center of mass should be at the origin.
But the average of R is certainly not zero because R is positive everywhere.
So, that doesn't work.
You cannot get the polar coordinates of a center of mass just by taking the average of R and the average of theta.
By the way, what is the average of theta?
If you take theta to from zero to 2pi, the average theta will be pi.
If you take it to go from minus pi to pi, the average theta will be zero.
So, there is a problem there.
That actually just doesn't work, so we really have to compute x bar and y bar.
But still we could set this up and then switch to polar coordinates to evaluate this integral.
But we still would be computing the average values of x and y.
We are basically re-exploring mechanics and motion of solids here.
The next thing is moment of inertia.
Just to remind you or in case you somehow haven't seen it in physics yet, the moment of inertia is basically to rotation of a solid where the mass is to translation.
In the following sense, the mass of a solid is what makes it hard to push it.
How hard it is to throw something is related to its mass.
How hard it is to spin something, on the other hand, is given by its moment of inertia.
Maybe I should write this down.
Mass is how hard it is to impart a translation motion to a solid.
I am using fancy words today.
And the moment of inertia -- The difference with a mass is that the moment of inertia is defined about some axis.
You choose an axis.
Then you would try to measure how hard it is to spin your object around that axis.
For example, you can try to measure how hard it is to spin this sheet of paper about an axis that is in the center of it.
We would try to spin it light that and see how much effort I would have to make.
Well, for a sheet of paper not very much.
That would measure the same thing but it would be rotation motion about that axis.
Maybe some of you know the definition but I am going to try to derive it again.
I am sorry but it won't be as quite as detailed as the way you have probably seen it in physics, but I am not trying to replace your physics teachers.
I am sure they are doing a great job.
What is the idea for the definition to find a formula for moment of inertia?
The idea is to think about kinetic energy.
Kinetic energy is really when you push something or when you try to make it move and you have to put some inertia to it.
Then it has kinetic energy.
And then, if you have the right device, you can convert back that kinetic energy into something else.
If you try to look at the kinetic energy of a point mass, so you have something with mass m going at the velocity v, well, that will be one-half of a mass times the square of the speed.
I hope you have all seen that formula some time before.
Now, let's say instead of just trying to push this mass, I am going to make it spin around something.
Instead of just somewhere, maybe I will have the origin, and I am trying to make it go around the origin in a circle at a certain angular velocity.
For a mass m at distance r, let's call r this distance.
And angular velocity, let's call the angular velocity omega.
I think that is what physicists call it.
Remember angular velocity is just the rate of the change of the angle over time.
It is d theta dt, if you want.
Well, what is the kinetic energy now?
Well, first we have to find out what the speed is.
What is the speed?
Well, if we are going on a circle of radius r at angular velocity omega that means that in unit time we rotate by omega and we go by a distance of r times omega.
The actual speed is the radius times angular velocity.
And so the kinetic energy is one-half mv squared, which is one-half m r squared omega squared.
And so, by similarity with that formula, the coefficient of v squared is the mass, and here we will say the coefficient of omega squared, so this thing is the moment of inertia.
That is how we define moment of inertia.
Now, that is only for a point mass.
And it is kind of fun to spin just a small bowl, but maybe you would like to spin actually a larger solid and try to define this moment of inertia.
Well, the moment inertia of a solid will be just the sum of the moments of inertia of all the little pieces.
What we will do is just cut our solid into little chunks and will sum this thing for each little piece.
For a solid with density delta, each little piece has mass which is the density times the amount of area.
This is equal actually.
And the moment of inertia of that small portion of a solid will be delta m, the small mass, times r squared, the square of a distance to the center of the axis along which I am spinning.
That means if I sum these things together, well, it has moment of inertia delta m times r squared, which is r squared times the density times delta A.
And so I will be summing these things together.
And so, the moment of inertia about the origin will be the double integral of r squared times density times dA.
The final formula for the moment of inertia about the origin is the double integral of a region of r squared density dA.
If you are going to do it in x, y coordinates, of course, r squared becomes x squared plus y squared, it is the square of the distance from the origin.
When you integrate this, that tells you how hard it is to spin that solid about the origin.
The motion that we try to do -- We keep this fixed and then we just rotate around the origin.
Sorry.
That is a pretty bad picture, but hopefully you know what I mean.
And the name we use for that is I0.
And then the rotational kinetic energy is one-half times this moment of inertia times the square of the angular velocity.
So that shows as that this replaces the mass for rotation motions.
OK. What about other kinds of rotations?
In particular, we have been rotating things about just a point in the plane.
What you could imagine also is instead you have your solid.
What I have done so far is I have skewered it this way, and I am rotating around the axis.
Instead, I could skewer it through, say, the horizontal axis.
And then I could try to spin about the horizontal axis so then it would rotate in space in that direction like that.
Let's say we do rotation about the x-axis.
Well, the idea would still be the same.
The moment of inertia for any small piece of a solid would be its mass element times the square of a distance to the x axes because that will be the radius of a trajectory.
If you take this point here, it is going to go in a circle like that centered on the x-axis.
So the radius will just be this distance here.
Well, what is this distance?
It is just y, or maybe absolute value of y.
Distance to x-axis is absolute value of y.
What we actually care about is the square of a distance, so it will just be y squared.
The moment of inertia about the x-axis is going to be obtained by integrating y squared times the mass element.
It is slightly strange but I have y in inertia about the x-axis.
But, if you think about it, y tells me how far I am from the x-axis, so how hard it will be to spin around the x-axis.
And I could do the same about any axis that I want.
Just I would have to sum the square of a distance to the axis of rotation.
Maybe I should do an example.
Yes?
Same thing as above, distance to the x-axis, because that is what we care about.
For the moment of inertia, we want the square of a distance to the axis of rotation.
Let's do an example.
Let's try to figure out if we have just a uniform disk how hard it is to spin it around its center.
That shouldn't be very hard to figure out.
Say that we have a disk of radius a and we want to rotate it about its center.
And let's say that it is of uniform density.
And let's take just the density to be a one so that we don't really care about the density.
What is the moment of inertia of that?
Well, we have to integrate of our disk r squared times the density, which is one, times dA.
What is r squared?
You have here to resist the urge to say the radius is just a.
We know the radius is a.
No, it is not a because we are looking at rotation of any point inside this disk.
And, when you are inside the disk, the distance to the origin is not a.
It is less than a.
It is actually anything between zero and a.
Just to point out a pitfall, r here is really a function on this disk.
And we are going to integrate this function.
Don't plug r equals a just yet.
What coordinates do we use to compute this integral?
They are probably polar coordinates, unless you want a repeat of what happened already with x and y.
That will tell us we want to integrate r squared time r dr d theta.
And the bounds for r, well, r will go from zero to a.
No matter which direction I go from the origin, if I fixed it, r goes from zero to r equals a.
The part of this ray that lives inside the disk is always from zero to a.
And theta goes from, well, zero to 2 pi for example.
And now you can compute this integral.
Well, I will let you figure it out.
But the inner integral becomes a to the four over four and the outer multiplies things by 2pi, so you get pi a to the four over two.
OK. That is how hard it is to spin this disk.
Now, what about instead of spinning it about the center we decided to spin it about a point on a second point.
For example, think of a Frisbee.
A Frisbee has this rim so you can actually try to make it rotate around the point on the circumference by holding it near the rim and spinning it there.
How much harder is that than around the center?
Well, we will try to compute now the moment of inertia about this point.
We have two options.
One is we keep the system of coordinates centers here.
But then the formula for distance to this point becomes harder.
The other option, which is the one I will choose, is to change the coordinate so that this point become the origin.
Let's do that.
About a point on the circumference, what I would have to do maybe is set up my region like that.
I have moved the origin so that it is on the circumference of a disk, and I will again try to find the moment of inertia of this disk about the origin.
It is still, for the the double integral of r squared dA.
But now I want to find out how to set up the integral.
I could try to use x, y coordinates and it would work.
Or I can use polar coordinates, and it works a little bit better that way.
But both are doable.
Let's say I do it this way.
I have to figure out how to set up my bounds.
What are the bounds for r?
Well, if I fix a value for theta, which means I chose an angle here, now I am shooting a ray from the origin in that direction.
I enter my region at r equals zero.
That hasn't changed.
The question is where do I exit the region?
What is that distance?
Maybe you have seen it in recitation, maybe not.
Let's see.
Actually, I should have written down the radius of a circle is a.
So this distance here is 2a.
If you draw this segment in here, you know that here you have a right angle.
You have a right triangle.
The hypotenuse here has length 2a.
This angle is theta.
Well, this length is 2a cosine theta.
The polar coordinates equation of this circle passing through the origin is r equals 2a cosine theta.
So, r will go from zero to 2a cosine theta.
That is the distance here.
Now, what are the bounds for theta?
It is not quite zero to 2pi because, actually, you see in this direction, if I shoot a ray in this direction I will never meet my region.
We have to actually think a bit more.
Well, the directions in which I will actually hit my circle are all the directions in the right half of a plane.
I mean, of course, if I shoot very close to the axis, you might think, oh, I won't be in there.
But, actually, that is not true because here the circle is tangent to the axis.
No matter which direction I take, I will still have a little tiny piece.
The angle actually goes from minus pi over two to pi over two.
If you compute that you will get, well, the inner integral will be r to the four over four between zero and 2a cosine theta, which will turn out to be 4a to the four cosine to the four theta.
And now you will integrate that for minus pi over two to pi over two.
And that is, again, the evil integral that we had yesterday.
Either we remember the method from yesterday or we remember from yesterday that actually there are formulas in the notes to help you.
On homework, you can use these formulas.
In the notes at the beginning of section 3b there are formulas for these particular kinds of integrals.
And that will end up being three-halves of pi a to the four.
In case you wanted to know, it is three times harder to spin a Frisbee about a point on a circumference than around the center.
We got three times the moment of inertia about the center.
OK. That is it.
Have a nice weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so far we've learned how to do double integrals in terms of xy coordinates, also how to switch to polar coordinates.
But, more generally, there's a lot of different changes of variables that you might want to do.
OK, so today we're going to see how to change variables, if you want, how to do substitutions in double integrals.
OK, so let me start with a simple example.
Let's say that we want to find the area of an ellipse with semi-axes a and b. OK, so that means an ellipse is just like a squished circle.
And so, there's a and there's b.
And, the equation of that ellipse is x over a squared plus y over b squared equals one.
That's the curve, and the inside region is where this is less than one.
OK, so it's just like a circle that where you have rescaled x and y differently.
So, let's say we want to find the area of it.
Maybe you know what the area is.
But let's do it as a double integral.
So, you know, if you find that the area is too easy, you can integrate any function other than ellipse, if you prefer.
But, let's do it just with area.
So, we know that we want to integrate just the area element, let's say, dx dy over the origin inside the ellipse.
That's x over a2 plus y over b2 less than 1.
Now, we can try to set this up in terms of x and y coordinates, you know, set up the bounds by solving for first four x as a function of y if we do it this order and, well, do the usual stuff.
That doesn't look very pleasant, and it's certainly not the best way to do it.
OK, if this were a circle, we would switch to polar coordinates.
Well, we can't quite do that yet.
But, you know, an ellipse is just a squished circle.
So, maybe we want to actually first rescale x and y by a and b.
So, to do that, what we'd like to do is set x over a to be u, and y over b be v. So, we'll have two new variables, u and v, and we'll try to redo our integral in terms of u and v. So, how do we do the substitution?
So, in terms of u and v, the condition, the region that we are integrating on will become u^2 v^2 is less than 1, which is arguably nicer than the ellipse.
That's why we are doing it.
But, we need to know what to do with dx and dy.
Well, here, the answer is pretty easy because we just change x and y separately.
We do two independent substitutions.
OK, so if we set u equals x over a, that means du is one over adx.
And here, dv is one over bdy.
So, it's very tempting to write, and here we actually can write, in this particular case, that dudv is (1/ab)dxdy, OK?
So, let me rewrite that.
OK, so I get dudv equals (1/ab)dxdy, or equivalently dxdy is ab times dudv.
OK, so in my double integral, I'm going to write (ab)dudv.
OK, so now, my double integral becomes, well, the double integral of a constant in terms of u and v. So, I can take the constant out.
I will get ab times double integral over u^2 v^2<1 of du dv.
And, that is an integral that we know how to do.
Well, it's just the area of a unit circle.
So, we can just say, this is ab times the area of the unit disk, which we know to be pi, or if somehow you had some function to integrate, then you could have somehow switched to polar coordinates, you know, setting u equals r times cos(theta), v equals r times sin(theta), and then doing it in polar coordinates.
OK, so here the substitution worked pretty easy.
The question is, if we do a change of variables where the relation between x and y and u and v is more complicated, what can we do?
Can we still do this, or do we have to be more careful?
And, actually, we have to be more careful.
So, that's what we are going to see next.
Any question about this, first?
No?
OK. OK, so, see the general problem when we try to do this is to figure out what is the scale factor?
What's the relation between dxdy and dudv?
We need to find the scaling factor.
So, we need to find dxdy versus dudv.
So, let's do another example that's still pretty easy, but a little bit less easy.
OK, so let's say that for some reason, we want to do the change of variables: u equals 3x-2y, and v equals x y.
Why would we want to do that?
Well, that might be to simplify the integrand because we are integrating a function that happens to be actually involving these guys rather than x and y.
Or, it might be to simplify the bounds because maybe we are integrating over a region whose equation in xy coordinates is very hard to write down.
But, it becomes much easier in terms of u and v. And then, the bounds would be much easier to set up with u and v. Anyway, so, whatever the reason might be, typically it would be to simplify the integrant or the bounds.
Well, how do we convert dxdy to dudv?
So, we want to understand, what's the relation between, let's call dA the area element in xy coordinates.
So, dA is dxdy, maybe dydx depending on the order.
And, the area element in uv coordinates, let me call that dA prime just to make it look different.
So, that would just be dudv, or dvdu depending on which order I will want to set it up in.
So, to find this relation, it's probably best to draw a picture to see what happens.
Let's consider a small piece of the xy plane with area delta(A) corresponding to just a box with sides delta(y) and delta(x).
OK, and let's try to figure out what it will look like in terms of u and v. And then, we'll say, well, when we integrate, we're really summing the value of the function of a lot of small boxes times their area.
But, the problem is that the area of the box in here is not the same as the area of the box in uv coordinates.
There, maybe it will look like, actually, if you see that these are linear changes of variables, you know that the rectangle will become a parallelogram after the change of variables.
So, the area of a parallelogram delta(A) prime, well, we will have to figure out how they are related so that we can decide what conversion factor, what's the exchange rate between these two currencies for area?
OK, any questions at this point?
No?
Still with me mostly?
I see a lot of tired faces.
Yes?
Why is delta(A) prime a parallelogram?
That's a very good question.
Well, see, if I look at the side of a rectangle, say there's a vertical side, it means I'm going to increase y, keeping x the same.
If I look at the formulas for u and v, they are linear formulas in terms of x and y.
So, if I just increase y, see that u is going to decrease at a rate of two.
v is going to increase at a rate of one at constant rates.
And, it doesn't matter whether I was looking at this site or at that site.
So, basically straight lines become straight lines.
And if they are parallel, they stay parallel.
So, if you just look at what the transformation, from xy to uv does, it does this kind of thing.
Actually, this transformation here you can express by a matrix.
And, remember, we've seen what matrices do the pictures.
We just take straight lines to straight lines.
They keep the notion of being parallel, but of course they mess up lengths, angles, and all that.
OK, so let's see.
So, let's try to figure out, what is the area of this guy?
Well, in fact, what I've been saying about this transformation being linear, and transforming all of the vertical lines in the same way, all the horizontal lines in the same way, it tells me, also, I should have a constant scaling factor, right, because how much I've scaled my rectangle doesn't depend on where my rectangle is.
If I move my rectangle to somewhere else, I have a rectangle of the same size, same shape, it will become a parallelogram of the same size, same shape somewhere else.
So, in fact, I can just take the simplest rectangle I can think of and see how its area changes.
And, if you don't believe me, then try with any other rectangle.
You will see it works exactly the same way.
OK, so I claim that the area scaling factor -- -- here in this case doesn't depend on the choice of the rectangle.
And I should say that because we are actually doing a linear change of variables -- So, you know, somehow, the exchange rate between uv and xy is going to be the same everywhere.
So, let's try to see what happens to the simplest rectangle I can think of, namely, just the unit square.
And, you know, if you don't trust me, then while I'm doing this one, do it with a different rectangle.
Do the same calculation, and see that you will get the same conversion ratio.
So, let's say that I take a unit square -- -- so, something that goes from zero to one both in x and y directions.
OK, and let's try to figure out what it looks like on the other side.
So, here the area is one.
Let's try to draw it in terms of u and v coordinates, OK?
So, here we have x equals 0, y equals 0.
Well, that tells us u and v are going to be 0.
Next, let's look at this corner.
Well, in xy coordinates, this is one zero.
If you plug x equals 1, y equals 0, you get u equals 3; v equals 1.
So, that goes somewhere here.
And so, this edge of the square will become this line here, OK?
Next, let's look at that point.
So that point here was (0,1).
If I plug x equals zero y equals one I will get (-2,1).
So, this edge goes here.
Then, if you put x equals one, y equals one, you will get u equals 1, v equals 2.
So, I want (1,2).
And, these edges will go to these edges here.
And, you see, it does look like a parallelogram.
OK, so now what the area of this parallelogram?
Well, we can get that by taking the determinant of these two vectors.
So, one of them is , and the other one is .
That will be 3 2.
That's 5.
OK, this parallelogram is apparently five times the size of this square.
Here, it looks like it's less because I somehow changed my scale.
I mean, my unit length is smaller here than here.
But, it should be a lot bigger than that.
OK, and if you do the same calculations not with zero and one, but with x and x plus delta x, and so on, you will still find that the area has been multiplied by five.
So, that tells us, actually for any other rectangle, area is also multiplied by five.
So, that tells us that dA prime, the area element in uv coordinate is worth five times more than the area element in the xy coordinate.
So, that means du dv is worth five times dx dy.
What's so funny?
What?
Oh.
[LAUGHTER] OK, rectangle.
OK, is that OK now?
Did I misspell other words?
No?
OK, it's really hard to see when you are up close.
It's much easier from a distance.
OK, so yeah, so we've said our transformation multiplies areas by five.
And so, dudv is five times dxdy.
So, if I'm integrating some function, dx dy, then when I switch to uv coordinates, I will have to replace that by one fifth dudv.
OK, and of course I would also, here my function would probably involve x and y. I will replace them by u's and v's.
And, the bounds, well, the shape of my origin in the xy coordinates I will have to switch to some shape in the uv coordinates.
And, that's also something that might be easy or might be tricky depending on what origin we are looking at.
So, usually we will do changes of variables to actually simplify the region so it becomes easier to set up the bounds.
So, anyway, so this is kind of an illustration of a general case.
And, why is that?
Well, here it looks very easy.
We are just using linear formulas, and somehow the relation between dx dy and du dv is the same everywhere.
If you take actually more complicated changes of variables that's not true because usually you will expect that there are some places where the rescaling is enlarging things, and some of other places where things are shrunk, so, certainly the exchange rate between dudv and dxdy will fluctuate from point to point.
It's the same as if you're trying to change dollars to euros.
It depends on where you do it.
You will get a better rate or a worse one.
So, of course, we'll get a formula where actually this scaling factor depends on x and y or on u and v. But, if you fix a point, then we have linear approximation.
And, linear approximation tells us, oh, we can do as if our function is just a linear function of x and y.
So then, we can do it the same way we did here.
OK, so let's try to think about that.
So, in the general case, well, that means we will replace x and y by new coordinates, u and v. And, u and v will be some functions of x and y.
So, well, we'll have an approximation formula which tells us that the change in u, if I change x or y a little bit, will be roughly (u sub x times change in x) (u sub y times change in y).
And, the change in v will be roughly (v sub x delta x) (v sub y delta y).
Or, the other way to say it, if you want in matrix form is delta u delta v is, sorry, approximately equal to matrix |u sub x, u sub y, v sub x, v sub y| times matrix |delta x, delta y|, OK?
So, if we look at that, what it tells us, in fact, is that if we take a small rectangle in xy coordinates, so that means we have a certain point, x, y, and then we have a certain width.
This is going to be too small.
Well, so, I have my width, delta x. I have my height, delta y.
This is going to correspond to a small uv parallelogram.
And, what the shape and the size of the parallelogram are depends on the partial derivatives of u and v. So, in particular, it depends on at which point we are.
But still, at a given point, it's a bit like that.
And, so if we do the same argument as before, what we will see is that the scaling factor is actually the determinant of this transformation.
So, that's one thing that maybe we didn't emphasize enough when we did matrices at the beginning of a semester.
But, when you have a linear transformation between variables, the determinant of that transformation represents how it scales areas.
OK, so one way to think about it is just to try it and see what happens.
Take this side.
This side in x, y coordinates corresponds to delta x and zero.
And, now, if you take the image of that, if you see what happens to delta u and delta v, that will be basically u sub x delta x and v sub x delta x.
There's no delta y.
For the other side, OK, so maybe I should do it actually.
So, you know, if we move in the x, y coordinates by delta x and zero, then delta u and delta v will be approximately u sub x delta x, and v sub x delta x.
And, on the other hand, if you move in the other direction along the other side of your rectangle, zero and delta y, then the change in u and the change in v will correspond to, well, how does u change?
That's u sub y delta y, and v changes by v sub y delta y.
And so, now, if you take the determinant of these two vectors, OK, so these are the sides of your parallelogram up here.
And, if you take these sides to get the area of the parallelogram, you'll need to take the determinant.
And, the determinant will be the determinant of this matrix times delta x times delta y.
So, the area in uv coordinates will be the determinant of a matrix times delta x, delta y.
And so, what I'm trying to say is that when you have a general change of variables, du dv versus dx dy is given by the determinant of this matrix of partial derivatives.
It doesn't matter in which order you write it.
I mean, you can put in rows or columns.
If you transpose a matrix, that doesn't change the determinant.
It's just any sensible matrix that you can write will have the correct determinant.
OK, so what we need to know is the following thing.
So, we define something called the Jacobian of a change of variables and used the letter J, or maybe a more useful notation is partial of u, v over partial of x, y.
That's a very strange notation.
I mean, that doesn't mean that we are actually taking the partial derivatives of anything.
OK, it's just a notation to remind us that this has to do with the ratio between dudv and dxdy.
And, it's obtained using the partial derivatives of u and v with respect to x and y.
So, it's the determinant of the matrix |u sub x, u sub y, v sub x, v sub y|, the matrix that I had up there.
OK, and what we need to know is that du dv is equal to the absolute value of J dx dy.
Or, if you prefer to see it in the easier to remember version, it's (absolute value of d of (u, v) over partial xy) times dx dy.
OK, so this is just what you need to remember, and it says that the area in uv coordinates is worth, well, the ratio to the xy coordinates is given by this Jacobian determinant except for one small thing.
It's given by, actually, the absolute value of this guy.
OK, so what's going on here?
What's going on here is when we are saying the determinant of the transformation tells us how the area is multiplied, there's a small catch.
Remember, the determinants are equal to areas up to sine.
Sometimes, the determinant is negative because of reversing the orientation of things.
But, the area is still the same.
Area is always positive.
So, the area elements are actually related by the absolute value of this guy.
OK, so if you find -10 as your answer, then du dv is still ten times dx dy.
OK, so I didn't put it all together because then you would have two sets of vertical bars.
See, this is a vertical bar for absolute value.
This is vertical bar for determinant.
They're not the same.
That's the one thing to remember.
OK, any questions about this?
No?
OK.
So, actually let's do our first example of that.
Let's check what we had for polar coordinates.
Last time I told you if we have dx dy we could switch it to r dr d theta.
And, we had some argument for that by looking at the area of a small circular sector.
But, let's check again using this new method.
So, in polar coordinates I'm setting x equals r cosine theta, y equals r sine theta.
So, the Jacobian for this change of variables, so let's say I'm trying to find the partial derivatives of x, y with respect to r, theta.
Well, what is, OK, let me actually write them here again for you.
And, so what does that become?
Partial x over partial r is just cosine theta.
Partial x over partial theta is negative r sine theta.
Sorry, I guess I'm going to run out of space here.
So, let me do it underneath.
So, we said x sub r is cosine theta; x sub theta is negative r sine theta.
y sub r is sine; y sub theta is r cosine.
And now, if we compute this determinant, we'll get (r cosine squared theta) (r sine squared theta).
And, that simplifies to r. So, dx dy is, well, absolute value of r dr d theta.
But, remember that r is always positive.
So, it's r dr d theta.
OK, so that's another way to justify how we did double integrals in polar coordinates.
OK, any questions on that?
Where?
Yeah, OK. Yeah, so this one seems to be switching.
Well, it depends what you do.
So, OK, actually here's an important thing that I didn't quite say.
So, I said, you know, we are going to switch from xy to uv.
We can also switch from uv to xy.
And, this conversion ratio, the Jacobian, works both ways.
Once you have found the ratio between du dv and dx dy, then it works one way or it works the other way.
I mean, here, of course, we get the answer in terms of r. So, this would let us switch from xy to r theta.
But, we can also switch from r theta to xy.
Just, we'd write dr d theta equals (1 over r) times dx dy.
And then we'd have, of course, to replace r by its formula in xy coordinates.
Usually, we don't do that.
Usually, we actually start with xy and switch to polar.
But, so in general, when you have this formula relating du dv with dx dy, you can use it both ways, either to switch from du dv to dx dy or the other way around.
And, the thing that I'm not telling you that now I should probably tell you is I could define two Jacobians because if I solve for xy in terms of uv instead of uv in terms of xy, then I can compute two different Jacobians.
I can compute partial uv over partial xy, or I can compute partial xy over partial uv if I have the formulas both ways.
Well, the good news is these guys are the inverse of each other.
So, the two formulas that you might get are consistent.
OK, so useful remark -- So, say that you can compute both -- -- these guys.
Well, then actually, the product will just be 1.
So, they are the inverse of each other.
So, it doesn't matter which one you compute.
You can compute whichever one is the easiest to compute no matter which one of the two you need.
And, one way to see that is that, in fact, we're looking at the determinant of these matrices that tell us the relation in variables.
So, if one of them tells you how delta u delta v relate to delta x delta y, the other one does the opposite thing.
It means they are the inverse matrices.
And, the determinant of the inverse matrix is the inverse of the determinant.
So, they are really interchangeable.
I mean, you can just compute whichever one is easiest.
So here, if you wanted, dr d theta in terms of dx dy, it's easier to do this and then move the r over there than to first solve for r and theta as functions of x and y and then do the entire thing again.
But, you can do it if you want.
I mean, it works.
Oh yeah, the other useful remark, so, I mentioned it, but let me emphasize again.
So, now, the ratio between du dv and dx dy, it's not a constant anymore, although there it used to be five.
But now, it's become r, or anything.
In general, it will be a function that depends on the variables.
So, it's not true that you can just say, oh, I'll put a constant times du dv.
Yes?
It would still work the same.
You could imagine drawing a picture where r and theta are the Cartesian coordinates, and your picture would be completely messed up.
It would be a very strange thing to do to try to draw, you know, I'm going to do it, but don't take notes on that.
You could try to draw picture like that, and then a circle would start looking like, you know, a disk would look like that.
It would be very counterintuitive.
But, you could do it.
And that would be equivalent to what we did with a previous change of variables.
So, in this case, certainly you would never draw a picture like that.
But, you could do it.
OK, so now let's do a complete example to see how things fit together, how we do everything.
So, let's say that we want to compute, so I have to warn you, it's going to be a very silly example.
It's an example where it's much easier to compute things without the change of variables.
But, you know, it's good practice in the sense that we're going to make it so complicated that if we can do this one, then we can do that one.
So, let's say that we want to compute this.
And, of course, it's very easy to compute it directly.
But let's say that for some evil reason we want to do that by changing variables to u equals x and v equals xy.
OK, that's a very strange idea, but let's do it anyway.
I mean, normally, you would only do this kind of substitution if either it simplifies a lot the function you are integrating, or it simplifies a lot the region on which you are integrating.
And here, neither happens.
But anyway, so the first thing we have to do here is figure out what we are going to be integrating.
OK, so to do that, we should figure out what dx dy will become in terms of u and v. So, that's what we've just seen using the Jacobian.
OK, so the first thing to do is find the area element.
And, for that, we use the Jacobian.
So, well, let's see, the one that we can do easily is partials of u and v with respect to x and y. I mean, the other one is not very hard because here you can solve easily.
But, the one that's given to you is partial of u and v with respect to x and y, so partial u partial x is one.
Partial u partial y is zero.
Partial v partial x is y.
And partial v partial y is x.
So that's just x.
So, that means that du dv is x dx dy.
Well, it would be absolute value of x, but x is positive in our origin.
So, at least we don't have to worry about that.
OK, so now that we have that, we can try to look at the integrand in terms of u and v. OK, so we were integrating x squared y dx dy.
So, let's switch it.
Well, let's first switch the dx dy that becomes one over x du dv.
So, that's actually xy du dv.
And, what is xy in terms of u and v?
Well, here at least we had a little bit of luck.
xy is just v. So, that's v du dv.
So, in fact, what we'll be computing is a double integral over some mysterious region of v du dv.
Now, last but not least, we'll have to find what are the bounds for u and v in the new integral so that we know how to evaluate this.
In fact, well, we could do it du dv or dv du.
We don't know yet.
Oh, amazing.
It went all the way down this time.
OK, so it could be dv du if that's easier.
So, let's try to find the bounds.
In this case, that's the hardest part.
OK, so let me draw a picture in xy coordinates and try to understand things using that.
OK, so x and y go from zero to one.
The region that we want to integrate over was just this square.
Let's try to figure out how u and v vary there.
So, let's say that we're going to do it du dv.
OK, so What we want to understand is how u and v vary in here.
What's going to happen?
So, the way we can think about it is we try to figure out how we are slicing our origin.
OK, so here, we are integrating first over u.
That means we start by keeping u constant, no, by keeping v constant as u changes.
OK, so u changes as v is constant.
What does it mean that I'm keeping v constant.
Well, what is v?
v is xy.
So, that means I keep xy equals constant.
What does the curve xy equals constant look like?
Well, it's just a hyperbola.
y equals constant over x.
So, if I look at the various values of v that I can take, for each value of v, if I fix a value of v, I will be moving on one of these red curves.
OK, and u, well, u is the same thing as x.
So, that means u will increase.
Here, maybe it will be 0.1 and it will increase all the way to one here.
OK, so we are just traveling on each of these slices.
Now, so the question we must answer here is for a given value of v, what are the bounds for u?
So, I'm traveling on my curve, v equals constant, and trying to figure out, when do I enter my origin?
When do I leave it?
Well, I enter it when I go through this side.
So, the question is, what's the value of u here?
Well, we don't know that very easily until we look at these formulas.
So, u equals x, OK, but we don't know what x is at that point.
v equals x and v equals xy.
What do we go here?
Well, we don't know x, but we know y certainly.
OK, so let's forget about trying to find u.
And, let's say, for now, we know y equals one.
Well, if we set y equals one, that tells us that u and v are both equal to x.
So, in terms of u and v, the equation of this uv coordinate is u equals v. OK, I mean, the other way to do it is, say that you know you want y equals one.
You want to know what is y in terms of u and v. Well, it's easy.
y is v over u.
So, let me actually add an extra step in case that's, so, we know that y is v over u equals one.
So, that means u=v is my equation.
OK, so when I'm here, when I'm entering my region, the value of u at this point is just v, u equals v. That's the hard part.
Now, we need to figure out, so, we started u equals v. u increases, increases, increases.
Where does it exit?
It exits one when we are here.
What's the value of u here?
One.
That one is easier, right?
This side here, so, this side here is x equals one.
That means u equals one.
So, we start at u equals one.
Now, we've done the inner integral.
What about the outer?
So, we have to figure out, what is the first and what is the last value of v that we'll want to consider?
Well, if you look at all these hyperbola's, xy equals constant.
What's the smallest value of xy that we'll ever want to look at in here?
Zero, OK. Let me actually, where's my yellow chalk?
Is it, no, ah.
So, this one here, that's actually v=0.
So, we'll start at v equals zero.
And, what's the last hyperbola we want to look at?
Well, it's the one that's right there in the corner.
It's this one here.
And, that's v equals one.
So, v goes from zero to one.
OK, and now, we can compute this.
I mean, it's not particularly easier than that one, but it's not harder either.
How else could we have gotten these bounds, because that was quite evil.
So, I would like to recommend that you try this way in case it works well.
Just try to picture, what are the slices in terms of u and v, and how you travel on them, where you enter, where you leave, staying in the xy picture.
If that somehow doesn't work well, another way is to draw the picture in the uv coordinates.
So, switch to a uv picture.
So, what do I mean by that?
Well, we had here a picture in xy coordinates where we had our sides.
And, we are going to try to draw what it looks like in terms of u and v. So, here we said this is x equals one.
That becomes u equals one.
So, we'll draw u equals one.
This side we said is y equals one becomes u equals v. That's what we've done over there.
OK, so u equals v. Now, we have the two other sides to deal with.
Well, let's look at this one first.
So, that was x equals zero.
What happens when x equals zero?
Well, both u and v are zero.
So, this side actually gets squished in the change of variables.
It's a bit strange, but it's a bit the same thing as when you switch to polar coordinates at the origin, r is zero but theta can be anything.
It's not always one point is one point.
So anyway, this is the origin, and then the last side, y equals zero, and x varies just becomes v equals zero.
So, somehow, in the change of variables, this square becomes this triangle.
And now, if we want to integrate du dv, it means we are going to slice by v equals constant.
So, we are going to integrate over slices like this, and you see for each value of v, we go from u equals v to u equals one.
And, v goes from zero to one.
OK, so you get the same bounds just by drawing a different picture.
So, it's up to you to decide whether you prefer to think on this picture or draw that one instead.
It depends on which problems you're doing.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK. Today we have a new topic, and we are going to start to learn about vector fields and line integrals.
Last week we had been doing double integrals.
For today we just forget all of that, but don't actually forget it.
Put it away in a corner of your mind.
It is going to come back next week, but what we do today will include line integrals.
And these are completely different things, so it helps, actually, if you don't think of double integrals at all while doing line integrals.
Anyway, let's start with vector fields.
What is a vector field?
Well, a vector field is something that is of a form, while it is a vector, but while M and N, the components, actually depend on x and y,on the point where you are.
So, they are functions of x and y.
What that means, concretely, is that every point in the plane you have a vector.
In a corn field, every where you have corn.
In a vector field, everywhere you have a vector.
That is how it works.
A good example of a vector field, I don't know if you have seen these maps that show the wind, but here are some cool images done by NASA.
Actually, that is a picture of wind patterns off the coast of California with Santa Ana winds, in case you are wondering what has been going on recently.
You have all of these vectors that show you the velocity of the air basically at every point.
I mean, of course you don't draw it every point, because if you drew a vector at absolutely all the points of a plane then you would just fill up everything and you wouldn't see anything.
So, choose points and draw the vectors at those points.
Here is another cool image, which is upside down.
That is a hurricane off the coast of Mexico with the winds spiraling around the hurricane.
Anyway, it is kind of hard to see.
You don't really see all the vectors, actually, because the autofocus is having trouble with it.
It cannot really do it, so I guess I will go back to the previous one.
Anyway, a vector field is something where at each point -- -- in the plane we have vector F that depends on x and y.
This occurs in real life when you look at velocity fields in a fluid.
For example, the wind.
That is what these pictures show.
At every point you have a velocity of a fluid that is moving.
Another example is force fields.
Now, force fields are not something out of Star Wars.
If you look at gravitational attraction, you know that if you have a mass somewhere, well, it will be attracted to fall down because of the gravity field of the earth, which means that at every point you have a vector that is pointing down.
And, the same thing in space, you have the gravitational field of planets, stars and so on.
That is also an example of a vector field because, wherever you go, you would have that vector.
And what it is depends on where you are.
The examples from the real world are things like velocity in a fluid or force field where you have a force that depends on the point where you are.
We are going to try to study vector fields mathematically.
We won't really care what they are most of the time, but, as we will explore with them defined quantities and so on, we will very often use these motivations to justify why we would care about certain quantities.
The first thing we have to figure out is how do we draw a vector field, you know, how do you generate a plot like that?
Let's practice drawing a few vector fields.
Well, let's say our very first vector field will be just 2i j.
It is kind of a silly vector field because it doesn't actually depend on x and y.
That means it is the same vector everywhere.
I take a plane and take vector .
I guess it points in that direction.
It is two units to the right and one up.
And I just put that vector everywhere.
You just put it at a few points all over the place.
And when you think you have enough so that you understand what is going on then you stop.
Here probably we don't need that many.
I mean here I think we get the picture.
Everywhere we have a vector .
Now, let's try to look at slightly more interesting examples.
Let's say I give you a vector field x times i hat.
There is no j component.
How would you draw that?
Well, first of all, we know that this guy is only in the i direction so it is always horizontal.
It doesn't have a j component.
Everywhere it would be a horizontal vector.
Now, the question is how long is it?
Well, how long it is depends on x.
For example, if x is zero then this will actually be the zero vector.
x is zero here on the y-axis.
I will take a different color.
If I am on the y-axis, I actually have the zero vector.
Now, if x becomes positive small then I will have actually a small positive multiple of i so I will be going a little bit to the right.
And then, if I increase x, this guy becomes larger so I get a longer vector to the right.
If x is negative then my vector field points to the left instead.
It looks something like that.
Any questions about that picture?
No.
OK. Usually, we are not going to try to have very accurate, you know, we won't actually take time to plot a vector field very carefully.
I mean, if we need to, computers can do it for us.
It is useful to have an idea of what a vector field does roughly.
Whether it is getting larger and larger, in what direction it is pointing, what are the general features?
Just to do a couple of more, actually, you will see very quickly that the examples I use in lecture are pretty much always the same ones.
We will be playing a lot with these particular vector fields just because they are good examples.
Let's say I give you xi yj.
That one has an interesting geometric significance.
If I take a point (x, y), there I want to take a vector x, y.
How do I do that?
Well, it is the same as a vector from the origin to this point.
I take this vector and I copy it so that it starts at one point.
It looks like that.
And the same thing at every point.
It is a vector field that is pointing radially away from the origin, and its magnitude increases with distance from the origin.
You don't have to draw as many as me, but the idea is this vector field everywhere points away from the origin.
And its magnitude is equal to the distance from the origin.
If these were, for example, velocity fields, well, you would see visually what is happening to your fluid.
Like here maybe you have a source at the origin that is pouring fluid out and it is flowing all the way away from that.
Let's do just a last one.
Let's say I give you minus y, x.
What does that look like?
That is an interesting one, actually.
Let's say that I have a point (x, y) here.
This vector here is <x, y>.
But the vector I want is <- y, x>.
What does that look like?
It is perpendicular to the position to this vector.
If I rotate this vector, let me maybe draw a picture on the side, and take vector x, y.
A vector with components negative y and x is going to be like this.
It is the vector that I get by rotating by 90 degrees counterclockwise.
And, of course, I do not want to put that vector at the origin.
I want to put it at the point x, y.
In fact, what I will draw is something like this.
And similarly here like that, like that, etc.
And if I am closer to the origin then it looks a bit the same, but it is shorter.
And at the origin it is zero.
And when I am further away it becomes even larger.
See, this vector field, if it was the motion of a fluid, it would correspond to a fluid that is just going around the origin in circles rotating at uniform speed.
This is actually the velocity field for uniform rotation.
And, if you figure out how long it takes for a particle of fluid to go all the way around, that would be actually 2(pi) because the length of a circle is 2(pi) times the radius.
That is actually at unit angular velocity, one radiant per second or per unit time.
That is why this guy comes up quite a lot in real life.
And you can imagine lots of variations on these.
Of course, you can also imagine vector fields given by much more complicated formulas, and then you would have a hard time drawing them.
Maybe you will use a computer or maybe you will just give up and just do whatever calculation you have to do without trying to visualize the vector field.
But if you have a nice simple one then it is worth doing it because sometimes it will give you insight about what you are going to compute next.
Any questions first about these pictures?
No.
OK. Oh, yes?
You are asking if it should be y, negative x. I think it would be the other way around.
See, for example, if I am at this point then y is positive and x is zero.
If I take y, negative x, I get a positive first component and zero for the second one.
So, y, negative x would be a rotation at unit speed in the opposite direction.
And there are a lot of tweaks you can do to it.
If you flip the sides you will get rotation in the other direction.
Yes?
How do know that it is at unit angular velocity?
Well, that is because if my angular velocity is one then that means the actually speed is equal to the distance from the origin.
Because the arch length on a circle of a certain radius is equal to the radius times the angle.
If the angle varies at rate one then I travel at speed equal to the radius.
That is what I do here.
The length of this vector is equal to the distance of the origin.
I mean, it is not obvious on the picture.
But, really, the vector that I put here is the same as this vector rotated so it has the same length.
That is why the angular velocity is one.
It doesn't really matter much anyway.
What are we going to do with vector fields?
Well, we are going to do a lot of things but let's start somewhere.
One thing you might want to do with vector fields is I am going to think of now the situation where we have a force.
If you have a force exerted on a particle and that particles moves on some trajectory then probably you have seen in physics that the work done by the force corresponds to the force dot product with the displacement vector, how much you have moved your particle.
And, of course, if you do just a straight line trajectory or if the force is constant that works well.
But if you are moving on a complicated trajectory and the force keeps changing then, actually, you want to integrate that over time.
The first thing we will do is learn how to compute the work done by a vector field, and mathematically that is called a line integral.
Physically, remember the work done by a force is the force times the distance.
And, more precisely, it is actually the dot product between the force as a vector and the displacement vector for a small motion.
Say that your point is moving from here to here, you have the displacement delta r. It is just the change in the position vector.
It is the vector from the old position to the new position.
And then you have your force that is being exerted.
And you do the dot product between them.
That will give you the work of a force during this motion.
And the physical significance of this, well, the work tells you basically how much energy you have to provide to actually perform this motion.
Just in case you haven't seen this in 8.01 yet.
I am hoping all of you have heard about work somewhere, but in case it is completely mysterious that is the amount of energy provided by the force.
If a force goes along the motion, it actually pushes the particle.
It provides an energy to do it to do that motion.
And, conversely, if you are trying to go against the force then you have to provide energy to the particle to be able to do that.
In particular, if this is the only force that is taking place then the work would be the variation in kinetic energy of a particle along the motion.
That is a good description for a small motion.
But let's say that my particle is not just doing that but it's doing something complicated and my force keeps changing.
Somehow maybe I have a different force at every point.
Then I want to find the total work done along the motion.
Well, what I have to do is cut my trajectory into these little pieces.
And, for each of them, I have a vector along the trajectory.
I have a force, I do the dot product and I sum them together.
And, of course, to get the actual answer, I should actually cut into smaller and smaller pieces and sum all of the small contributions to work.
So, in fact, it is going to be an integral.
Along some trajectory, let's call C the trajectory for curve.
It is some curve.
The work adds up to an integral.
We write this using the notation integral along C of F dot dr. We have to decode this notation.
One way to decode this is to say it is a limit as we cut into smaller and smaller pieces of the sum over each piece of a trajectory of the force of a given point dot product with that small vector along the trajectory.
Well, that is not how we will compute it.
To compute it, we do things differently.
How can we actually compute it?
Well, what we can do is say that actually we are cutting things into small time intervals.
The way that we split the trajectory is we just take a picture every, say, millisecond.
Every millisecond we have a new position.
And the motion, the amount by which you have moved during each small time interval is basically the velocity vector times the amount of time.
In fact, let me just rewrite this.
You do the dot product between the force and how much you have moved, well, if I just rewrite it this way, nothing has happened, but what this thing is, actually, is the velocity vector dr over dt.
What I am trying to say is that I can actually compute by integral by integrating F dot product with dr / dt over time.
Whatever the initial time to whatever the final time is, I integrate F dot product velocity dt.
And, of course, here this F, I mean F at the point on the trajectory at time t. This guy depends on x and y before it depends on t. I see a lot of confused faces, so let's do an example.
Yes?
Yes.
Here I need to put a limit as delta t to zero.
I cut my trajectory into smaller and smaller time intervals.
For each time interval, I have a small motion which is, essentially, velocity times delta t, and then I dot that with a force and I sum them.
Let's do an example.
Let's say that we want to find the work of this force.
I guess that was the first example we had.
It is a force field that tries to make everything rotate somehow.
Your first points along these circles.
And let's say that our trajectory, our particle is moving along the parametric curve.
x = t, y = t^2 for t going from zero to one.
What that looks like -- Well, maybe I should draw you a picture.
Our vector field.
Our trajectory.
If you try to plot this, when you see y is actually x squared, so it a piece of parabola that goes from the origin to (1,1).
That is what our curve looks like.
We are trying to get the work done by our force along this trajectory.
I should point out; I mean if you are asking me how did I get this?
That is actually the wrong question.
This is all part of the data.
I have a force and I have a trajectory, and I want to find what the work done is along that trajectory.
These two guys I can choose completely independently of each other.
The integral along C of F dot dr will be -- Well, it is the integral from time zero to time one of F dot the velocity vector dr over dt times dt.
That would be the integral from zero to one.
Let's try to figure it out.
What is F?
F, at a point (x, y), is <- y, x>.
But if I take the point where I am at time t then x is t and y is t squared.
Here I plug x equals t, y equals t squared, and that will give me negative t squared, t. Here I will put negative t squared, t dot product.
What is the velocity vector?
Well, dx over dt is just one, dy over dt is 2t.
So, the velocity vector is 1,2t dt.
Now we have to continue the calculation.
We get integral from zero to one of, what is this dot product?
Well, it is negative t squared plus 2t squared.
I get t squared.
Well, maybe I will write it.
Negative t squared plus 2t squared dt.
That ends up being integral from zero to one of t squared dt, which you all know how to integrate and get one-third.
That is the work done by the force along this curve.
Yes?
Well, I got it by just taking the dot product between the force and the velocity.
That is in case you are wondering, things go like this.
Any questions on how we did this calculation?
No.
Yes?
Why can't you just do F dot dr?
Well, soon we will be able to.
We don't know yet what dr means or how to use it as a symbol because we haven't said yet, I mean, see, this is a d vector r. That is kind of strange thing to have.
And certainly r is not a usual variable.
We have to be careful about what are the rules, what does this symbol mean?
We are going to see that right now.
And then we can do it, actually, in a slightly more efficient way.
I mean r is not a scalar quantity.
R is a position vector.
You cannot integrate F with respect to r. We don't know how to do that.
OK.
Yes?
The question is if I took a different trajectory from the origin to that point (1,1), what will happen?
Well, the answer is I would get something different.
For example, let me try to convince you of that.
For example, say I chose to instead go like this and then around like that, first I wouldn't do any work because here the force is perpendicular to my motion.
And then I would be going against the force all the way around.
I should get something that is negative.
Even if you don't see that, just accept it at face value that I say now.
The value of a line integral, in general, depends on how we got from point a to point b.
That is why we have to compute it by using the parametric equation for the curve.
It really depends on what curve you choose.
Any other questions.
Yes?
What happens when the force inflects the trajectory?
Well, then, actually, you would have to solve a differential equation telling you how a particle moves to find what the trajectory is.
That is something that would be a very useful topic.
And that is probably more like what you will do in 18.03, or maybe you actually know how to do it in this case.
What we are trying to develop here is a method to figure out if we know what the trajectory is what the work will be.
It doesn't tell us what the trajectory will be.
But, of course, we could also find that.
But here, see, I am not assuming, for example, that the particle is moving just based on that force.
Maybe, actually, I am here to hold it in my hand and force it to go where it is going, or maybe there is some rail that is taking it in that trajectory or whatever.
I can really do it along any trajectory.
And, if I wanted to, if I knew that was the case, I could try to find the trajectory based on what the force is.
But that is not what we are doing here.
Let's try to make sense of what you asked just a few minutes ago, what can we do directly with dr?
dr becomes somehow a vector.
I mean, when I replace it by dr over dt times dt, it becomes something that is a vector with a dt next to it.
In fact -- Well, it is not really new.
Let's see.
Another way to do it, let's say that our force has components M and N. I claim that we can write symbolically vector dr stands for its vector whose components are dx, dy.
It is a strange kind of vector.
I mean it is not a real vector, of course, but as a notion, it is a pretty good notation because it tells us that F of dr is M dx plus N dy.
In fact, we will very often write, instead of F dot dr line integral along c will be line integral along c of M dx plus N dy.
And so, in this language, of course, what we are integrating now, rather than a vector field, becomes a differential.
But you should think of it, too, as being pretty much the same thing.
It is like when you compare the gradient of a function and its differential, they are different notations but have the same content.
Now, there still remains the question of how do we compute this kind of integral?
Because it is more subtle than the notation suggests.
Because M and N both depend on x and y.
And, if you just integrate it with respect to x, you would be left with y's in there.
And you don't want to be left with y's.
You want a number at the end.
See, the catch is along the curve x and y are actually related to each other.
Whenever we write this, we have two variables x and y, but, in fact, along the curve C we have only one parameter.
It could be x.
It could be y.
It could be time.
Whatever you want.
But we have to express everything in terms of that one parameter.
And then we get a usual single variable integral.
How do we evaluate things in this language?
Well, we do it by substituting the parameter into everything.
The method to evaluate is to express x and y in terms of a single variable.
And then substitute that variable.
Let's, for example, redo the one we had up there just using these new notations.
You will see that it is the same calculation but with different notations.
In that example that we had, our vector field F was negative .
What we are integrating is negative y dx plus x dy.
And, see, if we have just this, we don't know how to integrate that.
I mean, well, you could try to come up with negative x, y or something like that.
But that actually doesn't make sense.
It doesn't work.
What we will do is we will actually have to express everything in terms of a same variable, because it is a single integral and we should only have on variable.
And what that variable will be, well, if we just do it the same way that would just be t. How do we express everything in terms of t?
Well, we use the parametric equation.
We know that x is t and y is t squared.
We know what to do with these two guys.
What about dx and dy?
Well, it is easy.
We just differentiate.
dx becomes dt, dy becomes 2t dt.
I am just saying, in a different language, what I said over here with dx over dt equals one, dy over dt equals 2t.
It is the same thing but written slightly differently.
Now, I am going to do it again.
I am going to switch from one board to the next one.
My integral becomes the integral over C of negative y is minus t squared dt plus x is t times dy is 2t dt.
And now that I have only t left, it is fine to say I have a usual single variable integral over a variable t that goes from zero to one.
Now I can say, yes, this is the integral from zero to one of that stuff.
I can simply it a bit and it becomes t squared dt, and I can compute it, equals one-third.
I have negative t squared and then I have plus 2t squared, so end up with positive t squared.
It is the same as up there.
Any questions?
Yes?
dy is the differential of y, y is t squared, so I get 2t dt.
I plug dt for dx, I plug 2t dt for dy and so on.
And that is the general method.
If you are given a curve then you first have to figure out how do you express x and y in terms of the same thing?
And you get to choose, in general, what parameter we use.
You choose to parameterize your curve in whatever way you want.
The note that I want to make is that this line integral depends on the trajectory C but not on the parameterization.
You can choose whichever variable you want.
For example, what you could do is when you know that you have that trajectory, you could also choose to parameterize it as x equals, I don't know, sine theta, y equals sine square theta, because y is x squared where theta goes from zero to pi over two.
And then you could get dx and dy in terms of d theta.
And you would be able to do it with a lot of trig and you would get the same answer.
That would be a harder way to get the same thing.
What you should do in practice is use the most reasonable way to parameterize your curve.
If you know that you have a piece of parabola y equals x squared, there is no way you would put sine and sine squared.
You could set x equals, y equals t squared, which is very reasonable.
You could even take a small shortcut and say that your variable will be just x.
That means x you just keep as it is.
And then, when you have y, you set y equals x squared, dy equals 2x dx, and then you will have an integral over x.
That works.
So, this one is not practical.
But you get to choose.
Now let me tell you a bit more about the geometry.
We have said here is how we compute it in general, and that is the general method for computing a line integral for work.
You can always do this, try to find a parameter, the simplest one, express everything in terms of its variable and then you have an integral to compute.
But sometimes you can actually save a lot of work by just thinking geometrically about what this all does.
Let me tell you about the geometric approach.
One thing I want to remind you of first is what is this vector dr?
Well, what is vector delta r?
If I take a very small piece of the trajectory then my vector delta r will be tangent to the trajectory.
It will be going in the same direction as the unit tangent vector t. And what is its length?
Well, its length is the arc length along the trajectory, which we called delta s. Remember, s was the distance along the trajectory.
We can write vector dr equals dx, dy, but that is also T times ds.
It is a vector whose direction is tangent to the curve and whose length element is actually the arc length element.
I mean, if you don't like this notation, think about dividing everything by dt.
Then what we are saying is dr over dt, which is the velocity vector.
Well, in coordinates, the velocity vector is dx over dt, dy over dt.
But, more geometrically, the direction of a velocity vector is tangent to the trajectory and its magnitude is speed ds over dt.
So, that is really the same thing.
If I say this, that means that my line integral F to dr, well, I say I can write it as integral of M dx plus N dy.
That is what I will do if I want to compute it by computing the integral.
But, if instead I want to think about it geometrically, I could rewrite it as F dot T ds.
Now you can think of this, F dot T is a scalar quantity.
It is the tangent component of my force.
I take my force and project it to the tangent direction to a trajectory and the I integrate that along the curve.
They are the same thing.
And sometimes it is easier to do it this way.
Here is an example.
This is bound to be easier only when the field and the curve are relatively simple and have a geometric relation to each other.
If I give you an evil formula with x cubed plus y to the fifth or whatever there is very little chance that you will be able to simplify it that way.
But let's say that my trajectory is just a circle of radius a centered at the origin.
Let's say I am doing that counterclockwise and let's say that my vector field is xi yj.
What does that look like?
Well, my trajectory is just this circle.
My vector field, remember, xi plus yj, that is the one that is pointing radially from the origin.
Hopefully, if you have good physics intuition here, you will already know what the work is going to be.
It is going to be zero because the force is perpendicular to the motion.
Now we can say it directly by saying if you have any point of a circle then the tangent vector to the circle will be, well, it's tangent to the circle, so that means it is perpendicular to the radial direction, while the force is pointing in the radial direction so you have a right angle between them.
F is perpendicular to T. F dot T is zero.
The line integral of F dot T ds is just zero.
That is much easier than writing this is integral of x over dx plus y over dy.
What do we do?
Well, we set x equals a cosine theta, y equals a sine theta.
We get a bunch of trig things.
It cancels out to zero.
It is not much harder but we saved time by not even thinking about how to parameterize things.
Let's just do a last one.
That was the first one.
Let's say now that I take the same curve C, but now my vector field is the one that rotates negative yi plus xj.
That means along my circle the tangent vector goes like this and my vector field is also going around.
So, in fact, at this point the vector field will always be going in the same direction.
Now F is actually parallel to the tangent direction.
That means that the dot product of F dot T, remember, if it is the component of F in this direction that will be the same of the length of F. But what is the length of F on this circle if this length is a?
It is just going to be a.
That is what we said earlier about this vector field.
At every point, this dot product is a.
Now we know how to integrate that quite quickly.
Because it becomes the integral of a ds, but a is a constant so we can take it out.
And now what do we get when we integrate ds along C?
Well, we should get the total length of the curve if we sum all the little pieces of arc length.
But we know that the length of a circle of radius a is 2pi a, so we get 2(pi)a squared.
If we were to compute that by hand, well, what would we do?
We would be computing integral of minus y dx plus x dy.
Since we are on a circle, we will probably set x equals a times cosine theta, y equals a times sine theta for theta between zero and 2pi.
Then we would get dx and dy out of these.
So, y is a sine theta, dx is negative a sine theta d theta, if you differentiate a cosine, plus a cosine theta times a cosine theta d theta.
Well, you will just end up with integral from zero to 2pi of a squared time sine squared theta plus cosine square theta times d theta.
That becomes just one.
And you get the same answer.
It took about the same amount of time because I did this one rushing very quickly, but normally it takes about at least twice the amount of time to do it with a calculation.
That tells you sometimes it is worth thinking geometrically.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, let me remind you, yesterday we've defined and started to compute line integrals for work as a vector field along a curve.
So, we have a curve in the plane, C. We have a vector field that gives us a vector at every point.
And, we want to find the work done along the curve.
So, that's the line integral along C of F dr, or more geometrically, line integral along C of F.T ds where T is the unit tangent vector, and ds is the arc length element.
Or, in coordinates, that they integral of M dx N dy where M and N are the components of the vector field.
OK, so -- Let's do an example that will just summarize what we did yesterday, and then we will move on to interesting observations about these things.
So, here's an example we are going to look at now.
Let's say I give you the vector field yi plus xj.
So, it's not completely obvious what it looks like, but here is a computer plot of that vector field.
So, that tells you a bit what it does.
It points in all sorts of directions.
And, let's say we want to find the work done by this vector field.
If I move along this closed curve, I start at the origin.
But, I moved along the x-axis to one.
That move along the unit circle to the diagonal, and then I move back to the origin in a straight line.
OK, so C consists of three parts -- -- so that you enclose a sector of a unit disk -- -- corresponding to angles between zero and 45�.
So, to compute this line integral, all we have to do is we have set up three different integrals and add that together.
OK, so we need to set up the integral of y dx plus x dy for each of these pieces.
So, let's do the first one on the x-axis.
Well, one way to parameterize that is just use the x variable.
And, say that because we are on the, let's see, sorry, we are going from the origin to (1,0).
Well, we know we are on the x-axis.
So, y there is actually just zero.
And, the variable will be x from zero to one.
Or, if you prefer, you can parameterize things, say, x equals t for t from zero to one, and y equals zero.
What doesn't change is y is zero, and therefore, dy is also zero.
So, in fact, we are integrating y dx x dy, but that becomes, well, zero dx 0, and that's just going to give you zero.
OK, so there's the line integral.
Here, it's very easy to compute.
Of course, you can also do it geometrically because geometrically, you can see in the picture along the x-axis, the vector field is pointing vertically.
If I'm on the x-axis, my vector field is actually in the y direction.
So, it's perpendicular to my curve.
So, the work done is going to be zero.
F dot T will be zero.
OK, so F dot T is zero, so the integral is zero.
OK, any questions about this first part of the calculation?
No?
It's OK?
OK, let's move on to more interesting part of it.
Let's do the second part, which is a portion of the unit circle.
OK, so I should have drawn my picture.
And so now we are moving on this part of the curve that's C2.
And, of course we have to choose how to express x and y in terms of a single variable.
Well, most likely, when you are moving on a circle, you are going to use the angle along the circle to tell you where you are.
OK, so we're going to use the angle theta as a parameter.
And we will say, we are on the unit circle.
So, x is cosine theta and y is sine theta.
What's the range of theta?
Theta goes from zero to pi over four, OK?
So, whenever I see dx, I will replace it by, well, the derivative of cosine is negative sine.
So, minus sine theta d theta, and dy, the derivative of sine is cosine.
So, it will become cosine theta d theta.
OK, so I'm computing the integral of y dx x dy.
That means -- -- I'll be actually computing the integral of, so, y is sine theta.
dx, that's negative sine theta d theta plus x cosine.
dy is cosine theta d theta from zero to pi/4.
OK, so that's integral from zero to pi / 4 of cosine squared minus sine squared.
And, if you know your trig, then you should recognize this as cosine of two theta.
OK, so that will integrate to one half of sine two theta from zero to pi over four, sorry.
And, sine pi over two is one.
So, you will get one half.
OK, any questions about this one?
No?
OK, then let's do the third one.
So, the third guy is when we come back to the origin along the diagonal.
OK, so we go in a straight line from this point.
Where's this point?
Well, this point is one over root two, one over root two.
And, we go back to the origin.
OK, so we need to figure out a way to express x and y in terms of the same parameter.
So, one way which is very natural would be to just say, well, let's say we move from here to here over time.
And, at time zero, we are here.
At time one, we are here.
We know how to parameterize this line.
So, what we could do is say, let's parameterize this line.
So, we start at one over root two, and we go down by one over root two in time one.
And, same with y.
That's actually perfectly fine.
But that's unnecessarily complicated.
OK, why is a complicated?
Because we will get all of these expressions.
It would be easier to actually just look at motion in this direction and then say, well, if we have a certain work if we move from here to here, then the work done moving from here to here is just going to be the opposite, OK?
So, in fact, we can do slightly better by just saying, well, we'll take x = t, y = t. t from zero to one over root two, and take, well, sorry, that gives us what I will call minus C3, which is C3 backwards.
And then we can say the integral for work along minus C3 is the opposite of the work along C3.
Or, if you're comfortable with integration where variables go down, then you could also say that t just goes from one over square root of two down to zero.
And, when you set up your integral, it will go from one over root two to zero.
And, of course, that will be the negative of the one from zero to one over root two.
So, it's the same thing.
OK, so if we do it with this parameterization, we'll get that, well of course, dx is dt, dy is dt.
So, the integral along minus C3 of y dx plus x dy is just the integral from zero to one over root two of t dt plus t dt.
Sorry, I'm messing up my blackboard, OK, which is going to be, well, the integral of 2t dt, which is t2 between these bounds, which is one half.
That's the integral along minus C3, along the reversed path.
And, if I want to do it along C3 instead, then I just take the negative.
Or, if you prefer, you could have done it directly with integral from one over root two, two zero, which gives you immediately the negative one half.
OK, so at the end, we get that the total work -- -- was the sum of the three line integrals.
I'm not writing after dr just to save space.
But, zero plus one half minus one half, and that comes out to zero.
So, a lot of calculations for nothing.
OK, so that should give you overview of various ways to compute line integrals.
Any questions about all that?
No?
OK.
So, next, let me tell you about how to avoid computing like integrals.
Well, one is easy: don't take this class.
But that's not, so here's another way not to do it, OK?
So, let's look a little bit about one kind of vector field that actually we've encountered a few weeks ago without saying it.
So, we said when we have a function of two variables, we have the gradient vector.
Well, at the time, it was just a vector.
But, that vector depended on x and y.
So, in fact, it's a vector field.
OK, so here's an interesting special case.
Say that F, our vector field is actually the gradient of some function.
So, it's a gradient field.
And, so f is a function of two variables, x and y, and that's called the potential for the vector field.
The reason is, of course, from physics.
In physics, you call potential, electrical potential or gravitational potential, the potential energy.
This function of position that tells you how much actually energy stored somehow by the force field, and this gradient gives you the force.
Actually, not quite.
If you are a physicist, that the force will be negative the gradient.
So, that means that physicists' potentials are the opposite of a mathematician's potential.
Okay?
So it's just here to confuse you.
It doesn't really matter all the time.
So to make things simpler we are using this convention and you just put a minus sign if you are doing physics.
So then I claim that we can simplify the evaluation of the line integral for work.
Perhaps you've seen in physics, the work done by, say, the electrical force, is actually given by the change in the value of a potential from the starting point of the ending point, or same for gravitational force.
So, these are special cases of what's called the fundamental theorem of calculus for line integrals.
So, the fundamental theorem of calculus, not for line integrals, tells you if you integrate a derivative, then you get back the function.
And here, it's the same thing in multivariable calculus.
It tells you, if you take the line integral of the gradient of a function, what you get back is the function.
OK, so -- -- the fundamental theorem of calculus for line integrals -- -- says if you integrate a vector field that's the gradient of a function along a curve, let's say that you have a curve that goes from some starting point, P0, to some ending point, P1.
All you will get is the value of F at P1 minus the value of F at P0.
OK, so, that's a pretty nifty formula that only works if the field that you are integrating is a gradient.
You know it's a gradient, and you know the function, little f. I mean, we can't put just any vector field in here.
We have to put the gradient of F. So, actually on Tuesday we'll see how to decide whether a vector field is a gradient or not, and if it is a gradient, how to find the potential function.
So, we'll cover that.
But, for now we need to try to figure out a bit more about this, what it says, what it means physically, how to think of it geometrically, and so on.
So, maybe I should say, if you're trying to write this in coordinates, because that's also a useful way to think about it, if I give you the line integral along C, so, the gradient field, the components are f sub x and f sub y.
So, it means I'm actually integrating f sub x dx plus f sub y dy.
Or, if you prefer, that's the same thing as actually integrating df.
So, I'm integrating the differential of a function, f. Well then, that's the change in F. And, of course, if you write it in this form, then probably it's quite obvious to you that this should be true.
I mean, in this form, actually it's the same statement as in single variable calculus.
OK, and actually that's how we prove the theorem.
So, let's prove this theorem.
How do we prove it?
Well, let's say I give you a curve and I ask you to compute this integral.
How will you do that?
Well, the way you compute the integral actually is by choosing a parameter, and expressing everything in terms of that parameter.
So, we'll set, well, so we know it's f sub x dx plus f sub y dy.
And, we'll want to parameterize C in the form x equals x of t. y equals y of t. So, if we do that, then dx becomes x prime of t dt.
dy becomes y prime of t dt.
So, we know x is x of t. That tells us dx is x prime of t dt.
y is y of t gives us dy is y prime of t dt.
So, now what we are integrating actually becomes the integral of f sub x times dx dt plus f sub y times dy dt times dt.
OK, but now, here I recognize a familiar guy.
I've seen this one before in the chain rule.
OK, this guy, by the chain rule, is the rate of change of f if I take x and y to be functions of t. And, I plug those into f. So, in fact, what I'm integrating is df dt when I think of f as a function of t by just plugging x and y as functions of t. And so maybe actually I should now say I have sometimes t goes from some initial time, let's say, t zero to t one.
And now, by the usual fundamental theorem of calculus, I know that this will be just the change in the value of f between t zero and t one.
OK, so integral from t zero to one of (df /dt) dt, well, that becomes f between t zero and t one.
f of what?
We just have to be a little bit careful here.
Well, it's not quite f of t. It's f seen as a function of t by putting x of t and y of t into it.
So, let me read that carefully.
What I'm integrating to is f of x of t and y of t. Does that sound fair?
Yeah, and so, when I plug in t1, I get the point where I am at time t1.
That's the endpoint of my curve.
When I plug t0, I will get the starting point of my curve, p0.
And, that's the end of the proof.
It wasn't that hard, see?
OK, so let's see an example.
Well, let's look at that example again.
So, we have this curve.
We have this vector field.
Could it be that, by accident, that vector field was a gradient field?
So, remember, our vector field was y, x.
Can we think of a function whose derivative with respect to x is y, and derivative with respect to y is x?
Yeah, x times y sounds like a good candidate where f( x, y) is xy.
OK, so that means that the line integrals that we computed along these things can be just evaluated from just finding out the values of f at the endpoint?
So, here's version two of my plot where I've added the contour plot of a function, x, y on top of the vector field.
Actually, they have a vector field is still pointing perpendicular to the level curves that we have seen, just to remind you.
And, so now, when we move, now when we move, the origin is on the level curve, f equals zero.
And, when we start going along C1, we stay on f equals zero.
So, there's no work.
The potential doesn't change.
Then on C2, the potential increases from zero to one half.
The work is one half.
And then, on C3, we go back down from one half to zero.
The work is negative one half.
See, that was much easier than computing.
So, for example, the integral along C2 is actually just, so, C2 goes from one zero to one over root two, one over root two.
So, that's one half minus zero, and that's one half, OK, because C2 was going here.
And, at this point, f is zero.
At that point, f is one half.
And, similarly for the others, and of course when you sum, you get zero because the total change in f when you go from here, to here, to here, to here, eventually you are back at the same place.
So, f hasn't changed.
OK, so that's a neat trick.
And it's important conceptually because a lot of the forces are gradients of potentials, namely, gravitational force, electric force.
The problem is not every vector field is a gradient.
A lot of vector fields are not gradients.
For example, magnetic fields certainly are not gradients.
So -- -- a big warning: everything today only applies if F is a gradient field.
OK, it's not true otherwise.
OK, still, let's see, what are the consequences of the fundamental theorem?
So, just to put one more time this disclaimer, if F is a gradient field -- -- then what do we have?
Well, there's various nice features of work done by gradient fields that are not too far off the vector fields.
So, one of them is this property of path independence.
OK, so the claim is if I have a line integral to compute, that it doesn't matter which path I take as long as it goes from point a to point b.
It just depends on the point where I start and the point where I end.
And, that's certainly false in general, but for a gradient field that works.
So if I have a point, P0, a point, P1, and I have two different paths that go there, say, C1 and C2, so they go from the same point to the same point but in different ways, then in this situation, the line integral along C1 is equal to the line integral along C2.
Well, actually, let me insist that this is only for gradient fields by putting gradient F in here, just so you don't get tempted to ever use this for a field that's not a gradient field -- -- if C1 and C2 have the same start and end point.
OK, how do you prove that?
Well, it's very easy.
We just use the fundamental theorem.
It tells us, if you compute the line integral along C1, it's just F at this point minus F at this point.
If you do it for C2, well, the same.
So, they are the same.
And for that you don't actually even need to know what little f is.
You know in advance that it's going to be the same.
So, if I give you a vector field and I tell you it's the gradient of mysterious function but I don't tell you what the function is and you don't want to find out, you can still use path independence, but only if you know it's a gradient.
OK, I guess this one is dead.
So, that will stay here forever because nobody is tall enough to erase it.
When you come back next year and you still see that formula, you'll see.
Yes, but there's no useful information here.
That's a good point.
OK, so what's another consequence?
So, if you have a gradient field, it's what's called conservative.
OK, so what a conservative field?
Well, the word conservative comes from the idea in physics; if the conservation of energy.
It tells you that you cannot get energy for free out of your force field.
So, what it means is that in particular, if you take a closed trajectory, so a trajectory that goes from some point back to the same point, so, if C is a closed curve, then the work done along C -- -- is zero.
OK, that's the definition of what it means to be conservative.
If I take any closed curve, the work will always be zero.
On the contrary, not conservative means somewhere there is a curve along which the work is not zero.
If you find a curve where the work is zero, that's not enough to say it's conservative.
You have show that no matter what curve I give you, if it's a closed curve, it will always be zero.
So, what that means concretely is if you have a force field that conservative, then you cannot build somehow some perpetual motion out of it.
You can't build something that will just keep going just powered by that force because that force is actually not providing any energy.
After you've gone one loop around, nothings happened from the point of view of the energy provided by that force.
There's no work coming from the force, while if you have a force field that's not conservative than you can try to actually maybe find a loop where the work would be positive.
And then, you know, that thing will just keep running.
So actually, if you just look at magnetic fields and transformers or power adapters, and things like that, you precisely extract energy from the magnetic field.
Of course, I mean, you actually have to take some power supply to maintain the magnetic fields.
But, so a magnetic field, you could actually try to get energy from it almost for free.
A gravitational field or an electric field, you can't.
OK, so and now why does that hold?
Well, if I have a gradient field, then if I try to compute this line integral, I know it will be the value of the function at the end point minus the value at the starting point.
But, they are the same.
So, the value is the same.
So, if I have a gradient field, and I do the line integral, then I will get f at the endpoint minus f at the starting point.
But, they're the same point, so that's zero.
OK, so just to reinforce my warning that not every field is a gradient field, let's look again at our favorite vector field from yesterday.
So, our favorite vector field yesterday was negative y and x.
It's a vector field that just rotates around the origin counterclockwise.
Well, we said, say you take just the unit circle -- -- for example, counterclockwise.
Well, remember we said yesterday that the line integral of F dr, maybe I should say F dot T ds now, because the vector field is tangent to the circle.
So, on the unit circle, F is tangent to the curve.
And so, F dot T is length F times, well, length T. But, T is a unit vector.
So, it's length F. And, the length of F on the unit circle was just one.
So, that's the integral of 1 ds.
So, it's just the length of the circle that's 2 pi.
And 2 pi is definitely not zero.
So, this vector field is not conservative.
And so, now we know actually it's not the gradient of anything because if it were a gradient, then it would be conservative and it's not.
So, it's an example of a vector field that is not conservative.
It's not path independent either by the way because, see, if I go from here to here along the upper half circle or along the lower half circle, in one case I will get pi.
In the other case I will get negative pi.
I don't get the same answer, and so on, and so on.
It just fails to have all of these properties.
So, maybe I will write that down.
It's not conservative, not path independent.
It's not a gradient.
It doesn't have any of these properties.
OK, any questions?
Yes?
How do you determine whether something is a gradient or not?
Well, that's what we will see on Tuesday.
Yes?
Is it possible that it's conservative and not path independent, or vice versa?
The answer is no; these two properties are equivalent, and we are going to see that right now.
At least that's the plan.
OK, yes?
Let's see, so you said if it's not path independent, then we cannot draw level curves that are perpendicular to it at every point.
I wouldn't necessarily go that far.
You might be able to draw curves that are perpendicular to it.
But they won't be the level curves of a function for which this is the gradient.
I mean, you might still have, you know, if you take, say, take his gradient field and scale it that in strange ways, you know, multiply by two in some places, by one in other places, by five and some other places, you will get something that won't be conservative anymore.
And it will still be perpendicular to the curves.
So, it's more subtle than that, but certainly if it's not conservative then it's not a gradient, and you cannot do what we said.
And how to decide whether it is or not, they'll be Tuesday's topic.
So, for now, I just want to figure out again actually, let's now state all these properties -- Actually, let me first do one minute of physics.
So, let me just tell you again what's the physics in here.
So, it's a force field is the gradient of a potential -- -- so, I'll still keep my plus signs.
So, maybe I should say this is minus physics.
[LAUGHTER] So, the work of F is the change in value of potential from one endpoint to the other endpoint.
[PAUSE ] And -- -- so, you know, you might know about gravitational fields, or electrical -- -- fields versus gravitational -- -- or electrical potential.
And, in case you haven't done any 8.02 yet, electrical potential is also commonly known as voltage.
It's the one that makes it hurt when you stick your fingers into the socket.
[LAUGHTER] Don't try it.
OK, and so now, conservativeness means no energy can be extracted for free -- -- from the field.
You can't just have, you know, a particle moving in that field and going on in definitely, faster and faster, or if there's actually friction, then keep moving.
So, total energy is conserved.
And, I guess, that's why we call that conservative.
OK, so let's end with the recap of various equivalent properties.
OK, so the first property that I will have for a vector field is that it's conservative.
So, to say that a vector field with conservative means that the line integral is zero along any closed curve.
Maybe to clarify, sorry, along all closed curves, OK, every closed curve; give me any closed curve, I get zero.
So, now I claim this is the same thing as a second property, which is that the line integral of F is path independent.
OK, so that means if I have two paths with the same endpoint, then I will get always the same answer.
Why is that equivalent?
Well, let's say that I am path independent.
If I am path independent, then if I take a closed curve, well, it has the same endpoints as just the curve that doesn't move at all.
So, path independence tells me instead of going all around, I could just stay where I am.
And then, the work would just be zero.
So, if I path independent, tonight conservative.
Conversely, let's say that I'm just conservative and I want to check path independence.
Well, so I have two points, and then I had to paths between that.
I want to show that the work is the same.
Well, how I do that?
C1 and C2, well, I observe that if I do C1 minus C2, I get a closed path.
If I go first from here to here, and then back along that one, I get a closed path.
So, if I am conservative, I should get zero.
But, if I get zero on C1 minus C2, it means that the work on C1 and the work on C2 are the same.
See, so it's the same.
It's just a different way to think about the situation.
More things that are equivalent, I have two more things to say.
The third one, it's equivalent to F being a gradient field.
OK, so this is equivalent to the third property.
F is a gradient field.
Why?
Well, if we know that it's a gradient field, that we've seen that we get these properties out of the fundamental theorem.
The question is, if I have a conservative, or path independent vector field, why is it the gradient of something?
OK, so this way is a fundamental theorem.
That way, well, so that actually, let me just say that will be how we find the potential.
So, how do we find potential?
Well, let's say that I know the value of my potential here.
Actually, I get to choose what it is.
Remember, in physics, the potential is defined up to adding or subtracting a constant.
What matters is only the change in potential.
So, let's say I know my potential here and I want to know my potential here.
What do I do?
Well, I take my favorite particle and I move it from here to here.
And, I look at the work done.
And that tells me how much potential has changed.
So, that tells me what the potential should be here.
And, this does not depend on my choice of path because I've assumed that I'm path independence.
So, that's who we will do on Tuesday.
And, let me just state the fourth property that's the same.
So, all that stuff is the same as also four.
If I look at M dx N dy is what's called an exact differential.
So, what that means, an exact differential, means that it can be put in the form df for some function, f, and just reformulating this thing, right, because I'm saying I can just put it in the form f sub x dx plus f sub y dy, which means my vector field was a gradient field.
So, these things are really the same.
OK, so after the weekend, on Tuesday we will actually figure out how to decide whether these things hold or not, and how to find the potential.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
We were looking at vector fields last time.
Last time we saw that if a vector field happens to be a gradient field -- -- then the line integral can be computed actually by taking the change in value of the potential between the end point and the starting point of the curve.
If we have a curve c, from a point p0 to a point p1 then the line integral for work depends only on the end points and not on the actual path we chose.
We say that the line integral is path independent.
And we also said that the vector field is conservative because of conservation of energy which tells you if you start at a point and you come back to the same point then you haven't gotten any work out of that force.
If we have a closed curve then the line integral for work is just zero.
And, basically, we say that these properties are equivalent being a gradient field or being path independent or being conservative.
And what I promised to you is that today we would see a criterion to decide whether a vector field is a gradient field or not and how to find the potential function if it is a gradient field.
So, that is the topic for today.
The question is testing whether a given vector field, let's say M and N compliments, is a gradient field.
For that, well, let's start with an observation.
Say that it is a gradient field.
That means that the first component of a field is just the partial of f with respect to some variable x and the second component is the partial of f with respect to y.
Now we have seen an interesting property of the second partial derivatives of the function, which is if you take the partial derivative first with respect to x, then with respect to y, or first with respect to y, then with respect to x you get the same thing.
We know f sub xy equals f sub yx, and that means M sub y equals N sub x.
If you have a gradient field then it should have this property.
You take the y component, take the derivative with respect to x, take the x component, differentiate with respect to y, you should get the same answer.
And that is important to know.
So, I am going to put that in a box.
It is a broken box.
The claim that I want to make is that there is a converse of sorts.
This is actually basically all we need to check.
Conversely, if, and I am going to put here a condition, My equals Nx, then F is a gradient field.
What is the condition that I need to put here?
Well, we will see a more precise version of that next week.
But for now let's just say if our vector field is defined and differentiable everywhere in the plane.
We need, actually, a vector field that is well-defined everywhere.
You are not allowed to have somehow places where it is not well-defined.
Otherwise, actually, you have a counter example on your problem set this week.
If you look at the last problem on the problem set this week, it gives you a vector field that satisfies this condition everywhere where it is defined.
But, actually, there is a point where it is not defined.
And that causes it, actually, to somehow -- I mean everything that I am going to say today breaks down for that example because of that.
I mean, we will shed more light on this a bit later with the notion of simply connected regions and so on.
But for now let's just say if it is defined everywhere and it satisfies this criterion then it is a gradient field.
If you ignore the technical condition, being a gradient field means essentially the same thing as having this property.
That is what we need to check.
Let's look at an example.
Well, one vector field that we have been looking at a lot was - yi xj.
Remember that was the vector field that looked like a rotation at the unit speed.
I think last time we already decided that this guy should not be allowed to be a gradient field and should not be conservative because if we integrate on the unit circle then we would get a positive answer.
But let's check that indeed it fails our test.
Well, let's call this M and let's call this guy N. If you look at partial M, partial y, that is going to be a negative one.
If you take partial N, partial x, that is going to be one.
These are not the same.
So, indeed, this is not a gradient field.
Any questions about that?
Yes?
Your question is if I have the property M sub y equals N sub x only in a certain part of a plane for some values of x and y, can I conclude these things?
And it is a gradient field in that part of the plane and conservative and so on.
The answer for now is, in general, no.
And when we spend a bit more time on it, actually, maybe I should move that up.
Maybe we will talk about it later this week instead of when I had planned.
There is a notion what it means for a region to be without holes.
Basically, if you have that kind of property in a region that doesn't have any holes inside it then things will work.
The problem comes from a vector field satisfying this criterion in a region but it has a hole in it.
Because what you don't know is whether your potential is actually well-defined and takes the same value when you move all around the hole.
It might come back to take a different value.
If you look carefully and think hard about the example in the problem sets that is exactly what happens there.
Again, I will say more about that later.
For now we basically need our function to be, I mean, I should still say if you have this property for a vector field that is not quite defined everywhere, you are more than welcome, you know, you should probably still try to look for a potential using methods that we will see.
But something might go wrong later.
You might end up with a potential that is not well-defined.
Let's do another example.
Let's say that I give you this vector field.
And this a here is a number.
The question is for which value of a is this going to be possibly a gradient?
If you have your flashcards then that is a good time to use them to vote, assuming that the number is small enough to be made with.
Let's try to think about it.
We want to call this guy M. We want to call that guy N. And we want to test M sub y versus N sub x. I don't see anyone.
I see people doing it with their hands, and that works very well.
OK.
The question is for which value of a is this a gradient?
I see various people with the correct answer.
OK. That a strange answer.
That is a good answer.
OK.
The vote seems to be for a equals eight.
Let's see.
What if I take M sub y?
That is going to be just ax.
And N sub x?
That is 8x.
I would like a equals eight.
By the way, when you set these two equal to each other, they really have to be equal everywhere.
You don't want to somehow solve for x or anything like that.
You just want these expressions, in terms of x and y, to be the same quantities.
I mean you cannot say if x equals z they are always equal.
Yeah, that is true.
But that is not what we are asking.
Now we come to the next logical question.
Let's say that we have passed the test.
We have put a equals eight in here.
Now it should be a gradient field.
The question is how do we find the potential?
That becomes eight from now on.
The question is how do we find the function which has this as gradient?
One option is to try to guess.
Actually, quite often you will succeed that way.
But that is not a valid method on next week's test.
We are going to see two different systematic methods.
And you should be using one of these because guessing doesn't always work.
And, actually, I can come up with examples where if you try to guess you will surely fail.
I can come up with trick ones, but I don't want to put that on the test.
The next stage is finding the potential.
And let me just emphasize that we can only do that if step one was successful.
If we have a vector field that cannot possibly be a gradient then we shouldn't try to look for a potential.
It is kind of obvious but is probably worth pointing out.
There are two methods.
The first method that we will see is computing line integrals.
Let's see how that works.
Let's say that I take some path that starts at the origin.
Or, actually, anywhere you want, but let's take the origin.
That is my favorite point.
And let's go to a point with coordinates (x1, y1).
And let's take my favorite curve and compute the line integral of that field, you know, the work done along the curve.
Well, by the fundamental theorem, that should be equal to the value of the potential at the end point minus the value at the origin.
That means I can actually write f of (x1, y1) equals -- -- that line integral plus the value at the origin.
And that is just a constant.
We don't know what it is.
And, actually, we can choose what it is.
Because if you have a potential, say that you have some potential function.
And let's say that you add one to it.
It is still a potential function.
Adding one doesn't change the gradient.
You can even add 18 or any number that you want.
This is just going to be an integration constant.
It is the same thing as, in one variable calculus, when you take the anti-derivative of a function it is only defined up to adding the constant.
We have this integration constant, but apart from that we know that we should be able to get a potential from this.
And this we can compute using the definition of the line integral.
And we don't know what little f is, but we know what the vector field is so we can compute that.
Of course, to do the calculation we probably don't want to use this kind of path.
I mean if that is your favorite path then that is fine, but it is not very easy to compute the line integral along this, especially since I didn't tell you what the definition is.
There are easier favorite paths to have.
For example, you can go on a straight line from the origin to that point.
That would be slightly easier.
But then there is one easier.
The easiest of all, probably, is to just go first along the x-axis to (x1,0) and then go up parallel to the y-axis.
Why is that easy?
Well, that is because when we do the line integral it becomes M dx N dy.
And then, on each of these pieces, one-half just goes away because x, y is constant.
Let's try to use that method in our example.
Let's say that I want to go along this path from the origin, first along the x-axis to (x1,0) and then vertically to (x1, y1).
And so I want to compute for the line integral along that curve.
Let's say I want to do it for this vector field.
I want to find the potential for this vector field.
Let me copy it because I will have to erase at some point.
4x squared plus 8xy and 3y squared plus 4x squared.
That will become the integral of 4x squared plus 8 xy times dx plus 3y squared plus 4x squared times dy.
To evaluate on this broken line, I will, of course, evaluate separately on each of the two segments.
I will start with this segment that I will call c1 and then I will do this one that I will call c2.
On c1, how do I evaluate my integral?
Well, if I am on c1 then x varies from zero to x1.
Well, actually, I don't know if x1 is positive or not so I shouldn't write this.
I really should say just x goes from zero to x1.
And what about y?
y is just 0.
I will set y equal to zero and also dy equal to zero.
I get that the line integral on c1 -- Well, a lot of stuff goes away.
The entire second term with dy goes away because dy is zero.
And, in the first term, 8xy goes away because y is zero as well.
I just have an integral of 4x squared dx from zero to x1.
By the way, now you see why I have been using an x1 and a y1 for my point and not just x and y.
It is to avoid confusion.
I am using x and y as my integration variables and x1, y1 as constants that are representing the end point of my path.
And so, if I integrate this, I should get four-thirds x1 cubed.
That is the first part.
Next I need to do the second segment.
If I am on c2, y goes from zero to y1.
And what about x?
x is constant equal to x1 so dx becomes just zero.
It is a constant.
If I take the line integral of c2, F dot dr then I will get the integral from zero to y1.
The entire first term with dx goes away and then I have 3y squared plus 4x1 squared times dy.
That integrates to y cubed plus 4x1 squared y from zero to y1.
Or, if you prefer, that is y1 cubed plus 4x1 squared y1.
Now that we have done both of them we can just add them together, and that will give us the formula for the potential.
F of x1 and y1 is four-thirds x1 cubed plus y1 cubed plus 4x1 squared y1 plus a constant.
That constant is just the integration constant that we had from the beginning.
Now you can drop the subscripts if you prefer.
You can just say f is four-thirds x cubed plus y cubed plus 4x squared y plus constant.
And you can check.
If you take the gradient of this, you should get again this vector field over there.
Any questions about this method?
Yes?
No.
Well, it depends whether you are just trying to find one potential or if you are trying to find all the possible potentials.
If a problem just says find a potential then you don't have to use the constant.
This guy without the constant is a valid potential.
You just have others.
If your neighbor comes to you and say your answer must be wrong because I got this plus 18, well, both answers are correct.
By the way.
Instead of going first along the x-axis vertically, you could do it the other way around.
Of course, start along the y-axis and then horizontally.
That is the same level of difficulty.
You just exchange roles of x and y.
In some cases, it is actually even making more sense maybe to go radially, start out from the origin to your end point.
But usually this setting is easier just because each of these two guys were very easy to compute.
But somehow maybe if you suspect that polar coordinates will be involved somehow in the answer then maybe it makes sense to choose different paths.
Maybe a straight line is better.
Now we have another method to look at which is using anti-derivatives.
The goal is the same, still to find the potential function.
And you see that finding the potential is really the multivariable analog of finding the anti-derivative in the one variable.
Here we did it basically by hand by computing the integral.
The other thing you could try to say is, wait, I already know how to take anti-derivatives.
Let's use that instead of computing integrals.
And it works but you have to be careful about how you do it.
Let's see how that works.
Let's still do it with the same example.
We want to solve the equations.
We want a function such that f sub x is 4x squared plus 8xy and f sub y is 3y squared plus 4x squared.
Let's just look at one of these at a time.
If we look at this one, well, we know how to solve this because it is just telling us we have to integrate this with respect to x.
Well, let's call them one and two because I will have to refer to them again.
Let's start with equation one and lets integrate with respect to x.
Well, it tells us that f should be, what do I get when I integrate this with respect to x, four-thirds x cubed plus, when I integrate 8xy, y is just a constant, so I will get 4x squared y.
And that is not quite the end to it because there is an integration constant.
And here, when I say there is an integration constant, it just means the extra term does not depend on x.
That is what it means to be a constant in this setting.
But maybe my constant still depends on y so it is not actually a true constant.
A constant that depends on y is not really a constant.
It is actually a function of y.
The good news that we have is that this function normally depends on x.
We have made some progress.
We have part of the answer and we have simplified the problem.
If we have anything that looks like this, it will satisfy the first condition.
Now we need to look at the second condition.
We want f sub y to be that.
But we know what f is, so let's compute f sub y from this.
From this I get f sub y.
What do I get if I differentiate this with respect to y?
Well, I get zero plus 4x squared plus the derivative of g. I would like to match this with what I had.
If I match this with equation two then that will tell me what the derivative of g should be.
If we compare the two things there, we get 4x squared plus g prime of y should be equal to 3y squared by 4x squared.
And, of course, the 4x squares go away.
That tells you g prime is 3y squared.
And that integrates to y cubed plus constant.
Now, this time the constant is a true constant because g did not depend on anything other than y.
And the constant does not depend on y so it is a real constant now.
Now we just plug this back into this guy.
Let's call him star.
If we plug this into star, we get f equals four-thirds x cubed plus 4x squared y plus y cubed plus constant.
I mean, of course, again, now this constant is optional.
The advantage of this method is you don't have to write any integrals.
The small drawback is you have to follow this procedure carefully.
By the way, one common pitfall that is tempting.
After you have done this, what is very tempting is to just say, well, let's do the same with this guy.
Let's integrate this with respect to y.
You will get another expression for f up to a constant that depends on x.
And then let's match them.
Well, the difficulty is matching is actually quite tricky because you don't know in advance whether they will be the same expression.
It could be you could say let's just take the terms that are here and missing there and combine the terms, you know, take all the terms that appear in either one.
That is actually not a good way to do it, because if I put sufficiently complicated trig functions in there then you might not be able to see that two terms are the same.
Take an easy one.
Let's say that here I have one plus tangent square and here I have a secan square then you might not actually notice that there is a difference.
But there is no difference.
Whatever.
Anyway, I am saying do it this way, don't do it any other way because there is a risk of making a mistake otherwise.
I mean, on the other hand, you could start with integrating with respect to y and then differentiate and match with respect to x.
But what I am saying is just take one of them, integrate, get an answer that involves a function of the other variable, then differentiate that answer and compare and see what you get.
By the way, here, of course, after we simplified there were only y's here.
There were no x's.
And that is kind of good news.
I mean, if you had had an x here in this expression that would have told you that something is going wrong.
g is a function of y only.
If you get an x here, maybe you want to go back and check whether it is really a gradient field.
Yes?
Yes, this will work with functions of more than two variables.
Both methods work with more than two variables.
We are going to see it in the case where more than two means three.
We are going to see that in two or three weeks from now.
I mean, basically starting at the end of next week, we are going to do triple integrals, line integrals in space and so on.
The format is first we do everything in two variables.
Then we will do three variables.
And then what happens with more than three will be left to your imagination.
Any other questions about either of these methods?
A quick poll.
Who prefers the first method?
Who prefers the second method?
Wow.
OK.
Anyway, you will get to use whichever one you want.
And I would agree with you, but the second method is slightly more effective in that you are writing less stuff.
You don't have to set up all these line integrals.
On the other hand, it does require a little bit more attention.
Let's move on a bit.
Let me start by actually doing a small recap.
We said we have various notions.
One is to say that the vector field is a gradient in a certain region of a plane.
And we have another notion which is being conservative.
It says that the line integral is zero along any closed curve.
Actually, let me introduce a new piece of notation.
To remind ourselves that we are doing it along a closed curve, very often we put just a circle for the integral to tell us this is a curve that closes on itself.
It ends where it started.
I mean it doesn't change anything concerning the definition or how you compute it or anything.
It just reminds you that you are doing it on a closed curve.
It is actually useful for various physical applications.
And also, when you state theorems in that way, it reminds you,oh..
I need to be on a closed curve to do it.
And so we have said these two things are equivalent.
Now we have a third thing which is N sub x equals M sub y at every point.
Just to summarize the discussion.
We have said if we have a gradient field then we have this.
And the converse is true in suitable regions.
We have a converse if F is defined in the entire plane.
Or, as we will see soon, in a simply connected region.
I guess some of you cannot see what I am writing here, but it doesn't matter because you are not officially supposed to know it yet.
That will be next week.
Anyway, I said the fact that Nx equals My implies that we have a gradient field and is only if a vector field is defined in the entire plane or in a region that is called simply connected.
And more about that later.
Now let me just introduce a quantity that probably a lot of you have heard about in physics that measures precisely fairly ought to be conservative.
That is called the curl of a vector field.
For the definition we say that the curl of F is the quantity N sub x - M sub y.
It is just replicating the information we had but in a way that is a single quantity.
In this new language, the conditions that we had over there, this condition says curl F equals zero.
That is the new version of Nx equals My.
It measures failure of a vector field to be conservative.
The test for conservativeness is that the curl of F should be zero.
I should probably tell you a little bit about what the curl is, what it measures and what it does because that is something that is probably useful.
It is a very strange quantity if you put it in that form.
Yes?
I think it is the same as the physics one, but I haven't checked the physics textbook.
I believe it is the same.
Yes, I think it is the same as the physics one.
It is not the opposite this time.
Of course, in physics maybe you have seen curl in space.
We are going to see curl in space in two or three weeks.
Yes?
Yes.
Well, you can also use it.
If you fail this test then you know for sure that you are not gradient field so you might as well do that.
If you satisfy the test but you are not defined everywhere then there is still a bit of ambiguity and you don't know for sure.
OK. Let's try to see a little bit what the curl measures.
Just to give you some intuition, let's first think about a velocity field.
The curl measures the rotation component of a motion.
If you want a fancy word, it measures the vorticity of a motion.
It tells you how much twisting is taking place at a given point.
For example, if I take a constant vector field where my fluid is just all moving in the same direction where this is just constants then, of course, the curl is zero.
Because if you take the partials you get zero.
And, indeed, that is not what you would call swirling.
There is no vortex in here.
Let's do another one where this is still nothing going on.
Let's say that I take the radial vector field where everything just flows away from the origin.
That is f equals x, y.
Well, if I take the curl, I have to take partial over partial x of the second component, which is y, minus partial over partial y of the first component, which is x. I will get zero.
And, indeed, if you think about what is going on here, there is no rotation involved.
On the other hand, if you consider our favorite rotation vector field -- -- negative y and x then this curl is going to be N sub x minus M sub y, one plus one equals two.
That corresponds to the fact that we are rotating.
Actually, we are rotating at unit angular speed.
The curl actually measures twice the angular speed of a rotation part of a motion at any given point.
Now, if you have an actual motion, a more complicated field than these then no matter where you are you can think of a motion as a combination of translation effects, maybe dilation effects, maybe rotation effects, possibly other things like that.
And what a curl will measure is how intense the rotation effect is at that particular point.
I am not going to try to make a much more precise statement.
A precise statement is what a curl measures is really this quantity up there.
But the intuition you should have is it measures how much rotation is taking place at any given point.
And, of course, in a complicated motion you might have more rotation at some point than at some others, which is why the curl will depend on x and y.
It is not just a constant because how much you rotate depends on where you are.
If you are looking at actual wind velocities in weather prediction then the regions with high curl tend to be hurricanes or tornadoes or things like that.
They are not very pleasant things.
And the sign of a curl tells you whether you are going clockwise or counterclockwise.
Curl measures twice the angular velocity of the rotation component of a velocity field.
Now, what about a force field?
Because, after all, how we got to this was coming from and trying to understand forces and the work they do.
So I should tell you what it means for a force.
Well, the curl of a force field -- -- measures the torque exerted on a test object that you put at any point.
Remember, torque is the rotational analog of the force.
We had this analogy about velocity versus angular velocity and mass versus moment of inertia.
And then, in that analogy, force divided by the mass is what will cause acceleration, which is the derivative of velocity.
Torque divided by moment of inertia is what will cause the angular acceleration, namely the derivative of angular velocity.
Maybe I should write that down.
Torque divided by moment of inertia is going to be d over dt of angular velocity.
I leave it up to your physics teachers to decide what letters to use for all these things.
That is the analog of force divided by mass equals acceleration, which is d over dt of velocity.
And so now you see if the curl of a velocity field measure the angular velocity of its rotation then, by this analogy, the curl of a force field should measure the torque it exerts on a mass per unit moment of inertia.
Concretely, if you imagine that you are putting something in there, you know, if you are in a velocity field the curl will tell you how fast your guy is spinning at a given time.
If you put something that floats, for example, in your fluid, something very light then it is going to start spinning.
And the curl of a velocity field tells you how fast it is spinning at any given time up to a factor of two.
And the curl of a force field tells you how quickly the angular velocity is going to increase or decrease.
OK. Well, next time we are going to see Green's theorem which is actually going to tell us a lot more about curl and failure of conservativeness.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so last time we've seen the curl of the vector field with components M and N. We defined that to be N sub x minus M sub y.
And, we said this measures how far that vector field is from being conservative.
If the curl is zero, and if the field is defined everywhere, then it's going to be conservative.
And so, when I take the line integral along a closed curve, I don't have to compute it.
I notes going to be zero.
But now, let's say that I have a general vector field.
So, the curl will not be zero.
And, I still want to compute the line integral along a closed curve.
Well, I could compute it directly or there's another way.
And that's what we are going to see today.
So, say that I have a closed curve, C, and I want to find the work.
So, there's two options.
One is direct calculation, and the other one is Green's theorem.
So, Green's theorem is another way to avoid calculating line integrals if we don't want to.
OK, so what does it say?
It says if C is a closed curve enclosing a region R in the plane, and I have to insist C should go counterclockwise.
And, if I have a vector field that's defined and differentiable everywhere not only on the curve, C, which is what I need to define the line integral, but also on the region inside.
Then -- -- the line integral for the work done along C is actually equal to a double integral over the region inside of curl F dA.
OK, so that's the conclusion.
And, if you want me to write it in coordinates, maybe I should do that.
So, the line integral in terms of the components, that's the integral of M dx plus N dy.
And, the curl is (Nx-My)dA.
OK, so that's the other way to state it.
So, that's a really strange statement if you think about it because the left-hand side is a line integral.
OK, so the way we compute it is we take this expression Mdx Ndy and we parameterize the curve.
We express x and y in terms of some variable, t, maybe, or whatever you want to call it.
And then, you'll do a one variable integral over t. This right-hand side here, it's a double integral, dA.
So, we do it the way that we learn how to couple of weeks ago.
You take your region, you slice it in the x direction or in the y direction, and you integrate dx dy after setting up the bounds carefully, or maybe in polar coordinates r dr d theta.
But, see, the way you compute these things is completely different.
This one on the left-hand side lives only on the curve, while the right-hand side lives everywhere in this region inside.
So, here, x and y are related, they live on the curve.
Here, x and y are independent.
There just are some bounds between them.
And, of course, what you're integrating is different.
It's a line integral for work.
Here, it's a double integral of some function of x and y.
So, it's a very perplexing statement at first.
But, it's a very powerful tool.
So, we're going to try to see how it works concretely, what it says, what are the consequences, how we could convince ourselves that, yes, this works, and so on.
That's going to be the topic for today.
Any questions about the statement first?
No?
OK, yeah, one remark, sorry.
So, here, it stays counterclockwise.
What if I have a curve that goes clockwise?
Well, you could just take the negative, and integrate counterclockwise.
Why does the theorem choose counterclockwise over clockwise?
How doesn't know that it's counterclockwise rather than clockwise?
Well, the answer is basically in our convention for curl.
See, we've said curl is Nx minus My, and not the other way around.
And, that's a convention as well.
So, somehow, the two conventions match with each other.
That's the best answer I can give you.
So, if you met somebody from a different planet, they might have Green's theorem with the opposite conventions, with curves going clockwise, and the curl defined the other way around.
Probably if you met an alien, I'm not sure if you would be discussing Green's theorem first, but just in case.
OK, so that being said, there is a warning here which is that this is only for closed curves.
OK, so if I give you a curve that's not closed, and I tell you, well, compute the line integral, then you have to do it by hand.
You have to parameterize the curve.
Or, if you really don't like that line integral, you could close the path by adding some other line integral to it, and then compute using Green's theorem.
But, you can't use Green's theorem directly if the curve is not closed.
OK, so let's do a quick example.
So, let's say that I give you C, the circle of radius one, centered at the point (2,0).
So, it's out here.
That's my curve, C. And, let's say that I do it counterclockwise so that it will match with the statement of the theorem.
And, let's say that I want you to compute the line integral along C of ye^(-x) dx plus (one half of x squared minus e^(-x)) dy.
And, that's a kind of sadistic example, but maybe I'll ask you to do that.
So, how would you do it directly?
Well, to do it directly you would have to parameterize this curve.
So that would probably involve setting x equals two plus cosine theta y equals sine theta.
But, I'm using as parameter of the angle around the circle, it's like the unit circle, the usual ones that shifted by two in the x direction.
And then, I would set dx equals minus sine theta d theta.
I would set dy equals cosine theta d theta.
And, I will substitute, and I will integrate from zero to 2pi.
And, I would probably run into a bit of trouble because I would have these e to the minus x, which would give me something that I really don't want to integrate.
So, instead of doing that, which looks pretty much doomed, instead, I'm going to use Green's theorem.
So, using Green's theorem, the way we'll do it is I will, instead, compute a double integral.
So, I will -- -- compute the double integral over the region inside of curl F dA.
So, I should say probably what F was.
So, let's call this M. Let's call this N. And, then I will actually just choose the form coordinates, (Nx minus My) dA.
And, what is R here?
Well, R is the disk in here.
OK, so, of course, it might not be that pleasant because we'll also have to set up this double integral.
And, for that, we'll have to figure out a way to slice this region nicely.
We could do it dx dy.
We could do it dy dx.
Or, maybe we will want to actually make a change of variables to first shift this to the origin, you know, change x to x minus two and then switch to polar coordinates.
Well, let's see what happens later.
OK, so what is, so this is R. So, what is N sub x?
Well, N sub x is x plus e to the minus x minus, what is M sub y, e to the minus x, OK?
This is Nx.
This is My dA.
Well, it seems to simplify a bit.
I will just get double integral over R of x dA, which looks certainly a lot more pleasant.
Of course, I made up the example in that way so that it simplifies when you use Green's theorem.
But, you know, it gives you an example where you can turn are really hard line integral into an easier double integral.
Now, how do we compute that double integral?
Well, so one way would be to set it up.
Or, let's actually be a bit smarter and observe that this is actually the area of the region R, times the x coordinate of its center of mass.
If I look at the definition of the center of mass, it's the average value of x.
So, it's one over the area times the double integral of x dA, well, possibly with the density, but here I'm thinking uniform density one.
And, now, I think I know just by looking at the picture where the center of mass of this circle will be, right?
I mean, it would be right in the middle.
So, that is two, if you want, by symmetry.
And, the area of the guy is just pi because it's a disk of radius one.
So, I will just get 2pi.
I mean, of course, if you didn't see that, then you can also compute that double integral directly.
It's a nice exercise.
But see, here, using geometry helps you to actually streamline the calculation.
OK, any questions?
Yes?
OK, yes, let me just repeat the last part.
So, I said we had to compute the double integral of x dA over this region here, which is a disk of radius one, centered at, this point is (2,0).
So, instead of setting up the integral with bounds and integrating dx dy or dy dx or in polar coordinates, I'm just going to say, well, let's remember the definition of a center of mass.
It's the average value of a function, x in the region.
So, it's one over the area of origin times the double integral of x dA.
If you look, again, at the definition of x bar, it's one over area of double integral x dA.
Well, maybe if there's a density, then it's one over mass times double integral of x density dA.
But, if density is one, then it just becomes this.
So, switching the area, moving the area to the other side, I'll get double integral of x dA is the area of origin times the x coordinate of the center of mass.
The area of origin is pi because it's a unit disk.
And, the center of mass is the center of a disk.
So, its x bar is two, and I get 2 pi.
OK, that I didn't actually have to do this in my example today, but of course that would be good review.
It will remind you of center of mass and all that.
OK, any other questions?
No?
OK, so let's see, now that we've seen how to use it practice, how to avoid calculating the line integral if we don't want to.
Let's try to convince ourselves that this theorem makes sense.
OK, so, well, let's start with an easy case where we should be able to know the answer to both sides.
So let's look at the special case.
Let's look at the case where curl F is zero.
Then, well, we'd like to conclude that F is conservative.
That's what we said.
Well let's see what happens.
So, Green's theorem says that if I have a closed curve, then the line integral of F is equal to the double integral of curl on the region inside.
And, if the curl is zero, then I will be integrating zero.
I will get zero.
OK, so this is actually how you prove that if your vector field has curve zero, then it's conservative.
OK, so in particular, if you have a vector field that's defined everywhere the plane, then you take any closed curve.
Well, you will get that the line integral will be zero.
Straightly speaking, that will only work here if the curve goes counterclockwise.
But otherwise, just look at the various loops that it makes, and orient each of them counterclockwise and sum things together.
So let me state that again.
So, OK, so a consequence of Green's theorem is that if F is defined everywhere in the plane -- -- and the curl of F is zero everywhere, then F is conservative.
And so, this actually is the input we needed to justify our criterion.
The test that we saw last time saying, well, to check if something is a gradient field if it's conservative, we just have to compute the curl and check whether it's zero.
OK, so how do we prove that now carefully?
Well, you just take a closed curve in the plane.
You switch the orientation if needed so it becomes counterclockwise.
And then you look at the region inside.
And then you know that the line integral inside will be equal to the double integral of curl, which is the double integral of zero.
Therefore, that's zero.
But see, OK, so now let's say that we try to do that for the vector field that was on your problems that was not defined at the origin.
So if you've done the problem sets and found the same answers that I did, then you will have found that this vector field had curve zero everywhere.
But still it wasn't conservative because if you went around the unit circle, then you got a line integral that was 2pi.
Or, if you compared the two halves, you got different answers for two parts that go from the same point to the same point.
So, it fails this property but that's because it's not defined everywhere.
So, what goes wrong with this argument?
Well, if I take the vector field that was in the problem set, and if I do things, say that I look at the unit circle.
That's a closed curve.
So, I would like to use Green's theorem.
Green's theorem would tell me the line integral along this loop is equal to the double integral of curl over this region here, the unit disk.
And, of course the curl is zero, well, except at the origin.
At the origin, the vector field is not defined.
You cannot take the derivatives, and the curl is not defined.
And somehow that messes things up.
You cannot apply Green's theorem to the vector field.
So, you cannot apply Green's theorem to the vector field on problem set eight problem two when C encloses the origin.
And so, that's why this guy, even though it has curl zero, is not conservative.
There's no contradiction.
And somehow, you have to imagine that, well, the curl here is really not defined.
But somehow it becomes infinite so that when you do the double integral, you actually get 2 pi instead of zero.
I mean, that doesn't make any sense, of course, but that's one way to think about it.
OK, any questions?
Yes?
Well, though actually it's not defined because the curl is zero everywhere else.
So, if a curl was well defined at the origin, you would try to, then, take the double integral.
no matter what value you put for a function, if you have a function that's zero everywhere except at the origin, and some other value at the origin, the integral is still zero.
So, it's worse than that.
It's not only that you can't compute it, it's that is not defined.
OK, anyway, that's like a slightly pathological example.
Yes?
Well, we wouldn't be able to because the curl is not defined at the origin.
So, you can actually integrate it.
OK, so that's the problem.
I mean, if you try to integrate, we've said everywhere where it's defined, the curl is zero.
So, what you would be integrating would be zero.
But, that doesn't work because at the origin it's not defined.
Yes?
Ah, so if you take a curve that makes a figure 8, then indeed my proof over there is false.
So, I kind of tricked you.
It's not actually correct.
So, if the curve does a figure 8, then what you do is you would actually cut it into its two halves.
And for each of them, you will apply Green's theorem.
And then, you'd still get, if a curl is zero then this line integral is zero.
That one is also zero.
So this one is zero.
OK, small details that you don't really need to worry too much about, but indeed if you want to be careful with details then my proof is not quite complete.
But the computation is still true.
Let's move on.
So, I want to tell you how to prove Green's theorem because it's such a strange formula that where can it come from possibly?
I mean, so let me remind you first of all the statement we want to prove is that the line integral along a closed curve of Mdx plus Ndy is equal to the double integral over the region inside of (Nx minus My)dA.
And, let's simplify our lives a bit by proving easier statements.
So actually, the first observation will actually prove something easier, namely, that the line integral, let's see, of Mdx along a closed curve is equal to the double integral over the region inside of minus M sub y dA.
OK, so that's the special case where N is zero, where you have only an x component for your vector field.
Now, why is that good enough?
Well, the claim is if I can prove this, I claim you will be able to do the same thing to prove the other case where there is only the y component.
And then, if the other together, you will get the general case.
So, let me explain.
OK, so a similar argument which I will not do, to save time, will show, so actually it's just the same thing but switching the roles of x and y, that if I integrate along a closed curve N dy, then I'll get the double integral of N sub x dA.
And so, now if I have proved these two formulas separately, then if you sum them together will get the correct statement.
Let me write it.
We get Green's theorem.
OK, so we've simplified our task a little bit.
We'll just be trying to prove the case where there's only an x component.
So, let's do it.
Well, we have another problem which is the region that we are looking at, the curve that we're looking at might be very complicated.
If I give you, let's say I give you, I don't know, a curve that does something like this.
Well, it will be kind of tricky to set up a double integral over the region inside.
So maybe we first want to look at curves that are simpler, that will actually allow us to set up the double integral easily.
So, the second observation, so that was the first observation.
The second observation is that we can decompose R into simpler regions.
So what do I mean by that?
Well, let's say that I have a region and I'm going to cut it into two.
So, I'll have R1 and R2.
And then, of course, I need to have the curves that go around them.
So, I had my initial curve, C, was going around everybody.
They have curves C1 that goes around R1, and C2 goes around R2.
OK, so, what I would like to say is if we can prove that the statement is true, so let's see, for C1 and also for C2 -- -- then I claim we can prove the statement for C. How do we do that?
Well, we just add these two equalities together.
OK, why does that work?
There's something fishy going on because C1 and C2 have this piece here in the middle.
That's not there in C. So, if you add the line integral along C1 and C2, you get these unwanted pieces.
But, the good news is actually you go twice through that edge in the middle.
See, it appears once in C1 going up, and once in C2 going down.
So, in fact, when you will do the work, when you will sum the work, you will add these two guys together.
They will cancel.
OK, so the line integral along C will be, then, it will be the sum of the line integrals on C1 and C2.
And, that will equal, therefore, the double integral over R1 plus the double integral over R2, which is the double integral over R of negative My.
OK and the reason for this equality here is because we go twice through the inner part.
What do I want to say?
Along the boundary between R1 and R2 -- -- with opposite orientations.
So, the extra things cancel out.
OK, so that means I just need to look at smaller pieces if that makes my life easier.
So, now, will make my life easy?
Well, let's say that I have a curve like that.
Well, I guess I should really draw a pumpkin or something like that because it would be more seasonal.
But, well, I don't really know how to draw a pumpkin.
OK, so what I will do is I will cut this into smaller regions for which I have a well-defined lower and upper boundary so that I will be able to set up a double integral, dy dx, easily.
So, a region like this I will actually cut it here and here into five smaller pieces so that each small piece will let me set up the double integral, dy dx.
OK, so we'll cut R in to what I will call vertically simple -- -- regions.
So, what's a vertically simple region?
That's a region that's given by looking at x between a and b for some values of a and b.
And, for each value of x, y is between some function of x and some other function of x. OK, so for example, this guy is vertically simple.
See, x runs from this value of x to that value of x.
And, for each x, y goes between this value to that value.
And, same with each of these.
OK, so now we are down to the main step that we have to do, which is to prove this identity if C is, sorry, if -- -- if R is vertically simple -- -- and C is the boundary of R going counterclockwise.
OK, so let's look at how we would do it.
So, we said vertically simple region looks like x goes between a and b, and y goes between two values that are given by functions of x. OK, so this is y equals f2 of x.
This is y equals f1 of x.
This is a.
This is b.
Our region is this thing in here.
So, let's compute both sides.
And, when I say compute, of course we will not get numbers because we don't know what M is.
We don't know what f1 and f2 are.
But, I claim we should be able to simplify things a bit.
So, let's start with the line integral.
How do I compute the line integral along the curve that goes all around here?
Well, it looks like there will be four pieces.
OK, so we actually have four things to compute, C1, C2, C3, and C4.
OK?
Well, let's start with C1.
So, if we integrate on C1 Mdx, how do we do that?
Well, we know that on C1, y is given by a function of x.
So, we can just get rid of y and express everything in terms of x. OK, so, we know y is f1 of x, and x goes from a to b.
So, that will be the integral from a to b of, well, I have to take the function, M. And so, M depends normally on x and y.
Maybe I should put x and y here.
And then, I will plug y equals f1 of x dx.
And, then I have a single variable integral.
And that's what I have to compute.
Of course, I cannot compute it here because I don't know what this is.
So, it has to stay this way.
OK, next one.
The integral along C2, well, let's think for a second.
On C2, x equals b.
It's constant.
So, dx is zero, and you would integrate, actually, above a variable, y.
But, well, we don't have a y component.
See, this is the reason why we made the first observation.
We got rid of the other term because it's simplifies our life here.
So, we just get zero.
OK, just looking quickly ahead, there's another one that would be zero as well, right?
Which one?
Yeah, C4.
This one gives me zero.
What about C3?
Well, C3 will look a lot like C1.
So, we're going to use the same kind of thing that we did with C. OK, so along C3, well, let's see, so on C3, y is a function of x, again.
And so we are using as our variable x, but now x goes down from b to a.
So, it will be the integral from b to a of M of (x and f2 of x) dx.
Or, if you prefer, that's negative integral from a to b of M of (x and f2 of x) dx.
OK, so now if I sum all these pieces together, I get that the line integral along the closed curve is the integral from a to b of M(x1f1 of x) dx minus the integral from a to b of M(x1f2 of x) dx.
So, that's the left hand side.
Next, I should try to look at my double integral and see if I can make it equal to that.
So, let's look at the other guy, double integral over R of negative MydA.
Well, first, I'll take the minus sign out.
It will make my life a little bit easier.
And second, so I said I will try to set this up in the way that's the most efficient.
And, my choice of this kind of region means that it's easier to set up dy dx, right?
So, if I set it up dy dx, then I know for a given value of x, y goes from f1 of x to f2 of x.
And, x goes from a to b, right?
Is that OK with everyone?
OK, so now if I compute the inner integral, well, what do I get if I get partial M partial y with respect to y?
I'll get M back, OK?
So -- So, I will get M at the point x f2 of x minus M at the point x f1 of x.
And so, this becomes the integral from a to b. I guess that was a minus sign, of M of (x1f2 of x) minus M of (x1f1 of x) dx.
And so, that's the same as up there.
And so, that's the end of the proof because we've checked that for this special case, when we have only an x component and a vertically simple region, things work.
Then, we can remove the assumption that things are vertically simple using this second observation.
We can just glue the various pieces together, and prove it for any region.
Then, we do same thing with the y component.
That's the first observation.
When we add things together, we get Green's theorem in its full generality.
OK, so let me finish with a cool example.
So, there's one place in real life where Green's theorem used to be extremely useful.
I say used to because computers have actually made that obsolete.
But, so let me show you a picture of this device.
This is called a planimeter.
And what it does is it measures areas.
So, it used to be that when you were an experimental scientist, you would run your chemical or biological experiment or whatever.
And, you would have all of these recording devices.
And, the data would go, well, not onto a floppy disk or hard disk or whatever because you didn't have those at the time.
You didn't have a computer in your lab.
They would go onto a piece of graph paper.
So, you would have your graph paper, and you would have some curve on it.
And, very often, you wanted to know, what's the total amount of product that you have synthesized, or whatever the question might be.
It might relate with the area under your curve.
So, you'd say, oh, it's easy.
Let's just integrate, except you don't have a function.
You can put that into calculator.
The next thing you could do is, well, let's count the little squares.
But, if you've seen a piece of graph paper, that's kind of time-consuming.
So, people invented these things called planimeters.
It's something where there is a really heavy thing based at one corner, and there's a lot of dials and gauges and everything.
And, there's one arm that you move.
And so, what you do is you take the moving arm and you just slide it all around your curve.
And, you look at one of the dials.
And, suddenly what comes, as you go around, it gives you complete garbage.
But when you come back here, that dial suddenly gives you the value of the area of this region.
So, how does it work?
This gadget never knows about the region inside because you don't take it all over here.
You only take it along the curve.
So, what it does actually is it computes a line integral.
OK, so it has this system of wheels and everything that compute for you the line integral along C of, well, it depends on the model.
But some of them compute the line integral of x dy.
Some of them compute different line integrals.
But, they compute some line integral, OK?
And, now, if you apply Green's theorem, you see that when you have a counterclockwise curve, this will be just the area of the region inside.
And so, that's how it works.
I mean, of course, now you use a computer and it does the sums.
Yes?
That costs several thousand dollars, possibly more.
So, that's why I didn't bring one.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Today I am going to tell you about flux of a vector field for a curve.
In case you have seen flux in physics, probably you have seen flux in space, and we are going to come to that in a couple of weeks, but for now we are still doing everything in the plane.
So bear with me if you have seen a more complicated version of flux.
We are going to do the easy one first.
What is flux?
Well, flux is actually another kind of line integral.
Let's say that I have a plane curve and a vector field in the plane.
Then the flux of F across a curve C is, by definition, a line integral, but I will use notation F dot n ds.
I have to explain to you what it means, but let me first box that because that is the important formula to remember.
That is the definition.
What does that mean?
First, mostly I have to tell you what this little n is.
The notation suggests it is a normal vector, so what does that mean?
I have a curve in the plane and I have a vector field.
Let's see.
The vector field will be yellow today.
And I will want to integrate along the curve the dot product of F with the normal vector to the curve, a unit normal vector to the curve.
That means a vector that is at every point of the curve perpendicular to the curve and has length one.
N everywhere will be the unit normal vector to the curve C pointing 90 degrees clockwise from T. What does that mean?
That means I have two normal vectors, one that is pointing this way, one that is pointing that way.
I have to choose a convention.
And the convention is that the normal vector that I take goes to the right of the curve as I am traveling along the curve.
You mentioned that you were walking along this curve, then you look to your right, that is that direction.
What we will do is just, at every point along the curve, the dot product between the vector field and the normal vector.
And we will sum that along the various pieces of the curve.
What this notation means is that if we actually break C into small pieces of length delta(s) then the flux will be the limit, as the pieces become smaller and smaller, of the sum of F dot n delta S. I take each small piece of my curve, I do the dot product between F and n and I multiply by the length of a piece.
And then I add these together.
That is what the line integral means.
Of course that is, again, not how I will compute it.
Just to compare this with work, conceptually it is similar to the line integral we did for work except the line integral for work -- Work is the line integral of F dot dr, which is also the line integral of F dot T ds.
That is how we reformulated it.
That means we take our curve and we figure out at each point how big the tangent component -- I guess I should probably take the same vector field as before.
Let's see.
My field was pointing more like that way.
What I do at any point is project F to the tangent direction, I figure out how much F is going along my curve and then I sum these things together.
I am actually summing -- -- the tangential component of my field F. Roughly-speaking the work measures, you know, when I move along my curve, how much I am going with or against F. Flux, on the other hand, measures, when I go along the curve, roughly how much the field is going to across the curve.
Counting positively what goes to the right, negatively what goes to the left.
Flux is integral F dot n ds, and that one corresponds to summing the normal component of a vector field.
But apart from that conceptually it is the same kind of thing.
Just the physical interpretations will be very different, but for a mathematician these are two line integrals that you set up and compute in pretty much the same way.
Let's see.
I should probably tell you what it means.
Why do we make this definition?
What does it correspond to?
Well, the interpretation for work made a lot of sense when F was representing a force.
The line integral was actually the work done by the force.
The interpretation for flux makes more sense if you think of F as a velocity field.
What is the interpretation?
Let's say that for F is a velocity field.
That means I am thinking of some fluid that is moving, maybe water or something, and it is moving at a certain speed.
And my vector field represents how things are moving at every point of the plane.
I claim that flux measures how much fluid passes through -- -- the curve C per unit time.
If you imagine that maybe you have a river and you are somehow building a damn here, a damn with holes in it so that the water still passes through, then this measures how much water passes through your membrane per unit time.
Let's try to figure out why this is true.
Why does this make sense?
Let's look at what happens on a small portion of our curve C. I am zooming in on my curve C. I guess I need to zoom further.
That is a little piece of my curve, of length delta S, and there is a fluid flow.
On my picture things are flowing to the right.
Here I am drawing a constant vector field because if you zoom in enough then your vectors will pretty much be the same everywhere.
If you enlarge the picture enough then things will be pretty much a uniform flow.
Now, how much stuff goes through this little piece of curve per unit time?
Well, what happens over time is the fluid is moving while my curve is staying the same place so it corresponds to something like this.
I claim that what goes through C in unit time is actually going to be a parallelogram.
Here is a better picture.
I claim that what will be going through C is this shaded parallelogram to the left of C. Let's see.
If I move for unit time it works.
That is the stuff that goes through my curve, for a small portion of curve in unit time.
And, of course, I would need to add all of these together to get the entire curve.
Let's try to understand how big this parallelogram is.
To know how big this parallelogram is I would like to use base times height or something like that.
And maybe I want to actually flip my picture so that the base and the height make more sense to me.
Let me actually turn it this way.
And, in case you have trouble reading the rotated picture, let me redo it on the board.
What passes through a portion of C in unit time is the contents of a parallelogram whose base is on C. So it has length delta s. That is a piece of C. And the other side is going to be given by my velocity vector F. And to find the height of this thing, I need to know what actually the normal component of this vector is.
If I call n the unit normal vector to the curve then the area is base times height.
The base is delta S and the height is the normal component of F, so it is F dot n. And so you see that when you sum these things together you get, what I said, flux.
Now, if you are worried about the fact that actually -- If your unit time is too long then of course things might start changing as it flows.
You have to take the time unit and the length unit that are sufficiently small so that really this approximation where C is a straight line and where flow is at constant speed are valid.
You want to take maybe a segment here that is a few micrometers.
And the time unit might be a few nanoseconds or whatever, and then it is a good approximation.
What I mean by per unit time is, well, actually, that works, but you want to think of a really, really small time.
And then the amount of matter that passes in that really, really small time is the flux times the amount of time.
Let's be a tiny bit more careful.
And what I am saying is the amount of stuff that passes through C depends actually on whether n is going this way or the opposite way.
Actually, what is implicit in this explanation is that I am counting positively all the stuff that flows across C in the direction of n and negatively what flows in the opposite direction.
What flows to the right of C, well, across C from left to right is counted positively.
While what flows right to left is counted negatively.
So, in fact, it is the net flow through C per unit time.
Any questions about the definition or the interpretation or things like that?
Yes?
Well, you can have both not in the same small segment.
But it could be that, well, imagine that my vector field accidentally goes in the opposite direction then this part of the curve, while things are flowing to the left, contributes negatively to flux.
And here maybe the field is tangent so the normal component becomes zero.
And then it becomes positive and this part of the curve contributes positively.
For example, if you imagine that you have a round tank in which the fluid is rotating and you put your dam just on a diameter across then things are going one way on one side, the other way on the other side, and actually it just evens out.
We don't have complete information.
It is just the total net flux.
OK.
If there are no other questions then I guess we will need to figure out how to compute this guy and how to actually do this line integral.
Well, let's start with a couple of easy examples.
Let's say that C is a circle of radius (a) centered at the origin going counterclockwise.
And let's say that our vector field is xi yj.
What does that look like?
Remember, xi plus yj is a vector field that is pointing radially away from the origin.
Because at every point it is equal to the vector from the origin to that point.
Now, if we have a circle and let's say we are going counterclockwise.
Actually, I have a nicer picture.
Let me do it here.
That is my curve and my vector field.
And the normal vector, see, when you go counterclockwise in a closed curve, this convention that a normal vector points to the right of curve makes it point out.
The usual convention, when you take flux for a closed curve, is that you are counting the flux going out of the region enclosed by the curve.
And, of course, if you went clockwise it would be the other way around.
You choose to do it the way you want, but the most common one is to count flux going out of the region.
Let's see what happens.
Well, if I am anywhere on my circle, see, the normal vector is sticking straight out of the circle.
That is a property of the circle that the radial direction is perpendicular to the circle.
Actually, let me complete this picture.
If I take a point on the circle, I have my normal vector that is pointing straight out so it is parallel to F. Along C we know that F is parallel to n, so F dot n will be equal to the magnitude of F times, well, the magnitude of n, but that is one.
Let me put it anywhere, but that is the unit normal vector.
Now, what is the magnitude of this vector field if I am at a point x, y?
Well, it is square root of x squared plus y squared, which is the same as the distance from the origin.
So if this distance, if this radius is a then the magnitude of F will just be a.
In fact, F dot n is constant, always equal to a.
So the line integral will be pretty easy because all I have to do is the integral of F dot n ds becomes the integral of a ds.
(a) is a constant so I can take it out.
And integral ds is just a length of C which is 2pi a, so I will get 2pi a squared.
And that is positive, as we expected, because stuff is flowing out of the circle.
Any questions about that?
No.
OK. Just out of curiosity, let's say that we had taken our other favorite vector field.
Let's say that we had the same C, but now the vector field .
Remember, that one goes counterclockwise around the origin.
If you remember what we did several times, well, along the circle that vector field now is tangent to the circle.
If it is tangent to the circle it doesn't have any normal component.
The normal component is zero.
Things are not flowing into the circle or out of it.
They are just flowing along the circle around and around so the flux will be zero.
F now is tangent to C. F dot n is zero and, therefore, the flux will be zero.
These are examples where you can compute things geometrically.
And I would say, generally speaking, with flux, well, if it is a very complicated field then you cannot.
But, if a field is fairly simple, you should be able to get some general feeling for whether your answer should be positive, negative or zero just by thinking about which way is my flow going.
Is it going across the curve one way or the other way?
Still no questions about these examples?
The next thing we need to know is how we will actually compute these things because here, yeah, it works pretty well, but what if you don't have a simple geometric interpretation.
What if I give you a really complicated curve and then you have trouble finding the normal vector?
It is going to be annoying to set up things this way.
Actually, there is a better way to do it in coordinates.
Just as we do work, when we compute this line integral, usually we don't do it geometrically like this.
Most of the time we just integrate M dx plus N dy in coordinates.
That is a similar way to do it because it is, again, a line integral so it should work the same way.
Let's try to figure that out.
How do we do the calculation in coordinates, or I should say using components?
That is the general method of calculation when we don't have something geometric to do.
Remember, when we were doing things for work we said this vector dr, or if you prefer T ds, we said just becomes symbolically dx and dy.
When you do the line integral of F dot dr you get line integral of n dx plus n dy.
Now let's think for a second about how we would express n ds.
Well, what is n ds compared to T ds?
Well, M is just T rotated by 90 degrees, so n ds is T ds rotated by 90 degrees.
That might sound a little bit outrageous because these are really symbolic notations but it works.
I am not going to spend too much time trying to convince you carefully.
But if you go back to where we wrote this and how we tried to justify this and you work your way through it, you will see that n ds can be analyzed the same way.
N is T rotated 90 degrees clockwise.
That tells us that n ds is -- How do we rotate a vector by 90 degrees?
Well, we swept the two components and we put a minus sign.
You have dy and dx.
And you have to be careful where to put the minus sign.
Well, if you are doing it clockwise, it is in front of dx.
Well, actually, let me just convince you quickly.
Let's say we have a small piece of C. If we do T delta S, that is also vector delta r. That is going to be just the vector that goes along the curve given by this.
Its components will be indeed the change in x, delta x, and the change in y, delta y.
And now, if I want to get n delta S, well, I claim now that it is perfectly valid and rigorous to just rotate that by 90 degrees.
If I want to rotate this by 90 degrees clockwise then the x component will become the same as the old y component.
And the y component will be minus delta x.
Then you take the limit when the segment becomes shorter and shorter, and that is how you can justify this.
That is the key to computing things in practice.
It means, actually, you already know how to compute line integrals for flux.
Let me just write it explicitly.
Let's say that our vector field has two components.
And let me just confuse you a little bit and not call them M and N for this time just to stress the fact that we are doing a different line integral.
Let me call them P and Q for now.
Then the line integral of F dot n ds will be the line integral of dot product .
That will be the integral of - Q dx P dy.
Well, I am running out of space here.
It is integral along C of negative Q dx plus P dy.
And from that point onwards you just do it the usual way.
Remember, here you have two variables x and y but you are integrating along a curve.
If you are integrating along a curve x and y are related.
They depend on each other or maybe on some other parameter like T or theta or whatever.
You express everything in terms of a single variable and then you do a usual single integral.
Any questions about that?
I see a lot of confused faces so maybe I shouldn't have called my component P and Q.
If you prefer, if you are really sentimentally attached to M and N then this new line integral becomes the integral of - N dx M dy.
If a problem tells you compute flux instead of saying compute work, the only thing you change is instead of doing M dx plus N dy you do minus N dx plus M dy.
And I am sorry to say that I don't have any good way of helping you remember which one of the two gets the minus sign, so you just have to remember this formula by heart.
That is the only way I know.
Well, you can try to go through this argument again, but it is really best if you just remember that formula.
I am not going to do an example because we already know how to do line integrals.
Hopefully you will get to see one at least in recitation on Monday.
That is all pretty good.
Let me tell you now what if I have to compute flux along a closed curve and I don't want to compute it?
Well, remember in the case of work we had Green's theorem.
We saw yesterday Green's theorem.
Let's us replace a line integral along a closed curve by a double integral.
Well, here it is the same.
We have a line integral along a curve.
If it is a closed curve, we should be able to replace it by a double integral.
There is a version of Green's theorem for flux.
And you will see it is not scarier than the other one.
It is perhaps less scarier or perhaps just as scary or just not as scary, depending on how you feel about it, but it works pretty much the same way.
What does Green's theorem for flux say?
It says if C is a curve that encloses a region R counterclockwise and if I have a vector field that is defined everywhere, not just on C but also inside, so also on R. Well, maybe I should give names to the components.
If you will forgive me for a second, I will still use P and Q for now.
You will see why.
It is defined and differentiable in R. Then I can actually -- -- replace the line integral for flux by a double integral over R of some function.
And that function is called the divergence of F dA.
This is the divergence of F. And I have to define for you what this guy is.
The divergence of a vector field with components P and Q is just P sub x Q sub y.
This one is actually easier to remember than curl because you just take the x component, take its partial with respect to x, take the y component, take its partial with respect to y and add them together.
No signs.
No switching things around.
This one is pretty straightforward.
The picture again is if I have my curve C going counterclockwise around a region R and I want to find the flux of some vector field F that is everywhere in here.
Maybe some parts of C will contribute positively and some parts will contribute negatively.
Just to reiterate what I said, positively here means, because we are going counterclockwise, the normal vector points out of the region.
This guy here is the flux out of R through C. That is the formula.
Any questions about what the statement says or how to use it concretely?
No.
OK.
It is pretty similar to Green's theorem for work.
Actually, I should say -- This is called Green's theorem in normal form also.
Not that the other one is abnormal, but just that the old one for work was, you could say, in tangential form.
That just means, well, Green's theorem, as seen yesterday was for the line integral F dot T ds, integrating the tangent component of F. The one today is for integrating the normal component of F. OK. Let's prove this.
Good news.
It is much easier to prove than the one we did yesterday because we are just going to show that it is the same thing just using different notations.
How do I prove it?
Well, maybe actually it would help if first, before proving it, I actually rewrite what it means in components.
We said the line integral of F dot n ds is actually the line integral of - Q dx P dy.
And we want to show that this is equal to the double integral of P sub x Q sub y dA.
This is really one of the features of Green's theorem.
No matter which form it is, it relates a line integral to a double integral.
Let's just try to see if we can reduce it to the one we had yesterday.
Let me forget what these things mean physically and just focus on the math.
On the math it is a line integral of something dx plus something dy.
Let's call this guy M and let's call this guy N. Let M equal negative Q and N equal P. Then this guy here becomes integral of M dx plus N dy.
And I know from yesterday what this is equal to, namely using the tangential form of Green's theorem.
Green for work.
This is the double integral of curl of this guy.
That is Nx minus My dA.
But now let's think about what this is in terms of M and N. Well, we said that M is negative Q so this is negative My.
And we said P is the same as N, so this is Nx.
Just by renaming the components, I go from one form to the other one.
So it is really the same theorem.
That's why it is also called Green's theorem.
But the way we think about it when we use it is different, because one of them computes the work done by a force along a closed curve, the other one computes the flux maybe of a velocity field out of region.
Questions?
Yes?
That is correct.
If you are trying to compute a line integral for flux, wait, where did I put it?
A line integral for flux just becomes this.
And once you are here you know how to compute that kind of thing.
The double integral side does not even have any kind of renaming to do.
You know how to compute a double integral of a function.
This is just a particular kind of function that you get out of a vector field, but it is like any function.
The way you would evaluate these double integrals is just the usual way.
Namely, you have a function of x and y, you have a region and you set up the bounds for the isolated integral.
The way you would evaluate the double integrals is really the usual way, by slicing the region and setting up the bounds for iterated integrals in dx, dy or dydx or maybe rd, rd theta or whatever you want.
In fact, in terms of computing integrals, we just have two sets of skills.
One is setting up and evaluating double integrals.
The other one is setting up and evaluating line integrals.
And whether these line integrals or double integrals are representing work, flux, integral of a curve, whatever, the way that we actually compute them is the same.
Let's do an example.
Oh, first.
Sorry.
This renaming here, see, that is why actually I call my components P and Q because the argument would have gotten very messy if I had told you now I call M ,N and I call N minus M and so on.
But, now that we are through with this, if you still like M and N better, you know, what this says -- The formulation of Green's theorem in this language is just integral of minus N dx plus M dy is the double integral over R of Mx plus Ny dA.
Now let's do an example.
Let's look at this picture again, the flux of xi plus yj out of the circle of radius A.
We did the calculation directly using geometry, and it wasn't all that bad.
But let's see what Green's theorem does for us here.
Example.
Let's take the same example as last time.
F equals xi yj.
C equals circle of radius a counterclockwise.
How do we set up Green's theorem.
Well, let's first figure out the divergence of F. The divergence of this field, I take the x component, which is x, and I take its partial respect to x.
And then I do the same with the y component, and I will get one plus one equals two.
So, the divergence of this field is two.
Now, Green's theorem tells us that the flux out of this region is going to be the double integral of 2 dA.
What is R now?
Well, R is the region enclosed by C. So if C is the circle, R is the disk of radius A.
Of course, we can compute it, but we don't have to because double integral of 2dA is just twice the double integral of dA so it is twice the area of R. And we know the area of a circle of radius A.
That is piA2.
So, it is 2piA2.
That is the same answer that we got directly, which is good news.
Now we can even do better.
Let's say that my circle is not at the origin.
Let's say that it is out here.
Well, then it becomes harder to calculate the flux directly.
And it is harder even to guess exactly what will happen because on this side here the vector field will go into the region so the contribution to flux will be negative here.
Here it will be positive because it is going out of the region.
There are positive and negative terms.
Well, it looks like positive should win because here the vector field is much larger than over there.
But, short of computing it, we won't actually know what it is.
If you want to do it by direct calculation then you have to parametize this circle and figure out what the line integral will be.
But if you use Green's theorem, well, we never used the fact that it is the circle of radius A at the origin.
It is true actually for any closed curve that the flux out of it is going to be twice the area of the region inside.
It still will be 2piA2 even if my circle is anywhere else in the plane.
If I had asked you a trick question where do you want to place this circle so that that the flux is the largest?
Well, the answer is it doesn't matter.
Now, let's just finish quickly by answering a question that some of you, I am sure, must have, which is what does divergence mean and what does it measure?
I mean, we said for curl, curl measures how much things are rotating somehow.
What does divergence mean?
Well, the answer is divergence measures how much things are diverging.
Let's be more explicit.
Interpretation of divergence.
You can think of it, you know, what do I want to say first?
If you take a vector field that is a constant vector field where everything just translates then there is no divergence involved because the derivatives will be zero.
If you take the guy that rotates things around you will also compute and find zero for divergence.
This is not sensitive to translation motions where everything moves together or to rotation motions, but instead it is sensitive to explaining motions.
A possible answer is that it measures how much the flow is expanding areas.
If you imagine this flow that we have here on the picture, things are moving away from the origin and they fill out the plane.
If we mention this fluid flowing out there, it is occupying more and more space.
And so that is what it means to have positive divergence.
If you took the opposite vector field that contracts everything to the origin that will have negative divergence.
That is a good way to think about it if you are thinking of a gas maybe that can expand to fill out more volume.
If you thinking of water, well, water doesn't really shrink or expand.
The fact that it is taking more and more space actually means that there is more and more water.
The other way to think about it is divergence is the source rate, it is the amount of fluid that is being inserted into the system, that is being pumped into the system per unit time per unit area.
What div F equals two here means is that here you actually have matter being created or being pumped into the system so that you have more and more water filling more and more space as it flows.
But, actually, divergence is not two just at the origin.
It is two everywhere.
So, in fact, to have this you need to have a system of pumps that actually is in something water absolutely everywhere uniformly.
That is the only way to do this.
I mean if you imagine that you just have one spring at the origin then, sure, water will flow out, but as you go further and further away it will do so more and more slowly.
Well, here it is flowing away faster and faster.
And that means everywhere you are still pumping more water into it.
So, that is what divergence measures.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
And, well let's see.
So, before we actually start reviewing for the test, I still have to tell you a few small things because I promised to say a few words about what's the difference, or precisely, what's the difference between curl being zero and a field being a gradient field, and why we have this assumption that our vector field had to be defined everywhere for a field with curl zero to actually be conservative for our test for gradient fields to be valid?
So -- More about validity of Green's theorem and things like that.
So, we've seen the statement of Green's theorem in two forms.
Both of them have to do with comparing a line integral along a closed curve to a double integral over the region inside enclosed by the curve.
So, one of them says the line integral for the work done by a vector field along a closed curve counterclockwise is equal to the double integral of a curl of a field over the enclosed region.
And, the other one says the total flux out of the region, so, the flux through the curve is equal to the double integral of divergence of a field in the region.
So, in both cases, we need the vector field to be defined not only, I mean, the left hand side makes sense if a vector field is just defined on the curve because it's just a line integral on C. We don't care what happens inside.
But, for the right-hand side to make sense, and therefore for the equality to make sense, we need the vector field to be defined everywhere inside the region.
So, I said, if there is a point somewhere in here where my vector field is not defined, then it doesn't work.
And actually, we've seen that example.
So, this only works if F and its derivatives are defined everywhere in the region, R. Otherwise, we are in trouble.
OK, so we've seen for example that if I gave you the vector field minus yi xj over x squared plus y squared, so that's the same vector field that was on that problem set a couple of weeks ago.
Then, well, f is not defined at the origin, but it's defined everywhere else.
And, wherever it's defined, it's curl is zero.
I should say everywhere it's -- And so, if we have a closed curve in the plane, well, there's two situations.
One is if it does not enclose the origin.
Then, yes, we can apply Green's theorem and it will tell us that it's equal to the double integral in here of curl F dA, which will be zero because this is zero.
However, if I have a curve that encloses the origin, let's say like this, for example, then, well, I cannot use the same method because the vector field and its curl are not defined at the origin.
And, in fact, you know that ignoring the problem and saying, well, the curl is still zero everywhere, will give you the wrong answer because we've seen an example.
We've seen that along the unit circle the total work is 2 pi not zero.
So, we can't use Green.
However, we can't use it directly.
So, there is an extended version of Green's theorem that tells you the following thing.
Well, it tells me that even though I can't do things for just this region enclosed by C prime, I can still do things for the region in between two different curves.
OK, so let me show you what I have in mind.
So, let's say that I have my curve C'.
Where's my yellow chalk?
Oh, here.
So, I have this curve C'.
I can't apply Green's theorem inside it, but let's get out the smaller thing.
So, that one I'm going to make going clockwise.
You will see why.
Then, I could say, well, let me change my mind.
This picture is not very well prepared.
That's because my writer is on strike.
OK, so let's say we have C' and C'' both going counterclockwise.
Then, I claim that Green's theorem still applies, and tells me that the line integral along C prime minus the line integral along C double prime is equal to the double integral over the region in between.
So here, now, it's this region with the hole of the curve.
And, well, in our case, that will turn out to be zero because curl is zero.
OK, so this doesn't tell us what each of these two line integrals is.
But actually, it tells us that they are equal to each other.
And so, by computing one, you can see actually that for this vector field, if you take any curve that goes counterclockwise around the origin, you would get two pi no matter what the curve is.
So how do you get to this?
Why is this not like conceptually a new theorem?
Well, just think of the following thing.
I'm not going to do it on top of that because it's going to be messy if I draw too many things.
But, so here I have my C''.
Here, I have C'.
Let me actually make a slit that will connect them to each other like this.
So now if I take, see, I can form a single closed curve that will enclose all of this region with kind of an infinitely thin slit here counterclockwise.
And so, if I go counterclockwise around this region, basically I go counterclockwise along the outer curve.
Then I go along the slit.
Then I go clockwise along the inside curve, then back along the slit.
And then I'm done.
So, if I take the line integral along this big curve consisting of all these pieces, now I can apply Green's theorem to that because it is the usual counterclockwise curve that goes around a region where my field is well-defined.
See, I've eliminated the origin from the picture.
And, so the total line integral for this thing is equal to the integral along C prime, I guess the outer one.
Then, I also need to have what I do along the inner side.
And, the inner side is going to be C double prime, but going backwards because now I'm going clockwise on C prime so that I'm going counterclockwise around the shaded region.
Well, of course there will be contributions from the line integral along this wide segment.
But, I do it twice, once each way.
So, they cancel out.
So, the white segments cancel out.
You probably shouldn't, in your notes, write down white segments because probably they are not white on your paper.
But, hopefully you get the meaning of what I'm trying to say.
OK, so basically that tells you, you can still play tricks with Green's theorem when the region has holes in it.
You just had to be careful and somehow subtract some other curve so that together things will work out.
There is a similar thing with the divergence theorem, of course, with flux and double integral of div f, you can apply exactly the same argument.
OK, so basically you can apply Green's theorem for a region that has several boundary curves.
You just have to be careful that the outer boundary must go counterclockwise.
The inner boundary either goes clockwise, or you put a minus sign.
OK, and the last cultural note, so, the definition, we say that a region in the plane, sorry, I should say a connected region in the plane, so that means -- So, connected means it consists of a single piece.
OK, so, connected, there is a single piece.
These two guys together are not connected.
But, if I join them, then this is a connected region.
We say it's simply connected -- -- if any closed curve in it, OK, so I need to gave a name to my region, let's say R, any closed curve in R, bounds, no, sorry.
If the interior of any closed curve in R -- -- is also contained in R. So, concretely, what does that mean?
That means the region, R, does not have any holes inside it.
Maybe I should draw two pictures to explain what I mean.
So, this guy here is simply connected while -- -- this guy here is not simply connected because if I take this curve, that's a curve inside my region.
But, the piece that it bounds is not actually entirely contained in my origin.
And, so why is that relevant?
Well, if you know that your vector field is defined everywhere in a simply connected region, then you don't have to worry about this question of, can I apply Green's theorem to the inside?
You know it's automatically OK because if you have a closed curve, then the vector field is, I mean, if a vector field is defined on the curve it will also be defined inside.
OK, so if the domain of definition -- -- of a vector field is defined and differentiable -- -- is simply connected -- -- then we can always apply -- -- Green's theorem -- -- and, of course, provided that we do it on a curve where the vector field is defined.
I mean, your line integral doesn't make sense so there's nothing to compute.
But, if you have, so, again, the argument would be, well, if a vector field is defined on the curve, it's also defined inside.
So, see, the problem with that vector field here is precisely that its domain of definition is not simply connected because there is a hole, namely the origin.
OK, so for this guy, domain of definition, which is plane minus the origin with the origin removed is not simply connected.
And so that's why you have this line integral that makes perfect sense, but you can't apply Green's theorem to it.
So now, what does that mean a particular?
Well, we've seen this criterion that if a curl of the vector field is zero and it's defined in the entire plane, then the vector field is conservative, and it's a gradient field.
And, the argument to prove that is basically to use Green's theorem.
So, in fact, the actual optimal statement you can make is if a vector field is defined in a simply connected region, and its curl is zero, then it's a gradient field.
So, let me just write that down.
So, the correct statement, I mean, the previous one we've seen is also correct.
But this one is somehow better and closer to what exactly is needed if curl F is zero and the domain of definition where F is defined is simply connected -- -- then F is conservative.
And that means also it's a gradient field.
It's the same thing.
OK, any questions on this?
No?
OK, some good news.
What I've just said here won't come up on the test on Thursday.
OK. (APPLAUSE) Still, it's stuff that you should be aware of generally speaking because it will be useful, say, on the next week's problem set.
And, maybe on the final it would be, there won't be any really, really complicated things probably, but you might need to be at least vaguely aware of this issue of things being simply connected.
And by the way, I mean, this is also somehow the starting point of topology, which is the branch of math that studies the shapes of regions.
So, in particular, you can try to distinguish domains in the plains by looking at whether they're simply connected or not, and what kinds of features they have in terms of how you can joint point what kinds of curves exist in them.
And, since that's the branch of math in which I work, I thought I should tell you a bit about it.
OK, so now back to reviewing for the exam.
So, I'm going to basically list topics.
And, if time permits, I will say a few things about problems from practice exam 3B.
I'm hoping that you have it or your neighbor has it, or you can somehow get it.
Anyway, given time, I'm not sure how much I will say about the problems in and of themselves.
OK, so the main thing to know about this exam is how to set up and evaluate double integrals and line integrals.
OK, if you know how to do these two things, then you are in much better shape than if you don't.
And -- So, the first thing we've seen, just to write it down, there's two main objects.
And, it's kind of important to not confuse them with each other.
OK, there's double integrals of our regions of some quantity, dA, and the other one is the line integral along a curve of a vector field, F.dr or F.Mds depending on whether it's work or flux that we are trying to do.
And, so we should know how to set up these things and how to evaluate them.
And, roughly speaking, in this one you start by drawing a picture of the region, then deciding which way you will integrate it.
It could be dx dy, dy dx, r dr d theta, and then you will set up the bound carefully by slicing it and studying how the bounds for the inner variable depend on the outer variable.
So, the first topic will be setting up double integrals.
And so, remember, OK, so maybe I should make this more explicit.
We want to draw a picture of R and take slices in the chosen way so that we get an iterated integral.
OK, so let's do just a quick example.
So, if I look at problem one on the exam 3B, it says to look at the line integral from zero to one, line integral from x to 2x of possibly something, but dy dx.
And it says, let's look at how we would set this up the other way around by exchanging x and y.
So, we should get to something that will be the same integral dx dy.
I mean, if you have a function of x and y, then it will be the same function.
But, of course, the bounds change.
So, how do we exchange the order of integration?
Well, the only way to do it consistently is to draw a picture.
So, let's see, what does this mean?
Here, it means we integrate from y equals x to y equals 2x, x between zero and one.
So, we should draw a picture.
The lower bound for y is y equals x.
So, let's draw y equals x.
That seems to be here.
And, we'll go up to y equals 2x, which is a line also but with bigger slope.
And then, all right, so for each value of x, my origin will go from x to 2x.
Well, and I do this for all values of x that go to x equals one.
So, I stop at x equals one, which is here.
And then, my region is something like this.
OK, so this point here, in case you are wondering, well, when x equals one, y is one.
And that point here is one, two.
OK, any questions about that so far?
OK, so somehow that's the first kill, when you see an integral, how to figure out what it means, how to draw the region.
And then there's a converse scale which is given the region, how to set up the integral for it.
So, if we want to set up instead dx dy, then it means we are going to actually look at the converse question which is, for a given value of y, what is the range of values of x?
OK, so if we fix y, well, where do we enter the region, and where do we leave it?
So, we seem to enter on this side, and we seem to leave on that side.
At least that seems to be true for the first few values of y that I choose.
But, hey, if I take a larger value of y, then I will enter on the side, and I will leave on this vertical side, not on that one.
So, I seem to have two different things going on.
OK, the place where enter my region is always y equals 2x, which is the same as x equals y over two.
So, x seems to always start at y over two.
But, where I leave to be either x equals y, or here, x equals y.
And, that depends on the value of y.
So, in fact, I have to break this into two different integrals.
I have to treat separately the case where y is between zero and one, and between one and two.
So, what I do in that case is I just make two integrals.
So, I say, both of them start at y over two.
But, in the first case, we'll stop at x equals y.
In the second case, we'll stop at x equals one.
OK, and now, what are the values of y for each case?
Well, the first case is when y is between zero and one.
The second case is when y is between one and two, which I guess this picture now is completely unreadable, but hopefully you've been following what's going on, or else you can see it in the solutions to the problem.
And, so that's our final answer.
OK, any questions about how to set up double integrals in xy coordinates?
No?
OK, who feels comfortable with this kind of problem?
OK, good.
I'm happy to see the vast majority.
So, the bad news is we have to be able to do it not only in xy coordinates, but also in polar coordinates.
So, when you go to polar coordinates, basically all you have to remember on the side of integrand is that x becomes r cosine theta.
Y becomes r sine theta.
And, dx dy becomes r dr d theta.
In terms of how you slice for your region, well, you will be integrating first over r. So, that means what you're doing is you're fixing the value of theta.
And, for that value of theta, you ask yourself, for what range of values of r am I going to be inside my origin?
So, if my origin looks like this, then for this value of theta, r would go from zero to whatever this distance is.
And of course I have to find how this distance depends on theta.
And then, I will find the extreme values of theta.
Now, of course, is the origin is really looking like this, then you're not going to do it in polar coordinates.
But, if it's like a circle or a half circle, or things like that, then even if a problem doesn't tell you to do it in polar coordinates you might want to seriously consider it.
OK, so I'm not going to do it but problem two in the practice exam is a good example of doing something in polar coordinates.
OK, so in terms of things that we do with double integrals, there's a few formulas that I'd like you to remember about applications that we've seen of double integrals.
So, quantities that we can compute with double integrals include things like the area of region, its mass if it has a density, the average value of some function, for example, the average value of the x and y coordinates, which we called the center of mass or moments of inertia.
So, these are just formulas to remember.
So, for example, the area of region is the double integral of just dA, or if it helps you, one dA if you want.
You are integrating the function 1.
You have to remember formulas for mass, for the average value of a function is the F bar, in particular x bar y bar, which is the center of mass, and the moment of inertia.
OK, so the polar moment of inertia, which is moment of inertia about the origin.
OK, so that's double integral of x squared plus y squared, density dA, but also moments of inertia about the x and y axis, which are given by just taking one of these guys.
Don't worry about moments of inertia about an arbitrary line.
I will ask you for a moment of inertia for some weird line or something like that.
OK, but these you should know.
Now, what if you somehow, on the spur of the moment, you forget, what's the formula for moment of inertia?
Well, I mean, I prefer if you know, but if you have a complete blank in your memory, there will still be partial credit were setting up the bounds and everything else.
So, the general rule for the exam will be if you're stuck in a calculation or you're missing a little piece of the puzzle, try to do as much as you can.
In particular, try to at least set up the bounds of the integral.
There will be partial credit for that always.
So, while we're at it about grand rules, how about evaluation?
How about evaluating integrals?
So, once you've set it up, you have to sometimes compute it.
First of all, check just in case the problem says set up but do not evaluate.
Then, don't waste your time evaluating it.
If a problem says to compute it, then you have to compute it.
So, what kinds of integration techniques do you need to know?
So, you need to know, you must know, well, how to integrate the usual functions like one over x or x to the n, or exponential, sine, cosine, things like that, OK, so the usual integrals.
You must know what I will call easy trigonometry.
OK, I don't want to give you a complete list.
And the more you ask me about which ones are on the list, the more I will add to the list.
But, those that you know that you should know, you should know.
Those that you think you shouldn't know, you don't have to know because I will say what I will say soon.
You should know also substitution, how to set U equals something, and then see, oh, this becomes u times du, and so substitution method.
What do I mean by easy trigonometrics?
Well, certainly you should know how to ingrate sine.
You should know how to integrate cosine.
You should be aware that sine squared plus cosine squared simplifies to one.
And, you should be aware of general things like that.
I would like you to know, maybe, the double angles, sine 2x and cosine 2x.
Know what these are, and the kinds of the easy things you can do with that, also things that involve substitution setting like U equals sine T or U equals cosine T. I mean, let me, instead, give an example of hard trig that you don't need to know, and then I will answer.
OK, so, not needed on Thursday; it doesn't mean that I don't want you to know them.
I would love you to know every single integral formula.
But, that shouldn't be your top priority.
So, you don't need to know things like hard trigonometric ones.
So, let me give you an example.
OK, so if I ask you to do this one, then actually I will give you maybe, you know, I will reprint the formula from the notes or something like that.
OK, so that one you don't need to know.
I would love if you happen to know it, but if you need it, it will be given to you.
So, these kinds of things that you cannot compute by any easy method.
And, integration by parts, I believe that I successfully test-solved all the problems without doing any single integration by parts.
Again, in general, it's something that I would like you to know, but it shouldn't be a top priority for this week.
OK, sorry, you had a question, or?
Inverse trigonometric functions: let's say the most easy ones.
I would like you to know the easiest inverse trig functions, but not much.
OK, OK, so be aware that these functions exist, but it's not a top priority.
I should say, the more I tell you I don't need you to know, the more your physics and other teachers might complain that, oh, these guys don't know how to integrate.
So, try not to forget everything.
But, yes?
No, no, here I just mean for evaluating just a single variable integral.
I will get to change variables and Jacobian soon, but I'm thinking of this as a different topic.
What I mean by this one is if I'm asking you to integrate, I don't know, what's a good example?
Zero to one t dt over square root of one plus t squared, then you should think of maybe substituting u equals one plus t squared, and then it becomes easier.
OK, so this kind of trig, that's what I have in mind here specifically.
And again, if you're stuck, in particular, if you hit this dreaded guy, and you don't actually have a formula giving you what it is, it means one of two things.
One is something's wrong with your solution.
The other option is something is wrong with my problem.
So, either way, check quickly what you've done it if you can't find a mistake, then just move ahead to the next problem.
Which one, this one?
Yeah, I mean if you can do it, if you know how to do it, which everything is fair: I mean, generally speaking, give enough of it so that you found the solution by yourself, not like, you know, it didn't somehow come to you by magic.
But, yeah, if you know how to integrate this without doing the substitution, that's absolutely fine by me.
Just show enough work.
The general rule is show enough work that we see that you knew what you are doing.
OK, now another thing we've seen with double integrals is how to do more complicated changes of variables.
So, when you want to replace x and y by some variables, u and v, given by some formulas in terms of x and y.
So, you need to remember basically how to do them.
So, you need to remember that the method consists of three steps.
So, one is you have to find the Jacobian.
And, you can choose to do either this Jacobian or the inverse one depending on what's easiest given what you're given.
You don't have to worry about solving for things the other way around.
Just compute one of these Jacobians.
And then, the rule is that du dv is absolute value of the Jacobian dx dy.
So, that takes care of dx dy, how to convert that into du dv.
The second thing to know is that, well, you need to of course substitute any x and y's in the integrand to convert them to u's and v's so that you have a valid integrand involving only u and v. And then, the last part is setting up the bounds.
And you see that, probably you seen on P-sets and an example we did in the lecture that this can be complicated.
But now, in real life, you do this actually to simplify the integrals.
So, probably the one that will be there on Thursday, if there's a problem about that on Thursday, it will be a situation where the bounds that you get after changing variables are reasonably easy.
OK, I'm not saying that it will be completely obvious necessarily, but it will be a fairly easy situation.
So, the general method is you look at your region, R, and it might have various sides.
Well, on each side you ask yourself, what do I know about x and y, and how to convert that in terms of u and v?
And maybe you'll find that the equation might be just u equals zero for example, or u equals v, or something like that.
And then, it's up to you to decide what you want to do.
But, maybe the easiest usually is to draw a new picture in terms of u and v coordinates of what your region will look like in the new coordinates.
It might be that it will actually much easier.
It should be easier looking than what you started with.
OK, so that's the general idea.
There is one change of variable problem on each of the two practice exams to give you a feeling for what's realistic.
The problem that's on practice exam 3B actually is on the hard side of things because the question is kind of hidden in a way.
So, if you look at problem six, you might find that it's not telling you very clearly what you have to do.
That's because it was meant to be the hardest problem on that test.
But, once you've reduced it to an actual change of variables problem, I expect you to be able to know how to do it.
And, on practice exam 3A, there's also, I think it's problem five on the other practice exam.
And, that one is actually pretty standard and straightforward.
OK, time to move on, sorry.
So, we've also seen about line integrals.
OK, so line integrals, so the main thing to know about them, so the line integral for work, which is line integral of F.dr, so let's say that your vector field has components, M and N. So, the line integral for work becomes in coordinates integral of Mdx plus Ndy while we've also seen line integral for flux.
So, line integral of F.n ds becomes the integral along C just to make sure that I give it to you correctly.
So, remember that just, I don't want to make the mistake in front of you.
So, T ds is dx, dy.
And, the normal vector, so, T ds goes along the curve.
Nds goes clockwise perpendicular to the curve.
So, it's going to be, well, it's going to be dy and negative dx.
So, you will be integrating negative Ndx plus Mdy.
OK, see, if you are blanking and don't remember the signs, then you can just draw this picture and make sure that you get it right.
So, you should know a little bit about geometric interpretation and how to see easily that it's going to be zero in some cases.
But, mostly you should know how to compute, set up and compute these things.
So, what do we do when we are here?
Well, it's year, we have both x and y together, but we want to, because it's the line integral, there should be only one variable.
So, the important thing to know is we want to reduce everything to a single parameter.
OK, so the evaluation method is always by reducing to a single parameter.
So, for example, maybe x and y are both functions of some variable, t, and then express everything in terms of some integral of, some quantity involving t dt.
It could be that you will just express everything in terms of x or in terms of y, or in terms of some angle or something.
It's up to you to choose how to parameterize things.
And then, when you're there, it's a usual one variable integral with a single variable in there.
OK, so that's the general method of calculation, but we've seen a shortcut for work when we can show that the field is the gradient of potential.
So, one thing to know is if the curl of F, which is an x minus My happens to be zero, well, and now I can say, and the domain is simply connected, or if the field is defined everywhere, then F is actually a gradient field.
So, that means, just to make it more concrete, that means we can find a function little f called the potential such that its derivative respect to x is M, and its derivative with respect to Y is N. We can solve these two conditions for the same function, f, simultaneously.
And, how do we find this function, little f?
OK, so that's the same as saying that the field, big F, is the gradient of little f. And, how do we find this function, little f?
Well, we've seen two methods.
One of them involves computing a line integral from the origin to a point in the plane by going first along the x axis, then vertically.
The other method was to first figure out what this one tells us by integrating it with respect to x.
And then, we differentiate our answer with respect to y, and we compare with that to get the complete answer.
OK, so I is that relevant?
Well, first of all it's relevant in physics, but it's also relevant just to calculation of line integrals because we see the fundamental theorem of calculus for line integrals which says if we are integrating a gradient field and we know what the potential is.
Then, we just have to, well, the line integral is just the change in value of a potential.
OK, so we take the value of a potential at the starting point, sorry, we take value potential at the endpoint minus the value at the starting point.
And, that will give us the line integral, OK?
So, important: this is only for work.
There's no statement like that for flux, OK, so don't tried to fly this in a problem about flux.
I mean, usually, if you look at the practice exams, you will see it's pretty clear that there's one problem in which you are supposed to do things this way.
It's kind of a dead giveaway, but it's probably not too bad.
OK, and the other thing we've seen, so I mentioned it at the beginning but let me mention it again.
To compute things, Green's theorem, let's just compute, well, let us forget, sorry, find the value of a line integral along the closed curve by reducing it to double integral.
So, the one for work says -- -- this, and you should remember that in there, so C is a closed curve that goes counterclockwise, and R is the region inside.
So, the way you would, if you had to compute both sides separately, you would do them in extremely different ways, right?
This one is a line integral.
So, you use the method to explain here, namely, you express x and y in terms of a single variable.
See that you're doing a circle.
I want to see a theta.
I don't want to see an R. R is not a variable.
You are on the circle.
This one is a double integral.
So, if you are doing it, say, on a disk, you would have both R and theta if you're using polar coordinates.
You would have both x and y.
Here, you have two variables of integration.
Here, you should have only one after you parameterize the curve.
And, the fact that it stays curl F, I mean, curl F is just Nx-My is just like any function of x and y. OK, the fact that we called it curl F doesn't change how you compute it.
You have first to compute the curl of F. Say you find, I don't know, xy minus x squared, well, it becomes just the usual double integral of the usual function xy minus x squared.
There's nothing special to it because it's a curl.
And, the other one is the counterpart for flux.
So, it says this, and remember this is mx plus ny.
I mean, what's important about these statements is not only remembering, you know, if you just know this formula by heart, you are still in trouble because you need to know what actually the symbols in here mean.
So, you should remember, what is this line integral, and what's the divergence of a field?
So, just something to remember.
And, so I guess I'll let you figure out practice problems because it's time, but I think that's basically the list of all we've seen.
And, well, that should be it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
All right, so the past few weeks, we've been looking at double integrals and the plane, line integrals in the plane, and will we are going to do now from now on basically until the end of the term, will be very similar stuff, but in space.
So, we are going to learn how to do triple integrals in space, flux in space, work in space, divergence, curl, all that.
So, that means, basically, if you were really on top of what we've been doing these past few weeks, then it will be just the same with one more coordinate.
And, you will see there are some differences.
But, conceptually, it's pretty similar.
There are a few tricky things, though.
Now, that also means that if there is stuff that you are not sure about in the plane, then I encourage you to review the material that we've done over the past few weeks to make sure that everything in the plane is completely clear to you because it will be much harder to understand stuff in space if things are still shaky in the plane.
OK, so the plan is we're going to basically go through the same stuff, but in space.
So, it shouldn't be surprising that we will start today with triple integrals.
OK, so the way triple integrals work is if I give you a function of three variables, x, y, z, and I give you some region in space, so, some solid, then I can take the integral over this region over function f dV where dV stands for the volume element.
OK, so what it means is we will just take every single little piece of our solid, take the value of f there, multiply by the small volume of each little piece, and sum all these things together.
And, so this volume element here, well, for example, if you are doing the integral in rectangular coordinates, that will become dx dy dz or any permutation of that because, of course, we have lots of possible orders of integration to choose from.
So, rather than bore you with theory and all sorts of complicated things, let's just do examples.
And, you will see, basically, if you understand how to set up iterated integrals into variables, that you basically understand how to do them in three variables.
You just have to be a bit more careful.
And, there's one more step.
OK, so let's take our first triple integral to be on the region.
So, of course, there's two different things as always.
There is the region of integration and there's the function we are integrating.
Now, the function we are integrating, well, it will come in handy when you actually try to evaluate the integral.
But, as you can see, probably, the new part is really hard to set it up.
So, the function won't really matter that much for me.
So, in the examples I'll do today, functions will be kind of silly.
So, for example, let's say that we want to look at the region between two paraboloids, one given by z = x ^2 y ^2.
The other is z = 4 - x ^2 - y ^2.
And, so, I haven't given you, yet, the function to integrate.
OK, this is not the function to integrate.
This is what describes the region where I will integrate my function.
And, let's say that I just want to find the volume of this region, which is the triple integral of just one dV.
OK, similarly, remember, when we try to find the area of the region in the plane, we are just integrating one dA.
Here we integrate one dV.
that will give us the volume.
Now, I know that you can imagine how to actually do this one as a double integral.
But, the goal of the game is to set up the triple integral.
It's not actually to find the volume.
So, what does that look like?
Well, z = x ^2 y ^2, that's one of our favorite paraboloids.
That's something that looks like a parabola with its bottom at the origin that you spin about the z axis.
And, z equals four minus x squared minus y squared, well, that's also a paraboloid.
But, this one is pointing down, and when you take x equals y equals zero, you get z equals four.
So, it starts at four, and it goes down like that.
OK, so the solid that we'd like to consider is what's in between in here.
So, it has a curvy top which is this downward paraboloid, a curvy bottom which is the other paraboloid.
And, what about the sides?
Well, do you have any idea what we get here?
Yeah, it's going to be a circle because entire picture is invariant by rotation about the z axis.
So, if you look at the picture just, say, in the yz plane, you get this point and that point.
And, when you rotate everything around the z axis, you will just get a circle here.
OK, so our goal is to find the volume of this thing, and there's lots of things I could do to simplify the calculation, or even not do it as a triple integral at all.
But, I want to actually set it up as a triple integral just to show how we do that.
OK, so the first thing we need to do is choose an order of integration.
And, here, well, I don't know if you can see it yet, but hopefully soon that will be intuitive to you.
I claim that I would like to start by integrating first over z.
What's the reason for that?
Well, the reason is if I give you x and y, then you can find quickly, what's the bottom and top values of z for that choice of x and y?
OK, so if I have x and y given, then I can find above that: what is the bottom z and the top z corresponding to the vertical line above that point?
The portion of it that's inside our solid, so somehow, there's a bottom z and a top z.
And, so the top z is actually on the downward paraboloid.
So, it's four minus x squared minus y squared.
The bottom value of z is x squared plus y squared.
OK, so if I want to start to set this up, I will write the triple integral.
And then, so let's say I'm going to do it dz first, and then, say, dy dx.
It doesn't really matter.
So then, for a given value of x and y, I claim z goes from the bottom surface.
The bottom face is z equals x squared plus y squared.
The top face is four minus x squared minus y squared.
OK, is that OK with everyone?
Yeah?
Any questions so far?
Yes?
Why did I start with z?
That's a very good question.
So, I can choose whatever order I want, but let's say I did x first .
Then, to find the inner integral bounds, I would need to say, OK, I've chosen values of, see, in the inner integral, you've fixed the two other variables, and you're just going to vary that one.
And, you need to find bounds for it.
So, if I integrate over x first, I have to solve, answer the following question.
Say I'm given values of y and z.
What are the bounds for x?
So, that would mean I'm slicing my solid by lines that are parallel to the x axis.
And, see, it's kind of hard to find, what are the values of x at the front and at the back?
I mean, it's possible, but it's easier to actually first look for z at the top and bottom.
Yes?
dy dx, or dx dy?
No, it's completely at random.
I mean, you can see x and y play symmetric roles.
So, if you look at it, it's reasonably clear that z should be the easiest one to set up first for what comes next.
xy or yx, it's the same.
Yes?
Yes, it will be easier to use cylindrical coordinates.
I'll get to that just as soon as I'm done with this one.
OK, so let's continue a bit with that.
And, as you mentioned, actually we don't actually want to do it with xy in the end.
In a few minutes, we will actually switch to cylindrical coordinates.
But, for now, we don't even know what they are.
OK, so I've done the inner integral by looking at, you know, if I slice by vertical lines, what is the top?
What is the bottom for a given value of x and y?
So, the bounds in the inner integral depend on both the middle and outer variables.
Next, I need to figure out what values of x and y I will be interested in.
And, the answer for that is, well, the values of x and y that I want to look at are all those that are in the shade of my region.
So, in fact, to set up the middle and outer bounds, what I want to do is project my solid.
So, my solid looks like this kind of thing.
And, I don't really know how to call it.
But, what's interesting now is I want to look at the shadow that it casts in the xy plane.
OK, and, of course, that shadow will just be the disk that's directly below this disk here that's separating the two halves of the solid.
And so, now I will want to integrate over, I want to look at all the xy's, x and y, in the shadow.
So, now I'm left with, actually, something we've already done, namely setting up a double integral over x and y.
So, if it helps, here, we don't strictly need it, but if it helps, it could be useful to actually draw a picture of this shadow in the xy plane.
So, here it would just look, again, like a disk, and set it up.
Now, the question is, how do we find the size of this disk, the size of the shadow?
Well, basically we have to figure out where our two paraboloids intersect.
There's nothing else.
OK, so, one way how to find the shadow in the xy plane -- -- well, here we actually know the answer a priori, but even if we didn't, we could just say, well, our region lives wherever the bottom surface is below the top surface, OK, so we want to look at things wherever bottom value of z is less than the top value of z, I mean, less or less than or equal, that's the same thing.
So, if the bottom value of z is x squared plus y squared should be less than four minus x squared minus y squared, and if you solve for that, then you will get, well, so let's move these guys over here.
You'll get two x squared plus two y squared less than four.
That becomes x squared plus y squared less than two.
So, that means that's a disk of radius square root of two, OK?
So, we kind of knew in advance it was going to be a disk, but what we've learned now is that this radius is square root of two.
So, if we want to set up, if we really want to set it up using dy dx like they started, then we can do it because we know, so, for the middle integral, now, we want to fix a value of x.
And, for that fixed value of x, we want to figure out the bounds for y.
Well, the answer is y goes from here to here.
What's here?
Well, here, y is square root of two minus x squared.
And, here it's negative square root of two minus x squared.
So, y will go from negative square root of two minus x squared to positive square root.
And then, x will go from negative root two to root two.
OK, if that's not completely clear to you, then I encourage you to go over how we set up double integrals again.
OK, does that make sense, kind of?
Yeah?
Well, so, when we set up, remember, we are setting up a double integral, dy dx here.
So, when we do it dy dx, it means we slice this region of a plane by vertical line segments.
So, this middle guy would be what used to be the inner integral.
So, in the inner, remember, you fix the value of x, and you ask yourself, what is the range of values of y in my region?
So, y goes from here to here, and what here and here are depends on the value of x.
How?
Well, we have to find the relation between x and y at these points.
These points are on the circle of radius root two.
So, if you want this circle maybe I should have written, is x squared plus y squared equals two.
And, if you solve for y, given x, you get plus minus root of two minus x squared, OK?
Yes?
Is there a way to compute this with symmetry?
Well, certainly, yeah, this solid looks sufficiently symmetric, but actually you could certainly, if you don't want to do the whole disk, you could just do quarter disks, and multiply by four.
You could even just look at the lower half of the solid, and multiply them by two, so, total by eight.
So, yeah, certainly there's lots of ways to make it slightly easier by using symmetry.
Now, the most spectacular way to use symmetry here, of course, is to use that we have this rotation symmetry and switch, actually, not do this guy in xy coordinates but instead in polar coordinates.
So -- So, the smarter thing to do would be to use polar coordinates instead of x and y.
Of course, we want to keep z. I mean, we are very happy with z the way it is.
But, we'll just change x and y to R cos theta, R sine theta, OK, because, well, let's see actually how we would evaluate this guy.
So, well actually, let's not.
It's kind of boring.
So, let me just point out one small thing here, sorry, before I do that.
So, if you start computing the inner integral, OK, so let me not do that yet, sorry, so if you try to compute the inner integral, you'll be integrating from x squared plus y squared to four minus x squared minus y squared dz.
Well, that will integrate to z between these two bounds.
So, you will get four minus two x squared minus two y squared.
Now, when you put that into the remaining ones, you'll get something that's probably not very pleasant of four minus two x squared minus two y squared dy dx.
And here, you see that to evaluate this, you would switch to polar coordinates.
Oh, by the way, so if your initial instincts had been to, given that you just want the volume, you could also have found the volume just by doing a double integral of the height between the top and bottom.
Well, you would just have gotten this, right, because this is the height between top and bottom.
So, it's all the same.
It doesn't really matter.
But with this, of course, we will be able to integrate all sorts of functions, not just one over the solid.
So, we will be able to do much more than just volumes.
OK, so let's see, how do we do it with polar coordinates instead?
Well, so -- Well, that would become, so let's see.
So, I want to keep dz.
But then, dx dy or dy dx would become r dr d theta.
And, if I try to set up the bounds, well, I probably shouldn't keep this x squared plus y squared around.
But, x squared plus y squared is easy in terms of r and theta.
That's just r squared.
OK, I mean, in general I could have something that depends also on theta.
That's perfectly legitimate.
But here, it simplifies, and this guy up here, four minus x squared minus y squared becomes four minus r squared.
And now, the integral that we have to do over r and theta, well, we look again at the shadow.
The shadow is still a disk of radius root two.
That hasn't changed.
And now, we know how to set up this integral in polar coordinates.
r goes from zero to root two, and theta goes from zero to two pi.
OK, and now it becomes actually easier to evaluate.
OK, so now we have actually a name for this because we're doing it in space.
So, these are called, actually, cylindrical coordinates.
So, in fact, you already knew about cylindrical coordinates even if you did not know the name.
OK, so the idea of cylindrical coordinates is that instead of x, y, and z, to locate a point in space, you will use three coordinates.
One of them is basically how high it is above the xy plane.
So, that will be z.
And then, you will use polar coordinates for the projection of your point on the xy plane.
So, r will be the distance from the z axis.
And theta will be the angle from the x axis counterclockwise.
So, the one thing to be careful about is because of the usual convention, that we make the x axis point toward us.
Theta equals zero is no longer to the right.
Now, theta equals zero is to the front, and the angel is measured from the front counterclockwise.
OK, so, and of course, if you want to know how to convert between x, y, z and r theta z, well, the formulas are just the same as in usual polar coordinates.
R cos theta, r sine theta, and z remain z. OK, so why are these called cylindrical coordinates, by the way?
Well, let's say that I gave you the equation r equals a, where a is some constant.
Say r equals one, for example.
So, r equals one in 2D, that used to be just a circle of radius one.
Now, in space, a single equation actually defines a surface, not just a curve anymore.
And, the set of points where r is a, well, that's all the points that are distance a from the z axis.
So, in fact, what you get this way is a cylinder of radius a centered on the z axis.
OK, so that's why they are called cylindrical coordinates.
By the way, so now, similarly, if you look at the equation theta equals some given value, well, so that used to be just a ray from the origin.
Now, that becomes a vertical half plane.
For example, if I set the value of theta and let r and z vary, well, r is always positive, but basically that means I am taking a vertical plane that comes out in this direction.
OK, any questions about cylindrical coordinates?
Yes?
Yeah, so I'm saying when you fix theta, you get only a half plane, not a full plane.
I mean, it goes all the way up and down, but it doesn't go back to the other side of the z axis.
Why?
That's because r is always positive by convention.
So, for example, here, we say theta is zero.
At the back, we say theta is pi.
We don't say theta is zero and r is negative.
We say r is positive and theta is pi.
It's a convention, largely.
But, sticking with this convention really will help you to set up the integrals properly.
I mean, otherwise there is just too much risk for mistakes.
Yes?
Well, so the question is if I were to use symmetry to do this one, would I multiply by four or by two?
Well, it depends on how much symmetry you are using.
So, I mean, it's your choice.
You can multiply by two, by four, by eight depending on how much you cut it.
So, it depends on what symmetry you use, if you use symmetry between top and bottom you'd say, well, the volume is twice the lower half.
If you use the left and right half, you would say it's twice each half.
If you cut it into four pieces, and so on.
So, and again, you don't have to use the symmetry.
If you don't think of using polar coordinates, then it can save you from doing, you know, you can just start at zero here and here, and simplify things a tiny bit.
But, OK, yes?
So, to define a vertical full plane, well, first of all it depends on whether it passes through the z axis or not.
If it doesn't, then you'd have to remember how you do in polar coordinates.
I mean, basically the answer is, if you have a vertical plane, so, it doesn't depend on z.
The equation does not involve z.
It only involves r and theta.
And, how it involves r and theta is exactly the same as when you do a line in polar coordinates in the plane.
So, if it's a line passing through the origin, you say, well, theta is either some value or the other one.
If it's a line that doesn't passes to the origin, but it's more tricky.
But hopefully you've seen how to do that.
OK, let's move on a bit.
So, one thing to know, I mean, basically, the important thing to remember is that the volume element in cylindrical coordinates, well, dx dy dz becomes r dr d theta dz.
And, that shouldn't be surprising because that's just dx dy becomes r dr d theta.
And, dz remains dz.
I mean, so, the way to think about it, if you want, is that if you take a little piece of solid in space, so it has some height, delta z, and it has a base which has some area delta A, then the small volume, delta v, is equal to the area of a base times the height.
So, now, when you make the things infinitely small, you will get dV is dA times dz, and you can use whichever formula you want for area in the xy plane.
OK, now in practice, you choose which order you integrate in.
As you have probably seen, a favorite of mine is z first because very often you'll know what the top and bottom of your solid look like, and then you will reduce to just something in the xy plane.
But, there might be situations where it's actually easier to start first with dx dy or r dr d theta, and then save dz for last.
I mean, if you seen how to, in single variable calculus, the disk and shell methods for finding volumes, that's exactly the dilemma of shells versus disks.
One of them is you do z first.
The other is you do z last.
OK, so what are things we can do now with triple integrals?
Well, we can find the volume of solids by just integrating dV.
And, we've seen that.
We can find the mass of a solid.
OK, so if we have a density, delta, which, remember, delta is basically the mass divided by the volume.
OK, so the small mass element, maybe I should have written that as dm, the mass element, is density times dV.
So now, this is the real physical density.
If you are given a material, usually, the density will be in grams per cubic meter or cubic inch, or whatever.
I mean, there is tons of different units.
But, so then, the mass of your solid will be just the triple integral of density, dV because you just sum the mass of each little piece.
And, of course, if the density is one, then it just becomes the volume.
OK, now, it shouldn't be surprising to you that we can also do classics that we had seen in the plane such as the average value of a function, the center of mass, and moment of inertia.
OK, so the average value of the function f of x, y, z in the region, r, that would be f bar, would be one over the volume of the region times the triple integral of f dV.
Or, if we have a density, and we want to take a weighted average -- Then we take one over the mass where the mass is the triple integral of the density times the triple integral of f density dV.
So, as particular cases, there is, again, the notion of center of mass of the solid.
So, that's the point that somehow right in the middle of the solid.
That's the point mass by which there is a point at which you should put point mass so that it would be equivalent from the point of view of dealing with forces and translation effects, of course, not for rotation.
But, so the center of mass of a solid is just given by taking the average values of x, y, and z. OK, so there is a special case where, so, x bar is one over the mass times triple integral of x density dV.
And, same thing with y and z.
And, of course, very often, you can use symmetry to not have to compute all three of them.
For example, if you look at this solid that we had, well, I guess I've erased it now.
But, if you remember what it looked, well, it was pretty obvious that the center of mass would be in the z axis.
So, no need to waste time considering x bar and y bar.
And, in fact, you can also find z bar by symmetry between the top and bottom, and let you figure that out.
Of course, symmetry only works, I should say, symmetry only works if the density is also symmetric.
If I had taken my guy to be heavier at the front than at the back, then it would no longer be true that x bar would be zero.
OK, next on the list is moment of inertia.
Actually, in a way, moment of inertia in 3D is easier conceptually than in 2D.
So, why is that?
Well, because now the various flavors that we had come together in a nice way.
So, the moment of inertia of an axis, sorry, with respect to an axis would be, again, given by the triple integral of the distance to the axis squared times density, times dV.
And, in particular, we have our solid.
And, we might skewer it using any of the coordinate axes and then try to rotate it about one of the axes.
So, we have three different possibilities, of course, the x, y, or z axis.
And, so now, rotating about the z axis actually corresponds to when we were just doing things for flat objects in the xy plane.
That corresponded to rotating about the origin.
So, secretly, we were saying we were rotating about the point.
But actually, it was just rotating about the z axis.
Just I didn't want to introduce the z coordinate that we didn't actually need at the time.
So -- [APPLAUSE] OK, so moment of inertia about the z axis, so, what's the distance to the z axis?
Well, we've said that's exactly r. That's the cylindrical coordinate, r. So, the square of a distance is just r squared.
Now, if you didn't want to do it in cylindrical coordinates then, of course, r squared is just x squared plus y squared.
Square of distance from the z axis is just x squared plus y squared.
Similarly, now, if you want the distance from the x axis, well, that will be y squared plus z squared.
OK, try to convince yourselves of the picture, or else just argue by symmetry: you know, if you change the positions of the axis.
So, moment of inertia about the x axis is the double integral of y squared plus z squared delta dV.
And moment of inertia about the y axis is the same thing, but now with x squared plus z squared.
And so, now, if you try to apply these things for flat solids that are in the xy plane, so where there's no z to look at, well, you see these formulas become the old formulas that we had.
But now, they all fit together in a more symmetric way.
OK, any questions about that?
No?
OK, so these are just formulas to remember.
So, OK, let's do an example.
Was there a question that I missed?
No?
OK, so let's find the moment of inertia about the z axis of a solid cone -- -- between z equals a times r and z equals b.
So, just to convince you that it's a cone, so, z equals a times r means the height is proportional to the distance from the z axis.
So, let's look at what we get if we just do it in the plane of a blackboard.
So, if I go to the right here, r is just the distance from the x axis.
The height should be proportional with proportionality factor A.
So, that means I take a line with slope A.
If I'm on the left, well, it's the same story except distance to the z axis is still positive.
So, I get the symmetric thing.
And, in fact, it doesn't matter which vertical plane I do it in.
This is the same if I rotate about.
See, there's no theta in here.
So, it's the same in all directions.
So, I claim it's a cone where the slope of the rays is A. OK, and z equals b.
Well, that just means we stop in our horizontal plane at height b. OK, so that's solid cone really just looks like this.
That's our solid.
OK, so it has a flat top, that circular top, and then the point is at v. The tip of it is at the origin.
So, let's try to compute its moment of inertia about the z axis.
So, that means maybe this is like the top that you are going to spin.
And, it tells you how hard it is to actually spin that top.
Actually, that's also useful if you're going to do mechanical engineering because if you are trying to design gears, and things like that that will rotate, you might want to know exactly how much effort you'll have to put to actually get them to spin, and whether you're actually going to have a strong enough engine, or whatever, to do it.
OK, so what's the moment of inertia of this guy?
Well, that's the triple integral of, well, we have to choose x squared plus y squared or r squared.
Let's see, I think I want to use cylindrical coordinates to do that, given the shape.
So, we use r squared.
I might have a density that let's say the density is one.
So, I don't have density.
I still have dV.
Now, it will be my choice to choose between doing the dz first or doing r dr d theta first.
Just to show you how it goes the other way around, let me do it r dr d theta dz this time.
Then you can decide on a case-by-case basis which one you like best.
OK, so if we do it in this direction, it means that in the inner and middle integrals, we've fixed a value of z.
And, for that particular value of z, we'll be actually slicing our solid by a horizontal plane, and looking at what we get, OK?
So, what does that look like?
Well, I fixed a value of z, and I slice my solid by a horizontal plane.
Well, I'm going to get a circle certainly.
What's the radius, well, a disk actually, what's the radius of the disk?
Yeah, the radius of the disk should be z over a because the equation of that cone, we said it's z equals ar.
So, if you flip it around, so, maybe I should switch to another blackboard.
So, the equation of a cone is z equals ar, or equivalently r equals z over a.
So, for a given value of z, I will get, this guy will be a disk of radius z over a. OK, so, moment of inertia is going to be, well, we said r squared, r dr d theta dz.
Now, so, to set up the inner and middle integrals, I just set up a double integral over this disk of radius z over a.
So, it's easy.
r goes from zero to z over a. Theta goes from zero to 2pi.
OK, and then, well, if I set up the bounds for z, now it's my outer variable.
So, the question I have to ask is what is the first slice?
What is the last slice?
So, the bottommost value of z would be zero, and the topmost would be b.
And so, that's it I get.
So, exercise, it's not very hard.
Try to set it up the other way around with dz first and then r dr d theta.
It's pretty much the same level of difficulty.
I'm sure you can do both of them.
So, and also, if you want to practice calculations, you should end up getting pi b to the five over 10a to the four if I got it right.
OK, let me finish with one more example.
I'm trying to give you plenty of practice because in case you haven't noticed, Monday is a holiday.
So, you don't have recitation on Monday, which is good.
But it means that there will be lots of stuff to cover on Wednesday.
So -- Thank you.
OK, so third example, let's say that I want to just set up a triple integral for the region where z is bigger than one minus y inside the unit ball centered at the origin.
So, the unit ball is just, you know, well, stay inside of the unit sphere.
So, its equation, if you want, would be x squared plus y squared plus z squared less than one.
OK, so that's one thing you should remember.
The equation of a sphere centered at the origin is x squared plus y squared plus z squared equals radius squared.
And now, we are going to take this plane, z equals one minus y.
So, if you think about it, it's parallel to the x axis because there's no x in its coordinate in its equation.
At the origin, the height is one.
So, it starts right here at one.
And, it slopes down with y with slope one.
OK, so it's a plane that comes straight out here, and it intersects the sphere, so here and here, but also at other points in between.
Any idea what kind of shape this is?
Well, it's an ellipse, but it's even more than that.
It's also a circle.
If you slice a sphere by a plane, you always get a circle.
But, of course, it's a slanted circle.
So, if you look at it in the xy plane, if you project it to the xy plane, that you will get an ellipse.
OK, so we want to look at this guy in here.
So, how do we do that?
Well, so maybe I should actually draw quickly a picture.
So, in the yz plane, it looks just like this, OK?
But, if I look at it from above in the xy plane, then its shadow, well, see, it will sit entirely where y is positive.
So, it sits entirely above here, and it goes through here and here.
And, in fact, when you project that slanted circle, now you will get an ellipse.
And, well, I don't really know how to draw it well, but it should be something like this.
OK, so now if you want to try to set up that double integral, sorry, the triple integral, well, so let's say we do it in rectangular coordinates because we are really evil.
[LAUGHTER] So then, the bottom surface, OK, so we do it with z first.
So, the bottom surface is the slanted plane.
So, the bottom value would be z equals one minus y.
The top value is on the sphere.
So, the sphere corresponds to z equals square root of one minus x squared minus y squared.
So, you'd go from the plane to the sphere.
And then, to find the bounds for x and y, you have to figure out what exactly, what the heck is this region here?
So, what is this region?
Well, we have to figure out, for what values of x and y the plane is below the ellipse.
So, the condition is that, sorry, the plane is below the sphere.
OK, so, that's when the plane is below the sphere.
That means one minus y is less than square root of one minus x squared minus y squared.
So, you have to somehow manipulate this to extract something simpler.
Well, probably the only way to do it is to square both sides, one minus y squared should be less than one minus x squared minus y squared.
And, if you work hard enough, you'll find quite an ugly equation.
But, you can figure out what are, then, the bounds for x given y, and then set up the integral?
So, just to give you a hint, the bounds on y will be zero to one.
The bounds on x, well, I'm not sure you want to see them, but in case you do, it will be from negative square root of 2y minus 2y squared to square root of 2y minus 2y squared.
So, exercise, figure out how I got these by starting from that.
Now, of course, if we just wanted the volume of this guy, we wouldn't do it this way.
We do symmetry, and actually we'd rotate the thing so that our spherical cap was actually centered on the z axis because that would be a much easier way to set it up.
But, depending on what function we are integrating, we can't always do that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So, last week we learned how to do triple integrals in rectangular and cylindrical coordinates.
And, now we have to learn about spherical coordinates, which you will see are a lot of fun.
So, what's the idea of spherical coordinates?
Well, you're going to represent a point in space using the distance to the origin and two angles.
So, in a way, you can think of these as a space analog of polar coordinates because you just use distance to the origin, and then you have to use angles to determine in which direction you're going.
So, somehow they are more polar than cylindrical coordinates.
So, how do we do that?
So, let's say that you have a point in space at coordinates x, y, z.
Then, instead of using x, y, z, you will use, well, one thing you'll use is the distance from the origin.
OK, and that is denoted by the Greek letter which looks like a curly p, but actually it's the Greek R. So -- That's the distance from the origin.
And so, that can take values anywhere between zero and infinity.
Then, we have to use two other angles.
And, so for that, let me actually draw the vertical half plane that contains our point starting from the z axis.
OK, so then we have two new angles.
Well, one of them is not really new.
One is new.
That's phi is the angle downwards from the z axis.
And the other one, theta, is the angle counterclockwise from the x axis.
OK, so phi, let me do it better.
So, there's two ways to draw the letter phi, by the way.
And, I recommend this one because it doesn't look like a rho.
So, that's easier.
That's the angle that you have to go down from the positive z axis.
And, so that angle varies from zero when you're on the z axis, increase to pi over two when you are on the xy plane all the way to pi or 180� when you are on the negative z axis.
It doesn't go beyond that.
OK, so -- Phi is always between zero and pi.
And, finally, the last one, theta, is just going to be the same as before.
So, it's the angle after you project to the xy plane.
That's the angle counterclockwise from the x axis.
OK, so that's a little bit overwhelming not just because of the new letters, but also because there is a lot of angles in there.
So, let me just try to, you know, suggest two things that might help you a little bit.
So, one is, these are called spherical coordinates because if you fix the value of rho, then you are moving on a sphere centered at the origin.
OK, so let's look at what happens on a sphere centered at the origin, so, with equation rho equals a.
Well, then phi measures how far south you are going, measures the distance from the North Pole.
So, if you've learned about latitude and longitude in geography, well, phi and theta you can think of as latitude and longitude except with slightly different conventions.
OK, so, phi is more or less the same thing as latitude in the sense that it measures how far north or south you are.
The only difference is in geography, latitude is zero on the equator and becomes something north, something south, depending on how far you go from the equator.
Here, you measure a latitude starting from the North Pole which is zero, increasing all the way to the South Pole, which is at pi.
And, theta or you can think of as longitude, which measures how far you are east or west.
So, the Greenwich Meridian would be here, now, the one on the x axis.
That's the one you use as the origin for longitude, OK?
Now, if you don't like geography, here's another way to think about it.
So -- Let's start again from cylindrical coordinates, which hopefully you're kind of comfortable with now.
OK, so you know about cylindrical coordinates where we have the z coordinates stay z, and the xy plane we do R and theta polar coordinates.
And now, let's think about what happens when you look at just one of these vertical planes containing the z axis.
So, you have the z axis, and then you have the direction away from the z axis, which I will call r, just because that's what r measures.
Of course, r goes all around the z axis, but I'm just doing a slice through one of these vertical half planes, fixing the value of theta.
Then, r of course is a polar coordinate seen from the point of view of the xy plane.
But here, it looks more like you have rectangular coordinates again.
So the idea of spherical coordinate is you're going to polar coordinates again in the rz plane.
OK, so if I have a point here, then rho will be the distance from the origin.
And phi will be the angle, except it's measured from the positive z axis, not from the horizontal axis.
But, the idea in here, see, let me put that between quotes because I'm not sure how correct that is, but in a way, you can think of this as polar coordinates in the rz plane.
So, in particular, that's the key to understanding how to switch between spherical coordinates and cylindrical coordinates, and then all the way to x, y, z if you want, right, because this picture here tells us how to express z and r in terms of rho and phi.
So, let's see how that works.
If I project here or here, so, this line is z.
But, it's also rho times cosine phi.
So, I get z equals rho cos phi.
And, if I look at r, it's the same thing, but on the other side.
So, r will be rho sine phi.
OK, so you can use this to switch back and forth between spherical and cylindrical.
And of course, if you remember what x and y were in terms of r and theta, you can also keep doing this to figure out, oops.
So, x is r cos theta.
That becomes rho sine phi cos theta.
Y is r sine theta.
So, that becomes rho sine phi sine theta.
And z is rho cos phi.
But, basically you don't really need to remember these formulas as long as you remember how to express r in terms of rho sine phi, and x equals r cos theta.
So, now, of course, we're going to use spherical coordinates in situations where we have a lot of symmetry, and in particular, where the z axis plays a special role.
Actually, that's the same with cylindrical coordinates.
Cylindrical and secure coordinates are set up so that the z axis plays a special role.
So, that means whenever you have a geometric problem, and you are not told how to choose your coordinates, it's probably wiser to try to center things on the z axis.
That's where these coordinates are the best adapted.
And, in case you ever need to switch backwards, I just want to point out, so, rho is the square root of r squared plus z squared, which means it's the square root of x squared plus y squared plus z squared.
OK, so that's basically all the formulas about spherical coordinates.
OK, any questions about that?
OK, let's see, who had seen spherical coordinates before just to see?
OK, that's not very many.
So, I'm sure for, one of you saw it twice.
That's great.
Sorry, oops, OK, so let's just look quickly at equations of some of the things.
So, as I've said, if I set rho equals a, that will be just a sphere of radius a centered at the origin.
More interesting things: let's say I give you phi equals pi over four.
What do you think that looks like?
Actually, let's take a quick poll on things.
OK, yeah, everyone seems to be saying it's a cone, and that's indeed the correct answer.
So, how do we see that?
Well, remember, phi is the angle downward from the z axis.
So, let's say that I'm going to look first at what happens if I'm in the right half of a plane of a blackboard, so, in the yz plane.
Then, phi is the angle downward from here.
So, if I want to get pi over four, that's 45�.
That means I'm going to go diagonally like this.
Of course, if I'm in the left half of a plane of a blackboard, it's going to be the same.
I also take pi over four.
And, I get the other half.
And, because the equation does not involve theta, it's all the same if I rotate my vertical plane around the z axis.
So, I get the same picture in any of these vertical half planes, actually.
OK, now, so this is phi equals pi over four.
And, just in case, to point out to you what's going on, when phi equals pi over four, cosine and sine are equal to each other.
They are both one over root two.
So, you can find, again, the equation of this thing in cylindrical coordinates, which I'll remind you was z equals r. OK, in general, phi equals some given number, or z equals some number times r. That will be a cone centered on the z axis.
OK, a special case: what if I say phi equals pi over two?
Yeah, it's just going to be the xy plane.
OK, that's the flattest of all cones.
OK, so phi equals pi over two is going to be just the xy plane.
And, in general, if phi is less than pi over two, then you are in the upper half space.
If phi is more than pi over two, you'll be in the lower half space.
OK, so that's pretty much all we need to know at this point.
So, what's next?
Well, remember we were trying to do triple integrals.
So now we're going to triple integrals in spherical coordinates.
And, for that, we first need to understand what the volume element is.
What will be dV?
OK, so dV will be something, d rho, d phi, d theta, or in any order that you want.
But, this one is usually the most convenient.
So, to find out what it is, well, we should look at how we are going to be slicing things now.
OK, so if you integrate d rho, d phi, d theta, it means that you are actually slicing your solid into little pieces that live, somehow, if you set an interval of rows, OK, sorry, maybe I should, so, if you first integrate over rho, it means that you will actually choose first the direction from the origin even by phi and theta.
And, in that direction, you will try to figure out, how far does your region extend?
And, of course, how far that goes might depend on phi and theta.
Then, you will vary phi.
So, you have to know, for a given value of theta, how far down does your solid extend?
And, finally, the value of theta will correspond to, in which directions around the z axis do we go?
So, we're going to see that in examples.
But before we can do that, we need to get the volume element.
So, what I would like to suggest is that we need to figure out, what is the volume of a small piece of solid which corresponds to a certain change, delta rho, delta phi, and delta theta?
So, delta rho means that you have two concentric spheres, and you are looking at a very thin shell in between them.
And then, you would be looking at a piece of that spherical shell corresponding to small values of phi and theta.
So, because I am stretching the limits of my ability to draw on the board, here's a picture.
I'm going to try to reproduce on the board, but so let's start by looking just at what happens on the sphere of radius a, and let's try to figure out the surface area elements on the sphere in terms of phi and theta.
And then, we'll add the rho direction.
OK, so -- So, let me say, let's start by understanding surface area on a sphere of radius a.
So, that means we'll be looking at a little piece of the sphere corresponding to angles delta phi and in that direction here delta theta.
OK, so when you draw a map of the world on a globe, that's exactly what the grid lines form for you.
So, what's the area of this guy?
Well, of course, all the sides are curvy.
They are all on the sphere.
None of them are straight.
But still, if it's small enough and it looks like a rectangle, so let's just try to figure out, what are the sides of your rectangle?
OK, so, let's see, well, I think I need to draw a bigger picture of this guy.
OK, so this guy, so that's a piece of what's called a parallel in geography.
That's a circle that goes east-west.
So now, this parallel as a circle of radius, well, the radius is less than a because if your vertical is to the North Pole, it will be actually much smaller.
So, that's why when you say you're going around the world it depends on whether you do it at the equator or the North Pole.
It's much easier at the North Pole.
So, anyway, this is a piece of a circle of radius, well, the radius is what I would call r because that's the distance from the z axis.
OK, that's actually pretty hard to see now.
So if you can see it better on this one, then so this guy here, this length is r. And, r is just rho, well, what was a times sine phi.
Remember, we have this angle phi in here.
I should use some color.
It's getting very cluttered.
So, we have this phi, and so r is going to be rho sine phi.
That rho is a.
So, let me just put a sine phi.
OK, and the corresponding angle is going to be measured by theta.
So, the length of this is going to be a sine phi delta theta.
That's for this side.
Now, what about that side, the north-south side?
Well, if you're moving north-south, it's not like east-west.
You always have to go all the way from the North Pole to the South Pole.
So, that's actually a great circle meridian of length, well, I mean, well, the radius is the radius of the sphere.
Total length is 2pi a.
So, this is a piece of a circle of radius a.
And so, now, the length of this one is going to be a delta phi.
OK, so, just to recap, this is a sine phi delta theta.
And, this guy here is a delta phi.
So, you can't read it because it's -- And so, that tells us if I take a small piece of the sphere, then its surface area, delta s, is going to be approximately a sine phi delta theta times a delta phi, which I'm going to rewrite as a squared sine phi delta phi delta theta.
So, what that means is, say that I want to integrate something just on the surface of a sphere.
Well, I would use phi and theta as my coordinates.
And then, to know how big a piece of a sphere is, I would just take a squared sine phi d phi d theta.
OK, so that's the surface element in a sphere.
And now, what about going back into the third dimension, so, adding some depth to these things?
Well, I'm not going to try to draw a picture because you've seen that's slightly tricky.
Well, let me try anyway just you can have fun with my completely unreadable diagrams.
So anyway, if you look at, now, something that's a bit like that piece of sphere, but with some thickness to it.
The thickness will be delta rho, and so the volume will be roughly the area of the thing on the sphere times the thickness.
So, I claim that we will get basically the volume element just by multiplying things by d rho.
So, let's see that.
So now, if I have a sphere of radius rho, and another one that's slightly bigger of radius rho plus delta rho, and then I have a little box in here.
Then, I know that the volume of this thing will be essentially, well, its thickness, the thickness is going to be delta rho times the area of its base, although it doesn't really matter, which is what we've called delta s. OK, so we will get, sorry, a becomes rho now.
Square sine phi, delta rho, delta phi, delta theta, and so out of that we get the volume element and spherical coordinates, which is rho squared sine phi d rho, d phi, d theta.
And, that's a formula that you should remember.
OK, so whenever we integrate a function, and we decide to switch to spherical coordinates, then dx dy dz or r dr d theta dz will become rho squared sine phi d rho d phi d theta.
OK, any questions on that?
No?
OK, so let's -- Let's see how that works.
So, as an example, remember at the end of the last lecture, I tried to set up an example where we were looking at a sphere sliced by a slanted plane.
And now, we're going to try to find the volume of that spherical cap again, but using spherical coordinates instead.
So, I'm going to just be smarter than last time.
So, last time, we had set up these things with a slanted plane that was cutting things diagonally.
And, if I just want to find the volume of this cap, then maybe it makes more sense to rotate things so that my plane is actually horizontal, and things are going to be centered on the z axis.
So, in case you see that it's the same, then that's great.
If not, then it doesn't really matter.
You can just think of this as a new example.
So, I'm going to try to find the volume of a portion of the unit sphere -- -- that lies above the horizontal plane, z equals one over root two.
OK, one over root two was the distance from the origin to our slanted plane.
So, after you rotate, that say you get this value.
Anyway, it's not very important.
You can just treat that as a good example if you want.
OK, so we can compute this in actually pretty much any coordinate system.
And also, of course, we can set up not only the volume, but we can try to find the moment of inertia about the central axis, or all sorts of things.
But, we are just doing the volume for simplicity.
So, actually, this would go pretty well in cylindrical coordinates.
But let's do it in spherical coordinates because that's the topic of today.
A good exercise: do it in cylindrical and see if you get the same thing.
So, how do we do that?
Well, we have to figure out how to set up our triple integral in spherical coordinates.
So, remember we'll be integrating one dV.
So, dV will become rho squared sign phi d rho d phi d theta.
And, now as we start, we're already facing some serious problem.
We want to set up the bounds for rho for a given, phi and theta.
So, that means we choose latitude/longitude.
We choose which direction we want to aim for, you know, which point of the sphere we want to aim at.
And, we are going to shoot a ray from the origin towards this point, and we want to know what portion of the ray is in our solid.
So -- We are going to choose a value of phi and theta.
And, we are going to try to figure out what part of our ray is inside this side.
So, what should be clear is at which point we leave the solid, right?
What's the value of rho here?
It's just one.
The sphere is rho equals one.
That's pretty good.
The question is, where do we enter the region?
So, we enter the region when we go through this plane.
And, the plane is z equals one over root two.
So, what does that tell us about rho?
Well, it tells us, so remember, z is rho cosine phi.
So, the plane is z equals one over root two.
That means rho cosine phi is one over root two.
That means rho equals one over root two cosine phi or, as some of you know it, one over root two times second phi.
OK, so if we want to set up the bounds, then we'll start with one over root two second phi all the way to one.
Now, what's next?
Well, so we've done, I think that's basically the hardest part of the job.
Next, we have to figure out, what's the range for phi?
So, the range for phi, well, we have to figure out how far to the north and to the south our region goes.
Well, the lower bound for phi is pretty easy, right, because we go all the way to the North Pole direction.
So, phi starts at zero.
The question is, where does it stop?
To find out where it stops, we have to figure out, what is the value of phi when we hit the edge of the region?
OK, so maybe you see it.
Maybe you don't.
One way to do it geometrically is to just, it's always great to draw a slice of your region.
So, if you slice these things by a vertical plane, or actually even better, a vertical half plane, something to delete one half of the picture.
So, I'm going to draw these r and z directions as before.
So, my sphere is here.
My plane is here at one over root two.
And, my solid is here.
So now, the question is what is the value of phi when I'm going to stop hitting the region?
And, if you try to figure out first what is this direction here, that's also one over root two.
And so, this is actually 45�, also known as pi over four.
The other way to think about it is at this point, well, rho is equal to one because you are on the sphere.
But, you are also on the plane.
So, rho cos phi is one over root two.
So, if you plug rho equals one into here, you get cos phi equals one over root two which gives you phi equals pi over four.
That's the other way to do it.
You can do it either by calculation or by looking at the picture.
OK, so either way, we've decided that phi goes from zero to pi over four.
So, this is pi over four.
Finally, what about theta?
Well, because we go all around the z axis we are going to go just zero to 2pi.
OK, any questions about these bounds?
OK, so note how the equation of this horizontal plane in spherical coordinates has become a little bit weird.
But, if you remember how we do things, say that you have a line in polar coordinates, and that line does not pass through the origin, then you also end up with something like that.
You get something like r equals a second theta or a cos second theta for horizontal or vertical lines.
And so, it's not surprising you should get this.
That's in line with the idea that we are just doing again, polar coordinates in the rz directions.
So of course, in general, things can be very messy.
But, generally speaking, the kinds of regions that we will be setting up things for are no more complicated or no less complicated than what we would do in the plane in polar coordinates.
OK, so there's, you know, a small list of things that you should know how to set up.
But, you won't have some really, really strange thing.
Yes?
D rho?
Oh, you mean the bounds for rho?
Yes.
So, in the inner integral, we are going to fix values of phi and theta.
So, that means we fix in advance the direction in which we are going to shoot a ray from the origin.
So now, as we shoot this ray, we are going to hit our region somewhere.
And, we are going to exit, again, somewhere else.
OK, so first of all we have to figure out where we enter, where we leave.
Well, we enter when the ray hits the flat face, when we hit the plane.
And, we would leave when we hit the sphere.
So, the lower bound will be given by the plane.
The upper bound will be given by the sphere.
So now, you have to get spherical coordinate equations for both the plane and the sphere.
For the sphere, that's easy.
That's rho equals one.
For the plane, you start with z equals one over root two.
And, you switch it into spherical coordinates.
And then, you solve for rho.
And, that's how you get these bounds.
Is that OK?
All right, so that's the setup part.
And, of course, the evaluation part goes as usual.
And, since I'm running short of time, I'm not going to actually do the evaluation.
I'm going to let you figure out how it goes.
Let me just say in case you want to check your answers, so, at the end you get 2pi over three minus 5pi over six root two.
Yes, it looks quite complicated.
That's basically because you get one over, well, you get a second square when you integrate C. When you integrate rho squared, you will get rho cubed over three.
But that rho cubed will give you a second cube for the lower bound.
And, when you integrate sine phi second cubed phi, you do a substitution.
You see that integrates to one over second squared with a factor in front.
So, in the second square, when you plug in, no, that's not quite all of it.
Yeah, well, the second square is one thing, and also the other bound you get sine phi which integrates to cosine phi.
So, anyways, you get lots of things.
OK, enough about it.
So, next, I have to tell you about applications.
And, of course, well, there's the same applications that we've seen that last time, finding volumes, finding masses, finding average values of functions.
In particular, now, we could say to find the average distance of a point in this solid to the origin.
Well, spherical coordinates become appealing because the function you are averaging is just rho while in other coordinate systems it's a more complicated function.
So, if you are asked to find the average distance from the origin, spherical coordinates can be interesting.
Also, well, there's moments of inertia, preferably the one about the z axis because if you have to integrate something that involves x or y, then your integrand will contain that awful rho sine phi sine theta or rho sine phi cosine theta, and then it won't be much fun to evaluate.
So, that anyway, there's the usual ones.
And then there's a new one.
So, in physics, you've probably seen things about gravitational attraction.
If not, well, it's what causes apples to fall and other things like that as well.
So, anyway, physics tells you that if you have two masses, then they attract each other with a force that's directed towards each other.
And in intensity, it's proportional to the two masses, and inversely proportional to the square of the distance between them.
So, if you have a given solid with a certain mass distribution, and you want to know how it attracts something else that you will put nearby, then you actually have to, the first approximation will be to say, well, let's just put a point mass at its center of mass.
But, if you're solid is actually not homogenous, or has a weird shape, then that's not actually the exact answer.
So, in general, you would have to just take every single piece of your object and figure out how it attracts you, and then compute the sum of these.
So, for example, if you want to understand why anything that you drop in this room will fall down, you have to understand that Boston is actually attracting it towards Boston.
And, Somerville's attracting it towards Somerville, and lots of things like that.
And, China, which is much further on the other side is going to attract towards China.
But, there's a lot of stuff on the other side of the Earth.
And so, overall, it's supposed to end up just going down.
OK, so now, how to find this out, well, you have to just integrate over the entire Earth.
OK, so let's try to see how that goes.
So, the setup that's going to be easiest for us to do computations is going to be that we are going to be the test mass that's going to be falling.
And, we are going to put ourselves at the origin.
And, the solid that's going to attract us is going to be wherever we want in space.
You'll see, putting yourself at the origin is going to be better.
Well, you have to put something at the origin.
And, the one that will stay a point mass, I mean, in my case not really a point, but anyway, let's say that I'm a point.
And then, I have a solid attracting me.
Well, so then if I take a small piece of it with the mass delta M, then that portion of the solid exerts a force on me, which is going to be directed towards it, and we'll have intensity.
So, the gravitational force -- -- exerted by the mass delta M at the point of x, y, z in space on a mass at the origin.
Well, we know how to express that.
Physics tells us that the magnitude of this force is going to be, well, G is just a constant.
It's the gravitational constant, and its value depends on which unit system you use.
Usually it's pretty small, times the mass delta M, times the test mass little m, divided by the square of the distance.
And, the distance from U to that thing is conveniently called rho since we've been introducing spherical coordinates.
So, that's the size, that's the magnitude of the force.
We also need to know the direction of the force.
And, the direction is going to be towards that point.
So, the direction of the force is going to be that of x, y, z.
But if I want a unit vector, then I should scale this down to length one.
So, let me divide this by rho to get a unit vector.
So, that means that the force I'm getting from this guy is actually going to be G delta M m over rho cubed times x, y, z. I'm just multiplying the magnitude by the unit vector in the correct direction.
OK, so now if I have not just that little p is delta M, but an entire solid, then I have to sum all these guys together.
And, I will get the vector that gives me the total force exerted, OK?
So, of course, there's actually three different calculations in one because you have to sum the x components to get the x components of a total force.
Same with the y, and same with the z.
So, let me first write down the actual formula.
So, if you integrate over the entire solid, oh, and I have to remind you, well, what's the mass, delta M of a small piece of volume delta V?
Well, it's the density times the volume.
So, the mass is going to be, sorry, density is delta.
There is a lot of Greek letters there, times the volume element.
So, you will get that the force is the triple integral over your solid of G m x, y, z over rho cubed, delta dV.
Now, two observations about that.
So, the first one, well, of course, these are just constants.
So, they can go out.
The second observation, so here, we are integrating a vector quantity.
So, what does that mean?
I just mean the x component of a force is given by integrating G m x over rho cubed delta dV.
The y components, same thing with y.
The z components, same thing with z. OK, there's no, like, you know, just integrate component by component to get each component of the force.
So, now we could very well to this in rectangular coordinates if we want.
But the annoying thing is this rho cubed.
Rho cubed is going to be x squared plus y squared plus z squared to the three halves.
That's not going to be a very pleasant thing to integrate.
So, it's much better to set up these integrals in spherical coordinates.
And, if we're going to do it in spherical coordinates, then probably we don't want to bother too much with x and y components because those would be unpleasant.
It would give us rho sine phi cos theta or sine theta.
So, the actual way we will set up things, set things up, is to place the solid so that the z axis is an axis of symmetry.
And, of course, that only works if the solid has some axis of symmetry.
Like, if you're trying to find the gravitational attraction of the Pyramid of Giza, then you won't be able to set up so that it has rotation symmetry.
Well, that's a tough fact of life, and you have to actually do it in x, y, z coordinates.
But, if at all possible, then you're going to place things.
Well, I guess even then, you could center it on the z axis.
But anyway, so you're going to mostly place things so that your solid is actually centered on the z-axis.
And, what you gain by that is that by symmetry, the gravitational force will be directed along the z axis.
So, you will just have to figure out the z component.
So, then the force will be actually, you know in advance that it will be given by zero, zero, and some z component.
And then, you just need to compute that component.
And, that component will be just G times m times triple integral of z over rho cubed delta dV.
OK, so that's the first simplification we can try to do.
The second thing is, well, we have to choose our favorite coordinate system to do this.
But, I claim that actually spherical coordinates are the best -- -- because let's see what happens.
So, G times mass times triple integral, well, a z in spherical coordinates becomes rho cosine phi over rho cubed.
Density, well, we can't do anything about density.
And then, dV becomes rho squared sine phi d rho d phi d theta.
Well, so, what happens with that?
Well, you see that you have a rho, a rho squared, and a rho cubed that cancel each other.
So, in fact, it simplifies quite a bit if you do it in spherical coordinates.
OK, so the z component of the force, sorry, I'm putting a z here to remind you it's the z component.
That is not a partial derivative, OK?
Don't get things mixed up, just the z component of the force becomes Gm triple integral of delta cos phi sine phi d rho d phi d theta.
And, so this thing is not dV, of course.
dV is much bigger, but we've somehow canceled out most of dV with stuff that was in the integrand.
And see, that's actually suddenly much less scary.
OK, so just to give you an example of what you can prove it this way, you can prove Newton's theorem, which says the following thing.
It says the gravitational attraction -- -- of a spherical planet, I should say with uniform density, or actually it's enough for the density to depend just on distance to the center.
But we just simplify the statement is equal to that of a point mass -- -- with the same total mass at its center.
OK, so what that means is that, so the way we would set it up is u would be sitting here and your planet would be over here.
Or, if you're at the surface of it, then of course you just put it tangent to the xy plane here.
And, you would compute that quantity.
Computation is a little bit annoying if a sphere is sitting up there because, of course, you have to find bounds, and that's not going to be very pleasant.
The case that we actually know how to do fairly well is if you are just at the surface of the planet.
But then, what the theorem says is that the force that you're going to feel is exactly the same as if you removed all of the planet and you just put an equivalent point mass here.
So, if the earth collapsed to a black hole at the center of the earth with the same mass, well, you wouldn't notice the difference immediately, or, rather, you would, but at least not in terms of your weight.
OK, that's the end for today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
We are going to continue to look at stuff in space.
We have been working with triple integrals and seeing how to set them up in all sorts of coordinate systems.
And the next topic we will be looking at are vector fields in space.
And so, in particular, we will be learning about flux and work.
So, just for a change, we will be starting with flux first.
And we will do work, actually, after Thanksgiving.
Just to remind you, a vector field in space is just the same thing as in the plane.
At every point you have a vector, and the components of this vector depend on the coordinates x, y and z.
Let's say the components might be P, Q, R, or your favorite three letters, where each of these things is a function of coordinates x, y, z.
You have seen that in the plane it is already pretty hard to draw a vector field.
Usually, in space, we won't really try too hard.
But it is still useful to try to have a general idea for what the vectors in there are doing, whether they are all going in the same direction, whether they may be all vertical or horizontal, pointing away from the origin, towards it, things like that.
But, generally-speaking, we won't really bother with trying to draw a picture because that is going to be quite hard.
Just to give you examples, well, the same kinds of examples as the plane, you can think of force fields.
For example, the gravitational attraction -- -- of a solid mass, let's call this mass big M, at the origin on a mass M at point x, y, z.
That would be given by a vector field that points toward the origin and whose magnitude is inversely proportional to the square of a distance from the origin.
Such a field would be directed towards the origin and its magnitude would be of the order of a constant over pho squared where pho is the distance from the origin.
The picture, if I really wanted to draw a picture, would be everywhere it is a field that points towards the origin.
And if I am further away then it gets smaller.
And, of course, I am not going to try to draw all these vectors in there.
If I wanted to give a formula for that -- A formula for that might be something of a form minus c times x, y, z over pho cubed.
Let's see.
Well, the direction of this vector, this vector is proportional to negative x, y, z. is the vector that goes from the origin to your point.
The negative goes towards the origin.
Then the magnitude of this guy, well, the magnitude of x, y, z is just the distance from the origin rho.
So the magnitude of this thing is one over rho cubed times some constant factor.
That would be an example of a vector field that comes up in physics.
Well, other examples would be electric fields.
Actually, if you look at the electric field generated by a charged particle at the origin, it is given by exactly the same kind of formula, and there are magnetic fields and so on.
Another example comes from velocity fields.
If you have a fluid flow, for example, if you want to study wind patterns in the atmosphere.
Well, wind, most of the time, is kind of horizontal, but maybe it depends on the altitude.
At high altitude you have jet streams, and the wind velocity is not the same at all altitudes.
And, just to give you more examples, in math we have seen that the gradient of a function of three variables gives you a vector field.
If you have a function u of x, y, z then its gradient field has just components, u sub x, u sub y and u sub z.
And, of course, the cases are not mutually exclusive.
For example, the electric field or gravitational field is given by the gradient of the gravitational or electric potential.
So, these are not like different cases.
There is overlap.
Anyway, hopefully, you are kind of convinced that you should learn about vector fields.
What are we going to do with them?
Well, let's start with flux.
Remember not so long ago we looked at flux of a two-dimensional field of a curve.
We had a curve in the plane and we had a vector field.
And we looked at the component of a vector field in the direction that was normal to the curve.
We formed the flux integral that was a line integral F dot n ds.
And that measured how much the vector field was going across the curve.
If you were thinking of a velocity field, that would measure how much fluid is passing through the curve in unit time.
Now let's say that we were in space.
Well, we cannot really think of flux as a line integral.
Because, if you have a curve in space and say that you have wind or something like that, you cannot really ask how much air is flowing through the curve.
See, to have a flow through something you need a surface.
If you have a net maybe then you can ask how much stuff is passing through that surface.
There is going to be a big difference here.
In the three-dimensional space, flux will be measured through a surface.
And so it will be a surface integral, not a line integral anymore.
That means we will be integrating, we will be summing over all the pieces of a surface in space.
Because a surface is a two-dimensional object, that will end up being a double integral.
But, of course, we will have to set it up properly because the surface that is in space, and we will probably have x, y and z to deal with at the same time, and we will have to somehow get rid of one variable so that we can set up and evaluate a double integral.
So conceptually it is very similar to line integrals.
In the line integral in the plane, you had two variables that you reduced to one by figuring out what the curve was.
Here you have three variables that you will reduce to two by figuring out what the surface is.
Let me give you a definition of flux in 3D.
Let's say that we have a vector field and s, a surface in space.
Let me draw some kind of a picture.
I have my surface and I have my vector field F. Well, at every point it changes with a point.
Well, I want to figure out how much my vector field is going across that surface.
That means I want to figure out the normal component of my vector field, so I will use, as in the plane case, the unit normal vector to s. I take my point on the surface and build a unit vector that is standing on it perpendicularly.
Now, we have to decide which way it is standing.
We can build our normal vector to go this way or to go the other way around.
There are two choices.
Basically, whenever you want to set up a flux integral you have to choose one side of the surface.
And you will count positively what flows toward that side and negatively what flows towards the other side.
There are two choices for n. We need to choose a side of the surface.
In the case of curves, we made that choice by deciding that because we were going along some direction on the curve we could choose one side by saying let's rotate clockwise from the tangent vector.
And, in a way, what we were doing was really it was a recipe to choose for us one of the two sides.
Here we don't have a notion of orienting the surface other than by precisely choosing one of the two possible normal vectors.
So, in fact, this is called choosing an orientation of a surface.
When you are saying you are orienting the surface that really means you are deciding which side is which.
Let's call that orientation.
Now, there is no set convention that will work forever.
But the usually traditional settings would be to take your normal vector pointing maybe out of the solid region because then you will be looking at flux that is coming out of that region of space.
Or, if you have a surface that is not like closed or anything but maybe you will want the flux going up through the region.
Or, there are various conventions.
Concretely, on problem sets it will either say which choice you have to make or you get to choose which one you want to make.
And, of course, if you choose the other one then the sign becomes the opposite.
Now, once we have made a choice then we can define the flux integral.
It will just be the double integral over a surface of F dot n dS.
Now I am using a big dS.
That stands for the surface area element on this surface.
I am using dS rather than dA because I still want to think of dA as maybe the area in one of the coordinate planes like the one we had in double integrals.
You will see later where this comes in.
But conceptually it is very similar.
Concretely what this means is I cut my surface into little pieces.
Each of them has area delta S. And, for each piece, I take my vector field, I take my normal vector, I dot them and I multiply by this surface area and sum all these things together.
That is what a double integral means.
In particular, an easy case where you know you can get away without computing anything is, of course, if your vector field is tangent to the surface because then you know that there is no flux.
Flux is going to be zero because nothing passes through the surface.
Otherwise, we have to figure out how to compute these things.
That is what we are going to learn now.
Well, maybe I should box this formula.
I have noticed that some of you seem to like it when I box the important formulas.
(APPLAUSE) By the way, a piece of notation before I move on, sometimes you will also see the notation vector dS.
What is vector dS?
Vector dS is this guy n dS put together.
Vector dS is a vector which points perpendicular to the surface and whose length corresponds to the surface element.
And the reason for having this shortcut notation, well, it is not only laziness like saving one n, but it is because this guy is very often easier to compute than it is to set up n and dS separately.
Actually, if you remember in the plane, we have seen that vector n little ds can be written directly as dy, - dx.
That was easier than finding n and ds separately.
And here the same is going to be true in many cases.
Well, any questions before we do examples?
No.
OK. Let's do examples.
The first example for today is we are going to look at the flux of vector field xi yj xk through the sphere of radius a -- -- centered at the origin.
What does the picture look like?
We have a sphere of radius a. I have my vector field.
Well, , see, that is a vector field that is equal to the vector from the origin to the point where I am, so it is pointing radially away from the origin.
My vector field is really sticking out everywhere away from the origin.
Now I have to find the normal vector to the sphere if I want to set up double integral over the sphere of F dot vector ds, or if you want F dot n dS.
What does the normal vector to the sphere look like?
Well, it depends, of course, whether I choose it pointing out or in.
Let's say I am choosing it pointing out then it will be sticking straight out of a sphere as well.
Hopefully, you can see that if I take a normal vector to the sphere it is actually pointing radially out away from the origin.
In fact, our vector field and our normal vector are parallel to each other.
Let's think a bit more about what a normal vector looks like.
I said it is sticking straight out.
It is proportional to this vector field.
Maybe I should start by writing because that is the vector that goes from the origin to my point so it points radially away from the origin.
Now there is a small problem with that.
It is not a unit vector.
So what is its length?
Well, its length is square root of x^2 y^2 z^2.
But, if I am on the sphere, then that length is just equal to a because distance from the origin is a.
In fact, I get my normal vector by scaling this guy down by a factor of a.
And let me write it down just in case you are still unsure.
This is unit because square root of x^2 y^2 z^2 is equal to a on the sphere.
OK. Any questions about this?
No.
It looks OK?
I see a lot of blank faces.
That physics test must have been hard.
Yes?
I could have put a rho but I want to emphasize the fact that here it is going to be a constant.
I mean rho has this connotation of being a variable that I will need to then maybe integrate over or do something with.
Yes, it would be correct to put rho but I then later will want to replace it by its actual value which is a number.
And the number is a.
It is not going to actually change from point to point.
For example, if this was the unit sphere then I would just put x, y, z. I wouldn't divide by anything.
Now let's figure out F dot n. Let's do things one at a time.
Well, F and n are parallel to each other.
F dot n, the normal component of F, is actually equal to the length of F. Well, times the length of n if you want, but that is going to be a one since F and n are parallel to each other.
And what is the magnitude of F if I am on the sphere?
Well, the magnitude of F in general is square root of x^2 y^2 z^2 on the sphere that is going be a.
The other way to do it, if you don't want to think geometrically like that, is to just to do the dot product x, y, z doted with x over a, y over a, z over a.
You will be x^2 y^2 z^2 divided by a.
That will simplify to a because we are on the sphere.
See, we are already using here the relation between x, y and z.
We are not letting x, y and z be completely arbitrary.
But the slogan is everything happens on the surface where we are doing the integral.
We are not looking at anything inside or outside.
We are just on the surface.
Now what do I do with that?
Well, I have turned my integral into the double integral of a dS.
And a is just a constant, so I am very lucky here.
I can just say this will be a times the double integral of dS.
And, of course, some day I will have to learn how to tackle that beast, but for now I don't actually need to because the double integral of dS just means I am summing the area of each little piece of the sphere.
I am just going to get the total area of the sphere which I know to be 4pi a2.
This guy here is going to be the area of S. I know that to be 4pi a^2.
So I will get 4pi a^3.
That one was relatively painless.
That was too easy.
Let's do a second example with the same sphere.
But now my vector field is going to be just z times k. Well, let me give it a different name.
Let me call it H instead of f or something like that just so that it is not called F anymore.
Well, the initial part of the setup is still the same.
The normal vector is still the same.
What changes is, of course, my vector field is no longer sticking straight out so I cannot use this easy geometric argument.
It looks like I will have to compute F dot n and then figure out how to integrate that with dS.
Let's do that.
We still have that n is <x, y, z>/a.
That tells us that H dot n will be dot <x, y, z> / a.
It looks like I will be left with z^2 over a. H dot n is z^2 over a.
The double integral for flux now becomes double integral on the sphere of z^2 over a dS.
Well, we can take out one over a, that is fine, but it looks like we will have to integrate z^2 on the surface of the sphere.
How do we do that?
Well, we have to figure out what is dS in terms of our favorite set of two variables that we will use to integrate.
Now, what is the best way to figure out where you are on the sphere?
Well, you could try to use maybe theta and z.
If you know how high you are and where you are around, in principle you know where you are on the sphere.
But since spherical coordinates we have actually learned about something much more interesting, namely spherical coordinates.
It looks like longitude / latitude is the way to go when trying to figure out where you are on a sphere.
We are going to use phi and theta.
And, of course, we have to figure out how to express dS in terms of d phi and d theta.
Well, if you were paying really, really close attention last time, you will notice that we have actually already done that.
Last time we saw that if I have a sphere of radius a and I take a little piece of it that corresponds to small changes in phi and theta then we said that -- Well, we argued that this side here, the one that is going east-west was a piece of the circle that has a radius a sin phi because that is r, so that side is a sin phi delta theta.
And the side that goes north-south is a piece of the circle of radius a corresponding to angle delta phi, so it is a delta phi.
And so, just to get to the answer, we got dS equals a^2 sin phi d phi d theta.
When we set up a surface integral on the surface of a sphere, most likely we will be using phi and theta as our two variables of integration and dS will become this.
Now, it is OK to think of them as spherical coordinates, but I would like to encourage you not to think of them as spherical coordinates.
Spherical coordinates are a way of describing points in space in terms of three variables.
Here it is more like we are parameterizing the sphere.
We are finding a parametric equation for the sphere using two variables phi and theta which happen to be part of the spherical coordinate system.
But, see, there is no rho involved in here.
I am not using any rho ever, and I am not going to in this calculation.
I have two variable phi and theta.
That is it.
It is basically in the same way as when you parameterize a line integral in the circle, we use theta as the parameter variable and never think about r. That being said, well, we are going to use phi and theta.
We know what dS is.
We still need to figure out what z is.
There we want to think a tiny bit about spherical coordinates again.
And we will know that z is just a cos phi.
In case you don't quite see it, let me draw a diagram.
Phi is the angle down from the positive z axes, this distance is a, so this distance here is a cos phi.
Now I have everything I need to compute my double integral.
z^2 over a dS will become a double integral.
z^2 becomes a^2 cos^2 phi over a times, ds becomes, a^2 sin phi d phi d theta.
Now I need to set up bounds.
Well, what are the bounds?
Phi goes all the way from zero to pi because we go all the way from the north pole to the south pole, and theta goes from zero to 2pi.
And, of course, I can get rid of some a's in there and take them out.
Let's look at what number we get.
First of all, we can take out all those a's and get a^3.
Second, in the inner integral, we are integrating cos^2 phi sin phi d phi.
I claim that integrates to cos3 up to some factor, and that factor should be negative one-third.
If you look at cos3 phi and you take its derivative, you will get that guy with a negative three in front between zero and pi.
And, while integrating over theta, we will just multiply things by 2pi.
Let me add the 2pi in front.
Now, if I evaluate this guy between zero and pi, well, at pi cos^3 is negative one, at zero it is one, I will get two-thirds out of this.
I end up with four-thirds pi a^3.
Sorry I didn't write very much because I am trying to save blackboard space.
Yes?
That is a very natural question.
That looks a lot like somebody we know, like the volume of a sphere.
And ultimately it will be.
Wait until next class when we talk about the divergence theorem.
I mean the question was is this related to the volume of a sphere, and ultimately it is, but for now it is just some coincidence.
Yes?
The question is there is a way to do it M dx plus N dy plus stuff like that?
The answer is unfortunately no because it is not a line integral.
It is a surface integral, so we need to have to variables in there.
In a way you would end up with things like some dx dy maybe and so on.
I mean it is not practical to do it directly that way because you would have then to compute Jacobians to switch from dx dy to something else.
We are going to see various ways of computing it.
Unfortunately, it is not quite as simple as with line integrals.
But it is not much harder.
It is the same spirit.
We just use two variables and set up everything in terms of these two variables.
Any other questions?
No.
OK. By the way, just some food for thought.
Never mind.
Conclusion of looking at these two examples is that sometimes we can use geometric.
The first example, we didn't actually have to compute an integral.
But most of the time we need to learn how to set up double integrals.
Use geometry or you need to set up for double integral of a surface.
And so we are going to learn how to do that in general.
As I said, we need to have two parameters on the surface and express everything in terms of these.
Let's look at various examples.
We are going to see various situations where we can do things.
Well, let's start with an easy one.
Let's call that number zero.
Say that my surface S is a horizontal plane, say z equals a.
When I say a horizontal plane, it doesn't have to be the entire horizontal plane.
It could be a small piece of it.
It could even be, to trick you, maybe an ellipse in there or a triangle in there or something like that.
What you have to recognize is my surface is a piece of just a flat plane, so I shouldn't worry too much about what part of a plane it is.
Well, it will become important when I set up bounds for integration.
But, when it comes to looking for the normal vector, be rest assured that the normal vector to a horizontal plane is just vertical.
It is going to be either k or negative k depending on whether I have chosen to orient it pointing up or down.
And which one I choose might depend on what I am going to try to do.
The normal vector is just sticking straight up or straight down.
Now, what about dS?
Well, it is just going to be the area element in a horizontal plane.
It just looks like it should be dx dy.
I mean if I am moving on a horizontal plane, to know where I am, I should know x and y.
So dS will be dx dy.
If I play the game that way, I have my vector field F. I do F dot n. That just gives me the z component which might involve x, y and z. x and y I am very happy with.
They will stay as my variables.
Whenever I see z, well, I want to get rid of it.
That is easy because z is just equal to a. I just plug that value and I am left with only x and y, and I am integrating that dx dy.
It is actually ending up being just a usual double integral in x, y coordinates.
And, of course, once it is set up anything is fair game.
I might want to switch to polar coordinates or something like that.
Or, I can set it up dx dy or dy dx.
All the usual stuff applies.
But, for the initial setup, we are just going to use these and express everything in terms of x and y.
A small variation on that.
Let's say that we take vertical planes that are parallel to maybe the blackboard plane, so parallel to the yz plane.
That might be something like x equals some constant.
Well, what would I do then?
It could be pretty much the same.
The normal vector for this guy would be sticking straight out towards me or away from me.
Let's say I am having it come to the front.
The normal vector would be plus/minus i.
And the variables that I would be using, to find out my position on this guy, would be y and z.
In terms of those, the surface element is just dy dz.
Similarly for planes parallel to the xz plane.
You can figure that one out.
These are somehow the easiest ones, because those we already know how to compute without too much trouble.
What if it is a more complicated plane?
We will come back to that next time.
Let's explore some other situations first.
Number one on the list.
Let's say that I gave you a sphere of radius a centered at the origin, or maybe just half of that sphere or some portion of it.
Well, we have already seen how to do things.
Namely, we will be saying the normal vector is x, y, z over a, plus or minus depending on whether we want it pointing in or out.
And dS will be a^2 sin phi d phi d theta.
In fact, we will express everything in terms of phi and theta.
If I wanted to I could tell you what the formulas are for x, y, z in terms of phi and theta.
You know them.
But it is actually better to wait a little bit.
It is better to do F dot n, because F is also going to have a bunch of x's, y's and z's.
And if there is any kind of symmetry to the problem then you might end up with things like x^2 y^2 z^2 or things that have more symmetry that are easier to express in terms of phi and theta.
The advice would be first do the dot product with F, and then see what you get and then turn it into phi and theta.
That is one we have seen.
Let's say that I have -- It is a close cousin.
Let's say I have a cylinder of radius a centered on the z-axis.
What does that look like?
And, again, when I say cylinder, it could be a piece of cylinder.
First of all, what does the normal vector to a cylinder look like?
Well, it is sticking straight out, but sticking straight out in a slightly different way from what happens with a sphere.
See, the sides of a cylinder are vertical.
If you imagine that you have this big cylindrical type in front of you, hopefully you can see that a normal vector is going to always be horizontal.
It is sticking straight out in the horizontal directions.
It doesn't have any z component.
I claim the normal vector for the cylinder, if you have a point here at (x, y, z), it would be pointing straight out away from the central axis.
My normal vector, well, if I am taking it two points outwards, will be going straight away from the central axis.
If I look at it from above, maybe it is easier if I look at it from above, look at x, y, then my cylinder looks like a circle and the normal vector just points straight out.
It is the same situation as when we had a circle in the 2D case.
The normal vector for that is just going to be x, y and 0 in the z component.
Well, plus/minus, depending on whether you want it sticking in or out.
We said in our cylinder normal vector is plus or minus x, y, zero over a.
What about the surface element?
Before we ask that, maybe we should first figure out what coordinates are we going to use to locate ourselves in a cylinder.
Well, yes, we probably want to use part of a cylindrical coordinate, except for, well, we don't want r because r doesn't change, it is not a variable here.
Indeed, you probably want to use z to tell how high you are and theta to tell you where you are around.
dS should be in terms of dz d theta.
Now, what is the constant?
Well, let's look at a small piece of our cylinder corresponding to a small angle delta theta and a small height delta z.
Well, the height, as I said, is going to be delta z.
What about the width?
It is going to be a piece of a circle of radius a corresponding to the angle delta theta, so this side will be a delta theta.
Delta S is a delta theta delta z. DS is just a dz d theta or d theta dz.
It doesn't matter which way you do it.
And so when we set up the flux integral, we will take first the dot product of f with this normal vector.
Then we will stick in this dS.
And then, of course, we will get rid of any x and y that are left by expressing them in terms of theta.
Maybe x becomes a cos theta, y becomes a sin theta.
These various formulas, you should try to remember them because they are really useful, for the sphere, for the cylinder.
And, hopefully, those for the planes you kind of know already intuitively.
What about marginals or faces?
Not everything in life is made out of cylinders and spheres.
I mean it is a good try.
Let's look at a marginal kind of surface.
Let's say I give you a graph of a function z equals f of x, y.
This guy has nothing to do with the integrand.
It is not what we are integrating.
We are just integrating a vector field that has nothing to do with that.
This is how I want to describe the surface on which I will be integrating.
My surface is given by z as a function of x, y.
Well, I would need to tell you what n is and what dS is.
That is going to be slightly annoying.
I mean, I don't want to tell them separately because you see they are pretty hard.
Instead, I am going to tell you that in this case, well, let's see.
What variables do we want?
I am going to tell you a formula for n dS.
What variables do we want to express this in terms of?
Well, most likely x and y because we know how to express z in terms of x and y.
This is an invitation to get rid of any z that might be left and set everything up in terms of dx dy.
The formula that we are going to see, I think we are going to see the details of why it works tomorrow, is that you can take negative partial f partial x, negative partial f partial y, one, dx dy.
Plus/minus depending on which way you want it to go.
If you really want to know what dS is, well, dS is the magnitude of this vector times dx dy.
There will be a square root and some squares and some stuff.
What is the normal vector?
Well, you take this vector and you scale it down to unit length.
Just to emphasize it, this guy here is not n and this guy here is not dS.
Each of them is more complicated than that, but the combination somehow simplifies nicely.
And that is good news for us.
Now, concretely, one way you can think about it is this tells you how to reduce things to an integral of x and y.
And, of course, you will have to figure out what are the bounds on x and y.
That means you will need to know what does the shadow of your surface look like in the x, y plane.
To set up bounds on whatever you will get dx dy, well, of course you can switch to dy dx or anything you would like, but you will need to look at the shadow of S in the x y plane.
But only do that after you gotten rid of all the z.
When you no longer have z then you can figure out what the bounds are for x and y.
Any questions about that?
Yes?
For the cylinder.
OK. Let me re-explain quickly how I got a normal vector for the cylinder.
If you know what a cylinder looks like, you probably can see that the normal vector sticks straight out of it horizontally.
That means the z component of n is going to be zero.
And then the x, y components you get by looking at it from above.
One last thing I want to say.
What about the geometric interpretation and how to prove it?
Well, if your vector field F is a velocity field then the flux is the amount of matter that crosses the surface that passes through S per unit time.
And the way that you would prove it would be similar to the picture that I drew when we did it in the plane.
Namely, you would consider a small element of a surface delta S. And you would try to figure out what is the stuff that flows through it in a second.
Well, it is the stuff that lives in a small box whose base is that piece of surface and whose other side is given by the vector field.
And then the volume of that is given by base times height, and the height is F dot n. It is the same argument as what we saw in the plane.
OK. Next time we will see more formulas.
We will first see how to prove this, more ways to do it, more examples.
And then we will get to the divergence theorem.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Yesterday we learned about flux and we have seen the first few examples of how to set up and compute integrals for a flux of a vector field for a surface.
Remember the flux of a vector field F through the surface S is defined by taking the double integral on the surface of F dot n dS where n is the unit normal to the surface and dS is the area element on the surface.
As we have seen, for various surfaces, we have various formulas telling us what the normal vector is and what the area element becomes.
For example, on spheres we typically integrate with respect to phi and theta for latitude and longitude angles.
On a horizontal plane, we would just end up degrading dx, dy and so on.
At the end of lecture we saw a formula.
A lot of you asked me how we got it.
Well, we didn't get it yet.
We are going to try to explain where it comes from and why it works.
The case we want to look at is if S is the graph of a function, it is given by z equals some function in terms of x and y.
Our surface is out here.
Z is a function of x and y.
And x and y will range over some domain in the x, y plane, namely the region that is the shadow of the surface on the x, y plane.
I said that we will have a formula for n dS which will end up being plus/minus minus f sub x, minus f sub y, one dxdy, so that we will set up and evaluate the integral in terms of x and y.
Every time we see z we will replace it by f of xy, whatever the formula for f might be.
Actually, if we look at a very easy case where this is just a horizontal plane, z equals constant, the function is just a constant, well, the partial derivatives become just zero.
You get  dx, dy.
That is what you would expect for a horizontal plane just from common sense.
This is more interesting, of course, if a function is more interesting.
How do we get that?
Where does this come from?
We need to figure out, for a small piece of our surface, what will be n delta S. Let's say that we take a small rectangle in here corresponding to sides delta x and delta y and we look at the piece of surface that is above that.
Well, the question we have now is what is the area of this little piece of surface and what is its normal vector?
Observe this little piece up here.
If it is small enough, it will look like a parallelogram.
I mean it might be slightly curvy, but roughly it looks like a parallelogram in space.
And so we have seen how to find the area of a parallelogram in space using cross-product.
If we can figure out what are the vectors for this side and that side then taking that cross-product and taking the magnitude of the cross-product will give us the area.
Moreover, the cross-product also gives us the normal direction.
In fact, the cross-product gives us two in one.
It gives us the normal direction and the area element.
And that is why I said that we will have an easy formula for n dS while n and dS taken separately are more complicated because you would have to actually take the length of a direction of this guy.
Let's carry out this problem.
Let's say I am going to look at a small piece of the x, y plane.
Here I have delta x, here I have delta y, and I am starting at some point (x, y).
Now, above that I will have a parallelogram on my surface.
This point here, the point where I start, I know what it is.
It is just (x, y).
And, well, z is f(x, y).
Now what I want to find, actually, is what are these two vectors, let's call them U and V, that correspond to moving a bit in the x direction or in the y direction?
And then U cross V will be, well, in terms of the magnitude of this guy will just be the little piece of surface area, delta S. And, in terms of direction, it will be normal to the surface.
Actually, I will get just delta S times my normal vector.
Well, up to sign because, depending on whether I do U V or V U, I might get the normal vector in the direction I want or in the opposite direction.
But we will take care of that later.
Let's find U and V. And, in case you have trouble with that small picture, I have a better one here.
Let's keep it just in case this one gets really too cluttered.
It really represents the same thing.
Let's try to figure out these vectors U and V. Vector U starts at the point x, y, f of x, y and it goes to -- Whereas, its head, well, I will have moved x by delta x.
So, x plus delta x and y doesn't change.
And, of course, the z coordinate has to change.
It becomes f of x plus delta x and y.
Now, how does f change if I change x a little bit?
Well, we have seen that it is given by the partial derivative f sub x.
This is approximately equal to f of x, y plus delta x times f sub x at the given point x, y. I am not going to add it because the notation is already long enough.
That means my vector U, well, approximately because I am using this linear approximation, <delta x, 0, f sub x times delta x>.
Is that OK with everyone?
Good.
Now, what about V?
Well, V works the same way so I am not going to do all the details.
When I move from here to here x doesn't change and y changes by delta y. X component nothing happens.
Y component changes by delta y.
What about the z component?
Well, f changes by f sub y times delta y.
That is how f changes if I increase y by delta y. I have my two sides.
Now I can take that cross-product.
Well, maybe I will first factor something out.
See, I can rewrite this as one, zero, f sub x times delta x.
And this one I will rewrite as zero, one, f sub y delta y.
And so now the cross-product, n hat delta S up to sign is going to be U cross V. We will have to do the cross-product, and we will have a delta x, delta y coming out.
I am just saving myself the trouble of writing a lot of delta x's and delta y's, but if you prefer you can just do directly this cross-product.
Let's compute this cross-product.
Well, the i component is zero minus f sub x.
The y component is going to be, well, f sub y minus zero but with the minus sign in front of everything, so negative f sub y.
And the z component will be just one times delta x delta y.
Does that make sense?
Yes.
Very good.
And so now we shrink this rectangle, we shrink delta x and delta y to zero, that is how we get this formula for n dS equals negative fx, negative fy, one, dxdy.
Well, plus/minus because it is up to us to choose whether we want to take the normal vector point up or down.
See, if you take this convention then the z component of n dS is positive.
That corresponds to normal vector pointing up.
If you take the opposite signs then the z component will be negative.
That means your normal vector points down.
This one is with n pointing up.
I mean when I say up, of course it is still perpendicular to the surface.
If the surface really has a big slope then it is not really going to go all that much up, but more up than down.
OK. That is how we get the formula.
Any questions?
No.
OK. That is a really useful formula.
You don't really need to remember all the details of how we got it, but please remember that formula.
Let's do an example, actually.
Let's say we want to find the flux of the vector field z times k, so it is a vertical vector field, through the portion of the paraboloid z equals x^2 y^2 that lives above the unit disk.
What does that mean?
z = x^2 y^2.
We have seen it many times.
It is this parabola and is pointing up.
Above the unit disk means I don't care about this infinite surface.
I will actually stop when I hit a radius of one away from the z-axis.
And so now I have my vector field which is going to point overall up because, well, it is z times k. The more z is positive, the more your vector field goes up.
Of course, if z were negative then it would point down, but it will live above.
Actually, a quick opinion poll.
What do you think the flux should be?
Should it be positive, zero, negative or we don't know?
I see some I don't know, I see some negative and I see some positive.
Of course, I didn't tell you which way I am orienting my paraboloid.
So far both answers are correct.
The only one that is probably not correct is zero because, no matter which way you choose to orient it you should get something.
It is not looking like it will be zero.
Let's say that I am going to do it with the normal pointing upwards.
Second chance.
I see some people changing back and forth from one and two.
Let's draw a picture.
Which one is pointing upwards?
Well, let's look at the bottom point.
The normal vector pointing up, here we know what it means.
It is this guy.
If you continue to follow your normal vector, see, they are actually pointing up and into the paraboloid.
And I claim that the answer should be positive because the vector field is crossing our paraboliod going upwards, going from the outside out and below to the inside and upside.
So, in the direction that we are counting positively.
We will see how it turns out when we do the calculation.
We have to compute the integral for flux.
Double integral over a surface of F dot n dS is going to be -- What are we going to do?
Well, F we said is <0,0, z>.
What is n dS.
Well, let's use our brand new formula.
It says negative f sub x, negative f sub y, one, dxdy.
What does little f in here?
It is x^2 y^2.
When we are using this formula, we need to know what little x stands for.
It is whatever the formula is for z as a function of x and y.
We take x^2 y^2 and we take the partial derivatives with minus signs.
We get negative 2x, negative 2y and one, dxdy.
Well, of course here it didn't really matter because we are going to dot them with zero.
Actually, even if we had made a mistake we somehow wouldn't have had to pay the price.
But still.
We will end up with double integral on S of z dxdy.
Now, what do we do with that?
Well, we have too many things.
We have to get rid of z.
Let's use z equals x^2 y^2 once more.
That becomes double integral of x^2 y^2 dxdy.
And here, see, we are using the fact that we are only looking at things that are on the surface.
It is not like in a triple integral.
You could never do that because z, x and y are independent.
Here they are related by the equation of a surface.
If I sound like I am ranting, but I know from experience this is where one of the most sticky and tricky points is.
OK. How will we actually integrate that?
Well, now that we have just x and y, we should figure out what is the range for x and y.
Well, the range for x and y is going to be the shadow of our region.
It is going to be this unit disk.
I can just do that for now.
And this is finally where I have left the world of surface integrals to go back to a usual double integral.
And now I have to set it up.
Well, I can do it this way with dxdy, but it looks like there is a smarter thing to do.
I am going to use polar coordinates.
In fact, I am going to say this is double integral of r^2 times r dr d theta.
I am on the unit disk so r goes zero to one, theta goes zero to 2pi.
And, if you do the calculation, you will find that this is going to be pi over two.
Any questions about the example.
Yes?
How did I get this negative 2x and negative 2y?
I want to use my formula for n dS.
My surface is given by the graph of a function.
It is the graph of a function x^2 y^2.
I will use this formula that is up here.
I will take the function x^2 y^2 and I will take its partial derivatives.
If I take the partial of f, so x^2 y^2 with respect to x, I get 2x, so I put negative 2x.
And then the same thing, negative 2y, one, dxdy.
Yes?
Which k hat?
Oh, you mean the vector field.
It is a different part of the story.
Whenever you do a surface integral for flux you have two parts of the story.
One is the vector field whose flux you are taking.
The other one is the surface for which you will be taking flux.
The vector field only comes as this f in the notation, and everything else, the bounds in the double integral and the n dS, all come from the surface that we are looking at.
Basically, in all of this calculation, this is coming from f equals zk.
Everything else comes from the information paraboloid z = x^2 y^2 above the unit disk.
In particular, if we wanted to now find the flux of any other vector field for the same paraboloid, well, all we would have to do is just replace this guy by whatever the new vector field is.
We have learned how to set up flux integrals for this paraboloid.
Not that you should remember this one by heart.
I mean there are many paraboloids in life and other surfaces, too.
It is better to remember the general method.
Any other questions?
No.
OK. Let's see more ways of taking flux integrals.
But, just to reassure you, at this point we have seen the most important ones.
90% of the problems that we will be looking at we can do with what we have seen so far in less time and this formula.
Let's look a little bit at a more general situation.
Let's say that my surface is so complicated that I cannot actually express z as a function of x and y, but let's say that I know how to parametize it.
I have a parametric equation for my surface.
That means I can express x, y and z in terms of any two parameter variables that might be relevant for me.
If you want, this one here is a special case where you can parameterize things in terms of x and y as your two variables.
How would you do it in the fully general case?
In a way, that will answer your question that, I think one of you, I forgot, asked yesterday how would I do it in general?
Is there a formula like M dx plus N dy?
Well, that is going to be the general formula.
And you will see that it is a little bit too complicated, so the really useful ones are actually the special ones.
Let's say that we are given a parametric description -- -- of a surface S. That means we can describe S by formulas saying x is some function of two parameter variables.
I am going to call them u and v. I hope you don't mind.
You can call them t1 and t2.
You can call them whatever you want.
One of the basic properties of a surface is because I have only two independent directions to move on.
I should be able to express x, y and z in terms of two variables.
Now, let's say that I know how to do that.
Or, maybe I should instead think of it in terms of a position vector if it helps you.
That is just a vector with components <x, y, z> is given as a function of u and v. It works like a parametric curve but with two parameters.
Now, how would we actually set up a flux integral on such a surface.
Well, because we are locating ourselves in terms of u and v, we will end up with an integral du dv.
We need to figure out how to express n dS in terms of du and dv.
N dS should be something du dv.
How do we do that?
Well, we can use the same method that we have actually used over here.
Because, if you think for a second, here we used, of course, a rectangle in the x, y plane and we lifted it to a parallelogram and so on.
But more generally you can think what happens if I change u by delta u keeping v constant or the other way around?
You will get some sort of mesh grid on your surface and you will look at a little parallelogram that is an elementary piece of that mesh and figure out what is its area and normal vector.
Well, that will again be given by the cross-product of the two sides.
Let's think a little bit about what happens when I move a little bit on my surface.
I am taking this grid on my surface given by the u and v directions.
And, if I take a piece of that corresponding to small changes delta u and delta v, what is going to be going on here?
Well, I have to deal with two vectors, one corresponding to changing u, the other one corresponding to changing v. If I change u, how does my point change?
Well, it is given by the derivative of this with respect to u.
This vector here I will call, so the sides are given by, let me say, partial r over partial u times delta u.
If you prefer, maybe I should write it as partial x over partial u times delta u.
Well, it is just too boring to write.
And so on.
It means if I change u a little bit, keeping v constant, then how x changes is, given by partial x over partial u times delta u, same thing with y, same thing with z, and I am just using vector notation to do it this way.
That is the analog of when I said delta r for line integrals along a curve, vector delta r is the velocity vector dr dt times delta t. Now, if I look at the other side -- Let me start again.
I ran out of space.
One side is partial r over partial u times delta u.
And the other one would be partial r over partial v times delta v. Because that is how the position of your point changes if you just change u or v and not the other one.
To find the surface element together with a normal vector, I would just take the cross-product between these guys.
If you prefer, that is the cross-product of partial r over partial u with partial r over partial v, delta u delta v. And so n dS is this cross-product times du dv up to sign.
It depends on which choice I make for my normal vector, of course.
That, of course, is a slightly confusing equation to think of.
A good exercise, if you want to really understand what is going on, try this in two good examples to look at.
One good example to look at is the previous one.
What is it?
It is when u and v are just x and y.
The parametric equations are just x equals x, y equals y and z is f of x, y.
You should end up with the same formula that we had over there.
And you should see why because both of them are given by a cross-product.
The other case you can look at just to convince yourselves even further.
We don't need to do that because we have seen the formula before, but in the case of a sphere we have seen the formula for n and for dS separately.
We know what n dS are in terms of d phi, d theta.
Well, you could parametize a sphere in terms of phi and theta.
Namely, the formulas would be x equals a sine phi cosine theta, y equals a sign phi sine theta, z equals a cosine phi.
The formulas for circle coordinates setting Ro equals a .
That is a parametric equation for the sphere.
And then, if you try to use this formula here, you should end up with the same things we have already seen for n dS, just with a lot more pain to actually get there because cross-product is going to be a bit complicated.
But we are seeing all of these formulas all fitting together.
Somehow it is always the same question.
We just have different angles of attack on this general problem.
Questions?
No.
OK. Let's look at yet another last way of finding n dS.
And then I promise we will switch to something else because I can feel that you are getting a bit overwhelmed for all these formulas for n dS.
What happens very often is we don't actually know how to parametize our surface.
Maybe we don't know how to solve for z as a function of x and y, but our surface is given by some equation.
And so what that means is actually maybe what we know is not really these kinds of formulas, but maybe we know a normal vector.
And I am going to call this one capital N because I don't even need it to be a unit vector.
You will see.
It can be a normal vector of any length you want to the surfaces.
Why would we ever know a normal vector?
Well, for example, if our surface is a plane, a slanted plane given by some equation, ax by cz = d. Well, you know the normal vector.
It is .
Of course, you could solve for z and then go back to that case, which is why I said that one is very useful.
But you can also just stay with a normal vector.
Why else would you know a normal vector?
Well, let's say that you know an equation that is of a form g of x, y, z equals zero.
Well, then you know that the gradient of g is perpendicular to the level surface.
Let me just give you two examples.
If you have a plane, ax by cz = d, then the normal vector would just be <a, b, c>.
If you have a surface S given by an equation, g(x, y, z) = 0, then you can take a normal vector to be the gradient of g. We have seen that the gradient is perpendicular to the level surface.
Now, of course, we don't necessarily have to follow what is going to come.
Because, if we could solve for z, then we might be better off doing what we did over there.
But let's say that we want to do it this.
What can we do?
Well, I am going to give you another way to think geometrically about n dS.
Let's start by thinking about the slanted plane.
Let's say that my surface is just a slanted plane.
My normal vector would be maybe somewhere here.
And let's say that I am going to try -- I need to get some handle on how to set up my integrals, so maybe I am going to express things in terms of x and y. I have my coordinates, and I will try to use x and y.
Then I would like to relate delta S or dS to the area in the x y plane.
That means I want maybe to look at the projection of this guy onto a horizontal plane.
Let's squish it horizontally.
Then you have here another area.
The guy on the slanted plane, let's call that delta S. And let's call this guy down here delta A.
And delta A would become ultimately maybe delta x, delta y or something like that.
The question is how do we find the conversion rate between these two areas?
I mean they are not the same.
Visually, I hope it is clear to you that if my plane is actually horizontal then, of course, they are the same.
But the more slanted it becomes the more delta A becomes smaller than delta S. If you buy land and it is on the side of a cliff, well, whether you look at it on a map or whether you look at it on the actual cliff, the area is going to be very different.
I am not sure if that is a wise thing to do if you want to build a house there, but I bet you can get really cheap land.
Anyway, delta S versus delta A depends on how slanted things are.
And let's try to make that more precise by looking at the angel that our plane makes with the horizontal direction.
Let's call this angle alpha, the angle that our plane makes with the horizontal direction.
See, it is all coming together.
The first unit about cross-products, normal vectors and so on is actually useful now.
I claim that the surface element is related to the area in the plane by delta A equals delta S times the cosine of alpha.
Why is that?
Well, let's look at this small rectangle with one horizontal side and one slanted side.
When you project this side does not change, but this side gets shortened by a factor of cosine alpha.
Whatever this length was, this length here is that one times cosine alpha.
That is why the area gets shrunk by cosine alpha.
In one direction nothing happens.
In the other direction you squish by cosine alpha.
What that means is that, well, we will have to deal with this.
And, of course, the one we will care about actually is delta S expressed in terms of delta A.
But what are we going to do with this cosine?
It is not very convenient to have a cosine left in here.
Remember, the angle between two planes is the same thing as the angle between the normal vectors.
If you want to see this angle alpha elsewhere, what you can do is you can just take the vertical direction.
Let's take k. Then here we have our angle alpha again.
In particular, cosine of alpha, I can get, well, we know how to find the angle between two vectors.
If we have our normal vector N, we will do N dot k, and we will divide by length N, length k. Well, length k is one.
That is one easy guy.
That is how we find the angle.
Now I am going to say, well, delta S is going to be one over cosine alpha delta A.
And I can rewrite that as length of N divided by N dot k times delta A.
Now, let's multiply that by the unit normal vector.
Because what we are about is not so much dS but actually n dS.
N delta S will be, I am just going to multiply by N. Well, let's think for a second.
What happens if I take a unit normal N and I multiply it by the length of my other normal big N?
Well, I get big N again.
This is a normal vector of the same length as N, well, up to sign.
The only thing I don't know is whether this guy will be going in the same direction as big N or in the opposite direction.
Say that, for example, my capital N has, I don't know, length three for example.
Then the normal unit vector might be this guy, in which case indeed three times little n will be big n. Or it might be this one in which case three times little n will be negative big N. But up to sign it is N. And then I will have N over N dot k delta A.
And so the final formula, the one that we care about in case you don't really like my explanations of how we get there, is that N dS is plus or minus N over N dot k dx dy.
That one is actually kind of useful so let's box it.
Now, just in case you are wondering, of course, if you didn't want to project to x, y, you would have maybe preferred to project to say the plane of a blackboard, y, z, well, you can do the same thing.
To express n dS in terms of dy dz you do the same argument.
Simply, the only thing that changes, instead of using the vertical vector k, you use the normal vector i.
So you would be doing N over N dot i dy dz.
The same thing.
So just keep an open mind that this also works with other variables.
Anyway, that is how you can basically project the vectors of this area element onto the x, y plane in a way.
Let's look at the special case just to see how this fits with stuff we have seen before.
Let's do a special example where our surface is given by the equation z minus f of x, y equals zero.
That is a strange way to write the equation.
z equals f of x, y.
That we saw before.
But now it looks like some function of x, y, z equals zero.
Let's try to use this new method.
Let's call this guy g(x, y, z).
Well, now let's look at the normal vector.
The normal vector would be the gradient of g, you see.
What is the gradient of this function?
The gradient of g -- Well, partial g, partial x, that is just negative partial f, partial x.
The y component, partial g, partial y is going to be negative f sub y, and g sub z is just one.
Now, if you take N over N dot k dx dy, well, it looks like it is going to be negative f sub x, negative f sub y, one divided by -- Well, what is N dot k?
If you dot that with k you will get just one, so I am not going to write it, dx dy.
See, that is again our favorite formula.
This one is actually more general because you don't need to solve for z, but if you cannot solve for z then it is the same as before.
I think that is enough formulas for n dS.
After spending a lot of time telling you how to compute surface integrals, now I am going to try to tell you how to avoid computing them.
And that is called the divergence theorem.
And we will see the proof and everything and applications on Tuesday, but I want to at least the theorem and see how it works in one example.
It is also known as the Gauss-Green theorem or just the Gauss theorem, depending in who you talk to.
The Green here is the same Green as in Green's theorem, because somehow that is a space version of Green's theorem.
What does it say?
It is 3D analog of Green for flux.
What it says is if S is a closed surface -- Remember, it is the same as with Green's theorem, we need to have something that is completely enclosed.
You have a surface and there is somehow no gaps in it.
There is no boundary to it.
It is really completely enclosing a region in space that I will call D. And I need to choose my orientation.
The orientation that will work for this theorem is choosing the normal vector to point outwards.
N needs to be outwards.
That is one part of the puzzle.
The other part is a vector field.
I need to have a vector field that is defined and differentiable -- -- everywhere in D, so same instructions as usual.
Then I don't have actually to compute the flux integral.
Double integral of f dot n dS of a closed surface S. I am going to put a circle just to remind you it is has got to be a closed surface.
It is just a notation to remind us closed surface.
I can replace that by the triple integral of a region inside of divergence of F dV.
Now, I need to tell you what the divergence of a 3D vector field is.
Well, you will see that it is not much harder than in the 2D case.
What you do is just -- Say that your vector field has components P, Q and R. Then you will take P sub x Q sub y R sub z.
That is the definition.
It is pretty easy to remember.
You take the x component partial respect to S plus partial respect to y over y component plus partial respect to z of the z component.
For example, last time we saw that the flux of the vector field zk through a sphere of radius a was four-thirds pi a cubed by computing the surface integral.
Well, if we do it more efficiently now by Green's theorem, we are going to use Green's theorem for this sphere because we are doing the whole sphere.
It is fine.
It is a closed surface.
We couldn't do it for, say, the hemisphere or something like that.
Well, for a hemisphere we would need to add maybe the flat face of a bottom or something like that.
Green's theorem says that our flux integral can actually be replaced by the triple integral over the solid bowl of radius a of the divergence of zk dV.
But now what is the divergence of this field?
Well, you have zero, zero, z so you get zero plus zero plus one.
It looks like it will be one.
If you do the triple integral of 1dV, you will get just the volume -- -- of the region inside, which is four-thirds by a cubed.
And so it was no accident.
In fact, before that we looked at also xi yj zk and we found three times the volume.
That is because the divergence of that field was actually three.
Very quickly, let me just say what this means physically.
Physically, see, this guy on the left is the total amount of stuff that goes out of the region per unit time.
I want to figure out how much stuff comes out of there.
What does the divergence mean?
The divergence means it measures how much the flow is expanding things.
It measures how much, I said that probably when we were trying to understand 2D divergence.
It measures the amount of sources or sinks that you have inside your fluid.
Now it becomes commonsense.
If you take a region of space, the total amount of water that flows out of it is the total amount of sources that you have in there minus the sinks.
I mean, in spite of this commonsense explanation, we are going to see how to prove this.
And we will see how it works and what it says.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so remember we left things with this statement of the divergence theorem.
So, the divergence theorem gives us a way to compute the flux of a vector field for a closed surface.
OK, it says if I have a closed surface, s, bounding some region, D, and I have a vector field defined in space, so that I can try to compute the flux of my vector field through my surface.
Double integral of F.dS or F.ndS if you want, and to set this up, of course, I need to use the geometry of the surface depending on what the surface is.
We've seen various formulas for how to set up the double integral.
But, we've also seen that if it's a closed surface, and if a vector field is defined everywhere inside, then we can actually reduce that to a calculation of the triple integral of the divergence of F inside, OK?
So, concretely, if I use coordinates, let's say that the coordinates of my vector field are, sorry, the components are P, Q, and R dot ndS, then that will become the triple integral of, well, so, divergence is P sub x plus Q sub y plus R sub z. OK, so by the way, how to remember this formula for divergence, and other formulas for other things as well.
Let me just tell you quickly about the del notation.
So, this guy usually pronounced as del, rather than as pointy triangle going downwards or something like that, it's a symbolic notation for an operator.
So, you're probably going to complain about putting these guys into a vector.
But, let's think of partial with respect to x, with respect to y, and with respect to z as the components of some formal vector.
Of course, it's not a real vector.
These are not like anything.
These are just symbols.
But, so see for example, the gradient of function, well, if you multiply this vector by scalar, which is a function, then you will get partial, partial x of f, partial, partial y of f, partial, partial z, f, well, that's the gradient.
That seems to work.
So now, the interesting thing about divergence is I can think of divergence as del dot a vector field.
See, if I do the dot product between this guy and my vector field P, Q, R, well, it looks like I will indeed get partial, partial x of P plus partial Q partial y plus partial R partial z.
That's the divergence.
and of course, similarly, when we have two variables only, x and y, we could have thought of the same notation, just with a two component vector, partial, partial x, partial, partial y.
So, now, this is like of slightly limited usefulness so far.
It's going to become very handy pretty soon because we are going to see curl.
And, the formula for curl in the plane was kind of complicated.
But, if you thought about it in terms of this, it was actually the determinant of del and f. And now, in space, we are actually going to do del cross f. But, I'm getting ahead of things.
So, let's not do anything with that.
Curl will be for next week.
Just getting you used to the notation, especially since you might be using it in physics already.
So, it might be worth doing.
OK, so the other thing I wanted to say is, what does this theorem say physically?
How should I think of this statement?
So, I think I said that very quickly at the end of last time, but not very carefully.
So, what's the physical interpretation of a divergence field?
So, I want to claim that the divergence of a vector field corresponds to what I'm going to call the source rate, which is somehow the amount of flux generated per unit volume.
So, to understand what that means, let's think of what's called an incompressible fluid.
OK, so an incompressible fluid is something like water, for example, where a fixed mass of it always occupies the same amount of volume.
So, guesses are compressible.
Liquids are incompressible, basically.
So, if you have an incompressible fluid flow -- -- well, so, again, what that means is really, given mass occupies always a fixed volume.
Then, well, let's say that we have such a fluid with velocity given by our vector field.
OK, so we're thinking of F as the velocity and maybe something containing water, a pipe, or something.
So, what does the divergence theorem say?
It says that if I take a region in space, let's call it D, sorry, D is the inside, and S is the surface around it, well, so if I sum the divergence in D, well, I'm going to get the flux going out through this surface, S. I should have mentioned it earlier.
The convention in the divergence theorem is that we orient the surface with a normal vector pointing always outwards.
OK, so now, we know what flux means.
Remember, we've been describing, flux means how much fluid is passing through this surface.
So, that's the amount of fluid that's leaving the region, D, per unit time.
And, of course, when I'm saying that, it means I'm counting everything that's going out of D minus everything that's coming into D. That's what the flux measures.
So, now, if there is stuff coming into D or going out of D, well, it must come from somewhere.
So, one possibility would be that your fluid is actually being compressed or expanded.
But, I've said, no, I'm looking at something like water that you cannot squish into smaller volume.
So, in that case, the only explanation is that there is something it here that actually is sucking up water or producing more water.
And so, integrating the divergence gives you the total amount of sources minus the amount of syncs that are inside this region.
So, the divergence itself measures basically the amount of sources or syncs per unit volume in a given place.
And now, if you think about it that way, well, it's basically the divergence theorem is just stating something completely obvious about all the matter that is leaving this region must come from somewhere.
So, that's basically how we think about it.
Now, of course, if you're doing 8.02, then you might actually have seen the divergence theorem already being used for things that are more like force fields, say, electric fields and so on.
Well, I'll try to say a few things about that during the last week of classes.
But, then this kind of interpretation doesn't quite work.
OK, any questions, generally speaking, before we move on to the proof and other applications?
Yes?
Oh, not the gradient.
So, yeah, the divergence of F measures the amount of sources or syncs in there.
Well, what makes it happen?
If you want, in a way, it's this theorem.
Or, in another way, if you think about it, try to look at your favorite vector fields and compute their divergence.
And, if you take a vector field where maybe everything is rotating, a flow that's just rotating about some axis, then you'll find that its divergence is zero.
If you, sorry?
No, divergence is not equal to the gradient.
Sorry, there's a dot here that maybe is not very big, but it's very important.
OK, so you take the divergence of a vector field.
Well, you take the gradient of a function.
So, if the gradient of a function is a vector, the divergence of a vector field is a function.
So, somehow these guys go back and forth between.
So, I should have said, with new notations comes new responsibility.
I mean, now that we have this nice, nifty notation that will let us do gradient divergence and later curl in a unified way, if you choose this notation you have to be really, really careful what you put after it because otherwise it's easy to get completely confused.
OK, so divergence and gradients are completely different things.
The only thing they have in common is that both are what's called a first order differential operator.
That means it involves the first partial derivatives of whatever you put into it.
But, one of them goes from functions to vectors.
That's gradient.
The other one goes from vectors to functions.
That's divergence.
And, curl later will go from vectors to vectors.
But, that will be later.
Let's see, more questions?
No?
OK, so let's see, so how are we going to actually prove this theorem?
Well, if you remember how we prove Green's theorem a while ago, the answer is we're going to do it exactly the same way.
So, if you don't remember, then I'm going to explain.
OK, so the first thing we need to do is actually a simplification.
So, instead of proving the divergence theorem, namely, the equality up there, I'm going to actually prove something easier.
I'm going to prove that the flux of a vector field that has only a z component is actually equal to the triple integral of, well, the divergence of this is just R sub z dV.
OK, now, how do I go back to the general case?
Well, I will just prove the same thing for a vector field that has only an x component or only a y component.
And then, I will add these things together.
So, if you think carefully about what happens when you evaluate this, you will have some formula for ndS, and when you do the dot product, you'll end up with the sum, P times something plus Q times something plus R times something.
And basically, we are just dealing with the last term, R times something, and showing that it's equal to what it should be.
And then, we the three such terms together.
We'll get the general case.
OK, so then we get the general case by summing one such identity for each component.
I should say three such identities, one for each component, whatever.
Now, let's make a second simplification because I'm still not feeling confident I can prove this right away for any surface.
I'm going to do it first or what's called a vertically simple region.
OK, so vertically simple means it will be something which I can setup an integral over the z variable first easily.
So, it's something that has a bottom face, and a top face, and then some vertical sides.
OK, so let's say first what happens if the given region, D, is vertically simple.
So, vertically simple means it looks like this.
It has top.
It has a bottom.
And, it has some vertical sides.
So, if you want, if I look at it from above, it projects to some region in the xy plane.
Let's call that R. And, it lives between the top face and the bottom face.
Let's say the top face is z equals z2 of (x, y).
Let's say the bottom face is z equals z1(x, y).
OK, and I don't need to know actual formulas.
I'm just going to work with these and prove things independently of what the formulas will be for these functions.
OK, so anyway, a vertically simple region is something that lives above a part of the xy plane, and is between two graphs of two functions.
So, let's see what we can do in that case.
So, the right-hand side of this equality, so that's the triple integral, let's start computing it.
OK, so of course we will not be able to get a number out of it because we don't know, actually, formulas for anything.
But at least we can start simplifying because the way this region looks like, I should say this is D, tells me that I can start setting up the triple integral at least in the order where I integrate first over z. OK, so I can actually do it as a triple integral with Rz dz dxdy or dydx, doesn't matter.
So, what are the bounds on z?
See, this is actually good practice to remember how we set up triple integrals.
So, remember, when we did it first over z, we start by fixing a point, x and y, and for that value of x and y, we look at a small vertical slice and see from where to where we have to go.
Well, we start at z equals whatever the value is at the bottom, so, z1 of x and y.
And, we go up to the top face, z2 of x and y.
Now, for x and y, I'm not going to actually set up bounds because I've already called R the quantity that I'm integrating.
So let me change this to, let's say, U or something like that.
If you already have an R, I mean, there's not much risk for confusion, but still.
OK, so we're going to call U the shadow of my region instead.
So, now I want to integrate over all values of x and y that are in the shadow of my region.
That means it's a double integral over this region, U, which I haven't described to you.
So, I can't actually set up bounds for x and y.
But, I'm going to just leave it like this.
OK, now you see, if you look at how you would start evaluating this, well, the inner integral certainly is not scary because you're integrating the derivative of R with respect to z, integrating that with respect to z.
So, you should get R back.
OK, so triple integral over D of Rz dV becomes, well, we'll have a double integral over U of, so, the inner integral becomes R at the point on the top.
So, that means, remember, R is a function of x, y, and z.
And, in fact, I will plug into it the value of z at the top, so, z of xy minus the value of R at the point on the bottom, x, y, z1 of x, y. OK, any questions about this?
No?
Is it looking vaguely believable?
Yeah?
OK.
So, now, let's compute the other side because here we are stuck.
We won't be able to do anything else.
So, let's look at the flux integral.
OK, we have to look at the flux of this vector field through the entire surface, S, which is the whole boundary of D. So, that consists of a lot of pieces, namely the top, bottom, and the sides.
OK, so the other side -- So, let me just remind you, S is bottom plus top plus side of this vector field, dot ndS equals, OK, so what do we have?
So first, we have to look at the bottom.
No, let's start with the top actually.
Sorry.
OK, so let's start with the top.
So, just remind you, let's do all of them.
So, let's look at the top first.
So, we need to set up the flux integral for a vector field dot ndS.
We need to know what ndS is.
Well, fortunately for us, we know that the top face is going to be the graph of some function of x and y.
So, we've seen a formula for ndS in this kind of situation, OK?
We have seen that ndS, sorry, so, just to remind you this is the graph of a function z equals z2 of x, y.
So, we've seen ndS for that is negative partial derivative of this function with respect to x, negative partial z2 with respect to y, one, dxdy.
OK, and, well, we can't compute these guys, but it's not a big deal because if we do the dot product with dot ndS, that will give us, well, if you dot this with zero, zero, R, these terms go away.
You just have R dxdy.
So, that means that the double integral for flux through the top of R vector field dot ndS becomes double integral of the top of R dxdy.
Now, how do we evaluate that, actually?
Well, so R is a function of x, y, z.
But we said, we have only two variables that we're going to use.
We're going to use x and y.
We're going to get rid of z.
How do we get rid of z?
Well, if we are on the top surface, z is given by this formula, z2 of x, y.
So, I plug z equals z2 of x, y into the formula for R, whatever it may be.
Then, I integrate dxdy.
And, what's the range for x and y?
Well, my surface sits exactly above this region U in the xy plane.
So, it's double integral over U, OK?
Any questions about how I set up this flux integral?
No?
OK, let me close the door, actually.
OK, so we've got one of the two terms that we had over there.
Let's try to get the others.
[LAUGHTER] No comment.
OK, so, we need to look, also, at the other parts of our surface for the flux integral.
So, the bottom, well, it will work pretty much the same way, right, because it's the graph of a function, z equals z1 of x, y.
So, we should be able to get ndS using the same method, negative partial with respect to x, negative partial with respect to y, one dxdy.
Now, there's a small catch.
OK, we have to think of it carefully about orientations.
So, remember, when we set up the divergence theorem, we need the normal vectors to point out of our region, which means that on the top surface, we want n pointing up.
But, on the bottom face, we want n pointing down.
So, in fact, we shouldn't use this formula here because that one corresponds to the other orientation.
Well, we could use it and then subtract, but I was just going to say that ndS is actually the opposite of this.
So, I'm going to switch all my signs.
OK, that's the other side of the formula when you orient your graph with n pointing downwards.
Now, if I do things the same way as before, I will get that <0,0, R> dot ndS will be negative R dxdy.
And so, when I do the double integral over the bottom, I'm going to get the double integral over the bottom of negative R dxdy, which, if I try to evaluate that, well, I actually will have to integrate.
Sorry, first I'll have to substitute the value of z.
The value of z that I will want to plug into R will be given by, now, z1 of x, y.
And, the bounds of integration will be given, again, by the shadow of our surface, which is, again, this guy, U. OK, so we seem to be all set, except we are still missing one part of our surface, S, because we also need to look at the sides.
Well, what about the sides?
Well, our vector field, , is actually vertical.
It's parallel to the z axis.
OK, so my vector field does something like this everywhere.
And, why that makes it very interesting on the top and bottom, that means that on the sides, really not much is going on.
No matter is passing through the vertical sides.
So, the side -- The sides are vertical.
So, <0,0, R> is tangent to the side, and therefore, the flux through the sides is just going to be zero.
OK, no calculation needed.
That's because, of course, that's the reason why a simplified first things so that my vector field would only have a z component, well, not just that but that's the main place where it becomes very useful.
So, now, if I compare my double integral and, sorry, my triple integral and my flux integral, I get that they are, indeed, the same.
Well, that's the statement of the theorem we are trying to prove.
I shouldn't erase it, OK?
[LAUGHTER] So, just to recap, we've got a formula for the triple integral of R sub z dV.
It's up there at the very top.
And, we got formulas for the flux through the top and the bottom, and the sides.
And, when you add them together, you get indeed the same formula, top plus bottom -- -- plus sides of, OK, and so we have, actually, completed the proof for this part.
Now, well, that's only for a vertically simple region, OK?
So, if D is not vertically simple, what do we do?
Well, we cut it into vertically simple pieces.
OK so, concretely, I wanted to integrate over a solid doughnut.
Then, that's not vertically simple because here in the middle, I have not only does top in this bottom, but I have this middle face.
So, the way I would cut my doughnut would be probably I would slice it not in the way that you'd usually slice the doughnut or a bagel, but at it's probably more spectacular if you think that it's a bagel.
Then, a mathematician's way of slicing it is like this into five pieces, OK?
And, that's not very convenient for eating, but that's convenient for integrating over it because now each of these pieces has a well-defined top and bottom face, and of course you've introduced some vertical sides for two reasons.
One is that we've said the flux through them is zero anyway.
So, who cares?
Why bother?
But, also, if you sum the flux through the surface of each little piece, well, you will see that you will be integrating twice over each of these vertical cuts.
Once, when you do this piece, you will be taking the flux through this red guy with normal vector pointing to the right, and once, when you take this middle little piece, you will be taking the flux through that cut surface again, but now with normal vector pointing the other way around.
So, in fact, you'll be summing the flux through these guys twice with opposite orientations.
They cancel out.
Well, and again, because of what you are doing actually, the integral was just zero anyway.
So, it didn't matter.
But, even if it hadn't simplified, that would do it for us.
OK, so that's how we do it with the general region.
And then, as I said at the beginning, when we can do it for a vector field that has only a z component, well, we can also do it for a vector field that has only an x or only a y component.
And then, we sum together and we get the general case.
So, that's the end of the proof.
OK, so you see, the idea is really the same as for Green's theorem.
Yes?
Oh, there's only four pieces, thank you.
Yes, there's three kinds of mathematicians: those who know how to count, and those who don't.
Well, OK.
So, OK, now I hope that you can see already the interest of this theorem for the divergence theorem for computing flux integrals just for the sake of computing flux integrals like might happen on the problem set or on the next test.
But let me tell you also why it's important physically to understand equations that had been observed empirically well before they were actually understood in terms of how things go.
So, let's look at something called the diffusion equation.
So, let me explain to you what it does.
So, the diffusion equation is something that governs, well, what's called diffusion.
Diffusion is when you have a fluid in which you are introducing some substance, and you want to figure out how that thing is going to spread out, the technical term is diffuse, into the ambient fluid.
So, for example, that governs the motion of, say, smoke in the air, or if you put dye in the solution or things like that.
That will tell you, say that you drop some ink into a glass of water.
Well, you can imagine that obviously it will get diluted into there.
And, that equation will tell you how exactly over time this thing is going to spread out and start filling the entire glass.
So, what's the equation?
Well, we need, first, to know what the unknown will be.
So, it's a partial differential equation, OK?
So the unknown is a function, and the equation will relate the partial derivatives of that function to each other.
So, u, the unknown, will be the concentration at a given point.
And, of course, that depends on the point where you are.
So, that depends on x, y, z, the location where you are.
But, since the goal is also to understand how things spread over time, it should also depend on time.
Otherwise, we're not going to get very far.
And, in fact, what the equation will give us is the derivative of u with respect to t. It will tell us how the concentration at a given point varies over time in terms of how the concentration varied in space.
So, it will relate partial u partial t to partial derivatives with respect to x, y, and z.
[APPLAUSE] OK, [LAUGHTER] so what's the equation?
The equation is partial u partial t equals some constant.
Let me call it constant k times something I will call del squared u, which is also called the Laplacian of u, and what is that?
Well, that means, OK, so just to scare you, del squared is this, which means it's the divergence of gradient u that we've used this notation for gradient.
OK, so if you actually expand it in terms of variables, that becomes partial u over partial x squared plus partial squared u over partial y squared plus partial squared u over partial z squared.
OK, so the equation is this equals that.
OK, so that's a weird looking equation.
And, I'm going to have to explain to you, where does it come from?
OK, but before I do that, well, let me point out actually that the equation is not just the diffusion equation.
It's also known as the heat equation.
And, that's because exactly the same equation governs how temperature changes over time when you have, again, so, sorry I should have been actually more careful.
I should have said this is in air that's not moving, OK?
OK, and same thing with the solution.
If you drop some ink into your glass of water, well, if you start stirring, obviously it will mix much faster than if you don't do anything.
OK, so that's the case where we don't actually, the fluid is not moving.
And, the heat equation now does the same, but for temperature in a fluid that's at rest, that's not moving.
It tells you how the heat goes from the warmest parts to the coldest parts, and eventually temperatures should somehow even out.
So, in the heat equation, that would just be, this u would just measure the temperature for temperature of your fluid at a given point.
Actually, it doesn't have to be a fluid.
It could be a solid for that heat equation.
So, for example, say that you have a big box made of metal or something, and you do some heating at one side.
You want to know how quickly is the other side going to get hot?
Well, you can use the equation to figure out, you know, say that you have a metal bar, and you keep one side at 400� because it's in your oven.
How quickly will the other side get warm?
OK, so it's the same equation for both phenomena even though they are, of course, different phenomena.
Well, the physical reason why they're the same is actually that phenomena are different, but not all that much.
They involve, actually, how the atoms and molecules are actually moving, and hitting each other inside this medium.
OK, so anyway, what's the explanation for this?
So, to understand the explanation, and given what we've been doing, we should have a vector field in there.
So, I'm going to think of the flow of, well, let's imagine that we are doing motion of smoke in air.
So, that's the flow of the smoke: that means at every point, it's a vector whose direction tells me in which direction the smoke is actually moving.
And, its magnitude tells me how fast it's moving, and also what amount of smoke is moving.
So, there's two things to understand.
One is how the disparities in the concentration between different points causes the flow to be there, how smoke will flow from the regions where there's more smoke to the regions where there's less smoke.
And, that's actually physics.
But, in a way, it's also common sense.
So, physics and common sense tell us that the smoke will flow from high concentration towards low concentration regions.
OK, so if we think of this function, U, that measures concentration, that means we are always going to go in the direction where the concentration decreases the fastest.
Well, what's that?
That's negative the gradient.
So, F is directed along minus gradient u.
Now, how big is F going to be?
Well, they are, you have to come up with some intuition for how the two are related.
And, the easiest relation I can think of is that they might be just proportional.
So, the steeper the differences in concentration, the faster the flow will be, or the more there will be flow.
And, if you try to think about it as molecules moving in random directions, you will see it makes actually complete sense.
Anyway, it should be part of your physics class, not part of what I'm telling you.
So, I'm just going to accept that the flow is just proportional to the gradient of u.
So, if you want, the differences between concentrations of different points are very small, then the flow will be very gentle.
And, if on the other hand you have huge disparities, then things will try to even out faster.
OK, so that's the first part.
Now, we need to understand the second part, which is once we know how flow is going, how does that affect the concentration?
If smoke is going that way, then it means the concentration here should be decreasing.
And there, it should be increasing.
So, that's the relation between F and partial u partial t. At that part is actually math, namely, the divergence theorem.
So, let's try to understand that part more carefully.
So, let's take a small piece of a small region in space, D, bounded by a surface, S. So, I want to figure out what's going on in here.
So, let's look at the flux out of D through S. Well, we said that this flux would be given by double integral on S of F dot n dS.
So, this flux measures how much smoke is passing through S per unit time.
That's the amount of smoke through S per unit time.
But now, how can I compute that differently?
Well, I can compute it just by looking at the total amount of smoke in this region, and seeing how much it changes.
If I'm gaining or losing smoke, it means it must be going up there.
Well, unless I have a smoker in here, but that's not part of the data.
So, that should be, sorry, that's the same thing as the derivative with respect to t of the total amount of smoke in the region, which is given by the triple integral of u.
If I integrate the concentration of smoke, which means the amount of smoke per unit volume over d, I will get the total amount of smoke in d, except, well, let's see.
This flux is counted positively if we go out of d. So, actually, it's minus the derivative.
This is the amount of smoke that we are losing per unit time.
OK, so now we are almost there.
Well, let me actually -- Because we know yet another way to compute this guy using the divergence theorem.
Right, so this part here is just common sense and thinking about what it means.
The divergence theorem tells me this is also equal to the triple integral, d, of div f dV.
So, what I got is that the triple integral over d of div F dV equals this derivative.
Well, let's think a bit about this derivative so, see, you are integrating function over x, y, and z.
And then, you are differentiating with respect to t. I claim that you can actually switch the order in which you do things.
So, when we think about it, is, here, you are taking the total amount of smoke and then see how that changes over time.
So, you're taking the derivative of the sum of all the small amounts of smoke everywhere.
Well, that will be the sum of the derivatives of the amounts of smoke inside each little box.
So, we can actually move the derivatives into the integral.
So, let's see, I said minus d dt of triple integral over d udV.
And, now I'm saying this is the same as the triple integral in d of du dt dv.
And the reason why this is going to work is you should think of it as d dt of a sum of u of some values.
You plug in the values of your points at that given time times the small volume.
You sum them, and then you take the derivative.
And now, you see, the derivative of this sum is the sum of the derivatives.
yi, zi, t, so, if you think about what the integral measures, that's actually pretty easy.
And it's because this variable here is not the same as the variables on which we are integrating.
That's why we can do it.
OK, so now, if we have this for any region, d. So, we have a function of x, y, z, t, and we have another function here.
And whenever we integrate them anywhere, we get the same answer.
Well, that must mean they're the same.
Just think of what happens if you just take d to be a tiny little box.
You will get roughly the value of div f at that point times the volume of the box.
Or, you will get roughly the value of du dt at that point times the value of a little box.
So, the values must be the same.
Well, let me actually explain that a tiny bit better.
So, what I get is that one over, let me divide by the volume of D, sorry.
I promise, I'm done in a minute.
Is the same thing as one over volume D of negative du dt, dV.
So, that means the average value, OK, maybe that's the best way of telling it, the average of div f in D is equal to the average of minus partial u partial t in D. And, that's true for any region, D, not just for some regions, but for, really, any region I can think of.
So, the outcome is that actually the divergence of f is equal to minus du dt.
And, that's another way to think about what divergence means.
The divergence, we said, is how much stuff is actually expanding, flowing out.
That's how much we're losing.
And so, now, if you combine this with that, you will get that du dt is minus divergence f, which is plus K del squared u. OK, so you combine this guy with that guy, and you get the diffusion equation.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
The other thing is the last lecture before the break ended a bit abruptly because I ran out of time, so just to summarize what the main point was, I mean probably you figured this out if you looked at the notes.
It is not that important, but anyway.
I just wanted to remind you, just to clarify what happened at the end, we got the diffusion equation from two bits of information.
I mean the unknown, in this partial differential equation, is a function that we call u that corresponds to the concentration of some substance.
And we used a vector field that represents the flow of whatever the substance is whose diffusion we are studying.
And so we got two bits of information, one that came from physics that said the flow goes from high concentration to smaller concentration.
And that told us that the flow is proportional to a negative gradient of a concentration.
And the second piece of information that we got was from the divergence theorem, and that was the one I spent time trying to explain.
And that one told us that the divergence of F is actually negative partial u over partial t. When you combine these two relations together that is how you get the diffusion equation.
Sorry, I should say this is not the statement of a divergence theorem.
This is something that would derive from it with quite a few steps involved.
And so what we got out of that is a diffusion equation because we end up getting that partial u over partial t is minus div F, which is therefore positive k times divergence of grad u, which is what we denoted by del square u, the Laplacian.
So, that is how we got the diffusion equation.
Anyway, I will let you have a look at the notes that were handed out in case you really want to see more.
I just wanted to give the missing part of the last lecture.
Let me just switch gears completely and switch to today's topic, which is line integrals and work in 3D.
That is going to look a lot like what we did in the plane, except, of course, there is a z coordinate.
You will see it doesn't change things much when it comes to computing a line integral.
It changes things quite a bit, however, when it comes to testing whether a field is a gradient field.
That is why we have to be more careful.
Let's start right away with line integrals in space.
Let's say that we have a vector field F with components P, Q and R. We should think of it maybe as representing a force.
And let's say that we have a curve C in space.
Then the work done by the field will be the line integral along C of F dot dr. That is a familiar formula.
And what we do with that formula is also familiar, except now, of course, we have a z coordinate.
We are going to think of vector dr as a space vector with components dx, dy and dz.
When we do the dot product of F with dr that will tell us that we have to integrate Pdx Qdy Rdz.
But it is still a line integral so it is still going to turn into a single integral when you plug in the correct values.
So the method will be exactly the same as in the plane, namely we will find some way to parameterize our curve, x plus x, y z in terms of a single variable, and then we will integrate with respect to that variable.
The way that we evaluate is by parameterizing C and express x, y, z, dx, dy, dz in terms of the parameter.
Let's do an example just to convince you that you actually know how to do this, or at least you should know how to do this.
Let's say that I give you the vector field with components yz, xz and xy.
And let's say that we have a curve given by x equals t^3, y equals t^2, z equals t for t going from zero to one.
The way we will set up the line integral for the work done will be -- Well, sorry.
Before we actually set up the line integral, we need to know how we will express everything in terms of t and dt.
x, y and z, in terms of t, are given here.
We just need to do also dx, dy and dz.
By differentiating you get dx is 3t^2 dt.
That is the derivative of t^3, dy will be 2t dt and dz will just be dt.
And we will evaluate the line integral for work.
That will be the integral of yz dx xz dy xy dz, which will become -- yz is t^3 times dx is 3t^2 dt plus xz is t^4 times dy is 2t dt plus xy is t^5 dt.
That just becomes the integral from, well, I guess t goes from zero to one, actually.
And we are integrating three plus two plus one.
That is 6t^5 dt which, I am sure you know, integrates to t^6, so we will just get one.
It is the same method as usual.
And if you are being given a geometric description of a curve then, of course, you will have to decide for yourself what the best parameter will be.
It might be some time parameter t like here.
It might be one of the coordinates.
Here we could have used z as our parameter because, in fact, this curve is x equals x3 and y equals z2.
And we could also have used maybe some angle.
Well, not here, but if we had been moving on a circle or something like that.
Any questions so far?
No.
OK. Well, because we can do a bit more practice, let's do another one where we do the same vector field F but our curve C will be going from the origin to the point (1,0, 0) along the x-axis.
Let's call that C1.
Then to (1,1, 0).
Let's call that C2 by moving parallel to the y-axis.
And then up to (1,1, 1) parallel to the z-axis, let's call that C3.
I am sure that some of you at least are suspecting what I am getting at here, but let's not spoil it for those who don't see it yet.
OK.
If we want to compute the line integral along this guy then we have to break it into a sum of three terms.
Well, maybe I should call that C', not C, because that is not the same C anymore.
I want to do the sum of the line integrals along C1, C2 and C3.
And, well, if I look at C1 and C2 they take place inside the x, y plane.
In fact, you know that z will be zero and dz will also be zero on both of these.
And if you just look at the formula for line integral in the role of yz dx plus xz dy plus xy dz, well, it looks like if you plug z equals zero and dz equals zero you will just get zero.
These are actually very fast.
Let me write it.
This is going to be zero, this is going to be zero, this is going to zero, and we will get zero.
Now, if we do C3, well, there we might have to do some calculation, but it won't be all that bad.
C3, well, x and y are both equal to one.
And, of course, because they are constant that means dx is zero and dy is zero.
On the other hand, z varies from zero to one.
If I look at the line integral on C3 -- The first two terms, yz, dx and xz dy go away because the dx and dy are zero, so I am just left with xy dz.
But because x and y are one it is just the integral of dz from zero to one, and that will just end up being one.
If add these numbers together, zero plus zero plus one, I get one again.
And, of course, it is not a coincidence because this vector field is a gradient field.
I am sure some of you have already figured out what it is the gradient of.
Otherwise, we will figure it out together.
And so that is why we get the same answer for these two paths going both from the origin to (1,1, 1).
Maybe I should point out, to make it clear, that if you plug t equals zero in up there you will get (0,0, 0).
If you plug t equals one, you will get (1,1, 1).
In fact -- F that we have here happens to be conservative.
And, if you plug the two curves together -- Well, I am not really sure if I know how to plot this correctly.
It is not exactly how it looks.
Whatever.
The first curve C goes from the origin to this point, and so does C ', just in a slightly more roundabout way.
They both go from the origin to (1,1, 1).
It is not a surprise that you will get the same answer for both line integrals.
And how do we see that?
Well, actually here it is not very hard to find a function whose gradient is this vector field.
Namely, the gradient of x, y, z looks like it should be exactly what we want.
If you take partial of this with respect to x, you will get yz, then with respect to y, xz, and with respect to z, xy.
And so, in fact, what was the easier way to compute these line integrals was to use the fundamental theorem of calculus.
Once we have this remark, we don't need to compute these line integrals anymore.
We can just use the fundamental theorem.
If we know this fundamental theorem -- -- for line integrals, that tells us that the line integral of a gradient field is equal to the value of the potential at the final point minus the value of the potential at the starting point.
And that, of course, only applies if you have a potential.
So, in particular, only if you have a conservative field, a gradient field.
Here, in our example, we have to look at, let's call little f of x, y, z that potential xyz, then we take f(1,1, 1) - f(0,0, 0).
And that indeed is one minus zero which is one.
Everything is consistent.
All this stuff so far works exactly as in the plane.
Any questions?
No.
OK. Let's try to see where things do get a little bit different.
And the first such place is when we try to test whether a vector field is a gradient field.
Remember when we had a vector field in the plane, to know whether it was a gradient of a function of two variables we just had to check one condition, N sub x equals M sub y.
Now we actually have three different conditions to check, and that means, of course, more work.
OK.
So what is our test for gradient fields?
We want to know whether a given vector field with components P, Q and R can be written as f sub x, f sub y and f sub z for a same function F. And for that to possibly happen, well, we need certainly some relations between P, Q and R. And, as before, this comes from the fact that the mixed second derivatives are the same, no matter in which order you take them.
If that is the case then I can compute f sub xy, which is the same as f sub yx in two different ways.
F sub xy should be P sub y. F sub yx, well, since f sub y is Q, that should be Q sub x.
That is a part of a criterion that we already had when we had only two variables.
But now, of course, we need to do the same thing when we look at x and z or y and z.
That gives us two more conditions.
P sub z is f sub xz, which is the same as f sub zx, so it should be the same as R sub x.
Finally, Q sub z, which is f sub yz, equals f sub zy equals R sub y.
We have three conditions, so our criterion -- Vector field F equals <P, Q, R>.
And here, to be completely truthful, I have to say defined in a simply connected region.
Otherwise, we might have the same kind of strange things happening as before.
Let's not worry too much about it.
For accuracy we need our vector field to be defined in a simply connected region.
And example is just if it is defined everywhere.
If you don't have any evil eliminators then you can just go ahead and there is no problem.
It is a gradient field.
We need three conditions.
Let's do it in order.
P sub y equals Q sub x.
And we have P sub z equals R sub x and Q sub z equals R sub y.
How do you remember these three conditions?
Well, it is pretty easy.
You pick any two components, say the x and the z component, and you take the partial of the x component with respect to z, the partial of the z component with respect to x and you must make them equal.
And the same with every pair of variables.
In fact, if you had a function of many more variables the criterion would still look exactly like that.
For every pair of components the mixed partials must be the same.
But we are not going to go beyond three variables so you don't need to know that.
This you need to know so let me box it.
That is pretty straightforward.
Let's do an example just to see how it goes.
By the way, we can also think of it in terms of differentials.
Before I do the example, let me just say in a different language.
If we have a differential given to us of a form Pdx Qdy Rdz is going to be an exact differential, which means it is equal to df for some function F exactly and of the same conditions.
That is the same thing.
Just in the language of differentials.
The example that I promised.
Of course, I could do again the same one over there and check that it satisfies the condition, but then it wouldn't be much fun.
So let's do a better one.
Actually, let's do it in a way that looks like an exam problem.
Let's say for which a and b is a xy dx plus -- Oh, it is not going to fit here.
But it will fit here.
a xy dx ( x^2 z^3) dy (byz^2 - 4z^3) dz, an exact differential.
Or, if you don't like exact differentials, for which a and b is the corresponding vector field with i, j and k instead, a gradient field.
Let's just apply the criterion.
And, of course, you can guess that what will follow is figuring out how to find the potential when there is one.
Let's do it one by one.
We want to compare P sub y with Q sub x, we want to compare P sub z with R sub x and we want to compare Q sub z with R sub y where we call P, Q and R these guys.
Let's see.
What is P sub y?
That seems to be ax.
What is Q sub x?
2x.
Q is this one.
Actually, let me write them down.
Because otherwise I am going to get confused myself.
This guy here, that is P, this guy here, that is Q and that guy here, that is R. This one tells us that a should be equal to two of the first product that you hold.
OK. Let's look at P sub z.
That is just zero.
R sub x?
Well, R doesn't have any x either so that is zero.
This one is not a problem.
Q sub z?
Well, that seems to be 3z2.
R sub y seems to be bz2, so b should be equal to three.
We need to have a equals two and, this is an and, not or, b equals three for this to be exact.
For those values of a and b, we can look for a potential using the method that we are going to see right now.
For any other values of a and b we cannot.
If we have to compute a line integral, we have to do it by finding a parameter and setting up everything.
Any questions at this point?
Yes?
I see.
Well, if I got the same answer, oh, did say bz^2 or 3bz^2?
Well, 3bz^2, for example, I would need b to be zero because the only time that 3bz2 equals bz2 as not just at one point but everywhere, I need them to be the same function of x, y, z.
Well, if a coefficient of z2 is the same that would be give b equals 3b, that would give me b equals zero.
If you got bz2 on both sides then it would mean for any value of b it works, and you wouldn't have to worry about what the value of b is.
Any other questions?
No.
OK. Now, how do we find the potential?
Well, there are two methods as before.
One of them, I don't remember if it was the first one or the second one last time, but it really doesn't matter.
One of them was just to say that the value of F at the point, let me call that x1, y1, z1, is equal to the line integral of my field along a well-chosen curve plus, of course, a constant, which is going to be the integration constant.
And the kind of curve that I will take to do this calculation will just be my favorite curve going from the origin to the point x1, y1, z1.
And so, typically the most common choice would be to go just first along the x-axis, then parallel to the y-axis and then parallel to the z-axis all the way to my point x1, y1, z1.
I would just calculate three easy line integrals.
Add them together and that would give me the value of my function.
That method works exactly the same way as it did in two variables.
Now, I seem to recall that you guys mostly preferred the other method.
I am going to tell you about the other method as well, but I just want to point out this one actually doesn't become more complicated.
The other one has actually more steps.
I mean, of course, here there are also a bit more steps because you have three parts to your path instead of two.
You have three line integrals to compute instead of two, but conceptually it remains exactly the same idea.
I should say it works the same way as in 2D.
Not much changes.
Let's look at the other method using anti-derivatives.
Remember we want to find a function little f whose partials are exactly the things we have been given.
We want to solve, well, let me plug in the values of a and b that will work.
We said a should be two, so f sub x should be 2xy, f sub y should be x2 plus z3, and f sub z should be 3yz^2 minus 4z^3.
We are going to look at them one at a time and get partial information on the function.
And then we will compare with the others to get more information until we are completely done.
The first thing we will do, we know that f sub x is 2xy.
That should tell us something about f. Well, let's just integrate that with respect to x.
Let me write integral dx next to that.
That tells us that f should be, well, if we integrate that with respect to x, 2x integrates to x^2, so we should get x2y.
Plus, of course, an integration constant.
Now, what do we mean by integration constant.
It means that for given values of y and z we will get a term that does not depend on x.
It still depends on y and z.
In fact, what we get is a function of y and z.
See, if you took the derivative of this with respect to x you will get 2xy and this guy will go away because there is no x in it.
That is the first step.
Now we need to get some information on g. How do we do that?
Well, we look at the other partials.
F sub y, we want that to be x^2 z^3.
But we have another way to find it, which is starting from this and differentiating.
Let me try to use color for this.
Now, if I take the partial of this with respect to y, I am going to get a different formula for f sub y.
That will be x^2 plus g sub y.
Well, if I compare these two expressions that tells me that g sub y should be z3.
Now, if I have this I can integrate with respect to y.
That will tell me that g is actually yz^3 plus an integration constant.
That constant, again, does not depend on y, but it can still depend on z because we still have not said anything about partial with respect to z.
In fact, that constant I will write as a function h of z.
If I have this function of z and I take its partial with respect to y, I will still get z^3 no matter what h was.
Now, how do I find h?
Well, obviously, I have to look at f sub z. F sub z.
We know from the given vector field that we want it to be 3yz^2 minus 4z^3.
In case you are wondering where that came from, that was R. But that is also obtained by differentiating with respect to z what we had so far.
Sorry.
What did we have so far?
Well, we had f equals x^2y plus g. And we said g is actually yz^3 plus h of z.
That is what we have so far.
If we take the derivative of that with respect to z, we will get zero plus 3yz^2 plus h prime of z, or dh dz as you want.
Now, if we compare these two, we will get the derivative of h. It will tell us that h prime is negative for z3.
That means that h is negative z^4 plus a constant.
And this it is at last an actual constant.
Because it does not depend on z and there is nothing else for it to depend on.
Now we plug this into what we had before, and that will give us our function f. We get that f=x^2y yz^3 - z^4 plus constant.
If you just wanted to find one potential, you can just forget the constant.
This guy was a potential.
If you want all the potentials they differ by this constant.
OK. Just to recap the method what did we do?
We started with -- And, of course, you can do it in whichever order you prefer, but you have to still follow the systematic method.
You start with f sub x and you integrate that with respect to x.
That gives you f up to a function of y and z only.
Now you compare f sub y as given to you by the vector field with the formula you get from this expression for f. And, of course, this one will involve g sub y.
Out of this, you will get the value of g sub y.
When you have g sub y that gives you g up to a function of z only.
And so now you have f up to a function of z only.
And what you will do is look at the derivative with respect to z, the one you want coming from the vector field and the one you have coming from this formula for f, match them and that will tell you h prime.
You will get h and then you will get f. Any questions?
Who still prefers this method?
OK, still most of you.
Who is thinking that maybe the other method was not so bad after all?
OK. That is still a minority.
You can choose whichever one you prefer.
I would encourage you to get some practice by trying both on least a couple of examples just to make sure that you know how to do them both and then stick to whichever one you prefer.
Any questions on that?
No.
I guess I already asked.
Still no questions?
OK.
The next logical thing is going to be curl.
And the theorem that is going to replace Green's theorem for work in this setting is going to be called Stokes' theorem.
Let me start by telling you about curl in 3D.
Here is the statement.
The curl is just going to measure how much your vector field fails to be conservative.
And, if you want to think about it in terms of motions, that also will measure the rotation part of the motion.
Well, let me first give a definition.
Let's say that my vector field has components P, Q and R. Then we define the curl of F to be R sub y minus Q sub z times i plus P sub z minus R sub x times j plus Q sub x minus P sub y times k. And of course nobody can remember this formula, so what is the structure of this formula?
Well, you see, each of these guys is one of the things that have to be zero for our field to be conservative.
If F is defined in a simply connected region then we have that F is conservative and is equivalent to if and only if curl F is zero.
Now, an important difference between curl here and curl in the plane is that now the curl of a vector field is again a vector field.
These expressions are functions of x, y, z and together you form a vector out of them.
The curl of a vector field in space is actually a vector field, not a scalar function.
I have delayed the inevitable.
I have to really tell you how to remember this evil formula.
The secret is that, in fact, you can think of this as del cross f. Maybe you have seen that in physics.
This is really where this del notation becomes extremely useful, because that is basically the only way to remember the formula for curl.
Remember we introduced the dell operator.
That was this symbolic vector operator in which the components are the partial derivative operators.
We have seen that if you apply this to a scalar function then that will give you the gradient.
And we have seen that if you do the dot product between dell and a vector field, maybe I should give it components P, Q and R, you will get partial P over partial x plus partial Q over partial y plus partial R over partial z, which is the divergence.
And so now what is new is that if I try to do dell cross F, well, what is dell cross F?
I have to set up a cross-product between this strange thing that is not really a vector.
I mean, I cannot really think of partial over partial x as a number.
And my vector field <P, Q, R>.
See, that is really a completely perverted use of a determinant notation.
Initially, determinants were just supposed to be you had a three by three table of numbers and you computed a number out of them.
These guys are functions so they count as numbers, but these are vectors and these are partial derivatives.
It doesn't really make much sense, except this notation.
If you try to enter this into a calculator or computer, it will just yell back at you saying are you crazy.
[LAUGHTER] We just use that as a notation to remember what is in there.
Let's try and see how that works.
The component of i in this cross-product, remember that is this smaller determinant, that smaller determinant is partial over partial y of R minus partial over partial z of Q, the coefficient of i.
And that seems to be what I had over there.
If not then I made a mistake.
Minus the next determinant times z.
Remember there is always a minus sign in front of a j component when you do a cross-product.
The other one is partial over partial x R minus partial over partial z of P plus the component of z which is going to be partial over partial x Q minus partial over partial y P. And that is indeed going to be the curl of F. In practice, if you have to compute the curl of a vector field, you know, don't try to remember this formula.
Just set up this cross-product with whatever formulas you have for the components of a field and then compute it.
Don't bother to try to remember the general formula, just remember this.
What is the geometric interpretation of curl, just to finish?
In a way, I will say just curl measures the rotation component in a velocity field.
An exercise that you can do, which is actually pretty easy to check, is say that we have a fluid that is just rotating about the x-axis uniformly.
Your fluid is just rotating like that about the z-axis.
If I take a rotation about the z-axis.
That is given by a velocity field with components at angular velocity omega.
That will be negative omega times y, then omega x and zero.
And the curl of that you can compute, and you will find two omega times k. Concretely, this curl gives you the angular velocity of the rotation, well, with a factor two but that doesn't matter, and the axis of rotation, the direction of the axis of rotation.
It tells you it is rotating about a vertical axis.
And, in general, if you have a complicated motion some of it might be, you know, there is a translation.
And then within that translation there is maybe expansion and rotation and sharing and everything.
And the curl will compute how much rotation is taking place.
It will tell you, say that you have a very small solid, I don't know like a ping pong ball in your flow, and it is just going with the flow, it tells you how it is going to start rotating.
That is what curl measures.
On Thursday we will see Stokes' theorem, which will be the last ingredient before the next exam.
And then on Friday we will review stuff.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Recall that yesterday we saw, no, two days ago we learned about the curl of a vector field in space.
And we said the curl of F is defined by taking a cross product between the symbol dell and the vector F. Concretely, the way we would compute this would be by putting the components of F into this determinant and expanding and then getting a vector with components Ry minus Qz, Pz minus Rx and Qx minus Py.
I think I also tried to explain very quickly what the significance of a curl is.
Just to tell you again very quickly, basically curl measures, if you mention that your vector field measures the velocity in some fluid then the curl measures how much rotation is taking place in that fluid.
Measures the rotation part of a velocity field.
More precisely the direction corresponds to the axis of rotation and the magnitude corresponds to twice the angular velocity.
Just to give you a few quick examples.
If I take a constant vector field, so everything translates at the same speed, then obviously when you take the partial derivatives you will just get a bunch of zeros so the curl will be zero.
If you take a vector field that stretches things, let's say, for example, we are going to stretch things along the x-axis, that would be a vector field that goes parallel to the x direction but maybe, say, x times i.
So that when you are in front of a plane of a blackboard you are moving forward, when you are behind you are moving backwards, things are getting expanded in the x direction.
If you compute the curl, you can check each of these.
Again, they are going to be zero.
There is no curl.
This is not what curl measures.
I mean, actually, what measures expansion, stretching is actually divergence.
If you take the divergence of this field, you would get one plus zero plus zero, it looks like it will be one, so in case you don't remember, I mean divergence precisely measures this stretching effect in your field.
And, on the other hand, if you take something that corresponds to, say, rotation about the z-axis at unit angular velocity -- That means they are going to moving in circles around the z-axis.
One way to write down this field, let's see, the z component is zero because everything is moving horizontally.
And in the x and y directions, if you look at it from above, well, it is just going to be our good old friend the vector field that rotates everything [at unit speed?].
And we have seen the formula for this one many times.
The first component is minus y, the second one is x.
Now, if you compute the curl of this guy, you will get zero, zero, two, two k. And so k is the axis of rotation, two is twice the angular velocity.
And now, of course, you can imagine much more complicated motions where you will have -- For example, if you look at the Charles River very carefully then you will see that water is flowing, generally speaking, towards the ocean.
But, at the same time, there are actually a few eddies in there and with water swirling.
Those are the places where there is actually curl in the flow.
Yes.
I don't know how to turn out the lights a bit, but I'm sure there is a way.
Does this do it?
Is it working?
OK. You're welcome.
Hopefully it is easier to see now.
That was about curl.
Now, why do we care about curl besides this motivation of understanding motions?
One place where it comes up is when we try to understand whether a vector field is conservative.
Remember we have seen that a vector field is conservative if and only if its curl is zero.
That is the situation in which we are allowed to try to look for a potential function and then use the fundamental theorem.
But another place where this comes up, if you remember what we did in the plane, curl also came up when we tried to convert nine integrals into double integrals.
That was Greene's theorem.
Well, it turns out we can do the same thing in space and that is called Stokes' theorem.
What does Stokes' theorem say?
It says that the work done by a vector field along a closed curve can be replaced by a double integral of curl F. Let me write it using the dell notation.
That is curl F. Dot ndS on a suitably chosen surface.
That is a very strange kind of statement.
But actually it is not much more strange than things we have seen before.
I should clarify what this means.
C has to be a closed curve in space.
And S can be any surface bounded by C. For example, what Stokes' theorem tells me is that let us say that I have to compute some line integral on maybe, say, the unit circle in the x, y plane.
Of course I can set a line integral directly and compute it by setting x equals cosine T, y equals sine T, z equals zero.
But maybe sometimes I don't want to do that because my vector field is really complicated.
And instead I will want to reduce things to a surface integral.
Now, I know that you guys are not necessarily fond of computing flux of vector fields for surfaces so maybe you don't really see the point.
But sometimes it is useful.
Sometimes it is also useful backwards because, actually, you have a surface integral that you would like to turn into a line integral.
What Stokes' theorem says is that I can choose my favorite surface whose boundary is this circle.
I could choose, for example, a half sphere if I want or I can choose, let's call that s1, I don't know, a pointy thing, s2.
Probably the most logical one, actually, would be just to choose a disk in the x, y plane.
That would probably be the easiest one to set up for calculating flux.
Anyway, what Stokes' theorem tells me is I can choose any of these surfaces, whichever one I want, and I can compute the flux of curl F through this surface.
Curl F is a new vector field when you have this formula that gives you a vector field you compute its flux through your favorite surface, and you should get the same thing as if you had done the line integral for F. That is the statement.
Now, there is a catch here.
What is the catch?
Well, the catch is we have to figure out what conventions to use because remember when we have a surface there are two possible orientations.
We have to decide which way we will counter flux positively, which way we will counter flux negatively.
And, if we change our choice, then of course the flux will become the opposite.
Well, similarly to define the work, I need to choose which way I am going to run my curve.
If I change which way I go around the curve then my work will become the opposite.
What happens is I have to orient the curve C and the surface S in compatible ways.
We have to figure out what the rule is for how the orientation of S and that of C relate to each other.
What about orientation?
Well, we need the orientations of C and S to be compatible and they have to explain to you what the rule is.
Let me show you a picture.
The rule is if I walk along C with S to my left then the normal vector is pointing up for me.
Let me write that.
If I walk along C, I should say in the positive direction, in the direction that I have chosen to orient C. With S to my left then n is pointing up for me.
Here is the example.
If I am walking on this curve, it looks like the surface is to my left.
And so the normal vector is going towards what is up for me.
Any questions about that?
I see some people using their right hands.
That is also right-handable which I am going to say in just a few moments.
That is another way to remember this.
Before I tell you about the right-handable version, let me just try something.
Actually, I am not happy with this orientation of C and I want to orient my curve C going clockwise on the picture.
So the other orientation.
Then, if I walk on it this way, the surface would be to my right.
You can remember, if it helps you, that if a surface is to your right then the normal vector will go down.
The other way to think about this rule is enough because if you are walking clockwise, well, you can change that to counterclockwise just by walking upside down.
This guy is walking clockwise on C. And while for him, if you look carefully at the picture, the surface is actually to his left when you flip upside down.
Yeah, it is kind of confusing.
But, anyway, maybe it's easier if you actually rotate in the picture.
And now it is getting actually really confusing because his walking upside up with, actually, the surface is to his left.
I mean where he is at here is actually at the front and this is the back, but that is kind of hard to see.
Anyway, whichever method will work best for you.
Perhaps it is easiest to first do it with the other orientation, this one, and this side, if you want the opposite one, then you will just flip everything.
Now, what is the other way of remembering this with the right-hand rule?
First of all, take your right hand, not your left.
Even if your right hand is actually using a pen or something like that in your right hand do this.
And let's take your fingers in order.
First your thumb.
Let's make your thumb go along the object that has only one dimension in there.
That is the curve.
Well, let's look at the top picture up there.
I want my thumb to go along the curve so that is kind of towards the right.
Then I want to make my index finger point towards the surface.
Towards the surface I mean towards the interior of the surface from the curve.
And when I am on the curve I am on the boundary of the surface, so there is a direction along the surface that is the curve and the other one is pointing into the surface.
That one would be pointing kind of to the back slightly up maybe, so like that.
And now your middle finger is going to point in the direction of the normal vector.
That is up, at least if you have the same kind of right hand as I do.
The other way of doing it is using the right-hand rule along C positively.
The index finger towards the interior of S. Sorry, I shouldn't say interior.
I should say tangent to S towards the interior of S. What I mean by that is really the part of S that is not its boundary, so the rest of the surface.
Then the middle finger points parallel to n. Let's practice.
Let's say that I gave you this curve bounding this surface.
Which way do you think the normal vector will be going?
Up.
Yes.
Everyone is voting up.
Imaging that I am walking around C. That is to my left.
Normal vector points up.
Imagine that you put your thumb along C, your index towards S and then your middle finger points up.
Very good.
N points up.
Another one.
It is interesting to watch you guys.
I think mostly it is going up.
The correct answer is it goes up and into the cone.
How do we see that?
Well, one way to think about it is imagine that you are walking on C, on the rim of this cone.
You have two options.
Imagine that you are walking kind of inside or imagine that you are walking kind of outside.
If you are walking outside then S is to your right, but it does not sound good.
Let's say instead that you are walking on the inside of a cone following the boundary.
Well, then the surface is to your left.
And so the normal vector will be up for you which means it will be pointing slightly up and into the cone.
Another way to think about it, through the right-hand rule, from this way index going kind of down because the surface goes down and a bit to the back.
And then the normal vector points up and in.
Yet another way, if you deform continuously your surface then the conventions will not change.
See, this is kind of [UNINTELLIGIBLE] in a way.
You can deform things and nothing will change.
So what if we somehow flatten our cone, push it a bit up so that it becomes completely flat?
Then, if you had a flat disk with the curve going counterclockwise, the normal vector would go up.
Now take your disk with its normal vector sticking up.
If you want to paint the face a different color so that you can remember that was beside with a normal vector and then push it back down to the cone, you will see that the painted face, the one with the normal vector on that side is the one that is inside and up.
Does that make sense?
Anyway, I think you have just to play with these examples for long enough and get it.
OK.
The last one.
Let's say that I have a cylinder.
So now this guy has actually two boundary curves, C and C prime.
And let's say I want to orient my cylinder so that the normal vector sticks out.
How should I choose the orientation of my curves?
Let's start with, say, the bottom one.
Would the bottom one be going clockwise or counterclockwise.
Most people seem to say counterclockwise, and I agree with that.
Let me write that down and claim C prime should go counterclockwise.
One way to think about it, actually, it's quite easy, you mentioned that you're walking on the outside of the cylinder along C prime.
If you want to walk along C prime so that the cylinder is to your left, that means you have to actually go counterclockwise around it.
The other way is use your right hand.
Say when you're at the front of C prime, your thumb points to the right, your index points up because that's where the surface is, and then your middle finger will point out.
What about C?
Well, C I claim we should be doing clockwise.
I mean think about just walking again on the surface of the cylinder along C. If you walk clockwise, you will see that the surface is to your left or use the right-hand rule.
Now, if a problem gives you neither the orientation of a curve nor that of the surface then it's up to you to make them up.
But you have to make them up in a consistent way.
You cannot choose them both at random.
All right.
Now we're all set to try to use Stokes' theorem.
Well, let me do an example first.
The first example that I will do is actually a comparison.
Stokes' versus Green.
I want to show you how Green's theorem for work that we saw in the plane, but also involved work and curl and so on, is actually a special case of this.
Let's say that we will look at the special case where our curve C is actually a curve in the x, y plane.
And let's make it go counterclockwise in the x, y plane because that's what we did for Green's theorem.
Now let's choose a surface bounded by this curve.
Well, as I said, I could make up any surface that comes to my mind.
But, if I want to relate to this stuff, I should probably stay in the x, y plane.
So I am just going to take my surface to be the piece of the x, y plane that is inside my curve.
So let's say S is going to be a portion of x, y plane bounded by a curve C, and the curve C goes counterclockwise.
Well, then I should look at [the table?].
For work along C of my favorite vector field F dot dr.
So that will be the line integral of Pdx plus Qdy.
Like I said, if I call the components of my field P, Q and R, it will be Pdx plus Qdy plus Rdz, but I don't have any Z here.
Dz is zero on C. If I evaluate for line integral, I don't have any term involving dz.
Z is zero.
Now, let's see what Strokes says.
Stokes says instead I can compute for flux through S of curve F. But now what's the normal vector to my surface?
Well, it's going to be either k or negative k. I just have to figure out which one it is.
Well, if you followed what we've done there, you know that the normal vector compatible with this choice for the curve C is the one that points up.
My normal vector is just going to be k hat, so I am going to replace my normal vector by k hat.
That means, actually, I will be integrating curl dot k. That means I am integrating the z component of curl.
Let's look at curl F dot k. That's the z component of curl F. And what's the z component of curl?
Well, I conveniently still have the values up there.
It's Q sub x minus P sub y.
My double integral becomes double integral of Q sub x minus P sub y.
What about dS?
Well, I am in a piece of the x, y plane, so dS is just dxdy or your favorite combination that does the same thing.
Now, see, if you look at this equality, integral of Pdx plus Qdy along a closed curve equals double integral of Qx minus Py dxdy.
That is exactly the statement of Green's theorem.
I mean except at that time we called things m and n, but really that shouldn't matter.
This tells you that, in fact, Green's theorem is just a special case of Stokes' in the x, y plane.
Now, another small remark I want to make right away before I forget, you might think that these rules that we've made up about compatibility of orientations are completely arbitrary.
Well, they are literally in the same way as our convention for which we guy curl is arbitrary.
We chose to make the curl be this thing and not the opposite which would have been pretty much just as sensible.
And, ultimately, that comes from our choice of making the cross-product be what it is but of the opposite.
Ultimately, it all comes from our preference for right-handed coordinate systems.
If we had been on the planet with left-handed coordinate systems then actually our conventions would be all the other way around, but they are this way.
Any other questions?
A surface that you use in Stokes' theorem is usually not going to be closed because its boundary needs to be the curve C. So if you had a closed surface you wouldn't know where to put your curve.
I mean of course you could make a tiny hole in it and get a tiny curve.
Actually, what that would say, and we are going to see more about that so not very important right now, but what we would see is that for a close surface we would end up getting zero for the flux.
And that is actually because divergence of curl is zero, but I am getting ahead of myself.
We are going to see that probably tomorrow in more detail.
Stokes' theorem only works if you can make sense of this.
That means you need your vector field to be continuous and differentiable everywhere on the surface S. Now, why is that relevant?
Well, say that your vector field was not defined at the origin and say that you wanted to do, you know, the example that I had first with the unit circling the x, y plane.
Normally, the most sensible choice of surface to apply Stokes' theorem to would be just the flat disk in the x, y plane.
But that assumes that your vector field is well-defined there.
If your vector field is not defined at the origin but defined everywhere else you cannot use this guy, but maybe you can still use, say, the half-sphere, for example.
Or, you could use a piece of cylinder plus a flat top or whatever you want but not pressing for the origin.
So you could still use Stokes but you'd have to be careful about which surface you choose.
Now, if instead your vector field is not defined anywhere on the z-axis then you're out of luck because there is no way to find a surface bounded by this unit circle without crossing the z-axis somewhere.
Then you wouldn't be able to Stokes' theorem at all or at least not directly.
Maybe I should write it F defines a differentiable everywhere on this.
But we don't care about what happens outside of this.
It's really only on the surface that we need it to be OK.
I mean, again, 99% of the vector fields that we see in this class are defined everywhere so that's not an urgent concern, but still.
OK. Should we move on?
Yes.
I have a yes.
Let me explain to you quickly why Stokes is true.
How do we prove a theorem like that?
Well, the strategy, I mean there are other ways, but the least painful strategy at this point is to observe what we already know is a special case of Stokes's theorem.
Namely we know the case where the curve is actually in the x, y plane and the surface is a flat piece of the x, y plane because that's Green's theorem which we proved a while ago.
We know it for C and S in the x, y plane.
Now, what if C and S were, say, in the y, z plane instead of the x, y plane?
Well, then it will not quite give the same picture because the normal vector would be i hat instead of k hat and they would be having different notations and it would be integrating with y and z.
But you see that it would become, again, exactly the same formula.
We'd know it for any of the coordinate planes.
In fact, I claim we know it for absolutely any plane.
And the reason for that is, sure, when we write it in coordinates, when we write that this line integral is integral of Pdx plus Qdy plus Rdz or when we write that the curl is given by this formula we use the x, y, z coordinate system.
But there is something I haven't quite told you about.
Which is if I switch to any other right-handed coordinate system, so I do some sort of rotation of my space coordinates, then somehow the line integral, the flux integral, the notion of curl makes sense in coordinates.
And the reason is that they all have geometric interpretations.
For example, when I think of this as the work done by a force, well, the force doesn't care whether it's being put in x, y coordinates this way or that way.
It still does the same work because it's the same force.
And when I say that the curl measures the rotation in a motion, well, that depends on which coordinates you use.
And the same for interpretation of flux.
In fact, if I rotated my coordinates to fit with any other plane, I could still do the same things.
What I'm trying to say is, in fact, if C and S are in any plane then we can still claim that it reduces to Green's theorem.
It will be Green's theorem not in x, y, z coordinates but in some funny rotated coordinate systems.
What I'm saying is that work, flux and curl makes sense independently of coordinates.
Now, this has to stop somewhere.
I can start claiming that I can somehow bend my coordinates to a plane, any surface is flat.
That doesn't really work.
But what I can say is if I have any surface I can cut it into tiny pieces.
And these tiny pieces are basically flat.
So that's basically the idea of a proof.
I am going to decompose my surface into very small flat pieces.
Given any S we are just going to decompose it into tiny almost flat pieces.
For example, if I have my surface like this, what I will do is I will just cut it into tiles.
I mean a good example of that is if you look at [UNINTELLIGIBLE], for example, it's made of all these hexagons and pentagons.
Well, actually, they're not quite flat in the usual rule, but you could make them flat and it would still look pretty much like a sphere.
Anyway, you're going to cut your surface into lots of tiny pieces.
And then you can use Stokes' theorem on each small piece.
What it says on each small flat piece -- It says that the line integral along say, for example, this curve is equal to the flux of a curl through this tiny piece of surface.
And now I will add all of these terms together.
If I add all of the small contributions to flux I get the total flux.
What if I add all of the small line integrals?
Well, I get lots of extra junk because I never asked to compute the line integral along this.
But this guy will come in twice when I do this little plate and when I do that little plate with opposite orientations.
When I sum all of the little line integrals together, all of the inner things cancel out, and the only ones that I go through only once are those that are at the outer most edges.
So, when I sum all of my works together, I will get the work done just along the outer boundary C. Sum of work around each little piece is just actually the work along C, the outer curve.
And the sum of the flux for each piece is going to be the flux through S. From Stokes' theorem for flat surfaces, I can get it for any surface.
I am cheating a little bit because you would actually have to check carefully that this approximately where you flatten the little pieces that are almost flat is [UNINTELLIGIBLE].
But, trust me, it actually works.
Let's do an actual example.
I mean I said example, but that was more like getting us ready for the proof so probably that doesn't count as an actual example.
I should probably keep these statements for now so I am not going to erase this side.
Let's do an example.
Let's try to find the work of vector field zi plus xj plus yk around the unit circle in the x, y plane counterclockwise.
The picture is conveniently already there.
Just as a quick review, let's see how we do that directly.
If we do that directly, I have to find the integral along C. So F dot dr becomes zdx plus xdy plus ydz.
But now we actually know that on this circle, well, z is zero.
And we can parameterize x and y, the unit circle in the x, y plane, so we can take x equals cosine t, y equals sine t. That will just become the integral over C. Well, z times dx, z is zero so we have nothing, plus x is cosine t times dy is -- Well, if y is sine t then dy is cosine tdt plus ydz but z is zero.
Now, the range of values for t, well, we are going counterclockwise around the entire circle so that should go from zero to 2pi.
We will get integral from zero to 2pi of cosine square tdt which, if you do the calculation, turns out to be just pi.
Now, let's instead try to use Stokes' theorem to do the calculation.
Now, of course the smart choice would be to just take the flat unit disk.
I am not going to do that.
That would be too boring.
Plus we have already kind of checked it because we already trust Green's theorem.
Instead, just to convince you that, yes, I can choose really any surface I want, let's say that I'm going to choose a piece of paraboloid z equals one minus x squared minus y squared.
Well, to get our conventions straight, we should take the normal vector pointing up for compatibility with our choice.
Well, we will have to compute the flux through S. We don't really have to because we could have chosen the disk, it would be easier, but if we want to do it this way we will compute the flux of curl F through our paraboloid.
How do we do that?
Well, we need to find the curl and we need to find ndS.
Let's start with the curl.
Curl F let's take the cross-product between dell and F which is zxy.
If we compute this, the i component will be one minus zero.
It looks like it is one i.
Minus the j component is zero minus one.
Plus the k component is one minus zero.
In fact, the curl of the field is one, one, one.
Now, what about ndS?
Well, this is a surface for which we know z is a function of x and y. ndS we can write as, let's call this F of xy, then we can use the formula that says ndS equals negative F sub x, negative F sub y, one dxdy, which here gives us 2x, 2y, one dxdy.
Now, when we want to compute the flux, we will have to do double integral over S of one, one, one dot product with 2x, 2y, one dxdy.
It will become the double integral of 2x plus 2y plus one dxdy.
And, of course, the region which we are integrating, the range of values of x and y will be the shadow of our surface.
That is just going to be, if you look at this paraboloid from above, all you will see is the unit disk so it will be a double integral of the unit disk.
And the way we will do that, one way is to switch to polar coordinates and do the calculation and then you will end up with pi.
The other way is to try to do it by symmetry.
Observe, when you integrate x above this, x is as negative on the left as it is positive on the right.
So the integral of x will be zero.
The integral of y will be zero also by symmetry.
Then the integral of one dxdy will just be the area of this unit disk which is pi.
That was our first example.
And, of course, if you're actually free to choose your favorite surface, there is absolutely no reason why you would actually choose this paraboloid in this example.
I mean it would be much easier to choose a flat disk.
OK.
Tomorrow I will tell you a few more things about curl fits in with conservativeness and with the divergence theorem, Stokes all together, and we will look at Practice Exam 4B so please bring the exam with you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so remember, we've seen Stokes theorem, which says if I have a closed curve bounding some surface, S, and I orient the curve and the surface compatible with each other, then I can compute the line integral along C along my curve in terms of, instead, surface integral for flux of a different vector field, namely, curl f dot n dS.
OK, so that's the statement.
And, just to clarify a little bit, so, again, we've seen various kinds of integrals.
So, line integrals we know how to evaluate.
They take place in a curve.
You express everything in terms of one variable, and after substituting, you end up with a usual one variable integral that you know how to evaluate.
And, surface integrals, we know also how to evaluate.
Namely, we've seen various formulas for ndS.
Once you have such a formula, due to the dot product with this vector field, which is not the same as that one.
But it's a new vector field that you can build out of f. You do the dot product.
You express everything in terms of your two integration variables, and then you evaluate.
So, now, what does this have to do with various other things?
So, one thing I want to say has to do with how Stokes helps us understand path independence, so, how it actually motivates our criterion for gradient fields, independence.
OK, so, we've seen that if we have a vector field defined in a simply connected region, and its curl is zero, then it's a gradient field, and the line integral is path independent.
So, let me first define for you when a simply connected region is.
So, we say that a region in space is simply connected -- -- if every closed loop inside this region bounds some surface again inside this region.
OK, so let me just give you some examples just to clarify.
So, for example, let's say that I have a region that's the entire space with the origin removed.
OK, so space with the origin removed, OK, you think it's simply connected?
Who thinks it's simply connected?
Who thinks it's not simply connected?
Let's think a little bit harder.
Let's say that I take a loop like this one, OK, it doesn't go through the origin.
Can I find a surface that's bounded by this loop and that does not pass through the origin?
Yeah, I can take the sphere, you know, for example, or anything that's just not quite the disk?
So, and similarly, if I take any other loop that avoids the origin, I can find, actually, a surface bounded by it that does not pass through the origin.
So, actually, that's kind of a not so obvious theorem to prove, but maybe intuitively, start by finding any surface.
Well, if that surface passes through the origin, just wiggle it a little bit, you can make sure it doesn't pass through the origin anymore.
Just push it a little bit.
So, in fact, this is simply connected.
That was a trick question.
OK, now on the other hand, a good example of something that is not simply connected is if I take space, and I remove the z axis -- -- that is not simply connected.
And, see, the reason is, if I look again, say, at the unit circle in the x axis, sorry, unit circle in the xy plane, I mean, in the xy plane, so, if I try to find a surface whose boundary is this disk, well, it has to actually cross the z axis somewhere.
There's no way that I can find a surface whose only boundary is this curve, which doesn't hit the z axis anywhere.
Of course, you could try to use the same trick as there, say, maybe we want to go up, up, up.
You know, let's start with a cylinder.
Well, the problem is you have to go infinitely far because the z axis goes infinitely far.
And, you'll never be able to actually close your surface.
So, the matter what kind of trick you might want to use, it's actually a theorem in topology that you cannot find a surface bounded by this disk without intersecting the z axis.
Yes?
Well, a doughnut shape certainly would stay away from the z axis, but it wouldn't be a surface with boundary just this guy.
Right, it would have to have either some other boundary.
So, maybe what you have in mind is some sort of doughnut shape like this that curves on itself, and maybe comes back.
Well, if you don't quite close it all the way around, so I can try to, indeed, draw some sort of doughnut here.
Well, if I don't quite close it, that it will have another edge at the other end wherever I started.
If I close it completely, then this curve is no longer its boundary because my surface lives on both sides of this curve.
See, I want a surface that stops on this curve, and doesn't go beyond it.
And, nowhere else does it have that kind of behavior.
Everywhere else, it keeps going on.
So, actually, I mean, maybe actually another way to convince yourself is to find a counter example to the statement I'm going to make about vector fields with curl zero and simply connected regions always being conservative.
So, what you can do is you can take the example that we had in one of our older problem sets.
That was a vector field in the plane.
But, you can also use it to define a vector field in space just with no z component.
That vector field is actually defined everywhere except on the z axis, and it violates the usual theorem that we would expect.
So, that's one way to check just for sure that this thing is not simply connected.
So, what's the statement I want to make?
So, recall we've seen if F is a gradient field -- -- then its curl is zero.
That's just the fact that the mixed second partial derivatives are equal.
So, now, the converse is the following theorem.
It says if the curl of F equals zero in, sorry, and F is defined -- No, is not the logical in which to say it.
So, if F is defined in a simply connected region, and curl F is zero -- -- then F is a gradient field, and the line integral for F is path independent -- -- F is conservative, and so on, all the usual consequences.
Remember, these are all equivalent to each other, for example, because you can use path independence to define the potential by doing the line integral of F. OK, so where do we use the assumption of being defined in a simply connected region?
Well, the way which we will prove this is to use Stokes theorem.
OK, so the proof, so just going to prove that the line integral is path independent; the others work the same way.
OK, so let's assume that we have a vector field whose curl is zero.
And, let's say that we have two curves, C1 and C2, that go from some point P0 to some point P1, the same point to the same point.
Well, we'd like to understand the line integral along C1, say, minus the line integral along C2 to show that this is zero.
That's what we are trying to prove.
So, how will we compute that?
Well, the line integral along C1 minus C2, well, let's just form a closed curve that is C1 minus C2.
OK, so let's call C, woops -- So that's equal to the integral along C of f dot dr where C is C1 followed by C2 backwards.
Now, C is a closed curve.
So, I can use Stokes theorem.
Well, to be able to use Stokes theorem, I need, actually, to find a surface to apply it to.
And, that's where the assumption of simply connected is useful.
I know in advance that any closed curve, so, C in particular, has to bound some surface.
OK, so we can find S, a surface, S, that bounds C because the region is simply connected.
So, now that tells us we can actually apply Stokes theorem, except it won't fit here.
So, instead, I will do that on the next line.
That's equal by Stokes to the double integral over S of curl F dot vector dS, or ndS.
But now, the curl is zero.
So, if I integrate zero, I will get zero.
OK, so I proved that my two line integrals along C1 and C2 are equal.
But for that, I needed to be able to find a surface which to apply Stokes theorem.
And that required my region to be simply connected.
If I had a vector field that was defined only outside of the z axis and I took two paths that went on one side and the other side of the z axis, I might have obtained, actually, different values of the line integral.
OK, so anyway, that's the customary warning about simply connected things.
OK, let me just mention very quickly that there's a lot of interesting topology you can do, actually in space.
So, for example, this concept of being simply connected or not, and studying which loops bound surfaces or not can be used to classify shapes of things inside space.
So, for example, one of the founding achievements of topology in the 19th century was to classify surfaces in space -- -- by trying to look at loops on them.
So, what I mean by that is that if I take the surface of a sphere, well, I claim the surface of a sphere -- -- is simply connected.
Why is that?
Well, let's take my favorite closed curve on the surface of a sphere.
I can always find a portion of the sphere that's bounded by it.
OK, so that's the definition of the surface of a sphere being simply connected.
On the other hand, if I take what's called a torus, or if you prefer, the surface of a doughnut, that's more, it's a less technical term, but it's -- -- well, that's not simply connected.
And, in fact, for example, if you look at this loop here that goes around it, well, of course it bounds a surface in space.
But, that surface cannot be made to be just a piece of the donut.
You have to go through the hole.
You have to leave the surface of a torus.
In fact, there's another one.
See, this one also does not bound anything that's completely contained in the torus.
And, of course, it bounds this disc, but inside of a torus.
But, that's not a part of the surface itself.
So, in fact, there's, and topologists would say, there's two independent -- -- loops that don't bound surfaces, that don't bound anything.
And, so this number two is somehow an invariant that you can associate to this kind of shape.
And then, if you consider more complicated surfaces with more holes in them, you can try, somehow, to count independent loops on them, and that's the beginning of the classification of surfaces.
Anyway, that's not really an 18.02 topic, but I thought I would mentioned it because it's kind of a cool idea.
OK, let me say a bit more in the way of fun remarks like that.
So, food for thought: let's say that I want to apply Stokes theorem to simplify a line integral along the curve here.
So, this curve is maybe not easy to see in the picture.
It kind of goes twice around the z axis, but spirals up and then down.
OK, so one way to find a surface that's bounded by this curve is to take what's called the Mobius strip.
OK, so the Mobius strip, it's a one sided strip where when you go around, you flip one side becomes the other.
So, you just, if you want to take a band of paper and glue the two sides with a twist, so, it's a one sided surface.
And, that gives us, actually, serious trouble if we try to orient it to apply Stokes theorem.
So, see, for example, if I take this Mobius strip, and I try to find an orientation, so here it looks like that, well, let's say that I've oriented my curve going in this direction.
So, I go around, around, around, still going this direction.
Well, the orientation I should have for Stokes theorem is that when I, so, curve continues here.
Well, if you look at the convention around here, it tells us that the normal vector should be going this way.
OK, if we look at it near here, if we walk along this way, the surface is to our right .
So, we should actually be flipping things upside down.
The normal vector should be going down.
And, in fact, if you try to follow your normal vector that's pointing up, it's pointing up, up, up.
It will have to go into things, into, into, down.
There's no way to choose consistently a normal vector for the Mobius strip.
So, that's what we call a non-orientable surface.
And, that just means it has only one side.
And, if it has only one side, that we cannot speak of flux for it because we have no way of saying that we'll be counting things positively one way, negatively the other way, because there's only one, you know, there's no notion of sides.
So, you can't define a side towards which things will be going positively.
So, that's actually a situation where flux cannot be defined.
OK, so as much as Mobius strips and climb-bottles are exciting and really cool, well, we can't use them in this class because we can't define flux through them.
So, if we really wanted to apply Stokes theorem, because I've been telling you that space is simply connected, and I will always be able to apply Stokes theorem to any curve, what would I do?
Well, I claim this curve actually bounds another surface that is orientable.
Yeah, that looks counterintuitive.
Well, let's see it.
I claim you can take a hemisphere, and you can take a small thing and twist it around.
So, in case you don't believe me, let me do it again with the transparency.
Here's my loop, and see, well, the scale is not exactly the same.
So, it doesn't quite match.
But, and it's getting a bit dark.
But, that spherical thing with a little slit going twisting into it will actually have boundary my loop.
And, that one is orientable.
I mean, I leave it up to you to stare at the picture long enough to convince yourselves that there's a well-defined up and down.
OK.
So now, I mean, in case you are getting really, really worried, I mean, there won't be any Mobius strips on the exam on Tuesday, OK?
It's just to show you some cool stuff.
OK, questions?
No?
OK, one last thing I want to show you before we start reviewing, so one question you might have about Stokes theorem is, how come we can choose whatever surface we want?
I mean, sure, it seems to work, but why?
So, I'm going to say a couple of words about surface independence in Stokes theorem.
So, let's say that I have a curve, C, in space.
And, let's say that I want to apply Stokes theorem.
So, then I can choose my favorite surface bounded by C. So, in a situation like this, for example, I might want to make my first choice be this guy, S1, like maybe some sort of upper half sphere.
And, if you pay attention to the orientation conventions, you'll see that you need to take it with normal vector pointing up.
Maybe actually I would rather make a different choice.
And actually, I will choose another surface, S2, that maybe looks like that.
And, if I look carefully at the orientation convention, Stokes theorem tells me that I have to take the normal vector pointing up again.
So, that's actually into things.
So, Stokes says that the line integral along C of my favorite vector field can be computed either as a flux integral for the curl through S1, or as the same integral, but through S2 instead of S1.
So, that seems to suggest that curl F has some sort of surface independence property.
It doesn't really matter which surface I take, as long as the boundary is this given curve, C. Why is that?
That's a strange property to have.
Where does it come from?
Well, let's think about it for a second.
So, why are these the same?
I mean, of course, they have to be the same because that's what Stokes tell us.
But, why is that OK?
Well, let's think about comparing the flux integral for S1 and the flux integral for S2.
So, if we want to compare them, we should probably subtract them from each other.
OK, so let's do the flux integral for S1 minus the flux integral for S2 of the same thing.
Well, let's give a name.
Let's call S the surface S1 minus S2.
So, what is S?
S is S1 with its given orientation together with S2 with the reversed orientation.
So, S is actually this whole closed surface here.
And, the normal vector to S seems to be pointing outwards everywhere.
OK, so now, if we have a closed surface with a normal vector pointing outwards, and we want to find a flux integral for it, well, we can replace that with a triple integral.
So, that's the divergence theorem.
So, that's by the divergence theorem using the fact that S is a closed surface.
That's equal to the triple integral over the region inside.
Let me call that region D of divergence, of curl F dV.
OK, and what I'm going to claim now is that we can actually check that if you take the divergence of the curl of a vector field, you always get zero.
OK, and so that will tell you that this integral will always be zero.
And that's why the flux for S1, and the flux for S2 were the same a priori and we didn't have to worry about which one we chose when we did Stokes theorem.
OK, so let's just check quickly that divergence of a curve is zero.
OK, in case you're wondering why I'm doing all this, well, first I think it's kind of interesting, and second, it reminds you of a statement of all these theorems, and all these definitions.
So, in a way, we are already reviewing.
OK, so let's see.
If my vector field has components P, Q, and R, remember that the curl was defined by this cross product between del and our given vector field.
So, that's Ry - Qz followed by Pz - Rx, and Qx - Py.
So, now, we want to take the divergence of this.
Well, so we have to take the first component, Ry minus Qz, and take its partial with respect to x.
Then, take the y component, Pz minus Rx partial with respect to y plus Qx minus Py partial with respect to z.
And, well, now we should expand this.
But I claim it will always simplify to zero.
OK, so I think we have over there, becomes R sub yx minus Q sub zx plus P sub zy minus R sub xy plus Q sub xz minus P sub yz.
Well, let's see.
We have P sub zy minus P sub yz.
These two cancel out.
We have R sub yx minus R sub xy.
These cancel out.
Q sub zx and Q sub xz, these two also cancel out.
So, indeed, the divergence of a curl is always zero.
OK, so the claim is divergence of curl is always zero.
Del cross F is always zero, and just a small remark, if we had actually real vectors rather than this strange del guy, indeed we know that if we have two vectors, U and V, and we do u dot u cross v, what is that?
Well, one way to say it is it's the determinant of u, u, and v, which is the volume of the box.
But, it's completely flat because u, u, and v are all in the plane defined by u and v. The other way to say it is that u cross v is perpendicular to u and v. Well, if it's perpendicular u, then its dot product with u will be zero.
So, no matter how you say it, this is always zero.
So, in a way, this reinforces our intuition that del, even though it's not at all an actual vector sometimes can be manipulated in the same way.
OK, I think that's it for new topics for today.
And, so, now I should maybe try to recap quickly what we've learned in these past three weeks so that you know, so, the exam is probably going to be similar in difficulty to the practice exams.
That's my goal.
I don't know if I will have reached that goal or not.
We'll only know that after you've taken the test.
But, the idea is it's meant to be more or less the same level of difficulty.
So, at this point, we've learned about three kinds of beasts in space.
OK, so I'm going to divide my blackboard into three pieces, and here I will write triple integrals.
We've learned about double integrals, and we've learned about line integrals.
OK, so triple integrals over a region in space, we integrate a scalar quantity, dV.
How do we do that?
Well, we can do that in rectangular coordinates where dV becomes something like, maybe, dz dx dy, or any permutation of these.
We've seen how to do it also in cylindrical coordinates where dV is maybe dz times r dr d theta or more commonly r dr d theta dz.
But, what I want to emphasize in this way is that both of these you set up pretty much in the same way.
So, remember, the main trick here is to find the bounds of integration.
So, when you do it, say, with dz first, that means for fixed xy, so, for a fixed point in the xy plane, you have to look at the bounds for z.
So, that means you have to figure out what's the bottom surface of your solid, and what's the top surface of your solid?
And, you have to find the value of z at the bottom, the value of z at the top as functions of x and y.
And then, you will put that as bounds for z.
Once you've done that, you are left with the question of finding bounds for x and y.
Well, for that, you just rotate the picture, look at your solid from above, so, look at its projection to the xy plane, and you set up a double integral either in rectangular xy coordinates, or in polar coordinates for x and y.
Of course, you can always do it a different orders.
And, I'll let you figure out again how that goes.
But, if you do dz first, then the inner bounds are given by bottom and top, and the outer ones are given by looking at the shadow of the region.
Now, there's also spherical coordinates.
And there, we've seen that dV is rho squared sine phi d rho d phi d theta.
So now, of course, if this orgy of Greek letters is confusing you at this point, then you probably need to first review spherical coordinates for themselves.
Remember that rho is the distance from the origin.
Phi is the angle down from the z axis.
So, it's zero, and the positive z axis, pi over two in the xy plane, and increases all the way to pi on the negative z axis.
And, theta is the angle around the z axis.
So, now, when we set up bounds here, it will look a lot like what you've done in polar coordinates in the plane because when you look at the inner bound down on rho, for a fixed phi and theta, that means you're shooting a straight ray from the origin in some direction in space.
So, you know, you're sending a laser beam, and you want to know what part of your beam is going to be in your given solid.
You want to solve for the value of rho when you enter the solid and when you leave it.
I mean, very often, if the origin is in your solid, then rho will start at zero.
Then you want to know when you exit.
And, I mean, there's a fairly small list of kinds of surfaces that we've seen how to set up in spherical coordinates.
So, if you're really upset by this, go over the problems in the notes.
That will give you a good idea of what kinds of things we've seen in spherical coordinates.
OK, and then evaluation is the usual way.
Questions about this?
No?
OK, so, I should say we can do something bad, but so we've seen, of course, applications of this.
So, we should know how to use a triple integral to evaluate things like a mass of a solid, the average value of a function, the moment of inertia about one of the coordinate axes, or the gravitational attraction on a mass at the origin.
OK, so these are just formulas to remember for examples of triple integrals.
It doesn't change conceptually.
You always set them up and evaluate them the same way.
It just tells you what to put there for the integrand.
Now, double integrals: so, when we have a surface in space, well, what we will integrate on it, at least what we've seen how to integrate is a vector field dotted with the unit normal vector times the area element.
OK, and this is sometimes called vector dS.
Now, how do we evaluate that?
Well, we've seen formulas for ndS in various settings.
And, once you have a formula for ndS, that will relate ndS to maybe dx dy, or something else.
And then, you will express, so, for example, ndS equals something dx dy.
And then, it becomes a double integral of something dx dy.
Now, in the integrand, you want to express everything in terms of x and y.
So, if you had a z, maybe you have a formula for z in terms of x and y.
And, when you set up the bounds, well, you try to figure out what are the bounds for x and y?
That would be just looking at it from above.
Of course, if you are using other variables, figure out the bounds for those variables.
And, when you've done that, it becomes just a double integral in the usual sense.
OK, so maybe I should be a bit more explicit about formulas because there have been a lot.
So, let me tell you about a few of them.
Let me actually do that over here because I don't want to make this too crowded.
OK, so what kinds of formulas for ndS have we seen?
Well, we've seen a formula, for example, for a horizontal plane, or for something that's parallel to the yz plane or the xz plane.
Well, let's do just the yz plane for a quick reminder.
So, if I have a surface that's contained inside the yz plane, then obviously I will express ds in terms of, well, I will use y and z as my variables.
So, I will say that ds is dy dz, or dz dy, whatever's most convenient.
Maybe we will even switch to polar coordinates after that if a problem wants us to.
And, what about the normal vector?
Well, the normal vector is either coming straight at us, or it's maybe going back away from us depending on which orientation we've chosen.
So, this gives us ndS.
We dot our favorite vector field with it.
We integrate, and we get the answer.
OK, we've seen about spheres and cylinders centered at the origin or centered on the z axis.
So, the normal vector sticks straight out or straight in, depending on which direction you do it in.
So, for a sphere, the normal vector is <x, y, z> divided by the radius of the sphere.
For a cylinder, it's <x, y, 0>, divided by the radius of a cylinder.
And, the surface element on a sphere, so, see, it's very closely related to the volume element of spherical coordinates except you don't have a rho anymore.
You just plug in a rho equals a.
So, you get a squared sine phi d phi d theta.
And, for a cylinder, it would be a dz d theta.
So, by the way, just a quick check, when you're doing an integral, if it's the surface integral, there should be two integral signs, and there should be two integration variables.
And, there should be two d somethings.
If you end up with a dx, dy, dz in the surface integral, something is seriously wrong.
OK, now, besides these specific formulas, we've seen two general formulas that are also useful.
So, one is, if we know how to express z in terms of x and y, and just to change notation to show you that it's not set in stone, let's say that z is known as a function z of x and y.
So, how do I get ndS in that case?
Well, we've seen a formula that says negative partial z partial x, negative partial z partial y, one dx dy.
So, this formula relates the volume, sorry, the surface element on our surface to the area element in the xy plane.
It lets us convert between dS and dx dy.
OK, so we just plug in this, and we dot with F, and then we substitute everything in terms of x and y, and we evaluate the integral over x and y.
If we don't really want to find a way to find z as a function of x and y, but we have a normal vector given to us, then we have another formula which says that ndS is, sorry, I should have said it's always up to sign because we have a two orientation convention.
We have to decide based on what we are trying to do, whether we are doing the correct convention or the wrong one.
So, the other formula is n over n dot k dx dy.
Sorry, are they all the same?
Well, if you want, you can put an absolute value here.
But, it doesn't matter because it's up to sign anyway.
So, I mean, this formula is valid as it is.
OK, and, I mean, if you're in a situation where you can apply more than one formula, they will all give you the same answer in the end because it's the same flux integral.
OK, so anyway, so we have various ways of computing surface integrals, and probably one of the best possible things you can do to prepare for the test is actually to look again at some practice problems from the notes that do flux integrals over various kinds of surfaces because that's probably one of the hardest topics in this unit of the class.
OK, anyway, let's move on to line integrals.
So, those are actually a piece of cake in comparison, OK, because all that this is, is just integral of P dx Q dy R dz.
And, then all you have to do is parameterize the curve, C, to express everything in terms of a single variable.
And then, you end up with a usual single integral, and you can just compute it.
So, that one works pretty much as it did in the plane.
So, if you forgotten what we did in the plane, it's really the same thing.
OK, so now we have three different kinds of integrals, and really, well, they certainly have in common that they integrate things somehow.
But, apart from that, they are extremely different in what they do.
I mean, this one involves a function, a scalar quantity.
These involve vector quantities.
They don't involve the same kinds of shapes over which to integrate.
Here, you integrate over a three-dimensional region.
Here, you integrate only over a two-dimensional surface, and here, only a one-dimensional curve.
So, try not to confuse them.
That's basically the most important advice.
Don't get mistaken.
Each of them has a different way of getting evaluated.
Eventually, they will all give you numbers, but through different processes.
So now, well, I said these guys are completely different.
Well, they are, but we still have some bridges between them.
OK, so we have two, maybe I should say three, well, two bridges between these guys.
OK, so we have somehow a connection between these which is the divergence theorem.
We have a connection between that, which is Stokes theorem.
So -- Just to write them again, so the divergence theorem says if I have a region in space, and I call its boundary S, so, it's going to be a closed surface, and I orient S with a normal vector pointing outwards, then whenever I have a surface integral over S, sorry, I can replace it by a triple integral over the region inside.
OK, so this guy is a vector field.
And, this guy is a function that somehow relates to the vector field.
I mean, you should know how.
You should know the definition of divergence, of course.
But, what I want to point out is if you have to compute the two sides separately, well, this is just, you know, your standard flux integral.
This is just your standard triple integral over a region in space.
Once you have computed what this guy is, it's really just a triple integral of the function.
So, the way in which you compute it doesn't see that it came from a divergence.
It's just the same way that you would compute any other triple integral.
The way we compute it doesn't depend on what actually we are integrating.
Stokes theorem says if I have a curve that's the boundary of a surface, S, and I orient the two in compatible manners, then I can replace a line integral on C by a surface integral on S. OK, and that surface integral, well, it's not for the same vector field.
This relates a line integral for one field to a surface integral from another field.
That other field is given from the first one just by taking its curl So, after you take the curl, you obtain a different vector field.
And, the way in which you would compute the surface integral is just as with any surface integral.
You just find a formula for ndS dot product, substitute, evaluate.
The calculation of this thing, once you've computed curl does not remember that it was a curl.
It's the same as with any other flux integral.
OK, and finally, the last bridge, so this was between two and three.
This was between one and two.
Let me just say, there's a bridge between zero and one, which is that if you have a function in its gradient, well, the fundamental theorem of calculus says that the line integral for the vector field given by the gradient of a function is actually equal to the change in value of a function.
That's if you have a curve bounded by P0 and P1.
So in a way, actually, each of these three theorems relates a quantity with a certain number of integral signs to a quantity with one more integral sign.
And, that's actually somehow a fundamental similarity between them.
But maybe it's easier to think of them as completely different stories.
So now, with this one, we additionally have to remember another topic is given a vector field, F, with curl equal to zero, find the potential.
And, we've seen two methods for that, and I'm sure you remember them.
So, if not, then try to remember them for Tuesday.
OK, so anyway, again, conceptually, we have, really, three different kinds of integrals.
We evaluated them in completely different ways, and we have a handful of theorems, connecting them to each other.
But, that doesn't have any impact on how we actually compute things.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Let's try to discuss a bit how things relate to physics.
There are two main things I want to discuss.
One of them is what curl says about force fields and, in particular,a nice consequence of that concerning gravitational attraction.
More about curl.
If we have a velocity field, then we have seen that the curl measures the rotation affects.
More precisely curl v measures twice the angular velocity, or maybe I should say the angular velocity vector because it also includes the axis of rotation.
I should say maybe for the rotation part of a motion.
For example, just to remind you, I mean we have seen this guy a couple of times, but if I give you a uniform rotation motion about the z, axes.
That is a vector field in which the trajectories are going to be circles centered in the z-axis and our vector field is just going to be tangent to each of these circles.
And, if you look at it from above, then you will have this rotation vector field that we have seen many times.
Typically, the velocity vector for this would be minus yi plus yj times maybe a number that represents how fast we are spinning, the angular velocity in gradients per second.
And then.
if you compute the curl of this, you will end up with two omega times k. Now, the other kinds of vector fields we have seen physically are force fields.
The question is what does the curl of a force field mean?
What can we say about that?
The interpretation is a little bit less obvious, but let's try to get some idea of what it might be.
I want to remind you that if we have a solid in a force field, we can measure the torque exerted by the force on the solid.
Maybe first I should remind you about what torque is in space.
Let's say that I have a piece of solid with a mass, delta m for example, and I have a force that is being exerted to it.
Let's say that maybe my force might be F times delta m. If you think, for example, a gravitational field.
The gravitational force is actually the gravitational field times the mass.
I mean you can forget delta m if you don't like it.
And let's say that the position vector, which should be aiming for the origin, R is here.
And now let's say that maybe this guy is at the end of some arm or some metal thing and I want to hold it in place.
The force is going to exert a torque relative to the origin that will try to measure how much I am trying to swing this guy around the origin.
And, consequently, how much effort I have to exert if I want to actually maintain its place by just holding it at the end of the stick here.
So the torque is now a vector, which is just the cross-product of a position vector with a force.
What the torque measures again is the rotation effects of the force.
And if you remember the principle that the derivative of velocity, which is acceleration, is force divided by mass then the derivative of angular velocity should be angular acceleration which is related to the torque per unit mass.
To just remind you, if I look at translation motions, say I am just looking at the point mass so there are no rotation effects then force divided by mass is acceleration, which is the derivative of velocity.
And so what I am claiming is that for rotation effects we have a similar law, which maybe you have seen in 8.01.
Well, it is one of the important things of solid mechanics, which is the torque of a force divided by the moment of inertia.
I am cheating a little bit here.
If you can see how I am cheating then I am sure you know how to state it correctly.
And if you don't see how I am cheating then let's just ignore the details.
[LAUGHTER] Is angular acceleration.
And angular acceleration is the derivative of angular velocity.
If I think of curl as an operation, which from a velocity field gives the angular velocity of its rotation effects, then you see that the curl of an acceleration field gives the angular acceleration in the rotation part of the acceleration effects.
And, therefore, the curl of a force field measures the torque per unit moment of inertia.
It measures how much torque its force field exerts on a small test solid placed in it.
If you have a small solid somewhere, the curl will just measure how much your solid starts spinning if you leave it in this force field.
In particular, a force field with no curl is a force field that does not generate any rotation motion.
That means if you put an object in there that is completely immobile and you leave it in that force field, well, of course it might accelerate in some direction but it won't start spinning.
While, if you put it in there spinning already in some direction, it should continue to spin in the same way.
Of course, maybe there will be friction and things like that which will slow it down but this force field is not responsible for it.
The cool consequence of this is if a force field F derives from a potential -- That is what we have seen about conservative forces.
Our main concern so far has been to say if we have a conservative force field it means that the work of a force is the change in the energy.
And, in particular, we cannot get energy for free out of it.
And the change in the potential energy is going to be the change in kinetic energy.
You have conservation of energy principles.
There is another thing that we know now because if a force derives from a potential then that means its curl is zero.
That is the criterion we have seen for a vector field to derive from a potential.
And if the curl is zero then it means that this force does not generate any rotation effects.
For example, if you try to understand where the earth comes from, well, the earth is spinning on itself as it goes around the sun.
And you might wonder where that comes from.
Is that causes by gravitational attraction?
And the answer is no.
Gravitational attraction in itself cannot cause the earth to start spinning faster or slower, at least if you assume the earth to be a solid, which actually is false.
I mean basically the reason why the earth is spinning is because it was formed spinning.
It didn't start spinning because of gravitational effects.
And that is a rather deep purely mathematical consequence of understanding gravitation in this way.
It is quite spectacular that just by abstract thinking we got there.
What is the truth?
Well, the truth is the earth, the moon and everything is slightly deformable.
And so there is deformation, friction effects, tidal effects and so on.
And these actually cause rotations to get slightly synchronized with each other.
For example, if you want to explain why the moon is always showing the same face to the earth, why the rotation of a moon on itself is synchronized with its revolution around the earth, which is actually explained by friction effects over time and the gravitational attraction of the earth and the moon.
There is something there, but if you took perfectly rigid, solid bodies then gravitation would never cause any rotation effects.
Of course that tells us that we do not know how to answer the question of why is the earth spinning.
That will be left for another physics class.
I don't have a good answer to that.
That was kind of 8.01-ish.
Let me now move forward to 8.02 stuff.
I want to tell you things about electric and magnetic fields.
And, in fact, something that is known as Maxwell's equations.
Just a quick poll.
How many of you have been taking 8.02 or something like that?
OK. That is not very many.
For most of you this is a preview.
If you have been taking 8.02, have you seen Maxwell's equations, at least part of them?
Yeah.
OK. Then I am sure, in that case, you know better than me what I am going to talk about because I am not a physicist.
But just in case.
Maxwell's equations govern how electric and magnetic fields behave, how they are caused by electric charges and their motions.
And, in particular, they explain a lot of things such as how electric devices work, but also how electromagnetic waves propagate.
In particular, that explains light and all sorts of waves.
It is thanks to them, you know, your cell phone, laptops and things like that work.
Anyway.
Hopefully most of you know that the electric field is a vector field that basically tells you what kind of force will be exerted on a charged particle that you put in it.
If you have a particle carrying an electric charge then this vector field will tell you, basically there will be an electric force which is the charge times E that will be exerted on that particle.
And that is what is responsible, for example, for the flow of electrons when you have a voltage difference.
Because classically this guy is a gradient of a potential.
And that potential is just electric voltage.
The magnetic field is a little bit harder to think about if you have never seen it in physics, but it is what is causing, for example, magnets to work.
Well, basically it is a force that is also expressed in terms of a vector field usually called B.
Some people call it H but I am going to use B.
And that force tends to cause it, if you have a moving charged particle, to deflect its trajectory and start rotating in a magnetic field.
What it does is not quite as easy as what an electric field does.
Just to give you formulas, the force caused by the electric field is the charge times the electric field.
And the force caused by the magnetic field, I am never sure about the sign.
Is that the correct sign?
Good.
Now, the question is we need to understand how these fields themselves are caused by the charged particles that are placed in them.
There are various laws in there that explain what is going on.
Let me focus today on the electric field.
Maxwell's equations actually tell you about div and curl of these fields.
Let's look at div and curl of the electric field.
The first equation is called the Gauss-Coulomb law.
And it says that the divergence of the electric field is equal to, so this is a just a physical constant, and what it is equal to depends on what units you are using.
And this guy rho, well, it is not the same rho as in spherical coordinates because physicists somehow pretended they used that letter first.
It is the electric charge density.
It is the amount of electric charge per unit volume.
What this tells you is that divergence of E is caused by the presence of electric charge.
In particular, if you have an empty region of space or a region where nothing has electrical charge then E has divergence equal to zero.
Now, that looks like a very abstract strange equation.
I mean it is a partial differential equation satisfied by the electric field E. And that is not very intuitive in any way.
What is actually more intuitive is what we get if we apply the divergence theorem to this equation.
If I think now about any closed surface, and I want to think about the flux of the electric field out of that surface, we haven't really thought about what the flux of a force field does.
And I don't want to get into that because there is no very easy answer in general, but I am going to explain soon how this can be useful sometimes.
Let's say that we want to find the flux of the electric field out of a closed surface.
Then, by the divergence theorem, that is equal to the triple integral of a region inside of div E dV, which is by the equation one over epsilon zero, that is this constant, times the triple integral of rho dV.
But now, if I integrate the charge density over the entire region, then what I will get is actually the total amount of electric charge inside the region.
That is the electric charge in D. This one tells us, in a more concrete way, how electric charges placed in here influence the electric field around them.
In particular, one application of that is if you want to study capacitors.
Capacitors are these things that store energy by basically you have two plates, one that contains positive charge and a negative charge.
Then you have a voltage between these plates.
And, basically, that can provide electrical energy to power maybe an electric circuit.
That is not really a battery because it doesn't store energy in large enough amounts.
But, for example, that is why when you switch your favorite gadget off it doesn't actually go off immediately but somehow you see things dimming progressively.
There is a capacitor in there.
If you want to understand how the voltage and the charge relate to each other, the voltage is obtained by integrating the electric field from one plate to the other plate.
And the charges in the plates are what causes the electric field between the plates.
That is how you can get the relation between voltage and charge in these guys.
That is an example of application of that.
Now, of course, if you haven't seen any of this then maybe it is a little bit esoteric, but that will tell you part of what you will see in 8.02.
Questions?
I see some confused faces.
Well, don't worry.
It will make sense some day.
[LAUGHTER] The next one I want to tell you about is Faraday's law.
In case you are confused, Maxwell's equations, there are four equations in the set of Maxwell's equations and most of them don't carry Maxwell's name.
That is a quirky feature.
That one tells you about the curl of the electric field.
Now, depending on your knowledge, you might start telling me that the curl of the electric field has to be zero because it is the gradient of the electric potential.
I told you this stuff about voltage.
Well, that doesn't account for the fact that sometimes you can create voltage out of nowhere using magnetic fields.
And, in fact, you have a failure of conservativity of the electric force if you have a magnetic field.
What this one says is the curl of E is not zero but rather it is the derivative of the magnetic field with respect to time.
More precisely it tells you that what you might have learned about electric fields deriving from electric potential becomes false if you have a variable magnetic field.
And just to tell you again that is a strange partial differential equation relating these two vector fields.
To make sense of it one should use Stokes' theorem.
If we apply Stokes' theorem to compute the work done by the electric field around a closed curve, that means you have a wire in there and you want to find the voltage along the wire.
Now there is a strange thing because classically you would say, well, if I just have a wire with nothing in it there is no voltage on it.
Well, a small change in plans.
If you actually have a varying magnetic field that passes through that wire then that will actually generate voltage in it.
That is how a transformer works.
When you plug your laptop into the wall circuit, you don't actually feed it directly 110 volts, 120 volts or whatever.
There is a transformer in there.
What the transformer does it takes some input voltage and passes that through basically a loop of wire.
Not much seems to be happening.
But now you have another loops of wire that is intertwined with it.
Somehow the magnetic field generated by it, and it has to be a donating current.
The donating current varies over time in the first wire.
That generates a magnetic field that varies over time, so that causes 2B by 2t and that causes curl of the electric field.
And the curl of the electric field will generate voltage between these two guys.
And that is how a transformer works.
It uses Stokes' theorem.
More precisely, how do we find the voltage between these two points?
Well, let's close the loop and let's try to figure out the voltage inside this loop.
To find a voltage along a closed curve places in a varying magnetic field, we have to do the line integral along a closed curve of the electric field.
And you should think of this as the voltage generated in this circuit.
That will be the flux for this surface bounded by the curve of curl E dot dS.
That is what Stokes' theorem says.
And now if you combine that with Faraday's law you end up with the flux trough S of minus dB over dt.
And, of course, you could take, if your loop doesn't move over time, I mean there is a different story if you start somehow taking your wire and somehow moving it inside the field.
But if you don't do that, if it is the field that is moving then you just can take the dB by dt outside.
But let's not bother.
Again, what this equation tells you is that if the magnetic field changes over time then it creates, just out of nowhere, and electric field.
And that electric field can be used to power up things.
I don't really claim that I have given you enough details to understand how they work, but basically these equations are the heart of understanding how things like capacitors and transformers work.
And they also explain a lot of other things, but I will leave that to your physics teachers.
Just for completeness, I will just give you the last two equations in that.
I am not even going to try to explain them too much.
One of them says that the divergence of the magnetic field is zero, which somehow is fortunate because otherwise you would run into trouble trying to understand surface independence when you apply Stokes' theorem in here.
And the last one tells you how the curl of the magnetic field is caused by motion of charged particles.
In fact, let's say that the curl of B is given by this kind of formula, well, J is what is called the vector of current density.
It measures the flow of electrically charged particles.
You get this guy when you start taking charged particles, like electrons maybe, and moving them around.
And, of course, that is actually part of how transformers work because I have told you running the AC through the first loop generates a magnetic field.
Well, how does it do that?
It is thanks to this equation.
If you have a current passing in the loop that causes a magnetic field and, in turn, for the other equation that causes an electric field, which in turn causes a current.
It is all somehow intertwined in a very intricate way and is really remarkable how well that works in practice.
I think that is basically all I wanted to say about 8.02.
I don't want to put your physics teachers out of a job.
[LAUGHTER] If you haven't seen any of this before, I understand that this is probably not detailed enough to be really understandable, but hopefully it will make you a bit curious about that and prompt you to take that class someday and maybe even remember how it relates to 18.02.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so anyway, let's get started.
So, the first unit of the class, so basically I'm going to go over the first half of the class today, and the second half of the class on Tuesday just because we have to start somewhere.
So, the first things that we learned about in this class were vectors, and how to do dot-product of vectors.
So, remember the formula that A dot B is the sum of ai times bi.
And, geometrically, it's length A times length B times the cosine of the angle between them.
And, in particular, we can use this to detect when two vectors are perpendicular.
That's when their dot product is zero.
And, we can use that to measure angles between vectors by solving for cosine in this.
Hopefully, at this point, this looks a lot easier than it used to a few months ago.
So, hopefully at this point, everyone has this kind of formula memorized and has some reasonable understanding of that.
But, if you have any questions, now is the time.
No?
Good.
Next we learned how to also do cross product of vectors in space -- -- and remember, we saw how to use that to find area of, say, a triangle or a parallelogram in space because the length of the cross product is equal to the area of a parallelogram formed by the vectors a and b.
And, we can also use that to find a vector perpendicular to two given vectors, A and B.
And so, in particular, that comes in handy when we are looking for the equation of a plane because we've seen -- So, the next topic would be equations of planes.
And, we've seen that when you put the equation of a plane in the form ax by cz = d, well, <a, b, c> in there is actually the normal vector to the plane, or some normal vector to the plane.
So, typically, we use cross product to find plane equations.
OK, is that still reasonably familiar to everyone?
Yes, very good.
OK, we've also seen how to look at equations of lines, and those were of a slightly different nature because we've been doing them as parametric equations.
So, typically we had equations of a form, maybe x equals some constant times t, y equals constant plus constant times t. z equals constant plus constant times t where these terms here correspond to some point on the line.
And, these coefficients here correspond to a vector parallel to the line.
That's the velocity of the moving point on the line.
And, well, we've learned in particular how to find where a line intersects a plane by plugging in the parametric equation into the equation of a plane.
We've learned more general things about parametric equations of curves.
So, there are these infamous problems in particular where you have these rotating wheels and points on them, and you have to figure out, what's the position of a point?
And, the general principle of those is that you want to decompose the position vector into a sum of simpler things.
OK, so if you have a point on a wheel that's itself moving and something else, then you might want to first figure out the position of a center of a wheel than find the angle by which the wheel has turned, and then get to the position of a moving point by adding together simpler vectors.
So, the general principle is really to try to find one parameter that will let us understand what has happened, and then decompose the motion into a sum of simpler effect.
So, we want to decompose the position vector into a sum of simpler vectors.
OK, so maybe now we are getting a bit out of some people's comfort zone, but hopefully it's not too bad.
Do you have any general questions about how one would go about that, or, yes?
Sorry?
What about it?
Parametric descriptions of a plane, so we haven't really done that because you would need two parameters to parameterize a plane just because it's a two dimensional object.
So, we have mostly focused on the use of parametric equations just for one dimensional objects, lines, and curves.
So, you won't need to know about parametric descriptions of planes on a final, but if you really wanted to, you would think of defining a point on a plane as starting from some given point.
Then you have two vectors given on the plane.
And then, you would add a multiple of each of these vectors to your starting point.
But see, the difficulty is to convert from that to the usual equation of a plane, you would still have to go back to this cross product method, and so on.
So, it is possible to represent a plane, or, in general, a surface in parametric form.
But, very often, that's not so useful.
Yes?
How do you parametrize an ellipse in space?
Well, that depends on how it's given to you.
But, OK, let's just do an example.
Say that I give you an ellipse in space as maybe the more, well, one exciting way to parameterize an ellipse in space is maybe the intersection of a cylinder with a slanted plane.
That's the kind of situations where you might end up with an ellipse.
OK, so if I tell you that maybe I'm intersecting a cylinder with equation x squared plus y squared equals a squared with a slanted plane to get, I messed up my picture, to get this ellipse of intersection, so, of course you'd need the equation of a plane.
And, let's say that this plane is maybe given to you.
Or, you can switch it to form where you can get z as a function of x and y.
So, maybe it would be z equals, I've already used a; I need to use a new letter.
Let's say c1x c2y plus d, whatever, something like that.
So, what I would do is first I would look at what my ellipse does in the directions in which I understand it the best.
And, those directions would be probably the xy plane.
So, I would look at the xy coordinates.
Well, if I look at it from above xy, my ellipse looks like just a circle of radius a.
So, if I'm only concerned with x and y, presumably I can just do it the usual way for a circle.
x equals a cosine t. y equals a sine t, OK?
And then, z would end up being just, well, whatever the value of z is to be on the slanted plane above a given xy position.
So, in fact, it would end up being ac1 cosine t plus ac2 sine t plus d, I guess.
OK, that's not a particularly elegant parameterization, but that's the kind of thing you might end up with.
Now, in general, when you have a curve in space, it would rarely be the case that you have to get a parameterization from scratch unless you are already being told information about how it looks in one of the coordinate planes, this kind of method.
Or, at least you'd have a lot of information that would quickly reduce to a plane problem somehow.
Of course, I could also just give you some formulas and let you figure out what's going on.
But, in general, we've done more stuff with plane curves.
With plane curves, certainly there's interesting things with all sorts of mechanical gadgets that we can study.
OK, any other questions on that?
No?
OK, so let me move on a bit and point out that with parametric equations, we've looked also at things like velocity and acceleration.
So, the velocity vector is the derivative of a position vector with respect to time.
And, it's not to be confused with speed, which is the magnitude of v. So, the velocity vector is going to be always tangent to the curve.
And, its length will be the speed.
That's the geometric interpretation.
So, just to provoke you, I'm going to write, again, that formula that was that v equals T hat ds dt.
What do I mean by that?
If I have a curve, and I'm moving on the curve, well, I have the unit tangent vector which I think at the time I used to draw in blue.
But, blue has been abolished since then.
So, I'm going to draw it in red.
OK, so that's a unit vector that goes along the curve, and then the actual velocity is going to be proportional to that.
And, what's the length?
Well, it's the speed.
And, the speed is how much arc length on the curve I go per unit time, which is why I'm writing ds dt.
That's another guy.
That's another of these guys for the speed, OK?
And, we've also learned about acceleration, which is the derivative of velocity.
So, it's the second derivative of a position vector.
And, as an example of the kinds of manipulations we can do, in class we've seen Kepler's second law, which explains how if the acceleration is parallel to the position vector, then r cross v is going to be constant, which means that the motion will be in an plane, and you will sweep area at a constant rate.
So now, that is not in itself a topic for the exam, but the kinds of methods of differentiating vector quantities, applying the product rule to take the derivative of a dot or cross product and so on are definitely fair game.
I mean, we've seen those on the first exam.
They were there, and most likely they will be on the final.
OK, so I mean that's the extent to which Kepler's law comes up, only just knowing the general type of manipulations and proving things with vector quantities, but not again the actual Kepler's law itself.
I skipped something.
I skipped matrices, determinants, and linear systems.
OK, so we've seen how to multiply matrices, and how to write linear systems in matrix form.
So, remember, if you have a 3x3 linear system in the usual sense, so, you can write this in a matrix form where you have a 3x3 matrix and you have an unknown column vector.
And, their matrix product should be some given column vector.
OK, so if you don't remember how to multiply matrices, please look at the notes on that again.
And, also you should remember how to invert a matrix.
So, how did we invert matrices?
Let me just remind you very quickly.
So, I should say 2x2 or 3x3 matrices.
Well, you need to have a square matrix to be able to find an inverse.
The method doesn't work, doesn't make sense.
Otherwise, then the concept of inverse doesn't work.
And, if it's larger than 3x3, then we haven't seen that.
So, let's say that I have a 3x3 matrix.
What I will do is I will start by forming the matrix of minors.
So, remember that minors, so, each entry is a 2x2 determinant in the case of a 3x3 matrix formed by deleting one row and one column.
OK, so for example, to get the first minor, especially in the upper left corner, I would delete the first row, the first column.
And, I would be left with this 2x2 determinant.
I take this times that minus this times that.
I get a number that gives my first minor.
And then, same with the others.
Then, I flip signs according to this checkerboard pattern, and that gives me the matrix of cofactors.
OK, so all it means is I'm just changing the signs of these four entries and leaving the others alone.
And then, I take the transpose of that.
So, that means I read it horizontally and write it down vertically.
I swept the rows and the columns.
And then, I divide by the inverse.
Well, I divide by the determinant of the initial matrix.
OK, so, of course, this is kind of very theoretical, and I write it like this.
Probably it makes more sense to do it on an example.
I will let you work out examples, or bug your recitation instructors so that they do one on Monday if you want to see that.
It's a fairly straightforward method.
You just have to remember the steps.
But, of course, there's one condition, which is that the determinant of a matrix has to be nonzero.
So, in fact, we've seen that, oh, there is still one board left.
We've seen that a matrix is invertible -- -- exactly when its determinant is not zero.
And, if that's the case, then we can solve the linear system, AX equals B by just setting X equals A inverse B.
That's going to be the only solution to our linear system.
Otherwise, well, AX equals B has either no solution, or infinitely many solutions.
Yes?
The determinant of a matrix real quick?
Well, I can do it that quickly unless I start waving my hands very quickly, but remember we've seen that you have a matrix, a 3x3 matrix.
Its determinant will be obtained by doing an expansion with respect to, well, your favorite.
But usually, we are doing it with respect to the first row.
So, we take this entry and multiply it by that determinant.
Then, we take that entry, multiply it by that determinant but put a minus sign.
And then, we take that entry and multiply it by this determinant here, and we put a plus sign for that.
OK, so maybe I should write it down.
That's actually the same formula that we are using for cross products.
Right, when we do cross products, we are doing an expansion with respect to the first row.
That's a special case.
OK, I mean, do you still want to see it in more details, or is that OK?
Yes?
That's correct.
So, if you do an expansion with respect to any row or column, then you would use the same signs that are in this checkerboard pattern there.
So, if you did an expansion, actually, so indeed, maybe I should say, the more general way to determine it is you take your favorite row or column, and you just multiply the corresponding entries by the corresponding cofactors.
So, the signs are plus or minus depending on what's in that diagram there.
Now, in practice, in this class, again, all we need is to do it with respect to the first row.
So, don't worry about it too much.
OK, so, again, the way that we've officially seen it in this class is just if you have a1, a2, a3, b1, b2, b3, c1, c2, c3, so if the determinant is a1 times b2 b3, c2 c3, minus a2 b1 b3 c1 c3 plus a3 b1 b2 c1 c2.
And, this minus is here basically because of the minus in the diagram up there.
But, that's all we need to know.
Yes?
How do you tell the difference between infinitely many solutions or no solutions?
That's a very good question.
So, in full generality, the answer is we haven't quite seen a systematic method.
So, you just have to try solving and see if you can find a solution or not.
So, let me actually explain that more carefully.
So, what happens to these two situations when a is invertible or not?
So, remember, in the linear system, you can think of a linear system as asking you to find the intersection between three planes because each equation is the equation of a plane.
So, Ax = B for a 3x3 system means that x should be in the intersection of three planes.
And then, we have two cases.
So, the case where the system is invertible corresponds to the general situation where your three planes somehow all just intersect in one point.
And then, the situation where the determinant, that's when the determinant is not zero, you get just one point.
However, sometimes it will happen that all the planes are parallel to the same direction.
So, determinant a equals zero means the three planes are parallel to a same vector.
And, in fact, you can find that vector explicitly because that vector has to be perpendicular to all the normals.
So, at some point we saw other subtle things about how to find the direction of this line that's parallel to all the planes.
So, now, this can happen either with all three planes containing the same line.
You know, they can all pass through the same axis.
Or it could be that they have somehow shifted with respect to each other.
And so, it might look like this.
Then, the last one is actually in front of that.
So, see, the lines of intersections between two of the planes, so, here they all pass through the same line, and here, instead, they intersect in one line here, one line here, and one line there.
And, there's no triple intersection.
So, in general, we haven't really seen how to decide between these two cases.
There's one important situation where we have seen we must be in the first case that when we have a homogeneous system, so that means if the right hand side is zero, then, well, x equals zero is always a solution.
It's called the trivial solution.
It's the obvious one, if you want.
So, you know that, and why is that?
Well, that's because all of your planes have to pass through the origin.
So, you must be in this case if you have a noninvertible system where the right hand side is zero.
So, in that case, if the right hand side is zero, there's two cases.
Either the matrix is invertible.
Then, the only solution is the trivial one.
Or, if a matrix is not invertible, then you have infinitely many solutions.
If B is not zero, then we haven't really seen how to decide.
We've just seen how to decide between one solution or zero,infinitely many, but not how to decide between these last two cases.
Yes?
I think in principle, you would be able to, but that's, well, I mean, that's a slightly counterintuitive way of doing it.
I think it would probably work.
Well, I'll let you figure it out.
OK, let me move on to the second unit, maybe, because we've seen a lot of stuff, or was there a quick question before that?
OK. OK, so what was the second part of the class about?
Well, hopefully you kind of vaguely remember that it was about functions of several variables and their partial derivatives.
OK, so the first thing that we've seen is how to actually view a function of two variables in terms of its graph and its contour plot.
So, just to remind you very quickly, if I have a function of two variables, x and y, then the graph will be just the surface given by the equation z equals f of xy.
So, for each x and y, I plot a point at height given with the value of the a function.
And then, the contour plot will be the topographical map for this graph.
It will tell us, what are the various levels in there?
So, what it amounts to is we slice the graph by horizontal planes, and we get a bunch of curves which are the points at given height on the plot.
And, so we get all of these curves, and then we look at them from above, and that gives us this map with a bunch of curves on it.
And, each of them has a number next to it which tells us the value of a function there.
And, from that map, we can, of course, tell things about where we might be able to find minima or maxima of our function, and how it varies with respect to x or y or actually in any direction at a given point.
So, now, the next thing that we've learned about is partial derivatives.
So, for a function of two variables, there would be two of them.
There's f sub x which is partial f partial x, and f sub y which is partial f partial y.
And, in terms of a graph, they correspond to slicing by a plane that's parallel to one of the coordinate planes, so that we either keep x constant, or keep y constant.
And, we look at the slope of a graph to see the rate of change of f with respect to one variable only when we hold the other one constant.
And so, we've seen in particular how to use that in various places, but, for example, for linear approximation we've seen that the change in f is approximately equal to f sub x times the change in x plus f sub y times the change in y.
So, you can think of f sub x and f sub y as telling you how sensitive the value of f is to changes in x and y.
So, this linear approximation also tells us about the tangent plane to the graph of f. In fact, when we turn this into an equality, that would mean that we replace f by the tangent plane.
We've also learned various ways of, before I go on, I should say, of course, we've seen these also for functions of three variables, right?
So, we haven't seen how to plot them, and we don't really worry about that too much.
But, if you have a function of three variables, you can do the same kinds of manipulations.
So, we've learned about differentials and chain rules, which are a way of repackaging these partial derivatives.
So, the differential is just, by definition, this thing called df which is f sub x times dx plus f sub y times dy.
And, what we can do with it is just either plug values for changes in x and y, and get approximation formulas, or we can look at this in a situation where x and y will depend on something else, and we get a chain rule.
So, for example, if f is a function of t time, for example, and so is y, then we can find the rate of change of f with respect to t just by dividing this by dt.
So, we get df dt equals f sub x dx dt plus f sub y dy dt.
We can also get other chain rules, say, if x and y depend on more than one variable, if you have a change of variables, for example, x and y are functions of two other guys that you call u and v, then you can express dx and dy in terms of du and dv, and plugging into df you will get the manner in which f depends on u and v. So, that will give you formulas for partial f partial u, and partial f partial v. They look just like these guys except there's a lot of curly d's instead of straight ones, and u's and v's in the denominators.
OK, so that lets us understand rates of change.
We've also seen yet another way to package partial derivatives into not a differential, but instead, a vector.
That's the gradient vector, and I'm sure it was quite mysterious when we first saw it, but hopefully by now, well, it should be less mysterious.
OK, so we've learned about the gradient vector which is del f is a vector whose components are just the partial derivatives.
So, if I have a function of just two variables, then it's just this.
And, so one observation that we've made is that if you look at a contour plot of your function, so maybe your function is zero, one, and two, then the gradient vector is always perpendicular to the contour plot, and always points towards higher ground.
OK, so the reason for that was that if you take any direction, you can measure the directional derivative, which means the rate of change of f in that direction.
So, given a unit vector, u, which represents some direction, so for example let's say I decide that I want to go in this direction, and I ask myself, how quickly will f change if I start from here and I start moving towards that direction?
Well, the answer seems to be, it will start to increase a bit, and maybe at some point later on something else will happen.
But at first, it will increase.
So, the directional derivative is what we've called f by ds in the direction of this unit vector, and basically the only thing we know to be able to compute it, the only thing we need is that it's the dot product between the gradient and this vector u hat.
In particular, the directional derivatives in the direction of I hat or j hat are just the usual partial derivatives.
That's what you would expect.
OK, and so now you see in particular if you try to go in a direction that's perpendicular to the gradient, then the directional derivative will be zero because you are moving on the level curve.
So, the value doesn't change, OK?
Questions about that?
Yes?
Yeah, so let's see, so indeed to look at more recent things, if you are taking the flux through something given by an equation, so, if you have a surface given by an equation, say, f equals one.
So, say that you have a surface here or a curve given by an equation, f equals constant, then the normal vector to the surface is given by taking the gradient of f. And that is, in general, not a unit normal vector.
Now, if you wanted the unit normal vector to compute flux, then you would just scale this guy down to unit length, OK?
So, if you wanted a unit normal, that would be the gradient divided by its length.
However, for flux, that's still of limited usefulness because you would still need to know about ds.
But, remember, we've seen a formula for flux in terms of a non-unit normal vector, and n over n dot kdxdy.
So, indeed, this is how you could actually handle calculations of flux through pretty much anything.
Any other questions about that?
OK, so let me continue with a couple more things we need to, so, we've seen how to do min/max problems, in particular, by looking at critical points.
So, critical points, remember, are the points where all the partial derivatives are zero.
So, if you prefer, that's where the gradient vector is zero.
And, we know how to decide using the second derivative test whether a critical point is going to be a local min, a local max, or a saddle point.
Actually, we can't always quite decide because, remember, we look at the second partials, and we compute this quantity ac minus b squared.
And, if it happens to be zero, then actually we can't conclude.
But, most of the time we can conclude.
However, that's not all we need to look for an absolute global maximum or minimum.
For that, we also need to check the boundary points, or look at the behavior of a function, at infinity.
So, we also need to check the values of f at the boundary of its domain of definition or at infinity.
Just to give you an example from single variable calculus, if you are trying to find the minimum and the maximum of f of x equals x squared, well, you'll find quickly that the minimum is at zero where x squared is zero.
If you are looking for the maximum, you better not just look at the derivative because you won't find it that way.
However, if you think for a second, you'll see that if x becomes very large, then the function increases to infinity.
And, similarly, if you try to find the minimum and the maximum of x squared when x varies only between one and two, well, you won't find the critical point, but you'll still find that the smallest value of x squared is when x is at one, and the largest is at x equals two.
And, all this business about boundaries and infinity is exactly the same stuff, but with more than one variable.
It's just the story that maybe the minimum and the maximum are not quite visible, but they are at the edges of a domain we are looking at.
Well, in the last three minutes, I will just write down a couple more things we've seen there.
So, how to do max/min problems with non-independent variables -- So, if your variables are related by some condition, g equals some constant.
So, then we've seen the method of Lagrange multipliers.
OK, and what this method says is that we should solve the equation gradient f equals some unknown scalar lambda times the gradient, g. So, that means each partial, f sub x equals lambda g sub x and so on, and of course we have to keep in mind the constraint equation so that we have the same number of equations as the number of unknowns because you have a new unknown here.
And, the thing to remember is that you have to be careful that the second derivative test does not apply in this situation.
I mean, this is only in the case of independent variables.
So, if you want to know if something is a maximum or a minimum, you just have to use common sense or compare the values of a function at the various points you found.
Yes?
Will we actually have to calculate?
Well, that depends on what the problem asks you.
It might ask you to just set up the equations, or it might ask you to solve them.
So, in general, solving might be difficult, but if it asks you to do it, then it means it shouldn't be too hard.
I haven't written the final yet, so I don't know what it will be, but it might be an easy one.
And, the last thing we've seen is constrained partial derivatives.
So, for example, if you have a relation between x, y, and z, which are constrained to be a constant, then the notion of partial f partial x takes several meanings.
So, just to remind you very quickly, there's the formal partial, partial f, partial x, which means x varies.
Y and z are held constant.
And, we forget the constraint.
This is not compatible with a constraint, but we don't care.
So, that's the guy that we compute just from the formula for f ignoring the constraints.
And then, we have the partial f, partial x with y held constant, which means y held constant.
X varies, and now we treat z as a dependent variable.
It varies with x and y according to whatever is needed so that this constraint keeps holding.
And, similarly, there's partial f partial x with z held constant, which means that, now, y is the dependent variable.
And, the way in which we compute these, we've seen two methods which I'm not going to tell you now because otherwise we'll be even more over time.
But, we've seen two methods for computing these based on either the chain rule or on differentials, solving and substituting into differentials.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
I guess last time on Friday we went over the first half of the class very quickly.
And so today we are going to go over the second half of the class very quickly.
And so that was stuff about double and triple integrals and vector calculus in the plane and in space.
As usual, what is on the final is basically what was on the other tests, exactly the same stuff.
Well, not the same problem, unfortunately.
The first thing we learned about was double integrals in the plane and how to set up the bounds and how to evaluate them.
Just to remind you quickly, the important thing with iterated integrals is when you integrate a function f of x, y, say dy dx for example, is that you have to draw a picture of a region.
Unless it is completely obvious you should really draw some picture of the domain of integration.
And once you have that picture you can use it to find the bounds.
Remember the general method is that we first look at the inner integral, here integral of f dy.
And in this inner integral the outer variable here, x, is fixed.
That means we are slicing our region by a vertical line corresponding to a fixed value of x.
We fix a value of x.
And what we have to find out is the bounds for y, so the value of y at this point, the value of y at that point.
Let me call that y some bottom of x, in general depends on x.
And this one will be y at that top, and it also depends on x.
And then the bounds for y would be this.
And then, when you look at the outer bound, things are different.
Because there you expect to have just numbers, no longer functions of anything.
And what you do is look at the shadow of your region.
We are doing it by shadow so you just project to the x-axis.
If you project to the x-axis your region will look like this.
Its shadow is going to be this integral form, some minimum value of x to some maximum value of x.
And that will give us the bounds for the outer integral.
And then, to evaluate, we evaluate the usual way.
Speaking of evaluation, what you need to know for the final, well, essentially the same kind of evaluation techniques that we were supposed to know for the other tests.
That means the usual functions, substitutions, basic trig, stuff like that.
Well, I don't expect that you would need integration by parts, although I still hope that some of you remember it from single variable calculus.
If there is a need to integrate some big power of cosine or sine then the formula will be given to you the way it is in the notes.
And, of course, we know also how to set up these integrals in polar coordinates.
And then the area element becomes r dr d theta.
And because you integrate first over r, well, first of all you should remember the polar coordinate formulas, namely x equals r cosine theta and y equals r sine theta.
And second you should remember that what we do, when we have our region, is for a fixed value of theta we look for the bounds for r, just like before, so the way we are slicing the region is now we are actually shooting rays straight from the origin.
And, in a given direction, we are asking ourselves how far does my region go?
You have to find a bound and you have to find whatever the value of r will be out here as a function of theta.
And ways to do that can be geometric or they can be by starting from the x, y equation of whatever curve you have and then expressing it in terms of r and theta and solving for r. For example, just to illustrate it, we have seen that one of our classics has been the circle of radius one centered at one, zero.
This guy, you have two different ways of getting its polar coordinate equation.
One is to argue geometrically that you have a right angle in here.
And this length is two, this angle is theta, this length is r, so the polar equation is r equals two cosine theta.
The other way to do it, if somehow you are missing the geometric trick, is to start from the x, y equation.
What is the x, y equation of this guy?
Well, it is x minus one squared plus y squared equals one.
If you expand that you will get x squared minus two x plus one plus y squared equals one.
The ones simplify.
X squared plus y squared becomes r squared minus two x becomes r cosine theta equals zero.
That gives you, when you simplify by r, r equals two cosine theta.
Two ways to get the same polar equation.
I should say this is an example, in case you were wondering what I was doing.
We have also actually seen how to change variables to more complicated coordinate systems.
Let's say u, v coordinates.
But, of course, you can call them whatever you want.
The main thing to remember is that you have to look for the Jacobian which will give you the conversion ratio between dx dy and du dv.
For example, if you know u and v as functions of x and y then you will write du dv equals absolute value of the Jacobian partial u, v over partial x, y times dx dy.
Or, if it is easier for you, you can do the Jacobian the other way around.
And this Jacobian, remember, is the determinant by a two by two matrix that you obtain by putting the partial derivatives of u and v with respect to x and y.
Then, when we have that, we can change the integrant, f of x, y, into something involving u and v possibly.
And then we have to find the bounds.
And to find the bounds perhaps the easiest is to draw a picture of a region in u, v coordinates.
Maybe you have some picture in the x, y plane that might actually be really hard to draw and maybe in terms of u and v the picture will become much simpler.
It might just become a rectangle.
Of course, if you see immediately what the bounds are in terms of u and v, and they turn out to be very easy, then maybe you don't even have to draw this picture.
But if it is not completely obvious then that might be a helpful way of figuring out what the bounds will be when you switch from x, y to u, v. We have seen some problems like that and there are more in the notes in case you need more.
Questions?
Yes?
That is the second time you've asked for something real quick in these review sessions.
You are in a hurry.
Take your time.
Partial u, v over partial x, y is just going to be the determinant of u sub x, u sub y, v sub x, v sub y.
That is the definition.
That is pretty direct.
And, of course, a general common sense thing that applies to actually all the integrals that we are going to see, there are two things in an integral.
One is whatever you integral is called the integrant.
It could be a function here.
It is a vector field in some of the flux things and so on.
There is another thing which is the region over which you integrate.
And the two have strictly nothing to do with each other.
When you are given a piece of data in the statement of a problem, you have to figure out whether that is part of a function to be integrated or whether that is part of the region of integration.
If it is the region of integration then it will go into the bounds of the integral and maybe in the choice of the coordinate system that you use for integrating.
While the function that you are integrating goes before the dx dy and not into the bounds or anything like that.
I know it sounds kind of silly but it is a good safety check.
Ask yourselves, when you have a piece of data, where in my formula should this go.
Yes?
I case you want the bounds for this region in polar coordinates, indeed it would be double integral.
For a fixed theta, r goes from zero to whatever it is on that curve.
So it would be zero to two cosine theta of whatever the function is r dr d theta.
And the bounds on theta would be from negative pi over two to pi over two.
We have seen that one several times, so hopefully by now it is clearer.
OK. Let me move on a bit because we have a lot of other kinds of integrals to see.
Other kinds of integrals we have seen are triple integrals.
And I am not doing things in the order that we did them in the class just so you can see parallels between stuff in the plane and in space.
When we do triple integrals in space, well, it is the same kind of story, except now we have, of course, more coordinate systems.
We have rectangular coordinates, we have cylindrical coordinates and we have spherical coordinates.
And cylindrical coordinates only mean that we are, instead of x, y and z, we are replacing x and y by the polar coordinate in the x, y plane, so the angle theta and the distance r. So R is somehow the distance from the z-axis and z is the height.
Usually you don't have to choose between rectangular and cylindrical until somewhat late in the process, especially if you integrate first of all z, because then the choice will come up mostly when you try to figure out what are the bounds for the shadow of your region.
I mean the z part looks exactly the same in rectangular and in cylindrical.
Spherical is, on the other hand, a little bit more annoying because it looks quite different.
You should think of it as doing polar coordinates not only in the horizontal direction but also in the vertical direction at the same time.
You have this angle phi.
That measures the angle down from the positive z-axis.
And you have rho which is the distance from the origin.
And if I project to the z-axis, r becomes rho sine phi and z becomes rho cosine phi.
I hope that you all know these two formulas, but if you ever have a small somehow memory lapse during the final then you should consider drawing this kind of picture because it will let you check very quickly which one is sine, which one is cosine.
Now, of course, we have to have formulas for dv in all these coordinate systems.
Here, for example, that might be dz r dr d theta or r dr d theta dz or anything like that.
Here it might be rho squared times phi times d rho d phi d theta.
And the general method for setting up bounds is pretty much the same as in the plane, just there is one more step.
If you are doing rectangular or cylindrical coordinates with z first, for example, that is the most common.
Well, if you do z first then you have to actually start by figuring out for a given value of x and y or r and theta what is the portion of a vertical line above x, y that lies within my region?
That will go from z on the bottom of my solid which depends on x and y to z at the top of my solid which also usually will depend on x and y.
And so that will give me the bounds for dz.
And then I will be left with the shadow of my region in the x, y plane.
And that one I will set up like a double integral over there.
Strictly-speaking, if you are curious, we could also change to weird coordinate systems using Jacobian with three variables at the same time.
But we haven't seen that so it won't be on the final.
But it would work just the same way, just with more pictures to do.
And, in fact, I just wanted to say this rho squared sine phi is actually the Jacobian for the change of variables for rectangular to spherical coordinates.
OK. Let's not think too much about that.
Applications.
Well, we have seen how to use double integrals to find the area of a volume of a piece of a plane or a piece of space, and find also the mass.
Remember, area is just double integral of one dA, volume is triple integral of one dV.
Sometimes if it's the volume between the x, y plane and the graph of some function, you can just set it up directly as a double integral.
But there is no harm in doing it as a triple integral if you feel better about that.
And mass will be double or triple integral, depending on how many dimensions you have, of whatever density function you have, dA or dV.
Then there is how to find the average value of some function.
Well, let me do the three-dimensional case.
You will just replace volume by area and dV by dA and so on, if need be.
That would be one over volume of the solid times the triple integral of f dV, or if it's a weighted average, one over mass times the triple integral of a function times density times dV.
If you don't have a density or if the density is constant then that reduces to that one.
In particular, we have seen the notion of center of mass.
The center of mass is just given by taking the average values of the coordinates, x bar, y bar, z bar.
It is just this formula but taking x, y or z as the function.
There are moments of inertia.
For example, the moment of inertia about the z-axis is the triple integral of x squared plus y squared density dV.
Or, if you have just a two-dimensional object, it is the same formula, but, of course, with dA.
And then we call that the polar moment of inertia because we thought of it as rotating the plane about the origin, but the origin is just where the z-axis hits the x, y plane so it is really the same thing.
And we have also seen gravitational attraction in space, and I will let you look at your notes for that.
It is just one formula to remember.
Questions about iterated integrals, things like that?
Yes?
The formula that you should know for gravitational attraction is that if yu have a point mass at the origin and you have some solid centered on the z-axis that is attracting it then the force will be given by G times the mass times the triple integral of density times cosine phi over rho squared times dV.
And, of course, you will actually do that in spherical coordinates because it is easier that way.
That is the formula I have in mind.
But, see, all these formulas just give you examples of things to integrate.
And how to set up the bounds and so on does not depend on what you are actually integrating.
It is done always using the same methods.
Let's move on to work and line integrals.
We have seen how to do that in the plane and in space.
And it looks very similar somehow.
Remember, you have to know how to set up and evaluate a line integral of this form.
Let me do it in the plane this time.
If you are in the plane you have two components, and then this becomes the line integral of M dx plus N dy.
If you have a space curve then you will have a third component here.
You will add that guy times dz.
Now, how do we evaluate that?
Well, it is very different from there because here we are just on a curve so there should be only one degree of freedom.
One variable should be enough to know where we are.
We will have to express x and y in terms of -- Well, I should and z optionally if there is one, in terms of a single parameters.
And that might be just one of the coordinates.
If you are told y equals z squared, that is easy.
You just substitute y equals x squared and dy equals two x dx into everything, and you are left with an integral over x.
Maybe it will be something in terms of time or in terms of an angle.
We express everything in terms of a single parameter, and that will give us a usual single integral.
Any questions about that?
Yes?
If you cannot parameterize the curve then it is really, really hard to evaluate the line integral.
Well, you might be able to evaluate it numerically into a computer, but that is the easiest way to describe a curve.
Indeed it could be that in the plane you have an equation in terms of x and y given by some completed formulas defining some curve.
Then actually there are ways you can use basically differentials and constrained partials to figure out what the tangent vector to the curve is and so on.
But we haven't really seen how to do that.
That would be a really nice topic for tying together the end of the second unit that we discussed last time, constrained partials, with this stuff.
But that is not going to be part of our topics.
Basically, all the curves we have seen in this class, there is a way to express the position of a point in terms of a parameter.
We haven't seen any curves that are so complicated that you cannot do that.
The other thing we have seen is that there are some special cases of vector fields where we don't actually have to compute this thing because maybe we know that it is the gradient of some potential function.
And then we have a fundamental theorem that gives us a way to compute this without computing it.
We've seen about gradient fields and path independence.
The thing to check is whether the curl of our vector field is zero.
And remember in the plane that is one condition, Nx equals My.
In 3D in space that is actually three conditions because you have to check all the mixed partials of the various components.
If the curl of f is zero that tells us we are likely to have a gradient field.
Strictly-speaking, I should mention and F is defined in a simply-connected region.
Then F is a gradient field.
That means that we can find a potential function.
You can write F as the gradient of little f for some potential function little f. And we have seen how to find the potential.
In fact, we have seen two methods for that.
And we have seen them twice.
We have seen them once for functions of two variables, once for functions of three variables.
They look very much the same.
I encourage you to compare your notes for the two side by side to see where they differ.
Where they differ, roughly-speaking, well, I never know if it is the first or the second, but one of the two methods was to compute a line integral.
In the plane, what we did is we set up and evaluated a line integral along our favorite path from the origin to a point with coordinates say x1, y1.
And then we had to evaluate the line integral for the work done along this path.
And that will give us the value of potential at that point.
If we are doing it with three variables, that method remains very similar.
The only difference is now we have to go also up in space to some point x1, y1, z1.
And so we actually sum three pieces together.
But on each piece it is the same story, only one variable changes.
Here it is only x that changes, it is only y that changes, and on the third one only z would be changing.
That is one possibility.
And the other possibility for finding the potential is that we start with the condition that the first component of our vector field should be equal to f sub x for the unknown potential function.
What we do is integrate with respect to x, and we will get our potential function up to an integration constant.
And that integration constant typically depends on the remaining variables that might be y or equal in space y and z.
And then what we have to do is take the partial of this with respect to y and compare it to what we want it to be, namely the y component of a vector field, and match them to get some information about this guy.
And if we have three variables then there is a third step because there you will still have an unknown function of z that you need to get by comparing the partials with respect to z. I see a lot of very quiet faces somehow.
Well, hopefully that is because you know that stuff.
If it is because you are hopelessly confused then please review a lot before the final, but I really hope that is not the case.
And so, in particular, what we have seen is once we have the potential then we can use the fundamental theorem of calculus to tell us that if we have a line integral to compute for work along a curve that goes from some point P zero to some point P one then the line integral for the work done by gradient F is actually going just to be the change in value of a potential.
And, in particular, that does not depend on how we got from P zero to P one.
That is why we say that we have path independence.
Next topic is flux in plane and space.
Flux looks quite different in the plane and in space because, in the plane, it is just another kind of line integral, while in space it is a surface integral.
If you were in four-dimensional space it would be a triple integral.
Generally, you do flux for something that is somehow a wall that separates regions of space from each other.
In the plane, the way we do it is we have a curve C and we look at its tangent vector, let's call that T, and we rotate it by 90 degrees clockwise.
That is our convention to get a unit normal vector that points to the right of the curve as we move along the curve.
That is our convention for orienting curves.
And we are always going to be using that one.
N equals T rotated 90 degrees clockwise.
In particular, that means that n ds, which will be what we integrate against when we try to compute flux, will just end up being dy, negative dx.
Concretely, when we have to evaluate a line integral of F dot n ds, geometrically we could try to take the dot product of our field with the normal vector and then sum the length element along the curve.
And, in some cases, for example, if you know that the vector field is tangent to the curve or if a dot product is constant or things like that then that might actually give you a very easy answer.
But, in general, the most efficient way to do it will be to say that if your vector field has components, I don't know, let's call them P and Q, then that will be just the line integral of PQ dot dy, negative dx, which means negative Q dx plus P dy.
And, from that point onward, you evaluate it exactly the same way as you would for a work integral.
But, of course, the geometric meaning is very different.
It is the same meaning that we have always seen for flux.
It measures how much a vector field goes across the curve.
Now, if we are in space then you take flux for a surface, not for a curve.
And the way it will work is that you have to choose an orientation of a surface, which just means choosing one of the two possible unit normal vectors.
And then you will do a surface integral for F dot n dS.
That is the surface i element.
The setup for this surface integral is that first we have to express n and dS in some way.
One possibility is that we can express the normal vector n dS geometrically.
That is, for example, what we do when we look at, say, a horizontal plane or a vertical plane or a sphere or a cylinder.
Then we have some geometric idea of why the normal vector is what it is and we have some formula for dS.
Or, we can use one of the standard formulas.
Basically, we have seen two formulas that work in fairly generate situations.
One of them says -- If S is given by an equation z equals some function of x, y then you can just say n dS equals minus f sub x, minus f sub y, one, dx dy.
And I need to rewrite that because I am running out of space.
But, while I erase, I would like to point out the most important there in here.
When I say n dS equals blah, blah, blah times dx dy, dx dy is not the same thing as dS at all.
If you make that mistake you are going to get into trouble the next time that you try to buy real estate in a region which hills or cliffs or things like that.
dS is the area on the slanted surface.
dx dy is the area on the map that shows the x, y plane.
And these are not the same thing.
In particular, you cannot just take one piece of it and not the other piece.
Let me give you formulas for n and for dS separately just to convince you.
That way, if you feel that you need them, then you will have them.
N is minus f sub x, minus f sub y, one, but scaled down to unit length.
This is not a unit vector.
It is actually divided by the length of this guy which is fx squared plus fy squared plus one.
And dS is that same vector times dx dy.
And so the square roots cancel out when you multiply them together.
But it would be completely wrong to just say I will replace n dS by minus f sub x, minus f sub y and one.
Then I end up again with the dS and I do something else with dS.
That is a pretty bad conceptual mistake because it gives you the wrong answer.
Another option more general than that.
If we have not seen how to solve for z, how to express z as a function of x and y, well, maybe we still know some normal vector to the surface.
Then there is another formula for n dS which is up to sine, N divided by N dot k dx dy.
And, see, that projection formula works also if you have to project to another coordinate plane.
For example, if you want to project to the x, z coordinate plane, the relation between n dS and dx dz is given by N over N dot j, because j is the direction perpendicular to the xz plane.
But this one is more useful.
What is a good example of that?
If you have a slanted plane given to you, you can easily find its normal vector.
That is just given by the coefficients of x, y, z in the equation.
Another situation where that might happen is if your surface is given by an equation of a form of g of x, y, z equals zero.
If that is the case then you know this is a level set of g. And we know how to find a normal vector to the level set, namely the gradient vector is always perpendicular to the level set.
You would take the gradient of g to be your big N. OK. Now, these are basically all the integrals we have seen how to set up.
Now we have a bunch of theorems relating them.
Let me think about how I am going to organize that.
Let me try like this.
This part of the board will be work, this part of the board will be about flux and the left part of the board will be about things in the plane and the right one will be about things in space.
What have we seen?
Well, we have seen Green's theorem for work.
That doesn't work so well because that is too small, so I am going to actually use more blackboards to do that.
This side will be space, this side will be the plane and we are going to start with theorems about work.
And we will see theorems about flux pretty soon.
We have two theorems about work.
In the plane that is called Green's theorem.
In space that is called Stokes' theorem.
Green's theorem says if I have a closed curve in the plane going counterclockwise enclosing entirely some region R then the line integral along C for the work of F is equal to the double integral of a region inside of the curl of F dA.
Concretely, if my components of F are called M and N that is the line integral of M dx plus N dy is equal to the double integral of R of N sub x minus M sub y dA.
This side here is a usual line integral.
This side here is a usual double integral in the plane.
And somehow their values end up being magically related.
Well, not quite magically.
We actually have seen how to prove it.
And now the analog of that in space is Stokes' theorem.
Stokes says if I have a closed curve in space, now I have to decide what kind of thing it bounds.
And the answer is it will have to bound some surface, but I have a choice of surface.
I choose my favorite surface bounded by C. I guess I will just draw it like that.
And I have to choose a compatible orientation.
Remember, we have seen this right hand rule for choosing how to orient the surface.
I believe, in this case, if I take C like that then the normal vector has to go up.
And then it tells me how to compute the work done by F along C. Namely, that becomes the double integral over that surface S of curl F, which I will write as dell cross F dot n dS.
This line integral is a usual line integral, but if for some reason we don't want to compute it directly we can actually replace it by a surface integral over any surface bounded by the curve.
It might be that a problem will tell you which surface you have to consider.
It might be that you will be left to choose the simplest possible surface you can think of that is somehow having this curve as its boundary.
And so now, remember, curl of a vector field in space is going to be another vector expression.
It has three components.
And the way you compute it is not by remembering the actual formula, which is really complicated by, but by instead computing the cross-product between dell and F. You set up the cross-product.
And, of course, it is a highly symbolic cross-product.
I mean it is not an cross-product of actual vectors but it works the same way.
Both of these formulas basically relate work on a curve with what happens to the curl on the surface that is enclosed by this curve, that is bounded by this curve.
And in this one you have less freedom of choice because you don't have somehow a z direction in which you could move your surface.
There is only possible choice of surface.
There is only one thing that is enclosed by this curve in the plane.
In both cases, these things tell you that you can think of curl as measuring how much the field fails to be conservative.
See, if your field was conservative -- If a curl was zero then the right-hand side would just be zero.
And that would be fortunate because if a curl is zero then your field is less conservative.
That means it comes from a potential.
That means when you go along a closed curve, well, the change of value of a potential should be zero.
Another way to say it is path independence tells you no work.
And, of course, if you have a vector field that is not a gradient field then the curl is not necessarily zero and then you get a more interesting answer.
Finally, let's move onto the theorems about flux.
That is Green for flux and that is the divergence theorem.
Flux theorems.
Here I say that will be divergence.
And here it will be Green again.
Green's theorem for flux says I have a closed curve that goes counterclockwise around some region.
In particular, counterclockwise means that the normal vector will be going out of the region.
And then it tells us that the flux out of the region, through the curve C, so that will be the line integral of F dot n ds is equal to the integral of a region inside of div F dA.
And remember the divergence of M, N is just Mx plus Ny.
This one here, the divergence theorem, tells you something similar but now for a region of space bounded by a closed surface.
So if you have some region of space and you call its boundary surface S and you let n be the normal vector that goes out of the region R. You orient S outwards.
Then the flux out of the region through S is going to be the same as the triple integral over the region of divergence F dV.
Remember, the divergence of a vector field with components P, Q, R is Px plus Qy plus Rz.
What do these two theorems say?
Well, they say essentially the same thing.
They say the total flux out of a region is equal to the integral of divergence over whatever is inside.
And the reason for that is, again, we have seen for a velocity field that divergence measures how much things are expanding or how much stuff is being created.
It tells you the amount of sources per unit portion of the region.
When you sum that over everything, you get the amount of fluid that is being, you know, the total amount of sources inside here, and that tells us how much stuff has to go out per unit time.
That is basically the interpretation.
In a way, I would be tempted to say that this table of four theorems is somehow the crucial point of 18.02.
And you would do well to remember them.
However, I would like also to point out that these theorems are completely useless if you don't know how to compute any of the integrals that are in there.
So all the stuff that was around there before is actually somehow more fundamental.
And if you don't know how to compute the double or triple integrals then this is of little use to you.
That is the end.
I guess I have to wish you happy holidays.
OK, let's get started.
I'm assuming that, A, you went recitation yesterday, B, that even if you didn't, you know how to separate variables, and you know how to construct simple models, solve physical problems with differential equations, and possibly even solve them.
So, you should have learned that either in high school, or 18.01 here, or, yeah.
So, I'm going to start from that point, assume you know that.
I'm not going to tell you what differential equations are, or what modeling is.
If you still are uncertain about those things, the book has a very long and good explanation of it.
Just read that stuff.
So, we are talking about first order ODEs
I'll only use two acronyms.
ODE is ordinary differential equations.
I think all of MIT knows that, whether they've been taking the course or not.
So, we are talking about first-order ODEs, which in standard form, are written, you isolate the derivative of y with respect to, x, let's say, on the left-hand side, and on the right-hand side you write everything else.
You can't always do this very well, but for today, I'm going to assume that it has been done and it's doable.
So, for example, some of the ones that will be considered either today or in the problem set are things like y prime equals x over y.
That's pretty simple.
The problem set has y prime equals, let's see, x minus y squared.
And, it also has y prime equals y minus x squared.
There are others, too.
Now, when you look at this, this, of course, you can solve by separating variables.
So, this is solvable.
This one is-- and neither of these can you separate variables.
And they look extremely similar.
But they are extremely dissimilar.
The most dissimilar about them is that this one is easily solvable.
And you will learn, if you don't know already, next time next Friday how to solve this one.
This one, which looks almost the same, is unsolvable in a certain sense.
Namely, there are no elementary functions which you can write down, which will give a solution of that differential equation.
So, right away, one confronts the most significant fact that even for the simplest possible differential equations, those which only involve the first derivative, it's possible to write down extremely looking simple guys.
I'll put this one up in blue to indicate that it's bad.
Whoops, sorry, I mean, not really bad, but recalcitrant.
It's not solvable in the ordinary sense in which you think of an equation is solvable.
And, since those equations are the rule rather than the exception, I'm going about this first day to not solving a single differential equation, but indicating to you what you do when you meet a blue equation like that.
What do you do with it?
So, this first day is going to be devoted to geometric ways of looking at differential equations and numerical.
At the very end, I'll talk a little bit about numerical ways.
And you'll work on both of those for the first problem set.
So, what's our geometric view of differential equations?
Well, it's something that's contrasted with the usual procedures, by which you solve things and find elementary functions which solve them.
I'll call that the analytic method.
So, on the one hand, we have the analytic ideas, in which you write down explicitly the equation, y prime equals f of x,y.
And, you look for certain functions, which are called its solutions.
Now, so there's the ODE.
And, y1 of x, notice I don't use a separate letter.
I don't use g or h or something like that for the solution because the letters multiply so quickly, that is, multiply in the sense of rabbits, that after a while, if you keep using different letters for each new idea, you can't figure out what you're talking about.
So, I'll use y1 means, it's a solution of this differential equation.
Of course, the differential equation has many solutions containing an arbitrary constant.
So, we'll call this the solution.
Now, the geometric view, the geometric guy that corresponds to this version of writing the equation, is something called a direction field.
And, the solution is, from the geometric point of view, something called an integral curve.
So, let me explain if you don't know what the direction field is.
I know for some of you, I'm reviewing what you learned in high school.
Those of you who had the BC syllabus in high school should know these things.
But, it never hurts to get a little more practice.
And, in any event, I think the computer stuff that you will be doing on the problem set, a certain amount of it should be novel to you.
It was novel to me, so why not to you?
So, what's a direction field?
Well, the direction field is, you take the plane, and in each point of the plane-- of course, that's an impossibility.
But, you pick some points of the plane.
You draw what's called a little line element.
So, there is a point.
It's a little line, and the only thing which distinguishes it outside of its position in the plane, so here's the point, (x,y), at which we are drawing this line element, is its slope.
And, what is its slope?
Its slope is to be f of x,y.
And now, You fill up the plane with these things until you're tired of putting then in.
So, I'm going to get tired pretty quickly.
So, I don't know, let's not make them all go the same way.
That sort of seems cheating.
How about here?
Here's a few randomly chosen line elements that I put in, and I putted the slopes at random since I didn't have any particular differential equation in mind.
Now, the integral curve, so those are the line elements.
The integral curve is a curve, which goes through the plane, and at every point is tangent to the line element there.
So, this is the integral curve.
Hey, wait a minute, I thought tangents were the line element there didn't even touch it.
Well, I can't fill up the plane with line elements.
Here, at this point, there was a line element, which I didn't bother drawing in.
And, it was tangent to that.
Same thing over here: if I drew the line element here, I would find that the curve had exactly the right slope there.
So, the point is the integral, what distinguishes the integral curve is that everywhere it has the direction, that's the way I'll indicate that it's tangent, has the direction of the field everywhere at all points on the curve, of course, where it doesn't go.
It doesn't have any mission to fulfill.
Now, I say that this integral curve is the graph of the solution to the differential equation.
In other words, writing down analytically the differential equation is the same geometrically as drawing this direction field, and solving analytically for a solution of the differential equation is the same thing as geometrically drawing an integral curve.
So, what am I saying?
I say that an integral curve, all right, let me write it this way.
I'll make a little theorem out of it, that y1 of x is a solution to the differential equation if, and only if, the graph, the curve associated with this, the graph of y1 of x is an integral curve.
Integral curve of what?
Well, of the direction field associated with that equation.
But there isn't quite enough room to write that on the board.
But, you could put it in your notes, if you take notes.
So, this is the relation between the two, the integral curves of the graphs or solutions.
Now, why is that so?
Well, in fact, all I have to do to prove this, if you can call it a proof at all, is simply to translate what each side really means.
What does it really mean to say that a given function is a solution to the differential equation?
Well, it means that if you plug it into the differential equation, it satisfies it.
Okay, what is that?
So, how do I plug it into the differential equation and check that it satisfies it?
Well, doing it in the abstract, I first calculate its derivative.
And then, how will it look after I plugged it into the differential equation?
Well, I don't do anything to the x, but wherever I see y, I plug in this particular function.
So, in notation, that would be written this way.
So, for this to be a solution means this, that that equation is satisfied.
Okay, what does it mean for the graph to be an integral curve?
Well, it means that at each point, the slope of this curve, it means that the slope of y1 of x should be, at each point, x1 y1.
It should be equal to the slope of the direction field at that point.
And then, what is the slope of the direction field at that point?
Well, it is f of that particular, well, at the point, x, y1 of x.
If you like, you can put a subscript, one, on there, send a one here or a zero there, to indicate that you mean a particular point.
But, it looks better if you don't.
But, there's some possibility of confusion.
I admit to that.
So, the slope of the direction field, what is that slope?
Well, by the way, I calculated the direction field.
Its slope at the point was to be x, whatever the value of x was, and whatever the value of y1 of x was, substituted into the right-hand side of the equation.
So, what the slope of this function of that curve of the graph should be equal to the slope of the direction field.
Now, what does this say?
Well, what's the slope of y1 of x?
That's y1 prime of x.
That's from the first day of 18.01, calculus.
What's the slope of the direction field?
This?
Well, it's this.
And, that's with the right hand side.
So, saying these two guys are the same or equal, is exactly, analytically, the same as saying these two guys are equal.
So, in other words, the proof consists of, what does this really mean?
What does this really mean?
And after you see what both really mean, you say, yeah, they're the same.
So, I don't how to write that.
It's okay: same, same, how's that?
This is the same as that.
Okay, well, this leaves us the interesting question of how do you draw a direction from the, well, this being 2003, mostly computers draw them for you.
Nonetheless, you do have to know a certain amount.
I've given you a couple of exercises where you have to draw the direction field yourself.
This is so you get a feeling for it, and also because humans don't draw direction fields the same way computers do.
So, let's first of all, how did computers do it?
They are very stupid.
There's no problem.
Since they go very fast and have unlimited amounts of energy to waste, the computer method is the naive one.
You pick the point.
You pick a point, and generally, they are usually equally spaced.
You determine some spacing, that one: blah, blah, blah, blah, blah, blah, blah, equally spaced.
And, at each point, it computes f of x, y at the point, finds, meets, and computes the value of f of (x, y), that function, and the next thing is, on the screen, it draws, at (x, y), the little line element having slope f of x,y.
In other words, it does what the differential equation tells it to do.
And the only thing that it does is you can, if you are telling the thing to draw the direction field, about the only option you have is telling what the spacing should be, and sometimes people don't like to see a whole line.
They only like to see a little bit of a half line.
And, you can sometimes tell, according to the program, tell the computer how long you want that line to be, if you want it teeny or a little bigger.
Once in awhile you want you want it narrower on it, but not right now.
Okay, that's what a computer does.
What does a human do?
This is what it means to be human.
You use your intelligence.
From a human point of view, this stuff has been done in the wrong order.
And the reason it's been done in the wrong order: because for each new point, it requires a recalculation of f of (x, y).
That is horrible.
The computer doesn't mind, but a human does.
So, for a human, the way to do it is not to begin by picking the point, but to begin by picking the slope that you would like to see.
So, you begin by taking the slope.
Let's call it the value of the slope, C. So, you pick a number.
C is two.
I want to see where are all the points in the plane where the slope of that line element would be two?
Well, they will satisfy an equation.
The equation is f of (x, y) equals, in general, it will be C. So, what you do is plot this, plot the equation, plot this equation.
Notice, it's not the differential equation.
You can't exactly plot a differential equation.
It's a curve, an ordinary curve.
But which curve will depend; it's, in fact, from the 18.02 point of view, the level curve of C, sorry, it's a level curve of f of (x, y), the function f of x and y corresponding to the level of value C. But we are not going to call it that because this is not 18.02.
Instead, we're going to call it an isocline.
And then, you plot, well, you've done it.
So, you've got this isocline, except I'm going to use a solution curve, solid lines, only for integral curves.
When we do plot isoclines, to indicate that they are not solutions, we'll use dashed lines for doing them.
One of the computer things does and the other one doesn't.
But they use different colors, also.
There are different ways of telling you what's an isocline and what's the solution curve.
So, and what do you do?
So, these are all the points where the slope is going to be C. And now, what you do is draw in as many as you want of line elements having slope C. Notice how efficient that is.
If you want 50 million of them and have the time, draw in 50 million.
If two or three are enough, draw in two or three.
You will be looking at the picture.
You will see what the curve looks like, and that will give you your judgment as to how you are to do that.
So, in general, a picture drawn that way, so let's say, an isocline corresponding to C equals zero.
The line elements, and I think for an isocline, for the purposes of this lecture, it would be a good idea to put isoclines.
Okay, so I'm going to put solution curves in pink, or whatever this color is, and isoclines are going to be in orange, I guess.
So, isocline, represented by a dashed line, and now you will put in the line elements of, we'll need lots of chalk for that.
So, I'll use white chalk.
Y horizontal?
Because according to this the slope is supposed to be zero there.
And at the same way, how about an isocline where the slope is negative one?
Let's suppose here C is equal to negative one.
Okay, then it will look like this.
These are supposed to be lines of slope negative one.
Don't shoot me if they are not.
So, that's the principle.
So, this is how you will fill up the plane to draw a direction field: by plotting the isoclines first.
And then, once you have the isoclines there, you will have line elements.
And you can draw a direction field.
Okay, so, for the next few minutes, I'd like to work a couple of examples for you to show how this works out in practice.
So, the first equation is going to be y prime equals minus x over y.
Okay, first thing, what are the isoclines?
Well, the isoclines are going to be y.
Well, negative x over y is equal to C. Maybe I better make two steps out of this.
Minus x over y is equal to C. But, of course, nobody draws a curve in that form.
You'll want it in the form y equals minus one over C times x.
So, there's our isocline.
Why don't I put that up in orange since it's going to be, that's the color I'll draw it in.
In other words, for different values of C, now this thing is aligned.
It's aligned, in fact, through the origin.
This looks pretty simple.
Okay, so here's our plane.
The isoclines are going to be lines through the origin.
And now, let's put them in, suppose, for example, C is equal to one.
Well, if C is equal to one, then it's the line, y equals minus x.
So, this is the isocline.
I'll put, down here, C equals minus one.
And, along it, no, something's wrong.
I'm sorry?
C is one, not negative one, right, thanks.
Thanks.
So, C equals one.
So, it should be little line segments of slope one will be the line elements, things of slope one.
OK, now how about C equals negative one?
If C equals negative one, then it's the line, y equals x.
And so, that's the isocline.
Notice, still dash because these are isoclines.
Here, C is negative one.
And so, the slope elements look like this.
Notice, they are perpendicular.
Now, notice that they are always going to be perpendicular to the line because the slope of this line is minus one over C. But, the slope of the line element is going to be C. Those numbers, minus one over C and C, are negative reciprocals.
And, you know that two lines whose slopes are negative reciprocals are perpendicular.
So, the line elements are going to be perpendicular to these.
And therefore, I hardly even have to bother calculating, doing any more calculation.
Here's going to be a, well, how about this one?
Here's a controversial isocline.
Is that an isocline?
Well, wait a minute.
That doesn't correspond to anything looking like this.
Ah-ha, but it would if I put C multiplied through by C. And then, it would correspond to C being zero.
In other words, don't write it like this.
Multiply through by C. It will read C y equals negative x.
And then, when C is zero, I have x equals zero, which is exactly the y-axis.
So, that really is included.
How about the x-axis?
Well, the x-axis is not included.
However, most people include it anyway.
This is very common to be a sort of sloppy and bending the edges of corners a little bit, and hoping nobody will notice.
We'll say that corresponds to C equals infinity.
I hope nobody wants to fight about that.
If you do, go fight with somebody else.
So, if C is infinity, that means the little line segment should have infinite slope, and by common consent, that means it should be vertical.
And so, we can even count this as sort of an isocline.
And, I'll make the dashes smaller, indicate it has a lower status than the others.
And, I'll put this in, do this weaselly thing of putting it in quotation marks to indicate that I'm not responsible for it.
Okay, now, we now have to put it the integral curves.
Well, nothing could be easier.
I'm looking for curves which are everywhere perpendicular to these rays.
Well, you know from geometry that those are circles.
So, the integral curves are circles.
And, it's an elementary exercise, which I would not deprive you of the pleasure of.
Solve the ODE by separation of variables.
In other words, we've gotten the, so the circles are ones with a center at the origin, of course, equal some constant.
I'll call it C1, so it's not confused with this C. They look like that, and now you should solve this by separating variables, and just confirm that the solutions are, in fact, those circles.
One interesting thing, and so I confirm this, I won't do it because I want to do geometric and numerical things today.
So, if you solve it by separating variables, one interesting thing to note is that if I write the solution as y equals y1 of x, well, it'll look something like the square root of C1 minus, let's make this squared because that's the way people usually put the radius, minus x squared.
And so, a solution, a typical solution looks like this.
Well, what's the solution over here?
Well, that one solution will be goes from here to here.
If you like, it has a negative side to it.
So, I'll make, let's say, plus.
There's another solution, which has a negative value.
But let's use the one with the positive value of the square root.
My point is this, that that solution, the domain of that solution, really only goes from here to here.
It's not the whole x-axis.
It's just a limited piece of the x-axis where that solution is defined.
There's no way of extending it further.
And, there's no way of predicting, by looking at the differential equation, that a typical solution was going to have a limited domain like that.
In other words, you could find a solution, but how far out is it going to go?
Sometimes, it's impossible to tell, except by either finding it explicitly, or by asking a computer to draw a picture of it, and seeing if that gives you some insight.
It's one of the many difficulties in handling differential equations.
You don't know what the domain of a solution is going to be until you've actually calculated it.
Now, a slightly more complicated example is going to be, let's see, y prime equals one plus x minus y.
It's not a lot more complicated, and as a computer exercise, you will work with, still, more complicated ones.
But here, the isoclines would be what?
Well, I set that equal to C. Can you do the algebra in your head?
An isocline will have the equation: this equals C. So, I'm going to put the y on the right hand side, and that C on the left hand side.
So, it will have the equation y equals one plus x minus C, or a nicer way to write it would be x plus one minus C. I guess it really doesn't matter.
So there's the equation of the isocline.
Let's quickly draw the direction field.
And notice, by the way, it's a simple equation, but you cannot separate variables.
So, I will not, today at any rate, be able to check the answer.
I will not be able to get an analytic answer.
All we'll be able to do now is get a geometric answer.
But notice how quickly, relatively quickly, one can get it.
So, I'm feeling for how the solutions behave to this equation.
All right, let's see, what should we plot first?
I like C equals one, no, don't do C equals one.
Let's do C equals zero, first.
C equals zero.
That's the line.
y equals x plus 1.
Okay, let me run and get that chalk.
So, I'll isoclines are in orange.
If so, when C equals zero, y equals x plus one.
So, let's say it's this curve.
C equals zero.
How about C equals negative one?
Then it's y equals x plus two.
It's this curve.
Well, let's label it down here.
So, this is C equals negative one.
C equals negative two would be y equals x, no, what am I doing?
C equals negative one is y equals x plus two.
That's right.
Well, how about the other side?
If C equals plus one, well, then it's going to go through the origin.
It looks like a little more room down here.
How about, so if this is going to be C equals one, then I sort of get the idea.
C equals two will look like this.
They're all going to be parallel lines because all that's changing is the y-intercept, as I do this thing.
So, here, it's C equals two.
That's probably enough.
All right, let's put it in the line elements.
All right, C equals negative one.
These will be perpendicular.
C equals zero, like this.
C equals one.
Oh, this is interesting.
I can't even draw in the line elements because they seem to coincide with the curve itself, with the line itself.
They write y along the line, and that makes it hard to draw them in.
How about C equals two?
Well, here, the line elements will be slanty.
They'll have slope two, so a pretty slanty up.
And, I can see if a C equals three in the same way.
There are going to be even more slantier up.
And here, they're going to be even more slanty down.
This is not very scientific terminology or mathematical, but you get the idea.
Okay, so there's our quick version of the direction field.
All we have to do is put in some integral curves now.
Well, it looks like it's doing this.
It gets less slanty here.
It levels out, has slope zero.
And now, in this part of the plain, the slope seems to be rising.
So, it must do something like that.
This guy must do something like this.
I'm a little doubtful of what I should be doing here.
Or, how about going from the other side?
Well, it rises, gets a little, should it cross this?
What should I do?
Well, there's one integral curve, which is easy to see.
It's this one.
This line is both an isocline and an integral curve.
It's everything, except drawable, [LAUGHTER] so, you understand this is the same line.
It's both orange and pink at the same time.
But I don't know what combination color that would make.
It doesn't look like a line, but be sympathetic.
Now, the question is, what's happening in this corridor?
In the corridor, that's not a mathematical word either, between the isoclines for, well, what are they?
They are the isoclines for C equals two, and C equals zero.
How does that corridor look?
Well: something like this.
Over here, the lines all look like that.
And here, they all look like this.
The slope is two.
And, a hapless solution gets in there.
What's it to do?
Well, do you see that if a solution gets in that corridor, an integral curve gets in that corridor, no escape is possible.
It's like a lobster trap.
The lobster can walk in.
But it cannot walk out because things are always going in.
How could it escape?
Well, it would have to double back, somehow, and remember, to escape, it has to be, to escape on the left side, it must be going horizontally.
But, how could it do that without doubling back first and having the wrong slope?
The slope of everything in this corridor is positive, and to double back and escape, it would at some point have to have negative slope.
It can't do that.
Well, could it escape on the right-hand side?
No, because at the moment when it wants to cross, it will have to have a slope less than this line.
But all these spiky guys are pointing; it can't escape that way either.
So, no escape is possible.
It has to continue on, there.
But, more than that is true.
So, a solution can't escape.
Once it's in there, it can't escape.
It's like, what do they call those plants, I forget, pitcher plants.
All they hear is they are going down.
So, it looks like that.
And so, the poor little insect falls in.
They could climb up the walls except that all the hairs are going the wrong direction, and it can't get over them.
Well, let's think of it that way: this poor trap solution.
So, it does what it has to do.
Now, there's more to it than that.
Because there are two principles involved here that you should know, that help a lot in drawing these pictures.
Principle number one is that two integral curves cannot cross at an angle.
Two integral curves can't cross, I mean, by crossing at an angle like that.
I'll indicate what I mean by a picture like that.
Now, why not?
This is an important principle.
Let's put that up in the white box.
They can't cross because if two integral curves, are trying to cross, well, one will look like this.
It's an integral curve because it has this slope.
And, the other integral curve has this slope.
And now, they fight with each other.
What is the true slope at that point?
Well, the direction field only allows you to have one slope.
If there's a line element at that point, it has a definite slope.
And therefore, it cannot have both the slope and that one.
It's as simple as that.
So, the reason is you can't have two slopes.
The direction field doesn't allow it.
Well, that's a big, big help because if I know, here's an integral curve, and if I know that none of these other pink integral curves are allowed to cross it, how else can I do it?
Well, they can't escape.
They can't cross.
It's sort of clear that they must get closer and closer to it.
You know, I'd have to work a little to justify that.
But I think that nobody would have any doubt of it who did a little experimentation.
In other words, all these curves joined that little tube and get closer and closer to this line, y equals x.
And there, without solving the differential equation, it's clear that all of these solutions, how do they behave?
As x goes to infinity, they become asymptotic to, they become closer and closer to the solution, x.
Is x a solution?
Yeah, because y equals x is an integral curve.
Is x a solution?
Yeah, because if I plug in y equals x, I get what?
On the right-hand side, I get one.
And on the left-hand side, I get one.
One equals one.
So, this is a solution.
Let's indicate that it's a solution.
So, analytically, we've discovered an analytic solution to the differential equation, namely, Y equals X, just by this geometric process.
Now, there's one more principle like that, which is less obvious.
But you do have to know it.
So, you are not allowed to cross.
That's clear.
But it's much, much, much, much, much less obvious that two integral curves cannot touch.
That is, they cannot even be tangent.
Two integral curves cannot be tangent.
I'll indicate that by the word touch, which is what a lot of people say.
In other words, if this is illegal, so is this.
It can't happen.
You know, without that, for example, it might be, I might feel that there would be nothing in this to prevent those curves from joining.
Why couldn't these pink curves join the line, y equals x?
You know, it's a solution.
They just pitch a ride, as it were.
The answer is they cannot do that because they have to just get asymptotic to it, ever, ever closer.
They can't join y equals x because at the point where they join, you have that situation.
Now, why can't you to have this?
That's much more sophisticated than this, and the reason is because of something called the Existence and Uniqueness Theorem, which says that there is through a point, x zero y zero, that y prime equals f of (x, y) has only one, and only one solution.
One has one solution.
In mathematics speak, that means at least one solution.
It doesn't mean it has just one solution.
That's mathematical convention.
It has one solution, at least one solution.
But, the killer is, only one solution.
That's what you have to say in mathematics if you want just one, one, and only one solution through the point x zero y zero.
So, the fact that it has one, that is the existence part.
The fact that it has only one is the uniqueness part of the theorem.
Now, like all good mathematical theorems, this one does have hypotheses.
So, this is not going to be a course, I warn you, those of you who are theoretically inclined, very rich in hypotheses.
But, hypotheses for those one or that f of (x, y) should be a continuous function.
Now, like polynomial, signs, should be continuous near, in the vicinity of that point.
That guarantees existence, and what guarantees uniqueness is the hypothesis that you would not guess by yourself.
Neither would I.
What guarantees the uniqueness is that also, it's partial derivative with respect to y should be continuous, should be continuous near x zero y zero.
Well, I have to make a decision.
I don't have time to talk about Euler's method.
I'll refer you to the, there's one page of notes, and I couldn't do any more than just repeat what's on those notes.
So, I'll trust you to read that.
And instead, let me give you an example which will solidify these things in your mind a little bit.
I think that's a better course.
The example is not in your notes, and therefore, remember, you heard it here first.
Okay, so what's the example?
So, there is that differential equation.
Now, let's just solve it by separating variables.
Can you do it in your head?
dy over dx, put all the y's on the left.
It will look like dy over one minus y.
Put all the dx's on the left.
So, the dx here goes on the right, rather.
That will be dx.
And then, the x goes down into the denominator.
So now, it looks like that.
And, if I integrate both sides, I get the log of one minus y, I guess, maybe with a, I never bothered with that, but you can.
It should be absolute values.
All right, put an absolute value, plus a constant.
And now, if I exponentiate both sides, the constant is positive.
So, this is going to look like y.
One minus y equals x And, the constant will be e to the C1.
And, I'll just make that a new constant, Cx.
And now, by letting C be negative, that's why you can get rid of the absolute values, if you allow C to have negative values as well as positive values.
Let's write this in a more human form.
So, y is equal to one minus Cx.
Good, all right, let's just plot those.
So, these are the solutions.
It's a pretty easy equation, pretty easy solution method, just separation of variables.
What do they look like?
Well, these are all lines whose intercept is at one.
And, they have any slope whatsoever.
So, these are the lines that look like that.
Okay, now let me ask, existence and uniqueness.
Existence: through which points of the plane does the solution go?
Answer: through every point of the plane, through any point here, I can find one and only one of those lines, except for these stupid guys here on the stalk of the flower.
Here, for each of these points, there is no existence.
There is no solution to this differential equation, which goes through any of these wiggly points on the y-axis, with one exception.
This point is oversupplied.
At this point, it's not existence that fails.
It's uniqueness that fails: no uniqueness.
There are lots of things which go through here.
Now, is that a violation of the existence and uniqueness theorem?
It cannot be a violation because the theorem has no exceptions.
Otherwise, it wouldn't be a theorem.
So, let's take a look.
What's wrong?
We thought we solved it modulo, putting the absolute value signs on the log.
What's wrong?
The answer: what's wrong is to use the theorem you must write the differential equation in standard form, in the green form I gave you.
Let's write the differential equation the way we were supposed to.
It says dy / dx equals one minus y divided by x.
And now, I see, the right-hand side is not continuous, in fact, not even defined when x equals zero, when along the y-axis.
And therefore, the existence and uniqueness is not guaranteed along the line, x equals zero of the y-axis.
And, in fact, we see that it failed.
Now, as a practical matter, it's the way existence and uniqueness fails in all ordinary life work with differential equations is not through sophisticated examples that mathematicians can construct.
But normally, because f of (x, y) will fail to be defined somewhere, and those will be the bad points.
Thanks.
The topic for today is -- Today we're going to talk, I'm postponing the linear equations to next time.
Instead, I think it's a good idea, since in real life, most of the differential equations are solved by numerical methods to introduce you to those right away.
Even when you see the compute where you saw the computer screen, the solutions being drawn.
Of course, what really was happening was that the computer was calculating the solutions numerically, and plotting the points.
So, this is the main way, numerically, it is the main way differential equations are actually solved, if they are of any complexity at all.
So, the problem is, that initial value problem, let's write up the first order problem the way we talked about it on Wednesday.
And now, I'll specifically add to that, the starting point that you used when you did the computer experiments.
And, I'll write the starting point this way.
So, y of x0 should be y0.
So, this is the initial condition, and this is the first order differential equation.
And, as you know, the two of them together are called an IVP, an initial value problem, which means two things, the differential equation and the initial value that you want to start the solution at.
Okay, now, the method we are going to talk about, the basic method of which many others are merely refinements in one way or another, is called Euler's method.
Euler, who did, of course, everything in analysis, as far as I know, didn't actually use it to compute solutions of differential equations.
His interest was theoretical.
He used it as a method of proving the existence theorem, proving that solutions existed.
But, nowadays, it's used to calculate the solutions numerically.
And, the method is very simple to describe.
It's so naive that you probably think that if you that living 300 years ago, you would have discovered it and covered yourself with glory for all eternity.
So, here is our starting point, (x0, y0).
Now, what information do we have?
At that point all we have is the little line element, whose slope is given by f of (x, y).
So, if I start the solution, the only way the solution could possibly go would be to start off in that direction, since I have no other information.
At least it has the correct direction at (x0, y0).
But, of course, it's not likely to have the correct direction anywhere else.
Now, what you do, then, is choose a step size.
I'll try just two steps of the method.
That's, I think, good enough.
Choose a uniform step size, which is usually called h. And, you continue that solution until you get to the next point, which will be x0 + h, as I've drawn it on the picture.
So, we get to here.
We stop at that point, and now you recalculate what the line element is here.
Suppose, here, the line element, now, through this point goes like that.
Well, then, that's the new direction that you should start out with going from here.
And so, the next step of the process will carry you to here.
That's two steps of Euler's method.
Notice it produces a broken line approximation to the solution.
But, in fact, you only see that broken line if you are at a computer if you are looking at the computer visual, for example, whose purpose is to illustrate for you Euler's method.
In actual practice, what you see is, the computer is simply calculating this point, that point, and the succession of points.
And, many programs will just automatically connect those points by a smooth looking curve if that's what you prefer to see.
Well, that's all there is to the method.
What we have to do now is derive the equations for the method.
Now, how are we going to do that?
Well, the essence of it is how to get from the nth step to the n plus first step?
So, I'm going to draw a picture just to illustrate that.
So, now we are not at x0.
But let's say we've already gotten to (xn, yn).
How do I take the next step?
Well, I take the line element, and it goes up like that, let's say, because the slope is this.
I'm going to call that slope A sub n. Of course, A sub n is the value of the right hand side at the point (xn, yn), and we will need that in the equation, -- -- but I think it will be a little bit clearer if I just give it a capital letter at this point.
Now, this is the new point, and all I want to know is what are its coordinates?
Well, the x n plus one is there.
The y n plus one is here.
Clearly I should draw this triangle, complete the triangle.
This side of the triangle, the hypotenuse has slope An.
This side of the triangle has length h. h is the step size.
Perhaps I'd better indicate that, actually put that up so that you know the word step.
It's the step size on the x axis, how far you have to go to get from each x to the next one.
What's this?
Well, if that slope has this, the slope An, this is h. Then this must be h times An, the length of that side, in order that the ratio of the height to this width should be An.
And, that gives us the method.
How do I get from, clearly, to get from xn to x n plus one, I simply add h. So, that's the trivial part of it.
The interesting thing is, how do I get the new y n plus one?
And so, the best way to write it as, that y n plus one minus yn divided by h, well, sorry, y n plus one minus yn is this line, the same as the line h times An.
So, that's the way to write it.
Or, since the computer is interested in calculating y n plus one itself, put this on the other side.
You take the old yn, the previous one, and to it, you add h times An.
And, what, pray tell, is An?
Well, the computer has to be told that An is the value of f. So, now, with that, let's actually write the Euler program, not the program, but the Euler-- the Euler method equations, let's just call it the Euler equations.
What will they be?
First of all, the new x is the old x plus h. The new y is just what I've written there, the old y plus h times a certain number, An, and finally, An has the value-- It's the slope of the line element here, and therefore by definition, that's f of (xn,yn).
So, if these three equations which define Euler's method.
I assume in 1.00 you must be asked to, at some point, as an exercise in the term at one point to program the computer in C or whatever they're using, Java, now, I guess, to do Euler's method.
And, these would be the recursive equations that you would put in to do that.
Okay, let's try an example, then.
So, what would be a good color for Euler?
Well, a purple.
I assume nobody can see purple.
Is that correct?
Can anyone in the back of the room see that that's purple?
Okay.
Sit closer.
So, let's calculate.
The example, I'll use a simple example, but it's not entirely trivial.
My example is going to be the equation, x squared minus y squared on the right hand side.
And, let's start with y of zero equals one, let's say.
And so, this is my initial value problem, that pair of equations.
And, I have to specify a step size.
So, let's take the step size to 0.1.
You choose the step size, or the computer does.
We'll have to talk about that in a few minutes.
Now, what do you do?
Well, I say this is a nontrivial equation because this equation, as far as I know, cannot be solved in terms of elementary functions.
So, this equation would be, in fact, a very good candidate for a numerical method like Euler's.
And, you had to use it, or maybe it was the other way around, I forget.
On your problem set, you drew a picture of the direction field and answered some questions about the isoclines, how the solutions behave.
Now, the main thing I want you to get, this is not just for Euler's, talking about Euler's equations.
But in general, for the calculations you have to do in this course, it's extremely important to be systematic because if you are not systematic, you know, if you just scribble, scribble, scribble, scribble, scribble, you can do the work, but it becomes impossible to find mistakes.
You must do the work in a form in which it can be checked, which you can look over it and find, and try to see where mistakes are if, in fact, there are any.
So, I strongly suggest, this is not a suggestion, this is a command, that you make a little table to do Euler's method by hand, I'd only ask you for a step or two, but since I'm just trying to make sure you have some idea of these equations and where they come from.
So, first, the value of n, then the value of xn, then the value of the yn, and then, a couple of more columns which tell you how to do the calculation.
You are going to need the value of the slope, and it's probably a good idea, also, because otherwise you'll forget it, to put in h An because that occurs in the formula.
All right, let's start doing it.
The first value of n is zero.
That's the starting point.
At the starting point, (x0, y0), x has the value zero, and y has the value one, so, zero and one.
In other words, starting, I'm carrying out exactly what I drew pictorially only now I'm doing it arithmetically using a table and substituting into the formulas.
Okay, the next thing we have to calculate is An.
Well, since An is the value of the right hand side, at the point zero one, you have to plug that in.
The right hand side is x squared minus y squared.
So, it's 0 squared minus 1 squared.
The value of the slope, there, is minus one, negative one.
Now, I have to multiply that by H. h is 0.1.
So, it's negative, I'll never learn that.
The way you learn to talk in kindergarten is the way you learn to talk the rest of your life, unfortunately.
In kindergarten, we said minus.
Negative 0.1. n is one now.
What's the value of xn?
Well, to the old value I add 1/10.
What's the value of y?
Well, at this point, you have to do the calculation.
It's the old value of y.
To get this new value, it's the old value plus this number.
Well, that's this plus that number is nine tenths.
An, now I have to calculate the new slope at this point.
Okay, that is one tenth squared minus nine tenths squared.
That's 0.01 minus 0.81, which makes minus 0.80, I hope.
Check it on your calculators.
Whip them out and press the buttons.
I now multiply that by h, which means it's going to be minus 0.08, perhaps with a zero after.
I didn't tell you how many decimal places.
Let's carry it out to two decimal places.
I think that will be good enough.
And finally, the last step, 2, here, add another one tenth, so the value of x is now two tenths.
And finally, what's the value of y?
Well, I didn't tell you where to stop.
Let's stop at y of 0.2 because there's no more room on the blackboard.
About approximately how big is that?
In other words, this is, then, this is going to be the old y plus this number, which seems to be 0.82 to me.
So, the answer is, the new value is 0.82.
Okay, we got a number.
We did what we are supposed to do.
We got a number.
Next question?
Now, let's ask a few questions.
One of the first, most basic things is, you know, how right is this?
How can I answer such a question if I have no explicit formula for that solution?
That's the basic problem with numerical calculation.
In other words, I have to wander around in the dark to some extent, and yet have some idea when I've arrived at the place that I want to go.
Well, the first question I'd like to answer, is this too high or too low?
Is Euler, sorry, he'll forgive me in heaven, I will use him.
By this, I mean, is the result, let me just say something first, and that I'll criticize it.
Is Euler too high or too low?
In other words, is the result of using Euler's method, i.e.
is this number too high or too low?
Is it higher than the right answer, what it should be?
Or, is it lower than the right answer?
Or, God forbid, is it exactly right?
Well, it's almost never exactly right.
That's not an option.
Now, how will we answer that question?
Well, let's answer it geometrically.
Basically, if the solution were a straight line, then the Euler method would be exactly right all the time.
But, it's not a line.
Then it's a curve.
Well, the critical question is, is it curved?
Is the solution?
So, here's a solution.
Let's call it y1 of x, and let's say here was the starting point.
Here, the solution is convex.
And, here the solution is concave, right?
Concave up or concave down, if you learn those words, but I think those have, by now, I hope pretty well disappeared from the curriculum.
Call it, if you haven't up until now, what mathematicians call it, convex is that, and the other one is concave.
Well, how do Euler's solutions look?
Well, I'll just sketch.
I think from this you can see already, when you start out on the Euler's solution, it's going to go like that.
Now you are too low.
Well, let's suppose after that, the line element here is approximately the same as what it is there, or roughly parallel.
After all, they are not too far apart.
And, the direction field is continuous.
That is, the directions don't change drastically from one point to another.
But now, you see it's still too low.
It's even lower as it pathetically tries to follow.
It's losing territory, and that's basically because the curve is convex.
Exactly the opposite what would happen if the curve were concave, if the solution curve were concave.
Now it's too high, and it's not going to be able to correct that as long as the solution curve stays concave.
Well, that's probably too optimistic.
It's probably more like this.
So, in other words, in this case, if the curve is convex, Euler is going to be too high, sorry, too low.
Let's put E for Euler.
How about that?
Euler is too low.
If it's concave, then Euler is too high.
Okay, that's great.
There's just one little problem left, namely, if we don't have a formula for the solution, and we don't have a computer that's busy drawing the picture for us, in which case we wouldn't need any of this anyway, how are we supposed to tell if it's convex or concave?
Back to calculus.
Calculus to the rescue!
When is a curve convex?
A curve is convex if its second derivative is positive because the first to be convex means the first derivative is increasing all the time.
And therefore, the second derivative, which is the derivative of the first derivative, should be positive.
Just the opposite here; the curve, the slope is, the first derivative, is decreasing all the time and therefore the second derivative is negative.
So, all we have to do is decide what the second derivative of the solution is.
We should probably call it a solution.
y of x is a little too vague.
y1 means the solution started at this point.
So, in fact, probably it would have been better from the beginning to call that y1, except there's no room, y1, let's say.
That means the solution which started out at the point, (0, 1).
So, I'm still talking about at a solution like that.
All right, so I want to know if this is positive, the second derivative is positive at the starting point, zero, or it's negative.
Now, again, how you can regulate the second derivative, if you don't know what the solution is explicitly, then the answer is you can do it from the differential equation itself.
How do I do that?
Well: easy.
y prime equals x squared minus y squared.
Okay, that tells me how to calculate y prime if I know the value of x and y, in other words, the 0.01.
What would be the value of y double prime?
Well, differentiate the equation.
It's two x minus two y y prime.
Don't forget to use the chain rule.
So, if I want to calculate at (0, 1), in other words, if my starting point is that curve convex or concave, well, let's calculate.
y of zero equals one.
Okay, what's y prime of zero?
Well, I don't have to repeat that calculation.
Using this, I've already calculated that it was negative one.
And now, the new thing, what's y double prime of zero?
Well, it is this.
I'll write it out.
It's two times zero minus two times negative y, which is one, two times one times y prime, which is negative one.
You want to see we are pulling ourselves up by our own boot straps, which is impossible.
But, it is not impossible because we are doing it.
So, what's the answer?
Zero here, two, I've calculated without having the foggiest idea of what the solution is or how it looks.
I've calculated that its second derivative at the starting point is two.
Therefore, my solution is convex at the starting point.
And therefore, this Euler approximation, if I don't carry it out too far, will be too low.
So, it's convex Euler, too low.
Now, you could argue, yeah, well, what about this?
[LAUGHTER] So, you could go like this, and then you can see it catches up.
Well, of course, if the curve changes from convex to concave, then it's really impossible to make any prediction at all.
That's a difficulty.
So, all this analysis is only if you stay very nearby.
However, I wanted to show you, the main purpose of it in my mind was to show you how do you use, it's these equations, how to use the differential equation itself to get information about the solutions, without actually being able to calculate the solutions?
Now, so that's the method, and that's how to find out something about it.
And now, what I'd like to talk about is errors.
How do I handle, right?
So, in a sense, I've started the error analysis.
In other words, the error, by definition, the error is this difference, e. So, in other words, what I'm asking here, is the error positive?
It depends which we measure it.
Usually, you take this minus that.
So, here, the error would be considered positive, and here it would be considered negative, although I'm sure there's a book somewhere in the world, which does the opposite.
Most hedge by just using the absolute value of the error plus a statement that the method is producing answers which are too low or too high.
The question, then, is, naturally, this is not the world's best method.
It's not as bad as it seems.
It's not the world's best method because that convexity and concavity means that you are automatically introducing a systematic error.
If you can predict which way the error is going to be by just knowing whether the curve is convex or concave, it's not what you want.
I mean, you want to at least have a chance of getting the right answer, whereas this is telling you you're definitely going to get the wrong answer.
All it tells you is, and it's telling you whether your answer is going to be too high or too low.
We've like a better chance of getting the right answer.
Now, so the question is, how do you get a better method?
A search is for a better method.
Now, the first method, which will occur, I'm sure, to anyone who looks at that picture, is, look, if you want this yellow line to follow the white one, the white solution, more accurately, for heaven's sake, don't take such big steps.
Take small steps, and then it will follow better.
All right, let's draw a picture.
Excuse me.
My little box of treasures, here.
[LAUGHTER] So, use a smaller step size.
And the picture, roughly, which is going to justify that, will look like this.
If the solution curve looks like this, then with a big step size, I'm liable to have something that looks like that.
But, if I take a smaller step size, suppose I halve the step size.
How's it going to look, then?
Well, I better switch to a different color.
If I halve the step size, I'll get a littler, goes like that.
And now it's following closer.
Of course, I'm stacking the deck, but see how close it follows?
I'm definitely not to be trusted on this.
Okay, let's do the opposite, make really big steps.
Suppose instead of the yellow ones I used the green one of double step size.
Well, what would have happened then?
Well, I've started out, but now I've gone all the way to there.
And now, on my way up, of course, it has a little further to go.
But, if for some reason, I stop there, you could see, I would be still lower.
In other words, the bigger the steps size, the more the error.
And, where are the errors that we are talking about?
Well, the way to think of the errors, this is the error, that number the error.
You can make it positive, negative, or just put it automatically an absolute value sign around it.
That's not so important.
So, in other words, the conclusion is, that the error e, the difference between the true value that I should have gotten, and the Euler value that the calculation produced, that the error e, depends on the step size.
Now, how does it depend on the step size?
Well, it's impossible to give an exact formula, but there's an approximate answer, which is, by and large, true.
So, the answer is, e is going to be a function of h. What function?
Well, asymptotically, which is another way of putting quotation marks around, what did I say?
It's going to be a constant, some constant, times H. [LAUGHTER] It looks like this, and for this reason it's called a first order, the Euler is a first order method.
And now, first order does not refer to the first order of the differential equation.
It's not the first order because it's y prime equals f of (x, y).
The first order means the fact that h occurs to the first power.
The way people usually say this is since the normal way of decreasing the step size, as you'll see as is you try to use a computer visual that deals with the Euler method, which I highly recommend, by the way, so highly recommended that you have to do it, is that the way to say it, each new step halves the step size.
That's the usual way to do it.
If you halve the step size, since this is a constant, if I halve the step size, I halve the error, approximately.
Halve the step size, halve the error.
That tells you how the error varies with step size for Euler's method.
Please understand, that's what people say, and please understand the grammatical construction.
Since everyone in the math department has a cold these days except me for the moment, everyone goes around chanting this mantra.
This is totally irrelevant.
This whole mantra, feed a cold, starve a fever.
And if you asked them what it means, they say eat a lot if you have a cold.
And if you have a fever, don't eat very much, which is not what it means at all.
Grammatically, it's exactly the same construction as this.
What this means is if you halve the step size, you will halve the error.
That's what feed a cold, starve a fever means.
And, remember this for the rest of your life.
If you feed a cold, if you eat too much when you have a cold, you will get a fever and end up still having to starve yourself because, of course, nobody, when you have a fever, nobody feels like eating, so they don't eat anything.
All right, you got that?
Good.
I want all of you to go home and tell that to your mothers.
You know, that's the way we always used to speak.
Grimmer ones: spare the rod, spoil the child does not mean that you should not hit your kid.
It means that if you fail to hit your kid, he or she will be spoiled, whatever that means.
So, you don't want to do that.
I guess the mantra today would be, I don't know.
Okay, so the first line of defense is simply to keep having the step size in Euler.
And, what people do is, if they don't want to use anything better than Euler's method, is you keep having the step size until the curve doesn't seem to change anymore.
And then you say, well, that must be the solution.
And, I asked you on the problems set, how much would you continue to have to halve the step size in order for that good thing to happen?
However, there are more efficient methods which get the results faster.
So if that's our good method, let's call this our still better method.
The better methods aim at being better.
They keep the same idea as Euler's method, but they say, look, let's try to improve that slope, An.
In other words, since the slope, An, that we start with is guaranteed to be wrong if the curve is convex or concave, can we somehow correct it?
So, for example, instead of immediately aiming there, can't we somehow aim it so that by luck, we just, at the next step just lands us back on the curve again?
In other words, with sort of looking for the short path, a shortcut path, which by good luck will end us up back on the curve again.
And, all the simple improvements on the Euler method, and they are the most stable in ways to solve differential equations numerically, aim at finding a better slope.
So, they find a better value for a better slope, find a better value than An.
Try to improve that slope that you found.
Now, once you have the idea that you should look for a better slope, it's not very difficult to see what, in fact, you should try.
Again, I think most of you would say, hey, I would have thought of that.
And, you would be closer in time, since these methods were only found about around the turn of the last century is when I place them, mostly by some German mathematicians interested in solving equations numerically.
All right, so what is the better method?
Our better slope, what should we look for in our better slope?
Well, the simplest procedure is, once again, we are starting from there, and the Euler slope would be the same as a line element.
So, the line element looks like this.
And, our yellow slope, A, and I'll still continue to call it An, goes like that, gets to here.
Okay, now if it were convex, if the curve were convex, this would be too low.
And therefore, the next step would be, I'm going to draw this next step in pink.
Well, let's continue in here, would be going up like that.
I'll call this Bn, just because it's the next step of Euler's method.
It could be called An prime or something like that.
But this will do.
And now what you do is, let me put an arrow on it to indicate parallelness, go back to the beginning, draw this parallel to Bn.
So, here is Bn.
Again, just a line of that same slope.
And now, what you should use as the simplest improvement on Euler's method, is take the average of these two because that's more likely to hit the curve than An will, which is sure to be too low if the curve is convex.
In other words, use this instead.
Use that.
So, this is our better slope.
Okay, what will we call that slope?
We don't have to call it anything.
What were the equations for the method be?
Well, x n plus one is gotten by adding the step size.
So, here's my step size just as it was before.
Just as it was before, the new thing is how to get the new value of y.
So, y n plus one should be the old yn, plus h times not this crummy slope, An, but the better, the pink slope.
What's the formula for the pink slope?
Well, let's do it in two steps.
It's the average of An and Bn.
Hey, but you didn't tell me, or I didn't tell you what Bn was.
So, you now must tell the computer, oh yes, by the way, you remember that An was what it always was.
The interesting thing is, what is Bn?
Well, to get Bn, Bn is the slope of the line element at this new point.
Now, what am I going to call that new point?
I don't want to call this y value, y n plus one, because that's, it's this up here that's going to be the y n plus one.
All this is, is a temporary value used to make another calculation, which will then be combined with the previous calculations to get the right value.
Therefore, give it a temporary name.
That point, we'll call it, it's not going to be the final, the real y n plus one.
We'll call it y n plus one twiddle, y n plus one temporary.
And, what's the formula for it?
Well, it's just going to be what the original Euler formula; it's going to be y n plus what you would have gotten if you had calculated, in other words, it's the point that the Euler method produced, but it's not, finally, the point that we want.
Now, do I have to say anything else?
Yeah, I didn't tell the computer what Bn was.
Okay, Bn is the slope of the direction field at the point n plus one.
And the computer knows what that is.
And, this point, y n plus one temporary.
So, you make a temporary choice of this, calculate that number, and then go back, and as it were, correct that value to this value by using this better slope.
Now, that's all there is to the method, except I didn't give you its name.
Well, it has three names, four names in fact.
Which one shall I give you?
I don't care.
Okay, the shortest name is Heun's method.
But nobody pronounces that correctly.
So, it's Heun's method.
It's called, also, the Improved Euler method.
It's called Modified Euler, very expressive word, Modified Euler's method.
And, it's also called RK2.
I'm sure you'll like that name best.
It has a Star Wars sort of sound.
RK stands for Runge Kutta, and the reason for the two is not that it uses, well, it is that it uses two slopes, but the real reason for the two is that it is a second order method.
So, that's the most important thing to put down about it.
It's a second order method, whereas Euler's was only a first order method.
So, Heun's method, or RK2, let's write it, is the shortest thing to write, is a second order method, meaning that the error varies with the step size like some constant, it will not be the same as the constant for Euler's method, times h squared.
That's a big saving because it now means that if you halve the step size, you're going to decrease the error by a factor of one quarter.
You will quarter the error.
Now, you say, hey, why should anyone use anything else?
Well, think a little second.
The real thing which determines how slowly one of these methods run is you look at the hardest step of the method and ask how long does the computer take, how many of those hardest steps are there?
Now, the answer is, the hardest step is always the evaluation of the function because the functions that are common use are not x squared minus y squared.
They take half a page and have, as coefficients, you know, ten decimal place numbers, whatever the engineers doing it, whatever their accuracy was.
So, the thing that controls how long a method runs is how many times the slope, the function, must be evaluated.
For Euler, I only have to evaluate it once.
Here, I have to evaluate it twice.
Now, roughly speaking, the number of function evaluations will each give you the exponent.
The method that's called Runge Kutta fourth order will require four evaluations of slope, but the accuracy will be like h to the fourth: very accurate.
You halve the step size, and it goes down by a factor of 16.
Great.
But you had to evaluate the slope four times.
Suppose, instead, you halve four times this thing.
And, what would you have done?
You would have decreased it to 1/16th to what it was.
You still would increase the number of function evaluations you needed by four, and you would have decreased the error by a 16th.
So, in some sense, it really doesn't matter whether you use a very fancy method, which requires more function evaluations.
That's true.
The error goes down faster, but you are having to more work to get it.
So, anyway, nothing is free.
Now, there is an RK4.
I think I'll skip that, since I wouldn't dare to ask you any questions about it.
But, let me just mention it, at least, because it's the standard.
It uses four evaluations.
It's the standard method, if you don't want to do anything fancier.
It's rather inefficient, but it's very accurate, standard method, accurate, and you'll see when you use the programs, it's, in fact, a program which is drawing those curves, the numerical method which draws all those curves that you believe in on the computer screen is the RK4 method.
The Runge Kutta, I should give them their names.
Runge Kutta, fourth order method.
Two mathematicians, I believe both German mathematicians around the turn of the last century, Runge Kutta fourth order method requires four slopes, requires you to calculate four slopes.
I won't bother telling you what you do, but it's a procedure like that.
It's just a little bit more elaborate.
And you take two of these, you make up a weighted average for the super slope.
You use weighted average.
What should I divide that by to get the right...?
Six.
Why six?
Well, because if all these numbers were the same, I'd want it to come out to be whatever that common value was.
Therefore, in a weighted average, you must always divide by the sum of the coefficients.
So, this is the super slope.
And, if you plug that super slope into here, you will be using the Runge Kutta method, and get the best possible results.
Now, I wanted to spend the last three minutes talking about pitfalls of numerical computation in general.
One pitfall I am leaving you on the homework to discover for yourself.
Don't worry, it won't cause you any grief.
It'll just destroy your faith in these things for the rest of your life, which is probably a good thing.
So, pitfalls, number one, you find, you'll find.
Let me talk, instead, briefly about number two, which I am not giving you an exercise in.
Number two is illustrated by the following equation.
What could be simpler?
This is a very bad equation to try to solve numerically.
Now, why?
Well, because if I separate variables, why don't I save a little time?
I'll just tell you what the solution is, okay?
You obviously separate variables.
Maybe you can do it in your head.
The solution will be, the solutions will have an arbitrary constant in them, and they won't be very complicated.
They will be one over c minus x.
C is an arbitrary constant, and as you give different values, you get, now, what do those guys look like?
Okay, so here I am.
I start out at the point, one.
And, I start out, I tell the computer, compute for me the value of the solution at one starting out at one.
And, it computes and computes a little while.
But the solution, how does this curve actually look?
So, in other words, suppose I say that y of zero equals one.
Find me y of two.
In other words, take a nice small step size.
Use the Runge Kutta fourth order method.
Calculate a little bit, and tell me, I just want to know what y of two is.
Well, what is y of two?
Well, unfortunately, how does that curve look?
The curve looks like this.
At that point, it drops to infinity in a manner of speaking, and then sort of comes back up again like that.
What is the value of y?
This is the point, one.
What is the value of y of two?
Is it here?
Is it this?
Well, I don't know, but I do know that the computer will not find it.
The computer will follow this along, and get lost in eternity, in infinity, and see no reason whatever why it should start again on this branch of the curve.
Okay, well, can't we predict that that's going to happen somehow, avoid what I should have.
The whole difficulty is, this is called a singular point.
The solution has a singularity, in other words, a single place where it goes to infinity or becomes discontinuous, maybe as a jump discontinuity.
It has a singularity at x equals c. This, in particular, at x equals one here, but from the differential equation, where is that c?
There is no way of predicting it.
Each solution, in other words, to this differential equation, has its own, private singularity, which only it knows about, and where it's going to blow up, and there's no way of telling from the differential equation where that's going to be.
That's one of the things that makes numerical calculation difficult, when you cannot predict where things are going to go bad in advance.
This time, we started solving differential equations.
This is the third lecture of the term, and I have yet to solve a single differential equation in this class.
Well, that will be rectified from now until the end of the term.
So, once you learn separation of variables, which is the most elementary method there is, the single, I think the single most important equation is the one that's called the first order linear equation, both because it occurs frequently in models because it's solvable, and-- I think that's enough.
If you drop the course after today you will still have learned those two important methods: separation of variables, and first order linear equations.
So, what does such an equation look like?
Well, I'll write it in there.
There are several ways of writing it, but I think the most basic is this.
I'm going to use x as the independent variable because that's what your book does.
But in the applications, it's often t, time, that is the independent variable.
And, I'll try to give you examples which show that.
So, the equation looks like this.
I'll find some function of x times y prime plus some other function of x times y is equal to yet another function of x.
Obviously, the x doesn't have the same status here that y does, so y is extremely limited in how it can appear in the equation.
But, x can be pretty much arbitrary in those places.
So, that's the equation we are talking about, and I'll put it up.
This is the first version of it, and we'll call them purple.
Now, why is that called the linear equation?
The word linear is a very heavily used word in mathematics, science, and engineering.
For the moment, the best simple answer is because it's linear in y and y prime, the variables y and y prime.
Well, y prime is not a variable.
Well, you will learn, in a certain sense, it helps to think of it as one, not right now perhaps, but think of it as linear.
The most closely analogous thing would be a linear equation, a real linear equation, the kind you studied in high school, which would look like this.
It would have two variables, and, I guess, constant coefficients, equal c. Now, that's a linear equation.
And that's the sense in which this is linear.
It's linear in y prime and y, which are the analogs of the variables y1 and y2.
A little bit of terminology, if c is equal to zero, it's called homogeneous, the same way this equation is called homogeneous, as you know from 18.02, if the right hand side is zero.
So, c of x I should write here, but I won't.
That's called homogeneous.
Now, this is a common form for the equation, but it's not what it's called standard form.
The standard form for the equation, and since this is going to be a prime course of confusion, which is probably completely correct, but a prime source of confusion is what I meant.
The standard linear form, and I'll underline linear is the first co efficient of y prime is taken to be one.
So, you can always convert that to a standard form by simply dividing through by it.
And if I do that, the equation will look like y prime plus, now, it's common to not call it b anymore, the coefficient, because it's really b over a.
And, therefore, it's common to adopt, yet, a new letter for it.
And, the standard one that many people use is p. How about the right hand side?
We needed a letter for that, too.
It's c over a, but we'll call it q.
So, when I talk about the standard linear form for a linear first order equation, it's absolutely that that I'm talking about.
Now, you immediately see that there is a potential for confusion here because what did I call the standard form for a first order equation?
So, I'm going to say, not this.
The standard first order form, what would that be?
Well, it would be y prime equals, and everything else on the left hand side.
So, it would be y prime.
And now, if I turn this into the standard first order form, it would be negative p of x y plus q of x.
But, of course, nobody would write negative p of x.
So, now, I explicitly want to say that this is a form which I will never use for this equation, although half the books of the world do.
In short, this poor little first order equation belongs to two ethnic groups.
It's both a first order equation, and therefore, its standard form should be written this way, but it's also a linear equation, and therefore its standard form should be used this way.
Well, it has to decide, and I have decided for it.
It is, above all, a linear equation, not just a first order equation.
And, in this course, this will always be the standard form.
Now, well, what on earth is the difference?
If you don't do it that way, the difference is entirely in the sin(p).
But, if you get the sign of p wrong in the answers, it is just a disaster from that point on.
A trivial little change of sign in the answer produces solutions and functions which have totally different behavior.
And, you are going to be really lost in this course.
So, maybe I should draw a line through it to indicate, please don't pay any attention to this whatsoever, except that we are not going to do that.
Okay, well, what's so important about this equation?
Well, number one, it can always be solved.
That's a very, very big thing in differential equations.
But, it's also the equation which arises in a variety of models.
Now, I'm just going to list a few of them.
All of them I think you will need either in part one or part two of problem sets over these first couple of problem sets, or second and third maybe.
But, of them, I'm going to put at the very top of the list of what I'll call here, I'll give it two names: the temperature diffusion model, well, it would be better to call it temperature concentration by analogy, temperature concentration model.
There's the mixing model, which is hardly less important.
In other words, it's almost as important.
You have that in your problem set.
And then, there are other, slightly less important models.
There is the model of radioactive decay.
There's the model of a bank interest, bank account, various motion models, you know, Newton's Law type problems if you can figure out a way of getting rid of the second derivative, some motion problems.
A classic example is the motion of a rocket being fired off, etc., etc., etc.
Now, today I have to pick a model.
And, the one I'm going to pick is this temperature concentration model.
So, this is going to be today's model.
Tomorrow's model in the recitation, I'm asking the recitations to, among other things, make sure they do a mixing problem, A) to show you how to do it, and B) because it's on the problem sets.
That's not a good reason, but it's not a bad one.
The others are either in part one or we will take them up later in the term.
This is not going to be the only lecture on the linear equation.
There will be another one next week of equal importance.
But, we can't do everything today.
So, let's talk about the temperature concentration model, except I'm going to change its name.
I'm going to change its name to the conduction diffusion model.
I'll put conduction over there, and diffusion over here, let's say, since, as you will see, the similarities, they are practically the same model.
All that's changed from one to the other is the name of the ideas.
In one case, you call it temperature, and the other, you should call it concentration.
But, the actual mathematics isn't identical.
So, let's begin with conduction.
All right, so, I need a simple physical situation that I'm modeling.
So, imagine a tank of some liquid.
Water will do as well as anything.
And, in the inside is a suspended, somehow, is a chamber.
A metal cube will do, and let's suppose that its walls are partly insulated, not so much that no heat can get through.
There is no such thing as perfect insulation anyway, except maybe an absolute perfect vacuum.
Now, inside, so here on the outside is liquid.
Okay, on the inside is, what I'm interested in is the temperature of this thing.
I'll call that T. Now, that's different from the temperature of the external water bath.
So, I'll call that T sub e, T for temperature measured in Celsius, let's say, for the sake of definiteness.
But, this is the external temperature.
So, I'll indicate it with an e. Now, what is the model?
Well, in other words, how do I set up a differential equation to model the situation?
Well, it's based on a physical law, which I think you know, you've had simple examples like this, the so called Newton's Law of cooling, -- -- which says that the rate of change, the temperature of the heat goes from the outside to the inside by conduction only.
Heat, of course, can travel in various ways, by convection, by conduction, as here, or by radiation, are the three most common.
Of these, I only want one, namely transmission of heat by conduction.
And, that's the way it's probably a little better to call it the conduction model, rather than the temperature model, which might involve other ways for the heat to be traveling.
So, dt, the independent variable, is not going to be x, as it was over there.
It's going to be t for time.
So, maybe I should write that down.
t equals time.
Capital T equals temperature in degrees Celsius.
So, you can put in the degrees Celsius if you want.
So, it's proportional to the temperature difference between these two.
Now, how shall I write the difference?
Write it this way because if you don't you will be in trouble.
Now, why do I write it that way?
Well, I write it that way because I want this constant to be positive, a positive constant.
In general, any constant, so, parameters which are physical, have some physical significance, one always wants to arrange the equation so that they are positive numbers, the way people normally think of these things.
This is called the conductivity.
The conductivity of what?
Well, I don't know, of the system of the situation, the conductivity of the wall, or the wall if the metal were just by itself.
At any rate, it's a constant.
It's thought of as a constant.
And, why positive, well, because if the external temperature is bigger than the internal temperature, I expect T to rise, the internal temperature to rise.
That means dT / dt, its slope, should be positive.
So, in other words, if Te is bigger than T, I expect this number to be positive.
And, that tells you that k must be a positive constant.
If I had turned it the other way, expressed the difference in the reverse order, K would then be negative, have to be negative in order that this turn out to be positive in that situation I described.
And, since nobody wants negative values of k, you have to write the equation in this form rather than the other way around.
So, there's our differential equation.
It will probably have an initial condition.
So, it could be the temperature at the starting time should be some given number, T zero.
But, the condition could be given in other ways.
One can ask, what's the temperature as time goes to infinity, for example?
There are different ways of getting that initial condition.
Okay, that's the conduction model.
What would the diffusion model be?
The diffusion model, mathematically, would be, word for word, the same.
The only difference is that now, what I imagine is I'll draw the picture the same way, except now I'm going to put, label the inside not with a T but with a C, C for concentration.
It's in an external water bath, let's say.
So, there is an external concentration.
And, what I'm talking about is some chemical, let's say salt will do as well as anything.
So, C is equal to salt concentration inside, and Ce would be the salt concentration outside, outside in the water bath.
Now, I imagine some mechanism, so this is a salt solution.
That's a salt solution.
And, I imagine some mechanism by which the salt can diffuse, it's a diffusion model now, diffuse from here into the air or possibly out the other way.
And that's usually done by vaguely referring to the outside as a semi permeable membrane, semi permeable, so that the salt will have a little hard time getting through but permeable, so that it won't be blocked completely.
So, there's a membrane.
You write the semi permeable membrane outside, outside the inside.
Well, I give up.
You know, membrane somewhere.
Sorry, membrane wall.
How's that?
Now, what's the equation?
Well, the equation is the same, except it's called the diffusion equation.
I don't think Newton got his name on this.
The diffusion equation says that the rate at which the salt diffuses across the membrane, which is the same up to a constant as the rate at which the concentration inside changes, is some constant, usually called k still, okay.
Do I contradict?
Okay, let's keep calling it k1.
Now it's different, times Ce minus C. And, for the same reason as before, if the external concentration is bigger than the internal concentration, we expect salt to flow in.
That will make C rise.
It will make this positive, and therefore, we want k to be positive, just k1 to be positive for the same reason it had to be positive before.
So, in each case, the model that I'm talking about is the differential equation.
So, maybe I should, let's put that, make that clear.
Or, I would say that this first order differential equation models this physical situation, and the same thing is true on the other side over here.
This is the diffusion equation, and this is the conduction equation.
Now, if you are in any doubt about the power of differential equations, the point is, when I talk about this thing, I don't have to say which of these I'm following.
I'll use neutral variables like Y and X to solve these equations.
But, with a single stroke, I will be handling those situations together.
And, that's the power of the method.
Now, you obviously must be wondering, look, these look very, very special.
He said he was going to talk about the first, general first order equation.
But, these look rather special to me.
Well, not too special.
How should we write it?
Suppose I write, let's take the temperature equation just to have something definite.
Notice that it's in a form corresponding to Newton's Law.
But it is not in the standard linear form.
Let's put it in standard linear form, so at least you could see that it's a linear equation.
So, if I put it in standard form, it's going to look like DTDTD little t plus KT is equal to K times TE.
Now, compare that with the general, the way the general equation is supposed to look, the yellow box over there, the standard linear form.
How are they going to compare?
Well, this is a pretty general function.
This is general.
This is a general function of T because I can make the external temperature.
I could suppose it behaves in anyway I like, steadily rising, decaying exponentially, maybe oscillating back and forth for some reason.
The only way in which it's not general is that this K is a constant.
So, I will ask you to be generous.
Let's imagine the conductivity is changing over time.
So, this is usually constant, but there's no law which says it has to be.
How could a conductivity change over time?
Well, we could suppose that this wall was made of slowly congealing Jell O, for instance.
It starts out as liquid, and then it gets solid.
And, Jell O doesn't transmit heat, I believe, quite as well as liquid does, as a liquid would.
Is Jell O a solid or liquid?
I don't know.
Let's forget about that.
So, with this understanding, so let's say not necessarily here, but not necessarily, I can think of this, therefore, by allowing K to vary with time.
And the external temperature to vary with time.
I can think of it as a general, linear equation.
So, these models are not special.
They are fairly general.
Well, I did promise you I would solve an equation, and that this lecture, I still have not solved any equations.
OK, time to stop temporizing and solve.
So, I'm going to, in order not to play favorites with these two models, I'll go back to, and to get you used to thinking of the variables all the time, that is, you know, be eclectic switching from one variable to another according to which particular lecture you happened to be sitting in.
So, let's take our equation in the form, Y prime plus P of XY, the general form using the old variables equals Q of X.
Solve me.
Well, there are different ways of describing the solution process.
No matter how you do it, it amounts to the same amount of work and there is always a trick involved at each one of them since you can't suppress a trick by doing the problem some other way.
The way I'm going to do it, I think, is the best.
That's why I'm giving it to you.
It's the easiest to remember.
It leads to the least work, but I have colleagues who would fight with me about that point.
So, since they are not here to fight with me I am free to do whatever I like.
One of the main reasons for doing it the way I'm going to do is because I want you to get what our word into your consciousness, two words, integrating factor.
I'm going to solve this equation by finding and integrating factor of the form U of X.
What's an integrating factor?
Well, I'll show you not by writing an elaborate definition on the board, but showing you what its function is.
It's a certain function, U of X, I don't know what it is, but here's what I wanted to do.
I want to multiply, I'm going to drop the X's a just so that the thing looks less complicated.
So, what I want to do is multiply this equation through by U of X.
That's why it's called a factor because you're going to multiply everything through by it.
So, it's going to look like UY prime plus PUY equals QU, and now, so far, it's just a factor.
What makes it an integrating factor is that this, after I do that, I want this to turn out to be the derivative of something with respect to X.
You see the motivation for that.
If this turns out to be the derivative of something, because I've chosen U so cleverly, then I will be able to solve the equation immediately just by integrating this with respect to X, and integrating that with respect to X.
You just, then, integrate both sides with respect to X, and the equation is solved.
Now, the only question is, what should I choose for U?
Well, if you think of the product formula, there might be many things to try here.
But there's only one reasonable thing to try.
Try to pick U so that it's the derivative of U times Y.
See how reasonable that is?
If I use the product rule on this, the first term is U times Y prime.
The second term would be U prime times Y.
Well, I've got the Y there.
So, this will work.
It works if, what's the condition that you must satisfy in order for that to be true?
Well, it must be that after it to the differentiation, U prime turns out to be P times U.
So, is it clear?
This is something we want to be equal to, and the thing I will try to do it is by choosing U in such a way that this equality will take place.
And then I will be able to solve the equation.
And so, here's what my U prime must satisfy.
Hey, we can solve that.
But please don't forget that P is P of X.
It's a function of X.
So, if you separate variables, I'm going to do this.
So, what is it, DU over U equals P of X times DX.
If I integrate that, so, separate variables, integrate, and you're going to get DU over U integrates to the be the log of U, and the other side integrates to be the integral of P of X DX.
Now, you can put an arbitrary constant there, or you can think of it as already implied by the indefinite integral.
Well, that doesn't tell us, yet, what U is.
What should U be?
Notice, I don't have to find every possible U, which works.
All I'm looking for is one.
All I want is a single view which satisfies that equation.
Well, U equals the integral, E to the integral of PDX.
That's not too beautiful looking, but by differential equations, things can get so complicated that in a week or two, you will think of this as an extremely simple formula.
So, there is a formula for our integrating factor.
We found it.
We will always be able to write an integrating factor.
Don't worry about the arbitrary constant because you only need one such U.
So: no arbitrary constant since only one U needed.
And, that's the solution, the way we solve the linear equation.
OK, let's take over, and actually do it.
I think it would be better to summarize it as a clear cut method.
So, let's do that.
So, what's our method?
It's the method for solving Y prime plus PY equals Q.
Well, the first place, make sure it's in standard linear form.
If it isn't, you must put it in that form.
Notice, the formula for the integrating factor, the formula for the integrating factor involves P, the integral of PDX.
So, you'd better get the right P. Otherwise, you are sunk.
OK, so put it in standard linear form.
That way, you will have the right P. Notice that if you wrote it in that form, and all you remembered was E to the integral PDX, the P would have the wrong sign.
If you're going to write, that P should have a negative sign there.
So, do it this way, and no other way.
Otherwise, you will get confused and get wrong signs.
And, as I say, that will produce wrong answers, and not just slightly wrong answers, but disastrously wrong answers from the point of view of the modeling if you really want answers to physical problems.
So, here's a standard linear form.
Then, find the integrating factor.
So, calculate E to the integral, PDX, the integrating factor, and that multiply both, I'm putting this as both, underlined that as many times as you have room in your notes.
Multiply both sides by this integrating factor by E to the integral PDX.
And then, integrate.
OK, let's take a simple example.
Suppose we started with the equation XY prime minus Y equals, I had X2, X3, something like that, X3, I think, yeah, X2.
OK, what's the first thing to do?
Put it in standard form.
So, step zero will be to write it as Y prime minus one over X times Y equals X2.
Let's do the work first, and then I'll talk about mistakes.
Well, we now calculate the integrating factor.
So, I would do it in steps.
You can integrate negative one over X, right?
That integrates to minus log X.
So, the integrating factor is E to the integral of this, DX.
So, it's E to the negative log X.
Now, in real life, that's not the way to leave that.
What is E to the negative log X?
Well, think of it as E to the log X to the minus one.
Or, in other words, it is E to the log X is X.
So, it's one over X.
So, the integrating factor is one over X. OK, multiply both sides by the integrating factor.
Both sides of what?
Both sides of this: the equation written in standard form, and both sides.
So, it's going to be one over XY prime minus one over X2 Y is equal to X2 times one over X, which is simply X.
Now, if you have done the work correctly, you should be able, now, to integrate the left hand side directly.
So, I'm going to write it this way.
I always recommend that you put it as extra step, well, put it as an extra step the reason for using that integrating factor, in other words, that the left hand side is supposed to be, now, one over X times Y prime.
I always put it that because there's always a chance you made a mistake or forgot something.
Look at it, mentally differentiated using the product rule just to check that, in fact, it turns out to be the same as the left hand side.
So, what do we get?
One over X times Y prime plus Y times the derivative of one over X, which indeed is negative one over X2.
And now, finally, that's 3A, continue, do the integration.
So, you're going to get, let's see if we can do it all on one board, one over X times Y is equal to X plus a constant, X, sorry, X2 over two plus a constant.
And, the final step will be, therefore, now I want to isolate Y by itself.
So, Y will be equal to multiply through by X. X3 over two plus C times X.
And, that's the solution.
OK, let's do one a little slightly more complicated.
Let's try this one.
Now, my equation is going to be one, I'll still keep two, Y and X, as the variables.
I'll use T and F for a minute or two.
One plus cosine X, so, I'm not going to give you this one in standard form either.
It's a trick question.
Y prime minus sine X times Y is equal to anything reasonable, I guess.
I think X, 2X, make it more exciting.
OK, now, I think I should warn you where the mistakes are just so that you can make all of them.
So, this is mistake number one.
You don't put it in standard form.
Mistake number two: generally people can do step one fine.
Mistake number two is, this is my most common mistake, so I'm very sensitive to it.
But that doesn't mean if you make it, you'll get any sympathy from me.
I don't give sympathy to myself.
You are so intense, so happy at having found the integrating factor, you forget to multiply Q by the integrating factor also.
You just handle the left hand side of the equation, if you forget about the right hand side.
So, the emphasis on the both here is the right hand, please include the Q.
Please include the right hand side.
Any other mistakes?
Well, nothing that I can think of.
Well, maybe only, anyway, we are not going to make any mistakes the rest of this lecture.
So, what do we do?
We write this in standard form.
So, it's going to look like Y prime minus sine X, sine X divided by one plus cosine X times Y equals, my heart sinks because I know I'm supposed to integrate something like this.
And, boy, that's going to give me problems.
Well, not yet.
With the integrating factor?
The integrating factor is, well, we want to calculate the integral of negative sine X over one plus cosine.
That's the integral of PDX.
And, after that, we have to exponentiate it.
Well, can you do this?
Yeah, but if you stare at it a little while, you can see that the top is the derivative of the bottom.
That is great.
That means it integrates to be the log of one plus cosine X.
Is that right, one over one plus cosine X times the derivative of this, which is negative cosine X.
Therefore, the integrating factor is E to that.
In other words, it is one plus cosine X.
Therefore, so this was step zero.
Step one, we found the integrating factor.
And now, step two, we multiply through the integrating factor.
And what do we get?
We multiply through the standard for equation by the integrating factor, if you do that, what you get is, well, Y prime gets the coefficient one plus cosine X, Y prime minus sign X equals 2X.
Oh, dear.
Well, I hope somebody would giggle at this point.
What's giggle able about it?
Well, that all this was totally wasted work.
It's called spinning your wheels.
No, it's not spinning your wheels.
It's doing what you're supposed to do, and finding out that you wasted the entire time doing what you were supposed to do.
Well, in other words, that net effect of this is to end up with the same equation we started with.
But, what is the point?
The point of having done all this was because now the left hand side is exactly the derivative of something, and the left hand side should be the derivative of what?
Well, it should be the derivative of one plus cosine X times Y, all prime.
Now, you can check that that's in fact the case.
It's one plus cosine X, Y prime, plus minus sine X, the derivative of this side times Y.
So, if you had thought, in looking at the equation, to say to yourself, this is a derivative of that, maybe I'll just check right away to see if it's the derivative of one plus cosine X sine.
You would have saved that work.
Well, you don't have to be brilliant or clever, or anything like that.
You can follow your nose, and it's just, I want to give you a positive experience in solving linear equations, not too negative.
Anyway, so we got to this point.
So, now this is 2X, and now we are ready to solve the equation, which is the solution now will be one plus cosine X times Y is equal to X2 plus a constant, and so Y is equal to X2 divided by X2 plus a constant divided by one plus cosine X.
Suppose I have given you an initial condition, which I didn't.
But, suppose the initial condition said that Y of zero were one, for instance.
Then, the solution would be, so, this is an if, I'm throwing in at the end just to make it a little bit more of a problem, how would I put, then I could evaluate the constant by using the initial condition.
What would it be?
This would be, on the left hand side, one, on the right hand side would be C over two.
So, I would get one equals C over two.
Is that correct?
Cosine of zero is one, so that's two down below.
Therefore, C is equal to two, and that would then complete the solution.
We would be X2 plus two over one plus cosine X.
Now, you can do this in general, of course, and get a general formula.
And, we will have occasion to use that next week.
But for now, why don't we concentrate on the most interesting case, namely that of the most linear equation, with constant coefficient, that is, so let's look at the linear equation with constant coefficient, because that's the one that most closely models the conduction and diffusion equations.
So, what I'm interested in, is since this is the, of them all, probably it's the most important case is the one where P is a constant because of its application to that.
And, many of the other, the bank account, for example, all of those will use a constant coefficient.
So, how is the thing going to look?
Well, I will use the cooling.
Let's use the temperature model, for example.
The temperature model, the equation will be DTDT plus KT is equal to.
Now, notice on the right hand side, this is a common error.
You don't put TE.
You have to put KTE because that's what the equation says.
If you think units, you won't have any trouble.
Units have to be compatible on both sides of a differential equation.
And therefore, whatever the units were for capital KT, I'd have to have the same units on the right hand side, which indicates I cannot have KT on the left of the differential equation, and just T on the right, and expect the units to be compatible.
That's not possible.
So, that's a good way of remembering that if you're modeling temperature or concentration, you have to have the K on both sides.
OK, let's do, now, a lot of this we are going to do in our head now because this is really too easy.
What's the integrating factor?
Well, the integrating factor is going to be the integral of K, the coefficient now is just K. P is a constant, K, and if I integrate KDT, I get KT, and I exponentiate that.
So, the integrating factor is E to the KT.
I multiply through both sides, multiply by E to the KT, and what's the resulting equation?
Well, it's going to be , I'll write it in the compact form.
It's going to be E to the KT times T, all prime.
The differentiation is now, of course, with respect to the time.
And, that's equal to KTE, whatever that is, times E to the KT.
This is a function of T, of course, the function of little time, sorry, little T time.
OK, and now, finally, we are going to integrate.
What's the answer?
Well, it is E to the, so, are we going to get E to the KT times T is, sorry, K little t, K times time times the temperature is equal to the integral of KTE.
I'll put the fact that it's a function of T inside just to remind you, E to the KT, and now I'll put the arbitrary constant.
Let's put in the arbitrary constant explicitly.
So, what will T be?
OK, T will look like this, finally.
It will be E to the negative KT.
That's on the outside.
Then, you will integrate.
Of course, the difficulty of doing this integral depends entirely upon how this external temperature varies.
But anyways, it's going to be K times that function, which I haven't specified, E to the KT plus C times E to the negative KT.
Now, some people, many, in fact, that almost always, in the engineering literature, almost never write indefinite integrals because an indefinite integral is indefinite.
In other words, this covers not just one function, but a whole multitude of functions which differ from each other by an arbitrary constant.
So, in a formula like this, there's a certain vagueness, and it's further compounded by the fact that I don't know whether the arbitrary constant is here.
I seem to have put it explicitly on the outside the way you're used to doing from calculus.
Many people, therefore, prefer, and I think you should learn this, to do what is done in the very first section of the notes called definite integral solutions.
If there's an initial condition saying that the internal temperature at time zero is some given value, what they like to do is make this thing definite by integrating here from zero to T, and making this a dummy variable.
You see, what that does is it gives you a particular function, whereas, I'm sorry I didn't put in the DT one minus two.
What it does is that when time is zero, all this automatically disappears, and the arbitrary constant will then be, it's T. So, in other words, C times this, which is one, is that equal to [T?].
In other words, if I make this zero, that I can write C as equal to this arbitrary starting value.
Now, when you do this, the essential thing, and we're going to come back to this next week, but right away, because K is positive, I want to emphasize that so much at the beginning of the period, I want to conclude by showing you what its significance is.
This part disappears because K is positive.
The conductivity is positive.
This part disappears as T goes to zero.
This goes to zero as T goes to infinity.
So, this is a solution that remains.
This, therefore, is called the steady state solution, the thing which the temperature behaves like, as T goes to infinity.
This is called the transient.
because it disappears as T goes to infinity.
It depends on the initial condition, but it disappears, which shows you, then, in the long run for this type of problem the initial condition makes no difference.
The function behaves always the same way as T goes to infinity.
The topic for today is how to change variables.
So, we're talking about substitutions and differential equations, or changing variables.
That might seem like a sort of fussy thing to talk about in the third or fourth lecture, but the reason is that so far, you know how to solve two kinds of differential equations, two kinds of first-order differential equations, one where you can separate variables, and the linear equation that we talked about last time.
Now, the sad fact is that in some sense, those are the only two general methods there are, that those are the only two kinds of equations that can always be solved.
Well, what about all the others?
The answer is that to a great extent, all the other equations that can be solved, the solution can be done by changing the variables in the equation to reduce it to one of the cases that we can already do.
Now, I'm going to give you two examples of that, two significant examples of that today.
But, ultimately, as you will see, the way the equations are solved is by changing them into a linear equation, or an equation where the variables are separable.
However, that's for a few minutes.
The first change of variables that I want to talk about is an almost trivial one.
But it's the most common kind there is, and you've already had it in physics class.
But I think it's so important in the science and engineering subjects that it's a good idea, even in 18.03, to call attention to it explicitly.
So, in that sense, the most common change of variables is the one simple one called scaling.
So, again, the kind of equation I'm talking about is a general first-order equation.
And, scaling simply means to change the coordinates, in effect, or axes, to change the coordinates on the axes to scale the axes, to either stretch them or contract them.
So, what does the change of variable actually look like?
Well, it means you introduce new variables, where x1 is equal to x times something or times a constant.
I'll write it as divided by a constant, since that tends to be a little bit more the way people think of it.
And y, the same.
So, the new variable y1 is related to the old one by an equation of that form.
So, a, b are constants.
So, those are the equations.
Why does one do this?
Well, for a lot of reasons.
But, maybe we can list them.
You, for example, could be changing units.
That's a common reason in physics.
Changing the units that he used, you would have to make a change of coordinates of this form.
Perhaps the even more important reason is to, sometimes it's used to make the variables dimensionless.
In other words, so that the variables become pure numbers, with no units attached to them.
Since you are well aware of the tortures involved in dealing with units in physics, the point of making variables, I'm sorry, dimensionless, I don't have to sell that.
Dimensionless, i.e.
no units, without any units attached.
It just represents the number three, not three seconds, or three grahams, or anything like that.
And, the third reason is to reduce or simplify the constants: reduce the number or simplify the constants in the equation.
Reduce their number is self explanatory.
Simplify means make them less, either dimensionless also, or if you can't do that, at least less dependent upon the critical units than the old ones were.
Let me give you a very simple example which will illustrate most of these things.
It's the equation.
It's a version of the cooling law, which applies at very high temperatures, and it runs.
So, it's like Newton's cooling laws, except it's the internal and external temperatures vary, what's important is not the first power as in Newton's Law, but the fourth power.
So, it's a constant.
And, the difference is, now, it's the external temperature, which, just so there won't be so many capital T's in the equation, I'm going to call M, to the forth power minus T to the forth power.
So, T is the internal temperature, the thing we are interested in.
And, M is the external constant, which I'll assume, now, is a constant external temperature.
So, this is valid if big temperature differences, Newton's Law, breaks down and one needs a different one.
Now, you are free to solve that equation just as it stands, if you can.
There are difficulties connected with it because you're dealing with the fourth powers, of course.
But, before you do that, one should scale.
How should I scale?
Well, I'm going to scale by relating T to M. So, that is very likely to use is T1 equals T divided by M. This is now dimensionless because M, of course, has the units of temperature, degrees Celsius, degrees absolute, whatever it is, as does T. And therefore, by taking the ratio of the two, there are no units attached to it.
So, this is dimensionless.
Now, how actually do I change the variable in the equation?
Well, watch this.
It's an utterly trivial idea, and utterly important.
Don't slog around doing it this way, trying to stuff it in, and divide first.
Instead, do the inverse.
In other words, write it instead as T equals MT1, the reason being that it's T that's facing you in that equation, and therefore T you want to substitute for.
So, let's do it.
The new equation will be what?
Well, dT-- Since this is a constant, the left-hand side becomes dT1 / dt times M equals k times M to the forth minus M to the forth T1 to the forth, so I'm going to factor out that M to the forth, and make it one minus T1 to the forth, okay?
Now, I could divide through by M and get rid of one of those, and so, the new equation, now, is dT1 / dt, d time, is equal to-- Now, I have k M cubed out front here.
I'm going to just give that a new name, k1.
Essentially, it's the same equation.
It's no harder to solve and no easier to solve than the original one.
But it's been simplified.
For one, I think it looks better.
So, to compare the two, I'll put this one up in green, and this one in green, too, just to convince you it's the same, but indicate that it's the same equation.
Notice, so, T1 has been rendered, is now dimensionless.
So, I don't have to even ask when I solve this equation, oh, please tell me what the units of temperature are.
How you are measuring temperature makes no difference to this equation.
k1 still has units.
What units does it have?
It's been simplified because it now has the units of, since this is dimensionless and this is dimensionless, it has the units of inverse time.
So, k1, whereas it had units involving both degrees and seconds before, now it has inverse time as its units.
And, moreover, there's one less constant.
So, one less constant in the equation.
It just looks better.
This business, I think you know that k1, the process of forming k1 out of k M cubed is called lumping constants.
I think they use standard terminology in physics and engineering courses.
Try to get all the constants together like this.
And then you lump them there.
They are lumped for you, and then you just give the lump a new name.
So, that's an example of scaling.
Watch out for when you can use that.
For example, it would have probably been a good thing to use in the first problem set when you were handling this problem of drug elimination and hormone elimination production inside of the thing.
You could lump constants, and as was done to some extent on the solutions to get a neater looking answer, one without so many constants in it.
Okay, let's now go to serious stuff, where we are actually going to make changes of variables which we hope will render unsolvable equations suddenly solvable.
Now, I'm going to do that by making substitutions.
But, it's, I think, quite important to watch up there are two kinds of substitutions.
There are direct substitutions.
That's where you introduce a new variable.
I don't know how to write this on the board.
I'll just write it schematically.
So, it's one which says that the new variable is equal to some combination of the old variables.
The other kind of substitution is inverse.
It's just the reverse.
Here, you say that the old variables are some combination of the new.
Now, often you'll have to stick in a few old variables, too.
But the basic, it's what appears on the left-hand side.
Is it a new variable that appears on the left-hand side by itself, or is it the old variable that appears on the left-hand side?
Now, right here, we have an example.
If I did it as a direct substitution, I would have written T1 equals T over M. That's the way I define the new variable, which of course you have to do if you're introducing it.
But when I actually did the substitution, I did the inverse substitution.
Namely, I used T equals T1, M times T1.
And, the reason for doing that was because it was the capital T's that faced me in the equation and I had to have something to replace them with.
Now, you see this already in calculus, this distinction.
But that might have been a year and a half ago.
Just let me remind you, typically in calculus, for example, when you want to do this kind of integral, let's say, x times the square root of one minus x squared dx, the substitution you'd use for that is u equals one minus x squared, right?
And then, you calculate, and then you would observe that this, the x dx, more or less makes up du, apart from a constant factor, there.
So, this would be an example of direct substitution.
You put it in and convert the integral into an integral of u.
What would be an example of inverse substitution?
Well, if I take away the x and ask you, instead, to do this integral, then you know that the right thing to do is not to start with u, but to start with the x and write x equals sine or cosine u.
So, this is a direct substitution in that integral, but this integral calls for an inverse substitution in order to be able to do it.
And notice, they would look practically the same.
But, of course, as you know from your experience, they are not.
They're very different.
Okay, so I'm going to watch for that distinction as I do these examples.
The first one I want to do is an example as a direct substitution.
So, it applies to the equation of the form y prime equals, there are two kinds of terms on the right-hand side.
Let's use p of x, p of x times y plus q of x times any power whatsoever of y.
Well, notice, for example, if n were zero, what kind of equation would this be?
y to the n would be one, and this would be a linear equation, which you know how to solve.
So, n equals zero we already know how to do.
So, let's assume that n is not zero, so that we're in new territory.
Well, if n were equal to one, you could separate variables.
So, that too is not exciting.
But, nonetheless, it will be included in what I'm going to say now.
If n is two or three, or n could be one half.
So anything: even zero is all right.
It's just silly.
Any number: it could be negative.
n equals minus five.
That would be fine also.
This kind of equation, to give it its name, is called the Bernoulli equation, named after which Bernoulli, I haven't the faintest idea.
There were, I think, three or four of them.
And, they fought with each other.
But, they were all smart.
Now, the key trick, if you like, method, to solving any Bernoulli equation, let me call another thing.
Most important is what's missing.
It must not have a pure x term in it.
And that goes for a constant term.
In other words, it must look exactly like this.
Everything multiplied by y, or a power of y, two terms.
So, for example, if I add one to this, the equation becomes non-doable.
Right, it's very easy to contaminate it into an equation that's unsolvable.
It's got to look just like that.
Now, you've got one on your homework.
You've got several.
Both part one and part two have Bernoulli equations on them.
So, this is practical, in some sense.
What do we got?
The idea is to divide by y to the n. Ignore all formulas that you're given.
Just remember that when you see something that looks like this, or something that you can turn into something that looks like this, divide through by y to the nth power, no matter what n is.
All right, so y prime over y to the n is equal to p of x times one over y to the n minus one, right, plus q of x.
Well, that certainly doesn't look any better than what I started with.
And, in your terms, it probably looks somewhat worse because it's got all those Y's at the denominator, and who wants to see them there?
But, look at it.
In this very slightly transformed Bernoulli equation is a linear equation struggling to be free.
Where is it?
Why is it trying to be a linear equation?
Make a new variable, call this hunk of it in new variable.
Let's call it V. So, V is equal to one over y to the n minus one.
Or, if you like, you can think of that as y to the one minus n. What's V prime?
So, this is the direct substitution I am going to use, but of course, the problem is, what am I going to use on this?
Well, the little miracle happens.
What's the derivative of this?
It is one minus n times y to the negative n times y prime In other words, up to a constant, this constant factor, one minus n, it's exactly the left-hand side of the equation.
Well, let's make N not equal one either.
As I said, you could separate variables if n equals one.
What's the equation, then, turned into?
A Bernoulli equation, divided through in this way, is then turned into the equation one minus n, sorry, V prime divided by one minus n is equal to p of x times V plus q of x.
It's linear.
And now, solve it as a linear equation.
Solve it as a linear equation.
You notice, it's not in standard form, not in standard linear form.
To do that, you're going to have to put the p on the other side.
That's okay, that term, on the other side, solve it, and at the end, don't forget that you put in the V. It wasn't in the original problem.
So, you have to convert the problem, the answer, back in terms of y.
It'll come out in terms of V, but you must put it back in terms of y.
Let's do a really simple example just to illustrate the method, and to illustrate the fact that I don't want you to memorize formulas.
Learn methods, not final formulas.
So, suppose the equation is, let's say, y prime equals y over x minus y squared.
That's a Bernoulli equation.
I could, of course, have concealed it by writing xy prime plus xy prime minus xy equals negative y squared.
Then, it wouldn't look instantly like a Bernoulli equation.
You would have to stare at it a while and say, hey, that's a Bernoulli equation.
Okay, but so I'm handing it to you a silver platter, as it were.
So, what do we do?
Divide through by y squared.
So, it's y prime over y squared equals one over x times one over y minus one.
And now, the substitution, then, I'm going to make, is for this thing.
V equals one over y.
It's a direct substitution.
V prime is going to be negative one over y squared times y prime.
Don't forget to use the chain rule when you differentiate with respect-- because the differentiation is with respect to x, of course, not with respect to y.
Okay, so what's this thing?
That's the left-hand side.
The only thing is it's got a negative sign.
So, this is minus V prime equals, one over x stays one over x, one over y.
So, it's V over x minus one.
So, let's put that in standard form.
In standard form, it will look like, first imagine multiplying it through by negative one, and then putting this term on the other side.
And, it will turn into V prime plus V over X is equal to one.
So, that's the linear equation in standard linear form that we are asked to solve.
And, the solution isn't very hard.
The integrating factor is, well, I integrate one over x.
That makes log x.
And, e to the log x, so, it's e to the log x, which is, of course, just x itself.
So, I should multiply this through by x squared, be able to integrate it.
Now, some of you, I would hope, just can see that right away, that if you multiply this through by x, it's going to look good.
So, after we multiply through by x, which I get?
(xV) prime for the-- maybe I shouldn't skip a step.
You are still learning this stuff, so let's not skip a step.
So, it becomes x V prime plus V equals x, okay?
After I multiplied through by the integrating factor, this now says this is xV prime, and I quickly check that that, in fact, is what it's equal to, equals x, and therefore xV is equal to one half x squared plus a constant.
And, therefore, V is equal to one half x plus C over x.
You can leave it at that form, or you can combine terms.
It doesn't matter much.
Am I done?
The answer is, no I am not done, because nobody reading this answer would know what V was.
V wasn't in the original problem.
It was y that was in the original problem.
And therefore, the relation is, one is the reciprocal of the other.
And therefore, I have to turn this expression upside down.
Well, if you're going to have to turn it upside down, you probably should make it look a little better.
Let's rewrite it as x squared plus 2c, combining fractions, I think they call it in high school or elementary school, plus 2c.
How's that?
x squared plus 2c divided by 2x.
Now, 2c, you don't call it constant 2c because this is just as arbitrary to call it c1.
So, I'll call that, so, my answer will be y equals 2x divided by x squared plus an arbitrary constant.
But, to indicate it's different from that one, I'll call it C1.
C1 is two times the old one, but that doesn't really matter.
So, there's the solution.
It has an arbitrary constant in it, but you note it's not an additive arbitrary constant.
The arbitrary constant is tucked into the solution.
If you had to satisfy an initial condition, you would take this form, and starting from this form, figure out what C1 was in order to satisfy that initial condition.
Thus, Bernoulli equation is solved.
Our first Bernoulli equation: isn't that exciting?
So, here was the equation, and there is its solution.
Now, the one I'm asking you to solve on the problem set in part two is no harder than this, except I ask you some hard questions about it, not very hard, but a little hard about it.
I hope you will find them interesting questions.
You already have the experimental evidence from the first problem set, and the problem is to explain the experimental evidence by actually solving the equation in the scene.
I think you'll find it interesting.
But, maybe that's just a pious hope.
Okay, I like, now, to turn to the second method, where a second class of equations which require inverse substitution, and those are equations, which are called homogeneous, a highly overworked word in differential equations, and in mathematics in general.
But, it's unfortunately just the right word to describe them.
So, these are homogeneous, first-order ODE's.
Now, I already used the word in one context in talking about the linear equations when zero is the right hand side.
This is different, but nonetheless, the two uses of the word have the same common source.
The homogeneous differential equation, homogeneous newspeak, is y prime equals, it's a question of what the right hand side looks like.
And, now, the supposed way to say it is, you should be able to write the right-hand side as a function of a combined variable, y divided by x.
In other words, after fooling around with the right hand side a little bit, you should be able to write it so that every time a variable appears, it's always in the combination y over x.
Let me give some examples.
For example, suppose y prime were, let's say, x squared y divided by x cubed plus y cubed.
Well, that doesn't look in that form.
Well, yes it is.
Imagine dividing the top and bottom by x cubed.
What would you get?
The top would be y over x, if you divided it by x cubed.
And, if I divide the bottom by x cubed, also, which, of course, doesn't change the value of the fraction, as they say in elementary school, one plus (y over x) cubed.
So, this is the way it started out looking, but you just said ah-ha, that was a homogeneous equation because I could see it could be written that way.
How about another homogeneous equation?
How about x y prime?
Is that a homogeneous equation?
Of course it is: otherwise, why would I be talking about it?
If you divide through by x, you can tuck it inside the radical, the square root, if you remember to square it when you do that.
And, it becomes the square root of x squared over x squared, which is one, plus y squared over x squared.
It's homogeneous.
Now, you might say, hey, this looks like you had to be rather clever to figure out if an equation is homogeneous.
Is there some other way?
Yeah, there is another way, and it's the other way which explains why it's called homogeneous.
You can think of it this way.
It's an equation which is, in modern speak, invariant, invariant under the operation zoom.
What is zoom?
Zoom is, you increase the scale equally on both axes.
So, the zoom operation is the one which sends x into a times x, and y into a times y.
In other words, you change the scale on both axes by the same factor, a.
Now, what I say is, gee, maybe you shouldn't write it like this.
Why don't we say, we introduce, how about this?
So, think of it as a change of variables.
We will write it like that.
So, you can put here an equals sign, if you don't know what this meaningless arrow means.
So, I'm making this change of variables, and I'm describing it as an inverse substitution.
But of course, it wouldn't make any difference.
It's exactly the same as the direct substitution I started out with underscaling.
The only difference is, I'm not using different scales on both axes.
I'm expanding them both equally.
That's what I mean by zoom.
Now, what happens to the equation?
Well, what happens to dy over dx?
Well, dx is a dx1.
dy is a dy1.
And therefore, the ratio, dy by dx is the same as dy1 over dx1.
So, the left-hand side becomes dy1 over dx1, and the right-hand side becomes F of, well, y over x is the same as y over, since I've scaled them equally, this is the same as y1 over x1.
So, it's y1 over x1, and the net effect is I simply, everywhere I have an x, I change it to x1, and wherever I have a y, I change it to y1, which, what's in a name?
It's the identical equation.
So, I haven't changed the equation at all via zoom transformation.
And, that's what makes it homogeneous.
That's not an uncommon use of the word homogeneous.
When you say space is homogeneous, every direction, well, that means, I don't know.
It means, okay, I'm getting into trouble there.
I'll let it go since I can't prepare any better, I haven't prepared any better explanation, but this is a pretty good one.
Okay, so, suppose we've got a homogeneous equation.
How do we solve it?
So, here's our equation, F of y over x.
Well, what substitution would you like to make?
Obviously, we should make a direct substitution, z equals y over x.
So, why did he say that this was going to be an example of inverse substitution?
Because I wanted to confuse you.
But look, that's fine.
If you write it in that form, you'll know exactly what to do with the right-hand side.
And, this is why everybody loves to do that.
But for Charlie, you have to substitute into the left-hand side as well.
And, I can testify, for many years of looking with sinking heart at examination papers, what happens if you try to make a direct substitution like this?
You say, oh, I need z prime.
z prime equals, well, I better use the quotient rule for differentiating that.
And, it comes out this long, and then either a long pause, what do I do now?
Because it's not at all obvious what to do at that point.
Or, much worse, two pages of frantic calculations, and giving up in total despair.
Now, the reason for that is because you tried to do it making a direct substitution.
All you have to do instead is use it, treat it as an inverse substitution, write y equals zx.
What's the motivation for doing that?
It's clear from the equation.
This goes through all of mathematics.
Whenever you have to change a variable, excuse me, whenever you have to change a variable, look at what you have to substitute for, and focus your attention on that.
I need to know what y prime is.
Okay, well, then I better know what y is.
If I know what y is, do I know what y prime is?
Oh, of course.
y prime is z prime x plus z times the derivative of this factor, which is one.
And now, I turned with that one stroke, the equation has now become z prime x plus z is equal to F of z.
Well, I don't know.
Can I solve that?
Sure.
That can be solved because this is x times dz / dx.
Just put the z on the other side, it's F of z minus z.
And now, this side is just a function of z.
Separate variables.
And, the only thing to watch out for is, at the end, the z was your business.
You've got to put the answer back in terms of z and y.
Okay, let's work an example of this.
Since I haven't done any modeling yet this period, let's make a little model, differential equations model.
It's a physical situation, which will be solved by an equation.
And, guess what?
The equation will turn out to be homogeneous.
Okay, so the situation is as follows.
We are in the Caribbean somewhere, a little isolated island somewhere with a little lighthouse on it at the origin, and a beam of light shines from the lighthouse.
The beam of light can rotate the way the lighthouse beams.
But, this particular beam is being controlled by a guy in the lighthouse who can aim it wherever he wants.
And, the reason he's interested in aiming it wherever he wants is there's a drug boat here, [LAUGHTER] which has just been caught in the beam of light.
So, the drug boat, which has just been caught in a beam of light, and feels it'd a better escape.
Now, the lighthouse keeper wants to keep the drug boat; the light is shining on it so that the U.S. Coast Guard helicopters can zoom over it and do whatever they do to drug boats, -- -- I don't know.
So, the drug boat immediately has to follow an escape strategy.
And, the only one that occurs to him is, well, he wants to go further away, of course, from the lighthouse.
On the other hand, it doesn't seem sensible to do it in a straight line because the beam will keep shining on him.
So, he fixes the boat at some angle, let's say, and goes off so that the angle stays 45 degrees.
So, it goes so that the angle between the beam and maybe, draw the beam a little less like a 45 degree angle.
So, the angle between the beam and the boat, the boat's path is always 45 degrees, goes at a constant 45 degree angle to the beam, hoping thereby to escape.
On the other hand, of course, the lighthouse guy keeps the beam always on the boat.
So, it's not clear it's a good strategy, but this is a differential equations class.
The question is, what's the path of the boat?
What's the boat's path?
Now, an obvious question is, why is this a problem in differential equations at all?
In other words, looking at this, you might scratch your head and try to think of different ways to solve it.
But, what suggests that it's going to be a problem in differential equations?
The answer is, you're looking for a path.
The answer is going to be a curve.
A curve means a function.
We are looking for an unknown function, in other words.
And, what type of information do we have about the function?
The only information we have about the function is something about its slope, that its slope makes a constant 45° angle with the lighthouse beam.
Its slope makes a constant known angle to a known angle.
Well, if you are trying to find a function, and all you know is something about its slope, that is a problem in differential equations.
Well, let's try to solve it.
Well, let's see.
Well, let me draw just a little bit.
So, here's the horizontal.
Let's introduce the coordinates.
In other words, there's the horizontal and here's the boat to indicate where I am with respect to the picture.
So, here's the boat.
Here's the beam, and the path of the boat is going to make a 45° angle with it.
So, this is the path that we are talking about.
And now, let's label what I know.
Well, this angle is 45°.
This angle, I don't know, but of course I can calculate it easily enough because it has to do with, if I know the coordinates of this point, (x, y), then of course that horizontal angle, I know the slope of this line, and that angle will be related to the slope.
So, let's call this alpha.
And now, what I want to know is what the slope of the whole path is.
So, y prime-- let's call y equals y of x, the unknown function whose path, whose graph is going to be the boat's path, unknown graph.
What's its slope?
Well, its slope is the tangent of the sum of these two angles, alpha plus 45°.
Now, what do I know?
Well, I know that the tangent of alpha is how much?
That's y over x.
In other words, if this was the point, x over y, this is the angle it makes with a horizontal, if you think of it over here.
So, this angle is the same as that one, and it's y over, its slope of that line is y over x.
So, the tangent of the angle is y over x.
How about the tangent of 45°?
That's one, and there's a formula.
This is the hard part.
All you have to know is that the formula exists, and then you will look it up if you have forgotten it, relating the tangent or giving you the tangent of the sum of two angles, and you can, if you like, clever, derive it from the formula for the sign and cosine of the sum of two angles.
But, one peak is worth a thousand finesses.
So, it is the tangent of alpha plus the tangent of 45°.
Let me read it out in all its gory details, divided by one, so you'll at least learn the formula, one minus tangent alpha times tangent 45°.
This would work for the tangent of the sum of any two angles.
That's the formula.
So, what do I get then?
y prime is equal to the tangent of alpha, which is y over x, oh, I like that combination, plus one, divided by (one minus y over x times one).
Now, there is no reason for doing anything to it, but let's make it look a little prettier, and thereby, make it less obvious that it's a homogeneous equation.
If I multiply top and bottom by x, it looks prettier.
x plus y over x minus y equals y prime.
That's our differential equation.
But, notice, that let step to make it look pretty has undone the good work.
It's fine if you immediately recognize this as being a homogeneous equation because you can divide the top and bottom by x.
But here, it's a lot clearer that it's a homogeneous equation because it's already been written in the right form.
Okay, let's solve it now, since we know what to do.
We're going to use as the new variable, z equals y over x.
And, as I wrote up there for y prime, we'll substitute z prime x plus z.
And, with that, let's solve.
Let's solve it.
The equation becomes z prime x plus z is equal to z plus one over one minus z.
We want to separate variables, so you have to put all the z's on one side.
So, this is going to be x, dz / dx equals this thing minus z, which is (z plus one) over (one minus z) minus z.
And now, as you realize, putting it on the other side, I'm going to have to turn it upside down.
Just as before, if you have to turn something upside down, it's better to combine the terms, and make it one tiny little fraction.
Otherwise, you are in for quite a lot of mess if you don't do this nicely.
So, z plus one minus z, that gets rid of the z's.
The numerator is one minus z squared over one minus z, I hope, one, is that right, (one plus z squared) over (one minus z).
And so, the question is dz, and put this on the other side and turn it upside down.
So, that will be (one minus z) over (one plus z squared) on the left-hand side and on the right-hand side, dx over x.
Well, that's ready to be integrated just as it stands.
The right-hand side integrates to be log x.
The left-hand side is the sum of two terms.
The integral of one over one plus z squared is the arc tangent of z, maybe?
The derivative of this is one over one plus z squared.
How about the term z over one plus z squared?
Well, that integrates to be a logarithm.
It is more or less the logarithm of one plus z squared.
If I differentiate this, I get one over one plus z^2 times 2z, but I wish I had negative z there instead.
Therefore, I should put a minus sign, and I should multiply that by half to make it come out right.
And, this is log x on the right hand side plus, put in that arbitrary constant.
And now what?
Well, let's now fool around with it a little bit.
The arc tangent, I'm going to simultaneously, no, two steps.
I have to remember your innocence, although probably a lot of you are better calculators than I am.
I'm going to change this, use as many laws of logarithms as possible.
I'm going to put this in the exponent, and put this on the other side.
That's going to turn it into the log of the square root of one plus z squared.
And, this is going to be plus the log of x plus c. And, now I'm going to make, go back and remember that z equals y over x.
So, this becomes the arc tangent of y over x equals.
Now, I combine the logarithms.
This is the log of x times this square root, right, make one logarithm out of it, and then put z equals y over z.
And, you see that if you do that, it'll be the log of x times the square root of one plus (y over x) squared, and what is that?
Well, if I put this over x squared and take it out, it cancels that.
And, what you are left with is the log of the square root of x squared plus y squared plus a constant.
Now, technically, you have solved the equation, but not morally because, I mean, my God, what a mess!
Incredible path.
It tells me absolutely nothing.
Wow, what is the screaming?
Change me to polar coordinates.
What's the arc tangent of y over x?
Theta.
In polar coordinates it's theta.
This is r. So, the curve is theta equals the log of r plus a constant.
And, I can make even that little better if I exponentiate everything, exponentiate both sides, combine this in the usual way, the and what you get is that r is equal to some other constant times e to the theta.
That's the curve.
It's called an exponential spiral, and that's what our little boat goes in.
And notice, probably if I had set up the problem in polar coordinates from the beginning, nobody would have been able to solve it.
But, anyone who did would have gotten that answer immediately.
Thanks.
Today, once again, a day of solving no differential equations whatsoever.
The topic is a special kind of differential equation, which occurs a lot.
It's one in which the right-hand side doesn't have any independent variable in it.
Now, since I'm going to use as the independent variable, t for time, maybe it would be better to write the left-hand side to let you know, since you won't be able to figure out any other way what it is, dy dt.
We will write it this time.
dy dt is equal to, and the point is that there is no t on the right hand side.
So, there's no t. There's a name for such an equation.
Now, some people call it time independent.
The only problem with that is that sometimes the independent variable is a time.
It's something else.
We need a generic word for there being no independent variable on the right-hand side.
So, the word that's used for that is autonomous.
So, that means no independent variable on the right-hand side.
It's a function of y alone, the dependent variable.
Now, your first reaction should be, oh, well, big deal.
Big deal.
If there's no t on the right hand side, then we can solve this by separating variables.
So, why has he been talking about it in the first place?
So, I admit that.
We can separate variables, and what I'm going to talk about today is how to get useful information out of the equation about how its solutions look without solving the equation.
The reason for wanting to do that is, A, it's fast.
It gives you a lot of insight, and the actual solution, I'll illustrate one, in the first place, take you quite a while.
You may not be able to actually do the integrations, the required and separation of variables to get an explicit solution, or it might simply not be worth the effort of doing if you only want certain kinds of separations about the solution.
So, the thing is, the problem is, therefore, to get qualitative information about the solutions without actually solving -- Without actually having to solve the equation.
Now, to do that, let's take a quick look at how the direction fields of such an equation, after all, it's the direction field is our principal tool for getting qualitative information about solutions without actually solving.
So, how does the direction field look?
Well, think about it for just a second, and you will see that every horizontal line is an isocline.
So, the horizontal lines, what are their equations?
This is the t axis.
And, here's the y axis.
The horizontal lines have the formula y equals a constant.
Let's make it y equals a y zero for different values of the constant y zero.
Those are the horizontal lines.
And, the point is they are isoclines.
Why?
Well, because along any one of these horizontal lines, I'll draw one in, what are the slopes of the line elements?
The slopes are dy / dt is equal to f of y zero, but that's a constant because there's no t to change as you move in the horizontal direction.
The slope is a constant.
So, if I draw in that isocline, I guess I've forgotten, as our convention, isoclines are in dashed lines, but if you have color, you are allowed to put them in living yellow.
Well, I guess I could make them solid, in that case.
I don't have to make a dash.
Then, all the line elements, you put them in at will because they will all have, they are all the same, and they have slope, that, f of y0.
And, similarly down here, they'll have some other slope.
This one will have some other slope.
Whatever, this is the y zero, the value of it, and whatever that happens to be.
I'll put it one more.
That's the x-axis.
I can use the x-axis.
That's an isocline, too.
Now, what do you deduce about how the solutions must look?
Well, let's draw one solution.
Suppose one solution looks like this.
Well, that's an integral curve, in other words.
Its graph is a solution.
Now, as I slide along, these slope elements stay exactly the same, I can slide this curb along horizontally, and it will still be an integral curve everywhere.
So, in other words, they integral curves are invariant under translation for an equation of this type.
They all look exactly the same, and you get them all by taking one, and just pushing it along.
Well, that's so simple it's almost uninteresting, except in that these equations occur a lot in practice.
They are often hard to integrate directly.
And, therefore, it's important to be able to get information about them.
Now, how does one do that?
There's one critical idea, and that is the notion of a critical point.
These equations have what are called critical points.
And, what it is is very simple.
There are three ways of looking at it: critical point, y zero; what does it mean for y0 to be a critical point?
It means, another way of saying it is that it should be a zero of the right-hand side.
So, if I ask you to find the critical points for the equation, what you will do is solve the equation f of y equals zero.
Now, what's interesting about them?
Well, for a critical point, what would be the slope of the line element along, if this is at a critical level, if that's a critical point?
Look at that isocline.
What's the slope of the line elements along it?
It is zero.
And therefore, for these guys, these are, in other words, our solution curves.
But let's prove it formally.
So, there are three ways of saying it.
y zero is a critical point.
It's a zero of the right-hand side, or, y equals y0 is a solution to the equation.
Now, that's perfectly easy to verify.
If y zero makes this right-hand side zero, it's certainly also y equals y0 makes the left-hand side zero because you're differentiating a constant.
So, the reasoning, if you want reasoning, is proof.
Maybe we can make one line out of a proof.
To say that it's a solution, what does it mean to say that it's a solution?
It means to say that when you plug it in, plug in this constant function, y0, the dy0 dt is equal to f of y0.
Is that true?
Yeah.
Both sides are zero.
It's true.
Now, y0 is not a number.
Well, it is.
It's a number on this side, but on this side, what I mean is a constant function whose constant value is y zero, this function, and its derivatives are zero because it has slope zero everywhere.
So, this guy is a constant function, has slope zero.
This is a number which makes the right-hand side zero.
Well, that's nice.
So, in other words, what we found are, by finding these critical points, solving that equation, we found all the horizontal solutions.
But, what's so good about those?
Surely, they must be the most interesting solutions there are.
Well, think of how the picture goes.
Let's draw in one of those horizontal solutions.
So, here's a horizontal solution.
That's a solution.
So, this is my y0.
That's the height at which it is.
And, I'm assuming that f of y0 equals zero.
So, that's a solution.
Now, the significance of that is, because it's a solution, in other words, it's an integral curve, remember what's true about integral curves.
Other curves are not allowed to cross them.
And therefore, these things are the absolute barriers.
So, for example, suppose I have two of them is y0, and let's say here's another one, another constant solution.
I want to know what the curves in between those can do.
Well, I do know that whatever those red curves do, the other integral curves, they cannot cross this, and they cannot cross that.
And, you must be able to translate them along each other without ever having two of them intersect.
Now, that really limits their behavior, but I'm going to nail it down even more.
So, other curves can't cross these.
Other integral curves can't cross these yellow curves, these yellow lines, these horizontal lines.
But, I'm going to show you more, and namely, so what I'm after is deciding, without solving the equation, what those curves must look like in between.
Now, the way to do that is you draw, so if we want to make steps, everybody likes steps, okay, so step one is going to be, find these.
Find the critical points.
And, you're going to do that by solving this equation, finding out where it's zero.
Once you have done that, you are going to draw the graph of f of y.
And, the interest is going to be, where is it positive?
Where is it negative?
You've already found where it's zero.
Everywhere else, therefore, it must be either positive or negative.
Now, once you have found that out, why am I interested in that?
Well, because dy / dt is equal to f of y, right?
That's what the differential equation says.
Therefore, if this, for example, is positive, that means this must be positive.
It means that y must be increasing.
It means the solution must be increasing.
Where it's negative, the solution will be decreasing.
And, that tells me how it's behaving in between these yellow lines, or on top of them, or on the bottom.
Now, at this point, I'm going to stop, or not stop, I mean, I'm going to stop talking generally.
And everything in the rest of the period will be done by examples which will get increasingly complicated, not terribly complicated by the end.
But, let's do one that's super simple to begin with.
Sorry, I shouldn't say that because some of you may be baffled even by here because after all I'm going to be doing the analysis not in the usual way, but by using new ideas.
That's the way you make progress.
All right, so, let's do our bank account.
So, y is money in the bank account.
r is the interest rate.
Let's assume it's a continuous interest rate.
All banks nowadays pay interest continuously, the continuous interest rate.
So, if that's all there is, and money is growing, you know the differential equation says that the rate at which it grows is equal to r, the interest rate times a principle, the amount that's in the bank at that time.
So, that's the differential equation that governs that.
Now, that's, of course, the solution is simply an exponential curve.
There's nothing more to say about it.
Now, let's make it more interesting.
Let's suppose there is a shifty teller at the bank, and your money is being embezzled from your account at a constant rate.
So, let's let w equal, or maybe e, but e has so many other uses in mathematics, w is relatively unused, w is the rate of embezzlement, thought of as continuous.
So, every day a little bit of money is sneaked out of your account because you are not paying any attention to it.
You're off skiing somewhere, and not noticing what's happening to your bank account.
So, since it's the rate, the time rate of embezzlement, I simply subtract it from this.
It's not w times y because the embezzler isn't stealing a certain fraction of your account.
It's simply stealing a certain number of dollars every day, the same number of dollars being withdrawn for the count.
Okay, now, of course, you could solve this.
This separates variables immediately.
You get the answer, and there's no problem with that.
Let's analyze the behavior of the solutions without solving the equation by using these two points.
So, I want to analyze this equation using the method of critical points.
So, the first thing I should do is, so, here's our equation, is find the critical points.
Notice it's an autonomous equation all right, because there's no t on the right-hand side.
Okay, so, the critical points, well, that's where ry minus w equals zero.
In other words, there's only one critical point, and that occurs when y is equal to w over r. So, that's the only critical point.
Now, I want to know what's happening to the solution.
So, in other words, if I plot, I can write away, of course, negative values aren't of particularly interesting here, there is definitely a horizontal solution, and it has the value, it's at the height, w over r. That's a solution.
The question is, what do the other solutions look like?
Now, watch how I make the analysis because I'm going to use two now.
So, this is step one, then step two.
What do I do?
Well, I'm going to graph f of y.
Well, f of y is ry minus w. What does that look like?
Okay, so, here is the y-axis.
Notice the y-axis is going horizontally because what I'm interested in is the graph of this function.
What do I call the other axis?
I'm going to use the same terminology that is used on the little visual that describes this.
And, that's dy.
You could call this other axis the f of y axis.
That's not a good name for it.
You could call it the dy / dt axis because it's, so to speak, the other variable.
That's not great either.
But, worst of all would be introducing yet another letter for which we would have no use whatsoever.
So, let's think of it.
We are plotting, now, the graph of f of y. f of y is this function, ry minus w. Well, that's a line.
Its intercept is down here at w, and so the graph looks something like this.
It's a line.
This is the line, ry minus w. It has slope r. Well, what am I going to get out of that line?
Just exactly this.
What am I interested in about that line?
Nothing other than where is it above the axis, and where is it below?
This function is positive over here, and therefore, I'm going to indicate that symbolically, this, by putting an arrow here.
The meeting of this arrow is that y of t is increasing.
See where it's the right-hand side of that last board?
y of t is increasing when f of y is positive.
f of y is positive here, and therefore, to the right of this point, it's increasing.
Here to the left of it, f of y is negative, and therefore over here it's going to be decreasing.
What point is this, in fact?
Well, that's where it crosses the axis.
That's exactly the critical point, w over r. Therefore, what this says is that a solution, once it's bigger than y over r, it increases, and it increases faster and faster because this function is higher and higher.
And, that represents the rate of change.
So, in other words, once the solution, let's say a solution starts over here at time zero.
So, this is the t axis.
And, here is the y axis.
So, now, I'm plotting solutions.
If it starts at t equals zero, above this line, that is, starts with the value w over r, which is bigger than zero, a value bigger than w over r, then it increases, and increases faster and faster.
If it starts below that, it decreases and decreases faster and faster.
Now, in fact, I only have to draw two of those because what do all the others look like?
They are translations.
All the other curves look exactly like those.
They are just translations of them.
This guy, if I start closer, it's still going to decrease.
Well, that's supposed to be a translation.
Maybe it is.
So, these guys look like that.
Let's do just a tiny bit more interpretation of that.
Well, I think I better leave it there because we've got harder things to do, and I want to make sure we've got time for it.
Sorry.
Okay, next example, a logistic equation.
Some of you have already solved this in recitation, and some of you haven't.
This is a population equation.
This is the one that section 7.1 and section 1.7 is most heavily concerned with, this particular equation.
The derivation of it is a little vague.
It's an equation which describes how population increases.
And one minute, the population behavior of some population, -- -- let's call it, y is the only thing I know to call anything today, but of course your book uses capital P for population, to get you used to different variables.
Now, the basic population equation runs dy / dt.
There's a certain growth rate.
Let's call it k y.
So, k is what's called the growth rate.
It's actually, sometimes it's talked about in terms of birthrate.
But, it's the net birth rate.
It's the rate at which people, or bacteria, or whatever are being born minus the rate at which they are dying.
So, it's a net birthrate.
But, let's just call it the growth rate.
Now, if this is the equation, we can think of this, if k is constant, that's what's called simple population growth.
And you are all familiar with that.
Logistical growth allows for slightly more complex situations.
Logistic growth says that calling k a constant is unrealistic because the Earth is not filled entirely with people.
What stops it from having unlimited growth?
Well, the fact that the resources, the food, the organism has to live on gets depleted.
And, in other words, the growth rate declines as y increases.
As the population increases, one expects the growth rate to decline because resources are being used up, and they are not indefinitely available.
Well, in other words, we should replace k by a function with this behavior.
What's the simplest function that declines as y increases?
The simplest choice, and if you are ignorant about what else to do, stick with the simplest, at least you won't work any harder than you have to, would be to take k equal to the simplest declining function of y there is, which is simply a linear function, A minus BY.
So, if I use that as the choice of the declining growth rate, the new equation is dy / dt equals, here's my new k. The y stays the same, so the equation becomes a minus by, the quantity times y, or in other words, ay minus b y squared.
This equation is what's called the logistic equation.
It has many applications, not just to population growth.
It's applied to the spread of disease, the spread of a rumor, the spread of many things.
Yeah, a couple pieces of chalk here.
Okay, now, those of you who have solved it know that the explicit solution involves, well, you separate variables, but you will have to use partial fractions, ugh, I hope you love partial fractions.
You're going to need them later in the term.
But, I could avoid them now by not solving the equation explicitly.
But anyway, you get a solution, which I was going to write on the board for you, but you could look it up in your book.
It's unpleasant enough looking to make you feel that there must be an easier way at least to get the basic information out.
Okay, let's see if we can get the basic information out.
What are the critical points?
Well, this is pretty easy.
A, I want to set the right-hand side equal to zero.
So, I'm going to solve the equation.
I can factor out a y.
It's going to be y times a minus by equals zero.
And therefore, the critical points are where y equals zero.
That's one.
And, the other factor is when this factor is zero, and that happens when y is equal to a over b.
So, there are my two critical points.
Okay, what does, let's start drawing pictures of solutions.
Let's put it in those right away.
Okay, the critical point, zero, gives me a solution that looks like this.
And, the critical point, a over b, those are positive numbers.
So, that's somewhere up here.
So, those are two solutions, constant solutions.
In other words, if the population by dumb luck started at zero, it would stay at zero for all time.
That's not terribly surprising.
But, it's a little less obvious that if it starts at that magic number, a over b, it will also stay at that magic number for all time without moving up or down or away from it.
Now, the question is, therefore, what happens in between?
So, for the in between, I'm going to make that same analysis that I made before.
And, it's really not very hard.
Look, so here's my dy/dt-axis.
I'll call that y prime, okay?
And, here's the y-axis.
So, I'm now doing step two.
This was step one.
Okay, the function that I want to graph is this one, ay minus b y squared, or in factor form, y times a minus by.
Now, this function, we know, has a zero.
It has a zero here, and it has a zero at the point a over b.
At these two critical points, it has a zero.
What is it doing in between?
Well, in between, it's a parabola.
It's a quadratic function.
It's a parabola.
Does it go up or does it go down?
Well, when y is very large, it's very negative.
That means it must be a downward-opening parabola.
And therefore, this curve looks like this.
So, I'm interested in knowing, where is it positive, and where is it negative?
Well, it's positive, here, for this range of values of y.
Since it's positive there, it will be increasing there.
Here, it's negative, and therefore it will be decreasing.
Here, it's negative, and therefore, dy / dt will be negative also, and therefore the function, y, will be decreasing here.
So, how do these other solutions look?
Well, we can put them in.
I'll put them in in white, okay, because this has got to last until the end of the term.
So, how are they doing?
They are increasing between the two curves.
They are not allowed to cross either of these yellow curves.
But, they are always increasing.
Well, if they're always increasing, they must start here and increase, and not allowed to cross.
It must do something like that.
This must be a translation of it.
In other words, the curves must look like that.
Those are supposed to be translations of each other.
I know they aren't, but use your imaginations.
But what's happening above?
So in other words, if I start with a population anywhere bigger than zero but less than a over b, it increases asymptotically to the level a over b.
What happens if I start above that?
Well, then it decreases to it because, this way, for the values of y bigger than a over b, it decreases as time increases.
So, these guys up here are doing this.
And, how about the ones below the axis?
Well, they have no physical significance.
But let's put them in anyway.
Whether they doing?
They are decreasing away from zero.
So, these guys don't mean anything physically, but mathematically they exist.
Their solutions, they're going down like that.
Now, you notice from this picture that there are, even though both of these are constant solutions, they have dramatically different behavior.
This one, this solution, is the one that all other solutions try to approach as time goes to infinity.
This one, the solution zero, is repulsive, as it were.
Any solution that starts near zero, if it starts at zero, of course, it stays there for all time, but if it starts just a little bit above zero, it increases to a over b.
This is called a stable solution because everybody tries to get closer and closer to it.
This is called, zero is also a constant solution, but this is an unstable solution.
And now, usually, solution is too general a word.
I think it's better to call it a stable critical point, and an unstable critical point.
But, of course, it also corresponds to a solution.
So, critical points are not all the same.
Some are stable, and some are unstable.
And, you can see which is which just by looking at this picture.
If the arrows point towards them, you've got a stable critical point.
If it arrows point away from them, you've got an unstable critical point.
Now, there is a third possibility.
Okay, I think we'd better address it because otherwise you're going to sit there wondering, hey, what did he do?
Suppose it looks like this.
Suppose it were just tangent.
Well, this is the picture of that curve, the pink curve.
What would the arrows look like then?
What would the arrows look like then?
Well, since they are positive, it's always positive, the arrow goes like this.
And then on the side, it also goes in the same direction.
So, is this critical point stable or unstable?
It's stable if you approach it from the left.
So, how, in fact, do the curves, how would the corresponding curves look?
Well, there's our long-term solution.
This corresponds to that point.
Let's call this a, and then this will be the value, a.
If I start below it, I rise to it.
If I start above it, I increase.
So, if I start above it, I do this.
Well, now, that's stable on one side, and unstable on the other.
And, that's indicated by saying it's semi-stable.
That's a brilliant word.
I wonder how long it to do think that one up, semi-stable critical point: stable on one side, unstable on the other depending on whether you start below it.
And, of course, it could be reversed if I had drawn the picture the other way.
I could have approached it from the top, and left it from below.
You get the idea of the behavior.
Okay, let's now take, I'm going to soup up this logistic equation just a little bit more.
So, let's talk about the logistic equation.
But, I'm going to add to it harvesting, with harvesting.
So, this is a very late 20th century concept.
So, we imagine, for example, a bunch of formerly free range Atlantic salmon penned in one of these huge factory farms off the coast of Maine or someplace.
They've made salmon much cheaper than it used to be, but at a certain cost to the salmon, and possibly to our environment.
So, what happens?
Well, the salmon grow, and grow, and do what salmon do.
And, they are harvested.
That's a word somewhere in the category of ethnic cleansing in my opinion.
But, it's, again, a very 20th-century word.
I think it was Hitler who discovered that, that all you had to do was call something by a sanitary name, and no matter how horrible it was, good bourgeois people would accept it.
So, the harvesting, which means, of course, picking them up and killing them, and putting them in cans and stuff like that, okay, so what's the equation?
I'm going to assume that the harvest is at a constant time rate.
In other words, it's not a certain fraction of all the salmon that are being caught each day and canned.
The factory has a certain capacity, so, 400 pounds of salmon each day are pulled out and canned.
So, it's a constant time rate.
That means that the equation is now going to be dy/dt is equal to, well, salmon grow logistically.
ay minus b y squared, so, that part of the equation is the same.
But, I need a term to take care of this constant harvesting rate, and that will be h. Let's call it h, not h times y.
Then, I would be harvesting a certain fraction of all the salmon there, which is not what I'm doing.
Okay: our equation.
Now, I want to analyze what the critical points of this look like.
Now, this is a little more subtle because there's now a new parameter, there.
And, what I want to see is how that varies with the new parameter.
The best thing to do is, I mean, the thing not to do is make this equal to zero, fiddle around with the quadratic formula, get some massive expression, and then spend the next half hour scratching your head trying to figure out what it means, and what information you are supposed to be getting out of it.
Draw pictures instead.
Draw pictures.
If h is zero, that's the smallest harvesting rate I could have.
The picture looks like our old one.
So, if h is zero, the picture looks like, what color did I, okay, pink.
Yellow.
Yellow is the cheapest, but I can't find it.
Okay, yellow is commercially available.
These are precious.
All right, purple if it's okay, purple.
So, this is the one, our original one corresponding to h equals zero.
Or, in other words, it's the equation ay minus b y squared.
h is zero.
Now, if I want to find, I now want to increase the value of h, well, if I increase the value of h, in other words, harvest more and more, what's happening?
Well, I simply lower this function by h. So, if I lower h somewhat, it will come to here.
So, this is some value, ay minus b y squared minus h1, let's say.
That's this curve.
If I lower it a lot, it will look like this.
So, ay minus b y squared minus h a lot, h twenty.
This doesn't mean anything.
Two.
Obviously, there's one interesting value to lower it by.
It's a value which would lower it exactly by this amount.
Let me put that in special.
If I lower it by just that amount, the curve always looks the same.
It's just been lowered.
I'm going to say this one is, so this one is the same thing, except that I've subtracted h sub m. Where is h sub m on the picture?
Well, I lowered it by this amount.
So, this height is h sub m. In other words, if I find the maximum height here, which is easy to do because it's a parabola, and lower it by exactly that amount, I will have lowered it to this point.
This will be a critical point.
Now, the question is, what's happened to the critical point as I did this?
I started with the critical points here and here.
As I lower h, the critical point changed to this and that.
And now, it changed to this one when I got to the purple line.
And, as I went still further down, there were no critical points.
So, this curve has no critical points attached to it.
What are the corresponding pictures?
Well, the corresponding pictures, well, we've already drawn, the picture for h equals zero is drawn already.
The pictures that I'm talking about are how the solutions look.
How would the solution look like for this one for h one?
For h1, the solutions look like, here is a over b.
Here is a over b, but the critical points aren't at zero and a over b anymore.
They've moved in a little bit.
So, they are here and here.
And, otherwise, the solutions look just like they did before, and the analysis is the same.
And, similarly, if h two goes very far, if h2 is very large, there are no critical points.
h, too large, no critical points.
Are the solutions decreasing all the time or increasing?
Well, they are always decreasing because the function is always negative.
Solutions always go down, always.
The interesting one is this last one, where I decreased it just to (h)m. And, what happens there is there is this certain, magic critical point whose value we could calculate.
There's one constant solution.
So, this is one that has the value.
Sorry, I'm calculating the solutions out.
So, y here and t here, so here it is value, (h)m is the value by which it has been lowered.
So, this is the picture for (h)m. And, how do the solutions look?
Well, to the right of that, they are decreasing.
And, to the left they are also decreasing because this function is always negative.
So, the solutions look like this, if you start above, and if you start below, they decrease.
And, of course, they can't get lower than zero because these are salmon.
What is the significance of (h)m?
(h)m is the maximum rate of harvesting.
It's an extremely important number for this industry.
If the maximum time rate at which you can pull the salmon daily out of the water, and can them without what happening?
Without the salmon going to zero.
As long as you start above, and don't harvest it more than this rate, it will be following these curves.
You will be following these curves, and you will still have salmon.
If you harvest just a little bit more, you will be on this curve that has no critical points, and the salmon in the tank will decrease to zero.
I assume from high school you know how to add and multiply complex numbers using the relation i squared equals negative one.
I'm a little less certain that you remember how to divide them.
I hope you read last night by way of preparation for that, but since that's something we're going to have to do a lot of a differential equations, so remember that the division is done by making use of the complex conjugate.
So, if z is equal to a plus bi, some people write a plus ib, and sometimes I'll do that too if it's more convenient.
Then, the complex conjugate is what you get by changing i to negative i.
And, the important thing is that the product of those two is a real number.
The product of these is a squared minus the quantity ib all squared, which makes a squared plus b squared because i squared is negative one.
So, the product of those, that's what you multiply if you want to multiply this by something to make it real.
You always multiplied by its complex conjugate.
And that's the trick that underlines the doing of the division.
So, for example, I better hang onto these or I'll never remember all the examples.
Suppose, for example, we wanted to calculate (two plus i) divided by (one minus 3 i).
To calculate it means I want to do the division; I want to express the answer in the form a plus bi.
What you do is multiply the top and bottom by the complex conjugate of the denominator in order to make it real.
So, it's (one plus 3i) divided by (one plus 3i), as they taught you in elementary school, that is one, in a rather odd notation; therefore, multiplying doesn't change the value of the fraction.
And so, the denominator now becomes 1 squared plus 3 squared, which is ten.
And, the numerator is, learn to do this without multiplying out four terms.
You must be able to do this in your head.
And, you always do it by the grouping, or post office method, whatever you want to call it, namely, first put down the real part, which is made out of two times one minus three times one.
So, that's negative one.
And then, the imaginary part, which is i times one.
That's one, coefficient one, plus 6i.
So, that makes 7i.
Now, some people feel this still doesn't look right, if you wish, and for some places and differential equations, it will be useful to write that as minus one tenth plus seven tenths i.
And, now it's perfectly clear that it's in the form a plus bi.
So, learn to do that if you don't know already.
It's going to be important.
Now, the main thing today is the polar representation, which sometimes they don't get to in high school.
And if they do, it's usually not in a grown up-enough in a form for us to be able to use it.
So, I have to worry about that little bit.
The polar representation, of course, is nominally just the switch to polar coordinates.
If here's a plus bi, then this is r, and that's theta.
And therefore, this can be written as, in the polar form, that would be r cosine theta plus i, or r cosine theta.
That's the A part.
And, the B part is, the imaginary part is r sin(theta) times i.
Now, it would be customary, at this point, to put the i in front, just because it looks better.
The complex numbers are commutative, satisfied to commutative law of multiplication, which means it doesn't matter in multiplication whether you put i in front or behind.
It's still the same answer.
So, this would be r cosine theta plus i times r sine theta, which, of course, will factor out, and will make it cosine theta plus i sine theta.
Now, it was Euler who took the decisive step and said, hey, look, I'm going to call that e to the i theta.
Now, why did he do that?
Because everything seemed to indicate that it should.
But that's certainly worth the best color we have, which is what?
We are getting low here.
Okay, nonetheless, it's worth pink.
I will even give him his due, Euler.
Sometimes it's called Euler's formula, but it really shouldn't be.
It's not a formula.
It's a definition.
So, in some sense, you can't argue with it.
If you want to call putting a complex number in a power, and calling it that, you can.
But, one can certainly ask why he did it.
And the answer, I guess, is that all the evidence seemed to point to the fact that it was the thing to do.
Now, I think it's important to talk about a little bit because I think it's, in my opinion, if you're seeing this for the first time, even if you read about it last night, it's a mysterious thing, and one needs to see it from every possible point of view.
It's something you get used to.
You will never see it in a sudden flash of insight.
It will just get as familiar to you as more common arithmetic, and algebraic, and calculus processes are.
But, look.
What is it we demand?
If you're going to call something an exponential, what is it we want an exponential to do, what gives an expression like this the right to be called e to the i theta?
The answer is I can't creep inside Euler's mind.
It must have been a very big day of his life.
He had a lot of big days, but when he realized that that was the thing to write down as the definition of e to the i theta.
But, what is it one wants of an exponential?
Well, the high school answer surely is you want it to satisfy the exponential law.
Now, to my shock, I realize a lot of people don't know.
In my analysis class, these are some math majors, or graduate engineers in various subjects, and if I say prove such and such using the exponential law, I'm sure to get at least half a dozen e-mails asking me, what's the exponential law?
Okay, the exponential law is a to the x times a to the y equals a to the x plus y: the law of exponents.
That's the most important reason why, that's the single most important thing about exponents, are the way one uses them.
And, this is the exponential function, called the exponential function because all this significant stuff is in the exponents.
All right, so it should satisfy-- we want, first of all, the exponential law to be true.
But that's not all.
That's a high school answer.
An MIT answer would be, I mean, why is e to the x such a popular function?
Well, of course, it does satisfy the exponential law, but for us, an even more reasonable thing.
It's the function, which, when you differentiate it, you get the same thing you started with.
And, it's apart from a constant factor, the only such function.
Now, in terms of differential equations, it means that it's the solution that e to the, let's be a little generous, make it e to the ax.
No, better not to use x because complex numbers tend to be called x plus iy.
Let's use t as a more neutral variable, which is standing outside the fray, as it were.
It satisfies the relationship that it's the solution, if you like, to the differential equation.
That's a fancy way of saying it.
dy / dt equals a times y.
Now, of course, that is not unique.
We could make it unique by putting in an initial value.
So, if I want to get this function and not a constant times it, I should make this an initial value problem and say that y of zero should be one.
And now, I will get only the function, e to the at.
So, in other words, that characterizes this function.
It's the only function in the whole world that has that property.
Now, if you're going to call something e to the i theta, we want that to be true.
So, here are my questions.
Is it true that e to the i theta one, let's use that, times e to the i theta two, see, I'm on a collision course here, but that's easily fixed.
Is that equal to e to the i (theta one plus theta two)?
If that turns out to be so, that's a big step.
What would we like to be true here?
Well, will it be true that the derivative, with respect to t of e to the i theta, I would like that to be equal to i times e to the i theta.
So, question, question.
I think those are the two most significant things.
Now, the nodes do a third thing, talk about infinite series.
Since we haven't done infinite series, anyway, it's not officially part of the syllabus, the kind of power series that are required.
But, I will put it down for the sake of completeness, as people like to say.
So, it should behave right.
The infinite series should be nice.
The infinite series should work out.
There is no word for this, should work out, let's say.
I mean, what's the little music?
Is that some weird music idea, or is it only me that hears it?
[LAUGHTER] Yes, Lord.
I feel I'm being watched up there.
This is terrible.
So, there's one guy.
Here's another guy.
And, I won't put a box around the infinite series, since I'm not going to say anything about it.
Now, these things, in fact, are both true.
Otherwise, why would I be saying them, and why would Euler have made the formula?
But, what's interesting to see is what's behind them.
And, that gives you little practice also in calculating with the complex numbers.
So, let's look at the first one.
What will it say?
It is asking the question.
It says, please, calculate the product of these two things.
Okay, I do it, I'm told.
I will calculate the product of cosine theta one plus i cosine theta two-- Sine.
Sine theta one.
That's e to the i theta one, right?
So, that corresponds to this.
The other factor times the other factor, cosine theta two plus i sine theta two.
Okay, what does that come out to be?
Well, again, we will use the method of grouping.
What's the real part of it?
The real part of it is cosine theta one cosine theta two.
And then, there's a real part, which comes from these two factors.
It's going to occur with a minus sign because of the i squared.
And, what's left is sine theta one sine theta two.
And then, the imaginary part, I'll factor out the i.
And then, what's left, I won't have to keep repeating the i.
So, it will have to be sine theta one cosine theta two.
And, the other factor will be cosine theta one sine theta two-- plus sine theta two cosine theta one.
Well, it looks like a mess, but, again, high school to the rescue.
What is this?
The top thing is nothing in disguise, but it's a disguised form of cosine (theta one plus theta two).
And the bottom is sine of (theta one plus theta two).
So, the product of these two things is this, and that's exactly the formula.
In other words, this formula is a way of writing those two trigonometric identities for the cosine of the sum and the sine of the sum.
Instead of the two identities taking up that much space, written one after the other, they take up as much space, and they say exactly the same thing.
Those two trigonometric identities are exactly the same as saying that e to the i theta satisfies the exponential law.
Now, people ask, you know, what's beautiful in mathematics?
To me, that's beautiful.
I think that's great.
Something long turns into something short, and it's just as good, and moreover, connects with all these other things in the world, differential equations, infinite series, blah, blah, blah, blah, blah.
Okay, I don't have to sell Euler.
He sells himself.
Now, how about the other one?
How about the other one?
Now, that's obviously, I haven't said something because for one thing, how do you differentiate if there's theta here, and t down there.
Okay, that's easily fixed.
But, how do I differentiate this?
What kind of a guy is e to the i theta?
Well, if I write it out, take a look at what it is.
It's cosine theta plus i sine theta.
As theta varies, it's a function.
The variable is real.
Theta is a real variable.
Its angle in radians, but it runs from negative infinity to infinity.
So, if you think of functions as a black box, what's going in is a real number.
But, what's coming out is a complex number.
So, schematically, here is the e to the i theta box, if you like to think that way, theta goes in, and that's real, and a complex number, this particular complex number goes out.
So, one, we'd call it, I'm not going to write this down because it's sort of pompous and takes too long.
But, it is a complex valued function of a real variable.
You got that?
Up to now, we studied real functions of real variables.
But now, real valued functions of real variables, those are the kind calculus is concerned with.
But now, it's a complex-valued function because the variable is real.
But, the output, the value of the function is a complex number.
Now, in general, such a function, well, maybe a better say, complex-valued, how about complex-valued function of a real variable, let's change the name of the variable.
t is always a real variable.
I don't think we have complex time yet, although I'm sure there will be someday.
But, the next Einstein appears.
A complex-valued function of a real variable, t, in general, would look like this.
t goes in, and what comes out?
Well: a complex number, which I would then have to write this way.
In other words, the real part depends on t, and the imaginary part depends upon t. So, a general function looks like this, a general complex-valued function.
This is just a special case of it, where the variable has a different name.
But, the first function would be cosine t, and the second function would be sine t. So, my only question is, how do you differentiate such a thing?
Well, I'm not going to fuss over this.
The general definition is, with deltas and whatnot, but the end result of a perfectly fine definition is, you differentiate it by differentiating each component.
The reason you don't have to work so very hard is because this is a real variable, and I already know what it means to differentiate a function of a real variable.
So, I could write it this way, that the derivative of u plus iv, I'll abbreviate it that way, this means the derivative, with respect to whatever variable, since I didn't tell you what the variable in these functions were, well, I don't have to tell you what I'm differentiating with respect to.
It's whatever was there because you can't see.
And the answer is, it would be the derivative of u plus i times the derivative of v. You differentiate it just the way you would if these were the components of a motion vector.
You would get the velocity by differentiating each component separately.
And, that's what you're doing here.
Okay, now, the importance of that is that it at least tells me what it is I have to check when I check this formula.
So, let's do it now that we know what this is.
We know how to differentiate the function.
Let's actually differentiate it.
That's fortunately, by far, the easiest part of the whole process.
So, let's do it.
So, what's the derivative?
Let's go back to t, our generic variable.
I want to emphasize that these functions, when we write them as functions, that theta will almost never be the variable outside of these notes on complex numbers.
It will normally be time or something like that, or x, a neutral variable like x.
So, what's the derivative of e to the i theta?
I'm hoping that it will turn out to be i e to the i theta, and that the yellow law may be true just as the green one was.
Okay, let's calculate it.
It's the derivative, with respect to, unfortunately I can convert t's to thetas, but not thetas to t's.
C'est la vie, okay.
Times cosine t plus i sine t, and what's that?
Well, the derivative of cosine t, differentiating the real and imaginary parts separately, and adding them up.
It's negative sine t, plus i times cosine t. Now, let's factor out at the i, because it says if I factor out the i, what do I get?
Well, now, the real part of what's left would be cosine t. And, how about the imaginary part?
Do you see, it will be i sine t because i times i gives me that negative one.
And, what's that?
e to the it.
i times e to the i t. So, that works too.
What about the initial condition?
No problem.
What is y of zero?
What's the function at zero?
Well, don't say right away, i times zero is zero, so it must be one.
That's illegal because, why is that illegal?
It's because in that formula, you are not multiplying i times theta.
I mean, sort of, you are, but that formula is the meaning of e to the i theta.
Now, it would be very nice if this is like, well, anyway, you can't do that.
So, you have to do it by saying it's the cosine of zero plus i times the sine of zero.
And, how much is that?
The sine of zero is zero.
Now, it's okay to say i times zero is zero because that's the way complex numbers multiply.
What is the cosine of zero?
That's one.
So, the answer, indeed, turns out to be one.
So, this checks, really, from every conceivable standpoint down as I indicated, also from the standpoint of infinite series.
So, we are definitely allowed to use this.
Now, the more general exponential law is true.
I'm not going to say much about it.
So, in other words, e to the a, this is really a definition.
e to the (a plus ib) is going to be, in order for the general exponential law to be true, this is really a definition.
It's e to the a times e to the ib.
Now, notice when I look at the-- at any complex number, -- -- so, in terms of this, the polar form of a complex number, to draw the little picture again, if here is our complex number, and here is r, and here is the angle theta, so the nice way to write this complex number is r e to the i theta.
The e to the i theta is, now, why is that?
What is the magnitude of this?
This is r. The length of the absolute value, I didn't talk about magnitude in argument.
I guess I should have.
But, it's in the notes.
So, r is called the modulus.
Well, the fancy word is the modulus.
And, we haven't given the complex number a name.
Let's call it alpha, modulus of alpha, and theta is called, it's the angle.
It's called the argument.
I didn't make up these words.
There, from a tradition of English that has long since vanished, when I was a kid, and you wanted to know what a play was about, you looked in the playbill, and it said the argument of the play, it's that old-fashioned use of the word argument.
Argument means the angle, and sometimes that's abbreviated by arg alpha.
And, this is abbreviated, of course, as absolute value of alpha, its length.
Okay, the notes give you a little practice changing things to a polar form.
I think we will skip that in favor of doing a couple of other things because that's pretty easy.
But let me, you should at least realize when you should look at polar form.
The great advantage of polar form is, particularly once you've mastered the exponential law, the great advantage of polar form is it's good for multiplication.
Now, of course, you know how to multiply complex numbers, even when they are in the Cartesian form.
That's the first thing you learn in high school, how to multiply a plus bi times c plus di.
But, as you will see, when push comes to shove, you will see this very clearly on Friday when we talk about trigonometric inputs to differential equations, -- -- that the changing to complex numbers makes all sorts of things easy to calculate, and the answers come out extremely clear, whereas if we had to do it any other way, it's a lot more work.
And worst of all, when you finally slog through to the end, you fear you are none the wiser.
It's good for multiplication because the product, so here's any number in its polar form.
That's a general complex number.
It's modulus times e to the i theta times r two e to the i theta two-- Well, you just multiply them as ordinary numbers.
So, the part out front will be r1 r2, and the e to the i theta parts gets multiplied by the exponential law and becomes e to the i (theta one plus theta two) -- -- which makes very clear that the multiply geometrically two complex numbers, you multiply the moduli, the r's, the absolute values, how long the arrow is from zero to the complex number, multiply the moduli, and add the arguments.
So the new number, its modulus is the product of r1 and r2.
And, its argument, its angle, polar angle, is the sum of the old two angles.
And, you add the angles.
And, you put down in your books angles, but I'm being photographed, so I'm going to write arguments.
In other words, it makes the geometric content of multiplication clear, in a sense in which this is extremely unclear.
From this law, blah, blah, blah, blah, blah, whatever it turns out to be, you have not the slightest intuition that this is true about the complex numbers.
That first thing is just a formula, whereas this thing is insightful representation of complex multiplication.
Now, I'd like to use it for something, but before we do that, let me just indicate how just the exponential notation enables you to do things in calculus, formulas that are impossible to remember from calculus.
It makes them very easy to derive.
A typical example of that is, suppose you want to, for example, integrate (e to the negative x) cosine x.
Well, number one, you spend a few minutes running through a calculus textbook and try to find out the answer because you know you are not going to remember how to do it.
Or, you run to a computer, and type in Matlab and something.
Or, you fish out your little pocket calculator, which will give you a formula, and so on.
So, you have aides for doing that.
But, the way to do it if you're on a desert island, and the way I always do it because I never have any of these little aides around, and I cannot trust my memory, probably a certain number of you remember how you did it at high school, or how you did it in 18.01, if you took it here.
You have to use integration by parts.
But, it's one of the tricky things that's not required on an exam because you had to use integration by parts twice in the same direction, and then suddenly by comparing the end product with the initial product and writing an equation.
Somehow, the value falls out.
Well, that's tricky.
And it's not the sort of thing you can waste time stuffing into your head, unless you are going to be the integration bee during IAP or something like that.
Instead, using complex numbers is the way to do this.
How do I think of this, cosine x?
What I do, is I think of that e to the negative x cosine x is the real part, the real part of what?
Well, cosine x is the real part of e to the ix.
So, this thing, this is real.
This is real, too.
But I'm thinking of it as the real part of e to the ix.
Now, if I multiply these two together, this is going to turn out to be, therefore, the real part of e to the minus x. I'll write it out very pompously, and then I will fix it.
I would never write this, you are you.
Okay, it's e to the minus x times, when I write cosine x plus i sine x, so it is the real part of that is cosine x.
So, it's the real part of, write it this way for real part of e to the, factor out the x, and what's up there is (negative one plus i) times x.
Okay, and now, so, the idea is the same thing is going to be true for the integral.
This is going to be the real part of that, the integral of e to the (minus one plus i) times x dx.
In other words, what you do is, this procedure is called complexifying the integral.
Instead of looking at the original real problem, I'm going to turn it into a complex problem by turning this thing into a complex exponential.
This is the real part of that complex exponential.
Now, what's the advantage of doing that?
Simple.
It's because nothing is easier to integrate than an exponential.
And, though you may have some doubts as to whether the familiar laws work also with complex exponentials, I assure you they all do.
It would be lovely to sit and prove them.
On the other hand, I think after a while, you find it rather dull.
So, I'm going to do the fun things, and assume that they are true because they are.
So, what's the integral of e to the (minus one plus i) x dx?
Well, if there is justice in heaven, it must be e to the (minus one plus i) times x divided by minus one plus i.
In some sense, that's the answer.
This does, in fact, give that.
That's correct.
I want the real part of this.
I want the real part because that's the way the original problem was stated.
I want the real part only.
So, I want the real part of this.
Now, this is what separates the girls from the women.
[LAUGHTER] This is why you have to know how to divide complex numbers.
So, watch how I find the real part.
I write it this way.
Normally when I do the calculations for myself, I would skip a couple of these steps.
But this time, I will write everything out.
You're going to have to do this a lot in this course, by the way, both over the course of the next few weeks, and especially towards the end of the term where we get into a complex systems, which involve complex numbers.
There's a lot of this.
So, now is the time to learn to do it, and to feel skillful at it.
So, it's this times e to the negative x times e to the ix, which is cosine x plus i sine x.
Now, I'm not ready, yet, to do the calculation to find the real part because I don't like the way this looks.
I want to go back to the thing I did right at the very beginning of the hour, and turn it into an a plus bi type of complex number.
In other words, what we have to do is the division.
So, the division is going to be, now, I'm going to ask you to do it in your head.
I multiply the top and bottom by negative one minus I.
What does that put in the denominator?
One squared plus one squared: Two.
And in the numerator, negative one minus i.
This is the same as that.
But now, it looks at the form a + bi.
It's negative one over two minus i times one half.
So, this is multiplied by e to the minus x and cosine x.
So, if you are doing it, and practice a little bit, please don't put in all these steps.
Go from here; well, I would go from here to here by myself.
Maybe you shouldn't.
Practice a little before you do that.
And now, what do we do with this?
Now, this is in a form to pick out the real part.
We want the real part of this.
So, you don't have to write the whole thing out as a complex number.
In other words, you don't have to do all the multiplications.
You only have to find the real part of it, which is what?
Well, e to the negative x will be simply a factor.
That's a real factor, which I don't have to worry about.
And, in that category, I can include the two also.
So, I only have to pick out the real part of this times that.
And, what's that?
It's negative cosine x.
And, the other real part comes from the product of these two things.
I times negative i is one.
And, what's left is sine x.
So, that's the answer to the question.
That's the integral of e to the negative x * cosine x.
Notice, it's a completely straightforward process.
It doesn't involve any tricks, unless you call going to the complex domain a trick.
But, I don't.
As soon as you see in this course the combination of e to ax times cosine bx or sine bx, you should immediately think, and you're going to get plenty of it in the couple of weeks after the exam, you are going to get plenty of it, and you should immediately think of passing to the complex domain.
That will be the standard way we solve such problems.
So, you're going to get lots of practice doing this.
But, this was the first time.
Now, I guess in the time remaining, I'm not going to talk about in the notes, i, R, at all, but I would like to talk a little bit about the extraction of the complex roots, since you have a problem about that and because it's another beautiful application of this polar way of writing complex numbers.
Suppose I want to calculate.
So, the basic problem is to calculate the nth roots of one.
Now, in the real domain, of course, the answer is, sometimes there's only one of these, one itself, and sometimes there are two, depending on whether n is an even number or an odd number.
But, in the complex domain, there are always n answers as complex numbers.
One always has n nth roots.
Now, where are they?
Well, geometrically, it's easy to see where they are.
Here's the unit circle.
Here's the unit circle.
One of the roots is right here at one.
Now, where are the others?
Well, do you see that if I place, let's take n equal five because that's a nice, dramatic number.
If I place these peptides equally spaced points around the unit circle, so, in other words, this angle is alpha.
Alpha should be the angle.
What would be the expression for that?
If there were five such equally spaced, it would be one fifth of all the way around the circle.
All the way around the circle is two pi.
So, it will be one fifth of two pi in radians.
Now, it's geometrically clear that those are the five fifth roots because, how do I multiply complex numbers?
I multiply the moduli.
Well, they all have moduli one.
So, if I take this guy, let's call that complex number, oh, I hate to give you, they are always giving you Greek notation.
All right, why not torture you?
Zeta.
At least you will learn how to make a zeta in this period, small zeta, so that's zeta.
There's our fifth root of unity.
It's the first one that occurs on the circle that isn't the trivial one, one.
Now, do you see that, how would I calculate zeta to the fifth?
Well, if I write zeta in polar notation, what would it be?
The modulus would be one, and therefore it will be simply, the r will be one for it because its length is one.
Its modulus is one.
What's up here?
I times that angle, and that angle is two pi over five.
So, there's just, geometrically I see where zeta is.
And, if I translate that geometry into the e to the i theta form for the formula, I see that it must be that number.
Now, let's say somebody gives you that number and says, hey, is this the fifth root of one?
I forbid you to draw any pictures.
What would you do?
You say, well, I'll raise it to the fifth power.
What's zeta to the fifth power?
Well, it's e to the i two pi / five, and now, if I think of raising that to the fifth power, by the exponential law, that's the same thing as putting a five in front of the exponent.
So, this times five, and what's that?
That's e to the i times two pi.
And, what is that?
Well, it's the angle.
If the angle is two pi, I've gone all the way around the circle and come back here again.
I've got the number one.
So, this is one.
Since the argument, two pi, is the same as an angle, it's the same as, well, let's not write it that way.
It's wrong.
It's just wrong since two pi and zero are the same angle.
So, I could replace this by zero.
Oh dear.
Well, I guess I have to stop right in the middle of things.
So, you're going to have to read a little bit about how to find roots in order to do that problem.
And, we will go on from that point Friday.
This is also written in the form, it's the k that's on the right hand side.
Actually, I found that source is of considerable difficulty.
And, in general, it is.
For these, the temperature concentration model, it's natural to have the k on the right-hand side, and to separate out the (q)e as part of it.
Another model for which that's true is mixing, as I think I will show you on Monday.
On the other hand, there are some common first-order models for which it's not a natural way to separate things out.
Examples would be the RC circuit, radioactive decay, stuff like that.
So, this is not a universal utility.
But I thought that that form of writing it was a sufficient utility to make a special case, and I emphasize it very heavily in the nodes.
Let's look at the equation.
And, this form will be good enough, the y prime.
When you solve it, let me remind you how the solutions look, because that explains the terminology.
The solution looks like, after you have done the integrating factor and multiplied through, and integrated both sides, in short, what you're supposed to do, the solution looks like y equals, there's the term e to the negative k out front times an integral which you can either make definite or indefinite, according to your preference.
q of t times e to the kt inside dt, it will help you to remember the opposite signs if you think that when q is a constant, one, for example, you want these two guys to cancel out and produce a constant solution.
That's a good way to remember that the signs have to be opposite.
But, I don't encourage you to remember the formula at all.
It's just a convenient thing for me to be able to use right now.
And then, there's the other term, which comes by putting up the arbitrary constant explicitly, c e to the negative kt.
So, you could either write it this way, where this is somewhat vague, or you could make it definite by putting a zero here and a t there, and change the dummy variable inside according to the way the notes tell you to do it.
Now, when you do this, and if k is positive, that's absolutely essential, only when that is so, then this term, as I told you a week or so ago, this term goes to zero because k is positive as t goes to infinity.
So, this goes to zero as t goes, and it doesn't matter what c is, as t goes to infinity.
This term stays some sort of function.
And so, this term is called the steady-state or long-term solution, or it's called both, a long-term solution.
And this, which disappears, gets smaller and smaller as time goes on, is therefore called the transient because it disappears at the time increases to infinity.
So, this part uses the initial condition, uses the initial value.
Let's call it y of zero, assuming that you started the initial value, t, when t was equal to zero, which is a common thing to do, although of course not necessary.
The starting value appears in this term.
This one is just some function.
Now, the general picture or the way that looks is, the steady-state solution will be some solution like, I don't know, like that, let's say.
So, that's a steady-state solution, the SSS.
Well, what do the other guys look like?
Well, the steady-state solution has this starting point.
Other solutions can have any of these other starting points.
So, in the beginning, they won't look like the steady-state solution.
But, we know that as time goes on, they must approach it because this term represents the difference between the solution and the steady-state solution.
So, this term is going to zero.
And therefore, whatever these guys do to start out with, after a while they must follow the steady-state solution more and more closely.
They must, in short, be asymptotic to it.
So, the solutions to any equation of that form will look like this.
Up here, maybe it started at 127.
That's okay.
After a while, it's going to start approaching that green curve.
Of course, they won't cross each other.
That's the rock star, and these are the groupies trying to get close to it.
Now, but something follows from that picture.
Which is the steady-state solution?
What, in short, is so special about this green curve?
All these other white solution curves have that same property, the same property that all the other white curves and the green curve, too, are trying to get close to them.
In other words, there is nothing special about the green curve.
It's just that they all want to get close to each other.
And therefore, though you can write a formula like this, there isn't one steady-state solution.
There are many.
Now, this produces vagueness.
You talk about the steady-state solution; which one are you talking about?
I have no answer to that; the usual answer is whichever one looks simplest.
Normally, the one that will look simplest is the one where c is zero.
But, if this is a peculiar function, it might be that for some other value of c, you get an even simpler expression.
So, the steady-state solution: about the best I can see, either you integrate that, don't use an arbitrary constant, and use what you get, or pick the simplest.
Pick the value of c, which gives you the simplest answer.
Pick the simplest function, and that's what usually called the steady-state solution.
Now, from that point of view, what I'm calling the input in this input response point of view, which we are going to be using, by the way, constantly, well, pretty much all term long, but certainly for the next month or so, I'm constantly going to be coming back to it.
The input is the q of t. In other words, it seems rather peculiar.
But the input is the right-hand side of the equation of the differential equation.
And the reason is because I'm always thinking of the temperature model.
The external water bath at temperature T external, the internal thing here, the problem is, given this function, the external water bath temperature is driving, so to speak, the temperature of the inside.
And therefore, the input is the temperature of the water bath.
I don't like the word output, although it would be the natural thing because this temperature doesn't look like an output.
Anyone might be willing to say, yeah, you are inputting the value of the temperature here.
This, it's more likely, the normal term is response.
This thing, this plus the water bath, is a little system.
And the response of the system, i.e.
the change in the internal temperature is governed by the driving external temperature.
So, the input is q of t, and the response of the system is the solution to the differential equation.
Now, if the thing is special, as it's going to be for most of this period, it has that special form, then I'm going to, I really want to call q sub e the input.
I want to call q sub e the input, and there is no standard way of doing that, although there's a most common way.
So, I'm just calling it the physical input, in other words, the temperature input, or the concentration input.
And, that will be my (q)e of t, and by the subscript e, you will understand that I'm writing it in that form and thinking of this model, or concentration model, or mixing model as I will show you on Monday.
By the way, this is often handled, I mean, how would you handle this to get rid of a k?
Well, divide through by k. So, this equation is often, in the literature, written this way: one over k times y prime plus y is equal to, well, now they call it q of t, not (q)e of t because they've gotten rid of this funny factor.
But I will continue to call it (q)e. So, in other words, and this part this is just, frankly, called the input.
It doesn't say physical or anything.
And, this is the solution, it's then the response, and this funny coefficient of y prime, that's not in standard linear form, is it, anymore?
But, it's a standard form if you want to do this input response analysis.
So, this is also a way of writing the equation.
I'm not going to use it because how many standard forms could this poor little course absorb?
I'll stick to that one.
Okay, you have, then, the superposition principle, which I don't think I'm going to-- the solution, which solution?
Well, normally it means any solution, or in other words, the steady-state solution.
Now, notice that terminology only makes sense if k is positive.
And, in fact, there is nothing like the picture, the picture doesn't look at all like this if k is negative, and therefore, the terms would steady state, transient would be totally inappropriate if k were negative.
So, this assumes definitely that k has to be greater than zero.
Otherwise, no.
So, I'll call this the physical input.
And then, you have the superposition principle, which I really can't improve upon what's written in the notes, this superposition of inputs.
Whether they are physical inputs or nonphysical inputs, if the input q of t produces the response, y of t, and q two of t produces the response, y two of t, -- then a simple calculation with the differential equation shows you that by, so to speak, adding, that the sum of these two, I stated it very generally in the notes but it corresponds, we will have as the response y1, the steady-state response y1 plus y2, and a constant times y1.
That's an expression, essentially, of the linear, it uses the fact that the special form of the equation, and we will have a very efficient and elegant way of seeing this when we study higher order equations.
For now, I will just, the little calculation that's done in the notes will suffice for first-order equations.
If you don't have a complicated equation, there's no point in making a fuss over proofs using it.
But essentially, it uses the fact that the equation is linear.
Or, that's bad, so linearity of the ODE.
In other words, it's a consequence of the fact that the equation looks the way it does.
And, something like this would not, in any sense, be true if the equation, for example, had here a y squared instead of t. Everything I'm saying this period would be total nonsense and totally inapplicable.
Now, today, what I wanted to discuss was, what's in the notes that I gave you today, which is, what happens when the physical input is trigonometric?
For certain reasons, that's the most important case there is.
It's because of the existence of what are called Fourier series, and there are a couple of words about them.
That's something we will be studying in about three weeks or so.
What's going on, roughly, is that, so I'm going to take the equation in the form y prime plus ky equals k times (q)e of t, and the input that I'm interested in is when this is a simple one that you use on the visual that you did about two points worth of work for handing in today, cosine omega t. So, if you like, k here.
So, the (q)e is cosine omega t. That was the physical input.
And, omega, as you know, is, you have to be careful when you use the word frequency.
I assume you got this from physics class all last semester.
But anyway, just to remind you, there's a whole yoga of five or six terms that go whenever you're talking about trigonometric functions.
Instead of giving a long explanation, the end of the second page of the notes just gives you a reference list of what you are expected to know for 18.03 and physics as well, with a brief one or two line description of what each of those means.
So, think of it as something to refer back to if you have forgotten.
But, omega is what's called the angular frequency or the circular frequency.
It's somewhat misleading to call it the frequency, although I probably will.
It's the angular frequency.
It's, in other words, it's the number of complete oscillations.
This cosine omega t is going up and down right?
So, a complete oscillation as it goes down and then returns to where it started.
That's a complete oscillation.
This is only half an oscillation because you didn't give it a chance to get back.
Okay, so the number of complete oscillations in how much time, well, in two pi, in the distance, two pi on the t-axis in the interval of length two pi because, for example, if omega is one, cosine t takes two pi to repeat itself, right?
If omega were two, it would repeat itself.
It would make two complete oscillations in the interval, two pi.
So, it's what happens to the interval, two pi, not what happens in the time interval, one, which is the natural meaning of the word frequency.
There's always this factor of two pi that floats around to make all of your formulas and solutions incorrect.
Okay, now, so, what I'm out to do is, the problem is for the physical input, (q)e cosine omega t, find the response.
In other words, solve the differential equation.
In short, for the visual that you looked at, I think I've forgot the colors now.
The input was in green, maybe, but I do remember that the response was in yellow.
I think I remember that.
So, find the response, yellow, and the input was, what color was it, green?
Blue, blue.
Light blue.
Okay, so we've got to solve the differential equation.
Now, it's a question of how I'm going to solve the differential equation.
I'm going to use complex numbers throughout, A because that's the way it's usually done.
B, to give you practice using complex numbers, and I don't think I need any other reasons.
So, I'm going to use complex numbers.
I'm going to complexify.
To use complex numbers, what you do is complexification of the problem.
So, I'm going to complexify the problem, turn it into the domain of complex numbers.
So, take the differential equation, turn it into a differential equation involving complex numbers, solve that, and then go back to the real domain to get the answer, since it's easier to integrate exponentials.
And therefore, try to introduce, try to change the trigonometric functions into complex exponentials, simply because the work will be easier to do.
All right, so let's do it.
To change this differential equation, remember, I've got cosine omega t here.
I'm going to use the fact that e to the i omega t, Euler's formula, that the real part of it is cosine omega t. So, I'm going to view this as the real part of this complex function.
But, I will throw at the imaginary part, too, since at one point we will need it.
Now, what is the equation, then, that it's going to turn into?
The complexified equation is going to be y prime plus ky equals, and now, instead of the right hand side, k times cosine omega t, I'll use the whole complex exponential, e i omega t. Now, I have a problem because y, here, in this equation, y means the real function which solves that problem.
I therefore cannot continue to call this y because I want y to be a real function.
I have to change its name.
Since this is complex function on the right-hand side, I will have to expect a complex solution to the differential equation.
I'm going to call that complex solution y tilda.
Now, that's what I would also use as the designation for the variable.
So, y tilda is the complex solution.
And, it's going to have the form y1 plus i times y2.
It's going to be the complex solution.
And now, what I say is, so, solve it.
Find this complex solution.
So, find the program is to find y tilde, -- -- that's the complex solution.
And then I say, all you have to do is take the real part of that, and that will answer the original problem.
Then, y1, that's the real part of it, right?
It's a function, you know, like this is cosine plus sine, as it was over here, it will naturally be something different.
It will be something different, but that part of it, the real part will solve the original problem, the original, real, ODE.
Now, you will say, you expect us to believe that?
Well, yes, in fact.
I think we've got a lot to do, so since the argument for this is given in the nodes, so, read this in the notes.
It only takes a line or two of standard work with differentiation.
So, read in the notes the argument for that, why that's so.
It just amounts to separating real and imaginary parts.
Okay, so let's, now, solve this.
Since that's our program, all we have to find is the solution.
Well, just use integrating factors and just do it.
So, the integrating factor will be, what, e to the, I don't want to use that formula.
So, the integrating factor will be e to the kt is the integrating factor.
If I multiply through both sides by the integrating factor, then the left-hand side will become y e to the kt, the way it always does, prime, Y tilde, sorry, and the right-hand side will be, now I'm going to start combining exponentials.
It will be k times e to the power i times omega t plus k. I'm going to write that k plus omega t. i omega t plus k. Thank you.
i omega t plus k, or k plus i omega t. kt?
Sorry.
So, it's k times e to the i omega t times e to the kt.
So, that's (k plus i omega) times t. Sorry.
So, y tilda e to the kt is k divided by, now I integrate this, so it essentially reproduces itself, except you have to put down on the bottom k plus i omega.
I'll take the final step.
What's y tilda equals, see, when you do it this way, then you don't get a messy looking formula that you substitute into and that is scary looking.
This is never scary.
Now, I'm going to do two things simultaneously.
First of all, here, if I multiply, after I get the answer, I'm going to want to multiply it by e to the negative kt, right, to solve for y tilda.
If I multiply this by e to the negative kt, then that just gets rid of the k that I put in, and left back with e to the i omega t. So, that side is easy.
All that is left is e to the i omega t. Now, what's interesting is this thing out here, k plus i omega.
I'm going to take a typical step of scaling it.
And you scale it.
I'm going to divide the top and bottom by k. And, what does that produce?
One divided by one plus i times omega over k. What I've done is take these two separate constants, and shown that the critical thing is their ratio.
Okay, now, what I have to do now is take the real part.
Now, there are two ways to do this.
There are two ways to do this.
Both are instructive.
So, there are two methods.
I have a multiplication.
The problem is, of course, that these two things are multiplied together.
And, one of them is, essentially, in Cartesian form, and the other is, essentially, in polar form.
You have to make a decision.
Either go polar, it sounds like go postal, doesn't it, or worse, like a bear, savage, attack it savagely, which that's a very good, aggressive attitude to have when doing a problem, or we can go Cartesian.
Going polar is a little faster, and I think it's what's done in the nodes.
The notes to do both of these.
They just do the first.
On the other hand, they give you a formula, which is the critical thing that you will need to go Cartesian.
I hope I can do both of them if we sort of hurry along.
How do I go polar?
To go polar, what you want to do is express this thing in polar form.
Now, one of the things I didn't emphasize enough, probably, when I talked to you about complex numbers last time is, so I will remind you, which saves my conscience and doesn't hurt yours, suppose you have alpha as a complex number.
See, this complex number is a reciprocal.
The good number is what's down below.
Unfortunately, it's downstairs.
You should know, like you know the back of your hand, which nobody knows, one over alpha.
So that's the form.
The number I'm interested in, that coefficient, it is of the form one over alpha.
One over alpha times alpha is equal to one.
And, from that, it follows, first of all, if I take absolute values, if the absolute value of one over alpha times the absolute value of this is equal to one, so, this is equal to one over the absolute value of alpha.
I think you all knew that.
I'm a little less certain you knew how to take care of the angles.
How about the argument?
Well, the argument of the angle, in other words, the angle of one over alpha plus, because when you multiply, angles add.
Remember that.
Plus, the angle associated with alpha has to be the angle associated with one.
But what's that?
One is out here.
What's the angle of one?
Zero.
Therefore, the argument, the absolute value of this thing is want over the absolute value.
That's easy.
And, you should know that the argument of want over alpha is equal to minus the argument of alpha.
So, when you take reciprocal, the angle turns into its negative.
Okay, I'm going to use that now, because my aim is to turn this into polar form.
So, let's do that someplace, I guess here.
So, I want the polar form for one over one plus i times omega over k. Okay, I will draw a picture.
Here's one.
Here is omega over k. Let's call this angle phi.
It's a natural thing to call it.
It's a right triangle, of course.
Okay, now, this is going to be a complex number times e to an angle.
Now, what's the angle going to be?
Well, this is a complex number, the angle for the complex number.
So, the argument of the complex number, one plus i times omega over k is how much?
Well, there's the complex number one plus i over one plus i times omega over k. Its angle is phi.
So, the argument of this is phi, and therefore, the argument of its reciprocal is negative phi.
So, it's e to the minus i phi.
And, what's A?
A is one over the absolute value of that complex number.
Well, the absolute value of this complex number is one plus omega over k squared.
So, the A is going to be one over that, the square root of one plus omega over k, the quantity squared, times e to the minus i phi.
See, I did that.
That's a critical step.
You must turn that coefficient.
If you want to go polar, you must turn is that coefficient, write that coefficient in the polar form.
And for that, you need these basic facts about, draw the complex number, draw its angle, and so on and so forth.
And now, what's there for the solution?
Once you've done that, the work is over.
What's the complex solution?
The complex solution is this.
I've just found the polar form for this.
So, I multiply it by e to the i omega t, which means these things add.
So, it's equal to A, this A, times e to the i omega t minus i times phi.
Or, in other words, the coefficient is one over, this is a real number, now, square root of one plus omega over k squared.
And, this is e to the, see if I get it right, now.
And finally, now, what's the answer to our real problem?
y1: the real answer.
I mean: the really real answer.
What is it?
Well, this is a real number.
So, I simply reproduce that as the coefficient out front.
And for the other part, I want the real part of that.
But you can write that down instantly.
So, let's recopy the coefficient.
And then, I want just the real part of this.
Well, this is e to the i times some crazy angle.
So, the real part is the cosine of that crazy angle.
So, it's the cosine of omega t minus phi.
And, if somebody says, yeah, well, okay, I got the omega k, I know what that is.
That came from the problem, the driving frequency, driving angular frequency.
That was omega, and k, I guess, k was the conductivity, the thing which told you how quickly the heat that penetrated the walls of the little inner chamber.
So, that's okay, but what's this phi?
Well, the best way to get phi is just to draw that picture, but if you want a formula for phi, phi will be, well, I guess from the picture, it's the arc tangent of omega, k, divided by k, over one, which I don't have to put in.
So, it's this number, phi, in reference to this function.
See, if the phi weren't there, this would be cosine omega t, and we all know what that looks like.
The phi is called the phase lag or phase delay, something like that, the phase lag of the function.
What does it represent?
It represents, let me draw you a picture.
Let's draw the picture like this.
Here's cosine omega t. Now, regular cosine would look sort of like that.
But, I will indicate that the angular frequency is not one by making my cosine squinchy up a little too much.
Everybody can tell that that's the cosine on a limp axis, something for Salvador Dali, okay.
So, there's cosine of something.
So, what was it?
Blue?
I don't have blue.
Yes, I have blue.
Okay, so now you will know what I'm talking about because this looks just like the screen on your computer when you put in the visual for this.
Frequency: your response order one.
So, this is cosine omega t. Now, how will cosine omega t minus phi look?
Well, it'll be moved over.
Let's, for example, suppose phi were pi over two.
Now, where's pi over two on the picture?
Well, what I do is cosine omega t minus this.
I move it over by one, so that this point becomes that one, and it looks like, the site will look like this.
In other words, I shove it over by, so this is the point where omega t equals pi over two.
It's not the value of t. It's not the value of t. It's the value of omega t. And, when I do that, then the blue curve has been shoved over one quarter of its total cycle, and that turns it, of course, into the sine curve, which I hope I can draw.
So, this goes up to there, and then, it's got to get back through.
Let me stop there while I'm ahead.
So, this is sine omega t, the yellow thing, but that's also, in another life, cosine of omega t minus pi over two.
The main thing is you don't subtract, the pi over two is not being subtracted from the t. It's being subtracted from the whole expression, and this whole expression represents an angle, which tells you where you are in the travel, a long cosine to this.
What this quantity gets to be two pi, you're back where you started.
That's not the distance on the t axis.
It's the angle through which you go through.
In other words, does number describes where you are on the cosine cycle.
It doesn't tell you, it's not aiming at telling you exactly where you are on the t axis.
The response function looks like one over the square root of one plus omega over k squared times cosine omega t minus phi.
And, I asked you on the problem set, if k goes up, in other words, if the conductivity rises, if heat can get more rapidly from the outside to the inside, for example, how does that affect the amplitude?
This is the amplitude, A, and the phase lag.
In other words, how does this affect the response?
And now, you can see.
If k goes up, this fraction is becoming smaller.
That means the denominator is becoming smaller, and therefore, the amplitude is going up.
What's happening to the phase lag?
Well, the phase lag looks like this: phi one omega over k. If k is going up, then the size of this side is going down, and the angle is going down.
Now, that part is intuitive.
I would have expected everybody to get that.
It's the heat gets in quickly, more quickly, then the amplitude will match more quickly.
This will rise, and get fairly close to one, in fact, and there should be very little lag in the way the response follows input.
But how about the other one?
Okay, I give you two minutes.
The other one, you will figure out yourself.
I usually like to start a lecture with something new, but this time I'm going to make an exception, and start with the finishing up on Friday because it involves a little more practice with complex numbers.
I think that's what a large number of you are still fairly weak in.
So, to briefly remind you, it will be sort of self-contained, but still, it will use complex numbers.
And, I think it's a good way to start today.
So, remember, the basic problem was to solve something with, where the input was sinusoidal in particular.
The k was on both sides, and the input looked like cosine omega t. And, the plan of the solution consisted of transporting the problem to the complex domain.
So, you look for a complex solution, and you complexify the right hand side of the equation, as well.
So, cosine omega t becomes the real part of this complex function.
The reason for doing that, remember, was because it's easier to handle when you solve linear equations.
It's much easier to handle exponentials on the right-hand side than it is to handle sines and cosines because exponentials are so easy to integrate when you multiply them by other exponentials.
So, the result was, after doing that, y tilda turned out to be one, after I scale the coefficient, one over one plus omega over k And then, the rest was e to the i times (omega t minus phi), where phi had a certain meaning.
It was the arc tangent of a, it was a phase lag.
And, this was then, I had to take the real part of this to get the final answer, which came out to be something like one over the square root of one plus the amplitude one omega / k squared, and then the rest was cosine omega t plus minus phi.
It's easier to see that that part is the real part of this; the problem is, of course you have to convert this.
Sorry, this should be i omega t, in which case you don't need the parentheses, either.
So, the problem was to use the polar representation of this complex number to convert it into something whose amplitude was this, and whose angle was minus phi.
Now, that's what we call the polar method, going polar.
I'd like, now, for the first few minutes of the period, to talk about the other method, the Cartesian method.
I think for a long while, many of you will be more comfortable with that anyway.
Although, one of the objects of the course should be to get you equally comfortable with the polar representation of complex numbers.
So, if we try to do the same thing going Cartesian, what's going to happen?
Well, I guess the same point here.
So, the starting point is still y tilde equals one over, sorry, this should have an i here, one plus i times omega over k, e to the i omega t. But now, what you're going to do is turn this into its Cartesian, turn both of these into their Cartesian representations as a plus ib.
So, if you do that Cartesianly, of course, what you have to do is the standard thing about dividing complex numbers or taking the reciprocals that I told you at the very beginning of complex numbers.
You multiply the top and bottom by the complex conjugate of this in order to make the bottom real.
So, what does this become?
This becomes one minus i times omega over k divided by the product of this in its complex conjugate, which is the real number, one plus omega over k squared So, I've now converted this to the a plus bi form.
I have also to convert the right-hand side to the a plus bi form.
So, it will look like cosine omega t plus i sine omega t. Having done that, I take the last step, which is to take the real part of that.
Remember, the reason I want the real part is because this input was the real part of the complex input.
So, once you've got the complex solution, you have to take its real part to go back into the domain you started with, of real numbers, from the domain of complex numbers.
So, I want the real part is going to be, the real part of that is, first of all, there's a factor out in front.
That's entirely real.
Let's put that out in front, so doesn't bother us particularly.
And now, I need the product of this complex number and that complex number.
But, I only want the real part of it.
So, I'm not going to multiply it out and get four terms.
I'm just going to look at the two terms that I do want.
I don't want the others.
All right, the real part is cosine omega t, from the product of this and that.
And, the rest of the real part will be the product of the two i terms.
But, it's i times negative i, which makes one.
And therefore, it's omega over k times sine omega t. Now, that's the answer.
And that's the answer, too; they must be equal, unless there's a contradiction in mathematics.
But, it's extremely important.
And that's the other reason why I'm giving you this, that you learn in this course to be able to convert quickly and automatically things that look like this into things that look like that.
And, that's done by means of a basic formula, which occurs at the end of the notes for reference, as I optimistically say, although I think for a lot of you will not be referenced, stuff in the category of, yeah, I think I've vaguely seen that somewhere.
But, well, we never used it for anything.
Okay, you're going to use it all term.
So, the formula is, the famous trigonometric identity, which is, so, the problem is to convert this into the other guy.
And, the thing which is going to do that, enable one to combine the sine and the cosine terms, is the famous formula that a times the cosine, I'm going to use theta to make it as neutral as possible, -- -- so, theta you can think of as being replaced by omega t in this particular application of the formula.
But, I'll just use a general angle theta, which doesn't suggest anything in particular.
So, the problem is, you have something which is a combination with real coefficients of cosine and sine, and the important thing is that these numbers be the same.
In practice, that means that the omega t, you're not allowed to have omega one t here, and some other frequency, omega two t here.
That would correspond to using theta one here, and theta two here.
And, though there is a formula for combining that, nobody remembers it, and it's, in general, less universally useful than the first.
If you're going to memorize a formula, and learn this one, it's best to start with the ones where the two are equal.
That's the basic formula.
The others are variations of it, but there is a sizable variations.
All right, so the answer is that this is equal to some other constant, real constant, times the cosine of theta minus phi.
Of course, most people remember this vaguely.
What they don't remember is what the c and the phi are, how to calculate them.
I don't suggest you memorize the formulas for them.
You can if you wish.
Instead, memorize the picture, which is much easier.
Memorize that a and b are the two sides of a right triangle.
Phi is the angle opposite the b side, and c is the length of the hypotenuse.
Okay, that's worth putting up.
I think that's a pink formula.
It's even worth two of those, but I will thrift.
Now, let's apply it to this case to see that it gives the right answer.
So, to use this formula, how I use it?
Well, I should take, I will reproduce the left-hand side.
So that part, I just copy.
And, how about the right?
Well, the amplitude, it's combined into a single cosine term whose amplitude is, well, the two sides of the right triangle are one, and omega over k. The hypotenuse in that case is going to be, well, why don't we write it here?
So, we have one, and omega over k. And, here's phi.
So, the hypotenuse is going to be the square root of one plus omega over k squared.
And, that's going to be multiplied by the cosine of omega t minus this phase lag angle phi.
You can write, if you wish, phi equals the arc tangent, but you are not learning a lot by that.
Phi is the arc tangent of omega over k. That's okay, but it's true.
But, notice there's cancellation now.
This over that is equal to what?
Well, it's equal to this.
And, so when we get in this way, by combining these two factors, one gets exactly the same formula that we got before.
So, as you can see, in some sense, there's not, if you can remember this trigonometric identity, there's not a lot of difference between the two methods except that this one requires this extra step.
The answer will come out in this form, and you then, to see what it really looks like, really have to convert it to this form, the form in which you can see what the phase lag and the amplitude is.
It's amazing how many people who should know, this includes working mathematicians, theoretical mathematicians, includes even possibly the authors of your textbooks.
I'm not sure, but I've caught them in this, too, who in this form, everybody remembers that it's something like that.
Unfortunately, when it occurs as the answer in an answer book, the numbers are some colossal mess here plus some colossal mess here.
And theta is, again, a real mess, involving roots and some cube roots, and whatnot.
The only thing is, these two are the same real mess.
That amounts to just another pure oscillation with the same frequency as the old guy, and with the amplitude changed, and with a phase shift, move to the right or left.
So, this is no more general than that.
Notice they both have two parameters in them, these two coefficients.
This one has the two parameters in an altered form.
Okay, well, I wanted, because of the importance of this formula, I wanted to take a couple of minutes out for a proof of the formula, -- -- just to give you chance to stare at it a little more now.
There are three proofs I know.
I'm sure there are 27.
The Pythagorean theorem now has several hundred.
But, there are three basic proofs.
There is the one I will not give you, I'll call the high school proof, which is the only one one normally finds in books, physics textbooks or other textbooks.
The high school proof takes the right-hand side, applies the formula for the cosine of the difference of two angles, which it assumes you had in trigonometry, and then converts it into this.
It shows you that once you've done that, that a turns out to be c cosine phi and b, the number b is c sine phi, and therefore it identifies the two sides.
Now, the thing that's of course correct and it's the simplest possible argument, the thing that's no good about it is that the direction at which it goes is from here to here.
Well, everybody knew that.
If I gave you this and told you, write it out in terms of cosine and sine, I would assume it dearly hope that practically all of you can do that.
Unfortunately, when you want to use the formula, it's this way you want to use it in the opposite direction.
You are starting with this, and want to convert it to that.
Now, the proof, therefore, will not be of much help.
It requires you to go in the backwards direction, and match up coefficients.
It's much better to go forwards.
Now, there are two proofs that go forwards.
There's the 18.02 proof.
Since I didn't teach most of you 18.02, I can't be sure you had it.
So, I'll spend one minute giving it to you.
What is the 18.02 proof?
It is the following picture.
I think this requires deep colored chalk.
This is going to be pretty heavy.
All right, first of all, the a and the b are the given.
So, I'm going to put in that vector.
So, there is the vector whose sides are, whose components are a and b. I'll write it without the i and j. I hope you had from Jerison that form for the vector, if you don't like that, write ai plus bj, okay?
Now, there's another vector lurking around.
It's the unit vector whose, I'll write it this way, u because it's a unit vector, and theta to indicate that it's angle is theta.
Now, the reason for doing that is because you see that the left-hand side is a dot product of two vectors.
The left-hand side of the identity is the dot product of the vector <a, b> with the vector whose components are cosine theta and sine theta.
That's what I'm calling this unit vector.
It's a unit vector because cosine squared plus sine squared is one.
Now, all this formula is, is saying that scalar product, the dot product of those two vectors, can be evaluated if you know their components by the left-hand side of the formula.
And, if you don't know their components, it can be evaluated in another way, the geometric evaluation, which goes, what is it?
It's a magnitude of one, times the magnitude of the other, times the cosine of the included angle.
Now, what's the included angle?
Well, theta is this angle from the horizontal to that unit vector.
The angle phi is this angle, from this picture here.
And therefore, the included angle between (u)theta and my pink vector is theta minus phi.
That's the formula.
It comes from two ways of calculating the scalar product of the vector whose coefficients are, and the unit vector whose components are cosine theta and sine theta.
All right, well, you should, that was 18.02.
There must be an 18.03 proof also.
Yes.
What's the 18.03 proof?
The 18.03 proof uses complex numbers.
It says, look, take the left side.
Instead of viewing it as the dot product of two vectors, there's another way.
You can think of it as the part of the products of two complex numbers.
So, the 18.03 argument, really, the complex number argument says, look, multiply together a minus bi and the complex number cosine theta plus i sine theta.
There are different ways of explaining why I want to put the minus i there instead of i.
But, the simplest is because I want, when I take the real part, to get the left-hand side.
I will.
If I take the real part of this, I'm going to get a cosine theta plus b sine theta because of negative i and i make one, multiplied together.
All right, that's the left-hand side.
And now, the right-hand side, I'm going to use polar representation instead.
What's the polar representation of this guy?
Well, if <a, b> has the angle theta, then a negative b, a minus bi goes down below.
It has the angle minus phi.
So, this is, has magnitude.
It is polar representation.
Its magnitude is a squared plus b squared, and its angle is negative phi, not positive phi because this a minus bi goes below the axis if a and b are positive.
So, it's e to the minus i phi.
That's the first guy.
And, how about the second guy?
Well, the second guy is e to the i theta.
So, what's the product?
It is a squared plus b squared, the square root, times e to the i times (theta minus phi).
And now, what do I want?
The real part of this, and I want the real part of this.
So, let's just say take the real parts of both sides.
If I take the real part of the left-hand side, I get a cosine theta plus b sine theta.
If I take a real part of this side, I get square root of a squared plus b squared times e, times the cosine, that's the real part, right, of theta minus phi, which is just what it's supposed to be.
Well, with three different arguments, I'm really pounding the table on this formula.
But, I think there's something to be learned from at least two of them.
And, you know, I'm still, for awhile, I will never miss an opportunity to bang complex numbers into your head because, in some sense, you have to reproduce the experience of the race.
As I mentioned in the notes, it took mathematicians 300 or 400 years to get used to complex numbers.
So, if it takes you three or four weeks, that's not too bad.
Now, for the rest of the period I'd like to go back to the linear equations, and try to put into perspective and summarize, and tell you a couple of things which I had to leave out, but which are, in my view, extremely important.
And, up to now, I don't want to leave you with any misapprehensions.
So, I'm going to summarize this way, whereas last lecture I went from the most general equation to the most special.
I'd like to just write them down in the reverse order, now.
So, we are talking about basic linear equations.
First order, of course, we haven't moved as a second order yet.
So, the most special one, and the one we talked about essentially all of the previous two times, or last Friday, anyway, was the equation where the k, the coefficient of y, is constant, and where you also get it on the right-hand side quite providentially.
So, this is the most special form, and it's the one which governed what I will call the temperature-concentration model, or if you want to be grown up, the conduction-diffusion model, conduction-diffusion which describes the processes, which the equation is modeling, whereas these simply described the variables of things, which you usually are trying to calculate when you use the equation.
Now, there are a class of things where the thing is constant, but where the k does not appear naturally on the right hand side.
And, you're going to encounter them pretty quickly in physics, for one place.
So, I better not try to sweep those under the rug.
Let's just call that q of t. And finally, there is the most general case, where you allow k to be non-constant.
That's the one we began, when we talked about the linear equation.
And you know how to solve it in general by a definite or an indefinite integral.
Now, there's one other thing, which I want to talk about.
I will do all these in a certain order.
But, from the beginning, you should keep in mind that there's another between the first two cases.
Between the first two cases, there's another extremely important distinction, and that is as to whether k is positive or not.
Up to now, we've always had k positive.
So, I'm going to put that here.
So, it's understood when I write these, that k is positive.
I want to talk about that, too.
But, first things first.
The first thing I wanted to do was to show you that this, the first case, the most special case, does not just apply to this.
It applies to other things, too.
Let me give you a mixing problem.
The typical mixing problem gives another example.
You've already done in recitation, and you did one for the problem set, the problem of the two rooms filled with smoke.
But, let me do it just using letters, so that the ideas stand out a little more clearly, and you are not preoccupied with the numbers, and calculating with the numbers, and trying to get numerical examples.
So, it's as simple as k sub mixing.
It looks like this.
You have a tank, a room, I don't know, where everything's getting mixed in.
It has a certain volume, which I will call v. Something is flowing in, a gas or a liquid.
And, r will be the flow rate, in some units.
Now, since it can't pile up inside this sealed container, which I'm sure is full, the flow rate out must also be r. And, what we're interested in is the amount of salt.
So, x, let's suppose these are fluid flows, and the dissolved substance that I'm talking about is not carbon monoxide, it's salt, any dissolved substance, some pollutant or whatever the problem calls for.
Let's use salt.
So, it's the amount of salt in the tank at time t. I'm interested in knowing how that varies with time.
Now, there's nothing to be said about how it flows out.
What flows out, of course, is what happens to be in the tank.
But, I do have to say what flows in.
Now, the only convenient way to describe the in-flow is in terms of its concentration.
The salt will be dissolved in the in-flowing water, and so there will be a certain concentration.
And, as you will see, for a secret reason, I'm going to give that the subscript e. So, e is the concentration of the incoming salt, in other words, in the fluid, how many grams are there per liter in the incoming fluid.
That's the data.
So, this is the data.
r is part of the data.
r is the flow rate.
v is the volume.
I think I won't bother writing that down, and the problem is to determine what happens to x of t. Now, I strongly recommend you not attempt to work directly with the concentrations unless you feel you have a really good physical feeling for concentrations.
I strongly recommend you work with a variable that you are given, namely, the dependent variable, which is the amount of salt, grams.
Well, because it's something you can physically think about.
It's coming in, it's getting mixed up, and some of it is going out.
So, the basic equation is going to be that the rate of change of salt in the tank is the rate of salt inflow, let me write salt inflow, minus the rate of salt outflow.
Okay, at what rate is salt flowing in?
Well, the flow rate is the flow rate of the liquid.
I multiply the flow rate, 1 L per minute times the concentration, 3 g per liter.
That means 3 g per minute.
It's going to be, therefore, the product of the flow rate and the concentration, incoming concentration.
How about the rate of the salt outflow?
Well, again, the rate of the liquid outflow is r. And, what is the concentration of salt in the outflow?
I must use, when you talk flow rates, the other factor must be the concentration, not the amount.
So, what is the concentration in the outflow?
Well, it's the amount of salt in the tank divided by its volume.
So, the analog, the concentration here is x divided by v. Now, here's a typical messy equation, dx / dt, let's write it in the standard, linear form, plus r times x over v equals r times the given concentration, which is a function of time.
Now, this is going to be some given function, and there will be no reason whatsoever why you can't solve it in that form.
And, that's normally what you will do.
Nonetheless, in trying to understand how it fits into this paradigm, which kind of equation is it?
Well, clearly there's an awkwardness in that on the right-hand side, we have concentration, and on the left-hand side, we seem to have amounts.
Now, the way to understand the equation as opposed to the way to solve it, well, it's a step on the way to solving it.
But, I emphasize, you can and normally will solve it in exactly that form.
But, to understand what's happening, it's better to express it in terms of concentration entirely, and that's why it's called the concentration, or the diffusion, concentration-diffusion equation.
So, I'm going to convert this to concentrations.
Now, there's no problem here.
x over v is the concentration in the tank.
And now, immediately, you see, hey, it looks like it's going to come out just in the first form.
But, wait a minute.
How about the x?
How do I convert that?
Well, what's the relation between x?
So, if the concentration in the tank is equal to x over v, so the tank concentration, then x is equal to C times the constant, V, and dx / dt, therefore, will be c times dC / dt.
You see that?
Now, that's not in standard form.
Let's put it in standard form.
To put it in standard form, I see, now, that it's not r that's the critical quantity.
It's r divided by v. So, it's dC / dt, C prime, plus r divided by v, -- -- I'm going to call that k, k C, no let's not, r divided by v is equal to r divided by v times Ce.
That's the equation expressed in a form where the concentration is the dependent variable, rather than the amount of salt itself.
And, you can see it falls exactly in this category.
That means that I can talk about it.
The natural way to talk about this equation is in terms of, the same way we talked about the temperature equation.
I said concentration.
I mean, that concentration has nothing to do with this concentration.
This is the diffusion model, where salt solution outside, cell in the middle, salt diffusing through a semi-permeable membrane into that, uses Newton's law of diffusion, except he didn't do a law of diffusion.
But, he is sticky.
His name is attached to everything.
So, that's this concentration model.
It's the one entirely analogous to the temperature.
And the physical setup is the same.
This one is entirely different.
Mixing in this form of this problem has really nothing to do with this model whatsoever.
But, nor does that concentration had anything to do with this concentration, which refers to the result of the mixing in the tank.
But, what happens is the differential equation is the same.
The language of input and response that we talked about is also available here.
So, everything is the same.
And, the most interesting thing is that it shows that the analog of the conductivity, the k, the analog of conductivity and diffusivity is this quantity.
I should not be considering r and v by themselves.
I should be considering as the basic quantity, the ratio of those two.
Now, why is that, is the basic parameter.
What is this?
Well, r is the rate of outflow, and the rate of inflow, what's r over v?
r over v is the fractional rate of outflow.
In other words, if r over v is one tenth, it means that 1/10 of the tank will be emptied in a minute, say.
In other words, we lumped these two constants into a single k, and at the same time have simplified the units.
What are the units?
This is volume per minute.
This is volume.
So, it's simply reciprocal minutes, reciprocal time, which was the same units of that diffusivity and conductivity had, reciprocal time.
The space variables have entirely disappeared.
So, it that way, it's simplified.
It's simplified conceptually, and now, you can answer the same type of questions we asked before about this.
I think it would be better for us to move on, though.
Well, just an example, one really simple thing, so, suppose since we spent so much time worrying about what was happening with sinusoid inputs, I mean, when could Ce be sinusoidal, for example?
Well, roughly sinusoidal if, for example, some factory were polluting.
If this were a lake, and some factory were polluting it, but in the beginning, at the beginning of the day, they produced a lot of the pollutant, and by the end of the day when it wound down, it might well happen that the concentration of pollutants in the incoming stream would vary sinusoidally with a 24 hour cycle.
And then, we would be asking, so, suppose this varies sinusoidally.
In other words, it's like cosine omega t. I'm asking, how closely does the concentration in the tank follow C sub e?
Now, what would that depend upon?
Think about it.
Well, the answer, suppose k is large.
Closely, let's just analyze one case, if k is big.
Now, what would make k big?
We know that from the temperature thing.
If the conductivity is high, then the inner temperature will follow the outer temperature closely, and the same thing with the diffusion model.
But we, of course, therefore, since the equation is the same, we must get the same result here.
Now, what would make k big?
If r is big, if the flow rate is very fast, we will expect the concentration of the inside of that tank to match fairly closely the concentration of the pollutant, of the incoming salt solution, or, if the tank is very small.
For fixed flow rates, if the tank is very small, well, then it gets emptied quickly.
So, both of these are, I think, intuitive results.
And, of course, as before, we got them from that, by trying to analyze the final form of the solution.
In other words, we got them by looking at that form of the solution up there, and seeing if k is big.
As k increases, what happens to the amplitude, and what happens to the phase lag?
But, that summarizes the two.
So, this means, closely means, that the phase lag is, big or small?
The lag is small.
And, the amplitude is, well, the amplitude, the biggest the amplitude could ever be is one because that's the amplitude of this.
So, the amplitude is near one, one because that's the amplitude of the incoming signal, input, whatever you want to call it.
Okay, now, I'd like to spend the rest of the time talking about the failures of number one, and when you have to use number two, and when even number two is no good.
So, let me end first-order equations by putting my worst foot forward.
Well, I'm just trying to avoid disappointment at misapprehensions from you.
I'll watch you leave this room and say, well, he said that, okay.
So, the first one you're going to encounter very shortly where one is not satisfied, but two is, so on some examples where you need two, well, it's going to happen right here.
Somebody, sooner or later, it's going to draw on that loathsome orange chalk, which is unerasable, something which looks like that.
Remember, you saw it here first.
r, yeah, we had that.
Okay, see, I had it in high school too.
That's the capacitance.
This is the resistance.
That's the electromotive force: battery, or a source of alternating current, something like that.
Now, of course, what you're interested in is how the current flows in the circuit.
Since current across the capacitance doesn't make sense, you have to talk about the charge on the capacitance.
So, q, it's customary in a circle this simple to use as the variable not current, but the charge on the capacitance.
And then Kirchhoff's, you are also supposed to know that the derivative, that the time derivative of q is what's called the current in the circuit.
That sort of intuitive.
But, i in a physics class, j in an electrical engineering class, and why, not the letter Y, but why is that?
That's because of electrical engineers use lots of lots of complex numbers and therefore, you have to call current j, I guess.
I think they do j in physics, too, now.
No, no they don't.
I don't know.
So, i is ambiguous if you are in that particular subject.
And it's customary to use, I don't know.
Now it's completely safe.
Okay, where are we?
Well, the law is, the basic differential equation is Kirchhoff's voltage law, but the sum of the voltage drops across these three has to be zero.
So, it's R times i, which is dq / dt.
That's Ohm's law.
That's the voltage drop across resistance.
The voltage drop across the capacitance is Coulomb's law, one form of Coulomb's law.
It's q divided by C. And, that has to be the voltage drop.
And then, there is some sign convention.
So, this is either plus or minus, depending on your sign conventions, but it's E of t. Now, if I put that in standard form, in standard form I probably should say q prime plus q over RC equals, well, I suppose, E over R. And, this is what would appear in the equation.
But, it's not the natural thing.
The k is one over RC.
And, that's the reciprocal.
The RC constant is what everybody knows is important when you talk about a little circuit of that form.
On the other hand, the right-hand side, it's quite unnatural to try to stick in the right-hand side that same RC.
Call this EC over RC.
People don't do that, and therefore, it doesn't really fall into the paradigm of that first equation.
It's the second equation that really falls into the category.
Another simple example of this is chained to k, radioactively changed to k. Well, let's say the radioactive substance, A, decays into, let's say, one atom of this produces one atom of that for simplicity.
So, it decays into B, which then still is radioactive and decays further.
Okay, what's the differential equation, which is going to be, it's going to govern this situation?
What I want to know is how much B there is at any given time.
So, I want a differential equation for the quantity of the radioactive product at any given time.
Well, what's it going to look like?
Well, it's the amount coming in minus the amount going out, so to speak.
The rate of inflow minus the rate of outflow, except it's not the same type of physical flow we had before.
How fast is it coming in?
Well, A is decaying at a certain rate, and so the rate at which A decays is by the basic radioactive law.
It's k1, it's constant, decay constant, times the amount of A present.
If I used the differential equation with A here, I'd have to put a negative sign because it's the rate at which that stuff is leaving A.
But, I'm interested in the rate at which it's coming in to B.
So, it has a positive sign.
And then, the rate at which B is decaying, and therefore the quantity of good B is gone, -- -- that will have some other constant, B.
So, that will be the equation, and to avoid having two dependent variables in there, we know how A is decaying.
So, it's k1, some constant times A, sorry, A will be e to the negative, you know, the decay law, so, times the initial amount that was there times e to the negative k1 t. That's how much A there is at any given time.
It's decaying by the radioactive decay law, minus k2 B.
Okay, so how does the differential equation look like?
It looks like B prime plus k2 B equals an exponential, k1 A zero e to the negative k1 t. But, there's no reason to expect that that constant really has anything to do with k2.
It's unnatural to put it in that form, which is the correct one.
Now, in the last two minutes, I wish to alienate half the class by pointing out that if k is less than zero, none of the terminology of transient, steady-state input response applies.
The technique of solving the equation is identical.
But, you cannot interpret.
So, the technique is the same, and therefore it's worth learning.
The technique is the same.
In other words, the solution will be still e to the negative kt integral q of t e to the kt dt plus a constant times e to the k, oh, this is terrible, no.
dy / dt, let's give an example.
The equation I'm going to look at is something that looks like this: y equals q of t, let's say, okay, a constant, but the constant a is positive.
So, the constant here is negative.
Then, when I solve, my k, in other words, is now properly written as negative a.
And therefore, this formula should now become not this, but the negative k is a t. And, here it's negative a t. And, here it is positive a t. Now, why is it, if this is going to be the solution, why are all those things totally irrelevant?
This is not a transient any longer because if a is positive, this goes to infinity.
Or, if I go to minus infinity, then C is negative.
So, it's not transient.
It's not going to zero, and it depends heavily on the initial conditions.
That means that of these two functions, this is the important guy.
This is just fixed, some fixed function.
Everything, in other words, is going to depend upon the initial conditions, whereas in the other cases we have been studying, the initial conditions after a while don't matter anymore.
Now, why did I say I would alienate half of you?
Well, because in what subjects will a be positive?
In what subjects will k be negative?
k is typically negative in biology, economics, Sloan.
In other words, the simple thing is think of it in biology.
What's the simplest equation for population growth?
Well, it is dP / dt equals some, if the population is growing, a times P, and a is a positive number.
That means P prime minus a P is zero.
So, the thing I want to leave you with is this.
If life is involved, k is likely to be negative.
k is positive when inanimate things are involved; I won't say dead, inanimate.
We're going to start.
We are going to start studying today, and for quite a while, the linear second-order differential equation with constant coefficients.
In standard form, it looks like, there are various possible choices for the variable, unfortunately, so I hope it won't disturb you much if I use one rather than another.
I'm going to write it this way in standard form.
I'll use y as the dependent variable.
Your book uses little p and little q. I'll probably switch to that by next time.
But, for today, I'd like to use the most neutral letters I can find that won't interfere with anything else.
So, of course call the constant coefficients, respectively, capital A and capital B. I'm going to assume for today that the right-hand side is zero.
So, that means it's what we call homogeneous.
The left-hand side must be in this form for it to be linear, it's second order because it involves a second derivative.
These coefficients, A and B, are understood to be constant because, as I said, it has constant coefficients.
Of course, that's not the most general linear equation there could be.
In general, it would be more general by making this a function of the dependent variable, x or t, whatever it's called.
Similarly, this could be a function of the dependent variable.
Above all, the right-hand side can be a function of a variable rather than simply zero.
In that case the equation is called inhomogeneous.
But it has a different physical meaning, and therefore it's customary to study that after this.
You start with this.
This is the case we start with, and then by the middle of next week we will be studying more general cases.
But, it's a good idea to start here.
Your book starts with, in general, some theory of a general linear equation of second-order, and even higher order.
I'm asking you to skip that for the time being.
We'll come back to it next Wednesday, it two lectures, in other words.
I think it's much better and essential for your problems at for you to get some experience with a simple type of equation.
And then, you'll understand the general theory, how it applies, a lot better, I think.
So, let's get experience here.
The downside of that is that I'm going to have to assume a couple of things about the solution to this equation, how it looks; I don't think that will upset you too much.
So, what I'm going to assume, and we will justify it in a couple lectures, that the general solution, that is, the solution involving arbitrary constants, looks like this.
y is equal-- The arbitrary constants occur in a certain special way.
There is c one y one plus c two y two.
So, these are two arbitrary constants corresponding to the fact that we are solving a second-order equation.
In general, the number of arbitrary constants in the solution is the same as the order of the equation because if it's a second-order equation because if it's a second-order equation, that means somehow or other, it may be concealed.
But you're going to have to integrate something twice to get the answer.
And therefore, there should be two arbitrary constants.
That's very rough, but it sort of gives you the idea.
Now, what are the y1 and y2?
Well, as you can see, if these are arbitrary constants, if I take c2 to be zero and c1 to be one, that means that y1 must be a solution to the equation, and similarly y2.
So, where y1 and y2 are solutions.
Now, what that shows you is that the task of solving this equation is reduced, in some sense, to finding just two solutions of it, somehow.
All we have to do is find two solutions, and then we will have solved the equation because the general solution is made up in this way by multiplying those two solutions by arbitrary constants and adding them.
So, the problem is, where do we get that solutions from?
But, first of all, or rather, second or third of all, the initial conditions enter into the, I haven't given you any initial conditions here, but if you have them, and I will illustrate them when I work problems, the initial conditions, well, the initial values are satisfied by choosing c1 and c2, are satisfied by choosing c1 and c2 properly.
So, in other words, if you have an initial value problem to solve, that will be taken care of by the way those constants, c, enter into the solution.
Okay, without further ado, there is a standard example, which I wish I had looked up in the physics syllabus for the first semester.
Did you study the spring-mass-dashpot system in 8.01?
I'm embarrassed having to ask you.
You did?
Raise your hands if you did.
Okay, that means you all did.
Well, just let me draw an instant picture to remind you.
So, this is a two second review.
I don't know how they draw the picture.
Probably they don't draw picture at all.
They have some elaborate system here of the thing running back and forth.
Well, in the math, we do that all virtually.
So, here's my system.
That's a fixed thing.
Here's a little spring.
And, there's a little car on the track here, I guess.
So, there's the mass, some mass in the little car, and motion is damped by what's called a dashpot.
A dashpot is the sort of thing, you see them in everyday life as door closers.
They're the thing up above that you never notice that prevent the door slamming shut.
So, if you take one apart, it looks something like this.
So, that's the dash pot.
It's a chamber with a piston.
This is a piston moving in and out, and compressing the air, releasing it, is what damps the motion of the thing.
So, this is a dashpot, it's usually called.
And, here's our mass in that little truck.
And, here's the spring.
And then, the equation which governs it is, let's call this x. I'm already changing, going to change the dependent variable from y to x, but that's just for the sake of example, and because the track is horizontal, it seems more natural to call it x.
There's some equilibrium position somewhere, let's say, here.
That's the position at which the mass wants to be, if the spring is not pulling on it or pushing on it, and the dashpot is happy.
I guess we'd better have a longer dashpot here.
So, this is the equilibrium position where nothing is happening.
When you depart from that position, then the spring, if you go that way, the spring tries to pull the mass back.
If it goes on the site, the spring tries to push the mass away.
The dashpot, meanwhile, is doing its thing.
And so, the force on the, m x double prime.
That's by Newton's law, the force, comes from where?
Well, there's the spring pushing and pulling on it.
That force is opposed.
If x gets to be beyond zero, then the spring tries to pull it back.
If it gets to the left of zero, if x gets to be negative, that that spring force is pushing it this way, wants to get rid of the mass.
So, it should be minus kx, and this is from the spring, the fact that is proportional to the amount by which x varies.
So, that's called Hooke's Law.
Never mind that.
This is a law.
That's a law, Newton's law, okay, Newton, Hooke with an E, and the dashpot damping is proportional to the velocity.
It's not doing anything if the mass is not moving, even if it's stretched way out of its equilibrium position.
So, it resists the velocity.
If the thing is trying to go that way, the dashpot resists it.
It's trying to go this way, the dashpot resists that, too.
It's always opposed to the velocity.
And so, this is a dash pot damping.
I don't know whose law this is.
So, it's the force coming from the dashpot.
And, when you write this out, the final result, therefore, is it's m x double prime plus c x prime, it's important to see where the various terms are, plus kx equals zero.
And now, that's still not in standard form.
To put it in standard form, you must divide through by the mass.
And, it will now read like this, plus k divided by m times x equals zero.
And, that's the equation governing the motion of the spring.
I'm doing this because your problem set, problems three and four, ask you to look at a little computer visual which illustrates a lot of things.
And, I didn't see how it would make, you can do it without this interpretation of spring-mass-dashpot, -- -- but, I think thinking of it of these constants as, this is the damping constant, and this is the spring, the constant which represents the force being exerted by the spring, the spring constant, as it's called, makes it much more vivid.
So, you will note is that those problems are labeled Friday or Monday.
Make it Friday.
You can do them after today if you have the vaguest idea of what I'm talking about.
If not, go back and repeat 8.01.
So, all this was just an example, a typical model.
But, by far, the most important simple model.
Okay, now what I'd like to talk about is the solution.
What is it I have to do to solve the equation?
So, to solve the equation that I outlined in orange on the board, the ODE, our task is to find two solutions.
Now, don't make it too trivial.
There is a condition.
The solution should be independent.
All that means is that y2 should not be a constant multiple of y1.
I mean, if you got y1, then two times y1 is not an acceptable value for this because, as you can see, you really only got one there.
You're not going to be able to make up a two parameter family.
So, the solutions have to be independent, which means, to repeat, that neither should be a constant multiple of the other.
They should look different.
That's an adequate explanation.
Okay, now, what's the basic method to finding those solutions?
Well, that's what we're going to use all term long, essentially, studying equations of this type, even systems of this type, with constant coefficients.
The basic method is to try y equals an exponential.
Now, the only way you can fiddle with an exponential is in the constant that you put up on top.
So, I'm going to try y equals e to the rt.
Notice you can't tell from that what I'm using as the independent variable.
But, this tells you I'm using t. And, I'm switching back to using t as the dependent variable.
So, T is the independent variable.
Why do I do that?
The answer is because somebody thought of doing it, probably Euler, and it's been a tradition that's handed down for the last 300 or 400 years.
Some things we just know.
All right, so if I do that, as you learned from the exam, it's very easy to differentiate exponentials.
That's why people love them.
It's also very easy to integrate exponentials.
And, half of you integrated instead of differentiating.
So, we will try this and see if we can pick r so that it's a solution.
Okay, well, I will plug in, then.
Substitute, in other words, and what do we get?
Well, for y double prime, I get r squared e to the rt.
That's y double prime because each time you differentiate it, you put an extra power of r out in front.
But otherwise, do nothing.
The next term will be r times, sorry, I forgot the constant.
Capital A times r e to the rt, and then there's the last term, B times y itself, which is B e to the rt.
And, that's supposed to be equal to zero.
So, I have to choose r so that this becomes equal to zero.
Now, you see, the e to the rt occurs as a factor in every term, and the e to the rt is never zero.
And therefore, you can divide it out because it's always a positive number, regardless of the value of t. So, I can cancel out from each term.
And, what I'm left with is the equation r squared plus ar plus b equals zero.
We are trying to find values of r that satisfy that equation.
And that, dear hearts, is why you learn to solve quadratic equations in high school, in order that in this moment, you would be now ready to find how spring-mass systems behave when they are damped.
This is called the characteristic equation.
The characteristic equation of the ODE, or of the system of the spring mass system, which it's modeling, the characteristic equation of the system, okay?
Okay, now, we solve it, but now, from high school you know there are several cases.
And, each of those cases corresponds to a different behavior.
And, the cases depend upon what the roots look like.
The possibilities are the roots could be real, and distinct.
That's the easiest case to handle.
The roots might be a pair of complex conjugate numbers.
That's harder to handle, but we are ready to do it.
And, the third case, which is the one most in your problem set is the most puzzling: when the roots are real, and equal.
And, I'm going to talk about those three cases in that order.
So, the first case is the roots are real and unequal.
If I tell you they are unequal, and I will put down real to make that clear.
Well, that is by far the simplest case because immediately, one sees we have two roots.
They are different, and therefore, we get our two solutions immediately.
So, the solutions are, the general solution to the equation, I write down without further ado as y equals c1 e to the r1 t plus c2 e to the r2 t. There's our solution.
Now, because that was so easy, and we didn't have to do any work, I'd like to extend this case a little bit by using it as an example of how you put in the initial conditions, how to put in the c. So, let me work a specific numerical example, since we are not going to try to do this theoretically until next Wednesday.
Let's just do a numerical example.
So, suppose I take the constants to be the damping constant to be a four, and the spring constant, I'll take the mass to be one, and the spring constant to be three.
So, there's more damping here, damping force here.
You can't really talk that way since the units are different.
But, this number is bigger than that one.
That seems clear, at any rate.
Okay, now, what was the characteristic equation?
Look, now watch.
Please do what I do.
I've found in the past, even by the middle of the term, there are still students who feel that they must substitute y equals e to the rt, and go through that whole little derivation to find that you don't do that.
It's a waste of time.
I did it that you might not ever have to do it again.
Immediately write down the characteristic equation.
That's not very hard.
r squared plus 4r plus three equals zero.
And, if you can write down its roots immediately, splendid.
But, let's not assume that level of competence.
So, it's r plus three times r plus one equals zero.
Okay, you factor it.
This being 18.03, a lot of the times the roots will be integers when they are not, God forbid, you will have to use the quadratic formula.
But here, the roots were integers.
It is, after all, only the first example.
So, the solution, the general solution is y equals c1 e to the negative, notice the root is minus three and minus one, minus 3t plus c2 e to the negative t. Now, suppose it's an initial value problem.
So, I gave you an initial condition.
Suppose the initial conditions were that y of zero were one.
So, at the start, the mass has been moved over to the position, one, here.
Well, we expected it, then, to start doing that.
But, this is fairly heavily damped.
This is heavily damped.
I'm going to assume that the mass starts at rest.
So, the spring is distended.
The masses over here.
But, there's no motion at times zero this way or that way.
In other words, I'm not pushing it.
I'm just releasing it and letting it do its thing after that.
Okay, so y prime of zero, I'll assume, is zero.
So, it starts at rest, but in the extended position, one unit to the right of the equilibrium position.
Now, all you have to do is use these two conditions.
Notice I have to have two conditions because there are two constants I have to find the value of.
All right, so, let's substitute, well, we're going to have to calculate the derivative.
So, why don't we do that right away?
So, this is minus three c1 e to the minus 3t minus c2 e to the negative t. And now, if I substitute in at zero, when t equals zero, what do I get?
Well, the first equation, the left says that y of zero should be one.
And, the right says this is one.
So, it's c1 plus c2.
That's the result of substituting t equals zero.
How about substituting?
What should I substitute in the second equation?
Well, y prime of zero is zero.
So, if the second equation, when I put in t equals zero, the left side is zero according to that initial value, and the right side is negative three c1 minus c2.
You see what you end up with, therefore, is a pair of simultaneous linear equations.
And, this is why you learn to study linear set of pairs of simultaneous linear equations in high school.
These are among the most important.
Solving problems of this type are among the most important applications of that kind of algebra, and this kind of algebra.
All right, what's the answer finally?
Well, if I add the two of them, I get minus 2c1 equals one.
So, c1 is equal to minus one half.
And, if c1 is minus a half, then c2 is minus 3c1.
So, c2 is three halves.
The final question is, what does that look like as a solution?
Well, in general, these combinations of two exponentials aren't very easy to plot by yourself.
That's one of the reasons you are being given this little visual which plots them for you.
All you have to do is, as you'll see, set the damping constant, set the constants, set the initial conditions, and by magic, the curve appears on the screen.
And, if you change either of the constants, the curve will change nicely right along with it.
So, the solution is y equals minus one half e to the minus 3t plus three halves e to the negative t. How does it look?
Well, I don't expect you to be able to plot that by yourself, but you can at least get started.
It does have to satisfy the initial conditions.
That means it should start at one, and its starting slope is zero.
So, it starts like that.
These are both declining exponentials.
This declines very rapidly, this somewhat more slowly.
It does something like that.
If this term were a lot, lot more negative, I mean, that's the way that particular solution looks.
How might other solutions look?
I'll draw a few other possibilities.
If the initial term, if, for example, the initial slope were quite negative, well, that would have start like this.
Now, just your experience of physics, or of the real world suggests that if I give, if I start the thing at one, but give it a strongly negative push, it's going to go beyond the equilibrium position, and then come back again.
But, because the damping is big, it's not going to be able to get through that.
The equilibrium position, a second time, is going to look something like that.
Or, if I push it in that direction, the positive direction, that it starts off with a positive slope.
But it loses its energy because the spring is pulling it.
It comes and does something like that.
So, in other words, it might go down.
Cut across the equilibrium position, come back again, it do that?
No, that it cannot do.
I was considering giving you a problem to prove that, but I got tired of making out the problems set, and decided I tortured you enough already, as you will see.
So, anyway, these are different possibilities for the way that can look.
This case, where it just returns in the long run is called the over-damped case, over-damped.
Now, there is another case where the thing oscillates back and forth.
We would expect to get that case if the damping is very little or nonexistent.
Then, there's very little preventing the mass from doing that, although we do expect if there's any damping at all, we expect it ultimately to get nearer and nearer to the equilibrium position.
Mathematically, what does that correspond to?
Well, that's going to correspond to case two, where the roots are complex.
The roots are complex, and this is why, let's call the roots, in that case we know that the roots are of the form a plus or minus bi.
There are two roots, and they are a complex conjugate.
All right, let's take one of them.
What does a correspond to in terms of the exponential?
Well, remember, the function of the r was, it's this r when we tried our exponential solution.
So, what we formally, this means we get a complex solution.
The complex solution y equals e to this, let's use one of them, let's say, (a plus bi) times t. The question is, what do we do that?
We are not really interested, I don't know what a complex solution to that thing means.
It doesn't have any meaning.
What I want to know is how y behaves or how x behaves in that picture.
And, that better be a real function because otherwise I don't know what to do with it.
So, we are looking for two real functions, the y1 and the y2.
But, in fact, what we've got is one complex function.
All right, now, a theorem to the rescue: this, I'm not going to save for Wednesday because it's so simple.
So, the theorem is that if you have a complex solution, u plus iv, so each of these is a function of time, u plus iv is the complex solution to a real differential equation with constant coefficients.
Well, it doesn't have to have constant coefficients.
It has to be linear.
Let me just write it out to y double prime plus A y prime plus B y equals zero.
Suppose you got a complex solution to that equation.
These are understood to be real numbers.
They are the damping constant and the spring constant.
Then, the conclusion is that u and v are real solutions.
In other words, having found a complex solution, all you have to do is take its real and imaginary parts, and voila, you've got your two solutions you were looking for for the original equation.
Now, that might seem like magic, but it's easy.
It's so easy it's the sort of theorem I could spend one minute proving for you now.
What's the reason for it?
Well, the main thing I want you to get out of this argument is to see that it absolutely depends upon these coefficients being real.
You have to have a real differential equation for this to be true.
Otherwise, it's certainly not.
So, the proof is, what does it mean to be a solution?
It means when you plug in A (u plus iv) plus, prime, plus B times u plus iv, what am I supposed to get?
Zero.
Well, now, separate these into the real and imaginary parts.
What does it say?
It says u double prime plus A u prime plus B u, that's the real part of this expression when I expand it out.
And, I've got an imaginary part, too, which all have the coefficient i.
So, from here, I get v double prime plus i times A v prime plus i times B v. So, this is the imaginary part.
Now, here I have something with a real part plus the imaginary part, i, times the imaginary part is zero.
Well, the only way that can happen is if the real part is zero, and the imaginary part is separately zero.
So, the conclusion is that therefore this part must be zero, and therefore this part must be zero because the two of them together make the complex number zero plus zero i.
Now, what does it mean for the real part to be zero?
It means that u is a solution.
This, the imaginary part zero means v is a solution, and therefore, just what I said.
u and v are solutions to the real equation.
Where did I use the fact that A and B were real numbers and not complex numbers?
In knowing that this is the real part, I had to know that A was a real number.
If A were something like one plus i, I'd be screwed, I mean, because then I couldn't say that this was the real part anymore.
So, saying that's the real part, and this is the imaginary part, I was using the fact that these two numbers, constants, were real constants: very important.
So, what is the case two solution?
Well, what are the real and imaginary parts of (a plus b i) t?
Well, y equals e to the at + ibt.
Okay, you've had experience.
You know how to do this now.
That's e to the at times, well, the real part is, well, let's write it this way.
The real part is e to the at times cosine b t. Notice how the a and b enter into the expression.
That's the real part.
And, the imaginary part is e to the at times the sine of bt.
And therefore, the solution, both of these must, therefore, be solutions to the equation.
And therefore, the general solution to the ODE is y equals, now, you've got to put in the arbitrary constants.
It's a nice thing to do to factor out the e to the at.
It makes it look a little better.
And so, the constants are c1 cosine bt and c2 sine bt.
Yeah, but what does that look like?
Well, you know that too.
This is an exponential, which controls the amplitude.
But this guy, which is a combination of two sinusoidal oscillations with different amplitudes, but with the same frequency, the b's are the same in both of them, and therefore, this is, itself, a purely sinusoidal oscillation.
So, in other words, I don't have room to write it, but it's equal to, you know.
It's a good example of where you'd use that trigonometric identity I spent time on before the exam.
Okay, let's work a quick example just to see how this works out.
Well, let's get rid of this.
Okay, let's now make the damping, since this is showing oscillations, it must correspond to the case where the damping is less strong compared with the spring constant.
So, the theorem is that if you have a complex solution, u plus iv, so each of these is a function of time, u plus iv is the complex solution to a real differential equation with constant coefficients.
A stiff spring, one that pulls with hard force is going to make that thing go back and forth, particularly at the dipping is weak.
So, let's use almost the same equation as I've just concealed.
But, do you remember a used four here?
Okay, before we used three and we got the solution to look like that.
Now, we will give it a little more energy by putting some moxie in the springs.
So now, the spring is pulling a little harder, bigger force, a stiffer spring.
Okay, the characteristic equation is now going to be r squared plus 4r plus five is equal to zero.
And therefore, if I solve for r, I'm not going to bother trying to factor this because I prepared for this lecture, and I know, quadratic formula time, minus four plus or minus the square root of b squared, 16, minus four times five, 16 minus 20 is negative four all over two.
And therefore, that makes negative two plus or minus, this makes, simply, i.
2i divided by two, which is i.
So, the exponential solution is e to the negative two, you don't have to write this in.
You can get the thing directly.
t, let's use the one with the plus sign, and that's going to give, as the real solutions, e to the negative two t times cosine t, and e to the negative 2t times the sine of t. And therefore, the solution is going to be y equals e to the negative 2t times (c1 cosine t plus c2 sine t).
If you want to put initial conditions, you can put them in the same way I did them before.
Suppose we use the same initial conditions: y of zero equals one, and y prime of zero equals one, equals zero, let's say, wait, blah, blah, blah, zero, yeah.
Okay, I'd like to take time to actually do the calculation, but there's nothing new in it.
I'd have to take, calculate the derivative here, and then I would substitute in, solve equations, and when you do all that, just as before, the answer that you get is y equals e to the negative 2t, so, choose the constants c1 and c2 by solving linear equations, and the answer is cosine t, so, c1 turns out to be one, and c2 turns out to be two, I hope.
Okay, I want to know, but what does that look like?
Well, use that trigonometric identity.
The e to the negative 2t is just a real factor which is going to reproduce itself.
The question is, what is cosine t plus 2 sine t look like?
What's its amplitude as a pure oscillation?
It's the square root of one plus two squared.
Remember, it depends on looking at that little triangle, which is one, two, this is a different scale than that.
And here is the square root of five, right?
And, here is phi, the phase lag.
So, it's equal to the square root of five.
So, it's the square root of five times e to the negative 2t, and the stuff inside is the cosine of, the frequency is one.
Circular frequency is one, so it's t minus phi, where phi is this angle.
How big is that, one and two?
Well, if this were the square root of three, which is a little less than two, it would be 60 infinity.
So, this must be 70 infinity.
So, phi is 70 infinity plus or minus five, let's say.
So, it looks like a slightly delayed cosine curve, but the amplitude is falling.
So, it has to start.
So, if I draw it, here's one, here is, let's say, the square root of five up about here.
Then, the square root of five times e to the negative 2t looks maybe something like this.
So, that's square root of five e to the negative 2t.
This is cosine t, but shoved over by not quite pi over two.
It starts at one, and with the slope zero.
So, the solution starts like this.
It has to be guided in its amplitude by this function out there, and in between it's the cosine curve.
But it's moved over.
So, if this is pi over two, the first time it crosses, it's 72 infinity to the right of that.
So, if this is pi over two, it's pi over two plus 70 infinity where it crosses.
So, it must be doing something like this.
And now, on the other side, it's got to stay within the same amplitude.
So, it must be doing something like this.
Okay, that gets us to, if this is the under-damped case, because if you're trying to do this with a swinging door, it means the door's going to be swinging back and forth.
Or, our little mass now hidden, but you could see it behind that board, is going to be doing this.
But, it never stops.
It never stops.
It doesn't realize, but not in theoretical life.
So, this is the under-damped.
All right, so it's like Goldilocks and the Three Bears.
That's too hot, and this is too cold.
What is the thing which is just right?
Well, that's the thing you're going to study on the problem set.
So, just right is called critically damped.
It's what people aim for in trying to damp motion that they don't want.
Now, what's critically damped?
It must be the case just in between these two.
Neither complex, nor the roots different.
It's the case of two equal roots.
So, r squared plus Ar plus B equals zero has two equal roots.
Now, that's a very special equation.
Suppose we call the root, since all of these, notice these roots in this physical case.
The roots always turn out to be negative numbers, or have a negative real part.
I'm going to call the root a.
So, r equals negative a, the root.
a is understood to be a positive number.
I want that root to be really negative.
Then, the equation looks like, the characteristic equation is going to be r plus a, right, if the root is negative a, squared because it's a double root.
And, that means the equation is of the form r squared plus two times a r plus a squared equals zero.
In other words, the ODE looked like this.
The ODE looked like y double prime plus 2a y prime plus, in other words, the damping and the spring constant were related in this special wake, that for a given value of the spring constant, there was exactly one value of the damping which produced this in between case.
Now, what's the problem connected with it?
Well, the problem, unfortunately, is staring us in the face when we want to solve it.
The problem is that we have a solution, but it is y equals e to the minus at.
I don't have another root to get another solution with.
And, the question is, where do I get that other solution from?
Now, there are three ways to get it.
Well, there are four ways to get it.
You look it up in Euler.
That's the fourth way.
That's the real way to do it.
But, I've given you one way as problem number one on the problem set.
I've given you another way as problem number two on the problem set.
And, the third way you will have to wait for about a week and a half.
And, I will give you a third way, too.
By that time, you won't want to see any more ways.
But, I'd like to introduce you to the way on the problem set.
And, it is this, that if you know one solution to an equation, which looks like a linear equation, in fact, the piece can be functions of t. They don't have to be constant, so I'll use the books notation with p's and q's.
y prime plus q y equals zero.
If you know one solution, there's an absolute, ironclad guarantee, if you'll know that it's true because I'm asking you to prove it for yourself.
There's another of the form, having this as a factor, one solution y one, let's call it, y equals y1 u is another solution.
And, you will be able to find u, I swear.
Now, let's, in the remaining couple of minutes carry that out just for this case because I want you to see how to arrange the work nicely.
And, I want you to arrange your work when you do the problem sets in the same way.
So, the way to do it is, the solution we know is e to the minus at.
So, we are going to look for a solution to this differential equation.
That's the differential equation.
And, the solution we are going to look for is of the form e to the negative at times u.
Now, you're going to have to make calculation like this several times in the course of the term.
Do it this way.
y prime equals, differentiate, minus a e to the minus a t u plus e to the minus a t u prime.
And then, differentiate again.
The answer will be a squared.
You differentiate this: a squared e to the negative at u. I'll have to do this a little fast, but the next term will be, okay, minus, so this times u prime, and from this are you going to get another minus.
So, combining what you get from here, and here, you're going to get minus 2a e to the minus a t u prime.
And then, there is a final term, which comes from this, e to the minus a t u double prime.
Two of these, because of a piece here and a piece here combine to make that.
And now, to plug into the equation, you multiply this by one.
In other words, you don't do anything to it.
You multiply this line by 2a, and you multiply that line by a squared, and you add them.
On the left-hand side, I get zero.
What do I get on the right?
Notice how I've arrange the work so it adds nicely.
This has a squared times this, plus 2a times that, plus one times that makes zero.
2a times this plus one times this makes zero.
All that survives is e to the a t u double prime, and therefore, e to the minus a t u double prime is equal to zero.
So, please tell me, what's u double prime?
It's zero.
So, please tell me, what's u?
It's c1 t plus c2.
Now, that gives me a whole family of solutions.
Just t would be enough because all I am doing is looking for one solution that's different from e to the minus a t. And, that solution, therefore, is y equals e to the minus a t times t. And, there's my second solution.
So, this is a solution of the critically damped case.
And, you are going to use it in three or four of the different problems on the problem set.
But, I think you can deal with virtually the whole problem set, except for the last problem, now.
This is a brief, so, the equation, and we got the characteristic equation from the last time.
The general topic for today is going to be oscillations, which are extremely important in the applications and in everyday life.
But, the oscillations, we know, are associated with a complex root.
So, they correspond to complex roots of the characteristic equation.
r squared plus br plus k equals zero.
I'd like to begin.
Most of the lecture will be about discussing the relations between these numbers, these constants, and the various properties that the solutions, oscillatory solutions, have.
But, before that, I'd like to begin by clearing up a couple of questions almost everybody has at some point or other when they study the case of complex roots.
Complex roots are the case which produce oscillations in the solutions.
That's the relation, and that's why I'm talking about this for the first few minutes.
Now, what is the problem?
The complex roots, of course, there will be two roots, and they occur at the complex conjugates of each other.
So, they will be of the form a plus or minus bi.
Last time, I showed you, I took the root r equals a plus bi, which leads to the solution.
The corresponding solution is a complex solution which is e to the at, (a plus i b)t. And, what we did was the problem was to get real solutions out of that.
We needed two real solutions, and the way I got them was by separating this into its real part and its imaginary part.
And, I proved a little theorem for you that said both of those give solutions.
So, the real part was e to the a t times cosine b t, and the imaginary part was e to the at sine b t. And, those were the two solutions.
So, here was y1.
And, the point was those, out of the complex solutions, we got real solutions.
We have to have real solutions because we live in the real world.
The equation is real.
Its coefficients are real.
They represent real quantities.
That's the way the solutions, therefore, have to be.
So, these, the point is, these are now real solutions, these two guys, y1 and y2.
Now, the first question almost everybody has, and I was pleased to see at the end of the lecture, a few people came up and asked me, yeah, well, you took a plus bi, but there was another root, a minus bi.
You didn't use that one.
That would give two more solutions, right?
Of course, they didn't say that.
They were too smart.
They just said, what about that other root?
Well, what about it?
The reason I don't have to talk about the other root is because although it does give to solutions, it doesn't give two new ones.
Maybe I can indicate that most clearly here even though you won't be able to take notes by just using colored chalk.
Suppose, instead of plus bi, I used a minus bi.
What would have changed?
Well, this would now become minus here.
Would this change?
No, because e to the minus ibt is the cosine of minus b, but that's the same as the cosine of b.
How about here?
This would become the sine of minus bt.
But that's simply the negative of the sine of bt.
So, the only change would have been to put a minus sign there.
Now, I don't care if I get y2 or negative y2 because what am I going to do with it?
When I get it, I'm going to write y, the general solution, as c1 y1 plus c2 y2.
So, if I get negative y2, that just changes that arbitrary constant from c2 to minus c2, which is just as arbitrary a constant.
So, in other words, there's no reason to use the other root because it doesn't give anything new.
Now, there the story could stop.
And, I would like it to stop, frankly, but I don't dare because there's a second question.
And, I'm visiting recitations not this semester, but in previous semesters.
In 18.03, so many recitations do this.
I have to partly inoculate you against it, and partly tell you that some of the engineering courses do do it, and therefore you probably should learn it also.
So, there is another way of proceeding, which is what you might have thought.
Hey, look, we got two complex roots.
That gives us two solutions, which are different.
Neither one is a constant multiple of the other.
So, the other approach is, use, as a general solution, y equals, now, I'm going to put a capital C here.
You will see why in just a second, times e to the (a plus b i) times t. And then, I will use the other solution: C2 times e to the (a minus b i) t. These are two independent solutions.
And therefore, can't I get the general solution in that form?
Now, in a sense, you can.
The whole problem is the following, of course, that I'm only interested in real solutions.
This is a complex function.
This is another complex function.
It's got an i in it, in other words, when I write it out as u plus iv.
If I expect to be able to get a real solution out of that, that means I have to make, allow these coefficients to be complex numbers, and not real numbers.
So, in other words, what I'm saying is that an expression like this, where the a plus bi and a minus bi are complex roots of that characteristic equation, is formally a very general, complex solution to the equation.
And therefore, the problem becomes, how, from this expression, do I get the real solutions?
So, the problem is, I accept these as the complex solutions.
My problem is, to find among all these guys where C1 and C2 are allowed to be complex, the problem is, which of the green solutions are real?
Now, there are many ways of getting the answer.
There is a super hack way.
The super hack way is to say, well, this one is C1 plus i d1.
This is C2 plus i d2.
And, I'll write all this out in terms of what it is, you know, cosine plus i sine, and don't forget the e to the at.
And, I will write it all out, and it will take an entire board.
And then, I will just see what the condition is.
I'll write its real part, and its imaginary part.
And then, I will say the imaginary part has got to be zero.
And, then I will see what it's like.
That works fine.
It just takes too much space.
And also, it doesn't teach you a few things that I think you should know.
So, I'm going to give another.
So, let's say we can answer this two ways: by hack, in other words, multiply everything out.
Multiply all out, make the imaginary part equal zero.
Now, here's a better way, in my opinion.
What I'm trying to do is, this is some complex function, u plus iv.
How do I know when a complex function is real?
I want this to be real.
Well, the hack method corresponds to, say, v must be equal to zero.
It's real if v is zero.
So, expand it out, and see why v is zero.
There's a slightly more subtle method, which is to change i to minus i.
And, what?
And, see if it stays the same because if I change i to minus i and it turns out, the expression doesn't change, then it must have been real, if the expression doesn't change when I change I to minus I.
Well, sure.
But you will see it works.
Now, that's what I'm going to apply to this.
If I want this to be real, I phrase the question, I rephrase the question for the green solution as change, so I'm going to change i to minus i in the green thing, and that's going to give me what conditions, and that will give conditions on the C's.
Well, let's do it.
In fact, it's easier done than talked about.
Let's change, take the green solution, and change.
Well, I better recopy it, C1.
So, these are complex numbers.
That's why I wrote them as capital letters because little letters you tend to interpret as real numbers.
So, C1 e to the (a plus b i)t, I'll recopy it quickly, plus C2 e to the (a minus b i).
Okay, we're going to change i to negative i.
Now, here's a complex number.
What happens to it when you change i to negative i?
You change it into its-- Class?
What do we change it to?
Its complex conjugate.
And, the notation for complex conjugate is you put a bar over it.
So, in other words, when I do that, the C1 changes to C1 bar, complex conjugate, the complex conjugate of C1.
What happens to this guy?
This guy changes to e to the (a minus b i) t. This changes to the complex conjugate of C2 now, times e to the (a plus b i) t. Well, I want these two to be the same.
I want the two expressions the same.
Why do I want them the same?
Because, if there's no change, that will mean that it's real.
Now, when is that going to happen?
That happens if, well, here is this, that.
If C2 should be equal to C1 bar, that's only one condition.
There's another condition.
C2 bar should equal C1.
So, I get two conditions, but there's really only one condition there because if this is true, that's true too.
I simply put bars over both things, and two bars cancel each other out.
If you take the complex conjugate and do it again, you get back where you started.
Change i to minus i, and then i to minus i again.
Well, never mind.
Anyway, these are the same.
This equation doesn't say anything that the first one didn't say already.
So, this one is redundant.
And, our conclusion is that the real solutions to the equation are, in their entirety, I now don't need both C2 and C1.
One of them will do, and since I'm going to write it out as a complex number, I will write it out in terms of its coefficient.
So, it's C1.
Let's just simply write it.
C plus i times d, that's the coefficient.
That's what I called C1 before.
And, that's times e to the (a plus b i) t. There's no reason why I put bi here and id there, in case you're wondering, sheer caprice.
And what's the other term?
Now, the other term is completely determined.
Its coefficient must be C minus i d times e to the (a minus b i) t. In other words, this thing is perfectly general.
Any complex number times that first root you use, exponentiated, and the second term can be described as the complex conjugate of the first.
The coefficient is the complex conjugate, and this part is the complex conjugate of that.
Now, it's in this form, many engineers write the solution this way, and physicists, too, so, scientists and engineers we will include.
Write the solution this way.
Write the real solutions this way in that complex form.
Well, why do they do something so perverse?
You will have to ask them.
But, in fact, when we studied Fourier series, we will probably have to do something, have to do that at one point.
If you work a lot with complex numbers, it turns out to be, in some ways, a more convenient representation than the one I've given you in terms of sines and cosines.
Well, from this, how would I get, suppose I insisted, well, if someone gave it to me in that form, I don't see how I would convert it back into sines and cosines.
And, I'd like to show you how to do that efficiently, too, because, again, it's one of the fundamental techniques that I think you should know.
And, I didn't get a chance to say it when we studied complex numbers that first lecture.
It's in the notes, but it doesn't prove anything since I don't think it made you use it in an example.
So, the problem is, now, by way of finishing this up, too, to change this to the old form, I mean the form involving sines and cosines.
Now, again, there are two ways of doing it.
The hack way is you write it all out.
Well, e to the (a plus b i)t turns into e to the a t times cosine this plus i sine that.
And, the other term does, too.
And then you've got stuff out front.
And, the thing stretches over two boards.
But you group all the terms together.
You finally get it.
By the way, when you do it, you'll find that the imaginary part disappears completely.
It has to because that's the way we chose the coefficients.
So, here's the hack method.
Write it all out: blah, blah, blah, blah, blah, blah, blah, and nicer.
Nicer, and teach you something you're supposed to know.
Write it this way.
First of all, you notice that both terms have an e to the a t factor.
Let's get rid of that right away.
I'm pulling it out front because that's automatically real, and therefore, isn't going to affect the rest of the answer at all.
So, let's pull out that, and what's left?
Well, what's left, you see, involves just the two parameters, C and d, so I'm going to have a C term.
And, I'm going to have a d term.
What multiplies the arbitrary constant, C?
Answer: after I remove the e to the a t, what multiplies it is, e to the b i t plus e to the -- e to the b i t. Let's write it i b t. And, the other term is plus e to the negative i b t. See how I got that, pulled it out?
And, how about the d?
What goes with d?
d goes with, well, first of all, there's an I in front that i better not forget.
And then, the rest of it is i.
So, it's i d times, it's e to the b i t, e to the i b t minus, now, e to the minus i b t. So, that's the way the solution looks.
It doesn't look a lot better, but now you must use the magic formulas, which, I want you to know as well as you know Euler's formula, even better than you know Euler's formula.
They're a consequence of Euler's formula.
They're Euler's formula read backwards.
Euler's formula says you've got a complex exponential here.
Here's how to write it in terms of sines and cosines.
The backwards thing says you've got a sine or a cosine.
Here is the way to write it in terms of complex exponentials.
And, remember, the way to do it is, cosine a is equal to e to the i a t, i a, plus e to the negative i a divided by two.
And, sine of a is almost the same thing, except you use a minus sign.
And, what everybody forgets, you have to divide by i.
So, this is a backwards version of Euler's formula.
The two of them taken together are equivalent to Euler's formula.
If I took cosine a, multiply this through by i, and added them up, on the right-hand side I'd get exactly e to the ia.
I'd get Euler's formula, in other words.
All right, so, what does this come out to be, finally?
This particular sum of exponentials, you should always recognize as real.
You know it's real because when you change i to minus i, the two terms switch.
And therefore, the expression doesn't change.
What is it?
This part is twice the cosine of bt.
What's this part?
This part is 2 i times the sine of bt.
And so, what does the whole thing come to be?
It is e to the a t times 2C cosine bt plus i times, did I lose possibly a, no it's okay, minus i times i is minus, so, minus 2d times the sine of bt.
Shall I write that out?
So, in other words, it's e to the a t times 2C cosine b t minus 2d times the sine of b t, which is, since 2C and negative 2d are just arbitrary constants, just as arbitrary as the constants of C and d themselves are.
This is our old form of writing the real solution.
Here's the way using science and cosines, and there's the way that uses complex numbers and complex functions throughout.
Notice they both have two arbitrary constants in them, C and d, two arbitrary constants.
That, you expect.
But that has two arbitrary constants in it, too, just the real and imaginary parts of that complex coefficient, C plus i d. Well, that took half the period, and it was a long, I don't consider it a digression because learning those ways of dealing with complex numbers of complex functions is a fairly important goal in this course, actually.
But let's get back now to studying what the oscillations actually look like.
Okay, well, I'd like to save a little time, but very quickly, you don't have to reproduce this sketch.
I remember very well from Friday to Monday, but I can't expect you to for a variety of reasons.
I mean, I have to think about this stuff all weekend.
And you, God forbid.
So, here's the picture, and I won't explain anymore what's in it, except there's the mass.
Here is the spring constant, the spring with its constant here.
Here's the dashpot with its constant.
The equation is from Newton's law: m x double, so this will be x, and here's, let's say, the equilibrium point is over here.
It looks like m x double prime; we derived this last time, plus c x prime plus k x equals zero.
And now, if I put that in standard form, it's going to look like x double prime plus c over m x prime plus k over m times x equals zero.
And, finally, the standard form in which your book writes it, which is good, it's a standard form in general that is used in the science and engineering courses.
One writes this as, just to be perverse, I'm going to change x back to y, okay, mostly just to be eclectic, to get you used to every conceivable notation.
So, I'm going to write this to change x to y.
So, that's going to become y double prime.
And now, this is given a new name, p, except to get rid of lots of twos, which would really screw up the formulas, make it 2p.
You will see why in a minute.
So, there's 2p times y prime, and this thing we are going to call omega nought squared.
Now, that's okay.
It's a positive number.
Any positive number is the square of some other positive number.
Take a square root.
You will see why, it makes the formulas much pretty to call it that.
And, it makes also a lot of things much easier to remember.
So, all I'm doing is changing the names of the constants that way in order to get better formulas, easier to remember formulas at the end.
Now, we are interested in the case where there is oscillations.
In other words, I only care about the case in which this has complex roots, because if it has just real roots, that's the over-damped case.
I don't get any oscillations.
By far, oscillations are by far the more important of the cases, I mean, just because, I don't know, I could go on for five minutes listing things that oscillate, oscillations, you know, like this.
So they can oscillate by going to sleep, and waking up, and going to sleep, and waking up.
They could oscillate.
So, that means we're going to get complex roots.
The characteristic equation is going to be r squared plus 2p.
So, p is a constant, now, right?
Often, p I use in this position to indicate a function of t. But here, p is a constant.
So, r squared plus 2p times r plus omega nought squared is equal to zero.
Now, what are its roots?
Well, you see right away the first advantage in putting in the two there.
When I use the quadratic formula, it's negative 2p over two.
Remember that two in the denominator.
So, that's simply negative p. And, how about the rest?
Plus or minus the square root of, now do it in your head.
4p squared minus 4 omega nought squared.
So, there's a four in both of those terms.
When I pull it outside becomes two.
And, the two in the denominator is lurking, waiting to annihilate it.
So, that two disappears entirely, and it will we are left with is, simply, p squared minus omega nought squared.
Now, whenever people write quadratic equations, and arbitrarily put a two in there, it's because they were going to want to solve the quadratic equation using the quadratic formula, and they don't want all those twos and fours to be cluttering up the formula.
That's what we are doing here.
Okay, now, the first case is where p is equal to zero.
This is going to explain immediately why I wrote that omega nought squared, as you probably already know from physics.
If p is equal to zero, the mass isn't zero.
Otherwise, nothing good would be happening here.
It must be that the damping is zero.
So, p is equal to zero corresponds to undamped.
There is no dashpot.
The oscillations are undamped.
And, the equation, then, becomes the solutions, then, are, well, the equation becomes the equation of simple harmonic motion, which, I think you already are used to writing in this form.
And, the reason you're writing in this form because you know when you do that, this becomes the circular frequency of the oscillations.
The solutions are pure oscillations, and omega nought is the circular frequency.
So, right away from the equation itself, if you write it in this form, you can read off what the frequency of the solutions is going to be, the circular frequency of the solutions.
Now, the solutions themselves, of course, look like, the general solutions look like y equal, in this particular case, the p part is zero.
This is zero.
It's simply, so, in this case, r is equal to omega nought i times omega naught plus or minus, but as before we don't bother with the minus sign since one of those roots is good enough.
And then, the solutions are simply c1 cosine omega nought t plus c2 sine omega nought t. That's if you write it out in the sign, and if you write it using the trigonometric identity, then the other way of writing it is a times the cosine of omega nought t. But now you will have to put it a phase lag.
So, you have those two forms of writing it.
And, I assume you remember writing the little triangle, which converts one into the other.
Okay, so this justifies calling this omega nought squared rather than k over m. And now, the question is what does the damp case look like?
It requires a somewhat closer analysis, and it requires a certain amount of thinking.
So, let's begin with an epsilon bit of thinking.
So, here's my question.
So, in the damped case, I want to be sure that I'm getting oscillations.
When do I get oscillations if, well, we get oscillations if those roots are really complex, and not masquerading.
Now, when are the roots going to be really complex?
This has to be, the inside has to be negative.
p squared minus omega squared must be negative.
p squared minus omega nought squared must be less than zero so that we are taking a square root of negative number, and we are getting a real complex roots, really complex roots.
In other words, now, this says, remember these numbers are all positive, p and omega nought are positive.
So, the condition is that p should be less than omega nought.
In other words, the damping should be less than the circular frequency, except p is not the damping.
It's half the damping, and it's not really the damping either because it involved the m, too.
You'd better just call it p. Naturally, I could write the condition out in terms of c, m, and k. So, your book does that, but I'm not going to.
It gives it in terms of c, m, and k, which somebody might want to know.
But, you know, we don't have to do everything here.
Okay, so let's assume that this is true.
What is the solution look like?
Well, we already experimented with that last time.
Remember, there was some guiding thing which was an exponential.
And then, down here, we wrote the negative.
So, this was an exponential.
In fact, it was the exponential, e to the negative pt.
And, in between that, the curve tried to do its thing.
So, the solution looks sort of like this.
It oscillated, but it had to use that exponential function as its guidelines, as its amplitude, in other words.
Now, this is a truly terrible picture.
It's so terrible, it's unusable.
Okay, this picture never happened.
Unfortunately, this is not my forte along with a lot of other things.
All right, let's try it better.
Here's our better picture.
Okay, there's the exponential.
At this point, I'm supposed to have a lecture demonstration.
It's supposed to go up on the thing, so you can all see it.
But then, you wouldn't be able to copy it.
So, at least we are on even terms now.
Okay, how does the actual curve look?
Well, I'm just trying to be fair.
That's all.
Okay, after a while, the point is, just so we have something to aim at, let's say, okay, here we are going to go, we're going to get down through there.
Okay then, this is our better curve.
Okay, so I am a solution, a particular solution satisfying this initial condition.
I started here, and that was my initial velocity.
The slope of that thing gave me the initial velocity.
Now, the interesting question is, the first, in some ways, the most interesting question, though there will be others, too, is what is this spacing?
Well, that's a period.
And now, it's half a period.
I clearly ought to think of this as the whole period.
So, let's call that, I'm going to call this pi over, so this spacing here, from there to there, I will call that pi divided by omega one because this, from here to here, should be, I hope, twice that, two pi over omega one.
Now, my question is, so this, for a solution, it's, in fact, is going to cross the axis regularly in that way.
My question is, how does this period, so this is going to be its half period.
I will put period in quotation marks because this isn't really a periodic function because it's decreasing all the time in amplitude.
But, it's trying to be periodic.
At lease it's doing something periodically.
It's crossing the axis periodically.
So, this is the half period.
Two pi over omega one would be its full period.
What I want to know is, how does that half period, or how does-- omega one is called its pseudo-frequency.
This should really be called its pseudo-period.
Everything is pseudo.
Everything is fake here.
Like, the amoeba has its fake foot and stuff like that.
Okay, so this is its pseudo-period, pseudo-frequency, pseudo-circular frequency, but that's hopeless.
I guess it should be circular pseudo-frequency, or I don't know how you say that.
I don't think pseudo is a word all by itself, not even in 18.03, circular.
Okay, here's my question.
If the damping goes up, this is the damping term.
If the damping goes up, what happens to the pseudo-frequency?
The frequency is how often the curve, this is high-frequency, and this is low-frequency, okay?
So, my question is, which way does the frequency go?
If the damping goes up, does the frequency go up or down?
Down.
I mean, I'm just asking you to answer intuitively on the basis of your intuition about how this thing explains, how this thing goes, and now let's get the formula.
What, in fact, is omega one?
What is omega one?
The answer is when I solve the equation, so, r is now, so in other words, if omega one is, sorry, if I have p, if p is no longer zero as it was in the undamped case, what is the root, now?
Okay, well, the root is minus p plus or minus the square root of p squared, -- -- now I'm going to write it this way, minus, to indicate that it's really a negative number, omega squared minus p squared.
Now, I'm going to call this, because you see when I change this to sines and cosines, the square root of this number is what's going to become that new frequency.
I'm going to call that minus p plus or minus the square root of minus omega one squared.
That's going to be the new frequency.
And therefore, the root is going to change so that the corresponding solution is going to look, how?
Well, it's going to be e to the negative pt times, let's write it out first in terms of sines and cosines, times the cosine of, well, the square root of omega one squared is omega one.
But, there's an i out front because of the negative sign in front of that.
So, it's going to be the cosine of omega one t plus c2 times the sine of omega one t. Or, if you prefer to write it out in the other form, it's e to the minus p t times some amplitude, which depends on c1 and c2, times the cosine of omega one t minus the phase lag.
Now, when I do that, you see omega one is this pseudo-frequency.
In other words, this number omega one is the same one that I identified here.
And, why is that?
Well, because, what are two successive times?
Suppose it crosses, suppose the solution crosses the x-axis, sorry, y-- the t-axis.
For the first time, at the point t1, what's the next time it crosses t2?
Let's jump to the two times across it.
So, I want this to be a whole period, not a half period.
What's t2?
Well, I say that t2 is nothing but 2 pi divided by omega one.
And, you can see that because when I plug in, if it's zero, if I have a point where it's zero, so, omega one t minus phi, when will it be zero for the first time?
Well, that will be when the cosine has to be zero.
So, it will be some multiple of, it will be, say, pi over two.
Then, the next time this happens will be, if that happens at t1, then the next time it happens will be at t1 plus 2 pi divided by omega one.
That will also be pi over two plus how much?
Plus 2 pi, which is the next time the cosine gets around and is doing its thing, becoming zero as it goes down, not as it's coming up again.
In other words, this is what you should add to the first time to get this second time that the cosine becomes zero coming in the direction from top to the bottom.
So, this is, in fact, the frequency with which it's crossing the axis.
Now, notice, I'm running out of boards.
What a disaster!
In that expression, take a look at it.
I want to know what depends on what.
So, p, in that, we got constants.
We got p. We got phi.
We got A.
What else we got?
Omega one.
What do these things depend upon?
You've got to keep it firmly in mind.
This depends only on the ODE.
It's basically the damping.
It depends on c and m. Essentially, it's c over 2m actually.
How about phi?
Well, phi, what else depends only on the ODE?
Omega one depends only on the ODE.
What's the formula for omega one?
Omega one squared.
Where do we have it?
Omega one squared, I never wrote the formula for you.
So, we have omega nought squared minus p squared equals omega one squared.
What's the relation between them?
That's the Pythagorean theorem.
If this is omega nought, then this omega one, this is p. They make a little, right triangle in other words.
The omega one depends on the spring.
So, it's equal to, well, it's equal to that thing.
So, it depends on the damping.
It depends upon the damping, and it depends on the spring constant.
How about the phi and the A?
What do they depend on?
They depend upon the initial conditions.
So, the mass of constants, they have different functions.
What's making this complicated is that our answer needs four parameters to describe it.
This tells you how fast it's coming down.
This tells you the phase lag.
This amplitude modifies, it tells you whether the exponential curve starts going, is like that or goes like this.
And, finally, the omega one is this pseudo-frequency, which tells you how it's bobbing up and down.
y prime and y double prime.
So, q of x times y equal zero.
The linearity of the equation, that is, the form in which it appears is going to be the key idea today.
Today is going to be theoretical, but some of the ideas in it are the most important in the course.
So, I don't have to apologize for the theory.
Remember, the solution method was to find two independent y one, y two independent solutions.
And now, I'll formally write out what independent means.
There are different ways to say it.
But for you, I think the simplest and most intelligible will be to say that y2 is not to be a constant multiple of y1.
And, unfortunately, it's necessary to add, nor is y1 to be a constant multiple.
I have to call it by different constants.
So, let's call this one c prime of y2.
Well, I mean, the most obvious question is, well, look.
If this is not a constant times that, this can't be there because I would just use one over c if it was.
Unfortunately, the reason I have to write it this way is to take account of the possibility that y1 might be zero.
If y1 is zero, so, the bad case that must be excluded is that y1 equals zero, y2 nonzero.
I don't want to call those independent.
But nonetheless, it is true that y2 is not a constant multiple of y1.
However, y1 is a constant multiple of y2, namely, the multiple zero.
It's just to exclude that case that I have to say both of those things.
And, one would not be sufficed.
That's a fine point that I'm not going to fuss over.
But I just have, of course.
Now, why do you do that?
That's because, then, all solutions, and this is what concerns us today, are what?
The linear combination with constant coefficients of these two, and the fundamental question we have to answer today is, why?
Now, there are really two statements involved in that.
On the one hand, I'm claiming there is an easier statement, which is that they are all solutions.
So, that's question one, or statement one.
Why are all of these guys solutions?
That, I could trust you to answer yourself.
I could not trust you to answer it elegantly.
And, it's the elegance that's the most important thing today because you have to answer it elegantly.
Otherwise, you can't go on and do more complicated things.
If you answer it in an ad hoc basis just by hacking out a computation, you don't really see what's going on.
And you can't do more difficult things later.
So, we have to answer this, and answer it nicely.
The second question is, so, if that answers why there are solutions at all, why are they all the solutions?
Why all the solutions?
In other words, to say it as badly as possible, why are all solutions, why all the solutions-- Never mind.
Why are all the solutions.
This is a harder question to answer, but that should make you happy because that means it depends upon a theorem which I'm not going to prove.
I'll just quote to you.
Let's attack there for problem one first.
q1 is answered by what's called the superposition.
The superposition principle says exactly that.
It says exactly that, that if y1 and y2 are solutions to a linear equation, to a linear homogeneous ODE, in fact it can be of higher order, too, although I won't stress that.
In other words, you don't have to stop with the second derivative.
You could add a third derivative and a fourth derivative.
As long as the former makes the same, but that implies automatically that c1 y1 plus c2 y2 is a solution.
Now, the way to do that nicely is to take a little detour and talk a little bit about linear operators.
And, since we are going to be using these for the rest of the term, this is the natural place for you to learn a little bit about what they are.
So, I'm going to do it.
Ultimately, I am aimed at a proof of this statement, but there are going to be certain side excursions I have to make.
The first side side excursion is to write the differential equation in a different way.
So, I'm going to just write its transformations.
The first, I'll simply recopy what you know it to be, q y equals zero.
That's the first form.
The second form, I'm going to replace this by the differentiation operator.
So, I'm going to write this as D squared y.
That means differentiate it twice.
D it, and then D it again.
This one I only have to differentiate once, so I'll write that as p D(y), p times the derivative of Y.
The last one isn't differentiated at all.
I just recopy it.
Now, I'm going to formally factor out the y.
So this, I'm going to turn into D squared plus pD plus q.
Now, everybody reads this as times y equals zero.
But, what it means is this guy, it means this is shorthand for that.
I'm not multiplying.
I'm multiplying q times y.
But, I'm not multiplying D times y. I'm applying D to y.
Nonetheless, the notation suggests this is very suggestive of that.
And this, in turn, implies that.
I'm just transforming it.
And now, I'll take the final step.
I'm going to view this thing in parentheses as a guy all by itself, a linear operator.
This is a linear operator, called a linear operator.
And, I'm going to simply abbreviate it by the letter L. And so, the final version of this equation has been reduced to nothing but Ly equals zero.
Now, what's L?
You can think of L as, well, formally, L you would write as D squared plus pD plus q.
But, you can think of L, the way to think of it is as a black box, a function of what goes into the black box, well, if this were a function box, what would go it would be a number, and what would come out with the number.
But it's not that kind of a black box.
It's an operator box, and therefore, what goes in is a function of x.
And, what comes out is another function of x, the result of applying this operator to that.
So, from this point of view, differential equations, trying to solve the differential equation means, what should come out you want to come out zero, and the question is, what should you put in?
That's what it means solving differential equations in an inverse problem.
The easy thing is to put it a function, and see what comes out.
You just calculate.
The hard thing is to ask, you say, I want such and such a thing to come out, for example, zero; what should I put in?
That's a difficult question, and that's what we're spending the term answering.
Now, the key thing about this is that this is a linear operator.
And, what that means is that it behaves in a certain way with respect to functions.
The easiest way to say it is, I like to make two laws of it, that L of u1, if you have two functions, I'm not going to put up the parentheses, x, because that just makes things look longer and not any clearer, actually.
What does L do to the sum of two functions?
If that's a linear operator, if you put in the sum of two functions, what you must get out is the corresponding L's, the sum of the corresponding L's of each.
So, that's a law.
And, the other law, linearity law, and this goes for anything in mathematics and its applications, which is called linear, basically anything is linear if it does the following thing to functions or numbers or whatever.
The other one is of a constant times any function, I don't have to give it a number now because I'm only using one of them, should be equal to c times L of u.
So, here, c is a constant, and here, of course, the u is a function, functions of x.
These are the two laws of linearity.
An operator is linear if it satisfies these two laws.
Now, for example, the differentiation operator is such an operator.
D is linear, why?
Well, because of the very first things you verify after you learn what the derivative is because the derivative of, well, I will write it in the D form.
I'll write it in the form in which you know it.
It would be D apply to u1 plus u2.
How does one write that in ordinary calculus?
Well, like that.
Or, maybe you write d by dx out front.
Let's write it this way, is equal to u1 prime plus u2 prime.
That's a law.
You prove it when you first study what a derivative is.
It's a property.
From our point of view, it's a property of the differentiation operator.
It has this property.
The D of u1 plus u2 is D of u1 plus D of u2.
And, it also has the property that c u prime, you can pull out the constant.
That's not affected by the differentiation.
So, these two familiar laws from the beginning of calculus say, in our language, that D is a linear operator.
What about the multiplication of law?
That's even more important, that u1 times u2 prime, I have nothing whatever to say about that in here.
In this context, it's an important law, but it's not important with respect to the study of linearity.
So, there's an example.
Here's a more complicated one that I'm claiming is the linear operator.
And, since I don't want to have to work in this lecture, the work is left to you.
So, the proof, prove that L is linear, is this particular operator.
L is linear.
That's in your part one homework to verify that.
And you will do some simple exercises in recitation tomorrow to sort of warm you up for that if you haven't done it already.
Well, you shouldn't have because this only goes with this lecture, actually.
It's forbidden to work ahead in this class.
All right, where are we?
Well, all that was a prelude to proving this simple theorem, superposition principle.
So, finally, what's the proof?
The proof of the superposition principle: if you believe that the operator is linear, then L of c1, in other words, the ODE is L. L is D squared plus pD plus q.
So, the ODE is Ly equals zero.
And, what am I being asked to prove?
I'm being asked to prove that if y1 and y2 are solutions, then so is that thing.
By the way, that's called a linear combination.
Put that in your notes.
Maybe I better write it even on the board because it's something people say all the time without realizing they haven't defined it.
This is called a linear combination.
This expression is called a linear combination of y1 and y2.
It means that particular sum with constant coefficients.
Okay, so, the ODE is Ly equals zero.
And, I'm trying to prove that fact about it, that if y1 and y2 are solutions, so is a linear combination of them.
So, the proof, then, I start with apply L to c1 y1 plus c2 y2.
Now, because this operator is linear, it takes the sum of two functions into the corresponding sum up what the operator would be.
So, it would be L of c1 y1 plus L of c2 y2.
That's because L is a linear operator.
But, I don't have to stop there.
Because L is a linear operator, I can pull the c out front.
So, it's c1 L of y1 plus c2 L of y2.
Now, where am I?
Trying to prove that this is zero.
Well, what is L of y1?
At this point, I use the fact that y1 is a solution.
Because it's a solution, this is zero.
That's what it means to solve that differential equation.
It means, when you apply the linear operator, L, to the function, you get zero.
In the same way, y2 is a solution.
So, that's zero.
And, the sum of c1 times zero plus c2 times zero is zero.
That's the argument.
Now, you could make the same argument just by plugging c1 y1, plugging it into the equation and calculating, and calculating, and calculating, grouping the terms and so on and so forth.
But, that's just calculation.
It doesn't show you why it's so.
Why it's so is because the operator, this differential equation is expressed as a linear operator applied to y is zero.
And, the only properties that are really been used as the fact that this operator is linear.
That's the key point.
L is linear.
It's a linear operator.
Well, that's all there is to the superposition principle.
As a prelude to answering the more difficult question, why are these all the solutions?
Why are there no other solutions?
We need a few definitions, and a few more ideas.
And, they are going to occur in connection with, so I'm now addressing, ultimately, question two.
But, it's not going to be addressed directly for quite awhile.
Instead, I'm going to phrase it in terms of solving the initial value problem.
So far, we've only talked about the general solution with those two arbitrary constants.
But, how do you solve the initial value problem, in other words, fit initial conditions, find the solution with given initial values for the function and its derivatives.
Now, -- -- the theorem is that this collection of functions with these arbitrary constants, these are all the solutions we have so far.
In fact, they are all the solutions there are, but we don't know that yet.
However, if we just focus on the big class of solutions, there might be others lurking out there somewhere lurking out there.
I don't know.
But let's use what we have, that just from this family is enough to satisfy any initial condition, to satisfy any initial values.
In other words, if you give me any initial values, I will be able to find the c1 and c2 which work.
Now, why is that?
Well, I'd have to do a song and dance at this point, if you hadn't been softened up by actually calculating for specific differential equations.
You've had exercises and actually how to calculate the values of c1 and c2.
So, I'm going to do it now in general what you have done so far for particular equations using particular values of the initial conditions.
So, I'm relying on that experience that you've had in doing the homework to make intelligible what I'm going to do now in the abstract using just letters.
So, why is this so?
Why is that so?
Well, we are going to need it, by the way, here, too.
I'll have to, again, open up parentheses.
But let's go as far as we can.
Well, you just try to do it.
Suppose the initial conditions are, how will we write them?
So, they're going to be at some initial point, x0.
You can take it to be zero if you want, but I'd like to be, just for a little while, a little more general.
So, let's say the initial conditions, the initial values are being given at the point x0, all right, that's going to be some number.
Let's just call it a.
And, the initial value also has to specify the velocity or the value of the derivative there.
Let's say these are the initial values.
So, the problem is to find the c which work.
Now, how do you do that?
Well, you know from calculation.
You write y equals c1 y1 plus c2 y2.
And you write y prime, and you take the derivative underneath that, which is easy to do.
And now, you plug in x equals zero.
And, what happens?
Well, these now turn into a set of equations.
What will they look like?
Well, y of x0 is a, and this is b.
So, what I get is let me flop it over onto the other side because that's the way you're used to looking at systems of equations.
So, what we get is c1 times y1 of x0 plus c2 times y2 of x0.
What's that supposed to be equal to?
Well, that's supposed to be equal to y of x0.
It's supposed to be equal to the given number, a.
And, in the same way, c1 y1 prime of x0 plus c2 y2 prime of x0, that's supposed to turn out to be the number, b.
In the calculations you've done up to this point, y1 and y2 were always specific functions like e to the x or cosine of 22x, stuff like that.
Now I'm doing it in the abstract, just calling them y1 and y2, so as to include all those possible cases.
Now, what am I supposed to do?
I'm supposed to find c1 and c2.
What kind of things are they?
This is what you studied in high school, right?
The letters are around us, but it's a pair of simultaneous linear equations.
What are the variables?
What are the variables?
What are the variables?
Somebody raise their hand.
If you have a pair of simultaneous linear equations, you've got variables and you've got constants, right?
And you are trying to find the answer.
What are the variables?
Yeah?
c1 and c2.
Very good.
I mean, it's extremely confusing because in the first place, how can they be the variables if they occur on the wrong side?
They're on the wrong side; they are constants.
How can constants be variables?
Everything about this is wrong.
Nonetheless, the c1 and the c2 are the unknowns, if you like the high school terminology.
c1 and c2 are the unknowns.
These messes are just numbers.
After you've plugged in x0, this is some number.
You've got four numbers here.
So, c1 and c2 are the variables.
The two find, in other words, to find the values of.
All right, now you know general theorems from 18.02 about when can you solve such a system of equations.
I'm claiming that you can always find c1 and c2 that work.
But, you know that's not always the case that a pair of simultaneous linear equations can be solved.
There's a condition.
There's a condition which guarantees their solution, which is what?
What has to be true about the coefficients?
These are the coefficients.
What has to be true?
The matrix of coefficients must be invertible.
The determinant of coefficients must be nonzero.
So, they are solvable if, for the c1 and c2, if this thing, I'm going to write it.
Since all of these are evaluated at x0, I'm going to write it in this way.
y1, the determinant, whose entries are y1, y2, y1 prime, and y2 prime, evaluated at zero, x0, that means that I evaluate each of the functions in the determinant at x0.
I'll write it this way.
That should be not zero.
So, in other words, the key thing which makes this possible, makes it possible for us to solve the initial value problem, is that this funny determinant should not be zero at the point at which we are interested.
Now, this determinant is important in 18.03.
It has a name, and this is when you're going to learn it, if you don't know it already.
That determinant is called the Wronskian.
The Wronskian of what?
If you want to be pompous, you say this with a V sound instead of a W. But, nobody does except people trying to be pompous.
The Wronskian, we'll write a W. Now, notice, you can only calculate it when you know what the two functions are.
So, the Wronskian of the two functions, y1 and y2, what's the variable?
It's not a function of two variables, y1 and y2.
These are just the names of functions of x.
So, when you do it, put it in, calculate out that determinant.
This is a function of x, a function of the independent variable after you've done the calculation.
Anyway, let's write its definition, y1, y2, y1 prime, y2 prime.
Now, in order to do this, the point is we must know that that Wronskian is not zero, that the Wronskian of these two functions is not zero at the point x0.
Now, enter a theorem which you're going to prove for homework, but this is harder.
So, it's part two homework.
It's not part one homework.
In other words, I didn't give you the answer.
You've got to find it yourself, alone or in the company of good friends.
So, anyway, here's the Wronskian.
Now, what can we say for sure?
Note, suppose y1 and y, just to get you a feeling for it a little bit, suppose they were not independent.
The word for not independent is dependent.
Suppose they were dependent.
In other words, suppose that y2 were a constant multiple of y1.
We know that's not the case because our functions are supposed to be independent.
But suppose they weren't.
What would the value of the Wronskian be?
If y2 is a constant times y1, then y2 prime is that same constant times y1 prime.
What's the value of the determinant?
Zero.
For what values of x is it zero for all values of x?
And now, that's the theorem that you're going to prove, that if y1 and y2 are solutions to the ODE, I won't keep, say, it's the ODE we've been talking about, y double prime plus py.
But the linear homogeneous with not constant coefficients, just linear homogeneous second order.
Our solutions, as there are only two possibilities, either-or.
Either the Wronskian of y, there are only two possibilities.
Either the Wronskian of y1 and y2 is always zero, identically zero, zero for all values of x.
This is redundant.
When I say identically, I mean for all values of x.
But, I am just making assurance doubly sure.
Or, or the Wronskian is never zero.
Now, there is no notation as for that.
I'd better just write it out, is never zero, i.e.
for no x is it, i.e.
for all x.
There's no way to say that.
I mean, for all values of x, it's not zero.
That means, there is not a single point for which it's zero.
In particular, it's not zero here.
So, this is your homework: problem five, part two.
I'll give you a method of proving it, which was discovered by the famous Norwegian mathematician, Abel, who is, I guess, the centenary of his birth, I guess, was just celebrated last year.
He has one of the truly tragic stories in mathematics, which I think you can read.
It must be a Simmons book, if you have that.
Simmons is very good on biographies.
Look up Abel.
He'll have a biography of Abel, and you can weep if you're feeling sad.
He died at the age of 26 of tuberculosis, having done a number of sensational things, none of which was recognized in his lifetime because people buried his papers under big piles of papers.
So, he died unknown, uncelebrated, and now he's Norway's greatest culture hero.
In the middle of a park in Oslo, there's a huge statue.
And, since nobody knows what Abel looked like, the statue is way up high so you can't see very well.
But, the inscription on the bottom says Niels Henrik Abel, 1801-1826 or something like that.
Now, -- -- the choice, I'm still, believe it or not, aiming at question two, but I have another big parentheses to open.
And, when I closed it, the answer to question two will be simple.
But, I think it's very desirable that you get this second big parentheses.
It will help you to understand something important.
It will help you on your problem set tomorrow night.
I don't have to apologize.
I'm just going to do it.
So, the question is, the thing you have to understand is that when I write this combination, I'm claiming that these are all the solutions.
I haven't proved that yet.
But, they are going to be all the solutions The point is, there's nothing sacrosanct about the y1 and y2.
This is exactly the same collection as a collection which I would write using other constants.
Let's call them u1 and u2.
They are exactly the same, where u1 and u2 are any other pair of linearly independent solutions.
Any other pair of independent solutions, they must be independent, either a constant multiple of each other.
In other words, u1 is some combination, now I'm really stuck because I don't know how to, c1 bar, let's say, that means a special value of c1, and a special value of c2, and u2 is some other special value, oh my God, c1 double bar, how's that?
The notation is getting worse and worse.
I apologize for it.
In other words, I could pick y1 and y2 and make up all of these.
And, I'd get a bunch of solutions.
But, I could also pick some other family, some other two guys in this family, and just as well express the solutions in terms of u1 and u2.
Now, well, why is he telling us that?
Well, the point is that the y1 and the y2 are typically the ones you get easily from solving the equations, like e to the x and e to the 2x.
That's what you've gotten, or cosine x and sine x, something like that.
But, for certain ways, they might not be the best way of writing the solutions.
There is another way of writing those that you should learn, and that's called finding normalized, the normalized.
They are okay, but they are not normalized.
For some things, the normalized solutions are the best.
I'll explain to you what they are, and I'll explain to you what they're good for.
You'll see immediately what they're good for.
Normalized solutions, now, you have to specify the point at which you're normalizing.
In general, it would be x nought, but let's, at this point, since I don't have an infinity of time, to simplify things, let's say zero.
It could be x nought, any point would do just as well.
But, zero is the most common choice.
What are the normalized solutions?
Well, first of all, I have to give them names.
I want to still call them y.
So, I'll call them capital Y1 and Y2.
And, what they are, are the solutions which satisfy certain, special, very special, initial conditions.
And, what are those?
So, they're the ones which satisfy, the initial conditions for Y1 are, of course there are going to be guys that look like this.
The only thing that's going to make them distinctive is the initial conditions they satisfy.
Y1 has to satisfy at zero.
Its value should be one, and the value of its derivative should be zero.
For Y2, it's just the opposite.
Here, the value of the function should be zero at zero.
But, the value of its derivative, now, I want to be one.
Let me give you a trivial example of this, and then one, which is a little less trivial, so you'll have some feeling for what I'm asking for.
Suppose the equation, for example, is y double prime plus, well, let's really make it simple.
Okay, you know the standard solutions are y1 is cosine x, and y2 is sine x.
These are functions, which, when you take the second derivative, they turn into their negative.
You know, you could go the complex roots are i and minus i, and blah, blah, blah, blah, blah, blah.
If you do it that way, fine.
But at some point in the course you have to be able to write down and, right away, oh, yeah, cosine x, sine x.
Okay, what are the normalized things?
Well, what's the value of this at zero?
It is one.
What's the value of its derivative at zero?
Zero.
This is Y1.
This is the only case in which you locked on immediately to the normalized solutions.
In the same way, this guy is Y2 because its value at zero is zero.
It's value of its derivative at zero is one.
So, this is Y2.
Okay, now let's look at a case where you don't immediately lock on to the normalized solutions.
Very simple: all I have to do is change the sign.
Here, you know, think through r squared minus one equals zero.
The characteristic roots are plus one and minus one, right?
And therefore, the solution is e to the x, and e to the negative x.
So, the solutions you find by the usual way of solving it is y1 equals e to the x, and y2 equals e to the negative x.
Those are the standard solution.
So, the general solution is of the form.
So, the general solution is of the form c1 e to the x plus c2 e to the negative x.
Now, what I want to find out is what is Y1 and Y2?
How do I find out what Y1 is?
Well, I have to satisfy initial conditions.
So, if this is y, let's write down here, if you can still see that, y prime is c1 e to the x minus c2 e to the negative x .
So, if I plug in, I want y of zero to be one, I want this guy at the point zero to be one.
What equation does that give me?
That gives me c1 plus c2, c1 plus c2, plugging in x equals zero, equals the value of this thing at zero.
So, that's supposed to be one.
How about the other guy?
The value of its derivative is supposed to come out to be zero.
And, what is its derivative?
Well, plug into this expression.
It's c1 minus c2.
Okay, what's the solution to those pair of equations?
c2 has to be equal to c1.
The sum of the two of them has to be one.
Each one, therefore, is equal to one half.
And so, what's the value of Y1?
Y1, therefore, is the function where c1 and c2 are one half.
It's the function e to the x plus e to the negative x divided by two.
In the same way, I won't repeat the calculation.
You can do yourself.
Same calculation shows that Y2, so, put in the initial conditions.
The answer will be that Y2 is equal to e to the x minus e to the minus x divided by two.
These are the special functions.
For this equation, these are the normalized solutions.
They are better than the original solutions because their initial values are nicer.
Just check it.
The initial value, when x is equal to zero, the initial value, this has is zero.
Here, when x is equal to zero, the value of the function is zero.
But, the value of its derivative, these cancel, is one.
So, these are the good guys.
Okay, there's no colored chalk this period.
Okay, there was colored chalk.
There's one.
So, for this equation, these are the good guys.
These are our best solutions.
e to the x and e to the minus x are good solutions.
But, these are our better solutions.
And, this one, of course, is the function which is called hyperbolic sine of x, and this is the one which is called hyperbolic cosine of x.
This is one of the most important ways in which they enter into mathematics.
And, this is why the engineers want them.
Now, why do the engineers want normalized solutions?
Well, I didn't explain that.
So, what's so good about normalized solutions?
Very simple: if Y1 and Y2 are normalized at zero, let's say, then the solution to the IVP, in other words, the ODE plus the initial values, y of zero equals, let's say, a and y prime of zero equals b.
So, the ODE I'm not repeating.
It's the one we've been talking about all term since the beginning of the period.
It's the one with the p of x and q of x.
And, here are the initial values.
I'm going to call them a and b.
You can also call them, if you like, maybe that's better to call them y0, as they are individual in the homework.
They are called, I'm using the, let's use those.
What is the solution?
I say the solution is, if you use y1 and y2, the solution is y0, in other words, the a times Y1, plus y0 prime, in other words, b times Y2.
In other words, you can write down instantly the solution to the initial value problem, if instead of using the functions, you started out with the little Y1 and Y2, you use these better functions.
The thing that's better about them is that they instantly solve for you the initial value problem.
All you do is use this number, initial condition as the coefficient of Y1, and use this number as the coefficient of Y2.
Now, just check that by looking at it.
Why is that so?
Well, for example, let's check.
What is its value of this function at zero?
Well, the value of this guy at zero is one.
So, the answer is y0 times one, and the value of this guy at zero is zero.
So, this term disappears.
And, it's exactly the same with the derivative.
What's the value of the derivative at zero?
The value of the derivative of this thing is zero.
So, this term disappears.
The value of this derivative at zero is one.
And so, the answer is y0 prime.
So, check, check, this works.
So, these better solutions have the property, what's good about them, and why scientists and engineers like them, is that they enable you immediately to write down the answer to the initial value problem without having to go through this business, which I buried down here, of solving simultaneous linear equations.
Okay, now, believe it or not, that's all the work.
We are ready to answer question number two: why are these all the solutions?
Of course, I have to invoke a big theorem.
A big theorem: where shall I invoke a big theorem?
Let's see if we can do it here.
The big theorem says, it's called the existence and uniqueness theorem.
It's the last thing that's proved at the end of an analysis course, at which real analysis courses, over which students sweat for one whole semester, and their reward at the end is, if they are very lucky, and if they have been very good students, they get to see the proof of the existence and uniqueness theorem for differential equations.
But, I can at least say what it is for the linear equation because it's so simple.
It says, so, the equation we are talking about is the usual one, homogeneous equation, and I'm going to assume, you have to have assumptions that p and q are continuous for all x.
So, they're good-looking functions.
Coefficients aren't allowed to blow up anywhere.
They've got to look nice.
Then, the theorem says there is one and only one solution, one and only solution satisfying, given initial values such that y of zero, let's say y of zero is equal to some given number, A, and y, let's make y0, and y prime of zero equals B.
The initial value problem has one and only one solution.
The existence is, it has a solution.
The uniqueness is, it only has one solution.
If you specify the initial conditions, there's only one function which satisfies them and at the same time satisfies that differential equation.
Now, this answers our question.
This answers our question, because, look, what I want is all solutions.
What we want are all solutions to the ODE.
And now, here's what I say: a claim that this collection of functions, c1 Y1 plus c2 Y2 are all solutions.
Of course, I began a period by saying I'd show you that c1 little y1 c little y2 are all the solutions.
But, it's the case that these two families are the same.
So, the family that I started with would be exactly the same as the family c1 prime Y1 because, after all, these are two special guys from that collection.
So, it doesn't matter whether I talk about the original ones, or these.
The theorem is still the same.
The final step, therefore, if you give me one more minute, I think that will be quite enough.
Why are these all the solutions?
Well, I have to take an arbitrary solution and show you that it's one of these.
So, the proof is, given a solution, u(x), what are its values?
Well, u of x zero is u zero, and u prime of x zero, zero, let's say, is equal to u zero, is equal to some other number.
Now, what's the solution?
Write down what's the solution of these using the Y1's?
Then, I know I've just shown you that u zero times Y1 plus u zero prime Y2 satisfies the same initial conditions, satisfies these initial conditions, initial values.
In other words, I started with my little solution.
u of x walks up to and says, hi there.
Hi there, and the differential equation looks at it and says, who are you?
You say, oh, I satisfy you and my initial, and then it says what are your initial values?
It says, my initial values are u0 and u0 prime.
And, it said, sorry, but we've got one of ours who satisfies the same initial conditions.
We don't need you because the existence and uniqueness theorem says that there can only be one function which does that.
And therefore, you must be equal to this guy by the uniqueness theorem.
Okay, we'll talk more about stuff next time, linear equation next time.
We are going to start today in a serious way on the inhomogenous equation, second-order linear differential, I'll simply write it out instead of writing out all the words which go with it.
So, such an equation looks like, the second-order equation is going to look like y double prime plus p of x, t, x plus q of x times y.
Now, up to now the right-hand side has been zero.
So, now we are going to make it not be zero.
So, this is going to be f of x.
In the most frequent applications, x is time.
x is usually time, often, but not always.
So, maybe just for today, I will use X in talking about the general theory.
And, from now on, I'll probably make X equal time because that's what is most of the time in the applications.
So, this is the part we've been studying up until now.
It has a lot of names.
It's input, signal, commas between those, a driving term, or sometimes it's called the forcing term.
You'll see all of these in the literature, and it pretty much depends upon what course you're sitting at, what the professor habitually calls it.
I will try to use all these terms now and then, probably most often I will lapse into input as the most generic term, suggesting nothing in particular, and therefore, equally acceptable or unacceptable to everybody.
The response, the solution, then, the solution as you know is then called the response.
The response, sometimes it's called the output.
I think I'll stick pretty much with response.
So, I'm using pretty much the same terminology we use for studying first-order equations.
Now, as you will see, the reason we had to study the homogeneous case first was because you cannot solve this without knowing the homogeneous solutions.
So, that's the inhomogeneous case.
But the homogeneous one, the corresponding homogeneous thing, y double prime plus p of x y prime plus q of x times y equals zero is an essential part of the solution to this equation.
That's called, therefore, it has names.
Now, unfortunately, it doesn't have a single name.
I don't know what to call it, but I think I'll probably call it the associated homogeneous equation, or ODE, the associated homogeneous equation, the one associated to the guy on the left.
It's also called the reduced equation by some people.
There is some other term for it, which escapes me totally, but what the heck.
Now, its solution has a name.
So, its solution, of course, doesn't depend on anything in particular, the general solution, because the right-hand side is always zero.
So, its solution, we know can be written as y equals in the form c1 y1 plus c2 y2, where y1 and y2 are any two independent solutions of that, and then c1's and c2's are arbitrary constants.
Now, what you are looking at this equation, you're going to need this also.
And therefore, it has a name.
It has various names.
Sometimes there is a subscript, c, there.
Sometimes there's a subscript, h. Sometimes there's no subscript at all, which is the most confusing of all.
But, anyway, what's the name given to it?
Well, there is no name.
Many books call it the solution to the associated homogeneous equation.
That's maximally long.
Your book calls it the complementary solution.
Many people call it that, and many will look at you with a blank, who know differential equations very well, and will not have the faintest idea what you're talking about.
If you call it (y)h, then you are thinking of it as the solution; the h is for homogeneous to indicate it's the solution.
So, it's the solution to the, I'm not going to write that.
You put it in her books if you like writing.
Write solution to the associated homogeneous equation, y(h).
But, it's all the same thing.
Now, or the solution to the reduced equation, I see I have in my notes.
Okay, good, the solution to the reduced equation, too.
Okay, now, the examples, there are, of course, two classical examples, of which you know one.
But, use them as the model for what solutions of these things should look like and how they should behave.
So, the model you know already is the one, I won't make the leading coefficient one because it usually isn't, is the one, m x double prime, so t is the independent variable, plus b x prime plus k x equals f of t. That's the spring-mass system, the spring-mass-dashpot system.
Mass, the damping constant and the spring constant, except up to now, it's always been zero here.
What does this f of t represent?
Well, if you think of the way in which I derived the equation, the mx, that was the Newton's law.
That's the acceleration.
So, it's the acceleration, the mass times the acceleration.
By Newton's law, this is equal to the imposed force on the little mass truck.
Okay, you got that truck, there.
I'm not going to draw the truck for the nth time.
You'll have to imagine it.
So, here's our truck.
Okay, forces are acting on it.
Remember, the forces were minus kx.
That came from the spring.
There was a force, minus b x prime.
That came from the dashpot, the damping force.
So, this other guy is f of t. What's this?
This is the external force, which is acting out.
In other words, instead of the little truck going back and forth and doing its own thing all by itself, here's someone with an electromagnet, and the mass it's carrying is a big pile of iron ore. You're turning it on and off, and pulling that thing from afar where nobody can see it.
So, this is the external force.
Now, think, that is the model you must have in your mind of how these equations are treated.
In other words, when f of t is zero, the system is passive.
There is no external force on it when this is zero.
The system is sitting, and just doing what it wants to do, all by itself.
You wanted up by giving it an initial push, and putting its initial position somewhere.
But after that, you lay your hands off.
The system then just passively responds to its initial conditions and does what it wants.
The other model is that you don't let it respond the way it wants to.
You force it from the outside by pushing it with an external force.
Now, those are clearly two entirely different problems: what it does by itself, or what it does when it's acted on from outside.
And, when I explained to you how the thing is to be solved, you have to keep in mind those two models.
So, this is the forced system.
I'll just use the word, forced system, that's where f of t is not zero, versus the passive system where there is no external applied force.
The passive system, the forced system, now, you have to both, even if you wanted to solve the forced system, the way the system would behave if nothing would be done to it from the outside is nonetheless going to be an important part of the solution.
And, I won't be able to give you that solution without knowing this also.
Now, I'd like to give you the other model very rapidly because it's in your book.
It's in the problems I have to give you.
You know, it's part of everybody's culture, whether they like it or not.
So, that's example number one.
Example number two, which follows the differential equation just as perfectly as the spring-mass-dashpot system is the simple electric circuit.
The inductance, you don't know yet what an inductance is, officially, but you will, a resistance, sorry, that's okay, put the capacitance up there, resistance, and then maybe a thing.
So, this is a resistance.
I think you know these symbols.
By now, you certainly know the system for capacitance.
What I mean when I say C is the capacitance, you may not know yet what L is.
That's called the inductance.
So, this is something called a coil because it looks like one.
L is what's called its inductance.
And, the differential equation, there are two differential equations which can be used in this.
They are essentially the same.
One is simply the derivative of the other.
Both differential equations come from Kirchhoff's voltage law, that the sum of the voltage drops as you move around the circuit -- -- has to be zero because otherwise, I don't have to, that's because of somebody's law, Kirchhoff, with two h's.
The sum of the voltage drops to zero, and now you know the voltage drop across this, and you know the voltage drop across that because you learned in 8.02.
You will, one day, learn the voltage drop across this.
But, I already know it.
It's Li.
So, i is the current.
I'll write this thing in its primitive form first.
So, i is the current that's flowing in the circuit.
q is the charge on the capacitance.
So, the voltage drop across the coil is L times i.
The voltage shop across the, Li prime, the voltage drop across the resistance is, well, you know that.
And, the voltage shop across the capacitance is q divided by C. And so, that's equal to, well, it's equal to zero, except if there's a battery here or something generating a voltage drop, so, let's call that E is a generic word.
E could be a battery.
It could be a source of alternating current, something like that.
But, there's a voltage drop across it, and I'm giving E the name of the voltage drop.
So, and then there's the question of the signs, which I know I'll never understand.
But, let's assume you've chosen the sign convention so that this comes out nicely on the right-hand side.
So, this might be varying sinusoidally, in which case you'd have source of alternating current.
Or, it might be constant, in which that would be a battery, a little dry cell giving you direct current of a constant voltage, stuff like that.
So, you could make this minus if you want, but everything will have the wrong signs, so don't do it.
Now, this doesn't look like what it's supposed to look like because it's got q and i.
So, the final thing you have to know is that q prime is equal to i.
The rate at which that charge leaves the condenser and hurries around the circuit to find its little soul mate on the other side is the current that's flowing in the circuit.
That's why current flows, except nothing really happens.
Electrons just push on each other, and they stay where they are.
I don't understand this at all.
So, if I differentiate this, you can do two things.
Either you could integrate i, and expressed the thing entirely in terms of q, or you can differentiate it, and express everything in terms of i.
Your book does nicely both, does not take sides.
So, let's differentiate it, and then it will look like L i double prime plus R i prime plus i divided by C equals, and now, watch out, you have now not the electromotive force, but its derivative.
So, if you were so unfortunate as to put a little dry cell there, now you've got nothing, and you've got the homogeneous case.
That's okay.
Where are the erasers?
One eraser?
I don't believe this.
So, there's the equation.
There are our two equations.
Why don't we put them up in colored chalk.
There's the spring equation.
And, here's the equation that governs the current, for how the current flows in that circuit.
And now, you can see, again, what does it mean?
If this is zero, for example, if I have a dry cell there, or if I have nothing at all in the circuit, then this represents the passive circuit.
It's just sitting there.
It wouldn't do anything at all, except that you've put a charge on the capacitor, and waited, and of course, when you put a charge on there, it's got a discharge, and discharges through the circuit, and swings back and forth a little bit if it's under-damped until finally towards the end the current dies away to zero.
But, what usually happens is that you drive this passive circuit by putting an effective E in it, and then you want to know how the current behaves.
So, those are the two problems, the passive circuit without an applied electromotive force, or plugging it into the wall, and wanting it to do things.
That's the normal state of affairs.
People don't want passive circuits, they want circuits which do things because, okay, that's why they want to solve inhomogeneous equations instead of homogeneous equations.
But as I said, you have to do the homogeneous case first.
Okay, you are now officially responsible for this, and I don't care that you haven't had it in physics yet.
You will before the next exam.
So, I don't even feel guilty.
But, you're going to start using it on the problem set right away.
So, it's never too soon to start learning it.
Okay, now, the main theorem, I now want to go, so that was just examples to give you some physical feeling for the sorts of differential equations we'll be talking about.
I now want to tell you briefly about the key theorem about solving the homogeneous equation.
So, the main theorem about solving the homogeneous equation is, the inhomogeneous equation.
So, I'm going to write the inhomogeneous equation out.
I'm going to make the left-hand side a linear operator, and am going to write the equation as Ly equals f of x.
That's the inhomogeneous equation.
So, L is the linear operator, second order because I'm only talking about second-order equations.
L is a linear operator, and then this is the differential equation.
So, here's our differential equation.
It's inhomogeneous because it's go the f of x on the right hand side.
And, what the theorem says is that the solution has the following form, y sub p, I'll explain what that is in just a moment, plus y sub c. So, the hypothesis is we've got the linear equation, and the conclusion is that that's what its solution looks like.
Now, you already know what y sub c looks like.
In other words, if I write this out in more detail, it would be i.e., department of fuller explanation, -- -- the general solution looks like y equals yp, and then this thing is going to look like an arbitrary constant times y1 plus an arbitrary constant times y2, where these are solutions of the homogeneous equation.
So, Yc looks like this part, and the yp, what's yp?
p stands for particular, the most confusing word in this subject.
But, you've got at least four weeks to learn what it means.
Okay, yp is a particular solution to Ly equals f of x.
Now, I'm not going to explain what particular means.
First, I'll chat as if you knew what it meant, and then we'll see if you have picked it up.
In other words, the procedure for solving this equation is composed of two steps.
First, to find this part.
In other words, to find the complementary solution, in other words, to do what we've been doing for the last week, solve not the equation you are given, but the reduced equation.
So, the first step is to find this.
The second step is to find yp.
Now, what's yp?
yp is a particular solution to the whole equation.
Yeah, but which one?
Well, if it's any one, then it's not a particular solution, yeah.
I say, unfortunately the word particular here is not being used in exactly the same sense in which most people use it in ordinary English.
It's a perfectly valid way to use it.
It's just confusing, and no one has ever come up with a better word.
So, particular means any one solution.
Any one will do.
Okay, even these have slightly different meanings.
Any questions about this?
I refuse to answer them.
[LAUGHTER] Now, well, examples of course will make it all clear.
But I'd like, first, to prove the theorem, to show you how simple it is.
It's extremely simple if you just use the fact that L is a linear operator.
We've got two things to prove.
What have we got to prove?
Well, I have to prove two statements, first of all, that all the yp plus c1 y1 plus c2 y2 are solutions.
How are we going to prove that?
Well, how do you know if something is a solution?
Well, you plug it into the equation, and you see if it satisfies the equation.
Good, let's do it, proof.
L, I'm going to plug it into the equation.
That means I calculate L of yp plus c1 y1 plus c2 y2.
Now, what's the answer?
Because this is a linear operator, and notice, the argument doesn't use the fact that the equation is second order.
It immediately generalizes to a linear equation of any order, whatever-- 47.
Okay, this is L of yp plus L of c1 y1 plus c2 y2.
Well, what's that?
What's L of the complementary solution?
What does it mean to be the complementary solution?
It means when you apply the operator L to it, you get zero because this satisfies the homogeneous equation.
So, this is zero.
What's L of yp?
Well, it was a particular solution to the equation.
Therefore, when I plugged it into the equation, I must have gotten out on the right-hand side, f of x.
So, this is since yp is a solution to the whole equation.
So, what's the conclusion?
That, if I take any one of these guys, no matter what c1 and c2 are, apply the linear operator, L to it, the answer comes out to be f of.
Therefore, this proves that this shows that these are all solutions because that's what it means.
Therefore, they satisfy L of y equals f of x.
They satisfy the whole inhomogeneous differential equation, and that's it.
Well, that's only half the story.
The other half of the story is to show that there are no other solutions.
Okay, so we got our little u of x coming up again, and he thinks he's a solution.
Okay, so, to prove there are no other solutions, it almost sounds biblical, thou shalt have no other solutions before me, okay.
There are no other solutions accept these guys for different values of c1 and c2.
Okay, so, u of x is a solution.
I have to show that u of x is one of these guys.
How am I going to do that?
Easy.
If it's a solution that, L of u, okay, I'm going to drop the x, okay, just to make the, like I dropped the x over there.
If it's a solution to the whole inhomogeneous equation, then this must come out to be f of x.
Now, what's L of yp?
That's f of x too, by secret little particular solution I've got in my pocket.
Okay, I pull it out, ah-ha, L of yp, that's f of x, too.
Now, I'm going to not add them.
I'm going to subtract them.
What is L of u minus yp?
Well, it's zero.
It's zero because this is a linear operator.
This would be L of u minus L of yp.
I guess the answer is zero on the right-hand side.
And therefore, what is the conclusion?
If that's zero, it must be a solution to the homogeneous equation.
Therefore, u minus yp is equal to, there must be c1 and c2.
I won't give them the generic names.
I'll give them a name, a particular one.
I'll put a tilde to indicate it's a particular one.
c1 plus c2 y2 tilde, so, in other words, for some choice of these constants, and I'll call those particular choices c1 tilde and c2 tilde, it must be that these are equal.
Well, what does that say?
It says that u is equal to yp plus c1 tilde, blah, blah, blah, blah, plus c2 tilde, blah, blah, blah, blah, and therefore chose that u wasn't a new solution.
It was one of these.
So, u isn't new.
So, I should write it down.
Otherwise some of you will have missed the punch line.
Okay, therefore, u is equal to yp plus c1 tilde y1 plus c2 tilde y2.
And, it shows.
This guy who thought he was new was not new at all.
It was just one of the other solutions.
Okay, well, now, since the coefficient's a constant, apparently we've done half the work.
We know what the complementary solution is because you know how to do those in terms of exponentials and complex exponentials, signs and cosines, and so on.
So, what's left to do?
All we have to do is find to solve equations, which are inhomogeneous.
All we have to do is find a particular solution, find one solution.
It doesn't matter which one, any one.
Just find one, okay?
Now, we're going to spend the next two weeks trying to do this.
I'll give you various methods.
I'll give you a general method involving Fourier series because it's a good excuse for learning what Fourier series are.
But, the answer is that in general, for a few standard functions, it's known how to do this.
You will learn those methods for finding those using operators.
For all the others, it's done by a series, or a method involving approximation.
Or, the worse comes to worst, you throw it on a computer and just take a graph and the numerical output of answers as the particular solution.
Okay, now before, we are going to start that work, not today.
We'll start it next Monday, and it will last, as I say the next two weeks.
And, we will be up to spring break.
But, before we do that, I'd like to relate this to what we did for first order equations because there is something to be learned from that.
Think back to the linear first-order equation, and I'm going to, since from now on for the rest of the period, I'm going to be considering the case for constant coefficients.
In other words, this case of springs or circuits or simple systems which behave like those and have constant coefficients.
So, for the linear, first-order equation, there, too, I'm going to think of constant coefficients.
We talked quite a bit about this equation.
What did I call the right-hand side?
I think we usually called it q of t, right?
This is in ancient history.
The definition of ancient history was before the first exam.
Okay, now how does that fit into this theorem that I've given you?
Remember what the solution looked like.
The solution looked like, remember, you took the integrating factor was e to the kt, and then after you integrated both sides, multiplied through, and then the final answer looked like this, y equaled, it was e to the negative kt times either an indefinite integral, or a definite integral depending on your preference, q of t, so, x is metamorphosed into t. I gather you've got that, e to the kt plus, what was the other term?
A constant times e to the negative kt How does this fit into the paradigm I've given you over there for solving the second order equation?
Which term is which?
Well, this has the arbitrary constant in it.
So, this must be the complementary solution.
Is it?
Is this the solution to the associated homogeneous equation?
What's the associated homogeneous equation?
Put zero here.
Okay, if you put zero there, what's the solution?
Now, this you ought to know.
y prime equals negative ky. What's the solution?
e to the negative kt.
You are supposed to come into this course knowing that, except there's an arbitrary constant in front.
So, right, this is exactly the solution to the associated homogeneous equation, where there is zero here.
Then, what's this thing?
This is a particular solution.
This is my yp.
But that's not a particular solution because this indefinite integral, you know, has an arbitrary constant in it.
In fact, it's just that arbitrary constant.
So, it's totally confusing.
But, this symbol, you know when you actually solve the equation this way, all you did was you found one function here.
You didn't throw in the arbitrary constant right away.
All you needed to do was find one function.
And, even if you really are bothered by the fact that this is so indefinite, and therefore, make it a particular solution by making this zero, make it a definite integral, zero, here, t there, and then change those t's to dummy t's, t1's or t tildes, or something like that.
So, this fits into that thing.
In other words, I could have done it at that time, but I didn't the point because this can be solved directly, whereas, of course, the general second order equation in homogeneous cannot be solved directly, and therefore you have to be willing to talk about what its solutions look like in advance.
Now, remember I said, we talked, I said there was two different cases, although both of them had the identical looking solution.
Their meaning in the physical world was so different that they really should be considered as solving the same equation.
And, one of these was the case.
Of the two, perhaps the more important was the case when k was positive, and of course the other is when k is negative.
When k is positive, that had the effect of separating that solution into this part, which was a transient, and the other part, which was a steady state.
The steady state solution, that was the yp part of it in that terminology.
And, the transient part, it was trangent because it went to zero.
If k is positive, the exponential dies regardless of what c is.
So, the transient, that's the yc part.
It goes to zero as Ttgoes to infinity.
The transient depends on, uses, the initial condition, whatever it is, because that's what determines the value of c. On the other hand, this initial condition makes no difference as t goes towards infinity.
All that's left is this steady state solution.
And, all solutions tend to the steady state solution.
So, if k is positive, one gets this analysis of the solutions into the sum of one basic solution, and the others, which just die away, have no influence on this, less and less influence as time goes to infinity.
For k less than zero, this analysis does not work because this term, if k is less than zero, this term goes to infinity or negative infinity, and typically tends to dominate that.
So, it's the start that the important one.
It depends on the initial conditions, and the analysis is meaningless.
So, the above is meaningless.
And now, what I'd like to do is try to see what the analog of that is for second order equations, and higher order equations.
If you understand second-order, that's good enough.
Higher order goes exactly the same way.
So, the question is, for second-order, let's make it with constant coefficients plus, I could call it b and k, oh, no, b k, or p. The trouble is, that wouldn't take care of the electrical circuits.
So, I just want to use neutral letters, which suggest nothing.
And, you can make them turn it into a circuit, so springs, or yet other examples undreamt of.
But these are constants.
And I'm going to think of it as time.
I think I'll switch back to time, let x be the time.
So, B y equals f of t. So, there is our equation.
A and B are constants.
And, the question is, the question I'm asking, can think of either of these two models or others, the question I'm asking is, under what circumstances can I make that same type of analysis into steady-state and transient?
Well, what does the solution look like?
The solution looks like y equals a particular solution plus c1 y1 plus c2 y2.
Therefore, to make that look like this, the c1 and c2 contain the initial conditions.
This part does not.
Therefore, if I want to say that the solutions look like a steady state solution plus something that dies away, which becomes less and less important as time goes on, what I'm really asking is, under what circumstances is this part guaranteed to go to zero?
So, the question is, when, in other words, under what conditions on the equation A and B, in effect, is what we are asking.
When does c1 y1 plus c2 y2 go to zero as t goes to infinity, regardless of what c1 and c2 are for all c1 c2.
Now, here there was no difficulty.
We had the thing very explicitly, and you could see k is positive: this goes to zero.
And if k is negative, it doesn't go to zero.
It goes to infinity.
Here, I want to make the same kind of analysis, except it's just going to take, it's a little more trouble.
But the answer, when it finally comes out is very beautiful.
So, when are all these guys going to go to zero?
First of all, you might as well just have the definition.
So, all the good things that this is going to imply, if this is so, in other words, if they all go to zero, everything in the complementary solution, then the ODE is called stable.
Some people call it asymptotically stable.
I don't know what to call it.
I can make the analysis, and then I use the identical terminology, c1 y1 plus c2 y2.
This is called the transient because it goes to zero.
This is called the particular solution now that we labored so hard to get for the next two weeks.
It's the important part.
It's the steady-state part.
It's what lasts out to infinity after the other stuff has disappeared.
So, this is the steady-state solution, steady-state solution, okay?
And, the differential equation is called stable.
Now, it's of the highest interest to know when a differential equation is stable, linear differential equation is stable in this sense because you have a control.
You know what its solutions look like.
You have some feeling for how it's behaving in the long term.
If this is not so, each equation is a law unto itself if you don't know.
So, let's do the work.
For the rest of the period, what I'd like to do is to find out what the conditions are, which make this true.
Those were the equations which we will have a right to call stable.
So, when does this happen, and where is it going to happen?
I don't know.
I guess, here.
Now, I think the first step is fairly easy, and it will give you a good review of what we've been doing up until now.
So, I'm simply going to make a case-by-case analysis.
Don't worry, it won't take very long.
What are the cases we've been studying?
Well, what do the characteristic roots look like?
The roots of the characteristic equation, in other words, remember, there are cases.
The first case is they are real and distinct, r1 not equal to r2, real and distinct.
What are the other cases?
Well, r1 equals r2.
And then, there's the case where there are complex.
So, I will write it r equals a plus or minus b i.
What do the solutions look like?
So, my ham-handed approach to this problem is going to be, in each case, I'll look at the solutions, and first get the condition on the roots.
So, in other words, I'm not going to worry right away about the a and the b. I'm going, instead, to worry about expressing this condition of stability in terms of the characteristic roots.
In fact, that's the only way in which many people know the conditions.
You're going to be smarter.
Okay, what do the solutions look like?
Well, the general solution looks like e to the r1 t plus c2 e to the r2 t. Okay, so, what's the stability condition?
In other words, if equation happened to have its characteristic roots, real and distinct, under what circumstances would it be stable?
Would it, in other words, all its solutions go to zero?
So, I'm talking about the homogeneous equation, the reduced equation, the associated homogeneous equation.
Why?
Because that's all that's involved in this.
In other words, when I write that, I am no longer interested in the whole equation.
All I'm interested in is the reduced equation, the equation where you turn the f of t on the right-hand side into zero.
So, what's the stability condition?
Well, let's write it out.
Under what circumstances will all these guys go to zero?
If r1 and r2 should be negative, can they be zero?
No, because then it will be a constant and it will go to zero.
How about this one?
Well, in this one, it's (c1 plus c2 times t) multiplied by e to the r1 t. Of course, both of these are the same.
I'll just arbitrarily pick one of them.
What happens to this as things go to zero?
Well, this part is rising, at least if c2 is positive.
This part is either helping or it's hindering.
But, I hope you know what these functions look like, and you know which of them go to zero.
They go to zero if r1 is negative.
It might rise in the beginning, but after a while they lose the energy.
Of course, if r1 is equal to zero, what do these guys do?
Linear, go to infinity.
Well, we are doing okay.
How about here?
Well, here, it's a little more complicated.
The solutions look like e to the at times (c1 cosine bt plus c2 sine bt).
Now, this part is a pure oscillation.
You know that.
It might have a big amplitude, but whatever it does, it does the same thing all the time.
So, whether this goes to zero depends entirely upon what that exponential is doing.
And, that exponential goes to zero if a is negative.
So here, the condition is negative.
And now, the only thing left to do is to say it nicely.
I've got three cases, and I want to say them all in one breath.
So, the stability condition is, the ODE is stable.
So, this is, or f of t. It doesn't matter.
But, psychologically, you can put this as zero there, is stable if what?
In case one, this is true.
In case two, that's true.
In case three, that's true.
But that's ugly.
Make it beautiful.
The beautiful way of saying it is if all the characteristic roots have negative real parts.
If the characteristic roots, the r's or the a plus or minus b i, have negative real part.
That's the form in which the electrical engineers will nod their head, tell you, yeah, that's right, negative real part, sorry.
Isn't it right?
Is that right here?
Yeah.
What's the real part of these guys?
They themselves, because they are real.
What's the real part of this?
Yeah.
The only case in which I really had to use real part is when I talk about the complex case because a is just the real part of a complex number.
It's not the whole thing.
The task for today is to find particular solutions.
So, let me remind you where we've gotten to.
We're talking about the second-order equation with constant coefficients, which you can think of as modeling springs, or simple electrical circuits.
But, what's different now is that the right-hand side is an input which is not zero.
So, we are considering, I'm going to use x as your book does, keeping to a neutral letter.
But, again, in the applications, and in many of the applications at any rate, it wants to be t. But, I make it x.
So, the independent variable is x, and the problem is, remember, that to find a particular solution, and the reason why we want to do that is then the general solution will be of the form y equals that particular solution plus the complementary solution, the general solution to the reduced equation, which we can write this way.
So, all the work depends upon finding out what that yp is, and that's what we're going to talk about today, or rather, talk about for two weeks.
But, the point is, not all functions that you could write on the right-hand side are equally interesting.
There's one kind, which is far more interesting, but more important in the applications than all the others.
And, that's the one out of which, in fact, as you will see later on this week and into next week, an arbitrary function can be built out of these simple functions.
So, the important function is on the right-hand side to be able to solve it when it's a simple exponential.
But, if you allow me to make it a complex exponential, so, here are the important right-hand sides where we want, we want to be able to do it when it's of the form, e to the ax.
In general, that will be, in most applications, a is not a growing exponential, but a decaying exponential.
So, typically, a is negative.
But, it doesn't have to be.
I'll put it in parentheses, though, often.
That's not any assumption that I'm going to make today.
It's just culture.
But, we want to be able to do it for sine omega x and cosine omega x.
In other words, when the right-hand side is a pure oscillation, that's another important type of input both for electrical circuits, think alternating current, or the spring systems.
That's a pure vibration is you're imposing pure vibration on the spring-mass-dashpot system, and you want to see how it responds to that.
Or, you could put them together, and make these decaying oscillations.
So, we could also have something like e to the ax times sine omega x, or times cosine omega x.
Now, the point is, all of these together are really just special cases of one general thing, exponential, if you allow the exponent not to be a real number, but to be a complex number.
So, they're all special cases of e to the, I'll write it alpha x, well, why don't we write it (a plus i omega) x, right?
If omega is zero, then I've got this case.
If a is zero, I got this case separating into its real and imaginary parts.
And, if neither is zero, I have this case.
But, I don't want to keep writing (a plus i omega) all the time.
So, I'm going to write that simply as e to the alpha x.
And, you understand that alpha is a complex number now.
It doesn't look like a real number.
Okay, so the complex number.
So, the equation we are solving is which one?
This pretty purple equation.
And, we are trying to find a particular solution of it.
And, the special functions we are going to use are these, well, this one in particular, e to the alpha x.
That's going to be our input.
Now, it turns out this is amazingly easy to do because it's an exponential because I write it in exponential form.
The idea is simply to use a rule which, in fact, you know already, the rule of substitution.
So, I'm going to write the equation in the form, so, there it is.
It's y double prime plus A y prime plus B y equals f of x.
But, I'm going to think of the left-hand side as the polynomial operator, AD plus B.
A and B are constants, applied to y equals f of x.
That's the way I write the thing.
And, this part, I'm going to think of in the form.
This is p of D, a polynomial in D. In fact, it's a simple quadratic polynomial.
But, most of what I'm going to say today would apply equally well if we were a higher order polynomial, a polynomial of higher degree.
And, just to reinforce the idea, I've given you one problem in your problem set when p is a polynomial of higher degree.
I should say, the notes are written for general polynomials, not just for quadratic ones.
I'm simplifying it by leaving it, today, I'll do what's in the notes, but I'll do it in the quadratic case to save a little time, and because that's the one you will be most concerned with in the problems.
All right, so p of D y equals f of x.
And now, there are just a couple of basic formulas that we're going to use all the time.
The first is that if you apply p of D to a complex exponential, or a real one, it doesn't matter, the answer is you get just what you started with, with D substituted by alpha.
So, it's p of alpha.
In other words, put an alpha wherever you saw a D in the polynomial.
And, what is this?
Well, this is now just an ordinary complex number, and multiply that by what you started with, e to the alpha x.
So, that's a basic formula.
It's called in the notes the substitution rule because the heart of it is, you substitute for the D, you substitute alpha.
Now, this hardly requires proof.
But, let's prove it just so you see, to reinforce things and make things go a little more slowly to make sure you are on board all the time.
How would I prove that?
Well, just calculate it out, what in fact is (D squared plus AD plus B) times e to the alpha x.
Well, it's D squared e to the alpha x by linearity plus AD times e to the alpha x plus B times e to the alpha x.
Well, what are these?
What's the derivative of e to the alpha x?
It's just alpha times e to the alpha x.
What's a second derivative?
Well, if you remember from the exam, you can do tenth derivatives now.
So, the second derivative is easy.
It's alpha squared times e to the alpha x.
In other words, this law, what I'm saying really is that this law is obviously, quote unquote, "true."
Okay, I'm not even going to put it in quotes.
It's obviously true for the operator, D, and the operator D squared.
In other words, D of e to the alpha x equals alpha times e to the alpha x.
D squared times e to the alpha x equals alpha squared times e to the alpha x.
And, therefore, it's true for linear combinations of these as well by linearity.
So, therefore, also true for p of D. And, in fact, so if you calculate it out, what is it?
This is alpha squared e to the alpha x plus alpha e to the alpha x times the coefficient plus b times e to the alpha x.
So, it's in fact exactly this.
It's e to the alpha x times (alpha squared plus A alpha plus B).
Now, how are we going to use this?
Well, the idea is very simple.
Remember, we're trying to solve this, I should have some consistent notation for these equations.
Purple, I think, will be the right thing here.
You are solving purple equations.
The formulas which will solve them will be orange formulas, and we will see what we need as we go along.
So, I would like to just formulate it, this solution, the particular solution now.
I'm going to call it a theorem.
It's really too simple to be a theorem.
On the other hand, it's too important not to be a theorem.
So, let's call it, as I called it in the notes, the exponential input theorem, which says it all.
Theorem says it's important.
Exponential input means it's taking f of x to be an exponential.
It's an exponential input, and the theorem tells you what the response is.
So, for that equation, I'm not going to recopy the equation for the purple equation, adequately indicated this way.
There.
Now try to take notes.
For the purple equation, a solution is e to the alpha x.
Somewhere I neglected to say that f of x, all right, so for the purple equals e to the alpha x, how about that?
That equation, y double prime plus a y prime plus b y equals e to the alpha x.
So, here's the exponential input.
The solution is e to the alpha x divided by p of alpha.
Now, that's a very useful formula.
In fact, Haynes Miller, who also teaches this course, in his notes calls of the most important theorem in the course.
Well, I don't have to totally agree with him, but it's certainly important.
It's probably the most important theorem for these two weeks, anyway.
But, you will have others as well.
Okay, so that's a theorem.
The theorem is going green.
You can tell what they are by their color code.
Well, in other words, what I've done is simply write down the solution for you, write down the particular solution.
But let's verify it in general.
So, the proof would be what?
Well, I have to substitute it into the equation.
So, the equation is p of D applied to y is equal to alpha x.
And, I want to know, when I substitute that expression in, is it the case that when I plug it in, that the right-hand side, I calculate it out, apply p of D to it.
Is it the case that I get e to the alpha x on the right?
Well, all you have to do is do it.
What is p of D applied to e to the alpha x divided by p of alpha?
Well, p of D applied to e to the alpha x is p of alpha times e to the alpha x.
That's the substitution rule.
What about this guy?
This guy is a constant, so it just gets dragged along because this operator is linear.
If this applied to that is this, then if I apply it to one half that, I get one half the answer, and so on.
So, the p of alpha is a constant and just gets dragged along.
And now, they cancel each other, and the answer is, indeed, e to the alpha x.
That's not much of a proof.
I hope that to at least half this class, you're wondering, yes, but what if Peter had not caught the wolf?
I mean, what if?
What if?
I'm looking stern.
Okay, we will take care of it in the simplest possible way.
We will assume that p of alpha is not zero.
The case p of alpha is zero is, in fact, an extremely important case, one that makes the world go 'round, one that contributes to all sorts of catastrophes, and they occur first here in the solution of differential equations, and that's what controls all the catastrophes.
But, there's a good side to it, too.
It also makes a lot of good things happen.
So, there are no moral judgments in mathematics.
For the time being, let's assume p of alpha is not zero.
And, that proof is okay because the p of alpha, being in the denominator, it's okay to be in the denominator if you're not zero.
Okay, let's work in a simple example.
Well, I'm picking the most complicated example I can think of.
Simple examples, I'll leave for your practice and for the recitations, can start off with simple examples if you are confused by this.
But, let's solve an equation, find a particular solution.
So, y double prime plus minus y prime plus 2y is equal to 10 e to the minus x sine x. Gulp.
Okay, so, the input is this function, 10 e to the minus x, it's a decaying oscillation.
You're seeing those already on the computer screen if you started your homework, if you've done problem one on your homework.
It's a decaying exponential, and I want to find a particular solution.
Well, let's find a particular and the general solution.
Find the general solution.
Well, the main part of the work is finding the particular solution, but let's quickly, the general solution, let's find first the complementary part of it, in other words, the solution to the homogeneous equation.
That's D squared minus D plus two.
No, let's not.
I don't want to solve messy quadratics.
Okay, we're going to find a particular solution.
I thought it was going to come out easy, and then I realized it wasn't because I picked the wrong signs.
Okay, so if you don't like, just change the problem.
I can do that, but you cannot.
Don't forget that.
So, we want a particular solution in our equation.
It is this equals that.
Now, let's complexify it to make this part of a complex exponential.
So, the complex exponential that's relevant is ten times e to the (minus one plus i), you see that?
x.
What is this?
This is the imaginary part of this complex exponential.
So, this is imaginary part of that guy, e to the negative x times e to the i x, and the imaginary part of e to the i x.
The ten, of course, just comes along for the ride.
Okay, well, now, since this is a complex equation, I shouldn't call this y anymore by my notation.
I like to call it y tilde to indicate that the solution we get to this is not going to be the original solution to the original problem, but you will have to take the imaginary part of it to get it.
So, we are looking, now, for the complex solution to this complexified equation.
Okay, what is it?
Well, the complex particular solution I can write down immediately.
It is ten, that, of course, just gets dragged along by linearity, times e to the (minus one plus i) times x.
And, it's over this polynomial evaluated at this alpha.
So, just write it down with, have faith.
So, what do I get?
The alpha is minus one plus i. I. square that, because I'm substituting this alpha into that polynomial.
The reason I'm doing that is because the formula tells me to do it.
That's going to be that solution.
Okay, so it's minus one plus i, the quantity squared, minus minus one plus i plus two.
All I've done is substitute minus one plus i for D in that polynomial, the quadratic polynomial.
And now, all I want is the imaginary part of this.
The imaginary part of this will be the solution to the original problem because this was the right hand side with the imaginary part of the complexified right hand side.
Okay, now, let's make it look a little better, yp tilde.
Clearly, what we have to do something nice to the denominator.
So, I'll copy the numerator.
That's e to the minus (one plus i) x and how about the denominator?
Well, again, don't expand things out because it's already this long.
And, what's the point of making it this long?
You want to make it as long, right?
Okay, then there is room here for one real number, and another real number times i, there's no more room.
Okay, what's the real number?
Okay, we're looking for the real part of this expression.
So, just put it in and keep it mentally.
So, minus one squared: that's one, plus i squared, that's minus one.
One minus one is zero.
I can forget about that term.
The term gives me plus one for the real part, plus two.
The answer is that the real part is three.
How about the imaginary part?
Well, from here, there's negative 2i, negative 2i.
I'm expanding that out by the binomial theorem, or whatever you like to call that, minus 2i minus i makes minus 3i.
Is that right?
Minus 2i, minus i, minus 3i.
So, it is ten thirds, and now in the denominator I have one minus i. I'll put that in the numerator, make it one plus i, but I have to divide by the product of one minus i and its complex conjugate.
In other words, I'm multiplying both top and bottom by one plus i.
And so, that makes here one squared plus one squared is two.
And now, what's left is e to the negative x times cosine x plus i sine x.
Now, of that, what we want is just the imaginary part.
Well, let's see.
Two goes into ten makes five, so that's five thirds.
So, we're practically at our solution.
The solution, then, finally, is going to be yp is the imaginary part of yp tilde.
And, what's that?
Well, what's the coefficient out front, first of all?
It's five thirds, so let's pull out to five thirds before we forget it.
And, we'll pull out the e to the negative x before we forget that.
And then, the rest is simply a question of seeing what's left.
Well, it's one.
I want the imaginary part.
So, the imaginary part is going to be one times cosine x, and then the other imaginary part comes from these two pieces, which is one times sine x.
And, that should be the particular solution.
Notice that most of the work is not getting this thing.
It's turning it into something human that you can take the real and imaginary parts of.
If we don't like this form, you can put it in the other form, which many engineers would do almost automatically, make it five thirds, e to the negative x, and what will that be?
Well, you can use the general formula if you want.
Remember, cosine, the two coefficients are one and one, so it's one and one.
So, this is the square root of two.
So, it is times, this part makes the square root of two times cosine of x minus pi over the angle.
This is a phi.
So, that's pi over four, minus pi over four.
Okay, all right, now let's address the case which is going to occupy a lot of the rest of today, and in a certain sense, all of next time.
What happens when p of alpha is zero?
Well, in order to be able to handle this decently, it's necessary to have one more formula, which is very slightly more complicated than the substitution rule.
But, it's the same kind of rule.
I'm going to call this, or it is called the exponential.
So, I'm going to first prove a formula, which is the analog of that, and then I will prove a green formula, which is what to do here if p of alpha turns out to be zero.
But, in order to be able to prove that, we're going to be the analog of the orange formula.
And, the analogue of the orange formula, that tells you how to apply p of D to a simple exponential.
I need a formula which applies p of D to that simple exponential times another function.
Now, I found I got into trouble by continuing to call that alpha.
So, I'm now going to change the name of alpha to change alpha's name to a.
But, it's still complex.
I don't mean it's guaranteed to be complex.
I mean it's allowed to be complex.
So, a is now allowed to be a complex number.
I'm thinking of it, in general, as a complex number, okay?
I hope this doesn't upset you too much, but you know, you change x to t's, and y's to x's.
This is no worse.
All right, what we going to do?
Well, I'm going to use this exponential shift rule, I'll call it, exponential shift rule or formula or law.
That's the substitution rule for me.
So, this is going to be exponential shift law.
And, to apply, it tells you how to apply the polynomial to not D, not just the exponential, but the exponential times some function of x.
What's that?
And now, the rule is very simple.
See, you can understand the difficulty.
If you try to start differentiating, you're going to have to calculate second derivatives of the stuff, and God forbid, higher order equations.
You would have to calculate fourth derivatives, fifth derivatives.
You barely even want to calculate the first derivative.
That's okay.
But, second derivative, do I have to?
No, not if you know the exponential shift rule, which says you can get rid of the e to the ax, make it pass to the left of the operator where it's not in any position to do any harm any longer, or upset the differentiation.
And, all you have to do is, when it passes over that operator, it changes D to D plus a.
So, the answer is, e to the ax.
There, it's passed over.
But, when it did so, it changed D to D plus a.
And, what about the u?
Well, the u just stayed there.
Nothing happened to it.
Okay, there's our orange formula.
I guess we better put the thing around the whole business.
Should I prove that, or the proof is quite easy.
So, let's do it just again to give you a chance to try to see, now, if somebody gives you a formula like that, you first stare at it.
You might try a couple of special cases, try it on a function and see if it works, but already, you probably don't want to do that.
I mean, even if you took a function like x here, you'd have to do a certain amount of differentiating, and some quadratic thing here.
You'd calculate and calculate away for a little while, and then if you did it correctly, the two would in fact turn out to be equal.
But, you would not necessarily feel any the wiser.
A better procedure in trying to understand something like this is say, well, let's keep the u general.
Suppose we make D simple.
For example, well, if D is a constant, of course there's nothing to happen because if this is just a constant, both sides of these are the same.
This doesn't make any sense if p doesn't really have a D in it.
Well, what's the simplest polynomial which would have a D in it?
Well, D itself.
So, let's take a special case.
p of D equals D, and check the formula in that case; see if it works.
So, the formula is asking us, what is D, that's the p of D, of e to the ax times u?
I'm not going to put in the variable here because it's just a waste of chalk.
Well, what is that?
I know how to calculate that.
I'll use the product rule.
So, it's the derivative.
I'll tell you what; let's do the other order first.
So, it's e to the ax times the derivative of u plus the derivative of e to the ax, which is a times e to the ax times u.
Do you follow that?
This is the product rule.
It's e to the ax times the derivative of u plus the derivative of e to the ax, which is this thing times u, the other factor.
Now, is that right?
I want to make it look like that.
Well, to make it look like that I should first factor e to the ax out.
And now, what's left?
Well, if I factor e to the ax out, what's left is Du plus au, which is exactly (D plus a) operating on u, D u plus a u.
Well, hey, that's just what the formula said it should be.
If you make e to the x pass over D, it changes D to D plus a.
Okay, now here's the main thing I want to show you.
All right, now, well let's try, if this is true, also works out for D squared, then the formula is clearly true by linearity because an arbitrary p of D is just a linear combination with constant coefficients of D, D squared, and that constant thing, which we agreed there was nothing to prove about.
Now, hack, you're a hack if you take D squared and start calculating the second derivative of this.
Okay, it's question about hacks.
I mean, it's just, you haven't learned the right thing to do.
Okay, that will work, but it's not what you want to do.
Instead, you bootstrap your way up.
I have already a formula telling me how to handle this.
And, you can be anything.
Look at this not as D squared all by itself.
Calculate, instead, D squared e to the ax times u.
Think of that as D, the derivative of the derivative of e to the a x u.
In other words, we will do it one step at a time.
But you see now immediately the advantage of this.
What's D of e to the a x u?
Well, I just calculated that.
Now, don't go back to the beginning.
Don't go back to here.
Use the formula.
After all, you worked to calculate it, or I did.
So, it's D of, and what's this inside?
It's e to the ax times (D plus a) times u.
Well, that looks like a mess, but it isn't because I'm taking D of e to the ax times something.
And I already know how to take D of e to the ax times something.
It doesn't matter what that something is.
Here, the something was u.
Here, the something is D plus (a times u) operating on u.
But, the principle is the same, and the answer is what?
Well, to take D of e to the ax times something, you pass the e to the ax over the D. That changes D to D plus a.
And, you apply that to the other guy, which is (D plus a) applied to u.
What's the answer?
e to the a x times (D plus a) squared u.
It's just what you would have gotten if you had taken e to the e to the x, pass it over, and then changed D to D plus a.
Now, another advantage to doing it this way is you can see that this argument is going to generalize to D cubed.
In other words, you would continue on in the same way by the process of mathematical, one word, mathematical, begins with an I, induction.
By induction, you would prove the same formula for the nth derivative.
If you don't know what mathematical induction is, shame on you.
But it's okay.
A lot of you will be able to go through life without ever having to learn what it is.
And, the rest of you will be computer scientists.
Okay, so that's the idea of this rule.
Now, we can use it to calculate something.
Let's see, I'm going to need green for this, I guess, for our better formula.
The formula, now, that tells you what to do if p of alpha is zero.
So, we're trying to solve the equation, D squared plus A D, we are trying to find a particular solution, e to the ax, let's say.
Remember, a is complex.
a could be complex.
It doesn't have to be real.
But, the problem is that p of alpha is zero.
How do I get a particular solution?
Well, I will write it down for you.
So, this is part of that exponential input theorem.
I think that's the way it is in the notes.
I gave all the cases together, but I thought pedagogically it's probably a little better to do the simplest case first, and then build up on the complexity.
So, what's yp?
The answer is yp is e to the a, except now you have to multiply it by x out front.
Where have you done something like that before?
Yes, don't tell me.
I know you know.
But, what should go in the denominator?
Clearly not p of alpha.
What goes in the denominator is the derivative.
Okay, but what if p prime of alpha is zero?
Couldn't that happen?
Yes, it could happen.
So, we better make cases.
This case is, the case where this is okay corresponds to the case where we're going to assume that alpha is a simple root, is a simple root of the polynomial, p. I don't know what to call the variable, p of D is okay.
A simple zero, in other words, it's not double.
Well, suppose is double.
One of the consequences you will see just in a second, if it's a simple zero, that means this derivative is not going to be zero.
That's automatic.
Yeah, well, suppose it's not as simple.
Well, suppose is a double root.
How did a-- How did that get changed to, argh!
[LAUGHTER] That's not an alpha.
Oh, well, yes it is, obviously.
Change!
All of you, I want you to change.
They should have something like in a search key where occurrences of alpha have been changed to a with a stroke of a, just your thumb.
They don't have that for the blackboard, unfortunately.
Well, too bad, for the future.
Correctable blackboards.
Well, what if a is double root?
It can't be more than a double root because you've only got a quadratic polynomial.
Quadratic polynomials only have two roots.
So, the worst that can happen is that both of them are a.
All right, in that case the formula should be yp is equal to, you are now going to need x squared up there times e to the ax, and in the denominator what you are going to need is the second derivative of, evaluated at a.
Now, you can guess the way this going to go on.
For higher degree things, if you've got a triple root, you will need here x cubed, except you're going to need a factorial there, too.
So, don't worry about it.
It's in the notes, but I'm not going to give you that for higher roots.
I don't even know if I will give it to you for double root.
Yes, I already did, so it's too late.
It's too late.
Okay, so we will make this two formulas according to whether a is a single or a double root.
Okay, let's prove one of these, and all of that will be good enough for my conscience.
Let's prove the first one.
Mostly, it's an exercise in using the first exponential shift rule.
Okay, this will be a first example actually seeing a work in practice as opposed to proving it.
Okay, so what does that thing look like?
So, what does the polynomial look like, which has a as a simple root?
So, we're going to try to prove the simple root case.
So, I'm just going to calculate what those guys actually look like.
What does p of D look like if a is a simple root?
Well, if it's a simple root, that means it has a factor.
When it factors, it factors into the product of (D minus a) times something which isn't, D minus some other root.
And, the point is that b is not equal to A.
The roots are really distinct.
Okay, what's, then, p prime, I'm going to have to calculate p prime of a.
What is that?
Well, let's calculate p prime of D first.
It is, well, by the ordinary product rule, it's the derivative of this times, which is one times (D minus a) plus, that's one thing plus the same thing on the other side, the derivative of this, which is one times (D minus b).
So, that's p prime.
And therefore, what's p prime of a?
It's nothing but, this part is zero, and that's a minus b.
Of course, this is not zero because it's a simple root.
And, that's the proof for you if you want, that if the root is simple, that p prime of a is guaranteed not to be zero.
And, you can see, it's going to be zero exactly when b equals a, and that root occurs twice.
But, I'm assuming that didn't happen.
Okay, then all the rest we have to do is calculate, do the calculation.
So, what I want to prove now is that with this p of D, what I'm trying to calculate that p of D times that guy, x e to the a x, except I'm going to write it as e to the a x times x, guess why, divided by p prime of a.
This is my proposed particular solution.
So, what I have to do is calculate it, and see that it turns out to be, what do I hope it turns out to be?
What the right hand side of the equation, the input?
The input is e to the ax.
If this is true, then yp, a particular solution, indeed, nothing will be a particular solution.
Of course, there could be others, but in this game, I only have to find one particular solution, and that certainly by far is the simple as one you could possibly find.
So, I have to calculate this.
And now, you see why I did the exponential shift rule because this is begging to be differentiated by something simpler than hack.
Okay, you can also see why I violated the natural order of things and put the e to the ax on the left in order that it pass over more easily.
So, the answer on the left-hand side is e to the ax times p of (D plus a).
Now, what is (p of D) plus a?
Write it in this form.
It's going to be a minus b.
So, p of (D plus a) is, change D to D plus a.
So, the first factor is going to be D plus a minus b.
And, what's the second factor?
Change D to D plus a.
It turns into D. All this is the result of taking that p of D, and changing D to D plus a.
And now, this is to be applied to what?
Well, e to the a x is already passed over.
So, what's left is x.
And, that's to be divided by the constant, p prime of a.
Now, what does this all come out to be?
e to the ax, what's D applied to x?
One, right?
And now, what's this thing applied to the constant one?
Well, the D kills it, so it has no effect.
It makes it zero.
The rest just multiplies it by a minus b.
So, the answer to the top is (a minus b) times one.
And, the answer to the bottom is p prime of a, which I showed you by just explicit calculation is a minus b.
And so the answer is, e to the a x comes out right.
Now, the other one, the other formula comes out the same way.
I'll leave that as an exercise.
Also, I don't dare do it because it's much too close to the problem I asked you to do for homework.
So, let's by way of conclusion, I'll do one more simple example, okay?
And then, you can feel you understand something.
I'm sort of bothered that I haven't done any examples of this more complicated case.
So, I'll pick an easy version instead of the one that you have in your notes, which is the one you have for homework, which is even easier.
So, this one's epsilon less easy.
Y double prime minus 3 y prime plus 2 y equals e to the x.
Okay, notice that one is a simple root.
The one I'm talking about is the a here, which is one.
One is a simple root of the polynomial D squared minus 3D plus two, isn't it?
It's a zero.
Put D equal one and you get one minus three plus two equals zero.
It's a simple root because anybody can see that one is not a double root because you know from critical damping, if one were a double root, you know just what the polynomial would look like, and it wouldn't look like that at all.
It would not look like D squared minus 3D plus two.
It would look differently.
Therefore, that proves that one is a simple root.
Okay, what's the particular solution, therefore?
The particular solution is x times e to the x divided by the derivative, the derivative evaluated at the point, so, what's p prime of D?
It is 2D minus three.
If I evaluate it at the point, one, it is negative one.
So, if this is to be divided by negative one, in other words, it's minus x e to the X.
And, if you don't believe it, you could plug it in and check it out.
Okay, I'm letting you out one minute early.
Remember that.
I'm trying to pay off the accumulated debt.
I just recalling some of the notation we are going to need for today, and a couple of the facts that we're going to use, plus trying to clear up a couple of confusions that the recitations report.
This can be thought of two ways.
It's a formal polynomial in D, in the letter D. It just has the shape of the polynomial, D squared plus AD plus B.
A and B are constant coefficients.
But, it's also, at the same time, if you think what it does, it's a linear operator on functions.
It's a linear operator on functions like y of t. You think of it both ways: formal polynomial because we want to do things like factoring it, substituting two for D and things like that.
Those are things you do with polynomials.
You do them algebraically.
You can take the formal derivative of the polynomial because it's just sums of powers.
On the other hand, as a linear operator, it does something to functions.
It differentiates them, multiplies them by constants or something like that.
So it's, so to speak, has a dual aspect this way.
And, that's one of the things we are exploiting what we use operator methods to solve differential equations.
Now, let me remind you of the key thing we were interested in.
f of t: not any old function, we'll get to that next time, but f of t, exponentials.
So, it should be an exponential or something like an exponential, or pretty close to it, for example, something with sine t and cosine t, or e to the, that could be thought of as part of the real or imaginary part of a complex exponential.
And, maybe by the end of today, we will have generalized that even little more.
But basically, I'm interested in exponentials.
Let's make it alpha complex.
That will at least take care of the cases, e to the ax times cosine bx, sin bx, which are the main cases.
Those are the main cases.
Then, remember the little table we made.
I simply gave you the formula for the particular solution.
So, what we're looking for is we already know how to solve the homogeneous equation.
What we want is that particular solution.
And then, the recipe for it I gave you, these things were proved by the substitution rules and exponential shift rules.
The recipe was that if f of t was, let's make a little table.
f of t is, well, it's always e to the a t. So, in other words, it's e to the a t. The cases are, so yp, what is the yp?
Well, it is the normal case is yp equals e to that alpha t divided by the polynomial where you substitute, you take that polynomial, and wherever you see a D, you substitute the complex number, alpha.
There, I'm thinking of it as a formal polynomial.
I'm not thinking of it as an operator.
Now, this breaks down.
So, that's the formula for the particular solution.
The only trouble is, it breaks down if p of alpha is zero.
So, we have to assume that it's not.
Now, if p of alpha is zero, that means alpha is a root of the polynomial, a zero of the polynomial is a better word.
So, in that case, it will be e to the alpha t divided by p prime of alpha.
Differentiate formally the polynomials, -- -- and you will get 2D plus A.
And now, substitute in the alpha.
And, this will be okay provided p prime of alpha is not zero.
That means that alpha is the simple root, simple zero of p. And then, there's one more case, which, since I won't need today, I won't write on the board.
But, you'll need it for homework.
So, make sure you know it.
Another words, if this is zero, then you've got a double root.
And, there is still a different formula.
And, this is wrong because I forgot the t. Yes?
I could tell on your faces.
That was before, and now we are up to today.
What we are interested in talking about today is what this has to do with the phenomenon of resonance.
Everybody knows at least one case of resonance, I hope.
A little kid is on his swing, right?
Back and forth, and they are very, very little, so they want a push.
Okay, well, everybody knows that to make the swing go, a swing has a certain natural frequency.
It swings back and forth like that.
It's a simple pendulum.
It's actually damped, but let's pretend that it isn't.
Everybody knows you want to push a kid on a swing so that they go high.
You have to push with essentially the same frequency that the natural frequency of the spring, of the swing is.
It's automatic, because when you come back here, it gets to there, and that's where you push.
So, automatically, you time your pushes.
But if you want the kid to stop, you just do the opposite.
Push at the wrong time.
So anyway, that's resonance.
Of course, there are more serious applications of it.
It's what made the Tacoma Bridge fall down, and I think movies of that are now being shown not merely on television, but in elementary school.
Resonance is what made, okay, more resonance stories later.
So, my aim is, what is this physical phenomenon, that to get a big amplitude you should have it match the frequency?
What does that have to do with a differential equation?
Well, the differential equation for that simple pendulum, let's assume it's undamped, will be of the type y double prime plus, I'm using t now since t is time.
That will be our new independent variable, plus omega nought squared is the natural frequency of the pendulum or of the spring, or whatever it is that's doing the vibrating.
Yeah, any questions?
What we're doing is driving that with the cosine, with something of a different frequency.
So, this is the input, or the driving term as it's often called, or it's sometimes called the forcing term.
And, the point is I'm going to assume that the frequency is different.
The driving frequency is different from the natural frequency.
So, this is the input frequency.
Okay, and now let's simply solve the equation and see what we get.
So, it's if I write it using the operator, it's D squared plus omega nought squared applied to y is equal to cosine.
It's a good idea to do this because the formulas are going to ask you to substitute into a polynomial.
So, it's good to have the polynomial right in front of you to avoid the possibility of error.
Well, really what I want is the particular solution.
It's the particular solution that's going to give me a pure oscillation.
And, the thing to do is, of course, since this cosine, you want to make it complex.
So, we are going to complexify the equation in order to be able to solve it more easily, and in order to be able to use those formulas.
So, the complex equation is going to be D squared plus omega nought squared.
Well, it's going to be a complex, particular solution.
So, I'll call it y tilde.
And, on the right-hand side, that's going to be e to the i omega1 t. Cosine is the real part of this.
So, when we get our answer, we want to be sure to take the real part of the answer.
I don't want the complex answer, I want its real part.
I want the real answer, in other words, the really real answer, the real real answer.
So, now without further ado, because of those beautiful, the problem has been solved once and for all by using the substitution rule.
I did that for you on Monday.
The answer is simply e to the i omega1 t divided by what?
This polynomial with omega one substituted in for D. So, sorry, i omega one, the complex coefficient of t. So, it is substitute i omega for D, I omega one for D, and you get (i omega one) squared plus omega nought squared.
Well, let's make that look a little bit better.
This should be e to the (i omega one t) divided by, now, what's this?
This is simply omega nought squared minus omega one squared.
But, I want the real part of it.
So, as one final, last step, the real part of that is what we call just the real particular solution, so, yp without the tilde anymore.
And, the real part of this, well, this cosine plus i sine.
And, the denominator, luckily, turns out to be real.
So, it's simply going to be cosine omega one t. That's the top, divided by this thing, omega nought squared minus omega one squared.
In other words, that's the response.
This is the input, and that's what came out.
Well, in other words, what one sees is, regardless of what natural frequency this system wanted to use for itself, at least for this solution, what it responds to is the driving frequency, the input frequency.
The only thing is that the amplitude has changed, and in a rather dramatic way, if omega1, depending on the relative sizes of omega1 and omega2.
Now, the interesting case is when omega one is very close to omega, the natural frequency.
When you push it with approximately it's natural frequency, then the solution is big amplitude.
The amplitude is large.
So, the solution looks like the frequency.
The input might have looked like this.
Well, it's cosine, so it ought to start up here.
The input might have looked like this, but the response will be a curve with the same frequency and still a pure oscillation.
But, it will have much, much bigger amplitude.
And, it's because the denominator, omega nought squared minus omega one squared, is always zero.
So, the response will, instead, look like this.
Now, to all intents and purposes, that's resonance.
You are pushing something with approximately the same frequency, something that wants to oscillate.
And, you are pushing it with approximately the same frequency that it would like to oscillate by itself.
And, what that does is it builds up the amplitude Well, what happens if omega one is actually equal to omega zero?
So, that's the case I'd like to analyze for you now.
Suppose the two are equal, in other words.
Well, the problem is, of course, I can't use that same solution.
It isn't applicable.
But that's why I gave you, derived for you using the exponential shift law last time, the second version, when it is a root.
So, if omega one equals omega nought, so now our equation looks like D squared plus omega nought squared, the natural frequency, y.
But this time, the driving frequency, the input frequency, is omega nought itself.
Then, the same analysis, a lot of it is, well, I'd better be careful.
I'd better be careful.
Let's go through the analysis again very rapidly.
What we want to do is first complexify it, and then solve.
So, the complex equation will be D squared plus omega nought squared times y tilde equals e to the i omega nought t, this time.
But now, i omega is zero of this polynomial.
That's why I picked it, right?
If I plug in i omega zero, I get i omega zero quantity squared plus omega nought squared.
That's zero.
So, I'm in the second case.
So, i omega nought is a simple root, simple zero, of D squared plus omega nought, that polynomial squared.
Therefore, the complex particular solution is now t e to the i omega nought t divided by p prime, where you plug in that root, the i omega nought.
Now, what's p prime?
p prime is 2D, right?
If I differentiate this formally, as if D were a variable, the way you differentiate polynomials, the derivative, this is a constant, and the derivative is 2D.
So, the denominator should have two times for D. You are going to plug in i omega zero.
So, it's 2 i omega zero.
And now, I want the real part of that, which is what?
Well, think about it.
The top is cosine plus i sine.
The real part is now going to come from the sine, right, because it's cosine plus i sine.
But this i is going to divide out the i that goes with this sine.
And, therefore, the real part is going to be t times the sine, this time, of omega nought t. And, that's going to be divided by, well, the i canceled out the i that was in front of the sine function.
And therefore, what's left is two omega nought down below.
So, that's our particular solution now.
Well, it looks different from that guy.
It doesn't look like that anymore.
What does it look like?
Well, it shows the way to plot such things is basically it's an oscillation of frequency omega nought.
But, its amplitude is changing.
So, the way to do it is, as always, if you have a basic oscillation which is neither too fast nor too slow, think of that as the thing, and the other stuff multiplying it, think of it as changing the amplitude of that oscillation with time.
So, the amplitude is that function, t divided by two omega zero.
So, just as we did when we talked about damping, you plot that and it's negative on the picture.
So, this is the function whose graph is t divided by two omega nought.
That's the changing amplitude, as it were.
And then, the function itself does what oscillation it can, but it has to stay within those lines.
So, the thing that's oscillating is sine omega nought t, which would like to be a pure oscillation, but can't because its amplitude is being changed by that thing.
So, it's doing this, and now the rest I have to leave to your imagination.
In other words, what happens when omega nought is equal to, when the driving frequency is actually equal to omega nought, mathematically this turns into a different looking solution, one with steadily increasing amplitude.
The amplitude increases linearly like the function t divided by two omega nought.
Well, many people are upset by this, slightly, in the sense that there is a funny feeling.
How is it that that solution can turn into this one?
If I simply let omega one go to omega zero, what happens?
Well, the pink curve just gets taller and taller, and after a while all you see of it is just a bunch of vertical lines which seem to be spaced at whatever the right period is for that function.
It's sort of like being in a first story window and watching a giraffe go by.
All you see is that.
Okay.
So, my concern is how does that function turn into this one?
I have something in mind to remind you of, and that's why we'll go through this little exercise.
It's a simple exercise.
But the function of it is, of course that as omega one goes to omega zero cannot possibly turn into this.
It's doing the wrong thing near zero.
It's already zooming up.
But, the point is, this is not the only particular solution on the block.
Any solution whatsoever of the differential equation, the inhomogeneous equation, is a particular solution.
It's like Fred Rogers: everybody is special.
Okay, so all solutions are special.
We don't have to use that one.
So, I will use, where are all the other solutions?
So, I'm going back to the equation D squared plus omega zero squared, applied to y, is equal to cosine omega one t. Now, the particular solution we found was that one, cosine omega one t divided by that omega nought squared minus omega one squared.
What do the other particular solutions look like?
Well, in general, any particular solution will look like that one we found, what is it, omega nought squared minus omega one squared, plus I'm allowed to add to it any piece of the complementary solution.
Equally particular, and equally good, as a particular solution is this plus anything which solved the homogeneous equation.
Now, all I'm going to do is pick out one good function which solves the homogeneous equation, and here it is.
It's the function minus cosine.
In fact, what does solve the homogeneous equation?
Well, it's solved by sine omega nought t, cosine omega nought t, and any linear combination of those.
So, out of all those functions, the one I'm going to pick is cosine omega nought t. And, I'm going to divide it by this same guy.
So, this is part of the complementary solution.
That's what we call the complementary solution, the solution to the associated homogeneous equation, to the reduced equation.
Call it what you like.
So, this is one of the guys in there, and it's still a particular solution to take the one I first found, and add to it anything which solves the homogeneous equation.
I showed you that when we first set out to solve the inhomogeneous equation in general.
Now, why do I pick that?
Well, I'm going to now calculate, what's the limit?
So, these guys are also good solutions to that.
This is a good solution to that equation, this equation.
All I'm going to do now is calculate the limit as omega one approaches omega zero of this function.
Well, what is that?
It's cosine omega one t minus cosine omega zero t divided by omega nought squared minus omega one squared.
Now, you see why I did that.
If I let just this guy, omega one approaches omega zero, I get infinity.
I don't get anything.
But, this is different here because I fixed it up, now.
The denominator becomes zero, but so does the numerator.
In other words, I've put myself in position to use L'Hopital rule.
So, let's L'Hopital it.
It's the limit.
As omega one approaches omega zero, and what do you do?
You differentiate the top and the bottom with respect to what?
Right, with respect to omega one.
Omega one is the variable.
That's what's changing.
The t that I'm thinking of is, I'm thinking, for the temporary fixed.
This has a fixed value.
Omega nought is fixed.
All that's changing in this limit operation is omega one.
And therefore, it's with respect to omega one that I differentiate it.
You got that?
Well, you are in no position to say yes or no, so I shouldn't even ask the question, but okay, rhetorical question.
All right, let's differentiate this expression, the top and bottom with respect to omega one.
So, the derivative of the top with respect to omega one is negative sine omega one t. But, I have to use the chain rule.
That's differentiating with respect to this argument, this variable.
But now, I must take times the derivative of this thing with respect to omega one.
And that is t is the constant, so times t. And, how about the bottom?
The derivative of the bottom with respect to omega one is, well, that's a constant.
So, it becomes zero.
And, this becomes negative two omega one.
So, it's the limit of this expression as omega one approaches omega zero.
And now it's not indeterminate anymore.
The answer is, the negative signs cancel.
It's simply t sine omega nought t divided by two omega nought.
So, that's how we get that solution.
It is a limit as omega one, but not of the particular solution we found first, but of this other one.
Now, it's still too much algebra.
I mean, what's going on here?
Well, that's something else you should know.
Okay, so my question is, therefore, what does this mean?
What's the geometric meaning of all this?
In other words, what does that function look like?
Well, that's another trigonometric identity, which in your book is just buried as half of one line sort of casual as if everybody knows it, and I know that virtually no one knows it.
But, here's your chance.
So, the cosine of B minus the cosine of A can be expressed as a product of signs.
It's the sine of (A minus B) over two times the sine of (A plus B) over two, I believe.
My only uncertainty: is there a two in front of that?
I think there has to be.
Let me check.
Sorry.
Is there a two?
I wouldn't trust my memory anyway.
I'd look it up.
I did look it up, two, yes.
If you had to prove that, you could use the sine formula to expand this out.
That would be a bad way to do it.
The best way is to use complex numbers.
Express the sign in terms of complex numbers, exponentials, you know, the backwards Euler formula.
Then do it here, and then just multiply those two expressions involving exponentials together, and cancel, cancel, cancel, cancel, cancel, and this is what you will end up with.
You see why I did this.
It's because this has that form.
So, let's apply that formula to it.
So, what's the left-hand side?
B is omega one t, and A is omega nought t. So, this is omega one t, and this is omega nought t All right, so what we get is that the cosine of omega one t minus the cosine of omega nought t, which is exactly the numerator of this function that I'm trying to get a handle on.
Then we will divide it by its amplitude.
So, that's this constant factor that's real.
It's a small number because I'm thinking of omega one as being rather close to omega zero, and getting closer and closer.
What does this tell us about the right-hand side?
Well, the right-hand side is twice the sine of A minus B.
Now, that's good because these guys sort of resemble each other.
So, that's (omega nought minus omega one) times t. That's A minus B, and I'm supposed to divide that by two.
And then, the other one will be the same thing with plus: sine omega nought plus omega one over two times t. Now, how big is this, approximately?
Remember, think of omega one as close to omega zero.
Then, this is approximately omega zero.
So this part is approximately sine of omega zero t. This part, on the other hand, that's a very small thing.
Okay, now what I want to know is what does this function look like?
The interest in knowing what the function looks like it is because we want to be able to see that it's limited is that thing.
You can't tell what's what its limit is, geometrically, unless you know it looks like.
So, what does it look like?
Well, again, the way to analyze it is the thing, that thing.
What you think of is, yeah, of course you cannot divide one side of equality without dividing the equation by the other side.
So, that's got to be there, too.
Now, what does that look like?
Well, the way to think of it is, here is something with a normal sort of frequency, omega nought.
It's doing its thing.
It's a sine curve.
It's doing that.
What's this?
Think of all this part as varying amplitude.
It's just another example of what I gave you before.
Here is a basic, pure oscillation, and now, think of everything else that's multiplying it as varying its amplitude.
All right, so what does that thing look like?
Well, first what we want to do is plot the amplitude lines.
Now, what will they be?
This is sine of an extremely small number times t. The frequency is small.
How does the sine curve look if its frequency is very low, very close to zero?
Well, that must mean its period is very large.
Here's something with a big frequency.
Here's something with a very, very low frequency.
Now, with a low frequency, it would hardly get off the ground and get up to one here, and it would do that.
But, it's made to look a little more presentable because of this coefficient in front, which is rather large.
And so, what this thing looks like, I won't pause to analyze it more exactly.
It's something which goes up at a reasonable rate for quite a while, and let's say that's quite awhile.
And then it comes down, and then it goes, and so on.
Of course, in figuring out its amplitude, we have to be willing to draw its negative, too.
And since I didn't figure things out right, I can at least make it cross, right?
Okay.
So, this is a picture of this slowly varying amplitude.
And in between, this is the function which is doing the oscillation, as well as it can.
But, it has to stay within that amplitude.
So, it's doing this.
Now, what happens?
As omega one approaches omega zero, this frequency gets closer and closer to zero, which means the period of that dotted line gets further and further out, goes to infinity, and you never do ultimately get a chance to come down again.
All you can see is the initial part, where it's rising and rising.
And, that's how this curve turns into that one.
Now, of course, this curve is enormously interesting.
You must have had this somewhere.
That's the phenomenon of what are called beats.
Too frequencies-- Your book has half a page explaining this.
That's the half a page where he gives you this identity, except it gives it in a wrong form, so that it's hard to figure out.
But anyway, the beats are two frequencies when you combine them, the two frequencies being two combined pure oscillations where the frequencies are very close to each other.
What you get is a curve which looks like that.
And, of course, what you hear is the envelope of the curve.
You hear the dotted lines.
Well, you hear this.
You hear that, too.
But, what you hear is-- And, that's how good violinists and cellists, and so on, tune their instruments.
They get one string right, and then the other strings are tuned by listening.
They don't actually listen for the sound of the note.
They listened just for the beats, wah, wah, wah, wah, and they turn the peg and it goes wah, wah, wah, wah, and then finally as soon as the wahs disappear, they know that the two strings are in tune.
A piano tuner does the same thing.
Of course, I, being a very bad cellist, use a tuner.
That's another solution, a more modern solution.
Okay.
Oh well.
Let's give it a try.
The bad news is that problem six in your problem set, I didn't ask you about the undamped case.
I thought, since you are mature citizens, you could be asked about the damped case.
I warn you, first of all you have to get the notation.
This is probably the most important thing I'll do with this.
Your book uses this, resonance.
I'm optimistic.
[LAUGHTER] Let's say zero or f of t. It doesn't matter.
In other words, the constants, the book uses two sets of constants to describe these equations.
If it's a spring, and not even talking about RLC circuits, the spring mass, damping, k, spring constant.
Then you divide out by m and you get this.
You're familiar with that.
And, it's only after you divided out by the m that you're allowed to call this the square of the natural frequency.
So, omega naught is the natural frequency, the natural undamped frequency.
If this term were not there, that omega nought would give the frequency with which the system, the little spring would like to vibrate by itself.
Now, further complication is that the visual uses neither of these.
The visual uses x double dot plus b times x prime, I think we will have to fix this in the future, but for now, just live with it, plus kx, and that's some function, again, a function.
So, in other words, the problem is that b is okay, can't be confused with c. On the other hand, this is not the same k as that.
What I'm trying to say is, don't automatically go to a formula one place, and assume it's the same formula in another place.
You have to use these equivalences.
You have to look and see how the basic equation was written, and then figure out what the constant should be.
Now, there was something called, when we analyzed this before, and this has happened in recitation, there was the natural, damped frequency.
I'll call it the natural, damped frequency.
The book calls it the pseudo-frequency.
It's called pseudo-frequency because the function, if you have zero on the right hand side, but have damping, the function isn't periodic.
It decays.
It does this.
Nonetheless, it still crosses the t-axis at regular intervals, and therefore, almost everybody just casually refers to it as the frequency, and understands it's the natural damped frequency.
Now, the relation between them is given by the little picture I drew you once.
But, I didn't emphasize it enough.
Here is omega nought.
Here is the right angle.
The side is omega one, and this side is the damping.
So, in other words, this is fixed because it's fixed by the spring.
That's the natural frequency of the spring, by itself.
If you are damping near the motion, then the more you damped it, the bigger this side gets, and therefore the smaller omega one is, the bigger the damping, then the smaller the frequency with which the damped thing vibrates.
That sort of intuitive, and vice versa.
If you decrease the damping to almost zero, well, then you'll make omega one almost the same size as omega zero.
This must be a right angle, and therefore, if there's very little damping, the natural damped frequency will be almost the same as the original frequency, the natural frequency.
So, the relation between them is that omega one squared is equal to omega nought squared minus p squared, and this comes from the characteristic roots from the characteristic roots of the damped equation.
So, we did that before.
I'm just reminding you of it.
Now, the third frequency which now enters, and that I'm asking you about on the problem set is if you've got a damped spring, okay, what happens when you impose a motion on it with yet a third frequency?
In other words, drive the damped spring.
I don't care.
I switched to y, since I'm in y mode.
So, our equation looks like this, just as it did before, except now going to drive that with an undetermined frequency, cosine omega t. And, my question, now, is, see, it's not going to be able to resonate in the correct-- you really only get true resonance when you don't have damping.
That's the only time where the amplitude can build up indefinitely.
But nonetheless, for all practical purposes, and there's always some damping unless you are a perfect vacuum or something, there's almost always some damping.
So, p isn't zero, can't be exactly zero.
So, the problem is, which omega gives, which frequency in the input, which input frequency gives the maximal amplitude for the response?
We solved that problem when it was undamped, and the answer was easy.
Omega should equal omega zero.
But, when it's damped, the answer is different.
And, I'm not asking you to do it in general.
I'm giving you some numbers.
But nonetheless, it still must be the case.
So, I'm giving you, I give you specific values of p and omega zero.
That's on the problem set.
Of course, one of them is tied to your recitation.
But, the answer is, I'm going to give you the general formula for the answer to make sure that you don't get wildly astray.
Let's call that omega r, the resonant omega.
This isn't true resonance.
Your book calls it practical resonance.
Again, most people just call it resonance.
So, you know what I mean, type of thing.
It is omega r is very much like that.
Maybe I should have written this one down in the same form.
Omega one is the square root of omega nought squared minus p squared.
What would you expect?
Well, what I would expect is that omega r should be omega one.
The damped system has a natural frequency.
The resonant frequency should be the same as that natural frequency with which the damped system wants to do its thing.
And the answer is, that's not right.
It is the square root.
It's a little lower.
It's a little lower.
It is omega nought squared minus two p squared.
Well, let's get started.
The topic for today is -- Sorry.
Thank you.
For today and the next two lectures, we are going to be studying Fourier series.
Today will be an introduction explaining what they are.
And, I calculate them, but I thought before we do that I ought to least give a couple minutes oversight of why and where we're going with them, and why they're coming into the course at this place at all.
So, the situation up to now is that we've been trying to solve equations of the form y double prime plus a y prime, constant coefficient second-order equations, and the f of t was the input.
So, we are considering inhomogeneous equations.
This is the input.
And so far, the response, then, is the solution equals the corresponding solution, y of t, maybe with some given initial conditions to pick out a special one we call the response, the response to that particular input.
And now, over the last few days, the inputs have been, however, extremely special.
For input, the basic input has been an exponential, or sines and cosines.
And, the trouble is that we learn how to solve those.
But the point is that those seem extremely special.
Now, the point of Fourier series is to show you that they are not as special as they look.
The reason is that, let's put it this way, that any reasonable f of t which is periodic, it doesn't have to be even very reasonable.
It can be somewhat discontinuous, although not terribly discontinuous, which is periodic with period, maybe not the minimal period, but some period two pi.
Of course, sine t and cosine t have the exact period two pi, but if I change the frequency to an integer frequency like sine 2t or sine 26 t, two pie would still be a period, although would not be the period.
The period would be shorter.
The point is, such a thing can always be represented as an infinite sum of sines and cosines.
So, it's going to look like this.
There's a constant term you have to put out front.
And then, the rest, instead of writing, it's rather long to write unless you use summation notation.
So, I will.
So, it's a sum from n equal one to infinity integer values of n, in other words, of a sine and a cosine.
It's customary to put the cosine first, and with the frequency, the n indicates the frequency of the thing.
And, the bn is sine nt.
Now, why does that solve the problem of general inputs for periodic functions, at least if the period is two pi or some fraction of it?
Well, you could think of it this way.
I'll make a little table.
I'll make a little table.
Let's look at, let's put over here the input, and here, I'll put the response.
Okay, suppose the input is the function sine nt.
Well, in other words, if you just solve the problem, you put a sine nt here, you know how to get the answer, find a particular solution, in other words.
In fact, you do it by converting this to a complex exponential, and then all the rigmarole we've been going through.
So, let's call the response something.
Let's call it y. I'd better index it by n because it, of course, is a response to this particular periodic function.
So, n of t, and if the input is cosine nt, that also will have a response, yn.
Now, I really can't call them both by the same name.
So, why don't we put a little s up here to indicate that that's the response to the sine.
And here, I'll put a little c to indicate what the answer to the cosine.
You're feeding cosine nt, what you get out is this function.
Now what?
Well, by the way, notice that if n is zero, it's going to take care of a constant term, too.
In other words, the reason there is a constant term out front is because that corresponds to cosine of zero t, which is one.
Now, suppose I input instead an cosine nt.
All you do is multiply the answer by an.
Same here.
Multiply the input by bn.
You multiply the response.
That's because the equation is a linear equation.
And now, what am I going to do?
I'm going to add them up.
If I add them up from the different ends and take a count also, the n equals zero corresponding to this first constant term, the sum of all these according to my Fourier formula is going to be f of t. What's the sum of this, the corresponding responses?
Well, that's going to be summation a n y n c t plus b n y n, the response to the sine.
That will be the sum from one to infinity, and there will be some sort of constant term here.
Let's just call it c1.
So, in other words, if this input produces that response, and these are things which we can calculate, we're led by this formula, Fourier's formula, to the response to things which otherwise we would have not been able to calculate, namely, any periodic function of period two pi will have, the procedure will be, you've got a periodic function of period two pi.
Find its Fourier series, and I'll show you how to do that today.
Find its Fourier series, and then the response to that general f of t will be this infinite series of functions, where these things are things you already know how to calculate.
They are the responses to sines and cosines.
And, you just formed the sum with those coefficients.
Now, why does that work?
It works by the superposition principle.
So, this is true.
The reason I can do the adding and multiplying by constant, I'm using the superposition principle.
If this input produces that response, then the sum of a bunch of inputs produces the sum of the corresponding responses.
And, why is that?
Why can I use the superposition principle?
Because the ODE is linear.
It's okay, since the ODE is linear.
That's what makes all this work.
Now, so what we're going to do today is I will show you how to calculate those Fourier series.
I will not be able to use it to actually solve any differential equation.
It will take us pretty much all the period to show how to calculate a Fourier series.
And, okay, so I'm going to solve differential equations on Monday.
Wrong.
I probably won't even get to it then because the calculation of a Fourier series is a sufficient amount of work that you really want to know all the possible tricks and shortcuts there are.
Unfortunately, they are not very clever tricks.
They are just obvious things.
But, it will take me a period to point out those obvious things, obvious in my sense if not in yours.
And, finally, the third day, we'll solve differential equations.
I will actually carry out the program.
But the main thing we're going to get out of it is another approach to resonance because the things that we are going to be interested in are picking out which of these terms may possibly produce resonance, and therefore a very crazy response.
Some of the terms in the response suddenly get a much bigger amplitude than this than you would normally have thought they had because it's picking out resonant terms in the Fourier series of the input.
Okay, well, that's a big mouthfu.
Let's get started on calculating.
So, the program today is calculate the Fourier series.
Given f of t periodic, having two pi as a period, find its Fourier series.
How, in other words, do I calculate those coefficients, an and bn.
Now, the answer is not immediately apparent, and it's really quite remarkable.
I think it's quite remarkable, anyway.
It's one of the basic things of higher mathematics.
And, what it depends upon are certain things called the orthogonality relations.
So, this is the place where you've got to learn what such things are.
Well, I think it would be a good idea to have a general definition, rather than immediately get into the specifics.
So, I'm going to call u of x, u of t, I think I will use, since Fourier analysis is most often applied when the variable is time, I think I will stick to independent variable t all period long, if I remember to, at any rate.
So, these are two continuous, or not very discontinuous functions on minus pi.
Let's make them periodic.
Let's say two pi is a period.
So, functions, for example like those guys, sine t, sine nt, sine 22t, and so on, say two pi is a period.
Well, I want them really on the whole real axis, not there.
Define for all real numbers.
Then, I say that they are orthogonal, perpendicular.
But nobody says perpendicular.
Orthogonal is the word, orthogonal on the interval minus pi to pi if the integral, so, two are orthogonal.
Well, these two functions, if the integral from minus pi to pi of u of t v of t, the product is zero, that's called the orthogonality condition on minus pi to pi.
Now, well, it's just the definition.
I would love to go into a little song and dance now on what the definition really means, and what its application, why the word orthogonal is used, because it really does have something to do with two vectors being orthogonal in the sense in which you live it in 18.02.
I'll have to put that on the ice for the moment, and whether I get to it or not depends on how fast I talk.
But, you probably prefer I talk slowly.
So, let's compromise.
Anyway, that's the condition.
And now, what I say is that that Fourier, that blue Fourier series, -- -- what finding the coefficients an and bn depends upon is this theorem that the collection of functions, as I look at this collection of functions, sine nt for any value of the integer, n, of course I can assume n is a positive integer because sine of minus nt is the same as sine of nt.
And, cosine mt, let's give it a different, so I don't want you to think they are exactly the same integers.
So, this is a big collection of functions, as n runs from one to infinity-- Here, I could let m be run from zero to infinity because cosine of zero t means something.
It's a constant, one-- that any two distinct ones, two distinct, you know, how can two things be not different?
Well, you know, you talk about two coincident roots.
I'm just killing, doing a little overkill.
Any two distinct ones of these, two distinct members of the set of this collection of, I don't know, there's no way to say that, any two distinct ones are orthogonal on this interval.
Of course, they all have two pi as a period for all of them.
So, they form into this general category that I'm talking about, but any two distinct ones are orthogonal on the interval for minus pi to pi.
So, if I integrate from minus pi to pi sine of three t times cosine of four t dt, answer is zero.
If I integrate sine of 3t times the sine of 60t, answer is zero.
The same thing with two cosines, or a sine and a cosine.
The only time you don't get zero is if you integrate, if you make the two functions the same.
Now, how do you know that you could not possibly get the answer is zero if the two functions are the same?
If the two functions are the same, then I'm integrating a square.
A square is always positive.
I'm integrating a square.
A square is always positive, and therefore I cannot get the answer, zero.
But, in the other cases, I might get the answer zero.
And the theorem is you always do.
Okay, so, why is this?
Well, there are three ways to prove this.
It's like many fundamental facts in mathematics.
There are different ways of going about it.
By the way, along with the theorem, I probably should have included, so, I'm far away.
But you might as well include, because we're going to need it.
What happens if you use the same function?
If I take U equal to V, and in that case, as I've indicated, you're not going to get the answer, zero.
But, what you will get is, so, in other words, I'm just asking, what is the sine of n t squared.
That's a case where two of them are the same.
I use the same function.
What's that?
Well, the answer is, it's the same as what you will get if you integrate the cosine, cosine squared n t dt.
And, the answer to either one of these is pi.
That's something you know how to do from 18.01 or the equivalent thereof.
You can integrate sine squared.
It's one of the things you had to learn for whatever exam you took on methods of integration.
Anyway, so I'm not going to calculate this out.
The answer turns out to be pi.
All right, now, the ways to prove it are you can use trig identities.
And, I'm asking you in one of the early problems in the problem set, identities, identities for the product of sine and cosine, expressing it in a form in which it's easy to integrate, and you can prove it that way.
Or, you can use, if you have forgotten the trigonometric identities and want to get some more exercise with complex-- you can use complex exponentials.
So, I'm asking you how to, in another part of the same problem I'm asking you how to do it, do one of these, at any rate, using complex exponentials.
And now, I'm going to use a mysterious third method another way.
I'm going to use the ODE.
I'm going to do that because this is the method.
It's not just sines and cosines which are orthogonal.
There are masses of orthogonal functions out there.
And, the way they are discovered, and the way you prove they're orthogonal is not with trig identities and complex exponentials because those only work with sines and cosines.
It is, instead, by going back to the differential equation that they solve.
And that's, therefore, the method here that I'm going to use here because this is the method which generalizes to many other differential equations other than the simple ones satisfied by sines and cosines.
But anyway, that is the source.
So, the way the proof of these orthogonality conditions goes, so I'm not going to do that.
And, I'm going to assume that m is different from n so that I'm not in either of these two cases.
What it depends on is, what's the differential equation that all these functions satisfy?
Well, it's a different differential equation depending upon the value of n, -- -- but they look at essentially the same.
These satisfy the differential equation, in other words, what they have in common.
The differential equation is, let's call it u.
It looks better.
It's going to look better if you let me call it u. u double prime plus, well, n squared, so for the function sine n t cosine n t, satisfy u double prime plus n squared times u.
In other words, the frequency is n, and therefore, this is a square of the frequency is what you put here, equals zero.
In other words, what these functions have in common is that they satisfy differential equations that look like that.
And the only thing that's allowed to vary is the frequency, which is allowed to change.
The frequency is in this coefficient of u.
Now, the remarkable thing is that's all you need to know.
The fact that they satisfy the differential equation, that's all you need to know to prove the orthogonality relationship.
Okay, let's try to do it.
Well, I need some notation.
So, I'm going to let un and vm be any two of the functions.
In other words, I'll assume m is different from n. For example, this one could be sine nt, and that could be sine of mt, or this could be sine nt and that could be cosine of mt.
You get the idea.
Any two of those in the subscript indicates whether what the n or the m is that are in that.
Any two, and I mean really two, distinct, well, if I say that m is not n, then they positively have to be different.
So, again, it's overkill with my two's-ness.
And, what I'm going to calculate, well, first of all, from the equation, I'm going to write the equation this way.
It says that u double prime is equal to minus n squared u.
That's true for any of these guys.
Of course, here, it would be v double prime is equal to minus m squared times v. You have to make those simple adjustments.
And now, what we're going to calculate is the integral from minus pi to pi of un double prime times vm dt.
Now, just bear with me.
Why am I going to do that?
I can't explain what I'm going to do that.
But you won't ask me the question in five minutes.
But the point is, this is highly un-symmetric.
The u is differentiated twice.
The v isn't.
So, those two functions-- but there is a way of turning them into an expression which looks extremely symmetric, where they are the same.
And the way to do that is I want to get rid of one of these primes here and put one on here.
The way to do that is if you want to integrate one of these guys, and differentiate this one to make them look the same, that's called integration by parts, the most important theoretical method you learned in 18.01 even though you didn't know that it was the most important theoretical method.
Okay, we're going to use it now as a basis for Fourier series.
Okay, so I'm going to integrate by parts.
Now, the first thing you do, of course, when you integrate by parts is you just do the integration.
You don't do differentiation.
So, the first thing looks like this.
And, that's to be evaluated between negative pi and pi.
In doing integration by parts between limits, minus what you get by doing both.
You do both, the integration and the differentiation.
And, again, evaluate that between limits.
Now, I'm just going to BS my way through this.
This is zero.
I don't care what the un's, which un you picked and which vm you picked.
The answer here is always going to be zero.
Instead of wasting six boards trying to write out the argument, let me wave my hands.
Okay, it's clear, for example, that a v is a sine, sine mt.
Of course it's zero because the sine vanishes at both pi and minus pi.
If the un were a cosine, after I differentiate it, it became a sine.
And so, now it's this side guy that's zero at both ends.
So, the only case in which we might have a little doubt is if this is a cosine, and after differentiation, this is also a cosine.
In other words, it might look like cosine, after, this cosine nt times cosine mt.
But, I claim that that's zero, too.
Why?
Because the cosines are even functions, and therefore, they have the same value at both ends.
So, if I subtract the value evaluated at pi, and subtract the value of minus pi, again zero because I have the same value at both ends.
So, by this entirely convincing argument, no matter what combination of sines and cosines I have here, the answer to that part will always be zero.
So, by calculation, but thought calculation; it's just a waste of time to write anything out.
You stare at it until you agree that it's so.
And now, I've taken, by this integration by parts, I've taken this highly un-symmetric expression and turned it into something in which the u and the v are treated exactly alike.
Well, good, that's nice, but why?
Why did I go to this trouble?
Okay, now we're going to use the fact that this satisfies the differential equation, in other words, that u double prime is equal to minus n, I'm sorry, I should have subscripted this.
If that's the solution, then this is equal to, times.
You have to put in a subscript otherwise.
The n wouldn't matter.
All right, I'm now going to take that expression, and evaluate it differently.
un double prime vm dt is equal to, well, un double prime, because it satisfies the differential equation is equal to that.
So, what is this?
This is minus n squared times the integral from negative pi to pi, and I'm replacing un double prime by minus n squared un.
I pulled the minus n squared out.
So, it's un here, and the other factor is vm dt.
Now, that's the proof.
Huh?
What do you mean that's the proof?
Okay, well, I'll first state it, why intuitively that's the end of the argument.
And then, I'll spell it out a little more detail, but the more detail you make for this, the more obscure it gets instead of, look, I just showed you that this is symmetric in u and v, after you massage it a little bit.
Here, I'm calculating it a different way.
Is this symmetric in u and v?
Well, the answer is yes or no.
Is this symmetric at u and v?
No.
Why?
Because of the n. The n favors u.
We have what is called a paradox.
This thing is symmetric in u and v because I can show it is.
And, it's not symmetric in u and v because I can show it is.
I can show it's not symmetric because it favors the n. Now, there's only one possible resolution of that paradox.
Both would be symmetric if what were true?
Pardon?
Negative pi.
All right, let me write it this way.
Okay, never mind.
You see, the only way this can happen is if this expression is zero.
In other words, the only way something can be both symmetric and not symmetric is if it's zero all the time.
And, that's what we're trying to prove, that this is zero.
But, instead of doing it that way, let me show you.
This is equal to that, and therefore, two things according to Euclid, two things equal to the same thing are equal to each other.
So, this equals that, which, in turn, therefore, equals what I would have gotten.
I'm just saying the symmetry of different way, what I would have gotten if I had done this calculation.
And, that turns out to be minus m squared times the integral from minus pi to pi of un vm dt.
So, these two are equal because they are both equal to this.
This is equal to that.
This equals that.
Therefore, how can this equal that unless the integral is zero?
How's that?
Remember, m is different from n. So, what this proves is, therefore, the integral from negative pi to pi of un vm dt is equal to zero, at least if m is different from n. Now, there is one case I didn't include.
Which case didn't I include?
un times un is not supposed to be zero.
So, in that case, I don't have to worry about, but there is a case that I didn't.
For example, something like the cosine of nt times the sine of nt.
Here, the m is the same as the n. Nonetheless, I am claiming that this is zero because these aren't the same function.
One is a cosine.
Why is that zero?
Can you see mentally that that's zero?
Mentally?
Well, this is trying to be in another life, it's trying to be one half the sine of two nt, right?
And obviously the integral of sine of two nt is zero between minus pi and pi because you integrate it, and it turns out to be zero.
You integrate it to a cosine, which has the same value of both ends.
Well, that was a lot of talking.
If this proof is too abstract for you, I won't ask you to reproduce it on an exam.
You can go with the proofs using trigonometric identities, and/or complex exponentials.
But, you ought to know at least one of those, and for the problem set I'm asking you to fool around a little with at least two of them.
Okay, now, what has this got to do with the problem we started with originally?
The problem is to explain this blue series.
So, our problem is, how, from this, am I going to get the terms of this blue series?
So, given f of t, two pi s a period.
Find the an and the bn.
Okay, let's focus on the an.
The bn is the same.
Once you know how to do one, you know how to do the other.
So, here's the idea.
Again, it goes back to the something you learned at the very beginning of 18.02, but I don't think it took.
But maybe some of you will recognize it.
So, what I'm going to do is write it.
Here's the term we're looking for here, this one.
Okay, and there are others.
It's an infinite series that goes on forever.
And now, to make the argument, I've got to put it one more term here.
So, I'm going to put in ak cosine kt.
I don't mean to imply that that k could be more than n, in which case I should have written it here.
I could have also used equally well bk sine kt here, and I could have put it there.
This is just some other term.
This is the an, and this is the one we want.
And, this is some other term.
Okay, all right, now, what you do is, to get the an, what you do is you multiply everything through by, you focus on the one you want, so it's dot, dot, dot, dot, dot, and you multiply by cosine nt.
So, it's ak cosine kt times cosine nt.
Of course, that gets multiplied, too.
But, the one we want also gets multiplied, an.
And, it becomes, when I multiply by cosine nt, cosine squared nt, and now, I hope you can see what's going to happen.
Now, oops, I didn't multiply the f of t, sorry.
It's the oldest trick in the book.
I now integrate everything from minus, so I don't endlessly recopy.
I'll integrate by putting it up in yellow chalk, and you are left to your own devices.
This is definitely a colored pen type of course.
Okay, so, you want to integrate from minus pi to pi?
Good.
Just integrate everything on the right hand side, also, from minus pi to pi.
Plus, these are the guys just to indicate that I haven't, they are out there, too.
And now, what happens?
What's this?
Zero.
Every term is zero because of the orthogonality relations.
They are all of the form, a constant times cosine nt times something different from cosine nt, sine kt, cosine kt, or even that constant term.
All of the other terms are zero, and the only one which survives is this one.
And, what's its value?
The integral from minus pi to pi of cosine squared, I put that up somewhere.
It's right here, down there?
It is pi.
So, this term turns into an pi, an, dragged along, but this, the integral of the square of the cosine turns out to be pi.
And so, the end result is that we get a formula for an.
What is an?
an is, well, an times pi, all these terms of zero, and nothing is left but this left-hand side.
And therefore, an times pi is the integral from negative pi to pi of f of t times cosine nt dt.
But, that's an times pi.
Therefore, if I want just an, I have to divide it by pi.
And, that's the formula for the coefficient an.
The argument is exactly the same if you want bn, but I will write it down for the sake of completeness, as they say, and to give you a chance to digest what I've done, you know, 30 seconds to digest it.
Sine nt dt.
And, that's because the argument is the same.
And, the integral of sine squared nt is also pi.
So, there's no difference there.
Now, there's only one little caution.
It have to be a little careful.
This is n one, two, and so on, and this is also n one, two, and unfortunately, the constant term is a slight exception.
We better look at that specifically because if you forget it, you can get them to gross, gross, gross errors.
How about the constant term?
Suppose I repeat the argument for that in miniature.
There is a constant term plus other stuff, a typical other stuff, an cosine, let's say.
How am I going to get that constant term?
Well, if you think of this as sort of like a constant times, the reason is the constant is because it's being multiplied by cosine zero t. So, that suggests I should multiply by one.
In other words, what I should do is simply integrate this from negative pi to pi, f of t dt.
What's the answer?
Well, this integrated from minus pi to pi is how much?
It's c zero times two pi, right?
And, the other terms all give me zero.
Every other term is zero because if you integrate cosine nt or sine nt over a complete period, you always get zero.
There is as much area above the axis or below.
Or, you can look at two special cases.
Anyway, you always get zero.
It's the same thing with sine here.
So, the answer is that c zero is equal to, is a little special.
You don't just put n equals zero here because then you would lose a factor of two.
So, c zero should be one over two pi times this integral.
Now, there are two kinds of people in the world, the ones who learn two separate formulas, and the ones who just learn two separate notations.
So, what most people do is they say, look, I want this to be always the formula for a zero.
That means, even when n is zero, I want this to be the formula.
Well, then you are not going to get the right leading term.
Instead of getting c zero, you're going to get twice it, and therefore, the formula is, the Fourier series, therefore, isn't written this way.
It's written-- If you want an a zero there, calculate it by this formula.
Then, you've got to write not c zero, but a zero over two.
I think you will be happiest if I have to give you advice.
I think you'll be happiest remembering a single formula for the an's and bn's, in which case you have to remember that the constant leading term is a zero over two if you insist on using that formula.
Otherwise, you have to learn a special formula for the leading coefficient, namely one over two pi instead of one over pi.
Well, am I really going to calculate a Fourier series in four minutes?
Not very likely, but I'll give it a brave college try.
Anyway, you will be doing a great deal of it, and your book has lots and lots of examples, too many, in fact.
It ruined all the good examples by calculating them for you.
But, I will at least outline.
Do you want me to spend three minutes outlining a calculation just so you have something to work on in the next boring class you are in?
Let's see, so I'll just put a few key things on the board.
I would advise you to sit still for this.
Otherwise you're going to hack it, and take twice as long as you should, even though I knew you've been up to 3:00 in the morning doing your problem set.
Cheer up.
I got up at 6:00 to make up the new one.
So, we're even.
This should be zero here.
So, here's minus pi.
Here's pi.
Here's one, negative one.
The function starts out like that, and now to be periodic, it then has to continue on in the same way.
So, I think that's enough of its path through life to indicate how it runs.
This is a typical square-away function, sometimes it's called.
It's an odd function.
It goes equally above and below the axis.
Now, the integrals, when you calculate them, the an is going to be, okay, look, the an is going to turn out to be zero.
Let me, instead, and you will get that with a little hacking.
I'm much more worried about what you'll do with the bn's.
Also, next Monday you'll see intuitively that the an is zero, in which case you won't even bother trying to calculate it.
How about the bn, though?
Well, you see, because the function is discontinuous, so, this is my input.
My f of t is that orange discontinuous function.
The bn is going to be, I have to break it into two parts.
In the first part, the function is negative one.
And there, I will be integrating from minus pi to pi of the function, which is minus one times the sine of nt dt.
And then, there's another part, sorry, minus pi to zero.
The other part I integrate from zero to pi of what?
Well, f of t is now plus one.
And so, I simply integrate sine nt dt.
Now, each of these is a perfectly simple integral.
The only question is how you combine them.
So, this is, after you calculate it, it will be (one minus cosine n pi) all over n. And, this part will turn out to be (one minus cosine n pi) over n also.
And therefore, the answer will be two minus two cosine, two over n times, right, two minus, two times (one minus cosine n pi) over n. No, okay, now, what's this?
This is minus one if n is odd.
It's plus one if n is even.
Now, either you can work with it this way, or you can combine the two of them into a single expression.
Its minus one to the nth power takes care of both of them.
But, the way the answer is normally expressed, it would be minus two over n, two over n times, if n is even, I get zero.
If n is odd, I get two.
So, times two, if n is odd, and zero if n is even.
So, it's four over n, or it's zero, and the final series is a sum of those coefficients times the appropriate-- cosine or sine?
Sine terms because the cosine terms were all coefficients, all turned out to be zero.
I'm sorry I didn't have the chance to do that calculation in detail.
But, I think that's enough sketch for you to be able to do the rest of it yourself.
Okay, that's, so to speak, the text for today.
The Fourier series, and the Fourier expansion for f of t, so f of t, if it looks like this should be periodic, and two pi should be a period.
Sometimes people rather sloppily say periodic with period two pi, but that's a little ambiguous.
So, this period could also be pi or a half pi or something like that as well.
The an's and bn's are calculated according to these formulas.
Now, we're going to need in just a minute a consequence of those formulas, which, it's not subtle, but because there are formulas for an and bn, it follows that once you know f of t, the an's and bn's are determined.
Or, to put it another way, a function cannot have two different Fourier series.
Or, to put it yet another way, if f of t, if two functions are equal, you'll see why I write it in this rather peculiar form.
Then, the Fourier series for f is the same as the Fourier series for g. And, the reason is because if f is equal to g, then this integral with an f there is the same as the integral with a g there.
And therefore, the an's come out to be the same.
In the same way, the bn's come out to be the same.
So, the Fourier series are the same, coefficient by coefficient, for f and g. Now, my ultimate goal-- let's all put down the argument since there are formulas, since we have formulas for an and bn.
Now, a consequence of that is, well, let me first say, what I'm aiming at is you will be amazed at how long it's going to take me to get to this.
I just want to calculate the Fourier series for some rather simple periodic function.
It's going to look like this.
So, here's pi, and here's negative pi.
So, the function which just looks like t in between those two, so, it goes up to, it's a function, t, more or less, goes up to pi here, minus pi there.
But, of course, it's got to be periodic of period two pi.
Well, then, it just repeats itself after that.
After this, it just does that, and so on.
It's a little ambiguous what happens at these endpoints.
Well, let's not worry about that for the moment, and frankly, it won't really matter because the integrals don't care about what happens in individual points.
So, there's my f of t. Now, I, of course, could start doing it right away.
But, you will quickly find, if you start doing these problems and hacking around with them, that the calculations seem really quite long.
And therefore, in the first half of the period, the first half of the period I want to show you how to shorten the calculations.
And in the second half of the period, after we've done that and calculated this thing successfully, I hope, I want to show you how to remove various restrictions on these functions, how to extend the range of Fourier series.
Well, one obvious thing, for example, is suppose the function isn't periodic of period two pi.
Suppose it has some other period.
Does that mean there's no formula?
Well, of course not.
There's a formula.
But, we need to know what it is, particularly in the applications, the period is rarely two pi.
It's normally one, or something like that.
But, let's first of all, I'm sure what you will appreciate is how the calculations can get shortened.
Now, the main way of shortening them is by using evenness and oddness.
And, what I claim is this, that if f of t is an even function, remember what that means, that f of negative t is equal to f of t. Cosine is a good example, of course, cosine nt; are all these functions are even functions.
If f of t is even, then its Fourier series contains only the cosine terms.
In other words, half the calculations you don't have to do if you start with an even function.
That's what I mean by shortening the work.
There are no odd terms, or let's put it positively.
All the bn's are zero.
Now, one way of doing this would be to say, well, y to the bn zero, well, we've got formulas, and fool around with the formula for the bn, and think about a little bit, and finally decide that that has to come out to be zero.
That's not a bad way, and it would remind you of some basic facts about integration, about integrals.
Instead of doing that, I'm going to apply my little principle that if two functions are the same, then their Fourier series have to be the same.
So, the argument I'm going to give is this, so, I'm going to try to prove this statement now.
And, I'm going to use the facts on the first board to do it.
So, what is f of minus t?
Well, if that's equal to f of t, then in terms of the Fourier series, how do I get the Fourier series for f of minus t?
Well, I take the Fourier series for f of t, and substitute t equals minus t. Now, what happens when I do that?
So, the Fourier series for this looks like a zero over two plus summation what?
Well, the an cosine nt, that does not change because when I change t to negative t, the cosine nt does not change, stays the same because it's an even function.
What happens to the sine term?
Well, the sine of negative nt is equal to minus the sine of nt.
So, the other terms, the sine terms change sign.
So, all that's the result of substituting t for negative t and f of t. On the other hand, what's f of t itself?
Well, f of t itself is what happened before that.
Now it's got a plus sign because nothing was done to the series.
Well, if the function is even, then those two right hand sides are the same function.
In other words, they're like my f of t equals g of t. And therefore, the Fourier series on the left must be the same.
In other words, if these are equal, therefore, these have to be equal, too.
Now, there's no problem with the cosine terms.
They look the same.
On the other hand, the sine terms have changed sign.
Therefore, it must be the case that bn is always equal to negative bn for all n. That's the only way this series can be the same as that one.
Now, if bn is equal to negative bn, that implies that bn is zero.
Zero is the only number which is equal to its negative.
And so, by this argument, in other words, using the uniqueness of Fourier series, we conclude that if the function is even, then its Fourier series can only have cosine terms in it.
Now, you say, hey, that's obvious.
The cosine, that's just a point of logic.
But, this is a mathematics course, after all.
It's not just about calculation.
Many of you would say, yeah, of course that's obvious because cosines are even, and the sines are odd.
I say, yeah, and so why does that make it true?
Well, the cosine's even.
Plus t into minus t, and what you are proving is the converse.
The converse is obvious.
Yeah, obvious, I don't care.
If the right-hand side is the sum of the functions, well, so is the left.
But I'm saying it the other way around.
If the left is an even function, why does the right-hand side have to have only even terms in it?
And, this is the argument which makes that true.
Now, there is a further simplification because if you've got an even function, oh, by the way, of course the same thing is true for the odd, I ought to put that down, and so also, if f of t is odd, then I think one of these proofs is enough.
The other you can supply yourself.
That will imply that all the an's are zero, even including this first one, a zero, and by the same reasoning.
So, an even function uses only cosines for its Fourier expansion.
An odd function uses only sines.
Good.
But, we still have to, suppose we got an even function.
We've still got to calculate this integral.
Well, even that can be simplified.
So, the second stage of the simplification, again, assuming that we have an even or odd function, and by the way, [LAUGHTER].
Totally unauthorized.
So, if f of t is even, what we'd like to do now is simplify the integral a little.
And, there is an easy way to do that, because, look, if f of t is an even function, then so is f of t cosine nt, is also even.
Imagine, we could make little rules about an even function times an even function is an even function.
There are general rules of that type, and some of you know them, and they are very useful.
But, let's just do it ad hoc here.
If I change t to negative t here, I don't change the function because it's even.
And, I don't change the cosine because that's even.
So, if I change t to negative t, I don't change the function.
Either factor that function, and therefore I don't change the product of those two things either.
So, it's also even.
Now, what about an even function when you integrate it?
Here's a typical looking even function, let's say, something like, I don't know, wiggle, wiggle, again.
Here's our better even function.
All right, so, minus pi to pi, even, even though the t-axis is somewhat curvy.
So, there is an even function.
The point is that if you integrate an even function from negative pi to pi, I think you all know even from calculus you were taught to do this simplification.
Don't do that.
Instead, integrate from zero to pi, and double the answer.
Why should you do that?
The answer is because it's always nice to have zero as one of the limits of integration.
I trust to your experience, I don't have to sell that.
Minus pi is a particularly unpleasant lower limit of integration because you are sure to get in trouble with negative signs.
There are bound to be at least three negative signs floating around.
And, if you miss one of them, you'll get the wrong signs of answer.
The answer will have the wrong sign.
So, the way the formula from this simplifies is that an, instead of integrating from negative pi to pi, I can integrate only from zero to pi, and double the answer.
So, our better formula is this.
If the function is even, this is the formula you should use: zero to pi, f of t cosine nt dt.
Of course, I don't have to tell you what bn should be because bn will be zero.
And, in the same way, if f is odd, the same reasoning shows that bn-- of course, an will be zero this time.
But it will be bn that will be two over pi times the integral from zero to pi of f of t sine nt dt.
Maybe we'd better just a word about that since, why is that so?
If it's odd, doesn't that mean things become zero?
If you integrate an odd function like that, the integral over minus pi to pi, you get zero.
Well, but this is not an odd function.
This is an odd function, and this is an odd function.
But the product of two odd functions is an even function.
Odd times odd is even.
I said I wasn't going to give you those rules, but since this is the one which trips everybody up, maybe I'd better say it just justbecause it looks wrong.
Right, this is odd.
That's odd.
Think about it.
If I change t to negative t, this multiplies by minus one.
This multiplies by minus one.
And therefore, the product multiplies by minus one times minus one.
In other words, it multiplies by plus one.
Nothing happens, so it stays the same.
Why does nobody believe this, even though it's true?
It's because they are thinking about numbers.
Everybody knows that an odd number times an odd number is an odd number.
So, I'm not multiplying numbers here, which also I'll put them in boxes to indicate that they are not numbers.
How's that?
Brand-new invented notation.
The box means caution.
The inside is not a number, it's the word odd or even.
It's just a symbolic statement that the product of an odd function and an odd function is an even function.
Even times even is even.
What's odd times even?
Yes, it has to get equal time.
Obviously, something must come out to be odd, right.
Okay, so, now that we've got our two simplifications, we are ready to do this problem.
Instead of attacking it with the original formulas, we are going to think about it and attack it with our better formulas.
So, now we are going to calculate the Fourier series for f of t. The first thing I see, so f of t is our little thing here.
Well, first of all, what kind of function is it: odd, even, or neither?
Most functions are neither, of course.
But, fortunately in the applications, functions tend to be one or the other.
Or, they can be converted into one to the other.
Maybe if I get a chance, I'll show you a little how, or the recitations will.
So, this function is odd.
Okay, half the work just disappeared.
I don't have to calculate any an's.
They will be zero.
So, I only have to calculate bn, and I'll calculate them by my better formula.
So, it's two over pi times the integral from zero to pi, and what I have to integrate, well, now, finally you've got to integrate something.
From zero to pi, this is the function, t. So, I have to integrate t times sine of nt dt.
Okay, so this is why you learned integration by parts, one of many reasons why you learned integration by parts, so that you wouldn't have to pull out your little calculators to do this.
Okay, now, let's do it.
So, it's two over pi.
Let's solve that away so we can forget about it.
And, what's then left is just the evaluation of the integral between limits.
So, if I integrate by parts, I'll want to differentiate the t, and integrate the sign, right?
So, the first step is you don't do the differentiation.
You only do the integration.
So, that integrates to be cosine nt over n, more or less.
The only thing is, if I differentiate this, I get negative sine nt instead of, so, I want to put a negative sign in front of all this.
And, I will evaluate that between the limits, zero and pi, and then subtract what you get by doing both things, both the differentiation and the integration.
So, I subtract the integral from zero to pi.
I now differentiate the t, and integrate.
Well, I just did the integration.
That's negative cosine nt over n. You see how the negative signs pile up?
And, if this is negative pi instead of zero, it's at that point when it starts to lose heart.
You see three negative signs, and then when you substitute, you're going to have to put in still something else negative, and you just have the feeling you're going to make a mistake.
And, you will.
Okay, now all we have to do is a little evaluation.
Let's see, at the lower limit I get zero, here.
Let's right away, as two over pi.
At the lower limit, I get zero.
That's nice.
At the upper limit, I get minus pi over n times the cosine of n pi.
Now, once and for all, the cosine of n pi-- If you like to make separate steps out of everything, okay, I'll let you do it this time, -- -- but in the long run, it's good to remember that that's negative one to the n'th power The cosine of pi is minus one .
The cosine of two pi is plus one, three pi, minus one, and so on.
So, at the upper limit, we get minus pi over n, oh, I didn't finish the calculation, times the cosine of n pi, which is minus one to the n'th power.
And now, how about the other guy?
Shall we do in our heads?
Well, I can do it in my head, but I'm not so sure about your heads.
Maybe just this once we won't.
What is it?
It's plus sine nt, right?
So, I combined the two negative signs to a plus sign by putting one this way and the other one that way.
And then, if I integrate that now, it's sine nt divided by n squared, right?
And that's evaluated between zero and pi.
And of course, the sign function vanishes at both ends.
So, that part is simply zero.
And so, the final answer is that bn is equal to, well, the pi's cancel.
This minus combines with those n to make one more.
And so, the answer is two over n times minus one to the n plus first power.
And therefore, the final result is that our Fourier series, the Fourier series for f of t, that funny function is, the Fourier series is summation bn, which is two, put the two out front because it's in every term.
There's no reason to repeat it, minus one to the n plus first power over n times the sign of nt.
That's summed from one to infinity.
Let's stop and take a look at that for a second.
Does that look right?
Okay, here's our function.
Here's our function.
What's the first term of this?
When n is one, this is plus one.
So, the first term is sine t. What's the next term?
When n is two, this is negative.
So, it's minus one to the third power.
So, that's negative one over two.
So, it's minus one half sine two t, and then it obviously continues in the same way plus a third sign three t. Now, watch carefully because what I'm going to say in the next minute is the heart of Fourier series.
I've given you that visual to look at to try to reinforce this, but it's really very important, as you go to the terminal yourself and do that work, simple as it is, and pay attention now.
Now, if you think old-fashioned, i.e.
if you think taylor series, you're not going to believe this because you will say, well, let's see, these go on and on.
Obviously, it's the first term that's the important one.
That's two sine t. Now, the derivative, two sine t, sine t would exactly follow the pink curve.
Sine t would look like this.
Two sine t goes up with the wrong angle.
The first term, in other words, does this.
It's going off with the wrong slope.
Now, that's the whole point of Fourier series.
Fourier series is not trying to approximate the function at zero at the central starting point the way Taylor series do.
Fourier series tries to treat the whole interval, and approximate the function nicely over the entire interval, in this case, minus pi to pi, as well as possible.
Taylor series concentrates at this point, does it the best it can at this point.
Then it tries, with the next term, to do a little better, and then a little better.
The whole philosophy is entirely different.
Taylor series are used for analyzing what a function of looks like which you stick close to the base point.
Fourier series analyze what a function looks like over the whole interval.
And, to do that, you should therefore aim to, so the first approximation is going to look like that, going to have entirely the wrong slope.
But, the next one will subtract off something which sort of helps to fix it up.
I can't draw this.
That's why I'm sending you to the visual because the visual draws them beautifully.
And, it shows you how each successive term corrects the Fourier series, and makes the sum a little closer to what you started with.
So, the next guy would, let's see, so it's 2t.
So, I'm subtracting off, probably I'm just guessing, but I don't dare draw this.
I haven't prepared to draw it, and I know I'll get it wrong.
So, okay, your exercise.
But, it'll look better.
It'll go, maybe, something like, let's see, it has to end up... some of it gets subtracted off...
I don't know what it looks like.
When you use the visual at the computer terminal, I've asked you to use it three times on a variety of functions.
I think this is maybe even one of them.
Notice that you can set the parameter, you can set the coefficients independently.
In other words, you can go back and correct your works, improving the earlier coefficients, and it won't affect anything you did before.
But, the most vivid way to do it is to try to get, visually, by moving the slider, to try to get the very best value for the first coefficient you can, and look at the curve.
Then get the very best value for the second coefficient and see how that improves the approximation, and the third, and so on.
And, the point is, watch the approximations approaching the function nicely over the whole interval instead of concentrating all their goodness at the origin the way a Taylor series would.
Now, there is still one mathematical point left.
It's that equality sign, which is wrong.
Why is it wrong?
Well, what I'm saying is that if I add that the series, it adds up to f of t. Now, it almost does but not quite.
And, I'd better give you the rule, the theorem.
Of all the theorems in this course that aren't being proved, this is the one that would be most outside the scope of this course, the one which I would most like to prove, in fact, just because I'm a mathematician but wouldn't dare.
The theorem tells you when a Fourier series converges to the function you started with.
And, the essence of it is this.
If f is continuous, is a continuous function, let's give the point, it's confusing just to keep calling it t. If you like, call it t, but I think it would be better to call it t zero just to indicate I'm looking at a specific point.
So, if the function is continuous there, the value of f of t is equal to, the Fourier series converges, and it's equal to its Fourier series, the sum of the Fourier series at t zero.
And, the fact that I can even use the word sum means that the Fourier series converges.
In other words, when you add up all these guys, you don't go to infinity or get something which just oscillates around crazily.
They really do add up to something.
Now, if f is not continuous at t zero, this emphatically will not be the case.
It will definitely not, but by far, the kinds of discontinuities which occur in the applications are ones like in this picture, where the discontinuities are jump discontinuities.
They are almost always jump discontinuities.
And, in that case, in other words, they are isolated.
The function looks good here and here, but there's a break.
Typically, electrical engineers just don't leave a gap because they like, I don't know why.
But electrical engineer, and others of his or her ilk would draw that function like this, like a rip saw tooth.
Even those vertical lines have no meaning whatever, but they make people look happier.
So, if f has a jump discontinuity at t zero, and as I said, that's the most important kind, then f of t, then the Fourier series adds up to, converges to, it converges, and it converges to the mid point of the jump.
Let me just write it out in words like that, the midpoint of the jump.
That's the way we'll be using it in this course.
There's a notation for this, and it's in your book.
But, those of you who would be interested in such things would know it anyway.
So, let's just call it the midpoint of the jump.
So, if I ask you, to what does this converge?
In other words, this series, what this shows is that the series, I'll write it out in the abbreviated form, summation minus one to the n plus one over n sine nt, what's the sum of the series?
What is it?
Let's call this not little f of t. Let's call it capital F of t. I want to know, what's the graph of capital F of t?
Well, the initial thing is to say, well, it must be the same as the graph of the function you started with.
And, my answer is almost, but not quite.
In fact, what will its graph look like?
Well, regardless of what definition I made for the endpoints of those pink lines, this function will converge to the following.
From here to here, I'll draw it.
I won't put in minus pi's.
I'll leave that to your imagination.
So, there's a hole at the end here.
In other words, the end of the line is not included.
And, the end of this line, regardless of whether it was included to start with or not, it's not now.
And here, similarly, I start it here with a hole, and then go down parallel to the function, t, slope one.
And now, how do I fill in, so the missing places, this is the point, pi.
This is the point, negative pi, and there are similar points as I go out.
Well, since the function is continuous here, the Fourier series will converge to this orange line.
But here, there's a jump discontinuity, and therefore, the Fourier series, this function converges to the midpoint of the jump, in other words, to here.
This function, in other words, converges to this very discontinuous looking function, and rather odd how these points are, I say, but in this case, I can prove to you that it converges here by calculating it.
Look, this is the point, pi.
What happens when you plug in t equals pi?
You get everyone of these terms is zero, and therefore the sum is zero.
So, it certainly converges, and it converges to zero.
Now, that's a general theorem.
It's rather difficult to prove.
You would have to take, again, an analysis course.
But, I don't even get to it in the analysis course which I teach.
If I had another semester I'd get to it, but I can't get everything.
Anyway, we're not going to get to it this semester to your infinite relief.
But, you should know the theorem anyway.
People will expect you to know it.
Well, that was half the period, and in the remaining half, you're going to stay a long time today.
Okay, no, don't panic.
I have to extend the Fourier series.
Okay, let me give you the hurry up version indicating the two ways in which it needs to be extended.
Extension number one -- The period is not two pi, but two times, I'll keep the two just to make the formulas look as similar as possible to the old ones.
The period, let's say, instead of two pi, is two times L. Now, I think you know enough mathematics by this point to sort of, I hope you can sort of shrug and say, well, you know, isn't that just kind of like changing the units on the t-axis?
You're just stretching.
Yeah, right.
All you do is make a change of variable.
Now, should we make it nicely?
I think I'll give you the final answer, and then I'll try to decide while I'm writing it down how much I'll try to make the argument.
First of all, the main thing to get is, if the period is not pi but L, what are the natural versions of the cosine and sine to use?
Use the natural functions.
Natural has no meaning, but it's psychologically important.
In other words, what kind of function should replace that?
I'll certainly have a t here.
What do I put in front?
I'll keep the n also.
The question is, what do I fix?
What should I put here in between in order to make the thing come out, so that it has period 2L?
You probably should learn to do this formally as well as just sort of psyching it out, and taking a guess, or memorizing the answer.
If this is the t-axis, here is t and L, zero and L. What you want to do is make a change of variable to the u-axis where the axis is the same.
This is still the point.
But, L, now, on the u coordinate, has the name pi.
Now, so I'm just describing a change of variable on the axis.
What's the one that does this?
Well, when t is L, u should be pi.
So, t should be L over pi.
When u is pi, t is L, and vice versa.
How about expressing u in terms, well, then u is equal to pi over L times t. That's the backwards form of writing it, or the forward form, depending upon how you like to think of these things.
Okay, so the cosine should be pi over L times t, in order that when t be L, it should be like cosine of n pi, which is what we would have had.
So, if t is equal to L, in other words, where is this from?
What am I trying to say?
That's the function.
This one is probably a little easier to see.
Where is this one zero?
The sine functions that we used before was zero at zero pi, two pi, three pi.
Where is this one zero?
It's zero at zero.
When t is equal to L, it's zero.
When t is equal to 2L, so, this is the right thing.
So, it's zero.
It's periodic, and it's zero plus or minus L plus or minus 2L.
And, in fact, formally you can verify that it's periodic with period 2L.
So, in other words, we want a Fourier expansion to use these functions as the natural analog of what would be up there.
So, the period of our function is 2L, and the formula is, I'll give you the formula.
It's f of t equals identical summation, an, except you'll use these as the natural functions instead of cosine nt and sine nt.
So, n pi t over L plus bn, okay, I'm tired, but I'll put it in anyway, n pi t over L. Yeah, but of course, what about the formulas for an?
Somebody up there is watching over us.
Here are the formulas.
They are exactly what you would guess if somebody said produce the formulas in ten seconds, and you'd better be right, and you didn't have time to calculate.
You say, well, it must be, let's do the cosine series.
Okay, let's not do a cosine.
So, it's one over L times the integral from negative L, in other words, wherever you see an L, wherever you see a pi, just put an L times the f of t cosine, and now we'll use our new function, not the old one.
I submit that's an easy, if you know the first formula, then this would be an easy one to remember.
All you do is change pi to L everywhere.
Except, you got to remember this part.
Make it a function periodic of period 2L, not 2pi.
And similarly, bn is similar.
It looks just the same way.
And, how about, and the same even-odd business goes, too, so that if f of t, for example, is even, and has period 2L, then the function, then the best formula for the an will not be that one.
It will be two over L, and where you integrate only from zero to L, f of t cosine.
So, now, the bn's will be zero, and you'll just have positive, etc.
for L. As I say, this is important case, particularly if the period is two, in other words, if the half period is one because in the literature, frequently one is used as the standard normal reference, not pi.
Pi is convenient mathematically because it makes the cosines and sines look simple.
But, in actual calculation, it tends to be where L is one.
So, usually you have a pi here.
You don't have just nt.
Well, I should do a calculation, but instead of doing that, let me give you the other extension.
Fortunately, there are plenty of calculations in your book.
So, let me give you in the last couple of minutes the other extension.
This is going to be a very important one for us next time.
Typically, in applications, well, I mean, the first thing, periodic functions are nice, but let's face it.
Most functions aren't periodic, I have to agree.
So, all this theory is just about periodic functions?
No.
It's about functions.
Really, it's about functions where the interval on which you are interested in them is finite.
It's a finite interval, not functions which go to infinity.
For those, you will have to use Fourier transforms, Fourier transforms, not Fourier series.
But, if you are interested in a function on a finite interval, then you can use Fourier series even though the function isn't periodic because you can make it periodic.
So, what you do is, if f of t is on, let's take the interval from zero to L. That's a sample finite interval.
I can always change the variable to make the interval from zero to L. I can even make it from zero to one, but that's a little too special.
It would be a little awkward.
So, if a function is defined on a finite interval, the way to apply the Fourier series to it is make a periodic extension.
Now, since I have so little time, I'm just going to get away with murder by just drawing pictures.
So, let me give you a function.
Here's my function defined on zero to L, colored chalk if you please.
Let's make it the function t squared, and let's make L equal to one.
That function is not periodic.
If I let it go off, it would just go off to infinity and never repeat its values, except on the left-hand side.
But, I'm not even going to let it be on the left hand side.
It's only defined from zero to one as far as I'm concerned.
Okay, that function has an even periodic extension.
And, its graph looks like this extended to be an even function.
Okay, now, that means from zero to negative L, you've got to make it look exactly as it looked on the right-hand side.
Otherwise, it would be even.
And now, what do I do?
Well, now I've got, from minus L to L. So, all I'm allowed to do is keep repeating the values.
In other words, apply the theory of Fourier series to this guy, use a cosine series because it's an even function, and then everything you want to do, you say, okay, all the rest of this is garbage.
I only really care about it from here to here.
And, that's what you will plug into your differential equation on the right-hand side, just that part of it, just this part of it.
How about the odd extension?
What would that look like?
Okay, the odd extension, here I start like this.
And now, to extend it to be an odd function, I have to make it go down in exactly the same way it went up.
And, what do I do here?
I have to make it start repeating its values so it will look like this.
So, the odd extension is going to be discontinuous in this case.
And, what's the Fourier series going to converge to?
Well, in each case, to the average, to the midpoint of the jump, and the odd extension looks like this, and this will give me assigned series.
Okay, you've got lots of problems to do.
I'd like to talk.
Thank you.
One of the things I'd like to give a little insight into today is the mathematical basis for hearing.
For example, if a musical tone, a pure musical tone would consist of a pure oscillation in terms of the vibration of the air.
It would be a pure oscillation.
So, [SINGS], and if you superimpose upon that, suppose you sing a triad, [SINGS], those are three tones.
Each has its own period of oscillation, and then another one, which is the top one, which is even faster.
The higher it is, the faster the thing.
Anyway, what you hear, then, is the sum of those things.
So, C plus E plus G, let's say, what you hear is the wave form.
It's periodics, still, but it's a mess.
I don't know, I can't draw it.
So, this is periodic, but a mess, some sort of mess.
Now, of course, if you hear the three tones together, most people, if they are not tone deaf, anyway, can hear the three tones that make up that.
So, in other words, if this is the function which is the sum of those three, some sort of messy function, f of t, you're able to do Fourier analysis on it, and break it up.
You're able to take that f of t, and somehow mentally express it as the sum of three pure oscillations.
That's Fourier analysis.
We've been doing it with an infinite series, but it's okay.
It's still Fourier analysis if you do it with just three.
So, in other words, the f of t is going to be the sum of, let's say, sine, I don't know, it's going to be the sign of one frequency plus the sine of another frequency plus the sine of a third, maybe with coefficients here.
So, somehow, since you were born, you have been able to take the f of t, and express it as the sum of the three signs.
And, here, therefore, the three tones that make up the triad.
Now, the question is, how did you do that Fourier analysis?
In other words, does your brain have a little integrator in it, which calculates the coefficients of that series?
Of course, the answer is no.
It has to do something else.
So, one of the things I'd like to aim at in this lecture is just briefly explaining what, in fact, actually happens to do that.
Now, to do that, we'll have to make some little detours, as always.
So, first I'm going to, throughout the lecture, in fact, I gave you last time a couple of shortcuts for calculating Fourier series based on evenness and oddness, and also some expansion of the idea of Fourier series where we use the different, but things didn't have to be periodic or period two pi, but it can have an arbitrary period, 2L, and we could still get a Fourier expansion for it.
Let me, therefore, begin just as a problem, another type of shortcut exercise, to do a Fourier calculation, which we are going to be later in the period to explain the music problem.
So, let's suppose we're starting with the function, f of t, which is a real square wave, and I'll make its period different from the one, not two pi.
So, suppose we had a function like this.
So, this is one, and this is one.
So, the height is one, and this point is one as well.
And then, it's periodic ever after that.
I'll tell you what, let's do like the electrical engineers do and put these vertical lines there even though they don't exist.
Okay, so the height is one and it goes over.
The half period is one.
This really is a square wave.
I mean, it's really a square, not what they usually call a square wave.
So, my question is, what's its Fourier series?
Well, it's neither even nor odd.
That's a little dismaying.
It sounds like we're going to have to calculate an's and bn's.
So, the shortcuts I gave you last time don't seem to be applicable.
Now, of course, nor is the period two pi, but that shouldn't be too bad.
In fact, you ought to look for an expansion in terms of things that look like sine of n, well, what should it be?
Since L is equal to one, the half period is equal to one.
Remember, the period is 2L, not L. It's n pi over L, but if L is one, we should be looking for an expansion in terms of functions that look like this.
Now, since we've already done the work for the official square way, which looks something like this, what you always try to do is reduce these things to problems that you've already solved.
This is a legitimate one, since I solved it in lecture for you.
So, we can consider it as something we know.
So, I observed that since I am very lazy, that if I lower this function by one half, it will become an odd function.
Now it's an odd function.
Okay, I just cut the work in half.
So, let's call this function, let's call this, I don't know, S of t. The green one is the one we wanted to start with.
So, f of t is a green function.
But, I can improve things even more because the function that we calculated in the lecture is a lot like this salmon function.
That's why I called it S. But, the difference is that the function we calculated with this one.
In the first place, it went down further.
It went not to negative one half, which is where that one goes.
But, it went down to negative one, and then went up here to plus one.
And, it went over to pi.
So, it came down again, but not, but at the point, pi.
And here, negative pi went up again.
Okay, let me remind you what this one was.
Suppose we call it, O doesn't look good, I don't know, how about g of u?
Let's, for a secret reason, call the variable u this time, okay?
So, the previous knowledge that I'm relying on was that I derived the Fourier series for you by an orthodox calculation.
And, it's not too hard to do because this is an odd function.
And therefore, you only have to calculate the bn's.
And, half of them turn out to be zero, although you don't know that in advance.
But anyway, the answer was four over pi times the sum of just the odd ones, the sine of n u, and that you had to divide by n. So, this is the expansion of g, this function, g of u, the Fourier expansion of this function.
Since it's an odd function, it only involves the signs.
There's no funny stuff here because the period is now two pi.
And, this came from the first lecture on Fourier series, or from the book, wherever you want it, or solutions to the notes.
There are lots of sources for that.
The solution's in the notes.
Okay, now, that looks so much like the salmon function, --- -- I ought to be able to convert one into the other.
Now, I will do that by shrinking the axis.
But, since this can get rather confusing, what I'll do is overlay this.
What I prefer to do is I think u, okay, I'm changing, I'm keeping the thing the same.
But, I'm going to change the name of the variable, the t, in such a way that on the t-axis, this becomes the point, one.
If I do that, then this function will turn exactly into that one, except it will go not from minus a half to a half, but it will go from negative one to one, since I haven't done anything to the vertical axis.
So, how I do that?
What's the relation between u and t?
Well, u is equal to pi times t, or the other way around.
You know that it's going to be approximately this.
Try one, and then check that it works.
When t is equal to one, u is pi, which is what it's supposed to be.
So, this is the relation between the two.
And therefore, without further ado, I can say that, let's write the relation between them.
f of t is what I want.
Well, what's f of t if I subtract one half of that?
So, that's going to be equal to the salmon function plus one half, right, or the salmon function is f of t lowered by one half.
One thing is the same as the other.
And, what's the relation between this salmon function and the orange function?
Well, the salmon function is, so, let's convert, so, S of t -- it's more convenient, as I wrote the formula g of u.
Let's start it from that end.
If I start from g of u, what do I have to do to convert it into S of-- into the salmon function?
Well, take one half of it.
So, if I put them all together, the conclusion is that f of t is equal to one half plus S of t, which is one half of g of u, but u is pi t. So, it's four pi, four over pi times the sum of the sine of n. And, for u, I will write pi t divided by n. And, sorry, I forgot to say that sum is only over the odd values of n, not all values of n. So, the sum over n odd of that, and, of course, the two will cancel that.
So, here we have, in other words, just by this business of shrinking or just stretching or shrinking the axis, lowering it and squishing it that way a little bit.
We get from this Fourier series, we get that one just by this geometric procedure.
I'd like you to be able to do that because it saves a lot of time.
Okay, so let's put this answer up in, I'm going to need it in a minute, but I don't really want to recopy it.
So, let me handle it by erasing.
So, let's call that plus two over pi, and there is our formula for that green function that we wrote before.
So, I'll put that in green.
So, we'll have a color-coded lecture again.
Now, what we're going to be doing ultimately, to getting at the music problem that I posed at the beginning of the lecture, is we want to solve, and this is what a study of Fourier series has been aiming at, to solve second-order linear equations with constant coefficients were the right-hand side was a more general function than the kind we've been handling.
So, now, in order to simplify, and we don't have a lot of time in the course, I'd have to take another day to make more complicated calculations, which I don't want to do since you will learn a lot from them, anyway.
I think you will find you've had enough calculation by the time Friday morning rolls around.
So, let's look at the undamped case, which is simpler, or undamped spring, or undamped anything because it doesn't have that extra term, which requires extra calculations.
So, I'll follow the book now and some of the notes and the visuals, and called the independent variable-- the dependent variable I'm going to call x now.
And, the independent variable is, as usual, time.
So, this is going to be, in general, f of t, and I'm going to use it by calculating example, this is the actual f of t I'm going to be using.
But, the general problem for a general f of t is to solve this, or at least to find a particular solution.
That's what most of the work is, because we already know how from that to get the general solution by adding the solution to the reduced equation, the associated homogeneous equation.
So, all our work has been, this past couple of weeks, in how you find a particular solution.
Now, the case in which we know what to do is, so we can find our particular solution.
Let's call that x sub p. We could find x sub p if the right hand side is cosine omega, well, in general, an exponential, but since we are not going to use complex exponentials today, all these things are real.
And I'd like to keep them real.
If it's either cosine omega t or sine omega t, or some multiple of that by linearity, it's just as good.
We already know how to find the thing, and to find a particular solution.
So, the procedure is use complex exponentials, and that magic formula I gave you.
But, right now, just to save a little time, since I already did that on the lecture on resonance, I solved it explicitly for that, and you've had adequate practice I think in the problem sets.
Let's simply write down the answer that comes out of that.
The answer for the particular solution is cosine omega t or sine omega t. That's the top.
And, it's over a constant.
And, the constant is omega naught squared.
That's the natural frequency which comes from the system, minus the imposed frequency, the driving frequency that the system, the spring or whatever it is, undamped spring, is being driven with.
Okay, understand the notation.
Cosine this over that, or sine, depending on whether you started driving it with cosine or sine.
So, this is from the lecture, if you like, from the lecture on resonance, but again it's, I hope by now, a familiar fact.
Let me remind you what this had to do with resonance.
Then, the observation was that if omega, the driving frequency is very close to the natural frequency, then this is close to that.
The denominator is almost zero, and that makes the amplitude of the response very, very large.
And, that was the phenomenon of resonance.
Okay, now what I'd like to do is apply those formulas to finding out what happens for a general f(t), or in particular this one.
So, in general, I'll keep using the notation, f of t, even though I've sorted used it for that.
But in general, what's the situation?
If f of t is a sine series, cosine series, all right, let's do everything.
Suppose it's, in other words, the procedure is, take your f of t, expand it in a Fourier series.
Well, doesn't that assume it's periodic?
Yes, sort of.
So, suppose it's a Fourier series.
I'll make a very general Fourier series, write it this way: cosine (omega)n t, and then the sine terms, sine (omega)n t from one to infinity where the omegas are, omega n is short for that.
Well, it's going to have the n in it, of course, but I want, now, to make the general period to be 2L.
So, it would be n pi over L. Of course, if L is equal to one, then it's n pi.
Or, if L equals pi, those are the two most popular cases, by far.
Then, it's simply n itself, the driving frequency.
But, this would be the general case, n pi over L if the period is the period of f of t is 2L.
So, that's what the Fourier series looks like.
Okay, then the particular solution will be what?
Well, I got these formulas.
In other words, what I'm using is superposition principle.
If it's just this, then I know what the answer is for the particular solution, the response.
So, if you make a sum of these things, a sum of these inputs, you are going to get a sum of the responses by superposition.
So, let's write out the ones we are absolutely certain of.
What's the response to here?
Well, it's (a)n cosine omega n t. The only thing is, now it's divided by omega naught squared.
This constant has changed, and the same thing here.
Of course, by linearity, if this is multiplied by a, then the answer is multiplied by, the response is also multiplied by a.
So, the same thing happens here.
Here, it's (b)n and over, again, omega naught squared minus omega times the sine of omega t. So, in other words, as soon as you have the Fourier expansion, the Fourier series for the input, you automatically get this by just writing it down the Fourier series for the response.
That's the fundamental idea of Fourier series, at least applied in this context.
They have many other contexts, approximations, so on and so forth.
But, that's the idea here.
All right, what about that constant term?
Well, this formula still works if omega equals zero.
If omega equals zero, then this is the constant, one.
The formula is still correct.
Omega is zero here.
The only thing you have to remember is that the original thing is written in this form.
So, the response will be, what will it be?
Well, it's one divided by omega naught squared, if I'm in the case omega zero is equal to zero.
So, it's a zero divided by two omega naught squared.
And, as you will see, it looks just like the others.
You're just taking omega, and making it equal to zero for that particular case.
Sorry, this should be omega n's all the way through here.
All right, well, let's apply this to the green function.
So, what have we got?
We have its Fourier series.
So, if the green function is, if the input in other words is this square wave, the green square wave, so in your notes, this guy, this particular f of t is the input.
And, the equation is x double prime plus omega naught squared x equals f of t. Then, the response is, well, I can't draw you a picture of the response because I don't know what the Fourier series actually looks like.
But, let's at least write down what the Fourier series is.
The Fourier series will be, well, what is it?
It's one half.
The constant out front is one half, except it's one over two omega naught squared.
So, this is my function, f of t. That's the general formula for how the input is related to the response.
And, I'm applying it to this particular function, f of t. And, the answer is plus.
Well, my Fourier series involves only odd sums, only the summation over odd, and only of the sign.
So, it is going to be two over pi, sorry, so it's going to be two over pi out front.
That constant will carry along by linearity.
And, I'm going to sum over odd, n odd values only.
The basic thing in the upstairs is going to be the sine of omega n t. But, what is (omega)n?
Well, (omega)n is n pi.
So, it's n pi t. And, how about the bottom?
The bottom is going to be omega naught squared minus omega n squared.
And, this is my (omega)n, minus n pi squared.
What's that?
Well, I don't know.
All I could do would be to calculate it.
You could put it on MATLAB and ask MATLAB to calculate and plot for you the first few terms, and get some vague idea of what it looks like.
That's nice, but it's not what's interesting to do.
What's interesting to do is to look at the size of the coefficients.
And, again, rather than do it in the abstract, let's take a specific value.
Let's suppose that the natural frequency of the system, in other words, the frequency at which that little spring wants to go vibrate back and forth, whatever you got vibrating.
Let's suppose the natural frequency that's omega naught is ten for the sake of definiteness, as they say.
Okay, if that's ten, all I want to do is calculate in the crudest possible way what a few of these terms are.
So, the response is, so let's see, we've got to give that a name.
The response is (x)p of t. What's (x)p of t?
I'm just going to calculate it very approximately.
This means, you know, throwing caution to the winds because I don't have a calculator with me.
And, I want you to look at this thing without a calculator.
The first term is one over 200.
Okay, that's the only term I can get exactly right.
[LAUGHTER] Or, I could if I could calculate.
I suppose it's 0.005, right?
That's the constant term.
Okay, so the next term, let's see, two over pi is two thirds.
I'll keep that in mind, right?
Plus two thirds, 0.6, let's say, that's an indication of the accuracy with which these things are going to be performed.
I think in Texas for a long while, the legislature declared pi to be three, anyways.
One of those states did it to save calculation time.
I'm not kidding, by the way.
All right, so what's the first term?
If n equals one, I have the sine of pi t. That's the n equals one term.
What's the denominator like?
That's about 100 minus 9 squared.
Let's say it's 91, sine t over 91.
What's the next term?
Sine of three pi t, remember, I am omitting, I'm only using the odd values of n because those are the only ones that enter into the Fourier expansion for this function, which is at the bottom of everything.
All right, what's the sine three pi t?
Well, now, I've got 100 minus three pi, -- -- that's 9 squared is 81.
So, no, what am I doing?
So, we have 100 minus three times pi is 9, squared.
Well, let's say a little more.
Let's say 85.
So, that's 15.
How bout the next one?
Well, it's sine 5 pi t. I think I'll stop here as soon as we do this one because at this point it's clear what's happening.
This is 100 squared minus, that's 15 squared is 225, so that's about 125 with a negative sign.
So, minus this divided by 125.
And, after this they are going to get really quite small because the next one will be seven pi squared.
That's 400, and this is becoming negligible.
So, what's happening?
So, it's approximately, in other words, 0.005 plus the next coefficient is, let's see, 6/10, let's say 100, sine pi t. And, what comes next?
Well, it's now 1/20th.
It's about a 20th.
Let's call that 0.005 sine three pi t, and now so small, minus 0.01, let's say times this last one, sine 5 pi t. What you find, in other words, is that the frequencies which make up the response do not occur with the same amplitude.
What happens is that this amplitude is roughly five times larger than any of the neighboring ones.
And after that, it's a lot larger than the ones that come later.
In other words, the main frequency which occurs in the response is the frequency three pi.
What's happened is, in other words, near resonance has occurred.
So, if omega is ten, very near resonance, that is, it's not too close, but it's not too far away either, occurs for the frequency three pi in the input.
Now, where's the frequency three pi in the input?
It isn't there.
It's just that green thing.
Where in that is the frequency three pi?
I can't answer that for you, but that's the function of Fourier series, to say that you can decompose that green function into a sum of frequencies, as it were, and the Fourier coefficients tell you how much frequency goes into each of those f of t's.
Now, so, f of t is decomposed into the sum of frequencies by the Fourier analysis.
But, the system isn't going to respond equally to all those frequencies.
It's going to pick out and favor the one which is closest to its natural frequency.
So, what's happened, these frequencies, the frequencies and their relative importance in f of t are hidden, as it were.
They're hidden because we can't see them unless you do the Fourier analysis, and look at the size of the coefficients.
But, the system can pick out.
The system picks out and favors, picks out for resonance, or resonates with, resonates with the frequencies closest to its natural frequency.
Well, suppose the system had natural frequency, not ten.
This is a put up job.
Suppose it had natural frequency five.
Well, in that case, none of them are close to the hidden frequencies in f of t, and there would be no resonance.
But, because of the particular value I gave here, I gave the value ten, it's able to pick out n equals three as the most important, the corresponding three pi as the most important frequency in the input, and respond to that.
Okay, so this is the way we hear, give or take a few thousand pages.
So, what does the ear do?
How does the ear, so, it's got that thing, messy curve, which I erased, which has a secret, which just has three hidden frequencies.
Okay, from now on I hand wave, right, like they do in other subjects.
So, we got our frequency.
So, it's got a [SINGS].
That's one frequency.
[SINGS] And, what goes in there is the sum of those three, and the ear has to do something to say out of all the frequencies in the world, I'm going to respond to that one, that one, and that one, and send a signal to the brain, which the brain, then, will interpret as a beautiful triad.
Okay, so what happens is that the ear, I don't talk physiology, and I never will again.
I know nothing about it, but anyway, the ear, when you get far enough in there, there are little three bones, bang, bang, bang; this is the eardrum, and then there's the part which has wax.
Then, there's the eardrum which vibrates, at least if there is not too much wax in your ear.
And then, the vibrations go through three little bones which send the vibrations to the inner ear, which nobody ever sees.
And, the inner ear, then, is filled with thick fluid and a membrane, and the last bone hits up against the membrane, and the membrane vibrates.
And, that makes the fluid vibrate.
Okay, good.
So, it's vibrating according to the function f of t. Well, what then?
Well, that's the marvelous part.
It's almost impossible to believe, but there is this, sort of like a snail thing inside.
I've forgotten the name.
It's cochlea.
And, it has these hairs.
They are not hairs really.
I don't know what else to call them.
They're not hairs.
But, there are things so long, you know, they stick up.
And, there are 20,000 of them.
And, they are of different lengths.
And, each one is tuned to a certain frequency.
Each one has a certain natural frequency, and they are all different, and they are all graded, just like a bunch of organ pipes.
And, when that complicated wave hits, the complicated wave hits, each one resonates to a hidden frequency in the wave, which is closest to its natural frequency.
Now, most of them won't be resonating at all.
Only the ones close to the frequency [SINGS], they'll resonate, and the nearby guys will resonate, too, because they will be nearby, almost have the same natural frequency.
And, over here, there will be a few which resonate to [SINGS], and finally over here a few which go [SINGS], and each of those little hairs, little groups of hairs will signal, send that signal to the auditory nerve somehow or other, which will then carry these three inputs to the brain, and the brain, then, will interpret that as you are hearing [SINGS].
So, the Fourier analysis is done by resonance.
You here resonance because each of these things has a certain natural frequency which is able, then, to pick out a resonant frequency in the input.
I'd like to finish our work on Fourier series.
So, for homework I'm asking you to do something similar.
Taken an input.
I gave you a frequency here, a different omega naught, a different input, as you by means of this Fourier analysis to find out which it will resonate, which of the hidden frequencies in the input the system will resonate to, just so you can work it out yourself and do it.
Now, I'd like to first try to match up what I just did by this formula with what's in your book, since your book handles the identical problem but a little differently, and it's essentially the same.
But I think I'd better say something about it.
So, the book's method, and to the extent which any of these problems are worked out in the notes, the notes do this, too.
Use substitution.
Base uses differentiation of Fourier series term by term.
The work is almost exactly the same as here.
And, it has a slight advantage, that it allows you, the book's method has a slight advantage that it allows you to forget this formula.
You don't have to know this formula.
It will come out in the wash. Now, for some of you, that may be of colossal importance, in which case, by all means, use the book's method, term by term.
So, it requires no knowledge of this formula because after all, I base this solution, I simply wrote down the solution and I based it on the fact that I was able to write down immediately the solution to this and put as being that response.
And for that, I had to remember it, or be willing to use complex exponentials quickly to remind myself.
There's very, very little difference between the two.
Even if you have to re-derive that formula, the two take almost about the same length of time.
But anyway, the idea is simply this.
With the book, you assume.
In other words, you take your function, f of t. You expand it in a Fourier series.
Of course, which signs and cosines you use will depend upon what the period is.
So, you assume the solution of the form-- Well, if I, for example, carried out in this particular case, I don't know if I will do all the work, but it would be natural to assume a solution of the form, since the input looks like the green guy.
Assume a solution which looks the same.
In other words, it will have a constant term because the input does.
But all the rest of the terms will be sines.
So, it will be something like (c)n times the sine of n pi t. The only question is, what are the (c)n's?
Well, I found one method up there.
But, the general method is just plug-in.
Substitute into the ODE.
Substitute into the ODE.
You differentiate this twice to do it.
So, I'll do the double differentiation and I won't stop the lecture there, but I will stop the calculation there because it has nothing new to offer.
And, this is the way all the calculations in the books and the solutions and the notes are carried out.
So, I don't think you'll have any trouble.
Well, this term vanishes.
This term becomes what?
If I differentiate this twice, I get summation, so, this is one to infinity because I don't know which of these are actually going to appear.
Summation one to infinity, (c)n times, well, if you differentiate the sine twice, you get negative sine, right?
Do it once: you get cosine.
Second time: you get negative sine.
But, each time you will get this extra factor n pi from the chain rule.
And so, the answer will be negative (c)n times n pi squared times the sine of n pi t. And so, the procedure is, very simply, you substitute (x)p double prime into the differential equation.
In other words, if you do it, we will multiply this by omega naught squared.
And, you add them.
And then, on the left-hand side, you are going to get a sum of terms, sine n pi t times coefficients involving the (c)n's.
And, on the right, so, you're going to get a sum involving the (c)n's, and the sines n pi t, and on the right, you're going to get the Fourier series for f of t, which is exactly the same kind of expression.
The only difference is, now the sines have come with definite coefficients.
And then, you simply click the coefficients on the left and the coefficients on the right, and figure out what the (c)n's are.
So, by equating coefficients, you get the (c)n's.
Would you like me to carry it out?
Yeah, okay, I was going to do something else, but I wouldn't have time to do it anyway.
So, why don't I take two minutes to complete the calculation just so you can see you get the same answer?
All right, what do we get?
If you add them up, you get c naught, out front, plus (c)n is multiplied by what?
Well, from the top it's multiplied by omega naught squared.
On the bottom, it's multiplied by n pi squared.
Ah-ha, where have I seen that combination?
The sum is equal to, sorry, one half plus what is it, sum over n odd of sine n pi t over n. So, the conclusion is that-- I'm sorry, it should be c naught times omega naught squared.
So, what's the conclusion?
If c zero is one over two omega naught squared, and that (c)n, only for n odd, the others will be even.
The others will be zero.
The (c)n is going to be equal to two over pi here.
So, it's going to be two pi, two over pi times one over n times one over omega naught squared minus n over pi squared.
This is terrible, which is the same answer we got before, I hope.
Did I cover it up?
Same answer.
So, that answer at the left-hand end of the board is the same one.
I've calculated, in other words, what the c zeros are.
And, I got the same answer as before.
Today, and for the next two weeks, we are going to be studying what, for many engineers and a few scientists is the most popular method of solving any differential equation of the kind that they happen to be, and that is to use the popular machine called the Laplace transform.
Now, you will get proficient in using it by the end of the two weeks.
But, there is always a certain amount of mystery that hangs around it.
People scratch their heads and can't figure out where it comes from.
And, that bothers them a lot.
In the past, I've usually promised to tell you, the students at the end of the two weeks, but I almost never have time.
So, I'm going to break that glorious tradition and tell you up front at the beginning, where it comes from, and then talk very fast for the rest of the period.
Okay, a good way of thinking of where the Laplace transform comes from, and a way which I think dispels some of its mystery is by thinking of power series.
I think virtually all of you have studied power series except possibly a few students who just had 18.01 here last semester, and probably shouldn't be taking 18.03 anyway, now.
But anyway, a power series looks like this: summation (a)n x to the n. And, you sum that from, let's say, zero to infinity.
And, the typical thing you want to do with it is add it up to find out what its sum is.
Now, the only way I will depart from tradition, instead of calling the sum some generic name like f of x, in order to identify the sum with the coefficients, a, I'll call it a of x.
Now, I want to make just one slight change in that.
I want to use computer notation, which doesn't use the subscript (a)n. Instead, this, it thinks of as a function of the discreet variable, n. In other words, it's a function which assigns to n equals zero, one, two, three real numbers.
That's what this sequence of coefficients really is.
So, the computer notation will look almost the same.
It's just that I will write this in functional notation as a of n instead of (a)n. But, it still means the real number associated with the positive integer, n, and everything else is the same.
See, what I'm thinking of this as doing is taking this discreet function, which gives me the sequence of coefficients of the power series, and associating that with the sum of the power series.
Let me give you some very simple examples, two very simple examples, which I think you know.
Suppose this is a function one.
Now, what do I mean by that?
I mean it's the constant function, one.
To every positive integer, it assigns the number one.
Okay, what's a of x?
What I'm saying is, in other words, in this fancy, mystifying form, is all of these guys are one, what's a of x?
One plus x plus x squared plus x cubed.
Look, you are supposed to be born knowing what that adds up to.
It adds up to one over one minus x, except that's the wrong answer.
What's wrong about it?
It's not true for every value of x.
That's only true when x is such that that series converges, and that is only true when x lies between negative one and one.
So, it's not this function.
It's this function with its domain restricted to be less than one in absolute value.
What does that converge to?
If x is bigger than one, the answer is it doesn't converge.
There's nothing else you can put here.
Okay, let's take another function.
Suppose this is, let's see, one over n you probably won't know.
Let's take one you will know, one over n factorial.
Suppose a of n is the function one over n factorial, what's a of x?
So, what I'm asking is, what does this add up to when the coefficient here is one over n factorial?
What's summation x to the n over n factorial?
It is e to the x.
And, this doesn't have to be qualified because this is true for all values of x.
So, in other words, from this peculiar point of view, I think of a power as summing the operation, of summing a power series as taking a discreet function defined for positive integers, or nonnegative integers, and doing this funny process.
And, out of it comes a continuous function of some sort.
And, notice what goes in is the variable, n. But, what comes out is the variable, x.
Well, that's perfectly natural.
That's the way a power series is set up.
So, the question I ask is, this is a discreet situation, a discreet summation.
Suppose I made the summation continuous instead of discreet.
So, I want the continuous analog of what I did over there.
Okay, what would a continuous analog be?
Well, instead of, I'll replace n zero, one, two, that will be replaced by a continued, that's a discreet variable.
I'll replace it by a continuous variable, t, which runs from zero to infinity, and is allowed to take every real value in between instead of being only allowed to take the values of the positive nonnegative integers.
Okay, well, if I want to use t instead of n, I clearly cannot sum in the usual way over all real numbers.
But, the way the procedure which replaces summation over all real numbers is integration.
So, what I'm going to do is replace that sum by the integral from zero to infinity.
That's like the sum from zero to infinity of what?
Well, of some function, but now n is being replaced by the continuous variable, t. So, this is going to be a function of t. And, how about the rest of it?
The rest I will just copy, x to the n'th.
Well, instead of n I have to write t and dt.
And, what's the sum?
Well, I'll call the sum, what's the sum a function of?
I integrate out the t. So, that doesn't appear in the answer.
All that appears is this number, x, this parameter, x.
For each value of x, like one, two, or 26.3, this integral has a certain value, and I can calculate it.
So, this is going to end up as a function of x, just as it did before.
Now, I could leave it in that form, but no mathematician would like to do that, and very few engineers either.
The reason is, in general, when you do integration and differentiation, you do not want to have as the base of an exponential something like x.
The only convenient thing to have is e, and the reason is because it's only e that people really like to differentiate, e to the something.
The only thing is that people really like to differentiate or integrate.
So, I'm going to make this look a little better by converting x to the t to the base e. I remember how to do that.
You write x equals e to the log x and so x to the t will be e to the log x times t, if you want.
Now, the only problem is I want to make one more little change.
After all, I want to be able to calculate this integral.
And, it's clear that if t is going to infinity, if I have a number here, for example, like x equals two, that integral is really quite unlikely to converge.
For example, if a of t were just the constant function, one, the integral certainly wouldn't converge.
It would be horrible.
That integral only has a chance of converging if x is a number less than one, so that when I take bigger and bigger powers of it, I get smaller and smaller numbers.
Don't forget, this is an improper integral going all the way up to infinity.
Those need treatment, delicate handling.
All right, so I really want x to be less than one.
Otherwise, that integral is very unlikely to converge.
I'd better have it positive, because if I allow it to be negative I'm going to get into trouble with negative powers, see what's minus one, for example, to the one half when t is one half.
That's already imaginary.
I don't want that.
If you've got an exponential, the base has got to be a positive number.
So, I want x to be a positive number.
All right, if x in my actual practices going to lie between zero and one in order to make the integral converge, how about log x?
Well, log x, if x is less than one, so log x is going to be less than zero, and it's going to go all the way down to negative infinity.
So, this means log x is negative.
In this interesting range of x, the log x is always going to be negative.
And now, I don't like that.
The first place I'd like to call this by a new variable since no one uses log x as a variable.
And, it would make sense to make it a negative, to make it negative, that is, to write log x is equal to negative s. Let's put it on the other side, in order that since log x is always going to be less than zero, then s will always be positive.
And it's always more convenient to work with positive numbers instead of negative numbers.
So, if I make those changes, what happens to the integral?
Well, I stress, all these changes are just cosmetic to make things a little easier to work with in terms of symbols.
First of all, the a I'm going to change.
I don't want to call it a of t because most people don't call functions a of t. They call them f of t. So, I'll call it f of t. x is e to the log x, which is e to the minus s. So, x has its name changed to e to the minus s. In other words, I'm using as the new variable not x any longer but s in order that the base be e. t, I now raise this to the t'th power, but by the laws of exponents, that means I simply multiply the exponent by t, and dt.
And now, since I'm calling the function f of t, the output ought to be called capital F. But it's now a function, since I've changed the variable, of s. It's no longer a function of x.
If you like, you may think of this as a of, what's x?
x is e to the negative s, I guess.
I mean, no one would leave a function in that form.
It's simply a function of s. And, what is that?
So, what have we got, finally?
What we have, dear hearts, is this thing, which I stress is nothing more than the continuous analog of the summation of a power series.
This is the discrete version.
This is by these perfectly natural transformations the continuous version of the same thing.
It starts with a function defined for positive values of t, and turns it into a function of s. And, this is called the Laplace transform.
Now, if I've done my work correctly, you should all be saying, oh, is that all?
But, I know you aren't.
So, it's okay.
You'll get used to it.
The first thing you have to get used to is one thing some people never get used to, which is you put in a function of t, and you get out a function of s. How could that be?
You know, for an operator, you put in 3x, and you get out three if it's a differentiation operator.
In other words, when you have an operator, the things we've been talking about the last two or three weeks in one form or another, at least the variable doesn't get changed.
Well, but for a transform it does, and that's why it's called a transform.
So, the difference between a transform and an operator is that for a transform a function of t comes in, but a function of s comes out.
The variable gets changed, whereas for an operator, f of t goes in and what comes out is g of t, a function using the same variable like differentiation is a typical example of an operator, or the linear differential operators we've been talking about.
Well, but this doesn't behave that way.
The variable does get changed.
That's, in fact, extremely important in the applications.
In the applications, t usually means the time, and s very often, not always, but very often is a variable measuring frequency, for instance.
But, so that's a peculiar thing that's hard to get used to.
But, a good thing is the fact that it's a linear transform.
In other words, it obeys the laws we'd love and like that the Laplace transform-- oh, I never gave you any notation for the laplace transform.
Hey, I'd better do that.
Okay, so, some notation: there are two notations that are used.
Your book mostly uses the notation that the laplace transform of f of t is capital F of s, uses the same letter but with the same capital.
Now, as you will see, there are some places you absolutely cannot use that notation.
It may seem strange, looks perfectly natural.
There are certain laws you cannot express using that notation.
It's baffling.
But, if you can't do it this way, you can do it using this notation instead.
One or the other will almost always work.
So, I'll use my little squiggly notation, but that's what I use.
I think it's a little more vivid, and the trouble is that this piles up too many parentheses.
And, that's always hard to read.
So, I like this better.
So, these are two alternate ways of saying the same thing.
The Laplace transform of this function is that one.
Okay, well, let's use, for the linearity law, it's definitely best.
I really cannot express the linearity law using the second notation, but using the first notation, it's a breeze.
The Laplace transform of the sum of two functions is the sum of their Laplace transforms of each of them separately.
Or, better yet, you could write it that way.
Let's write it this way.
That way, it looks more like an operator, L of f plus L of g. And, of the same way, if you take a function and multiply it by a constant and take the laplace transform, you can pull the constant outside.
And, of course, why are these true?
These are true just because of the form of the transform.
If I add up f and g, I simply add up the two corresponding integrals.
In other words, I'm using the fact that the integral, this definite integral, is itself a linear operator.
Well, that's the general setting.
That's where it comes from, and that's the notation for it.
And, now we have to get to work.
The first thing to do to get familiar with this is, obviously what we want to do is say, okay, these were the transforms of some simple discreet functions.
Okay, suppose I put in some familiar functions, f of t. What do their Laplace transforms look like?
So, let's do that.
So, one of the boards I should keep stored.
Why don't I store on this board?
I'll store on this board the formulas as we get them.
So, let's see, what should we aim at, first?
Let's first find, and I'll do the calculations on the sideboard, and we'll see how it works out.
I'm not very sure.
In other words, what's the Laplace transform of the function, one?
Well, there's an even easier one.
What's the Laplace transform of the function zero?
Answer: zero.
Very exciting.
What's the Laplace transform of one?
Well, it doesn't turn out the constant anymore than it turned out to be a constant up there.
Let's calculate it.
Now, you can do these calculations carefully, dotting all the i's, or pretty carefully, or not carefully at all, i.e.
sloppily.
I'll let you be sloppy after, generally speaking, you could be sloppy unless the directions tell you to be less sloppy or to be careful, okay?
So, I'll do one carefully.
Let's calculate the Laplace transform of one carefully.
Okay, in the beginning, you've got nothing to use with the definition.
So, I have to calculate the integral from zero to infinity of one, that's the f of t times e to the negative s t, so I don't have to put in the one, dt.
All right, now, let me remind you, this is an improper integral.
This is just about the first time in the course we've had an improper integral.
But, there are going to be a lot of them over the next couple of weeks, nothing but.
All right, it's an improper integral.
That means we have to go back to the definition.
If you want to be careful, you have to go back to the definition of improper integral.
So, it's the limit, as R goes to infinity, of what you get by integrating only up as far as R. That's a definite integral.
That's a nice Riemann integral.
So, this is what I have to calculate.
And, I have to take the limit as R goes to infinity.
Now, how do I calculate that?
Well, this integral is equal to, that's easy.
It's just integrating.
Remember that you're integrating with respect to t. So, s is a parameter.
It's like a constant, in other words.
So, it's e to the minus s t, and when I differentiated, the derivative of this would have negative s. So, to get rid of that negative s, so the derivative is e to the minus s t. You have to put minus s in the denominator.
And now, I'll want to evaluate that between zero and R. And, what do I get?
Well it is at the upper limit.
So, it's e to the minus s times R minus, at the lower limit, it's t is equal to zero, so whatever s is, it's one.
And that's divided by this constant up front, negative s. So, the answer is, it is equal to the limit of, as R goes to infinity, of e to the negative s R minus one divided by minus s. Now, what's that?
Well, as R goes to infinity, e to the minus 2R, or minus 5R goes to zero, and the answer is minus one over minus s. So, that's one over s. And so, that's our answer.
Let's put it up here.
It's one over s, except it isn't.
I made a mistake.
Well, not mistake, a little oversight.
What's the oversight?
This is okay.
This is okay.
This is okay.
This is not okay.
This is okay.
But that's not okay.
What's wrong?
I did slight a verbal hand.
Maybe some of you have picked it up and were too embarrassed to correct me, but I said like e to the minus 2R obviously goes to zero, and e to the minus 5R goes to zero.
How about e to the minus minus 3 R?
Does that go to zero?
No, that's e to the 3R, which goes to infinity.
The only time this goes to zero is if s is a positive number.
Minus s looks like a negative number, but it's not, if s is equal to minus two.
So, this is only true if s is positive because only if s is positive is this exponent really negative and large, and therefore going to infinity, going to zero as R goes to infinity.
So, the answer is not one over s. It is one over s, s must positive.
Now, once again, here, people don't worry about this sort of thing with power series because it seems very obvious, you know, one over x, absolute value of x is less than one, when it gets to be the Laplace transform, just because the Laplace transform is mysterious, the question is, okay, the Laplace transform is one over s of one, well, Laplace transform of one I understand is one over s if s is positive.
What is it if s is negative?
Okay, right down in your little books, this, but that down, what is it if s is negative, and write underneath that, this question is meaningless.
It doesn't mean anything.
I'll draw you a picture.
This is a picture of the Laplace transform of one.
It is that.
It's one branch of this curve.
It does not include the branch on the left.
It doesn't because I showed you it doesn't.
That's all there is to it.
Okay, so I did that carefully.
Now I'm going to get a little less careful.
What's the Laplace transform of e to the a t?
First of all, in general, the kind of functions for which people like to calculate the Laplace transform, and basically the only ones there will be in the tables are exactly the sort of functions that you used in solving linear equations with constant coefficients.
What kinds of functions entered in there?
Exponentials, sines and cosines, but they were really complex exponentials, right?
e to the t sine t, but that was really a complex exponential, too, just a little more complicated one, polynomials, and that's about it.
t times e to the t, that was okay, too.
These are the functions for which people calculate the Laplace transform, and all the other functions they don't calculate the Laplace transforms.
So, I don't mean to disappoint you here.
You're going to say, oh, what, that same old stuff?
For two more weeks, we've got that same, well, the Laplace transform does a lot of things much better than the methods we've been using.
And, I won't.
I'll sell it when I get a chance to, for now, let's just get familiar with it.
All right, so while I'm not going to calculate e to the a t for you, because I'd like instead to just prove a simple formula which will just give that, and will also give us e to the a t sine t. It will give us a lot more, instead.
I'm going to calculate a formula for the Laplace transform of this guy if you already know the Laplace transform of it.
Now, see, this falls in that category because this is really e to the a t times one.
But, I already know the Laplace transform of one.
So that's, if I can get a general formula for this, I'll be able to get the formula for e to the a t as a consequence.
So, let's look for this Laplace transform.
Now, it's really easy.
Let's see, where am I doing calculations?
Over here.
Okay, so we've got e. So, I want to calculate the Laplace transform e to the a t f of t. So I'm going to say that's the integral from zero to infinity of e to the a t times f of t. And now, the rest I copy.
That's the function part of it that goes to the input, and then there's the other part.
This part is called the kernel, by the way, but don't worry about that.
However, if you drop it in conversation, people will look at you and say, gee, they know something I don't.
And you will.
You know that it's the kernel.
Okay, well, now, what kind of formula can I be looking for?
Clearly, I can only be looking for a formula which expresses it in terms of the Laplace transform of f of t. Let's calculate and see what we get.
Now, what would you do to that thing to make?
Well, obviously, the thing to do is to combine the two exponentials.
So, that's going to be the integral from zero to infinity of f of t. e, now, I'd like to put it, to combine the exponentials in such a way that it has, still, that same form, so, I'm going to begin with that negative sign, and then see what the rest of it has to be.
What is it going to be?
Well, minus s t and plus a t, but I can make that minus a here, and it will come out right.
So, it's minus s t plus a t, and there are the two parts, those two factors, dt.
So, what's that?
That's the Laplace transform.
If the a weren't there, this would be the Laplace transform of f of t. What is it with the a there?
It's the Laplace transform of f of t, except that instead of the variable, s has been replaced by the variable s minus a. I'll give you a second to digest that.
Well, you digest it while I'm writing it because that's the answer.
And, the way this is most often used, I have to qualify it for the value.
So, if F of s is good for s positive, the way it would be, for example, if I used the function one here, then to finish that off, then, F of s minus a will be, this will be good when s is bigger than a.
Why is that?
Well, because this is true.
This is true.
If s minus a is positive, that's the condition.
That's what this Laplace transform is good.
But that simply says that s should be bigger than a.
And, since this doesn't look pretty, let me try to make it look a little bit prettier.
So, let's write it.
So, this is assuming F of s is for s greater than zero.
Now, this is called something.
This is called, well, what would you call it?
On the left side, you multiply by an exponential.
On the right, you translate.
You shift the argument over by a.
So, this is called, gulp, the exponential shift.
What?
Well, I'll call it the formula.
The thing before, when we talked about operators, we called it the exponential shift rule or the exponential shift law.
But, in fact, this is, in a way, a disguised form of the same law.
And, engineers who typically do all their work using the Laplace transform and don't use operators, this is the form of the exponential shift law that they would know.
What you can do with one, you can do with the other.
You can now use both.
So, what's the answer to e to the a t?
Well, the answer is, I'm supposed to, e to the a t times one, the Laplace transform of one is one over s. And, therefore, what I do is to multiply by e to the a t, I change s to s minus a .
And so, that's the answer.
Let's see, what else don't we know?
Well, how about sines and cosines?
Well, the way to do sines and cosines is by making the observation that this formula also works when a is a complex number.
So, can use also for a a complex number, for e to the a plus b i times t. The Laplace transform of e to the a plus b i times t is one over s minus a plus b i.
And again, it will be for s bigger than a.
So, let's calculate the Laplace transform of, let's say, well, I've got to cover up something.
Okay, so, that's the Laplace transform.
I've got to remember that.
So, let's calculate the Laplace transform of, let's say, sine of a t and cosine a t. What do you get for that?
Well, just for a little variety, we could do it by using that formula, and taking its real and imaginary parts.
Since some of you had so much difficulty with the backwards Euler formula, he is a good case where you could use it.
Suppose you want to calculate the Laplace transform of cosine a t. Well, I'm going to write that using, I want to calculate using complex exponentials.
The way I will do it is by using the backwards Euler formula.
So, this is e to the i a t plus e to the minus i a t divided by two.
Remember, the foreword Euler formula would say e to the i a t equals cosine a t plus i sine a t. That expresses the complex exponential in terms of sines and cosines.
This is the backward formula, which just read it backwards, expressing cosines and sines in terms of complex exponentials instead.
Both formulas are useful, almost equally useful, in fact.
And anyway, just remind you of it, let's use this one.
Okay, what's the Laplace transform, then, of cosine a t?
Well, by linearity, it's equal to one half the Laplace transform of this guy plus the Laplace transform of that guy.
And, what are those?
Well, the Laplace transform of e to the i a t is one over s minus i a, and the Laplace transform of the other guy is one divided by s plus i a.
Now, of course, this has become out to be a real function.
This is real.
Every integral is real.
This must come out to be real.
This looks kind of complex, but it isn't.
I know automatically that this is going to be a real function.
How I know that?
Well, mentally, you can combine the terms and calculate.
But, I know even before that.
Remember, there are two ways to see that something is real.
You can calculate it and see that its imaginary part is zero, hack, or without any calculation, if you change i to minus i, and you get the same thing, it must be real.
Now, if I change i to minus i in this expression, what happens?
If I change i to minus i, this term turns into that one, and this one turns into that one.
Conclusion: the sum of the two is unchanged.
And therefore, this is real.
Well, of course, in the time I took to make that argument, I could have actually calculated it.
So, what the heck, let's calculate it?
So, you do the high school thing, and it's this guy plus that guy on top, which makes 2s.
I on the bottom is the product of those, which by now you should know the product of two complex numbers.
A product of a number and its complex conjugate is the sum of the squares.
So, what's the answer?
The twos cancel, and the answer is that the Laplace transform of cosine a t is s over s squared plus a squared.
And, that will be true as, in general, it's true up there for positive values of s only.
And, the sine a t, you can calculate that in recitation tomorrow.
The answer to that is a divided by s squared plus a squared.
You would get the same answers if you took the real and imaginary parts of that expression.
It's another way of getting at the recitations tomorrow; we'll get practice in calculating other functions related to these by using these formulas, and also from scratch directly from the definition of the Laplace transform.
Well, there are two things which we still should do.
The first is I want to get you started with calculating inverse Laplace transforms.
And, the reason for doing that is, in other words, I've started with f of t, and we've been focusing on what is capital F of s?
But, you will find that when you go to solve differential equations, by far, the hardest part of the procedure is you get F of s. The Laplace transform of the answer, and you have to convert that back into the answer in terms of t that you were looking for.
In other words, the main step in the procedure that you are going to be using for solving differential equations is, and the hardest part of the step will be to calculate inverse laplace transforms.
Now, you think that could be done by tables, but, in fact, it can't unless the tables are too long to be useful.
You have to do a certain amount of work yourself.
And, the certain amount of work that you have to do yourself involves partial fractions decompositions.
And, in case you were wondering which you are not, the reason you learned partial fractions in 18.01 was not to learn those silly integrals, but he learned it so that when you got to 18.03 you would be able to calculate, solve differential equations by using Laplace transforms.
Sorry.
That's life.
Now, so a certain amount of the recitation time tomorrow will be devoted to reminding you how to do partial fractions since you haven't done it in a while, and I assume, yeah, we had that, I think.
Okay, now, they also remind you of the most efficient method, which about half of you have had, and the rest think you might have had, but really aren't sure.
So, here's the answer.
We want to find out what it's inverse Laplace transform is.
What you have to do, it normally won't be in the tables like this.
You have to put it in a form in which it will be in the tables.
As you do that, you have to make partial fractions decompositions, which, to do it quickly, so if you don't know what I'm doing now, or you think you once knew but don't quite remember, go to recitation tomorrow.
To get the coefficient here, I cover up s, and I put s equals zero because that's the law.
To get this coefficient, I cover up s plus three and I put s equals a negative three because that's what you're supposed to do.
Put s equal negative three, you get minus one third.
This is equal to that.
In this form, I don't know what the inverse Laplace form is, but in this form, I certainly do know with the inverse Laplace transform because the inverse Laplace transform is linear, and because each of these guys especially occurs in those tables.
Well, what's this?
Well, it's whatever the Laplace transform of, inverse Laplace transform of one over s is multiplied by one third.
Well, the inverse Laplace transform of one over s is one.
So, it's one third times one.
How about the other guy?
Minus one third, the inverse Laplace transform of one over s plus three, that's this formula.
a is negative three, and that makes e to the minus 3t.
So, if this was the Laplace transform of the solution to the differential equation, then the solution in terms of t was this function.
Now, you'll get lots of practice in that.
All I'm doing now is signaling that that's the most important and difficult step of the procedure, and that, please, start getting practice.
Get up to snuff doing that procedure.
Okay, in the time remaining, I want to add one formula to this list, and that is going to be the Laplace transform of, we still haven't done polynomials.
And now, to polynomials, because the Laplace transform is linear, all I have to do is know what the Laplace transform of, the individual term of a polynomial.
In other words, what the Laplace transform of t to the n, where n is some positive integer?
Well, let's bravely start trying to calculate it.
Integral from zero to infinity t to the n e to the negative st dt.
Now, I think you can see that the method you should use is integration by part because this is a product of two things, one of which you would like to differentiate a lot of times, in fact, and the other won't hurt to integrate it because it's very easy to integrate.
So, this factor is going to be the one that's to be differentiated, and this is the factor that will be pleased to integrate it.
Let's get started and see what we can get out of it.
Well, this time I'm going to be, well, I'd better be a little careful because there's a point here that's tricky.
Okay, the first step of integration by parts is you only do the integration.
You don't do the differentiation.
Remember, the variable is t. The s is just a parameter.
It's just a constant.
It's hanging around, not knowing what to do.
Okay, so the first step is you don't do the differentiation.
You only do the integration.
Evaluate it between limits, and then you put a minus sign before you forget to do it.
And then, integral zero to infinity.
Now you do both operations.
So, it's n t to the n minus one, and you also do the integration.
Okay, let's consider each of these pieces in turn.
Now, this piece, well, there's no problem with the lower limit, zero, because when t is equal to zero, this factor is zero, and the thing disappears as long as n is one or higher.
So, it's minus zero here at the lower limit.
The question is, what is at the upper limit?
So, what I have to do is find out, what is the limit?
The limit, as t goes to infinity, that's what's happening up there, of t to the n times e to the negative s t divided by minus s. Well, as t goes to infinity, this goes to infinity, of course.
This had better go to zero unless I want an answer, infinity, which won't do me any good.
If this goes to zero, s had better be positive.
So, I'd better be restricting myself to that case.
Okay, so let's assume that s is positive so that this minus s really is a negative number.
Okay, then I have a chance.
So, this is going to be the limit.
Let's write it in a more familiar form with that down below.
So, it's t to the n. That's going to infinity.
But, the bottom is e to the minus s t. But now, it's plus s t. And, that's going to infinity, too, because s is positive.
So, the two guys are racing, and the question is, oh, I lost a minus s here.
So, oh... equals minus one over s. How's that?
So, the question is only, which guy wins?
In the race to infinity, which one wins, and how do you decide?
And, the answer, of course, is that's the bottom that wins.
The exponential always wins, and it's because of L'Hopital's rule.
You differentiate top and bottom.
Nothing much happens to the bottom.
It gets another factor of s, but the top goes down to t to the n minus one.
L'Hopital it again, and again, and again, and again, and again until finally you've reduced the top to t to the zero where it's defenseless and just sitting there, and nothing's happened to the bottom.
It's still got e to the s t. and that goes to infinity.
So, the answer is, this is zero by n applications of L'Hopital's rule.
Or, if you're very clever, you can do it in one, but I won't tell you how.
So, the answer is that this is zero.
At the upper limit, it's also zero at least if s is positive, which is the case we're considering.
That leaves the rest of this.
All right, let's pull the constants out front.
That's plus.
Two negatives make a plus.
n over s, now, what's left?
The integral from zero to infinity of t to the n minus one, e to the minus s t dt.
But, what on Earth is that?
That is n over s times the Laplace transform of t to the n minus one.
We got a reduction for it.
We don't get the answer in one step.
But, we get a reduction formula.
And, it says that the Laplace transform, let me write it this way for once.
The first way is now better, is equal to n over s times the Laplace transform of n minus t to the n minus one.
Okay, the next step, this would be n over s times n minus one over s times the Laplace transform of t to the n minus two.
If I can continue, I finally get in the top n times n minus one times all the way down to one divided by the same number of s's, n of them, times the Laplace transform of t to the zero, finally.
See, one, zero, n minus one.
And so, what's the final answer?
It is n factorial over s to the what power?
Well, the Laplace transform of this is one over s. So, the answer is it's s to the n plus one, n of them here plus an extra one coming from the one over s here.
And, that's the answer.
The Laplace transform of t to the n, oddly enough, is more complicated, and looks a little different from these.
It's n factorial over s to the n plus one.
And, with that, you can now calculate the Laplace transform of anything in sight, and tomorrow you will.
Okay, those are the formulas.
You will get all of those on the test, plus a couple more that I will give you today.
Those will be the basic formulas of the Laplace transform.
If I think you need anything else, I'll give you other stuff, too.
So, I'm going to leave those on the board all period.
The basic test for today is to see how Laplace transforms are used to solve linear differential equations with constant coefficients.
Now, to do that, we're going to have to take the Laplace transform of a derivative.
And, in order to make sense of that procedure, we're going to have to ask, I apologize in advance, but a slightly theoretical question, namely, we have to have some guarantee in advance that the Laplace transform is going to exist.
Now, how could the Laplace transform fail to exist?
Can't I always calculate this?
And the answer is, no, you can't always calculate it because this is an improper integral.
I'm integrating all the way up to infinity, and you know that improper integrals don't always converge.
You know, if the integrand for example just didn't have the exponential factor there, were simply t dt, that it might look like it made sense, but that integral doesn't converge.
And, anyway, it has no value.
So, I need conditions in advance, which guarantee that the Laplace transforms will exist.
Only under those circumstances will the formulas make any sense.
Now, there is a standard condition that's in your book.
But, I didn't get a chance to talk about it last time.
So, I thought I'd better spent the first few minutes today talking about the condition because it's what we're going to need in order to be able to solve differential equations.
The condition that makes the Laplace transform definitely exist for a function is that f of t shouldn't grow too rapidly.
It can grow rapidly.
It can grow because the e to the minus s t is pulling it down, trying hard to pull it down to zero to make the integral converge.
All we have to do is to guarantee that it doesn't grow so rapidly that the e to the minus s t is powerless to pull it down.
Now, the condition is it's what's called a growth condition.
These are very important in applications, and unfortunately, it's always taught in 18.01, but it's not always taught in high school calculus.
And, it's a question of how fast the function is allowed to grow.
And, the condition is universally said this way, should be of exponential type.
So, what I'm defining is the phrase "exponential type."
I'll put it in quotation marks for that reason.
What does this mean?
It's a condition, a growth condition on a function, says how fast it can get big.
It says that f of t in size, since f of t might get negatively very large, and that would hurt, make the integral hard to converge, not likely to converge, use the absolute value.
In other words, I don't care if f of t is going up or going down very low.
Whichever way it goes, its size should not be bigger than a rapidly growing exponential.
And, here's a rapidly growing exponential.
c is some positive constant, for some positive constant c and some positive constant k. And, this should be true for all values of t. All t greater than or equal to zero.
I don't have to worry about negative values of t because the integral doesn't care about them.
I'm only doing the integration as t runs from zero to infinity.
In other words, f of t could have been an extremely wild function, sewn a lot of oats or whatever functions do for negative values of t, and we don't care.
It's only what's happening from now from time zero onto infinity.
As long as it behaves now, from now on, it's okay.
All right, so, the way it should behave is by being an exponential type.
Now, to try to give you some feeling for what this means, these functions, for example, if k is 100, do you have any idea what the plot of e to the 100t looks like?
It goes straight up.
On every computer you try to plot it on, e to the 100t goes like that unless, of course, you make the scale t equals zero to, over here, is one millionth.
Well, even that won't do.
Okay, so these functions really can grow quite rapidly.
Let's take an example and see what's of exponential type, and then perhaps even more interestingly, what isn't.
The function sine t, is that of exponential type?
Well, sure.
Its absolute value is always less than or equal to one.
So, it's also this paradigm.
If I take c equal to one, and what should I take k to be?
Zero.
Take k to be zero, c equals one, and in fact sine t plays that condition.
Here's one that's more interesting, t to the n. Think of t to the 100th power.
Is that smaller than some exponential with maybe a constant out front?
Well, t to the 100th power goes straight up, also.
Well, we feel that if we make the exponential big enough, maybe it will win out.
In fact, you don't have to make the exponential big.
k equals one is good enough.
In other words, I don't have to put absolute value signs around the t to the n because I'm only thinking about t as being a positive number, anyway.
I say that that's less than or equal to some constant M, positive constant M times e to the t will be good enough for some M and all t. Now, why is that?
Why is that?
The way to think of that, so, what this proves is that, therefore, t to the n is of exponential type, which we could have guessed because after all we were able to calculate its Laplace transform.
Now, just because you can calculate the Laplace transform doesn't mean it's of exponential type, but in practical matters, it almost always does.
So, t to the n is of exponential type.
How do you prove that?
Well, the weighted secret is to look at t to the n divided by e to the t. In other words, look at the quotient.
What I'd like to argue is that this is bounded by some number, capital M. That's the question I'm asking.
Now, why is this so?
Well, I think I can convince you of it without having to work very hard.
What does the graph of this function look like?
It starts here, so I'm graphing this function, this ratio.
When t is equal to zero, its value is zero, right, because of the numerator.
What happens as t goes to infinity?
What happens to this?
What does it approach?
Zero.
And, why?
By L'Hop.
By L'Hopital's rule.
Just keep differentiating, reapply the rule over and over, keep differentiating it n times, and finally you'll have won the numerator down to t to the zero, which isn't doing anything much.
And, the denominator, no matter how many times you differentiate it, it's still t, to the t all the time.
So, by using Lopital's rule n times, you change the top to one or n factorial, actually; the bottom stays e to the t, and the ratio clearly approaches zero, and therefore, it approached zero to start with.
So, I don't know what this function's doing in between.
It's a positive function.
It's continuous because the top and bottom are continuous, and the bottom is never zero.
So, it's a continuous function which starts out at zero and is positive, and as t goes to infinity, it gets closer and closer to the t-axis, again.
Well, what does t to the n do?
It might wave around.
It doesn't actually.
But, the point is, because it's continuous, starts at zero, ends at zero, it's bounded.
It has a maximum somewhere.
And, that maximum is M. So, it has a maximum.
All you have to know is where it starts, and where it ends up, and the fact that it's continuous.
That guarantees that it has a maximum.
So, it is less than some maximum, and that shows that it's of exponential type.
Now, of course, before you get the idea that everything's of exponential type, let's see what isn't.
I'll give you two functions that are not of exponential type, for different reasons.
One over t is not of exponential type.
Well, of course, it's not defined that t equals zero.
But, you know, it's okay for an integral not to be defined at one point because you're measuring an area, and when you measure an area, what happened to one point doesn't really matter much.
That's not the thing.
What's wrong with one over t is that the integral doesn't converge at zero times one over t dt.
That integral, when t is near zero, this is approximately equal to one, right?
If t is zero, this is one.
So, it's like the function, integral from zero to infinity of one over t, near zero it's close to, it's like the integral from zero to someplace of no importance, dt over t. But, this does not converge.
This is like log t, and log zero is minus infinity.
So, it doesn't converge.
So, one over t is not of exponential type.
So, what's the Laplace transform of one over t?
It doesn't have a Laplace transform.
Well, what if I put t equals negative n?
What about t to the minus one?
Well, that only works for positive integers, not negative integers.
Okay, so it's not of exponential type.
However, that's because it never really gets started properly.
It's more fun to look at a function which is not of exponential type because it grows too fast.
Now, what's a function that grows faster that it grows so rapidly that you can't find any function e to the k t which bounds it?
A function which grows too rapidly, a simple one is e to the t squared, grows too rapidly to be of exponential type.
And, the argument is simple.
No matter what you propose, it's always, for the K, no matter how big a number, use Avogadro's number, use anything you want.
Ultimately, this is going to be bigger than k t no matter how big k is, no matter how big k is.
When is this going to happen?
This will happen if t squared is bigger than k t. In other words, as soon as t becomes bigger than k, you might have to wait quite a while for that to happen, but, as soon as t gets bigger than 10 to the 10 to the 23, this e to the t squared will be bigger than e to the 10 to the 10 to the 23 times t. So, e to the t squared, it's a simple function, a simple elementary function.
It grows so rapidly it doesn't have a Laplace transform.
Okay, so how are we going to solve differential equations if e to the t squared?
I won't give you any.
And, the reason I won't give you any: because I never saw one occur in real life.
Nature, like sines, cosines, exponentials, are fine, I've never seen a physical, you know, this is just my ignorance.
But, I've never seen a physical problem that involved a function growing as rapidly as e to the t squared.
That may be just my ignorance.
But, I do know the Laplace transform won't be used to solve differential equations involving such a function.
How about e to the minus t squared?
That's different.
It looks almost the same, but e to the minus t squared does this.
It's very well-behaved.
That's the curve, of course, that you're all afraid of.
Don't panic.
Okay.
So, I'd like to explain to you now how differential equations, maybe I should save-- I'll tell you what.
We need more formulas.
So, I'll put them, why don't I save this board, and instead, I'll describe to you the basic way Laplace transforms are used to solve differential equations, what are they called, a paradigm.
I'll show you the paradigm, and then we'll fill in the holes so you have some overall view of how the procedure goes, and then you'll understand where the various pieces fit into it.
I think you'll understand it better that way.
So, what do we do?
Start with the differential equation.
But, right away, there's a fundamental difference between what the Laplace transform does, and what we've been doing up until now, namely, what you have to start with is not merely the differential equation.
Let's say we have linear with constant coefficients.
It's almost never used to solve any other type of problem.
And, let's take second order so I don't have to do, because that's the kind we've been working with all term.
But, it's allowed to be inhomogeneous, so, f of t. Let's call the something else, another letter, h of t. I'll want f of t for the function I'm taking the Laplace transform of.
All right, now, the difference is that up to now, you know techniques for solving this just as it stands.
The Laplace transform does not know how to solve this just doesn't stands.
The Laplace transform must have an initial value problem.
In other words, you must supply from the beginning the initial conditions that the y is to satisfy.
Now, I don't want to say any specific numbers, so I'll use generic numbers.
Well, but look, what do we do if we get a problem and there are no initial conditions; does that mean we can't use the Laplace transform?
No, of course you can use it.
But, you will just have to assume the initial conditions are on the numbers.
You'll say it but the initial conditions be y sub zero and y zero prime, or whatever, a and b, whatever you want to call it.
And now, the answer, then, will involve the a and the b or the y zero and the y zero prime.
But, you must, at least, give lip service to the initial conditions, whereas before we didn't have to do that.
Now, depending on your point of view, that's a grave defect, or it is, so what?
Let's adopt the so what point of view.
So, there's our problem.
It's an initial value problem.
How is it solved by the Laplace transform?
Well, the idea is you take the Laplace transform of this differential equation and the initial conditions.
So, I'm going to explain to you how to do that.
Not right now, because we're going to need, first, the Laplace transform of a derivative, the formula for that.
You don't know that yet.
But when you do know it, you will be able to take the Laplace transform of the initial value problem.
So, I'll put the little l here, and what comes out is, well, y of t is the solution to the original problem.
If y of t is the function which satisfies that equation and these initial conditions, its Laplace transform, let's call it capital Y, that's our standard notation, but it's going to be of a new variable, s. So, when I take the Laplace transform of the differential equation with the initial conditions, what comes out is an algebraic-- the emphasis is on algebraic: no derivatives, no transcendental functions, nothing like that, an algebraic equation, m Y of s. And, now what?
Well, now, in the domain of s, it's easy to solve this algebraic equation.
Not all algebraic equations are easy to solve for the capital Y.
But, the ones you will get will always be, not because I am making life easy for you, but that's the way the Laplace transform works.
So, you will solve it for Y.
And, the answer will always come out to be Y equals, Y of s equals some rational function, some quotient of polynomials in s, a polynomial in s divided by some other polynomial in s. And, now what?
Well, this is the Laplace transform of the answer.
This is the Laplace transform of the solution we are looking for.
So, the final step is to go backwards by taking the inverse Laplace transform of this guy.
And, what will you get?
Well, you will get y equals the y of t that we are looking for.
It's really a wildly improbable procedure.
In other words, instead of going from here to here, you have to imagine there's a mountain here.
And, the only way to get around it is to go, first, here, and then cross the stream here, and then go back up, and go back up.
It looks like a senseless procedure, what do they call it, going around Robin Hood's barn, it was called when I was a, I don't know why it's called that.
But that's what we used to call it; not Laplace transform.
That was just a generic thing when you had to do something like this.
But, the answer is that it's hard to go from here to here, but trivial to go from here to here.
This solution step is the easiest step of all.
This is not very hard.
It's easy, in fact.
This is easy and straightforward.
This is trivial, essentially, yeah, trivial.
But, this step is the hard step.
This is where you have to use partial fractions, look up things in the table to get back there so that most of the work of the procedure isn't going from here to here.
Going from here to there is a breeze.
Okay, now, in order to implement this, what is it we have to do?
Well, the basic thing is I have to explain to you, you already know at least a little bit, a reasonable amount of technique for taking that step if you went to recitation yesterday and practiced a little bit.
This part, I assure you, is nothing.
So, all I have to do now is explain to you how to take the Laplace transform of the differential equation.
And, that really means, how do you take the Laplace transform of a derivative?
So, that's our problem.
What I want to form, in other words, is a formula for the Laplace transform f prime of t. Now, in terms of what?
Well, since f is an arbitrary function, the only thing I could hope for is somehow to express the Laplace transform of the derivative in terms of the Laplace transform of the original function.
So, that's what I'm aiming for.
Okay, where are we going to start?
Well, starting is easy because we know nothing.
If you don't know anything, then there's no place to start but the definition.
Since I know nothing whatever about the function f of t, and I want to calculate the Laplace transform, I'd better start with the definition.
Whatever this is, it's the integral from zero to infinity of e to the minus s t times f prime of t dt.
Now, what am I looking for?
I'm looking for somehow to transform this so that what appears here is not f prime of t, which I'm clueless about, but f of t because if this were f of t, this expression would be the Laplace transform of f of t. And, I'm assuming I know that.
So, the question is how do I take this and somehow do something clever to it that turns this into f of t instead of f prime of t?
Now, to first the question that way, I hope I would get 100% response on what to do.
But, I'll go for 1%.
So, what should I do?
I want to change that, so that instead of f prime of t, f of t appears there instead.
What should I do?
Integrate by parts, the most fundamental procedure in advanced analysis.
Everything important and interesting depends on integration by parts.
And, when you consider that integration by parts is nothing more than just the formula for the derivative of a product read backwards, it's amazing.
It never fails to amaze me, but it's okay.
That's what mathematics are so great.
Okay, so let's use integration by parts.
Integration by parts: okay, so, we have to decide, of course, there's no doubt that this is the factor we want to integrate, which means we have to be willing to differentiate this factor.
But that will be okay because it looks practically, like any exponential, it looks practically the same after you've differentiated it.
So, let's do the work.
First step is you don't do the differentiation.
You only do the integration.
So, the first step is e to the negative s t. And, the integral of f prime of t is just f of t. And, that's to be evaluated between the limits zero and infinity.
And then, minus, again, before you forget it, put down that minus sign.
The integral between the limits of what you get by doing both operations, both the differentiation and the integration.
So, the differentiation will be by using the chain rule.
Remember, I'm differentiating with respect to t. The variable is t here, not s. s is just a parameter.
It's just a constant, a variable constant, if you get my meaning.
That's not an oxymoron.
A variable constant: a parameter is a variable constant, variable because you can manipulate the little slider and make a change its value, right?
That's why it's variable.
It's not a variable.
It's variable, if you get the distinction.
Okay, well, I mean, it becomes a variable [LAUGHTER].
But right now, it's not a variable.
It's just sitting there in the integral.
All right, so, minus s, e to the negative s t, f of t dt.
Now, this part's easy.
The interesting thing is this expression.
So, and the most interesting thing is I have to evaluate it at infinity.
Now, of course, that means take the limit as you go towards, as you let t goes to infinity.
Now, so what I'm interested in knowing is what's the limit of that expression?
I'll write it as f of t divided by e to the s t. Remember, s is a positive number.
s t goes to infinity, and I want to know what the limit of that is.
Well, the limit is what it is.
But really, if that limit isn't zero, I'm in deep trouble since the whole process is out of control.
What will make that limit zero?
Well, that f of t should not grow faster than e to the s t if s is a big enough number.
And now, that's just what will happen if f of t is of exponential type.
It's for this step right here that is the most crucial place at which we need to know that f of t is of exponential type.
So, that limit is zero since f of t is of exponential type, in other words, that the value, the absolute value of f of t, becomes less than, let's say, put in the c if you want, but it's not very important, c e to the k t efor all values of t. And, therefore, this will go to zero as soon as s becomes bigger than that k. In other words, if f of t isn't growing any faster than e to the k t , then as soon as s is a number, that parameter has the value bigger than k, this ratio is going to go to zero because the denominator will always be bigger than the numerator, and getting bigger faster.
So, this goes to zero if s is bigger than k. At the upper limit, therefore, this is zero.
Again, assuming that s is bigger than that k, the k of the exponential type, how about at the lower limit?
We're used to seeing zero there, but we're not going to get zero.
If I plug in t equals zero, this factor becomes one.
And, what happens to that one?
f of zero.
You mean, I'm going to have to know what f of zero is before I can take the Laplace transform of this derivative?
The answer is yes, and that's why you have to have an initial value problem.
You have to know in advance what the value of the function that you are looking for is at zero because it enters into the formula.
I didn't make up these rules; I'm just following them.
So, what's the rest?
The two negatives cancel, and you get plus s. It's just a parameter, so I can pull it out of the integral.
I'm integrating with respect to t, and what's left is, well, what is left?
If I take out minus s, combine it there, I get what's left is just the Laplace transform of the function I started with.
So, it's F of s. And, that's the magic formula for the Laplace transform of the derivative.
So, it's worth putting up on our little list.
So, f prime of t, assuming it's of exponential type, has as its Laplace transform, well, what is it?
Let's put down the major part of it is s times whatever the Laplace transform of the original function, F of t, was.
So, I take the original Laplace transform.
When I multiply it by s, that corresponds to taking the derivative.
But there's also that little extra piece.
I must know the value of the starting value of the function.
This is the formula you'll used to take a Laplace transform of the differential equation.
Now, but you see I'm not done yet because that will take care of the term a y prime.
But, I don't know what the Laplace transform of the second derivative is.
Okay, so, we need a formula for the Laplace transform of a second derivative as well as the first.
Now, the hack method is to say, secondary, all right.
I've got to do this.
I'll second derivative here, second derivative here, what do I do with that?
Ah-ha, I integrate by parts twice.
Yes, you can do that.
But that's a hack method.
And, of course, I know you're too smart to do that.
What you would do instead is-- How are we going to fill that in?
Well, a second derivative is also a first derivative.
A second derivative is the first derivative of the first derivative.
Okay, now, we'll just call this glop, something.
So, it's glop prime.
What is the formula for the Laplace transform of glop prime?
It is, well, I have my formula.
It is the glop prime.
The formula for it is s times the Laplace transform of glop, okay, glop.
Well, glop is f prime of t. I'm not done yet, minus glop evaluated at zero.
What's glop evaluated at zero?
Well, f prime of zero.
Now, I don't want the formula in that form, but I have to have it in that form because I know what the Laplace transform of f prime of t is.
I just calculated that.
So, this is equal to s times the Laplace transform of f prime of t, which is s times F of s, capital F of s, minus f of zero.
All that bracket stuff corresponds to this guy.
And, don't forget the stuff that's tagging along, minus f prime of zero.
And now, put that all together.
What is it going to be?
Well, there's the principal term which consists of s squared multiplied by F of s. That's the main part of it.
And, the rest is the sort of fellow travelers.
So, we have minus s times f of zero, little term tagging along.
This is a constant times s. And then, we've got another one, still another constant.
So, what we have is to calculate the Laplace transform of the second derivative, I need to know both f of zero and f prime of zero, exactly the initial conditions that the problem was given for the initial value problem.
But, notice, there's a principal part of it.
That's the s squared F of s. That's the guts of it, so to speak.
The rest of it, you know, you might hope that these two numbers are zero.
It could happen, and often it is made to happen and problems to simplify them.
And I case, you don't have to worry; they're not there.
But, if they are there, you must put them in or you get the wrong answer.
So, that's the list of formulas.
So, those formulas on the top board and these two extra ones, those are the things you will be working with on Friday.
But I stress, the Laplace transform won't be a big part of the exam.
The exam, of course, doesn't exist, let's say a maximum of 20%, maybe 15.
I don't know, give or take a few points.
Yeah, what's a point or two?
Okay, let's solve, yeah, we have time.
We have time to solve a problem.
Let's solve a problem.
See, I can't touch that.
It's untouchable.
Okay, this, we've got to keep.
Problem?
Okay.
Okay, now you know how to solve this problem by operators.
Let me just briefly remind you of the basic steps.
You have to do two separate tasks.
You have to first solve the homogeneous equation, putting a zero there.
That's the first thing you learned to do.
That's easy.
You could almost do that in your head now.
You get the characteristic polynomial, get its roots, get the two functions, e to the t and e to the negative t, which are the solutions.
You make up c1 times one, and c2 times the other.
That's the complementary function that solves the homogeneous problem.
And then you have to find a particular solution.
Can you see what would happen if you try to find the particular solution?
The number here is negative one, right?
Negative one is a root of the characteristic polynomial, so you've got to use that extra formula.
It's okay.
That's why I gave it to you.
You've used the exponential input theorem with the extra formula.
Then, you will get the particular solution.
And now, you have to make the general solution.
The particular solution plus the complementary function, and now you are ready to put in the initial conditions.
At the very end, when you've got the whole general solution, now you put in, not before, you put in the initial conditions.
You have to calculate the derivative of that thing and substitute this.
You take it as it stands to substitute this.
You get a pair of simultaneous equations for c1 and c2.
You solve them: answer.
It's a rather elaborate procedure, which has at least three or four separate steps, all of which, of course, must be done correctly.
Now, the Laplace transform, instead, feeds the entire problem into the Laplace transform machine.
You follow that little blue pattern, and you come out with the answer.
So, let's do the Laplace transform way.
Okay, so, the first step is to say, if here's my unknown function, y of t, it obeys this law, and here are its starting values, a bit of its derivative.
What I'm going to take is the Laplace transform of this equation.
In other words, I'll take the Laplace transform of this side, and this side also.
And, they must be equal because if they were equal to start with, the Laplace transforms also have to be equal.
Okay, so let's take the Laplace transform of this equation.
Okay, first ID the Laplace transform of the second derivative.
Okay, that's going to be, don't forget the principal terms.
There is some people who get so hypnotized by this.
I just know I'm going to forget this.
So, they read it.
Then they forget this.
But that's everything.
That's the important part.
Okay, so it's s times, I'm calling the Laplace transform not capital F but capital Y because my original function is called little y.
So, it's s squared Y.
It's Y of s, but I'm not going to put that, the of s in because it just makes the thing look more complicated.
And now, before you forget, you have to put in the rest.
So, minus s times the value at zero, which is one, minus the value of the derivative.
But, that's zero.
So, this is not too hard a problem.
So, minus s minus zero, so I don't have to put that in.
So, all this is the Laplace transform of y double prime.
And now, minus the Laplace transform of y, well, that's just capital Y.
What's that equal to?
The Laplace transform of the right-hand side.
Okay, look up the formula.
It is e to the negative t, a is minus one, so, it's one over s minus minus one; so, it is s plus one.
This is that.
Okay, the next thing we have to do is solve for Y.
That doesn't look too hard.
Solve it for y.
Okay, the best thing to do is put s squared, group all the Y terms together unless you're really quite a good calculator.
Maybe make one extra line out of it.
Yeah, definitely do this.
And then, the extra garbage I refer to as the garbage, this stuff, and this stuff, the stuff, the linear polynomials which are tagging along move to the right-hand side because they don't involve capital Y.
So, this we will move to the other side.
And so, that's equal to (one over (s plus one)) plus s. Now, you have a basic choice.
About half the time, it's a good idea to combine these terms.
The other half of the time, it's not a good idea to combine those terms.
So, how do we know whether to do it or not to do it?
Experience, which you will get by solving many, many problems.
Okay, I'm going to combine them because I think it's a good thing to do here.
So, what is that?
That's s squared plus s plus one.
So, it's s squared plus s plus one divided by s plus one, okay?
I'm still not done because now we have to know, what's Y?
All right, now we have to think.
What we're going to do is get Y in this form.
But, I want it in the form in which it's most suited for using partial fractions.
In other words, I want the denominator as factored as I possibly can be.
Okay, well, the numerator is going to be just what it was.
How should I write the denominator?
Well, the denominator is going to have the factor s plus one in it from here.
But after I divide through, the other factor will be s squared minus one, right?
But, s squared minus one is s minus one times s plus one.
So, I have to divide this by s squared minus one.
Factored, it's this.
So, the end result is there are two of these and one of the other.
And now, it's ready to be used.
It's better to be a partial fraction.
So, the final step is to use a partial fraction's decomposition on this so that you can calculate its inverse Laplace transform.
So, let's do that.
Okay, (s squared plus s plus one) divided by that thing, (s plus one) squared times (s minus one) equals s plus one squared plus s plus one plus s minus one.
In the top will be constants, just constants.
Let's do it this way first, and I'll say at the very end, something else.
Maybe now.
Many of you are upset.
Some of you are upset.
I know this for a fact because in high school, or wherever you learned to do this before, there weren't two terms here.
There was just one term, s plus one squared.
If you do it that way, then it's all right.
Then, it's all right, but I don't recommend it.
In that case, the numerators will not be constants.
But, if you just have that, then because this is a quadratic polynomial all by itself.
You've got to have a linear polynomial, a s plus b in the top, see?
So, you must have a s plus b here, as I'm sure you remember if that's the way you learned to do it.
But, to do cover-up, the best way as much as possible to separate out the terms.
If this were a cubic term, God forbid, s plus one cubed, then you'd have to have s plus one cubed, s plus one squared.
Okay, I won't give you anything bigger than quadratic.
[LAUGHTER] You can trust me.
Okay, now, what can we find by the cover up method?
Well, surely this.
Cover up the s minus one, put s equals one, and I get three divided by two squared, four.
So, this is three quarters.
Now, in this, you can always find the highest power by cover-up because, cover it up, put s equals negative one, and you get one minus one plus one.
So, one up there, negative one here makes negative two here.
So, one over negative two.
So, it's minus one half.
Now, this you cannot determine by cover-up because you'd want to cover-up just one of these s plus ones.
But then you can't put s equals negative one because you get infinity.
You get zero there, makes infinity.
So, this must be determined some other way, either by undetermined coefficients, or if you've just got one thing, for heaven's sake, just make a substitution.
See, this is supposed to be true.
This is an algebraic identity, true for all values of the variable, and therefore, it ought to be true when s equals zero, for instance.
Why zero?
Well, because I haven't used it yet.
I used negative one and positive one, but I didn't use zero.
Okay, let's use zero.
Put s equals zero.
What do we get?
Well, on the left-hand side, I get one divided by one squared, negative.
So, I get minus one on the left hand side equals, what do I get on the right?
Put s equals zero, you get negative one half.
Well, this is the number I'm trying to find.
So, let's write that simply as plus c, putting s equals zero.
s equals zero here gives me negative three quarters.
Okay, what's c?
This is minus a half, minus three quarters, is minus five quarters.
Put it on the other side, minus one plus five quarters is plus one quarter.
So, c equals one quarter.
And now, we are ready to do the final step.
Take the inverse Laplace transform.
You see what I said when I said that all the work is in this last step?
Just look how much of the work, how much of the board is devoted to the first two steps, and how much is going to be devoted to the last step?
Okay, so we get e to the inverse Laplace transform.
Well, the first term is the hardest.
Let's let that go for the moment.
So, I leave a space for it, and then we will have one quarter.
Well, one over s plus one is, that's just the exponential formula.
One over s plus one would be e to the negative t,e to the minus one times t. So, it's one quarter e to the minus one times t. And, how about the next thing would be three quarters times, well, here it's negative one, so that's e to the plus t. Notice how those signs work.
And, that just leaves us the Laplace transform of this thing.
Now, you look at it and you say, this Laplace transform happened in two steps.
I took something and I got, essentially, one over s squared.
And then, I changed s to s plus one.
All right, what gives one over s squared?
The Laplace transform of what is one over s squared?
t, you say to yourself, one over s to some power is essentially some power of t. And then, you look at the formula.
Notice at the top is one factorial, which is one, of course.
Okay, now, then how do I convert this to one over s plus one squared?
That's the exponential shift formula.
If you know what the Laplace transform, so the first formula in the middle of the board on the top, there, if you know what, change s to s plus one, corresponds to multiplying by e to the t. So, it is t times e to the negative t. Sorry, that corresponds to this.
So, this is the exponential shift formula.
If t goes to one over s squared, then t e to the minus t goes to one over s plus one squared.
Okay, but there's a constant out front.
So, it's minus one half t e to the negative t. Now, tell me, what parts of this solution, oh boy, we're over time.
But, notice, this is what would have been the particular solution, (y)p before, and this is the stuff that occurs in the complementary function, but already the appropriate constants have been supplied for the coefficients.
You don't have to calculate them separately.
They were built into the method.
Okay, good luck on Friday, and see you there.
Today is going to be one of the more difficult lectures of the term.
So, put on your thinking caps, as they would say in elementary school.
The topic is going to be what's called a convolution.
The convolution is something very peculiar that you do to two functions to get a third function.
It has its own special symbol.
f of t asterisk is the universal symbol that's used for that.
So, this is a new function of t, which bears very little resemblance to the ones, f of t, that you started with.
I'm going to give you the formula for it, but first, there are two ways of motivating it, and both are important.
There is a formal motivation, which is why it's tucked into the section on Laplace transform.
And, the formal motivation is the following.
Suppose we start with the Laplace transform of those two functions.
Now, the most natural question to ask is, since Laplace transforms are really a pain to calculate is from old Laplace transforms, is it easy to get new ones?
And, the first thing, of course, summing functions is easy.
That gives you the sum of the transforms.
But, a more natural question would be, suppose I want to multiply F of t and G of t. Is there, hopefully, some neat formula?
If I multiply the product of the, take the product of these two, is there some neat formula for the Laplace transform of that product?
That would simply life greatly.
And, the answer is, there is no such formula.
And there never will be.
Well, we will not give up entirely.
Suppose we ask the other question.
Suppose instead I multiply the Laplace transforms.
Could that be related to something I cook up out of F of t and G of t?
Could it be the transform of something I cook up out of F of t or G of t?
And, that's what the convolution is for.
The answer is that F of s times G of s turns out to be the Laplace transform of the convolution.
The convolution, and that's one way of defining it, is the function of t you should put it there in order that its Laplace transform turn out to be the product of F of s times G of s. Now, I'll give you, in a moment, the formula for it.
But, I'll give you one and a quarter minutes, well, two minutes of motivation as to why there should be such formula.
Now, I won't calculate this out to the end because I don't have time.
But, here's the reason why there should be such formula.
And, you might suspect, and therefore it would be worth looking for.
It's because, remember, I told you where the Laplace transform came from, that the Laplace transform was the continuous analog of a power series.
So, when you ask a general question like that, the place to look for is if you know an analogous idea, say, does it work.
something like that work there?
So, here I have a power series summation, (a)n x to the n. Remember, you can write this in computer notation as a of n to make it look like f of n, f of t. And, the analog is turned into t when you turn a power series into the Laplace transform, and x gets turned into e to the negative s, and one formula just turns into the other.
Okay, so, there's a formula for F of x.
This is the analog of the Laplace transform.
And, similarly, G of x here is summation (b)n x to the n. Now, again, the naīve question would be, well, suppose I multiply the two corresponding coefficients together, and add up that power series, summation (a)n (b)n times x to the n. Is that somehow, that sum related to F and G?
And, of course, everybody knows the answer to that is no.
It has no relation whatever.
But, suppose instead I multiply these two guys.
In that case, I'll get a new power series.
I don't know what its coefficients are, but let's write them down.
Let's just call them (c)n's.
So, what I'm asking is, this corresponds to the product of the two Laplace transforms.
And, what I want to know is, is there a formula which says that (c)n is equal to something that can be calculated out of the (a)i and the (b)j.
Now, the answer to that is, yes, there is.
And, the formula for (c)n is called the convolution.
Now, you could figure out this formula yourself.
You figure it out.
Anyone who's smart enough to be interested in the question in the first place is smart enough to figure out what that formula is.
And, it will give you great pleasure to see that it's just like the formula for the convolution of going to give you now.
So, what is that formula for the convolution?
Okay, hang on.
Now, you are not going to like it.
But, you didn't like the formula for the Laplace transform, either.
You felt wiser, grown-up getting it.
But it's a mouthful to swallow.
It's something you get used to slowly.
And, you will get used to the convolution equally slowly.
So, what is the convolution of f of t and g of t?
It's a function calculated according to the corresponding formula.
It's a function of t. It is the integral from zero to t of f of u, -- u is a dummy variable because it's going to be integrated out when I do the integration, g of (t minus u) dt.
That's it.
I didn't make it up.
I'm just varying the bad news.
Well, what do you do when you see a formula?
Well, the first thing to do, of course, is try calculating just to get some feeling for what kind of a thing, you know.
Let's try some examples.
Let's see, let's calculate, what would be a modest beginning?
Let's calculate the convolution of t with itself.
Or, better yet, let's calculate the convolution just so that you could tell the difference, t with t squared, t squared with t, to make it a little easier.
By the way, the convolution is symmetric.
f star g is the same thing as g star f. Let's put that down explicitly.
I forgot to last period.
So, tell all the guys who came to the one o'clock lecture that you know something that they don't.
Now, that's a theory.
It's commutative.
This operation is commutative, in other words.
Now, that has to be a theorem because the formula is not symmetric.
The formula does not treat f and g equally.
And therefore, this is not obvious.
It's at least not obvious if you look at it that way, but it is obvious if you look at it that way.
Why?
In other words, f star g is the guy whose Laplace transform is F of s times G of s. Well, what would g star f?
That would be the guy whose Laplace transform is G times F. But capital F times capital G is the same as capital G times capital F. So, it's because the Laplace transforms are commutative.
Ordinary multiplication is commutative.
It follows that this has to be commutative, too.
So, I'll write that down, since F times G is equal to GF.
And, you have to understand that here, I mean that these are the Laplace transforms of those guys.
But, it's not obvious from the formula.
Okay, let's calculate the Laplace transform of, sorry, the convolution of t star, let's do it by the formula.
All right, by the formula, I calculate integral zero to t. Now, I take the first function, but I change its variable to the dummy variable, u.
So, that's u squared.
I take the second function and replace its variable by u minus t. So, this is times t minus u, sorry.
Okay, do you see that to calculate this is what I have to write down?
That's what the formula becomes.
Anything wrong?
Oh, sorry, the du, the integration's with expect to u, of course.
Thanks very much.
Okay, let's do it.
So, it is, integral of u squared t is, remember, it's integrated with respect to u.
So, it's u cubed over three times t. The rest of it is the integral of u cubed, which is u to the forth over four.
All this is to be evaluated between zero and t at the upper limit.
So, I put u equal t, I get t to the forth over three minus t to the forth over four.
Of course, at the lower limit, u is zero.
So, both of these are terms of zero.
There's nothing there.
And, the answer is, therefore, t to the forth divided by, a third minus a quarter is a twelfth.
So, that's doing it from the formula.
But, of course, there is an easier way to do it.
We can cheat and use the Laplace transform instead.
If I Laplace transform it, the Laplace transform of t squared is what?
It's two factorial divided by s cubed.
The Laplace transform of t is one divided by s squared.
And so, because this is the convolution of these, it should correspond to the product of the Laplace transforms, which is two over s to the 5th power.
Well, is that the same as this?
What's the Laplace transform of, in other words, what's the inverse Laplace transform of two over s to the fifth?
Well, the inverse Laplace transform of four factorial over s to the fifth is how much?
That's t to the forth, right?
Now, how does this differ?
Well, to turn that into that, I should divide by four times three.
So, this should be one twelfth t to the forth, one over four times three because this is 24, and that's two, so, divide by 12 to determine what constant, yeah.
So, it works, at least in that case.
But now, notice that this is not an ordinary product.
The convolution of t squared and t is not something like t cubed.
It's something like t to the forth, and there's a funny constant in there, too, very unpredictable.
Let's look at the convolution.
Let's take another example of the convolution.
Let's do something really humble just assure you that this, even at the simplest example, this is not trivial.
Let's take the convolution of f of t with one.
Can you take, yeah, one is a function just like any function.
But, you get something out of the convolution, yes, yes.
Let's just write down the formula.
Now, I can't use the Laplace transform here because you won't know what to do with it.
You don't have that formula yet.
It's a secret one that only I know.
So, let's do it.
Let's calculate it out the way it was supposed to.
So, it's the integral from zero to t of f of u, and now, what do I do with that one?
I'm supposed to take, one is the function g of t, and wherever I see a t, I'm supposed to plug in t minus u.
Well, I don't see any t there.
But that's something for rejoicing.
There's nothing to do to make the substitution.
It's just one.
So, the answer is, it's this curious thing.
The convolution of a function with one, you integrate it from zero to t. Well, as they said in Alice in Wonderland, things are getting curiouser and curiouser.
I mean, what is going on with this crazy function, and where are we supposed to start with it?
Well, I'm going to prove this for you, mostly because the proof is easy.
In other words, I'm going to prove that that's true.
And, as I give the proof, you'll see where the convolution is coming from.
That's number one.
And, number two, the real reason I'm giving you the proof: because it's a marvelous exercise in changing the variables in a double integral.
Now, that's something you all know how to do, even the ones who are taking 18.02 concurrently, and I didn't advise you to do that.
But, I've arranged the course so it's possible to do.
But, I knew that by the time we got to this, you would already know how to change variables at a double integral.
So, and in fact, you will have the advantage of remembering how to do it because you just had it about a week or two ago, whereas all the other guys, it's something dim in their distance.
So, I'm reviewing how to change variables at a double integral.
I'm showing you it's good for something.
So, what we are out to try to prove is this formula.
Let's put that down in, so you understand.
Okay, let's do it.
Now, we'll use the desert island method.
So, you have as much time as you want.
You're on a desert island.
In fact, I'm going to even go it the opposite way.
I'm going to start with-- you've got a lot of time on your hands and say, gee, I wonder if I take the product of the Laplace transforms, I wonder if there's some crazy function I could put in there, which would make things work.
You've never heard of the convolution.
You're going to discover it all by yourself.
Okay, so how do you begin?
So, we'll start with the left hand side.
We're looking for some nice way of calculating that as the Laplace transform of a single function.
So, the way to begin is by writing out the definitions.
We couldn't use anything else since we don't have anything else to use.
Now, looking ahead, I'm going to not use t. I'm going to use two neutral variables when I calculate.
After all, the t is just a dummy variable anyway.
You will see in a minute the wisdom of doing this.
So, it's this times the integral, which gives the Laplace transform of g. So, that's e to the negative s v, let's say, times g of v, dv.
Okay, everybody can get that far.
But now we have to start looking.
Well, this is a single integral, an 18.01 integral involving u, and this is an 18.01 integral involving v. But when you take the product of two integrals like that, remember when you evaluate a double integral, there's an easy case where it's much easier than any other case.
If you could write the inside, if you are integrating over a rectangle, for example, and you can write the integral as a product of a function just of u, and a product of a function just as v, then the integral is very easy to evaluate.
You can forget all the rules.
You just take all the u part out, all the v part out, and integrate them separately, a to b, c to d. That's the easy case of evaluating a double integral.
It's what everybody tries to do, even when it's not appropriate.
Now, here it is appropriate, except I'm going to use it backwards.
This is the result of having done that.
If this is the result of having done it, what was the step just before it?
Well, I must have been trying to evaluate a double integral as u runs from zero to infinity and v runs from zero to infinity, of what?
Well, of the product of these two functions.
Now, what is that?
e to the minus s u times e to the minus s v. Well, I must surely want to combine those.
e to the minus s u times e to the minus s v. And, what's left?
Well, what gets dragged along?
du dv.
This is the same as that because of that law I just gave you this is the product of a function just of u, and a function just of v. And therefore, it's okay to separate the two integrals out that way because I'm integrating sort of a rectangle that goes to infinity that way and infinity that way.
But, what I'm integrating is over the plane, in other words, this region of the plane as u, v goes from zero to infinity, zero to infinity.
Now, let's take a look.
What are we looking for?
Well, we're looking for, we would be very happy if u plus v were t. Let's make it t. In other words, I'm introducing a new variable, t, u plus v, and it's suggested by the form in which I'm looking for the answer.
Now, of course you then have to, we need another variable.
We could keep either u or v. Let's keep u.
That means v, we just gave a musical chairs.
v got dropped out.
Well, we can't have three variables.
We only have room for two.
But, we will remember it.
Rest in peace, v was equal to t minus u in case we ever need him again.
Okay, let's now put in the limits.
Let's put in the integral, the rest of the change of variable.
So, I'm now changing it to these new variables, t and u, so it's e to the negative s t. Well, f of u I don't have to do anything to.
But, g of v, I'm not allowed to keep v, so v has to be changed to t minus u. Amazing things are happening.
Now, I want to change this to an integral du dt.
Now, for that, you have to be a little careful.
We have two things to do to figure out this; what goes with that?
And, we have to put in the limits, also.
Now, those are the two nontrivial operations, when you change variables in a double integral.
So, let's be really careful.
Let's do the easier of the two, first.
I want to change from du dv to du dt.
And now, to do that, you have to put in the Jacobian matrix, the Jacobian determinant.
Ah-ha!
How many of you forgot that?
I won't even bother asking.
Oh, come on, you only lose two points.
It doesn't matter if you put it in the Jacobian.
As you see, you're going to forget something.
You will lose less credit for forgetting than anything else.
So, it's the Jacobian of u and v with respect to u and t. So, to calculate that, you write u equals u, v equals t minus u, and then the Jacobian is the partial of the matrix, the determinant of partial derivatives.
So, it's the determinant whose entries are the partial of u with respect to u, the partial of u with respect to t, but these are independent variables.
So, that's zero.
The partial of v with respect to u is negative one.
The partial of v with respect to t is one.
So, the Jacobian is one.
So, if you forgot it, no harm.
So, the Jacobian is one.
Now, more serious, and in some ways, I think, for most of you, the most difficult part of the operation, is putting in the new limits.
Now, for that, you look at the region over which you're integrating.
I think I'd better do that carefully.
I need a bigger picture.
That's really what I'm trying to say.
So, here's the (u, v) coordinates.
And, I want to change these to (u, t) coordinates.
The integration is over the first quadrant.
So, what you do is, when you do the integral, the first step is u is varying, and t is held fixed.
So, in the first integration, u varies.
t is held fixed.
Now, what is holding t fixed in this picture mean?
Well, t is equal to u plus v. So, u plus v is fixed, is a constant, in other words.
Now, where are the curves along which u plus v is a constant?
Well, they are these lines.
These are the lines along which u plus v equals a constant, or t is a constant.
The reason I'm holding t a constant is because the first integration only allows u to change.
t is held fixed.
Okay, you let u increase.
As u increases, and t is held fixed, I'm traversing these lines in this direction.
That's the direction on which u is increasing.
I integrate from the point, from the u value where they leave the region.
And, to enter the region, what's the u value where they enter the region?
u is equal to zero.
Everybody would know that.
Not so many people would be able to figure out what to put for where it leaves the region.
What's the value of u when it leaves the region?
Well, this is the curve, v equals zero.
But, v equals zero is, in another language, u equals t. t minus u equals zero, or u equals t. In other words, they enter the region where u equals zero, and they leave where u is t, has the value of t. And, how about the other guys?
For which t's do I want to do this?
Well, I want to do it for all these t values.
Well, now, the t value here, that's the starting one.
Here, t is zero, and here t is not zero.
And, if I go out and cover the whole first quadrant, I'll be letting t increase to infinity.
The sum of u and v, I will be letting increase to infinity.
So, it's zero to infinity.
So, all this is an exercise in taking this double integral in (u, v) coordinates, and changing it to this double integral, an equivalent double integral over the same region, but now in (u, t) coordinates.
And now, that's the answer.
Somewhere here is the answer because, look, since the first integration is with respect to u, this guy can migrate outside because it doesn't involve u.
That only involves t, and t is only caught by the second integration.
So, I can put this outside.
And, what do I end up with?
The integral from zero to infinity of e to the negative s t times, what's left?
A funny expression, but you're on your desert island and found it.
This funny expression, integral from zero to t, f of u, g of t minus u vu, in short, the convolution, exactly the convolution.
So, all you have to do is get the idea that there might be a formula, sit down, change variables and double integral it, ego, you've got your formula.
Well, I would like to spend much of the rest of the period--- in other words, that's how it relates to the Laplace transform.
That's how it comes out of the Laplace transform.
Here's how to use it, calculate it either with the Laplace transform or directly from the integral.
And, of course, you will solve problems, Laplace transform problems, differential equations using the convolution.
But, I have to tell you that most people, convolution is very important.
And, most people who use it don't use it in connection with the Laplace transform.
They use it for its own sake.
The first place I learned that outside of MIT people used a convolution was actually from my daughter.
She's an environmental engineer, an environmental consultant.
She does risk assessment, and stuff like that.
But anyway, she had this paper on acid rain she was trying to read for a client, and she said something about calculating acid rain falls on soil.
And then, from there, the stuff leeches into a river.
But, things happen to it on the way.
Soil combines in various ways, reduces the acidity, and things happen.
Chemical reactions take place, blah, blah, blah, blah.
Anyways, she said, well, then they calculated in the end how much the river gets polluted.
But, she said it's convolution.
She said, what's the convolution?
So, I told her she was too young to learn about the convolution.
And she knows that I thought I'd better look it up first.
I mean, I, of course, knew the convolution was, but I was a little puzzled at that application of it.
So, I read the paper.
It was interesting.
And, in thinking about it, other people have come to me, some guy with a problem about, they drilled ice cores in the North Pole, and from the radioactive carbon and so on, deducing various things about the climate 60 billion years ago, and it was all convolution.
He asked me if I could explain that to him.
So, let me give you sort of all-purpose thing, a simple all-purpose model, which can be adapted, which is very good way of thinking of the convolution, in my opinion.
It's a problem of radioactive dumping.
It's in the notes, by the way.
So, I'm just, if you want to take a chance, and just listen to what I'm saying rather that just scribbling everything down, maybe you'll be able to figure it out for the notes, also.
So, the problem is we have some factory, or a nuclear plant, or some thing like that, is producing radioactive waste, not always at the same rate.
And then, it carts it, dumps it on a pile somewhere.
So, radioactive waste is dumped, and there's a dumping function.
I'll call that f of t, the dump rate.
That's the dumping rate.
Let's say t is in years.
You like to have units, and quantity, kilograms, I don't know, whatever you want.
Now, what does the dumping rate mean?
The dumping rate means that if I have two times that are close together, for example, two successive days, midnight on two successive days, then there's a time interval between them.
I'll call that delta t. To say the dumping rate is f of t means that the amount dumped in this time interval, in the time interval from t1 to t1 plus one is approximately, not exactly, because the dumping rate isn't even constant within this time interval.
But it's approximately the dumping rate times the time over which the dumping is taking place.
That's what I mean by the dump rate.
And, it gets more and more accurate, the smaller the time interval you take.
Okay, now here's my problem.
The problem is, you start dumping at time t equals zero.
At time t equal t, how much radioactive waste is in the pile?
Now, what makes that problem slightly complicated is radioactive waste decays.
If I put some at a certain day, and then go back several months later and nothing's happened in between, I don't have the same amount that I dumps because a fraction of it decayed.
I have less.
And, our answer to the problem must take account of, for each piece of waste, how long it has been in the pile because that takes account of how long it had to decay, and what it ends up as.
So, the calculation, the essential part of the calculation will be that if you have an initial amount of this substance, and it decays for a time, t, this is the amount left at time t. This is the law of radioactive decay.
You knew that coming into 18.03, although, it's, of course, a simple differential equation which produces it, but I'll assume you simply know the answer.
k depends on the material, so I'm going to assume that the nuclear plant dumps the same radioactive substance each time.
It's only one substance I'm calculating, and k is it.
So, assume the k is fixed.
I don't have to change from one k from one material to a k for another because it's mixing up the stuff, just one material.
Okay, and now let's calculate it.
Here's the idea.
I'll take the t-axis, but now I'm going to change its name to the u-axis.
You will see why in just a second.
It starts at zero.
I'm interested in what's happening at the time, t. How much is left at time t?
So, I'm going to divide up the interval from zero to t on this time axis into, well, here's u0, the starting point, u1, u2, let's make this u1.
Oh, curses!
u1, u2, u3, and so on.
Let's call this (u)n. So they're u(n + 1), not that it matters.
It doesn't matter.
Okay, now, the amount, so, what I'm going to do is look at the amount, take the time interval from ui to ui plus one.
This is a time interval, delta u. Divide it up into equal time intervals.
So, the amount dumped in the time interval from u(i) to u(i plus one) is equal to approximately f of u(i), the dumping function there, times delta u.
We calculated that before.
That's what the meaning of the dumping rate is.
By time t, how much has it decayed to?
It has decayed.
How much is left, in other words?
Well, this is the starting amount.
So, the answer is going to be it's f of (u)i times delta u times this factor, which tells how much it decays, so, time.
So, this is the starting amount at time (u)i.
That's when it was first dumped, and this is the amount that was dumped, times, multiply that by e to the minus k times, now, what should I put up in there?
I have to put the length of time that it had to decay.
What is the length of time that it had to decay?
It was dumped at u(i).
I'm looking at time, t, it decayed for time length t minus u i, the length of time it had all the pile.
So, the stuff that was dumped in this time interval, at time t when I come to look at it, this is how much of it is left.
And now, all I have to do is add up that quantity for this time, the stuff that was dumped in this time interval plus the stuff dumped in, and so on, all the way up to the stuff that was dumped yesterday.
And, the answer will be the total amount left at time, t, that is not yet decayed will be approximately, you add up the amount coming from the first time interval plus the amount coming, and so on.
So, it will be f of u(i), I'll save the delta u for the end, times e to the minus k times t minus u(i) times delta u.
So, these two parts represent the amount dumped, and this is the decay factor.
And, I had those up as I runs from, well, where did I start?
From one to n, let's say.
And now, let delta t go to zero, in other words, make this delta u go to zero, make this more accurate by taking finer and finer subdivisions.
In other words, instead of looking every month to see how much was dumped, let's look every week, every day, and so on, to make this calculation more accurate.
And, the answer is, this approach is the exact amount, which will be the integral.
This sum is a Riemann sum.
It approaches the integral from zero to, well, I'm adding it up from u1 equals zero to un equals t, the final value.
So, it will be the integral from the starting point to the ending point of f of u e to the minus k times t minus u to u.
That's the answer to the problem.
It's given by this rather funny looking integral.
But, from this point of view, it's entirely natural.
It's a combination of the dumping function.
This doesn't care what the material was.
It only wants to know how much was put on everyday.
And, this part, which doesn't care how much was put on each day, it just is an intrinsic constant of the material involving its decay rate.
And, this total thing represents the total amount.
And that is, what is it?
It's the convolution of f of t with what function?
e to the minus k t. It's the convolution of the dumping function and the decay function.
And, the convolution is exactly the operation that you have to have to do that.
Okay, so, I think this is the most intuitive physical approach to the meaning of the convolution.
In this particular, you can say, well, that's very special.
Okay, so it tells you what the meaning of the convolution with an exponential is.
But, what about the convolution with all the other functions we're going to have to use in this course.
They can all be interpreted just by being a little flexible in your approach.
I'll give you two examples of this, well, three.
First of all, I'll use it for, in the problem set I ask you about a bank account.
That's not something any of you are interested in.
Okay, so, suppose instead I dumped garbage -- -- undecaying.
So, something that doesn't decay at all, what's the answer going to be?
Well, the calculation will be exactly the same.
It will be the convolution of the dumping function.
The only difference is that now the garbage isn't going to decay.
So, no matter how long it's left, the same amount is going to be left at the end.
In other words, I don't want to exponential decay function.
I want to function, one, the constant function, one, because once I stick it on the pile, nothing happens to it.
It just stays there.
So, it's going to be the convolution of this one because this is constant.
It's undecaying -- -- by the identical reasoning.
And so, what's the answer going to be?
It's going to be the integral from zero to t of f of u du.
Now, that's an 18.01 problem.
If I dump with a dumping rate, f of u, and I dump from time zero to time t, how much is on the pile?
They don't give it.
They always give velocity problems, and problems of how to slice up bread loaves, and stuff like that.
But, this is a real life problem.
If that's the dumping rate, and you dump for t days from zero to time t, how much do you have left at the end?
Answer: the integral of f of u du from zero to t. I'll give you another example.
Suppose I wanted a dumping function, suppose I wanted a function, wanted to interpret something which grows like t, for instance.
All I want is a physical interpretation.
Well, I have to think, I'm making a pile of something, a metaphorical pile, we don't actually have to make a physical pile.
And, the thing should be growing like t. Well, what grows like t?
Not bacteria, they grow exponentially.
Before the lecture, I was trying to think of something.
So, I came up with chickens on a chicken farm.
Little baby chickens grow linearly.
All little animals, anyway, I've observed that babies grow linearly, at least for a while, thank God.
After a while, they taper off.
But, at the beginning, they eat every four hours or whatever.
And they eat the same amount, pretty much.
And, that adds up.
So, let's suppose this represents the linear growth of chickens, of baby chicks.
That makes them sound cuter, less offensive.
Okay, so, a farmer, chicken farmer, whatever they call them, is starting a new brood.
So anyway, the hens lay at a certain rate, and each of those are incubated.
And after a while, little baby chicks come out.
So, this will be the production rate for new chickens.
Okay, and it will be the convolution which will tell you at time, t, the number of kilograms.
We'd better do this in kilograms, I'm afraid.
Now, that's not as heartless as it seems.
The number of kilograms of chickens times t. [LAUGHTER] It really isn't heartless because, after all, why would the farmer want to know that?
Well, because a certain number of pounds of chicken eat a certain number of pounds of chicken feed, and that's how much he has to dump, must have to give them every day.
That's how he calculates his expenses.
So, he will have to know the convolution is, or better yet, he will hire you, who knows what the convolution is.
And you'll be able to tell him.
Okay, why don't we stop there and go to recitation tomorrow.
I'll be doing important things.
Today we are going to do a last serious topic on the Laplace transform, the last topic for which I don't have to make frequent and profuse apologies.
One of the things the Laplace transform does very well and one of the reasons why people like it, engineers like it, is that it handles functions with jump discontinuities very nicely.
Now, the OR function with a jump discontinuity is-- Purple.
Is a function called the unit step function.
I will draw a graph of it.
Even the graph is controversial, but everyone is agreed that it zero here and one there.
What people are not agreed upon is its value at zero.
And some people make it zero, some people make it one, and some equivocate like me.
I will leave it undefined.
It is u of t. It is called the unit step.
Because that is what it is.
And let's say we will leave u of zero undefined.
If that makes you unhappy, get over it.
Of course, we don't always want the jump to be at zero.
Sometimes we will want to jump in another place.
If I want the function to jump, let's say at the point a instead of jumping at zero, I am going to start doing what everybody does.
You put in the vertical lines, even though I have no meaning, whatever, but it makes the graph look more connected and a little easier to read.
So that function I will call u sub a is the function which jumps at the point a.
How shall I give its definition?
Well, you can see it is just the translation by a of the unit step function.
So that is the way to write it, u of t minus a.
Now I am not done.
There is a unit box function, which we will draw in general terms like this.
It gets to a, then it jumps up to one, falls down again at b and continues onto zero.
This happens between a and b.
And the value to which it arises is one.
I will call this the unit box.
It is a function of t, a very simple one but an important one.
And what would be the formula for the unit box function?
Well, in general, almost all of these functions, as you will see when you use jump discontinuity, the idea is to write them all cleverly using nothing but u of.
Because it is that will have the Laplace transformer.
The way to write this is (u)ab.
And, if you like, you can treat this as the definition of it.
Let's make it a definition.
Okay, three lines.
Or, better yet, a colon and two lines.
I am defining this to be, what would it be?
Make the unit step at a step up at a, but then I would continue at one all the time.
I should, therefore, step down at b.
Now, the way you step down is just by taking the negative of the unit step function.
I step down at b by subtracting u sub b of t .
In other words, it is u of t minus a minus u of t minus b.
And now I have expressed it entirely in terms of the unit step function.
That will be convenient when I want take the Laplace transform.
What is so good about these things?
Well, these functions, when you use them in multiplications, they transform other functions in a nice way.
Not transform.
That is not the right word.
They operate on them.
They turn them into other strange creatures, and it might be these strange creatures that you are interested in.
Let me just draw you a picture.
That will be good enough.
Suppose we have some function like that, f of t, what would the function u sub ab, I will put in the variable, t times f of t, what function would that be?
I am just going to draw its graph.
What would its graph be?
Well, in between a and b this function (u)ab of t has the value one.
All I am doing is multiplying f of t by one.
In short, I am not doing anything to it at all.
Outside of that interval, (u)ab has the value zero, so that zero times f of t makes zero.
And, therefore, outside of this it is zero.
The effect of multiplying an arbitrary function by this unit box function is, you wipe away all of its graph except the part between a and b.
Now, that is a very useful thing to be able to do.
Well, that is enough of that.
Now, let's get into the main topic.
That is just preliminary.
I will be using these functions all during the period, but the real topic is the following.
Let's calculate the Laplace transform of the unit step function.
Well, this is no very big deal.
It is the integral from zero to infinity e to the minus s t times y of t, dt.
But, look, when t is bigger than zero, this has the value one.
So it is the same of the Laplace transform of one.
In other words, it is one over s for positive values of s. Or, to make it very clear, the Laplace transform of one is exactly the same thing.
As you see, the Laplace transform really is not interested in what happens when t is less than zero because that is not part of the domain of integration, the interval of integration.
That is fine.
They both have Laplace transform of one over s. What is the big deal?
The big deal is, what is the inverse Laplace transform of one over s?
Will the real function please stand up?
Which of these two should I pick?
Up to now in the course we have been picking one just because I never made a fuss over it and one was good enough.
For today one is no longer going to be good enough.
And we have to first investigate the thing in a slightly more theoretical way because this problem, I have illustrated it on the inverse Laplace transform of one over x, but it occurs for any inverse Laplace transform.
Suppose I have, in other words, that a function f of t has as its Laplace transform capital F of s?
And now, I ask what the inverse Laplace transform of capital F of s is.
Well, of course you want to write f of t. But the same thing happens.
I will draw you a picture.
Suppose, in other words, that here is our function f of t. Well, one answer certainly is f of t. That is okay.
That is the answer we have been using up until now.
But, you see, I can complete this function in many other ways.
Suppose I haven't told you what it was for s less than zero.
Any of these possibilities all will produce the same Laplace transform.
In fact, I can even make it this.
That is okay.
Each of these, f of t with any one of these tails, all have the same Laplace transform.
Because the Laplace transform, remember the definition, integral zero to infinity, e to the negative s t, f of t, dt because the Laplace transform does not care what the function was doing for negative values of t. Now, if we have to have a unique answer -- And most of the time you don't because, in general, the Laplace transform is only used for problems for future time.
That is the way the engineers and physicists and other people who use it habitually think of it.
If your problem is starting now and going on into the future and you don't have to know anything about the past, that is a Laplace transform problem.
If you also have to know about the past, then it is a Fourier transform problem.
That is beyond the scope of this course, you will never hear that word again, but that is the difference.
We are starting at time zero and going forward.
All right.
It does not care what f of t was doing for negative values of t. And that gives us a problem when we try to make the Laplace transform unique.
Now, how will I make it unique?
Well, there is a simple way of doing it.
Let's agree that wherever it makes a difference, and most of the time it doesn't, but today it will, whenever it makes a difference we will declare, we will by brute force make our function zero for negative values of t. That makes it unique.
I am going to say that to make it unique, now, how do I make f of t zero for negative values of t?
The answer is multiply it by the unit step function.
That leaves it what it was.
It multiplies it by one for positive values but multiplies it by zero for negative values.
The answer is going to be u of t times f of t. That will be the function that will look just that way that I drew, but I will draw it once more.
It is the function that looks like this.
And when I do this, it makes the inverse Laplace transform unique.
Out of all the possible tails I might have put on f of t, it picks the least interesting one, the tail zero.
That is a start.
But what we have to do now is -- What I want is a formula.
What we are going to need is, as you see right even in the beginning, if for example, if I want to calculate the Laplace transform of this, what I would like to have is a nice Laplace transform for the translate.
If you translate a function, how does that effect this Laplace transform?
In other words, the formula I am looking for is -- I want to express the Laplace transform of f of t minus a.
In other words, the function translated, let's say a is positive, so I translate it to the right along the t axis by the distance a. I want a formula for this in terms of the Laplace transform of the function I started with.
Now, my first task is to convince you that, though this would be very useful and interesting, there cannot possibly be such a formula.
There is no such formula.
Why not?
Well, I think I will explain it over there since there is a little piece of board I did not use.
Waste not want not.
Why can't there be such a formula?
What is it we are looking for?
Let's take a nice average function f of t. It has a Laplace transform.
And now I am going to translate it.
Let's say this is the point negative a.
And so the corresponding point positive a will be around here.
I am going to translate it to the right by a.
What is it going to look like?
Well, then it is going to start here and is going to look like this dashy thing.
That is f of t minus a.
That is not too bad a picture.
It will do.
I just took that curve and shoved it to the right by a.
Now, why is it impossible to express the Laplace transform of the dashed line in terms of the Laplace transform of the solid line?
The answer is this piece.
I will write it this way.
The trouble is, this piece is not used for the Laplace transform of f of t. Why isn't it used?
Well, because it occurs to the left of the vertical axis.
It occurs for negative values of t. And the Laplace transform of f of t simply does not care what f of t was doing to the left of that line, for negative values of t. It does not enter into the integral.
It was not used when I calculated this piece of the curve.
It was not used when I calculated the Laplace transform of f of t. On the other hand, it is going to be needed.
It occurs here, after I shift it to the right.
It is going to be needed for the Laplace transform of f of t minus a, because I will have to start the integration here, and I will have to know what that is.
In other words, when I took the Laplace transform, I automatically lost all information about the function for negative values of t. If I am later going to want some of that information for calculating this, I won't have it and, therefore, there cannot be a formula expressing one in terms of the other.
Now, of course, that cannot be the answer, otherwise I would not have raised your expectations merely to dash them.
I don't want to do that, of course.
There is a formula, of course.
It is just, I want to emphasize that you must write it my way because, if you write it any other way, you are going to get into the deepest trouble.
The formula is-- the good formula, the right formula -- -- accepts the given.
It says look, we have lost that pink part of it.
Therefore, I can never recover that.
Therefore, I won't ask for it.
The translation formula I will ask for is not one for the Laplace transform of f of t minus a, but rather for the Laplace transform of this thing where I have wiped away that pink part from the translated function.
In other words, the function I am talking about now is the formula for, I will put it over here to show you the function what we are talking about.
It is the function f of, well, in terms of the pink function it is, I will have to reproduce some of that picture.
There is f of t. f of t minus a, then, looked like this.
And so the function I am looking for is, this is the thing translated, but when I get down to the corresponding, this is the point that corresponds to that one, I wipe it away and just go with zero after that.
So this is u of t minus a times f of t minus a times f of t minus a.
What is this Laplace transform?
Now that does have a simple answer.
The answer is it is e to the minus as, a funny exponential, times the Laplace transform of the original function.
Now, this formula occurs in two forms.
This one is not too bad looking.
The trouble is, when you want to solve differential equations you are going to be extremely puzzled because the function that you will have to take to do the calculation on will not be given to you in the form f of t minus a.
It will look sine t or t squared or some polynomial in t. It will not be written as t minus a.
What do you do?
If your function does not look like that but instead, in terms of symbols looks like this, you can still use the formula.
Just a trivial change of variable means that you can write it instead.
Now, this is one place, there is no way of writing the answer in terms of capital F of s. This is one of those cases where this notation is just no good anymore.
I am going to have to write it using the L notation.
The Laplace transform of f of, and wherever you see a t, you should write t plus a.
Basically, this is the same formula as that one.
But I will have to stand on my head for one minute to try to convince you of it.
I won't do that now.
I would like you just to take a look at the formula.
You should know what it is called.
There are a certain number of idiots who call this the exponential shift formula because on the right side you multiply by an exponential, and that corresponds to shifting the function.
Unfortunately, we have preempted that.
We are not going to call it this.
I will call it what your book calls it.
The difficulty is there is no universal designation for this formula, important as it is.
However, your book calls this t-axis translation formula.
Translation because I am translating on the t-axis.
And that is what I do to the function, essentially.
And this tells me what its new Laplace transform is.
The other formula, remember it?
The exponential shift formula, the shift or the translation occurs on the s-axis.
In other words, the formula said that F of s minus a, you do the translation in s variable corresponded to multiplying this by e to the a t. In other words, the formulas are sort of dual to each other.
This guy translates on the left side and multiplies by the exponential on the right.
The formula that you know translates on the right and multiplies by the exponential on the left.
What are we going to calculate?
I am trying to calculate, so I am trying to prove this first formula.
The second one will be an easy consequence.
I am trying to calculate the Laplace transform of that thing.
What is it?
Well, it is the integral from zero to infinity of e to the minus st times u of t minus a, f of t minus a times dt.
That is the formula for it.
But I am trying to express it in terms of the Laplace transform of f itself.
Now, it is trying to be the Laplace transform of f. The problem is that here, a (t minus a) occurs, which I don't like.
I would like that to be just a t. Now, in order not to confuse you, and this is what confused everybody, I will set t1 equal to t minus a. I will change the variable.
This is called changing the variable in a definite integral.
How do you change the variable in a definite integral?
You do it.
Well, let's leave the limits for the moment.
e to the minus s times -- Now, t, remember you can change the variable forwards, direct substitution, but now I have to use the inverse substitution.
It's trivial, but t is equal to t1 plus a.
To change this I must substitute backwards and make that t1 plus a.
How about the rest of it?
Well, this becomes u of t1.
This is f of t1.
I have to change the dt, too, but that's no problem.
dt1 equals dt because a is a constant.
That is dt1.
And the last step is to put in the limits.
Now, when t is equal to zero, t1 has the value negative a.
So this has to be negative a when t is infinity.
Infinity minus a is still infinity, so that is still infinity.
In other words, this changes to that.
These two things, whatever they are, they have the same value.
All I have done is changed the variable.
Make it change a variable.
But now, of course, I want to make this look better.
How am I going to do that?
Well, first multiply out the exponential and then you get a factor e to the minus s(t1).
That is good.
That goes with this guy.
Now I get a factor e to the minus s times a from the exponential law.
But that does not have anything to do with the integral.
It is a constant as far as the integral is concerned because it doesn't involve t1.
And, therefore, I can pull it outside of the integral sign.
And write that e to the minus s times a.
Let's write it the other way.
Times the integral of what?
Well, e to the negative st1.
Now, u of t1, f of t1 times dt1.
Still integrated from minus a to infinity.
And now the final step.
This u of t1 is zero for negative values of t. And, therefore, it is equal to one for positive values of t. It is equal to zero for negative values of t, which means I can forget about the part of the integral that goes from negative a to zero.
I better rewrite this.
Okay, leave that.
In other words, this is equal to e to the minus as times the integral from zero to infinity of e to the minus s -- Let me do the shifty part now.
And this is since u of t1 is equal to zero for t1 less than zero.
That is why I can replace this with zero.
Because from negative a to zero, nothing is happening.
The integrand is zero.
And why can I get rid of it after that?
Well, because it is one after that.
And what is this thing?
This is the Laplace transform.
No, it is not the Laplace transform they said.
Because you had t1 there, not t. It is the Laplace transform because this is a dummy variable.
The t1 is integrated out.
It is a dummy variable.
It doesn't matter what you call it.
It is still the Laplace transform if I make that wiggly t or t star or tau or u. I can call it anything I want and it is still the Laplace transform of f of t. What is the answer?
That is e to the negative as times the Laplace transform of the function f. That is what I promised you in that formula.
Now, how about the other formula?
Well, let's look at that quickly.
That is, as I say, just sleight of hand.
But since that is the formula you will be using at least half the time you better learn it.
This little sleight of hand is also reproduced in one page of notes that I give you, but maybe you will find it easy to understand if I talk it out loud.
The problem now is for the second formula.
I am going to have to recopy out the first one in order to make the argument in a form in which you will understand it, I hope.
This goes to e to the minus as F s, except I am now going to write that not in F of s; since I will not be able to write the second formula using F of s, I am not going to write the first formula that way either.
I will write it as the Laplace transform with f of t. Now, formally if somebody says, okay, how do I calculate the Laplace transform of this thing?
I say put down this.
Well, that has no t in it.
It doesn't have the f in it either.
Then write this.
What formula did I do?
I looked at that and changed t minus a to t. Now, how did I change t minus a?
The way to say it is I changed t. Because the t is always there.
t to t plus a.
You get this by replace t by t plus a to get the right-hand side.
I replace this t by t plus a, and that turns this into f of t. And that is the f of t that went in there.
That is the universal rule for doing it.
Now I am going to use that same rule for transforming u of t minus a times f of t. See, the problem is now I have a function like t squared or sine t, which is not written in terms of t minus a.
And I don't know what to do with it.
The answer is, by brute force, write it in terms of t minus a.
What is brute force?
Brute force is the following.
I am going to put a t minus a there if it kills me.
t minus a plus a.
No harm in that, is there?
Now there is a t minus a there, just the way there was up there.
And now what is the rule?
I am just going to follow my nose.
What's sauce for the goose is sauce for the gander.
Minus as, Laplace transform of f of, now what am I going to write here?
Wherever I see a t, I am going to change it from t plus a.
Here I see a t. I will change that to t plus a.
What do I have?
t plus a minus a plus a, well, if you can keep count, what does that make?
It makes t plus a in the end.
The peace that passeth understanding.
Let's do some examples and suddenly you will breathe a sigh of relief that this all is doable anyway.
Let's calculate something.
I hope I am not covering up any crucial, yes I am.
I am covering up the u of t's, but you know that by now.
Let's see.
What should we calculate first?
What I just covered up.
Let's calculate the Laplace transform of (u)ab of t. What is that going to be?
Well, first of all, write out what it is in terms of the unit step function.
Remember that formula?
There.
Now you see it.
Now you don't.
Its Laplace transform is going to be what?
Well, the Laplace transform of t minus a, that is a special case here where this function is one.
Well, that one.
Either one.
It makes no difference.
It is simply going to be the Laplace transform of what f of t would have been, which is -- See, the Laplace transform of u of t is what?
That's one over s, right?
Because this is the function one, and we don't care the fact that it is zero or negative values of t. That is my f of s. And so I multiply it by e to the minus as times one over s. I am using this formula, e to the minus as times the Laplace transform of the unit step function, which is one over s. How about the translation?
That was taken care of by the exponential factor.
And it's minus because this is minus.
The same thing with the b.
This is the Laplace transform of the unit box function.
It looks a little hairy.
You will learn to work with it, don't worry about it.
How about the Laplace transform of -- Okay.
Let's use the other formula.
What would be the Laplace transform of u of t minus one times t squared, for example?
See, if I gave this to you and you only had the first formula, you would say, hey, but there is no t minus one in there.
There is only t squared.
What am I supposed to do?
Well, some of you might dig way back into high school and say every polynomial can be written in powers of t minus one, that is what I will do.
That would give the right answer.
But in case you had forgotten how to do that, you don't have to know because you could use the other formula instead, which, by the way, is the way you do it.
What are we going to do?
It goes into e to the minus s. The a is one in this case, plus one.
e to the minus s times the Laplace transform of what function?
Change t to t plus one.
The Laplace transform of t plus one squared.
What is that?
That is e to the minus s times the Laplace transform of t squared plus 2t plus one.
What's that?
Well, by the formulas which I am not bothering to write on the board anymore because you know them, it is e to the minus s times -- Laplace transform of t squared is two factorial over s cubed.
Remember you always have to raise the exponent by one.
This is two factorial, but that is the same as two.
Plus two.
This two comes from there.
The Laplace transform of t is one over s squared.
And, finally, the Laplace transform of one is one over s. You mean all that mess from this simple-looking function?
This function is not so simple.
What is its graph?
What is it we are calculating the Laplace transform of?
Well, it is the function t squared.
But multiplying it by that factor u of t minus one means that the only part of it I am using is this part, because u of t minus one is one when t is bigger than one.
But when t is less than one it is zero.
That function doesn't look all that simple to me.
And that is why its Laplace transform has three terms in it with this exponential factor.
Well, it is a discontinuous function.
And it gets discontinuous at a very peculiar spot.
You have to expect that.
Where in this does it tell you it becomes discontinuous at one?
It is because this is e to the minus one times s. This tells you where the discontinuity occurs.
The rest of it is just stuff you have to take because it is the function t squared.
It's what it is.
All right.
I think most of you are going to encounter the worst troubles when you try to calculate inverse Laplace transforms, so let me try to explain how that is done.
I will give you a simple example first.
And then I will try to give you a slightly more complicated one.
But even the simple one won't make your head ache.
We are going to calculate the inverse Laplace transform of this guy, one plus e to the negative pi s -- -- divided by s squared plus one.
All right.
Now, the first thing you must do is as soon as you see exponential factors in there like that you know that these functions, the answer is going to be a discontinuous function.
And you have got to separate out the different pieces of it that go with the different exponentials.
Because the way the formula works, it has to be used differently for each value of a.
Now, in this case, there is only one value of a that occurs.
Negative pi.
But it does mean that we are going to have to begin by separating out the thing into one over s squared plus one and this other factor e to the negative pi s divided by s squared plus one.
Now all I have to do is take the inverse Laplace transform of each piece.
The inverse Laplace transform of one over s squared plus one is -- Well, up to now we have been saying its sine t, right?
If you say it is sine t you are going to get into trouble.
How come?
We didn't get into trouble before.
Yes, but that was because there were no exponentials in the expression.
When there are exponentials you have to be more careful.
Make the inverse transform unique.
Make it not sine t, but u of t sine t. You will see why in just a moment.
If this weren't there then sine t would be perfectly okay.
With that factor there, you have got to put in the u of t, otherwise you won't be able to get the formula to work right.
In other words, I must use this particular one that I picked out to make it unique at the beginning of the period.
Otherwise, it just won't work.
Now, I know that is fine.
But now what is the inverse Laplace transform of e to the minus pi?
In other words, it is the same function, except I am now multiplying it by e to the negative pi s. Well, now I will use that formula.
My f of s is one over s squared plus one, and that corresponds it to sine t. If I multiply it by e to the minus pi s, just copy it down.
It now corresponds, the inverse Laplace transform, to what the left side says it does.
u of t minus pi times, in other words, this corresponds to that.
Then if I multiply it by e to the minus pi s, it corresponds to change the t to t minus pi.
What is the answer?
The answer is you sum these two pieces.
The first piece is u of time sine t. The second piece is u of t minus pi sine t of minus pi.
Now, if you leave the answer in that form it is technically correct, but you are going to lose a lot of credit.
You have to transform it to make it look good.
You have to make it intelligible.
You are not allowed to leave it in that form.
What could we do to it?
Well, you see, this part of it is interesting whenever t is positive.
This part of it is only interesting when t is greater than or equal to pi because this is zero.
Before that this is zero.
What you have to do is make cases.
Let's call the answer f of t. The function has to be presented in what is called the cases format.
That is what it is called when you type in tech, which I think a certain number of you can do anyway.
You have to make cases.
The first case is what happened between zero and pi?
Well, between zero and pi, only this term is operational.
The other one is zero because of that factor.
Therefore, between zero and pi the function looks like, now, I don't have to put in the u of t because that is equal to one.
It is equal to sine t between zero and pi.
What is it equal to bigger than pi?
Well, the first factor, the first term still obtains, so I have to include that.
But now I have to add the second one.
Well, what is the second term?
I don't include the t minus pi, u of t minus pi anymore because that is now one.
That has the value one.
It is sine of t minus pi.
But what is sine of t minus pi?
You take the sine curve and you translate it to the right by pi.
So what happens to it?
It turns into this curve.
In other words, it turns into the curve, what curve is that?
Minus sine t. The other factor, this factor is one and this becomes negative sine t. And so the final answer is f of t is equal to sine t between zero and pi and zero for t greater than or equal to pi.
That is the right form of the answer.
Well, today is the last day on Laplace transform and the first day before we start the rest of the term, which will be spent on the study of systems.
I would like to spend it on one more type of input function which, in general, your teachers in other courses will expect you to have had some acquaintance with.
It is the kind associated with an impulse, so an input consisted of what is sometimes called a unit impulse.
Now, what's an impulse?
It covers actually a lot of things.
It covers a situation where you withdraw from a bank account.
For example, take half your money out of a bank account one day.
It also would be modeled the same way.
But the simplest way to understand it the first time through is as an impulse, if you know what an impulse is.
If you have a variable force acting over time, and we will assume it is acting along a straight line so I don't have to worry about it being a vector, then the impulse, according to physicists, the physical definition, the impulse of f of t over some time interval.
Let's say the time interval running from a to b is, by definition, the integral from a to b of f of t dt.
Actually, I am going to do the most horrible thing this period.
I will assume the force is actually a constant force.
So, in that case, I wouldn't even have to bother with the integral at all.
If f of t is a constant, let's say capital F, then the impulse is -- Well, that integral is simply the product of the two, the impulse over that time interval is simply F times b minus a.
Just the product of those two.
The force times the length of time for which it acts.
Now, that is what I want to calculate, want to consider in connection with our little mass system.
So, once again, I think this is probably the last time you'll see the little spring.
Let's bid a tearful farewell to it.
There is our little mass on wheels.
And let's make it an undamped mass.
It has an equilibrium point and all the other little things that go with the picture.
And when I apply an impulse, what I mean is applying a constant force to this over a definite time interval.
And that is what I mean by applying an impulse over that time interval.
Now, what is the picture of such a thing?
Well, the force is only going to be applied, in other words, I am going to push on the mass or pull on the mass with a constant force.
With a little electromagnet here, we will assume, there is a pile of iron filings or something inside there.
I turn on the electromagnet.
It pulls with a constant force just between time zero and time two seconds.
And then I stop.
That is going to change the motion of the thing.
First it is going to start pulling it toward the thing.
And then, when it lets go, it will zoom back and there will be a certain motion after that.
What the question is, if I want to solve that problem of the motion of that in terms of the Laplace transform, how am I going to model this force?
Well, let's draw a picture of it first.
It starts here.
It is zero for t, let's say the force is applied between time zero to time h. And then its force is turned on, it stays constant and then it is turned off.
And those vertical lines shouldn't be there.
But, since in practice, it takes a tiny bit of time to turn a force on and off.
It is, in practice, not unrealistic to suppose that there are approximately vertical lines there.
They are slightly slanted but not too much.
Now, I want it to be unit impulse.
This is the force access and this is the time access.
Since the impulse is the area under this curve, if I want that to be one, then if this is h, the height to which I -- In other words, the magnitude of the force must be one over h in order that the area be one, in order, in other words, that this integral be one, the area under that curve be one.
So the unit impulse looks like that.
The narrower it is here, the higher it has to be that way.
The bigger the force must be if you want the end result to be a unit impulse.
Now, to solve a problem, a typical problem, then, would be a spring.
The mass is traveling on the track.
Let's suppose the spring constant is one, so there would be a differential equation.
And the right-hand side would be this f of t. Well, let's give it its name, the name I gave it before.
Remember, I called the unit box function the thing which was one between zero and h and zero everywhere else.
The notation we used for that was u, and then it had a double subscript from the starting point and the finishing point.
So oh-- u(oh) of t. This much represents the thing if it only rose to the high one.
But if it, instead, rises to the height one over h in order to make that area one, I have to multiply it by the factor one over h. Now, if you want to solve this by the Laplace transform.
In other words, see what the motion of that mass is as I apply this unit impulse to it over that time interval.
You have to take the Laplace transform, if that is the way we are doing it.
Now, the left-hand side is just routine and would involve the initial conditions.
The whole interest is taking the Laplace transform of the right-hand side.
And that is what I want to do now.
The problem is what is the Laplace transform of this guy?
Well, remember, to do everything else, you do everything by writing in terms of the unit step function?
This function that we are talking about is one over h times what you get by first stepping up to one.
That is the unit step function, which goes up by one and tries to stay at one ever after.
And then, when it gets to h, it has got to step down.
Well, the way you make it step down is by subtracting off the function, which is the unit step function but where the step takes place, not at time zero but at time h. In other words, I translate the unit step function of course with, I don't think I have to draw that picture again.
The unit step function looks like zing.
And if you translate it to the right by h it looks like zing.
And then make it negative to subtract it off.
And what you will get is this box function.
So we want to take the Laplace transform of this thing.
Well, let's assume, for the sake of argument that you didn't remember.
Well, you had to use the formula at 2:00 AM this morning and, therefore, you do remember it.
[LAUGHTER] So I don't have to recopy the formula onto the board.
Maybe if there is room there.
All right, let's put it up there.
It says that u of t minus a times f, any f, so let's call it g so you won't confuse it with this particular one, times g translated.
If you translate a function from t, if you translate it to the right by a then its Laplace transform is e to the minus a s times whatever the old Laplace transform was, g of s. Multiply by an exponential on the right.
On the left that corresponds to translation.
Except you must remember to put in that factor u for a secret reason which I spent half of Wednesday explaining.
What do we have here?
The Laplace transform of u of t, that is easy.
That is simply one over s. The Laplace transform of this other guy we get from the formula.
It is basically one over s. No, the Laplace transform of u of t. But because it has been translated to the right by h, I have to multiply it by that factor e to the minus h times s. That is the answer.
And, if you want to solve problems, this is what you would feed into the equation.
And you would calculate and calculate and calculate it.
But that is not what I want to do now because that was Wednesday and this is Friday.
You have the right to expect something new.
Here is what I am going to do new.
I am going to let h go to zero.
As h goes to zero, this function gets narrower and narrower, but it also has to get higher and higher because its area has to stay one.
What I am interested in, first of all, is what happens to the Laplace transform as h goes to zero.
In other words, what is the limit, as h goes to zero of -- Well, what is that function?
One minus e to the negative hs divided by hs.
Well, this is an 18.01 problem, an ordinary calculus problem, but let's do it nicely.
You see, the nice way to do it is to make a substitution.
We will change h s to u because it is occurring as a unit in both cases.
This is going to be the same as the limit as u goes to zero.
I think there are too many u's here already.
I cannot use u, you cannot use t, v is velocity, w is wavefunction.
There is no letter.
All right, u.
It is one minus e to the negative u over u.
So what is the answer?
Well, either you know the answer or you replace this by, say, the first couple of terms of the Taylor series.
But I think most of you would use L'Hopital's rule, so let's do that.
The derivative of the top is zero here.
The derivative by the chain rule of e to the negative u is e to the negative u times minus one.
And that minus one cancels that minus.
So the derivative of the top is simply e to the negative u and the derivative of the bottom is one.
So, as u goes to zero, that limit is one.
Interesting.
Let's draw a picture this way.
I will draw it schematically.
Up here is the function one over h times u zero h of t, our box function, except it has the height one over h instead of the height one.
We have just calculated that its Laplace transform is that funny thing, one minus e to the minus hs divided by hs.
That is the top line.
All this is completely kosher, but now I am going to let h go to zero.
And the question is what do we get now?
Well, I just calculated for you that this thing approaches one, has the limit one.
And now, let's fill in the picture.
What does this thing approach?
Well, it approaches a function which is zero everywhere.
As h approaches zero, this green box turns into a box which is zero everywhere except at zero.
And there, it is infinitely high.
So, keep going up.
Now, of course, that is not a function.
People call it a function but it isn't.
Mathematicians call it a generalized function, but that is not a function either.
It is just a way of making you feel comfortable by talking about something which isn't really a function.
It was given the name, introduced formally into mathematics by a physicist, Dirac.
And he, looking ahead to the future, did what many people do who introduce something into the literature, a formula or a function or something which they think is going to be important.
They never name it directly after themselves, but they always use as the symbol for it the first letter of their name.
I cannot tell you how often that has happened.
Maybe even Euler called e for that reason, although he claims it was in Latin because it has to do with exponentials.
Well, luckily his name began with an E, too.
That is Paul Dirac's delta function.
I won't dignify it by the name function by writing that out, by putting the world function here, too, but it is called the delta function.
From this point on, the entire rest of the lecture has a slight fictional element.
The entire rest of the lecture is in figurative quotation marks, so you are not entirely responsible for anything I say.
This is a non-function, but you put it in there and call it a function.
And you naturally want to complete, if it's a function then it must have a Laplace transform, even though it doesn't, so the diagram is completed that way.
And its Laplace transform is declared to be one.
So let's start listing the properties of this weird thing.
The delta function, its Laplace transform is one.
Now, one of the things is we have not yet expressed the fact that it is a unit impulse.
In other words, since the areas of all of these boxes, they all have areas one as they are shrunk this way they get higher that way.
By convention, one says that the area under the orange curve also remains one in the limit.
Now, how am I going to express that?
Well, it is done by the following formula that the integral, the total impulse of the delta function should be one.
Now, where do I integrate?
Well, from any place that it is zero to any place that it is zero on the other side of that vertical line.
But, in order to avoid controversy, people integrate all the way from negative infinity to infinity since it doesn't hurt.
Does it?
It is zero practically all the time.
This is the function whose Laplace transform is one.
Its integral from minus infinity to infinity is one.
How else can we calculate for it?
Well, I would like to calculate its convolution.
Here is f of t. What happens if I convolute it with the delta function?
Well, if you go back to the definition of the convolution, you know, it is that funny integral, you are going to do a lot of head scratching because it is not really all that clear how to integrate with the delta function.
Instead of doing that let's assume that it follows the laws of the Laplace transform.
In that case, its Laplace transform would be what?
Well, the whole thing of a convolution is that the Laplace transform of the convolution is the product of the two separate Laplace transforms.
So that is going to be F of s times the Laplace transform of the delta function, which is one.
Now, what must this thing be?
Well, there is some ambiguity as to what it is for negative values of t. But if we, by brute force, decide for negative values of t it is going to have the value zero, that is the way we make things unique.
In fact, why don't we make f of unique that way to start with?
This is a function now that is allowed to do anything it wants on the right-hand side of zero starting at zero, but on the left-hand side of zero it is wiped away and must be zero.
This is a definite thing now.
Its convolution is this.
And the inverse Laplace transform is -- The answer, in other words, is the same thing as what u of t f of t would be.
It's the same thing, F of s. And so, the conclusion is that these are equal, since they must be unique.
They have been made unique by making them zero for t negative.
In other words, apply to a function, well, I won't recopy it.
But the point is that delta t, for the convolution operation, is acting like an identity.
If I multiply, in the sense of convolution, it is a peculiar operation.
But algebraically, it has a lot of the properties of multiplication.
It is communitive.
It is linear in both factors.
In other words, it is almost anything you would want with multiplication.
It has an identity element, identity function.
And the identity function is the Dirac delta function.
Anything else here?
Yeah, I will throw in one more thing.
It would just require one more phony argument, which I won't bother giving you, but it is not totally implausible.
After all, u of t, the unit step function is not differentiable, is not a differentiable function.
It looks like this.
Here its derivative is zero, here its derivative is zero, and in this class it is not even defined in between.
But, I don't care, I will make it go straight up.
The question is what's its derivative?
Well, zero here, zero there and infinity at zero, so it must be the delta function.
That has exactly the right properties.
So the same people who will tell you this will tell you that also.
And, in fact, when you use it to solve differential equations it acts as if that is true.
I think I have given you an example on your homework.
Let me now show you a typical example of the way the Dirac delta function would be used to solve a problem.
Let's go back to our little spring, since it is the easiest thing.
You are familiar with it from a physical point of view, and it is the easiest thing to illustrate on.
We have our spring mass system.
Where is it?
Is it on the board?
Up there.
That one.
And the differential equation we are going to solve is y double prime plus y equals -- And now, I am going to assume that the spring is kicked with impulse a. I am not going to kick it at time t equals zero, since that would get us into slight technical difficulties.
Anyway, it is more fun to kick it at time pi over two.
The thing is, what is happening?
Well, we have got to have initial conditions.
The initial conditions are going to be, let's start at time zero.
We will start it at the position one.
So I take my spring, I drag it to the position one, I take the little mass there and then let it go.
And so it starts going birr.
But right when it gets to the equilibrium point I give it a, "cha!"
with unit impulse.
I started it from rest.
Those will be the initial conditions.
And I want to say that I kicked it, not with unit impulse, but with the impulse a.
Bigger.
And I did that at time pi over two.
So how are we going to say that?
Well, kick it means delivered that impulse over an extremely short time interval, but in such a way kicked it sufficiently hard that the total impulse was a.
The way to say that is kick it with the Dirac delta function.
Translate it to the point time pi over two.
Not at zero any longer.
t minus pi over two.
But that would kick it with a unit impulse.
I want it to kick it with the impulse a, so I will just multiply that by the constant factor a.
Let's put this over here.
y of zero equals one, that's the starting value.
Now we have a problem.
The only thing new in solving this with the Laplace transform is I have this funny right-hand side.
But it corresponds to a physical situation.
Let's do it.
You take the Laplace transform of both sides of the equation.
Remember how to do that?
You have to take account of the initial conditions.
The Laplace transform of the second derivative is you multiply by s squared, and then you have to subtract.
You have to use these initial conditions.
This one won't give you anything, but the first one means I have to subtract one times s. That is the Laplace transform of y double prime.
The Laplace transform of y, of course, is just capital Y.
And how about the Laplace transform of the right-hand side.
Well, we will have the constant factor a because the Laplace transform is linear.
And now, the delta function would have the transform one.
But when I translate it, pi over two, that means I have to use that formula.
Translate it by pi over two means take the one that it would have been otherwise and multiply it by e, that exponential factor.
It would be e to the minus pi over two, that is the A times s times one, which would be the g of s, the Laplace transform or the delta function before it had been translated.
But I don't have to put that in because it's one.
I am multiplying by one.
And to do everything now is routine.
Solve for the Laplace transform.
Well, what is it?
It is y is equal to.
I put the s on the other side.
That makes the right-hand side the sum of two terms.
And I divide by the coefficient of y, which is s squared plus one.
The s is over on the right-hand side and it is divided by s squared plus one.
And the other factor is there, too.
And it, too, is divided by s squared plus one.
Now, we take the inverse Laplace transform of those two terms and add them up.
What will we get?
Well, y is equal to, the inverse Laplace transform of s over s squared plus one is cosine t. Now, for this thing we will have to use our formula.
If this weren't here, the inverse Laplace transform of a over s squared plus one would be what?
Well, it would be a times the sine of t. In other words, if this is the g of s then the function on the left would be basically A sine t. But because it has been multiplied by that exponential factor, e to the minus as where a is pi over two, the left-hand side has to be changed from A sine t to what it would be with the translated form.
So the rest of it is u of t minus pi over two, because a is pi over two, times what it would have been just from the factor g of s itself.
In other words, A times the sine of, again, t minus pi over two.
I am applying that formula, but I am applying it in that direction.
I started with this, and I want to recover the left-hand side.
And that is what it must look like.
The A, of course, just gets dragged along for the free ride.
Now, as I emphasized to you last time, and I hope you did on your homework that you handed in, you mustn't leave it in that form.
You have to make cases because people will expect you to tell them what the meaning of this is.
Now, if t is less than pi over two, this is zero.
And, therefore, that term does not exist.
So the first part of it is just the cosine t term if t lies between zero and pi over two.
If t is bigger than pi over two then this factor is one.
It's the unit step function.
And I, therefore, must add in this term.
Now, what is that term?
What is the sine of t minus pi over two?
The sine of t looks like that.
The sine of t, if I translate it, looks like this.
If I translate it by pi over two.
And let's finish it up, the pi that was over here moved into position.
That curve is the curve negative cosine t. And so the answer is if t is bigger than pi over two, it is cosine t minus A times cosine t. Or, in other words, it is one minus A times cosine t. Now, do those match up?
They have always got to match up, or you have made a mistake.
You always have to get a continuous function when you have just discontinuities.
Do we get a continuous function?
Yeah, when t is pi over two the value here is zero.
The value of this is also zero at pi over two.
There is no conflict in the values.
Values doesn't suddenly jump.
The function is continuous.
It is not differential but it is continuous.
Well, what function does that look like?
There are cases.
It starts out life as the function cosine t. So it gets to here.
And at t equals pi over two, the mass gets kicked and that changes the function.
Now, what are the values?
Well, if A is bigger than one this is a negative number and it therefore becomes the function negative cosine t. Now, negative cosine t looks like this, the blue guy.
Negative cosine t is a function that looks like this.
So it goes from here, it reverses direction, the mass reverses direction from what you thought it was going to do.
And it does that because A is so large that that impulse was enough to make it reverse direction.
Of course it might only do this, but this is what will happen if A is bigger than one.
This will be A, which is a lot bigger than one.
If it's not so much bigger than one it might look like that.
So A is just bigger than one.
How's that?
Well, what if A is less than one?
Well, in that case it stays positive.
If A is less than one, this is now still a positive number.
And, therefore, the cosine continues on its merry way.
The only thing is it might be a little more sluggish or it might be very peppy and do that.
Let's just go that far.
This will be the case A less than one.
Well, of course, the most interesting case is what happens if A is exactly equal to one?
The porridge is exactly just right, I think that's the phrase.
Too hot.
Too cold.
Just right.
When A is equal to one, it is zero.
It starts out as cosine t. When it gets to t, it continues on ever after as the function zero.
I have a visual aid for the only time this term.
It didn't work at all.
I mean, on the other hand, the last hour, the people who worked it were not intrinsically baseball players, so we will use the equation of the pendulum instead.
That is a lot easier than mass spring.
This is a pendulum.
It is undamped because I declare it to be and it swings back and forth.
And here I am releasing it.
The variable is not x or y but theta, the angle through.
Here theta is one, let's say.
That's about one radian.
It starts there and swings back and forth.
It is not damped, so it never loses amplitude, particularly if I swish it, if I move my hand a little bit.
I want someone who knows how to bat a baseball.
That was the problem last hour.
Two people.
One to release it.
I will stand up and try to hold it here.
Somebody releases it.
And then somebody who has to be very skillful should apply a unit impulse of exactly one when it gets to the equilibrium point.
So who can do that?
Who can play baseball here?
Come on.
Somebody elected?
All right.
Come on.
[APPLAUSE] Somebody release it, too.
Somebody tall to handle it all.
I think that will be me.
Just hold it at what you would take to be one radian.
He releases it.
When it gets to the bottom, you will have to get way down, and maybe on this side.
Are you a lefty or a righty?
Rightly.
Okay.
Bat it what part.
Give it a good swat.
I will stand up higher.
Help.
I'm not very stable.
[APPLAUSE] A trial run.
Again.
Okay.
A little further out.
First of all, you have to see where it's going.
Why don't you stand, oh, you bat rightly.
That's right.
Okay.
Let's try it again.
Strike one.
It's okay.
It's the beginning of the baseball season.
One more.
The Red Sox are having trouble, too.
Not bad.
[APPLAUSE] If he had hit even harder it would have reversed direction and gone that way.
If you hadn't hit it quite as hard it would have continued on, still at cosine t, but with less amplitude.
But if you hit it exactly right -- It is fun to try to do.
Toomre in our department is a master at this, but he has been practicing for years.
He can take a little mallet and go blunk, and it stops absolutely dead.
It is unbelievable.
I should have had him give the lecture.
Now, I would like to do something truly serious.
Here, I guess.
Because there is a certain amount of engineering lingo you have to learn.
It is used by almost everybody.
Not architects and biologists probably quite yet, but anybody that uses the Laplace transform will use these words in connection with it.
I really think, since it is such a widespread technique, that these are things you should know.
Anyway, it will be easy.
It is just the enrichment of your vocabulary.
It is always fun to learn new vocabulary words.
So, let's just consider a general second order equation.
By the way, all this applies to higher order equations, too.
It applies to systems.
The same words are used, but let's use something that you know.
Here is a system.
It could be a spring mass dashpot system.
It could be an RLC circuit.
Or that pendulum, a damped pendulum, anything that is modeled by that differential equation with constant coefficients, second-order.
This is the input.
The input can be any kind of a function.
Exponential functions, sine, cosine.
It could be a Dirac delta function.
It could be a sum of these things.
It could be a Fourier series.
Anything of the sort of stuff we have been talking about throughout the last few weeks.
And let's have simple initial conditions so that doesn't louse things up, the simplest possible ones.
The mass starts at the equilibrium point from rest.
Of course, it doesn't stay that way because there is an input that is asking it to move along.
Now all I want to do is solve this in general with a Laplace transform.
If I do it in general, that is always easier than doing it in particular since you don't ever have to do any calculations.
It is s squared Y.
There are no other terms here because the initial conditions are zero.
This part will be a times s Y.
Again, no other terms because the initial conditions are zero.
Plus b times Y.
And all that is equal to whatever the Laplace transform is of the right-hand side.
So it is F of s. Next step.
Boy, this is an easy problem.
You solve for Y.
Well, Y is F of s times one over s squared plus as plus b.
Now, what is that?
The next step now is to figure out what the answer to the problem is, what's the Y of t?
Well, you do that by taking the inverse Laplace transform.
But because these are general functions, I don't have to write down any specific answer.
The only thing is to use the convolution because this is the product of two functions of s. The inverse transform will be the convolution of their respective things.
The answer is going to be the convolution of F of t, the input function in other words, convoluted with the inverse Laplace transform of that thing.
Now, we have to have a name for that, and those are the two words I want to introduce you to because they are used everywhere.
The function, on the right-hand side, this function one over s squared plus as plus b, notice it only depends upon the left-hand side of the differential equation, on the damping constant.
The spring constant if you are thinking of a mass spring dashpot system.
So this depends only on the system, not on what input is going into it.
And it is called the transfer function.
Is usually called capital W of, sometimes it is capital H of s, there are different things, but it is always called the transfer function.
What we are interested in putting here its inverse Laplace transform.
Well, I will call that W of t to go with the capital W of s by the usual notation.
Its inverse Laplace transform, well, I cannot calculate that.
I will just give it a name, W of t. And that is called the weight function of the system.
This is the transfer function of the system, so put in "of the system" if you are taking notes.
And so the answer is that always the solution is the convolution to this differential equation that we have been solving for the last three or four weeks.
It is the convolution of that.
And, therefore, the solution is expressed as a definite integral of the function of the input on the right-hand side, what is forcing the equation, times this magic function but flipped and translated by t. That says du for you guys over there.
In other words, the solution to the differential equation is presented as a definite integral.
Marvelous.
And the only thing is the definite integral involves this funny function W of t. To understand why that is the solution, you have to understand what W of t is.
Well, formally, of course, it's that.
But what does it really mean?
The problem is what is W of t really?
Not just formally, but what does it really mean?
I mean, is it real?
I think the simplest way of thinking of it, once you know about the delta function is just to think of this differential equation y double prime plus a y prime plus b.
Except use as the input the Dirac delta function.
In other words, we are kicking the mass.
The mass starts at rest, so the initial conditions are going to be what they were before.
y of zero, y prime of zero.
Both zero.
The mass starts at rest from the equilibrium position, and it is kicked in the positive direction, I guess that's this way, with unit impulse.
At time zero with unit impulse.
In other words, kick it just hard enough so you impart a unit impulse.
So that situation is modeled by this differential equation.
The kick at time zero is modeled by this input, the Dirac delta function.
And now, what happens if I solve it?
Well, you see, everything in the solution is the same.
The left stays the same, but on the right-hand side I should have not f of s here.
Since this is the delta function, I should have one.
What I get is, on the left-hand side, s squared Y plus as Y plus bY equals, for the Laplace transform of the right-hand side is simply one.
And, therefore, Y is what?
Y is one over exactly the transform function.
And therefore its inverse Laplace transform is that weight function.
That is the simplest interpretation I know of what this magic weight function is, which gives the solution to all the differential equations, no matter what the input is.
The weight function is the response of the system at rest to a sharp kick at time zero with unit impulse.
And read the notes because they will explain to you why this could be thought of as the superposition of a lot of sharp kicks times zero a little later.
Kick, kick, kick, kick.
And that's what makes the solution.
Next time we start systems.
For the rest of the term, we are going to be studying not just one differential equation at a time, but rather what are called systems of differential equations.
Those are like systems of linear equations.
They have to be solved simultaneously, in other words, not just one at a time.
So, how does a system look when you write it down?
Well, since we are going to be talking about systems of ordinary differential equations, there still will be only one independent variable, but there will be several dependent variables.
I am going to call, let's say two.
The dependent variables are going to be, I will call them x and y, and then the first order system, something involving just first derivatives, will look like this.
On the left-hand side will be x prime, in other words.
On the right-hand side will be the dependent variables and then also the independent variables.
I will indicate that, I will separate it all from the others by putting a semicolon there.
And the same way y prime, the derivative of y with respect to t, will be some other function of (x, y) and t. Let's write down explicitly that x and y are dependent variables.
And what they depend upon is the independent variable t, time.
A system like this is going to be called first order.
And we are going to consider basically only first-order systems for a secret reason that I will explain at the end of the period.
This is a first-order system, meaning that the only kind of derivatives that are up here are first derivatives.
So x prime is dx over dt and so on.
Now, there is still more terminology.
Of course, practically all the equations after the term started, virtually all the equations we have been considering are linear equations, so it must be true that linear systems are the best kind.
And, boy, they certainly are.
When are we going to call a system linear?
I think in the beginning you should learn a little terminology before we launch in and actually try to start to solve these things.
Well, the x and y, the dependent variables must occur linearly.
In other words, it must look like this, ax plus by.
Now, the t can be a mess.
And so I will throw in an extra function of t there.
And y prime will be some other linear combination of x and y, plus some other messy function of t. But even the a, b, c, and d are allowed to be functions of t. They could be one over t cubed or sine t or something like that.
So I have to distinguish those cases.
The case where a, b, c, and d are constants, that I will call -- Well, there are different things you can call it.
We will simply call it a constant coefficient system.
A system with coefficients would probably be better English.
On the other hand, a, b, c, and d, this system will still be called linear if these are functions of t. Can also be functions of t. So it would be a perfectly good linear system to have x prime equals tx plus sine t times y plus e to the minus t squared.
You would never see something like that but it is okay.
What else do you need to know?
Well, what would a homogenous system be?
A homogenous system is one without these extra guys.
That doesn't mean there is no t in it.
There could be t in the a, b, c and d, but these terms with no x and y in them must not occur.
So, a linear homogenous.
And that is the kind we are going to start studying first in the same way when we studied higher order equations.
We studied first homogenous.
You had to know how to solve those first, and then you could learn how to solve the more general kind.
So linear homogenous means that r1 is zero and r2 is zero for all time.
They are identically zero.
They are not there.
You don't see them.
Have I left anything out?
Yes, the initial conditions.
Since that is quite general, let's talk about what would initial conditions look like?
Well, in a general way, the reason you have to have initial conditions is to get values for the arbitrary constants that appear in the solution.
The question is, how many arbitrary constants are going to appear in the solutions of these equations?
Well, I will just give you the answer.
Two.
The number of arbitrary constants that appear is the total order of the system.
For example, if this were a second derivative and this were a first derivative, I would expect three arbitrary constants in the system -- -- because the total, the sum of two and one makes three.
So you must have as many initial conditions as you have arbitrary constants in the solution.
And that, of course, explains when we studied second-order equations, we had to have two initial conditions.
I had to specify the initial starting point and the initial velocity.
And the reason we had to have two conditions was because the general solution had two arbitrary constants in it.
The same thing happens here but the answer is it is more natural, the conditions here are more natural.
I don't have to specify the velocity.
Why not?
Well, because an initial condition, of course, would want me to say what the starting value of x is, some number, and it will also want to know what the starting value of y is at that same point.
Well, there are my two conditions.
And since this is going to have two arbitrary constants in it, it is these initial conditions that will satisfy, the arbitrary constants will have to be picked so as to satisfy those initial conditions.
In some sense, the giving of initial conditions for a system is a more natural activity than giving the initial conditions of a second order system.
You don't have to be the least bit cleaver about it.
Anybody would give these two numbers.
Whereas, somebody faced with a second order system might scratch his head.
And, in fact, there are other kinds of conditions.
There are boundary conditions you learned a little bit about instead of initial conditions for a second order equation.
I cannot think of any more general terminology, so it sounds like we are going to actually have to get to work.
Okay, let's get to work.
I want to set up a system and solve it.
And since one of the things in this course is supposed to be simple modeling, it should be a system that models something.
In general, the kinds of models we are going to use when we study systems are the same ones we used in studying just first-order equations.
Mixing, radioactive decay, temperature, the motion of temperature.
Heat, heat conduction, in other words.
Diffusion.
I have given you a diffusion problem for your first homework on this subject.
What else did we do?
That's all I can think of for the moment, but I am sure they will occur to me.
When, out of those physical ideas, are we going to get a system?
The answer is, whenever there are two of something that there was only one of before.
For example, if I have mixing with two tanks where the fluid goes like that.
Say you want to have a big tank and a little tank here and you want to put some stuff into the little tank so that it will get mixed in the big tank without having to climb a big ladder and stop and drop the stuff in.
That will require two tanks, the concentration of the substance in each tank, therefore, that will require a system of equations rather than just one.
Or, to give something closer to home, closer to this backboard, anyway, suppose you have dah, dah, dah, don't groan, at least not audibly, something that looks like that.
And next to it put an EMF there.
That is just a first order.
That just leads to a single first order equation.
But suppose it is a two loop circuit.
Now I need a pair of equations.
Each of these loops gives a first order differential equation, but they have to be solved simultaneously to find the current or the charges on the condensers.
And if I want a system of three equations, throw in another loop.
Now, suppose I put in a coil instead.
What is this going to lead to?
This is going to give me a system of three equations of which this will be first order, first order.
And this will be second order because it has a coil.
You are up to that, right?
You've had coils, inductance?
Good.
So the whole thing is going to count as first-order, first-order, second-order.
To find out how complicated it is, you have to add up the orders.
That is one and one, and two.
This is really fourth-order stuff that we are talking about here.
We can expect it to be a little complicated.
Well, now let's take a modest little problem.
I am going to return to a problem we considered earlier in the problem of heat conduction.
I had forgotten whether it was on the problem set or I did it in class, but I am choosing it because it leads to something we will be able to solve.
And because it illustrates how to add a little sophistication to something that was unsophisticated before.
A pot of water.
External temperature Te of t. I am talking about the temperature of something.
And what I am talking about the temperature of will be an egg that is cooking inside, but with a difference.
This egg is not homogenous inside.
Instead it has a white and it has a yolk in the middle.
In other words, it is a real egg and not a phony egg.
That is a small pot, or it is an ostrich egg.
[LAUGHTER] That is the yoke.
The yolk is contained in a little membrane inside.
And there are little yucky things that hold it in position.
And we are going to let the temperature of the yolk, if you can see in the back of the room, be T1.
That is the temperature of the yolk.
The temperature of the white, which we will assume is uniform, is going to be T2.
Oh, that's the water bath.
The temperature of the white is T2, and then the temperature of the external water bath.
In other words, the reason for introducing two variables instead of just the one variable for the overall temperature of the egg we had is because egg white is liquid pure protein, more or less, and the T1, the yolk has a lot of fat and cholesterol and other stuff like that which is supposed to be bad for you.
It certainly has different conducting.
It is liquid, at the beginning at any rate, but it certainly has different constants of conductivity than the egg white would.
And the condition of heat through the shell of the egg would be different from the conduction of heat through the membrane that keeps the yoke together.
So it is quite reasonable to consider that the white and the yolk will be at different temperatures and will have different conductivity properties.
I am going to use Newton's laws but with this further refinement.
In other words, introducing two temperatures.
Whereas, before we only had one temperature.
But let's use Newton's law.
Let's see.
The question is how does T1, the temperature of the yolk, vary with time?
Well, the yolk is getting all its heat from the white.
Therefore, Newton's law of conduction will be some constant of conductivity for the yolk times T2 minus T1.
The yolk does not know anything about the external temperature of the water bath.
It is completely surrounded, snug and secure within itself.
But how about the temperature of the egg white?
That gets heat and gives heat to two sources, from the external water and also from the internal yolk inside.
So you have to take into account both of those.
Its conduction of the heat through that membrane, we will use the same a, which is going to be a times T1 minus T2.
Remember the order in which you have to write these is governed by the yolk outside to the white.
Therefore, that has to come first when I write it in order that a be a positive constant.
But it is also getting heat from the water bath.
And, presumably, the conductivity through the shell is different from what it is through this membrane around the yolk.
So I am going to call that by a different constant.
This is the conductivity through the shell into the white.
And that is going to be T, the external temperature minus the temperature of the egg white.
Here I have a system of equations because I want to make two dependent variables by refining the original problem.
Now, you always have to write a system in standard form to solve it.
You will see that the left-hand side will give the dependent variables in a certain order.
In this case, the temperature of the yolk and then the temperature of the white.
The law is that in order not to make mistakes -- And it's a very frequent source of error so learn from the beginning not to do this.
You must write the variables on the right-hand side in the same order left to right in which they occur top to bottom here.
In other words, this is not a good way to leave that.
This is the first attempt in writing this system, but the final version should like this.
T1 prime, I won't bother writing dT / dt, is equal to -- T1 must come first, so minus a times T1 plus a times T2.
And the same law for the second one.
It must come in the same order.
Now, the coefficient of T1, that is easy.
That's a times T1.
The coefficient of T2 is minus a minus b, so minus (a plus b) times T2.
But I am not done yet.
There is still this external temperature I must put into the equation.
Now, that is not a variable.
This is some given function of t. And what the function of t is, of course, depends upon what the problem is.
So that, for example, what might be some possibilities, well, suppose the problem was I wanted to coddle the egg.
I think there is a generation gap here.
How many of you know what a coddled egg is?
How many of you don't know?
Well, I'm just saying my daughter didn't know.
I mentioned it to her.
I said I think I'm going to do a coddled egg tomorrow in class.
And she said what is that?
And so I said a cuddled egg?
She said why would someone cuddle an egg?
I said coddle.
And she said, oh, you mean like a person, like what you do to somebody you like or don't like or I don't know.
Whatever.
I thought a while and said, yeah, more like that.
[LAUGHTER] Anyway, for the enrichment of your cooking skills, to coddle an egg, it is considered to produce a better quality product than boiling an egg.
That is why people do it.
You heat up the water to boiling, the egg should be at room temperature, and then you carefully lower the egg into the water.
And you turn off the heat so the water bath cools exponentially while the egg inside is rising in temperature.
And then you wait four minutes or six minutes or whatever and take it out.
You have a perfect egg.
So for coddling, spelled so, what will the external temperature be?
Well, it starts out at time zero at 100 degrees centigrade because the water is supposed to be boiling.
The reason you have it boiling is for calibration so that you can know what temperature it is without having to use a thermometer, unless you're on Pike's Peak or some place.
It starts out at 100 degrees.
And after that, since the light is off, it cools exponential because that is another law.
You only have to know what K is for your particular pot and you will be able to solve the coddled egg problem.
In other words, you will then be able to solve these equations and know how the temperature rises.
I am going to solve a different problem because I don't want to have to deal with this inhomogeneous term.
Let's use, as a different problem, a person cooks an egg.
Coddles the egg by the first process, decides the egg is done, let's say hardboiled, and then you are supposed to drop a hardboiled egg into cold water.
Not just to cool it but also because I think it prevents that dark thing from forming that looks sort of unattractive.
Let's ice bath.
The only reason for dropping the egg into an ice bath is so that you could have a homogenous equation to solve.
And since this a first system we are going to solve, let's make life easy for ourselves.
Now, all my work in preparing this example, and it took considerably longer time than actually solving the problem, was in picking values for a and b which would make everything come out nice.
It's harder than it looks.
The values that we are going to use, which make no physical sense whatsoever, but a equals 2 and b equals 3.
These are called nice numbers.
What is the equation?
What is the system?
Can somebody read it off for me?
It is T1 prime equals, what is it, minus 2T1 plus 2T2.
That's good.
Minus 2T1 plus 2T2.
T2 prime is, what is it?
I think this is 2T1.
And the other one is minus a plus b, so minus 5.
This is a system.
Now, on Wednesday I will teach you a fancy way of solving this.
But, to be honest, the fancy way will take roughly about as long as the way I am going to do it now.
The main reason for doing it is that it introduces new vocabulary which everyone wants you to have.
And also, more important reasons, it gives more insight into the solution than this method.
This method just produces the answer, but you want insight, also.
And that is just as important.
But for now, let's use a method which always works and which in 40 years, after you have forgotten all other fancy methods, will still be available to you because it is method you can figure out yourself.
You don't have to remember anything.
The method is to eliminate one of the dependent variables.
It is just the way you solve systems of linear equations in general if you aren't doing something fancy with determinants and matrices.
If you just eliminate variables.
We are going to eliminate one of these variables.
Let's eliminate T2.
You could also eliminate T1.
The main thing is eliminate one of them so you will have just one left to work with.
How do I eliminate T2?
Beg your pardon?
Is something wrong?
If somebody thinks something is wrong raise his hand.
No?
Why do I want to get rid of T1?
Well, I can add them.
But, on the left-hand side, I will have T1 prime plus T2 prime.
What good is that?
[LAUGHTER] I think you will want to do it my way.
[APPLAUSE] Solve for T2 in terms of T1.
That is going to be T1 prime plus 2T1 divided by 2.
Now, take that and substitute it into the second equation.
Wherever you see a T2, put that in, and what you will be left with is something just in T1.
To be honest, I don't know any other good way of doing this.
There is a fancy method that I think is talked about in your book, which leads to extraneous solutions and so on, but you don't want to know about that.
This will work for a simple linear equation with constant coefficients, always.
Substitute in.
What do I do?
Now, here I do not advise doing this mentally.
It is just too easy to make a mistake.
Here, I will do it carefully, writing everything out just as you would.
T1 prime plus 2T1 over 2, prime, equals 2T1 minus 5 time T1 prime plus 2T1 over two.
I took that and substituted into this equation.
Now, I don't like those two's.
Let's get rid of them by multiplying.
This will become 4.
And now write this out.
What is this when you look at it?
This is an equation just in T1.
It has constant coefficients.
And what is its order?
Its order is two because T1 prime primed.
In other words, I can eliminate T2 okay, but the equation I am going to get is no longer a first-order.
It becomes a second-order differential equation.
And that's a basic law.
Even if you have a system of more equations, three or four or whatever, the law is that after you do the elimination successfully and end up with a single equation, normally the order of that equation will be the sum of the orders of the things you started with.
So two first-order equations will always produce a second-order equation in just one dependent variable, three will produce a third order equation and so on.
So you trade one complexity for another.
You trade the complexity of having to deal with two equations simultaneously instead of just one for the complexity of having to deal with a single higher order equation which is more trouble to solve.
It is like all mathematical problems.
Unless you are very lucky, if you push them down one way, they are really simple now, they just pop up some place else.
You say, oh, I didn't save anything after all.
That is the law of conservation of mathematical difficulty.
[LAUGHTER] You saw that even with the Laplace transform.
In the beginning it looks great, you've got these tables, take the equation, horrible to solve.
Take some transform, trivial to solve for capital Y.
Now I have to find the inverse Laplace transform.
And suddenly all the work is there, partial fractions, funny formulas and so on.
It is very hard in mathematics to get away with something.
It happens now and then and everybody cheers.
Let's write this out now in the form in which it looks like an equation we can actually solve.
Just be careful.
Now it is all right to use the method by which you collect terms.
There is only one term involving T1 double prime.
It's the one that comes from here.
How about the terms in T1 prime?
There is a 2.
Here, there is minus 5 T1 prime.
If I put it on the other side it makes plus 5 T1 prime plus this two makes 7 T1 prime.
And how many T1's are there?
Well, none on the left-hand side.
On the right-hand side I have 4 here minus 10.
4 minus 10 is negative 6.
Negative 6 T1 put on this left-hand side the way we want to do makes plus 6 T1.
There are no inhomogeneous terms, so that is equal to zero.
If I had gotten a negative number for one of these coefficients, I would instantly know if I had made a mistake.
Why?
Why must those numbers come out to be positive?
It is because the system must be, the system must be, fill in with one word, stable.
And why must this system be stable?
In other words, the long-term solutions must be zero, must all go to zero, whatever they are.
Why is that?
Well, because you are putting the egg into an ice bath.
Or, because we know it was living but after being hardboiled it is dead and, therefore, dead systems are stable.
That's not a good reason but it is, so to speak, the real one.
It's clear anyway that all solutions must tend to zero physically.
That's obvious.
And, therefore, the differential equation must have the same property, and that means that its coefficients must be positive.
All its coefficients must be positive.
If this weren't there, I would get oscillating solutions, which wouldn't go to zero.
That is physical impossible for this egg.
Now the rest is just solving.
The characteristic equation, if you can remember way, way back in prehistoric times when we were solving these equations, is this.
And what you want to do is factor it.
This is where all the work was, getting those numbers so that this would factor.
So it's r plus 1 times r plus 6 And so the solutions are, the roots are r equals negative 1.
I am just making marks on the board, but you have done this often enough, you know what I am talking about.
So the characteristic roots are those two numbers.
And, therefore, the solution is, I could write down immediately with its arbitrary constant as c1 times e to the negative t plus c2 times e to the negative 6t.
Now, I have got to get T2.
Here the first worry is T2 is going to give me two more arbitrary constants.
It better not.
The system is only allowed to have two arbitrary constants in its solution because that is the initial conditions we are giving it.
By the way, I forgot to give initial conditions.
Let's give initial conditions.
Let's say the initial temperature of the yolk, when it is put in the ice bath, is 40 degrees centigrade, Celsius.
And T2, let's say the white ought to be a little hotter than the yolk is always cooler than the white for a soft boiled egg, I don't know, or a hardboiled egg if it hasn't been chilled too long.
Let's make this 45.
Realistic numbers.
Now, the thing not to do is to say, hey, I found T1.
Okay, I will find T2 by the same procedure.
I will go through the whole thing.
I will eliminate T1 instead.
Then I will end up with an equation T2 and I will solve that and get T2 equals blah, blah, blah.
That is no good, A, because you are working too hard and, B, because you are going to get two more arbitrary constants unrelated to these two.
And that is no good.
Because the correct solution only has two constants in it.
Not four.
So that procedure is wrong.
You must calculate T2 from the T1 that you found, and that is the equation which does it.
That's the one we have to have.
Where is the chalk?
Yes.
Maybe I can have a little thing so I can just carry this around with me.
That is the relation between T2 and T1.
Or, if you don't like it, either one of these equations will express T2 in terms of T1 for you.
It doesn't matter.
Whichever one you use, however you do it, that's the way you must calculate T2.
So what is it?
T2 is calculated from that pink box.
It is one-half of T1 prime plus T1.
Now, if I take the derivative of this, I get minus c1 times the exponential.
The coefficient is minus c1, take half of that, that is minus a half c1 and add it to T1.
Minus one-half c1 plus c1 gives me one-half c1.
And here I take the derivative, it is minus 6 c2.
Take half of that, minus 3 c2 and add this c2 to it, minus 3 plus 1 makes minus 2.
That is T2.
And notice it uses the same arbitrary constants that T1 uses.
So we end up with just two because we calculated T2 from that formula or from the equation which is equivalent to it, not from scratch.
We haven't put in the initial conditions yet, but that is easy to do.
Everybody, when working with exponentials, of course, you always want the initial conditions to be when T is equal to zero because that makes all the exponentials one and you don't have to worry about them.
But this you know.
If I put in the initial conditions, at time zero, T1 has the value 40.
So 40 should be equal to c1 + c2.
And the other equation will say that 45 is equal to one-half c1 minus 2 c2.
Now we are supposed to solve these.
Well, this is called solving simultaneous linear equations.
We could use Kramer's rule, inverse matrices, but why don't we just eliminate.
Let me see.
If I multiply by, 45, so multiply by two, you get 90 equals c1 minus 4 c2.
Then subtract this guy from that guy.
So, 40 taken from 90 makes 50.
And c1 taken from c1, because I multiplied by two, makes zero.
And c2 taken from minus 4 c2, that makes minus 5 c2, I guess.
I seem to get c2 is equal to negative 10.
And if c2 is negative 10, then c1 must be 50.
There are two ways of checking the answer.
One is to plug it into the equations, and the other is to peak.
Yes, that's right.
[LAUGHTER] The final answer is, in other words, you put a 50 here, 25 there, negative 10 here, and positive 20 there.
That gives the answer to the problem.
It tells you, in other words, how the temperature of the yolk varies with time and how the temperature of the white varies with time.
As I said, we are going to learn a slick way of doing this problem, or at least a very different way of doing the same problem next time, but let's put that on ice for the moment.
And instead I would like to spend the rest of the period doing for first order systems the same thing that I did for you the very first day of the term.
Remember, I walked in assuming that you knew how to separate variables the first day of the term, and I did not talk to you about how to solve fancier equations by fancier methods.
I instead talked to you about the geometric significance, what the geometric meaning of a single first order equation was and how that geometric meaning enabled you to solve it numerically.
And we spent a little while working on such problems because nowadays with computers it is really important that you get a feeling for what these things mean as opposed to just algorithms for solving them.
As I say, most differential equations, especially systems, are likely to be solved by a computer anyway.
You have to be the guiding genius that interprets the answers and can see when mistakes are being made, stuff like that.
The problem is, therefore, what is the meaning of this system?
Well, you are not going to get anywhere interpreting it geometrically, unless you get rid of that t on the right-hand side.
And the only way of getting rid of the t is to declare it is not there.
So I hereby declare that I will only consider, for the rest of the period, that is only ten minutes, systems in which no t appears explicitly on the right-hand side.
Because I don't know what to do if it does up here.
We have a word for these.
Remember what the first order word was?
A first order equation where there was no t explicitly on the right-hand side, we called it, anybody remember?
Just curious.
Autonomous, right.
This is an autonomous system.
It is not a linear system because these are messy functions.
This could be x times y or x squared minus 3y squared divided by sine of x plus y.
It could be a mess.
Definitely not linear.
But autonomous means no t. t means the independent variable appears on the right-hand side.
Of course, it is there.
It is buried in the dx/dt and dy/dt.
But it is not on the right-hand side.
No t appears on the right-hand side.
Because no t appears on the right-hand side, I can now draw a picture of this.
But, let's see, what does a solution look like?
I never even talked about what a solution was, did I?
Well, pretend that immediately after I talked about that, I talked about this.
What is the solution?
Well, the solution, maybe you took it for granted, is a pair of functions, x of t, y of t if when you plug it in it satisfies the equation.
And so what else is new?
The solution is x equals x of t, y equals y of t. If I draw a picture of that what would it look like?
This is where your previous knowledge of physics above all 18.02, maybe 18.01 if you learned this in high school, what is x equals x of t and y equals y of t?
How do you draw a picture of that?
What does it represent?
A curve.
And what will be the title of the chapter of the calculus book in which that is discussed?
Parametric equations.
This is a parameterized curve.
So we know what the solution looks like.
Our solution is a parameterized curve.
And what does a parameterized curve look like?
Well, it travels, and in a certain direction.
Okay.
That's enough.
Why do I have several of those curves?
Well, because I have several solutions.
In fact, given any initial starting point, there is a solution that goes through it.
I will put in possible starting points.
And you can do this on the computer screen with a little program you will have, one of the visuals you'll have.
It's being made right now.
You put down starter point, put down a click, and then it just draws the curve passing through that point.
Didn't we do this early in the term?
Yes.
But there is a difference now which I will explain.
These are various possible starting points at time zero for this solution, and then you see what happens to it afterwards.
In fact, through every point in the plane will pass a solution curve, parameterized curve.
Now, what is then the representation of this?
Well, what is the meaning of x prime of t and y prime of t?
I am not going to worry for the moment about the right-hand side.
What does this mean by itself?
If this is the curve, the parameterized motion, then this represents its velocity vector.
It is the velocity of the solution at time t. If I think of the solution as being a parameterized motion.
All I have drawn here is the trace, the path of the motion.
This hasn't indicated how fast it was going.
One solution might go whoosh and another one might go rah.
That is a velocity, and that velocity changes from point to point.
It changes direction.
Well, we know its direction at each point.
That's tangent.
What I cannot tell is the speed.
From this picture, I cannot tell what the speed was.
Too bad.
Now, what is then the meaning of the system?
What the system does, it prescribes at each point the velocity vector.
If you tell me what the point (x, y) is in the plane then these equations give you the velocity vector at that point.
And, therefore, what I end up with, the system is what you call in physics and what you call in 18.02 a velocity field.
So at each point there is a certain vector.
The vector is always tangent to the solution curve through there, but I cannot predict from just this picture what its length will be because at some points, it might be going slow.
The solution might be going slowly.
In other words, the plane is filled up with these guys.
Stop me.
Not enough here.
So on and so on.
We can say a system of first order equations, ODEs of first order equations, autonomous because there must be no t on the right-hand side, is equal to a velocity field.
A field of velocity.
The plane covered with velocity vectors.
And a solution is a parameterized curve with the right velocity everywhere.
Now, there obviously must be a connection between that and the direction fields we studied at the beginning of the term.
And there is.
It is a very important connection.
It is too important to talk about in minus one minute.
When we need it, I will have to spend some time talking about it then.
The last time I spent solving a system of equations dealing with the chilling of this hardboiled egg being put in an ice bath.
We called T1 the temperature of the yoke and T2 the temperature of the white.
What I am going to do is revisit that same system of equations, but basically the topic for today is to learn to solve that system of equations by a completely different method.
It is the method that is normally used in practice.
Elimination is used mostly by people who have forgotten how to do it any other way.
Now, in order to make it a little more general, I am not going to use the dependent variables T1 and T2 because they suggest temperature a little too closely.
Let's change them to neutral variables.
I will use x equals T1, and for T2 I will just use y. I am not going to re-derive anything.
I am not going to resolve anything.
I am not going to repeat anything of what I did last time, except to write down to remind you what the system was in terms of these variables, the system we derived using the particular conductivity constants, two and three, respectively.
The system was this one, minus 2x plus 2y.
And the y prime was 2x minus 5y.
And so we solved this by elimination.
We got a single second-order equation with constant coefficients, which we solved in the usual way.
From that I derived what the x was, from that we derived what the y was, and then I put them all together.
I will just remind you what the final solution was when written out in terms of arbitrary constants.
It was c1 times e to the negative t plus c2 e to the negative 6t, and y was c1 over 2 e to the negative t minus 2c2 e to the negative 6t.
That was the solution we got.
And then I went on to put in initial conditions, but we are not going to explore that aspect of it today.
We will in a week or so.
This was the general solution because it had two arbitrary constants in it.
What I want to do now is revisit this and do it by a different method, which makes heavy use of matrices.
That is a prerequisite for this course, so I am assuming that you reviewed a little bit about matrices.
And it is in your book.
Your book puts in a nice little review section.
Two-by-two and three-by-three will be good enough for 18.03 mostly because I don't want you to calculate all night on bigger matrices, bigger systems.
So nothing serious, matrix multiplication, solving systems of linear equations, end-by-end systems.
I will remind you at the appropriate places today of what it is you need to remember.
The very first thing we are going to do is, let's see.
I haven't figured out the color coding for this lecture yet, but let's make this system in green and the solution can be in purple.
Invisible purple, but I have a lot of it.
Let's abbreviate, first of all, the system using matrices.
I am going to make a column vector out of (x, y).
Then you differentiate a column vector by differentiating each component.
I can write the left-hand side of the system as (x, y) prime.
How about the right-hand side?
Well, I say I can just write the matrix of coefficients to negative 2, 2, 2, negative 5 times x,y.
And I say that this matrix equation says exactly the same thing as that green equation and, therefore, it is legitimate to put it up in green, too.
The top here is x prime.
What is the top here?
After I multiply these two I get a column vector.
And what is its top entry?
It is negative 2x plus 2y.
There it is.
And the bottom entry the same way is 2x minus 5y, just as it is down there.
Now, what I want to do is, well, maybe I should translate the solution.
What does the solution look like?
We got that, too.
How am I going to write this as a matrix equation?
Actually, if I told you to use matrices, use vectors, the point at which you might be most hesitant is this one right here, the very next step.
Because how you should write it is extremely well-concealed in this notation.
But the point is, this is a column vector and I am adding together two column vectors.
And what is in each one of the column vectors?
Think of these two things as a column vector.
Pull out all the scalars from them that you can.
Well, you see that c1 is a common factor of both entries and so is e to the negative t, that function.
Now, if I pull both of those out of the vector, what is left of the vector?
Well, you cannot even see it.
What is left is a 1 up here and a one-half there.
So I am going to write that in the following form.
I will put out the c1, it's the common factor in both, and put that out front.
Then I will put in the guts of the vector, even though you cannot see it, the column vector 1, one-half.
And then I will put the other scalar function in back.
The only reason for putting one of these in front and one in back is visual so to make it easy to read.
There is no other reason.
You could put the c1 here, you could put it here, you could put the e negative t in front if you want to, but people will fire you.
Don't do that.
Write it the standard way because that is the way that it is easiest to read.
The constants out front, the functions behind, and the column vector of numbers in the middle.
And so the other one will be written how?
Well, here, that one is a little more transparent.
c2, 1, 2 and the other thing is e to the negative 6t.
There is our solution.
That is going to need a lot of purple, but I have it.
And now I want to talk about how the new method of solving the equation.
It is based just on the same idea as the way we solve second-order equations.
Yes, question.
Oh, here.
Sorry.
This should be negative two.
Thanks very much.
What I am going to use is a trial solution.
Remember when we had a second-order equation with constant coefficients the very first thing I did was I said we are going to try a solution of the form e to the rt.
Why that?
Well, because Oiler thought of it and it has been known for or 300 years that that is the thing you should do.
Well, this has not been known nearly as long because matrices were only invented around or so, and people did not really use them to solve systems of differential equations until the middle of the last century, 1950-1960.
If you look at books written in 1950, they won't even talk about systems of differential equations, or talk very little anyway and they won't solve them using matrices.
This is only 50 years old.
I mean, my God, in mathematics that is very up to date, particularly elementary mathematics.
Anyway, the method of solving is going to use as a trial solution.
Now, if you were left to your own devices you might say, well, let's try x equals some constant times e to the lambda1 t and y equals some other constant times e to the lambda2 t. Now, if you try that, it is a sensible thing to try, but it will turn out not to work.
And that is the reason I have written out this particular solution, so we can see what solutions look like.
The essential point is here is the basic solution I am trying to find.
Here is another one.
Their form is a column vector of constants.
But they both use the same exponential factor, which is the point.
In other words, I should not use here, in my trial solution, two different lambdas, I should use the same lambda.
And so the way to write the trial solution is (x, y) equals two unknown numbers, that or that or whatever, times e to a single unknown exponent factor.
Let's call it lambda t. It is called lambda.
It is called r. It is called m. I have never seen it called anything but one of those three things.
I am using lambda.
Your book uses lambda.
It is a common choice.
Let's stick with it.
Now what is the next step?
Well, we plug into the system.
Substitute into the system.
What are we going to get?
Well, let's do it.
First of all, I have to differentiate.
The left-hand side asks me to differentiate this.
How do I differentiate this?
Column vector times a function.
Well, the column vector acts as a constant.
And I differentiate that.
That is lambda e to the lambda t. So the (x, y) prime is (a1, a2) times e to the lambda t times lambda.
Now, it is ugly to put the lambda afterwards because it is a number so you should put it in front, again, to make things easier to read.
But this lambda comes from differentiating e to the lambda t and using the chain rule.
This much is the left-hand side.
That is the derivative (x, y) prime.
I differentiate the x and I differentiated the y.
How about the right-hand side.
Well, the right-hand side is negative 2, 2, 2, negative 5 times what?
Well, times (x, y), which is (a1, a2) e to the lambda t. Now, the same thing that happened a month or a month and a half ago happens now.
The whole point of making that substitution is that the e to the lambda t, the function part of it drops out completely.
And one is left with what?
An algebraic equation to be solved for lambda a1 and a2.
In other words, by means of that substitution, and it basically uses the fact that the coefficients are constant, what you have done is reduced the problem of calculus, of solving differential equations, to solving algebraic equations.
In some sense that is the only method there is, unless you do numerical stuff.
You reduce the calculus to algebra.
The Laplace transform is exactly the same thing.
All the work is algebra.
You turn the original differential equation into an algebraic equation for Y of s, you solve it, and then you use more algebra to find out what the original little y of t was.
It is not different here.
So let's solve this system of equations.
Now, the whole problem with solving this system, first of all, what is the system?
Let's write it out explicitly.
Well, it is really two equations, isn't it?
The first one says lambda a1 is equal to negative 2 a1 plus 2 a2.
That is the first one.
The other one says lambda a2 is equal to 2 a1 minus 5 a2.
Now, purely, if you want to classify that, that is two equations and three variables, three unknowns.
The a1, a2, and lambda are all unknown.
And, unfortunately, if you want to classify them correctly, they are nonlinear equations because they are made nonlinear by the fact that you have multiplied two of the variables.
Well, if you sit down and try to hack away at solving those without a plan, you are not going to get anywhere.
It is going to be a mess.
Also, two equations and three unknowns is indeterminate.
You can solve three equations and three unknowns and get a definite answer, but two equations and three unknowns usually have an infinity of solutions.
Well, at this point it is the only idea that is required.
Well, this was a little idea, but I assume one would think of that.
And the idea that is required here is, I think, not so unnatural, it is not to view these a1, a2, and lambda as equal.
Not all variables are created equal.
Some are more equal than others.
a1 and a2 are definitely equal to each other, and let's relegate lambda to the background.
In other words, I am going to think of lambda as just a parameter.
I am going to demote it from the status of variable to parameter.
If I demoted it further it would just be an unknown constant.
That is as bad as you can be.
I am going to focus my attention on the a1, a2 and sort of view the lambda as a nuisance.
Now, as soon as I do that, I see that these equations are linear if I just look at them as equations in a1 and a2.
And moreover, they are not just linear, they are homogenous.
Because if I think of lambda just as a parameter, I should rewrite the equations this way.
I am going to subtract this and move the left-hand side to the right side, and it is going to look like (minus 2 minus lambda) times a1 plus 2 a2 is equal to zero.
And the same way for the other one.
It is going to be 2a1 plus, what is the coefficient, (minus 5 minus lambda) a2 equals zero.
That is a pair of simultaneous linear equations for determining a1 and a2, and the coefficients involved are parameter lambda.
Now, what is the point of doing that?
Well, now the point is whatever you learned about linear equations, you should have learned the most fundamental theorem of linear equations.
The main theorem is that you have a square system of homogeneous equations, this is a two-by-two system so it is square, it always has the trivial solution, of course, a1, a2 equals zero.
Now, we don't want that trivial solution because if a1 and a2 are zero, then so are x and y zero.
Now that is a solution.
Unfortunately, it is of no interest.
If the solution were x, y zero, it corresponds to the fact that this is an ice bath.
The yoke is at zero, the white is at zero and it stays that way for all time until the ice melts.
So that is the solution we don't want.
We don't want the trivial solution.
Well, when does it have a nontrivial solution?
Nontrivial means non-zero, in other words.
If and only if this determinant is zero.
In other words, by using that theorem on linear equations, what we find is there is a condition that lambda must satisfy, an equation in lambda in order that we would be able to find non-zero values for a1 and a2.
Let's write it out.
I will recopy it over here.
What was it?
Negative 2 minus lambda, two, here it was 2 and minus 5 minus lambda.
All right.
You have to expand the determinant.
In other words, we are trying to find out for what values of lambda is this determinant zero.
Those will be the good values which lead to nontrivial solutions for the a's.
This is the equation lambda plus 2.
See, this is minus that and minus that, the product of the two minus ones is plus one.
So it is lambda plus 2 times lambda plus 5, which is the product of the two diagonal elements, minus the product of the two anti-diagonal elements, which is 4, is equal to zero.
And if I write that out, what is that, that is the equation lambda squared plus 7 lambda, 5 lambda plus 2 lambda, and then the constant term is 10 minus 4 which is 6.
How many of you have long enough memories, two-day memories that you remember that equation?
When I did the method of elimination, it led to exactly the same equation except it had r's in it instead of lambda.
And this equation, therefore, is given the same name and another color.
Let's make it salmon.
And it is called the characteristic equation for this method.
All right.
Now I am going to use now the word from last time.
You factor this.
From the factorization we get its root easily enough.
The roots are lambda equals negative 1 and lambda equals negative 6 by factoring the equation.
Now what I am supposed to do?
You have to keep the different parts of the method together.
Now I have found the only values of lambda for which I will be able to find nonzero values for the a1 and a2.
For each of those values of lambda, I now have to find the corresponding a1 and a2.
Let's do them one at a time.
Let's take first lambda equals negative one.
My problem is now to find a1 and a2.
Where am I going to find them from?
Well, from that system of equations over there.
I will recopy it over here.
What is the system?
The hardest part of this is dealing with multiple minus signs, but you had experience with that in determinants so you know all about that.
In other words, there is the system of equations over there.
Let's recopy them here.
Minus 2, minus minus 1 makes minus 1.
What's the other coefficient?
It is just plain old 2.
Good.
There is my first equation.
And when I substitute lambda equals negative one for the second equation, what do you get?
2 a1 plus negative 5 minus negative 1 makes negative 4.
There is my system that will find me a1 and a2.
What is the first thing you notice about it?
You immediately notice that this system is fake because this second equation is twice the first one.
Something is wrong.
No, something is right.
If that did not happen, if the second equation were not a constant multiple of the first one then the only solution of the system would be a1 equals zero, a2 equals zero because the determinant of the coefficients would not be zero.
The whole function of this exercise was to find the value of lambda, negative 1, for which the system would be redundant and, therefore, would have a nontrivial solution.
Do you get that?
In other words, calculate the system out, just as I have done here, you have an automatic check on the method.
If one equation is not a constant multiple of the other you made a mistake.
You don't have the right value of lambda or you substituted into the system wrong, which is frankly a more common error.
Go back, recheck first the substitution, and if convinced that is right then recheck where you got lambda from.
But here everything is going fine so we can now find out what the value of a1 and a2 are.
You don't have to go through a big song and dance for this since most of the time you will have two-by-two equations and now and then three-by-three.
For two-by-two all you do is, since we really have the same equation twice, to get a solution I can assign one of the variables any value and then simply solve for the other.
The natural thing to do is to make a2 equal one, then I won't need fractions and then a1 will be a2.
So the solution is (2, 1).
I am only trying to find one solution.
Any constant multiple of this would also be a solution, as long as it wasn't zero, zero which is the trivial one.
And, therefore, this is a solution to this system of algebraic equations.
And the solution to the whole system of differential equations is, this is only the (a1, a2) part.
I have to add to it, as a factor, lambda is negative, therefore, e to the minus t. There is our purple thing.
See how I got it?
Starting with the trial solution, I first found out through this procedure what the lambda's have to be.
Then I took the lambda and found what the corresponding a1 and a2 that went with it and then made up my solution out of that.
Now, quickly I will do the same thing for lambda equals negative 6.
Each one of these must be treated separately.
They are separate problems and you are looking for separate solutions.
Lambda equals negative 6.
What do I do?
How do my equations look now?
Well, the first one is minus 2 minus negative 6 makes plus 4.
It is 4a1 plus 2a2 equals zero.
Then I hold my breath while I calculate the second one to see if it comes out to be a constant multiple.
I get 2a1 plus negative 5 minus negative 6, which makes plus 1.
And, indeed, one is a constant multiple of the other.
I really only have on equation there.
I will just write down immediately now what the solution is to the system.
Well, the (a1, a2) will be what?
Now, it is more natural to make a1 equal 1 and then solve to get an integer for a2.
If a1 is 1, then a2 is negative 2.
And I should multiply that by e to the negative 6t because negative 6 is the corresponding value.
There is my other one.
And now there is a superposition principle, which if I get a chance will prove for you at the end of the hour.
If not, you will have to do it yourself for homework.
Since this is a linear system of equations, once you have two separate solutions, neither a constant multiple of the other, you can multiply each one of these by a constant and it will still be a solution.
You can add them together and that will still be a solution, and that gives the general solution.
The general solution is the sum of these two, an arbitrary constant.
I am going to change the name since I don't want to confuse it with the c1 I used before, times the first solution which is (2, 1) e to the negative t plus c2, another arbitrary constant, times 1 negative 2 e to the minus 6t.
Now you notice that is exactly the same solution I got before.
The only difference is that I have renamed the arbitrary constants.
The relationship between them, c1 over 2, I am now calling c1 tilda, and c2 I am calling c2 tilda.
If you have an arbitrary constant, it doesn't matter whether you divide it by two.
It is still just an arbitrary a constant.
It covers all values, in other words.
Well, I think you will agree that is a different procedure, yet it has only one coincidence.
It is like elimination goes this way and comes to the answer.
And this method goes a completely different route and comes to the answer, except it is not quite like that.
They walk like this and then they come within viewing distance of each other to check that both are using the same characteristic equation, and then they again go their separate ways and end up with the same answer.
There is something special of these values.
You cannot get away from those two values of lambda.
Somehow they are really intrinsically connected.
Occurs the exponential coefficient, and they are intrinsically connected with the problem of the egg that we started with.
Now what I would like to do is very quickly sketch how this method looks when I remove all the numbers from it.
In some sense, it becomes a little clearer what is going on.
And that will give me a chance to introduce the terminology that you need when you talk about it.
Well, you have notes.
Let me try to write it down in general.
I will first write it out two-by-two.
I am just going to sketch.
The system looks like (x, y) equals, I will still put it up in colors.
Except now, instead of using twos and fives, I will use (a, b; c, d).
The trial solution will look how?
The trial is going to be (a1, a2).
That I don't have to change the name of.
I am going to substitute in, and what the result of substitution is going to be lambda (a1, a2).
I am going to skip a step and pretend that the e to the lambda t's have already been canceled out.
Is equal to (a, b; c, d) times (a1, a2).
What does that correspond to?
That corresponds to the system as I wrote it here.
And then we wrote it out in terms of two equations.
And what was the resulting thing that we ended up with?
Well, you write it out, you move the lambda to the other side.
And then the homogeneous system is we will look in general how?
Well, we could write it out.
It is going to look like a minus lambda, b, c, d minus lambda.
That is just how it looks there and the general calculation is the same.
Times (a1, a2) is equal to zero.
This is solvable nontrivially.
In other words, it has a nontrivial solution if an only if the determinant of coefficients is zero.
Let's now write that out, calculate out once and for all what that determinant is.
I will write it out here.
It is a minus lambda times d minus lambda, the product of the diagonal elements, minus the anti-diagonal minus bc is equal to zero.
And let's calculate that out.
It is lambda squared minus a lambda minus d lambda plus ad, the constant term from here, negative bc from there, plus ad minus bc, where have I seen that before?
This equation is the general form using letters of what we calculated using the specific numbers before.
Again, I will code it the same way with that color salmon.
Now, most of the calculations will be for two-by-two systems.
I advise you, in the strongest possible terms, to remember this equation.
You could write down this equation immediately for the matrix.
You don't have to go through all this stuff.
For God's sakes, don't say let the trial solution be blah, blah, blah.
You don't want to do that.
I don't want you to repeat the derivation of this every time you go through a particular problem.
It is just like in solving second order equations.
You have a second order equation.
You immediately write down its characteristic equation, then you factor it, you find its roots and you construct the solution.
It takes a minute.
The same thing, this takes a minute, too.
What is the constant term?
Ad minus bc, what is that?
Matrix is (a, b; c, d).
Ad minus bc is its determinant.
This is the determinant of that matrix.
I didn't give the matrix a name, did I?
I will now give the matrix a name A.
What is this?
Well, you are not supposed to know that until now.
I will tell you.
This is called the trace of A.
Put that down in your little books.
The abbreviation is trace A, and the word is trace.
The trace of a square matrix is the sum of the d elements down its main diagonal.
If it were a three-by-three there would be three terms in whatever you are up to.
Here it is a plus b, the sum of the diagonal elements.
You can immediately write down this characteristic equation.
Let's give it a name.
This is a characteristic equation of what?
Of the matrix, now.
Not of the system, of the matrix.
You have a two-by-two matrix.
You could immediately write down its characteristic equation.
Watch out for this sign, minus.
That is a very common error to leave out the minus sign because that is the way the formula comes out.
Its roots.
If it is a quadratic equation it will have roots; lambda1, lambda2 for the moment let's assume are real and distinct.
For the enrichment of your vocabulary, those are called the eigenvalues.
They are something which belonged to the matrix A.
They are two secret numbers.
You can calculate from the coefficients a, b, and c, and d, but they are not in the coefficients.
You cannot look at a matrix and see what its eigenvalues are.
You have to calculate something.
But they are the most important numbers in the matrix.
They are hidden, but they are the things that control how this system behaves.
Those are called the eigenvalues.
Now, there are various purists, there are a fair number of them in the world who do not like this word because it begins German and ends English.
Eigenvalues were first introduced by a German mathematician, you know, around the time matrices came into being in or so.
A little while after eigenvalues came into being, too.
And since all this happened in Germany they were named eigenvalues in German, which begins eigen and ends value.
But people who do not like that call them the characteristic values.
Unfortunately, it is two words and takes a lot more space to write out.
An older generation even calls them something different, which you are not so likely to see nowadays, but you will in slightly older books.
You can also call them the proper values.
Characteristic is not a translation of eigen, but proper is, but it means it in a funny sense which has almost disappeared nowadays.
It means proper in the sense of belong to.
The only example I can think of is the word property.
Property is something that belongs to you.
That is the use of the word proper.
It is something that belongs to the matrix.
The matrix has its proper values.
It does not mean proper in the sense of fitting and proper or I hope you will behave properly when we go to Aunt Agatha's or something like that.
But, as I say, by far the most popular thing, slowly the word eigenvalue is pretty much taking over the literature.
Just because it's just one word, that is a tremendous advantage.
Okay.
What now is still to be done?
Well, there are those vectors to be found.
So the very last step would be to solve the system to find the vectors a1 and a2.
For each (lambda)i, find the associated vector.
The vector, we will call it (alpha)i.
That is the a1 and a2.
Of course it's going to be indexed.
You have to put another subscript on it because there are two of them.
And a1 and a2 is stretched a little too far.
By solving the system, and the system will be the system which I will write this way, (a minus lambda, b, c, d minus lambda).
It is just that system that was over there, but I will recopy it, (a1, a2) equals zero, zero.
And these are called the eigenvectors.
Each of these is called the eigenvector associated with or belonging to, again, in that sense of property.
Eigenvector, let's say belonging to, I see that a little more frequently, belonging to lambda i.
So we have the eigenvalues, the eigenvectors and, of course, the people who call them characteristic values also call these guys characteristic vectors.
I don't think I have ever seen proper vectors, but that is because I am not old enough.
I think that is what they used to be called a long time ago, but not anymore.
And then, finally, the general solution will be, by the superposition principle, (x, y) equals the arbitrary constant times the first eigenvector times the eigenvalue times the e to the corresponding eigenvalue.
And then the same thing for the second one, (a1, a2), but now the second index will be 2 to indicate that it goes with the eigenvalue e to the lambda 2t.
I have done that twice.
And now in the remaining five minutes I will do it a third time because it is possible to write this in still a more condensed form.
And the advantage of the more condensed form is A, it takes only that much space to write, and B, it applies to systems, not just the two-by-two systems, but to end-by-end systems.
The method is exactly the same.
Let's write it out as it would apply to end-by-end systems.
The vector I started with is (x, y) and so on, but I will simply abbreviate this, as is done in 18.02, by x with an arrow over it.
The matrix A I will abbreviate with A, as I did before with capital A.
And then the system looks like x prime is equal to -- x prime is what?
Ax.
That is all there is to it.
There is our green system.
Now notice in this form I did not even tell you whether this a two-by-two matrix or an end-by-end.
And in this condensed form it will look the same no matter how many equations you have.
Your book deals from the beginning with end-by-end systems.
That is, in my view, one of its weaknesses because I don't think most students start with two-by-two.
Fortunately, the book double-talks.
The theory is end-by-end, but all the examples are two-by-two.
So just read the examples.
Read the notes instead, which just do two-by-two to start out with.
The trial solution is x equals what?
An unknown vector alpha times e to the lambda t. Alpha is what we called a1 and a2 before.
Plug this into there and cancel the e to the lambda t's.
What do you get?
Well, this is lambda alpha e to the lambda t equals A alpha e to the lambda t. These two cancel.
And the system to be solved, A alpha equals lambda alpha.
And now the question is how do you solve that system?
Well, you can tell if a book is written by a scoundrel or not by how they go -- A book, which is in my opinion completely scoundrel, simply says you subtract one from the other, and without further ado writes A minus lambda, and they tuck a little I in there and write alpha equals zero.
Why is the I put in there?
Well, this is what you would like to write.
What is wrong with this equation?
This is not a valid matrix equation because that is a square end-by-end matrix, a square two-by-two matrix if you like.
This is a scalar.
You cannot subtract the scalar from a matrix.
It is not an operation.
To subtract matrices they have to be the same size, the same shape.
What is done is you make this a two-by-two matrix.
This is a two-by-two matrix with lambdas down the main diagonal and I elsewhere.
And the justification is that lambda alpha is the same thing as the lambda I times alpha because I is an identity matrix.
Now, in fact, jumping from here to here is not something that would occur to anybody.
The way it should occur to you to do this is you do this, you write that, you realize it doesn't work, and then you say to yourself I don't understand what these matrices are all about.
I think I'd better write it all out.
And then you would write it all out and you would write that equation on the left-hand board there.
Oh, now I see what it should look like.
I should subtract lambda from the main diagonal.
That is the way it will come out.
And then say, hey, the way to save lambda from the main diagonal is put it in an identity matrix.
That will do it for me.
In other words, there is a little detour that goes from here to here.
And one of the ways I judge books is by how well they explain the passage from this to that.
If they don't explain it at all and just write it down, they have never talked to students.
They have just written books.
Where did we get finally here?
The characteristic equation from that, I had forgotten what color.
That is in salmon.
The characteristic equation, then, is going to be the thing which says that the determinant of that is zero.
That is the circumstances under which it is solvable.
In general, this is the way the characteristic equation looks.
And its roots, once again, are the eigenvalues.
And from then you calculate the corresponding eigenvectors.
Okay.
Go home and practice.
In recitation you will practice on both two-by-two and three-by-three cases, and we will talk more next time.
I just want to remind you of the main facts.
The first thing that you have to do is, of course, we are going to have to be doing it several times today.
That is the system we are trying to solve.
And the first thing you have to do is find a characteristic equation which is general form, although this is not the form you should use for two-by-two, is A minus lambda I equals zero.
And its roots are the eigenvalues.
And then with each eigenvalue you then have to calculate its eigenvector, which you do by solving the system (A minus lambda1, let's say, times I) alpha equals zero because the solution is the eigenvector alpha 1.
And then the final solution that you make out of the two of them looks like alpha 1 times e to the lambda 1t.
Of course you do that for each eigenvalue.
You get the associated eigenvector.
And then the general solution is made up out of a linear combination of these individual guys with constant coefficients.
The lecture today is devoted to the two cases where things do not go as smoothly as they seem to in the homework problems you have been doing up until now.
The first one will take probably most of the period.
It deals with what happens when an eigenvalue gets repeated.
But I think since the situation is a little more complicated than it is where the case of a characteristic root gets repeated in the case of a second-order equation as we saw it, you know what to do in that case, here there are different possibilities.
And I thought the best thing to do would be to illustrate them on an example.
So here is a problem.
It came out of a mild nightmare, but I won't bore you with the details.
Anyway, we have this circular fish tank.
It is a very modern fish tank.
It is divided into three compartments because one holds Siamese fighting fish and one goldfish, and one-- They should not eat each other.
And it is going to be a simple temperature problem.
The three actual compartments have to be kept at different temperatures because one is for tropical fish and one is for arctic fish and one is for everyday garden variety fish.
But the guy forgets to turn on the heater so the temperatures start out what they are supposed to be, tropical, icy, and normal.
But as the day wears on, of course, the three compartments trade their heat and sort of tend to all end up at the same temperature.
So we are going to let (x)i equal the temperature in tank i.
Now, these are separated from each other by glass things.
Everything is identical, each has the same volume, and the same glass partition separates them out and no heat can escape.
This is very well-insulated with very double-thick Thermopane glass or something like that.
You can see in, but heat cannot get out very well.
Heat essentially is conducted from one of these cells to the other.
And let's assume that the water in each tank is kept stirred up because the fish are swimming around in it.
That should be a pretty decent way of stirring a fish tank.
The question is how do each of these, as a function of time, and I want to know how they behave over time, so find these functions.
Well, we are going to find them in solutions to differential equations.
And the differential equations are not hard to set up.
They are very much like the diffusion equation you had for homework or the equations we studied in the beginning of the term.
Let's do one carefully because the others go exactly the same way.
What determines the flow, the change in temperature?
Well, it is the conductivity across the barriers.
But there are two barriers because heat can flow into this first cell, both from this guy and it can flow across this glass pane from the other cell.
We have to take account of both of those possibilities.
It is like in your homework.
The little diffusion cell that was in the middle could get contributions from both sides, whereas, the two guys on the end could only get contribution from one.
But here, nobody is on the end.
It is circular table.
Everyone is dying equally.
Everybody can get input from the other two cells.
x1 prime is some constant of conductivity times the temperature difference between tank three and tank one.
And then there is another term which comes from tank two.
So a times tank two minus the temperature difference, tank two minus tank one.
Let's write this out.
Remember there will be other equations, too.
But instead of doing this, let's do a more careful job with this first equation.
When I write it out, remember, the important thing is you are going to have x1, x2, x3 down the left, so they have to occur in the same order on the right in order to use these standard eigenvalue techniques.
The coefficient of x1 is going to be minus a x1 and then another minus a x1.
In other words, it is going to be minus 2 ax1.
And then the x2 term will be plus a x2.
And the x3 term will be plus a x3.
Well, you can see now that is the equation for x1 prime in terms of the other variables.
But there is symmetry.
There is no difference between this tank, that tank, and that tank as far as the differential equations are concerned.
And, therefore, I can get the equations for the other two tanks by just changing 1 to 2, just switching the subscripts.
When I finally do it all, the equations are going to be, I will write them first out as a system.
Let's take a equal 1 because I am going to want to solve them numerically, and I want you to be able to concentrate on what is important, what is new now and not fuss because I don't want to have an extra a floating around everywhere just contributing nothing but a mild confusion to the proceedings.
So x1 prime, I am going to take a equal 1 and simply write it minus 2 x1 plus x2 plus x3.
And so now what would the equation for x2 prime be?
Well, here x2 plays the role that x1 played before.
And the only way to tell that x1 was the main guy here was it occurred with a coefficient negative 2, whereas, the other guys occurred with coefficient 1.
That must be what happens here, too.
Since x2 prime is our main man, this is minus x2 and this must be x1 here plus x3.
And finally the last one is no different, x3 prime is x1 plus x2.
And now it is the x3 that should get negative 2 for the coefficient.
There is a perfectly reasonable-looking set of equations.
Just how reasonable they are depends upon what their characteristic polynomial turns out to be.
And all the work in these problems is trying to find nice models where you won't have to use Matlab to calculate the roots, the eigenvalues, the roots of the characteristic polynomial.
So we have to now find the characteristic polynomial.
The matrix that we are talking about is the matrix, well, let's right away write A minus lambda I I cannot use the trace and determinant form for this equation because it is not a two-by-two matrix.
It is a three-by-three matrix.
I have to use the original form for the characteristic equation.
But what is this going to be?
Well, what is A?
A is minus 2.
I am going to leave a little space here.
1, 1, 1 minus 2, 1.
And finally 1, 1, negative 2. subtract lambda from the main diagonal, minus 2 minus lambda minus 2 minus lambda, minus 2 minus lambda.
And now that equals zero is the characteristic equation.
The term with the most lambdas in it is the main diagonal.
That is always true, notice.
Now, each of these I would be happier writing lambda plus 2, so there would be a negative sign, negative sign, negative sign.
The product of three negative signs is still a negative sign because three is an odd number.
So it is minus the principle term.
The product of these three is minus lambda plus 2 cubed.
Now, the rest of the terms are going to be easy.
There is another term 1 times 1 times 1, another term 1 times 1 times 1.
So to that I add 2, 1 and 1 for those two other terms.
And now I have the three going in this direction, but each one of them has to be prefaced with a minus sign.
What does each one of them come to?
Well, this is minus 2 minus lambda when I multiply those three numbers together.
And so are the other guys.
This is 1 times 1 times minus 2 minus lambda, the same thing.
There are three of them.
Minus because they are going this way, minus 3 because there are three of them, and what each one of them is is negative 2 negative lambda.
That is equal to zero, and that is the characteristic equation.
Now, it doesn't look very promising.
On the other hand, I have selected it for the lecture.
Simple psychology should tell you that it is going to come out okay.
What I am going to do is expand this.
First imagine changing the sign.
I hate to have a minus sign in front of a lambda cubed, so let's make this plus and we will make this minus and we will make this plus.
I will just change all the signs, which is okay since it is an equation equals zero.
That doesn't change its roots any.
And now we are going to expand it out.
What is this?
Lambda plus 2 cubed.
Lambda cubed plus, and don't get confused because it is this 2 that will kill you when you use the binomial theorem.
If there is 1 here everybody knows what to do.
If there is an A there everybody knows what to do.
It is when that is a number not 1 that everybody makes mistakes, including me.
The binomial coefficients are 1, 3, 3, 1 because it is a cubed, I am expanding.
So it is lambda cubed plus 3 times lambda squared times 2.
I won't explain to you what I am doing.
I will just do it and hope that you all know what I am doing.
Plus 3 times lambda times 2 squared plus the last term, which is 2 cubed.
And now we have the other term.
All that is plus because I changed its sign.
The next thing is negative 2.
And then the last thing is plus 3 times (minus 2 minus lambda).
Let's keep it.
So what is the actual characteristic equation?
Maybe I can finish it.
I should stay over here instead of recopying all of it.
Well, there is a lot more work to do.
Let's see if we can at least write down the equation.
What is it?
It is lambda cubed.
What is the lambda squared erm?
It is six and that is all there is.
How about the lambda term?
Well, we have 12 lambda minus 3 lambda which makes plus 9 lambda.
That looks good but constant terms have a way of screwing everything up.
What is the constant term?
It is A minus 2 minus 6.
Zero.
The constant term is zero.
That converts this from a hard problem to an easy problem.
Now it is a cinch to calculate the stuff.
Let's go to this board and continue the work over here.
The equation is lambda cubed plus 6 lambda squared plus 9 lambda is zero.
It is very easy to calculate the roots of that.
You factor it.
Lambda is a common factor.
And what is left?
Lambda squared plus 6 lambda plus 9.
That is the sort of thing you got all the time when you were studying critical damping.
It is the square of lambda plus 3.
Lambda squared plus 6 lambda plus 9 equals zero.
So the eigenvalues, the roots are what?
Well, they are lambda equals zero from this factor and then lambda equals minus 3.
But what is new is that the minus 3 is a double root.
That is a double root.
Now, that, of course, is what is going to cause the trouble.
Because, for each one of these, I am supposed to calculate the eigenvector and make up the solution.
But that assumed that I had three things to get three different solutions.
Here I have only got two things.
It is the same trouble we ran into when there was a repeated root.
We were studying second or third order differential equations and the characteristic equation had a repeated root.
And I had to go into a song and dance and stand on my head and multiply things by t and so on.
And then talk very hard arguing why that was a good thing to do to get the answer.
Now, I am not going to do the same thing here.
Instead, I am going to try to solve the problem instead.
Let's get two points by at least doing the easy part of it.
Lambda equals zero.
What am I supposed to do with lambda equals zero?
I am looking for the alpha that goes with that.
And I find that eigenvector by solving this system of equations.
Let's write out what that system of equations is.
Well, if lambda is zero, this isn't there.
It is just the matrix A times alpha equals zero.
And the matrix A is, I never even wrote it anywhere.
I never wrote A. I thought I would get away without having to do it, but you never get away with anything.
It's the principle of life.
That is A.
If I subtract zero from the main diagonal, that doesn't do a great deal to A.
And the resulting system of equations is those same things, except you have the a1's there, too.
There is one.
a1 minus 2 a2 plus a3 equals zero.
I am just subtracting zero from the main diagonal so there is nothing to do.
a2 minus 2 a3 equals zero.
Now I am supposed to solve those.
Of course we could do it.
Well, how do you know how to solve a system of three linear equations?
Well, elimination.
You can always solve by elimination.
Now we are much more sophisticated than that.
You all have pocket calculators so you could use the inverse matrix, right?
No.
You cannot use the inverse matrix.
What will happen if you punch in those coefficients and then punch in A inverse.
What answer will it give you?
0, 0, 0.
No, I am sorry.
It won't give you any answer.
What will it say?
It will say I cannot calculate the inverse to that matrix because the whole purpose of this exercise was to find a value of lambda such that this system of equations is dependent.
The coefficient determinant is zero and, therefore, the coefficient matrix does not have an inverse matrix.
You cannot use that method.
In other words, the inverse matrix will never work in these problems because the system of equations you will be trying to solve is always a non-independent system.
And, therefore, its determinant is always zero.
And, therefore, there is no inverse matrix because the determinant of the coefficient is zero.
All you can do is use elimination or physical insight and common sense.
Now, because I teach differential equations everybody assumes, mistakenly, as I think, that I really know something about them.
I get now and then graduate students, not in mathematics, but some obscure field of engineering or whatever drift into my office and say I see you teach differential equations.
Do you have a minute here?
And before I can say no they write their differential equation on the board.
And almost invariably it is nothing I have ever seen before.
And they so look at me hopefully and expectantly.
So what do I ask them?
I don't ask them what they have tried.
What I ask them is where did this come from?
What field did it come from?
Because each field has its own little tricks.
It gets the same differential equations all the time and has its own little tricks for solving them.
You should do the same thing here.
Well, of course we can solve this.
And by now most of you have solved it just by inspection, just by sort of psyching out the answer.
But a better way is to say look, suppose we had the solution, what would the solution look like?
Well, it would look like (a1, a2, a3), whatever the values of those variables were which gave me the solution to the equation, times e to the 0t.
But what is this?
e to the 0t is one for all time.
And, therefore, this is a constant solution.
What I am asking is to find a constant solution.
Now, can I, by inspection, find a constant solution to this?
If so it must be the one.
Well, there is an obvious constant solution.
All the cells have the same temperature.
If that is true then there is no reason why it should ever change as time goes on.
The physical problem itself suggests what the answer must be.
You don't have to solve equations.
In other words, any constant like (1, 1, 1).
Well, could it be (20, 20, 20)?
Yeah, that is a constant multiple of (1, 1, 1).
That is included.
My basic constant solution, therefore, is simply (1, 1, 1) times e to the 0t.
You don't have to include e to the 0t because it is one.
Now, just to check, is (1, 1, 1) a solution to these equations?
It certainly is.
1 plus 1 minus 2 is zero in every case.
The equations are essentially the same, except they use different variables.
By inspection or, if you like, by elimination, but not by finding the inverse matrix you solve those equations.
And we have our first solution.
Now let's go onto the second one.
For the second one, we are going to have to use the eigenvalue lambda equals negative 3.
And now what is the system of equations?
Well, now I have to take this and I have to subtract negative 3 from the diagonal elements.
Minus 2 minus negative 3 is plus 1, right?
Got that?
Each of the diagonal elements, after I subtract minus 3 turns into plus 1.
And, therefore, the system becomes, the system I have to solve is a1 plus a2 plus a3 equals zero.
And what is the second equation?
Symmetry is preserved.
All the equations are essentially the same, except for the names of the variables so they all must give you the same thing after I subtract minus 3 from the main diagonal.
Well, that is what we call a dependent system of equations.
All I have is the same equation repeated twice, but I still have to solve it.
Now, what you see is that there are lots of solutions to this.
Let me write down one of them.
For example, suppose I made a1 equal to 1 and I made a2 a 0, then a3 would be negative 1.
So here is a solution.
That is the eigenvector.
And with it, I can make the solution by multiplying by e to the negative 3t.
There is a solution.
But that is not the only alpha I could have chosen.
Suppose I chose this one instead.
Suppose I kept this 1, but this time made a3 zero.
Well, in that case, there would be a2 that had to be minus 1.
Now, is this essentially different from that one?
It would still be multiplied by e to the minus 3t, but don't be fooled by the e to the minus 3t.
That is our scalar.
That is not what is essential.
What is essential is the content of these two vectors.
Is either one a multiple of the other?
The answer is no.
Therefore, they are independent.
They are pointing in two different directions in three space, these two vectors.
And, therefore, I have two independent solutions just by picking two different vectors that solve those three equations.
This is also a solution.
If I call this the eigenvector alpha 1, then I ought to call this one the alpha 2.
Hey, we can keep on going through this.
Why not make the first one zero?
Well, what would happen if I made the first one 0, and then 1, and minus 1?
The answer is this one is no longer independent of those two.
I can get it by taking a combination of those two.
Do you see what combination I should take?
This one minus that one.
This guy minus that guy gives me that guy, isn't that right?
1 minus 1, 0 minus minus 1, minus 1 minus 0.
This is not a new one.
It looks new, but it is not.
I can get it by taking a linear combination of these two.
It is not independent delta.
And that would be true for any other possible solution you could get for these equations.
Once you found two solutions, all the others will be linear combinations of them.
Well, I cannot use that one.
It is not new.
And the general solution, therefore, will be a combination, c1 times that one plus a constant times this one.
Plus the first one that I found c3 times (1, 1, 1) e to the 0t, which I don't have to write in.
That is the general solution to the system, (x1,x2, x3).
What happens as time goes to infinity?
Regardless of what the values of these two C's this term goes to zero, that term goes to zero and what I am left with is a constant solution.
So all of these solutions tend to be the solution where all the cells are at the same temperature.
Well, of course there must be some vocabulary word in this.
There is.
There are two vocabulary words.
This is a good eigenvalue.
There are also bad eigenvalues.
This is a good repeated eigenvalue, but good is not the official word.
An eigenvalue like this, which is repeated but where you can find enough eigenvectors, if lambda is a repeated eigenvalue, it occurs multiply in the characteristic polynomial as a root.
But you can find enough independent eigenvectors -- Forget the "but."
-- to make up the needed number of independent solutions.
For example, if it is repeated once, that is it occurs doubly then somehow I have got to get two solutions out of that as I was able to here.
If it occurred triply, I have got to get three solutions out of it.
I would look for three independent eigenvectors and hope I could find them.
That is the good case because it tells you how to make up as many solutions as you need.
And this kind of eigenvalue is called in the literature the complete eigenvalue.
Now, how about the kind in which you cannot?
Well, unfortunately, all my life I have called it incomplete, which seems to be a perfectly reasonable thing to call it.
However, terminology changes slowly over time.
The notes, because I wrote them, call it an incomplete eigenvalue.
But the accepted term nowadays is defective.
I don't like that.
It violates the "eigenvalues with disabilities act" or something.
But I have to give it to you because that is the word I am going to try to use from now on, at least if I remember to use it.
It would be the word, for example, used in the linear algebra course 18.06 "plug, plug," defective otherwise.
A defective eigenvalue is one where you can get one eigenvector.
If it is double, for example, if it a double eigenvalue.
It is defective if you can get one eigenvector that goes with it, but you cannot find an independent one.
The only other ones you can find are multiples of the first one.
Then you are really in trouble because you just don't have enough solutions that you are supposed to get out of that, and you have to do something.
What you do is turn to problem two on your problem set and solve it because that tells you what to do.
And I even give you an example to work.
Problem two, that little matrix has a defective eigenvalue.
It doesn't look defective, but you cannot tell.
It is defective.
But you, nonetheless, will be able to find two solutions because you will be following instructions.
Now, the only other thing I should tell you is one of the most important theorems in linear algebra, which is totally beyond the scope of this course and is beyond the scope of most elementary linear algebra courses as I have taught around the country but, of course, not at MIT.
But, nonetheless, it is the last theorem in the course.
That means it is liable to use stuff.
The theorem goes by different names.
Sometimes it is called the principle axis theorem.
Sometimes it is called the spectral theorem.
But, anyway, what it says is, if A is a real end-by-end matrix which is symmetric, you know what a symmetric matrix is?
The formal definition is it is equal to its transpose.
What that means is if you flip it around the main diagonal it looks just the same as before.
Somewhere on this board, right there, in fact, is a symmetric matrix.
What happened to it?
Right here was the symmetric matrix.
I erased the one thing which I had to have.
Minus 2, 1, 1; 1, minus 2, 1; 1, 1 minus 2.
That was our matrix A.
The matrix is symmetric because if I flip it around the diagonal it looks the same as it did before.
Well, not exactly.
The ones are sort of lying on their side, but you have to take account of that.
Is that right?
The twos are backward.
Well, you know what I mean.
Put that element there, this one here, that one there.
Exchange these two.
Notice the diagonal elements don't all have to be minus 2 for that.
No matter what they were, they are the guys that aren't moved when you do the flipping.
Therefore, there is no condition on them.
It is these other guys.
Each guy here has to use the same guy there.
This one has to be the same as that one, and so on.
Then it will be real and symmetric.
If you have a matrix that is real and symmetric, like the one we have been working with, the theorem is that all its eigenvalues are complete.
That is a very unobvious theorem.
All its eigenvalues are automatically complete.
And it is a remarkable fact that you can prove that purely generally with a certain amount of pure reasoning no calculation at all.
But it has to be true, and it is true.
You will find there are whole branches of applied differential equations.
You know, equilibrium theory, all the matrices that you deal with are always symmetric.
And, therefore, this repeated eigenvalues is not something you have to worry about, finding extra solutions.
Well, I guess that is the end of the first part of the lecture.
I have a third of it left.
Let's talk fast.
I would like to, with the remaining time, explain to you what to do if you were to get complex eigenvalues.
Now, actually, the answer is follow the same program.
In other words, if you solve the characteristic equation and you get a complex root, follow the program, calculate the corresponding complex eigenvectors.
In other words, solve the equations.
Everything will be the same except that the eigenvectors will turn out to be complex, that is will have complex entries.
Don't worry about it.
Then form the solutions.
The solutions are now going to look once again like alpha times e to the a plus bi to the t. This will be complex and that will be complex, too.
This will have complex entries.
And then, finally, take the real and imaginary parts.
Those will be real and they will give real and two solutions.
In other words, the program is exactly like what we did for second-order differential equations.
We used the complex numbers, got complex solutions.
And then, at the very last step, we took the real and imaginary parts to get two real solutions out of each complex number.
I would like to give you a simple example of working that out.
And it is the system x prime equals x plus 2y.
And y prime equals minus x minus y.
Because it is springtime, it doesn't feel like spring but it will this weekend as it is getting warmer.
And since, when I am too tired to make up problem sets for you late at night, I watch reruns of Seinfeld.
I am from New York.
It is just in my bloodstream.
Of course, the most interesting character on Seinfeld is George.
We are going to consider Susan who is the girlfriend who got killed by licking poison envelopes.
And George carried on their love affair until Susan was disposed of by the writers by this strange death.
And we are going to consider x is modeling Susan's love for George.
That is x.
And George's love for Susan will be y.
Now, I don't mean the absolute love.
If x and y are zero, I don't mean that they don't love each other.
I just mean that that is the equilibrium value of the love.
Everything else is measured as departures from that.
So (0, 0) represents the normal amount of love, if love is measured.
I don't know what love units are.
Hearts, I guess.
Six hearts, let's say.
Now, in what sense does this model it?
This is a normal equation and this is a neurotic equation.
That is why this is George and this is Susan who seemed very normal to me.
Susan is a normal person.
When y is positive that means that George seems to be loving her more today than yesterday, and her natural response is to be more in love with him.
That is what most people are.
If y is negative, hey, what's the matter with George?
He doesn't feel so good.
Maybe there is something wrong with him.
She gets a little mad at him and this goes down.
x prime is negative.
And the same way why is this positive?
Well, again, it is a psychological thing, but all the world loves a lover.
When Susan is in love, as she feels x is high, that makes her feel good.
And she loves everything, in fact.
Not just George.
It is one of those things.
You all know what I am talking about.
Now, George, of course, is what makes the writers happy.
George is neurotic and, therefore, is exactly the opposite.
He sees one day that he feels more in love with Susan than he was yesterday.
Does this make him happy?
Not at all.
Not at all.
It makes y prime more negative.
Why?
Because all he can think of is, my God, suppose I am really in love with this girl?
Suppose I marry her.
Oh, my God, 40 years of seeing the same person at breakfast all the time.
I must be crazy.
And so it goes down.
Here is our neurotic model.
The question for differential equations is, what do the solutions to that look like?
In other words, how does, in fact, their love affair go?
Now, there is a reason why the writers picked that model, as you will see.
It means they were able to get a year's worth of episodes out of it.
And why is that so?
Well, let's solve it.
The characteristic equation is lambda squared.
The matrix that governs this system is A equals (1, 2; negative 1, negative 1) The trace of that matrix, the sum of the diagonal elements is zero.
There is the zero lambda here.
The determinant, which is the constant term, is negative 1, minus negative 2, which is plus 1.
So the characteristic equation, by calculating the trace and determinant is lambda squared plus 1 equals 0.
The eigenvalues are plus and minus i.
Now, you don't have to pick both of them because the negative one lead to essentially the same solutions but with negative signs.
Either one will do just as we solved second order equations.
The system for finding the eigenvectors, well, we are going to have to accept the complex eigenvector.
What is the system going to be?
Well, I take the matrix and I subtract i.
We will use i. Subtract i from the main diagonal.
So the system is (1 minus i) times a1 plus 2a2 is zero.
And let's, for good measure, write the other one down, too.
It is negative a1 plus minus (1 minus i) times a2.
Then what is the solution?
Well, you get the solution the usual way.
Let's take a1 equal to 1.
Then what is a2?
a2 is 1 minus i divided by 2 from the first equation.
So the complex solution is 1 minus i over 2 times e to the it.
Now you have to take the real and imaginary parts of that.
This is the only part which technically I would not trust you to do without having someone show you how to do it.
What do you do?
Well, of course, you know how to separate the real and imaginary parts of that.
It is the first thing is to separate the vectors.
I don't know how to explain this.
Just watch.
The real part of it is 1, one-half.
It should be negative 1, so minus this plus that because I didn't put that on the right side.
It is minus one-half plus i times (0, one-half).
Anybody want to fight?
1 plus i times 0 minus one-half plus one-half times i.
You saw how I did that?
Okay.
When you do these problem you do it the same way, but don't ask me to explain what I just did.
Here it is cosine t plus i sine t. And so the real part will give me one solution.
The imaginary part will give me another.
Since I have a limited amount of time, let's just calculate the real part.
What is it?
Well, it is (1, minus ½) times cosine t, i squared is negative 1, so minus (0, one-half), the negative 1 from the i squared, times sine t. Now, what solution is that?
This is (x, y).
Take the final step.
It doesn't have to look like that.
x equals cosine t. Do you see that?
x equals cosine t plus 0 times sine t. What is y?
y is minus one-half times cosine t, minus one-half times sine t, plus sine t. Now, you may have the pleasure of showing eliminating t. You get a quadratic polynomial in x and y equals zero.
This is an ellipse.
As t varies, you can see this repeats its values at intervals of 2pi, this gives an ellipse.
And if you want to use a little computer program, linear phase, this is not in the assignment, but the ellipses look like this and go around that way.
And that is the model of George and Susan's love.
x, Susan.
y, George.
They go round and round in this little love circle, and it stretches on for 26 episodes.
As a matter of fact, it plots them very accurately.
But it is something you also need to learn to do yourself, as you will see when we study nonlinear equations.
It is a skill.
And since a couple of important mathematical ideas are involved in it, I think it is a very good thing to spend just a little time on, one lecture in fact, plus a little more on the problem set that I will give out.
The last problem set that I will give out on Friday.
I thought it might be a little more fun to, again, have a simple-minded model.
No romance this time.
We are going to have a little model of war, but I have made it sort of sublimated war.
Let's take as the system, I am going to let two of those be parameters, you know, be variable, in other words.
And the other two I will keep fixed, so that you can concentrate on them better.
I will take a and d to be negative 1 and negative 3.
And the other ones we will leave open, so let's call this one b times y, and this other one will be c times x. I am going to model this as a fight between two states, both of which are trying to attract tourists.
Let's say this is Massachusetts and this will be New Hampshire, its enemy to the North.
Both are busy advertising these days on television.
People are making their summer plans.
Come to New Hampshire, you know, New Hampshire has mountains and Massachusetts has quaint little fishing villages and stuff like that.
So what are these numbers?
Well, first of all, what do x and y represent?
x and y basically are the advertising budgets for tourism, you know, the amount each state plans to spend during the year.
However, I do not want zero value to mean they are not spending anything.
It represents departure from the normal equilibrium.
x and y represent departures -- -- from the normal amount of money they spend advertising for tourists.
The normal tourist advertising budget.
If they are both zero, it means that both states are spending what they normally spend in that year.
If x is positive, it means that Massachusetts has decided to spend more in the hope of attracting more tourists and if negative spending less.
What is the significance of these two coefficients?
Those are the normal things which return you to equilibrium.
In other words, if x gets bigger than normal, if Massachusetts spends more there is a certain poll to spend less because we are wasting all this money on the tourists that are not going to come when we could be spending it on education or something like that.
If x gets to be negative, the governor tries to spend less.
Then all the local city Chamber of Commerce rise up and start screaming that our economy is going to go bankrupt because we won't get enough tourists and that is because you are not spending enough money.
There is a push to always return it to the normal, and that is what this negative sign means.
The same thing for New Hampshire.
What does it mean that this is negative three and that is negative one?
It just means that the Chamber of Commerce yells three times as loudly in New Hampshire.
It is more sensitive, in other words, to changes in the budget.
Now, how about the other?
Well, these represent the war-like features of the situation.
Normally these will be positive numbers.
Because when Massachusetts sees that New Hampshire has budgeted this year more than its normal amount, the natural instinct is we are fighting.
This is war.
This is a positive number.
We have to budget more, too.
And the same thing for New Hampshire.
The size of these coefficients gives you the magnitude of the reaction.
If they are small Massachusetts say, well, they are spending more but we don't have to follow them.
We will bucket a little bit.
If it is a big number then oh, my God, heads will roll.
We have to triple them and put them out of business.
This is a model, in fact, for all sorts of competition.
It was used for many years to model in simper times armaments races between countries.
It is certainly a simple-minded model for any two companies in competition with each other if certain conditions are met.
Well, what I would like to do now is try different values of those numbers.
And, in each case, show you how to sketch the solutions at different cases.
And then, for each different case, we will try to interpret if it makes sense or not.
My first set of numbers is, the first case is -- -- x prime equals negative x plus 2y.
And y prime equals, this is going to be zero, so it is simply minus 3 times y.
Now, what does this mean?
Well, this means that Massachusetts is behaving normally, but New Hampshire is a very placid state, and the governor is busy doing other things.
And people say Massachusetts is spending more this year, and the Governor says, so what.
The zero is the so what factor.
In other words, we are not going to respond to them.
We will do our own thing.
What is the result of this?
Is Massachusetts going to win out?
What is going to be the ultimate effect on the budget?
Well, what we have to do is, so the program is first let's quickly solve the equations using a standard technique.
I am just going to make marks on the board and trust to the fact that you have done enough of this yourself by now that you know what the marks mean.
I am not going to label what everything is.
I am just going to trust to luck.
The matrix A is negative 1, 2, 0, negative 3.
The characteristic equation, the second coefficient is the trace, which is minus 4, but you have to change its sign, so that makes it plus 4.
And the constant term is the determinant, which is 3 minus 0, so that is plus 3 equals zero.
This factors into lambda plus 3 times lambda plus one.
And it means the roots therefore are, one root is lambda equals negative 3 and the other root is lambda equals negative 1.
These are the eigenvalues.
With each eigenvalue goes an eigenvector.
The eigenvector is found by solving an equation for the coefficients of the eigenvector, the components of the eigenvector.
Here I used negative 1 minus negative 3, which makes 2.
The first equation is 2a1 plus 2a2 is equal to zero.
The second one will be, in fact, in this case simply 0a1 plus 0a2 so it won't give me any information at all.
That is not what usually happens, but it is what happens in this case.
What is the solution?
The solution is the vector alpha equals, well, 1, negative 1 would be a good thing to use.
That is the eigenvector, so this is the e-vector.
How about lambda equals negative 1?
Let's give it a little more room.
If lambda is negative 1 then here I put negative 1 minus negative 1.
That makes zero.
I will write in the zero because this is confusing.
It is zero times a1.
And the next coefficient is 2 a2, is zero.
People sometimes go bananas over this, in spite of the fact that this is the easiest possible case you can get.
I guess if they go bananas over it, it proves it is not all that easy, but it is easy.
What now is the eigenvector that goes with this?
Well, this term isn't there.
It is zero.
The equation says that a2 has to be zero.
And it doesn't say anything about a1, so let's make it 1.
Now, out of this data, the final step is to make the general solution.
What is it?
(x, y) equals, well, a constant times the first normal mode.
The solution constructed from the eigenvalue and the eigenvector.
That is going to be 1, negative 1 e to the minus 3t.
And then the other normal mode times an arbitrary constant will be (1, 0) times e to the negative t. The lambda is this factor which produces that, of course.
Now, one way of looking at it is, first of all, get clearly in your head this is a pair of parametric equations just like what you studied in 18.02.
Let's write them out explicitly just this once.
x equals c1 times e to the negative 3t plus c2 times e to the negative t. And what is y?
y is equal to minus c1 e to the minus 3t plus zero.
I can stop there.
In some sense, all I am asking you to do is plot that curve.
In the x,y-plane, plot the curve given by this pair of parametric equations.
And you can choose your own values of c1, c2.
For different values of c1 and c2 there will be different curves.
Give me a feeling for what they all look like.
Well, I think most of you will recognize you didn't have stuff like this.
These weren't the kind of curves you plotted.
When you did parametric equations in 18.02, you did stuff like x equals cosine t, y equals sine t. Everybody knows how to do that.
A few other curves which made lines or nice things, but nothing that ever looked like that.
And so the computer will plot it by actually calculating values but, of course, we will not.
That is the significance of the word sketch.
I am not asking you to plot carefully, but to give me some general geometric picture of what all these curves look like without doing any work.
Without doing any work.
Well, that sounds promising.
Okay, let's try to do it without doing any work.
Where shall I begin?
Hidden in this formula are four solutions that are extremely easy to plot.
So begin with the four easy solutions, and then fill in the rest.
Now, which are the easy solutions?
The easy solutions are c1 equals plus or minus 1, c2 equals zero, or c1 equals zero, or c1 = 0, c2 equals plus or minus 1.
By choosing those four values of c1 and c2, I get simple solutions corresponding to the normal mode.
If c1 is one and c2 is zero, I am talking about (1, negative 1) e to the minus 3t, and that is very easy plot.
Let's start plotting them.
What I am going to do is color-code them so you will be able to recognize what it is I am plotting.
Let's see.
What colors should we use?
We will use pink and orange.
This will be our pink solution and our orange solution will be this one.
Let's plot the pink solution first.
The pink solution corresponds to c1 equals 1 and c2 equals zero.
Now, that solution looks like-- Let's write it in pink.
No, let's not write it in pink.
What is the solution?
It looks like x equals e to the negative 3t, y equals minus e to the minus 3t.
Well, that's not a good way to look at it, actually.
The best way to look at it is to say at t equals zero, where is it?
It is at the point 1, negative 1.
And what is it doing as t increases?
Well, it keeps the direction, but travels.
The amplitude, the distance from the origin keeps shrinking.
As t increases, this factor, so it is the tip of this vector, except the vector is shrinking.
It is still in the direction of 1, negative 1, but it is shrinking in length because its amplitude is shrinking according to the law e to the negative 3t.
In other words, this curve looks like this.
At t equals zero it is over here, and it goes along this diagonal line until as t equals infinity, it gets to infinity, it reaches the origin.
Of course, it never gets there.
It goes slower and slower and slower in order that it may never reach the origin.
What was it doing for values of t less than zero?
The same thing, except it was further away.
It comes in from infinity along that straight line.
In other words, the eigenvector determines the line on which it travels and the eigenvalue determines which way it goes.
If the eigenvalue is negative, it is approaching the origin as t increases.
How about the other one?
Well, if c1 is negative 1, then everything is the same except it is the mirror image of this one.
If c1 is negative 1, then at t equals zero it is at this point.
And, once again, the same reasoning shows that it is coming into the origin as t increases.
I have now two solutions, this one corresponding to c1 equals 1, and the other one c2 equals zero.
This one corresponds to c1 equals negative 1.
How about the other guy, the orange guy?
Well, now c1 is zero, c2 is one, let's say.
It is the vector (1, 0), but otherwise everything is the same.
I start now at the point (1, 0) at time zero.
And, as t increases, I come into the origin always along that direction.
And before that I came in from infinity.
And, again, if c2 is 1 and if c2 is negative 1, I do the same thing but on the other side.
That wasn't very hard.
I plotted four solutions.
And now I roll up my sleeves and waive my hands to try to get others.
The general philosophy is the following.
The general philosophy is the differential equation looks like this.
It is a system of differential equations.
These are continuous functions.
That means when I draw the velocity field corresponding to that system of differential equations, because their functions are continuous, as I move from one (x, y) point to another the direction of the velocity vectors change continuously.
It never suddenly reverses without something like that.
Now, if that changes continuously then the trajectories must change continuously, too.
In other words, nearby trajectories should be doing approximately the same thing.
Well, that means all the other trajectories are ones which come like that must be going also toward the origin.
If I start here, probably I have to follow this one.
They are all coming to the origin, but that is a little too vague.
How do they come to the origin?
In other words, are they coming in straight like that?
Probably not.
Then what are they doing?
Now we are coming to the only point in the lecture which you might find a little difficult.
Try to follow what I am doing now.
If you don't follow, it is not well done in the textbook, but it is very well done in the notes because I wrote them myself.
Please, it is done very carefully in the notes, patiently follow through the explanation.
It takes about that much space.
It is one of the important ideas that your engineering professors will expect you to understand.
Anyway, I know this only from the negative one because they say to me at lunch, ruin my lunch by saying I said it to my students and got nothing but blank looks.
What do you guys teach them over there?
Blah, blah, blah.
Maybe we ought to start teaching it ourselves.
Sure.
Why don't they start cutting their own hair, too?
Here is the idea.
Let me recopy that solution.
The solution looks like (1, negative 1) e to the minus 3t plus c2, (1, 0) e to the negative t. What I ask is as t goes to infinity, I feel sure that the trajectories must be coming into the origin because these guys are doing that.
And, in fact, that is confirmed.
As t goes to infinity, this goes to zero and that goes to zero regardless of what the c1 and c2 are.
That makes it clear that this goes to zero no matter what the c1 and c2 are as t goes to infinity, but I would like to analyze it a little more carefully.
As t goes to infinity, I have the sum of two terms.
And what I ask is, which term is dominant?
Of these two terms, are they of equal importance, or is one more important than the other?
When t is 10, for example, that is not very far on the way to infinity, but it is certainly far enough to illustrate.
Well, e to the minus 10 is an extremely small number.
The only thing smaller is e to the minus 30.
The term that dominates, they are both small, but relatively-speaking this one is much larger because this one only has the factor e to the minus 10, whereas, this has the factor e to the minus 30, which is vanishingly small.
In other words, as t goes to infinity -- Well, let's write it the other way.
This is the dominant term, as t goes to infinity.
Now, just the opposite is true as t goes to minus infinity.
t going to minus infinity means I am backing up along these curves.
As t goes to minus infinity, let's say t gets to be negative 100, this is e to the 100, but this is e to the 300, which is much, much bigger.
So this is the dominant term as t goes to negative infinity.
Now what I have is the sum of two vectors.
Let's first look at what happens as t goes to infinity.
As t goes to infinity, I have the sum of two vectors.
This one is completely negligible compared with the one on the right-hand side.
In other words, for a all intents and purposes, as t goes to infinity, it is this thing that takes over.
Therefore, what does the solution look like as t goes to infinity?
The answer is it follows the yellow line.
Now, what does it look like as it backs up?
As it came in from negative infinity, what does it look like?
Now, this one is a little harder to see.
This is big, but this is infinity bigger.
I mean very, very much bigger, when t is a large negative number.
Therefore, what I have is the sum of a very big vector.
You're standing on the moon looking at the blackboard, so this is really big.
This is a very big vector.
This is one million meters long, and this is only 20 meters long.
That is this guy, and that is this guy.
I want the sum of those two.
What does the sum look like?
The answer is a sum is approximately parallel to the long guy because this is negligible.
This does not mean they are next to each other.
They are slightly tilted over, but not very much.
In other words, as t goes to negative infinity it doesn't coincide with this vector.
The solution doesn't, but it is parallel to it.
It has the same direction.
I am done.
It means far away from the origin, it should be parallel to the pink line.
Near the origin it should turn and become more or less coincident with the orange line.
And those were the solutions.
That's how they look.
How about down here?
The same thing, like that, but then after a while they turn and join.
Here, they have to turn around to join up, but they join.
And that is, in a simple way, the sketches of those functions.
That is how they must look.
What does this say about our state?
Well, it says that the fact that the governor of New Hampshire is indifferent to what Massachusetts is doing produces ultimately harmony.
Both states revert ultimately their normal advertising budgets in spite of the fact that Massachusetts is keeping an eye peeled out for the slightest misbehavior on the part of New Hampshire.
Peace reins, in other words.
Now you should know some names.
Let's see.
I will write names in purple.
There are two words that are used to describe this situation.
First is the word that describes the general pattern of the way these lines look.
The word for that is a node.
And the fact that all the trajectories end up at the origin for that one uses the word sink.
This could be modified to nodal sink.
That would be better.
Nodal sink, let's say.
Nodal sink or, if you like to write them in the opposite order, sink node.
In the same way there would be something called a source node if I reversed all the arrows.
I am not going to calculate an example.
Why don't I simply do it by giving you -- For example, if the matrix A produced a solution instead of that one.
Suppose it looked like 1, negative 1 e to the 3t.
The eigenvalues were reversed, were now positive.
And I will make the other one positive, too.
c2 1, 0 e to the t. What would that change in the picture?
The answer is essentially nothing, except the direction of the arrows.
In other words, the first thing would still be 1, negative 1.
The only difference is that now as t increases we go the other way.
And here the same thing, we have still the same basic vector, the same basic orange vector, orange line, but it has now traversed the solution.
We traverse it in the opposite direction.
Now, let's do the same thing about dominance, as we did before.
Which term dominates as t goes to infinity?
This is the dominant term.
Because, as t goes to infinity, 3t is much bigger than t. This one, on the other hand, dominates as t goes to negative infinity.
How now will the solutions look like?
Well, as t goes to infinity, they follow the pink curve.
Whereas, as t starts out from negative infinity, they follow the orange curve.
As t goes to infinity, they become parallel to the pink curve, and as t goes to negative infinity, they are very close to the origin and are following the yellow curve.
This is pink and this is yellow.
They look like this.
Notice the picture basically is the same.
It is the picture of a node.
All that has happened is the arrows are reversed.
And, therefore, this would be called a nodal source.
The word source and sink correspond to what you learned in 18.02 and 8.02, I hope, also, or you could call it a source node.
Both phrases are used, depending on how you want to use it in a sentence.
And another word for this, this would be called unstable because all of the solutions starting out from near the origin ultimately end up infinitely far away from the origin.
This would be called stable.
In fact, it would be called asymptotically stable.
I don't like the word asymptotically, but it has become standard in the literature.
And, more important, it is standard in your textbook.
And I don't like to fight with a textbook.
It just ends up confusing everybody, including me.
That is enough for nodes.
I would like to talk now about some of the other cases that can occur because they lead to completely different pictures that you should understand.
Let's look at the case where our governors behave a little more badly, a little more combatively.
It is x prime equals negative x as before, but this time a firm response by Massachusetts to any sign of increased activity by stockpiling of advertising budgets.
Here let's say New Hampshire now is even worse.
Five times, quintuple or whatever increase Massachusetts makes, of course they don't have an income tax, but they will manage.
Minus 3y as before.
Let's again calculate quickly what the characteristic equation is.
Our matrix is now negative 1, 3, 5 and negative 3.
The characteristic equation now is lambda squared.
What is that?
Again, plus 4 lambda.
But now the determinant is 3 minus 15 is negative 12.
And this, because I prepared very carefully, all eigenvalues are integers.
And so this factors into lambda plus 6 times lambda minus 2, does it not?
Yes.
6 lambda minus 2 is four lambda.
Good.
What do we have?
Well, first of all we have our eigenvalue lambda, negative 6.
And the eigenvector that goes with that is minus 1.
This is negative 1 minus negative 6 which makes, shut your eyes, 5.
We have 5a1 plus 3a2 is zero.
And the other equation, I hope it comes out to be something similar.
I didn't check.
I am hoping this is right.
The eigenvector is, okay, you have been taught to always make one of the 1, forget about that.
Just pick numbers that make it come out right.
I am going to make this one 3, and then I will make this one negative 5.
As I say, I have a policy of integers only.
I am a number theorist at heart.
That is how I started out life anyway.
There we have data from which we can make one solution.
How about the other one?
The other one will correspond to the eigenvalue lambda equals 2.
This time the equation is negative 1 minus 2 is negative 3.
It is minus 3a1 plus 3a2 is zero.
And now the eigenvector is (1, 1).
Now we are ready to draw pictures.
We are going to make this similar analysis, but it will go faster now because you have already had the experience of that.
First of all, what is our general solution?
It is going to be c1 times 3, negative 5 e to the minus 6t.
And then the other normal mode times an arbitrary constant will be 1, 1 times e to the 2t.
I am going to use the same strategy.
We have our two normal modes here, eigenvalue, eigenvector solutions from which, by adjusting these constants, we can get our four basic solutions.
Those are going to look like, let's draw a picture here.
Again, I will color-code them.
Let's use pink again.
The pink solution now starts at 3, negative 5.
That is where it is when t is zero.
And, because of the coefficient minus 6 up there, it is coming into the origin and looks like that.
And its mirror image, of course, does the same thing.
That is when c1 is negative one.
How about the orange guy?
Well, when t is equal to zero, it is at 1, 1.
But what is it doing after that?
As t increases, it is getting further away from the origin because the sign here is positive.
e to the 2t is increasing, it is not decreasing anymore, so this guy is going out.
And its mirror image on the other side is doing the same thing.
Now all we have to do is fill in the picture.
Well, you fill it in by continuity.
Your nearby trajectories must be doing what similar thing?
If I start out very near the pink guy, I should stay near the pink guy.
But as I get near the origin, I am also approaching the orange guy.
Well, there is no other possibility other than that.
If you are further away you start turning a little sooner.
I am just using an argument from continuity to say the picture must be roughly filled out this way.
Maybe not exactly.
In fact, there are fine points.
And I am going to ask you to do one of them on Friday for the new problem set, even before the exam, God forbid.
But I want you to get a little more experience working with that linear phase portrait visual because it is, I think, one of the best ones this semester.
You can learn a lot from it.
Anyway, you are not done with it, but I hope you have at least looked at it by now.
That is what the picture looks like.
First of all, what are we going to name this?
In other words, forget about the arrows.
If you just look at the general way those lines go, where have you seen this before?
You saw this in 18.02.
What was the topic?
You were plotting contour curves of functions, were you not?
What did you call contours curves that formed that pattern?
A saddle point.
You called this a saddle point because it was like the center of a saddle.
It is like a mountain pass.
Here you are going up the mountain, say, and here you are going down, the way the contour line is going down.
And this is sort of a min and max point.
A maximum if you go in that direction and a minimum if you go in that direction, say.
Without the arrows on it, it is like a saddle point.
And so the same word is used here.
It is called the saddle.
You don't say point in the same way you don't say a nodal point.
It is the whole picture, as it were, that is the saddle.
It is a saddle.
There is the saddle.
This is where you sit.
Now, should I call it a source or a sink?
I cannot call it either because it is a sink along these lines, it is a source along those lines and along the others, it starts out looking like a sink and then turns around and starts acting like a source.
The word source and sink are not used for saddle.
The only word that is used is unstable because definitely it is unstable.
If you start off exactly on the pink lines you do end up at the origin, but if you start anywhere else ever so close to a pink line you think you are going to the origin, but then at the last minute you are zooming off out to infinity again.
This is a typical example of instability.
Only if you do the mathematically possible, but physically impossible thing of starting out exactly on the pink line, only then will you get to the origin.
If you start out anywhere else, make the slightest error in measure and get off the pink line, you end off at infinity.
What is the effect with our war-like governors fighting for the tourist trade willing to spend any amounts of money to match and overmatch what their competitor in the nearby state is spending?
The answer is, they all lose.
Since it is mostly this section of the diagram that makes sense, what happens is they end up all spending an infinity of dollars and nobody gets any more tourists than anybody else.
So this is a model of what not to do.
I have one more model to show you.
Maybe we better start over at this board here.
Massachusetts on top.
New Hampshire on the bottom.
x prime is going to be, that is Massachusetts, I guess as before.
Let me get the numbers right.
Leave that out for a moment.
y prime is 2x minus 3y.
New Hampshire behaves normally.
It is ready to respond to anything Massachusetts can put out.
But by itself, it really wants to bring its budget to normal.
Now, Massachusetts, we have a Mormon governor now, I guess.
Imagine instead we have a Buddhist governor.
A Buddhist governor reacts as follows, minus y.
What does that mean?
It means that when he sees New Hampshire increasing the budget, his reaction is, we will lower ours.
We will show them love.
It looks suicidal, but what actually happens?
Well, our little program is over.
Our matrix a is negative 1, negative 1, 2, negative 3.
The characteristic equations is lambda squared plus 4 lambda.
And now what is the other term?
3 minus negative 2 makes 5.
This is not going to factor because I tried it out and I know it is not going to factor.
We are going to get lambda equals, we will just use the quadratic formula, negative 4 plus or minus the square root of 16 minus 4 times 5, that is 16 minus 20 or negative 4 all divided by 2, which makes minus 2, pull out the 4, that makes it a 2, cancels this 2, minus 1 inside.
It is minus 2 plus or minus i.
Complex solutions.
What are we doing to do about that?
Well, you should rejoice when you get this case and are asked to sketch it because, even if you calculate the complex eigenvector and from that take its real and imaginary parts of the complex solution, in fact, you will not be able easily to sketch the answer anyway.
But let me show you what sort of thing you can get and then I am going to wave my hands and argue a little bit to try to indicate what it is that the solution actually looks like.
You are going to get something that looks like -- A typical real solution is going to look like this.
This is going to produce e to the minus 2t times e to the i t. e to the minus 2 plus i all times t. This will be our exponential factor which is shrinking in amplitude.
This is going to give me sines and cosines.
When I separate out the eigenvector into its real and imaginary parts, it is going to look something like this.
a1, a2 times cosine t, that is from the e to the it part.
Then there will be a sine term.
And all that is going to be multiplied by the exponential factor e to the negative 2t.
That is just one normal mode.
It is going to be c1 times this plus c2 times something similar.
It doesn't matter exactly what it is because they are all going to look the same.
Namely, this is a shrinking amplitude.
I am not going to worry about that.
My real question is, what does this look like?
In other words, as a pair of parametric equations, if x is equal to a1 cosine t plus b1 sine t and y is a2 cosine plus b2 sine, what does it look like?
Well, what are its characteristics?
In the first place, as a curve this part of it is bounded.
It stays within some large box because cosine and sine never get bigger than one and never get smaller than minus one.
It is periodic.
As t increases to t plus 2pi, it comes back to exactly the same point it was at before.
We have a curve that is repeating itself periodically, it does not go off to infinity.
And here is where I am waving my hands.
It satisfies an equation.
Those of you who like to fool around with mathematics a little bit, it is not difficult to show this, but it satisfies an equation of the form A x squared plus B y squared plus C xy equals D. All you have to do is figure out what the coefficients A, B, C and D should be.
And the way to do it is, if you calculate the square of x you are going to get cosine squared, sine squared and a cosine sine term.
You are going to get those same three terms here and the same three terms here.
You just use undetermined coefficients, set up a system of simultaneous equations and you will be able to find the A, B, C and D that work.
I am looking for a curve that is bounded, keeps repeating its values and that satisfies a quadratic equation which looks like this.
Well, an earlier generation would know from high school, these curves are all conic sections.
The only curves that satisfy equations like that are hyperbola, parabolas, the conic sections in other words, and ellipses.
Circles are a special kind of ellipses.
There is a degenerate case.
A pair of lines which can be considered a degenerate hyperbola, if you want.
It is as much a hyperbola as a circle, as an ellipse say.
Which of these is it?
Well, it must be those guys.
Those are the only guys that stay bounded and repeat themselves periodically.
The other guys don't do that.
These are ellipses.
And, therefore, what do they look like?
Well, they must look like an ellipse that is trying to be an ellipse, but each time it goes around the point is pulled a little closer to the origin.
It must be doing this, in other words.
And such a point is called a spiral sink.
Again sink because, no matter where you start, you will get a curve that spirals into the origin.
Spiral is self-explanatory.
And the one thing I haven't told you that you must read is how do you know that it goes around counterclockwise and not clockwise?
Read clockwise or counterclockwise.
I will give you the answer in 30 seconds, not for this particular curve.
That you will have to calculate.
All you have to do is put in somewhere.
Let's say at the point (1, 0), a single vector from the velocity field.
In other words, at the point (1, 0), when x is 1 and y is 0 our vector is minus 1, 2, which is the vector minus 1, 2, it goes like this.
Therefore, the motion must be counterclockwise.
And, by the way, what is the effect of having a Buddhist governor?
Peace.
Everything spirals into the origin and everybody is left with the same advertising budget they always had.
Thanks.
The real topic is how to solve inhomogeneous systems, but the subtext is what I wrote on the board.
I think you will see that really thinking in terms of matrices makes certain things a lot easier than they would be otherwise.
And I hope to give you a couple of examples of that today in connection with solving systems of inhomogeneous equations.
Now, there is a little problem.
We have to have a little bit of theory ahead of time before that, which I thought rather than interrupt the presentation as I try to talk about the inhomogeneous systems it would be better to put a little theory in the beginning.
I think you will find it harmless.
And about half of it you know already.
The theory I am talking about is, in general, the theory of the systems x prime equal a x. I will just state it when n is equal to two.
A two-by-two system likely you have had up until now.
It is also true for end-by-end.
It is just a little more tedious to write out and to give the definitions.
Here is a little two-by-two system.
It is homogeneous.
There are no zeros.
And it is not necessary to assume this, but since the matrix is going to be constant until the end of the term let's assume it in and not go for a spurious generality.
So constant matrices like you will have on your homework.
Now, there are two theorems, or maybe three that I want you to know, that you need to know in order to understand what is going on.
The first one, fortunately, is already in your bloodstream, I hope.
Let's call it theorem A.
It is simply the one that says that the general solution to the system, that system I wrote on the board, the two-by-two system is what you know it to be.
Namely, from all the examples that you have calculated.
It is a linear combination with arbitrary constants for the coefficients of two solutions.
In other words, to solve it, to find the general solution you put all your energy into finding two independent solutions.
And then, as soon as you found them, the general one is gotten by combining those with arbitrary constants.
The only thing to specify is what the x1 and the x2 are.
"Where," I guess, would be the right word to use.
Where x1 and x2 are two solutions, but neither must be a constant multiple of the other.
That is the only thing I want to stress, they have to be independent.
Or, as it is better to say, linearly independent.
Are two linearly independent solutions.
And department of fuller explanation, i.e., neither is a constant multiple of the other.
That is what it means to be linearly independent.
Now, this theorem I am not going to prove.
I am just going to say that the proof is a lot like the one for second order equations.
It has an easy part and a hard part.
The easy part is to show that all of these guys are solutions.
And, in fact, that is almost self-evident by looking at the equation.
For example, if x1 and x2, each of those solve that equation so does their sum because, when you plug it in, you differentiate the sum by differentiating each term and adding.
And here A times x1 plus x2 is Ax1 plus Ax2.
In other words, you are using the linearity and the superposition principle.
It is easy to show that all of these, well, maybe I should actually write something down instead of just talking.
Easy that all these are solutions.
Every one of those guys, regardless of what c1 and c2 is, is a solution.
That is linearity, if I use that buzz word, plus the superposition principle, that the sum of two solutions is a solution.
The hard thing is not to show that these are solutions but to show that these are all the solutions, that there are no other solutions.
No matter how you do that it is hard.
The hard thing is that there are no other solutions.
These are all.
Now, you could sort of say, well, it has two arbitrary constants in it.
That is sort of a rough and ready reason, but it is not considered adequate by mathematicians.
And, in fact, I could go into a song and dance as to just why it is inadequate.
But we have other things to do, bigger fish to fry, as they say.
Let's fry a fish.
No, we have another theorem first.
This one it is mostly the words that I am interested in.
Once again, we have our old friend the Wronskian back.
The Wronskian of what?
Of two solutions.
It is the Wronskian of the solution x1 and x2.
They don't, by the way, have to be independent.
Just two solutions to the system.
And what is it?
Hey, didn't we already have a Wronskian?
Yeah.
Forget about that one for the moment.
Postpone it for a minute.
This is a determinant, just like the old one way.
This is going to be a great lecture.
(x1, x2).
Now what is this?
x1 is a column vector, right?
x2 is a column vector.
Two things in it.
Two things in it.
Together they make a square matrix.
And this means it is determinant.
It is the determinant of this.
It is a determinant, in other words, of a square matrix.
And that is what it is.
I will change this equality.
To indicate it is a definition, I will put the colon there, which is what you add, to indicate this is only equal because I say so.
It is a definition, in other words.
Now, there is a connection between this and the earlier Wronskian which I, unfortunately, cannot explain to you because you are going to explain it to me.
I gave it to you as part one of your homework problem.
Make sure you do it.
And, if you cannot remember what the old Wronskian is, please look it up in the book.
Don't look it up in the solution to the problem.
If you do that you will learn something.
Then you will see how, in a certain sense, this is a more general definition than I gave you before.
The one I gave you before is, in a certain sense, a special case of it.
Now that is just the definition.
There is a theorem.
And the theorem is going to look just like the one we had for second order equations, if you can remember back that far.
The theorem is that if these are two solutions there are only two possibilities for the Wronskian.
So either or.
Either the Wronskian is -- Now, the Wronskian, these are functions, the column vectors are the solutions, so those are functions of the variable t, so are these.
The Wronskian as a whole is a function of the independent variable t after you have calculated out that determinant.
I will write it now this way to indicate that it s a function of t. Either the Wronskian is -- One possibility is identically zero.
That is zero for all values of t, in other words.
And this happens if x1 and x2 are not linearly independent.
Usually people just say dependent and hope they are interpreted correctly.
Are dependent.
But since I did not explain what dependent means, I will say it.
Not linearly independent.
I know that is horrible, but nobody has figured out another way to say it.
That is one possibility, or the opposite of this is never zero for any t value.
I mean a normal function is zero here and there, and the rest of the time not zero.
Well, not this Wronskian.
You only have two choices.
Either it is zero all the time or it is never zero.
It is like the function e to the t. In other words, an exponential which is never zero, always positive and never zero.
Or, it could be a constant.
Anyway, it has to be a function which is never zero.
And this happens in the other case, so this is -- There is no place to write it.
This is the case if x1 and x2 are independent, by which I mean linearly independent.
It is just I didn't have room to write it.
That is pretty much the end of the theory.
And now, let's start in on the matrices.
The basic new matrix we are going to be talking about this period and next one on Monday also is the way that most people who work with systems actually look at the solutions to systems, so it is important you learn this word and this way of looking at it.
What they do is look not at each solution separately, as we have been doing up until now.
They put them all together in a single matrix.
And it is the properties of that matrix that they study and try to do the calculations using.
And that matrix is called the fundamental matrix for the system.
Sometimes people don't bother writing in the whole system.
They just say it is a fundamental matrix for A because, after all, A is the only thing that is varying there.
Once you know A, you know what the system is.
So what is this guy?
Well, it is a two-by-two matrix.
And it is the most harmless thing.
It is the precursor of the Wronskian.
It is what the Wronskian was before the determinant was taken.
In other words, it is the matrix whose two columns are those two solutions.
The other question is what we are going to call it.
I kept trying everything and settled on calling it capital X because I think that is the one that guides you in the calculations the best.
This is definition two, so colon equality.
Notice I am not using vertical lines now because that would mean a determinant.
It is the matrix whose columns are two independent solutions.
Is that all?
Yeah.
You just put them side-by-side.
Why?
That will come out.
Why should one do this?
Well, first of all, in order not to interrupt the basic calculation that I want to make with this during the period, it has two basic properties that we are going to need during this period.
These are the properties.
Just two.
And one is obvious and the other you will think, I hope, is a little less familiar.
I think you will see there is nothing to it.
It is just a way of talking, really.
The first is the one that is already embedded in the theorem, namely that the determinant of the fundamental matrix is not zero for any t. Why?
Well, I just told you it wasn't.
This is the Wronskian.
The Wronskian is never zero?
Why is it never zero?
Well, because I said these columns had to be independent solutions.
So this is not just not zero, it is never zero.
It is not zero for any value of t. That is good.
As you will see, we are going to need that property.
But the other one is a little stranger.
The only thing I can say is, get used to it.
Namely that X prime equals AX.
Now, why is that strange?
That is not the same as this.
This is a column vector.
That is a square matrix and this is a column vector.
This is not a column vector.
This is a square matrix.
This is what is called a matrix differential equation where the variable is not a single x or a column vector of a set of x's like the x and the y.
It is a whole matrix.
Well, first of all, I should say what is it saying?
This is a two-by-two matrix.
When I multiply them I get a two-by-two matrix.
What is this?
This is a two-by-two matrix, every entry of which has been differentiated.
That is what it means to put that prime there.
To differentiate a matrix means nothing fancy.
It just means differentiate every entry.
It is just like to differentiate a vector (x, y), to make a velocity vector you differentiate the x and the y.
Well, a column vector is a special kind of matrix.
The definition applies to any matrix.
Well, why is that so?
I state it as a property, but I will continue it by giving you, so to speak, the proof of it.
In fact, there is nothing in this.
It is nothing more than a little matrix calculation of the most primitive kind.
Namely, what does this mean?
Let's try to undo that.
What does the left-hand side really mean?
Well, if that is what x means, the left-hand side must mean the derivative of the first column.
That is its first column.
And the derivative of the second column.
That is what it means to differentiate the matrix X.
You differentiate each column separately.
And to differentiate the column you need to differentiate every function in it.
Well, what does the right-hand side mean?
Well, I am supposed to take A and multiply that by [x1,x2].
Now, I don't know how to prove this, except ask you to think about it.
Or, I could write it all out here.
But think of this as a bing, bing, bing, bing.
And this is a bing, bing.
And this is a bong, bong.
How do I do the multiplication?
In other words, what is in the first column of the matrix?
Well, it is dah, dah, and the lower thing is dah, dah.
In other words, it is A times x1.
Shut your eyes and visualize it.
Got it?
Dah, dah is the top entry, and dah, dah is the bottom entry.
It is what you get by multiplying A by the column vector x1.
And the same way the other guy is -- -- what you get by multiplying A by the column vector x2.
This is just matrix multiplication.
That is the law of matrix multiplication.
That is how you multiply matrices.
Well, good, but where does this get us?
What does it mean for those two guys to be equal?
That is going to happen, if and only if x1 prime is equal to A x1.
This guy equals that guy.
And similarly for the x2's.
The end result is that this matrix, saying that the fundamental matrix satisfies this matrix differential equation is only a way of saying, in one breath, that its two columns are both solutions to the original system.
It is, so to speak, an efficient way of turning these two equations into a single equation by making a matrix.
I guess it is time, finally, to come to the topic of the lecture.
I said the thing the matrices were going to be used for is solving inhomogeneous systems, so let's take a look at those.
I thought I would give you an example.
Inhomogeneous systems.
Well, what is one going to look like?
So far what we have done is, up until now has been solving, we spent essentially two weeks solving and plotting the solutions to homogeneous systems.
There was nothing over there.
And homogeneous systems, in fact, with constant coefficients.
Stuff that looked like that that we abbreviated with matrices.
Now, to make the system inhomogeneous what I do is add the extra term on the right-hand side, which is some function of t. Except, I will have to have two functions of t because I have two equations.
Now it is inhomogeneous.
And what makes it inhomogeneous is the fact that these are not zero anymore.
There is something there.
Functions of t are there.
These are given functions of t like exponentials, polynomials, the usual stuff you have on the right-hand side of the differential equation.
What is confusing here is that when we studied second order equations it was homogeneous if the right-hand side was zero, and if there was something else there it was inhomogeneous.
Unfortunately, I have stuck this stuff on the right-hand side so it is not quite so clear anymore.
It has got to look like that, in other words.
How would the matrix abbreviation look?
Well, the left-hand side is x prime.
The homogenous part is ax, just as it has always been.
The only extra part is those functions r. And this is a column vector, after the multiplication this is a column vector, what is left is column vector.
Now, explicitly it is a function of t, given by explicit functions of t, again, like exponentials.
Or, they could be fancy functions.
That is the thing we are trying to solve.
Why don't I put it up in green?
Our new and better and improved system.
Think back to what we did when we studied inhomogeneous equations.
We are not talking about systems but just a single equation.
What we did was the main theorem -- I guess there are going to be three theorems today, not just two.
Theorem C. Is that right?
Yes, A, B.
We are up to C. Theorem C says that the general solution, that is, the general solution to the system, is equal to the complimentary function, which is the general solution to x prime equals Ax, -- -- the homogeneous equation, in other words, plus, what am I going to call it?
(x)p, right you are, a particular solution.
But the principle is the same and is proved exactly the same way.
It is just linearity and superposition.
The linearity of the original system and the superposition principle.
The essence is that to solve this inhomogeneous system, what we have to do is find a particular solution.
This part I already know how to do.
We have been doing that for two weeks.
The new thing is to find this.
Now, if you remember back before spring break, most of the work in solving the second order equation was in finding that particular solution.
You quickly enough learned how to solve the homogeneous equation, but there was no real general method for finding this.
We had an exponential input theorem with some modifications to it.
We took a week's detour in Fourier series to see how to do it for periodic functions or functions defined on finite intervals.
There were other techniques which I did not get around to showing you, techniques involving the so-called method of undetermined coefficients.
Although, some of you peaked in your book and learned it from there.
But the work is in finding (x)p. The miracle that occurs here, by contrast, is that it turns out to be easy to find (x)p. And easy in this further sense that I do not have to restrict the kind of function I use.
For example, the second homework problem I have given you, the second part two homework problem.
You will see how to use systems.
For example, to solve this simple equation, I will write it out for you, consider that equation, tangent t. What technique will you apply to solve that?
In other words, suppose you wanted to find a particular solution to that.
The right-hand side is not an exponential.
It is not a polynomial.
It is not like sine or cosine of bt.
I could use the Laplace transform.
No, because you don't know how to take the Laplace transform of tangent t. Neither, for that matter, do I. Fourier series.
Not a good choice for a function that goes to infinity at pi over two.
So you cannot do this until you do your homework.
Now you will be able to do it.
In other words, one of the big things is not only will I give you a formula for the Xp but that formula will work even for tangent t, any function at all.
Well, I thought I would try to put a little meat on the bones of the inhomogeneous systems by actually giving you a physical problem so we would actually be able to solve a physical problem instead of just demonstrate a solution method.
Here is a mixing problem.
Just to illustrate what makes a system of equations inhomogeneous, here at two ugly tanks.
I am not going to draw these carefully, but they are both 1 liter.
And they are connected by pipes.
And I won't bother opening holes in them.
There is a pipe with fluids flowing back there and this direction it is flowing this way, but that is not the end.
The end is there is stuff coming in to both of them.
And I think I will just make it coming out of this one.
There is something realistic.
The numbers 2, 3, 2.
Let's start there and see what the others have to be.
So these are flow rates.
One liter tanks.
The flow rates are in, let's say, liters per hour.
And I have some dissolved substance in, so here is going to be x salt in there and the same chemical in there, whatever it is.
x is the amount of salt, let's say, in tank one.
And y, the same thing in tank two.
Now, if you have stuff flowing unequally this way, you must have balance.
You have to make sure that neither tank is getting emptied or bursting and exploding.
What is flowing in?
What is x?
Three is going out, two is coming in, so this has to be one in order that tank x stay full and not explode.
And how about y?
How much is going out?
Two there and two here.
Four is going out, three is coming in.
This also has to be one.
Those are just the flow rates of water or the liquid that is coming in.
Now, the only thing I am going to specify is the concentration of what is coming in.
Here the concentration is 5 e to the minus t. And that is what makes the problem inhomogeneous.
Here the concentration is going to be zero.
In other words, pure water is flowing in here to create the liquid balance.
Here, on the other hand, salt solution is flowing in but with a steadily declining concentration.
So, what is the system?
Well, you have set it up exactly the way you did when you studied first order equations.
It is inflow minus outflow.
What is the outflow?
The outflow is all in this pipe.
The flow rates are liters per hour.
Three liters per hour flowing out.
How much salt does that represent?
It is negative three times the concentration of salt.
But the concentration, notice, equals x divided by one.
In other words, x represents both the concentration and the amount.
So I don't have to distinguish.
If I had made it two liter tanks then I would have had to divide this by two.
I am cheating, but it is enough already.
x prime equals minus 3x.
That is what is going out.
What is coming in?
Well, 2y is coming in.
Concentration here.
What is coming in?
Is it y 2 liter?
Plus what is coming in from the outside.
We have to add that in, and that will be plus 5 e to the negative t. How about y?
y prime is changing.
What comes in from x?
That is 3x.
What goes out?
Well, two is leaving here and two is leaving here.
It doesn't matter that they are going out through separate pipes.
They are both going out.
It is minus 4, 2 and 2.
How about the inhomogeneous term?
There is one coming in, but there is no salt in it.
Therefore, that is not changing.
What is coming through that pipe is necessary for the liquid balance.
But it has no effect whatsoever.
I will put a zero here but, of course, you don't have to put that in.
This is now an inhomogeneous system.
In other words, the system is x prime equals this matrix, negative 3, the same sort of stuff we always had, plus the inhomogeneous term which is the column vector 5 e to the minus t and zero.
It is the presence of this term that makes this system inhomogeneous.
And what that corresponds to is this little closed system being attacked from the outside by these external pipes which are bringing salt in.
Without those, of course the balance would be all wrong.
I would have to change this to three and cut that out, I guess.
But then, it would be just a simple homogenous system.
It is these pipes that make it inhomogeneous.
Now, I should start to solve that.
I did this just to illustrate where a system might come from.
Before I solve that, what I want to do is, of course, is solve it in general.
In other words, how do you solve this in general?
Because I promised you that you would be able to do in general, regardless of what sort of functions were in the r of t, that column vector.
So let's do it.
First of all, you have to learn the name of the method.
This method is for solving x prime equals Ax.
It is a method for finding a particular solution.
Of course, to actually solve it then you have to add the complimentary function.
We are looking for a particular solution for this system.
Now, the whole cleverness of the method, which I think was discovered a couple hundred years ago by, I think, Lagrange, I am not sure.
The method is called variation of parameters.
I am giving you that so that when you forget you will be able to look it up and be indexes to some advanced engineering mathematics book or something, whatever is on your shelf.
But, if course, you won't remember the name either so maybe this won't work.
Variation of parameters, I will explain to you why it is called that.
All the cleverness is in the very first line.
If you could remember the very first line then I trust you to do the rest yourself.
I don't know any motivation for this first step, but mathematics is supposed to be mysterious anyway.
It keeps me eating.
It says, look for a solution and there will be one of the following form.
Now, it will look exactly like -- Look carefully because it is going to be gone in a moment.
It will look exactly like this.
But, of course, it cannot be this because this solves the homogeneous system.
If I plug this in with these as constants it cannot possibly be a particular solution to this because it will stop there and satisfy that with r equals zero.
The whole trick is you think of these are parameters which are now variable.
Constants that are varying.
That is why it is called variation of parameters.
You think of these, in other words, as functions of t. We are going to look for a solution which has the form, since they are functions of t, I don't want to call them c1 and c2 anymore.
I will call them v because that is what most people call them, v or u, sometimes.
The method says look for a solution of that form.
The variation parameters, these are the parameters that are now varying instead of being constants.
Now, if you take it in that form and start trying to substitute into the equation you are going to get a mess.
I think I was wrong in saying I could trust you from this point on.
I will take the first step from you, and then I could trust you to do the rest after that first step.
The first step is to change the way this looks by using the fundamental matrix.
Remember what the fundamental matrix was?
Its entries were the two columns of solutions.
These are solutions to the homogeneous system.
And I am going to write it using the fundamental matrix as, now thinks about it.
The fundamental matrix has columns x1 and x2.
Your instinct might be using matrix multiplication to put the v1 and the v2 here, but that won't work.
You have to put them here.
This says the same thing as that.
Let's just take a second out to calculate.
The x is going to look like (x1, y1).
That is my first solution.
My second solution, here is the fundamental matrix, is (x2, y2).
And I am multiplying this on the right by (v1, v2).
Does it come out right?
Look.
What is it?
The top is x1 v1 plus x2 v2.
The top, x1 v1 plus x2 v2.
It is in the wrong order, but multiplication is commutative, fortunately.
And the same way the bottom thing will be v1 y1 plus v2 y2.
If I had written it on the other side instead, which is tempting because the v's occur on the left here, that won't work.
What will I get?
I will get v1 x1 plus v2 y1, which is not at all what I want.
You must put it on the right.
But this is a very important thing.
This is going to plague us on Monday, too.
It must be written on the right and not on the left as a column vector.
The rest of the program is very simple.
I will write it out as a program.
Substitute into the system, into that, in other words, and see what v has to be.
That is what we are looking for.
We know what the x1 and x2 are.
It is a question of what those coefficients are.
And see what v is.
Let's do it.
Let's substitute.
Let's see.
The system is x prime equals Ax plus r. I want to put in (x)p, this proposed particular solution.
And it is a fundamental matrix, and the v is unknown.
How do I differentiate the product of two matrices?
You differentiate the product of two matrices using the product rule that you learned the first day of 18.01.
Trust me.
Let's do it.
I am going to substitute in.
In other words, here is my (x)p, (x)p, and I am going to write in what that is.
The left-hand side is the derivative of, X prime times v, plus X times the derivative of v. Notice that one of these is a column vector and the other is a square matrix.
That is perfectly Okay.
Any two matrices which are the rate shape so you can multiply them together, if you want to differentiate their product, in other words, if the entries are functions of t it is the product rule.
The derivative of this times time plus that times the derivative of this.
You have to keep them in the right order.
You are not allowed to shuffle them around carelessly.
So that is that.
What is it equal to?
Well, the right-hand side is A.
And now I substitute just (x)p in, so that is X times v plus r. Is this progress?
What is v?
It looks like a mess but it is not.
Why not?
It is because this is not any old matrix X.
This is a matrix whose columns are solutions to the system.
And what does that do?
That means X prime satisfies that matrix differential equation.
X prime is the same as Ax.
And, by a little miracle, the v is tagging along in both cases.
This cancels that and now there is very little left.
The conclusion, therefore, is that Xv is equal to r. What is v?
It is v that we are looking for, right?
You have to solve a matrix equation, now.
This is a square matrix so you have to do it by inverting the matrix.
You don't just sloppily divide.
You multiply on which side by what matrix?
Choice of left or right.
You multiply by the inverse matrix on the left or on the right?
It has to be on the left.
Multiply both sides of the equation by X inverse on the left, and then you will get v is equal to X inverse r. How do I know the X inverse exists?
Does X inverse exist?
For a matrix inverse to exist, the matrix's determinant must be not zero.
Why is the determinant of this not zero?
Because its columns are independent solutions.
Of course this is not right.
I forgot the prime here.
I am not failing this course after all.
v prime equals that.
This is done by differentiating each entry in the column vector.
And, therefore, we should integrate it.
It will be the integral, just the ordinary anti-derivative of x inverse times r. This is a column vector.
The entries are functions of t. You simply integrate each of those functions in turn.
So integrate each entry.
There is my v. Sorry, you cannot tell the v's from the r's here.
And so, finally, the particular solution is (x)p is equal to -- It is really not bad at all.
It is equal to X times v. It's equal to X times the integral of X inverse r dt.
Now, actually, there is not much work to doing that.
Once you have solved the homogeneous system and gotten the fundamental matrix, taking the inverse of a two-by-two matrix is almost trivial.
You flip those two and you change the signs of these two and you divide by the determinant.
You multiply it by r. And the hard part is if you can do the integration.
If not, you just leave the integral sign the way you have learned to do in this silly course and you still have the answer.
What about the arbitrary constant of integration?
The answer is you don't need to put it in.
Just find one particular solution.
It is good enough.
You don't have to put in the arbitrary constants of integration.
Because they are already in the complimentary function here.
Therefore, you don't have to add them.
I am sorry I didn't get a chance to actually solve that.
I will have to let it go.
The recitations will do it on Tuesday, will solve that particular problem, which means you will, in effect.
We are going to need a few facts about fundamental matrices, and I am worried that over the weekend this spring activities weekend you might have forgotten them.
So I will just spend two or three minutes reviewing the most essential things that we are going to need later in the period.
What we are talking about is, I will try to color code things so you will know what they are.
First of all, the basic problem is to solve a system of equations.
And I am going to make that a two-by-two system, although practically everything I say today will also work for end-by-end systems.
Your book tries to do it end-by-end, as usual, but I think it is easier to learn two-by-two first and generalize rather than to wade through the complications of end-by-end systems.
So the problem is to solve it.
And the method I used last time was to describe something called a fundamental matrix.
A fundamental matrix for the system or for A, whichever you want, remember what that was.
That was a two-by-two matrix of functions of t and whose columns were two independent solutions, x1, x2.
These were two independent solutions.
In other words, neither was a constant multiple of the other.
Now, I spent a fair amount of time showing you the two essential properties that a fundamental matrix had.
We are going to need those today, so let me remind you the basic properties of X and the properties by which you could recognize one if you were given one.
First of all, the easy one, its determinant shall not be zero, is not zero for any t, for any value of the variable.
That simply expresses the fact that its two columns are independent, linearly independent, not a multiple of each other.
The other one was more bizarre, so I tried to call a little more attention to it.
It was that the matrix satisfies a differential equation of its own, which looks almost the same, except it's a matrix differential equation.
It is not our column vectors which are solutions but matrices as a whole which are solutions.
In other words, if you take that matrix and differentiate every entry, what you get is the same as A multiplied by that matrix you started with.
This, remember, expressed the fact, it was just really formal when you analyzed what it was, but it expressed the fact that it says that the columns solved the system.
The first thing says the columns are independent and the second says each column separately is a solution to the system.
That is as far, more or less.
Then I went in another direction and we talked about variation of parameters.
I am not going to come back to variation of parameters today.
We are going in a different tack.
And the tack we are going on is I want to first talk a little more about the fundamental matrix and then, as I said, we will talk about an entirely different method of solving the system, one which makes no mention of eigenvalues or eigenvectors, if you can believe that.
But, first, the one confusing thing about the fundamental matrix is that it is not unique.
I have carefully tried to avoid talking about the fundamental matrix because there is no "the" fundamental matrix, there is only "a" fundamental matrix.
Why is that?
Well, because these two columns can be any two independent solutions.
And there are an infinity of ways of picking independent solutions.
That means there is an infinity of possible fundamental matrices.
Well, that is disgusting, but can we repair it a little bit?
I mean maybe they are all derivable from each other in some simple way.
And that is, of course, what is true.
Now, as a prelude to doing that, I would like to show you what I wanted to show you on Friday but, again, I ran out of time, how to write the general solution -- -- to the system.
The system I am talking about is that pink system.
Well, of course, the standard naďve way of doing it is it's x equals, the general solution is an arbitrary constant times that first solution you found, plus c2, times another arbitrary constant, times the second solution you found.
Okay.
Now, how would you abbreviate that using the fundamental matrix?
Well, I did something very similar to this on Friday, except these were called Vs.
It was part of the variation parameters method, but I promised not to use those words today so I just said nothing.
Okay.
What is the answer?
It is x equals, how do I write this using the fundamental matrix, x1, x2?
Simple.
It is capital X times the column vector whose entries are c1 and c2.
In other words, it is x1, x2 times the column vector c1, c2, isn't it?
Yeah.
Because if you multiply this think top row, top row, top row c1, plus top row times c2, that exactly gives you the top row here.
And the same way the bottom row here, times this vector, gives you the bottom row of that.
It is just another way of writing that, but it looks very efficient.
Sometimes efficiency isn't a good thing, you have to watch out for it, but here it is good.
So, this is the general solution written out using a fundamental matrix.
And you cannot use less symbols than that.
There is just no way.
But that gives us our answer to, what do all fundamental matrices look like?
Well, they are two columns are solutions.
The answer is they look like -- Now, the first column is an arbitrary solution.
How do I write an arbitrary solution?
There is the general solution.
I make it a particular one by giving a particular value to that column vector of arbitrary constants like two, three or minus one, pi or something like that.
The first guy is a solution, and I have just shown you I can write such a solution like X, c1 with a column vector, a particular column vector of numbers.
This is a solution because the green thing says it is.
And side by side, we will write another one.
And now all I have to do is, of course, there is supposed to be a dependent.
We will worry about that in just a moment.
All I have to do is make this look better.
Now, I told you last time, by the laws of matrix multiplication, if the first column is X c1 and the second column is X c2, using matrix multiplication that is the same as writing it this way.
This square matrix times the matrix whose entries are the first column vector and the second column vector.
Now, I am going to call this C. It is a square matrix of constants.
It is a two-by-two matrix of constants.
And so, the final way of writing it is just what corresponds to that, X times C. And so X is a given fundamental matrix, this one, that one, so the most general fundamental matrix is then the one you started with, and multiply it by an arbitrary square matrix of constants, except you want to be sure that the determinant is not zero.
Well, the determinant of this guy won't be zero, so all you have to do is make sure that the determinant of C isn't zero either.
In other words, the fundamental matrix is not unique, but once you found one all the other ones are found by multiplying it on the right by an arbitrary square matrix of constants, which is nonsingular, it has determinant nonzero in other words.
Well, that was all Friday.
That's Friday leaking over into Monday.
And now we begin the true Monday.
Here is the problem.
Once again we have our two-by-two system, or end-by-end if you want to be super general.
There is a system.
What do we have so far by way of solving it?
Well, if your kid brother or sister when you go home said, a precocious kid, okay, tell me how to solve this thing, I think the only thing you will be able to say is well, you do this, you take the matrix and then you calculate something called eigenvalues and eigenvectors.
Do you know what those are?
I didn't think you did, blah, blah, blah, show how smart I am.
And you then explain what the eigenvalues and eigenvectors are.
And then you show how out of those make up special solutions.
And then you take a combination of that.
In other words, it is algorithm.
It is something you do, a process, a method.
And when it is all done, you have the general solution.
Now, that is fine for calculating particular problems with a definite model with definite numbers in it where you want a definite answer.
And, of course, a lot of your work in engineering and science classes is that kind of work.
But the further you get on, well, when you start reading books, for example, or god forbid start reading papers in which people are telling you, you know, they are doing engineering or they are doing science, they don't want a method, what they want is a formula.
In other words, the problem is to fill in the blank in the following.
You are writing a paper, and you just set up some elaborate model and A is a matrix derived from that model in some way, represents bacteria doing something or bank accounts doing something, I don't know.
And you say, as is well-known, the solution is, of course, you only have letters here, no numbers.
This is a general paper.
The solution is given by the formula.
The only trouble is, we don't have a formula.
All we have is a method.
Now, people don't like that.
What I am going to produce for you this period is a formula, and that formula does not require the calculation of any eigenvalues, eigenvectors, doesn't require any of that.
It is, therefore, a very popular way to fill in to finish that sentence.
Now the question is where is that formula going to come from?
Well, we are, for the moment, clueless.
If you are clueless the place to look always is do I know anything about this sort of thing?
I mean is there some special case of this problem I can solve or that I have solved in the past?
And the answer to that is yes.
You haven't solved it for a two-by-two matrix but you have solved it for a one-by-one matrix.
A one-by-one matrix also goes by the name of a constant.
It is just a thing.
It's a number.
Just putting brackets around it doesn't conceal the fact that it is just a number.
Let's look at what the solution is for a one-by-one matrix, a one-by-one case.
If we are looking for a general solution for the end-by-end case, it must work for the one-by-one case also.
That is a good reason for us starting.
That looks like x, doesn't it?
A one-by-one case.
Well, in that case, I am trying to solve the system.
The system consists of a single equation.
That is the way the system looks.
How do you solve that?
Well, you were born knowing how to solve that.
Anyway, you certainly didn't learn it in this course.
You separate variables, blah, blah, blah, and the solution is x equals, the basic solution is e to the at, and you multiply that by an arbitrary constant.
Now, that is a formula for the solution.
And it uses the parameter in the equation.
I didn't have to know a special number.
I didn't have to put a particular number here to use that.
Well, the answer is that the same idea, whatever the answer I give here has got to work in this case, too.
But let's take a quick look as to why this works.
Of course, you separate variables and use calculus.
I am going to give you a slightly different argument that has the advantage of generalizing to the end-by-end case.
And the argument goes as follows for that.
It uses the definition of the exponential function not as the inverse to the logarithm, which is where the fancy calculus books get it from, nor as the naďve high school method, e squared means you multiply e by itself and e cubed means you do it three times and so on.
And e to the one-half means you do it a half a time or something.
So, the naďve definition of the exponential function.
Instead, I will use the definition of the exponential function that comes from an infinite series.
Leaving out the arbitrary constant that we don't have to bother with.
e to the a t is the series one plus at plus a squared t squared over two factorial.
I will put out one more term and let's call it quits there.
If I take this then argument goes let's just differentiate it.
In other words, what is the derivative of e to the at with respect to t?
Well, just differentiating term by term it is zero plus the first term is a, the next term is a squared times t. This differentiates to t squared over two factorial.
And the answer is that this is equal to a times, if you factor out the a, what is left is one plus a t plus a squared t squared over two factorial In other words, it is simply e to the at.
In other words, by differentiating the series, using the series definition of the exponential and by differentiating it term by term, I can immediately see that is satisfies this differential equation.
What about the arbitrary constant?
Well, if you would like, you can include it here, but it is easier to observe that by linearity if e to the a t solves the equation so does the constant times it because the equation is linear.
Now, that is the idea that I am going to use to solve the system in general.
What are we doing to say?
Well, what could we say?
The solution to, well, let's get two solutions at once by writing a fundamental matrix.
"A" fundamental matrix, I don't claim it is "the" one, for the system x prime equals A x.
That is what we are trying to solve.
And we are going to get two solutions by getting a fundamental matrix for it.
The answer is e to the a t. Isn't that what it should be?
I had a little a.
Now we have a matrix.
Okay, just put the matrix up there.
Now, what on earth?
The first person who must have thought of this, it happened about 100 years ago, what meaning should be given to e to a matrix power?
Well, clearly the two na•ve definitions won't work.
The only possible meaning you could try for is using the infinite series, but that does work.
So this is a definition I am giving you, the exponential matrix.
Now, notice the A is a two-by-two matrix multiplying it by t. What I have up here is that it's basically a two-by-two matrix.
Its entries involve t, but it's a two-by-two matrix.
Okay.
We are trying to get the analog of that formula over there.
Well, leave the first term out just for a moment.
The next term is going to surely be A times t. This is a two-by-two matrix, right?
What should the next term be?
Well, A squared times t squared over two factorial.
What kind of a guy is that?
Well, if A is a two-by-two matrix so is A squared.
How about this?
This is just a scalar which multiplies every entry of A squared.
And, therefore, this is still a two-by-two matrix.
That is a two-by-two matrix.
This is a two-by-two matrix.
No matter how many times you multiply A by itself it stays a two-by-two matrix.
It gets more and more complicated looking but it is always a two-by-two matrix.
And now I am multiplying every entry of that by the scalar t cubed over three factorial.
I am continuing on in that way.
What I get, therefore, is a sum of two-by-two matrices.
Well, you can add two-by-two matrices to each other.
We've never made an infinite series of them, we haven't done it, but others have.
And this is what they wrote.
The only question is, what should we put in the beginning?
Over there I have the number one.
But I, of course, cannot add the number one to a two-by-two matrices.
What goes here must be a two-by-two matrix, which is the closest thing to one I can think of.
What should it be?
The I. Two-by-two I. Two-by-two identity matrix looks like the natural candidate for what to put there.
And, in fact, it is the right thing to put there.
Okay.
Now I have a conjecture, you know, purely formally, changing only with a keystroke of the computer, all the little a's have been changed to capital A's.
And now all I have to do is wonder if this is going to work.
Well, what is the basic thing I have to check to see that it is the fundamental matrix?
The question is, I wrote it down all right, but is this a fundamental matrix for the system?
Well, I have a way of recognizing a fundamental matrix when I see one.
The critical thing is that it should satisfy this matrix differential equation.
That is what I should verify.
Does this guy that I have written down satisfy this equation?
And the answer is, number two is, it satisfies x prime equals Ax.
In other words, plugging in x equals this e to the at, whose definition I just gave you.
If I substitute that in, does it satisfy that matrix differential equation?
The answer is yes.
I am not going to calculate it out because the calculation is actually identical to what I did there.
The only difference is when I differentiated it term by term, how do you differentiate something like this?
Well, you differentiate every term in it.
But, if you work it out, this is a constant matrix, every term of which is multiplied by t squared over two factorial.
Well, if you differentiate every entry of that constant, of that matrix, what you are going to get is A squared times just the derivative of that part, which is simply t. In other words, the formal calculation looks absolutely identical to that.
So the answer to this is yes, by the same calculation as before, as for the one-by-one case.
And now the only other thing to check is that the determinant is not zero.
In fact, the determinant is not zero at one point.
That is all you have to check.
What is x of zero?
What is the value of the determinant of x is e to the At?
What is the value of this thing at zero?
Here is my function.
If I plug in t equals zero, what is it equal to?
I.
What is the determinant of I?
One.
It is certainly not zero.
By writing down this infinite series, I have my two solutions.
Its columns give me two solutions to the original system.
There were no eigenvalues, no eigenvectors.
I have a formula for the answer.
What is the formula?
It is e to the At.
And, of course, anybody reading the paper is supposed to know what e to the At is.
It means that.
This is just marvelous.
There must be a fly in the ointment somewhere.
Only a teeny little fly.
There is a teeny little fly because it is almost impossible to calculate that series for all reasonable times.
However, once in a while it is.
Let me give you an example where it is possible to calculate the series and were you get a nice answer.
Let's work out an example.
By the way, you know, nowadays, we are not back 50 years, the exponential matrix has the same status on, say, a Matlab or Maple or Mathematica, as the ordinary exponential function does.
It is just a command you type in.
You type in your matrix.
And you now say EXP of that matrix and out comes the answer to as many decimal places as you want.
It will be square matrix with entries carefully written out.
So, in that sense, the fact that we cannot calculate it shouldn't bother us.
There are machines to do the calculations.
What we are interested in is it as a theoretical tool.
But, in order to get any feeling for this at all, we certainly have to do a few calculations.
Let's do an easy one.
Let's consider the system x prime equals y, y prime equals x.
This is very easily done by elimination, but that is forbidden.
First of all, we write it as a matrix.
It's the system x prime equals zero, one, one, zero, x.
Here is my A.
And so e to the At is going to be -- A is zero, one, one, zero.
What we want to calculate is we are going to get both solutions at once by calculating it one fell swoop e to the At.
Okay.
E to the At equals.
I am going to actually write out these guys.
Well, obviously the hard part, the part which is normally going to prevent us from calculating this series explicitly, by hand anyway, because, as I said, the computer can always do it.
The value, how do we raise a matrix to a high power?
You just keep multiplying and multiplying and multiplying.
That looks like a rather forbidding and unpromising activity.
Well, here it is easy.
Let's see what happens.
If that is A, what is A squared?
I am going to have to calculate that as part of the series.
That is going to be zero, one, one, zero times zero, one, one, zero, which is one, zero, zero, one.
We got saved.
It is the identity.
Now, from this point on we don't have to do anymore calculations, but I will do them anyway.
What is A cubed?
Don't start from scratch again.
No, no, no.
A cubed is A squared times A.
And A squared is, in real life, the identity.
Of course, you would do all this in your head, but I am being a good boy and writing it all out.
This is I, the identity, times A, which is A. I will do one more.
What is A to the fourth?
Now, you would be tempted to say A to the fourth is A squared, which is I times I, which is I, but that would be wrong.
A to the fourth is A cubed times A, which is, I have just calculated is A times A, right?
And now that is A squared, which is the identity.
It is clear, by this argument, it is going to continue in the same way each time you add an A on the right-hand side, you are going to keep alternating between the identity, A, the next one will be identity, the next will be A.
The end result is that the first term of the series is simply the identity; the next term of the series is A, but it is multiplied by t. I will keep the t on the outside.
Remember, when you multiply a matrix by a scalar, that means multiply every entry by that scalar.
This is the matrix zero, t, t, zero.
I will do a couple more terms.
The next term would be, well, A squared we just calculated as the identity.
That looks like this.
Except now I multiply every term by t squared over two factorial.
All right.
I'll go for broke.
The next one will be this times t cubed over three factorial.
And, fortunately, I have run out of room.
Okay, let's calculate then.
What is the final answer for e to At?
I have an infinite series of two-by-two matrices.
Let's look at the term in the upper left-hand corner.
It is one plus zero times t plus one times t squared over two factorial plus zero times t. It is going to be, in other words, one plus t squared over two factorial plus the next term, which is not on the board but I think you can see, is this.
And it continues on in the same way.
How about the lower left term?
Well, that is zero plus t plus zero plus t cubed over three factorial and so on.
It is t plus t cubed over three factorial plus t to the fifth over five factorial.
And the other terms in the other two corners are just the same as these.
This one, for example, is zero plus t plus zero plus t cubed over three factorial.
And the lower one is one plus zero plus t squared and so on.
This is the same as one plus t squared over two factorial and so on, and up here we have t plus t cubed over three factorial and so on.
Well, that matrix doesn't look very square, but it is.
It is infinitely long physically, but it has one term here, one term here, one term here and one term there.
Now, all we have to do is make those terms look a little better.
For here I have to rely on the culture, which you may or may not posses.
You would know what these series were if only they alternated their signs.
If this were a negative, negative, negative then the top would be cosine t and this would be sine t, but they don't.
So they are the next best thing.
They are what?
Hyperbolic.
The topic is not cosine t, but cosh t. The bottle is sinh t. And how do we know this?
Because you remember.
And what if I don't remember?
Well, you know now.
That is why you come to class.
Well, for those of you who don't, remember, this is e to the t plus e to the negative t. It should be over two, but I don't have room to put in the two.
This doesn't mean I will omit it.
It just means I will put it in at the end by multiplying every entry of this matrix by one-half.
If you have forgotten what cosh t is, it's e to the t plus e to the negative t divided by two.
And the similar thing for sinh t. There is your first explicit exponential matrix calculated according to the definition.
And what we have found is the solution to the system x prime equals y, y prime equals x.
A fundamental matrix.
In other words, cosh t and sinh t satisfy both solutions to that system.
Now, there is one thing people love the exponential matrix in particular for, and that is the ease with which it solves the initial value problem.
It is exactly what happens when studying the single system, the single equation x prime equals Ax, but let's do it in general.
Let's do it in general.
What is the initial value problem?
Well, the initial value problem is we start with our old system, but now I want to plug in initial conditions.
I want the particular solution which satisfies the initial condition.
Let's make it zero to avoid complications, to avoid a lot of notation.
This is to be some starting value.
This is a certain constant vector.
It is to be the value of the solution at zero.
And the problem is find what x of t is.
Well, if you are using the exponential matrix it is a joke.
It is a joke.
Shall I derive it or just do it?
All right.
The general solution, let's derive it, and then I will put up the final formula in a box so that you will know it is important.
What is the general solution?
Well, I did that for you at the beginning of the period.
Once you have a fundamental matrix, you get the general solution by multiplying it on the right by an arbitrary constant vector.
The general solution is going to be x equals e to the At.
That is my super fundamental matrix, found without eigenvalues and eigenvectors.
And this should be multiplied by some unknown constant vector c. The only question is, what should the constant vector c be?
To find c, I will plug in zero.
When t is zero, here I get x of zero, here I get e to the A times zero times c. Now what is this?
This is the vector of initial conditions?
What is e to the A times zero?
Plug in t equals zero.
What do you get?
I.
Therefore, c is what?
c is x zero.
It is a total joke.
And the solution is, the initial value problem is x equals e to the At times x zero.
It is just what it would have been at one variable.
The only difference is that here we are allowed to put the c out front.
In other words, if I asked you to put in the initial condition, you would probably write x equals little x zero times e to the At.
And you would be tempted to do the same thing here, vector x equals vector x zero times e to the At.
Now, you cannot do that.
And, if you try to Matlab will hiccup and say illegal operation.
What is the illegal operation?
Well, x is a column vector.
From the system it is a column vector.
That means the initial conditions are also a column vector.
You cannot multiply a column vector out front and a square matrix afterwards.
You cannot.
If you want to multiply a matrix by a column vector, it has to come afterwards so you can do zing, zing.
There is no zing, you see.
You cannot put it in front.
It doesn't work.
So it must go behind.
That is the only place you might get tripped up.
And, as I say, if you try to type that in using Matlab, you will immediately get error messages that it is illegal, you cannot do that.
Anyway, we have our solution.
There is our system.
Our initial value problem anyway is in pink, and its solution using the exponential matrix is in green.
Now, the only problem is we still have to talk a little bit more about calculating this.
Now, the principle warning with an exponential matrix is that once you have gotten by the simplest things involving the fact that it solves systems, it gives you the fundamental matrix for a system, then you start flexing your muscles and say, oh, well, let's see what else we can do with this.
For example, the reason exponentials came into being in the first place was because of the exponential law, right?
I will kill anybody who sends me emails saying, what is the exponential law?
The exponential law would say that e to the A plus B is equal to e to the A times e to the B.
The law of exponents, in other words.
It is the thing that makes the exponential function different from all other functions that it satisfies something like that.
Now, first of all, does this make sense?
That is are the symbols compatible?
Let's see.
This is a two-by-two matrix, this is a two-by-two matrix, so it does make sense to multiply them, and the answer will be a two-by-two matrix.
How about here?
This is a two-by-two matrix, add this to it.
It is still a two-by-two matrix.
e to a two-by-two matrix still comes out to be a two-by-two matrix.
Both sides are legitimate two-by-two matrices.
The only question is, are they equal?
And the answer is not in a pig's eye.
How could this be?
Well, I didn't make up these laws.
I just obey them.
I wish I had time to do a little calculation to show that it is not true.
It is true in certain special cases.
It is true in the special case, and this is pretty much if and only if, the only case in which it is true is if A and B are not arbitrary square matrices but commute with each other.
You see, if you start writing out the series to try to check whether that law is true, you will get a bunch of terms here, a bunch of terms here.
And you will find that those terms are pair-wise equal only if you are allowed to let the matrices commute with each other.
In other words, if you can turn AB plus BA into twice AB then everything will work fine.
But if you cannot do that it will not.
Now, when do two square matrices commute with each other?
The answer is almost never.
It is just a lucky accident if they do, but there are three cases of the lucky accident which you should know.
The three cases, I feel justified calling it "the" three cases.
Oh, well, maybe I shouldn't do that.
The three most significant examples are, example number one, when A is a constant times the identity matrix.
In other words, when A is a matrix that looks like this.
That matrix commutes with every other square matrix.
If that is A, then this law is always true and you are allowed to use this.
Okay, so that is one case.
Another case, when A is more general, is when B is equal to negative A. I think you can see that that is going to work because A times minus A is equal to minus A times A. Yeah, they are both equal to A squared, except with a negative sign in front.
And the third case is when B is equal to the inverse of A because A A inverse is the same as A inverse A.
They are both the identity.
Of course, A must have an inverse.
Okay, let's suppose it does.
Now, of them this is, I think, the most important one because it leads to this law.
That is forbidden, but there is one case of it which is not forbidden and that is here.
What will it say?
Well, that will say that e to the A minus A is equal to e to the A times e to the negative A.
This is true, even though the general law is false.
That is because A and negative A commute with each other.
But now what does this say?
What is e to the zero matrix?
In other words, suppose I take the matrix that is zero and plug it into the formula for e?
What do you get?
e to the zero times t is I.
It has to be a two-by-two matrix if it is going to be anything.
It is the matrix I.
This side is I.
This side is the exponential matrix.
And what does that show?
It shows that the inverse matrix, the e to the A, is e to the negative A.
That is a very useful fact.
This is the main survivor of the exponential law.
In general it is false, but this standard corollary to the exponential law is true, is equal to e to the minus A, just what you would dream and hope would be true.
Okay.
I have exactly two and a half minutes left in which to do the impossible.
All right.
The question is, how do you calculate e to the At?
You could use series, but it rarely works.
It is too hard.
There are a few examples, and you will have some more for homework, but in general it is too hard because it is too hard to calculate the powers of a general matrix A.
There is another method, which is useful only for matrices which are symmetric, but like that -- Well, it is more than symmetric.
These two have to be the same.
But you can handle those, as you will see from the homework problems, by breaking it up this way and using the exponential law.
This would be zero, b, b, zero.
See, these two matrices commute with each other and, therefore, I could use the exponential law.
This leaves all other cases.
And here is the way to handle all other cases.
All other cases.
In other words, if you cannot calculate the series, this trick doesn't work, I have done as follows.
You start with an arbitrary fundamental matrix, not the exponential matrix.
You multiply it by its value at zero, that is a constant matrix, and you take the inverse of that constant matrix.
It will have one because, remember, the fundamental matrix never has the determinant zero.
So you can always take its inverse-ready value of t. Now, what property does this have?
It is a fundamental matrix.
How do I know that?
Well, because I found all fundamental matrices for you.
Take any one, multiply it by a square matrix on the right-hand side, and you get still a fundamental matrix.
And what is its value at zero?
Well, it is x of zero times x of zero inverse.
Its value at zero is the identity.
Now, e to the At has these same two properties.
Namely, it is a fundamental matrix and its value at zero is the identity.
Conclusion, this is e to the At.
And that is the garden variety method of calculating the exponential matrix, if you want to give it explicitly.
Start with any fundamental matrix calculated, you should forgive the expression using eigenvalues and eigenvectors and putting the solutions into the columns.
Evaluate it at zero, take its inverse and multiply the two.
And what you end up with has to be the same as the thing calculated with that infinite series.
Okay.
You will get lots of practice for homework and tomorrow.
Everything I say today is going to be for n-by-n systems, but for your calculations and the exams two-by-two will be good enough.
Our system looks like that.
Notice I am talking today about the homogeneous system, not the inhomogenous system.
So, homogenous.
And we have so far two basic methods of solving it.
The first one, on which we spent the most time, is the method of where you calculate the eigenvalues of the matrix, the eigenvectors, and put them together to make the general solution.
So eigenvalues, e-vectors and so on.
The second method, which I gave you last time, I called "royal road," simply calculates the matrix e to the At and says that the solution is e to the At times x zero, the initial condition.
That is very elegant.
The only problem is that to calculate the matrix e to the At, although sometimes you can do it by its definition as an infinite series, most of the time the only way to calculate the matrix e to the At is by using the fundamental matrix.
In other words, the normal way of doing it is you have to calculate it as the fundamental matrix time normalized at zero.
So, as I explained at the end of last time and you practiced in the recitations, you have to find the fundamental matrix, which, of course, you have to do by eigenvalues and eigenvectors.
And then you multiply it by its value at zero, inverse.
And that, by magic, turns out to be the same as the exponential matrix.
But, of course, there has been no gain in simplicity or no gain in ease of calculation.
The only difference is that the language has been changed.
Now, today is going to be devoted to yet another method which saves no work at all and only amounts to a change of language.
The only reason I give it to you is because I have been begged by various engineering departments to do so -- -- because that is the language they use.
In other words, each person who solves systems, some like to use fundamental matrices, some just calculate, some immediately convert the system by elimination into a single higher order equation because they are more comfortable with that.
Some, especially if they are writing papers, they talk exponential matrices.
But there are a certain number of engineers and scientists who talk decoupling, express the problem and the answer in terms of decoupling.
And that is, therefore, what I have to explain to you today.
So, the third method, today's method, I stress is really no more than a change of language.
And I feel a little guilty about the whole business.
Instead of going more deeply into studying these equations, what I am doing is like giving a language course and teaching you how to say hello and good-bye in French, German, Spanish, and Italian.
It is not going very deeply into any of those languages, but you are going into the outside world, where people will speak these things.
Here is an introduction to the language of decoupling in which for some people is the exclusive language in which they talk about systems.
Now, I think the best way to, well, in a general way, what you try to do is as follows.
You try to introduce new variables.
You make a change of variables.
I am going to do it two-by-two just to save a lot of writing out.
And it's going to be a linear change of variables because we are interested in linear systems.
The problem is to find u and v such that something wonderful happens, such that when you make the change of variables to express this system in terms of u and v it becomes decoupled.
And that means the system turns into a system which looks like u prime equals k1 times u and v prime equals k2 times v. Such a system is called decoupled.
Why?
Well, a normal system is called coupled.
Let's write out what it would be.
Well, let's not write that.
You know what it looks like.
This is decoupled because it is not really a system at all.
It is just two first-order equations sitting side by side and having nothing whatever to do with each other.
This is two problems from the first day of the term.
It is not one problem from the next to last day of the term, in other words.
To solve this all you say is u is equal to some constant times e to the k1 t and v equals another constant times e to the k2 t. Coupled means that the x and y occur in both equations on the right-hand side.
And, therefore, you cannot solve separately for x and y, you must solve together for both of them.
Here I can solve separately for u and v and, therefore, the system has been decoupled.
Now, obviously, if you can do that it's an enormous advantage, not just to the ease of solution, because you can write down the solution immediately, but because something physical must be going on there.
There must be some insight.
There ought to be some physical reason for these new variables.
Now, that is where I plan to start with.
My plan for the lecture is first to work out, in some detail, a specific example where decoupling is done to show how that leads to the solution.
And then we will go back and see how to do it in general -- -- because you will see, as I do the decoupling in this particular example, that that particular method, though it is suggested, will not work in general.
I would need a more general method.
But let's first go to the example.
It is a slight modification of one you should have done in recitation.
I don't think I worked one of these in the lecture, but to describe it I have to draw two views of it to make sure you know exactly what I am talking about.
Sometimes it is called the two compartment ice cube tray problem, a very old-fashion type of ice cube tray.
Not a modern one that is all plastic where there is no leaking from one compartment to another.
The old kind of ice cube trays, there were compartments and these were metal separated and you leveled the liquid because it could leak through the bottom that didn't go right to the bottom.
If you don't know what I am talking about it makes no difference.
This is the side view.
This is meant to be twice as long.
But, to make it quite clear, I will draw the top view of this thing.
You have to imagine this is a rectangle, all the sides are parallel and everything.
This is one and this is two.
All I am trying to say is that the cross-sectional area of these two chambers, this one has twice the cross-sectional area of this one.
So I will write a two here and I will write a one there.
Of course, it is this hole here through which everything leaks.
I am going to let x be the height of this liquid, the water here, and y the height of the water in that chamber.
Obviously, as time goes by, they both reach the same height because of somebody's law.
Now, what is the system of differential equations that controls this?
Well, the essential thing is the flow rate through here.
That flow rate through the hole in units, let's say, in liters per second.
Just so you understand, I am talking about the volume of liquid.
I am not talking about the velocity.
That is proportional to the area of the hole.
So the cross-sectional area of the hole.
And it is also times the velocity of the flow, but the velocity of the flow depends upon the pressure difference.
And that pressure difference depends upon the difference in height.
All those are various people's laws.
So times the height difference.
Of course, you have to get the sign right.
I have just pointed out the height difference is proportional to the pressure at the hole.
And it is that pressure at the hole that determines the velocity with which the fluid flows through.
Where does this all produce our equations?
Well, x prime is equal to, therefore, some constant, depending on the area of the hole and this constant of proportionality with the pressure and the units and everything else times the pressure difference I am talking about.
Well, if fluid is going to flow in this direction that must mean the y height is higher than the x height.
So, to make x prime positive, it should be y minus x here.
Now, the y prime is different.
Because, again, the rate of fluid flow is determined.
This time, if y prime is positive, if this is rising, as it will be in this case, it's because the fluid is flowing in that direction.
It is because x is higher than y.
So this should be the same constant x minus y.
But notice that right-hand side is the rate at which fluid is flowing into this tank.
That is not the rate at which y is changing.
It is the rate at which 2y is changing.
Why isn't there a constant here?
There is.
It's one.
That is the one, this one cross-section.
The area here is one and the cross-sectional area here is two.
And that is the reason for the one here and the two here, because we are interested in the rate at which fluid is being added to this, which is only related to the height, the rate at which the height is rising if you take into account the cross-sectional area.
So there is the system.
In order to use nothing but integers here, I am going to take c equals to two, so I don't have to put in halves.
The final system is x prime equals minus 2x, you have to write them in the correct order, and y prime equals, the twos cancel because c is two, is x minus y.
So there is our system.
Now the problem is I want to solve it by decoupling it.
I want, in other words, to find new variables, u and v, which are more natural to the problem than the x and y that are so natural to the problem that the new system will just consistently be two side-by-side equations instead of the single equation.
The question is, what should u and v be?
Now, the difference between what I am going to do now and what I am going to do later in the period is later in the period I will give you a systematic way of finding what u and v should be.
Now we are going to psyche out what they should be in the way in which people who solve systems often do.
I am going to use the fact that this is not just an abstract system of equations.
It comes from some physical problem.
And I ask, is there some system of variables, which somehow go more deeply into the structure of what's going on here than the naīve variables, which simply tell me how high the two tank levels are?
That is the obvious thing I can see, but there are some variables that go more deeply.
Now, one of them is sort of obvious and suggested both the form of the equation and by this.
Simply, the difference in heights is, in some ways, a more natural variable because that is directly related to the pressure difference, which is directly related to the velocity of flow.
They will differ by just constant.
I am going to call that the second variable, or the difference in height let's call it.
That's x minus y.
That is a very natural variable for the problem.
The question is, what should the other one be?
Now you sort of stare at that for a while until it occurs to you that something is constant.
What is constant in this problem?
Well, the tank is sitting there, that is constant.
But what thing, which might be a variable, clearly must be a constant?
It will be the total amount of water in the two tanks.
These things vary, but the total amount of water stays the same because it is a homogenous problem.
No water is coming in from the outside, and none is leaving the tanks through a little hole.
Okay.
What is the expression for the total amount of water in the tanks?
x plus 2y.
Therefore, that is a natural variable also.
It is independent of this one.
It is not a simple multiply of it.
It is a really different variable.
This variable represents the total amount of liquid in the two tanks.
This represents the pressure up to a constant factor.
It is proportional to the pressure at the hole.
Okay.
Now what I am going to do is say this is my change of variable.
Now let's plug in and see what happens to the system when I plug in these two variables.
And how do I do that?
Well, I want to substitute and get the new system.
The new system, or rather the old system, but what makes it new is in terms of u and v. What will that be?
Well, u prime is x prime plus 2y prime.
But I know what x prime plus 2y prime is because I can calculate it for this.
What will it be?
x prime plus 2y prime is negative 2x plus twice y prime, so it's plus 2x, which is zero.
And how about these two?
2y minus twice this, because I want this plus twice that, so it 2y minus 2y, again, zero.
The right-hand side becomes zero after I calculate x prime plus 2y.
So that is zero.
That would just, of course, clear.
Now, that makes sense, of course.
Since the total amount is constant, that says that u prime is zero.
Okay.
What is v prime?
v prime is x prime minus y prime.
What is that?
Well, once again we have to calculate.
x prime minus y prime is minus 2x minus x, which is minus 3x, and 2y minus negative y, which makes plus 3y.
All right.
What is the system?
The system is u prime equals zero and v prime equals minus three times x minus y.
But x minus y is v. In other words, these new two variables decouple the system.
And we got them, as scientists often do, by physical considerations.
These variables go more deeply into what is going on in that system of two tanks than simply the two heights, which are too obvious as variables.
All right.
What is the solution?
Well, the solution is, u equals a constant and v is equal to?
Well, the solution to this equation is a different arbitrary constant from that one.
These are side-by-side equations that have nothing whatever to do with each other, remember?
Times e to the minus 3t.
Now, there are two options.
Either one leaves the solution in terms of those new variables, saying they are more natural to the problem, but sometimes, of course, one wants the answer in terms of the old one.
But, if you do that, then you have to solve that.
In order to save a little time, since this is purely linear algebra, I am going to write -- Instead of taking two minutes to actually do the calculation in front of you, I will just write down what the answer is -- -- in terms of u and x and y.
In other words, this is a perfectly good way to leave the answer if you are allowed to do it.
But if somebody says they want the answer in terms of x and y, well, you have to give them what they are paying for.
In terms of x and y, you have first to solve those equations backwards for x and y in terms of u and v in which case you will get x equals one-third of u plus 2v.
Use the inverse matrix or just do elimination, whatever you usually like to do.
And the other one will be one-third of u minus v. And then, if you substitute in, you will see what you will get is one-third of c1.
Sorry.
u is c1.
c1 plus 2 c2 e to the negative 3t.
And this is one-third of c1 minus c2 e to the minus 3t.
And so, the final solution is, in terms of the way we usually write out the answer, x will be what?
Well, it will be one-third c1 times the eigenvector one, one plus one-third times c2 times the eigenvector two, negative one times e to the minus 3t.
That is the solution written out in terms of x and y either as a vector in the usual way or separately in terms of x and y.
But, notice, in order to do that you have to have these backwards equations.
In other words, I need the equations in that form.
I need the equations because they tell me what the new variables are.
But I also have to have the equations the other way in order to get the solution in terms of x and y, finally.
Okay.
That was all an example.
For the rest of the period, I would like to show you the general method of doing the same thing which does not depend upon being clever about the choice of the new variables.
And then, at the very end of the period, I will apply the general method to this problem to see whether we get the same answer or not.
What is the general method?
Our problem is the decouple.
Now, the first thing is you cannot always decouple.
To decouple the eigenvalues must all be real and non-defective.
In other words, if they are repeated they must be complete.
You must have enough independent eigenvectors.
So they must be real and complete.
If repeated, they must be complete.
They must not be defective.
As I told you at the time when we studied complete and incomplete, the most common case in which this occurs is when the matrix is symmetric.
If the matrix is real and symmetric then you can always decouple the system.
That is a very important theorem, particularly since many of the equilibrium problems normally lead to symmetric matrices and are solved by decoupling.
Okay.
So what are we looking for?
We are assuming this and we need it.
In general, otherwise, you cannot decouple if you have complex eigenvalues and you cannot decouple if you have defective eigenvalues.
Well, what are we looking for?
We are looking for new variables.
u, v equals a1, b1, a2, b2 times the x, y.
And this matrix is called D, the decoupling matrix and is what we are looking for.
How do I choose those new variables u and v when I don't have any physical considerations to guide me as I did before?
Now, the key is to look instead at what you are going to need.
Remember, we are changing variables.
And, as I told you from the first days of the term, when you change variables look at what you are going to need to substitute in to make the change of variables.
Don't just start writing equations.
What we are going to need to plug into that system and change it to the (u, v) coordinates is not u and v in terms of x and y.
What we need is x and y in terms of u and v to do the substitution.
What we need is the inverse of this.
So, in order to do the substitution, what we need is (x, y).
Oops.
Let's call them prime.
Let's call these a1, b1, a2, b2 because these are going to be much more important to the problem than the other ones.
Okay.
I am going to, I should call this matrix D inverse, that would be a sensible thing to call it.
Since this is the important matrix, this is the one we are going to need to do the substitution, I am going to give it another letter instead.
And the letter that comes after D is E. Now, E is an excellent choice because it is also the first letter of the word eigenvector.
And the point is the matrix E, which is going to work, is the matrix whose columns are the two eigenvectors.
The columns are the two eigenvectors.
Now, even if you didn't know anything that would be practically the only reasonable choice anybody could make.
What are we looking for?
To make a linear change of variables like this really means to pick new i and j vectors.
You know, from the first days of 18.02, what you want is a new coordinate system in the plane.
And the coordinate system in the plane is determined as soon as you tell what the new i is and what the new j is in the new system.
To establish a linear change of coordinates amounts to picking two new vectors that are going to play the role of i and j instead of the old i and the old j.
Okay, so pick two vectors which somehow are important to this matrix.
Well, there are only two, the eigenvectors.
What else could they possibly be?
Now, what is the relation?
I say with this, what happens is I say that alpha one corresponds, and alpha two, these are vectors in the xy-system.
Well, if I change the coordinates to u and v, in the uv-system they will correspond to the vectors one, zero.
In other words, the vector that we would normally call i in the u, v system.
And this one will correspond to the vector zero, one.
Now, if you don't believe that I will calculate it for you.
The calculation is trivial.
Look.
What have we got?
(x, y) equals a1, b1, a2, b2.
This is the column vector alpha one.
This is the column vector alpha two.
Now, here is u and v. Suppose I make u and v equal to one, zero, what happens to x and y?
Your matrix multiply.
One, zero.
So a1 plus zero, b1 plus zero.
It corresponds to the column vector (a1, b1).
And in the same way zero, one corresponds to (a2, b2).
Just by matrix multiplication.
And that shows that these correspond.
In the uv-system the two eigenvectors are now called i and j.
Well, that looks very promising, but the program now is to do the substitution to substitute into the system x prime equals Ax and see if it is decoupled in the uv-coordinates.
Now, I don't dare let you do this by yourself because you will run into trouble.
Nothing is going to happen.
You will just get a mess and will say I must be missing something.
And that is because you are missing something.
What you are missing, and this is a good occasion to tell you, is that, in general, three-quarters of the civilized world does not introduce eigenvalues and eigenvectors the way you learn them in 18.03.
They use a different definition that is identical.
I mean it is equivalent.
The concept is the same, but it looks a little different.
Our definition is what?
Well, what is an eigenvalue and eigenvector?
The basic thing is this equation.
This is a two-by-two matrix, right?
This is a column vector with two entries.
The product has to be a column vector with two entries, but both entries are supposed to be zero so I will write it this way.
This way first defines what an eigenvalue is.
It is something that makes the determinant zero.
And then it defines what an eigenvector is.
It is, then, a solution to the system that you can get because the determinant is zero.
Now, that is not what most people do.
What most people do is the following.
They write this equation differently by having something on both sides.
Using the distributive law, what goes on the left side is A alpha one.
What is that?
That is a column vector with two entries.
What goes on the right?
Well, lambda one times the identity times alpha one.
Now, the identity matrix times anything just reproduces what was there.
There is no difference between writing the identity times alpha one and just alpha one all by itself.
So that is what I am going to do.
This is the definition of eigenvalue and eigenvector that all the other people use.
Most linear algebra books use this definition, or most books use a different approach and say, here is an eigenvalue and an eigenvector.
And it requires them to define them in the opposite order.
First what alpha one is and then what lambda one is.
See, I don't have any determinant now.
So what is the definition?
And they like it because it has a certain geometric flavor that this one lacks entirely.
This is good for solving differential equations, which is why we are using it in 18.03, but this has a certain geometric content.
This way thinks of A as a linear transformation of the plane, a shearing of the plane.
You take the plane and do something to it.
Or, you squish it like that.
Or, you rotate it.
That's okay, too.
And the matrix defines a linear transformation to the plane, every vector goes to another vector.
The question it asks is, is there a vector which is taken by this linear transformation and just left alone or stretched, is kept in the same direction but stretched?
Or, maybe its direction is reversed and it is stretched or it shrunk.
But, in general, if there are real eigenvalues there will be such vectors that are just left in the same direction but just stretched or shrunk.
And what is the lambda?
The lambda then is the amount by which they are stretched or shrunk, the factor.
This way, first we have to find the vector, which is left essentially unchanged, and then the number here that goes with it is the stretching factor or the shrinking factor.
But the end result is the pair alpha one and lambda one, regardless of which order you find them, satisfied the same equation.
Now, a consequence of this definition we are going to need in the calculation that I am going to do in just a moment.
Let me calculate that out.
What I want to do is calculate the matrix A times E. I am going to need to calculate that.
Now, what is that?
Remember, E is the matrix whose columns are the eigenvectors.
That is the matrix alpha one, alpha two.
Now, what is this?
Well, in both Friday's lecture and Monday's lecture, I used the fact that if you do a multiplication like that it is the same thing as doing the multiplication A alpha one and putting it in the first column.
And then A alpha two is the column vector that goes in the second column.
But what is this?
This is lambda one alpha one.
And this is lambda two alpha two by this other definition of eigenvalue and eigenvector.
And what is this?
Can I write this in terms of matrices?
Yes indeed I can.
This is the matrix alpha one, alpha two times this matrix lambda one, lambda two, zero, zero.
Check it out.
Lambda one plus zero, lambda one times this thing plus zero, the first entry is exactly that.
And the same way the second column doing the same calculation is exactly this.
What is that?
That is e times this matrix lambda one, zero, zero, lambda two.
Okay.
We are almost finished now.
Now we can carry out our work.
We are going to do the substitution.
I start with a system.
Remember where we are.
I am starting with this system.
I am going to make the substitution x equal to this matrix E, whose columns are the eigenvectors.
I am in introducing, in other words, new variables u and v according to that thing.
u is the column vector, u and v. And x, as usual, is the column vector x and y.
So I am going to plug it in.
Okay.
Let's plug it in.
What do I get?
I take the derivative.
E is a constant matrix so that makes E times u prime, is equal to A times, x is E times u again.
Now, at this point, you would be stuck, except I calculated for you A times E is E times that funny diagonal matrix with the lambdas.
So this is E times that funny matrix of the lambdas, the eigenvalues, and still the u at the end of it.
So where are we?
E times u prime equals E times this thing.
Well, multiply both sides by E inverse and you can cancel them out.
And so the end result is that after you have made the substitution in terms of the new variables u, what you get is u prime equals lambda one, lambda two, zero, zero times u.
Let's write that out in terms of a system.
This is u prime is equal to, well, this is u, v here.
It is lambda one times u plus zero times v. And the other one is v prime equals zero times u plus lambda two times v. We are decoupled.
In just one sentence you would say -- In other words, if you were reading a book that sort of assumed you knew what was going on, all it would say is as usual.
That is to make you feel bad.
Or, as is well-known to make you feel even worse.
Or, the system is decoupled by choosing as the new basis for the system the eigenvectors of the matrix and in terms of the resulting new coordinates, the decoupled system will be the following where the constants are the eigenvalues.
And so the solution will be u equals c1 times e to the lambda1 t and v is equal to c2 times e to the lambda2 t. Of course, if you want it back in terms now of x and y, you will have to go back to here, to these equations and then plug in for u and v what they are.
And then you will get the answer in terms of x and y.
Okay.
We have just enough time to actually carry out this little program.
It takes a lot longer to derive than it does actually to do, so let's do it for this system that we were talking about before.
Decouple the system, x, y prime equals the matrixes negative two, two, one, negative one.
Okay.
What do I do?
Well, I first have to calculate the eigenvalues in the eigenvectors, so the Ev's and Ev's.
The characteristic equation is lambda squared.
The trace is negative three, but you have to change the sign.
The determinant is two minus two, so that is zero.
There is no constant term here.
It is zero.
That is the characteristic equation.
The roots are obviously lambda equals zero, lambda equals negative three.
And what are the eigenvectors that go with that?
With lambda equals zero goes the eigenvector, minus two.
Well, I subtract zero here, so the equation I have to solve is minus 2 a1 plus -- I am not going to write 2 a2, which is what you have been writing up until now.
The reason is because I ran into trouble with the notation and I had to use, as the eigenvector, not a1, a2 but a1, b1.
So it should be a b1 here, not the a2 that you are used to.
The solution, therefore, is alpha one equals one, one.
And for lambda equals negative three, the corresponding eigenvector this time will be -- Now I have to subtract negative three from here, so negative two minus negative three makes one.
That is a1 plus 2 b1 equals zero, a logical choice for the eigenvector here.
The second eigenvector would be make b1 equal to one, let's say, and then a1 will be negative two.
Okay.
Now what do we have to do?
Now, what we want is the matrix E. The matrix E is the matrix of eigenvectors, so it is the matrix one, one, negative two, one.
The next thing we want is what the new variables u and v are.
For that, we will need E inverse.
How do you calculate the inverse of a two-by-two matrix?
You switch the two diagonal elements, there I have switched them, and you leave the other two where they are but change their sign.
So it is two up here and negative one there.
Maybe I should make this one purple and then that one purple to indicate that I have switched them.
I am not done yet.
I have to divide by the determinant.
What is the determinant?
It is one minus negative two, which is three, so I have to divide by three.
I multiply everything here by one-third.
Okay.
And what is the decoupled system?
The new variables are u equals one-third.
In other words, the new variables are given by D. It is u, v equals one, two, negative one, one times one-third times x,y.
That is the expression for u, v in terms of x and y.
It's this matrix D, the decoupling matrix which is the one that is used.
And that gives this system u equals one-third of x plus 2y on top.
And what is the v entry?
v is one-third of minus x plus y.
Now, are those the same variables that I used before?
Yes.
This is my new and better you, the one I got by just blindly following the method instead of looking for physical things with physical meaning.
It differs from the old one just by a constant factor.
Now, that doesn't have any effect on the resulting equation because if the old one is u prime equals zero the new one is one-third u prime equals zero.
It is still the same equation, in other words.
And how about this one?
This one differs from the other one by the factor minus one-third.
If I multiply that v through by minus one-third, I get this v. And, therefore, that too does not affect the second equation.
I simply multiply both sides by minus one-third.
The new v still satisfies the equation minus three times v.
So the topic for today is we have a system like the kind we have been studying, but there is now a difference.
A system of first order differential equations, just two of them.
It is an autonomous system meaning, of course, that there is no t explicitly on the right-hand side.
But what makes this different, now, is that it is nonlinear.
In other words, the functions on the right-hand side are no longer simple things like ax plus by, cx plus dy.
Those are the kind we have been studying.
But we are going to allow them to have quadratic terms, sines, cosines, different stuff there that are not linear functions anymore.
And the problem is, if it's a linear system you know how to get a sketch of its trajectories without using the computer by using eigenlines.
You were very good at that on the exam on Friday.
Most of you could do that very well.
But what do you do if you have a nonlinear system?
The problem is to sketch its trajectories.
In general, there are not analytic formulas for the solutions to nonlinear systems like that.
There are only computer-drawn things.
But sometimes you have to get qualitative information, a quick idea of how the trajectories look.
And, especially on Friday, I will give you examples of stuff that you can do that the computer cannot do very well at all.
Okay, so the problem is to sketch those trajectories.
Now, what I am going to do is -- The way I will give the lecture is, this is the general problem.
We have to do two things sort of simultaneously.
I will give a general explanation using x and y, but then, as we do each step of the process and talk about it in general, I would like to carry it out on a specific example.
And so we will do it with a specific example.
The example I am going to carry out is that of the nonlinear pendulum.
I am using this because it illustrates virtually everything.
And, in addition, it has the great advantage that, since we know how a pendulum swings, we will be able to, when we get the answer, verify it and, at various stages of the procedure, verify that the mathematics is, in fact, in agreement with our physical intuition.
It is going to be a lightly damped pendulum because I am going to have to put in numbers in order to do the calculations.
And that seems like a good case which illustrates several types of behavior.
Let's first of all, before we talk in general, remind you of the pendulum.
The pendulum I am talking about has the vertex from which it swings.
This is a rigid rod.
It is not one of these string-type pendulums.
There is a mass here.
The rigid rod is of length l. And so it swings in a circular orbit like that back and forth in a circle.
And let's put in the vertical distance, the vertical position rather.
And now, as variables, of course normally we use neutral variables like x and y.
But here x and y are not relevant variables to describing the way the mass moves.
The obviously relevant variable is theta, this angle.
Now, I am taking it in the positive direction.
Here theta is zero.
As it swings, theta becomes positive.
Over here, when it is horizontal, theta has the value pi over two and then so on it goes.
Values here correspond to negative values of theta.
That is how it swings.
Now, just to remind you of the equation that this satisfies, it satisfies F equals ma, rather ma equals F. Now, the acceleration is along the circular path.
And that is different from the angular acceleration.
I have to put in the factor of the length.
You had that a lot in 8.01 so I am simply going to write it down.
It is the mass.
Therefore, the linear acceleration along the circular path is equal to the angular acceleration times l. It is l times theta prime prime, or double dot if you prefer.
And so this much of it is the acceleration vector.
Now, once the force is acting on it, well, there is a force of gravity which is pulling it straight down.
But, of course, that is not the relevant force.
I am interested in the force that acts along that circular line.
And that will not be all of the pink line but only its component in that direction.
And the component, I fill out the little right triangle.
And then the way to get the component in that direction, the vertical pink part has the magnitude mg.
But, since I only want this part, I have to multiply by the sign of this angle.
Now, sometimes I have given on a diagnostic test to students when they enter to what angle is that?
But, of course, anybody can guess it must be theta.
Otherwise, why would he be asking it?
So this is still the angle theta.
You can prove these two triangles are similar.
One of them I haven't even written in, but it would be the right triangle whose leg is perpendicular to it.
So the right angle is here.
If that is theta then the length of this small pink line is mg times the sine of theta.
That is the force due gravity.
It is mg sine theta.
Except in what direction is it?
It is acting in the negative direction.
This is theta increasing.
This is the opposite direction, so I should put a negative sign in front of it.
But that is not the only force.
There is also a damping force that goes with a velocity.
And that also occurs if the angular velocity is positive, the angle theta is increasing, in other words, the damping force resists that.
It is opposite to the velocity.
So the velocity is going to be l times theta prime.
There is my velocity v. Linear velocity, not angular velocity.
And so this is going to be a negative times some constant times that c1.
Now, that is the equation.
But let's make it look a little better by getting rid of some of these constants.
If I write it out this way and put everything on the left-hand side, the way it is usually done in writing a second order differential equation.
Theta I am going to divide through by ml and put everything on the left-hand side and in the right order.
Next should come the theta prime term.
And so that is going to be c1l divided by ml, so that is c1 over m. The l's cancel out.
And, finally, the last term on the right, I will move this over to the left, but remember everything is being divided by ml, so the m's cancel out, and it is plus g over l times the sine of theta.
That is our differential equation.
But let's make it look still a little bit better by lumping these constants and giving them new names.
It is going to be finally theta double prime.
I will simply call this thing the damping constant.
I will lump those two together into single damping constant.
And then g over l, I will lump those together, too.
And we usually call that k. It is k sine theta.
Now, this is a second-order different equation, but it is not linear.
If this were a month and a half ago and I said solve that, you would stare at me.
But, anyway, you couldn't solve that.
And nobody can, in some sense.
It is a nonlinear equation.
It doesn't have any exact solution.
The only thing you could do is look for a solution in infinite series or something like that.
Well, what do you do?
You throw it on the computer, that is the easy answer, but what does the computer do?
Well, the first thing the computer does is turns it into a system because the computer is going to use numerical methods to solve it.
But only those methods, the formulas it uses, Euler or modified or improved Euler or the Runge-Cutta method, they are always expressed not for single higher-order equations, but instead they always assume that the equation has been converted to a first order system.
Let's do that for the computer, even though it will do it itself if nobody tells it not to.
Theta prime is equal to, now I have to figure out what new variable to introduce.
Normally we use x prime and call that y.
That really doesn't seem to be very suitable here.
But what do the physicists call it?
This is the angular velocity.
And the standard designation for that is omega.
Two Greek letters.
I told you this was going to be hard.
Omega prime equals what?
Well, omega prime equals, now you do it in the standard way, you convert the system, but remember you have to put the theta first.
So it is minus k sine theta.
The theta first and then the omega term first.
So minus c times omega, minus c theta prime, but theta prime is, in real life, omega.
And now we have our acceptable system.
The only problem is I have not put in any numbers yet.
The numbers I am going to put in will make it lightly damped.
I am going to give c, think of it here, this is the damping, and this is the stuff representing, well, if I want to make it lightly damped all I am saying is that c should be small compared with k, but it doesn't have to be very small.
I am going to take c equal one and k equal two, and that will make it lightly enough damped.
This is the lightly damped value, values which give underdamping.
In other words, they are going to allow the pendulum to swing back and forth instead of strictly going ug and ending up there.
Finally, therefore, the system that we are going to calculate is where theta prime equals omega and omega prime equals negative two sine theta minus omega.
And now what do we do with that?
There is our example.
That is our system that represents a pendulum swinging back and forth, damped away.
And now let's go back to the general theory.
And, in general, if you have a nonlinear system, how do you go about analyzing it?
The first step is to find the simplest possible solutions, solutions that you hope can be found by inspection.
Now, what would they be?
They are the solutions that consist of a single point.
How could a solution be a single point?
Well, like the origin for a linear system, those points which form solutions all by themselves are called the critical points.
I am looking for the critical points of the system.
That is the first step.
The definition is a critical point x zero, y zero.
For that to be a critical point means that it makes the right-hand side zero.
The f is zero there, and the g is zero there.
See the significance of that?
If you have such a point, let's say there is a critical point, what is the velocity field at that point?
Well, it is given by the vectors on the right-hand side, but the components are zero.
That means, at this point the velocity vector is zero.
Well, that means if a solution starts there, you put the mouse there and tell it to move, where do you go?
It has no reason to go anywhere since the velocity vector is zero there.
So it sits there for all time.
And indeed it solves the system, doesn't it?
It makes the right-hand side zero and it makes the left-hand side zero because x equals x zero, y equals y zero for all time.
If that is true for all time it sits there then the derivatives, with respect to time are zero, so the left-hand sides are zero, the right-hand sides are zero and everybody is happy.
Well, these are great points.
How do I find them?
Well, by looking for points that make these two functions zero.
I find them by solving simultaneously the equations f of (x, y) equals zero and g of (x, y) equals zero.
A pair of equations.
But the trouble is those are not linear equations.
Linear equations you know how to solve, but they are not linear equations.
They are nonlinear equations that you don't know how to solve.
And, to some extent, nobody else does either.
There are very fat books in the library whose topic is how to solve just a pair of equations, f of (x, y) equals zero, g of (x, y) equals zero.
And it is quite a hard problem.
It is even a hard problem by computer.
Because, if you know approximately where the solution is going to be, you can make up the little screen and then the computer will find it for you.
Or, even without a screen, it will calculate it by Newton's method or something else, it will zero in.
The problem is, if you don't know in advance roughly where the critical point is that you are looking for, there are a lot of numbers.
They go to infinity that way and infinity that way.
In general, it is almost an impossible problem.
The only thing that makes it possible is that these problems always come from the real world and one has some physical feeling, one hopes, for where the critical point is.
I am going to assume breezily and cheerily we can solve those.
And I am only going to give you examples where it is possible to solve them.
But even there, you have to watch out.
There is a certain trickiness that will be talked about in the recitations tomorrow.
Okay, so we found the critical points.
Let's do it for our example.
Let's find the critical point.
What is the pair of equations we have to solve?
We have to solve the equations omega equals zero, minus two sine theta minus omega equals zero.
Now, it is not always this easy.
But the solution is omega is zero.
If omega is zero, then sine theta is zero, and sine theta is zero at the integral multiples of pi.
The critical points are omega is always zero and theta is zero, or it could be plus or minus pi, plus or minus two pi and so on.
In other words, there are an infinity of critical points.
That seems a little discouraging.
On the other hand, there are really only two.
There are really physically only two because omega equals zero means the mass is not moving.
The angular velocity is zero, so it is only the theta position which is changing.
Now, what are the possible theta positions?
Well, here is our nonlinear pendulum.
Here is the critical point, theta equals zero, omega equals zero.
Theta equals zero means the rod is vertical.
Omega equals zero means that it is not moving, despite the fact that it is moving.
Theoretically it is not moving.
Now, what is the other one?
Well, there is theta equals pi.
Theta is now, starting from zero and increasing, I hope, through positive theta.
And when it gets to pi, it is sticking straight up in the air.
And so the claim is that another critical point is theta equals pi and omega equals zero.
In other words, if it gets to this position, it starts out in this position and stays there for all time, as you see.
[LAUGHTER] But my point is what about negative pi?
That is the same as pi.
Two pi is the same as zero.
So physically there really are only two critical points, this one and that one.
And they obviously have something very different about them.
This critical point is stable.
If I start near there, I approach that critical point in infinite time.
This one, if I start near there, I do not stay near there.
I always leave it.
Of the two critical points, physically it is clear that the critical point is zero, zero and the other guys that look like it, two pi zero and so on is a stable critical point.
Whereas, pi zero, when theta is sticking straight up in the air, is unstable.
Now, of course, we will want to see that mathematically also, but basically there are just physically two point.
There are just two critical points.
Well, that raises the question what about all the others?
As you will see, we have to have those.
They are an essential part of the problem.
They are not just redundant baggage that is trailing along.
They are really important.
But you will see that when we talk about finally how the trajectories look and how the solutions look.
Now what do we do?
Well, we found the critical points, and now the work begins.
Virtually all the work is in this next step.
What do we do?
Step two.
I can only describe it in general terms, but here is what you do.
For each critical point x zero, y zero, a procedure that has to be done at each one, you linearize the system near that point.
In other words, you may find a linear system, the good kind, the kind you know about, which is a good approximation to the nonlinear system at that critical point.
Plot the trajectories of this linearized system.
And you do that near the critical point.
How do you plot the trajectories?
Well, that you knew how to do on Friday so I am assuming you still know how to do it on Monday.
In other words, if the system is linear you know how to plot the trajectories of it by calculating eigenvalues and eigenvectors and maybe the direction of motion if it is a spiral.
I will give you a couple of examples of that when we work out the pendulum.
But, on the other hand, how do I line arise a system?
Well, there are two methods.
There is one method the book gives you, which by and large I do not want you to use, although I will give you an example of it now.
I want you to use another method because it is much faster.
Especially if you have to handle several critical points it is much, much faster.
Let's first carry this out on an easy case, and then I will show you how to do it in general.
Just this once I will use the book's method because I think it is the method which would naturally occur to you.
Let's linearize this example at the point zero, zero.
What should be the linearized system?
In other words, it's only the nonlinear terms I have to worry about.
Well, the minus omega is fine.
It is that stupid sine theta that I don't like.
But if theta is small, in other words, if I stay near zero, I could replace sine theta by theta.
The linearized system is minus two.
You replace sine theta by theta, since sine theta is approximately theta if theta is small, if theta is near zero.
It is the first term of its Taylor series you can think of, or it's just the linear approximation starts out sine theta equals theta.
That is it.
Or, you draw a picture.
I don't know.
There are a millions of ways to do it.
So we have it.
Now what do I do?
Okay, now we will plot that.
The matrix, let's do our little routine, in other words.
I am writing right to left for no reason.
The matrix is, now I am just going to make marks on a board the way you did on your exam, zero, one.
[LAUGHTER] I don't know what I am doing, but you know.
Negative two, negative one, and then I write down the eigen-whatchamacallits.
Lambda squared, plus lambda, minus the trace.
The determinant is minus, minus two, so it is plus two equals zero.
And then, since it doesn't occur to me how to factor this, I will use the quadratic formula.
It is negative one plus or minus the square root of, b squared minus 4ac, minus seven.
Complex.
That means it is going to be a spiral.
I am going to get a spiral.
Will it be a source or a sink since they are complex roots?
Complex eigenvalues give a spiral.
A source or a sink?
I tell that from the sine of the real part.
The real part is negative one-half.
Therefore, the amplitude is shrinking like e to the minus t over two.
And, therefore, the spiral is coming into the origin.
It is a spiral sink since lambda equals minus one-half plus some number times i. Spiral sink.
And the other thing to determine is its direction of motion, which will be what?
I determine its direction of motion by putting in a single vector from the velocity field.
Here it is.
The vector at one, zero will be the same as the first column of the matrix.
So that is zero, negative two.
Here is a vector from the velocity field, and that shows that the motion is clockwise and is spiraling into the origin.
That is a picture, therefore, at the origin of how that looks.
Now, it is of the utmost importance for your understanding of what comes now that you understand in what sense this picture corresponds to the physical behavior of the pendulum.
Let's start it over here.
What is it doing?
That means that theta is some number like one, for example.
Let's make it smaller.
Let's say a little bit.
And this is the omega access.
If it starts over here that means the angular velocity is zero and theta is a small positive number.
Theta is a small positive number.
The angular velocity is zero.
It's velocity zero, theta small and positive.
I release it and it does that.
What does this have to do with the spiral?
Well, the spiral is exactly a mathematical picture of this motion.
What happens?
Theta starts to decrease.
And the angular velocity increases but in the negative direction.
This is negative angular velocity.
Theta is decreasing.
The angular velocity gets bigger and bigger.
And it is biggest, most negative when theta is zero.
It has reached the vertical position is when the angular velocity is biggest.
It continues then.
Theta gets negative, but the angular velocity then decreases to zero.
Now the angular velocity is zero and theta is at its most negative, and then it reverses.
Angular velocity gets positive as theta increases again, and so on.
These represent the successive swings back and forth.
Notice the fact that it is damped is reflected in the fact that each successive swing, the biggest that theta gets is a little less than it was before.
In other words, this point is not quite as far out as that one.
And this one isn't as far out as that one.
In other words, it is spiraling in.
Well, I hope that is clear because we now have to go to the next critical point.
And now we have a little problem.
If I want to do the next critical point, so what I want to do now is, in other words, I want to linearize at the point pi zero where theta is pi and the thing is sticking up in the air.
The question is, how am I going to do that?
This trick of replacing sine theta by theta, that doesn't work at pi.
That works at zero.
Now we have to go to the next step of the method.
The way to do this, in general, as you will read in the notes because, as I say, this is not in the book, is to calculate the Jacobian.
I mean the Jacobian matrix.
The Jacobian is the determinant.
I mean, before you put the two bars down and made a determinant, you called it just the Jacobian matrix.
And the formula for it is, it is calculated from f and g. The top line is the partial of f with respect to x and y, and the bottom line is the partial of g with respect to x and y.
So that is the Jacobian matrix.
I hope I get a chance at the end of the period to explain to you why, but I am most anxious right now to at least get you familiar with the algorithm, how to do it.
The notes describe the y of it, if we don't get a chance to get to it, but I hope we will.
What do you do?
The Jacobian matrix.
You calculate it at the point x zero, y zero.
I will indicate that by putting a subscript zero on it.
This means without the subscript zero it is the Jacobian matrix calculated out of those four partial derivatives.
When I put a subscript zero, I mean I evaluated it at the critical point by plugging in each entry as a function of x and y.
You plug in for x equals x zero, y equals y zero, and you get a numerical matrix.
That is the matrix which is the matrix of the linearized system.
This is the matrix of the linearized system.
Trust me, it is.
Now, since I don't expect you to trust me, let's calculate it.
Here, we got the matrix another way, by this procedure of saying sine theta is theta-- What would we have gotten if we had done it instead by the linearized system?
Let's do it that way.
Let's do it via the Jacobian.
I need to know the Jacobian of the system, which I have conveniently covered up.
There is the system.
The Jacobian matrix is what?
The top line, I take the partial derivative of omega first with respect to theta and then with respect to omega.
I then take the second line on the right-hand side.
I take its partial with respect to theta first.
That is negative two cosine theta.
And, thinking ahead, I erase the one and move it over a little bit.
And what is the partial of that thing with respect to omega?
It is negative one.
Does everyone see how I calculated that Jacobian matrix?
And now I want to evaluate it.
Let's do our old case first.
At zero, zero what would this have amounted to?
This would have given me zero, one.
The cosine of theta at zero is one, so this is negative two, negative one.
I'm screwed.
[LAUGHTER] That is the same as that.
Now you see it, now you don't.
We got our old answer back.
That should give us enough confidence to use it in the new case, where I don't have an old answer to compare it with.
What is it going to be at pi, zero?
The answer is J zero is now going to be -- Well, everything is the same.
Zero, one, negative one.
And here cosine of pi is negative one, so negative two times negative one is two.
Everything is the same, except there is a two there now instead of negative two.
Lambda squared plus lambda, the determinant, is now negative two.
And this factors into lambda plus two times lambda minus one.
So lambda equals one.
The corresponding eigenvector.
I subtract one here, so the equation is minus a1 plus a2 is zero.
The solution is one, one, e to the t. And for the other one, it's lambda is equal to negative two.
This is the sort of stuff you can do, so I am doing it fast.
Zero minus negative two is two.
So the equation is 2a1 plus a2 equals zero.
And the solution is now, I give a1 the value one, a2 will be negative two, and that is times e to the minus 2t.
Well, what does the thing then actually look like?
What I am now going to do is, I drew a picture before, that spiral picture we had before of the way the thing looked at the point zero, zero.
So at the point pi, zero how does it look, now?
Well, it looks like the origin, but I am thinking of it really as the point pi, zero.
In other words, I am thinking of a linear variable change sliding along the axis so that the point pi, zero now looks like the origin.
If I do that then those two basic solutions, there is the one, one solution which is going out that way.
And it is going out this way.
But the other guy is coming in along the vector one, negative two.
So one, negative two looks like this.
This guy is coming in at a somewhat sharper angle, coming in because it is e to the negative 2t.
And we recognize this, of course, as a saddle.
And I would complete the trajectories by putting in some of the typical saddle lines like that.
Now, I say that, too, gives a picture of what is happening to the pendulum near that point.
Let's, for example, look here.
What is happening here?
This is theta, or really it is theta minus pi, is this axis.
In other words, this will be zero.
When theta is pi this will correspond to the point zero.
Here is omega.
What is theta doing?
Starting up there this represents a value of theta a little bit less than pi, a little bit to the negative.
A little bit less than pi.
Here is pi, so a little bit less than pi is over here.
A little bit less than pi.
A little bit less.
And omega is zero.
What happened?
Theta started decreasing ever more rapidly so that the omega was zero here, now omega is negative and gets much more negative.
In other words, both theta decreases from that point and omega decreases also.
What happened here?
Here, theta is a little bit more than pi.
Now it is a little bit more than pi.
Theta now increases until it gets to 2pi and then oscillates around 2pi.
So theta increases.
And omega increases, too, because the angular velocity started at zero but, as theta gets more positive, omega gets positive, too, because theta is increasing.
So omega increases and theta increases and it goes off like that.
Well, the final step is to put them all together to be the big picture.
Here we are for three, let's say, the big picture.
Plot trajectories around each critical point and then add some.
It's that last step that can cause you a little grief, but we will see how it works out.
Add some more, according to your best judgment.
Let's make a big picture now of our pendulum the way it apparently ought to look.
Nice big axis since we are going to accommodate a lot of critical points here.
Let's put in some critical points.
Here is the origin.
And now here is the one at pi, let's say, here is one at 2pi.
I am going to add some of these others, 3pi, 4pi.
I won't in their values.
You can figure out what I mean.
Zero here.
And then here it will be negative pi, and here negative 2pi.
I guess I can stop there.
That is the theta axis, and here is the omega axis.
And now, at each one of these, I stay nearby and I draw the linear trajectories, the trajectories of the corresponding linearized system.
We decided here that the spiral went clockwise.
Now, this point is physically, of course, the same as that one.
But mathematically, the Jacobian matrix is the same also because if theta is 2pi the cosine of 2pi is also one.
And this is the same matrix.
So the analysis is identical.
And, therefore, this point will also correspond to a counterclockwise spiral, as will this one.
I am sorry, a clockwise spiral.
Here, too, all of these points are the same.
The behavior near them, clockwise spirals everywhere.
How about the other ones?
Well, the other ones correspond to these saddles, so let's draw them efficiently by doing the same thing on every one, mass production of saddles.
There.
And these guys go out.
And the other guys come in, etc.
And so here we have a little bit there, a little here, here, there, everywhere, etc.
Now what?
Now you pray for inspiration.
And what you have to do is add trajectories that are compatible with the ones you have.
Let's start with this guy.
Where is it going?
Well, a trajectory either goes off to infinity, but generally they get trapped around critical points.
This guy must be surely doing this.
How about this one?
Yeah, sure.
How about this one?
Why not?
How about that one?
Yeah.
But notice you are in trouble when two arrows you want to put in are near each other and going in opposite directions.
That you cannot have.
Continuity forbids it.
But notice if I, for example, had gotten these eigenlines wrong, if I made this come in and those go out because I made a simple error in drawing the thing, I would have said but I cannot draw this picture because this spiral wants to go that way but this, right next to it, wants to go the other way.
That is the way you would know if you made a mistake.
If you didn't make a mistake you won't have any trouble filling these things out.
The directions of motions of the spirals, everything will be compatible.
Okay.
What is this guy doing?
Oh, well, it must be joining up with that.
How about this one?
Well, it must be coming back there.
How about this one?
Well, trajectories cannot cross.
This guy cannot cross so it must be doing this.
All right.
What is that trajectory?
Starts zero.
Omega, on the other hand, is big and positive.
Omega big and positive.
[LAUGHTER] I am scared.
Omega starts.
Theta is zero.
Omega big and positive.
It went around, slowed up, but continued beyond pi.
And, in fact, went too far.
It continued to go here and then finally wound around that one.
Now do you see why we had to have all the critical points?
You have to have all the critical points.
And not just the two physicals ones because the other critical points are necessary to describe a complicated motion that goes round and round and round until finally it crashes.
You are going to practice drawing these pictures and interpreting them in recitation tomorrow.
First of all, the way a nonlinear autonomous system looks, you have had some practice with it by now.
This is nonlinear.
The right-hand side are no longer simple combinations ax plus by.
Nonlinear and autonomous, these are function just of x and y.
There is no t on the right-hand side.
Now, most of today will be geometric.
The way to get a geometric picture of that is first by constructing the velocity field whose components are the functions f and g. This is a velocity field that gives a picture of the system and has solutions.
The solutions to the system, from the point of view of functions, they would look like pairs of functions, x of t, y of t. But, from the point of view of geometry, when you plot them as parametric equations, they are called trajectories of the field F, which simply means that they are curves everywhere having the right velocity.
So a typical curve would look like -- There is a trajectory.
And we know it is a trajectory because at each point the vector on it has, of course, the right direction, the tangent direction, but more than that, it has the right velocity.
So here, for example, the point is traveling more slowly.
Here it is traveling more rapidly because the velocity vector is bigger, longer.
So this is a picture of a typical trajectory.
The only other things that I should mention are the critical points.
If you have worked the problems for this week, the first couple of problems, you have already seen the significance of the critical points.
Well, from Monday's lecture you know from the point of view of solutions they are constant solutions.
From the point of view of the field they are where the field is zero.
There is no velocity vector, in other words.
The velocity vector is zero.
And, therefore, a point being there has no reason to go anywhere else.
And, spelling it out, it's where the partial derivatives, where the values of the functions on the right-hand side, which give the two components, the i and j components of the field, where they are zero.
That is all I will need by way of a recall today.
I don't think I will need anything else.
The topic for today is another kind of behavior that you have not yet observed at the computer screen, unless you have worked ahead, and that is there are trajectories which go along to infinity or end up at a critical point.
They are the critical points that just sit there all the time.
But there is a third type of behavior that a trajectory can have where it neither sits for all time nor goes off for all time.
Instead, it repeats itself.
Such a thing is called a closed trajectory.
What does it look like?
Well, it is a closed curve in the plane that at every point, it is a trajectory, i.e., the arrows at each point, let's say it is traced in the clockwise direction.
And so the arrows of the field will go like this.
Here it is going slowly, here it is very slow and here it picks up a little speed again and so on.
Now, for such a trajectory what is happening?
Well, it goes around in finite time and then repeats itself.
It just goes round and round forever if you land on that trajectory.
It represents a system that returns to its original state periodically.
It represents periodic behavior of the system.
Now, we have seen one example of that, a simple example where this simple system, x prime equals y, y prime equals negative x.
We could write down the solutions to that directly, but if you want to do eigenvalues and eigenvectors the matrix will look like this.
The equation will be lambda squared plus zero lambda plus one equals zero, so the eigenvalues will be plus or minus i.
In fact, from then on you could work out in the usual ways the eigenvectors, complex eigenvectors and separate them.
But, look, you can avoid all that just by writing down the solution.
The solutions are sines and cosines.
One basic solution will be x equals cosine t, in which case what is y?
Well, y is the derivative of that.
That will be minus sine t. Another basic solution, we will start with x equals sine t. In which case y will be cosine t, its derivative.
Now, if you do that, what do these things look like?
Well, either of these two basic solutions looks like a circle, not traced in the usual way but in the opposite way.
For example, when t is equal to zero it starts at the point one, zero.
Now, if the minus sign were not there this would be x equals cosine t, y equals sine t, which is the usual counterclockwise circle.
But if I change y from sine t to negative sine t it is going around the other way.
So this circle is traced that way.
And this is a family of circles, according to the values of c1 and c2, concentric, all of which go around clockwise.
So those are closed trajectories.
Those are the solutions.
They are trajectories of the vector field.
They are closed.
They come around and they repeat in finite time.
Now, these are no good.
These are the kind I am not interested in.
These are commonplace, and we are interested in good stuff today.
And the good stuff we are interested in is limit cycles.
A limit cycle is a closed trajectory with a couple of extra hypotheses.
It is a closed trajectory, just like those guys, but it has something they don't have, namely, it is king of the roost.
They have to be isolated, no other guys nearby.
And they also have to be stable.
See, the problem here is that none of these stands out from any of the others.
In other words, there must be, isolated means, no others nearby.
That is just what goes wrong here.
Arbitrarily close to each of these circles is yet another circle doing exactly the same thing.
That means that there are some that are only of mild interest.
What is much more interesting is to find a cycle where there is nothing nearby.
Something, therefore, that looks like this.
Here is our pink guy.
Let's make this one go counterclockwise.
Here is a limit cycle, it seems to be.
And now what do nearby guys do?
Well, they should approach it.
Somebody here like that does this, spirals in and gets ever and every closer to that thing.
Now, it can never join it because, if it joined it at the joining point, I would have two solutions going through this point.
And that is illegal.
All it can do is get arbitrarily close.
On the computer screen it will look as if it joins it but, of course, it cannot.
It is just the resolution, the pixels.
Not enough pixels.
The resolution isn't good enough.
And the ones that start further away will take longer to find their way to the limit cycle and they will always stay outside of the earlier guys, but they will get arbitrarily close, too.
How about inside?
Inside, well, it starts somewhere and does the same thing.
It starts here and will try to join the limit cycle.
That is what I mean by stability.
Stability means that nearby guys, the guys that start somewhere else eventually approach the limit cycle, regardless of whether they start from the outside or start from the inside.
So that is stable.
An unstable limit cycle -- But I am not calling it a limit cycle if it is unstable.
I am just calling it a closed trajectory, but let's draw one which is unstable.
Here is the way we will look if it is unstable.
Guys that start nearby will be repelled, driven somewhere else.
Or, if they start here, they will go away from the thing instead of going toward it.
This is unstable.
And I don't call it a limit cycle.
It is just a closed trajectory.
Cycle because it cycles round and round.
Limit because it is the limit of the nearby curves.
The other case where it is unstable is not the limit.
Of course, you could have a case also where the curves outside spiral in toward it but the ones inside are repelled and do this.
That would be called semi-stable.
And you can make up all sorts of cases.
And I think I, at one point, drew them in the notes, but I am not going to.
The only interesting one, of permanent importance that people study, are the actual limit cycles.
No, it was the stable closed trajectories.
Notice, by the way, a closed trajectory is always a simple curve.
Remember what that means from 18.02?
Simple means it doesn't cross itself.
Why doesn't it cross itself?
It cannot cross itself because, if it tried to, what is wrong with that point?
At that point which way does the vector field go, that way or that way?
Why the interest of limit cycles?
Well, because there are systems in nature in which just this type of behavior, they have a certain periodic motion.
And, if you disturb it, gradually it returns to its original periodic state.
A simple example is breathing.
Now I have made you all self-conscious.
All of you are breathing.
If you are here you are breathing.
At what rate are you breathing?
Well, you are unaware of it, of course, except now.
If you are sitting here listening.
There is a certain temperature and a certain air circulation in the room.
You are not thinking of anything, certainly not of the lecture, and the lecture is not unduly exciting, you will breathe at a certain steady rate which is a little different for every person but that is your rate.
Now, you can artificially change that.
You could say now I am going to breathe faster.
And indeed you can.
But, as soon as you stop being aware of what you are doing, the levels of various hormones and carbon dioxide in your bloodstream and so on will return your breathing to its natural state.
In other words, that system of your breathing, which is controlled by various chemicals and hormones in the body, is exhibiting exactly this type of behavior.
It has a certain regular periodic motion as a system.
And, if disturbed, if artificially you set it out somewhere else, it will gradually return to its original state.
Now, of course, if I am running it will be different.
Sure.
If you are running you breathe faster, but that is because the parameters in the system, the a's and the b's in the equation, the f of (x, y) and g of (x, y), the parameters in those functions will be set at different levels.
You will have different hormones, a different of carbon dioxide and so on.
Now, I am not saying that breathing is modeled by a limit cycle.
It is the sort of thing which one might look for a limit cycle.
That is, of course, a question for biologists.
And, in general, any type of periodic behavior in nature, people try to see if there is some system of differential equations which governs it in which perhaps there is a limit cycle, which contains a limit cycle.
Well, what are the problems?
In a sense, limit cycles are easy to lecture about because so little is known about them.
At the end of the period, if I have time, I will show you that the simplest possible question you could ask, the answer to it is totally known after 120 years of steady trying.
But let's first talk about what sorts of problems people address with limit cycles.
First of all is the existence problem.
If I give you a system, you know, the right-hand side is x squared plus 2y cubed minus 3xy, and the g is something similar.
I say does this have limit cycles?
Well, you know how to find its critical points.
But how do you find out if it has limit cycles?
The answer to that is nobody has any idea.
This problem, in general, there are not much in the way of methods.
Not much.
Not much is known.
There is one theorem that you will find in the notes, a simple theorem called the Poincare-Bendixson theorem which, for about 60 or 70 years was about the only result known which enabled people to find limit cycles.
Nowadays the theorem is used relatively little because people try to find limit cycles by computer.
Now, the difficulty is you have to know where to look for them.
In other words, the computer screen shows that much and you set the axes and it doesn't show any limit cycles.
That doesn't mean there are not any.
That means they are over there, or it means there is a big one like there.
And you are looking in the middle of it and don't see it.
So, in general, people don't look for limit cycles unless the physical system that gave rise to the pair of differential equations suggests that there is something repetitive going on like breathing.
And, if it tells you that, then it often gives you approximate values of the parameters and the variables so you know where to look.
Basically this is done by computer search guided by the physical problem.
Therefore, I cannot say much more about it today.
Instead I am going to focus my attention on nonexistence.
When can you be sure that a system will not have any limit cycles?
And there are two theorems.
One, again, due to Bendixson who was a Swedish mathematician who lived around 1900 or so.
There is a criterion due to Bendixson.
And there is one involving critical points.
And I would like to describe both of them for you today.
First of all, Bendixson's criterion.
It is very simply stated and has a marvelous proof, which I am going to give you.
We have D as a region of the plane.
And what Bendixson's criterion tells you to do is take your vector field and calculate its divergence.
We are set back in 1802, and this proof is going to be straight 18.02.
You will enjoy it.
Calculate the divergence.
Now, I am talking about the two-dimensional divergence.
Remember that is fx, the partial of f with respect to x, plus the partial of the g, the j component with respect to y.
And assume that that is a continuous function.
It always will be with us.
Practically all the examples I will give you f and g will be simple polynomials.
They are smooth, continuous and nice and behave as you want.
And you calculate that and assume -- Suppose, in other words, that the divergence of f, I need more room.
The hypothesis is that the divergence of f is not zero in that region D. It is never zero.
It is not zero at any point in that region.
The conclusion is that there are no limit cycles in the region.
If it is not zero in D, there are no limit cycles.
In fact, there are not even any closed trajectories.
You couldn't even have those bunch of concentric circles, so there are no closed trajectories of the original system whose divergence you calculated.
There are no closed trajectories in D. For example, let me give you a simple example to put a little flesh on it.
Let's see.
What do I have?
I prepared an example.
x prime equals, here is a simple nonlinear system, x cubed plus y cubed.
And y prime equals 3x plus y cubed plus 2y.
Does this system have limit cycles?
Well, even to calculate its critical points would be a little task, but we can easily answer the question as to whether it has limit cycles or not by Bendixson's criterion.
Let's calculate the divergence.
The divergence of the vector field whose components are these two functions is, well, 3x squared, it's the partial of the first guy with respect to x plus the partial of the second guy with respect to y, which is 3y squared plus two.
Now, can that be zero anywhere in the x,y-plane?
No, because it is the sum of these two squares.
This much of it could be zero only at the origin, but that plus two eliminates even that.
This is always positive in the entire x,y-plane.
Here my domain is the whole x,y-plane and, therefore, the conclusion is that there are no closed trajectories in the x,y-plane, anywhere.
And we have done that with just a couple of lines of calculation and nothing further required.
No computer search.
In fact, no computer search could ever proof this.
It would be impossible because, no matter where you look, there is always some other place to look.
This is an example where a couple lines of mathematics dispose of the matter far more effectively than a million dollars worth of calculation.
Well, where does Bendixson's theorem come from?
Yes, Bendixson's theorem comes from 18.02.
And I am giving it to you both to recall a little bit of 18.02 to you.
Because it is about the first example in the course that we have had of an indirect argument.
And indirect arguments are something you have to slowly get used to.
I am going to give you an indirect proof.
Remember what that is?
You assume the contrary and you show it leads to a contradiction.
What would assuming the contrary be?
Contrary would be I will assume the divergence is not zero, but I will suppose there is a closed trajectory.
Suppose there is a closed trajectory that exists.
Let's draw a picture of it.
And let's say it goes around this way.
There is a closed trajectory for our system.
Let's call the curve C. And I am going to call the inside of it R, the way one often does in 18.02.
D is all this region out here, in which everything is taking place.
This is to exist in D. Now, what I am going to do is calculate a line integral around that curve.
A line integral of this vector field.
Now, there are two things you can calculate.
One of the line integrals, I will put in a few of the vectors here.
The vectors I know are pointing this way because that is the direction in which the curve is being traversed in order to make it a trajectory.
Those are a few of the typical vectors in the field.
I am going to calculate the line integral around that curve in the positive sense.
In other words, not in the direction of the salmon-colored arrow, but in the normal sense in which you calculate it using Green's theorem, for example.
The positive sense means the one which keeps the region, the inside on your left, as you walk around like that the region stays on your left.
That is the positive sense.
That is the sense in which I am integrating.
I am going to use Green's theorem, but the integral that I am going to calculate is not the work integral.
I am going to calculate instead the flux integral, the integral that represents the flux of F across C. Now, what is that integral?
Well, at each point, you station a little ant and the ant reports the outward flow rate across that point which is F dotted with the normal vector.
I will put in a few normal vectors just to remind you.
The normal vectors look like little unit vectors pointing perpendicularly outwards everywhere.
These are the n's.
F dotted with the unit normal vector, and that is added up around the curve.
This quantity gives me the flux of the field across C. Now, we are going to calculate that by Green's theorem.
But, before we calculate it by Green's theorem, we are going to psych it out.
What is it?
What is the value of that integral?
Well, since I am asking you to do it in your head there can only be one possible answer.
It is zero.
Why is that integral zero?
Well, because at each point the field vector, the velocity vector is perpendicular to the normal vector.
Why?
The normal vector points perpendicularly to the curve but the field vector always is tangent to the curve because this curve is a trajectory.
It is always supposed to be going in the direction given by that white field vector.
Do you follow?
A trajectory means that it is always tangent to the field vector and, therefore, always perpendicular to the normal vector.
This is zero since F dot n is always zero.
Everywhere on the curve, F dot n has to be zero.
There is no flux of this field across the curve because the field is always in the same direction as the curve, never perpendicular to it.
It has no components perpendicular to it.
Good.
Now let's do it the hard way.
Let's use Green's theorem.
Green's theorem says that the flux across C should be equal to the double integral over that region of the divergence of F. It's like Gauss theorem in two dimensions, this version of it.
Divergence of F, that is a function, I double integrate it over the region, and then that is dx / dy, or let's say da because you might want to do it in polar coordinates.
And, on the problem set, you certainly will want to do it in polar coordinates, I think.
All right.
How much is that?
Well, we haven't yet used the hypothesis.
All we have done is set up the problem.
Now, the hypothesis was that the divergence is never zero anywhere in D. Therefore, the divergence is never zero anywhere in R. What I say is the divergence is either greater than zero everywhere in R. Or less than zero everywhere in R. But it cannot be sometimes positive and sometimes negative.
Why not?
In other words, I say it is not possible the divergence here is one and here is minus two.
That is not possible because, if I drew a line from this point to that, along that line the divergence would start positive and end up negative.
And, therefore, have to be zero some time in between.
It's because it is a continuous function.
It is a continuous function.
I am assuming that.
And, therefore, if it sometimes positive and sometimes negative it has to be zero in between.
You cannot get continuously from plus one to minus two without passing through zero.
The reason for this is, since the divergence is never zero in R it therefore must always stay positive or always stay negative.
Now, if it always stays positive, the conclusion is then this double integral must be positive.
Therefore, this double integral is either greater than zero.
That is if the divergence is always positive.
Or, it is less than zero if the divergence is always negative.
But the one thing it cannot be is not zero.
Well, the left-hand side, Green's theorem is supposed to be true.
Green's theorem is our bedrock.
18.02 would crumble without that so it must be true.
One way of calculating the left-hand side gives us zero.
If we calculate the right-hand side it is not zero.
That is called the contradiction.
Where did the contradiction arise from?
It arose from the fact that I supposed that there was a closed trajectory in that region.
The conclusion is there cannot be any closed trajectory of that region because it leads to a contradiction via Green's theorem.
Let me see if I can give you some of the argument for the other, well, let's at least state the other criterion I wanted to give you.
Suppose, for example, we use this system, x prime equals -- Does this have limit cycles?
Does that have limit cycles?
Let's Bendixson it.
We will calculate the divergence of a vector field.
It is 2x from the top function.
The partial with respect to x is 2x.
The second function with respect to y is negative 2y.
That certainly could be zero.
In fact, this is zero along the entire line x equals y.
Its divergence is zero here along that whole line.
The best I could conclude was, I could conclude that there is no limit cycle like this and there is no limit cycle like this, but there is nothing so far that says a limit cycle could not cross that because that would not violate Bendixson's theorem.
In other words, any domain that contained part of this line, the divergence would be zero along that line.
And, therefore, I could conclude nothing.
I could have limit cycles that cross that line, as long as they included a piece of that line in them.
The answer is I cannot make a conclusion.
Well, that is because I am using the wrong criterion.
Let's instead use the critical point criterion.
Now, I am going to say that it makes a nice positive statement but nobody ever uses it this way.
Nonetheless, let's first state it positively, even though that is not the way to use it.
The positive statement will be, once again, we have our region D and we have a region of the xy plane and we have our C, a closed trajectory in it.
A closed trajectory of what?
Of our system.
And that is supposed to be in D. The critical point criterion says something very simple.
If you have that situation it says that inside that closed trajectory there must be a critical point somewhere.
It says that inside C is a critical point.
Now, this won't help us with the existence problem.
This won't help us find a closed trajectory.
We will take our system and say it has a critical point here and a critical point there.
Does it have a closed trajectory?
Well, all I know is the closed trajectory, if it exists, will have to go around one or more of those critical points.
But I don't know where.
It is not going to go around it like this.
It might go around it like this.
And my computer search won't find it because it is looking at too small a part of the screen.
It doesn't work that way.
It works negatively by contraposition.
Do you know what the contrapositive is?
You will at least learn that.
A implies B says the same thing as not B implies not A.
They are different statements but they are equivalent to each other.
If you prove one you prove the other.
What would be the contrapositive here?
If you have a closed trajectory inside is a critical point.
The theorem is used this way.
If D has no critical points, it has no closed trajectories and therefore has no limit cycle.
Because, if it did have a closed trajectory, inside it would be a critical point.
But I said B had no critical point.
That enables us to dispose of this system that Bendixson could not handle at all.
We can dispose of this system immediately.
Namely, what is it?
Where are its critical points?
Well, where is that zero?
x squared plus y squared is one, plus one is never zero.
This is positive.
Or, worse, zero.
And then I add the one to it and it is not zero anymore.
This has no zeros and, therefore, it does not matter that this one has a lot of zeros.
It makes no difference.
It has no critical points.
It has none, therefore, no limit cycles.
Now, I desperately wanted to give you the proof of this.
It is clearly impossible in the time remaining.
The proof requires a little time.
I haven't decided what to do about that.
It might leak over until Friday's lecture.
Instead, I will finish by telling you a story.
How is that?
And along side of it was little y prime.
I am not going to continue on with the letters of the alphabet.
I will prime the earlier one.
This has a total of 12 parameters in it.
But, in fact, if you change variables you can get rid of all the linear terms.
The important part of it is only the quadratic terms in the beginning.
This sort of thing is called a quadratic system.
After you have departed from linear systems, it is the simplest kind there is.
And the predictor-prey, the robin-earthworm example I gave you is of a typical quadratic system and exhibits typical nonlinear quadratic system behavior.
Now, the problem is the following.
A, b, c, d, e, f and so on, those are just real numbers, parameters, so I am allowed to give them any values I want.
And the problem that has bothered people since 1880 when it was first proposed is how many limit cycles can a quadratic system have?
After 120 years this problem is totally unsolved, and the mathematicians of the world who are interested in it cannot even agree with each other on what the right conjecture is.
But let me tell you a little bit of its history.
There were attempts to solve it in the 20 or 30 years after it was first proposed, through the 1920 and `30s which all seemed to have gaps in them.
Until finally around 1950 two Russians mathematicians, one of whom is extremely well-known, Petrovski, a specialist in systems of ordinary differential equations published a long and difficult, complicated 100 page paper in which they proved that the maximum number is three.
I won't put down their names.
Petrovski-Landis.
The maximum number was three.
And then not many people were able to read the paper, and those who did there seemed to be gaps in the reasoning in various places until finally Arnold who was the greatest Russian, in my opinion, one of the greatest Russian mathematician, certainly in this field of analysis and differential equations, but in other fields, too, he still is great, although he is somewhat older now, criticized it.
He said look, there is a really big gap in this argument and it really cannot be considered to be proven.
People tried working very hard to patch it up and without success.
Then about 1972 or so, '75 maybe, a Chinese mathematician found a system with four.
Wrote down the numbers, the number a is so much, b is so much, and they were absurd numbers like 10^-6 and 40 billion and so on, nothing you could plot on a computer screen, but found a system with four.
Nobody after this tried to fill in the gap in the Petrovski-Landis paper.
I was then chairman of the math department, and one of my tasks was protocol and so on.
Anyway, we were trying very hard to attract a Chinese mathematician to our department to become a full professor.
He was a really outstanding analyst and specialist in various fields.
Anyway, he came in for a courtesy interview and we chatted.
At the time, I was very much interested in limit cycles.
And I had on my desk the Math Society's translation of the Chinese book on limit cycles.
A collection of papers by Chinese mathematicians all on limit cycles.
After a certain point he said, oh, I see you're interested in limit cycle problems.
I said yeah, in particular, I was reading this paper of the mathematician who found four limit cycles.
And I opened to that system and said the name is, and I read it out loud.
I said do you by any chance know him?
And he smiled and said yes, very well.
That is my mother.
[LAUGHTER] Well, bye-bye.
Today's lecture is going to be basically devoted to working out a single example of a nonlinear system, but it is a very good example because it illustrates three things which you really have to know about nonlinear systems.
I have indicated them by three cryptic words on the board, but you will see at different points in the lecture what they refer to.
Each of these represents something you need to know about nonlinear systems to be able to effectively analyze them.
In addition, I am not allowed to collect any more work, according to the faculty rules, after today.
And, of course, I won't.
But, nonetheless, the final will contain material from the reading assignment G7, and I will be touching on all of these today, and 7.4, just a couple of pages of that.
You will see its connection.
It discusses the same example we are going to do today.
And then I suggest those three exercises to try to solidify what the lecture is about.
Instead of introducing the nonlinear system right away that we are going to talk about, I would like to first explain, so that it won't interrupt the presentation later, what I mean by a conversion to a first-order equation.
For that why don't we look at -- Maybe I can do it here.
Our system looks like this, dx over dt.
For once, I am not writing x prime because I want to explicitly indicate what the independent variable is.
It is a nonlinear autonomous system, so there is no t on the right-hand side.
But this is not a simple linear function, ax plus by.
It is more complicated, like you had in the predator-prey robin-earthworm problem that you worked on last night.
And the other equation will be g of (x, y).
This is the system, and explicitly it is nonlinear in general and autonomous, as I have indicated on the right-hand side.
Now, remember that the geometric picture of this was as a velocity field.
You made a velocity field out of the vectors whose components were f and g, so that is (f)i plus (g)j, but what it looked like was a plane filled up with a lot of vectors pointing different ways according to the velocity at the point was.
And side-by-side with that went the solutions.
A typical solution was x equals x of t, y equals y of t represented as a column vector.
And that is a parametric equation.
And the geometric picture of that was given by its trajectory.
When you plotted it, it was a trajectory.
It had not only the right direction at each point, in other words, but it also had to have the right velocity at each point.
In other words, sometimes the point was moving rapidly and sometimes it was moving more slowly along that path.
And the way it moved was an important part of the solution.
Now, what I plan to do is -- The conversation that I am talking about takes place by eliminating t. Now, why would one want to do that?
t, after all, is an essential part of the solution.
It is an essential part of this picture because without t you would not know how long the arrows were supposed to be.
You would still know their direction.
And, of course, it occurs in here.
Now, let's take the first step and eliminate t from the system itself.
As you see, that is very easy to do because the right-hand side has no t in it anyway and the left-hand side has the t only as the denominators of the differential quotients.
The obvious way to eliminate t is just to divide one equation by the other.
And since usually we think of not dx by dy, but dy by dx, I will vow -- What is dy over dx?
Well, if I divide this equation by that equation, dy by dt divided by dx by dt, according to the chain rule, or according to commonsense, you cancel the dt's and you get dy over dx.
And, on the right-hand side, you get g of (x, y) divided by f of (x, y).
But what is that?
That is the equation of the first day of the term.
By taking that single step, I convert this nonlinear system, which we virtually never find explicit solutions to, into an ordinary first order equation which, in fact, you also don't usually find explicit solutions to, except with our narrow blinders on.
We only get problems at the beginning of 18.03 where there will be an explicit solution to this.
I have converted the nonlinear system to a single first order equation to which I can apply the usual first order methods that include hope that I will be able to solve it.
Now, what do I lose?
Well, what happens to this picture?
If you don't have the t in it anymore then you don't have velocity, you don't have arrows with a length to them because the length gives you how fast the point is going.
Without t I don't know at any point how fast it is supposed to be going.
All I know at any point (x, y) is no longer dx over dt and dy over dt.
All I know is dy over dx.
In other words, the corresponding picture, if I eliminate t from this picture all that is left is the directions of each of these arrows.
What disappears is their length.
And, in fact, since dy over dx is purely a slope, I cannot even tell whether the point is traveling this way or that way.
It doesn't make any sense to have the point traveling anymore since there is no time in which it can do its traveling.
So that picture gets changed to the direction field.
The corresponding picture now, all you do is take each of these arrows, you snip it off to a standard length, being careful to snip off the little pointy end, and it becomes nothing but a line element.
And all it has is a slope.
What corresponds to the velocity field here is now just our slope field or our direction field.
All we know is the slope at each point.
How about the solution?
Well, if I have eliminated time, the solution is no longer a pair of parametric equations.
It is just an equation involving x and y. I can hope that the solution will be explicit, y equals y of x, but you already know from the first days of the term that sometimes it isn't.
Let's call that explicit solution.
Often you have to settle for it.
And sometimes it is not a pain.
It is something that is even better.
Settle for a solution which looks like this which is y defined implicitly as a function of x.
And what does the picture of that look like?
Well, the picture of this is now simply an integral curve.
It is not a trajectory anymore.
It is what we called an integral curve.
I am reminding you of the very first day of the term, or maybe the second day.
And all it has at each point the right slope because that is all the field is telling me now.
It only has a slope at each point.
It doesn't have any magnitude or direction.
It has the direction, but it doesn't have direction in the sense of an arrow telling me whether it is going this way or the opposite way.
That is the picture with t in it.
These are the corresponding degeneration of that.
It is a coarsening, it's a cheapening of it.
It is throwing away information.
Throw away all the information that had to do with t, and we are back in the first days of the terms with a single first order equations involving only x and y with solutions that involve only x and y with curves that are the graphs of these solutions but again have no t in them.
In effect, they are just the paths of the trajectories.
Whereas, the trajectory is the point that shows actually how the point is moving along in time faster or slower at various points.
Now, why would one want to lose information?
Well, the great gain is that this might be solvable.
Whereas, this almost certainly is not.
That is a big plus.
For example, let's illustrate it on the very simplest possible case.
I cannot illustrate it on a nonlinear system very well, or not right now because, in general, nonlinear systems are not solvable.
Let's take an easy example.
Suppose the system were a linear one, let's say this one that we have talked about before, in fact.
That is a simple linear two-by-two system.
This means the derivatives with respect to time.
Well, you know that the solutions we talked about in fact, last time, typical solutions that would be something like (x, y) equals a constant times, let's say if x is cosine t then y would be the derivative of that, which is negative sine t. Or, another one would be c2 times sine t, cosine t. And those, of course, are circles.
They are parametrized circles, so they are circles that go around, in fact, in this direction.
And they have a certain velocity to them.
Now what do I do if I follow this plan of eliminating t?
Well, if I eliminate t directly from the system, what will I get?
I will get dy by dx.
I divide this equation by that one.
-- is equal to negative x over.
Oh, well, of course, that is solvable.
You were able to solve by 18.01 methods before you ever come into this course.
By separation of variables, it is y dy equals negative x dx, which integrates to be one-half y squared equals negative one-half x squared plus a constant.
And after multiplying through by two and moving things around it becomes x squared plus y squared.
The nicest way to say it is implicitly, x squared plus y squared equals some positive constant.
These are the circles.
Now, can I eliminate t?
I could also take one of these solutions and eliminate t. If I square this, the best way to do it is not to use our cosines and arcsines and whatnot, which will just get you totally lost.
Square this, square that and add them together.
And you conclude that x squared plus y squared is equal to one.
Or, if c1 is not one it is equal to c1 squared.
You could eliminate t from the solution the way you eliminate t from a pair of parametric equations, and you get it the same way that x squared plus y squared, in this case, would be c1 squared, I guess.
In some sense, what we are doing is cycling around to the beginning of the term.
In fact, this whole lecture, as you will see, is about cycles of one sort or another.
But I keep thinking of that great line of poetry, "in my end is my beginning" or maybe it's "in my beginning is my end".
I think both lines occur.
But that is what is happening here.
This is the beginning of the course and this is the end of the course, and they have this almost trivial relation between them.
But notice the totally different methods used for analyzing this, where the t is included, than from analyzing this where there is no t. The goals are different, what you look for are different, everything is different, and yet it is almost the same problem.
I promised you a nonlinear example.
I guess it is time to see what that is.
It is going to be another predator-prey equation, but one which is, in some ways, simpler than the one I gave you for homework.
The predator is going to be x because you are used to that from the robins.
I am keeping the same predator-prey variables.
And the prey will be y.
And now just notice the small difference from what I gave you before.
For one thing, I am not giving you specific numbers.
I am going to, at the beginning at least, do some of the analysis at the beginning using letters, using parameters.
a, b, c, d are always going to be positive constants.
I am going to assume that x prime equals minus ax.
This represents the predator dying out if there is no prey there, but if there is something to eat that term represents the predator meeting up with the prey and gobbling it up.
And b is the coefficient.
How about without predictors, the prey will multiply and be fruitful.
Unfortunately they get eaten, and so there will be a term that looks like this.
Now, there are two basic differences between these simple equations and the slightly more complicated ones you had with the robin-earthworm equations.
Namely, in the first place, I am assuming that these guys, let's give them names.
I want you to remember which is x and which is y, so we are going to think of these are "sharx".
[LAUGHTER] And these will be food fish, so let's make them "yumfish".
The fact that with your robins you had a positive term here.
I made this 2x.
And now it is minus a times x.
What is the difference?
The difference is that robins have other things to eat, so even if there are not any worms, a robin will survive.
It could eat other insects, grubs, Japanese beetle grubs.
I think it can even eat seeds.
Anyway, the robins in my garden seem to be pecking at things that don't seem to be insects.
And the other is that I am assuming a very naīve growth law.
For example, if there are no sharks, how are the food fish fruitful and multiplying?
They multiple exponentially.
Now, obviously you cannot have unlimited growth like that.
With the worms we added the term minus a constant times y squared to indicate that even worms cannot multiply purely exponentially forever, but ultimately their growth levels off because they cannot find enough organic matter to plow their way through.
Those are the two differences.
I am not assuming a logistic growth law.
This is less sophisticated than the one I gave you for homework.
This assumes that sharks have absolutely nothing to eat except these fish, which is not so bad.
That's true, more or less.
My plan is now, with this model, let's start the analysis, as you learned to do it.
And we are going to run into trouble at various places.
And the troubles will then illustrate these three points that I wanted to add to your repertoire of things to do with nonlinear systems when you run into trouble.
And it will also increase your understanding of the nonlinear systems.
The first thing we have to do is find the critical points.
I am going to assume that you are good at this and, therefore, not spend a lot of time detailing the calculations.
I will simply write them on the board.
The first equation I am going to write down is x times minus a plus by equals zero.
And I assume you know why I am writing that down.
And the other equation will be y times c minus dx is zero.
Those are the simultaneous equations I have to find to find the critical point.
The first one is if the product of these is zero, either x is zero or the other factor is.
So, from the second equation, if x is zero, y has to be zero also.
That is one critical point.
Now, if x is not zero then this factor has to be zero which says that y must be equal to a over b.
And if y is equal to a over b, it is not zero here.
Therefore, this factor must be zero which says that x is equal to c over d. You see right away that this must be a simpler system because it is only producing two critical points.
Whereas, the system that you did for homework had four.
Here is one.
Well, let's just write them up here.
The critical points are zero, zero.
That doesn't look terribly interesting, but the other one looks more interesting.
It is c over d, a over b.
Well, let's take a look first at the zero, zero critical point.
The origin, in other words.
What does that look like?
Well, at the origin, the linearization is extremely easy to do because I simply ignore the quadratic terms, which are the product of two small numbers, where these only have a single small number in it.
It is minus ax, cy.
In other words, the linearization matrix is minus a, zero, zero and cy gives me a coefficient c there.
Now, I don't think at any point I have ever explicitly told you that I hope you have learned from the homework or maybe your recitation teacher, but for heaven's sake, put this down in your little books, if you have a diagonal matrix, for god's sake, don't calculate its eigenvalues.
They are right in front of you.
They are always the diagonal elements.
The eigenvalue, you can check this out if you insist on writing the equation, but trust me it is clear.
If I, for example, subtract c from the main diagonal, I am going to get a determinant zero because the bottom row will be all zero.
The eigenvalues are negative a and c. In other words, they have opposite signs.
This is a negative number.
That is a positive number -- -- because a, b, c and d are always positive.
And, therefore, this is automatically a saddle.
You don't have to calculate anything.
It is all right in front of you.
It must be a saddle and, therefore, unstable because all saddles are.
And, in fact, you can even draw the little picture of what the stuff looks like near the origin without even bothering to calculate eigenvectors, although it is extremely easy to do.
Just from common sense, these are the sharks and these are the yumfish.
Well, if there are zero yumfish, in other words, if I am on the sharks axis, the axis of sharks, I die out.
Well, forget about this side.
It's on the positive side.
That makes sense.
But I die out because the sharks have nothing to eat.
Whereas, if I am on the yumfish axis I go this way.
I grow because, without any sharks to eat them, the yumfish increase.
So it must look like that.
And, therefore, the saddle must look like this.
The saddle curves hug those and go nearby.
Now, the other critical point is the interesting one.
And for that the analysis, in order that you don't spend the rest of this period writing a's, b's, c's and d's, I am going to make the simplifying assumption.
But it doesn't change qualitatively any of the mathematics.
It just makes it a little easier to write everything out.
I am going to assume that everything is one.
Well, in fact, even this would be good enough, but let's make everything one.
I am going to assume this.
And it is only to make the writing simpler.
It doesn't really change the mathematics at all.
Well, if everything is one, in order to calculate the linearized system, I'd better use the Jacobian.
But perhaps it would be better to write out explicitly what the system is.
The system now is x prime equals minus x plus xy, all the coefficients are one.
And y prime is equal to y minus xy.
What is the Jacobian?
Well, the Jacobian is the partial of this with respect to x which is minus one plus y, the partial with respect to y which is x, the partial of this with respect to x, which is minus y, and the partial with respect to y, which is one minus x.
But I want to evaluate that at the point one, one, which is the critical point.
It is the critical point because I have assumed that all these parameters have the value one.
And when I do that, evaluating it at one, one, I get what matrix?
Well, I get zero, one, negative one, zero.
In other words, the linearized system is x prime equals y and y prime equals negative x.
Well, that is just the one we analyzed before in terms of -- It's the one whose solutions are circles.
In other words, what we find out is that the linearized system is what geometric type?
Saddle?
Node?
Spiral?
None of those.
It is a center.
The linearized system is a center.
It consists, in other words, of a bunch of curves going round and round next to each other.
Concentric circles, in fact.
Well, what is wrong with that?
Now, we are in deep trouble.
We are now in trouble because that is a borderline case.
Let me remind you of what the borderline cases are.
When we drew that picture, this is from last week's problem set so you have a perfect excuse for forgetting it totally.
But I am trying to remind you of it.
That is another reason why I am doing this.
Remember that picture I asked you about that appeared on the computer screen?
Let's make it a little flatter so that I can have room to write in.
This is a certain parabola whose equation you are dying to tell me, but I am not asking.
There is the trace, this is the determinant, and the characteristic equation is related to the values of these numbers like plus d equals zero.
Then these were spiral synchs.
This was the region of spiral sources.
That is the abbreviation for sources.
These were nodal synchs and these were the nodal sources.
And down here were the saddles.
And where were the centers?
The centers were along this line.
The centers, there weren't many of them, and they were the separation for these two regions.
I will put that down.
These are the centers.
These other borderlines correspond to other things.
These are defective eigenvalues, zero eigenvalues and so on.
But let's concentrate on just one of them.
You will find the others discussed in the notes, GS7.
But if you get this idea then the rest is just details.
I think it will be perfectly clear.
What is wrong with the center?
The answer is that if we are on the center, for example, this system corresponds to the trace being zero and the determinant being plus one.
It corresponds to t equals zero and d equals one.
This point, in other words.
Now, if I wiggle the coefficients of the matrix just a little bit, just change them a little bit, what I am going to do is move off this pink line.
And I might move to here or I might move a little way to there.
But, if I do that, I change the geometric type.
In other words, being a borderline, a slight change of the parameters can change what it changes to.
And, in fact, that is geometrically clear.
What does a center look like?
A center looks like this, a bunch of curves going around all in the same direction, like concentric circles or maybe ellipses or something like that.
If I deform the picture just a little bit, well, I might change it into something that looks like this where, after they go around they don't quite meet up with where they were to start with.
And I am going to get a spiral synch.
Or, I might do the deformation by going around once and I'm a little outside of where I was.
In which case it's going to be a spiral source.
The fact that just changing these curves a little bit can change the picture to this or to that corresponds to the fact that if you are on here with this value of t and d, zero and one, and just change t and d a little bit, you are going to move off into these regions.
You might, of course, stay on the pink line.
It is not very likely.
Where does this leave us?
Well, if the linear system is not stable in the sense that if you change the parameters a little bit it doesn't change the type.
This is, after all, only an approximation to the nonlinear system.
If in this approximation you cannot really predict the behavior of very well when you change the parameters, then from it you cannot tell what the original system looked like.
In other words, the nonlinear system at one, one might be any one of the possibilities, still a center or it might be a spiral synch or it might be a spiral source.
Any one of those three is a possibility.
It couldn't be a saddle because that is too far away.
It couldn't be a node.
That is too far away, too.
But it could wander into either of these regions and, therefore, the picture you cannot tell which of these three it is just from this type of critical point analysis.
Well, that was Volterra's problem.
By the way, the person who introduced these equations and studied them systematically in the way in which we are doing it here was Volterra.
And, in fact, he was interested in sharks and food fish, as they were called.
What do we do?
Well, you have to be smart.
What Volterra did was he went to method number one.
Let's do it.
Volterra said I got my equations x prime equals minus x plus xy.
y prime is equal to, these are the food fish, y minus xy.
Let's eliminate t. And my problem is, of course, I am trying to determine what type of critical point one, one is.
And the method we have used up until now has failed because it gave us a borderline case, which is from that we cannot deduce.
He said let's eliminate t. And we get dy over dx equals, I am going to factor, y times one minus x on top.
And on the bottom factor x negative one plus y.
You could solve that before you came into 18.03, right?
This you can separate variables.
Let's separate variables.
The y's all go on one side, so y goes down here.
It is (y minus one) over y dy equals -- On the other side the dx gets moved up, one minus x over x dx.
Now, of course, you wouldn't dream of using partial fractions on this.
It would be illegal because, even though it is a rational function, the quotient of two polynomials, the degree of the top is not smaller than the degree of the bottom.
In other words, it is a partial fraction so this degree must be bigger than that one.
And it isn't so you would have to first divide.
And if you divide by that then there is no point in using partial fractions at all.
Of course, you would have done this by basic instant, I know.
What is the solution?
It is y minus log y equals log x minus x plus some constant.
That is not the final constant I want so I will give it a subscript one to indicate I want more.
This is very hard to see what that curve looks like.
We can make it look better by exponentiating.
If I exponentiate it, going back to high school mathematics, but I know from experience that many of you are not too good at this step, so that is another reason I am doing it, it will be e to the y, times, that part is easy because pluses and minuses change into times, times e to the negative log y.
Well, e to the log y is y.
If I put a minus sign in front of that, that sends it into the denominator, so instead of being y it is one over y Equals x, e to log x is x, e to the negative x is, therefore, times one over e to the x.
And that is times the exponential of c1, which I will call c2.
And now if I combine them all and put them all on one side it is x over e to the x times y over e to the y is equal to yet another constant, one over c2.
This is my final constant so let's call that c. In other words, the integral curves are the graphs of this equation for different values of the constant c. Well, of course you've graphed an equation like that all the time.
What am I going to do with this stupid thing?
Well, I deliberately picked something which involved all your learning up until now.
We are now going into 18.02, right?
You looked at that in 18.02 you would say this is the contour curve, these are contour curves of the function -- Well, let's write it out the other way.
It doesn't make any difference, but you are more likely to have seen the function in this form.
x over e to the x you won't recognize, but this you will.
In fact, we have had it before this term.
It is one of the kinds of solutions you can add to second order equations.
Times y e to the negative y equals c. It is the contour curves of this function.
Let's call that function, let's say, h of (x,y).
Well, of course you could throw it on Matlab, as you did in 18.02 maybe, and ask Matlab to plot the function.
But Volterra didn't have that luxury.
He was smart, too.
So let's be smart instead.
What is the function x?
Let's use a neutral variable like u and plot u e to the negative u.
18.01.
At zero, it has the value zero.
When u is small this has approximately the value one and, therefore, it starts out like the function u.
As u goes to infinity, you know by L'Hopital's rule that this goes to zero so it ends up like this.
Well, what on earth could it possibly do except rise to a maximum?
But where is that maximum?
It is easy to see there is a unique maximum just by 18.01.
Take the derivative, find out where the maximum point is and you will find, perhaps you have already done it, this is at one.
The maximum occurs at one, that is where the derivative is zero and the value there is one times e to the minus one.
It is one over e, in other words.
That is what the curve looks like.
What do the contour curves of this look like?
Well, if we are looking at the contour curves of the function h, so here is x, here is y, and we want the contour curves of the function h of (x, y), what do I know?
Where is its maximum?
The maximum point of h of x, y is where?
Well, h is the product x e to the negative x times y e to the minus y.
This has its maximum at one, this has its maximum at one, so where does h of (x, y) head?
Well, you have two factors.
One makes each of them the biggest they can be.
The maximum must be exactly at that critical point one, one.
That is our maximum point.
Let's make it conspicuous.
This is the maximum point.
It is the point one, one.
It is the point that makes the function biggest.
Now, how about the contour curves?
Well, this is the top of the mountain.
What is it along the axes?
Along the axes, when either x or y is zero, the function has the value zero, so it is biggest there.
It has the value zero here.
So it is zero value.
What are the contour curves?
Well, we must have a mountain peak here.
This is the unique maximum point.
In fact, it is easy to see the contour curves just surround it like that.
Now, the reason they cannot be spirals, well, in the first place, you never see contour curves that were spirals.
That is a good reason.
It is a mountain.
That is the way contour curves of a mountain look.
But, all right, that is not a good argument.
But notice that along each horizontal line here, I want to know how many times can it intersect the contour curve?
That is the same as asking along one of these lines how many times could it intersect?
See, this is a graph of the function along this horizontal line.
And, therefore, how many times can it intersect one of the contour curves the same number of times that a horizontal line can intersect this curve?
Twice.
Only once if you are exactly at that height and after that never.
But here it can intersect each contour curve only twice which means these things cannot be spirals.
Otherwise, they would intersect those horizontal and vertical lines many times instead of just twice.
That is what it looks like.
In fact, we can even put in the direction without any effort.
We know that the sharks die out without any food fish and the food fish grow.
Well, I think this has to be clockwise.
Of course, the guys nearby must be going the same way.
And, therefore, the guys near them must be going the same way.
And it is sort of a domino effect.
The direction spreads and there is only one compatible way of making these.
They all must be going clockwise.
What is happening, actually?
At this point there are a lot of sharks but not so many food fish.
The sharks eat the few remaining food fish, and now the sharks start to die out because they have nothing to eat.
When they are practically gone, the food fish can start growing again and grow and grow.
For a while they grow happily, and then the sharks suddenly start being able to find them again.
And the few remaining sharks start eating them, the population of sharks grows, the food fish start dying out again, and the cycle starts all over again.
Now, I wanted to talk about the qualitative behavior.
This is, in some ways, the most important part of the lecture.
I wanted to discuss, just as we did at the beginning of the term, the effect of fishing with nets at a constant rate k. Just as near the beginning of the term, we talked about fishing a single population.
The only difference is, now there are two populations.
Both sharks and the food fish.
Now, what happens to the equations?
Well, the equations start out just as they were.
I am now reverting to a and b.
You will see why.
It's because we need the general coefficients.
Plus bxy.
But, if we are fishing with nets, then a certain fraction of the sharks in the sea get hauled in by the net.
And say the fishing rate is k. There is a fishing term.
We remove minus k times a certain fraction of the sharks in the sea.
How about the food fish?
Well, we remove them, too.
And we don't distinguish between sharks and food fish.
This is cy minus dxy.
But we also remove a certain fraction of them.
Minus k times y.
What, therefore, is the new system?
The new system is therefore x prime equals minus a plus k. I am combining those two x terms.
That is times x.
Plus bxy.
And the y prime is c minus k times y minus dxy.
To solve these, I do not have to go through the analysis.
All I have to do is change the numbers, change these coefficients from a to a plus k and c minus k. The old critical point was the point c over d, a over b.
The new critical point with fishing is what?
Well, the parameters have been changed by the addition of k. The new critical point is c minus k over d and a plus k over d. What is the effect of fishing?
Well, if the old critical point was over here, let's say this is the point c over d and this is the point a over b, that is the old critical point that was over here.
The result of fishing is to lower the value of x and raise the value here.
The new critical point is, this gets lowered to c minus k over d, and this gets raised to a plus k over b.
In other words, the new point is there.
The effect of fishing is to -- It does not treat the sharks and the yumfish equally.
The effect of fishing lowers the shark population.
See, the critical point gives sort of the average shark population.
Of course, it cycles around these.
But, on the average, this gives how many sharks there are and how many food fish there are.
The new critical point with fishing lowers the shark population and raises the food fish population.
That is not intuitive.
And, in fact, that was observed experimentally at a slightly different context.
And that is why Volterra started working on the problem.
I will need three more minutes.
The most interesting application of all is not to sharks and food fish.
I cannot assume that you are dramatically interested in how many of them there are in the ocean, but you might be more interested in this.
That thing about lowering is called Volterra's principle.
Put that in your books.
Volterra's principle.
Volterra is spelt over there.
-- has found more modern applications than sharks.
Suppose you consider fish in a pond.
You have mosquito larvae which breed in the pond.
This happens.
And then suddenly there is a plague of mosquitoes and concerned citizens.
And this is what happened in the '50s.
That was before you were born, but not before I was.
What happened was a lot of mosquitoes.
Everybody said the mosquitoes breed in the little stagnant ponds so spray DDT on them.
DDT them.
Dump it in the ponds.
That will kill all the larvae and we won't get bitten anymore.
When you put DDT in the pond, as people did not realize at the time because these things were new, of course you kill the mosquitoes, but you also kill the fish because DDT is poisonous to fish.
What, in effect, you are doing mathematically is the same as harvesting the fish population with the sharks and the food fish.
The result is, the fish are the predators because they eat the mosquito larvae, big food, there was a certain equilibrium, according to Volterra's principle.
With DDT that equilibrium moves to here.
In other words, the effect of indiscriminately spraying the pond with DDT is to increase the number of mosquitoes and kill fish.
And, in fact, that is exactly what was observed.
The same thing was observed with the bird population and insects.
Spraying trees for insects to get rid of some pests ends up killing more birds than it does insects and the insects increase.
Thanks.
Hi.
This is the first lecture in MIT's course 18.06, linear algebra, and I'm Gilbert Strang.
The text for the course is this book, Introduction to Linear Algebra.
And the course web page, which has got a lot of exercises from the past, MatLab codes, the syllabus for the course, is web.mit.edu/18.06.
And this is the first lecture, lecture one.
So, and later we'll give the web address for viewing these, videotapes.
Okay, so what's in the first lecture?
This is my plan.
The fundamental problem of linear algebra, which is to solve a system of linear equations.
So let's start with a case when we have some number of equations, say n equations and n unknowns.
So an equal number of equations and unknowns.
That's the normal, nice case.
And what I want to do is -- with examples, of course -- to describe, first, what I call the Row picture.
That's the picture of one equation at a time.
It's the picture you've seen before in two by two equations where lines meet.
So in a minute, you'll see lines meeting.
The second picture, I'll put a star beside that, because that's such an important one.
And maybe new to you is the picture -- a column at a time.
And those are the rows and columns of a matrix.
So the third -- the algebra way to look at the problem is the matrix form and using a matrix that I'll call A.
Okay, so can I do an example?
The whole semester will be examples and then see what's going on with the example.
So, take an example.
Two equations, two unknowns.
So let me take 2x -y =0, let's say.
And -x 2y=3.
Okay.
let me -- I can even say right away -- what's the matrix, that is, what's the coefficient matrix?
The matrix that involves these numbers -- a matrix is just a rectangular array of numbers.
Here it's two rows and two columns, so 2 and -- minus 1 in the first row minus 1 and 2 in the second row, that's the matrix.
And the right-hand -- the, unknown -- well, we've got two unknowns.
So we've got a vector, with two components, x and y, and we've got two right-hand sides that go into a vector 0 3.
I couldn't resist writing the matrix form, right -- even before the pictures.
So I always will think of this as the matrix A, the matrix of coefficients, then there's a vector of unknowns.
Here we've only got two unknowns.
Later we'll have any number of unknowns.
And that vector of unknowns, well I'll often -- I'll make that x -- extra bold.
A and the right-hand side is also a vector that I'll always call b.
So linear equations are A x equal b and the idea now is to solve this particular example and then step back to see the bigger picture.
Okay, what's the picture for this example, the Row picture?
Okay, so here comes the Row picture.
So that means I take one row at a time and I'm drawing here the xy plane and I'm going to plot all the points that satisfy that first equation.
So I'm looking at all the points that satisfy 2x-y =0.
It's often good to start with which point on the horizontal line -- on this horizontal line, y is zero.
The x axis has y as zero and that -- in this case, actually, then x is zero.
So the point, the origin -- the point with coordinates (0,0) is on the line.
It solves that equation.
Okay, tell me in -- well, I guess I have to tell you another point that solves this same equation.
Let me suppose x is one, so I'll take x to be one.
Then y should be two, right?
So there's the point one two that also solves this equation.
And I could put in more points.
But, but let me put in all the points at once, because they all lie on a straight line.
This is a linear equation and that word linear got the letters Okay, thanks.
for line in it.
That's the equation -- this is the line that ... of solutions to 2x-y=0 my first row, first equation.
So typically, maybe, x equal a half, y equal one will work.
And sure enough it does.
Okay, that's the first one.
Now the second one is not going to go through the origin.
It's always important.
Do we go through the origin or not?
In this case, yes, because there's a zero over there.
In this case we don't go through the origin, because if x and y are zero, we don't get three.
So, let me again say suppose y is zero, what x do we actually get?
If y is zero, then I get x is minus three.
So if y is zero, I go along minus three.
So there's one point on this second line.
Now let me say, well, suppose x is minus one -- just to take another x.
If x is minus one, then this is a one and I think y should be a one, because if x is minus one, then I think y should be a one and we'll get that point.
Is that right?
If x is minus one, that's a one.
If y is a one, that's a two and the one and the two make three and that point's on the equation.
Okay.
Now, I should just draw the line, right, connecting those two points at -- that will give me the whole line.
And if I've done this reasonably well, I think it's going to happen to go through -- well, not happen -- it was arranged to go through that point.
So I think that the second line is this one, and this is the all-important point that lies on both lines.
Shall we just check that that point which is the point x equal one and y was two, right?
That's the point there and that, I believe, solves both equations.
Let's just check this.
If x is one, I have a minus one plus four equals three, okay.
Apologies for drawing this picture that you've seen before.
But this -- seeing the row picture -- first of all, for n equal 2, two equations and two unknowns, it's the right place to start.
Okay.
So we've got the solution.
The point that lies on both lines.
Now can I come to the column picture?
Pay attention, this is the key point.
So the column picture.
I'm now going to look at the columns of the matrix.
I'm going to look at this part and this part.
I'm going to say that the x part is really x times -- you see, I'm putting the two -- I'm kind of getting the two equations at once -- that part and then I have a y and in the first equation it's multiplying a minus one and in the second equation a two, and on the right-hand side, zero and three.
You see, the columns of the matrix, the columns of A are here and the right-hand side b is there.
And now what is the equation asking for?
It's asking us to find -- somehow to combine that vector and this one in the right amounts to get that one.
It's asking us to find the right linear combination -- this is called a linear combination.
And it's the most fundamental operation in the whole course.
It's a linear combination of the columns.
That's what we're seeing on the left side.
Again, I don't want to write down a big definition.
You can see what it is.
There's column one, there's column two.
I multiply by some numbers and I add.
That's a combination -- a linear combination and I want to make those numbers the right numbers to produce zero three.
Okay.
Now I want to draw a picture that, represents what this -- this is algebra.
What's the geometry, what's the picture that goes with it?
Okay.
So again, these vectors have two components, so I better draw a picture like that.
So can I put down these columns?
I'll draw these columns as they are, and then I'll do a combination of them.
So the first column is over two and down one, right?
So there's the first column.
The first column.
Column one.
It's the vector two minus one.
The second column is -- minus one is the first component and up two.
It's here.
There's column two.
So this, again, you see what its components are.
Its components are minus one, two.
Good.
That's this guy.
Now I have to take a combination.
What combination shall I take?
Why not the right combination, what the hell?
Okay.
So the combination I'm going to take is the right one to produce zero three and then we'll see it happen in the picture.
So the right combination is to take x as one of those and two of these.
It's because we already know that that's the right x and y, so why not take the correct combination here and see it happen?
Okay, so how do I picture this linear combination?
So I start with this vector that's already here -- so that's one of column one, that's one times column one, right there.
And now I want to add on -- so I'm going to hook the next vector onto the front of the arrow will start the next vector and it will go this way.
So let's see, can I do it right?
If I added on one of these vectors, it would go left one and up two, so we'd go left one and up two, so it would probably get us to there.
Maybe I'll do dotted line for that.
Okay?
That's one of column two tucked onto the end, but I wanted to tuck on two of column two.
So that -- the second one -- we'll go up left one and up two also.
It'll probably end there.
And there's another one.
So what I've put in here is two of column two.
Added on.
And where did I end up?
What are the coordinates of this result?
What do I get when I take one of this plus two of that?
I do get that, of course.
There it is, x is zero, y is three, that's b.
That's the answer we wanted.
And how do I do it?
You see I do it just like the first component.
I have a two and a minus two that produces a zero, and in the second component I have a minus one and a four, they combine to give the three.
But look at this picture.
So here's our key picture.
I combine this column and this column to get this guy.
That was the b.
That's the zero three.
Okay.
So that idea of linear combination is crucial, and also -- do we want to think about this question?
Sure, why not.
What are all the combinations?
If I took -- can I go back to xs and ys?
This is a question for really -- it's going to come up over and over, but why don't we see it once now?
If I took all the xs and all the ys, all the combinations, what would be all the results?
And, actually, the result would be that I could get any right-hand side at all.
The combinations of this and this would fill the whole plane.
You can tuck that away.
We'll, explore it further.
But this idea of what linear combination gives b and what do all the linear combinations give, what are all the possible, achievable right-hand sides be -- that's going to be basic.
Okay.
Can I move to three equations and three unknowns?
Because it's easy to picture the two by two case.
Let me do a three by three example.
Okay, I'll sort of start it the same way, say maybe 2x-y and maybe I'll take no zs as a zero and maybe a -x 2y and maybe a -z as a -- oh, let me make that a minus one and, just for variety let me take, -3z, -3ys, I should keep the ys in that line, and 4zs is, say, 4.
Okay.
That's three equations.
I'm in three dimensions, x, y, z.
And, I don't have a solution yet.
So I want to understand the equations and then solve them.
Okay.
So how do I you understand them?
The row picture one way.
The column picture is another very important way.
Just let's remember the matrix form, here, because that's easy.
The matrix form -- what's our matrix A?
Our matrix A is this right-hand side, the two and the minus one and the zero from the first row, the minus one and the two and the minus one from the second row, the zero, the minus three and the four from the third row.
So it's a three by three matrix.
Three equations, three unknowns.
And what's our right-hand side?
Of course, it's the vector, zero minus one, four.
Okay.
So that's the way, well, that's the short-hand to write out the three equations.
But it's the picture that I'm looking for today.
Okay, so the row picture.
All right, so I'm in three dimensions, x, find out when there isn't a solution.
y and z.
And I want to take those equations one at a time and ask -- and make a picture of all the points that satisfy -- let's take equation number two.
If I make a picture of all the points that satisfy -- all the x, y, z points that solve this equation -- well, first of all, the origin is not one of them.
x, y, z -- it being 0, 0, 0 would not solve that equation.
So what are some points that do solve the equation?
Let's see, maybe if x is one, y and z could be zero.
That would work, right?
So there's one point.
I'm looking at this second equation, here, just, to start with.
Let's see.
Also, I guess, if z could be one, x and y could be zero, so that would just go straight up that axis.
And, probably I'd want a third point here.
Let me take x to be zero, z to be zero, then y would be minus a half, right?
So there's a third point, somewhere -- oh my -- okay.
Let's see.
I want to put in all the points that satisfy that equation.
Do you know what that bunch of points will be?
It's a plane.
If we have a linear equation, then, fortunately, the graph of the thing, the plot of all the points that solve it are a plane.
These three points determine a plane, but your lecturer is not Rembrandt and the art is going to be the weak point here.
So I'm just going to draw a plane, right?
There's a plane somewhere.
That's my plane.
That plane is all the points that solves this guy.
Then, what about this one?
Two x minus y plus zero z.
So z actually can be anything.
Again, it's going to be another plane.
Each row in a three by three problem gives us a plane in three dimensions.
So this one is going to be some other plane -- maybe I'll try to draw it like this.
And those two planes meet in a line.
So if I have two equations, just the first two equations in three dimensions, those give me a line.
The line where those two planes meet.
And now, the third guy is a third plane.
And it goes somewhere.
Okay, those three things meet in a point.
Now I don't know where that point is, frankly.
But -- linear algebra will find it.
The main point is that the three planes, because they're not parallel, they're not special.
They do meet in one point and that's the solution.
But, maybe you can see that this row picture is getting a little hard to see.
The row picture was a cinch when we looked at two lines meeting.
When we look at three planes meeting, it's not so clear and in four dimensions probably a little less clear.
So, can I quit on the row picture?
Or quit on the row picture before I've successfully found the point where the three planes meet?
All I really want to see is that the row picture consists of three planes and, if everything works right, three planes meet in one point and that's a solution.
Now, you can tell I prefer the column picture.
Okay, so let me take the column picture.
That's x times -- so there were two xs in the first equation minus one x is, and no xs in the third.
It's just the first column of that.
And how many ys are there?
There's minus one in the first equations, two in the second and maybe minus three in the third.
Just the second column of my matrix.
And z times no zs minus one zs and four zs.
And it's those three columns, right, that I have to combine to produce the right-hand side, which is zero minus one four.
Okay.
So what have we got on this left-hand side?
A linear combination.
It's a linear combination now of three vectors, and they happen to be -- each one is a three dimensional vector, so we want to know what combination of those three vectors produces that one.
Shall I try to draw the column picture, then?
So, since these vectors have three components -- so it's some multiple -- let me draw in the first column as before -- x is two and y is minus one.
Maybe there is the first column.
y -- the second column has maybe a minus one and a two and the y is a minus three, somewhere, there possibly, column two.
And the third column has -- no zero minus one four, so how shall I draw that?
So this was the first component.
The second component was a minus one.
Maybe up here.
That's column three, that's the column zero minus one and four.
This guy.
So, again, what's my problem?
What this equation is asking me to do is to combine these three vectors with a right combination to produce this one.
Well, you can see what the right combination is, because in this special problem, specially chosen by the lecturer, that right-hand side that I'm trying to get is actually one of these columns.
So I know how to get that one.
So what's the solution?
What combination will work?
I just want one of these and none of these.
So x should be zero, y should be zero and z should be one.
That's the combination.
One of those is obviously the right one.
Column three is actually the same as b in this particular problem.
I made it work that way just so we would get an answer, (0,0,1), so somehow that's the point where those three planes met and I couldn't see it before.
Of course, I won't always be able to see it from the column picture, either.
It's the next lecture, actually, which is about elimination, which is the systematic way that everybody -- every bit of software, too -- production, large-scale software would solve the equations.
So the lecture that's coming up.
If I was to add that to the syllabus, will be about how to find x, y, z in all cases.
Can I just think again, though, about the big picture?
By the big picture I mean let's keep this same matrix on the left but imagine that we have a different right-hand side.
Oh, let me take a different right-hand side.
So I'll change that right-hand side to something that actually is also pretty special.
Let me change it to -- if I add those first two columns, that would give me a one and a one and a minus three.
There's a very special right-hand side.
I just cooked it up by adding this one to this one.
Now, what's the solution with this new right-hand side?
The solution with this new right-hand side is clear.
took one of these and none of those.
So actually, it just changed around to this when I took this new right-hand side.
Okay.
So in the row picture, I have three different planes, three new planes meeting now at this point.
In the column picture, I have the same three columns, but now I'm combining them to produce this guy, and it turned out that column one plus column two which would be somewhere -- there is the right column -- one of this and one of this would give me the new b.
Okay.
So we squeezed in an extra example.
But now think about all bs, all right-hand sides.
Can I solve these equations for every right-hand side?
Can I ask that question?
So that's the algebra question.
Can I solve A x=b for every b?
Let me write that down.
Can I solve A x =b for every right-hand side b?
I mean, is there a solution?
And then, if there is, elimination will give me a way to find it.
I really wanted to ask, is there a solution for every right-hand side?
So now, can I put that in different words -- in this linear combination words?
So in linear combination words, do the linear combinations of the columns fill three dimensional space?
Every b means all the bs in three dimensional space.
Do you see that I'm just asking the same question in different words?
Solving A x -- A x -- that's very important.
A times x -- when I multiply a matrix by a vector, I get a combination of the columns.
I'll write that down in a moment.
But in my column picture, that's really what I'm doing.
I'm taking linear combinations of these three columns and I'm trying to find b.
And, actually, the answer for this matrix will be yes.
For this matrix A -- for these columns, the answer is yes.
This matrix -- that I chose for an example is a good matrix.
A non-singular matrix.
An invertible matrix.
Those will be the matrices that we like best.
There could be other -- and we will see other matrices where the answer becomes, no -- oh, actually, you can see when it would become no.
What could go wrong?
find out -- because if elimination fails, How could it go wrong that out of these -- out of three columns and all their combinations -- when would I not be able to produce some b off here?
When could it go wrong?
Do you see that the combinations -- let me say when it goes wrong.
If these three columns all lie in the same plane, then their combinations will lie in that same plane.
So then we're in trouble.
If the three columns of my matrix -- if those three vectors happen to lie in the same plane -- for example, if column three is just the sum of column one and column two, I would be in trouble.
That would be a matrix A where the answer would be no, because the combinations -- if column three is in the same plane as column one and two, I don't get anything new from that.
All the combinations are in the plane and only right-hand sides b that I could get would be the ones in that plane.
So I could solve it for some right-hand sides, when b is in the plane, but most right-hand sides would be out of the plane and unreachable.
So that would be a singular case.
The matrix would be not invertible.
There would not be a solution for every b.
The answer would become no for that.
Okay.
I don't know -- shall we take just a little shot at thinking about nine dimensions?
Imagine that we have vectors with nine components.
Well, it's going to be hard to visualize those.
I don't pretend to do it.
But somehow, pretend you do.
Pretend we have -- if this was nine equations and nine unknowns, then we would have nine columns, and each one would be a vector in nine-dimensional space and we would be looking at their linear combinations.
So we would be having the linear combinations of nine vectors in nine-dimensional space, and we would be trying to find the combination that hit the correct right-hand side b.
And we might also ask the question can we always do it?
Can we get every right-hand side b?
And certainly it will depend on those nine columns.
Sometimes the answer will be yes -- if I picked a random matrix, it would be yes, actually.
If I used MatLab and just used the random command, picked out a nine by nine matrix, I guarantee it would be good.
It would be non-singular, it would be invertible, all beautiful.
But if I choose those columns so that they're not independent, so that the ninth column is the same as the eighth column, then it contributes nothing new and there would be right-hand sides b that I couldn't get.
Can you sort of think about nine vectors in nine-dimensional space an take their combinations?
That's really the central thought -- that you get kind of used to in linear algebra.
Even though you can't really visualize it, you sort of think you can after a while.
Those nine columns and all their combinations may very well fill out the whole nine-dimensional space.
But if the ninth column happened to be the same as the eighth column and gave nothing new, then probably what it would fill out would be -- I hesitate even to say this -- it would be a sort of a plane -- an eight dimensional plane inside nine-dimensional space.
And it's those eight dimensional planes inside nine-dimensional space that we have to work with eventually.
For now, let's stay with a nice case where the matrices work, we can get every right-hand side b and here we see how to do it with columns.
Okay.
There was one step which I realized I was saying in words that I now want to write in letters.
Because I'm coming back to the matrix form of the equation, so let me write it here.
The matrix form of my equation, of my system is some matrix A times some vector x equals some right-hand side b.
Okay.
So this is a multiplication.
A times x. Matrix times vector, and I just want to say how do you multiply a matrix by a vector?
Okay, so I'm just going to create a matrix -- let me take two five one three -- and let me take a vector x to be, say, 1and 2.
How do I multiply a matrix by a vector?
But just think a little bit about matrix notation and how to do that in multiplication.
So let me say how I multiply a matrix by a vector.
Actually, there are two ways to do it.
Let me tell you my favorite way.
It's columns again.
It's a column at a time.
For me, this matrix multiplication says I take one of that column and two of that column and add.
So this is the way I would think of it is one of the first column and two of the second column and let's just see what we get.
So in the first component I'm getting a two and a ten.
I'm getting a twelve there.
In the second component I'm getting a one and a six, I'm getting a seven.
So that matrix times that vector is twelve seven.
Now, you could do that another way.
You could do it a row at a time.
And you would get this twelve -- and actually I pretty much did it here -- this way.
Two -- I could take that row times my vector.
This is the idea of a dot product.
This vector times this vector, two times one plus five times two is the twelve.
This vector times this vector -- one times one plus three times two is the seven.
So I can do it by rows, and in each row times my x is what I'll later call a dot product.
But I also like to see it by columns.
I see this as a linear combination of a column.
So here's my point.
A times x is a combination of the columns of A.
That's how I hope you will think of A times x when we need it.
Right now we've got -- with small ones, we can always do it in different ways, but later, think of it that way.
Okay.
So that's the picture for a two by two system.
And if the right-hand side B happened to be twelve seven, then of course the correct solution would be one two.
Okay.
So let me come back next time to a systematic way, using elimination, to find the solution, if there is one, to a system of any size and
2 times this equation, Okay.
This is it.
The second lecture in linear algebra, and I've put below my main topics for today.
I put right there a system of equations that's going to be our example to work with.
But what are we going to do with it?
We're going to solve it.
And the method of solution will not be determinants.
Determinants are something that will come later.
The method we'll use is called elimination.
And it's the way every software package solves equations.
And elimination, well, if it succeeds, it gets the answer.
And normally it does succeed.
If the matrix A that's coming into that system is a good matrix, and I think this one is, then elimination will work.
We'll get the answer in an efficient way.
But why don't we, as long as we're sort of seeing how elimination works -- it's always good to ask how could it fail?
So at the same time, we'll see how elimination decides whether the matrix is a good one or has problems.
Then to complete the answer, there's an obvious step of back substitution.
In fact, the idea of elimination is -- you would have thought of it, right?
I mean Gauss thought of it before we did, but only because he was born earlier.
It's a natural idea... and died earlier, too.
Okay, and you've seen the idea.
But now, the part that I want to show you is elimination expressed in matrix language, because the whole course -- all the key ideas get expressed as matrix operations, not as words.
And one of the operations, of course, that we'll meet is how do we multiply matrices and why?
Okay, so there's a system of equations.
Three equations and three unknowns.
And there's the matrix, the three by three matrix -- so this is the system Ax = b.
This is our system to solve, Ax equal -- and the right-hand side is that vector 2, 12, 2.
Okay.
Now, when I describe elimination -- it gets to be a pain to keep writing the equal signs and the pluses and so on.
It's that matrix that totally matters.
Everything is in that matrix.
But behind it is those equations.
So what does elimination do?
What's the first step of elimination?
We accept the first equation, it's okay.
I'm going to multiply that equation by the right number, the right multiplier and I'm going to subtract it from the second equation.
With what purpose?
So that will decide what the multiplier should be.
Our purpose is to knock out the x part of equation two.
So our purpose is to eliminate x.
So what do I multiply -- and again, I'll do it with this matrix, because I can do it short.
What's the multiplier here?
What do I multiply -- equation one and subtract.
Notice I'm saying that word subtract.
I'd like to stick to that convention.
I'll do a subtraction.
First of all this is the key number that I'm starting with.
And that's called the pivot.
I'll put a box around it and write its name down.
That's the first pivot.
The first pivot.
Okay.
So I'm going to use -- that's sort of like the key number in that equation.
And now what's the multiplier?
So I'm going to -- my first row won't change, that's the pivot row.
But I'm going to use it -- and now, finally, let me ask you what the multiplier is.
Yes?
3 times that first equation will knock out that 3.
Okay.
So what will it leave?
So the multiplier is 3.
3 times that will make that 0.
That was our purpose.
3 2s away from the 8 will leave a 2 and three 1s away from 1 will leave a minus 2.
And this guy didn't change.
Now the next step -- this is forward elimination and that Okay.
step's completed.
Oh, well, you could say wait a minute, what about the right hand side?
Shall I carry -- the right-hand side gets carried along.
Actually MatLab finishes up with the left side before -- and then just goes back to do the right side.
Maybe I'll be MatLab for a moment and do that.
Okay.
I'm leaving a room for a column of b, the right-hand side.
But I'll fill it in later.
Okay.
Now the next step of elimination is what?
Well, strictly speaking... this position that I cleaned up was like the 2, 1 position, row 2, column 1.
So I got a 0 in the 2, 1 position.
I'll use 2,1 as the index of that step.
The next step should be to finish the column and get a 0 in that position.
So the next step is really the 3,1 step, row three, column one.
But of course, I already have 0.
Okay.
So the multiplier is 0.
I take 0 of this equation away from this one and I'm all set.
So I won't repeat that, but there was a step there which, MatLab would have to look -- it would look at this number and, do that step, unless you told it in advance that it was 0.
Okay.
Now what?
Now we can see the second pivot, which is what?
The second pivot -- see, we've eliminated -- x is now gone from this equation, right?
We're down to two equations in y and z.
And so now I just do it again.
Like, everything's very cursive at this -- this is like -- such a basic algorithm and you've seen it, but carry me through one last step.
So this is still the first pivot.
Now the second pivot is this guy, who has appeared there.
And what's the multiplier, the appropriate multiplier now?
And what's my purpose?
Is it to wipe out the 3, 2 position, right?
This was the 2, 1 step.
And now I'm going to take the 3, 2 step.
So this all stays the same, 1 2 1, 0 2 -1 and the pivots are there.
Now I'm using this pivot, so what's the multiplier?
this row, gets subtracted from this row and makes that a 0.
So it's 0, 0 and is it a 5?
Yeah, I guess it's a 5, is that right?
Because I have a one there and I'm subtracting twice of twice this, so I think it's a 5 there.
There's the third pivot.
So let me put a box around all three pivots.
Is there a -- oh, did I just invent a negative one?
I'm sorry that the tape can't, correct that as easily as I can.
Okay.
Thank you very much.
You get an A in the course now.
Is that correct?
Is it correct now?
Okay.
So the three pivots are there -- I know right away a lot about this matrix.
This elimination step from A -- this matrix I'm going to call U. U for upper triangular.
So the whole purpose of elimination was to get from A to U.
And, literally, that's the most common calculation in scientific computing.
And people think of how could I do that faster?
Because it's a major, major thing.
But we're doing it the straightforward way.
We found three pivots, and by the way, I didn't say this, pivots can't be 0.
I don't accept 0 as a pivot.
And I didn't get 0.
So this matrix is great.
It gave me three pivots, I didn't have to do anything special, I just followed the rules and, and the pivots are 1, 2 and 5.
By the way, just because I always anticipate stuff from a later day, if I wanted to know the determinant of this matrix -- which I never do want to know, but I would just multiply the pivots.
The determinant is 10.
So even things like the determinant are here.
Okay.
Now -- oh, let me talk about failure for a moment, and then -- and then come back to success.
How could this have failed?
How could -- by fail, I mean to come up with three pivots.
I mean, there are a couple of points.
I would have already been in trouble if this very first number here was 0.
If it was a 0 there -- suppose that had been a 0, there were no Xs in that equation -- first equation.
Does that mean I can't solve the problem?
Does that mean I quit?
No.
What do I do?
I switch rows.
I exchange rows.
So in case of a 0, I will not say 0 pivot.
I will never be heard to utter those words, 0 pivot.
But if there's a 0 in the pivot position, maybe I can say that, I would try to exchange for a lower equation and get a proper pivot up there.
Okay.
Now, for example, this second pivot came out two.
Could it have come out 0?
What -- actually, if I change that 8 a little bit, I would have got a little trouble.
What should I change that 8 to so that I run into trouble?
A 6.
If that had been a 6, then this would have been 0 and I couldn't have used that as the pivot.
But I could have exchanged again.
In this case.
In this case, because when can I get out of trouble?
I can get out of trouble if there's a non-0 below this troublesome 0.
And there is here.
So I would be okay in this case.
If this was a 6, I would survive by a row exchange.
Now -- of course, it might have happened that I couldn't do the row, that -- that there was 0s below it, but here there wasn't.
Now, I could also have got in trouble if this number 1 was a little different.
See, that 1 became a 5, I guess, by the end.
So can you see what number there would have got me trouble that I really couldn't get out of?
Trouble that I couldn't get out of would mean if 0 is in the pivot position and I've got no place to exchange.
So there must be some number which if I had had here it would have meant failure.
Negative 4, good.
If it was a negative 4 here -- if it happened to be a negative 4, I'll temporarily put it up here.
If this had been a negative 4 z, then I would have gone through the same steps.
This would have been a minus 4, it still would have been a minus 4.
But at the last minute it would have become 0.
And there wouldn't have been a third pivot.
The matrix would have not been invertible.
Well, of course, the inverse of a matrix is coming next week, but, you've heard these words before.
So, that's how we identify failure.
There's temporary failure when we can do a row exchange -- and get out of it, or there's complete failure when we get a 0 and -- and there's nothing below that we can use.
Okay.
Let's stay with -- back to success now.
In fact, I guess the next topic is back substitution.
So what's back substitution?
Well, now I'd better bring the right-hand side in.
So what would MatLab do and what should we do?
Let me bring in the right-hand side as an extra column.
So there comes B.
So it's 2, 12, I would call this the augmented matrix.
"Augment" means you've tacked something on.
I've tacked on this extra column.
Because, when I'm working with equations, I do the same thing to both sides.
So, at this step, I subtracted 2 of the first equation away from the second equation so that this augmented -- I even brought some colored chalk, but I don't know if it shows up.
So this is like the augmented -- no!
Damn, circled the wrong thing.
Okay.
Here is b.
Okay, that's the extra column.
Okay.
So what happened to that extra column, the right-hand side of the equations, when I did the first step?
So that was 3 of this away from this, so it took -- the 2 stayed the same, but three 2s got taken away from 12, leaving 6, and that 2 stayed the same.
So this is how it's looking halfway along.
And let me just carry to the end.
The 2 and the 6 stay the same, but -- what do I have here?
Oh, gosh.
Help me out, now.
What -- so now I'm -- This is still like forward elimination.
I got to this point, which I think is right, and now what did I do at this step?
I multiplied that pivot by 2 or that whole equation by 2 and subtracted from that, so I think I take two 6s, which is 12, away from the 2.
Do you think minus 10 is my final right-hand side -- the right-hand side that goes with U, and let me call that once and forever the vector c. So c is what happens to b, and U is what happens to A.
Okay.
There you've seen elimination clean.
Okay.
Oh, what's back substitution?
So what are my final equations, then?
Can I copy these equations?
x+2y+z=2 is still there and 2y-2z=6 is there, and 5z=-10.
Okay.
Those are the equations that these numbers are telling me about.
Those are the equations U x equals c. Okay, how do I solve them?
What one do I solve for first?
z. I see immediately that the correct value of z is negative And what do I do next?
I go back upwards.
I now know z here.
So, if z is negative 2, that's 4 there, is that right?
And so 2 y plus a 4 is 6, maybe y is 1.
Going -- this is back substitution.
We're doing it on the fly because it's so easy.
And then x is -- so x -- 2y is 2 minus 2, maybe x is 2?
So you see what back substitution is.
It's the simple step solving the equations in reverse order because the system is triangular.
Okay.
Good.
So that's elimination and back substitution, and I kept the right-hand side along.
Okay, now what do I -- that, like, is first piece of the lecture.
What's the second piece?
Matrices are going to get in.
So I wrote stuff with x, y-s and z-s in there, then I really, got the right shorthand, just writing the matrix entries, and now I want to write the operations that I did in matrices, right?
I've carried the matrices along, but I haven't said the operation those elimination steps, I now want to express as matrices.
Okay.
Here they come.
So now this is elimination matrices.
Okay.
Let me take that first step, which took me from 1 2 1 3 8 1 0 4 1.
I want to operate on that -- I want to do elimination on that.
Okay.
Okay, now I'm remembering a point I want to single out as especially important.
Let me move the board up for that.
Because when we do matrix operations, we've got to, like, be able to see the big picture.
Okay.
Last time, I spoke about the big picture of -- when I multiply a matrix by a right-hand side.
If I have some matrix there and I multiply it by 3 4 5, let's say -- so here's a matrix -- what did I say -- well, I guess I only said it on the videotape, but -- do you remember how I look at that matrix multiplication?
The result of multiplying a matrix by some vector is a combination of the columns of the matrix.
It's 3 times the first column.
It's 3 times column one plus 4 times column two plus 5 times column three.
Okay.
I'm going to come back to that multiple times.
What I wanted to do now was to emphasize the parallel thing with rows.
Why?
Because all our operations here for this two weeks of the course are row operations.
So this isn't what I need for row operations.
Let me do a row operation.
Suppose I have my matrix again and suppose I multiply on the left by some -- let's say 1 2 7.
Again, I'm just, like, saying what the result is.
And then we'll say how matrix multiplication works and we'll see that it's true.
Okay.
But maybe already I'm making -- I'm sort of bringing up -- the central idea of linear algebra is how these matrices work by rows as well as by columns.
Okay.
How does it work by rows?
What -- so that's a row vector.
I could say that's a one by three matrix, a row vector multiplying a three by three matrix.
What's the output?
What's the product of a row times a matrix?
And -- okay, it's a row.
A row -- a column -- I'm sorry.
A matrix times a column is a column.
So matrix times a -- yeah.
Matrix times a column is a column.
And we know what column it is.
Over here, I'm doing a row times a matrix.
And what's the answer?
It's one of that first row, so it's 1 times -- 1 times row one, plus 2 times row two plus 7 times row three.
When -- as we do matrix multiplication, keep your eye on what it's doing with whole vectors.
And what it's doing -- what it's doing in this case is it's combining the rows.
And we have a combination, a linear combination of the rows.
Okay, I want to use that.
Okay, so my question is what's the matrix that does this first step, that takes -- subtracts 3 of equation one from equation two?
That's what I want to do.
So this is going to be a matrix that's going to subtract 3 times row one from row two, and leaves the other rows the same.
Just in -- I mean, the answer is going to be that.
So whatever matrix this is -- and you're going to, like, tell me what matrix will do it, it's the matrix that leaves the first row unchanged, leaves the last row unchanged, but takes 3 of these away from this so it puts a 0 there, a 2 there and a minus 2.
Good.
What matrix will do it?
It's these.
It should be a pretty simple matrix, because we're doing a very simple step.
We're just doing this step that changes row two.
So actually, row one is not changing.
So tell me how the matrix should begin.
One -- the first row of the matrix will be 1 0 0, because that's just the right thing that takes one of that row and none of the other rows, and that's what we want.
What's the last row of the matrix?
0 0 1, because that takes one of the third row and none of the other rows, that's great.
Okay.
Now, suppose I didn't want to do anything at all.
Suppose my row -- well, I guess maybe I had a case here when I already had a 0 and, didn't have to do anything.
What matrix does nothing, like, just leaves you where you were?
If I put in -- if I put in 0 1 0, that would be -- that would be -- that's the matrix -- what's the name of that matrix?
The identity matrix, right.
So it does absolutely nothing.
It just multiplies everything and leaves it where it is.
It's like a one, like the number one, for matrices.
But that's not what we want, because we want to change this row to -- so what's the correct -- what should I put in here now to do it right?
I want to get -- what do I want?
What I -- I'm after -- I want 3 of row one to get subtracted off.
So what's the right matrix, finish that matrix for me.
Negative 3 goes here?
And what goes here?
That 1.
And what goes here?
The 0.
That's the good matrix.
That's the matrix that takes minus 3 of row one plus the row two and gives the new row 2.
Should we just, like, check some particular entry?
How do I check a particular entry of a matrix in matrix multiplication?
Like, suppose I wanted to check the entry here that's in row two, column three.
So where does the entry in row two, column three come from?
I would look at row two of this guy and column three of this one to get that number.
That number comes from the second row and the third column and I just take this dot product minus 3 -- I'm multiplying -- minus 3 plus 1 and 0 gives the minus 2.
Yeah.
It works.
So we got various ways to multiply matrices now.
We're sort of, like -- informally.
We've got by columns, we've got -- well, we will have by columns, by rows, by each entry at a time.
But it's good to see that matrix multiplication when one of the matrices is so simple.
So this guy is our elementary matrix.
Let's call it E for elementary or elimination.
And let me put the indexes 2 1, because it's the matrix that we needed to fix the 2 1 position.
It's the matrix that we needed to get this 2 1 position to be Okay.
Good enough.
So what do I do next?
I need another matrix, right?
I need to -- there's another step here.
And I want to express the whole elimination process in matrix language.
So tell me what -- so next step, step two, which was what?
Subtract -- what was -- what was the actual step that we did?
I think I subtracted -- do you remember?
I had a 2 in the pivot and a 4 below it, so I subtracted two times -- times row two from row three.
From row three.
Tell me the matrix that will do that.
And tell me its name.
Okay, it's going to be E, for elementary or elimination matrix and what's the index number that I used to tell me what E -- 3, 2, right?
Because it's fixing this 3 2 position.
And what's the matrix, now?
Okay, you remember -- so E 3 2 is supposed to multiply my guy that I have and it's supposed to produce the right result, which was -- it leaves -- it's supposed to leave the first row, it's supposed to leave the second row and it's supposed to straighten out that third row to this.
And what's the matrix that does that?
1 0 0, right?
Because we don't change the first row and the next row we don't change either, and the last row is the one we do change.
And what do I do?
Let's see, I subtract two times -- so what's this row?
What's this here?
0, right, because the first row's not involved.
It's just in the 3 2 position, isn't it?
This the key number is this minus the multiplier that goes -- sitting there in that 3 2 position.
Is it a minus 2 to subtract 2 and then this is a 1 so that -- the overall effect is to take minus 2 of this row plus 1 of that.
Okay.
So, I've now given you the pieces, the elimination matrices, the elementary matrices that take each step.
So now what?
Now the next point in the lecture is to put those steps together into a matrix that does it all and see how it all happens.
So now I'm going to express the whole -- everything we did today so far on A was to start with A, we multiplied it by E 2 1, that was the first step -- and then we multiplied that result by E 3 2 and that led us to this thing and what was that matrix?
U.
You see why I like matrix notation, because there in, like, little space -- a few bits when its compressed on the web -- is everything -- is this whole lecture.
Okay.
Now there -- there are important facts about matrix multiplication.
And we're close to maybe the most important.
And that is this.
Suppose I ask you this question.
Suppose I start with a matrix A and I want to end with a matrix U and I want to say what matrix does the whole job?
What matrix takes me from A to U, using the letters I've got?
And the answer is simple.
I'm not asking this as -- but it's highly important.
How would I create the matrix that does the whole job at once, that does all of elimination in one shot?
It would be -- I would just put these together, right?
In other words, this is the thing I'm struggling to say.
I can move those parentheses.
If I keep the matrices in order -- I can't mess around with the order of the matrices, but I can change the order that I do the multiplications.
I can multiply these two first -- in other words, you see what those parentheses are doing?
It's saying -- multiply the Es first and that gives you the matrix that does everything at once.
Okay.
So this fact, that this is automatically the same as this -- for every matrix multiplication, which I'm conscious of still not telling you in every detail, but, like, you're seeing how it works -- and this is highly important -- and maybe tell me the long word that describes this law for matrices, that you can move the parentheses?
It's called the associative law.
I think you can now forget that.
But don't forget the law.
I mean, like, forget the word associative.
I don't know.
But don't forget the law.
Because actually, we'll see so many steps in linear algebra, so many proofs, even, of main fact come from just moving the parentheses.
And it's not that easy to prove that this is correct, you have to go into the gory details of matrix multiplication, do it both ways and see that you come out the same.
Maybe I'll leave the author to do that.
Okay.
So there we go.
So there's a single matrix, I could call it E -- while we're talking about these matrices, tell me one other -- there's another type of elementary matrix, and we already said why we might need it.
We didn't need it in this case.
But it's the matrix that exchanges two rows.
It's called a permutation matrix.
Can you just, like, tell me what that would So I'm just -- like, this is a slight digression and be?
we'll -- yes, so let me get some -- let me figure out where I'm going to put a permutation matrix.
You'll see I'm always squeezing stuff in.
So permutation.
Or, in fact this one you'll, like, exchange rows -- shall I exchange rows one and two, just to make life easy?
So if I had my matrix -- no, let -- let me just do two by two.
|a b; c d|.
Suppose I want to find the matrix that exchanges those rows.
What is it?
So the matrix that exchanges those rows -- the row I want is c d and it's there.
So I better take one of it.
And the row I want here is up top, so I'll take one of that.
So actually, I'm just -- the easy way -- this is my matrix that I'll call P, for permutation.
It's the matrix -- actually, the easy way to find it is just do the thing to the identity matrix.
Exchange the rows of the identity matrix and then that's the matrix that will do row exchanges for you.
Suppose I wanted to exchange columns instead.
Columns have hardly got into today's lecture, but they certainly are going to be around.
How could I -- if I started with this matrix |a b; c d| then I wouldn't -- I'm not even going to write this down, I'm just going to ask you, because in elimination, we're doing rows.
But suppose we wanted to exchange the columns of a matrix.
How would I do that?
What matrix multiplication would do that job?
Actually, why not?
I'll write it down.
So this is -- I'll write it under here and then hide it again.
Okay.
Suppose I had my matrix |a b; c d| and I want to get to a c over here and b d here.
What matrix does that job?
Can I multiply -- can I cook up some matrix that produces that answer?
You can see from where I put my hand I was really asking can I put a matrix here on the left that will exchange columns?
And the answer is no.
I'm just bringing out again this point that when I multiply on the left, I'm doing row operations.
So if I want to do a column operation, where do I put that permutation matrix?
On the right.
If I put it here, where I just barely left room for it -- so I'll exchange the two columns of the identity.
Then it comes out right, because now I'm multiplying a column at a time.
This is the first column and says take one -- take none of that column, one of this one and then you got it.
Over here, take one of this one, none of this one and you've got a c. So, in short, to do column operations, the matrix multiplies on the right.
To do row operations, it multiplies on the left.
Okay, okay, and it's row operations that we're really doing.
Okay.
And of course, I mentioned in passing, but I better say it very clearly that you can't exchange the orders of matrices.
And that's just the point I was making again here.
A times B is not the same as B times A.
You have to keep these matrices in their Gauss given order here, right?
But you can move the parentheses, so that, in other words, the commutative law, which would allow you to take it in the other order is false.
So we have to keep it in that order.
Okay.
So what next?
I could do this multiplication.
I could do E 32.
So let me come back to see what that was.
Here was E 2 1.
And here is E 3 2.
And if I multiply those two matrices together -- E 3 2 and then E 2 1, I'll get a single matrix that does elimination.
I don't want to do it that -- if I do that multiplication -- there -- there's a better way to do this.
And so in this last few minutes of today's lecture, can I anticipate that better way?
The better way is to think not how do I get from A to U, but how do I get from U back to A?
So reversing steps is going to come in.
Inverse -- I'll use the word inverse here.
Okay.
So let me make the first step at what's the inverse matrix?
All the matrices you've seen on this board have inverses.
I didn't write any bad matrices down.
We spoke about possible failure, and for a moment, we put in a matrix that would fail.
But right now, all these matrices are good, they're all invertible.
And let's take the inverse -- well, let me say first what does the inverse mean and find it?
Okay.
So we're getting a little leg up on inverses.
Okay, so this is the final moments of today.
Sorry, he's still there.
Okay.
Inverses.
Okay, and I'm just going to take one example and then we're done.
The example I'll take will be that E. So my matrix is 1 0 0 minus 3 1 0 0 0 1.
And I want to find the matrix that undoes that step.
So what was that step?
The step was subtract 3 times row one from row two.
So what matrix will get me back?
What matrix will bring back -- you know, if I started with a 2 12 2 and I changed it to a 2 6 2 because of this guy, I want to get back to the 2 12 I want to find the matrix which -- which undoes elimination, the matrix which multiplies this to give the identity.
And you can tell me what I should do in words first, and then we'll write down the matrix that does it.
If this step subtracted 3 times row 1 from row 2, what's the inverse step?
I add 3 times row one to row two, right?
I add it back.
The -- what I subtracted away, I add back.
So the inverse matrix in this case is -- I now want to add 3 times row one to row two, so I won't change row one, I won't change row three and I'll add 3 times row one to row two.
That's a case where the inverse is clear.
It's clear in words what to do, because what this did was simple to express.
It just changed row two by subtracting 3 of row one.
So to invert it, I go that way.
And if you -- if we do that calculation, 3 times this row plus 1 times this row, comes out the right row of the identity.
Okay, so inverses are an -- so if this matrix was E and this matrix is I for identity, then what's the notation for this guy?
E to the minus one.
E inverse.
Okay.
Let's stop there for today.
That's a little jump on what's coming on Monday.
So, see you Monday.
I've been multiplying matrices already, but certainly time for me to discuss the rules for matrix multiplication.
And the interesting part is the many ways you can do it, and they all give the same answer.
And they're all important.
So matrix multiplication, and then, come inverses.
So we mentioned the inverse of a matrix.
That's a big deal.
Lots to do about inverses and how to find them.
Okay, so I'll begin with how to multiply two matrices.
First way, okay, so suppose I have a matrix A multiplying a matrix B and -- giving me a result -- well, I could call it C. A times B.
Okay.
So, let me just review the rule for this entry.
That's the entry in row i and column j.
So that's the i j entry.
Right there is C i j.
We always write the row number and then the column number.
So I might -- I might -- maybe I take it C 3 4, just to make it specific.
So instead of i j, let me use numbers.
C 3 4.
So where does that come from, the three four entry?
It comes from row three, here, row three and column four, as you know.
Column four.
And can I just write down, or can we write down the formula for it?
If we look at the whole row and the whole column, the quick way for me to say it is row three of A -- I could use a dot for dot product.
I won't often use that, actually.
Dot column four of B.
But this gives us a chance to just, like, use a little matrix notation.
What are the entries?
What's this first entry in row three?
That number that's sitting right there is... A, so it's got two indices and what are they?
3 1.
So there's an a 3 1 there.
Now what's the first guy at the top of column four?
So what's sitting up there?
B 1 4, right.
So that this dot product starts with A 3 1 times B 1 4.
And then what's the next -- so this is like I'm accumulating this sum, then comes the next guy, A 3 2, second column, times B 2 4, second row.
So it's b A 3 2, B 2 4 and so on.
Just practice with indices.
Oh, let me even practice with a summation formula.
So this is -- most of the course, I use whole vectors.
I very seldom, get down to the details of these particular entries, but here we'd better do it.
So it's some kind of a sum, right?
Of things in row three, column K shall I say?
Times things in row K, column four.
Do you see that that's what we're seeing here?
This is K is one, here K is two, on along -- so the sum goes all the way along the row and down the column, say, one to N. So that's what the C three four entry looks like.
A sum of a three K b K four.
Just takes a little practice to do that.
Okay.
And -- well, maybe I should say -- when are we allowed to multiply these matrices?
What are the shapes of these things?
The shapes are -- if we allow them to be not necessarily square matrices.
If they're square, they've got to be the same size.
If they're rectangular, they're not the same size.
If they're rectangular, this might be -- well, I always think of A as m by n. m rows, n columns.
So that sum goes to n. Now what's the point -- how many rows does B have to have?
n. The number of rows in B, the number of guys that we meet coming down has to match the number of ones across.
So B will have to be n by something.
Whatever.
P. So the number of columns here has to match the number of rows there, and then what's the result?
What's the shape of the result?
What's the shape of C, the output?
Well, it's got these same m rows -- it's got m rows.
And how many columns?
P. m by P. Okay.
So there are m times P little numbers in there, entries, and each one, looks like that.
Okay.
So that's the standard rule.
That's the way people think of multiplying matrices.
I do it too.
But I want to talk about other ways to look at that same calculation, looking at whole columns and whole rows.
Okay.
So can I do A B C again?
A B equaling C again?
But now, tell me about...
I'll put it up here.
So here goes A, again, times B producing C. And again, this is m by n. This is n by P and this is m by P. Okay.
Now I want to look at whole columns.
I want to look at the columns of -- here's the second way to multiply matrices.
Because I'm going to build on what I know already.
How do I multiply a matrix by a column?
I know how to multiply this matrix by that column.
Shall I call that column one?
That tells me column one of the answer.
The matrix times the first column is that first column.
Because none of this stuff entered that part of the answer.
The matrix times the second column is the second column of the answer.
Do you see what I'm saying?
That I could think of multiplying a matrix by a vector, which I already knew how to do, and I can think of just P columns sitting side by side, just like resting next to each other.
And I multiply A times each one of those.
And I get the P columns of the answer.
Do you see this as -- this is quite nice, to be able to think, okay, matrix multiplication works so that I can just think of having several columns, multiplying by A and getting the columns of the answer.
So, like, here's column one shall I call that column one?
And what's going in there is A times column one.
Okay.
So that's the picture a column at a time.
So what does that tell me?
What does that tell me about these columns?
These columns of C are combinations, because we've seen that before, of columns of A.
Every one of these comes from A times this, and A times a vector is a combination of the columns of A.
And it makes sense, because the columns of A have length m and the columns of C have length m. And every column of C is some combination of the columns of A.
And it's these numbers in here that tell me what combination it is.
Do you see that?
That in that answer, C, I'm seeing stuff that's combinations of these columns.
Now, suppose I look at it -- that's two ways now.
The third way is look at it by rows.
So now let me change to rows.
Okay.
So now I can think of a row of A -- a row of A multiplying all these rows here and producing a row of the product.
So this row takes a combination of these rows and that's the answer.
So these rows of C are combinations of what?
Tell me how to finish that.
The rows of C, when I have a matrix B, it's got its rows and I multiply by A, and what does that do?
It mixes the rows up.
It creates combinations of the rows of B, thanks.
Rows of B.
That's what I wanted to see, that this answer -- I can see where the pieces are coming from.
The rows in the answer are coming as combinations of these rows.
The columns in the answer are coming as combinations of those columns.
And so that's three ways.
Now you can say, okay, what's the fourth way?
The fourth way -- so that's -- now we've got, like, the regular way, the column way, the row way and -- what's left?
The one that I can -- well, one way is columns times rows.
What happens if I multiply -- So this was row times column, it gave a number.
Okay.
Now I want to ask you about column times row.
If I multiply a column of A times a row of B, what shape I ending up with?
So if I take a column times a row, that's definitely different from taking a row times a column.
So a column of A was -- what's the shape of a column of A?
m by one.
A column of A is a column.
It's got m entries and one column.
And what's a row of B?
It's got one row and P columns.
So what's the shape -- what do I get if I multiply a column by a row?
I get a big matrix.
I get a full-sized matrix.
If I multiply a column by a row -- should we just do one?
Let me take the column two three four times the row one six.
That product there -- I mean, when I'm just following the rules of matrix multiplication, those rules are just looking like -- kind of petite, kind of small, because the rows here are so short and the columns there are so short, but they're the same length, one entry.
So what's the answer?
What's the answer if I do two three four times one six, just for practice?
Well, what's the first row of the answer?
Two twelve.
And the second row of the answer is three eighteen.
And the third row of the answer is four twenty four.
That's a very special matrix, there.
Very special matrix.
What can you tell me about its columns, the columns of that matrix?
They're multiples of this guy, right?
They're multiples of that one.
Which follows our rule.
We said that the columns of the answer were combinations, but there's only -- to take a combination of one guy, it's just a multiple.
The rows of the answer, what can you tell me about those three rows?
They're all multiples of this row.
They're all multiples of one six, as we expected.
But I'm getting a full-sized matrix.
And now, just to complete this thought, if I have -- let me write down the fourth way.
A B is a sum of columns of A times rows of B.
So that, for example, if my matrix was two three four and then had another column, say, seven eight nine, and my matrix here has -- say, started with one six and then had another column like zero zero, then -- here's the fourth way, okay?
I've got two columns there, I've got two rows there.
So the beautiful rule is -- see, the whole thing by columns and rows is that I can take the first column times the first row and add the second column times the second row.
So that's the fourth way -- that I can take columns times rows, first column times first row, second column times second row and add.
Actually, what will I get?
What will the answer be for that matrix multiplication?
Well, this one it's just going to give us zero, so in fact I'm back to this -- that's the answer, for that matrix multiplication.
I'm happy to put up here these facts about matrix multiplication, because it gives me a chance to write down special matrices like this.
This is a special matrix.
All those rows lie on the same line.
All those rows lie on the line through one six.
If I draw a picture of all these row vectors, they're all the same direction.
If I draw a picture of these two column vectors, they're in the same direction.
Later, I would use this language.
Not too much later, either.
I would say the row space, which is like all the combinations of the rows, is just a line for this matrix.
The row space is the line through the vector one six.
All the rows lie on that line.
And the column space is also a line.
All the columns lie on the line through the vector two three four.
So this is like a really minimal matrix.
And it's because of these ones.
Okay.
So that's a third way.
Now I want to say one more thing about matrix multiplication while we're on the subject.
And it's this.
You could also multiply -- You could also cut the matrix into blocks and do the multiplication by blocks.
Yet that's actually so, useful that I want to mention it.
Block multiplication.
So I could take my matrix A and I could chop it up, like, maybe just for simplicity, let me chop it into two -- into four square blocks.
Suppose it's square.
Let's just take a nice case.
And B, suppose it's square also, same size.
So these sizes don't have to be the same.
What they have to do is match properly.
Here they certainly will match.
So here's the rule for block multiplication, that if this has blocks like, A -- so maybe A1, A2, A3, A4 are the blocks here, and these blocks are B1, B2,3 and B4?
Then the answer I can find block.
And if you tell me what's in that block, then I'm going to be quiet about matrix multiplication for the rest of the day.
What goes into that block?
You see, these might be -- this matrix might be -- these matrices might be, like, twenty by twenty with blocks that are ten by ten, to take the easy case where all the blocks are the same shape.
And the point is that I could multiply those by blocks.
And what goes in here?
What's that block in the answer?
A1 B1, that's a matrix times a matrix, it's the right size, ten by ten.
Any more?
Plus, what else goes in there?
A2 B3, right?
It's just like block rows times block columns.
Nobody, I think, not even Gauss could see instantly that it works.
But somehow, if we check it through, all five ways we're doing the same multiplications.
So this familiar multiplication is what we're really doing when we do it by columns, by rows by columns times rows and by blocks.
Okay.
I just have to, like, get the rules straight for matrix multiplication.
Okay.
All right, I'm ready for the second topic, which is inverses.
Okay.
Ready for inverses.
And let me do it for square matrices first.
Okay.
So I've got a square matrix A.
And it may or may not have an inverse, right?
Not all matrices have inverses.
In fact, that's the most important question you can ask about the matrix, is if it's -- if you know it's square, is it invertible or not?
If it is invertible, then there is some other matrix, shall I call it A inverse?
And what's the -- if A inverse exists -- there's a big "if" here.
If this matrix exists, and it'll be really central to figure out when does it exist?
And then if it does exist, how would you find it?
But what's the equation here that I haven't -- that I have to finish now?
This matrix, if it exists multiplies A and produces, I think, the identity.
But a real -- an inverse for a square matrix could be on the right as well -- this is true, too, that it's -- if I have a -- yeah in fact, this is not -- this is probably the -- this is something that's not easy to prove, but it works.
That a left -- square matrices, a left inverse is also a right inverse.
If I can find a matrix on the left that gets the identity, then also that matrix on the right will produce that identity.
For rectangular matrices, we'll see a left inverse that isn't a right inverse.
In fact, the shapes wouldn't allow it.
But for square matrices, the shapes allow it and it happens, if A has an inverse.
Okay, so give me some cases -- let's see.
I hate to be negative here, but let's talk about the case with no inverse.
So -- these matrices are called invertible or non-singular -- those are the good ones.
And we want to be able to identify how -- if we're given a matrix, has it got an inverse?
Can I talk about the singular case?
No inverse.
All right.
Best to start with an example.
Tell me an example -- let's get an example up here.
Let's make it two by two -- of a matrix that has not got an inverse.
And let's see why.
Let me write one up.
No inverse.
Let's see why.
Let me write up -- one three two six.
Why does that matrix have no inverse?
You could answer that various ways.
Give me one reason.
Well, you could -- if you know about determinants, which you're not supposed to, you could take its determinant and you would get -- Zero.
Okay.
Now -- all right.
Let me ask you other reasons.
I mean, as for other reasons that that matrix isn't invertible.
Here, I could use what I'm saying here.
Suppose A times other matrix gave the identity.
Why is that not possible?
Because -- oh, yeah -- I'm thinking about columns here.
If I multiply this matrix A by some other matrix, then the -- the result -- what can you tell me about the columns?
They're all multiples of those columns, right?
If I multiply A by another matrix that -- the product has columns that come from those columns.
So can I get the identity matrix?
No way.
The columns of the identity matrix, like one zero -- it's not a combination of those columns, because those two columns lie on the -- both lie on the same line.
Every combination is just going to be on that line and I can't get one zero.
So, do you see that sort of column picture of the matrix not being invertible.
In fact, here's another reason.
This is even a more important reason.
Well, how can I say more important?
All those are important.
This is another way to see it.
A matrix has no inverse -- yeah -- here -- now this is important.
A matrix has no -- a square matrix won't have an inverse if there's no inverse because I can solve -- I can find an X of -- a vector X with A times -- this A times X giving zero.
This is the reason I like best.
That matrix won't have an inverse.
Can you -- well, let me change I to U.
So tell me a vector X that, solves A X equals zero.
I mean, this is, like, the key equation.
In mathematics, all the key equations have zero on the right-hand side.
So what's the X?
Tell me an X here -- so now I'm going to put -- slip in the X that you tell me and I'm going to get zero.
What X would do that job?
Three and negative one?
Is that the one you picked, or -- yeah.
Or another -- well, if you picked zero with zero, I'm not so excited, right?
Because that would always work.
So it's really the fact that this vector isn't zero that's important.
It's a non-zero vector and three negative one would do it.
That just says three of this column minus one of that column is the zero column.
Okay.
So now I know that A couldn't be invertible.
But what's the reasoning?
If A X is zero, suppose I multiplied by A inverse.
Yeah, well here's the reason.
Here -- this is why this spells disaster for an inverse.
The matrix can't have an inverse if some combination of the columns gives z- it gives nothing.
Because, I could take A X equals zero, I could multiply by A inverse and what would I discover?
Suppose I take that equation and I multiply by -- if A inverse existed, which of course I'm going to come to the conclusion it can't because if it existed, if there was an A inverse to this dopey matrix, I would multiply that equation by that inverse and I would discover X is zero.
If I multiply A by A inverse on the left, I get X.
If I multiply by A inverse on the right, I get zero.
So I would discover X was zero.
But it -- X is not zero.
X -- this guy wasn't zero.
There it is.
It's three minus one.
So, conclusion -- only, it takes us some time to really work with that conclusion -- our conclusion will be that non-invertible matrices, singular matrices, some combinations of their columns gives the zero column.
They they take some vector X into zero.
And there's no way A inverse can recover, right?
That's what this equation says.
This equation says I take this vector X and multiplying by A gives zero.
But then when I multiply by A inverse, I can never escape from zero.
So there couldn't be an A inverse.
Where here -- okay, now fix -- all right.
Now let me take -- all right, back to the positive side.
Let's take a matrix that does have an inverse.
And why not invert it?
Okay.
Can I -- so let me take on this third board a matrix -- shall I fix that up a little?
Tell me a matrix that has got an inverse.
Well, let me say one three two -- what shall I put there?
Well, don't put six, I guess is -- right?
Do I any favorites here?
One?
Or eight?
I don't care.
What, seven?
Seven.
Okay.
Seven is a lucky number.
All right, seven, okay.
Okay.
So -- now what's our idea?
We believe that this matrix is invertible.
Those who like determinants have quickly taken its determinant and found it wasn't zero.
Those who like columns, and probably that -- that department is not totally popular yet -- but those who like columns will look at those two columns and say, hey, they point in different directions.
So I can get anything.
Now, let me see, what do I mean?
How I going to computer A inverse?
So A inverse -- here's A inverse, now, and I have to find it.
And what do I get when I do this multiplication?
The identity.
You know, forgive me for taking two by two-s, but -- lt's good to keep the computations manageable and let the ideas come out.
Okay, now what's the idea I want?
I'm looking for this matrix A inverse, how I going to find it?
Right now, I've got four numbers to find.
I'm going to look at the first column.
Let me take this first column, A B.
What's up there?
What -- tell me this.
What equation does the first column satisfy?
The first column satisfies A times that column is one zero.
The first column of the answer.
And the second column, C D, satisfies A times that second column is zero one.
You see that finding the inverse is like solving two systems.
One system, when the right-hand side is one zero -- I'm just going to split it into two pieces.
I don't even need to rewrite it.
I can take A times -- so let me put it here.
A times column j of A inverse is column j of the identity.
I've got n equations.
I've got, well, two in this case.
And they have the same matrix, A, but they have different right-hand sides.
The right-hand sides are just the columns of the identity, this guy and this guy.
And these are the two solutions.
Do you see what I'm going -- I'm looking at that equation by columns.
I'm looking at A times this column, giving that guy, and A times that column giving that guy.
So -- Essentially -- so this is like the Gauss -- we're back to Gauss.
We're back to solving systems of equations, but we're solving -- we've got two right-hand sides instead of one.
That's where Jordan comes in.
So at the very beginning of the lecture, I mentioned Gauss-Jordan, let me write it up again.
Okay.
Here's the Gauss-Jordan idea.
Gauss-Jordan solve two equations at once.
Okay.
Let me show you how the mechanics go.
How do I solve a single equation?
So the two equations are one three two seven, multiplying A B gives one zero.
And the other equation is the same one three two seven multiplying C D gives zero one.
Okay.
That'll tell me the two columns of the inverse.
I'll have inverse.
In other words, if I can solve with this matrix A, if I can solve with that right-hand side and that right-hand side, I'm invertible.
I've got it.
Okay.
And Jordan sort of said to Gauss, solve them together, look at the matrix -- if we just solve this one, I would look at one three two seven, and how do I deal with the right-hand side?
I stick it on as an extra column, right?
That's this augmented matrix.
That's the matrix when I'm watching the right-hand side at the same time, doing the same thing to the right side that I do to the left?
So I just carry it along as an extra column.
Now I'm going to carry along two extra columns.
And I'm going to do whatever Gauss wants, right?
I'm going to do elimination.
I'm going to get this to be simple and this thing will turn into the inverse.
This is what's coming.
I'm going to do elimination steps to make this into the identity, and lo and behold, the inverse will show up here.
K--- let's do it.
Okay.
So what are the elimination steps?
So you see -- here's my matrix A and here's the identity, like, stuck on, augmented on
I'm sorry
Yeah?
-- is the two and the three supposed to be switched?
Did I -- oh, no, they weren't supposed to be switched.
Sorry.
Thanks.
Okay.
Thank you very much.
And there -- I've got them right.
Okay, thanks.
Okay.
So let's do elimination.
All right, it's going to be simple, right?
So I take two of this row away from this row.
So this row stays the same and two of those come away from this.
That leaves me with a zero and a one and two of these away from this is that what you're getting -- after one elimination step -- Let me sort of separate the -- the left half from the right half.
So two of that first row got subtracted from the second row.
Now this is an upper triangular form.
Gauss would quit, but Jordan says keeps going.
Use elimination upwards.
Subtract a multiple of equation two from equation one to get rid of the three.
So let's go the whole way.
So now I'm going to -- this guy is fine, but I'm going to -- what do I do now?
What's my final step that produces the inverse?
I multiply this by the right number to get up to ther to remove that three.
So I guess, I -- since this is a one, there's the pivot sitting there.
I multiply it by three and subtract from that, so what do I get?
I'll have one zero -- oh, yeah that was my whole point.
I'll multiply this by three and subtract from that, which will give me seven.
And I multiply this by three and subtract from that, which gives me a minus three.
And what's my hope, belief?
Here I started with A and the identity, and I ended up with the identity and who?
That better be A inverse.
That's the Gauss Jordan idea.
Start with this long matrix, double-length A I, eliminate, eliminate until this part is down to I, then this one will -- must be for some reason, and we've got to find the reason -- must be A inverse.
Shall I just check that it works?
Let me just check that -- can I multiply this matrix this part times A, I'll carry A over here and just do that multiplication.
You'll see I'll do it the old fashioned way.
Seven minus six is a one.
Twenty one minus twenty one is a zero, minus two plus two is a zero, minus six plus seven is a one.
Check.
So that is the inverse.
That's the Gauss-Jordan idea.
So, you'll -- one of the homework problems or more than one for Wednesday will ask you to go through those steps.
I think you just got to go through Gauss-Jordan a couple of times, but I -- yeah -- just to see the mechanics.
But the, important thing is, why -- is, like, what happened?
Why did we -- why did we get A inverse there?
Let me ask you that.
We got -- so we take -- We do row reduction, we do elimination on this long matrix A I until the first half Then a second half is A inverse.
is up.
Well, how do I see that?
Let me put up here how I see that.
So here's my Gauss-Jordan thing, and I'm doing stuff to it.
So I'm -- well, whole lot of E's.
Remember those are those elimination matrices.
Those are the -- those are the things that we figured out last time.
Yes, that's what an elimination step is it's in matrix form, I'm multiplying by some Es.
And the result -- well, so I'm multiplying by a whole bunch of Es.
So, I get a -- can I call the overall matrix E?
That's the elimination matrix, the product of all those little pieces.
What do I mean by little pieces?
Well, there was an elimination matrix that subtracted two of that away from that.
Then there was an elimination matrix that subtracted three of that away from that.
I guess in this case, that was all.
So there were just two Es in this case, one that did this step and one that did this step and together they gave me an E that does both steps.
And the net result was to get an I here.
And you can tell me what that has to be.
This is, like, the picture of what happened.
If E multiplied A, whatever that E is -- we never figured it out in this way.
But whatever that E times that E is, E times A is -- What's E times A?
It's I.
That E, whatever the heck it was, multiplied A and produced So E must be -- E A equaling I tells us what E is, I. namely it is --
It's the inverse of
It's the inverse of A.
Great.
And therefore, when the second half, when E multiplies I, it's E -- Put this A inverse.
You see the picture looking that way?
E times A is the identity.
It tells us what E has to be.
It has to be the inverse, and therefore, on the right-hand side, where E -- where we just smartly tucked on the identity, it's turning in, step by step -- It's turning into A inverse.
There is the statement of Gauss-Jordan elimination.
That's how you find the inverse.
Where we can look at it as elimination, as solving n equations at the same time -- -- and tacking on n columns, solving those equations and up goes the n columns of A inverse Okay, thanks.
See you on Wednesday.
Okay, this is linear algebra, lecture four.
And, the first thing I have to do is something that was on the list for last time, but here it is now.
What's the inverse of a product?
If I multiply two matrices together and I know their inverses, how do I get the inverse of A times B?
So I know what inverses mean for a single matrix A and for a matrix B.
What matrix do I multiply by to get the identity if I have A B?
Okay, that'll be simple but so basic.
Then I'm going to use that to -- I will have a product of matrices and the product that we'll meet will be these elimination matrices and the net result of today's lectures is the big formula for elimination, so the net result of today's lecture is this great way to look at Gaussian elimination.
We know that we get from A to U by elimination.
We know the steps -- but now we get the right way to look at it, A equals L U.
So that's the high point for today.
Okay.
Can I take the easy part, the first step first?
So, suppose A is invertible -- and of course it's going to be a big question, when is the matrix invertible?
But let's say A is invertible and B is invertible, then what matrix gives me the inverse of A B?
So that's the direct question.
What's the inverse of A B?
Do I multiply those separate inverses?
Yes.
I multiply the two matrices A inverse and B inverse, but what order do I multiply?
In reverse order.
And you see why.
So the right thing to put here is B inverse A inverse.
That's the inverse I'm after.
We can just check that A B times that matrix gives the identity.
Okay.
So why -- once again, it's this fact that I can move parentheses around.
I can just erase them all and do the multiplications any way I want to.
So what's the right multiplication to do first?
B times B inverse.
This product here I is the identity.
Then A times the identity is the identity and then finally A times A inverse gives the identity.
So forgive the dumb example in the book.
Why do you, do the inverse things in reverse order?
It's just like -- you take off your shoes, you take off your socks, then the good way to invert that process is socks back on first, then shoes.
Sorry, okay.
I'm sorry that's on the tape.
And, of course, on the other side we should really just check -- on the other side I have B inverse, A inverse.
That does multiply A B, and this time it's these guys that give the identity, squeeze down, they give the identity, we're in shape.
Okay.
So there's the inverse.
Good.
While we're at it, let me do a transpose, because the next lecture has got a lot to -- involves transposes.
So how do I -- if I transpose a matrix, I'm talking about square, invertible matrices right now.
If I transpose one, what's its inverse?
Well, the nice formula is -- let's see.
Let me start from A, A inverse equal the identity.
And let me transpose both sides.
That will bring a transpose into the picture.
So if I transpose the identity matrix, what do I have?
The identity, right?
If I exchange rows and columns, the identity is a symmetric matrix.
It doesn't know the difference.
If I transpose these guys, that product, then again it turns out that I have to reverse the order.
I can transpose them separately, but when I multiply, those transposes come in the opposite order.
So it's A inverse transpose times A transpose giving the identity.
So that's -- this equation is -- just comes directly from that one.
But this equation tells me what I wanted to know, namely what is the inverse of this guy A transpose?
What's the inverse of that -- if I transpose a matrix, what'ss the inverse of the result?
And this equation tells me that here it is.
This is the inverse of A transpose.
Inverse of A transpose.
Of A transpose.
So I'll put a big circle around that, because that's the answer, that's the best answer we could hope for.
That if you want to know the inverse of A transpose and you know the inverse of A, then you just transpose that.
So in a -- to put it another way, transposing and inversing you can do in either order for a single matrix.
Okay.
So these are like basic facts that we can now use, all right -- so now I put it to use.
I put it to use by thinking -- we're really completing, the subject of elimination.
Actually, -- the thing about elimination is it's the right way to understand what the matrix has got.
This A equal L U is the most basic factorization of a matrix.
I always worry that you will think this course is all elimination.
It's just row operations.
And please don't.
We'll be beyond that, but it's the right algebra to do first.
Okay.
So, now I'm coming near the end of it, but I want to get it in a decent form.
So my decent form is matrix form.
I have a matrix A, let's suppose it's a good matrix, I can do elimination, no row exchanges -- So no row exchanges for now.
Pivots all fine, nothing zero in the pivot position.
I get to the very end, which is U.
So I get from A to U.
And I want to know what's the connection?
How is A related to U?
And this is going to tell me that there's a matrix L that connects them.
Okay.
Can I do it for a two by two first?
Okay.
Two by two, elimination.
Okay, so I'll do it under here.
Okay.
So let my matrix A be -- We'll keep it simple, say two and an eight, so we know that the first pivot is a two, and the multiplier's going to be a four and then let me put a one here and what number do I not want to put there?
Four.
I don't want a four there, because in that case, the second pivot would not -- we wouldn't have a second pivot.
The matrix would be singular, general screw-up.
Okay.
So let me put some other number here like seven.
Okay.
Okay.
Now I want to operate on that with my elementary matrix.
So what's the elementary matrix?
Strictly speaking, it's E21, because it's the guy that's going to produce a zero in that position.
And it's going to produce U in one shot, because it's just a two by two matrix.
So two one and I'm going to take four of those away from those, produce that zero and leave a three there.
And that's U.
And what's the matrix that did it?
Quick review, then.
What's the elimination elementary matrix E21 -- it's one zero, thanks.
And -- negative four one, right.
Good.
Okay.
So that -- you see the difference between this and what I'm shooting for.
I'm shooting for A on one side and the other matrices on the other side of the equation.
Okay.
So I can do that right away.
Now here's going to be my A equals L U.
And you won't have any trouble telling me what -- so A is still two one eight seven.
L is what you're going to tell me and U is still two one zero three.
Okay.
So what's L in this case?
Well, first -- so how is L related to this E guy?
It's the inverse, because I want to multiply through by the inverse of this, which will put the identity here, and the inverse will show up there and I'll call it L. So what is the inverse of this?
Remember those elimination matrices are easy to invert.
The inverse matrix for this one is 1 0 4 1, it has the plus sign because it adds back what this removes.
Okay.
Do you want -- if we did the numbers right, we must -- this should be correct.
Okay.
And of course it is.
That says the first row's right, four times the first row plus the second row is eight seven.
Good.
Okay.
That's simple, two by two.
But it already shows the form that we're headed for.
It shows -- so what's the L stand for?
Why the letter L?
If U stood for upper triangular, then of course L stands for lower triangular.
And actually, it has ones on the diagonal, where this thing has the pivots on the diagonal.
Oh, sometimes we may want to separate out the pivots, so can I just mention that sometimes we could also write this as -- we could have this one zero four one -- I'll just show you how I would divide out this matrix of pivots -- two three.
There's a diagonal matrix.
And I just -- whatever is left is here.
Now what's left?
If I divide this first row by two to pull out the two, then I have a one and a one half.
And if I divide the second row by three to pull out the three, then I have a one.
So if this is L U, this is maybe called L D or pivot U.
And now it's a little more balanced, because we have ones on the diagonal here and here.
And the diagonal matrix in the middle.
So both of those... Matlab would produce either one.
I'll basically stay with L U.
Okay.
Now I have to think about bigger than two by two.
But right now, this was just like easy exercise.
And, to tell the truth, this one was a minus sign and this one was a plus sign.
I mean, that's the only difference.
But, with three by three, there's a more significant difference.
Let me show you how that works.
Let me move up to a three by three, let's say some matrix A, okay?
Let's imagine it's three by three.
I won't write numbers down for now.
So what's the first elimination step that I do, the first matrix I multiply it by, what letter will I use for that?
It'll be E two one, because it's -- the first step will be to get a zero in that two one position.
right?
And then the next step will be to get a zero in the three one position.
And the final step will be to get a zero in the three two That's what elimination is, and it produced U. position.
And again, no row exchanges.
I'm taking the nice case, now, the typical case, too -- when I don't have to do any row exchange, all I do is these elimination steps.
Okay.
Now, suppose I want that stuff over on the right-hand side, as I really do.
That's, like, my point here.
I can multiply these together to get a matrix E, but I want it over on the right.
I want its inverse over there.
So what's the right expression now?
If I write A and U, what goes there?
Okay.
So I've got the inverse of this, I've got three matrices in a row now.
And it's their inverses that are going to show up, because each one is easy to invert.
Question is, what about the whole bunch?
How easy is it to invert the whole bunch?
So, that's what we know how to do.
We know how to invert, we should take the separate inverses, but they go in the opposite order.
So what goes here?
E three two inverse, right, because I'll multiply from the left by E three two inverse, then I'll pop it up next to U.
And then will come E three one inverse.
And then this'll be the only guy left standing and that's gone when I do an E two one inverse.
So there is L. That's L U. L is product of inverses.
Now you still can ask why is this guy preferring inverses?
And let me explain why.
Let me explain why is this product nicer than this one?
This product turns out to be better than this one.
Let me take a typical case here.
Let me take a typical case.
So let me -- I have to do three by three for you to see the improvement.
Two by two, it was just one E, no problem.
But let me go up to this case.
Suppose my matrices E21 -- suppose E21 has a minus two in there.
Suppose that -- and now suppose -- oh, I'll even suppose E31 is the identity.
I'm going to make the point with just a couple of these.
Okay.
Now this guy will have -- do something -- now let's suppose minus five one.
Okay.
There's typical.
That's a typical case in which we didn't need an E31.
Maybe we already had a zero in that three one position.
Okay.
Let me see -- is that going to be enough to, show my point?
Let me do that multiplication.
So if I do that multiplication it's like good practice to multiply these matrices.
Tell me what's above the diagonal when I do this multiplication?
All zeroes.
When I do this multiplication, I'm going to get ones on the diagonal and zeroes above.
Because -- what does that say?
That says that I'm subtracting rows from lower rows.
So nothing is moving upwards as it did last time in Gauss-Jordan.
Okay.
Now -- so really, what I have to do is check this minus two one zero, now this is -- what's that number?
This is the number that I'm really have in mind.
That number is ten.
And this one is -- what goes here?
Row three against column two, it looks like the minus five.
What – it's that ten.
How did that ten get in there?
I don't like that ten.
I mean -- of course, I don't want to erase it, because it's right.
But I don't want it there.
It's because -- the ten got in there because I subtracted two of row one from row two, and then I subtracted five of that new row two from row three.
So doing it in that order, how did row one effect row three?
Well, it did, because two of it got removed from row two and then five of those got removed from row three.
So altogether ten of row one got thrown into row three.
Now my point is in the reverse direction -- so now I can do it -- below it I'll do the inverses.
Okay.
And, of course, opposite order.
Reverse order.
Reverse order.
Okay.
So now this is going to -- this is the E that goes on the left side.
Left of A.
Now I'm going to do the inverses in the opposite order, so what's the -- So the opposite order means I put this inverse first.
And what is its inverse?
What's the inverse of E21?
Same thing with a plus sign, right?
For the individual matrices, instead of taking away two I add back two of row one to row two, so no problem.
And now, in reverse order, I want to invert that.
Just right?
I'm doing just this, this.
So now the inverse is again the same thing, but add in the five.
And now I'll do that multiplication and I'll get a happy result.
I hope.
Did I do it right so far?
Yes, okay.
Let me do the multiplication.
I believe this comes out.
So row one of the answer is one zero zero.
Oh, I know that all this is going to be left, right?
Then I have two one zero.
So I get two one zero there, right?
And what's the third row?
What's the third row in this product?
Just read it out to me, the third row?
0 5 1 Because one way to say is – this is saying take one of the last row and there it is.
And this is my matrix L. And it's the one that goes on the left of U.
It goes into -- what do I mean here?
Maybe rather than saying left of A, left of U, let me right down again what I mean.
E A is U, whereas A is L U.
Okay.
Let me make the point now in words.
The order that the matrices come for L is the right order.
The two and the five don't sort of interfere to produce this ten one.
In the right order, the multipliers just sit in the matrix L. That's the point -- that if I want to know L, I have no work to do.
I just keep a record of what those multipliers were, and that gives me L. So I'll draw the -- let me say it.
So this is the A=L U.
So if no row exchanges, the multipliers that those numbers that we multiplied rows by and subtracted, when we did an elimination step -- the multipliers go directly into L. Okay.
So L is -- this is the way, to look at elimination.
You go through the elimination steps, and actually if you do it right, you can throw away A as you create L U.
If you think about it, those steps of elimination, as when you've finished with row two of A, you've created a new row two of U, which you have to save, and you've created the multipliers that you used -- which you have to save, and then you can forget A.
So because it's all there in L and U.
So that's -- this moment is maybe the new insight in elimination that comes from matrix -- doing it in matrix form.
So it was -- the product of Es is -- we can't see what that product of Es is.
The matrix E is not a particularly attractive one.
What's great is when we put them on the other side -- their inverses in the opposite order, there the L comes out just right.
Okay.
Now -- oh gosh, so today's a sort of, like, practical day.
Can we think together how expensive is elimination?
How many operations do we do?
So this is now a kind of new topic which I didn't list as -- on the program, but here it came.
Here it comes.
How many operations on an n by n matrix A. I mean, it's a very practical question.
Can we solve systems of order a thousand, in a second or a minute or a week?
Can we solve systems of order a million in a second or an hour or a week?
I mean, what's the -- if it's n by n, we often want to take n bigger.
I mean, we've put in more information.
We make the whole thing is more accurate for the bigger matrix.
But it's more expensive, too, and the question is how much more expensive?
If I have matrices of order a hundred.
Let's say a hundred by a hundred.
Let me take n to be a hundred.
Say n equal a hundred.
How many steps are we doing?
How many operations are we actually doing that we -- And let's suppose there aren't any zeroes, because of course if a matrix has got a lot of zeroes in good places, we don't have to do those operations, and, it'll be much faster.
But -- so just think for a moment about the first step.
So here's our matrix A, hundred by a hundred.
And the first step will be -- that column, is got zeroes down here.
So it's down to 99 by 99, right?
That's really like the first stage of elimination, to get from this hundred by hundred non-zero matrix to this stage where the first pivot is sitting up here and the first row's okay the first column is okay.
So, eventually -- how many steps did that take?
You see, I'm trying to get an idea.
Is the answer proportional to n?
Is the total number of steps in elimination, the total number, is it proportional to n -- in which case if I double n from a hundred to two hundred -- does it take me twice as long?
Does it square, so it would take me four times as long?
Does it cube so it would take me eight times as long?
Or is it n factorial, so it would take me a hundred times as long?
I think, you know, from a practical point of view, we have to have some idea of the cost, here.
So these are the questions that I'm -- let me ask those questions again.
Is it proportional -- does it go like n, like n squared, like n cubed -- or some higher power of n?
Like n factorial where every step up multiplies by a hundred and then by a hundred and one and then by a hundred and two -- which is it?
Okay, so that's the only way I know to answer that is to think through what we actually had to do.
Okay.
So what was the cost here?
Well, let's see.
What do I mean by an operation?
I guess I mean, well an addition or -- yeah.
No big deal.
I guess I mean an addition or a subtraction or a multiplication or a division.
Okay.
And actually, what operation I doing all the time?
When I multiply row one by multiplier L and I subtract from row six.
What's happening there individually?
What's going on?
If I multiply -- I do a multiplication by L and then a subtraction.
So I guess operation -- Can I count that for the moment as, like, one operation?
Or you may want to count them separately.
The typical operation is multiply plus a subtract.
So if I count those together, my answer's going to come out half as many as if -- I mean, if I count them separately, I'd have a certain number of multiplies, certain number of subtracts.
That's really want to do.
Okay.
How many have I got here?
So, I think -- let's see.
It's about -- well, how many, roughly?
How many operations to get from here to here?
Well, maybe one way to look at it is all these numbers had to get changed.
The first row didn't get changed, but all the other rows got changed at this step.
So this step -- well, I guess maybe -- shall I say it cost about a hundred squared.
I mean, if I had changed the first row, then it would have been exactly hundred squared, because -- because that's how many numbers are here.
A hundred squared numbers is the total count of the entry, and all but this insignificant first row got changed.
So I would say about a hundred squared.
Okay.
Now, what about the next step?
So now the first row is fine.
The second row is fine.
And I'm changing these zeroes are all fine, so what's up with the second step?
And then you're with me.
Roughly, what's the cost?
If this first step cost a hundred squared, about, operations then this one, which is really working on this guy to produce this, costs about what?
How many operations to fix?
About ninety-nine squared, or ninety-nine times ninety-eight.
But less, right?
Less, because our problem's getting smaller.
About ninety-nine squared.
And then I go down and down and the next one will be ninety-eight squared, the next ninety-seven squared and finally I'm down around one squared or -- where it's just like the little numbers.
The big numbers are here.
So the number of operations is about n squared plus that was n, right?
n was a hundred?
n squared for the first step, then n minus one squared, then n minus two squared, finally down to three squared and two squared and even one squared.
No way I should have written that -- squeezed that in.
Let me try it so the count is n squared plus n minus one squared plus -- all the way down to one squared.
That's a pretty decent count.
Admittedly, we didn't catch every single tiny operation, but we got the right leading term here.
And what do those add up to?
Okay, so now we're coming to the punch of this, question, this operation count.
So the operations on the left side, on the matrix A to finally get to U.
And anybody -- so which of these quantities is the right ballpark for that count?
If I add a hundred squared to ninety nine squared to ninety eight squared -- ninety seven squared, all the way down to two squared then one squared, what have I got, about?
It's just one of these -- let's identify it first.
Is it n?
Certainly not.
Is it n factorial?
No.
If it was n factorial, we would -- with determinants, it is n factorial.
I'll put in a bad mark against determinants, because that -- okay, so what is it?
It's n -- well, this is the answer.
It's this order -- n cubed.
It's like I have n terms, right?
I've got n terms in this sum.
And the biggest one is n squared.
So the worst it could be would be n cubed, but it's not as bad as -- it's n cubed times -- it's about one third of n cubed.
That's the magic operation count.
Somehow that one third takes account of the fact that the numbers are getting smaller.
If they weren't getting smaller, we would have n terms times n squared, but it would be exactly n cubed.
But our numbers are getting smaller -- actually, row two and row one moves down to row three.
do you remember where does one third come in this -- I'll even allow a mention of calculus.
So calculus can be mentioned, integration can be mentioned now in the next minute and not again for weeks.
It's not that I don't like 18.01, but18.06 is better.
Okay.
So, -- so what's -- what's the calculus formula that looks like?
It looks like -- if we were in calculus instead of summing stuff, we would integrate.
So I would integrate x squared and I would get one third x cubed.
So if that was like an integral from one to n, of x squared b x, if the answer would be one third n cubed -- and it's correct for the sum also, because that's, like, the whole point of calculus.
The whole point of calculus is -- oh, I don't want to tell you the whole -- I mean, you know the whole point of calculus.
Calculus is like sums except it's continuous.
Okay.
And algebra is discreet.
Okay.
So the answer is one third n cubed.
Now I'll just -- let me say one more thing about operations.
What about the right-hand side?
This was what it cost on the left side.
This is on A.
Because this is A that we're working with.
But what's the cost on the extra column vector b that we're hanging around here?
So b costs a lot less, obviously, because it's just one column.
We carry it through elimination and then actually we do back substitution.
Let me just tell you the answer there.
It's n squared.
So the cost for every right hand side is n squared.
So let me -- I'll just fit that in here -- for the cost of b turns out to be n squared.
So you see if we have, as we often have, a a matrix A and several right-hand sides, then we pay the price on A, the higher price on A to get it split up into L and U to do elimination on A, but then we can process every right-hand side at low cost.
Okay.
So the -- We really have discussed the most fundamental algorithm for a system of equations.
Okay.
So, I'm ready to allow row exchanges.
I'm ready to allow -- now what happens to this whole -- today's lecture if there are row exchanges?
When would there be row exchanges?
There are row -- we need to do row exchanges if a zero shows up in the pivot position.
So moving then into the final section of this chapter, which is about transposes -- well, we've already seen some transposes, and -- the title of this section is, "Transposes and Permutations."
Okay.
So can I say, now, where does a permutation come in?
Let me talk a little about permutations.
So that'll be up here, permutations.
So these are the matrices that I need to do row exchanges.
And I may have to do two row exchanges.
Can you invent a matrix where I would have to do two row exchanges and then would come out fine?
Yeah let's just, for the heck of it -- so I'll put it here.
Let me do three by threes.
Actually, why don't I just plain list all the three by three permutation matrices.
There're a nice little group of them.
What are all the matrices that exchange no rows at all?
Well, I'll include the identity.
So that's a permutation matrix that doesn't do anything.
Now what's the permutation matrix that exchanges -- what is P12?
The permutation matrix that exchanges rows one and two would be -- 0 1 0 -- 1 0 0, right.
I just exchanged those rows of the identity and I've got it.
Okay.
Actually, I'll -- yes.
Let me clutter this up.
Okay.
Give me a complete list of all the row exchange matrices.
So what are they?
They're all the ways I can take the identity matrix and rearrange its rows.
How many will there be?
How many three by three permutation matrices?
Shall we keep going and get the answer?
So tell me some more
Zero one –
Zero – What one are you going to do now?
I'm going to switch the –
Switch rows one and -- One and three, okay.
One and three, leaving two alone.
Okay.
Now what else?
Switch -- what would be the next easy one -- is switch two and three, good.
So I'll leave one zero zero alone and I'll switch -- I'll move number three up and number two down.
Okay.
Those are the ones that just exchange single -- a pair of rows.
This guy, this guy and this guy exchanges a pair of rows, but now there are more possibilities.
What's left?
So tell -- there is another one here.
What's that?
It's going to move -- it's going to change all rows, right?
Where shall we put them?
So -- give me a first row
Zero one zero?
Zero one zero.
Okay, now a second row -- say zero zero one and the third guy One zero zero.
So that is like a cycle.
That puts row two moves up to row one, row three moves up to row two and row one moves down to row three.
And there's one more, which is -- let's see.
What's left?
I'm lost
Is it zero zero one?
Is it zero zero one?
Okay
One zero zero
One zero zero, okay.
Zero one zero, okay.
Great.
Six.
Six of them.
Six P. And they're sort of nice, because what happens if I write, multiply two of them together?
If I multiply two of these matrices together, what can you tell me about the answer?
It's on the list.
If I do some row exchanges and then I do some more row exchanges, then all together I've done row exchanges.
So if I multiply -- but, I don't know.
And if I invert, then I'm just doing row exchanges to get back again.
So the inverses are all there.
It's a little family of matrices that -- they've got their own -- if I multiply, I'm still inside this group.
If I invert I'm inside this group -- actually, group is the right name for this subject.
It's a group of six matrices, and what about the inverses?
What's the inverse of this guy, for example?
What's the inverse -- if I exchange rows one and two, what's the inverse matrix?
Just tell me fast.
The inverse of that matrix is -- if I exchange rows one and two, then what I should do to get back to where I started is the same thing.
So this thing is its own inverse.
That's probably its own inverse.
This is probably not -- actually, I think these are inverses of each other.
Oh, yeah, actually -- the inverse is the transpose.
There's a curious fact about permutations matrices, that the inverses are the transposes.
And final moment -- how many are there if I -- how many four by four permutations?
So let me take four by four -- how many Ps?
Well, okay.
Make a good guess.
Twenty four, right.
Twenty four Ps.
Okay.
So, we've got these permutation matrices, and in the next lecture, we'll use them.
So the next lecture, finishes Chapter 2 and moves to Chapter 3.
Thank you.
Okay.
This is lecture five in linear algebra.
And, it will complete this chapter of the book.
So the last section of this chapter is two point seven that talks about permutations, which finished the previous lecture, and transposes, which also came in the previous lecture.
There's a little more to do with those guys, permutations and transposes.
But then the heart of the lecture will be the beginning of what you could say is the beginning of linear algebra, the beginning of real linear algebra which is seeing a bigger picture with vector spaces -- not just vectors, but spaces of vectors and sub-spaces of those spaces.
So we're a little ahead of the syllabus, which is good, because we're coming to the place where, there's a lot to do.
Okay.
So, to begin with permutations.
Can I just -- so these permutations, those are matrices P and they execute row exchanges.
And we may need them.
We may have a perfectly good matrix, a perfect matrix A that's invertible that we can solve A x=b, but to do it -- I've got to allow myself that extra freedom that if a zero shows up in the pivot position I move it away.
I get a non-zero.
I get a proper pivot there by exchanging from a row below.
And you've seen that already, and I just want to collect the ideas together.
And principle, I could even have to do that two times, or more times.
So I have to allow -- to complete the -- the theory, the possibility that I take my matrix A, I start elimination, I find out that I need row exchanges and I do it and continue and I finish.
Okay.
Then all I want to do is say -- and I won't make a big project out of this -- what happens to A equal L U?
So A equal L U -- this was a matrix L with ones on the diagonal and zeroes above and multipliers below, and this U we know, with zeroes down here.
That's only possible.
That description of elimination assumes that we don't have a P, that we don't have any row exchanges.
And now I just want to say, okay, how do I account for row exchanges?
Because that doesn't.
The P in this factorization is the identity matrix.
The rows were in a good order, we left them there.
Maybe I'll just add a little moment of reality, too, about how Matlab actually does elimination.
Matlab not only checks whether that pivot is not zero, as every human would do.
It checks for is that pivot big enough, because it doesn't like very, very small pivots.
Pivots close to zero are numerically bad.
So actually if we ask Matlab to solve a system, it will do some elimination some row exchanges, which we don't think are necessary.
Algebra doesn't say they're necessary, but accuracy -- numerical accuracy says they are.
Well, we're doing algebra, so here we will say, well, what do row exchanges do, but we won't do them unless we have to.
But we may have to.
And then, the result is -- it's hiding here.
It's the main fact.
This is the description of elimination with row exchanges.
So A equal L U becomes P A equal L U.
So this P is the matrix that does the row exchanges, and actually it does them -- it gets the rows into the right order, into the good order where pivots will not -- where zeroes won't appear in the pivot position, where L and U will come out right as up here.
So, that's the point.
Actually, I don't want to labor that point, that a permutation matrix -- and you remember what those were.
I'll remind you from last time of what the main points about permutation matrices were -- and then just leave this factorization as the general case.
This is -- any invertible A we get this.
For almost every one, we don't need a P. But there's that handful that do need row exchanges, and if we do need them, there they are.
Okay, finally, just to remember what P was.
So permutations, P is the identity matrix with reordered rows.
I include in reordering the possibility that you just leave them the same.
So the identity matrix is -- okay.
That's, like, your basic permutation matrix -- your do-nothing permutation matrix is the identity.
And then there are the ones that exchange two rows and then the ones that exchange three rows and then then ones that exchange four -- well, it gets a little -- it gets more interesting algebraically if you've got four rows, you might exchange them all in one big cycle.
One to two, two to three, three to four, four to one.
Or you might have -- exchange one and two and three and four.
Lots of possibilities there.
In fact, how many possibilities?
The answer was (n)factorial.
This is n(n-1)(n-2)... (3)(2)(1).
That's the number of -- this counts the reorderings, the possible reorderings.
So it counts all the n by n permutations.
And all those matrices have these -- have this nice property that they're all invertible, because we can bring those rows back into the normal order.
And the matrix that does that is just P -- is just the same as the transpose.
You might take a permutation matrix, multiply by its transpose and you will see how -- that the ones hit the ones and give the ones in the identity matrix.
So this is a -- we'll be highly interested in matrices that have nice properties.
And one property that -- maybe I could rewrite that as P transpose P is the identity.
That tells me in other words that this is the inverse of that.
Okay.
We'll be interested in matrices that have P transpose P equal the identity.
There are more of them than just permutations, but my point right now is that permutations are like a little group in the middle -- in the center of these special matrices.
Okay.
So now we know how many there are.
Twenty four in the case of -- there are twenty four four by four permutations, there are five factorial which is a hundred and twenty, five times twenty four would bump us up to a hundred and twenty -- so listing all the five by five permutations would be not so much fun.
Okay.
So that's permutations.
Now also in section two seven is some discussion of transposes.
And can I just complete that discussion.
First of all, I haven't even transposed a matrix on the board here, have I?
So I'd better do it.
So suppose I take a matrix like (1 2 4; 3 3 1).
It's a rectangular matrix, three by two.
And I want to transpose it.
So what's -- I'll use a T, also Matlab would use a prime.
And the result will be -- I'll right it here, because this was three rows and two columns, this was a three by two matrix.
The transpose will be two rows and three columns, two by three.
So it's short and wider.
And, of course, that row -- that column becomes a row -- that column becomes the other row.
And at the same time, that row became a column.
This row became a column.
Oh, what's the general formula for the transpose?
So the transpose -- you see it in numbers.
What I'm going to write is the same thing in symbols.
The numbers are the clearest, of course.
But in symbols, if I take A transpose and I ask what number is in row I and column J of A transpose?
Well, it came out of A.
It came out A by this flip across the main diagonal.
And, actually, it was the number in A which was in row J, column I.
So the row and column -- the row and column numbers just get reversed.
The row number becomes the column number, the column number becomes the row number.
No problem.
Okay.
Now, a special -- the best matrices, we could say.
In a lot of applications, symmetric matrices show up.
So can I just call attention to symmetric matrices?
What does that mean?
What does that word symmetric mean?
It means that this transposing doesn't change the matrix.
A transpose equals A.
And an example.
So, let's take a matrix that's symmetric, so whatever is sitting on the diagonal -- but now what's above the diagonal, like a one, had better be there, a seven had better be here, a nine had better be there.
There's a symmetric matrix.
I happened to use all positive numbers as its entries.
That's not the point.
The point is that if I transpose that matrix, I get it back again.
So symmetric matrices have this property A transpose equals A. I guess at this point -- I'm just asking you to notice this family of matrices that are unchanged by transposing.
And they're easy to identify, of course.
You know, it's not maybe so easy before we had a case where the transpose gave the inverse.
That's highly important, but not so simple to see.
This is the case where the transpose gives the same matrix back again.
That's totally simple to see.
Okay.
Could actually -- maybe I could even say when would we get such a matrix?
For example, this -- that matrix is absolutely far from symmetric, right?
The transpose isn't even the same shape -- because it's rectangular, it turns the -- lies down on its side.
But let me tell you a way to get a symmetric matrix out of this.
Multiply those together.
If I multiply this rectangular, shall I call it R for rectangular?
So let that be R for rectangular matrix and let that be R transpose, which it is.
Then I think that if I multiply those together, I get a symmetric matrix.
Can I just do it with the numbers and then ask you why, how did I know it would be symmetric?
So my point is that R transpose R is always symmetric.
Okay?
And I'm going to do it for that particular R transpose R which was -- let's see, the column was one two four three three one.
I called that one R transpose, didn't I, and I called this guy one two four three three one.
I called that R. Shall we just do that multiplication?
Okay, so up here I'm getting a ten.
Next to it I'm getting two, a nine, I'm getting an eleven.
Next to that I'm getting four and three, a seven.
Now what do I get there?
This eleven came from one three times two three, right?
Row one, column two.
What goes here?
Row two, column one.
But no difference.
One three two three or two three one three, same thing.
It's going to be an eleven.
That's the symmetry.
I can continue to fill it out.
What -- oh, let's get that seven.
That seven will show up down here, too, and then four more numbers.
That seven will show up here because one three times four one gave the seven, but also four one times one three will give that seven.
Do you see that it works?
Actually, do you want to see it work also in matrix language?
I mean, that's quite convincing, right?
That seven is no accident.
The eleven is no accident.
But just tell me how do I know if I transpose this guy -- How do I know it's symmetric?
Well, I'm going to transpose it.
And when I transpose it, I'm hoping I get the matrix back again.
So can I transpose R transpose R?
So just -- so, why?
Well, my suggestion is take the transpose.
That's the only way to show it's symmetric.
Take the transpose and see that it didn't change.
Okay, so I take the transpose of R transpose R. Okay.
How do I do that?
This is our little practice on the rules for transposes.
So the rule for transposes is the order gets reversed.
Just like inverses, which we did prove, same rule for transposes and -- which we'll now use.
So the order gets reversed.
It's the transpose of that that comes first, and the transpose of this that comes -- no.
Is that -- yeah.
That's what I have to write, right?
This is a product of two matrices and I want its transpose.
So I put the matrices in the opposite order and I transpose them.
But what have I got here?
What is R transpose transpose?
Well, don't all speak at once.
R transpose transpose, I flipped over the diagonal, I flipped over the diagonal again, so I've got R. And that's just my point, that if I started with this matrix, I transposed it, I got it back again.
So that's the check, without using numbers, but with -- it checked in two lines that I always get symmetric matrices this way.
And actually, that's where they come from in so many practical applications.
Okay.
So now I've said something today about permutations and about transposes and about symmetry and I'm ready for chapter three.
Can we take a breath -- the tape won't take a breath, but the lecturer will, because to tell you about vector spaces is -- we really have to start now and think, okay, listen up.
What are vector spaces?
And what are sub-spaces?
Okay.
So, the point is, The main operations that we do -- what do we do with vectors?
We add them.
We know how to add two vectors.
We multiply them by numbers, usually called scalers.
If we have a vector, we know what three V is.
If we have a vector V and W, we know what V plus W is.
Those are the two operations that we've got to be able to do.
To legitimately talk about a space of vectors, the requirement is that we should be able to add the things and multiply by numbers and that there should be some decent rules satisfied.
Okay.
So let me start with examples.
So I'm talking now about vector spaces.
And I'm going to start with examples.
Let me say again what this word space is meaning.
When I say that word space, that means to me that I've got a bunch of vectors, a space of vectors.
But not just any bunch of vectors.
It has to be a space of vectors -- has to allow me to do the operations that vectors are for.
I have to be able to add vectors and multiply by numbers.
I have to be able to take linear combinations.
Well, where did we meet linear combinations?
We met them back in, say in R^2.
So there's a vector space.
What's that vector space?
So R two is telling me I'm talking about real numbers and I'm talking about two real numbers.
So this is all two dimensional vectors -- real, such as -- well, I'm not going to be able to list them all.
But let me put a few down.
|3; 2|, |0;0|, |pi; e|.
So on.
And it's natural -- okay.
Let's see, I guess I should do algebra first.
Algebra means what can I do to these vectors?
I can add them.
I can add that to that.
And how do I do it?
A component at a time, of course.
Three two added to zero zero gives me, three two.
Sorry about that.
Three two added to pi e gives me three plus pi, two plus e. Oh, you know what it does.
And you know the picture that goes with it.
There's the vector three two.
And often, the picture has an arrow.
The vector zero zero, which is a highly important vector -- it's got, like, the most important here -- is there.
And of course there's not much of an arrow.
Pi -- I'll have to remember -- pi is about three and a little more, e is about two and a little more.
So maybe there's pi e. I never drew pi e before.
It's just natural to -- this is the first component on the horizontal and this is the second component, going up the vertical.
Okay.
And the whole plane is R two.
So R two is, we could say, the plane.
The xy plane.
That's what everybody thinks.
But the point is it's a vector space because all those vectors are in there.
If I removed one of them -- Suppose I removed zero zero.
Suppose I tried to take the -- considered the X Y plane with a puncture, with a point removed.
Like the origin.
That would be, like, awful to take the origin away.
Why is that?
Why do I need the origin there?
Because I have to be allowed -- if I had these other vectors, I have to be allowed to multiply three two -- this was three two -- by anything, by any scaler, including zero.
I've got to be allowed to multiply by zero and the result's got to be there.
I can't do without that point.
And I have to be able to add three two to the opposite guy, minus three minus two.
And if I add those I'm back to the origin again.
No way I can do without the origin.
Every vector space has got that zero vector in it.
Okay, that's an easy vector space, because we have a natural picture of it.
Okay.
Similarly easy is R^3.
This would be all -- let me go up a little here.
This would be -- R three would be all three dimensional vectors -- or shall I say vectors with three real components.
Okay.
Let me just to be sure we're together, let me take the vector three two zero.
Is that a vector in R^2 or R^3?
Definitely it's in R^3.
It's got three components.
One of them happens to be zero, but that's a perfectly okay number.
So that's a vector in R^3.
We don't want to mix up the -- I mean, keep these vectors straight and keep R^n straight.
So what's R^n?
R^n.
So this is our big example, is all vectors with n components.
And I'm making these darn things column vectors.
Can I try to follow that convention, that they'll be column vectors, and their components should be real numbers.
Later we'll need complex numbers and complex vectors, but much later.
Okay.
So that's a vector space.
Now, let's see.
What do I have to tell you about vector spaces?
I said the most important thing, which is that we can add any two of these and we -- still in R^2.
We can multiply by any number and we're still in R^2.
We can take any combination and we're still in R^2.
And same goes for R^n.
It's -- honesty requires me to mention that these operations of adding and multiplying have to obey a few rules.
Like, we can't just arbitrarily say, okay, the sum of three two and pi e is zero zero.
It's not.
The sum of three two and minus three two is zero zero.
So -- oh, I'm not going to -- the book, actually, lists the eight rules that the addition and multiplication have to satisfy, but they do.
They certainly satisfy it in R^n and usually it's not those eight rules that are in doubt.
What's -- the question is, can we do those additions and do we stay in the space?
Let me show you a case where you can't.
So suppose this is going to be not a vector space.
Suppose I take the xy plane -- so there's R^2.
That is a vector space.
Now suppose I just take part of it.
Just this.
Just this one -- this is one quarter of the vector space.
All the vectors with positive or at least not negative components.
Can I add those safely?
Yes.
If I add a vector with, like, two -- three two to another vector like five six, I'm still up in this quarter, no problem with adding.
But there's a heck of a problem with multiplying by scalers, because there's a lot of scalers that will take me out of this quarter plane, like negative ones.
If I took three two and I multiplied by minus five, I'm way down here.
So that's not a vector space, because it's not -- closed is the right word.
It's not closed under multiplication by all real numbers.
So a vector space has to be closed under multiplication and addition of vectors.
In other words, linear combinations.
It -- so, it means that if I give you a few vectors -- yeah look, here's an important -- here -- now we're getting to some really important vector spaces.
Well, R^n -- like, they are the most important.
But we will be interested in so- in vector spaces that are inside R^n.
Vector spaces that follow the rules, but they -- we don't need all of -- see, there we started with R^2 here, and took part of it and messed it up.
What we got was not a vector space.
Now tell me a vector space that is part of R^2 and is still safely -- we can multiply, we can add and we stay in this smaller vector space.
So it's going to be called a subspace.
So I'm going to change this bad example to a good one.
Okay.
So I'm going to start again with R^2, but I'm going to take an example -- it is a vector space, so it'll be a vector space inside R^2.
And we'll call that a subspace of R^2.
Okay.
What can I do?
It's got something in it.
Suppose it's got this vector in it.
Okay.
If that vector's in my little subspace and it's a true subspace, then there's got to be some more in it, right?
I have to be able to multiply that by two, and that double vector has to be included.
Have to be able to multiply by zero, that vector, or by half, or by three quarters.
All these vectors.
Or by minus a half, or by minus one.
I have to be able to multiply by any number.
So that is going to say that I have to have that whole line.
Do you see that?
Once I get a vector in there -- I've got the whole line of all multiples of that vector.
I can't have a vector space without extending to get those multiples in there.
Now I still have to check addition.
But that comes out okay.
This line is going to work, because I could add something on the line to something else on the line and I'm still on the line.
So, example.
So this is all examples of a subspace -- our example is a line in R^2 actually -- not just any line.
If I took this line, would that -- so all the vectors on that line.
So that vector and that vector and this vector and this vector -- in lighter type, I'm drawing something that doesn't work.
It's not a subspace.
The line in R^2 -- to be a subspace, the line in R^2 must go through the zero vector.
Because -- why is this line no good?
Let me do a dashed line.
Because if I multiplied that vector on the dashed line by zero, then I'm down here, I'm not on the dashed line.
Z- zero's got to be.
Every subspace has got to contain zero -- because I must be allowed to multiply by zero and that will always give me the zero vector.
Okay.
Now, I was going to make -- create some subspaces.
Oh, while I'm in R^2, why don't we think of all the possibilities.
R two, there can't be that many.
So what are the possible subspaces of R^2?
Let me list them.
So I'm listing now the subspaces of R^2.
And one possibility that we always allow is all of R two, the whole thing, the whole space.
That counts as a subspace of itself.
You always want to allow that.
Then the others are lines -- any line, meaning infinitely far in both directions through the zero.
So that's like the whole space -- that's like whole two D space.
This is like one dimension.
Is this line the same as R^1 ?
No.
You could say it looks a lot like R^1.
R^1 was just a line and this is a line.
But this is a line inside R^2.
The vectors here have two components.
So that's not the same as R^1, because there the vectors only have one component.
Very close, you could say, but not the same.
Okay.
And now there's a third possibility.
There's a third subspace that's -- of R^2 that's not the whole thing, and it's not a line.
It's even less.
It's just the zero vector alone.
The zero vector alone, only.
I'll often call this subspace Z, just for zero.
Here's a line, L. Here's a plane, all of R^2.
So, do you see that the zero vector's okay?
You would just -- to understand subspaces, we have to know the rules -- and knowing the rules means that we have to see that yes, the zero vector by itself, just this guy alone satisfies the rules.
Why's that?
Oh, it's too dumb to tell you.
If I took that and added it to itself, I'm still there.
If I took that and multiplied by seventeen, I'm still there.
So I've done the operations, adding and multiplying by numbers, that are required, and I didn't go outside this one point space.
So that's always -- that's the littlest subspace.
And the largest subspace is the whole thing and in-between come all -- whatever's in between.
Okay.
So for example, what's in between for R^3?
So if I'm in ordinary three dimensions, the subspace is R, all of R^3 at one extreme, the zero vector at the bottom.
And then a plane, a plane through the origin.
Or a line, a line through the origin.
So with R^3, the subspaces were R^3, plane through the origin, line through the origin and a zero vector by itself, zero zero zero, just that single vector.
Okay, you've got the idea.
But, now comes -- the reality is -- what are these -- where do these subspaces come -- how do they come out of matrices?
And I want to take this matrix -- oh, let me take that matrix.
So I want to create some subspaces out of that matrix.
Well, one subspace is from the columns.
Okay.
So this is the important subspace, the first important subspace that comes from that matrix -- I'm going to -- let me call it A again.
Back to -- okay.
I'm looking at the columns of A.
Those are vectors in R^3.
So the columns are in R^3.
The columns are in R^3.
So I want those columns to be in my subspace.
Now I can't just put two columns in my subspace and call it a subspace.
What do I have to throw in -- if I'm going to put those two columns in, what else has got to be there to have a subspace?
I must be able to add those things.
So the sum of those columns -- so these columns are in R^3, and I have to be able -- I'm, you know, I want that to be in my subspace, I want that to be in my subspace, but therefore I have to be able to multiply them by anything.
Zero zero zero has got to be in my subspace.
I have to be able to add them so that four five five is in the subspace.
I've got to be able to add one of these plus three of these.
That'll give me some other vector.
I have to be able to take all the linear combinations.
So these are columns in R^3 and all there linear combinations form a subspace.
What do I mean by linear combinations?
I mean multiply that by something, multiply that by something and add.
The two operations of linear algebra, multiplying by numbers and adding vectors.
And, if I include all the results, then I'm guaranteed to have a subspace.
I've done the job.
And we'll give it a name -- the column space.
Column space.
And maybe I'll call it C of A.
C for column space.
There's an idea there that -- Like, the central idea for today's lecture is -- got a few vectors.
Not satisfied with a few vectors, we want a space of vectors.
The vectors, they're in -- these vectors in -- are in R^3 , so our space of vectors will be vectors in R^3.
The key idea's -- we have to be able to take their combinations.
So tell me, geometrically, if I drew all these things -- like if I drew one two four, that would be somewhere maybe there.
If I drew three three one, who knows, might be -- I don't know, I'll say there.
There's column one, there's column two.
What else -- what's in the whole column space?
How do I draw the whole column space now?
I take all combinations of those two vectors.
Do I get -- well, I guess I actually listed the possibilities.
Do I get the whole space?
Do I get a plane?
I get more than a line, that's for sure.
And I certainly get more than the zero vector, but I do get the zero vector included.
What do I get if I combine -- take all the combinations of two vectors in R^3 ?
So I've got all this stuff on -- that whole line gets filled out, that whole line gets filled out, but all in-between gets filled out -- between the two lines because I -- I allowed to add something from one line, something from the other.
You see what's coming?
I'm getting a plane.
That's my -- and it's through the origin.
Those two vectors, namely one two four and three three one, when I take all their combinations, I fill out a whole plane.
Please think about that.
That's the picture you have to see.
You sure have to see it in R^3 , because we're going to do it in R^10, and we may take a combination of five vectors in R^10, and what will we have?
God knows.
It's some subspace.
We'll have five vectors.
They'll all have ten components.
We take their combinations.
We don't have R^5 , because our vectors have ten components.
And we possibly have, like, some five dimensional flat thing going through the origin for sure.
Well, of course, if those five vectors were all on the line, then we would only get that line.
So, you see, there are, like, other possibilities here.
It depends what -- it depends on those five vectors.
Just like if our two columns had been on the same line, then the column space would have been only a line.
Here it was a plane.
Okay.
I'm going to stop at that point.
That's the central idea of -- the great example of how to create a subspace from a matrix.
Take its columns, take their combinations, all their linear combinations and you get the column space.
And that's the central sort of -- we're looking at linear algebra at a higher level.
When I look at A -- now, I want to look at Ax=b.
That'll be the first thing in the next lecture.
How do I understand Ax=b in this language -- in this new language of vector spaces and column spaces.
And what are other subspaces?
So the column space is a big one, there are others to come.
Okay, thanks.
Okay.
This is lecture six in linear algebra, and we're at the start of this new chapter, chapter three in the text, which is really getting to the center of linear algebra.
And I had time to make a first start on it at the end of lecture five.
But now is lecture six is officially the lecture on vector spaces and subspaces.
And then especially -- there are two subspaces that we're specially interested in.
One is the column space of a matrix, the other is the null space of the matrix.
So, I got to tell you what those are.
Okay.
So, first to remember from lecture five, what is a vector space?
It's a bunch of vectors that -- where I'm allowed -- where I can add -- I can add any two vectors in the space and the answer stays in the space.
Or I can multiply any vector in the space by any constant and the result stays in the space.
So that's -- in fact if I combine those two into one, you can see that -- if I can add and I can multiply by numbers, that really means that I can take linear combinations.
So the quick way to say it is that all linear combinations, C -- any multiple of V plus any multiple of W stay in the space.
So, can I give you examples that are vector spaces and also some examples that are not, to make that point clear?
So, suppose I'm in three dimensions.
Then one way to get us one space is the whole three dimensional space.
So the whole space R^3, three dimensional space, would be a vector space, because if I have a couple of vectors I can add them and I'm certainly okay and they follow all the rules.
So R^3 is easy.
Now I'm interested also in subspaces.
So there's this key word, subspaces.
That's a space -- that's some vectors inside the given space, inside R three that still make up a vector space of their own.
It's a vector space inside a vector space.
And the simplest example was a plane.
So, like, can I just sketch it -- there is a plane.
It's got to go through the origin, and of course it goes infinitely far.
That's of that's a subspace now.
Do you see that if I have two vectors on the plane and I add them, the result stays in the plane.
If I take a vector in the plane and I multiply by minus two, I'm still in the plane.
So that plane is a subspace.
So let me just make that point.
Plane through zero, through that zero zero zero is a subspace.
Okay.
And also, another subspace would be a line.
A line through zero zero zero -- yeah, the line has to go through the origin.
All subspaces have got to contain the origin, contain zero -- the zero vector.
So this line is a subspace.
Really, if I want to say it really correctly, I should say a subspace of R^3.
That of R^3 was, like, understood there.
Now -- so let me call this plane P. And let me call this line L. And let me ask you about other sets of vectors.
Suppose I took -- yeah -- so here's a first question.
Suppose I take two subspaces, like P and L. And I just put them together, take their union, take all the vectors -- so now you've got P and L in mind, here.
So I have two subspaces.
I have two subspaces and, for example, P -- a plane and L a line.
Okay.
Now I want to ask you about the union of those.
So P union L. This is all vectors in P or L or both.
Is that a subspace?
Is this a subspace?
This is or is not a subspace?
Because we're -- I just want to be sure that I've got the central idea.
Suppose I take the vectors in the plane and also the vectors on that line, put them together, so I've got a bunch of vectors, is it a subspace?
Can you give me, like, so the camera can hear it or maybe the tape.
Can you say yes or no?
Do I have a subspace if I put -- if I take all the vectors on the plane plus all -- and all the ones on the line and just join them together -- but I'm not taking this guy that's -- actually, I'm not taking most of them, because most vectors are not on the line or the plane, they're off somewhere else.
Do I have a subspace?
No
Right.
Thank you.
No.
Because -- why not?
Because I can't add.
Because if I that requirement isn't satisfied.
If I take one vector like this guy and another vector that happens to come from L and add, I'm off somewhere else.
You see that I've gone outside the union if I just add something from P and something from L, then normally what'll happen is I'm outside the union -- and I don't have a subspace.
So the correct answer is -- is not.
Okay.
Now let me ask you about -- the other thing we do is take the intersection.
So what does intersection mean?
Intersection means all vectors that are in both P and L. Is this a subspace.
Yeah, so I guess I want to go back up to the same question.
This is or is not a subspace?
And you can answer me -- answer the question first for this particular example, this picture I drew.
What is P intersect L for this case?
It's only zero
It's only zero.
At least, sort of this was the artist's idea as he drew it that, that that line L was not in the plane and, went off somewhere else -- and then the only point that was in common was the zero vector.
Is the zero vector by itself a subspace?
Yes
Yes, absolutely.
And what about, if I don't have this plane and this line but any subspace and any other subspace?
So now -- can I ask that question for any two subspaces?
So maybe I'll write it up here.
If I'm strong enough.
Okay.
So this is the general question.
I have subspaces, say S and T. And I want to ask you about their intersection S intersect T and I want -- it is a subspace.
Do you see why?
Do you see why if I take the vectors that are in both one- th- that are in both of the subspaces -- so that's like a smaller set of vectors, probably, because it's -- we've added requirements.
It has to be in S and in T. How do I know that's a subspace?
Can we just think through that abstract stuff and then I get to the examples.
Okay.
So why?
Suppose I take a couple of vectors that are in the intersection.
Why is the sum also in the intersection?
Okay, so let me give a name to these vectors, say V and W. They're in the intersection.
So that means they're both in S. Also means they're both in T. So what can I say about V plus W?
Is it in S?
Yes.
Right?
If I take two vectors, V and W that are both in S, then the sum is in S, because S was a subspace.
And if they're both in T and I add them, then the result is also in T, because T was a subspace.
So the result V plus W is in the intersection.
It's in both and requirement one is satisfied.
Requirement two's the same.
If I take a vector that's in both, I multiply by seven.
Seven times that vector is in S, because the vector was in S. Seven times that vector's in T because the original one was in T. So seven times that vector is in the intersection.
In other words, when you take the intersection of two subspaces, you get probably a smaller subspace, but it is a subspace.
Okay.
So that's like sort of just emphasizing what these two requirements mean.
Again -- Let me circle those, because those are so important.
The sum and the scale of multiplication which combines into linear combinations.
That's what you have to do inside the subspace.
Okay.
On to the column space.
Okay.
So my lecture last time started that and I want to continue it.
Okay.
Column space of a matrix.
Of A.
Okay.
Can I take an example?
Say one two three four.
One one one one.
Two three four five.
Okay.
That's my matrix A.
So, it's got columns, three columns.
Those columns are vectors, so the column space of this A, of this A -- let's stay with this example for a while.
The column space of this matrix is a subspace of R -- R what?
So what space are we in if I'm looking at the columns of this matrix?
R^4 , right?
These are vectors in R^4, they're four dimensional vectors.
So it's this column space of A is a subspace of R^4 here, because A was four by -- A is a four by three matrix.
This tells me how many rows there are, how many components in a column, and so we're in R^4.
Okay, now what's in that subspace?
So the column space of A, it's a subspace of R^4.
I call it the column space of A, like that.
So that's my little symbol for some subspace of R^4.
What's in that subspace?
Well, that column certainly is.
One two three four.
This column is in.
This column is in, and what else?
So it's got the columns of A in it, but that's not enough, certainly.
Right?
I don't have a subspace if I just put in three vectors.
So how do I fill that out to be a subspace?
I take their linear combinations.
So the column space of A is all linear combinations -- combinations of the columns.
And that does give me a subspace.
It does give me a vector space, because if I have one linear combination and I multiply by eleven, I've got another linear combination.
If I have a linear combination, I add to another linear combination I get a third combination.
So that -- this is like the smallest space -- like, it's got to have those three columns in it, and it has to have their combinations and that's where we stop.
Okay.
Now I'm going to be interested in that space.
So I, like -- get some idea of what's in that space.
How big is that space?
Is that space the whole four dimensional space?
Or is it a subspace inside?
Can you -- let me just see if we can get a yes or no answer sometimes without being ready for the complete proof.
What do you think?
Is the subspace that I'm talking about here, the combinations of those three guys, does that fill the full four dimensional space?
Maybe yes or no on that one.
No.
No.
Somehow our feeling is, and it happens to be right, that if we start with three vectors and take their combinations, we can't get the whole four dimensional space.
Now -- so somehow we get a smaller space.
But how much smaller?
That's going to come up here.
That's not so immediate.
Let me first make this critical connection with -- with, linear equations, because behind our abstract definition, we have a purpose.
And that is to understand Ax=b.
So suppose I make the connection -- w- w- does A x=b always have a solution for every b?
Have a solution for every right-hand side?
I guess that's going to be a yes or no question.
And then I'm going to ask which right-hand sides are okay?
That's really the question I'm after, is which right-hand sides (b) do make up -- you can see from the way I'm speaking what the question -- As it is.
The answer is no.
A x=b does not have a solution for every b.
Why do I say no?
Because A x=b is -- like, this is four equations, and only three unknowns.
Right?
X is -- let me right out that whole -- what the whole thing looks like.
Yeah.
Let me write out A x=b.
A x is -- these columns are one two three four.
One one one one and two three four five.
Then x, of course, has three components, x1, x2, x3.
And I'm trying to get the -- hit the right-hand side, b1,b2,b3 and b4.
So my first point is, I can't always do it.
In a way, that just says again what you told me five minutes ago -- that the combinations of these columns don't fill the whole four dimensional space.
There's going to be some vectors b, a lot of vectors b, that are not combinations of these three columns, because the combinations of those columns are, like, going to be just a little plane or something inside -- inside R^4.
Now, so and you see that I do have four equations and only three unknowns.
So, like anybody is going to say, no you dope, you can't usually solve four equations with only three unknowns.
But now I want to say sometimes you can.
For some right-hand sides, I can solve this.
So that's the bunch of right-hand sides that I'm interested in right now.
Which right-hand sides allow me to solve this?
This is the question for today.
It's going to have, like, a nice clear answer.
So my question is -- is which bs, which vectors b, allow this system to be solved?
And I want to ask you -- so that's, like, gets two question marks to indicate that's -- this is the important question.
Okay, first, before we give a total answer, give me just a partial answer.
Tell me one right-hand side that I know I can solve this thing for.
So -- all zeroes.
Okay.
That's the, like, guaranteed.
If these were all zero, then I know I can solve it, let the x-s all be zero, no problem.
So that's always a -- okay.
Okay.
A x=0 I can always solve.
Now tell me another right-hand side, just a specific set of numbers for which I can solve these three -- these four equations with only three unknowns, but if you give me a good right-hand side, I can do it.
So tell me one?
1 2 3 4
1 2 3 4?
If I -- can I solve -- is that a good right-hand side?
Can you solve -- can you find a solution that -- X one plus X two plus two X three is one, two X one plus X two plus three X three is two and two more equations -- so I'm asking you to solve in your head in -- within five seconds, four equations and three unknowns, but you can do it, because the right-hand side is, like, showing up here is -- it's one of the columns.
So tell me what's the X that does solve it?
One zero zero.
One zero zero solves it, because -- well, so you can multiply this out by rows, but oh God, it's much nicer to say -- okay, this is one of this column, zero of this, zero of this, so it's one of that column, which is exactly what we wanted.
Okay.
So there is a b that's okay.
Now tell me another B that's okay, another right-hand side that would be all right?
Well -- all ones?
Actually -- and then what's the solution in that case?
0 1 0, thanks.
And, in fact, it's much e- like, one way to do it is think of a solution first, right, and then just see what b turns out to be.
What b turns out to be, right.
Okay.
So I think of a solution -- so I think of an x, I think of any -- x1, x2, x3, I do this multiplication and what have I got?
Now I'm ready to answer the big question.
I can solve A x=b exactly when the right-hand side B is a vector in the column space.
Good.
I can solve A x=b when b is a combination of the columns, when it's in the column space -- so let me write that answer down.
I can solve Ax=b exactly when B is in the column space.
Let me just say again why that is.
Because it -- the column space by its definition contains all the combinations.
It contains all the Ax-s.
The column space really consists of all vectors A times any X.
So those are the bs that I can deal with.
If b is a combination of the columns, then that combination tells me what X should be.
If b is not a combination of the columns, then there is no x.
There's no way to solve A x equal b.
Okay.
So the column space -- that's really why we're interested in this column space, because it's the central guy.
It says when we can solve, and that -- we got to understand this column space better.
Let's see.
Do I want to think -- yeah, somehow -- oh, well, let's just -- as long as we've got it here, what do I get for this particular example?
If I take combinations of this and this and this, I'll tell you the question that's in my mind.
It's not even proper to use this word yet, but you'll know what it means.
Are those three columns independent?
If I take the combinations of the three columns -- does each column contribute something new or now?
So that if I take the combinations of those three columns, do I, like, get some three dimensional subspace -- do I have three vectors that are, like, you know, independent, whatever that means?
Or do I -- is one of those columns, like, contributing nothing new -- So that actually only two of the columns would have given the same column space?
Yeah -- that's a good way to ask the question.
Finally I think of it.
Can I throw away any columns -- and have the same column space?
Yes
Yes.
And which one do you suggest I throw away?
Column three -- three
Well, three is the natural, like, guy to target.
So if I -- and why?
Because -- what's so bad about three here?
Column three?
It's the sum of these, right?
So it's not -- if I'm taking -- if I have combinations of these two and I put in this one, actually, I don't get anything more.
So later on I will call these pivot columns.
And the third guy will not be a pivot column in this -- with those numbers.
Now actually -- honesty makes me ask you this question.
Could I have thrown away column one?
Yes, I could.
I could.
So when I say pivot columns, my convention is, okay, I'll keep the first ones as long as they're not dependent.
So I keep this guy, he's fine, he's a line.
I keep the second guy.
It's in a second direction.
But the third one, which is in the same plane as the first two gives me nothing new.
It's dependent in the language that we will use and I don't need it.
Okay.
So I would describe the column space of this matrix as a two dimensional subspace of R^4.
A two dimensional subspace of R^4.
Okay.
So you're seeing how these vector spaces work and you -- you're seeing that we -- some idea of dependence or independence is in our future.
Okay.
Now I want to speak about another vector space, the null space.
So again I'm getting a little ahead because it's in section three point two, but that's okay.
All right.
Now I'm ready for the null space.
Let me keep the same matrix.
And this is going to be a different -- totally different subspace.
Totally different.
Okay.
Now -- so let me make space for it.
Now -- here comes a completely different subspace, the null space of A.
What's in it?
It contains not right-hand sides b.
It contains x-s.
It contains all x-s that solve -- this word null is going to -- I mean, that's the key word here, meaning zero.
So this contains -- this is all solutions x, and of course x is our vectors, x1, x2 and x3, to the equation A x=0.
Well, four equations, because we've got -- so, do you see what I'm doing?
I'm now saying, okay, columns were great, the column space we understood.
Now I'm interested in x-s.
I'm not -- the only b I'm interested in now is the b of all zeroes.
The right-hand side is now zeroes.
And I'm interested in solutions.
x-s.
So t- where is this null space for this example?
These x-s are -- have three components.
So the null space is a subspace -- we still have to show it is a subspace -- of R^3.
So this is -- and we will show -- these vectors x, this is in R^3, where the column space was in R^4 in our example.
For an m by n matrix, this is m and this is n, because the number of columns, n, tells me how many unknowns, how many x-s multiply those columns, so it tells me the big space, in this case R three that I'm in.
Now tell me -- why don't we figure out what the null space is for this example, just by looking at it.
I mean, that's the beauty of small examples, that my official way to find null spaces and column spaces and get all the facts straight would be elimination, and we'll do that.
But with a small example, we can see that -- see what's going on without going through the mechanics of elimination.
So this null space -- so I'm talking about -- again, the null space, and let me copy again the matrix.
One two three four, one one one one and two three four five.
What's in the null space?
So I'm taking A times x, so let me right it again, and I want you to solve those four equations.
In fact, I want you to find all solutions to those four equations.
Well, actually, just first of all find one.
Why should I ask you for all of them?
Tell me one -- well, tell me one solution that y- you don't even have to look at the matrix to know one solution to this set of equations.
It is zero vector.
Whatever that matrix is, its null space contains zero -- because A times the zero vector sure gives the zero right-hand side.
So the null space certainly contains zero.
A- so it's got a chance to be a vector space now, and it will turn out it is.
Okay.
Tell me another solution.
So this particular null space -- and of course I'm going to call it N(A) for null space -- this contains-- well we've already located the zero vector, and now you're going to tell me another vector that's in the null space, another solution, another x, another -- you see what I'm asking you for is a combination of those columns.
That's what I'm always looking at combinations of columns, but now I'm looking at the weights, the coefficients in the combination.
So tell me a good set of numbers to put in there.
One one --
Minus one
One one minus one.
Thanks.
One one minus one.
So there's a vector that's in it.
Okay.
But have I got a subspace at this point?
Certainly not, right?
I've got just a couple of vectors.
No way they make a subspace.
Tell me -- actually, why don't I jump the whole way now?
Tell me -- well, tell me one more solution, one more X that would work.
Student: 2 2 -2
2 2 -2?
Oh, well, tell me all of them, that would have been easier.
Tell me the whole lot, now.
What is the null space for this matrix?
It's all vectors of the form -- what could this be?
It could be one one minus one, it could be it could be any number C, any number -- the same number again and --
Minus
Minus C. In other words -- actually, any multiple of this guy.
Oh, that's the perfect description, because now the zero vector's automatically included because C could be zero.
The vector I had is included, because C could be one.
But now any vector.
And that's actually it.
And do I have a subspace?
And what does it look like?
It's in -- how would you describe this, the null space, this -- all these vectors of this form C C minus C, like, seven seven minus seven.
Minus eleven minus eleven plus eleven.
What have I got here?
If -- describe that whole null space of -- what -- if I drew it, what do I draw?
A line, right?
The null space is a line.
It's the line through -- in R^3 and the vector one one negative one maybe goes down here, I don't know where it goes, say, down here.
There's the vector one one negative one that you gave me.
And where is the vector C C negative C?
It's on this line.
Of course, there's zero zero zero that we had.
And what we've got is that whole -- oops -- that whole line, going both ways, through the origin.
The null space is a line in R^3.
Okay.
For that example, we could find all the combinations of the columns that gave zero at sight.
Now, can I just take one more time, to go back to the definition of subspace, vector space, and ask you -- how do I know that the null space is a vector space?
How I entitled to use this word space?
I'll never use that word space without meaning that the two requirements are satisfied.
Can we just check that they are?
So I'm going to check that -- can I just continue here?
Check that -- that the solutions to A x=0 always give a subspace.
And, of course, the key word is that= "Space."
So what do I have to check?
I have to show that if I have one solution, call it x, and another solution, call it x*, that their sum is also a solution, right?
That's a requirement.
To use that word space, I have to say -- I have to convince myself that if A x is zero and also -- and A x* is zero, or maybe I should have said if A v is zero and A w is zero, then what about v plus w?
Shall I -- let me use those letters.
If A v is zero and A w is zero, then what -- if that and that, then what's my point here?
That A times (v+w) must be zero.
That says that if v is in the null space and w's in the null space, then their sum v+w is in the null space.
And of course, now that I've written it down, it's totally absurd, ridiculously simple -- because matrix multiplication allows me to separate that out into A v plus A w. I shouldn't say absurdly simple.
That was a dumb thing to say.
Could -- we've used, here, a basic law of matrix multiplication.
Actually, we've used it without proving it, but that's okay.
We only live so long, we just skip that proof.
I think it's called the distributive law that I can split these -- split this into two pieces.
But now you see the point, that A v is zero and A w is zero so I have zero plus zero and I do get zero.
It checks.
And, similarly, I have to show that if A v is zero, then A times any multiple, say 12v is also zero.
And how do I know that?
Because I'm allowed to s- bring that twelve outside.
A number, a scaler can move outside, so I have twelve A vs, twelve zeroes -- I have zero.
Okay.
Just to -- it's really critical to understand the -- oh yeah.
Here -- I was going to say, understand what's the point of a vector space?
Let me make that point by changing the right-hand side.
Oops.
Okay.
Let me change the right-hand side to one two three four.
Oh, okay.
Why don't we do all of linear algebra in one lecture, then we -- okay.
I would like to know the solutions to this equation.
For those four equations.
So I have four equations.
I have only three unknowns, so if I don't have a pretty special right-hand side there won't be any solution at all.
But that is a very special right-hand side.
And we know that there is a solution, one zero zero.
Were there any more solutions?
And did they form a vector space?
Okay.
So I'm asking two questions there.
One is, do -- so my right-hand side now is not zero anymore.
I'm not looking at the null space because I changed from zeroes.
So my first question is, do the solutions, if there are any and there are, do they form a subspace?
Let's answer that question first.
Yes or no.
Do I get a subspace if I look at the solutions to -- let me go back to x1 x2 x3.
I'm looking at all the x-s, at all those vectors in R^3 that solve A x -b.
The only thing I've changed is b isn't zero anymore.
Do the x-s, the solutions, form a vector space?
The solutions to this do not form a subspace.
The solutions don't, because -- how shall I see that?
The zero vector is not a solution, so I never even got started.
The zero vector doesn't solve this system.
I can't -- solutions can't be a vector space.
Now what are they like?
Well, we'll see this, but let's do it for this example.
So one zero zero was a solution.
You saw that right away.
Are there any other solutions?
Can you tell me a second solution to this system of equations?
0 -1 1
0 -1 1.
Boy, that's -- 0 -1 1.
Yes.
Because that says I take minus this column plus this one and sure enough.
That's right.
So there are -- there's a bunch of solutions here.
But they're not a subspace.
I'll tell you what it's like.
It's like a plane that doesn't go through the origin, or a line that doesn't go through the origin.
Maybe in this case it's a line that doesn't go through the origin, if I graft the solutions to A x equal B.
So you -- I think you've got the idea.
Subspaces have to go through the origin.
If I'm looking at x-s, then they'd better solve Ax=0.
In a way I've got -- my two subspaces that I -- talking about today are kind of the two ways I can tell you what a -- about subspace.
If I want to tell you about the column space, I tell you a few columns and I say take their combinations.
Like I build up this subspace.
I put in a few vectors, their combinations make a subspace.
Now, when I went to -- let me come back to the one that is a subspace here.
Here, when I talked about the null space, I didn't tell you what's in it.
We had to figure out what was in it.
What I told you was the equations that I'm -- that has to be satisfied.
You see those -- like, those are the two natural ways to tell you what's in a subspace.
I can either give you a few vectors and say fill it out, take combinations -- or I can give you a system of equations, the requirements that the x-s have to satisfy.
And both of those ways produce subspaces and they're the important ways to construct subspaces.
Okay, so today's lecture actually got, the essentials of three point two, the idea of the null space.
Now we have to tackle, Wednesday, the job of how do we actually get hold of that subspace in an example that's bigger and we can't see it just by eye.
Okay.
See you Wednesday.
Thanks.
OK, here's linear algebra lecture seven.
I've been talking about vector spaces and specially the null space of a matrix and the column space of a matrix.
What's in those spaces.
Now I want to actually describe them.
How do you describe all the vectors that are in those spaces?
How do you compute these things?
So this is the, turning the idea, the definition, into an algorithm.
What's the algorithm for solving A x =0?
So that's the null space that I'm interested in.
So can I take a particular matrix A and describe the natural algorithm, and I'll execute it for that matrix -- here we go.
So let me take the matrix as an example.
So we're definitely talking rectangular matrices in this chapter.
So I'll make, I'll have four columns.
And three rows.
Two four six eight and three six eight ten.
OK.
If I just look at those columns, and rows, well, I notice right away that column two is a multiple of column one.
It's in the same direction as column one.
It's not independent.
I'll expect to discover that in the process.
Actually, with rows, I notice that that row plus this row gives the third row.
So the third row is not independent.
So, all that should come out of elimination.
So now what I -- my algorithm is elimination, but extended now to the rectangular case, where we have to continue even if there's zeros in the pivot position, we go on.
OK, so let me execute elimination for that matrix.
My goal is always, while I'm doing elimination -- I'm not changing the null space.
That's very important, right?
I'm solving A x equals zero by elimination, and when I do these operations that you already know, when I subtract a multiple of one equation from another equation, I'm not changing the solutions.
So I'm not changing the null space.
Actually, I changing the column space, as you'll see.
So you have to pay attention.
What does elimination leave unchanged?
And the answer is the solutions to the system are not changed because I'm doing the same thing to -- I'm doing a legitimate operations on the equations.
Of course, on the right hand side it's always zero, and I don't plan to write zero all the time.
OK, so I'm really just working on the left side, but the right side is, is keeping up always zeros.
OK, so what's elimination?
Well, you know where the first pivot is and you know what to do with it.
So can I just take the first step below here?
So that pivot row is fine.
I take two times that row away from this one and I get zero zero.
That's signaling a difficulty.
Two, two twos away from the six leaves me with a two.
Two twos away from the eight leaves me with a four.
And now three of those away from here is zero, again another zero, three twos away from that eight is the two, three twos away from that ten is a four.
OK. That's the first stage of elimination.
I've got the first column straight.
So of course I move on to the second column.
I look in that position, I see a zero.
I look below it, hoping for a non-zero that I can do a row exchange.
But it's zero below.
So that's telling me that that column is -- well, what it's really going to be telling me is that that column is a combination of the earlier columns.
It's that second column is dependent on the earlier columns.
But I don't stop to think here.
In that column there's nothing to do.
I go on to the next.
So here's the next pivot.
So there's the first pivot and there's the second pivot, and I just keep this elimination going downwards.
So, so the next step keeps the first row, keeps the second row with its pivot, so I've got my two pivots, and does elimination to clear out the column below that pivot.
So actually you see the multiplier is one.
It subtracts row two from row three and produces a row of zeros.
OK. That I would call that matrix U, right?
That's our upper -- well, I can't quite say upper triangular.
Maybe upper -- I don't know -- upper something.
It's in this so-called echelon form.
The word echelon means, like, staircase form.
It's the, the non-zeros come in that staircase form.
If there was another pivot here, then the staircase would include that.
But here's a case where we have two pivots only.
OK, so actually we've already discovered the most important number about this matrix.
The number of pivots is two.
That number we will call the rank of the matrix.
So let me put immediately.
The rank of A -- so I'm telling you what this word rank means in the algorithm.
It's equal to the number of pivots.
And in this case, two.
OK, for me that number two is the crucial number.
OK, now I go to -- you remember I'm always solving A x equals zero, but now I can solve U x equals zero, right?
Same solution, same null space.
OK.
So I could stop here -- why don't I stop here and do the back substitution.
So now I have to ask you, how do I describe the solutions?
There are solutions, right, to A x equals zero.
I knew there would be.
I had three equations in four unknowns.
I certainly expected some solutions.
Now I want to see what are they.
OK, here's the critical step.
I refer to it up here as separating out the pivot variables, the pivot columns, which are these two.
Here I have two pivot columns.
Those, obviously, they're the columns with the pivots.
So I have two pivot columns.
And I have the other columns, I'll call free.
These are free columns, OK. Why do I use those words?
Why do I use that word free?
Because now I want to write, I want to find the solutions to U x equals zero.
Here is the way I do it.
These free columns I can assign any number freely to those -- to the variables x2 and x4, the ones that multiply columns two and four.
So I can assign anything I like to x2 and x4.
And then I can solve the equations for x1 and x3.
Let me say that again.
If I give -- let me, let me assign.
So, so one particular x is to assign, say, the value one to the, to x2, and zero to x4.
Those are -- that was a free choice, but it's a convenient choice.
OK. Now I want to solve U x equals zero and find numbers one and three, complete the solution.
Can I write down -- let's see.
OK. Shall we just remember what U x equals zero represents?
What are my equations?
That first equation is x1 plus just -- I'm just saying what are these matrices meaning.
That's the first equation.
And the second equation was 2x3 + 4x4=0.
Those are my two equations.
OK. Now the point is I can find x1 and x3 by back substitution.
So we're building on what we already know.
The new thing is that there were some free variables that I could give any numbers to.
And I'm systematically going to make a choice like this, Now what is x3?
1 and 0.
Let's, let's go backwards, back up.
I look at the last equation.
x3 is zero, from the last equation, because, because x4 we've set x4 to zero, and then we get x3 as zero.
OK. Now we set x2 to be one, so what is x1?
Negative two, right?
So then I have negative two plus two, zero and zero, correctly giving zero.
There is a vector in the null space.
There is a solution to A x equals zero.
In fact, what solution is that?
That simply says that minus two times the first column plus one times the second column is the zero column.
Of course that's right.
Minus two of that column plus one of that, or minus two of that plus one of that.
That solution is -- that, that's just what we saw immediately, that the second column is twice as big as the first column.
OK, tell me some more vectors in the null space.
I found one.
Tell me, how to get a bunch more immediately out of that one.
Just take multiples of it.
Any multiple of -- I could multiply this by anything.
I might as well call it, I could say, C, some multiple of this.
So let me -- so X could be any multiple of this one.
OK, that, that describes now a line, an infinitely long line in four dimensional space.
But -- which is in the null space.
Is that the whole null space?
No.
I've got two free variables here.
I made this choice, one and zero, for the free variables, but I could have made another choice.
Let me make the other choice zero and one.
You see my system.
So let me repeat the system.
This is the algorithm that you, you just learned to do.
Do elimination.
Decide which are the pivot columns and which are the free columns.
That tells you that, that variables one and three are pivot variables, two and four are free variables.
Then those free variables, you assign them -- you give one of them the value one and the others the value zero -- in this case, we had a one and a zero -- and complete the solution.
And you do -- you give the other one the value one and zero.
And now complete the solution.
So let's complete that solution.
I'm looking for a vector in the null space, and it's absolutely going to be different from this guy, because, because any multiple of that zero is never going to give the one.
So I have somebody new in the null space, and let me finish it out.
What's x3 here?
So we're going by back substitution, looking at this equation.
Now x4 we've changed, we're doing the other possibility, where x2 is zero and x4 is one.
So x3 will happen to be minus two.
And now what do I get for that first equation?
x1 -- let's see.
Two x3s is minus four plus two -- do I get a two there?
Perhaps, yeah.
Two for x1, minus four, and two.
OK. That's in the null space.
What does that say about the columns?
That says that two of this column minus two of this column plus this column gives zero, which it does.
Two of that minus two of that and one of that gives the zero column.
OK, now, now I've found another vector in the null space.
Now we're ready to tell me the whole null space.
What are all the solutions to Ax=0?
I've got this guy and when I have him, what else is, goes into the null space along with that?
These are my two special solutions.
I call them special -- I just invented that name.
Special solutions.
What's special about them is the special numbers I gave to the free variables, the values zero and one for the free variables.
I could have given the free variables any values and got vectors in the null space.
But this was a good way to be sure I got t- got everybody.
OK, so once I have him, I also have any multiple, right?
So I could take any multiple of that and it's in the null space.
And now what else -- I left a little space for what?
What -- a plus sign.
I can take any combination.
Here is a line of vectors in the null space.
A bunch of solutions.
Would you rather I say in the null space or would you rather I say, OK, I'm solving Ax=0?
Well, really I'm solving Ux=0.
Well, OK, let me put in that crucial plus sign.
I'm taking all the combinations of my two special solutions.
That's my conclusion there.
The null space contains, contains exactly all the combinations of the special solutions.
And how many special solutions are there?
There's one for every free variable.
And how many free variables are there?
Oh, I mean, we can see all the whole picture now.
If the rank R was two, this is the, this is the number of pivot variables, right, because it counted the pivots.
So how many free variables?
Well, you know it's two, right?
What is it in -- for a matrix that's m rows, n columns, n variables that means, with rank r?
How many free variables have we got left?
If r of the variables are pivot variables, we have n-r -- in this case four minus two -- free variables.
Do you see that first of all we get clean answers here?
We get r pivot variables -- so there really were r equations here.
There looked like three equations, but there were really only two independent equations.
And there were n-r variables that we could choose freely, and we gave them those special zero one values, and we got the special solutions.
OK. For me -- we could stop at that point.
That gives you a complete algorithm for finding all the solutions to A x equals zero.
OK. Again, you do elimination -- going onward when a column, when there's nothing to be done on one column, you just continue.
There's this number r, the number of pivots, is crucial, and it leaves n-r free variables, which you give values zero and one to.
I would like to take one more step.
I would like to clean up this matrix even more.
So now I'm going to go to -- this is in its, this is in echelon form, upper triangular if you like.
I want to go one more step to make it as good as it can be.
OK, so now I'm going to speak about the reduced row echelon form.
OK, so now I'm going to speak about the matrix R, which is the reduced row echelon form.
So what does that mean?
That means I just -- I can, I can work harder on U.
So let me start, let me suppose I got as far as U, which was good.
Notice how that row of zeros appeared.
I didn't comment on that, but I should have.
That row of zeros up here is because row three was a combination of rows one and two, and elimination discovered that fact.
When we get a row of zeros, that's telling us that the -- original row that was there was a combination of other rows, and elimination knocked it out.
OK, so we got this far.
Now how can I clean that up further?
I can do, elimination upwards.
I can get zero above the pivots.
So this reduced row echelon form has zeros above and below the pivots.
So let me do that.
So now I'll subtract one of this from the row above.
That will leave a zero and a minus two in there.
And that's good.
OK, and I can clean it up even one more step.
I can make the pivots -- the pivots I'm going to make equal to one, because I can divide equation two by the pivot.
That won't change the solutions.
So let me do that.
And then I really -- I'm ready to stop.
One, two, zero, minus two, zero, zero, one, two.
I divided the second equation by two, because now I have a one in the pivot and zeros below.
OK.
This is my matrix R. I guess I'm hoping that you could now execute the whole algorithm.
Matlab will do it immediately with the command -- reduced row echelon form of A.
So if I input that original matrix A and then I write, then I type that command, press return, that matrix will appear.
That's the reduced row echelon form, and it's got all the information as clear as can be.
What, what information has it got?
Well, of course it immediately tells me the pivot rows, pivot rows, one and two, pivot columns, one and three.
And in fact it's got the identity matrix in there, It's, it's got zeros above and below the pivots, right?
and the pivots are one, so it's, so it's got a -- so notice the two by two identity matrix that's sitting in the pivot rows and pivot columns.
it's I in the pivot rows and columns.
It's got zero rows below.
Those are always indicating that original rows were, were combinations of other rows.
So we really only had two rows there.
And now it also -- so there's the identity.
Now it's also got its free columns.
And, they're cleaned up as much as possible.
Actually, actually it's now so cleaned up that the special solutions, I can read off -- you remember I went through the steps of computing this -- doing back substitution?
Let me, let me, instead of that system, let me take this improved system.
So I'm going to use these numbers, right.
In these equations, what did I do?
I divided this equation by two and, oh yeah and I had subtracted two of this, which knocked out this guy and made that a minus sign.
Is that what -- I've now written Rx equals zero.
Now I guess I'm hoping everybody in this room understands the solutions to the original A x equals zero, the midway, halfway, U x equals zero, and the final R x equals zero are all the same.
Because going from one of those to another one I didn't mess up.
I just multiplied equations and subtracted from other equations, which I'm allowed to do.
OK.
But my point is that now if I do this free variables and back substitution, it's just, the numbers are there.
When I let x -- so in this guy, I let x2 be one and x4 be zero.
I, I guess, what I seeing here?
Let me, let me sort of separate this out here.
I'm seeing in the pivot, in the pivot columns, if I, if I put the pivot columns here, I'm seeing those.
And I'm -- in the free columns I'm seeing -- what I seeing in the free columns?
A two, zero in that first free column, the x2 column, and a minus two, two in the fourth column, the other free column.
And the row of zeros below, which of course have no -- that equation is zero equals zero.
That's satisfied.
Here's my point.
That when I do back substitution, these numbers are exactly what shows up -- oh, their signs get switched.
I was going to say those numbers, two, minus two, zero, two, can I just circle the -- this is the free part of the matrix.
This is the identity part.
This is the free part, maybe I'll call it F. This, of course, I call I, because it's the identity.
The free part is a, I mean, I'm just doing back substitution.
And those free numbers will show up, with a minus sign, because they pop onto the other side of the equation -- so I see minus two, zero, and I see two, minus two.
So that wasn't magic.
It had to happen.
Let me, show you clearly why it happened.
OK, so that's -- this is what I'm interested in here.
And now let me, let me just, like, do it, do it for -- let's suppose we've, we've got to -- let's suppose we've got this system already in, in rref form.
So my matrix R is -- what does it look like?
OK, and I'll -- let me pretend that the pivot columns come first and then whatever's in the free columns.
And there might be some zero rows below.
There's a typical -- a pretty typical reduced row echelon form.
You see what's typical.
It's got -- this is r by r. This is r pivot rows.
This is r pivot columns.
And here are n-r free columns.
OK. Tell me what are the special solutions?
What are the -- what's x?
If I want to solve R x equals zero -- in fact, let me -- I'm really going to, do the whole -- since these -- this is now block matrices, I might as well do all of the special solutions at once.
So I want to solve R x equals zero, and I'll have some special solutions.
Let me, actually -- can I do them all at once?
I'm going to create a null space matrix, OK. A matrix.
Its, its, its columns are the special -- the columns are the special solutions.
This is, I'm making it sound harder, it's going to be totally easy.
N will be this null space matrix.
I want R N to be the zero matrix.
These columns of N are supposed to multipl- to get multiplied by R and give zero columns.
So what N will do the job?
Let me put -- I'm going to put the identity in the free variable part and then minus F will show up in the pivot variables, just the way it did in that example.
There we had the identity and F. Here -- in the special solution.
So these columns are -- there's the matrix of special solutions.
And actually, there -- so there's a Matlab command or a teaching code command, NULL -- N equal, so this is the -- produces the null basis, the null space matrix, NULL of A, and there it is.
And how does that command actually work?
It uses Matlab to compute R, then it picks out the pivot variables, the free variables, puts, puts ones and zeros in for the free variables, and copies out the pivot variables.
It, it does back substitution, but back substitution for this system is totally simple.
What is this system?
R x equals zero.
So this is R is I F, and x is the pivot variables and the free variables, and it's supposed to give zero.
So what does that mean?
That means that the pivot variables plus F times the free variables give zero.
So let me put F times the free variables on the other side.
I get minus F times the free variables.
There's my, equation, as simple as it can be.
That's what back substitution comes to when I've reduced and reduced and reduced this system to the, to the best form, OK. And, then if the free variables, I make this special choice of the identity, then the pivot variables are minus F. OK, can I do, another example?
Could you do another example?
Can I -- let me just take another matrix and, and let's go through this algorithm once more, OK.
Here we go.
Here's a blackboard for another matrix, OK.
So I'll call the matrix A again, but let me make it -- yeah, how big shall we make it this time?
Why don't I do this?
Just for the heck of it.
Let me take the transpose of this A and see what happens to that.
Two four six eight and three six eight ten.
Before we do the calculations, tell me what's coming?
How many pivot variables do you expect here?
How many columns are going to have pivots?
How many -- we have three columns in that matrix, but are we going to, are we going to have three pivots?
No, because this third columns is the sum of the first two columns.
I'm totally expecting, totally expecting that these will be pivot columns -- because they're independent, but that this third guy, the third column, which is dependent on the first two, is going to be a free column.
Elimination better discover that.
And elimination will also straighten out the rows, dependent rows and some independent rows.
What's the, what's the row reduced echelon form for this?
Let's just do it, OK.
So, so that's the first pivot.
Two times that away from that gives me a row of zeros.
Two times that away from that gives me a zero two two.
And two times that away from that gives me a zero four four.
OK, first column is straight.
First variable is a pivot variable.
No problem.
On to the second column.
I look at the second pivot, it's a zero.
I look below it.
There's a two.
OK, I do a row exchange.
So this zero is now there.
I now have a perfectly good pivot, and I use it.
OK, and I subtract two of that row away from this row.
All right if I do it like that?
I've got to the form U now.
This was my A.
Now there's my U. I can see now -- I have to stop, right?
I would go on to the third column.
I should have tried.
I quit, but without trying.
I shouldn't have done that.
On to the third column, look at the pivot position.
It's got a zero in it.
Look below, all zeros.
Now I'm entitled to stop, OK.
So the rank is two again.
What about the null space?
How many special solutions are there this time?
How many special solutions for this matrix?
I've got -- and which are the free variables and which are the pivot variables and so on?
Pivot columns, I've got two pivot columns, and that's no accident.
The number of pivot columns for a matrix A, that's a great fact, that the number of pivot columns for A and A transpose are the same.
And then I have a free column.
There's a free column.
One free column, because the count is three minus two.
Three minus two gives me one free column.
OK, so now let me solve, what's in the null space.
OK, so how do I -- let's see.
These vectors have length three.
They only have three components.
I'm making too much space for the, to write x. x has just got three components, and what are they?
I'm looking for the null space.
OK, so how do I start?
I give the free variable some convenient value.
And what's that?
I set it to one.
I set the free variable to one.
If I set the free variable to zero and solve for the pivot variables, I'll get all zeros: no progress.
But by setting the free variable to one -- you see w- my two equations now are -- my equations are x1+ 2x2+ 3 x3 is zero, that's my first equation.
And my second equation is now 2x2+2x3 equals zero.
And, OK.
So if x3 is one, then x2 is minus one.
And if x3 is one and x2 is minus one, then maybe x1 is minus one.
And actually I go back to check now.
I don't, like -- I did a quick calculation mentally.
Can I mentally do a quick check?
That matrix, that solution x says that minus this column minus this column plus this one is the zero column.
And it is.
Minus that minus that plus that is zero.
So that's in the null space.
And now you can tell me what else is in the null space.
What's, what's the whole null space now?
I multiply by C, right.
The whole null space is a line.
So that's the description.
You know, if I ask you on a homework or a quiz or the final what -- give me, describe, tell me the null space, find the null space of this matrix, you can take those steps.
And that's the answer I'm looking for.
And I'm looking for that C too, because that's telling me that you're remembering that it's a whole space and not just one vector.
Later I will ask you for a basis for the null space.
Then I just want this vector.
But if I ask for the whole null space, then there's the whole line through that vector.
OK, now one more natural thing to do with this example, right, is keep going to the reduced matrix, R. So can I push onwards to R?
That should be quick, but let's just practice.
Let me keep going to R. OK, so what do I do here?
I subtract -- I clear out above the pivot, so I subtract that from that, that's one zero one is left, right?
When I subtracted this row from this it produced a zero above this pivot.
And now I want that pivot to be a one.
So for the R matrix, I'll divide this equation by two, and of course these zero, zeros are great, they don't change.
There's R. That's R. You see what R is?
You see the identity matrix sitting up here?
You see the free part F, the F part here?
And you see the zeros below.
This is I F zero zero.
And what's the x?
The x has the identity -- well, it's only a single number one, but it's the identity matrix in the free, in the free part.
And what does it have in the pivot variables?
What did back substitution give?
It gave minus these guys.
You see that what this is is any multiple of -- this is the identity there, and this is minus F here.
This is our null space matrix N for this.
Our, our null space matrix is the guy whose columns are the special solutions.
So their free variables have the special values one and, pivot variables have minus F. So do you see, though, how the minus F just automatically shows up in the special solutions.
That's it really.
I don't think there's anything more I can say about A x equals zero.
There's more I can say about A x equal b, but that'll be on Friday.
OK, so that's, that's the null space.
Thanks.
OK, when the camera says, we'll start.
You want to give me a signal?
OK, this is lecture eight in linear algebra, and this is the lecture where we completely solve linear equations.
So Ax=b.
That's our goal.
If it has a solution.
It certainly can happen that there is no solution.
We have to identify that possibility by elimination.
And then if there is a solution we want to find out is there only one solution or are -- is there a whole family of solutions, and then find them all.
OK. Can I use as an example the same matrix that I had last time when we were looking for the null space.
So the, the matrix has rows 1 2 2 2, 2 4 6 8, and the third row -- you remember the main point was the third row, 3 6 8 10, is the sum of row one plus row two.
In other words, if I add those left-hand sides, I get the third left-hand side.
So you can tell me right away what elimination is going to discover about the right-hand sides.
What's -- there is a condition on b1, b2, and b3 for this system to have a solution.
Most cases -- if I took these numbers to be one -- 5, and 17, there would not be a solution.
In fact, if I took those first numbers to be 1 and 5, what is the only b3 that would be OK?
Six.
If the left-hand -- if these left-hand sides add up to that, then B -- I need b1 plus b2 to equal b3.
Let's just see how elimination discovers that.
But we can see it coming, right?
That if -- let me say it in other words.
If some combination on the left-hand side gives all 0s then the same combination on the right-hand side must give 0.
OK.
So let me take that example and write down instead of copying out all the plus signs, let me write down the matrix.
1 2 2 2, 2 4 6 8, and that 6 3 8 10, where the third row is the sum of the first two rows.
Now how do we deal with the right-hand side?
That's -- we want to do the same thing to the right-hand side that we're doing to these rows on the left side, so we just tack on the right-hand side as another vector, another column.
So this is the augmented matrix.
It's, it's the matrix A with the vector b tacked on.
In Matlab, that's all you would need to type.
OK.
So we do elimination on that.
Can we just do elimination quickly?
The first pivot is fine, I subtract two of this away from this, three of this away from this, so I have 1 2 2 2 b1.
Two of those away will give me 0 0 2 and 4, and that was b2 minus two b1.
I, I have to do the same thing to that third, that last column.
And then three of these away from this gave me 0 0 2 4 b3 minus three b1s.
So that's the, that's elimination with the first column completed.
We move on.
There's the first pivot still.
Here is the second pivot.
We're always remembering, now, these are then going to be the pivot columns.
And let me get the final result -- well, let me -- can I do it by eraser?
We're capable of subtracting this row from this row, just by -- that'll knock this out completely and give me the row of 0s, and on the right-hand side, when I subtract this away from this, what do I have?
I think I have b3 minus a b2, and I had minus three b1s.
This is going to, it's going to be a minus a b1.
Oh yeah that's exactly what I expect.
So now the -- what's the last equation?
The last equation, this represented by this zero row, that last equation is, says 0 equals b3 minus b2 minus b1.
So that's the condition for solvability.
That's the condition on the right-hand side that we expected.
It says that b1+b2 has to match b3, and if our numbers happen to have been 1, 5, and 6 -- so let me take, suppose b is 1 5 6.
That's an OK b.
And when I do this elimination, what will I have?
The b1 will still be a 1. b2 would be 5 minus 2, this would be a 3.
5 -- my 6 minus 5 minus 1, this will be -- this is the main point -- this will be a 0, thanks.
OK.
So the last equation is OK now.
And I can proceed to solve the two equations that are really there with four unknowns.
OK, I, I, I want to do that, so this, this b is OK.
It allows a solution.
We're going to be, naturally, interested to keep track what are the conditions on b that make the equation solvable.
So let me write down what we already see before I continue to solve it.
Let me first -- solvability, solvability.
So which -- so this is the condition on the right-hand sides.
And what is that condition?
This is solvability always of Ax=b.
So Ax=b is solvable -- well, actually, we had an answer in the language of the column space.
Can you remind me what that answer is?
That, that was like our answer from earlier lecture.
b had to be in the column space.
Solvable if -- when -- exactly when b is in the column space of A.
Right?
That just says that b has to be a combination of the columns, and of course that's exactly what the equation is looking for.
So that -- now I want to answer it -- the same answer but in different language.
Another way to answer this -- if a combination of the rows of A gives the zero row, and this was an example where it happened, some combination of the rows of A produced the zero row -- then what's the requirement on b?
Since we're going to do the same thing to both sides of all equations -- the same combination of the components of b has to give 0.
Right?
That's -- so if there's a combination of the rows that gives the zero row, then the same combination of the entries of b must give 0.
And this isn't the zero row, that's the zero number.
Tthis is another way of saying -- and it is not immediate, OK. right, that these two statements are equivalent.
But somehow they must be, because they're both equivalent to the solvability of the system.
OK.
So we've got this, this sort of -- like question zero is, does the system have a solution?
OK, I'll come back to discuss that further.
Let's go forward when it does.
When there is a solution.
And so what's our job now?
Abstractly we sit back and we say, OK, there's a solution, finished.
It exists.
But we want to construct it.
So what's the algorithm, the sequence of steps to find the solution?
That's what I -- and of course the quiz and the final, I'm going to give you a system Ax=b and I'm going to ask you for the solution, if there is one.
And so this algorithm that you want to follow.
OK, let's see.
So what's the -- so now to find the complete solution to Ax=b.
OK. Let me start by finding one solution, one particular solution.
I'm expecting that I can, because my system of equations now, that last equation is zero equals zero, so that's all fine.
I really have two equations -- actually I've got four unknowns, so I'm expecting to find not only a solution but a whole bunch of them.
But let's just find one.
So step one, a particular solution, x particular.
How do I find one particular solution?
Well, let me tell you how I, how I find it.
So this is -- since there are lots of solutions, you could have your own way to find a particular one.
But this is a pretty natural way.
Set all free variables to zero.
Since those free variables are the guys that can be anything, the most convenient choice is zero.
And then solve Ax=b for the pivot variables.
So what does that mean in this example?
Which are the free variables?
Which, which are the variables that we can assign freely and then there's one and only one way to find the pivot variables?
They're x2 and -- so x2 is zero, because that's in a column without a pivot, the second column has no pivot.
And the -- what's the other one?
The fourth, x4 is zero.
Because that, those are the, the free ones.
Those are in the columns with no pivots.
So you see what my -- so when I knock -- when x2 and x4 are zero, I'm left with the -- what I left with here?
I'm just left with -- see, now I'm not using the two free columns.
I'm only using the pivot columns.
So I'm really left with x1 -- the first equation is just x1 and two x3s should be the right-hand side, which we picked to be a one.
And the second equation is two x3s, as it happened, turned out to be, three.
I just write it again here with the x2 and the x4 knocked out, since we're set them to zero.
And you see that we're back in the normal case of having back -- where back substitution will do it.
So x3 is three halves, and then we go back up and x1 is one minus two x3.
That's probably minus two.
Good.
So now we have the solution, x particular is the vector minus two zero three halves zero.
OK, good.
That's one particular solution, and we should and could plug it into the original system.
Really if -- on the quiz, please, it's a good thing to do.
So we did all this, these, row operations, but this is supposed to solve the original system, and I think it does.
OK.
So that's x particular which we've got.
So that's like what's new today.
The particular solution comes -- first you check that you have zero equals zero, so you're OK on the last equations.
And then you set the free variables to zero, solve for the pivot variables, and you've got a particular solution, the particular solution that has zero free variables.
OK. Now -- but that's only one solution, and now I'm looking for all.
So how do I find the rest?
The point is I can add on x -- anything out of the null space.
We know how to find the vectors in the null space -- because we did it last time, but I'll remind you what we got.
And then I'll add.
So the final result will be that the complete solution -- this is now the complete guy -- the complete solution is this one particular solution plus any, any vector, all different vectors out of the null space.
xn, OK. Well why, why this pattern, because this pattern shows up through all of mathematics, because it shows up everywhere we have linear equations.
Let me just put here the, the reason.
A xp, so that's x particular, so what does Ax particular give?
That gives the correct right-hand side b.
And what does A times an x in the null space give?
Zero.
So I add, and I put in parentheses.
So xp plus xn is b plus zero, which is b.
So -- oh, what I saying?
Let me just say it in words.
If I have one solution, I can add on anything in the null space, because anything in the null space has a zero right-hand side, and I still have the correct right-hand side B.
So that's my system.
That's my complete solution.
Now let me write out what that will be for this example.
So in this example, x general, x complete, the complete solution, is x particular, which is minus two zero three halves zero, with those zeroes in the free variable, plus -- you remember there were the special solutions in the null space that had a one in the free variables -- or one and zero in the free variables, and then we filled in to find I've forgotten what they were, but maybe it was that.
the others?
That was a special solution, and then there was another special solution that had that free variable zero and this free variable equal one, and I have to fill those in.
Let's see, can I remember how those fill in?
Maybe this was a minus two and this was a two, possibly?
I think probably that's right.
I'm not -- yeah.
Does that look write to you?
I would have to remember what are my equations.
Can I, rather than go way back to that board, let me remember the first equation was two x3 plus two x4 equaling zero now, because I'm looking for the guys in the null space.
So I set x4 to be one and the second equation, that I didn't copy again, gave me minus two for this and then -- yeah, so I think that's right.
Two minus four and two gives zero, check.
OK. Those were the special solutions.
What do we do to get the complete solution?
How do I get the complete solution now?
I multiply this by anything, c1, say, and I multiply this by anything -- I take any combination.
Remember that's how we described the null space?
The null space consists of all combinations of -- so this is xn -- all combinations of the special solutions.
There were two special solutions because there were two free variables.
And we want to make that count -- carefully now.
Just while I'm up here.
So there's, that's what the -- that's the kind of answer I'm looking for.
Is there a constant multiplying this guy?
Is there a free constant that multiplies x particular?
No way.
Right?
x particular solves A xp=b.
I'm not allowed to multiply that by three.
But Axn, I'm allowed to multiply xn by three, or add to another xn, because I keep getting zero on the right.
OK.
So, so again, xp is one particular guy.
xn is a whole subspace.
Right?
It's one guy plus, plus anything from a subspace.
Let me draw it.
Let me try to -- oh.
I want to draw, I want to graph all this -- I want to, I want to plot all solutions.
Now x.
So what dimension I in?
This is a unfortunate point.
How many components does x have?
Four.
There are four unknowns.
So I have to draw a four dimensional picture on this MIT cheap blackboard.
OK.
So here we go.
x1 -- Einstein could do it, but, this, this is -- those are four perpendicular axes in -- representing four dimensional space.
OK. Where are my solutions?
Do my solutions form a subspace?
Does the set of solutions to Ax=b form a subspace?
No way.
What does it actually look like, though?
A subspace is in this picture.
This part is a subspace, right?
That part is some, like, two dimensional, because I've got two parameters, so it's -- I'm thinking of this null space as a two dimensional subspace inside R^4.
Now I have to tell you and will tell you next time, what does it mean to say a subspace, what's the dimension of a subspace.
But you see what it's going to be.
It's the number of free independent constants that we can choose.
So somehow there'll be a two dimensional subspace, not a line, and not a three dimensional plane, but only a two dimensional guy.
But it's doesn't go through the origin because it goes through this point.
So there's x particular.
x particular is somewhere here.
x particular.
So it's somehow a subspace -- can I try to draw it that way?
It's a two dimensional subspace that goes through x particular and then onwards by -- so there's x particular, and I added on xn, and there's x.
There's x=xp+xn.
But the xn was anywhere in this subspace, so that filled out a plane.
It's a subspace -- it's not a subspace, what I saying?
It's like a flat thing, it's like a subspace, but it's been shifted, away from the origin.
It doesn't contain zero.
Thanks.
OK. That's the picture, and that's the algorithm.
So the algorithm is just go through elimination and, find the particular solution, and then find those special solutions.
You can do that.
Let me take our time here in the lecture to think, about the bigger picture.
So let me think about -- so this is my pattern.
Now I want to think -- I want to ask you about a question -- I want to ask you some questions.
So when I mean think bigger, I mean I'll think about an m by n matrix A of rank r. OK. What's our definition of rank?
Our current definition of rank is number of pivots.
OK. First of all, how are these numbers related?
Can you tell me a relation between r and m?
If I have m rows in the matrix and R pivots, -- then I certainly know, always -- what relation do I know between r and m?
r is less or equal, right?
Because I've got m rows, I can't have more than m pivots, I might have m and I might have fewer.
Also, I've got n columns.
So what's the relation between r and n?
It's the same, less or equal, because a column can't have more than one pivot.
So I can't have more than n pivots altogether.
OK, OK.
So I have an m by n matrix of rank r. And I always know r less than or equal to m, r less than or equal to n. Now I'm specially interested in the case of full rank, when the rank r is as big as it can be.
Well, I guess I've got two separate possibilities here, depending on what these numbers m and n are.
So let me talk about the case of full column rank.
And by that I mean r=n.
And I want to ask you, what does that imply about our solutions?
What does that tell us about the null space?
What does that tell us about, the complete solution?
OK, so what does that mean?
So I want to ask you, well, OK, if the rank is n, what does that mean?
That means there's a pivot in every column.
So how many pivot variables are there?
n. All the columns have pivots in this case.
So how many free variables are there?
None at all.
So no free variables.
r=n, no free variables.
So what does that tell us about what's going to happen then in our, in our little algorithms?
What will be in the null space?
The null space of A has got what in it?
Only the zero vector.
There are no free variables to give other values to.
So the null space is only the zero vector.
And what about our solution to Ax=b?
Solution to Ax=b?
What, what's the story on that one?
So now that's coming from today's lecture.
The solution x is -- what's the complete solution?
It's just x particular, right?
If, if, if there is an x, if there is a solution.
It's x equal x particular.
There's nothing -- you know, there's just one solution.
If there's one at all.
So it's unique solution -- unique means only one -- unique solution if it exists, if it exists.
In other words, I would say -- let me put it a different way.
There're either zero or one solutions.
This is all in this case r=n.
So I'm -- because many, many applications in reality, the columns will be what I'll later call independent.
And we'll have, nothing to look for in the null space, and we'll only have particular solutions.
OK. Everybody see that possibility?
But I need an example, right?
So let me create an example.
What sort of a matrix -- what's the shape of a matrix that has full column rank?
So can I squeeze in an, an example here?
If it exists.
Let me put in an example, and it's just the right space to put in an example.
Because the example will be like tall and thin.
It will have -- well, I mean, here's an example, one two six five, three one one one.
Brilliant example.
OK.
So there's a matrix A, and what's its rank?
What's the rank of that matrix?
How many pivots will I find if I do elimination?
Two, right?
Two.
I see a pivot there -- oh certainly those two columns are headed off in different directions.
When I do elimination, I'll certainly get another pivot here, fine, and I can use those to clean out below and above.
So the -- actually, tell me what its row reduced row echelon form would be.
Can you carry that, that elimination process to the bitter end?
So what do, what does that mean?
I subtract a multiple of this row from these rows.
So I clean up, all zeros there.
Then I've got some pivot here.
What do I do with that?
I go subtract it below and above, and then I divide through, and what's R for that example?
Maybe I can -- you'll allow me to put that just here in the next board.
What's the row reduced echelon form, just out of practice, for that matrix?
It's got ones in the pivots.
It's got the identity matrix, a little two by two identity matrix, and below it all zeros.
That's a matrix that really has two independent rows, and they're the first two, actually.
The first two rows are independent.
They're not in the same direction.
But the other rows are combinations of the first two, so -- is there always a solution to Ax=b?
Tell me what's the picture here?
For this matrix A, this is a case of full column rank.
The two columns are -- give two pivots.
There's nothing in the null space.
There's no combination of those columns that gives the zero column except the zero zero combination.
So there's nothing in the null space.
But is there always a solution to A X equal B?
What's up with A X equal B?
I've got four, four equations here, and only two Xs.
So the answer is certainly no.
There's not always a solution.
I may have zero solutions, and if I make a random choice, I'll have zero solutions.
Or if I make a great particular choice of the right-hand side, which just happens to be a combination of those two guys -- like tell me one right-hand side that would have a solution.
Tell me a right-hand side that would have a solution.
Well, 0 0 0 0, OK. No prize for that one.
Tell me another one.
Another right-hand side that has a solution would be 4 3 7 6.
I could add the two columns.
What would be the total complete solution if the Right?
right-hand side was 4 3 7 6?
There would be the particular solution one one, one of that column plus one of that, and that would be the only solution.
So there would be -- x particular would be one one in the case when the right side is the sum of those two columns, and that's it.
So that would be a case with one solution.
OK. That, this is the typical setup with full column rank.
Now I go to full row rank.
You see the sort of natural symmetry of this discussion.
Full row rank means r=m.
So this is what I'm interested in now, r=m.
OK, what's up with that?
How many pivots?
m. So what happens when we do elimination in that case?
I'm going to get m pivots.
So every row has a pivot, right?
Every row has a pivot.
Then what about solvability?
What about this business of -- for which right-hand sides can I solve it?
So that's my question.
I can solve Ax=b for which right-hand sides?
Do you see what's coming?
I do elimination, I don't get any zero rows.
So there aren't any requirements on b. I can solve Ax=b for every b. I can solve Ax=b for every right-hand side.
So this is the existence, exists a solution.
Now tell me, so the, u- u- so every row has a pivot in it.
So how many free variables are there?
How many free variables in this case?
If I had n variables to start with, how many are used up by pivot variables?
r, which is m. So I'm left with, left with n-r free variables.
OK.
So this case of full row rank I can always solve, and then this tells me how many variables are free, and this is of course n-m.
This is n-m free variables.
Can I do an example?
You know, the best way for me to do an example is just to transpose that example.
So let me take, let me take that matrix that had column one two six five and make it a row.
And let me take three one one one as the second row.
And let me ask you, this is my matrix A, what's its rank?
What's the rank of that matrix?
Sorry to ask, but not sorry really, because we're just getting the idea of rank.
What's the rank of that matrix?
Two, exactly, two.
There will be two pivots.
What will the row reduced echelon form be?
Anybody know that one?
Actually, tell me not only -- you have to tell me not only the, there'll be two pivots but which will be the pivot columns.
Which columns of this matrix will be pivot columns?
So the first column is fine, and then I go on to the next column, and what do I get?
Do I get a second pivot out of -- will I get a second pivot in this position?
Yes.
So the pivots, when I get all the way to R, will be there.
And here will be some numbers.
This is the part that I previously called F. This is the part that -- the pivot columns in R will be the identity matrix.
There are no zero rows, no zero rows, because the rank is two.
But there'll be stuff over here.
And that will, enter the special solutions and the null space.
OK.
So this is a typical matrix with r=m smaller than n. Now finally I've got a space here for r=m=n.
I'm off in the corner here with the most important case of all.
So what's up with this matrix?
So let me give an example.
OK, brilliant example, 1 2 3 1.
Tell me what -- how do I describe a matrix that has rank r=m=n?
So the matrix is square, right, it's a square matrix.
And if I know its rank is -- it's full rank, now.
I don't have to say full column rank or full row rank -- I just say full rank, because the count, column count and the row count are the same, and the rank is as big as it can be.
And what kind of a matrix have I got?
It's invertible.
So that's exactly the invertible matrices.
r=m=n means the -- what's the row echelon form, the reduced row echelon form, for an invertible matrix?
For a square, nice, square, invertible matrix?
It's I.
Right.
So you see that the, the good matrices are the ones that kind of come out trivially in R. You reduce them all the way to the identity matrix.
What's the null space for this, for this matrix?
Can I just hammer away with questions?
What's the null space for this matrix?
The null space of that matrix is the zero vector only.
The zero vector only.
What are the conditions to solve Ax=b?
Which right-hand sides b are OK?
If I want to solve Ax=b for this example, so A is this, b is b1 b2, what are the conditions on b1 and b2?
None at all, right.
So this is the case, this is the case where I can solve -- so I've coming back here, I can -- since the rank equals m, I can solve for every b.
And since the rank is also n, there's a unique solution.
Let me summarize the whole picture here.
Here's the whole picture.
I could have r=m=n.
This is the case where this is the identity matrix.
And this is the case where there is one solution.
That's the square invertible chapter two case.
Now we're into chapter three.
We could have r=m smaller than n. Now that's what we had over there, and the row echelon form looked like the identity with some zero rows.
And that was the case where there are zero or one solution.
When I say solution I mean to Ax=b.
So this case, there's always one.
This case there's zero or one.
And now let me take the case of full column rank, but some, extra rows.
So now R has -- well, the identity -- I'm almost tempted to write the identity matrix and then F, but that isn't necessarily right.
I have -- is that right?
Am I getting this correct here?
Oh, I'm not!
My God!
This is the case R equals n, the columns, the columns are, are OK. That's the case that was on that board, r=n, full column rank.
Now I want the case where m is smaller than n and I've got extra columns.
OK.
There we go.
So this is now the case of full row rank, and it looks like I F except that I can't be sure that the pivot columns are the first columns.
So the I and the F, could be partly mixed into the I.
Can I write that with just like that?
So the F could be sort of partly into the I if the first columns weren't the pivot columns.
Now how many solutions in this case?
There's always a solution.
This is the existence case.
There's always a solution.
We're not getting any zero rows.
There are no zero rows here.
So there's always either one or infinitely many solutions.
OK. Actually, I guess there's always an infinite number, because we always have some null space to deal with.
Then the final case is where r is smaller than m and smaller than n. OK. Now that's the case where R is the identity with some free stuff but with some zero rows too.
And that's the case where there's either no solution -- because we didn't get a zero equals zero for some bs, or infinitely many solutions.
OK. Do you -- this board really summarizes the lecture, and this sentence summarizes the lecture.
The rank tells you everything about the number of solutions.
That number, the rank r, tells you all the information except the exact entries in the solutions.
For that you go to the matrix.
OK, good.
Have a great weekend, and I'll see you on Monday.
OK, this is linear algebra lecture nine.
And this is a key lecture, this is where we get these ideas of linear independence, when a bunch of vectors are independent -- or dependent, that's the opposite.
The space they span.
A basis for a subspace or a basis for a vector space, that's a central idea.
And then the dimension of that subspace.
So this is the day that those words get assigned clear meanings.
And emphasize that we talk about a bunch of vectors being independent.
Wouldn't talk about a matrix being independent.
A bunch of vectors being independent.
A bunch of vectors spanning a space.
A bunch of vectors being a basis.
And the dimension is some number.
OK, so what are the definitions?
Can I begin with a fact, a highly important fact, that, I didn't call directly attention to earlier.
Suppose I have a matrix and I look at Ax equals zero.
Suppose the matrix has a lot of columns, so that n is bigger than m. So I'm looking at n equations -- I mean, sorry, m equations, a small number of equations m, and more unknowns.
I have more unknowns than equations.
Let me write that down.
More unknowns than equations.
More unknown x-s than equations.
Then the conclusion is that there's something in the null space of A, other than just the zero vector.
The conclusion is there are some non-zero x-s such that Ax is zero.
There are some special solutions.
And why?
We know why.
I mean, it sort of like seems like a reasonable thing, more unknowns than equations, then it seems reasonable that we can solve them.
But we have a, a clear algorithm which starts with a system and does elimination, gets the thing into an echelon form with some pivots and pivot columns, and possibly some free columns that don't have pivots.
And the point is here there will be some free columns.
The reason, so the reason is there must -- there will be free variables, at least one.
That's the reason.
That we now have this -- a complete, algorithm, a complete systematic way to say, OK, we take the system Ax equals zero, we row reduce, we identify the free variables, and, since there are n variables and at most m pivots, there will be some free variables, at least one, at least n-m in fact, left over.
And those variables I can assign non-zero values to.
I don't have to set those to zero.
I can take them to be one or whatever I like, and then I can solve for the pivot variables.
So then it gives me a solution to Ax equals zero.
And it's a solution that isn't all zeros.
So, that's an important point that we'll use now in this lecture.
So now I want to say what does it mean for a bunch of vectors to be independent.
OK.
So this is like the background that we know.
Now I want to speak about independence.
OK. Let's see.
I can give you the abstract definition, and I will, but I would also like to give you the direct meaning.
So the question is, when vectors x1, x2 up to -- Suppose I have n vectors are independent if.
Now I have to give you -- or linearly independent -- I'll often just say and write independent for short.
OK.
I'll give you the full definition.
These are just vectors in some vector space.
I can take combinations of them.
The question is, do any combinations give zero?
If some combination of those vectors gives the zero vector, other than the combination of all zeros, then they're dependent.
They're independent if no combination gives the zero vector -- and then I have, I'll have to put in an except the zero combination.
So what do I mean by that?
No combination gives the zero vector.
Any combination c1 x1+c2 x2 plus, plus cn xn is not zero except for the zero combination.
This is when all the c-s, all the c-s are zero.
Then of course.
That combination -- I know I'll get zero.
But the question is, does any other combination give zero?
If not, then the vectors are independent.
If some other combination does give zero, the vectors are dependent.
OK. Let's just take examples.
Suppose I'm in, say, in two dimensional space.
OK.
I give you -- I'd like to first take an example -- let me take an example where I have a vector and twice that vector.
So that's two vectors, V and 2V.
Are those dependent or independent?
Those are dependent for sure, right, because there's one vector is twice the other.
One vector is twice as long as the other, so if the word dependent means anything, these should be dependent.
And they are.
And in fact, I would take two of the first -- so here's, here is a vector V and the other guy is a vector 2V, that's my -- so there's a vector V1 and my next vector V2 is 2V1.
Of course those are dependent, because two of these first vectors minus the second vector is zero.
That's a combination of these two vectors that gives the zero vector.
OK, that was clear.
Suppose, suppose I have a vector -- here's another example.
It's easy example.
Suppose I have a vector and the other guy is the zero vector.
Suppose I have a vector V1 and V2 is the zero vector.
Then are those vectors dependent or independent?
They're dependent again.
You could say, well, this guy is zero times that one.
This one is some combination of those.
But let me write it the other way.
Let me say -- what combination, how many V1s and how many V2s shall I take to get the zero vector?
If, if V1 is like the vector two one and V2 is the zero vector, zero zero, then I would like to show that some combination of those gives the zero vector.
What shall I take?
How many V1s shall I take?
Zero of them.
Yeah, no, take no V1s.
But how many V2s?
Six.
OK. Or five.
Then -- in other words, the point is if the zero vector's in there, if the zero -- if one of these vectors is the zero vector, independence is dead, right?
If one of those vectors is the zero vector then I could always take -- include that one and none of the others, and I would get the zero answer, and I would show dependence.
OK. Now, let me, let me finally draw an example where they will be independent.
Suppose that's V1 and that's V2.
Those are surely independent, right?
Any combination of V1 and V2, will not be zero except, the zero combination.
So those would be independent.
But now let me, let me stick in a third vector, V3.
Independent or dependent now, those three vectors?
So now n is three here.
I'm in two dimensional space, whatever, I'm in the plane.
I have three vectors that I didn't draw so carefully.
I didn't even tell you what exactly they were.
But what's this answer on dependent or independent?
Dependent.
How do I know those are dependent?
How do I know that some combination of V1, V2, and V3 gives me the zero vector?
I know because of that.
That's the key fact that tells me that three vectors in the plane have to be dependent.
Why's that?
What's the connection between the dependence of these three vectors and that fact?
OK.
So here's the connection.
I take the matrix A that has V1 in its first column, V2 in its second column, V3 in its third column.
So it's got three columns.
And V1 -- I don't know, that looks like about two one to me.
V2 looks like it might be one two.
V3 looks like it might be maybe two, maybe two and a half, minus one.
OK. Those are my three vectors, and I put them in the columns of A.
Now that matrix A is two by three.
It fits this pattern, that where we know we've got extra variables, we know we have some free variables, we know that there's some combination -- and let me instead of x-s, let me call them c1, c2, and c3 -- that gives the zero vector.
Sorry that my little bit of art got in the way.
Do you see the point?
When I have a matrix, I'm interested in whether its columns are dependent or independent.
The columns are dependent if there is something in the null space.
The columns are dependent because this, this thing in the null space says that c1 of that plus c2 of that plus c3 of this is zero.
So in other words, I can go out some V1, out some more V2, back on V3, and end up zero.
OK.
So let -- here I've give the general, abstract definition, but let me repeat that definition -- this is like repeat -- let me call them Vs now.
V1 up to Vn are the columns of a matrix A.
In other words, this is telling me that if I'm in m dimensional space, like two dimensional space in the example, I can answer the dependence-independence question directly by putting those vectors in the columns of a matrix.
They are independent if the null space of A, of A, is what?
If I have a bunch of columns in a matrix, I'm looking at their combinations, but that's just A times the vector of c-s. And these columns will be independent if the null space of A is the zero vector.
They are dependent if there's something else in there.
If there's something else in the null space, if A times c gives the zero vector for some non-zero vector c in the null space.
Then they're dependent, because that's telling me a combination of the columns gives the zero column.
I think you're with be, because we've seen, like, lecture after lecture, we're looking at the combinations of the columns and asking, do we get zero or don't we?
And now we're giving the official name, dependent if we do, independent if we don't.
So I could express this in other words now.
I could say the rank -- what's the rank in this independent case?
The rank r of the, of the matrix, in the case of independent columns, is?
So the columns are independent.
So how many pivot columns have I got.
All n. All the columns would be pivot columns, because free columns are telling me that they're a combination of earlier columns.
So this would be the case where the rank is n. This would be the case where the rank is smaller than n. So in this case the rank is n and the null space of A is only the zero vector.
And no free variables.
No free variables.
And this is the case yes free variables.
If you'll allow me to stretch the English language that far.
That's the case where we have, a combination that gives the zero column.
I'm often interested in the case when my vectors are popped into a matrix.
So the, the definition over there of independence didn't talk about any matrix.
The vectors didn't have to be vectors in N dimensional space.
And I want to give you some examples of vectors that aren't what you think of immediately as vectors.
But most of the time, this is -- the vectors we think of are columns.
And we can put them in a matrix.
And then independence or dependence comes back to the null space.
OK.
So that's the idea of independence.
Can I just, yeah, let me go on to spanning a What does it mean for a bunch of vectors to span a space?
space.
Well, actually, we've seen it already.
You remember, if we had a columns in a matrix, we took all their combinations and that gave us the column space.
Those vectors that we started with span that column space.
So spanning a space means -- so let me move that important stuff right up.
OK.
So vectors -- let me call them, say, V1 up to -- call you some different letter, say Vl -- span a space, a subspace, or just a vector space I could say, span a space means, means the space consists of all combinations of those vectors.
That's exactly what we did with the column space.
So now I could say in shorthand the columns of a matrix span the column space.
So you remember it's a bunch of vectors that have this property that they span a space, and actually if I give you a bunch of vectors and say -- OK, let S be the space that they span, in other words let S contain all their combinations, that space S will be the smallest space with those vectors in it, right?
Because any space with those vectors in it must have all the combinations of those vectors in it.
And if I stop there, then I've got the smallest space, and that's the space that they span.
OK.
So I'm just -- rather than, needing to say, take all linear combinations and put them in a space, I'm compressing that into the word span.
Straightforward.
So if I think of a, of the column space of a OK. matrix.
I've got their -- so I start with the columns.
I take all their combinations.
That gives me the columns space.
They span the column space.
Now are they independent?
Maybe yes, maybe no.
It depends on the particular columns that went into that matrix.
But obviously I'm highly interested in a set of vectors that spans a space and is independent.
That's, that means like I've got the right number of vectors.
If I didn't have all of them, I wouldn't have my whole space.
If I had more than that, they probably wouldn't -- they wouldn't be independent.
So, like, basis -- and that's the word that's coming -- is just right.
So here let me put what that word means.
A basis for a vector space is, is a, is a sequence of vectors -- shall I call them V1, V2, up to let me say Vd now, I'll stop with that letters -- that has two properties.
I've got enough vectors and not too many.
It's a natural idea of a basis.
So a basis is a bunch of vectors in the space and it's a so it's a sequence of vectors with two properties, with two properties.
One, they are independent.
And two -- you know what's coming?
-- they span the space.
OK. Let me take -- so time for examples, of course.
So I'm asking you now to put definition one, the definition of independence, together with definition two, and let's look at examples, because this is -- this combination means the set I've -- of vectors I have is just right, and the -- so that this idea of a basis will be central.
I'll always be asking you now for a basis.
Whenever I look at a subspace, if I ask you for -- if you give me a basis for that subspace, you've told me what it is.
You've told me everything I need to know about that subspace.
Those -- I take their combinations and I know that I need all the combinations.
OK.
Examples.
OK, so examples of a basis.
Let me start with two dimensional space.
Suppose the space -- say example.
The space is, oh, let's make it R^3.
Real three dimensional space.
Give me one basis.
One basis is?
So I want some vectors, because if I ask you for a basis, I'm asking you for vectors, a little list of vectors.
And it should be just right.
So what would be a basis for three dimensional space?
Well, the first basis that comes to mind, why don't we write that down.
The first basis that comes to mind is this vector, this vector, and this vector.
OK. That's one basis.
Not the only basis, that's going to be my point.
But let's just see -- yes, that's a basis.
Are, are those vectors independent?
So that's the like the x, y, z axes, so if those are not independent, we're in trouble.
Certainly, they are.
Take a combination c1 of this vector plus c2 of this vector plus c3 of that vector and try to make it give the zero vector.
What are the c-s?
If c1 of that plus c2 of that plus c3 of that gives me 0 0 0, then the c-s are all -- 0, right.
So that's the test for independence.
In the language of matrices, which was under that board, I could make those the columns of a matrix.
Well, it would be the identity matrix.
Then I would ask, what's the null space of the identity matrix?
And you would say it's only the zero vector.
And I would say, fine, then the columns are independent.
The only thing -- the identity times a vector giving zero, the only vector that does that is zero.
OK. Now that's not the only basis.
Far from it.
Tell me another basis, a second basis, another basis.
So, give me -- well, I'll just start it out.
One one two.
Two two five.
Suppose I stopped there.
Has that little bunch of vectors got the properties that I'm asking for in a basis for R^3?
We're looking for a basis for R^3.
Are they independent, those two column vectors?
Yes.
Do they span R^3?
No.
Our feeling is no.
Our feeling is no.
Our feeling is that there're some vectors in R3 that are not combinations of those.
OK.
So suppose I add in -- I need another vector then, because these two don't span the space.
OK. Now it would be foolish for me to put in three three seven, right, as the third vector.
That would be a goof.
Because that, if I put in three three seven, those vectors would be dependent, right?
If I put in three three seven, it would be the sum of those two, it would lie in the same plane as those.
It wouldn't be independent.
My attempt to create a basis would be dead.
But if I take -- so what vector can I take?
I can take any vector that's not in that plane.
Let me try -- I hope that 3 3 8 would do it.
At least it's not the sum of those two vectors.
But I believe that's a basis.
And what's the test then, for that to be a basis?
Because I just picked those numbers, and if I had picked, 5 7 -14 how would we know do we have a basis or don't we?
You would put them in the columns of a matrix, and you would do elimination, row reduction -- and you would see do you get any free variables or are all the columns pivot columns.
Well now actually we have a square -- the matrix would be three by three.
So, what's the test on the matrix then?
The matrix -- so in this case, when my space is R^3 and I have three vectors, my matrix is square and what I asking about that matrix in order for those columns to be a basis?
So in this -- for R^n, if I have -- n vectors give a basis if the n by n matrix with those columns, with those columns, is what?
What's the requirement on that matrix?
Invertible, right, right.
The matrix should be invertible.
For a square matrix, that's the, that's the perfect answer.
Is invertible.
So that's when, that's when the space is the whole space R^n.
Let me, let me be sure you're with me here.
Let me remove that.
Are those two vectors a basis for any space at all?
Is there a vector space that those really are a basis for, those, that pair of vectors, this guy and this 1, 1 1 2 and 2 2 5?
Is there a space for which that's a basis?
Sure.
They're independent, so they satisfy the first requirement, so what space shall I take for them to be a basis of?
What spaces will they be a basis for?
The one they span.
Their combinations.
It's a plane, right?
It'll be a plane inside R^3.
So if I take this vector 1 1 2, say it goes there, and this vector 2 2 5, say it goes there, those are a basis for -- because they span a plane.
And they're a basis for the plane, because they're independent.
If I stick in some third guy, like 3 3 7, which is in the plane -- suppose I put in, try to put in 3 3 7, then the three vectors would still span the plane, but they wouldn't be a basis anymore because they're not independent anymore.
So, we're looking at the question of -- again, OK. the case with independent columns is the case where the column vectors span the column space.
They're independent, so they're a basis for the column space.
OK.
So now there's one bit of intuition.
Let me go back to all of R^n.
So I -- where I put 3 3 8.
OK.
The first message is that the basis is not unique, right.
There's zillions of bases.
I take any invertible three by three matrix, its columns are a basis for R^3.
The column space is R^3, and if those, if that matrix is invertible, those columns are independent, I've got a basis for R^3.
So there're many, many bases.
But there is something in common for all those bases.
There's something that this basis shares with that basis and every other basis for R^3.
And what's that?
Well, you saw it coming, because when I stopped here and asked if that was a basis for R^3, you said no.
And I know that you said no because you knew there weren't enough vectors there.
And the great fact is that there're many, many bases, but -- let me put in somebody else, just for variety.
There are many, many bases, but they all have the same number of vectors.
If we're talking about the space R^3, then that number of vectors is three.
If we're talking about the space R^n, then that number of vectors is n. If we're talking about some other space, the column space of some matrix, or the null space of some matrix, or some other space that we haven't even thought of, then that still is true that every basis -- that there're lots of bases but every basis has the same number of vectors.
Let me write that great fact down.
Every basis -- we're given a space.
Given a space.
R^3 or R^n or some other column space of a matrix or the null space of a matrix or some other vector space.
Then the great fact is that every basis for this, for the space has the same number of vectors.
If one basis has six vectors, then every other basis has six vectors.
So that number six is telling me like it's telling me how big is the space.
It's telling me how many vectors do I have to have to have a basis.
And of course we're seeing it this way.
That number six, if we had seven vectors, then we've got too many.
If we have five vectors we haven't got enough.
Sixes are like just right for whatever space that is.
And what do we call that number?
That number is -- now I'm ready for the last definition today.
It's the dimension of that space.
So every basis for a space has the same number of vectors in it.
Not the same vectors, all sorts of bases -- but the same number of vectors is always the same, and that number is the dimension.
This is definitional.
This number is the dimension of the space.
OK. OK. Let's do some examples.
Because now we've got definitions.
Let me repeat the four things, the four words that have now got defined.
Independence, that looks at combinations not being zero.
Spanning, that looks at all the combinations.
Basis, that's the one that combines independence and spanning.
And now we've got the idea of the dimension of a space.
It's the number of vectors in any basis, because all bases have the same number.
OK. Let's take examples.
Suppose I take, my space is -- examples now -- space is the, say, the column space of this matrix.
Let me write down a matrix.
1 1 1, 2 1 2, and I'll -- just to make it clear, I'll take the sum there, 3 2 3, and let me take the sum of all -- oh, let me put in one -- yeah, I'll put in one one one again.
OK.
So that's four vectors.
OK, do they span the column space of that matrix?
Let me repeat, do they span the column space of that matrix?
By definition, that's what the column space -- Yes.
where it comes from.
Are they a basis for the column space?
Are they independent?
No, they're not independent.
There's something in that null space.
Maybe we can -- so let's look at the null space of the matrix.
Tell me a vector that's in the null space of that matrix.
So I'm looking for some vector that combines those columns and produces the zero column.
Or in other words, I'm looking for solutions to A X equals zero.
So tell me a vector in the null space.
Maybe -- well, this was, this column was that one plus that one, so maybe if I have one of those and minus one of those that would be a vector in the null space.
So, you've already told me now, are those vectors independent, the answer is -- those column vectors, the answer is -- no.
Right?
They're not independent.
Because -- you knew they weren't independent.
Anyway, minus one of this minus one of this plus one of this zero of that is the zero vector.
OK, so they're not independent.
OK.
They span, but they're not independent.
Tell me a basis for that column space.
What's a basis for the column space?
These are all the questions that the homework asks, the quizzes ask, the final exam will ask.
Find a basis for the column space of this matrix.
OK. Now there's many answers, but give me the most natural answer.
Columns one and two.
Columns one and two.
That's the natural answer.
Those are the pivot columns, because, I mean, we s- we begin systematically.
We look at the first column, it's OK. We can put that in the basis.
We look at the second column, it's OK. We can put that in the basis.
The third column we can't put in the basis.
The fourth column we can't, again.
So the rank of the matrix is -- what's the rank of our matrix?
Two.
Two.
And, and now that rank is also -- we also have another word.
We, we have a great theorem here.
The rank of A, that rank r, is the number of pivot columns and it's also -- well, so now please use my new word.
This, it's the number two, of course, two is the rank of my matrix, it's the number of pivot columns, those pivot columns form a basis, of course, so what's two?
It's the dimension.
The rank of A, the number of pivot columns, is the dimension of the column space.
Of course, you say.
It had to be.
Right.
But just watch, look for one moment at the, the language, the way the English words get involved here.
I take the rank of a matrix, the rank of a matrix.
It's a number of columns and it's the dimension of -- not the dimension of the matrix, that's what I want to say.
It's the dimension of a space, a subspace, the column space.
Do you see, I don't take the dimension of A.
That's not what I want.
I'm looking for the dimension of the column space of A.
If you use those words right, it shows you've got the idea right.
Similarly here.
I don't talk about the rank of a subspace.
It's a matrix that has a rank.
I talk about the rank of a matrix.
And the beauty is that these definitions just merge so that the rank of a matrix is the dimension of its column space.
And in this example it's two.
And then the further question is, what's a basis?
And the first two columns are a basis.
Tell me another basis.
Another basis for the columns space.
You see I just keep hammering away.
I apologize, but it's, I have to be sure you have the idea of basis.
Tell me another basis for the column space.
Well, you could take columns one and three.
That would be a basis for the column space.
Or columns two and three would be a basis.
Or columns two and four.
Or tell me another basis that's not made out of those columns at all?
So -- I guess I'm giving you infinitely many possibilities, so I can't expect a unanimous answer here.
I'll tell you -- but let's look at another basis, though.
I'll just -- because it's only one out of zillions, I'm going to put it down and I'm going to erase it.
Another basis for the column space would be -- let's see.
I'll put in some things that are not there.
Say, oh well, just to make it -- my life easy, 2 2 2.
That's in the column space.
And, that was sort of obvious.
Let me take the sum of those, say 6 4 6.
Or the sum of all of the columns, 7 5 7, why not.
That's in the column space.
Those are independent and I've got the number right, I've got two.
Actually, this is a key point.
If you know the dimension of the space you're working with, and we know that this column -- we know that the dimension, DIM, the dimension of the column space is two.
If you know the dimension, then -- and we have a couple of vectors that are independent, they'll automatically be a basis.
If we've got the number of vectors right, two vectors in this case, then if they're independent, they can't help but span the space.
Because if they didn't span the space, there'd be a third guy to help span the space, but it couldn't be independent.
So, it just has to be independent if we've got the numbers right.
And they span.
OK.
Very good.
So you got the dimension of a space.
So this was another basis that I just invented.
OK. Now, now I get to ask about the null space.
What's the dimension of the null space?
So we, we got a great fact there, the dimension of the column space is the rank.
Now I want to ask you about the null space.
That's the other part of the lecture, and it'll go on to the next lecture.
OK.
So we know the dimension of the column space is two, the rank.
What about the null space?
This is a vector in the null space.
Are there other vectors in the null space?
Yes or no?
Yes.
So this isn't a basis because it's doesn't span, right?
There's more in the null space than we've got so far.
I need another vector at least.
So tell me another vector in the null space.
Well, the natural choice, the choice you naturally think of is I'm going on to the fourth column, I'm letting that free variable be a one, and that free variable be a zero, and I'm asking is that fourth column a combination of my pivot columns?
Yes, it is.
And it's -- that will do.
So what I've written there are actually the two special solutions, right?
I took the two free variables, free and free.
I gave them the values 1 0 or 0 I figured out the rest.
So do you see, let me just say it in words.
This vector, these vectors in the null space are telling me, they're telling me the combinations of the columns that give zero.
They're telling me in what way the, the columns are dependent.
That's what the null space is doing.
Have I got enough now?
And what's the null space now?
We have to think about the null space.
These are two vectors in the null space.
They're independent.
Are they a basis for the null space?
What's the dimension of the null space?
You see that those questions just keep coming up all the time.
Are they a basis for the null space?
You can tell me the answer even though we haven't written out a proof of that.
Can you?
Yes or no?
Do these two special solutions form a basis for the null space?
In other words, does the null space consist of all combinations of those two guys?
Yes or no?
Yes.
Yes.
The null space is two dimensional.
The null space, the dimension of the null space, is the number of free variables.
So the dimension of the null space is the number of free variables.
And at the last second, give me the formula.
This is then the key formula that we know.
How many free variables are there in terms of R, the rank, m -- the number of rows, n, the number of columns?
What do we get?
We have n columns, r of them are pivot columns, so n-r is the number of free columns, free variables.
And now it's the dimension of the null space.
OK. That's great.
That's the key spaces, their bases, and their dimensions.
Thanks.
OK, here is lecture ten in linear algebra.
Two important things to do in this lecture.
One is to correct an error from lecture nine.
So the blackboard with that awful error is still with us.
And the second, the big thing to do is to tell you about the four subspaces that come with a matrix.
We've seen two subspaces, the column space and the null space.
There's two to go.
First of all, and this is a great way to OK. recap and correct the previous lecture -- so you remember I was just doing R^3.
I couldn't have taken a simpler example than R^3.
And I wrote down the standard basis.
That's the standard basis.
The basis -- the obvious basis for the whole three dimensional space.
And then I wanted to make the point that there was nothing special, nothing about that basis that another basis couldn't have.
It could have linear independence, it could span a space.
There's lots of other bases.
So I started with these vectors, one one two and two two five, and those were independent.
And then I said three three seven wouldn't do, because three three seven is the sum of those.
So in my innocence, I put in three three eight.
I figured probably if three three seven is on the plane, is -- which I know, it's in the plane with these two, then probably three three eight sticks a little bit out of the plane and it's independent and it gives a basis.
But after class, to my sorrow, a student tells me, "Wait a minute, that ba- that third vector, three three eight, is not independent."
And why did she say that?
She didn't actually take the time, didn't have to, to find w- w- what combination of this one and this one gives three three eight.
She did something else.
In other words, she looked ahead, because she said, wait a minute, if I look at that matrix, it's not invertible.
That third column can't be independent of the first two, because when I look at that matrix, it's got two identical rows.
I have a square matrix.
Its rows are obviously dependent.
And that makes the columns dependent.
So there's my error.
When I look at the matrix A that has those three columns, those three columns can't be independent because that matrix is not invertible because it's got two equal rows.
And today's lecture will reach the conclusion, the great conclusion, that connects the column space with the row space.
So those are -- the row space is now going to be another one of my fundamental subspaces.
The row space of this matrix, or of this one -- well, the row space of this one is OK, but the row space of this one, I'm looking at the rows of the matrix -- oh, anyway, I'll have two equal rows and the row space will be only two dimensional.
The rank of the matrix with these columns will only be two.
So only two of those columns, columns can be independent too.
The rows tell me something about the columns, in other words, something that I should have noticed and I didn't.
OK.
So now let me pin down these four fundamental subspaces.
So here are the four fundamental subspaces.
This is really the heart of this approach to linear algebra, to see these four subspaces, how they're related.
So what are they?
The column space, C of A.
The null space, N of A.
And now comes the row space, something new.
The row space, what's in that?
It's all combinations of the rows.
That's natural.
We want a space, so we have to take all combinations, and we start with the rows.
So the rows span the row space.
Are the rows a basis for the row space?
Maybe so, maybe no.
The rows are a basis for the row space when they're independent, but if they're dependent, as in this example, my error from last time, they're not -- those three rows are not a basis.
The row space wouldn't -- would only be two dimensional.
I only need two rows for a basis.
So the row space, now what's in it?
It's all combinations of the rows of A.
All combinations of the rows of A.
But I don't like working with row vectors.
All my vectors have been column vectors.
I'd like to stay with column vectors.
How can I get to column vectors out of these rows?
I transpose the matrix.
So if that's OK with you, I'm going to transpose the matrix.
I'm, I'm going to say all combinations of the columns of A transpose.
And that allows me to use the convenient notation, the column space of A transpose.
Nothing, no mathematics went on there.
We just got some vectors that were lying down to stand up.
But it means that we can use this column space of A transpose, that's telling me in a nice matrix notation what the row space is.
OK. And finally is another null space.
The fourth fundamental space will be the null space of A transpose.
The fourth guy is the null space of A transpose.
And of course my notation is N of A transpose.
That's the null space of A transpose.
Eh, we don't have a perfect name for this space as a -- connecting with A, but our usual name is the left null space, and I'll show you why in a moment.
So often I call this the -- just to write that word -- the left null space of A.
So just the way we have the row space of A and we switch it to the column space of A transpose, so we have this space of guys l- that I call the left null space of A, but the good notation is it's the null space of A transpose.
OK. Those are four spaces.
Where are those spaces?
What, what big space are they in for -- when A is m by n?
In that case, the null space of A, what's in the null space of A?
Vectors with n components, solutions to A x equals zero.
So the null space of A is in R^n.
What's in the column space of A?
Well, columns.
How many components dothose columns have?
m. So this column space is in R^m.
What about the column space of A transpose, which are just a disguised way of saying the rows of A?
The rows of A, in this three by six matrix, have six components, n components.
The column space is in R^n.
And the null space of A transpose, I see that this fourth space is already getting second, you know, second class citizen treatment and it doesn't deserve it.
It's, it should be there, it is there, and shouldn't be squeezed.
The null space of A transpose -- well, if the null space of A had vectors with n components, the null space of A transpose will be in R^m.
I want to draw a picture of the four spaces.
OK. OK.
Here are the four spaces.
OK, Let me put n dimensional space over on this side.
Then which were the subspaces in R^n?
The null space was and the row space was.
So here we have the -- can I make that picture of the row space?
And can I make this kind of picture of the null space?
That's just meant to be a sketch, to remind you that they're in this -- which you know, how -- what type of vectors are in it?
Vectors with n components.
Over here, inside, consisting of vectors with m components, is the column space and what I'm calling the null space of A transpose.
Those are the ones with m components.
OK. To understand these spaces is our, is our job now.
Because by understanding those spaces, we know everything about this half of linear algebra.
What do I mean by understanding those spaces?
I would like to know a basis for those spaces.
For each one of those spaces, how would I create -- construct a basis?
What systematic way would produce a basis?
And what's their dimension?
OK.
So for each of the four spaces, I have to answer those questions.
How do I produce a basis?
And then -- which has a somewhat long answer.
And what's the dimension, which is just a number, so it has a real short answer.
Can I give you the short answer first?
I shouldn't do it, but here it is.
I can tell you the dimension of the column space.
Let me start with this guy.
What's its dimension?
I have an m by n matrix.
The dimension of the column space is the rank, r. We actually got to that at the end of the last lecture, but only for an example.
So I really have to say, OK, what's going on there.
I should produce a basis and then I just look to see how many vectors I needed in that basis, and the answer will be r. Actually, I'll do that, before I get on to the others.
What's a basis for the columns space?
We've done all the work of row reduction, identifying the pivot columns, the ones that have pivots, the ones that end up with pivots.
But now I -- the pivot columns I'm interested in are columns of A, the original A.
And those pivot columns, there are r of them.
The rank r counts those.
Those are a basis.
So if I answer this question for the column space, the answer will be a basis is the pivot columns and the dimension is the rank r, and there are r pivot columns and everything great.
OK.
So that space we pretty well understand.
I probably have a little going back to see that -- to prove that this is a right answer, but you know it's the right answer.
Now let me look at the row space.
OK. Shall I tell you the dimension of the row space?
Yes.
Before we do even an example, let me tell you the dimension of the row space.
Its dimension is also r. The row space and the column space have the same dimension.
That's a wonderful fact.
The dimension of the column space of A transpose -- that's the row space -- is r. That, that space is r dimensional.
Snd so is this one.
OK. That's the sort of insight that got used in this example.
If those -- are the three columns of a matrix -- let me make them the three columns of a matrix by just erasing some brackets.
OK, those are the three columns of a matrix.
The rank of that matrix, if I look at the columns, it wasn't obvious to me anyway.
But if I look at the rows, now it's obvious.
The row space of that matrix obviously is two dimensional, because I see a basis for the row space, this row and that row.
And of course, strictly speaking, I'm supposed to transpose those guys, make them stand up.
But the rank is two, and therefore the column space is two dimensional by this wonderful fact that the row space and column space have the same dimension.
And therefore there are only two pivot columns, not three, and, those, the three columns are dependent.
OK. Now let me bury that error and talk about the row space.
Well, I'm going to give you the dimensions of all the spaces.
Because that's such a nice answer.
OK.
So let me come back here.
So we have this great fact to establish, that the row space, its dimension is also the rank.
What about the null space?
OK. What's a basis for the null space?
What's the dimension of the null space?
Let me, I'll put that answer up here for the null space.
Well, how have we constructed the null space?
We took the matrix A, we did those row operations to get it into a form U or, or even further.
We got it into the reduced form R. And then we read off special solutions.
Special solutions.
And every special solution came from a free variable.
And those special solutions are in the null space, and the great thing is they're a basis for it.
So for the null space, a basis will be the special solutions.
And there's one for every free variable, right?
For each free variable, we give that variable the value one, the other free variables zero.
We get the pivot variables, we get a vector in the -- we get a special solution.
So we get altogether n-r of them, because that's the number of free variables.
If we have r -- this is the dimension is r, is the number of pivot variables.
This is the number of free variables.
So the beauty is that those special solutions do form a basis and tell us immediately that the dimension of the null space is n -- I better write this well, because it's so nice -- n-r. And do you see the nice thing?
That the two dimensions in this n dimensional space, one subspace is r dimensional -- to be proved, that's the row space.
The other subspace is n-r dimensional, that's the null space.
And the two dimensions like together give n. The sum of r and n-R is n. And that's just great.
It's really copying the fact that we have n variables, r of them are pivot variables and n-r are free variables, and n altogether.
OK. And now what's the dimension of this poor misbegotten fourth subspace?
It's got to be m-r.
The dimension of this left null space, left out practically, is m-r. Well, that's really just saying that this -- again, the sum of that plus that is m, and m is correct, it's the number of columns in A transpose.
A transpose is just as good a matrix as A.
It just happens to be n by m. It happens to have m columns, so it will have m variables when I go to A x equals 0 and m of them, and r of them will be pivot variables and m-r will be free variables.
A transpose is as good a matrix as A.
It follows the same rule that the this plus the dimension -- this dimension plus this dimension adds up to the number of columns.
And over here, A transpose has m columns.
OK. OK.
So I gave you the easy answer, the dimensions.
Now can I go back to check on a basis?
We would like to think that -- say the row space, because we've got a basis for the column space.
The pivot columns give a basis for the column space.
Now I'm asking you to look at the row space.
And I -- you could say, OK, I can produce a basis for the row space by transposing my matrix, making those columns, then doing elimination, row reduction, and checking out the pivot columns in this transposed matrix.
But that means you had to do all that row reduction on A transpose.
It ought to be possible, if we take a matrix A -- let me take the matrix -- maybe we had this matrix in the last lecture.
1 1 1, 2 1 2, 3 2 3, 1 1 1.
OK. That, that matrix was so easy.
We spotted its pivot columns, one and two, without actually doing row reduction.
But now let's do the job properly.
So I subtract this away from this to produce a zero.
So one 2 3 1 is fine.
Subtracting that away leaves me minus 1 -1 0, right?
And subtracting that from the last row, oh, well that's easy.
OK?
I'm doing row reduction.
Now I've -- the first column is all set.
The second column I now see the pivot.
And I can clean up, if I -- actually, OK. Why don't I make the pivot into a 1.
I'll multiply that row through by by -1, and then I have 1 1.
That was an elementary operation I'm allowed, multiply a row by a number.
And now I'll do elimination.
Two of those away from that will knock this guy out and make this into a 1.
So that's now a 0 and that's a OK.
Done.
That's R. I'm seeing the identity matrix here.
I'm seeing zeros below.
And I'm seeing F there.
OK. What about its row space?
What happened to its row space -- well, what happened -- let me first ask, just because this is, is -- sometimes something does happen.
Its column space changed.
The column space of R is not the column space of A, right?
Because 1 1 1 is certainly in the column space of A and certainly not in the column space of R. I did row operations.
Those row operations preserve the row space.
So the row, so the column spaces are different.
Different column spaces, different column spaces.
But I believe that they have the same row space.
Same row space.
I believe that the row space of that matrix and the row space of this matrix are identical.
They have exactly the same vectors in them.
Those vectors are vectors with four components, right?
They're all combinations of those rows.
Or I believe you get the same thing by taking all combinations of these rows.
And if true, what's a basis?
What's a basis for the row space of R, and it'll be a basis for the row space of the original A, but it's obviously a basis for the row space of R. What's a basis for the row space of that matrix?
The first two rows.
So a basis for the row -- so a basis is, for the row space of A or of R, is, is the first R rows of R. Not of A.
Sometimes it's true for A, but not necessarily.
But R, we definitely have a matrix here whose row space we can, we can identify.
The row space is spanned by the three rows, but if we want a basis we want independence.
So out goes row three.
The row space is also spanned by the first two rows.
This guy didn't contribute anything.
And of course over here this 1 2 3 1 in the bottom didn't contribute anything.
We had it already.
So this, here is a basis.
1 0 1 1 and 0 1 1 0.
I believe those are in the row space.
I know they're independent.
Why are they in the row space?
Why are those two vectors in the row space?
Because all those operations we did, which started with these rows and took combinations of them -- I took this row minus this row, that gave me something that's still in the row space.
That's the point.
When I took a row minus a multiple of another row, I'm staying in the row space.
The row space is not changing.
My little basis for it is changing, and I've ended up with, sort of the best basis.
If the columns of the identity matrix are the best basis for R^3 or R^n, the rows of this matrix are the best basis for the row space.
Best in the sense of being as clean as I can make it.
Starting off with the identity and then finishing up with whatever has to be in there.
OK. Do you see then that the dimension is r?
For sure, because we've got r pivots, r non-zero rows.
We've got the right number of vectors, r. They're in the row space, they're independent.
That's it.
They are a basis for the row space.
And we can even pin that down further.
How do I know that every row of A is a combination?
How do I know they span the row space?
Well, somebody says, I've got the right number of them, so they must.
But -- and that's true.
But let me just say, how do I know that this row is a combination of these?
By just reversing the steps of row reduction.
If I just reverse the steps and go from A -- from R back to A, then what do I, what I doing?
I'm starting with these rows, I'm taking combinations of them.
After a couple of steps, undoing the subtractions that I did before, I'm back to these rows.
So these rows are combinations of those rows.
Those rows are combinations of those rows.
The two row spaces are the same.
The bases are the same.
And the natural basis is this guy.
Is that all right for the row space?
The row space is sitting there in R in its cleanest possible form.
OK. Now what about the fourth guy, the null space of A transpose?
First of all, why do I call that the left null space?
So let me save that and bring that down.
OK.
So the fourth space is the null space of A transpose.
So it has in it vectors, let me call them y, so that A transpose y equals 0.
If A transpose y equals 0, then y is in the null space of A transpose, of course.
So this is a matrix times a column equaling zero.
And now, because I want y to sit on the left and I want A instead of A transpose, I'll just transpose that equation.
Can I just transpose that?
On the right, it makes the zero vector lie down.
And on the left, it's a product, A, A transpose times y.
If I take the transpose, then they come in opposite order, right?
So that's y transpose times A transpose transpose.
But nobody's going to leave it like that.
That's -- A transpose transpose is just A, of course.
When I transposed A transpose I got back to A.
Now do you see what I have now?
I have a row vector, y transpose, multiplying A, and multiplying from the left.
That's why I call it the left null space.
But by making it -- putting it on the left, I had to make it into a row instead of a column vector, and so my convention is I usually don't do that.
I usually stay with A transpose y equals 0.
OK. And you might ask, how do we get a basis -- or I might ask, how do we get a basis for this fourth space, this left null space?
OK.
I'll do it in the example.
As always -- not that one.
The left null space is not jumping out at me here.
I know which are the free variables -- the special solutions, but those are special solutions to A x equals zero, and now I'm looking at A transpose, and I'm not seeing it here.
So -- but somehow you feel that the work that you did which simplified A to R should have revealed the left null space too.
And it's slightly less immediate, but it's there.
So from A to R, I took some steps, and I guess I'm interested in what were those steps, or what were all of them together.
I don't -- I'm not interested in what particular ones they were.
I'm interested in what was the whole matrix that took me from A to R. How would you find that?
Do you remember Gauss-Jordan, where you tack on the identity matrix?
Let's do that again.
So I, I'll do it above, here.
So this is now, this is now the idea of -- I take the matrix A, which is m by n. In Gauss-Jordan, when we saw him before -- A was a square invertible matrix and we were finding its inverse.
Now the matrix isn't square.
It's probably rectangular.
But I'll still tack on the identity matrix, and of course since these have length m it better be m by m. And now I'll do the reduced row echelon form of this matrix.
And what do I get?
The reduced row echelon form starts with these columns, starts with the first columns, works like mad, and produces R. Of course, still that same size, m by n. And we did it before.
And then whatever it did to get R, something else is going to show up here.
Let me call it E, m by m. It's whatever -- do you see that E is just going to contain a record of what we did?
We did whatever it took to get A to become R. And at the same time, we were doing it to the identity matrix.
So we started with the identity matrix, we buzzed along.
So we took some -- all this row reduction amounted to multiplying on the left by some matrix, some series of elementary matrices that altogether gave us one matrix, and that matrix is E. So all this row reduction stuff amounted to multiplying by E. How do I know that?
It certainly amounted to multiply it by something.
And that something took I to E, so that something was E. So now look at the first part, E A is R. No big deal.
All I've said is that the row reduction steps that we all know -- well, taking A to R, are in a, in some matrix, and I can find out what that matrix is by just tacking I on and seeing what comes out.
What comes out is E. Let's just review the invertible square case.
What happened then?
Because I was interested in it in chapter two also.
When A was square and invertible, I took A I, I did row, row elimination.
And what was the R that came out?
It was I.
So in chapter two, in chapter two, R was I.
The row the, the reduced row echelon form of a nice invertible square matrix is the identity.
So if R was I in that case, then E was -- then E was A inverse, because E A is I.
Good.
That's, that was good and easy.
Now what I'm saying is that there still is an E. It's not A inverse any more, because A is rectangular.
It hasn't got an inverse.
But there is still some matrix E that connected this to this -- oh, I should have figured out in advanced what it was.
Shoot.
I didn't -- I did those steps and sort of erased as I went -- and, I should have done them to the identity too.
Can I do that?
Can I do that?
I'll keep the identity matrix, like I'm supposed to do, and I'll do the same operations on it, and see what I end up with.
OK.
So I'm starting with the identity -- which I'll write in light, light enough, but -- OK. What did I do?
I subtracted that row from that one and that row from that one.
OK, I'll do that to the identity.
So I subtract that first row from row two and row three.
Good.
Then I think I multiplied -- Do you remember?
I multiplied row two by minus one.
Let me just do that.
Then what did I do?
I subtracted two of row two away from row one.
I better do that.
Subtract two of this away from this.
That's minus one, two of these away leaves a plus 2 and 0.
I believe that's E. The way to check is to see, multiply that E by this A, just to see did I do it right.
So I believe E was -1 2 0, 1 -1 0, and -1 0 1.
OK. That's my E, that's my A, and that's R. All right.
All I'm struggling to do is right, the reason I wanted this blasted E was so that I could figure out the left null space, not only its dimension, which I know -- actually, what is the dimension of the left null space?
So here's my matrix.
What's the rank of the matrix?
And the dimension of the null -- of the left null space is supposed to be m-r.
It's 3 -2, 1.
I believe that the left null space is one dimensional.
There is one combination of those three rows that produces the zero row.
There's a basis -- a basis for the left null space has only got one vector in it.
And what is that vector?
It's here in the last row of E. But we could have seen it earlier.
What combination of those rows gives the zero row?
-1 of that plus one of that.
So a basis for the left null space of this matrix -- I'm looking for combinations of rows that give the zero row if I'm looking at the left null space.
For the null space, I'm looking at combinations of columns to get the zero column.
Now I'm looking at combinations of these three rows to get the zero row, and of course there is my zero row, and here is my vector that produced it.
-1 of that row and one of that row.
Obvious.
OK.
So in that example -- and actually in all examples, we have seen how to produce a basis for the left null space.
I won't ask you that all the time, because -- it didn't come out immediately from R. We had to keep track of E for that left null space.
But at least it didn't require us to transpose the matrix and start all over again.
OK, those are the four subspaces.
Can I review them?
The row space and the null space are in R^n.
Their dimensions add to n. The column space and the left null space are in R^m, and their dimensions add to m. OK.
So let me close these last minutes by pushing you a little bit more to a new type of vector space.
All our vector spaces, all the ones that we took seriously, have been subspaces of some real three or n dimensional space.
Now I'm going to write down another vector space, a new vector space.
Say all three by three matrices.
My matrices are the vectors.
Is that all right?
I'm just naming them.
You can put quotes around vectors.
Every three by three matrix is one of my vectors.
Now how I entitled to call those things vectors?
I mean, they look very much like matrices.
But they are vectors in my vector space because they obey the rules.
All I'm supposed to be able to do with vectors is add them -- I can add matrices -- I'm supposed to be able to multiply them by scalar numbers like seven -- well, I can multiply a matrix by And that -- and I can take combinations of matrices, I can take three of one matrix minus five of another matrix.
And those combinations, there's a zero matrix, the matrix that has all zeros in it.
If I add that to another matrix, it doesn't change it.
All the good stuff.
If I multiply a matrix by one it doesn't change it.
All those eight rules for a vector space that we never wrote down, all easily satisfied.
So now we have a different -- now of course you can say you can multiply those matrices.
I don't care.
For the moment, I'm only thinking of these matrices as forming a vector space -- so I only doing A plus B and c times A. I'm not interested in A B for now.
The fact that I can multiply is not relevant to th- to a vector space.
OK.
So I have three by three matrices.
And how about subspaces?
What's -- tell me a subspace of this matrix space.
Let me call this matrix space M. That's my matrix space, my space of all three by three matrices.
Tell me a subspace of it.
What about the upper triangular matrices?
OK.
So subspaces.
Subspaces of M. All, all upper triangular matrices.
Another subspace.
All symmetric matrices.
The intersection of two subspaces is supposed to be a subspace.
We gave a little effort to the proof of that fact.
If I look at the matrices that are in this subspace -- they're symmetric, and they're also in this subspace, they're upper triangular, what do they look like?
Well, if they're symmetric but they have zeros below the diagonal, they better have zeros above the diagonal, so the intersection would be diagonal matrices.
That's another subspace, smaller than those.
How can I use the word smaller?
Well, I'm now entitled to use the word smaller.
I mean, well, one way to say is, OK, these are contained in those.
These are contained in those.
But more precisely, I could give the dimension of these spaces.
So I could -- we can compute -- let's compute it next time, the dimension of all upper -- of the subspace of upper triangular three by three matrices.
The dimension of symmetric three by three matrices.
The dimension of diagonal three by three matrices.
Well, to produce dimension, that means I'm supposed to produce a basis, and then I just count how many vecto- how many I needed in the basis.
Let me give you the answer for this one.
What's the dimension?
The dimension of this -- say, this subspace, let me call it D, all diagonal matrices.
The dimension of this subspace is -- as I write you're working it out -- three.
Because here's a matrix in this -- it's a diagonal matrix.
Here's another one.
Here's another one.
Better make it diagonal, let me put a seven there.
That was not a very great choice, but it's three diagonal matrices, and I believe that they're a basis.
I believe that those three matrices are independent and I believe that any diagonal matrix is a combination of those three.
So they span the subspace of diagonal matrices.
Do you see that idea?
It's like stretching the idea from R^n to R^(n by n), three by three.
But the -- we can still add, we can still multiply by numbers, and we just ignore the fact that we can multiply two matrices together.
OK, thank you.
That's lecture ten.
OK.
This is linear algebra lecture eleven.
And at the end of lecture ten, I was talking about some vector spaces, but they're -- the things in those vector spaces were not what we usually call vectors.
Nevertheless, you could add them and you could multiply by numbers, so we can call them vectors.
I think the example I was working with they were matrices.
So the -- so we had like a matrix space, the space of all three by three matrices.
And I'd like to just pick up on that, because -- we've been so specific about n dimensional space here, and you really want to see that the same ideas work as long as you can add and multiply by scalars.
So these new, new vector spaces, the example I took was the space M of all three by three matrices.
OK.
I can add them, I can multiply by scalars.
I can multiply two of them together, but I don't do that.
That's not part of the vector space picture.
The vector space part is just adding the matrices and multiplying by numbers.
And that's fine, we stay within this space of three by three matrices.
And I had some subspaces that were interesting, like the symmetric, the subspace of symmetric matrices, symmetric three by threes.
Or the subspace of upper triangular three by threes.
Now I, I use the word subspace because it follows the rule.
If I add two symmetric matrices, I'm still symmetric.
If I multiply two symmetric matrices, is the product automatically symmetric?
No.
But I'm not multiplying matrices.
I'm just adding.
So I'm fine.
This is a subspace.
Similarly, if I add two upper triangular matrices, I'm still upper triangular.
And, that's a subspace.
Now I just want to take these as example and ask, well, what's a basis for that subspace?
What's the dimension of that subspace?
And what's bd- dimension of the whole space?
So, there's a natural basis for all three by three matrices, and why don't we just write it down.
So, so M, a basis for M. Again, all three by threes.
OK. And then I'll just count how many members are in that basis and I'll know the dimension.
And OK, it's going to take me a little time.
In fact, what is the dimension?
Any idea of what I'm coming up with next?
How many numbers does it take to specify that three by three matrix?
Nine.
Nine is the, is the dimension I'm going to find.
And the most obvious basis would be the matrix that's that matrix and then this matrix with a one there and that's two of them, shall I put in the third one, and then onwards, and the last one maybe would end with the one.
OK. That's like the standard basis.
In fact, our space is practically the same as nine dimensional space.
It's just the nine numbers are written in a square instead of in a column.
But somehow it's different and, and ought to be thought of as -- natural for itself.
Because now what about the symmetric three by threes?
So that's a subspace.
Just let's just think, what's the dimension of that subspace and what's a basis for that subspace.
OK. And I guess this question occurs to me.
If I look at this subspace of symmetric three by threes, well, how many of these original basis members belong to the subspace?
I think only three of them do.
This one is symmetric.
This last one is symmetric.
And the one in the middle with a, with a one in that position -- in the two two position, would be symmetric.
But so I've got three of these original nine are symmetric, but, so this is an example where -- but that's, that's not all, right?
What's the dimension?
Let's put the dimensions down.
Dimension of the, of M, was nine.
What's the dimension of -- shall we call this S -- is what?
What's the dimension of this?
I'm sort of taking simple examples where we can, we can, spot the answer to these questions.
So how many -- if I have a symmetric -- think of all symmetric matrices as a subspace, how many parameters do I choose in three by three symmetric matrices?
Six, right.
If I choose the diagonal that's three, and the three entries above the diagonal, then I know what the three entries below.
So the dimension is six.
I guess what's the dimension of this here?
Let's call this space U for upper triangular.
So what's the dimension of that space of all upper triangular three by threes?
Again six.
Again six.
And, but we haven't got a -- we haven't seen -- well, actually, maybe we have got a basis here for, the upper triangulars.
I guess six of these guys, one, two, three, four, and a, and a couple more, would be upper triangular.
So there's a accidental case where the big basis contains in it a basis for the subspace.
But with the symmetric guy, it didn't have.
The symmetric guy the, basis -- so you see -- a basis is the basis for the big space, we generally need to think it all over again to get a basis for the subspace.
And then how do I get other subspaces?
Well, we spoke before about, the subspace the symmetric matrices and the upper triangular.
This is symmetric and upper triangular.
What's the, what's the dimension of that space?
OK. Well, what's in that space?
So what's -- if a matrix is symmetric and also upper triangular, that makes it diagonal.
So this is the same as the diagonal matrices, diagonal three by threes.
And the dimension of this, of S intersect U, right -- you're OK with that symbol?
That's, that's the vectors that are in both S and U, and that's D. So S intersect U is the diagonals.
And the dimension of the diagonal matrices is three.
And we've got a basis, no problem.
OK, as I write that, I think, OK, what about putting -- so this is like, this intersection -- is taking all the vectors that are in both, that are symmetric and also upper triangular.
Now we looked at the union.
Suppose I take the matrices that are symmetric or upper triangular.
What -- why was that no good?
So why is it no -- why I not interested in the union, putting together those two subspaces?
So this, these are matrices that are in S or in U, or possibly both, so they, the diagonals included.
But what's bad about this?
It's not a subspace.
It's like having, taking, you know, a couple of lines in the plane and stopping there.
A line -- this is -- so there's a three dimensional subspace of a nine dimensional space, there's -- ooh, sorry, six.
There's a six dimensional subspace of a nine dimensional space.
There's another one.
But they, they're headed in different directions, so we, we can't just put them together.
We have to fill in.
So that's what we do.
To get this bigger space that I'll write with a plus sign, this is combinations of things in S and things in U. OK.
So that's the final space I'm going to introduce.
I have a couple of subspaces.
I can take their intersection.
And now I'm interested in not their union but their sum.
So this would be the, this is the intersection, and this will be their sum.
So what do I need for a subspace here?
I take anything in S plus anything in U. I don't just take things that are in S and pop in also, separately, things that are in U.
This is the sum of any element of S, that is, any symmetric matrix, plus any in U, any element of U. OK. Now as long as we've got an example here, tell me what we get.
If I take every symmetric matrix, take all symmetric matrices, and add them to all upper triangular matrices, then I've got a whole lot of matrices and it is a subspace.
And what's -- it's a vector space, and what vector space would I then have?
Any idea what, what matrices can I get out of a symmetric plus an upper triangular?
I can get anything.
I get all matrices.
I get all three by threes.
It's worth thinking about that.
It's just like stretch your mind a little, just a little, to, to think of these subspaces and what their intersection is and what their sum is.
And now can I give you a little -- oh, well, let's figure out the dimension.
So what's the dimension of S plus U?
In this example is nine, because we got all three by threes.
So the original spaces had, the original symmetric space had dimension six and the original upper triangular space had dimension six.
And actually I'm seeing here a nice formula.
That the dimension of S plus the dimension of U -- if I have two subspaces, the dimension of one plus the dimension of the other -- equals the dimension of their intersection plus the dimension of their sum.
Six plus six is three plus nine.
That's kind of satisfying, that these natural operations -- and we've -- this is it, actually, this is the set of natural things to do with, with subspaces.
That, the dimensions come out in a good way.
OK. Maybe I'll take just one more example of a vector space that doesn't have vectors in it.
It's come from differential equations.
So this is a one more new vector space that we'll give just a few minutes to.
Suppose I have a differential equation like d^2y/dx^2+ y=0.
OK.
I look at the solutions to that equation.
So what are the solutions to that equation?
y=cos(x) is a solution.
y=sin(x) is a solution.
y equals -- well, e to the (ix) is a solution, if you want, if you allow me to put that in.
But why should I put that in?
It's already there.
You see, I'm really looking at a null space here.
I'm looking at the null space of a differential equation.
That's the solution space.
And describe the solution space, all solutions to this differential equation.
So the equation is y''+y=0.
Cosine's, cosine's a solution, sine is a solution.
Now tell me all the solutions.
They're -- so I don't need e^(ix).
Forget that.
What are all the complete solutions?
Is what?
A combination of these.
The complete solution is y equals some multiple of the cosine plus some multiple of the sine.
That's a vector space.
That's a vector space.
What's the dimension of that space?
What's a basis for that space?
OK, let me ask you a basis first.
If I take the set of solutions to that second order differential equation -- there it is, those are the solutions.
What's a basis for that space?
Now remember, what's the, what question I asking?
Because if you know the question I'm asking, you'll see the answer.
A basis means all the guys in the space are combinations of these basis vectors.
Well, this is a basis.
sin x, cos x there is a basis.
Those two -- they're like the special solutions, right?
We had special solutions to Ax=b.
Now we've got special solutions to differential equations.
Sorry, we had special solutions to Ax=0, I misspoke.
The special solutions were for the null space just as here we're talking about the null space.
Do you see that here is a -- those two -- and what's the dimension of the solution space?
How many vectors in this basis?
Two, the sine and cosine.
Are those the only basis for this space?
By no means.
e^(ix) and e^(-ix) would be another basis.
Lots of bases.
But do you see that really what a course in differential -- in linear differential equations is about is finding a basis for the solution space.
The dimension of the solution space will always be -- will be two, because we have a second order equation.
So that's, like there's 18.03 in -- five minutes of 18.06 is enough to, to take care of 18.03.
So there's a -- that's one more example.
OK. And of course the point of the example is these things don't look like vectors.
They look like functions.
But we can call them vectors, because we can add them and we can multiply by constants, so we can take linear combinations.
That's all we have to be allowed to do.
So that's really why this idea of linear algebra and basis and dimension and so on plays a wider role than -- our constant discussions of m by n matrices.
OK. That's what I wanted to say about that topic.
Now of course the key, number associated with matrices, to go back to that number, is the rank.
And the rank, what do we know about the rank?
Well, we know it's not bigger than m and it's not bigger than n. So but I'd like to have a little discussion on the rank.
Maybe I'll put that here.
So I'm picking up this topic of rank one matrices.
And the reason I'm interested in rank one matrices is that they ought to be simple.
If the rank is only one, the matrix can't get away from us.
So for example, let me take -- let me create a rank one matrix.
OK.
Suppose it's three -- suppose it's two by three.
And let me give you the first row.
What can the second row be?
Tell me a possible second row here, for, for this matrix to have rank one.
A possible second row is?
Two eight ten.
The second row is a multiple of the first row.
It's not independent.
So tell me a basis for the -- oh yeah, sorry to keep bringing up these same questions.
After the quiz I'll stop, but for now, tell me a basis for the row space.
A basis for the row space of that matrix is the first row, right?
The first row, one four five.
A basis for the column space of this matrix is?
What's the dimension of the column space?
The dimension of the column space is also one, right?
Because it's also the rank.
The dimension -- you remember the dimension of the column space equals the rank equals the dimension of the column space of the transpose, which is the row space of A. OK, and in this case it's one, r is one.
And sure enough, all the columns are -- all the other columns are multiples of that column.
Now there's -- there ought to be a nice way to see that, and here it is.
I can write that matrix as its pivot column, one two, times its -- times one four five.
A column times a row, one column times one row gives me a matrix, right?
If I multiply a column by a row, that, g- that's a two by one matrix times a one by three matrix, and the result of the multiplication is two by three.
And it comes out right.
So what I want to -- my point is the rank one matrices that every rank one matrix has the form some column times some row.
So U is a column vector, V is a column vector -- but I make it into a row by putting in V transpose.
So that's the -- complete picture of rank one matrices.
We'll be interested in rank one matrices.
Later we'll find, oh, their determinant, that'll be easy, their eigenvalues, that'll be interesting.
Rank one matrices are like the building blocks for all matrices.
And actually maybe you can guess.
If I took any matrix, a five by seventeen matrix of rank four, then it seems pretty likely -- and it's true, that I could break that five by seventeen matrix down as a combination of rank one matrices.
And probably how many of those would I need?
If I have a five by seventeen matrix of rank four, I'll need four of them, right.
Four rank one matrices.
So the rank one matrices are the, are the building blocks.
And out -- I can produce every, I can produce every five by -- every rank four matrix out of four rank one matrices.
That brings me to a question, of course.
OK. Would the rank four matrices form a subspace?
Let me take all five by seventeen matrices and think about rank four -- the subset of rank four matrices.
Let me -- I'll write this down.
You seem I'm reviewing for the quiz, because I'm asking the kind of questions that are short enough but -- that bring out do you know what these words mean.
So I take -- my matrix space M now is all five by seventeen matrices.
And now the question I ask is the subset of, of rank four matrices, is that a subspace?
If I add a matrix of -- so if I multiply a matrix of rank four by -- of rank four or less, let's say, because I have to let the zero matrix in if it's going to be a subspace.
But, but that doesn't just because the zero matrix got in there doesn't mean I have a subspace.
So if I -- so the, the question really comes down to -- if I add two rank four matrices, is the sum rank four?
What do you think?
If -- no, not usually.
Not usually.
If I add two rank four matrices, the sum is probably -- what could I say about the sum?
Well, actually, well, the rank could be five.
It's a general fact, actually, that the rank of A plus B can't be more than rank of A plus the rank of B.
So this would say if I added two of those, the rank couldn't be larger than eight, but I know actually the rank couldn't be as large as eight anyway.
What -- how big could the rank be, for, for the rank of a matrix in M?
Could be as large as five, right, right.
So they're all sort of natural ideas.
So it's rank four matrices or rank one matrices -- let me, let me change that to rank one.
Let me take the subset of rank one matrices.
Is that a vector space?
If I add a rank one matrix to a rank one matrix?
No.
It's most likely going to have rank two.
So this is -- So I'll just make that point.
Not a subspace.
OK. OK. Those are topics that I wanted to, just fill out the, the previous lectures.
The I'll ask one more subspace question, a, a more, a more, likely example.
Suppose I'm in -- let me put, put this example on a new board.
Suppose I'm in R, in R^4.
So my typical vector in R^4 has four components, v1, v2, v3, and v4.
Suppose I take the subspace of vectors whose components add to zero.
So I let S be all v, all vectors v in four dimensional space with v1+v2+v3+v4=0.
So I just want to consider that bunch of vectors.
Is it a subspace, first of all?
It is a subspace.
It is a subspace.
What's -- how do we see that?
It is a subspace.
I -- formally I should check.
If I have one vector that with whose components add to zero and I multiply that vector by six -- the components still add to zero, just six times as -- six times zero.
If I have a couple of v and a w and I add them, the, the components still add to zero.
OK, it's a subspace.
What's the dimension of that space and what's a basis for that space?
So you see how I can just describe a space and we -- we can ask for the dimension -- ask for the basis first and the dimension.
Of course, the dimension's the one that's easy to tell me in a single word.
What's the dimension of our subspace S here?
And a basis tell me -- some vectors in it.
Well, I'm going to make ask you again to guess the dimension.
Again I think I heard it.
The dimension is three.
Three.
Now how does this connect to our Ax=0?
Is this the null space of something?
Is that the null space of a matrix?
And then we can look at the matrix and, and we know everything about those subspaces.
This is the null space of what matrix?
What's the matrix where the null space is then Ab=0.
So I want this equation to be Ab=0.
b is now the vector.
And what's the matrix that, that we're seeing there?
It's the matrix of four ones.
Do you see that that's -- that if I look at Ab=0 for this matrix A, I multiply by b and I get this requirement, that the components add to zero.
So I'm really when I speak about S -- I'm speaking about the null space of that matrix.
OK. Let's just say we've got a matrix now, we want its null space.
Well, we -- tell me its rank first.
The rank of that matrix is one, thanks.
So r is one.
What's the general formula for the dimension of the null space?
The dimension of the null space of a matrix is -- in general, an m by n matrix of rank r?
How many independent guys in the null space?
n-r, right?
n-r.
In this case, n is four, four columns.
The rank is one, so the null space is three dimensions.
So of course y- you could see it in this case, but you can also see it here in our systematic way of dealing with the four fundamental subspaces of a matrix.
So what actually what, what are all four subspaces then?
The row space is clear.
The row space is in R^4.
Yeah, can we take the four fundamental subspaces of this matrix?
Let's just kill this example.
The row space is one dimensional.
It's all multiples of that, of that row.
The null space is three dimensional.
Oh, you better give me a basis for the null space.
So what's a basis for the null space?
The special solutions.
To find the special solutions, I look for the free variables.
The free variables here are -- there's the pivot.
The free variables are two, three, and four.
So the basis, basis for S, for S will be -- I'm expecting three vectors, three special solutions.
I give the value one to that free variable, and what's the pivot variable if the -- this is going to be a vector in S?
Minus one.
Now they're always added to -- the entries add to zero.
The second special solution has a one in the second free variable, and again a minus one makes it right.
The third one has a one in the third free variable, and again a minus one makes it right.
That's my answer.
That's the answer I would be looking for.
The -- a basis for this subspace S, you would just list three vectors, and those would be the natural three to list.
Not the only possible three, but those are the special three.
OK, tell me about the column space, What's the column space of this matrix A?
So the column space is a subspace of R^1, because m is only one.
The columns only have one component.
So the column space of S, the column space of A is somewhere in the space R^1, because we only have -- these columns are short.
And what is the column space actually?
I just, it's just talking with these words is what I'm doing.
The column space for that matrix is R^1.
The column space for that matrix is all multiples of that column.
And all multiples give you all of R^1.
And what's the, the remaining fourth space, the null space of A transpose is what?
So we transpose A.
We look for combinations of the columns now that give zero for A transpose.
And there aren't any.
The only thing, the only combination of these rows to give the zero row is the zero combination.
OK.
So let's just check dimensions.
The null space has dimension three.
The row space has dimension one.
Three plus one is four.
The column space has dimension one, and what's the dimension of this, like, smallest possible space?
What's the dimension of the zero space?
It's a subspace.
Zero.
What else could it be?
I mean, let's -- we have to take a reasonable answer -- and the only reasonable answer is zero.
So one plus zero gives -- this was n, the number of columns, and this is m, the number of rows.
And let's just, let me just say again then the, the, the subspace that has only that one point, that point is zero dimensional, of course.
And the basis is empty, because if the dimension is zero, there shouldn't be anybody in the basis.
So the basis of that smallest subspace is the empty set.
And the number of members in the empty set is zero, so that's the dimension.
OK. Good.
Now I have just five minutes to tell you about -- well, actually, about some, some, some, this is now, this last topic of small world graphs, and leads into, a lecture about graphs and linear algebra.
But let me tell you -- in these last minutes the graph that I interested in.
It's the graph where -- so what is a graph?
Better tell you that first.
OK. What's a graph?
OK.
This isn't calculus.
We're not, I'm not thinking of, like, some sine curve.
The word graph is used in a completely different way.
It's a set of, a bunch of nodes and edges, edges connecting the nodes.
So I have nodes like five nodes and edges -- I'll put in some edges, I could put, include them all.
There's -- well, let me put in a couple more.
There's a graph with five nodes and one two three four five six edges.
And some five by six matrix is going to tell us everything about that graph.
Let me leave that matrix to next time and tell you about the question I'm interested in.
Suppose, suppose the graph isn't just, just doesn't have just five nodes, but suppose every, suppose every person in this room is a node.
And suppose there's an edge between two nodes if those two people are friends.
So have I described a graph?
It's a pretty big graph, hundred, hundred nodes.
And I don't know how many edges are in there.
There's an edge if you're friends.
So that's the graph for this class.
A, a similar graph you could take for the whole country, so two hundred and sixty million nodes.
And edges between friends.
And the question for that graph is how many steps does it take to get from anybody to anybody?
What two people are furthest apart in this friendship graph, say for the US?
By furthest apart, I mean the distance from -- well, I'll tell you my distance to Clinton.
It's two.
I happened to go to college with somebody who knows Clinton.
I don't know him.
So my distance to Clinton is not one, because I don't, happily or not, don't know him.
But I know somebody who does.
He's a Senator and so I presume he knows him.
OK.
I don't know what your -- well, what's your distance to Clinton?
Well, not more than three, right.
Actually, true.
You know me.
I take credit for reducing your Clinton distance to three -- what's your distance to Monica.
Not, anybody below -- below four is in trouble here.
Or maybe three, but, right.
So -- and what's Hillary's distance to Monica?
I don't think we'd better put that on tape here.
That's one or two, I guess.
Is that right?
I don't -- well, we won't, think more about that.
So actually, the, the real question is what are large distances?
How, how far apart could people be separated?
And roughly this number six degrees of separation has kind of appeared as the movie title, as the book title, and it's with this meaning.
That roughly speaking -- six might be a fairly -- not too many people.
If you sit next to somebody on an airplane, you get talking to them.
You begin to discuss mutual friends to sort of find out, OK, what connections do you have, and very often you'll find you're connected in, like, two or three or four steps.
And you remark, it's a small world, and that's how this expression small world came up.
But six, I don't know if you could find -- if it took six, I don't know if you would successfully discover those six in a, in an airplane conversation.
But here's the math question, and I'll leave it for next, for lecture twelve, and do a lot of linear algebra in lecture twelve.
But the interesting point is that with a few shortcuts, the distances come down dramatically.
That, I mean, all your distances to Clinton immediately drop to three by taking linear algebra.
That's, like, an extra bonus for taking linear algebra.
And to understand mathematically what it is about these graphs -- or like the graphs of the World Wide Web.
There's a fantastic graph.
So many people would like to understand and model the web.
What the -- where the edges are links and the nodes are, sites, websites.
I'll leave you with that graph, and I'll see you -- have a good weekend, and see you on Monday.
This is lecture twelve.
OK. We've reached twelve lectures.
And this one is more than the others about applications of linear algebra.
And I'll confess.
When I'm giving you examples of the null space and the row space, I create a little matrix.
You probably see that I just invent that matrix as I'm going.
And I feel a little guilty about it, because the truth is that real linear algebra uses matrices that come from somewhere.
They're not just, like, randomly invented by the instructor.
They come from applications.
They have a definite structure.
And anybody who works with them gets, uses that structure.
I'll just report, like, this weekend I was at an event with chemistry professors.
OK, those guys are row reducing matrices, and what matrices are they working with?
Well, their little matrices tell them how much of each element goes into the -- or each molecule, how many molecules of each go into a reaction and what comes out.
And by row reduction they get a clearer picture of a complicated reaction.
And this weekend I'm going to -- to a sort of birthday party at Mathworks.
So Mathworks is out Route 9 in Natick.
That's where Matlab is created.
It's a very, very successful, software, tremendously successful.
And the conference will be about how linear algebra is used.
And so I feel better today to talk about what I think is the most important model in applied math.
And the discrete version is a graph.
So can I draw a graph?
Write down the matrix that's associated with it, and that's a great source of matrices.
You'll see.
So a graph is just, so a graph -- to repeat -- has nodes and edges.
OK. And I'm going to write down the graph, a graph, so I'm just creating a small graph here.
As I mentioned last time, we would be very interested in the graph of all, websites.
Or the graph of all telephones.
I mean -- or the graph of all people in the world.
Here let me take just, maybe nodes one two three -- well, I better put in an -- I'll put in that edge and maybe an edge to, to a node four, and another edge to node four.
How's that?
So there's a graph with four nodes.
So n will be four in my -- n equal four nodes.
And the matrix will have m equal the number -- there'll be a row for every edge, so I've got one two three four five edges.
So that will be the number of rows.
And I have to to write down the matrix that I want to, I want to study, I need to give a direction to every edge, so I know a plus and a minus direction.
So I'll just do that with an arrow.
Say from one to two, one to three, two to three, one to four, three to four.
That just tells me, if I have current flowing on these edges then I know whether it's -- to count it as positive or negative according as whether it's with the arrow or against the arrow.
But I just drew those arrows arbitrarily.
OK. Because I -- my example is going to come -- the example I'll -- the words that I will use will be words like potential, potential difference, currents.
In other words, I'm thinking of an electrical network.
But that's just one possibility.
My applied math class builds on this example.
It could be a hydraulic network, so we could be doing, flow of water, flow of oil.
Other examples, this could be a structure.
Like the -- a design for a bridge or a design for a Buckminster Fuller dome.
Or many other possibilities, so many.
So l- but let's take potentials and currents as, as a basic example, and let me create the matrix that tells you exactly what the graph tells you.
That's --, that's it.
So now I'll call it the incidence matrix, incidence matrix.
So let me write it down, and you'll see, OK. what its properties are.
So every row corresponds to an edge.
I have five rows from five edges, and let me write down again what this graph looks like.
OK, the first edge, edge one, goes from node one to two.
So I'm going to put in a minus one and a plus one in th- this corresponds to node one two three and four, That's a basis for the null space.
the four columns.
The five rows correspond -- the first row corresponds to edge one.
Edge one leaves node one and goes into node two, and that -- and it doesn't touch three and four.
Edge two, edge two goes -- oh, I haven't numbered these edges.
I just figured that was probably edge one, but I didn't say so.
Let me take that to be edge one.
Let me take this to be edge two.
Let me take this to be edge three.
This is edge four.
Ho, I'm discovering -- no, wait a minute.
Did I number that twice?
Here's edge four.
And here's edge five.
All right.
OK?
So, so edge one, as I said, goes from node one to two.
Edge two goes from two to three, node two to three, so minus one and one in the second and third columns.
Edge three goes from one to three.
I'm, I'm tempted to stop for a moment with those three edges.
The null space is actually one dimensional, and it's the line Edges one two three, those form what would we, A basis for the null space will be just that.1 what do you call the, the little, the little, the subgraph formed by edges one, two, and three?
That's a loop.
And the number of loops and the position of the loops will be crucial.
OK. Actually, here's a interesting point about loops.
If I look at those rows, corresponding to edges one two three, and these guys made a loop.
You want to tell me -- if I just looked at that much of the matrix it would be natural for me to ask, are those rows independent?
Are the rows independent?
And can you tell from looking at that if they are or are not independent?
Do you see a, a relation between those three rows?
Yes.
If I add that row to that row, I get this row.
So, so that's like a hint here that loops correspond to dependent, linearly dependent column -- linearly dependent give me a basis for the null space.
rows.
OK, let me complete the incidence matrix.
Number four, edge four is going from node one to node four.
And the fifth edge is going from node three to node four.
OK.
There's my matrix.
It came from the five edges and the four nodes.
And if I had a big graph, I'd have a big matrix.
And what questions do I ask about matrices?
Can I ask -- here's the review now.
There's a matrix that comes from somewhere.
of all vectors through that one.
If, if it was a big graph, it would be a large matrix, but a lot of zeros, right?
Because every row only has two non-zeros.
So the number of -- it's a very sparse matrix.
The number of non-zeros is exactly two times five, it's two m. Every row only has two non-zeros.
And that's with a lot of structure.
And -- that was the point I wanted to begin with, that graphs, that real graphs from real -- real matrices from genuine problems have structure.
OK. We can ask, and because of the structure, we can answer, If it -- yeah, let me ask you just always, the, the main questions about matrices.
So first question, what about the null space?
So what I asking if I ask you for the null space of that matrix?
I'm asking you if I'm looking at the columns of the matrix, four columns, and I'm asking you, So there's a basis for it, and here is the whole null are those columns independent?
If the columns are independent, then what's in the null space?
Only the zero vector, right?
The null space contains -- tells us what combinations of the columns -- it tells us how to combine columns to get zero.
Can -- and is there anything in the null space of this matrix other than just the zero vector?
In other words, are those four columns independent or dependent?
What else is in the null space?
OK. That's our question.
Let me, I don't know if you see the answer.
Whether there's -- so let's see.
I guess we could do it properly.
space.
We could solve Ax=0.
So let me solve Ax=0 to find the null space.
OK. What's Ax?
Can I put x in here in, in little letters?
x1, x2, x3, x4, that's -- it's got four columns.
Ax now is that matrix times x.
And what do I get for Ax?
If the camera can keep that matrix multiplication there, I'll put the answer here.
Ax equal -- what's the first component of Ax?
Can you take that first row, minus one one zero zero, and multiply by the x, and of course you get x2-x1.
space.
The second row, I get x3-x2.
From the third row, I get x3-x1.
Any multiple of one one one one, it's the whole line in four From the fourth row, I get x4-x1.
And from the fifth row, I get x4-x3.
And I want to know when is the thing zero.
This is my equation, Ax=0.
Notice what that matrix A is doing, what we've created a matrix that computes the differences across every edge, the differences in potential.
differences are zero, and that x is in the null Let me even begin to give this interpretation.
I'm going to think of this vector x, which is x1 x2 x3 x4, as the potentials at the nodes.
So I'm introducing a word, potentials at the nodes.
And now if I multiply by A, I get these -- I get these five components, x2-x1, et cetera.
And what are they?
They're potential differences.
That's what A computes.
If I have potentials at the nodes and I multiply by A, it gives me the potential differences, the differences in potential, across the edges.
OK.
When are those differences all zero?
So I'm looking for the null space.
Of course, if all the (x)s are zero then I get zero.
That, that just tells me, of course, the zero vector is in the null space.
But w- there's more in the null space.
Those columns are -- of A are dependent, right -- because I can find solutions to that equation.
dimensional space.
Tell me -- the null space.
Tell me one vector in the null space, so tell me an x, it's got four components, and it makes that thing zero.
So what's a good x to do that?
One one one one, constant potential.
If the potentials are constant, then all the potential Do you see that that's the null space?
So the, so the dimension of the null space of A is one.
And there's a basis for it, and there's everything that's in it.
Good.
And what does that mean physically?
I mean, what does that mean in the application?
That guy in the null space.
It means that the potentials can only be determined up to a constant.
Potential differences are what make current flow.
That's what makes things happen.
It's these potential differences that will make something move in the, in our network, between x2- between node two and node one.
Nothing will move if all potentials are the same.
If all potentials are c, c, c, and c, then nothing will move.
So we're, we have this one parameter, this arbitrary constant that raises or drops all the potentials.
It's like ranking football teams, whatever.
We have a, there's a, there's a constant -- or looking at temperatures, you know, there's a flow of heat from higher temperature to lower temperature.
If temperatures are equal there's no flow, and therefore we can measure -- we can measure temperatures by, Celsius or we can start at absolute zero.
And that arbitrary -- it's the same arbitrary constant that, that was there in calculus.
In calculus, right, when you took the integral, the indefinite integral, there was a plus c, and you had to set a starting point to know what that c was.
So here what often happens is we fix one of the potentials, like the last one.
So a typical thing would be to ground that node.
To set its potential at zero.
And if we do that, if we fix that potential so it's not unknown anymore, then that column disappears and we have three columns, and those three columns are independent.
So I'll leave the column in there, but we'll remember that grounding a node is the way to get it out.
And grounding a node is the way to -- setting a node -- setting a potential to zero tells us the, the base for all potentials.
Then we can compute the others.
But what's the -- now I've talked enough to ask OK. what the rank of the matrix is?
What's the rank then?
The rank of the matrix.
So we have a five by four matrix.
We've located its null space, one dimensional.
How many independent columns do we have?
What's the rank?
It's three.
And the first three columns, or actually any three columns, will be independent.
Any three potentials are independent, good variables.
The fourth potential is not, we need to set, and typically we ground that node.
OK. Rank is three.
Rank equals three.
OK. Let's see, do I want to ask you about the column space?
The column space is all combinations of those columns.
I could say more about it and I will.
Let me go to the null space of A transpose, because the equation A transpose y equals zero is probably the most fundamental equation of applied mathematics.
All right, let's talk about that.
That deserves our attention.
A transpose y equals zero.
Let's -- let me put it on here.
So A transpose y equals zero.
OK.
So now I'm finding the null space of A transpose.
Oh, and if I ask you its dimension, you could tell me what it is.
What's the dimension of the null space of A transpose?
We now know enough to answer that question.
What's the general formula for the dimension of the null space of A transpose?
A transpose, let me even write out A transpose.
This A transpose will be n by m, right?
In this case, it'll be four by five.
n by m. Those columns will become rows.
Minus one zero minus one minus one zero is now the first row.
The second row of the matrix, one minus one and three zeros.
The third column now becomes the third row, zero one one zero minus one.
And the fourth column becomes the fourth row.
OK, good.
There's A transpose.
That multiplies y, y1 y2 y3 y4 and y5.
OK. Now you've had time to think about this question.
What's the dimension of the null space, if I set all those -- wow.
Usually -- sometime during this semester, I'll drop one of these erasers behind there.
That's a great moment.
There's no recovery.
There's -- centuries of erasers are back there.
OK. OK, what's the dimension of the null space?
Give me the general formula first in terms of r and m and n. This is like crucial, you -- we struggled to, to decide what dimension meant, and then we figured out what it equaled for an m by n matrix of rank r, and the answer was m-r, right?
There are m=5 components, m=5 columns of A transpose.
And r of those columns are pivot columns, because it'll have r pivots.
It has rank r. And m-r are the free ones now for A transpose, so that's five minus three, so that's two.
And I would like to find this null space.
I know its dimension.
Now I want to find out a basis for it.
And I want to understand what this equation is.
So let me say what A transpose y actually represents, why I'm interested in that equation.
I'll put it down with those old erasers and continue this.
Here's the great picture of applied mathematics.
So let me complete that.
There's a matrix that I'll call C that connects potential differences to currents.
So I'll call these -- these are currents on the edges, y1 y2 y3 y4 and y5.
Those are currents on the edges.
And this relation between current and potential difference is Ohm's Law.
This here is Ohm's Law.
Ohm's Law says that the current on an edge is some number times the potential drop.
That's -- and that number is the conductance of the edge, one over the resistance.
This is the old current is, is, the relation of current, resistance, and change in potential.
So it's a change in potential that makes some current happen, and it's Ohm's Law that says how much current happens.
OK. And then the final step of this framework is the equation A transpose y equals zero.
And that's -- what is that saying?
It has a famous name.
It's Kirchoff's Current Law, KCL, Kirchoff's Current Law, A transpose y equals zero.
So that when I'm solving, and when I go back up with this blackboard and solve A transpose y equals zero, it's this pattern of -- that I want you to see.
That we had rectangular matrices, but -- and real applications, but in those real applications comes A and A transpose.
So our four subspaces are exactly the right things to know about.
All right.
Let's know about that null space of A transpose.
Wait a minute, where'd it go?
There it is.
OK. OK. Null space of A transpose.
We know what its dimension should be.
Let's find out -- tell me a vector in it.
Tell me -- now, so what I asking you?
I'm asking you for five currents that satisfy Kirchoff's Current Law.
So we better understand what that law says.
That, that law, A transpose y equals zero, what does that say, say in the first row of A transpose?
That says -- the so the first row of A transpose says minus y1 minus y3 minus y4 is zero.
Where did that equation come from?
Let me -- I'll redraw the graph.
Can I redraw the graph here, so that we -- maybe here, so that we see again -- there was node one, node two, node three, node four was off here.
That was, that was our graph.
We had currents on those.
We had a current y1 going there.
We had a current y -- what were the other, what are those edge numbers?
y4 here and y3 here.
And then a y2 and a y5.
I'm, I'm just copying what was on the other board so it's ea- convenient to see it.
What is this equation telling me, this first equation of Kirchoff's Current Law?
What does that mean for that graph?
Well, I see y1, y3, and y4 as the currents leaving node one.
So sure enough, the first equation refers to node one, and what does it say?
It says that the net flow is zero.
That, that equation A transpose y, Kirchoff's Current Law, is a balance equation, a conservation law.
Physicists, be overjoyed, right, by this stuff.
It, it says that in equals out.
And in this case, the three arrows are all going out, so it says y1, y3, and y4 add to zero.
Let's take the next one.
The second row is y1-y2, and that's all that's in that row.
And that must have something to do with node two.
And sure enough, it says y1=y2, current in equals current out.
The third one, y2 plus y3 minus y5 equals zero.
That certainly will be what's up at the third node.
y2 coming in, y3 coming in, y5 going out has to balance.
And finally, y4 plus y5 equals zero says that at this node, y4 plus y5, the total flow, We don't -- you know, charge doesn't accumulate at is zero.
the nodes.
It travels around.
OK. Now give me -- I come back now to the linear algebra question.
What's a vector y that solves these equations?
Can I figure out what the null space is for this matrix, A transpose, by looking at the graph?
I'm happy if I don't have to do elimination.
I can do elimination, we know how to do, we know how to find the null space basis.
We can do elimination on this matrix, and we'll get it into a good reduced row echelon form, and the special solutions will pop right out.
But I would like to -- even to do it without that.
Let me just ask you first, if I did elimination on that, on that, matrix, what would the last row become?
What would the last row -- if I do elimination on that matrix, the last row of R will be all zeros, right?
Why?
Because the rank is three.
We only going to have three pivots.
And the fourth row will be all zeros when we eliminate.
So elimination will tell us what, what we spotted earlier, what's the null space -- all the, all the information, what are the dependencies.
We'll find those by elimination, but here in a real example, we can find them by thinking.
OK. Again, my question is, what is a solution y?
How could current travel around this network without collecting any charge at the nodes?
Tell me a y. OK.
So a basis for the null space of A transpose.
How many vectors I looking for?
Two.
It's a two dimensional space.
My basis should have two vectors in it.
Give me one.
One set of currents.
Suppose, let me start it.
Let me start with y1 as one.
OK.
So one unit of -- one amp travels on edge one with the arrow.
OK, then what?
What is y2?
It's one also, right?
And of course what you did was solve Kirchoff's Current Law quickly in the second equation.
OK. Now we've got one amp leaving node one, coming around to node three.
What shall we do now?
Well, what shall I take for y3 in other words?
Oh, I've got a choice, but why not make it what you said, negative one.
So I have just sent current, one amp, around that loop.
What shall y4 and y5 be in this case?
We could take them to be zero.
This satisfies Kirchoff's Current Law.
We could check it patiently, that minus y1 minus y3 gives zero.
We know y1 is y2.
The others, y4 plus y5 is certainly zero.
Any current around a loop satisfies -- satisfies the Current Law.
Now you know how to get another one.
OK. Take current around this loop.
So now let y3 be one, y5 be one, and y4 be minus one.
And so, so we have the first basis vector sent current around that loop, the second basis vector sends current around that loop.
And I've -- and those are independent, and I've got two solutions -- two vectors in the null space of A transpose, two solutions to Kirchoff's Current Law.
Of course you would say what about sending current around the big loop.
What about that vector?
One for y1, one for y2, nothing f- on y3, one for y5, and minus one for y4.
What about that?
Is that, is that in the null space of A transpose?
Sure.
So why don't we now have a third vector in the basis?
Because it's not independent, right?
It's not independent.
This vector is the sum of those two.
If I send current around that and around that -- then on this edge y3 it's going to cancel out and I'll have altogether current around the whole, the outside loop.
That's what this one is, but it's a combination of those two.
Do you see that I've now, I've identified the null space of A transpose -- but more than that, we've solved Kirchoff's Current Law.
And understood it in terms of the network.
OK.
So that's the null space of A transpose.
I guess I -- there's always one more space to ask you about.
Let's see, I guess I need the row space of A, the column space of A transpose.
So what's N, what's its dimension?
Yup?
What's the dimension of the row space of A?
If I look at the original A, it had five rows.
How many were independent?
Oh, I guess I'm asking you the rank again, right?
And the answer is three, right?
Three independent rows.
When I transpose it, there's three independent columns.
Are those columns independent, those three?
The first three columns, are they the pivot columns of the matrix?
No.
Those three columns are not independent.
There's a in fact, this tells me a relation between them.
There's a vector in the null space that says the first column plus the second column equals the third column.
They're not independent because they come from a loop.
So the pivot columns, the pivot columns of this matrix will be the first, the second, not the third, but the fourth.
One, columns one, two, and four are OK. Where are they -- those are the columns of A transpose, those correspond to edges.
So there's edge one, there's edge two, and there's edge four.
So there's a -- that's like -- is a, smaller graph.
If I just look at the part of the graph that I've, that I've, thick -- used with thick edges, it has the same four nodes.
It only has three edges.
And the, those edges correspond to the independent guys.
And in the graph there -- those three edges have no loop, right?
The independent ones are the ones that don't have a loop.
All the -- dependencies came from loops.
They were the things in the null space of A transpose.
If I take three pivot columns, there are no dependencies among them, and they form a graph without a loop, and I just want to ask you what's the name for a graph without a loop?
So a graph without a loop is -- has got not very many edges, right?
I've got four nodes and it only has three edges, and if I put another edge in, I would have a loop.
So it's this graph with no loops, and it's the one where the rows of A are independent.
And what's a graph called that has no loops?
It's called a tree.
So a tree is the name for a graph with no loops.
And just to take one last step here.
Using our formula for dimension.
Using our formula for dimension, let's look -- once at this formula.
The dimension of the null space of A transpose is m-r. OK.
This is the number of loops, number of independent loops.
m is the number of edges.
And what is r?
What is r for our -- we'll have to remember way back.
The rank came -- from looking at the columns of our matrix.
So what's the rank?
Let's just remember.
Rank was -- you remember there was one -- we had a one dimensional -- rank was n minus one, that's what I'm struggling to say.
Because there were n columns coming from the n nodes, so it's minus, the number of nodes minus one, because of that C, that one one one one vector in the null space.
The columns were not independent.
There was one dependency, so we needed n minus one.
This is a great formula.
This is like the first shall I, -- write it slightly differently?
The number of edges -- let me put things -- have I got it right?
Number of edges is m, the number -- r- is m-r, OK.
So, so I'm getting -- let me put the number of nodes on the other side.
So I -- the number of nodes -- I'll move that to the other side -- minus the number of edges plus the number of loops is -- I have minus, minus one is one.
The number of nodes minus the number of edges plus the number of loops is one.
These are like zero dimensional guys.
They're the points on the graph.
The edges are like one dimensional things, they're, they connect nodes.
The loops are like two dimensional things.
They have, like, an area.
And this count works for every graph.
And it's known as Euler's Formula.
We see Euler again, that guy never stopped.
OK. And can we just check -- so what I saying?
I'm saying that linear algebra proves Euler's Formula.
Euler's Formula is this great topology fact about any graph.
I'll draw, let me draw another graph, let me draw a graph with more edges and loops.
Let me put in lots of -- OK.
I just drew a graph there.
So what are the, what are the quantities in that formula?
How many nodes have I got?
Looks like five.
How many edges have I got?
One two three four five six seven.
How many loops have I got?
One two three.
And Euler's right, I always get one.
That, this formula, is extremely useful in understanding the relation of these quantities -- the number of nodes, the number of edges, and the number of loops.
OK. Just complete this lecture by completing this picture, this cycle.
So let me come to the -- so this expresses the equations of applied math.
This, let me call these potential differences, say, E. So E is A x.
That's the equation for this step.
The currents come from the potential differences.
y is C E. The potential -- the currents satisfy Kirchoff's Current Law.
Those are the equations of -- with no source terms.
Those are the equations of electrical circuits of many -- those are like the, the most basic three equations.
Applied math comes in this structure.
The only thing I haven't got yet in the picture is an outside source to make something happen.
I could add a current source here, I could, I could add external currents going in and out of nodes.
I could add batteries in the edges.
Those are two ways.
If I add batteries in the edges, they, they come into here.
Let me add current sources.
If I add current sources, those come in here.
So there's a, there's where current sources go, because the F is a like a current coming from outside.
So we have our edges, we have our graph, and then I send one amp into this node and out of this node -- and that gives me, a right-hand side in Kirchoff's Current Law.
And can I -- to complete the lecture, I'm just going to put these three equations together.
So I start with x, my unknown.
I multiply by A.
That gives me the potential differences.
That was our matrix A that the whole thing started with.
I multiply by C. Those are the physical constants in Ohm's Law.
Now I have y. I multiply y by A transpose, and now I have F. So there is the whole thing.
There's the basic equation of applied math.
Coming from these three steps, in which the last step is this balance equation.
There's always a balance equation to look for.
These are the -- when I say the most basic equations of applied mathematics -- I should say, in equilibrium.
Time isn't in this problem.
I'm not -- and Newton's Law isn't acting here.
I'm, I'm looking at the -- equations when everything has settled down, how do the currents distribute in the network.
And of course there are big codes to solve the -- this is the basic problem of numerical linear algebra for systems of equations, because that's how they come.
And my final question.
What can you tell me about this matrix A transpose C A?
Or even A transpose A?
I'll just close with that question.
What do you know about the matrix A transpose A?
It is always symmetric, right.
OK, thank.
So I'll see you Wednesday for a full review of these chapters, and Friday you get to tell me.
Thanks.
OK.
Uh this is the review lecture for the first part of the course, the Ax=b part of the course.
And the exam will emphasize chapter three.
Because those are the --0 chapter three was about the rectangular matrices where we had null spaces and null spaces of A transpose, and ranks, and all the things that are so clear when the matrix is square and invertible, they became things to think about for rectangular matrices.
So, and vector spaces and subspaces and above all those four subspaces.
OK, I'm thinking to start at least I'll just look at old exams, read out questions, write on the board what I need to and we can see what the answers are.
The first one I see is one I can just read out.
Well, I'll write a little.
Suppose u, v and w are nonzero vectors in R^7.
What are the possible -- they span a -- a vector space.
They span a subspace of R^7, and what are the possible dimensions?
So that's a straightforward question, what are the possible dimensions of the subspace spanned by u, v and w?
OK, one, two, or three, right.
One, two or three.
Couldn't be more because we've only got three vectors, and couldn't be zero because -- because I told you the vectors were nonzero.
Otherwise if I allowed the possibility that those were all the zero vector -- then the zero-dimensional subspace would have been in there.
OK. Now can I jump to a more serious question?
OK. We have a five by three matrix.
And I'm calling it U. I'm saying it's in echelon form.
And it has three pivots, r=3.
Three pivots.
OK. First question, what's the null space?
What's the null space of this matrix U, so this matrix is five by three, and I find it helpful to just see visually what five something?
How to be a millionaire?
by three means, what that shape is.
It was some crazy guy, what was his name?
so that --1 does the matrix have three Three columns.
Three columns in U then, five rows, three pivots, It's -- Regis, right.
and what's the null space?
Regis.
OK.2 If you saw this, like when you should have been The null space of U is -- and it asks for a spec-of course I'm looking for an answer that isn't just the definition of the null space, but is the null space of this matrix, with this information.
And what is it?
It's only the zero vector.
Because we're told that the rank is three, so those three columns must be independent, no combination -- of those columns is the zero vector except -- so the only thing in this null space is the zero vector, and I -- I could even say what that vector is, zero, columns also?
zero, zero.
That's OK.
So that's what's in the null space.
All right?
let me go on with -- this question has multiple parts.
What's the -- oh now it asks you about a ten by three matrix, B, which is the matrix U and two U.
It actually -- I would probably be writing R -- and maybe I should be writing R here now.
This exam goes back a few years when I emphasized U more than R. Now, what's the echelon form for that matrix?
So the echelon form, what's the rank, Yes.
and what's the echelon form?
Let's suppose this is in reduced echelon form, so that I could be using the letter R. So I'll ask for the reduced row echelon form so imagine that So I'm seeing the same length in b, three, and also in x. these are -- U is in reduced row echelon form but now I've doubled the height of the matrix, So the (x)-s that it multiplies have three components, what will happen when we do row reduction?
What row reduction will take us to what matrix here?
So you start doing elimination.
You're doing elimination on single rows.
But of course we're allowed to think of blocks.
So what, well, what's the answer look like?
U and z- or R -- let's -- I'll stay with this letter U but I'm what was the name of that --2 winning a million dollars or really thinking it's in reduced form, and zero.
OK. Fine.
Then it asks oh, further, it asks about this matrix.
U, U, U, and zero.
OK, what's the echelon form of this?
So it's just like practice in thinking through what would row elimination, what would row reduction do.
Have I thought this through, so what -- what are we -- if we start doing elimination, basically we're going to subtract these rows from these -- In the column space, because I do know that it can so it's going to take us to U, U, zero, and minus U, For example, I'm going to ask you for a I guess, right?
But is it a- is it three by three?
Take the thing all the way to R -- let's suppose U is really R. Suppose that we're really going for the reduced row echelon doing linear algebra of course -- but I didn't have to do the form.
Then would we stop there?
No.
We would clean out, we would -- we could use this to -- is that right, can I so I took this row -- these rows away from these to get there.
Now I take these rows away from these, so that gives me zero.
There.
there?1 And now what more would I do if I'm really shooting for R, the reduced row echelon form?
I would -- then I want plus ones in the pivot, so I would multiply through by minus one to get plus there.
So essentially I'm seeing reduced row echelon form there and there, and there's just one little twist still to go.
Do you see what that final twist might be?
It certainly has three rows.
To have if -- if U is in reduced row echelon form and now I'm looking at U, U, there's one little step to take, this isn't like a big deal at all, but -- but if I really want this to be in reduced form, what would I still -- might I still have to do?
I might have some zero rows here, I might have some zero rows here that strictly should move to the bottom.
Well, I'm not going to make a project out of that.
first of all?
OK. What's the rank of that matrix?
What's the rank of this matrix C?
Given that I know that the original U has rank three, what's the rank of this guy?
Six, right.
That has rank six, I can tell.
What was -- what's the rank of this B, while -- while we're at on TV for a few weeks, did you see that, it?
The rank of B, is that six or three?
So A is a three by three matrix.
Three is right.
Three is right.
Because we actually got it to where we could just see three pivots.
OK, and oh, now finally this easy one, what's the dimension be solved exactly when B is in the column space, of the null space -- of the null space of C transpose?
Oh, boy.
OK, so what do I -- if I want the dimension of a null space, I want to know the size of the matrix -- so what's the size of the matrix C?
It looks like it's ten by six, is it?
Ten by six, so C is ten by six, so m is ten, so C has ten rows, C transpose has ten columns, so there are ten columns there.
So how many free variables have I got, once I -- There's quite a few trues, shall I take a poll, if I start with the ten columns in C transpose, that's the m for the original C. And what do I subtract off?
Six.
Because we said that was the rank.
So I'm left with four.
Thanks.
OK.
If b is in the -- and what would -- what does the exam say So I think that's the right answer -- the dimension of the null space of C transpose would be four.
Right.
OK. Yeah.
OK.
So that's one question, at least it brought in some -- some of the dimension counts.
OK.
Here's another type of question.
think about, what's the -- what's the shape of the matrix, I give you an equation, Ax equals two four two.
And I give you the complete solution.
And what's its rank?
But I don't give you the matrix.
And another -- there's another vector, zero, zero, one.
OK. All right.
My first question is what's the dimension of the row space?
Of the matrix A?
So the main thing that you want to get from this question is that a question could start this way.
exam, don't just tell me.
Sort of backward way.
so I guess I'm asking you what is the column space for this basis for the null space.
By giving you the answer and not telling you what the problem homework, so I was watching it, so there were three -- the is.
But we can get a lot of information, and sometimes we can get complete information about that matrix A. OK.
So what's the dimension of the row space of A?
What's the rank?
Tell me about what's the size of the matrix, yeah, just -- These are the things we want to matrix b and I'll answer them without multiplying it out.2 Its rank -- tell me something about its null space, I heard the right answer for the rank, the rank is one in this case.
Why?
Because the dimension of the null space, so the dimension of this instant?3 the null space of A is from knowing that that's the complete solution, it's two.
I'm seeing two vectors here, and they're independent in the null space of A, because they have to be in the OK, which -- did you watch that quiz, there was a quiz program null space of A if I'm allowed to throw into the solution any amount of those vectors, that tells me that's the null space part then.
So the dimension of the null space is two, and then I -- of course I know the dimensions of all the -- four subspaces.
Now actually it asks what's the matrix?
Well, what's the matrix in this case?
Do we want to -- shall I try to figure that out?
Sure.
Let's -- you'd like me to do it, OK. Well, what's the ---- I actually say in this in the So what about the matrix, or let me at least start it, OK.
If A times this x gives two, four, two, what does that tell me about the matrix A?
If A times that x solves that equation then it tells me that the first column is -- the first column of A is -- one, two, one, right.
The first column of A has to be one, two, one, because if I multiply by x, that's going to multiply just matrix?
the first column, and give me two, four, two.
And then I've got two more columns to find, and what information have I got to find them with?
A basis for the null space.
I've got the null space.
So the fact that this is in the null space, what does that tell So what is if b has the form -- so I guess I'm asking what's me about the matrix?
A matrix that has zero, zero, one in its null space?
That tells me that the last column of the matrix is zeroes.
so how many think true?
Because this is in the null space, the last column has to be When could it be solved?
zeroes.
And because this is in the null space, what's the second column?
Well, this in the null space means that if I multiply A by that vector I must be getting zeroes, so I think that better be minus one, minus two, and minus one.
OK. That's a type of question that just brings out the information that's in that complete solution.
And then actually I go on to ask what vectors -- for what OK. vectors B can Ax=b be solved?
Ax=b can be solved if what -- so I'm looking for a condition Definitely not.
on b, if any.
interesting -- the novel point was there were three ways that Can it be solved for every right-hand side b?
No.
And the answer is -- yes or no -- right.
solvable?
the column space of this matrix, and what is it?
It's so the column space of that matrix is all multiples b Can you tell me something -- so I'll ask questions about this -- b is a multiple of one, two, one.
Right?
I can solve the thing if it's a multiple of one, two, one, and of course sure enough -- yeah, that was a multiple of one, two, one, and so I had a solution.
So this is a case where we've got lots of null space.
Let me just recall rank is big, don't forget those cases, don't forget the other cases when r is as big as it can be, OK. r equal m or r equal n. Those are -- we had a full lecture on that, the full rank, full lecture, and important -- important case.
I gave you every chance to think about that.
OK.
I'll just move on.
I think this is the best type of review.
So I'm going to solve Bx equals zero.
It's just brings these ideas out.
Apologies to the camera while I recover glasses and exam.
OK. How about a few true-false ones?
Actually there won't be a true-false on the quiz.
But it gives us a moment of quick review.
True or false, how do you feel about it at Here's one.
If the null space -- I have a square matrix.
If its null space is just the zero vector, what about the null space of A transpose?
If the null space of A is just the zero vector, and the matrix is square, what do I know about the null space of A transpose?
Also the zero vector.
Good.
And that's a very very important fact.
OK. How about this?
These -- look at the space of five by five matrices as a vector space.
So it's actually a twenty-five-dimensional vector space.
All five by five matrices.
Look at the invertible matrices.
Do they form a subspace?
So I have this five by -- a space of all five by five matrices.
I can add them, I can multiply by numbers.
But now I narrow down to the invertible ones.
And I ask are they a subspace?
And you -- your answer is -- quiet, but nevertheless definite, no.
Right?
Because if I add two invertible matrices I have no idea if the No.
answer is invertible.
If I multiply that invertible -- well, it doesn't even have the zero matrix in it, it couldn't be a subspace.
I have to be able to multiply by zero -- and stay in my subspace, and the invertible ones wouldn't work.
Well, the singular ones wouldn't work either.
They have zero -- the zero matrix is in the singular matrices, but if I add two singular matrices I don't know if the answer is singular or not.
OK.
So another true-false.
If b squared equals zero then b equals zero.
columns then the question is does Ax=b, is it always True or false?
If b squared equals zero, true, false?
you could get help, right -- b squared equals zero, b has to be a square -- square matrix, so that I can multiply it by itself, does that imply that B is zero?
Are there matrices whose square could be the zero matrix?
Yes or no?
Yes there are.
There are matrices whose square is the zero matrix.
So this statement is false.
If b squared is zero, we don't know that b is zero.
For example -- the best example is that matrix.
That matrix is a dangerous matrix.
It will come up in later parts of this course as an example of what can go wrong.
And here is a real simple -- so this -- so if I square that without doing the multiplication and finding the matrix b. matrix, I do get the zero matrix,and it shows -- OK. A system of m equations in m unknowns is solvable for every right-hand side if the columns are independent.
So can I say that again, I'm -- I'll write it down then OK. for short.
m by m matrix independent So give me a basis for -- for the null space of B.
Let's see.
but you could only use each way once, so you couldn't like use them all the time.
So remember that?
You could -- so you could poll the audience, and that was a very -- that was a hundred percent successful way, so I'll poll the audience on this.
If the other possibility -- another possibility you could call your friend, right, or he's your friend until he gives you the wrong answer, which -- that turned out subspaces, then A is some multiple of B. to be very unreliable, you know, you'd call up your brother or something and ask him for the capital of whatever, Bosnia.
What does he know, he makes some guess, So what -- I would just take extreme cases if it was me, matrix, independent columns, is Ax=b always solvable?
Maybe just hands up for that?
A few.
And who says no?
Oh, gosh, this audience is not reliable.
Fifty fifty.
I guess I'd say, I'd vote yes.
Because independent columns, that means that the rank is the full size m, I have a matrix of rank m. That means it's -- I mean it's square, so it's an invertible matrix, and nothing could go wrong.
Yes.
So that's the good case and we always expect it in chapter two, but of course chapter three is -- only one of the possibilities.
OK. Let me move on to another question from an old quiz.
OK. OK. Let's see.
I'm going to give you a matrix, but I'm going to give it to you OK. as a product of a couple of matrices, one, one, zero, zero, one, zero, one, zero, one, times another matrix, one, zero, I would like to ask you questions about that matrix minus one, two, zero, one, one, minus one, and all zeroes.
OK. Let's see, what dimension I in -- the null space of B is a subspace of R. What size vectors I looking for here?
Because if we don't know the size, we aren't going to find it, right?
the null -- this matrix is three by four obviously.
So if we're looking for the null space we're looking for those vectors x in R^4.
OK.
So the null space of B is certainly a subspace of R^4.
What do you think its dimension is?
Of course once we find the basis we would know the dimension immediately, but let's stop first, what's the rank of this matrix B?
Let's see, what -- is that matrix invertible, that square one there?
Let's say sure, I think it is, yes, that matrix B looks invertible.
Is that pretty clear?
Yeah.
Yeah.
So I've gone wrong in this course already, but I'll still hope that that matrix is invertible.
Yeah, yeah, because if I look for a combination of those three columns -- well, I couldn't use this middle column because it would have a one and in a position that I -- column is otherwise all zero, so a combination that gives zero can't give us that problem, and then the other two are clearly independent sets -- so that matrix is invertible.
Later we could take a determinant or other things.
OK. What's the setup?
If I have an invertible matrix, a nice invertible square matrix, times this guy, times this second factor, and I'm looking for the null space, does this have any effect?
Is the null -- so what I'm asking is is the null space of B the same as the null space of just this part?
I think so.
I think so.
If Bx is zero, then multiplying by that guy I'll still have zero.
But also if this times some x give zero, I could always multiply on the left by the inverse of that, because it is invertible, and I would discover that this kind of Bx is zero.
You want me to write some of that down -- if I have a product here, C times -- times D, say, and if C is invertible, the null space of CD, well, it will the same as the null space of D. If C is invertible.
Multiplying by an invertible matrix on the left can't change the null space.
OK.
So basically I'm asking you for the null space of this.
so when do I know -- well, I would say suppose the matrix I don't have to do the multiplication because I have C is invertible.
That first factor C is invertible.
It's not going to change the null space.
OK.
So can we just write down a basis now for the null space?
So what's the basis for the null space of -- of that?
So basis for the null space I'm looking for the two -- there are two pivot columns obviously.
It clearly has rank two.
If -- so true or false, if A and B have the same four I'm looking for the two special solutions.
They'll come from the third and the fourth.
The free variables.
OK. so if the third free variable is a one, then I think probably I need a minus one there and a one there, it looks like.
Do you agree that if I then do that multiplication I'll get zero?
And if I have one in the fourth variable, then maybe I need a one in the second variable and maybe a minus two in the third.
So I just reasoned that through and then if I look back I see sure enough that the free variable part that I sometimes call F, that up -- that two by two corner, is sitting here with all its signs reversed.
So that's -- here I'm seeing minus F, and here I'm seeing the identity in the null space matrix.
OK, so that's the null space.
Another question is solve Bx equal one, zero, one.
OK.
So that's one question, now solve complete solutions.
To Bx equal one, zero, one.
OK. Yeah, so I guess I'm seeing if I wanted to get one, zero, one - What's our particular solution?
So I'm looking for a particular solution and then the null space part.
OK.
I-- actually the first column of B, so what's the first column of our matrix B?
It's the vector one, zero, one.
The first column of our matrix agrees with the right-hand side.
So I guess I'm thinking x particular plus x null space will be the particular solution, since the first column of B is exactly right, that's great.
And then I have C times that first null space vector and D times the other null space vector.
Right?
The two -- the null space part of the solution, as always has the arbitrary constants, the particular solution doesn't have any arbitrary constants, it's one particular solution, and in this case it'll -- that one would do.
OK. Fine.
so those are questions taken from old quizzes, any questions coming to mind?
Yeah
value.
OK. Well, so that particular x particular, it says that let's see, when I multiply by this guy, I'm going to get the first column of B.
That -- if that's a solution, I multiply B, B times this x will be the first column of B, and so I'm saying that the first column of this B agrees with the right-hand side.
So I'm saying that look at the first column of that matrix B.
If you do the multiplication, it's -- so what's the first column of that matrix?
Is that how you do that multiplication?
I multiply that matrix by that first column.
And it picks out one, zero, one.
So the first column of B is exactly that.
And therefore a particular solution will be this guy.
So I'll repeat that question.
Yeah.
OK.
Yes
particular solution.
Any of the solutions can be the particular one that we pick out.
So like this plus -- plus this would be another particular OK. solution.
It would be another solution.
The particular is just telling us only take one.
But it's not telling us which one we have to take.
We take the most convenient one.
I guess in this -- in this problem that was that one.
Good.
Other questions?
Yes.
And this pattern of particular plus null space, of course, that's going throughout mathematics of linear systems.
We're really doing mathematics of linear systems here.
Our systems are discrete and they're finite-dimensional -- and so it's linear algebra, but this particular plus null space goes -- that doesn't depend on having finite matrices -- that spreads much -- that spreads everywhere.
OK, I'm going to just like to encourage you to take problems out of the book, let me do the same myself.
OK well here's some easy true or falses.
I don't know why the author put these in here.
OK.
If m=n, then the row space equals the column space.
is invertible -- suppose A is an invertible So these are true or falses.
If m equals n, so that means the matrix is square, then the row space equals the column space?
False, good.
Good, what is equal there?
What can I say is equal, if M -- well, yeah.
Yeah it -- so that's definitely false -- the row space and the column space, and this matrix is like always a good example to consider.
So there's a square matrix but it's row space is the multiples of zero, one, and its column space is the multiples of one, zero.
Very different.
The row space and the column space are totally different for that matrix.
Now of course if the matrix was symmetric, well, then clearly the row space equals the column space.
OK. How about this question?
The matrices A and minus A share the same four subspaces?
Do the matrices A and minus A have the same column space, do they have the same null space, do they have the same row space?
What's the answer on that?
Yes or no.
Yes.
Good.
OK. How about this?
If A and B have the same four subspaces, then A is a multiple of B.
If -- suppose those subspaces are the same.
Then is A a multiple of B?
OK. How how do you answer a question like that?
Of course if you want to answer it yes, then I would -- then they'd have to think of a reason why.
If you want to answer no way, then you would -- and I would sort of like first I would try to think no, I would say can I come up with an example where it isn't true?
Let me repeat the question.
And then write the answer.
matrix, then what -- suppose it's six by six invertible matrix, then what's its row space, and its column space is all of R^6, and the null space, and the null space of A transpose would be the zero vector.
So every invertible matrix is going to give that answer.
If I have a six by six invertible matrix, I know what those subspaces are.
Heck, that was back in chapter two, when I didn't even know what subspaces were.
The row space and column space are both all six-dimensional space -- the whole space, and the rank is six, in other words, and the null spaces have zero dimension.
So do you see now the answer?
So A and B could be.
So A and B could be for example any -- so I'm going to say false.
Because A and B for example -- So an example: A and B any invertible six by six, six by six.
So those would have the same four subspaces but they wouldn't be the same.
Of course th- there should be something about those matrices that would be the same.
It's sort of a natural problem, so now actually we're getting to a math question.
The answer is this is not true.
One matrix doesn't have to be a multiple of the other.
But there must be something that's true.
And that would be sort of like a natural question to ask.
If they have the same subspaces, same four subspaces, then what -- what could you -- instinct wasn't necessarily right.
But I hope you now see that the correct answer is false.
And then you might think OK, well, they certainly do have the same rank.
But do -- obviously if they have the same four subspaces, they have the same rank.
I might say if they have the well, I could extend that question and think about other possibilities and finally come up with something that was true.
But I won't press that one.
Let me keep going with practice questions.
And these practice questions are quite appropriate I think for the exam.
OK. let's see.
If I exchange two rows of A which subspaces stay the same?
So I'm trying to take out questions that we can answer without you know we can answer quickly.
If I have a matrix A, and I exchange two of its rows, which subspaces stay the same?
The row space does stay the same.
And the null space stays the same.
Good.
Good.
Correct.
Column space would be a wrong answer.
OK. all right, here's a question.
Oh, this leads into the next chapter.
Why can the vector one, two, three not be a row and also in the null space?
Fitting we close with this question.
Close is -- so V equal this one, two, three can't be in the null space of a matrix and the row space.
And my question is why not?
Why not?
So this is a question that we can because it's sort of asked in a straightforward way, we can figure out an answer.
Well, actually yeah -- I'll even pin it down, it can't be in the null space -- and be a row.
I'll even pin it down further.
Ask it to be a row of A.
Why not?
So I'm -- we know the dimensions of these spaces.
But now I'm asking you sort of like the overlap between -- so the null space and the row space, those are in the same n-dimensional space.
Those are -- well, those are both subspaces of n-dimensional space, and I'm basically saying they can't overlap.
I can't have a vector like this, a typical vector, that's in the null space and it's also a row of the matrix.
Why is that?
So that's a new sort of idea.
Let's just see what it would mean.
I mean that A times this V, why can this A times this V it can't be zero.
Oh well, if it's zero, so this is -- I'm getting it into the null space here.
So this is -- now let's put that vector's in the null space, why can't the first row of a matrix be one, two, three?
I can fill out the matrix as I like.
Why is that impossible?
Well, you're seeing it's impossible, right?
That if that was a row of the matrix and in the null space, that number would not be zero, it would be fourteen.
Right.
So now we actually are beginning to get a more complete picture of these four subspaces.
The two that are over in n-dimensional space, they actually only share the zero vector.
The intersection of the null space and the row space is only the zero vector.
And in fact the null space is perpendicular to the row space.
That'll be the first topic let's see, we have a holiday Monday -- and I'll see you Wednesday with perpendiculars.
And I'll see you Friday.
So good luck on the quiz.
OK. cameras are rolling.
This is lecture fourteen, starting a new chapter.
Chapter about orthogonality.
What it means for vectors to be orthogonal.
What it means for subspaces to be orthogonal.
What it means for bases to be orthogonal.
So ninety degrees, this is a ninety-degree chapter.
So what does it mean -- let me jump to subspaces.
Because I've drawn here the big picture.
This is the 18.06 picture here.
And hold it down, guys.
So this is the picture and we know a lot about that picture already.
We know the dimension of every subspace.
We know that these dimensions are r and n-r. We know that these dimensions are r and m-r. What I want to show now is what this figure is saying, that the angle -- the figure is just my attempt to draw what I'm now going to say, that the angle between these subspaces is ninety degrees.
And the angle between these subspaces is ninety degrees.
Now I have to say what does that mean?
What does it mean for subspaces to be orthogonal?
But I hope you appreciate the beauty of this picture, that that those subspaces are going to come out to be orthogonal.
Those two and also those two.
So that's like one point, one important point to step forward in understanding those subspaces.
We knew what each subspace was like, we could compute bases for them.
Now we know more.
Or we will in a few minutes.
OK.
I have to say first of all what does it mean for two vectors to be orthogonal?
So let me start with that.
Orthogonal vectors.
The word orthogonal is -- is just another word for perpendicular.
It means that in n-dimensional space the angle between those vectors is ninety degrees.
It means that they form a right triangle.
It even means that the going way back to the Greeks that this angle that this triangle a vector x, a vector x, and a vector x+y -- of course that'll be the hypotenuse, so what was it the Greeks figured out and it's neat.
It's the fact that the -- so these are orthogonal, this is a right angle, if -- so let me put the great name down, Pythagoras, I'm looking for -- what I looking for?
I'm looking for the condition if you give me two vectors, how do I know if they're orthogonal?
How can I tell two perpendicular vectors?
And actually you probably know the answer.
Let me write the answer down.
Orthogonal vectors, what's the test for orthogonality?
I take the dot product which I tend to write as x transpose y, because that's a row times a column, and that matrix multiplication just gives me the right thing, that x1y1+x2y2 and so on, so these vectors are orthogonal if this result x transpose y is zero.
That's the test.
OK. Can I connect that to other things?
I mean -- it's just beautiful that here we have we're in n dimensions, we've got a couple of vectors, we want to know the angle between them, and the right thing to look at is the simplest thing that you could imagine, the dot product.
OK. Now why?
So I'm answering the question now why -- let's add some justification to this fact, that's the test.
OK, so Pythagoras would say we've got a right triangle, if that length squared plus that length squared equals that length squared.
OK, can I write it as x squared plus y squared equals x plus y squared?
Now don't, please don't think that this is always true.
This is only going to be true in this -- it's going to be equivalent to orthogonality.
For other triangles of course, it's not true.
For other triangles it's not.
But for a right triangle somehow that fact should connect to that fact.
Can we just make that connection?
What's the connection between this test for orthogonality and this statement of orthogonality?
Well, I guess I have to say what is the length squared?
So let's continue on the board underneath with that equation.
Give me another way to express the length squared of a vector.
And let me just give you a vector.
The vector one, two, three.
That's in three dimensions.
What is the length squared of the vector x equal one, two, three?
So how do you find the length squared?
Well, really you just, you want the length of that vector that goes along one -- up two, and out three -- and we'll come back to that right triangle stuff.
The length squared is exactly x transpose x.
Whenever I see x transpose x, I know I've got a number that's positive.
It's a length squared unless it -- unless x happens to be the zero vector, that's the one case where the length is zero.
So right -- this is just x1 squared plus x2 squared plus so on, plus xn squared.
So one -- in the example I gave you what was the length squared of that vector one, two, three?
So you square those, you get one, four and nine, you add, you get fourteen.
So the vector one, two, three has length fourteen.
So let me just put down a vector here.
Let x be the vector one, two, three.
Let me cook up a vector that's orthogonal to it.
So what's the vector that's orthogonal?
So right down here, x squared is one plus four plus nine, fourteen, let me cook up a vector that's orthogonal to it, we'll get right that that -- those two vectors are orthogonal, the length of y squared is five, and x plus y is one and two making three, two and minus one making one, three and zero making three, and the length of this squared is nine plus one plus nine, nineteen.
And sure enough, I haven't proved anything.
I've just like checked to see that my test x transpose y equals zero, which is true, right?
Everybody sees that x transpose y is zero here?
That's maybe the main point.
That you should get really quick at doing x transpose y, so it's just this plus this plus this and that's zero.
And sure enough, that clicks with fourteen plus five agreeing with nineteen.
Now let me just do that in letters.
So that's y transpose y.
And this is x plus y transpose x plus y. OK.
So I'm looking, again, this isn't always true.
I repeat.
This is going to be true when we have a right angle.
And let's just -- well, of course, I'm just going to simplify this stuff here.
There's an x transpose x there.
And there's a y transpose y there.
And there's an x transpose y.
And there's a y transpose x. I knew I could do that simplification because I'm just doing matrix multiplication and I've just followed the rules.
OK.
So x transpose x-s cancel.
Y transpose y-s cancel.
And what about these guys?
What can you tell me about the inner product of x with y and the inner product of y with x?
Is there a difference?
I think if we -- while we're doing real vectors, which is all we're doing now, there isn't a difference, there's no difference.
If I take x transpose y, that'll give me zero, if I took y transpose x I would have the same x1y1 and x2y2 and x3y3, it would be the same, so this is -- this is the same as that, this is really I'll knock that guy out and say two of these.
So actually that's the -- this equation boiled down to this thing being zero.
Right?
Everything else canceled and this equation boiled down to that one.
So that's really all I wanted.
I just wanted to check that Pythagoras for a right triangle led me to this -- of course I cancel the two now.
No problem.
To x transpose y equals zero as the test.
Fair enough.
OK. You knew it was coming.
The dot product of orthogonal vectors is zero.
It's just -- I just want to say that's really neat.
That it comes out so well.
All right.
Now what about -- so now I know if two -- when two vectors are orthogonal.
By the way, what about if one of these guys is the zero vector?
Suppose x is the zero vector, and y is whatever.
Are they orthogonal?
Sure.
In math the one thing about math is you're supposed to follow the rules.
So you're supposed to -- if x is the zero vector, you're supposed to take the zero vector dotted with y and of course you always get zero.
So just so we're all sure, the zero vector is orthogonal to everybody.
But what I want to -- what I now want to think about is subspaces.
What does it mean for me to say that some subspace is orthogonal to some other subspace?
So OK. Now I've got to write this down.
So because we're defining definition of subspace S is orthogonal so to subspace let's say T, so I've got a couple of subspaces.
And what should it mean for those guys to be orthogonal?
It's just sort of what's the natural extension from orthogonal vectors to orthogonal subspaces?
Well, and in particular, let's think of some orthogonal subspaces, like this wall.
Let's say in three dimensions.
So the blackboard extended to infinity, right, is a -- is a subspace, a plane, a two-dimensional subspace.
It's a little bumpy but anyway, it's a -- think of it as a subspace, let me take the floor as another subspace.
Again, it's not a great subspace, MIT only built it like so-so, but I'll put the origin right here.
So the origin of the world is right there.
OK.
Thereby giving linear algebra its proper importance in this.
OK.
So there is one subspace, there's another one.
The floor.
And are they orthogonal?
What does it mean for two subspaces to be orthogonal and in that special case are they orthogonal?
All right.
Let's finish this sentence.
What does it mean means we have to know what we're talking about here.
So what would be a reasonable idea of orthogonal?
Well, let me put the right thing up.
It means that every vector in S, every vector in S, is orthogonal to -- what I going to say?
Every vector in T. That's a reasonable and it's a good and it's the right definition for two subspaces to be orthogonal.
But I just want you to see, hey, what does that mean?
So answer the question about the -- the blackboard and the floor.
Are those two subspaces, they're two-dimensional, right, and we're in R^3.
It's like a xz plane or something and a xy plane.
Are they orthogonal?
Is every vector in the blackboard orthogonal to every vector in the floor, starting from the origin right there?
Yes or no?
I could take a vote.
Well we get some yeses and some noes.
No is the answer.
They're not.
You can tell me a vector in the blackboard and a vector in the floor that are not orthogonal.
Well you can tell me quite a few, I guess.
Maybe like I could take some forty-five-degree guy in the blackboard, and something in the floor, they're not at ninety degrees, right?
In fact, even more, you could tell me a vector that's in both the blackboard plane and the floor plane, so it's certainly not orthogonal to itself.
So for sure, those two planes aren't orthogonal.
What would that be?
So what's a vector that's -- in both of those planes?
It's this guy running along the crack there, in the intersection, the intersection.
A vector, you know -- if two subspaces meet at some vector, well then for sure they're not orthogonal, because that vector is in one and it's in the other, and it's not orthogonal to itself unless it's zero.
So the only I mean so orthogonal is for me to say these two subspaces are orthogonal first of all I'm certainly saying that they don't intersect in any nonzero vector.
But also I mean more than that just not intersecting isn't good enough.
So give me an example, oh, let's say in the plane, oh well, when do we have orthogonal subspaces in the plane?
Yeah, tell me in the plane, so we don't -- there aren't that many different subspaces in the plane.
What what have we got in the plane as possible subspaces?
The zero vector, real small.
A line through the origin.
Or the whole plane.
OK. Now so when is a line through the origin orthogonal to the whole plane?
Never, right, never.
When is a line through the origin orthogonal to the zero subspace?
Always.
Right.
When is a line through the origin orthogonal to a different line through the origin?
Well, that's the case that we all have a clear picture of, they -- the two lines have to meet at ninety degrees.
They have only the -- so that's like this simple case I'm talking about.
There's one subspace, there's the other subspace.
They only meet at zero.
And they're orthogonal.
OK. Now.
So we now know what it means for two subspaces to be orthogonal.
And now I want to say that this is true for the row space and the null space.
OK.
So that's the neat fact.
So row space is orthogonal to the null space.
Now how did I come up with that?
But you see the rank it's great, that means that these -- that these subspaces are just the right things, they're just cutting the whole space up into two perpendicular subspaces.
OK.
So why?
Well, what have I got to work with?
All I know is the null space.
The null space has vectors that solve Ax equals zero.
So this is a guy x. x is in the null space.
Then Ax is zero.
So why is it orthogonal to the rows of A?
If I write down Ax equals zero, which is all I know about the null space, then I guess I want you to see that that's telling me, just that equation right there is telling me that the rows of A, let me write it out.
There's row one of A.
Row two.
Row m of A. that's A.
And it's multiplying X.
And it's producing zero.
OK.
Written out that way you'll see it.
So I'm saying that a vector in the row space is perpendicular to this guy X in the null space.
And you see why?
Because this equation is telling you that row one of A multiplying that's a dot product, right?
Row one of A dot product with this x is producing this zero.
So x is orthogonal to the first row.
And to the second row.
Row two of A, x is giving that zero.
Row m of A times x is giving that zero.
So x is -- the equation is telling me that x is orthogonal to all the rows.
Right, it's just sitting there.
That's all we -- it had to be sitting there because we didn't know anything more about the null space than this.
And now I guess to be totally complete here I'd now check that x is orthogonal to each separate row.
But what else strictly speaking do I have to do?
To show that those subspaces are orthogonal, I have to take this x in the null space and show that it's orthogonal to every vector in the row space, every vector in the row space, so what -- what else is in the row space?
This row is in the row space, that row is in the row space, they're all there, but it's not the whole row space, right, we just have to like remember, what does it mean, what does that word space telling us?
And what else is in the row space?
Besides the rows?
All their combinations.
So I really have to check that sure enough if x is perpendicular to row one, row two, all the different separate rows, then also x is perpendicular to a combination of the rows.
And that's just matrix multiplication again.
You know, I have row one transpose x is zero, so on, row two transpose x is zero, so I'm entitled to multiply that by some c1, this by some c2, I still have zeroes, I'm entitled to add, so I have c1 row one so -- so all this when I put that together that's big parentheses c1 row one plus c2 row two and so on.
Transpose x is zero.
Right?
I just added the zeroes and got zero, and I just added these following the rule.
No big deal.
The whole point was right sitting in that.
OK.
So if I come back to this figure now I'm like a happier person.
Because I have this -- I now see how those subspaces are oriented.
And these subspaces are also oriented.
Well, actually why is that orthogonality?
Well, it's the same statement for A transpose that that one was for A.
So I won't take time to prove it again because we've checked it for every matrix and A transpose is just as good a matrix as A.
So we're orthogonal over there.
So we really have carved up this -- this was like carving up m-dimensional space into two subspaces and this one was carving up n-dimensional space into two subspaces.
And well, one more thing here.
One more important thing.
Let me move into three dimensions.
Tell me a couple of orthogonal subspaces in three dimensions that somehow don't carve up the whole space, there's stuff left there.
I'm thinking of a couple of orthogonal lines.
If I -- suppose I'm in three dimensions, R^3.
And I have one line, one one-dimensional subspace, and a perpendicular one.
Could those be the row space and the null space?
Could those be the row space and the null space?
Could I be in three dimensions and have a row space that's a line and a null space that's a line?
No.
Why not?
Because the dimensions aren't right.
Right?
The dimensions are no good.
The dimensions here, r and n-r, they add up to three, they add up to n. If I take -- just follow that example -- if the row space is one-dimensional, suppose A is what's a good in R^3, I want a one-dimensional row space, let me take one, two, five, two, four, ten.
What's the dimension of that row space?
One.
What's the dimension of the null space?
Tell what's the null space look like in that case?
The row space is a line, right?
One-dimensional, it's just a line through one, two, five.
Geometrically what's the row space look like?
What's its dimension?
So here r here n is three, the rank is one, so the dimension of the null space, so I'm looking at this x, x1, x2, x3.
To give zero.
So the dimension of the null space is we all know is two.
Right.
It's a plane.
And now actually we know, we see better, what plane is it?
What plane is it?
It's the plane that's perpendicular to one, two, five.
Right?
We now see.
In fact the two, four, ten didn't actually have any effect at all.
I could have just ignored that.
That didn't change the row space or the null space.
I'll just make that one equation.
Yeah.
OK. Sure.
That's the easiest to deal with.
One equation.
Three unknowns.
And I want to ask -- what would the equation give me the null space, and you would have said back in September you would have said it gives you a plane, and we're completely right.
And the plane it gives you, the normal vector, you remember in calculus, there was this dumb normal vector called N. Well there it is.
One, two, five.
OK. What is the what's the point I want to make here?
I want to make -- I want to emphasize that not only are the -- let me write it in words.
So I want to write the null space and the row space are orthogonal, that's this neat fact, which we've -- we've just checked from Ax equals zero, but now I want to say more because there's a little more that's true.
Their dimensions add to the whole space.
So that's like a little extra information.
That it's not like I could have -- I couldn't have a line and a line in three dimensions.
Those don't add up one and one don't add to three.
So I used the word orthogonal complements in R^n.
And the idea of this word complement is that the orthogonal complement of a row space contains not just some vectors that are orthogonal to it, but all.
So what does that mean?
That means that the null space contains all, not just some but all, vectors that are perpendicular to the row space.
OK. Really what I've done in this half of the lecture is just notice some of the nice geometry that -- that we didn't pick up before because we didn't discuss perpendicular vectors before.
But it was all sitting there.
And now we picked it up.
That these vectors are orthogonal complements.
And I guess I even call this part one of the fundamental theorem of linear algebra.
The fundamental theorem of linear algebra is about these four subspaces, so part one is about their dimension, maybe I should call it part two now.
Their dimensions we got.
Now we're getting their orthogonality, that's part two.
And part three will be about bases for them.
Orthogonal bases.
So that's coming up.
OK.
So I'm happy with that geometry right now.
OK. OK. Now what's my next goal in this chapter?
Here's the main problem of the chapter.
The main problem of the chapter is -- so this is coming.
It's coming attraction.
This is the very last chapter that's about Ax=b.
I would like to solve that system of equations when there is no solution.
You may say what a ridiculous thing to do.
But I have to say it's done all the time.
In fact it has to be done.
You get -- so the problem is solve -- the best possible solve I'll put quote Ax=b when there is no solution.
And of course what does that mean?
b isn't in the column space.
And it's quite typical if this matrix A is rectangular if I -- maybe I have m equations and that's bigger than the number of unknowns, then for sure the rank is not m, the rank couldn't be m now, so there'll be a lot of right-hand sides with no solution, but here's an example.
Some satellite is buzzing along.
You measure its position.
You make a thousand measurements.
So that gives you a thousand equations for the -- for the parameters that -- that give the position.
But there aren't a thousand parameters, there's just maybe six or something.
Or you're measuring the -- you're doing questionnaires.
You're measuring resistances.
You're taking pulses.
You're measuring somebody's pulse rate.
There's just one unknown.
OK.
The pulse rate.
So you measure it once, OK, fine, but if you really want to know it, you measure it multiple times, but then the measurements have noise in them, so there's -- the problem is that in many many problems we've got too many equations and they've got noise in the right-hand side.
So Ax=b I can't expect to solve it exactly right, because I don't even know what -- there's a measurement mistake in b.
But there's information too.
There's a lot of information about x in there.
And what I want to do is like separate the noise, the junk, from the information.
And so this is a straightforward linear algebra problem.
How do I solve, what's the best solution?
OK. Now.
I want to say so that's like describes the problem in an algebraic way.
I got some equations, I'm looking for the best solution.
Well, one way to find it is -- one way to start, one way to find a solution is throw away equations until you've got a nice, square invertible system and solve that.
That's not satisfactory.
There's no reason in these measurements to say these measurements are perfect and these measurements are useless.
We want to use all the measurements to get the best information, to get the maximum information.
But how?
OK. Let me anticipate a matrix that's going to show up.
This A is typically rectangular.
But a matrix that shows up whenever you have -- and we chapter three was all about rectangular matrices.
And we know when this is solvable, you could do elimination on it, right?
But I'm thinking hey, you do elimination and you get equation zero equal other non-zeroes.
I'm thinking we really -- elimination is going to fail.
So that's our question.
Elimination will get us down to -- will tell us if there is a solution or not.
But I'm now thinking not.
So what are we going to do?
OK. All right.
I want to tell you to jump ahead to the matrix that will play a key role.
So this is the matrix that you want to understand for this chapter four.
And it's the matrix A transpose A.
What's -- tell me some things about that matrix.
So A is this m by n matrix, rectangular, but now I'm saying that the good matrix that shows up in the end is A transpose A.
So tell me something about that.
Is it -- what's the first thing you know about A transpose A.
It's square.
Right?
Square because this is m by n and this is n by m. So this is the result is n by n. Good.
Square.
What else?
It's symmetric.
Good.
It's symmetric.
Because you remember how to do that.
If we transpose that matrix let's transpose it, A transpose A, if I transpose it, then that comes first transposed, this comes second, transposed, and then transposing twice is leaves it -- brings it back to the same so it's symmetric.
Good.
Now we now know how to ask more about a matrix.
I'm interested in is it invertible?
If not, what's its null space?
So I want to know about -- because you're going to see, well, let me -- let me even, well I shouldn't do this, but I will.
Let me tell you what equation to solve when you can't solve that one.
The good equation comes from multiplying both sides by A transpose, so the good equation that you get to is this one.
A transpose Ax equals A transpose b.
That will be the central equation in the chapter.
So I think why not tell it to you.
Why not admit it right away.
OK.
I have to -- I should really give x. I want to sort of indicate that this x isn't I mean this x was the solution to that equation if it existed, but probably didn't.
Now let me give this a different name, x hat.
Because I'm hoping this one will have a solution.
And I'm saying that it's my best solution.
I'll have to say what does best mean.
But that's going to be my -- my plan.
I'm going to say that the best solution solves this equation.
So you see right away why I'm so interested in this matrix A transpose A.
And in its invertibility.
OK. Now, when is it invertible?
OK. Let me take a case, let me just do an example and then -- I'll just pick a matrix here.
Just so we see what A transpose A looks like.
So let me take a matrix A one, one, one, one, two, five.
Just to invent a matrix.
So there's a matrix A.
Notice that it has M equal three rows and N equal two columns.
Its rank is -- the rank of that matrix is two.
Right, yeah, the columns are independent.
Does Ax equal b?
If I look at Ax=b, so x is just x1 x2, and b is b1 b2 b3.
Do I expect to solve Ax=b?
What's -- no way, right?
I mean linear algebra's great, but solving three equations with only two unknowns usually we can't do it.
We can only solve it if this vector is b is what?
I can solve that equation if that vector b1 b2 b3 is in the column space.
If it's a combination of those columns then fine.
But usually it won't be.
The combinations just fill up a plane and most vectors aren't on that plane.
So what I'm saying is that I'm going to work with the matrix A transpose A.
And I just want to figure out in this example what A transpose A is.
So it's two by two.
The first entry is a three, the next entry is an eight, this entry is -- what's that entry?
Eight, for sure.
We knew it had to be, and this entry is, what's that now, getting out my trusty calculator, thirty, is that right?
And is that matrix invertible?
Thirty.
There's an A transpose A.
And it is invertible, right?
Three, eight is not a multiple of eight, thirty -- and it's invertible.
And that's the normal, that's what I expect.
So this is I want to show.
So here's the final -- here's the key point.
The null space of A transpose A -- it's not going to be always invertible.
Tell me a matrix -- I have to say that I can't say A transpose A is always invertible.
Because that's asking too much.
I mean what could the matrix A be, for example, so that A transpose A was not invertible?
Well, it even could be the zero matrix.
I mean that's like extreme case.
Suppose I make this rank -- suppose I change to that A.
Now I figure out A transpose A again and I get -- what do I get?
I get nine, I get nine of course and here I get what's that entry?
Twenty-seven.
And is that matrix invertible?
No.
And why do I -- I knew it wouldn't be invertible anyway.
Because this matrix only has rank one.
And if I have a product of matrices of rank one, the product is not going to have a rank bigger than one.
So I'm not surprised that the answer only has rank one.
And that's what I -- always happens, that the rank of A transpose A comes out to equal the rank of A.
So, yes, so the null space of A transpose A equals the null space of A, the rank of A transpose A equals the rank of A.
So let's -- as soon as I can why that's true.
But let's draw from that what the fact that I want.
This tells me that this square symmetric matrix is invertible if -- so here's my conclusion.
A transpose A is invertible if exactly when -- exactly if this null space is only got the zero vector.
Which means the columns of A are independent.
So A transpose A is invertible exactly if A has independent columns.
That's the fact that I need about A transpose A.
And then you'll see next time how A transpose A enters everything.
Next lecture is actually a crucial one.
Here I'm preparing for it by getting us thinking about A transpose A.
And its rank is the same as the rank of A, and we can decide when it's invertible.
OK.
So I'll see you Friday.
Thanks.
OK, guys the -- we're almost ready to make this lecture immortal.
OK. Are we on?
All right.
This is an important lecture.
It's about projections.
Let me start by just projecting a vector b down on a vector a.
So just to so you see what the geometry looks like in when I'm in -- in just two dimensions, I'd like to find the point along this line so that line through a is a one-dimensional subspace, so I'm starting with one dimension.
I'd like to find the point on that line closest to a.
Can I just take that problem first and then I'll explain why I want to do it and why I want to project on other subspaces.
So where's the point closest to b that's on that line?
It's somewhere there.
And let me connect that and -- and what's the whole point of my picture now?
What's the -- where does orthogonality come into this picture?
The whole point is that this best point, that's the projection, P, of b onto the line, where's orthogonality?
It's the fact that that's a right angle.
That this -- the error -- this is like how much I'm wrong by -- this is the difference between b and P, the whole point is -- that's perpendicular to a.
That's got to give us the equation.
That's got to tell us -- that's the one fact we know, that's got to tell us where that projection is.
Let me also say, look -- I've drawn a triangle there.
So if we were doing trigonometry we would do like we would have angles theta and distances that would involve sine theta and cos theta that leads to lousy formulas compared to linear algebra.
The formula that we want comes out nicely and what's the -- what do we know?
We know that P, this projection, is some multiple of a, right?
It's on that line.
So we know it's in that one-dimensional subspace, it's some multiple, let me call that multiple x, of a.
So really it's that number x I'd like to find.
So this is going to be simple in 1-D, so let's just carry it through, and then see how it goes in high dimensions.
OK.
The key fact is -- the key to everything is that perpendicular.
The fact that a is perpendicular to a is perpendicular to e. Which is (b-ax), xa.
I don't care what -- xa.
That that equals zero.
Do you see that as the central equation, that's saying that this a is perpendicular to this -- correction, that's going to tell us what x is.
Let me just raise the board and simplify that and out will come x. OK.
So if I simplify that, let's see, I'll move one to -- one term to one side, the other term will be on the other side, it looks to me like x times a transpose a is equal to a transpose b.
Right?
I have a transpose b as one f- one term, a transpose a as the other, so right away here's my a transpose a.
But it's just a number now.
And I divide by it.
And I get the answer.
x is a transpose b over a transpose a.
And P, the projection I wanted, is -- that's the right multiple.
That's got a cosine theta built in.
But we don't need to look at angles.
It's -- we've just got vectors here.
And the projection is P is a times that x.
Or x times that a.
But I'm really going to -- eventually I'm going to want that x coming on the right-hand side.
So do you see that I've got two of the three formulas already, right here, I've got the -- that's the equation -- that leads me to the answer, here's the answer for x, and here's the projection.
OK. can I do add just one more thing to this one-dimensional problem?
One more like lift it up into linear algebra, into matrices.
Here's the last thing I want to do -- but don't forget those formulas.
a transpose b over a transpose a.
Actually let's look at that for a moment first.
Suppose -- Let me take this next step.
So P is a times x.
So can I write that then?
P is a times this neat number, a transpose b over a transpose a.
That's our projection.
Can I ask a couple of questions about it, just while we look, get that digest that formula.
Suppose b is doubled.
Suppose I change b to two b.
What happens to the projection?
So suppose I instead of that vector b that I drew on the board make it two b, twice as long -- what's the projection now?
It's doubled too, right?
It's going to be twice as far, if b goes twice as far, the projection will go twice as far.
And you see it there.
If I put in an extra factor two, then P's got that factor too.
Now what about if I double a?
What if I double the vector a that I'm projecting onto?
What changes?
The projection doesn't change at all.
Right?
Because I'm just -- the line didn't change.
If I double a or I take minus a.
It's still that same line.
The projection's still in the same place.
And of course if I double a I get a four up above, and I get a four -- an extra four below, they cancel out, and the projection is the same.
OK.
So really, this -- I want to look at this as the projection -- there's a matrix here.
The projection is carried out by some matrix that I'm going to call the projection matrix.
And in other words the projection is some matrix that acts on this guy b and produces the projection.
The projection P is the projection matrix acting on whatever the input is.
The input is b, the projection matrix is P. OK. Actually you can tell me right away what this projection matrix is.
So this is a pretty interesting matrix.
What matrix is multiplying b?
I'm just -- just from my formula -- I see what P is.
P, this projection matrix, is -- what is it?
I see a a transpose above, and I see a transpose a below.
And those don't cancel.
That's not one.
Right?
That's a matrix.
Because down here, the a transpose a, that's just a number, a transpose a, that's the length of a squared, and up above is a column times a row.
Column times a row is a matrix.
So this is a full-scale n by n matrix, if I -- if I'm in n dimensions.
And it's kind of an interesting one.
And it's the one which if I multiply by b then I get this, you see once again I'm putting parentheses in different places.
I'm putting the parentheses right there.
I'm saying OK, that's really the matrix that produces this projection.
OK. Now, tell me -- all right, what are the properties of that matrix?
I'm just using letters here, a and b, I could put in numbers, but I think it's -- for once, it's clearer with letters, because all formulas are simple, a transpose b over a transpose a -- that's the number that multiplies the a, and then I see wait a minute, there's a matrix here and what's the rank of that matrix, by the way?
What's the rank of that matrix, yeah -- let me just ask you about that matrix.
Which looks a little strange, a a transpose over this number.
But well, I could ask you its column space.
Yeah, let me ask you its column space.
So what's the column space of a matrix?
If you multiply that matrix by anything you always get in the column space, right?
The column space of a matrix is when you multiply any vector by that matrix -- any vector b, by the matrix, you always land in the column space.
That's what column spaces work.
Now what space do we always land in?
What's the column space of -- what's the result when I multiply this any vector b by my matrix?
So I have P times b, where I?
I'm on that line, right?
The column space, so here are facts about this matrix.
The column space of P, of this projection matrix, is the line through a.
And the rank of this matrix is you can all say it at once one.
Right.
The rank is one.
This is a rank one matrix.
Actually it's exactly the form that we're familiar with a rank one matrix.
A column times a row, that's a rank one matrix, that column is the basis for the column space.
Just one dimension.
OK.
So I know that much about the matrix.
But now there are two more facts about the matrix that I want to notice.
First of all is the matrix symmetric?
That's a natural question for matrices.
And the answer is yes.
If I take the transpose of this -- there's a number down there, the transpose of a a transpose is a a transpose.
So P is symmetric.
P transpose equals P. So this is a key property.
That the projection matrix is symmetric.
One more property now and this is the real one.
What happens if I do the projection twice?
So I'm looking for something, some information about P squared.
But just give me in terms of that picture, in terms my picture, I take any vector b, I multiply it by my projection matrix, and that puts me there, so this is Pb.
And now I project again.
What happens now?
What happens when I apply the projection matrix a second time?
To this, so I'm applying it once brings me here and the second time brings me I stay put.
Right?
The projection for a point on this line the projection is right where it is.
The projection is the same point.
So that means that if I project twice, I get the same answer as I did in the first projection.
So those are the two properties that tell me I'm looking at a projection matrix.
It's symmetric and it's square is itself.
Because if I project a second time, it's the same result as the first result.
OK.
So that's -- and then here's the exact formula when I know what I'm projecting onto, that line through a, then I know what P is.
So do you see that I have all the pieces here for projection on a line?
Now, and those -- please remember those.
So there are three formulas to remember.
The formula for x, the formula for P, which is just ax, and then the formula for capital P, which is the matrix.
Good.
Good.
OK. Now I want to move to more dimensions.
So we're going to have three formulas again but you'll have to -- they'll be a little different because we won't have a single line but -- a plane or three-dimensional or a n-dimensional subspace.
OK.
So now I'll move to the next question.
Maybe -- let me say first why I want this projection, and then we'll figure out what it is, we'll go completely in parallel there, and then we'll use it.
OK, why do I want this projection?
Well, so why project?
It's because I'm as I mentioned last time this new chapter deals with equations Ax=b may have no solution.
So that's really my problem, that I'm given a bunch of equations probably too many equations, more equations than unknowns, and I can't solve them.
OK.
So what do I do?
I solve the closest problem that I can solve.
And what's the closest one?
Well, ax will always be in the column space of a.
That's my problem.
My problem is ax has to be in the column space and b is probably not in the column space.
So I change b to what?
I choose the closest vector in the column space, so I'll solve Ax equal P instead.
That one I can do.
Where P is this is the projection of b onto the column space.
That's why I want to be able to do this.
Because I have to find a solution here, and I'm going to put a little hat there to indicate that it's not the x, it's not the x that doesn't exist, it's the x hat that's best possible.
So I must be able to figure out what's the good projection there.
What's the good right-hand side that is in the column space that's as close as possible to b and then I'm -- then I know what to do.
OK.
So now I've got that problem.
So that's why I have the problem again but now let me say I'm in three dimensions, so I have a plane maybe for example, and I have a vector b that's not in the plane.
And I want to project b down into the plane.
OK.
So there's my question.
How do I project a vector and I'm -- what I'm looking for is a nice formula, and I'm counting on linear algebra to just come out right, a nice formula for the projection of b into the plane.
The nearest point.
So this again a right angle is going to be crucial.
OK. Now so what's -- first of all I've got to say what is that plane.
To get a formula I have to tell you what the plane is.
How I going to tell you a plane?
I'll tell you a basis for the plane, I'll tell you two vectors a one and a two that give you a basis for the plane, so that -- let us say -- say there's an a one and here's an a -- a vector a two.
They don't have to be perpendicular.
But they better be independent, because then that tells me the plane.
The plane is the -- is the plane of a one and a two.
And actually going back to my -- to this connection, this plane is a column space, it's the column space of what matrix?
What matrix, so how do I connect the two questions?
I'm thinking how do I project onto a plane and I want to get a matrix in here.
Everything's cleaner if I write it in terms of a matrix.
So what matrix has these -- has that column space?
Well of course it's just the matrix that has a one in the first column and a two in the second column.
Right, just just let's be sure we've got the question before we get to the answer.
So I'm looking for again I'm given a matrix a with two columns.
And really I'm ready once I get to two I'm ready for n. So it could be two columns, it could be n columns.
I'll write the answer in terms of the matrix a.
And the point will be those two columns describe the plane, they describe the column space, and I want to project.
OK. And I'm given a vector b that's probably not in the column space.
Of course, if b is in the column space, my projection is simple, it's just b.
But most likely I have an error e, this b minus P part, which is probably not zero.
OK.
But the beauty is that I know -- from geometry or I could get it from calculus or I could get it from linear algebra that that this this vector -- this is the part of b that's that's perpendicular to the plane.
That e is perpendicular is perpendicular to the plane.
If your intuition is saying that that's the crucial fact.
That's going to give us the answer.
OK.
So let me, that's the problem.
Now for the answer.
So this is a lecture that's really like moving along.
Because I'm just plotting that problem up there and asking you what combination -- now, yeah, so what is it?
What is this projection P?
P. This is projection P, is some multiple of these basis guys, right, some multiple of the columns.
But I don't like writing out x one a one plus x two a two, I would rather right that as ax.
Well, actually I should put if I'm really doing everything right, I should put a little hat on it -- to remember that this x -- that those are the numbers and I could have a put a hat way back there is right, so this is this is the projection, P. P is ax bar.
And I'm looking for x bar.
So that's what I want an equation for.
So now I've got hold of the problem.
The problem is find the right combination of the columns so that the error vector is perpendicular to the plane.
Now let me turn that into an equation.
So I'll raise the board and just turn that -- what we've just done into an equation.
So let me I'll write again the main point.
The projection is ax b- x hat.
And our problem is find x hat.
And the key is that b minus ax hat, that's the error.
This is the e. Is perpendicular to the plane.
That's got to give me well what I looking for, I'm looking for two equations now because I've got an x one and an x two.
And I'll get two equations because so this thing e is perpendicular to the plane.
So what does that mean?
I guess it means it's perpendicular to a one and also to a two.
Right, those are two vectors in the plane and the things that are perpendicular to the plane are perpendicular to a one and a two.
Let me just repeat.
This this guy then is perpendicular to the plane so it's perpendicular to that vector and that vector.
Not -- it's perpendicular to that of course.
But it's perpendicular to everything I the plane.
And the plane is really told me by a one and a two.
So really I have the equations a one transpose b minus ax is zero.
And also a two transpose b minus ax is zero.
Those are my two equations.
But I want those in matrix form.
I want to put those two equations together as a matrix equation and it's just comes out right.
Look at the matrix a transpose.
Put a one a one transpose is its first row, a two transpose is its second row, that multiplies this b-ax, and gives me the zero and the zero.
I'm you see the -- this is one way -- to come up with this equation.
So the equation I'm coming up with is a transpose b-ax hat is zero.
OK. That's my equation.
All right.
Now I want to stop for a moment before I solve it and just think about it.
First of all do you see that that equation back in the very first problem I solved on a line, what was -- what was on a line the matrix a only had one column, it was just little a.
So in the first problem I solved, projecting on a line, this for capital a you just change that to little a and you have the same equation that we solved before.
a transpose e equals zero.
OK. Now a second thing, second comment.
I would like to since I know about these four subspaces, I would like to get them into this picture.
So let me ask the question, what subspace is this thing in?
Which of the four subspaces is that error vector e, this is this is nothing but e -- this is this guy, coming in down perpendicular to the plane.
What subspace is e in?
From this equation.
Well the equation is saying a transpose e is zero.
So I'm learning here that e is in the null space of a transpose.
Right?
That's my equation.
And now I just want to see hey of course that that was right.
Because things that are in the null space of a transpose, what do we know about the null space of a transpose?
So that last lecture gave us the sort of the geometry of these subspaces.
And the orthogonality of them.
And do you remember what it was?
What on the right side of our big figure we always have the null space of a transpose and the column space of a, and they're orthogonal.
So e in the null space of a transpose is saying e is perpendicular to the column space of a.
Yes.
I just feel OK, the damn thing came out right.
The equation for the equation that I struggled to find for e really said what I wanted, that the error e is perpendicular to the column space of a, just right.
And from our four fundamental subspaces we knew that that is the same as that.
To say e is in the null space of a transpose says e's perpendicular to the column space.
OK.
So we've got this equation.
Now let's just solve it.
All right.
Let me just rewrite it as a transpose a x hat equals a transpose b.
That's our equation.
That gives us x.
And -- allow me to keep remembering the one-dimensional case.
The one-dimensional case, this was little a.
So this was just a number, little a transpose, a transpose a was just a vector row times a column, a number.
And this was a number.
And x was the ratio of those numbers.
But now we've got matrices, this one is n by n. a transpose a is an n by n matrix.
OK.
So can I move to the next board for the solution?
OK.
This is the -- the key equation.
Now I'm ready for the formulas that we have to remember.
What's x hat?
What's the projection, what's the projection matrix, those are my three questions.
That we answered in the 1-D case and now we're ready for in the n-dimensional case.
So what is x hat?
Well, what can I say but a transpose a inverse, a transpose b.
That's the solution to -- to our equation.
OK. What's the projection?
That's more interesting.
What's the projection?
The projection is a x hat.
That's how x hat got into the picture in the first place.
x hat was the was the combination of columns in the I had to look for those numbers and now I found them.
Was the combination of the columns of a that gave me the projection.
OK.
So now I know what this guy is.
So it's just I multiply by a. a a transpose a inverse a transpose b.
That's looking a little messy but it's not bad.
That that combination is is our like magic combination.
This is the thing which is which use which is like what's it like, what was it in one dimension?
What was that we had this we must have had this thing way back at the beginning of the lecture.
What did we -- oh that a was just a column so it was little a, little a transpose over a transpose a, right, that's what it was in 1-D. You see what's happened in more dimensions, I -- I'm not allowed to to just divide because because I don't have a number, I have to put inverse, because I have an n by n matrix.
But same formula.
And now tell me what's the projection matrix?
What matrix is multiplying b to give the projection?
Right there.
Because there it -- I even already underlined it by accident.
The projection matrix which I use capital P is this, it's it's that thing, shall I write it again, a times a transpose a inverse times a transpose.
Now if you'll bear with me I'll think about what have I done here.
I've got this formula.
Now the first thing that occurs to me is something bad.
Look why don't I just you know here's a product of two matrices and I want its inverse, why don't I just use the formula I know for the inverse of a product and say OK, that's a inverse times a transpose inverse, what will happen if I do that?
What will happen if I say hey this is a inverse times a transpose inverse, then shall I do it?
It's going to go on videotape if I do it, and I don't -- all right, I'll put it there, but just like don't take the videotape quite so carefully.
OK.
So if I put that thing it -- it would be a a inverse a transpose inverse a transpose and what's that?
That's the identity.
But what's going on?
So why -- you see my question is somehow I did something wrong.
That that wasn't allowed.
And and and why is that?
Because a is not a square matrix.
a is not a square matrix.
It doesn't have an inverse.
So I have to leave it that way.
This is not OK.
If if a was a square invertible matrix, then then I couldn't complain.
Yeah I think -- let me think about that case.
But you but my main case, the whole reason I'm doing all this, is that a is this matrix that has x too many rows, it's just got a couple of columns, like a one and a two, but lots of rows.
Not square.
And if it's not square, this a transpose a is square but I can't pull it apart like this -- I'm not allowed to do this pull apart, except if a was square.
Now if a is square what's what's going on if a is a square matrix?
a nice square inv- invertible matrix.
Think.
What's up with that what's with that case.
So this is that the formula ought to work then too.
If a is a nice square invertible matrix what's its column space, so it's a nice n by n invertible everything great matrix, what's its column space, the whole of R^n.
So what's the projection matrix if I'm projecting onto the whole space?
It's the identity matrix right?
If I'm projecting b onto the whole space, not just onto a plane, but onto all of 3-D, then b is already in the column space, the projection is the identity, and this is gives me the correct formula, P is I.
But if I'm projecting onto a subspace then I can't split those apart and I have to stay with that formula.
OK. And what can I say if -- so I remember this formula for 1-D and that's what it looks like in n dimensions.
And what are the properties that I expected for any projection matrix?
And I still expect for this one?
That matrix should be symmetric and it is.
P transpose is P. Because if I transpose this, this guy's symmetric, and its inverse is symmetric, and if I transpose this one when I transpose it will pop up there, become a, that a transpose will pop up here, and I'm back to P again.
And do we dare try the other property which is P squared equal P?
It's got to be right.
Because we know geometrically that the first projection pops us onto the column space and the second one leaves us where we are.
So I expect that if I multiply by let me do it -- if I multiply by another P, so there's another a, another a transpose a inverse a transpose, can you -- eight (a)-s in a row is quite obscene but -- do you see that it works?
So I'm squaring that so what do I do-- how do I see that multiplication?
Well, yeah, I just want to put parentheses in good places, so I see what's happening, yeah, here's an a transpose a sitting together -- so when that a transpose a multiplies its inverse, all that stuff goes, right.
And leaves just the a transpose at the end, which is just what we want.
So P squared equals P. So sure enough those two properties hold.
OK. OK we really have got now all the formulas.
x hat, the projection P, and the projection matrix capital P. And now my job is to use them.
OK.
So when would I have a bunch of equations, too many equations and yet I want the best answer and the -- the most important example, the most common example is if I have points so here's the -- here's the application.
v squared.
Fitting by a line.
OK.
So I'll start this application today and there's more in it than I can do in this same lecture.
So that'll give me a chance to recap the formulas and there they are, and recap the ideas.
So let me start the problem today.
I'm given a bunch of data points.
And they lie close to a line but not on a line.
Let me take that.
Say a t equal to one, two and three, I have one, and two and two again.
So my data points are this is the like the time direction and this is like well let me call that b or y or something.
I'm given these three points and I want to fit them by a line.
By the best straight line.
So the problem is fit the points one, one is the first point -- the second point is t equals two, b equal one, and the third point is t equal three, b equal to two.
So those are my three points, t equal sorry,that's two.
Yeah, OK.
So this is the point one, one.
This is the point two, two, and that's the point three, two.
And of course there isn't a -- a line that goes through them.
So I'm looking for the best line.
I'm looking for a line that probably goes somewhere, do you think it goes somewhere like that?
I didn't mean to make it go through that point, it won't.
It'll kind of -- it'll go between so the error there and the error there and the error there are as small as I can get them.
OK, what I'd like to do is find the matrix a.
Because once I've found the matrix a the formulas take over.
So what I'm looking for this line, b is C+Dt.
So this is in the homework that I sent out for today.
Find the best line.
So I'm looking for these numbers.
C and D. That tell me the line and I want them to be as close to going through those three points as I can get.
I can't get exactly so there are three equations to go through the three points.
It would it will go exactly through that point if let's see that first point has t equal to one, so that would say C+D equaled 1.
This is the one, one.
The second point t is two.
So C+2D should come out to equal 2.
But I also want to get the third equation in and at that third equation t is three so C+3D equals only 2.
That's the key.
Is to write down what equations we would like to solve but can't.
Reason we if we could solve them that would mean that we could put a line through all three points and of course if these numbers one, two, two were different, we could do it.
But with those numbers, one, two, two, we can't.
So what is our equation Ax equal Ax equal b that we can't solve?
I just want to say what's the matrix here, what's the unknown x, and what's the right-hand side.
So this is the matrix is one, one, one, one, two, three.
The unknown is C and D. And the right-hand side if one, two, two.
Right, I've just taken my equations and I've said what is Ax and what is b.
Then there's no solution, this is the typical case where I have three equations -- two unknowns, no solution, but I'm still looking for the best solution.
And the best solution is taken is is to solve not this equation Ax equal b which has which has no solution but the equation that does have a solution, which was this one.
So that's the equation to solve.
That's the central equation of the subject.
I can't solve Ax=b but magically when I multiply both sides by a transpose I get an equation that I can solve and its solution gives me x, the best x, the best projection, and I discover what's the matrix that's behind it.
OK.
So next time I'll complete an example, numerical example.
today was all letters, numbers next time.
Thanks.
OK.
Here's lecture sixteen and if you remember I ended up the last lecture with this formula for what I called a projection matrix.
And maybe I could just recap for a minute what is that magic formula doing?
For example, it's supposed to be -- it's supposed to produce a projection, if I multiply by a b, so I take P times b, I'm supposed to project that vector b to the nearest point in the column space.
OK. Can I just -- one way to recap is to take the two extreme cases.
Suppose a vector b is in the column space?
Then what do I get when I apply the projection P?
So I'm projecting into the column space but I'm starting with a vector in this case that's already in the column space, so of course when I project it I get B again, right.
And I want to show you how that comes out of this formula.
Let me do the other extreme.
Suppose that vector is perpendicular to the column space.
So imagine this column space as a plane and imagine b as sticking straight up perpendicular to it.
What's the nearest point in the column space to b in that case?
So what's the projection onto the plane, the nearest point in the plane, if the vector b that I'm looking at is -- got no component in the column space, it's sticking completely -- ninety degrees with it, then Pb should be zero, right.
So those are the two extreme cases.
The average vector has a component P in the column space and a component perpendicular to it, and what the projection does is it kills this part and it preserves this part.
OK. Can we just see why that's true?
Just -- that formula ought to work.
So let me start with this one.
What vectors are in the -- are perpendicular to the column space?
How do I see that I really get zero?
I have to think, what does it mean for a vector b to be perpendicular to the column space?
So if it's perpendicular to all the columns, then it's in some other space.
We've got our four spaces so the reason I do this is it's perfectly using what we know about our four spaces.
What vectors are perpendicular to the column space?
Those are the guys in the null space of A transpose, right?
That's the first section of this chapter, that's the key geometry of these spaces.
If I'm perpendicular to the column space, I'm in the null space of A transpose.
OK.
So if I'm in the null space of A transpose, and I multiply this big formula times b, so now I'm getting Pb, this is now the projection, Pb, do you see that I get zero?
Of course I get zero.
Right at the end there, A transpose b will give me zero right away.
So that's why that zero's here.
Because if I'm perpendicular to the column space, then I'm in the null space of A transpose and A transpose b is OK, what about the other possibility.
zilch.
How do I see that this formula gives me the right answer if b is in the column space?
So what's a typical vector in the column space?
It's a combination of the columns.
How do I write a combination of the columns?
So tell me, how would I write, you know, your everyday vector that's in the column space?
It would have the form A times some x, right?
That's what's in the column space, A times something.
That makes it a combination of the columns.
So these b's were in the null space of A transpose.
These guys in the column space, those b's are Ax-s.
Right?
If b is in the column space then it has the form Ax.
I'm going to stick that on the quiz or the final for sure.
That you have to realize -- because we've said it like a thousand times that the things in the column space are vectors A times x. OK. And do you see what happens now if we use our formula?
There's an A transpose A.
Gets canceled by its inverse.
We're left with an A times x.
So the result was Ax.
Which was b.
Do you see that it works?
This is that whole business.
Cancel, cancel, leaving Ax.
And Ax was b.
So that turned out to be b, in this case.
OK, so geometrically what we're seeing is we're taking a vector -- we've got the column space and perpendicular to that is the null space of A transpose.
And our typical vector b is out here.
There's zero, so there's our typical vector b, and what we're doing is we're projecting it to P. And the -- and of course at the same time we're finding the other part of it which is e. So the two pieces, the projection piece and the error piece, add up to the original b. OK. That's like what our matrix does.
So this is P -- P is -- this P is Ab, is sorry -- is Pb, it's the projection, applied to b, and this one is -- OK, that's a projection too.
That's a projection down onto that space.
What's a good formula for it?
Suppose I ask you for the projection of the projection matrix onto the -- this space, this perpendicular space?
So if this projection was P, what's the projection that gives me e?
It's the -- what I want is to get the rest of the vector, so it'll be just I minus P times b, that's a projection too.
That's the projection onto the perpendicular space.
OK.
So if P's a projection, I minus P is a projection.
If P is symmetric, I minus P is symmetric.
If P squared equals P, then I minus P squared equals I minus P. It's just -- the algebra -- is only doing what your -- picture is completely telling you.
But the algebra leads to this expression.
That expression for P given -- given a basis for the subspace, given the matrix A whose columns are a basis for our column space.
OK, that's recap because you -- you need to see that formula more than once.
And now can I pick up on using it?
So now -- and the -- it's like, let me do that again, I'll go right through a problem that I started at the end, which is find a best straight line.
You remember that problem, I -- I picked a particular set of points, they weren't specially brilliant, t equal one, two, three, the heights were one, two, and then two again.
So they were -- heights were that point, that point, which makes it look like I've got a nice forty-five-degree line -- but then the third point didn't lie on the line.
And I wanted to find the best straight line.
So I'm looking for the -- this line, y=C+Dt.
And it's not going to go through all three points, because no line goes through all three points.
So I'm going to pick the best line, the -- the best being the one that makes the overall error as small as I can make it.
Now I have to tell you, what is that overall error?
And -- because that determines what's the winning line.
If we don't know -- I mean we have to decide what we mean by the error -- and then we minimize and we find the right -- the best C and D. So if I went through this -- if I went through that point, OK.
I would solve the equation C+D=1.
Because at t equal to one -- I'd have C plus D, and it would come out right.
If it went through this point, I'd have C plus two D equal to two.
Because at t equal to two, I would like to get the answer two.
At the third point, I have C plus three D because t is three, but the -- the answer I'm shooting for is two again.
So those are my three equations.
And they don't have a solution.
But they've got a best solution.
What do I mean by best solution?
So let me take time out to remember what I'm talking about for best solution.
So this is my equation Ax=b.
A is this matrix, one, one, one, one, two, three.
x is my -- only have two unknowns, C and D, and b is my right-hand side, one, two, three.
OK. No solution.
Three eq- I have a three by two matrix, I do have two independent columns -- so I do have a basis for the column space, those two columns are independent, they're a basis for the column space, but the column space doesn't include that vector.
So best possible in this -- what would best possible mean?
The way that comes out to linear equations is I -- I want to minimize the sum of these -- I'm going to make an error here.
I'm going to make an error here.
I'm going to make an error there.
And I'm going to sum and square and add up those errors.
So it's a sum of squares.
It's a least squares solution I'm looking for.
So if I -- those errors are the difference between Ax and b.
That's what I want to make small.
And the way I'm measuring this -- this is a vector, right?
This is e1,e2 ,e3.
The Ax-b, this is the e. The error vector.
And small means its length.
The length of that vector.
That's what I'm going to try to minimize.
And it's convenient to square.
If I make something small, I make -- this is a never negative quantity, right?
The length of that vector.
The length will be zero exactly when the -- when I have the zero vector here.
That's exactly the case when I can solve exactly, b is in the column space, all great.
But I'm not in that case now.
I'm going to have an error vector, e. What's this error vector in my picture?
I guess what I'm trying to say is there's -- there's two pictures of what's going on.
There's two pictures of what's going on.
One picture is -- in this is the three points and the line.
And in that picture, what are the three errors?
The three errors are what I miss by in this equation.
So it's this -- this little bit here.
That vertical distance up to the line.
There's one -- sorry there's one, and there's C plus D. And it's that difference.
Here's two and here's C+2D.
So vertically it's that distance -- that little error there is e1.
This little error here is e2.
This little error coming up is e3.
e3.
And what's my overall error?
Is e1 square plus e2 squared plus e3 squared.
That's what I'm trying to make small.
I -- some statisticians -- this is a big part of statistics, fitting straight lines is a big part of science -- and specifically statistics, where the right word to use would be regression.
I'm doing regression here.
Linear regression.
And I'm using this sum of squares as the measure of error.
Again, some statisticians would be -- they would say, OK, I'll solve that problem because it's the clean problem.
It leads to a beautiful linear system.
But they would be a little careful about these squares, for -- in this case.
If one of these points was way off.
Suppose I had a measurement at t equal zero that was way off.
Well, would the straight line, would the best line be the same if I had this fourth point?
Suppose I have this fourth data point.
No, certainly the line would -- it wouldn't be the -- that wouldn't be the best line.
Because that line would have a giant error -- and when I squared it it would be like way out of sight compared to the others.
So this would be called by statisticians an outlier, and they would not be happy to see the whole problem turned topsy-turvy by this one outlier, which could be a mistake, after all.
So they wouldn't -- so they wouldn't like maybe squaring, if there were outliers, they would want to identify them.
OK.
I'm not going to -- I don't want to suggest that least squares isn't used, it's the most used, but it's not exclusively used because it's a little -- overcompensates for outliers.
Because of that squaring.
OK.
So suppose we don't have this guy, we just have these three equations.
And I want to make -- minimize this error.
OK. Now, what I said is there's two pictures to look at.
One picture is this one.
The three points, the best line.
And the errors.
Now, on this picture, what are these points on the line, the points that are really on the line?
So they're -- points, let me call them P1, P2, and P3, those are three numbers, so this -- this height is P1, this height is P2, this height is P3, and what are those guys?
Suppose those were the three values instead of -- there's b1, ev- everybody's seen all these -- sorry, my art is as usual not the greatest, but there's the given b1, the given b2, and the given b3.
I promise not to put a single letter more on that picture.
OK.
There's b1, P1 is the one on the line, and e1 is the distance between.
And same at points two and same at points three.
OK, so what's up?
What's up with those Ps?
P1, P2, P3, what are they?
They're the components, they lie on the line, right?
They're the points which if instead of one, two, two, which were the b's, suppose I put P1, P2, P3 in here.
I'll figure out in a minute what those numbers are.
But I just want to get the picture of what I'm doing.
If I put P1, P2, P3 in those three equations, what would be good about the three equations?
I could solve them.
A line goes through the Ps.
So the P1, P2, P3 vector, that's in the column space.
That is a combination of these columns.
It's the closest combination.
It's this picture.
See, I've got the two pictures like here's the picture that shows the points, this is a picture in a blackboard plane, here's a picture that's showing the vectors.
The vector b, which is in this case, in this example is the vector one, two, two.
The column space is in this case spanned by the -- well, you see A there.
The column space of the matrix one, one, one, one, two, three.
And this picture shows the nearest point.
There's the -- that point P1, P2, P3, which I'm going to compute before the end of this hour, is the closest point in the column space.
OK. Let me -- t I don't dare leave it any longer -- can I just compute it now.
So I want to compute -- find P. All right.
Find P. Find x, which is CD, find P and P. OK. And I really should put these little hats on to remind myself that they're the estimated the best line, not the perfect line.
OK. OK. How do I proceed?
Let's just run through the mechanics.
What's the equation for x?
The -- or x hat.
The equation for that is A transpose A x hat equals A transpose x -- A transpose b.
The most -- I'm -- will venture to call that the most important equation in statistics.
And in estimation.
And whatever you're -- wherever you've got error and noise this is the estimate that you use first.
OK.
Whenever you're fitting things by a few parameters, that's the equation to use.
OK, let's solve it.
What is A transpose A?
So I have to figure out what these matrices are.
One, one, one, one, two, three and one, one, one, one, two, three, that gives me some matrix, that gives me a matrix, what do I get out of that, three, six, six, and one and four and nine, fourteen.
OK. And what do I expect to see in that matrix and I do see it, just before I keep going with the calculation?
I expect that matrix to be symmetric.
I expect it to be invertible.
And near the end of the course I'm going to say I expect it to be positive definite, but that's a future fact about this crucial matrix, A transpose A. OK. And now let me figure A transpose b.
So let me -- can I tack on b as an extra column here, one, two, two?
And tack on the extra A transpose b is -- looks like five and one and four and six, eleven.
I think my equations are three C plus six D equals five, and six D plus fourt-six C plus fourteen D is eleven.
Can I just for safety see if I did that right?
One, one, one times one, two, two is five.
One, two, three, that's one, four and six, eleven.
Looks good.
These are my equations.
That's my -- they're called the normal equations.
I'll just write that word down because it -- so I solve them.
I solve that for C and D. I would like to -- before I solve them could I do one thing that's on the -- that's just above here?
I would like to -- I'd like to find these equations from calculus.
I'd like to find them from this minimizing thing.
So what's the first error?
The first error is what I missed by in the first equation.
C plus D minus one squared.
And the second error is what I miss in the second equation.
C plus two D minus two squared.
And the third error squared is C plus three D minus two squared.
That's my -- overall squared error that I'm trying to minimize.
OK.
So how would you minimize that?
OK, linear algebra has given us the equations for the minimum.
But we could use calculus too.
That's a function of two variables, C and D, and we're looking for the minimum.
So how do we find it?
Directly from calculus, we take partial derivatives, right, we've got two variables, C and D, so take the partial derivative with respect to C and set it to zero, and you'll get that equation.
Take the partial derivative with respect -- I'm not going to write it all out, just -- you will.
The partial derivative with respect to D, it -- you know, it's going to be linear, that's the beauty of these squares,that if I have the square of something and I take its derivative I get something And this is what I get.
linear.
So this is the derivative of the error with respect to C being zero, and this is the derivative of the error with respect to D being zero.
Wherever you look, these equations keep coming.
So now I guess I'm going to solve it, what will I do, I'll subtract, I'll do elimination of course, because that's the only thing I know how to do.
Two of these away from this would give me -- let's see, six, so would that be two Ds equals one?
Ha.
So it wasn't -- I was afraid these numbers were going to come out awful.
But if I take two of those away from that, the equation I get left is two D equals one, so I think D is a half and C is whatever back substitution gives, six D is three, so three C plus three is five, I'm doing back substitution now, right, three, can I do it in light letters, three C plus that six D is three equals five, so three C is two, so I think C is two-thirds.
One-half and two-thirds.
So the best line, the best line is the constant two-thirds plus one-half t. And I -- is my picture more or less right?
Let me write, let me copy that best line down again, two-thirds and a half.
Let me -- I'll put in the two-thirds and the half.
OK.
So what's this P1, that's the value at t equal to one.
At t equal to one, I have two-thirds plus a half, which is -- what's that, four-sixths and three-sixths, so P1, oh, I promised not to write another thing on this -- I'll erase P1 and I'll put seven-sixths.
OK. And yeah, it's above one, and e1 is one-sixth, right.
You see it all.
Right?
What's P2?
OK. At point t equal to two, where's my line here?
At t equal to two, it's two-thirds plus one, right?
That's five-thirds.
Two-thirds and t is two, so that's two-thirds and one make five-thirds.
And that's -- sure enough, that's smaller than the exact two.
And then final P3, when t is three, oh, what's two-thirds plus three-halves?
It's the same as three-halves plus two-thirds.
It's -- so maybe four-sixths and nine-sixths, maybe thirteen-sixths.
OK, and again, look, oh, look at this, OK. You have to admire the beauty of this answer.
What's this first error?
So here are the errors.
e1, e2 and e3.
OK, what was that first error, e1?
Well, if we decide the errors counting up, then it's one-sixth.
And the last error, thirteen-sixths minus the correct two is one-sixth again.
And what's this error in the middle?
Let's see, the correct answer was two, two.
And we got five-thirds and it's the other direction, minus one-third, minus two-sixths.
That's our error vector.
In our picture, in our other picture, here it is.
We just found P and e. e is this vector, one-sixth, minus two-sixths, one-sixth, and P is this guy.
Well, maybe I have the signs of e wrong, I think I have, let me fix it.
Because I would like this one-sixth -- I would like this plus the P to give the original b. I want P plus e to match b.
So I want minus a sixth, plus seven-sixths to give the correct b equal one.
OK. Now -- I'm going to take a deep breath here, and ask what do we know about this error vector e?
You've seen now this whole problem worked completely through, and I even think the numbers are right.
So there's P, so let me -- I'll write -- if I can put it down here, B is P plus e. b I believe was one, two, two.
The nearest point had seven-sixths, what were the others?
Five-thirds and thirteen-sixths.
And the e vector was minus a sixth, two-sixths, one-third in other words, and minus a sixth.
OK. Tell me some stuff about these two vectors.
Tell me something about those two vectors, well, they add to b, right, great.
OK. What else?
What else about those two vectors, the P, the projection vector P, and the error vector e. What else do you know about them?
They're perpendicular, right.
Do we dare verify that?
Can you take the dot product of those vectors?
I'm like getting like minus seven over thirty-six, can I change that to ten-sixths?
Oh, God, come out right here.
Minus seven over thirty-six, plus twenty over thirty-six, minus thirteen over thirty-six.
Thank you, God.
OK. And what else should we know about that vector?
Actually we know -- I've got to say we know even a little more.
This vector, e, is perpendicular to P, but it's perpendicular to other stuff too.
It's perpendicular not just to this guy in the column space, this is in the column space for sure.
This is perpendicular to the column space.
So like give me another vector it's perpendicular to.
Another because it's perpendicular to the whole column space, not just to this -- this particular projection that's -- that is in the column space, but it's perpendicular to other stuff, whatever's in the column space, so tell me another vector in the -- oh, well, I've written down the matrix, so tell me another vector in the column space.
Pick a nice one.
One, one, one.
That's what everybody's thinking.
OK, one, one, one is in the column space.
And this guy is supposed to be perpendicular to one, one, one.
Is it?
Sure.
If I take the dot product with one, one, one I get minus a sixth, plus two-sixths, minus a sixth, zero.
And it's perpendicular to one, two, three.
Because if I take the dot product with one, two, three I get minus one, plus four, minus three, zero again.
OK, do you see the -- I hope you see the two pictures.
The picture here for vectors and, the picture here for the best line, and it's the same picture, just -- this one's in the plane and it's showing the line, this one never did show the line, this -- in this picture, C and D never showed up.
In this picture, C and D were -- you know, they determined that line.
But the two are exactly the same.
C and D is the combination of the two columns that gives P. OK.
So that's these squares.
And the special but most important example of fitting by straight line, so the homework that's coming then Wednesday asks you to fit by straight lines.
So you're just going to end up solving the key equation.
You're going to end up solving that key equation and then P will be Ax hat.
That's it.
OK. Now, can I put in a little piece of linear algebra that I mentioned earlier, mentioned again, but I never did write?
And I've -- I should do it right.
It's about this matrix A transpose A.
There.
I was sure that that matrix would be invertible.
And of course I wanted to be sure it was invertible, because I planned to solve this system with with that matrix.
So and I announced like before -- as the chapter was just starting, I announced that it would be invertible.
But now I -- can I come back to that?
OK.
So what I said was -- that if A has independent columns, then A transpose A is invertible.
And I would like to -- first to repeat that important fact, that that's the requirement that makes everything go here.
It's this independent columns of A that guarantees everything goes through.
And think why.
Why does this matrix A transpose A, why is it invertible if the columns of A are independent?
OK, there's -- so if it wasn't invertible, I'm -- so I want to prove that.
If it isn't invertible, then what?
I want to reach -- I want to follow that -- follow that line -- of thinking and see what I come to.
Suppose, so proof.
Suppose A transpose Ax is zero.
I'm trying to prove this.
This is now to prove.
I don't like hammer away at too many proofs in this course.
But this is like the central fact and it brings in all the stuff we know.
OK.
So I'll start the proof.
Suppose A transpose Ax is zero.
What -- and I'm aiming to prove A transpose A is invertible.
So what do I want to prove now?
So I'm aiming to prove this fact.
I'll use this, and I'm aiming to prove that this matrix is invertible, OK, so if I suppose A transpose Ax is zero, then what conclusion do I want to reach?
I'd like to know that x must be zero.
I want to show x must be zero.
To show now -- to prove x must be the zero vector.
Is that right, that's what we worked in the previous chapter to understand, that a matrix was invertible when its null space is only the zero vector.
So that's what I want to show.
How come if A transpose Ax is zero, how come x must be zero?
What's going to be the reason?
Actually I have two ways to do it.
Let me show you one way.
This is -- here, trick.
Take the dot product of both sides with x.
So I'll multiply both sides by x transpose.
x transpose A transpose Ax equals zero.
I shouldn't have written trick.
That makes it sound like just a dumb idea.
Brilliant idea, I should have put.
OK.
I'll just put idea.
OK. Now, I got to that equation, x transpose A transpose Ax=0, and I'm hoping you can see the right way to -- to look at that equation.
What can I conclude from that equation, that if I have x transpose A -- well, what is x transpose A transpose Ax?
Does that -- what it's giving you?
It's again going to be putting in parentheses, I'm looking at Ax and what I seeing here?
Its transpose.
So I'm seeing here this is Ax transpose Ax.
Equaling zero.
Now if Ax transpose Ax, so like let's call it y or something, if y transpose y is zero, what does that tell me?
That the vector has to be zero, right?
This is the length squared, that's the length of the vector Ax squared, that's Ax times Ax.
So I conclude that Ax has to be zero.
Well, I'm getting somewhere.
Now that I know Ax is zero, now I'm going to use my little hypothesis.
Somewhere every mathematician has to use the hypothesis.
Right?
Now, if A has independent columns and we've -- we're at the point where Ax is zero, what does that tell us?
I could -- I mean that could be like a fill-in question on the final exam.
If A has independent columns and if Ax equals zero then what?
Please say it.
x is zero, right.
Which was just what we wanted to prove.
That -- do you see why that is?
If Ax eq- equals zero, now we're using -- here we used this was the square of something, so I'll put in little parentheses the observation we made, that was a square which is zero, so the thing has to be zero.
Now we're using the hypothesis of independent columns at the A has independent columns.
If A has independent columns, this is telling me x is in its null space, and the only thing in the null space of such a matrix is the zero vector.
OK.
So that's the argument and you see how it really used our understanding of the -- of the null space.
OK. That's great.
All right.
So where are we then?
That board is like the backup theory that tells me that this matrix had to be invertible because these columns were independent.
OK. there's one case of independent -- there's one case where the geometry gets even better.
When the -- there's one case when columns are sure to be independent.
And let me put that -- let me write that down and that'll be the subject for next time.
Columns are sure -- are certainly independent, definitely independent, if they're perpendicular.
Oh, I've got to rule out the zero column, let me give them all length one, so they can't be zero if they are perpendicular unit vectors.
Like the vectors one, zero, zero, zero, one, zero and zero, zero, one.
Those vectors are unit vectors, they're perpendicular, and they certainly are independent.
And what's more, suppose they're -- oh, that's so nice, I mean what is A transpose A for that matrix?
For the matrix with these three columns?
It's the identity.
So here's the key to the lecture that's coming.
If we're dealing with perpendicular unit vectors and the word for that will be -- see I could have said orthogonal, but I said perpendicular -- and this unit vectors gets put in as the word normal.
Orthonormal vectors.
Those are the best columns you could ask for.
Matrices with -- whose columns are orthonormal, they're perpendicular to each other, and they're unit vectors, well, they don't have to be those three, let me do a final example over here, how about one at an angle like that and one at ninety degrees, that vector would be cos theta, sine theta, a unit vector, and this vector would be minus sine theta cos theta.
That is our absolute favorite pair of orthonormal vectors.
They're both unit vectors and they're perpendicular.
That angle is ninety degrees.
So like our job next time is first to see why orthonormal vectors are great, and then to make vectors orthonormal by picking the right basis.
OK, see you.
Thanks.
OK, here's the last lecture in the chapter on orthogonality.
So we met orthogonal vectors, two vectors, we met orthogonal subspaces, like the row space and null space.
Now today we meet an orthogonal basis, and an orthogonal matrix.
So we really -- this chapter cleans up orthogonality.
And really I want -- I should use the word orthonormal.
Orthogonal is -- so my vectors are q1,q2 up to qn -- I use the letter "q", here, to remind me, I'm talking about orthogonal things, not just any vectors, but orthogonal ones.
So what does that mean?
That means that every q is orthogonal to every other q.
It's a natural idea, to have a basis that's headed off at ninety-degree angles, the inner products are all zero.
Of course if q is -- certainly qi is not orthogonal to itself.
But there we'll make the best choice again, make it a unit vector.
Then qi transpose qi is one, for a unit vector.
The length squared is one.
And that's what I would use the word normal.
So for this part, normalized, unit length for this part.
OK.
So first part of the lecture is how does having an orthonormal basis make things nice?
It certainly does.
It makes all the calculations better, a whole lot of numerical linear algebra is built around working with orthonormal vectors, because they never get out of hand, they never overflow or underflow.
And I'll put them into a matrix Q, and then the second part of the lecture will be suppose my basis, my columns of A are not orthonormal.
How do I make them so?
And the two names associated with that simple idea are Graham and Schmidt.
So the first part is we've got a basis like this.
Let's put those into the columns of a matrix.
So a matrix Q that has -- I'll put these orthonormal vectors, q1 will be the first column, qn will be the n-th column.
And I want to say, I want to write this property, qi transpose qj being zero, I want to put that in a matrix form.
And just the right thing is to look at Q transpose Q.
So this chapter has been looking at A transpose A.
So it's natural to look at Q transpose Q.
And the beauty is it comes out perfectly.
Because Q transpose has these vectors in its rows, the first row is q1 transpose, the nth row is qn transpose.
So that's Q transpose.
And now I want to multiply by Q.
That has q1 along to qn in the columns.
That's Q.
And what do I get?
You really -- this is the first simplest most basic fact, that how do orthonormal vectors, orthonormal columns in a matrix, what happens if I compute Q transpose Q?
Do you see it?
If I take the first row times the first column, what do I get?
A one.
If I take the first row times the second column, what do I get?
Zero.
That's the orthogonality.
The first row times the last column is zero.
And so I'm getting ones on the diagonal and I'm getting zeroes everywhere else.
I'm getting the identity matrix.
You see how that's -- it's just like the right calculation to do.
If you have orthonormal columns, and the matrix doesn't have to be square here.
We might have just two columns.
And they might have four, lots of components.
So but they're orthonormal, and when we do Q transpose times Q, that Q transpose times Q or A transpose A just asks for all those dot products.
Rows times columns.
And in this orthonormal case, we get the best possible answer, the identity.
OK, so this is -- so I mean now we have a new bunch of important matrices.
What have we seen previously?
We've seen in the distant past we had triangular matrices, diagonal matrices, permutation matrices, that was early chapters, then we had row echelon forms, then in this chapter we've already seen projection matrices, and now we're seeing this new class of matrices with orthonormal columns.
That's a very long expression.
I sorry that I can't just call them orthogonal matrices.
But that word orthogonal matrices -- or maybe I should be able to call it orthonormal matrices, why don't we call it orthonormal -- I mean that would be an absolutely perfect name.
For Q, call it an orthonormal matrix because its columns are orthonormal.
OK, but the convention is that we only use that name orthogonal matrix, we only use this -- this word orthogonal, we don't even say orthonormal for some unknown reason, matrix when it's square.
So in the case when this is a square matrix, that's the case we call it an orthogonal matrix.
And what's special about the case when it's square?
When it's a square matrix, we've got its inverse, so -- so in the case if Q is square, then Q transpose Q equals I tells us -- let me write that underneath -- tells us that Q transpose is Q inverse.
There we have the easy to remember property for a square matrix with orthonormal columns.
That -- I need to write some examples down.
Let's see.
Some examples like if I take any -- so examples, let's do some examples.
Any permutation matrix, let me take just some random permutation matrix.
Permutation Q equals let's say oh, make it three by three, say zero, zero, one, one, zero, zero, zero, one, zero.
OK. That certainly has unit vectors in its columns.
Those vectors are certainly perpendicular to each other.
And if I -- and so that's it.
That makes it a Q.
And -- if I took its transpose, if I multiplied by Q transpose, shall I do that -- and let me stick in Q transpose here.
Just to do that multiplication once more, transpose it'll put the -- make that into a column, make that into a column, make that into a column.
And the transpose is also -- another Q.
Another orthonormal matrix.
And when I multiply that product I get I. OK, so there's an example.
And actually there's a second example.
But those are real easy examples, right, I mean to get orthogonal columns by just putting ones in different places is like too easy.
So let me keep going with examples.
So here's another simple example.
Cos theta sine theta, there's a unit vector, oh, let me even take it, well, yeah.
Cos theta sine theta and now the other way I want sine theta cos theta.
But I want the inner product to be zero.
And if I put a minus there, it'll do it.
So that's -- unit vector, that's a unit vector.
And if I take the dot product, I get minus plus zero.
OK. For example Q equals say one, one, one, minus one, is that an orthogonal matrix?
I've got orthogonal columns there, but it's not quite an orthogonal matrix.
How shall I fix it to be an orthogonal matrix?
Well, what's the length of those column vectors, the dot product with themselves is -- right now it's two, right, the -- the length squared.
The length squared would be one plus one would be two, the length would be square root of two, so I better divide by square root of two.
OK.
So there's a -- there now I have got an orthogonal matrix, in fact, it's this one -- when theta is pi over four.
The cosines and well almost, I guess the minus sine is down there, so maybe, I don't know, maybe minus pi over four or something.
OK. Let me do one final example, just to show that you can get bigger ones.
Q equals let me take that matrix up in the corner and I'll sort of repeat that pattern, repeat it again, and then minus it down here.
That's one of the world's favorite orthogonal matrices.
I hope I got it right, is -- can you see whether -- if I take the inner product of one column with another one, let's see, if I take the inner product of that column with that I have two minuses and two pluses, that's good.
When I take the inner product of that with that I have a plus and a minus, a minus and a plus.
Good.
I think it all works out.
And what do I have to divide by now?
To make those into unit vectors.
Right now the vector one, one, one, one has length two.
Square root of four.
So I have to divide by two to make it unit vector, so there's another.
That's my entire array of simple examples.
This construction is named after a guy called Adhemar and we know how to do it for two, four, sixteen, sixty-four and so on, but we -- nobody knows exactly which size matrices have -- which size -- which sizes allow orthogonal matrices of ones and minus ones.
So Adhemar matrix is an orthogonal matrix that's got ones and minus ones, and a lot of ones -- some we know, some other sizes, there couldn't be a five by five I think.
But there are some sizes that nobody yet knows whether there could be or can't be a matrix like that.
OK. You see those orthogonal matrices.
Now let me ask what -- why is it good to have orthogonal matrices?
What calculation is made easy?
If I have an orthogonal matrix.
And -- let me remember that the matrix could be rectangular.
Shall I put down -- I better put a rectangular example down.
So the -- these were all square examples.
Can I put down just -- a rectangular one just to be sure that we realize that this is possible.
let's help me out.
Let's see, if I put like a one, two, two and a minus two, minus one, two.
That's -- a matrix -- oh its columns aren't normalized yet.
I always have to remember to do that.
I always do that last because it's easy to do.
What's the length of those columns?
So if I wanted them -- if I wanted them to be length one, I should divide by their length, which is -- so I'd look at one squared plus two squared plus two squared, that's one and four and four is nine, so I take the square root and I need to divide by three.
OK.
So there is -- well, without that, I've got one orthonormal vector.
I mean just one unit vector.
Now put that guy in.
Now I have a basis for the column space for a two-dimensional space, an orthonormal basis, right?
These two columns are orthonormal, they would be an orthonormal basis for this two-dimensional space that they span.
Orthonormal vectors by the way have got to be independent.
It's easy to show that orthonormal vectors since they're headed off all at ninety degrees there's no combination that gives zero.
Now if I wanted to create now a third one, I could either just put in some third vector that was independent and go to this Graham-Schmidt calculation that I'm going to explain, or I could be inspired and say look, that -- with that pattern, why not put a one in there, and a two in there, and a two in there, and try to fix up the signs so that they worked.
Hmm.
I don't know if I've done this too brilliantly.
Let's see, what signs, that's minus, maybe I'd make a minus sign there, how would that be?
Yeah, maybe that works.
I think that those three columns are orthonormal and they -- the beauty of this -- this is the last example I'll probably find where there's no square root, the -- the punishing thing in Graham-Schmidt, maybe we better know that in advance, is that because I want these vectors to be unit vectors, I'm always running into square roots.
I'm always dividing by lengths.
And those lengths are square roots.
So you'll see as soon as I do a Graham-Schmidt example, square roots are going to show up.
But here are some examples where we did it without any square root.
OK. OK.
So -- so great.
Now next question is what's the good of having a Q?
What formulas become easier?
Suppose I want to project, so suppose Q -- suppose Q has orthonormal columns.
I'm using the letter Q to mean this, I'll write it this one more time, but I always mean when I write a Q, I always mean that it has orthonormal columns.
So suppose I want to project onto its column space.
So what's the projection matrix?
What's the projection matrix is I project onto a column space?
OK, that gives me a chance to review the projection section, including that big formula, which used to be -- those four As in a row, but now it's got Qs, because I'm projecting onto the column space of Q, so do you remember what it was?
It's Q Q transpose Q inverse Q transpose.
That's my four Qs in a row.
But what's good here?
What -- what makes this formula nice if I'm projecting onto a column space when I have orthonormal basis for that space?
What makes it nice is this is the identity.
I don't have to do any inversion.
I just get Q Q transpose.
So Q Q transpose is a projection matrix.
Oh, I can't help -- I can't resist just checking the properties, what are the properties of a projection matrix?
There are two properties to know for any projection matrix.
And I'm saying that this is the right projection matrix when we've got this orthonormal basis in the columns.
OK.
So there's the projection matrix.
Suppose the matrix is square.
First just tell me first this extreme case.
If my matrix is square and it's got these orthonormal columns, then what's the column space?
If I have a square matrix and I have independent columns, and even orthonormal columns, then the column space is the whole space, right?
And what's the projection matrix onto the whole space?
The identity matrix.
If I'm projecting in the whole space, every vector B is right where it's supposed to be and I don't have to move it by projection.
So this would be -- I'll put in parentheses this is I if Q is square.
Well that we said that already.
If Q is square, that's the case where Q transpose is Q inverse, we can put it on the right, we can put it on the left, we always get the identity matrix, if it's square.
But if it's not a square matrix then it's not -- we don't get the identity matrix.
We have Q Q transpose, and just again what are those two properties of a projection matrix?
First of all, it's symmetric.
OK, no problem, that's certainly a symmetric So what's that second property of a projection?
matrix.
That if you project and project again you don't move the second time.
So the other property of a projection matrix should be that Q Q transpose twice should be the same as Q Q transpose once.
That's projection matrices.
And that property better fall out right away because from the fact we know about orthonormal matrices, Q transpose Q is I. OK, you see it.
In the middle here is sitting Q Q t- Q transpose Q, sorry, that's what I meant to say, Q transpose Q is I.
So that's sitting right in the middle, that cancels out, to give the identity, we're left with one Q Q transpose, and we're all set.
OK.
So this is the projection matrix -- all the equation -- all the messy equations of this chapter become trivial when our matrix -- when we have this orthonormal basis.
I mean what do I mean by all the equations?
Well, the most important equation was the normal equation, do you remember old A transpose A x hat equals A transpose b?
But now -- now A is Q.
Now I'm thinking I have Q transpose Q X hat equals Q transpose b.
And what's good about that?
What's good is that matrix on the left side is the identity.
The matrix on the left is the identity, Q transpose Q, normally it isn't, normally it's that matrix of inner products and you've to compute all those dopey inner products and -- and -- and solve the system.
Here the inner products are all one or zero.
This is the identity matrix.
It's gone.
And there's the answer.
There's no inversion involved.
Each component of x is a Q times b.
What that equation is saying is that the i-th component is the i-th basis vector times b.
That's -- probably the most important formula in some major parts of mathematics, that if we have orthonormal basis, then the component in the -- in the i-th, along the i-th -- the projection on the i-th basis vector is just qi transpose b.
That number x that we look for is just a dot product.
OK. OK, so I'm ready now for the sort of like second half of the lecture.
Where we don't start with an orthogonal matrix, orthonormal vectors.
We just start with independent vectors and we want to make them orthonormal.
So I'm going to -- can I do that now?
Now here comes Graham-Schmidt.
So -- Graham-Schmidt.
So this is a calculation, I won't say -- I can't quite say it's like elimination, because it's different, our goal isn't triangular anymore.
With elimination our goal was make the matrix triangular.
Now our goal is make the matrix orthogonal.
Make those columns orthonormal.
So let me start with two columns.
So I start with vectors a and b.
And they're just like -- here, let me draw them.
Here's a.
Here's b.
For example.
A isn't specially horizontal, wasn't meant to be, just a is one vector, b is another.
I want to produce those two vectors, they might be in twelve-dimensional space, or they might be in two-dimensional space.
They're independent, anyway.
So I better be sure I say that.
I start with independent vectors.
And I want to produce out of that q 1 and q2, I want to produce orthonormal vectors.
And Graham and Schmidt tell me how.
OK. Well, actually you could tell me how, we don't need -- frankly, I don't know -- there's only one idea here, if Graham had the idea, I don't know what Schmidt did.
But OK.
So you'll see it.
We don't need either of them, actually.
OK, so what I going to do.
I'll take that -- this first guy.
OK. Well, he's fine.
That direction is fine except -- yeah, I'll say OK, I'll settle for that direction.
So I'm going to -- I'm going to get, so what I going to -- my goal is I'm going to get orthogonal vectors and I'll call those capital A and B.
So that's the key step is to get from any two vectors to two orthogonal vectors.
And then at the end, no problem, I'll get orthonormal vectors, how will -- what will those will be my qs, q1 and q2, and what will they be?
Once I've got A and B orthogonal, well, look, it's no big deal -- maybe that's what Schmidt did, he, brilliant Schmidt, thought OK, divide by the length, all right.
That's Schmidt's contribution.
OK.
But Graham had a little more thinking to do, right?
We haven't done Graham's part.
This part except OK, I'm happy with A, A can be A.
That first direction is fine.
Why should -- no complaint about that.
The trouble is the second direction is not fine.
Because it's not orthogonal to the first.
I'm looking for a vector that's -- starts with B, but makes it orthogonal to A.
What's the vector?
How do I do that?
How do I produce from this vector a piece that's orthogonal to this one?
And the -- remember these vectors might be in two dimensions or they might be in twelve dimensions.
I'm just looking for the idea.
So what's the idea?
Where did we have orthogonal -- a vector showing up that was orthogonal to this guy?
Well, that was the first basic calculation of the whole chapter.
We -- we did a projection and the projection gave us this part, which was the part in the A direction.
Now, the part we want is the other part, the e part.
This part.
This is going to be our -- that guy is that guy.
This is our vector B.
That gives us that ninety-degree angle.
So B is you could say -- B is really what we previously called e. The error vector.
And what is it?
I mean what do I -- what is B here?
A is A, no problem.
B is -- OK, what's this error piece?
Do you remember?
It's I start with the original B and I take away what?
Its projection, this P. This -- the vector B, this error vector, is the original vector removing the projection.
So instead of wanting the projection, now that's what I want to throw away.
I want to get the part that's perpendicular.
And there will be a perpendicular part, it won't be zero.
Because these vectors were independent, so B -- if B was along the direction of A, then if the original B and A were in the same direction, then I'm -- I've only got one direction.
But here they're in two independent directions and all I'm doing is getting that guy.
So what's its formula?
What's the formula for that if -- I want to subtract the projection, so do you remember the projection?
It's some multiple of A and what's that multiple?
It's -- it's that thing we called x in the very very first lecture on this chapter.
There's an A transpose A in the bottom and there's an A transpose B, isn't that it?
I think that's Graham's formula.
Or Graham-Schmidt.
No, that's Graham.
Schmidt has got to divide the whole thing by the length, so he -- his formula makes a mess which I'm not willing to write down.
So let's just see that what I saying here?
I'm saying that this vector is perpendicular to A.
That these are orthogonal.
A is perpendicular to B.
Can you check that?
How do you see that yes, of course, we -- our picture is telling us, yes, we did it right.
How would I check that this matrix is perpendicular to A?
I would multiply by A transpose and I better get zero, right?
I should check that.
A transpose B should come out zero.
So this is A transpose times -- now what did we say B was?
We start with the original B, and we take away this projection, and that should come out zero.
Well, here we get an A transpose B minus -- and here is another A transpose B, and the -- and it's an A transpose A over A transpose A, a one, those cancel, and we do get zero.
Right.
Now I guess I can do numbers in there.
But I have to take a third vector to be sure we've got this system down.
So now I have to say if I have independent vectors A, B and C, I'm looking for orthogonal vectors A, B and capital C, and then of course the third guy will just be C over its length, the unit vector.
So this is now the problem.
I got B here.
I got A very easily.
And now -- if you see the idea, we could figure out a formula for C. So now that -- so this is like a typical homework quiz problem.
I give you two vectors, you do this, I give you three vectors, and you have to make them orthonormal.
So you do this again, the first vector's fine, the second vector is perpendicular to the first, and now I need a third vector that's perpendicular to the first one and the second one.
Right?
Tthis is the end of a -- the lecture is to find this guy.
Find this vector -- this vector C, that's perpendicular we n- at this point we know A and B.
But C, the little c that we're given, is off in some -- it's got to come out of the blackboard to be independent, so -- so can I sort of draw off -- off comes a c somewhere.
I don't know, where I going to put the darn thing?
Maybe I'll put it off, oh, I don't know, like that somehow, C, little c. And I already know that perpendicular direction, that one and that one.
So now what's the idea?
Give me the Graham-Schmidt formula for C. What is this C here?
Equals what?
What I going to do?
I'll start with the given one.
As before.
Right?
I start with the vector I'm given.
And what do I do with it?
I want to remove out of it, I want to subtract off, so I'll put a minus sign in, I want to subtract off its components in the A, capital A and capital B directions.
I just want to get those out of there.
Well, I know how to do that.
I did it with B.
So I'll just -- so let me take away -- what if I do this?
What have I done?
I've got little c and what have I subtracted from it?
Its component, its projection if you like, in the A direction.
And now I've got to subtract off its component B transpose C over B transpose B, that multiple of B, is its component in the B direction.
And that gives me the vector capital C that if anything is -- if there's any justice, this C should be perpendicular to A and it should be perpendicular to B.
And the only thing it's -- hasn't got is unit vector, so we divide by its length to get that too.
OK. Let me do an example.
Can I -- I'll make my life easy, I'll just take two vectors.
So let me do a numerical example.
If I'll give you two vectors, you give me back the Graham-Schmidt orthonormal basis, and we'll see how to express that in matrix form.
OK.
So let me give you the two vectors.
So I'll take the vector A equals let's say one, one, one, why not?
And B equals let's say one, zero, two, OK?
I didn't want to cheat and make them orthogonal in the first place because then Graham-Schmidt wouldn't be needed.
OK.
So those are not orthogonal.
So what is capital A?
Well that's the same as big A.
That was fine.
What's B?
So B is this b -- is the original B, and then I subtract off some multiple of the A.
And what's the multiple?
What goes in here?
B -- here's the A -- this is the -- this is the little b, this is the big A, also the little a, and I want to multiply it by that right -- that right ratio, which has A transpose A, here's my ratio.
I'm just doing this.
So it's A transpose B, what is A transpose B, it looks like three.
And what is A -- oh, my -- what's A transpose A?
Three.
I'm sorry.
I didn't know that was going to happen.
OK.
But it happened.
Why should we knock it?
OK.
So do you see it all right?
That's A transpose B, there's A transpose A, that's the fraction, so I take this away, and I get one take away one is a zero, zero minus this one is a minus one, and two minus the one is a one.
OK. And what's this vector that we finally found?
This is B.
And how do I know it's right?
How do I know I've got a vector I want?
I check that B is perpendicular to -- that A and B are perpendicular.
That A is perpendicular to B.
Just look at that.
That one -- the dot product of that with that is zero.
OK.
So now what is my q1 and q2?
Why don't I put them in a matrix?
Of course.
Since I'm always putting these -- so the Q, I'll put the q1 and the q2 in a matrix.
And what are they?
Now when I'm writing q-s I'm supposed to make things normalized.
I'm supposed to make things unit vectors.
So I'm going to take that A but I'm going to divide it by square root of three.
And I'm going to take this B but I'm going to divide it by square root of two to make it a unit vector, and there is my matrix.
That's my matrix with orthonormal columns coming from Graham-Schmidt and it sort of it -- it came from the original one, one, one, one, zero, two, right?
That was my original guys.
These were the two I started with.
These are the two that I'm happy to end with.
Because those are orthonormal.
So that's what Graham-Schmidt did.
It -- well, tell me about the column spaces of these matrices.
How is the column space of Q related to the column space of A?
So I'm always asking you things like this, and that makes you think, OK, the column space is all combinations of the columns, it's that plane, right?
I've got two vectors in three-dimensional space, their column space is a plane, the column space of this matrix is a plane, what's the relation between the planes?
Between the two column spaces?
They're one and the same, right?
It's the same column space.
All I'm taking is here this B thing that I computed, this B thing that I computed is a combination of B and A, and A was little A, so I'm always working here with this in the same space.
I'm just like getting ninety-degree angles in there.
Where my original column space had a perfectly good basis, but it wasn't as good as this basis, because it wasn't orthonormal.
Now this one is orthonormal, and I have a basis then that -- so now projections, all the calculations I would ever want to do are -- are a cinch with this orthonormal basis.
One final point.
One final point in this chapter.
And it's -- just like elimination.
We learned how to do elimination, we know all the steps, we can do it.
But then I came back to it and said look at it as a matrix in matrix language and elimination gave me -- what was elimination in matrix language?
I'll just put it up there.
A was LU.
That was matrix, that was elimination.
Now, I want to do the same for Graham-Schmidt.
Everybody who works in linear algebra isn't going to write out the columns are orthogonal, or orthonormal.
And isn't going to write out these formulas.
They're going to write out the connection between the matrix A and the matrix Q.
And the two matrices have the same column space, but there's some -- some matrix is taking the -- and I'm going to call it R, so A equals QR is the magic formula here.
It's the expression of Graham-Schmidt.
And I'll -- let me just capture that.
So that's the -- my final step then is A equal QR.
Maybe I can squeeze it in here.
So A has columns, let's say a1 and a2.
Let me suppose n is two, just two vectors.
OK.
So that's some combination of q1 and q2.
And times some matrix R. They have the same column space.
This is just -- this matrix just includes in it whatever these numbers like three over three and one over square root of three and one over square root of two, probably that's what it's got.
One over square root of three, one over square root of two, something there, but actually it's got a zero there.
So the main point about this A equal QR is this R turns out to be upper triangular.
It turns out that this zero is upper triangular.
We could see why.
Let me see, I can put in general formulas for what these This I think in here should be the inner product of a1 with q1.
are.
And this one should be the -- the inner product of a1 with q2.
And that's what I believe is zero.
This will be something here, and this will be something here with inner -- a1 transpose q2, sorry a2 transpose q1 and a2 transpose q2.
But why is that guy zero?
Why is a1 q2 zero?
That's the key to this being -- this R here being upper triangular.
You know why a1q2 is zero, because a1 -- that was my -- this was really a and b here.
This was really a and b.
So this is a transpose q2.
And the whole point of Graham-Schmidt was that we constructed these later q-s to be perpendicular to the earlier vectors, to the earlier -- all the earlier vectors.
So that's why we get a triangular matrix.
The -- result is extremely satisfactory.
That if I have a matrix with independent columns, the Graham-Schmidt produces a matrix with orthonormal columns, and the connection between those is a triangular matrix.
That last point, that the connection is a triangular matrix, please look in the book, you have to see that one more time.
OK.
Thanks, that's great.
OK, this lecture is like the beginning of the second half of this is to prove.
this course because up to now we paid a lot of attention to rectangular matrices.
Now, concentrating on square matrices, so we're at two big topics.
The determinant of a square matrix, so this is the first lecture in that new chapter on determinants, and the reason, the big reason we need the determinants is for the Eigen values.
So this is really determinants and Eigen values, the next big, big chunk of 18.06.
OK, so the determinant is a number associated with every square matrix, so every square matrix has this number associated with called the, its determinant.
I'll often write it as D E T A or often also I'll write it as, A with vertical bars, so that's going to mean the determinant of the matrix.
That's going to mean this one, like, magic number.
Well, one number can't tell you what the whole matrix was.
But this one number, just packs in as much information as possible into a single number, and of course the one fact that you've seen before and we have to see it again is the matrix is invertible when the determinant is not zero.
The matrix is singular when the determinant is zero.
So the determinant will be a test for invertibility, but the determinant's got a lot more to it than that, so let me start.
OK, now the question is how to start.
Do I give you a big formula for the determinant, all in one gulp?
I don't think so!
That big formula has got too much packed in it.
I would rather start with three properties of the determinant, three properties that it has.
And let me tell you property one.
The determinant of the identity is one.
OK.
I...
I wish the other two properties were as easy to tell you as that.
But actually the second property is pretty straightforward too, and then once we get the third we will actually have the determinant.
Those three properties define the determinant and we can -- then we can figure out, well, what is the determinant?
What's a formula for it?
Now, the second property is what happens if you exchange two rows of a matrix.
What happens to the determinant?
So, property two is exchange rows, reverse the sign of the determinant.
A lot of plus and minus signs.
I even wrote here, "plus and minus signs," because this is, like, that's what you have to pay attention to in the formulas and properties of determinants.
So that -- you see what I mean by a property here?
I haven't yet told you what the determinant is, but whatever it is, if I exchange two rows to get a different matrix that reverses the sign of the determinant.
And so now, actually, what matrices do we now know the determinant of?
From one and two, I now know the determinant.
Well, I certainly know the determinant of the identity matrix and now I know the determinant of every other matrix that comes from row exchanges from the identities still.
So what matrices have I gotten at this point?
The permutations, right.
At this point I know every permutation matrix, so now I'm saying the determinant of a permutation matrix is one or minus one.
One or minus one, depending whether the number of exchanges was even or the number of exchanges was odd.
So this is the determinant of a permutation.
Now, P is back to standing for permutation.
OK. if I could carry on this board, I could, like, do the two-by-two's.
So, property one tells me that this two-by-two matrix.
Oh, I better write absolute -- I mean, I'd better write vertical bars, not brackets, for that determinant.
Property one said, in the two-by-two case, that this matrix has determinant one.
Property two tells me that this matrix has determinant -- what?
Negative one.
This is, like, two-by-twos.
Now, I finally want to get -- well, ultimately I want to get to, the formula that we all know.
Let me put that way over here, that the determinant of a general two-by-two is ad-bc.
OK.
I'm going to leave that up, like, as the two by two case I'm down to the product of the diagonal and if I transpose, that we already know, so that every property, I can, like, check that it's correct for two-by-twos.
But the whole point of these properties is that they're going to give me a formula for n-by-n. That's the whole point.
They're going to give me this number that's a test for invertibility and other great properties for any size matrix.
OK, now you see I'm like, slowing down because property three is the key property.
Property three is the key property and can I somehow describe it -- maybe I'll separate it into 3A I said that if you do a row exchange, the determinant and 3B.
Property 3A says that if I multiply one of the rows, say the first row, by a number T -- I'm going to erase that.
That's, like, what I'm headed for but I'm not there yet.
It's the one we know and you'll see that it's checked out by each property.
OK, so this is for any matrix.
For any matrix, if I multiply one row by T and leave the other row or other n-1 rows alone, what happens to the determinant?
The factor T comes out.
It's T times this determinant.
That's not hard.
I shouldn't have made a big deal out of property 3A, and 3B is that, if is, is if I keep -- I'm always keeping this second row the same, or that last n-1 rows are all staying the same.
I'm just working -- I'm just looking inside the first row and if I have an a+a' there and a b+b' there -- sorry, I didn't.
Ahh.
Why don't -- I'll use an eraser, do it right.
b+b' there.
You see what I'm doing?
This property and this property are about linear combinations, of the first row only, leaving the other rows unchanged.
They'll copy along.
Then, then I get the sum -- this breaks up into the sum of this determinant and this one.
I'm putting up formulas.
Maybe I can try to say it in words.
The determinant is a linear function.
It behaves like a linear function of first row if all the other rows stay the same.
I not saying that -- let me emphasize.
I not saying that the determinant of A plus B is determinant of A plus determinant of B. I not saying that.
I'd better -- can I -- how do I get it onto tape that I'm not saying that?
You see, this would allow all the rows -- you know, A to have a bunch of rows, B to have a bunch of rows.
That's not the linearity I'm after.
I'm only after linearity in each row.
Linear for each row.
Well, you may say I only talked about the first row, but I claim it's also linear in the second row, if I had this -- but not, I can't, I can't have a combination in both first and second.
If I had a combination in the second row, then I could use rule two to put it up in the first row, use my property and then use rule two again to put it back, so each row is OK, not only the first row, but each row separately.
OK, those are the three properties, and from those properties, so that's properties one, two, three.
From those, I want to get all -- I'm going to learn a lot more about the determinant.
Let me take an example.
What would I like to learn?
I would like to learn that -- so here's our property four.
Let me use the same numbering as here.
Property four is if two rows are equal, the determinant is zero.
OK, so property four.
Two equal rows lead to determinant equals zero.
Right.
Now, of course I can -- in the two-by-two case I can check, sure, the determinant of ab ab comes out zero.
But I want to see why it's true for n-by-n.
Suppose row one equals row three for a seven-by-seven matrix.
So two rows are the same in a big matrix.
And all I have to work with is these properties.
The exchange property, which flips the sign, and the linearity property which works in each row separately.
OK, can you see the reason?
How do I get this one out of properties one, two, three?
Because -- that's all I have to work with.
Everything has to come from properties one, two, three.
OK, so suppose I have a matrix, and two rows are even.
How do I see that its determinant has to be zero from these properties?
I do an exchange.
Property two is just set up for this.
Use property two.
Use exchange -- exchange rows.
Exchange those rows, and I get the same matrix.
Of course, because those rows were equal.
So the determinant didn't change.
But on the other hand, property two says that the sign did change.
So the -- so I, I have a determinant whose sign doesn't change and does change, and the only possibility then is that the determinant is zero.
You see the reasoning there?
Straightforward.
Property two just told us, hey, if we've got two equal rows we.
we've got a zero determinant.
And of course in our minds, that matches the fact that if I have two equal rows the matrix isn't invertible.
If I have two equal rows, I know that the rank is less changes sign.
than n. OK, ready for property five.
Now, property five you'll recognize as P. It says that the elimination step that I'm always doing, or U and U transposed, when they're triangular,4 subtract a multiple, subtract some multiple l times row one from another row, row k, let's say.
You remember why I did that.
In elimination I'm always choosing this multiplier so as to produce zero in that position.
What I -- way, way back in property two,4 Or row I from row k, maybe I should just make very clear that there's nothing special about row one here.
OK, so that, you can see why I want that who cares?
one, because that will allow me to start with this full matrix whose determinant I don't know, and I can do elimination and clean it out.
I can get zeroes below the diagonal by these elimination steps and the point is that the determinant, the determinant doesn't change.
So all those steps of elimination are OK not changing the determinant.
In our language in the early chapter the determinant of A is So if I do seven row exchanges, the determinant changes sign, going to be the same as the determinant of U, the upper triangular one.
It just has the pivots on the diagonal.
That's why we'll want this property.
OK, do you see where that property's coming from?
Let me do the two-by-two case.
Let me do the two-by-two case here.
So, I'll keep property five going along.
So what I doing?
I'm going to keep -- I'm going to have ab cd, but I'm going to subtract l times the first row from the second row.
And that's the determinant and of OK, that's not -- I didn't put in every comma and, course I can multiply that out and figure out, sure enough, ad-bc is there and this minus ALB plus ALB cancels out, but I just cheated, right?
I've got to use the properties.
So what property?
Well, of course, this is a com -- I'm keeping the first row the same and the second row has a c and a d, and then there's the determinant of the A and the B, and the minus LA, and the minus LB.
So what property was that?
3B.
I kept one row the same and I had a combination in the second, in the other row, and I just separated it out.
OK, so that's property 3.
That's by number 3, 3B if you like.
OK, now use 3A.
How do you use 3A, which says I can factor out an l, I can factor out a minus l here.
I can factor a minus l out from this row, no problem.
That was 3A.
So now I've used property three and now I'm ready for the kill.
Property four says that this determinant is zero, has two equal rows.
You see how that would always work?
I subtract a multiple of one row from another one.
It gives me a combination in row k of the old row and l times this copy of the higher row, and then if -- since I have two equal rows, that's zero, so the determinant after elimination is the same as before.
Good.
OK. Now, let's see -- if I rescue my glasses, I can see what's property six.
Oh, six is easy, plenty of space.
Row of zeroes leads to determinant of A equals zero.
A complete row of zeroes.
So I'm again, this is like, something I'll use in the singular case.
Actually, you can look ahead to why I need these properties.
So I'm going to use property five, the elimination, use this stuff to say that this determinant is the same as that determinant and I'll produce a zero there.
But what if I also produce a zero there?
What if elimination gives a row of zeroes?
That, that used to be my test for, mmm, singular, not invertible, rank two -- rank less than N, and now I'm seeing it's also gives determinant zero.
How do I get that one from the previous properties?
'Cause I -- this is not a new law, this has got to come from the old ones.
So what shall I do?
Yeah, use -- that's brilliant.
If you use 3A with T equals zero.
Right.
So I have this zero zero cd, and I'm trying to show that that determinant is zero.
triangular matrices, l and l transposed, OK, so the zero is the same is -- five, can I take T equals five, just to, like, pin it down?
I'll multiply this row by five.
Five, well then, five of this should -- if I, if there's a factor five in that, you see what -- so this is property 3A, with taking T as five.
If I multiply a row by five, out comes a five.
So why I doing this?
Oh, because that's still zero zero, right?
So that's this determinant equals five times this determinant, and the determinant has to be zero.
I think I didn't do that the very best way.
You were, yeah, you had the idea better.
Multiply, yeah, take T equals zero.
Was that your idea?
Take T equals zero in rule 3B.
If T is zero in rule 3B, and I bring the camera back to rule 3B -- sorry.
If T is zero, then I have a zero zero there and the determinant is zero.
OK, one way or another, a row of zeroes means zero determinant.
OK, now I have to get serious.
The next properties are the ones that we're building up to.
OK, so I can do elimination.
I can reduce to a triangular matrix and now what's the determinant of that triangular matrix?
OK, so they had to wait until the last minute.
Suppose, suppose I -- all right, rule seven.
So suppose my matrix is now triangular.
So it's got -- so I even give these the names of the pivots, d1, d2, to dn, and stuff is up here, I don't know what that is, but what I do know is this is all zeroes.
That's all zeroes, and I would like to know the determinant, because elimination will get me to this.
So once I'm here, what's the determinant then?
Let me use an eraser to get those, get that vertical bar again, so that I'm taking the determinant of U so that, so, what is the determinant of an upper triangular matrix?
Do you know the answer?
It's just the product of the d's.
for it.
The -- these things that I don't even put in letters for, because they don't matter, the determinant is d1 times d2 times dn.
If I have a triangular matrix, then the diagonal is all I have to work with.
And that's, that's telling us then.
We now have the way that MATLAB, any reasonable software, would compute a determinant.
If I have a matrix of size a hundred, the way I would actually compute its determinant would be elimination, make it triangular, multiply the pivots together, but it -- would it be possible t- to produce the same matrix the product of the pivots, the product of pivots.
Now, there's always in determinants a plus or minus and cross every T in that proof, but that's really the sign to remember.
Where -- where does that come into this rule?
Could it be, could the determinant be minus the product of the pivots?
The determinant is d1, d2, to dn.
No doubt about that.
But to get to this triangular form, we may have had to do a row exchange, so, so this -- it's the product of the pivots if there were no row exchanges.
If, if SLU code, the simple LU code, the square LU went right through.
If we had to do some row exchanges, then we've got to watch plus or minus.
OK, but though -- this law is simply that.
OK, how do I prove that?
Let's see, let me suppose that d's are not zeroes.
The pivots are not zeroes.
And tell me, how do I show that none of this upper stuff makes any difference?
How do I get zeroes there?
By elimination, right?
I just multiply this row by the right number, subtract from that row, kills that.
I multiply this row by the right number, kills that, by the right number, kills that.
Now, you kill every one of these off-diagonal terms, no problem and I'm just left with the diagonal.
Well, strictly speaking, I still have to figure out why is, for a diagonal matrix now, why is that the right determinant?
I mean, we sure hope it is, but why?
I have to go back to properties one, two, three.
Why is -- now that the matrix is suddenly diagonal, how do I know that the determinant is just a product of That's my proof, really, that once I've got those diagonal entries?
Well, what I going to use?
I'm going to use property 3A, is that right?
I'll factor this, I'll factor this, I'll factor that d1 out and have one and have the first row will be that.
And then I'll factor out the d2, shall I shall I put the d2 here, and the second row will look like that, and so on.
So I've factored out all the d's and what I left with?
The identity.
And what rule do I finally get to use?
Rule one.
Finally, this is the point where rule one finally chips in and says that this determinant is one, so it's the product of the d's.
So this was rules five, to do elimination, 3A to factor out the D's, and, and our best friend, rule one.
And possibly rule two, the exchanges may have been needed also.
OK. Now I guess I have to consider also the case if some d is zero, because I was assuming I could use the d's to clean out and make a diagonal, but what if -- what if one of those diagonal entries is zero?
Well, then with elimination we know that we can get a row of zeroes, and for a row of zeroes I'm using rule six, the determinant is zero, and that's right.
So I can check the singular case.
In fact, I can now get to the key point that determinant of A is zero, exactly when, exactly when A is singular.
And otherwise is not singular, so that the determinant is a fair test for invertibility or non-invertibility.
So, I must be close to that because I can take any matrix and get there.
Do I -- did I have anything to say?
The, the proofs, it starts by saying by elimination go from A to U. Oh, yeah.
Actually looks to me like I don't -- haven't said anything brand-new here, that, that really, I've got this, because let's just remember the By elimination, I can go from the original A to reason.
Well, OK, now suppose the matrix is U. singular.
If the matrix is singular, what happens?
Then by elimination I get a row of zeroes and therefore the determinant is zero.
And if the matrix is not singular, I don't get zero, so maybe -- do you want me to put this, like, in two parts?
The determinant of A is not zero when A is invertible.
Because I've already -- in chapter two we figured out when is the matrix invertible.
It's invertible when elimination produces a full set of pivots and now, and we now, we know the determinant is the product of those non-zero numbers.
So those are the two cases.
If it's singular, I go to a row of zeroes.
If it's invertible, I go to U and then to the diagonal D, and then which -- and then to d1, d2, up to dn.
As the formula -- so we have a formula now.
We have a formula for the determinant and it's actually a very much more practical formula than the but they didn't matter anyway.
ad-bc formula.
Is it correct, maybe you should just -- let's just check that.
Two-by-two.
What are the pivots of a two-by-two matrix?
What does elimination do with a two-by-two matrix?
It -- there's the first pivot, fine.
What's the second pivot?
We subtract, so I'm putting it in this upper triangular form.
What do I -- my multiplier is c over a, right?
I multiply that row by c over a and I subtract to get that zero, and here I have d minus c over a times b.
That's the elimination on a two-by-two.
So I've finally discovered that the determinant of this matrix -- I've got it from the properties, not by knowing the answer from last year, that the determinant of this two-by-two is the product of A times that, and of course when I multiply A by that, the product of that and that is ad minus, the a is canceled, bc.
So that's great, provided a isn't zero.
because all math professors watching this will be waiting If a was zero, that step wasn't allowed, with seven row exchanges and with ten row exchanges?
zero wasn't a pivot.
So that's what I -- I've covered all the bases.
I have to -- if a is zero, then I have to do the exchange, and if the exchange doesn't work, it's because a is proof.
singular.
OK, those are -- those are the direct properties of the determinant.
And now, finally, I've got two more, nine and ten.
And that's -- I think you can... Like, the ones we've got here are totally connected with our elimination process and whether pivots are available and whether we get a row of zeroes.
I think all that you can swallow in one shot.
Let me tell you properties nine and ten.
They're quick to write down.
They're very, very useful.
So I'll just write them down and use them.
Property nine says that the determinant of a product -- if I That's the, like, concrete proof that, multiply two matrices.
So if I multiply two matrices, A and B, that the determinant of the product is determinant of A times determinant of B, and for me that one is like, that's a very valuable property, and it's sort of like partly coming out of the blue, because we haven't been multiplying matrices and here suddenly this determinant has this multiplying property.
Remember, it didn't have the linear property, it didn't have the adding property.
The determinant of A plus B is not the sum of the determinants, but the determinant of A times B is the product, is the product of the determinants.
OK, so for example, what's the determinant of A inverse?
Using that property nine.
Let me, let me put that under here because the camera is happier if it can focus on both at once.
So let me put it underneath.
The determinant of A inverse, because property ten will come in that space.
What does this tell me about A inverse, its determinant?
OK, well, what do I know about A inverse?
I know that A inverse times A is odd.
So what I going to do?
I'm going to take determinants of both sides.
The determinant of I is one, and what's the determinant of A inverse A?
That's a product of two matrices, A and B.
So it's the product of the determinant, so what I learning?
I'm learning that the determinant of A inverse times the determinant of A is the determinant of I, that's this one.
Again, I happily use property one.
OK, so that tells me that the determinant of A inverse is one over.
Here's my -- here's my conclusion -- is one over the determinant of A. I guess that that -- I, I always try to think, well, do we know some cases of that?
Of course, we know it's right already if A is diagonal.
If A is a diagonal matrix, then its determinant is just a product of those numbers.
So if A is, for example, two-three, then we know that A-inverse is one-half one-third, and sure enough, that has determinant six, and that has determinant one-sixth.
And our rule checks.
So somehow this proof, this property has to -- somehow the proof of that property -- if we can boil it down to diagonal matrices then we can read it off, whether it's A and A-inverse, or two different diagonal matrices A and B.
For diagonal -- so what I saying?
I'm saying for a diagonal matrices, check.
But we'd have to do elimination steps, we'd have to patiently do the, the, argument if we want to use these previous properties to get it for other matrices.
And it also tells me -- what, just let's, see what else it's telling me.
What's the determinant of, of A-squared?
If I take a matrix and square it?
Then the determinant just got squared.
Right?
The determinant of A-squared is the determinant of A times the determinant of A.
So if I square the matrix, I square the determinant.
If I double the matrix, what do I do to the non-zeroes flipped to the other side of the diagonal, determinant?
Think about that one.
If I double the matrix, what -- so the determinant of A, since I'm writing down, like, facts that follow, the determinant of A-squared is the determinant of A, all squared.
The determinant of 2A is what?
That's A plus A.
But wait, er, I don't want the answer to determinant of A here.
That's wrong.
It's not two determinant of A, What is it?
OK, one more coming, which I I have to make, what's the number that I have to multiply determinant of A by if I double the whole matrix, if I double every entry in the matrix?
What happens to the determinant?
If that were possible, that would be a bad thing, Supposed it's an n-by-n matrix.
that gets -- get down to triangular Two to the n, right.
Two to the nth.
Now, why is it two to the nth, and not just two?
So why is it two to the nth?
Because I'm factoring out two from every row.
There's a factor -- this has a factor two in every row, so I can factor two out of the first row.
I factor a different two out of the second row, a different two out of the nth row, so I've got all those twos coming out.
So it's like volume, really, and that's one of the great applications of determinants.
If I -- if I have a box and I double all the sides, I multiply the volume by two to the nth.
If it's a box in three dimensions, I multiply the volume by 8.
So this is rule 3A here.
This is rule nine.
And notice the way this rule sort of checks out with the singular/non-singular stuff, that if A is invertible, what does that mean about its determinant?
It's not zero, and therefore this makes sense.
The case when determinant of A is zero, that's the case where my formula doesn't work anymore.
If determinant of A is zero, this is ridiculous, and that's ridiculous.
A-inverse doesn't exist, and one over zero doesn't make sense.
So don't miss this property.
It's sort of, like, amazing that it can... And the tenth property is equally simple to state, that the determinant of A transposed equals the determinant of A.
And of course, let's just check it on the ab cd guy.
We could check that sure enough, that's ab cd, it works.
It's ad - bc, it's ad - bc, sure enough.
So that transposing did not change the determinant.
But what it does change is -- well, what it does is it lists, so all -- I've been working with rows.
If a row is all zeroes, the determinant is zero.
But now, with rule ten, I know what to do is a column is all zero.
If a column is all zero, what's the determinant?
Zero, again.
So, like all those properties about rows, exchanging two rows reverses the sign.
Now, exchanging two columns reverses the sign, because I can always, if I want to see why, I can transpose, those columns become rows, I do the exchange, I transpose back.
And I've done a column operation.
So, in, in conclusion, there was nothing special about row one, 'cause I could exchange rows, and now there's nothing special about rows that isn't equally true for columns because this is the same.
OK. And again, maybe I won't -- oh, let's see.
Do we...?
Maybe it's worth seeing a quick proof of this number ten, quick, quick, er, proof of number ten.
Er, let me take the -- this is number ten.
A transposed equals A. Determinate of A transposed equals determinate of A.
That's what I should have said.
OK.
So, let's just, er.
A typical matrix A, if I use elimination, this factors into LU.
And the transpose is U transpose, l transpose.
Er... let me.
So this is proof, this is proof number ten, using -- well, I don't know which ones I'll use, so I'll put 'em all in, one to nine.
OK.
I'm going to prove number ten by using one to nine.
I won't cover every case, but I'll cover almost every case.
So in almost every case, A can factor into LU, and A transposed can factor into that.
Now, what do I do next?
So I want to prove that these are the same.
I see a product here.
So I use rule nine.
So, now what I want to prove is, so now I know that this is LU, this is U transposed and l transposed.
Now, just for a practice, what are all those determinants?
So this is, this is, this is prove this, prove this, prove this, and now I'm ready to do it.
What's the determinant of l?
You remember what l is, it's this lower triangular matrix with ones on the diagonals.
So what is the determinant of that guy?
I- It's one.
Any time I have this triangular matrix, I can get rid of all the non-zeroes, down to the diagonal case.
The determinate of l is one.
How about the determinant of l transposed?
That's triangular also, right?
Still got those ones on the diagonal, it's just the matrices and then get down to diagonal matrices.
right?
If If I could -- why would it be bad?
My whole lecture would die, right?
Because rule two said that if you do seven row exchanges, then the sign of the determinant reverses.
But if you do ten row exchanges, the sign of the determinant stays the same, because minus one ten times is plus one.
OK, so there's a hidden fact here, that every -- like, every permutation, the permutations are either odd or even.
I could get the permutation with seven row exchanges, then I could probably get it with twenty-one, or twenty-three, or a hundred and one, if it's an odd one.
Or an even number of permutations, so, but that's the key fact that just takes another little algebraic trick to see, and that means that the determinant is well-defined by properties one, two, three and it's got properties four to ten.
OK, that's today and I'll try to get the homework for next Wednesday onto the web this afternoon.
Thanks.
OK, this is the second lecture on determinants.
There are only three.
With determinants it's a fascinating, small topic inside linear algebra.
Used to be determinants were the big thing, and linear algebra was the little thing, but they -- those changed, that situation changed.
Now determinants is one specific part, very neat little part.
And my goal today is to find a formula for the determinant.
It'll be a messy formula.
So that's why you didn't see it right away.
But if I'm given this n by n matrix then I use those entries to create this number, the determinant.
So there's a formula for it.
In fact, there's another formula, a second formula using something called cofactors.
So you'll -- you have to know what cofactors are.
And then I'll apply those formulas for some, some matrices that have a lot of zeros away from the three diagonals.
OK.
So I'm shooting now for a formula for the determinant.
You remember we started with these three properties, three simple properties, but out of that we got all these amazing facts, like the determinant of A B equals determinant of A times determinant of B.
But the three facts were -- oh, how about I just take two by twos.
I know, because everybody here knows, the determinant of a two by two matrix, but let's get it out of these three formulas.
OK, so here's my, my two by two matrix.
I'm looking for a formula for this determinant.
a b c d, OK.
So property one, I know what to do with the identity.
Right?
Property two allows me to exchange rows, and I know what to do then.
So I know that that determinant is one.
Property two allows me to exchange rows and know that this determinant is minus one.
And now I want to use property three to get everybody, to get everybody.
And how will I do that?
OK.
So remember that if I keep the second row the same, I'm allowed to use linearity in the first row.
And I'll just use it in a simple way.
I'll write this vector a b as a 0 + 0 b.
So that's one step using property three, linearity in the first row when the second row's the same.
OK.
But now you can guess what I'm going to do next.
I'll -- because I'd like to -- if I can make the matrices diagonal, then I'm clearly there.
So I'll take this one.
Now I'll keep the first row fixed and split the second row, so that'll be an a 0 and I'll split that into a c 0 and, keeping that first row the same, a 0 d. I used, for this part, linearity.
And now I'll -- whoops, that's plus because I've got more coming.
This one I'll do the same.
I'll keep this first row the same and I'll split c d into c 0 and 0 d. OK. Now I've got four easy determinants, and two of them are -- well, all four are extremely easy.
Two of them are so easy as to turn into zero, right?
Which two of these determinants are zero right away?
The first guy is zero.
Why is he zero?
Why is that determinant nothing, forget him?
Well, it has a column of zeros.
And by the -- well, so one way to think is, well, it's a singular matrix.
Oh, for, for like forty-eight different reasons, that determinant is zero.
It's a singular matrix that has a column of zeros.
It's, it's dead.
And this one is about as dead too.
Column of zeros.
OK.
So that's leaving us with this one.
Now what do I -- how do I know its determinant, following the rules?
Well, I guess one of the properties that we actually got to was the determinant of that -- diagonal matrix, then -- so I, I'm finally getting to that determinant is the a d. And this determinant is what?
What's this one?
Minus -- because I would use property two to do a flip to make it c b, then property three to factor out the b, property c to factor out the c -- the property again to factor out the c, and that minus, and of course finally I got the answer that we knew we would get.
But you see the method.
You see the method, because it's method I'm looking for here, not just a two by two answer but the method of doing -- now I can do three by threes and four by fours and any size.
So if you can see the method of taking each row at a time -- so let's -- what would happen with three by threes?
Can we mentally do it rather than I write everything on the board for three by threes?
So what would we do if I had three by threes?
I would keep rows two and three the same and I would split the first row into how many pieces?
Three pieces.
I'd have an A zero zero and a zero B zero and a zero zero C or something for the first row.
So I would instead of going from one piece to two pieces to four pieces, I would go from one piece to three pieces to -- what would it be?
Each of those three, would, would it be nine?
Or twenty-seven?
Oh yeah, I've actually got more steps, right.
I'd go to nine but then I'd have another row to straighten out, twenty-seven.
Yes, oh God.
OK, let me say this again then.
If I -- if it was three by three, I would -- separating out one row into three pieces would give me three, separating out the second row into three pieces, then I'd be up to nine, separating out the third row into its three pieces, I'd be up to twenty-seven, three cubed, pieces.
But a lot of them would be zero.
So now when would they not be zero?
Tell me the pieces that would not be zero.
Now I will write the non-zero ones.
OK, so I have this matrix.
I think I have to use these, start using these double symbols here because otherwise I could never do n by n. OK. OK.
So I split this up like crazy.
A bunch of pieces are zero.
Whenever I have a column of zeros, I know I've got zero.
When do I not have zero?
When do I have -- what is it that's like these guys?
These are the survivors, two survivors there.
So my question for three by three is going to be what are the survivors?
How many survivors are there?
What are they?
And when do I get a survivor.
Well, I would get a survivor -- for example, one survivor will be that one times that one times that one, with all zeros everywhere else.
That would be one survivor.
a one one zero zero zero a two two zero zero zero a three three.
That's like the a d survivor.
Tell me another survivor.
What other thing -- oh, now here you see the clue.
Now can -- shall I just say the whole clue?
That I'm having -- the survivors have one entry from each row and each column.
One entry from each row and column.
Because if some column is missing, then I get a singular matrix.
And that, that's one of these guys.
See, you see what happened with -- this guy?
Column one never got used in 0 b 0 d. So its determinant was zero and I forget it.
So I'm going to forget those and just put -- so tell me one more that would be a survivor?
Well -- well, here's another one.
a one one zero zero -- now OK, that's used up row -- row one is used.
Column one is already used so it better be zero.
What else could I have?
Where could I pick the guy -- which column shall I use in row two?
Use column three, because here if I use column -- here I used column one and row one.
This was like the column -- numbers were one two three, right in order.
Now the column numbers are going to be one three, column three, and column two.
So the row numbers are one two three, of course.
The column numbers are some -- OK, some permutation of one two three, and here they come in the order one three two.
It's just like having a permutation matrix with, instead of the ones, with numbers.
And actually, it's very close to having a permutation matrix, because I, what I do eventually is I factor out these numbers and then I have got.
So what is that determinant equal?
I factor those numbers out and I've got a one one times a two two times a three three.
And what does this determinant equal?
Yeah, now tell me the, this -- I mean, we're really getting to the heart of these formulas now.
What is that determinant?
By the laws of -- by, by our three properties, I can factor these out, I can factor out the a one one, the a two three, and the a three two.
They're in separate rows.
I can do each row separately.
And then I just have to decide is that plus sign or is that a minus sign?
And the answer is it's a minus.
Why minus?
Because these is one row exchange to get it back to the identity.
So that's a minus.
Now I through?
No, because there are other ways.
What I'm really through with, what I've done, what I've, what I've completed is only the part where the a one one is there.
But now I've got parts where it's a one two.
And now if it's a one two that row is used, that column is used.
You see that idea?
I could use this row and column.
Now that column is used, that column is used, and this guy has to be here, a three three.
And what's that determinant?
That's an a one two times an a two one times an a three three, and does it have a plus or a minus?
A minus is right.
It has a minus.
Because it's one flip away from an id- from the, regular, the right order, the diagonal order.
And now what's the other guy with a -- with, a one two up there?
I could have used this row.
I could have put this guy here and this guy here.
Right?
You see the whole deal?
Now that's an a one two, a two three, a three one, and does that go with a plus or a minus?
Yeah, now that takes a minute of thinking, doesn't it, because one row exchange doesn't get it in line.
So what is the answer for this?
Plus or minus?
Plus, because it takes two exchanges.
I could exchange rows one and three and then two and three.
Two exchanges makes this thing a plus.
And then finally we have -- we're going to have two more.
OK.
Zero zero a one three, a two one zero zero, zero a three two zero.
And one more guy.
Zero zero a one three, zero a two two zero, A three one zero zero.
And let's put down what we get from those.
An a one three, an a two one, and an a three two, and I think that one is a plus.
And this guys is a minus because one exchange would put it -- would order it.
And that's a minus.
All right, that has taken one whole board just to do the three by three.
But do you agree that we now have a formula for the determinant which came from the three properties?
And it must be it.
And I'm going to keep that formula.
That's a famous -- that three by three formula is one that if, if the cameras will follow me back to the beginning here, I, I get the ones with the plus sign are the ones that go down like down this way.
And the ones with the minus signs are sort of the ones that go this way.
I won't make that precise.
For two reasons, one, it would clutter up the board, and second reason, it wouldn't be right for four by fours.
For four by four, let me just say right away, four by four matrix -- the, the cross diagonal, the wrong diagonal happens to come out with a plus sign.
Why is that?
If I have a four by four matrix with ones coming on the counter diagonal, that determinant is plus.
Why?
Why plus for that guy?
Because if I exchange rows one and four and then I exchange rows two and three, I've got the identity, and I did two exchanges.
So this down to this, like, you know, down toward Miami and down toward LA stuff is, like, three by three only.
OK.
But I do want to get now -- I don't want to go through this for a four by four.
I do want to get now the general formula.
So this is what I refer to in the book as the big formula.
So now this is the big formula for the determinant.
I'm asking you to make a jump from two by two and three by three to n by n. OK, so this will be the big formula.
That the determinant of A is the sum of a whole lot of terms.
And what are those terms?
And, and is it a plus or a minus sign, and I have to tell you which, which it is, because this came in -- in the three by three case, I had how many terms?
Six.
And half were plus and half were minus.
How many terms are you figuring for four by four?
If I get two terms in the two by two case, three -- six terms in the three by three case, what's that pattern?
How many terms in the four by four case?
Twenty-four.
Four factorial.
Why four factorial?
This will be a sum of n factorial terms.
Twenty-four, a hundred and twenty, seven hundred and twenty, whatever's after that.
OK. Half plus and half minus.
And where do those n factorial -- terms come from?
This is the moment to listen to this lecture.
Where do those n factorial terms come from?
They come because the first, the guy in the first row can be chosen n ways.
And after he's chosen, that's used up that, that column.
So the one in the second row can be chosen n minus one ways.
And after she's chosen, that second column has been used.
And then the one in the third row can be chosen n minus two ways, and after it's chosen -- notice how I'm getting these personal pronouns.
But I've run out.
And I'm not willing to stop with three by three, so I'm just going to write the formula down.
So the one in the first row comes from some column alpha.
I don't know what alpha is.
And the one in the -- I multiply that by somebody in the second row that comes from some different column.
And I multiply that by somebody in the third row who comes from some yet different column.
And then in the n-th row, I don't know what -- I don't know how to draw.
Maybe omega, for last.
And the whole point is then that -- that those column numbers are different, that alpha, beta, gamma, omega, that set of column numbers is some permutation, permutation of one to n. It, it, the n column numbers are each used once.
And that gives us n factorial terms.
And when I choose a term, that means I'm choosing somebody from every row and column.
And then I just -- like the way I had this from row and column one, row and column two, row and column three, so that -- what was the alpha beta stuff in that, for that term here?
Alpha was one, beta was two, gamma was three.
The permutation was, was the trivial permutation, one two three, everybody in the right order.
You see that formula?
It's -- do you see why I didn't want to start with that the first day, Friday?
I'd rather we understood the properties.
Because out of this formula, presumably I could figure out all these properties.
How would I know that the determinant of the identity matrix was one, for example, out of this formula?
Why is -- if A is the identity matrix, how does this formula give me a plus one?
You see it, right?
Because, because almost all the terms are zeros.
Which term isn't zero, if, if A is the identity matrix?
Almost all the terms are zero because almost all the As are zero.
It's only, the only time I'll get something is if it's a one one times a two two times a three three.
Only, only the, only the permutation that's in the right order will, will give me something.
It'll come with a plus sign.
And the determinant of the identity is one.
So, so we could go back from this formula and prove everything.
We could even try to prove that the determinant of A B was the determinant of A times the determinant of B.
But like next week we would still be working on it, because it's not -- clear from -- if I took A B, my God.
You know --.
The entries of A B would be all these pieces.
Well, probably, it's probably -- historically it's been done, but it won't be repeated in eighteen oh six.
OK.
It would be possible probably to see, why the determinant of A equals the determinant of A transpose.
That was another, like, miracle property at the end.
That would, that would, that's an easier one, which we could find.
OK. Is that all right for the big formula?
I could take you then a, a typical -- let me do an example.
Which I'll just create.
I'll take a four by four matrix.
I'll put some, I'll put some ones in and some zeros in.
OK. Let me -- I don't know how many to put in, to tell the truth.
I've never done this before.
I don't know the determinant of that matrix.
So like mathematics is being done for the first time in, in front of your eyes.
What's the determinant?
Well, a lot of -- there are twenty-four terms, because it's four by four.
Many of them will be zero, because I've got all those zeros there.
Maybe the whole determinant is zero.
I mean, I -- is that a singular matrix?
That possibility definitely exists.
I could, I could, So one way to do it would be elimination.
Actually, that would probably be a fairly reasonable way.
I could use elimination, so I could use -- go back to those properties, that -- and use elimination, get down, eliminate it down, do I have a row of zeros at the end of elimination?
The answer is zero.
I was thinking, shall I try this big formula?
OK. Let's try the big formula.
How -- tell me one way I can go down the matrix, taking a one, taking a one from every row and column, and make it to the end?
So it's -- I get something that isn't zero.
Well, one way to do it, I could take that times that times that times that times that.
That would be one and, and, and I just said, that comes in with what sign?
Plus.
That comes with a plus sign.
Because, because that permutation -- I've just written the permutation about four three two one, and one exchange and a second exchange, two exchanges puts it in the correct order.
Keep walking away, don't.... OK, we're executing a determinant formula here.
Uh as long as it's not periodic, of course.
If he comes back I'm in -- no.
All right, all right.
OK, so that would give me a plus one.
All right.
Are there any others?
Well, of course we see another one here.
This times this times this times this strikes us right away.
So that's the order three, the order -- let me make a little different mark here.
Three two one four.
And is that a plus or a minus, three two one four?
Is that, is that permutation a plus or a minus permutation?
It's a minus.
How do you see that?
What exchange shall I do to get it in the right order?
If I exchange the one and the three I'm in the right orders, took one exchange to do it, so that would be a plus -- that would be a minus one.
And now I don't know if there're any more here.
Let's see.
Let me try again starting with this.
Now I've got to pick somebody from -- oh yeah, see, you see what's happening.
If I I start there, OK, column three is used.
So then when I go to next row, I can't use that, I must use that.
Now columns two and three are used.
When I come to this row I must use that.
And then I must use that.
So if I start there, this is the only one I get.
And similarly, if I start there, that's the only one I get.
So what's the determinant?
What's the determinant?
Zero.
The determinant is zero for that case.
Because we, we were able to check the twenty-four terms.
Twenty-two of them were zero.
One of them was plus one.
One of them was minus one.
Add up the twenty-four terms, zero is the answer.
OK. Well, I didn't know it would be zero, I -- because I wasn't, like, thinking ahead.
I was a little scared, actually.
I said, that, apparition went by.
So and I don't know if the camera caught that.
So whether the rest of the world will realize that I was in danger or not, we don't know.
But anyway, I guess he just wanted to be sure that we got the right answer, which is determinant zero.
And then that makes me think, OK, the matrix must be, the matrix must be singular.
And then if the matrix is singular, maybe there's another way to see that it's singular, like find something in its null space.
Or find a combination of the rows that gives zero.
And like what d- what, what combination of those rows does give zero.
Suppose I add rows one and rows three.
If I add rows one and rows three, what do I get?
I get a row of all ones.
Then if I add rows two and rows four I get a row of all ones.
So row one minus row two plus row three minus row four is probably the zero row.
It's a singular matrix.
And I could find something in its null space the same way.
That would be a combination of columns that gives zero.
OK, there's an example.
All right.
So that's, well, that shows two things.
That shows how we get the twenty-four terms and it shows the great advantage of having a lot of zeros in there.
OK.
So we'll use this big formula, but I want to pick -- I want to go onward now to cofactors.
Onward to cofactors.
Cofactors is a way of breaking up this big formula that connects this n by n -- this is an n by n determinant that we've just have a formula for, the big formula.
So cofactors is a way to connect this n by n determinant to, determinants one smaller.
One smaller.
And the way we want to do it is actually going to show up in this.
Since the three by three is the one that we wrote out in full, let's, let me do this three by -- so I'm talking about cofactors, and I'm going to start again with three by three.
And I'm going to take the, the exact formula, and I'm just going to write it as a one one -- this is the determinant I'm writing.
I'm just going to say a one one times what?
A one one times what?
And it's a one one times a two two a three three minus a two three a three two.
Then I've got the a one two stuff times something.
And I've got the a one three stuff times something.
Do you see what I'm doing?
I'm taking our big formula and I'm saying, OK, choose column -- out of the first row, choose column one.
And take all the possibilities.
And those extra factors will be what we'll call the cofactor, co meaning going with a one one.
So this in parenthesis are, these are in, the cofactors are in parens.
A one one times something.
And I figured out what that something was by just looking back -- if I can walk back here to the, to the a one one, the one that comes down the diagonal minus the one that comes that way.
That's, those are the two, only two that used a one one.
So there they are, one with a plus and one with a minus.
And now I can write in the -- I could look back and see what used a one two and I can see what used a one three, and those will give me the cofactors of a one two and a one three.
Before I do that, what's this number, what is this cofactor?
What is it there that's multiplying a one one?
Tell me what a two two a three three minus a two three a three two is, for this -- do you recognize that?
Do you recognize -- let's see, I can -- and I'll put it here.
There's the a one one.
That's used column one.
Then there's -- the other factors involved these other columns.
This row is used.
This column is used.
So this the only things left to use are these.
And this formula uses them, and what's the, what's the cofactor?
Tell me what it is because you see it, and then -- I'll be happy you see what the idea of cofactors.
It's the determinant of this smaller guy.
A one one multiplies the determinant of this smaller guy.
That gives me all the a one one part of the big formula.
You see that?
This, the determinant of this smaller guy is a two two a three three minus a two three a three two.
In other words, once I've used column one and row one, what's left is all the ways to use the other n-1 columns and n-1 rows, one of each.
All the other -- and that's the determinant of the smaller guy of size n-1.
So that's the whole idea of cofactors.
And we just have to remember that with determinants we've got pluses and minus signs to keep straight.
Can we keep this next one straight?
Let's do the next one.
OK, the next one will be when I use a one two.
I'll have left -- so I can't use that column any more, but I can use a two one and a two three and I can use a three one and a three three.
So this one gave me a one times that determinant.
This will give me a one two times this determinant, a two one a three three minus a two three a three one.
So that's all the stuff involving a one two.
But have I got the sign right?
Is the determinant of that correctly given by that or is there a minus sign?
There is a minus sign.
I can follow one of these.
If I do that times that times that, that was one that's showing up here, but it should have showed -- it should have been a minus.
So I'm going to build that minus sign into the cofactor.
So, so the cofactor -- so I'll put, put that minus sign in here.
So because the cofactor is going to be strictly the thing that multiplies the, the factor.
The factor is a one two, the cofactor is this, is the parens, the stuff in parentheses.
So it's got the minus sign built in.
And if I did -- if I went on to the third guy, there w- there'll be this and this, this and this.
And it would take its determinant.
It would come out plus the determinant.
So now I'm ready to say what cofactors are.
So this would be a plus and a one three times its cofactor.
And over here we had plus a one one times this determinant.
But and there we had the a one two times its cofactor, but the -- so the point is the cofactor is either plus or minus the determinant.
So let me write that underneath them.
What is the, what are cofactors?
The cofactor if any number aij, let's say.
This is, this is all the terms in the, in the big formula that involve aij.
We're especially interested in a1j, the first row, that's what I've been talking about, but any row would be all right.
All right, so -- what terms involve aij?
So -- it's the determinant of the n minus one matrix -- with row i, column j erased.
So it's the, it's a matrix of size n-1 with -- of course, because I can't use this row or this column again.
So I have the matrix all there.
But now it's multiplied by a plus or a minus.
This is the cofactor, and I'm going to call that cij.
Capital, I use capital c just to, just to emphasize that these are important and emphasize that they're, they're, they're different from the (a)s. OK.
So now is it a plus or is it a minus?
Because we see that in this case, for a one one it was a plus, for a one two I -- this is ij -- it was a minus.
For this ij it was a plus.
So any any guess on the rule for plus or minus when we see those examples, ij equal one one or one three was a plus?
It sounds very like i+j odd or even.
That, that's doesn't surprise us, and that's the right answer.
So it's a plus if i+j is even and it's a minus if i+j is odd.
So if I go along row one and look at the cofactors, I just take those determinants, those one smaller determinants, and they come in order plus minus plus minus plus minus.
But if I go along row two and, and, and take the cofactors of sub-determinants, they would start with a minus, because the two one entry, two plus one is odd, so the -- like there's a pattern plus minus plus minus plus if it was five by five, but then if I was doing a cofactor then this sign would be minus plus minus plus minus, plus minus plus -- it's sort of checkerboard.
OK. OK. Those are the signs that, that are given by this rule, i+j even or odd.
And those are built into the cofactors.
The thing is called a minor without th- before you've built in the sign, but I don't care about those.
Build in that sign and call it a cofactor.
So what's the cofactor formula?
OK. What's the cofactor formula then?
Let me come back to this board and say, what's the cofactor formula?
Determinant of A is -- let's go along the first row.
It's a one one times its cofactor, and then the second guy is a one two times its cofactor, and you just keep going to the end of the row, a1n times its cofactor.
So that's cofactor for -- along row one.
And if I went along row I, I would -- those ones would be Is.
That's worth putting a box over.
That's the cofactor formula.
Do you see that -- actually, this would give me another way I could have started the whole topic of determinants.
And some, some people might do it this -- choose to do it this way.
Because the cofactor formula would allow me to build up an n by n determinant out of n-1 sized determinants, build those out of n-2, and so on.
I could boil all the way down to one by ones.
So what's the cofactor formula for two by two matrices?
Yeah, tell me that.
What's the cofactor for us?
Here is the, here is the world's smallest example, practically, of a cofactor formula.
OK. Let's go along row one.
I take this first guy times its cofactor.
What's the cofactor of the one one entry?
d, because you strike out the one one row and column and you're left with d. Then I take this guy, b, times its cofactor.
What's the cofactor of b?
Is it c or it's -- minus c, because I strike out this guy, I take that determinant, and then I follow the i+j rule and I get a minus, I get an odd.
So it's b times minus c. OK, it worked.
Of course it, it worked.
And the three by three works.
So that's the cofactor formula, and that is, that's an -- that's a good formula to know, and now I'm feeling like, wow, I'm giving you a lot of algebra to swallow here.
Last lecture gave you ten properties.
Now I'm giving you -- and by the way, those ten properties led us to a formula for the determinant which was very important, and I haven't repeated it till now.
What was that?
The, the determinant is the product of the pivots.
So the pivot formula is, is very important.
The pivots have all this complicated mess already built in.
As you did elimination to get the pivots, you built in all this horrible stuff, quite efficiently.
Then the big formula with the n factorial terms, that's got all the horrible stuff spread out.
And the cofactor formula is like in between.
It's got easy stuff times horrible stuff, basically.
But it's, it shows you, how to get determinants from smaller determinants, and that's the application that I now want to make.
So may I do one more example?
So I remember the general idea.
But I'm going to use this cofactor formula for a matrix -- so here is going to be my example.
It's -- I promised in the, in the lecture, outline at the very beginning to do an example.
And let me do -- I'm going to pick tri-diagonal matrix of ones.
I could, I'm drawing here the four by four.
So this will be the matrix.
I could call that A4.
But my real idea is to do n by n. To do them all.
So A -- I could -- everybody understands what A1 and A2 are.
Yeah.
Maybe we should just do A1 and A2 and A3 just for -- so this is What's the determinant of A1?
A4.
What's the determinant of A1?
So, so what's the matrix A1 in this formula?
It's just got that.
So the determinant is one.
What's the determinant of A2?
So it's just got this two by two, and its determinant is -- zero.
And then the three by three.
Can we see its determinant?
Can you take the determinant of that three by three?
Well, that's not quite so obvious, at least not to me.
Being three by three, I don't know -- so here's a, here's a good example.
How would you do that three by three determinant?
We've got, like, n factorial different ways.
Well, three factorial.
So we've got six ways.
OK.
I mean, one way to do it -- actually the way I would probably do it, being three by three, I would use the complete the big formula.
I would say, I've got a one from that, I've got a zero from that, I've got a zero from that, a zero from that, and this direction is a minus one, that direction's a minus one.
I believe the answer is minus one.
Would you do it another way?
Here's another way to do it, look.
Subtract row three from -- I'm just looking at this three by three.
Everybody's looking at the three by three.
Subtract row three from row two.
Determinant doesn't change.
So those become zeros.
OK, now use the cofactor formula.
How's that?
How can, how -- if this was now zeros and I'm looking at this three by three, use the cofactor formula.
Why not use the cofactor formula along that row?
Because then I take that number times its cofactor, so I take this number -- let me put a box around it -- times its cofactor, which is the determinant of that and that, which is what?
That two by two matrix has determinant one.
So what's the cofactor?
What's the cofactor of this guy here?
Looking just at this three by three.
The cofactor of that one is this determinant, which is one times negative.
So that's why the answer came out minus one.
OK.
So I did the three by three.
I don't know if we want to try the four by four.
Yeah, let's -- I guess that was the point of my example, of course, so I have to try it.
Sorry, I'm in a good mood today, so you have to stand for all the bad jokes.
OK. OK.
So what was the matrix?
Ah.
OK, now I'm ready for four by four.
Who wants to -- who wants to guess the, the -- I don't know, frankly, this four by four, what's, what's the determinant.
I plan to use cofactors.
OK, let's use cofactors.
The determinant of A4 is -- OK, let's use cofactors on the first row.
Those are easy.
So I multiply this number, which is a convenient one, times this determinant.
So it's, it's one times the, this three by three determinant.
Now what is -- do you recognize that matrix?
It's A3.
So it's one times the determinant of A3.
Coming along this row is a one times this determinant, and it goes with a plus, right?
And then we have this one.
And what is its cofactor?
Now I'm looking at, now I'm looking at this three by three, this three by three, so I'm looking at the three by three that I haven't X-ed out.
What is that -- oh, now it, we did a plus or a -- is it plus this determinant, this three by three determinant, or minus it?
It's minus it, right, because this is -- I'm starting in a one two position, and that's a minus.
So I want minus this determinant.
But these guys are X-ed out.
OK.
So I've got a three by three.
Well, let's use cofactors again.
Use cofactors of the column, because actually we could use cofactors of columns just as well as rows, because, because the transpose rule.
So I'll take this one, which is now sitting in the plus position, times its determinant -- oh!
Oh, hell.
What -- oh yeah, I shouldn't have said hell, because it's all right.
OK. One times the determinant.
What is that matrix now that I'm taking the, this smaller one of?
Oh, but there's a minus, right?
The minus came from, from the fact that this was in the one two position and that's odd.
So this is a minus one times -- and what's -- and then this one is the upper left, that's the one one position in its matrix, so plus.
And what's this matrix here?
Do you recognize that?
That matrix is -- yes, please say it -- A2.
And we -- that's our formula for any case.
A of any size n is equal to the determinant of A n minus one, that's what came from taking the one in the upper corner, the first cofactor, minus the determinant of A n minus two.
What we discovered there is true for all n. I didn't even mention it, but I stopped taking cofactors when I got this one.
Why did I stop?
Why didn't I take the cofactor of this guy?
Because he's going to get multiplied by zero, and no, no use wasting time.
Or this one too.
The cofactor, her cofactor will be whatever that determinant is, but it'll be multiplied by zero, so I won't bother.
OK, there is the formula.
And that now tells us -- I could figure out what A4 is now.
Oh yeah, finally I can get A4.
Because it's A3, which is minus one, minus A2, which is zero, so it's minus one.
You see how we're getting kind of numbers that you might not have guessed.
So our sequence right now is one zero minus one minus one.
What's the next one in the sequence, A5?
A5 is this minus this, so it is zero.
What's A6?
A6 is this minus this, which is one.
What's A7?
I'm, I'm going to be stopped by either the time runs out or the board runs out.
OK, A7 is this minus this, which is one.
I'll stop here, because time is out, but let me tell you what we've got.
What -- these determinants have this series, one zero minus one minus one zero one, and then it starts repeating.
It's pretty fantastic.
These determinants have period six.
So the determinant of A sixty-one would be the determinant of A1, which would be one.
OK.
I hope you liked that example.
A non-trivial example of a tri-diagonal determinant.
Thanks.
See you on Wednesday.
OK, this is lecture twenty.
And this is the final lecture on determinants.
And it's about the applications.
So we worked hard in the last two lectures to get a formula for the determinant and the properties of the determinant.
Now to use the determinant and, and always this determinant packs all this information into a single number.
And that number can give us formulas for all sorts of, things that we've been calculating already without formulas.
Now what was A inverse?
So, so I'm beginning with the formula for A inverse.
Two, two by two formula we know, right?
The two by two formula for A inverse, the inverse of a b c d inverse is one over the determinant times d a -b -c. Somehow I want to see what's going on for three by three and n by n. And actually maybe you can see what's going on from this two by two case.
So there's a formula for the inverse, and what did I divide by?
The determinant.
So what I'm looking for is a formula where it has one over the determinant and, and you remember why that makes good sense, because then that's perfect as long as the determinant isn't zero, and that's exactly when there is an inverse.
But now I have to ask can we recognize any of this stuff?
Do you recognize what that number d is from the past?
From last, from the last lecture?
My hint is think cofactors.
Because my formula is going to be, my formula for the inverse is going to be one over the determinant times a matrix of cofactors.
So you remember that D?
What's that the cofactor of?
Remember cofactors?
If -- that's the one one cofactor, because if I strike out row and column one, I'm left with d. And what's minus b?
OK.
Which cofactor is that one?
Oh, minus b is the cofactor of c, right?
If I strike out the c, I'm left with a b there.
And why the minus sign?
Because this c was in a two one position, and two plus one is odd.
So there was a minus went into the cofactor, and that's it.
OK.
I'll write down next what my formula is.
Here's the big formula for the A -- for A inverse.
It's one over the determinant of A and then some matrix.
And that matrix is the matrix of cofactors, c. Only one thing, it turns -- you'll see, I have to, I transpose.
So this is the matrix of cofactors, the -- what I'll just -- but why don't we just call it the cofactor matrix.
So the one one entry of, of c is the cof- is the one one cofactor, the thing that we get by throwing away row and column one.
It's the d. And, because of the transpose, what I see up here is the cofactor of this guy down here, right?
That's where the transpose came in.
What I see here, this is the cofactor not of this one, because I've transposed.
This is the cofactor of the b.
When I throw away the b, the b row and the b column, I'm left with c, and then I have that minus sign again.
And of course the two two entry is the cofactor of d, and that's this a.
So there's the formula.
OK.
But we got to think why.
I mean, it worked in this two by two case, but a lot of other formulas would have worked just as well.
We, we have to see why that's true.
In other words, why is it -- so this is what I aim to find.
And, and let's just sort of look to see what is that telling us.
That tells us that the -- the expression for A inverse -- let's look at a three by three.
Can I just write down a a b c d e f g h i?
And I'm looking for its inverse.
And what kind of a formula -- do I see there?
I mean, what -- the determinant is a bunch of products of three factors, right?
The determinant of this matrix'll involve a e i, and b f times g, and c times d times h, and minus c e g, and so on.
So things with three factors go in here.
Things with how many factors do things in the cofactor matrix have?
What's a typical cofactor?
What's the cofactor of a?
The cofactor of a, the one one entry up here in the inverse is?
I throw away the row and column containing a and I take the determinant of what's left, that's the cofactor.
And that's e i minus f h. Products of two things.
Now, I'm just making the observation that the determinant of A involves products of n entries.
And the cofactor matrix involves products of n minus 1 entries.
And, like, we never noticed any of this stuff when we were computing the inverse by the Gauss-Jordan method or whatever.
You remember how we did it?
We took the matrix A, we tucked the identity next to it, we did elimination till A became the identity.
And then that, the identity suddenly was A inverse.
Well, that was great numerically.
But we never knew what was going on, basically.
And now we see what the formula is in terms of letters, what's the algebra instead of the algorithm.
OK.
But I have to say why this is right, right?
I still -- that's a pretty magic formula.
Where does it come from?
Well, I'll just check it.
Having, having got it up there, let me -- I'll say, how can we check -- what do I want to check?
I want to check that A times its inverse gives the identity.
So I want, I want to check that A times this thing, A times this -- now I'm going to write in the inverse -- gives the identity.
So I check that A times C transpose -- let me bring the determinant up here.
Determinant of A times the identity.
That's my job.
That's it, that if this is true, and it is, then, then I've correctly identified A inverse as C transpose divided by the determinant.
OK.
But why is this true?
Why is that true?
Let me, let me put down what I'm doing here.
I have A -- here, here's A, here's a11 -- I'm doing this multiplication -- along to a1n.
And then down in this last row will be an an1 along to ann.
And I'm multiplying by the cofactor matrix transposed.
So when I transpose, it'll be c11 c12 down to c1n.
Notice usually that one coming first would mean I'm in row one, but I've transposed, so that's, those are the cofactors.
This first column are the cofactors from row one.
And then the last column would be the cofactors from row n. And why should that come out to be anything good?
In fact, why should it come out to be as good as this?
Well, you can tell me what the one one entry in the product is.
This is like you're seeing the main point if you just tell me one entry.
What do I get up there in the one one entry when I do this row of this row from A times this column of cofactors?
What, what will I get there?
Because we have seen this.
I mean, we're, right, building exactly on what the last lecture reached.
So this is a11 times c11, a12 times c12, a1n times c1n.
What does that what does that sum up to?
That's the cofactor formula for the determinant.
That's the, this cofactor formula, which I wrote, which we got last time.
The determinant of A is, if I use row one, let, let I equal one, then I have a11 times its cofactor, a12 times its cofactor, and so on.
And that gives me the determinant.
And it worked in this, in this case.
This row times this thing is, sure enough, ad minus bc.
But this formula says it always works.
So up here in this, in this position, I'm getting determinant of A.
What about in the two two position?
Row two times column two there, what, what is that?
That's just the cofactors, that's just row two times its cofactors.
So of course I get the determinant again.
And in the last here, this is the last row times its cofactors.
It's exactly -- you see, we're realizing that the cofactor formula is just this sum of products, so of course we think, hey, we've got a matrix multiplication there.
And we get determinant of A.
But there's one more idea here, right?
Great.
What else, what have I not -- so I haven't got that formula completely proved yet, because I've still got to do all the off-diagonal stuff, which I want to be zero, right?
I just want this to be determinant of A times the identity, and then I'm, I'm a happy person.
So why should that be?
Why should it be that one row times the cofactors from a different row, which become a column because I transpose, give zero?
In other words, the cofactor formula gives the determinant if the row and the, and the cofactors -- you know, if the entries of A and the cofactors are for the same row.
But for some reason, if I take the cofactors from the -- entries from the first row and the cofactors from the second row, for some reason I automatically get zero.
And it's sort of like interesting to say, why does that happen?
And can I just check that -- of course, we know it happens, in this case.
Here are the numbers from row one and here are the cofactors from row two, right?
Those are the numbers in row one.
And th- these are the cofactors from row two, because the cofactor of c is minus b and the cofactor of d is a.
And sure enough, that row times this column gives -- please say it.
Zero, right.
OK.
So now how come?
How come?
Can we even see it in this two by two case?
Why did -- well, I mean, I guess we, you know, in one way we certainly do see it, because it's right here.
I mean, do we just do it, and then we get zero.
But we want to think of some reason why the answer's zero, some reason that we can use in the n by n case.
So let -- here, here is my thinking.
We must be, if we're getting the answer's zero, we suspect that what we're doing somehow, we're taking the determinant of some matrix that has two equal rows.
So I believe that if we multiply these by the cofactors from some other row, we're taking the determinant -- ye, what matrix are we taking the determinant of?
Here it's, this is it.
We're, when we do that times this, we're really taking -- can I put this in little letters down here?
I'm taking -- let me look at the matrix a b a b.
Let me call that matrix AS, A screwed up.
OK. All right.
So now that matrix is certainly singular.
So if we find its determinant, we're going to get zero.
But I claim that if we find its determinant by the cofactor rule, go along the first row, we would take a times the cofactor of a.
And what is the -- see, how -- oh no -- let me go along the second row.
OK.
So let's see, which -- if I take -- I know I've got a singular matrix here.
And I believe that when I do this multiplication, what I'm doing is using the cofactor formula for the determinant.
And I know I'm going to get zero.
Let me try this again.
So the cofactor formula for the determinant says I should take a times its cofactor, which is this b, plus b times its cofactor, which is this minus a. OK. That's what we're doing, apart from a sign here.
Oh yeah, so you know, there might be a minus multiplying everything.
So if I take this determinant, it's A -- the determinant of this, the determinant of A screwed up is a times its cofactor, which is b, plus the second guy times its cofactor, which is minus a.
And of course I get the answer zero, and this is exactly what's happening in that, in that, row times this wrong column.
OK. That's the two by two picture, and it's just the same here.
That the reason I get a zero up in there is, the reason I get a zero is that when I multiply the first row of A and the last row of the cofactor matrix, it's as if I'm taking the determinant of this screwed up matrix that has first and last rows identical.
The book pins this down more specific -- and more carefully than I can do in the lecture.
I hope you're seeing the point.
That this is an identity.
That it's a beautiful identity and it tells us what the inverse of the matrix is.
So it gives us the inverse, the formula for the inverse.
OK.
So that's the first goal of my lecture, was to find this formula.
It's done.
OK. And of course I could invert, now, I can, I sort of like I can see what -- I can answer questions like this.
Suppose I have a matrix, and let me move the one one entry.
What happens to the inverse?
Just, just think about that question.
Suppose I have some matrix, I just write down some nice, non-singular matrix that's got an inverse, and then I move the one one entry a little bit.
I add one to it, for example.
What happens to the inverse matrix?
Well, this formula should tell me.
I have to look to see what happens to the determinant and what happens to all the cofactors.
And, the picture, it's all there.
It's all there.
We can really understand how the inverse changes when the matrix changes.
OK. Now my second application is to -- let me put that over here -- is to Ax=b.
Well, the -- course, the solution is A inverse b.
But now I have a formula for A inverse.
A inverse is one over the determinant times this C transpose times B. I now know what A inverse is.
So now I just have to say, what have I got here?
Is there any way to, to make this formula, this answer, which is the one and only answer -- it's the very same answer we got on the first day of the class by elimination.
Now I'm -- now I've got a formula for the answer.
Can I play with it further to see what's going on?
And Cramer's, this Cramer's Rule is exactly, that -- a way of looking at this formula.
OK.
So this is a formula for x.
Here's my formula.
Well, of course.
The first thing I see from the formula is that the answer x always has that in the determinant.
I'm not surprised.
There's a division by the determinant.
But then I have to say a little more carefully what's going on And let me tell you what Cramer's Rule is.
up here.
Let, let me take x1, the first component.
So this is the first component of the answer.
Then there'll be a second component and a, all the other components.
Can I take just the first component of this formula?
Well, I certainly have determinant of A down under.
And what the heck is the first -- so what do I get in C transpose b?
What's the first entry of C transpose b?
That's what I have to answer myself.
Well, what's the first entry of C transpose b?
This B is -- let me tell you what it is.
OK.
Somehow I'm multiplying cofactors by the entries of B, right, in this product.
Cofactors from the matrix times entries of b.
So any time I'm multiplying cofactors by numbers, I think, I'm getting the determinant of something.
And let me call that something B1.
So this is a matrix, the matrix whose determinant is coming out of that.
And we'll, we'll see what it is.
x2 will be the determinant of some other matrix B2, also divided by determinant of A.
So now I just -- Cramer just had a good idea.
He realized what matrix it was, what these B1 and B2 and B3 and so on matrices were.
Let me write them on the board underneath.
OK.
So what is this B1?
This B1 is the matrix that has b in its first column and otherwise the rest of it is A.
So it otherwise it has the rest, the, the n-1 columns of A.
It's the matrix with -- it's just the matrix A with column one replaced by the right-hand side, by the right-hand side b.
Because somehow when I take the determinant of this guy, it's giving me this matrix multiplication.
Well, how could that be?
How could -- so what's, what's the determinant formula I'll use here?
I'll use cofactors, of course.
And I might as well use cofactors down column one.
So when I use cofactors down column one, I'm taking the first entry of b times what?
Times the cofactor c11.
Do you see that?
When I, when I use cofactors here, I take the first entry here, B one let's call it, times the cofactor there.
But what's the cofactor in -- my little hand-waving is meant to indicate that it's a matrix of one size smaller, the cofactor.
And it's exactly c11.
Well, that's just what we wanted.
This first entry is c11 times b1.
And then the next entry is whatever, is c21 times b2, and so on.
And sure enough, if I look here, when I'm, when I do the cofactor expansion, b2 is getting multiplied by the right thing, and so on.
So there's Cramer's Rule.
And the book gives another kind of cute proof without, without building so much on, on cofactors.
But here we've got cofactors, so I thought I'd just give you this proof.
So what is B -- in general, what is Bj?
This is the, this is A with column j replaced by, by b.
So that's -- the determinant of that matrix that you divide by determinant of A to get xj.
So x -- let me change this general formula.
xj, the j-th one, is the determinant of Bj divided by the determinant of A.
And now we've said what Bj is.
Well, so Cramer found a rule.
And we could ask him, OK, great, good work, Cramer.
But is your rule any good in practice?
So he says, well, you couldn't ask about a rule in mine, right, because it's just -- all you have to do is find the determinant of A and these other determinants, so I guess -- oh, he just says, well, all you have to do is find n+1 determinants, the, the n Bs and the A.
And actually, I remember reading -- there was a book, popular book that, that kids interested in math read when I was a kid interested in math called Mathematics for the Million or something, by a guy named Bell.
And it had a little page about linear algebra.
And it said,-- so it explained elimination in a very complicated way.
I certainly didn't understand it.
And, and it made it, you know, it sort of said, well, there is this formula for elimination, but look at this great formula, Cramer's Rule.
So it certainly said Cramer's Rule was the way to go.
But actually, Cramer's Rule is a disastrous way to go, because to compute these determinants, it takes, like, approximately forever.
So actually I now think of that book title as being Mathematics for the Millionaire, because you'd have to be able to pay for, a hopelessly long calculation where elimination, of course, produced the x-s, in an instant.
But having a formula allows you to, with, with letters, you know, allows you to do algebra instead of, algorithms.
So the, there's some value in the Cramer's Rule formula for x and in the explicit formula for, for A inverse.
They're nice formulas, but I just don't want you to use them.
That'ss what it comes to.
If you had to -- and Matlab would never, never do it.
I mean, it would use elimination.
OK. Now I'm ready for number three in today's list of amazing connections coming through the determinant.
And that number three is the fact that the determinant gives a volume.
OK.
So now -- so that's my final topic for -- among these -- this my number three application, that the determinant is actually equals the volume of something.
Can I use this little space to consider a special case, and then I'll use the far board to think about the general rule.
So what I going to prove?
Or claim.
I claim that the determinant of the matrix is the volume of a box.
OK, and you say, which box?
Fair enough.
OK.
So let's see.
I'm in -- shall we say we're in, say three by three?
Shall we suppose -- let's, let's say three by three.
So, so we can really -- we're, we're talking about boxes in three dimensions, and three by three matrices.
And so all I do -- you could guess what the box is.
Here is, here is, three dimensions.
OK. Now I take the first row of the matrix, a11, a22, A -- sorry.
a11, a12, a13.
That row is a vector.
It goes to some point.
That point will be -- and that edge going to it, will be an edge of the box, and that point will be a corner of the box.
So here is zero zero zero, of course.
And here's the first row of the matrix: a11, a12, a13.
So that's one edge of the box.
Another edge of the box is to the second row of the matrix, row two.
Can I just call it there row two?
And a third row of the box will be to -- a third row -- a third edge of the box will be given by row three.
So, so there's row three.
That, the coordinates, what are the coordinates of that corner of the box?
a31, a32, a33.
So I've got that edge of the box, that edge of the box, that edge of the box, and that's all I need.
Now I just finish out the box, right?
I just -- the proper word, of course, is parallelepiped.
But for obvious reasons, I wrote box.
OK.
So, OK.
So there's the, there's the bottom of the box.
There're the four edge sides of the box.
There's the top of the box.
Cute, right?
It's the box that has these three edges and then it's completed to a, to a, each, you know, each side is a, is a parallelogram.
And it's that box whose volume is given by the determinant.
That's -- now it's -- possible that the determinant is negative.
So we have to just say what's going on in that case.
If the determinant is negative, then the volume, we, we should take the absolute value really.
So the volume, if we, if we think of volume as positive, we should take the absolute value of the determinant.
But the, the sign, what does the sign of the determinant -- it always must tell us something.
And somehow it, it will tell us whether these three is a -- whether it's a right-handed box or a left-handed box.
If we, if we reversed two of the edges, we would go between a right-handed box and a left-handed box.
We wouldn't change the volume, but we would change the, the cyclic, order.
So I won't worry about that.
And, so one special case is what?
A equal identity matrix.
So let's take that special case.
A equal identity matrix.
Is the formula determinant of identity matrix, does that equal the volume of the box?
Well, what is the box?
What's the box?
If A is the identity matrix, then these three rows are the three coordinate vectors, and the box is -- it's a cube.
It's the unit cube.
So if, if A is the identity matrix, of course our formula is Well, actually that proves property one -- that the volume right.
has property one.
Actually, we could, we could, we could get this thing if we -- if we can show that the box volume has the same three properties that define the determinant, then it must be the determinant.
So that's like the, the, the elegant way to prove this.
To prove this amazing fact that the determinant equals the volume, first we'll check it for the identity matrix.
That's fine.
The box is a cube and its volume is one and the determinant is one and, and one agrees with one.
Now let me take one -- let me go up one level to an orthogonal matrix.
Because I'd like to take this chance to bring in chapter -- the, the previous chapter.
Suppose I have an orthogonal matrix.
What did that mean?
I always called those things Q.
What was the point of -- suppose I have, suppose instead of the identity matrix I'm now going to take A equal Q, an orthogonal matrix.
What was Q then?
That was a matrix whose columns were orthonormal, right?
Those were its columns were unit vectors, perpendicular unit vectors.
So what kind of a box have we got now?
What kind of a box comes from the rows or the columns, I don't mind, because the determinant is the determinant of the transpose, so I'm never worried about that.
What kind of a box, what shape box have we got if the matrix is an orthogonal matrix?
It's another cube.
It's a cube again.
How is it different from the identity cube?
It's just rotated.
It's just the orthogonal matrix Q doesn't have to be the identity matrix.
It's just the unit cube but turned in space.
So sure enough, it's the unit cube, and its volume is one.
Now is the determinant one?
What's the determinant of Q?
We believe that the determinant of Q better be one or minus one, so that our formula is -- checks out in that -- if we can't check it in these easy cases where we got a cube, we're not going to get it in the general case.
So why is the determinant of Q equal one or minus one?
What do we know about Q?
What's the one matrix statement of the properties of Q?
A matrix with orthonormal columns has -- satisfies a certain equation.
What, what is that?
It's if we have this orthogonal matrix, then the fact -- the way to say what it, what its properties are is this.
Q prime, u- u- Q transpose Q equals I.
Right?
That's what -- those are the matrices that get the name Q, the matrices that Q transpose Q is I. OK. Now from that, tell me why is the determinant one or minus one.
How do I, out of this fact -- this may even be a homework problem.
It's there in the, in the list of exercises in the book, and let's just do it.
How do I get, how do I discover that the determinant of Q is one or maybe minus one?
I take determinants of both sides, everybody says, so I won't -- I take determinants of both sides.
On the right-hand side -- so I, when I take determinants of both sides, let me just do it.
Take the determinant of -- take determinants.
Determinant of the identity is one.
What's the determinant of that product?
Rule nine is paying off now.
The determinant of a product is the determinant of this guy -- maybe I'll put it, I'll use that symbol for determinant.
It's the determinant of that guy times the determinant of the other guy.
And then what's the determinant of Q transpose?
It's the same as the determinant of Q.
Rule ten pays off.
So this is just this thing squared.
So that determinant squared is one and sure enough it's one or minus one.
Great.
So in these special cases of cubes, we really do have determinant equals volume.
Now can I just push that to non-cubes.
Let me push it first to rectangles, rectangular boxes, where I'm just multiplying the e- the edges are -- let me keep all the ninety degree angles, because those are -- that, that makes my life easy.
And just stretch the edges.
Suppose I stretch that first edge, suppose this first edge I double.
Suppose I double that first edge, keeping the other edges the same.
What happens to the volume?
It doubles, right?
We know that the volume of a cube doubles.
In fact, because we know that the new cube would sit right on top -- I mean, the new, the added cube would sit right on -- would fit -- probably a geometer would say congruent or something -- would go right in, in the other.
We'd have two.
We have two identical cubes.
Total volume is now two.
OK.
So I want -- if I double an edge, the volume doubles.
What happens to the determinant?
If I double, the first row of a matrix, what ch- ch- what's the effect on the determinant?
It also doubles, right?
And that was rule number 3a.
Remember rule 3a was that if I, I could, if I had a factor in, in row one, T, I could factor it out.
So if, if I have a factor two in that row one, I can factor it out of the determinant.
It agrees with the -- the volume of the box has that factor two.
So, so volume satisfies this property 3a.
And now I really close, but I -- but to get to the very end of this proof, I have to get away from right angles.
I have to allow the possibility of, other angles.
And -- or what's saying the same thing, I have to check that the volume also satisfies 3b.
So can I -- This is end of proof that the -- so I'm -- determinant of A equals volume of box, and where I right now?
This volume has properties, properties one, no problem.
If the box is the cube, everything is -- if the box is the unit cube, its volume is one.
Property two was if I reverse two rows, but that doesn't change the box.
And it doesn't change the absolute value, so no problem there.
Property 3a was if I mul- you remember what 3a was?
So property one was about the identity matrix.
Property two was about a plus or minus sign that I don't care about.
Property 3a was a factor T in a row.
But now I've got property three B to deal with.
What was property 3b?
This is a great way to review these, properties.
So that 3b, the property 3b said -- let's do, let's do two by two.
So said that if I had a+a', b+b', c, d that this equaled what?
So this is property 3b.
This is the linearity in row one by itself.
So c d is staying the same, and I can split this into a b and a' b'.
That's property 3b, at least in the two by two case.
And what I -- I wanted now to show that the volume, which two, two by two, that means area, has this, has this property.
Let me just emphasize that we have got -- we're getting -- this is a formula, then, for the area of a parallelogram.
The area of this parallelogram -- can I just draw it?
OK, here's the, here's the parallelogram.
I have the row a b.
That's the first row.
That's the point a b.
And I tack on c d. c d, coming out of here.
And I complete the parallelogram.
So this is -- well, I better make it look right.
It's really this one that has coordinates c d and this has coordinates -- well, whatever the sum is.
And of course starting at zero zero.
So we all know, this is a+c, b+d.
Rather than -- I'm pausing on that proof for a minute just to going back to our formula.
Because I want you to see that unlike Cramer's Rule, that I wasn't that impressed by, I'm very impressed by this formula for the area of a parallelogram.
And what's our formula?
What, what's the area of that parallelogram?
If I had asked you that last year, you would have said OK, the area of a parallelogram is the base times the height, right?
So you would have figured out what this base, the -- how long that base was.
It's like the square root of A squared plus b squared.
And then you would have figured out how much is this height, whatever it is.
It's horrible.
This, I mean, we got square roots, and in that height there would be other revolting stuff.
But now what's the formula that we now know for the area?
It's the determinant of our little matrix.
It's just ad-bc.
No square roots.
Totally rememberable, because it's exactly a formula that we've been studying the whole, for three lectures.
OK. That's, you know, that's the most important point I'm making here.
Is that if you know the coordinates of a box, of the corners, then you have a great formula for the volume, area or volume, that doesn't involve any lengths or any angles or any heights, but just involves the coordinates that you've got.
And similarly, what's the area of this triangle?
Suppose I chop that off and say what about -- because you might often be interested in a triangle instead of a parallelogram.
What's the area of this triangle?
Now there again, everybody would have said the area of a triangle is half the base times the height.
And in some cases, if you know the base that a, that's -- and the height, that's fine.
But here, we, what we know is the coordinates of the corners.
We know the vertices.
And so what's the area of that triangle?
If I know these, if I know a b, c d, and zero zero, what's the area?
It's just half, so it's just half of this.
So this is, this is a- a b -- a d - b c for the parallelogram and one half of that, one half of ad-bc for the triangle.
So I mean, this is a totally trivial remark, to say, well, divide by two.
But it's just that you more often see triangles, and you feel you know the formula for the area but the good formula for the area is this one.
And I'm just going to -- I'm just going to say one more thing about the area of a triangle.
It's just because it's -- you know, it's so great to have a good formula for something.
What if our triangle did not start at zero zero?
What if our triangle, what if we had this -- what if we had -- so I'm coming back to triangles again.
But let me, let me put this triangle somewhere, it's -- I'm staying with triangles, I'm just in two dimensions, but I'm going to allow you to give me any three corners.
And in -- those six numbers must determine the area.
And what's the formula?
The area is going to be, it's going to be, there'll be that half of a parallelogram.
I mean, basically this can't be completely new, right?
We've got the area when -- we, we know the area when this is zero zero.
Now we just want to lift our sight slightly and get the area when all th- so let me write down what it, what it comes out to be.
It turns out that if you do this, x1 y1 and a 1, x2 y2 and a 1, x3 y3 and a 1, that that works.
That the determinant symbol, of course.
It's just -- if I gave you that determinant to find, you might subtract this row from this.
It would kill that one.
Subtract this row from this, it would kill that one.
Then you'd have a simple determinant to do with differences, and it would -- this little subtraction, what I did was equivalent to moving the triangle to start at the origin.
I did it fast, because time is up.
And I didn't complete that proof of 3b.
I'll leave -- the book has a carefully drawn figure to show why that works.
But I hope you saw the main point is that for area and volume, determinant gives a great formula.
OK. And next lectures are about eigenvalues, so we're really into the big stuff.
Thanks.
OK.
So this is the first lecture on eigenvalues and eigenvectors, and that's a big subject that will take up most of the rest of the course.
It's, again, matrices are square and we're looking now for some special numbers, the eigenvalues, and some special vectors, the eigenvectors.
And so this lecture is mostly about what are these numbers, and then the other lectures are about how do we use them, why do we want them.
OK, so what's an eigenvector?
Maybe I'll start with eigenvector.
What's an eigenvector?
So I have a matrix A. OK. What does a matrix do?
It acts on vectors.
It multiplies vectors x.
So the way that matrix acts is in goes a vector x and out comes a vector Ax.
It's like a function.
With a function in calculus, in goes a number x, out comes f(x).
Here in linear algebra we're up in more dimensions.
In goes a vector x, out comes a vector Ax.
And the vectors I'm specially interested in are the ones the come out in the same direction that they went in.
That won't be typical.
Most vectors, Ax is in -- points in some different direction.
But there are certain vectors where Ax comes out parallel to x.
And those are the eigenvectors.
So Ax parallel to x.
Those are the eigenvectors.
And what do I mean by parallel?
Oh, much easier to just state it in an equation.
Ax is some multiple -- and everybody calls that multiple lambda -- of x.
That's our big equation.
We look for special vectors -- and remember most vectors won't be eigenvectors -- that -- for which Ax is in the same direction as x, and by same direction I allow it to be the very opposite direction, I allow lambda to be negative or zero.
Well, I guess we've met the eigenvectors that have eigenvalue zero.
Those are in the same direction, but they're -- in a kind of very special way.
So this -- the eigenvector x. Lambda, whatever this multiplying factor is, whether it's six or minus six or zero or even some imaginary number, that's the eigenvalue.
So there's the eigenvalue, there's the eigenvector.
Let's just take a second on eigenvalue zero.
From the point of view of eigenvalues, that's no special deal.
That's, we have an eigenvector.
If the eigenvalue happened to be zero, that would mean that Ax was zero x, in other words zero.
So what would x, where would we look for -- what are the x-s?
What are the eigenvectors with eigenvalue zero?
They're the guys in the null space, Ax equals zero.
So if our matrix is singular, let me write this down.
If, if A is singular, then that -- what does singular mean?
It means that it takes some vector x into zero.
Some non-zero vector, that's why -- will be the eigenvector into zero.
Then lambda equals zero is an eigenvalue.
But we're interested in all eigenvalues now, lambda equals zero is not, like, so special anymore.
OK.
So the question is, how do we find these x-s and lambdas?
And notice -- we don't have an equation Ax equal B anymore.
I can't use elimination.
I've got two unknowns, and in fact they're multiplied together.
Lambda and x are both unknowns here.
So, we need to, we need a good idea of how to find them.
But before I, before I do that, and that's where determinant will come in, can I just give you some matrices?
Like here you go.
Take the matrix, a projection matrix.
OK.
So suppose we have a plane and our matrix P is -- what I've called A, now I'm going to call it P for the moment, because it's -- I'm thinking OK, let's look at a then this, this other new matrix, I just have an Ax, projection matrix.
What are the eigenvalues of a projection matrix?
So that's my question.
What are the x-s, the eigenvectors, and the lambdas, the eigenvalues, thing,4 but the roots of that quadratic for -- and now let me say a projection matrix.
My, my point is that we -- before we get into determinants and, and formulas and all that stuff, let's take some matrices where we know what they do.
We know that if we take a vector b, what this matrix does is it projects it down to Pb.
So is b an eigenvector in, in that picture?
Is that vector b an eigenvector?
No.
Not so, so b is not an eigenvector c- because Pb, its projection, is in a different direction.
So now tell me what vectors are eigenvectors of P?
What vectors do get projected in the same direction that they start?
So, so answer, tell me some x-s. Do you see what3 so it's if Ax equals lambda x, In this picture, where could I start with a vector b or x, do its projection, and end up in the same direction?
Well, that would happen if the vector was right in that plane already.
If the vector x was -- so let the vector x -- so any vector, any x in the plane will be an eigenvector.
And what will happen when I multiply by P, when I project a vector x -- I called it b here, because this is our familiar picture, but now I'm going to say that b was no good for, for the, for our purposes.
I'm interested in a vector x that's actually in the plane, and I project it, and what do I get back?
x, of course.
Doesn't move.
can be complex numbers.
So any x in the plane is unchanged by P, and what's that telling me?
That's telling me that x is an eigenvector, and it's also telling me what's the eigenvalue, which is -- just compare it with that.
The eigenvalue, the multiplier, is just one.
Good.
So we have actually a whole plane of eigenvectors.
Now I ask, are there any other eigenvectors?
And I expect the answer to be yes, because I would like to get three, if I'm in three dimensions, I would like to hope for three independent eigenvectors, two of them in the plane and one not in the plane.
OK.
So this guy b that I drew there was not any good.
What's the right eigenvector that's not in the plane?
The, the good one is the one that's perpendicular to the plane.
There's an, another good x, because what's the projection?
So these are eigenvectors.
Another guy here would be another eigenvector.
But now here is another one.
two.
Any x that's perpendicular to the plane, what's Px for that, for that, vector?
What's the projection of this guy perpendicular to the plane?
It is zero, of course.
So -- there's the null space.
Px and n- for those guys are zero, or zero x if we like, and the eigenvalue is zero.
So my answer to the question is, what are the eigenvalues for In our example, the one we worked out, a projection matrix?
There they are.
One and zero.
OK. We know projection matrices.
We can write them down as that A, A transpose, A inverse, A transpose thing, but without doing that from the picture we could see what are the eigenvectors.
OK. Are there other matrices?
Let me take a second example.
How about a permutation matrix?
What about the matrix, I'll call it A now.
Zero one, one zero.
A equals zero one one zero, that had eigenvalue one and Can you tell me a vector x -- see, we'll have a system soon enough, so I, I would like to just do these e- these couple of examples, just to see the picture before we, before we let it all, go into a system where that, matrix isn't anything special.
Because it is special.
And what, so what vector could I multiply by and end up in the same direction?
Can you spot an eigenvector for this guy?
That's a matrix that permutes x1 and x2, right?
It switches the two components of x.
How could the vector with its x2 x1, with -- permuted turn out to be a multiple of x1 x2, the vector we start with?
Can you tell me an eigenvector here for this guy?
x equal -- what is -- actually, can you tell me one vector that which is lambda x, and I have a three x, And of course you -- everybody knows that they're -- what, has eigenvalue one?
So what, what vector would have eigenvalue one, just above what we2 found here.
so that if I, if I permute it it doesn't change?
right?
There, that could be one one, thanks.
One one.
OK, take that vector one one.
That will be an eigenvector, because if I do Ax I get one one.
So that's the eigenvalue is one.
Great.
That's one eigenvalue.
But I have here a two by two matrix, and I figure there's going to be a second eigenvalue.
And eigenvector.
Now, what about that?
What's a vector, OK, maybe we can just, like, guess it.
A vector that the other -- actually, this one that I'm thinking of is going to be a vector that has eigenvalue minus one.
That's going to be my other eigenvalue for this matrix.
It's a -- notice the nice positive or not negative matrix, but an eigenvalue is going to come out negative.
And can you guess, spot the x that will work for Times x is supposed to give me zero, right?
that?
So I want a, a vector.
When I multiply by A, which reverses the two components, I want the thing to come out minus the original.
So what shall I send in in that case?
If I send in negative one one.
Then when I apply A, I get I do that multiplication, and I get one negative one, so it reversed sign.
So Ax is -x. Lambda is minus one.
Ax -- so Ax was x there and Ax is minus x here.
Can I just mention, like, jump ahead, have, give a perfectly innocent-looking quadratic and point out a special little fact about eigenvalues.
n by n matrices will have n eigenvalues.
And I get this matrix4 zero zero zero one, And it's not like -- suppose n is three or four or more.
It's not so easy to find them.
We'd have a third degree or a fourth degree or an n-th degree equation.
But here's one nice fact.
There, there's one pleasant fact.
we -- the eigenvalues came out four and two.
That the sum of the eigenvalues equals the sum down the diagonal.
That's called the trace, and I put that in the lecture Now I add three I to that matrix.
content specifically.
So this is a neat fact, the fact that sthe sum of the lambdas, add up the lambdas, equals the sum -- what would you like me to, shall I write that down?
What I'm want to say in words is the sum down the diagonal of A. Shall I write a11+a22+...+ ann.
That's add up the diagonal entries.
In this example, it's zero.
In other words, once I found this eigenvalue of one, I knew the other one had to be minus one in this two by two case, because in the two by two case, which is a good one to, to, play with, the trace tells you right away what the other eigenvalue is.
So if I tell you one eigenvalue, you can tell me the other one.
We'll, we'll have that -- we'll, minus one and eigenvectors one one and eigenvector minus one we'll see that again.
OK. Now can I -- I could give more examples, but maybe it's time to face the, the equation, Ax equal lambda x, and figure how are we going to find x and lambda.
And that is lambda one times lambda3 OK.
So this, so the question now is how to find eigenvalues and eigenvectors.
How to solve, how to solve Ax equal lambda x from the three x, so it's just I mean, when we've got two unknowns both in the equation.
OK.
Here's the trick.
Simple idea.
Bring this onto the same side.
Rewrite.
Bring this over as A minus lambda times the identity x One.
equals zero.
Right?
I have Ax minus lambda x, so I brought that over and I've got zero left on the, on the right-hand side.
What's the relation between that problem and -- let me write OK.
I don't know lambda and I don't know x, but I do know something here.
What I know is if I, if I'm going to be able to solve this thing, for some x that's not the zero vector, that's not, that's a useless eigenvector, doesn't count.
What I know now is that this matrix must be what?
If I'm going to be -- if there is an x -- I don't -- right now I don't know what it is.
I'm going to find lambda first, actually.
And -- but if there is an x, it tells me that this matrix, this special combination, which is like the matrix A with lambda -- shifted by lambda, shifted by lambda I, that it has to be singular.
This matrix must be singular, otherwise the only x would be the zero x, and zero matrix.OK.
So this is singular.
And what do I now know about singular matrices?
So, so take three away.
Their determinant is zero.
So I've -- so from the fact that that has to be singular, I know that the determinant of A minus lambda I has to be zero.
And that, now I've got x out of it.
I've got an equation for lambda, that the key equation -- it's called the characteristic equation or the eigenvalue equation.
And that -- in other words, I'm now in a position to find lambda first.
So -- the idea will be to find lambda first.
And actually, I won't find one lambda, I'll find N different lambdas.
Well, n lambdas, maybe not n different ones.
A lambda could be repeated.
A repeated lambda is the source of all trouble in 18.06.
So, let's hope for the moment that they're not repeated.
There, there they were different, right?
One and minus one in that, in that, for that permutation.
OK.
So and after I found this lambda, can I just look ahead?
How I going to find x?
After I have found this lambda, the lambda being this -- one of the numbers that makes this matrix singular.
Their product was eight.
Then of course finding x is just by elimination.
Right?
It's just -- now I've got a singular matrix, I'm looking for the null space.
We're experts at finding the null space.
You know, you do elimination, you identify the, the, the pivot columns and so on, you're -- and, give values to the free variables.
Probably there'll only be one free variable.
We'll give it the value one, like there.
And we find the other variable.
OK.
So let's -- find the x second will be a doable job.
That's my big equation for x.
Let's go, let's look at the first job of finding lambda.
Can I take another example?
OK. And let's, let's work that one out.
OK.
So let me take the example, say, let me make it easy.
it's just sitting there.
Three three one and one.
what do you know about the complex numbers?
So I've made it easy.
I've made it two by two.
I've made it symmetric.
And I even made it constant down the diagonal.
That a matrix, a perfectly real matrix could So that -- so the more, like, special properties I stick into the matrix, the more special outcome I get for the eigenvalues.
For example, this symmetric matrix, I know that it'll come out with real eigenvalues.
one.
The eigenvalues will turn out to be nice real numbers.
And up in our previous example, that was a symmetric matrix.
Actually, while we're at it, that was a symmetric matrix.
Its eigenvalues were nice real numbers, one and minus one.
And do you notice anything about its eigenvectors?
And what do you notice?
Anything particular about those two vectors, one one and minus And now comes that thing that I wanted to be reminded of.
one one?
They just happen to be -- no, I can't say they just happen to be, because that's the whole point, is that they had to be -- what?
What are they?
They're perpendicular.
The vector, when I -- if I see a vector one one and a one -- and a minus one one, my mind immediately takes that dot product.
It's zero.
what's the determinant of that matrix?
Those vectors are perpendicular.
That'll happen here too.
Well, let's find the eigenvalues.
Actually, oh, my example's too easy.
My example is too easy.
Let me tell you in advance what's going to happen.
May I?
Or shall I do the determinant of A minus lambda, and then point out at the end?
Will you remind me at the -- after I've found the eigenvalues to say why they were -- why they were easy from That -- it had to be eight, because we factored into lambda the, from the example we did?
OK, let's do the job here.
Let's compute determinant of A minus lambda I.
So that's a determinant.
And what's, what is this thing?
It's the matrix A with lambda removed from the diagonal.
for this matrix?
So the diagonal matrix is shifted, and then I'm taking the determinant.
OK.
So I multiply this out.
So what is that determinant?
Do you notice, I didn't take lambda away from all the entries.
It's lambda I, so it's lambda along the Lambda plus three x. diagonal.
So I get three minus lambda squared and then minus one, right?
And I want that to be zero.
And what is A minus lambda I x?
Well, I'm going to simplify it.
And what will I get?
So if I multiply this out, I get lambda squared minus six What's -- how is this matrix related to that matrix?
lambda plus what?
Plus eight.
But it's out there.
And that I'm going to set to zero.
And I'm going to solve it.
So and it's, it's a quadratic equation.
I can use factorization, I can use the quadratic formula.
I'll get two lambdas.
Before I do it, tell me what's that number six that's showing up in this equation?
It's the trace.
That number six is three plus three.
And while we're at it, what's the number eight that's showing up in this equation?
It's the determinant.
That our matrix has determinant eight.
So in a two by two case, it's really nice.
It's lambda squared minus the trace times lambda -- the trace is the linear coefficient -- and plus the determinant, the constant term.
OK.
So let's -- can, can we find the roots?
I guess the easy way is to factor that as something times something.
If we couldn't factor it, then we'd have to use the old b^2-4ac formula, but I, I think we can factor that into lambda minus what times lambda minus what?
Can you do that factorization?
Four and two?
Lambda minus four times lambda minus two.
So the, the eigenvalues are four and two.
So the eigenvalues are -- one eigenvalue, lambda one, Now I'm looking for x, the eigenvector.
let's say, is four.
Lambda two, the other eigenvalue, is two.
The eigenvalues are four and two.
And then I can go for the eigenvectors.
Suppose I have a matrix A, and Ax equal lambda x. equals zero.
You see I got the eigenvalues first.
So if they, if this had eigenvalue lambda, Four and two.
Now for the eigenvectors.
So what are the eigenvectors?
They're these guys in the null space when I take away, when I make the matrix singular by taking four I or two I away.
So we're -- we got to do those separately.
I'll -- let me find the eigenvector for four first.
So I'll subtract four, so A minus four I is -- so taking four away will put minus ones there.
And what's the point about that matrix?
If four is an eigenvalue, then A minus four I had better be a what kind of matrix?
Singular.
If that matrix isn't singular, the four wasn't correct.
But we're OK, that matrix is singular.
And what's the x now?
The x is in the null space.
So what's the x1 that goes with, with the lambda one?
eigenvalue, eigenvector, eigenvalue for this, So that A -- so this is -- now I'm doing A x1 is lambda one x1.
So I took A minus lambda one I, that's this matrix, and now I'm looking for the x1 in its null space, and who is he?
What's the vector x in the null space?
Of course it's one one.
So that's the eigenvector that goes with that eigenvalue.
So, so now -- Let's just spend one more minute on this bad Now how about the eigenvector that goes with the other eigenvalue?
Can I do that with, with erasing?
I take A minus two I.
So now I take two away from the diagonal, and that leaves me with a one and a one.
So A minus two I has again produced a singular matrix, as it had to.
I'm looking for the null space of that guy.
What vector is in its null space?
Well, of course, a whole line of vectors.
So when I say the eigenvector, I'm not speaking correctly.
There's a whole line of eigenvectors, and you just -- I just want a basis.
And for a line I just want one vector.
But -- You could, you're -- there's some freedom in choosing that one, but choose a reasonable one.
What's a vector in the null space of that?
Well, the natural vector to pick as the eigenvector with, with lambda two is minus one one.
If I did elimination on that vector and set that, the free variable to be one, I would get minus one and get that eigenvector.
So you see then that I've got eigenvector, Now the other neat fact is that the determinant, How are those two matrices related?
Well, one is just three I more than the other one, right?
two.
I just took that matrix and I -- I took this matrix and I added three I.
So my question is, what happened to the minus four times lambda minus two.
eigenvalues and what happened to the eigenvectors?
That's the, that's like the question we keep asking now in this chapter.
If I, if I do something to the matrix, what happens if I -- or I know something about the matrix, what's the what's the conclusion for its eigenvectors and eigenvalues?
Because -- those eigenvalues and eigenvectors are going to tell us important information about the matrix.
And here what are we seeing?
What's happening to these eigenvalues, one and minus one, when I add three I?
It just added three to the eigenvalues.
I got four and two, three more than one and minus one.
What happened to the eigenvectors?
Nothing at all.
One one is -- and minus -- and one -- and minus one one are -- is still the eigenvectors.
In other words, simple but useful observation.
If I add three I to a matrix, its eigenvectors don't change and its eigenvalues are three bigger.
Let's, let's just see why.
Let me keep all this on the same board.
but just so you see -- so I'll try to do that.
this has eigenvalue lambda plus three.
And x, the eigenvector, is the same x for both matrices.
OK.
So that's, great.
Of course, it's special.
We got the new matrix by adding three I.
Suppose I had added another matrix.
Suppose I know the eigenvalues and eigenvectors of A.
So I took A minus lambda I x, and what kind of a matrix I So this is, this, this little board here is going to be not so great.
Suppose I have a matrix A and it has an eigenvector x with an eigenvalue lambda.
You remember, I solve A minus lambda I x And now I add on some other matrix.
So, so what I'm asking you is, if you know the eigenvalues of A and you know the eigenvalues of B, let me say suppose B -- so this is if -- let me put an if here.
If Ax equals lambda x, fine, and B has, eigenvalues, has eigenvalues -- what shall we call them?
Alpha, alpha one and alpha -- let's say -- I'll use alpha for the eigenvalues of B for no good reason.
What a- you see what I'm going to ask is, how about A plus B?
Let me, let me give you the, let me give you, what you might think first.
OK.
If Ax equals lambda x and if B has an eigenvalue alpha, then I allowed to say -- what's the matter with this argument?
That gave us the constant term eight.
It's wrong.
What I'm going to write up is wrong.
I'm going to say Bx is alpha x.
Add those up, and you get A plus B x equals lambda plus alpha x.
So you would think that if you know the eigenvalues of A and you knew the eigenvalues of B, then if you added you would know the eigenvalues of A plus B.
But that's false.
A plus B -- well, when B was three I, that worked great.
But this is not so great.
And what's the matter with that argument there?
We have no reason to believe that x is also an eigenvector of B has some eigenvalues, B. but it's got some different eigenvectors normally.
It's a different matrix.
I don't know anything special.
If I don't know anything special, then as far as I know, it's got some different eigenvector y, and when I add I get just rubbish.
I mean, I get -- I can add, but I don't learn anything.
So not so great is A plus B.
Or A times B.
Normally the eigenvalues of A plus B or A times B are not eigenvalues of A plus eigenvalues of B. Ei- eigenvalues are not, like, linear.
Or -- and they don't multiply.
Because, eigenvectors are usually different and, and there's just no way to find out what A plus B does to affect What do I do now?
it.
OK.
So that's, like, a caution.
Don't, if B is a multiple of the identity, great, but if B is some general matrix, then for A plus B you've got to find -- you've got to solve the eigenvalue problem.
Now I want to do another example that brings out a, OK. another point about eigenvalues.
Let me make this example a rotation matrix.
possibility of complex numbers.
OK.
So here's another example.
So a rotate -- oh, I'd better call it Q. I often use Q for, for, rotations because those are the, like, very important examples of orthogonal matrices.
Let me make it a ninety degree rotation.
So -- my matrix is going to be the one that rotates every And that's the sum, that's lambda one plus lambda vector by ninety degrees.
So do you remember that matrix?
It's the cosine of ninety degrees, which is zero, the sine of ninety degrees, which is one, minus the sine of ninety, the cosine of ninety.
So that matrix deserves the letter Q.
It's an orthogonal matrix, very, very orthogonal matrix.
Now I'm interested in its eigenvalues and eigenvectors.
Two by two, it can't be that tough.
We know that the eigenvalues add to zero.
Actually, we know something already here.
The eigen- what's the sum of the two eigenvalues?
Just tell me what I just said.
Zero, right.
From that trace business.
The sum of the eigenvalues is, is going to come out zero.
And the product of the eigenvalues, did I tell you about the determinant being the product of the eigenvalues?
No.
But that's a good thing to know.
We pointed out how that eight appeared in the, in the quadratic equation.
eigenvalues, we can postpone that evil day, So let me just say this.
The trace is zero plus zero, obviously.
And that was the determinant.
OK. What I'm leading up to with this example is that something's going to go wrong.
Something goes wrong for rotation because what vector can come out parallel to itself after a rotation?
If this matrix rotates every vector by ninety degrees, what could be an eigenvector?
Do you see we're, we're, we're going to have trouble.
eigenvectors are -- Well.
Our, our picture of eigenvectors, of, of coming out in the same direction that they went in, there won't be it.
And with, and with eigenvalues we're going to have trouble.
From these equations.
Let's see.
Why I expecting trouble?
The, the first equation says that the eigenvalues add to zero.
So there's a plus and a minus.
So I take the eigenvalue.
But then the second equation says that the product is plus one.
We're in trouble.
But there's a way out.
So how -- let's do the usual stuff.
Look at determinant of Q minus lambda I.
So I'll just follow the rules, take the determinant, subtract lambda from the diagonal, where I had zeros, the rest is the same.
Rest of Q is just copied.
Compute that determinant.
OK, so what does that determinant equal?
Lambda squared minus minus one plus what?
What's up?
There's my equation.
My equation for the eigenvalues is lambda squared plus one equals zero.
What are the eigenvalues lambda one and lambda two?
They're I, whatever that is, and minus it, right.
Those are the right numbers.
To be real numbers even though the matrix was perfectly real.
So this can happen.
Complex numbers are going to -- have to enter eighteen oh six at this moment.
Boo, right.
All right.
If I just choose good matrices that have real supposed to have here?
We do know a little information about the, the two complex numbers.
They're complex conjugates of each other.
If, if lambda is an eigenvalue, then when I change, when I go -- you remember what complex conjugates are?
You switch the sign of the imaginary part.
Well, this was only imaginary, had no real part, so we just switched its sign.
So that eigenvalues come in pairs like that, but they're complex.
A complex conjugate pair.
And that can happen with a perfectly real matrix.
And as a matter of fact -- so that was my, my point earlier, that if a matrix was symmetric, it wouldn't happen.
So if we stick to matrices that are symmetric or, like, close to symmetric, then the eigenvalues will stay real.
But if we move far away from symmetric -- and that's as far as you can move, because that matrix is -- how is Q transpose related to Q for that matrix?
That matrix is anti-symmetric.
Q transpose is minus Q.
That's the very opposite of symmetry.
When I flip across the diagonal I get -- I reverse all the signs.
Those are the guys that have pure imaginary eigenvalues.
So they're the extreme case.
And in between are, are matrices that are not symmetric or anti-symmetric but, but they have partly a symmetric part and an anti-symmetric part.
OK.
So I'm doing a bunch of examples here to show the possibilities.
The good possibilities being perpendicular eigenvectors, real eigenvalues.
The bad possibilities being complex eigenvalues.
We could say that's bad.
There's another even worse.
I'm getting through the, the bad things here today.
Then, then the next lecture can, can, can be like pure happiness.
OK.
Here's one more bad thing that could happen.
So I, again, I'll do it with an example.
Suppose my matrix is, suppose I take this three three one and I change that guy to zero.
What are the eigenvalues of that matrix?
What are the eigenvectors?
This is always our question.
Of course, the next section we're going to show why are, why do we care.
But for the moment, this lecture is introducing them.
And let's just find them.
OK. What are the eigenvalues of that matrix?
Let me tell you -- at a glance we could answer that question.
Because the matrix is triangular.
It's really useful to know -- if you've got properties like a triangular matrix.
It's very useful to know you can read the eigenvalues off.
They're right on the diagonal.
So the eigenvalue is three and also three.
Three is a repeated eigenvalue.
But let's see that happen.
Let me do it right.
The determinant of A minus lambda I, what I always have to do is this determinant.
I take away lambda from the diagonal.
I leave the rest.
I compute the determinant, so I get a three minus lambda times a three minus lambda.
And nothing.
So that's where the triangular part came in.
Triangular part, the one thing we know about triangular matrices is the determinant is just the product down the diagonal.
And in this case, it's this same, repeated -- so lambda one is one -- sorry, lambda one is three and lambda two is three.
That was easy.
I mean, no -- why should I be pessimistic about a matrix whose eigenvalues can be read off right away?
The problem with this matrix is in the eigenvectors.
So let's go to the eigenvectors.
So how do I find the eigenvectors?
I'm looking for a couple of eigenvectors.
Singular, right?
It's supposed to be singular.
And then it's got some vectors -- which it is.
So it's got some vector x in the null space.
And what, what's the, what's -- give me a basis for the null space for that guy.
Tell me, what's a vector x in the null space, so that'll be the, the eigenvector that goes with lambda one equals three.
The eigenvector is -- so what's in the null space?
One zero, is it?
Great.
Now, what's the other eigenvector?
What's, what's the eigenvector that goes with lambda two?
Well, lambda two is three again.
So I get the same thing again.
Give me another vector -- I want it to be independent.
If I'm going to write down an x2, I'm never going to let it be dependent on x1.
I'm looking for independent eigenvectors, and what's the conclusion?
There isn't one.
This is a degenerate matrix.
It's only got one line of eigenvectors instead of two.
It's this possibility of a repeated eigenvalue opens this further possibility of a shortage of eigenvectors.
And so there's no second independent eigenvector x2.
So it's a matrix, it's a two by two matrix, but with only one independent eigenvector.
So that will be -- the matrices that -- where eigenvectors are -- don't give the complete story.
OK. My lecture on Monday will give the complete story for all the other matrices.
Thanks.
Have a good weekend.
A real New England weekend.
OK. Shall we start?
This is the second lecture on eigenvalues.
So the first lecture was -- reached the key equation, A x equal lambda x. x is the eigenvector and lambda's the eigenvalue.
Now to use that.
And the, the good way to, after we've found -- so, so job one is to find the eigenvalues and find the eigenvectors.
Now after we've found them, what do we do with them?
Well, the good way to see that is diagonalize the matrix.
So the matrix is A.
And I want to show -- first of all, this is like the basic fact.
This, this formula.
That's, that's the key to today's lecture.
This matrix A, I put its eigenvectors in the columns of a matrix S. So S will be the eigenvector matrix.
And I want to look at this magic combination S inverse A S. So can I show you how that -- what happens there?
And notice, there's an S inverse.
We have to be able to invert this eigenvector matrix S. So for that, we need n independent eigenvectors.
So that's the, that's the case.
OK.
So suppose we have n linearly independent eigenvectors of A.
Put them in the columns of this matrix S. So I'm naturally going to call that the eigenvector matrix, because it's got the eigenvectors in its columns.
And all I want to do is show you what happens when you multiply A times S. So A times S. So this is A times the matrix with the first eigenvector in its first column, the second eigenvector in its second column, the n-th eigenvector in its n-th column.
And how I going to do this matrix multiplication?
Well, certainly I'll do it a column at a time.
And what do I get.
A times the first column gives me the first column of the answer, but what is it?
That's an eigenvector.
A times x1 is equal to the lambda times the x1.
And that lambda's we're -- we'll call lambda one, of course.
So that's the first column.
Ax1 is the same as lambda one x1.
A x2 is lambda two x2.
So on, along to in the n-th column we now how lambda n xn.
Looking good, but the next step is even better.
So for the next step, I want to separate out those eigenvalues, those, those multiplying numbers, from the x-s.
So then I'll have just what I want.
OK.
So how, how I going to separate out?
So that, that number lambda one is multiplying the first column.
So if I want to factor it out of the first column, I better put -- here is going to be x1, and that's going to multiply this matrix lambda one in the first entry and all zeros.
Do you see that that, that's going to come out right for the first column?
Because w- we remember how -- how we're going back to that original punchline.
That if I want a number to multiply x1 then I can do it by putting x1 in that column, in the first column, and putting that number there.
Th- u- what I going to have here?
I'm going to have lambda -- I'm going to have x1, x2, ... ,xn.
These are going to be my columns again.
I'm getting S back again.
I'm getting S back again.
But now what's it multiplied by, on the right it's multiplied by?
If I want lambda n xn in the last column, how do I do it?
Well, the last column here will be -- I'll take the last column, use these coefficients, put the lambda n down there, and it will multiply that n-th column and give me lambda n xn.
There, there you see matrix multiplication just working for us.
So I started with A S. I wrote down what it meant, A times each eigenvector.
That gave me lambda time the eigenvector.
And then when I peeled off the lambdas, they were on the right-hand side, so I've got S, my matrix, back again.
And this matrix, this diagonal matrix, the eigenvalue matrix, and I call it capital lambda.
Using capital letters for matrices and lambda to prompt me that it's, that it's eigenvalues that are in there.
So you see that the eigenvalues are just sitting down that diagonal?
If I had a column x2 here, I would want the lambda two in the two two position, in the diagonal position, to multiply that x2 and give me the lambda two x2.
That's my formula.
A S is S lambda.
OK. That's the -- you see, it's just a calculation.
Now -- I mentioned, and I have to mention again, this business about n independent eigenvectors.
As it stands, this is all fine, whether -- I mean, I could be repeating the same eigenvector, but -- I'm not interested in that.
I want to be able to invert S, and that's where this comes in.
This n independent eigenvectors business comes in to tell me that that matrix is invertible.
So let me, on the next board, write down what I've got.
A S equals S lambda.
And now I'm, I can multiply on the left by S inverse.
So this is really -- I can do that, provided S is invertible.
Provided my assumption of n independent eigenvectors is satisfied.
And I mentioned at the end of last time, and I'll say again, that there's a small number of matrices for -- that don't have n independent eigenvectors.
So I've got to discuss that, that technical point.
But the great -- the most matrices that we see have n di- n independent eigenvectors, and we can diagonalize.
This is diagonalization.
I could also write it, and I often will, the other way round.
If I multiply on the right by S inverse, if I took this equation at the top and multiplied on the right by S inverse, I could -- I would have A left here.
Now S inverse is coming from the right.
So can you keep those two straight?
A multiplies its eigenvectors, that's how I keep them straight.
So A multiplies S. A multiplies S. And then this S inverse makes the whole thing diagonal.
And this is another way of saying the same thing, putting the Ss on the other side of the equation.
A is S lambda S inverse.
So that's the, that's the new factorization.
That's the replacement for L U from elimination or Q R for -- from Gram-Schmidt.
And notice that the matrix -- so it's, it's a matrix times a diagonal matrix times the inverse of the first one.
It's, that's the combination that we'll see throughout this chapter.
This combination with an S and an S inverse.
OK. Can I just begin to use that?
For example, what about A squared?
What are the eigenvalues and eigenvectors of A squared?
That's a straightforward question with a, with an absolutely clean answer.
So let me, let me consider A squared.
So I start with A x equal lambda x.
And I'm headed for A squared.
So let me multiply both sides by A.
That's one way to get A squared on the left.
So -- I should write these if-s in here.
If A x equals lambda x, then I multiply by A, so I get A squared x equals -- well, I'm multiplying by A, so that's lambda A x.
That lambda was a number, so I just put it on the left.
And what do I -- tell me how to make that look better.
What have I got here for if, if A has the eigenvalue lambda and eigenvector x, what's up with A squared?
A squared x, I just multiplied by A, but now for Ax I'm going to substitute lambda x.
So I've got lambda squared x.
So from that simple calculation, I -- my conclusion is that the eigenvalues of A squared are lambda squared.
And the eigenvectors -- I always think about both of those.
What can I say about the eigenvalues?
They're squared.
What can I say about the eigenvectors?
They're the same.
The same x as in -- as for A.
Now let me see that also from this formula.
How can I see what A squared is looking like from this formula?
So let me -- that was one way to do it.
Let me do it by just taking A squared from that.
A squared is S lambda S inverse -- that's A -- times S lambda S inverse -- that's A, which is?
This is the beauty of eigenvalues, eigenvectors.
Having that S inverse and S is the identity, so I've got S lambda squared S inverse.
Do you see what that's telling me?
It's, it's telling me the same thing that I just learned here, but in the -- in a matrix form.
It's telling me that the S is the same, the eigenvectors are the same, but the eigenvalues are squared.
Because this is -- what's lambda squared?
That's still diagonal.
It's got little lambda one squared, lambda two squared, down to lambda n squared o- on that diagonal.
Those are the eigenvalues, as we just learned, of A squared.
OK.
So -- somehow those eigenvalues and eigenvectors are really giving you a way to -- see what's going on inside a matrix.
Of course I can continue that for -- to the K-th power, A to the K-th power.
If I multiply, if I have K of these together, do you see how S inverse S will keep canceling in the, in the inside?
I'll have the S outside at the far left, and lambda will be in there K times, and S inverse.
So what's that telling me?
That's telling me that the eigenvalues of A, of A to the K-th power are the K-th powers.
The eigenvalues of A cubed are the cubes of the eigenvalues of A.
And the eigenvectors are the same, the same.
OK.
In other words, eigenvalues and eigenvectors give a great way to understand the powers of a matrix.
If I take the square of a matrix, or the hundredth power of a matrix, the pivots are all over the place.
L U, if I multiply L U times L U times L U times L U a hundred times, I've got a hundred L Us.
I can't do anything with them.
But when I multiply S lambda S inverse by itself, when I look at the eigenvector picture a hundred times, I get a hundred or ninety-nine of these guys canceling out inside, and I get A to the hundredth is S lambda to the hundredth S inverse.
I mean, eigenvalues tell you about powers of a matrix in a way that we had no way to approach previously.
For example, when does -- when do the powers of a matrix go to zero?
I would call that matrix stable, maybe.
So I could write down a theorem.
I'll write it as a theorem just to use that word to emphasize that here I'm getting this great fact from this eigenvalue picture.
OK. A to the K approaches zero as K goes, as K gets bigger if what?
What's the w- how can I tell, for a matrix A, if its powers go to zero?
What's -- somewhere inside that matrix is that information.
That information is not present in the pivots.
It's present in the eigenvalues.
What do I need for the -- to know that if I take higher and higher powers of A, that this matrix gets smaller and smaller?
Well, S and S inverse are not moving.
So it's this guy that has to get small.
And that's easy to -- to understand.
The requirement is all eigenvalues -- so what is the requirement?
The eigenvalues have to be less than one.
Now I have to wrote that absolute value, because those eigenvalues could be negative, they could be complex numbers.
So I'm taking the absolute value.
If all of those are below one.
That's, in fact, we practically see why.
And let me just say that I'm operating on one assumption here, and I got to keep remembering that that assumption is still present.
That assumption was that I had a full set of, of n independent eigenvectors.
If I don't have that, then this approach is not working.
So again, a pure eigenvalue approach, eigenvector approach, needs n independent eigenvectors.
If we don't have n independent eigenvectors, we can't diagonalize the matrix.
We can't get to a diagonal matrix.
This diagonalization is only possible if S inverse makes sense.
OK. Can I, can I follow up on that point now?
So you see why -- what we get and, and why we want it, because we get information about the powers of a matrix just immediately from the eigenvalues.
OK. Now let me follow up on this, business of which matrices are diagonalizable.
Sorry about that long word.
So a matrix is, is sure -- so here's, here's the main point.
A is sure to be -- to have N independent eigenvectors and, and be -- now here comes that word -- diagonalizable if, if -- so we might as well get the nice case out in the open.
The nice case is when -- if all the lambdas are different.
That means, that means no repeated eigenvalues.
OK. That's the nice case.
If my matrix, and most -- if I do a random matrix in Matlab and compute its eigenvalues -- so if I computed if I took eig of rand of ten ten, gave, gave that Matlab command, the -- we'd get a random ten by ten matrix, we would get a list of its ten eigenvalues, and they would be different.
They would be distinct is the best word.
I would have -- a random matrix will have ten distinct -- a ten by ten matrix will have ten distinct eigenvalues.
And if it does, the eigenvectors are automatically independent.
So that's a nice fact.
I'll refer you to the text for the proof.
That, that A is sure to have n independent eigenvectors if the eigenvalues are different, if.
If all the, if all eigenvalues are different.
It's just if some lambdas are repeated, then I have to look more closely.
If an eigenvalue is repeated, I have to look, I have to count, I have to check.
Has it got -- say it's repeated three times.
So what's a possibility for the -- so here is the, here is the repeated possibility.
And, and let me emphasize the conclusion.
That if I have repeated eigenvalues, I may or may not, I may or may not have, have n independent eigenvectors.
I might.
I, I, you know, this isn't a completely negative case.
The identity matrix -- suppose I take the ten by ten identity matrix.
What are the eigenvalues of that matrix?
So just, just take the easiest matrix, the identity.
If I look for its eigenvalues, they're all ones.
So that eigenvalue one is repeated ten times.
But there's no shortage of eigenvectors for the identity matrix.
In fact, every vector is an eigenvector.
So I can take ten independent vectors.
Oh, well, what happens to everything -- if A is the identity matrix, let's just think that one through in our head.
If A is the identity matrix, then it's got plenty of eigenvectors.
I choose ten independent vectors.
They're the columns of S. And, and what do I get from S inverse A S?
I get I again, right?
If A is the identity -- and of course that's the correct lambda.
The matrix was already diagonal.
So if the matrix is already diagonal, then the, the lambda is the same as the matrix.
A diagonal matrix has got its eigenvalues sitting right there in front of you.
Now if it's triangular, the eigenvalues are still sitting there, but so let's take a case where it's triangular.
Suppose A is like, two one two zero.
So there's a case that's going to be trouble.
There's a case that's going to be trouble.
First of all, what are the -- I mean, we just -- if we start with a matrix, the first thing we do, practically without thinking is compute the eigenvalues and eigenvectors.
OK.
So what are the eigenvalues?
You can tell me right away what they are.
They're two and two, right.
It's a triangular matrix, so when I do this determinant, shall I do this determinant of A minus lambda I?
I'll get this two minus lambda one zero two minus lambda, right?
I take that determinant, so I make those into vertical bars to mean determinant.
And what's the determinant?
It's two minus lambda squared.
What are the roots?
Lambda equal two twice.
So the eigenvalues are lambda equals two and two.
OK, fine.
Now the next step, find the eigenvectors.
So I look for eigenvectors, and what do I find for this guy?
Eigenvectors for this guy, when I subtract two minus the identity, so A minus two I has zeros here.
And I'm looking for the null space.
What's, what are the eigenvectors?
They're the -- the null space of A minus lambda I.
The null space is only one dimensional.
This is a case where I don't have enough eigenvectors.
My algebraic multiplicity is two.
I would say, when I see, when I count how often the eigenvalue is repeated, that's the algebraic multiplicity.
That's the multiplicity, how many times is it the root of the polynomial?
My polynomial is two minus lambda squared.
It's a double root.
So my algebraic multiplicity is two.
But the geometric multiplicity, which looks for vectors, looks for eigenvectors, and -- which means the null space of this thing, and the only eigenvector is one zero.
That's in the null space.
Zero one is not in the null space.
The null space is only one dimensional.
So there's a matrix, my -- this A or the original A, that are not diagonalizable.
I can't find two independent eigenvectors.
There's only one.
OK.
So that's the case that I'm -- that's a case that I'm not really handling.
For example, when I wrote down up here that the powers went to zero if the eigenvalues were below one, I didn't really handle that case of repeated eigenvalues, because my reasoning was based on this formula.
And this formula is based on n independent eigenvectors.
OK. Just to say then, there are some matrices that we're, that, that we don't cover through diagonalization, but the great majority we do.
OK. And we, we're always OK if we have different distinct eigenvalues.
OK, that's the, like, the typical case.
Because for each eigenvalue there's at least one eigenvector.
The algebraic multiplicity here is one for every eigenvalue and the geometric multiplicity is one.
There's one eigenvector.
And they are independent.
OK. OK. Now let me come back to the important case, when, when we're OK.
The important case, when we are diagonalizable.
Let me, look at -- so -- let me solve this equation.
The equation will be each -- I start with some -- start with a given vector u0.
And then my equation is at every step, I multiply what I have by A.
That, that equation ought to be simple to handle.
And I'd like to be able to solve it.
How would I find -- if I start with a vector u0 and I multiply by A a hundred times, what have I got?
Well, I could certainly write down a formula for the answer, so what, what -- so u1 is A u0.
And u2 is -- what's u2 then?
u2, I multiply -- u2 I get from u1 by another multiplying by A, so I've got A twice.
And my formula is uk, after k steps, I've multiplied by A k times the original u0.
You see what I'm doing?
The next section is going to solve systems of differential equations.
I'm going to have derivatives.
This section is the nice one.
It solves difference equations.
I would call that a difference equation.
It's -- at first order, I would call that a first-order system, because it connects only -- it only goes up one level.
And I -- it's a system because these are vectors and that's a matrix.
And the solution is just that.
OK.
But, that's a nice formula.
That's the, like, the most compact formula I could ever get.
u100 would be A to the one hundred u0.
But how would I actually find u100?
How would I find -- how would I discover what u100 is?
Let me, let me show you how.
Here's the idea.
If -- so to solve, to really solve -- shall I say, to really solve -- to really solve it, I would take this initial vector u0 and I would write it as a combination of eigenvectors.
To really solve, write u nought as a combination, say certain amount of the first eigenvector plus a certain amount of the second eigenvector plus a certain amount of the last eigenvector.
Now multiply by A.
You want to -- you got to see the magic of eigenvectors working here.
Multiply by A.
So Au0 is what?
So A times that.
A times -- so what's A -- I can separate it out into n separate pieces, and that's the whole point.
That each of those pieces is going in its own merry way.
Each of those pieces is an eigenvector, and when I multiply by A, what does this piece become?
So that's some amount of the first -- let's suppose the eigenvectors are normalized to be unit vectors.
So that says what the eigenvector is.
It's a -- And I need some multiple of it to produce u0.
OK. Now when I multiply by A, what do I get?
I get c1, which is just a factor, times Ax1, but Ax1 is lambda one x1.
When I multiply this by A, I get c2 lambda two x2.
And here I get cn lambda n xn.
And suppose I multiply by A to the hundredth power now.
Can we, having done it, multiplied by A, let's multiply by A to the hundredth.
What happens to this first term when I multiply by A to the one hundredth?
It's got that factor lambda to the hundredth.
That's the key.
That -- that's what I mean by going its own merry way.
It, it is pure eigenvector.
It's exactly in a direction where multiplication by A just brings in a scalar factor, lambda one.
So a hundred times brings in this a hundred times.
Hundred times lambda two, hundred times lambda n. Actually, we're -- what are we seeing here?
We're seeing, this same, lambda capital lambda to the hundredth as in the, as in the diagonalization.
And we're seeing the S matrix, the, the matrix S of eigenvectors.
That's what this has got to -- this has got to amount to.
A lambda to the hundredth power times an S times this vector c that's telling us how much of each one is in the original thing.
So if, if I had to really find the hundredth power, I would take u0, I would expand it as a combination of eigenvectors -- this is really S, the eigenvector matrix, times c, the, the coefficient vector.
And then I would immediately then, by inserting these hundredth powers of eigenvalues, I'd have the answer.
So -- huh, there must be -- oh, let's see, OK.
It's -- so, yeah.
So if u100 is A to the hundredth times u0, and u0 is S c -- then you see this formula is just this formula, which is the way I would actually get hold of this, of this u100, which is -- let me put it here.
u100.
The way I would actually get hold of that, see what, what the solution is after a hundred steps, would be -- expand the initial vector into eigenvectors and let each eigenvector go its own way, multiplying by a hundred at -- by lambda at every step, and therefore by lambda to the hundredth power after a hundred steps.
Can I do an example?
So that's the formulas.
Now let me take an example.
I'll use the Fibonacci sequence as an example.
So, so Fibonacci example.
You remember the Fibonacci numbers?
If we start with one and one as F0 -- oh, I think I start with zero, maybe.
Let zero and one be the first ones.
So there's F0 and F1, the first two Fibonacci numbers.
Then what's the rule for Fibonacci numbers?
Ah, they're the sum.
The next one is the sum of those, so it's one.
The next one is the sum of those, so it's two.
The next one is the sum of those, so it's three.
Well, it looks like one two three four five, but somehow it's not going to do that way.
The next one is five, right.
Two and three makes five.
The next one is eight.
The next one is thirteen.
And the one hundredth Fibonacci number is what?
That's my question.
How could I get a formula for the hundredth number?
And, for example, how could I answer the question, how fast are they growing?
How fast are those Fibonacci numbers growing?
They're certainly growing.
It's not a stable case.
Whatever the eigenvalues of whatever matrix it is, they're not smaller than one.
These numbers are growing.
But how fast are they growing?
The answer lies in the eigenvalue.
So I've got to find the matrix, so let me write down the Fibonacci rule.
F(k+2) = F(k+1)+F k, right?
Now that's not in my -- I want to write that as uk plus one and Auk.
But right now what I've got is a single equation, not a system, and it's second-order.
It's like having a second-order differential equation with second derivatives.
I want to get first derivatives.
Here I want to get first differences.
So the way, the way to do it is to introduce uk will be a vector -- see, a small trick.
Let uk be a vector, F(k+1) and Fk.
So I'm going to get a two by two system, first order, instead of a one -- instead of a scalar system, second order, by a simple trick.
I'm just going to add in an equation F(k+1) equals F(k+1).
That will be my second equation.
Then this is my system, this is my unknown, and what's my one step equation?
So, so now u(k+1), that's -- so u(k+1) is the left side, and what have I got here on the right side?
I've got some matrix multiplying uk.
Can you, do -- can you see that all right?
if you can see it, then you can tell me what the matrix is.
Do you see that I'm taking my system here.
I artificially made it into a system.
I artificially made the unknown into a vector.
And now I'm ready to look at and see what the matrix is.
So do you see the left side, u(k+1) is F(k+2) F(k+1), that's just what I want.
On the right side, this remember, this uk here -- let me for the moment put it as F(k+1) Fk.
So what's the matrix?
Well, that has a one and a one, and that has a one and a zero.
There's the matrix.
Do you see that that gives me the right-hand side?
So there's the matrix A.
And this is our friend uk.
So we've got -- so that simple trick -- changed the second-order scalar problem to a first-order system.
Two b- u- with two unknowns.
With a matrix.
And now what do I do?
Well, before I even think, I find its eigenvalues and eigenvectors.
So what are the eigenvalues and eigenvectors of that matrix?
Let's see.
I always -- first let me just, like, think for a minute.
It's two by two, so this shouldn't be impossible to do.
Let's do it.
OK.
So my matrix, again, is one one one zero.
It's symmetric, by the way.
So what I will eventually know about symmetric matrices is that the eigenvalues will come out real.
I won't get any complex numbers here.
And the eigenvectors, once I get those, actually will be orthogonal.
But two by two, I'm more interested in what the actual numbers are.
What do I know about the two numbers?
Well, should do you want me to find this determinant of A minus lambda I?
Sure.
So it's the determinant of one minus lambda one one zero, right?
Minus lambda, yes.
God.
OK. OK.
There'll be two eigenvalues.
What will -- tell me again what I know about the two eigenvalues before I go any further.
Tell me something about these two eigenvalues.
What do they add up to?
Lambda one plus lambda two is?
Is the same as the trace down the diagonal of the matrix.
One and zero is one.
So lambda one plus lambda two should come out to be one.
And lambda one times lambda one times lambda two should come out to be the determinant, which is minus one.
So I'm expecting the eigenvalues to add to one and to multiply to minus one.
But let's just see it happen here.
If I multiply this out, I get -- that times that'll be a lambda squared minus lambda minus one.
Good.
Lambda squared minus lambda minus one.
Actually, I -- you see the b- compare that with the original equation that I started with.
F(k+2) - F(k+1)-Fk is zero.
The recursion that -- that the Fibonacci numbers satisfy is somehow showing up directly here for the eigenvalues when we set that to zero.
WK.
Let's solve.
Well, I would like to be able to factor that, that quadratic, but I'm better off to use the quadratic formula.
Lambda is -- let's see.
Minus b is one plus or minus the square root of b squared, which is one, minus four times that times that, which is plus four, over two.
So that's the square root of five.
So the eigenvalues are lambda one is one half of one plus square root of five, and lambda two is one half of one minus square root of five.
And sure enough, they -- those add up to one and they multiply to give minus one.
OK. Those are the two eigenvalues.
How -- what are those numbers approximately?
Square root of five, well, it's more than two but less than three.
Hmm.
It'd be nice to know these numbers.
I think, I think that -- so that number comes out bigger than one, right?
That's right.
This number comes out bigger than one.
It's about one point six one eight or something.
Not exactly, but.
And suppose it's one point six.
Just, like, I think so.
Then what's lambda two?
Is, is lambda two positive or negative?
Negative, right, because I'm -- it's, obviously negative, and I knew that the -- so it's minus -- and they add up to one, so minus point six one eight, I guess.
OK. A- and some more.
Those are the two eigenvalues.
One eigenvalue bigger than one, one eigenvalue smaller than one.
Actually, that's a great situation to be in.
Of course, the eigenvalues are different, so there's no doubt whatever -- is this matrix diagonalizable?
Is this matrix diagonalizable, that original matrix A?
Sure.
We've got two distinct eigenvalues and we can find the eigenvectors in a moment.
But they'll be independent, we'll be diagonalizable.
And now, you, you can already answer my very first question.
How fast are those Fibonacci numbers increasing?
How -- those -- they're increasing, right?
They're not doubling at every step.
Let me -- let's look again at these numbers.
Five, eight, thirteen, it's not obvious.
The next one would be twenty-one, thirty-four.
So to get some idea of what F one hundred is, can you give me any -- I mean the crucial number -- so it -- these -- it's approximately -- what's controlling the growth of these Fibonacci numbers?
It's the eigenvalues.
And which eigenvalue is controlling that growth?
The big one.
So F100 will be approximately some constant, c1 I guess, times this lambda one, this one plus square root of five over two, to the hundredth power.
And the two hundredth F -- in other words, the eigenvalue -- the Fibonacci numbers are growing by about that factor.
Do you see that we, we've got precise information about the, about the Fibonacci numbers out of the eigenvalues?
OK. And again, why is that true?
Let me go over to this board and s- show what I'm doing here.
The -- the original initial value is some combination of eigenvectors.
And then when we start -- when we start going out the theories of Fibonacci numbers, when we start multiplying by A a hundred times, it's this lambda one to the hundredth.
This term is, is the one that's taking over.
It's -- I mean, that's big, like one point six to the hundredth power.
The second term is practically nothing, right?
The point six, or minus point six, to the hundredth power is an extremely small, extremely small number.
So this is -- there're only two terms, because we're two by two.
This number is -- this piece of it is there, but it's, it's disappearing, where this piece is there and it's growing and controlling everything.
So, so really the -- we're doing, like, problems that are evolving.
We're doing dynamic u- instead of Ax=b, that's a static problem.
We're now we're doing dynamics.
A, A squared, A cubed, things are evolving in time.
And the eigenvalues are the crucial, numbers.
OK.
I guess to complete this, I better write down the eigenvectors.
So we should complete the, the whole process by finding the eigenvectors.
OK, well, I have to -- up in the corner, then, I have to look at A minus lambda I.
So A minus lambda I is this one minus lambda one one and minus lambda.
And now can we spot an eigenvector out of that?
That's, that's, for these two lambdas, this matrix is singular.
I guess the eigenvector -- two by two ought to be, I mean, easy.
So if I know that this matrix is singular, then u- seems to me the eigenvector has to be lambda and one, because that multiplication will give me the zero.
And this multiplication gives me -- better give me also zero.
Do you see why it does?
This is the minus lambda squared plus lambda plus one.
It's the thing that's zero because these lambdas are special.
There's the eigenvector.
x1 is lambda one one, and x2 is lambda two one.
I did that as a little trick that was available in the two by two case.
So now I finally have to -- oh, I have to take the initial u0 now.
So to complete this example entirely, I have to say, OK, what was u0?
u0 was F1 F0.
So u0, the starting vector is F1 F0, and those were one and zero.
So I have to use that vector.
So I have to look for, for a multiple of the first eigenvector and the second to produce u0, the one zero vector.
This is what will find c1 and c2, and then I'm done.
Do you -- so let me instead of, in the last five seconds, grinding out a formula, let me repeat the idea.
Because I'd really -- it's the idea that's central.
When things are evolving in time -- let me come back to this board, because the ideas are here.
When things are evolving in time by a first-order system, starting from an original u0, the key is find the eigenvalues and eigenvectors of A.
That will tell -- those eigenvectors -- the eigenvalues will already tell you what's happening.
Is the solution blowing up, is it going to zero, what's it doing.
And then to, to find out exactly a formula, you have to take your u0 and write it as a combination of eigenvectors and then follow each eigenvector separately.
And that's really what this formula, the formula for, -- that's what the formula for A to the K is doing.
So remember that formula for A to the K is S lambda to the K S inverse.
OK. That's, that's difference equations.
And you just have to -- so the, the homework will give some examples, different from Fibonacci, to follow through.
And next time will be differential equations.
Thanks.
-- and lift-off on differential equations.
So, this section is about how to solve a system of first order, first derivative, constant coefficient linear equations.
And if we do it right, it turns directly into linear algebra.
The key idea is the solutions to constant coefficient linear equations are exponentials.
So if you look for an exponential, then all you have to find is what's up there in the exponent and what multiplies the exponential and that's the linear algebra.
So -- and the result -- one thing we will fine -- it's completely parallel to powers of a matrix.
So the last lecture was about how would you compute A to the K or A to the 100?
How do you compute high powers of a matrix?
Now it's not powers anymore, but it's exponentials.
That's the natural thing for differential equation.
Okay.
But can I begin with an example?
And I'll just go through the mechanics.
How would I solve the differential -- two differential equations?
So I'm going to make it -- I'll have a two by two matrix and the coefficients are minus one two, one minus two and I'd better give you some initial condition.
So suppose it starts u at times zero -- this is u1, u2 -- let it -- let it -- suppose everything is in u1 at times zero.
So -- at -- at the start, it's all in u1.
But what happens as time goes on, du2/dt will -- will be positive, because of that u1 term, so flow will move into the u2 component and it will go out of the u1 component.
So we'll just follow that movement as time goes forward by looking at the eigenvalues and eigenvectors of that matrix.
That's a first job.
Before you do anything else, find the -- find the matrix and its eigenvalues and eigenvectors.
So let me do that.
Okay.
So here's our matrix.
Maybe you can tell me right away what -- what are the eigenvalues and -- eigenvalues anyway.
And then we can check.
But can you spot any of the eigenvalues of that matrix?
We're looking for two eigenvalues.
Do you see -- I mean, if I just wrote that matrix down, what -- what do you notice about it?
It's singular, right.
That -- that's a singular matrix.
That tells me right away that one of the eigenvalues is lambda equals zero.
I can -- that's a singular matrix, the second column is minus two times the first column, the determinant is zero, it's -- it's singular, so zero is an eigenvalue and the other eigenvalue will be -- from the trace.
I look at the trace, the sum down the diagonal is minus three.
That has to agree with the sum of the eigenvalue, so that second eigenvalue better be minus three.
I could, of course -- I could compute -- why don't I over here -- compute the determinant of A minus lambda I, the determinant of this minus one minus lambda two one minus two minus lambda matrix.
But we know what's coming.
When I do that multiplication, I get a lambda squared.
I get a two lambda and a one lambda, that's a three lambda.
And then -- now I'm going to get the determinant, which is two minus two which is zero.
So there's my characteristic polynomial, this determinant.
And of course I factor that into lambda times lambda plus three and I get the two eigenvalues that we saw coming.
What else do I need?
The eigenvectors.
So before I even think about the differential equation or what -- how to solve it, let me find the eigenvectors for this matrix.
Okay.
So take lambda equals zero -- so that -- that's the first eigenvalue.
Lambda one equals zero and the second eigenvalue will be lambda two equals minus three.
By the way, I -- I already know something important about this.
The eigenvalues are telling me something.
You'll see how it comes out, but let me point to -- these numbers are -- this eigenvalue, a negative eigenvalue, is going to disappear.
There's going to be an e to the minus three t in the answer.
That e to the minus three t as times goes on is going to be very, very small.
The other part of the answer will involve an e to the zero t. But e to the zero t is one and that's a constant.
So I'm expecting that this solution'll have two parts, an e to the zero t part and an e to the minus three t part, and that -- and as time goes on, the second part'll disappear and the first part will be a steady It won't move.
state.
It will be -- at the end of -- as t approaches infinity, this part disappears and this is the -- the e to the zero t part is what I get.
And I'm very interested in these steady states, so that's -- I get a steady state when I have a zero eigenvalue.
Okay.
What about those eigenvectors?
So what's the eigenvector that goes with eigenvalue zero?
Okay.
The matrix is singular as it is, the eigenvector is -- is the guy in the null space, so what vector is in the null space of that matrix?
Let's see.
I guess I probably give the free variable the value one and I realize that if I want to get zero I need a two up here.
Okay?
So Ax1 is zero x1.
A x1 is zero x1.
Fine.
Okay.
What about the other eigenvalue?
Lambda two is minus three.
Okay.
How do I get the other eigenvalue, then?
For the moment -- can I mentally do it?
I subtract minus three along the diagonal, which means I add three -- can I -- I'll just do it with an erase -- erase for the moment.
So I'm going to add three to the diagonal.
So this minus one will become a two and -- I'll make it in big loopy letters -- and when I add three to this guy, the minus two becomes -- well, I can't make one very loopy, but how's that?
Okay.
Now that's A minus three I -- A plus three I, sorry.
That's A plus three I.
It's supposed to be singular, right?
I-- if things -- if I did it right, this matrix should be singular and the x2, the eigenvector should be in its null space.
Okay.
What do I get for the null space of this?
Maybe minus one one, or one minus one.
Doesn't matter.
Those are both perfectly good.
Right?
Because that's in the null space of this.
Now I'll -- because A times that vector is three times that vector.
Ax2 is minus three x2.
Good.
Okay.
Can I get A again so we see that correctly?
That was a minus one and that was a minus two.
Good.
Okay.
That -- that's the first job.
eigenvalues and eigenvectors.
And already the eigenvalues are telling me the most important information about the answer.
But now, what is the answer?
The answer is -- the solution will be U of T -- okay.
Now, wh- now I use those eigenvalues and eigenvectors.
The solution is some -- there are two eigenvalues.
So I -- it -- so there're going to be two special solutions here.
Two pure exponential solutions.
The first one is going to be either the lambda one tx1 and the -- so that solves the equation, and so does this one.
They both are solutions to the differential equation.
That's the general solution.
The general solution is a combination of that pure exponential solution and that pure exponential solution.
Can I just see that those guys do solve the equation?
So let me just check -- check on this one, for example.
I -- I want to check that the -- my equation -- let's Check.
remember, the equation -- du/dt is Au.
I plug in e to the lambda one t x1 and let's just see that the equation's okay.
I believe this is a solution to that equation.
So just plug it in.
On the left-hand side, I take the time derivative -- so the left-hand side will be lambda one, e to the lambda one t x1, right?
The time derivative -- this is the term that depends on time, it's just ordinary exponential, its derivative brings down a lambda one.
On the other side of the equation it's A times this thing.
A times either the lambda one t x one, and does that check out?
Do we have equality there?
Yes, because either the lambda one t appears on both sides and the other one is Ax1 equal lambda one x1 -- check.
Do you -- so, the -- we've come to the first point to remember.
These pure solutions.
Those pure solutions are the -- those pure exponentials are the differential equations analogue of -- last time we had pure powers.
Last time -- so -- so last time, the analog was lambda -- lambda one to the K-th power x1, some amount of that, plus some amount of lambda two to the K-th power x2.
That was our formula from last time.
I put it up just to -- so your eye compares those two formulas.
Powers of lambda in the -- in the difference equation -- that -- this was in the -- this was for the equation uk plus one equals A uk.
That was for the finite step -- stepping by one.
And we got powers, now this is the one we're interested in, the exponentials.
So -- so that's -- that's the solution -- what are c1 and c2?
Then we're through.
What are c1 and c2?
Well, of course we know these actual things.
Let me just -- let me come back to this.
c1 is -- we haven't figured out yet, but e to the lambda one t, the lambda one is zero so that's just a one times x1 which is two one.
So it's c1 times this one that's not moving times the vector, the eigenvector two one and c2 times -- what's e to the lambda two t?
Lambda two is minus three.
So this is the term that has the minus three t and its eigenvector is this one minus one.
So this vector solves the equation and any multiple of it.
This vector solves the equation if it's got that factor e to the minus three t. We've got the answer except for c1 and c2.
So -- so everything I've done is immediate as soon as you know the eigenvalues and eigenvectors.
So how do we get c1 and c2?
That has to come from the initial condition.
So now I -- now I use -- u of zero is given as one zero.
So this is the initial condition that will find c1 and c2.
So let me do that on the board underneath.
At t equals zero, then -- I get c1 times this guy plus c2 and now I'm at times zero.
So that's a one and this is a one minus one and that's supposed to agree with u of zero one zero.
Okay.
That should be two equations.
That should give me c1 and c2 and then I'm through.
So what are c1 and c2?
Let's see.
I guess we could actually spot them by eye or we could solve two equations in two unknowns.
Let's see.
If these were both ones -- so I'm just adding -- then I would get three zero.
So what's the -- what's the solution, then?
If -- if c1 and c2 are both ones, I get three zero, so I want, like, one third of that, because I want to get one zero.
So I think it's c1 equals a third, c2 equals a third.
So finally I have the answer.
Let me keep it in the -- in this board here.
Finally the answer is one third of this plus one third of this.
Do you see what -- what's actually happening with this flow?
This flow started out at -- the solution started out at one zero.
Started at one zero.
Then as time went on, people moved, essentially.
Some fraction of this one moved here.
And -- and in the limit, there's -- there's the limit, as -- right?
As t goes to infinity, as t gets very large, this disappears and this is the steady state.
So the steady state is -- so the steady state -- u -- we could call it u at infinity is one third of two and one.
It's -- it's two thirds of one third.
So that's the -- we really -- I mean, you're getting, like, total, insight into the behavior of the solution, what the differential equation does.
Of course, we don't -- wouldn't always have a steady state.
Sometimes we would approach zero.
Sometimes we would blow up.
Can we straighten out those cases?
The eigenvalue should tell us.
So when do we get -- so -- so let me ask first, when do we get stability?
That's u of t going to zero.
When would the solution go to zero no matter what the initial condition is?
Negative eigenvalues, right.
Negative eigenvalues.
But now I have to -- I have to ask you for one more step.
Suppose the eigenvalues are complex numbers?
Because we know they could be.
Then we want stability -- this -- this -- we want -- we need all these e to the lambda t-s all going to zero and somehow that asks us to have lambda negative.
But suppose lambda is a complex number?
Then what's the test?
What -- if lambda's a complex number like, oh, suppose lambda is negative plus an imaginary part?
Say lambda is minus three plus six i?
What -- what happens then?
Can we just, like, do a -- a case here?
If -- if this lambda is minus three plus six it, how big is that number?
Does this -- does this imaginary part play a -- play a -- play a role here or not?
Or how big is -- what's the absolute value of that -- of that quantity?
It's just e to the minus three t, right?
Because this other part, this -- the -- the magnitude -- the -- this -- e to the six it -- what -- that has absolute value one.
Right?
That's just this cosine of six t plus i, sine of six t. And the absolute value squared will be cos squared plus sine squared will be one.
This is -- this complex number is running around the unit circle.
This com- this -- the -- it's the real part that matters.
This is what I'm trying to do.
Real part of lambda has to be negative.
If lambda's a complex number, it's the real part, the minus three, that either makes us go to zero or doesn't, or let -- or blows up.
The imaginary part won't -- will just, like, oscillate between the two components.
Okay.
So that's stability.
And what about -- what about a steady state?
When would we have, a steady state, always in the same direction?
So let me -- I'll take this part away -- when -- so that was, like, checking that it's -- that it's the real part that we care about.
Now, we have a steady state when -- when lambda one is zero and the other eigenvalues have what?
So I'm looking -- like, that example was, like, perfect for a steady state.
We have a zero eigenvalue and the other eigenvalues, we want those to disappear.
So the other eigenvalues have real part negative.
And we blow up, for sure -- we blow up if any real part of lambda is positive.
So if I -- if I reverse the sign of A -- of the matrix A -- suppose instead of the matrix I had, the A that I had, I changed it -- I changed all its sines.
What would that do to the eigenvalues and eigenvectors?
If I -- if -- if I know the eigenvalues and eigenvectors of A, tell me about minus A.
What happens to the eigenvalues?
For minus A, they'll all change sine.
So I'll have blow up.
This -- instead of the e to the minus three t, if I change that to minus -- if I change the sines in that matrix, I would change the trace to plus three, I would have an e to the plus three t and I would have blow up.
Of course the zero eigenvalue would stay at zero, but the other guy is taking off in -- if I reversed all the sines.
Okay.
So this is -- this is the stability picture.
And for a two by two matrix, we can actually pin down even more closely what that means.
Can I -- let -- can I do that?
Let me do that -- I want to -- for a two by two matrix, I can tell whether the real part of the eigenvalues is negative, I -- well, let me -- let me tell you what I have in mind for that.
So two by two stability -- let me -- this is just a little comment.
Two by two stability.
So my matrix, now, is just a b c d and I'm looking for the real parts of both eigenvalues to be negative.
Okay.
What -- how can I tell from looking at the matrix, without computing its eigenvalues, whether the two eigenvalues are negative, or at least their real parts are negative?
What would that tell me about the trace?
So -- so the trace -- that's this a plus d -- what can you tell me about the trace in the case of a two by two stable matrix?
That means the eigenvalues have -- are negative, or at least the real parts of those eigenvalues are negative -- then, when I take the -- when I look at the matrix and find its trace, what -- what do I know about that?
It's negative, right.
This is the sum of the -- this equals -- this equals lambda one plus lambda two, so it's negative.
The two eigenvalues, by the way, will have -- if they're complex -- will have plus six i and minus six i.
The complex parts will -- will be conjugates of each other and disappear when we add and the trace will be negative.
Okay, the trace has to be negative.
Is that enough -- is a negative trace enough to make the matrix stable?
Shouldn't be enough, right?
Can I -- can you make -- what's a matrix that has a negative trace but still it's not stable?
So it -- it has a blow -- it still has a blow-up factor and a -- and a -- and a decaying one.
So what would be a -- so just -- just to see -- maybe I just put that here.
This -- now I'm looking for an example where the trace could be negative but still blow up.
Of course -- yeah, let's just take one.
Oh, look, let me -- let me make it minus two zero zero one.
Okay.
There's a case where that matrix has negative trace -- I know its eigenvalues of course.
They're minus two and one and it blows up.
It's got -- it's got a plus one eigenvalue here, so there would be an e to the plus t in the solution and it'll blow up if it has any second component at all.
I need another condition.
And it's a condition on the determinant.
And what's that condition?
If I know that the two eigenvalues -- suppose I know they're negative, both negative.
What does that tell me about the determinant?
Let me ask again.
If I know both the eigenvalues are negative, then I know the trace is negative but the determinant is positive, because it's the product of the two eigenvalues.
So this determinant is lambda one times lambda two.
This is -- this is lambda one times lambda two and if they're both negative, the product is positive.
So positive determinant, negative trace.
I can easily track down the -- this condition also for the -- if -- if there's an imaginary part hanging around.
Okay.
So that's a -- like a small but quite useful, because two by two is always important -- comment on stability.
Okay.
So let's just look at the picture again.
Okay.
The main part of my lecture, the one thing you want to be able to, like, just do automatically is this step of solving the system.
Find the eigenvalues, find the eigenvectors, find the coefficients.
And notice -- what's the matrix -- in this linear system here, I can't help -- if I -- if I have equations like that -- that's my equations column at a time -- what's the matrix form of that equation?
So -- so this -- this system of equations is -- is some matrix multiplying c1, c2 to give u of zero.
One zero.
What's the matrix?
Well, it's obviously two one, one minus one, but we have a name, or at least a letter -- actually a name for that matrix.
Wh- what matrix are we s- are we -- are we dealing with here in this step of finding the c-s?
Its letter is S -- it's the eigenvector matrix.
Of course.
These are the eigenvectors, there in the columns of our matrix.
So this is S c equals u of zero -- is the -- that step where you find the actual coefficients, you find out how much of each pure exponential is in the solution.
By getting it right at the start, then it stays right forever.
I -- you're seeing this picture that each -- each pure exponential goes on its own way once you start it from u of zero.
So you start it by figuring out how much each one is present in u of zero and then off they go.
Okay.
So -- so that's a system of two equations in two unknowns coupled -- the matrix sort of couples u1 and u2 and the eigenvalues and eigenvectors uncouple it, diagonalize it.
Actually -- okay, now can I -- can I think in terms of S and lambda?
So I want to write the solution down, again in terms of S and lambda.
Okay.
I'll do that on this far board.
Okay.
So I'm coming back to -- I'm coming back to our equation du/dt equals Au.
Now this matrix A couples them.
The whole point of eigenvectors is to uncouple.
So one way to see that is introduce set u equal A -- not A.
It's S, the eigenvector matrix that uncouples it.
So I'm -- I'm taking this equation as I'm given, coupled with -- with A has -- is probably full of non-zeroes, but I'm -- by uncoupling it, I mean I'm diagonalizing it.
If I can get a diagonal matrix, I'm -- I'm in.
Okay.
So I plug that in.
This is A S v. And this is S dv/dt.
S is a constant.
It's -- this it the eigenvector matrix.
This is the eigenvector matrix.
Okay.
Now I'm going to bring S inverse over here.
And what have I got?
That combination S inverse A S is lambda, the diagonal matrix.
That's -- that's the point, that in -- if I'm using the eigenvectors as my basis, then my system of equations is just diagonal.
I -- each -- there's no coupling anymore -- dv1/dt is lambda one v1.
So let's just write that down.
dv1/ dt is lambda one v1 and so on for all n of the equations.
It's a system of equations but they're not connected, so they're easy to solve and why don't I just write down the solution?
v -- well, v is now some e to the lambda one t -- well, okay -- I guess my idea here now is to use, the natural notation -- v of T is e to the lambda tv of zero.
And u of t will be Se to the lambda t S inverse, u of zero.
This is the -- this is the, formula I'm headed for.
This -- this matrix, S e to the lambda t S inverse, that's my exponential.
That's my e to the A t, is this S e to the lambda t S inverse.
So my -- my job really now is to explain what's going on with this matrix up in the exponential.
What does that mean?
What does it mean to say e to a matrix?
This ought to be easier because this is e to a diagonal matrix, but still it's a matrix.
What do we mean by e to the A t?
Because really e to the A t is my answer here.
My -- my answer to this equation is -- this u of t, this is my -- this is my e to the A t u of zero.
So it's -- my job is really now to say what's -- what does that mean?
What's the exponential of a matrix and why is that formula correct?
Okay.
So I'll put that on the board underneath.
What's the exponential of a matrix?
Let me come back here.
So I'm talking about matrix exponentials.
e to the At.
Okay.
How are we going to define the exponential of a -- of something?
The trick -- the idea is -- the thing to go back to is exponential -- there's a power series for exponentials.
That's how you -- you -- the good way to define e to the x is the power series one plus x plus one half x squared, one six x cubed and we'll do it now when the -- when we have a matrix.
So the one becomes the identity, the x is At, the x squared is At squared and I divide by two.
The cube, the x cube is At cubed over six, and what's the general term in here?
At to the n-th power divided by -- and it goes on.
But what do I divide by?
So, you see the pattern -- here I divided by one, here I divided by one by two by six, those are the factorials.
It's n factorial.
That was, like, the one beautiful Taylor series.
The one beautiful Taylor series -- well, there are two -- there are two beautiful Taylor series in this world.
The Taylor series for e to the x is the s with x to the n-th over n factorial.
And all I'm doing is doing the same thing for matrixes.
The other beautiful Taylor series is just the sum of x to the n-th not divided by n factorial.
Can you -- do you know what function that one is?
So if I take -- this is the series, all these sums are going from zero to infinity.
What's -- what function have I got -- this is like a side comment -- this is one plus x plus x squared plus x cubed plus x to the fourth not divided by anything, what's -- what's that function?
One plus x plus x squared plus x cubed plus x fourth forever is one over one minus x.
It's the geometric series, the nicest power series of all.
So, actually, of course, there would be a similar thing here.
If -- if I wanted, I minus A t inverse would be -- now I've got matrixes.
I've got matrixes everywhere, but it's just like that series with -- and just like this one without the divisions.
It's I plus At plus At squared plus At cubed and forever.
So that's actually a -- a reasonable way to find the inverse of a matrix.
If I chop it off -- well, it's reasonable if t is small.
If t is a small number, then -- then t squared is extremely small, t cubed is even smaller, so approximately that inverse is I plus At.
I can keep more terms if I like.
Do you see what I'm doing?
I'm just saying we can do the same thing for matrixes that we do for ordinary functions and the good thing about the exponential series -- so in a way, this series is better than this one.
Why?
Because this one always converges.
I'm dividing by these bigger and bigger numbers, so whatever matrix A and however large t is, that series -- these terms go to zero.
The series adds up to a finite sum, e to the At is a -- is -- is completely defined.
Whereas this second guy could fail, right?
If At is big -- somehow if At has an eigenvalue larger than one, then when I square it it'll have that eigenvalue squared, when I cube it the eigenvalue will be cubed -- that series will blow up unless the eigenvalues of At are smaller than one.
So when the eigenvalues of At are smaller than one -- so I'd better put that in.
The -- all eigenvalues of At below one -- then that series converges and gives me the inverse.
Okay.
So this is the guy I'm chiefly interested in, and I wanted to connect it to -- oh, okay.
What's -- how do I -- how do I get -- this is my, like, main thing now to do -- how do I get e to the At -- how do I see that e to the At is the same as this?
In other words, I can find e to the At by finding S and lambda, because now e to the lambda t is easy.
Lambda's a diagonal matrix and we can write down either the lambda t -- and will right -- in a minute.
But how -- do you see what -- do you see that we're hoping for a -- we're hoping that we can compute either the A T from S and lambda -- and I have to look at that definition and say, okay, how do -- how do I get an S and the lambda to come out of that?
Okay, can -- do you see how I -- I want to connect that to that, from that definition.
So let me erase this -- the geometric series, which isn't part of the differential equations case and get the S and the lambda into this picture.
Oh, okay.
Here we go.
So identity is fine.
Now -- all right, you -- you -- you'll see how I'm -- how I'm -- how I going to get A replaced by S and S is in lambda's?
Well I use the fundamental formula of this whole chapter.
A is S lambda S inverse and then times t. That's At.
Okay.
What's A squared t?
I can -- I've got the divide by two, I've got the t squared and I've got an A squared.
All right, I -- so I've got a -- there's A -- there's A.
Now square it.
So what happens when I square it?
We've seen that before.
When I square it, I get S lambda squared S inverse, right?
When I square that thing, the -- there's an S and a -- an S cancels out an S inverse.
I'm left with the S on the left, the S inverse on the right and lambda squared in the middle.
And so on.
The next one'll be S lambda cubed, S inverse -- times t cubed over three factorial.
And now -- what do I do now?
I want to pull an S out from everything.
I want an S out of the whole thing.
Well, look, I'd better write the identity how?
I -- I want to be able to pull an S out and an S inverse out from the other side, so I just write the identity as S times S inverse.
So I have an S there and an S inverse from this side and what have I got in the middle?
If I pull out an S and an S inverse, what have I got in the middle?
I've got the identity, a lambda t, a lambda squared t squared over two -- I've got e to the lambda t. That's what's in the middle.
That's my formula for e to the At.
Oh, now I have to ask you.
Does this formula always work?
This formula always works -- well, except it's an infinite series.
But what do I mean by always work?
And this one doesn't always work and I just have to remind you of what assumption is built into this formula that's not built into the original.
The assumption that A can be diagonalized.
You'll remember that there are some small -- sm- some subset of matrixes that don't have n independent eigenvectors, so we don't have an S inverse for those matrixes and the whole diagonalization breaks down.
We could still make it triangular.
I'll tell you about that.
But diagonal we can't do for those particular degenerate matrixes that don't have enough independent eigenvectors.
But otherwise, this is golden.
Okay.
So that's the formula -- that's the matrix exponential.
Now it just remains for me to say what is e to the lambda t?
Can I just do that?
Let me just put that in the corner here.
What is the exponential of a diagonal matrix?
Remember lambda is diagonal, lambda one down to lambda n. What's the exponential of that diagonal matrix?
Because our whole point is that this ought to be simple.
Our whole point is that to take the exponential of a diagonal matrix ought to be completely decoupled -- it ought to be diagonal, in other words, and it is.
It's just e to the lambda one t, e to the lambda two t, e to the lambda n t, all zeroes.
So -- so if we have a diagonal matrix and I plug it into this exponential formula, everything's diagonal and the diagonal terms are just the ordinary scaler exponentials e to the lambda one t. Okay, so that's -- that's -- in a sense, I'm doing here, on this board, with -- with, like, formulas what I did on the -- in the first half of the lecture with specific matrix A and specific eigenvalues and eigenvectors.
The -- so let me show you the formulas again.
But the -- so you -- I guess -- what should you know from this lecture?
You should know what this matrix exponential is and, like, when does it go to zero?
Tell me again, now, the answer to that.
When does e to the At approach -- get smaller and smaller as t increases?
Well, the S and the S inverse aren't moving.
It's this one that has to get smaller and smaller and that one has this simple diagonal form.
And it goes to zero provided every one of these lambdas -- I -- I need to have each one of these guys go to zero, so I need every real part of every eigenvalue negative.
Right?
If the real part is negative, that's -- that takes the exponential -- that forces -- the exponential goes to zero.
Okay, so that -- that's really the difference.
If I can just draw the -- here's a picture of the -- of the -- this is the complex plain.
Here's the real axis and here's the imaginary axis.
And where do the eigenvalues have to be for stability in differential equations?
They have to be over here, in the left half plain.
So the left half plain is this plain, real part of lambda, less than zero.
Where do the ma- where do the eigenvalues have to be for powers of the matrix to go to zero?
Powers of the matrix go to zero if the eigenvalues are in here.
So this is the stability region for powers -- this is the region -- absolute value of lambda, less than one.
That's the stability for -- that tells us that the powers of A go to zero, this tells us that the exponential of A goes to zero.
Okay.
One final example.
Let me just write down how to deal with a final example.
Let's see.
So my final example will be a single equation, u''+bu'+Ku=0.
One equation, second order.
How do I -- and maybe I should have used -- I'll use -- I prefer to use y here, because that's what you see in differential equations.
And I want u to be a vector.
So how do I change one second order equation into a two by two first order system?
Just the way I did for Fibonacci.
I'll let u be y prime and y.
What I'm going to do is I'm going to add an extra equation, y prime equals y prime.
So I take this -- so by -- so using this as the vector unknown, now my equation is u prime.
My first order system is u prime, that'll be y double prime y prime, the derivative of u, okay, now the differential equation is telling me that y double prime is m- so I'm just looking for -- what's this matrix?
y prime y. I'm looking for the matrix A.
What's the matrix in case I have a single first order equation and I want to make it into a two by two system?
Okay, simple.
The first row of the matrix is given by the equation.
So y''-by'-ky -- no problem.
And what's the second row on the matrix?
Then we're done.
The second row of the matrix I want just to be the trivial equation y prime equals y prime, so I need a one and a zero there.
So matrixes like these, the gen- the general case, if I had a five by five -- if I had a fifth order equation and I wanted a five by five matrix, I would see the coefficients of the equation up there and then my four trivial equations would put ones here.
This is the kind of matrix that goes from a fifth order to a five by five first order.
So the -- and the eigenvalues will come out in a natural way connected to the differential equation.
Okay, that's differential equations.
The -- a parallel lecture compared to powers of a matrix we can now do exponentials.
Thanks.
-- two, one and -- okay.
Here is a lecture on the applications of eigenvalues and, if I can -- so that will be Markov matrices.
I'll tell you what a Markov matrix is, so this matrix A will be a Markov matrix and I'll explain how they come in applications.
And -- and then if I have time, I would like to say a little bit about Fourier series, which is a fantastic application of the projection chapter.
Okay.
What's a Markov matrix?
Can I just write down a typical Markov matrix, say .1, .2, .7, .01, .99 0, let's say, .3, .3, .4.
Okay.
There's a -- a totally just invented Markov matrix.
What makes it a Markov matrix?
Two properties that this -- this matrix has.
So two properties are -- one, every entry is greater equal zero.
All entries greater than or equal to zero.
And, of course, when I square the matrix, the entries will still be greater/equal zero.
I'm going to be interested in the powers of this matrix.
And this property, of course, is going to -- stay there.
It -- really Markov matrices you'll see are connected to probability ideas and probabilities are never negative.
The other property -- do you see the other property in there?
If I add down the columns, what answer do I get?
One.
So all columns add to one.
All columns add to one.
And actually when I square the matrix, that will be true again.
So that the powers of my matrix are all Markov matrices, and I'm interested in, always, the eigenvalues and the eigenvectors.
And this question of steady state will come up.
You remember we had steady state for differential equations last time?
When -- what was the steady state -- what was the eigenvalue?
What was the eigenvalue in the differential equation case that led to a steady state?
It was lambda equals zero.
When -- you remember that we did an example and one of the eigenvalues was lambda equals zero, and that -- so then we had an E to the zero T, a constant one -- as time went on, there that thing stayed steady.
Now what -- in the powers case, it's not a zero eigenvalue.
Actually with powers of a matrix, a zero eigenvalue, that part is going to die right away.
It's an eigenvalue of one that's all important.
So this steady state will correspond -- will be totally connected with an eigenvalue of one and its eigenvector.
In fact, the steady state will be the eigenvector for that eigenvalue.
Okay.
So that's what's coming.
Now, for some reason then that we have to see, this matrix has an eigenvalue of one.
This property, that the columns all add to one -- turns out -- guarantees that one is an eigenvalue, so that you can actually find the eigenvalue -- find that eigenvalue of a Markov matrix without computing any determinants of A minus lambda I -- that matrix will have an eigenvalue of one, and we want to see why.
And then the other thing is -- so the key points -- let me -- let me write these underneath.
The key points are -- the key points are lambda equal one is an eigenvalue.
I'll add in a little -- an additional -- well, a thing about eigenvalues -- key point two, the other eigenval- values -- all other eigenvalues are, in magnitude, smaller than one -- in absolute value, smaller than one.
Well, there could be some exceptional case when -- when an eigen -- another eigenvalue might have magnitude equal one.
It never has an eigenvalue larger than one.
So these two facts -- somehow we ought to -- linear algebra ought to tell us.
And then, of course, linear algebra is going to tell us what the -- what's -- what happens if I take -- if -- you remember when I solve -- when I multiply by A time after time the K-th thing is A to the K u0 and I'm asking what's special about this -- these powers of A, and very likely the quiz will have a problem to computer s- to computer some powers of A or -- or applied to an initial vector.
So, you remember the general form?
The general form is that there's some amount of the first eigenvalue to the K-th power times the first eigenvector, and another amount of the second eigenvalue to the K-th power times the second eigenvector and so on.
A -- just -- my conscience always makes me say at least once per lecture that this requires a complete set of eigenvectors, otherwise we might not be able to expand u0 in the eigenvectors and we couldn't get started.
But once we're started with u0 when K is zero, then every A brings in these lambdas.
And now you can see what the steady state is going to be.
If lambda one is one -- so lambda one equals one to the K-th power and these other eigenvalues are smaller than one -- so I've sort of scratched over the equation there to -- we had this term, but what happens to this term -- if the lambda's smaller than one, then the -- when -- as we take powers, as we iterate as we -- as we go forward in time, this goes to zero, Can I just -- having scratched over it, I might as well scratch right?
further.
That term and all the other terms are going to zero because all the other eigenvalues are smaller than one and the steady state that we're approaching is just -- whatever there was -- this was -- this was the -- this is the x1 part of un- of the initial condition u0 -- is the steady state.
This much we know from general -- from -- you know, what we've already done.
So I want to see why -- let's at least see number one, why one is an eigenvalue.
And then there's actually -- in this chapter we're interested not only in eigenvalues, but also eigenvectors.
And there's something special about the eigenvector.
Let me write down what that is.
The eigenvector x1 -- x1 is the eigenvector and all its components are positive, so the steady state is positive, if the start was.
If the start was -- so -- well, actually, in general, I -- this might have a -- might have some component zero always, but no negative components in that eigenvector.
Okay.
Can I come to that point?
How can I look at that matrix -- so that was just an example.
How could I be sure -- how can I see that a matrix -- if the columns add to zero -- add to one, sorry -- if the columns add to one, this property means that lambda equal one is an eigenvalue.
Okay.
So let's just think that through.
What I saying about -- let me ca- let me look at A, and if I believe that one is an eigenvalue, then I should be able to subtract off one times the identity and then I would get a matrix that's, what, -.9, -.01 and -.6 -- wh- I took the ones away and the other parts, of course, are still what they were, and this is still .2 and .7 and -- okay, what's -- what's up with this matrix now?
I've shifted the matrix, this Markov matrix by one, by the identity, and what do I want to prove?
I -- what is it that I believe this matrix -- about this matrix?
I believe it's singular.
Singular will -- if A minus I is singular, that tells me that one is an eigenvalue, right?
The eigenvalues are the numbers that I subtract off -- the shifts -- the numbers that I subtract from the diagonal -- to make it singular.
Now why is that matrix singular?
I -- we could compute its determinant, but we want to see a reason that would work for every Markov matrix not just this particular random example.
So what is it about that matrix?
Well, I guess you could look at its columns now -- what do they add up to?
Zero.
The columns add to zero, so all columns -- let me put all columns now of -- of -- of A minus I add to zero, and then I want to realize that this means A minus I is singular.
Okay.
Why?
So I could I -- you know, that could be a quiz question, a sort of theoretical quiz question.
If I give you a matrix and I tell you all its columns add to zero, give me a reason, because it is true, that the matrix is singular.
Okay.
I guess actually -- now what -- I think of -- you know, I'm thinking of two or three ways to see that.
How would you do it?
We don't want to take its determinant somehow.
For the matrix to be singular, well, it means that these three columns are dependent, right?
The determinant will be zero when those three columns are dependent.
You see, we're -- we're at a point in this course, now, where we have several ways to look at an idea.
We can take the determinant -- here we don't want to.
B- but we met singular before that -- those columns are dependent.
So how do I see that those columns are dependent?
They all add to zero.
Let's see, whew -- well, oh, actually, what -- another thing I know is that the -- I would like to be able to show is that the rows are dependent.
Maybe that's easier.
If I know that all the columns add to zero, that's my information, how do I see that those three rows are linearly dependent?
What -- what combination of those rows gives the zero row?
How -- how could I combine those three rows -- those three row vectors to produce the zero row vector?
And that would tell me those rows are dependent, therefore the columns are dependent, the matrix is singular, the determinant is zero -- well, you see it.
I just add the rows.
One times that row plus one times that row plus one times that row -- it's the zero row.
The rows are dependent.
In a way, that one one one, because it's multiplying the rows, is like an eigenvector in the -- it's in the left null space, right?
One one one is in the left null space.
It's singular because the rows are dependent -- and can I just keep the reasoning going?
Because this vector one one one is -- it's not in the null space of the matrix, but it's in the null space of the transpose -- is in the null space of the transpose.
And that's good enough.
If we have a square matrix -- if we have a square matrix and the rows are dependent, that matrix is singular.
So it turned out that the immediate guy we could identify was one one one.
Of course, the -- there will be somebody in the null space, too.
And actually, who will it be?
So what's -- so -- so now I want to ask about the null space of -- of the matrix itself.
What combination of the columns gives zero?
I -- I don't want to compute it because I just made up this matrix and -- it will -- it would take me a while -- it looks sort of doable because it's three by three but wh- my point is, what -- what vector is it if we -- once we've found it, what have we got that's in the -- in the null space of A?
It's the eigenvector, right?
That's where we find X one.
Then X one, the eigenvector, is in the null space of A.
That's the eigenvector corresponding to the eigenvalue one.
Right?
That's how we find eigenvectors.
So those three columns must be dependent -- some combination of columns -- of those three columns is the zero column and that -- the three components in that combination are the eigenvector.
And that guy is the steady state.
Okay.
So I'm happy about the -- the thinking here, but I haven't given -- I haven't completed it because I haven't found x1.
But it's there.
Can I -- another thought came to me as I was doing this, another little comment that -- you -- about eigenvalues and eigenvectors, because of A and A transpose.
What can you tell me about eigenvalues of A -- of A and eigenvalues of A transpose?
Whoops.
They're the same.
They're -- so this is a little comment -- we -- it's useful, since eigenvalues are generally not easy to find -- it's always useful to know some cases where you've got them, where -- and this is -- if you know the eigenvalues of A, then you know the eigenvalues of A transpose.
eigenvalues of A transpose are the same.
And can I just, like, review why that is?
So to find the eigenvalues of A, this would be determinate of A minus lambda I equals zero, that gives me an eigenvalue of A -- now how can I get A transpose into the picture here?
I'll use the fact that the determinant of a matrix and the determinant of its transpose are the same.
The determinant of a matrix equals the determinant of a -- of the transpose.
That was property ten, the very last guy in our determinant list.
So I'll transpose that matrix.
This leads to -- I just take the matrix and transpose it, but now what do I get when I transpose lambda I?
I just get lambda I.
So that's -- that's all there was to the reasoning.
The reasoning is that the eigenvalues of A solved that equation.
The determinant of a matrix is the determinant of its transpose, so that gives me this equation and that tells me that the same lambdas are eigenvalues of A transpose.
So that, backing up to the Markov case, one is an eigenvalue of A transpose and we actually found its eigenvector, one one one, and that tell us that one is also an eigenvalue of A -- but, of course, it has a different eigenvector, the -- the left null space isn't the same as the null space and we would have to find it.
So there's some vector here which is x1 that produces zero zero zero.
Actually, it wouldn't be that hard to find, you know, I -- as I'm talking I'm thinking, okay, I going to follow through and actually find it?
Well, I can tell from this one -- look, if I put a point six there and a point seven there, that's what -- then I'll be okay in the last row, right?
Now I only -- remains to find one guy.
And let me take the first row, then.
Minus point 54 plus point 21 -- there's some big number going in there, right?
So I have -- just to make the first row come out zero, I'm getting minus point 54 plus point 21, so that was minus point 33 and what -- what do I want?
Like thirty three hundred?
This is the first time in the history of linear algebra that an eigenvector has every had a component thirty three hundred.
But I guess it's true.
Because then I multiply by minus one over a hundred -- oh no, it was point 33.
So is this just -- oh, shoot.
Only 33.
Okay.
Only 33.
Okay, so there's the eigenvector.
Oh, and notice that it -- that it turned -- did turn out, at least, to be all positive.
So that was, like, the theory -- predicts that part, too.
I won't give the proof of that part.
So 30 -- 33 -- point six 33 point seven.
Okay.
Now those are the ma- that's the linear algebra part.
Can I get to the applications?
Where do these Markov matrices come from?
Because that's -- that's part of this course and absolutely part of this lecture.
Okay.
So where's -- what's an application of Markov matrices?
Okay.
Markov matrices -- so, my equation, then, that I'm solving and studying is this equation u(k+1)=Auk.
And now A is a Markov matrix.
A is Markov.
And I want to give an example.
Can I just create an example?
It'll be two by two.
And it's one I've used before because it seems to me to bring out the idea.
It's -- because we have two by two, we have two states, let's say California and Massachusetts.
And I'm looking at the populations in those two states, the people in those two states, California and Massachusetts.
And my matrix A is going to tell me in a -- in a year, some movement has happened.
Some people stayed in Massachusetts, some people moved to California, some smart people moved from California to Massachusetts, some people stayed in California and made a billion.
Okay.
So that -- there's a matrix there with four entries and those tell me the fractions of my population -- so I'm making -- I'm going to use fractions, so they won't be negative, of course, because -- because only positive people are in- involved here -- and they'll add up to one, because I'm accounting for all people.
So that's why I have these two key properties.
The entries are greater equal zero because I'm looking at probabilities.
Do they move, do they stay?
Those probabilities are all between zero and one.
And the probabilities add to one because everybody's accounted for.
I'm not losing anybody, gaining anybody in this Markov chain.
It's -- it conserves the total population.
Okay.
So what would be a typical matrix, then?
So this would be u, California and u Massachusetts at time t equal k+1.
And it's some matrix, which we'll think of, times u California and u Massachusetts at time k. And notice this matrix is going to stay the same, you know, forever.
So that's a severe limitation on the example.
The example has a -- the same Markov matrix, the same probabilities act at every time.
Okay.
So what's a reasonable, say -- say point nine of the people in California at time k stay there.
And point one of the people in California move to Massachusetts.
Notice why that column added to one, because we've now accounted for all the people in California at time k. Nine tenths of them are still in California, one tenth are here at time k+1.
Okay.
What about the people who are in Massachusetts?
This is going to multiply column two, right, by our fundamental rule of multiplying matrix by vector, it's the -- it's the population in Massachusetts.
Shall we say that -- that after the Red Sox, fail again, eight -- only 80 percent of the people in Massachusetts stay and 20 percent move to California.
Okay.
So again, this adds to one, which accounts for all people in Massachusetts where they are.
So there is a Markov matrix.
Non-negative entries adding to one.
What's the steady state?
If everybody started in Massachusetts, say, at -- you know, when the Pilgrims showed up or something.
Then where are they now?
Where are they at time 100, let's say, or maybe -- I don't know, how many years since the Pilgrims?
300 and something.
Or -- and actually where will they be, like, way out a million years from now?
I -- I could multiply -- take the powers of this matrix.
In fact, you'll -- you would -- ought to be able to figure out what is the hundredth power of that matrix?
Why don't we do that?
But let me follow the steady state.
So what -- what's my starting -- my starting u Cal, u Mass at time zero is, shall we say -- shall we put anybody in California?
Let's make -- let's make zero there, and say the population of Massachusetts is -- let's say a thousand just to -- okay.
So the population is -- so the populations are zero and a thousand at the start.
What can you tell me about this population after -- after k steps?
What will u Cal plus u Mass add to?
A thousand.
Those thousand people are always accounted for.
But -- so u Mass will start dropping from a thousand and u Cal will start growing.
Actually, we could see -- why don't we figure out what it is after one?
After one time step, what are the populations at time one?
So what happens in one step?
You multiply once by that matrix and, let's see, zero times this column -- so it's just a thousand times this column, so I think we're getting 200 and 800.
So after the first step, 200 people have -- are in California.
Now at the following step, I'll multiply again by this matrix -- more people will move to California.
Some people will move back.
Twenty people will come back and, the -- the net result will be that the California population will be above 200 and the Massachusetts below 800 and they'll still add up to a thousand.
Okay.
I do that a few times.
I do that 100 times.
What's the population?
Well, okay, to answer any question like that, I need the eigenvalues and eigenvectors, right?
As soon as I've -- I've created an example, but as soon as I want to solve anything, I have to find eigenvalues and eigenvectors of that matrix.
Okay.
So let's do it.
So there's the matrix .9, .2, .1, .8 and tell me its eigenvalues.
Lambda equals -- so tell me one eigenvalue?
One, thanks.
And tell me the other one.
What's the other eigenvalue -- from the trace or the determinant -- from the -- I -- the trace is what -- is, like, easier.
So the trace of that matrix is one point seven.
So the other eigenvalue is point seven.
And it -- notice that it's less than one.
And notice that that determinant is point 72-.02, which is point seven.
Right.
Okay.
Now to find the eigenvectors.
This is -- so that's lambda one and the eigenvector -- I'll subtract one from the diagonal, right?
So can I do that in light let -- in light here?
Subtract one from the diagonal, I have minus point one and minus point two, and of course these are still there.
And I'm looking for its -- here's -- here's -- this is going to be x1.
It's the null space of A minus I.
Okay, everybody sees that it's two and one.
Okay?
And now how about -- so that -- and it -- notice that that eigenvector is positive.
And actually, we can jump to infinity right now.
What's the population at infinity?
It's a multiple -- this is -- this eigenvector is giving the steady state.
It's some multiple of this, and how is that multiple decided?
By adding up to a thousand people.
So the steady state, the c1x1 -- this is the x1, but that adds up to three, so I really want two -- it's going to be two thirds of a thousand and one third of a thousand, making a total of the thousand people.
That'll be the steady state.
That's really all I need to know at infinity.
But if I want to know what's happened after just a finite number like 100 steps, I'd better find this eigenvector.
So can I -- can I look at -- I'll subtract point seven time -- ti- from the diagonal and I'll get that and I'll look at the null space of that one and I -- and this is going to give me x2, now, and what is it?
So what's in the null space of -- that's certainly singular, so I know my calculation is right, and -- one and minus one.
One and minus one.
So I'm prepared now to write down the solution after 100 time steps.
The -- the populations after 100 time steps, right?
Can -- can we remember the point one -- the -- the one with this two one eigenvector and the point seven with the minus one one eigenvector.
So I'll -- let me -- I'll just write it above here.
u after k steps is some multiple of one to the k times the two one eigenvector and some multiple of point seven to the k times the minus one one eigenvector.
Right?
That's -- I -- this is how I take -- how powers of a matrix work.
When I apply those powers to a u0, what I -- so it's u0, which was zero a thousand -- that has to be corrected k=0.
So I'm plugging in k=0 and I get c1 times two one and c2 times minus one one.
Two equations, two constants, certainly independent eigenvectors, so there's a solution and you see what it is?
Let's see, I guess we already figured that c1 was a thousand over three, I think -- did we think that had to be a thousand over three?
And maybe c2 would be -- excuse me, let -- get an eraser -- we can -- I just -- I think we've -- get it here.
c2, we want to get a zero here, so maybe we need plus two thousand over three.
I think that has to work.
Two times a thousand over three minus two thousand over three, that'll give us the zero and a thousand over three and the two thousand over three will give us three thousand over three, the thousand.
So this is what we approach -- this part, with the point seven to the k-th power is the part that's disappearing.
That's -- that's Markov matrices.
Okay.
That's an example of where they come from, from modeling movement of people with no gain or loss, with total -- total count conserved.
Okay.
I -- just if I can add one more comment, because you'll see Markov matrices in electrical engineering courses and often you'll see them -- sorry, here's my little comment.
Sometimes -- in a lot of applications they prefer to work with row vectors.
So they -- instead of -- this was natural for us, right?
For all the eigenvectors to be column vectors.
So our columns added to one in the Markov matrix.
Just so you don't think, well, what -- what's going on?
If we work with row vectors and we multiply vector times matrix -- so we're multiplying from the left -- then it'll be the then we'll be using the transpose of -- of this matrix and it'll be the rows that add to one.
So in other textbooks, you'll see -- instead of col- columns adding to one, you'll see rows add to one.
Okay.
Fine.
Okay, that's what I wanted to say about Markov, now I want to say something about projections and even leading in -- a little into Fourier series.
Because -- but before any Fourier stuff, let me make a comment about projections.
This -- so this is a comment about projections onto -- with an orthonormal basis.
So, of course, the basis vectors are q1 up to qn.
Okay.
I have a vector b.
Let -- let me imagine -- let me imagine this is a basis.
Let -- let's say I'm in n by n. I'm -- I've got, eh, n orthonormal vectors, I'm in n dimensional space so they're a complete -- they're a basis -- any vector v could be expanded in this basis.
So any vector v is some combination, some amount of q1 plus some amount of q2 plus some amount of qn.
So -- so any v. I just want you to tell me what those amounts are.
What are x1 -- what's x1, for example?
So I'm looking for the expansion.
This is -- this is really our projection.
I could -- I could really use the word expansion.
I'm expanding the vector in the basis.
And the special thing about the basis is that it's orthonormal.
So that should give me a special formula for the answer, for the coefficients.
So how do I get x1?
What -- what's a formula for x1?
I could -- I can go through the projection -- the Q transpose Q, all that -- normal equations, but -- and I'll get -- I'll come out with this nice answer that I think I can see right away.
How can I pick -- get a hold of x1 and get these other x-s out of the equation?
So how can I get a nice, simple formula for x1?
And then we want to see, sure, we knew that all the time.
Okay.
So what's x1?
The good way is take the inner product of everything with q1.
Take the inner product of that whole equation, every term, with q1.
What will happen to that last term?
The inner product -- when -- if I take the dot product with q1 I get zero, right?
Because this basis was orthonormal.
If I take the dot product with q2 I get zero.
If I take the dot product with q1 I get one.
So that tells me what x1 is.
q1 transpose v, that's taking the dot product, is x1 times q1 transpose q1 plus a bunch of zeroes.
And this is a one, so I can forget that.
I get x1 immediately.
So -- do you see what I'm saying -- is that I have an orthonormal basis, then the coefficient that I need for each basis vector is a cinch to find.
Let me -- let me just -- I have to put this into matrix language, too, so you'll see it there also.
If I write that first equation in matrix language, what -- what is it?
I'm writing -- in matrix language, this equation says I'm taking these columns -- are -- are you guys good at this now?
I'm taking those columns times the Xs and getting V, right?
That's the matrix form.
Okay, that's the matrix Q. Qx is v. What's the solution to that equation?
It's -- of course, it's x equal Q inverse v. So x is Q inverse v, but what's the point?
Q inverse in this case is going to -- is simple.
I don't have to work to invert this matrix Q, because of the fact that the -- these columns are orthonormal, I know the inverse to that.
And it is Q transpose.
When you see a Q, a square matrix with that letter Q, the -- that just triggers -- Q inverse is the same as Q transpose.
So the first component, then -- the first component of x is the first row times v, and what's that?
The first component of this answer is the first row of Q transpose.
That's just -- that's just q1 transpose times v. So that's what we concluded here, too.
Okay.
So -- so nothing Fourier here.
The -- the key ingredient here was that the q-s are orthonormal.
And now that's what Fourier series are built on.
So now, in the remaining time, let me say something about Fourier series.
Okay.
So Fourier series is -- well, we've got a function f of x.
And we want to write it as a combination of -- maybe it has a constant term.
And then it has some cos(x) in it.
And it has some sin(x) in it.
And it has some cos(2x) in it.
And a -- and some sin(2x), and forever.
So what's -- what's the difference between this type problem and the one above it?
This one's infinite, but the key property of things being orthogonal is still true for sines and cosines, so it's the property that makes Fourier series work.
So that's called a Fourier series.
Better write his name up.
Fourier series.
So it was Joseph Fourier who realized that, hey, I could work in function space.
Instead of a vector v, I could have a function f of x.
Instead of orthogonal vectors, q1, q2 , q3, I could have orthogonal functions, the constant, the cos(x), the sin(x), the s- cos(2x), but infinitely many of them.
I need infinitely many, because my space is infinite dimensional.
So this is, like, the moment in which we leave finite dimensional vector spaces and go to infinite dimensional vector spaces and our basis -- so the vectors are now functions -- and of course, there are so many functions that it's -- that we've got an infin- infinite dimensional space -- and the basis vectors are functions, too.
a0, the constant function one -- so my basis is one cos(x), sin(x), cos(2x), sin(2x) and so on.
And the reason Fourier series is a success is that those are orthogonal.
Okay.
Now what do I mean by orthogonal?
I know what it means for two vectors to be orthogonal -- y transpose x equals zero, right?
Dot product equals zero.
But what's the dot product of functions?
I'm claiming that whatever it is, the dot product -- or we would more likely use the word inner product of, say, cos(x) with sin(x) is zero.
And cos(x) with cos(2x), also zero.
So I -- let me tell you what I mean by that, by that dot product.
Well, how do I compute a dot product?
So, let's just remember for vectors v trans- v transpose w for vectors, so this was vectors, v transpose w was v1w1 +...+vnwn.
Okay.
Now functions.
Now I have two functions, let's call them f and g. What's with them now?
The vectors had n components, but the functions have a whole, like, continuum.
To graph the function, I just don't have n points, I've got this whole graph.
So I have functions -- I'm really trying to ask you what's the inner product of this function f with another function g?
And I want to make it parallel to this the best I can.
So the best parallel is to multiply f (x) times g(x) at every x -- and here I just had n multiplications, but here I'm going to have a whole range of x-s, and here I added the results.
What do I do here?
So what's the analog of addition when you have -- when you're in a continuum?
It's integration.
So that the -- the dot product of two functions will be the integral of those functions, dx.
Now I have to say -- say, well, what are the limits of integration?
And for this Fourier series, this function f(x) -- if I'm going to have -- if that right hand side is going to be f(x), that function that I'm seeing on the right, all those sines and cosines, they're all periodic, with -- with period two pi.
So -- so that's what f(x) had better be.
So I'll integrate from zero to two pi.
My -- all -- everything -- is on the interval zero two pi now, because if I'm going to use these sines and cosines, then f(x) is equal to f(x+2pi).
This is periodic -- periodic functions.
Okay.
So now I know what -- I've got all the right words now.
I've got a vector space, but the vectors are functions.
I've got inner products and -- and the inner product gives a number, all right.
It just happens to be an integral instead of a sum.
I've got -- and that -- then I have the idea of orthogonality -- because, actually, just -- let's just check.
Orthogonality -- if I take the integral -- s- I -- let me do sin(x) times cos(x) -- sin(x) times cos(x) dx from zero to two pi -- I think we get zero.
That's the differential of that, so it would be one half sine x squared, was that right?
Between zero and two pi -- and, of course, we get zero.
And the same would be true with a little more -- some trig identities to help us out -- of every other pair.
So we have now an orthonormal infinite basis for function space, and all we want to do is express a function in that basis.
And so I -- the end of my lecture is, okay, what is a1?
What's the coefficient -- how much cos(x) is there in a function compared to the other harmonics?
How much constant is in that function?
That'll -- that would be an easy question.
The answer a0 will come out to be the average value of f. That's the amount of the constant that's in there, its average value.
But let's take a1 as more typical.
How will I get -- here's the end of the lecture, then -- how do I get a1?
The first Fourier coefficient.
Okay.
I do just as I did in the vector case.
I take the inner product of everything with cos(x) Take the inner product of everything with cos(x).
Then on the left -- on the left I have -- the inner product is the integral of f(x) times cos(x) cx.
And on the right, what do I have?
When I -- so what I -- when I say take the inner product with cos(x), let me put it in ordinary calculus words.
Multiply by cos(x) and integrate.
That's what inner products are.
So if I multiply that whole thing by cos(x) and I integrate, I get a whole lot of zeroes.
The only thing that survives is that term.
All the others disappear.
So -- and that term is a1 times the integral of cos(x) squared dx zero to 2pi equals -- so this was the left side and this is all that's left on the right-hand side.
And this is not zero of course, because it's the length of the function squared, it's the inner product with itself, and -- and a simple calculation gives that answer to be pi.
So that's an easy integral and it turns out to be pi, so that a1 is one over pi times there -- times this integral.
So there is, actually -- that's Euler's famous formula for the -- or maybe Fourier found it -- for the coefficients in a Fourier series.
And you see that it's exactly an expansion in an orthonormal basis.
Okay, thanks.
So I'll do a quiz review on Monday and then the quiz itself in Walker on Wednesday.
Okay, see you Monday.
Thanks.
OK, this is quiz review day.
The quiz coming up on Wednesday will before this lecture the quiz will be this hour one o'clock Wednesday in Walker, top floor of Walker, closed book, all normal.
I wrote down what we've covered in this second part of the course, and actually I'm impressed as I write it.
so that's chapter four on orthogonality and you're remembering these -- what this is suggesting, these are those columns are orthonormal vectors, and then we call that matrix Q and the -- what's the key -- how do we state the fact that those v- those columns are orthonormal in terms of Q, it means that Q transpose Q is the identity.
So that's the matrix statement of the -- of the property that the columns are orthonormal, the dot products are either one or zero, and then we computed the projections onto lines and onto subspaces, and we used that to solve problems Ax=b in -- in the least square sense, when there was no solution, we found the best solution.
And then finally this Graham-Schmidt idea, which takes independent vectors and lines them up, takes -- subtracts off the projections of the part you've already done, so that the new part is orthogonal and so it takes a basis to an orthonormal basis.
And you -- those calculations involve square roots a lot because you're making things unit vectors, but you should know that step.
OK, for determinants, the three big -- the big picture is the properties of the determinant, one to three d- properties one, two and three, d- that define the determinant, and then four, five, six through ten were consequences.
Then the big formula that has n factorial terms, half of them have plus signs and half minus signs, and then the cofactor formula.
So and which led us to a formula for the inverse.
And finally, just so you know what's covered in from chapter three, it's section six point one and two, so that's the basic idea of eigenvalues and eigenvectors, the equation for the eigenvalues, the mechanical step, this is really Ax equal lambda x for all n eigenvectors at once, if we have n independent eigenvectors, and then using that to compute powers of a matrix.
So you notice the differential equations not on this list, because that's six point three, that that's for the third quiz.
OK. Shall I what I usually do for review is to take an old exam and just try to pick out questions that are significant and write them quickly on the board, shall I -- shall I proceed that way again?
This -- this exam is really old.
November nineteen -- nineteen eighty-four, so that was before the Web existed.
So not only were the lectures not on the Web, nobody even had a Web page, my God.
OK, so can I nevertheless linear algebra was still as great as ever.
So may I and that wasn't meant to be a joke, OK, all right, so let me just take these questions as they come.
All right.
OK.
So the first question's about projections.
It says we're given the line, the -- the vector a is the vector two, one, two, and I want to find the projection matrix P that projects onto the line through a.
So my picture is, well I'm in three dimensions, of course, so there's a vector, two -- there's the vector a, two, one, two, let me draw the whole line through it, and I want to project any vector b onto that line, and I'm looking for the projection matrix.
So -- so this -- the projection matrix is the matrix that I multiply b by to get here.
And I guess this first part, this is just part one a, I'm really asking you the -- the quick way to answer, to find P, is just to remember what the formula is.
And -- and we're in -- we're projecting onto a line, so our formula, our -- our usual formula is AA transpose A inverse A transpose, but now A is just a column, one-column matrix, so it'll be just a, so I'll just call it little a, little a transpose, and this is just a number now, one by one, so I can put it in the denominator, so there's our -- that's really what we want to remember, and -- and using that two, one, two, what will I get?
I'm dividing by -- what's the length squared of that vector?
So what's a transpose a?
Looks like nine, and what's the matrix, well, I'm -- I'm doing two, one, two against two, one, two, so it's one-ninth of this matrix four, two, four, two, one, two, four, two, four.
Now the next part asked about eigenvalues.
So you see we're -- since we're learning, we know a lot more now, we can make connections between these chapters, so what's the eigenvector, what are the eigenvalues of P?
I could ask what's the rank of P. What's the rank of that matrix?
Uh -- one.
Rank is one.
What's the column space?
If I apply P to all vectors then I fill up the column space, it's combinations of the columns, so what's the column space?
Well, it's just this line.
The column space is the line through two, one, two.
And now what's the eigenvalue?
So, or since that matrix has rank one, so tell me the eigenvalues of this matrix.
It's a singular matrix, so it certainly has an eigenvalue zero.
Actually, the rank is only one, so that means that there like going to be t- a two-dimensional null space, there'll be at least two, this lambda equals zero will be repeated twice because I can find two independent eigenvectors with lambda equals zero, and then of course since it's got three eigenvalues, what's the third one?
It's one.
How do I know it's one?
Either from the trace, which is nine over nine, which is one, or by remembering what -- what is the eigenvector, and actually now it's going to ask for the eigenvector, so what's the eigenvector for that eigenvalue?
What's the eigenvector for that eigenvalue?
It's the -- it's the vector that doesn't move, eigenvalue one, so the vector that doesn't move is a.
This a is the -- is -- is also the eigenvector with lambda equal one, because if I apply the projection matrix to a, I get a again.
Everybody sees that if I apply that matrix to a, can I do it in little letters, if I apply that matrix to a, then I have a transpose a canceling a transpose a and I get a again.
So sure enough, Pa equals a.
And the eigenvalue is one.
OK. Good.
now, actually it asks you further to solve this difference equation, so this will be -- this is -- this is solve u(k+1)=Puk, starting from u0 equal nine, nine, zero.
And find uk.
So -- so what's up?
Shall we find u1 first of all?
So just to get started.
So what is u1?
It's Pu0 of course.
So if I do the projection of -- of this vector onto the line, so this is like my vector b now that I'm projecting onto the line, I get a times a transpose u0 over a transpose a.
Well, one way or another I just do this multiplication.
but maybe this is the easiest way to do it.
a transpose, can I remember what a is on this board?
Two, one, two, so I'm projecting onto the line through there.
This is the projection, it's P times the vector u0, so what do I have for a transpose u0 looks like eighteen, looks like twenty-seven, and a transpose a we figured was nine, so it's three a, so that this is the -- this is the x hat, the -- the multiple of a, in -- in our formulas, and of course that's six, three, six.
So that's u1.
Computed out directly.
That's on the line through a and it's the closest point to u0, and it's just Pu0.
You just straightforward multiplication produces that.
OK. Now, what's u2?
Well, u2 is Pu1, I agree.
Do I need to compute that again?
No, because once I'm already on the line through A, uk will be, I could do the projection k times, but it's enough just to do it once.
It's the same, it's the same, six, three, six.
So this is a case where I could and -- and actually on the quiz if you see one of these, which could very well be there, and it could very well be not a projection matrix, then we would use all the eigenvalues and eigenvectors.
Let's think for a moment, how do you do those?
M - the point of this small part of a question was that when P is a projection matrix, so that P squared equals P and P cubed equals P, then -- then we don't need to get into the mechanics of all knowing all the other eigenvalues and eigenvectors.
We just can go directly.
But if P was now some other matrix, can you just -- let's just remember from these very recent lectures how you would proceed.
from these very recent lectures we know that uk we would -- we would expand u0 as a combination of eigenvectors.
Let me leave -- yeah, as a combination of eigenvectors, c1x1, some multiple of the second eigenvector, some multiple of the third eigenvector, and then A to the ku0 would be c1, so this -- we have to find these numbers here, that's the work actually.
The work is find the eigenvalues, find the eigenvectors and find the c-s because they all come into the formula.
We have -- so -- so to do this, you can see what you have to compute.
You have to compute the eigenvalues, you have to compute the eigenvectors, and then to match u0 you compute the c-s, and then you've got it.
So it's -- it's just that's a formula that shows what pieces we need.
And what would actually happen in the case of this projection matrix?
If this A is a projection matrix, then a couple of eigenvalues are zero.
That's why we just throw those away.
The other eigenvalue was a one, so that we got the same thing every time, c3x3.
From the first time, second time, third, all iterations pro- left us with this constant, left us right here at six, three, six.
But maybe I take -- I'm taking this chance to remind you of what to do for other matrices.
OK.
So that's part way through.
OK.
The next question in nineteen eighty-four is fitting a straight line to points.
And actually a straight line through the origin.
A straight line through the origin.
So can I go to question two?
So this is fitting a straight line to these points, can I -- I'll just give you the points at t=1 the y is four, at t=2, y is five, at t=3, y is eight.
So we've got points one, two, three, four, five, and eight.
And I'm trying to fit a straight line through the origin to these three values.
OK, so my equation that I'm allowing myself is just y equal Dt.
So I have only one unknown.
One degree of freedom.
One parameter D. So I'm expecting to end up my matrix so my -- my -- when I try to -- when I try to fit a straight line, that goes through the origin, that's because it goes through the origin, I've lost the constant c here, so I have just this should be a quick calculation.
and I can write down the three equations that -- that -- that would -- I'd like to solve if the line went through the points, that's a good start.
Because that displays the matrix.
So can I continue that problem?
We would like to solve-- so y is Dt, so I'd like to solve D times one times D equals four, two times D equals five and three times D equals eight.
That would be perfection.
If I could find such a D, then the line y equal Dt would satisfy all three equations, would go through all three points, but it doesn't exist.
So -- so I have to solve this -- so the -- my matrix is now you can see my matrix, it just has one column.
Multiplying a scalar D. And you can see the right-hand side.
This is my Ax=b.
I don't need three equals signs now because I've got vectors.
OK.
There's Ax=b and you take it from there.
You the -- the best x will be -- will come from -- so what's the -- the key equation?
So this is the A, this is the Ax hat equal b equation.
Well, Ax=b.
And what's the equation for x hat?
The best D, so to find the best D, the best x, the equation is A transpose A, the best D, is A transpose times the right-hand side.
This is all coming from projection on a line, our -- our matrix only has one column.
So A transpose A would be maybe fourteen, D hat, and A transpose b I'm getting four, ten, and twenty-four.
Is that right?
Four, ten and twenty-four.
So thirty-eight.
So that tells me the best D hat is D hat is thirty-eight over fourteen.
OK. Fine.
All right.
so we found the best line.
And now here's a -- here's the next question.
What vector did I just project onto what line?
See in this section on least squares here's the key point, I'm -- I'm asking you to think of the least squares problem in two ways.
Two different pictures.
Two different graphs.
One graph is this.
This is a graph in the -- in the b -- in the tb plane, ty plane.
The -- the -- the line itself.
The other picture I'm asking you to think of is like my projection picture.
What -- what projection -- what -- what vector I -- I projecting onto what line or what subspace when I -- when I do this?
So the -- my second picture is a projection picture that -- that sees the whole thing with vectors.
Here's my vector of course that I'm projecting.
I'm projecting that vector b onto the column space of A.
Of if you like -- it's just a line onto that's the line it's just a line, of course.
That's what this calculation is doing.
This is computing the best D, which is -- this is the x hat.
So -- so seeing it as a projection means I don't see the projection in this figure, right?
In this figure I'm not projecting those points onto that line or anything of the sort.
The projection s-picture for -- for least squares is in the -- in the space where b lies, the whole vector b, and the columns of A.
And then the x is the best combination that gives the projection.
OK.
So that's a chance to tell me that.
OK.
I'll go -- OK now finally in orthogonality there's the Graham-Schmidt idea.
So that's problem two D here.
It asks me if I have two vectors, a1 equal one, two, three, and a2 equal one, one, one, find two orthogonal vectors in that plane.
So those two vectors give a plane, they give a plane.
Which is of course the -- the column space of the -- of the matrix.
And I'm looking for an orthogonal basis for that plane.
So I'm looking for two orthogonal vectors.
And of course there are lots of -- I mean, I've got a plane there.
If I get one orthogonal pair, I can rotate it.
There's not just one answer here.
But Graham-Schmidt says OK, start with the first vector, and let that be -- and keep that one.
And then take the second one orthogonal to this.
So -- so Graham-Schmidt says start with this one and then make a second vector B, can I call that second vector B, this is going to be orthogonal to, so perpendicular to a1.
If I can with my chalk create the key equation.
This vector B is going to be this one, one, one, but that one, one -- one, one, one is not perpendicular to a1, so I have to subtract off its projection, I have to subtract off the B, the -- the B trans- ye the B -- the -- the I should say the a1 transpose b over a1 transpose a1, that multiple of a1, I've got to remove.
So I just have to compute what that is, and I get ano- I get a vector B that's orthogonal to a1.
It's the -- it's -- it's the original vector minus its projection.
Oh, so what is -- I mean this to be a2.
So I'm projecting a2 onto the line through a1.
Yeah.
That's the part that I don't want because that's in the direction I already have, so I subtract off that projection and I get the part I want, the orthogonal part.
So that's the Graham-Schmidt thing and we can put numbers in.
OK. one, one, one take away a1 transpose a2 is six, a1 transpose a1 is fourteen,multiplying a1.
And that gives us the new orthogonal vector B.
Because I only ask for orthogonal right now, I don't have to divide by the length which will involve a square root.
OK. Third question.
Third question.
All right, let me -- I'll move this board up.
third question will probably be about eigenvalues.
OK. Three.
This is a four-by-four matrix.
Its eigenvalues are lambda one, lambda two, lambda three, lambda four.
Question one.
What's the condition on the lambdas so that the matrix is invertible?
OK.
So under what conditions on the lambdas will the matrix be invertible?
So that's easy.
Invertible if what's the condition on the lambdas?
None of them are zero.
A zero eigenvalue would mean something in the null space would mean a solution to Ax=0x, but we're invertible, so none of them is zero, the product -- however you want to say, no -- no zero eigenvalues.
Good.
OK, what's the determinant of A inverse?
The determinant of A inverse?
So where is that going to come from?
Well, if we knew the eigenvalues of A inverse, we could multiply them together to find the determinant.
And we do know the eigenvalues of A inverse.
What are they?
They're just one over lambda one times one over lambda two, that's the second eigenvalue, the third eigenvalue and the fourth.
So the product of the four eigenvalues of the inverse will give us the determinant of the inverse.
Fine.
OK. And what's the trace of A plus I?
So what do we know about trace?
It's the sum down the diagonal, but we don't know what our matrix is.
The trace is also the sum of the eigenvalues, and we do know the eigenvalues of A plus I.
So we just add them up.
So what -- what's the first eigenvalue of A plus I?
When the matrix A has eigenvalues lambda one, two, three and four, then the eigenvalues if I add the identity, that moves all the eigenvalues by one, so I just add up lambda one plus one, lambda two plus one, and so on, lambda three plus one, lambda four plus one, so it's lambda one plus lambda two plus lambda three plus lambda four plus four.
Right.
That movement by the identity moved all the eigenvalues by one, so it moved the whole trace by four.
So it was the trace of A plus four more.
OK. Let's see.
We may be finished this quiz twenty minutes early.
No.
There's another question.
Oh, God, OK. How did this class ever do it?
Well, you'll see.
you'll be able to do it.
OK. this has got to be a determinant question.
All right.
More determinants and cofactors and big formula question.
OK. Let me do that.
So it's about a matrix, a -- a whole family of matrices.
Here's the four-by-four one.
The four-by-four one is, and -- and all the matrices in this family are tridiagonal with -- with ones.
Otherwise zeroes.
So that's the pattern, and we've seen this matrix.
OK.
So the -- it's tridiagonal with ones on the diagonal, ones above and ones below, and you see the general formula An, so I'll use Dn for the determinant of An.
OK. All right.
So I'm going to do a -- the first question is use cofactors to show that Dn is something times D(n-1) plus something times D(n-2).
And find those somethings.
OK.
So this -- the fact that it's tridiagonal with these constant diagonals means that there is such a recurrence formula.
And so the first question is find it.
Well, what's the recurrence formula?
OK, how does it go?
So I'll use cofactors along the first row.
So I take that number times its cofactor.
So it's one times its cofactor and what is its cofactor?
D(n-1), right, exactly, the cofactor is this -- is this guy uses up row one and column one, so the cofactor is down here, so it's one of those.
OK, that's the first cofactor term.
Now the other cofactor term is this guy.
Which uses up row one and column two and what's surprising about that?
When you use row one and column two that brings in a minus.
There'll be a minus because the -- the cofactor is this determinant times minus one.
The the one-two cofactor is that determinant with its sign changed.
OK.
So I have to look at that determinant and I have to remember in my head a sign is going to get changed.
OK. Now how do I do that determinant?
How do I make that one clear?
I -- the -- the neat way to do is -- is here I see I -- I'll use cofactors down the first column.
Because the first column is all zeroes except for that one, so this one is now -- and what's its cofactor?
Within this three-by-three its cofactor will be two-by-two, and what is it?
It's this, right?
So -- so that part is all gone, so I'm taking that times its cofactor, then zero times whatever its cofactor is, so it's really just one times and what's this in the general n-by-n case?
It's Dn minus two.
But now so is this a plus or sign or a minus sign, it's -- it's just a one, because there's a one from there and a one from there.
And is it a plus or a minus?
It's minus I guess because there was a minus the first time and then the second time it's a plus, so it's overall it's a minus.
So there's my a and b were one and minus one.
Those constants.
Th- that's the -- that's the recurrence.
OK. And oh, then it asks you to then it asks you to solve this thing first by writing it as a -- as a system.
So now I'd like to know the solution.
I -- I better know how it starts, right?
It starts with D1, what was D1, that's just the one-by-one case, so D1 is one, and what is D2?
Just to get us started and then this would give us D3, D4, and forever.
D2 is this two-by-two that I'm seeing here and that determinant is obviously zero.
So those little ones will start the recurrence and then we take off.
And then the idea is to write this recurrence as -- as a Dn, D(n-1) is some matrix times the one before, the D(n-1), D(n-2).
What's the matrix?
You see, you remember this step of taking a single second order equation and by introducing a vector unknown to make it into a -- to a first order system.
OK.
So Dn is one of Dn minus one minus one, I think that -- that goes in the first row, right?
From the equation above?
And the second one is this is the same as this, so one and zero are fine.
So there's the matrix.
OK.
So now how do I proceed?
We can guess what this examiner's got in his little mind.
well, find the eigenvalues.
And actually it tells us that the sixth power of these eigenvalues turns out to be one.
Uh, well, can -- can we get the equation for the eigenvalues?
Let's do it and let's get a formula for them.
OK.
So what are the eigenvalues?
I look at the -- the matrix, this determinant one minus lambda and zero minus lambda, and these guys are still there, I compute that determinant, I get lambda squared minus lambda and then plus one.
And I set that to zero.
OK.
So we're not Fibonacci here.
We're -- we're not seeing Fibonacci numbers.
Because the sign -- we had a sign change there.
And it's not clear right away whether these -- whether this -- is it clear?
Is this matrix stable or unstable?
When we take -- when we go further and further out?
Are these Ds increasing?
Are they going to zero?
Are they bouncing around periodically?
the answers have to be here.
I would like to know how big these lambdas are, right?
And the point is probably these -- let's -- let's see, what's lambda?
From the quadratic formula lambda is one, I switch the sign of that, plus or minus the square root of one minus 4ac, I getting a minus three there?
Over two.
What's up?
They're complex.
The -- the eigenvalues are one plus square root of three I over two and one minus square root of three I over two.
What's the magnitude of lambda?
That's the key point for stability.
These are two numbers in the complex plane.
One plus some -- somewhere here, and its complex conjugate there.
I want to know how far from the origin are those numbers.
What's the magnitude of lambda?
And do you see what it is?
Do you recognize this -- a number like that?
Take the real part squared and the imaginary part squared and add.
What do you get?
So the real part squared is a quarter.
The imaginary part squared is three-quarters.
They add to one.
That's a number with -- that's on the unit circle.
That's an e to the i theta.
That's a cos(theta)+isin(theta).
And what's theta?
This -- this is like a complex number that's worth knowing, it's not totally obvious but it's nice.
That's -- I should see that as cos(theta)+isin(theta), and the angle that would do that is sixty degrees, pi over three.
So that's a -- let me improve my picture.
So those -- that's e to the i pi over six -- pi over three.
This is -- this number is e to the i pi over three and e to the minus i pi over three.
We'll be doing more complex numbers briefly but a little more in the next two days.
next two lectures.
Anyway, the -- so what's the deal with stability, what do the Dn-s do?
Well, look, if -- if I take the sixth power I'm around at one, the problem actually told me this.
The sixth power of those eigenvalues brings me around to What does that tell you about the matrix, by the way?
one.
Suppose you know -- this was a great quiz question, so I should never have just said it, but popped out.
Suppose lambda one to the sixth and lambda two to the sixth are -- are one, which they are.
What does that tell me about a m- a matrix?
About my matrix A here.
Well, what -- what matrix is connected with lambda one to the sixth and lambda two to the sixth?
It's got to be the matrix A to the sixth.
So what is A to the sixth for that matrix?
It's got eigenvalues one and one.
Because when I take the sixth power, actually, ye, if I take the sixth power b- all the sixth power of that is one and the sixth power of that is one, the sixth power of this is e to the two pi i, that's one, the sixth power of this is e to the minus two pi i, that's one.
So the sixth powers, the -- the sixth power of that matrix has eigenvalues one and one, so what is it?
It's the identity, right.
So if I operate this -- if I run this thing six times, I'm back where I was.
The sixth power of that matrix is the identity.
Good.
OK.
So it'll loop around, it's -- it doesn't go to zero, it doesn't blow up, it just periodically goes around with period six.
OK. let's just see if there's a -- all right.
I'll -- let's see.
Could I also look at a -- at a final exam from nineteen ninety-two.
I think that's yeah, let me do that on this last board.
It starts -- a lot of the questions in this exam are about a family of matrices.
Let me give you the fourth, the fourth guy in the family is -- has a one, so it's zeroes on the diagonal, but these are going one, two, three and so on.
One, two, three, and so on.
But, for the four-by-four case I'm stopping at four.
You see the pattern?
It's a family of matrices which is growing, and actually the numbers -- it's symmetric, right, it's equal to A4 transpose.
And we can ask all sorts of questions about its null space, its range, r- its column space find the projection matrix onto the column space of A3, for example, is in here.
So -- so one -- so A3 is zero, one, zero, one, zero, two, zero, two, zero.
OK, find the projection matrix onto the column space.
By the way, is that matrix singular or invertible?
Singular.
Why do we know it's singular?
I see that column three is a multiple of column one.
Or we could take its determinant.
So it's certainly singular.
The projection will be matrix will be three-by-three but it will project onto the column space, it'll project onto this plane.
The column space of A3, and I guess I would find it from the formula AA -- AA transpose A inverse, I would have to -- I would -- I guess I would do all this.
There may be a better way, perhaps I could think there might be a slightly quicker way, but that would come out pretty fast.
OK.
So that's be the projection matrix.
Next question.
Find the eigenvalues and eigenvectors of that matrix.
OK.
There's a three-by-three matrix, oh, yeah, so what are its eigenvalues and eigenvectors, we haven't done any three-by-threes.
Let's do one.
I want to find, so how do I find eigenvalues?
I take the determinant of A3 minus lambda I.
So this is you just have to -- so I'm subtracting lambda from the diagonal, and I have a one, one, zero, zero, two, two there, and I just have to find that determinant.
OK, since it's three-by-three I'll just go for it.
This way gives me minus lambda cubed and a zero and zero.
Then in this direction which has the minus sign, that's a zero, four lambdas, I mean minus four lambdas, and minus another lambda, so that's minus five lambdas, but that direction goes with a minus sign, so I think it's plus five lambda.
That looks like the determinant of A3 minus lambda I, so I set it to zero.
So what are the eigenvalues?
Well, lambda equals zero -- lambda factors out of this, times minus lambda squared plus four, so the eigenvalues are five, thanks, thanks, so the eigenvalues are zero, square root of five, and minus square root of five.
And I would never write down those three eigenvalues without checking the trace to tell the truth.
Because -- because we did a bunch of calculations here but then I can quickly add up the eigenvalues to get zero, add up the trace to get zero, and feel that I'm -- well, I guess that wouldn't have caught my error if I'd made it -- if -- if that had been a four I wouldn't have noticed,the determinant isn't anything greatly useful here, right, because the determinant is just zero.
And so I never would know whether that five was right or wrong, but thanks for making it right.
OK. Ha.
Question two c, whoever wrote this, probably me, said this is not difficult.
I don't know why I put that in.
just -- it asks for the projection matrix onto the column space of A4.
How could I have thought that wasn't difficult?
It looks extremely difficult.
what's the projection matrix onto the column space of A4?
I don't know whether that -- this is not difficult is just like helpful or -- or insulting.
Uh, what do you think?
The -- what's the column space of A4 here?
Well, what's our first question is is the matrix singular or invertible?
If the answer is invertible, then what's the column space?
If -- if this matrix A4 is invertible, so that's my guess, if this problem's easy it has to be because this matrix is probably invertible.
Then its column space is R^4, good, the column space is the whole space, and the answer to this easy question is the projection matrix is the identity, it's the four-by-four identity matrix.
If this matrix is invertible.
Shall we check invertibility?
How would you find its determinant?
Can we just like take the determinant of that matrix?
I could ask you how -- so there -- there are twenty-four terms, do we want to write all twenty-four terms down?
not in the remaining ten seconds.
Better to use cofactors.
So I go along row one, I see one -- the only nonzero is this guy, so I should take that one times the cofactor.
Now so I'm down to this determinant.
OK.
So now I'm -- look at this first column, I see one times this, there's the cofactor of the one, so I'm using up row one -- row one and column one of this three-by-three matrix, I'm down to this cofactor, and by the way, those were both plus signs, right?
No, they weren't.
That was a minus sign.
That was a -- that was a minus, and then that was a plus, and then this, so what's the determinant?
Nine.
Nine.
Determinant is nine.
Determinant of A4 is nine.
OK. Where A3, so my guess is I'll put that on the final this year, the -- probably the odd- numbered ones are singular and the even-numbered ones are invertible.
And I don't know what the determinants are but I'm betting that they have some nice formula.
OK.
So, recitations this week will also be quiz review and then the quiz is Wednesday at one o'clock.
Thanks.
-- one and -- the lecture on symmetric matrixes.
So that's the most important class of matrixes, symmetric matrixes.
A equals A transpose.
So the first points, the main points of the lecture I'll tell you right away.
What's special about the eigenvalues?
What's special about the eigenvectors?
This is -- the way we now look at a matrix.
We want to know about its eigenvalues and eigenvectors and if we have a special type of matrix, that should tell us something about eigenvalues and eigenvectors.
Like Markov matrixes, they have an eigenvalue equal one.
Now symmetric matrixes, can I just tell you right off what the main facts -- the two main facts are?
The eigenvalues of a symmetric matrix, real -- this is a real symmetric matrix, we -- talking mostly about real matrixes.
The eigenvalues are also real.
So our examples of rotation matrixes, where -- where we got E- eigenvalues that were complex, that won't happen now.
For symmetric matrixes, the eigenvalues are real and the eigenvectors are also very special.
The eigenvectors are perpendicular, orthogonal, so which do you prefer?
I'll say perpendicular.
Perp- well, they're both long words.
Okay, right.
So -- I have a -- you should say "why?"
and I'll at least answer why for case one, maybe case two, the checking the Eigen -- that the eigenvectors are perpendicular, I'll leave to, the -- to the book.
But let's just realize what -- well, first I have to say, it -- it could happen, like for the identity matrix -- there's a symmetric matrix.
Its eigenvalues are certainly all real, they're all one for the identity matrix.
What about the eigenvectors?
Well, for the identity, every vector is an eigenvector.
So how can I say they're perpendicular?
What I really mean is the -- they -- this word are should really be written can be chosen perpendicular.
That is, if we have -- it's the usual case.
If the eigenvalues are all different, then each eigenvalue has one line of eigenvectors and those lines are perpendicular here.
But if an eigenvalue's repeated, then there's a whole plane of eigenvectors and all I'm saying is that in that plain, we can choose perpendicular ones.
So that's why it's a can be chosen part, is -- this is in the case of a repeated eigenvalue where there's some real, substantial freedom.
But the typical case is different eigenvalues, all real, one dimensional eigenvector space, Eigen spaces, and all perpendicular.
So, just -- let's just see the conclusion.
If we accept those as correct, what happens -- and I also mean that there's a full set of them.
so forgive me for doing such a thing, but, I'll look at the I -- so that's part of this picture here, that there -- there's a complete set of eigenvectors, perpendicular ones.
So, having a complete set of eigenvectors means -- so normal -- so the usual -- maybe I put the -- usually -- usual -- usual case is that the matrix A we can write in terms of its eigenvalue matrix and its eigenvector matrix this way, right?
We can do that in the usual case, but now what's special when the matrix is symmetric?
So this is the usual case, and now let me go to the symmetric case.
So in the symmetric case, A, this -- this should become somehow a little special.
Well, the lambdas on the diagonal are still on the diagonal.
They're -- they're real, but that's where they are.
What about the eigenvector matrix?
So what can I do now special about the eigenvector matrix when -- when the A itself is symmetric, that says something good about the eigenvector matrix, so what is this -- what does this lead to?
This -- these perpendicular eigenvectors, I can not only -- I can not only guarantee they're perpendicular, I could also make them unit vectors, no problem, just s- scale their length to one.
So what do I have?
I have orthonormal eigenvectors.
And what does that tell me about the eigenvector matrix?
What -- what letter should I now use in place of S -- I've got -- those two equations are identical,1 remember S has the eigenvectors in its columns, but now those columns are orthonormal, so the right letter to use is Q.
So that's where -- so we've got the letter all set up, book.
so this should be Q lambda Q inverse.
Q standing in our minds always for this matrix -- in this case it's square, it's -- so these are the Okay.
columns of Q, of course.
And one more thing.
What's Q inverse?
For a matrix that has these orthonormal columns, So I took the dot product -- ye, somehow, it didn't -- I we know that the inverse is the same as the transpose.
So here is the beautiful -- there is the -- the great haven't learned anything.
description, the factorization of a symmetric matrix.
And this is, like, one of the famous theorems of linear algebra, that if I have a symmetric matrix, it can be factored in this form.
An orthogonal matrix times diagonal times the transpose of that orthogonal matrix.
And, of course, everybody immediately says yes, and if this is possible, then that's clearly symmetric, right?
That -- take -- we've looked at products of three guys like that and taken their transpose and we got it back again.
So do you -- do you see the beauty of this -- of this factorization, then?
It -- it completely displays the eigenvalues and eigenvectors the symmetry of the -- of the whole thing, because -- that product, Q times lambda times Q transpose, if I transpose it, it -- this comes in this position and we get that matrix back again.
So that's -- in mathematics, that's called the spectral Spectrum is the set of eigenvalues of a matrix.
theorem.
Spec- it somehow comes from the idea of the spectrum of light as a combination of pure things -- where our matrix is broken down into pure eigenvalues and eigenvectors -- in mechanics it's often called the principle axis theorem.
It's very useful.
It means that if you have -- we'll see it geometrically.
It means that if I have some material -- if I look at the right axis, it becomes diagonal, it becomes -- the -- the I- I've done something dumb, because I've got the -- I should've taken the dot product of this guy here with -- that's directions don't couple together.
Okay.
So that's -- that -- that's what to remember from -- from this lecture.
Now, I would like to say why are the eigenvalues real?
Can I do that?
So -- so -- because that -- something useful comes out.
So I'll just come back -- come to that question why real eigenvalues?
Okay.
So I have to start from the only thing we know, Ax equal lambda x.
Okay.
But as far as I know at this moment, lambda could be complex.
I'm going to prove it's not -- and x could be complex.
In fact, for the moment, even A could be -- we could even think, well, what happens if A is complex?
Well, one thing we can always do -- this is -- this is like always -- always okay -- I can -- if I have an equation, I can take the complex conjugate of everything.
That's -- no -- no -- so A conjugate x conjugate equal lambda conjugate x conjugate, it just means that everywhere over here that there was a -- an equals x bar transpose lambda bar x bar.
i, then here I changed it to a-i.
That's -- that -- you know that that step -- that conjugate business, that a+ib, if I conjugate it it's a-ib.
That's the meaning of conjugate -- and products behave right, I can conjugate every factor.
So I haven't done anything yet except to say what would be true if, x -- in any case, even if x and lambda were complex.
Of course, our -- we're speaking about real matrixes A, so I can take that out.
Actually, this already tells me something about real matrixes.
I haven't used any assumption of A -- A transpose yet.
Symmetry is waiting in the wings to be used.
This tells me that if a real matrix has an eigenvalue lambda what I was going to do.
and an eigenvector x, it also has -- another of its eigenvalues is lambda bar with eigenvector x bar.
Real matrixes, the eigenvalues come in lambda, lambda bar -- the complex eigenvalues come in lambda and lambda bar pairs.
But, of course, I'm aiming to show that they're not complex at all, here, by getting symmetry in.
So how I going to use symmetry?
I'm going to transpose this equation to x bar transpose A transpose equals x bar transpose lambda bar.
That's just a number, so I don't mind wear I put that number.
This is -- this is -- this is a -- then again okay.
Ax equals lambda x bar transpose x, right?
But now I'm ready to use symmetry.
I'm ready -- so this was all just mechanics.
Now -- now comes the moment to say, okay, if the matrix is this from the right with x bar, I get x bar transpose Ax bar symmetric, then this A transpose is the same as A.
You see, at that moment I used the assumption.
Now let me finish the discussion.
Here -- here's the way I finish.
I look at this original equation and I take the inner product.
I multiply both sides by -- oh, maybe I'll do it with this one.
I take -- I multiply both sides by x bar transpose.
x bar transpose Ax bar equals lambda bar x bar transpose x bar.
Okay, fine.
All right, now what's the other one?
Oh, for the other one I'll probably use this guy.
A- I happy about this?
No.
For some reason I'm not.
I'm -- I want to -- if I take the inner product of Okay.
So that -- that was -- that's fine.
That comes directly from that, multiplying both sides by x bar transpose, but now I'd like to get -- why do I have x bars over there?
Oh, yes.
Forget this.
Okay.
On this one -- right.
On this one, I took it like that, I multiply on the right by x.
That's the idea.
Okay.
Now why I happier with this situation now?
A proof is coming here.
Because I compare this guy with this one.
And they have the same left hand side.
So they have the same right hand side.
So comparing those two, can -- I'll raise the board to do this comparison -- this thing, lambda x bar transpose x is equal to lambda bar x bar transpose x.
Okay.
And the conclusion I'm going to reach -- I -- I on the right track here?
The conclusion I'm going to reach is lambda equal lambda bar.
I would have to track down the other possibility that this -- this thing is zero, but let me -- oh -- oh, yes, that's important.
It's not zero.
So once I know that this isn't zero, I just cancel it and I learn that lambda equals lambda bar.
And so what can you -- do you -- have you got the reasoning altogether?
What does this tell us?
Lambda's an eigenvalue of this symmetric matrix.
We've just proved that it equaled lambda bar, so we have just proved that lambda is real, right?
If, if a number is equal to its own complex conjugate, then there's no imaginary part at all.
The number is real.
So lambda is real.
Good.
Good.
Now, what -- but it depended on this little expression, on knowing that that wasn't zero, so that I could cancel it out -- so can we just take a second on that one?
Because it's an important quantity.
x bar transpose x.
Okay, now remember, as far as we know, x is complex.
So this is -- here -- x is complex, x has these components, x1, x2 down to xn.
And x bar transpose, well, it's transposed and it's conjugated, so that's x1 conjugated x2 conjugated up to xn conjugated.
I'm -- I'm -- I'm really reminding you of crucial facts about complex numbers that are going to come into the next lecture as well as this one.
So w- what can you tell me about that product -- I -- I guess what I'm trying to say is, if I had a complex vector, this would be the quantity I would -- I would like.
This is the quantity I like.
I would take the vector times its transpose -- now what -- what happens usually if I take a vector -- a -- a -- x transpose x?
I mean, that's a quantity we see all the time, x transpose x.
That's the length of x squared, right?
That's this positive length squared, it's Pythagoras, it's x1 squared plus x2 squared and so on.
Now our vector's complex, and you see the effect?
I'm conjugating one of these guys.
So now when I do this multiplication, I have x1 bar times x1 and x2 bar times x2 and so on.
So this is an -- this is sum a+ib.
And this is sum a-ib.
I mean, what's the point here?
What's the point -- when I multiply a number by its conjugate, a complex number by its conjugate, what do I get?
I get a n- the -- the imaginary part is gone.
When I multiply a+ib by its conjugate, what's -- what's the result of that -- of each of those separate little multiplications?
There's an a squared and -- and what -- how many -- what's -- b squared comes in with a plus or a minus?
A plus.
i times minus i is a plus b squared.
And what about the imaginary part?
Gone, right?
An iab and a minus iab.
So this -- this is the right thing to do.
If you want a decent answer, then multiply numbers by their conjugates.
Multiply vectors by the conjugates of x transpose.
So this quantity is positive, this quantity is positive -- the whole thing is positive except for the zero vector and that allows me to know that this is a positive number, which I safely cancel out and I reach the conclusion.
So actually, in this discussion here, I've done two things.
If I reached the conclusion that lambda's real, which I wanted to do.
But at the same time, we sort of saw what to do if things were complex.
If a vector is complex, then it's x bar transpose x, this is its length squared.
And as I said, the next lecture Monday, we'll -- we'll repeat that this is the right thing and then do the right thing for matrixes and all other -- all other, complex possibilities.
Okay.
But the main point, then, is that the eigenvalues of a symmetric matrix, it just -- do you -- do -- where did we use symmetry, by the way?
We used it here, right?
Let -- can I just -- let -- suppose A was a complex.
Suppose A had been a complex number.
Could -- could I have made all this work?
If A was a complex number -- complex matrix, then here I should have written A bar.
I erased the bar because I assumed A was real.
But now let's suppose for a moment it's not.
Then when I took this step, what should I have?
What did I do on that step?
I transposed.
So I should have A bar transpose.
In the symmetric case, that was A, and that's what made everything work, right?
This -- this led immediately to that.
This one led immediately to this when the matrix was real, so that didn't matter, and it was symmetric, so that didn't matter.
Then I got A.
But -- so now I just get to ask you.
Suppose the matrix had been complex.
What's the right equivalent of sym- symmetry?
So the good matrix -- so here, let me say -- good matrixes -- by good I mean real lambdas and perpendicular x-s. And tell me now, which matrixes are good?
If they're -- If they're real matrixes, the good ones are symmetric, because then everything went through.
The -- so the good -- I'm saying now what's good.
This is -- this is -- these are the good matrixes.
They have real eigenvalues, perpendicular eigenvectors -- good means A equal A transpose if real.
Then -- then that was what -- our proof worked.
But if A is complex, all -- our proof will still work provided A bar transpose is A.
Do you see what I'm saying?
I'm saying if we have complex matrixes and we want to say are they -- are they as good as symmetric matrixes, then we should not only transpose the thing, but conjugate it.
Those are good matrixes.
And of course, the most important s- the most important case is when they're real, this part doesn't matter and I just have A equal A transpose symmetric.
Do you -- I -- I'll just repeat that.
The good matrixes, if complex, are these.
If real, that doesn't make any difference so I'm just saying symmetric.
And of course, 99% of examples and applications to the matrixes are real and we don't have that and then symmetric is the key property.
Okay.
So that -- that's, these main facts and now let me just -- let me just -- so that's this x bar transpose x, okay.
So I'll just, write it once more in this form.
So perpendicular orthonormal eigenvectors, real eigenvalues, transposes of orthonormal eigenvectors.
That's the symmetric case, A equal A transpose.
Okay.
Good.
Actually, I'll even take one more step here.
Suppose -- I -- I can break this down to show you really what that says about a symmetric matrix.
I can break that down.
Let me here -- here go these eigenvectors.
I -- here go these eigenvalues, lambda one, lambda two and so on.
Here go these eigenvectors transposed.
And what happens if I actually do out that multiplication?
Do you see what will happen?
There's lambda one times q1 transpose.
So the first row here is just lambda one q1 transpose.
If I multiply column times row -- you remember I could do that?
When I multiply matrixes, I can multiply columns times rows?
So when I do that, I get lambda one and then the column and then the row and then lambda two and then the column and the row.
Every symmetric matrix breaks up into these pieces.
So these pieces have real lambdas and they have these Eigen -- these orthonormal eigenvectors.
And, maybe you even could tell me what kind of a matrix have I got there?
Suppose I take a unit vector times its transpose?
So column times row, I'm getting a matrix.
That's a matrix with a special name.
What's it's -- what kind of a matrix is it?
We've seen those matrixes, now, in chapter four.
It's -- is A A transpose with a unit vector, so I don't have to divide by A transpose A.
That matrix is a projection matrix.
That's a projection matrix.
It's symmetric and if I square it there'll be another -- there'll be a q1 transpose q1, which is one.
So I'll get that matrix back again.
Every -- so every symmetric matrix -- every symmetric matrix is a combination of -- of mutually perpendicular -- so perpendicular projection matrixes.
Projection matrixes.
Okay.
That's another way that people like to think of the spectral theorem, that every symmetric matrix can be broken up that way.
That -- I guess at this moment -- first I haven't done an example.
I could create a symmetric matrix, check that it's -- find its eigenvalues, they would come out real, find its eigenvectors, they would come out perpendicular and you would see it in numbers, but maybe I'll leave it here for the moment in letters.
Oh, I -- maybe I will do it with numbers, for this reason.
Because there's one more remarkable fact.
Can I just put this further great fact about symmetric matrixes on the board?
When I have symmetric matrixes, I know their eigenvalues are So then I can get interested in the question are they positive real.
or negative?
And you remember why that's important.
For differential equations, that decides between instability and stability.
So I'm -- after I know they're real, then the next question is are they positive, are they negative?
And I hate to have to compute those eigenvalues to answer that question, right?
Because computing the eigenvalues of a symmetric matrix of order let's say 50 -- compute its 50 eigenvalues -- is a job.
I mean, by pencil and paper it's a lifetime's job.
I mean, which -- and in fact, a few years ago -- well, say, 20 years ago, or 30, nobody really knew how to do it.
I mean, so, like, science was stuck on this problem.
If you have a matrix of order 50 or 100, how do you find its eigenvalues?
Numerically, now, I'm just saying, because pencil and paper is -- we're going to run out of time or paper or something before we get it.
Well -- and you might think, okay, get Matlab to compute the determinant of lambda minus A, A minus lambda I, this polynomial of 50th degree, and then find the roots.
Matlab will do it, but it will complain, because it's a very bad way to find the eigenvalues.
I'm sorry to be saying this, because it's the way I taught you to do it, right?
I taught you to find the eigenvalues by doing that determinant and taking the roots of that polynomial.
But now I'm saying, okay, I really meant that for two by twos and three by threes but I didn't mean you to do it on a 50 by 50 and you're not too unhappy, probably, because you didn't want to do it.
But -- good, because it would be a very unstable way -- the 50 answers that would come out would be highly unreliable.
So, new ways are -- are much better to find those 50 eigenvalues.
That's a -- that's a part of numerical linear algebra.
But here's the remarkable fact -- that Matlab would quite happily find the 50 pivots, right?
Now the pivots are not the same as the eigenvalues.
But here's the great thing.
If I had a real matrix, I could find those 50 pivots and I could see maybe 28 of them are positive and 22 are negative pivots.
And I can compute those safely and quickly.
And the great fact is that 28 of the eigenvalues would be positive and 22 would be negative -- that the sines of the pivots -- so this is, like -- I hope you think this -- this is kind of a nice thing, that the sines of the pivots -- for symmetric, I'm always talking about symmetric matrixes -- so I'm really, like, trying to convince you that symmetric matrixes are better than the rest.
So the sines of the pivots are same as the sines of the eigenvalues.
The same number.
The number of pivots greater than zero, the number of positive pivots is equal to the number of positive eigenvalues.
So that, actually, is a very useful -- that gives you a g- a good start on a decent way to compute eigenvalues, because you can narrow them down, you can find out how many are positive, how many are negative.
Then you could shift the matrix by seven times the identity.
That would shift all the eigenvalues by seven.
Then you could take the pivots of that matrix and you would know how many eigenvalues of the original were above seven and below seven.
So this -- this neat little theorem, that, symmetric matrixes have this connection between the -- nobody's mixing up and thinking the pivots are the eigenvalues -- I mean, the only thing I can think of is the product of the pivots equals the product of the eigenvalues, why is that?
So if I asked you for the reason on that, why is the product of the pivots for a symmetric matrix the same as the product of the eigenvalues?
Because they both equal the determinant.
Right.
The product of the pivots gives the determinant if no row exchanges, the product of the eigenvalues always gives the determinant.
So -- so the products -- but that doesn't tell you anything about the 50 individual ones, which this does.
Okay.
So that's -- those are essential facts about symmetric matrixes.
Okay.
Now I -- I said in the -- in the lecture description that I would take the last minutes to start on positive definite matrixes, because we're right there, we're ready to say what's a positive definite matrix?
It's symmetric, first of all.
On -- always I will mean symmetric.
So this is the -- this is the next section of the book.
It's about this -- if symmetric matrixes are good, which was, like, the point of my lecture so far, then positive, definite matrixes are -- a subclass that are excellent, okay.
Just the greatest.
so what are they?
They're matrixes -- they're symmetric matrixes, so all their eigenvalues are real.
You can guess what they are.
These are symmetric matrixes with all -- the eigenvalues are -- okay, tell me what to write.
What -- well, it -- it's hinted, of course, by the name for these things.
All the eigenvalues are positive.
Okay.
Tell me about the pivots.
We can check the eigenvalues or we can check the pivots.
All the pivots are what?
And then I'll -- then I'll finally give an example.
I feel awful that I have got to this point in the lecture and I haven't given you a single example.
So let me give you one.
Five three two two.
That's symmetric, fine.
It's eigenvalues are real, for sure.
But more than that, I know the sines of those eigenvalues.
And also I know the sines of those pivots, so what's the deal with the pivots?
The Ei- if the eigenvalues are all positive and if this little fact is true that the pivots and eigenvalues have the same sines, then this must be true -- all the pivots are positive.
And that's the good way to test.
This is the good test, because I can -- what are the pivots for that matrix?
The pivots for that matrix are five.
So pivots are five and what's the second pivot?
Have we, like, noticed the formula for the second pivot in a matrix?
It doesn't necessarily -- you know, it may come out a fraction for sure, but what is that fraction?
Can you tell me?
Well, here, the product of the pivots is the determinant.
What's the determinant of this matrix?
Eleven?
So the second pivot must be eleven over five, so that the product is eleven.
They're both positive.
Then I know that the eigenvalues of that matrix are both positive.
What are the eigenvalues?
Well, I've got to take the roots of -- you know, do I put in a minus lambda?
You mentally do this -- lambda squared minus how many lambdas?
Eight?
Right.
Five and three, the trace comes in there, plus what number comes here?
The determinant, the eleven, so I set that to zero.
So the eigenvalues are -- let's see, half of that is four, look at that positive number, plus or minus the square root of sixteen minus eleven, I think five.
The eigenvalues -- well, two by two they're not so terrible, but they're not so perfect.
Pivots are really simple.
And this is a -- this is the family of matrixes that you really want in differential equations, because you know the sines of the eigenvalues, so you know the stability or not.
Okay.
There's one other related fact I can pop in here in -- in the time available for positive definite matrixes.
The related fact is to ask you about determinants.
So what's the determinant?
What can you tell me if I -- remember, positive definite means all eigenvalues are positive, all pivots are positive, so what can you tell me about the determinant?
It's positive, too.
But somehow that -- that's not quite enough.
Here -- here's a matrix minus one minus three, what's the determinant of that guy?
It's positive, right?
Is this a positive, definite matrix?
Are the pivots -- what are the pivots?
Well, negative.
What are the eigenvalues?
Well, they're also the same.
So somehow I don't just want the determinant of the whole matrix.
Here is eleven, that's great.
Here the determinant of the whole matrix is three, that's positive.
I also -- I've got to check, like, little sub-determinants, say maybe coming down from the left.
So the one by one and the two by two have to be positive.
So there -- that's where I get the all.
All -- can I call them sub-determinants -- are -- see, I have to -- I need to make the thing plural.
I need to test n things, not just the big determinant.
All sub-determinants are positive.
Then I'm okay.
Then I'm okay.
This passes the test.
Five is positive and eleven is positive.
This fails the test because that minus one there is negative.
And then the big determinant is positive three.
So t- this -- these -- this fact -- you see that actually the course, like, coming together.
And that's really my point now.
In the next -- in this lecture and particularly next Wednesday and Friday, the course comes together.
These pivots that we met in the first week, these determinants that we met in the middle of the course, these eigenvalues that we met most recently -- all matrixes are square here, so coming together for square matrixes means these three pieces come together and they come together in that beautiful fact, that if -- that all the -- that if I have one of these, I have the others.
That if I -- but for symmetric matrixes.
So that -- this will be the positive definite section and then the real climax of the course is to make everything come together for n by n matrixes, not necessarily symmetric -- bring everything together there and that will be the final thing.
Okay.
So have a great weekend and don't forget symmetric matrixes.
Thanks.
Okay.
This is a lecture where complex numbers come in.
It's a -- complex numbers have slipped into this course because even a real matrix can have complex eigenvalues.
So we met complex numbers there as the eigenvalues and complex eigenvectors.
And we -- or -- this is probably the last -- we have a lot of other things to do about eigenvalues and eigenvectors.
And that will be mostly real.
But at one point somewhere, we have to see what you do when the numbers become complex numbers.
What happens when the vectors are complex, when the matrixes are complex, when the -- what's the inner product of two, the dot product of two complex vectors -- we just have to make the change, just see -- what is the change when numbers become complex?
Then, can I tell you about the most important example of complex matrixes?
It comes in the Fourier matrix.
So the Fourier matrix, which I'll describe, is a complex matrix.
It's certainly the most important complex matrix.
It's the matrix that we need in Fourier transform.
And the -- really, the special thing that I want to tell you about is what's called the fast Fourier transform, and everybody refers to it as the FFT and it's in all computer and it's used -- it's being used as we speak in a thousand places, because it has, like, transformed whole industries to be able to do the Fourier transform fast, which means multiplying -- how do I multiply fast by that matrix -- by that n by n matrix?
Normally, multiplications by an n by n matrix -- would normally be n squared multiplications, because I've got n squared entries and none of them is zero.
This is a full matrix.
And it's a matrix with orthogonal columns.
I mean, it's just, like, the best matrix.
And this fast Fourier transform idea reduces this n squared, which was slowing up the calculation of Fourier transforms down to n log(n).
n log(n), log to the base two, actually.
And it's this -- when that hit -- when that possibility hit, it made a big difference.
Everybody realized gradually what, -- that this simple idea -- you'll see it's just a simple matrix factorization -- but it changed everything.
Okay.
So I want to talk about complex vectors and matrixes in general, recap a little bit from last time, and the Fourier matrix in particular.
Okay.
So what's the deal?
All right.
The main point is, what about length?
I'm given a vector, I have a vector x.
Or let me call it z as a reminder that it's complex, for the moment.
But I can -- later I'll call the components x.
They'll be complex numbers.
But it's a vector -- z1, z2 down to zn.
So the only novelty is it's not in R^n anymore.
It's in complex n dimensional space.
Each of those numbers is a complex number.
So this z,z1 is in C^n, n dimensional complex space instead of R^n.
So just a different letter there, but now the point about its length is what?
The point about its length is that z transpose z is no good.
z transpose z -- if I just put down z transpose here, it would be z1, z2, to zn.
Doing that multiplication doesn't give me the right thing.
W-Why not?
Because the length squared should be positive.
And if I multiply -- suppose this is, like, 1 and i.
What's the length of the vector with components 1 and i?
What if I do this, so n is just two.
I'm in C^2, two dimensional space, complex space with the vector whose components are 1 and i.
All right.
So if I took one times one and i times i and added, z transpose z would be zero.
But I don't -- that vector is not -- doesn't have length zero -- the vector with the components 1 and i -- this multiplication -- what I really want is z1 conjugate z1.
You remember that z1 conjugate z1 is -- so you see that first step will be z1 conjugate z1, which is the magnitude of z1 squared, which is what I want.
That's, like, three squared or five squared.
Now, if it's -- if z1 is i, then I multiplied by minus i gives one plus one, so the component of length -- the component i, its modulus squared is plus one.
That's great.
So what I want to do then is do that -- I want z1 bar z1, z2 bar z2, zn bar zn.
And remember that -- you remember this complex conjugate.
So -- so there's the point.
Now I can erase the no good and put is good, because that now gives the answer zero for the zero vector, of course, but it gives a positive length squared for any other vector.
So it's a -- it's the right definition of length, and essentially the message is that we're always going to be taking -- when we transpose, we also take complex conjugate.
So let's -- let's find the length of one -- so the vector one i, that's z, that's that vector z.
Now I take the conjugate of one is one, the conjugate of i is minus i. I take this vector, I get one plus one -- I get two.
So that's a vector and that's a vector of length -- square root of two.
Square root of two is the length and not the zero that we would have got from one minus i squared.
Okay.
So the message really is whenever we transpose, we also take conjugates.
So here's a symbol -- one symbol to do both.
So that symbol H, it stands for a guy named Hermite, who didn't actually pronounce the H, but let's pronounce it -- so I would call that z Hermitian z. I'll -- let me write that word, Herm- so his name was Hermite, and then we make it into an adjective, Hermitian.
So z Hermitian z. z H z.
Okay.
So, that's the -- that's the, length squared.
Now what's the inner product?
Well, it should match.
The inner product of two vectors -- so inner product is no longer -- used to be y transpose x.
That's for real vectors.
For complex vectors, whenever we transpose, we also take the conjugate.
So it's y Hermitian x.
Of course it's not real anymore, usually.
That -- the inner product will usually be complex number.
But if y and x are the same, if they're the same z, then we have z -- z H z, we have the length squared, and that's what we want, the inner product of a vector with itself should be its length squared.
So this is, like, forced on us because this is forced on us.
So -- so this z -- this -- everybody's picking up what this equals.
This is z1 squared plus zn squared.
That's the length squared.
And that's the inner product that we have to go with.
So it could be a complex number now.
One more change.
Well, two more changes.
We've got to change the idea of a symmetric matrix.
So I'll just recap on symmetric matrixes.
Symmetric means A transpose equals A, but not -- no good if A is complex.
So what do we instead -- that applies perfectly to real matrixes.
But now if my matrixes were complex, I want to take the transpose and the conjugate to equal A.
So there's -- that's the -- the right complex version of symmetry.
The com- the symmetry now means when I transpose it, flip across the diagonal and take conjugates.
So, for example -- here would be an example.
On the diagonal, it had better be real, because when I flip it, the diagonal is still there and it has to -- and then when I take the complex conjugate it has to be still there, so it better be a real number, let me say two and five.
What about entries off the diagonal?
If this entry is, say, three plus i, then this entry had better be -- because I want whatever this -- when I transpose, it'll show up here and i conjugate.
So I need three minus I there.
So there's a matrix with -- that corresponds to symmetry, but it's complex.
And those matrixes are called Hermitian matrixes.
Hermitian matrixes.
A H equals A.
Fine.
Okay, that's -- and those matrixes have real eigenvalues and they have perpendicular eigenvectors.
What does perpendicular mean?
Perpendicular means the inner product -- so let's go on to perpendicular.
Well, when I had perpendicular vectors, for example, they were like q1, q2 up to qn.
That's my -- q is my letter that I use for perpendicular.
Actually, I usually -- I also mean unit length.
So those are perpendicular unit vectors.
But now what does -- so it's a -- orthonormal basis, I'll still use those words, but how do I compute perpendicular?
How do I check perpendicular?
This means that the inner product of qi with qj -- but now I not only transpose, I must conjugate, right, to get zero if i is not j and one if i is j.
So it's a unit vector, meaning unit length, orthogonal -- all the angles are right angles, but these are angles in complex n dimensional space.
So it's q1, q on- qi bar transpose.
Or, for short, qi H qj.
So it will still be true -- so let me -- again I'll create a matrix out of those guys.
The matrix will have these q-s in its columns, q2 to qn.
And I want to turn that into matrix language, just like before.
What does that mean?
That means I want all these inner products, so I take these columns of Q, multiply by their rows -- so it was -- it used to be Q -- it used to be Q transpose Q equals I, right?
This was an orthogonal matrix.
But what's changed?
These are now complex vectors.
Their inner products are -- involve conjugating the first factor.
So it's not -- it's the conjugate of Q transpose.
It's Q bar transpose Q. Q H. So can I call this -- let me call it Q H Q, which is I.
So that's our new -- you -- you see I'm just translating, and the -- the book h- on one page gives a little dictionary of the right words in the real case, R^n, and the corresponding words in the complex case for the vector space C^n.
Of course, C^n is a vector space, the numbers we multiply are now complex numbers -- we're just moving into complex n dimensional space.
Okay.
Now -- actually, I have to say we changed the word symmet- symmetric to Hermitian for those matrixes.
People also change this word orthogonal into another word that happens to be unitary, as a word that applies -- that signals that we might be dealing with a complex matrix here.
So what's a unitary matrix?
It's a -- it's just like an orthogonal matrix.
It's a square, n by n matrix with orthonormal columns, perpendicular columns, unit vectors -- unit vectors computed by -- and perpendicularity computed by remembering that there's a conjugate as well as a transpose.
Okay.
So those are the words.
Now I'm ready to get into the substance of the lecture which is the most famous complex matrix, which happens to be one of these guys.
It has orthogonal columns, and it's named after Fourier because it comes into the Fourier transform, so it's the matrix that's all around us.
Okay.
Let me tell you what it is first of all in the n by n case.
Then often I'll let n be four because four is a good size to work with.
But here's the n by n Fourier matrix.
Its first column is the vector of ones.
It's n by n, of course.
Its second column is the powers, the -- actually, better if I move from the math department to EE for this one half hour and then, please, let me move back again.
Okay.
What's the difference between those two departments?
It's just math starts counting with one and electrical engineers start counting at zero.
Actually, they're probably right.
So anyway, we'll give them -- humor them.
So this is really the zeroes column.
And the first column up to the n-1, that's the one inconvenient spot in electrical engineering.
All these expressions start at zero, no problem, but they end at n-1.
Well, that's -- that's the difficulty of Course 6.
So what's -- they're the powers of a number that I'm going to call W -- W squared, W cubed, W to the -- now what is the W here?
What's the power?
This was the zeroes power, first power, second power, this will be n minus first power.
That's the column.
What's the next column?
It's the powers of W squared, W to the fourth, W to the sixth, W to the two n-1.
And then more columns and more columns and more columns and what's the last column?
It's the powers of -- let's see.
We -- actually, if we look around rows, w- this matrix is symmetric.
It's symmetric in the old not quite perfect way, not perfect because these numbers are complex.
And so it's -- that first row is all ones.
One W W squared up to W to the n-1.
That's the last column is the powers of W to the n-1, so this guy matches that, and finally we get W to something here.
I guess we could actually figure out what that something is.
What are the entries of this matrix?
The i j entry of this matrix are -- I going to -- are you going to allow me to let i go from zero to n minus one?
So i and g go from zero to n-1.
So the one -- the zero zero entry is a one -- it's just this same W guy to the power i times j.
Let's see.
I'm jumping into formulas here and I have to tell you what W is and then you know everything about this matrix.
So W is the -- well, shall we finish here?
What was this -- this is the (n-1) (n-1) entry.
This is W to the n-1 squared.
Everything's looking like a mess here, because we have -- not too bad, because all the entries are powers of W. There -- none of them are zero.
This is a full matrix.
But W is a very special number.
W is the special number whose n-th power is one.
In fact -- well, actually, there are n numbers like that.
One of them is one, of course.
But the one we -- the W we want is -- the angle is two pi over n. Is that what I mean?
n over two pi.
No, two pi over n. W is E to the I and the angle is two pi over n. Right.
Where is this W in the complex plane?
It's -- it's on the unit circle, right?
This is -- it's the cosine of two pi over n plus I times the sine of two pi over n. But actually, forget this.
It's never good to work with the real and imaginary parts, the rectangular coordinates, when we're taking powers.
To take that to the tenth power, we can't see what we're doing.
To take this form to the tenth power, we see immediately what we're doing.
It would be e to the i 20 pi over n. So when our matrix is full of powers -- so it's this formula, and where is this on the complex plain?
Here are the real numbers, here's the imaginary axis, here's the unit circle of radius one, and this number is on the unit circle at this angle, which is one n-th of the full way round.
So if I drew, for example, n equals six, this would be e to the two pi, two pi over six, it would be one sixth of the way around, it would be 60 degrees.
And where is W squared?
So I -- my W is e to the two pi I over six in this case, in this six by -- for the six by six Fourier transform, it's totally constructed out of this number and its powers.
So what are its powers?
Well, its powers are on the unit circle, right?
Because when I square a number, a complex number, I square its absolute value, which gives me one again.
All the powers have -- are on the unit circle.
And the -- the angle gets doubled to a hundred and twenty, so there's W squared, there's W cubed, there's W to the fourth, there's W to the fifth and there is W to the sixth, as we hoped, W to the sixth coming back to one.
So those are the six -- can I say this on TV?
The six sixth roots of one, and it's this one, the primitive one we say, the first one, which is W. Okay, so what -- let me change -- let me -- I said I would probably switch to n equal four.
What's W for that?
It's the fourth root of one.
W to the fourth will be one.
W will be e to the two pi i over four now.
What's that?
This is e to the i pi over two.
This is a quarter of the way around the unit circle, and that's exactly i, a quarter of the way around.
And sure enough, the powers are i, i squared, which is minus one, i cubed, which is minus i and finally i to the fourth which is one, right.
So there's W, W squared, W cubed, W to the fourth -- I'm really ready to write down this Fourier matrix for the four by four case, just so we see that clearly.
Let me do it here.
F4 is -- all right, one one one one one one one W -- it's I. I squared.
That's minus one.
i cubed is minus i. I'll -- I could write i squared and i cubed.
Why don't I, just so we see the pattern for sure.
i squared, i cubed, i squared, i cubed, i fourth, i sixth -- i fourth, i sixth and i ninth.
You see the exponents fall in this nice -- the exponent is the row number times the column number, always starting at zero.
Okay.
And now I can put in those numbers if you like -- one one one one, one i minus one minus i, one minus one, one minus one and one minus i minus one i.
No.
Yes.
Right.
What's -- why do I think that matrix is so remarkable?
It's the four by four matrix that comes into the four point Fourier transform.
When we want to find the Fourier transform, the four point Fourier transform of a vector with four components, we want to multiply by this F4 or we want to multiply by F4 inverse.
One way we're taking the transform, one way we're taking the inverse transform.
Actually, they're so close that it's easy to confuse the two.
The inverse of this matrix will be a nice matrix also.
So -- and that's, of course, what makes it -- that -- I guess Fourier knew that.
He knew the inverse of this matrix.
A- as you'll see, it just comes from the fact that the columns are orthogonal -- from the fact that the columns are orthogonal, we will quickly figure out what is the inverse.
What Fourier didn't know -- didn't notice -- I think Gauss noticed it but didn't make a point of it and then it turned out to be really important was the fact that this matrix is so special that you can break it up into nice pieces with lots of zeroes, factors that have lots of zeroes and multiply by it or by its inverse very, very fast.
Okay.
But how did it get into this lecture first?
Because the columns are orthogonal.
Can I just check that the columns of this matrix are orthogonal?
So the inner product of that column with that column is zero.
The inner product of column one with column three is zero.
How about the inner product of two and four?
Can I take the inner product of column two with column four?
Or even the inner product of two with three, let's -- let's see, does that -- let me do two and four.
Okay.
What -- oh, I see, yes, hmm.
Hmm.
Let's see, I believe that those two columns are orthogonal.
So let me take their inner product and hope to get zero.
Okay, now if you hadn't listened to the first half of this lecture, when you took the inner product of that with that, you would have multiplied one by one, i by minus i, and that would have given you one, minus one by minus one would have given you another one minus I by I would have been minus I squared, that's another one.
So do I conclude that the inner product of columns -- I said columns two and four, that's because I forgot those are columns one and three.
I'm interested in their inner product and I'm hoping it's zero, but it doesn't look like zero.
Nevertheless, it is zero.
Those columns are perpendicular.
Why?
Because the inner product -- we conjugate.
Do you remember that the -- one of the vectors in the inner product has to get conjugated.
So when I conjugated, it changes that i to a minus i, changes this to a plus i, changes those -- that second sine and that fourth sine and I do get zero.
So those columns are orthogonal.
So columns are orthogonal.
They're not quite orthonormal.
But I could fix that easily.
They -- all those columns have length two.
Length squared is four, like this -- the four I had there -- this length squared, one plus -- one squared one squared one squared one squared is four, square root is two -- so if I really wanted them -- suppose I really wanted to fix life perfectly, I could divide by two, and now I have columns that are actually orthonormal.
So what?
So I can invert right away, right?
O- orthonormal columns means -- now I'm keeping this one half in here for the moment -- c- means F4 Hermitian, can I use that, conjugate transpose times F4 is i.
So I see what the inverse is.
The inverse of F4 is -- it's just like an -- an orthogonal matrix.
The inverse is the transpose -- here the inverse is the conjugate transpose.
So, fine.
That -- that tells me that anything good that I learn about F4 I'll know the same -- I'll know a similar fact about its inverse, because its inverse is just its conjugate transpose.
Okay, now -- so what's good?
Well, first, the columns are orthogonal.
That's a key fact, then.
That's the thing that makes the inverse easy.
But what property is it that leads to the fast Fourier transform?
So now I'm going to talk, in these last minutes, about the fast Fourier transform.
What -- here's the idea.
F6, our six by six matrix, will c- there's a neat connection to F3, half as big.
There's a connection of F8 to F4.
There's a connection of F(64) to F(32).
Shall I write down what that connection is?
What's the connection of F(64) to F(32)?
So F(64) is a 64 by 64 matrix whose W is the 64th root of one.
So it's one 64th of the way round in F(64).
And it -- do- and F(32) is a 32 by 32 matrix.
Remember, they're different sizes.
And the W in that 32 by 32 matrix is the 32nd root of one, which is twice as far -- that -- you sh- see that key point -- that's the -- that's how 32 and 64 are connected in the Ws.
The W for 64 is one 64th of the way -- so all I'm saying is that if I square the W -- W(64), that's what I'm using for the one over -- the -- W sixty f- this Wn is either the i two pi over n -- so W(64) is one 64th of the way around it.
When I square that, what do I get but W(32)?
Right?
If I square this matrix, I double the angle -- if I square this number, I double the angle, I get, the -- the W(32).
So somehow there's a little hope here to connect F(64) with F(32).
And here's the connection.
Okay.
Let me -- let me go back, -- yes, let me -- I'll do it here.
Here's the connection.
F(64).
The 64 by 64 Fourier matrix is connected to two copies of F(32).
Let me leave a little space for the connection.
So this is 64 by 64.
Here's a matrix of that same size, because it's got two copies of F(32) and two zero matrixes.
Those zero matrixes are the key, because when I multiply by this matrix, just as it is, regular multiplication, I would take -- need 64 -- I would -- I would have 64 squared little multiplications to do.
But this matrix is half zero.
Well, of course, the two aren't equal.
I'm going to put an equals sign, but there has to be some fix up factors -- one there and one there -- to make it true.
The beauty is that these fix up factors will be really -- almost all zeroes.
So that as soon as we get this formula right, we've got a great idea for how to get from the sixty- from the 64 squared calculations -- so this original -- originally we have 64 squared calculations from there, but this one will give us -- this is -- this will -- we don't need that many -- we only need two times 32 squared, because we've got that twice.
And -- plus the fix-up.
So I have to tell you what's in this fix-up matrix.
The one on the right is actually a permutation matrix, a very simple odds and evens permutation matrix, the -- ones show up -- I haven't put enough ones, I really need a -- 32 of these guys at -- double space and then -- you see it's -- it's a permutation matrix.
What it does -- shall I call it P for permutation matrix?
So what that P does when it multiplies a vector, it takes the odd -- the even numbered components first and then the odds.
You see this -- this one skipping every time is going to pick out x0, x2, x4, x6 and then below that will come -- will pick out x1, x3, x5.
And of course, that can be hard wired in the computer to be instantaneous.
So that says -- so far, what have we said?
We're saying that the 64 by 64 Fourier matrix is really separated into -- separate your vector into the odd -- into the even components and the odd components, then do a 32 size Fourier transform onto those separately, and then put the pieces together again.
So the pieces -- putting them together turns them out to be I and a diagonal matrix and I and a minus, that same diagonal matrix.
So the fix-up cost is really the cost of multiplying by D, this diagonal matrix, because there's essentially no cost in -- in the I part or in the permutation part, so really it's -- the fix-up cost is essentially because D is diagonal -- is 32 multiplications.
That's the -- there you're seeing -- of course we didn't check the formula or we didn't even say what D is yet, but I will -- this diagonal matrix D is powers of W -- one W W squared down to W to the 31st.
So you see that when I -- to do a multiplication by D, I need to do 32 multiplications.
There they are.
Then -- but the other, the more serious work is to do the F(32) twice on the -- separately on the even numbered and odd numbered components, so twice 32 squared.
So 64 squared is gone now.
And that's the new count.
Okay, great, but what next?
So that's -- I -- we now have the key idea -- we would have to check the algebra, but it's just checking a lot of sums that come out correctly.
This is right -- the right way to see the fast Fourier transform, or one right way to see it.
Then you've got to see what's the next idea.
The next idea is to break the 32s down.
Break those 32s down.
So we have this factor, and now we have the F(32), but that breaks into some guy here -- F thirty- F six- F(16) -- F(16).
Each -- each F(32) is breaking into two copies of F(16), and then we have a permutation and then the -- so this is a -- like, this was a 64 size permutation, this is a 32 size permutation -- I guess I've got it twice.
So it's -- I'm -- I'm just using the same idea recursively -- recursion is the key word -- that on each of those F(32)s -- so here's zero zero -- it's just -- to get F(32) -- this is the odd even permutations -- so you see, we're -- the combination of those permutations, what's it doing?
This guy separates into odds -- in -- into evens and odds, and then this guy separates the evens into the ones -- the numbers that are mult- the even evens, which means zero four eight sixteen -- and even odds, which means two, six, ten, fourteen -- and then odd evens and odd odds.
You see, together these permutations then break it -- break our vector down into x, even even and three other pieces.
Those are the four pieces that separately get multiplied by F(16) -- separately fixed up by these Is and Ds and Is and minus Ds -- so this count is now reduced.
This count is now -- what's it reduced to?
So that's going to be gone, because 32 squared -- that's -- that's the change I'm making, right?
The 32 squared -- w- so -- so it's this that's now reduced.
So I still have two times it, but now what's 32 squared?
It's gone in favor of two sixteen squareds plus sixteen.
That's -- and then the original 32 to fix.
Maybe you see what's happening.
Even easier than this formula is w- what's -- when I do the recursion more and more times, I get simpler and simpler factors in the middle.
Eventually I'll be down to two point or one point Fourier transforms.
But I get more and more factors piling up on the right and left.
On the right, I'm just getting permutation matrixes.
On the left, I'm getting these guys, these Is and Ds, so that there was a 32 there and -- each one of these is costing 32.
Each one of those is costing And how many will there be?
So you see the 32 for this original fix up, because D had 32 numbers, 32 for this next fix up, because D has 16 and 16 more.
I keep going.
So the count in the middle goes down to zip, but these fix up counts are all that I'm left with, and how many factors -- how many fix-ups have I got -- log in -- from 64, one step to 32, one step to 16, one step to eight, four, two and one.
Six steps.
So I have six fix-up -- six fix up factors.
Finally I get to six times the That's my final count.
Instead of 64 squared, this is log to the base two of 64 times 64 -- actually, half of 64.
So actually, the final count is n log to the base two of n -- that's the 32 -- a half.
So can I put a box around that wonderful, extremely important and satisfying conclusion -- that the fast Fourier transform multiplies by an n by n matrix, but it does it not in n squared steps, but in one half n log n steps.
And if we just -- complete by doing a count, let's suppose -- suppose -- a typical case would be two to the tenth.
Now n squared is bigger than a million.
So it's a thousand twenty four times a thousand twenty four.
But what is n -- what is one half -- what is the new count, done the right way?
It's n -- the thousand twenty four times one half, and what's the logarithm?
It's ten.
So times ten over two.
So it's five times -- it's five times a thousand twenty four, where this one was a thousand twenty four times a thousand twenty four.
We've reduced the calculation by a factor of 200 just by factoring the matrix properly.
This was a thousand times n, we're now down to five times n. So we can do 200 Fourier transforms, where before we could do one, and in real scientific calculations where Fourier transforms are happening all the time, we're saving a factor of in one of the major steps of modern scientific computing.
So that's the idea of the fast Fourier transform, and you see the whole thing hinged on being a special matrix with orthonormal columns.
Okay, that's actually it for complex numbers.
I'm back next time really to -- to real numbers, eigenvalues and eigenvectors and the key idea of positive definite matrixes is going to show up.
What's a positive definite matrix?
And it's terrific that this course is going to reach positive definiteness, because those are the matrixes that you see the most in applications.
Okay, see you next time.
Thanks.
OK, this is the lecture on positive definite matrices.
I made a start on those briefly in a previous lecture.
One point I wanted to make was the way that this topic brings the whole course together, pivots, determinants, eigenvalues, and something new- four plot instability and then something new in this expression, x transpose Ax, actually that's the guy to watch in this lecture.
So, so the topic is positive definite matrix, and what's my goal?
First, first goal is, how can I tell if a matrix is positive definite?
So I would like to have tests to see if you give me a, a five by five matrix, how do I tell if it's positive definite?
More important is, what does it mean?
Why are we so interested in this property of positive definiteness?
And then, at the end comes some geometry.
Ellipses are connected with positive definite things.
Hyperbolas are not connected with positive definite things, so we- it's this, we, there's a geometry too, but mostly it's linear algebra and -- this application of how do you recognize 'em when you have a minim is pretty neat.
OK.
I'm gonna begin with two by two.
All matrices are symmetric, right?
That's understood; the matrix is symmetric, now my question is, is it positive definite?
Now, here are some -- each one of these is a complete test for positive definiteness.
If I know the eigenvalues, my test is are they positive?
Are they all positive?
If I know these -- so, A is really -- I look at that number A, here, as the, as the one by one determinant, and here's the two by two determinant.
So this is the determinant test.
This is the eigenvalue test, this is the determinant test.
Are the determinants growing in s- of all, of all end, sort of, can I call them leading submatrices, they're the first ones the northwest, Seattle submatrices coming down from from there, they all, all those determinants have to be positive, and then another test is the pivots.
The pivots of a two by two matrix are the number A for sure, and, since the product is the determinant, the second pivot must be the determinant divided by A.
And then in here is gonna come my favorite and my new idea, the, the, the the one to catch, about x transpose Ax being positive.
But we'll have to look at this guy.
This gets, like a star, because for most, presentations, the definition of positive definiteness would be this number four and these numbers one two three would be test four.
OK. Maybe I'll tuck this, where, you know, OK.
So I'll have to look at this x transpose Ax.
Can you, can we just be sure, how do we know that the eigenvalue test and the determinant test, pick out the same matrices, and let me, let's just do a few examples.
Some examples.
Let me pick the matrix two, six, six, tell me, what number do I have to put there for the matrix to be positive definite?
Tell me a sufficiently large number that would make it positive definite?
Let's just practice with these conditions in the two by two case.
Now, when I ask you that, you don't wanna find the eigenvalues, you would use the determinant test for that, so, the first or the pivot test, that, that guy is certainly positive, that had to happen, and it's OK. How large a number here -- the number had better be more than what?
More than eighteen, right, because if it's eight -- no.
More than what?
Nineteen, is it?
If I have a nineteen here, is that positive definite?
I get thirty eight minus thirty six, that's OK.
If I had an eighteen, let me play it really close.
If I have an eighteen there, then I positive definite?
Not quite.
I would call this guy positive, so it's useful just to see that this the borderline.
That matrix is on the borderline, I would call that matrix positive semi-definite.
And what are the eigenvalues of that matrix, just since we're given eigenvalues of two by twos, when it's semi-definite, but not definite, then the -- I'm squeezing this eigenvalue test down, -- what's the eigenvalue that I know this matrix has?
What kind of a matrix have I got here?
It's a singular matrix, one of its eigenvalues is zero.
That has an eigenvalue zero, and the other eigenvalue is -- from the trace, twenty.
OK.
So that, that matrix has eigenvalues greater than or equal to zero, and it's that "equal to" that brought this word "semi-definite" in.
And, the what are the pivots of that matrix?
So the pivots, so the eigenvalues are zero and twenty, the pivots are, well, the pivot is two, and what's the next pivot?
There isn't one.
We got a singular matrix here, it'll only have one pivot.
You see that that's a rank one matrix, two six is a -- six eighteen is a multiple of two six, the matrix is singular it only has one pivot, so the pivot test doesn't quite The -- let me do the x transpose Ax.
pass.
Now this is -- the novelty now.
OK. What I looking at when I look at this new combination, x transpose Ax.
x is any vector now, so lemme compute, so any vector, lemme call its components x1 and x2, so that's x.
And I put in here A.
Let's, let's use this example two six, six eighteen, and here's x transposed, so x1 x2.
We're back to real matrices, after that last lecture that- that said what to do in the complex case, let's come back to real matrices.
So here's x transpose Ax, and what I'm interested in is, what do I get if I multiply those together?
I get some function of x1 and x2, and what is it?
Let's see, if I do this multiplication, so I do it, lemme, just, I'll just do it slowly, x1, x2, if I multiply that matrix, this is 2x1, and 6x2s, and the next row is 6x1s and 18x2s, and now I do this final step and what do I have?
I've got 2x1 squareds, got 2X1 squareds is coming from this two, I've got this one gives me eighteen, well, shall I do the ones in the middle?
How many x1 x2s do I have?
Let's see, x1 times that 6x2 would be six of 'em, and then x2 times this one will be six more, I get twelve.
So, in here is going, this is the number a, this is the number 2b, and in here is -- x2 times eighteen x2 will be eighteen x2 squareds and this is the number c. So it's ax1 -- it's like ax squared.
2bxy and cy squared.
For my first point that I wanted to make by just doing out a multiplication is, that is as soon as you give me the matrix, as soon as you give me the matrix, I can -- those are the numbers that appear in -- I'll call this thing a quadratic, you see, it's not linear anymore.
Ax is linear, but now I've got an x transpose coming in, I'm up to degree two, up to degree two, maybe quadratic is the -- use the word.
A quadratic form.
It's purely degree two, there's no linear part, there's no constant part, there certainly no cubes or fourth powers, it's all second degree.
And my question is -- is that quantity positive or not?
That's -- for every x1 and x2, that is my new definition -- that's my definition of a positive definite matrix.
If this quantity is positive, if, if, if, it's positive for all x's and y's, all x1 x2s, then I call them -- then that's the matrix is positive definite.
Now, is this guy passing our test?
Well we have, we anticipated the answer here by, by asking first about eigenvalues and pivots, and what happened?
It failed.
It barely failed.
If I had made this eighteen down to a seven, it would've totally failed.
I do that with the eraser, and then I'll put back eighteen, because, seven is such a total disaster, but if -- I'll keep seven for a second.
Is that thing in any way positive definite?
No, absolutely not.
I don't know its eigenvalues, but I know for sure one of them is negative.
Its pivots are two and then the next pivot would be the determinant over two, and the determinant is -- what, what's the determinant of this thing?
Fourteen minus thirty six, I've got a determinant minus twenty two.
The next pivot will be -- the pivots now, of this thing are two and minus eleven or something.
Their product being minus twenty two the determinant.
This thing is not positive definite.
What would be -- let me look at the x transpose Ax for this guy.
What's -- if I do out this multiplication, this eighteen is temporarily changing to a seven.
This eighteen is temporarily changing to a seven, and I know that there's some numbers x1 and x2 for which that thing, that function, is negative.
And I'm desperately trying to think what they are.
Maybe you can see.
Can you tell me a value of x1 and x2 that makes this quantity negative?
Oh, maybe one and minus one?
Yes, that's -- in this case, that, will work, right, if I took x1 to be one, and x2 to be minus one, then I always get something positive, the two, and the seven minus one squared, but this would be minus twelve and the whole thing would be negative; I would have two minus twelve plus seven, a negative.
If I drew the graph, can I get the little picture in here?
If I draw the graph of this thing?
So, graphs, of the function f(x,y), or f(x), so I say here f(x,y) equal this -- x transpose Ax, this, this this ax squared, 2bxy, and cy squared.
And, let's take the example, with these numbers.
OK, so here's the x axis, here's the y axis, and here's -- up is the function.
z, if you like, or f. I apologize, and let me, just once in my life here, put an arrow over these, these, axes so you see them.
That's the vector and I just, see, instead of x1 and x2, I made them x- the components x and y. OK.
So, so, what's a graph of 2x squared, twelve xy, and seven y squared?
I'd like to see -- I not the greatest artist, but let's -- tell me something about this graph of this function.
Whoa, tell me one point that it goes through.
The origin.
Right?
Even this artist can get this thing to go through the origin, when these are zero, I, I certainly get zero.
OK.
Some more points.
If x is one and y is zero, then I'm going upwards, so I'm going up this way, and I'm, I'm going up, like, two x squared in that direction.
So -- that's meant to be a parabola.
And, suppose x stays zero and y increases.
Well, y could be positive or negative; it's seven y squared.
Is this function going upward?
In the x direction it's going upward, and in the y direction it's going upwards, and if x equals y then the forty-five degree direction is certainly going upwards; because then we'd have what, about, everything would be positive, but what?
This function -- what's the graph of this function?
Look like?
Tell me the word that describes the graph of this function.
This is the non-positive definite here, everybody's with me here, for some reason got started in a negative direction, your case that isn't positive definite.
And what's the graph look like that goes up, but does it -- do we have a minimum here, does it go from, from the origin?
Completely?
No, because we just checked that this thing failed.
It failed along the direction when x was minus y -- we have a saddle point, let me put myself, let me, to the least, tell you the word.
This thing, goes up in some directions, but down in other directions, and if we actually knew what a saddle looked like or thinks saddles do that -- the way your legs go is, like, down, up, the way, you, looking like, forward, and, the, and drawing the thing is even worse than describing -- I'm just going to say in some directions we go up and in other directions, there is, a saddle -- Now I'm sorry I put that on the front board, you have no way to cover it, but it's a saddle.
OK. And, and this is a saddle point, it's the, it's the point that's at the maximum in some directions and at the minimum in other directions.
And actually, the perfect directions to look are the eigenvector directions.
We'll see that.
So this is, not a positive definite matrix.
OK. Now I'm coming back to this example, getting rid of this seven, let's move it up to twenty.
Let's, let's let's make the thing really positive definite.
OK.
So this is, this number's now twenty.
c is now twenty.
OK. Now that passes the test, which I haven't proved, of course, it passes the test for positive pivots.
It passes the test for positive eigenvalues.
How can you tell that the eigenvalues of that matrix are positive without actually finding them?
Of course, two by two I could find them, but can you see -- how do I know they're positive?
I know that their product is -- I know that lambda one times lambda two is positive, why?
Because that's the determinant, right, lambda one times lambda two is the determinant, which is forty minus thirty-six is four.
So the determinant is four.
And the trace, the sum down the diagonal, is twenty-two.
So, they multiply to give four.
So that leaves the possibility they're either both positive or both negative.
But if they're both negative, the trace couldn't be So they're both positive.
twenty-two.
So both of the eigenvalues that are positive, both of the pivots are positive -- the determinants are positive, and I believe that this function is positive everywhere except at zero, zero, of course.
When I write down this condition, So I believe that x transposed, let me copy, x transpose Ax is positive, except, of course, at the minimum point, at zero, of course, I don't expect miracles.
So what does its graph look like, and how do I check, and how do I check that this really is positive?
So we take it's graph for a minute.
What would be the graph of that function -- it does not have a saddle point.
Let me -- I'll raise the board, here, and stay with this example for a while.
So I want to do the graph of -- here's my function, two x squared, twelve xy-s, that could be positive or negative, and twenty y squared.
But my point is, so you're seeing the underlying point is, that, the things are positive definite when in some way, these, these pure squares, squares we know to be positive, and when those kind of overwhelm this guy, who could be m- positive or negative, because some like or have same or have same or different signs, when these are big enough they overwhelm this guy and make the total thing positive, and what would the graph now look like?
Let me draw the x - well, let me draw the x direction, the y direction, and the origin, at zero, zero, I'm there, where do I go as I move away from the origin?
Where do I go as I move away from the origin?
I'm sure that I go up.
The origin, the center point here, is a minim because this thing I believe, and we better see why, it's, the graph is like a bowl, the graph is like a bowl shape, it's -- here's the minimum.
And because we've got a pure quadratic, we know it sits at the origin, we know it's tangent plane, the first derivatives are zero, so, we know, first derivatives, First derivatives are all zero, but that's not enough for a minimum.
It's first derivatives were zero here.
So, the partial derivatives, the first derivatives, are zero.
Again, because first derivatives are gonna have an x or an a y, or a y in them, they'll be zero at the origin.
It's the second derivatives that control everything.
It's the second derivatives that this matrix is telling us, and somehow -- here's my point.
You remember in Calculus, how did you decide on a minimum?
First requirement was, that the derivative had to be zero.
But then you didn't know if you had a minimum or a maximum.
To know that you had a minimum, you had to look at the second derivative.
The second derivative had to be positive, the slope had to be increasing as you went through the minimum point.
The curvature had to go upwards, and that's what we're doing now in two dimensions, and in n dimensions.
So we're doing what we did in Calculus.
Second derivative positive, m- will now become that the matrix of second derivatives is positive definite.
Can I just -- like a translation of -- this is how minimum are coming in, ithe beginning of Calculus -- we had a minimum was associated with second derivative, being positive.
And first derivative zero, of course.
Derivative, first derivative, but it was the second derivative that told us we had a minimum.
And now, in 18.06, in linear algebra, we'll have a minim for our function now, our function will have, for your function be a function not of just x but several variables, the way functions really are in real life, the minimum will be when the matrix of second derivatives, the matrix here was one by one, there was just one second derivative, now we've got lots.
Is positive definite.
So positive for a number translates into positive definite for a matrix.
And it this brings everything you check pivots, you check determinants, you check all your values, or you check this minimum stuff.
OK. Let me come back to this graph.
That graph goes upwards.
And I'll have to see why.
How do I know that this, that this function is always positive?
Can you look at that and tell that it's always positive?
Maybe two by two, you could feel pretty sure, but what's the good way to show that this thing is always If we can express it, as, in terms of squares, positive?
because that's what we know for any x and y, whatever, if we're squaring something we certainly are not negative.
So I believe that this expression, this function, could be written as a sum of squares.
Can you tell me -- see, because all the problems, the headaches are coming from this xy term.
If we can get expressions -- if we can get that inside a square, so actually, what we're doing is something called, that you've seen called completing the square.
Let me start the square and you complete it.
OK, I think we have two of x plus, now I don't remember how many y-s we need, but you'll figure it out, squared.
How many y-s should I put in here, to make -- what do I want to do, the two x squared-s will be correct, right?
What I want to do is put in the right number of y-s to get the twelve xy correct.
And what is that number of y-s?
Let's see, I've got two times, and so I really want six xy-s to come out of here, I think maybe if I put three there, does that look right to you?
I have two- this is, we can mentally, multiply out, that's X squared, that's right, that's six X Y, times the two gives from, right, and how many Y squareds have I now got?
How many Y squareds have I now got from this term?
Eighteen.
Eighteen was the key number, remember?
Now if I want to make it twenty, then I've got two left.
Two y squared-s. That's completing the square, and it's, now I can see that that function is positive, because it's all squares.
I've got two squares, added together, I couldn't go negative.
What if I went back to that seven?
If instead of twenty that number was a seven, then what would happen?
This would still be correct, I'd still have this square, to get the two x squared and the twelve xy, and I'd have eighteen y squared and then what would I do here?
I'd have to remove eleven y squared-s, right, if I only had a seven here, then instead of -- when I had a twenty I had two more to put in, when I had an eighteen, which was the marginal case, I had no more to put in.
When I had a seven, which was the case below zero, the indefinite case, I had minus eleven.
Now, so, you can see now, that this thing is a bowl.
OK.
It's going upwards, if I cut it at a plane, z equal to one, say, I would get, I would get a curve, what would be the equation for that curve?
If I cut it at height one, the equation would be this thing equal to one.
And that curve would be an ellipse.
So actually, already, I've blocked into the lecture, the different pieces that we're aiming for.
We're aiming for the tests, which this passed; we're aiming for the connection to a minimum, which this -- which we see in the graph, and if we chop it up, if we set this thing equal to one, if I set that thing equal to one, that -- what that gives me is, the cross-section.
It gives me this, this curve, and its equation is this thing equals one, and that's an ellipse.
Whereas if I cut through a saddle point, I get a hyperbola.
But this minimum stuff is really what I'm most interested OK. in.
OK. By -- I just have to ask, do you recognize, I mean, these numbers here, the two that appeared outside, the three that appeared inside, the two that appeared there -- actually, those numbers come from elimination.
Completing the square is our good old method of Gaussian elimination, in expressed in terms of these squares.
The -- let me show you what I mean.
I just think those numbers are no accident, If I take my matrix two, six, six, and twenty, and I do elimination, then the pivot is two and I take three, what's the multiplier?
How much of row one do I take away from row two?
Three.
So what you're seeing in this, completing the square, is the pivots outside and the multiplier inside.
Just do that again?
The pivot is two, three -- three of those away from that gives me two, six, zero, and what's the second pivot?
Three of this away from this, three sixes'll be eighteen, and the second pivot will also be a two.
So that's the U, this is the A, and of course the L was one, zero, one, and the multiplier was three.
So, completing the square is elimination.
Why I happy to see, happy to see that coming together?
Because I know about elimination for m by m matrices.
I just started talking about completing the square, here, for two by twos.
But now I see what's going on.
Completing the square really amounts to splitting this thing into a sum of squares, so what's the critical thing -- I have a lot of squares, and inside those squares are multipliers but they're squares, and the question is, what's outside these squares?
When I complete the square, what are the numbers that go outside?
They're the pivots.
They're the pivots, and that's why positive pivots give me sum of squares.
Positive pivots, those pivots are the numbers that go outside the squares, so positive pivots, sum of squares, everything positive, graph goes up, a minimum at the origin, it's all connected together; all connected together.
And in the two by two case, you can see those connections, but linear algebra now can go up to three by three, m by m. Let's do that next.
Can I just, before I leave two by two, I've written this expression "matrix of second derivatives."
What's the matrix of second derivatives?
That's one second derivative now, but if I'm in two dimensions, I have a two by two matrix, it's the second x derivative, the second x derivative goes there -- shall I write it -- fxx, if you like, fxx, that means the second derivative of f in the x direction.
fyy, second derivative in the y direction.
Those are the pure derivatives, second derivatives.
They have to be positive.
For a minimum.
This number has to be positive for a minimum.
That number has to be positive for a minimum.
But, that's not enough.
Those numbers have to somehow be big enough to overcome this cross-derivative, Why is the matrix symmetric?
Because the second derivative f with respect to x and y is equal to -- I can, that's the beautiful fact about second derivatives, is I can do those in either order and I get the same thing.
So this is the same as that, and so, that's the matrix of second derivatives.
And the test is, it has to be positive definite.
You might remember, from, tucked in somewhere near the end of eighteen o' two or at least in the book, was the condition for a minimum, For a function of two variables.
Let's -- when do you have a minimum?
For a function of two variables, believe me, that's what Calculus is for.
The condition is first derivatives have to be zero.
And the matrix of second derivatives has to be positive definite.
So you maybe remember there was an fxx times an fyy that had to be bigger than an an fxy squared, that's just our determinant, two by two.
But now, we now know the answer for three by three, m by m, because we can do elimination by m by m matrices, we can connect eigenvalues of m by m matrices, we can do sum of squares, sum of m squares instead of only two squares; and so let's take a, let me go over here to do a three by three example.
So, three by three example.
OK. Oh, let me -- shall I use my favorite matrix?
You've seen this matrix before.
Yes, let's use the good matrix, four by one, oops, open.
Is that matrix positive definite?
What's -- so I'm going to ask questions about this matrix, is it positive definite, first of all?
What's the function associated with that matrix, what's the x transpose Ax?
Is -- do we have a minimum for that function, at zero?
And then even what's the geometry?
OK. First of all, is the matrix positive definite, now I've given you the numbers there so you can take the determinants, maybe that's the quickest, is that what you would do mentally, if I give you all a matrix on a quiz and say is it positive definite or not?
I would take that determinant and I'd give the answer two.
I would take that determinant and I would give the answer for that two by two determinant, I'd give the answer three, and anybody remember the answer for the three by three determinant?
It was four, you remember for these special matrices, when we do determinants, they went up two, three, four, five, six, they just went up linearly.
So that matrix has -- the determinants are two, three, and four.
Pivots.
What are the pivots for that matrix?
I'll tell you, they're -- the first pivot is two, the next pivot is three over two, the next pivot is four over three.
Because, the product of the pivots has to give me those determinants.
The product of these two pivots gives me that determinant; the product of all the pivots gives me that determinant.
What are the eigenvalues?
Oh, I don't know.
The eigenvalues I've got, what do I have a cubic equation -- a degree three equation?
There are three eigenvalues to find.
If I believe what I've said today, what do I know about these eigenvalues, even though I don't know the exact numbers.
I -- I remember the numbers.
Because these matrices are so important that people figure them.
But -- what do you believe to be true about these three eigenvalues -- you believe that they are all positive.
They're all positive.
I think that they are two minus square root of two, two, and two plus the square root of two.
I think.
Let me just -- I can't write those numbers down without checking the simple checks, what the first simple check is the trace, so if I add those numbers I get six and if I add those numbers I get six.
The other simple test is the determinant, if I -- can you do this, can you multiply those numbers together?
I guess we can multiply by two out there.
What's two minus square root of two times two plus square root of two, that'll be four minus two, that'll be two, yeah, two times two, that's got the determinant, right, so it's got, it's got a chance of being correct and I think it is.
Now, what's the x transpose Ax?
I better give myself enough room for that.
x transpose Ax for this guy.
It's two x1 squareds, and two x2 squareds, and two x3 squareds.
Those come from the diagonal, those were easy.
Now off the diagonal there's a minus and a minus, they come together there'll be a minus two minus two whats?
Are coming from this one two and two one position, is the x1 x2.
I'm doing mentally a multiplication of this matrix times a row vector on the left times a column vector on the right, and I know that these numbers show up in the answer.
The diagonal is the perfect square, this off diagonal is a minus two x1 x2, and there are no x1 x3-s, and there're minus two x2 x3-s. And I believe that that expression is always positive.
I believe that that curve, that graph, really, of that function, this is my function f, and I'm in more dimensions now than I can draw, it -- but the graph of that function goes upwards.
It's a bowl.
Or maybe the right word is -- just forgot, what's a long word for bowl?
Hm, maybe paraboloid, I think, paraboloid comes in.
I'll edit the tape and get that word in.
Bowl, let's say, is, that, so that, and if I can -- I could complete the squares, I could write that as the sum of three squares, and those three squares would get multiplied by the three pivots.
And the pivots are all positive.
So I would have positive pivots times squares, the net result would be a positive function and a bowl which goes upwards.
And then, finally, if I cut -- if I slice through this bowl, if I -- now I'm asking you to stretch your visualization here, because I'm in four dimensions, I've got x1 x2 x3 in the base, and this function is z, or f, or something.
And its graph is going up.
But I'm in four dimensions, because I've got three in the base and then the upward direction, but now if I cut through this four-dimensional picture, at level one, so, suppose I cut through this thing at height one.
So I take all the points that are at height one.
That gives me -- it gave me an ellipse over there, in that two by two case, in this case, this will be the equation of an ellipsoid, a football in other words.
Well, not quite a football.
A lopsided football.
What would be, can I try to describe to you what the ellipsoid will look like, this ellipsoid, I'm sorry that the, that I've ended the matrix right -- at the point, let's -- let me be sure you've seen the equation.
Two x1 squared, two x2 squared, two x3 squared, minus two of the cross parts, equal what?
That is the equation of a football, so what do I mean by a football or an ellipsoid?
I mean that, well, I'll draw a few.
It's like that, it's got a center, and it's got it's got three principal directions.
This ellipsoid.
So -- you see what I'm saying, if we have a sphere then all directions would be the same.
If we had a true football, or it's closer to a rugby ball, really, because it's more curved than a football, it would have one long direction and the other two would be equal.
That would be like having a matrix that had one eigenvalue repeated.
And then one other different.
So this sphere comes from, like, the identity matrix, all eigenvalues the same.
Our rugby ball comes from a case where -- three, the three, two of the three eigenvalues are the same.
But we know how the case where -- the typical case, where the three eigenvalues were all different.
So this will have -- How do I say it, if I look at this ellipsoid correctly, it'll have a major axis, it'll have a middle axis, and it'll have a minor axis.
And those three axes will be in the direction of the eigenvectors.
And the lengths of those axes will be determined by the eigenvalues.
I can get -- turn this all into linear algebra, because we have -- the right thing we know about eigenvectors and eigenvalues, for that matrix is what?
Of -- let me just tell you that, repeat the main linear algebra point.
How could we turn what I said into algebra; we would write this A as Q, the eigenvector matrix, times lambda, the eigenvalue matrix times Q transposed.
The principal axis theorem, we'll call it, now.
The eigenvectors tell us the directions of the principal axes.
The eigenvalues tell us the lengths of those axes, actually the lengths, or the half-lengths, or one over the eigenvalues, it turns out.
And that is the matrix factorization which is the most important matrix factorization in our eigenvalue material so far.
That's diagonalization for a symmetric matrix, so instead of the inverse I can write the transposed.
OK.
I've -- so what I've tried today is to tell you the -- what's going on with positive definite matrices.
Ah, you see all how all these pieces are there and linear algebra connects them.
OK. See you on Friday.
Okay.
This lecture is mostly about the idea of similar matrixes.
I'm going to tell you what that word similar means and in what way two matrixes are called similar.
But before I do that, I have a little more to say about positive definite matrixes.
You can tell this is a subject I think is really important and I told you what positive definite meant -- it means that this -- this expression, this quadratic form, x transpose I x is always positive.
But the direct way to test it was with eigenvalues or pivots or determinants.
So I -- we know what it means, we know how to test it, but I didn't really say where positive definite matrixes come from.
And so one thing I want to say is that they come from least squares in -- and all sorts of physical problems start with a rectangular matrix -- well, you remember in least squares the crucial combination was A transpose A.
So I want to show that that's a positive definite matrix.
Can -- so I -- I'm going to speak a little more about positive definite matrixes, just recapping -- so let me ask a question.
It may be on the homework.
Suppose a matrix A is positive definite.
I mean by that it's all -- I'm assuming it's symmetric.
That's always built into the definition.
So we have a symmetric positive definite matrix.
What about its inverse?
Is the inverse of a symmetric positive definite matrix also symmetric positive definite?
So you quickly think, okay, what do I know about the pivots of the inverse matrix?
Not much.
What do I know about the eigenvalues of the inverse matrix?
Everything, right?
The eigenvalues of the inverse are one over the eigenvalues of the matrix.
So if my matrix starts out positive definite, then right away I know that its inverse is positive definite, because those positive eigenvalues -- then one over the eigenvalue is also positive.
What if I know that A -- a matrix A and a matrix B are both positive definite?
But let me ask you this.
Suppose if A and B are positive definite, what about -- what about A plus B?
In some way, you hope that that would be true.
It's -- positive definite for a matrix is kind of like positive for a real number.
But we don't know the eigenvalues of A plus B.
We don't know the pivots of A plus B.
So we just, like, have to go down this list of, all right, which approach to positive definite can we get a handle on?
And this is a good one.
This is a good one.
Can we -- how would we decide that -- if A was like this and if B was like this, then we would look at x transpose A plus B x. I'm sure this is in the homework.
Now -- so we have x transpose A x bigger than zero, x transpose B x positive for all -- for all x, so now I ask you about this guy.
And of course, you just add that and that and we get what we want.
If A and B are positive definites, so is A plus B.
So that's what I've shown.
So is A plus B.
Just -- be sort of ready for all the approaches through eigenvalues and through this expression.
And now, finally, one more thought about positive definite is this combination that came up in least squares.
Can I do that?
So now -- now suppose A is rectangular, m by n. I -- so I'm sorry that I've used the same letter A for the positive definite matrixes in the eigenvalue chapter that I used way back in earlier chapters when the matrix was rectangular.
Now, that matrix -- a rectangular matrix, no way its positive definite.
It's not symmetric.
It's not even square in general.
But you remember that the key for these rectangular ones was A transpose A.
That's square.
That's symmetric.
Those are things we knew -- we knew back when we met this thing in the least square stuff, in the projection stuff.
But now we know something more -- we can ask a more important question, a deeper question -- is it positive definite?
And we sort of hope so.
Like, we -- we might -- in analogy with numbers, this is like -- sort of like the square of a number, and that's positive.
So now I want to ask the matrix question.
Is A transpose A positive definite?
Okay, now it's -- so again, it's a rectangular A that I'm starting with, but it's the combination A transpose A that's the square, symmetric and hopefully positive definite matrix.
So how -- how do I see that it is positive definite, or at least positive semi-definite?
You'll see that.
Well, I don't know the eigenvalues of this product.
I don't want to work with the pivots.
The right thing -- the right quantity to look at is this, x transpose Ax -- A -- x transpose times my matrix times x. I'd like to see that this thing -- that that expression is always positive.
I'm not doing it with numbers, I'm doing it with symbols.
Do you see -- how do I see that that expression comes out positive?
I'm taking a rectangular matrix A and an A transpose -- that gives me something square symmetric, but now I want to see that if I multiply -- that if I do this -- I form this quadratic expression that I get this positive thing that goes upwards when I graph it.
How do I see that that's positive, or absolutely it isn't negative anyway?
We'll have to, like, spend a minute on the question could it be zero, but it can't be negative.
Why can this never be negative?
The argument is -- like the one key idea in so many steps in linear algebra -- put those parentheses in a good way.
Put the parentheses around Ax and what's the first part?
What's this x transpose A transpose?
That is Ax transpose.
So what do we have?
We have the length squared of Ax.
We have -- that's the column vector Ax that's the row vector Ax, its length squared, certainly greater than or possibly equal to zero.
So we have to deal with this little possibility.
Could it be equal?
Well, when could the length squared be zero?
Only if the vector is zero, right?
That's the only vector that has length squared zero.
So we have -- we would like to -- I would like to get that possibility out of there.
So I want to have Ax never -- never be zero, except of course for the zero vector.
How do I assure that Ax is never zero?
The -- in other words, how do I show that there's no null space of A?
The rank should be -- so now remember -- what's the rank when there's no null space?
By no null space, you know what I mean.
Only the zero vector in the null space.
So if I have a -- if I have an 11 by 5 matrix -- so it's got 11 rows, 5 columns, when is there no null space?
So the columns should be independent -- what's the rank?
n 5 -- rank n. Independent columns, when -- so if I -- then I conclude yes, positive definite.
And this was the assumption -- then A transpose A is invertible -- the least squares equations all work fine.
And more than that -- the matrix is even positive definite.
And I just to say one comment about numerical things, with a positive definite matrix, you never have to do row exchanges.
You never run into unsuitably small numbers or zeroes in the pivot position.
They're the right -- they're the great matrixes to compute with, and they're the great matrixes to study.
So that's -- I wanted to take this first ten minutes of grab the first ten minutes away from similar matrixes and continue a -- this much more with positive definite.
I'm really at this point, now, coming close to the end of the heart of linear algebra.
The positive definiteness brought everything together.
Similar matrixes, which is coming the rest of this hour is a key topic, and please come on Monday.
Monday is about what's called the SVD, singular values.
It's the -- has become a central fact in -- a central part of linear algebra.
I mean, you can come after Monday also, but -- Monday is, -- that singular value thing has made it into this course.
Ten years ago, five years ago it wasn't in the course, now it has to be.
Okay.
So can I begin today's lecture proper with this idea of similar matrixes.
This is what similar matrixes mean.
So here -- let's start again.
I'll write it again.
So A and B are similar.
A and B are -- now I'm -- these matrixes -- I'm no longer talking about symmetric matrixes, in -- at least no longer expecting symmetric matrixes.
I'm talking about two square matrixes n by n. A and B, they're n by n matrixes.
And I'm introducing this word similar.
So I'm going to say what does it mean?
It means that they're connected in the way -- well, in the way I've written here, so let me rewrite it.
That means that for some matrix M, which has to be invertible, because you'll see that -- this one matrix is -- take the other matrix, multiply on the right by M and on the left by M inverse.
So the question is, why that combination?
But part of the answer you know already.
You remember -- we've done this -- we've taken a matrix A -- so let's do an example of similar.
Suppose A -- the matrix A -- suppose it has a full set of eigenvectors.
They go in this eigenvector matrix S. Then what was the main point of the whole -- the main calculation of the whole chapter was -- is -- use that eigenvector matrix S and its inverse comes over there to produce the nicest possible matrix lambda.
Nicest possible because it's diagonal.
So in our new language, this is saying A is similar to lambda.
A is similar to lambda, because there is a matrix, and this particular -- there is an M and this particular M is this important guy, this eigenvector matrix.
But if I take a different matrix M and I look at M inverse A M, the result won't come out diagonal, but it will come out a matrix B that's similar to A.
Do you see that I'm -- what I'm doing is, like -- I'm putting these matrixes into families.
All the matrixes in one -- in the family are similar to each other.
They're all -- each one in this family is connected to each other one by some matrix M and the -- like the outstanding member of the family is the diagonal guy.
I mean, that's the simplest, neatest matrix in this family of all the matrixes that are similar to A, the best one is lambda.
But there are lots of others, because I can take different -- instead of S, I can take any old matrix M, any old invertible matrix and -- and do it.
I'd better do an example.
Okay.
Suppose I take A as the matrix two one one two.
Okay.
Do you know the eigenvalue matrix for that?
The eigenvalues of that matrix are -- well, three and one.
So that -- and the eigenvectors would be easy to find.
So this matrix is similar to this one.
But my point is -- but also, I can also take my matrix, two one one two, I could multiply it by -- let's see, what -- I'm just going to cook up a matrix M here.
I'm -- I'll -- let me just invent -- one four one zero.
And over here I'll put M inverse, and because I happened to make that triangular, I know that its inverse is that, right?
So there's M inverse A M, that's going to produce some matrix -- oh, well, I've got to do the multiplication, so hang on a second, let -- I'll just copy that one minus four zero one and multiply these guys so I'm getting two nine one and six, I think.
Can you check it as I go, because you -- see I'm just -- so that's two minus four, I'm getting a minus two nine minus 24 is a minus 15, my God, how did I get this?
And that's probably one and six.
So there's my matrix B.
And there's my matrix lambda, there's my matrix A and my point is these are all similar matrixes.
They all have something in common, besides being just two by two.
They have something in common.
And that's -- and what is it?
What's the point about two matrixes that are built out of -- the B is built out of M inverse A M. What is it that A and B have in common?
That's the main -- now I'm telling you the main fact about similar matrixes.
They have the same eigenvalues.
This is -- this chapter is about eigenvalues, and that's why we're interested in this family of matrixes that have the same eigenvalues.
What are the eigenvalues in this example?
Lambda.
The eigenvalues of that I could compute.
The eigenvalues of that I can compute really fast.
So the eigenvalues are three and one -- for this for sure.
Now did we -- do you see why the eigenvalues are three and one for that one?
If I tell you the eigenvalues are three and one, you prick -- quickly process the trace, which is -- and four -- agrees with four and you process the determinant, three times one -- the determinant is three and you say yes, it's right.
Now I'm hoping that the eigenvalues of this thing are three and one.
May I process the trace and the determinant for that one?
What's the trace here?
The trace of this matrix is four minus two and six, and that's what it should be.
What's the determinant minus twelve plus fifteen is three.
The determinant is three.
The eigenvalues of that matrix are also three and one.
And you see I created this matrix just like -- I just took any M, like, one that popped into my head and computed M inverse A M, got that matrix, it didn't look anything special but it's -- like A itself, it has those eigenvalues three and one.
So that's the main fact and let me write it down.
Similar matrixes have the same eigenvalues.
So I'll just put that as an important point.
And think why.
Why is that?
So that's what that family of matrixes is.
The matrixes that are similar to this A here are all the matrixes with eigenvalues three and one.
Every matrix with eigenvalues three and one, there's some M that connects this guy to the one you think of.
And then of course, the most special guy in the whole family is the diagonal one with eigenvalues three and one sitting there on the diagonal.
But also, I could find -- I mean, tell me just a couple more members of the family.
Another -- tell me another matrix that has eigenvalues three and one.
Well, let's see, I -- oh, I'll just make it triangular.
That's in the family.
There is some M that -- that connects to this one.
And -- and also this.
There's some matrix M -- so that M inverse A M comes out to be that.
There's a whole family here.
And they all share the same eigenvalues.
So why is that?
Okay.
I'm going to start -- the only possibility is to start with Ax equal lambda x.
Okay, so suppose A has the eigenvalue lambda.
Now I want to get B into the picture here somehow.
You remember B is M inverse A M. Let's just remember that over here.
B is M inverse A M. And I want to see its eigenvalues.
How I going to get M inverse A M into this equation?
Let me just sort of do it.
I'll put an M times an M inverse in there, right?
That was -- I haven't changed the left-hand side, so I better not change the right-hand side.
So everybody's okay so far, I just put in there -- see, I want to get a -- so now I'll multiply on the left by M inverse -- I have to do the same to this side and that number lambda's just a number, so it factors out in the front.
So what I have here is this was safe.
I did the same thing to both sides.
And now I've got B.
There's B.
That's B times this vector M inverse x is equal to lambda times this vector M inverse x.
So what have I learned?
I've learned that B times some vector is lambda times that vector.
I've learned that lambda is an eigenvalue of B also.
So this is -- if -- so this is -- if lambda's an eigenvalue of A, then I can write it this way and I discover that lambda's an eigenvalue of B.
That's the end of the proof.
The eigenvector didn't stay the same.
Of course I don't expect the eigenvectors to stay the same.
If all the eigenvalues are the same and all the eigenvectors are the same, then probably the matrix is the same.
Here the eigenvector changes, so the eigenvector -- so the point is then the eigenvector of B -- of B is M inverse times the eigenvector of A.
Okay.
That's all that this says here.
The eigenvector of A was X, and so the M inverse -- similar matrixes, then have the same eigenvalues and their eigenvectors are just moved around.
Of course, that's what we -- that's what happened way back -- and the most important similar matrixes are to diagonalize.
So what was the point when we diagonalized?
The eigenvalues stayed the same, of course.
Three and one.
What about the eigenvectors?
The eigenvectors were whatever they were for the matrix A, but then what were the eigenvectors for the diagonal matrix?
They're just -- what are the eigenvectors of a diagonal matrix?
They're just one zero and zero one.
So this step made the eigenvectors nice, didn't change the eigenvalues, and every time we don't change the eigenvalues.
Same eigenvalues.
Okay.
Now -- so I've got all these matrixes in -- I've got this family of matrixes with eigenvalues three and one.
Fine.
That's a nice family.
It's nice because those two eigenvalues are different.
I now have to -- to get into that -- the -- into the less happy possibility that the two eigenvalues could be the same.
And then it's a little trickier, because you remember when two eigenvalues are the same, what's the bad possibility?
That there might not be enough -- a full set of eigenvectors and we might not be able to diagonalize.
So I need to discuss the bad case.
So the bad -- can I just say bad?
If lambda one equals lambda two, then the matrix might not be diagonalizable.
Suppose lambda one equals lambda two equals four, say.
Now if I look at the family of matrixes with eigenvalues four and four, well, one possibility occurs to me.
One family with eigenvalues four and four has this matrix in it, four times the identity.
Then another -- but now I want to ask also about the matrix four four one zero.
And my point -- here's the whole point of this -- of this bad stuff, is that this guy is not in the same family with that one.
The family of a -- of matrixes that have eigenvalues four and four is two families.
There's this total loner here who's in a family off -- right?
Just by himself.
And all the others are in with this guy.
So the big family includes this one.
And it includes a whole lot of other matrixes, all -- in fact, in this two by two case, it -- you see where -- what do I mean -- so what I using, this word family -- in a family, I mean they're similar.
So my point is that the only matrix that's similar to this is itself.
The only matrix that's similar to four times the identity is four times the identity.
It's off by itself.
Why is that?
The -- if this is my matrix, four times the identity, and I take it, I multiply on the right by any matrix M, I multiply on the left by M inverse, what do I get?
This is any M, but what's the result?
Well, factoring out a four, that's just the identity matrix in there.
So then the M inverse cancels the M, so I've just got this matrix back again.
So whatever the M is, I'm not getting any more members of the family.
So this is one small family, because it only has one person.
One matrix, excuse me.
I think of these matrixes as people by this point, in eighteen oh six.
Okay, the other family includes all the rest -- all other matrixes that have eigenvalues four and four.
This is somehow the best one in that family.
See, I can't make it diagonal.
If I -- if it's diagonal, it's this one.
It's in its own, by itself.
So I have to think, okay, what's the nearest I can get to diagonal?
But it will not be diagonalizable.
That -- do you know that that matrix is not diagonalizable?
Of course, because if it was diagonalizable, it would be similar to that, which it isn't.
The eigenvalues of this are four and four, but what's the catch with that matrix?
It's only got one eigenvector.
That's a non-diagonalizable matrix.
Only one eigenvector.
And somehow, if I made that one into a ten or to a million, I could find an M, it's in the family, it's similar.
But the best -- so the best guy in this family is this one.
And this is called the Jordan -- so this guy Jordan picked out -- so he, like, studied, these families of matrixes, and each family, he picked out the nicest, most diagonal one.
But not completely diagonal, because there's nobody -- there isn't a diagonal matrix in this family, so there's a one up there in the Jordan form.
Okay.
I think we've got to see some more matrixes in that family.
So, all right, let me -- let's just think of some other matrixes whose eigenvalues are four and four but they're not four times the identity.
So -- and I believe that -- that this -- that all the examples we pick up will be similar to each other and -- do you see why -- in this topic of similar matrixes, the climax is the Jordan form.
So it says that every matrix -- I'll write down what the Jordan form -- what Jordan discovered.
He found the best looking matrix in each family.
And that's -- then we've got -- then we've covered all matrixes including the non-diagonalizable one.
That -- that's the point, that in some way, Jordan completed the diagonalization by coming as near as he could, which is his Jordan form.
And therefore, if you want to cover all matrixes, you've got to get him in the picture.
It used to be -- when I took eighteen oh six, that was the climax of the course, this Jordan form stuff.
I think it's not the climax of linear algebra anymore, because -- it's not easy to find this Jordan form for a general matrix, because it depends on these eigenvalues being exactly the same.
You'd have to know exactly the eigenvalues and it -- and you'd have to know exactly the rank and the slightest change in numbers will change those eigenvalues, change the rank and therefore the whole thing is numerically not an -- a good thing.
But for algebra, it's the right thing to understand this family.
So just tell me another matrix -- a few more matrixes -- so more members of the family.
Let me put down again what the best one is.
Okay.
All right.
Some more matrixes.
Let's see, what I looking for?
I'm looking for matrixes whose trace is what?
So if I'm looking for more matrixes in the family, they'll all have the same eigenvalues, four and four.
So their trace will be eight.
So why don't I just take, like, five and three -- I've got the trace right, now the determinant should be what?
Sixteen.
So I just fix this up -- shall I put maybe a one and a minus one there?
Okay.
There's a matrix with eigenvalues four and four, because the trace is eight and the determinant is sixteen.
And I don't think it's diagonalizable.
Do you know why it's not diagonalizable?
Because if it was diagonalizable, the diagonal form would have to be this.
But I can't get to that form, because whatever I do with any M inverse and M I stay with that form.
I could never get -- connect those.
So I can put down more members -- here -- here's another easy one.
I could put the four and the four and a seventeen down there.
All these matrixes are similar.
If I'm -- I could find an M that would show that that one is similar to that one.
And in -- you can see the general picture is I can take any a and any 8-a here and any -- oh, I don't know, whatever you put it'd be -- anyway, you can see.
I can fill this in, fill this in to make the trace equal eight, the determinant equal 16, I get all that family of matrixes and they're all similar.
So we see what eigenvalues do.
They're all similar and they all have only one eigenvector.
So I -- if I'm -- if you were going to -- allow me to add to this picture, they have the same lambdas and they also have the same number of independent eigenvectors.
Because if I get an eigenvector for x I get one for -- for A, I get one for B also.
So -- and same number of eigenvectors.
But even more than that -- even more than that -- I mean, it's not enough just to count eigenvectors.
Yes, let me give you an example why it's not even enough to count eigenvectors.
So another example.
So here are some matrixes -- oh, let me make them four by four -- okay, here -- here's a matrix.
I mean, like if you want nightmares, think about matrixes like these.
Uh, so a one off the diagonal -- say a one there, how many -- what are the eigenvalues of that matrix?
Oh, I mean -- okay.
What are the eigenvalues of that matrix?
Please.
Four 0s, right?
So we're really getting bad matrixes now.
So I mean, this is, like -- Jordan was a good guy, but he had to think about matrixes that all -- that had, like -- an eigenvalue repeated four times.
How many eigenvectors does that matrix have?
Well, I'm -- eigenvectors will be -- since the eigenvalue is zero, eigenvectors will be in the null space, right?
I'm -- eigenvectors have got to be A x equal zero x.
So what's the dimension of the null space?
Two.
Somebody said two.
And that's right.
How -- why?
Because you ask what's the rank of that matrix, the rank is obviously two.
The number of independent rows is two, the number of independent columns is two, the rank is two so the null -- the dimension of the null space is four minus two, so it's got two eigenvectors.
Two eigenvectors.
Two independent eigenvectors.
All right.
The dimension of the null space is two.
Now, suppose I change this zero to a seven.
The eigenvalues are all still zero, how -- what about -- how many eigenvectors?
What's the dimension of the -- what's the rank of this matrix now?
Still two, right?
So it's okay.
And actually, this would be similar to the one that had a zero in there.
But it's not as beautiful, Jordan picked this one.
He picked -- he put ones -- we have a one on the -- above the diagonal for every missing eigenvector, and here we're missing two because we've got two, so we've got two eigenvectors and two are missing, because it's a four by four matrix.
Okay, now -- but I was going to give you this second example.
0 1 0 0, let me just move the one.
Oop, not there.
Off the diagonal and zero zero zero zero zero.
Okay.
So now tell me about this matrix.
Its eigenvalues are four zeroes again.
Its rank is two again.
So it has two eigenvectors and two missing.
But the darn thing is not similar to that one.
A -- a count of eigenvectors looks like these could be similar, but they're not.
Jordan -- see, this is like -- a little three by three block and a little one by one block.
And this one is like a two by two block and a two by two block, and those blocks are called Jordan blocks.
So let me say what is a Jordan block?
J block number I has -- so a Jordan block has a repeated eigenvalue, lambda I, lambda I on the diagonal.
Zeroes below and ones above.
So there's a block with this guy repeated, but it only has one eigenvector.
So a Jordan block has one eigenvector only.
This one has one eigenvector, this block has one eigenvector and we get two.
This block has one eigenvector and that block has one eigenvector and we get two.
So -- but the blocks are different sizes.
And that -- it turns out Jordan worked out -- then this is not similar, not similar to this one.
So the -- so I'm, like, giving you the whole story -- well, not the whole story, but the main themes of the story -- is here's Jordan's theorem.
Every square matrix A is similar to A Jordan matrix J.
And what's a Jordan matrix J?
It's a matrix with these blocks, block -- Jordan block number one, Jordan block number two and so on.
And let's say Jordan block number d. And those Jordan blocks look like that, so the eigenvalues are sitting on the diagonal, but we've got some of these ones above the diagonal.
We've got the number of -- so the number of blocks -- the number of blocks is the number of eigenvectors, because we get one eigenvector per block.
So what I'm -- so if I summarize Jordan's idea -- start with any A.
If its eigenvalues are distinct, then what's it similar to?
This is the good case.
if I start with a matrix A and it has different eigenvalues -- it's n eigenvalues, none of them are repeated, then that's a diagonal -- diagonalizable matrix -- the Jordan blocks is -- has -- the Jordan matrix is diagonal.
It's lambda.
So the good case -- the good case, J is lambda.
All -- there are -- d=n.
There are n eigenvectors, n blocks, diagonal, everything great.
But Jordan covered all cases by including these cases of repeated eigenvalues and missing eigenvectors.
Okay.
That's a description of Jordan.
That -- that's -- I haven't told you how to compute this thing, and it isn't easy.
Whereas the good case is the -- the good case is what 18.06 is about.
The -- this case is what 18.06 was about 20 years ago.
So you can see you probably won't have on the final exam the computation of a Jordan matrix for some horrible thing with four repeated eigenvalues.
I'm not that crazy about the Jordan form.
But I'm very positive about positive definite matrixes and about the idea that's coming Monday, the singular value decomposition.
So I'll see you on Monday, and have a great weekend.
Bye.
Okay.
This is the lecture on the singular value decomposition.
But everybody calls it the SVD.
So this is the final and best factorization of a matrix.
Let me tell you what's coming.
The factors will be, orthogonal matrix, diagonal matrix, orthogonal matrix.
So it's things that we've seen before, these special good matrices, orthogonal diagonal.
The new point is that we need two orthogonal matrices.
A can be any matrix whatsoever.
Any matrix whatsoever has this singular value decomposition, so a diagonal one in the middle, but I need two different -- probably different orthogonal matrices to be able to do this.
Okay.
And this factorization has jumped into importance and is properly, I think, maybe the bringing together of everything in this course.
One thing we'll bring together is the very good family of matrices that we just studied, symmetric, positive, definite.
Do you remember the stories with those guys?
Because they were symmetric, their eigenvectors were orthogonal, so I could produce an orthogonal matrix -- this is my usual one.
My usual one is the eigenvectors and eigenvalues In the symmetric case, the eigenvectors are orthogonal, so I've got the good -- my ordinary s has become an especially good Q.
And positive definite, my ordinary lambda has become a positive lambda.
So that's the singular value decomposition in case our matrix is symmetric positive definite -- in that case, I don't need two -- U and a V -- one orthogonal matrix will do for both sides.
So this would be no good in general, because usually the eigenvector matrix isn't orthogonal.
So that's not what I'm after.
I'm looking for orthogonal times diagonal times orthogonal.
And let me show you what that means and where it comes from.
Okay.
What does it mean?
You remember the picture of any linear transformation.
This was, like, the most important figure in And what I looking for now?
A typical vector in the row space -- typical vector, let me call it v1, gets taken over to some vector in the column space, say u1.
So u1 is Av1.
Okay.
Now, another vector gets taken over here somewhere.
What I looking for?
In this SVD, this singular value decomposition, what I'm looking for is an orthogonal basis here that gets knocked over into an orthogonal basis over there.
See that's pretty special, to have an orthogonal basis in the row space that goes over into an orthogonal basis -- so this is like a right angle and this is a right angle -- into an orthogonal basis in the column space.
So that's our goal, is to find -- do you see how things are coming together?
First of all, can I find an orthogonal basis for this row space?
Of course.
No big deal to find an orthogonal basis.
Graham Schmidt tells me how to do it.
Start with any old basis and grind through Graham Schmidt, out comes an orthogonal basis.
But then, if I just take any old orthogonal basis, then when I multiply by A, there's no reason why it should be orthogonal over here.
So I'm looking for this special set up where A takes these basis vectors into orthogonal vectors over there.
Now, you might have noticed that the null space I didn't include.
Why don't I stick that in?
You remember our usual figure had a little null space and a little null space.
And those are no problems.
Those null spaces are going to show up as zeroes on the diagonal of sigma, so that doesn't present any difficulty.
Our difficulty is to find these.
So do you see what this will mean?
This will mean that A times these v-s, v1, v2, up to -- what's the dimension of this row space?
Vr.
Sorry, make that V a little smaller -- up to vr.
So that's -- Av1 is going to be the first column, so here's what I'm achieving.
Oh, I'm not only going to make these orthogonal, but why not make them orthonormal?
Make them unit vectors.
So maybe the unit vector is here, is the u1, and this might be a multiple of it.
So really, what's happening is Av1 is some multiple of u1, right?
These guys will be unit vectors and they'll go over into multiples of unit vectors and the multiple I'm not going to call lambda anymore.
I'm calling it sigma.
So that's the number -- the stretching number.
And similarly, Av2 is sigma two u2.
This is my goal.
And now I want to express that goal in matrix language.
That's the usual step.
Think of what you want and then express it as a matrix multiplication.
So Av1 is sigma one u1 -- actually, here we go.
Let me pull out these -- u1, u2 to ur and then a matrix with the sigmas.
Everything now is going to be in that little part of the blackboard.
Do you see that this equation says what I'm trying to do with my figure.
A times the first basis vector should be sigma one times the other basis -- the other first basis vector.
These are the basis vectors in the row space, these are the basis vectors in the column space and these are the multiplying factors.
So Av2 is sigma two times u2, Avr is sigma r times ur.
And then we've got a whole lot of zeroes and maybe some zeroes at the end, but that's the heart of it.
And now if I express that in -- as matrices, because you knew that was coming -- that's what I have.
So, this is my goal, to find an orthogonal basis in the orthonormal, even -- basis in the row space and an orthonormal basis in the column space so that I've sort of diagonalized the matrix.
The matrix A is, like, getting converted to this diagonal matrix sigma.
And you notice that usually I have to allow myself two different bases.
My little comment about symmetric positive definite was the one case where it's A Q equal Q sigma, where V and U are the same Q.
But mostly, you know, I'm going to take a matrix like -- oh, let me take a matrix like four four minus three three.
Okay.
There's a two by two matrix.
It's invertible, so it has rank two.
So I'm going to look for two vectors, v1 and v2 in the row space, and U -- so I'm going to look for v1, v2 in the row space, which of course is R^2.
And I'm going to look for u1, u2 in the column space, which of course is also R^2, and I'm going to look for numbers sigma one and sigma two so that it all comes out right.
So these guys are orthonormal, these guys are orthonormal and these are the scaling factors.
So I'll do that example as soon as I get the matrix picture straight.
Okay.
Do you see that this expresses what I want?
Can I just say two words about null spaces?
If there's some null space, then we want to stick in a basis for those, for that.
So here comes a basis for the null space, v(r+1) down to vm.
So if we only had an r dimensional row space and the other n-r dimensions were in the null space -- okay, we'll take an orthogonal -- orthonormal basis there.
No problem.
And then we'll just get zeroes.
So, actually, w- those zeroes will come out on the diagonal matrix.
So I'll complete that to an orthonormal basis for the whole space, R^m.
I complete this to an orthonormal basis for the whole space R^n and I complete that with zeroes.
Null spaces are no problem here.
So really the true problem is in a matrix like that, which isn't symmetric, so I can't use its eigenvectors, they're not orthogonal -- but somehow I have to get these orthogonal -- in fact, orthonormal guys that make it work.
I have to find these orthonormal guys, these orthonormal guys and I want Av1 to be sigma one u1 and Av2 to be sigma two u2.
Okay.
That's my goal.
Here's the matrices that are going to get me there.
Now these are orthogonal matrices.
I can put that -- if I multiply on both sides by V inverse, I have A equals U sigma V inverse, and of course you know the other way I can write V inverse.
This is one of those square orthogonal matrices, so it's the same as U sigma V transpose.
Okay.
Here's my problem.
I've got two orthogonal matrices here.
And I don't want to find them both at once.
So I want to cook up some expression that will make the Us disappear.
I would like to make the Us disappear and leave me only with the Vs. And here's how to do it.
It's the same combination that keeps showing up whenever we have a general rectangular matrix, then it's A transpose A, that's the great matrix.
That's the great matrix.
That's the matrix that's symmetric, and in fact positive definite or at least positive semi-definite.
This is the matrix with nice properties, so let's see what will it be?
So if I took the transpose, then, I would have -- A transpose A will be what?
What do I have?
If I transpose that I have V sigma transpose U transpose, that's the A transpose.
Now the A -- and what have I got?
Looks like worse, because it's got six things now together, but it's going to collapse into something good.
What does U transpose U collapse into?
I, the identity.
So that's the key point.
This is the identity and we don't have U anymore.
And sigma transpose times sigma, those are diagonal matrixes, so their product is just going to have sigma squareds on the diagonal.
So do you see what we've got here?
This is V times this easy matrix sigma one squared sigma two squared times V transpose.
This is the A transpose A.
This is -- let me copy down -- A transpose A is that.
Us are out of the picture, now.
I'm only having to choose the Vs, and what are these Vs?
And what are these sigmas?
Do you know what the Vs are?
They're the eigenvectors that -- see, this is a perfect eigenvector, eigenvalue, Q lambda Q transpose for the matrix A transpose A.
A itself is nothing special.
But A transpose A will be special.
It'll be symmetric positive definite, so this will be its eigenvectors and this'll be its eigenvalues.
And the eigenvalues'll be positive because this thing's positive definite.
So this is my method.
This tells me what the Vs are.
And how I going to find the Us?
Well, one way would be to look at A A transpose.
Multiply A by A transpose in the opposite order.
That will stick the Vs in the middle, knock them out, and leave me with the Us.
So here's the overall picture, then.
The Vs are the eigenvectors of A transpose A.
The Us are the eigenvectors of A A transpose, which are different.
And the sigmas are the square roots of these and the positive square roots, so we have positive sigmas.
Let me do it for that example.
This is really what you should know and be able to do for the SVD.
Okay.
Let me take that matrix.
So what's my first step?
Compute A transpose A, because I want its eigenvectors.
Okay.
So I have to compute A transpose A.
So A transpose is four four minus three three, and A is four four minus three three, and I do that multiplication and I get sixteen -- I get twenty five -- I get sixteen minus nine -- is that seven?
And it better come out symmetric.
And -- oh, okay, and then it comes out 25.
Okay.
So, I want its eigenvectors and its eigenvalues.
Its eigenvectors will be the Vs, its eigenvalues will be the squares of the sigmas.
Okay.
What are the eigenvalues and eigenvectors of this guy?
Have you seen that two by two example enough to recognize that the eigenvectors are -- that one one is an eigenvector?
So this here is A transpose A. I'm looking for its eigenvectors.
So its eigenvectors, I think, are one one and one minus one, because if I multiply that matrix by one one, what do I get?
If I multiply that matrix by one one, I get 32 32, which is 32 of one one.
So there's the first eigenvector, and there's the eigenvalue for A transpose A.
So I'm going to take its square root for sigma.
Okay.
What's the eigenvector that goes -- eigenvalue that goes with this one?
If I do that multiplication, what do I get?
I get some multiple of one minus one, and what is that multiple?
Looks like 18.
Okay.
So those are the two eigenvectors, but -- oh, just a moment, I didn't normalize them.
To make everything absolutely right, I ought to normalize these eigenvectors, divide by their length, square root of two.
So all these guys should be true unit vectors and, of course, that normalization didn't change the 32 and the 18.
Okay.
So I'm happy with the Vs.
Here are the Vs.
So now let me put together the pieces here.
Here's my A.
Here's my A.
Let me write down A again.
If life is right, we should get U, which I don't yet know -- U I don't yet know, sigma I do now know.
What's sigma?
So I'm looking for a U sigma V transpose.
U, the diagonal guy and V transpose.
Okay.
Let's just see that come out right.
So what are the sigmas?
They're the square roots of these things.
So square root of 32 and square root of 18.
Zero zero.
Okay.
What are the Vs?
They're these two.
And I have to transpose -- maybe that just leaves me with ones -- with one over square root of two in that row and the other one is one over square root of two minus one over square root of two.
Now finally, I've got to know the Us.
Well, actually, one way to do -- since I now know all the other pieces, I could put those together and figure out what the Us are.
But let me do it the A A transpose way.
Okay.
Find the Us now.
u1 and u2.
And what are they?
I look at A A transpose -- so A is supposed to be U sigma V transpose, and then when I transpose that I get V sigma transpose U transpose.
So I'm just doing it in the opposite order, A times A transpose, and what's the good part here?
That in the middle, V transpose V is going to be the identity.
So this is just U sigma sigma transpose, that's some diagonal matrix with sigma squareds and U transpose.
So what I seeing here?
I'm seeing here, again, a symmetric positive definite or at least semi-definite matrix and I'm seeing its eigenvectors and its eigenvalues.
So if I compute A A transpose, its eigenvectors will be the things that go into U.
Okay, so I need to compute A A transpose.
I guess I'm going to have to go -- can I move that up just a little?
Maybe a little more and do A A transpose.
So what's A?
Four four minus three and three.
And what's A transpose?
Four four minus three and three.
And when I do that multiplication, what do I get?
Sixteen and sixteen, thirty two.
Uh, that one comes out zero.
Oh, so this is a lucky case and that one comes out 18.
So this is an accident that A A transpose happens to come out diagonal, so we know easily its eigenvectors and eigenvalues.
So its eigenvectors -- what's the first eigenvector for this A A transpose matrix?
It's just one zero, and when I do that multiplication, I get 32 times one zero.
And the other eigenvector is just zero one and when I multiply by that I get 18.
So this is A A transpose.
Multiplying that gives me the 32 A A transpose.
Multiplying this guy gives me First of all, I got 32 and 18 again.
Am I surprised?
You know, it's clearly not an accident.
The eigenvalues of A A transpose were exactly the same as the eigenvalues of -- this one was A transpose A.
That's no surprise at all.
The eigenvalues of A B are the same as the eigenvalues of B A.
That's a very nice fact, that eigenvalues stay the same if I switch the order of multiplication.
So no surprise to see 32 and What I learned -- first the check that things were numerically correct, but now I've learned these eigenvectors, and actually they're about as nice as can be.
They're the best orthogonal matrix, just the identity.
Okay.
So my claim is that it ought to all fit together, that these numbers should come out right.
The numbers should come out right because the matrix multiplications use the properties that we want.
Okay.
Shall we just check that?
Here's the identity, so not doing anything -- square root of 32 is multiplying that row, so that square root of 32 divided by square root of two means square root of 16, four, correct?
And square root of 18 is divided by square root of two, so that leaves me square root of 9, which is three, but -- well, Professor Strang, you see the problem?
Why is that -- okay.
Why I getting minus three three here and here I'm getting three minus three?
Phooey.
I don't know why.
It shouldn't have happened, but it did.
Now, okay, you could say, well, just -- the eigenvector there could have -- I could have had the minus sign here for that eigenvector, but I'm not happy about that.
Hmm.
Okay.
So I realize there's a little catch here somewhere and I may not see it until Wednesday.
Which then gives you a very important reason to come back on Wednesday, to catch that sine difference.
So what did I do illegally?
I think I put the eigenvectors in that matrix V transpose -- okay, I'm going to have to think.
Why did that come out with with the opposite sines?
So you see -- I mean, if I had a minus there, I would be all right, but I don't want that.
I want positive entries down the diagonal of sigma squared.
Okay.
It'll come to me, but, I'm going to leave this example to finish.
Okay.
And the beauty of, these sliding boards is I can make that go away.
Can I,-- let me not do it, though, yet.
Let me take a second example.
Let me take a second example where the matrix is singular.
So rank one.
Okay, so let me take as an example two, where my matrix A is going to be rectangular again -- let me just make it four three eight six.
Okay.
That's a rank one matrix.
So that has a null space and only a one dimensional row space and column space.
So actually, my picture becomes easy for this matrix, because what's my row space for this one?
So this is two by two.
So my pictures are both two dimensional.
My row space is all multiples of that vector four three.
So the whole -- the row space is just a line, right?
That's the row space.
And the null space, of course, is the perpendicular line.
So the row space for this matrix is multiples of four three.
Typical row.
Okay.
What's the column space?
The columns are all multiples of four eight, three six, one two.
The column space, then, goes in, like, this direction.
So the column space -- when I look at those columns, the column space -- so it's only one dimensional, because the rank is one.
It's multiples of four eight.
Okay.
And what's the null space of A transpose?
It's the perpendicular guy.
So this was the null space of A and this is the null space of A transpose.
Okay.
What I want to say here is that choosing these orthogonal bases for the row space and the column space is, like, no problem.
They're only one dimensional.
So what should V be?
V should be -- v1, but -- yes, v1, rather -- v1 is supposed to be a unit vector.
There's only one v1 to choose here, only one dimension in the row space.
I just want to make it a unit vector.
So v1 will be -- it'll be this vector, but made into a unit vector, so four -- point eight point six.
Four fifths, three fifths.
And what will be u1?
u1 will be the unit vector there.
So I want to turn four eight or one two into a unit vector, so u1 will be -- let's see, if it's one two, then what multiple of one two do I want?
That has length square root of five, so I have to divide by square root of five.
Let me complete the singular value decomposition for this matrix.
So this matrix, four three eight six, is -- so I know what u1 -- here's A and I want to get U the basis in the column space.
And it has to start with this guy, one over square root of five two over square root of five.
Then I want the sigma.
Okay.
What are we expecting now for sigma?
This is only a rank one matrix.
We're only expecting a sigma one, which I have to find, but zeroes here.
Okay.
So what's sigma one?
It should be the -- where did these sigmas come from?
They came from A transpose A, so I -- can I do that little calculation over here?
A transpose A is four three -- four three eight six times four three eight six.
This had better -- this is a rank one matrix, this is going to be -- the whole thing will have rank one, that's 16 and 64 is 80, 12 and 48 is 60, 12 and 48 is 60, 9 and 36 is 45.
Okay.
It's a rank one matrix.
Of course.
Every row is a multiple of four three.
And what's the eigen -- what are the eigenvalues of that matrix?
So this is the calculation -- this is like practicing, now.
What are the eigenvalues of this rank one matrix?
Well, tell me one eigenvalue, since the rank is only one, one eigenvalue is going to be zero.
And then you know that the other eigenvalue is going to be a hundred and twenty five.
So that's sigma squared, right, in A transpose A.
So this will be the square root of a hundred and twenty five.
And then finally, the V transpose -- the Vs will be -- there's v1, and what's v2?
What's v2 in the -- how do I make this into an orthonormal basis?
Well, v2 is, in the null space direction.
It's perpendicular to that, so point six and minus point eight.
So those are the Vs that go in here.
Point eight, point six and point six minus point eight.
Okay.
And I guess I better finish this guy.
So this guy, all I want is to complete the orthonormal basis -- it'll be coming from there.
It'll be a two over square root of five and a minus one over square root of five.
Let me take square root of five out of that matrix to make it look better.
So one over square root of five times one two two minus one.
Okay.
So there I have -- including the square root of five -- that's an orthogonal matrix, that's an orthogonal matrix, that's a diagonal matrix and its rank is only one.
And now if I do that multiplication, I pray that it comes out right.
The square root of five will cancel into that square root of one twenty five and leave me with the square root of 25, which is five, and five will multiply these numbers and I'll get whole numbers and out will come A.
Okay.
That's like a second example showing how the null space guy -- so this -- this vector and this one were multiplied by this zero.
So they were easy to deal with.
Tthe key ones are the ones in the column space and the row space.
Do you see how I'm getting columns here, diagonal here, rows here, coming together to produce A.
Okay, that's the singular value decomposition.
So, let me think what I want to add to complete this topic.
So that's two examples.
And now let's think what we're really doing.
We're choosing the right basis for the four subspaces of linear algebra.
Let me write this down.
So v1 up to vr is an orthonormal basis for the row space.
u1 up to ur is an orthonormal basis for the column space.
And then I just finish those out by v(r+1), the rest up to vn is an orthonormal basis for the null space.
And finally, u(r+1) up to is an orthonormal basis for the null space of A transpose.
Do you see that we finally got the bases right?
They're right because they're orthonormal, and also -- again, Graham Schmidt would have done this in chapter four.
Here we needed eigenvalues, because these bases make the matrix diagonal.
A times V I is a multiple of U I.
So I'll put "and" -- the matrix has been made diagonal.
When we choose these bases, there's no coupling between Vs and no coupling between Us.
Each A -- A times each V is in the direction of the corresponding U.
So it's exactly the right basis for the four fundamental subspaces.
And of course, their dimensions are what we know.
The dimension of the row space is the rank r, and so is the dimension of the column space.
The dimension of the null space is n-r, that's how many vectors we need, and m-r basis vectors for the left null space, the null space of A transpose.
Okay.
I'm going to stop there.
I could develop further from the SVD, but we'll see it again in the very last lectures of the course.
So there's the SVD.
Thanks.
OK, this is the lecture on linear transformations.
Actually, linear algebra courses used to begin with this lecture, so you could say I'm beginning this course again by talking about linear transformations.
In a lot of courses, those come first before matrices.
The idea of a linear transformation makes sense without a matrix, and physicists and other -- some people like it better that way.
They don't like coordinates.
They don't want those numbers.
They want to see what's going on with the whole space.
But, for most of us, in the end, if we're going to compute anything, we introduce coordinates, and then every linear transformation will lead us to a matrix.
And then, to all the things that we've done about null space and row space, and determinant, and eigenvalues -- all will come from the matrix.
But, behind it -- in other words, behind this is the idea of a linear transformation.
Let me give an example of a linear transformation.
So, example.
Example one.
A projection.
I can describe a projection without telling you any matrix, anything about any matrix.
I can describe a projection, say, this will be a linear transformation that takes, say, all of R^2, every vector in the plane, into a vector in the plane.
And this is the way people describe, a mapping.
It takes every vector, and so, by what rule?
So, what's the rule, is, I take a -- so here's the plane, this is going to be my line, my line through my line, and I'm going to project every vector onto that line.
So if I take a vector like b -- or let me call the vector v for the moment -- the projection -- the linear transformation is going to produce this vector as T(v).
So T -- it's like a function.
Exactly like a function.
You give me an input, the transformation produces the output.
So transformation, sometimes the word map, or mapping is used.
A map between inputs and outputs.
So this is one particular map, this is one example, a projection that takes every vector -- here, let me do another vector v, or let me do this vector w, what is T(w)?
You see?
There are no coordinates here.
I've drawn those axes, but I'm sorry I drew them, I'm going to remove them, that's the whole point, is that we don't need axes, we just need -- so guts -- get it out of there, I'm not a physicist, so I draw those axes.
So the input is w, the output of the projection is, project on that line, T(w).
OK. Now, I could think of a lot of transformations T. But, in this linear algebra course, I want it to be a linear transformation.
So here are the rules for a linear transformation.
Here, see, exactly, the two operations that we can do on vectors, adding and multiplying by scalars, the transformation does something special with respect to those operations.
So, for example, the projection is a linear transformation because -- for example, if I wanted to check that one, if I took v to be twice as long, the projection would be twice as long.
If I took v to be minus -- if I changed from v to minus v, the projection would change to a minus.
So c equal to two, c equal minus one, any c is OK.
So you see that actually, those combine, I can combine those into one statement.
What the transformation does to any linear combination, it must produce the same combination of T(v) and T(w).
Let's think about some -- I mean, it's like, not hard to decide, is a transformation linear or is it not.
Let me give you an example so you can tell me the answer.
Suppose my transformation is -- here's another example two.
Shift the whole plane.
So here are all my vectors, my plane, and every vector v in the plane, I shift it over by, let's say, three by some vector v0.
Shift whole plane by v0.
So every vector in the plane -- this was v, T(v) will be v+v0.
There's T(v).
Here's v0.
There's the typical v. And there's T(v).
You see what this transformation does?
Takes this vector and adds to it.
Adds a fixed vector to it.
Well, that seems like a pretty reasonable, simple transformation, but is it linear?
The answer is no, it's not linear.
Which law is broken?
Maybe both laws are broken.
Let's see.
If I double the length of v, does the transformation produce something double -- do I double T(v)?
No.
If I double the length of v, in this transformation, I'm just adding on the same one -- same v0, not two v0s, but only one v0 for every vector, so I don't get two times the transform.
Do you see what I'm saying?
That if I double this, then the transformation starts there and only goes one v0 out and doesn't double T(v).
In fact, a linear transformation -- what is T of zero?
That's just like a special case, but really worth noticing.
The zero vector in a linear transformation must get transformed to zero.
It can't move, because, take any vector V here -- well, so you can see why T of zero is zero.
Take v to be the zero vector, take c to be three.
Then we'd have T of zero vector equaling three T of zero vector, the T of zero has to be zero.
OK.
So, this example is really a non-example.
Shifting the whole plane is not a linear transformation.
Or if I cooked up some formula that involved squaring, or the transformation that, also non-example, how about the transformation that, takes any vector and produces its length?
So there's a transformation that takes any vector, say, any vector in R^3, let me just -- I'll just get a chance to use this notation again.
Suppose I think of the transformation that takes any vector in R^3 and produces this number.
So that, I could say, is a member of R^1, for example, if I wanted.
Or just real numbers.
That's certainly not linear.
It's true that the zero vector goes to zero.
But if I double a vector, it does double the length, that's true.
But suppose I multiply a vector by minus two.
What happens to its length?
It just doubles.
It doesn't get multiplied by minus two.
So when c is minus two in my requirement, I'm not satisfying that requirement.
So T of minus v is not minus v -- minus, the length, it's just the length.
OK, so that's another non-example.
Projection was an example, let me give you another example.
I can stay here and have a -- this will be an example that is a linear transformation, a rotation.
Rotation by -- what shall we say?
By 45 degrees.
OK?
So again, let me choose this, this will be a mapping, from the whole plane of vectors, into the whole plane of vectors, and it just -- here is the input vector v, and the output vector foam this 45 degree rotation is just rotate that thing by 45 degrees, T(v).
So every vector got rotated.
You see that I can describe this without any coordinates.
And see that it's linear.
If I doubled v, the rotation would just be twice as far out.
If I had v+w, and if I rotated each of them and added, the answer's the same as if I add and then rotate.
That's what the linear transformation is.
OK, so those are two examples.
Two examples, projection and rotation, and I could invent more that are linear transformations where I haven't told you a matrix yet.
Actually, the book has a picture of the action of linear transformations -- actually, the cover of the book has it.
So, in this section seven point one, we can think of a -- actually, here let's take this linear transformation, rotation, suppose I have, as the cover of the book has, a house in R^2.
So instead of this, let me take a small house in R^2.
So that's a whole lot of points.
The idea is, with this linear transformation, that I can see what it does to everything at once.
I don't have to just take one vector at a time and see what T of V is, I can take all the vectors on the outline of the house, and see where they all go.
In fact, that will show me where the whole house goes.
So what will happen with this particular linear transformation?
The whole house will rotate, so the result, if I can draw it, will be, the house will be sitting there.
OK. And, but suppose I give some other examples.
Oh, let me give some examples that involve a matrix.
Example three -- and this is important -- coming from a matrix at -- we always call A.
So the transformation will be, multiply by A.
There is a linear transformation.
And a whole family of them, because every matrix produces a transformation by this simple rule, just multiply every vector by that matrix, and it's linear, right?
Linear, I have to check that A(v) -- A times v plus w equals Av plus A w, which is fine, and I have to check that A times vc equals c A(v).
Check.
Those are fine.
So there is a linear transformation.
And if I take my favorite matrix A, and I apply it to all vectors in the plane, it will produce a bunch of outputs.
See, the idea is now worth thinking of, like, the big picture.
The whole plane is transformed by matrix multiplication.
Every vector in the plane gets multiplied by A.
Let's take an example, and see what happens to the vectors of the house.
So this is still a transformation from plane to plane, and let me take a particular matrix A -- well, if I cooked up a rotation matrix, this would be the right picture.
If I cooked up a projection matrix, the projection would be the picture.
Let me just take some other matrix.
Let me take the matrix one zero zero minus one.
What happens to the house, to all vectors, and in particular, we can sort of visualize it if we look at the house -- so the house is not rotated any more, what do I get?
What happens to all the vectors if I do this transformation?
I multiply by this matrix.
Well, of course, it's an easy matrix, it's diagonal.
The x component stays the same, the y component reverses sign, so that like the roof of that house, the point, the tip of the roof, has an x component which stays the same, but its y component reverses, and it's down here.
And, of course, what we get is, the house is, like, upside down.
Now, I have to put -- where does the door go?
I guess the door goes upside down there, right?
So here's the input, here's the input house, and this is the output.
OK.
This idea of a linear transformation is like kind of the abstract description of matrix multiplication.
And what's our goal here?
Our goal is to understand linear transformations, and the way to understand them is to find the matrix that lies behind them.
That's really the idea.
Find the matrix that lies behind them.
Um, and to do that, we have to bring in coordinates.
We have to choose a basis.
So let me point out what's the story -- if we have a linear transformation -- so start with -- start.
Suppose we have a linear transformation.
Let -- from now on, let T stand for linear transformations.
I won't be interested in the nonlinear ones.
Only linear transformations I'm interested in.
OK.
I start with a linear transformation T. Let's suppose its inputs are vectors in R^3.
OK?
And suppose its outputs are vectors in R^2, for example.
OK. What's an example of such a transformation, just before I go any further?
Any matrix of the right size will do this.
So what would be the right shape of a matrix?
So, for example -- I'm wanting to give you an example, just because, here, I'm thinking of transformations that take three-dimensional space to two-dimensional space.
And I want them to be linear, and the easy way to invent them is a matrix multiplication.
So example, T of v should be any A v. Those transformations are linear, that's what 18.06 is about.
And A should be what size, what shape of matrix should that be?
I want V to have three components, because this is what the inputs have -- so here's the input in R^3, and here's the output in R^2.
So what shape of matrix?
So this should be, I guess, a two by three matrix?
Right?
A two by three matrix.
A two by three matrix, we'll multiply a vector in R^3 -- you see I'm moving to coordinates so quickly, I'm not a true physicist here.
A two by three matrix, we'll multiply a vector in R^3 an produce an output in R^2, and it will be a linear transformation, and OK.
So there's a whole lot of examples, every two by three matrix give me an example, and basically, I want to show you that there are no other examples.
Every linear transformation is associated with a matrix.
Now, let me come back to the idea of linear transformation.
Suppose I've got this linear transformation in my mind, and I want to tell you what it is.
Suppose I tell you what the transformation does to one vector.
OK. You know one thing, then.
All right.
So this is like the -- what I'm speaking about now is, how much information is needed to know the transformation?
By knowing T, I -- to know T of v for all v. All inputs.
How much information do I have to give you so that you know what the transformation does to every vector?
OK, I could tell you what the transformation -- so I could take a vector v1, one particular vector, tell you what the transformation does to it -- fine.
But now you only know what the transformation does to one vector.
So you say, OK, that's not enough, tell me what it does to another vector.
So I say, OK, give me a vector, you give me a vector v2, and we see, what does the transformation do to v2?
Now, you only know -- or do you only know what the transformation does to two vectors?
Have I got to ask you -- answer you about every vector in the whole input space, or can you, knowing what it does to v1 and v2, how much do you now know about the transformation?
You know what the transformation does to a larger bunch of vectors than just these two, because you know what it does to every linear combination.
You know what it does, now, to the whole plane of vectors, with bases v1 and v2.
I'm assuming v1 and v2 were independent.
If they were dependent, if v2 was six times v1, then I didn't give you any new information in T of v2, you already knew it would be six times T of v1.
So you can see what I'd headed for.
If I know what the transformation does to every vector in a basis, then I know everything.
So the information needed to know T of v for all inputs is T of v1, T of v2, up to T of vm, let's say, or vn, for any basis -- for a basis v1 up to vn.
This is a base for any -- can I call it an input basis?
It's a basis for the space of inputs.
The things that T is acting on.
You see this point, that if I have a basis for the input space, and I tell you what the transformation does to every one of those basis vectors, that is all I'm allowed to tell you, and it's enough to know T of v for all v-s, because why?
Because every v is some combination of these basis vectors, c1v1+...+cnvn, that's what a basis is, right?
It spans the space.
And if I know what T does to this, and what T does to v2, and what T does to vn, then I know what T does to V. By this linearity, it has to be c1 T of v1 plus O one plus cn T of vn.
There's no choice.
So, the point of this comment is that if I know what T does to a basis, to each vector in a basis, then I know the linear transformation.
The property of linearity tells me all the other vectors.
All the other outputs.
OK.
So now, we got -- so that light we now see, what do we really need in a linear transformation, and we're ready to go to a matrix.
OK. What's the step now that takes us from a linear transformation that's free of coordinates to a matrix that's been created with respect to coordinates?
The matrix is going to come from the coordinate system.
These are the coordinates.
Coordinates mean a basis is decided.
Once you decide on a basis -- this is where coordinates come from.
You decide on a basis, then every vector, these are the coordinates in that basis.
There is one and only one way to express v as a combination of the basis vectors, and the numbers you need in that combination are the coordinates.
Let me write that down.
So what are coordinates?
Coordinates come from a basis.
Coordinates come from a basis.
The coordinates of v, the coordinates of v are these numbers that tell you how much of each basis vector is in v. If I change the basis, I change the coordinates, right?
Now, we have always been assuming that were working with a standard basis, right?
The basis we don't even think about this stuff, because if I give you the vector v equals three two four, you have been assuming completely -- and probably rightly -- that I had in mind the standard basis, that this vector was three times the first coordinate vector, and two times the second, and four times the third.
But you're not entitled -- I might have had some other basis in mind.
This is like the standard basis.
And then the coordinates are sitting right there in the vector.
But I could have chosen a different basis, like I might have had eigenvectors of a matrix, and I might have said, OK, that's a great basis, I'll use the eigenvectors of this matrix as my basis vectors.
Which are not necessarily these three, but some other basis.
So that was an example, this is the real thing, the coordinates are these numbers, I'll circle them again, the amounts of each basis.
OK.
So, if I want to create a matrix that describes a linear transformation, now I'm ready to do that.
OK, OK.
So now what I plan to do is construct the matrix A that represents, or tells me about, a linear transformation, linear transformation T. OK.
So I really start with the transformation -- whether it's a projection or a rotation, or some strange movement of this house in the plane, or some transformation from n-dimensional space to -- or m-dimensional space to n-dimensional space.
n to m, I guess.
Usually, we'll have T, we'll somehow transform n-dimensional space to m-dimensional space, and the whole point is that if I have a basis for n-dimensional space -- I guess I need two bases, really.
I need an input basis to describe the inputs, and I need an output basis to give me coordinates -- to give me some numbers for the output.
So I've got to choose two bases.
Choose a basis v1 up to vn for the inputs, for the inputs in -- they came from R^n.
So the transformation is taking every n-dimensional vector into some m-dimensional vector.
And I have to choose a basis, and I'll call them w1 up to wn, for the outputs.
Those are guys in R^m.
Once I've chosen the basis, that settles the matrix -- I now working with coordinates.
Every vector in R^n, every input vector has some coordinates.
So here's what I do, here's what I do.
Can I say it in words?
I take a vector v. I express it in its basis, in the basis, so I get its coordinates.
Then I'm going to multiply those coordinates by the right matrix A, and that will give me the coordinates of the output in the output basis.
I'd better write that down, that was a mouthful.
What I want -- I want a matrix A that does what the linear transformation does.
And it does it with respecting these bases.
So I want the matrix to be -- well, let's suppose -- look, let me take an example.
Let me take the projection example.
The projection example.
Suppose I take -- because we've got that -- we've got that projection in mind -- I can fit in here.
Here's the projection example.
So the projection example, I'm thinking of n and m as two.
The transformation takes the plane, takes every vector in the plane, and, let me draw the plane, just so we remember it's a plane -- and there's the thing that I'm projecting onto, that's the line I'm projecting onto -- so the transformation takes every vector in the plane and projects it onto that line.
So this is projection, so I'm going to do projection.
OK.
But, I'm going to choose a basis that I like better than the standard basis.
My basis -- in fact, I'll choose the same basis for inputs and for outputs, and the basis will be -- my first basis vector will be right on the line.
There's my first basis vector.
Say, a unit vector, on the line.
And my second basis vector will be a unit vector perpendicular to that line.
And I'm going to choose that as the output basis, also.
And I'm going to ask you, what's the matrix?
What's the matrix?
How do I describe this transformation of projection with respect to this basis?
OK?
So what's the rule?
I take any vector v, it's some combination of the first basis ve- vector, and the second basis vector.
Now, what is T of v?
Suppose the input is -- well, suppose the input is v1.
What's the output?
v1, right?
The projection leaves this one alone.
So we know what the projection does to this first basis vector, this guy, it leaves it.
What does the projection do to the second basis vector?
It kills it, sends it to zero.
So what does the projection do to a combination?
It kills this part, and this part, it leaves alone.
Now, all I want to do is find the matrix.
I now want to find the matrix that takes an input, c1 c2, the coordinates, and gives me the output, c1 0.
You see that in this basis, the coordinates of the input were c1, c2, and the coordinates of the output are c1, And of course, not hard to find a matrix that will do that.
The matrix that will do that is the matrix one, zero, zero, zero.
Because if I multiply input by that matrix A -- this is A times input coordinates -- and I'm hoping to get the output coordinates.
And what do I get from that multiplication?
I get the right answer, c1 and zero.
So what's the point?
So the first point is, there's a matrix that does the job.
If there's a linear transformation out there, coordinate-free, no coordinates, and then I choose a basis for the inputs, and I choose a basis for the outputs, then there's a matrix that does And what's the job?
the job.
It multiplies the input coordinates and produces the output coordinates.
Now, in this example -- let me repeat, I chose the input basis was the same as the output basis.
The input basis and output basis were both along the line, and perpendicular to the line.
They're actually the eigenvectors of the projection.
And, as a result, the matrix came out diagonal.
In fact, it came out to be lambda.
This is like, the good basis.
So the good -- the eigenvector basis is the good basis, it leads to the matrix -- the diagonal matrix of eigenvalues lambda, and just as in this example, the eigenvectors and eigenvalues of this linear transformation were along the line, and perpendicular.
The eigenvalues were one and zero, and that's the matrix that we got.
OK.
So that's a, like, the great choice of matrix, that's the choice a physicist would do when he had to finally -- he or she had to finally bring coordinates in unwillingly, the coordinates to be chosen, the good coordinates are the eigenvectors, because, if I did this projection in the standard basis -- which I could do, right?
I could do the whole thing in the standard basis -- I better try, if I can do that.
What are we calling -- so I'll have to tell you now which line we're projecting on.
Say, the 45 degree line.
So say we're projecting onto 45 degree line, and we use not the eigenvector basis, but the standard basis.
The standard basis, v1, is one, zero, and v2 is zero, one.
And again, I'll use the same basis for the outputs.
Then I have to do this -- I can find a matrix, it will be the matrix that we would always think of, it would be the projection matrix.
It will be, actually, it's the matrix that we learned about in chapter four, it's what I call the matrix -- do you remember, P was A, A transpose over A transpose A?
And I think, in this example, it will come out, one-half, one-half, one-half, one-half.
I believe that's the matrix that comes from our formula.
And that's the matrix that will do the job.
If I give you this input, one, zero, what's the output?
The output is one-half, one-half.
And that should be the right projection.
And if I give you the input zero, one, the output is, again, one-half, one-half, again the projection.
So that's the matrix, but not diagonal of course, because we didn't choose a great basis, we just chose the handiest basis.
Well, so the course has practically been about the handiest basis, and just dealing with the matrix that we got.
And it's not that bad a matrix, it's symmetric, and it has this P squared equal P property, all those things are good.
But in the best basis, it's easy to see that P squared equals P, and it's symmetric, and it's diagonal.
So that's the idea then, is, do you see now how I'm associating a matrix to the transformation?
I'd better write the rule down, I'd better write the rule down.
The rule to find the matrix A.
All right, first column.
So, a rule to find A, we're given the bases.
Of course, we don't -- because there's no way we could construct the matrix until we're told what the bases are.
So we're given the input basis, and the output basis, v1 to vn, w1 to wm.
Those are given.
Now, in the first column of A, how do I find that column?
The first column of the matrix.
So that should tell me what happens to the first basis vector.
So the rule is, apply the linear transformation to v1.
To the first basis vector.
And then, I'll write it -- so that's the output, right?
The input is v1, what's the output?
The output is in the output space, it's some combination of these guys, and it's that combination that goes into the first column -- so, let me -- I'll put this word -- right, I'll say it in words again.
How to find this matrix.
Take the first basis vector.
Apply the transformation, then it's in the output space, T of v1, so it's some combination of these outputs, this output basis.
So that combination, the coefficients in that combination will be the first column -- so a1, a row 2, column 1, w2, am1, wm.
There are the numbers in the first column of the matrix.
Let me make the point by doing the second column.
Second column of A.
What's the idea, now?
I take the second basis vector, I apply the transformation to it, that's in -- now I get an output, so it's some combination in the output basis -- and that combination is the bunch of numbers that should go in the second column of the matrix.
OK. And so forth.
So I get a matrix, and the matrix I get does the right job.
Now, the matrix constructed that way, and following the rules of matrix multiplication.
The result will be that if I give you the input coordinates, and I multiply by the matrix, so the outcome of all this is A times the input coordinates correctly reproduces the output coordinates.
Why is this right?
Let me just check the first column.
Suppose the input coordinates are one and all zeros.
What does that mean?
What's the input?
If the input coordinates are one and other -- and the rest zeros, then the input is v1, right?
That's the vector that has coordinates one and all zeros.
OK?
When I multiply A by the one and all zeros, I'll get the first column of A, I'll get these numbers.
And, sure enough, those are the output coordinates for T of v1.
So we made it right on the first column, we made it right on the second column, we made it right on all the basis vectors, and then it has to be right on every vector.
So there is a picture of the matrix for a linear OK. transformation.
Finally, let me give you another -- a different linear transformation.
The linear transformation that takes the derivative.
That's a linear transformation.
Suppose the input space is all combination c1 plus c2x plus c3 x squared.
So the basis is these simple functions.
Then what's the output?
Is the derivative.
The output is the derivative, so the output is c2+2c3 x.
And let's take as output basis, the vectors one and x.
So we're going from a three-dimensional space of inputs to a two-dimensional space of outputs by the derivative.
And I don't know if you ever thought that the derivative is linear.
But if it weren't linear, taking derivatives would take forever, right?
We are able to compute derivatives of functions exactly because we know it's a linear transformation, so that if we learn the derivatives of a few functions, like sine x and cos x and e to the x, and another little short list, then we can take all their combinations and we can do all the derivatives.
OK, now what's the matrix?
What's the matrix?
So I want the matrix to multiply these input vectors -- input coordinates, and give these output coordinates.
So I just think, OK, what's the matrix that does it?
I can follow my rule of construction, or I can see what the matrix is.
It should be a two by three matrix, right?
And the matrix -- so I'm just figuring out, what do I want?
No, I'll -- let me write it here.
What do I want from my matrix?
What should that matrix do?
Well, I want to get c2 in the first output, so zero, one, zero will do it.
I want to get two c3, so zero, zero, two will do it.
That's the matrix for this linear transformation with those bases and those coordinates.
You see, it just clicks, and the whole point is that the inverse matrix gives the inverse to the linear transformation, that the product of two matrices gives the right matrix for the product of two transformations -- matrix multiplication really came from linear transformations.
I'd better pick up on that theme Monday after Thanksgiving.
And I hope you have a great holiday.
I hope Indian summer keeps going.
OK, see you on Monday.
OK.
So, coming nearer the end of the course, this lecture will be a mixture of the linear algebra that comes with a change of basis.
And a change of basis from one basis to another basis is something you really do in applications.
And, I would like to talk about those applications.
I got a little bit involved with compression.
Compressing a signal, compressing an image.
And that's exactly change-of-basis.
And then, the main theme in this chapter is th- the connection between a linear transformation, which doesn't have to have coordinates, and the matrix that tells us that transformation with respect to coordinates.
So the matrix is the coordinate-based description of the linear transformation.
Let me start out with the nice part, which is just to tell you something about image compression.
Those of you -- well, everybody's going to meet compression, because you know that the amount of data that we're getting -- well, these lectures are compressed.
So that, actually, probably you see my motion as jerky?
Shall I use that word?
Have you looked on the web?
I should like to find a better word.
Compressed, let's say.
So the complete signal is, of course, in those video cameras, and in the videotape, but that goes to the bottom of building nine, and out of that comes a jumpy motion because it uses a standard system for compressing images.
And, you'll notice that the stuff that sits on the board comes very clearly, but it's my motion that needs a whole lot of bits, right?
So, and if I were to run up and back up there and back, that would need too many bits, and I'd be compressed even more.
So, what does compression mean?
Let me just think of a still image.
And of course, satellites, and computations of the climate, computations of combustion, the computers and sensors of all kinds are just giving us overwhelming amounts of data.
The Web is, too.
Now, some compression can be done with no loss.
Lossless compression is possible just using, sort of, the fact that there are redundancies.
But I'm talking here about lossy compression.
So I'm talking about -- here's an image.
And what does an image consist of?
It consists of a lot of little pixels, right?
Maybe five hundred and twelve by five hundred and twelve.
Two to the ninth by two to the ninth pixels, and so this is pixel number one, one, so that's a pixel.
And if we're in black and white, the typical pixel would tell us a gray-scale, from zero to two fifty five.
So a pixel is usually a value of one of the xi, so this would be the i-th pixel, is -- it's usually a real number on a scale from zero to two fifty five.
In other words, two to the eighth possibilities.
So usually, that's the standard, so that's eight -- eight bits.
But then we have that for every pixel, so we have five hundred and twelve squared pixels, we're really operating x is a vector in R^n, but what is n?
n is five hundred and twelve squared.
That's our problem, right there.
A pixel is a vector that gives us the information about the image.
I'm sorry.
The image that comes through is a vector of that length that -- that's the information that we have about the image, if it's a color image, we would have three times that length, because we'd need three coordinates to get color.
So it would be three times five hundred and twelve squared.
It's an enormous amount of information, and we couldn't send out the image for these lectures without compressing it.
It would overload the system.
So it has to be compressed.
The standard compression, and still used with lectures is, called JPEG.
I think that stands for Joint Photographic Experts Group.
They established a system of compression.
And I just want to tell you what it's about.
It's a change-of-basis.
What basis do we have?
The current basis we have is, you could say, the standard basis is, every pixel, give a value.
So that's like we have a vector x which is five hundred and twelve squared long and, in the i-th position, we get a number like one twenty one or something.
The pixel next to it might be one twenty four, maybe where my tie begins to enter, so if it was mostly blue shirt, this would be a slight difference in shading, but pretty close, then the tie would be a different color, so we might have quite a few pixels for the blue shirt, and a whole lot more for the blackboard, that are very close.
And that's what are very correlated.
And that's what gives us the possibility of compression.
For example, before the lecture starts, if we had a blank blackboard, then there's an image, but it would make no sense to take that image and tell you what it is pixel by pixel.
I mean, there's a case in which all pixel values, all gray levels are the same -- or practically the same, depending on the erasing of the board, but extremely close -- and, so that's an image where the standard basis is lousy.
That's the basic fact, that the standard basis which gives the value of every pixel makes no use of the fact that we're getting a whole lot of pixels whose gray levels -- the neighboring pixels tend to have the same gray level as their neighbors.
So how do we take advantage of that fact?
Well, one basis vector that would be extremely nice to include in the basis would be a vector of all ones.
That's not in our standard basis, so let me just write again, the standard basis is our one, and all the rest zeroes, zero, one, and all the rest, zeroes, everybody knows what these standard basis is.
Now, any other basis for R -- so this is -- for this very high-dimensional space -- now I'm going to speak about a better basis.
Better basis -- and let me just emphasize, one vector that would be extremely nice to have in that basis is the vector of all ones.
Why is that?
Let me just say again, because that vector of all ones, by itself, one vector is able to completely give the information on a solid image.
Of course, our image won't be solid, it will have a mix of solid and signal.
So having that one vector in the basis is going to save us a whole lot.
Now, the question is, what other vectors should be in the basis?
The extreme vector in the basis might be a vector of one minus one, one minus one, one minus one.
That would be a vector that shows -- I mean, that's like a checkerboard vector, right?
That's a vector that would, if the image was like a huge checkerboard of plus, minus, plus, minus, plus, minus, that vector would carry the whole signal.
But much more common would be maybe to have half the image, darker and the other half lighter.
So another vector that might be quite useful in here would be half ones and half minus ones.
I'm just trying to get across the idea of that a basis could be where, that first of all, we've got the bases at our disposal.
Like, we're free to choose that.
And it's a billion-dollar decision what we choose.
So, and TV people would rather pre- would prefer one basis based on the way the signal is scanned, and movie people would prefer another, I mean, there's giant politics in this question that really reduces to a linear algebra problem, what basis to choose.
I'll just mention the best known basis, which JPEG uses, -- let me put that here -- is the Fourier basis.
So when you use the Fourier basis, that includes -- this is the constant vector, the D C vector if we're electrical engineers, the l- vector of all ones, so it would include one, one, one, one.
Often eight by eight is a good choice.
Eight by eight is a good choice.
So, what do I mean by this eight by eight?
I mean that the big signal, which is five twelve by five twelve, gets broken down, and JPEG does this, into eight by eight blocks.
And we -- sort of, this is too much to deal with at once.
So what JPEG does is take this eight by eight block, which is sixty four coefficients, sixty four, pixels, and changes the basis on that piece.
And then, now, let's see, I was going to write down Fourier, so you remember Fourier as this vector of all ones, and then, the vector -- oh, well, actually, I gave a lecture earlier about the Fourier matrix, this matrix whose columns are powers of a complex number w. I won't repeat that, because I don't want to go into the details of the Fourier basis, just to tell you how compression works.
So what happens in JPEG?
What happens to the video, to each image, of these lectures?
It gets broken into eight by eight blocks.
OK.
Within each block, we have sixty four coefficients, sixty four basis vectors, sixty four pixels, and we change basis in sixty four dimensional space using these Fourier vectors.
Just note, that was a lossless step.
Let me emphasize.
In comes the signal x.
We change basis.
This is the basis change.
Change basis.
Choose a better basis.
So it produces, the coefficients c. So sixty four pixels come in, sixty four coefficients come out.
Now comes the compression.
Now come -- this was lossless.
It's just -- we know that R -- R sixty four has plenty of bases, and we've chosen one.
Now, in that basis, we write the signal in that basis, and that's what my lecture -- that's the math part of my lecture.
Now here's the application part.
The next part is going to be the compression step.
And that's lossy.
We're going to lose information.
And what will actually happen at that step?
Well, one thing we could do is just throw away the small coefficients.
So that's called thresholding, we set some threshold.
Every coefficient, every basis vector that's not in there more than the threshold value, and we set them threshold so that our eye can't see the difference, or can hardly see the difference, whether we throw away that little bit of that basis vector or keep it.
So this compression step produces a compressed set of I'll just keep going here.
coefficients.
So it keeps going, this compression step produces some coefficient c hat.
And with many zeroes.
So that's where the compression came.
Probably, there is enough of this vector of all ones -- we very seldom throw that away.
Usually, its coefficient will be large.
But the coefficient of something like this, that quickly alternative vector, there's probably very little of that in any smooth signal.
That's high-frequency -- this is low-frequency, zero frequency.
This stuff is the highest frequency we could have, and if the noise, the jitter is producing that sort of output, but a smooth lecture like this one is, has very little of that highest frequency, very little noise in this lecture.
OK, so we throw away whatever there is, and we're left with just a few coefficients, and then we reconstruct a signal using those coefficients.
We take those coefficients, times their basis vectors, but this sum doesn't have sixty four terms any more.
Probably, it has about two or three terms.
So that would -- say it has three terms.
From sixty four down to three, that's compression of twenty one to one.
That's the kind of compression you're looking for.
And everybody is looking for that sort of compression.
Let's see, I guess I met the problem with the FBI and fingerprints.
So there's a whole lot of still images.
You know, with your thumb, you make these inky marks which go somewhere.
it used to go to Washington and get stored in a big file.
So Washington had a file of thirty million murderers, cheaters on quizzes, other stuff, and actually, there was no way to retrieve them in time.
So suppose you're at the police station, they say, OK, this person may have done this, check with Washington, have they got -- are his or her fingerprints on file?
Well, Washington won't know the answer within a week if it's got filing cabinets full of fingerprints.
So of course, the natural step is digitizing.
So all fingerprints are now digitized, so now it's at least electronic, but still there's too much information in each one.
I mean, you can't search through that many, fingerprints if the digital image is five twelve squared by five twelve squared, if it's that many pixels.
So you get compressed.
So the FBI had to decide what basis to choose for compression of fingerprints.
And then they built a big new facility in West Virginia, and that's where fingerprints now are sent.
So I think, if you get your fingerprints done now at the police station, if it's an up-to-date police station, it happens digitally, and the signal is sent digitally, and then in West Virginia, it's compressed and indexed.
And then, if they want to find you, they can do it within minutes instead of within a week.
OK.
So this compression comes up for signals, for images, for video -- which is, like these lectures -- there's another aspect.
You could treat the video as one still image after another one, and compress each one, and then run them and make a video.
But that misses -- well, you can see why that's not optimal.
In a video thing, you have a sequence of images, so video is really a sequence of images but what about one image to the next image?
They're extremely correlated.
I mean that I'm getting an image every split-second, and also, I'm moving slightly.
That's what's producing the, jumpy motion on the video.
But I'm not, like, you know -- each image in the sequence is pretty close to the one before.
So you have to use, like, prediction and correction.
I mean, the image of me one instant -- one time-step later, you would assume would be the same, and then plus a small correction.
And you would only code and digitize the correction, and compress the correction.
So a sequence of images that's highly correlated and the problem in compression is always to use this correlation, this fact that, in time, or in space, things don't change instantly, they're very often smooth changes, and, you can predict one value from the previous value.
OK.
So those are applications which are pure linear algebra.
I could, well, maybe you'll allow me to tell you, and the book describes, the new basis that's the competition for Fourier.
So the competition for Fourier is called wavelets, and I can describe what that basis is like, say, in the eight by eight case.
So the eight by eight wavelet basis is the vector of all ones, eight ones, then the vector of four ones and four minus ones, then the vector of two ones, and two minus ones, and four zeroes.
And also the vector of four zeroes and two ones and two minus ones.
So now I'm up to four, and I need four more, right?
For R^8?
The next basis vector will be one minus one and six zeroes, and then three more like that, with the one minus one there, and there, and there.
So those are eight vectors in eight-dimensional space, those are called wavelets, and it's a very simple wavelet choice, it's a more sophisticated choice.
This is a little jumpy, to jump between one and minus one.
And, actually, you can see, now, suppose you compare the wavelet basis with the Fourier basis above.
How could I write this guy, which is in the Fourier basis, it's an eight -- it's a vector in R^8.
How would I write that as a combination of the wavelet basis?
Have I told you enough about the wavelet basis that you can see, how does this very fast guy -- what combination of the wavelet basis is that very fast guy?
It would be this one -- it would be the sum of these four, right?
That very fast guy will be that one minus one, and the next one, and the next one, and the next one.
So this is the sum of those last four wavelets.
This one, we've kept, and so on.
So, each -- well, every -- well, that's what a basis does.
Every vector in R^8 is some combination of those, and for the linear algebra -- so the linear algebra is this step, find the coefficient.
That's the step we want to take.
What if I give you the basis, like this wavelet basis, and I give you the pixel -- so here are the pixel values, P1, P2, down to P8 -- what's the job?
What's the linear algebra here?
So these are the values, this is in the standard basis, right?
Those are just the values at eight successive points.
I guess I'm dropping down to one dimension, instead of eight by eight, I'm just going to take eight pixel values along that first top row.
So what do I want to do?
In standard basis, here are the pixel values.
I want to write that as a combination of c1 times this guy, plus c2 times this guy, plus c3, these are the coefficients, plus c4 times this one -- do you see what I'm doing?
I want to write this vector P as a combination of c1 times the first wavelet plus c8 times the eighth wavelet.
That's the transform step.
That's the lossless step.
That's the step from P -- oh, I'm calling it P here, and I called it x there, so let me -- at the risk of moving, and therefore making this jumpy -- suppose the signal I'm now calling P, that a pixel values, and I'm looking for the coefficients.
OK, tell me how to do it.
If I give you eight basis vectors, and I give you the input signal, and I ask for the coefficients, what do I do?
What's the step?
I'm trying to solve this, I want to know the eight coefficients, so I'm changing from the standard basis, which is just the eight gray-scale values to the wavelet basis, where the same vector is represented by eight numbers.
It's got to take eight numbers to tell you a vector in R^8, and those eight numbers are the coefficients of the basis.
Look, we've done this thing before.
There is the equation in vector notation, we want to see it as a matrix.
This is a combination of columns of the wavelet matrix, right?
This is P equals c1, c2, down to c8, and these guys are the columns.
I mean, this is the step that we're constantly taking in this course, the first basis vector goes in the first column, the second basis vector goes in the second column, and so on, the eight columns of this wavelet matrix are the eight basis vectors.
This is a wavelet matrix W. So, the step to change basis -- so now I'm finally coming to this change-of-basis, so the change of basis that, let me stay with this board, but -- well, let me just go above it, here.
So the standard basis, we know, the wavelet basis we have here, and the transform is simply, solve the equations, P=W C. So the coefficients are W inverse P. Right.
This shows a critical point.
A good basis has a nice, fast, inverse.
So good basis means what?
So this is like the billion-dollar competition, Eh?
and it's not over yet.
People are going to come up with better bases than these.
So a good basis will be, first good thing would be fast.
I have to be able to multiply by W fast, and multiply by W -- by its inverse fast.
That's -- if a basis doesn't allow you to do that fast, then it's going to take so much time that you can't afford it.
So these bases -- the Fourier basis, everybody said, OK, I know how to deal quickly with the Fourier basis, because we have something called the Fast Fourier Transform.
So there's a FFT that came in my earlier lecture, and comes in the last chapter of the book, so change-of-basis is done -- if, for the Fourier basis, it's done fast by the FFT and there's a fast wavelet transform.
I can change, for this wavelet example, this matrix is easy to invert.
It's just somebody had a smart idea in choosing that wavelet basis and inverting it, it has a nice inverse.
Actually, you can see why it has a nice inverse.
Do you see any property of these eight basis vectors?
Well, I've only written five of them, but if you see that property for those five, you'll see it for the three remaining.
Well, if I give you those eight vectors and ask, what's a nice property?
Well, you would say, first, they're all ones and minus ones and zeroes.
So every multiplication is very fast using -- just in binary.
But what's the other great property of those vectors?
Anybody see it?
So, of course, when I think about a basis, one nice property -- I don't have to have it, but I'm happy if it's there -- is that they're orthogonal.
If the basis vectors are orthogonal, then I'm in good shape.
And these are... do you see?
Take the dot product of that with that, you get four plus ones and four minus ones, you get zero.
Take the dot product of that with that.
You get two plus ones and two minus ones.
Or the dot product of that with that.
Two plus ones and two minus ones.
You can easily check that that's an orthogonal basis.
It's not orthonormal.
To fix it up, I should divide by the length, to make them unit vectors.
Let's suppose I do that.
So somewhere in here, I've got to account for the fact that this has length square root of eight, that has length square root of four, that has length square root of two.
But that's just a constant factor that's easy to -- so suppose we've done that.
Then, tell me what's W inverse?
That's what chapter four, section four point four was about.
If we have orthonormal columns then the inverse is the same as the transpose.
So if we have a fast way to multiply by W, which we do, the inverse is going to look just the same, and we'll have a fast way to do W inverse.
So that's the wavelet basis passes this requirement for fast.
We can use it fast.
But there's a second requirement, is it any good?
Because the the very fastest thing we could do is not to change basis at all.
Right?
The fastest thing would be, OK, stay with the standard basis, stay with eight pixel values.
But that was poor from compression point of view, right?
Those eight pixel values, if I just took those eight numbers, I can't throw some of those away.
If I throw away ninety percent -- if I compress ten to one, and throw away ninety percent of my pixel values, well, my picture's just gone dark.
Whereas, the basis that was good, the wavelet basis or the Fourier basis, if I throw away c5, c6, c7, and c8, all I'm throwing away is little blips that are probably there in very small amounts.
So the second property that we need is good compression.
So first, it has to be fast, and secondly, a few basis vectors should come close to the signal.
So a few is enough.
Can I write it that way?
A few basis vectors are enough to reproduce the image just exactly as on a video of these 18.06 lectures.
Uh, I don't know what the compression rate is, I'll ask, David, who does the compression -- and, by the way, I'll try to get the lectures, that are relevant for the quiz up onto the Web in time.
So I'll send them a message today.
So, he's using the Fourier basis because the JPEG -- so JPEG two thousand, which will be the next standard for image compression, will include wavelets.
So, I mean, you're actually getting a kind of up-to-date, picture of where this big world of signal and image processing is.
That Fourier is what everybody knew, and what people automatically used, and the new one is wavelets, where this is the simplest set of wavelets.
And this isn't the one that the FBI uses, by the way, the FBI uses a smoother wavelet, instead of jumping from one to minus one, it's a smooth, Cutoff.
and, that's what we'll be in in JPEG two thousand.
OK, so that's that application.
Now, let me come to the math, the linear algebra part of the lecture.
Well, we've actually seen a change-of-basis.
So let -- let me just review that eh-eh change-of-basis idea, and then the i- and then the transformation to a matrix.
OK.
So this, I hope you see that these applications are really big.
Now, I have to talk a little about change-of-basis, and a little about that.
The matrix.
OK. OK. OK.
So change-of-basis.
Basically, forgive that put, OK, I have, I have my vector in one basis, and I want to change to a different one.
Actually, you saw it for the wavelet case.
So I need the -- let the matrix W, and the columns of W be the new basis vectors.
Then the change-of-basis involves, just as it did there, W inverse.
So we have the vector, say, x, in the old basis, and that converts to a vector, let's say, c, in the new basis, and the relation is exactly what we had there, that x is W c. That's the step we have to take.
There's a matrix W that gives us a change-of-basis.
OK. What I want to do is think about transformations on matrices.
So here's the question to complete this lecture.
Suppose I have a linear transformation T. So we would think of it as an eight -- as a n by n matrix.
And it's computed with respect to a certain basis.
So T -- no, I'm sorry.
I've got the transformation T, period.
That's taking eight-dimensional space to eight-dimensional space.
Now, let's get matrices in there.
OK.
So, with respect to a first basis, say v1 up to v8, it has a matrix A. I'm just setting up letters here.
With respect to a second basis, say, I'll make it u1 up to -- or w1, since I've used (w)s, w1 up to w8, it has a matrix B.
And my question is, what's the connection between A How is the matrix -- the transformation T is settled.
and B?
We could say, it's a rotation, for example.
So that would be one transformation of eight-dimensional space, just spin it a little.
Or project it.
Or whatever linear transformation we've got.
Now, we have to remember -- my first step is to remind you how you create that matrix A.
Then my second step is, we would use the same method to create B, but because it came from the same transformation, there's got to be a relation between A and B.
What's the relation between A and B?
And let me jump to the answer on that one.
That if I have the same transformation, and I'm compute on its matrix in one basis, and then I computer it in another basis, those two matrices are similar.
So these two matrices are similar.
Now, do you remember what similar matrices meant?
Similar.
A is similar to -- the two matrices are similar.
Similar.
And what do I mean by that?
I mean that I take the matrix B, and I can compute it from the matrix A using some similarity, some matrix M on one side, and M inverse on the other.
And this M will be the change-of-basis matrix.
This part of the lecture is, admittedly, compressed.
What I wanted you to -- it's really the conclusion that I want you to spot.
Now, I have to go back and say, what does it mean for A to be the matrix of this transformation T. So I have to remind you what that meant, that was in the last lecture.
Then this is the conclusion that if I change to a different basis, we now know -- see, if I change to a different basis, two things happen.
Every vector has new coordinates.
There, the rule is this one, between the old coordinates and the new ones.
Every matrix changes, every transformation has a new matrix.
And the new matrix is related this way, the M could be the same as the W. The M there would be the W here.
OK.
So, can I, in the remaining minutes, recapture my lecture -- the end of my lecture that was just before Thanksgiving, about the matrix?
OK. What's the matrix?
And I'll just take one basis.
So now this part is going to go onto this board here.
What is the matrix?
What is A?
OK.
Using a basis v1 up to v8.
Mm.
OK. What's the point?
The point is, if I know what the transformation does to those eight basis vectors, I know it completely.
I know T, I know everything about T, I know T completely from knowing T of V -- what T does to v1, what T does to v2, what T does to v8.
Why is that?
It's because T is a linear transformation.
So that if I know what these outputs are -- so these are the inputs v1 up to v8, these are the outputs from the transformation, like everyone rotated, everyone projected, whatever transformation I've done, then why is it that I know everything?
How does linearity work?
Why?
This is because every x is some combination of these basis vectors, right?
c1v1, c2v2, c8v8, they were a basis.
That's the whole point of a basis, that every vector is a combination of the basis vectors in exactly one way.
And then, what is T of x?
The point is, I claim that we know T of x completely for every x, because every x is a combination of those -- and now we use the linear transformation part to say that the output from x has to be c1 times the output from v1 plus v2 times the output from v2, and so on.
Up through c8 times the output from v8.
So this is like just saying, OK. We know everything when we know what T does to each basis vector.
OK.
So those are the eight things we need.
Now -- but we need these answers in this basis.
So this first output is some combination of the eight basis vectors.
So write T acting on the first input -- in other words, write the first output as a combination of the basis vectors, say a11 v1 + a21 v2 and so on a81 v8.
Write T of v2 as some combination a12 of v1, a22 of v2 and so on.
I'm creating the matrix A, column by column.
Those numbers go in the first column, these numbers go in the second column, the matrix A that thi- this -- this is our matrix that represents T in this basis is these numbers.
a11 down to a18, a21 down to a28, and so on.
OK. That's the recipe.
In other words, if I give you a transformation, and a basis.
So that's what I have to give you.
The inputs are the basis and to tell you what the transformation is.
And then, you tell me -- you compute T for each basis, expand that result in the basis, and that gives you the sixty four numbers that go into the matrix A.
Let me suppose -- let's close with the best example of all.
Suppose v1 to v8, this basis, is the eigenvectors.
Suppose we have an eigenvector basis so that T(vi) is in the same direction of vi.
Now, my question is, what is A?
Can you carry through the steps?
Let's do them together, because we can do it in one minute.
So, we've chosen this perfect basis.
And, actually, with signal image processing, they might look for the eigenvectors.
But that would take more calculation time that just saying, OK, we'll use the wavelet basis.
Or, OK, we'll use the Fourier basis.
But the very best basis is the eigenvector basis.
OK, what's the matrix?
So, what's the first column of the matrix?
How do I get the first column?
I take the first basis vector v1.
I opt -- I look to see, what does the transformation do to it?
The output is lambda one v1.
I express that output as a combination so the first input is v1.
Its output is lambda one v1.
Now write lambda one v1 as a combination of the basis vectors, well, it's already done.
It's just lambda one times the first basis vector and zero times the others.
So this first column will have lambda one and zeroes.
OK. Second input is v2.
Output is lambda two v2.
OK, write that output as a combination of the (v)s. It's already done.
It's just lambda two times the second v. So we need, in the second column, we have lambda two times the second v. Well, you see what's coming, that in that basis, in the eigenvector basis, the matrix is diagonal.
So that's the perfect basis, that's the basis we'd love to have for image processing, but to find the eigenvectors of our pixel matrix would be too expensive.
So we do something cheaper and close, which is to choose a good basis like wavelets.
OK, thanks.
So I'll -- quiz review on Wednesday, all day.
Thanks.
OK, here we go with, quiz review for the third quiz that's coming on Friday.
So, one key point is that the quiz covers through chapter six.
Chapter seven on linear transformations will appear on the final exam, but not on the quiz.
So I won't review linear transformations today, but they'll come into the full course review on the very last lecture.
So today, I'm reviewing chapter six, and I'm going to take some old exams, and I'm always ready to answer questions.
And I thought, kind of help our memories if I write down the main topics in chapter six.
So, already, on the previous quiz, we knew how to find eigenvalues and eigenvectors.
Well, we knew how to find them by that determinant of A minus lambda I equals zero.
But, of course, there could be shortcuts.
There could be, like, useful information about the eigenvalues that we can speed things up with.
OK. Then, the new stuff starts out with a differential equation, so I'll do a problem.
I'll do a differential equation problem first.
What's special about symmetric matrices?
Can we just say that in words?
I'd better write it down, though.
What's special about symmetric matrices?
Their eigenvalues are real.
The eigenvalues of a symmetric matrix always come out real, and there always are enough eigenvectors.
Even if there are repeated eigenvalues, there are enough eigenvectors, and we can choose those eigenvectors to be orthogonal.
So if A equals A transposed, the big fact will be that we can diagonalize it, and those eigenvector matrix, with the eigenvectors in the column, can be an orthogonal matrix.
So we get a Q lambda Q transpose.
That, in three symbols, expresses a wonderful fact, a fundamental fact for symmetric matrices.
OK. Then, we went beyond that fact to ask about positive definite matrices, when the eigenvalues were positive.
I'll do an example of that.
Now we've left symmetry.
Similar matrices are any square matrices, but two matrices are similar if they're related that way.
And what's the key point about similar matrices?
Somehow, those matrices are representing the same thing in different basis, in chapter seven language.
In chapter six language, what's up with these similar matrices?
What's the key fact, the key positive fact about similar matrices?
They have the same eigenvalues.
Same eigenvalues.
So if one of them grows, the other one grows.
If one of them decays to zero, the other one decays to zero.
Powers of A will look like powers of B, because powers of A and powers of B only differ by an M inverse and an M way on the outside.
So if these are similar, then B to the k-th power is M inverse A to the k-th power M. And that's why I say, eh, this M, it does change the eigenvectors, but it doesn't change the eigenvalues.
So same lambdas.
And then, finally, I've got to review the point about the SVD, the Singular Value Decomposition.
OK.
So that's what this quiz has got to cover, and now I'll just take problems from earlier exams, starting with a differential equation.
OK. And always ready for questions.
So here is an exam from about the year zero, and it has a three by three.
So that was -- but it's a pretty special-looking matrix, it's got zeroes on the diagonal, it's got minus ones above, and it's got plus ones like that.
So that's the matrix A. OK.
Step one is, well, I want to solve that equation.
I want to find the general solution.
I haven't given you a u(0) here, so I'm looking for the general solution, so now what's the form of the general solution?
With three arbitrary constants going to be inside it, because those will be used to match the initial condition.
So the general form is u at time t is some multiple of the first special solution.
The first special solution will be growing like the eigenvalue, and it's the eigenvector.
So that's a pure exponential solution, just staying with that eigenvector.
Of course, I haven't found, yet, the eigenvalues and eigenvectors.
That's, normally, the first job.
Now, there will be second one, growing like e to the lambda two, and a third one growing like e to the lambda three.
So we're all done -- well, we haven't done anything yet, actually.
I've got to find the eigenvalues and eigenvectors, and then I would match u(0) by choosing the right three constants.
OK.
So now I ask -- ask you about the eigenvalues and eigenvectors, and you look at this matrix and what do you see in that matrix?
Um, well, I guess we might ask ourselves right away, is it singular?
Is it singular?
Because, if so, then we really have a head start, we know one of the eigenvalues is zero.
Is that matrix singular?
Eh, I don't know, do you take the determinant to find out?
Or maybe you look at the first row and third row and say, hey, the first row and third row are just opposite signs, they're linear-dependent?
The first column and third column are dependent -- it's singular.
So one eigenvalue is zero.
Let's make that lambda one.
Lambda one, then, will be zero.
OK. Now we've got a couple of other eigenvalues to find, and, I suppose the simplest way is to look at A minus lambda I So let me just put minus lambda in here, minus ones above, ones below.
But, actually, before I do it, that matrix is not symmetric, for sure, right?
In fact, it's the very opposite of symmetric.
That matrix A transpose, how is A transpose connected to A?
It's negative A.
It's an anti-symmetric matrix, skew-symmetric matrix.
And we've met, maybe, a two-by-two example of skew-symmetric matrices, and let me just say, what's the deal with their eigenvalues?
They're pure imaginary.
They'll be on the imaginary axis, there be some multiple of I if it's an anti-symmetric, skew-symmetric matrix.
So I'm looking for multiples of I, and of course, that's zero times I, that's on the imaginary axis, but maybe I just do it out, here.
Lambda cubed.
well, maybe that's minus lambda cubed, and then a zero and a zero.
Zero, and then maybe I have a plus a lambda, and another plus lambda, but those go with a minus sign.
Am I getting minus two lambda equals zero?
So.
So I'm solving lambda cube plus two lambda equals zero.
So one root factors out lambda, and the the rest is lambda squared plus two.
OK.
This is going the way we expect, right?
Because this gives the root lambda equals zero, and gives the other two roots, which are lambda equal what?
The solutions of when is lambda squared plus two equals zero then the eigenvalues those guys, what are they?
They're a multiple of i, they're just square root of two i.
When I set this equals to zero, I have lambda squared equal to minus two, right?
To make that zero?
And the roots are square root of two i and minus the square root of two i.
So now I know what those are.
I'll put those in, now.
Either the zero t is just a one.
That's just a one.
This is square root of two I and this is minus square root of two I.
So, is the solution decaying to zero?
Is this a completely stable problem where the solution is going to zero?
No.
In fact, all these things are staying the same size.
This thing is getting multiplied by this number.
e to the I something t, that's a number that has magnitude one, and sort of wanders around the unit circle.
Same for this.
So that the solution doesn't blow up, and it doesn't go to zero.
OK. And to find out what it actually is, we would have to plug in initial conditions.
But actually, the next question I ask is, when does the solution return to its initial value?
I won't even say what's the initial value.
This is a case in which I think this solution is periodic after.
At t equals zero, it starts with c1, c2, and c3, and then at some value of t, it comes back to that.
So that's a very special question, Well, let's just take three seconds, because that special question isn't likely to be on the quiz.
But it comes back to the start, when?
Well, whenever we have e to the two pi i, that's one, and we've come back again.
So it comes back to the start.
It's periodic, when this square root of two i -- shall I call it capital T, for the period?
For that particular T, if that equals two pi i, then e to this thing is one, and we've come around again.
So the period is T is determined here, cancel the i-s, and T is pi times the square root of two.
So that's pretty neat.
We get all the information about all solutions, we haven't fixed on only one particular solution, but it comes around again.
So this was probably my first chance to say something about the whole family of anti-symmetric, skew-symmetric matrices.
OK. And then, finally, I asked, take two eigenvectors (again, I haven't computed the eigenvectors) and it turns out they're orthogonal.
They're orthogonal.
The eigenvectors of a symmetric matrix, or a skew-symmetric matrix, are always orthogonal.
I guess may conscience makes me tell you, what are all the matrices that have orthogonal eigenvectors?
And symmetric is the most important class, so that's the one we've spoken about.
But let me just put that little fact down, here.
Orthogonal x-s. eigenvectors.
A matrix has orthogonal eigenvectors, the exact condition -- it's quite beautiful that I can tell you exactly when that happens.
It happens when A times A transpose equals A transpose times A.
Any time that's the condition for orthogonal eigenvectors.
And because we're interested in special families of vectors, tell me some special families that fit.
This is the whole requirement.
That's a pretty special requirement most matrices have.
So the average three-by-three matrix has three eigenvectors, but not orthogonal.
But if it happens to commute with its transpose, then, wonderfully, the eigenvectors are orthogonal.
Now, do you see how symmetric matrices pass this test?
Of course.
If A transpose equals A, then both sides are A squared, we've got it.
How do anti-symmetric matrices pass this test?
If A transpose equals minus A, then we've got it again, because we've got minus A squared on both sides.
So that's another group.
And finally, let me ask you about our other favorite family, orthogonal matrices.
Do orthogonal matrices pass this test, if A is a Q, do they pass the test for orthogonal eigenvectors.
Well, if A is Q, an orthogonal matrix, what is Q transpose Q?
It's I.
And what is Q Q transpose?
It's I, we're talking square matrices here.
So yes, it passes the test.
So the special cases are symmetric, anti-symmetric (I'll say skew-symmetric,) and orthogonal.
Those are the three important special classes that are in this family.
OK. That's like a comment that, could have been made back in, section six point four.
OK, I can pursue the differential equations, also this question, didn't ask you to tell me, how would I find this matrix exponential, e to the At?
So can I erase this?
I'll just stay with this same... how would I find e to the At?
Because, how does that come in?
That's the key matrix for a differential equation, because the solution is -- the solution is u(t) is e^(At) u(0).
So this is like the fundamental matrix that multiplies the given function and gives the answer.
And how would we compute it if we wanted that?
We don't always have to find e to the At, because I can go directly to the answer without any e to the At-s, but hiding here is an e to the At, and how would I compute it?
Well, if A is diagonalizable.
So I'm now going to put in my usual if A can be diagonalized (and everybody remember that there is an if there, because it might not have enough eigenvectors) this example does have enough, random matrices have enough.
So if we can diagonalize, then we get a nice formula for this, because an S comes way out at the beginning, and S inverse comes way out at the end, and we only have to take the exponential of lambda.
And that's just a diagonal matrix, so that's just e the lambda one t, these guys are showing up, now, in e to the lambda nt.
OK?
That's a really quick review of that formula.
It's something we can compute it quickly if we have done the S and lambda part.
If we know S and lambda, then it's not hard to take that step.
OK, that's some comments on differential equations.
I would like to go on to a next question that I started here.
And it's, got several parts, and I can just read it out.
What we're given is a three-by-three matrix, and we're told its eigenvalues, except one of these is, like, we don't know, and we're told the eigenvectors.
And I want to ask you about the matrix.
OK.
So, first question.
Is the matrix diagonalizable?
And I really mean for which c, because I don't know c, so my questions will all be, for which is there a condition on c, does one c work.
But your answer should tell me all the c-s that work.
I'm not asking for you to tell me, well, c equal four, yes, that checks out.
I want to know all the c-s that make it diagonalizable.
OK?
What's the real on diagonalizable?
We need enough eigenvectors, right?
We don't care what those eigenvalues are, it's eigenvectors that count for diagonalizable, and we need three independent ones, and are those three guys independent?
Yes.
Actually, let's look at them for a moment.
What do you see about those three vectors right away?
They're more than independent.
Can you see why those three got chosen?
Because it will come up in the next part, they're orthogonal.
Those eigenvectors are orthogonal.
They're certainly independent.
So the answer to diagonalizable is, yes, all c, all c. Doesn't matter.
c could be a repeated guy, but we've got enough eigenvectors, so that's what we care about.
OK, second question.
For which values of c is it symmetric?
OK, what's the answer to that one?
If we know the same setup if we know that much about it, we know those eigenvectors, and we've noticed they're orthogonal, then which c-s will work?
So the eigenvalues of that symmetric matrix have to be real.
So all real c. If c was i, the matrix wouldn't have been symmetric.
But if c is a real number, then we've got real eigenvalues, we've got orthogonal eigenvectors, that matrix is symmetric.
OK, positive definite.
OK, now this is a sub-case of symmetric, so we need c to be real, so we've got a symmetric matrix, but we also want the thing to be positive definite.
Now, we're looking at eigenvalues, we've got a lot of tests for positive definite, but eigenvalues, if we know them, is certainly a good, quick, clean test.
Could this matrix be positive definite?
No.
No, because it's got an eigenvalue zero.
It could be positive semi-definite, you know, like consolation prize, if c was greater or equal to zero, it would be positive semi-definite.
But it's not, no.
Semi-definite, if I put that comment in, semi-definite, that the condition would be c greater or equal to zero.
That would be all right.
OK. Next part.
Is it a Markov matrix?
Hm.
Could this matrix be, if I choose the number c correctly, a Markov matrix?
Well, what do we know about Markov matrices?
Mainly, we know something about their eigenvalues.
One eigenvalue is always one, and the other eigenvalues are smaller.
Not larger.
So an eigenvalue two can't happen.
So the answer is, no, not a ma- that's never a Markov matrix.
OK?
And finally, could one half of A be a projection matrix?
So could it- could this -- eh-eh could this be twice a projection matrix?
So let me write it this way.
Could A over two be a projection matrix?
OK, what are projection matrices?
They're real.
I mean, th- they're symmetric, so their eigenvalues are real.
But more than that, we know what those eigenvalues have to be.
What do the eigenvalues of a projection matrix have to be?
See, that any nice matrix we've got an idea about its eigenvalues.
So the eigenvalues of projection matrices are zero and one.
Zero and one, only.
Because P squared equals P, let me call this matrix P, so P squared equals P, so lambda squared equals lambda, because eigenvalues of P squared are lambda squared, and we must have that, so lambda equals zero or one.
OK. Now what value of c will work there?
So, then, there are some value that will work, and what will work?
c equals zero will work, or what else will work?
c equal to two.
Because if c is two, then when we divide by two, this Eigenvalue of two will drop to one, and so will the other one, so, or c equal to two.
OK, those are the guys that will work, and it was the fact that those eigenvectors were orthogonal, the fact that those eigenvectors were orthogonal carried us a lot of the way, here.
If they weren't orthogonal, then symmetric would have been dead, positive definite would have been dead, projection would have been dead.
But those eigenvectors were orthogonal, so it came down to the eigenvalues.
OK, that was like a chance to review a lot of this chapter.
Shall I jump to the singular value decomposition, then, as the third, topic for, for the review?
OK, so I'm going to.
jump to this.
OK.
So this is the singular value decomposition, known to everybody as the SVD.
And that's a factorization of A into orthogonal times diagonal times orthogonal.
And we always call those U and sigma and V transpose.
OK. And the key to that -- this is for every matrix, every A, every A. Rectangular, doesn't matter, whatever, has this decomposition.
So it's really important.
And the key to it is to look at things like A transpose A.
Can we remember what happens with A transpose A?
If I just transpose that I get V sigma transpose U transpose, that's multiplying A, which is U, sigma V transpose, and the result is V on the outside, s- U transpose U is the identity, because it's an orthogonal matrix.
So I'm just left with sigma transpose sigma in the middle, that's a diagonal, possibly rectangular diagonal by its transpose, so the result, this is orthogonal, diagonal, orthogonal.
So, I guess, actually, this is the SVD for A transpose A.
Here I see orthogonal, diagonal, and orthogonal.
Great.
But a little more is happening.
For A transpose A, the difference is, the orthogonal guys are the same.
It's V and V transpose.
What I seeing here?
I'm seeing the factorization for a symmetric matrix.
This thing is symmetric.
So in a symmetric case, U is the same as V. U is the same as V for this symmetric matrix, and, of course, we see it happening.
OK.
So that tells us, right away, what V is.
V is the eigenvector matrix for A transpose A. OK. Now, if you were here when I lectured about this topic, when I gave the topic on singular value decompositions, you'll remember that I got into trouble.
I'm sorry to remember that myself, but it happened.
OK. How did it happen?
I was in great shape for a while, cruising along.
So I found the eigenvectors for A transpose A.
Good.
I found the singular values, what were they?
What were the singular values?
The singular value number i, or -- these are the guys in sigma -- this is diagonal with the number sigma in it.
This diagonal is sigma one, sigma two, up to the rank, sigma r, those are the non-zero ones.
So I found those, and what are they?
Remind me about that?
Well, here, I'm seeing them squared, so their squares are the eigenvalues of A transpose A.
Good.
So I just take the square root, if I want the eigenvalues of A transpose -- If I want the sigmas and I know these, I take the square root, the positive square root.
OK. Where did I run into trouble?
Well, then, my final step was to find U.
And I didn't read the book.
So, I did something that was practically right, but -- well, I guess practically right is not quite the same.
OK, so I thought, OK, I'll look at A A transpose.
What happened when I looked at A A transpose?
Let me just put it here, and then I can feel it.
OK, so here's A A transpose.
So that's U sigma V transpose, that's A, and then the transpose is V sigma transpose, U sigma transpose.
Fine.
And then, in the middle is the identity again, so it looks great.
U sigma sigma transpose, U transpose.
Fine.
All good, and now these columns of U are the eigenvectors, that's U is the eigenvector matrix for this guy.
That was correct, so I did that fine.
Where did something go wrong?
A sign went wrong.
A sign went wrong because -- and now -- now I see, actually, somebody told me right after class, we can't tell from this description which sign to give the eigenvectors.
If these are the eigenvectors of this matrix, well, if you give me an eigenvector and I change all its signs, we've still got another eigenvector.
So what I wasn't able to determine (and I had a fifty-fifty change and life let me down,) the signs I just happened to pick for the eigenvectors, one of them I should have reversed the sign.
So, from this, I can't tell whether the eigenvector or its negative is the right one to use in there.
So the right way to do it is to, having settled on the signs, the Vs also, I don't know which sign to choose, but I choose one.
I choose one.
And then, instead, I should have used the one that tells me what sign to choose, the rule that A times a V is sigma times the U.
So, having decided on the V, I multiply by A, I'll notice the factor sigma coming out, and there will be a unit vector there, and I now know exactly what it is, and not only up to a change of sign.
So that's the good and, of course, this is the main point about the SVD.
That's the point that we've diagonalized, that's A times the matrix of Vs equals U times the diagonal matrix of sigmas.
That's the same as that.
OK.
So that's, like, correcting the wrong sign from that earlier lecture.
And that would complete that, so that's how you would compute the SVD.
Now, on the quiz, I going to ask -- well, maybe on the final.
So we've got quiz and final ahead.
Sometimes, you might be asked to find the SVD if I give you the matrix -- let me come back, now, to the main board -- or, I might give you the pieces.
And I might ask you something about the matrix.
For example, suppose I ask you, oh, let's say, if I tell you what sigma is -- OK. Let's take one example.
Suppose sigma is -- so all that's how we would compute them.
But now, suppose I give you these.
Suppose I give you sigma is, say, three two.
And I tell you that U has a couple of columns, and V has a couple of columns.
OK. Those are orthogonal columns, of course, because U and V are orthogonal.
I'm just sort of, like, getting you to think about the SVD, because we only had that one lecture about it, and one homework, and, what kind of a matrix have I got here?
What do I know about this matrix?
All I really know right now is that its singular values, those sigmas are three and two, and the only thing interesting that I can see in that is that they're not zero.
I know that this matrix is non-singular, right?
That's invertible, I don't have any zero eigenvalues, and zero singular values, that's invertible, there's a typical SVD for a nice two-by-two non-singular invertible good matrix.
If I actually gave you a matrix, then you'd have to find the Us and the Vs as we just spoke.
But, there.
Now, what if the two wasn't a two but it was -- well, let me make an extreme case, here -- suppose it was minus five.
That's wrong, right away.
That's not a singular value decomposition, right?
The singular values are not negative.
So that's not a singular value decomposition, and forget it.
OK.
So let me ask you about that one.
What can you tell me about that matrix?
It's singular, right?
It's got a singular matrix there in the middle, and, let's see, so, OK, it's singular, maybe you can tell me, its rank?
What's the rank of A?
It's clearly -- somebody just say it -- one, thanks.
The rank is one, so the null space, what's the dimension of the null space?
One.
Right?
We've got a two-by-two matrix of rank one, so of all that stuff from the beginning of the course is still with us.
The dimensions of those fundamental spaces is still central, and a basis for them.
Now, can you tell me a vector that's in the null space?
And then that will be my last point to make about the SVD.
Can you tell me a vector that's in the null space?
So what would I multiply by and get zero, here?
I think the answer is probably v2.
I think probably v2 is in the null space, because I think that must be the eigenvector going with this zero eigenvalue.
Yes.
Have a look at that.
And I could ask you the null space of A transpose.
And I could ask you the column space.
All that stuff.
Everything is sitting there in the SVD.
The SVD takes a little more time to compute, but it displays all the good stuff about a matrix.
OK. Any question about the SVD?
Let me keep going with further topics.
Now, let's see.
Similar matrices we've talked about, let me see if I've got another, -- OK.
Here's a true false, so we can do that, easily.
So.
Question, A given.
A is symmetric and orthogonal.
OK.
So beautiful matrices like that don't come along every day.
But what can we say first about its eigenvalues?
Actually, of course.
Here are our two most important classes of matrices, and we're looking at the intersection.
So those really are neat matrices, and what can you tell me about what could the possible eigenvalues be?
Eigenvalues can be what?
What do I know about the eigenvalues of a symmetric matrix?
Lambda is real.
What do I know about the eigenvalues of an orthogonal matrix?
Ha.
Maybe nothing.
But, no, that can't be.
What do I know about the eigenvalues of an orthogonal matrix?
Well, what feels right?
Basing mathematics on just a little gut instinct here, the eigenvalues of an orthogonal matrix ought to have magnitude one.
Orthogonal matrices are like rotations, they're not changing the length, so orthogonal, the eigenvalues are one.
Let me just show you why.
So the matrix, can I call it Q for orthogonal Why?
for the moment?
If I look at Q x equal lambda x, how do I see that this thing has magnitude one?
I take the length of both sides.
This is taking lengths, taking lengths, this is whatever the magnitude is times the length of x.
And what's the length of Q x if Q is an orthogonal matrix?
This is something you should know.
It's the same as the length of x. Orthogonal matrices don't change lengths.
So lambda has to be one.
Right.
OK. That's worth committing to memory, that could show up again.
OK.
So what's the answer now to this question, what can the eigenvalues be?
There's only two possibilities, and they are one and the other one, the other possibility is negative one, right, because these have the right magnitude, and they're real.
OK. TK.
true -- OK.
True or false?
A is sure to be positive definite.
Well, this is a great matrix, but is it sure to be positive definite?
No.
If it could have an eigenvalue minus one, it wouldn't be positive definite.
True or false, it has no repeated eigenvalues.
That's false, too.
In fact, it's going to have repeated eigenvalues if it's as big as three by three, one of these c- one of these, at least, will have to get repeated.
Sure.
So it's got repeated eigenvalues, but, is it diagonalizable?
It's got these many, many, repeated eigenvalues.
If it's fifty by fifty, it's certainly got a lot of repetitions.
Is it diagonalizable?
Yes.
All symmetric matrices, all orthogonal matrices can be diagonalized.
And, in fact, the eigenvectors can even be chosen orthogonal.
So it could be, sort of, like, diagonalized the best way with a Q, and not just any old S. OK. Is it non-singular?
Is a symmetric orthogonal matrix non-singular?
Orthogonal matrices are always non-singular.
Sure.
And, obviously, we don't have any zero Eigenvalues.
Is it sure to be diagonalizable?
Yes.
Now, here's a final step -- show that one-half of A plus I is A -- that is, prove one-half of A plus I is a projection matrix.
OK?
Let's see.
What do I do?
I could see two ways to do this.
I could check the properties of a projection matrix, which are what?
A projection matrix is symmetric.
Well, that's certainly symmetric, because A is.
And what's the other property?
I should square it, and hopefully get the same thing back.
So can I do that, square and see if I get the same thing back?
So if I square it, I'll get one-quarter of A squared plus two A plus I, right?
And the question is, does that agree with p- the thing itself?
One-half A plus I. Hm.
I guess I'd like to know something about A squared.
What is A squared?
That's our problem.
What is A squared?
If A is symmetric and orthogonal, A is symmetric and orthogonal.
This is what we're given, right?
It's symmetric, and it's orthogonal.
So what's A squared?
I.
A squared is I, because A times A -- if A equals its own inverse, so A times A is the same as A times A inverse, which is I.
So this A squared here is I.
And now we've got it.
We've got two identities over four, that's good, and we've got two As over four, that's good.
OK.
So it turned out to be a projection matrix safely.
And we could also have said, well, what are the eigenvalues of this thing?
What are the eigenvalues of a half A plus I?
If the eigenvalues of A are one and minus one, what are the eigenvalues of A plus I?
Just stay with it these last thirty seconds here.
What if I know these eigenvalues of A, and I add the identity, the eigenvalues of A plus I are zero and two.
And then when I divide by two, the eigenvalues are zero and one.
So it's symmetric, it's got the right eigenvalues, it's a projection matrix.
OK, you're seeing a lot of stuff about eigenvalues, and special matrices, and that's what the quiz is about.
OK, so good luck on the quiz.
Yes, OK, four, three, two, one, OK, I see you guys are in a happy mood.
I don't know if that means 18.06 is ending, or, the quiz was good.
Uh, my birthday conference was going on at the time of the quiz, and in the conference, of course, everybody had to say nice things, but I was wondering, what would my 18.06 class be saying, because it was at the exactly the same time.
But, what I know from the grades so far, they're basically close to, and maybe slightly above the grades that you got on quiz two.
So, very satisfactory.
And, then we have a final exam coming up, and today's lecture, as I told you by email, will be a first step in the review, and then on Wednesday I'll do all I can in reviewing the whole course.
So my topic today is -- actually, this is a lecture I have never given before in this way, and it will -- well, four subspaces, that's certainly fundamental, and you know that, so I want to speak about left-inverses and right-inverses and then something called pseudo-inverses.
And pseudo-inverses, let me say right away, that comes in near the end of chapter seven, and that would not be expected on the final.
But you'll see that what I'm talking about is really the basic stuff that, for an m-by-n matrix of rank r, we're going back to the most fundamental picture in linear algebra.
Nobody could forget that picture, right?
When you're my age, even, you'll remember the row space, and the null space.
Orthogonal complements over there, the column space and the null space of A transpose column, orthogonal complements over here.
And I want to speak about inverses.
OK. And I want to identify the different possibilities.
So first of all, when does a matrix have a just a perfect inverse, two-sided, you know, so the two-sided inverse is what we just call inverse, right?
And, so that means that there's a matrix that produces the identity, whether we write it on the left or on the right.
And just tell me, how are the numbers r, the rank, n the number of columns, m the number of rows, how are those numbers related when we have an invertible matrix?
So this is the matrix which was -- chapter two was all about matrices like this, the beginning of the course, what was the relation of th- of r, m, and n, for the nice case?
They're all the same, all equal.
So this is the case when r=m=n.
Square matrix, full rank, period, just -- so I'll use the words full rank.
OK, good.
Everybody knows that.
OK. Then chapter three.
We began to deal with matrices that were not of full rank, and they could have any rank, and we learned what the rank was.
And then we focused, if you remember on some cases like full column rank.
Now, can you remember what was the deal with full column rank?
So, now, I think this is the case in which we have a left-inverse, and I'll try to find it.
So we have a -- what was the situation there?
It's the case of full column rank, and that means -- what does that mean about r?
It equals, what's the deal with r, now, if we have full column rank, I mean the columns are independent, but maybe not the rows.
So what is r equal to in this case?
n. Thanks.
n. r=n.
The n columns are independent, but probably, we have more rows.
What's the picture, and then what's the null space for this?
So the n columns are independent, what's the null space in this case?
So of course, you know what I'm asking.
You're saying, why is this guy asking something, I know that-- I think about it in my sleep, right?
So the null space of this matrix if the rank is n, the null space is what vectors are in the null space?
Just the zero vector.
Right?
The columns are independent.
Independent columns.
No combination of the columns gives zero except that one.
And what's my picture over, -- let me redraw my picture -- the row space is everything.
No.
Is that right?
Let's see, I often get these turned around, right?
So what's the deal?
The columns are independent, right?
So the rank should be the full number of columns, so what does that tell us?
There's no null space, right.
OK.
The row space is the whole thing.
Yes, I won't even draw the picture.
And what was the deal with -- and these were very important in least squares problems because -- So, what more is true here?
If we have full column rank, the null space is zero, we have independent columns, the unique -- so we have zero or one solutions to Ax=b.
There may not be any solutions, but if there's a solution, there's only one solution because other solutions are found by adding on stuff from the null space, and there's nobody there to add on.
So the particular solution is the solution, if there is a particular solution.
But of course, the rows might not be - are probably not independent -- and therefore, so right-hand sides won't end up with a zero equal zero after elimination, so sometimes we may have no solution, or one solution.
OK. And what I want to say is that for this matrix A -- oh, yes, tell me something about A transpose A in this case.
So this whole part of the board, now, is devoted to this case.
What's the deal with A transpose A?
I've emphasized over and over how important that combination is, for a rectangular matrix, A transpose A is the good thing to look at, and if the rank is n, if the null space has only zero in it, then the same is true of A transpose A.
That's the beautiful fact, that if the rank of A is n, well, we know this will be an n by n symmetric matrix, and it will be full rank.
So this is invertible.
This matrix is invertible.
That matrix is invertible.
And now I want to show you that A itself has a one-sided inverse.
Here it is.
The inverse of that, which exists, times A transpose, there is a one-sided -- shall I call it A inverse?
-- left of the matrix A.
Why do I say that?
Because if I multiply this guy by A, what do I get?
What does that multiplication give?
Of course, you know it instantly, because I just put the parentheses there, I have A transpose A inverse times A transpose A so, of course, it's the identity.
So it's a left inverse.
And this was the totally crucial case for least squares, because you remember that least squares, the central equation of least squares had this matrix, A transpose A, as its coefficient matrix.
And in the case of full column rank, that matrix is invertible, and we're go.
So that's the case where there is a left-inverse.
So A does whatever it does, we can find a matrix that brings it back to the identity.
Now, is it true that, in the other order -- so A inverse left times A is the identity.
Right?
This matrix is m by n. This matrix is n by m. The identity matrix is n by n. All good.
All good if you're n. But if you try to put that matrix on the other side, it would fail.
If the full column rank -- if this is smaller than m, the case where they're equals is the beautiful case, but that's all set.
Now, we're looking at the case where the columns are independent but the rows are not.
So this is invertible, but what matrix is not invertible?
A A transpose is bad for this case.
A transpose A is good.
So we can multiply on the left, everything good, we get the left inverse.
But it would not be a two-sided inverse.
A rectangular matrix can't have a two-sided inverse, because there's got to be some null space, right?
If I have a matrix that's rectangular, then either that matrix or its transpose has some null space, because if n and m are different, then there's going to be some free variables around, and we'll have some null space in that direction.
OK, tell me the corresponding picture for the opposite case.
So now I'm going to ask you about right-inverses.
A right-inverse.
And you can fill this all out, this is going to be the case of full row rank.
And then r is equal to m, now, the m rows are independent, but the columns are not.
So what's the deal on that?
Well, just exactly the flip of this one.
The null space of A transpose contains only zero, because there are no combinations of the rows that give the zero row.
We have independent rows.
And in a minute, I'll give an example of all these.
So, how many solutions to Ax=b in this case?
The rows are independent.
So we can always solve Ax=b.
Whenever elimination never produces a zero row, so we never get into that zero equal one problem, so Ax=b always has a solution, but too many.
So there will be some null space, the null space of A -- what will be the dimension of A's null space?
How many free variables have we got?
How many special solutions in that null space have we got?
So how many free variables in this setup?
We've got n columns, so n variables, and this tells us how many are pivot variables, that tells us how many pivots there are, so there are n-m free variables.
So there are infinitely many solutions to Ax=b.
We have n-m free variables in this case.
OK. Now I wanted to ask about this idea of a right-inverse.
OK.
So I'm going to have a matrix A, my matrix A, and now there's going to be some inverse on the right that will give the identity matrix.
So it will be A times A inverse on the right, will be I.
And can you tell me what, just by comparing with what we had up there, what will be the right-inverse, we even have a formula for it.
There will be other -- actually, there are other left-inverses, that's our favorite.
There will be other right-inverses, but tell me our favorite here, what's the nice right-inverse?
The nice right-inverse will be, well, there we had A transpose A was good, now it will be A A transpose that's good.
The good matrix, the good right -- the thing we can invert is A A transpose, so now if I just do it that way, there sits the right-inverse.
You see how completely parallel it is to the one above?
Right.
So that's the right-inverse.
So that's the case when there is -- In terms of this picture, tell me what the null spaces are like so far for these three cases.
What about case one, where we had a two-sided inverse, full rank, everything great.
The null spaces were, like, gone, right?
The null spaces were just the zero vectors.
Then I took case two, this null space was gone.
Case three, this null space was gone, and then case four is, like, the most general case when this picture is all there -- when all the null spaces -- this has dimension r, of course, this has dimension n-r, this has dimension r, this has dimension m-r, and the final case will be when r is smaller than m and n. But can I just, before I leave here look a little more at this one?
At this case of full column rank?
So A inverse on the left, it has this left-inverse to give the identity.
I said if we multiply it in the other order, we wouldn't get the identity.
But then I just realized that I should ask you, what do we get?
So if I put them in the other order -- if I continue this down below, but I write A times A inverse left -- so there's A times the left-inverse, but it's not on the left any more.
So it's not going to come out perfectly.
But everybody in this room ought to recognize that matrix, right?
Let's see, is that the guy we know?
Am I OK, here?
What is that matrix?
P. Thanks.
P. That matrix -- it's a projection.
It's the projection onto the column space.
It's trying to be the identity matrix, right?
A projection matrix tries to be the identity matrix, but you've given it, an impossible job.
So it's the identity matrix where it can be, and elsewhere, it's the zero matrix.
So this is P, right.
A projection onto the column space.
And if I asked you this one, and put these in the opposite OK. order -- so this came from up here.
And similarly, if I try to put the right inverse on the left -- so that, like, came from above.
This, coming from this side, what happens if I try to put the right inverse on the left?
Then I would have A transpose A, A transpose inverse A, if this matrix is now on the left, what do you figure that matrix is?
It's going to be a projection, too, right?
It looks very much like this guy, except the only difference is, A and A transpose have been reversed.
So this is a projection, this is another projection, onto the row space.
Again, it's trying to be the identity, but there's only so much the matrix can do.
And this is the projection onto the column space.
So let me now go back to the main picture and tell you about the general case, the pseudo-inverse.
These are cases we know.
So this was important review.
You've got to know the business about these ranks, and the free variables -- really, this is linear algebra coming together.
And, you know, one nice thing about teaching 18.06, It's not trivial.
But it's -- I don't know, somehow, it's nice when it comes out right.
I mean -- well, I shouldn't say anything bad about calculus, but I will.
I mean, like, you know, you have formulas for surface area, and other awful things and, you know, they do their best in calculus, but it's not elegant.
And, linear algebra just is -- well, you know, linear algebra is about the nice part of calculus, where everything's, like, flat, and, the formulas come out right.
And you can go into high dimensions where, in calculus, you're trying to visualize these things, well, two or three dimensions is kind of the limit.
But here, we don't -- you know, I've stopped doing two-by-twos, I'm just talking about the general case.
OK, now I really will speak about the general case here.
What could be the inverse -- what's a kind of reasonable inverse for a matrix for the completely general matrix where there's a rank r, but it's smaller than n, so there's some null space left, and it's smaller than m, so a transpose has some null space, and it's those null spaces that are screwing up inverses, right?
Because if a matrix takes a vector to zero, well, there's no way an inverse can, like, bring it back to life.
My topic is now the pseudo-inverse, and let's just by a picture, see what's the best inverse we could have?
So, here's a vector x in the row space.
I multiply by A.
Now, the one thing everybody knows is you take a vector, you multiply by A, and you get an output, and where is that output?
Where is Ax?
Always in the column space, right?
Ax is a combination of the columns.
So Ax is somewhere here.
So I could take all the vectors in the row space.
I could multiply them all by A. I would get a bunch of vectors in the column space and what I think is, I'd get all the vectors in the column space just right.
I think that this connection between an x in the row space and an Ax in the column space, this is one-to-one.
We got a chance, because they have the same dimension.
That's an r-dimensional space, and that's an r-dimensional space.
And somehow, the matrix A -- it's got these null spaces hanging around, where it's knocking vectors to And then it's got all the vectors in between, zero.
which is almost all vectors.
Almost all vectors have a row space component and a null space component.
And it's killing the null space component.
But if I look at the vectors that are in the row space, with no null space component, just in the row space, then they all go into the column space, so if I put another vector, let's say, y, in the row space, I positive that wherever Ay is, it won't hit Ax.
Do you see what I'm saying?
Let's see why.
All right.
So here's what I said.
If x and y are in the row space, then A x is not the same as A y.
They're both in the column space, of course, but they're different.
That would be a perfect question on a final exam, because that's what I'm teaching you in that material of chapter three and chapter four, especially chapter three.
If x and y are in the row space, then Ax is different from Ay.
So what this means -- and we'll see why -- is that, in words, from the row space to the column space, A is perfect, it's an invertible matrix.
If we, like, limited it to those spaces.
And then, its inverse will be what I'll call the pseudo-inverse.
So that's that the pseudo-inverse is.
It's the inverse -- so A goes this way, from x to y -- sorry, x to A x, from y to A y, that's A, going that way.
Then in the other direction, anything in the column space comes from somebody in the row space, and the reverse there is what I'll call the pseudo-inverse, and the accepted notation is A plus.
So y will be A plus x. I'm sorry.
No, y will be A plus times whatever it started with, A y.
Do you see my picture there?
Same, of course, for x and A x.
This way, A does it, the other way is the pseudo-inverse, and the pseudo-inverse just kills this stuff, and the matrix just kills this stuff.
So everything that's really serious here is going on in the row space and the column space, and now, tell me -- this is the fundamental fact, that between those two r-dimensional spaces, our matrix is perfect.
Why?
Suppose they weren't.
Why do I get into trouble?
Suppose -- so, proof.
I haven't written down proof very much, but I'm going to use that word once.
Suppose they were the same.
Suppose these are supposed to be two different vectors.
Maybe I'd better make the statement correctly.
If x and y are different vectors in the row space -- maybe I'll better put if x is different from y, both in the row space -- so I'm starting with two different vectors in the row space, I'm multiplying by A -- so these guys are in the column space, everybody knows that, and the point is, they're different over there.
So, suppose they weren't.
Suppose A x=A y.
Suppose, well, that's the same as saying A(x-y) is zero.
So what?
So, what do I know now about (x-y), what do I know about this vector?
Well, I can see right away, what space is it in?
It's sitting in the null space, right?
So it's in the null space.
But what else do I know about it?
Here it was x in the row space, y in the row space, what about x-y?
It's also in the row space, right?
Heck, that thing is a vector space, and if the vector space is anything at all, if x is in the row space, and y is in the row space, then the difference is also, so it's also in the row space.
So what?
Now I've got a vector x-y that's in the null space, and that's also in the row space, so what vector is it?
It's the zero vector.
So I would conclude from that that x-y had to be the zero vector, x-y, so, in other words, if I start from two different vectors, I get two different vectors.
If these vectors are the same, then those vectors had to be the same.
That's like the algebra proof, which we understand completely because we really understand these subspaces of what I said in words, that a matrix A is really a nice, invertible mapping from row space to columns pace.
If the null spaces keep out of the way, then we have an inverse.
And that inverse is called the pseudo inverse, and it's a very, very, useful in application.
Statisticians discovered, oh boy, this is the thing that we needed all our lives, and here it finally showed up, the pseudo-inverse is the right thing.
Why do statisticians need it?
And because statisticians are like least-squares-happy.
I mean they're always doing least squares.
And so this is their central linear regression.
Statisticians who may watch this on video, please forgive that description of your interests.
One of your interests is linear regression and this problem.
But this problem is only OK provided we have full column rank.
And statisticians have to worry all the time about, oh, God, maybe we just repeated an experiment.
You know, you're taking all these measurements, maybe you just repeat them a few times.
You know, maybe they're not independent.
Well, in that case, that A transpose A matrix that they depend on becomes singular.
So then that's when they needed the pseudo-inverse, it just arrived at the right moment, and it's the right quantity.
OK.
So now that you know what the pseudo-inverse should do, let me see what it is.
Can we find it?
So this is my -- to complete the lecture is -- how do I find this pseudo-inverse A plus?
OK. OK. Well, here's one way.
Everything I do today is to try to review stuff.
One way would be to start from the SVD.
The Singular Value Decomposition.
And you remember that that factored A into an orthogonal matrix times this diagonal matrix times this orthogonal matrix.
But what did that diagonal guy look like?
This diagonal guy, sigma, has some non-zeroes, and you remember, they came from A transpose A, and A A transpose, these are the good guys, and then some more zeroes, and all zeroes there, and all zeroes there.
So you can guess what the pseudo-inverse is, I just invert stuff that's nice to invert -- well, what's the pseudo-inverse of this?
That's what the problem comes down to.
What's the pseudo-inverse of this beautiful diagonal matrix?
But it's got a null space, right?
What's the rank of this matrix?
What's the rank of this diagonal matrix?
r, of course.
It's got r non-zeroes, and then it's otherwise, zip.
So it's got n columns, it's got m rows, and it's got rank r. It's the best example, the simplest example we could ever have of our general setup.
OK?
So what's the pseudo-inverse?
What's the matrix -- so I'll erase our columns, because right below it, I want to write the pseudo-inverse.
OK, you can make a pretty darn good guess.
If it was a proper diagonal matrix, invertible, if there weren't any zeroes down here, if it was sigma one to sigma n, then everybody knows what the inverse would be, the inverse would be one over sigma one, down to one over s- but of course, I'll have to stop at sigma r. And, it will be the rest, zeroes again, of course.
And now this one was m by n, and this one is meant to have a slightly different, you know, transpose shape, n by m. They both have that rank r. My idea is that the pseudo-inverse is the best -- is the closest I can come to an inverse.
So what is sigma times its pseudo-inverse?
Can you multiply sigma by its pseudo-inverse?
Multiply that by that?
What matrix do you get?
They're diagonal.
Rectangular, of course.
But of course, we're going to get ones, R ones, and all the rest, zeroes.
And the shape of that, this whole matrix will be m by m. And suppose I did it in the other order.
Suppose I did sigma plus sigma.
Why don't I do it right underneath?
in the opposite order?
See, this matrix hasn't got a left-inverse, it hasn't got a right-inverse, but every matrix has got a pseudo-inverse.
If I do it in the order sigma plus sigma, what do I get?
Square matrix, this is m by n, this is m by m, my result is going to m by m -- is going to be n by n, and what is it?
Those are diagonal matrices, it's going to be ones, and then zeroes.
It's not the same as that, it's a different size -- it's a projection.
One is a projection matrix onto the column space, and this one is the projection matrix onto the row space.
That's the best that pseudo-inverse can do.
So what the pseudo-inverse does is, if you multiply on the left, you don't get the identity, if you multiply on the right, you don't get the identity, what you get is the projection.
It brings you into the two good spaces, the row space and column space.
And it just wipes out the null space.
So that's what the pseudo-inverse of this diagonal one is, and then the pseudo-inverse of A itself -- this is perfectly invertible.
What's the inverse of V transpose?
Just another tiny bit of review.
That's an orthogonal matrix, and its inverse is V, good.
This guy has got all the trouble in it, all the null space is responsible for, so it doesn't have a true inverse, it has a pseudo-inverse, and then the inverse of U is U transpose, thanks.
Or, of course, I could write U inverse.
So, that's the question of, how do you find the pseudo-inverse -- so what statisticians do when they're in this -- so this is like the case of where least squares breaks down because the rank is -- you don't have full rank, and the beauty of the singular value decomposition is, it puts all the problems into this diagonal matrix where it's clear what to do.
It's the best inverse you could think of is clear.
You see there could be other -- I mean, we could put some stuff down here, it would multiply these zeroes.
It wouldn't have any effect, but then the good pseudo-inverse is the one with no extra stuff, it's sort of, like, as small as possible.
It has to have those to produce the ones.
If it had other stuff, it would just be a larger matrix, so this pseudo-inverse is kind of the minimal matrix that gives the best result.
Sigma sigma plus being r ones.
SK.
so I guess I'm hoping -- pseudo-inverse, again, let me repeat what I said at the very beginning.
This pseudo-inverse, which appears at the end, which is in section seven point four, and probably I did more with it here than I did in the book.
The word pseudo-inverse will not appear on an exam in this course, but I think if you see this all will appear, because this is all what the course was about, chapters one, two, three, four -- but if you see all that, then you probably see, well, OK, the general case had both null spaces around, and this is the natural thing to do.
So, this is one way to find the pseudo-inverse.
Yes.
The point of a pseudo-inverse, of computing a pseudo-inverse is to get some factors where you can find the pseudo-inverse quickly.
And this is, like, the champion, because this is where we can invert those, and those two, easily, just by transposing, and we know what to do with a diagonal.
OK, that's as much review, maybe -- let's have a five-minute holiday in 18.06 and, I'll see you Wednesday, then, for the rest of this course.
Thanks.
OK. Good.
The final class in linear algebra at MIT this Fall is to review the whole course.
And, you know the best way I know how to review is to take old exams and just think through the problems.
So it will be a three-hour exam next Thursday.
Nobody will be able to take an exam before Thursday, anybody who needs to take it in some different way after Thursday should see me next Monday.
I'll be in my office Monday.
OK. May I just read out some problems and, let me bring the board down, and let's start.
OK.
Here's a question.
This is about a 3-by-n matrix.
And we're given -- so we're given -- given -- A x equals 1 0 0 has no solution.
And we're also given A x equals 0 1 0 has exactly one solution.
OK.
So you can probably anticipate my first question, what can you tell me about m?
It's an m-by-n matrix of rank r, as always, what can you tell me about those three numbers?
So what can you tell me about m, the number of rows, n, the number of columns, and r, the rank?
OK. See, do you want to tell me first what m is?
How many rows in this matrix?
Must be three, right?
We can't tell what n is, but we can certainly tell that m is three.
OK. And, what do these things tell us?
Let's take them one at a time.
When I discover that some equation has no solution, that there's some right-hand side with no answer, what does that tell me about the rank of the matrix?
It's smaller m. Is that right?
If there is no solution, that tells me that some rows of the matrix are combinations of other rows.
Because if I had a pivot in every row, then I would certainly be able to solve the system.
I would have particular solutions and all the good So any time that there's a system with no solutions, stuff.
that tells me that r must be below m. What about the fact that if, when there is a solution, there's only one?
What does that tell me?
Well, normally there would be one solution, and then we could add in anything in the null space.
So this is telling me the null space only has the 0 vector in it.
There's just one solution, period, so what does that tell me?
The null space has only the zero vector in it?
What does that tell me about the relation of r to n?
So this one solution only, that means the null space of the matrix must be just the zero vector, and what does that tell me about r and n?
They're equal.
The columns are independent.
So I've got, now, r equals n, and r less than m, and now I also know m is three.
So those are really the facts I know.
n=r and those numbers are smaller than three.
Sorry, yes, yes.
r is smaller than m, and n, of course, is also.
So I guess this summarizes what we can tell.
In fact, why not give me a matrix -- because I would often ask for an example of such a matrix -- can you give me a matrix A that's an example?
That shows this possibility?
Exactly, that there's no solution with that right-hand side, but there's exactly one solution with this right-hand side.
Anybody want to suggest a matrix that does that?
Let's see.
What do I -- what vector do I want in the column space?
I want zero, one, zero, to be in the column space, because I'm able to solve for that.
So let's put zero, one, zero in the column space.
Actually, I could stop right there.
That would be a matrix with m equal three, three rows, and n and r are both one, rank one, one column, and, of course, there's no solution to that one.
So that's perfectly good as it is.
Or if you, kind of, have a prejudice against matrices that only have one column, I'll accept a second column.
So what could I include as a second column that would just be a different answer but equally good?
I could put this vector in the column space, too, if I wanted.
That would now be a case with r=n=2, but, of course, three m eq- m is still three, and this vector is not in the column space.
So you're -- this is just like prompting us to remember all those things, column space, null space, all that stuff.
Now, I probably asked a second question about this type of thing.
OK. Oh, I even asked, write down an example of a Ah.
matrix that fits the description.
Hm.
I guess I haven't learned anything in twenty-six years.
CK.
Cross out all statements that are false about any matrix with these -- so again, these are -- this is the preliminary sta- these are the facts about my matrix, this is one example.
But, of course, by having an example, it will be easy to check some of these facts, or non-facts.
Let me, let me write down some, facts.
Some possible facts.
So this is really true or false.
The determinant -- this is part one, the determinant of A transpose A is the same as the determinant of A A transpose.
Is that true or not?
Second one, A transpose A, is invertible.
Is invertible.
Third possible fact, A A transpose is positive definite.
So you see how, on an exam question, I try to connect the different parts of the course.
So, well, I mean, the simplest way would be to try it with that matrix as a good example, but maybe we can answer, even directly.
Let me take number two first.
Because I'm -- you know, I'm very, very fond of that matrix, A transpose A.
And when is it invertible?
When is the matrix A transpose A, invertible?
The great thing is that I can tell from the rank of A that I don't have to multiply out A transpose A.
A transpose A, is invertible -- well, if A has a null space other than the zero vector, then it -- it's -- no way it's going to be invertible.
But the beauty is, if the null space of A is just the zero vector, so the fact -- the key fact is, this is invertible if r=n, by which I mean, independent columns of A.
In A.
In the matrix A.
If r=n -- if the matrix A has independent columns, then this combination, A transpose A, is square and still that same null space, only the zero vector, independent columns all good, and so, what's the true/false?
Is it -- is this middle one T or F for this, in this setup?
Well, we discovered that -- we discovered that -- that r was n, from that second fact.
So this is a true.
That's a true.
And, of course, A transpose A, in this example, would probably be -- what would A transpose A, be, for that matrix?
Can you multiply A transpose A, and see what it looks like for that matrix?
What shape would it be?
It will be two by two.
And what matrix will it be?
The identity.
So, it checks out.
OK, what about A A transpose?
Well, depending on the shape of A, it could be good or not so good.
It's always symmetric, it's always square, but what's the size, now?
This is three by n, and this is n by three, so the result is three by three.
Is it positive definite?
I don't think so.
False.
If I multiply that by A transpose, A A transpose, what would the rank be?
It would be the same as the rank of A, that's -- it would be just rank two.
And if it's three-by-three, and it's only rank two, it's certainly not positive definite.
So what could I say about A A transpose, if I wanted to, like, say something true about it?
It's true that it is positive semi-definite.
If I made this semi-definite, it would always be true, always.
But if I'm looking for positive definite, then I'm looking at the null space of whatever's here, and, in this case, it's got a null space.
So A, A -- eh, shall we just figure it out, here?
A A transpose, for that matrix, will be three-by-three.
If I multiplied A by A transpose, what would the first row be?
All zeroes, right?
First row of A A transpose, could only be all zeroes, so it's probably a one there and a one there, or something like that.
But, I don't even know if that's right.
But it's all zeroes there, so it's certainly not positive definite.
Let me not put anything up I'm not sh- don't check.
What about this determinant?
Oh, well, I guess -- that's a sort of tricky question.
Is it true or false in this case?
It's false, apparently, because A transpose A, is invertible, we just got a true for this one, and we got a false, we got a z- we got a non-invertible one for this one.
So actually, this one is false, number one.
That surprises us, actually, because it's, I mean, why was it tricky?
Because what is true about determinants?
This would be true if those matrices were square.
If I have two square matrices, A and any other matrix B, could be A transpose, could be somebody else's matrix.
Then it would be true that the determinant of B A would equal the determinant of A B.
But if the matrices are not square and it would actually be true that it would be equal -- that this would equal the determinant of A times the determinant of A transpose.
We could even split up those two separate determinants.
And, of course, those would be equal.
But only when A is square.
So that's just, that's a question that rests on the, the falseness rests on the fact that the matrix isn't square in the first place.
OK, good.
Let's see.
Oh, now, even asks more.
Prove that A transpose y equals c -- hah-God, it's -- this question goes on and on.
now I ask you about A transpose y=c.
So I'm asking you about the equation -- about the matrix A transpose.
And I want you to prove that it has at least one solution -- one solution for every c, every right-hand side c, and, in fact -- in fact, infinitely many solutions for every c. OK. Well, none -- none of this is difficult, but, it's been a little while.
So we just have to think again.
When I have a system of equations -- this is -- this matrix A transpose is now, instead of being three by n, it's n by three, it's n by m. Of course.
To show that a system has at least one solution, when does this, when does this system -- when is the system always solvable?
When it has full row rank, when the rows are independent.
Here, we have n rows, and that's the rank.
So at least one solution, because the number of rows, which is n, for the transpose, is equal to r, the rank.
This A transpose had independent rows because A had independent columns, right?
The original A had independent columns, when we transpose it, it has independent rows, so there's at least one solution.
But now, how do I even know that there are infinitely many solutions?
Oh, what do I -- I want to know something about the null space.
What's the dimension of the null space of A transpose?
So the answer has got to be the dimension of the null space of A transpose, what's the general fact?
If A is an m by n matrix of rank r, what's the dimension of A transpose?
The null space of A transpose?
Do you remember that little fourth subspace that's tagging along down in our big picture?
It's dimension was m-r. And, that's bigger than zero.
m is bigger than r. So there's a lot in that null space.
So there's always one solution because n i- this is speaking about A transpose.
So for A transpose, the roles of m and n are reversed, of course, so I'm -- keep in mind that this board was about A transpose, so the roles -- so it's the null space of a transpose, and there are m-r free variables.
OK, that's, like, just some, review.
Can I take another problem that's also sort of -- suppose the matrix A has three columns, v1, v2, v3.
Those are the columns of the matrix.
All right.
Question A.
Solve Ax=v1-v2+v3.
Tell me what x is.
Well, there, you're seeing the most -- the one absolutely essential fact about matrix multiplication, how does it work, when we do it a column at a time, the very, very first day, way back in September, we did multiplication a column at a time.
So what's x?
Just tell me?
One minus one, one.
Thanks.
OK. Everybody's got that.
OK?
Then the next question is, suppose that combination is zero -- oh, yes, OK, so question (b) says -- part (b) says, suppose this thing is zero.
Suppose that's zero.
Then the solution is not unique.
Suppose I want true or false.
-- and a reason.
Suppose this combination is zero.
v1-v2+v3.
Show that -- what does that tell me?
So it's a separate question, maybe I sort of saved time by writing it that way, but it's a totally separate question.
If I have a matrix, and I know that column one minus column two plus column three is zero, what does that tell me about whether the solution is unique or not?
Is there more than one solution?
What's uniqueness about?
Uniqueness is about, is there anything in the null space, right?
The solution is unique when there's nobody in the null space except the zero vector.
And, if that's zero, then this guy would be in the null space.
So if this were zero, then this x is in the null space of A.
So solutions are never unique, because I could always add that to any solution, and Ax wouldn't change.
So it's always that question.
Is there somebody in the null space?
OK. Oh, now, here's a totally different question.
Suppose those three vectors, v1, v2, v3, are orthonormal.
So this isn't going to happen for orthonormal vectors.
OK, so part (c), forget part (b).
c. If v1, v2, v3, are orthonormal -- so that I would usually have called them q1, q2, q3.
Now, what combination -- oh, here's a nice question, if I say so myself -- what combination of v1 and v2 is closest to v3?
What point on the plane of v1 and v2 is the closest point to v3 if these vectors are orthonormal?
So let me -- I'll start the sentence -- then the combination something times v1 plus something times v2 is the closest combination to v3?
And what's the answer?
What's the closest vector on that plane to v3?
Zeroes.
Right.
We just imagine the x, y, z axes, the v1, v2, th- v3 could be the standard basis, the x, y, z vectors, and, of course, the point on the xy plane that's closest to v3 on the z axis is zero.
So if we're orthonormal, then the projection of v3 onto that plane is perpendicular, it hits right at zero.
OK, so that's like a quick -- you know, an easy question, but still brings it out.
OK. Let me see what, shall I write down a Markov matrix, and I'll ask you for its eigenvalues.
OK.
Here's a Markov matrix -- this -- and, tell me its eigenvalues.
So here -- I'll call the matrix A, and I'll call this as point two, point four, point four, point four, point four, point two, point four, point three, point three, point four.
OK. Let's see -- it helps out to notice that column one plus column two -- what's interesting about column one plus column two?
It's twice as much as column three.
So column one plus column two equals two times column three.
I put that in there, column one plus column two equals twice column three.
That's observation.
OK. Tell me the eigenvalues of the matrix.
OK, tell me one eigenvalue?
Because the matrix is singular.
Tell me another eigenvalue?
One, because it's a Markov matrix, the columns add to the all ones vector, and that will be an eigenvector of A transpose.
And tell me the third eigenvalue?
Let's see, to make the trace come out right, which is point eight, we need minus point two.
OK. And now, suppose I start the Markov process.
Suppose I start with u(0) -- so I'm going to look at the powers of A applied to u(0).
This is uk.
And there's my matrix, and I'm going to let u(0) be -- this is going to be zero, ten, zero.
And my question is, what does that approach?
If u(0) is equal to this -- there is u(0).
Shall I write it in?
Maybe I'll just write in u(0).
A to the k, starting with ten people in state two, and every step follows the Markov rule, what does the solution look like after k steps?
Let me just ask you that.
And then, what happens as k goes to infinity?
This is a steady-state question, right?
I'm looking for the steady state.
Actually, the question doesn't ask for the k step answer, it just jumps right away to infinity -- but how would I express the solution after k steps?
It would be some multiple of the first eigenvalue to the k-th power -- times the first eigenvector, plus some other multiple of the second eigenvalue, times its eigenvector, and some multiple of the third eigenvalue, times its eigenvector.
OK. Good.
And these eigenvalues are zero, one, and minus point two.
So what happens as k goes to infinity?
The only thing that survives the steady state -- so at u infinity, this is gone, this is gone, all that's left is c2x2.
So I'd better find x2.
I've got to find that eigenvector to complete the answer.
What's the eigenvector that corresponds to lambda equal one?
That's the key eigenvector in any Markov process, is that eigenvector.
Lambda equal one is an eigenvalue, I need its eigenvector x2, and then I need to know how much of it is in the starting vector u0.
OK.
So, how do I find that eigenvector?
I guess I subtract one from the diagonal, right?
So I have minus point eight, minus point eight, minus point six, and the rest, of course, is just -- still point four, point four, point four, point four, point three, point three, and hopefully, that's a singular matrix, so I'm looking to solve A minus Ix equal zero.
Let's see -- can anybody spot the solution here?
I don't know, I didn't make it easy for myself.
What do you think there?
Maybe those first two entries might be -- oh, no, what do you think?
Anybody see it?
We could use elimination if we were desperate.
Are we that desperate?
Anybody just call out if you see the vector that's in that null space.
Eh, there better be a vector in that null space, or I'm quitting.
Uh, ha- OK, well, I guess we could use elimination.
I thought maybe somebody might see it from further away.
Is there a chance that these guys are -- could it be that these two are equal and this is whatever it takes, like, something like three, three, two?
Would that possibly work?
I mean, that's great for this -- no, it's not that great.
Three, three, four -- this is, deeper mathematics you're watching now.
Three, three, four, is that -- it works!
Don't mess with it!
It works!
Uh, yes.
OK, it works, all right.
And, yes, OK, and, so that's x2, three, three, four, and, how much of that vector is in the starting vector?
Well, we could go through a complicated process.
But what's the beauty of Markov things?
That the total number of the total population, the sum of these doesn't change.
That the total number of people, they're moving around, but they don't get born or die or get dead.
So there's ten of them at the start, so there's ten of them there, so c2 is actually one, yes.
So that would be the correct solution.
OK. That would be the u infinity.
OK.
So I used there, in that process, sort of, the main facts about Markov matrices to, to get a jump on the answer.
OK. let's see.
OK, here's some, kind of quick, short questions.
Uh, maybe I'll move over to this board, and leave that for the moment.
I'm looking for two-by-two matrices.
And I'll read out the property I want, and you give me an example, or tell me there isn't such a matrix.
All right.
Here we go.
First -- so two-by-twos.
First, I want the projection onto the line through A equals four minus three.
So it's a one-dimensional projection matrix I'm looking for.
And what's the formula for it?
What's the formula for the projection matrix P onto a line through A.
And then we'd just plug in this particular A.
Do you remember that formula?
There's an A and an A transpose, and normally we would have an A transpose A inverse in the middle, but here we've just got numbers, so we just divide by it.
And then plug in A and we've got it.
So, equals.
OK. You can put in the numbers.
Trivial, right.
OK.
Number two.
So this is a new problem.
The matrix with eigenvalue zero and three and eigenvectors -- well, let me write these down.
eigenvalue zero, eigenvector one, two, eigenvalue three, eigenvector two, one.
I'm giving you the eigenvalues and eigenvectors instead of asking for them.
Now I'm asking for the matrix.
What's the matrix, then?
What's A?
Here was a formula, then we just put in some numbers, what's the formula here, into which we'll just put the given numbers?
It's the S lambda S inverse, right?
So it's S, which is this eigenvector matrix, it's the lambda, which is the eigenvalue matrix, it's the S inverse, whatever that turns out to be, let me just leave it as inverse.
That has to be it, right?
Because if we went in the other direction, that matrix S would diagonalize A to produce lambda.
So it's S lambda S inverse.
Good.
OK, ready for number three.
A real matrix that cannot be factored into A -- I'm looking for a matrix A that never could equal B transpose B, for any B.
A two-by-two matrix that could not be factored in the form B transpose B.
So all you have to do is think, well, what does B transpose B, look like, and then pick something different.
What do you suggest?
Let's see.
What shall we take for a matrix that could not have this form, B transpose B.
Well, what do we know about B transpose B?
It's always symmetric.
So just give me any non-symmetric matrix, it couldn't possibly have that form.
OK. And let me ask the fourth part of this question -- a matrix that has orthogonal eigenvectors, but it's not symmetric.
What matrices have orthogonal eigenvectors, but they're not symmetric matrices?
What other families of matrices have orthogonal eigenvectors?
We know symmetric matrices do, but others, also.
So I'm looking for orthogonal eigenvectors, and, what do you suggest?
The matrix could be skew-symmetric.
It could be an orthogonal matrix.
It could be symmetric, but that was too easy, so I ruled that out.
It could be skew-symmetric like one minus one, like that.
Or it could be an orthogonal matrix like cosine sine, minus sine, cosine.
All those matrices would have complex orthogonal eigenvectors.
But they would be orthogonal, and so those examples are fine.
OK. We can continue a little longer if you would like to, with these -- from this exam.
From these exams.
Least squares?
OK, here's a least squares problem in which, to make life quick, I've given the answer -- it's like Jeopardy!, right?
I just give the answer, and you give the question.
OK. Whoops, sorry.
Let's see, can I stay over here for the next question?
OK. least squares.
So I'm giving you the problem, one, one, one, zero, one, two, c d equals three, four, one, and that's b, of course, this is Ax=b.
And the least squares solution -- Maybe I put c hat d hat to emphasize it's not the true solution.
So the least square solution -- the hats really go here -- is eleven-thirds and minus one.
Of course, you could have figured that out in no time.
So this year, I'll ask you to do it, probably.
But, suppose we're given the answer, then let's just remember what happened.
OK, good question.
What's the projection P of this vector onto the column space of that matrix?
So I'll write that question down, one.
What is P?
The projection.
The projection of b onto the column space of A is what?
Hopefully, that's what the least squares problem solved.
What is it?
This was the best solution, it's eleven-thirds times column one, plus -- or rather, minus one times column two.
Right?
That's what least squares did.
It found the combination of the columns that was as close as possible to b.
That's what least squares was doing.
It found the projection.
OK?
Secondly, draw the straight line problem that corresponds to this system.
So I guess that the straight line fitting a straight line problem, we kind of recognize.
So we recognize, these are the heights, and these are the points, and so at zero, one, two, the heights are three, and at t equal to one, the height is four, one, two, three, four, and at t equal to two, the height is one.
So I'm trying to fit the best straight line through those points.
God.
I could fit a triangle very well, but, I don't even know which way the best straight line goes.
Oh, I do know how it goes, because there's the answer,yes.
It has a height eleven-thirds, and it has slope minus one, so it's something like that.
Great.
OK. Now, finally -- and this completes the course -- find a different vector b, not all zeroes, for which the least square solution would be zero.
So I want you to find a different B so that the least square solution changes to all zeroes.
So tell me what I'm really looking for here.
I'm looking for a b where the best combination of these two columns is the zero combination.
So what kind of a vector b I looking for?
I'm looking for a vector b that's orthogonal to those columns.
It's orthogonal to those columns, it's orthogonal to the column space, the best possible answer is zero.
So a vector b that's orthogonal to those columns -- let's see, maybe one of those minus two of those, and one of those?
That would be orthogonal to those columns, and the best vector would be zero, zero.
OK.
So that's as many questions as I can do in an hour, but you get three hours, and, let me just say, as I've said by e-mail, thanks very much for your patience as this series of lectures was videotaped, and, thanks for filling out these forms, maybe just leave them on the table up there as you go out -- and above all, thanks for taking the course.
Thank you.
Thanks.
I love teaching this course, and I tell the class that the first day.
But because of them, I've learned that they are really serious.
And I have something that I hope they get, which is the idea-- many of them took calculus, differential equations, survived it, didn't especially enjoy it.
And I'm always hoping that they see what math is about, and how it looks for the pattern in a whole lot of different applications.
Often engineering, but many others too.
Science, economics, biology, wherever.
And if they see the pattern, then they're really seeing what mathematics brings beyond just formulas, which is not what the subject's about, but getting answers and understanding.
So one part of the course is the modeling, getting the equations, what's the structure.
And the second part of the course is how do you solve the equations?
Sometimes analytically, sometimes numerically.
And that's, of course, the reality of being a scientist or engineer now.
So this course is three times a week.
Very typical hour's lecture.
And a fair amount of homework, including some Matlab homework, because that's the reality of how people get answers.
And also, of course, analytical, pencil and paper homework to see the central ideas and use.
And office hours from TAs.
Because a lot of people haven't had linear algebra.
They haven't had [?
18.06 ?]
when they take this course.
And so they ask, do I have to know linear algebra?
And I say, well, maybe not, but you soon will.
Well, I think it's so fantastic to be able to have OpenCourseWare and try to do something beyond MIT.
And so, what am I trying to do?
I guess, I think that the teaching of applied math has a big step forward still to take.
That we, in many situations, we're doing what we did a while ago, and not really bringing it home, making the subject what it can be.
And part of that has got to be numerical solution of the equations, and also new applications.
Well, 18.085 , or really, 18.075 has been an MIT course for a long time.
And Professor Hildebrand's textbooks were used.
He was an exceptionally good teacher.
But computing wasn't part of it, really.
And it has become part of what all users do to get answers.
So the course had to move forward, and the first step was about 20 years ago with a new textbook, a different approach.
And I have had a wonderful time teaching classes over those years, and now it's ready for the next generation.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Just to give an overview in three lines: the text is the book of that name, Computational Science and Engineering.
That was completed just last year, so it really ties pretty well with the course.
I don't cover everything in the book, by all means.
And I don't, certainly, don't stand here and read the book.
That would be no good.
But you'll be able, if you miss a class -- well, don't miss a class.
But if you miss a class, you'll be able, probably, to see roughly what we did.
OK, so the first part of the semester is applied linear algebra.
And I don't know how many of you have had a linear algebra course, and that's why I thought I would start with a quick review.
And you'll catch on.
I want matrices to come to life, actually.
You know, instead of just being a four by four array of numbers, there are four by four, or n by n or m by n array of special numbers.
They have a meaning.
When they multiply a vector, they do something.
And so it's just part of this first step is just, like, getting to recognize, what's that matrix doing?
Where does it come from?
What are its properties?
So that's a theme at the start.
Then differential equations, like Laplace's equation, are beautiful examples.
So here we get, especially, to numerical methods; finite differences, finite elements, above all.
So I think in this class you'll really see how finite elements work, and other ideas.
All sorts of ideas.
And then the last part of the course is about Fourier.
That's Fourier series, that you may have seen, and Fourier integrals.
But also, highly important, Discrete Fourier Transform, DFT.
That's a fundamental step for understanding what a signal contains.
Yeah, so that's great stuff, Fourier.
OK, what else should I say before I start?
I said this was my favorite course, and maybe I elaborate a little.
Well, I think what I want to say is that I really feel my life is here to teach you and not to grade you.
I'm not going to spend this semester worrying about grades, and please don't.
They come out fine.
We've got lots to learn.
And I'll do my very best to explain it clearly.
And I know you'll do your best.
I know from experience.
This class goes for it and does it right.
So that's what makes it so good.
OK. Homeworks, by the way, well, the first homework will simply be a way to get a grade list, a list of everybody taking the course.
They won't be graded in great detail.
Too large a class.
And you're allowed to talk to each other about homework.
So homework is not an exam at all.
So let me just leave any discussion of exams and grades for the future.
I'll tell you, you'll see how informally the first homework will be.
And I hope it'll go up on the website.
The first homework will be for Monday.
So it's a bit early, but it's pretty open-ended.
If you could take three problems from 1.1, the first section of the book, any three, and any three problems from 1.2, and print your name on the homework -- because we're going to use that to create the grade list -- I'll be completely happy.
Well, especially if you get them right and do them neatly and so on.
But actually we won't know.
So that's for Monday.
OK. And we'll talk more about it.
I'll announce the TA on the website and the TA hours, the office hours, and everything.
There'll be a Friday afternoon office hour, because homeworks will typically come Monday.
OK.
Questions about the course before I just start?
OK. Another time for questions, too.
OK, so can we just start with that matrix?
So I said about matrices, I'm interested in their properties.
Like, I'm going to ask you about that.
And then, I'm interested in their meaning.
Where do they come from?
You know, why that matrix instead of some other?
And then, the numerical part is how do we deal with them?
How do we solve a linear system with that coefficient matrix?
What can we say about the solution?
So the purpose.
Right.
OK, now help me out.
So I guess my plan with the video taping is, whatever you say, I'll repeat.
So say it as clearly as possible, and it's fantastic to have discussion, conversation here.
So I'll just repeat it so that it safely gets on the tape.
So tell me its properties.
Tell me the first property that you notice about that matrix.
Symmetric.
Symmetric.
Right.
I could have slowed down a little and everybody probably would have said that at once.
So that's a symmetric matrix.
Now we might as well pick up some matrix notation.
How do I express the fact that this a symmetric matrix?
In simple matrix notation, I would say that K is the same as K transpose.
The transpose, everybody knows, it comes from -- oh, I shouldn't say this -- flipping it across the diagonal.
That's not a very "math" thing to do.
But that's the way to visualize it.
And let me use a capital T for transpose.
So it's symmetric.
Very important.
Very, very important.
That's the most important class of matrices, symmetric matrices.
We'll see them all the time, because they come from equilibrium problems.
They come from all sorts of -- they come everywhere in applications.
And we will be doing applications.
The first week or week and a half, you'll see pretty much discussion of matrices and the reasons, what their meaning is.
And then we'll get to physical applications; mechanics and more.
OK. All right.
Now I'm looking for properties, other properties, of that matrix.
Let me write "2" here so that you got a spot to put it.
What are you going to tell me next about that matrix?
Periodic.
Well, okay.
Actually, that's a good question.
Let me write periodic down here.
You're using that word, because somehow that pattern is suggesting something.
But you'll see I have a little more to add before I would use the word periodic.
So that's great to see that here.
What else?
Somebody else was going to say something.
Please.
Sparse!
Oh, very good.
Sparse.
That's also an obvious property that you see from looking at the matrix.
What does sparse mean?
Mostly zeros.
Well that isn't mostly zeros, I guess.
I mean, that's got what, out of sixteen entries, it's got six zeros.
That doesn't sound like sparse.
But when I grow the matrix -- because this is just a four by four.
I would even call this one K_4.
When the matrix grows to 100 by 100, then you really see it as sparse.
So if that matrix was 100 by 100, how many non-zeros would it have?
So if n is 100, then the number of non-zeros -- wow, that's the first MATLAB command I've written.
A number of non-zeros of K would be -- anybody know what it would be?
I'm just asking to go up to five by five.
I'm asking you to keep that pattern alive.
Twos on the diagonal, minus ones above and below.
So yeah, so 298, would it be?
A hundred diagonal entries, 99 and 99, maybe 298?
298 out of 100 by 100 would be what?
It's been a long summer.
Yeah, a lot of zeros.
A lot.
Right.
Because the matrix has got what 100 x 100, 10,000 entries.
Out of 10,000.
So that's sparse.
But we see those all the time, and fortunately we do.
Because, of course, this matrix, or even 100 by 100, we could deal with if it was dense.
But 10,000, 100,000, or 1 million, which happens all the time now in scientific computation.
A million by million dense matrix is not a nice thing to think about.
A million by million matrix like this is a cinch.
OK.
So sparse.
What else do you want to say?
Toeplitz.
Holy Moses.
Exactly right.
But I want to say, before I use that word, so that'll be my second MATLAB command.
Thanks.
Toeplitz.
What's that mean?
So this matrix has a property that we see right away, which is?
I want to stay with Toeplitz but everybody tell me something more about properties of that matrix.
Tridiagonal.
Tridiagonal, so that's almost a special subcase of sparse.
It has just three diagonals.
Tridiagonal matrices are truly important.
They come in all the time, we'll see that they come from second order differential equations, which are, thanks to Newton, the big ones.
Ok, now it's more than tridiagonal and what more?
So what further, we're getting deeper now.
What patterns do you see beyond just tridiagonal, because tridiagonal would allow any numbers there but those are not, there's more of a pattern than just three diagonals, what is it?
Those diagonals are constant.
If I run down each of those three diagonals, I see the same number.
Twos, minus ones, minus ones, and that's what the word Toeplitz means.
Toeplitz is constant diagonal.
Ok. And that kind of matrix is so important.
It corresponds, yeah, if we were in EE, I would use the words time invariant filter, linear time invariant.
So it's linear because we're dealing with a matrix.
And it's time invariant, shift invariant.
I just use all these equivalent words to mean that we're seeing the same thing row by row, except of course, at shall I call that the boundary?
That's like, the end of the system and this is like the other end and there it's chopped off.
But if it was ten by ten I would see that row eight times.
100 by 100 I'd see it 98 times.
So it's constant diagonals and the guy who first studied that was Toeplitz.
And we wouldn't need that great historical information except that MATLAB created a command to create that matrix.
K, MATLAB is all set to create Toeplitz matrices.
Yeah, so I'll have to put what MATLAB would put.
I realize I'm already using the word MATLAB.
I think that MATLAB language is really convenient to talk about linear algebra.
And how many know MATLAB or have used it?
Yeah.
You know it better than I. I talk a good line with MATLAB but I, the code never runs.
Never!
I always forget some stupid semicolon.
You may have had that experience.
And I just want to say it now that there are other languages, and if you want to do homeworks and want to do your own work in other languages, that makes sense.
So the older established alternatives were Mathematica and Maple and those two have symbolic, they can deal with algebra as well as numbers.
But there are newer languages.
I don't know if you know them.
I just know my friends say, Yes they're terrific.
Python is one.
And R. I've just had a email saying, Tell your class about R. And others.
Ok, so but we'll use MATLAB language because that's really a good common language.
Ok, so what is a Toeplitz matrix?
A Toeplitz matrix is one with constant diagonals.
You could use the word time invariant, linear time invariant filter.
And to create K, this is an 18.085 command.
It's just set up for us.
I can create K by telling the system the first row.
Two, minus one, zero, zero.
That would, then if it wasn't symmetric I would have to give the first column also.
Toeplitz would be constant diagonal, it doesn't have to be symmetric.
But if it's symmetric, then the first row and first column are the same vector, so I just have to give that vector.
Okay, so that's the quickest way to create K. And of course, if it was bigger then I would, rather than writing 100 zeros, I could put zeros of 98 and one.
Wouldn't I have to say that?
Or is it one and 98?
You see why it doesn't run.
Well I guess I'm thinking of that as a row.
I don't know.
Anyway.
I realize getting this videotaped means I'm supposed to get things right!
Usually it's like, we'll get it right later.
But anyway, that might work.
Okay.
So there's a command that you know.
Zeros that creates a matrix of this size with all zeros.
Okay.
That would create the 100 by 100.
Good.
Ok. Oh, by the way, as long as we're speaking about computation I've gotta say something more.
We said that the matrix is sparse.
And this 100 by 100 matrix is certainly sparse.
But if I create it this way, I've created all those zeros and if I ask MATLAB to work with that matrix, to square it or whatever, it would carry all those zeros and do all those zero computations.
In other words, it would treat K like a dense matrix and it would just, it wouldn't know the zeros were there until it looked.
So I just want to say that if you have really big systems Sparse MATLAB is the way to go.
Because Sparse MATLAB keeps track only of the non-zeros.
So it knows-- and their locations, of course.
What the numbers are and their location.
So I could create a sparse matrix out of that, like KS for K sparse.
I think if I just did sparse(K) that would create a sparse matrix.
And then if I do stuff to it, MATLAB would automatically know those zeros were there and not spend it's time multiplying by zero But of course, this isn't perfect because I've created the big matrix before sparsifying it.
And better to have created it in the first place as a sparse matrix.
Ok.
So those were properties that you could see.
Now I'm looking for little deeper.
What's the first question I would ask about a matrix if I have to solve a system of equations, say KU=F or something.
I got a 4 by 4 matrix, four equations, four unknowns.
What would I want to know next?
Is it invertible?
Is the matrix invertible?
And that's an important question and how do you recognize an invertible matrix?
This one is invertible.
So let me say K is invertible.
And what does that mean?
That means that there's another matrix, K inverse such that K times K inverse is the identity matrix.
The identity matrix in MATLAB would be eye(n) and it's the diagonal matrix of one.
It's the unit matrix is the matrix that doesn't do anything to a vector.
So this K has an inverse.
But how do you know?
How can you recognize that a matrix is invertible?
Because obviously that's a critical question and many, many-- since our matrices are not-- a random matrix would be invertible, for sure, but our matrices have patterns, they're created out of a problem and the question of whether that matrix is invertible is fundamental.
I mean finite elements has these, zero energy modes that you have to watch out for because, what are they?
They produce non-invertible stiffness matrix.
Ok.
So how did we know, or how could we know that this K is invertible?
Somebody said invertible and I wrote it down.
Yeah?
Well ok. Now I get to make a speech about determinants.
Don't deal with them!
Don't touch determinants.
I mean this particular four by four happens to have a nice determinant.
I think it's five.
But if it was a 100 by 100 how would we show that the matrix was invertible?
And what I mean by this is the whole family is invertible.
All sizes are invertible.
K_ n is invertible for every n, not just this particular guy, whose determinant we could take.
But as five by five, six by six, we would be up in the-- but you're completely right.
The determinant is a test.
Alright.
But I guess I'm saying that it's not the test that I would use.
So what I do?
I would row reduce.
That's the default option in linear algebra.
If you don't know what to do with a matrix, if you want to see what's going on, row reduce.
What does that mean?
That means, shall I try it?
So let me just start it just so I'm not using a word that we don't need.
Ok. And actually, maybe the third lecture, maybe next Monday we'll come back to row reduce.
So I won't make heavy weather of that, certainly not now.
So what is row reduce, just so you know.
I want to get that minus one to be a zero.
I'm aiming for a triangular matrix.
I want to clean out below the diagonal because if my matrix is triangular then I can see immediately everything.
Right?
Ultimately I'll reach a matrix U that'll be upper triangular and that first row won't change but the second row will change.
And what does it change to?
How do I clean out, get a zero in that where the minus one is right now?
Well I want to use the first row, the first equation.
I want to add some multiple of the first row to the second row.
And what should that multiple be?
I want to multiply that row by something.
And I'll say "add" today.
Later I'll say "subtract."
But what shall I do?
Just tell me what the heck to do.
I've got that row and I want to use it, I want to take a combination of these two rows.
This row and some multiple of this one that'll produce a zero.
This is called the pivot.
That's the first pivot P-I-V-O-T. Pivot.
And then that's the pivot row.
And what do I do?
Tell me what to do.
Add half this row to this one.
When I add half of that row to that one, what do I get?
I get that zero.
What do I get here for the second pivot?
What is it?
1.5, 3/2.
Because half of that is, so 3/2.
And the rest won't change.
So I'm happy with that zero.
Now I've got a couple more entries below that first pivot, but they're already zero.
That's where the sparseness pays off.
The tridiagonal really pays off.
So those zeros say the first column is finished.
So I'm ready to go on to the second column.
It's like I got to this smaller problem with the 3/2 here.
And a zero there.
What do I do now?
There is the second pivot, 3/2.
Below it is a non-zero.
I gotta get rid of it.
What do I multiply by now?
2/3.
2/3 of that new, second row added to the third row will clean out the third row.
This was already cleaned out.
This is already a zero.
But I want to have 2/3 of this row added to this one so what's my new third row?
Starts with zero and what's the third pivot now?
You see the pivots appearing?
The third pivot will be 4/3 because I've got 2/3 this minus one and two is 6/3 so I have 6/3.
I'm taking 2/3 away, I get 4/3 and that minus one is still there.
So you see that I'm-- this is fast.
This is really fast.
And the next step, maybe you can see the beautiful patterns that are coming.
Do you want to just guess the fourth pivot?
5/4, good guess, right.
5/4.
Now this is actually how MATLAB would find the determinant.
It would do elimination.
I call that elimination because it eliminated all those numbers below the diagonal and got zeros.
Now what's the determinant?
If I asked you for the determinant, and I will very rarely use the word determinant, but I guess I'm into it now, so tell me the determinant.
Five.
Why's that?
I guess I did say five earlier.
But how do you know it's five?
Whatever the determinant of that matrix is, why is it five?
Because it's a triangular matrix.
Triangular matrices, you've got all these zeros.
You can see what's happening.
And the determinant of a triangular matrix is just the product down the diagonal.
The product of these pivots.
The determinant is the product of the pivots.
And that's how MATLAB would compute a determinant.
And it would take two times 3/2 times 4/3 times 5/4 and it would give answer five.
My friend Alan Edelman told me something yesterday.
MATLAB computes in floating point.
So 4/3, that's 1.3333, etc.
So MATLAB would not, when it does that multiplication, get a whole number.
Right?
Because in MATLAB that would be 1.333 and probably it would make that last pivot a decimal, a long decimal.
And then when it multiplies that it gets whatever it gets.
But it's not exactly five I think.
Nevertheless MATLAB will print the answer five.
It's cheated actually.
It's done that calculation and I don't know if it takes the nearest integer when it knows that the-- I shouldn't tell you this, this isn't even interesting.
If the determinant of an integer matrix, whole number is a whole number, so MATLAB says, Better get a whole number.
And somehow it gets one.
Actually, it doesn't always get the right one.
So maybe later I'll know the matrix whose determinant might not come out right.
But ours is right, five.
Now where was this going?
It got thrown off track by the determinant.
What's the real test?
Well so I said there are two ways to see that a matrix is invertible.
Or not invertible.
Here we're talking about the first way.
How do I know that this matrix-- I've got an upper triangular matrix.
When is it invertible?
When is an upper triangular matrix invertible?
Upper triangular is great.
When you've got it in that form you should be able to see stuff.
So this key question of invertible, which is not obvious for a typical matrix is obvious for a triangular matrix.
And why?
What's the test?
Well, we could do the determinant but we can say it without using that long word.
The diagonal is non-zero.
K as invertible because the diagonal-- no, it's got a full set of pivots.
It's got four non-zero pivots.
That's what it takes.
That's what it's going to take to solve systems.
So this is the first step in solving this system.
In other words, to decide if a matrix is invertible, you just go ahead and use it.
You don't stop first necessarily to check invertibility.
You go forward, you get to this point and you see non-zeros there and then you're practically got to the answer here.
I'll leave for another day the final back to going back upwards that gives you the answer.
So K is invertible.
That means full set of pivots.
n non-zero pivots.
And here they are, two, 3/2, 4/3 and 5/4.
Worth knowing because this matrix K is so important.
We'll see it over and over again.
Part of my purpose today is to give some matrices a name because we'll see them again and you'll know them and you'll recognize them.
While I'm on this invertible or not invertible business I want to ask you to change K. To make it not invertible.
Change that matrix.
How could I change that matrix?
Well, of course, many ways.
But I'm interested in another matrix and this'll be among my special matrices.
And it will start out the same.
It'll have these same diagonals.
It'll be Toeplitz.
I'm going to call it C and I want to say the reason I'm talking about it now is that it's not going to be invertible.
And I'm going to tell you a C and see if you can tell me why it is not invertible.
So here's the difference; I'm going to put minus one in the corners.
Still zeros there.
So that matrix C still has that pattern.
It's still a Toeplitz matrix, actually.
That would still be the matrix Toeplitz of two, minus one, zero, minus one.
I claim that matrix is not invertible and I claim that we can see that without computing determinants, we can see it without doing elimination, too.
MATLAB would see it by doing elimination.
We can see it by just human intelligence.
Now why?
How do I recognize a matrix that's not invertible?
And then, by converse, how a matrix that is invertible.
I claim-- and let may say first, let me say why that letter C. That letter C stands for circulant.
it's because this word circulant, why circulant, it's because that diagonal which only had three guys circled around to the fourth.
This diagonal that only had three entries circled around to the fourth entry.
This diagonal with two zeros circled around to the other two zeros.
The diagonal are not only constant, they loop around.
And you use the word periodic.
Now for me, that's the periodic matrix.
See, a circulant matrix comes from a periodic problem.
Because it loops around.
It brings numbers, zero is the same as number four or something.
And why is that not invertible?
The thing is can you find a vector?
Because matrices multiply vectors, that's their whole point.
Can you see a vector that it takes to zero?
Can you see a solution to Cu=0?
I'm looking for a u with four entries so that I get four zeros.
Do you see it?
All ones.
All ones.
That will do it.
So that's a nice, natural entry, a constant.
And do you see why when I-- we haven't spoken about multiplying matrices times vectors.
And most people will do it this way.
And let's do this one this way.
You take row one times that, you get two, minus one, zero, minus one.
You get the zero because of that new number.
Here we always got zero from the all ones vector and now over here that minus one, you see it's just right.
If all the rows add to zero then this vector of all ones will be, I would use the word in the null space if you wanted a fancy word, a linear algebra word.
What does that mean?
It solves Cu=0.
And why does that show that the matrix isn't invertible?
Because that's our point here.
I have a solution to Cu=0.
I claim that the existence of such a solution has wiped out the possibility that the matrix is invertible because if it was invertible, what would this lead to?
If invertible, if C inverse exists what would I do to that equation that would show me that C inverse can't exist?
Multiply both sides by C inverse.
So you're seeing, just this first day you're seeing some of the natural steps of linear algebra.
Row reduction, multiply when you want to see what's happening, multiply both sides by C inverse.
That's the same as in ordinary language, Do the same thing to all the equations.
So I multiply both sides by the same matrix.
And here I would get (C inverse)(Cu)=(C inverse)(0).
So what does that tell me?
I made it long, I threw in this extra step.
You were going to jump immediately to C inverse C is I is the identity matrix and when the identity matrix multiplies a vector u, you get u.
And on the right side, C inverse, whatever it is if it existed, times zero would have to be zero.
So this would say that if C inverse exists, then the only solution is u equals u.
That's a good way to recognize invertible matrices.
If it is invertible then the only solution to Cu=0 u=0.
And that wasn't true here.
So we conclude C is not invertible.
C is therefore not invertible.
Now can I even jump in.
I've got two more matrices that I want to tell you about that are also close cousins of K and C. But let me just explain physically a little bit about where these matrices are coming from.
So maybe next to K-- so I'm not going to put periodic there.
Right?
That's the one that I would call periodic.
This one is fixed at the ends.
Can I draw a little picture that aims to show that?
Aims to show where this is coming from.
It's coming from I think of this as controlling like four masses.
Mass one, mass two, mass three and mass four with springs attached and with endpoints fixed.
So if I put some weights on those masses-- we'll do this; masses and springs is going to be the very first application and it will connect to all these matrices.
And all I'm doing now is just asking to draw the system.
Draw the mechanical system.
Actually I'll usually draw it vertically.
But anyway, it's got four masses and the fact that this minus one here got chopped off, what would I call that end?
I'd call that a fixed end.
So this is a fixed, fixed matrix.
Both ends or fixed.
And it's the matrix that would govern and the springs and masses all the same is what tells me that the thing is Toeplitz.
Now what's the picture that goes with C?
What's the picture with C?
Do you have an instinct of that?
So C is periodic.
So again we've got four masses connected by springs.
But what's up with those masses to make the problem cyclic, periodic, circular, whatever word you like.
They're arranged in a ring.
The fourth guy comes back to the first one.
So the four masses would be, so in some kind of a ring, the springs would connect them.
I don't know if that's suggestive, but I hope so.
And what's the point of, can we just speak about mechanics one moment?
How does that system differ from this fixed system?
Here the whole system can't move, right?
If there no force, then nothing can happen.
Here the whole system can turn.
They can all displace the same amount and just turn without any compression of the springs, without any force having to do anything.
And that's why the solution that kills this matrix is one, one, one, one.
So one, one, one, one would describe a case where all the displacements were equal.
In a way it's like the arbitrary constant in calculus.
You're always adding plus C. So here we've got a solution of all ones that produces zero the way the derivative of a constant function is the zero function.
So this is just like an indication.
Yes, perfect.
I've got two more matrices.
Are you okay for two more?
Yes okay, what are they?
Okay a different blackboard for the last two.
So one of them is going to come by freeing up this end.
So I'm going to take that support away.
And you might imagine like a tower oscillating up and down or you might turn it upside down and like a hanging spring, or rather four springs with four masses hanging onto them.
But this end is fixed and this is not fixed anymore, this is now free.
And can I tell you the matrix, the free-fixed matrix.
Free-fixed.
Because it's the top end that I changed, I'm going to call it T. So all the other guys are going to be the same but the top one, the top row, the boundary row, boundary conditions are always the tough part, the tricky part, the key part of a model, and here the natural boundary condition is to have a one there.
That two changed to a one.
Now if I asked you for the properties of that matrix-- so that's the third.
shall I do the fourth one?
So you have them all, you'll have the whole picture.
The fourth one, well you can guess.
What's the fourth?
What am I going to do?
Free up the other end.
So this guy had one free end and the other guy has B for both ends.
B for both ends are going to be free.
So this is free-fixed.
This'll be free-free.
So that means I have this free end, the usual stuff in the middle, no change, and the last row is what?
What am I going to put in the last row?
Minus one, one.
Minus one, one.
So I've changed the diagonal.
There I put a single one in because I freed up one end.
With B I freed both ends and I got two minus ones.
Now what do you think?
So we've drawn the free-fixed one and what's your guess?
They're all symmetric.
That's no accident.
They're all tridiagonal, no accident again.
Why are they tridiagonal?
Physically they're tridiagonal because that mass is only connected to it's two neighbors, it's not connected to that mass.
That's why we get a zero in the two, four position.
Because two is not connected to four.
So it's tridiagonal.
And it's not Toeplitz anymore, right?
Toeplitz says constant diagonals and these are not quite constant.
I would create K, I would take T equal K if I was going to create this matrix and then I would say T of one, one equal one.
That command would fix up the first entry.
Yeah, that's a serious question.
Maybe, can I hang on until Friday, and even maybe next week.
Because it's very important.
When I said boundary conditions are the key to problems, I'm serious.
If I had to think okay, what do people come in my office ask about questions, I say right away, What's the boundary condition?
Because I know that's where the problem is.
And so here we'll see these guys clearly.
Fixed and free, very important.
But also let me say two more words, I never can resist.
So fixed means the displacement is zero.
Something was set to zero.
The fifth guy, the fifth over here, that fifth column was knocked out.
Free means that in here it could mean that the fifth guy is the same as the fourth.
The slope is zero.
Fixed is u is zero.
Free is slope is zero.
So here I have a slope of zero at that end, here I have it at both ends.
So maybe that's a sort of part answer.
Now I wanted to get to the difference between these two matrices.
And the main properties.
So what are we see?
Symmetric again, tridiagonal again, not quite Toeplitz, but almost, sort of morally Toeplitz.
But then the key question was invertible or not.
Key question was invertible or not.
Right.
And what's your guess on these two?
Do you think that one's invertible or not?
Make a guess.
You're allowed to guess.
Yeah it is.
Why's that?
Because this thing has still got a support.
It's not free to shift forever.
It's held in there.
So that gives you a hint about this guy.
Invertible or not for B?
No.
And now prove that it's not.
Physically you were saying, well this free guy with this thing gone now, this is now free-free.
Physically we're saying the whole thing can move, there's nothing holding it.
But now, for linear algebra, that's not the proper language.
You have to say something about that matrix.
Maybe tell me something about Bu=0.
u What are you going to take for u?
Yeah.
Same u.
We're lucky in this course, u equal  is the guilty main vector many times.
Because again the rows are all adding to zero and the all ones vector is in the null space.
If I could just close with one more word.
Because it's the most important.
Two words, two words.
Because they're the most important words, they're the words that we're leading to in this chapter.
And I'm assuming that for most people they will be new words, but not for all.
It's a further property of this matrix.
So we've got, how many?
Four properties, or five?
I'm going to go for one more.
And I'm just going to say that name first so you know it's coming.
And then I'll say, I can't resist saying a tiny bit about it.
I'll use a whole blackboard for this.
So I'm going to say that K and T are, here it comes, take a breath; positive definite matrices.
So if you don't know what that means, I'm happy.
Right?
Because well, I can tell you one way to recognize a positive definite matrix.
And while we're at it, let me tell you about C and B.
Those are positive semi-definite because they hit zero somehow.
Positive means up there, greater than zero.
And what is greater than zero that we've already seen?
And we'll say more.
The pivots were.
So if I have a symmetric matrix and the pivots are all positive then that matrix is not only invertible, because I'm in good shape, the determinant isn't zero, I can go backwards and do everything, those positive numbers are telling me that more than that, the matrix is positive definite.
So that's a test.
We'll say more about positive definite, but one way to recognize it is compute the pivots by elimination.
Are they positive?
We'll see that all the eigenvalues are positive.
The word positive definite just brings the whole of linear algebra together.
It connects to pivots, it connects to eigenvalues, it connects to least squares, it's all over the place.
Determinants too.
Questions or discussion.
It's a big class and we're just meeting for the first time but there's lots of time to, chance to ask me.
I'll always be here after class.
So shall we stop today?
I'll see you Friday or this afternoon.
If this wasn't familiar, this afternoon would be a good idea.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Starting with a differential equation.
So key point here in this lecture is how do you start with a differential equation and end up with a discrete problem that you can solve?
But simple differential equation.
It's got a second derivative and I put a minus sign for a reason that you will see.
Second derivatives are essentially negative definite things so that minus sign is to really make it positive definite.
And notice we have boundary conditions that at one end the solution is zero, at the other end it's zero.
So this is fixed-fixed.
And it's a boundary value problem.
That's different from an initial value problem.
We have x space, not time.
So we're not starting from some thing and oscillating or growing or decaying in time.
We have a fixed thing.
Think of an elastic bar.
An elastic bar fixed at both ends, maybe hanging by its own weight.
So that load f(x) could represent the weight.
Maybe the good place to sit is over there, there are tables just, how about that?
It's more comfortable.
So we can solve that equation.
Especially when I change f(x) to be one.
As I plan to do.
So I'm going to change, I'm going to make it-- it's a uniform bar because there's no variable coefficient in there and let me make it a uniform load, just one.
So it actually shows you that, I mentioned differential equations and we'll certainly get onto Laplace's equation, but essentially our differential equations will not-- this isn't a course in how to solve ODEs or PDEs.
Especially not ODEs.
It's a course in how to compute solutions.
So the key idea will be to replace the differential equation by a difference equation.
So there's the difference equation.
And I have to talk about that.
That's the sort of key point.
That up here you see what I would call a second difference.
Actually with a minus sign.
And on the right-hand side you see the load, still f(x).
Can I move to this board to explain differences?
Because this is like, key step is given the differential equation replace it by difference equation.
And the interesting point is you have many choices.
There's one differential equation but even for a first derivative, so this if you remember from calculus, how did you start with the derivative?
You started by something before going to the limit.
h or delta x goes to zero in the end to get the derivative.
But this was a finite difference.
You moved a finite amount.
And this is the one you always see in calculus courses.
U(x + h)-U(x) just how much did that step go.
You divide by the delta x, the h and that's approximately the derivative, U'(x).
Let me just continue with these others.
I don't remember if calculus mentions a backward difference.
But you won't be surprised that another possibility, equally good more or less, would be to take the point and the point before, take the difference, divide by delta x.
So again all these approximate U'.
And now here's one that actually is really important.
A center difference.
It's the average of the forward and back.
If I take that plus that, the U(x)'s cancel and I'm left with, I'm centering it.
This idea of centering is a good thing actually.
And of course I have to divide by 2h because this step is now 2h, two delta x's.
So that again is going to represent U'.
But so we have a choice if we have a first derivative.
And actually that's a big issue.
You know, one might be called upwind, one might be called downwind, one may be called centered.
It comes up constantly in aero and mechanical engineering, everywhere.
You have these choices to make.
Especially for the first difference.
We don't have, I didn't allow a first derivative in that equation because I wanted to keep it symmetric and first derivatives, first differences tend to be anti-symmetric.
So if we want to get our good matrix K, and I better remember to divide by h squared because the K just has those that I introduced last time and will repeat, the K just has the numbers -1, 2, -1.
Now, first point before we leave these guys what's up with them?
How do we decide which one is better?
There's something called the order of accuracy.
How close is the difference to the derivative?
And the answer is the error is of size h. So I would call that first order accurate.
And I can repeat here but the text does it, how you recognize what this is the, sort of local error, truncation error, whatever, you've chopped off the exact answer and just did differences.
This one is also order of h. And in fact, the h terms, the leading error, which is going to multiply the h, has opposite sign in these two and that's the reason center differences are great.
Because when you average them you center things.
This is correct to order h squared.
And I may come back and find out why that h squared term is.
Maybe I'll do that.
Yeah.
Why don't I just?
Second differences are so important.
Why don't we just see.
And center differences.
So let me see.
How do you figure U(x + h)?
This is a chance to remember something called Taylor series.
But that was in calculus.
If you forgot it, you're a normal person.
So but what does it say?
That's the whole point of calculus, in a way.
That if I move a little bit, I start from the point x and then there's a little correction and that's given by the derivative and then there's a further correction if I want to go further and that's given by half of h squared, you see the second order correction, times the second derivative.
And then, of course, more.
But that's all you ever have to remember.
It's pretty rare.
Second order accuracy is often the goal in scientific computing.
First order accuracy is, like, the lowest level.
You start there, you write a code, you test it and so on.
But if you want production, if you want accuracy, get to second order if possible.
Now, what about this U(x - h)?
Well, that's a step backwards, so that's U(x).
Now the step is -h, but then when I square that step I'm back to +h squared, U''(x) and so on.
Ooh!
Am I going to find even more accuracy?
I could tell you what the next turn is.
Plus h cubed upon six, that's three times two times one, U''' .
And this would be, since this step is -h now, it would be -h cubed upon six U''' .
So what happens when I take the difference of these two?
Remember now that center differences subtract this from this.
So now that U(x + h)-U(x-h) is zero.
2hU' subtracting that from that is zero, two of these, so I guess that we really have an h cubed over three, U'''.
And now when I divide by the 2h, can I just divide by 2h here?
Oh yeah, it's coming out right, divide by 2h, divide this by 2h, that'll make it an h squared over six.
I've done what looks like a messy computation.
I'm a little sad to start a good lecture, important lecture by such grungy stuff.
But it makes the key point.
That the center difference gives the correct derivative with an error of order h squared.
Where the error if for the first differences the h would have been there.
And we can test it.
Actually we'll test it.
Okay for that?
This is first differences.
And that's a big question; what do you replace the first derivative by if there is one?
And you've got these three choices.
And usually this is the best choice.
Now to second derivatives.
Because our equation has got U'' in it.
So what's a second derivative?
It's the derivative of the derivative.
So what's the second difference?
It's the difference, first difference of the first difference.
So the second difference, the natural second difference would be-- so now let me use this space for second differences.
Second differences.
I could take the forward difference of the backward difference.
Or I could take the backward difference of the forward difference.
Or you may say why don't I take the center difference of the center difference.
All those, in some sense it's delta squared, but which to take?
Well actually those are the same and that's the good choice, that's the 1, -2, 1 choice.
So let me show you that.
Let me say what's the matter with that.
Because now having said how great center differences are, first differences, why don't I just repeat them for second differences?
Well the trouble is, let me say in a word without even writing, well I could even write a little, the center difference, suppose I'm at a typical mesh point here.
The center difference is going to take that value minus that value But then if I take the center difference of that I'm going to be out here.
I'm going to take this value, this value, and this value.
I'll get something correct.
Its accuracy will be second order, good.
But it stretches too far.
We want compact difference molecules.
We don't want this one, minus two of this, plus one of that.
So this would give us a 1, 0, -2 , 0, 1.
I'm just saying this and then I'll never come back to it because I don't like this one, these guys give 1, -2, 1 without any gaps.
And that's the right choice.
And that's the choice made here.
So I'm not thinking you can see it in your head, the difference of the difference.
But well, you almost can.
If I take this, yeah.
Can you sort of see this without my writing it?
If I take the forward difference and then I subtract the forward difference to the left, do you see that I'll have minus two.
So there is what I started with.
I subtract U(x)-U(x-h) and I get two -U(x)'s.
This is what I get.
Now I'm calling that ui.
I better make completely clear about the minus sign.
The forward difference or the backward difference, what this leads is 1, -2, 1.
That's the second difference.
Very important to remember, the second difference of a function is the function, the value ahead, minus two of the center, plus one of the left.
It's centered obviously, symmetric, right?
Second differences are symmetric.
And because I want a minus sign I want minus the second difference and that's why you see here -1, 2, -1 .
Because I wanted positive twos there.
Are you ok?
This is the natural replacement for -U''.
And I claim that this second difference is like the second derivative, of course.
And why don't we just check some examples to see how like the second derivative it is.
So I'm going to take the second difference or some easy functions.
It's very important that these come out so well.
So I'm going to take the second difference.
I'm going to write it as sort of a matrix.
So this is like the second different.
Yeah, because this is good.
I'm inside the region, here.
I'm not worried about the boundaries now.
Let me just think of myself as inside.
So I have second differences and suppose I'm applying it to a vector of all ones.
What answer should I get?
So if I think of calculus it's the second derivative of one, of the constant function.
So what answer am I going to get?
Zero.
And do I get zero?
Of course.
I get zero.
Right?
All these second differences are zero.
Because I'm not worrying about the boundary yet.
So that's like, check one.
It passed that simple test.
Now let me move up from constant to linear.
And so on.
So let me apply second differences to a vector that's growing linearly.
What answer do I expect to get for that?
So remember I'm doing second differences, like second derivatives, or minus second derivatives, actually.
So what do second derivatives do to a linear function?
If I take a straight line I take the-- sorry, second derivatives.
If I take second derivatives of a linear function I get?
Zero, right.
So I would hope to get zero again here and I do.
Right?
-1+4-3=0.
Minus one, sorry, let me do it here, -2+6-4.
And actually, that's consistent with our little Taylor series stuff.
The function x should come out right.
Now what about-- now comes the moment.
What about x squared?
So I'm going to put squares in now.
Do I expect to get zeroes?
I don't think so.
Because let me again test it by thinking about what second derivative.
So now I'm sort of copying second derivative of x squared, which is?
Second derivative of x squared is?
Two, right?
First derivative's 2x, second derivative is just two.
So it's a constant.
And remember I put in a minus sign so I'm wondering, do I get the answer minus two?
All the way down.
-4+8-9.
Whoops.
What's that?
What do I get there?
What do I get from that second difference of these squares?
-4+8-9 is?
Minus two, good.
So can we keep going?
-4+18-16.
What's that?
-4+18-16, so I've got -20+18 .
I got minus two again.
-9, 32, -25, it's right.
The second differences of the vector of squares, you could say, is a constant vector with the right number.
And that's because that second difference is second order accurate.
It not only got constants right and linears right, it got quadratics right.
So that's, you're seeing second differences.
We'll soon see that second differences are also on the ball when you apply them to other vectors.
Like vectors of sines or vectors of cosines or exponentials, they do well.
So that's just a useful check which will help us over here.
Okay, can I come back to the part of the lecture now?
Having prepared the way for this.
Well, let's start right off by solving the differential equation.
So I'm bringing you back years and years and years, right?
Solve that differential equation with these two boundary conditions.
How would you do that in a systematic way?
You could almost guess after a while, but systematically if I have a linear, I notice-- What do I notice about this thing?
It's linear.
So what am I expecting?
I'm expecting, like, a particular solution that gives the correct answer one and some null space solution or whatever I want to call it, homogenous solution that gives zero and has some arbitrary constants in it.
Give me a particular solution.
So this is going to be our answer.
This'll be the general solution to this differential equation.
What functions have minus the second derivative equal one, that's all I'm asking.
What are they?
So what is one of them?
One function that has its second derivative as a constant and that constant is minus one.
So if I want the second derivative to be a constant, what am I looking at?
x squared.
I'm looking at x squared.
And I just want to figure out how many x squareds to get a one.
So some number of x squareds and how many do I want?
-1/2, good.
Good.
-1/2.
Because x squared would give me two but I want minus one so I need -1/2.
Okay that's the particular solution.
Now throw in all the solutions, I can add in any solution that has a zero on the right side, so what functions have second derivatives equals zero?
x is good.
I'm looking for two because it's a second derivative, second order equation.
What's the other guy?
Constant, good.
So let me put the constant first, C, say, and Dx.
Two constants that I can play with and what use am I going to make of them?
I'm going to use those to satisfy the two boundary conditions.
And it won't be difficult.
You could say plug in the first boundary condition, get an equation for the constants, plug in the second, got another equation, we'll have two boundary conditions, two equations, two constants.
Everything's going to come out.
So if I plug in U(0)=0, what do I learn?
C is zero, right?
If I plug in, is that right?
If I plug in zero, then that's zero already, this is zero already, so I just learned that C is zero.
So C is zero.
So I'm down to one constant, one unused boundary condition.
Plug that in.
U(1)=-1/2.
What's D?
It's 1/2, right.
D is 1/2.
So can I close this up?
There's 1/2.
Dx is 1/2.
Now it just always pays to look back.
At x=0, that's obviously zero.
At x=1 it's zero because those are the same and I get zero.
So -1/2x squared plus 1/2x.
That's the kind of differential equation and solution that we're looking for.
Not complicated nonlinear stuff.
So now I'm ready to move to the difference equation.
So again, this is a major step.
I'll draw a picture of this from zero to one.
And if I graph that I think I get a parabola, right?
A parabola that has to go through here.
So it's some parabola like that.
That would be always good, to draw a graph of the solution.
Now, what do I get here?
Moving to the difference equation.
So that's the equation, and notice it's boundary conditions.
Those boundary conditions just copied this one because I've chopped this up.
I've got i equal one, two, three, four, five and this is one, the last point then is 6h .
h is 1/6.
What's going to be the size of my matrix and my vector and my unknown u here?
How many unknowns am I going to have?
Let's just get the overall picture right.
What are the unknowns going to be?
They're going to be u_1, u_2, u_3, u_4, u_5.
Those are unknown.
Those will be some values, I don't know where, maybe something like this because they'll be sort of like that one.
And this is not an unknown, u_6 , this is not an unknown, u_0 , those are the ones we know.
So this is what the solution to a difference equation looks like.
It gives you a discreet set of unknowns.
And then, of course MATLAB or any code could connect them up by straight lines and give you a function.
But the heart of it is these five values.
Okay, good.
And those five values come from these equations.
I'm introducing this subscript stuff but I won't need it all the time because you'll see the picture.
This equation applies for i equal one up to five.
Five inside points and then you notice how when i is one, this needs u_0 , but we know u_0.
And when I is five, this needs u_6 , but we know u_6 .
So it's a closed five by five system and it will be our matrix.
That -1, 2, -1 is what sits on the matrix.
When we close it with the two boundary conditions it chops off the zero column, you could say and chops off the six column and leaves us with a five by five problem and yeah.
I guess this is a step not to jump past because it takes a little practice.
You see I've written the same thing two ways.
Let me write it a third way.
Let me write it out clearly.
So now here I'm going to complete this matrix with a two and a minus one and a two and a minus one and now it's five by five.
And those might be u but I don't know if they are so let me put in u_1, u_2, u_3, u_4, and u_5.
Oh and divide by h squared.
I'll often forget that.
So I'm asking you to see something that if you haven't, after you get the hang of it it's like, automatic.
But I have to remember it's not automatic.
Things aren't automatic until you've done them a couple of times.
So do you see that that is a concrete statement of this?
This delta x squared is the h squared.
And do you see those differences when I do that multiplication that they produce those differences?
And now, what's my right-hand side?
Well I've changed the right-hand side to one to make it easy.
So this right-hand side is all ones.
And this is the problem that MATLAB would solve or whatever code.
Find a difference code.
I've got to a linear system, five by five, it's fortunately, the matrix is not singular, there is a solution.
How does MATLAB find it?
It does not find it by finding the inverse of that matrix.
Monday's lecture will quickly review how to solve five equations and five unknowns.
It's by elimination, I'll tell you the key word.
And that's what every code does.
And sometimes you would have to exchange rows, but not for a positive definite matrix like that.
It'll just go bzzz, right through.
When it's tridiagonal it'll go like with the speed of light and you'll get the answer.
And those five answers will be these five heights.
u_1, u_2, u_3, u_4, and u_5.
And we could figure it out.
Actually I think section 1.2 gives the formula for this particular model problem for any size, and particular for five by five.
And there is something wonderful for this special case.
The five points fall right on the correct parabola, they're exactly right.
So for this particular case when the solution was a quadratic, the exact solution was a quadratic, a parabola, it will turn out, and that quadratic matches these boundary conditions, it will turn out that those points are right on the money.
So that's, you could call, is like, super convergence or something.
I mean that won't happen every time, otherwise life would be like, too easy.
It's a good life, but it's not that good as a rule.
So they fall right on that curve.
And we can say what those numbers are.
Actually, we know what they are.
Actually, I guess I could find them.
What are those numbers then?
And of course, one over h squared is-- What's one over h squared, just to not forget?
One over h squared there, h is what?
1/6.
Squared is going to be a 36.
So if I bring it up here, bring the h squared up here, it would be times a 36.
Well let me leave it here, 36.
And I'm just saying that these numbers would come out right.
Maybe I'll just do the first one.
What's the exact u_1, u_2?
u_1 and u_2 would be what?
The exact u_1, ooh!
Oh shoot, I've got to figure it out.
If I plug in x=1/6... Do we want to do this?
Plug in x=1/6?
No, we don't.
We don't.
We've got something better to do with our lives.
But if we put that number in, whatever the heck it is, in this one, we would find out came out right.
The fact that it comes out right is important.
But I'd like to move on to a similar problem.
But this one is going to be free-fixed.
So if this problem was like having an elastic bar hanging under its own weight and these would be the displacements points on the bar and fixed at the ends, now I'm freeing up the top end.
I'm not making u_0, zero anymore.
I better maybe use a different blackboard because that's so important that I don't want to erase it.
So let me take the same problem, uniform bar, uniform load, but I'm going to fix U over one, that's fixed, but I'm going to free this end.
And from a differential equation point of view, that means I'm going to set the slope at zero to be zero.
U'(0)=0.
That's going to have a different solution.
Change the boundary conditions is going to change the answer.
Let's find the solution.
So here's another differential equation.
Same equation, different boundary conditions, so how do we go?
Well I had the general solution over there.
It still works, right?
U(x) is still -1/2 of x squared.
The particular solution that gives me the one.
Plus the Cx plus D that gives me zero, one, zero for second derivatives but gives me the possibility to satisfy the two boundary conditions.
And now again, plug in the boundary conditions to find C and D. Slope is zero at zero.
What does that tell me?
I have to plug that in.
Here's my solution, I have to take it's derivative and set x to zero.
So it's derivative is a 2x or a minus x or something which is zero.
The derivative of that is C and the derivative of that is zero.
What am I learning from that left, the free boundary condition?
C is zero, right?
C is zero because the slope here is C and it's supposed to be zero.
So C is zero.
Now the other boundary condition.
Plug in x=1.
I want to get the answer zero.
The answer I do get is minus 1/2 at x=1 , plus D .
So what is D then?
What's D?
Let me raise that.
What do I learn about D?
It's 1/2.
I need 1/2.
So the answer is -1/2 of x squared plus 1/2.
Not 1/2x as it was over there, but 1/2.
And now let's graph it.
Always pays to graph these things between x equals zero and one.
What does this looks like?
It starts at 1/2, right?
At x=0.
And it's a parabola, right?
It's a parabola.
And I know it goes through this point.
What else do I know?
Slope starts at?
The slope starts to zero.
The other, the boundary condition, the free condition at the left-hand end, so slope starts at zero, so the parabola comes down like that.
It's like half a-- where that was a symmetric bit of a parabola, this is just half of it.
The slope is zero.
And so that's a graph of U(x) .
Now I'm ready to replace it by a difference equation.
So what'll be the difference equation?
It'll be the same equation for the -u''.
No change.
So minus u_(i+1) minus 2u_i , minus u_(i-1) over h squared equals.
I'm taking f(x) to be one, so let's stay with one.
Okay, big moment.
What boundary conditions?
What boundary conditions?
Well, this guy is pretty clear.
That says u_(n+1) is zero.
What do I do for zero slope?
What do I do for a zero slope?
Okay, let me suggest one possibility.
It's not the greatest, but one possibility for a zero slope is (u_1-u_0)/h .
That's the approximate slope, should be zero.
So that's my choice for the left-hand boundary condition.
It says u_1 is u_0 .
It says that u_1 is u_0 .
So now I've got again five equations for five unknowns, u_1, u_2, u_3, u_4, and u_5.
I'll write down what they are.
Well, you know what they are.
So this thing divided by h squared is all ones, just like before.
And of course all these rows are not changed.
But the first row is changed because we have a new boundary condition at the left end.
And it's this.
So u_1, well u_0 isn't in the picture, but previously what happened to u_0 , when i is one, I'm in the first equation here, that's where I'm looking.
i is one.
It had a u_0.
Gone.
In this case it's not gone.
u_0 comes back in, u_0 is u_1.
That might-- Ooh!
Don't let me do this wrong.
Ah!
Don't let me do it worse!
All right.
There we go.
Good.
Okay.
Please, last time I videotaped lecture 10 had to fix up lecture 9, because I don't go in.
Professor Lewin in the physics lectures, he cheats, doesn't cheat, but he goes into the lectures afterwards and fixes them.
But you get exactly what it looks like.
So now it's fixed, I hope.
But don't let me screw up.
So now, what's on this top row?
When i is one.
I have minus u_2, that's fine.
I have 2u_1 as before, but now I have a minus u_1 because u_0 and u_1 are the same.
So I just have a one in there.
That's our matrix that we called T. The top row is changed, the top row is free.
This is the equation T * U divided by h squared is the right-hand side ones.
Ones of five, would call that.
Properly I would call it ones of five one, because the MATLAB command ones wants matrix and it's a matrix with five rows, one column.
But it's T, that's the important thing.
And would you like to guess what the solution looks like?
In particular, is it again exactly right?
Is it right on the money?
Or if not, why not?
The computer will tell us, of course.
It will tell us whether we get agreement with this.
This is the exact solution here and this is the exact parabola starting with zero slope.
So but I solved this problem.
Oh, let me see, I didn't get u_1, u_2 to u_5 in there.
So it didn't look right.
u_1, u_2, u_3, u_4, and u_5.
And that's the right-hand side.
Sorry about that.
So that's T divided by h squared, T with that top row changed times U is the right-hand side.
By the way, I better just say what was the reason that we came out exactly right on this problem?
Would we come out exactly right if it was some general load f(x) ?
No.
Finding differences can't do miracles.
They have no way to know what's happening to f(x) between the mesh points, right?
If I took this to be f(x) and took this at the five points, at these five points, this wouldn't know what f(x) is in between, couldn't be exactly right.
It's exactly right in this lucky special case because, of course, it has the right ones.
But also because, the reason it's exactly right is that second differences of quadratics are exactly right.
That's what we checked on this board that's underneath there.
Second differences of squares came out perfectly.
And that's why the second differences of this guy give the right answer, so that guy is the answer to both the differential and the difference equation.
I had to say that word about why was that exactly right.
It was exactly right because second differences of squares are exactly right.
Now, again, we have second differences of squares.
So you could say exactly right or no?
What are you betting?
How many think, yeah, it's going to come out on the parabola?
Nobody.
Right.
Everybody thinks there's something going to miss here.
And why?
Why am I going to miss something?
Yes?
It's a first order approximation at the left boundary.
Exactly right, exactly right.
It's a first order approximation to take this and I'm not going to get it right.
That first order approximation, that error of size h is going to penetrate over the whole interval.
It'll be biggest here.
Actually I think it turns out, and the book has a graph, I think it comes out wrong by 1/2h there.
1/2h, first order.
And then it, of course, it's discrete and of course it's straight across because that's the boundary condition, right?
And then it starts down, it gets sort of closer, closer, closer and gets, of course, that's right at the end.
But there's an error.
The difference between U, the true U and the computed U is of order h. So you could say alright, if h is small I can live with that.
But as I said in the end you really want to get second order accuracy if you can.
And in a simple problem like this we should be able to do it.
What I've done already covers section 1.2 but then there's a note, a worked example at the end of 1.2 that tells you how to upgrade to second order.
And maybe we've got a moment to see how would we do it.
What would you suggest that I do differently?
I'll get a different matrix.
I'll get a different discrete problem.
But that'll be ok.
I can solve that just as well.
And what shall I replace that by?
because that was the guilty party, as you said.
That was guilty.
That's only a first order approximation to zero slope at zero.
A couple of ways we could go.
This is a correct second order approximation at what mesh point?
That is a correct second order approximation to U'=0, but not at that point or at that point where?
Halfway between.
If I was looking at a point halfway between that, that would be centered there, that would be a centered difference and it would be good.
But we're not looking there.
So I'm looking here.
So what do you suggest I do?
Well I've got to center it.
Essentially I'm going to use U, minus one.
I'm going to use U, minus one.
And let me just say what the effect is.
You remember we started with the usual second difference here, 2, -1, -1.
This is what got chopped off for the fixed method.
It got brought back here by our first order method.
Our second order method will-- You see what's likely to happen?
That -1 is going to show up where?
Over here.
To center it around zero.
So that guy will make this into a minus two.
Now that matrix is still fine.
It's not one of our special matrices.
When I say fine, it's not beautiful is it?
It's got one, like, flaw, it needs what do you call it when you have your face-- cosmetic surgery or something.
It needs a small improvement.
So what's the matter with it?
It's not symmetric.
It's not symmetric and a person isn't happy with a un-symmetric problem, approximation to a perfectly symmetric thing.
So I could just divide that row by two.
If I divide that row by 2, which you won't mind if I do that, make that one, minus one and makes this 1/2.
I divided the first equation by two.
Look in the notes in the text if you can.
And the result is now it's right on.
It's exactly on.
Because again, the solution, the true solution is squares.
This is now second order and we'll get it exactly right.
And I say all this for two reasons.
One is to emphasize again that the boundary conditions are critical and that they penetrate into the region.
The second reason for my saying this is looking forward way into October.
So let me just say the finite element method, which you may know a little about, you may have heard about, it's another-- this was finite differences.
Courses starting with finite differences, because that's the most direct way.
You just go for it.
You've got derivatives, you replace them by differences.
But another approach which turns out to be great for big codes and also turns out to be great for making, for keeping the properties of the problem, the finite element method, you'll see it, it's weeks away, but when it comes, notice, the finite element method automatically produces that first equation.
Automatically gets it right.
So that's pretty special.
And so, the finite element method just has, it produces that second order accuracy that we didn't get automatically for finite differences.
Ok, questions on today or on the homework.
So the homework is really wide open.
It's really just a chance to start to see.
I mean, the real homework is read those two sections of the book to capture what these two lectures have done.
So Monday I'll see.
We'll do elimination.
We'll solve these equations quickly and then move on to the inverse matrix.
More understanding of these problems.
Thanks.
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Starting today on section 1.3, actually we'll finish that today.
And that's about the big problem in, or the most common problem in scientific computing, solving a linear system.
Au=b.
How would you do that?
So we will certainly do it for some examples.
Like our free-fixed or our fixed-fixed matrices.
But I thought I'd just put some comments.
Often I make comments that go beyond what specific things we will do in detail.
Certainly the workhorse of, if I'm in MATLAB notation, the workhorse would be backslash.
That command is the quick, natural way to get the answer u.
And I want to say more about backslash.
I won't be able to say all the things it does.
There's a lot built into that command.
And the MATLAB helpdesk would give the details of all the different things that backslash actually does.
It looks first, can it divide the problem into blocks, into smaller problems.
It looks to see if A is symmetric.
If it's symmetric, because of course, that would cut the work in half because you know half the matrix from the other half in the symmetric case.
It goes hopefully along and eventually gives you the answer.
Ok, so I'm going to say more about that.
Because that's by elimination.
The method there is elimination.
I just thought I would mention that if I had a large sparse matrix, so I was in sparse math MATLAB and used backslash, it would try to reorder the rows.
Or it will try to reorder the rows in an optimal way.
Now, what would be an optimal way?
Just so you have an idea of what.
So elimination takes this matrix, right?
It's got all it's rows.
And we'll do examples.
So the idea of elimination is that it produces a zero.
So let's suppose it accepts the first row as the pivot row.
And this 1, 1 entry would be the pivot.
And then how does it use the pivot?
It subtracts multiples of that pivot row from the other rows in order to get zeroes in the first column.
So then it has a new matrix that's like it's one size smaller.
And then it just continues all the way until it gets to a triangular matrix with all the pivots sitting there on the diagonal.
So that's the idea of elimination.
And we'll do small examples.
So why would you want to reorder the rows?
Two reason, two good reasons for reordering the rows.
One reason which MATLAB will always unless it has some good reason not to, is if that pivot is too small compared to numbers below it it will pick the largest and reorder to put the largest pivot up there.
So you like large pivots.
Why do you like large pivots?
Because for numerical stability that gives you small multipliers.
If this is the biggest number then all the multiples that you need, the multiple of that row that you want to subtract from the other rows will be less than one.
The multiplier is, so just let's remember, the multiplier is the entry to eliminate divided by the pivot.
So if that entry to be eliminated is smaller then the pivot, then the multiplier's less than one and things stay stable.
So that's a picture of elimination, a sort of first picture.
And if it's sparse that means there's a lot of zeroes.
And where would we like those zeroes to be?
Where would we like to find zeroes to save time and use to avoid all the steps, to avoid going through every step.
If this was already a zero then it's there and we don't need to get it there.
The job is already done.
We can go forward.
And if we have a zero here then it's already zero in the next one and we don't have to do, that would be the second pivot row, we don't have to subtract it from this row because a zero is already there.
So we like zeroes.
And sparse matrices have a lot of them.
But we only like them if they're in the right place.
And what's the right place?
It's, roughly speaking, it's the beginning of a row.
So we're very happy with, we would like to get an ordering that looks somehow like this.
We would like to get a bunch of zeroes in there.
Well that's, I really overdid it.
Maybe we can't get that many.
But my point is, you see the point that those are at the beginning of the rows.
So let's say we can't quite get to that, but maybe something more like that.
Zero.
So we get some rows starting with zeroes and then we get rows that don't.
Which of course, I mean, they were there in the first place.
What we've done is move them to the bottom.
And the reason for moving them to the bottom is that they don't destroy rows below them.
Yeah.
I didn't show how that would happen.
Suppose I had a number there.
Suppose that wasn't a zero.
And all these were.
Then when I do that elimination I subtract a multiple of that row from this row.
Do you see that those zeroes will fill in?
So fill in is the bad thing that you're trying to avoid.
So in this quick discussion of reordering you're trying to order the rows so that they start with zeroes for as long as you can because those zeroes will never fill in.
Zeroes inside a band, yes, just to make the point again, suppose I have a long row.
Then if I have some zeroes here, here, here, useless.
Because when I go to do elimination on these rows, when I subtract a multiple of that row from that row, that zero will fill in, it didn't help, it's gone.
And this and this.
So zeroes sort of near the diagonal are often going to fill in.
It's zeroes at the far left that are good.
So that's a discussion of a topic that actually comes into 18.086.
And so does the third words that I thought I would just write on the board.
So this would be topic specializing in how do you solve large systems.
And reordering with elimination is one way.
And a second approach is, I just put these words up here so that you've seen them.
Conjugate gradient method, that's a giant success in solving symmetric problems.
Large symmetric problems.
Something called multigrid is a terrific, incomplete LU.
People have worked hard on ideas for solving problems so large that this becomes too expensive.
But backslash is the natural choice.
If you can do it, it's like it's so simple.
So let me focus on backslash.
So backslash is the key to talk about.
Let me see, let's just think what does backslash do?
Suppose I had two equations, two different right-hand sides, b and c with the same matrix A.
Would I solve them, would I do u=A\b and then separately A\c?
No.
If I had two equations with the same matrix, there's a big saving in, suppose I have two equations, two, well, can I put it?
I'll make a matrix.
I mean this is what MATLAB would always do.
Put those two right-hand sides into a matrix.
Then backslash is happy with that.
That backslash command would solve both equations.
It would give you the answer then, would be, it would be two parts to it.
What would be the first column of u then?
So, do you see what I have now?
I have a square matrix A times an unknown u.
And I've got two right-hand sides.
The point is this often happens.
If you're doing, say, a design problem.
You might try different designs.
So those different designs give you different right-hand sides.
And then you have a problem to solve.
What's the response to those designs?
What are the displacements, what happens?
So you want to solve with two right-hand sides and the point is you don't want to go through the elimination process on A twice.
That's crazy.
You see how that elimination process on A has this, what I described here, didn't look at b. I didn't even get to the right-hand side part.
I was dealing with the expensive part, which is the matrix A.
So we don't want to pay that price twice.
And therefore backslash is all set up.
So what's the first column of u then?
It's A\b.
And now what's another way to write, a more mathematical way to write A?
What's the answer to Au=b.
It's u equal A inverse b, right?
That's the answer in the first column and A inverse c would be the answer in the second column.
So it would produce those answers.
Backslash will produce the answer to the first equation and the answer the second equation.
Will it do that by finding A inverse?
No.
A inverse, for multiple reasons, we don't often compute, if it's 3 by 3, 4 by 4, then it's not a bad idea to see what A inverse looks like.
We'll do that actually, because it's very enlightening if the matrix is small, you can see what's going on.
But for a large problem we don't want A inverse.
We want the answer.
And backslash goes to the answer.
It doesn't get there by computing A inverse.
And our examples will show how.
So I'm giving A inverse a little bad comments right now.
So I maybe should finish that sentence and then you'll see me turn around and compute the darn thing.
But why do we not use A inverse?
Two reasons.
One is it's more expensive.
If I have to compute A inverse and multiply by b, that's taking too long.
Second reason is A inverse could easily be a full, dense matrix.
All non-zero.
Where A itself was like, tridiagonal.
So if A is tridiagonal, all the numbers we need are there in three diagonals, we don't want A inverse, you'll see, A inverse is full.
So two reasons for not using A inverse.
Takes too long in the first place even in the good case and often our matrix A has got lots is zeroes that are not there in A inverse so we've wasted time.
Nevertheless let me say something about A inverse on the next board.
I don't know if you ever thought about the inverse matrix.
Let me ask you this question.
Suppose I use the command A\I.
What's that doing?
That's putting the identity on the right-hand side instead of a single vector b or instead of two vectors b and c I'm now putting n. That's ok. Backslash will work with that.
That's a shorthand for solving.
This solves all these equations.
A, it'll get my different answers, u_1, u_2.
It'll get n different answers from the n right-hand sides, the columns of I.
Let me take to be three.
0, 1, 0; 0, 0, 1.
But I'll take n to be three.
That's the identity.
So this solves that equation.
A\I will output u, will output u.
And here's the question.
What have I got?
If you have a matrix A, square matrix and of course you have to create I as eye(3), that I would be, this would be eye(3).
Cleve Moler's lousy pun.
So what would I get?
What would I get?
The inverse, yes.
I'd get the inverse.
I'd get the inverse matrix.
A backslash-- why's that?
Because what matrix solves, what's the solution u to A times something equal I?
The solution to this equation is Au=I.
The solution to that equation is the inverse matrix.
u will be A inverse.
So that is a pretty, I mean, that's pretty, if I wanted A inverse that's a good way to do it.
Other ways to do it would be inv(A) in MATLAB and other ways, but this is about as good as you get.
Do you see that you get the inverse matrix that way?
And it's worth giving some words to that fact.
How would I describe this?
This is a set of three different problems.
I would describe <1, 0, 0>, that right-hand side as an impulse.
That's an impulse.
A delta vector with an impulse in the first component.
I'd call that an impulse in the second component.
I'd call that an impulse in the third component.
So my inputs are three impulses and my outputs are u_1, u_2, u_3.
What words might I use?
I could call those impulse response.
If I were in Course 6, I certainly would.
These would be impulses, these would be the responses to that impulse from our system.
So those are impulse responses but in linear algebra the words I would use, u_1, u_2, u_3 are the columns of A inverse, right?
That's what we just said.
u_1, u_2, u_3.
So the columns of-- Let me write that down.
The columns of A inverse are the responses, the u's, the solutions to the impulses, to n impulses.
And these are the columns of I.
Do you see?
Nothing I've said so far is deep or anything.
But it's just, this comes up so much that it's nice to have different ways to think about this.
And actually, yeah, if I had to solve by hand, if I had to find the inverse by hand I would use elimination on this system of equations.
I would take A and put the identity next to it.
I would do elimination a lot.
Actually I'll put that in one.
If I had to find the inverse I would take this block matrix.
Is that the first block matrix we've seen?
Block matrices are really, you should just get familiar, when you're getting familiar with matrices, they often come in blocks.
So here's a three by three block, here's a three by three block, the whole matrix is three by six.
I can go through the elimination steps on it.
If I really go haywire on elimination and keep going and going and going all the way until A gets to the identity, so I do elimination and I get it triangular then I even clean out up above the pivots and then I change all the pivots to one, I can get all the way to the identity if the matrix is invertible.
And what do you think will show up in the right half?
A inverse, yeah.
So A inverse will show up there.
So in 18.06 I would explain, go through examples of this computation just to see A inverse appear.
There's no reason for us to go through long examples like that.
M one three by three would be worth doing, but in the big picture we're going to use backslash.
Questions or discussion.
So this is the sort of overall picture about the inverse.
Well this is about the inverse.
This was about elimination.
I've gotta take a little time on elimination.
No it's just too important.
It's sort of straightforward and mechanical but it's like, too important to blow away.
So let me remove that and put something better there.
So now I'm talking about elimination.
I'm talking about one equation.
I'm back to Au=b.
Au=b.
And notice I'm using the letter A rather than K or one of our special letters because right now I don't know that that's a special matrix.
In a minute the example I do will be one of our special matrices of course, and then I'll use it's letter.
Just a word, though.
I thought I would take-- you're getting a lot of big picture here for a minute and then we'll get into the details.
I thought I would just, like, this is an occasion to look ahead.
To say a word about the big picture of linear algebra.
It's got four major problems, linear algebra.
And there are four commands to solve those problems.
And those commands, why not know?
So those commands are LU.
I'm speaking about MABLAB notation but Octave, Scilab, Python, R, all other would have those.
Would do these same things.
So LU is the command that produces this, but I didn't say what that is yet.
Ooh.
So that's my job, right.
What does this mean?
What's up there?
So, okay, to nobody's surprise MATLAB thought, ok, LU was a good letter for that.
And what did MATLAB think of as a good letter for the command that does this?
QR.
So if I did lu(A) or qr(A) I would get-- I mean, this is sometimes associated with the names of Gram-Schmidt.
It makes vectors orthogonal.
Not to worry about this stuff.
You can like, close your eyes for a moment here.
lu is the first command and that's what today's about.
qr is the key command for least squares problems.
Maybe the biggest application of rectangular matrices, I'm sure that's the big.
Eigenvalues, do you know about eigenvalues?
Well we'll just name the command eig(A).
And the singular value D composition, which you may never have heard of, but you will, is svd(A).
Can I leave that?
It's in the videotape now.
And my point is that when we've spoken about those four, we really have got numerical linear algebra and a lot of pure linear algebra explained.
These are the four big ones.
And I guess what I'm saying is, the four big problems of linear algebra turn out to, a good way to describe the answer is as a factorization of the matrix.
This is a factor.
So now let me say what this one is.
I start with a matrix and what elimination is really doing, if you look to see what is it doing, it's producing a lower triangular times an upper triangular.
Let's go directly to that.
Let me go directly to that.
Let me take an example.
So here's my matrix.
Well, I don't have to call it A because you recognize it.
So what's our name for that matrix?
T. T because the top boundary condition is free.
Oh, that reminds me.
Some good comments after class Friday brought out something that I sloughed over.
That the free-fixed matrix, the free-fixed problem is usually one unknown larger than the fixed-fixed.
Because remember the fixed-fixed problem had both ends fixed.
They were not unknowns.
The only unknowns were one, two, three to n in the middle.
But people noticed when I was talking about the free boundary condition that u_0 came into it and u_0 is not known.
So really, the free boundary condition like, has an extra unknown, an extra row and column in the matrix and that's correct.
We'll see later in Fourier transforms cosine matrices are one size bigger than sine matrices.
The cosine matrices are free-free and the sine matrices are fixed-fixed.
And now here we're at free-fixed.
I want to do elimination on that matrix.
And while I'm at it, we'll find the inverse.
But let's do elimination.
Just on that matrix.
Just to see what this L and U stuff is.
What do we do?
The first pivot is?
One, it's fine.
Not going to worry about that.
We'll use it now.
So how do I use it?
I use a pivot, now listen because here is a convention here, I'm going to use the word subtract.
You would say add that row to that row, right?
Because you want to get a zero here.
Forgive me for making it sound harder.
I'm going to say subtract because I like subtraction.
Subtract minus one of that row, my multiplier is minus one here.
I'm going to say subtract minus one of that row from that.
Same thing.
You'll say okay.
No problem.
Let's just do it.
So there's the pivot row and now when I-- shall I just add?
When I add that to or does my superego thing subtract minus one of that from that.
What do I get?
I get the zero, the one and the minus one.
And then what do I get?
What's the multiplier?
So let's just put these L, these multipliers, the l_21.
That's the multiplier.
2, 1 refers to row two, column one.
And this step got the zero in row two, column one.
And what was the multiplier that did it?
It's the number that I multiplied row one by and subtracted from row two, so it was minus one.
What's l_31?
What's the multiplier that produces a zero in the three, row three, column one position?
It's zero.
I take zero of this row away from this row because it's zero already.
So I'm not going to change, that row won't change and l_31 was zero.
Now I know the next pivot.
I'm ready to use it.
I want to get a zero below it because I'm aiming at this upper triangular u.
And what's the multiplier now?
And what's its number?
What's the multiplier number?
3, 2 because I'm trying to fix row three, column two.
And what do I multiply this by and subtract from this to make it zero?
It's negative one again.
Negative one, right.
It's negative one.
Sorry.
Right.
And now what happens when I do that?
Can I just do it in place here?
Forgive me if I just add that to that.
And I'll get zero and one.
And now what do I know at this point, what have I learned?
The most important thing I've learned is the matrix is invertible.
Because the pivots one, one, and one, well they're all here on the diagonal.
This is my matrix U.
That's my upper triangular matrix.
And-- yeah, of course?
I'm subtracting from this, so I've got the two is there, yeah, yeah, that's right.
So the two is sitting there and I'm subtracting minus one of that row from it and that would mean taking, yeah, yeah.
Right.
That would mean I'm subtracting one from the two and getting the one.
Yeah, yeah.
So the row, the typical entry is, the typical result is the row you have minus L times the pivot row.
The row you have minus the multiplier times the pivot row.
That's the operation that elimination lives on.
Elimination does that all the time.
It's one of the basic linear algebra subroutines.
B L A S. Now, this is my U.
So that's the goal of elimination, get upper triangular.
And the reason is, you can solve upper triangular systems really fast.
These multipliers l, l_21, 2, and so on, they and go into the L matrix.
And now, let me just say it here that in a way, that example is too beautiful.
Seldom am I sorry to see an example come out beautifully, but why do I say this is too beautiful?
It's not typical.
If I had other numbers here, I would get to other numbers here and what would be the difference, typically?
The pivots wouldn't be all ones.
That's what's too beautiful here, but let's go with it.
I mean, it was worth it because everything came out simple.
But the pivots for another problem, ooh, let me just do a second problem here.
I'll do the fixed-fixed guy.
Ok, so let's just do elimination on that.
That's the first pivot.
Subtract.
Now what's the multiplier now?
You're not as quick as MATLAB.
MATLAB is ahead of you.
So the multiplier is negative 1/2.
So the multiplier is, l_21 is negative 1/2.
l_31 will again be zero and let's use it, so it knocks that guys out.
And what did that number come out to be?
Do you remember?
That was 3/2.
I think we looked at that once.
And that would be all the same.
And then the next multiplier, l_32 will be negative 2/3 because when I multiply that by 2/3 it gives me the negative one and then I subtract and it kills this and I get 4/3.
I just did that quickly.
And my main point was the pivots are on the diagonal.
They're not all ones now.
So this is a more typical one.
This is, again our u.
And our L matrix will be, oh, oh, that's the point.
That these l's, these multipliers fit right into a lower triangular matrix.
All these multipliers, and we'll put ones on the diagonal of that guy and these lower triangular ones will fit in just right perfectly in there.
Over here the L would be, let me construct the L. Ones on the diagonal representing the pivot rows that stayed put and minus one, zero, and minus one as the multipliers that, so this was L, the multipliers that we used.
One more.
We're doing lots of good stuff here and it's not deep, but it's-- Suppose the matrix had been singular.
We have to realize, okay, this elimination method is great.
But it can break down and it's going to break down, it has to break down somehow if the matrix is singular.
Now what's our example of a singular matrix here?
The matrix, this is free-fixed and that by fixing one support it wasn't singular, but if I want to make it singular, what'll I take?
Free-free.
Free-free matrix.
So can I, if I had thought to bring colored chalk, I'll just erase for a moment for the bad case.
The bad case would be free-free.
And how would it show up as bad in elimination.
How does a singular matrix reveal itself as elimination goes forward?
Because you can't tell at the beginning.
What would have gone wrong?
We would have had a zero there.
We had a two that dropped to one.
But if we start with a one, it'll drop to zero.
That would have been a zero there.
The matrix would not have had three pivots.
This upper triangular matrix is singular, no good.
And that tells us back there that the original matrix is singular, no good.
So if I can't get to three pivots somehow, the matrix'll be singular and that's an example that is.
And MATLAB would immediately tell us, of course.
So let's go back to the good case for the main point.
The good case for the main point.
So the good case was three pivots.
In fact it was extra good because they all turned out to be ones.
Now, oh, now we're ready for LU.
Here's the magic.
And I'm not giving a proof.
The magic is that the result U, if I multiply the multiplier matrix L times the result U, I'll bring back A. I'll bring back A.
So let me just see.
If I multiply L by U, so this is now L times U, maybe you can see that I get A.
So what is U?
I just have to copy it.
[1, 1, 1; -1, -1, 0].
I could fill in the zeroes but I know they're there.
That's L times U.
And sure enough, if I do the multiplication, this-- How would you to that multiplication?
I would say this is one of the first row when I see that.
1, 0, 0 multiplying these.
I'd say get one of the first row.
That's correct in A.
Here I would say this is minus one of the first row, plus one of the second row, and sure enough it's the right part of A.
And this is minus one of the second row, plus one of the third row, and sure enough it's the right third row of A. I get A.
And that's when elimination goes through with no zero pivots, no problems, just a bunch of multipliers, then that wonderful description of it, A=LU is correct.
I don't know how many that's new to.
I should maybe have thought ahead.
How many have seen like, L times U before?
Just to give me an idea?
Quite a few.
Ok.
So it's terrific.
Oh, here I would get L times U.
Now this is like a little more interesting because the pivots were not ones.
So that's my matrix U.
And here's my matrix L, right?
Okay, big point.
Because we're so interested in symmetric matrices and this one in particular, or that one, symmetric matrices are good.
Now, I'm unhappy about one aspect.
So now there's just one part of this.
This was great, we got three non-zero pivots, we got to U, we got the multiplier matrix all fine and we would be ready for the right-hand side and we would be ready for two right-hand sides, we would even be ready for all three right-hand sides, whatever.
But I have one criticism.
The matrix A which was our K, this was really K, was symmetric.
That was the very first thing you did, told me on the very first day.
And now it's equal to L times U, but what's happened?
The symmetry is lost.
Somehow the L has ones on the diagonal, the U as we have it has pivots on the diagonal, now the pivots are not all ones.
So you see the symmetry of the problem got lost, and that shouldn't happen.
And there ought to be a way to get back.
Ok. And now I want to describe the way to get back to symmetry.
So LU doesn't keep the symmetry.
L has ones, U has pivots.
Different.
But a very simple idea will bring back the symmetry.
That is peel off the pivots into a diagonal matrix.
In other words, there's a matrix, I'll call it D, D for diagonal.
I'll divide those numbers out of each row.
And can I just do that?
So I'm just going to write this U as a product of this diagonal D where I'm going to be dividing the two out.
So when I divide the two out from that row I'm left with one, minus 1/2, zero.
And when I divide 3/2, the pivot then, it makes that pivot into a one and what does it produce for that guy?
When I divide 3/2, when I divide that minus one by 3/2, what do I get?
I get negative, division will be 2/3, I'll get a negative 2/3.
And now, on the last row I'm dividing that row by 4/3.
When I divide that row by 4/3, what row do I get here?
.
Because I've made the pivots one, well they're not pivots.
What I've done is separate out the pivots.
So I've made the diagonal ones just by separating it out.
And what's happened?
My goal was to get back some symmetry that was there at the start.
Now so I have a pivot matrix D, and what's that matrix?
You could say, well, it's the rest.
But that's not what I'm looking for.
What is it?
Can everybody have a look at it?
I can't raise it.
If you look at what we got there, what is it?
What's the right name to give it?
L transpose, exactly!
That's the right name.
L transpose.
So what am I concluding then?
I'm concluding that, let's see, where shall I put this?
And it'll come back to it.
Well, here we had just to show it wasn't an accident, here we had L, L transpose and what was the pivot matrix in this too beautiful problem case?
It was the identity.
So we didn't notice it.
So can I squeeze in the identity?
That's the pivot matrix there.
But and again, we had L times L transpose.
The beauty was there, the symmetry was there.
And now what's the usual thing?
So really I'm completing this to one more thought.
In that case when A is symmetric.
I'm completing, I have the L. I'm factoring out the D. And what's left is L transpose.
I hope you like LD*L transpose.
Seeing a matrix on one side and the transpose on the other side, the matrix L at the left and L transpose at the right is just right.
So the point of symmetric case we have, and I'll use the letter K rather than A because now we're getting the matrix that's more special.
It's that K or it's this T or it's any other symmetric matrix.
The elimination leads to, uses multipliers L and if I factor out the pivot matrix then the other part is L transpose.
We've seen that just by example.
By two examples.
Now I want to just look at that.
Because this that describes not only the result of elimination which is the key operation, but it also keeps the symmetry.
In fact every matrix of that sort is symmetric.
No, yeah, that's important.
This is sure to be symmetric.
We will often see matrices multiplied by their transpose.
So what I'm saying is that if you gave me any matrix L, any diagonal matrix D, and then the transpose of L, if I multiplied those out, I would get a symmetric matrix.
And going the other way, if I started with a symmetric matrix and I did elimination and got an L, then the D factoring out would leave me L transpose.
So what you've seen by example is what will happen all the time.
Now why is that matrix symmetric?
Here we get a chance to show the power of matrix notation, really.
I just think that if I have any matrix L, in this case it happened to be lower triangular, but if I have any matrix L and I have a nice, symmetric diagonal guy in the middle and I have the transpose of this matrix on the other side I think the result is a symmetric matrix when I multiply.
So it's these symmetric factorizations that we're getting to and are important problems because our important problems are symmetric.
Ok, why is that sure to be symmetric?
Suppose I asked you as a exercise, prove that L times a diagonal times L transpose is always a symmetric matrix.
How could you do that?
How could you do that?
You could certainly create an example that did it and check it out, multiply, it would work.
But we want to see that this is going to be true always.
So how would you do that?
I guess, let me get started.
I would take its transpose.
If I want to show something's symmetric, I transpose it and see if I get the same matrix again.
So let me take the transpose of this.
So I'm answering, Why is it sure to be symmetric?
So let me take K transpose.
So this is the transpose of, I have K equals something times something times something, A times B times C, you could say.
If I transpose a matrix, how can I create transposes out of a, B and C. Do you remember what happens?
They reverse the order.
It's like inverses.
Transposes and inverses both have that key rule.
When you have a product and you invert it, they come in the opposite order.
When you transpose it, they come in the opposite order.
So let me try put these separate transposes in the opposite order.
So I've used the most important fact there.
Which is just a fact about transposing a product.
Ok, what have I got now?
What's L transpose transposed?
It's L, great.
What's L transpose L transposed?
Nothing but L. Transpose twice and I'm back to L. What about D transpose?
Same as D. Because D was symmetric, in fact diagonal.
So what have I learned?
The proof is done.
I've got K back again.
This was the original K. So I've learned that K transpose is K. So you're going to see time after time, let me just put these things there, you're going to see an A transpose times an A.
That's the most important, most highly important multiplication.
Take a matrix, maybe rectangular, multiply by A transpose.
So this matrix is certainly square because A could be m by n. And then A transpose would be n by m and the result would be n by n. So it's certainly square.
But now what's the new property we now know?
It's symmetric.
It's symmetric because if I transpose it, the transpose of A will go on this side, the double transpose will go on this side, but the double transpose is A again, so symmetric.
So I'm plugging away here on symmetric matrices because they're just-- yeah, what does symmetry mean in, yeah, can I just come back to this idea of responses?
And by the way, if this was symmetric, would it's inverse be symmetric?
The answer is yes.
If a matrix is symmetric, it's inverse is symmetric.
These symmetric matrices are a fantastic family.
So I could add that to this.
K inverse will also symmetric without having yet said why.
But maybe in words, I'll just say a few words here at the end.
So what's a typical entry?
Say the 2, 1 entry, just to carry on with this language one more moment.
This is A inverse here.
Now the 2, 1 entry in the inverse is an impulse is in the first, the first mass, whatever gets an impulse.
And that is the response of the second mass.
The response in position two to the impulse in position one.
Now my matrix is symmetric, thinking about symmetric matrices here.
So what about here?
Here, if I take the impulse in position two and look at the response in position one, so do you see the difference?
In general, those could be different.
This is the response at position two to an impulse at one.
This is the response at one to an impulse at two.
You see that I'm multiplying those columns and getting these columns.
And what's the point about symmetry?
Those are the same.
Symmetry is expressing this physical meaning that the response that i, to an impulse at j is the same as the response at j to an impulse at i.
And that's sort of, that's such an important property, you want to notice it.
And it goes into symmetry.
So many, many problems will be symmetric and then some won't.
We'll have to admit this won't cover everything, but it covers such an important and beautiful part.
So that's today's lecture on LU elimination, solving linear systems, and then let's move forward to understanding the actual inverses.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Ok, so this is I could say delta function day.
Break from linear algebra mostly.
So we're looking on another type of right-hand side.
Before in the differential equation and in the difference equation.
So the right-hand sides up to now, the one we looked at was a uniform constant load second derivative equal one.
Now a point load.
Well in a way, we're now solving a whole bunch of problems because the point load can be in different places.
So instead of solving one problem with one on the right-hand side, we're solving with a delta function.
Now a delta function is, you probably have seen and heard the words and seen the symbol, but maybe not done much with a delta function.
It takes a little practice but it's really worth it.
It's a great model of maybe what can't quite happen physically, to have a load acting exactly at a point and nowhere else.
So the delta function is, I drew it's picture, the delta function is zero, this is delta of x is zero except at that one point, the origin, x=0, and then all along back to zero again.
So nothing's happening, no load except at that one point.
And let me just, so there's no hesitation in when I change from x to x-a, what does that do to a graph?
If I have a function of x and I instead shift the function to f(x-a), I shift x to x-a, well in this case, and in all cases, it will just shift the graph.
So if I drew a picture of delta, of x-a, the load now would happen when this is zero, because it's delta at zero is the impulse, and now this is zero at x=a.
In other words, the load moved to the point a.
So there is the shifting load, but the load could fall anywhere between zero and one.
So delta of x, the load actually falls at zero.
Well we don't quite want that load at the boundary.
So let's think of the point a, the load point as somewhere between zero and one.
Can I just take a little time to recall the main fact about delta functions?
When I say recall, it could very well be new to you.
So that's what the delta function, that's my best graph of the delta function.
But of course I'm, in using the word function, I'm kind of breaking the rules because no function, I mean the function is, functions can be zero there, can be zero there, but they're not supposed to be infinite at a single point in between, but this one is.
Let me go back to delta of x to match these figures.
Of course, they would also just shift along by a.
Maybe no harm in that.
Sorry, I'll stay there and now I want to integrate.
And that's when a delta function comes into its own.
It's value of infinity is a little bit uncertain.
What does that mean?
But when we integrate it, what's the key fact about delta function?
That the integral of a delta function from, let's say, let's integrate the whole thing, we can safely start way at the far left and go away to the far right because it's zero all the time there except at one point, and you know.
So what's the area under that spike?
It is one.
That's right.
So that's the fact, the sort of central fact about a delta function .
That the area is one.
Oh well, let me, while I'm really writing down the central fact, let me write it more specifically, more generally.
Suppose I integrate, and this is delta functions now really showing, if I integrate a delta function against some, at times, some nice function.
Now have you ever thought about that?
What would be the answer if I integrate the delta function against some nice function?
So I'm still getting zero from this term all the way along until I hit the spike and then after it goes back to zero again.
So, whatever, it's gotta be at the spike, at x=0, because I put the spike here at zero, the impulse.
So what do you think's the answer for that one?
Yeah, It's the function.
So, yes, tell me again and I'll write it down.
g, it's a value of this function g. We don't care what it is to the left and to the right at zero because it's really at zero that this thing turns on and its value at that point is just, gives us the amplitude of the impulse, which is g(0).
And of course if g is the constant function one, I'm back to that formula.
But this is maybe the thing to watch for.
Actually there's a lot built into that little thing.
We'll come back to that.
So that's delta functions integrated and now here are some pictures.
These are the good pictures.
So here's one integral of the delta function.
It's a step function.
And the step of course will occur at the point a if the integral of the delta function at a point a will be the step function.
Where the action happens.
The jump happens, I could call it a jump function.
At that point a.
Because, just for the reason we said.
That if we integrate, the integral is zero.
And then as soon as our integral passes this point, so this is integral of the, this is, I integrated.
I integrate to get to this picture.
I start with that delta function and I integrate and it suddenly jumps to one as soon as the integral goes past the spike, the impulse.
So a step function.
Very handy function, step function.
Sometimes called a heavy side function named after the guy who, the electrical engineer I think who first sort of work out the rules for using these.
Let's integrate one more time because we have second order equations, second derivatives, so we better integrate twice to see what sort of answer we get.
Now integrate the step function.
So again, the integral is zero all the way to the left, so I'm still getting zero, but now beyond this point I'm integrating one.
And the integral of one is x.
So now that I would call a ramp function.
That's a nice short word for this valuable function.
A ramp function is the function that's zero and then x.
So tell me about that ramp function.
Just think about it.
what happens to its derivative at the point a?
As I run along and I hit this key point, what happens to the derivative of the ramp?
What does the derivative do?
Focus on that ramp now.
What does the derivative do at that point?
It jumps.
The derivative jumps the slope.
Is the derivative, the slope jumps from zero and here the slope is one.
And of course that's what that's telling us.
Here's the picture of the derivative.
What does the second derivative do?
Well, since I integrated twice I guess going back two steps I'll find out what the second derivative is.
So the first derivative takes a jump.
The second derivative is the derivative of that jump, so it's got the impulse.
So the second derivative, it's a straight line here, second derivative a straight line.
This is straight line here, second derivative of a straight line is a straight line.
But at that point the first derivative jumps, the second derivative has that delta function.
In other words, that's that stuff.
If I keep integrating and I don't need higher integrals in today's lecture, another integral would be what?
If I integrate this function, then it's running along the zero.
What's the integral of this?
Doesn't quite turn that steeply.
What's that curve there?
If I've integrated the ramp.
Here is the integral.
First, the next step up, the integral of the ramp would be?
It'll be x squared, yeah, it'll be a parabola.
x squared over two, the integral of that.
And now what do I get when I integrate this one?
I get something very important.
Not important today, but important in a few weeks.
And very useful in computing.
These have turned out to be just the right thing.
So again, I'm integrating that.
Everybody can tell me, what is that?
What's that curve now?
It's the next integral of course.
The area under that will be x cubed over six.
So now that is a function.
Yeah, it's worth maybe just for practice.
What's the deal with that function?
That's pretty smooth function.
Because it certainly passes right, it meets at that point.
The first derivative meets at that point.
The second derivative meets at that point.
The third derivative does what?
Of this line.
The third derivative takes three steps back down the line and you see that the third derivative jumps.
Right?
The third derivative of that is the third derivative, would be, shall I for C, for cubic spline or something, the third derivative will be zero there.
And the third derivative of that is exactly like back to that, back to that, back to one is one.
So the third derivative.
so the cubic spline's so smooth your eye doesn't see that.
They're very useful for drawing many, many purposes.
CAD programs would use such things constantly because they're convenient, they have nice pieces that you can fit together and they fit together very smoothly.
But they really are two separate functions.
So that's up to cubic spline.
But our focus is-- These would solve, what equations would those solve?
Well, that takes how many derivatives to get to a delta?
So what would be the equation?
What would be the right-hand side?
Let me take the fourth derivative.
I'll just ask the question that way.
What would be the fourth derivative of that cubic spline?
A delta, right?
Four steps back.
So what is, physically, what are we seeing here?
Do you recognize what kind?
if I ask now people in mechanics, When will we meet a fourth order equation?
Fourth derivative equals a load.
Anybody know the physical situation where fourth derivative?
Beams, yeah.
It's the equation for a beam.
A beam has, the bending of a beam.
So it's a beam.
This eraser isn't too very much like a beam, but anyway I put the chalk on it, well nothing happened.
Sit on it, whatever.
It'll bend and that bending will be given by a beam equation.
So later we'll meet the beam equation.
So most equations of physics, mechanics, biology, everything are second order, Newton's Laws often the reason.
But we get up to fourth order sometimes.
And very seldom get higher.
Hopefully.
Beams or plates, that table would be a plate and it would have a fourth order equation.
Let's start solving this problem.
What's the solution, what's the general solution to that equation?
Minus the second derivative, so notice the minus that I like, and the load has now moved to the point a.
So the solution u(x), let's write down all solutions.
Tell me one solution, first.
One particular solution.
What is one function for which minus the second derivative would be the delta.
That's what we've got over there.
So just bring that blackboard over here.
Change its sign because that minus, and what are you going to tell me?
Minus a ramp.
Minus a ramp.
And the ramp, of course, will ramp up at the point a so that it's the second derivative of that, the second derivative of R will be delta.
The minus is correct and the point is correct.
Now does that solve our problem?
No.
The ramp is going upwards.
It's not zero.
What am I forgetting?
What do I not yet have?
There's more to this solution.
Just as there was for a uniform load.
What was the more?
Constant and I want two homogeneous solutions, null solutions, two solutions with second derivative equal zero.
One of them is C and the other one is Dx.
That's the whole solution.
So what I want to, I mean we need that C+Dx.
We've got two boundary conditions to satisfy, just as before.
So I need two constants, that'll do it perfectly and I'll get an exact answer.
And so this is a ramp.
Oh yeah.
Before I go further, how would I think about this?
This is a ramp that turns which way?
Down.
Right?
With that minus sign, that ramp turns down at the point x=a.
Right?
It's derivative goes from zero to minus one.
The slope of this guy drops by one because of the minus sign.
Sorry the slope, of the ramp function of minus the ramp.
It goes at zero, drops by one.
And what this is going to do is take that ramp and adjust it to go through the fixed ends.
Oh, let's just do it.
Let's just do it.
What are C and D?
What are C and D?
My point a-- let me draw a graph, that's always the best thing.
Always draw a graph of these solutions.
So let me put in the point a.
So I'm drawing now a picture of the solution from zero to one.
I'll graph it.
What do I have here?
Shall we just plug in the boundary conditions and find C and D?
That's the direct way.
What is C?
C I'm going to plug in.
Hopefully I might find it from just the first boundary condition.
If I'm starting from zero, well this guy certainly starts at zero, right?
The ramp hasn't done anything until it gets to a.
And this guy is certainly zero.
So what is C?
Gone, right.
Now what is D?
Well alright, what's D?
Let's see.
Let me draw the-- so there's a Dx and D won't be zero.
I want that thing to be zero at point one.
So I want to determine D. Let me determine D. So what is minus the ramp at x=1?
I'm plugging in x=1.
Is that right?
I'm going straight forward here.
Plugging in x=1 into this boundary condition, ready for this guy.
What's the ramp?
So it's minus and the ramp is, well if the ramp is shifted over then that's shifted over.
So at x=1, what's the ramp?
How high has that ramp gone?
1-a.
Right?
The ramp is x-a.
At the point x=1 it will be 1-a.
So I think I get 1, -1 - a out of that.
Minus the ramp plus D times what?
One, I'm plugging in x=1.
And that's supposed to equal?
Zero, good.
So I'm doing this sort of the systematic way of writing down the general solution.
Discovering that D, what do I discover D is?
Put it on the other side.
D is 1-a.
And of course, don't forget that it's multiplying the x.
Let me just draw the picture.
Here's how I think about it.
The solution is, away from x=a, what does the solution look like?
To the left of x=a what's my graph going to be?
It's going to be?
A straight line, right?
To the left of here there is no load.
The equation is second derivative equals zero.
The solution to that is a straight line.
In other words, until I get to a, this thing hasn't started.
It's only this straight line.
The solution does something like that.
It's a straight line.
And I guess, actually, that's what it is.
Because the C isn't here and that's all we've got left.
So that's that straight line.
What is it for the second half?
Tell me what the solution looks like in the second half.
In between a and one.
It's going downhill.
Why?
Because it gotta get back to zero.
And how's it going downhill?
It has to be linear.
In this region, has to be linear.
Why?
How do I know it's linear here?
Because one way is to say the equation in that region is second derivative equal zero.
Second derivative equal zero, straight line.
This is my solution.
It's (1-a)x here and it's whatever it is to get back to zero.
What will it take to get back to zero?
Let's see.
Well we could plug in, we've got one expression here.
Or I could just look at that.
I could say, okay what's the equation for the straight line that's at this point, what is the, yeah, it's (1-a)x. I want it to be linear.
I want it to get to zero.
Let's see.
If I want that, it would be great to have 1-x times something.
I have to figure out what.
Because with the 1-x that x=1, that'll drop off.
That's linear.
What number, what's the key here?
That slope, I want to match them up there.
And that's the point x=a.
This is supposed to match that at x=a.
Do you have an idea for what I should take?
What do I put right there?
a.
Look at the symmetry in those two sides.
(1-a)x going up.
(1-x)a x going down.
At x=a it hits that point, right.
So we've solved it.
We could think about this different ways.
I could have got that 1-x, let's see, I could have got it from the formula.
In a way I like to get it from the picture, I see it, sort of, I see the point.
What happened at that point?
What are the jump conditions?
This is another way to ask, to see how the delta function works.
What are they jump conditions?
I want to know, when I ask about jump conditions, I want to know what are the conditions on u(x), the displacement?
What are the conditions on the slope, u'(x).
That'll be the strain when we're speaking about elasticity.
Just for u(x), what's the statement about u(x) from the left and from the right at that critical point, the point of the load.
From the left and from the right u(x) is?
The same.
u(x) matches up.
u(x) from the left is that height.
u(x) from the right is that.
I want to write down those jump conditions.
Because that's another way to see this.
u(x), u(a) from the left should equal u-- do you want me to say u is continuous?
I'll just say it in words.
u(x) is continuous, that just means it doesn't jump, at x=a.
So that's, you could say that's a non-jump condition.
The function itself doesn't jump.
Why not?
Because we're talking about some elastic bar on which we put a point load.
The thing isn't going to break.
The displacement is going to be continuous.
But what's the condition on u'(x), the derivative, the slope?
So that's the function and now tell me what's the deal on the slope?
What's the comparison between the-- I have a slope of whatever it is going along here and I have a slope of a new slope.
So u'(x), the slope jumps, right?
And how much does it jump?
Minus one.
It drops by one.
The slope, because of my minus.
So this tells me that, yeah, let me write that down.
u'(x) drops by one.
This is another way to say what the equation is asking.
The equation is looking for two pieces of straight lines that meet at a but their slope drops by one.
By the way, what were the slopes?
It's good to graph the slopes, too.
Let me graph the slopes.
The slope u', the derivative du/dx.
What's the slope here?
Slope is 1-a at this point, right?
The derivative is 1-a along here.
So slope is 1-a.
And now at x=a the slope changes to this one.
And what's the slope of that second part?
Minus a.
Look.
It did it right.
Minus a is the slope along here.
Do you see 1-a?
It dropped by one.
The one disappeared to leave a slope of minus a. I guess if I just imagine a bar.
I'm fixing it at both ends.
There's a bar.
I'm just thinking for people who like to see a physical picture of what's happening, that's what this is, we'll do it properly very, very soon.
I've got a bar.
It's a very light bar.
It's weight is not a problem here.
But it's got a load at the point.
So I'll measure a going downwards.
And at the point x=a I'm hanging a heavy load.
A load.
How do I draw a load?
Maybe I'll make a big weight or something.
What's going to happen to this dumb bar when I do that?
Just tell me physically.
What's going to happen?
What's going to happen above the load?
It's going to stretch, right, tension.
The load is going to pull the bar down, it's going to stretch this part.
And because nothing special is happening, it's going to stretch it linearly.
And then what's going to happen below the load?
Compression.
So the slope will go negative.
And nothing special happened so the slope will be negative but it'll be constant.
The slope will drop from this to this.
The displacement, that point will go down a little bit.
That little bit it goes down is actually the height of this, because that's the displacement.
It'll go down a little bit.
It'll stretch above, it'll compress below, and we see that in that picture of the displacement.
The displacement's all down.
Right?
Displacement, you know, nature is still going to, all the bar is going to move down.
That's why this function doesn't, this function, the displacement function is positive.
It goes all down.
But the slope function is positive here, so tension is positive slope.
Stretch and compression is negative.
Well all that to solve this equation.
Maybe while we're on a roll, let's solve the free-fixed guy.
So this is our-- might as well be systematic.
This with the fixed-fixed problem.
Let me below it solve the free-fixed problem.
So it'll be u''.
That's the second derivative equals delta at x-a.
Same set up.
But now the top end is, so it's free at the top.
What does that mean?
Slope is zero at the top but is still fixed at the bottom.
So this will be now free-fixed.
Let me go straight to the picture.
Let me go straight to the picture of u(x).
So there is x=0, there's x=1, here's the load at a.
What's up?
And while you're thinking about that, let me draw a picture to match this picture.
A bar fixed at the bottom but not at the top.
And it's got its load here hanging down.
But let's do it math first, and then check with the picture.
What do we got, two or three ways now to try to get the answer?
The systematic way would be to write down this solution and plug in the two boundary conditions.
That'd be a straightforward way.
Yeah, we could even start by that.
So u(x) is the particular solution, the ramp plus any Cx+D.
And just plug in x=0 that'll be easy.
If I plug in x=0 in the free condition, what does that tell me?
At x=0, this corner, this ramp hasn't started so the slope is zero.
The slope of the constant is zero.
What do I learn from this boundary condition?
u'(0)=0.
That C is zero.
Before I learned that D was zero, but now from that condition I'm going to learn C is zero.
Do the picture for me.
Do the picture for me.
What's the graph of-- this is a graph of u(x).
Remember now it starts from zero slope because it's free at the top.
What does the graph look like in the first part?
It's a straight line, has to be a straight line because there's no force.
And what kind of a line?
It's going to be horizontal because it starts off horizontal.
The slope has to be zero at zero and nothing changes until a.
So it comes along there.
Right?
Now I've started out with the right, left, the correct boundary condition at zero, which was no slope.
And now what's it going to do the other half?
From a to one.
It's going to be again, it'll be a straight line, right?
Because there's no force there.
And what happens at-- all the action of course is at this point a, and what action is it?
Tell me what sort of a line.
How do I finish the picture?
What do I do?
I start here, right?
Because the bar's not falling apart.
u is continuous.
I don't get a gap suddenly.
And now what do I do from there?
Only thing I can possibly do, because I have to end up here and it has to be a straight line, that's it.
That's what the picture will have to look like.
What does that correspond to in the picture for the bar?
Well what happens with this bar?
Above the weight what's what happens to this top part of the bar in that picture?
And what happens to the lower part of the bar?
So this was at the point x=a, this is x=0, this is x=1.
What happens above the bar, above the weight?
It just, a rigid motion, just goes down.
Because what happens below the weight?
The same compression or compression still happening.
This is still squeezed.
Shall I try to draw it?
So this is after the weight.
This got squeezed but this part did not get squeezed.
And that's what we're seeing here.
A fixed displacement.
So this means, that picture means that all the pieces of the bar here got moved down by the same amount, whatever this, we don't know that number yet.
And then below it they got compressed.
Well we're almost there but we don't yet have that solution.
Come back to this picture.
u(x) is continuous, got it.
And what's the real condition that's going to determine where we are, what those heights are, the numbers in there.
It's gotta look like that, but we get more than that, we gotta no what are the actual, what is that height.
What is this?
What's the slope?
Here the slope is zero.
Here the slope is what?
What's the slope in the second part?
That's the key.
And you know what it has to be because what happens to the slope?
If I have the second derivative as a delta function with that minus sign, the slope drops by one.
And the slope here is zero, so the slope here is minus one.
And now it has to get through there, so what is the function?
What's the function that has a slope of minus one and comes down to zero?
It's gotta have a minus x in it and what's the constant to make it come out right?
What do I write now here for u(x).
1-x.
That has a slope of minus one, the derivative is minus one, at x=1 it comes to zero, that's it.
And what do I write, what's u(x) up here?
And therefore, right there?
What's the displacement there, of all this bit that moves down, how much does it move down?
1-a.
Why 1-a?
That's the right answer.
1-a.
Why's that?
Because it had to match up at x=a.
At x=a, this and that match up.
At x=a, that slope, that function and that function match up.
So the slope picture is zero and, oh I'm sorry, can't draw it because I'm at the bottom of the board.
The slope picture, maybe I can draw it here, the slope picture is zero along here and then it drops by one to 1-a.
So that's a picture of u'.
Zero and minus one.
This is the thing to look at.
That's hard work.
When you're seeing delta functions the first time.
But of course the functions did not get complicated.
We kept a clean example.
And which we matched up with a figure and we've got the answer and we've got a couple of ways to do it.
One is this standard, systematic, plug-in boundary condition way.
The other way is this.
u(x) does something here, then the slope has to drop by one.
And that's the key to everything with a boundary condition.
So in a way, we have a piece to the left and a piece to the right.
Two constants here, two constants here, and somewhere there are four conditions that settle those four constants.
You know, we could have a straight line here, a straight line here, that's two and two.
But what are the four conditions that settle those four constants?
Well we have a boundary condition here, that's one.
Boundary condition here is two.
We need two more conditions to settle the two pairs of constants, and there they are.
Two conditions at the jump, at the discontinuity.
Now I've got to do the discrete case.
Are you up for the discrete case?
The case where we're doing, we have a difference equation, so we're doing KU equal a column of the identity.
Column of I.
Let me take a specific column.
Say, .
Let's suppose we have five.
I'm going to draw a picture now.
We have five because I made it five by five.
One, two, three, four, five, here is zero and here is six.
So h is 1/(5+1), 1/6, that's the delta x.
So what does my equation say?
Remember what K is.
U is the n u_1, u_2, u_3, u_4, and u_5, the unknowns.
K is our old friend with twos and minus ones and minus ones.
I'm going to find the solution.
And this'll be the solution that has a load at this point.
This is like my point a, right?
Here in the continuous case, a could run anywhere between zero and one.
In the discrete case, I've got five possible load points and I've picked the second one.
Five columns of the identity matrix, five places to put that one, I put it there.
Now can I draw the picture here?
Which should we do first?
Should we do free-fixed?
Because that came out even easier than fixed-fixed.
Notice the solution here had two parts.
This is the way I would write that answer.
Because you could draw a picture, but if you want to write the formula, what would I do?
I would break it into two pieces.
1-a up to the point a because that's what it was running along here.
And then down here it was 1-x, x>=a.
That's important to mention.
You have to have some guidance on how to write the answer.
And when the answer has two parts, this is a good way to write it, in two parts.
It's a little too, you're compressing it too much to write, to use that ramp function.
Better to split it apart into before a and after a.
What's going to happen over here?
Oh yeah, can we take a shot at this problem?
And let me mention again in the review that'll be in here this afternoon and every Wednesday afternoon I'll just be ready for questions.
Please bring questions.
They can be questions on the homework.
Even better if they're questions on other problems, questions on the lecture.
Questions are essential to make that help session helpful.
What do you think's cooking here?
At a typical, somewhere in the middle here, I'm going to draw the u's.
Shall I just draw them?
And now what's my condition?
I gotta put the boundary conditions on.
Oh, I have put the boundary conditions on it.
By putting that two there, I'm up to here.
Ok, let's do that one.
When I chose K and put a two in there I was picking the fixed-fixed boundary condition.
So can I just say it's going to be beautiful.
The solution over there is going to look like this.
The solution over here is going to be up, up, up.
It's going to be a straight line but only points in a line and it'll be straight line down.
That value, that value, that value.
Those will be u_1, u_2, u_3, u_4, and u_5.
And once more, this is going to drop by one again.
Actually I didn't have to redraw the picture.
It falls right on.
In case x is 2/6 so that it fits that picture, I'm claiming we have another extremely lucky case.
If we can use the word lucky for math.
That I'm claiming that the way, you remember for the uniform load with a one, when we had second derivative equal one, the solution was a perfect parabola and the discrete solution, the difference equation was right on the parabola for this fixed-fixed case.
It's going to happen again.
It won't always happen.
Those are the only two important right-hand sides I know.
They're the two most important right-hand sides and those are the two lucky ones.
If we have a constant that lies right on a parabola, if we have a delta function, it lies right on a ramp.
And there it is.
So that's what the solution looks like.
Now, I have to figure out what these numbers are, I guess.
Yes, what are those numbers?
Oh, well.
Actually, if it falls right on, I know the numbers.
So a is 2/6.
So let me keep 2/6.
So a is 2/6.
That's that value.
So let me say what I think U is.
So this was a picture of U.
That's u_1, 2, 3, 4, and 5 and now I think it lies right on that.
So it's going to be (1-2/6)x going up and (1-x)2/6 going down.
My point is that I'll be able to figure out what that-- this is u, this is the u.
You're going to say, why?
Let me pause before putting in numbers and say why is it, how do I know that the solution is right on the function, the continuous solution.
Well, can I draw a set of pictures just like those guys for discrete?
Yeah, let me just draw those for discrete here.
That shows you the magic.
So there is a, I'm going to draw a vector now.
I'm going to have to lift the chalk, it won't be a function and it'll be the delta vector.
So it'll be the delta vector, delta with, so there is point one.
Zero, one, two, up to six.
It'll be the delta vector.
Well if I just draw the delta vector, the delta vector has a one there.
So this is the delta vector.
Do I need?
Well you can see that the delta vector is now going to be the vector of all zeroes and it's got a one at the key, at the impulse and then zero.
So it's a discrete impulse.
That would be a better word.
Discrete impulse.
Impulse at zero.
So let's stay with an impulse at zero.
Alright.
What's my next picture?
Again let me put in zero.
One, two, three, onwards.
Minus one, so on.
What do I want to do now?
What do I draw second?
I always look over here.
What did I draw second over here?
The step.
Now why did I draw a step function?
How did I get from here to here?
I integrate.
I took the integral.
So how will I get from here to this picture?
I don't integrate, I add, sum.
So coming along from the left, all these all along here, this sum is all zero because it was always zero.
So it's zero, zero, zero, zero.
And then, whoops, wait a minute.
It says it a one there?
Yeah, I think it must be.
So here it wasn't a zero, wrong.
Here it's a one.
And what is it next?
What's next to it?
One, because I'm adding more and more zeroes but I have that one now, okay.
A discrete step.
It's a discrete step, zeroes and then ones.
Now comes the second.
So what am I going to call that?
A step, right?
It'll be a step function, step vector.
If the sums of the delta vector gave me the step vector, how do I go the other way?
What do I do to the step vector to get back to the delta vector?
Differences, right?
Sums in one direction, differences in the other.
So the differences of the step vector are the delta vector.
The step is the sum of the deltas and the delta is the differences of the step.
Now for the crucial next guy.
What's it going to be?
I add.
Wait a minute.
What's up?
I'm looking for that picture.
Do I get it?
Yeah, I hope so.
Oh, look, we ran out of time.
I don't have to do this, but I will.
So as I add I get zeroes and then it's one, and then I add on one more one.
Look.
You see what's happening.
I right along at zero but I'm going to look at the book to see whether that jump should come here or here.
So I've got a little bit of this to finish next time and I'm open for any questions this afternoon.
Okay, thanks and sorry to keep you late.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Shall we start?
The main job of today is eigenvalues and eigenvectors.
The next section in the book and a very big topic and things to say about it.
I do want to begin with a recap of what I didn't quite finish last time.
So what we did was solve this very straightforward equation.
Straightforward except that it has a point source, a delta function.
And we solved it, both the fixed-fixed case when a straight line went up and back down and in the free-fixed case when it was a horizontal line and then down with slope minus one.
And there are different ways to get to this answer.
But once you have it, you can look at it and say, well is it right?
Certainly the boundary conditions are correct.
Zero slope went through zero, that's good.
And then the only thing you really have to check is does the slope drop by one at the point of the impulse?
because that's what this is forcing us to do.
It's saying the slope should drop by one.
And here the slope is 1-a going up.
And if I take the derivative, it's -a going down.
1-a dropped to -a, good.
Here the slope was zero.
Here the slope was minus one, good.
So those are the right answers.
And this is simple, but really a great example.
And then, what I wanted to do was catch the same thing for the matrices.
So those matrices, we all know what K is and what T is.
So I'm solving, I'm really solving K inverse equal identity.
That's the equation I'm solving.
So I'm looking for K inverse and trying to get the columns of the identity.
And you realize the columns of the identity are just like delta vectors.
They've got a one in one spot, they're a point load just like this thing.
So can I just say how I remember K inverse?
I finally, you know, again there are different ways to get to it.
One way is MATLAB, just do it.
But I guess maybe the whole point is, the whole point of these and the eigenvalues that are coming too, is this.
That we have here the chance to see important, special cases that work out.
Normally we don't find the inverse, print out the inverse of a matrix.
It's not nice.
Normally we just let eig find the eigenvalues.
Because that's an even worse calculation, to find eigenvalues, in general.
I'm talking here about our matrices of all sizes n by n. Nobody finds the eigenvalues by hand of n by n matrices.
But these have terrific eigenvalues and especially eigenvectors.
So in a way this is a little bit like, typical of math.
That you ask about general stuff or you write the equation with a matrix A.
So that's the general information.
And then there's the specific, special guys with special functions.
And here there'll be sines and cosines and exponentials.
Other places in applied math, there are Bessel functions and Legendre functions.
Special guys.
So here, these are special.
And how do I complete K inverse?
So this four, three, two, one.
Let me complete T inverse.
You probably know T inverse already.
So T, this is, four, three, two, one, is when the load is way over at the far left end and it's just descending.
And now I'm going to, let me show you how I write it in.
Pay attention here to the diagonal.
So this will be three, three, two, one.
Do you see that's the solution that's sort of like this one?
That's the second column of the inverse so it's solving, I'm solving T times T inverse equals I here.
It's the second column is the guy with a one in the second place.
So that's where the load is, in position number two.
So I'm level, three, three up to that load.
And then I'm dropping after the load.
What's the third column of T inverse?
I started with that first column and I knew that the answer would be symmetric because T is symmetric, so that allowed me to write the first row.
And now we can fill in the rest.
So what do you think, if the point load is now, I'm looking at the third column, third column of the identity, the load has moved down to position number three.
So what do I have there and there?
Two and two.
And what do I have last?
One.
It's dropping to zero.
You could put zero in green here if you wanted.
Zero is the unseen last boundary, you know, row at this end.
And finally, what's happening here?
What do I get from that?
All one, one, one to the diagonal.
And then sure enough it drops to zero.
So this would be a case where the load is there.
It would be one, one, one, one and then boom.
No, it wouldn't be.
It'd be more like this.
One, one, one, one and then down to.
That's a pretty clean inverse.
That's a very beautiful matrix.
Don't you admire that matrix?
I mean, if they were all like that, gee, this would be a great world.
But of course it's not sparse.
That's why we don't often use the inverse.
Because we had a sparse matrix T that was really fast to compute with.
And here, if you tell me the inverse, you've actually slowed me down.
Because you've given me now a dense matrix, no zeroes even and multiplying T inverse times the right side would be slower than just doing elimination.
Now this is the kind of more interesting one.
Because this is the one that has to go up to the diagonal and then down.
So let me, can I fell in what I the way this one goes?
I'm going upwards to the diagonal and then I'm coming down to zero.
Remember that I'm coming down to zero on this K. So Zero, zero, zero, zero is kind of the row number.
If that's row number zero, here's one, two, three, four, the real thing.
And then row five is getting back to zero again.
So what do you think, finish the rest of that column.
So you're telling me now the response to the load in position two.
So it's going to look like this.
In fact, it's going to look very like this.
There's the three and then this is in position two.
And then I'm going to have something here and something here and it'll drop to zero.
What do I get?
Four, two.
Six, four, two, zero.
It's dropping to Zero.
I'm going to finish this in but then I'm going to look back and see have I really got it right.
How does this go now?
Two, let's see.
Now it's going up from zero to two to four to six.
That's on the diagonal.
Now it starts down.
It's got to get to zero, so that'll be a three.
Here is a one going up to two to three to four.
Is that right?
And then dropped fast to zero.
Is that correct?
Think so, yep.
Except, wait a minute now.
We've got the right overall picture.
Climbing up, dropping down.
Climbing up, dropping down.
Climbing up, dropping down.
All good.
But we haven't yet got, we haven't checked yet that the change in the slope is supposed to be one.
And it's not.
Here the slope is like, three, It's going up by threes and then it's going down by twos.
So we've gone from going up at a slope of three to down to a slope of two.
Up three, down just like this.
But that would be a change in slope of five.
Therefore there's a 1/5.
So this is going up with a slope of four and down with a slope of one.
four dropping to one when I divide by the five that's what I like.
Here is up by twos, down by threes, again it's a change of five so I need the five.
Up by ones, down by four.
Sudden, that's a fast drop of four.
Again, the slope changed by five, dividing by five, that's got it.
So that's my picture.
You could now create K inverse for any size.
And more than that, sort of see into K inverse what those numbers are.
Because if I wrote the five by five or six by six doing it a column at a time, it would look like a bunch of numbers.
But you see it now.
Do you see the pattern?
Right.
This is one way to get to those inverses, and homework problems are offering other ways.
T, in particular, is quite easy to invert.
Do I have any other comment on inverses before the lecture on eigenvalues really starts?
Maybe I do have one comment, one important comment.
It's this, and I won't develop it in full, but let's just say it.
What if the load is not a delta function?
What if I have other loads?
Like the uniform load of all ones or any other load?
What if the discrete load here is not a delta vector?
I now know the responses to each column of the identity, right?
If I put a load in position one, there's the response.
If I put a load in position two, there is the response.
Now, what if I have other loads?
Let me take a typical load.
What if the load was, well, the one we looked at before.
If the load was .
So that I had, the bar was hanging by its own weight, let's say.
In other words, could I solve all problems by knowing these answers?
That's what I'm trying to get to.
If I know these special delta loads, then can I get the solution for every load?
Yes, no?
What do you think?
Yes, right.
Now with this matrix it's kind of easy to see because if you know the inverse matrix, well you're obviously in business.
If I had another load, say another load f for load, I would just multiply by K inverse, no problem.
But I want to look a little deeper.
Because if I had other loads here than a delta function, obviously if I had two delta functions I could just combine the two solutions.
That's linearity that we're using all the time.
If I had ten delta functions I could combine them.
But then suppose I had instead of a bunch of spikes, instead of a bunch of point loads, I had a distributed load.
Like all ones, how could I do it?
Main point is I could.
Right?
If I know these answers, I know all answers.
If I know the response to a load at each point, then-- come back to the discrete one.
What would be the answer if the load was ?
Suppose I now try to solve the equation Ku=ones(4,1), so all ones.
What would be the answer?
How would I get it?
I would just add the columns.
Now why would I do that?
Right.
Because this, the right-hand side, the input is the sum of the four columns, the four special inputs.
So the output is the sum of the four outputs, right.
In other words, as you saw, we must know everything.
And that's the way we really know it.
By linearity.
If the input is a combination of these, the output is the same combination of those.
Right.
So, for example, in this T case, if input was, if I did Tu=ones, I would just add those and the output would be .
That would be the output from .
And now, oh boy.
Actually, let me just introduce a guy's name for these solutions and not today show you.
You have the idea, of course.
Here we added because everything was discrete.
So you know what we're going to do over here.
We'll take integrals , right?
A general load will be an integral over point loads.
That's the idea.
A fundamental idea.
That some other load, f(x) is an integral of these guys.
So the solution will be the same integral of these guys.
Let me not go there except to tell you the name, because it's a very famous name.
This solution u with the delta function is called the Green's function.
So I've now introduced the idea, this is the Green's function.
This guy is the Green's function for the fixed-fixed problem.
And this guy is the Green's function for the free-fixed problem.
And the whole point is, maybe this is the one point I want you to sort of see always by analogy.
The Green's function is just like the inverse.
What is the Green's function?
The Green's function is the response at x to the u(x) when the input, when the impulses is at a.
So it sort of depends on two things.
It depends on the position a of the input and it tells you the response at position x.
And often we would use the letter G for Green's.
So it depends on x(a).
And maybe I'm happy if you just sort of see in some way what we did there is just like what we did here.
And therefore the Green's function must be just a differential, continuous version of an inverse matrix.
Let's move on to eigenvalues with that point sort of made, but not driven home by many, many examples.
Question, I'll take a question, shoot.
Why did I increase zero, three, six and then decrease six?
Well intuitively it's because this is copying this.
What's wonderful is that it's a perfect copy.
I mean, intuitively the solution to our difference equation should be like the solution to our differential equation.
That's why if we have some computational, some differential equation that we can't solve, which would be much more typical than this one, that we couldn't solve it exactly by pencil and paper, we would replace derivatives by differences and go over here and we would hope that they were like pretty close.
Here they're right, they're the same.
Oh the other columns?
Absolutely.
These guys?
Zero, two, four, six going up.
Six, three, zero coming back.
So that's a discrete thing of one like that.
And then the next guy and the last guy would be going up one, two, three, four and then sudden drop.
Thanks for all questions.
I mean, this sort of, by adding these guys in, the first one actually went up that way.
You see the Green's functions.
But of course this has a Green's function for every a. x and a are running all the way from zero to one.
Here they're just discrete positions.
Thanks.
So playing with these delta functions and coming up with this solution, well, as I say, different ways to do it.
I worked through one way in class last time.
It takes practice.
So that's what the homework's really for.
You can see me come up with this thing, then you can, with leisure, you can follow the steps, but you've gotta do it yourself to see.
Eigenvalues and, of course, eigenvectors.
We have to give them a fair shot.
Square matrix.
So I'm talking about general, what eigenvectors and eigenvalues are and why do we want them.
I'm always trying to say what's the purpose, you know, not doing this just for abstract linear algebra.
We do this, we look for these things because they tremendously simplify a problem if we can find it.
So what's an eigenvector?
The eigenvalue is this number, lambda, and the eigenvector is this vector y.
And now, how do I think about those?
Suppose I take a vector and I multiply by A.
So the vector is headed off in some direction.
Here's a vector v. If I multiply, and I'm given this matrix, so I'm given the matrix, whatever my matrix is.
Could be one of those matrices, any other matrix.
If I multiply that by v, I get some result, Av.
What do I do?
I look at that and I say that v was not an eigenvector.
Eigenvectors are the special vectors which come out in the same direction.
Av comes out parallel to v. So this was not an eigenvector.
Very few vectors are eigenvectors, they're very special.
Most vectors, that'll be a typical picture.
But there's a few of them where I've a vector y and I multiply by A.
And then what's the point?
Ay is in the same direction.
It's on that same line as y.
It could be, it might be twice as far out.
That would be Ay=2y.
It might go backwards.
This would be a possibility, Ay=-y.
It could be just halfway.
It could be, not move at all.
That's even a possibility.
Ay=0y.
Count that.
Those y's or eigenvectors and the eigenvalue is just, from this point of view, the eigenvalue has come in second because it's, so y was a special vector that kept its direction.
And then lambda is just the number, the two, the zero, the minus one, the 1/2 that tells you stretching, shrinking, reversing, whatever.
That's the number.
But y is the vector.
And notice that if I knew y and I knew it was an eigenvector, then of course if I multiply by A, I'll learn the eigenvalue.
And if I knew an eigenvalue, you'll see how I could find the eigenvector.
Problem is you have to find them both.
And they multiply each other.
So we're not talking about linear equations anymore.
Because one unknown is multiplying another.
But we'll find a way to look to discover eigenvectors and eigenvalues.
I said I would try to make clear what's the purpose.
The purpose is that in this direction on this y line, line of multiples of yA is just acting like a number.
A is some big n by n, 1,000 by 1,000 matrix.
So a million numbers.
But on this line if we find an eigenline you could say, an eigendirection in that direction, all the complications of A are gone.
It's just acting like a number.
So in particular we could solve 1,000 differential equations with 1,000 unknown u's with this 1,000 by 1,000 matrix.
We can find a solution and this is where the eigenvector eigenvalue are going to pay off.
You recognize this.
Matrix A is of size 1,000.
And u is a vector of 1,000 unknowns.
So that's a system of 1,000 equations.
But if we have found an eigenvector and it's eigenvalue then the equation will, if it starts in that direction it'll stay in that direction and the matrix will just be acting like a number.
And we know how to solve U'=lambda*u.
That one by one scalar problem we know how to solve.
The solution to that is e to the lambda*t. And of course it could have a constant do that.
Don't forget that these equations are linear.
If I multiply it, if I take 2e^(lambda*t), I have a two here and a two here and it's just as good.
So I better allow that as well.
A constant times e^(lambda*t)y.
Notice this is a vector.
It's a number times a number, the growth.
So the lambda is now, for the differential equation, the lambda, this number lambda is crucial.
It's telling us whether the solution grows, whether it decays, whether it oscillates.
And we're just looking at this one normal mode, you could say normal mode for eigenvector y.
We certainly have not found all possible solutions.
If we have an eigenvector, we found that one.
And there's other uses and then, let me think.
Other uses, what?
So let me write again the fundamental equation, Ay=lambda*y.
So that was a differential equation.
Going forward in time.
Now if we go forward in steps we might multiply by A at every step.
Tell me an eigenvector of A squared.
I'm looking for a vector that doesn't change direction when I multiply twice by A.
You're going to tell me it's y. y will work.
If I multiply once by A I get lambda times y.
When I multiply again by A I get lambda squared times y.
You see eigenvalues are great for powers of a matrix, for differential equations.
The nth power will just take the eigenvalue to the nth.
The nth power of A will just have lambda to the nth there.
You know, the pivots of a matrix are all messed up when I square it.
I can't see what's happening with the pivots.
The eigenvalues are a different way to look at a matrix.
The pivots are critical numbers for steady-state problems.
The eigenvalues are the critical numbers for moving problems, dynamic problems, things are oscillating or growing or decaying.
And by the way, let's just recognize since this is the only thing that's changing in time, what would be the, I'll just go down here, e^(lambda*t).
Let's just look and see.
When would I have decay?
Which you might want to call stability.
A stable problem.
What would be the condition on lambda to get for this to decay.
Lambda less than zero.
Now there's one little bit of bad news.
Lambda could be complex.
Lambda could be 3+4i.
It can be a complex number, these eigenvalues even if A is real.
You'll say, how'd that happen, let me see?
I didn't think.
Well, let me finish this thought.
Suppose lambda was 3+4i.
So I'm thinking about what would either the lambda*t do in that case?
So this is small example.
If I had lambda (3+4i)t. What does that do as time grows?
It's going to grow and oscillate.
And what decides the growth?
The real part.
So it's really the decay or growth is decided by the real part.
The three, either the 3t, that would be a growth.
Let me put growth.
And that would be, of course, unstable.
And that's a problem when I have a real part of lambda bigger than zero.
And then if lambda has a zero real part, so it's pure oscillation, let me just take a case like that.
So e^(4it).
So that would be, oscillating, right?
It's cos(4t) + i*sin(4t), it's just oscillating.
So in this discussion we've seen growth and decay.
Tell me that parallels because I'm always shooting for the parallels.
What about the growth of A?
What matrices, how can I recognize a matrix whose powers grow?
How can I recognize a matrix whose powers go to zero?
I'm asking you for powers here, over there for exponentials somehow.
So here would be A to higher and higher powers goes to zero, the zero matrix.
In other words, when I multiply, multiply, multiply by that matrix I get smaller and smaller and smaller matrices, zero in the limit.
What do you think's the test on the lambda now?
So what are the eigenvalues of A to the K?
Let's see.
If A had eigenvalues lambda, A squared will have eigenvalues lambda squared, A cubed will have eigenvalues lambda cubed, A to the thousandth will have eigenvalues lambda to the thousandth.
And what's the test for that to be getting small?
Lambda less than one.
So the test for stability will be in the discrete case.
It won't be the real part of lambda, it'll be the size of lambda less than one.
And growth would be the size of lambda greater than one.
And again, there'd be this borderline case when the eigenvalue has magnitude exactly one.
So you're seeing here and also here the idea that we may have to deal with complex numbers here.
We don't have to deal with the whole world of complex functions and everything but it's possible for complex numbers to come in.
Well while I'm saying that, why don't I give an example where it would come in.
This is going to be a real matrix with complex eigenvalues.
Complex lambdas.
It'll be an example.
So I guess I'm looking for a matrix where y and Ay never come out in the same direction.
For real y's I know, okay, here's a good matrix.
Take the matrix that rotates every vector by 90 degrees.
Or by theta.
But let's say here's a matrix that rotates every vector by 90 degrees.
I'm going to raise this board and hide it behind there in a minute.
I just wanted to just to open up this thought that we will have to face complex numbers.
If you know how to multiply two complex numbers and add them, you're ok.
This isn't going to turn into a big deal.
But let's just realize that suppose that matrix, if I put in a vector y and I multiply by that matrix, it'll turn it through 90 degrees.
So y couldn't be an eigenvector.
That's the point I'm trying to make.
No real vector could be the eigenvector of a rotation matrix because every vector gets turned.
So that's an example where you'd have to go to complex vectors.
and I think if I tried the vector 1i, so I'm letting the square root of minus one into here, then I think it would come out.
If I do that multiplication I get minus i.
And I get one.
And I think that this is, what is it?
This is probably minus i times that.
So this is minus i times the input.
No big deal.
That was like, you can forget that.
It's just complex numbers can come in.
Now let me come back to the main point about eigenvectors.
Things can be complex.
So the main point is how do we use them?
And how many are there?
Here's the key.
A typical, good matrix which includes every symmetric matrix, so it includes all of our examples and more, if it's of size 1,000, it will have 1,000 different eigenvectors.
And let me just say for our symmetric matrices those eigenvectors will all be real.
They're great, the eigenvectors of symmetric matrices.
Oh, let me find them for one particular symmetric matrix.
Say this guy.
So that's a matrix.
two by two.
How many eigenvectors am I now looking for?
Two.
You could say, how do I find them?
Maybe with a two by two, I can even just wing it.
We can come up with a vector that is an eigenvector.
Actually that's what we're going to do here is we're going to guess the eigenvectors and then we're going to show that they really are eigenvectors and then we'll know the eigenvalues and it's fantastic.
So like let's start here with the two by two case.
Anybody spot an eigenvector?
Is  an eigenvector?
Try .
What comes out of ?
Well that picks the first column, right?
That's how I see multiplying by .
That says take one of the first column.
And is it an eigenvector?
Yes, no?
No.
This vector is not in the same direction as that one.
No good.
Now can you tell me one that is?
You're going to guess it.
.
try .
Do the multiplication and what do you get?
Right?
If I input this vector y, what do I get out?
Actually I get y itself.
Right?
The point is it didn't change direction, and it didn't even change length.
So what's the eigenvalue for that?
So I've got one eigenvalue now, one eigenvector.
.
And I've got the eigenvalue.
So here are the vectors, the y's.
And here are the lambdas.
And I've got one of them and it's one, right?
Would you like to guess the other one?
I'm only looking for two because it's a two by two matrix.
So let me erase here, hope that you'll come up with another one.
is certainly worth a try.
Let's test it.
If it's an eigenvector, then it should come out in the same direction.
What do I get when I do that?
So I do that multiplication.
Three and I get three and minus three, so have we got an eigenvector?
Yep.
And what's, so if this was y, what is this vector?
3y.
So there's the other eigenvector is  and the other eigenvalue is three.
So we did it by spotting it here.
MATLAB can't do it that way.
It's got to figure it out.
But we're ahead of MATLAB this time.
So what do I notice?
What do I notice about this matrix?
It was symmetric.
And what do I notice about the eigenvectors?
If I show you those two vectors,  and , what do you see there?
They're orthogonal.
is orthogonal to , perpendicular is the same as orthogonal.
These are orthogonal, perpendicular.
I can draw them, of course and see that.
will go, if this is one, it'll go here.
So that's .
And  will go there, it'll go down, this would be the other one.
.
So there's y_1.
There's y_2.
And they are perpendicular.
But of course I don't draw pictures all the time.
What's the test for two vectors being orthogonal?
The dot product.
The dot product.
The inner product.
y transpose, y_1 transpose * y_2.
Do you prefer to write it as y_1 with a dot, y_2?
This is maybe better because it's matrix notation.
And the point is orthogonal, the dot product is zero.
So that's good.
Very good, in fact.
So here's a very important fact.
Symmetric matrices have orthogonal eigenvectors.
What I'm trying to say is eigenvectors and eigenvalues are like a new way to look at a matrix.
A new way to see into it.
And when the matrix is symmetric, what we see is perpendicular eigenvectors.
And what comment do you have about the eigenvalues of this symmetric matrix?
Remembering what was on the board for this anti-symmetric matrix.
What was the point about that anti-symmetric matrix?
It's eigenvalues were imaginary actually, an i there.
Here is the opposite.
What's the property of the eigenvalues for a symmetric matrix that you just guess?
They're real.
They're real.
Symmetric matrices are great because they have real eigenvalues and they have perpendicular eigenvectors and actually, probably if a matrix has real eigenvalues and perpendicular eigenvectors, it's going to be symmetric.
So symmetry is a great property and it shows up in a great way in this real eigenvalue, real lambdas, and orthogonal y's shows up perfectly in the eigenpicture.
Here's a handy little check on the eigenvalues to see if we got it right.
Course we did.
That's one and three we can get.
But let me just show you two useful checks if you haven't seen eigenvalues before.
If I add the eigenvalues, what do I get?
Four.
And I compare that with adding down the diagonal of the matrix.
Two and two, four.
And that check always works.
The sum of the eigenvalues matches the sum down the diagonal.
So that's like, if you got all the eigenvalues but one, that would tell you the last one.
Because the sum of the eigenvalues matches the sum down the diagonal.
You have no clue where that comes from but it's true.
And another useful fact.
If I multiply the eigenvalues what do I get?
Three?
And now, where do you see a three over here?
The determinant.
4-1=3.
Can I just write those two facts with no idea of proof.
The sum of the lambdas, I could write "sum" this is for any matrix, the sum of the lambdas is equal to the, it's called the trace of the matrix.
The trace of the matrix is the sum down the diagonal.
And the product of the lambdas, lambda_1 times lambda_2 is the determinant of the matrix.
Or if I had ten eigenvalues, I would multiply all ten and I'd get the determinant.
So that's some facts about eigenvalues.
There's more, of course, in section 1.5 about how you would find eigenvalues and how you would use them.
That's of course the key point, is how would we use them.
Let me say something more about that, how to use eigenvalues.
Suppose I have this system of 1,000 differential equations.
Linear, constant coefficients, starts from some u(0).
How do eigenvalues and eigenvectors help?
Well, first I have to find them, that's the job.
So suppose I find 1,000 eigenvalues and eigenvectors.
A times eigenvector number i is eigenvalue number i times eigenvector number i.
So these, y_1 to y_1,000, so y_1 to y_1,000 are the eigenvectors.
And each one has its own eigenvalue, lambda_1 to lambda_1,000.
And now if I did that work, sort of like, in advance, now I come to the differential equation.
How could I use this?
This is now going to be the most-- it's three steps to use it, three steps to use these to get the answer.
Ready for step one.
Step one is break u_0 into eigenvectors.
Split, separate, write, express u(0) as a combination of eigenvectors.
Now step two.
What happens to each eigenvector?
So this is where the differential equation starts from.
This is the initial condition.
1,000 components of u at the start and it's separated into 1,000 eigenvector pieces.
Now step two is watch each piece separately.
So step two will be multiply say, c_1 by e^(lambda_1*t), by it's growth.
This is following eigenvector number one.
And in general, I would multiply every one of the c's by e to those guys.
So what would I have now?
This is one piece of the start.
And that gives me one piece of the finish.
So the finish is, the answer is to add up the 1,000 pieces.
And if you're with me, you see what those 1,000 pieces are.
Here's a piece, some multiple of the first eigenvector.
Now if we only were working with that piece, we follow it in time by multiplying it by either the lambda_1 * t, and what do we have at a later time?
c_1*e^(lambda_1*t)y_1.
This piece has grown into that.
And other pieces have grown into other things.
And what about the last piece?
So what is it that I have to add up?
Tell me what to write here.
c_1,000, however much of eigenvector 1,000 was in there, and then finally, never written left-handed before, e to the who?
Lambda number 1,000, not 1,000 itself, but it's eigenvalue, 1,000t.
This is just splitting, this is constantly, constantly the method, the way to use eigenvalues and eigenvectors.
Split the problem into the pieces that go, that are eigenvectors.
Watch each piece, add up the pieces.
That's why eigenvectors are so important.
Yeah?
Yes, right.
Well, now, very good question.
Let's see.
Well, the first thing we have to know is that we do find 1,000 eigenvectors.
And so my answer is going to be for symmetric matrices, everything always works.
For symmetric matrices, if size is 1,000, they have 1,000 eigenvectors, and next time we'll have a shot at some of these.
What some of them are for these special matrices.
So this method always works if I've got a full family of independent eigenvectors.
If it's of size 1,000, I need, you're right, exactly right.
To see that this was the questionable step.
If I haven't got 1,000 eigenvectors, I'm not going to be able to take that step.
And it happens.
I am sad to report that some matrices haven't got enough eigenvectors.
Some matrices, they collapse.
This always happens in math, somehow.
Two eigenvectors collapse into one and the matrix is defective, like it's a loser.
So now you have to, of course, the equation still has a solution.
So there has to be something there, but the pure eigenvector method is not going to make it on those special matrices.
I could write down one but why should we give space to a loser?
But what happens in that case?
You might remember from differential equations when two of these roots, these are like roots, these lambdas are like roots that you found in solving a differential equation.
When two of them come together, that's when the danger is.
When I have a double eigenvalue, then there's a high risk that I've only got one eigenvector.
And I'll just put in this little thing what the other, so the e^(lambda_1*t) is fine.
But if that y_1 is like, if the lambda_1's in there twice, I need something new.
And the new thing turns out to be t*e^(lambda* t).
I don't know if anybody remembers.
This was probably hammered back in differential equations that if you had repeated something or other then this, you didn't get pure e^(lambda*t)'s, you got also a t*e^(lambda*t).
Anyway that's the answer.
That if we're short eigenvectors, and it can happen, but it won't for our good matrices.
Ok, so Monday I've got lots to do.
Special eigenvalues and vectors and then positive definite.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Actually, two things to say about eigenvalues.
One is about matrices in general and then the second is to focus on our favorites, those second derivatives and second differences.
There's a lot to say about eigenvalues but then we'll have the main ideas.
So the central idea of course is to find these special directions and we expect to find n directions, n eigenvectors y where this n by n matrix is acting like a number in each of those directions.
So we have this for n different y's and each one has its own eigenvalue lambda.
And of course the eig command in MATLAB will find the y's and the lambdas.
So it finds the y's and the lambdas in a matrix.
So that's what I'm going to do now, straightforward.
Any time I have n vectors, so I have n of these y's, I've n y's and n lambdas.
Well, if you give me n vectors, I put them into the columns of a matrix, almost without thinking.
So can I just do that?
So there is y_1, the first eigenvector.
That's y_2 to y_n.
Ok, that's my eigenvector matrix.
Often I call it S. So I'll stick with that.
S will be the eigenvector matrix.
Since these are eigenvectors I'm going to multiply that matrix by A.
That should bring out the key point.
I'm just going to repeat this which is one at a time by doing them all that once.
So what happens if I multiply a matrix by a bunch of columns?
Matrix multiplication is wonderful.
It does the right thing.
It multiplies A times the first column.
So let's put that there.
A times the first column along to A times the last column.
Just column by column.
But now we recognize these.
They're special y's.
They're special because they're eigenvectors.
So this is lambda_1*y_1 along to that column is lambda_n*y_n.
Y Right?
Now I've used the fact that they were eigenvectors.
And now, one final neat step of matrix multiplication is to factor out this same eigenvector matrix again and realize, and I'll look at it, that it's being multiplied by this diagonal that's now a diagonal matrix of eigenvalues.
So let's just look at that very last step here.
Here I had the first column was lambda_1*y_1.
I just want to see, did I get that right?
If I'm looking at the first column where that lambda_1 is sitting, it's going to multiply y_1 and it'll be all zeroes below so I'll have none of the other eigenvectors.
So I'll have lambda_1*y_1, just what I want.
Got a little squeezed near the end there, but so let me write above.
The result is just A times this eigenvector matrix that I'm going to call S equals what?
This is Ay=lambda*y for all n at once.
A times S is, what have I got here?
What's this?
That's S. And what's the other guy?
That's the eigenvalue matrix.
So it's just got n numbers.
They automatically go on the diagonal and it gets called capital Lambda.
Capital Lambda for the matrix, little lambda for the numbers.
So this is n, this is all n at once.
Straightforward.
Now I'm going to assume that I've got these n eigenvectors, that I've been able to find n independent directions.
And almost always, you can.
For every symmetric matrix you automatically can.
So these y's are independent directions.
If those are the columns of a matrix, yeah, here's a key question about matrices.
What can I say about this matrix S if its n columns are independent?
Whatever that, you know, we haven't focused in careful detail, but we have an idea.
That means sort of none of them are combinations of the others.
We really have n separate directions.
Then that matrix is?
Invertible.
A matrix that's got n columns, independent, that's what we're hoping for.
That matrix has an inverse.
We can produce, well all the good facts about matrices.
This is a square matrix.
So I can invert it if you like.
And I can write A as S lambda.
I'm multiplying on the right by this S inverse And there I have the diagonalization of a matrix.
The matrix has been diagonalized.
And what does that mean?
Well this is, of course the diagonal that we're headed for.
And what it means is that if I look at my matrix and I separate out the different eigendirections, I could say, that the matrix in those directions is just this diagonal matrix.
So that's a short way of saying it.
Let me just carry one step further.
What would A squared be?
Well now that I have it in this cool form, S*lambda*S inverse, I would multiply two of those together and what would I learn?
If I do that multiplication what comes out?
First an S from here.
And then what?
Lambda squared.
Why lambda squared?
Because in the middle is the S S inverse that's giving the identity matrix.
So then the lambda multiplies the lambda and now here is S inverse.
Well A squared is S*lambda squared*S inverse.
What does that tell me in words?
That tells me that the eigenvectors of A squared are?
The same.
As for A.
And it tells me that the eigenvalues of A squared are?
The squares.
So I could do this.
Maybe I did it before, one at a time.
Ay=lambda*y, multiply again by A.
A squared*y is lambda*Ay, but Ay is lambda*y so I'm up to lambda squared*y.
You should just see that when you've got these directions then your matrix is really simple.
Effectively it's a diagonal matrix in these good directions.
So that just shows one way of seeing.
And of course what about A inverse?
We might as well mention A inverse.
Suppose A is invertible.
Then what do I learn about A inverse?
Can I just invert that?
I'm just playing with that formula, so you'll kind of, like, get handy with it.
What would the inverse be if I have three things in a row multiplied together?
What's the inverse?
So I'm going to take the inverses in the opposite order, right?
So the inverse of that will come first.
So what's that?
Just S. The lambda in the middle gets inverted and then the S at the left, its inverse comes at the right.
Well what do I learn from that?
I learn that the eigenvector matrix for A inverse is?
Same thing, again.
Same.
Let me put just Same.
What's the eigenvalue matrix for A inverse?
It's the inverse of this guy, so what does it look like?
It's got one over lambdas.
That's all it says.
The eigenvalues for A inverse are just one over the eigenvalues for A.
If that is so, and it can't be difficult, we could again, we could prove it sort of like, one at a time.
This is my starting point, always.
How would I get to A inverse now and recover this fact that the eigenvalues for the inverse, just turn them up.
If A has an eigenvalue seven, A inverse will have an eigenvalue 1/7.
What do I do?
Usually multiply both sides by something sensible.
Right?
What shall I multiply both sides by?
A inverse sounds like a good idea, right.
So I'm multiplying both sides by A inverse, so that just leaves y and here is that number, here is A inverse times y.
Well, maybe I should do one more thing.
What else shall I do?
Divide by lambda.
Take that number lambda and put it over here as one lambda.
Well, just exactly what we're looking for.
The same y has A inverse, the same y as an eigenvector of A inverse and the eigenvalue is one over lambda.
Oh, and of course, I should have said before I inverted anything, what should I have said about the lambdas?
Not zero.
Right?
A zero eigenvalue is a signal the matrix isn't invertible.
So that's perfect test.
If the matrix is invertible, all its eigenvalues are not zero.
If it's singular, it's got a zero eigenvalue.
If a matrix is singular, then Ay would be 0y for some, there'd be a vector that that matrix kills.
If A is not invertible, there's a reason for it.
It's because it takes some vector to zero, and of course, you can't bring it back to life.
So shall I just put that up here?
Lambda=0 would tell me I have a singular matrix.
All lambda not equal zero would tell me I have an invertible matrix.
These are straightforward facts.
It's taken down in this row and it's just really handy to have up here.
Well now I'm ready to move toward the specific matrices, our favorites.
Now, those are symmetric.
So maybe before I leave this picture, we better recall what is special when the matrix is symmetric.
So that's going to be the next thing.
So if A is symmetric I get some extra good things.
So let me take instead of A, I'll use K. So that'll be my letter for the best matrices.
So symmetric.
So now what's the deal was symmetric matrices?
The eigenvalues, the lambdas.
I'll just call them the lambdas and the y's.
The lambdas are, do you remember from last time?
If I have a symmetric matrix, the eigenvalues are all?
Anybody remember?
They're all real numbers.
You can never run into complex eigenvalues if you start with a symmetric matrix.
We didn't prove that but it's just a few steps like those.
And what about, most important, what about the y's?
The eigenvectors.
They are, or can be chosen to be, or whatever, anybody remember that fact?
These are, like, the golden facts.
Every sort of bunch of matrices reveals itself through what its eigenvalues are like and what its eigenvectors are like.
And the most important class is symmetric and that reveals itself through real eigenvalues and?
orthogonal, good.
Orthogonal eigenvectors, orthogonal.
And in fact, since I'm an eigenvector, I can adjust it's length as I like.
Right?
If y is an eigenvector, 11y is an eigenvector because I would just multiply both sides by 11.
That whole line of eigenvectors is getting stretched by lambda.
So what I want to do is make them unit vectors.
MATLAB will automatically produce, eig would automatically give you vectors that have been normalized to u.
Here's something good.
So what does orthogonal mean?
That means that one of them, the dot product of one of them with another one is?
Now that's not, I didn't do the dot product yet.
What symbol do I have to write on left-hand side?
Well you could say, just put a dot.
Of course.
But dots are not cool, right?
So maybe I should say inner product, that's the more upper class word.
But I want to use transpose.
So it's the transpose.
That's the dot product.
And that's the test for perpendicular.
So what's the answer then?
I get a zero if i is different from j.
If I'm taking two different eigenvectors and I take their dot product, that's what you told me, they're orthogonal.
And now what if i equals j?
If I'm taking the dot product with itself, each eigenvector with itself.
So what does the dot product of a vector with itself give?
It'll be one because I'm normal.
Exactly.
What it always gives, the dot product of a vector with itself, you just realize that that'll be y_1 squared, y_2 squared, it'll be the length squared.
And here we're making the length to be one.
Well once again, if I write something down like this which is straightforward I want to express it as a matrix statement.
So I want to multiply, it'll involve my good eigenvector matrix.
And this will be what?
I want to take all these dots products at once.
I want to take the dot product of every y with every other y.
Well here you go.
Just put these guys in the rows, now that we see that it really was the transpose multiplying y, do you see that that's just done it?
In fact, you'll tell me what the answer is here.
Don't shout it out, but let's take it two or three entries and then you can shout it out.
So what's the 1, 1 entry here of I guess that's what we called S. And now this would be its transpose.
And what I'm saying is if I take-- yeah, this is important because throughout this course we're going to be taking A transpose A, S transpose S, Q transpose Q, often, often, often.
So here we got the first time at it.
So why did I put a zero there, because it's not it.
What is it?
What is that first entry?
One.
Because that's the row times the column, that's a one.
And what's the entry next to it?
Zero.
Right?
y_1, dot product with y_2 is, we're saying, zero.
So what matrix have I got here?
I've got the identity.
Because y_2 with y_2 will put a one there and all zeroes elsewhere.
Zero, zero.
And y_3 times y_3 will be the one.
I get the identity.
So this is for symmetric matrices.
In general, we can't expect the eigenvectors to be orthogonal.
It's these special ones that are.
But they're so important that we notice.
Now so this is the eigenvector matrix S and this is its transpose.
So I'm saying that for a symmetric matrix, S transpose times S is I.
Well that's pretty important.
In fact, that's important enough that I'm going to give an extra name to S, the eigenvector matrix when it comes from a symmetric matrix, when it has a matrix with S transpose times S equaling the identity is really a good matrix to know.
So let's just focus on those guys.
I can put that up here.
So here's a matrix.
Can I introduce a different letter than S?
It just helps you to remember that this remarkable property is in force.
That we've got it.
So I'm going to call it when K is a symmetric matrix, I'll just repeat that, then its eigenvector matrix has this S transpose times S, I'm going to call it Q. I'm going to call the eigenvectors, so for this special situation, A times-- So I'm going to call the eigenvector matrix Q.
It's the S but it's worth giving it this special notation to remind us that this is, so Q is an orthogonal matrix.
There's a name for matrices with this important property.
And there's a letter Q that everybody uses.
An orthogonal matrix.
And what does that mean?
Means just what we said, Q transpose times Q is I.
What I've done here is just giving a special, introducing a special letter Q, a special name, orthogonal matrix for what we found in the good, in this for eigenvectors of a symmetric matrix.
And this tells me one thing more.
Look what's happening here.
Q transpose times Q is giving the identity.
What does that tell me about the inverse of Q?
That tells me here some matrix is multiplying Q and giving I.
So what is this matrix?
What's another name for this Q transpose?
Is also Q inverse.
Because that's what defines the inverse matrix, at times Q should give I.
So Q transpose is Q inverse.
I'm moving along here.
Yes, please.
The question was, shouldn't I call it an orthonormal matrix?
The answer is yes, I should.
But nobody does.
Damnit!
So I'm stuck with that name.
But orthonormal is the proper name.
If you call it an orthonormal matrix, I'm happy because that's really the right name for that matrix, orthonormal.
Because orthogonal would just mean orthogonal columns but we've taken this extra little step to make all the lengths one.
And then that gives us this great property.
Q transpose is Q inverse.
Orthogonal matrices are like rotations.
I better give an example of an orthogonal matrix.
I'll do it right under here.
Here is an orthogonal matrix.
So what's the point?
It's supposed to be a unit vector in the first column so I'll put cos(theta), sin(theta).
And now what can go in the second column of this orthogonal matrix?
It's gotta be a unit vector again because we've normalized and it's gotta be, what's the connection to the first column?
Orthogonal, gotta be orthogonal.
So I just wanted to put something here that sum of squares is one, so I'll think cos(theta) and sin(theta) again.
But then I've got to flip them a little to make it orthogonal to this.
So if I put minus sin(theta) there and plus cos(theta) there that certainly has length one, good.
And the dot product, can you do the dot product of that column with that column?
It's minus sine, cosine, plus sine, cosine, zero.
So there is a two by two, actually that's a fantastic building block out of which you good build many orthogonal matrices of all sizes.
That's a rotation by theta.
That's a useful matrix to know.
It takes every vector, swings it around by an angle theta.
What do I mean?
I mean that Qx, Q times a vector x rotates x by theta.
Let me put it.
Qx rotates whatever vector x you give it, you multiply by Q, it rotates it around by theta, it doesn't change the length.
So that would be an eigenvector matrix of a pretty typical two by two.
I see as I talk about eigenvectors, eigenvalues there's so much to say.
Because everything you know about a matrix shows up somehow in its eigenvectors and eigenvalues and we're focusing on symmetric guys.
What happens to this A=S*lambda*S inverse?
Let's write that again.
Now we've got K. It's S*lambda*S inverse like any good diagonalization but now I'm giving S a new name, which is what?
Q. because when I give K, when I use that letter K I'm thinking symmetric so I'm in this special situation of symmetric.
I have the lambda, the eigenvalue matrix, and here I have Q inverse.
But there's another little way to write it and it's terrifically important in mechanics and dynamics, everywhere.
It's simple now.
We know everything.
Q lambda what?
Q transpose.
Do you say the beauty of that form?
That's called the principal access theorem in mechanics.
It's called the spectral theorem in mathematics.
It's diagonalization, it's quantum mechanics, everything.
Any time you have a symmetric matrix there's the wonderful statement of how it breaks up when you look at its orthonormal eigenvectors and its real eigenvalues.
Do you see that once again the symmetry has reappeared in the three factors?
The symmetry has reappeared in the fact that this vector is the transpose of this one.
We saw that for elimination when these were triangular.
That makes me remember what we had in a different context, in the elimination when things were triangular we had K=L*D*L transpose.
I just squeezed that in to ask you to sort of think of the two as two wonderful pieces of linear algebra in such a perfect shorthand, perfect notation.
This was triangular times the pivot matrix times the upper triangular.
This is orthogonal times the eigenvalue matrix times its transpose.
And the key point here was triangular and the key point here is orthogonal.
That took some time, but it had to be done.
This is the right way to understand.
That the central theme, it's a highlight of a linear algebra course and we just went straight to it.
And now what I wanted to do was look now at the special K. Oh, that's an awful pun.
The special matrices that we have, so those are n by n. And as I said last time, usually it's not very likely that we find all the eigenvalues and eigenvectors of this family of bigger and bigger matrices.
So now I'm going to specialize to my n by n matrix K equals twos down the diagonal, minus ones above and minus ones below.
What are the eigenvalues of that matrix and what are the eigenvectors?
How to tackle that?
The best way is the way we've done with the inverse and other ways of understanding K was to compare it with the continuous problem.
So this is a big matrix which is a second difference matrix, fixed-fixed.
Everybody remembers that the boundary conditions associated with this are fixed-fixed.
I want to ask you to look at the corresponding differential equation.
So you may not have thought about eigenvectors of differential equations.
And maybe I have to call them eigenfunctions but the idea doesn't change one bit.
So what shall I look at?
K corresponds what?
Continuous differential business, what derivative, what?
So I would like to look at Ky=lambda*y. I'm looking for the y's and lambdas and the way I'm going to get them is to look at, what did you say it was?
K, now I'm going to write down a differential equation that's like this but we'll solve it quickly.
So what will it be?
K is like, tell me again.
Second derivative of y with respect to x squared.
And there's one more thing you have to remember.
Minus.
And here we have lambda*y(x).
That's an eigenvalue and an eigenfunction that we're looking at for this differential equation.
Now there's another thing you have to remember.
And you'll know what it is and you'll tell me.
I could look for all the solutions.
Well, let me momentarily do that.
What functions have minus the second derivative is a multiple of the function?
Can you just tell me a few?
Sine and cosine.
I mean this is a fantastic eigenvalue problem because it's solutions are sines and cosines.
And of course we could combine them into exponential.
We could have sine(omega*x) or cos(omega*x) or we could combine those into e^(i*omega*x), would be a combination of those, or e^(-i*omega*x).
Those are combinations of these, so those are not new.
We've gotten lots of eigenfunctions.
Oh, for every frequency omega this solves the equation.
What's the eigenvalue?
If you guess the eigenfunction you've got the eigenvalue just by seeing what happens.
So what would the eigenvalue be?
Tell me again.
Omega squared.
Because I take the second derivative of the sine, that'll give me the cosine back to the sine, omega squared comes out, omega comes out twice.
Comes out with a minus sign from the cosine and that minus sign is just right to make it plus.
Lambda is omega squared.
So omega squared.
All the way of course.
Those are the eigenvalues.
All our differential examples had something more than just the differential equation.
What's the additional thing that a differential equation comes with?
Boundary conditions.
With boundary conditions.
Otherwise we got too many.
I mean we don't want all of these guys.
What boundary conditions, if we're thinking about K, our boundary conditions should be fixed and fixed.
So that's the full problem.
This is part of the problem not just an afterthought.
Now these conditions, that will be perfect.
Instead of having all these sines and cosines we're going to narrow down to a family that satisfies the boundary conditions.
First boundary condition is it has to be zero at x=0.
What does that eliminate now?
Cosines are gone, keeps the sines.
Cosines are gone by that first boundary condition.
These are guys that are left.
I won't deal with these at this point because I'm down to sines already from one boundary condition.
And now, the other boundary condition.
The other boundary condition has to at x=1 if it's going to work sin(omega*x) has to be?
Nope, what do I put now?
sin(omega), right?
x is one.
I'm plugging in here.
I'm just plugging in x=1 to satisfy.
And it has to equal zero.
So that means, that pins down omega.
Doesn't give me just one of it, well tell me one omega that's okay then.
The first omega that occurs to you is?
Pi.
The sine comes back to pi.
So we've got one.
y_1.
Our first guy is with omega=pi is sin(pi*x).
That's our fundamental mode.
That's the number one eigenfunction.
And it is an eigenfunction, it satisfies the boundary condition.
Everybody would know its picture, just one arch of the sine function.
And the lambda that goes with it, lambda_1, so this is the first eigenfunction, what's the first eigenvalue?
Pi squared, right.
Because omega, we took to be pi.
So lambda_1 is pi squared.
We've got one.
We were able to do it because we could solve this equation in an easy way.
Ready for a second one?
What will the next one be?
The next eigenfunction it's got to, whatever its frequency is, omega, it's got to have sin(omega)=0.
What's your choice?
2pi.
So the next one is going to be sin(2pi*x).
And what will be the eigenvalue that goes with that guy?
lambda_2 will be omega squared, which is 2pi squared, 2pi all squared, so that's four pi squared.
You see the whole list.
The sines with these correct frequencies are the eigenfunctions of the second derivative with fixed-fixed boundary conditions.
And this is entirely typical.
We don't have just n of them.
The list goes on forever, right?
The list goes on forever because we're talking here about a differential equation.
A differential equation's somehow like a matrix of infinite size.
And somehow these sines are the columns of the infinite size eigenvector matrix.
And these numbers, pi squared, four pi squared, nine pi squared, 16pi squared are the eigenvalues of the infinite eigenvalue matrix.
We got those answers quickly.
And let's just mention that if I changed to free-fixed or to free-free I could repeat.
I'd get different y's.
If I have different boundary conditions I expect to get different y's.
In fact, what it look like if that was y'=0 as the left end?
What would you expect the eigenfunctions to look like?
They'd be cosines.
They'd be cosines.
And then we would have to adjust the omegas to make them come out right at the right-hand end.
So this y(0)=0, the fixed ones gave us sines, the free ones give us cosines, the periodic ones if I had y(0)=y(1) so that I'm just circling around, then I would expect these e^(ikx)'s -- the textbook will, so I'm in the eigenvalue section of course, and the textbook lists the answers for the other possibilities.
Let's go with this one.
Because this is the one that corresponds to K. We're now ready for the final moment.
And it is can we guess the eigenvectors for the matrix?
Now I'm going back to the matrix question.
And as I say, normally the answer's no.
Who could guess?
But you can always hope.
You can try.
So what well I try?
Here, let me draw sin(x), sin(pi*x).
And let me remember that my matrix K was a finite difference matrix.
Let's make it four by four.
One, two, three, four let's say.
What would be the best I could hope for, for the eigenvector, the first eigenvector?
I'm hoping that the first eigenvector of K is very, very like the first eigenfunction in the differential equation, which was this sin(pi*x), so that's sin(pi*x).
Well, what do you hope for?
What shall I hope for as the components of y_1, the first eigenvector?
It's almost too good.
And as far as I know, basically it only happens with these sines and cosines example.
These heights, I just picked these, what I might call samples, of the thing.
Those four values and of course zero at that end and zero at that end, so because K, the matrix K is building in the fixed-fixed, these four heights, these four numbers, those four sines-- In other words, what I hope is that for Ky=lambda*y, I hope that y_1, the first eigenvector, it'll be sin(pi*x), but now what is x?
So this is x here from zero to one.
So what's x there, there, there and there?
Instead of sin(pi*x), the whole curve, I'm picking out those four samples.
So it'll be the sine of, what'll it be here?
Pi.
Pi divided by n+1.
Which in my picture would be, we'll make it completely explicit.
Five.
It's 1/5 away along.
Maybe I should make these y's into column vectors since we're thinking of them as columns.
So here's y_1.
sin(pi/5), sin(2pi/5), sin(3pi/5), sin(4pi/5).
That's the first eigenvector.
And it works.
And you could guess now the general one.
Well when I say it works, I haven't checked that it works.
I better do that.
But the essential point is that it works.
I may not even do it today.
So, in fact, tell me the second eigenvector.
Or tell me the second eigenfunction over here.
What's the second eigenfunction?
Let me draw it with this green chalk.
So I'm going to draw y_2.
Now what does y_2 look like?
sin(2pi*x).
What's the new picture here?
It goes up.
What does it do?
By here it's got back, oh no, damn.
I would've been better with three points in the middle, but it's correct.
It comes down here.
Right?
That's sin(2pi*x).
That's halfway along.
I'll finish this guy.
This'll be sin(2pi/5), sin(4pi/5).
See I'm sampling this same thing.
I'm sampling 2pi*x at those same points.
sin(6pi/5) and sin(8pi/5).
Maybe let's accept this as correct.
It really works.
It's the next eigenvector.
And then there's a third one and then there's a fourth one.
And how many are there?
n usually.
And in my case, what is n in the picture I've drawn?
n here is four.
One, two, three, four.
n is four in that picture and that means that I'm dividing by n+1.
That's really sin(pi*h).
You remember I used h as the step size.
So h is 1/5, 1/(n+1), 1/5.
So it's sin(pi*h), sin(2pi*h), 4pi*h, 3pi*h, 4pi*h. Here's 2sin(2pi*h), sin(4pi*h), sin(6pi*h), sin(8pi*h).
So I have two things to do.
One is to remember what is the remarkable property of these y's.
So there's a y that we've guessed.
Right now you're taking my word for it that it is the eigenvector and this is the next one.
I copied them out of those functions.
And just remind me, what is it that I'm claiming to be true about y_1 and y_2.
They are orthogonal, there are orthogonal.
Well to check that I'd have to do some trig stuff.
But what I was going to do was come over here and say this was a symmetric differential equation.
We found its eigenfunctions.
What do you think's up with those?
Those are orthogonal too.
So this would be a key fact in any sort of advanced applied math is that the sine function is orthogonal to the sin(2x).
That function as orthogonal to this one.
And actually that's what makes the whole world of Fourier series work.
So that was really a wonderful fact.
That this is orthogonal to this.
Now you may, quite reasonably, ask what do I mean by that?
What does it mean for two functions to be orthogonal?
As long as we're getting all these parallels, let's get that one too.
I claim that this function, is orthogonal to this function.
What does that mean?
What should these functions could I write dot or transpose or something?
But now I'm doing it for functions.
I just want you to see the complete analogy.
So for vectors, what did I do?
If I take a dot product I multiply the first component times the first component, second component times the second, so on, so on.
Now what'll I do for functions?
I multiply sin(pi*x) * sin(2pi*x) at each x.
Of course I've got a whole range of x's.
And then what do I do?
I integrate.
I can't add.
I integreate instead.
So I integrate one function sin(pi*x) against the other function, sin(2pi*x), dx, and I integrate from zero to one and the answer comes out zero.
The answer comes out zero.
The sine functions are orthogonal.
The sines are orthogonal functions.
The sine vectors are orthogonal vectors.
I normalize to length one and they go right into my queue.
So if I multiply, if I did that times that, that dot product would turn out to be zero.
If I had been a little less ambitious and taken n to be two or three or something we would have seen it completely.
But maybe doing with four is ok.
So great lecture except for that.
Didn't get there.
So Wednesday's lecture is sort of the bringing all these pieces together, positive eigenvalues, positive pivots, positive definite.
So come on Wednesday please.
Come Wednesday.
And Wednesday afternoon I'll have the review session as usual.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Finally we get to positive definite matrices.
I've used the word and now it's time to pin it down.
And so this would be my thank you for staying with it while we do this important preliminary stuff about linear algebra.
So starting the next lecture we'll really make a big start on engineering applications.
But these matrices are going to be the key to everything.
And I'll call these matrices K and positive definite, I will only use that word about a symmetric matrix.
So the matrix is already symmetric and that means it has real eigenvalues and many other important properties, orthogonal eigenvectors.
And now we're asking for more.
And it's that extra bit that is terrific in all kinds of applications.
So if I can do this bit of linear algebra.
So what's coming then, my review session this afternoon at four, I'm very happy that we've got, I think, the best MATLAB problem ever invented in 18.085 anyway.
So that should get onto the website probably by tomorrow.
And Peter Buchak is like the MATLAB person.
So his review sessions are Friday at noon.
And I just saw him and suggested Friday at noon he might as well just stay in here.
And knowing that that isn't maybe a good hour for everybody.
So you could see him also outside of that hour.
But that's the hour he will be ready for all kinds of questions about MATLAB or about the homeworks.
Actually you'll be probably thinking more also about the homework questions on this topic.
Ready for positive definite?
You said yes, right?
And you have a hint about these things.
So we have a symmetric matrix and the beauty is that it brings together all of linear algebra.
Including elimination, that's when we see pivots.
Including determinants which are closely related to the pivots.
And what do I mean by upper left?
I mean that if I have a three by three symmetric matrix and I want to test it for positive definite, and I guess actually this would be the easiest test if I had a tiny matrix, three by three, and I had numbers then this would be a good test.
The determinants, by upper left determinants I mean that one by one determinant.
So just that first number has to be positive.
Then the two by two determinant, that times that minus that times that has to be positive.
Oh I've already been saying that.
Let me just put in some letters.
So a has to be positive.
This is symmetric, so a times c has to be bigger than b squared.
So that will tell us.
And then for two by two we finish.
For three by three we would also require the three by three determinant to be positive.
But already here you're seeing one point about a positive definite matrix.
Its diagonal will have to be positive.
And somehow its diagonal has to be not just above zero, but somehow it has to defeat b squared.
So the diagonal has to be somehow more positive than whatever negative stuff might come from off the diagonal.
That's why I would need a*c > b squared.
So both of those will be positive and their product has to be bigger than the other guy.
And then finally, a third test is that all the eigenvalues are positive.
And of course if I give you a three by three matrix, that's probably not the easiest test since you'd have to find the eigenvalues.
Much easier to find the determinants or the pivots.
Actually, just while I'm at it, so the first pivot of course is a itself.
No difficulty there.
The second pivot turns out to be the ratio of a*c - b squared to a.
So the connection between pivots and determinants is just really close.
Pivots are ratios of determinants if you work it out.
The second pivot, maybe I would call that d_2, is the ratio of a*c - b squared over a.
In other words it's (c - b squared)/a.
Determinants are positive and vice versa.
Then it's fantastic that the eigenvalues come into the picture.
So those are three ways, three important properties of a positive definite matrix.
But I'd like to make the definition something different.
Now I'm coming to the meaning.
If I think of those as the tests, that's done.
Now the meaning of positive definite.
The meaning of positive definite and the applications are closely related to looking for a minimum.
And so what I'm going to bring in here, so it's symmetric.
Now for a symmetric matrix I want to introduce the energy.
So this is the reason why it has so many applications and such important physical meaning is that what I'm about to introduce, which is a function of x, and here it is, it's x transpose times A, not A, I'm sticking with K for my matrix, times x. I think of that as some f(x).
And let's just see what it would be if the matrix was two by two, [a, b; b, c].
Suppose that's my matrix.
We want to get a handle on what, this is the first time I've ever written something that has x's times x's.
So it's going to be quadratic.
They're going to be x's times x's.
And x is a general vector of the right size so it's got components x_1, x_2.
And there it's transpose, so it's a row.
And now I put in the [a, b; b, c].
And then I put in x again.
This is going to give me a very nice, simple, important expression.
Depending on x_1 and x_2.
Now what is, can we do that multiplication?
Maybe above I'll do the multiplication of this pair and then I have the other guy to bring in.
So here, that would be ax_1+bx_2.
And this would be bx_1+cx_2.
So that's the first, that's this times this.
What am I going to get?
What shape, what size is this result going to be?
This K is n by n. x is a column vector.
n by one.
x transpose, what's the shape of x transpose?
One by n?
So what's the total result?
One by one.
Just a number.
Just a function.
It's a number.
But it depends on the x's and the matrix inside.
Can we do it now?
So I've got this to multiply by this.
Do you see an x_1 squared showing up?
Yes, from there times there.
And what's it multiplied by?
The a.
The first term is this times the ax_1 is a(x_1 squared).
So that's our first quadratic.
Now there'd be an x_1, x_2.
Let me leave that for a minute and find the x_2 squared because it's easy.
So where am I going to get x_2 squared?
I'm going to get that from x_2, second guy here times second guy here.
There's a c(x_2 squared).
So you're seeing already where the diagonal shows up.
The diagonal a, c, whatever is multiplying the perfect squares.
And it'll be the off-diagonal that multiplies the x_1, x_2.
We might call those the crossterms.
And what do we get for that then?
We have x_1 times this guy.
So that's a crossterm.
bx_1*x_2, right?
And here's another one coming from x_2 times this guy.
And what's that one?
It's also bx_1*x_2.
So x_1, multiply that, x_2 multiply that, and so what do we have for this crossterm here?
Two of them.
2bx_1*x_2.
In other words, that b and that b came together in the 2bx_1*x_2.
So here's my energy.
Can I just loosely call it energy?
And then as we get to applications we'll see why.
So I'm interested in that because it has important meaning.
Well, so now I'm ready to define positive definite matrices.
So I'll call that number four.
But I'm going to give it a big star.
Even more because it's the sort of key.
So the test will be, you can probably guess it, I look at this expression, that x transpose Ax.
And if it's a positive definite matrix and this represents energy, the key will be that this should be positive.
This one should be positive for all x's.
Well, with one exception, of course.
All x's except, which vector is it?
x=0 would just give me-- See, I put K. My default for a matrix, but should be, it's K. Except x=0, except the zero vector.
Of course.
If x_1 and x_2 are both zero.
Now that looks a little maybe less straightforward, I would say, because it's a statement about this is true for all x_1 and x_2.
And we better do some examples and draw a picture.
Let me draw a picture right away.
So here's x_1 direction.
Here's x_2 direction.
And here is the x transpose Ax, my function.
So this depends on two variables.
So it's going to be a sort of a surface if I draw it.
Now, what point do we absolutely know?
And I put A again.
I am so sorry.
Well, we know one point.
It's there whatever that matrix might be.
It's there.
Zero, right?
You just told me that if both x's are zero then we automatically get zero.
Now what do you think the shape of this curve, the shape of this graph is going to look like?
The point is, if we're positive definite now.
So I'm drawing the picture for positive definite.
So my definition is that the energy goes up.
It's positive, right?
When I leave, when I move away from that point I go upwards.
That point will be a minimum.
Let me just draw it roughly.
So it sort of goes up like this.
These cheap 2-D boards and I've got a three-dimensional picture here.
But you see it somehow?
What word or what's your visualization?
It has a minimum there.
That's why minimization, which was like, the core problem in calculus, is here now.
But for functions of two x's or n x's.
We're up the dimension over the basic minimum problem of calculus.
It's sort of like a parabola It's cross-sections cutting down through the thing would be just parabolas because of the x squared.
I'm going to call this a bowl.
It's a short word.
Do you see it?
It opens up.
That's the key point, that it opens upward.
And let's do some examples.
Tell me some positive definite.
So positive definite and then let me here put some not positive definite cases.
Tell me a matrix.
Well, what's the easiest, first matrix that occurs to you as a positive definite matrix?
The identity.
That passes all our tests, its eigenvalues are one, its pivots are one, the determinants are one.
And the function is x_1 squared plus x_2 squared with no b in it.
It's just a perfect bowl, perfectly symmetric, the way it would come off a potter's wheel.
Now let me take one that's maybe not so, let me put a nine there.
So I'm off to a reasonable start.
I have an x_1 squared and a nine x_2 squared.
And now I want to ask you, what could I put in there that would leave it positive definite?
Well, give me a couple of possibilities.
What's a nice, not too big now, that's the thing.
Two.
Two would be fine.
So if I had a two there and a two there I would have a 4x_1*x_2 and it would, like, this, instead of being a circle, which it was for the identity, the plane there would cut out a ellipse instead.
But it would be a good ellipse.
Because we're doing squares, we're really, the Greeks understood these second degree things and they would have known this would have been an ellipse.
How high can I go with that two or where do I have to stop?
Where would I have to, if I wanted to change the two, let me just focus on that one, suppose I wanted to change it.
First of all, give me one that's, how about the borderline.
Three would be the borderline.
Why's that?
Because at three we have nine minus nine for the determinant.
So the determinant is zero.
Of course it passed the first test.
One by one was ok.
But two by two was not, was at the borderline.
What else should I think?
Oh, that's a very interesting case.
The borderline.
You know, it almost makes it.
But can you tell me the eigenvalues of that matrix?
Don't do any quadratic equations.
How do I know, what's one eigenvalue of a matrix?
You made it singular, right?
You made that matrix singular.
Determinant zero.
So one of its eigenvalues is zero.
And the other one is visible by looking at the trace.
I just quickly mentioned that if I add the diagonal, I get the same answer as if I add the two eigenvalues.
So that other eigenvalue must be ten.
And this is entirely typical, that ten and zero, the extreme eigenvalues, lambda_max and lambda_min, are bigger than, these diagonal guys are inside.
They're inside, between zero and ten and it's these terms that enter somehow and gave us an eigenvalue of ten and an eigenvalue of zero.
I guess I'm tempted to try to draw that figure.
Just to get a feeling of what's with that one.
It always helps to get the borderline case.
So what's with this one?
Let me see what my quadratic would be.
Can I just change it up here?
Rather than rewriting it.
So I'm going to, I'll put it up here.
So I have to change that four to what?
Now that I'm looking at this matrix.
That four is now a six.
Six.
This is my guy for this one.
Which is not positive definite.
Let me tell you right away the word that I would use for this one.
I would call it positive semi-definite because it's almost there, but not quite.
So semi-definite allows the matrix to be singular.
So semi-definite, maybe I'll do it in green what semi-definite would be.
Semi-def would be eigenvalues greater than or equal zero.
Determinants greater than or equal zero.
Pivots greater than zero if they're there or then we run out of pivots.
You could say greater than or equal to zero then.
And energy, greater than or equal to zero for semi-definite.
And when would the energy, what x's, what would be the like, you could say the ground states or something, what x's, so greater than or equal to zero, emphasize that possibility of equal in the semi-definite case.
Suppose I have a semi-definite matrix, yeah, I've got one.
But it's singular.
So that means a singular matrix takes some vector x to zero.
Right?
If my matrix is actually singular, then there'll be an x where Kx is zero.
And then, of course, multiplying by x transpose, I'm still at zero.
So the x's, the zero energy guys, this is straightforward, the zero energy guys, the ones where x transpose Kx is zero, will happen when Kx is zero.
If Kx is zero, and we'll see it in that example.
Let's see it in that example.
What's the x for which, I could say in the null space, what's the vector x that that matrix kills?
, right?
The vector .
gives me .
That's the vector that, so I get 3-3, the zero, 9-9, the zero.
So I believe that this thing will be-- Is it zero at three, minus one?
I think that it has to be, right?
If I take x_1 to be three and x_2 to be minus one, I think I've got zero energy here.
Do I?
x_1 squared will be at the nine and nine x_2 squared will be nine more.
And what will be this 6x_1*x_2?
What will that come out for this x_1 and x_2?
Minus 18.
Had to, right?
So I'd get nine from there, nine from there, minus 18, zero.
So the graph for this positive semi-definite will look a bit like this.
There'll be a direction in which it doesn't climb.
It doesn't go below the base, right?
It's never negative.
This is now the semi-definite picture.
But it can run along the base.
And it will for the vector x_1=3, x_2=-1, I don't know where that is, one, two, three, and then maybe minus one.
Along some line here the graph doesn't go up.
It's sitting, can you imagine that sitting in the base?
I'm not Rembrandt here, but in the other direction it goes up.
Oh, the hell with that one.
Do you see, sort of?
It's like a trough, would you say?
I mean, it's like a, you know, a bit of a drainpipe or something.
It's running along the ground, along this direction and in the other directions it does go up.
So it's shaped like this with the base not climbing.
Whereas here, there's no bad direction.
Climbs every way you go.
So that's positive definite and that's positive semi-definite.
Well suppose I push it a little further.
Let me make a place here for a matrix that isn't even positive semi-definite.
Now it's just going to go down somewhere.
I'll start again with one and nine and tell me what to put in now.
So this is going to be a case where the off-diagonal is too big, it wins.
And prevents positive definite.
So what number would you like here?
Five?
Five is certainly plenty.
So now I have [1, 5; 5, 9].
Let me take a little space on a board just to show you.
Sorry about that.
So I'm going to do the [1, 5; 5, 9] just because they're all important, but then we're coming back to positive definite.
So if it's [1, 5; 5, 9] and I do that usual x, x transpose Kx and I do the multiplication out, I see the one x_1 squared and I see the nine x_2 squareds.
And how many x_1*x_2's do I see?
Five from there, five from there, ten.
And I believe that can be negative.
The fact of having all nice plus signs is not going to help it because we can choose, as we already did, x_1 to be like a negative number and x_2 to be a positive.
And we can get this guy to be negative and make it, in this case we can make it defeat these positive parts.
What choice would do it?
Let me take x_1 to be minus one and tell me an x_2 that's good enough to show that this thing is not positive definite or even semi-definite, it goes downhill.
Take x_2 equal?
What do you say?
1/2?
Yeah, I don't want too big an x_2 because if I have too big an x_2, then this'll be important.
Does 1/2 do it?
So I've got 1/4, that's positive, but not very.
9/4, so I'm up to 10/4, but this guy is what?
Ten and the minus is minus five.
Yeah.
So that absolutely goes, at this one I come out less than zero.
And I might as well complete.
So this is the case where I would call it indefinite.
Indefinite.
It goes up like if x_2 is zero, then it's just got x_1 squared, that's up.
If x_1 is zero, it's only got x_2 squared, that's up.
But there are other directions where it goes downhill.
So it goes either up, it goes both up in some ways and down in others.
And what kind of a graph, what kind of a surface would I now have for x transpose for this x transpose, this indefinite guy?
So up in some ways and down in others.
This gets really hard to draw.
I believe that if you ride horses you have an edge on visualizing this.
So it's called, what kind of a point's it called?
Saddle point, it's called a saddle point.
So what's a saddle point?
That's not bad, right?
So this is a direction where it went up.
This is a direction where it went down.
And so it sort of fills in somehow.
Or maybe, if you don't, I mean, who rides horses now?
Actually maybe something we do do is drive over mountains.
So the path, if the road is sort of well-chosen, the road will go, it'll look for the, this would be-- Yeah, here's our road.
We would do as little climbing as possible.
The mountain would go like this, sort of.
So this would be like, the bottom part looking along the peaks of the mountains.
But it's the top part looking along the driving direction.
So driving, it's a maximum, but in the mountain range direction it's a minimum.
So it's a saddle point.
So that's what you get from a typical symmetric matrix.
And if it was minus five it would still be the same saddle point, would still be 9-25, it would still be negative and a saddle.
Positive guys are our thing.
Alright.
So now back to positive definite.
With these four tests and then the discussion of semi-definite.
Very key, that energy.
Let me just look ahead a moment.
Most physical problems, many, many physical problems, you have an option.
Either you solve some equations, either you find the solution from our equations, Ku=f, typically.
Matrix equation or differential equation.
Or there's another option of minimizing some function.
Some energy.
And it gives the same equations.
So this minimizing energy will be a second way to describe the applications.
Now can I get a number five?
There's an important number five and then you know really all you need to know about symmetric matrices.
This gives me, about positive definite matrices, this gives me a chance to recap.
So I'm going to put down a number five.
Because this is where the matrices come from.
Really important.
And it's where they'll come from in all these applications that chapter two is going to be all about, that we're going to start.
So they come, these positive definite matrices, so this is another way to, it's a test for positive definite matrices and it's, actually, it's where they come from.
So here's a positive definite matrix.
They come from A transpose A.
A fundamental message is that if I have just an average matrix, possibly rectangular, could be a square but not symmetric, then sooner or later, in fact usually sooner, you end up looking at A transpose A.
We've seen that already.
And we already know that A transpose A is square, we already know it's symmetric and now we're going to know that it's positive definite.
So matrices like A transpose A are positive definite or possibly semi-definite.
There's that possibility.
If A was the zero matrix, of course, we would just get the zero matrix which would be only semi-definite, or other ways to get a semi-definite.
So I'm saying that if K, if I have a matrix, any matrix, and I form A transpose A, I get a positive definite matrix or maybe just semi-definite, but not indefinite.
Can we see why?
Why is this positive definite or semi-?
So that's my question.
And the answer is really worth, it's just neat and worth seeing.
So do I want to look at the pivots of A transpose A?
No.
They're something, but whatever they are, I can't really follow those well.
Or the eigenvalues very well, or the determinants.
None of those come out nicely.
But the real guy works perfectly.
So look at x transpose Kx.
So I'm just doing, following my instinct here.
So if K is A transpose A, my claim is, what am I saying then about this energy?
What is it that I want to discover and understand?
Why it's positive.
Why does taking any matrix, multiplying by its transpose produce something that's positive?
Can you see any reason why that quantity, which looks kind of messy, I just want to look at it the right way to see why that should be positive, that should come out positive.
So I'm not going to get into numbers, I'm not going to get into diagonals and off-diagonals.
I'm just going to do one thing to understand that particular combination, x transpose A transpose Ax.
What shall I do?
Anybody see what I might do?
Yeah, you're seeing here if you look at it again, what are you seeing here?
Tell me again.
If I take Ax together, then what's the other half?
It's the transpose of Ax.
So I just want to write that as, I just want to think of it that way, as Ax.
And here's the transpose of Ax.
Right?
Because transposes of Ax, so transpose guys in the opposite order, and the multiplication-- This is the great.
I call these proof by parenthesis because I'm just putting parentheses in the right place, but the key law of matrix multiplication is that, that I can put (AB)C is the same as A(BC).
That rule, which is just multiply it out and you see that parentheses are not needed because if you keep them in the right order you can do this first, or you can do this first.
Same answer.
What do I learn from that?
What was the point?
This is some vector, I don't know especially what it is times its transpose.
So that's the length squared.
What's the key fact about that?
That it is never negative.
It's always greater than zero or possibly equal.
When does that quantity equal zero?
When Ax is zero.
When Ax is zero.
Because this is a vector.
That's the same vector transposed.
And everybody's got that picture.
When I take any y transpose y, I get y_1 squared plus y_2 squared through y_n squared.
And I get a positive answer except if the vector is zero.
So it's zero when Ax is zero.
So that's going to be the key.
If I pick any matrix A, and I can just take an example, but chapter, the applications are just going to be full of examples.
Where the problem begins with a matrix A and then A transpose shows up and it's the combination A transpose A that we work with.
And we're just learning that it's positive definite.
Unless, shall I just hang on since I've got here, I have to say when is it, have to get these two possibilities.
Positive definite or only semi-definite.
So what's the key to that borderline question?
This thing will be only semi-definite if there's a solution to Ax=0.
If there is an x, well, there's always the zero vector.
Zero vector I can't expect to be positive.
So I'm looking for if there's an x so that Ax is zero but x is not zero, then I'll only be semi-definite.
That's the test.
If there is a solution to Ax=0.
When we see applications that'll mean there's a displacement with no stretching.
We might have a line of springs and when could the line of springs displace with no stretching?
When it's free-free, right?
If I have a line of springs and no supports at the ends, then that would be the case where it could shift over by the  vector.
So that would be the case where the matrix is only singular.
We know that.
The matrix is now positive semi-definite.
We just learned that.
So the free-free matrix, like B, both ends free, or C. So our answer is going to be that K and T are positive definite.
And our other two guys, the singular ones, of course, just don't make it.
B at both ends, the free-free line of springs, it can shift without stretching.
Since Ax will measure the stretching when it just shifts rigid motion, the Ax is zero and we see only positive definite.
And also C, the circular one.
There it can displace with no stretching because it can just turn in the circle.
So these guys will be only positive semi-definite.
Maybe I better say this another way.
When is this positive definite?
Can I use just a different sentence to describe this possibility?
This is positive definite provided, so what I'm going to write now is to remove this possibility and get positive definite.
This is positive definite provided, now, I could say it this way.
The A has independent columns.
So I just needed to give you another way of looking at this Ax=0 question.
If A has independent columns, what does that mean?
That means that the only solution to Ax=0 is the zero solution.
In other words, it means that this thing works perfectly and gives me positive.
When A has independent columns.
Let's just remember our K, T, B, C. So here's a matrix, so let me take the T matrix, that's this one, this guy.
And then the third column is .
Those three columns are independent.
They point off.
They don't lie in a plane.
They point off in three different directions.
And then there are no solutions to, no x's that's go Kx=0.
So that would be a case of independent columns.
Let me make a case of dependent columns.
So and I'm going to make it B now.
Now the columns of that matrix are dependent.
There's a combination of them that give zero.
They all lie in the same plane.
There's a solution to that matrix times x equal zero.
What combination of those columns shows me that they are dependent?
That some combination of those three columns, some amount of this plus some amount of this plus some amount of that column gives me the zero vector.
You see the combination.
What should I take?
again.
No surprise.
That's the vector that we know is in the everything shifting the same amount, nothing stretching.
Talking fast here about positive definite matrices.
This is the key.
Let's just ask a few questions about positive definite matrices as a way to practice.
Suppose I had one.
Positive definite.
What about its inverse?
Is that positive definite or not?
So I've got a positive definite one, it's not singular, it's got positive eigenvalues, everything else.
It's inverse will be symmetric, so I'm allowed to think about it.
Will it be positive definite?
What do you think?
Well, you've got a whole bunch of tests to sort of mentally run through.
Pivots of the inverse, you don't want to touch that stuff.
Determinants, no.
What about eigenvalues?
What would be the eigenvalues if I have this positive definite symmetric matrix, its eigenvalues are one, four, five.
What can you tell me about the eigenvalues of the inverse matrix?
They're the inverses.
So those three eigenvalues are?
1, 1/4, 1/5, what's the conclusion here?
It is positive definite.
Those are all positive, it is positive definite.
So if I invert a positive definite matrix, I'm still positive definite.
All the tests would have to pass.
It's just I'm looking each time for the easiest test.
Let me look now, for the easiest test on K_1+K_2.
Suppose that's positive definite and that's positive definite.
What if I add them?
What do you think?
Well, we hope so.
But we have to say which of my one, two, three, four, five would be a good way to see it.
Would be a good way to see it.
Good question.
Four?
We certainly don't want to touch pivots and now we don't want to touch eigenvalues either.
Of course, if number four works, others will also work.
The eigenvalues will come out positive.
But not too easy to say what they are.
Let's try test number four.
So K_1.
What's the test?
So test number four tells us that this part, x transpose K_1*x, that that part is positive, right?
That that part is positive.
If we know that's positive definite.
Now, about K_2 we also know that for every x, you see it's for every x, that helps, don't let me put x_2 there, for every x this will be positive.
And now what's the step I want to take?
To get some information on the matrix K_1+K_2.
I should add.
If I add these guys, you see that it just, then I can write that as, I can write that this way.
And what have I learned?
I've learned that that's positive, even greater than, except for the zero vector.
Because this was greater than, this is greater than.
If I add two positive numbers, the energies are positive and the energies just add.
The energies just add.
So that definition four was the good way, just nice, easy way to see that if I have a couple of positive definite matrices, a couple of positive energies, I'm really coupling the two systems.
This is associated somehow.
I've got two systems, I'm putting them together and the energy is just even more positive.
It's more positive either of these guys because I'm adding.
As I'm speaking here, will you allow me to try test number five, this A transpose A business?
Suppose K_1 was A transpose A.
If it's positive definite, it will.
Be And suppose K_2 is B transpose B.
If it's positive definite, it will be.
Now I would like to write the sum somehow as, in this something transpose something.
And I just do it now because I think it's like, you won't perhaps have thought of this way to do it.
Watch.
Suppose I create the matrix [A; B].
That'll be my new matrix.
Say, call it C. Am I allowed to do that?
I mean, that creates a matrix?
These A and B, they had the same number of columns, n. So I can put one over the other and I still have something with n columns.
So that's my new matrix C. And now I want C transpose.
By the way, I'd call that a block matrix.
You know, instead of numbers, it's got two blocks in there.
Block matrices are really handy.
Now what's the transpose of that block matrix?
You just have faith, just have faith with blocks.
It's just like numbers.
If I had a matrix [1; 5] then I'd get a row one, five.
But what do you think?
This is worth thinking about even after class.
What would be, if this C matrix is this block A above B, what do you think for C transpose?
A transpose, B transpose side by side.
Just put in numbers and you'd see it.
And now I'm going to take C transpose times C. I'm calling it C now instead of A because I've used the A in the first guy and I've used B in the second one and now I'm ready for C. How do you multiply block matrices?
Again, you just have faith.
What do you think?
Tell me the answer.
A transpose, I multiply that by that just as if they were numbers.
And I add that times that just as if they were numbers.
And what do I have?
I've got K_1+K_2.
So I've written K_1, this is K_1+K_2 and this is in my form C transpose C that I was looking for, that number five was looking for.
So it's done it.
It's done it.
The fact of getting A, K_1 in this form, K_2 in this form.
And I just made a block matrix and I got K_1+K_2.
That's not a big deal in itself, but block matrices are really handy.
It's good to take that step with matrices.
Think of, possibly, the entries as coming in blocks and not just one at a time.
Well, thank you, okay.
I swear Friday we'll start applications in all kinds of engineering problems and you'll have new applications.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we move to a topic I really like.
It's the beginning of the applications.
So the particular application that comes first will be springs and masses, a pretty classical problem.
But what we're looking for is how do we model it, what's the main framework to look at a whole series of problems.
So this number one in the series and it's the most straightforward.
Let me draw it with four springs connecting three masses.
And let me fix both ends.
So this will be a fixed-fixed picture.
So the masses have some weight.
The weight pulls the springs down.
When there was no weight acting they were not stretched.
The masses will stretch the springs.
And the question is how much do those, we're looking for the displacements.
How much does mass one go down?
mass two, mass three, and of course, essentially the displacement here is zero and here is zero.
I don't know if you can imagine these masses have acted so that the position before gravity was turned on was somewhere up here and then it came here.
So this moved down by a distance u_1.
Let's use u for the displacements.
So if I look at this main picture here I have displacements, movements, u_1, u_2, u_3.
Now what happens physically?
Important in every one of these examples to see what's happening physically.
Of course, this one moved down by some u_2, this one moved down from its total rest position to u_3.
These are not oscillating.
Next week they'll start moving, time will enter.
Here I'm just looking for a steady state.
They come to rest, they stretch.
So what's your feeling of what's going to happen here somehow The displacements look to me like they'll all be positive.
What's the key equation going to be?
That when this moves down it will stretch that spring.
Hooke's Law will say there's a force, the spring pulls back.
The spring pulls back with a force proportional to the stretch.
So u_1, u_2, and u_3 are movements.
Here is a key question.
What's the stretching and spring number two?
So this is spring one, two, three and four.
How much does spring number two stretch?
u_2-u_1.
A difference is coming in there.
So let me put that up here.
So stretching or elongation, I'll use two words, elongation I'll say sometimes because that starts with a letter e. So these are the elongations in the springs, in the four springs.
It's the amount the spring stretches.
Or what's the opposite of stretching?
Compression somehow.
Looks to me like this last spring, at least, is going to be compressed, and I'm not sure about the others.
So we've got four springs.
And each one has a stretching or compression, an elongation.
And then there's a link then that you already told me.
That e_2 is, just from the picture, e_2 is the difference between u_2 and u_1.
Because the lower mass goes down by distance u_2, the upper mass by u_1 and spring is stretched by the difference u_2-u_1.
So that's a first key fact.
So that expresses somehow a fact of geometry.
Of sort of the way things are connected.
The material properties of the springs have not got into the picture yet.
But now Hooke's Law brings them into the picture.
By stretching a spring that produces a force that pulls back.
So we get, can I say, forces in the spring.
And let me give those a name w. w_1, two, three and four.
And then the link between the stretching and the force that it produces is, so that's somehow where the properties of the material come in.
So I have to say, what are the properties of the springs?
So this will be Hooke's Law, this step.
Hooke's Law for this particular application.
And so I have to say these springs have spring constants.
So I haven't completed the description of the problem until I've told you about springs themselves and the masses.
So the spring constants will be c_1, c_2, c_3, and c_4.
And now what does Hooke's Law say?
Usually this physical law in the middle we keep it linear.
Of course, we all understand that it if these springs were enormously stretched the elastic property could become non-linear.
It could become plastic.
The first law always has somebody's name.
Was the person to see that in some range of small displacements, so I guess that's the answer.
We're speaking here about small displacements, small stretching up to the point where Hooke's Law continues to hold.
And now what does Hooke's Law say?
It says that each force in the spring is proportional to the stretching of the spring.
You could say it's a diagonal matrix is showing up here.
The vector of w's, the vector of forces in the spring is a diagonal matrix C, which it just has these numbers on the diagonal, c_4 times the e's.
So of course I'm going to write that in matrix notation as W equals a matrix C times e. So there in the middle is the physics.
The material properties, the constitutive law.
C can stand for constants, for constitutive law, later for conductances.
It's the place where the material enters.
And now how do we complete this picture?
In the end we have to bring in the masses.
Gravity is the external force that's making things happen.
We need a force term from outside to move us away from zeroes.
And that will be the downward forces f_1, f_2, f_3 on the three masses.
So I plan to complete this picture with a force balance equation on the masses, on each mass.
When I use the word framework there, this is what I was talking about.
I guess what I want to say is I really have found that this way of describing, modeling the problem is successful for so many applications.
You have somehow a geometry, a step which'll involve a matrix A.
Then you have a physical step which involves a matrix C. And then finally you have a force balance.
In a way this force balance or its analog, the analog would be Kirchoff's current law.
We'll see that for networks.
Flow in equals flow out.
Force on one side equals force on the other.
If we're talking about equilibrium we can expect our model to have an equation like that.
And for me it really helps to know when a new model comes in.
Like somebody'll come into my office with a problem in chemistry or biology.
but if it fits in this framework I'll be looking for a balance equation, a continuity equation at the end.
This part was easy and it's these two parts that I want to pin down.
Well you told me how to start here.
So the elongation, so I want to take this step again.
I want to find the elongations from some matrix that multiplies the displacements.
So I'm just completing this step.
And you told me what is the stretching in spring two.
Again, do you mind just saying it again?
The stretching in that second spring, the amount, it's made longer by the action of gravity was?
u_2-u_1.
u_2-u_1.
So e_2 will be a minus one here for u_1, a plus one and a zero.
That will be a typical row of this matrix, the displacement stretching matrix, you could say.
Now what about the stretching in e_1?
What's the stretching in e_1?
Only u_1.
Because essentially it's u_1-u_0 but u_0 is set to zero by the support.
So we only have u_1.
Because that multiplication just gives us.
So e_1 is u_1.
e_2 is u_2-u_1.
e_3 is what?
The stretching in the third spring.
What is it?
u_3-u_2.
So I need a one for u_3 and a minus one for u_2.
And the stretching in the fourth spring?
What's the stretching in the fourth spring?
I've sort of, and you have too, mentally given a plus sign when the spring is extended and a minus sign when it's compressed.
Plus for retention, minus for compression.
So since I fixed that one, u_4 was zero, so what do I have in this last row?
Just minus u_3.
I guess what I'm saying here is that if we get a systematic approach to problems then we know we're looking for a matrix that connects these.
We're looking for the material constitutive law that does this and now we're looking for this one.
We kind of know where we are.
What to look for.
And so this matrix is the matrix I'm going to call A.
So this is e=Au.
Well one more step to go.
And that will be the force balance step.
So now, what's the equation for balance?
The external forces are the masses.
Well, I guess to get the units right, it should be mass times g, the gravitational constant.
So let me put external forces f_1, f_2, and f_3.
The three masses will be m_1*g, m_2*g, and m_3*g. So those are the forces from outside.
Now it's the balance equation I'm after.
So this is in this position.
It's in equilibrium.
And what does that tell us?
That tells us that the total force on this mass, so I'm going to take each mass, it's like a free body force diagram here.
I'm looking now at that mass.
I'm saying what forces are acting on it and I'm making them balance.
So what equation will that give me?
So let me write that.
This is now the force balance equation.
Force balance at each mass.
How much force is pulling up?
What's the force pulling up on?
So this spring is pulling upwards.
And it's pulling upwards by w_1, right?
Just getting these letters right.
The w's were the internal resisting force, reacting force in the spring.
w_1 is pulling up.
What other forces are acting?
w_2 is pulling down.
And also pulling down is?
Gravity m_1*g. So the balance of forces there says that w_1, the force up is w_2, the force down and m_1*g. And similarly the next one will have, the next mass if I look just at that I see a force up, a force down and gravity down.
So w_2 will be, well, that's the pull up will be w_3+m_2*g. And the third one, the force up on the third one will be the force down on the third one.
so I think those are the equations of force balance written one at a time.
And now, of course I'm going to write that.
So that's three equations with four w's.
So I want to write that as, I want to bring the W's all to the left-hand side.
Can I do that?
Can I just bring those over with minus signs?
And make these equal signs.
So now we've got internal force balancing external force.
This vector of external forces is the f's and this is the internal forces.
Now somewhere there we're going to see a matrix.
So I'm going to write this equation as some matrix.
Well, let's figure out what that matrix is.
So it's shape is what?
I've got three equations, so I need three rows in the matrix.
I've got four w's so I need four columns.
So it's going to multiply w_1, w_2, w_3, w_4 to give these three masses, can I call them f_1, f_2, f_3 just to have a good letter.
We're almost there.
What's the matrix?
What's the matrix for this final step, the force balance equation?
I just read it off.
w_1-w_2.
I think I've got that.
w_2-w_3.
Tell me what the second row of the matrix looks like to give me w_2-w_3.
0, 1 for the w_2, -1.
Good.
And for the third, the final row?
.
So that completes the third piece.
If I'd given you the problem as I did, drawn the problem, described it, you know that there's going to be a connection between the external forces and the displacements.
But what I'm trying to say is a good way to see the connection is to see it in three simple steps.
The simple step that gets you from the displacements to the springs.
A second step within the springs.
A third step back to the nodes, you could say, back to the masses.
And of course, the key question is, what's that matrix?
And do you recognize it?
Do we need a new name for that matrix?
The matrix in the third step?
So this third step is going to be that some matrix times w is f and what's that matrix?
What's the good name for us to give it?
A transpose is the best possible name.
If we've given this matrix the name A, the stretching displacement matrix, the strain in elasticity, this becomes the strains, these become the stresses.
But the beauty is, just beautiful, that the matrix in this law is the transpose of this one.
So it's A transpose.
So that's the framework seen now here for the first time.
So the key point was that A and A transpose both appeared but with physical material properties, constitutive matrix in between.
So if we put the pieces together, then we're golden.
And then, let's do an example to see what actually happened.
So the equations were e=Aw, e=Au, then w=Ce, that's Hooke's Law, and then A transpose-- or maybe I'll write it as f=A transpose*w. That's the three steps.
So in this problem the source term showed up at that point.
The source term came from external forces.
I've got three equations.
Now I'm going to put them together into one.
I'll put them into one equation.
So this w I'll just substitute.
So it's A transpose w is Ce , and e is Au.
So I have A transpose C Au.
So that's the ultimate.
That's put the whole structure together.
That's the equation you have to solve.
This would be called the stiffness matrix.
And I use the letter K for that one.
So our equation is Ku=f.
This is our final equation.
Well, we didn't know w. There are two unknowns here.
Two physical things that you want to find.
If you're designing a bridge or a structure you want to know the displacements and then you want to know the internal forces w. It's really beautiful.
The two unknowns of u and w are somehow dual, we can work with one, work with the other, work with both.
Oh let me just mention that the finite element method will fit this framework and somehow this name stiffness matrix has become famous for finite elements in structures and then it's just exploded to appear all over the place.
I guess we should look at A transpose C A.
We can see what it looks like.
And also just from the way it looks there.
So I can write it out explicitly.
I think we want to.
But at the same time I can learn something from just seeing how it's put together.
What can you tell me about A transpose C A?
Let's get the shape first.
Just to see the shape of these things.
The matrix A is what?
What's the shape of A?
It's over here.
four by three.
Four by three.
And the shape of C was, three by three is it?
Where have I got, that C matrix better be here somewhere.
Oh, no, it's four by four.
Four springs.
Of course, it had to be four by four to do that multiplication.
There's the C matrix.
Four by four, thanks.
And the A transpose matrix?
Three by four, thanks.
So the net result is three by three.
Good.
So it's a square matrix.
K is a square matrix.
What else can you tell me about it?
Now we're going to begin to use some of the, sort of the matrix preparation.
These matrices are kind of friends by now.
This is a difference matrix, somehow.
Right?
The stretchings are differences and displacements.
That's its transpose.
And then the C matrix, which is the new thing, sort of the new guy to appear today, is diagonal.
Well if I asked you now, without writing out the matrix for one more property, it's square, what else could you tell me about it?
Symmetric is going to be a very good guess and let's see why.
Why is it symmetric?
How do we show that that?
What do I do?
I take the transpose.
If I take my K transpose, now I write it as, what do I do?
It's a product of things.
So when I transpose a product I have the individual transposes in the opposite order.
So A, its transpose comes first.
C, its transpose comes next.
A transpose, its transpose comes last.
So that's just the rules of matrix transposes.
Now what?
Now I'm ready to use the wonderful fact of what we've got here.
So what is C transpose?
So notice we wanted a symmetric matrix in the middle to be able to knock that T out.
And what is A transpose transpose?
That's A.
We've learned that the thing is symmetric, that if I transpose it I get it back again.
We're going to see more about that.
But let me do the multiplication.
So I'm going to take that, oh, boy.
How am I going to do that?
I want to multiply three matrices to see what K actually looks like here.
One question first.
Eventually the solution, the short formula for the solution will be u=K inverse f. Right?
So the answer will be u=K inverse f in matrix notation but I'm looking for numbers.
And then if I know u then I know the stretching.
e is A times K inverse f. And w is, I'm just going down the list, is C times A times K inverse f. We've got everything.
So that's the key.
This is the key equation.
That's the answer.
Let me ask you about inverses.
What about K inverse?
We took three steps.
Now what if I just ask you about inverses?
This is K inverse that we would like to know.
So again, for inverses I'm going to start this and I'm going to stop halfway and you'll tell me why.
If you give me a product of matrices and I don't think particularly much I'll take the inverse of that times the inverse of that times the inverse of that.
And what's the matter with that?
You would say, why not just undo each step?
Why not find the w's from the f's and then the e's from the w's by dividing and then the u's from the e's?
Why don't we just go backwards around the loop rather than what I'm saying we have to do.
We eventually get this step across with a matrix K that does all three at once.
Well sometimes we might be able to, but I don't think we can in this time.
What's the trouble with A that I don't want to write A inverse?
Well I don't say singular.
What do I say here?
Look at this matrix A here.
It's not square.
It's not square, that's right.
So I'm not comfortable, I'm not willing to write A inverse When A is not a square matrix.
And this distinction, is the matrix A square or not, is the first issue.
It's just the picture.
Let me show you an example of where it would be square.
May I?
Before I do this multiplication, can I jump to a, I'll change the line of springs in a way that'll change A.
And let me show you what happens.
Suppose I take out that spring.
So I've removed the fourth spring.
It's a line of springs now, hanging from a support.
It's a perfectly good problem.
It's problem two, but it's a different problem.
And what's different now?
There is no fourth spring.
If this was my problem, what would be different?
There's no fourth spring.
So that's gone.
I just have three springs stretching from three masses.
Then the force balance is the same.
Everything looks the same except there's no force, there's no fourth spring, so there's no force there, that's gone.
And of course, how does C change?
So in my new picture now I have, let me write now, A transpose C A.
A is now three by three, right, I've lost a row.
A transpose is now three by three, I've lost a column, that fourth spring is gone.
And what is C?
Well of course there's no guy here anymore.
What I'm trying to say is for this problem the matrices have become square.
This would be correct.
So this is an especially nice kind of problem.
It's called statically determinate.
It means I can determine the three w's from the three f's.
I can go backwards.
Everything is determined.
The long word for the fixed-fixed one, our main example, is statically indeterminate.
I cannot determine four w's from three forces.
I can't determine what these internal forces are until I put the whole loop into one matrix K. So that's like a warning, and at the same time, an important separation.
A few nice problems where you don't have too many springs, you don't have too many bars in a truss.
You just have like, the minimum number to hold it together.
Could be statically determinate and square matrices.
But here we're not square.
Now I go back.
So that would be fixed-free.
Right?
That example that I just described would be fixed-free and we can kind of carry that along because we know that what happens is we lose a row and a column and a c_4 is just not in the picture anymore.
But now I want to go back to the fixed-fixed one and finish it.
So that's got a support down there, too.
Key question, what's this matrix K?
This A transpose C A.
We know it's a square matrix, we know it's a symmetric matrix, but it would be really nice to know what does it look like.
What does that matrix look like?
Can I do the multiplication?
So this is going to be K. So it starts with a three by four.
1, -1; 1, -1; 1 -1.
Then its got the four by four, c_1, c_2, c_3, c_4.
And then it's got the transpose of that, which is the 1, -1; 1, -1; 1, -1.
With zero square, I didn't write anything.
We've got three matrices to multiply together.
What's going to happen here?
Well, let's see.
I guess, why don't I multiply that by that?
Can I do that?
So that's like getting two steps together.
It's going to be easy because of this.
This is usually an easy matrix.
Often diagonal.
So when I do that multiplication, so let me, I'll just copy this guy.
And now c_1 multiplies that row, c_2 multiplies this row, c_3 multiplies this row and c_4 multiplies the last row.
c_1 in that row, c_2, c_3, and c_4.
And now I'm ready to put those together into K. So K will be three by three.
What does it have?
It has c_1+c_2.
And then next to that is going to be this row one against column two, there'll be a zero or they'll be a -c_2 here.
And then when row one goes against column three there's nothing.
Why nothing?
When do I expect to see a zero in the overall matrix?
What is it about?
So that zero is in the position 1, 3.
What is it about masses one and theww that is putting that zero in there.
We kind of expect to see that zero even before we find it.
If I look at the picture, what do you notice about masses one and three that is going to produce the zero?
They're not connected.
They're not connected.
If I had another spring, which I could have, connecting mass one to mass three that would produce, I'd have another.
I'd be up to five.
Instead of four, there'd be a fifth spring.
It would have its own constant.
It would show up.
Absolutely could.
Here we don't have it.
Now let me keep going.
I know from symmetry that the second row times this is going to be zero, is going to be -c_2.
Symmetric as I expected.
What are you expecting on the diagonal there?
c_2+c_3.
That's certainly the right pattern.
Zero, c_2+c_3.
c_2+c_3.
And what are you expecting over here?
-c_3 is a good guess.
It's seeing that pattern.
Let's just see it happen.
That second row times this third guy will give me zero, two rows, two zeroes, and then a -c_3, good.
And now we know the zeroes going to show up here, the -c_3 is going to show up here.
And what will show up here?
c_3+c_4.
So we've got it.
That's the matrix K that controls this whole problem.
Now we check.
It's square, yes.
It's symmetric, yes.
And notice also it's the kind of matrix we've seen already.
In fact, it's exactly the matrix we've seen already Suppose all the c's are one.
Suppose every, c_1, c_2, c_3, c_4 is one.
Then what's the matrix capital C in that standard case?
C will just be the identity if these are all ones.
And then I'm only left with A transpose A.
So let me take that special case below it.
Special IF, so this is IF C is I, what matrix do we have then?
Just to see that we have a matrix that we know about.
So I'm copying this now here in the case when all the c's are one.
So if you put all those c's to be one, what matrix do you get?
You get, yes.
You get the special K. Right, you get the special.
So the work we did to understand that special matrix pays off here.
Because we know how that matrix works.
And this matrix, well, it's got four spring constants in it.
But we can guess the important facts about this one from this one.
So what are they important questions about that matrix?
This is my matrix K now.
What would be, we know it's square, we know it's symmetric.
What else do we ask about a matrix?
Well, positive definite, that's the perfect question, right.
And built into positive definiteness would be a property that we mentioned the very first day.
Is it invertible?
What's your guess?
Is that matrix invertible?
Everybody's going to guess yes because, you could guess no, where would you be, the whole course would end.
In fact, the world would end because the problem is correctly posed.
Those displacements are determined by the forces and that just says K is an invertible matrix.
So but how do we see that it's invertible and, even more, positive definite, because that's the property we now know.
So why is that matrix positive definite?
Do we want to check determinants?
We could say, ok, that guy's positive.
We could evaluate this product and find that it came out well.
Would you want to do that one?
We could probably do the two by two determinant.
Could you take that times that and subtract that?
Let's just write it above what we would get.
Just to see it.
That number times that number would be a c_1*c_2 and a c_1*c_3 and a c_2*c_2 twice.
And a c_2*c_3.
And then I would subtract off this guy.
So it would knock out that, right?
And it would leave something that would be positive.
All this spring constants are positive here.
We're talking normal materials.
I guess, actually, people are producing now really amazing materials with amazing properties.
And the amazing property is a material with a negative c. But that's like 18.085 does not allow such a thing.
Right?
All these c's are positive.
And you might guess that the whole determinant is positive.
But now I'd like you to tell me why.
So now we can use our growing familiarity with matrices to say why is this matrix positive definite.
Is symmetric, of course.
Positive definite.
Why?
So that's what the previous lecture helped us to answer.
We've got these various tests, but what was the core idea of positive definiteness?
The core idea was positive energy.
The core idea was I looked at the energy x trans-- no, u, sorry.
Have to call it u now.
u transpose times that matrix times u.
And there was a reason why that matrix was, why this number, it's going to be a number, right?
This combination will involve all four of these c's, it'll involve three u's.
I don't want to write out that quantity.
It would be, I'll have some u_1 squareds and some u_1*u_2.
I won't have any u_1*u_3, because that 1, 3 entry is zero.
But why was this positive?
Where do I put the parentheses.
Where do I put the parentheses to see that that's positive?
I put them around where?
Around that, good.
And around this?
This is really, since we now have a letter for Au, this is really e transpose Ce, right?
That's e. This is eAu and this is its transpose.
And now what?
So now we've narrowed it down to C. Oh, we can actually see why it's an energy.
Remember C is that diagonal matrix.
What will this be?
This is the row of stretchings, the diagonal matrix of c's and the column of stretchings.
And now if I do that multiplication, what do I get?
Do you see it?
Because the physics is coming in.
What do I get?
This will multiply that.
So what's the first term I should write here?
e_1?
What will it be?
I only have diagonal.
In other words, I only have perfect squares when I look at this thing.
I think I just have c_1*e_1 squared coming from that diagonal.
That c_1 there, this e_1 here, this e_1 there is going to give me that c_1.
What else will I have?
c_2*e_2 squared, c_3*e_3 squared and c_4*e_4 squared.
And do you remember about springs and Hooke's Law and energy?
What's the energy in a spring?
This is a stretched spring.
So the energy in a stretched spring, what I wanted to say, this is the sum of four internal energies in the four springs but it properly should have a factor 1/2.
There probably, to really use the word energy properly, it should be 1/2 of all this,1/2 of all this, 1/2 of that's the energy in the first spring, the energy in the second, the energy in the third and the energy in the fourth.
But of course our matrix point was, it's positive.
It's a sum of squares multiplied now by these positive numbers, these elastic constants, c_1, two, three and four.
So we know the main facts about that matrix.
We're really at the point here of we've got some problem formulated, we've got the essential facts about the matrix, it's symmetric, positive definite, certainly invertible.
Then there'd be the step of actually computing U by solving the stiffness equation.
Say, for example, Professor Bathe big finite element code, ADINA.
What's the big picture for ADINA for any big finite element code?
NASTRAN, ANSYS, whatever.
Abacus.
There are so many really good ones.
And they've taken years and years of work to create.
But if you look to see what are the elements that go in, you choose the model, and we'll see in the next chapter, in October we'll see what finite elements is about, you have the material properties, you assemble the matrix K. That's a key step, is assembling this matrix K. And then the final step is solve the system.
Ku=f.
But it's assembling that matrix.
Now one thing popped into my head.
Do I have time to mention it or not?
And there's no class Monday I think, right?
Can I mention?
Can you hang on one more second to mention a really remarkable way to do matrix multiplication.
You may say, we know matrix multiplication.
We got it.
Right?
We did it and we got the right answer.
Can I just show you another way.
And you can like, see if it works.
I did this multiplication by like, I'll say rows times columns.
I took rows times columns.
That's the usual way.
But finite elements and other, often the right way is the opposite.
It's columns times rows.
And of course, this guy's in here too.
You might say, ok, what do I get from column one times that number, times row one?
Can you do that multiplication just mentally?
Multiply that column by that row.
First of all, what shape will the answer have?
What shape will the answer have if I multiply a three by one times a one by three.
Three by three.
It's a full matrix.
Columns times rows.
And it's a totally legitimate way to multiply matrices.
That column times that row will be?
Well you can see what will it be.
And then the c_1 is going to come into it.
If I just did those multiplications, it would just be that.
And then the c_1 puts that there.
What do I see there?
I see the element matrix.
Do you see that this is the piece that involved c_1 in the answer?
Well I guess you'll see it better when I do column two times c_2 times row two.
So I have to add that guy on.
And then I'll leave the other.
What do I get if I do that column, three by one, times itself as a row times the c_2.
I don't know if you see what I'm going to get.
If you just do that, you'll see a c_2 will appear here.
And a -c_2 will appear there.
And these will be zeroes.
So this was column one times row one.
This is column two times row two.
And third and then the fourth.
But do you see that this part is telling me all about the second spring?
This part is telling me, what does the first spring, the c_1, contribute to K. This part tells me what does the c_2 part, do you see the c_2 part in K?
There, there, minus there and minus there.
The third part from the column row would be the c_3 part.
And the fourth part from the fourth spring would be the c_4 part.
So that's a way you won't have thought of.
But it's the way ADINA would assemble this matrix.
It would not do that multiplication.
It would do it this way, columns times rows.
We'll see it again.
So, hope you have a great weekend and a holiday Monday that we all happy about.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
Ready?
So let me recap the important lecture from last time that gave the framework that we'll see in Chapter two and Chapter three, really a major part of the course.
And our example was a line of masses and springs.
So that's like, the first example to start with.
And you remember, there were three steps.
If we want to know how much the masses displace we have to get to the springs.
Difference in displacements gave the stretching of the springs.
Then comes the material law, Hooke's Law in this case with a matrix C that led to CAu.
And then finally, those spring forces are balanced by the external forces.
That brought in A transpose.
So that's the picture to look for.
And I just mention it again because it's so central.
So we'll be doing other examples of that.
But I wanted to stay with springs and masses for today.
To do another type of problem.
A problem in which time enters.
A problem in which we're not looking for the steady state.
But it's Newton's Law is going to come in.
This is actually Newton's Law.
You recognize mass.
You recognize acceleration.
And the force on the spring is -Ku.
So when - Ku, when the force came over to that side, that's the equation we're looking at.
So it's like time-out for a discussion of the very important subject of time derivatives.
How do I solve initial value problems now?
So I have some starting time, I'm given u(0), the starting position of the springs, and I'm given the velocity at zero.
So I'm given two facts at zero.
That's very different from the steady state problem where you're given stuff around the boundary.
Here we're given stuff at the start.
Initial values instead of boundary values.
So what does this mean?
It means that maybe I pull the springs down and I let go.
And what do they do?
They oscillate.
And you will have studied this equation.
I hope you've seen that equation.
Because I think of it as the fundamental equation of mechanics.
In fact, I've even used those words.
The fundamental equation of mechanical engineering.
Well, it's Newton's Law for a system.
So the matrix M now has two masses.
In this case, it would be a two by two system.
The matrix M would have two masses.
Suppose mass one may be like, nine and mass two is one.
So copying what our problem is we'd have a nine and a one multiplying u_1'' and u_2''.
Can I use prime for a derivative?
Rather than dots, because I'm never too sure if the dot is there or not.
So prime I can see.
And then plus our Ku.
And actually you know what K will look like for this system.
It's fixed-fixed.
Well, let me write down what K would look like.
Just because we saw it at the very end.
What it looks like if there are spring constants, c_1, c_2, and c_3.
Can I just remind you what that will look like?
So this will multiply u and give zero.
So the mass matrix is diagonal in the problem.
That's diagonal matrices, we certainly expect to be easy.
Almost as easy as the identity matrix where we would have two equal masses.
Here it's just that little bit more general with two different masses.
And in here, the kind of thing that we created just at the end of the last lecture was something like c_1+c_2 on the diagonal and a -c_2 and a -c_2 and a c_2 + c_3.
So c_2 for the middle spring is the one that's really there with its full little element matrix, c_2, -c_2, -c_2, c_2.
The c_1 part is only coming in at the top because it's connected to a fixed support.
And c_3 is only coming in at the bottom.
Anyway, that's the matrix K. This is the matrix M. And how do we solve the problem?
You've seen before and now ADINA Professor Bathe's code or any other code would offer, like two different ways.
One way is because we have constant coefficients here we can expect a pretty clean formula for the answer.
And because we're growing, we're oscillating in time, things are happening in time, we expect eigenvectors, eigenvalues to come in.
Into that formula.
I think it's worth just repeating that formula.
How do you approach it?
And then the second, the really serious issue is how do you solve equation like this or even more general, time-dependant problems.
I mean this is what finite element codes are created for.
How do you study the crash of a car?
So this isn't quite the equation for a car crash.
What would be different in a car crash?
There's a short section later in Chapter two called the reality of computational engineering.
And the reality is, cars crash, people drop their cell phones.
And those are very difficult problems to compute with.
And of course, absolutely impossible to use eigenvectors because, well they're not linear at all.
Right?
If you crash two cars, if you want to study, everything's happening in like, hundredth of a second or something.
But what's happening there is highly non-linear so it's very much more complicated.
And you take thousands of time steps maybe within the crash time.
And you're going to use finite differences or finite elements.
So this is a chance to say something about this special equation for finite difference method.
It's really 18.086 that takes up these questions seriously.
If I choose a finite difference method.
See, the thing is with finite differences you've got many, many choices.
There's only one way to go in step one.
Eigenvectors, this thing has got eigenvectors, you find them, you're in.
But for the much more general, typical problems that you'll be solving the rest of your lives, finite differences or finite elements, you've got lots of choices.
And the issues that come up are the accuracy of the choice.
Center differences often gives you that extra order of accuracy.
Stability is something we have not seen yet.
So what does stability mean?
Stability means that as I follow my difference equation forward in time it stays near the true equation.
You'll see by example.
So some methods are more stable than others, some are completely unstable and unusable.
And then, of course, the other condition, another desired requirement is speed of calculation.
So these balance each other.
It's a beautiful subject.
And let me just open the door to that subject today.
May I start with the eigenvector solution.
How would I solve this equation by eigenvectors?
Well again, I'm going to look for special solutions.
So let me write that equation down again.
Mu''+Ku=0.
I'm taking zero external force.
So the springs and masses are just oscillating.
Their total energy won't change, it's a closed system.
The kinetic plus the potential energy will stay constant and we can show why.
There's a lot in this section, by the way, section 2.2.
More than I would be able to do in the lecture.
But let me capture the key ideas.
So one key idea is the natural idea when we have constant coefficients.
Look for special solutions.
So let me say look for special solutions of the form-- well, let's keep in mind always the simplest model of all.
Let me put it over here because I'll focus on it particularly, especially.
The simplest case would be u'' with a mass normalized to one plus u=0.
Just a single equation.
So that's just a single spring.
Single mass.
So the mass is just oscillating up and down.
And we all know that it's going to oscillate up and down, sines and cosines are going to come into here.
In fact, we know that the solution to that would be, this could have cosine, this would have some cos(t)'s and some, u could be a combination of cos(t) and sin(t).
Right?
And the A and B would be used to match the two initial conditions.
So that's what we've got in the scalar case.
This is one spring.
n is one.
We don't have a matrix here, just a number.
But that's our guide to the matrix case.
So now what am I going to look for?
I'm going to look for u(t).
I'll look for special solutions of the form cos, they'll go at different frequencies.
So the frequency has to be found.
So that's the time dependence.
And then some, if I'm lucky all the springs will be going together.
So this x-- Am I going to use x for eigenvector?
I think so, yes.
Let's use x for eigenvector.
So this is a vector, this is a constant vector.
And this gives the oscillation.
Let me just already start with that.
I've got two unknowns here, right?
Two things that I'm free to choose.
I'm free to choose the frequency omega and I'm free to choose this vector x.
And I've like, separated out time.
The way we've been doing it with an e^(lambda*).
We used an e^(lambda*t).
So that was sort of right for a first order equation.
Cosine is right for a second order equation like this.
When is that a solution to my equation?
May I just plug it in?
So I'm going to just plug it in?
Substitute is a better word than plug in, right?
So shall I just plug it in?
So I get M times that second derivative.
So the second derivative of the cosine is the cosine with a minus M omega squared.
Is that right?
Yes?
Times what?
So that's what came from the time derivative.
Times cos times this, cos(omega*t)x.
Two time derivatives brought down a minus omega squared.
And the other part is just K times u, cos(omega*t)x.
And that should be zero.
And this is supposed to be a good solution that works for all time.
This is what I want to be zero.
Do you see what's left?
Kx from here.
And putting that on the other side, M(omega squared)x.
Can I write the omega squared that way?
(omega squared)Mx.
Then if that is satisfied then the differential equation is satisfied.
I've got a solution.
So I'm looking for K and I'm looking for x and omega.
And by the way, I could also have sin(omega*t) here.
As well as cosine just the way over there.
Sines and cosines both possible.
So sin(omega*t)x.
It would be exactly the same except that it'd be a sin(omega*t) that I'd be dividing out.
And again, I'd come back to this.
This is key problem.
And do you see that somehow here we have an eigenvector x and an eigenvalue omega squared?
An eigenvalue omega squared, right.
But there is one little twist.
It's not quite our standard eigenvalue problem.
What's the extra guy that's present in that box that we don't usually see?
M. M is the new person there, new thing.
The mass matrix.
Which, if all masses where one, the mass matrix would be the identity.
We would be back.
We would just have our standard problem.
I just have to say a word about the case when the mass matrix is there.
It's still an eigenvalue problem.
If you like, I can bring M inverse over here.
I could write it as (M inverse*K)x=(omega squared)x.
So I'm looking for the eigenvalues of M inverse K. That's really what I'm doing.
The eigenvalues and eigenvectors of M inverse K. But I'm always trying to be like, aesthetically right.
And what's M inverse K, has just a little something wrong with it.
Which is?
It's not symmetric.
If I take my matrix K, which is beautifully symmetric, and my M, which is beautifully symmetric, but when I do M inverse K, do see what'll happen?
The inverse of that matrix will have a 1/9.
When I do M inverse*K, that row will get divided by nine.
And the second row won't change because I've got a one.
So it just like, spoiled things a little.
You can deal with it.
But actually, in some way it's better to keep it right.
And MATLAB is totally okay with that.
So MATLAB would use the command eig, and other systems too, of K, M. Just tell MATLAB the two matrices.
Then it solves that problem.
It will print out the, well if you just typed eig, it would print out the eigenvalues.
If you ask it for the eigenvector there matrix, it'll print those too.
So that has the grand name generalized eigenvalue problem.
Generalized because it's got an M in there.
I don't know if you've met these problems where there could be an M. It's just a natural, and it's not really a big deal, particularly when M is a positive diagonal matrix.
It's not a big deal at all.
It's the same codes essentially finding the eigenvalues and eigenvectors.
So suppose we have them.
We expect n positive eigenvalues, and those'll be the omega squareds.
And each one comes with its eigenvector.
And complete set, everything is good.
We're talking about symmetric positive definite matrices.
Notice that K is positive definite.
That was what our whole last weeks have been about.
And M is obviously positive definite.
So it's a complete, it's a perfect set-up for the eigenvalue problem.
I just want to think through the final step.
After we find the eignevalues, lambda, the omega squareds, then we know the omegas and we find the eigenvectors, x, how do you write the answer?
So what have I done so far?
I've looked for some special solutions.
And I will find them.
So I've found these omegas.
They can go with sines or cosines, I found the x's.
Now what's the general solution?
If I have these solutions to my great equation there, what's the general solution?
u(t).
So this would be the big picture.
What can I do in linear equations?
Linear combinations.
That's what linear algebra is all about, linear combinations.
So I could take a linear combination of these cosines, so cos(omega_1*t)x_1.
So that would be a solution coming from the first time eigenvector and eigenvalue or the square root times any constant.
And then, of course, I could have a b_1.
I'll use b's for the sines.
(omega_1*t)x_1.
So that's just like that.
But it's used the sines, which would also solve the equation.
And then all the way down to x_n's.
Right?
So I've got 2n.
2n simple solutions.
n cosines times eigenvectors and n sines times those same eigenvectors.
Why do I want 2n?
I've got 2n constants then to choose, the a's and the b's.
And how do I choose them?
What's the next step?
Using eigenvectors, maybe I just mention again the three steps.
The three step method.
Step one; find a's and b's.
Oh, now you have to answer my question.
Where are the a's and b's going to come from?
What's going to determine these 2n constants, the a's and the b's?
The initial conditions, of course.
We've got two initial conditions, this is a vector.
We're talking about a system here.
I've got position of both masses.
I've got the initial velocity of both masses.
And the a's, they come from, because the a's go with the cosines and so they're going to be the ones associated with u(0), will give this.
Will give a combination, will give the a's.
u(0) will be a_1*x_1 + a_n*x_n.
This is at t=0.
So I'm using the initial conditions.
u(0) is a combination of those eigenvectors.
So this is from the initial conditions.
And the b's of course are going to come from u'(0), the initial velocity because they go with sines.
So sines start at zero but they have an initial velocity.
So that's step one.
That finds the constants.
It splits the problem into these normal modes.
It finds how much of each eigenvector is in there at the Start Step two?
So it's really worth seeing these three steps because they just repeat and repeat for all applications of eigenvalues.
Step two is what?
Step two is follow each of these guys forward in time.
Follow them to any time.
What does that mean?
That means just put in the cosines and the sines.
So the a's go to a*cos(omega*t).
So I'm just saying what happens to those coefficients.
Each one.
Let's see.
a_i, say, goes to a_i is multiplied by the cosine.
And the b's, b_i's go to b_i*sin(omega_i*t).
So now we followed the coefficients forward in time.
And step three?
The final step is?
So here, we're following each one.
So the pattern is always the same one.
Take your starting conditions, split them up into eigenvectors.
Follow each eigenvector.
And then step three is?
Reassemble.
Put them back together.
Step three is the solution.
u at that later time t is a_1*cos(omega_1*t)x_1.
And b_1*sin(omega_1*t)x_1.
Those are the two guys that are reflecting the fundamental mode.
And then we have a_2's and b_2's multiplied by cosines and sines and times x_2 and so on.
Put the pieces together.
So it's split the initial conditions, follow each eigenvector, reassemble.
It's the heart of Fourier methods, it's the heart of using eigenvectors.
And there's a little code in the book that does exactly that.
So let's just pause a moment.
This was the eigenvector solution.
And now I want to talk about computational science.
For which this problem is a model, but the problem has probably got more complications and things change in time, whatever.
This was an exact solution.
That's more than we hope for in real computations.
We hope for accuracy, but we don't hope for 100% accuracy unless we have this exact model problem.
So we use finite differences.
Ready for finite differences?
So this is method one, which allows you to understand the model problem.
And now comes method two, which allows you to compute much more, many more problems.
And if I do it just for this, let me come back to just this model.
One spring and one mass.
Shall we start the mass at rest?
Let me make that the answer.
I want that to be the answer.
So I'm going to start with u(0) to be zero-- to be one, right?
I'm going to start with u(0) to be one and u'(0) to be zero.
Because the cosine starts at one and its derivative starts at zero.
Apologies that this is such a simple model.
I'm just pulling this this mass down and let go.
I just pull it down an amount one, I let go, it oscillates forever.
How do I draw the motion of a single mass?
A picture is important and we have to think what's in the picture.
So I think the picture should be, maybe u in this direction and maybe u' in this direction.
So it's the u, u' plane.
The position velocity plane.
Sometimes called the phase plane.
I'll just write that word down, phase plane.
And what will be our picture here?
It starts there, right?
u(0) is one, no velocity, starts there.
Oh, and then it just follows cos(t).
And of course, I know what u' is.
If u is cos(t), u' is -sin(t).
Let me put that somewhere where my picture won't run over it, up here maybe.
u' is -sin(t).
Help me out.
If as time increases, I follow this point, the x coordinate is cos(t).
The y coordinate is -sin(t).
It's a circle.
It's a circle.
So I just buzz along it, because of that minus sign, the circle goes this way around.
It's a unit circle.
Goes on forever.
And actually cos squared plus sine squared is one, of course, that's why it's a circle of radius one.
And that represents the total energy actually.
I'm trying to draw a circle.
Actually how would you get your computer to draw a circle?
I'm thinking not just one circle.
I want to refresh it.
I mean, I want to follow this in time.
So I want to draw, I want to follow a point around a circle.
Well I suppose what you would do would be to ask it to plot those points.
That's not allowed.
I want you to follow the solution to the equation.
I want to take that equation whose exact solution is a circle and I want to use finite differences in time.
So I'm going to take finite steps.
Of course that's what the computer has to do.
The computer will take finite steps.
So I'd be very happy if those finite steps keep me on the circle.
And I'd be even happier if it came around exactly right at 2pi but it's not going to.
I want finite differences.
I want to replace that differential equation by finite differences.
And when we do this equation, we're practically doing that one at the same time.
So this model will be fine.
So I'm going to introduce finite differences for this equation.
And everybody, we've been talking about how to replace second derivatives by second differences.
So that looks good.
Let me write down, maybe I can write down three or four possible finite differences.
For u'' I think I'll write down u_(n+1)-2u_n+u_(n-1).
That's the second difference that replaces the second derivative divided by what?
What do you think goes in the denominator now?
Square of what?
Yeah, now what's the right name?
It's going to be something squared.
What do I put down there?
The step size.
I don't want to call it delta x, right?
What should I call that?
Delta t would be natural, delta t. Or h, I could again use h just to have a shorthand.
That's my second difference.
It's centered, it's the natural choice.
And not the only choice, by the way.
Oh.
If I'm an astronomer or like, Professor Wisdom here.
So he followed Pluto around, right?
And discovered that Pluto wasn't a planet, right?
He discovered that the motion of Pluto was not like, regular, like the Earth.
I think the Earth is regular.
But Pluto has chaotic motion.
So it does crazy stuff.
Right.
Well we're looking for very periodic, circular motion here.
So what I was going to say, Professor Wisdom, he would sneer at this second differences, right?
I mean, as we'll see that's got decent accuracy, but not accuracy that would allow you to follow Pluto for 100 million years.
But it'll do for us.
We're not going 100 million years here.
So now comes -u.
Now, question.
I'm taking this, the u term and I'm putting it on the other side.
So mass is one.
Acceleration is this.
The force is -u.
Now comes the big question.
For -u, do I use the newest value?
Do I use the middle value?
Or do I use the oldest value?
Or some combination?
If I want high accuracy, oh, then I go way up in these differences and I find tricky combinations that get me above first and second order accuracy.
But let's not go there.
So I've gotta make one of those choices.
And that choice is going to decide, so I mean, this is a choice you have to make when you have such a problem.
Where are you going to evaluate this?
I guess one choice jumps out as natural.
What would you think of doing?
Well how many would do that one?
So those are people, that's the conservative choice, the stable.
And then, how many would do this?
Yeah, you would do that, right.
I would call that the leapfrog choice.
Somehow, this second difference is leaping over this middle point.
So that would be the leapfrog method.
And then if I use a very low, to use the first value would be.
So my point is those are three different choices.
And each one has these questions of accuracy.
The middle one will be more accurate because it's centered.
Each one has a question of stability.
Ah, you don't see stability yet.
So that's the point of the rest of the lecture is to see.
What is this stability question?
The issue of speed, speed would be, this is the slow one.
Because it involves, I have to bring u_(n+1) over there.
And if I've got a system of equations and then I, it would take me some time.
This would be called an implicit method.
The new u_(n+1) is only given implicitly because it's showing up on the right-hand side.
I've got to move it over to the left.
If it's non-linear I've got a system to solve.
It's safer, but more expensive.
But this would be the natural choice.
I want to get eigenvalues into this picture.
So I'm going to do the step we often do when we see a second order equation.
That is, reduce it to two first order equations.
Can I do that and see the same thing?
So what would be my two first order equations.
My unknowns will be u and u'.
So it'll be first order, the derivative of u and u'.
Shall I call it v for velocity?
Let me write down, I don't need to write this fancy.
The two equations will be, u'=v.
And v' is what?
So u'=v sort of told me what v was.
v is the velocity, the time derivative of du/dt.
And now the derivative of the velocity, now that should reflect my true equation.
So this is all coming from u''+u=0.
So v' is what?
v' is u''.
Do I want -u there?
Yeah, that's my equation.
v', which is u'' is -u.
Good.
That's my system.
So I have a matrix here.
I have a two by two system with a matrix [0, 1; -1, 0] I guess.
So while thinking about difference methods, so this, I was thinking about a second order equation.
I went right to it.
Here I'm thinking about a first order system.
A lot of problems come in first order systems.
Gas dynamics comes as a big first order system for those components of mass, momentum, energy, typically five non-linear equations in gas dynamics.
I mean, that's a very, very serious problem, how to solve those.
Here we've got a much simpler model problem, linear, constant coefficients.
But what do we do?
Can I propose three possibilities here?
And actually they are identical to these three possibilities.
If I just switch over to a first order system.
So possibility one.
Possibility one is that u_(n+1) and v_(n+1), n so the u_ (n+1) would be, how do I model that equation?
u_(n+1) is u_n+delta t*v_n.
That would be a natural, right?
Let me just pause there, because that's meant to be the natural forward difference.
So I replaced u' by u_(n+1)-u_n over delta t. And I replaced v by the value I know.
And do you know whose name is associated with that?
I'm almost going to say his name.
It's just I replace a differential equation by a difference equation where each step I just could figure out what the slope is and I go along a straight line.
Delta t further along.
Do you know his name?
Euler, Euler, right.
It's Euler's method.
The very first method you would think of.
Right?
So I'm doing the same for v. This is my position I've reached.
So Euler's method is replace this by v_(n+1)-v_n over delta t equals -u_n.
So this is Euler's idea.
Predict the new value by a forward difference starting from where the slope of the line is this old one.
That's the first difference method you would think of.
So now if I multiply up by delta t I have a minus delta t*u_n.
Good time to pay attention.
This is forward Euler.
Forward Euler.
The time step, you follow the derivative, what the derivative is at the start of the step.
You follow it for a little step.
Ah, let me draw what the thing would do.
Shall I draw what forward Euler will do?
So we're here.
And what's our first step?
Our first step.
So we're starting at v is zero at the start.
So u won't change in one step.
But u is one, so v will go down.
I think the first step will take us there.
That point was (1, 0).
And I think the second step, u came from the zero, so it'll still be a one.
And v came from a zero, but minus delta.
I think it'll just be 1-delta t. It's left the circle, of course.
And what do you think happens when I do 1,000 steps?
This is two lines in MATLAB.
Just write those equations for the next u coming from the previous u starting at (1, 0) and see what happens.
And what do you think?
It's going to sort of go around.
I mean, it's a reasonable, Euler wasn't dumb.
Anything true here is the fact that Euler was not dumb.
He had a bunch of ideas.
This wasn't his greatest.
But it's the starting point of any, if you have to code some complicated problem, start with Euler, forward Euler.
What'll happen?
It'll spiral out.
Exactly.
It'll spiral out.
And if I was an eigenvalue person, I guess which I probably am, I would look at the, there's some matrix here that's finding the new u, v from the old u, v. Maybe we can even find out, write above what that matrix is.
The new u, v come from the old u, v, well there is the identity matrix.
Here is a delta t. And here is our minus delta t. That would be my matrix.
That's the forward Euler matrix.
The growth matrix for forward Euler is that.
Now what do you think about its eigenvalues before?
Spiral out was the right answer.
If delta t is small it'll stay close to the circle, but it'll spiral out from it.
So if we were following a planet, it would just be off.
So I can't immediately, well you could almost see the eigenvalues of this matrix.
What's the determinant of that matrix?
What's the determinant of G?
I'm just asking you to think through.
You see a matrix like this.
You wonder about its eigenvalues so you take the determinant.
And what do you get?
One, something bigger than one or less than one?
Bigger.
So that tells me what?
Since the determinant is the product of lambda_1*lambda_2, whatever those guys are, at least one of those eigenvalues is bigger than one.
This has an eigenvalue, actually, the actual eigenvalues of this happen to be one from the identity matrix and then this guy is what was our example of an imaginary eigenvalue, i*delta t. Bigger than one in absolute value.
It goes out.
So that's forward Euler corresponding to that choice.
Can I change to a second, to Euler's other choice?
And you can guess what its name is.
Instead of forward Euler it's going to be?
Backward Euler.
Euler said, if you don't like it, I've got another one.
And what will happen then?
Backward Euler is going to use, instead of the old value, it will use the new values here.
So these differences, you see that this is now a backward difference?
So I'm at time n+1.
I'm at the new time.
And this goes back from that.
So this is at the new time and this is a difference backward.
And what happens now?
Well, we could follow backward Euler.
We could follow its first step.
What would its first step be?
u_n+u_1.
No, I can't follow that first step so easily.
Help me out by just making a guess.
What do you think backward Euler does?
It spirals in.
Right, the cover of the book has forward Euler, I think, is on the cover of the textbook.
Spiraling out in the front.
But maybe the back cover also has Euler spiraling in.
So here is backward Euler.
And after I take a bunch of steps it comes back somewhere there, but it's spiraled in.
No good.
Have we got one minute for the good method?
You're guessing that it's this one?
How does that work?
Actually, it's pretty neat.
So my question is, where do I evaluate these guys?
So this is like forward and backward.
The u equation, so I know u_n and v_n, right?
Everybody with me?
I've got to time in, I know u_n and v_n, the position and velocity approximately at point n. And I want to go n+1.
what I know is v_n.
So that's great.
But what's going to change here?
Now I know u_(n+1) from the first equation.
What shall I do, where shall I evaluate u in the second equation?
At n+1.
Why not?
I know it already from the first step, from the forward step with this equation.
I've taken u forward.
So I'll use that forward value of u in the v equation.
So it's forward n backward at the same time.
And the net result if I go back from my system of two first order equations back to one second order equation, I'll find the centered guy.
And what do you think happens with that center difference?
I think MATLAB would have to show it.
And look in Section 2.2 for the picture that shows the leapfrog method.
So what do you think?
I'm having to depend on my memory now.
It stays almost on the circle.
I think it maybe, it stays very close to the circle.
And comes back almost at the right time, but not exactly.
Because we have an error here.
No, we haven't got the real equation.
We've got a difference equation.
So maybe that's the point to say.
When you start with a differential equation you've got various choices.
Backward is actually safer.
Probably for a car crash.
No, actually I think for a car crash they would use forward differences because they have to take so many.
Let me check on that.
But for our problem, this one, leapfrog is the winner.
Actually, if anybody does computation in computational chemistry, I mean that's a subject that uses giant supercomputers to follow molecular dynamics, to follow what molecules are doing at high speed, very high speed.
So this, there's a giant number multiplying that u.
And they use leapfrog method.
I'll say a little more Friday if I can about this general subject, because it's so important.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses visit MIT OpenCourseWare at ocw.mit.edu
You can pick up the homeworks after if you didn't get one.
And the good grade is five.
And the MATLABs are coming in, that's all good.
And I'll have some thoughts about the MATLAB Monday.
I wanted to say a little more about finite differences in time just because they're so important.
And I got the idea of a forward Euler and a backward Euler but I want to give you a couple of more possibilities just so you see how finite difference methods are created.
And then I want to get started on least squares, the next major topic, the next case where A transpose A shows up at the core of everything.
So what we did with finite differences was this was our model problem for one spring and this is our model problem for n springs with n different masses.
So this is like, scalar.
u is just a single unknown.
Here u is a vector unknown.
If I reduce things to like, even this model problem, if I introduce the velocity city, because everybody's also interested in computing velocity, then the velocity is u' and the equation tells me that v' is minus u.
So I have a system.
Better to think in terms of first order systems because they include everything.
So the first order system there, do you see what the matrix is on that right-hand side?
I'm always seeing a matrix that's multiplying uv.
So this would be our example of this general set-up.
This general set-up.
First order systems, taking them to be linear here.
And then I better say a little bit about what happens when they're not linear today and later.
So this is my problem.
du/dt=Au.
And the exact solution would come from the eigenvalues and eigenvectors of A.
We would have the e^(lambda*t)'s giving us the time growth or decay or oscillation times their eigenvectors.
And a combination of those would be the exact answer.
So that's the general method for using eigenvalues and eigenvectors to get exact solutions.
I'm not speaking now about exact solutions.
I'm going to talk about finite differences in time because for more general problems I must expect to go to finite differences.
I can't expect exact solutions like pure cos(t) or sin(t).
So here's my problem, model problem.
Here is Euler's first idea.
So idea one from Euler was replace the derivative by a finite difference taking time steps forward in time and use the equation to tell you the slope at the start of the step.
Do you see Euler's equation there?
And that is definitely a reasonable thing to start with.
It's not very accurate.
It's not perfectly stable.
It doesn't preserve energy.
We saw the answer spiraling out but if you only want to compute up to a limited time you could use it with a pretty small time step.
But you can do better.
Now backward Euler was the other way.
So it's just the contrast of the two that you want to see.
Neither one is an outstanding method.
They're both only first order accurate.
So backward Euler, this time step in time is still the difference that replaces the derivative.
But now everybody notices the big difference.
The slope is being computed at the end of the time step.
And that's more stable.
That's the one that spiraled in.
And I would call this method explicit.
What does explicit mean?
It means that I can find u_(n+1) directly.
This method I would call implicit.
And what does implicit mean?
Implicit means that u_(n+1) is appearing on the right-hand side as well as the left so I've got to move it over and I have to solve a system of equations to find the next u_(n+1).
So you see that basic separation of ideas here.
Forward, faster, but less stable.
Backward, slower, generally more stable and in fact, in this case somehow, too stable.
It dissipates more energy.
You don't want to lose all your energy.
So you look for something better.
And also you look for something more accurate.
I want to suggest a more accurate method, the trapezoidal method.
Which just follows that basic principle that if I center things at halfway I'm probably going to pick up an order of accuracy.
So centering it halfway I took half of backward Euler and half of forward Euler.
So is this one explicit or implicit?
Implicit.
Implicit.
u_(n+1) is still appearing here, has to move over there.
So if I move it over I could say here I have, let me multiply up by delta t and bring the u_(n+1) over here.
So I'd have u_(n+1).
I'll have minus delta t over 2A.
All that's what's multiplying u_(n+1).
Right?
When I multiply up, let me do that.
Let me make it easy for your eyes.
I'll multiply up by the delta t. And then I'm bringing this part, the implicit part.
So that's my left-hand side.
And what's my right-hand side?
u_n is going over as the identity plus delta t over 2A.
All that's what's multiplying u_n.
Good.
So this is the matrix that has to get inverted at every step.
Of course, in this model problem that's not expensive to do.
if we have the same matrix at every step, if we have a linear time-invariant system, well we can just invert it once or factor it in into L times U once.
We don't have to do it at every step.
So that's, of course, very fast.
But when the problem becomes non-linear we're going to start paying a price for a more complicated implicit part.
What I'm interested in right now just to sort of see these differences, there's a particular matrix A that is my model here.
It's antisymmetric.
Notice it's antisymmetric and that sort of goes with conserving energy.
I'm not going to in a first order system, that antisymmetric tells me that the eigenvalues of that matrix are pure imaginary.
And so I'm going to get e^(it).
I'm going to get things that don't blow up, that don't decay.
So I want to see what's the growth factor?
Suppose you want to understand is this method good, what's its growth factor?
So that will tell me about stability.
It'll tell me about accuracy, too.
So its growth factor, all I want to do is put everything on the right side of the equation.
I want to write it in this form.
So that G will be the growth matrix.
And what is G?
Well, I just have to invert that.
That's I minus delta t over 2A.
The inverse of that, so that's the implicit part, times the explicit part.
I'm not doing anything difficult here.
Just seeing how would you approach to understand whether the solution to this is growing, decaying, or staying as we hope with maybe, constant energy.
Because that's what the true solution is doing.
The true solution, you remember, is just going around in a circle in our model problem and just oscillating forever in our model system.
What about the growth or decay or whatever of that growth factor?
Well again, this is the point where I would look for eigenvalues.
If I have a matrix G, and everybody recognizes that after n time steps, what matrix am I going to have?
What matrix connects u_n to the original u_0?
That matrix is?
G^n.
At every step I'm multiplying by a G. So with finite differences, you have powers.
That's the rule.
With differential equations, you have exponentials.
With finite differences, finite steps, you have powers of G. So I'm interested in G^n and what tells me about that is the eigenvalues here.
If I had to ask for the eigenvalues of this matrix, they would be the eigenvalues of G. So eig(G), can I say for this case.
Actually, can I just say what it would be?
What are the eigenvalues of this guy?
The eigenvalues of that factor are one plus delta t over two times the eigenvalues of A.
That's what we're getting from here.
And here, this is the inverse.
So its eigenvalues will come in the denominator.
This'll be one minus delta t over two times the eigenvalues of A.
That's pretty good.
That's pretty good.
Actually this sort of shows me a lot.
Well, in the case of this model example, the eigenvalues were i and minus i.
This is for the special case A equals [0, 1; -1, 0].
This will be one plus i delta t over two divided by one minus i delta t over two.
And the complex conjugate for the other eigenvalue.
It'd be two eigenvalues.
Let me work with the one, let me take the one that's i and then there would be a similar deal with minus i.
That's our eigenvalue of G. Where in the complex plane is that number?
If I give you that complex number.
We're meeting complex numbers here.
We're not meeting complex functions, just we have to be able to deal with complex numbers.
Actually, when I look at those two numbers, what do I see right away about them?
How are they related?
They're conjugates, right?
Conjugates of each other.
This one is in the complex plane, I would go along one, the real axis and up delta t over two.
And this one I would go down delta t over two.
And that symmetry is what I, the word you used, the complex conjugate.
Compare that length for the top with that length for the bottom and what do you get?
Same length.
So I'm concluding that in this case the magnitude of these eigenvalues of G is what?
So magnitude, absolute value, is just the absolute value of that divided by the absolute value of that.
It's this distance divided by this distance.
And you told me already the answer is one.
The eigenvalues are right on the unit circle.
In some ways that's wonderful.
The solution if you compute exactly will stay on the unit circle.
Of course, it will not be exactly the same as the continuous solution.
But the energy won't change.
We're not going to spiral out.
We're not going to spiral in.
It's this average and it's more accurate.
So trapezoidal method is like, the workhorse for finite element codes.
Trapezoidal method is a pretty successful method.
What's the price?
The price is this implicit stuff that you have to solve.
So I should say it's the workhorse among implicit methods.
And it's simple.
So just to have a picture, finite element codes usually are not shooting for great accuracy.
Many finite element calculations, you're happy with two or three decimal places.
So we're not shooting for great accuracy.
Second order is very adequate.
What we don't want, of course, is to be unstable.
We don't want to lose all the energy.
So this is really a good method.
I have to say it's a good method, but not perfect.
For linear problems, for my model problem this is fine.
For a real problem, a difficult problem in mechanics you might find that the energy, you might find it goes a little unstable and non-linearity tends to grab onto a little instability and make it worse.
So like Professor Bathe if you take his course on finite elements, this trapezoidal rule-- and many people have reinvented it.
There's a Newmark family of methods and with special parameter, you get back this one and everybody makes that choice practically.
There's just a host of finite difference methods.
I'm wondering whether you want me to tell you one more.
Do you want one more finite difference method?
Just to like, see what.
You said yes, right?
One more method.
One more and then, this is a subject on its own.
But just to see what else could you do.
What else might you do?
And let me say why you would.
And Professor Bathe had to do it.
In some problems he found that with non-linear problems, he found that this method, which for a perfect linear problem stays exactly one that's great, but you're playing with fire.
To be right on the unit circle, if non-linearity pushes you off then you wish you had not tried tried for that perfection.
So let me describe a backward difference.
Let me write down, I'll call this BDF2.
All these formulas have names and numbers.
So this would be Backward Difference Formula second order accurate.
You'll see, it goes two steps back.
So here it is.
u_(n+1)-u_(n-1) over delta t. And then there's another term which gets the second order accuracy.
It happens to be u_(n+1)-2u_n+u_(n-1) over delta t. And then that really is delta t. Equals Au(n+1).
It's good practice.
That formula came from somewhere.
What if we look at it.
What do we see?
What's the picture for a formula like that?
Is it implicit or explicit first, our first question.
Implicit.
Because it's using, this right-hand side involves this.
And if it was a non-linear equation, whatever that right-hand side is, not linear, would be evaluated at the new step and therefore would have to go back over here.
It is second order accurate and I won't go through the checking on that.
And is it stable?
That's another question.
We have to find eigenvalues here.
Let me not go through all details there.
It is stable.
And it's slightly dissipative.
It's not as dissipative as backward Euler.
There you're losing energy fast.
Here the eigenvalues, the thing would stay much closer to the unit circle than backward Euler.
I'll just put up there so that you see.
What are other features that you see right away from this method?
The fact that it involves not only u_(n+1) and u_n, but also u_(n-1).
What does that mean?
How do I get started with that method?
Right?
This calls to find the new u. I need the previous one and the one before that.
No problem, I have them.
Except at the start I don't.
So it'll need a sort of special start to be able to figure out what should u_n be.
It'll need a separate formula to decide.
And it could use one step of backward Euler to find u_1.
And then take off, now finding u_2 from u_1 and u_0.
And then onward.
So that's quite fast.
More stable, good method.
And you maybe you can see that I could get more formulas by going back further in time.
And by doing that I can get the accuracy higher.
So that's good to see that particular BDF2.
So that's a backward difference formula.
Oh, just to mention what's developed.
So this is stable.
It actually loses a little energy.
So in fact now both Professor Bathe and I have studied the possibility of taking a trapezoidal step, which was a little dangerous in the non-linear case because you were playing with fire, you were right on the circle.
And then put in a backward difference step to recover a little stability.
And then trapezoidal backward difference.
So split step.
Split the step into a trapezoidal part and a backward difference part.
That's actually discussed later in Chapter 2.
Section 2.6 and I might come back to that.
I just wanted to say that much this morning.
Having got as far as forward and backward Euler.
I didn't want to leave you without a better method which is the trapezoidal method.
I could take any question.
I would like to devote the second half, if you're ok to just change gear, to beginning on least squares.
So this was our kind of excitement to have time dependence for a little while.
But now I'm going back to steady state problems.
So they're not moving.
I'm looking at equilibrium.
And what I'm going to do now in then next lectures is more to see this framework that we identified once for the masses and springs, to see it again and again.
Because it's the basic framework of applied math.
So now I'm ready for least squares.
Least squares.
What's the problem?
Well the problem is I'm given a system of equations Au=f.
These f's are observations.
The u is the unknown vector.
You say what's different here?
It's just a linear system.
What's different is too many equations.
This is an m by n matrix with m bigger than n. Maybe much bigger than n. So what do we do?
We've got too many equations and no solution, no exact solution.
I would say probably that what we're coming to, what we're starting on today is the most important, the application of linear algebra that I see the most.
So it interests everybody.
It interests engineers, scientists, statisticians, everybody has to deal with this problem of too many equations.
And those equations come from measurements so you don't want to throw them away.
I don't want to just throw away.
I want to somehow get the best solution.
I'm looking for the u that comes closest when I can't find an exact u.
So that's the idea.
There's no exact solution and the problem is what is the best u.
And I'm going to call that u hat.
My favorite choice of u will be u hat.
So I'm going to get an equation for u hat.
That is my goal.
And what I'm starting here just goes all the way to, there will be weighted least squares, there will be Kalman filters.
It's a giant world here of estimating the best solution when there's noise in the right-hand side.
And what's the model problem?
Always good to have a model problem.
Let me draw a model problem.
Model problem is fit by a straight line.
So say C+Dt shall I use that as the C+Dx.
It's got two unknowns.
So n is two.
But m is big.
What do I have?
Let me draw the picture.
You've seen this often.
This is the t direction, this is the f. These are the measurements.
So I measure at time t=0 let's say.
I measure my position.
That would be f_1.
At time t=1 I've moved somewhere, I've measured where I am.
I'm tracking a satellite, let's say.
So I'm tracking this satellite.
Well the times don't have to be equally spaced.
I'll take the next time to be three.
Let's say the engine is off this satellite.
If the measurements were perfect-- Does that look too perfect to you?
It's almost on a straight line, isn't it?
Of course my point is that measurements, well I mean, of course in reality measurements would be close to a straight line.
I'm going to have to draw, in order for you to see anything, I'm going to have to draw a really.
Suppose f_1 is one.
Suppose it starts at position one and suppose f_2 is two and this guy will be.
Where do you want me to take it?
Let's see, if it was linear, what would be the?
It would be four, right?
So can I take a different number?
Three.
Is three okay?
Because five I haven't got space for.
And you don't want to see pi or some dumb thing or e. So let me take three.
I want to fit that data which is we're saying close to a straight line, I want to fit it by the best straight line.
So the best straight line would go probably, I don't know what your eye suggests for the best straight line through three points.
Do you see I've got three equations, two unknowns?
That's the first point to see.
Somehow I'm trying to fit three things with only two degrees of freedom and I'm not going to succeed usually.
But I'm going to do my best and probably the best line goes sort of, it won't exactly go through any of them.
So I'm doing the best least squares approximation.
And what does that mean?
Well, what would the three equations be?
What does my linear equations, my unsolvable ones, say that at time zero?
So at time zero, at time one and at time three, at each of those times I have an equation C+Dt should agree with.
So that C+D time zero should match f_1.
At t=1 my line will be C+D*1.
It should equal f_2.
And at t=3, the height of the line will be C+3d and I would like it to go through that height f_3.
But I'm not going to be able.
If there was noise in the measurements that system, that's my unsolvable system.
What's the matrix?
I want to write three equations and you're getting good at seeing three equations like so.
So I've a 3 by 2 matrix.
And my unknown u is C, D. Those are my unknowns.
And my right-hand sides are these heights, well, I decided on particular numbers, one, two and three.
One, two, three.
And what's the matrix?
What's the matrix A that you read off when you see that system of equations?
The first column of the matrix is?
All ones because that's multiplying the C's.
And the second column of the matrix is the times.
Zero, one, three, is that right?
That multiply the D. So this is the same as that.
So here I'm in my set-up.
I'll erase m equal big because m was only three, not that big.
What's the best answer?
What's the best u hat?
The best u hat will now be C hat and D hat.
The best I can do.
I need some idea of what does best mean.
And there is not a single possible meaning.
There are many possible ways I could say the best line.
One way would be to make the, well, what could a best line be?
I'm going to have three errors here, right?
That did not go right through the point.
This did not go right through the point.
They came pretty close.
I've got three small errors.
e_1, e_2, e_3.
Those are the errors in my equations.
So I will get equality when I add in the e_1, e_2, e_3, the little bits that will bring it onto the line.
One idea.
Make the largest error as small as I can.
Minimize the maximum of the e's Try to balance them so no e, no error is bigger than the others.
Look for that balance.
That's a reasonable idea.
But it's not the least squares idea.
So what's the least squares idea?
The least squares idea makes the sum of the squares of the errors as small as possible.
So the least squares idea will be to minimize the sum of the squares of the errors.
e_1 squared plus e_m squared.
It would be just e_1 squared plus e_2 squared plus e_3 squared.
What is this?
Let me began to write this in matrix.
I want to bring in the matrix here.
This is the error.
The error is the difference between the measurements and Au.
So that's what I'm trying to make small.
I'd love to make it zero but I can't.
i've got more equations than unknowns.
There's no two unknowns that will make all three errors zero.
So I want to make the errors small.
And this is the length of e squared.
The length in this sum of squares method.
It's a pretty good measure of the error.
Gauss was the first to apply least squares.
What I'm going to do today is Gauss.
Who was, by the way, the greatest mathematician of all time.
And here, he was doing astronomy actually.
And writing in latin.
The message got out somehow.
So his idea was sum of squares.
So this e is the distance between f and Au.
I have to begin to write.
I have to write some things.
I can write some things out in detail, but then I also, at the same time, have to carry along the way I would look at it for any matrix A and any right-hand side f. So do you see that this is the error?
The meaning of this double bars squared is exactly that.
That it's the sum of the squares of the components.
So that's where the word least squares come in.
Can I just say what's better about least squares and what's maybe a drawback.
So actually this next sentence is pretty important in practice.
What's better about least squares, what's really nice about least squares is well, for one thing, the equations we get for the best C, D will be linear, will be linear equations.
You may say, not surprising, I started out with a linear system and I'm going to end up with a linear system.
Actually I prefer to use well, might be too late.
Next lecture I'm going to put b in there for the right-hand side.
But I'll leave it with that for now.
So good point is we'll get linear equations.
The not so good point in some applications is when I look at the squares of errors, well big errors, outliers, really bad measurements have a big influence on the answer because of getting squared.
So if I have ten readings that are very accurate but then in an eleventh reading that is way off and I don't know it and if I don't realize that that's way off, then that eleventh error will-- It's like having a whole lot of points close to a line and then another point way off.
That will have a significant effect on the best line.
So you might say too great an effect.
Depends on the application.
I just had to say before starting on least squares, as always, there are advantages and disadvantages but the advantages are very, very great.
So it's an important idea here, least squares.
I'm ready now to ask for the equation for u hat.
So the equation for u hat is the u that minimizes here.
So we have touched on minimizing quadratics.
This is squares.
I could expand that out as f minus Au transpose times f minus Au just to see another way to write it.
The length squared of a vector is always the transpose.
It's inner product with itself.
And I could split this out into all these different terms.
I would have then, some quadratic equation to minimize.
In other words, let me jump to the answer.
Let me jump to the equation for the best u.
And then come back to see why.
Because you must see what that equation is.
It's the fundamental equation of, this might be called linear regression, fitting data.
You're just constantly doing it.
So what is the equation for the best u hat?
Can I put it here?
This'll be equation that we get to.
It'll be A transpose A.
You're not surprised to see A transpose A up here.
First of all because this is 18.085 and also because this A is rectangular and when you have rectangular matrices, sooner or later A transpose A comes up.
So that's the matrix.
And then the right-hand side is A transpose f. So that's the key equation for least squares.
That's the central equation of least squares.
And let's just see what it looks like.
You could say the way I arrived at it, Imean the short way is this is an equation that I can't satisfy.
I multiply both sides by A transpose and now this is the equation for u hat.
And what that did was kind of average out the m equations.
Because how many equations do I now have?
A as m by n. What's the shape of A transpose A?
Everybody's on top of that, right?
The shape of A transpose A is square, n by n. Because A transpose is n by m. n by m times m by n leaves us an n by n. So we've averaged the m equations that were too many to get n equations.
And of course this is what it should be, n by m m by one.
So it's m by one.
It's a good right-hand side.
That's the equation of least squares.
That's the equation I want to explain, understand and solve.
Actually why don't we solve it for this particular problem just to see the whole thing for this example.
Just to do it.
So there is A.
Here is u.
And here is f. what shall I call these?
They're mostly called the normal equations.
that's one possible word for the key equation of least squares, the normal equations.
Can you tell me these matrices A transpose A.
And u hat I know.
That'll be the best C and the best D. And over here can you compute A transpose f?
If I write A transpose above it, will that help you do these multiplications?
Let me just do.
So there was the matrix A.
Let me write A transpose above it.
So it has a row of ones and then a row of times.
So what shape is the matrix?
The A transpose A matrix.
It's going to be that A transpose times that A.
The size will be two by two, right?
Two by three times three by two, it's averaging out to get me a two by two matrix.
What's the first entry of this?
Can you do A transpose times A just so we see this matrix.
Three.
And off the diagonal?
Four.
And here?
And you knew it would be symmetric.
And here?
Ten.
And tell me A transpose f while we're at it.
So that's just a vector.
If you multiply that by the right-hand sides, looks like a six.
Is that right?
11, maybe.
Is that right?
Two, nine making 11, yeah.
So those are the numbers.
I can't write those numbers, write A transpose A without asking you to tell me one more time what kind of a matrix have I got here?
It's symmetric positive definite, right?
We know that's going to be.
And we see it.
Our test for positive definite might be the determinant, that one by one determinant is three, that two by two determinant is 30 minus 16; 14 positive.
We got a good problem here.
And I could solve for C hat and D hat.
I see the numbers are not coming out fantastically but they would produce a line that would, I'm pretty sure, it would be, this looks optimal to me.
If I rotated any, I'm going to make things worse.
I think it would look like that.
So that's the system that you end up with.
But why?
You really have to understand where did this equation come from.
Where did it come from?
It's worth understanding, this least squares stuff.
So I'm going to try to draw a picture that makes it clear where that equation comes from.
So what am I doing here?
Au=f.
Start there.
And the particular A was, I'll even copy the A.
It was 1, 1, 1; 0, 1, 3 multiplied u to give me f as .
But of course, I couldn't solve it because I don't have enough unknowns.
What's the picture?
Everybody likes to see what's happening by a picture as well as by algebra.
So the picture here is I'm in three dimensions and I have a vector .
So 0 in that direction, one in that, three up.
So somewhere there is my f. Now I'll put in C, D here.
What is the equation asking me to do?
Which actually, I won't be able to do because I can't solve it.
But the equation, how do we see a system of linear equations?
If I have a system of linear equations I'm looking for numbers C and D so that C times column one plus D times column two gives me that.
That's how I think of a system of equations.
A combination of the columns.
Tell me what vectors do I get if I take combinations of the columns.
Well, if I took the combination C=1, D=0 I would just get the first column.
So that's a candidate.
<1, 1, 1>, I don't know where that is.
Wherever  might be.
I'm not too sure where to draw .
I want to go one there, one there and one there.
Damn.
Let's define that to be the vector  right there.
Wait.
You let me put that up there and I didn't mean to, right?
f should be zero, what, sorry?
f was , yeah.
Damn!
Don't let me make mistakes.
These mistakes are permanent if you let them slide by.
That's it, same point.
I didn't have the point right in the first place so now it's just perfect.
There it is.
Before of course, if I had  I could've solved the equation.
But with  I can't.
Here's the situation.
This vector is not a combination of those two.
Because the combinations of two vectors, what's the picture?
If I try to draw, if I look at all combinations of two vectors,  which is that vector,  which is maybe this vector let's just say.
If I take the combinations of these two column vectors, what do I get?
Now this is for everybody to know.
If I take the combinations of two vectors here in three-dimensional space I get a plane.
I get the plane that contains those vectors.
So this I could call the column plane or the column space.
This is all combinations of the columns.
That's the same thing as saying this is all the f's that have exact solutions.
So let's just see this picture.
This particular right-hand side is not in the plane, right?
That's my problem.
This particular vector f points out of the plane.
But if I change it a little, like if I change it to .
Do you see that that would?
What's different now that I've changed it to <1, 2, 3> for a moment?
What's different about ?
Where is in my picture?
Do you see what's great about ?
It is a combination.
Right?
is a combination.
With C=1, D=1.
It would satisfy the equation.
So where is <1, 2, 4> in my picture?
It's in the plane.
The plane are the heights that do lie on a straight line.
So the plane are all the ones that I can get exactly.
But this vector, these observations, I couldn't get exactly.
So let me, in 30 seconds or less, let me tell you the best thing to do.
Or let you tell me the best thing to do.
I have a right-hand side that's not in the plane.
I can get my straight lines correspond to vectors, right-hand sides that are in the plane.
So what do I do?
I project.
I take the nearest point that is the plane as my right-hand side.
I project down.
And it's that projection that's going to lead us to the equation that I'm shooting for, A transpose Au hat equals A transpose f. This comes from projecting f down into the plane where straight lines do work exactly.
So there's an error here that I can't deal with.
And there's a part here, the projection part, that I can deal with.
This is important and it's fun and we'll come back to it Monday.
Thanks for patience.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare ocw.mit.edu
Ready for the least squares lecture, lecture 11?
Homework is just being posted on the web.
It'll be due, it's really to help you practice, get some experience on these sections for the first exam.
That's Tuesday evening.
So eight days away.
So the homework will be due the day after.
And actually, we'll try to move the review session to Monday next week so you can ask me any questions about the homework or any review material.
So that's all a week away and this week we get two great examples.
Least squares is one that comes today.
But could I first, because I keep learning more, and I've got your MATLAB homeworks to return, I keep sort of learning a little more from your MATLAB results and I think because we spoke about it, it would be worth speaking just a little more.
So I'm going to take ten minutes about this convection diffusion equation in which I put in a coefficient d, a diffusivity just to help get the units right.
So this is your example.
And it had d=1 of course.
Well first I realized that later in the book I completely forgot that I discuss this problem.
About page 509 I think.
I discussed it a little bit.
And just because it's worth, since we invested a little time, the little bit more will pay off.
So first of all, the point is here we have convection competing with diffusion.
And always there's some non-dimensional number.
Here it's called the Peclet number.
Actually, there's an accent on one of those e's, Peclet number.
Which measures the ratio, the importance of convection relative to diffusion.
So it's V times a length scale in the problem divided by d. So then that has the same units as that if the result is dimensionless.
Maybe you know the Reynolds number.
This is very like the Reynolds number, which also measures in Navier-Stokes equation the importance of convection, advection and diffusion.
There in that equation the velocity, V, that's a non-linear equation, Navier-Stokes, it's tremendously important and many codes to solve it, lots of discussion, theory still not complete.
In that problem, the V is u.
It's non-linear and the term there that we took as a constant, as a given constant V, it's the same as u.
So in the Reynolds number, this would be u, a typical velocity u, times a typical length scale, which would be like one in our zero to one problem, divided by d or mu or nu, whatever number we use.
So it's like the Reynolds number.
And then it's turned out for this problem that people also use a number that gets called the cell Peclet number where the length is taken to be half the cell size, delta x over two.
Let me call that number P. So that's P. And what's my point?
This equation's important enough to sort of see a little more about it than just the numbers that come out.
So the MATLAB homework which you did really well set up finite differences for this.
Right?
And found the eigenvalues and solutions.
It's the eigenvalues I want to say a little more about.
Because you set up a matrix K over delta x squared and V times the center difference over delta x.
And I guess I call that whole combination L asked you about the eigenvalues of L. And you printed them out correctly.
But there's more there than I think we have understood.
And I want to make some more comments about that.
Because it's quite important.
And the comments are clearest if I just reduce to a n equal 2.
So that matrix, well the off-diagonal part of that matrix had some number v and some number c. Actually we could figure out what was the b in this.
This produced a minus 1, 2, minus 1, right?
So part of the b was the minus 1 over delta x squared.
And then from this was a plus V and a 1 over, well it's a center difference so I should divide by 2 delta x.
Is that right?
Is that what a typical off-diagonal thing that in the matrix that you displayed?
That's what's coming from the off-diagonal of K. And this is what's coming from the center difference c. And then what would this c thing be?
Well the c is below the diagonal so it's also at minus 1 over delta x squared.
But now this is a difference, so it's going to be a minus, right?
I think those would have been your entries for b And c. So can we just think first, what are the eigenvalues of that matrix?
It's a two by two simple problem.
The trace is zero plus zero.
So that the eigenvalues will be a plus minus pair because they have to add to zero.
And I think that's the plus minus pair you get.
Let's just check.
What's our other check?
They will add to zero, the plus the square root and minus the square root.
And the product of the two eigenvalues, lambda one times lambda two, will be, we have one of them is plus, one with a minus, so it'd minus b c. And that's correctly the determinant.
So it's good.
These are the correct eigenvalues.
Now let me ask you about the signs of b and c. If b and c have the same signs, like maybe even equal, one, one, what are the eigenvalues?
So in that symmetric case if b and c are equal the eigenvalues are?
Right here.
If b and c are equal, say equal to one, the eigenvalues are plus and minus one.
But what if the signs are opposite?
Everything changes.
What if b is one and c is minus one?
That matrix would then be a 90 degree rotation.
It would be anti-symmetric if b was one and c was minus one.
Our formula is still correct, but what does it give us?
If b is one and c is minus one what have I got here?
I've got i.
So the eigenvalues change from plus and minus one in the symmetric case to plus and minus i in the anti-symmetric case.
And I think that's what you guys saw at a certain level of V. I hope you did because that was the point about eigenvalues.
Now you may say, what about the diagonal?
Well I claim diagonal is very simple.
What's the diagonal?
Now I'm going to allow myself a diagonal and I'm just going to change.
What happens if I have a and a?
Same entry on the diagonal.
What are the eigenvalues now?
This is just like, a great chance to do some basic eigenvalue stuff.
What are the eigenvalues of that matrix?
Well I've added a times the identity.
I've just shifted that matrix by a.
So the eigenvalues all shift by a.
So the eigenvalues are now a plus and minus.
So no big deal.
So you say that the a is actually not important, not the key to this question of are they real or do they go complex.
So the eigenvalues of this are real when b and c have the same sign.
If b and c have the same sign, I have a square root, no problem.
When b and c have opposite sign, what do I get?
When b and c have opposite sign, I'm taking the square root of a negative number and I've gone complex.
Do you see that the change from real eigenvalues, which gives a nice curve, to complex eigenvalues, which gives a very bumpy curve for the solution, just happens when like, for example, b-- is it b that's going to go to zero maybe?
And then beyond that?
Well this sign is for sure negative, right?
So c is staying negative.
And originally for a little delta x, b is also negative.
What's happening here?
I think that the transition that you I hope observed comes when b hits zero.
When the combination of V and delta x is such that at b=0 we switch from real eigenvalues to complex eigenvalues.
And when is b=0?
That's when this negative guy off the diagonal just exactly cancels this one.
So b is zero when what?
So if this equals this, one over delta x squared is equal to V over two delta x, let me multiply both sides by delta x squared so that I have a nice one there, multiplying by delta x squared will put a delta x up here.
And what have we discovered?
This is why I wanted you to see it.
That the transition comes when the Peclet number is one.
So that Peclet number, that cell Peclet number is exactly the point but we observed of transition from real eigenvalues to complex eigenvalues.
And that's the transition.
So it's that combination, this is the Peclet number, cell Peclet number, it's that combination, P_cell maybe.
We've done the computations and now we gradually get back to the meaning.
And I just wanted to take this step back to the meaning to see when do those numbers start going complex.
You may have noticed or you may not have noticed that it'll happen when one of those, when that upper diagonal changes sign.
Now you could say, ok that's the eigenvalues.
What's the consequences for the shape of the solution?
Well, I haven't figured all that out.
I'd be happy to have some more thoughts about that.
But what you noticed, I think, in the computations is if V got too big so that that P was bigger than one, if V got too big, so convection was dominating and our delta x was not small enough to deal with it, you should have seen the points on the discrete values were oscillating instead of a proper smooth.
I mean, the proper, with a large V, the correct solution, I think, is practically nothing for here and then it goes, this is a really large V, take V to a thousand or something.
It climbs up like mad.
Here's the halfway point where the load is.
And then it goes along here and then it climbs down like mad to satisfy the boundary condition.
I didn't know that that's what would happen for large V. What I'm saying is, and undoubtedly it could be understood physically, so I guess what I'm saying is there's just more good stuff in any computation than purely the numbers.
And this is part of the good stuff in that example.
I hope you liked that.
Because I mean, here you did the work but then, to understand it is frankly still under way.
More thinking to do.
That's back to least squares.
Here's today's lecture.
So remember where we started last time.
Au=b.
Last time I wrote f. I regret it terribly.
I can't fix it.
But it's b. I want b there to be the right-hand side.
And I jumped to the equation that determines the best u.
There's no exact u because we've got too many equations.
You remember the set-up, we have too many equations.
There's noise in the measurements and we can't get the error down to zero.
There's some error.
And the best u was given by that equation and we want to say why.
And understand it from two or three ways.
Calculus, geometry, everything.
Can I first, because I love my little framework here, fit it in because it's quite important, this example and then others fit in.
So u is our unknown as always.
Then the matrix A in the problem produces an Au.
Now two things to notice about e, which, that's the same letter I used for elongation, here it's standing for error.
Two things to notice.
One is that the source term, which is b, comes in at this point of the framework.
When we had external forces on springs and on masses it came in at this point.
We had an f there.
So that's why I'd like to keep those two separate.
The b's are like voltage sources, they come in here.
The f's are will be like current sources, they'll come in there.
Actually it's beautiful.
One more thing to notice.
A is coming with a minus sign.
In mechanics in masses and springs we had e=Au.
Here it's natural to work with this, the error or the residual b-Au.
And that minus sign is natural in physics and in electrical engineering and hydraulics.
Where's that minus sign coming from in flow?
Well, flow goes from the higher point to the lower.
Higher voltage to the lower voltage.
And that usually produces that minus sign.
No big deal, of course.
So that step is fine with the framework.
What do we expect in that middle step?
So what's our name for the matrix that goes there?
Everybody's gotta know this framework.
C, right?
Only I've been taking unweighted least squares.
So for unweighted least squares, C will be the identity.
And C doesn't show in our equations.
So C is the identity when there are no weights, when all the equations are equally reliable.
And that's pretty common, of course.
But not always.
And we'll think, ok there is a weight e. So w, which is Ce, is weighted errors, you could say.
So the letter w comes up appropriately again.
Weighted errors.
And then what's the good weighting?
May I stay with C equal the identity for the moment?
Unweighted least squares, because that's by far the most common.
And then w and e are the same.
C is the identity.
And finally, there's the last step in our framework where we always expect to see A transpose.
And we do.
And we have to say why.
So that's where I left it last time.
That this was the picture.
This is the equation.
If I had a matrix C, it would go there and there.
Right?
Because I'd have b-Au and then I'd apply C before A transpose.
So C would slip in there before A transpose on both sides.
So that would, with the C's there, that would be the weighted least squares equation.
You see that it would be A transpose C A instead of A transpose A, but still the main facts are there.
So where does the equation come from?
So one source, one way to get the equation is from calculus.
From minimizing, from minimizing.
Set a derivative to zero, calculus.
And what's the quantity we're minimizing?
We're minimizing that squared error because this is least squares.
We're minimizing this, e transpose e, the length of e squared.
The sum of the squares of the errors.
Which is (b-Au) transpose b-Au.
Again I could say where to slip in the C matrix.
If there was one, it would go in there.
C would go in there, C would go in there.
There'd be a C in the equation.
But let's keep C to be the identity.
So I minimized.
It's a quadratic.
It's got u's times u's, so second degree.
And what's the coefficient in that second degree part?
Well, the second degree part is coming from Au transpose Au.
Right?
This times this is going to be linear.
This times this is going to be linear.
That times that is just going to be a constant, its derivative is zero.
But this times this is altogether, that times that is the u transpose A transpose Au.
Right?
So that's the quadratic part.
And my only point is it's like our old stiffness matrix.
We're seeing the matrix in here is A transpose A.
In other words, when I do calculus and maybe I'd prefer to see something than just compute away, take derivatives mechanically.
So I'm going to leave that which is done in the text, finding the derivative, setting to zero.
And what does it give?
It gives us our equation.
So that equation will come when I set the derivatives of this thing to zero.
So that's one totally ok approach.
But I like to see a picture with it.
I hope that's alright.
To take the second approach is to see why A transpose w equal zero.
Why is that?
What's going on in that key step?
This is always the key step.
This is like the set-up step.
This is the weighting step with constants coming in.
And here's the key step.
Let's see that.
So my picture.
Let me draw that picture again.
And my example was in three dimensions, so m=3.
I've got three equations.
The matrix A, oh I'm afraid I don't remember what it was, but I think it was something like [1, 1, 1; 0, 1, 3], was that maybe it?
Just to connect to last time.
And what I'm now calling V was the vector  was it?
Or was it not?
It was  maybe?
That's right?
And what was the point?
If I draw the vector b it goes there somewhere.
If I draw the first column of A, it goes here somewhere.
If I draw the second column of A, it goes there somewhere.
And if I draw all combinations of these columns, all combinations of that vector and that vector, what do I get?
I get a plane.
I get a plane.
There it is.
That's the plane.
That's the plane.
This is from column one.
Here's column two.
This plane is the column plane or column space.
It's the column space of A because it comes from the columns of A.
Now what's the point about this plane?
The point is that if b is on the plane then I'm golden.
If b is on the plane then b is a combination of the columns, that's what the plane is, and I have a solution to Au=b.
So b on a plane, b on the plane means Au=b is solvable.
And it could happen, of course.
Like perfect measurements.
But we can't expect it.
When we have three measurements or 100 measurements or 10,000 measurements we can't expect perfection.
So usually b will be off the plane.
Now what?
What happens when b is off the plane?
Let me just complete that picture.
And you know what's coming.
If we're going to get-- Au or Au hat is going to be on the plane so I'm looking for the best u hat.
Can I just erase this to make space for what you know I'm going to draw?
Here are these little columns, let me put them there.
What am I going to draw?
The projection.
The projection.
I'm going to draw, what's the projection?
The projection is the nearest point that is in the plane to the b that's not in the plane.
So here's the projection p. I drop down this thing.
There's the projection p, little p. That's the projection of b onto the plane.
I think your mind says yeah, that's the right choice.
And do you want to tell me what this?
That is the part that we can't deal with.
The part we can't improve.
We've made it as small as we could and it's e. That's the error e and this p is the best guy that is in the plane.
Do you see that this is the picture.
You get an actual picture of what's going on.
You're splitting b, the measurements into the part you can deal with, the projection, the Au hat that is in the column space.
It is a combination of the columns.
Those points do lie on a line if I'm doing straight line fitting.
And the part that you can't deal with, the e, the difference, b-Au, which is not in the plane.
And now I'm still looking for the equations.
Right?
I've just named some stuff.
But I haven't got an equation for that projection.
So what's the key fact?
What's the key fact in this picture that's going to lead me to an equation for p and e and u hat and everything?
The key fact is that that dotted line is perpendicular, perpendicular to the plane.
If I'm looking for the closest point, everybody knows project, that's what projection involves.
Go perpendicular.
This is a right angle.
That e is perpendicular to the whole plane.
Not only perpendicular to p, it's perpendicular to everybody in that plane.
Right?
I'm dropping the perpendicular to the plane.
Do you accept that?
Because if you do, we're through.
We just write down the equations for perpendicular and we've got what we want from the picture instead of from a calculation.
So what's the idea?
So e is perpendicular to the first column.
So b in the plane, we would be golden.
Let's suppose we're not in the plane.
So now we have this 90 degree angle, this perpendicular projection.
And it tells me that the first column-- oh I better name the columns.
Can I just call this column a_1?
That first column is a_1 and the second column is a_2.
So those two columns, whatever they are, are the guys whose combinations give us the plane.
And it's the plane that we're projecting onto.
It's the plane of all combinations that comes up here.
So what's this 90 degree angle?
It says that a_1 is perpendicular to p, right?
Sorry!
Say that right for me.
The first equation says that a_1 and what are perpendicular?
e, thank you, e. So the first equation says that a_1 transpose e is zero.
And the second equation says that a_2 transpose e is zero.
Those are my two equations.
I have to convert those now into matrix language because I've done them two separate-- vector, I mean vector language, and I want to get into matrix language.
But it's easy to do.
Here I have, look, if I have two equations, let's get a matrix here.
What's it saying?
a_1 transpose and a_2 transpose, what are those?
They're the rows of A transpose.
So the matrix way to say that is A transpose e equal zero.
In other words, this is saying both at once, right?
The first row of A transpose times e gives zero, the second row of A transpose times e is zero.
So it's A transpose e equal zero which is what we wanted in this case where w and e are the same.
Because C is the identity.
And let's just go one step further and see.
That's A transpose (b-Au hat) is zero.
Remember this zero stands for , right?
I wanted to put the two equations together.
So I've got two components on the right-hand side.
And then I just plugged in what b is.
And now everybody sees it, right?
Everybody sees that we've got the picture, this 90 degree angle was the key to these equations.
Because if I put A transpose Au hat onto the other side, I've got exactly the normal equations that I wanted.
We're taking the time to see the picture and the form of the equations.
Then I can plug in the numbers, but the thinking is where the equations come from.
so we're there.
Now what to do next?
Now we've understood where the equations come from.
I didn't go through the steps of taking the derivatives, but that would work.
Or this picture.
I love this picture.
Let me stay with that a little bit longer.
What is u hat?
Can I just go over here to say, ok what have we got here?
We started with Au=b and then we got the projection was Au hat.
But now what is u hat?
I'm just going to assemble things here.
u hat we figured out by the 90 degree angle comes from this equation, which is that equation, which is A transpose Au hat equal A transpose b, the central equation.
That's the central equation.
Now plug in u hat here so I get a formula for the projection.
While we're doing all this stuff we might just as well put those two pieces together and have a formula for the projection.
So it's A times u hat-- I hope you like this formula.
It's kind of goofy-looking but you'll remember it.
What is u hat?
The whole point is that this matrix is good.
It's square, it's symmetric, it's invertible, we'll have another word about that.
And now I'll invert it times A transpose b.
That's the goofy formula that I wanted you to see.
The projection of vector b onto these columns of A comes from applying this matrix, sometimes I call it the matrix of four A's.
Now it's worth looking at that matrix.
Often I'll call that matrix capital P. It's the projection matrix.
You give me any vector b, I multiply it by this matrix and I get the projection.
It's just worth seeing what this matrix P, these four A's, what are projection matrices like.
Now first of all, when I have an inverse of a product any reasonable person would say ok, split that into A inverse times A transpose inverse and simplify the whole thing.
And what will happen?
It's not going to be legal, but let's just pretend.
If I split this into A inverse times A transpose inverse and simplify, what do I get for P?
Do you see it?
I'll get A and if I try to split this into that, what do I have here?
I've got the identity.
That's the identity.
That's the identity.
The result is the identity.
That doesn't look good, right?
P is not the same as P. This matrix cannot be split into these two pieces.
A is rectangular, that's its problem.
If A was square, oh yeah, think about the case when A is square.
Suppose m equals n. That case'll be included here.
If m equals n and my matrix is square and invertible and golden then all this works.
The projection is the identity matrix.
And what's with my picture?
What's my picture look like in the case where A is a square matrix?
Give it another column.
Fit this thing by a quadratic.
So if I was fitting instead of by a straight line, by a quadratic, it turns out I'd have zero squared, one squared and three squared in that column.
I'd have a three by three matrix.
It comes out to be invertible.
Now what's going on?
Now what's my problem Au=b?
Now suddenly m is still three, but now n is three.
b is?
And what happened to the plane?
b is in there.
And now what's there?
It's now the combinations of what?
Why did that plane come in?
That was the combinations of two columns.
But now I've got three.
The combinations of three columns, those three columns of an invertible matrix is what?
Are you with me?
If I have a three by three invertible matrix, these three columns independent, pointing off different directions, not in a plane, then when I take the combinations I get?
I get R^3.
I get the whole space.
I get everybody.
Every vector including this b and any other b you want to suggest will be a combination of these three guys.
So what's my picture here?
My picture is that plane grew to be the whole space.
So what's the projection of b onto the whole space?
b itself.
And what's the error?
Zero.
Good.
So that's the nice case.
That's the standard case that we've thought about in the past when m equalled n. In that case P is the identity and that'd be all true.
But normally it's not.
So I want to come back to this P just to mention an important fact about P. And it comes again from the picture.
So this is a projection.
This is what I'm calling the projection matrix.
It's the matrix that does the projection.
And there it is.
Four A's in a row that multiplies b.
Now here's my little question.
So linear algebra's full of these different kinds of matrices.
Rotations, reflections, symmetric matrices, Markov matrices, so it's just every problem has matrices.
Now here we have a projection matrix.
Now what I want to know is what happens if I project again?
If I take the vector b, any vector b, I project it and then I project again.
So project twice and just tell me, you know what will happen.
I'm back to this picture.
I project b to P and now I project again.
Where do I go?
Same place, right?
Once I'm in the plane the projection stays right where it is.
So what does that tell me?
That tells me that P squared on b is the same as P on b.
If I project twice, no change.
It's the same as projecting once.
So the projection matrix has the property that P squared is P. And actually, we should be able to see it if I write out this whole miserable thing twice.
So now I'm going to be up to eight A's.
Sorry about this, but I promise not to do P cubed.
A times A transpose A inverse times A transpose, that's one P. I'll write it again.
There's the second P. So that's P squared.
Do you see anything good there?
Do you see in here A transpose A, that combination and that combination there.
This cancels that to give the identity.
And what am I left with?
I'm left with A, the inverse times A transpose, which was exactly P. The algebra is just coming along with the understanding that we know.
So that's the projection matrix.
So this is the theory of projections in a nutshell, in a nutshell.
This is projections onto the column space of A.
Now I have to remind you about one little math point.
Not so little, I guess.
How could I say little for math?
Is A transpose A invertible?
We're plowing along as if it is, that's going to be our assumption.
But what's the condition for A transpose A to be invertible, which allows all this to work?
When is A transpose A invertible?
What I'm doing here now is I'm separating the positive definite one.
When A transpose A is positive definite, the good normal case when all our equations work, from the semi-definite one where we overlook the fact that A transpose A, where somehow the experiment wasn't well set up, we got an A transpose A that is singular.
And just to see when could that happen.
Let me just remind you.
This is important.
Why don't I give it some space.
It's really straightforward.
Let me just go through those steps again.
If it's not invertible, if some A transpose A u is zero.
This is always the risk that we have to check out and be sure we don't have and understand.
So if A transpose Au is zero, then that would lead us, I could multiply both sides by u transpose.
u transpose zero, right?
Safe.
Multiply whatever that u might be, multiply both sides by u transpose.
But what is u transpose zero?
Zero, nothing there.
Now how do I understand this guy?
Well you remember the key.
Everybody remembers the key?
You look at that thing and you say hey, if I put in parentheses in the right place that's the length of Au squared.
So that's the small trick that this multiplying by u transpose and then seeing what you've got that we've done and you should know it.
And now if the length squared is zero, what does that tell me about Au?
If I have a vector here who's length is zero, that vector must be?
Zero.
Zero vector's the only one for which the sum of the squares will give zero.
And if Au is zero I could multiply both sides by A transpose and complete the loop.
Actually I thought of that when I was swimming this morning, that line.
Just to see once again when, it's sort of interesting then.
A transpose Au equal zero which is the bad thing we hope we don't deal with.
And when does it happen?
It happens when A u is zero.
So our assumption always has to be this; that there aren't any u's, except the zero vector of course, that's always going to happen, but we always have to assume that Au is never zero.
So we have to avoid this.
So to avoid that assume A has, this is the key word, independent columns.
Since this is a combination of the columns, independent columns means what?
It means that the only combination of the columns to give zero is the zero combination.
So did I have independent columns over here?
I sure did.
That column and that column were off in different directions, they were independent.
And that's why I knew we were fine.
A transpose A was zero.
I would have to really struggle to find a, well I'd have to think a bit to find an example where we run into trouble.
These squares, well I certainly could in many applications, but the straightforward applications of fitting a straight line, A is going to be a column vector of ones and a column vector of times and those are different directions and no problem.
So that's A transpose A.
What else to do with this topic?
Because there's a whole world of estimation.
I mean, statistics is looking over our shoulder I guess.
Really, we should realize that a statistician say yeah, I know that but, and then going on.
And what is that guy, what more does he have to say?
So you've got the central ideas.
I guess the statistician comes in in this, that's the statistical constant now.
And what do statisticians compute?
They say you've got errors, right?
And of course, in any particular case we don't know what that error is, otherwise we could take it out and we'd get exact solutions.
We don't know what the error is.
What is reasonable to know about errors?
We're doing a little statistics here.
Somehow that error, that particular error of the experiment that we happen to run, and if we ran it again we'd get a different error, those errors come out of some sort of error population.
Like dark matter or something.
Just like, a bunch of errors are out there, noise.
And what could we reasonably assume that we know about the noise?
We could assume that its average is zero, mean zero.
So statisticians always, that just resets the meter.
Right?
If you had a meter or a clock that was always three minutes ahead (like this one) you would reset it.
And we'll do that one day.
So you'd reset to get the average zero.
But that doesn't mean every error is zero, right?
That just means the average error is zero.
So what's the other number?
What's the other number that statisticians live on?
It's the deviation or its square, which is called the variance.
Right.
Variance.
So that's the thing that you could assume that the errors have mean zero and have some variance.
You could suppose that you knew something about the variance.
You don't know the individual errors, but you know whether the errors are like, are very small or close to zero or large.
So this is a small variance.
So one over sigma is sort of that distance.
One over sigma.
Here, this is a large variance where the magnitude of the error could be much larger from this.
So those are the two numbers, mean zero, that leaves us just one number, and the variance, the standard deviation sigma or the variance sigma squared.
One moment on these squares.
Let me just say what the weighting matrix would be.
And then I can tell you in a moment why.
What would the weighting matrix be if our three equations, you know, that came from one measurement and this came from a second measurement and this came from a third measurement.
If they came from different meter readers with different variances, suppose, then the right C matrix will be a diagonal matrix, beautiful.
And what sits up there, what sits there, what sits there?
We don't have spring constants anymore.
We have statistics constants.
And what's the number that goes there?
That one is the third guy.
So it's associated with the third measurement.
It's one over sigma three squared.
Those are the numbers that go on the diagonal, the inverses of the variances.
And just to see that that makes sense.
If that number is unreliable, if it has a large variance then I want to give it little weight, right?
If this third meter is very unreliable I'm not going to throw it out entirely, but I know that it's variance is large and therefore I'll weight that equation only a little, with a small weight.
Suppose sigma two, so this guy is one over sigma two squared, suppose this is an extremely reliable meter.
That measurement has little expected error.
Then I want to weight it heavily.
So it has a small sigma two and that gives it a large weight.
And sigma one similarly.
So that's the weighting for the case that you can actually hope to use in practice.
I'll just mention that statisticians would also say, wait a minute.
Measurement two and measurement three might be interconnected.
They might not be independent.
There might be a covariance.
And then that gets them into more great linear algebra actually.
But if I want a diagonal matrix C that's the case when my measurements are independent.
And basically, I'm whitening the system.
I'm making the system white, making it all equal variances by rescaling.
By weighting the equations.
Ok thanks.
Wednesday is the next big example of the framework with b and f. See you then.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so this is the start.
I won't be able to do it all in one day of what I think of as the number one model in applied math, in discrete implied math, I'll say.
Let me review what our four examples are.
Just so you see the big picture.
So the first example was the springs and masses.
That was beautiful.
It's simple.
The masses are all in a line, and the matrix K, the free-fixed and fixed-fixed and free-free come out closely related to our K t b matrices.
So that was the natural place to start, and actually we also got a chance to do the most important equation in time.
Ku''.
Sorry Mu''+Ku=0.
So that was a key example.
Then least squares.
Very important, I'm already getting questions from the class about problems that come up in your work, least square problems.
Maybe I'll just mention that the professional numerical guys don't always go to A transpose A.
If it's a badly conditioned problem, in that conditioning is a topic that was in 1.7 and we'll eventually come back to, if it's a badly conditioned problem matrix a then, a transpose a kind of makes it worse.
So there's another way to orthogonalize in advance.
And if you're working with orthogonal vectors, or orthonormal vectors, numerical calculations are as safe as they can be.
Yeah.
Wall Street is more like A transpose A.
And the orthonormal is the safe way.
Alright, this is today's lecture.
You'll see the matrix a for a graph, for a network.
It's simple to construct, and it just shows up everywhere.
Because networks are everywhere.
And, just, looking ahead, trusses are there partly because they're the most fun.
You'll enjoy trusses.
I mean, it's kind of fun to figure out is the truss going to collapse or not.
It's good.
And actually, what's the linear algebra in there?
The collapsing or not will depend on solutions to Au=0.
Let me just recall the equation Au=0.
If A is our key matrix in each example, it's different in each example.
And we sort of hope that Au=0 doesn't have solutions, or that it has solutions we know.
Because if Au=0 has solutions that's the case where a transpose a is not invertable and we have to do something.
Very useful to review.
What were the solutions to Au=0, in the case of springs?
Well, there were some in the free-free case.
The all ones vector was the solution u or all constant, was the solution u in the free-free case and that's why we couldn't invert it.
But the fixed-free or the fixed-fixed, when we have one support or two supports, that removed the all ones solution.
Good.
These squares, we assume there weren't any.
We assumed because we wanted to work directly with a transpose a, the normal equations, so we assumed that the columns of a were independent.
We assumed that there were no non-zero solutions to Au=0.
Because if there were, that would have made a transpose a singular, and we would have had to do something different.
Here, this'll be a lot like this one.
Today, once you see A, you'll spot the solutions to Au=0.
This is A for a network.
And the solution is going to be that same guy, all ones.
And that only tells us again that we have to ground a node, I may use an electrical term.
Grounding a node is like fixing a displacement.
Once you've fixed one of those, say at zero, whatever, but zero's the natural choice.
Once you've said one of the potentials, one of the voltages is zero, then you know all the rest.
You can find all the rest from our equations.
So this is like this in having this all ones solution.
And as you'll see with trusses, that could, depending on the truss, have more solutions.
And if there are more solutions that's when the truss collapses.
So the trusses need more than just a single support to hold up a whole truss.
OK.
So that's the Au=0.
Now we're ready for the lecture itself.
Graphs and networks.
OK, let me start with, what's a graph.
A graph is a bunch of nodes and some or all of the edges between them.
Let me take just a particular example of a graph.
And this of course you spot in the book.
Oh and everybody recognized that, and it's probably now corrected, that in the homework where it said 3.4 it meant 2.4, of course.
And this is Section 2.4 now.
Let me draw a different graph.
Maybe it'll have four nodes, at those four edges, let me put in a fifth edge.
OK, that's a graph.
It's not a complete graph because I didn't include that extra edge.
It's not a tree because there are some loops here.
So complete graphs are one extreme where all the edges are in.
A tree is the other extreme, where you have a minimum number of edges.
It would only take probably three edges.
So just while we're looking at it, there are a bunch of possible trees that would be sort of inside this graph.
Sub-graphs of this graph, if I knock out those two edges I have a tree.
Going out, or a tree could be like this.
Or a tree could be like this.
Anyway, five edges is in this graph, six in a complete graph, it would be three edges in a tree.
OK, and the number of edges is always m. So five edges.
And the number of nodes is always n, for nodes.
So a will be five by four.
OK. And it's called, so we get a special name in this world, it's called the incidence matrix of the graph.
The incidence matrix.
Or, of course, these things come up so often they have other names, too.
But incidence matrix is a pretty general name.
OK, I have to number the nodes just so we can create the matrix, A one, two, three, four.
And I have to number the edges.
If I don't number them, I don't know which is which.
So let me call this edge one, from one to two, and I'll draw an arrow on the edges.
So from one to two, maybe this'll be edge two, from one to three.
This'll be edge three.
Oh no, let me put edge three there, would be a natural one, say from two to three.
And how about edge four there, from two to four.
And edge five going from three to four.
OK, so now I have numbered, I've identified the nodes, and I've identified the edges.
And there were five edges and four nodes.
Usually m is bigger than n. We're in this, except for trees, m will be at least as large as n. And I've put arrows on, so you could say it's a directed graph.
Because I've given a direction.
You'll see that the directions, those arrow directions, which are just to tell me which way current should count as plus, if it's with the arrow, or which way it should count as minus if it's against the arrow.
Of course, current could go either way.
It's just, now I have a convention of which is plus and which is minus.
OK, so now let me tell you the incidence matrix.
So everybody can get it right away, how do you create this incidence matrix?
A five by four.
So it's going to have five rows, one for every edge.
So what's the row for edge one?
And it's got four columns, one for every node.
So these are the nodes.
Nodes one, two, three, four.
So there's a column for every node and a row for every edge.
OK, edge one.
This is just going to tell me everything about the graph.
So exactly what's in that picture will be in this matrix.
If I've erased one, I could reproduce it's by knowing the other one.
OK, edge one goes from node one to node two.
So it leaves node one, I'll put a minus one there.
In the first column.
And a plus one in the second column.
Edge one doesn't touch nodes three and four.
So there you go, that's edge one.
Let me do edge two and then you'll be able to fill in the rest.
So edge two goes from one to three, minus one, and a one.
Edge three goes from two to three, I'll just keep going.
Minus one and a one.
Edge four goes from two to four.
And edge five goes from three to four.
OK.
Simple, right?
Got it.
That matrix has got all the information that's in my picture, and the matrix, but the point about matrices is, they do something.
They multiply vector u to produce something.
They have a meaning beyond just a record of the picture.
So a is a great thing.
In fact, what does it do?
Let's see.
So that's the matrix a that we work with.
Oh, first tell me about Au=0.
Because we brought up that subject already?
Are those four columns independent?
I've got four columns, they're sitting in five-dimensional space, there's plenty of room there for four independent vectors.
Are these four columns independent vectors?
No.
No, they're not.
Because what combination of them produces the zero vector?
.
If I take that column plus that, plus that, plus that, I'm multiplying by.
So, A I'll just put that up here and then I won't have to write it again.
A times <1, 1, 1, 1>, is five zeroes.
So that u, that particular u, of all ones is, I would say, in the null space of the matrix?
The null space is all the solutions at Au=0.
In other words, so these four columns, tell me about the geometry again.
These four columns, if I take all their combinations, yeah.
Think about this.
If I take all four combinations, all combination, any amount of this column, this column, this column, that fourth column, those are all vectors in five-dimensional space.
Now, this isn't essential but it's good.
Do you have an idea of what you'd get?
What would you get if you took, so this, think of four vectors, pointing along, take all their combinations, that kind of fills in.
Whatever fill in may mean.
And what does it fill in?
What do I get?
What's your image?
Frankly, I don't know.
I can't visualize five-dimensional space.
That well.
But still, we can use words.
What do you think?
You get a something subspace.
You got a something, you get something flat.
I don't know if you do.
It's pretty flat, somehow.
Like I'm just asking you to jump up from a case we know.
Where we had columns in three-dimensional space and we took a combination and they gave us a plane.
Right, when they were dependent?
Now, how would you visualize the combinations in five-dimensional space?
Just for the heck of it?
It's some kind of a subspace, I would say.
And what's its dimension, maybe that's what I want to ask you.
What's the dimension?
Do I get, like, a four-dimensional subspace of five-dimensional space when I take the combinations of these particular four guys?
Yes or no?
Do I get a four-dimensional subspace, whatever that may mean?
No.
Right answer, I don't.
I don't.
Somehow the dimension of that subspace, whatever I get, isn't four because this fourth guy is not contributing anything new.
The fourth one is a combination of the first three.
So I get a three-dimensional subspace.
The rank of this matrix is three.
If you allow me to introduce that key word, rank, is the number of independent columns.
It tells you how big the matrix really is.
You know, if the matrix, if I pile on a whole lot of zero columns, or a lot of zero rows, the matrix looks bigger.
But of course it isn't truly bigger.
The heart of the matrix, the core of the matrix is somehow just three.
And actually, I tell you now and we'll see it happen, can I tell you the key result in the first half of linear algebra?
It's this.
That if I have three independent columns, and by the way any three are independent, it's just all four together are dependent.
This has three independent columns, then the great fact is, it has three independent rows.
That's kind of fantastic.
Since it's such a beautiful and remarkable and basic fact, look at the rows.
That what linear algebra is all about.
Looking at a matrix by columns, and then by rows, and seeing what are the connections.
And the connection is, the key connection is, that these five rows, now what space are they in?
What what space are these rows in?
four-dimensional space.
They only have four components.
So I had four columns in 5-D, I have five rows in 4-D.
But now, are those five rows independent?
Let me just ask that question.
Are those five independent rows, are they pointing in different directions, or could any combination give the zero vector in 4-D, looking at those five rows?
What do you say, wait a minute.
Five vectors, in four-dimensional space?
Dependent, of course.
Right.
So they're dependent.
There couldn't be five independent vectors in 4-D.
But are there four in this particular case?
And here's the great fact, no, there are three.
If there are three independent columns and no more, then there are three independent rows and no more.
And we'll get to see which rows are independent.
And which are not.
That's a question about A transpose, and we haven't got to A transpose yet.
OK, are you OK with that incidence matrix?
Because this is like the central matrix of our subject.
We can figure out A transpose A, that's kind of fun.
I do a transpose a then you'll see the core computations of this neat section.
So if I do A transpose A, so I'm going to bring in a transpose and you know that I'm not just bringing it in from nowhere, that networks.
The balance law is going to involve a transpose.
So let's just anticipate.
What do you think a transpose a looks like?
Now, how am I going to do this for you?
May I write may I erase this for a moment, and try to squeeze in a transpose here?
So that you can multiply it by site and see the answer, and then you'll see the pattern.
That's the great thing about math.
You do a few examples, and you hope that a pattern reveals itself.
So let me show a transpose.
So now I'm going to take that column and make it a row.
I'm going to take that column and make it a row, it's going to be a little squeezed but we can do it.
Take that column, .
And the last column, .
OK.
So I just wrote a transpose here.
And now could you help me with A transpose A.
Which is the key matrix in the graph here.
What size will it be?
Everybody knows it's going to be square, it's going to be symmetric, and just tell me the size.
Four by four.
Right, we have a four by five times a five by four, we're expecting this to be four by four.
And what's the first entry?
Two.
Right, take row one, dot it with column one.
I get two ones and then a bunch of zeroes, so I just get a two.
What's the next entry?
Take row one against column two, can you do that in your head?
Row one, column two, the top one is going to hit on a minus one, and I think that's all there is, right?
Then this one hits a zero and those three zeroes, so.
And then what about the next guy here?
A minus one.
And the last guy?
A zero.
So that's row one of A transpose A.
Can I just look at that for a moment before I fill in the rest?
And then, when you fill in the rest it'll confirm the idea.
Why do I have a zero there?
Why did a zero appear in the 1, 4 position?
If I look back at the graph, what is it about nodes one and four that told me ahead of time?
You're going to get a zero in that A transpose A.
Everybody see what nodes one and four are?
Yeah, say it again.
Not connected.
No edge.
Here there was an edge from node one to two.
Here is an edge from node one to three.
Those both produce the minus ones.
And on the diagonal came the two to balance it.
What does that two represent?
That two represents the number of edges that do go into node one.
See, that row is all about node one.
So they're two edges into it, and then an edge out, and an edge out, and the edge out and the no edge.
OK.
So, now I know it's going to be a symmetric matrix, so I could speed up and fill those in.
What's the next entry here?
What's the guy on this diagonal?
So that's row two against column two, so I have a one there, a one there, a one there, that makes a three.
Why a three?
Because there are, yeah, you got it.
There are three edges into node number two.
Three edges into node number two, and now I'm going to have some minus ones off the diagonal for those edges.
So what are these entries going to be here?
They're both minus ones.
Edge two is connected to all three other nodes.
So I'm going to see a minus one and a minus one there, and it's going to be symmetric.
And I'm nearly there.
Of course, I'm describing a pattern that you're just seeing unfold, but I'm doing it that way so that you'll feel hey, I can write down a transpose a, or check it quite quickly, without doing this complete matrix multiplication.
So what number goes there?
That's to do with node three, and I see node three connected to all three other nodes, and so what do you expect there?
Minus one there, and a minus one there, and what do you expect here?
Two.
And so now I have my matrix.
The a transpose a matrix.
And that's square and it's symmetric.
Now I ask you, is it positive definite?
Or is it only semi-definite?
Right, we know that A transpose A is always positive definite in the best case.
But only positive semi-definite if it's singular, if there's some vector in its null space, if a transpose a times some vector gives zero.
If some combination of those columns gives me the zero column.
Which is it?
Have I got a singular matrix or an invertible matrix here?
Singular.
Why singular?
Because a had some solutions to Au=0.
So if Au equaled zero, then I could multiply both sides by a transpose, that same u, A transpose times zero will still be zero, it might be a different size zero, but it'll be zero.
And what's the u, then?
It's the all ones vector.
What am I saying about the columns of A transpose A?
They're dependent.. they add up because it's the <1, 1, 1, 1> vector that's guilty.
Every row adds to zero.
Every row adds to zero.
Let me just say for a moment, introduce two notation for the diagonal matrix.
D, that's the diagonal matrix, two, three, three, two.
And then I'll put in a minus sign, and this is and I'll call it W. So you can pick out what D and W are, but let me do it for sure.
So D, the degree matrix, OK this is this is like fun because I'm not doing anything yet.
I'm just giving names here.
Two, three, three, two.
The degree of a node, the degree means how many edges go from it.
How many edges touch it.
And W is also a great matrix, it's called the adjacency matrix.
It's also beautiful.
Now it'll have plus ones because I wanted minus W, so it has, these nodes are not adjacent to themselves but it's got this one and this one and this one this one and that one, and that's a zero.
So there are five, the adjacency matrix tells me which nodes are connected to which other nodes.
And of course the connections are going both ways.
So I see five ones from five edges.
And I see five more ones below the diagonal, because the edges are connecting both ways.
Ones connected to three, and three is connected to one.
One is not connected to four, and four is not connected to one.
One is not connected to itself.
By an edge.
If we allowed, like, little self loops, then a one could appear on the diagram.
But we don't.
OK, so that's D and W. Here are the key matrices.
This is actually, I venture to say that any afternoon at MIT there's a seminar that involves these matrices.
One name for this is the graph, Laplacian.
From Laplace's equation and we'll see pretty soon where that name's coming from.
But it's there.
And should I think, I think I should, just about networks.
Like where, does the networks come from?
I think we've got networks all around us.
Right?
Electrical networks are the simplest, maybe in some ways the simplest to visualize.
So that's the example, that's the language I'll use.
Now, I get a network, I use the word network when there's a c_1, c_2, c_3, c_4, c_5.
Those extra numbers.
I've got the A, and now the network comes from the c part.
That diagonal matrix, and if I'm talking electricity, these could be resistors.
Status springs, they're resistors.
So it's the conductance in those five resistors, are c_1, c_2, c_3, c_4, and c_5.
So I'm ready for that.
Ready for the C matrix, because we got the a matrix.
And we've got A transpose A, but the the applications throw in a c matrix also.
What are other applications, I was saying, like this one is the one, I'll use the word current, for flow in the edges, or I'll use the word flow.
A network of oil, or natural gas, or water pipes would be just that, and then the electrical people study.
Professor Vergasian in Course 6 studies the electric grid.
The US electric grid, or the western, off in the western half of the US electric grid.
So that's got a whole lot of things.
Pumping stations.
You see it?
Actually, the world wide web, the internet, is a giant network that people would love to understand.
And the phone company would love to understand those networks of phone calls.
I mean, those are really, that's what, giant businesses are are dependent on understanding and maintaining networks.
OK, so I'm going to use resistors.
Of course, I'm staying linear.
And I'm staying steady state.
So by staying linear there aren't any transistors in this net.
By staying steady state, there aren't any capacitors or inductors.
Those guys would be linear elements, but they would be coming in a time-dependent problem.
A UTT problem.
And I'm just staying now with Ku=f, I'm trying to create K. The stiffness matrix, which maybe here we might call the conductance matrix.
OK, so ready for the picture now?
That these come into?
You know what the picture looks like, it's going to have the usual four, we'll start with these potentials u at the nodes, potentials at nodes, so those will be u_1, u_2, u_3, u_4.
Voltages, if I'm really speaking, those units would be volts, and now comes the matrix A.
And now I get, what do I get from A?
What do I get from A?
Key question.
If I multiply A times u, and you know that's coming, right?
If I multiply A times u, so I'll erase A transpose now, because we've got that.
So there's A, and now I'll make space to multiply by u, alright?
So now I want to look at Au.
So A multiplies a bunch of potentials, a bunch of voltages.
And let's just do this multiplication and see what it produces.
This is the great thing about matrices, they produce something.
OK, what's the first component of Au?
Of course, Au is going to be five by five.
It's going to be associated with edges.
Right, u's associated with nodes, a u with edges.
Just, the pattern is so nice.
Alright, what's the first component?
Just do that multiplication and what do you get?
u_2-u_1.
What do you get in the second component?
Do that multiplication and you get u_3-u_1.
The third one will be u_3-u_2.
The fourth one would be u_4-u_2, and the fifth one will be u_4-u_3.
Just like our first difference matrices.
But this one deals with, I mean, our first difference matrices were exactly like this when the graph was all in a line.
The big step now is that the graph is not in a line, not even necessarily in a plane.
Could be in, it's a bunch of points, and edges.
Actually, the position of those points, we don't have to know are they in a plane.
I think of them as nodes and edges.
OK, what's the natural name for Au?
I would call those potential differences, right?
Voltage differences.
So that's what we see here and those will be e. e_1, e_2, e_3, e_4, e_5 will be potential or voltage differences.
Voltage drops, you might say.
Potential differences, voltage drops.
Oh well, now.
When I say voltage drops, that's because, as we noted before, the current goes from a higher to a lower potential.
It goes in the direction of the drop.
And I think that what we need now is minus Au, for e. So I think we need a minus sign and it's quite common to have the minus sign.
We saw it already with least squared.
And let me say also, so this is e. I'll abbreviate those always five e's I just wrote down, five of them.
So you would remember there are five.
We're talking about the currents.
We're talking about, this is the e in e=IR.
The voltage drop.
That makes some current go.
Now, also, just as with least squares, so it was great that we saw it before, there could be a source term here.
So I'm completing the picture here, allowing the source term.
And we'll come back to what does that mean, physically.
But at that point could enter b, and b is really standing for batteries.
I work hard to make the language match the initials.
These letters.
OK, now what?
That step just involved A, nothing physical.
Now comes the step that involves A, so w will be Ce.
And these will be the currents on the edges.
And that's the law, then, with a matrix C, of course C is our old friend c_1 to c_5.
And tell me first, the name.
Whose law is this?
That the current is proportional to the voltage drop?
Ohm.
So this is Ohm's law.
Instead of Hooke's law, it's Ohm's law.
And I've written it with conductances, not resistances.
So resistances are 1/R, the usual R in e=IR, would be, I'm more looking at it as i current equals Ce, instead of e=IR.
So I'm flipping the the the resistance, or the impedance to give the conductance.
OK, and now finally can you tell me what the last step is going to be?
If life is good, well you might wonder whether life is good, reading the papers, but it's still good here.
OK, what matrix shows up there?
Everybody knows it.
A transpose.
So the final equation, the balance equation, will be, let me write it so I don't catch it up here.
Will be A transpose w equals whatever.
Will be the balance equation.
The current balance, it's the balance of currents, balance of charge, whatever you like to say.
At each node, it's the balance at the nodes.
Because when we're up on this line, we're in the node picture.
We have four equations here, right?
We're talking about at each node.
Here we're talking about on each edge.
There is so critical.
These two variables.
Which we're seeing physically as node variables and edge variables.
That pair of variables just shows up everywhere.
In displacements and stresses, it's fundamental in elasticity.
And oh, there are just so many in optimization, it's everywhere.
And a big part of this course is to see it everywhere.
OK, why don't I, just so you see the main picture.
We're going to have the A transpose C A matrix that I'm going to maybe call K again.
And now of course there could be current sources.
Just the way there could be forces that we had to balance.
There could be, not always but there could be, current sources from outside.
External current sources.
So these are external voltage sources.
These are external current sources.
So in a way, we now have combined our first two examples, our springs and masses only had forces external.
Our least squares problem had an external b.
Measurement.
This picture is the whole deal.
It's gotta b and f, and actually I could put in even a little more.
Sources like, well, we already kind of caught on to the fact that we'd better ground the node or A transpose C A as it stands, A transpose C A as it stands will be singular.
You know, it's the matrix, there's A transpose A and the C in the middle isn't going to help any.
That's singular.
If we wanted to be able to compute voltages, we've got to set one of them.
It's like setting one temperature, it's like deciding where is absolute zero.
Let's put absolute zero down here.
u_4=0.
Grounded the node.
OK, so we've fixed a potential.
So here's a boundary condition coming in u_4=0.
That's another source term, another thing coming, you could say sort of from outside the A transpose C A.
We could fix another voltage at, I mean, I'm thinking now about what's the picture.
What's the whole problem?
So the problem could have batteries, in the edges.
It could have current sources into the nodes.
It could fix u_1 at some voltage like ten.
Our problem could fix - we must fix one of them.
Otherwise our matrix isn't as singular.
But once we've set up the matrix, and when we fix u_4=0 by the way, what happens to our matrix?
Let me take u_4=0, so this is a key step here.
When I set u_4=0, I now know u_4.
It's not an unknown any more.
So I've removed u_4 from the problem.
And then it'll be also removed from A transpose A.
So this, is you could say, like a reduced A, or a grounded matrix A.
It's now five by three.
And A transpose A, what shape will the a transpose a matrix be?
It'll be three by three, right?
I now have five by three, three by five.
Multiplying five by three gives me three by three.
This column is gone, and that row is gone.
Because the row came from A transpose and the column came from A, and we've just thrown them away.
By grounding that node.
Now give me the key fact about that A transpose A matrix?
What what do you see there?
Now, you see a reduced, a grounded A transpose A.
What kind of a matrix have I got?
Positive def.
Good.
Positive definite.
It's now not singular any more, its determinant is some positive number.
And everything is positive, its eigenvalues are all positive, everything's good about that matrix.
OK, and I guess what I was starting to say here, if I wanted to fix, this would be a natural problem.
Fix the top voltage at one, say.
Fix u_1=1 and see how much current flows.
That would be a natural question.
What's the system resistance between the top node and the bottom, if I'm given, or the system conductance.
If I'm given a c_1, a c_2, a c_3, a c_4, and a c_5, I could say I could fix that voltage at one, I could fix this at zero.
Maybe one of the homework problems asks you for something like this.
And then you find all the currents.
And the voltages, you solve the problem.
And you know what the currents are.
You know the total current that leaves node one, enters node four when the voltages drop by one, between, right?
So current can flow down here, cross over here, down here whatever.
Somehow all these five numbers are going to play a part in that system resistance.
So that would be an interesting number to know.
Out of those five numbers, somehow five c's, there's a system resistance between that node and that node.
And we can find it by setting this to be one, this to be zero, having the reduced matrix - oh, well what will happen?
How many unknowns well I have?
Just do this mental experiment.
Suppose I introduce u_1 to be one, for example.
This is just one type of possible problem.
If I take u_1 to be one, what happens to my matrix A?
It loses its first column, too.
u_1 is not unknown any more.
u_1 will not be unknown.
And that value one is somehow going to move to the right-hand side, right?
People have asked me after class, well what happens if a boundary condition isn't zero?
Suppose we have this fixed springs and we pull this spring down to make its displacement 12.
Well, somehow that 12 is going to show up on the right side of the equation.
It's a source, it's an external term.
OK, so if we had u_1 equals whatever, this u_1 would disappear.
I would only have a two by two problem.
Because I would only have two, I now have only two unknown u's, right?
So that's where sources can come.
And can I just complete the picture of the source stuff?
We could fix, we could.
Look, here's what I'm going to say.
External stuff.
Sources can come into here.
They can come into here.
They can come into here, so of course everybody says why shouldn't they come in here?
And the answer is we could send them here.
So we could fix, we could fix some w's.
Of course, you understand we can't do everything.
I mean, there's a limit to how much we can put on the system.
We want to have some unknowns left.
Some matrix still, but anyway.
I like this picture now, it's more complete.
That you now see the node variables and node equations, the edge variables, e and w. The currents.
These guys are the big ones.
w and u are what I think of as the crucial unknowns.
e is sort of on the way.
f is the source.
But now we have the possibility of sources at all four positions.
OK, let's see.
If I wrote out, If I looked at A transpose C A, would you like to tell me, yeah.
Have we got?
No, we don't.
I was going to say, what's a typical row of A transpose C A, can I just say it in words?
It'll be too quick to really catch.
So without the C, this is what we had.
So what do you think that two becomes if there's an A transpose C A, if there's a C in the middle.
Have you got the pattern yet?
That two was there because of two edges.
Edges one and two, it happened to be.
So instead of the two, I'm going to see c_1+c_2.
Right.
When those were ones, I got the two.
So this will be c_1+c_2, this'll be a minus c_1, and that'll be a minus c_1, when we do it out.
And you could do it out for yourself.
Just tell me what would show up there.
In A transpose C A, so I'm talking now about A transpose C A.
So instead of one plus one plus one, what do I have?
What am I going to have, and you really want to multiply it out, because it's so nice to see it happen.
What do I have?
I'm looking at node two, I'm seeing three edges out of it.
And instead of one, one, one, I'll have c_1+c_3+c_4.
c_1+c_3+c_4 will be sitting here.
And minus c_1 will be here, and minus c_3 will be here, and minus c_4 will be there.
The pattern's just nice.
So if you can read this part of the section, I'll have more to say Friday about the a transpose w, the balance.
That critical point we didn't do yet.
But the main thing, you've got it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseware continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This is my second lecture on the example, the model of graphs and networks.
Really perfect, beautiful model for the whole framework.
And important in itself.
So, I guess a major thing that I still have to do is discuss A transpose.
See how A transpose just naturally appears in the balance slope.
Kirchhoff's current law, KCL, is just like a model for balance equations.
By balance I mean, n equals out, essentially.
Flow in equals flow out because we're talking about steady state.
So in each node, at node three, for example, I have three edges.
So the law, Kirchhoff's current law at that point is going to tell me that the flow w_2 plus w_3 minus w_5 would be zero if there's no current source, or if I'm feeding current into there, some current f_3, then it would match f_3.
So maybe having just said those words, let me just say w_2, I'll say it again, w_2 plus w_3 minus w_5, and I'm hoping that I'll find that in the third column here.
And I do.
Because I'm thinking, and we'll write it down, I'm thinking to take a transpose, so it'll be the third row, and here I see that w_2, w_3 and minus w_5 that will show up in the third equation there, the n equals out at node three.
Well I'll write that down, because it's -- So, two or three jobs for today.
And then Monday I plan to spend a part of the lecture and all of the review session in review, it's a great chance to go back to the things we've done and collect them, assemble them, organize them and see them again.
So that's my goal for both sessions on Monday.
And questions, then, about every aspect.
So, before I get to A transpose, this part I had written down last time, but there's a little more to say.
And what I was saying and want to continue with is the source terms.
I put in this b but I didn't draw anything on the network.
So let me draw what these b's represent.
So this second, lower, row is about edge equations, edge variables.
So those batteries b are on the edges.
And there will be, b has length five.
There are five edges.
This is a vector of length five, this matrix, A, you remember was five by four.
It produced from four inputs, from four potentials at the four nodes, A produces Au, five potential differences.
So those are the differences in potentials.
But then also there can be source terms from batteries in the edges, and the minus sign is there because I'm talking about voltage drops.
So when I say differences I'm really speaking about voltage drops.
Because that's the way current flows.
OK, this is the moment that I hate.
Putting the batteries in I think you draw a battery with a long and a short?
Is that right?
And then you put a plus and a minus?
Well, can I just say, life is too short to get those, to get this sign right.
You may say when my car battery stalls how do I, because it's important at that moment, right?
When you start it up you're supposed to put the right battery, the right lead on the positive and the right on the negative, or your blow yourself up.
OK.
So what's my solution?
Because I literally refuse to deal with these signs.
So my solution is call AAA.
OK, they're paid to blow themselves up.
They can do it.
But so this is serious now, I don't want to be asked about these signs.
And I forgive you for messing up the signs.
So possibly that's plus, possibly minus, I don't know.
But there's a battery in there of length b_1, of voltage, of a nine-volt battery, b_1 would be nine.
And depending how it was placed in there, the b here would be a plus nine or a minus nine in the first component.
And then if I had another battery here, a b that's on edge four, there would be a battery b_4, and that would show up there in Ohm's law.
Because Ohm's law will look at the difference in these potentials, but then it'll also account for the battery, right?
The voltage that comes from the battery, and somehow combining the Au, which is the difference in these guys, with the b_4, I'll know what is the voltage drop across the resistor.
And that's what Ohm's law applies to.
Ohm's law says the voltage drop across the resistor times the conductance, so this is Ohm's law, that on that edge the voltage drop across the resistor times the conductance gives the current.
So that's the physical law.
Is that OK?
For batteries.
Now a comment on current sources.
So what's with current sources?
How would I draw those?
Well, very often maybe I might draw a current going into node one, so that would be in f_1 that goes into node one and maybe comes out at ground.
So that would be a typical f. That if I imposed a current source, if I sent a current source through there it would go down and come out at ground, and our question is what are the currents in the five edges?
What are the potentials at the four nodes?
Well, I'm making this one ground.
So I'm grounding this one to be u_4=0.
And you remember of course why I had to do that, because this matrix a transpose a, as it stands is, what's the matter with a transpose a as it stands?
It's singular, right?
And as I, looking ahead, just a small comment, that this will have exactly the same thing in so many other applications in big, finite element codes, you produce a stiffness matrix that's initially singular.
And then you impose the boundary conditions.
That's the efficient way to do it, is create the matrix first, that's the big job, then impose the boundary conditions, that's the small but occasionally tricky part.
Now I indicated what happened here.
When u_4 was zero, that means that a times u, the u_4 there, the zero is multiplying this, is not unknown any more.
It's known.
And so that column is not really any more leaded in A.
Because u_4 is gone from my list of unknown potentials.
Now, at the same time, when I went over to A transpose A, I'm making this comment because there were several good questions about it.
I claim that also, that row coming, which of course comes from the fourth row of A transpose.
The fourth column of A is gone, so the fourth row A transpose should be gone.
And we might just think why's that, what's going on?
Of course, it produces exactly what we want.
It leaves us with a three by three matrix that's not singular any more.
I've removed that <1, 1, 1, 1> from the null space by fixing a potential.
Grounding a node, and the problem becomes just what we want.
And I'll write down the equations when I get the Kirchhoff current law to complete the loop.
Now, what's going on?
Let me remove this current source, just so we focus on what I'm speaking about now.
On this type of boundary condition, this is like fixing a support, right?
It's like fixing a support in our spring mass problem.
Can I squeeze in a little spring mass problem, so I have a spring, and then I'm fixing this displacement, u_4 to be zero.
Now, I want to try to think through what happens in a balance equation, A transpose w, let me make it A transpose w equal f, because that's the case with right hand side allowed.
And now what happens when I fix this, I'm asking you to think back about force balance, which we certainly saw was an A transpose w equal f business.
And then parallel will be the current balance, at that node.
OK, so what was the deal on force balance?
A transpose for this fixed in here.
What was the thing with a transpose did that fixed in?
We did not write the equation A transpose w equal f at this point.
We did not write the force balance equation at that point.
When I fixed u_4, in this case it was the displacement so I'm fixing it at zero displacement, I didn't have a force balance in the A transpose part at this node.
Now, you could say why?
Because of course forces have to balance.
But what's going on?
What's really happening here is I don't have to write out, I the displacement here is known.
It's not unknown.
And let me say it in a sentence.
Force balance does hold because the support supplies whatever force it takes to balance the internal forces.
So, in other words, let me say that again, the force balance will hold and it will tell me, after I've solved the rest of the problem, it'll tell me what the support has to supply, how much force the support is actually supplying.
It's a reaction force, it would be called.
So a reaction force is whatever the support has to do to fix that displacement.
And so I solved the fixed problems for all the other displacements and all the spring forces.
And then I could come back at the end and say OK, what was the force in that spring and therefore how much is that support, what's the force being supplied by that support.
See what I'm saying?
That the force balance at this node comes afterwards.
That equation is like knocking that one out of this problem.
It would be the same here I fix the potential at zero.
I fix ground at zero.
Current flows.
Maybe some, maybe I might fix that potential at one.
I fix that potential at zero.
Current flows.
OK. ta-da, I find out, I compute what they are, and this row and column will be gone.
Find out what they are.
At this node, what's happened?
What's happening to the balance of currents?
Does this current have to balance that current at the ground?
No.
The ground, whatever current comes here plus whatever current comes here, goes into ground.
Do you see the point?
Kirchhoff's current law, the balance of currents that in equals out, is true but it's not an equation I have to solve in finding the currents, it's something I can discover at the end.
I can say OK, how much current flowed from ground?
And similarly up here.
If I fix u_1 to be zero, then you, no, I don't want to fix it to be zero, two that would be a way to make very little happen.
Let me fix u_1 to be one, so this is a standard important problem.
It's like what's the resistance in the net, and what's the net system resistance?
If I fix this at one fix this at zero, some current will flow, it'll come out here.
And that'll be the current going into here.
Somehow there'll be a balance there, and a balance there.
But it's found later.
Just the way the force in the support is found later.
OK, that takes a little thinking.
I just wanted to, because I had blithely knocked this row out.
And you could do it on the basis, well if you knock out this column of a then you're knocking out the row of A transpose, and therefore A transpose A will be three by three.
And it'll go down to two by two if I fix that potential.
And if I don't fix it at zero, if I fix it at one then something will move to the right hand side.
OK. that's that point I hope at least discussed.
So now we have the whole pattern here.
except that I really have still to justify the fact that it truly is A transpose.
The transpose of that matrix that comes into Kirchhoff's current law.
I guess in the first minute of the lecture I looked at that particular node.
Let's look at all the nodes.
Let me look at A transpose w equals here.
OK.
So I'll copy a transpose because I truly believe that Kirchhoff would want me to do it.
OK, so that becomes a row, that becomes the next row, of course I see three guys going into node one, then the one that I looked at before.
That's the three edges, node three, and then the last one.
Let's keep the last one in for now.
And because if I didn't fix that I'd have that last one.
There we go, so that multiplies, what does that multiply now in Kirchhoff's current law?
Multiplies w, so currents.
So here the currents, one, two, three, four and five.
All I'm doing now is just, like, convincing you that it really is a transpose, that if I look at, let me pick that node now, if I look at that node I see edge 1 coming in, I see edge 3 going out, I see edge 4 going out and when I look at that second node, I look here at the second row, I see edge one coming in, edge three and edge four going out, multiplying those currents, and that will give, that current balance there gives that zero.
Right?
That current balance at node two gives that zero in the right hand side.
And then, of course, other zeroes are here too.
This is minus w_1, minus w_2, so I have a zero.
This is with no current sources.
OK. And this one, of course.
I guess I'm hoping that you say yes, it's the A transpose really was the right matrix to express in=out, Kirchhoff's current law.
OK. Of course, by now you're probably getting blase.
You expect it to be a transpose, you don't need any convincing.
But it's kind of good to see each time because it's like, well, it's not a miracle.
But it's like a miracle.
It's as good as a miracle.
Because to get a transpose is just what makes everything right.
Now, here's a question.
This is a question worth thinking about.
What are the solutions?
If I only look at this piece of the framework, if I just look at Kirchhoff's current law, it's telling me, and I have zero current sources.
Well let's take that, let me take just this piece of the whole framework and ask you how many vectors, w, how many solutions?
Are there any solutions?
Well, of course, there's always the zero solution.
But I always ask you, what are the solutions when zero is on the right hand side?
So, what are the w's that solve Kirchhoff's current law.
Now that's a new question.
We haven't asked that before.
What we asked before was the question Au, Au=0, remember it was then A.
This five by four matrix, u had four components.
Just remind me, and I'm putting that column back in, so this is still here.
In the un-reduced, un-grounded case.
Well, just so we get started, remind me what the solutions u were for Au equals zero.
.
Or any multiple of it.
A whole line of vectors.
any constant c. OK, so this had, there were four columns.
But only three were independent.
OK. Now, now I've made them into rows.
And I made the rows into columns.
So now I have five columns.
What I'm leading to is, I want to count in advance how many w's I should be looking for, and then look for them.
OK, so the first question is how many different solutions w. First of all, are there some solutions?
Is there is a solution w, other than zero of course, to this system?
Well, as we said last time, we've only got four equations here.
We've got five unknowns.
Of course there's at least one solution.
Five for equations, five unknowns, I can do elimination.
Whatever systematic procedure you want me to do.
In the end, I'm going to find a solution.
Now, I might find more solutions.
So that's the question.
So what's the, now this is the key fact of linear algebra.
Which is, it just tells us the numbers of everything.
So this told us that there were three independent columns.
Of A.
Now, for that key theorem, which tells me that how many independent rows of a are there?
Three.
That number is equal.
I just want to say that's a pretty remarkable fact.
If I have a 50 by 80 matrix and that 50 by 80 matrix has 17 independent columns, then this great fact tells me that there are 17 independent rows.
And if those 50 times 80, 4,000 numbers are random, boy, you can't look at it and see what are they.
Independent rows.
But if you know there are 17 independent columns then there are 17 independent rows.
So, what does that mean here?
That means that out of the five columns of A transpose, which were rows of A, three are independent.
So just tell me, how many solutions I'm looking for.
Before I look for them.
So the number of the number of independent w's, independent solutions will be what?
What's your guess?
Two!
Two is the right guess.
Two, because I have all together five unknowns I subtract three equations that are really there, I have three real equations there even though it looks like four.
And I get two.
So that general picture is n - oh, I'm sorry it's actually m because I'm doing the transpose here.
So it's m w's minus r, the rank.
So that's m is five, the rank is three and this counts the number of independent solutions.
So it's a nice, it couldn't be better.
It's a fundamental count of how many solutions are there.
You really taking the number of unknowns, five, and you're subtracting the number of equations that are really there, three and that leaves you with two solutions which we will have to find in a minute.
Can you say why there's really only three equations there?
Why do I say that that fourth equation is not contributing anything new?
I believe that that fourth equation is a consequence of the first three.
And, therefore, if it's there or if it's not there, it's not telling me anything new about in=out.
In other words, if I have a closed system, closed because it's zero on the right side, if I have a closed system and I have in=out at those three nodes, then I'll automatically have in=out at the fourth node, because the total in is zero.
And the total out is zero.
So if I'm right at three I'll be right at the fourth one.
And now just tell me with the numbers, how would I get this equation w_4+w_5=0 from the first three?
Well, it's probably the same way that I got that column, that that column was connected to those columns.
What do I do?
Add them.
If you add that equation to that equation to that equation, add those three equations, what happens?
the w_1's cancel, the w_2's cancel, the w_3's cancel, this says there's a minus w_4 and a minus w_5 that adds to zero plus zero plus zero, minus w_4, minus w_5 equalling zero is the same as plus w_4 plus w_5 equals zero.
The four equations add to zero equals zero.
That's the central thing is.
The four equations add to zero equals zero just the way the four columns up here added to the zero column.
OK.
So we got the count.
Now, this is the interesting part, always.
What are the solutions?
What are the actual w's?
OK. You could say, wait a minute, you're asking me to solve four equations and five unknowns, and just say what the answer is.
Well, normally that's not reasonable.
But here we can get help from the graph.
Let me give you an idea here.
So what are we looking for on the graph?
That solves it.
We're looking for a bunch of currents that balance themselves.
Right?
We've got zero on the right hand side.
So we're not getting any help from outside.
How could you send current in this loop in a way that would satisfy Kirchhoff.
He'd be happy.
The balance load would be true.
OK. b, I'm just looking at currents.
Current w_1, w_2, w_3.
Is there any combination of w_1, w_2, w_3 that would balance itself, that would make Kirchhoff OK?
Well, here's the idea.
Send that current around a loop.
Loop currents are solutions to Kirchhoff's balance law.
If I send an amp on that edge, on that edge and backward on that edge, right?
It's around a loop at every node, it's totally OK.
So I believe that a particular solution will be for these things to be, let's see what did I say?
w_1, I'll send one around.
w_3 will be a one.
w_2 was backwards, it wasn't traveling on w_4.
I think that's a solution.
That loop current gives me a solution so let me call this solution, that's the first solution.
And if we do these multiplications, of course it's going to come out right.
OK, so that's solution number one.
A w that works.
OK, with that hint, tell me a second w that works.
In fact, since there are only two, you'll be giving me the rest of the answer.
So that was a loop current that went around that loop.
Tell me another one.
Well, we're a big class but everybody's seen it.
How about around that loop?
So that would be another thing and it's pretty clearly not the same as this one.
So I'm really truly finding two independent solutions.
And what is that second solution?
Let me maybe just put it in here.
And they're both giving me .
And now, what is this number two solution the loop number two?
One and two are not involved now.
Number three, see I'm usually sending it counterclockwise, that's the sort of convention.
But you know, of course you just have to follow some convention in connection with the arrows.
So that would go backwards on three, I think.
Forwards on four, and backwards on five.
So that would be solution number two.
OK. Now, tell me what all the solutions are.
I found two particular solutions, two particular loop currents, particularly easy.
What would be all the solutions to Kirchhoff's current law, A transpose w equals zero?
Every w now since I've found the right number, every w will be a combination of those two.
Ah, well wait a minute.
Have I got them all?
I should have thought, what about current around the big loop?
That would certainly satisfy Kirchhoff.
Plus one, one, so why is this not number three?
Around the big loop I have a one and then a one on the fourth position.
Backwards on five, backwards on two.
And so there is number, so I'll put number three with a question mark.
What's up with that guy?
It, you know if unless mathematics has got to close up shop, this better be a combination of those.
And of course, it is.
If I send something around the top loop and something around the second loop and add them together, they'll cancel on the middle edge there and produce number three.
So this is probably just the sum.
If I add that one to that one, I think I get number three.
So number three is true.
It's a solution but it's not a new one.
OK. That was simple, right, once we saw that loops gave the answer.
But, it's, actually it's quite important and appears everywhere.
In fact the theory of electrical networks, current laws and so on.
I mean that used to be a, like, basic course in electrical engineering.
There was a text by Professor Ernst Guillemin, I remember.
It's sort of not so central to the world any more.
And now, but the structure is just right somehow.
And what I wanted to say is you could take, in those days you maybe took loop currents as the unknowns.
You could think of currents in the loops as your principal unknowns.
We don't do that now.
But, oh, there's, yeah it comes up again.
Knowing all the solutions to A transpose w equals zero, well, you'll see, what's ahead?
It will be the continuous analog of this, where I have flows, not just around a graph, but in a region.
And Laplace's equation is going to come up, and the equations of divergence and gradient, all that great stuff is coming in Chapter 3.
And this is somehow the discrete case.
If by these these loop currents that has something to do with the curl.
And these differences that a takes has something to do with gradients.
And this, Kirchhoff's current law has something to do with divergence.
Can I just say ahead of time, what we're doing is really good to see and get it because then you have a way to understand vector calculus.
This is discrete vector calculus we're doing.
OK. Now, it's just right.
Forgive me for my sermon here.
Alright.
Now, may I bring the pieces together finally?
May I bring the three steps together into, well first you would say bring them into one equation.
Put the three steps, combine the three steps into one.
OK.
So, what happens if I do that?
I take that last step A transpose f, is A transpose w. So now I'm going to get one equation.
Which is the equation that's going to have the stiffness matrix in it, A transpose C A.
It's the conductance matrix.
And it's the equation that a big finite element code, a circuit simulation code.
It's the matrix they have to find and work with.
And those codes are enormous, and spice by the way is the sort of grandfather of circuit simulation code.
Somebody at Berkeley had the sense to see hey, we've got giant circuits now.
Modern circuits have thousands of elements.
And you can't do it by eye the way we can do this one by eye.
You've got to organize it and write a code, and spice is the start.
So one way to do it is to end up with one equation.
So that was f equals A transpose w. I'm now going, I'm just assembling the whole loop.
But w is Ce.
So that's A transpose Ce, but e is b minus Au.
Nothing new there.
Nothing new maybe except that it involves both the f and the b, where our earlier examples involved either an f, in masses and springs, or a b in the squares.
Now they're both here.
So now, that's my equation, f equals this.
And now let me just move that to the, let me just recollect it, that's A transpose C A, the big thing that I wanted to see.
I'll put it on the left side.
And what will I have on the right side?
I'll have A transpose C b.
And I'll have f is now coming over to the other side with a minus x.
That's the big equation.
You could say that's the fundamental equation of equilibrium.
And you see how it's right?
It involves the A transpose C A, which we expect.
Over here was the A transpose C b.
Now, which problem, before networks, produced an A transpose b or an A transpose C b?
That was least squared.
And now, so that's the least squares, the b term.
The b is there with a couple of matrices because b entered the problem just one step around.
It had two more steps to go.
It had a C step and then an A transpose step.
Whereas f up here is at the very end and now it appears with a minus sign.
That's different from springs and masses simply because, the sign conventions, you could say.
OK, there is the equation.
OK, fine.
So that's what you have to solve.
And actually I think of that as the fundamental problem of numerical analysis.
How to solve that equation.
More effort, more thinking goes into that than probably any other single problem.
In some form.
OK, and here's some part of that thinking.
Part of that thinking and another important possibility is to keep, this was like the one equation, the one field problem, this corresponds to the displacement method.
Can I use words that I'm not going to use seriously for another few weeks, this would correspond to the displacement method in finite elements.
In f em, f em for finite element method.
That's the displacement method, it's the method that that most people use.
It's the standard method.
OK.
But it's the only possibility, and let me show you a second one that involves two equations.
Because that's also very important.
with many other applications.
That we will see but haven't seen yet.
So my two equations, I really should say two systems, because one equation, that's a vector of course.
So I have a system of two vector equations, it would go into a block matrix and you'll see it.
OK, so what what two unknowns am I going to keep?
u, I'm keeping.
Displacement, I'm keeping.
But I'm also going to keep what I think of as the other important unknown, w. So the other important unknown is w. So now I have two equations and one of them is just that, is just the current, is just the current law, the balance law A transpose w equal f. The only guy I'm eliminating is e. Initially you could say I've a three field system.
u, e and w. Now, e and w are so easily connected that I'm going to eliminate e. e is c inverse w. I did it actually here.
This w is Cb-Au, that's we know if I multiply by the C inverse, then I have C inverse w is, and I bring the Au over to the far left and I have only the b left.
Everybody saw that, I did a C inverse there to get e off by itself and then I substituted for e, I put in the u part so e's now gone, and the equation is C inverse w plus Au equals b.
That's my two field system.
Now, there's a matrix here.
This is really nice.
I just want to write that.
I've got to what I want but now I want to look at it.
So I think of a two by two block matrix multiplying wu and giving me bf.
And I want to ask you about that matrix.
So this is the matrix for when I've only eliminated e and I've still got w as well as u. OK, you can read off what's in that matrix.
What goes up here?
C inverse, of course.
Positive diagonal, usually.
Easy.
What goes here is a rectangular, that guy is rectangular.
A transpose is multiplying w, so it goes down here and this equation has no u in it.
That matrix is worth noticing.
And let's spend the rest of this, the remaining minutes just to think about that matrix.
I just want to say what if I keep w and u, this is an important possibility.
And it's important in finite elements which as you know is just a terrific way to solve a whole lot of continuum problems.
And what's it called when I have w and u together, both unknowns, not eliminating w now, it's called the mixed method.
So this corresponds to the mixed method in finite elements.
It corresponds to the possibility of keeping w and u.
Well, and you might say, wait, isn't there a third possibility?
And what would that be?
Keep only w. Here we kept only u, here we've got them both, this is kind of the mother of all equilibrium equations.
And another possibly would be to keep only the w's, to make the currents the primary unknowns.
And that in the finite element structural context, that would be saying make the stresses.
So of course it'd be called the stress method.
It'd be called the stress method, and Professor Pian in Course 16, now retired, was one of the major developers of the stress method.
The difficulty with the stress method, the reason it didn't win big time, is that the w's, if you make them the unknowns you've got a constraint on them.
Kirchhoff's, not all w's are allowed.
Somehow over here all u's are allowed, and that made it much easier to set up the problem.
So the displacement method is the 95 percent winner.
But there are problems where maybe C inverse is complicated, or C is too complicated and you're better to.
We can see that.
That's later in the book, but we want to see now about that matrix.
Well, if I wrote that matrix down, and let me write just so you, I want to ask you about that block matrix.
What's its size.
Now just focus entirely on that block matrix, because that's what I care about.
What's the size of that matrix?
Let's see, what's the size of C?
m by m. What's the size of A?
n by n. So what do I have here?
I've got n rows and m plus n columns, and there's n more rows.
It's of size m+n.
It's got the n u's and the n w's.
Of course, m+n.
And the natural size, right.
So it's got more unknowns but we'll see, oh in optimization you bring in Lagrange multipliers, that's just exactly parallel to what we're doing here.
You have more, you have extra bunch of unknowns.
That's what we have.
Now what else about that matrix?
I was going to write down a very, very tiny model for that matrix.
I'll just make it two by two.
Here's a model for that matrix where C is just a one and A is just a one.
I mean, it's kind of laughable, right?
That model, this is the real thing.
But it gives you an example to check against.
OK, what's a property of that matrix?
It's, again?
Symmetric.
Good.
That's a symmetric matrix.
Because what happens if I transpose that block matrix?
That A block will flip over here as A transpose, the A transpose block will flip up there as A, what happens to the C inverse block?
C is a symmetric guide.
In fact, it was just diagonal in our imagination.
The key point it's symmetric, its inverse is symmetric, its transpose is the same.
So that's symmetric matrix.
That's a good thing.
Right, now we've got a matrix that's symmetric, square symmetric.
OK, what's my other question about that matrix?
Is it or is it not positive definite, right?
We've got to answer that question.
Have we got a positive definite matrix?
Would all the pivots be positive.
Would the eigenvalues be positive?
What's your guess?
No.
That matrix is not positive.
No way that a matrix with a zero there, a zero block, or that matrix with a zero number could be positive definite.
No, no way.
The energy in this guy, this u transpose Au, you remember, would be u_1 squared and u_1*u_2, twice.
And no u_2 squareds.
And that thing is definitely indefinite.
Right?
In the u_1 direction it looks good, that's things positive there.
But if I took u_1 and u_2 to have opposite signs, and made u_2 big enough then of course this just brings it down.
So the graph of that would be a saddle point.
The graph of that would be a saddle.
OK, and now here I have the same thing on an m+n size.
So what do I have?
Actually, you could see.
The last exercise is mentally do elimination on that matrix.
Mentally do elimination on that matrix.
So start with the first m rows.
We'll work with those first.
What will elimination do, what will the pivots be like, what will happen at the beginning of elimination?
When I start with this matrix?
Well it meets C inverse right away, that diagonal matrix, and it's extremely happy.
Those will be the pivots, right?
They're sitting on the diagonals, zero off the diagonals.
They'll be positive pivots, I'll have m positive pivots here.
And then I get down to where a comes in the picture.
So on the last board here, let me just copy this matrix, [C inverse, A; A transpose, 0], an elimination is going to, it'll be very happy with that.
But it's going to put, so it's happy with that row.
Block row.
It's going to do an elimination to get a bunch of zeroes there and what did it do?
This was elimination, this was subtracting, yeah what did it subtract here?
It multiplied these pivot rows by something and subtracted from these lower rows and got the zero block.
And what did it multiply by?
What do I multiply that block row, and this is a perfect, perfect exercise to see how blocks are just like numbers.
You can deal with them.
What do I multiply that block row by and subtract from the row below to get a zero.
You said C A transpose, but I don't think that's it.
A transpose C. You've got to multiply by A transpose C. First of all, C A transpose wouldn't be a possibility.
Wrong shapes.
a transpose those C is the four by five, five by five guy.
So you multiply A transpose C, that cancels that, leaves the a transpose, when you subtract it gives you a zero, and what does it give you there?
What shows up there?
A transpose C multiplies at A, subtracts so it's actually what shows up there is minus A transpose C A.
Let me write it in there.
Minus A transpose C A.
So that matrix is exactly what comes from this one.
It's exactly what we do when we eliminate w. That's what elimination is.
I just eliminated w by getting a zero there.
And I got only an equation, but notice the minus.
So, final question, what are the signs of the last n pivots?
The first m were all positive, and they were sitting on the diagonal already.
The last n are not so easy to see, but we can see what sign they have.
And what sign do the last n pivots have?
Minus.
Because they come from a negative definite.
Minus A transpose C A is shown up there.
So that's the saddle point.
Saddle points are when you have two field problems, you're talking about saddle points, and the mixed method in finite elements is exactly that.
And the tricky part is then with the mixed method, you're sort of not so perfectly guaranteed that the matrix is invertible.
Because we have plot plus stuff and minus stuff.
OK, thank you that's great.
And I'll see you Monday all about the exam and review, it's a great chance to think back.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So this is Lecture 14, it's also the lecture before the exam on Tuesday evening.
I thought I would just go ahead and tell you what questions there are, so you could see.
I haven't filled in all the numbers, but this will tell you, it's a way of reviewing of course, to sort of see the things that we've done.
Which is quite a bit, really.
And also of course topics that will not be in the exam.
You can see that they're not in the exam, for example.
Topics from 1.7 on condition number, or 2.3 on Gram-Schmidt that I can speak about a little bit.
So those are the, only thing I didn't fill in is the n time there.
So I don't usually ask four questions, three is more typical.
So it's a little bit longer but it's not a difficult exam.
And I never want time to be the essential ingredient.
So 7:30 to 9 is the nominal time, but it would not be a surprise if you're still going on after 9 o'clock a little bit.
And I'll try not to, like, tear a paper away from you.
So, just figure that if you move along at a reasonable speed, and so you can bring papers, the book, anything with you.
And I'm open for questions now about the exam.
You know where 54-100 is, it's a classroom unfortunately.
Not a, sometimes we'll have the top of Walker where you have a whole table to work on, that's this 54-100 in the tallest building of MIT out in the middle of the middle space out there, is a large classroom.
So if you kind of spread out your papers we'll be OK.
It's a pretty big room.
Questions.
And, of course, don't forget this afternoon in 1-190 rather than here.
So, review is in 1-190, 4 to 5 today.
But if you're not free at that time don't feel that you've missed anything essential.
I thought this would be the right way to tell you what the exam is and guide your preparation for it.
Questions.
Yes, thanks
[INAUDIBLE]
So these are four different questions.
So this would be a question about getting to this matrix and what it's about, A transpose
[INAUDIBLE]
OK, I don't use beam bending for that.
I'm thinking of elastic bar.
Yeah, stretching equation, yeah.
So the stretching equation, so the first one we ever saw, the stretching equation, was just u'', or -u''=1.
And now that allows a C to sneak in and that allows a matrix C to sneak in there, but I think you'll see what they should look like, but yeah
[INAUDIBLE]
Yeah, yeah.
So this is Section 2.2, directly out of the book.
M will be the mass matrix, K will be the stiffness matrix, yep
[INAUDIBLE]
OK.
So, good point to say.
In the very first section, 1.1, we gave the name K to a very special matrix, a specific one.
But then later, now, I'm using the same letter K for matrices of that type.
That was the most special, simplest completely understood case, but now I'll use K for stiffness matrices and when we're doing finite elements in a few weeks again it'll be K. Yeah, same name, right.
So here you'll want to create K and M and know how to deal with, this was our only time dependent thing.
So I guess what you're seeing here is not only what time dependent equation will be there but also that I'm not going in detail into those trapezoidal difference methods.
Important as they are, we can't do everything on the quiz so I'm really focusing on things that were central to our course.
Good.
Other questions.
I'm very open for more questions this afternoon.
Yep
[INAUDIBLE]
Others.
OK.
So, let me, I don't want to go on and do new material, because we're focused on these things.
And this course, the name of this course is computational science and engineering.
And by the way I just had an email last week from the dean of engineering, or a bunch of us did, to say that the school of engineering is establishing a center for computational engineering, CCE.
Several faculty members there and, like myself in the School of Science and the Sloan School are involved with computation, and this new center is going to organize that.
So, it's a good development.
And it's headed by people in Course 2 and Course 16.
So.
If we're talking about computations, and I do have to say something about how you would actually do the computations.
And what are the issues about accuracy.
Speed and accuracy is what you're aiming for in the computations.
Of course, the first step is to know what problem is that you want to compute.
What do you want to solve, what's the equation?
That's what we've been doing all along.
Now, I just take a little time out to say, suppose I have the equation.
When I write K, I'm thinking of a symmetric positive, definite or at least semi definite matrix.
When I write A I'm thinking of any general, usually tall, thin, matrix.
Rectangular.
So that I would need the squares for this guy, where straightforward elimination would work for that one.
And so my first question is let's take this, so these are two topics for today.
This one would come out of 1.7, that discussion with condition number.
This one would come out of 2.3, the least squares section.
OK, so if I give you, and I'm thinking that the computational questions emerge when the systems are large.
So I'm thinking thousands of unknowns here.
Thousands of equations, at the least.
OK.
So, and the question is, I do Gaussian elimination here, ordinary elimination.
Backslash.
And how accurate is the answer?
And how do you understand?
I mean, the accuracy of the answer is going to kind of depend on two things.
And it's good to separate them.
One is the method you use, like elimination.
Whatever adjustments you might make.
Pivoting.
Exchanging rows to get larger pivots.
All that is in the algorithm, in the code.
And then the second, very important aspect, is the matrix K itself.
Is this a tough problem to solve whatever method you're using, or is it a simple problem?
Is the problem ill condition, meeting K would be like nearly singular, and then we would know we had a tougher problem to solve whatever method, or is K quite well conditioned, I mean the best condition would be when all the columns are unit vectors and all orthogonal to each other.
Yeah, I mean that would be the best conditioning of all.
That condition number would be one if this K, which, not too likely is a matrix that I would call Q. Q, which is going to show up over here, in the second problem is, Q stands for a matrix which has orthonormal columns.
So, you remember what orthonormal means.
Ortho is telling us perpendicular, that's the key point.
Normal is telling us that they're unit vectors, lengths one.
So that's the Q, and then you might ask what's the R. And the R is upper triangular.
OK. OK.
So what I said about this problem, that there's the method you use, and also the sensitivity, the difficulty of the problem in the first place, applies just the same here.
There's the method you use, do you use A transpose A to find u hat, this is, now we're looking for u hat, of course, the best solution.
Do I use A transpose A?
Well, you would say of course, what else.
That equation, that least squares equation has A transpose Au hat equal A transpose b, what's the choice?
But, if you're interested in high accuracy, and stability, numerical stability, maybe you don't go to A transpose A.
Going to A transpose A kind of squares the condition number.
You get to an A transpose A, that'll be our ok, but its condition number will somehow be squared and if the problem is nice, you're OK with that.
But if the problem is delicate?
Now, what does delicate mean for Au=b?
I'm kind of giving you a overview of the two problems before I start on this one.
And then that one.
So with this one the problem was, is the matrix nearly singular.
What does that mean?
Does MATLAB tell you, what does MATLAB tell you about the matrix?
And that is measured by the condition number.
What's the issue here is, when would this be a numerically difficult, sensitive problem?
Well, the columns of A are not orthonormal.
If they are, then you're golden.
If the columns of A are orthonormal, then you're all set.
So what's the opposite?
Well, the extreme opposite would be when the columns of A are dependent.
If the columns of A are linearly dependent and some column is a combination of other columns, you're in trouble right away.
So that's like big trouble, that's like K being singular.
Those are the cases.
K singular here, dependent columns here.
Not full rank.
So, again we're supposing we're not facing disaster.
Just near disaster.
So we want to know, is K near the singular and how to measure that, and we want to know what to do when the columns of A are independent but maybe not very.
And that would show up in a large condition number for A transpose A.
And this happens all the time; if you don't set up your problem well, your experimental problem, you can easily get matrices A, whose columns are not very independent.
Measured by A transpose A being close to singular.
Right, everybody here's got that idea.
If the columns of A are independent, A transpose A is non-singular.
Fact, positive, definite.
Then now we're talking about when we have that property, but the columns of A are not very independent and the matrix A transpose A is not very invertible.
OK, so that's what the things are.
And then just to, because, say on this one, what's the good thing to do?
the good thing to do is to call the qr code which gets its name because it takes the matrix A and it factors it.
Of course, we all know that lu is the code here.
It factors K. And qr is the code that factors A into a very good guy.
An optimal queue.
It couldn't be b. and an R, that's upper triangular, and therefore in the simplest form you see exactly what you're dealing with.
Let me continue this least squares idea.
Because Q and R are probably not so familiar.
Maybe the name Gram-Schmidt is familiar?
How many have seen Gram-Schmidt?
The Gram-Schmidt idea I'll describe quickly, but just do those words, do those guys' names mean anything to?
Yes, if they do.
Quite a few.
But not all.
OK. OK. And I can I just say, also Gram-Schmidt is kind of our name for getting these two factors.
And you'll see why, it's very cool to have, why this is a good first step.
It costs a little to take that step, but if you're interested in safety, take it.
It might cost twice as much as solving the A transpose A equation, so you double the cost by going this safer route.
And double is not a big deal, usually.
OK.
So, I was going to say that Gram-Schmidt, that's the name everybody uses.
But actually their method is no longer the winner.
And in Section 2.3 and I'll try to describe a slightly better method than the Gram-Schmidt idea to arrive at Q and R. But let's suppose you got to Q and R. Then, what would be the least squares equation?
A transpose Au hat is A transpose b, right?
That's the equation everybody knows.
But now if we have A factored into Q times R, let me see how that simplifies.
So now A is QR, and A transpose, of course, is R transpose Q transpose u hat, and this is R transpose Q transpose b.
Same equation, I'm just supposing that I've got a into this nice form where Q has, where I've taken these columns that possibly lined up too close to each other like, you know, angles of one degree.
And I've got better angles.
I've got these columns of A are too close.
So I spread them out, two columns of qQ that are at 90 degrees.
Orthogonal columns.
Now, what's the deal with orthogonal columns?
Let me just remember the main point about Q.
It has orthogonal columns, right, and I'll call those q's.
q_1 to q_n.
OK. And the good deal is what happens when I do Q transpose Q.
So I do q_1 transpose, these now rows to qn transpose.
q_1 columns to q_n.
And what do I get when I multiply those matrices?
Q transpose times Q. I get I. q_1 transpose q_1, that's the length of q_1 squared is one, and q_1 is orthogonal to all the others.
And then q_2, you see I get the I. q_3, get an n by n. I get the identity matrix.
Q transpose Q is I.
That's the beautiful, just remember that fact.
And use it right away.
You see where it's used, Q transpose Q is I in the middle of that.
So I can just delete that, I just have R transpose R and I can even simplify this further.
What can I do now?
So that's the identity, so I have R transpose R, but now I have an R transpose over here so am I left with?
I'll multiply both sides by R transpose inverse and that will lead me to, Ru hat equals, knocking out our transpose inverse on both sides.
Q transpose b.
Well, that's, our least squares equation has become completely easy to solve.
We've got a triangular matrix here, I mean it's just back substitution.
It's just back substitution now, and a Q transpose b over there.
So a very simple solution for our equation after the initial work of A=QR.
OK.
But very safe, Q is a great matrix to work with.
In fact people, codes are written so as to use orthogonal matrices Q as often as they can.
Alright, so you had a look ahead of the computational side of 2.3, let me come back to the most basic equations just symmetric, positive, definite equations.
Ku=f, and consider OK, how do we measure whether K is nearly singular.
OK, let me just ask that question.
That's the central question.
How to measure, when K, which is, we're assuming be symmetric positive definite, nearly singular?
How to measure that?
How to know whether we're in any danger or not?
OK. Well, first you might think OK, if it is singular its determinate is zero.
So why not take its determinant?
Well determinants, as we've said, are not a good idea numerically.
First, they're not fun to compute.
Second, they depend on the number of unknowns, right?
If I just have twice the identity, suppose k is twice the identity matrix.
You could not get a better problem than that, right?
If k was twice the identity matrix the whole thing's simple.
Or if K is, suppose K is one millionth of the identity matrix.
OK, again, that's a perfect problem, right?
If K is one millionth of the identity matrix, well to solve the problem you just multiply by a million, you've got the answer.
So those are good.
And we have to have some measure of bad or good that tells us those are good.
OK.
So the determinant won't do.
Because the determinant of two I would be two to the nth, the size of the matrix.
Or the determinant of one millionth identity would be one millionth to the n. Those are not numbers we want.
What's a better number?
Maybe you could suggest a better number to measure how close is the matrix to being singular.
What would you say?
I think if you think about it a little, so what numbers do we know?
Well eigenvalues jumps to mind.
Eigenvalues jumps to mind.
Because this matrix K, being symmetric positive definite, has eigenvalues say lambda_1 less than lambda_2, so on.
So on.
Up to, so this is lambda_max and that's lambda_min, and they're all positive.
And so what's your idea of whether the thing's nearly singular now?
Look at lambda_1, right?
If lambda_1 is near zero, that somehow indicates near singular.
So lambda_1 is sort of a natural test.
Not that I intend to compute lambda_1, that would take longer than solving the system.
But an estimate of lambda_1 would be enough.
OK.
But my answer is not just lambda_1.
And why is that?
Because the examples I gave you, when I had twice the identity, what would lambda_1 be there in that case?
If my matrix K was beautiful, twice the identity matrix, lambda_1 would be two.
All the eigenvalues are two for the identity matrix.
Now if my matrix was one millionth of the identity, again I have a beautiful problem.
Just as good, just as beautiful problem.
What's lambda_1 for that one?
One millionth.
It looks not as good, it looks much more singular, but that's not really there.
So you could say, we'll scale your matrix.
And scaling the matrices, in fact scaling individual rows and columns to get it, you might have used, your unknowns might be somehow in the wrong units.
So one of the answers is way big and the second component is way small.
That's not good.
So scaling is important.
But even then you still end up with a matrix K, some eigenvalues and I'll tell you the condition number.
The condition number of K is the ratio of this guy to this one.
In other words, two K, or a million K, or one millionth K, all have the same condition number.
Because those problems are identical problems.
Multiplying by two, multiplying by a million, dividing by a million didn't change reality there.
So if we're in floating points, it just didn't change.
So the condition number is going to be lambda_max/lambda_min.
And this is for symmetric positive definite matrices.
And and MATLAB will print out that number.
Or print an estimate for that number; as I said we don't want to compute it exactly.
Lambda_max/lambda_min.
That measures how sensitive, how tough your problem is.
OK. And then I have to think how does that come in, why is that an appropriate number?
I guess I've tried to give an instinct for why it's appropriate, but we can be pretty specific about it.
In fact, let's do that now.
So what would be the condition number of twice the identity?
It would be one.
Perfectly conditioned problem.
What would be the condition, yeah, OK. What would be the condition of a diagonal matrix suppose K was a diagonal matrix two, three, four?
The condition number of that matrix is two, right?
Lambda_max is sitting there, lambda_min is sitting there, the ratio is two.
Of course, any condition number under 100 or 1000 is no problem.
Roughly the rule of thumb is that the, what's the rule of thumb?
I think that maybe the number of digits in the condition number, the number of digits, maybe if the condition number was 1000 you would be taking a chance that your last three digits, single precision that would be three out of six, three bits, somehow the log of the condition number, the number of digits in it, is some measure of the number of digits you'd lose.
Because you're doing floating point, of course, here.
That's it, totally well conditioned matrix.
I wouldn't touch that one.
I mean that's just fine.
But we can figure out, here's the point that I should make.
Because here's the computational science point.
When this is our special K, our -1, 2, -1 matrix.
-1, 2, -1 matrix of size n, the condition number goes, like, n squared.
Because we know the eigenvalues of that matrix, we could see it.
The largest eigenvalue, when n is big, say n is 1000.
And we're dealing with our standard second difference matrix, the most important example I could possibly present.
Then the largest eigenvalues we actually are there in Section 1.5, we didn't do them in detail, we'll probably come back to them when we need them.
But the largest one is about four.
And the smallest one is pretty small.
The smallest one is sort of like a sine squared of a small number.
And so the smallest eigenvalue is of order 1/n squared.
And then when I do that ratio of four, lambda_max is just like four, this lambda_min is like 1/n squared, quite small.
That ratio gives me the n squared.
So there's an indication.
Basically, that's not bad.
That's not bad if n is 1000 in most engineering problems, that gives you extremely, extremely good accuracy.
Condition number of a million, you could live with.
If n is 100 more typical, condition number of 10,000 is basically, I think OK. And I would go with it.
But if the condition number is way up, then I'd think again, did I model the problem well.
OK. Alright.
So that's, now I have to tell you why is this inappropriate?
How do you look at the error?
So can I write down a way of approaching this does Ku=f?
So this is the first time I've used the word round off error.
So in all the calculations, in all the calculations that you have to do, to get to u, those row operations and you're doing them to the right side too, so all those are floating point operations in which small errors are sneaking in.
And it was very unclear, in the early years, whether the millions and millions of operations that you do addition, subtractions, multiplications, and elimination, do those, could those add up, if they don't cancel, you've got problems, right?
But in general you would expect somehow that these are just round off errors, you're making them millions and millions of times, it would be pretty bad luck, I mean like Red Sox twelfth inning bad luck to have them pile up on you.
So you don't expect that.
Now, what you do solve, so what you actually compute, so this is the exact.
This it would be the computed.
Let me suppose that the computed one is sort of a, there's an error.
I'll call it delta u.
That's our error.
And it's equal to an f plus delta f. And this is our round off error, this is error we make, and this is error in the answer.
In the final answer.
OK, now I would like to have an error equation.
An equation for that error, delta u, because that's what I'm trying to get an idea of.
No problem.
If I subtract the exact equation from this equation, I have a simple error equation.
So this is my error equation.
OK.
So I want to estimate the size of that error, compared to the exact.
You might say, and you would be right, in saying, well wait a minute as you do all these operations you're also creating errors in K. So I could have a K plus delta K here, too.
And actually it wouldn't be difficult to deal with.
And would certainly be there in a proper error analysis.
And it wouldn't make a big difference, the condition number would still be the right measure.
So let me concentrate here on the error in f, when subtracting one from the other gives me this simple error equation.
So my question is, when is that error, delta u, big?
When do I get a large error?
And delta f, I'm not controlling.
I might control the size, but the details of it I can't know.
So what delta f, now I'll take worst possible here.
Suppose this is of some small size, ten to the minus something, times some vector of errors, but I don't know anything about that vector and therefore I'd better take the worst possibility.
What would be the worst possibility?
What right hand side would give me the biggest delta u?
Yeah.
Maybe that's the right question to ask.
So now we're being a little pessimistic.
We're saying what right hand side, what set of errors in the measurements or from the calculations would give me the largest delta u?
Well, so let's see.
I'm thinking the worst case would be if delta f was an eigenvector with the smallest eigenvalue, right?
If delta f is an eigenvector, is x_1, the eigenvector that goes with lambda_1, the worst case would be for that to be the first eigenvector.
That would be the worst direction.
Of course, it would be multiplied by some little number.
Epsilon is every mathematician's idea of a little number.
OK.
So epsilon x_1, then what is delta u?
Then the worst delta u is what?
What would be the solution to that equation?
If the right hand side was epsilon and an eigenvector?
This is the whole point of eigenvectors.
You can tell me what the solution is.
Is it a multiple of that eigenvector?
You bet.
If this is an eigenvector, then I can put in the same eigenvector there, I just have to scale it properly.
So it'll be just this side and what do I need?
I think I just need lambda_1.
The worst K inverse can be is like 1/lambda_1.
Right, if I claim that that's the answer, if the right hand side is sort of in the worst direction, then the answer is that same right hand side divided by lambda_1.
Let me just check.
If I multiply both sides by K, I have K delta u equals K, what's K*x_1?
Everybody with me?
What's K*x_1?
Lambda_1*x_1, so the lambda_1's cancel, then I got the epsilon x_1 I want.
So, no surprise.
That's just telling us that the worst error is an error in the direction of the low eigenvector, and that error gets amplified by 1/lambda_1.
OK.
So that's brought lambda_1, lambda_min into it, in the denominator.
Now, here's another point.
Second point now.
So that would be the absolute error.
But we saw for those factors of two and one millionth and so on, that really it's the relative error.
So I want to estimate not the absolute error, delta u, but the error delta u relative to u itself.
So that if I scale the whole problem, my relative error wouldn't change.
So, in other words, what I want to do is ask in this case, what's the right hand side?
How big, yeah, I know, I want to know how small u could be, right?
I'm shooting for the worst.
The relative error is the size of u, maybe I should put it this way.
The relative error is the size of the error relative to the size of u.
And I want to know how big that could be.
OK.
So now I know how big delta u could be, it could be that big.
But u itself, how big could u be?
How small could u be, right?
u's in the denominator.
So if I'm trying to make this big, I'll try to make that small.
So when is u the smallest?
Over there I said when is delta u the biggest, now I'm going to say when is u the smallest?
What f would point me in the direction in which u was the smallest?
Got to be the other eigenvector.
This end.
The worst case would be when this is in the direction of x_n.
The top eigenvector.
In that case, what is u?
So I'm saying the worst f is the one that makes u smallest, and the worst delta f is the one that makes delta u biggest.
I'm going for the worst case here, so if the right side is x_n, what is u?
x_n/lambda_n.
Because when I multiply by K, K times x_n brings me a lambda_n, cancel that lambda_n I get it right.
So there is the smallest u, and here is the largest delta u.
And the epsilon is coming from the method we use, so that's not involved with the matrix K. So do you see on there, if I'm trying to estimate delta u over u, that's b.
The size of delta to u over this, the size of delta u over the size of u is what?
Delta u has some epsilon that measures the machine number of digits we're keeping, the machine length, word length and so on.
This is a unit vector over lambda_1.
That's delta u over lambda_1, this u is in the denominator.
lambda_n u is down here, so the lambda_n flips up.
Do you see it?
By taking the worst case, I've got the worst relative error.
So that for other methods, other f's and delta f's, they won't be the very worst ones.
But here I've written down what's the worst.
And that's the reason that this is the condition number.
So I'm speaking about topics that are there in 1.7, trying to give you the main point.
The main point is, look at relative error because that's the right thing to look at.
Look at the worst cases, which are in the directions of the top and bottom eigenvectors.
In that case, that relative error has this condition number, lambda_n/lambda_1, and that's the good measure for how singular the matrix is.
So one millionth of the identity is not a nearly singular matrix, because lambda_max and lambda_min are equal, that's a perfectly conditioned matrix.
This matrix has condition number two, 4/2.
It's quite good.
This matrix is getting worse with an n squared in there; if n is big and other matrices could be worse than that.
OK. so that's my discussion of condition numbers.
I'll add one more thing.
These eigenvalues were a good measure when my matrix was symmetric positive definite.
If I have a matrix that I would never call K, a matrix like one, one, zero and a million, OK, that I would never write K for that.
I would shoot myself first before writing K. So what are the eigenvalues of lambda_min lambda_max for that matrix?
Are you up now on eigenvalues?
We haven't done a lot of eigenvalues but triangular matrices are really easy.
The eigenvalues of that matrix are?
One and one.
The condition number should not be 1/1, that would be bad.
So instead, if this was my matrix and I wanted to know its condition number, what would I do?
How would I define the condition number of A?
You know what I do whenever I have a matrix that is not symmetric.
I get to a symmetric matrix by forming A transpose A, I get a K, I take its condition number by my formula, which I like in a symmetric case, and then I would take the square root.
So that would be a pretty big number.
For this, for that matrix A, the condition number of that matrix A is up around 10^(6).
The condition number of that matrix is up around 10^(6) even though it's eigenvalues are one and one because when I form A transpose A, those eigenvalues will jump all over.
And then probably this thing will have eigenvalues way up and the condition number will be high.
OK.
So that's a little, you'll meet condition number.
MATLAB shows it, and you naturally wonder what is this?
Well, if it's a positive definite matrix, it's just the ratio lambda_max to lambda_min and it tells you as it gets bigger that means the matrix is tougher to work with.
OK. We have just five minutes left to say a few words about QR.
OK, can I do that in just a few minutes?
And much more is in the codes.
OK, what's the deal with QR?
I'm starting with a matrix A.
Let's make it two by two, a two by two.
OK, it's got a couple of columns.
Can I draw them?
So it's got a column there, that's its first column and it's got another column there, maybe that's not a very well conditioned matrix.
Those are the columns of a.
Plotted in the plane, two space.
OK, so now the Gram-Schmidt idea is out of those columns, get forced to normal columns.
Get from A to Q.
So the Gram-Schmidt idea is, out of these two vectors, two axes that are not at 90 degrees produce vectors that are at 90 degrees.
Actually, you can guess how you're going to do it.
Let me say, OK I'll settle for that direction, that can be my first direction, q_1.
What should be q_2?
If that direction is the right one for q_1 I say OK I'll settle for that, what's the q_2 guy?
Well, what am I going to do?
I mean, Gram thought of it and Schmidt thought of it.
Schmidt was a little later, but it wasn't, like that hard to think of it.
What do you do here?
Well, we know how to do projections from this least squares.
What am I looking for?
Subtract off the projection, right.
Take the projection and subtract it off and be left with the component that's perpendicular.
So this will be the q_1 direction and this will be, this guy e, that what we call e would tell me the q_2 direction.
And then we could make those into unit vectors and we'd be golden.
And if we did that we would discover that the original a_1, a_2 and a_1, so this is the original first column, and the original second column, would be the good q_1 and the good q_2 times some matrix R. So here's our A=QR.
It's your chance to see this second major factorization of linear algebra.
LU being the first, QR being the second.
So what's up?
Well, compare first columns.
First columns, I didn't change direction.
So all I have is here is some scaling r_1,1, zero.
Some number times q_1 is a_1.
That direction was fine.
The second direction, q_2 and a_2, those involve also a_1.
So there's an r_1,2 and an r_2,2 there.
The point was, this came out triangular.
And that's what makes things good.
It came out triangular because of the order that Gram and Schmidt worked.
Gram and Schmidt settled the first one in the first direction.
Then they settled the first two in the first two directions.
If we were in three dimensions, there'd be an a_3 somewhere here, coming out of the board.
And then the q_3 would come straight out of the board.
Right?
If you just see that you've got Gram-Schmidt completely.
a_1 is there.
So is q_1.
a_2 is there.
I'm in the board still, in the plane of a_1 and a_2, is the plane of q_1 and q_2, I'm just getting right angles in.
a_3, the third column in a three by three case, is coming out at some angle.
I want q_3 to come out at a 90 degree angle.
So that q_3 will involve some combination of all the a's.
So if it was three by three, this would grow to q_1, q_2, q_3, and this would then have three guys in its third column.
But maybe you see that picture.
So that's what Gram-Schmidt achieves.
And I just can't let time right now without saying that this is a pretty good way.
Actually, nobody's thought there was a better one for centuries.
But then a guy named Householder came up with a different way, and a numerically little better way.
Numerically a little better way.
So this is the Gram-Schmidt way.
Can I just put those words up here?
So there's the Gram-Schmidt, the classical Gram-Schmidt idea, which was what I described, was the easy one, easy to describe.
And then there's a method called Householder, just named after him, MATLAB would follow.
That every good qr code now uses Householder matrices.
It achieves the same results.
And if I had a little bit more time I could draw a picture of what it does.
But there you go.
So that's my lecture on, my quick lecture on numerical linear algebra.
These two essential points and I'll see you this afternoon.
Let me bring those quiz questions down again, for any discussion about the quiz.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Alright.
So this is Lecture 15.
It's the last topic, today and Friday, like just 15 and 16, trusses within Chapter 2.
The last topic we'll do for discrete systems.
Then it's a lot of fun.
I wanted to say a few words first about last night's exam.
Several words first.
Overall, I'm sure it's going to be fine.
Ramis is grading the first two problems, he'll pass them to Peter for the next two.
And I'll get them back.
I'm pretty sure it'll be next week.
I felt it was a fair exam, except I should have done a better job in helping you with the matrix A.
Especially in problem one.
I'm glad that hint was there, the matrix A_0, that goes with free-free to sort of say what kind of matrix to be looking for.
And I thought I'd just repeat, make the connections that I should have made earlier.
So we all see the point about these.
These A's and the A transpose A's.
So if I take, for that A_0, free-free one.
Everybody sees that this, and this connects of course with our graphs.
Our graph is just this simple graph with well actually is that how many, is it five nodes?
I guess there are five.
Because as it stands I have one, two, three, four, five columns, I've got five u's, u_0 down to u_4.
And if I take A_0 transpose A_0, that would be the free-free matrix.
What size would it be?
And what matrix would it be?
Just if we do that multiplication, this is a first difference matrix.
When I do A_0 transpose A_0 I'll get one of our second difference matrices.
So it'll be one of our special ones.
Which special one would it be?
B.
It'll be the matrix B that has both ends free.
And what size will it be?
I guess it'll be five by five.
That's right; that would be five by four times A_0, which is four by five.
So it'll be the five by five matrix B.
Can I call it B_5?
OK.
So that was there as a hint.
That isn't the correct matrix for problem one because problem one was fixed fixed.
Let's get there in two steps.
Suppose it's fixed free.
So suppose I make u_0=0.
So I ground the top node, I support the top node - oh no, shall I do u_0?
Yeah.
I'll do u_0.
So that would knock out this one.
If I say fix u_0, say, at zero or whatever.
Now I've got, the next a, I won't call it A_0 anymore.
So now four by four, now if I do A transpose A, which of our special matrices am I going to get?
t. It'll be T. It'll be the one that has the first 1, 1 entry will only be a one.
So that'll be the fixed free matrix.
It'll be of what size?
Four.
Now I've only got four unknowns.
u_1 to u_4.
OK, that still is not what problem one is.
Problem one was fixed fixed.
So as I did in the review, that would knock out both of these columns.
So this now is the matrix that I was looking for in problem one, and I wish I had emphasized these steps in advance.
I apologize.
OK, so what fixed-fixed if, without the C part in it.
Just focusing on the A, what fixed-fixed matrix would I get?
Which one of our guys would it be?
K of size three.
And while we're at it, what would be the story, how would I get one of these circular ones, which is sort of on our special list.
For Fourier it's the really special guy.
So a circular one I'm going to connect u_4 back to u_0.
So I'm going to put these guys back in.
And what else would change, if u_4 was connected back to u_0, now I'm aiming for this circulant.
What matrix A is going to give me the circulant?
So again, these guys are the A transpose A's.
This over here was the A, and over here is the A transpose A.
And now I want to fix A, and then I want to see that A transpose A.
So suppose I give you that graph, then.
Oops, I should have, well.
Just connect the whole guy.
So fifth node coming back to the first.
So that's my circle of nodes.
That's a simple graph.
What's the a circulant now?
So this would be the A for the circulant case.
So it's got that back in.
That shouldn't be erased, that shouldn't be erased.
And what else is it got?
If I ask you for the incidence matrix, now I'm in Section 2.4, like I've given you a graph, or you can think of masses and springs in a circle.
So I've got five masses, five springs.
What's my matrix A missing?
It needs another row.
We just put in another edge, it needs another row.
That edge went the node back to the first node.
So we've got a fifth row.
So you see, now it really is circulant.
I would call this one also a circulant matrix.
The diagonals are constant.
That what I and MATLAB and everybody else would call a templates matrix, and the command templates could create this.
That diagonal is constant.
That diagonal is constant.
The other diagonals are constant.
But more than that, what's additional here in the circulant, which is the thing that makes Fourier happy?
The diagonal circles around.
That diagonal has only got four entries in it, but it circles around sort of periodically to its fifth entry.
So that's more than templates.
It's circulant because it's coming around again.
This we'll see in the discrete Fourier transform.
It's really all good stuff.
And now there is my a circulant.
And what would be my A transpose circulant A circulant?
What would be A transpose A if I take that five by five matrix?
C. Finally I've created, so I've already got B, I've got T, I've got K, all those three special guys.
And now the A transpose A for this circulant, so that's a first difference matrix for a periodic problem.
And A transpose A will be a second difference matrix for a periodic problem; c_5, I guess.
It'll be five.
OK.
I hope that brings together what I, if I was on the ball, I would of brought it together before the quiz.
Can I just say a few words about the quiz and grades?
They come out fine.
Really they do.
I've been doing this a long time.
And just, enjoy October.
I'm sorry to give you any exam at all, but it's a chance for you yeah, I'm working on this stuff.
I'm learning it.
Everybody doesn't it first time.
I don't learn it first time, every time I teach the course I learn something more.
And if you're learning from this course then I'm totally happy.
And I believe that's the case.
So I am entirely happy.
And I hope the quiz, some points of it I wish I'd prepared better.
But I feel pretty good about it.
I feel good about it, let me just say.
So, and I'm happy to have any comments, email or in person.
But allow me to go forward with trusses.
However, I'm ready always for a comment.
Yeah.
OK.
Anyway, enjoy trusses.
Enjoy life.
Yeah.
And this should have been in the book, this page.
So if I wasn't too late I would paste it in.
Because this connects A transpose A to a special matrices, and the way I had in my mind but I didn't put it on the board until just now.
OK, so I'll cover that up and ready to go with trusses.
OK.
So trusses, we want to know what's up.
We want to get the setup right.
Once we get the setup we'll know we're looking for.
OK, so a truss is a bunch of elastic bars with pin joints connecting them.
Now, what do I mean by a pin joint?
I mean that stretching the bars takes force.
Turning around the pin joint doesn't take force.
So the pin just lets them turn, so we'll have forces in those bars.
So it's like masses and springs.
Exactly like masses and springs.
But yet we have a 2-D problem.
So it's a two dimensional problem with masses and springs.
And we could certainly have a 3-D truss, but 2-D makes all the important points.
And then I can count the bars.
One, two, three, four, five.
And I can count the nodes.
There happen to be five here.
But now comes the moment.
I have to tell you, what are the unknowns?
What are the u's.
Because of course, you know that I'm going to go from u's to e's to w's, to forces.
f. And you know that a matrix A is going to do that.
A matrix C is going to do that.
A matrix A transpose is going to do that.
You're all ready, we need to know what's the setup.
What are these matrices.
OK, and how many.
So let me explain the setup.
Typical node, node one.
we have forces in these bars, so that node one could have a force.
We're in the plane.
So we have a horizontal force and a vertical force.
Together, that would produces a force in any direction whatever.
So this is the key point.
That there is a horizontal force.
f, horizontal one.
The one being the force on node one.
But there's also a vertical force.
And let me take horizontal to the right, positive to the right.
Vertical positive upwards.
Just to have a convention.
So how many f's have I got?
Well, the point is I now have two per node.
That's the difference.
I have two per node, two forces.
and I have two displacements per node.
Because that point under, there will be more forces.
Some maybe pulling this way, whatever.
Maybe let's look at node two.
So node two could have a couple of forces on it.
F H two, and F V two.
And it moves like the other nodes.
So now I'm introducing the unknowns.
u is the movement.
u, again horizontal, and again, now we're talking about node two.
And it moves up.
Or doesn't, or moves down.
But that's an unknown.
u, a displacement, a vertical displacement of node two.
Do you see the setup?
Two forces per node.
Two displacements per node.
So that's like, the number of unknown is like double.
Like, doubled, and that produces an interesting situation.
I've marked supports here.
So let's just speak about supports.
So what's happening at the supports?
At the support there's no movement.
The whole that point is pinned.
So this is telling me that u horizontal five is zero and u vertical, sorry that was four and it'll be the same for five.
u vertical five is zero.
It's like grounding a node in the electrical case.
We just see this pattern over and over.
And we want to see OK, what does it look like for trusses?
So here's a support that fixes those.
So those are not unknowns.
And similarly, they're not unknowns there.
Still saying five when I mean four.
So those are boundary conditions, n conditions, whatever.
And similarly here.
So how many unknowns are there?
Now look at this picture, how many unknown displacements are there in this truss?
Six.
Six, right?
Two here, two here, two here and none there.
So the number of actual unknowns is six.
My idea would be that it's twice the number of nodes minus the number of fixed things, that R would be four in this case.
I've got two fixed here and two fixed here, so this would be two times five.
Ten possible displacements but R counts the number of fixed displacements, four, and leaves us with six.
OK.
So my matrix A will now be, it's always m by n. My matrix A will be five by six.
OK. Now you're going to ask what is that matrix.
But let me hold that off for a little moment.
I want to just see its shape first.
So you could now do this for a large truss, right?
You count the bars, and you could count the nodes.
And then you could count the unknown displacements, u.
So there are six u's here.
And there are five e's.
And there are five bar forces.
And there are six equilibrium, balance, force balances.
Six, six for the node count, for the unknowns count, five, five for the bars count.
OK, now here's a point about this particular truss.
It's not safe to get on it.
Right?
And I want to say why is it not safe.
So this is a feature that comes into the truss question that makes it a little new and more interesting.
A little twist compared to the previous examples.
That bar, that truss, I wouldn't stand on it.
Now, why not?
Well, purely for linear algebra reasons.
Of course.
The matrix A is five by six.
So now what do we know about a matrix that's five by six?
So A is five by six.
Five rows, as always the m; six columns, because now we have six unknowns.
And what do I know about any five by six matrix?
I want to ask about the equation Au=0.
So I want to ask about it in linear algebra language and then I want to ask about it in physical language.
And the beauty is the thing that makes trusses sort of fun is, these matrices, A, get pretty big fast.
Because when I put a few more nodes on, the book has a picture of a sort of treehouse.
Then A is growing.
And I don't, all the time, write down the matrix A. I haven't written it down here.
What I've written down it's just its size, because that's enough to tell us something about this set of equations Au=0.
What's the story Au=0?
Well of course it has the solution u=0.
Nothing moving.
If I have no displacement, if the u's are all zero, then I have no stretching.
The e's are stretching.
Elongation, as before.
How far does the bar stretch?
OK.
So if I have zero u's and zero e's, but what other possibility am I going to have here?
I'm going to have probably one solution to this system that isn't zero.
I'm probably going to have one set of displacements u, look what's happening here.
This is Au is the e. So I'm going to have at least one and probably in this case it will be one, there will be one, the neat word for it is a mechanism.
And what does that mean?
A mechanism is a solution to Au=0.
So that, a mechanism is a movement of the bar.
So it's going to be non-zero.
The bars are going to move a little.
Sorry, the nodes are going to move a little.
The nodes will move a little bit.
In this u, because it isn't zero.
But the bars won't stretch.
So tells us we've got instability here.
If there's a solution to that, that's always telling us that A transpose A is singular.
So let me just put that A transpose A, or A transpose C A, C couldn't save it.
Will be singular.
It's just like our free-free thing in being singular, but the picture doesn't look free free, does it?
It's got supports in here, just not good enough.
And I believe that if you look at this truss, you could describe, you could tell me, and you could draw, a movement of that truss in which there is displacement but no stretching.
Let me ask you how to draw that.
And I believe, everybody understood why was there a solution.
It was because we have six unknowns and we only have five equations.
So this was five equations.
Any time you have five equations with a zero on the right hand side, so five homogeneous equations, whatever you want to say when that's zero on the right and six unknowns.
Six u's.
Then you're going to have solution.
You can't help it.
You've got that many degrees of freedom, you've only got that many constraints, there's going to be solution.
OK, tell me how to draw that.
Let me put in the truss now.
What's the solution?
So this is the fun part in a particular example at the start.
How could that move without stretching bars?
Let me see.
What could happen?
What do you mean now, who's going to move where?
What's the movement here?
And I want to draw it over there.
So you give the answer by drawing it as well as by telling me the six unknown u's.
So what can happen at this thing?
So you're going to say the truss could, these bars could, turn a little?
And notice that word a little.
We're talking small displacement, small stretches all the time here.
I'll show you why we're always making that linearity assumption, or small assumption.
OK, those move a little.
And what happens to that triangle at the top?
It sort of just moves along, right?
So the picture you would draw would be that you started there.
And it moved along a little, I'll make it a larger displacement than I really have in mind.
These guys of course are here.
So they come out and the rest of the truss, the top of the truss just kind of goes with it.
Goes with the flow.
That would be the answer that I would be looking for, to draw the mechanism.
That would show it.
And if I wanted to write down the u that goes with it, what would it be?
Let me again number these guys; this is one, two, and this is three.
So what are the displacements of nodes one, two, and three?
I'll always write u_1 horizontal before vertical.
Can we make an agreement?
So I want to know about the horizontal movement then the vertical movement of node one, then node two, then node three.
So I'll have six numbers there.
And what could I put in for those six numbers?
So the horizontal, let me suppose that first guy, I'll put a one.
Really that's a bigger number than I should put, but it's a convenient number.
So I'll just take it to be one even so I really have in mind.
Let's say that's one angstrom, or one tiny little person.
OK, so what about the rest?
What's the vertical cool, oh yeah this is a key point here, what's the vertical movement.
This movement to me is horizontal.
I'm going to say that the vertical moment is zero.
Of node one, just moves over.
And node two does the same.
And node three does the same.
So that's my solution.
.
That's a simple movement, a simple set of displacements, think most to the right.
I have not written down the matrix A, but probably won't even do it until next time.
But you will see that when we do the matrix A for this particular truss will have this particular u as a mechanism.
In linear algebra, u is in the null space of A. Au=0, that's all that means.
OK, do you see more or less what's up?
But now there's one little thing that may be bothering you.
Which is what?
If I come back to the zero, zero, zero there, you could correctly say wait a minute, if those bars didn't stretch, if they just rotated as you told me to do, then this was mostly across but a little bit down, right?
And I'm saying no.
I'm saying zero.
OK, how do I get away with that?
So I'm saying in 18.085 it's a zero.
And why?
So this is like a little time out just to focus in on, let me focus in on node two.
So here's the bottom node four, so it used to be vertical up to two.
This was node two and this was number four.
And then it rotated a little, to there.
To this position.
So it went, if this angle was, let's say, theta, then what is that actual position?
So let's say this was, let's say the bar had length one.
This is the origin, zero, zero.
This is the point zero, one above it, OK?
And now, that's before it moved.
Then it moved a little bit.
It moved to an angle theta.
What's the position of that bar?
Of that node?
What's the new position of the node, and then we'll look at the difference and we'll see the movement u, the displacement.
So how far did it move?
What's the x coordinate?
How far did it go across?
If I put in that line you'll know.
So the movement across was, sin(theta)?
Good.
sin(theta).
And the movement down, well, yes, so let's find its position and then we'll take the difference.
So what's the vertical new position for that guy?
It moved by, it moved across by sin(theta).
And down by 1-cos(theta).
Are you agreed with that?
Because here is cosine theta, right there.
And here's the little bit it moved down.
OK.
So these are exactly correct.
Yeah this is in the position of sin(theta), cos(theta), and the difference was the 1-cos(theta).
OK, so now here comes the key point.
Approximately sin(theta) is approximately, if theta is small and now here comes the smallness, sin(theta) is approximately theta.
sin(theta)'s approximately theta.
And 1-cos(theta) is approximately what?
Now, this is the important point.
So theta is like the first term.
If I expand, I mean, the exact term would be theta minus theta cubed over six, dot dot dot.
But I'm only keeping that term.
And 1-cos(theta), now what's the formula for cos(theta)?
This is like worth, just should?
It's a one, because of course cos(0) is one, and then you subtract what?
Theta squared over two.
And so on.
And then plus theta fourtg over 24 or whatever.
OK, so the ones cancel as we expect.
And I'm getting theta squared over two.
And this, here was theta to the first power.
Theta cubed was, we didn't care.
And we don't care about theta squared.
So that's why it's zero.
Because it's a second order movement.
If theta is small, as I'm going to assume, small displacement, theta squared would, if I allowed theta squared and cos(theta) in here, I'd have a non linear problem.
And I don't want that.
And I don't need it.
I mean, finite elements, structures, bridges, whatever.
Your first hope and expectation and calculation is small theta linear problem.
So to a linear person theta squared is zero.
That's why those guys are zero.
OK, so that's an assumption we'll often see, so it kind of was.
There are two kinds of non-linearities in structures and elasticity.
One would be to allow this geometric non-linearity.
Theta's large displacements, theta large enough so that you can't neglect theta squared.
That's a tough one.
If you allow geometric non-linearity in, as finite element codes have to do.
If you have ABAQUS is a code that does major finite element calculations, nonlinear ones.
I mean they, at the beginning they were studying what happens, what are the stresses on cables under the Atlantic.
I mean, those are fascinating problems.
Or I mention car crashes.
I mean, car crashes, the geometry changes, you have big displacements.
But we're talking here about linear small displacement cases.
OK, so don't forget that part.
That when the truss gets more complicated, the principle stays the same.
That we distinguish between the thetas that matter and the theta squareds that don't.
OK, so now what?
Now I guess I'm ready to complete this picture a little more.
OK.
So let me, so we've understood what the idea of a mechanism.
Oh, how could I prevent a mechanism?
In other words, if I stood on this truss, the slightest bit of wind would crash it down, right?
So that's unstable.
That's an unstable truss.
How could I make it stable?
I mean if you were designing this thing, what would you do?
Add another edge.
You'd stick in another bar.
Maybe stick in a bar there.
What would happen now?
Would it now be stable?
You'd have to answer that question.
You couldn't just put in bars, whatever.
You want to put bars that do the job.
OK, now how many bars?
We've now got six bars.
So m is now up to six.
The matrix A is six by six now, whatever that matrix may be.
We have six bars, six displacements.
We can hope that we now have a six by six, well, we do have a six by six matrix, whatever it looks like.
And we can hope that it's not singular.
We can hope it's invertible, we can hope that that mechanism is killed.
And you see it is killed.
The six by six, that truss is now stable.
No mechanism there, right?
Again I haven't written down the matrix, but I'm really calling for engineering intuition here.
That this truss is now stable, and of course I can make it even more stable by adding a seventh edge.
A seventh bar.
So when it was six I had square matrices, A transpose and then C and then A would have been square.
Now I've got seven bars and, so now I've put in a seventh guy.
m is now up to seven.
My matrix a would now be seven by six.
Mechanism will be gone because I've now got, what, seven equations.
Same six u's.
So we begin to get a feel of, are there solutions or not?
What I'm saying is I can't tell just from the count that a is not singular.
I could have a lot of bars and still be unstable.
Invent a truss for me.
Just because, how could you invent a truss that had, maybe it has seven bars.
With those seven bars, those diagonal guys, that did it.
That made it stable.
Our eye tells us that before we do any linear algebra.
Tell me a seven by, a thing.
Well, yeah.
OK, so here would be, shall we support both of these?
I'll start out the same, OK. Now, yeah, how could I, let's see, I've haven't prepared.
How could I get a whole lot of bars.
I might not get seven by six exactly, but how could I have plenty of bars and still unstable?
Well, suppose I do this.
Oh yeah, that's a good example.
That's not stable, right?
OK. Every let's practice with that one.
That's just my idea, and problems in the book just ask you to practice with things like that.
Tell me the count, first.
What is m, the number of bars?
Six.
What is n, the number of unknowns, little n, the number of unknowns?
What's the shape of my matrix here?
A is, it's got six bars and how many unknowns?
Eight.
Eight, right?
two here, two, two, two.
None here.
Six by eight.
OK. And how many mechanisms am I now expecting?
Probably two.
Probably two, there would be two independent mechanisms here.
Can you tell me what they look like?
Draw them.
What would they look like?
What would be two different things that could happen, could go wrong with that truss?
You see it, right?
This could turn.
As in our example with the top part moving with it.
Or, second one possibility would be the top part goes.
And the bottom part stays.
Or any combination.
So the whole thing could go like that.
That would be one.
But that wouldn't be the only one, of course.
So in other words, we have a two dimensional space of mechanisms and you could give me two different, and there are not just two guys, all their combinations are there.
So this would have two mechanisms.
Two mechanisms.
And I could put in bars, of course, that would try to save it.
Well, how many bars, what's the minimum number of bars I absolutely need to make this thing stable again?
Two.
Well, now suppose I put in these two bars.
Right?
I've got enough bars, I've got an eight by eight matrix, but I haven't saved it.
Right?
Because it still has that mechanism.
So you can't assume that because the count is right you've avoided mechanisms because in that example you haven't.
OK, so that would be a case of square eight by eight, but not good.
So as soon as I say there's a solution to Au=0, I know that A transpose A will be singular.
And unstable.
OK, before I go to the framework let's just do one more thing.
Suppose I take away the supports.
All right, let me put in some bars, though.
I'll put in some bars.
OK, plenty of bars.
Want another one?
OK, how many bars have I got?
Lots, right?
OK, now the matrix A, what do you think about this?
Are there solutions?
You haven't even seen the matrix A, of course, but you've seen the truss, that's what matters.
How many solutions, are there solutions to Au=0?
Are there ways that this truss could move without stretching?
Are there ways that this truss could move without stretching?
And what are they, and how many are there?
And what name should we use?
OK, what are they?
How could that move without stretching?
Well, it's got no supports at all.
It's just free out there in space.
So it could move.
How many ways could it move?
Three.
It could move, everybody could move this way.
All ones on the horizontal guys.
Everybody could move this way, all six ones on the vertical guys, or be 12 unknowns here.
And it could also rotate, what would be the rotation?
I'm not talking about this rotation.
This could not happen.
What could happen?
What rotation could happen here?
For this, there's a third rigid motion.
Translation, translation, and rotation around, well take this one as an example.
The whole thing could swing around this.
That would be a motion.
Well now you're going to say well, why didn't I swing it around that one?
And of course it could.
But what would be the deal?
It would have to be, there are only three rigid motions, right, up and around.
So if you give me another one, like, around this one, then somehow it had to be a combination of those.
I don't even want to think what combination it is.
But there are three rigid motions.
So I sort of distinguish mechanisms, this word mechanism.
So that's where the truss deforms.
In these and rigid motions, so I'll say plus, possibly.
Plus rigid motions, and rigid motions would be, you know, it doesn't deform internally, the whole thing moves.
And this is of course what we get in the case of not enough supports.
And this is what we get in the case of not enough bars.
Yeah.
So maybe it's worth separating those two.
In the examples we do, we'll usually put in enough supports to kill the rigid motions.
And then the question would be are there some mechanisms.
OK. Now, I have to start on what this is.
Well it'll be just a very quick start.
So what I'll do at the beginning of Friday, so Friday's the other lecture on this topic.
And then the homework will ask you to do some trusses in this section.
It's probably Section 2. something.
2.7, maybe.
What's the matrix C?
Last second question, what's the matrix C, what size is it?
What size is the matrix C for our original problem?
Or no.
What size is C, is C involving, if I know these numbers, what size is C?
Five by five.
m by m, right?
C is the diagonal matrix, one entry for each bar, C is just C. It has a c_1, c_2, c_3, and this w=Ce, it's just Hooke's Law on each bar.
So, simple.
It gets there in the middle, just the way that C in the first exam problem popped in, and other C's.
That gets there in the middle.
But it's very, extremely, simple.
OK, so the real attention is on A, as usual.
And that will come Friday morning.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so this is the second last lecture on trusses.
Then we've got a holiday on Monday.
And then after that we'll be into Chapter 3.
I thought I'd write down just in case it's use to you, the four problems that I intend to include with the next homework.
That won't be due for quite a while, a week from Monday.
So these will be the problems on trusses that come from particular trusses drawn in the book.
And then there'll be some problems from the new material, that we do next week.
So trusses and really, there's two main jobs for today.
One is to identify this matrix A, the strained displacement matrix or the stretching matrix.
How far do the bars stretch?
Everybody remembers A is going to come in this step if we have displacements then of the nodes like this would be like a u_1, this would be a u_1 h and a u_1 v, this would be a u_2 h and a u_2 v, so there are four movements of the ends of the truss and of one particular bar, and then we'll stretch that bar.
And the question, is how much?
So that will be one row of A.
So if we follow one bar, you remember in the matrix A, there's going to be a row for every bar.
So a row for each, row of A.
For each bar.
And if we track down one of those rows, we'll have the idea.
And then of course at the end we'd maybe be constructing A without, sort of a free-free A.
And then at the end, any fixed displacements that will knock out columns of A.
So that's one job.
And then to see, so the A is going to be a little more messy.
It's because we're in two dimensions.
So compared to the network problems, and and the line of springs, now we have more happening.
We've got more columns because every node has now two unknowns.
A horizontal and vertical.
So A is kind of bigger.
And therefore A transpose C A, you might think, it's going to be hard to see what's going on.
But you'll see the right way to look at A transpose C A is a bar at a time.
That's the nice fact about A transpose C A, I might focus on that first.
And then comes the fun part.
I'll draw some more trusses, that may or may not have mechanisms.
They may or may not be stable.
And we can try to identify the mechanisms.
Actually, as before.
We'll do it by engineering instinct rather than by solving.
I mean, in principle, we could always use elimination.
Or ask MATLAB or any other system to do it, and look for the solutions to Au=0.
And decide are the columns of A independent.
In that case the truss is stable.
This matrix is invertible, we know all the good possibilities.
And then there's the more interesting possibility, of having some solutions to that.
In which case that matrix will be singular.
There'll be some modes in our big system that will cause it to fail.
But it's kind of fun to find those.
OK, while I've written A transpose C A, may I remind you about a good way to do that multiplication.
OK, so imagine I'm just putting a number.
Here's going to be the matrix A.
So the matrix A will have a bunch of rows, row one, row two, so on.
These rows will correspond to bar one, bar two, and bar three.
OK. OK, then we have C, so that's a square matrix.
That each bar has a spring constant, so c_1, c_2, c_3, and then we have A transpose.
And those rows, or columns of A transpose.
So that's the sort of picture of A transpose C A, for a three bar, three bars only.
But the point is made right here.
There's a row of A for every bar.
Right?
Because our matrix A is m by n. If there are m bars, a row for every bar, and it tells us how far that bar is stretched.
And we'll figure out what its entries are.
That's our main job.
I'm just looking ahead.
Suppose we've got that row.
And that row, and that row.
So a row for every bar.
Now, here I've taken three bars.
Now, how do I multiply those matrices?
Well, I can do it different ways.
But here's a cool way to do it.
Just the way I want to point out is column times row.
If you multiply matrices you're allowed to, the effective c_1, c_2, c_3 is going to be very simple.
So I'm really paying attention here to A transpose A.
If I want to multiply A transpose A, I can do row times column as usual and get one number.
Or I can do column times row and get a whole little matrix.
And that's the bar one matrix.
It's the element matrix, and that's how finite elements will be assembled, and that's why I should keep mentioning this point.
So the way to do that column times row thing, and then of course that c_1 just multiplies that row, that'll be c_1.
Row one, transpose, that's the column, times row one.
That's what's coming from bar one.
That column multiplies the c_1 and that row.
You see how nice, that's the element matrix associated with the first bar.
And then there'll be a second column times the c_2, times the second row.
So plus c_2, row two, transpose row two.
That's a matrix again.
Plus c_3, row three transpose.
Row three.
I focus on that because you don't think of this as a way to multiply matrices, but it's really a nice thing to notice.
And it's better to notice it now when we have three bars or something, than in a big finite element code.
Yeah.
So this is, just if I complete it, complete this thought, this I would call up a one bar matrix.
That's the matrix A transpose A if there's only one bar.
Actually, one of the problems at the end of this section is find the element matrix for one bar.
And I guess it's about what we're going to get to when we do that one bar.
Do you remember what it was in the, just to connect this thought, what was the little matrix?
In the case of networks?
So in the case of networks, there was just one unknown for each, not two.
So for networks, just, I'm just going to put down the, and you'll recognize it immediately.
The little element matrix was the c for that.
And there was an 1, -1, -1 what?
Do you remember that guy that was the -1, 1 from a row?
-1, 1 from a column?
So this was exactly c times the -1, 1 from the column times the -1, 1 from the row.
That's where this simple little matrix came from.
And you remember that the, so what's involved in creating this big A transpose C A is just create all these little pieces.
Which are like this, but they're going to be a little bigger.
Fact, in a minute I'm going to ask you what size they'll be.
Well, they're really big matrices.
There are a whole lot of zeroes there that I didn't even put.
Zeroes are there for rows and columns that aren't touching this particular edge.
And again, this matrix.
There'll be all kinds of zeroes at A.
Because a typical row of A, bar one, is going to have non-zeroes only for the, yeah what's the size?
How many non-zeroes in a typical row of A?
Getting the count right first is like half the battle.
How many non-zeroes in a typical row of a?
This was the network case where we had a couple of nodes.
And they were connected.
And we had an unknown at each end.
So two unknowns were involved.
The little matrix was two by two.
It properly has lots of zeroes for all the other nodes that are not involved.
And then that matrix kind of gets, assembled is the word I think usually used.
All these little guys get assembled, you know, pasted, stamped, I hear the verb sometimes now.
Take these little matrices for this little element.
And stamp them into the big A transpose C A.
This is the c_1, so this gives the c_1's in the matrix.
And the c_2's 2 and the c_3's.
Alright.
Now, just before we, I'm like doing this preliminary, before I write down anything, the exact row.
What's the size of, for trusses, how many non-zeroes in a row of A?
So that's my question.
How many non-zeroes in a typical row, like for that bar, non-zeroes in a row of A?
So A is the matrix that tells us how much, that row of A is the row that tells us how much this bar stretched when this moved along by u H 1, and up by you u V 1, and this moved along by say, u H 2, and up by u V 2.
Well, I've written all those in.
So that you can tell me this number.
How many?
What's your guess?
When I tell you, you'll say of course.
How many u's are involved in the stretching of that bar?
Four.
Four.
Exactly.
Instead of one at each end, we have two at each end.
So the answer is four.
How many, the answer is four.
And now the only remaining question is, what are they?
What are those four numbers?
The four non-zeroes in the row?
So let me just answer that.
They are, so here is that row.
So we the two non-zeroes associated with it.
Well, the way I've numbered these nodes one and two.
Since I've numbered them one and two, the non-zeroes are going to come right at the start.
And then a whole lot, then this is all going to be zero after that.
Because those will be nodes three, four, or five, whatever that don't involve bar one.
So bar one just connects node one to node two.
Now, what do you think?
Well, let me put in the key quantity here.
This bar is at an angle.
It's at an angle theta.
So there's a theta.
Angle theta.
OK. And so that angle is going to enter these things.
In fact, here's what you get.
You get, I think if I put the one up there and the two down there, let's see.
What am I thinking now?
I'm saying if u 1 H is positive, that's going to stretch the bar.
That's a positive stretching.
So I'm expecting a positive u 1 H to give me, I'm expecting that sort of to come in with a plus sign.
Now suppose the bar is horizontal.
Suppose the bar is horizontal, then how much does the u 1 H stretch it?
It stretches it by the whole u 1 H, right?
If the bar was horizontal, so theta equals zero.
I'm just doing these, we got to sort out this theta angle stuff.
So here's my thing.
If the bar happens to be horizontal, then that stretching by u H 1, will completely stretch the bar.
If the bar happened to be, yeah yeah.
And of course, this way.
So that for a horizontal bar, I'll just be back to this step.
I'll have a one and a minus one u, oh yeah, remind me about that.
Why doesn't u vertical, for a horizontal bar like this, why does this one not stretch the bar?
You remember that from last time, that was the little bit of trig that we did when we were forced ourselves to stay linear.
So we dropped the second order correction, that would come from going this way.
Right?
I mean you must have noticed, like walking?
Suppose you want to walk from here to the end of the bar, OK?
Well, if somebody moves the end of the bar forward, you have to take those extra steps.
The bar really stretches.
But, if somebody moves the bar this way, then the extra bit of length is much less.
In fact, it's zero to first order.
This is like taking shortcuts when you walk across the courtyard.
So when the angle's theta, I'm only expecting a one and a minus one.
On the horizontal.
And zeroes on the vertical.
OK, now I'm ready to write it.
I think when the angle's theta, when the angle's theta, any theta, that was when the angle was zero, I think we get a cos theta.
Doesn't your instinct say that this is on the u_1, u horizontal 1.
And then the u vertical 1, tell me what these should be.
And then we'll make, what do you suppose is the entry, second non-zero, the one that corresponds to a vertical movement.
Here it would be, for a horizontal bar when theta is zero, I'm going to see a zero there.
But if the bar is at an angle like this, what am I going to see?
Everybody's going to get it right?
What do I put in there?
Sine theta.
What else could it be?
Right, OK, sin(theta).
And now what about the next guy, the other end of the bar?
u H 2 and u V 2, those are the other two non-zeroes.
What your guess for u 2 H, u H 2?
If I move this forward, What's the change in length of the bar?
What would your guess be that goes into there?
Say it again?
-cos(theta).
-cos(theta), right, yeah.
The movement of the other end, like if I move this guy a little bit to this side.
That will shorten the bar.
Forget about that one.
If I move this over, the bar becomes shorter.
And the cosine tells me the key number there, how much it becomes shorter.
If the bar was horizontal, the cos(theta) was one, it counts a hundred percent.
If the bar is vertical, and I move it horizontally, it comes zero percent.
Because the linearity says there was no first order change.
And now tell me the final non-zero entry.
And I see I didn't leave much room for all the zeroes.
OK, what's the u 2 V entry?
-sin(theta), of course.
And then come all the zeroes, four whatever other joints are not involved with bar one.
So let me, maybe to make this picture best, I should move that over to where it belongs.
Now, if I add up along the bar, add up the four numbers there, what do I get?
Zero.
You expected that, right?
In fact, if I add just that and that I get zero.
If I add that and that I get zero.
Just the way I got zero here.
In the incidence matrices.
The column of all ones is certainly going to solve Au=0.
Unless the supports remove those, of course.
If the supports don't allow all ones because some have to stay at zero, then I could have a stable truss.
OK, that's a typical bar.
A typical row.
That's a typical row.
OK, and now maybe while I'm on the same subject, what is the size, what is this thing look like now?
This is in A.
This is in a matrix A, and now I want to ask you before I even come back to all this stuff, what about in A transpose A?
In A transpose A, A transpose C, it the whole deal.
The element for the little matrix, the element matrix, can I call it that?
Or the one bar matrix, call it the one bar matrix.
Will be, is what?
So I want this, it would be typical.
c_1, row one.
Transpose row one, that's the typical guy.
And how many non-zeroes in that?
Multiplying a row.
Sorry, multiplying a column that has four non-zeroes times a row that has four non-zeroes times a number, which is just fine.
How many non-zeroes are going to sit in this element matrix, this one-bar matrix?
16.
16 non-zeroes.
And they're going to be, I have cosine, sine, minus cosine, minus sine, multiplying cosine, sine, minus cosine, minus sine.
And all multiplied by c_1.
So that's the matrix.
And you see what it looks like.
c squared, cs, so on.
16 guys.
So we have four squared as our element matrix, where here we had two squared.
And in finite elements, when you get to elasticity, and you've got triangles, you've got triangular elements, then there are three nodes involved.
So you're up to higher numbers.
But this gives you the idea.
And remember that this four by four, the way I've done it, the way I've numbered it, one, two, happens to sit up in the upper left corner.
Of A transpose C A.
But can you sort of imagine how the code would be written?
The code would be written, take each bar, and what do I have to know about the bar?
Just imagine a code that would do trusses.
Actually, the final problem that I'm not assigning in this section says what would the code be like?
Can we just have a think about what the code would look like if we were to write it.
What would the input have to be?
For each bar, what input do I need?
For this bar, I need to know, and for the whole truss.
What do I have to tell, what's the information that I need for the whole truss?
I have to know the positions of all of the joints, right?
So I'd have to know the coordinates of that, x y, the coordinates of this one, x_1, y_1.
For joint one, x_2, y_2.
So I'd have to have a little list of what would that be?
m by two?
I don't know.
n. n by two.
I have n, what do I have now?
Think of what information do I have to report about this truss?
I guess I have N, capital N, joints.
And I need two coordinates, x, y for each position.
So that's N by two this.
OK, and then for every bar, what do I need to tell it?
What do I need to put in the code for a typical bar?
I certainly have to put in the c for that bar.
And what else do I need to know?
I need to know which joints it's connected.
Right?
I have to tell the system that this bar is between two and one, one and two.
I have to tell it which pair.
So I guess I have a list of m bars, and for each bar I must tell the system the two node numbers, and the c, the stiffness.
The constant for Hooke's Law.
Right, do you see this picture?
Just sort of visualizing, creating a code here.
And then the code would do all this, oh, have I given enough information to find theta?
Or do I have to import theta also?
No.
I told you the positions, so it'll figure out cos(theta) theta and sin(theta).
It actually won't figure out theta, that's always a dumb thing to do find the actual angle.
cos(theta) and sin(theta) is the quantities we want.
So given that position, x, y and this position x_2, y_2, it would know cos(theta) and sin(theta).
And having drawn his picture allows me to make once more the key point about small displacements.
What's the angle of the bar after it's moved?
After it's displaced?
It was theta before it was displaced, and the angle after is theta.
To first order, the angle doesn't change.
Because these are little tiny movements of the ends.
I've drawn them much bigger than they should be drawn.
They're little, tiny movements of the ends so that the angle is not significantly changed.
Otherwise we're into geometric nonlinearity and that stuff, that makes the problem much, much harder.
OK, are you seeing sort of the picture?
I guess what I haven't completely, I've really depended more on your intuition than on a calculation to say that these are the four non-zeroes.
What did I ask you to do?
I asked you to check that that was right in the extreme cases, like if theta is zero, the bar is horizontal, then we just have a one, zero minus one, zero vertical isn't happening.
If the bar is vertical so that the angle is 90 degrees then we would have a zero, one, zerom minus one, everything's vertical.
And the book draws a little picture, and computes.
Computes delta l from these four small movements.
And takes the leading term and sure enough it produces that row of the matrix.
Gosh, I talk real fast.
But do you think you could now create the stiffness?
If you had a real truss, you could create the matrix A for it?
C is simple, it's given to you.
You could create A transpose C A?
You might just want to write the command as A'*C*A or something.
And let MATLAB do the thinking.
But I wanted to just see what these, how this four by four piece appears in this product from each bar.
The 16 non-zeroes will appear in different positions and you told the code what those positions are.
You had to give the code a local to global picture.
This is the local picture.
Watch one bar.
Then it has to fit in this big n by n matrix, and that means you have to know what joints was that bar connecting.
So which positions do these 16 non-zeroes assemble into?
That's some time devoted to a job that I actually don't plan to require you to do.
Creating this truss problem.
What I think is kind of more fun and that's these homework problems would deal with it, is part two of the lecture going back to mechanisms.
And now thinking about more complicated trusses.
We now in principle could find the solutions to Au=0 because we now have constructed A, and we could get MATLAB to do the work or Python or whoever.
But can I go to part two now and draw a truss and ask you about the mechanisms?
Let's see.
I guess somewhere in the problem set, but not one of the assigned ones is, start with those six bars.
And six joints, so these are six joints.
And OK, as it stands how many, that's a good question.
As it stands, what's the shape of the matrix A?
How many rows has it got?
So as it stands, so I'll call it A_0, for no supports have been added.
A_0, just the full matrix.
Is what shape?
six by 12.
Good.
Six by 12 is six by 12.
OK, of course it's not stable.
We know that.
We haven't supported anything.
So in a typical case, how many solutions to A, so I'm going to ask you how many solutions to Au=0?
And what's your guess?
Six.
Got six equations, we've got 12 u's, 12-6, so this is going to be 12-6.
And of course six solutions to that equation.
It's what I would expect.
There could be, it could be possible that the six equations are not independent.
If they really dropped to five then this would bump up to seven, I don't think it's going to happen.
Now, can you describe those six solutions, not with numbers, just with, tell me.
I hope you can, because I can't right now.
OK, three of them we know.
So with no supports at all, what are three rigid motions?
And what are they?
The whole truss could move to the right, the whole hexagon.
It could all move up, it could all rotate about one point.
All three of those would be movements, displacements that don't stretch anything.
OK, three rigid motions.
Across, up, and rotate.
OK, and I can get rid of those by supporting some nodes.
But let me see, I don't know what's going to happen.
When I describe this topic as the fun one in 18.085, it's more fun for you than for me.
Because I draw something like that and I start worrying can I think of three, how many mechanisms to look for?
Three.
That's a big number.
I bet you I can find one but you guys have got to, alright, tell me some mechanisms.
Let me try to draw them.
What would be one mechanism?
Collapses, yeah, somehow.
How shall I make it collapse?
Can squeeze in, yeah, maybe that's the first one.
This guy comes in, this guy, what does that do?
I've got 15 minutes here, I could pull that board down and draw another one.
What is this one here?
Let's see, if that comes in, do these guys have to go up a bit?
Yeah, because that angle's no 90 degrees.
So we've got a first order change in this.
So this comes in a little, this goes up a little, this guy maybe stays straight.
Would you go for this, I mean please say yes?
Something happens there, right?
These things go in and those go out.
Could you create the, I mean is this was A equal side, regular hexagon, you could put in all the numbers for all 12, I would be looking for 12 numbers.
Six joints and they each have two u's, so that wouldn't be so simple but you could do it.
So that would be one.
Looking for number two.
What would be another one?
So sort of squeezing in like whatever.
What do you think?
Any others?
Maybe that's possible.
Maybe I just look at it, you think that would work?
We could hope those were independent, but I wouldn't put my life on it.
I have this squeeze in, this squeeze in and this squeeze in, I would worry a little bit.
I can see another one
Half, for--
Fold it in half
Along, I guess that sort of gets into 3-D, but
Yeah, we've got to stay in the plane, right.
I have instead of what,
[INAUDIBLE]
Squeeze that out.
Yeah.
Good.
Number two.
And then I can see, here's one, here's an easy one to think of.
Leave these three alone and just rotate these guys.
Bring this down, right?
Just bring these three vertically down, or something.
You see why it's sort of, did you like that one alright?
It seems simple to me, looking at it, just leave these guys in rotation, let this turn down.
This turn down, let's say, and this go down.
Maybe these would all drop by the same amount.
Maybe.
So anyway, whatever.
Let's put some supports on them.
And get these numbers down.
So let's support, as usual, the bottom guy.
OK, so different problem now I won't call that A_0, I'll call it A. I'll ask myself, is it stable or unstable?
The matrix is now six by what?
Eight, because I've taken away, I have four reaction forces, two at each support, horizontal and vertical.
I've got four free nodes.
And six by eight.
Let me put in, so six by eight, what am I expecting now?
Any rigid motions?
No, no rigid motions now.
How many solutions am I expecting?
Two, I think.
Probably two.
How many mechanisms?
Well, no rigid motions.
So probably two mechanisms.
Now, can we find two mechanisms?
Alright, this is like more reasonable.
We can see whether whether we get two mechanisms and whether they're really different.
OK, what are the mechanisms now?
These guys are fixed.
So forget my little sketch here, and think again.
What do you see?
Alright, let's have one mechanism.
What would one mechanism be now?
There have to be two.
What do you think?
Sit on it.
Alright, bring these guys down, and then these guys will go out, is that it?
OK, so a number, mechanism number one, sit on truss.
OK. Alright.
Now, I don't know what number two is, that's why I'm taking time
[INAUDIBLE]
Yeah.
Or could we do this, could we bring these guys in and let's go up?
It's the same thing.
OK, so squeeze truss.
Squeeze sides.
Is that the same thing that I had, number one the same as two.
Oh jeez, ok. Is that what everybody is agreeing with this?
OK.
So I didn't get, my number two was no good.
OK. What's a better number two?
Hold an edge?
Like that one.
I'm just doing what you say, I'm not
[INAUDIBLE] Just this guy, rotates like so.
And this guy will rotate, and this guy.
That looks pretty good to me.
Good, is that correct?
Say that one again?
So the first one was when these two came down and these went out.
Right, OK. And now your suggestion is, you picked on this guy and held it fixed.
And then this one came down a little bit.
It'll, of course, how will it move?
It will move perpendicular, right?
Small movement.
The bar is not going to change length, that's the whole point, right?
The bar is not changing length.
So the movement must be, it must be a simple rotation.
Around here.
OK.
Right, and of course again you might say well, the bar really did change length because that's not quite the same as that.
But then again that's my second order basis.
So that one came down, and what did this one do?
Came down the same.
OK, and this one it also moves.
What is that?
OK, so what am I going to call this one?
Fixed one.
Yeah.
Fixed one node.
And that makes sense.
Fixed one joint, yeah.
And then, and rotate the rest.
I think that would be possible.
Yep.
OK, a small prize for anybody who, maybe handwritten, a picture of two really nice mechanisms.
Somehow this one seems a little um-symmetric in a problem that so symmetric, so I would guess that somewhere along the line we could find a kind of more some symmetric one.
But I don't see what it is right now.
Can the whole thing rotate a little?
Could that rotate, could the whole thing rotate?
Yeah, maybe it could.
This guy would go up, maybe that's possible.
That's somehow got everybody into the action.
So I'll put or rotate.
OK, so you see what questions you get into.
May I just draw a different truss?
So those homework questions are other trusses.
Here's one that I drew in the book itself.
Yeah, may I draw this, I called it a treehouse.
OK, so I have, so here's one that's actually in the book.
And it's got a couple of bars going up, and one over.
So that's the start.
Then it's got a diagonal and that one.
And then here comes the treehouse.
OK, right.
Well, just to get, let's again get the count right.
So what's the matrix A for this treehouse?
A is how many by how many?
How many bars are you seeing here?
One, two, three, four, five, six, seven, eight, eight bars.
And how many unknown displacements?
Ten.
We got five joints that are not supported, and each one has two unknowns.
So A is eight by ten.
So I expect two mechanisms.
OK, so again I'm looking for two mechanisms.
OK, what's one?
[INAUDIBLE]
It's what?
This guy just falls, right.
This looks unfortunately very much like the treehouses that I built for my kids.
Well, so linear algebra sentenced them to fall, right?
OK, that's one.
I probably propped it up with one more bar, but of course that wouldn't be enough, because it's got two mechanisms, so if I make it nine by ten I haven't saved the kids.
OK, with eight by ten, what's the other mechanisms?
The whole thing could turn the, nothing preventing turning here.
They can't move but they could turn.
So the whole thing could go over, right, the whole thing could just, that would be a horizontal movement of all five nodes.
The horizontal of all five nodes.
And again, slightly downwards, but that's a second order effect.
OK, so that's the second truss.
OK.
So this is really like practice for discrete problems, for the problems of plane elasticity.
And the point is that there are two unknowns for each point.
If we have differential equations.
So the differential equations of plane elasticity are not really simple.
They're not really simple.
And 3-D elasticity even more.
Because the points are physical points and they can move three ways, and it gets quite interesting.
And those are the major problems of computational mechanics.
OK, let's say, holiday time and I'll see you next Wednesday for Chapter 3.
Which moves to partial differential equations.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so.
This is a big day.
Part One of the course is completed, and I have your quizzes for you, and that was a very successful result, I'm very pleased.
I hope you are, too.
Quiz average of 85, that's on the first part of the course.
And then the second part so this is, Chapter 3 now.
Starts in one dimension with an equation of a type that we've already seen a little bit.
So there's some more things to say about the equation and the framework, but then we get to make a start on the finite element approach to solving it.
We could of course in 1-D finite differences are probably the way to go, actually.
In one dimension the special success of finite elements doesn't really show up that much because finite elements have been, I mean one great reason for their success is that they handle different geometries.
They're flexible; you could have regions in the plane, three dimensional bodies of different shapes.
Finite differences doesn't really know what to do on a curved boundary in in 2- or 3-D. Finite elements cope much better.
So, we'll make a start today, more Friday on one dimensional finite elements and then, a couple of weeks later will be the real thing, 2-D and 3-D. OK, so, ready to go on Chapter 3?
So, that's our equation and everybody sees right away what's the framework that that's A transpose in some way; this is A transpose C A, but what's new of course is that we're dealing with functions, not vectors.
So we're dealing with, you could say, operators, not matrices.
And nevertheless, the big picture is still as it was.
So let me take u(x) to be the displacements again.
So I'm thinking more of mechanics than electronics here.
Displacements, and then we have du, the e(x) will be du/dx, that'll be the stretching, the elongation and, of course at that step you already see the big new item, the fact that the A, the one that gets us from u to du/dx, instead of being a difference matrix which it has been, our matrix A is now a derivative.
A is d/dx.
So maybe I'll just take out that arrow.
So A is d/dx.
OK, but if we dealt OK with difference matrices, we're going to deal OK with derivatives.
Then, of course, this is the C part, that produces w(x).
And it's a multiplication by this possibly varying, possibly jumping, stiffness constancy of x.
So w(x) is c(x) c(x), that's our old w=Ce, this is Hooke's Law.
I'll put Hooke's Law, but that's, or who's ever law it is.
It's like a diagonal matrix; I hope you see that it's like a diagonal matrix.
This function u is kind of like of a vector but a continuum vector instead of just a fixed, finite number of values.
Then at each value we used to multiply by c_i, now our values are continuous with x, so we multiply by c(x).
And then you're going to expect that, going up here, there's going to be an A transpose w f, and of course that A transpose we have to identify and that's the first point of the lecture, really.
To identify what is A transpose.
What do I mean by A transpose?
And I've got to say right away that I'm a little, the notation, writing a transpose of a derivative is like, that's not legal.
Because we think of the transpose of a matrix; you sort of flip it over the main diagonal, but obviously it's got to be something more to it than that.
And so that's a central math part of this lecture is what's really going on when you transpose?
Because then we can copy what's going on and it's quite important to get it.
Because the transpose, well, other notations and other words for it would be notation might be a star.
Star would be way more common than transpose, I'll just stay with transpose because I want to keep pressing the parallel with A transpose C A.
And the name for it would be the adjoint.
And the adjoint method, and adjoint operator, those appear a lot.
And you'll see them up here in finite elements.
So this is a good thing to catch on to.
Why?
Why should the transpose or the adjoint of the derivative be minus the derivative?
And by the way, just while we're fixing this, this is a key factor which is certainly, we have a very strong hint from center difference, right?
If I think of derivatives, if I associate them with differences, the center difference matrix, so the a matrix may be centered, would be.
Just to remind us, it's a center difference has onws and minus one, one, zeroes on the diagonal, right?
Minus one, one.
Takes that difference at every row.
Except possibly boundary rows.
And of course as soon as you look at that matrix you see, yeah, it's anti symmetric.
It's an anti symmetric matrix.
So a transpose is minus a for center differences and therefore we're not so surprised to see a minus sign up here when we go to the continuous case, the derivative.
But, we still have to say what it means.
So that's what I'll do next, OK?
So this is a good thing to know.
And I was just going to comment, what would be the transpose of the second derivative?
I won't even write this down.
If the derivative transpose sort of flips its sign to minus, what would you guess for this x transpose of second derivative, our more familiar d second by dx squared?
Well we'll have two minus signs.
So it'll come out fine.
So second derivatives, even order derivatives are sort of like symmetric guys.
Odd order derivatives, first and third and fifth derivatives, well, God forbid we ever meet a fifth derivative, but first derivative anyway, is anti symmetric.
Except for boundary conditions.
So I really have to emphasize that the boundary conditions come in.
And you'll see them come in.
They have to come in.
OK, so what meaning can I assign to the transpose, or what was the real thing happening when we flipped the matrix across its diagonal?
I claim that we really define the transpose by this rule.
By we know what inner products are.
I'll do vectors first, we know about inner products, dot products, we know what the dot product of two vectors is.
So, this is the transpose of A.
How am I going to define the transpose of A?
Well, I look at the dot product of Au with w. I'll use a dot here for once, I may erase it and replace it.
If at the dot product of Au with w, then that equals for all u and w, all vectors u and w, that equals the dot product of u with something.
Because u is coming.
If I write out what the dot product is, I see u_1 multiplies something, u_2 multiplies something.
And what goes in that little space?
This is just an identity.
I mean, it's like, you'll say no big deal.
But I'm saying there is at least a small deal.
OK.
So if I write it this way, you'll tell me right away this should be the same as u transpose times something.
And again, so I'm asking for the same something on both lines.
What is that something?
A transpose w. Whatever A transpose is, it's the matrix that makes this right.
That's really my message.
That A transpose is the reason we flipped the matrix across the diagonal, is that it makes that equation correct.
And I'm writing the same thing here.
OK.
So again, if we knew what dot products were, what inner product of vectors were, then A transpose is the matrix that makes this identity correct.
And of course if you write it all out in terms of i, j every component, you find it is correct.
So that defines the transpose of a matrix.
And of course it coincides with flipping across the diagonal.
Now, how about the transpose of a derivative.
OK, so I'm going to follow the same rule.
Here A is now going to be the derivative, and A transpose is going to be whatever it takes to make this true.
But what do I mean?
Now I have functions, so I have to think again, what do I mean by the inner product, the dot product?
So for this to make sense I need to say, and it's a very important thing anyway, and it's the right natural choice, I need to say the dot product, or the inner product is a better word, of functions.
Of two functions.
A, e(x), and w. if I have two functions, what do I mean by their inner products?
Well, really I just think back what did we mean in the finite dimensional case, I multiplied each e by a w, each component of e by w, and I added, so what am I going to do here?
Maybe my notation should be parentheses with a comma would be better than a dot, for function.
So I have a function.
I'm in function space now.
We moved out of our n, today, into function space.
Our vectors have become functions.
And now what's the dot product of two vectors?
Well, what am I going to do?
I'm going to do what I have to do.
I'm going to multiply each e by its corresponding w, and now they depend on this continuous variable x, so that's e(x) times w(x).
And what do I do now?
Integrate.
Here, I added e_i times w_i, of course.
Over here I have functions.
I integrate dx.
Over whatever the region of the problem is.
And then our example's in 1-D, be zero to one.
If these are functions of two variables I'd be integrating over some 2-D region, but we're in 1-D today.
OK, so you see that I'm prepared to say this now makes sense.
I now want to say, I'm going to let A be the derivative, and I'm going to figure out what A transpose has to be.
So if A is the derivative, so now is this key step.
y is the transpose x, OK?
So I look at the derivative, du/dx, with w, so that's this integral, zero to one of du/dx*w(x)dx, so that's my left side.
Now I want to get u by itself.
I want to get the dot product, so I want to get another integral here that has u(x) by itself.
Times something, and that something is what I'm looking for.
That something will be A transpose w. Right?
Do you see what I'm doing?
This is is the dot product, this Auw, so I've written out what a u in a product with w is.
And now I want to get u out by itself and what it multiplies here will be the a transpose w, and my rule will be extended to the function case and I'll be ready to go.
Now do you recognize, this is a basic calculus step, what rule of calculus am I going to use?
We're back to 18.01.
I have the integral of a derivative times w, and what do I want to do?
I want to get the derivative off of u.
What happens?
What's it called?
Integration by part.
Very important thing.
Very important.
If you miss it's important in calculus.
It gets sometimes introduced as a rule, or a trick to find some goofy integral, but it's really the real thing.
So what is integration by parts?
What's the rule?
You take the derivative off of u, you put it on to the other one just what we hope for, and then you also have to remember that there is a minus.
Integration by parts has a minus.
And usually you'd see it out there but here I've left more room for it there.
So I have identified now, A transpose w. A transpose w has, if this is Au, in a product with w, then this is u in a product with A transpose w, it had to be what was.
And so that one integration by parts brought out a minus sign.
If I was looking at second derivatives there would probably be somewhere two integration by parts; I'd have minus twice, I'd be back to plus.
And you're going to ask about boundary conditions.
And you're right to ask about boundary conditions.
I even circled that, because that is so important.
So what we've done so far is to get the interior of the interval right.
Between zero and one, if A is the derivative, then A transpose is minus the derivative.
That's all we've done.
We have not got the boundary conditions yet.
And we can't go on without that.
OK, so I'm ready now to say something about boundary conditions.
And it will bring up this square versus rectangular also, so we're getting the rules straight before we tackle finite elements.
OK, let me take an example of a matrix and its transpose.
Just so you see how boundary conditions.
Suppose I have a free fixed problem.
Suppose I have a free fixed line of springs.
What's the matrix A for that?
Well - question?
Yes
[INAUDIBLE]
Yeah.
that's Yes.
When I learned it, it was also that stupid trick.
So you would like me to put plus, can I put plus, whatever.
What do you want me to call that?
An integrated term?
It would be, yeah.
I even remember what it is.
As you do better than me.
u times w at the, is that good?
Yeah, I think.
So it's really this part that I'm now coming to.
It's the boundary part that I'm now coming to.
And let me say, so I'm glad you asked that question because I made it seem unimportant, where that's not true at all.
The boundary condition is part of the definition of A, and part of the definition of A transpose.
Just the way I'm about to say free fixed, I had to tell you that for you to know what a was.
Until I tell you the boundary condition, you don't know what the boundary rows are.
You only know the inside of the matrix.
Or one possible inside.
So I'm thinking my inside is going to be minus one, one, minus one, one.
So on, as given we find the differences.
Minus one, one.
But.
Oh no, let's see.
So I'm doing free fixed.
Is that right?
Am I doing free fixed?
OK, so am I taking free at the left end?
Yes.
Alright, so if I'm free at the left and fixed at the right end, what's my A?
We're getting better at this, right?
Minus one, one.
Minus one, one.
Minus one, one.
Minus one, and the one here gets chopped off.
You could say if you want the fifth row of A_0, remembering A_0 as the hint on the quiz, where it had five rows for the full thing, free free.
And then when an n got fixed, the fifth column got removed, and that's my free fixed matrix.
At the left hand end, at the zero end, it's got the difference in there.
Difference, difference, difference.
But here at the right hand end, it's the fixing, the setting u, whatever it would be.
u_5 to zero, or maybe it's u_4, because it's like one, two, three.
Setting u_4 to zero knocked that out.
OK. All I want to do is transpose that.
And you'll see something that we maybe didn't notice before.
So I transpose it, that's minus one, one becomes a column, minus one, one becomes a column.
Minus one, one becomes a column.
Minus one, all there is.
That row becomes, so this was, so, have I got it right?
Yes.
What's happened?
A transpose, what are the boundary conditions going with A transpose?
The boundary conditions that went with a were, let me say first, what were the boundary conditions with A?
Those are going to be boundary conditions on u.
So A has boundary conditions on u.
And A transpose has boundary conditions on w. Because A transpose acts on w, and A acts on u.
So there was no choice.
So now what was the boundary condition here?
The boundary condition was u_4=0, right?
That was what I meant by that guy getting fixed.
Now, and no boundary condition at u_0, it was free.
Now, what are the boundary conditions that go with A transpose?
And remember, A transpose is multiplying w. I'm going to put w here, so what are the boundary conditions that go with A transpose?
This thing, nothing got knocked off.
The boundary condition came up here for A transpose, the battery condition was w_0=0.
That got knocked out, w_0 to be zero.
No surprise.
Free fixed, this is the free end at the left, this is the fixed end at the right.
Did you ever notice that the matrix does it for you?
I mean, when you transpose that matrix it just automatically built in the correct boundary conditions on w by, you started with the conditions on u, you transpose the matrix and you've discovered what the boundary conditions on w are.
And I'm going to do the same for the continuous problem.
I'm going to do the same for the continuous problem, so the continuous free fixed.
OK, what's the boundary condition on u?
If it's free fixed, I just want you to repeat this, on the integral zero to one for functions u(x), w(x) instead of for vectors.
What's the boundary condition on u, if I have a free fixed problem?
u(1)=0.
u over one equals zero.
So this is the boundary conditions that goes with A in the free fixed case.
And this is part of A.
That is part of A. I don't know what a is until I know its boundary condition.
Just the way I don't know what this matrix is.
It could have been A_0, it could have lost one column, it could have lost two columns, whatever.
I don't know until I've told you the boundary condition on u.
And then transposing is going to tell me, automatically, without any further input, the boundary condition that goes on the adjoint.
So what's the boundary condition on w that goes as part of A transpose?
Well, you're going to tell me.
Tell me.
w(0) should be zero.
It came out automatically, naturally.
This is a big distinction between boundary conditions.
I would call that an essential boundary condition.
I had to start with it, I had to decide on that.
And then this, I call a natural boundary condition.
Or there are even two guys' names, which are associated with these two types.
So maybe a first chance to just mention these names.
Because you'll often see, reading some paper, maybe a little on the mathematical side, you'll see that word used.
The guy's name with this sort of boundary condition is a French name.
Not so easy to say, Dirichlet.
I'll say it more often in the future.
Anyway, I would call that a Dirichlet condition and you would say it's a fixed boundary condition.
And if you were doing heat flow you would say it's a fixed temperature.
Whatever.
Fixed, is really the word to remember there.
OK, and then I guess I better give Germany a shot here, too.
So the boundary condition is associated with the name of Neumann.
So if I said a Dirichlet problem, a total Dirichlet problem, I would be speaking about fixed fixed.
And if I spoke about a Neumann problem I would be talking about free free.
And this problem is Dirichlet at one end, Neumann at the other.
Anyway, so essential and natural.
And now of course I'm hoping that that's going to make this this boundary term go away.
OK, now I'm paying attention to this thing that you made me write.
uw.
OK, what happens there?
uw?
Oh, right this isn't bad because it shows that there's a boundary condition, I've got some little deals going.
But do you see that that becomes zero?
Why isn't zero at the top end, at one?
When I take u times w at one, why do I get zero?
Because u(1) is zero.
Good.
And it's the bottom end.
When I take uw at the other boundary, why do I get zero?
Because of w. You see that w was needed.
That w(0) was needed because there was no controlling u(0).
I had no control of u at the left hand end, because it was free.
So the control has to come from w. And so w naturally had to be zero, because I wasn't controlling u at that left hand, free end.
So one way or the other, the integration by parts is the key.
So that said, what I critically wanted to say about transposing, taking the adjoint, except I was just going to add a comment about this square A versus rectangular.
And this was a case of square, right?
This was a case, this free fixed case, this example I happened to pick was squared.
A_0, the free-free guy that was a hint on the quiz, was rectangular.
The fixed fixed, which was also on the quiz, was also rectangular.
It was what, four by three or something.
This A is four by four.
And what is especially nice when it's squared?
If our problem happens to give a square matrix, in the trust case if the number of displacement unknowns happens to equal the number of bars, so m equals n, I have a square matrix A.
And this guy's invertible, so it's all good.
Oh, that may be the whole point.
That if it's rectangular I wouldn't talk about its inverse.
But this is a square matrix, so A itself has an inverse.
Instead of having, as I usually have, to deal with A transpose C A all at once, let me put this comment because it's just a small one up here.
Right under these words.
Square versus rectangular.
Square A, and let's say invertible, otherwise we're in the unstable K, so you know what I mean, in the network problems the number of nodes matches the number of edges?
In the spring problem we have free fixed situations.
Anyway, A comes out square.
Whatever the application.
If it comes out square, what is especially good?
It comes out square, what's especially good is that it has an inverse.
So that in this square case, this K inverse is A transpose C A inverse, can be split.
This allows us to separate, to do what you better not do otherwise.
In other words, we are three steps, which usually mash together and we can't separate them and we have to deal with the whole matrix at once.
In this square case, they do separate.
And so that's worth noticing.
It means that we can solve backwards, we can solve these three one at a time.
The inverses can be done separately.
When A and A transpose are square, then from this equation I can find w. From knowing w I can find e, just by inverting C. By knowing e I can find u, just by inverting A.
You see the three steps?
You could invert that, and then you can invert the middle step, and you can invert A.
And you've got u.
So the square case is worth noticing.
It special enough that in this case we would have an easy problem.
And this case is called, for trusses and mechanics, there's a name for this.
And unfortunately it's a little long.
Statically, that's not the key word, determinant is the key word.
Statically determinate, that goes with squares.
And you can guess what the rectangular matrix a, what word would I use, what's the opposite of determinate?
It's got to be indeterminate.
Rectangular a will be indeterminate.
And all that is referring to is the fact that in the determinate case the forces determine the stresses.
You don't have to take that, we'd get all three together; mix them, invert, go backwards.
All that.
You just can do them one at the time in this determinate case.
And now I guess I'd better say, so here's the matrix case.
But now in this chapter I always have to do the continuous part.
So let me just stay with free fixed, and what is this balance equation?
So this is my force balance.
I didn't give it its moment but its moment has come now.
So the force balance equation is -dw/dx, because A transpose is minus a derivative equal f(x).
and my free boundary condition, my free n, gave me w of what was it?
The Neumann guy gave me w(0)=0.
And what's the point?
Do you see what I'm saying?
I'm saying that this free fixed is a beautiful example of determinate.
Square matrix A in the matrix case and the parallel and the continuous cases.
I can solve that for w(x).
I can solve directly for w(x).
Without involving, you see I didn't have to know c(x).
I hadn't even got that far.
I'm just going backwards now.
I can solve this, just the way I can invert that matrix.
Inverting the matrix here is the same as solving the equation.
You see I have a first order first derivative?
I mean, it's so trivial, right?
It's the equation you solved in the final problem of the quiz, where an f was a delta function.
It was simple because it was a square determinate problem with one condition on w. When both conditions are on u, then it's not square any more.
OK for that point?
Determinate versus indeterminate.
OK.
So that's sort of, and I could do examples.
Maybe I've asked you on the homework to take a particular f(x).
I hope it was a free fixed problem, if I was feeling good that day, because free fixed you'll be able to get w(x) right away.
If it was fixed-fixed then I apologize it's going to take you a little bit longer to get to w. To get to u. OK.
But this, of course- I just integrate.
Inverting a difference matrix is just integrating a function.
Good.
OK, so this lecture so far was the transition from vectors and matrices to functions and continuous problems.
And then, of course we're going to get deep into that because we got partial differential equations ahead.
But today let's stay in one dimension and introduce finite elements.
OK. Ready for finite elements, so that's now a major step.
Finite differences, maybe I'll mention this, probably in this afternoon's review session, where I'll just be open to homework problems.
I'll say something more about truss examples, and I might say something about finite differences for this.
But really, it's finite elements that get introduced right now.
So let me do that.
Finite elements and introducing them.
OK.
So the prep, the getting ready for finite elements is to get hold of something called the weak form of the equation.
So that's going to be a statement of, the finite elements aren't appearing yet.
Matrices are not appearing yet.
I'm talking about the differential equation.
But what do I mean by this weak form?
OK, let me just go for it directly.
You see the equation up there?
Let me copy it.
So here's the strong form.
The strong form is, you would say, the ordinary equation.
Strong form is what our equation is, minus d/dx of c(x), du/dx=f(x).
OK, that's the strong form.
That's the equation.
Now, how do I get to the weak form?
Let me just go to it directly and then over the next days we'll see why it's so natural and important.
If I go for it directly, what I do is this.
I multiply both sides of the equation by something I'll call a test function.
And I'll try to systematically use the letter v for the test function.
u will be the solution.
v isn't the solution, v is like any function that I test this equation this way.
I'm just multiplying both sides by the same thing, some v(x).
Any v(x).
We'll see if there are any limitations, OK?
And I integrate.
OK.
So you're going to say nope, no problem.
I integrate from zero to one.
Alright, this would be true for f. So now I'll erase the word strong form, because the strong form isn't on the board anymore.
It's the weak form now that we're looking at.
And this is for any, and I'll put "any" in quotes just because eventually I'll say a little more about this.
I'll write the equation this way.
And you might think, OK, if this has to hold for every v(x), I could let v(x) be concentrated in a little area.
And this would have to hold, then I could try another v(x), concentrated around other points.
You can maybe feel that if this holds for every v(x), then I can get back to the strong form.
If this holds for every v(x), then somehow that had better be the same as that.
Because if this was f(x+1) and this is f(x), then I wouldn't have the equality any more.
Should I just say that again?
It's just like, at this point it's just a feeling, that if this is true for every v(x), then that part had better equal that part.
That'll be my way back to the strong form.
It's a little bit like climbing a hill.
Going downhill was easy, I just multiplied by v and integrated.
Nobody objected to that.
I'm saying I'll be able to get back to the strong form with a little patience.
But I like the week form.
That's the whole point.
You've got to begin to like the week form.
If you begin to take it in and think OK. Now, why do I like it?
What am I going to do to that left side?
The right side's cool, right?
It looks good.
Left side does not look good to me.
When you see something like that, what do you think?
Today's lecture has already said what to think.
What should I do to make that look better?
I should, yep.
Integrate by parts.
If I integrate by parts, you see what I don't like about it as it is, is two derivatives are hitting u, and v is by itself.
And I want it to be symmetric.
I'm going to integrate this by parts, this is minus the derivative of something.
Times v. And when I integrate by parts, I'm going to have, it'll be an integral.
And can you integrate by parts now?
I mean, you probably haven't thought about integration by parts for a while.
Just think of it as taking the derivative off of this, so it leaves that by itself.
And putting it onto v, so it's dv/dx, and remembering the minus sign, but we have a minus sign so now it's coming up plus.
That's the weak form.
Can I put a circle around the weak form?
Well, that wasn't exactly a circle.
OK.
But that's the weak form.
For every v, this is, I could give you a physical interpretation but I won't do it just this minute.
This is going to hold for any v. That's the weak form.
OK. Good.
Now, why did I want to do that?
The person who reminds me about boundary conditions should remind me again.
That when I did this integration by parts, there should have been also.
What's the integrated part now, that has to be evaluated at zero and one?
This c, so it's that times that, right?
It's that c(x)du/dv times v(x).
Maybe minus.
Yeah, you're right.
Minus.
Good.
What do I want this to come out?
Zero, of course.
I don't want to think that the same.
Alright, so now I'm doing this free fixed problem still.
So what's the deal on the free fixed problem?
Well, let's see.
OK, I got the two ends and I want them to be zero.
OK, now at the free end, I'm not controlling v. I wasn't controlling u and I'm not going to be controlling its friend v. So this had to be zero.
So this part will be zero at the free end.
That boundary condition has just appeared again naturally.
I had to have it because I had no control over b.
And what about at the fixed end.
At the fixed end, which is that, at the free end.
Now, what's up at the fixed end?
What was the fixed end?
That's where u was zero.
I'm going to make v also zero.
So there's, when I said any v(x), I better put in with v=0 at the Dirichlet point, at fixed point, at fixed end.
I need that.
I need to know that v is zero at that end.
I had u=0.
Here's why I'm fine.
So I'm saying that any time I have a Dirichlet condition, a fixed condition that tells me u, I think of v and you'll begin to think of v as a little movement away from u. u is the solution.
Now, remind me, this was free fixed.
So the u might have been something like this.
I just draw that.
That's my u.
This guy was fixed, right?
By u.
Now, I'm thinking of v's as, the letter v is very fortunate because it stands for virtual displacement.
A virtual displacement is a little displacement away from u, but it has to satisfy the zero, the fixed condition that u satisfied.
In other words, the little virtual v can't move away from zero.
So I get this term is zero at the fixed end.
OK. that's the little five minute time out state to check the boundary condition part.
The net result is that that term's gone and I've got the weak format I've wanted.
OK, three minutes to start to tell you how to use the weak form.
So this is called Galerkin's method.
And it starts with the weak form.
So he's Russian.
Russia gets into the picture now.
We had France and Germany with the boundary conditions, now we've got Russia with this fundamental principle of how to turn a continuous problem into a discrete problem.
That's what Galerkin's idea does.
Instead of a function unknown I want to have n unknowns.
I want to get a discrete equation which will eventually be kKU=F.
So I'm going to get to an equation KU=F, but not by finite difference, right?
I could but, I'm not.
I'm doing it this weak Galerkin finite element way.
OK, so if I tell you the Galerkin idea then next time we bring in, we have libraries of finite elements.
But you have to get the principle straight.
So it's Galerkin's idea.
Galerkin's idea was was choose trial functions.
Let me call them call them t?
Have to get the names right.
Phi.
OK, the Greeks get a shot ok. Trial functions, phi_1(x) to phi_n(x).
OK, so that's a choice you make.
And we have a free choice.
And it's a fundamental choice for all of applied math here.
You choose some functions, and if you choose them well you get a great method, if you choose them badly you got a lousy method.
OK, so you choose trial functions, and now what's the idea going to be?
Your approximate U, approximate solution will be some combination of this.
So combinations of those, let me call the coefficients U's, because those are the unknowns.
Plus U_n*phi_n.
So those are the unknowns.
The n unknowns.
I'll even remove that for the moment.
You see, these are functions of x.
And these are numbers.
So our unknown, our n unknown numbers are the coefficients to be decided of the functions we chose.
OK., now I need n equations.
I've got n unknowns now, they're the unknown coefficients of these functions.
I need an equation so I get n equations by choose test functions.
V_1, V_2, up to V(x).
Each V will give me an equation.
So I'll have n equations at the end, I have n unknowns, I'll have a square matrix.
And that'll be a linear system.
I'll get to KU=F.
But do you see how I'm getting there?
I'm getting there by using the weak form, By using Galerkin's idea of picking some trial functions, and some test functions, and putting them into the weak form.
So Galerkin's idea is, take these functions and these functions.
And apply the weak form just to those guys.
Not to, the real weak form, the continuous weak form, was for a whole lot of V. We'll get n equations by picking n V's, and we'll get n unknowns by picking n phis.
So this method, this idea, was a hundred years older than finite elements.
The finite element idea was a particular choice of these guys, a particular choice of the phis and the V's as simple polynomials.
And you might think well, why didn't Galerkin try those first, maybe he did.
But key is that now with the computing power we now have compared to Galerkin, we can choose thousands of functions.
If we keep them simple.
So that's really what the finite element brought, finite element brought is.
Keep the functions as simple polynomials and take many of them.
Where Galerkin, who didn't have MATLAB, he probably didn't even have a desk computer, he used pencil and paper, he took one function.
Or maybe two.
I mean, that took him a day.
But we take thousands of functions, simple functions, and we'll see on Friday the steps that get us to KU=F.
So this is the prep for finite elements.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So.
I got the preparation for finite elements.
Again we're in one dimension, because that's where you can see first and most clearly how the system works.
So the system was, really to begin with the weak form that I introduced last time.
The Galerkin idea that I introduced just at the very end of last time, and that idea is that instead of the continuous differential equations where Galerkin's idea is how do you make it discrete, and he'll choose some trial functions.
Those are functions.
And some test function.
And I didn't get to say, but I'll say now, very often they're the same.
So very often the phis are the same as the v's.
And probably they will be in all my examples today.
And then what's new today is what choices would you make.
You have pretty wide choice, but there are some natural ones to start with.
And so that's where we are today.
And then also the other part of today is how do we get from all that preparation to the equations that we actually solve.
The KU=F; where does the K come from, where does the F come from?
The F is going to come, of course, somehow from this right hand side and the K is going to come from the left side.
OK, so that's the overall direction if you want to know how finite elements, work that's the key line there.
And then here I've recalled what the step was, here's our differential equation.
And there's its weak form.
This is the weak form.
And let me put this in.
Here I've reproduced what we reached last time, the weak form.
That was the strong form with its boundary conditions; now we get the weak form with its boundary conditions.
And the weak form involves u, so it's the boundary conditions on u, the fixed ones that get used in the weak form.
The free ones don't appear in the weak form, but by the magic of integration by parts, the free ones will sort of come out in a natural way.
But we have to build in the essential Dirichlet boundary conditions like that fixed right hand n. OK, and because I think of v as a little movement away from u, but my u's all fixed at the end, therefore v has to be fixed, too.
OK, so this is important.
but we haven't made it down to Earth yet.
We were doing a lot of ideas last time, which are the center of things, but now let's get down to Earth.
And down to Earth really means, what functions do we choose?
And then how do we get the equation.
So that's the job today, the principle job and this is of course what you would actually code up to run a finite element simulation.
You would make a decision on these functions.
And let's start with piecewise linear functions.
So a typical phi is maybe centered at a node, so it goes up and down.
So if that's node two, let's number these nodes, one, two, three, four, and five.
Well, that's five there but I'm going to have, let's see; what do I have here?
At a typical node two, my function is zero except in the integrals that touch node two.
So here's phi_2.
phi_2(x).
That's it's graph.
It comes along, it's piecewise linear up, it's peacewise linear back down, and it's zero again.
So that would be phi_2.
Then phi_1, there'll be a phi_1.
Like so, exactly similar.
The whole point is keep the system simple.
That's what the finite element idea is.
Use simple functions for the phis.
And I'm taking them also to be the test function v. Keep those simple.
Now, what about at the boundaries?
OK, well we've said our functions have to be, I'm doing free-fixed.
So fixed comes to zero there and all my functions will do that.
But this end is free, so watch this.
This is another what I'll call a half-hat.
If I call those hat functions is that OK?
It makes a nice short word.
So these are hat functions.
And then a fuller description would be piecewise linear, but hat functions is clear.
OK, now notice I'm sticking in here what I maybe could call a half-hat.
Because my v and my phis, my functions are not constrained at that left hand end.
That's free.
So there's a phi_1, and there's a phi_0.
And a phi_3, and a phi_4.
4 So altogether, I'm going to have five.
And I've started the numbering at zero, I guess it just happens.
So let's accept that.
So I'm going to have five functions.
And they will also be my five test functions.
So let me first think of them as the trial functions.
Phi.
So what was the point about trial functions?
The point about is my approximate, my finite element solution u(x) is going to be a combination of those.
Some u_0, times the phi_0 function up to u_4 times the phi_4 function.
And these are my unknowns.
u_0, u_1, those numbers.
I have five unknowns.
And I'm using hat functions.
So I should, really, why don't I draw a graph of u(x).
So I don't need the words piecewise linear.
You see, I have a kind of space of function.
My functions are all, my approximations are combinations of these fixed basis functions.
You could call these phis, trial functions, basis functions, you're always choosing in applied math.
A bunch of functions whose combinations are going to be, is what you're going to work with.
And here I'm making them hat functions.
OK, so what does a combination of those guys look like?
I'll even erase the word hat functions; we won't forget that.
So what would a combination of, here's the same integral, zero, one, with the same mesh, one, two, three, four guys inside.
What would, it just helps the visualization to see.
If I combine these guys, so those were despite the way it might look, that's one separate function.
Two, three, four and five.
And now suppose I take a combination.
What kind of a function do I get?
How would you describe this, a combination like this?
Of those five guys?
What will it look like?
Between every node it will be a straight line, right?
Because all these guys are straight, between those.
So a typical function will start at what?
This height will be u_0, because, well let me draw a few things.
OK, so I'll put u_0 a little higher because I want to end up down at zero.
So that might be the height u_0, u_1 might be pretty close.
u_2 might be coming down a bit.
Coming down a bit, coming down a bit.
And ending at zero.
So those were meant to be corners.
That wasn't a very good bit.
There, OK.
It wasn't meant to be, yeah.
So that height is u_0.
Now, notice why?
Why is that?
Why is the coefficient of phi_0 exactly the height, the displacement, at zero?
Because all the other phis are zero.
All the other phis are zero at this point.
Only this guy's coming in.
So we'll only see in this, at this point see, even phi_1 has dropped to zero.
So we're only going to see phi_0 times u_0.
So, I should have said, we take all these heights to be one.
Height is one.
Course it doesn't matter because we're multiplying by u's, but so let's settle.
They all have heights one.
And this is sort of a key part of the finite that element idea.
You see, Galerkin, when he created just two or three functions phi, he tried to follow the exact solution.
He had an idea in his mind what the solution to the problem will be.
And he wanted to take two or three functions that would get him close to it.
Here we choose the functions, they're in the finite element library before we even know what the equation is, or the boundary conditions.
These functions are like, the hat function choice.
And they have this beautiful sort of a connect to nodes.
In a way where in fact I got involved in finite elements in the first place just to understand what's the difference between finite elements and finite differences.
Because the finite elements, as you'll see, are associated with nodes.
phi_1 is the only guy that's not zero at node one.
And therefore, what will this height be?
What's that height?
So that maybe comes down a little.
What's this height?
u_1.
It's the coefficient of phi_1, because phi_1 is sitting there.
And all the other phis are zero there, so this height is really u_1.
And u_2, and u_3, and u_4, and u_5 is zero.
So that was u_1, u_2, and so on.
What I'm saying is, and we'll see it happen, is that this KU=F equation that we finally get to is going to look very like a finite difference equation.
But it's coming from this different direction.
And this way allows many more possibilities, gets things sort of more naturally right.
It less, with the finite difference equation, we had to go in there and decide what it should be.
With finite elements, our decision is just the phis.
Once we've decided the phis, Galerkin tells us what the equation is.
And we'll get to the KU=F, what it is.
But here I'm getting you to see what our approximate solution can look like.
And this beautiful fact that the coefficients in here have a physical meaning.
They're actually the displacements at the nodes for these simple functions.
OK. You got a picture of what the trial functions are, some people would think about the functions as these guys.
Other people might think of this picture, the combinations.
So those are the individual basis functions.
That's a typical combination of the typical one.
OK, so we're looking for an equation for these u's.
Five equations, of course.
Because we've got five u's.
So that's my final step here.
What are the five equations for the five u's.
And those are the equations that I'm going to call KU=F.
OK.
So here's now a critical moment.
Where do the equations come from?
Well, the equations come from the weak form.
So I take the weak form.
And in for u, for u here, I guess it's just there.
I put this.
I put capital U.
So so can I just copy, this is the weak form for before we've made it discrete.
Before we've chosen n phis.
OK, now let's choose the n phis, so now this'll be the weak form.
Again, the weak form with Galerkin.
After the decision.
So it'll be the integral, from zero to one of this c(x), whatever it is.
Times the u(x), times the dU/dx, right?
dU/dx, so what is dU/dx?
Oh, you have to pay attention.
This was a true solution.
Little u.
But now this is where I'm working.
I'm working with capital U.
So instead of the d little u/dx, it's d capital U/dx.
Maybe I'll put it in there.
d capital U/dx.
And then I'll put down here what it is.
What is it?
It's u_0*phi_0, can I use prime just to, or d phi_0/dx, whatever?
Can I use prime for derivative?
Just save a little space.
So this is the derivative of my guy.
u_1*phi_1'(x), up to whatever it was.
u_4*phi_4'(x).
That's what that term is.
And that multiplies dV/dx.
Where V is, here V is any test function.
Any test, function only required to have V(1)=0.
But now, I'm going discrete.
So instead of any test function, I'll use these five functions.
So I've got five functions.
The phis are the same as the V's, then.
V_1, V_2, V_3 and V_4.
Same guys.
So now I'll put in dV, can I say dV_i/dx?
And on the right hand side I have the integral from zero to one of F(x)*V_i(x)dx, i is zero, one, two, three, or four.
I'm testing against five V's.
So I have this equation for five different V's.
So i equals zero, one, two, three, four, gives me my five equations.
Here are my five unknowns.
This is my five by five system.
Let me just step back a minute so you see what happened there.
So what do you have to do?
You chose the basis functions, the phis and the V's.
Then you just plug into the weak form, you plug in dU/dx, is coming from there.
So this is dU/dx.
You have to plug dV/dx, you have to do the integrals.
But to do the integrals, that's something we didn't have in finite differences.
Finite elements involves doing the integrals, left side and right hand side.
OK, so and here we have five different integrals to do.
We have F(x) times each V. This will be, this number will be F_i.
So my f vector is going to be an F_0, F_1, down to F_4.
The five guys that I get from these five integrals.
Alright.
And the K matrix is sitting here somewhere.
That's the last thing, that's the final thing is to see what is the K matrix.
Which is coming.
This is somehow K's times U's are sitting here.
F's are sitting over there.
So it may be good to see the f first.
So do you see this now?
We made the choices, then what's our job?
Our next job is to do all the integrations.
Integrate my function against V. Let's make the natural first example, let F be one.
First example, let F be one.
All right.
So if F is one, I'm going to find the system KU=F.
So if I know F(x) is one, then I have everything I need to find these numbers.
OK, actually we can probably do those by, you can probably tell me what they are.
If F(x) is one, now.
So this is example one.
F(x) is a constant.
One we have solved before.
What's the integral of V_0?
Right, that's what I have to do.
What's the integral of this F(x) being one, I'm just asking, I just have V_0(x)?
The integral of V_0(x), and let me again draw V_0, which is phi_0.
It's a half hat and then it goes along at zero.
What's the integral of that function?
One, yeah.
How do I think about that integral?
It's the area of the triangle.
It's the area.
That's what an integral is, it's the area.
So the area is, I've got delta x there.
Right, delta x as the base.
One as the height, and you see the formula for the area of a triangle, it's got a half in there somewhere, right?
A half.
OK, can I factor out the delta x?
Because the delta x is going to come in.
I think there's a half there.
And then what about F_1?
What's the integral of this times V_1(x), the next V?
It's the area under this dashed function.
Which is now the basis 2 delta x, so I get a one.
Is that right?
I get a one, one, one, one.
OK, so that was obviously not too tough, right?
That was straightforward and notice something.
Even here; in fact, we see it here.
I don't know if you remember about that half.
Do you remember something about when we did finite differences and we had a free boundary?
And we lost an order of accuracy if we didn't do it right?
Do you remember that?
At the fixed boundary we were fine, but with finite differences that's a free boundary where I was using the matrix T. With one minus one at the top row.
I lost an order of accuracy.
Unless I made some change on the right hand side.
Look what's happening.
The finite element method is making the change for me on the right hand side.
So the finite element method is going to automatically keep the second order accuracy.
Keep the second order accuracy.
So that's a key point.
That these piecewise linear functions are associated with second order accuracy.
Later we'll move up to parabolas.
To cubics that will move up the order of accuracy in a nice way.
Where with finite differences we would have had to create new finite difference formulas.
Our minus one, two, minus one formula, that was good for second order accuracy.
Then we would have to figure out in the quiz and partly started it, what if there's a c(x) in there, what do you do?
Finite differences, is more thinking involved.
Finite elements is like just press the button.
Well, there's a little more to it than that of course, because it's taking a whole lecture.
But in the end it's more systematic, you could say.
So that's the F. Now are you ready for the K?
So this is the key part, OK?
So you have to can get this thing to simplify.
So what am I looking for here?
This whole left hand side should be K times U.
So I'm looking to see what multiplies, I'm looking to make sense out of this.
What's the first equation?
Right, so the first equation or the zeroth equation, I guess.
The zeroth equation, the one that'll run along and have this right hand side, the zeroth equation is the equation when i is zero.
It's the equation that comes from testing our weak form for V_0.
For that particular form.
Maybe I'll just start over on this board.
Then I can write a formula, but I'd rather you see how it comes.
So I'm looking at equation zero.
So take i=0.
So I have my left side is my integral, of c(x).
Times this combination that I wrote, U_0*phi_0' +...+u_4*phi_4'.
Times dV_0/dx*dx equal, and on the right side is where I got the F_0, which I already figured out to be delta x times a half.
It's the left side that I'm worrying about.
OK, you see what's happening here?
This is some matrix.
Its zeroth row is what we're finding.
Multiplying U_0, U_1, U_2, U_3, and U_4 equaling the F vector.
I'm supposed to be getting the first row of the matrix, the top row of the matrix, from the top V. OK, so let's just do these.
We've got integrals to do again.
Alright, what is dV_0/dx?
Do you see?
Let me just see?
What number is going in here?
What number is going in there?
Yeah, if we see that we're golden.
What number is going in there?
That's the thing that multiplies U_0 in the first row that means I should use V_0, so this is the point, this is K_00.
And what's its formula?
You realize I'm starting the count at zero because all these counts.
So what is K_00?
It's an integral.
Of what?
c(x), good.
Times this guy, because it's multiplying times this guy, V_0.
dx.
That's what you have to do.
That's what you have to do.
c(x) times phi', it's phi_0', that's what would sit there.
And maybe, well, let's figure that one, shall we?
I have to know c(x), right, that's part of the problem.
What would you like me to choose for c(x)?
One.
Thank you.
I'll choose one.
Let this be one.
Or I could make it capital C and you would see a capital C appearing everywhere, but let's make it one.
So what are we doing now?
What's our equation?
Our right hand side is one, our c(x) is one, our equation has reduced to -U''=1.
The first equation in the course.
So we're back to September the 3rd or whatever it was.
But doing it now by finite elements.
OK, so let c(x) be one and tell me what this integral is.
So c(x) is now, we're taking in our problem we're supposing it's one.
Let me just say suppose it's function.
Then we have lots of integrals to do involving that function.
And we might not do them exactly, that would be alright.
It's certainly totally OK to do the integrals approximately, because we're doing everything else approximately.
So we just have to be sure that we do the integrals with sufficient accuracy so that we don't lose accuracy in the integrals.
Of course, with a one we're going to do the integral exactly.
But if c(x) was some variable function, I wouldn't have to do it exactly, I would just have to do it with enough accuracy so that I don't lose extra accuracy beyond what I'm losing in the whole Galerkin approximation.
OK, ready for that number.
What number comes out of that?
phi_0', let's graph phi_0'.
And of course it's the same as V_0', so can I put a little graph here?
Here is zero to one, and I'm going to graph phi_0'.
So what's phi_0'?
Oh, it negative, isn't it?
My little graph isn't going to work.
I didn't even have room.
It's negative.
So I'll just write it in words.
It's the same as V_0', and what is it?
Tell me what it is, what's the derivative of that function?
It's what?
Negative one.
Wait a minute.
Yeah, it's going to have a certain value, yeah.
You can tell me what it is beyond that point real fast.
So it's something up to delta.
So what is the slope?
What's that slope there, of phi_0?
It's not negative one, because remember, what's the base here?
That's not the point one, I'm sorry.
All these were delta x's.
Those were just numbering the nodes.
But the actual length is scale is the delta x scale.
So now tell me what it is.
The derivative is negative one over delta x, right?
It dropped by one when it went across by delta x.
And this is only up to node one.
Up to delta x and then zero afterwards.
This is a key point.
That all our functions are local.
Our functions are local.
What does that mean?
You can tell me what am I going to get when I integrate, for example, when later on I might be integrating phi_1' against V_4'.
What's the answer?
This is the key point.
Later, when I'm looking for the one, four entry, when I'm looking there, I'm going to do an integral of phi_1'.
I'll erase for a moment and do this in my head.
When I integrate phi_1' against V_4'.
Maybe it's the fourth row.
And the first guy over, maybe it's this guy I'm doing.
Doesn't matter a whole lot.
Because the answer is, when I integrate phi_1' against V_4' just, it's nice to get the easy ones.
It's zero.
Why is it zero?
Why is the integral of phi_1' against V_4' zero?
Because these phis are local.
phi_1' is only non-zero here.
V_4' is only non-zero over here.
The two don't overlap.
Anywhere the one is not zero, the other is zero.
So that's a zero there.
In fact, our overlaps, I'm just sort of looking ahead here.
Our overlaps, a phi overlaps itself, of course.
And its right hand neighbor and its left hand neighbor.
But nobody two or three or more away.
I think our K, all our integrals are going to be zero outside, we'll have another tri-diagonal matrix.
We're going to have zeroes all here.
And we'll only have entries where phi against V when they're either the same or just differ by one.
So we'll only have three diagonals.
OK, we were about to find out what that number is.
So the slope of this is minus one over delta x, and that's - I'm sorry, let me go back to zero, zero.
OK.
This is what we're keeping our fingers crossed for.
What's that number?
So I have this thing, actually is it just squared?
And that's the slope.
And then the phi and the V I'm choosing the same, so that's the slope again.
I think I'm just getting one over delta x squared for that times that times the one.
So what's K_00?
One over delta x.
Where'd the delta x come from?
Because we're only integrating over, it looks like zero to one but they're all zero.
We're really only integrating, the only reality was out to node one.
Out to delta x.
You see that the number there, the number here on the diagonal is one over delta x. OK, how about doing K_11 for me?
So again, now these guys will be the same guys.
It's a square.
No it's the integral phi_1' against V_1', they're the same.
And what is phi_1', which is the same as V_1'?
What's the derivative now?
It's, ah.
What's the slope of this function?
It goes up and goes back down, right?
I have a plus part, so the slope going up is the one over delta x.
And then the slope coming down is minus one over delta x.
So this was up to delta x and then to two delta x, and then zero.
That's a much more typical thing, this smoke goes the function, the hat function goes up to the top of the hat.
Back down.
The slope up and the slope down are easy.
And now the integral's easy.
So I'm just squaring, well, when I square it, this squared is the one over delta x squared.
This is the same, because the minus will get squared.
So what's K_11?
What's K_11 now?
Have you got K_11 in your head?
This is one over delta x squared.
And now what is the integral?
Two over delta x.
Because now we're integrating from zero to two delta x, because that's where my functions are going out from zero to node two.
If the function's numbered one.
So it's two delta x times this; I think we get a two over delta x on that diagonal.
Would you care to guess the rest of the diagonal?
Yes, you tell me what's K_22 and K_33 and K_44?
They're all the same.
We're just shifting over.
So two over delta x goes down there.
Alright, one more to do.
One more integral to do.
This next guy.
So can you tell me what do I get now for K_01?
K_01, so now this is the case where I'm in row zero, so this should be V_0, because that tells me the row I'm in.
But phi_1.
What happens when I integrate, just see the picture here.
Let me just draw it small.
phi_1', so let me draw phi_1.
And V_0.
OK.
But it's the derivatives that I want.
It's the slopes that I want, OK?
So what do I get from here on out?
Zero, because this guy only got to there.
That's the half hat, the first guy stopped at delta x.
So whatever is happening here is going to be multiplied by zero.
So it's just here.
One delta x integral for this one.
They just overlap in one interval, of course.
This guy and its neighbor only overlap in one interval.
And what's the deal about the two slopes?
They're opposite.
One's coming down, one's going up.
But the slopes are one over delta x and minus one over delta x.
Do you see what's happening here?
I'm integrating.
Here I have a slowpoke of one over delta x, and here I have a slope of minus one over delta x, so I should multiply those.
Minus one over delta x squared integrate, what goes in K_01?
What's that number?
It's that times that integrated, but now the integral is only going really out to delta x because basically I'm just stopping there.
So but there's a minus now.
Because it's not the square.
It's this times its neighbor.
One's going up, and one's going down.
So it's delta x squared, and then the length of the integral, this is a minus one over delta x.
Would you care to guess the rest of this matrix?
What's the rest of that diagonal, above the main diagonal?
It's all the same.
That stays the same, because when I do phi_2*V_1, my picture is just like shifted over.
But I still have one coming down, and one going up.
When I do phi_3*V_2, same thing.
And if I do phi_1*V_2, it'll be the same.
I'm going to get this minus one over delta x all the way on that diagonal, also.
Symmetry.
It's going to come out symmetric.
Actually, since the course started by speaking about properties of matrices, let me just say K is going to turn out to be symmetric positive definite.
And what's more, for this example we recognize K completely.
You will say why did you take so long to get to this result.
K is the one over delta x part times, what's the matrix?
It's T. It's T. So it's one, minus one, minus one, two, minus one, minus one, two, minus one, minus one, two, minus one, and I guess we had five of them.
Oh, but nobody there, right?
That's not there.
That fixed, why do we not have a minus one?
Because we've got no, there isn't a six.
That would be column.
We've got one, two, three, four, five columns; there's no U_5, there's no V_5, we've got them all.
And the F, so that, so KU, this thing multiplies U_0 to U_4, to U_4, and it produces F, which is one over delta, which is what?
Oh, delta x is in the numerator, right.
Times a half, one, one, one and one.
That is the finite element system KU=F.
For this simple problem.
It's exactly what finite differences did.
So you can see why my first introduction to finite elements was with the question what's the difference.
The finite element community at that point, this was like the golden age of finite elements, all this was just beginning to be created.
These elements were being used.
Especially in civil and structural engineering, that's where a lot of the earliest papers came out of.
And then, in a model problem it didn't look anything new.
It looked like our original finite difference matrix.
But there were some new things.
First, there was this new 1/2, that we hadn't particularly noticed with finite differences.
We we could catch onto that.
Here's a minor difference.
You notice that the delta x is strictly speaking the delta x is up here then, but when I divide by delta x then I'm back to the finite difference.
I have the one over delta x squared, it looks like finite differences again.
So everything looks the same.
But, of course, if c(x) isn't one or if F(x) isn't one, oh yeah.
If c(x) isn't one then I've got integrals to do.
I would approximate those.
And I could then come out with something that would look like a finite differences.
Let me take our other favorite model problem.
What would be the F if, yeah, here's a question.
What would be the right side if my vector, instead of being one, what's my other favorite choice?
Delta.
So I take delta at x minus, let me take delta at x minus 1/4, first.
Suppose that's my F. Then I've got to change all these guys.
And what would they be?
What would be the new right hand side when I have this point low?
I have to go back to the integrals, right?
I have to go back to these guys.
These integrals, with that new F, this is now delta of x minus a 1/4, times each V. Times dx, I have to integrate delta of x minus 1/4 against every hat function.
And see what it equals?
And what will I get?
You're going to tell me right away.
What are those integrals?
That's a point load at node one.
Times the V integrated over the whole thing.
What do I get?
I get a one, yeah.
That integral is going to pick out the value at a quarter.
Right, that's what the delta function does, the spike is at a quarter.
Has area one, so it picks out the V_i at a quarter.
V_i at a quarter will be?
One for the, I think we get a .
Again a little bit what our finite differences suggested that we should do.
Alright, here's one final one for today.
Suppose the delta function is not at a node.
Suppose it's at 3/8.
Or it could be at any point a, but let me just take a typical, a special one where I can do it.
Suppose the load is at 3/8.
What do I get for the integrals now?
So now, it's delta of x minus 3/8.
The spike is at this point here.
That's where delta is now, spiking at 3/8.
Is that right?
We had 1/4, tell me what - oh, I should have had 1/5 before, sorry.
Change that on the videotape.
All those 1/4s where that 1/4 was 1/5, and now what do I want?
Three?
I wanted to take a nice one that was halfway.
I just forgot what halfway was.
Where is halfway there?
3/10 now for delta.
So that was before I had it for when delta.
So previously was delta at x minus 1/5 and now delta at x minus 3/10, what's the F now?
What's the F now?
So the spike is right in the middle between one and two.
What's do those integrals come out to be?
If I integrate delta function times the different hats, what do I get?
What do I get, yeah.
I get a zero for this first guy because it didn't touch the half hat.
And then what do I get there?
Half.
And what do I get at the next one?
Half again.
And then the other guys it doesn't touch.
You see, it automatically does it.
Does those smart things.
It automatically makes the smart choice.
And if the spike was at a, at any point a, then at that typical point a wherever it is, like there, spike could be there.
Then I would have, what would I have if the spike was there?
I'd have a little bit of phi_3, and a big bit of phi_4.
And the two parts would add to one.
It would take the right proportion.
It would be the proportion, by however much this spike was near there, it would give that extra weight to phi_4.
OK.
So there is the finite element method.
It produced something that you might say, oh, we knew that.
But you've got to see that it deals automatically with c(x), it deals automatically with F(x), it deals automatically with the free boundary.
You see, the solution there is going to take sort of a balance, a pretty close balance, of this half hat with this one, and the solution will actually be a pretty close to free.
It'll be pretty close to having the right zero slope there.
OK, good.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. All right.
Good morning.
So we're doing finite elements.
The element that we considered so far was the basic linear element, continuous.
But of course, the slopes have jumped.
The slope was minus one over delta x for that one.
This one was a plus and then a minus.
So a jump in slope but no jump in the function.
So actually, my abbreviation for that would be C^0, saying that it's continuous but no derivative is continuous.
And now we'll get to some elements where the slope is continuous.
It's sort of fun to create these finite elements of higher degree.
It's pretty straightforward in 1-D. And that's where we are now.
So we'll get second degree elements and third degree elements.
And that gives us, as we'll see, higher accuracy.
So I want to connect the degree of the polynomials to the accuracy of the approximation.
Part of that connection is to recognize that these problems have a strong form, as we know, the equation; a weak form, that's the one that has test functions.
And also a minimum form that we'll see.
So how would I get some quadratics, so second degree elements, parabolas into the picture?
You remember Gelerkin's idea?
Choose trial functions.
And we're taking those to be the same as the test function.
So these are the trial functions we've chosen.
One, two, three, four of them.
And they're linear.
And that limits the accuracy that you can get, because your approximations then are combinations of those.
So they're like broken line functions, linear approximations.
And the accuracy is not great.
It's sort of the lowest level possible.
So how would you get parabolas?
So this was first guy.
The second guy is going to be continuous and quadratic.
So it's going to have new trial functions.
In addition to these, I'm going to put in some more.
Gelerkin's happy with that.
I still proceed as usual.
My approximation is some combination of those.
It's only going to be continuous.
So it'll just be a C^0 guy again.
That means jump in slope.
The first derivative isn't there.
Eventually, I want to get to a C^1 where the slopes are continuous.
OK, but how would I get some quadratics?
All I want now is my functions, my space, my combinations should be the piecewise parabolas instead of piecewise linear.
And the pieces are broken at the nodes.
OK, so here is a way to do it.
Inside each interval, I'm going to add, I'll just call them bubble functions.
So these will be new additional guys.
So this will be my first phi.
You remember that half hat?
Because the problem I was doing was a free fixed problem.
That's why I had a half hat at this end, because there was no boundary condition that my functions had to satisfy.
At this end, there was.
It was fixed.
So that's why the hat ended, and there was no half hat, there's no extra function there.
So I have right now one, two, three, four.
I'm going to add four more bubble functions.
Each one will be inside an interval.
So it'll be a little parabola.
This is function number whatever.
If I number this number one, let's say, phi_1 now -- that's probably a change in numbering -- phi_2 is going to be my bubble.
And you see what my bubble function is?
It's a function that goes there and straight.
So it is continuous, no jumps, and it is second degree.
It's a parabola, and I'll make its height one.
And then there'll be another function.
If I had another color I could draw it.
Well, I'll just do it with broken lines, maybe.
So there'll be another bubble function in here, a third bubble function in the third interval, and a fourth in the fourth interval.
You see that I've now got my old phi_1, phi_3, phi_5, and phi_7 were the hat functions.
But now I've got a phi_2, phi_4, phi_6, and phi_8 that are these new trial functions.
So part of the message is, we can throw an additional functions.
They don't have to be polynomials, but those are the simplest choices.
Why are they simple?
Because, you remember, that in the end when I made the choice, I have to do various integrations.
So you remember that I have to integrate to find entries K_ij.
Do you remember what that interval looked like?
You certainly remember F_i.
That was the integral from zero to one of whatever function phi_i we had, times the F(x)dx times the load.
And we computed these.
You remember we computed these for the piecewise linear guys.
But I don't think I wrote down the expression that were really doing, so let me just do that.
It's c(x)du, no.
d phi_j/dx and a dV_i/dx.
Those were the integrals that we had to do.
And we were taking phis to be the same as V's.
Maybe I'll just do that here, because I don't plan to make any other choices at all.
d phi_i/dx.
It's a symmetric matrix now.
K_ji, because when I'm choosing phis the same as the V's, this is what it looks like, and if I switch j and i, I don't see any difference.
So these are the things that have to be integrated.
And those are the ones we did integrate when phi was piecewise linear.
When phi was piecewise linear, the slope was piecewise constant, and we had really easy integrals.
Very easy integrals.
We had to pay attention to where we were was the slope on a minus interval or on a plus interval, but they were easy to compute.
And they led us back to the kind of matrix that we've seen before.
The twos and minus ones.
And our right hand sides looked familiar.
Now, we've got new functions.
We still have the same formulas.
No change in formulas.
The system is really quite successful, because these are the things that we have to compute.
So now I'll have to integrate these parabolas, these little parabolas, half of the phis will be little parabolas, and their derivatives will be linear.
So you see, I'll have more calculations to do.
Which I don't plan to do, but more integrations.
For example, the diagonal entry, say, 2, 2, which will come from that bubble with itself.
K_22, then, will be the integral of c(x), times the derivative of that bubble, which will be a straight line times itself, so it would be squared.
And c is positive, so this K_22 is going to be some nice positive number.
But we'll have to figure out what it is.
Maybe I'll just say one fact that we'll come back to.
That this K is symmetric positive definite.
You thought it would be.
By using the letter K, we kind of expected it to be.
And it will be.
It'll be symmetric because the phis and the V's are the same.
And it turns out it's positive definite.
So it's just great.
Just great.
We have a little more effort, either to use a formula for integrating polynomials, or using numerical integration.
One way or another, and I won't concentrate right now on that point, we get these numbers.
Okay.
Here's something to concentrate on.
What kind of a matrix K do we have?
Where will it be non-zero?
So it'll be eight by eight, right?
I'll follow through on that choice.
Just to say, where will I see non-zeros here?
Because if you get that point, you see the way things come together.
I'll just put a little x for non-zero.
So K_11.
So what's that first row of K?
It's coming from the first function, integrated against itself.
K_11, if for 1, 1, we'll get something there.
Will we have something in the 1, 2 position?
That's my question, do we have something in the 1, 2 position?
What's 1, 2?
That's this function against the bubble function, yes?
Right?
They're non-zero at the same place.
We can expect something there.
What about K_13?
That's what we've done before, that's this one against this one.
Yes?
We expect a non-zero there.
But then what?
After that, what will the rest of that row be?
Zero.
Because that first half hat doesn't touch any of the others.
So let's go on.
Of course it'll be symmetric.
I know this much.
So this is the half hat row, and this is the first bubble row.
Because the half hat was phi_1 and now the first bubble is phi_2.
What non-zeros do we get in the stiffness matrix?
Again, we could unconstruct it entry by entry.
Another way to construct it will be element by element, stamp them in.
You're beginning to see the idea of that.
So what do I get for that bubble?
I just look to see which elements touch that bubble.
And which ones do?
One, two and three, and not four.
Right?
In that row, we only get -- so from that bubble, I think we only get that much.
Now, we're not quite seeing the picture yet.
Let me go to the next hat.
The hat, phi_2, and then I'll do the bubble.
Oh, no, sorry, the hat's numbered phi_3, and then the next bubble is numbered phi_4.
Where do I get zeros?
You can tell me, where do I get zeros?
From inner products, from these guys, when i is three.
So which phis does phi number three overlap?
That's all I'm asking.
Does it overlap number one?
Yes.
Does it overlap phi number two?
You want to highlight, so we're now looking at phi_3, at this hat.
God, where's it gone?
That's the one we're doing now?
So what does it overlap?
It overlaps the half hat, does it overlap the first bubble?
Yes.
Does it overlap itself?
Yes.
Does it overlap the second bubble?
Yes.
Does it overlap the next hat?
Yes.
And then all zeros.
Okay, and now do one more row.
Bubble four.
So now I'm looking at this guy, this next bubble.
phi_4.
What does that overlap?
Does it overlap the first half hat?
Nope.
Of course, symmetry told us that.
Does the second bubble overlap the first bubble?
No.
Big point: zero there.
Does the second bubble overlap the hat?
Yes.
Does the second bubble overlap itself?
Certainly, on the diagonal we have something.
Does the second bubble overlap the next hat, phi_5?
Yes.
And that's it.
I think.
The second level does not overlap the following bubble.
I don't know if you see what pattern we're getting here.
Those were special rows, because that was only a half hat.
These are typical rows.
A typical hat function, that row is showing us five non-zeros, because it overlaps itself, the neighboring hats, and the neighboring bubbles.
But the bubble row only has three, because a bubble overlaps itself, the neighboring hat on each side, but not the neighboring bubbles.
So we have only three non-zeros.
Do you see that the next row will have five?
Will I get it right?
I hope so.
The next row we'll have, I think they'd be here.
And then the next row will have only three guys, maybe here, here, here.
Well, it's certainly a band matrix.
So you could say, okay, it's a band matrix.
I wouldn't call it tri-diagonal anymore.
If I showed you that matrix and said, what kind of a matrix, you'd say a band matrix.
If you wanted to tell me that it had five bands, you could maybe say penta-diagonal, or something.
But it's easy to work with, of course.
That's the point of finite elements, is that all the functions are local, so that we get all zeros when trial functions don't overlap.
My additional point was just a small one that's not a big deal, but it's a little bit worth noticing.
These rows with only three entries, three non-zeros.
I guess what I want to say is I have to solve eight equations and eight unknowns.
And the normal way to do it would be just elimination.
LU, that would work fine.
Start from the top, eliminate, and you've got it.
And of course in one dimension, nobody would do anything else.
That would be simple.
I just want to say, these bubbles, by giving me extra zeros, I could eliminate the bubbles first.
Can I just make this point but not labor it?
I could eliminate the bubbles first.
I could use this equation to express the bubble coefficient in terms of its neighbors.
I could use this one to express the bubble coefficient in terms of it neighbors.
And I could plug back into the other equations.
I could simplify this.
I could get the bubbles done first if I wanted.
I can see that to go into the gory details is probably not wise.
But bubbles are easy to do.
However there are better elements.
So that's my discussion of quadratic elements, almost complete.
It's not a big favorite, because cubics are better.
So why are cubics better?
Why are cubics better?
So you're going to say, okay, upgrade to cubics.
How shall I do that?
And I want to say a word about the error here.
Of course, the reason quadratics are better than cubics...
Sorry, the reason why quadratics are better than linear, and cubics will be better than quadratics is I'm getting more accuracy.
Suppose my true solution may be some curve like that.
Okay.
My piecewise linear elements, suppose the piecewise linear elements happen to be, as they would in a special model problem, right on the money, at the nodes.
Usually they won't be.
But what would be the error in that one?
Well, no error at all at the nodes as I've drawn it.
But that's not what I'm interested in.
I'm interested in, how big is that?
How far off is the displacement?
What's the maximum error in the displacement.
Do you have any idea?
If this is size h delta x. Shall I call it delta x or h. How far does a curving function escape from the...
I need to blow that up, don't I?
So I have a curving function and a linear function, and I want to know how far apart they are over a distance of length delta x.
What's this scale?
That's the question.
It's just good, it'll have a simple answer and it's great to know it.
Anybody want to make a guess?
Is that scale of size delta x?
Is it of size delta x squared, size delta x cubed?
It's that exponent of delta x that is telling me how big is the error?
And it's easy to find once you get the hang of it.
Anybody want to make a guess?
Delta x?
Squared.
Squared would be the right guess.
Squared would be the right guess.
I could just turn that picture if we wanted, to this is delta x.
Now it would look like that, pretty much.
Doesn't have to be symmetric, of course, because this could be a complicated function.
But when I focus on a little delta x interval, every function looks like a little polynomial.
The error there, let's see.
What would that function be?
I could go forever on this.
But look, if the slope is something, whatever, let me change numbers here.
Let me call it from zero to y, what would be a little parabola that has a slope of one, let's say, at both ends.
What would that parabola be?
We probably have seen that before.
If I wanted a slope of one at both ends, the polynomial would be something like... what would it be?
Sorry, tell me that little polynomial.
It's a polynomial in x, it's just a quadratic.
Its slope is one, so it maybe starts with an x. I've got to bring it down here.
It's x times one minus x over....
I didn't like y ever in the first place.
What do I want to put there?
I don't want to put a one.
That would make it look big.
y is there.
Okay, I think that quadratic is zero at zero, because of that term.
It's zero at x=y, because of that term.
It's second degree.
And I think its height is a maximum right there.
And what is that height?
At y/2, this is y/2, this is y/2.
That height is y squared over four.
That's what I was shooting for.
The square.
That in a little interval of length y, for length delta x, if I draw a little parabola and I'm matching at the ends, then the height it reaches is like y squared.
That's the scale.
So my conclusion is that if I use these basic hat function elements, the error I get is -- so can I list the errors?
-- the error is delta x squared.
That's the displacement error.
The error in U. I'm not proving anything.
The careful discussion of the accuracy is a later section in the book.
But I'm trying to make the main point, is that if we're fitting functions by straight lines, then we have an error of delta x squared.
And what's the slope error?
What do you think is the slope error?
Because for us that slope is important.
That's the error in the stretching and the strain.
So the error in the function is delta x squared.
The error in the slope will be one order less, just delta x.
Okay, I'll come back to all this.
Now, make a guess.
Suppose I include these bubble functions.
With delta x as my length scale horizontally, what will be the scale of the error?
What do you guess is the expected error in displacement for a general problem, for a general c(x) and F(x).
Which I won't get exactly right, but how close will I come?
I'll come within delta x to what power?
Make a guess, please.
Four is an optimist.
I won't get up to four.
Cubed.
I'd only get cubed.
I'll get one by increasing the degree of the polynomial by one, I'll get one degree better.
So it you could look at it this way.
Suppose I have any function.
This is a another way to think about the accuracy.
Suppose I have any function F(x).
The whole point of calculus is that I could start, if I start where it is at zero, then I add in F'(0), the slope times x.
Then I add in 1/2 F''(0) times x squared, and so on.
Right?
It's called the Taylor series.
And we're not paying any attention to convergence, or high order.
It's the early terms that I'm interested in.
And the point is that if my functions include linear functions, which the hats did, they will be able to get these terms right, and this will be the error that I missed.
I'm just looking to see what's the first term in the Taylor series that I will not get.
And if I only have hat functions, I can't get an x squared.
I can't get a parabola.
But when I go here and include the x squareds, I can get that term right.
So then it'll be the 1/6 f triple prime x cubed that I miss.
So the error will be the next missing term.
Okay, so that's thoughts about the error.
And of course that's why those elements are better than these.
They take more work, but they are worth it.
But now I want to tell you about the next elements.
Cubics.
Where you're going to expect to get delta x to the fourth.
So now we're getting serious accuracy.
Now we're getting good accuracy.
Of course our problem is not the most difficult problem.
It's in 1-D.
But this is good.
Okay.
This was now the fun in the golden age of finite elements.
To construct cubics.
What shall I use as basis functions for cubics?
So I want to have a cubic in each piece.
First of all suppose I just want no more than that.
Suppose I'm happy with just continuous functions and I let the slope jump.
What new trial function shall I put in?
So I'm going to put in new trial functions.
What will they look like?
Little cubics?
Little third degree bit pieces.
Instead of parabolas, they'll be little pieces of third degree.
And I could put in four more bubbles.
Four cubic bubbles.
So I would be up to twelve degrees, twelve by twelve matrices, twelve functions.
And for that size delta x, that would give me delta x to the fourth.
So that would be okay.
There's a better idea.
You can see that I left space.
I'm going to make the slope also continuous.
I'm not going to allow jumps in slope.
Think how will I do that?
So I'm going to call those C -- what will I call that when the slope is continuous?
The first derivative, I'll call that C^1, continuous first derivative.
Okay.
Now I'm actually in section 3.2, where these better elements, these really nifty elements are constructed.
C^1 continuous slope cubics.
Okay.
Ready for those?
What shall be my trial function for continuous slope cubics?
So I have to start again.
I have to start again because the hat functions are out now.
Those hat functions have a jump in slope.
The bubble functions have a jump in slope.
I'm rethinking here to create a better element.
Okay.
So let's just think, if we've got a chance at it, how could these elements work?
Okay, so here is the idea, then.
Here is my interval.
Zero to one, and here's a typical interval.
And now at a typical node, like node one, I plan to have as unknowns the height of the function, as before, and also the slope.
So I want the function, my trial function is going to have some height and some slope.
And at node two, it's going to have some height and some slope.
And here's the question.
Here here's the good point.
That those four numbers, the two heights and the two slopes, that gives me four things, four quantities.
How many quantities do I need to determine a cubic?
So by a cubic, of course, I mean by a cubic something like a zero plus a one x plus a two x squared and a three x cubed.
It's called a cubic because it's x cubed.
So how many numbers here?
Four.
Perfect match.
There's exactly one cubic that has a specified height and a specified slope at these two ends.
There's one cubic that'll do that.
And then whatever the height here is and whatever the slope there is, there'll be one cubic with that height and that slope that comes into this one.
And you see that they will have continuous slope.
Because of course the slope is continuous in between; it's a polynomial.
The question is always at the nodes.
But I use the same number coming from the left and from the right.
The slope has become an extra unknown.
The slope has become an extra unknown.
So I have height slope at every point.
So that's one way to describe these trial functions now.
The trial functions have height and also slope at each node.
So what does that mean?
That means that I'm going to have two unknowns.
Two functions, two trial functions, each with its own coefficient at each node.
So if I take a typical node there, I want two functions.
Okay, this is interesting.
But you see what I'm creating.
I think I'm going to get two functions there, two functions there, two functions there, two functions there, right?
Because nobody's constraining that.
So I'm up to eight.
And how many functions do you think I'm going to have associated with that node.
Only one.
Why?
Because the height is fixed.
So I think I've got nine trial functions here.
And if we can see what those are, then the system will take over.
They're my phi_1 to phi_9, whatever they plug in here, they plug in the right hand side, I'll have a nine by nine stiffness matrix.
It'll be local again.
Well, let's see if we can figure out these functions.
Okay, so you have the idea?
I'm expecting two trial functions.
One is sort of a round hat.
All right, let me draw that.
The round hat function will be the function -- These will be the round hats, and they'll be associated with, they give me heights.
And then I'll also have an additional one, except at the last node.
And these will be -- I don't know what to call them yet.
You'll have to give me a name.
These will give me the slopes.
Okay.
So what does a round hat look like?
Now these have to be, follow my rules, they have to be continuous, their slope has to be continuous.
And I want to take the one that has height one and zero slope there.
And it should have height zero and zero slope, here.
Height zero, zero slope.
You see what it's going to be?
This phi, whatever number it is, it'll be the phi whose coefficient tells me the height at node one.
So here's node one.
What will it look like?
What will this function do?
Well, there is exactly one cubic, that starts from zero with slope zero and ends there, ends at one with slope zero.
Right?
That's what we said; four numbers determine that cubic in that interval.
Then there's another cubic that, with those two numbers again, that keeps the continuous slope, and these two numbers in this interval.
And of course it'll just be symmetric.
You see the round hat?
So that's the basis function, the trial function that has continuous slopes and heights, of course, and it has height one at that point.
And now let me draw the one that has height zero slope zero, height zero slope zero.
And what do I want it to do there?
What should this function be like?
It should be the one that it's coefficient will tell me the slope.
So I want it to have a slope of one and a height of zero.
Do you see these functions, shall I call these the height functions, phi h 1?
That's the phi, that's the trial function that tells me the height at node one.
When I take combinations, it gets multiplied by U h 1, which is exactly the height at node one.
Now what about this guy?
This guy is going to start with zero slope at zero.
It's going to be a cubic, and there's exactly one cubic that'll do it.
It'll look a little like.
Then there'll be exactly one cubic that does that and gets back to zero.
You see that that's possible?
In each interval, I've got four numbers: two heights, two slopes.
So this would be a picture of the phi slope at node one function.
So that's a standard function, it's a cubic, piecewise cubic.
Local again, because in all these intervals it's zero.
And it will be, when I go to take combinations of all these guys, it'll be multiplied by its coefficient, U slope one.
And then I'll have nine all together.
But those two are the typical ones.
Do you do see how that's going?
It's more subtle than hat functions.
Suppose whoever's writing the finite element code gets a formula for those phis and plugs them into the integrals, comes out with a stiffness matrix.
Actually, we could even look at that stiffness matrix.
This is a good way to understand the picture.
Now it'll be nine by nine.
Right?
So here we'll have a typical, this'll be our phi height 1 row, and this'll be our phi slope 1 row, and this'll be our phi height 2 row, and so on.
Of course, I didn't leave room for all.
What will a typical row of this stiffness matrix have in it?
I'm just asking about the overlaps.
phi 1 height certainly overlaps itself.
Does phi 1 height overlap phi_1 slope?
Yes or no?
Sure.
Sure.
Does phi_1 height overlap phi_2 height?
Yes.
Yes.
Because the phi_2 height will go up like that.
You see?
And the phi_2 slope.
So actually we'll have, I think we'll have six non-zeros on a typical row.
Is that right?
Six non-zeros?
Because a typical h -- this is maybe not so typical, because to the left of it there's only one -- No, there are two?
Right?
There's a phi_0, phi h 0 and a phi s 0.
Sure, there are two here, the two guys here, there's one height guy, and there's one -- what's cooking in that?
Oh, it's got a slope of one and it gets back to zero.
What I'm drawing now in little dashed lines was the phi slope 0.
The one that gives me a slope at node zero, and this is the one that gives me a height.
Yes.
Do you see it?
So above this was a phi slope 0, and stuck in there was a phi height 0.
Six diagonal matrix.
I think it helps to draw that little thing with x's and zeroes, because then you sort of see how things are fitting together.
Okay.
So these functions now, I've gone into section 3.2 for that.
I want to go to a slightly different topic, and then I'll come back in section 3.2 to these cubics.
So these are C^1 cubics, continuous slope cubics.
Very interesting construction.
Are you seeing how it could go in more dimensions?
I mean, that's what we'll see for Laplace's Equation, how can you construct quadratics, cubics in a plane.
It gets interesting.
But you'll get the knack of these guys.
These are pretty direct, and very useful.
So what's the effect?
The effect is that we get a matrix.
It looks quite like a difference matrix.
Well, actually, the height rows and the numbers in the height rows and the slopes rows look different.
We're getting something new here.
We're getting matrix, a KU=F, that's going to give us fourth order accuracy.
So the accuracy has moved up.
So we've got up to fourth order accuracy, which we could get by finite differences by a lot of patience.
We get them from finite elements in a straight way.
Okay, any question or discussion?
I'm talking real fast to get this new idea of constructing finite elements here.
I do want to say something about that line.
Because that's a part of this business of estimating the accuracy.
It's a key idea in the background of the Galerkin method, and the minimum form would be associated with names like Raleigh and Ritz.
All right.
I'll just go directly to that, if I may.
So what I want to do is tell you, for our model problem, I want to tell you the strong form-- let me do it this way.
I'll put the strong form, the weak form, and then I want to add in the minimum form.
Okay.
So the strong form of our equation was minus the derivative of c*du/dx=f.
Okay.
What was the weak form?
This is an f(x).
The weak form, how do you get to the weak form?
You multiply both sides by a test function, you integrate, you integrate by parts, and you get this beautifully symmetric form that we have up there, du/dx*dv/dx*dx, equals the integral of f(x)*v(x)*dx.
I write that again, just so you see the nice symmetry of that weak form.
And it's for all test functions v Okay.
I'm shooting for a third description.
A third description of the same problem.
And it's really neat to see that you have that.
Let me just see it first in the discrete case.
The discrete case, the strong form would be A transpose C Au=f.
That's the strong form.
Right?
I always like to see the discrete one first, and then the continuous.
Okay, what would be the weak form in the discrete case?
I would multiply by a vector v, and I would take inner products A transpose C Au inner product with v, equals f inner product with v. You can use dot.
So that would be the weak form.
I've just taking the dot product of both sides with v. Now you'll see the weak form better if, what should I do?
What would make that look nice?
So that's the dot product of A transpose C Au with v. And what do I do to make that look nice?
Do you get the idea yet?
It doesn't look pretty to me.
It's all lopsided.
Right?
So what can I do with A transpose?
What's the rule about A transpose?
That if I have A transpose times something, dotted with something, what can I do?
I can move the A transpose over to the other guy.
And what will it be when I do that?
So I take it away from here, and what do I put there?
A.
That's the whole point of transposes.
Transposes, you put them on the other side of the dot product, you take the transpose, so it would be literally, maybe A transpose transpose, which is A.
What I just did there is integration by parts.
Well, summation by parts, because I'm in the discrete case.
The whole idea of integration by parts amounted to taking A transpose off of u, off of this, and putting a over there.
Isn't that neat?
And you see that this CAuAv is just what I have here.
C, a is derivative, so this is CAuAv.
Inner product.
That's cool.
That's just like how it should be.
I just followed that rule, that A transpose times something, shall I call it w, inner product with u, is the same as wAu.
That if I bring A transpose over, it becomes an A.
If I bring an A over, it would become an A transpose.
All right, what about the minimum form?
Have I got one minute to do the minimum form?
Yes.
So what's the minimization that's hiding behind this?
The minimization in the discrete case, do you remember?
We're looking at Ku=f.
And some quadratic quantity from least squares has its minimum when Ku=f.
And it's 1/2 u transpose Ku minus u transpose f. Where K is A transpose C A.
This is the minimum statement of the problem.
That if I look for the u that minimizes that quadratic, it leads me to the equation Ku=f.
So that's the minimum statement.
And if we want it to really look perfectly like the others, I would put in A transpose C A.
Okay.
Can I write down next time, because our time is really up.
It's not fair to -- all I'm going to do is write down the same thing here.
I'm minimizing 1/2 -- oh, I'm going to do it anyway.
c(x)*du/dx squared, minus the integral of f(x)u(x).
So that's the minimum problem.
Minimize over all u, this quadratic.
This is the right way to see these problems.
You see a differential equation, which we use for finite differences; you see a weak form, which we use for finite elements; and now you see a minimum form.
Okay, that gives you something to think about.
And there'll be a homework on finite elements that'll give you a chance to use them.
Okay, thank you.
The following content is provided under a Creative Commons License.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional material from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Okay.
Hi.
So, our goal, certainly, to reach next week is partial differential equations.
Laplace's equation.
Additional topics still to see clearly in 1-D.
So one of those topics, these will both come today, one of those topics is the idea, still in the finite element world, of element matrices.
So you remember, we saw those, that each bar in the truss could give a piece of A transpose A that could be stamped in, or assembled, to use the right word -- I think assembled is maybe used more than stamped in, but both ok -- into K. For graphs, an edge in the graph gave us a little [1, -1 ;-1, 1] matrix that could be stamped in.
And now we want to see, how does that process work for finite elements.
Because that's how finite element matrices are really put together out of these element matrices.
Okay, so that's step one, that's half today's lecture.
Step two, problem two now, coming from the next section, is fourth order equations.
Up to now, all our differential equations have been second order.
Are there fourth order equations that are important?
Yes, there are.
For beam bending.
So I'll describe that application, which leads to fourth order equations.
Do they fit our A transpose C A framework?
You bet.
You know they will.
Each additional application in the framework kind of get us comfortable, familiar with that framework, what it can do.
The A transpose C A.
So let me start with element matrices.
I'll get the homework, those numbers will get posted on the website later today.
I just thought I'd put them down, and I have to figure out what would be a suitable MATLAB question.
So my idea for this homework, as it really was for the last homework, was that you get some, not a large number of ordinary questions, paper and pencil questions, and one MATLAB question.
When I wrote MATLAB last, people interpreted that as last MATLAB.
But those words are not commutative.
Right?
I just meant -- and it doesn't really matter, it was a dumb thing to say -- I meant you to put the MATLAB question at the end after the regular questions.
Sorry.
So that MATLAB is due.
Maybe a bunch of those turned in on Monday didn't include the MATLAB.
No penalty.
I'm talking now about the MATLAB for the trusses.
How many have still got a MATLAB for the truss still to turn in?
A number.
Oh, not too many.
Okay.
Anyway, just when you can.
In my envelope is good.
Okay.
So it'll be a similar thing for this week, and because it's short I said okay, let's get that cleaned up by Monday.
And then we're ready for vector calculus, partial differential equations, two and three dimensions, the next big step.
If I want to illustrate element matrices, the best place I could do it would be to go back to our piecewise linear elements in 1-D, and see how an element now is just one of those little intervals.
Little pieces of the whole structure.
If it's a bar, or whatever it is, I've cut it into pieces.
Here's one piece, here's another piece.
With finite differences, if I gave you unequally spaced meshes, if my little h is different from my big H, we'd have to think again.
I mean there would be some three point, instead of minus one, two, minus one for the second difference.
It would be a little lopsided, of course, when the mesh is unequal.
We would have to think that through for finite differences; for finite elements, the system thinks for us.
So I just want to show how that happens.
So I'll take this.
What's the element matrix for our standard equation for that element?
Okay.
And then it would fit into the big matrix K. I'm going to come up with a matrix K that I could do in any other way, but this is the way it's really done.
So I look in that interval.
I'm focusing on that interval.
So here I've drawn the basis function.
But now I'm not going to operate so much with basis function.
Let me just think about that interval again.
That's the same interval.
So those were the basis functions that went up to height one.
One started at height one and went down, one started at height zero and went up.
But it's their combination, of course.
Let me number this node number zero and this node number one.
So we have some height, U_0, here.
This is coming from U_0 times that hat.
And some other height, U_1, here, which is coming from U_1 times that hat.
But inside this interval, my U in this interval is U_0 times that first hat function, the phi_0, the coming down hat.
Plus U_1 times the phi_1 function, the going up hat.
It's a linear function, so it's going to be there.
That's my graph -- no, yes.
This is U(x).
U(x).
Okay.
Right?
I'm focusing inside one element.
One interval.
I've numbered them zero to one, but of course that distance is H. Okay, so there's my function.
Right?
Everybody agrees that this combination, it starts out right, it ends up right, at height U_1, and it's linear.
So that's got to be right.
So really, what's the contribution from that element?
I look at the quantity, and I'm always taking the phi functions to equal the V functions.
So I can write this as c(x) times dU/dx squared, dx.
Over the whole thing, over the whole interval, that would be the U transpose K U.
This, everybody remembers the K part, of course, comes from the left side of the problem.
It comes from integrations.
And this dU is now a combination of all the -- with weights U_0, U_1, U_2, U_3 -- if I plug in for capital U and do all the integrals, I'll see what K is.
Now what's the point?
The point is, when I did those integrals, the question is, how am I going to do the integrals?
Do I do them the way I did before, was I watched what phi_0, one phi times another phi and I integrated.
That was successful.
But the new way is, integrate it an interval at a time.
Do the integrals.
So this would be K global.
Now I'm going to go just from zero to H. If I go from just zero to H, that's going to give me the K element piece.
The little piece that comes from this little interval.
And on that little interval, this is my formula for U. I'm just hoping you'll sort of see this as a reasonable idea, and then when we do the integral you'll see it.
It clicks.
Okay, how does it click?
So what's my plan?
My plan is, here's my function.
There is its picture.
I'm just going to plug it into here.
Oh, it's going to be simple, isn't it?
What's the slope of that function in that element?
On that interval?
What's the dU/dx?
So instead of doing every full integral for each separate phi, I'm doing every element integral for both phis.
See, these two phis are both coming in to that element.
Okay, so what's dU/dx.
I didn't realize how neat this that's going to be.
So what's dU/dx in this element?
I'll use a different board here.
So it's an integral, then, from zero to H of whatever my c(x) might be.
And I'll make a comment on that.
But my focus here is, what's dU/dx?
What's the derivative?
And then I'm going to square it.
For this function, for that picture, the slope is obviously (U_1-U_0)/H Right?
The slope is (U_1-U_0)/H.
And I'm squaring it.
dx.
Okay.
Let me take c(x)=1.
Just to see clearly what's going on.
So c(x) is just going to be one.
Okay, so I'm claiming that this is my U transpose.
My little piece.
Why is it only a little piece?
Because it only involves two of the U's.
It's going to be a little two by two element matrix, that comes from this element.
And then it's going to to be put into the big K, the global K, in its proper place.
Okay, well.
It's trivial, right?
This is a constant, this is a constant, the integral is just H times U_1-U_0, squared, over H squared.
Because that's getting squared, the interval was length H. I think I just have a 1/H.
So this is my U transpose K element U.
And now I want to pick out, what's that matrix?
What's that little two by two matrix that only touches these two U's?
What's the matrix -- well, since it only touches these two U's, you can tell me.
I want a U_0, U_1.
Sorry, let me make a little more space.
And you can tell me what matrix goes in there.
I want this all to match up.
U_0, U_1.
Now here is the two by two element matrix, .
What's the two by two matrix that will correctly produce this answer?
I'm just shooting for that answer.
What do I have here?
This is U_1 squared minus 2U_0*U_1, plus a U_0 squared.
Right?
And I have to remember the 1/H, so it's automatically going to come out right.
So there's a 1/H, shall I remember that first off.
1/H is part of my element matrix.
And then, what are the numbers that go inside that matrix?
We had practice with this.
You remember when we talked about positive definite matrices, way back in chapter one?
The point was that we could look at eigenvalues, or pivots, or something.
But the core idea was energy.
The core idea was that energy, that quadratic, and that's what we're looking at again.
That's the energy.
That's the energy, right there, and this is the energy, this is the energy in the finite element subspace.
All I'm saying is, what matrix, what two by two matrix, goes with U_1 squared minus 2U_0*U_1 plus U_0 squared.
Just tell me what to put in that matrix.
What do I put in here?
One, right.
Because it's multiplying U_0 U_0.
What do I put there?
Minus one.
Good.
Because I have a minus two, it comes in, minus one comes in twice, and a one goes there.
So there, with the 1/H included, is K_e.
K element, for that element.
For the big H element.
You'll say big deal, because we've seen this thing before.
Notice what is nice.
First of all, notice how nice that is.
It's particularly nice, of course, because I took c(x) to be one.
So let me make a comment.
Suppose c(x) was not one.
What would I do?
Suppose c(x), suppose I have some variable stiffness in the material.
Suppose the material could be changing width, so its stiffness would change.
So in other words, I'd have a variable, c(x), that I should do the integral.
Probably finite element systems aren't set up to actually do the exact integral.
What would they do?
They would take, for that simple integration, they would probably just take c(x) at the midpoint.
So there's a numerical integration here in the creation of these element matrices.
And numerical immigration is just, take a suitable combination of the values at a few points.
You do know Simpson's rule?
Simpson's rule, that's a pretty high level rule.
My suggestion there was just a midpoint rule.
Just take c(x) at the middle of the interval.
Then it would factor out.
So I should really put a c here.
A c should really be coming in there.
And you expected that, right, from the A transpose C A.
We always saw a C in the middle.
It really should be there.
When I took c to be 1, I didn't see it.
So what am I doing?
I'm approximating c(x) by c at halfway.
Approximately.
I would replace this unpleasant, possibly varying, function by c at a point and use that value.
So numerical integration is one part of the picture that we won't go into all the different rules.
There's a rectangle rule, there's a trapezoid rule, very good.
There's the Simpson's rule that's better.
As I get higher order elements, like those cubic elements I spoke about, the numerical integration has to keep up.
If I was integrating cubic stuff, I wouldn't use such a cheap rule.
I would go up to Simpson's rule, or Gauss's rule, or somebody's rule.
Anyway, that's the c part.
Here's the part that stamps into the matrix.
Notice, by the way, when I stamp it in, tell me how it's going to look stamped in.
And then I've completed it.
So here's my big K. I wish I had a little more room for it.
Okay, here's my big K. So that interval that I drew there, the H interval, will stamp in here, some two by two.
Right?
Now where will the similar thing coming from this guy -- maybe it has to be numbered minus one.
Sorry about that.
That's another little interval.
I'll do the same thing on that interval.
I'll get a little element matrix, two by two guy, for K for that element.
And where will it fit in to the big k?
Does it fit in up here, let me just ask you.
Does it fit in up there?
Yes, no?
I'm assembling, stamping in the small two by twos into the full n by n. And if I draw the picture you'll see it.
So when I do the two by two for that big H interval, it goes there, let's say, then I just want to say where does the two by two go for the interval to the left?
Does it go there?
Nope.
How does it go?
It overlaps.
Right?
It overlaps.
Why does it overlap?
Because the phi_0, this guy, is acting on the right, and also acting on the left.
The U_0 is active, is partly controlling the slope this way, and also that way.
The U_0 is in common the two intervals.
Anytime any unknowns, any mesh points that are in common to two elements, we're going to have an overlap when we assemble.
And so it'll just sit, it'll sit right there.
And so there is the diagonal guy.
And maybe you could tell me what number will go there.
What number would actually go there?
And then you'll see the whole point of assembly, stamping in.
Well, what number goes there from this?
One times the c/H.
So in here would be the c, can I call it c right, or c_H, c on the big H interval, divided by the H. Because that one, we had to get it right.
And then what will go in that very same spot?
So add it in coming from the small h interval.
The same thing, it'll be this one, like shifted up, moved over.
So it'll be coming from that one, so it'll be a c on the little h interval, divided by little h. That would be the diagonal entry of K. That's what we would see right there.
Over here, should I write a typical row of K?
Typical row of K, when I do that, is going to have this c_h/h, plus c_H/H.
That's like the two, right?
That's like the two.
And what goes here?
What will the entry be that sits there when I assemble?
Just this guy.
Right?
Just this guy, times that.
The entry here will be the minus c_H/H.
That'll be the entry.
This is the diagonal one, this is the one to the right, and what's the one to the left?
What's the one here?
Well, you know what it is.
It's going to come from the minus and it'll be the minus c_h/h.
Look at that.
That just shows you how it works.
And again, you can look at that, page 299 to 300 in the book.
You see that if the c's are the same, if the h's are the same, then we're looking again at our minus one, two, minus one.
Times whatever c over h, to keep it dimensionally right.
Do you see that?
It's just simple.
Simple idea.
The point is that each interval can be done separately.
It's a simple idea in 1-D.
It's a key idea in 2-D, where we have triangles, we have tetrahedra, tets.
We'll see this in two dimensions, later in this chapter, when we're doing Laplace's equation.
It's just fun to see it work.
You'll have different triangles, say column triangles.
So that phi -- do you want me to look ahead?
Just ten seconds, to triangles?
So imagine we have triangles here, so we have piecewise, we have little pyramids.
Instead of hat functions, they grow to pyramids.
So there's a pyramid guy whose height is one there, and drops to zero in all these places.
And that's our phi.
Our trial function, test function, will be pyramid function, then.
And I can do integrals that way, or I can take the integral over a typical triangle.
So a typical triangle is involved with three, now I've three functions, in the linear case, controlling inside that triangle.
So what will be the size of the element matrix?
Can you sort of see how the system is going to work?
And then we'll make it work in 2-D. Every node, every mesh point, corresponds, has a pyramid function, has a U that goes with it.
Those U's are the unknowns.
And how many of those unknowns are operating inside that triangle?
Three.
So what will be the size of the element matrix, the non-zero part of the element matrix?
Three by three.
What else could it be?
So we'll see what it looks like.
And we'll have integrals over triangles.
So that's good.
Okay, thanks.
Exactly halfway through the hour is exactly that first topic of element matrices.
Done.
Okay.
Let me take two deep breaths, and move to fourth order equations.
Fourth order equations.
For the bending of a beam.
So I'd better draw a beam.
This is a 1-D problem, still.
This is a 1-D problem, still.
To keep it 1-D, this better be a thin beam.
So this is a thin beam.
And the loads, what's the difference?
What's here?
It looks like a bar, pretty much, right?
But the difference is, the load is acting this way.
The load is acting that way on the beam.
Maybe two loads.
Maybe a uniform load.
Maybe the weight of the beam.
But it's transverse.
It's in the perpendicular, it's transverse to the beam.
It's this way.
So the beam bends.
Let me do a fixed free.
So this'll be a fixed free beam.
Fixed free, the word for fixed free would be cantilever beam.
Okay.
So what happens if I impose those loads.
Well, the beam bends.
So the displacement is now downwards, is now not the direction of the rod, the displacement I'm interested in is perpendicular to the beam.
Downwards.
Okay, so we can start on our framework.
So this is displacement.
u(x).
I'll stay with the same letters.
So you know I'm going to have an A, that will take me to whatever this is going to get called.
What's the quantity there?
It's going to be, let's see.
What happens?
So this is just geometric now.
Let me put in the easy part, C. That'll be sort of the bending stiffness.
Right?
This'll be the bending stiffness.
Because the beam is going to bend.
And over here I'll get a suitable w. So when the beam bends, it's curvature.
Curvature of the beam is what's produced.
It's not stretching of the beam; it's curving of the beam.
So this quantity, e, will be the curvature that comes from the displacement.
If I displace these beams, suppose I put a node here, it's going to bend that down.
The bar will curve.
So the curvature, e. Now then, the question is, what is this A; what is the curvature?
Well, do you remember?
You're on the ball if you remember the formula for curvature.
It's a horrible formula, actually.
But that's only because we're going to make it better.
You remember the curvature, it was in calculus.
Yes, you all remember this.
Suppose I have a graph.
I know its slope, that's become easy now, right?
Calculus.
But the curvature of it, what derivative did it involve?
Second derivative.
And was it the second derivative exactly?
No, unfortunately there's some term which is horrible.
One plus the first derivative squared, all square root.
But I'm just going to take u double prime.
So this is an approximation.
So what is A now?
What's my matrix, A, that gets me from u -- or my operator, A, that gets me from u to e?
What's the e=Au equation?
A is just second derivative.
That's something new.
A is second derivative.
And why do I do that?
Because I assume small curvature, small displacement.
I assume that u' squared is very small compared to one.
So it's just slightly bent.
A beam that goes way down here, I'd have to go nonlinear.
But if I want to keep things linear, I approximate this will be much smaller than this, so one is fine that term goes.
And now the next step will be easy.
What's the bending moment?
This will be called the bending moment.
And let me use the letter w again.
I should really capital M for bending moment.
That will be the stiffness times the curvature.
So that's the force, the way the spring had a restoring force by Hooke's law.
This is the equivalent of Hooke's law.
But this restoring force is not pulling back, it's bending back.
It's torquing back.
And then, of course you know, there'll be an equilibrium equation to balance the load.
So this load is the f(x).
And you know what that'll be.
Because you know it'll involve A transpose.
And the transpose of that, do you want to just make a guess?
I shouldn't use transpose, of course, but I'll use it again.
It'll be the same; it'll be second derivative.
I mentioned we had a minus sign with the first derivative, but now we're going to have two minus signs, so it's going to come out symmetric, second derivative, and the equation here will be w''=f(x).
Our framework is working.
We have plus boundary conditions on u.
And here we have plus boundary conditions on w. So those parts we have not yet mentioned.
And of course, that depends on my picture.
While we're at it, why don't we figure out, what do you think is the boundary condition here?
And how many boundary conditions am I going to look for all together?
Four all together.
Because I have a fourth order equation.
There will be four arbitrary constants until I plug in boundary conditions.
So I'm looking for four boundary conditions, two at this end, two at that end now.
And what will be the two at the fixed end?
At the fixed end, obviously, it's built in.
Built in.
Slightly different words sometimes for the beam problem.
Here I'll have u=0 and u'=0.
Those apply with A; those go with A.
Those are the essential conditions, the Dirichlet conditions, the ones I must impose all the way.
And now at this free end, what do you think?
Well, it's great.
It's w=0, and w'=0.
It's just beautiful, the way it all works.
So that's a completely fixed free.
Then why don't I draw in, just while we're talking about boundary conditions, an alternative.
So here's my beam.
And now you see, it's under a load, f(x), transverse load.
Now that would be different boundary conditions.
Anybody know the name of a beam that's set up like that?
Simply supported.
You don't need beam theory, and I don't know beam theory, to tell the truth, to do these problems.
So that's a simply supported beam.
And what are the boundary conditions that go with that?
Well this is u=0.
No displacement, it sits there on that support.
And what else is happening at that support?
There's no bending moment.
Nobody's here.
Right?
So it's w=0.
And at this end, too.
Also at this end, u(1) is sitting there, and w(1) is zero.
Yeah.
That would be the boundary conditions, four of them, for a fourth order equation that we'll just write down in a minute, for simply supported.
And we could have a mix.
This could be simply supported here, free here.
I think.
Or maybe, could it be, or maybe that's too risky.
Would that be a singular case, simply supported?
Huh.
So as always with boundary conditions, some are unstable.
Some are not going to determine all four constants.
Just the way free free didn't work, right?
Free free for a rod didn't determine anything, it left a whole rigid motion.
Maybe u=0, w=0 at one end, and free at the other end; it sounds risky to me.
But we can see.
Okay.
So, do you get the general picture of the beam?
So what's the equation?
What's A transpose C A, when I put it all together?
I'll use that space to put in A transpose C A.
Continuous, we're talking here.
Right now we've got differential equations.
So what's the differential equation, A transpose C A?
So it's the second derivative.
I'm just going backwards around the framework, as always.
The second derivative of this, and this is c(x) times e(x), and e(x) is second derivative of u=f(x).
Good.
In ten minutes, we've written down the framework, some possible boundary conditions, and the combined A transpose C A equation.
I mean, we're ready to go.
We've got the pattern to think about this.
So let's see, what should we do first?
I would say the first thing to do is, let c be one and solve some problems.
Let c be one, and consider -- so if c is one, it's a fourth derivative equation.
Should we take uniform load?
Yeah.
How does a beam bend under its own weight?
So it's just one, or whatever constant.
So it's constant load, it's just its own weight, it's going to sag a little in the middle.
What's the solution to that equation?
And what shall I take as boundary conditions?
Let me do the simply supported one.
Because that would be kind of nice.
So it's simply supported, it's sagging under its own weight, with u(0)=0, u''(0)=0, because that's the w. And u(1)=0, u''(1)=0.
Whatever.
I don't know that I'll have the patience to go through and plug in all four boundary conditions to determine all four constants.
Just get me to that point.
Get me to a solution u(x), the general solution here that's got four constants in it is what?
Okay.
Okay, think again.
What are we looking at?
We're looking at a linear differential equation.
Linear problem.
I'm asking for the general solution to a linear equation.
What's the general set up?
General set up is, particular solution plus nullspace solution.
Right?
You see an equation like that, looking for the general solution, tell me one particular solution and then tell me all the solutions when it has zero on the right, and we've got everybody.
So that was true for matrices, it was true for Ax=b, it's just as true for differential equations.
So what's one particular solution?
What's one function whose fourth derivative is one?
Yes?
What am I looking for here?
1/4 of x to the -- no, what?
1/24, is it?
1/24 of x to the fourth?
x to the fourth over 24.
Because four derivatives -- so we're thinking we're in the polynomial world here.
Just as we were with u''.
With the bar it was x squared over two, the particular solution, now we're up to x fourth over 24.
So that's the fourth derivative is one, good.
So I'm seeing a fourth degree bending there.
And now what about the null space solutions, the homogeneous solutions.
This accounts for the one, now what are the possibilities if it was a zero?
You're going to tell me the whole bunch, right?
A plus Bx plus Cx squared plus Dx cubed.
Because all of those have fourth derivatives equal zero.
So that's the general solution.
Okay.
So whatever the boundary conditions are, they determine A, B, C, D. We're not that far away from Monday's lecture on fitting cubics.
Actually we're really close to it.
When we use finite elements, we're going to use exactly those cubics.
I'll get to that point.
Let me take the other model problem, that everybody knows what's coming.
What's the other right hand side that this course lives and dies on -- lives on.
Delta function.
Right.
Delta at some point.
So that's a point load then.
I can make it whatever the boundary conditions are.
Right.
Good point.
This boring stuff will just repeat, right?
That's the null space solution.
But now, what is a particular solution?
The particular solution has become interesting.
The particular solution here was straightforward, simple, a good one to do first.
What's a particular solution to a point load?
So instead of having distributed load here, I'm putting a heavy weight here at this point, a.
And it's heavy weight, I'm multiplying the delta function by one, I could multiply by some l for load or something, but let's just keep it simple.
What's a solution to that?
Fourth derivative equals delta.
So that means I've now got to integrate, one way to get the answer here would be to integrate four times.
Right?
If I integrate delta four times, then I've got something whose fourth derivative will match delta.
So do you remember the integrals of delta?
Okay, so I integrate.
First integral is, step.
Second integral is, ramp.
Third integral is, quadratic, right?
This was linear, boop boop, linear pieces.
The next integral is going to bring me up to quadratic ramp.
And the next, fourth one is going to bring me up to cubic ramp.
Cubic.
So it's going to be cubic ramp, is what I get there.
So one particular solution would be a function that's zero, and then at the point a, it suddenly goes up cubically.
So it's zero here, and it's x cubed over six there, I think.
If I do integrate three times, I'll be up to x cubed over six.
Is that right?
Yes, cubic.
So that's an interesting function.
Of course, these parts will tilt the function, will change it.
So our solution won't look like this, because I've only got one particular function, and I'll need these to satisfy the boundary conditions.
So there is one particular solution.
The general solution -- yes, good for us to think out the general solution.
What does that picture look like when I add in this stuff?
Very, very important.
I'm sorry I don't have more space for this highly important picture.
Okay, so here's my point a.
Keep your eye on that point a.
Okay, so to the left of it, I've got some curve, whatever, dut dut dut dut dut, the beam.
And to the right of it, I've got some other curve, whatever it is, dut dut dut dut dut.
And what's cooking at point a?
What's the jump condition at point a?
That's the critical question.
What changes at point a?
You remember, this is the corresponding thing, the analog of our ramp.
So what changed for the ramp at point a?
What jumped?
The slope.
Okay, now the question is what's going to jump here?
What jumped there?
Here, did the function jump?
Certainly not.
Did the slope jump?
Certainly not.
Did the second derivative jump?
No, no.
The second derivative was zero and then zero.
What jumped?
Third derivative.
The third derivative is allowed to jump.
And of course.
A jump in the third derivative produces a delta in the fourth.
Right?
It just works.
So this is a cubic of some sort, coming from that jump.
This is another, a different cubic, coming from this sort, from this junk, and this.
So it's cubic in each piece.
Why is it cubic in each piece?
Because, what's the equation in the middle of that piece?
What's our differential equation if I look here?
Here's my equation.
What is it in the middle of that piece?
u fourth equal zero.
The delta function is zero there.
And u fourth is zero here.
So of course this is a cubic spline.
We're meeting that neat word, cubic spline.
Those turn out to be very, very handy functions for other things, too.
So we see them here as a solution to fourth order equations with point loads are cubic splines.
Because the big key point is that there's a jump here in u''', the third derivative.
A jump in the third derivative, that's what we saw here.
And we'll see it if we have any cubic meeting any cubic.
Let me just say, a jump in the third derivative, your eye probably won't notice it.
I mean, it's a discontinuity, somehow.
We don't have the same polynomial from here to here.
But that discontinuity in u''', it's pretty darn smooth still.
The slope is continuous, so your eye doesn't see a ramp.
And even more, the curving is continuous, the curvature is continuous.
e and w are good.
It's just a jump in the third derivative.
Okay, so I want to speak about splines, and more about this, and about finite elements for beam problems on Friday.
And then that will take care of 1-D and we'll move into 2-D.
Okay.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So.
Today's lecture is partly cleaning up some pieces left from 1-D problems.
3.1 was second order equations, 3.2 was fourth order equation for beams.
And just a few more comments to make.
But then I do want to say something about splines.
And I included a homework question that asked you to use the spline command and get a result.
I'm not planning to make that a serious topic in the course.
In other words, you're not like going to be responsible on an exam for a discussion of splines.
But it's such a major event, and major requirement in scientific computing -- if you're given some values, put a curve through them -- that I have to say something about that.
So that's interpolation.
Because you'll have this all the time.
If your experiment produces some values at specific times or a finite number of values, and you want to fit a curve through those points.
How do you do it?
So splines give a very, very good answer.
And then, the next and big topic of the course we'll make a beginning on.
Which is gradient, divergence, leading to Laplace's equation.
Problems in 2-D.
So partial differential equations are going to show up.
OK.
Some small points.
First point, about the MATLAB homework for Monday.
A question came by email.
I put the spike and also the jump in c in the coefficient, not at mesh point.
If you've started on that problem, you probably noticed that, and maybe a little swearing went on.
[LAUGHTER] Because it makes it not so easy.
And then the question came, did I really expect you to compute, use the exact position and figure out, when you have some integrals to do, they'll change at the place where c changes values.
Where we have integrals of c, if c has one of those jumps, on the quiz the jump came right at a neat point, so it was clear.
c was one on one side and two on the other.
Now the jump is going to come at an in-between point.
So the answer is yes.
I had a nice email this morning from somebody who has done that MATLAB homework.
And he said, quote, it's a good problem.
I learned more from it than I did from the other problems in the pset.
So I'll blame it on him.
I'll stay with that problem, and ask you to do your best to deal with the calculation.
It's second order and I used linear elements.
I didn't want to push you up to these cubic elements, even though those would give much better accuracy.
One reason I didn't put the spike and the jump right at the node is that, if I did, the finite element solution would be exactly right.
Because the correct solution is going to be piecewise linear.
And if I put the breaks between pieces right at node points, well, then I have a function that's in my finite element space, in my linear space.
So it'll come out exactly.
So I thought well, let's at least have some chance to see the error between the finite element solution and the exact one.
You see what I'm saying?
I'm just anticipating, that the exact solution will -- let's see.
Something happened at 1/3 and something happened at 2/3.
I'm afraid I don't remember the boundary conditions.
Does anybody remember what I took for boundary conditions?
Probably free fixed, or something.
So maybe, if it was free fixed, probably the solution would be maybe something like that.
I don't know.
Whatever.
Piecewise linear with breaks, where there's a jump in c, or where the point load is hitting.
Yeah.
I don't know if that picture is accurate.
But my point was, if I chose those points to be also mesh points, then the finite element solution would be exact, and we wouldn't learn anything about accuracy.
OK.
So anyway, for that MATLAB problem, I'm hoping you can do the integrals, which will mean noticing which integral -- if that doesn't fall, so if the mesh points are like this, in the finite element integral over that mesh, over that interval, you're going to have to split it into two pieces.
That's it.
So see if you can do that split, do the split into two pieces.
OK.
So that's a comment on that homework.
OK. What else do I need to comment on, when I'm sort of catching up here?
Oh, about boundary conditions.
I thought I would just repeat clearly on the board the rules about and the difference between fixed and free when we're doing this weak form approach to the problem.
And then I gave the other names: a fixed condition, in this theory it's often called an essential boundary condition, but many people just name Dirichlet as the person's name that's associated with such conditions.
Like u(0)=0.
Right, fixed.
And then free conditions are natural conditions in the finite element method, and that means that we don't have to impose them.
Right.
I discussed this in the earlier lecture.
I just thought I'd bring it back here, because it's easy to describe in one lie.
Now somebody asked, on Wednesday, a good question.
Suppose u' is given, but not given as zero.
So suppose we have a condition like u'=A, let's say.
What to do?
So I'll just put question mark.
How do I deal with -- how do I?
-- how does the weak form approach and the Galerkin approach deal with a non-zero, natural condition?
A nonzero Neumann condition, that'd be the right word.
So this would be, in a second order problem, this would be a Neumann condition.
And I'm thinking, what do you do if A isn't zero.
Somehow that has to show up in your finite element equations, right?
So I guess the right way is to go back to the way they came from.
So you remember how the weak form came?
I took the differential equation, I multiplied by any test function, and I integrated by parts.
That's where the weak form started.
So the integration by parts gave me this nice symmetric integral.
It's symmetric in u and v. It only requires one derivative of u, so that I'm allowed to use hat functions.
And I'm allowed to use hat functions for v, because their derivatives have just a jump, and a jump function I can integrate, no problem.
Now what about this u'=A?
And I guess we're seeing it there.
If u' was zero, if we had a totally free end, u' was zero, nothing happening there, then that term would drop out of the integration by parts.
And you see why I don't have to impose anything on v, because that term is already accounted for by the u' going away.
Now, what about if u' is given, but not given zero?
Suppose it's giving the value A.
All I want to say is, then this term has to stay.
I have to pay attention to it.
What what happens when Galerkin takes over?
Galerkin will put in one of his test functions.
Maybe I just indicate that by changing the v to a cap V. It'll be one of them.
There'll be n of them.
Each V_i, each test guy gives me an equation.
And of course u is now going to -- the Galerkin idea is that instead of any u, I only have capital U's.
Combinations of these phis.
I'm not going to say anything very big here, or very clear.
All I want to say is that if u prime is given at an end point, and given by A, then this whole stuff is equalling something with an f, right?
f*V_i*dx.
I should remember the other side of the equation.
If u' is given, then c at that end point, times the given value of A, times the V will be -- whatever value of V that is -- will show up in equation i.
That will be a term that goes on the right hand side.
That's all I'm saying, and all you would expect.
That any time I'm given some data, I'm given some non-zero boundary conditions, or I'm given some non-zero f in the inside, that stuff is going to show up on the right hand side.
It'll show up as part of the f. The vector of these.
So up to now, f has just come from little f(x), integrated against V. And all I'm saying is that if we had one of these guys, then that would contribute to big F, also.
Yeah.
You can see, it's not beautiful.
But we have to realize what we would do.
OK, that's not my favorite topic.
But I'll just say quit on that one.
I'm not expecting you to do problems or anything that involved that possibility.
Just to see that it could happen.
OK. Quit.
Now, interpolation.
All right.
So I guess if I had to list problems that people face all the time and need numerical guidance on, this is one of them.
And so let's say we're in 1-D -- 1-D is certainly a lot easier -- so suppose I have this.
I'll call this x, just to have a name for the variable.
Suppose I have some values.
Say six values.
And I've measured them, I've worked hard to find those six values.
But now you may say, well I'm thinking I have some function F(x) here.
But right now I don't have a function.
Right now I just have six values of that function.
You know, what is the stretching constant, what is the c(x).
If you had a physical experiment with an actual spring, you might measure, you might put on different forces and measure the stretching, and have six values of that.
How do I fit those with a curve?
In other words, how do I know -- interpolation means, inter- means asking, what about between the points?
What value should it have there?
OK, well there's one simple rule would be, just interpolate linearly between.
OK. That's certainly pretty stable.
It will not get out of control.
But it's not very accurate.
For the function that's probably lying behind this, this isn't that great of a representation.
OK, you say, I want something smoother.
Well another idea that naturally comes up is, the other extreme would be -- so that was completely local.
Just, every two values determine the broken line.
The second idea that's completely natural would be, fit a polynomial.
Fit a polynomial through those points, and then you have a nice, totally simple function.
Nice curve.
And so what degree polynomial would we be looking for here?
If I have six points, I could fit a polynomial of degree -- what do you think?
I want to have six coefficients, so I can get six equations to make the polynomial go through those six points.
So my polynomial P(x), which actually goes through those points, would be some polynomial of the form, it's got some constant term, and it's got some linear term.
And how far am I going to go?
Fifth, right.
That'll give me six coefficients.
OK.
So I could put a fifth degree polynomial, that gives me a_0, a_1, up to a_5.
Six coefficients through six points.
Well, six isn't too bad.
But if six became 60, don't do it.
That's the message.
Don't do it.
If I'm going on up to, say I have an a 59x to the 59th, And that fits my 60 points.
And you say, well look, I've got 60 values, that should be way better than having only six.
But the polynomial that would come out of that, even if these values were pretty smooth, the polynomial that fits -- well I'd have to put in a whole lot more to get 60, and I won't try -- the polynomial would go crazy.
Can I just draw something crazy?
I mean, whatever.
It's unstable.
And there are classical examples of that.
In fact, the famous example is the function one over one plus x squared.
And later, there's a figure in the book, I think it's in section 5.4, which shows the result.
That's a terrific function.
It's infinitely differentialable, I would even call it an analytic function.
It's great.
But if I tried to fit it with a high degree polynomial at points, that polynomial gets out of hand.
And a figure is-- a figure there.
So, MATLAB or any computing system would have a subroutine that does interpolation.
And this is named after Legrange.
So this is called Legrange interpolation, fitting a polynomial.
and if the degree is small, and maybe degree five would be okay, then that's an important thing to be able to do.
In other words, you're finding these six coefficients from these six heights.
You've got some six by six matrix that connects the six heights to the six a's.
But when you get up to 60, that matrix stops behaving well.
It becomes very ill conditioned, and your coefficients go all over the place.
OK, so my question is, what do you do?
And one answer, one good answer is fit those points, not with straight lines -- that's too crude -- not with a single polynomial -- that's too unstable -- but fit it with splines.
So interpolation by splines.
OK.
So it's just sort of appearing in this section 3.2 -- I mean by that, and people often do mean, when they say spline, they mean cubic spline.
So you could have splines of other degrees.
But often, sort of the natural choice is the cubic.
So can I just briefly describe what that does.
What the cubic spline would be like.
So let me draw in again these six points that I'm going to fit.
And just say, for these few minutes, what's a cubic spline.
I touched on that Wednesday, and now I just want to say a little bit more.
And then the homework asked you to actually do it with values of a particular function.
And see how close does the cubic spline come to the given function.
OK, so what's a cubic spline?
What does that word, spline, what should that mean in your mind?
So a cubic spline is a cubic in each piece.
A cubic in each piece.
Now we've met cubics in each piece as finite elements.
And part of this short discussion is to keep those two slightly different ideas separate.
The finite element idea was also piecewise cubic.
But it used values and slopes.
So it was completely local.
If I had a value, as I have here, six values, and if I also had six slopes, if I knew the slopes, then I would use those cubic elements.
And I would fit A -- if I knew the height and slope there, and I knew the height and slope there, there would be exactly one cubic.
Because I would have four conditions, two conditions there, two conditions there, would be four.
I could fit a cubic between, another cubic there, another cubic there.
And because I'm using the same slope from the left and from the right, the slope would be good.
It would be continuous.
The second derivative, the curvature, would not be continuous with those cubic elements.
And that's the difference.
So the difference is, splines have continuous, no jump -- let me just put it this way.
No jump in the function.
I'll use S, maybe, for spline.
No jump in its slope.
And no jump in the second derivative.
So that's the difference.
That the spline functions, the only jumps you see for those are jumps in the third derivative.
So that makes this extremely smooth.
So if I just try to draw now what a spline would do, it'll go through those points, coming out of the spline fit MATLAB command.
And it'll be as smooth as I drew it.
There is a change in the third derivative at these points.
Actually, have you ever seen this word, "spline," before?
It came out of naval engineering.
When people were designing the shape of the ship.
A naval architect is fitting the proposed shape of a ship -- this was before MATLAB, before life started.
[LAUGHTER] They had little physical, slightly bendable -- I'll call them splines.
I guess that's what they were.
That's where the word came from.
Some physical thing which was like a curve that you use -- I don't know if you guys ever did mechanical drawing.
That was a freshman subject when I came to MIT.
Mechanical drawing.
I was terrible, terrible, at mechanical drawing.
I don't know.
I had a friend who helped.
l probably helped him in some other course.
Anyway, whatever.
So there were physical things, curves you used to use.
And these spline curves were used, and maybe still are used, by naval architects in creating a drawing.
But I'm assuming that those things are now all computerized, and the spline command is used.
Anyway, the result is that.
Now, I have to draw one picture of a spline.
Of the most important spline.
And then I'm done with splines.
So, what I'm going to draw is now a B-spline.
And that's a basic spline.
It could be one of our functions, it could be one of our functions phi(x).
One of our trial functions.
It could be, and let me comment on that.
So let me remind myself, good or bad, with a question.
And let me try to answer that.
But let me first draw the B-spline.
OK, so it's like a hat function.
Actually a hat function is a linear spline.
A hat function is that low level spline that's linear between pieces.
Now the B-spline is going to have the value one there.
It's going to have no jump in the function.
So the function will go through there.
No jump in the slope.
No jump in the second derivative.
And I want to get down to zero.
I want to get down to zero.
I want it to be as local as I can make it.
So I want it to get to zero.
Here's the point.
With those cubic finite elements, I got them down to zero.
They came in here with zero slope, and then they continued as zero, no problem.
If I'm wanting the second derivative also to be zero, to be continuous, I won't be able to do it in one interval.
It's going to take me a total of four intervals.
This one can come down to some point, here, where another cubic starts.
Maybe I should do the one here.
Going up, you remember the allowed cubic spline would be an x cubed over six, some multiple of x cubed.
I don't know what multiple it'll take, maybe x cubed over six is right.
It'll come up to some point here.
Now I've got to get it beginning to curve downwards.
So I'm going to have to change the third derivative.
Change to another cubic there, change to another cubic there, and a final cubic here.
So there's a picture of a B-spline.
And we could figure out, by requiring all these continuity conditions, we could figure out the formula for the cubic in these four pieces.
And it would be symmetric, of course, across that center point.
But I think I won't try to do it.
I'll just leave that idea, then.
There's a figure in the book showing a picture of the B-spline.
So those are functions, extremely valuable in this interpolation problem.
Because all splines are combinations of B-splines.
Yeah, now let me pull this topic together.
Every spline, every spline function, is combination of these B-splines.
Let B_i(x) be the one centered on node i.
And then there's a neighbor centered on node i+1.
And a neighbor to the left centered on node i-1.
What's the point?
The point is that, at a typical node, i+1, three of these functions will be non-zero.
With these very local hat functions at that node, the only one of the phi functions that wasn't zero is the one I've drawn.
All the others were zero there, right?
All the other hats were zero.
There was just the one.
But now there'll be a B-spline starting from here, going up to here, coming down from here.
There'll be the B-spline starting from here, going up to here, up, down, so on.
So at a typical node, I'm getting the one centered at that node, and also the one to the left, which is on its way down, and also the one to the right, which is on its way up.
In other words, it's not as local as the other construction.
So I would say good, because it's smooth, but bad because it's not fully local.
Not completely local.
It's reasonably local, in that there are only three functions that are affecting these points.
If I use those polynomials, they weren't local at all.
All the points -- it was a typical x to the fifth is not zero at any of the points.
These B-splines are zero at a lot of points, but three of them, one, two, and three, will be non-zero at a typical mesh point.
OK, so they're not popular, as a result, for finite elements.
I'm just wanting to be sure you make the distinction.
They're very popular for the interpolation job.
They're very popular for the interpolation job.
I've got some combination of these at particular mesh points.
It's supposed to agree with F at those mesh points.
This is my system to solve.
I create these functions, these B-splines.
I'm given F at typical points.
And I choose a combination which matches F at those points.
Yeah, OK. Is there a question or discussion on this?
I don't like to not say anything about such an important problem as putting curves through points.
But I don't want to make it a course on splines.
Yes?
[UNINTELLIGIBLE]
I don't know that I'll -- oh, OK, yes.
All right.
Two or three comments, and that suggests a good question, a very good question.
Can I make a separate comment that if I go into two dimensions, this gets much tougher.
What happens if I had a function of x and y?
So that I've got points on a surface, and I'm trying to fit a surface to it.
Just one message first: that's not as easy.
It's pretty easy if those points are on a rectangular grid.
So this is like typical.
And then I'll come to your Nyquist question, which is a very good one.
Can I just -- because we're coming into 2-D now.
Suppose, there's two dimensions.
I have a grid.
Let's suppose it's a nice grid.
And at every point, at every grid point, I have a height.
And I'm fitting a surface.
Right?
My function is now a function of x and y. x is here, y is here, F is the surface coming out of board, in the third dimension.
And fitting those points, if they're regularly spaced like that, my life would be OK.
I could use sort of products of splines in 1-D.
I could use products of basis, of splines in the x direction times splines in the y direction.
And it would be pretty successful.
It would be, not quite as nice, but almost OK.
But if I had irregularly spaced points from a general grid, it's not as easy.
And I won't -- people have obviously had to figure out how to do it, and to repeat again, that's what the CAD/CAM world is having to do all the time, is fit a curve, fit a surface through points.
It's significant.
Now, you asked about Nyquist.
So that's a good question.
Can I just say, I have to say -- so what's a function called?
Band-limited.
How many have heard Nyquist's name?
Okay, some of you may know a lot more than I about it.
But let me just get some context for band-limited functions.
Okay.
This is actually a topic that will belong in the third part of our course, in the Fourier part.
So this is a Fourier idea.
So I'll come back to it, actually.
So right here I'm just going to say, very briefly, how it might connect us to what I've said today.
But then let's make a plan to, when we've got the idea of Fourier coefficients.
What is a band-limited function?
You know the Fourier idea is to take F(x), and write it as some combination of pure frequencies.
e^(i*K*x), let me say.
e^(i*K*x).
So this is Fourier that's coming.
I take the function F(x), and I write it.
I could think of it as a combination of pure exponentials, pure frequencies.
That would be a Fourier series.
So that's a Fourier series because I'm using -- K has integer values.
The frequencies are zero, one, two, three, whatever.
Now that we'll use, so that Fourier series comes for functions that are periodic.
They repeat every 2pi.
Because those functions, if I increase by 2pi, don't change.
So I'm repeating every 2pi.
Now I have to say a word about the other possibility, which would be to have all frequencies.
I'll integrate, now, dK.
Instead of summing on K, I'll integrate on K. What does band-limited mean?
Band-limited means that only frequencies in a certain range, say a range around zero, are included.
So a band-limited function would be a function whose frequencies go from some value, say minus omega to omega, instead of going from minus infinity to infinity.
This would be band-limited frequencies between minus omega and omega.
So that's another kind of smooth function.
That's another way -- functions that have only low frequencies are associated with smoothness.
High frequencies are associated with fast oscillations.
So the class of band-limited functions gives me another, a Fourier way, to talk about smoothness.
So for us, smoothness was something about how many derivatives were continuous.
That's the sort of smoothness in the x domain.
How many derivatives.
Smoothness in the frequency domain is, how fast do the frequencies drop off.
And here, band-limited means they drop like a shot.
Band-limited means that the frequencies in the function, that these e^(i*K*x)'s are not there for high frequencies.
High frequencies are out for a band-limited function.
And then Shannon has a formula for the natural way to fit -- so our same interpolation problem.
So now, completing the answer to your question.
So I have these points.
So Shannon could fit a function, and it would look smooth, through those points.
And his function would be band-limited.
So it would be smooth again.
It would be -- yeah, it would be smooth.
In some way, this Shannon band-limited stuff is the limit of splines as the spline degree goes way up.
So we did hat functions, degree one splines.
Cubic splines I recommended as a pretty reliable construction.
But you could do fifth degrees splines, seventh degree splines, you could keep going.
And in the limit, you would get this.
So maybe that's some partial answer to the connection between splines and this Fourier world.
OK.
Thanks.
So these are topics now.
Wow, today's lecture is kind of -- can I do one really important thing now in today's lecture?
For you to remember?
Gradient and divergence?
I don't want you to spend the weekend without thinking about gradient and divergence.
[LAUGHTER] OK.
Here's the idea.
For lots and lots of applications, for a region, let's say, in the plane, -- what physical example shall I pick now?
Maybe I'll let u be the temperature.
So instead of being displacement, let me make it temperature.
u. OK. Then I have a temperature gradient.
A slope.
But now, the whole point is, that's a function of x and y.
Then I have a temperature gradient.
And if I'm consistent with the notation, that'll be e(x,y).
And then you'll expect that there's some c(x,y).
Some operator, c, that tells me how much heat flows.
This will tell me something about the the, the thermal conductivity.
Right?
Really, as I speak about this framework, I'm just uttering the correct words.
Having started with temperature, this thing should be a temperature gradient.
Then I should have some physical thermal conductivity, different for different metals or different materials.
And I'll have a heat flow, w(x,y).
And everybody knows that w will be c times e. And then there will be some A transpose.
Of course, there will be some A transpose here, and some A here.
And up here I'll have a balance equation.
OK.
I just want to think, what's the operator A?
If we can focus on that question, then that's what's going to occupy us for, certainly the whole of next week.
So I've actually used the word gradient.
We have functions of two variables.
We're looking for the change, the rate of change, the steepness of those functions.
So this A, Au, is going to give me two derivatives.
I've got two variables, there are two first derivatives.
Both of them are important.
That's what the A is.
For the next big example in the course.
The final major example of the course is, when a acts on a function of two variables, because I'm in a region in the plane, to find its rate of change.
And this is called the gradient.
The shorthand is the gradient of u.
So we have to understand that.
We have to understand what the gradient is.
And, of course, we want to know its transpose.
So can I just think, what should be the transpose of the gradient?
OK.
I'll take that picture out.
OK.
Thinking now about the transpose of the gradient.
OK.
So a itself is, you could say, is d/dx and d/dy.
You notice how I'm separating out A from Au.
When this acts on a function, this is the gradient operator that acts on a function, u, to produce du/dx and du/dy.
OK. Now what's the transpose of this?
OK. You can guess what it should be, and then we'll see that yes, that guess is correct.
So what should a transpose look like?
If there's any justice, a transpose should be -- OK, this is two by one.
The transpose you would expect to be a row, a row vector.
I should have the transpose of that there.
And what is the transpose of that?
Just tell me.
Because we have an idea.
What should be the transpose of d/dx?
Negative d/dx.
And what should be the transpose of d/dy?
It should be negative d/dy.
Those are two different pieces.
This is not run together.
There's a big space in there.
That's two pieces.
So what is A transpose applied to -- heat flow w. I want to say, what is A transpose applied to w?
You're going to see this again, but we'll just take these minutes to show it for the first time.
So A transpose -- wait a minute.
Is w a function, or is it a vector?
Yeah we've got to get that straight before we start.
Here's an ordinary function, a scalar function.
Just whatever, x squared plus y squared.
What is e?
Suppose this is x squared plus y squared.
Let's have a specific example.
What would e then be?
It's got two components, right?
It's got an x derivative and a y derivative.
e is a vector, .
Then I multiply by c. So this w has two components.
This has got a w_1(x,y), w_2(x,y).
So just keep things straight.
So it's that that I want to apply A transpose to.
So A transpose w is minus d/dx, minus d/dy.
And everything's coming out right because it's applied to a function w_1 and w_2.
Two is a vector field.
It's not a scalar field, it's a vector field.
And the result is just what it should be: minus dw_1/dx, minus dw_2/dy.
OK, good.
This has got to come out of integration by parts, right?
So we'll have to think: what does integration by parts mean in two variables?
And it's a famous formula named after Gauss and Green.
The Green's formula, often.
But do you recognize what I'm looking at here?
This is so important it has a name.
And what's the name?
And then we're ready to go.
What's the name of, if I take a vector field, like .
Let me take , as a specific example.
Suppose w is , and was multiplied by, let me take c to be one.
Then what is A transpose w?
Specifically.
What am I getting out of it?
What do I get from here?
That's minus the x derivative of the first guy, and the x derivative of that guy is two, so I'm getting a minus two.
And this is minus the y derivative of the second guy, so that's another minus two.
So I'm getting a number.
It happens to be a number here.
I chose such a simple function it came out to be a number.
And what's the name?
So this is minus the what of w?
Just tell me, what's the name everybody uses for that operation?
The divergence.
Minus the divergence of w. So I what I'm saying here is that this -- I'm saying it because somehow I remember studying vector calculus.
And in that process, I learned about the gradient, and I learned about the divergence.
But I never learned that one was the transpose of the other.
I think, looking back, that was criminal.
To describe those -- with a minus sign, of course.
I learned Green's formula, but now we'll see what it means.
OK, that's next week's job.
Have a great weekend and see you Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So this is lecture 22, gradient and divergence, headed for Laplace's equation.
So the gradient will be our operator A, the divergence, or minus the divergence, will be A transpose, and then A transpose A will be the Laplacian.
We get to Laplace's equation Wednesday.
Today I wanted to take them separately.
To understand the meaning of gradient, the meaning of divergence, the connection between them.
I mentioned at the end of last time that one is the transpose of the other, or minus the transpose.
I'll try to keep gradient on this side, and if I could only transpose the blackboard, I could do divergence -- I'll do divergence on that side.
And I guess if I could get a rotating blackboard, right in the middle I could do curl.
That would be perfect.
[LAUGHTER] OK.
So some of this will not be new to you.
But maybe some of the insights or the ways of looking at it could be new.
This is the background of vector calculus.
So we have things like vector fields.
That means I have a vector, , at each point, (x, y).
So I could draw a little arrow at every point to show the direction and magnitude of that vector.
I have a field of vectors.
OK, so.
From last time, our basic setup is, the gradient is this first operator, A.
The one we see at the beginning.
One change.
Instead of calling the result e, let me connect to velocity.
I'll be thinking of u as a potential, I'll use the word potential for u, and I'll use v instead of e for the velocity component.
So that's the A.
And then on the other side, I start with a w. Again, it's a vector field, it's actually a momentum.
Very often the step between here and here -- well most often, the step between here and here will be the identity.
That's what gives Laplace's equation.
So you'll have to watch, I'm sometimes confusing the v's with the w's.
Because when I go to Laplace's equation and c is the identity, they're the same.
But I would like, today, to try to keep this left side separate, the gradient, from the right side, the divergence.
And understand what they mean, and how to work with them.
And of course, a big connection is the divergence theorem, or the Gauss-Green connection, identity.
We'll get to that.
OK, gradient first.
First, what does it mean?
If I have a function u, what does it is gradient tell me?
And then the second is kind of a backwards question.
Suppose I have the v. Is it the gradient of some u?
So one direction, from u to v, and the second direction will be from v back to u when possible.
OK, meaning of the gradient.
So the gradient is just the obvious thing, the two derivatives in the x and y direction.
So of course the gradient gives you the rate of change, the partial derivatives of u.
But how do you see that in a picture?
Let me draw an important curve.
I'll start with a very simple u. u is x squared plus y squared.
So that's my example.
Example one.
And what I've drawn is an equipotential curve.
Or isopotential might be a more appropriate word these days, but we still say equipotential.
It means that, along this curve, u is a constant.
And for this particular potential, this simple one to work with, x squared plus y squared, the curve is a circle.
So the curve would be a circle.
OK, now what do I learn by taking the gradient?
So the gradient of u, this is my v, is the x derivative, which is 2x, and the y derivative, which is 2y.
OK. Let me take a typical point on the curve and draw that gradient vector.
So this is the x, y plane, this is the curve u equal constant.
When I have a curve u equal constant and I draw the gradient of u, where does it point?
This is the first and most important, simple idea about the gradient vector.
The gradient vector points -- does the gradient vector point, could it point any old way?
No.
The gradient vector is perpendicular to the curve.
And we can see that, for this simple example, that vector , that's a vector radially outwards, right, if here's the origin.
And if, at this point, I don't know its coordinates, whatever they are, maybe 2, 1 or something, the gradient vector would be .
It would be a multiple -- here's the position vector, .
The point is, the gradient vector points out.
Perpendicular to the curve.
That's what the gradient tells you.
It tells you, in this situation it's telling me which direction is perpendicular to the curve.
Now how do I understand that?
How do I see that?
I think of this -- and let me try to draw the whole surface, u=x+y^2.
What would that whole surface look like?
Let me try to put that picture -- this is a 2-D picture right now.
It's sort of a cross-section.
Let me put this 2-D picture into a 3-D picture to see.
So the 3-D picture is a picture of the whole surface.
u going up, x and y going around.
And what kind of a curve, what kind of a surface do I have for that function, x squared plus y squared.
Well, it grows, right?
It's sort of, I use the word bowl.
And we've seen it before.
It sort of goes up, right, and it goes up faster and faster as I go out, because of the squares.
And now what I want to do is, suppose I take the cross-section, suppose I take this bowl.
Do you see a beautiful bowl there?
And now, I cut through it, horizontal cross-section, at the height c. What do I get if I take this surface in 3-D, and I cut through it by a plane, the plane at that height, c. So if the plane went through there, that distance was c. It cuts us a cross-section out of the bowl.
And what's the cross section?
It's the circle.
It's this one.
So out of the bowl, this plane is cutting this cross-section.
This is height u equal constant.
So I'm really intersecting one surface, the plane u equal constant, with the bowl.
And the intersection of those two is the equipotential, the curve, u=c.
Right?
You see the two pictures?
And now, what does the gradient tell me in that picture?
What does the gradient of u tell me in that picture?
Over here, it pointed outwards, but now we're going to kind of see why.
So, it pointed outwards for this nice function, u. I could have made a more general function u, but let me just stay with this simple one.
So suppose I'm climbing.
This is like, I'm climbing out of a volcano or something.
Usually, I would say a mountain, but the way I've drawn the thing, it's not much of a mountain.
So OK, volcano.
Climbing out of it.
And I got up to this point, which was this point.
Now, what does the gradient tell me?
That when I'm climbing away, I reach -- duh duh duh duh -- I get up to here.
What does the gradient tell me?
It tells me which way to go.
It tells me the steepest direction upwards, and it tells me how steep.
Right.
So it tells me those two things, direction and magnitude.
Direction is, well for this special function you know the direction is, like, straight outwards, straight upwards.
And how steep is it?
What's the steepness?
It's the size of, it's the magnitude of this vector, grad u.
Which is the-- this is not -- how big is this vector?
This is my vector, grad u.
This is another shorthand notation for gradient.
The size of that vector is the square root of, what?
The length of a vector is the square root of the sum of the squares.
So I have the length of v is the square root of 4x^2 + 4y^2.
OK. Now, it gets steeper and steeper, of course.
The cross sections here would be all circles for this simple function.
And the gradients would keep pointing out, and we'd have a function that comes from a surface like that.
OK, this is what I wanted to say about question one, the meaning of the gradient.
The thing to remember, if you remember the two pictures, you've really got the idea of what the gradient is telling you.
For a function of one variable, the derivative is telling you the slope.
Well, we've got slopes in the x direction, slopes in the y direction, and the gradient direction tells us the steepest slope.
And it tells us -- yeah, tells us what that slope is.
OK, so much for the meaning of the gradient.
Now, backwards.
Suppose I have v. Because these gradient fields are extremely important.
I mean, they're wonderful fields.
If you've got a v_1 and a v_2 that is the gradient of some u, you want to know that.
I mean, that's good news.
Most fields will not.
So let's figure out, when is -- can I life that up now, and do the other gradient thing now?
Or maybe I'll put the final gradient blackboard here, and then make way for divergence.
OK, so I'm asking now question two.
I'm given v_1 and v_2, and I want this to be du/dx for some u, and I want this to be du/dy.
So I've got two equations.
If I'm looking for u.
Remember, I'm now starting with these, looking for the u. OK.
So, am I going to find Au?
Am I going to find a function, u, whose x derivative is my v_1 that was given, and the y derivative is my v_2?
Well, chances are not good.
Right?
I've got two equations here, but only one unknown.
So generally, it's like a rectangular system.
Usually there's no solution, but sometimes there is.
So we have to find out, what's the test for consistency for these to have a solution?
And this brings us right away to the key identity from partial derivatives.
OK. Can you see some condition that has to-- how can I connect these two equations, and therefore connect v_1 and v_2?
That'll be the test for weather -- you could say, in matrix language, I'm asking whether is in the column space of the gradient.
Is it something I can get from this tall thin matrix?
OK, they key idea is take the y derivative of that equation, d second u/dydx.
And take the x derivative of this equation.
dv_2/dy.
dx is the second derivative of u, respect to the x derivative of the y derivative.
OK.
So it's like I'm operating on the equations.
I'm doing what I'm allowed to do, I'm doing the same thing to both sides of each equation.
Now I'm going to eliminate.
And what am I going to find?
What's the key point here?
The key, key point about second derivatives is that that equals that.
Right?
Take any function.
Do we want to practice with a function, just to see it be true?
Take, let u be x cubed y.
Just for the heck of it.
OK, I don't know what -- is that going to produce anything interesting?
Then maybe I'd better make it y squared.
I don't know why I'm doing this, it just is like an example, to make it believable.
OK, so du/dx, u_x is -- and I'll often use u_x as a sort of shorthand -- will be what?
3x squared y squared.
And u_y, taking the y derivative is just, x is constant, 2x cubed y.
And now let me do u_xy, the y derivative of this.
Which is what?
It's been a long weekend, but hey, we can do these.
The y derivative of that is going to be 6x squared y.
And the x derivative of this is going to be, so I should say y -- I've got the y derivative and now I should take the x derivative of that.
And what do I get?
The x derivative of that is 6x squared y.
And look!
They're the same.
Hooray.
OK.
So if the function is smooth and has these derivatives, they'll come out the same.
And therefore, so what do I conclude?
What's the test on v_1 and v_2 that it must pass to be a gradient field?
For there to be a function, u, that solves these equations.
These are solvable only when what?
Well, if these are the same, these have to be the same.
So it's solvable only when -- I need dv_2/dx-dv_1/dy to be what?
Zero.
OK. That's the conclusion.
Those have to be the same.
dv_2/dx-dv_1/dy has to be zero.
OK. Because those are the same.
I guess I came to this early because it's the key identity of vector calculus.
Well, the key identity behind vector calculus is this fact about the derivatives.
Can I just throw out a question that you might think about?
We already have seen, so often now, the second derivative, like v_xx.
Or u_xx, suppose.
So here's a little question to think about.
So think.
OK.
I just want to bring finite differences in for a moment.
u_xx we've got a handle on.
We know that that's like u(x+h).
And if there was a y, put in the y, minus 2u at x.
And I can put in a y there, plus u(x-h,y).
We've seen that.
That'll be the x derivative, second x derivative -- sorry!
Second x difference.
Since we're taking x derivatives and x differences, it's x that moves, and y doesn't move.
Just the central idea of partial derivatives.
u_yy will be similar with y moving.
And my question to you is -- and we could have asked it way way back -- what's a finite difference approximation to u(x,y), to the cross derivative?
That equals what?
I want to go to finite differences.
OK, what finite differences?
Which finite differences?
Maybe that's a question to think about.
If I can remember, I'll include it just as a small homework question for the homework on this material.
So that's looking ahead, really, today.
We're not making things discrete.
We're in continuous x and y.
We have vector fields.
And we now know the test for a gradient field.
I'm tempted to use the word curl here.
I'm tempted to use the word curl.
I want to connect that test -- may I use the word curl without -- and I'll say why I'm not going to do everything properly with curl right away.
I would describe this as curl.
The test is, the curl of v has to be zero.
So for me, that's the curl of v. You're going to say, wait a minute.
I learned about curl, and that doesn't look like the curl to me.
So I'll say wait another minute.
It's not that far off.
OK, what's your objection?
Your objection is that the curl is in three dimensional space.
Right?
When you saw curl, and of course it comes in this section of the book, we had functions of x, y, z.
And the curl had three components.
And those three components -- do you remember curl?
I mean, if you remember curl, you're a good person.
Because it's got this -- you sort of remember that it has things like this.
Right?
Sort of, differences of derivatives, and the indices follow a certain pattern.
I'm saying that this is the natural -- this is the curl in a plane.
What do I mean by the curl in a plane?
So in 3-D, v has components of v_1, v_2, and v_3.
Right?
That depend on x, y, z.
In the plane, in 3-D, I have a vector field, v, which has components v_1, v_2, v_3.
All depending on x, y, z.
That's the general 3-D picture, where you usually see the curl.
Now, in the plane, what's happening?
In the plane, we have two components.
So what's happening?
Think of this.
So in 3-D, our velocity field is like, the flow is going all over the place.
Right?
It's a three dimensional flow.
But now suppose my flow stays in the plane.
So in 2-D -- so now I have to put 2-D up above here.
So in 2-D, a plane field is what I'm working with today.
My v is some v_1(x,y).
No z, no dependence on z. v_2(x,y), the y direction of the velocity doesn't depend on z, because this is a plane field, same on every plane.
And the third component of this plane field, the velocity perpendicular in the z direction, is zero.
OK. one.
So what I want to say is that if I look, if I specialize too plane fields, to fields like these, then the only component of the curl that survives is this one.
See, the other components of the curl -- which I'm not even writing down -- the other components of a curl have derivatives of v_3, but v_3 is zero.
And they also have derivatives of these guys with respect to z.
But they don't depend on z.
So that's why all the other pieces of the curl, like, are automatically zero for a plane field.
So that the only component that's significant, the test curl v equals zero, boils down to a test not on three things, but just on one.
And that's the one.
So because I want to stay mostly with plane fields and two dimensional problems, I just had to comment that, if the curl was to get in here, it would fit fine.
And if I restrict the curl to the fields I'm working with, plane fields, then there's only one component I'll have to think about, it has to be zero to have a gradient field.
OK, now I guess I should just do an example or two.
Can I give you a v_1 and v_2, and you tell me, is it a gradient field or is it not a gradient field.
Let me give you a different, let me just change these guys.
Suppose I change that to y and x.
So there is a v, a different v. That's a vector field.
At every point x, y, I've got a little vector.
I could try, even, to draw them.
And I'm going to ask you, is it the gradient of any u.
And if it is, what's that u?
So let me show you what I mean by a vector field.
I mean, at a typical point like x=1, y=0, the vector -- let's see, if x is one and y is zero, then what's the gradient at that point?
, am I right?
I won't draw it too big, or you won't be able to see a darn thing.
OK, what about at the point (1, 1)?
Which way is my vector field going?
.
So what's that look like?
Plotting the vector field, v, at a bunch of points.
So you get like a map of little arrows.
So here it would go that way.
Is that right?
Huh.
I wasn't expecting that, to tell the truth.
Let's see, so can I get some more points?
Let's see.
What if I have, there.
What's -- at (1/2, 0).
Then, v is .
Where is this flow going?
See, if the point is along this line, where y is equal to x, then the flow was going out along this line.
Can you give me some other point here, just so we get some handle on this?
There's the point (2, 1), let's say.
Let me put it over a little bit.
How about the point (2, 1).
What's the vector if I just want to draw -- I'm just drawing here.
And of course, code would do it much better than I'm doing.
(2, 1), that gives me , right?
So at (2, 4) is over two and up, it's like this.
I think -- but I don't swear to it -- that if I connect all this -- see, now you have to take a big leap of faith.
Imagine, like at every point we've got these little arrows, and I want to connect them up.
Let me do something.
I'll do that, but let me come back to the question I should have asked you first.
Is this a gradient field?
Does it satisfy the curl zero condition that we put in a box here?
Does that satisfy dv -- there is v_2 and there's v_1.
Is dv_2/dx, whatever that test was, minus dv_1/dy equal zero?
Yes, no?
Yes, right?
dv_2/dx is two, and dv_1/dy is two.
So what's the conclusion then?
It satisfied my little test, so this must be the gradient of some u.
Right?
That's the question we have.
Which vector fields -- and we found that this is one of them, it passes the test.
It's the gradient of some u.
What's the u?
What's the function, u, which is supposed to exist, whose gradient is that.
2xy.
Did everybody spot that one?
2xy.
Because the x derivative has to be 2y.
So I just integrate with respect to x.
This is the x derivative.
It's 2y.
Take the integral with respect to x, and I get 2xy.
And there could be some term that depended only on y.
Anyway, this works.
I think that this, maybe gives me somehow -- oh, yeah.
What's the equipotential curve now?
Oh, yeah, this picture's going to come together.
What is the equipotential curve for this potential?
It was a circle for the first guy, but circles are out now.
I changed v. I've got a new potential function.
And now I want to draw, in this graph, the equipotentials.
Suppose u is one.
Suppose I draw the curve 2xy=1 in that picture.
What kind of a curve is it?
Do you recognize this?
The Greeks would.
Recognize 2xy=1.
Or you could say y=1/2x.
That gives you a quick handle on the curve.
It comes down like that.
Right?
And what's the Greek name for that curve?
Oh, come on.
It's a hyperbola.
It's a hyperbola.
Hyperbolas -- you remember the Greeks, they had these conic sections.
They had ellipses, they had parabolas, the marginal case, and then they had hyperbolas.
And they all come from second degree things.
If I have a x squared and 2bxy and cy squared equal one, that's one of those curves.
And if it was x squared plus y squared equal one, it was a circle.
If it was x squared plus 7y squared equal one, it would be an ellipse.
The positive definite -- it all comes down to linear algebra, of course.
If that little matrix is positive definite, so that means a and c are positive, and a c is bigger than b squared, you know the test for positive definite.
What kind of curve do the Greeks have?
What kind of equipotential -- what kind of a curve have we got here?
An ellipse.
If this curve, if that little matrix is indefinite, as, for example, here.
So with this one, what would be the matrix, what's the matrix that goes with 2xy, if I match this with this.
That's the matrix.
There's no a squareds, there's no x squareds, there's no y squareds.
And there are 2xy's, I think the matrix is that.
So it's this nice symmetric matrix.
Is that a positive definite matrix?
Certainly not.
It's indefinite.
It's Eigenvalues are plus one and minus one, adding to the trace zero.
So indefinite matrices correspond to hyperbolas.
And later on, definite matrices will correspond to elliptic partial differential equations.
And indefinite matrices -- like Laplace -- and indefinite matrices will correspond to hyperbolic partial differential equations, like the wave equation.
What's the -- now we're here.
I didn't expect to get here -- what's the marginal case?
What's the marginal case between positive definite and indefinite is -- semidefinite.
Great.
And what kind of a curve do you think comes when this little matrix is semidefinite.
It's the one in between ellipses and hyperbolas.
The marginal guy is a parabola.
Right.
So semidefinite would correspond to a parabola.
Right.
OK. Good.
Anyway, all I was going to say is, this u=2xy, that's our potential.
If I draw equipotential curves, they're hyperbolas.
And now, what's the point about these little arrows that I got started on.
What was the very first point about the answer to the meaning of the gradient was what?
These are the gradients of u, so those arrows point where?
Perpendicular to the hyperbolas.
Perpendicular to the hyperbola.
We're trying to see the geometry -- it's beautiful geometry -- behind the gradient.
So if v is a gradient, then it comes from some u. I can plot the u equal constant, the equipotential, and then the gradients will be perpendicular.
So they really are a little -- OK, good.
OK, those are pieces of information that you have, but always need saying again.
And to get the picture in your mind -- I suppose, finally, I should choose a v which is not a gradient.
Just to finish.
How shall I adjust that v?
This v was a gradient.
Can you just change it a little bit -- practically anything you do will screw it up -- to make it not a gradient?
So I just changed this two, what shall I change the two to?
To three.
That would totally foul it up.
So that vector field, which I could draw little pictures of, but there would be no u that it's coming from.
There would be no u.
These little arrows would not line up perpendicular to some beautiful curves.
I don't get a u from that.
Because the y derivative of that is three, and it doesn't equal the x derivative of that.
So that's a no good one.
Let's go back to the good one.
OK. OK, good.
Is that OK for gradients?
We got the meaning of gradients.
They point perpendicular to equipotentials.
They tell how steeply those -- they tell the separation between the equipotentials, right.
It's like, if you're a mountain climber, you're looking at your map, your contour map, and that's all I'm drawing here.
I'm drawing a contour map that every guy who goes climbing in New Hampshire is going to have.
And it shows little circles, those are level heights, right?
Those are level contours.
And if you want to climb as fast as possible, you go perpendicular to those contours.
And the distance between contours tells you how steep it is.
So it's all nice geometry.
OK.
I've got to get to divergence here.
Divergence.
I should have said though, -- damn.
There's more to say about gradients.
That question of whether <v_1, v_2>, the vector field, is this question.
The question of is it the gradient of some u.
So we now have a test.
We now have a test.
This is our test, right?
That's our test.
But I have to connect it with Kirchhoff's voltage law.
Do you remember, we haven't talked so much about Kirchhoff's voltage law, but I'm connecting it with the discrete case, to add in a little more insight.
What did Kirchhoff's voltage law say?
In that case, a was a difference matrix.
It was the incidence matrix for our graph.
And the question was -- I have to take two moments to think about that.
So Kirchhoff's voltage law.
For a graph.
a is an incidence matrix.
You know, the minus one, one guys, one for every edge?
And let me call it v again, or e. I called it e at that time.
Let's just look at an x.
Right.
A is this long thin matrix, times -- sorry, u's.
u's.
Let me say it all at once.
Which vectors have the form Au?
Which vectors are combinations of the columns of A?
The test is, Kirchhoff's voltage law, that if I go around any loop in the graph -- so if I have a u_1 here, u_2 here, u_5 here, and u_7 here -- then a u will produce u_1-u_7 on that edge.
It'll produce a u_2-u_1 on that edge.
It'll produce a u_5-u_2 on that edge, if the edges are all going that way.
And it'll produce a u_7-u_5 on that edge.
So I've got four components of Au, four differences.
And what does Kirchhoff's voltage law tell me about those four differences?
Which I can certainly see directly.
Those four differences, u_1-u_7, u_2-u_1, u_5-u_2 and u_7-u_5.
What's the obvious fact about those four guys?
They add to zero.
The total drop around a loop is zero.
You see, if I cancel those, if I add them, the u_1's cancel, the u_2's cancel, the u_5's cancel, the u_7's cancel.
We know this.
OK.
So that's Kirchhoff's voltage law.
It's got to have a continuous form.
This tells me, this is the test on v at a point.
What's the test on v around a loop?
I just want to connect that -- I have to connect that to a second test.
I'll just mention it, and you'll find it in the book.
That's the pointwise test.
That was the easy test.
We applied it to this and we got the answer yes.
If it was 3y, 2x, we got the answer no.
Now let me give you a test that looks like Kirchhoff's voltage law.
So I'm going to integrate around a closed loop.
What am I going to integrate?
I think I integrate v -- oh boy, I'd better look.
It's easy to get these wrong.
Yeah.
So I would call this the vorticity.
And then I would say the vorticity is zero for a gradient field.
Now my integral guy is going to be the circulation.
Oh yeah.
Because I'm following the path.
So it's just v_1*dx+v_2*dy should be zero.
Around every closed loop -- that's idea of this thing, that it tells me the integral goes around a closed loop -- if I follow the velocity field, the total circulation is zero.
I put this up here as a fact in vector calculus that's connected to that.
These, one is zero and the other is zero.
There's a Stokes' theorem that tells me that this integral is found from a double integral of this.
So if one is zero, the other is zero.
I'm just saying, here is the natural analog of Kirchhoff's voltage law.
OK.
I had to say something about voltage law, because for the divergence, which I'm now going to get to -- whatever.
Let me ask about divergence of w=0.
What does that mean?
That's going to be the equivalent of who's law.
Please tell me.
Which law is going to be the equivalent of -- divergence of w=0 is going to mean there's no source.
Whatever goes in, comes out.
Whose law is that?
That's Kirchhoff again.
Well, yeah, other people in physics.
Right.
But in our little world, it's the other Kirchhoff law.
It's Kirchhoff's current law.
It's the one, it's the A transpose Right?
This is what we're thinking of as A transpose w equal zero.
Kirchhoff's current law, in equals out.
How will I translate that in equal out for functions?
Now I don't have -- on a graph, I just had the total flow, the net flow at every node.
Notice the divergence is at every node.
The circulation was around every loop.
OK.
So in equals out was just the sum of four things.
OK, here I'm going to have in equal out -- how am I going to express in equal out?
Divergence of w equals 0.
Yeah, what I need is the divergence theorem.
Let's just face it, we've got to have that.
So I have a region here.
I have a w everywhere, w, .
Then the divergence theorem.
This is the great identity, which of course has a discrete form.
OK.
The divergence theorem says that if I integrate over the region, over this region, R, the divergence, that's (dw_1/dx+dw_2/dy)dxdy.
So that's like telling me the source, I'm integrating over the source at every point.
At every point here, this measures in minus out.
But now, when I put the whole thing together by integrating, what's the right hand side of this equation?
Do you know the divergence theorem?
And let's remember it and see why it's so.
What I'm doing is in equals out for the whole region at once.
Right?
When I this is like in equal out at a point.
But now I'm putting all the points together.
So the only way out will be out through the boundary.
And so I'll need to say how much flows out.
This is the total source, the total in equal out inside.
The only way to get out is through the boundary.
So this is the integral around the boundary of -- so what's the flow out?
It's, yeah, it's somehow -- think now what should go there.
This is flux I'm talking about.
Flux is short word for the total flow out.
OK.
So now I've got to get this right.
In vector notation, it would be -- w tells me the flow.
But flow outwards, see, suppose w points that way.
Then the actual flow out is not all that.
Because a lot of that is just going sideways.
It's this part.
It's the flow perpendicular to the boundary.
So it's w dot n, the normal component of flow.
And I integrate that around the boundary.
There you have a key, key theorem.
In 2-D. And it's an equation for the flux.
It's like the fundamental theorem of calculus, but now we're in two dimensions.
And this is what it looks like.
OK, so I'm obviously not going to finish with the divergence theorem today.
So what's the conclusion?
If the divergence is zero, then what?
If the divergence is zero, if this is zero at every point, then this is zero across every loop.
Can I call this thing a loop?
That closed loop.
That's the conclusion that we want to reach.
So this is the divergence theorem.
The text gives a proof, not to repeat in class, but it's a crucial formula to know.
That the integral of the divergence is the flux.
OK. Let's come back to that Wednesday, and I'll have lots of homework for you.
Thanks for turning in these today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So this is a fun lecture.
This is the lecture where understanding the gradient, which we did last time, the divergence, that we almost completed, those two pieces now come together into Laplace's equation.
Things work in a fantastic way, so I enjoy this one.
So the result will be that we have the potential function, u, that we've spoken about, and take its gradient.
Now is coming a stream function, S, that's connected with the divergence.
When the divergence is zero, there's something called a stream function, S. And these, the connections between those two functions, u and S, are crucial.
And connecting gradient to divergence will take us to Laplace's equation.
And then you'll see the special, special role of x+iy in 2-D.
So this is in two dimensions.
OK.
So let me begin with divergence.
This is the divergence of w, of course.
And solve it.
Just as we wanted to solve Kirchoff's current law, A transpose w equal zero, so now in the continuous case, we want to find solutions to -- we want to find divergence-free fields.
Source-free fields, you could say.
That's about the best word we have, divergence-free free, meaning there is no divergence.
So what have we got here?
We've got one equation in two unknowns.
Last time, with the gradient business, we had two equations for u, and only that one unknown.
Now we've got one equation, two unknowns, because we're looking at the transpose.
So there should be a lot of solutions.
Right?
If you give me a w_1, then probably I'll be able to find a w_2.
But there's a neat way to describe the solutions to that equation, the divergence-free fields.
It's to introduce something called a source function -- a stream function, sorry, stream function.
OK, and now the idea will be that if I let -- let me try.
I take any function S(x,y), any function whatever.
So now I try -- let w_1 be dS/dy.
Maybe I should say, maybe I should go this way.
I'm looking for solutions.
OK.
So I take any function, S, I take its y derivative to be w_1.
And you can tell me what w_2 has to be.
So if w_1 is dS/dy, this will be what?
This'll be the second -- the x derivative of the y derivative.
Right?
If w_1 is the y derivative, then when I take the x derivative, I've got this cross derivative.
So what would be the smart choice for w_2?
So I'll say then, then w_2 will be -- now I've sort of said what w_1 is, what's the w_2 that goes with it?
Well, it's whatever it takes to cancel this.
I want this to be, in other words I want it to get zero, so this should be the second derivative S. And what am I going to put now?
dydx.
I'm using the same very crucial fact that the cross derivative can be in either order.
So what do I see for w_2?
Do you see what w_2 is?
This is supposed to match that.
There's a minus sign.
w_2 is what?
Minus dS/dx.
Right?
Minus dS/dx.
That's the minus dS/dx.
If I take any function, S, I let w_1 be its y derivative and w_2 be minus its x derivative, then the divergence will be zero.
Because I'll have the cross derivative minus the cross derivative, of course that'll be zero.
So these will be my w_1's, my w's.
w_1, w_2, coming from S. So I have what I expect, with two unknowns, only one equation, I've got lots of solutions.
I create any function, S, and that will be one.
So that's a stream function.
And it has a physical meaning, so we get to see it.
And it has a fantastic connection to the potential.
So up to now -- so this is now the moment pieces come together.
Up to now, we had the divergence and the gradient separately.
So up to now, what we had was, we started with the potential u, and we went to v -- I called it v for this application -- .
OK. And now, over on this side, I had the divergence of w equals zero.
And that lead me to w being, what we just said, dS/dy, and minus dS/dx.
OK, two separate pictures.
Now we're going to connect the framework, going to give the connection between v and w. And the connection will be the easiest possible; they'll be equal.
So the c in our framework is the identity.
I want to say, I want to look at our framework when c is the identity.
So v and w are the same.
In other words, what equation do we get then?
We started with u, we take the gradient of u, that's v. We go over here, and we still have the gradient of u, because this was the v and now it's also the w. And then we take -- what do we take?
Minus the divergence of the gradient of u, equals, let's say, f. Shall we say f, to make something happen?
Or zero -- I'll say zero.
I'll say zero.
Yeah, because zero is the one that give us Laplace's equation.
The official name would be -- that's Laplace's equation.
Do you recognize it?
We'd better write it out properly.
The divergence of the gradient is the Laplacian.
So this is Laplace.
Famous equation.
Steady state type of equation.
It goes with steady state problems.
Let's just figure out what this is.
So w is, since these are the same, w is now .
It's the gradient.
Now what happens, do you see what happens when I take the divergence of the gradient, the divergence of this w. I just want to write that equation out.
It's crucial.
I need brilliant colors and lights shining, now, for what goes in there.
Because I want to take the divergence of the gradient.
So what does that mean?
I take the x derivative of the first component, plus the y derivative of the second component.
The minus sign is not going to matter, with a zero on the right hand side.
I could cancel the minus.
So, but let me say it again.
The divergence is, it applies to a back vector, I've got a vector.
I take the component of the first -- of w_1.
So that'll give me d second u/dx squared.
And I take the y derivative -- I should have said derivative -- y derivative of the second component.
And the y derivative of that is d second u/dy squared.
And I get zero.
So that's Laplace's equation.
You see, the whole idea is Laplace's equation, in working with Laplace's equation, we have three elements, here.
The gradient comes in, the divergence comes in, and equality comes in.
Would you like to see a more general Laplace's equation?
Well, a more general Poisson's equation.
So underneath Laplace, let me write Poisson.
You got the pronunciation, the brilliant French pronunciation there, of Poisson?
That was my best.
I can't improve on that one.
So it comes in when there's a right hand side.
So the normal Poisson equation is second derivative with respect to x, second derivative with respect to y -- well, actually, it should have a minus there.
Really should be a minus there.
You remember why the minus is there.
The minus is there to make the whole thing positive.
Right?
That sounds crazy, that the minus makes it positive.
But these second derivatives are negative definite, as always, and the minus makes them positive definite.
So I don't remember whether -- maybe I often include the minus over here -- equal f(x,y).
So there's a source.
Poisson has a source term.
Laplace doesn't.
And just while I'm talking about this framework, if there was a c(x,y), so this would be or.
I'll put or.
The more general one would be the x derivative, because I'm taking the divergence, of a c(x,y)du/dx, and the y derivative of a c -- so you see the difference.
I'm now allowing some variable conductivity.
Variable whatever, variable material.
du/dy equals f, again.
So that would be the more general one.
I don't think we plan to study that.
Well, that's not the most general, I could have more and more things there.
But that shows you a variable material.
Yeah, that material is variable.
I would use the word -- what word would I use?
It doesn't depend on the direction.
I'm using the same c(x,y) in the x direction and the y direction.
Therefore in all directions.
And I didn't come prepared with that word.
What's the word for when it doesn't depend on the direction?
Isotropic!
Thanks.
Isotropic.
So that's isotropic.
So that's really our framework there.
At least for isotropic materials.
And then we could have more general with an isotropic material.
That would be fun.
All of those things are fun.
But Laplace's equation is the most fun of all.
So let me take f to be zero, c to be one, get back up to Laplace's equation, and begin to make connections.
Begin to make connections.
OK.
The beautiful connection is the one right here.
When v and u are the same, when v and w are the same, then I just read off.
If v is the same as w, then how is the potential function, which is on the left side, connected to the strain function, which is coming from, any time div w is zero, I've got a stream function in 2-D, here.
The connection is just, those two match.
du/dx is the same as dS/dy, and du/dy is the same as minus dS/dx.
You see that the two sides of our world are coming together.
We're dealing with flows that are both gradient flows, they come from a potential, you could often say potential flows, I see the word ideal flows coming in.
These are very special flows.
So aero couldn't work entirely with these flows, of course.
These are such ideal flows.
Proper aerodynamics, you've got shocks, you've got all sorts of stuff going on.
But in a region where everything's beautiful, then you get back to this, total steady state, steady flow, steady potential flow.
We've got those equations that connect u on one side with the divergence business, the stream function on the other side.
So I want to focus on these.
Actually, I mean, so the heart of Laplace's quation is in there.
OK. First, do you know the names of those two guys?
I shouldn't call them guys, they're the greatest mathematicians ever.
Maybe after Gauss.
Do you know whose names are associated with those two equations.
Well, Lagrange was great.
I'm not saying anything about Lagrange.
But he didn't do this.
So two, one French and one German.
So the French guy's name is Cauchy.
And the German is -- so these are Cauchy, and the German is a really fantastic guy.
Anybody know his name?
Cauchy-Riemann.
Yeah.
The other name is Riemann.
Cauchy-Riemann equations, that connect the two pieces, u and S. And they're the subject of an enormous theory that we'll just touch on here.
OK. and we'll see them graphically, and we'll find solutions.
So this is one way to pose our problem.
Notice something here.
I think that if we have these equations, a solution, u, should satisfy Laplace's equation.
Because this equality will take us around the loop.
So do you see that if I could solve these two -- here's the point.
I'm going to get Laplace's equation solved by u. Laplace's equation will also be solved by S. The string function.
So what's going to happen is, I get two solutions.
A pair of solutions.
Laplace's equation, solutions to that come in pairs, u and S. So we get them two at a time.
And they're connected in this remarkable way.
Let's see, can you see that u will satisfy-- this is the key to everything.
Does u satisfy Laplace's equation?
Sure.
I take the x derivative of that, and I add to the y derivative of that.
Do you see, it works.
The x derivative of this, plus the y derivative of this is exactly the cancellation of the cross that I wanted.
So I do these, I combine those into Laplace.
Shall I just go through that verbally again?
I take the x derivative of this, which gives me the cross derivative of S. This asked me to take the y derivative, so I get the cross derivative again.
With the minus sign, they add to zero.
I just want to point out that also, S satisfies Laplace's equation.
Can we do that one?
I claim that also, the stream function solves Laplace's equation.
Because we want the x derivative of dS/dx.
So dS/dx is minus this.
Do you see what's happening?
When I take the x derivative of dS/dx, I get minus the cross derivative of u for this guy.
It's the y derivative of this, which is plus the cross derivative of u.
They cancel.
So u and S are just together.
Oh, let's find some solutions.
They're great to find.
And then draw them.
OK, can I find some solutions to Laplace's equation.
And I'm going to find them in pairs.
So I'm going to have a list of u's and their corresponding S's.
OK.
These are solutions to Laplace.
These are, solve Laplace's equation.
So we've got Laplace's equation in our minds.
Actually, furthermore, they solve Cauchy-Riemann.
Because they're going to be connected by our Cauchy-Riemann equation.
So, solve Laplace and Cauchy-Riemann.
OK.
Suppose I take u(x,y)=x.
I'm going to start with an easy solution.
That certainly solves Laplace's equation.
You've got Laplace's equation in mind?
Let me write it up here again.
u_xx+u_yy=0.
I take the chance to write it again, to do it in this little bit shorter notation, just subscripts instead of partials.
And also, S_xx+S_yy=0.
But most of all, the Cauchy-Riemann that connects the two.
Well, does u=x solve Laplace's equation?
Of course it does.
The second x derivative, if the function is x, is zero.
And the second y derivative is very, very zero.
[LAUGHTER] So what's S?
What's the S that goes with it?
It'll be simple, too.
So the S at u is x, right?
I'm starting with this x.
So du/dx is one.
So what do you figure s is?
If du/dx -- see, u is just x itself, so du/dx is only a one.
That derivative was easy.
Then dS/dy is supposed to be one, and dS/dx is supposed to be zero, I guess.
Do you see what S is?
y.
S is y.
S is y.
Of course, that solves Laplace's equation, too, and it solves Cauchy-Riemann.
The x derivative of this is the y derivative of that.
One equal one.
And the y derivative of that is minus the x derivative of that, zero equal zero.
OK, so that's an easy one.
I'm going to go up a level.
I want to take a second degree.
So my next guy in the list will be -- a pair, it's a list of pairs -- will be x squared minus y squared.
First of all, it better not be in that list unless it solves Laplace's equation.
And then if it is, we'll find an S. So plug it in mentally, can you plug this into Laplace's equation?
What's the second x derivative of this function?
Two.
Right?
xx brings down a two.
What's the second y derivative?
Minus two, from this term.
And then put them into Laplace's equation, two minus two.
Correct.
Zero.
All right.
Now I'm looking for the S that goes with it.
The other one in the pair.
OK, maybe I'd better think through what -- so this is supposed to give me the S, du/dx.
Let me copy Cauchy-Riemann here, so we can just focus entirely on that board.
So du/dx, this is 2x in my example.
du/dy is minus 2y.
So what am I learning?
dS/dy should be 2x.
dS/dx should be 2y.
Because our minus signs both there.
What's S?
Do you see S?
The y derivative is 2x, the x derivative is 2y, and that stream function is 2xy.
Right?
2xy.
Because the x derivative of this is 2y, and the y derivative of a this is 2x.
And we saw 2xy last time also.
And of course, it solves Laplace's equation easily.
Plug that in, the second derivative is zero.
The second x derivative is zero, second y derivative is zero, everything.
So that's a pair.
This is a nice pair.
You want to shoot for third degree?
We could maybe figure out third degree, just by jiggling it.
After that, we're going to need an idea to get up to fourth degree.
Let me try third degree.
Cubics, now.
So I'm looking, first I just want to get somebody here.
So it's some x cubed.
And then I'm going to need some more stuff, because x cubed by itself certainly won't work.
I need something more.
And I want to plug it into Laplace's equation and figure out what should it be?
OK, so when I plug this into Laplace's equation, what do I get?
Let me do Laplace's equation over here, to try to get the u of degree three.
So what's u_xx, so far?
6x, right?
Bring down, we've got 3x squared, then we get 6x, x so I've got a 6x.
And I'm looking for -- so u_yy should be minus 6x to cancel that.
So what do I want there?
What do I need?
I need a minus, I'm sure of that.
So the second y derivative should be this 6x deal.
What do I want?
3xy squared?
That sounds good.
Let me write it down and see if it is good.
OK.
The second y derivative.
Yes.
We'll bring down a two, and then the y's will disappear and I'll have the minus 6x.
Golden.
OK, that's great.
That's great.
Is that correct?
I mean, it's great, but is it right?
Yes.
Yes.
OK. Now, you have faith that there's another one?
Well, yeah, there is another one.
Cauchy-Riemann never let us down.
There will be an S that'll go with that, that'll solve a Cauchy-Riemann equation, and it'll look like it.
And I think, I think -- and I just sort of reverse x and y to get another one, because if I exchange x and y, I'm still OK with Laplace's equation.
I think something like 3yx squared minus y cubed.
If that worked, then this one should work, too.
Because I just switched x and y, and I think it'll work right here.
The second x derivative will be 6y, and the second y derivative will be minus 6y.
I think that's good.
OK. Let's put this list on hold for a moment.
Something's, obviously, there's some pattern here that we've got to locate.
Can I put it on hold for a moment and take, for example, the graph of this one.
And I want to draw the pictures.
Before I get a complete list, I want to draw the pictures of these functions.
And what do I mean by pictures?
I mean draw the -- so now I'm taking the u to be x squared minus y squared, and the S to be 2xy.
And I want to draw those, I want to draw the vector field of the gradients of those guys.
OK, so this is the u -- I want to draw -- this is the potential, x squared minus y squared.
So what are they equipotential curves?
So this is now, I'm drawing the flow.
And to draw the flow, I draw the curves on which the potential is a constant.
x squared minus y squared equal constant.
So what kind of a curve is x squared minus y squared equal constant?
It's a hyperbola.
It's a hyperbola.
So x squared minus y squared equal, let's take one, for example.
As one constant.
So x=1 will be on the curve, when y is zero.
And then what else will be on the curve?
If x is a little bigger, like two -- so here's (1, 0).
That point is certainly, x squared minus y squared is one for that.
Suppose I go out to x=2.
What should y be then, to make this right?
Where's the curve going?
When x is two, what is y?
Square root of three, plus or minus.
So square root of three is something like this, up or down.
It's a curve, like so.
It's a hyperbola.
And now, if I change c to four, let's say, then it'll go through (2, 0), and it'll go up this way.
And if I change x to something very small, it'll still -- oh, there's a Greek word, asymptotes.
Oh, yeah.
What do I get if -- the asymptote is when this is zero.
Yeah.
When that is zero, what's my curve?
What are x and y?
They'll be the same.
x will be y, or minus y. I'll be on that straight line, or on this straight line.
So all these other hyperbolas are kind of asymptotic, whatever the word is.
Right, do you see them?
As a bunch of hyperbolas?
And actually, more hyperbolas -- well, yeah.
Let's see, back when I had a one there, I took x and y to be positive, and I got that hyperbola.
But since I'm squaring them there, also these hyperbolas are here.
So these same guys are on that side of the picture.
Those are the equipotentials.
So these are the equipotentials.
OK. Now, let me draw S. So those will be the equi -- no, I don't want to say equistream functions.
That's awkward.
So now, I want to draw S equal constant, like one or whatever.
So I've drawn this with a whole lot of constants, and now I want to draw the other guys.
What do those curves look like?
And what's their name?
A curve on which the stream function is a constant has a nice name streamline.
So now I'm going to draw the streamlines.
And what are the streamlines?
The streamlines will be the curves that the actual material flows.
If you drop a leaf into this flow, and you watch it, it'll flow along a streamline.
And we can draw those lines.
So what are those?
2xy=1, or the equation y=1/2x.
That's also a hyperbola, right?
That's also a hyperbola.
This is a fantastic picture, in which we have two sets of hyperbolas.
We're second degree, that's why we're getting two hyperbolas.
I'm not going to tackle drawing -- MATLAB could do it -- drawing the equipotentials and the streamlines for this guy.
Oh, but I'm willing to tackle this one.
What are the equipotentials and the streamlines for the easiest one in the list, there?
Can I just draw that one?
Because it makes a point very clear, that we'll see when we draw these.
Okay, so what's the picture, the corresponding picture?
Here is the xy plane again.
What are the equipotentials for this pair?
And the streamlines.
The equipotentials are x equal constant, what are those?
Those are lines, vertical lines.
So the equipotentials are just vertical lines.
Equipotentials, x equal a constant.
And what are the streamlines?
Horizontal lines.
Streamlines go this way.
And what's the great point about these?
These are the streamlines, S equal constant.
And of course, what do you notice here?
They're perpendicular.
The streamlines are perpendicular to the equipotentials.
And why?
It's because -- you remember we talked about, what does the gradient mean?
Which way does the gradient point?
It points perpendicular to these equipotentials.
And in this case, all these equipotentials are parallel, and the perpendicular lines are the streamlines, and they're all parallel.
Now over here, we haven't got straight lines.
but we still have the beautiful figure.
Now I'm ready to tackle it.
I'll draw these curves.
They're hyperbolas, like y=1/2x.
If I just make that y=1/2x.
That's a line that comes down this way.
Let me try to draw it.
i'll use dashed lines, of course.
As x gets bigger, y gets smaller.
But y never makes it to zero, because if y was zero, no x would work.
But if I change that one to to a four, I've got a bigger -- this is coming out here.
If I change that to something very small, I'll get one that's coming -- Do you see how the picture works?
These are all right angles.
That's the great thing.
Right angles.
90 degree angles.
Between the streamlines and the equipotentials.
You may have seen this before, and now I just want to ask you why.
Why are those 90 degrees?
We kind of see it physically, that the gradient, the flow is in the gradient direction.
And we know that gradients are always perpendicular to equipotential lines.
The gradient of any function is perpendicular to the level curves.
That's all we're seeing here.
But we're seeing these two fantastic families of curves.
Equipotentials perpendicular to the streamline.
And the reason they're perpendicular is Cauchy-Riemann.
Cauchy-Riemann is telling us they're perpendicular.
Because the gradient -- yeah, you see that they're perpendicular, and this may be overkill to try to give a proof.
The gradient of u -- what I want to say is, the gradient of u is a 90 degree rotation of the gradient of S. Let me put it that way.
The gradient of S -- the gradient of u, rotate 90 degrees, rotate gradient of u by 90 degrees, pi/2, and you get grad S. That's what these equations say.
That if I take the gradient of u -- yeah, let me try to do that.
So I take the gradient of u.
That's .
So if I have a vector in 2-D, .
And I want to rotate a vector by 90 degrees.
What's the result?
Suppose I have a vector , and I'm looking to rotate it.
What's the vector that goes that way?
If that vector is , that vector should be what?
You may not have done this, but it's worth just noticing, and you won't forget it.
It's got to have a zero dot product, right?
So I'm looking for a vector, here.
This went out a and up b. I'm looking for a vector there that's perpendicular to this vector.
So what should it do?
What am I going to put there?
A b.
And what am I going to put -- oh no.
It went backwards, sorry.
When I put there, I should have put -- minus b.
And what goes there?
a. Yeah.
That's the perpendicular one.
Right?
That's the 90 degree rotation.
And that's what Cauchy-Riemann is doing for us.
I take the gradient, I take this vector.
Now I rotate by 90 degrees, means I reverse these, reverse the sign, and I've got the gradient of S. Or minus the gradient of S. I won't say whether the rotation is plus 90 degrees or minus 90 degrees.
This can remain an exercise to see it slowly and clearly.
I'm happy if you see it in the picture there.
And of course, you saw it in this picture, here.
So is this is a moment, then, to take a little pause.
Because we've got ten minutes for the great event.
We've got the general idea.
The u equal constants and the S equal constant.
These two families of perpendicular curves, one telling us where level sets for the potential, the other telling us the direction of the flow.
The flow goes perpendicular to the level sets.
It's just wonderful.
And now I would like to get the pattern that's going on here and complete that list.
OK, well that pattern, like, comes out of the blue, I have to admit.
You might sort of recognize it, that something -- here is, obviously, stuff to the first power.
Here we've squared something to get here.
Here we've cubed something.
You sort of recognize these numbers, 1 3 3 1, or one, minus three, whatever.
We're taking powers of something.
And that something is what comes out of the blue.
It's this quantity, x+iy.
Complex variables.
Everything here was real until this moment.
And then I'm saying that the complex number, i, the imaginary number, i, the square root of minus one.
Which of course, it's not a real number, but it has the property, whenever we see i squared, we write minus one.
So then, we know how to deal with it.
OK.
Sort of.
Anyway, so that complex number I'll call z.
And here's what I think.
I think that these two pieces are the real and the imaginary parts of z.
Now, these two pieces are the real and the imaginary parts of z squared.
These two pieces will be the real part and the imaginary part of z cubed.
And if we check that, and then we begin to see, why should these satisfy Laplace's equation, we'll have the whole pattern.
It'll just be x+iy, fourth power, fifth power, sixth power.
We can make a complete, infinite list of pairs of solutions to Laplace's equation.
So let me just check what I said about the squares first.
How do you do x+iy squared.
Because that's what I believe we're seeing in the quadratic list.
So I claim that if I take x+iy squared, just do it normally, I get x squared, and I get 2ixy's, and I get i squared, y squared.
Right?
I just squared it, following normal algebra rules.
Now what's the real part?
What's the real part of this x+iy squared?
x squared, this is real.
And this is real, because i squared is minus one, this says minus y squared, that's our guy.
And the imaginary part is 2xy.
The imaginary part is the part that multiplies i.
So by some magic -- next time is the fun of exploring this magic -- we get solutions to a real equation.
We get two solutions to a real equation, by working with this complex thing, x+iy, and taking things like -- now, if I go to x+iy cubed, well, let me just say it'll work.
We'll have -- it's like, if you cube something, you're going to see 1 3 3 1.
You'll have an x cubed, and a 3x squared iy, and a 3xiy twice, and a one of the iy cubed.
And then you look to see what part is real, and you say, x cubed is real.
And i squared is the minus, so I have minus 3xy squared.
Golden.
And you look for what part is imaginary, and you see the 3x squared y, and you see the i cubed -- so what's i cubed?
-- is minus i.
So that's an imaginary term, with the minus we wanted.
So now we've got the whole thing.
We've got solutions to Laplace's equation, coming from all the powers.
This is now the moment to celebrate.
Because we've got a giant family of solutions to Laplace's equation.
We've got the real parts.
So u is the real part of x+iy to any power.
And S will be the imaginary part.
And I claim that, just as we've held for n equal one, two, three, for every n, these will be solutions to Laplace's equation.
Not only that, they'll be connected by Cauchy-Riemann, so they'll be the potential and the stream function for a flow.
And we've got lots of them.
And, why don't we get even more, because we have a linear equation here.
So what are we allowed to do, if we have solutions to a linear equation, what are we allowed to do with those solutions to get more solutions?
Combine them.
I can take any combination of these guys.
I can take "or."
"And," "or," I don't know which, should I put "or," "and?"
The real part of any combination of these.
So I could take a combination of, I can take coefficients c_k, or c_n, maybe I should say c_n(x+iy) to the nth.
So if I take a combination of solutions, with coefficient c_n, I still have solutions.
And what will be the twin solution, or the stream function that goes with this u?
So this is another u, this is virtually a complete family of u.
Because we have all these coefficients to choose.
And what will be the S?
So what's the corresponding S, then?
It's the imaginary part of -- what?
-- the same thing.
We're just taking that same combination of the special ones.
So the special ones were (x+iy)^n.
And the combinations are those.
Yeah.
These are fantastic solutions.
And now, since I'm blessed with two more minutes, I get to make them much easier.
Because already when I got up to x cubed and y cubed, they're looking messy.
You say, okay, great to have all these solutions, but how am I going to use them?
They're getting more and more complicated as n increases.
But switch to polar coordinates.
Make the same list in polar coordinates.
So again, I'm just going to list the same guys in polar coordinates r, theta.
Then you'll see the pattern, then the pattern really jumps out.
So what is x in polar coordinates?
If I switch from xy rectangular coordinates, to r, theta, polar coordinates, x is r*cos(theta).
And y is r*sin(theta).
Well, so far it doesn't look any easier.
x and y look fine.
But let me go to the second one.
So this is now r squared cos squared theta, right?
Minus, this is r squared sine squared theta.
Trigonometry comes in.
I have r squared cos squared theta, minus r squared sine squared theta, I want to simplify that.
So it's got an r squared, and it's got a cos squared theta minus sine squared theta.
Anybody remember that?
cos(2theta)!
And this one.
Well, what do you think is coming?
What do you think is coming here?
I have 2r*cos(theta) times r*sin(theta).
I have two, I've got an r squared, again, times 2cos(theta)*sin(theta).
So what's 2cos(theta)*sin(theta)?
sin(2theta).
You get it.
sin(2theta).
And you know what's coming next, right?
You know that now, the whole family in polar coordinates -- what's the nth power?
We can now write down the nth one in this list.
It is r^n -- for the nth pair, r^n times cos(n*theta), and r^n sin(n*theta).
So those are the twins, the u and the S that we get if we use polar coordinates.
So that's just terrific.
Now we have a whole lot of solutions to Laplace's equation.
And we have, don't forget, all combinations of them.
And we're really ready to go.
So Friday we'll be solving Laplace's equation in the cases that we can do it, by pencil and paper, by chalk.
And then, after that, comes solving Laplace's equation by finite differences and finite elements.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseware continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, it's Laplace again today.
Laplace's equation.
And trying to describe, that's a big area that's a lot of people have worked on for centuries.
And for the early centuries, there were always analysis methods.
And what we you got started on last time.
And we'll do a bit more.
There's no way we could do everything that people have worked on.
Year and years, trying to find ideas about solving.
But we can get the idea.
And this part, then is in the section called Laplace's equation.
And the exam Wednesday would include some of these constructions.
So this is what we did last time, we identified a whole family of solutions to Laplace's equation as polynomials in x and y.
Of increasing degree n, and then when we wrote them in polar form they were fantastic.
r^n*cos(n*theta) and r^n*sin(n*theta).
So my idea is just, we've got them, now let's use them.
So how to use these solutions.
So because we can take combinations of them, we can create series of sines and cosines.
So I'll do that first.
Series.
And then Green's function, that's the name you remember we've seen it before.
That's the solution when the right side is a delta function.
When we have Poisson's equation with a delta there.
So that an important one.
And then well, a big part of two dimensional and three dimensional problems is that the region itself, not just the equation but the region itself, can be all over the place.
We'll solve when the region is nice, like a circle or a square, and then there is a way to, in principle, to get other regions.
To change from a crazy region to a circle or a square, and then solve it there.
So that's called conformal mapping.
And I can't let the whole course go without saying a word or two about that.
But somehow among numerical methods, it's conformal mapping, there are packages that do conformal mapping.
But they're not the central way to solve these equations numerically.
Finite differences, finite elements are.
And that's what's coming next week.
So this is the future, this is the present, right there.
So, can I just start with an example or two?
Like, how would you solve Laplace's equation in a circle?
So, in a circle.
This is the idea here.
I have Laplace's equation.
OK, so I've got a whole lot of solutions.
And I've even got chalk to write them down.
OK, so here's my circle, might as well make it the unit circle.
Radius one.
And inside here is Laplace.
u_xx+u_yy=0.
No sources inside.
So we have to have sources from somewhere, and they will come from the boundary.
So on the boundary, we keep, let me think of u as temperature.
So I set the temperature on the boundary.
u equals some u_0.
Some known function.
This is given.
This is the boundary condition.
This is the given boundary condition.
And it's a function of, I'm going to use polar coordinates.
Polar coordinates are natural for a circle.
So this is at r=1, so maybe I should say on the boundary, which is r=1, and going around the angle, theta is given.
This is u_0(theta).
That's my given boundary conditions.
This problem is named after Dirichlet, because it's like giving fixed conditions and now Neumann conditions.
OK, so I'm just looking for a combination.
For a function that solves Laplace's equation inside the circle, and takes on some values around the boundary.
And of course the boundary values might be plus one on the top and minus one on the bottom.
Or the boundary condition might vary around, it might, in variable then come around.
But notice that this is a periodic function.
This is 2pi periodic, because the problem's the same.
If I increase theta by 2pi I've come back to the same point.
So it's got to have that value.
OK, so well let me give you a couple of examples first.
Suppose u_0, suppose this function - Example 1, easy.
Suppose u_0 is sin(3theta).
So that means I've got a region here, I'm prescribing its temperature on the boundary.
And I want to say what does it look like inside?
And I'm prescribing right now the sin(3theta), so there theta is zero, so it's zero, the boundary condition's zero, there climbs to one, back to zero, down to minus 1, back to 0.
3 times, and again comes back to 0 again there.
So, I'm looking for a solution to Laplace's equation and I've got a pretty good list.
That will match u_0 when r is one.
So that's the boundary, r is one.
So you can tell me what it is.
So you can solve this problem, right away.
The answer is u(r,theta) is, what function will?
Remember I've got my eye on that list.
You too, right?
I'm just trying to get one that when r is one it will match sin(3theta).
What's the good guy?
It'll be on that list.
Which of those, by itself, here I don't need a series because I've got such a neat u_0 function.
I'll get it right with one answer, and what is that answer?
I look there.
I say what do I do, so that at r=1, I'll match sin(3theta).
I'll use r cubed sin(3theta).
So the good winner will be r cubed sin(3theta).
That solves Laplace's equation.
We checked it out, it's the imaginary part of x+iy cubed.
We could write it in x and y coordinates if we wanted but we don't want to.
And it matches when r is one, it gives us sin(3theta).
That it.
And of course I could take any one.
Now suppose I'm trying to match something that's not as simple as sin(3theta).
In that case, I may have to use all of them.
I mean, it's very, very, fluky that one term is going to do it.
Usually, so my main examples would be I'll have to match all of them.
So what do I do?
At r=1, so my general solution is a combination of these guys I worked so hard to get.
The solution is of this form.
It's some a_0, the constant.
And then a_1*rcos(theta), and b_1*rsin(theta).
And a_2*r squared cos(2theta).
And so on.
I'm just taking any combination of, I'm using the a's as the coefficients for the cosine guys.
And the b's, b_1, b_2, b_3 would be the coefficients for the sine one.
OK, that's my general solution.
That solves Laplace's equation.
Every term did, so every combination will.
Now, set r=1.
To match that r=1 and match the boundary.
And match u_0(theta), the required temperature around the on the boundary.
The boundary being where r is one.
So this is set r=1.
So then u_0(theta), this given thing, has to match this, when r is one.
So it's a_0 plus a_1, now what do I write here?
r is one, so it's just cos(theta).
Now, b_1, r is one, so I just have sin(theta).
And I have an a_2*cos(2theta), and a b_2*sin(2theta), and so on.
Here, just let me put it together now.
I'm given any temperature distribution around the boundary.
It's in equilibrium, the temperature, where if the temperature's high near that point and low over here the temperature inside will gradually go from that high point, dot dot dot dot, to the lower one.
By matching on the boundary.
And this is the match on the boundary.
Now, this is really a lead in to the last part of this course.
So whose name is associated with a series like that?
Fourier.
You recognize that as what's called a Fourier series.
So the idea is, I'm given these boundary values.
I find their expansion in sines and cosines, and that's what we'll do in November.
And then I've got it.
Then I know the a's and the b's.
And then basically I just put in the r's.
r and r squareds and r cubeds and so on.
So then I've got the answer inside.
In principle it's so easy.
So, why is it easy, though?
First, it's easy because it's a circle we're working in.
If I was in an ellipse or a strange shape, forget it.
I mean, so this is quite special.
And secondly, it's easy because these functions are so nice.
Fourier works with the best functions ever.
These sines and cosines.
So I'll find a way to find those coefficients, the a's and the b's.
Even though there are lots of them, I'll be able to pick them off one at the time, the a's and b's.
Once I know the a's and b's, I know the answer.
So do you see this is in principle a great way to solve it?
In fact, it's the way we used over here, when my u_0 was sin(3theta) then the only term in its Fourier series was one sin(3theta).
And then the solution was one r cubed sin(3theta).
So you can learn things from this.
For example, oh, what can you learn?
One thing I noticed, an important feature of Laplace's equation is that this solution inside the circle gets very smooth.
The boundary conditions could be like a delta function.
I could say that on the boundary, the temperature is zero everywhere except at that point it spikes.
So I could take u_0.
So example 2, and I won't do it in full, would be u_0 on the boundary equal a delta function.
A spike at that one point.
So all the heat is coming from the source at that one point.
Like I've got a fire going there.
Keeping the rest of the boundary frozen, the heat's kind of going to come inside.
So then how would I proceed?
Well, if I have this boundary value as a delta function, I look for its Fourier series, and it's a very important, beautiful, Fourier series for a delta function.
Would you want to know it?
I mean, we'll know it well in November.
Would you want to know it in October?
This is Halloween, I guess, so delta.
I'll tell you what it is.
Since you insist.
Delta theta, I think, will, I think there's a 1/2pi or something.
Ah, shoot.
We'll get it exactly right.
It's something like 1 and 2 cos(theta) and 2cos(2theta), I'm not sure about the 2pi.
2cos(2theta), and 2cos(3theta), and so on.
We'll know it well when we get there.
What I notice about this delta function, of course you're going to expect the delta function being somehow a little bit strange.
At theta=0, what does that series add up to?
Just so you begin to get a hang of Fourier series.
At theta=0, what does that series look like?
Well, all these cosine thetas are?
One.
So this series at theta=0 is 1+2+2+2+2.
It's infinite.
And that's what we want.
The delta function is infinite at theta=0.
And it's periodic, of course, so that if I go around to theta=2pi I'll come back to zero again.
At theta=pi, you could sort of see, well, yeah, theta=pi is a sort of interesting point.
At theta is pi, what's the cosine?
Is negative one, right?
But then the cos(2pi) will be plus one.
So at theta=pi, I think I'm getting a one minus a two plus a two, minus a two, plus a two.
You see, it's doing its best to cancel itself out and give me the zero that I want, the theta=pi over on the left side of the circle.
Anyway, so that's an extreme example.
But now, what's the temperature inside?
Can you just follow the same rule?
What will be the temperature inside?
If that's the delta function, if that's the right series, whatever, it may be a 4pi, I'm not sure, for that.
Now, you can tell me what's the solution, what's the temperature distribution inside a circle when one point on the boundary has a heat source, a delta function.
What do I do?
How do I match this with this guy?
I just put in the r's, right?
If this is what it's supposed to match when r is one, then when r is, so maybe I'll put it under here.
So the u(r,theta), from the delta guy, is just put in the r's.
1+2rcos(theta), and 2r squared cos(2theta), and so on.
OK, and eventually 2r to cos(100theta), and more.
OK, I write this out, you could say why did he write this down?
I wanted to make this point that the important feature of the solution to Laplace's equation is how smooth it gets when you go inside the region.
And why is that?
Because at r=1/2, this term is practically gone, right?
If I go halfway into the circle, this term is practically gone.
1/2 to the hundredth power.
And if I go to the center of the circle, it's completely gone.
In fact, what's the value at the center of the circle?
What's the temperature at the center?
1/2pi.
This is the only term that's remaining.
And it's the average, around the circle.
That makes physical sense, I guess.
Since the whole thing's completely isotropic, we've got a perfect circle.
The value at the center of the circle is always the average going around.
The constant term in the Fourier series, this guy.
We'll get to know that one very well.
That's the average.
You're just seeing a little bit of Fourier series early, here.
But my point is that you could have high oscillation around the boundary, that damps out because of these powers of r. And inside the circle it's only the low order terms that begin to take over.
This is the kind of trick you have, or not trick but the kind of method that you can use for solving Laplace's equation by an infinite series.
Of course, a person who wants a number can complain that, wait a minute, how do I use that infinite series?
Well, of course, if you wanted to know the temperature at a particular point you'd have to plug in that value of r, that value of theta, add up the terms until you hope that they become so small that you can ignore them.
So infinite series is one form of a solution.
And somehow these are examples, I should use the words separation of variables.
Separation of variables is the golden idea in this analysis stuff.
Separation of variables means I got the r part separated from the theta part.
And that worked great, worked well for a circle.
Let's see, maybe for a square I could try to separate x from y.
Maybe there's a homework problem, a solution that separates x from y, I think is something like.
So this would be another family.
Good for squares, something like sin(kx) cinch times, so this separation is something in x times something in y.
Again I'm just mentioning things.
I think that that solves Laplace's equation because if I take two x derivatives, that'll bring down k squared, but it'll flip the sine, right?
These two derivatives of the sine will be a minus.
And if I take I need a ky there.
And if I took two derivatives of this hyperbolic sine, you remember that's the e^(ky) and e^(-ky).
The two derivatives of that will bring out a k squared with a plus sign.
So two x derivatives bring out the minus k squared, two y derivatives bring out a plus k squared and together that solves Laplace's equation.
We'll check that in our homework problem.
So there would be an example, good for a square.
So, there's hope to do an exact solution in a special region.
Now, what's this Green's function idea?
OK, that's now this is another thing.
So last time we appreciated that this combination x+iy was magic.
The idea was that we could take any function of x+iy, and it solves Laplace's equation.
Can we just see, sort of very crudely why that is?
We saw the pattern, we saw x+iy to the nth.
Sort of, we went as far as n=3, checked it all out.
But now, really if I want to be able to, why does that solve Laplace's equation for any n?
Should I just plug that into Laplace's equation?
What happens if I take the two x derivatives of this thing?
So this going to be a typical function of x+iy, typically nice one.
If I take two x derivatives, I want to plug it in and see that it really does solve Laplace's equation.
So two x derivatives of that will give me what?
The first x derivative will bring down an n times this thing to the n-1.
And then the next x derivative will bring down an n-1 times this thing to the n-2.
So that'll be the u_xx.
And what about u_yy?
This is my u. I'm sort of just checking that yes, this scene again, see if it still works Friday what worked Wednesday.
That this x+iy is magic and functions of it like powers, exponentials, logarithims, whatever, all solve Laplace's equation.
OK, so we did u_xx, and we got easy.
Now, what happens with u_yy?
Do you see the point?
[INAUDIBLE]
Sorry opposite sign.
And why does the sign come out opposite?
Because of that guy.
Yeah, it's the chain rule, right the derivative of this with respect to y will give me an n times this thing to one lower power.
Times the derivative of what's inside.
And the derivative of what's inside is an i.
And then the second derivative will bring down an n-1, this guy will be down to n-2, another i will come out and just what you want, right?
Because the i squared is minus one, those cancel.
When those are equal opposite signs.
And we get u_xx+u_yy equaling 0.
So that works.
And, actually, the same idea would work for any function of x+iy.
The two x derivatives just give f''.
Two y derivatives will give f'' but the chain rule will bring out i both times and we've got it.
OK, I think we just need another couple of examples.
And this of course could be in polar coordinates, f(r*e^(i*theta)).
That's just, everybody recognizes re^(i*theta) is the same as x+iy?
Better just be sure we've got that.
x is some point here in the complex plane.
iy takes us up to here.
So there's x+iy.
That's x+iy there, but it's also, so let me put those in better.
So there's x and there's y.
Everybody knows this picture, right?
This x and this y, now if I want to go to polar coordinates, that angle is theta, this x is r*cos(theta), this y is r*sin(theta), and this guy is re^(i*theta).
cos(theta)+i*sin(theta) is the same as re^i*theta).
That's utterly fundamental.
Everybody's responsible for that picture of putting the complex numbers into their beautiful polar form.
That's what made our r to the nth cos(n*theta) all so simple.
Now, what was I aiming to do?
Give a particular f. Now I want to give a particular function f, or maybe a couple of choices.
A couple of functions f, and see that they're real parts and their imaginary parts solve Laplace's equation.
Let me take first a one that works completely.
Take the real part and the imaginary part.
Let me take e^(x+iy).
It's a function of x+iy, extremely nice function of x+iy, and we can figure out its real and imaginary parts, and we get two solutions to Laplace's equation.
The good way is to write this thing as e^x times e^(iy).
And again we'll write it as e^x times cos(y)+i*sin(y).
So now I can see that the real part, I can see what the real part is.
And I can see what the imaginary part is.
The real part will be, that's real.
And that's real.
so this will so give me e^x*cos(y).
And the imaginary part will be e^x*sin(y).
You see it.
And those will solve Laplace's equation.
Can I give a name to this whole field of analysis?
This e^z is an analytic, I should just use that word, an analytic function.
And these guys, the real and imaginary parts, are two harmonic functions.
Maybe it's not so important to know the word harmonic function.
But analytic function, yeah I would say that's an important word.
Actually, what does it mean?
It's a function of z.
So we're in the complex plane here now.
It's a function of z, e^z, and it can be written as a power series, of course, one plus z plus 1/2 factorial z squared and all those guys.
So it has a power series.
That makes it a combination of our special one.
The great thing about that series is it converges.
So an analytic function, an analytic function is the sum of a power series that converges.
And this one does.
So there's an example.
Yeah, so the whole theory of analytic functions is actually, that's Chapter 5 of the textbook.
And we won't get beyond this point, I think, in one semester with analytic functions.
So what am I saying, though?
I'm saying that the theory of analytic functions is closely tied to Laplace's equation.
Because the real and the imaginary parts give me this pair u and S that satisfy, they each satisfy Laplace's equation.
And they're connected by the Cauchy-Riemann equations.
Boy, it's a lot of mathematics coming real fast here.
Now I'd like to take one more example.
Instead of the exponential, can we take the logarithm.
I want to take the log of x+iy, and I want you to split it into its real and imaginary parts, and get the u and the S that go with that.
So this was like the nicest possible.
We got a series of, e^z is good for every z, the series converges, fantastic.
It's an analytic function everywhere.
Best possible.
Now we go to one that's not best possible but nevertheless highly valuable.
OK, so e^z, I've done.
Let me erase e^z, take log z. OK, so now I'm not doing e^z any more.
And I want to find the logarithm, OK.
So, what's the deal with the logarithm?
Real and imaginary parts.
Now I'm going to take the log of x+iy.
That is a function of x+iy, except at one point it has a problem, right?
There's a point where this is not going to be analytic, and there's going to be a special point in the flow which is singular somehow.
But away from that point, we have a nice-looking function, the logarithm of x+iy, and now I'd like to get its real and imaginary parts.
I'd like to know the u and the S. But nobody in their right mind wants to take the logarithm of a sum, right?
That's a very foolish thing to try to do, the log of a sum.
What's the good way to get somewhere with this?
Real and imaginary part.
I can take the log of a product.
So the polar is way better again.
I want to write this as a log of re^i, I want to write it that way.
And now what's the log of a product?
The sum of the two pieces.
So I have log r, and the log of e^(i*theta), which is?
Which is i*theta.
Boy, look, this is fantastic.
Fantastic except it's zero.
I mean, it's fantastic but it's got a big problem at zero.
But it's an extremely important example.
So what's the real part?
It's sitting there.
This is my u.
This is my u(r,theta), my u(x,y), whatever you want is the log of r. The log of the square root of x squared plus y squared.
I claim that again by this magic combination, this log, this r is the square root of x squared plus y squared.
I claim if you substitute that into Laplace's equation you get zero.
It works.
And what's the imaginary part, the S?
The twin?
Is the imaginary part, which is theta.
Oh, what is theta in x, if I wanted it in x and y?
What would theta be?
It's the arctan, it's the angle whose tangent is something.
y/x, so if I really want it in rectangular xy stuff, it's the angle whose tangent is y/x.
And again, if you remember in calculus how to take derivatives of this thing and you plug it into Laplace's equation you get zero.
It works.
So that's a great solution except where?
At zero.
Except at zero.
And this doesn't tell us what's happening at zero.
It's an excellent solution.
What's the picture?
So by Wednesday's exam I'm not expecting you to be an expert on the theory of analytic functions.
I don't expect you to know any conformal mappings.
By Wednesday, God, that's.
But, I do expect you to have these pictures in mind.
So when I draw those axes, what picture is it that I'm planning on?
I'm planning on the equipotentials u equal constant, and the, who are the other guys?
The stream lines.
The places where the stream functions.
So here is the potential function.
So what are the equipotential curves?
For that guy.
Circles.
This is a constant when r is a constant, so the equipotential functions would be circles.
I don't want to draw that circle with radius zero, though.
I'm nervous about that one.
But all the others are great.
And what are the stream lines, now?
The stream lines are, well, what will the stream lines be?
If I've drawn one family, you can tell me the other family.
The stream lines will be?
Radial lines.
Because they're going to be perpendicular to this.
And so what do I get, this is the stream function, theta.
So what's a stream line?
The stream function should be a constant.
Theta's a constant.
That means I'm going out on rays.
Those are all streamlined.
Again, everything fantastic.
If you look in a little region here you see just a beautiful picture of of equipotentials and stream lines crossing them at right angles.
Everything great.
Just that point is obviously a problem.
Now, and I'm suspecting that there's a source here.
I think this flow, which is given by these guys, comes from some kind of a delta function right there.
And the flow goes outwards.
So I know u, I know v is the gradient of u, right?
I could take the x and y derivatives, I'd know the velocity.
I know the stream function, the divergence would be zero.
Everything great, except at the origin.
I think we've got some action at the origin.
Because, here's the way to test it.
I want to see what's happening at the origin.
And I'm going to use the divergence theorem.
Yeah.
Yeah.
I'm going to use the divergence theorem.
So the divergence theorem says, what is the divergence theorem?
So this is the key thing that connects double integrals.
Let me take a circle of radius r. So that's the circle of radius r. r could be big, or little.
So I integrate over the circle of radius r. So what's the deal?
v is the same as w. What does the divergence theorem tell me?
It tells me that if I integrate, what do I integrate, the divergence of w?
dx/dy, or r*dr*d theta.
Then I get the flux.
So this is a key identity.
Fundamentally, more than just the key identity, it's central here.
The total flow out of the region must make it through the boundary.
So I integrate this boundary and this boundary is a circle of radius r, and what do I integrate along that circle?
What's the other side of the divergence theorem?
w dot n. w dot n, around the boundary.
And remember, I have this nice, my curve here is this nice circle.
So I'm going to integrate around that circle.
First, of all what is n?
By definition, n is the norm that points outward, straight out.
So it's actually going out that way.
At every point it's pointing straight out.
And dS, yeah, I think we can figure out exactly what that right-hand side is.
How do I get that right-hand side?
I'm looking for w, and then I have to integrate.
OK, here is my u.
My u is log r. So what's the gradient of log r?
It points outwards.
And how large is the derivative?
So the derivative of this log r is 1/r.
I think that this comes down to, this is the integral.
Around the circle.
I think that this thing is 1/r.
I went pretty quickly there, so I'll ask you to look in the book because this is such an important example it's done there in more detail.
So I'm claiming that the derivative is 1/r, and that it points directly out.
So the gradient points out.
The normal points out, so that I just get exactly 1/r.
Now, what is dS?
For integrating around the circle what's a little tiny piece of r on a circle?
Of radius r?
r d theta.
Good man.
r d theta.
Now that's an integral I can do, right?
And what do I get?
2 pi.
r cancels r, I'm integrating d theta around from zero to 2pi.
The answer is 2pi.
So what do I learn from that?
I learn that somehow this source in the inside has strength 2pi.
What's sitting in there is 2pi times a delta function.
This is the solution to Laplace's equation except at that source term, so I really should say Poisson's equation.
This has turned out to be the solution to Poisson with a delta, or with 2pi times a delta.
We have just solved this important equation.
Poisson's equation with a point source.
And, of course, that's important because when you can solve with a point source, you can put together all sorts of sources.
And this is called the Green's function.
The Green's function is the solution when the source is a delta.
So if I divide by 2pi, now I've got it.
I divide this by 2pi and there is the Green's function.
I have to put that in bold letters.
Green's function.
It's the solution to the equation when the source is a delta and the answer is u is the log of r over 2pi.
So that's the Green's function in 2-D.
Physicists, you know, they live and die with these Green's function.
Live, let's say, with Green's function.
And they would want to know the Green's function in 3-D.
So the Green's function in three dimensions also turns out beautifully.
This is in, they would say, in free space.
This is the Green's function when there's no other charges.
Nothing is happening, except for the charge right at the center.
And if I'm in two dimensions the Green's function is this log r. So it grows more slowly.
It behaves like log r. And in 3-D I think the answer is 1/4pi*r. It's just amazing that those Green's functions, when the right side is a delta, have such nice formulas.
OK, let me take one moment here.
I'll tell you what conformal mapping is about.
But what's your take-home from this lecture?
Your take-home is two methods that we can really use to get a formula for the answer.
One method was for Laplace's equation in a circle.
Get the boundary conditions in a series of sines and cosines, and then just put in the r's that we need.
That's a simple, simple method.
Provided we can get started with the Fourier series.
The second method is, look at functions of x+iy, and try to pick one that matches your problem.
And if your problem has a point source, at the origin we found that one.
So the literature for hundreds of years is aimed at solving other problems.
If the point source is somewhere else, what happens?
That's not hard.
If it's not a point source but some other kind of source, or if the region is not a circle.
Can I say in one final sentence just what to do, this conformal mapping idea, when the region is not a circle.
Well, I can say it in one word, make it a circle.
I mean, that's what Riemann said you could do it.
You could think of a function, so Riemann said that there's always some function of x+iy, let me call this Riemann's function capital F(x,y).
So this is now the idea of conformal mapping.
Change variables.
Conformal mapping is a change of variables.
He picked some function and let its real part be and let its imaginary part be Y.
Capital Y. OK, this is totally ridiculous to put conformal mapping in 30 seconds.
But, never mind, let's just do it.
The book describes conformal mappings and classical applied math courses do much more with conformal mapping.
But the truth is, computationally they're not anything like as much used as these.
So what's the idea?
The idea is to find a neat function of x+iy, so that your crazy boundary becomes a circle.
In the capital X, capital Y variable.
So you're mapping the region, ellipse, whatever it looks like, by changing from little x, little y, where it was an ellipse, to capital X, capital Y, where it's a circle.
And the point is Laplace's equation stays Laplace's equation.
That change of variables does not mess up Laplace's equation.
So that then you've got it in a circle.
You solve it in a circle, for these guys.
And then you go back.
In a word, you're able to solve Laplace's equation in this crazy region because you never leave the magic x+iy.
You find a combination with that magic x+iy that makes your region into a circle.
In the circle we now know how to use capital X+iY.
You're staying with that magic combination and getting the region to be what you like.
So people know a lot of these conformal mappings.
A famous one is the Joukowski one, that takes something that looks very like an airfoil, and you can get a circle out of it.
So I'll put down Joukowski's name.
So that's one that I trust Course 16 still finds valuable.
It's a transformation that takes certain shapes and they include shapes that look like airfoils, and produce circles.
OK, so sorry about such a quick presentation of such a basic subject.
Conformal mapping, not on any exam, that'd be impossible.
It's really this stuff that you're number one responsible for.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, OK.
So I thought I would, as last time, before Quiz 1 and now before Quiz 2, I thought I would just tell you what the topics of the four problems are.
Remember that trusses are still to be reviewed, and of course the fun was to find mechanisms, to find solutions, to Au=0 in the case when the truss was unstable.
So you can guess that I'll probably pick an unstable truss.
And ask for mechanisms.
Then maybe the key problem for this third of the course is finite elements.
That idea, and finding elements we've really only done in one dimension.
So and particularly linear hat function elements.
So you see what's not included there is like cubics.
I mean, those are really great finite elements, but maybe not so great for an exam.
So 1-D is enough; linear elements are enough to do on an exam.
And then questions three and four, so this one will be weighted more heavily.
Then questions three and four are the topics that we've been doing the last week, two weeks.
And I realize we haven't had full time to digest them, to work a whole lot of exercises.
So those will be, right now they're too simple, my questions on those topics.
Three and four, I'll try to make them a little harder for you.
But I might not succeed.
So, anyway.
There we go.
So I just felt we can't talk about a course in applied math without dealing with these topics in vector calculus that lead to Laplace's equation.
But my heart is really in solving Laplace's equation or Poisson's equation by finite differences.
That'll be today, or by finite elements, that'll be Wednesday and probably Friday.
So this is like the core of the computational side but we had to do this preparation.
And then you know that Peter kindly changed his weekly review session to Tuesday, I think it's Tuesday at noon; you could look on the website.
So Peter will be a Tuesday review and then regular, that's me, on Wednesday.
And then the exam, I think, is Wednesday evening.
I think, yeah.
Let me just do a review.
Oh, Peter sent me a note about a question on an old exam.
I think it was 2005, question three.
I think it was, 2005 question three.
Peter said he didn't understand, at all, the solution to 3c.
Nor do I. I think somehow, I think there was a little confusion in that, and the answer in there is the answer to a different question.
Forget it.
Having looked at this one, a and b are worth looking at in this div grad potential stream function world that we're in.
Those are typical exercises in that area.
Yes, question
[INAUDIBLE]
Oh, good question.
Yes.
Thanks for reminding me.
So, solutions to the homework that you're practicing with.
If anybody has typed up solutions or could, we'll post them right away.
So I'll just hope for an email from somebody who solved some or all of those problems.
And of course you'll see me again Wednesday.
But that's an invitation for anybody who worked through those homeworks that I'll never collect.
The current homework.
OK, thanks for that.
Other questions about the exam, and then you'll have another chance Wednesday to ask about that, so those are the topics.
OK. And before I get to the fast Poisson solvers there's so much about Laplace's equation to say and I think it still remains to write Green's formula on the board.
And that's part of the big picture of Laplace's equation is to see this.
So we have double integrals now, that's the difference.
But we're still integrating Au against w. So Au is the gradient end of u, w is w_1, w_2, and this is now for any u and any w, so I want the inner product.
Of the gradient view with any w, so this is like the Auw part, so the gradient view, that would be a du/dx times the w_1.
And a du/dy times w_2.
dx/dy, that would be.
So inner products, if I'm in two dimensions, inner products are integrals, of course, if I'm dealing with function.
So this is like the Au against w. Inner product.
Au, is those two pieces, w is those, there's the dot product.
And now the key idea behind what Green's formula is about, what we use it for in 1-D is integration by parts.
So an integration by parts is going to produce a doubled integral in which, so I'm just sort of thinking through Green's formula.
Of course, we could just say OK, remember that formula.
But you want to just see it as an integration by parts.
See how it evolved.
And then we can think.
So I plan to take two integration by parts, that x derivative will get taken off of u and put onto w_1.
So I'll have u*dw_1.
Oh, there'll be a minus right, from the integration by part.
u will multiply minus dw_1/dx, so that was this guy integrated by parts.
Now this one integrated by parts, a y derivative is coming off of u and onto w_2.
So that'll leave me with u times dw_2/dy with the minus sign.
That comes also from that integration by parts.
And then there's the boundary term.
So there's the term which, now the boundary of the region is the curve around it instead of just the two points.
And what do we see in the integration by parts?
Well, from the boundary term we expect a u and then the key point is it's (w dot n)ds.
It's the normal component of of this vector w. Because we're looking, well, that's just what it is.
So that's Green's formula written out in detail.
And this is the formula that tells us that here, if this was a, a being gradient, on the left side I'm seeing grad u in a product with w, and over here I'm seeing the inner product of u with, what's that?
If I look at minus, if I have this quantity there that's in parentheses, what's its name?
It's the divergence.
It's minus the divergence of w. Plus the boundary term.
So this is why I allowed myself this gradient transpose equal minus divergence to see that if A is the gradient then A transpose will be minus the divergence.
And this shows how that, just sort of shows you the integration by parts that makes that true.
So, you know there's so many integral formulas in this world of vector calculus.
But this is like, one to know.
And actually, if you want to know, OK what about the details, where did that come from?
This actually turns out to be, and the text will make it clear, is the divergence theorem.
The divergence theorem gives us this, this, exactly this, so the divergence theorem applied to the vector field u times w. It turns out if I apply the divergence theorem to that, then I get a whole lot of terms from the divergence of that, and they are these.
And then I get this for the boundary.
To flex out.
OK, so.
I just didn't want to leave the subject without writing down the central thing.
What does this tell us, then?
I have a region.
In that region I have Laplace's equation or Poisson's equation.
Say Poisson.
On the boundary.
Now, this is the final thing to ask you about.
What are suitable boundary conditions for this problem in 2-D?
Right?
In 1-D, by now we know boundary conditions.
That we know the main ones.
There's fixed, where u or given.
Or there's free where the slope is given.
Now, what do we do here?
So this is where we used to be in 1-D. That's a straight line.
Not perfectly straight but anyway, straight line.
So 1-D there are only two boundary points.
The normal vector goes that way.
And that way, this is the end in this formula.
I think if I just wrote ordinary integration by parts it would look just the same and my boundary reduces to two points.
Here my boundary's the whole curve around.
So what corresponds to fixed boundary conditions, and what corresponds to free boundary conditions?
So I want the one corresponds to fixed, those are the ones that named after Dirichlet.
Other names?
Or the rest will be free, because the ones named after Neumann, and those are what we call natural boundary conditions.
So I haven't, just ready to finish this story here.
What are the possibilities?
Well, in each we could have part of the boundary could be fixed.
Part could be free.
So fixed would mean say some other boundary, let me call this part of the boundary fixed.
So the temperature, for example is fixed on this part of the boundary.
So this would be like, and this might be the whole boundary.
It might be fixed on the whole one, but I'm now allowing the possibility that we have here a fixed part and here we have a free part.
In fluids, here we have fixed corresponds to, like, there's some obstacle there.
Some wall.
Free is where the fluid is coming in or flowing out.
So there it's a velocity condition.
OK, so fixed conditions, these Dirichlet ones will be like, tell me what u is, u is given.
On this part.
So on part of the boundary, or possibly all, I could be given u=u_0.
It could be zero, as it often was in 1-D. Often just took u to be zero.
Now, what I'm asking maybe for the first time, is what's the free boundary conditions?
What do we prescribe w=0 on the free part of the boundary?
Is w=0 the correct free condition?
Or w=w_0?
If we wanted to allow it to be non-zero.
That would sound right, right?
u on one side, w on the other.
But now we're in 2-D. u is still a scalar.
That's fine.
But w is now a vector.
So if I was to prescribe w on this part of the boundary, I'm giving you two conditions, not one And that's not good.
I can't give you more than one boundary condition.
And the clue is there.
It's w dot n, it's the outward flow.
That you could prescribe.
You can't say in advance what the tangential flow should be.
So the free conditions would be like w dot n, which is of course, since w is v, we have Laplace's equation here with c=1 in between w and v, so this is the gradient of u dot n, grad u dot n, and it's sometimes written, the derivative of u in the normal direction.
And that we can prescribe as some w_0.
Oh, I don't to say w_0 because w_0 makes it sound like a vector would be allowed, and that's the whole point here.
We give one boundary condition; we either give u or w dot n. So let me give it some different name, like F. So.
I'll just ask one more question to bring this home.
And then I'll get onto the fun today of fast Poisson solvers.
Suppose what was the deal on in 1-D when we had free-free boundary conditions?
What was the deal in one dimension when we had free-free boundary conditions?
Both conditions were slope conditions.
u'=0.
What matrix did we get?
What was different about that matrix, the free-free case?
I'm always looking back to 1-D because that's the case we really understood.
What was the the problem with the free-free matrix?
It was singular.
What was its name?
Was K, was it?
You remember the first day of class, the unforgettable four matrices?
So this was fixed-fixed, fixed-fixed.
This one was, you could tell me better than I can remember.
T was free-fixed.
And that was still OK. By OK I mean it was still invertable.
What was B?
Free-free, that's what I'm looking at.
And the problem with free-free and then this was periodic, that's Fourier stuff that's the next part of the course.
But the point is that this free-free was singular.
Right?
And it'll be the same here.
If the whole boundary is free, then I've got a singular problem.
If I've got a whole boundary that's free, yeah actually I'll just put it here.
Suppose I have Laplace's equation, zero, and suppose I make the whole all free.
So grad u dot n is zero on the whole boundary.
So it's like solving Bu=0.
And the point is that that has solutions.
You remember, what was in the null space of B, this free-free case?
Bu=0, for what vector?
Our favorite guilty vector.
It was constants.
Right, all ones.
All constants.
u=1, all constants.
Right, so that was the free-free case in 1-D. Now, I just want you to tell me the same thing over here.
That equation does not have a unique solution.
So it's a singular problem.
Here's the equation, something u equals zero, but you can tell me solutions to that pure Neumann problem.
That are not zero.
So this is a problem where there's a null space.
And what is the solution?
What's a solution to this?
To the Laplace's equation in the inside and slope zero all around the boundary.
There's a simple solution to that one, which is just like this guy.
And do you see what it is?
u equal constant, right.
So u equal constant.
So I'd like you to see all the whole picture connecting with what we fully understood in the 1-D case for those matrices.
We don't expect to be able to have a pure Neumann problem.
A totally free problem.
Because these, it'll be singular and we'll have to do something special.
So the point is that it would be enough for me just to fix that little bit.
If I just fix that little bit and prescribe u=u_0 on a little bit, I could make the rest of it free.
But I can't make it all free.
OK.
So those are things which, just to complete the big picture of Laplace's equation.
OK, now I'm going to turn to solving.
So now I want to solve Laplace's equation and then we'll have, because we had the review sessions coming on Tuesday and Wednesday of the past, now we're looking future.
Forward.
Ready to look forward.
Laplace's equation.
On a square.
On a square.
OK.
So let me draw this square.
Laplace's equation or Poisson's equation.
In a square.
OK, I'm going to make it the boundary conditions, so I'm going to have a mesh.
This is going to be finite differences.
Finite differences will do fine on a square.
As soon as I draw this curved region, I better be thinking about finite elements.
So that's what, today is the, like, square day.
OK, so square day I could think of finite differences.
I just draw a mesh, let's make it small.
So this is all known.
These are all known values, let me just say u is given.
On the boundary.
I'll do the Dirichlet problem.
So at all these mesh points, I know u.
And I want to find u inside.
So I've got nine unknowns, right?
Nine interior mesh points.
Three times three three.
So I've got nine unknowns, I want nine equations and I'm thinking here about difference equations.
So let me write my equation -u_xx-u_yy=f.
Poisson.
What finite difference, I want to turn this into finite differences.
That'll be my system of nine equations.
And the unknowns will be the nine values of u, so these are unknown.
Right?
Those are the unknown.
OK, and I need equations.
So let me take a typical mesh point, like this one.
I'm looking for the finite difference approximation to the Laplace.
If you give me Laplace's equation and I say OK, what's a good finite difference approximation.
That I can then put in the computer and solve, I'm ready to do it.
So what should we do for this one?
For that term?
What's the natural, totally natural thing to do to replace the finite difference replacement of minus u_xx?
Second difference.
Second difference.
And with the minus sign there, second differences around this point will be a minus one of there, two of those, a minus one of those.
That second difference in the x direction is my replacement.
My approximation.
And what about in the y direction?
Same thing.
Second differences, but now vertically.
So now I have minus one, two more, that makes this guy a four.
And this is minus one.
And on the right hand side, I'll take the value of f at the center point.
Do you see that molecule?
Let me highlight the molecule there.
Four in the center minus ones beside it.
Our matrix, we're going to produce a KU=f matrix, and what's the K going to be?
It's k2D.
Let's give a name.
A new name.
We've had k forever.
Let's have K2D.
And u is, so tell me now again just so we got, what size is K2D for this particular problem?
What's the shape of it?
How many unknowns have I got in u?
Nine.
Nine unknowns.
And notice, of course, near the boundary, if this was the, well let me take the first one.
What does the very first row of K look like?
If I number nodes like this.
So this will be node number one, two, three, four, five, six, seven, eight, nine.
So number one is sort of close to a boundary.
So I have here a minus one, a two, and this is gone.
That's like a column of our matrix.
removed.
Like a grounded node, or whatever.
But then I have the vertical one minus one, two, making this into a four.
And this guy will also be removed.
So I think if I look at K2D, it's nine by nine.
I don't plan to write all 81 entries.
But I could write the first row.
The first row comes from the first equation.
So the first row has an f_1 on the right hand side, f at this point.
And what do you see, what's on the diagonal?
Of K2D?
Four.
And what's the rest of the row?
Well there's the next point and it has a minus one.
And then this point is not connected to that, so it's a zero.
I'd better leave room for nine by nine here.
Four, minus one, zero.
And now what about number, the rest of the row?
So there's for is the center, minus one, zero.
And now I come to here, so what's the next entry in my matrix?
Minus one.
We're multiplying the unknown, the fourth unknown.
And what about the rest of the row?
This guy is not connected to five, six, seven, eight, or nine.
So now I have zero, zero, zero, zero, zero.
OK.
So that would be a row that would be a row that's sort of different because it corresponds to a mesh point that's way over near the corner.
How about the fifth row?
What would be the fifth row of K2D corresponding to this guy that's right in the middle?
What's on the diagonal, what's the 5, 5 entry of K2D?
Four.
Good.
I'm going to have a diagonal of fours.
I'm going to have fours all the way down the diagonal.
Because every point the molecule is centered at gives me the four, and I just have a minus one for its neighbor.
And this guy has all four neighbors.
They're all alive and well, so there's a guy, this is the fifth row.
There's somebody right next door on the left.
Somebody right next door on the right.
And then do the the other minus ones go after that?
This is the whole point.
I want you to see how this K2D matrix looks.
Because it's the one, it's the equation we have to solve.
So we've got to know what this matrix looks like.
So what's the point here?
Four, it's got a neighbor there.
This was unknown number five.
Its fourth unknown and the sixth unknown, what are the numbers of these unknowns?
Two and eight.
So this guy is going to have a minus one in the two position.
And then zero.
And this guy, it'll also have a minus one, is that nine?
Yeah, do you see how it's going?
I have a diagonal, I have five diagonals somehow.
I have diagonal, the neighbor on the left, the neighbor on the right, the neighbor underneath, and the neighbor above.
Across the street you put something.
So our matrix, this very important matrix, more attention has gone into how to solve this equation for that matrix than any other single specific example in numerical analysis.
It's just a model problem.
And what do I notice about this matrix?
So now you see it's got five diagonals.
But, and what's the big but?
The big but is that those five diagonals don't run, it's not just a band of five because you don't get to the one up above until you've gone all the way, you see the bandwidth is big.
If my matrix, if this is one, two up to n, and this is one, two up to n, then how many unknowns have I got?
Just think big now.
We're in 2-D, think 2-D. How many unknowns have I got?
n was three in the picture, but now you're kind of visualizing a much finer grid.
How many unknowns?
n squared unknowns, right?
One to n in each direction.
So we've got the matrix of size n squared.
Way bigger, way bigger.
I mean if n was ten, our matrix is of size 100.
And of course, that's completely simple.
The real question is when n becomes 1,000 and our matrix is at size a million.
And those get solved every day.
So, question is how.
Actually this, for a million by million matrix, this fast Poisson solver that I want to describe does it great.
Actually, so you can deal with a matrix of order a million without turning a hair.
OK now, but can we understand that matrix?
So suppose n is 100.
Just to have us.
What's the story?
This matrix, then.
What's the size of that matrix?
Did I say 100?
Or 1,000, do you want to think really big?
I guess I said 1,000.
And I can get up to a million.
OK, so n is 1,000, so we have a 1,000 by 1,000, a million unknowns.
My matrix is a million by a million.
Let me ask you this.
Here's the question that I want to ask you.
What's the bandwidth?
How far from the main diagonal out to that minus one.
In other words, this is really the full band of the matrix.
It's got a lot of zeroes inside the band.
A whole bunch.
Maybe 990 something zeroes in here.
And in here, and how far is it from here to here?
1,000, right?
It's 1,000.
So the bandwidth is 1,000.
And what if I do ordinary, so now I want to begin to think how do I solve this equation?
How do I work with that matrix?
And you have an idea of that matrix?
It's a beautiful matrix to work with.
It's very like our K matrix; it's just up to 2-D.
In fact, it's closely connected to our K2D will be closely connected to K and that's what makes life possible here.
But suppose I had to solve this equation.
Alright, so I give it to MATLAB just as a system of equations.
A million equations, a million unknowns.
So MATLAB takes a gulp and then it tries, OK?
So ordinary elimination, that's what backslash does.
By the way this matrix is symmetric, and you won't be surprised it's positive definite.
Everything is nice about that matrix.
In fact more than I've said.
OK, what happens if I do elimination on that matrix?
How long does it take, how fast is it?
Well, MATLAB will take advantage, or any code, will take advantage of all these zeroes, right?
In other words, what do I mean by take advantage?
And what do I mean by elimination?
You remember I'm going to subtract multiples of this row.
They'll be a minus one below it on this guy.
This row that has some fours and some minus ones, whatever.
Elimination, I'm subtracting a multiple of this row from this row to get that minus one to be a zero, right?
I'm eliminating this entry in the matrix.
This is ordinary, LU, this is ordinary LU of K2D.
I'm sort of thinking through LU of k 2-D. And what I'm going to conclude is that there ought to be a better way.
It's such a special matrix, and it's going to be messed up by LU.
And what do I mean by messed up?
I mean that all these great zeroes, 990-something zeroes there and there, inside the band, are going to fill in.
That's the message, if you want to talk about solving large linear systems, the big issue is fill in, it's a sparse matrix.
That's typical.
We have a local operator here, so we expect a very local, sparse matrix.
Large sparse matrices is what we're doing now.
And the problem is that zeroes here will never get touched because we don't need any elimination, they're already zero.
But all this diagonal of minus ones when we do eliminations to make those zero, then some non-zeroes are going to come in and fill in the zeroes.
So the work, how much work would elimination be, actually?
We could even think about that.
Can I just give the result for how much time would elimination take?
Ordinary elimination.
So I have n squared rows to work on.
n squared rows to work on, and then the distance, that distance was what in terms of n, the width there is?
n, right?
It was about 1,000.
So I get n numbers here and I have to work on n numbers below.
To clean up a column I'm working with a vector of length n and I'm subtracting it from n rows below.
So cleaning up a column takes me n squared operation.
The total then is N^4.
N^4, for elimination.
And that's not acceptable.
Right?
If n is 1,000, I'm up to 10^12.
So elimination is not a good way to solve.
Elimination, in this ordering, ha.
Yeah.
So can I tell you two ways, two better ways to deal with this problem?
So two better ways to deal with this problem.
One is, and MATLAB have a code that would do it, one is to change from this ordering one, two, three, four, five, six, seven, eight, nine, change to a different ordering that had less fill-in.
So that's a substantial part of scientific computing.
Is to re-order the unknowns.
To reduce the number of wasted, fill-in things that you have to fill in, and then you have to eliminate out again.
And I can get that N^4 down quite a bit that way.
OK, so that's a topic of importance in scientific computing.
Is given a large linear system, put the rows and columns in a good order.
OK, and that's something you hope the machine can do.
The machine will have a record of where are the non-zeroes.
And then so you run the little, you really run the program twice.
You run it once without using the numbers.
Just the positions of non-zeroes to get a good ordering so that the number of elimination steps will be way down.
And then you run it with the numbers in that good order to do the elimination.
So that's a topic of, one topic in this.
But.
Today's topic is using the fact that this is a square.
That this matrix has a special form.
That we can actually find its eigenvalues and eigenvectors.
And that's what makes a fast Poisson solver.
So that's the topic of the lecture and the topic of the next section in the book.
And I will get started on it now, and then complete it Wednesday.
So I want to take this particular K2D for a square and show a different way to solve the equations that uses the fast Fourier transform.
When I see, when you see, a regular mesh, regular numbers.
Nothing varying, constant diagonals, think fast Fourier transform.
The FFT has got a chance when you have matrices like this one.
And this I want to show you how it works.
I want to show you the underlying idea, and then we'll see the details of the fast Fourier transform in the Fourier section.
So, of course, everybody knows the third part of this course, which is coming up next week, we'll be well into it, is the Fourier part.
OK.
But I can give you, because we've seen the eigenvectors and eigenvalues of K, I can do this now.
Ready for the smart idea?
The good way, and it only works because this is a square, because the coefficients are constant, because everything's beautiful, here's the idea.
OK. Central idea.
The idea is, so I have this matrix K2D and I'm trying to solve a system with that particular matrix.
And the fantastic thing about this matrix is that I know what it's eigenvalues and eigenvectors are.
So let's suppose we know them.
So I know (K2D)y=lambda*y.
That's when y is an eigenvector, and remember it's got n squared components.
Lambda's a number.
And I've got a whole lot of these, y_k, lambda_k, y_k, and I've got k going from one up to n squared.
Now I want to use them.
I want to use them to solve (K2D)u=f.
Let me say, it's totally amazing that I would, in fact this is the only important example I know in scientific computing, in which I would use the eigenvalues and eigenvectors of the matrix to solve a linear system.
You see, normally you think of eigenvalue problems as like, those are harder.
It's more difficult to solve this problem than this one.
Normally, right?
That's the way to think about it.
But for this special matrix, we can figure out the eigenvalues and eigenvectors by pencil and paper.
So we know what these guys are.
And that will give us a way to solve a linear system.
So can I remember, this is now central message.
What are the three steps that to use eigenvectors and eigenvalues in solving linear equations, in solving differential equations, in solving difference equations, they're always the same three steps.
Can I remember what those three steps are?
So step one, so I'm supposing that I know these guys.
First of all, that's amazing.
To know them in advance.
And the second amazing thing is that they have beautiful structure.
And those facts make it possible to do something you would never expect to do.
Use eigenvalues for a linear system.
So here's the way to do it, three steps.
One, expand f as a combination of the eigenvectors.
c_1*y_1, up to however many we've got.
We've got n squared of them.
Gosh, we're loaded with eigenvectors.
Do you remember, this a column of size n squared.
Our problem has size n squared.
So it's big, a million.
So I'm going to expand f in terms of the million eigenvectors.
OK.
So that tells me the right hand side.
Now, you remember step two in this one, two, three process?
Step two is the easy one, because I want to get, what am I looking for?
I'm aiming for, of course, the answer I'm aiming for is K2D inverse f. Right?
That's what I'm shooting for.
That's my goal, that's the solution.
So here I've got f in terms of the eigenvectors.
Now I want to get, how do I get u in terms of the eigenvectors?
Well, what's the eigenvalue for the inverse, K to the inverse?
Do you remember?
It's one over.
It's one over the eigenvalue, right?
If this is true, then this will be true with lambda_k inverse.
It's simple and I won't stop to do it, but we could come back to it.
So I know how to get K2D inverse.
So that's what I'm going to do.
I'm going to divide, so here's the fast step.
Divide c_k, each of those c_k's, by lambda_k.
What do I have now?
I've got c_1/lambda_1*y_1 plus c_n squared, the last guy divided by its eigenvalue times its eigenvector.
And that's the answer.
Right, that's the correct answer.
If this is the right hand side, then this is the solution.
Because why?
Because when I multiply this solution by K2D it multiplies every eigenvector by eigenvalue, it cancels all the lambdas, and I get x.
Do you see, I can't raise that board, I'm sorry.
Can you see it at the back?
Should I write it up here again, because everything hinges on that.
The final, the answer u, the K2D inverse f, is the same combination that gave f. But I divide by lambda_1, c_2 y, the second eigenvector over lambda_2 and so on.
Because when I multiply by K2D, multiplying each y, each eigenvector will bring out an eigenvalue, it will cancel at and I'll have the original f. So do you see that those are the steps?
The steps are, you have to do a, you could say it's a transform into eigenspace.
I take my vector in physical space, it's got its n squared component.
At physical points.
I express it in eigenspace.
By that I mean it's a combination of the eigenvectors and now the n squared physical values are n squared amplitudes, or amounts of each eigenvector.
So that was a, that's the Fourier series idea that's the Fourier transform idea, it just appears everywhere.
Express the function in the good basis.
We're using these good functions.
In these good functions, the answer is simple.
I'm just dividing by lambda.
Do you see that I'm just dividing by lambda, where if it was a differential equation, this is just what we did for differential equations.
What did I do for differential equations?
Do you mind if I just ask you?
If I was solving dU/dt=(K2D)U.
Starting from f, start with U(0) as f. I just want to draw the analogy.
How did we solve these equations?
I expanded f in terms of the eigenvector, and then what was step two?
For the differential equation, I didn't divide by lambda_k, what did I do?
I multiplied by e^(lambda*k*t).
You see, it's the same trick.
Just get it in eigenspace.
In eigenspace it's like one tiny problem at a time.
You're just following that normal mode, and so here instead of dividing by lambda, I multiplied by either the lambda*t. And for powers of for K2D to the 100th power, I would multiply by lambda to the hundredth.
Here, I have K2D to the minus first power.
So I'm multiplying by lambdas to the minus one.
OK, and the key point is that these c's can be found fast.
And this summing up can be done fast.
Now that's what also very exceptional.
And that's where the FFT comes in.
The fast Fourier transform is a fast way to get into eigenspace and to get back.
When eigenspace happens to be frequency space.
You see that's the key.
In other words these y's and lambdas that I said we knew, these y's, the eigenvectors, are pure sine vectors.
So that this is an expansion is a sine transform.
S-I-N-E. A pure sine transform.
And that can be done fast, and it can be inverted fast.
So you see what makes this fast Poisson solver work, is I have to know eigenvectors and eigenvalues, and they have to be fantastic vectors like sign vectors.
OK, so I'll give you more, the final picture the details, Wednesday and then move on to finite elements.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK. Well, we had an election since I saw you last.
I hope you're happy about the results.
I'm very happy.
Except one thing.
There was a senator who was in 18.085.
And he lost his seat.
So we don't have a senator from 18.085 any more.
It was Senator Sununu, from New Hampshire.
So he took 18.085.
Actually, it was a funny experience.
So, some years ago I had to go to Washington to testify before the committee about increasing the funding for mathematics.
So I was nervous, of course, trying to remember what I've got to say.
More money is the main point, but you have to say it more delicately than that.
And so I got in, a lot of people were taking turns, you only get five minutes to do it.
And so I went and sat down, waited for my turn.
And here on the committee was my student, Senator Sununu.
Well at that time, I think, Representative Sununu.
So that was nice.
He's a nice guy.
Actually, the lobbyist who kind of guided me there, and found the room for me and sort of told me what to do, said I hope you gave him an A.
And the truth is I didn't.
So I guess you guys should all get As, just to be on the safe side.
Or let me know if you're going to be in the Senate anyway.
So there you go.
Anyway.
So alright.
But he wasn't great in 18.085.
OK, so you know what, this evening I just called the schedules office to confirm that it is tonight, in 54-100.
It's not a long, difficult, exam so see you at 7:30, or see you earlier at 4:00 for a review of everything that you want to ask about.
So I won't try to do an exam review this morning; we've got lots to do and we're into November already.
So I want to finish the discussion of the fast Poisson solver.
Because it's just a neat idea which is so simple.
Anytime you have a nice square grid.
I guess the message is, anytime you have a nice square or rectangular grid, you should be thinking, use the fast Fourier transform somehow.
And here it turns out that the eigenvectors are for this problem, this K2D, so we're getting this matrix K2D.
Which is of size n squared by n squared, so a large matrix.
But its eigenvectors will be, well, actually they'll be sine functions for the K2D, the fixed-fixed.
And they would be cosine functions for the B2D, free-free.
And the point is the fast Fourier transform does all those fast.
I mean, the fast Fourier transform is set up for the complex exponentials as we'll see in a couple of weeks.
That's the most important algorithm I could tell you about.
After maybe Gaussian elimination, I don't know.
But my point here is that if you can do complex exponentials fast, you can do sines fast, you can do cosines fast, and the result will be that instead of I did an operation count when I did elimination in the order one, two, three, four, five, six, ordered by along each row.
That gave us a matrix that I want to look at again, this K2D matrix.
If I did elimination as it stands, I think the count was n^4.
I think the count was size n squared, I had n squared columns to work on and I think each column took n squared steps, so I was up to n^4.
And that's manageable.
2-D problems, you really can use backslash.
But for this particular problem, using the FFT way, it comes down to n squared log n. So that's way better.
Way better.
Big saving.
In 1-D, you don't see a saving.
But in 2-D it's a big saving, and in 3-D an enormous saving.
So a lot of people would like to try to use the FFT also when the region isn't a square.
You can imagine all the thinking that goes into this.
But we'll focus on the one where it's a square.
Oh, one thing more to add.
I spoke about reordering the unknowns.
But you probably were wondering, OK, what's that about because I only just referred to it.
Let me suggest an ordering and you can tell me what the matrix K2D will look like in this ordering.
I'll call it the red black ordering.
I guess with the election just behind us I could call it the red blue ordering.
OK, red blue ordering, OK, so the ordering, instead of ordering it by, let me draw this again, I'll put in a couple more, make n=4 here, so now I've got 16, n squared is now 16.
So here's the idea.
The red blue ordering is like a checkerboard.
The checkerboard ordering, you could think of.
This will be one, this'll be two, this'll be three, this'll be four, five, six, seven, eight.
So I've numbered from one to eight.
And now nine to 16 will be the other guys.
These will be nine, ten, 11, 12, 13, 14, 15, 16.
So suppose I do that, what K2D look like?
Maybe you could, it's a neat example.
It gives us one more chance to look at this matrix.
So K2D is always going to have fours on the diagonal.
So I have 16 fours.
Whatever order I take, the equation at a typical point like say this one, that's point number one, two.
That's point number three, at a point number three I have my five point molecule with a four in the middle.
So it'd be a four in this, on that position.
And then I have the four guys around it, the minus ones, and where will they appear in the matrix?
With this ordering.
So you get to see the point of an ordering.
So this was point one, point two, point three, I'm focusing on point three.
It's got a four on the diagonal and then it's got a minus one, a minus one, a minus one, a minus one, where do they appear?
Well, all those guys are what color?
They're all color blue.
Right?
All the neighbors are colored blue.
So I won't get to them until I've gone through the red, the eight reds, and then I come back to nine.
Whatever, 11, 12, 13.
The point is they'll all be over here.
So this is like, you see that the only nodes that are next to a red node are all blue.
And the only nodes next to a blue node are red.
So this is, sorry filled in a little bit, that's where I'm sort of like four diagonals of minus ones or something.
But there you go.
That gives you an idea of what a different ordering could be.
And you see what will happen now with elimination?
Elimination is completed already on this stuff.
So all the, with elimination you want to push the non-zeroes way to the end.
That's sort of like the central idea is, it's kind of a greedy approach.
Greedy algorithm.
Use up zeroes while the sun's shining.
And then near the end, where the problem is elimination's reduced it to a much smaller problem, then you've got some work to do.
So that's what would happen here.
By the way there's a movie, on the 18.086 page.
I hope some of you guys will consider taking 18.086.
It's a smaller class, people do projects like creating this elimination movie.
So that would be math at MIT.edu/18.086.
And with the movie has a movie of different orderings.
That to try to figure out what's the absolute best ordering with the absolute least fill-in is, that's one of these NP-Hard combinatorial problems.
But to get an ordering that's much better than bye rows is not only doable but should be done.
OK, so that's the ordering for elimination.
As this is for elimination.
OK.
So that's a whole world of its own.
Ordering the thing for Gaussian elimination.
My focus in this section 3.5, so this is Section 3.5 of the book and then this is Section 3.6, which is our major one.
That's our big deal.
But my focus in 3.5 is first to try to see what the 2-D matrix is.
I mean, a big part of this course, I think, a big part of what I can do for you, is to, like, get comfortable with matrices.
Sort of see what do you look at when you see a matrix?
What's its shape?
What are its properties?
And so this K2D is our first example of a 2-D matrix, and it's highly structured.
The point is we'll have K2D matrices out of finite elements.
But the finite elements might be, well, they could be those.
This could be a finite element picture.
And then the finite element matrix on such a regular mesh would be quite structured too.
But if I cut them all up into triangles and I have a curved region and everything, an unstructured mesh, then I'll still have the good properties, this will still be symmetric positive definite, all the good stuff.
But the FFT won't come in.
OK, so now I want to take a look again at this K2D matrix, OK?
So one way to describe the K2D matrix is the way I did last time, to kind of write down the typical row, typical row in the middle.
It's got a four on the diagonal and four minus ones.
And here we've re-ordered it so the four minus ones came off.
Came much later.
But either way, that's what the matrix looks like.
Let me go back to this ordering.
And let's get 2-D, another better, clearer look at K2D.
I want to construct K2D from K. Which is this K1D.
I don't use that name, but here it is.
Let me just show it to you.
OK, so let me take the matrix that does the second differences in the x direction.
This'll be an n squared by n squared matrix.
And it'll take second differences on every row.
Let me just write in what it'll be.
It'll be K, it'll be a block matrix.
K, K, n blocks.
Each n by n. You see that that will be the second difference matrix?
K takes the second differences along a row.
This K will take the second x differences along the next row.
Row by row, simple.
So this is the u_xx part.
Now what will the u_yy, the second y derivative.
That gives second differences up the columns, right?
So can I see what that matrix will look like, second differences up the columns?
Well, I think it will look like this.
It will have twos on the diagonal.
2I, 2I, 2I, this is second differences, right?
Down to 2I.
And next to it will be a minus the identity - let me write it, and you see if you think it looks good.
So minus the identity.
It's like a blown up K. Somehow.
Right?
I have, do you see, it's every row has got a two and two minus ones.
That's from the minus 1one the two and the minus one in the vertical second difference.
But you see how the count, you see how the numbering changed it from here?
Here the neighbors were right next to each other, because we're ordering by rows.
Here I have to wait a whole n to get the guy above.
And I'm also n away from the guy below.
So this is the two minus one minus one centered there, above and below.
Do you see that?
If you think about a little it's not sort of difficult to see.
And I guess the thing I want also to do, is to tell you that there's a neat little MATLAB command, or neat math idea really, and they just made a MATLAB command out of it, that produces this matrix and this one out of K. Can I tell you what this is?
It's something you don't see in typical linear algebra courses.
So I'm contracting K2D from K1D by, this is called a Kronecker product.
It's named after a guy, some dead German, or sometimes called tensor product.
The point is, this is always the simplest thing to do in two dimensions or three dimensions, or so on.
Is like product of 1-D things, like copy the 1-D idea both ways.
That's all this thing is doing.
Copying the one idea in the x direction and in the y direction.
So the MATLAB command, I've got two pieces here.
For this first piece it's kron, named after Kronecker, of - now, let's see.
I'm going to put two n by n matrices, and Kronecker product is going to be a matrix, a giant matrix, of size n squared.
Like these.
OK, so I want to write the right thing in there.
I think the right thing is the identity and K. OK, and so I have to explain what this tensor product, Kronecker product, is.
And this guy happens to be also a Kronecker product.
But it's K and I.
So I'm just like mentioning here.
That because if you have 2-D problems and 3-D problems and they're on a nice square grid so you can like just take products of things, this is what you want to know about.
OK.
So what is this Kronecker product?
Kronecker product says take the first matrix, I.
It's n by n. Just n by n, that's the identity.
And now take this K and multiply every one of those numbers, these are all numbers, let me put more numbers in, so I've got 16 numbers.
This is going to work for this mesh that has four guys in each direction.
And four directions.
Four levels.
So I is four by four, and K is four by four and now, each number here gets multiplied by K. So because of all those zeroes I get just K, K, K, K. Which is exactly what I wanted.
So that Kronecker product just in one step you've created an n squared by n squared matrix that you really would not want to type in entry by entry.
That would be a horrible idea.
OK, and if I follow the same principle here, I'll take the Kronecker product here, as I start with a matrix K, so I start with two, minus one, minus one, two, minus one, minus one, two, minus one, minus one, two.
That's my K, and now what's the Kronecker idea?
Each of those numbers gets multiplied by this matrix.
So that two becomes 2I, minus one becomes minus I, minus one becomes minus I, it all clicks.
And it's producing exactly the u_yy part, the second part.
So that's the good construction thing.
Just to tell you about.
And the beauty is, this is just like cooked up, set up for separation of variables.
If I want to know the eigenvalues and the eigenvectors of this thing, I start by knowing the eigenvalues and eigenvectors of K. Can you remind me what those are?
We should remember them.
So, anybody remember eigenvectors of K?
Well, this is going back to Section 1.5, way early in the semester.
And there's a lot of writing involved and I probably didn't do it all.
But let me remind you of the idea.
We guessed those eigenvectors.
And how did we guess them?
We guessed them by comparing this difference equation to the differential equation.
So we're in 1-D now, I'm just reminding you of what we did a long time ago.
OK, so what did we do?
I want to know the eigenvectors of these guys.
So I better know the eigenvictors of the 1-D one first.
OK, so what did we do in 1-D?
We found the eigenvectors of K by looking first at the differential equation -u''=lambda*u u with, and remember we're talking about the fixed-fixed-fixed here.
And anybody remember the eigenvectors of this guy, eigenfunctions I guess I should say for those?
So sines and cosines look good here, right?
Because you take their second derivative, it brings down the constant.
And then do I want sines or cosines?
Looking at the boundary conditions, that's going to tell me everything.
I want sines, cosines are wiped out by this first condition that u(0) should be zero.
And then the sine has to come back to zero at one, so the eigenfunctions were u equals the sine of something, I think.
I want a multiple of pi, right?
pi*k*x. I think that would be right.
Because at x=1, that's come back to zero.
And the eigenvalue of lambda, if I just plug that in, two derivatives bring down pi*k twice.
And with a minus, and that cancels that minus.
So the eigenvalues were pi squared, K squared.
And that's the beautiful construction for differential equations.
And these eigenfunctions are a complete set, it's all wonderful.
So that's a model problem of what classical applied math does for Bessel's equation, Legendre's equation, a whole long list of things.
Now, note all those equations that I just mentioned would have finite difference analogs.
But to my knowledge, it's only this one, this simplest, best one of all, we could call it the Fourier equation if we needed a name for it.
I just thought of that.
That sounds good to me.
Fourier equation.
OK.
So what's great about this one is that the eigenvectors in the matrix case are just found by sampling these functions.
You're right on the dot if you just sample these functions.
So, for K, the eigenvectors, what do I call eigenvectors?
Maybe y's, I think I sometimes call them y.
So a typical eigenvector y would be a sine vectors.
I'm sampling it at, let's see, can I use h for this step size?
So h is 1/(n+1).
As we saw in the past.
h is 1/(n+1).
So a typical eigenvector would sample this thing at h, 2h, sin(2*pi*k*h), and so on, dot dot dot dot.
And that would be an eigenvector.
That would be the eigenvector number K, actually.
And do you see that that sort of eigenvector is well set up.
It's going to work because it ends, where does it end?
Who's the last person, the last component of this eigenvector?
It's sin(N*pi*k*h).
Sorry about all the symbols, but compared to most eigenvectors this is, like, the greatest.
OK, now why do I like that?
Because the pattern, what would be the next one if there was an n plus first?
If there was an n plus first component, what would it be?
What would be the sin((N+1)pi*k*h)?
That's the golden question.
What would be, I'm trying to say that these have a nice pattern because the guy that's the next person here would be sin((N+1_pi*k*h), which is what?
So N+1 is in there.
And h is in there, so what do I get from those guys?
The (N+1)h is just one, right?
(N+1)h carries me all the way to the end.
That's a guy that's out, it's not in our matrix because that's the part, that's a known one, that's a zero in a fixed boundary condition.
So like the next guy would be sin((N+1)h), that's one.
sin(pi*k).
And what is sin(pi*k)?
The sin(pi*k) is always?
Zero.
So we're getting it right, like the guy that got knocked off following this pattern will be zero.
And what about the guy that was knocked off of that end?
If this was sin(2pi*k*h), and sin(pi*k*h), this would be sin(0pi*k*h), which would be?
Also zero, sin(0).
So, anyway you could check just by, it takes a little patience with trig identities, if I multiply K by that vector, the pattern keeps going, the two minus one, minus one, you know at a typical point I'm going to have two of these, minus one of these, minus one of these.
And it'll look good.
And when I get to the end I'll have two minus one of these.
The pattern will still hold, because the minus one of these I can say is there.
But it is a zero, so it's OK. And similarly here.
Anyway, the pattern's good and the eigenvalue is?
Something.
OK, it has some formula.
It's not going to be exactly this guy.
Because we're taking differences of sines and not derivatives of sines.
So it has some formula.
Tell me what you would know about lambda.
I'm not going to ask you for the formula, I'm going to write it down.
Tell me, before I write it down, or I'll write it down and then you can tell me.
I have two on the diagonal, so that's just going to give me a two.
And then the guy on the left and the guy on the right, two, two with minus ones I think gives us two.
I think it turns out to be a cos(k).
A k has to be in there, a pi has to be in there, and h. I think that'll be it.
I think that's the eigenvalue.
That's the k'th eigenvalue to go with this eigenvector.
What do you notice about two minus two times the cosine of something?
What is it?
Positive, negative, zero, what's up?
Two minus twp times a cosine is always?
Positive.
What does that tell us about K that we already knew?
It's positive definite, right?
All eigenvalues, here we actually know what they all are, they're all positive.
I'm never going to get zero here.
These k*pi*h's are not hitting the ones where the cosine is one and, of course, you can imagine.
So I started this sentence and realized I should add another sentence.
These are all positive.
We expected that.
We knew that k was a positive definite matrix.
All its eigenvalues are positive, and now we actually know what they are.
And if I had the matrix B instead for a free-free one?
Then this formula would change a little bit.
And what would be different about B, the free-free matrix?
Its eigenvectors would be maybe at half angles or some darned thing happened.
And so we get something slightly different here.
Maybe h is 1/n for that, I've forgotten.
And what's the deal with the free-free K?
One of the eigenvalues is zero.
The free-free is the positive semi-definite example.
OK, that was like a quick review of stuff we did earlier.
And so I'm coming back to that early point in the book, because of this great fact that my eigenvectors are sines.
So the point is, my eigenvectors being sines, that just lights up a light saying use the fast Fourier transform.
You've got a matrix full of sine vectors.
Your eigenvector matrix is a sine transform.
It's golden, so use it.
So I'll just remember then how, recall that.
So what are the eigenvectors in 2-D?
First of all, so let's go to 2-D. Let me do the continuous one first.
Yeah, -u_xx-u_yy=lambda*u.
The eigenvalue problem for Laplace's equation now in 2-D, and again I'm going to make it on the square.
This unit square.
And I'm going to have zero boundary conditions.
Fixed-fixed.
Anybody want to propose an eigenfunction for the 2-D problem?
So the 2-D one, the whole idea is hey, this square is like a product of intervals somehow.
We know the answer in each direction.
What do you figure, what would be a good eigenfunction, u(x,y)?
Or eigenfunction, yeah, u(x,y).
For this problem?
What do you think?
This is like this.
The older courses on applied math did this until you were blue in the face.
Because there wasn't finite elements and good stuff at that time.
It was exact formulas.
You wondered about exact eigenfunctions and for this problem variables separate.
And you get u(x) is the product of sine.
Of this guy, sin(pi*k*x), times the sin(pi*l*y).
Well, once you have the idea that it might look like that, you just plug it in to see, does it really work?
And what's the eigenvalue?
So I claim that that's a good eigenfunction function and I need, it's got two indices, k and l, two frequencies.
It's got an x frequency and a y frequency.
So I need double index.
k and l will go, yeah.
All the way out to infinity in the continuous problem.
And what's the eigenvector?
Can you plug this guy in?
What happens when you plug that in?
Take the second x derivative, what happens?
A constant comes out and what's the constant?
pi squared k squared.
Then plug it into that term.
A constant comes out again.
What's that constant?
pi squared l squared.
They come out with a minus and then you already built in the minus, so that they've made it a plus.
So the lambda_kl is just pi squared, k squared, plus pi squared l squared.
You see how it's going?
If we knew it in 1-D now we get it in 2-D, practically for free.
Just the idea of doing this.
OK, and now I'm going to do it for finite differences.
So now I have K2D and I want to ask you about its eigenvectors, and what do you think they are?
Well, of course, they're just like those.
They're just the eigenvectors, shall I call them z, k, l. So these will be the eigenvectors.
The eigenvectors z_kl, will be, their components, I don't even know the best way to write them.
The trouble is this matrix is of size n squared.
Its eigenvectors have got n squared components, and what are they?
Just you could tell me in words.
What do you figure are going to be the components of the eigenvectors in K2D when you know them in 1-D?
Products, of course.
This construction, however I write it, is just like this.
z_kl, a typical component, typical components so that components of the eigenvectors are products of these guys, like sin(k*pi*h), something like that.
I've got indices that I don't want to get into.
Just damn, it I'll just put that.
Sine here or something, right.
We could try to sort out the indices, the truth is a kron operation does it for us.
So all I'm saying is we've got an eigenvector with n components in the x direction.
We've got another one with index l and components in the y direction.
Multiply these end guys by these end guys, you get n squared guys.
Whatever order you write them in.
And that's the eigenvector.
And then the eigenvalue is, so the eigenvalue will be, well what do we got?
We have a second x difference, so it will be a 2-2cos(k*pi*h).
From the xx term.
And it'll plus the other term will be a 2-2cos(l*pi*h).
That's cool, yeah.
I didn't get into writing the details of the eigenvector, that's a mess.
This is not a mess.
This is nice.
What do I see again?
I see four coming from the diagonal.
I see two minus ones coming from left and right.
I see two minus ones coming from up and below.
And do I see a positive number?
You bet.
Right, this is positive, this is positive.
Sum is positive.
The sum of two positive definite matrices is positive definite.
I know everything about k. And the fast Fourier transform makes all those calculations possible.
So maybe I, just to conclude this subject, would be to remind you, how do you use the eigenvalues and eigenvectors in solving the equation?
I guess I'd better put that on a board.
But this is what we did last time.
So the final step would be how do I solve (K2D)U=F?
You remember, what were the three steps?
I do really want you to learn those three steps.
It's the way eigenvectors and eigenvalues are use.
It's the whole point of finding them.
The whole point of finding them is to split this problem into n squared little 1-D problems, where each eigenvector is just doing its own thing.
So what do you do?
You write F as a combination with some coefficients, c_kl of the eigenvectors y_kl.
So these are vectors.
Put an arrow over them just to emphasize.
Those are vectors.
Then what's the answer, let's skip the middle step which was so easy.
Tell me the answer.
Supposed the right hand side is a combination of eigenvectors.
Then the left hand side, the answer, is also a combination of eigenvectors.
And just tell me, what are the coefficients in that combination?
This is like the whole idea is on two simple lines here.
The coefficient there is?
What do I have?
c_kl, does that come in?
Yes, because c_kl tells me how much of this eigenvector is here.
But now, what else comes in?
lambda_kl, and what do I do with that?
Divide by it, because when I multiply it'll multiply by it and bring back F. So there you go.
That's the, this u is a vector, of course.
These are all vectors.
2-D, we'll give them an arrow, just as a reminder that these are vectors with n squared components.
They're giant vectors, but the point is this y_kl, its n squared components are just products, as we saw here, of the 1-D problem.
So the fast Fourier transform, the 2-D fast Fourier transform just takes off.
Takes off and gives you this fantastic speed.
So that's absolutely the right way to solve the problem.
And everybody sees that picture?
Don't forget that picture.
That's like a nice part of this subject, is to get it's formulas and not code.
But it's easily turned into code.
Because the FFT and the sine transform are all coded.
Actually, Professor Johnson in the Math Department, he created the best FFT code there is.
Do you know that?
I'll just mention.
His code is called FFTW, Fastest Fourier Transform in the West, and the point is it's set up to be fast on whatever computer, whatever architecture you're using.
It figures out what that is.
And optimizes the code for that.
And gives you the answer.
OK. That's Part 2 complete of Section 3.5.
I guess I'm hoping after we get through the quiz, the next natural step would be some MATLAB, right?
We really should have some MATLAB case of doing this.
I haven't thought of it, but I'll try.
And some MATLAB for finite elements.
And there's a lot of finite element code on the course page, ready to download.
But now we have to understand it.
Are you ready to tackle, so in ten minutes we can get the central idea of finite elements in 2-D, and then after the quiz, when our minds are clear again, we'll do it properly Friday.
This is asking you a lot, but can I do it?
Can I go to finite elements in 2-D?
OK. Oh, I have a choice.
Big, big choice.
I could use triangles, or quads.
That picture, this picture would naturally set up for quads.
Let me start with triangles.
So I have some region with lots of triangles here.
Got to have some interior points, or I'm not going to have any unknowns.
Gosh, that's a big mesh with very little unknown.
Let me put in another few here.
Have I got enough?
Oops, that's not a triangle.
How's that?
Is that now a triangulation?
So that's an unstructured triangular mesh.
Its quality is not too bad.
The angles, some angles are small but not very small.
And they're not near, that angle's getting a little big too, it's getting toward a 180 degrees which you have to stay away from.
It was a 180, the triangle would squash in.
So it's a bit squashed, that one.
This one's a little long and narrow.
But not bad.
And you need a mesh generator to generate a mesh like this.
And I had a thesis student just a few years ago who wrote a nice mesh generator in MATLAB.
So we get a mesh.
OK. Now, what do you remember the finite element idea?
Well, first you have to use the weak form.
Let's save the weak form for first thing Friday.
The weak form of Laplace.
I need the weak form of Laplace's equation.
I'll just tell you what it is.
We all have double integrals, oh, Poisson.
I'm making it Poisson.
I want a right-hand side.
Double integral, this will be du/dx.
d test function/dx.
And now what's changed in 2-D, I'll also have a du/dy*dv/dy.
dx/dy.
That's what I'll get.
And on the right-hand side I'll just have F, the load times the test.
dx/dy.
Yeah I guess, I have to write that down because that's our starting point.
Do you remember the situation, and of course that's on the exam this evening, is remembering about the weak form in 1-D, so in 2-D I expect to see x's and y's, my functions are functions of x and y. I have my, this is my solution.
The V's are all possible test functions, and this hold for every, this is like the virtual work.
You could call it the equation of virtual work, if your were a little old-fashioned.
Those V's are virtual displacements, they're test functions.
That's what it looks like.
I'll come back to that at the start of Friday.
What's the finite element idea?
Just as in 1-D, the finite element idea is choose some trial functions, right?
Choose some trial functions.
And our approximate guy is going to be some combination of these trial functions.
Some coefficient U_1 times the first trial function plus so on, plus U_n times the nth trial function.
So this will be our, and these will also be our test functions again.
These will also be, the phis are also the V's.
I'll get an equation.
By using n tests, by using the weak form n times for these n functions.
And so each equation comes.
V is one of the phis, U is this combination of phis.
Plug it in.
You get an equation.
What do I got to do in three minutes, is speak about the most important point.
The phis.
Choosing the phis is what matters.
That's what made finite elements win.
The fact that I choose nice simple functions, which are polynomials, little easy polynomials on each element, on each triangle.
And the ones I'm going to speak about today are the linear ones.
They're like hat functions, right?
But now we're in 2-D.
They're going to be pyramids.
So if I take, this is unknown number one.
Unknown number one, it's got its hat function.
Pyramid function, sorry.
Pyramid function.
So its pyramid function is a one there, right?
The pyramid, top of the pyramid is over that point.
And it goes down to zero so it's zero at all these points.
Well, at all the other points.
That's the one beauty of finite elements, is that then this U, this coefficient U_1 that we compute, will actually be our approximation at this point.
Because all the others are zero at that point.
So can you just, this is all you have to do in the last 60 seconds is visualize this function.
You see it, maybe tent would be better than pyramid?
Either one.
What's the base of the pyramid?
The base of the pyramid, I'm thinking of the surface as a way to visualize this function.
It's a function that's one here, it goes down to zero linearly.
So these are linear triangular elements.
What's the base of the pyramid?
Just this, right?
That's the base.
Because over in these other triangles, nothing ever got off zero.
Nothing got off the ground.
It's zero, zero, zero at all three.
If I know it at all three corners, I know it everywhere.
For linear functions, it's just a piece of flat roof.
Piece of flat roof.
So I have how many pieces have I got around this point?
Oh, where is the point?
I guess I've got one, two, three, four, five, six, is it?
It's a six, it's a pyramid with six sloping faces sloping down to zero along the sides, right?
So it's like a six-sided teepee, six-sided tent.
And here are the rods that hold the tent up would be these six things.
And then the six sides would be six flat pieces.
If you would see what I've tried to draw there, that's phi_1.
And you can imagine that when I've got that, I can take the x derivative and the y derivative.
What will I know about the x derivative and the y derivative in a typical triangle?
If my function is in a typical triangle, say in this triangle, my function looks like a+bx+cy.
a+bx, it's flat.
It's linear.
And these three numbers, a and b, are decided by the fact that the function should be one there, zero there, and zero there.
Three facts about the function, three coefficients to match those facts.
And what's the deal on the x derivative?
It's just b.
It's a constant.
The y derivative is just c, a constant.
So the integrals are easy and the the whole finite element system just goes smoothly.
So then what the finite element system has to do, and we'll talk about it Friday, is gotta keep track of which triangles go to which nodes.
How do you assemble, where do these pieces go in?
But every finite, every element matrix is going to be simple.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, let's see, you probably guessed on that quiz problem three, it wasn't what I meant.
I get a zero for that problem.
But you'll get probably good numbers, so that's an election gift if it comes out that way.
So they're all in the hands of the TAs to be graded.
We have a holiday Monday, I think.
We come back to Fourier.
Now, so we just have a concentrated shot at Fourier, just about eight or nine lectures in November.
So stay with it and that'll be of course the subject of the third quiz.
Which will have no mistakes.
It'll be solved by the TAs in advance and we'll spot things.
So, and if we have the quizzes to return to you by Wednesday that will be great.
I hope so, but they have a big job.
A little bit, looking far ahead at the end of Fourier the quiz is December 4th, I think that's a Thursday.
And that's the end of the course.
So December 4th, so we'll be ending the course a little bit early.
Because I'll be in Hong Kong, to tell the truth.
And and we've done a lot, and with the review sessions we're really doing well.
So, that's the future.
Fourier, today is an important day too, finite elements in 2-D, that's a major part of computational science and engineering.
The finite element idea, the idea of using polynomials, you can find in some early papers by Courant, a mathematician in New York, and by a guy in China neat guy named Fung Kong But those papers were sort of, you could do it this way if you wanted.
It was really the structural engineers in Berkeley and elsewhere who made it happen ten years later.
And the whole idea has just blossomed.
Continues to grow.
So I had an early book, in the `70s, actually, about the mathematical underpinnings.
The math basis for the finite element method.
And many other finite element books.
Professor Bathe you know, teaches a full of course on that.
But, I think we can get the idea of finite elements here.
We did them in 1-D, and now there's a MATLAB problem and I'd like to just describe that particular problem if I can, as an example.
And, of course, you would use the code that's printed in the book, and that's available on the website just to download.
But the problem is not on a square domain.
It starts on a circle, so that the first lines of the code, the calling the MATLAB command square grid, are not applicable.
So you have to create, then, a mesh.
Well, I have a suggested mesh so I'll draw that, and then from that you want to make a list of all the node points.
A list, P, of - so what the code needs is two lists.
Well, let me draw a picture of, well, it's a circle.
And I'm going to be solving Poisson's equation.
The equation will be -u_xx-u_yy=4, in the circle.
So it's Poisson but with a constant right hand side.
That will mean that all the integrals of F times v, all the right hand side of our discrete equation will be, the integrals are all easy because we just have a constant there times the trial function.
OK, and then on the boundary is going to be u=0.
On the boundary.
So it's a classic problem.
And we can say what the solution is.
So it's one with a known solution.
I think it would be x squared.
No, I guess one, one minus x squared minus y squared.
This should all be on the .086 site.
I just didn't have a chance to look this morning to be sure it got up.
So you can watch, here.
So that, I hope, does solve the problem.
Two x derivatives give us a two, two y derivatives another two, so we get four.
So we know the answer; the question is, and I'm interested in this question, for research reasons too, is what's the error when you go to a polygon?
You go to a, these curved boundaries don't get correctly saved.
You approximate them by straight lines.
That would be the first idea.
And with this, all this symmetry, let's keep the problem nice and use a regular polygon.
So maybe I'll try to draw one with about eight sides, but, OK.
So we impose u=0 at these nodes.
So u is zero at those nodes, and then we have a mesh.
So we want to create a mesh.
OK, so with all the symmetry here, the natural idea would be to start with eight pieces, or M pieces if I have, this is a regular M side, let's say, and I'll take M to be eight in this picture.
And I think we can work on just one triangle.
By rotational symmetry, all those triangles are going to be the same.
So I think our domain is really this one triangle here.
That's where we're working.
And in that triangle, I think we have zero boundary conditions.
And across this edge I think we have natural boundary conditions.
Slope zero, if I see the picture correctly.
The rotational symmetry would mean that things are not changing.
That every triangle is the same.
So I think on these boundaries it's the Neumann condition, dU/dn=0.
And I'm frankly not sure what to do at the origin, so I'll maybe just try both ways and see.
OK, so there is a real problem.
Of course, it's artificial in the sense that we know the answer.
But it's a real open question of what does the error look like, from doing that.
So that's the goal and let me just say the problem I'll ask you to do, and it probably is quite enough to be ready for next Friday, is to use piecewise linear elements.
Which is what I'm going to do.
What every discussion of finite elements will begin with, linear elements.
Those pyramids that I spoke about it at the end of last time.
So that's what I hope, but actually I would be highly interested if anybody got into the problem, to try quadratic elements.
So I'll just say here, second-degree quadratic polynomials would be more accurate.
Would be more accurate.
So, in other words, this is a first type of finite element called P_1, for polynomials of degree one.
These guys, I would call P_2, for polynomials of degree two.
And I've mentioned here the possibility of using quads.
Instead of triangles, if I had squares for example, the simplest element would be a Q_1.
So these are, if I manage today to tell you about how to use P_1 and P_2 and Q_1, you're on your way.
And for the requirements of this course, P_1 is the first point to understand.
OK.
While I'm speaking about codes and meshes, let me draw the mesh I proposed in the homework problem.
So I thought OK, we just have to have a simple mesh here.
So let me draw that line in.
I know all these points, right?
This is 0, 0 here.
And that point's on the circle.
And so is this, that point might be, so what's the angle there?
That angle is probably pi/8.
The whole angle would be 2pi/8, and eight of them would go all the way around.
So I think that angle is pi/8.
And so this point would be cos(pi/8).
sin(pi/8).
We know where they are.
But that's going to be what we have to list.
We have to list where are the coordinates of all the mesh points.
So let me describe the rest of the mesh points and then see what that list would look like.
And once you've created the list, the code will take over.
And then you plot the results and see what's going on.
So let me suggest a mesh.
It's pretty straightforward.
I just divided this piece into N, this center thing into N, so I called that distance h. So Nh gets me out to here.
Whatever that is, that's cos(pi/8), I guess, that's the x coordinate of that line.
So those are mesh points and then let me keep drawing these.
So these will be mesh points too, and these will be mesh points too.
So at this point, I've got probably 12 or 13 mesh points.
But I've got quads, right?
Well, I've got a couple of triangles here that I'm not going to touch, those are fine.
But these are quads and they could be used with the Q_1 element, but I'm thinking let's stay with triangles.
So I just suggested to put in triangles, put in these diagonals, keeping symmetry.
And so there's the mesh.
There's the mesh.
And then what does the code ask for?
So it's got 13 mesh points.
And the code, first of all it wants a list of the coordinates of all those mesh points.
So that the code will, and I better number them, of course.
So let me number them one, shall I number them the center guys first, one, two, three, four, five, and then up here six, seven, eight, nine,, now I don't know if this is a good numbering.
Ten, 11, 12, 13.
Why don't we, just to have some consistency within plans, why don't you, don't have to take N=4.
I hope you'll take, what did I take N as four or five?
Yeah, right.
N is 4.
But you'll want to try different N's.
N=4 would be a good crude start to see what's going on, but then I hope you'll go higher and get better accuracy.
And you can see how the accuracy improves, how you get closer to that as n gets bigger.
OK, but got the nodes numbered.
Oh, I better number the triangles.
OK, how shall we number the triangles?
Shall we do along the top, or this?
I don't know.
What do you want to do for the numbering of the triangles?
Maybe run along the top, and then run along the bottom, because then it'll practically be a copy.
So I didn't leave myself much space, but one, two, three, four, five, six, seven.
Seven triangles along the top and seven along the bottom, so I have 14 triangles in this mesh.
So it's a mesh with 14 triangles and 13 nodes.
And I know the positions of everyone, right?
I know the x,y coordinates of every one.
So what the code will want is a list of those coordinates.
So a list, P, P will be a list of coordinates.
The first guy will be of, the nodes so 13 rows, three columns.
So it's a little 13 by three matrix that tells you where all the nodes are.
So the first one on that list would be (0,0).
that's for node one.
And the second one would be whatever the coordinates of that are, something zero.
(h,0), I guess it is.
The third one will be (2h,0), and so on, and then complete the list of 13 positions.
So you are then told the code where all the nodes are.
What else do you have to tell it?
Not much.
You now have to tell it about the triangles.
So now for every, why do I say three columns?
Maybe only two, is it?
You see the point already.
I've forgotten, maybe, yeah.
I don't see why.
Well for triangles, I'm going to need three, for nodes maybe it's only got two.
Maybe it's 13 by two.
I don't see why I need three.
But, anyway.
Then the other list is triangles.
So this will be the list, t, and so it takes triangle number one.
Which is right here.
That very first triangle.
And what does it have to tell us about triangle one?
The three nodes.
If it tells us the three node numbers, and this list, P, gave their positions, we've got it.
So how many triangles did that I have?
14?
So t will be 14 by three, and so the first guy will be just one, node number one, node number two, and node number six.
That will tell us which is the first triangle.
And the second triangle, I guess I've drawn from two.
Two, seven to six, right?
That's a very skinny triangle up there but it's the one that started at node two, went up to seven and back to six.
So a list like that.
And then the code will do the rest.
I hope.
Almost all the rest.
The code will create the matrix K. It'll create the matrix K with, it'll be singular.
Boundary conditions won't yet be in there.
And then a final step after that K, or maybe we could call it K_0 is created, a final step will be to fix u.
At least at these three points.
So these three will be boundary nodes.
And as I say, I'm not too sure about that one, I apologize.
At those boundary nodes I'm going to take the values to be zero.
So this is going to be zero along the whole edge, because if it's zero there, zero there, zero there and zero there, and if it's linear, it's zero.
So the final, sort of, subroutine in the code, the final group of commands you want to impose, zeroes here, that should then make the matrix K invertible, and then you've got KU=F to solve.
So what the code is doing is creating K and F. You see the overall picture?
I jumped right into this particular mesh, particular problem, but now I really should back up to where it starts.
This this is going to be the weak form of Laplace, Poisson, maybe I'll make a little space to put in Poisson's name too.
You have a picture already of what this weak form is about, so now I'm really backing up to the start.
I take the equation and I get its weak form.
And remember that's in the continuous case, as it was in the quiz problem.
The first step is the continuous weak form, and then the second step is choose test functions and, trial - I'm sorry, gosh, I'm in bad shape here.
Because they're the same.
This isn't my worst error, today.
But those are trial functions, and these are test functions.
OK, questions at this point, because I've, yeah, thank you.
Good.
Let's look at this picture
[INAUDIBLE]
Because, we did.
That's right.
So because my question is what is the continuous problem, I would like to solve has u=0 on the polygon.
So in a way you can forget the circle, where we know the answer now.
We really are looking on the polygon, and I would like to know what's the solution like on that polygon.
And then so there are two steps, the first step was start with a circle, we have the answer.
Second step is go to a polygon, continuous problem, Poisson's equation in the polygon.
How different is that from this?
Because this will not satisfy the polygon boundary conditions.
So that's the circle answer.
Then the question is, what's the polygon answer.
And I don't know that.
You may say a regular polygon, you can't do that.
I didn't think you can.
It's amazing, but probably a triangle or a square.
So if M is three or four, probably some formulas would be available.
But I think once we get higher, I don't know the answer to the Dirichlet problem, to Poisson's equation on a polygon.
On a regular polygon and that's what I would really like to know more about.
And how do I find out more about it?
By finite elements.
With your help.
Taking that polygon, breaking it into a mesh, looking only at one triangle just for simplicity, and getting u finite elements.
Well, I should say u_p_1.
That's the finite element solution using linear.
I would really like to know u_p_2, the finite element solution which will be better.
If I use quadratics.
So now I get the fun of describing the linear elements, the quadratic elements, the quads.
But did I answer that question OK?
Yeah.
So this is the problem I would like to know the answer to.
If I have this equation, zero boundary conditions on a regular polygon with M sides, what's the answer?
And it's going to be close to this, but it won't be the same.
Because this does not vanish on the polygon edges.
And I would like to compare the slopes, too.
So the homework problem asked you not only to compare u circle with u_p_1, but also the slopes.
The slopes here are easy, slopes here are easy because it's a bunch of flat functions.
So the slopes are just constant in each triangle.
OK, I'm guessing that the error gets smaller as you go in.
I think that if you plot the error, it'll be largest out here and get small there.
But remains to be seen.
So I hope you enjoy - yeah, good
[INAUDIBLE]
Rather than seven
[INAUDIBLE]
No, the middle.
There's nothing magic about any particular mesh.
I just chose this mesh as pretty good, and actually, I'm imagining M could get pretty big.
That would be interesting.
M=8 would be interesting, M=16, M=1,024, now then I'd really get interested.
OK, but so if M is 1,024, then this side would be very small.
Right?
And I just wanted more triangles.
Actually, I would like more than I've got.
I'd like, if M was really big, then probably N should be at least that big.
So I should have a thousand this way, if this is just a tiny bit.
I just want little tiny h, and then, yeah.
Actually, that might not be too bad.
If m anM N were roughly comparable, then that length would be roughly comparable to these lengths.
And the triangles would be pretty good shape.
And that's what you're looking for.
I think there's a lot of experiments to be done here.
So, I'm thinking then of M and N. Here I took N to be just four.
When M was eight.
That's fine.
But if you keep M and N roughly the same size, then you've got triangles that are not too long and skinny.
I'll tell you when you might want.
So generally you want nice shaped triangles.
You don't want angles very small or very large, usually.
But there would be, anybody in Course 16 can imagine that if I am computing the flow field past a wing, that long, thin triangles in the direction of the wing are natural.
I mean, somehow a problem like true aerodynamics is by no means isotropic.
I mean, the direction of the wing is kind of critical to whether the plane flies, right?
So don't make the wing vertical.
And if you want accuracy, then you have long, thin triangles in the direction of the flow.
But here we're not doing a flow problem.
We haven't got shocks, or trailing edges, and other horrible stuff that makes planes fly.
We just got Poisson's equation.
OK.
Thanks for those good questions, another one
[INAUDIBLE]
What would the dimension look like?
Ah, would you like me to show you something about quadratics?
Yeah.
Shall I jump into quadratics, it's kind of fun.
Quadratics, so let me just do, so I'll come back to the weak form, right it's totally, oh I'll do it now.
It's so simple I don't want to forget it.
The weak form, so I write the equation down, -u_xx-u_yy=f.
This is the continuous weak form equal f(x,y), OK?
So that's the strong form.
And I've made it the Laplace in here to keep it simple, on any right hand side.
OK, how do I get to the weak form?
Just remind me, I multiply both sides by any test function v(x,y).
Multiply by v(x,y), and then what do I do?
I integrate over the whole region.
So that's the weak form, dxdy, this is for all v, all v(x,y), all, I'll say all admissible v(x,y).
So that's the weak form.
If this holds for all this great family of v's, the idea behind it is, that if this holds for all these trial functions, test functions, v(x,y), the only way that can happen is for this to actually equal that.
That's a fundamental lemma in this part of math, and of course it has to be spelled out more than I'm doing in words.
But the idea is that if these hold for such a large class of v(x,y), then the only way that can happen is for the strong form to hold.
For this to actually match this.
OK, so that's the start.
But then what's the next step in the weak form?
I I like the right hand side but I'm not so crazy about the left hand side.
I'm not crazy about it because this says second derivatives of u, and my little roof functions, pyramid functions, haven't got second derivatives.
So I would be dead in the water without doing the natural step that makes everything beautiful, which is?
Integration by parts.
Integrate by parts.
Move derivatives of of u, on to v. One derivative onto v, off of u, so then u and v each have one derivative.
I can use my piecewise linear, piecewise quadratics, all my finite elements are going to go fine.
So I integrate by parts.
So integrate by parts, and what is that mean in 2-D?
Of course I have a double integral here.
So integrate by parts, that mean you the Green's formula.
That was the key point of this Green, or Gauss-Green's, formula.
Can I do it first in, this is -div(grad u), times vdxdy, we can write out all the terms.
We can use vector notation.
I could use that nabla, that upside down triangle notation, or whatever.
But maybe good to see it a few different ways.
So what's the point?
When I integrate by parts, that minus disappears to a plus, I have a double integral then, and these derivatives move off of, I'm taking one derivative off of here, the divergence moves over there, but when the divergence moves onto v it becomes?
The transpose.
It becomes gradient.
And so this is gradient view, gradient of v. dxdy, plus boundary terms.
The integral of, what is, let's see.
What do I have in this integral, I have grad u dot n, times v around the boundary.
And that's with my boundary conditions that's going to be gone, so I can come back to that.
Now, you all looked a little uncertain when I wrote Green's formula this way.
For this problem I can write it more easily.
This is my left side.
I want to write the answer, I just want to write this weak form in a much simpler form.
So let say, what have I got here.
Well, all I've got is one derivative is moving off of u and on to v. And the minus sign is disappearing, so I have du/dx times dv/dx.
Right?
One off of u, onto v. The other term, one y derivative, moving off of this and onto v. Minus sign again going to a plus.
du/dy, dv/dy.
That's the integral.
That's it, that's cool.
Easy to do.
And on the right hand side of course I have no change.
The integral of f(x,y)*v(x,y)*dy.
Now, that's the weak form, dx/dy.
Here it is, weak form.
That's pretty nice.
Beautifully symmetric, though the matrix that comes up when we plug in finite specific trial functions and test functions is going to be a symmetric matrix K. And the integrals of first derivatives, so as long as our functions, our trial functions and test functions are continuous, that is, they shouldn't jump.
If the trial functions or test functions jump, then if I have a jump, then the derivative would be a delta.
I'd have another delta here, I'd have an integral delta, a delta times delta, and I don't want that.
That's infinite.
Those discontinuous elements would not be conforming, and that's a whole new world of discontinuous Galerkin.
I'd have to impose penalty stuff, and Professor Peraire I mentioned.
And others, Professor Darmofal in aero are experts on this.
We're doing continuous form.
CG.
Our piecewise linear, piecewise quadratic, they'll be continuous.
All I have to do is these derivatives.
Integrate those things and that's what the code will do.
OK, I've got to the weak form.
That's the weak form.
Now comes the finite element idea.
So there is our weak form, now ready for the finite element idea.
OK, so what was that idea?
That's the continuous problem.
Now, the finite element idea is, plug in U as a combination.
Let me write out the terms.
You know what's coming here.
If I'm using finite elements, I'm going to choose nice polynomials, phi, say, N of them.
That would be like, one for every node, so I would have 13 functions here.
I'm going to choose the v's to be the same as the phis.
And then, I'm working then in 13 dimensions instead of infinite dimensions.
So what do I do?
For this limited subspace, this finite element subspace, this piecewise polynomial, piecewise linear subspace, I plug that into the weak form and I test it against 13 V's, which are phis.
So I plug that in, so now what is K?
Now let me just say, so I now have the integral of, yeah I guess I'd better plug it in.
K_ij would then be the integral.
I'm just copying the weak form in.
Of dU/dx, no, sorry I'd better just plug it in first.
dU/dx*dV/dx plus dU/dy*dV/dy, those are the integrals I have to do.
And on the right hand side I have to do the fV.
OK, plug that in.
That's the integral over the whole domain.
When I plug it in this U is a combination of known functions and the V's will be the same guys.
So what am I going to get here?
It's just as in 1-D.
So no new ideas entering here.
The new idea's going to enter when I construct these phis.
Let me just say, though, one thing.
In 1-D, we've pretty much had a choice of, when it was one dimension.
Just remember that.
In one dimension, when I had these hat functions, when I had these guys, integrated against these guys, I pretty much had a choice of did I want to think about integrating that hat function against that one.
Or actually it was their derivatives.
It was the integral of U, yeah.
Of phi, what I needed was all the integrals of phi_i', phi_j'.
Those are what I needed, these go into K. Into the matrix K. In fact, that's what equals K_ij, the integral of phi prime.
In 1-D. OK. Now, what I was going to say, I could do it this way if I wanted.
But you remember the other way to do it?
Was elements at a time.
So this was one method here.
That found the entries of K separately, one by one.
The other way was take the elements, one by one.
So the other way was take an element like this element.
It's got two functions, two trial functions are involved there.
There's a little two by two, so this is four.
Two by two element matrices.
K equals.
And the quiz recalled that part.
That approach.
So what I want to say is that's the right way to do it in two dimensions.
A triangle at a time.
That's the way the code will do it.
It creates these little element matrices, and then it stamps them into the big matrix K. Alright.
So I want to do this integral one triangle at a time.
Is the good way.
OK, and that's what the code will do.
Actually, I think that the best way to learn these steps is just to read the lines of the code.
You can read them in the book, Page 303 or something.
And you'll see it just doing all the steps that need to be done.
One triangle at a time.
So, now.
Now comes the fun.
I get to answer what do these piecewise linear elements look like.
What do the quadratic elements look like.
What do the Q_1 quad elements look like?
This was the golden age of finite elements, when people invented these ways to create piecewise polynomials.
And it continues.
People are still inventing, I had a email this week, somebody says I've got spectral elements.
People are going higher and higher degrees.
You, know sixth degree, eighth degree.
In order to get more accuracy.
OK, let's start with P_1.
How do I describe a P_1 element inside a triangle?
So in a triangle, the unknowns will be the value, this has a height U_1, this has a height U_2, and a height U_3 at those nodes.
Inside the triangle, the function U is linear.
a+bx+cy.
Then, you see that if I know these three values, then I know these three numbers.
And vice versa.
There's a three by three matrix, right?
There has to be a three by - any time you see pictures like this, this is like the good part of 18.085 is to realize that if I have three numbers here, three values and I've got three coefficients, that there's some three by three matrix that connects them that you're going to need.
That's like a meta-message of this course.
Is, you've got to translate between the node values and the coefficients.
Because the node values are the unknowns, right?
These are the guys that are multiplying the pyramid function, this is multiplying a pyramid function with height one.
At that point, going down to zero, so this one will be a pyramid function of height U_2 times one, going down to zero.
And U_3.
So we've got a flat function in here.
And it looks exactly like that.
OK?
So what do I want to say?
When we know the positions of these three nodes from our list, P, right?
These were the crucial things we did.
The positions of all the nodes, we know where they are.
Then there has to be a three by three matrix that will now connect to the coefficients.
Why do we want the coefficients?
Because those are what we do when we integrate.
The coefficients are what we need, we need to integrate dU/dx, dU/dy, dU/dz.
Sorry, dU/dx, dU/dy.
Are you visualizing this overall solution capital U, yeah.
So what the overall solution capital U, you should visualize with a combination of all the little u's, is zero here and then it's going to go up and these triangles and bend around and, I don't know, maybe down again.
Or maybe, no, maybe it keeps going up.
This is probably the largest value, because it's the largest value and the correct solution.
So is this is probably going to be the highest point of this.
What's the Forbidden City, right?
In China, is in Beijing is like, or a single, do pagodas have flat?
No.
We we will meet pagoda functions.
But this would be just an ordinary western roof, I guess.
Just flat pieces.
Yeah.
OK, see, you've got to see the whole thing and then you look at each piece.
Each piece looks like that, and the integrals are doable.
OK, so while I'm going here, I want to do quadratics.
You'll get the idea right away.
So, same triangle, now I'm going to have quadratics.
So I'm now going to have, so this won't be the arrow, this arrow will now go this way.
I'm going to have dx squared, exy, and f y squared.
So now how many coefficients have I got to determine a quadratic?
Six, right?
a, b, c, d, e, f. How many nodes do I need?
Six.
Where are they?
Well, the natural positions are those guys in the mid-point.
So now, those are all nodes now.
Some nodes are at vertices of triangles, some nodes are at midpoint.
But remember, we've got other triangles hooking on here, many other triangles, all with their own six nodes.
Well, not their own, because they share.
That's a big point.
So there's a grid of triangles, with nodes for quadratic.
And we've got one, two, three, four, five, six, seven, eight, nine, ten, 11, 12, 13, 14, 15, 16 nodes, I think.
And within each triangle, this is what we've got.
So there's a six by six matrix for each triangle.
A six by six matrix which will connect the values U_1, U_2, U_3, U_4, U_5, U_6 for this triangle.
Connect those six heights with these six numbers.
And what will the roof look like within that triangle?
Well, sort of curved.
A parabola, right?
A parabola somehow in 2-D, it'll look like this, yeah.
Yeah.
And here's the key question.
Will that roof, that curvy roof, fit the one over there?
Because if it didn't fit, we're in trouble.
This derivative would have a delta function, and we've got delta functions, and integral squaring them would give infinite.
So here's the question.
Why does this roof, using these six points, fit on to the roof that uses U_7, U_8, U_9, and U_3 U_4 and U_6?
Why do those two roofs fit together?
This one piecewise polynomials?
Of course, the slope will change.
But the roof won't have a gap.
Water won't go through it.
Why's that?
Do you see why?
Because what do they share, what do those two curvy roofs share?
They share a side.
They share the same values along the side.
And are those three values that are shared along the side sufficient to make it match all along the side?
Yes.
That's the important question.
Finite elements lives or dies on that question.
The answer is yes, because along that side, if I just focus on that side, where these three values are shared on both sides, by the triangle on both sides.
Along that edge, what kind of a function have I got?
It's second degree.
This is whatever, when I restrict this to just run along a line, it's a parabola.
And the parabola is determined by those three values.
So having it right at three points means I have it right the whole way.
Yeah.
So there you see what quadratic elements would look like, and you could extend the code in the book and on the CSE site to work for quadratic elements.
And you want to just guess what cubic elements could look like?
I'm sorry, we've run five minutes over, but maybe finite elements is worth it.
So if I had cubic elements, any idea how many?
So I'm now going up to, I'm adding g x cubed, h, i, j, any idea how many coefficients I now have?
Four new ones plus these six is ten.
I need ten nodes.
Where I am I going to put ten nodes in this triangle?
I want to put them, I'd like to have some on the edges.
Because the edges help me make triangles match each other.
They'll just be like bowling balls.
So here's six, oops, that wouldn't be believable.
Is that right?
Four, three, two, and one.
Yeah.
Yeah.
OK.
So, now I've got a bubble node inside and I've got four nodes of vertices and two points, at two 1/3 points, and that will then match the triangle next to it.
Because four points determine a cubic.
There you go, I hope you have fun, I hope you have a great holiday.
I'll see you Wednesday for Fourier and always open for questions on the MATLAB.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
I hoped I might have Exam 2 for you today, but it's not quite back from the grader.
It's already gone to the second grader, so it will not be long.
And I hope you've had a look at the MATLAB homework for a variety of possible.
I think we've got, there were some errors in the original statement, location of the coordinates, but I think they're fixed now.
So ready to go on that MATLAB.
Don't forget that it's four on the right-hand side and not one, so if you get an answer near 1/4 at the center of the circle, that's the reason.
Just that factor four is to remember.
I'll talk more about the MATLAB this afternoon in the review session right here.
Just to say, I'm highly interested in that problem.
Not just increasing N, the number of mesh points in the octagon, but also increasing the number of sides.
So there are two numbers there, we had N points on a ray, out from the center.
But we have M sides of the polygon.
And I'm interested in both of those, getting big.
Growing.
I don't know how.
And maybe a reasonable balance is to take, I think N proportional to M is a pretty good balance.
So I'd be very happy; I mean I'm very happy with whatever you do.
But I'm really interested to know what happens as both of these increase.
How close, how quickly do you approach the eigenvalues of a circle.
And you might keep the two proportional as you increase them.
So let me say more about that this afternoon, because it's a big day today, to start Fourier.
Fourier series, the new chapter, the new topic.
In fact, the final major topic of the course.
So I tried to list here, so here I'm in Section 4.1, so I'm talking about Fourier series.
So Fourier series is for functions that have period 2pi.
It involves things like sin(x), like cos(x) like e^(ikx), all of those if I increase x by 2pi, I'm back where I started.
So that's the sort of functions that have Fourier series.
Then we'll go on to the other two big forms, crucial forms of the Fourier world.
But 4.1 starts with the classical Fourier series.
So I realize, you will have seen, many of you will have seen Fourier series before.
I hope you'll see some new aspects here.
So, let me just get organized.
It's nice to have some examples that just involve sine.
And since the sine is an odd function, that means it's sort of anti-symmetric across zero, those are the functions that will have only sine.
That will have a sine expansion.
Cosines are the opposite.
Cosines are symmetric across zero.
Like a constant, or like cos(x).
Zero comes right at the symmetric point.
So those will have only cosines.
And a lot of examples fit in one or the other of those, and it's easy to see them.
The general function, of course, is a combination odd and even.
It has cosines and it has sines, it's just the some of the two pieces.
So, this is the standard Fourier series, which I couldn't get onto one line, but it has all the cosines including this slightly different cos(0), and all the sines.
But because this one has these three different pieces, the constant term, the other cosines, all the sines, three slightly different formulas, it's actually nicest of all, to use this final form.
Because there's just one formula.
There's just one kind.
And I'll call its coefficient c_k, and now they multiply e^(ikx), so we have to get used to e^(ikx).
We may be more familiar with cos, sin(kx) and cos(kx), but everybody knows e^(ikx) is a combination of them.
And if we let k go from minus infinity to infinity, so we've got all the terms.
Including e^(-i3x), and e^(+i3x), those would combine to give cosines and sines of 3x.
We get one nice formula.
There's just one formula for the C's.
So that's one good reason to look at the complex form.
Even if our function is actually real.
That form is kind of neat, and the second good reason, the really important reason, is then when we go to the discrete Fourier transform, the DFT, everybody writes that with complex numbers.
So it's good to see complex numbers first and then we can just translate the formulas from.
And these are also almost always written with complex numbers.
So this is the way to see it.
OK, so what do we do about Fourier series?
What do we have to know how to do and what should we understand?
Well, if you've met Fourier series you may have met the formula for these coefficients.
That's sort of like step one.
If I'm given the function, whatever the function might be, might be a delta function.
Interesting case, always.
Always interesting.
Always crazy right?
But it's always interesting, the delta function.
The coefficients can be computed.
The coefficients, you'll see, I'll repeat those formulas.
They involve integrals.
What I want to say right now is that this isn't a course in integration.
So I'm not interested in doing more and more complicated integrals and finding Fourier coefficients of weird functions.
No way.
I want to understand the simple, straight, the important examples.
And here's a point that's highly interesting.
In practice, in computing practice, we're close to computing practice here.
In everything we do.
I mean, this is really constantly used.
And one important question is, is the Fourier series quickly convergent?
Because if we're going to compute, we don't want to compute a thousand terms.
Hopefully ten terms, 20 terms would give us good accuracy.
So that question comes down to how quickly does those a's and b's and c's go to zero?
That's highly important.
And you'll connect this decay rate, we'll connect this with the smoothness of the function.
Oh, I can tell you even at a start.
OK, so I just want to emphasize this point.
We'll see it over and over that like for a delta function, which is not smooth at all, we'll see no decay at all.
In the coefficients.
They're constant.
They don't decrease as we go to higher and higher frequencies.
I think of k here, I'll use the word frequency for k. So high frequency means high k, far off the Fourier series, and the question is, are the coefficients staying up there big, and we have to worry about them.
Or do they get very small?
So a delta function is a key example and then a step function.
So what will be the deal with those?
If I have a function that's a step function, I'll have decay at rate is 1/k.
So they do go to zero.
The thousandth coefficient will be roughly of size 1/1000.
That's not fast.
That's not really fast enough to compute with.
Well, we meet step functions, I mean, functions with jumps.
And we'll see that their Fourier series, the coefficients do go to zero but not very fast.
And we get something highly interesting.
So when we do these examples, so I've sort of moved on to examples, so these are two basic examples.
What would be the next example?
Step function.
Well, yeah, or maybe a hat next.
A hat function would be, you see what I'm doing at each step?
I'm integrating.
A hat function might be the next, yeah, a ramp, exactly.
Hat function, which is a ramp with a corner.
Now, so that's one integral better.
You want to guess the decay rate on that one?
k squared.
Now we're getting better.
That's a faster follow-up.
1/k^2.
And then we integrate again, we'd get 1/k^3.
Then one more integral, 1/k^4 would be a cubic spline with, you remember the cubic spline is continuous.
Its derivative is continuous, that gives us a 1/k^3.
Its second derivative is continuous, that gives us a 1/k^4, and then you really can compute with that, if you have such a function.
So, point, pay attention to decay rate.
That, and the connection to smoothness.
So examples, we'll start right off with these guys.
And then we'll see the rules for the derivative.
Oh yeah, rules for the derivative.
The beauty of Fourier series is, well, actually you can see this.
You can see the rule.
Let me just show you the rule for this.
So the rule for derivatives, the whole point about Fourier is, it connects perfectly with calculus.
With taking derivatives.
So suppose I have F(x) equals, I'll use this form, the sum of c_k*e^(ikx).
And now I take its derivative.
dF/dx.
What do you think is the derivative, what's the Fourier series for the derivative?
Suppose I have the Fourier series for some function, and then I take Fourier series for the derivative.
So I'm kind of going the backwards way.
Less smooth.
I'm going from, the derivative of the step function involves delta functions, so I'm going less smooth as I take derivatives.
It's so easy, it jumps at you.
What's the rule?
Just take the derivative of every term, so I'll have the sum of, now what happens when I take the derivative?
Everybody see what happens when I take the derivative of that typical term in the Fourier series?
What happens?
The derivative brings down a factor, ik.
With k being the thing that, so it's ik times what we have.
So these are the Fourier coefficients of the derivative.
And that again makes exactly the same point about the decay rate or the opposite, the non decay rate.
As I take the derivative you got a rougher function, right?
Derivative of a step function is a delta, derivative of a hat would have some steps.
We're going less smooth as we take more derivatives.
And every time we do it, we see, you understand the decay rate now?
Because the derivative just brings a factor ik, so its high frequencies are more present.
Have larger coefficients.
So and of course, the second derivative would bring down (ik)^2.
So that our equations, for example, let me just do an application here.
Without pushing it.
Our application, we started this course with equations like -u''(x)=delta(x-a).
Right?
If we wanted to apply to a differential equation, how would I do it?
I would take the Fourier series of both sides.
I would look at, I'd jump into what people would call the frequency domain.
So this is a differential equation written as usual in the physical domain.
And with physical variable x position.
Or it could be time.
And now let me take Fourier transforms.
So what would happen here?
If I take the Fourier transform of this, well, we'll soon see, right?
We get Fourier coefficients of the deltas.
Of the delta function.
That's a key example, and you see why.
Over here, what will we get?
And now I'm taking two derivatives, so I bring down ik twice.
So I'm looking.
Here it would be the sum of whatever the delta's coefficients are.
Shall we call those d?
The alphabet's coming out right.
d for delta.
So the right side has coefficients, d_k.
And what about the left side?
What are the coefficients if the solution u has coefficients c_k, so let's call this u now.
Has coefficients c_k, then what happens to the second derivative?
ik, ik again, that's i squared k squared, the minus sign.
So we would have the sum of k squared c_k*e^(ikx).
This is if u itself has coefficient c_k, then -u'' has these coefficients.
So what's up?
How would we use that?
It's going to be easy.
We'll just match terms.
Right?
I can see, what's my formula, what should c_k be if I know the d_k?
I'm given the right-hand side.
We're just doing what's constantly happening, this three step process.
You're given the right side.
Step one, expand it in Fourier series now.
Step two, match the two sides.
So what's the formula for c_k?
In this application, which by the way I had no intention to do this.
But it jumped into my head and I thought why not just do it.
What would be the formula for c_k?
It'll be d_k divided by?
k squared.
You're just matching terms.
Just the way, when we expanded things in eigenvectors, we'd match the coefficients of the eigenvectors, and that involved just the simple step, here it's d_k over k squared.
Good.
And then what's the final step?
The final step is, now you know the right coefficients, add them back up.
Add the thing back up, like here.
Only I'm temporarily calling it u, to find the solution.
Right?
Three steps.
Go into the frequency domain.
Write the right-hand side as a Fourier series.
Second quick step is look at the equation for each separate Fourier coefficient.
Match the coefficients of these eigenvectors.
Eigenfunctions.
And that's this quick middle step.
And then you've got the answer, but you're still in Fourier space, you're still in frequency space.
So you have to use these, put them back to get the answer in physical space.
Right?
That's the pattern.
Over and over.
So that's sort of the general plan of applying Fourier.
And when does it work?
When does it work?
Because, I mean it's fantastic when it works.
So what is it about this problem that made it work?
What is Fourier happy?
You know, when does he raise his hand, say yes I can solve that problem?
OK, what do I need here for this plan to work?
I certainly don't need always just -u'', Fourier could do better than that.
But what's the requirement for Fourier to work perfectly?
Well, linear equation, right?
If we didn't have linear equations we couldn't do all this adding and matching and stuff.
So linear equations.
Well, OK. Now, what other linear equations?
Could I have a c(x) in here?
My familiar c(x), variable material property inside this equation?
No.
Well, not easily, anyway.
That would really mess things up if there's a variable coefficient in here then it's going to have its own Fourier series.
We're going to be multiplying Fourier series.
That comes later and it's not so clean.
So we want, it works perfectly when it's constant coefficients.
Constant coefficients in the differential equations.
And then one more thing.
Very important other thing.
The boundary conditions.
Everybody remembers now, it's a part of the message of this course is that boundary conditions are often a source of trouble.
They're part of the problem, you have to deal with them.
Now, what boundary conditions do we think about here?
Well, fixed-fixed was where we started.
So if we had fixed-fixed boundary conditions what would I expect?
Then things would give me a sine series, possibly.
Because those are the eigenfunctions we're used to.
Fixed-fixed, it's sines that go from zero back to zero.
Fixed-free will have some sines or cosines.
Periodic would be the best of all.
Yeah, so we need nice boundary conditions.
So the boundary conditions, let me just say, periodic would be great.
Or sometimes a fixed-free, are familiar ones.
At least in simple cases can be dealt with.
OK.
So now, boy, that board is already full of formulas.
But, let's go back to the start and say how do we find the coefficients?
So because that was the first step.
Take the right-hand side, find its coefficient.
If we want to, just as applying eigenvalues, the first step is always find eigenvalues.
Here, in applying Fourier, the first step is always find the coefficients.
So, how do we do that?
And at the beginning it doesn't look too easy, right?
Because let me take the first guy, sin(x).
Let me take an example.
Particular S(x).
The most important, interesting function, S(x).
I want it to be an odd function, so that it will have only sine.
And I should have two period, 2pi.
So let me just graph it.
So it's going to have coefficients, and I use b for sine, so it's going to have b_1*sin(x), and b_2*sin(2x), and so on.
And so it's got a whole infinity of coefficients.
Right?
We're in function space.
We're not dealing with vectors now.
So how is it possible to find those coefficients?
And let me chose a particular S(x) so I'll put, since it's 2pi periodic, if I tell you what it is over a 2pi interval, just, repeat, repeat, repeat.
So I'll pick the 2pi interval to be minus pi to pi here.
Just because it's a nice way, and so that's a 2pi length.
There's zero, I want to function to be odd across zero.
And I want it to be simple, because it's going to be an important example that I can actually compute.
So I'm going to make it a one.
And a minus one there.
So, a step function.
A step function, a square.
And if I repeat it, of course, it would go down, up, down, up, so on.
But we only have to look over this part.
OK. Now, well, you might say wait a minute how are we going to expand this function in sine.
Well, sines are odd functions.
Everybody knows what odd means?
Odd means that S(-x) is -S(x).
So that's the anti-symmetric that we see in that graph.
We also see a few problems in this graph.
At x=0, what is our sine series going to give us?
If I plug in x=0 on the right-hand side I get zero, certainly.
So this sine series is going to do that.
And actually Fourier series tend to do this.
In the middle of a jump it'll pick the middle point of a jump.
Fourier series generally, it's the best possible, will pick the middle point of the jump.
And what about at x=pi?
At the end of the interval?
What does my series add up at x=pi?
Zero again, because sin(pi), sin(2pi), all zero.
And that'll be in the middle of that jump.
So it's pretty good.
But now what I'm hoping is that my sine series is going to somehow get real fast up to one, and level out at one.
We're asking a lot.
In fact, when Fourier proposed this idea, Fourier series, there was a lot of doubters.
Was it really possible to represent other functions, maybe even including a step function, in terms of sines or maybe cosines?
And Fourier said yes, go with it.
So let's do it.
OK, so and he turned out to be incredibly right.
How do I find b_2?
Do you remember how to, I don't want to know the formula.
I want to know why.
What's the step to find the coefficient b_2?
Well, the step is, the key point.
Which makes everything possible.
Why don't I identify the key point without which we would be in real trouble.
The key point is that all these sine functions, sin(2x), sin(3x), sin(4x), are orthogonal.
Now, what do I mean by two functions being orthogonal?
Somehow my picture in function space, so my picture in function space is that here is, this is the sine x coordinate.
And somewhere there's a sin(2x) coordinate and it's 90 degrees and then there's a sin(3x) coordinate, and then there's a sine, I don't know where to point now.
But there is a sin(4x), and we're in infinite dimensions.
And the sine vectors are an orthogonal basis.
They're orthogonal to each other.
What does that mean?
Vectors we take the dot product.
Functions, we take, we don't use the word dot product as much as inner product.
So let me take the inner product of, the whole point is orthogonality.
Let me write that word down.
Orthogonal.
The sines are orthogonal.
And what does that mean?
That means that the integral over our 2pi interval, or any 2pi interval, of one sine, sin(kx), let's say, multiplied by another sine, sin(lx), the x is, you can guess the answer.
And everything is depending on this answer.
And it is?
Zero.
It's just terrific.
If k is different from l, of course.
If k is equal to l then I have to figure that one out.
I'll need that one.
What is it if sine, if k=l so I'm integrating sine squared of kx, then it's certainly not zero.
I getting like, the length squared of the sin(kx) function.
If k=l, what is it?
It has some nice formula.
Very nice.
Let's see.
Sine squared, do I need to think about sine squared kx?
Sine squared kx, what does it do?
Well, just graph sine squared x.
What would the graph of sine squared x look like, from minus pi to pi?
So it goes up, right?
Doesn't it go up?
And then it goes back down.
OK.
Sorry, I made that a little hard.
Is that right?
And then it keeps it up.
Right.
So, what's the integral of that?
I'm not seeing quite why.
The answer is its average value is 1/2.
The integral of sine squared is 1/2 of the length.
The whole interval is of length 2pi, and we're taking the area under sine squared.
I may have to come back to it, but the answer would be half of 2pi, which is pi.
Yeah, yeah.
So you could say the length of the sine function is square root of pi.
So these are integrals.
You told me the answer was zero.
And I agreed with you, but we haven't computed it.
And nor have we really got that.
So a little bit to fix, still.
But the crucial fact, I mean, those are highly important integrals that just come out beautifully.
And beautifully really means zero.
I mean, that's the beautiful number, right, for an integral.
OK, so now how do I use that?
Again, I'm looking for b_2.
How do I pick off b_2, using the fact that sin(2x) times any other sine integrates to zero.
Ready for the moment?
To find the coefficient b_2?
I should, let me start this sentence and if you finish it.
I'll multiply both sides of this equation by sin(2x).
And then I will integrate.
I'll multiply both sides by sin(2x), so I take S(x)sin(2x).
And on the right hand, I have b_1*sin(x)sin(2x).
And then I have b_2.
Now, here's the one that's going to live through the integration.
It's going to survive, because it's the sin(2x) times sin(2x) sin(2x) squared.
And then comes the b_3 guy, would be b_3*sin(3x)sin(2x).
Everybody sees what I'm doing?
As we did with the weak form in differential equations, I'm multiplying through by these guys.
And then I'm integrating over the interval.
And what do I get?
Integrate everyone dx.
And what's the result?
What is that integral?
Zero.
It's gone.
What is this integral, the integral of sin(3x) times sin(2x)?
Zero.
All those sines integrate to zero, and I have to come back and see it's a simple trig identity to do it.
To see why that's zero.
Do you see that everything is disappearing, except b_2.
So we finally have the formula that we want.
Let me just with put these formulas down.
So b_k, b_2 or b_k, yeah tell me the formula for b_k.
Let me go back, here.
What did b_2 come out to be?
So I have b_2, that's a number.
It's got this right-hand side.
That's the integral that I mentioned.
You'd have to compute that integral.
And then what about this stuff?
This sin(2x) squared?
I've integrated that.
And what did I get for that?
This is b_2, and then this is some number.
And it's pi.
So this is b_2, and multiplying, right?
That b_2 comes out, and then I have the integral of sine squared 2x, and that's what's pi.
So that's b_2 times pi here, and I just divide by the pi.
So I divide by pi and I get the integral from minus pi to pi of my function times my sine.
That's the model for all the coefficients of orthogonal series.
That's the model.
Cosines, the complete ones, the complex coefficients.
The Legendre series, the Bessel series, everybody's series will follow this same model.
Because all those series are series of orthogonal functions.
Everything is hinging on this orthogonality.
The fact that one term times another gives zero.
What that means, really.
I want to say it with a picture, too.
so let me draw two orthogonal directions.
I intentionally didn't make them just x and y axes.
This might be the direction of sin(x), and this might be the direction of sin(2x).
And then I have a function.
And I'm trying to find out how much of sin(2x) has it got in it?
How much of sin(x) has it got in it, and then of course there's also a sin(3x) and all the other sin(kx)'s.
The point is, the point of this 90 degree angle there is, that if I can split this S(x), whatever it might be, I can find its sin(x) piece directly.
By just projecting it, it's the projection of my function on that coordinate.
If you don't like sin(x), sin(2x), S(x), write v_1, v_2, whatever.
To think of it as vectors.
What's the sin 2?
So that is b_1*sin(x).
That's the right amount of sin(x).
And the whole point is that that calculation didn't involve b_2 and b_3 and all the other b's.
When I'm projecting onto orthogonal directions, I can do them one at a time.
I can do one one-dimensional projection at a time.
This b_ksin(kx) is the, so I'm just saying this in words, is the projection of my function onto sin(kx).
And the point is, I could do this and get this answer because of that 90 degree angle.
If I didn't have 90 degrees, do you see that this wouldn't work?
Suppose my two basis functions are at some 40 degree angle.
Then I take my function.
Can I project that onto this guy?
And project that onto this guy, so the projections are there?
And there?
Do they add back to the function that I started with?
The given function?
No way.
I mean, these are much too big, right?
If I add that one to this one I'm way out here somewhere.
But over here, with 90 degrees, these are the two projections, project there.
Project there.
Add those two pieces and I got back exactly.
I just want to emphasize the importance of orthogonality.
It breaks the problem down into one-dimensional projections.
So here we go with b_k*sin(kx).
OK, let me do the key example now.
This example.
Let me find the coefficients of that particular function S(x).
This is the step function, the square wave, S(x), let's find its coefficients.
I'll just use this formula.
OK, maybe I'll erase so that I can write the integration right underneath.
OK. Oh, one little point here.
Well, not so little, but it's a saving.
It's worth noticing.
The reward for picking off the odd function is, I think that this integral is the same from minus pi to zero as zero to a pi.
In other words, I think that for an odd function, I get the same answer if I just do the integral from zero to pi, that I have to do.
And double it.
So I think if I just double it, I don't know if you regard that as a saving.
In some way, the work is only half as much.
It'll make this particular example easy, so let me do this example.
What are the Fourier coefficients of the square wave?
OK, so I'll do this integral.
So from zero to pi, what is my function?
My N from the graph?
Just one.
This is going to be a picnic, right?
The function is one here.
So S(x) is one, so I want 2/pi, the integral from zero to pi of just sin(kx)dx, right?
Which is, so I've got 2/pi, now I integrate sin(kx), I get minus cos(kx), right?
Between zero and pi.
And what else?
What have I forgotten?
The most important point.
The integral of sin(kx) k x is not minus cos(kx).
I have to divide by k. It's the division by k that's going to give me the correct decay rate.
2/(pi*k).
Alright, now I've got a little calculation to do.
I have to figure out what is cos(kx) at zero, no problem, it's one.
And at the other point, at x=pi.
So what am I getting, then?
I'm getting 2/(pi*k).
With that minus sign, I'll evaluate it at x=0, I have one minus whatever I get at the top.
cos(k*pi).
That's b_k.
So there's a typical, well not typical but very nice, answer.
Now let's see what these numbers are.
So let me take a 2/pi out here.
And then just list these numbers.
So k is one, two, three, four, five, right?
Tell me what these numbers are for, let me put the k in here because that's part of it.
So it's a constant, 2/pi.
At k=1, what do I get?
At k=1?
This is the little bit that needs the patience.
At k=1, the cos(pi) is?
Negative one.
So I have net minus minus one, I get a two.
I get a two over a one.
k is one.
Alright, that is the coefficient for k=1.
Now, what's b_2, the coefficient for k=2?
I have 1-cos(2pi), what's cos(2pi)?
One.
So they cancel, so I get a zero.
There is no b_2.
What about b_3?
So now b_3, I have 1-cos(3pi).
What's the cos(3pi)?
It's negative one again.
Right, same as the cos(pi).
So that gives me a two, and now I'm dividing by three.
2/3, alright, let's do two more.
k=4, what do I get?
Zero, because the cos(4pi) has come back to one.
So I get a zero.
And what do I get from k=5?
1-cos(5pi), which is?
cos(5pi) is back to negative one, so 1--1 is a two.
You see the pattern.
And so let me just copy the famous series for this, S(x).
This S(x) is, let's see.
The twos, I'll make that 4/pi, right?
I'll take out all those twos.
So I have 4/pi*sin(x).
I have no sin(2x), forget that.
Now I do have some sin(3x)'s, how much do I have?
4/pi*sin(3x)'s, But divide by three, right?
And then there's no 4x's, no sin(4x)'s.
But then there will be a 4/pi*sine, what's the next term now?
Are you with me?
So this is a typical nice example, an important example.
Sine of what?
5x divided by five.
OK. That's a great example, it's worth remembering.
Factor the 4/pi out if you want to.
4/pi time sin(x), sin(3x)/3, sin(5x)/5, it's a beautiful example of an odd function.
OK, and let's see.
So what do you think, MATLAB can draw this graph far better than we can.
But let me draw enough so you see what's really interesting here.
Interesting and famous.
So the leading term is 4 over pi sine x, that would be something like that.
That's as close as sin(x) can get, 4/pi is the optimal number.
The optimal coefficient.
The projection, this 4/pi*sin(x) is the best, the closest I can get to one.
On that integral.
With just sin(x).
But now when I put in sin(3x), I think it'll do something more like this.
Do you see what's happening there?
That's what I've got with sin(3x), and of course odd on the other side.
What do you think it looks like with sin(5x)?
It's just so great you have to let the computer draw it a couple of times.
You see, it goes up here.
And then it's sort of, you know, it's getting closer.
It's going to stay closer to that.
But I don't know if you can see from my picture, I'm actually proud of that picture.
It's not as bad as usual.
And it makes the crucial point, two crucial points.
One is, I am going to get closer and closer to one.
These oscillations, these ripples, will be smaller.
But here is the great fact and it's a big headache in calculation.
At the jump, the first ripple doesn't get smaller.
The first ripple gets thinner, the first ripple gets thinner.
You see the ripples moving over there, but their height doesn't change.
Do you know whose name is associated with that, in that phenomenon?
Gibbs.
Gibbs noticed that the ripple height as you add more and more terms, you're closer and closer to the function over more and more of the interval.
So the ripples get squeezed to the left.
The area under the ripples goes to zero, certainly.
But the height of the ripples doesn't.
And it doesn't stay constant, but nearly constant.
It approaches a famous number.
And of course we'll have the same odd picture down here.
And it'll bump up again, the same thing is happening at every jump.
In other words, if you're computing shock.
If you're computing air flow around shocks, with Fourier type method, Gibbs is going to get you.
You'll have to deal with Gibbs.
Because the shock has that extra ripple.
OK, that's a lot of Section 4.1.
Energy, we didn't get to, so that'll be the first point on Friday.
And I'll see you this afternoon and talk about the MATLAB or anything else.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so I've got Quiz 2 give back to with good scores.
So it's Christmas early.
Or, good work on that exam.
That's fine and so we all know Fourier series is like a central topic on the final third of the course, and I'll just keep going on Fourier series.
I'll give you homework on 4.1 and 4.2.
So 4.1 is what we complete today, the Fourier series.
4.2 is the discrete Fourier series.
So those are two major, major topics for this part of the course.
This is the periodic one and 4.2 will be the finite one, with the Fourier matrix showing up.
OK, so can I pick out, I've made a list of topics last time that were important for 4.1, for Fourier series.
And I think these are the remaining entries on the list.
I did the the Fourier series for the odd square wave, the minus one stepping up to plus one.
And you remember that, well just let me put that example down here.
That was the step function; easier if I draw it than if I try to write equations, so it was minus one up to plus one, ending of course period 2pi.
And I guess we called that S(x), for S signaling step function, S signaling square wave.
And S the general a signal that the function is odd.
And that means that it goes with sine functions.
And I think the numbers that we found from the formula was sin(x)/1, sin(3x)/3, sin(5x)/5, it's just a great one to remember.
And that's the example that has the important, and I don't know if I wrote down Gibbs' name.
He was the great physicist at Yale, well, a hundred years ago or more, and did this key idea that appears all the time, that any time you have a step function then the Fourier series it does its best, I if I take a thousand term, it'll do its best but it will overshoot by an amount that Gibbs found, and then it will get really close and then it will overshoot again and then, symmetrically.
Or anti-symmetrically, I should say.
So that's the Gibbs phenomenon of great importance.
I write this one out because first it's an important one to remember.
Second it'll give us a good example for this important equality, that the energy in the function is the energy in the coefficient.
That'll be good.
OK, actually maybe I should do that one first because the delta function has got infinite energy and we don't learn anything from this equation.
So let me jump to the energy in the function and the energy in the coefficient.
So what do I mean by energy?
Well, it's quadratic.
Right?
It's the length squared here.
It's the length squared of the function.
So let me compute, maybe I'll do it on this board underneath and leave space for the delta function.
The energy in x space is just, the integral is the length squared.
The integral of S(x) squared.
dx.
It's just what you would expect.
We have a function, not a vector, so we can't sum coefficients squared.
Instead we integrate all the values squared.
And of course, this is a number that we can quickly compute for that function.
So what does it turn out to be?
Well, what is S(x) squared for that function?
One, obviously.
The function is one here.
I'm looking at the original S(x), not the series.
The function is one there and minus one there.
When I squared those S(x) squared is one everywhere so I'm integrating one everywhere from minus pi to pi.
So I get the answer 2pi.
So that's a case where the energy in the physical space, the x space, was totally easy to compute.
Now, what about energy, what is this equality?
This really neat easy to remember equality?
I'm just going to find it by taking this thing squared.
What's the integral of the right-hand side?
The two are equal, so suppose I just fire away?
I integrate the square of that infinite series.
You're going to say, well that's going to take a while.
But what's going to be good.
The key point, the first point in last time's lecture, the first point in every discussion of Fourier series, is orthogonality.
Sines times other sines integrated are zero.
So a whole lot of terms will go.
So I take that thing, I square it.
So let me let me do that one here.
The interval from minus pi to pi.
May I take out the (4/pi) squared?
Just so it's not confusing.
Now, this is the sin(x)/1, sin(3x)/3, sin(kx)/k, and so on.
All squared.
dx, and so what do I get?
The (4/pi) squared.
And now I've got a whole lot of terms.
But the thing is, I can do this.
Because when I square this, I'll have a lot of terms like sin(x), sin(3x), and when I integrate those I get zero.
So the only ones that I don't get zero are when sin(x) integrates against itself.
And sin(3x) against itself.
So when sin(x) integrates against itself, that's sine squared.
Its integral is, you remember the integral of sine squared, which is, its average value is 1/2.
We're over an integral.
I think I'm going to get pi, for sine squared.
Because sine squared, we could do that calculation separately.
It's just a standard integral.
The integral of sine squared is pi.
Actually, yeah it just uses the fact that sine squared x is the same as whatever it is the same as.
Is it 1-cos(2x) or something?
Over two.
Or plus, who cares?
Because the integral of whichever, plus or minus, let me, well I suppose for history's sake we should get it right.
Which is it?
Is it a minus, so it looks OK now?
OK, alright, if it's wrong I didn't say.
OK, but I'm going to integrate.
So the integral of the cosine is zero, and the integral of the 1/2 is the part I'm talking about.
That 1/2 is there all the way from minus pi to pi.
So I get a pi.
From all these sines.
And now, what are all the terms?
Well, one over one squared, that just had a coefficient one, but what's the next guy?
You remember I'm squaring it, I'm integrating.
But I have a 1/3 squared.
And 1/5 squared.
And so on.
And here's a great point.
These two are equal.
I've got the same function, expressed in x space, and here it's expressed in sine space, you could say.
In harmonic space.
OK, so that's equal.
And that's going to be the fact in general.
In general that the integral of S(x) squared, so the general fact will be the integral of - well, I'll write it down below.
But let's just see what we got for numbers here.
So I had pi on both sides.
And so if I lift that over there, I get something like - what do I have?
Pi squared over 16, maybe I have pi squared over eight.
You just get a remarkable formula.
Putting that up there would be pi squared over 16, and the two makes it an eight.
And here I have the sum of 1/1 squared plus 1/3 squared.
Plus 1/5 squared.
So, that's an infinite sum that I would not have known how to do except it appears here.
The sum one over all those squares.
If I picked another example, I could get the sum - oh, this was all the odd numbers squared.
If I picked a different function, I could have got one that also had the sin(2x)/2 and the sin(4x)/4.
So this would have been the sum of all the squares.
Do you happen to know what that comes out to be?
I mean, here's a way to compute pi.
We have a formula for pi.
And we'd have another formula that involved all the sums.
Maybe I have room for it up here.
This would be the sum of 1/n squared, right?
This here I have only the odd ones.
And I get pi squared over eight.
Do you happen to know what I get for all of them?
So I'm also including 1/2 squared, there's a quarter also in here.
And also a 16.
And also a 36.
In this one, and the answer happens to be pi squared over six.
Pi squared over six.
The important point about this energy equality is not being able to get a few very remarkable formulas for pi.
There's another remarkable formula in the homework.
This is a little famous.
Do you know what this is?
This is the famous Riemann zeta function.
The sum of (1/n)^x is the zeta function at x.
Here's the zeta function at two.
So if I could draws a zeta.
Maybe?
There's a Greek guy in this class who could do it properly, but anyway.
I'll chicken out.
Zeta of, at the value two, zeta(2).
So we know zeta(2), we know zeta(4).
I don't think we know zeta(3), I think it's not a special number like pi squared over six.
So the zeta function, the sum of 1/n to this thing is, actually that's the subject of a problem that Riemann did not solve.
There's a problem Riemann did not solve, and nobody has succeeded to find it.
To solve it since.
There's a million-dollar prize for its solution.
My neighbors think I should be working on this, but I know better.
It's going to be solved one day, but it's pretty difficult.
And that is so nowhere this zeta function, where it's zero.
Of course, it isn't zero at two.
Because it's pi squared six.
And actually the conjecture is that it's zero, all the zeroes are at points, complex numbers with real part 1/2.
So they're on this famous line, the imaginary line with real part 1/2.
And that's the most important problem in pure mathematics.
So here we go.
We got a formula for pi out of this energy identity.
And I'll write it again, once I have the complex form.
OK, but you see where it comes from.
It just comes from orthogonality.
The fact that we could integrate that square is what made it all work.
OK, let's do the delta function.
So that's an even function, the delta, right?
Now I'm looking at the delta function.
Minus pi to pi.
It has the spike at zero and it's certainly even so we expect cosines.
And what are the coefficients?
So it's just an important one to know.
Very important example.
So what's a_0?
In general, the coefficient a_0 in the Fourier series is the, if I have a function, delta(x), S(x), whatever my function, the a_0 coefficient is the average.
a for average, a_0 is the average value.
So this is 1/2pi, the integral of minus pi to pi of my function.
Where did that come from?
I just integrated.
I just multiplied both sides by one.
Or by 1/2pi, and integrated.
And those terms disappeared, and I was left with a_0, and what's the answer?
Everybody knows that integral.
The integral of the delta function is one, so I just get 1/2pi.
So 1/2pi.
OK, now ready for a_1.
How much of cos(x) do I have?
Can I just change this formula to give me a_1 and you can tell me what it gives?
Well let me do it here.
Here's a_1.
What's the formula for a_1?
It's just like b_1, like the sine formulas.
You have to remember you're only dividing by pi.
Because that average value was 1/2, as we saw.
And then you have the integral of whatever your function is.
delta(x) in this case.
Times the cos(1x).
If we're looking for a_1 we've multiplied both sides by cos(x) and integrated.
And what answer do we get?
For a_1?
What's the integral of delta(x) times cos(x)?
dx, so I should put in a dx.
And the answer is?
One, also one.
The delta function, this spike, picks out the value of this function at the spike.
Because of course it doesn't matter what that function is away from the spike, because the other factor, delta, is zero.
Everything is at that spike, all the action, and this happens to be one at the spike.
So I get a 1/pi.
And actually, the same formula for all these guys.
This will be the same with cos(x) changed to cost(2x).
And what will be the answer now?
Again, one.
Right?
It's the value of this integral.
The formula for a_2 would have come by multiplying both sides by cos(2x), integrating.
And the integral of cos squared would give me the pi, and I'm dividing by the pi, and I just need that integral and it's easy.
It's also 1/pi.
So all these are 1/pi.
Let me just put in the formula.
1/2pi, and all the rest are 1/pi.
cos(3x).
All the cosines are there.
All in the same amount.
And the constant term is slightly different.
OK, that's the formal Fourier series for the delta function.
Formal meaning you can use it to compute, of course some things will fail, like what's the energy?
If I integrate, I tried to do energy in x space of the delta function, or energy in k space, what answer would I get?
Right, the integral of delta squared, its energy is infinite, right?
The integral of delta, if I have delta times delta then I'm really in trouble, right?
Because this delta says, if I can speak informally, that delta says take the value of this function at zero, but of course that's infinite, so that would be infinite.
That would be the energy in x space.
What about the energy in k space?
Well, let's think, what's the energy in k space?
I'm going to do the squares of the coefficients, you remember that's what I had down here?
And I'll have it again in a moment.
It's the sum of the squares of the coefficients, fixed up by factors of pi.
And here, all the coefficients are constants.
So that sum is infinite.
Again, it's the sum of squares, of constant, constant, constant, constants, and that series doesn't converge, it flows up.
So the energy is infinite.
That's OK.
The key is that formula.
OK, there's the formula to remember.
There's the formula.
On forever.
Every frequency is in here to the same amount.
OK, good.
That's the delta function example.
OK, ready to go to complex?
Complex is no big deal because we know, actually, you can tell me the complex series for the delta function.
I'll write it right underneath.
The complex series for the delta function, just turn these guys into e^(i*theta), e^(i*x)'s, and e^(i*2x)'s, and e^(-i*2x)'s.
Just term by term, just to see it.
To see it clearly for this great example.
So what does cos(2x) look like in terms of complex exponentials?
Everybody knows cos(x) is the same as e^(ix), and e^(-ix), right?
Divided by two, because this is cos(x)+i*sin(x), this is cos(x)-i*sin(x), when I add them I get two cos(x)'s.
So I must divide by two.
Let me divide by two all the way.
So I'm dividing this 1/pi by two.
You'll see it's so much nicer.
So 1/2pi times the one, that's our first guy.
And now I've got the next guy.
Right?
Because I need to divide by the two to get the cosine, and there's my two.
OK, what's the next?
What do I have next?
From this guy.
1/pi, still there.
What's cos(2x), if I want to write it in terms of complex exponentials?
So this guy now, I'm ready for him.
Is e^(i*2x).
And e^(-i*2x).
Divided by two, and there's my two.
Do you see what's happening?
There's an example to show you why the complex case is so nice.
Here we had to remember a different number for a_0.
Here it's just, so the next one will be e^(i*3x), and e^(-i*3x), so that the delta function in the complex Fourier series, all the terms have coefficients one divided by the 2pi.
That's a great example.
And of course we see again that the sum of squares, oh yeah.
So let's do the complex formula, by which I mean I'm taking any function, F, not necessarily even, not necessarily odd, not necessarily real.
Any function can now be a complex function, because we're going to use complex things here.
So I'll have all the complex exponentials.
For integer k. This k is an integer but it can go from minus infinity to infinity.
That's the complex form.
F(x) is is a series again.
The beauty is that every term looks the same.
The thing you have to remember is that k negative k is allowed, as well as positive k. You see that we needed the negative k to get cosines.
k was minus 1 there, k was minus 2 there.
And for sines we would also need them.
So cosines go into it, sines go into it.
F(x) could be complex.
That's the complex series.
So maybe we could have started with that series.
But we didn't, we came to it here.
But what's the formula for its coefficient?
OK, actually, so the next half-hour now, we have to think complex.
and that will bring a few changes.
So watch for the changes.
You see we're almost in the same ballpark, but there are a couple of things to notice.
So let me write down that series again.
Minus infinity to infinity of some coefficient e^(ikx).
Now, what is c_k?
What is the formula for c_k?
As soon as we answer that question, you'll see the new aspect for complex.
How do I find coefficients?
I multiply by something.
I integrate, and I use orthogonality.
Same idea, just repeat after repeat.
The question is, what do I multiply by?
If I wanted to know c_3, suppose I want to know a formula for c_3, the coefficient.
What am I going to multiply by that's going to give me the orthogonality I need, that all the other integrals are going to disappear, that's the key?
I want to multiply by something so that when I integrate, all the other integrals are going to disappear.
And let's just do it.
Here, suppose I have e^(i5x).
And I'm looking for c_3.
So I'll look at e^(i3x).
So watch.
This is the small point we have to make.
So I'm looking for e^(i3x).
So I'm going to multiply by something, and I'm going to integrate.
And what would be the good thing to multiply by?
Well, you would say, if you were just a real person, you would say multiply by e^(i3x), integrated.
And hope for getting zero.
You won't get zero.
If I multiply e^(i3x), I'm sorry, you might say what am I doing here?
I'm trying to check orthogonality.
Let me instead of three use kx.
So that's a typical complex one.
It's any one of these guys.
And I want to see what's orthogonality.
That's what I'm asking.
Everything hinged on orthogonality.
We've got to have orthogonality here.
But let me show you what it is.
The thing you multiply by to get the c_3 is not e^(i3x), it is?
e^(-i3x).
You take the conjugate.
You change i to minus i.
In a complex case.
So if I take e^(ikx) times e^(-ilx), and notice that minus.
I claim that I get zero.
Except if k is l. And when k is l, I probably get 2pi.
If k is l. I get zero if k is not l. That's the beautiful orthogonality.
I'm not too worried about the 2pi, I'll figure that out.
So what I'm saying is, when you're taking inner products, dot products, and you've got complex stuff, one of the factors takes a complex conjugate.
Change i to minus i.
Do you see that we've got a completely easy integral now?
What is the integral of this guy?
How do I see that I really get zero?
So this is the first time I'm actually doing an integration and seeing orthogonality.
Anybody likes integrating these things, because they're so simple to integrate.
Before I plug in the limits, what's the integral of this guy?
How do I rewrite that to integrate it easily?
I put the two exponents together.
This is the same as e^i(k-l)x, right?
The exponentials follow that rule.
If I multiply exponentials, I combine the exponents.
And now I'm ready to integrate.
And so what is the integral of e^(i(k-l)x?
When I integrate e to the something x, I get that same thing again divided by the something.
So now I've integrated, and now I just want to go from zero to 2pi, plug in the limits from x=0 to x=2pi.
This is what the integral is asking me for.
So I'm actually doing the integral here.
I put these together into that, I integrated, which just brought this term down below, because the derivative will bring that term above.
Oh, they weren't meant to change.
Actually, it's wrong but right.
To change those integration limits.
I mean, any 2pi would work, but thank you.
You're totally right, I should have done minus pi to pi.
Why do we get zero?
That's the whole point.
Here we actually did the integral, and we can just plug in x=pi and x=-pi.
Or we could plug in zero and 2pi, or I could plug in any guys that were 2pi apart, any period of 2pi.
Why do we get zero?
Do you have to do the plugging in part to see it?
You can, certainly.
But the point is, this function is periodic.
That's a function that has period 2pi.
So it has to be the same at the lower and the upper limit.
That's what it's coming to.
That's a periodic function.
It's equal at these two limits.
And therefore, when I do the subtraction I take it at the top limit, minus the answer at the bottom limit, it's the same at both.
So I get zero.
So there is the actual check orthogonality.
So the key point was, orthogonality, or inner product, or complex functions, one of them has to take the complex conjugate.
Let me just do for vectors, too.
Complex vectors.
I may have mentioned it, let me take the vector as an extreme example.
And suppose I wanted to find the inner product with itself.
Which will be the length squared.
What's the inner product of that vector, that complex vector  with itself?
Let me just raise it up so we see it.
Focus on this.
Usually the length squared would be one squared plus i squared.
No good.
Why no good?
Because it's zero.
One squared plus i squared is zero.
We want absolute values squared.
Absolute values, we need squares.
We need positive numbers here.
So the length of this squared, I would, so let me call that vector v. I want to take not v transpose v, that's the thing that would give me zero.
If I did , I have to take the complex conjugate.
One of the two factors, and it's just a matter of convention which one you do, this is the thing that gives me the length of v squared.
And it's the same thing for functions.
It's the integral of F(x).
Do you want me to write it as, it's the integral of F(x) times its conjugate.
That gives me the length of F squared.
And of course I can rewrite that as the integral of F. We have this handy notation for F times its conjugate.
It's like re^(i*theta) times re^(-i*theta), the product of re^(i*theta), times re^(-i*theta) is what?
This is the complex number, this is its conjugate, when I multiply, you notice that i went to minus i when I drop below the real axis, I just change i to minus i.
So those cancel and I get r squared.
The length, the size of the number.
It's just, if you haven't thought complex just make this change and you're ready to go.
Yeah just v bar transpose v, F bar transpose F, if you like, or F squared.
And when we make the correct change we still have the orthogonality.
OK?
So that's what orthogonality is, now what's the coefficient?
So now, after all that speech, tell me what do I multiply both sides by?
Let me start again and remember here.
So my f of x is going to be sum of c k, e to the i k x.
And if I want to get a coefficient, I'm looking for a formula for c 3.
So how do I find c 3?
Let me slow down a second.
How to find c 3?
Multiply both sides by what?
And integrate.
Minus pi to pi.
What goes there?
If I want c 3, I should multiply both sides by, should I multiply both sides by e to the i 3 x?
Nope.
e to the minus i 3 x.
Multiply both sides by e to the minus i 3 x, and integrate.
e to the minus i 3 x, d x.
And then on the right side I get, what?
I get c 3, because that'll be the minus i 3 x times e to the plus i 3 x, I'll be integrating 1 so I get a 2 pi.
And all 0's.
From all the other terms.
Orthogonality doing its job again.
All these 0's are by orthogonality.
By exactly this integration that we did.
If k is 3 and l is 7, or k is 7 and l is 3, the integral gives 0.
So what's the formula then?
This is all gone, divide by 2 pi and you've got it.
There you are.
That's the coefficient.
OK. We have to move to complex numbers, and now in those 20 minutes the only change to make is conjugate one of them.
One of the things when it can be complex.
So that's a complex series.
What's the energy inequality now?
Let me do the energy equality in x space and in k space.
If I take this, now I'd like, so that's an equality of two functions.
Now I'd like to get it, I want to do energy.
This is fantastic.
And extremely practical and useful.
The fact that the energy is going to come out beautifully, too.
So what am I going to do for energy?
I'm going to integrate F(x) squared.
dx.
That's the energy.
That's the energy in x space, in the function.
What do I do now?
I integrate this series.
c_k*e^(ikx) squared.
Oh no, whoa.
If I put a square there, right?
I'm fired.
Right?
What do I have to do?
I will put a square there, but I have to straighten out something.
What do I do?
Those curvy lines, which just meant take the thing and square, should be changed to?
Straight.
Just a matter of getting straight.
Straightening this out.
OK, straight.
There we are.
OK. Now, that means the thing times its conjugate.
That's the thing times its conjugate.
So what do I get?
All the terms disappear except the perfect, the ones that are squared.
So what do I get here?
And the ones that are squared I'm integrating, one, so I get a 2pi.
So there's a 2pi.
This whole subject's full of these 2pi's and 2pi's, it's just part of the deal.
Now, what's left?
And now I'm looking only at the terms where I'm integrating something by its conjugate, by its own conjugate.
And then I'm getting c_k times its own conjugate, so I'm getting all the terms.
I'm getting no cross terms, just the terms that come from that times it's own conjugate.
Which is c_k squared.
That's the energy in the coefficients.
That's the energy in k space, and of course that sum goes minus infinity to infinity.
I've got them all.
That's the energy in k space, here's the energy in x space.
You can expect that this orthogonality is going to give you something nice for the energy.
For the integral of the square.
Alright, so you saw that.
And let's see.
So we saw it for, we actually got a good number for the square wave.
We got infinity for the delta function.
I've put on here as a last topic, to just give the word function.
Part of what this course is doing is to speak the language, teach the language of applied math.
So that when you see something, you see this, you recognize hey I've seen that word before.
So I want to know, what's the right space of functions?
This is my measure for the length squared of a function.
My square wave is great.
It's length squared was whatever it was, pi squared over something.
And that's its length squared.
The delta function gave infinity.
So it's not going to be allowed into the function space.
It's a vector of infinite length.
Like the vector <1, 1, 1, 1, 1> forever.
Too long.
Pythagoras fails, because we sum the squares, we get infinite.
So what functions, what vectors should we allow in our space?
So we're going to have a space that's going to be infinite dimensional because our coefficients, we've got infinitely many coefficients.
Our functions have infinitely many values.
So we've moved up from n dimensional space to infinite dimensional space.
And everybody calls it after the guy who, Hilbert space.
So I don't know if you've seen that word before, that name before, Hilbert space.
It's the space of functions with finite energy.
Finite length.
So this function is in it, the delta function is not in it.
And the point is that this space of functions, we've got these guys are a great basis for the space of functions.
The sines and cosines are another basis for this space of functions.
We just have a whole lot of functions, and all the facts of n-dimensional space.
So what are important facts about n-dimensional space?
One that comes to mind that involves length is the, a key fact about length is, length and angle, I could say actually, many people would say this is the most important inequality in mathematics.
That the dot product of two vectors, it's called the Schwarz inequality.
Several people found it, independently Schwarz is the single name most often used.
What do you know about the dot product of two vectors?
Somehow it tells you the angle between them, right?
Somehow the dot product of two vectors is, if I divide by the length of the vectors, so the dot product of vectors divided by the length, do you know what this is, in geometry?
It's a cosine.
It's the cosine of the angle between them.
And cosines are never larger than one.
So this quantity here is never larger than one; in other words, this is never larger than the length of one vector times the length of the other vector.
I could do an example.
Let v be .
And let w be .
I don't know how this is going to work.
What's the dot product of those two vectors?
Oh, it's 19.
Sorry about that.
Let's change this, I'd like a nice number here.
What do you suggest?
Make it five somewhere?
Five wouldn't be bad.
It wouldn't be too good either, but.
Ah, OK. What's the dot product of those?
16.
And now what am I claiming, that that's length less than the length of this vector, which is what?
What's the length of ?
Square root of ten.
It's good to do these small ones, just to remember.
The length of that vector is the sum of the square root of the sum of the squares.
Square root of ten.
And the length of this guy?
Is the square root of 26.
And so I hope, and Schwarz hopes, that 16 is less than that square root.
Can we check it?
Let's square both sides, that would make it easier.
So the right-hand side when I square both sides will be?
260.
When I square both sides, and what's the square of 16?
256.
That was close.
But, it worked.
I'll admit to you, oops, not equal.
Ah.
Lesser equal, Schwarz would say.
And it's actually less than because these vectors are not in the same direction.
If they were exactly in the same direction, or opposite directions, the cosine would be one and we would have equal.
But since the angle, you see the angle between those two vectors is a pretty small angle.
The cosine is quite near one.
But it's not exactly one.
So I'm glad somebody knew 16 squared.
Does anybody know 99 squared?
The reason I ask that is, or 999.
I'll make it sound harder.
The hope, when I was about 11 or something, I was I was always hoping somebody would ask me 999 squared.
Because I was all ready with the answer.
Nobody ever asked.
Anyway.
But you've asked, I think.
So 998,001.
And now I I've finally got a chance to show that I know it.
OK, have a great weekend and see you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so I thought I'd quickly list the four central topics for the Fourier part of the course, and I would guess that there'll be a question on each of those four in the final quiz.
Final exam.
And we covered 4.1, Fourier series.
I don't plan to do discussion in 4.2 of other series like Bessel and Legendre and so on.
If I can, I'll come back to those in the very last lectures.
But I want to pick up now on the second key example, which is the discrete Fourier series that has only N terms.
Instead this series, as you're looking at it, has infinitely many terms.
But I'm going to cut back to n terms.
So this K, well I can go one to N, but the best way is zero to N-1.
So that's the discrete series.
We're going to be dealing with a vector, c_0 up to c_(n-1).
And a vector of function values.
Vector to vector.
So a matrix is going to be the key to everything here.
An n by n matrix.
So that's what's coming.
Then, after that probably by Friday will be the integral transform.
And early next week, and a little after Thanksgiving would be this key idea of convolution, which is really important in so many applications and needs to be separated out.
Convolution will apply to all of these.
And you'll see the point there.
OK, ready for the idea of the discrete one.
OK, so the key idea is we only have N terms, so I can't expect to reproduce a whole function.
I can get n function value, so I'll get this at n different points.
And if I'm thinking of my F(x), on, let me take zero to 2pi now, and I'm thinking again of an F(x) that's periodic.
So it goes, does whatever.
If it was like that, if that was my F(x), just remember about Fourier series, what could you tell me about the Fourier series for that F(x)?
Well, when I continue, periodically am I going to see a jump?
Yes.
There'll be a jump at every 2pi, so the underlying function is not continuous, has a jump, so that our special examples with jumps and slow decay, 1/k decay rate for the Fourier coefficients would apply here.
OK, that's remembering Fourier series.
Now, Fourier integrals, I'm just going to take, let me just take N=4.
So I'll take this point, you know what four points am I going to take?
I'm not going to repeat, ah.
Just for the heck of it, let's make it look periodic.
Yeah, let's make it look periodic.
I never should have done it otherwise.
OK, so there's one point.
I'm just going to take four equally spaced, equally spaced being key, zero, one, two, three.
So N-1 is three.
And these four values I can get right.
So those are my four x's.
The four x's that I'm going to take.
So this is x=0, of course.
And this one is, what's the x there?
x is 2pi, the whole deal.
The whole interval, but it's divided into N pieces, so 2pi/N.
So we are going to see a lot of fractions involving pi/N.
Or 2pi/N.
They're just going to come up everywhere.
Because that's the delta x, the delta h, the step in the discrete form.
OK.
So those are the x's.
So I'm only going to get this at x=0, 2pi/N, 4pi/N, up to whatever it comes out to, (N-1)*2pi/N.
That doesn't look elegant so we probably will try to avoid writing that again.
But this was the three times 2pi/4 that got us to there.
So this is 2pi/8, pi/4.
Sorry, 2pi/4, 4pi/4, 6pi - oh, ok.
So we're only going to get equality at those x's.
So it's those x's that I should plug in here.
OK, so let me do that.
Let me plus in, what does this series look like at these four points?
Let me see.
So writing out all four equations, F(0) is going to match the right side.
At zero, because zero is one of my points.
So I get c_0+c_1+c_2+c_3.
At x=0, right, all the e to the whatevers were one.
Now at the next x, 2pi/N, now I'm matching the series with a function at that value, there.
2pi/N.
So now I have k=0, so what does k=0 give me?
When k is zero, what's my term look like?
It's just c_0 times one.
Because if k is zero, e to the whatever is again one.
So this is just c_0.
Now, here's the key.
What is that business there at x=2pi/N?
Here comes the important number in this whole business.
The important number this whole business is, I'll call it w, is e^(2pi*i)/N.
That's a complex number, and we'll place it in the complex plane.
So you must get to know that number.
w. Because when I plug in x equal - k is now one here, so I have c_1.
Times this e to the i one, 2pi/N, what is that?
w. So it's w*c_1.
And what do I get from the next term in this series?
I have c_2*e^(i2x).
x is 2pi/N, do you see what's happening?
w squared is e^(4pi*i/N).
We're using the beauty of exponentials, that's making everything go.
It's w squared.
And this guy would be w cubed, c_3.
That's the series.
At our points.
So this, like, we'll just see this once.
What were we doing?
But this is what we're going to live with here.
So I'll rewrite it all in terms of w, so you just, you've got the idea of what are we doing?
We're matching at N points and now we get the good notation w. OK, and at the next point, 4pi/N.
Well, again, k=0 term, it's c_0.
The k=1 term, can you tell me what I get when k is one, I have something times c_1, it's e^(i*4pi/N), what's that?
w squared.
e^(i*4pi/N) is w squared.
Look, this matrix is coming out symmetric.
And this will be w^4, and that'll turn out to be w^6, and then the last row of the matrix is going to come from matching at 6pi/N, and it will turn out to be c_0, and it'll be w^3, w^6, and w^9.
So I did the last row or two a bit fast.
Just because the patterns there, and you'll spot it.
Now I want to pull out of the matrix.
The Fourier matrix, the four by four Fourier matrix that multiplies the c's and gives me the F's.
Let me call these numbers are y_0, y_1, y_2, and y_3, just to have a, so it's a vector of values of the function.
And this is a vector of coefficients.
So it's wise to - the Fourier transform goes between y's and c's, and y's.
Connects the vector, and this is N values, N function values in physical space.
These are N coefficients in frequency space, and one way is the discrete Fourier transform and the other way is the inverse discrete Fourier transform.
So, and it's a little bit confused, which is which, actually.
There was no question with Fourier series, it was easy to tell the function from the coefficients because the function was f x and the coefficients were a bunch of numbers.
Here we only have N numbers.
N y's and N c's and we're transforming back and forth.
So I'm finding right now the matrix, this Fourier matrix is going to take the c's and give me the y's.
So I'm talking here about the matrix F. And let's see what F is.
Can you just see what, that was y_0, to y_3, equals this matrix F, the four by four matrix.
Times c_0 to c_3.
This is really the inverse.
DFT.
It reconstructs the y's from the c's, or an easier way to say that is add up the series.
So that's the usual DFT is taking us, starting with the y's and giving us the c's.
So this would be the discrete Fourier transform.
Take a function, produce its coefficients.
The reverse step is take the coefficients, add back, add up the series, get the function.
And that's what F does.
So I'm going to focus on F. Which is the ones that involves this w guy.
OK, can you read off, peel off from here what the matrix F is, the Fourier matrix?
Its first row is all ones.
Its first column is all ones.
And then it's one, w, w^2, w^3, w^2, w^4, w^6, w^3, w^6, and w^9.
To the ninth power.
OK, so it's made up of these powers of w. Zero first, second, and third power and then higher powers.
But higher powers are - let's draw a picture.
Where is w?
So this is the complex plane.
Here's the real, and here's the imaginary direction.
And let me take N to be, here N was four.
Let me take N to be eight in my picture, just to have some, then I'll be able to spot the fours and the eights.
If N is eight, where would w^8 be?
So w^8 is meant to be the e^(2pi*i/8), now.
Where's that number in that picture?
Is it on the circle?
Absolutely.
This discrete Fourier transform never gets off the circle.
And in fact, it never gets off eight points on the circle.
Which are the eight equally spaced points.
This says go 1/8 of the way of the whole way around, which will be here.
So there's w^8.
When I square it, what's the square of w^8?
The angle.
What happens to the angle when I multiply numbers?
I add the exponents.
Here, if I'm multiplying by itself, it'll double the exponent.
Double the angle.
It'll be, there's w^8 squared, so that point is w^8 squared.
And it's the same as w^4 because it's 1/4 of the way around.
And what is that number in ordinary language?
i.
It's i, right?
It's on the unit circle at 90 degrees.
And on the imaginary axis it's just i.
So all this here was i, i^2 i^3.
i^2, i^4, i^6.
i^3, i^6, i^9.
The four by four Fourier matrix is just powers of i.
And the N by N, the eight by eight Fourier matrix is powers of w^8.
And let's find all the rest of the powers.
So there's w^8, w^8 squared, where's w^8 cubed?
Here.
I multiply those guys, I add the angle, I'm out to here.
Here's the fourth power, here's the fifth power, sixth power, seventh power and finally we get to w^8, to the eighth power, which is one.
Because if I take the eighth power here, I've got e^(2pi*8), and that's one.
So that's the picture to remember.
And while looking at that picture, could you tell me the sum of those eight numbers?
What do those eight members add up to?
So I just want to ask what do w^1 plus w up to w^7, when I'm doing the eighth guy, adds up to?
So the sum of those eight numbers, well I wouldn't ask the question if it didn't have a nice answer.
Zero.
That's of course the usual answer in math.
We do all this work and we get zero.
But now, of course the real math is why.
I mean, so math is, you get equations but then always there's that kicker, why.
And why do they add up to zero?
Here.
Can you see that?
Why those eight numbers would add to zero?
Well, yeah.
So what do these add up to?
Just those two guys?
One and minus one add to zero.
That and that add to zero, I can pair them off.
This and this add to zero.
That and that add to zero.
Yeah, so that's one way of seeing it.
But actually, even if N was three or five or some odd number, and I had - so should I do the picture for N=3?
Just a little, doesn't need a very big picture for N=3.
So where would the roots be for N=3?
One of them is always one.
These are the three cube roots of one.
That I'm going to draw.
Those were the eight eighth roots of one.
Because if I take any one of those numbers its eighth power brings me back to one.
So where are these guys, so this is three numbers equally spaced around the circle.
Where do you think they are?
Well, it's gotta be at 120 degrees, 240 degrees and 360.
And again, those will add to zero.
You can't pair them off because we got an odd number, but it's safely zero and we could see why.
OK, so this section, this topic is all about that matrix.
It's all about that four by four, or N by N matrix.
Which has powers of w. Notice that the matrix is, it hasn't got any zeroes.
Hasn't even got anything close to zero.
All those numbers are on the unit circle.
So multiplying by that matrix, which is the same as adding up the Fourier series at the four points, doing this calculation, finding these y's from the c's, reconstructing the y's, given the coefficients, has 16 terms.
Right?
The matrix has got 16 entries, all the entries in the matrix are showing up here in powers of w. We've got 16 multiplications.
Well, maybe a couple of them were easy, but basically 16 multiplications and additions.
Pretty much N squared work.
And that's not bad.
Except if you want to do it a million times, right?
Suppose we have a typical N might be 1024.
So the matrix has then got, is 1024 squared numbers, which is about a million.
So if we have a matrix with a million entries, a million operations to do, and then we do it a million times then it's getting up to money that Congress would give to the banks.
Whatever.
Anyway.
Serious money.
Serious computing time.
And the wonderful thing is that there is a faster way than n squared.
There's a faster way to do these 16 terms than the obvious way.
You might say, well, they're pretty nice.
And they are.
But they're nicer than you can know.
I mean, nicer than you can see immediately.
This matrix will break up into simple steps with lots of zeroes in each step, and the final result is that instead of n squared, capital N squared, which is done the old fashioned way, let's say.
That that there's a fast Fourier transform.
So the FFT, I should maybe find a better space for the most important algorithm in the last hundred years, and here it is.
Right there.
So my one goal for Wednesday will be to give you some insight into the FFT.
But here I'm just looking at the result.
The number of computations is, instead of N squared, it's N times log N. It's log to the base two, or maybe 1/2 N log N. So I'll erase some of the other things close by so you can see it.
So that, where n squared was, if N was a thousand, N squared was a million, If N is a thousand, this is a thousand, and what's the logarithm of a thousand, to the base two?
That's the ten, right?
The 2^10 gave us that 1,024.
So this is 10^4.
So N is 10^3, the logarithm of 1,024 more exactly, the logarithm is ten, so we're 10^4 instead of 10^6.
So that's a saving of a factor of 100 that's true.
I mean it just comes from doing the addition and multiplication in the right order.
You get this incredible saving.
And it's sort of makes whole calculations that were previously impossible are now possible because instead of where it might have been 100 minutes, it's now one minute.
So everything focuses on this Fourier matrix and its inverse, which we still have to find but it'll come out beautifully.
In fact, I could tell you what F inverse, just so you see it coming.
F inverse will, so what happened to w, that all-important number w?
Well, let me repeat what w is.
I'll put it on this board.
F has powers of w. w is e^(2pi*i/N).
And what power has it got?
In the j,k position?
So now you can see the formula for these entries.
Only, you have to let me start the count at j and k equals zero.
This is zero based, where MATLAB is one based.
And to do this stuff you always have to shift things by one.
Some other, more recent, software like Python on is zero based, because this is so common.
So here's the formula.
It's w to the power j times k. That's neat.
That's what goes into the matrix F. This here is row three.
It looks like row four but I'm starting with zero, so that's row three.
That's column three.
This is w^3 times three.
w^9, which by the way is the same as what?
If w is i in this four by four, it is.
w is i, what's w^9, what's i to the 9th power?
Same as?
Can you do i^9 in your head?
Sure.
Because you're starting here.
You're taking its ninth power, so one, two, three, four, you're back to one.
i^4 is one.
One, two, three, four more, i^8 is one.
i^9.
That will actually be just i. i^9.
So they're all powers of i.
But this is the good way to see it.
It's three times three, because it's in.
And this, of course, is three times two, and two times two, and three times one.
You see the entries of F?
Yeah.
So now I'd better write down, I promised to write this formula in a better form.
All those formulas in a better - nobody is going to write this out for matrices of order 1000, right?
So we've got to write the decent formula.
So the decent formula is that the jth y, because it's going to come in equation number j, will be the sum on k=0 to N-1, of c_k times w^jk.
Those are the entries, those are the c's that it multiplies and the y's.
This is just vector y's, this is y=Fc.
Now we've finally got a good notation.
So this was horrible notation.
I mean, that's exhausting.
This shows you the algebra, but you have to be sort of prepared - well, the information is here because this is telling you what the entries of F, of the matrix, are But this is with indices, and this is with whole vectors.
So that would be y, in MATLAB that would be the inverse fast Fourier transform of c. So you see that that's our matrix.
Now, I was going to say before I show why, I was going to say what about the inverse?
Because if the FFT gave us a fast way to multiply by F, we want it also to give us a fast way to do the inverse.
We have to go both ways here.
We get our data, we put it into frequency space, we look at it in frequency space, we understand what's going on.
By separating out the frequencies.
We maybe smooth it, we maybe convolve it, whatever we do.
Compress it.
And then we've got to go back to physical space.
So we have to go both ways.
And the question is what about F inverse?
And let me just say, what goes into F inverse?
The point is that the number that goes into F inverse is just like w. But it's the complex conjugate, and I'll call it w bar.
And that's e^(-2pi*i/N).
So the thing that's going to go into the inverse matrix is the powers of w bar.
The complex conjugate.
jk.
Let me just say right away, that just as there were 2pi's in the Fourier series world, that got in our way, and physicists can't stand those 2pi's.
They try to get rid of them but course they can't, they're there.
So they put it up into the exponent.
Well, I guess, I'm blaming physicists, I've done it too.
I've put the 2pi up there.
For the Fourier series.
Anyway whatever, 2pi's are all in your hair in the Fourier series.
And in this, for the discrete one, the corresponding thing is an N. There's a factor N that I have to deal with.
Because look, here's what I'm saying.
Let me multiply F by F inverse.
OK. Say, 1, 1, 1, 1, 1 i, i^2, i^3, this is my F, right?
Whoops.
Don't ever let me do that.
i^2, fourth, sixth.
i^3, i^6, i^9.
Now, my claim is that F inverse is going to involve the complex conjugate, w bar.
i bar.
What is the complex conjugate of i?
You see that we really have to use complex numbers here, but they're on the unit circle.
You couldn't ask for a better line.
What's the complex conjugate that's going to go into F inverse?
If I take i and I take its conjugate, what does that mean?
So the conjugate of that is just, flip it across the real axis to the other one.
You see, it's great?
The complex conjugate of i is minus i.
Here's w^8, here's w bar^8.
It's just across the axis.
So it still has size one.
And, of course, it's e to the?
This angle, which is just minus this angle.
So that's why we get this minus.
Because when I flip it across, i changes to minus i, angles change it to minus angle.
And finally I was going to do this multiplication just to see.
So now I'm claiming that this is 1, 1, 1 1, 1 1, 1, -i, -i^2, -i^3.
So on; -i^2, -i^3, fourth, sixth, sixth, and ninth.
And now, if you do that multiplication, you will get the identity matrix.
That's the great thing.
You get the identity.
1, 1, 1, 1.
Except for a factor N. And do you see that coming?
When I multiply this by this, what do I get?
Four.
If they were N by N, I'd have N ones against N ones, so I'd have to expect an N there.
So F times F inverse, so, in other words if I want to get the identity, I'd better divide this by N. So now I have F F inverse equal I.
So F inverse is just like F, except complex conjugate and divide by N. So if I want this formula to be really correct, I have to divide by N. OK?
So and that's what w and w bar is.
And another letter that often gets used in this topic is omega.
And I figure that's a good way to separate out the complex conjugates, let's call this guy w and call this guy omega.
So this is w and here's omega.
I don't know if that helps.
But you do have to keep straight whether you're talking about this number or that number.
And this number could be called w, because omega wouldn't be easy to type.
So you have to pay attention, which one you're doing and when, somewhere you have to divide by N. But otherwise it's fantastic.
It's just beautiful.
Can we do that multiplication, just to see that it really works?
Well, the ones worked fine.
What about the ones times, or what about the ones times that column.
So I'm looking at the (1,2) entry in this multiplication.
Or rather, the (0,1), entry I should say.
This is the zeroth row, and this is the row number one, and what do I get out of that?
Zero, right.
That was my point.
That this is adding up, oh well, going the other way around.
Because it's the complex conjugate, but it's adding up the four, same four numbers and giving me that zero.
And now how about the ones on the diagonal?
When I do this times this, so this is the same, the row times itself.
What am I seeing there?
What are these terms?
That's certainly one, what is i times minus i?
i times minus i is one again.
A number's getting multiplied by its conjugate.
That gives you something real and positive when you multiply a number by its conjugate.
In fact, we get one every time so we get four ones, we divide by four and we get that.
Maybe you actually see here, just so you remember, so F inverse is, I'm concluding that F inverse, I'm looking now, what is F inverse?
It's F conjugate divided by this nuisance number N. I could put square root of N with F, and then square root of N would show up with F inverse, just the way I could have done, square the 2pi with one, and square the 2pi with the other.
But let's leave it like so.
So that's not proved in a way.
Just, checked for a couple of cases with N=4, but it always works.
That's the beauty.
In other words, the two directions, the y to c and the c to y, are both going to be speeded up by the FFT.
The FFT, whatever it did, whatever it could do with the w and powers of w, it can do the same with w bar.
And powers of w bar.
If the FFT is a fast way to to multiply by that, it'll also give me a fast way to multiply, to do the other transform.
So the FFT is what makes every - this would all be important if the FFT didn't exist.
And of course actually, Fourier didn't notice it.
Gauss again had the idea, but he had so many ideas it wasn't very clear in his notes.
The FFT idea.
But then Cooley and Tukey, guys at Bell Labs in the 1950s discovered it.
In a small paper, I don't think they had any idea how important it would be.
But they published their paper and people saw what was happening, and realized that they should, that that gave the right way to do the transforms.
OK, I was going to say here, do you notice that this one, when I take a vector, that vectors and its complex conjugate, what am I getting?
I'm getting the length squared.
I'm getting the right inner product.
This is the right inner product when things are complex.
Something times its conjugate.
I'm just reminding you that the inner product of two vectors, uv, u dot v or u comma v, should now be the sum - it used to be the sum of u_i*v_i.
But now we're in the complex case, so we should take the conjugate one of them.
And here I guess that's the v, and this is the u, and so the point is that we're taking the right, yeah.
We have, oh yeah.
let me just say it this way.
These columns are orthogonal.
Those four columns, the columns of the Fourier matrix, are orthogonal.
In other words, we have an orthogonal basis.
And it's those four columns.
I almost missed saying that.
But you remember from last week, the whole point about sines and cosines and exponentials in the function case, was orthogonal.
That's what made everything work, and it makes everything work here.
Here we have vectors, not functions.
Orthogonal just means dot products are zero.
But they're complex, so that in the dot product, one factor or the other has to get conjugated.
Anyway, that's an orthogonal matrix.
Apart from this N, this capital N thing.
So if I normalize it, if I take out one over square root of N, now the length of every column is one.
So now I have orthonormal columns.
One squared, one squared, one squared, one squared is four, divided by four is one.
And every column has length is a vector in four-dimensional complex space.
Of length one.
And orthogonal.
So that means orthonormal.
It's a fantastic matrix.
That's the best basis we'll ever have for complex four-dimensional space.
That basis.
Well, actually when I say the best basis we'll ever have, I realize I wrote a book about wavelets.
And the idea of wavelets is, there's another basis.
A wavelet basis.
And I hope to tell you about that at the very end of the semester.
So wavelets would be another basis, would you like to know the wavelet basis?
Have we got?
Just in case the end of the semester doesn't come, can I write down here the wavelet basis?
OK. And you can decide you maybe like it better.
So the four by four wavelet basis, alright.
So this will be the wavelet matrix.
Well, this was too good to lose, right?
That's a terrific vector.
All ones, constant vector.
It's the thing that gives us the average, this c_0 is going to again be the average of the y's.
How do I get that average?
Somehow I'm doing, the c_0, the constant term is again I'm going to take one of everything and I'll divide by four and I'll have the average.
So I like that vector.
That's the DC vector, if I'm talking about direct current vector.
If I'm talking about networks.
In an image, it would be the whole, you could call it the blackboard vector.
The whole image of a empty blackboard would be constant.
OK, so that's too good to lose.
Now, the obvious thing orthogonal to that would be .
Right?
Now I've got two orthogonal columns.
And probably that's pretty close to something I have over here.
Now, here's the point about wavelets.
You'll see it with the next guy.
The next one, you saw that those two are orthogonal.
And real, and simple, nice numbers.
Now, the next one will be .
Do you see that that's orthogonal?
Obviously, because its dot product with that is zero and that is zero.
And you want to tell me the last column of the wavelet, of the simple wavelet matrix?
This would be the Haar, named after Haar, the Haar wavelets.
The whole point about wavelets was a new basis.
Somebody realized, well actually, Haar lived a long time ago, but they got improved and improved, but here's the simplest one.
What's the good fourth column?
Fourth orthogonal vector here?
What shall I start with?
Zeroes, and then I'll put one minus one.
So the beauty of wavelets is, I've got this averaging vector.
And then I've got this difference, sort of on a big scale, and then I've got differences on a little scale with zero, differences here with zeroes so as I keep going I could do eight by eight, 16 by 16, it would - I get the averaging guy.
And then I get differencing, I get large scale differences and then I get finer differences.
So coarse differences, finer differences, finer finer differences.
And that's a really good way to - if your function, if your signal has jumps.
We saw that Fourier wasn't perfect, right?
And a lot of signals have jumps.
Right?
Your shocks, whatever you're trying to represent.
This is a better basis to represent a jump, a discontinuity, than Fourier.
Because Fourier, at a discontinuity, does its best but it's not a local basis.
It can't, like, especially focus on that jump point.
And and the result is the coefficients decay very slowly and it's hard to compute with.
Whereas the wavelet is great for signals that have jumps.
If something happens suddenly at a certain time.
Anyway, that would be another, and of course I haven't made these unit vectors, I'd have to divide that by the square root of four and divide these guys by the square root of two just to normalize them.
But normalizing isn't important.
Orthogonal is what is important.
OK, so that's competition, you could say.
That's the competition.
OK, back to main guy, so this is the famous formula that gives the y's from the c's, and what's the other formula that gives the c's from the y's?
So now this is the transform in the other direction.
So it's going to be a sum from j=0 to N-1.
And what's going to go in there?
Have you got the idea of the two transforms?
In one direction this is e^i.
You see why we don't want to write it.
That is, that number is e^(2pi*i*j*k/N).
That's what that number is, in the Fourier matrix.
So you see using w makes it much cleaner.
OK, what goes there?
w bar.
The complex conjugate, e to the minus all that stuff.
w bar to the same power, jk.
And then I have to remember to divide by N somewhere.
I can put it here, or I can put it there.
But the all ones vector has length squared equal to N and I've got to divide by an N somewhere.
OK, those are the two formulas that many people would just remember.
For them that's the discrete Fourier transform and the inverse, and maybe they would put the N over there.
Probably that would be more common.
I got started with the Fourier matrix with this side, and then the DFT matrix comes here, and I needed the N. So those are the formulas.
I don't know how you'd prefer to remember these.
Somehow you have to commit something to memory here, certainly what w is.
And the fact that it's w and w bar?
And the fact that the sums start at zero.
And the fact that you have just these simple expressions for the entries in the matrix.
And this, then would be c=F inverse*y.
That's what I've written down there in detail.
OK, so that's the discrete Fourier transform, and I think we're out of time.
I've got more to say in a little bit about the FFT, how that works.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
OK, so.
Pretty full day again.
I had last time introduced the Fourier matrix, the discrete Fourier transform.
Well, more strictly, the discrete Fourier transform is usually this one.
It takes the function values and produces the coefficients.
And then I started with the coefficients added back, added up the series to get the function values.
So F or F inverse.
So we didn't do examples yet.
And one natural example is the discrete delta function that has a one in the zero position.
That's easy to do.
And then we should do also a shift, to see what's the effect, if you shift the function, what happens to the transform.
That's an important rule, important for Fourier series and Fourier integrals, too.
Because often you do that, and it's going to be a simple rule.
If you shift the function, the transform does something nice.
OK, and then I want to describe a little about the FFT and then start on the next section, convolutions.
So that's fun and that's a big deal.
OK, about reviews, I'll be here as usual today.
I think maybe the 26th, just hours before Thanksgiving we can give ourselves a holiday.
So not next Wednesday but then certainly the Wednesday of the following week would be a sort of major quiz review on in the review session.
And in class.
OK, ready to go?
On this example gives us a chance just to remember what the matrix looks like, because we're just going to, if I multiply this inverse matrix by that vector it's just going to pick off the first column and it'll be totally easy so let's just do it.
So if y is this one, I want to know about f. What are the Fourier coefficients of the delta function?
Discrete delta function?
OK, before I even do it, we got a pretty good idea what to expect.
Because we remember what happened to the ordinary delta function, in continuous time.
Or rather, I guess it was the periodic delta function.
Do you remember the coefficients, what were the coefficients for the periodic delta function?
You remember those?
We took the integral from minus pi to pi of our function, which was delta(x).
And then we had to remember to divide by 2pi, and you remember the coefficients are e^(-ikx).
This is c_k in the periodic case, the 2pi periodic case the function is the delta function, and you remember that if we want coefficient k we multiply by e^(-ikx).
That's the thing that will pick out the e^(+ikx) term and of course everybody knows what we get here, this delta, this spike at x=0, means we take the value of that function at zero, which is one, so we just get 1/2pi.
For all the Fourier coefficients of the delta function.
The point being that they're all the same.
That all frequencies are in the delta function to the same amount.
I mean that's kind of nice.
That we created the delta function for other reasons, but then here in Fourier space it's just clean as could be.
And we'll expect something here, too.
You remember what F inverse is?
F inverse 1/N.
Instead of 1/2pi, and then the entries of F inverse come from F bar, the conjugate.
So it's just one, one, one, minus - well, I've made it four by four here.
This is minus omega - no, it isn't minus omega.
It's it's omega bar, which is minus i, in this case.
Omega bar, so it's minus i, that's omega bar.
And the next one would be omega bar squared, and cubed, and so on.
All the way up to the ninth power.
But we're multiplying by one, zero, zero, so none of that matters.
What's the answer?
I'm doing this discrete Fourier transform, so I'm multiplying by the matrix with the complex conjugate guys.
But I'm multiplying by that simple thing so it's just going to pick out the 0th column.
In other words, constant.
All the Fourier discrete Fourier coefficients of the discrete delta are the same.
Just again.
And what are they?
So it picks out this column, but of course it divides by N, so the answer was 1/N.
.
It's just constant with the 1/N, where in the continuous case we had 1/2pi.
No problem.
OK. And, of course, everybody knows, suppose that I now start with these coefficients and add back to get the function.
What would I get?
Because just to be sure that we believe that F and F inverse really are what they are supposed to be.
If I start with these coefficients, add back to put those in here and reconstruct, so 1/N, <1, 1, 1, 1>, what will I get?
Well, what am I supposed to get?
The delta, right?
I'm supposed to get back to y.
If I started with that, did F inverse to get the coefficients, that was the discrete Fourier transform, now I add back to get, add the Fourier series up again to come back here, well I'll certainly get , and you see why?
If I multiply F, that zeroth row of F is , times will give me N. The N will cancel.
I get the one.
And all the other guys add to zeroes.
So, sure enough, it works.
We're really just seeing an example, an important example of the DFT.
And the homework, then, would have some other examples.
But I've forgotten whether the homework has this example.
But let's think about it now.
Suppose that's my function value instead.
OK, so now I'm starting with the delta.
Again it's a delta, but it's moved over.
And I could ask, I really should ask first, in the continuous case, suppose I, I can do it with just a little erasing here.
Let me do the continuous case for the delta that we met first in this course.
I'll shift it to a.
If I shift the delta function to a point a, well, I said we'd met the delta function first in this course.
At a, good.
But when we did it wasn't 2pi periodic.
So we still have, in fact, the Fourier integrals next week.
Will have a similar formula.
The integral will go from minus infinity to infinity and then we'll have the real delta, not periodic.
Here, we have, and people call it a train of deltas.
A train of spikes.
Sort of you have one every 2pi.
Anyway, that's what we've got.
Now, you can see the answer.
This is like in perfect practice in doing an integral with a delta.
What's the integral equal?
Well, the spike is at x=a.
So it picks this function at x=a, which is e^(-ika).
So not constant any more.
They depend on k. The 1/2pi's still there.
So it's, but still, the delta function shifted over.
I mean, it didn't change energy.
It didn't change, it just changed phase, so to speak.
And we see that, I would call this like a modulation.
So it's staying of absolute value one, still.
But it's not the number one, it's going around the circle.
Going around the unit circle.
So it's a phase factor, right.
And that's what I'm going to expect to see here in the discrete case too.
If I do this multiplication by one there, it picks out this column, right?
That one will pick out this column, so you see it's maybe I come up here now.
Shall I just, when I pick out that column, the answer then, I guess I've got the column circle, there it is minus i.
Minus i squared, minus i cubed.
You see it's like k equals zero, one, two, three.
Just the way here, we had k, well we had all integers.
k in that function case.
Here we've got four integers, k equals zero, one, two, and three, but again it's the minus i, it's the e to the, it's the w bar.
In other words, the answer was one w bar w bar squared w bar cubed.
Just the powers of w with this factor 1/N.
Here we had a modulation.
It's the same picture.
Absolute value's one.
And and what about energy?
Having mentioned energy, so that's another key rule.
The key rules for the Fourier series, just let's think back.
What were the key rules?
First, the rule to find the coefficients.
Good.
Then the rule for the derivatives.
This is so important.
These are rules.
Let's say, for Fourier series.
For Fourier series.
Let's just make this a quick review.
What were the important rules?
The important rules were, if I had the Fourier series of f. Start with the Fourier series of f. Then the question was, what's the Fourier series of df/dx.
And now I'm saying the next important rule is the Fourier series of f, shifted.
And then the last important rule is the energy.
OK, and let's just, maybe this is a bad idea to, since we're kind of doing all of Fourier in, it's coming in three parts.
Functions, discrete, integrals, but they all match.
So this is what happens to the function.
What happens to the coefficient?
So this starts with coefficient c_k, for f, what are the coefficients for the derivative, just remind me?
If f(x), so I'm starting with f(x) equals sum of c_k*e^(ikx).
Start with that.
And now take the derivative.
When I take the derivative, down comes ik.
So you remember that rule.
Those are the Fourier coefficients of the derivative.
Now what's the Fourier coefficients of the shift?
If I've just shifted, translated the function, if my original x was this, now let me look at f(x-a).
You'll see it.
It'll jump out at us, it'll be a sum of the same c_k's e^ik(x-a).
So what are the Fourier coefficients of that?
Well there is the e^(ikx).
Whatever's multiplying it has got to be the Fourier coefficient, and we see it as c_k times e^ik(-a).
e^(-ika), times c_k.
Right?
And, of course, that's just what we discovered here.
That's just what we found there, that when we shifted the delta, we've multiplied by this modulation, this phase factor came into the Fourier coefficients.
And now finally, the energy stuff.
You remember the energy was, what's the energy?
The integral from minus pi to pi, of f(x) squared.
dx is the same as the sum from minus infinity to infinity of the coefficient squared.
And somebody correctly sent me an email to say energy and length squared are you really, is there much difference?
No.
No.
You could say length squared here, I'm just using the word energy.
Now, I left a space because I know that there's a stupid 2pi somewhere.
Where does it come?
You remember how to get this?
You put that whole series in there, multiply by its complex conjugate to get squared.
And integrate.
Right?
That was the idea.
Isn't that how we figured out, we got to this?
We started with this, length squared.
We plugged in the Fourier series.
This is f times f bar, so that's this times its conjugate.
And we integrated, and all the cross terms vanished.
And only the ones were, e^(ikx) multiplied e^(-ikx) came.
And those had c_k squared.
And when we integrated that one, we probably got a 2pi.
So that's the energy inequality, right.
For functions.
And now what I was going to say is, you shouldn't miss the fact that in the discrete case, there'll be a similar energy inequality.
So we had y was the Fourier matrix times c. Now, if I take the length squared of both, y, so I'm going to right?
That's the same as that.
Now I'm going to do the same as this.
I'm going to find the length squared, which will be y transpose y.
No, it won't be y transpose y.
What is length squared?
y bar transpose y. I have to do that right.
That will be, substituting that's c bar F bar transpose times y is Fc.
Just plugged it in, and now what do I use?
The key fact, the fact that the columns are orthogonal.
That's what made all these integrals simple, right?
When I put that into there, a whole lot of integrals had to be zero.
When I put this in, a whole lot of dot products have to be zero.
Rows of F bar times columns of F, all zero.
Except when I'm hitting the same row, and when I'm hitting that same row I get an N. So I get this, is N c bar transpose c. And that's c squared.
That's the energy inequality, it's just orthogonality once again.
Everything in these weeks is coming out of orthogonality.
Orthogonality is the fact that this is N times the identity.
Right?
Well, OK that's a quick recall of a bunch of stuff for functions.
And just seeing maybe for the first time the discrete analogs.
I guess I don't have a brilliant idea for the discrete analog of the derivative.
Well, guess there's a natural idea, it would be a finite difference, but somehow that isn't a rule that gets like, high marks.
But we saw the discrete analog of the shift and now we see the energy inequality is just that the length of the function squared is equal to N times the length of the coefficient squared.
OK with that?
Lots of formulas here.
Let's see, and do some examples.
I mean, these were simple examples.
And I think the homework gives you some more.
You should be able to take the Fourier transform and go backwards.
And when we do convolution in a few minutes, we're certainly going to be taking the Fourier, we're going to be going both ways.
And use all these facts.
OK, I'll pause a moment.
That's topic one.
Topic two, fast Fourier transform.
Wow.
What's the good way to - I mean, any decent machine comes with the FFT hardwired in.
So you might say, OK I'll just use it.
And that seems totally reasonable to me.
But you might like to see just on one board, what's the key idea?
What's the little bit of algebra that makes it work?
So I'll just have one board here for the FFT and a little bit of algebra.
Simple, simple but once it hit the world, well, computer scientists just love the recursion that comes in there.
So they look for that in every possible other algorithm, now.
OK, let me see that point.
So here's the main point.
That if I want to take the multiply by F of size 1,024, the fast Fourier transform connects that full matrix to a half-full matrix.
It connects that to the half-full matrix that takes the half-size transforms separately.
So it's half-full because of these zeroes.
That's the point.
That the 1,024 matrix is connected to the 512 matrix.
And what's underlying that?
The 1,024 matrix is full of e^(2pi*i), the w, over a 1,024, right?
That's the w for this guy.
And then the w for this guy, for both of these, is e^(2pi*i) over 512.
So if there's a connection between that matrix and this matrix, there'd better be a connection between that number and that number.
Because this is the number that fills this one, and this is the number that fills these.
So what's the connection?
It's just perfect, right?
If I take this number, which is one part of the whole circle, 1/1,024, a fraction of the whole circle, what do I do to get this guy?
To get 1/512th of the way around the circle?
I square it.
The square of this w is this w. Let me call this w_N, and this one w_M.
Maybe I'll use caps, yeah I'm using cap N, this is my w. This is the one I want.
The w_N, the N by N one, N is 1,024, and the point is, everybody saw that, when I squared it I doubled the angle?
When I doubled that angle the two over that gave me 1/512.
Fantastic.
Of course, that doesn't make this equal to that, but it suggests that there is a close connection.
So let me finish here the key idea of the Fourier transform in block matrix form.
What's the key idea?
So instead of doing the big transform, the full size, I'm going to do two half-sizes.
But what am I going to apply those to?
Here's the trick.
These two separate guys apply to the odd-numbered coefficients.
The odd-numbered component.
And the even.
So I have to first do a little permutation, and even comes first, always.
Even means zero, two, four, up to a 1,022.
So this is zero, two, zero, up to 1,022.
And then come all the odd guys.
One up to 1,023.
So this is a permutation, a simple permutation.
Just take your 1,024 numbers, pick out the even ones, put them on top.
Right?
In other words, put them on top where 512 is going to act on it.
Put the odd ones at the bottom, the last 512, that guy will act on that.
So there's 512 numbers there, with the even coefficients, this acts.
OK, now we've got two half-size transforms.
Because we're applying this to the y, to a to a typical y. OK.
But I've just written F without a y so I don't really need a y here.
This is a matrix identity.
It's a matrix identity that's got a bunch of zeroes there.
Of course, that matrix is full of zeroes.
I mean, this is instant speed to do that permutation.
Grab the evens, put them in front of the odds.
OK, so now I've got two half-sizes, but then I have to put them back together to get the full-size matrix.
And that is also a matrix.
Turns out to be a diagonal there, and a minus the diagonal goes there.
So actually, that looks great too, right?
Full of zeroes, the identity.
No multiplications whatever.
Well, these are the only multiplications, because sometimes they're called twiddle factors, give it a fancy name.
Official sounding name, twiddle factors.
OK, so that diagonal matrix D, what's that?
That diagonal matrix D happens to be just the powers of w, sit along D. 1 w up to w to the, this is W_N, we're talking, the big W, and it's only half-size so it goes up to M-1.
Half, M is half of N. Everybody's got that, right?
M is half of N here.
I'll get that written on the board.
M is 512, N is 1,024, here we have the powers of this guy up to 511.
The total size being 512 because that's a 512 by 512 matrix.
I guess I can remember somewhere, being at a conference, this was probably soon after the FFT became sort of famous.
And then somebody who was just presenting the idea and as soon as it was presented that way, I was happy.
I guess.
I thought OK, there you see the idea.
Permutation, reorder the even-odd.
Two half-size transforms, put them back together.
And what's happened here?
The work of this matrix, multiplying by this matrix, which would be 1,024 squared is now practically cut in half.
Because this is nothing.
And we have just this diagonal multiplication to do, and of course this is the same as that, just with minus signs.
So the total multiplications we have to do, the total number of twiddle factors, is just 512, twelve and then we're golden.
So we've got half the work plus 512, 512 operations.
That's pretty good.
And of course it gets better.
How?
Once you have the idea of getting down to 512, what are you going to do now?
This is the computer scientist's favorite idea.
Do it again.
That's what it comes to.
Whatever worked for 1,024 to get to 512 is going to work.
So now I'll split this up into, well, each 512, so now I have to do, yeah.
Let me write F_512 will be, it's now smaller.
An I, an I, and a D and a minus D for the 512 size of F_256, 256 and then the permutation, the odd-even permutation P. So we're doing this idea in there, and in there.
So it's just recursive.
Recursive.
And now, if we go all the way, so you see why I keep taking powers of two.
It's not natural to have powers of two.
Two or three.
Three is also good.
I mean, all this gets so optimized that powers of two or three are pretty good.
And you just use the same idea.
There'd be a similar idea here for, if I was doing instead of odd-even, even-odd I was doing maybe three groups.
But stick with two, that's fine.
Then you might ask, if you were a worrier I guess you might ask what if it's not a power of two.
I think you just add in zeroes.
Just pad it out to be the next power of two.
Nothing difficult there.
I think that's right.
Hope that's right.
And once this idea came out, of course, people started looking.
What if the number here was prime?
And found another neat bit of algebra that worked OK for prime numbers.
Using a little bit of number theory.
But the ultimate winner was this one.
So maybe I'll just refer you to those pages in the book, which were, you'll spot this matrix equality.
And then next to it is the algebra that you have to do to check it.
I can just say, because I want you to look to the right spot there, maybe I'll take out the great - this is sometimes called after Parseval, or some other person.
Yeah, the algebra.
Let me start the algebra that made this thing work.
We want the sum, when we multiply Fourier F times something, we want the sum of w^jk, right?
That's the coefficient, that's the entry of F, times c_k.
Sum from k=0 to N-1.
To 1,023.
That's the y_j that we're trying to compute.
We're computing a 1,024 y's from 1,024 c's by adding up the Fourier series when we multiply by F. This is F, this is the equation y=Fc written with subscripts.
So this is what the matrices are doing, and now where do I find that 512 thing.
How do I get M into the picture, remembering that the w_N squared was w_M, right?
That's what we saw.
This is the big number, this is half of it, so this is a little bit of the part around the circle.
When I go twice as far I get to the other one.
So that's the thing that we've got to use.
Everything is going to depend on that.
So this was w_N, of course.
This is the N by N transpose So now comes what, what's the key idea?
The key idea is split into even and odd.
And then use this.
So split into the even ones and the odd ones.
So I write this as two separate sums, a sum for the even ones w_N, now, so now the even c_k, so I'm going to multiply by c_2k.
These are the ones with even, and the sum is only going to go from zero to M-1, right?
This is only half of the terms.
And then look on the other half, plus the same sum of, but I didn't finish here.
Let me finish.
So I'm just picking out, I'm taking, instead of k I'm doing 2k.
So I have j times 2k here.
And now these will be the odd ones.
c_(2k+1), and omega_N^j(2k+1).
It's a lot to ask you.
To focus on this bit of algebra.
I hope you're going to go away feeling, well it's pretty darn simple.
I mean there'll be a lot of indices, and if I push it all the way through there'll be a few more.
But the point is, it's pretty darn simple.
For example, this term.
What have I got there?
Look at that term, that's beautiful.
That's w_N squared.
What is w_N squared?
It's w_N.
So instead of w_N squared, I'm going to replace that w_N squared by w_M.
And the sum goes from zero to M-1, and what does that represent?
That represents the F_512 multiplication, the half-size transform.
It goes halfway, it operates on the even ones, and it uses the M. It's just perfectly, so this is nothing but the Fourier matrix, the M by M Fourier matrix, acting on the even c's.
That's what that is.
And that's why we get these identities.
That's why we get these identities, because they're acting on the even - the top half is the even guy.
OK, this is almost as good.
This is the odd c's.
Here I have, now do I have w_M here?
I've got to have w_M.
Well, you can see.
It's not quite coming out right, and that's the twiddle factor.
I have to take out a w_N to the power j to make things good.
And when I take out that w_N to the power j, that's what goes in the D, in the diagonal matrix.
Yeah.
I won't go more than that.
You couldn't, there's no reason to do everything here on the board when the main point is there.
And then the point was recursion and then, oh, let me complete the recursion.
So I recurse down to 256, then 128, then 64.
And what do I get in the end?
What do I get altogether, once I've got all the way down to size two?
I have a whole lot of these factors down in the middle, the part that used to be hard is now down to F_2 or F_1 or something.
So let's say F_1, one by one, just the identity.
So I go all the way.
Ten steps from 1024, 512, 256, every time I get twiddle factors.
Every time I get P's.
A lot of P's, but the F_512, what used to be the hard part, is gone to the easy part.
And then what do I have?
I've got just a permutation there.
And this is the actual work.
This is the only work left, is the matrices like this, for different sizes.
And I have to do those.
I have to do all those twiddle factors.
So how many matrices are there, there?
It's the log, right?
Every time I divided by two.
So if I started at 1,024, I do ten times, I have ten of those matrices and have me down to N=1.
- So I've got ten of these, and each one takes 1,024 or maybe only 1/2 of that.
Actually, only half because this is a copy of that.
I think the final count, and can I just put in here, this is the great number, is each of these took n multiplications.
But there were only log to the base two - oh, no.
Each of them took half of N multiplications, because the D and the minus D are just, I don't have to repeat.
So half N for each factor and the number of factors is log to the base two of N. Ten, for 1,024.
So that's the magic of the FFT.
OK.
It's almost all on one board, one and a half boards.
To tell you the key point, odds and evens, recursion, twiddle factors, getting down to the point where you only have twiddle factors left and then those multiplications are only N log N. Good?
Yes.
Right, OK. Now.
that's discrete transform.
The theory behind it and the fantastic algorithm that executes it.
Are you ready for a convolution?
Can we start on a topic that's really quite nice, and then Friday will be the focus on convolution.
Friday will certainly be all convolution day.
But maybe it's not a bad idea to see now, what's the question.
Let me ask that question.
OK, convolution.
So we're into the next section of the book, Section 4.4, it must be.
And let me do it first for a Fourier series.
I have convolution of series, convolution of discrete.
Convolution of integrals, but we haven't got there yet.
So I'll do this one, this series.
So let me start with a couple of series.
f(x) is the sum of c_k*e^(ikx).
g(x) is the sum of some other coefficients.
And I'm going to ask you a simple question.
What are the Fourier coefficients of f times g?
If I multiply those functions, equals something.
And let me call those coefficients, h maybe.
h_k*e^(ikx), and my question is what are the coefficients h_k of f times g?
That's the question that convolution answers.
Actually, both this series and the discrete series are highly interesting.
Highly interesting.
So here I wrote it for this series.
If I write it for the discrete ones, you'll see, so let me do it over here for the discrete one.
Because I can write it out for the discrete ones.
My y's are c_0+c_1 e - no, w I have this nice notation w, plus c_N-1*w^(N-1), right?
That's the - ooh.
What's that?
I haven't got that right.
Yes, what do I want now?
I need, yep.
Sorry, I'm looking.
Really, I'm looking at y_j, the j'th component of y.
So I need a w^j, w^j(N-1).
Yeah, OK, let me - OK. Alright.
And so that's my f. I'll come back to that, let me stay with this.
I'll stay with this to make the main point, and then Friday we'll see it in a neat way for the discrete one.
So I'm coming back to this.
f has its Fourier series g has its Fourier series.
I multiply.
What happens when I multiply this times this?
I'm not going to integrate.
I mean, when I do this multiplication, I'm going to get a mass of terms.
A real lot of terms.
And I'm not going to integrate them away.
So they're all there.
So what am I asking?
I'm asking to pick out all the terms that have the same exponential with them.
Like, what's h_0?
Yes, tell me what h_0 is?
If you can pick out h_0 here, you'll get the idea of convolution.
What's the constant term if I multiply this mess times this mess, and I look for the constant term, h_0, where do I get the constant terms when I multiply that by that?
Just think about that.
Where do I get a constant, without any k, without an e^(ikx)?
If I multiply that by that.
Well tell me one place I get something.
c_0 times?
d_0.
Good.
Is that the end of the story?
No.
If you thought that multiplying the functions, I just multiplied the Fourier coefficients, the first point is no.
There's more stuff.
Where else do I get a constant out of this?
Just look at it, do that multiplication and ask yourself where's the constant.
Another one, yep.
You were going to say it is?
c_1 times d_-1.
Right.
Right.
c_1 times d_-1.
And tell me all of them, now.
c_2 times d_-2.
And what about c_-1?
There's a c_-1.
It multiplies d_1.
And onwards.
So the coefficient comes from, now how could you describe that?
I guess I'll describe it as, I'll need a sum to multiply.
This will be the sum of c_k times d_1.
Minus k, right?
That's what you told me.
Piece at the start, and that's the pattern that keeps going.
OK, that's h_0, the sum of c_k times d_-k. Now, we have just time to do the next one.
We've got time but not space, where the heck am I going to put it?
I want to do h_k, I guess.
Or I better use a different letter h. l, let me use the letter h_l, and God there's no space.
Alright, so can I - yes.
h_l.
OK.
So this was h_0, let me keep things sort of looking right for the moment.
OK, now you're going to fix h_l.
So what does c_0 multiply if I'm looking for h_l, I'm looking for the coefficient of e^(ilx).
So ask yourself how do I get e^(ilx) when that multiplies that?
When that multiplies that, and I'm looking for an e^(ilx), I get one when c_0 all multiplies what?
This is it.
dl, right.
And what about for c_1?
Think of here, I have a c_1*e^(ilx).
What does it multiply down here to get the exponential to be l?
ilx?
What doesn't multiply d_-1.
It multiplies, c_1 multiplies?
d_(l-1).
Good, good.
l-1, right.
l-1, and what are you noticing here?
c minus, I'll have to fill that in.
But you're seeing the pattern here?
And what was the pattern here?
Those numbers added to this number.
And now these numbers add to l Those numbers add to l, whatever it is, the two indices have to add to l, so that when I multiply the exponential they'll add to e^(ilx).
They'll multiply to e^(ilx).
So what goes there?
It's probably l+1, right?
So that l+1 combined with minus one gives me the l. If you tell me what goes there, I'm a happy person.
Let's make it h_l.
We're ready for the final formula for convolutions.
Big star.
To find h_l, the coefficient of e^(ilx), when you multiply that by that, you look at c_k, and which d is going to show up in the e^(ilx) term?
l-k, is that what you said?
I hope, yeah.
l-k.
Right, that's it.
That's it.
So we've got a lot of computation here.
But we've got the idea of what, we've got a formula.
And most of all we have the magic rule.
In convolutions, convolutions are, things multiply but indices add.
Things multiply, numbers multiply, while their indices add.
That's the key idea of convolution that we'll see clearly and completely on Friday.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, thanks for coming today.
This is a key lecture in the application of Fourier, you could say.
So convolution is the big word.
And a major application of convolution is filtering signal processing.
So we'll develop that application.
But it's nothing but convolution.
So the key idea is these convolution rules, where they come from.
And what these new symbols, that's the symbol for the convolution of two functions.
They could be functions here.
Here they're long vectors of coefficients, and so these are the rules.
So it's just a little bit of algebra.
But it just is so central to all this subject.
Signal processing is certainly the most important little thing to know.
That if I multiply two functions, so that's where we started last time.
If I have a function f with Fourier coefficient c, and a function g with coefficients d, then, oh, wrong.
I convolve the coefficients, right.
So f has coefficient c, g has coefficient d. So if I multiply the functions I definitely do not just multiply each c times the d. I do this convolution operation, which we have to remember.
That's our main first step is to remember what that was about.
And then this is the other direction.
So we didn't see this before.
That I mean, there's always this fantastic symmetry between physical space and frequency space.
And convolution in one is multiplication in the other.
It's so easy to remember.
If I multiply in one space, I do a convolution in the other space.
If I do a convolution of functions, I do a multiplication of coefficients.
So that's the rule to know.
And now if we expand on it, by sort of seeing again what it means.
So let me do a quick repeat of this first step to remember what this symbol, convolution symbol, means.
And then another thing I have to do.
Here, I'm talking about the infinite case, functions and with a whole infinite sequence of coefficients.
I've got to do the cyclic case, too.
Which goes with the discrete transform.
OK, but let's start with the infinite case as we did last time.
Maybe, here's something.
Here's a suggestion.
We had f(x).
We started with f(x), as the sum.
And remember to have nice formulas, we're doing the complex version.
c_k*e^(ikx).
Let me suggest something.
Let me write z for e^(ikx).
So I'm going to just write that at the sum of c_k*z^k.
z to So z is e^(ix).
This is like a good point to make anyway.
When we have this e^(ikx), it's natural to think of that as a complex number on the unit circle.
Right?
That's always the message, e to the i real is on the unit circle.
Absolute value one.
And this is great for periodic function.
Because if these functions have period 2pi, and if we think of our function as being on the circle, it obviously has period 2pi.
I mean, the picture says, yes.
If I go 2pi, I come back.
So x is the angle, right?
x is the in this picture, a little bit unusual maybe, x there would be the angle.
And it's just a little bit easier to write.
And maybe it even has an official name, the z transform.
How do you like that?
You learn a transform just, in fourteen seconds.
z transform.
It'll make it easier to multiply by g, so g is going to be the sum of d_k*z^k.
So just think of these as long, well I'm tempted to say, long polynomials.
I mean, very long, because they can be infinite series.
Infinite negative powers and positive powers.
But just think of it as a bunch of powers of z times a bunch of powers of z.
And if you multiply a couple of polynomials, you're doing convolution.
I guess my message is, you've been doing convolution since the second grade.
That's the real message.
Here, let me show you.
This is convolution, too.
Suppose I have to multiply a 123 times 456?
OK, so what do you do?
Remember back, it's a long way back but we can do this.
One, two, three times four, five, six.
So I multiply the three, oh there's a little point.
Where that second-grade teacher's going to panic.
I'm going to write that as 18.
And that's 15, and that's 12.
Sorry about that, yeah.
And 12, ten and eight, right?
Four, or five and six.
So you see the nine multiplications that you have to do?
Nine multiplications, three times three.
OK, right now imagine those were, they could have been longer.
But they were finite length filters, we could say.
And now, what does convolution do?
Just what you did in multiplication.
When I add 12, 10 and 6, what am I doing?
I'm putting together the three times the four, the two times the five, and the one times the six.
That's what convolution, those are the things that convolution puts together.
So we got an 18, a 27, 28.
13, and four.
So I guess I'm saying that, alright, here's what I really want to say.
I want to say that the convolution of those, with these, four, five, six, is this sequence here - oh.
Yeah, that's right.
Four, 13, 28.
27, and 18.
If you just look at that.
Where did that 13 come from?
Let's just remember, where does that 13 come from?
That came from, this was z^0.
This 13 is 13z to the first power, right?
We're just checking all the powers here.
There's 13z to the first power.
Where do we get a first power?
We get a z^0 times 5z^1.
So that's 5z^1.
And we also have 2z^1, times 4z^0.
Right?
Two times four gave the eight.
So the eight from there and the five from there produce that 13.
And that's just what you did in multiplication.
Right?
So that's multiplication of two series.
That's not cyclic.
This is definitely not yet cyclic, and but we'll make it cyclic in a minute.
This is the infinite one, except that we had all zeroes beyond.
OK, so that if you in non-cyclic convolution like this, if I have length m and length n, then I get length m+n, maybe m+n-1.
OK, so that's convolution.
Without carrying numbers.
Without doing it right.
OK, so and what does that correspond to?
Let me just, so you see it every way.
That corresponds to 1+2z+3z^2, multiplying 4+5z+6z^2.
And it gave, that's times.
And it gave this thing up to 18z^4.
Just, exactly the multiplication that you've always done.
OK, so that's the idea.
Over on that board I'm going to put a formula for this convolution operation.
But my point on this board is, you've done it always.
When you multiply a couple of polynomials, you collect powers.
And that's all convolution is doing, collecting each power separately.
OK, let's do it.
So then f(x), g(x), is, when I multiply that polynomial or series by that polynomial, I get some polynomial in, with coefficients, oh I was changed to l, just to have a different symbol there.
And what was the formula for l?
For h_l?
What is the coefficient of z^l?
If I multiply that by that, do you remember the story there?
When I multiply that by that and I looked for the terms that gave me z^l?
OK, well that means that this power times this power is going to be z^l.
So that the index, do you remember what happened?
It was a lot of different, just the way this h_l is here.
Here's h_2 or something.
I've got to do an addition.
Because a bunch of c's come in with different d's, and what's the deal then?
c_k comes in with which d?
This is the magic number there.
What's the subscript that if I look at the coefficient of z^l, I look at each of these.
And then I pick out the one of these that will give me a z^l.
And which one is it?
d_(l-k).
It's that magic quantity that the eye spots perfectly.
k and l-k, adding to l simply because z to the k is z^(l-k), multiplies to z^l.
Same thing.
Right?
OK, this is, now I'll use that notation.
This is c convolved with the d. c convolved with d is h. c convolved with d is h. So this is the l'th component.
c convolved with d is my symbol.
This h is the convolution.
And that's the convolution rule.
It's just whatever operation you have to do to get the right answer.
The right answer when you multiply.
So that one.
Ready for the discrete case?
The finite case?
The case when, you have power, when z becomes w. The discrete case.
The cyclic case, sorry.
Maybe emphasize the cyclic case, meaning it circles around, is the case when z becomes this very special z, on the unit circle that we know as w. OK, and we have to say, and of course it's w_N, I have to tell you in the cyclic case how long the cycle is.
So this would be a case.
Watch what you do here to make this cyclic.
OK, I have three inputs, so N is three here.
Now I'm going to do the cyclic.
So instead of z's, I should be putting w's.
I will.
Just to emphasize, it's good to think of it with the w there, because the w has this special property that's critical to everything, OK so now I'm going to, I think of this is as 1w^0, 2w^1, and 3w^2.
I'm thinking of the same multiplication here, but now w's.
OK, so I'll end up with 18w^4, and four was the constant and 3w's and so on.
All these numbers.
What's the difference?
Ready for the key point?
Now, what's happened in this cyclic case?
Well, the difference is what is w^4?
If we're in the cyclic case, N is three now, our guys have length three, our circle is w now is 1/3 of the way around.
So that's my w, here's my w squared.
Here is my one.
But here it is also w cubed.
So w is the same as w^4, w^2 is the same as w^5.
So, what's the difference?
What can I do now?
If I'm in this discrete case, then my inputs are a vector of length N, three.
A vector of length N, three.
And I want to get out to a vector of length three.
I'm not happy with that in the cyclic case, because I'm not happy with w^4.
So now tell me again that last, when I do the multiplication and I just do it, there's no difference except in how I write the answer.
18w^4 is the same as?
w. w^4 is the same as w, when N is three.
So that 18w^4 cycles back in, with this 13.
So now if I do the, can I show you the symbol?
I'll just do a little circle there.
To say this is now the cyclic convolution.
Then this isn't the answer any more.
The answer now for cyclic convolution, I only want three numbers.
And if you can tell me what those three numbers are, we've got it.
Move those over a little to make room for the three numbers.
So there's the answer, the space for the answer.
What do I write in?
How many, what's the constant term?
It's 31, right.
It's 31.
Where did 31 come from?
It came from, you could say, cycling that 27w^3 back with the four.
Because there's no w^3 is the same as one.
So I've gone around the circle when I come around to w^3.
So that 27 and four combined into that 31.
And let's see so so what multiplications am I doing here?
One times four gives me the constant.
Two times six, that's 2w's, and six w squareds, that's 12w^3, that's 12.
And then 3w^2, and five w's is 15w^3.
But that's the same as 15.
So that's why we get 31.
We got four, we've got 12 and we've got 15.
OK, now what's the second component, the w component of the cyclic convolution?
Tell me what number do I write in, in that middle position?
How many w's do I have?
31 again.
This 28, this 18 is coming back by three.
To 13.
You see, I could have done that multiplication.
So coming back to 31, am I going to get another 31?
No, what's the w^2 guy?
28.
Yeah.
28, because there's no to the fifth to come back.
So 28 uses three multiplications.
The 31 there used the four and these two.
This came back over to here.
And this 31 used, this 18 came back.
I could have put the 18 here.
You know, I could have lined it up just three.
So I'll write a formula for it.
So that's the answer.
31, 31, 28.
Could I just suggest a little check on that?
Just to check on the numbers.
I think that if I add up these numbers, I get six.
And if I add up those numbers I get 15.
And if I multiply that, I get 90.
And if I add those numbers I get 90, right?
So those add to 90.
So I'm just saying the miracle check is add these, multiply by the sum of those.
And you get the sum of those.
Why is that?
Somehow seems right, doesn't it?
Because somehow I've taken all nine products here, and so when I add all the results, I'll have the sum of all nine of these possible products.
So I'll have six times 15.
Actually, here's a good way to look at it.
In doing this, I just set w=1.
I just set w=1 in the polynomial.
When I set w to one, this becomes six, this becomes 15 and the answer becomes 90, when w is one.
Yeah.
So that's another way to see.
And actually, this multiplication, the second grade version, had w, well, I'm almost going to say w=10, but not quite that.
Because it's written in the opposite order, right?
If w was ten, this would be one.
Well, anyway, w can't be ten, it's got to stay on the unit circle, so.
So somewhere in the non-cyclic case, it's something like w=10, or w=1/10, maybe.
Whatever.
OK. Could you take the convolution now, let me give you just another example.
Do it mentally.
What's the convolution of , let me make a little longer.
So I take the, first of all, the non-cyclic convolution of .
OK. What's the ordinary - how long is the answer now?
This is just practice.
How many components am I going to have in the ordinary convolution of those two guys of length four?
I think it's seven.
I think it'll be seven.
Because we'll have here one, we have no, we have it z^0, z^1, z^2, z^3.
And here we'll have again the same, it would go up to z^6 but remember there's a z^0, so that's why we have seven.
And what will it be?
What will it be?
I guess, actually, this wasn't a brilliant example, was it?
But let's finish it.
So I'm just multiplying z by z squared, so what do you get for an answer?
I think the one shows up in the z^3.
Is that right?
Yeah.
z^1, here z^2 here, z^3 here, and we always have to remember everything in Chapter 4 starts at zero.
Zero, z^0's the first one.
OK, that's not too great an example, because what happens if I do the circular cyclic convolution?
What would be the cyclic convolution?
Now I'm expecting four guys only, right?
A cyclic keeps the same length.
And what would be the answer?
Well, there's nobody to fold back, so it would be just .
So let me update this a little bit with one here.
OK, just to practice.
So suppose I do that convolution.
Un-cyclic, first.
What do I change here now?
I've now got a z^2 and I've also got a z^3, but I only have a single one there.
Let's make it a little more interesting.
OK, make it like so.
Alright, z+z^2 is what we're looking at, z+z^2 is multiplying z^2+z^3.
And in the long form, what do I get?
Let me make space, tell me what numbers to put in.
If I multiply z+z^2+z^3, I get what?
1z^3, how many z^4?
Two of them.
How many z^5?
One, and nobody there.
OK. And now the cyclic version would be what?
What's what's my answer now for the cyclic version?
Let me take those out.
So the cyclic version would bring the two back to the zero.
Would bring that one back so there.
That zero will still be zero.
And I checked that I haven't missed anything by adding those up to get four, and adding this up to get two times two.
Yeah.
OK, so that's the rule.
And it's a lot cleaner to see these answers than to see this formula.
And I need, actually of course, now mentioning that formula, I need a cyclic formula.
So can I write above it the cyclic formula?
What do I get when I'm, instead of this sum, which went from k equal minus infinity to infinity, in the cyclic case, h_k is just going to be a sum from zero to N-1, and there'll be a c_k, and a d something.
And now this is the cyclic case, so I guess this makes us, I think what our situation now is we understand the cyclic case from examples.
And now we just have the job of how do I put it into algebra.
How do I put it into symbols?
What's the point?
c_k d_n, let's say.
But oh no, I'm looking for h_l, yeah.
So what's the deal?
Here l was k. That one is the sum of that one and that one.
So here, l is the sum of k and n, but.
What's the but?
I mean somehow I've got some wraparound to do, right?
When I'm doing the cyclic multiplication and I'm doing the wraparound because w^n, the wraparound comes from that, right?
That's why I never get as high as n, because when I get to n I go back to the zeroth power.
OK, so what's the relation of k and n and l here?
We just need the right word to express it.
What's the word?
Mod.
So that's the word I'm looking for.
k+n is l. With wraparound and wraparound means that the nice notation that people use is a mod m. Let's practice.
What is two plus two mod seven?
Four.
Two plus two is four, even in 18.085.
Right, OK.
But two plus two mod three is, two plus two mod three is?
one.
Everybody sees it?
I'm taking z^2 times z^2, z^4, but I'm doing with N=3, so z^3 is on3, so that z^4 is really just z to the first power.
So this is the little nifty notation that says make it cyclic.
Bring it back so that l only has the values here, zero up to N-1.
And then stops.
OK. OK, so we'll have more practice with examples when we do some filtering.
Have you got that fundamental, so we've talked about this rule one here.
f times g goes to those coefficients.
And if it's the cyclic case then I put a circle around that star.
And I do the wraparound.
But it's just, it's the z transform, it's polynomials in z or polynomials in w, and when it's polynomials in w, you use that special property that w_N is one.
Yeah, OK. Now, I see I've written another line, there.
That I could convolve functions.
Let me do a couple more examples.
Couple of examples.
First, before I go to that line, OK.
So I'm up to this line.
A couple of examples here.
Let's see, what example would I want to do?
Let's see, OK, I want to do one example with a delta function.
One example with a delta function.
One example with the delta vector.
Yeah, let me take the function g(x) identically one.
OK, constant function.
In this rule.
I want to see what happens with the rule.
OK, then f(x) g(x) is the same as f(x), right?
Because this function g(x) is so simple, it's just one.
Now, what about the coefficient?
So I have the coefficients of c, what are the Fourier coefficients, what are the d's?
Ah yes, what are the d's?
So I'm testing my rule on a really really simple case, g(x) identically one.
What, you have to tell me, in order to check the right side of the rule you have to tell me the Fourier coefficients for that very special function.
What would be the Fourier coefficients?
If I expand the function one in a Fourier series, what do I see?
I see a one.
Yeah, that's it, I see one.
So what are its coefficients?
d_0, right, is one?
And the other d's are all zero, right?
So my vector of d, my vector of d's is a whole lot of zeroes on the negative side.
A one right there in the center, and then a lot of zeroes.
And now I want to convolve that with c. I'm practicing the convolution rule on a case that's so simple it's confusing, right?
I mean, it's a big mess, this multiplication.
What do I get out of this?
If d is this vector, if d has this property that d_0 is one and others are zero, all others are zero, so this is my little example, what does this sum boil down to?
Well, I only get something when l=k, right?
I only got something when l=k, because then I have d_0 and that's the only d that's around.
So in this sum, something happens only when k and l are the same.
And then what happens?
Then I have a one, I have c_l, and that's h_l, so that's all I'm concluding then.
That this h is the same as c. I'm sorry, it's so dumb.
My point is that in convolution, this is the thing that acts like one.
Because in multiplication, that's the thing, that's the function that acts like one.
That's the function that is one.
So this is the one in, oh, would you allow me to do this?
I'm going to create a matrix with these d's.
There's another way to see convolution.
Yeah, there's another way to see convolution and discrete convolution.
Maybe the discrete one's the better.
Yeah can you stand one more way to write the formula?
One more way to write, now I'm going to do, I'm going to do discrete convolution.
Discrete cyclic.
Just So how am I going to write it?
I'm going to write it by a matrix multiplication.
Because you know that in this course a matrix was going to show up.
So it's going to be a matrix multiplication.
So I just have to tell you the matrix, so this is going to be some matrix.
Let me take N to be four.
So then I have, you watch.
So I have four d's, and the output is the four h's.
And the rule I'm following is this rule.
Is this, the same old rule but with the cyclic part.
And now I want to show you the matrix that'll just do this.
Look, I've put the c's in the first column.
And then I go, yeah, here's another.
So it's a cyclic matrix.
So let me finish it up.
It's going to be four by four, it's going to be cyclic.
So I have a c_0, c_0, c_0, c_0 on the diagonal.
That's fine, that's because z to the zeroes multiplying all the d's and leaving them in place.
And then I have c_1's, and then I think I come around again here for a c_1.
And I have c_2's, see where see c_3, c_2.
And I come around again, to a c_2 and a c_2.
And c_3 comes around to a c_3, a c_3 and a c_3.
Well, can you, I hope you can see, this is cyclic matrix.
It's only got one, it starts with a vector c, and those are on the diagonal and the diagonals wrap around.
That's the other word that you often see when you see the word cyclic, wraparound.
It's because you think of a circle.
If you go the second time around, it's wrapped around the first time.
OK, just can you look and see that this is the right formula for h_0?
h_0 is c_0*d_0.
Where does that come from?
Remember, h_0 is the coefficient of z^0 in the answer.
So it comes from c_0*d_0 to the zeroth power in the input.
And then why is there is a c_3*d_1?
Why is there a c_3*d_1, and then a c_2*d_2 and then a c_1*d_3 all piling up into h_0?
Tell me now, why is there a c_3*d_1?
Because we're doing mod four is one way to say it.
Three and one add to four.
Because c_3 is the w cubed guy, and d_1 is the coefficient of w^1 t and w^3 times w^1 piles back into the constant.
And you see the pattern of that matrix?
So these matrices are very important.
So they circle around.
Oh, we've actually met a matrix of this type, the first day of 18.085.
What was that matrix?
It was one of our four great matrices.
And now here it is back again.
Which one was it?
Well, you remember the letter for it.
Which isn't going to change.
And do you remember the particular matrix?
Well, everybody does remember that matrix, right?
Twos were on the diagonal, minus ones were on the diagonal, and the diagonal curve continued.
Minus one was on this diagonal and that, continued and zeroes was on this diagonal.
So this is cyclic convolution, the circulant matrix, cyclic convolution by c, what's the c that produces that convolution matrix?
It's just, it's got - well there it is.
The first column is it.
Right, right.
And somehow I would say that that's an even vector.
It's sort of, I associate it with cosine.
It's an even vector.
Here is the zero term, and then these are the same, not to worry about that part.
Do you see that we've seen that matrix before?
And the cyclic convolution means you take its second differences, of course.
We're taking second differences, but our everything in our world is cyclic.
So the result, the x_4 is x_0.
So we're taking second differences - well, maybe I should say d - we're taking second differences d_i, 2d_i's, -d_(i-1), -d(i+1), I don't know if this is.
So there's a minus one, two, minus one.
And we're cycling around so that d_0 is d_4.
And d_1 is d_5, and d_-1 is d_3, whatever.
OK, I'm just reminding you, we've seen these before.
OK, so this is another way to remember the formula.
OK now can I ask you a practical question?
A practical question.
Let me bring back this second grade multiplication.
Well, I have a granddaughter named Elizabeth, I'll have to admit I didn't think about mentioning Elizabeth.
She's six.
And she delights in sending me long multiplications.
I mean, really long.
And then every time I talk to her on the phone, she says have you done that one yet?
And I say, I'm working on it I've got MATLAB at work.
Because they're ridiculous and I haven't figured out how to tell her.
I mean, she just writes page after page.
Times 100, plus three, minus seven, just whatever she things of.
OK, now I need help from the convolution groupal, here, actually.
So let's suppose that Elizabeth has given me a multiplication in which I have a thousand digits times a thousand, right.
Which Mathematica is prepared to do exactly, right?
MATLAB will mess up, but Mathematica and Maple and symbolic packages will do exact computations.
So what would be the right way, well let may make it 1,024.
1,024 digits times 1,024 digits.
Let's do the cyclic version first.
Elizabeth doesn't know about cyclic.
Maybe I could teach her that.
That'll keep her busy while I'm doing the multiplication.
OK, right.
Only her older brother would explain it to her, that's the trouble.
OK, so how am I going to do, or how are you going to do on the quiz, multiplication of a 1,024 digits times 1,024?
And I'll make it easy by making it cyclic, so I just want 1,024 digits in the answer.
OK. How would you do it?
Well, before today, you would have just multiplied, right?
You would have written down 1,024 two, lines of 1,024, done an addition.
And you would have had a million multiplications to do.
But how would you do it now?
Apart from giving it to Mathematica.
What's a faster way to do it?
What's a faster way to do a convolution?
The fast way to do a convolution is to use the convolution rule, go this way.
So take these numbers, these 1,024 numbers, in c and these 1,024 numbers in d, and, well what do I have to do?
I want to use the convolution rule, because multiplying is fast.
Now I've got functions.
But I'm in the cyclic case.
So I'm in the cyclic case, so what should I do?
How can I change this to be the cyclic case?
This is like f, j, g, g, j.
So multiplication of components of things in function space is convolution of coefficients.
So now, this is the cyclic.
So let me make it cyclic.
So again, what's your problem?
The problem is to do this cyclic multiplication.
What's the idea?
The idea is to transform c back to f, to transform d back to g. Do the multiplications, now I have only 1,024 multiplications.
Not 1,024 squared.
That's the point.
And if I do this directly, I've got 1,024 squared multiplications to do.
Much better.
Transform back to here, do just 1,024 - what's the MATLAB command for that, when you're multiplying each component by itself?
It's not the dot product, notice.
It's not the dot product because I'm not summing.
Do you know the MATLAB command, if I have a sequence of numbers of vector f, of length 1,024 and I want to get that result?
What's the result?
It's a vector of length 1,024 that takes each f times its g. But doesn't do any adds.
That's what's there.
What's the MATLAB command for that?
Dot, yeah.
Dot star, right.
So that dot says component by component.
OK, so what's the plan here?
I do c's back to F. By the Fourier matrix.
d back to g, by the Fourier matrix, then I do a very quick multiplication.
And then what?
Then I mustn't forget.
That I'm in frequency space, and what do I have to do?
I've got to get back into coefficient space.
So I do the inverse transform - here's the formula, then.
I'm doing the inverse transform of F, so the transform of c dot star, the transform of d. To get c, d. Is that right?
So I took c, and I got back into the function.
I took d, and got back to its function, with the Fourier matrix.
OK, I'm in the Fourier and now I'm in this space.
I've added up coefficients to get in this space.
Now I do the dot star, the fast one.
And then I transform back.
So why is that faster?
Than just doing it?
Because what's the cost of F times c?
And how am I going to do that?
I'm going to do with the fast Fourier transform, right.
That's the point.
I can multiply by F, or by F inverse faster so I have three of these transforms.
I've got to get two guys into the other space, and the answer back out.
So I have sort of three of these n log n, but that will easily be n squared.
Right?
So if you have a convolution to do, and it's possible to do this, get into the other space where it's just an element by element multiplication.
And that would apply in either direction.
Because the rule goes both ways.
If I have this convolution to do, I would find the coefficients here, the c's, the coefficients of d of the g's.
I would do this one by one multiplication, and then I have the Fourier coefficients of the convolution.
Right, OK?
Do I have a moment?
Well, hardly.
Just can I write down what the formula for a convolution of two functions would look like?
Sorry, f(x) convolved with g(x).
Let me make it cyclic.
Just to see what it would look like.
What am I expecting for that convolution?
I'm expecting a function, and somehow there's going to be an integral instead of a sum, where I had sums, but now I have integrals.
And here's the point.
I'll have f(t) times g(x-t)dt.
All I'm asking you to look at is the fact that the way here I had k, and l-k. For functions, your eye sees that right away as a t, and an (x-t)dt.
These add to the answer.
These add to the result, that x.
That would be the cyclical one, yeah.
So I could go zero- these are all periodic functions, so all 2pi periods are the same.
The book will do that properly.
OK, we've got the filtering to discuss on Monday.
You can see that this convolution stuff just takes a little new thinking, but it comes out nicely.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
we he doesn't he's got n e t at you
OK, so.
These are our two topics.
And, Thanksgiving is coming up, of course.
With convolutions, where are we?
I'm probably starting a little early.
Oh, I am.
Is that right?
Yeah, OK.
So, with convolutions, I feel we've got a whole lot of formulas.
We practiced on some specific examples, but we didn't see the reason for them.
We didn't say the use of them.
And I'm unwilling to let a whole topic, important topic like convolutions, go on with just formulas.
So I want to talk about signal processing.
So that's a nice, perfect application of convolutions.
And you'll see it.
You'll see the point, and it's something that you're going to end up doing.
And it's quite a simple idea, once you understand convolutions you've got it.
OK, but then I do want to go on to the Fourier integral part.
And basically, today I'll just give the formulas.
So you've got the Fourier integral formulas that take from a function f, defined for all x now.
It's on the whole line, like some bell-shaped curve or some exponential decaying.
And then you get its transform, I could call that c(k), but a more familiar notation is f hat of k. So that will be the Fourier integral transform, or just for short, Fourier transform of f(x), and it'll involve all frequencies.
So we are getting away from the periodic case, and integer frequencies to the whole line case with a whole line of frequencies.
OK, so is that alright?
Finishing, I'll have more to say about 4.4 convolutions, but I want to say this much.
OK, let's move to an example.
So here's a typical block diagram.
In comes the signal.
Vector x, values x_k.
Often, in engineering and EE, people tend to write that x[k], these days.
OK, so, as being easier to type.
And sort of better.
But I'll stay with the subscript.
So that signal comes in.
And it goes through a filter.
And I'm taking the simplest filter I can think of, the one that just averages the current value with the previous value.
And we want to see what's the effect of doing that.
So we want to see, we want to understand the outputs, the y_k's, which are just the current value and the previous value averaged.
We want to see that as a convolution, and see what it's doing.
OK, so that's the - and I guess that here, it's a frequent convention in signal processing to basically pretend that the signal is infinitely long.
That there's no start and no finish.
Of course, in reality there has to be a start and a finish.
But if it's a long, you know if it's a cd or something, and you're sampling it every, thousands and thousands of times.
And the input signal is so long and you're not really caring about the very start and the very end.
It's simpler to just pretend that you've got numbers for all of k. OK, then let's see what kind of a filter this is.
What does it do to the signal?
OK. Well, one way filters are - well, first of all, let's see it as a convolution.
So I want to see that this formula is a convolution that the output y, that output of all the averages y, is the convolution of some filter, some, and I'll give it a proper name, but with the input.
And notice that, again, there's no circle around here.
We're not doing the cyclic case, we're just pretending infinitely long.
So OK, now I just want to ask you what is the h, I want to recognize this as a filter.
So that convolution, let's remember the notation.
And then we can match this to that.
So remember the notation for that would be, for a convolution is, I take a sum.
Of all x, of h_l.
x_(k-l), right?
That's what convolution is.
This is our famous, and it would be in principle the sum over f. Sum over l's.
So I the filter is defined by these numbers h, these h's, h_0, h_1, so on.
Those numbers h multiply the x's, where this familiar rule to give the k'th output.
OK, let's have no mystery here.
What are the h's if this is the output?
Well, I want to match that with that.
So here I see that this takes x_k, multiplies by half.
So that tells me that when l is zero, I'm getting h_0 times x_k, so what's h_0?
So what's the h_0?
It's 1/2.
That's what this formula is saying.
When l is zero, so l is zero, take h_0, 1/2 times x_k, ta-da.
Now, what's the other h that's showing up here?
This is a very, very short filter.
It's only going to have two coefficients, h_0 and h what?
One.
And what's the coefficient h_1?
What's the number h_1?
Now you've told me everything about h when you tell me that number.
Everybody sees it.
It's also 1/2, right?
Because I'm taking 1/2 of x_(k-1).
So when l is that one, we have h, the h is 1/2, times x_(k-1).
Do you see that that simple averaging, running average, you could call it.
Running average, it's the most, the first thing you would think of to - why would you do such a thing?
Why is filtering done?
This filter, this averaging filter, would smooth the data.
So the data comes with noise, of course.
And what you'd like, so noise is high frequency stuff.
So what you want to do is like damp those high frequencies a little bit, because much of it is not, it hasn't got information in it.
It's just noise, but you want to keep the signal.
So it's always this signal to noise ratio.
That's the key - SNR.
PSNR.
That's the constant expression, signal to noise ratio.
And we're sort of expecting here that the signal to noise ratio is pretty good.
High.
Mostly signal.
But there's some noise.
This is a very simple, extremely short, filter.
So this vector h, it's a proper convolution.
You could say h has infinitely many components.
But they're all zero, except for those two.
Right, do you see it?
Another way, just at the end of last time, I asked you to think of a matrix that's doing the same thing.
Why do I bring a matrix in?
Because anytime I see something linear, and that's incredibly linear, right?
I think OK, there's a matrix doing it.
So these y's, like y_k, y_(k+1), all the y's are coming out.
The x's are going in, x_k, x_(k+1), x_(k-1), bunch of x's.
And there's a matrix doing exactly that.
And what does that matrix have?
Well, it has 1/2 on the diagonal.
So that y_k will have a 1/2 of x_k, and what's the other entry in that row?
I want 1/2 of x_k, and I want 1/2 of x_(k-1), right?
So I just put 1/2 next to it.
So there is the main diagonal of the halves, and there is the sub-diagonal of the half.
So it's just constant diagonal.
Now, yeah, let me tell you the word.
When an engineer, electrical engineer looks at this, first thing, or this, any of these.
First letters he uses is linear time invariant.
LTI.
So linear we understand, right?
What does this time invariant mean?
Time invariant means that you're not changing the filter as the signal comes through.
You're keeping a half and a half.
You're keeping the formula.
The formula doesn't depend on k, the numbers are just 1/2 and 1/2.
They're the same, so if I shift the whole signal by a thousand, the output shifts by a thousand, right?
If I take the whole signal and delay it, delay it by a thousand times?
Clock times?
Then the same output will come a thousand clock times delayed.
So linear time invariant.
That would be - I mean, linear time invariant is just talking convolution.
I mean, that's what it comes to if we're in discrete problems.
It's just that, for some h. Now, our h deserves like, the other initials you see.
OK, that was linear time invariant.
Now, the next initials you'll see will be F-I-R.
It's an FIR filter.
So that's finite impulse response.
What's the impulse response mean?
It means the h. The vector h is the impulse response.
The vector h is what you get if I put an impulse in, what comes out?
Just tell me, what happens here.
Suppose an impulse, by an impulse I mean I stick a one in one position and all zeroes.
Our usual delta.
Our impulse, our spike, is just, suppose the x's have a one here, otherwise all zero, what comes out?
Well, suppose there's just x_0 is one.
What is y?
Suppose the only input is boom, you know a bell sounds at time zero.
What comes out from the filter?
Well, y_0 will be what?
If the input has just a single x, and it's one, then at that time zero, so x_0 is one.
Then y_0 will be?
1/2, and what will be y_1?
Also 1/2, right?
Because of y_1 will take x_1, that's already dropped back to zero.
Plus x_0, that's the bell.
It'll be 1/2.
In other words, the output is this.
No big deal.
The impulse responses is exactly h. So you can say they've created a long word for a small idea, true.
And the idea, the word finite is the important number.
Finite, meaning that it's finite length, I only have a finite number of h's, and in this case two h's.
So that's - part of every subject is just learning the language.
So LTI, it means you've got a convolution.
FIR means that the convolution has finite length.
OK, now for the question.
What is this filter doing to the signal?
It's certainly averaging.
That's clear.
But we want to be more precise.
Well, let me take some examples.
Suppose the input is all ones.
All x_k are one.
So a constant input.
Natural thing to test.
That's the constant input, that's zero frequency.
Zero frequency.
What's the output?
From all ones going in wow, sorry to ask you such a trivial question.
You came in for some good math here, and I'm just taking 1/2 and 1/2.
So the output is all y's equal, right?
So, to me, that's telling, just to introduce an appropriate word, low frequencies; in fact, bottom frequencies, zero frequency, is passed straight through.
That's a low pass filter.
That's telling me I have a low pass filter.
So that's an expression.
That's so simple that you might as well know those words.
Low pass means that the lowest frequencies pass through virtually unchanged.
In this case, the very zero frequency the DC term, the constant term, passes through completely unchanged.
Now, what about another input all - well, now I want high frequencies.
Top frequencies.
What's the the highest oscillation I can get would be x equal, say, it starts one minus one, one, minus one, so on.
Both directions.
Oscillating as fast as possible.
I couldn't get a faster frequency of oscillation in a discrete signal than up, down, up, down.
What's the output for that?
So that's really oscillation.
That's the fastest oscillation.
What would be the output from my averaging filter for this input?
Zero.
At every step, I'm averaging this with the guy before and they add to zero.
I'm averaging this with the guy before, this with the guy - output is, y equals all zeroes.
OK, so that confirms in my mind that I have a low pass filter.
The high frequencies are getting wiped out.
OK, so that's two examples.
Now, what about frequencies in between?
Because ultimately we want to see what's happening to frequencies in between.
OK, so what's an in-between frequency?
So in between x_k could be e^(ikn), let's say.
e^(ik*omega).
e^(ik*omega).
Where this omega is somewhere between minus pi and pi.
OK, why do I say minus pi and pi?
If the frequency - so that's the frequency.
If omega is zero, what's my signal?
All ones, right?
If omega is zero, everything is my all ones.
This is this k. So I now have a letter for it, omega=0.
What's this top frequency?
One, minus one, one, minus one, what omega will give me alternating signs?
Omega equal?
Pi, right?
Omega=pi.
Because if this is pi, I have e^(i*pi), which is minus one.
So when omega=pi, my inputs are e^(i*omega), to the k'th power.
But this is minus one.
e^(i*pi), to the k'th power, and that's minus one.
So that's the top frequency.
And also the bottom frequency is pi.
And the zero frequency is the all one.
And this is what happens - ah.
Now, comes the point.
What's the output if this is the input?
What's the output when this is the input?
We can easily figure that out.
We can take that average.
OK, so let me do that input.
Input x_k is e^(ik*omega).
And now what's the output?
y_k is the average of that.
And the one before.
Divided by two, right?
OK, now you're certainly going to factor out, anybody who sees this is going to factor out e^(ik*omega), right?
I mean that's sitting there, that's the whole point of these exponentials is they factor out of all linear stuff.
So if I factor that out, I get a very very important thing.
I get, well, it's over two, I get a one.
And I get, what's this term?
e^(ik*omega) is here.
So I only want e^(-i*omega).
OK, that is called the frequency response.
So that's telling me the response of what the filter does to frequency omega.
It multiplies the signal.
If I have a signal that's purely with frequency omega, that signal is getting multiplied by that response factor.
1+e^(i*omega).
When omega is zero, what is this quantity?
So let me call this cap h(omega).
what.
Is this factor, if omega is zero?
Then h(omega)=0 is?
One.
That's telling me again that at zero frequency the output is the same as the input.
Multiplied by one.
And at omega equal to pi, what is this frequency response?
Zero, right.
At omega=pi, this is minus one so I get zero.
And it's telling me again that this is the response.
And now it's also telling me what the response factor is for the frequencies in between.
And everybody would draw a graph of the darn thing, right?
So this was simple, let me do its graph over here.
So I'm going to graph h(omega).
Well, I've a little problem.
h(omega)'s a complex number.
I'll graph the magnitude response.
So here I'm going to do a graph from minus pi to pi.
This is the picture.
This is the picture people look at.
This is the picture of what the filter is doing.
All the information about the filter is in here.
All the information is in there.
So if I graph that, I know what the filter's doing.
So you said at omega=0, I get a value of one.
At omega=pi, I get a value of zero.
At omega equal minus pi, I get a value of zero.
And I think if you figure out the magnitude, it's just a cosine.
It's just an arc of a cosine.
OK, for that really, really simple filter.
So any engineer, any signal processing person, looks at this graph of h(omega) and says that is a very fuzzy filter.
A good, an ideal filter, an ideal low pass filter, would do something like this.
An ideal filter would stay at one up to some frequency - say, pi/2.
And drop instantly to zero.
There is a really good filter.
I mean, people would pay money for that filter.
Because what happens when you send a signal through that ideal filter?
It completely wipes out the top frequencies.
Let's say, up after pi/2.
And it completely saves the in-between ones.
So that's really a sharp filter.
Actually, what people would like to do would be to have that filter available.
And then also to have a perfect ideal high pass filter.
What would be an ideal high pass filter?
Yeah, let's talk about high pass filters just a moment.
Because this is, you're seeing the reality of what people do, and how they - and that little easy bit of math they do.
Do you want to suggest a high pass filter, let me come back to this?
And just change it a little?
So I plan to do not - I'm now going to do a different filter.
That's going to be a high pass filter.
And what do I mean by that?
A high pass filter will kill the x_k=1.
I now want the output from - this is now going to be, I'm going to change.
Can I just erase, change a lot of things?
I'm now going to produce a high pass filter.
Sorry, pi.
And what's the difference?
When all x's are one, the output is going to be?
Zero.
And when I have the highest frequency, the output is going to be?
The input.
And what am I - and then in between, I'll do something in between.
OK, what do you think would be a high pass filter, like the simplest high pass filter we can think of?
Anybody think of it?
You're only getting, like, 15 seconds to think in this class.
That's a small drawback, 15 seconds.
But, the high pass filter that I think of first is, take the difference.
Take the difference.
Put minus 1/2s on the sub-diagonal.
This is the same, this is also a convolution, but now what h_0 is still a half.
But now h_1 is?
Minus 1/2.
We're still convolving.
We're still convolving with - it's still linear time invariant, that just means it's a convolution.
It's still a finite impulse response.
But the response, the impulse response is now 1/2 minus 1/2.
So what happens if I, in my picture over here, if I send in any pure frequency, I'm now doing minus 1/2 here.
So I'll just keep the plus.
But I'll also add in the minus.
So now I'm looking at 1-e^(-1*omega/2).
And again, let's plot a few points for that guy.
So what, at x at omega - so this is omega in this direction.
And this is h in this direction.
So at omega=0, what's my high pass guy?
When I send in a zero frequency, constant, I get what output?
Zeroes, because now - I I'll call it a differencing filter.
So I'll just, instead of averaging I'm differencing.
OK, so now for this one, maybe I'll put an x to indicate I'm now doing, I'll do x's for the high pass.
So this now, the high pass guy, kills the low frequency and preserves the high frequency.
And you won't be surprised to find it's some cosine or something that, well yeah, it's got so sorry that's not much of a cosine.
It's the mirror image of the low pass guy.
And maybe the sum of squares adds to one or two or something.
One, probably.
The sum of squares probably adds to one.
And they're kind of complementary filters.
But they're very poor.
Very crude, I mean that's so far from the ideal filter.
So how would we create a closer to ideal filter?
Well, we need more h's.
With two h's, we're doing the best we can, with just h_0 and h_1.
With a longer filter, for which we're going to have to pay a little more to use, but we'll get a lot more.
We'll get something, we could get a filter that stays pretty close to this, drops pretty fast.
There's a whole world.
Bell Labs had a little team of filter experts.
Creating, and now MATLAB will create it for you, The coefficients, h, that would give you a response, a frequency response, it'll stay up toward, up close to one as long as possible.
And drop as fast as possible, and bounce around there.
So next week, if I come back to that topic, I can say a little more about these really good filters.
What was I trying to do today?
Trying to see how convolution is used.
And this is a use you will really make.
So now I just have, I think, about two more things to say about this example.
Let's see, what are they?
Well, first, so all the information is in this h(omega).
Oh, yeah.
This simple example gives us a way to visualize convolution.
And I think we need that.
Right?
Because up to now, convolution has been a formula.
Right?
It's been this formula.
That's the formula for convolution, and how do I visualize that?
Just think of, may I try to visualize that?
Here I have, this is the time line.
The different k's.
k equals zero, one, two, minus one.
And I have x_-1, x_0, x_1, x_2, x_3.
So that would be a little bouncy up and down.
And the averaging filter, let me go back to the averaging one.
The averaging filter would smooth out the bumps.
Because it would take the, like, average neighbors.
And that's a smoothing process.
As we see here, it's a process that kills high frequency.
Now, what is this visualization I want you to think of?
I want you to just think of, like, a moving window.
So here is the input.
Now, I move a window along.
And that window, so let's say here's the window, I should have another.
So that's the window.
When the window is there, it takes the average of those two.
That gives me the new output.
Now, think of the window as moving along here, taking the average of these.
Move the window along, take the average of these.
Move the window along.
Do you see the sort of, this is what a convolution is doing.
This is a picture of my formula, sum of h_k*x_(l-k).
So the window is the h's, is the width of the h's, and as that window moves along.
I mean, you could write, you could create, design a little circuit that would do exactly this.
That would do the convolution.
You just have to put together some multipliers, because you have these h's, these, like, halves.
And you have to put in an adder, that'll add the pieces.
And those are the essential little electronic pieces of an actual filter.
Then you just move it along.
So it needs a delay.
That's about the content of a filter.
Is multipliers that will multiply by the h's, so in come the x, multiply by the h's, do the addition, and do a shift to get onto the next one.
You see how a filter works?
I think that image, or convoluted is a little bit vague, maybe?
This window moving along?
But it's quite meaningful.
And then the final thing I'll say about filters is this.
That, what's the connection between h(omega) and h(k)?
Or h_k, let me call it h_k?
What's the connection between the numbers in impulse response?
Just, which were the h's, and the function, which is the frequency response?
Which tells me what happens to a particular frequency?
Each frequency, e to the - you notice how the frequency that went in is the frequency that comes out.
It's just amplified or diminished by this h(omega) factor.
So you see the h's are the coefficients here of h(omega).
In other words, h(omega) is the sum of the h_k's, e to the, e^-ik.
Here's the beautiful formula.
That's obvious, right?
Here you're seeing the formula in the simplest case, with just an h_0 and an h_1.
But of course, it would have worked if I had several h's.
So this h(omega), this factor that comes out, is just this guy.
Now, if I look at that, what am I saying?
I've seen things that connect a function of omega with a number of filter coefficients.
I saw that in Section 4.1, in Fourier series.
This is the Fourier series for that function.
Right?
You might say, OK, why that minus?
I say, it's there because the electrical engineers put it there.
They liked it.
And the rest of the world has to live with it.
So, you notice I don't concede on i. I refuse to write j.
But they all would.
I speak about they, but probably some you would write j.
So I'm hoping it's OK if I write i. i is for imaginary.
I don't see how you could say the word imaginary starting with a j.
And what was the matter with i, anyway?
Current.
Well, current used to be i. I mean who, is it still?
Well, let's just accept it.
OK, they can call the current i, and the square root of minus one j, but not in 18.085.
So OK, here we are.
So my point is just that we have a Fourier series.
Here we a 2pi periodic function.
Here we have its Fourier coefficients.
The only difference is that we started with the coefficients.
And created the function.
But otherwise, we're back to Section 4.1, Fourier series.
But that fact that we started with the coefficients and built the function, sometimes you could say, OK that sounds a little different from the regular Fourier series, where you go the other way.
So people give it the name discrete time Fourier transform.
You might see those letters sometime.
The discrete time Fourier transform goes from the coefficients to the function.
Where the standard Fourier series starts with a function, goes to the coefficients.
But really, it doesn't matter.
The point is, yeah.
So you could say, maybe we have now a force transform.
Like the first transform was Fourier series.
The second one was the discrete.
The third one is the Fourier integral that's coming in one minute.
And the fourth is this one.
But hey, it's just the coefficients and the function have switched places.
In the, which one is the start and which one is at the end.
OK, let me pause a minute because that's everything I wanted to say about simple filters.
And you can see that this is a very simple filter, and could be improved.
Better numbers would give, I mean what would be better numbers?
I suppose that 1/4, 1/2, 1/4 would probably be better.
If I took those numbers, I'm pretty sure that this thing would be closer to ideal by quite a bit.
If I plotted - so what do I mean by, those are the h's.
So I would take 1/4 plus 1/2 e^(-i*omega).
Plus 1/4 e^(-2i*omega).
This would be my better h(omega).
This would be my frequency response to a better averaging filter, sort of this is like averaged averaged, right?
If I do a half an average and then I do the average again.
In other words, if I just send these signals y_k through that same averaging filter, so average again to get a z_k.
I think probably the coefficients would be 1/4, 1/2, 1/4, and it would be, I've taken out more noise.
Right?
Each time I did that averaging, I damp the high frequencies, so if I do it twice I get more damping.
But I lose signal, of course.
I mean, presumably there's some information in the signal in these frequencies.
And I'm reducing it.
And if I average twice I'm reducing it further.
So a better one would be to get a sharp cutoff.
OK, that's filters.
I guess what I hope is that we have the idea of a convolution, and now we see what we can use it for.
Right?
And there are many others.
So we'll have another.
We'll come back to convolutions and de-convolution.
Because if you have a CT scanner, that doing a little convolution.
I mean, you're the input, right, to the CT scanner?
You march in, hoping for the best.
OK, CT scanner convolves you with their little filter.
And then it does a deconvolution, to an approximate deconvolution, to have a better image of you.
OK, let's leave that.
Can I change direction and just write down the formulas for the Fourier integral transform?
And do one example?
OK.
I don't know what you think about a lecture that stops and starts a new topic.
Is it - maybe it's tough on the listener?
Or maybe it's a break.
I don't know.
Let's look at it positively.
Alright, break.
Alright.
So let me remember the Fourier series formulas.
So I'm just going to break, and now we go to the integral transform.
OK, so let me remember the formula for the coefficients, which was 1/2pi.
The integral of f(x)e^(-ikx)dx, right?
And then when we added it up to get f(x) back again, we added up a sum of the c_k's e^(ikx)'s.
That's 4.1.
We know those formulas.
And we notice again.
Complex conjugate.
One direction is the conjugate compared to the other direction.
Now, all I plan to do is write down the formula.
And remember, I'm going to use f hat of k, instead of the coefficients.
Because it's a function of k, all k's and not just integers are allowed.
And then I'm going to recover f(x).
OK, now this integral went from minus pi to pi, because that was periodic.
But now all the integrals are going to go from minus infinity to infinity.
We've got every k, every x.
So we take, what do you expect here?
f(x)?
e^(-ikx)?
dx?
Yes.
Fine.
Same thing, f(x) is there, but now any k is allowed so I have a function of all k's.
And now I want to recover f(x).
So what do I do?
You can guess.
I've got an integral now.
Not a sum.
Because a sum was when I had only integer numbers.
Now, I've got f hat of k, and would you like to tell me what the magic factor is, there in the integral formula?
It's just what you hope.
It's the e^(ik*omega).
Where d what?
Now this is where it's easy to make a mistake.
d, I'm integrating here.
I'm putting the whole, reconstructing the function.
I'm putting back the harmonics with the amount f hat of k tells me how much e^(ik*omega) there is in the function.
I put them all together, so I integrate dk.
I'm integrating over the frequencies.
This was the sum over k. From minus infinity to infinity.
Now this is an integral, because we've got, it's all filled in.
And it remains to deal with this 2pi.
And I see in the book that the 2pi went there.
I don't know why.
Anyway, there it is.
So let's follow that convention.
Put the 2pi here.
So there's the formula.
The pair of formulas, the twin formulas.
The transform, from f to f hat, and the inverse transform, from f hat back to f. And it's just like the one you've seen for Fourier series.
Well, I think the only good way to remember those is to put in a function and find its transform.
So my final thing for today would be take a particular function, f(x).
Here, let me take ever f(x) to be, here's one.
f to be zero here, and then a jump to one.
And then an exponential decay.
So e^-ax.
OK, so that's the input.
It's not odd, it's not even.
So I expect sort of a complex f hat of k, which I can compute.
So f hat of k is what?
Now, let's just figure out f hat of k and look at the decay rate and all the other good stuff.
So what do I do?
I'm just doing this integral.
For practice.
OK, so f is zero in the first half.
So I really only integrate zero to infinity.
And in that region it's e^-ax, and I multiply by e^(-ikx), and I integrate dx, and what do I get?
I'll get this, is an integral we can do, and it's easy because this is e^-(a+ik)x.
You're always going to see it that way, right?
That we're integrating.
And then the integral of an exponential is the exponential divided by the factor that will come down when we take the derivative.
So I think we just have this, right?
Don't you think, to integrate that exponential, we just get the exponential divided by its little factor.
And now we have to stick in the limits.
And what do I get at the limits?
This is like, a fun part of Fourier integral formulas.
What do I get at the upper limit, x equal infinity?
If x is very large, what does this thing do?
Goes to zero.
It's gone.
The e^(ikx) is oscillating around, it's of size one.
But the e^(-ax), so I needed a to be positive here.
That picture had to be the right one.
A positive.
Then at infinity, I get zero.
So now I just plug in this lower limit, that comes with a minus sign.
So what do I get?
The minus sign will make this a+ik, and what does it thing equal at x=0?
One.
e^0 is one.
So there is the Fourier transform.
Of my one-sided exponential.
Now, just a quick look at that and then I'll do some more, this example's Example one in Section 4.5, and we'll do more examples.
But let's just look at that one.
I see a jump in the function.
What do I expect in the decay rate of the transform?
So a jump in the function, I expect a decay rate of 1/k.
Decay rate, right?
Just as for Fourier coefficients, so for the integral transform.
So a decay rate in f hat.
And it's here.
1/k in the denominator.
Yeah.
So that's a a good example.
You might say, wait a minute, OK that's fine but what about the second one?
Could I put in 1/(a+ik) and get back the pulse?
The exponential pulse?
The answer is yes, but maybe I don't know how to do that integral.
So I'm sort of fortunate that these formulas are proved for any function including this function.
So this example shows the decay rate.
The possibility sometimes of doing one integral but maybe the other integral going the other direction is not so easy.
And that's normal.
So that's, like, Fourier transforms and inverse transforms, we don't expect to be able to do them all by hand.
I mean, I'll just say that actually, anybody who studies complex variables and residues, I don't know if you know any of this, heard these words about, there are ways to integrate.
I could put 1/(a+ik) in here.
And, actually, and do this integral for minus infinity to infinity.
By stuff that's in Chapter 5.
Can I just point ahead without any plan to discuss it.
That some integrals can be done by x+iy tricks, by using complex numbers.
But I won't do more.
OK, thanks.
So that's the formulas.
And that's one example.
Wednesday there will be more examples and then no review session Wednesday evening.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, thank you for coming today.
The day before Thanksgiving.
Day before my birthday, actually.
So it's a special day.
Everybody gets an A for showing up.
Even you.
So, let's see.
Last time, I wrote down these formulas for the Fourier integral transform.
And I thought I'd just write them again so you kind of photograph them and remember them.
They're easy to remember.
As always, you take the function, you multiply by e^(-ikx), and you integrate.
To get the amount - so k is my frequency variable.
It could well have been omega or some other variable.
I stayed with k because it was k in the Fourier series.
So that's the calculation which as always, I mean, these are integrals that we may be able to do if the function is especially nice, or we may not.
But that's the formula.
And then to reconstruct the function, we combine all they e^(ikx)'s in that amount to get f(x) back.
OK, nice formula.
So I did one example last time, and now could I just double it up?
This is also in the textbook and so this is now going to be an even function.
Last time the example I did was zero, up to x=0.
This time I'll make it symmetric, make the function even.
And then I have two pieces.
In the integral.
And if you remember what it was, you remember that this, I'll just remind you what we did.
We wrote that it's e^-(a+ik)x.
That was clear.
And then when we integrated we got that same function divided by -(a+ik).
a And then we put in the limits.
And the answer was, let me maybe write the answer down here.
Was just at x equal infinity the limit was zero because this thing is tailing off.
At x=0 this is one.
It comes in with a minus because that's the lower limit.
So it was 1/(a+ik) for the first half.
And we were not surprised to see this 1/k because the first half all by itself has that jump, from zero to one.
So we see that jump reflected in slow decay.
Alright, but now I'm making it even.
What are you going to guess for the rate of decay of f hat of k for this function?
This function no longer has a jump.
But it does have - I don't know were we saying ramp, or corner?
This is not a smooth point here, because the derivative going up is plus a, so I'll just put a circle right, the derivative is a e^(ax), and at x=0 that would be plus a going up.
And here the derivative is minus a, e^(-ax).
Put in x=0, and the derivative coming down is minus x.
So there's a jump in the derivative.
So what, just before we see it, what will you expect for the rate of decay of the transform?
1/k to what power, now?
So it didn't have a jump, a jump was 1/k.
This has a jump in the derivative, so we're expecting 1/k squared.
k squared, it'll be one order smoother.
OK, you can easily see that happen.
Because this part, well this is just e^(a-ik)x, which I'm going to integrate to get this thing over a-ik.
And I'm going to plug in the limits minus infinity and zero.
And at minus infinity I'll get nothing, this e^(ax), that minus infinity will be zero.
Where the function starts.
Way down at zero.
So and at x=0, this is a one.
So I just get 1/(a+ik).
Over a minus, thank you.
Right, over a minus.
And somehow it can't be an accident that this is the complex conjugate of that somehow.
That's not a surprise.
OK, so let's put those together into a single fraction and see what we have.
So the denominator of that fraction will be this times this.
And that's the most basic multiplication of complex numbers.
That one times its conjugate gives me what?
It gives me an a squared.
And what else?
a+k squared because i times minus i is plus k squared, and no imaginary part.
There's a plus i k a and a minus i k a, all we're seeing here is the sum of squares.
The usual z times z bar.
And in the numerator, let's see.
When I put it over this, so this was putting it over this common denominator.
So I should have an a-ik going up on top there.
And an a+ik going up on top here.
Right?
Those are my two fractions.
That over this, and that over this.
And now that numerator simplifies, oh look it's great.
I'm getting a real answer.
And because the minus ik and the plus ik cancel, and it's just the two way.
And probably no surprise that somehow that that's the jump in slope.
That must have something to do with that 2a.
So we got a real even, Fourier f hat from my real even f. And it decays like k squared.
OK, so that's another good example.
A very useful example.
Right.
I could add other examples.
One quite, before I use that in application, let's do just a few more examples.
Suppose f(x) is the delta function.
What's f hat of k?
Can you just plug in f(x)=delta(x) here?
Do that integration, and what does f hat of k come out to be?
One.
Because if it's a delta function in there at x, at x=0 the spike is at x=0, so I plug in x=0, I get one.
So we're kind of, we'd be surprised if it wasn't a constant, right?
A delta function in physical space goes in, has all frequencies in equal amounts.
And it's a constant in frequency space.
Then there's one more that takes a little trick to do, but it's a very neat one.
f(x) is e to the minus x squared over two.
Do you recognize that as an important function, e to the minus x squared or it usually has that e to the minus x squared over two or sometimes an e to the minus x squared over two sigma squared, a rescaling?
But this would be the bell shaped curve.
The bell shaped curve.
It decays, very quickly the variance, because that's a two and not a two sigma squared, the standard deviation is one here.
The variance is one.
So it's a bell shaped curve that has about 2/3 of its area between minus one and one.
This is the all important function for probability.
The normal distribution, the Gaussian, both of those words are used, it's the most important probability distribution.
I need a one over square root of 2pi to make the total probability be one.
But let me just leave it there.
That's a very, very important function.
It's also going to be important in the heat equation.
In math finance, shows up all over the place.
And its integral would not be easy to do from zero to one.
The integral of that function, from zero to one, we have tables of it.
To the nth place.
But so there's no simple, elementary function whose derivative is this.
That x squared is what's making the integral tricky.
So from zero to one, we just have to give it a name.
So, error function.
This would be ERF, error function, the integral of that thing correctly normalized.
I'm just saying, important, important function.
And it turns out that integrals from minus infinity to infinity can be done so beautifully by some trickery.
We can find the transform of this.
We can find the transform of this, we can do this integral from minus infinity to infinity, where we could not do it from zero to one.
So I'll just write down the answer for this guy.
Only because it's such a key example.
It's some constant that involves 2pi times e to the minus k squared over two.
Boy, that's pretty amazing.
Right, the Fourier integral transform, f hat of k, has the same form as the function.
And of course this function is infinitely smooth.
So its transform to k is infinitely fast.
Yeah, there's no problems like one over k squared here, there are no bumps in the and bell shaped curve.
So I won't push that example except I'll use it.
What else should I say just to, like, emphasize that this is such an important distribution in probability?
Why is it important in probability?
That's the question.
Why does everybody assume if you can get away with it and don't have any natural alternative, everybody assumes that noise, whatever, is coming with a normal distribution.
So, in other words, with a sigma squared in there.
So a normal distribution, that has mean zero because it's absolutely centered at the origin and it has variance one, but I could change the variance and that would just spread out or tighten the bell shaped curve.
Why is the bell shaped curve so important?
That's certainly, we're not going to launch into theory of probability but it's the central limit theorem.
So let me just use those words.
The central limit theorem that says that if I start with other probability distributions, like I'm flipping a coin.
I flip a coin a million times.
Then the mean, and let's say zero for tails, one for heads.
OK, so I flip, flip, flip.
Well, the mean of that, the expected mean is what, half a million, right?
Half tails, half heads.
So if I give zero for tails, one for heads and flip a million times the mean would be about half a million.
And then, so let me center the mean.
I could have centered it by taking minus one and one.
That would have been smarter.
Minus one and one.
Minus one for tails, one for heads would have had a mean of zero.
And then it would be natural if I have a million of these to divide by a thousand, I think.
Of course, the answer won't be zero, right?
If I do a million flips it's not going to come out exactly half a million and half a million.
I'm remembering.
I used to have a long discussion with a nice guy in college.
He ran for Mayor of Boston, actually.
But he had the idea that after a million flips, suppose there had been more heads than tails.
Then the next flip, he figured, was more likely to be tails.
I couldn't convince them that this was not mathematically the right thing to think about.
And all I did was say don't go to Las Vegas.
I mean, if you're thinking that way save your money.
So, anyway.
But this is much studied.
The variation what that curve looks like, that's quite interesting.
But my point is, that as the number gets bigger and bigger and we scale it properly, the distribution will approach the norm.
All sorts of distributions.
If I just repeat and repeat experiments and scale it, the central limit theorem says you're always going to the normal distribution.
So that's highly important.
OK, and it comes up different places.
And it's quite a neat function.
OK, so that's some examples.
Now, let me use, like every topics that I introduce, I want to find a use for.
So now, can I start on this one?
Constant coefficient differential equations.
I'm going to write down a differential equation, which will look pretty much like the ones we started this course with.
And I could, well, let me write it down.
Minus d second u/dx squared, we're used to that.
Now let me put in an a^2*u.
Which is a lower lower order term, we could deal with that.
Equals sum f(x).
And now, because I want to do Fourier integrals, I'm thinking all x.
We're on the whole line.
Instead of the interval (0,1) where I might use Fourier series and have sine series or cosine series, depending on the boundary conditions.
Here, the boundary condition is just everything drops off at infinity.
And minus infinity.
So all these functions we can do these integrals.
OK, so there's a good question.
What's the solution?
We could tackle it, but I want to suggest to use Fourier.
So it's not the only way, but it's one way to see it.
So now if I use, what do I mean by using Fourier?
It means I'm going to take the Fourier integral transform of every term.
So when I take the Fourier transform of the right-hand side, I'm going to get f hat of k, whatever.
This is known, of course.
This guy is given.
That's the source term.
And u is the unknown.
OK, so I'm going to take the Fourier transform of every term, well this is, a is a constant.
a had to be a constant, or I couldn't do, you know if a depended on x this would be some multiplication and the transform would be a mess.
Fourier applies when you've got constant coefficients and nice boundary conditions.
And here our boundary conditions are nice, they just go to zero fast.
OK, so the transform of this is, that's a constant.
u hat of k. And what's the transform of that?
So this is our chance to use probably the most important rule for Fourier integrals.
Maybe you'll tell me what it is.
You should think what it is.
If I take a derivative, that's the rule.
If I take a derivative of the function, what's happening in frequencies?
I could make that happen here.
If I took the derivative, yeah.
So maybe if I take the derivative here, so here it's just remembering the rule.
Suppose I take the derivative of this equation.
I get this integral, and what would f' be?
What would f hat, sorry.
If I take the x derivative of this, if I take the x derivative of this equation, what happens on the right-hand side when I take the x derivative?
Down comes i k. Down comes ik.
So when ik is coming down, I won't even finish that equation.
And ik is coming down, when I take the derivative.
So the derivative, the transform, is multiplied by ik, higher frequencies are emphasized now because of that k factor.
And now if I take two derivatives, I bring ik twice.
Because i squared k squared, the i squared and the minus give me a plus.
So that's just k squared.
u hat, of k. OK with that?
And now, we get an immediate formula for u hat of k, the solution.
Well, it's the solution but it's in frequency space.
If we wanted to know it in x space, as we do, we've got to transform back.
But what do we get here?
It's just f hat of k. Divided by, this is just multiplied by a squared plus k squared.
OK, so that's the answer.
In frequency space.
That was simple.
And then if I wanted it in x space, I take the reverse transform.
Notice that this is, it here are the same three steps that I emphasize all the time about using eigenvectors and eigenvalues.
Do you remember those three steps for solving differential equations?
Difference equations, linear equations, whatever?
The three steps were, find that coefficients, expand everything in eigenfunctions.
I won't write, I'll talk.
The three steps were expand in eigenfunctions, follow each eigenfunction function separately, that was the trivial step with just a division like this division.
And then use those coefficients of the eigenfunctions, combine them all back to get the answer.
Right?
Step one, write it in the right basis, step two easy in that basis.
Step three go back to your physical space.
We're doing exactly the same thing here.
These e^(ikx)'s are the eigenfunctions of this thing.
They're the eigenfunctions.
And here the eigenvalue of this stuff is k squared plus a squared.
And that's what we divided by.
And then the final job of going back from u hat to u, so write u there.
Can I do, this f now, it's really u that I'm wanting to bring back to physical space.
So just for the sake of your -- I see -- it, let me put a u in.
So that's the answer in a way.
It's the answer, it's a formula for the answer.
It did depend on our being able to do two integrals.
By the integral from f to f hat may not have been easy, and then the integral from u hat back to u, this integral, might not have been easy.
So it's a formula.
OK, now I want to go with it a little longer.
Because I want to show you how the delta function pays off.
So let me do the example where f(x) is the delta function.
So now we're really close to where this course began.
Differential equation with a delta function.
The only new thing is, we're not on an interval we're on the whole line.
So I take transforms, so what's the transform now of this specific f(x) is?
One.
We just saw.
OK, so now we get a one there.
Now we just divide by here.
And we've got a one here.
So we were able to go, this was an integral we could easily do, to get from delta to delta hat, which was just one.
And fantastically, this is an integral to go back to u(x), to go back to u(x), that's an integral we can do.
Well, you may ask how can we do it.
How do I find the u(x) that has this transform?
Well, I either use complex variables to do integrals like this, residue methods that are in Chapter 5, or I look in a table.
Or I look at the blackboard over there.
I think that's the best way.
Look at this blackboard.
Right?
Because this is the answer we got.
We got that same answer apart from a constant factor 2a.
So this is our function.
This is our solution, u(x) is this.
What am I going to call that?
Two-sided pulse?
I'll call that the two-sided pulse?
Maybe I should give it a name but I'll just write out those words two-sided pulse.
That's it divided by 2a.
So we've got the answer.
Let me just make a little more space here.
This was one over a squared plus k squared, and now having seen that already, I just say yep, that must be it.
It's the two-sided pulse, and I have to divide by 2a.
Do you see that that's the correct answer?
We can substitute that in the equation and see that it works.
I mean, so we have solved the problem.
We have solved the problem when the right side was delta.
Let's put it into the equation.
So this is just because we did this, it's nice to do it again after all this time.
So I put it in the equation.
This two-sided pulse over 2a.
So what's my equation?
Well, this is zero most of the time.
So I believe that if I plug in this function, it will give me the zero.
Do you want to just plug it in?
I believe that if I plug in a^(-x), or a^x, either one, can I just check that you try u=e^(-ax).
Put it in and just see that I get zero.
Because yeah, two derivatives bring down a squared with a minus.
And there's a squared with a plus.
It works, right?
Two derivatives of this function bring down minus a twice, so that's a squared.
So it's minus a squared, plus a squared.
Works.
And then, of course, the important point is x=0 where the spike is.
What happens at the spike, going back to the beginning of the course?
This term is going to be unimportant compared to this term.
What do I see?
With -u'' equal a spike, what was the solution to that?
u had a corner, right?
The slope of u, what did the slope of u do?
It dropped by one, was that right?
The slope of u dropped by one.
We used to have corners going up and down and the difference between the slopes was one.
And here, the difference between the slopes, ah, look.
The slope has dropped by 2a, and when we divided by the 2a, it was just right.
And now, when I divide by the 2a, this has a slope of 1/2.
This has a slope of minus 1/2, the drop is one.
And we're right.
It solves the equation.
Nobody doubted that.
OK, so that's great.
We have found the solution to this equation, when the right side is delta.
Good.
Now, can I ask you do you remember the name?
There's a special name for the solution when the right side is a delta function.
Whose name is associated with that?
So that this particular u, I'm going to give it another letter.
It's the particular u, the special u when the right side is delta, and whose name is associated with that solution?
Green.
It's the Green's function.
Green's function.
The famous Green's function.
Green's function is just like an inverse to the problem.
This is like having an identity on the right-hand side.
It's like there it is.
So let me just use G for Green's function.
So that's the Fourier transform of the Green's function, and this is the Green's function.
This is the Green's function.
Now I can give it its name, Green's function, when I divide by the 2a.
And now the slope is 1/2 going up, and minus 1/2 coming down.
And it's all right.
OK, so we found the Green's function.
We found the fundamental solution to the equation, and this is it.
It was straight lines, right?
It was straight lines in the first weeks of the course.
But now there's an exponential drop-off caused by this additional term.
OK, good.
So that's straightforward.
Depending on our being able to recognize or do the transform back to the x space.
Now comes the question, what about the original f(x)?
How can the Green's function be used?
So you're seeing now what use is this Green's function?
With that right-hand side, when the right-hand side is something different?
When the right-hand side is some different f(x)?
So let me go back to an f(x) on the right.
And then there's an f hat of k, after the transform.
How can I use the Green's function for a general source?
The general source term, a general load?
This is a fundamental idea.
I would say fundamental.
How do you use the Green's function?
And remember, the Green's function is like telling you the inverse matrix.
So it can't be too hard.
It's like solving a linear system when you know the inverse matrix.
So that's the analogy, but let's just focus on the particular question.
I think the intuition, you should have an intuition for how the Green's function works.
So the Green's function was the solution when the source term was a delta.
And here's the intuition.
It's rough, but it works.
Any source term, f(x), is in some way a combination of delta.
If f(x) is a combination of deltas, then our answer u(x) is the same combination of the Green's function, right?
If the right-hand side is some combination of this special delta, then the solution will be the same.
This is just linearity.
Super position, whatever short or long word you like to use.
So if I can make sense of that statement, that f(x) is a combination of deltas, then I'm in.
Now, what do I mean by a combination of deltas?
I mean, well, those deltas are going to be shifted deltas.
Obviously the single delta, delta(x), is a spike at the origin.
That's only one point.
I want to combine delta of x and its shifts.
So I'm going to have to expecting to be using G(x), and its shifts, right?
OK so now I'll just say this again.
I'm thinking of f(x) as a combination of delta and its shifts, and then the solution u will be the same combination of G(x) and its shifts.
So now you just have to tell me what combination.
What combination of delta and its shifts?
Maybe you'll allow me.
Let me just do this maybe on that board.
I just can't help writing down the discrete case.
So, in the discrete case, the delta vector corresponds to something like .
Right?
That was a typical delta vector with a one in the zeroth position.
Then its shifts would be <0, 1, 0, 0>, that'd be a shift.
And another shift would be .
And another shift would be .
So now there is the delta vector and its shifts.
These four guys.
OK, now suppose my f, my right-hand side, is .
I want to write that as a combination of those deltas.
This would be in the case when if I know the solution for each of these guys, the Green's function, the inverse matrix.
Everybody sees that if I know the solution to those four, I know the inverse matrix.
Right?
Because if I can solve with those four right-hand sides, those four solutions are the columns of the inverse matrix.
Right?
You remember that if I had a matrix A and I was looking for its inverse, I solve A A inverse equal I.
And I is just these four guys.
So a inverse is the solution from these four guys.
OK, now everybody's going to tell me what's the solution for this right-hand side ?
Suppose this guy has solution, so a inverse, the columns of a inverse are this Green's function.
This Green's function with a shift.
Maybe SG, Green's function with a shift.
Yeah, S squared and G, Green's function with a double shift.
S cubed G; I'm just cooking up, I never used these letters before.
But what's the answer?
Then u is what?
It's one times the Green's function.
And it's two times the guy, right?
This f is just, this f is just, is one times the Green's function, two times the shifted.
Three times the double shifted, and seven times the triple shift.
Right?
Just taking four minutes to do something simple because over there, when I use the continuous case it'll look a little strange, but here it's so easy.
That'll involve integrals, this involves a sum.
So what is it?
I have G, twice the shift of G. Three times the double shift of G, and seven times the triple shift of G. Right?
By linearity, by superposition, if this is my f, this is my u.
Everybody's with me here, right?
That if I take a combination of f's, the answer is the corresponding combination of G's.
OK, good.
I didn't mean that.
That wasn't so great.
That shouldn't have been called, I didn't mean to call those the Green's functions.
I meant to call those the deltas, right?
And the Green's functions were the answers.
So this a shift to delta, this is a doubly shift to delta, this is a triply shift to delta, and now this is one of the delta, two of the shifted delta, three of the doubly shifted delta.
The f is a combination of deltas and its shifts.
The u is a combination of Green's function, second shift.
Apologies for making that mistake, but maybe it's brought us back to the point.
So the point is, express your function f, your source as a combination of deltas, just exactly our plan.
Then u is the same combination of the G's with the same shift.
Alright, now back here.
How do I express f(x), in what way is f(x) a combination of deltas?
I mean, slow down just to see that point.
How much of delta, of the spike at x=3, how much of delta(x-3), so that's a spike at three, right?
How much of that do you think I need in f(x)?
f of?
f(3).
Whatever f is at that point, x=3, that's the amount I need there.
So this would be f(3).
So that that part would sort of have the right pep, the right punch at the point three.
Now, three could be any point, that's any point along the line.
Let me, I can't use x for other points on the line because I've got an x in the formula.
Let me use t. So this was t equal to three.
Now I want to do it at all points.
I want to take f(2), and f(pi), and f at everything and put them together.
And of course putting them together in the continuous case means not sum, as I did there, but integrate.
So I have to, now I'm going to change three to a t, because this is the amount of f of, so that's the amount, that's the delta functions which spike at t, multiplied by f(t), and now how do I get f(x) out of this?
I put them together.
dt.
I add up, this is the combination I've been talking about.
So this is like any f. This is like a crazy delta function identity.
Actually, it's not crazy, it's the identity we've used our whole lives.
Or at least our whole 18.085 lives, which is all that matters, right?
OK, so I'm integrating a delta function.
And I want to see, do I get that answer?
And you're going to say absolutely, clearly you get that answer.
No.
Everybody knows that if I integrate something times the delta function that I plug in at the point t=x, where that spike happens, I put t=x, I get f(x), correct.
So that's like any, that's an identity or whatever you would like to call it.
And now just tell me the final answer.
Let me put it on the board above.
Now, so this was, this expressed my f(x) as a combination of delta.
Just the way over here I expressed  as a combination of delta.
Now, what's that u?
What's the function u?
The solution that comes from f(x).
Just erase here so that I can put all of them.
The point of the whole example now is for you to tell me what's u(x)?
Can you do it?
You see what u(x) is going to come out?
Going to come out nicely.
I've written the right-hand side f as a combination of deltas.
I know the answer, for delta.
It's G. What's the answer when the delta is shifted along?
What's the Green's function when the spike is moved along to a point, t?
Just because this constant coefficient translation and variant LTI problem is shifting in variant, the answer, when the spike is moved along, is just the answer G moved along.
So here's my answer.
All this, this is my input and my output is the same integral, this from minus infinity to infinity, of where it was f, now it's G. Well no, what do I want?
Help me out here.
No, f(t) is just the amount of the delta, but now what's the solution?
What do I write now?
G of?
x-t. That's it.
You see why that works?
Because this was the input, G's the output.
The problem was shift in variant, so if I shifted the input I shifted the output.
It was a linear, so if I add up a bunch of deltas, the solution is add up a bunch of G. That's the answer.
Oh, I could just say one thing more.
But you've got it if you see math.
So that's the point, that if you know the Green's function, well yeah.
Maybe from a practical point of view, what have we done?
The original way we did it involved our computing two integrals.
We had to, if we were given an f(x), we had to find it's transform f hat of k, that we weren't sure we could do with pencil and paper.
And then we got an answer with an f hat of k up here and then we had to transform back.
Step one and step three, we had to do.
Now this is better.
This is like better, because we were able to get an explicit answer, G, when this was a delta.
You could say I've got it down to one integral.
Well, for whatever that's worth.
I was going to say, probably can't do that one either depending what f(x) is.
But that's a nice way to see the answer, you have to admit.
And it's because the problem is is a shift in variance and I can write the answer that way.
OK, and now one more thing about this.
And I've written the word, the key word, down here.
Convolution.
Do you see the nice way to write that answer?
It's the convolution of f with G. We didn't do integral convolutions, we just did the discrete sums, but I mentioned that in the integral case, you have this same thing.
t and x-t adding up to x, just the way we had k and n-k adding up to n. In other words, this is just notation.
But I'm just going to write the answer in a nice way.
So after all that lecture, the answer to the differential equations is in three symbols f star G. Where this just simply means this.
This is the continuous convolution, not cyclic, and there's the answer.
I'll allow myself one more thing.
Here we had a convolution for the right-hand side.
We started with this, this is a convolution.
Now, what are the three symbols that I write down?
For the shorthand for this equation?
So this was to be true for any f. And now can I write down, how do I write?
f(x) is equal to, what is this right-hand side?
It's f convolved with?
So any f is the same f convolved with delta.
In convolution, delta is one.
Because when I go to the other space and I get a multiplication, it really is one, right?
In the other space.
So in the other space, so this convolution in x space turns into multiplication in frequency space.
And it just tells me that f hat is f hat times one.
So that's the way to look at it in physical space.
And this is the way to look at the solution.
So, one more sod and I'll come back to that.
So this G, the Green's function, this is what a CAT scan does, what an X-ray telescope does.
What all sorts of physical things do.
Provided we can assume this translation in variance, which is never perfectly true because the telescope is finite.
But a telescope takes the star, takes the light signal, convolves it with the telescope's own little Green's function.
It blurs it, it's the point spread function.
It's the blurring function, G. The Green's function on a telescope is somehow, that's what's you're convolving with.
And if you want to find that star as a bright, single point.
You've got to do deconvolution.
You've got to do a division to get the G out.
May I just say those words and then it's Thanksgiving?
That a machine, a sensor which is translation in variant, or you could say close enough to pretend it is, because nothing is going to be perfect translation in variant all the way out to infinity.
But if it's near the star.
So I look at this point star in the telescope I see a blur.
That's because the telescope has convolved the correct thing that I should have seen, with G. It's blurred it by its point spread function.
So what if the person, the factory where the telescope was built can test this whole thing on points.
And it can find the point spread function.
And if I knew G, then I could undo it.
And get a clear picture.
So it's that step that won the Nobel Prize for the CAT scan, and I'm sure is winning Nobel Prizes for astronomers.
OK, have a great Thanksgiving, I'll see you Monday.
Good.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well, hope you had a good Thanksgiving.
So this is partly review today, even, Wednesday even more review.
Wednesday evening.
Or Wednesday at 4 I'll be here for any questions.
And then the exam is Thursday at 7:30 in Walker.
Top floor of Walker this time, not the same 54-100.
OK, and then, no lectures after that.
Holiday, whatever.
Yes.
Right, you get a chance to do something.
Catch up with all those other courses that are being neglected in favor of 18.085 Right.
OK, so here's a bit of review right away.
We really had four cases.
We started with Fourier series, that was periodic functions.
And then discrete Fourier series, also periodic in a way.
Because w^n was one.
So that we have n numbers and then we could repeat them if we wanted.
So those are the two that repeat.
This is the f(x), this is all x, so that would be the Fourier integral that we did just last week.
Fourier integral transform.
And this was the - well, these are all, this is the discrete all the way.
So that's - oh, you can see these pair off, right?
The periodic function, the 2pi periodic function has Fourier coefficients for all k, so that's the pair that we started with.
Section 4.1.
This sort of pairs, I don't know whether to say with itself.
I mean, we start with n numbers and we end with n numbers.
We have n numbers in physical space.
And we have n numbers in frequency space.
Right, so we call those, so those went to c_0 up to c_(N-1).
And this, all x pair for the function, paired off with itself.
Or with this went to f - well maybe I use small f. I guess I did in last week.
So that, and I called its Fourier transform f hat of k, all k. So that's the pairing kind of inside n-dimensional space with the Fourier matrix.
This is the pairing of the formula for f, and its similar formula for f hat, and these are the guys that connect with each other.
OK, so that's what we know.
What we haven't done is anything into two dimensions.
So I would like to include that today.
I think my real message about 2-D, and I'm not going to include it on the exam, but you might wonder, OK, can I have a function of x and y?
And will the whole setup work.
And the answer is yes.
So really, my message is not to be afraid in any way of 2-D.
It's just the same formulas with x,y or two indices, k,l.
Yeah.
You'll see that.
OK, now for the new part.
What's a convolution equation?
That's my word for an equation where instead of doing a convolution and finding the right-hand side, instead we're given the right-hand side.
And the unknown is in the convolution.
So let me write examples of convolution equation.
Every one of these would allow a convolution.
So the convolution equation would be something the integral of F(t)u, for the unknown, at x-t is - oh no, sorry.
F will be the right-hand side.
F of, well, can I - yeah, better if I put it on the right-hand side.
Wouldn't want to call it the right-hand side.
So this would be some, shall I call it often K for kernel is sometimes the word.
So what I'm saying is equations come this way.
This is really K convolved with u.
Equals F. You see, the only novelty is the unknown is here.
So that's why the word d convolution is up there.
Because that's what we have to do.
We have to undo the convolution, this unknown function is convolved with a known, K is known, some known kernel that tells us the point spread of the telescope or whatever we're doing.
And gives us the output that we're looking at.
And then we have to find the input.
OK, can I write down the similar equations for the other three here?
And then we'll just think how would we find u, how would we solve them?
So the equation here might be that some kernel circle, convolved with the unknown u is some y.
These are now vectors.
This is known.
This is known.
And those, the end-components of u, are unknown.
OK, so that would be the same problem here.
What would be here?
Same thing.
Now the integral will go from - the only difference is the integral will go from minus infinity to infinity, K(t)u(x-t)dt=f(x).
u(x-t) And finally regular convolution.
What am I going to call it?
K would be a sequence, maybe I should call it a, known convolved with u, unknown, is some c, known.
Yeah.
So those would be four equations.
You might say, wait a minute where is Professor Strang come up with these problems at the last week of the course.
But, these are exactly the type of problems that we know and love.
These come from constant coefficient time invariant, shift invariant.
Linear problems.
LTI, linear time and variant.
And my lecture Wednesday, just before Thanksgiving, took a differential equation for u and found, and put it in this form.
I'll come back to that.
So suddenly we're seeing, I mean, we're actually seeing some new things but also it includes all the old ones.
These are all of the best problems in the world.
These linear constant coefficient problems.
Time in variant, of any of these types.
This one was an integral from minus pi to pi, where this one went all the way.
So this is not brand new stuff.
But it sort of looks new.
And now the question is, so my immediate question is, before doing any example, how would you solve such an equation.
And I saw on old exams, some of this sort for example, let me focus on this one.
Let me, instead of K there, I'm not used to using K for a vector, I'm used to, well maybe I'll use c. For the vector there.
So this is N equations, N unknowns.
Oops, capital N is our usual here.
For the number.
N u, N unknown u's.
It's a matrix equation with a circulant matrix.
So all these equations are sort of the special best kind.
Because they're convolutions.
And now tell me the main point.
How do we solve equations like this?
How do we do a deconvolution, so the unknown is convolved with c here, it's convolved with K, it's convolved with a, how do we deconvolve it to get u by itself?
So what's the central idea here?
Central idea: go into frequency space.
Use the convolution rule.
In frequency space, where these transform, we're looking at multiplication.
And multiplication, we can undo.
We can de-multiply.
De-multiply is just a big word for divide, right?
So that's the point.
Get into that space.
That's what we've been doing all the time.
I better get one example, the example from the problem Wednesday, just up here.
Just so you see it.
This won't look like a convolution equation, but do you remember that it was -u'' plus a squared u equal some f(x)?
So that's a constant, it's certainly constant coefficient linear.
Time invariant.
Right, OK. And how did we solve that?
We took Fourier transforms.
So this was the second derivative, the Fourier transform.
What is the rule for the Fourier transform of derivative?
Every derivative brings down an ik, in the transform.
So we get ik twice.
So it's k squared.
i squared cancels the minus one.
So that's the transform of -u''.
This is the transform of ordinary a squared u, just a squared.
And this is f hat.
So we've got into frequency space.
Where we are just seeing a multiplication, K squared plus a squared, u hat.
Of, this is u hat of k, equals f hat of k, right?
Oh well, sorry we were - yeah, that's right.
f hat of k, right.
So we're in frequency space, where we just see a multiplication.
So again, this is now we just demultiply, just divide.
And then we have the answer, but we have its transform.
And then we have to transform back.
So we have to do the Fourier transform to get to the, I'll say the inverse Fourier transform.
To get back to u(x).
The answer.
That's the model.
That's the model.
And that's maybe the one that we've seen, now we're able to think about all these four topics.
Right.
OK, so what was the key idea?
Get into frequency space and then it's just a, the equation is just a multiplication, so the solution is just a division.
So can I do that now with these four examples, just see.
So this is like bring the pieces together.
OK, and deconvolution is a very key thing to do.
OK, so I'll take all four of those and bring them into frequency space.
So this will be maybe, you'll let me use k hat of, oh, k hat of k, that's not too good.
Well, stuck with it.
What am I doing here?
In this 2pi periodic one?
That's the one I started with, but now I've got, I I'm using hats and so on.
I didn't do that in Section 4.1.
What the heck am I going to, what notation am I going to do?
And I really didn't do convolution that much, for functions.
So let me jump to here.
I'll come back.
It follows exactly the same pattern.
So let me jump to this one.
OK, so I have a convolution equation now.
This is one where you could do this one.
This could appear on the quiz because I can do all of it.
So what is this convolution?
OK.
I've got n equations, n unknowns.
Let me write them in matrix form, just so you see it that way too.
c_1, c_2, c_3>.
I'll make n=4.
And then these are, this convolution has c_0, c_2, c_1.
c_3, c_2, c_1, c_0.
This'll be c_3, c_3, c_3, I'm writing down all the right numbers in the right places.
So that when I do that multiplication with the unknown, , I get the right-hand, the known right-hand side.
Maybe b would be a little better.
Because we're more used to b as as a known.
It's just an Ax=b problem, or an Au=b problem.
But it looks like a convolution but now it's just a matrix multiplication.
So this is just <b_0, b_1, b_2, b_3>.
OK. That's our equation.
Special type of matrix.
Circulant matrix.
So this is just literally the same as c circularly convolved with u equals b. I just wrote it out in matrix language.
So you could call MATLAB with that matrix, and so one way to answer it would be get the inverse of the matrix.
But if it was large, a better way would be switch over to frequency space.
Think, now.
What happens when I switch these vectors to frequency space?
It becomes a multiplication.
So this becomes a multiplication.
Now so c, I want the Fourier, from the c's, what am I going to - so these are all in the space where it's a convolution.
What am I going to call it where it's in the space where it's a multiplication?
I just need three new names.
Maybe I'll use c hat, u hat, and b hat just because there's no doubt in anybody's mind that when you see that hat, you've gone into frequency space.
Now, what's the equation in frequency space?
And then I'll do an example.
It's a multiplication, but I don't usually see a vector and nothing there.
What's the multiplication in frequency space?
It's component by component.
c_0*u_0=b_0.
c hat 1 u hat 1 equals b hat 1. c hat 2, u hat 2, equals b hat 2.
And finally, c hat 3, u hat 3, equals b hat 3.
And there might be, I don't swear that there isn't, a 1/4 somewhere.
Right?
But the point is, we're in frequency space now.
We just have a component by component, each component of c hat times each component of u hat gives us a component of b hat; now we're ready for a deconvolution; just divide.
So now u hat, obviously I don't have to write all these, b hat 0 over c hat 0.
Right?
I just do a division.
So on down to u hat 3 is b hat, is the third component of b, divided by the third component of c. OK, now don't forget here.
That in going from here to here, I had to figure out what the c hats were, right?
I had to do the Fourier matrix, or the inverse Fourier matrix to go from c to c hat, from b to b hat, so everything got Fourier transforms.
But the object was to make the equation easy.
And of course, now we've got four trivial equations that we just solved that way.
Alright, let me see if I can just pull this down with some questions.
Here's a good question.
When is a circulant matrix invertible.
When will this method work?
The circulant matrix could fail to be invertible.
How would I know that?
If it's singular, and how would I, if I proceed this way, here I've got an answer.
But if it's singular I'm not really expecting to get an answer.
Let me left the board a little.
So where would I get, oops.
Have to stop this method.
In solving those four equations.
Where would I learn that it's is singular?
What could go wrong in this?
Yes
[INAUDIBLE]
That's right.
Always in math, the question is are you dividing by zero.
So the question of whether the matrix is singular, is the same as the question of whether c_0 hat, c_01, c_02, and c_03, - sorry, c_0 hat, c_1 hat, c_2 hat, and c_3 hat, can't be zero.
That's, in fact, even better those four numbers, those four c hats, are actually the eigenvalues of the matrix.
We've switched, what the Fourier transform did, was switch over to the eigenvalues and eigenvectors.
And there, that's the whole message of those guys is, you follow each one separately.
Just the way we're doing here.
So this is the component of the b in the for eigenvector directions.
Those are the four eigenvalues, and I have to divide by them.
You see, the idea is, like, we've diagonalized the matrix.
We've had that matrix, which is full.
And we take by taking the Fourier transforms, that's the same thing as as putting in the eigenvectors, switching the matrix to this diagonal matrix, right?
Our problem has become like the diagonalized form is c_0 hat down to c_3 hat, sitting on the diagonal.
All zeroes elsewhere.
That's when we switched, when we did Fourier transform we were switching to eigenvectors.
OK, so that's the message.
That the test for singularity is the Fourier, the transform of c hits zero.
Then we're in trouble.
Let me do an example you know.
Let me do an example you know.
OK here's, so finally now we get a numerical example.
The example we really know is this one, right?
As I start writing that, you may say in your mind, oh no not again.
But give it to me, one more week with these matrices.
But it'll be the C matrix, so it's going to be the circulant.
Recognize this?
And it's got those minus ones in the corners, too.
OK, let's go back to day one.
Is that matrix invertible?
Yes or no.
Please, no.
Everybody knows that matrix is not invertible.
And do you remember what's in the null space?
Yes, what's the vector in the null space of that matrix?
All ones.
Now, when I take, just think now.
When I take Fourier transform, that all ones is going to transform to what?
It's going to transform to the delta.
It'll transform to the one that is like, .
Or maybe it's .
But it's that, well, OK, now I'm ready to take.
So here's my c. So what's my method now?
I'm going to do this method, and I'm going to run into, this thing is going to be zero.
Because that's the eigenvalue that goes with the <1, 1, 1, 1, 1> column, the constant, the zero frequency in frequency space.
You'll see it happen.
So let's take the Fourier transform of that.
And then we would have to take the Fourier transform of the right-hand side, b, whatever that happened to be.
But it's always the left side.
The singular or not matrix.
I believe we'll be singular here.
So, OK, just remind me how do I take transforms of this guy?
Gosh, we have to be able to do that.
That's Section 4.- well, 4.3 isn't, yeah.
The DFT of that vector.
What do I get?
Yes.
How do I take the DFT of a vector?
I multiply by the Fourier matrix, right?
Yes.
So I have to multiply that thing by the Fourier matrix.
So to get c hat, this was big C for the matrix, little c for the vector that goes into it, into column zero.
And c hat for its transform.
OK, so now here comes the Fourier matrix that we know, 1, i, i^2, i^3, 1, i^2, i^4, i^6.
1, i^3, i^6, and i^9.
So I want to transform that c to get, to find out c hat.
OK, and what do I get up there?
What's the first component, the zeroth component I should say, when I take this guy, this four, this vector with four components and I get back four components, the frequency components, what's the first one?
Ones times this, what am I getting?
Zero.
That's what I expected.
That tells me the matrix is not going to be invertible.
Because in a different language, I'm finding the eigenvalues and that's one of them.
And if an eigenvalue is zero, that means the eigenvector is getting knocked out completely.
And there's snow way a c inverse could recover when that eigenvector is gone.
OK, let's do the other ones.
Two minus i, nothing.
What's the other one here?
Two, this is minus i, and that's plus i, I think.
So I think it's just two.
Alright, this is 2 i squared, can I write in some of these just so I have a little, i squared is minus one, and i^4 is one, and that's minus one.
So that's two plus one, plus one, I think is four.
And then i^3 is the same as minus i.
And i^9 is the same as plus i.
So I think I'm getting two plus i, nothing, minus i, two.
So what's my claim?
My claim is that these are the four eigenvalues that the Fourier - Fourier diagonalizes these problems.
That's what it comes to.
Fourier diagonalizes all constant coefficients, shift invariant, linear problems.
And tells us here are eigenvalues.
So .
Would you like to, how do I check the eigenvalues of a matrix?
Let's just remember.
If I give you four numbers and I say those are the eigenvalues, and you look at that matrix, what quick check does everybody do?
Compute the - the trace.
Add up the diagonal of the matrix, add up the proposed eigenvalues, they had better be the same.
And they are.
I get eight both ways.
That doesn't mean, of course, that these four numbers are right, but I think they are.
Yeah, yeah.
So those added up to eight, those numbers added up to eight.
And yep.
And these are real.
They came out real, and how did I know that would happen from the matrix?
What matrices am I certain to get real eigenvalues for?
Symmetric.
Right.
Now, what about - I heard the word positive.
Of course, that's the other question I have to ask.
Is this matrix positive definite?
OK, everybody this is the language we've learned in 18.085.
Is that matrix positive definite, yes or no?
No.
What is it?
It's positive semi-definite.
What does that tell me about eigenvalues?
Their one is zero, that's why it's not positive definite.
But the others are positive.
So that sure enough, in other words, what we've done here, for that matrix that came on day one and now we're seeing it on day N-1 here.
We're we're seeing sort of in a new way, because at that time we didn't know these four were the eigenvectors of that matrix.
But they are.
And we're coming to the same conclusion we came to on day one.
That the matrix is positive semi-definite and that we know its eigenvalues.
And we can, actually, let me even take it one more step, just because this example is so perfect.
Some right-hand sides we could solve for, right?
If I have a matrix that's singular, way, way back.
Even, I think it was like a worked example in Section 1.1, I could ask the question when is Cx=b solvable.
Because there are some right-hand sides that'll work.
Because if I just take an x and multiply by C, I'll get a right-hand side that works.
But for which vectors right-hand sides, b, well my method work?
The ones that have...?
Yeah, the ones that have which?
What do I need with these c's, c_0, c_1, c_2, and c_3, for my solution to be possible.
I need b_0 hat.
Equals zero.
I need b_0 hat equals zero.
And then what does that say?
That means that the b, the vector b, has no constant term in the Fourier series.
It means that the vector b is orthogonal to the <1, 1, 1, 1>, eigenvector.
So this is like a subtle point but just driving home the point that what Fourier does is diagonalize everything.
It diagonalizes all the important problems of, all the simplest problems, of differential equations.
You know, I mean this is like 18.03 looked at from Fourier's point of view.
OK, what more could I do with that equation?
I think you really are seeing all the good stuff here.
You're seeing the matrix.
We're recognizing it as a circulant.
We're realizing that we could take its Fourier transform.
We get the eigenvalues.
We're diagonalizing the matrix, the convolution becomes a multiplication, and the solution becomes, inversion becomes division.
I hope you see that.
That's really a model problem for this course.
OK. yeah.
Questions.
Good
[INAUDIBLE]
Would I give you a six by six Fourier matrix on a test?
Probably not.
No.
I just about could.
I mean, it's, six by six, those are pretty decent numbers.
Right.
Those six roots of unit e, but not quite.
Right, yeah.
Yeah.
Yeah.
So four by four is, five by five would not be nice, certainly.
Who knows the cosine of 72 degrees?
Crazy.
But, at 60 degrees we could do.
So the Fourier matrix would be full of square roots of three over two, and one over two, an i's, and so on.
But it wouldn't be, so really four by four is sort of the model.
Yeah, yeah.
So four by four is that model.
Other questions?
Because this is really a key example.
Yeah.
When I calculated the eigenvalues, yeah.
Ah.
Because this matrix, I know everything about that matrix when I know its first vector
[INAUDIBLE]
Yeah, it's because it's a circulant matrix.
It's because that matrix is expressing convolution with this vector.
.
That circulant matrix essentially is built from four numbers, right.
Yeah.
Yeah, and they go in the zeroth column.
Right, yeah.
Yeah.
Right, so there is an example where we could like do everything.
Now, and let me just remember that with this example, we could do everything.
So this is an example of, you could say this type of problem.
But with a very special kernel there, so it turned out to be, it looks like an integral equation here but if that kernel involves delta functions and so on then it can be just a differential equation.
And then that's what we got there.
So we took all the same steps we did here, we did here.
We took the Fourier transform, and I emphasize there, just to remember Wednesday, this was a delta function.
When I took the Fourier transform I got a one, so this was a one over this.
And I did the inverse transform and I got back to the function that I drew.
Which was e^(-ax) over 2a.
And even.
So, yeah.
this was the answer u(x).
So I was able to do that, I mean this step was easy, that step is easy.
That step is easy, the division is easy.
And then I just recognize this as the transform of this one, this example that we had done.
Once I divided by 2a.
So you should be able to do this.
So those are two that you should really be able to do.
I'm not going to, obviously I'm not going to, ask you a 2-D problem on the exam or even on a homework.
But now if you'll allow me, I'd like to spend a few minutes to get into 2-D. Because really, you've got the main thoughts here.
That Fourier is the same as finding eigenvectors and eigenvalues.
That's the main thought for these LTI problems.
OK, now suppose I have, let's just get the formalities straight here.
Suppose I have a function of x and y.
2pi periodic in x, and in y.
So if I bump x by 2pi, or if I bump y by 2pi - oh, I'm using capital F for the periodic guys.
So let me stay with capital F(x,y+2pi).
OK, so I have a function.
This is given.
This is, it's in 2-D now.
And I want to write its Fourier series.
So I'm just asking the question what does the Fourier series look like for a function of two variables.
The point is, it's going to be a nice answer.
And so everything, what you know how to do in 1-D you can do in 2-D.
So let me write the complex form, the e^(ik) stuff.
So what would I write, how would I write this?
I would write that as a sum, but it'll have, I'll make it a double sum, I'll write two sigmas just to emphasize that we're summing from k equal minus infinity, to infinity, and from l equal minus infinity to infinity.
We have coefficients c_kl.
They depend on two indices, this is the pattern to know.
Multiplying our e^(ikx), and our e^(ily).
Right, good.
So, alright.
Let me ask you.
How would I find c_23?
Just to know that - we could find all these coefficients, find formulas for them.
We could do examples.
How would I find c_23?
So this is my F. I know F. I want to find c_23.
What's the magic trick?
Then and a 2pi periodic, so all integrals.
All the integrals, and I'm giving you a hint, of course.
I'm going to integrate.
And the integrals will all go from minus pi to pi in x, and in y.
They'll integrate over the period square.
Here's the period square from, there's the center.
x direction, y direction, goes out to pi and goes up to pi.
So all integrals will be over dxdy.
But what do I integrate?
To find c_23.
Well, these guys are orthogonal.
That's what's making everything work, they're orthogonal and very special.
So that by u's orthogonality, what do I do?
I multiply by?
Just tell me what to multiply by.
By this and integrate.
OK, what is it that I multiply by if I'm shooting for c_23, for example?
e^(i2x), is it e^(i2x)?
Minus, right.
I multiply by e^(-i2x), e^(-i3y), and integrate.
Yeah.
So when I multiply by that and integrate, everything will go except the c_23 term.
Which will be multiplied by what?
So I'll just have c_23 times probably 2pi squared.
I guess 2pi will come in from both integrals, so the formula will be c_kl.
c_kl will be, do you want me to write this formula?
I'll write it here and then forget it right away.
c_kl will be one over 2pi squared.
The integral of my function.
Times my e^(-ikx), times my e^(-ily)dxdy.
So that just makes the point.
That there's nothing new here, it's just up a dimension.
But the formulas all look the same, and if f was a- well, if F is a delta function, if f is now a 2-D delta function.
We haven't done delta functions in 2-D, why don't we?
Suppose F is the delta function in 2-D. Then what are the coefficients?
What do you think you this means, this delta function in 2-D?
So if I put in the delta here, and I integrate.
And what do I get then?
So if this guy is a delta, a two-dimensional delta function, the rule is that when I integrate over a region that includes the spike, so it's a spike sitting up above a plane now, instead of sitting above a line it's sitting above a plane.
Then I get the value, so this is the delta function at the origin.
So I get the value of this at the origin.
So what answer do I get?
I get one out of the integral and then I just have this constant.
So it's constant again.
And it's just one.
So the Fourier coefficients of the delta function are constant.
All frequencies there are the same.
What about a line of delta functions?
And what does that mean?
What about, yeah let me try to draw delta(x).
Suppose I have a function of x and y - it's just worth imagining a line of delta functions.
So I'm in the x,y, let me look again at this thing.
I have delta functions all along this line.
Now.
Here is a crazy example, just to say well there is something new in 2-D.
So previously my delta function was just at that point.
And the all integrals just picked out the value at that point.
But now think of a delta function's a sort of line of spikes.
Going up here, and then of course it's periodic.
Everything's periodic so that line continues and this line appears here, and this line appears here.
But I only have to focus on one period square.
What's my answer now?
If this function suddenly changes from a one point delta function to a line of delta functions?
Now tell me what the coefficients are.
What are the Fourier coefficients in 2-D for a line of delta functions?
A straight line of delta functions going up the y axis?
It'll be - let's see.
What do I do?
I'm go to integrate - oh yeah, what is it?
Good question.
OK.
So what is the - when do I get zero and when do I not get zero out of this?
Yeah, tell me first when do I get zero out of this integral?
And when do I not?
What am I doing here?
Help me.
I said 2-D was easy and I've got in over my head here.
So look.
I can do the x integral, right?
We all know how to do the x integral.
Yes, is that right?
If I integrate with respect to x, what do I get?
Let's see, I'll keep that one over 2pi squared.
Now I'm trying to do this integral.
Do I get a one?
If I get a one from the x integral.
So then I'm down to one integral, just the y integral is left.
e^(-ily)dy, right?
I did the x part, which said, OK, take the value at x=0, which was one.
So the x integral was one, good.
And now I've got down to this part.
Now what is that integral?
It's two - wait a minute.
Depends on l, doesn't it?
When l - yeah.
So it's going to depend on whether l is zero or not.
Is that right?
Yeah, that's sort of interesting.
If l is zero, then I'm getting - then this is a 2pi.
So the answer, so I'm getting c_k0, when l is zero I'm getting a 2pi out of that.
If l is zero, I'm integrating one, I get a 2pi cancels one of those.
I get a one over 2pi.
And otherwise the other c_kl's, when l is not zero, are what?
Just zero, I think.
The integral of this thing, this is a periodic guy, if I integrate it from minus pi to pi it's zero.
What am I - I'm making a big deal out of something that shouldn't be a big deal.
The delta(x) function, this is just, its Fourier series is just the one we know.
Sum of e^(ikx)'s.
Do you see what's happened here?
It was supposed to be a double sum.
But the ones, when l wasn't zero aren't there.
The only ones - yeah.
So I'm back to the, for a line of spikes, a line of deltas, I'm back to - it so it only depended on x, so the Fourier series is just the one I already know.
All ones.
When there's no - when l is zero, all ones or 1/2 2pi's, all constants when l is zero, but there's no y.
There's no oscillation in the y direction.
OK, I don't know why I got into that example, because the conclusion was just it's the Fourier series that we already know and it doesn't depend on l, because the function didn't depend on y. OK, then we could imagine delta functions in other positions, or a general function.
OK, so that's 2-D. Would I want to tackle a 2-D - ha.
We've got two minutes.
that's one dimension a minute.
Right, OK. What happens, what's a 2-D discrete convolution?
What's a 2-D discrete convolution?
Now, you might say OK, why is Professor Strang inventing these problems?
Because a 2-D discrete convolution is the core idea of image processing.
If I have an image, what does image processing do?
Image processing takes my image, it separates it into pixels, right, that's all the image is, bunch of pixels.
Then many 2-D image processing algorithms, we'll jpeg, all jpeg for example, would take an eight by eight, eight by eight 2-D, in other words.
Eight by eight, 2-D is the main point.
Set of pixels.
And transform it.
Do a 2-D transform.
So what is a 2-D transform?
What would be the 2-D transform that would correspond to this?
First of all how big's the matrix?
Just so we get an idea.
I probably won't get to the end of this example.
But just, so in 1-D, my matrix was four by four.
Now I've got, so that was for four points on a line.
Now I've got a square of points.
So how big is my matrix?
16, right?
16 by 16, because it's operating on 16 pixels.
It's operating on 16 pixels, where in 1-D it only had four to act on.
So I'm going to end up with a 16 by 16 matrix here.
And I think - let me see, what do I need?
Oh, wait a minute.
Uh-oh.
Yeah, I think the time's up here.
Yeah, because my C, has my C got to have 16 components?
Yes.
My u has to have 16, my right-hand side has got these 16 components.
Yeah.
So I'm up to 16 but a very special circulant of a circulant.
It'll be a circulant of a circulant somehow.
OK, enough for 2-D.
I'll see you Wednesday and we're back to reality.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, hi.
So I've got homework nine for you.
Ready to return at the end.
Also, the department asked me to do evaluations, but that's the end of the lecture.
Then, so everybody knows there's a quiz tomorrow night.
And shall I just remember the four questions on the quiz?
I mean, not the details but the general idea of the questions.
Details OK too.
Yes.
Yeah, so there'll be one question on a Fourier series.
And you should know the energy equality for all of these possibilities, connecting the function squared with the coefficient squared.
A second one on the discrete Fourier transform.
Cyclic stuff.
The third one on the Fourier integral.
And have a look at the applications to solving an ODE.
I did one in class.
The one in class was the one in the book -u''+a^2*u=f(x), so this will be.
So have a look at that application.
This is, of course, on minus infinity to infinity, and then a fourth question on convolution.
OK. And this afternoon, of course, I'll be here to answer any questions from the homework, from any source for these topics.
Are there any questions just now, though?
I'm OK to take questions.
I thought I'd discussed today a topic that involves both Fourier series and Fourier integrals.
It's a kind of cool connection and it's linked the name of Claude Shannon who created information theory, who was a Bell Labs guy and then an MIT professor.
So I should put his name in.
Shannon.
OK, so this is, yeah.
You'll see.
So it's not on the quiz but it gives me a chance to say something important, and at the same time review Fourier series and Fourier integrals.
So let me start with the problem.
The problem comes for a A to D converter.
So what does that mean?
That means this a is analog.
That means we have a function, A for analog.
And D for digital.
So we have a function.
Like, so f(x).
Say, all the way minus infinity to infinity, so we'll be doing, that's where the Fourier integral's going to come up.
So that's analog.
All x, it's some curve.
And people build, and you can buy, and they're sold in large quantities, something that just samples that function.
Say, at the integers.
So now I'll sample that function and let me take the period of the sample to be one, so that I'm going to take the values f(n).
so now I've got something digital like I work with I can come here with and so the saplings here I'm well he I mean and they're going to question it yeah selling sampling say army is about this question and it seems like crazy question when do these numbers that's just a sequence of numbers.
This x was all the way from minus infinity to infinity.
And similarly, n is numbers all the way from minus infinity to infinity, they're just samples.
When does that tell me the function?
When can I learn from those samples, when do I have total information about the function?
Now, you'll say impossible.
Right?
So suppose I draw function f(x), OK?
And I'm going to sample it at these points.
All the way, so these are the numbers, these are my f(n).
Sampling it, equal intervals because if we want to use Fourier ideas, equal spacing is the right thing to have.
So when could I recover the function in between?
Well, you'd say, never.
Because how do I know what that function could be doing in between.
So let me take the case when all the samples are zero.
And let's think about that case.
Suppose, what could the function be if all the samples, if these are all zeroes, forever.
OK, well there's one leading candidate for the function, the zero function.
Now, you'll see the whole point of the sampling theorem if you think about other.
What other functions?
Familiar functions, yeah.
I mean, we could, obviously, any, all sorts of things.
But since we're doing Fourier, we like to pick on the sines, cosines, the special function, and think about those in particular.
So, somebody said sines.
Now, what function, so a sine function certainly, the sine function hits zero at infinitely often.
What frequency, so what sine of what would give me the same answer?
The same samples.
i put this sine function that you're going to tell me, so you're going to tell me it's sine of something.
Will have these zero values at all the integers, at zero, one, two, minus one, minus two, and so on.
So what would do the job?
Sine of?
Of what will hit zero.
So I'm looking for a sine function, I guess I'm looking first for the function that just does that.
And what is it?
sin(pi*x).
And now tell me some more.
Tell me another function.
Which will also, it won't be that graph.
sin(2pi*x).
And all the rest, OK?
Let me just use a word that's kind of a handy word.
Of course, let's put zero on the list here.
OK.
So this is where k, the frequency, usually appears.
This is where k. The word I want to introduce is alias.
This frequency, pi, is an alias for this at frequency zero.
Here's the, it's a different function but yet the samples are the same.
So if you're only looking at the samples you're getting the same answer but somehow the function has a different name.
So that frequency and this frequency, and all those others would be alias.
Can I just write that word down, because you it often.
Alias.
That means two frequencies, like pi and 2 pi, and zero or whatever, that give you the same samples.
OK, so now comes Shannon's question.
So we have to make some assumption on the function.
To knock out those possibilities.
We want to know a limited class of functions.
Which don't include these guys.
So that within this limited class of functions, this is the only candidate and we have this possibility of doing the impossible.
Of determining that if I know zeroes here, the function has to be zero everywhere.
OK, now the question is what class of functions?
We want to eliminate these guys, and sort of, your instinct is, you want to eliminate functions that, you know if it's not zero then got to get up and back down in everything.
It could do different things in different intervals.
But somehow it's got to have some of these frequencies.
Pi or higher.
Would have to be in there.
So this is the instinct.
That if I limit the frequency band, so I'm going to say f(x) is band-limited, can I introduce that word?
I'll maybe take a moment just ask you if you've seen that word before.
How many have seen this word, band-limited?
Quite a few but not half.
OK. Band-limited means the band is a band of frequencies.
So the function's band-limited when its transform, this tells me how much of each frequency there is.
If this is zero, in some band.
In some band, let's say, all frequencies below something.
And let's not even put equal in there.
OK.
But that's not critical.
Band-limited, I have to tell you the size of the band.
And the size of the band, the limiting frequency is this famous Nyquist frequency, so Nyquist is a guy's name.
And the Nyquist frequency in our problem here is pi.
This is the Nyquist frequency.
If we let that frequency, that's the borderline frequency.
And there would be a similar Nyquist sampling rate.
So Nyquist is the guy who studied the sort of borderline case and so the point is that if our, say, band-limited by pi, I have to tell you, so band-limited means there's some limit on the band.
And our interest is when that limit is the Nyquist frequency.
The one we don't want to allow, so we, this is the point.
So this will be the idea.
That if we take this class of function, that band limit at that, those are called band-limited functions, and they're band-limited specifically by the Nyquist limit.
If we take those, then the idea is that then we can reconstruct from the samples.
Because the only function that has zero samples in that class is the zero function.
You see that class has knocked out, is not allowing these guys.
Of course, haven't proved anything yet.
And I haven't shown how to reconstruct.
Well, of course, we quickly reconstructed the zero function out of those zeroes, but now let me take another obviously important possible sample.
Suppose I get zero samples except at that point, where it's one.
OK, now the question is what function, f(x), can I fill in, in between zero, zero, zero, zero, one, zero, zero, zero, can I fill in exactly one function.
That comes from this class.
So that I have now the answer for this highly important sample?
The sample that's all zeroes except for the delta sample, you could say.
OK, so I'm looking for the function now which is one at that point.
And zero at the others.
So here's a key function.
And I'll show you what it is.
So the function, this function that I'm going to mention, will get down here, it'll oscillate, it'll go forever.
It's not like a spline.
Splines made it to zero and stayed there, right?
The cubic spline, for example.
OK, but I guess, yeah, that somehow that function, we're not in that league.
We're in this band-limited league.
In a way you could say that, I mean, what's the key connection between dropoff of the Fourier transform?
So if the Fourier transform drops off fast, what does that tell me about the function?
It's smooth, thanks.
That's exactly the right word.
If the Fourier transform drops off fast, the function is smooth.
OK, this is really an extreme case.
That it's dropped off totally.
You know, it's not just decay rate, it's just zonk, out.
Beyond this band of frequencies.
And so that gives, you could say, sort of a hyper-smooth function.
I mean, so smooth that you know everything by knowing the sample.
OK, now I'm ready to write down the key function, a famous function that has those samples.
And that function is sin(pi*x)/(pi*x).
I don't know if you ever thought about this function, and it has a name.
Do you know its name?
sinc.
It's the sinc function.
Which is a little - you know, a name's a little unfortunate.
Mainly because you know, you're using those same letters S I N, but it's a c that turns it, that gives it, so this is called the sinc function.
sinc(x).
But the main thing is its formula.
OK, well everybody sees that at x=1, the sin(pi) is zero, at x=2, the sin(pi) is zero, all these ones we've seen already.
And now what happens at equals zero?
Do you recognize that this function, as x goes to zero, is a perfectly good function?
I mean, it becomes 0/0 at x=0.
But the limit, we know to be one.
Right?
This sin(theta)/theta is one.
As theta approaches zero, right?
So that does have that correct sample.
And now what's the, I claim that that is a band-limited function.
And you'll see that that function pushes the limit.
It's right, Nyquist barely lets it in.
Now here's a calculation.
So this is our practice.
What is f hat of k for that function?
Now, let me think how to do this one.
So just to understand this better, I want to see that that is a band-limited function and what is its Fourier transform.
OK, now once again here we have a function where if I want to do - how best to do this.
You could say well, just do it.
As I would say on the quiz, just go ahead and do it.
But you'll see I'm going to have a problem, I think.
But this gives us a chance to remember the formula.
So what's the formula for the Fourier integral transform of this particular?
So my function is the sinc function, sin(pi*x) over sin(pi*x).
So how do I get its, I do a what here?
e^(-ikx), and am I doing dx or dk?
dx, right?
And I'm going from minus infinity to infinity.
And am I, do I have a 2pi?
Yes or no?
Who knows, anyway?
Right.
In the book I didn't?
OK. Now, well, I don't know the answer.
But so let's - it's much better to start with the answer, right, and check that - so let me say what I think the answer is.
I think the answer is, it's a function that's exactly as I say, it pushes the limit from, this is k. It's the square wave, it's zero, the height is one.
It's the function that's zero all the way here, all the way there.
I think that that's the Fourier transform of that function.
And just before we check it, see how is Nyquist got really pushed up to the wall, right?
Because the frequency is non-zero right all the way through pi.
But pi is just one point there.
And anyway, I think we won't get into philosophical discussion about whether is that is that limited to pi, I don't know whether to put, you saw me chicken out here.
I didn't know whether to put less or equal or not, and I still don't.
But this is making the point that that's the key frequency.
So this particular function, what I'm saying is this particular function has all the frequencies in equal amounts over a band, and nothing outside that band.
And that's the Nyquist band.
OK, now why is this the correct answer here?
I guess the smart way would be, this is a good function, easy function.
So let's take the transform in the other direction.
Start from here and get to here.
Right?
That would be convincing.
Because we do know that that pair of formulas for f connected to f hat, connected to f, they go together.
So if I can show that I go from here, that that Fourier integral takes me from here to there, then this guy will take me back.
So let me just do that.
Because that's a very very important one that you should be prepared for.
Right.
So now what do I want to do?
Here's my function of k, and I I'm hoping that I recall.
Now, what do I do when I want to do the transform in the opposite direction?
It'll be an e^(+ikx), right?
d what?
dk, now.
From k equal minus infinity to to infinity.
And now I think I do put in the 2pi, is that right?
And the question is, does that bring back the sinc function?
If it does, then this was OK in the other direction.
If the transform's correct in one direction then the inverse transform will be correct.
So I just plan to do this integral.
f hat of k, of course, is an easy integral now.
f hat of k is one over, between minus pi and pi, so I only have to do over that range where f hat of k is just a one.
And now that's an integral we can certainly do.
So I have 1/2pi, integrating e^(ikx) will give me e^(ikx)/ix.
Now, remember I'm integrating dk.
Oh, look.
See, we're showing this x now in the denominator.
That we're hoping for.
And now I have to do that between k is minus pi and pi.
So this is like, so I get 1/2pi, e^(i*pi*k) -e^(-i*pi*x), right?
Over the ix.
OK so far?
I was doing a k integral and I get an x answer.
And I want to be sure that this x answer is the x answer I want, it's the sinc function.
OK, it is.
Right?
I recognize the sine, e^(i*theta)-e^(-i*theta), divided by two, I guess.
Is the sine, right?
So I have 1/2pi.
And here is ix - well, no, the i is part of that sine.
So I'm just using the fact that e^(i*theta)-e^(-i*theta) is, this is cosine plus i sine, subtract cosine.
But subtract minus i sine, so that will be two i sin(theta), right?.
We all know, and now theta is pi*x here.
So I have two - let me keep the i there, and 2i*sin(pi*x).
You see it works.
Just using this, replacing this by the sine, the two cancels the two.
The i cancels the i, and I have sin(pi*x) over pi*x, that's the sinc function.
OK, so that's the function we've now checked.
We've checked two things about this function.
It has the right samples, zero, zero, zero, zero, one at x=0 and then back to zero.
It is band-limited, so it's the guy, if this was my sample, if this was my f(n), zero, zero, zero, one, zero, zero, zero, then I've got it.
It's the right function.
OK, we can create Shannon's sampling formula.
Shannon's sampling formula gives me the f(x) for any f(n).
Maybe you can spot that, now.
So this is going to use the shift in variance.
Oh yeah, let's - tell me what the, let's take one step here.
Suppose my f(n), were zero, zero, zero, zero, and a one there.
So suppose this is, for the exam too, this idea of shifting is simple and basic.
And it's a great thing to be able to do.
So this wouldn't be the right answer that that function turn it was the one at zero.
Tell me what function.
Copying this idea will produce all zeroes except for a one at this point.
So I'll put this question over here.
Suppose I'm all zeroes at that point but at this point I'm up and then I go back to zeroes.
What function is giving me that?
You see it does decay because of the x in the denominator, it kind of goes to zero but not very fast.
OK, what's that function?
I replace x by x-1, right.
So the function here is sin(pi(x-1)), divided by pi(x-1).
I just change the x to x-1.
Now again, at x=0, this is sin(pi), sin(-pi) is safely zero, but at x=1 that's now the point where I'm getting 0/0.
And the numbers are right to give me the exact answer, one.
OK, so now we see what to do if the sampling turned out to give us this answer.
And now can you tell me the whole formula?
Can you tell me the whole formula, so now I'm ready for Shannon's sampling theorem.
Is that f(x), if f(x) is band-limited, then I can tell you what it is at all x.
So i'm going back to the beginning of this lecture.
It's a miracle that this is possible.
That we can write down a formula for f(x) at all x, only using f(n)'s.
OK, now it's going to be a sum.
From n equal minus infinity to infinity, because I'm going to use all the f(n)'s to produce the f(x).
And now what do I put in there?
So I want my formula to be correct.
I want my formula to be correct in this case, so that if all the f's were zero except for the middle one, then I want to put sin(pi*x)/(pi*x) in there.
The sinc function.
And I also want to get this one right.
If all the f's are zero, so there'll only be one term.
If this is the term, I want that to show up.
Ok what do I - yeah, you can tell me.
Suppose this is hitting at n. We'll just fix this and then you'll see it.
Suppose that all the others, n, n-1, n+1, all those at zero, but at x=n, it's one.
Now what should I have chosen?
What's the correct - I'm just going to make it easy for all of us to, yes.
What's the good sinc function which peaks at a point n?
Again, I'm just shifting it over.
So what do I do?
Put in, what do I write here?
n. I shift the whole thing by n. So that's the right answer when this hits at n. So now maybe you see that this is going to be the right answer for all of them.
You see that, we're using linearity shift invariance.
The shift invariance is telling us this answer for wherever the one hits.
That's what we need.
And then by linearity, I put together whatever the f is at that point, that would just amplify the sinc.
And then I have to put them in for all the other values.
That's the Shannon formula.
That's the Shannon formula, and this function is band-limited, let's see.
What's the - oh, yeah.
What's the, do you see that this one, that this guy is band-limited?
We checked, right?
We checked that this one, sin(pi*x)/(pi*x), that was band-limited because we actually found the band.
Now, that just gives us another chance to think.
Allowed.
What's the Fourier transform of this guy?
My claim is that it's also in this band.
Non-zero only in the band, and zero outside the Nyquist band.
What is the transform of that?
What happens if you shift a function, what happens to its transform?
So that's one of the key rules that makes Fourier so special.
If I took this sine, let me write this guy again.
This was the un-shifted one.
That connected to the, what am I going to call that, the box function.
The box function, the square wave.
Well, box is good.
Now, what if I shift the function?
If I shift a function, what happens to its Fourier transform?
Anybody remember?
You multiply it by, so if I shift the function, I just multiply this box function, this is a box function in the k, times something e to the i shift, and the shift was one, right?
Is it just e^(ik), d being the shift distance.
Oh, the shift distance was n. Right.
And possibly minus, who knows.
OK, but what's the point here?
The point is that it's still zero outside the box.
Inside the box, instead of being one, it's this complex guy.
But no change.
It's still zero outside the box.
It's still band-limited.
So this is the transform of this guy.
And then the transform of this combination would be still in the box, multiplied by some messy expression.
So what was I doing there?
I was just checking that, sure enough, this guy is band-limited, and it's band-limited it, gives us the right f(n)'s, of course.
Everybody sees that at x=n, let's just have a look now.
We've got this great formula.
Plug in x=n.
What happens when you plug in at one of the samples, you look to see what this a to d converter produced at time n, and let's just see.
So at x=n, the left side is f(n).
Why is the right side f of that N?
That particular N?
Maybe I should give a specific letter to that N. So at that particular sample, this left side is f(N), and I hope that the right side gives me f at that capital N. That particular one.
Why does it?
You're all seeing that at x equal capital N, these guys are all zero, except for one of them.
Except for the one when little n and capital N are the same.
Then that becomes the one.
And I'm getting f at capital N. So it will give me that, for the n=1, the one term, yeah.
I don't know if it was necessary to say that.
You've got the idea of the sampling formula.
I could say more about the sampling, just to realize that the technology, communication theory is always trying to, like, to have a greater bandwidth.
You always want a greater bandwidth.
But if the bandwidth, which is this, is increased, well by the way, what does happen?
Suppose it's band-limited by pi, oh, by pi/T.
Let's just, I normalize things to choose samples every integer.
Zero, one, two, three.
And that turned out that the Nyquist frequency was pi.
Now, what sampling rate would correspond to this band, which could be - well, let me just say what it is.
That would be the Nyquist frequency for sampling every T. Instead of a sampling interval of one, if I sample every T, 2T, 3T, -T, my sampling rate is T, so if T is small, I'm sampling much more.
Suppose T is 1/4.
If T is 1/4, then I'm doing four samplings.
I'm taking four samples, I'm paying more.
For this A to D you converter, because it's taking four samples where previously it took one.
How do I get paid back?
What's the reward?
The reward is if T is 1/4, then the Nyquist limit is 4pi.
I can get a broader band of singles by sampling them more often.
Let me just say that again, because that's the fundamental idea behind it.
If I sample more often, say, so fast sampling would be small, fast samples, would be small t and then a higher Nyquist.
A higher band limit.
More functions allowed.
If I sample more often I'm able to catch on to more functions.
If I sample, and that's what, I mean, now, communications want wide bands.
And this is where they get limited.
I mean, this is, could say, the fundamental, I don't know whether to say physical limit, sort of maybe Fourier limit on sampling theory.
Is exactly this Nyquist frequency.
OK, questions or discussion about that.
OK.
So that's an example that allowed us to do a lot of things.
I did want to ask for your help doing these evaluations.
Let me say what I'm going to do this afternoon.
I'm going to answer all the questions I can, and I planned, when there is a pause, and nobody else asks, I plan to compute the Fourier integral and Fourier series, say, Fourier series, for a function that has, it's going to be like the one today except this is going to have a height of 1/h, and a width of h. So that's, in case you're not able to be here this afternoon, I thought I'd just say in advance what calculations I thought I would do.
So there's a particular function f(x), it happens to be an even function.
We'll compute its Fourier coefficients, in and we'll let h go to zero.
To see what happens.
It's just a good example that you may have seen on older exams.
OK, well can I just say a personal word before I pass out.
So evaluations, if you're willing to help, and just leave them on the table would be much appreciated.
I'll stretch out the homeworks.
I just want to say I've enjoyed teaching you guys.
Very much.
Thank you all, and, thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This is the one and only review, you could say, of linear algebra.
I just think linear algebra is very important.
You may have got that idea.
And my website even has a little essay called Too Much Calculus.
Because I think it's crazy for all the U.S. universities do this pretty much, you get semester after semester in differential calculus, integral calculus, ultimately differential equations.
You run out of steam before the good stuff, before you run out of time.
And anybody who computes, who's living in the real world is using linear algebra.
You're taking a differential equation, you're taking your model, making it discrete and computing with matrices.
The world's digital now, not analog.
I hope it's OK to start the course with linear algebra.
But many engineering curricula don't fully recognize that and so if you haven't had an official course, linear algebra, stay with 18.085.
This is a good way to learn it.
You're sort of learning what's important.
So my review would be-- and then this, future Wednesdays will be in our regular room for homework, review, questions of all kinds, and today questions too.
Shall I just fire away for the first half of the time to give you a sense of how I see the subject, or at least within that limited time.
And then questions are totally welcome.
Always welcome, actually.
Right?
So I'll just start right up.
So essentially linear algebra progresses starting with vectors to matrices and then finally to subspaces.
So that's, like, the abstraction.
You could say abstraction, but it's not difficult, that you want to see.
Until you see the idea of a subspace, you haven't really got linear algebra.
Okay, so I'll start at the beginning.
What do you do with vectors?
Answer; you take their linear combinations.
That's what you can do with a vector.
You can multiply it by a number and you can add or subtract.
So that's the key operation.
Suppose I have vectors u, v and w. Let me take three of them.
So I can take their combinations.
So some combination will be, say some number times u plus some number times v plus some number times w. So these numbers are called scalers.
So these would be called scalers.
And the whole thing is a linear combination.
Let me abbreviate those words, linear combination.
And you get some answer, say b.
But let's put it down, make this whole discussion specific.
Yeah, I started a little early, I think.
I'm going to take three vectors; u, v and w and take their combinations.
They're carefully chosen.
My u is going to be .
And I'll take vectors in three dimensions.
So that means their combinations will be in three dimensions, R^3, three-dimensional space.
So that'll be u and then v, let's take zero, I think, one and minus one.
Okay.
Suppose I stopped there and took their linear combinations.
It's very helpful to see a picture in three-dimensional space.
I mean the great thing about linear algebra, it moves into n-dimensional space, 10-dimensional, 100-dimensional, where we can't visualize, but yet, our instinct is right if we just follow.
So what's your instinct if I took those two vectors, and notice they're not on the same line, one isn't a multiple of the other, they go in different directions.
If I took their combinations, say x_1*u+x_2*v. Oh now, let me push, this is a serious question.
If I took all their combinations.
So let me try to draw a little bit.
Okay.
I'm in three-dimensional space and u goes somewhere, maybe there and v goes somewhere, maybe here.
Now suppose I take all the combinations, so I could multiply that first guy by any number, that would fill the line.
I can multiply that second guy, v. So this was u and this was v. I can multiply that by any number x_2, that would fill its line.
Each of those lines I would later call a one-dimensional subspace, just a line.
But now, what happens if I take all combinations of the two?
What do you think?
You got a plane.
Get a plane.
If I take anything on this line and anything on this line and add them up you can see that I'm not going to fill 3-D.
But I'm going to fill a plane and that maybe takes a little thinking.
It just, then it becomes sort of, you see that that's what it has to be.
Ok, now I'm going to have a third vector.
Ok, my third vector will be .
Ok, so that  is zero in the x, zero in the y and one in the z direction.
So there's W. Now I want to take their combinations.
So let me do that very specifically.
How do I take combinations?
This is important.
Seems it's very simple, but important.
I like to think of taking the combinations of some vectors, I'm always putting vectors into the columns of a matrix.
So now I'm going to move to step two; matrix.
I'm going to move to step two and maybe I'll put it-- well not, I better put it here.
Ok, step two is the matrix has those vectors in its columns.
So in this case, it's three by three.
That's my matrix and I'm going to call it A.
How do I take combinations of vectors?
I should have maybe done it in detail here, but I'll just do it with a matrix here.
Watch this now.
If I multiply A by the vector of x's, what that does, so this is now A times x, so very important, a matrix times a vector.
What does it do?
The output is just what I want.
This is the output.
It takes x_1 times the first column plus x_2 times the second plus x_3 times the third.
That's the way matrix multiplication works; by columns.
And you don't always see that.
Because what do you see?
You probably know how to multiply that matrix by that vector.
Let me ask you to do it.
What do you get?
So everybody does it a component at a time.
So what's the first component of the answer?
x_1, yeah.
How do you get that?
It's row times the vector.
And when I say "times", I really mean that dot product.
This plus this plus this is x_1.
And what about the second row?
-x_1+x_2.
Or I'll just say x_2-x_1.
And the third guy, the third component would be x_3-x_2, right?
So right away I'm going to say, I'm going to call this matrix A a difference matrix.
It always helps to give names to things.
So this A is a difference matrix because it takes differences of the x's.
And I would even say a first difference matrix because it's just the straightforward difference and we'll see second differences in class Friday.
So that's what A does.
But you remember my first point was that when a matrix multiplies a vector, the result is a combination of the columns.
And that's not always, because see, I'm looking at the picture not just by numbers.
You know, with numbers I'm just doing this stuff.
But now I'm stepping back a little bit and saying I'm combining.
It's this vector times x_1.
That vector times x_1 plus this vector times x_2 plus that one times x_3 added together gives me this.
Saying nothing complicated here.
It's just look at it by vectors, also.
It's a little interesting, already.
Here we multiplied these vectors by numbers.
x_1, x_2, x_3.
That was our thinking here.
Now our thinking here is a little, we've switched slightly.
Now I'm multiplying the matrix times the numbers in x.
Just a slight switch, multiply the matrix times the number.
And I get some answer, b.
Which is this, this is b.
And of course, I can do a specific example like, suppose I take, well, I could take the squares to be in x.
So suppose I take A times the first three squares, .
What answer would I get?
Just to keep it clear that we're very specific here.
So what would be the output b?
I think of this as the input, the , the x's.
Now the machine is a multiply by A and here's the output.
And what would be the output?
What numbers am I going to get there?
Yeah?
One, three, something?
.
Which is actually a little neat that you find the differences of the squares are the odd numbers.
That appealed to me in school somehow.
That was already a bad sign, right?
This dumb kid notices that you take differences of squares and get odd numbers, whatever.
So now is a big step.
This was the forward direction, right?
Input and there's the output.
But now the real reality, that's easy and important, but the more deep problem is, what if I give you b and ask for x?
So again, we're switching the direction here.
We're solving an equation now, or three equations and three unknowns, Ax=b.
So if I give you this b, can you get x?
How do you solve three equations?
We're looking backwards.
Now that won't be too hard for this particular matrix that I chose because its triangular will be able to go backwards.
So let me do that.
Let me take b to be.
It's a vector, it's got three components.
And now I'm going to go backwards to find x.
Or we will.
So do you see the three equations?
Here they are; x_1 is b_1, this is b_2, that difference is b_3.
Those are my three equations.
Three unknown x's, three known right-hand sides.
Or I think of it as A times x, as a matrix times x giving a vector b.
What's the answer?
As I said, we're going to be able to do this.
We're going to be able to solve this system easily because it's already triangular.
And it's actually lower triangular so that means we'll start from the top.
So the answers, the solution will be what?
Let's make room for it.
x_1, x_2, and x_3 I want to find.
And what's the answer?
Can we just go from top to bottom now?
What's x_1?
b_1, great.
What's x_2?
So x_2-x_1.
These are my equations.
So what's x_2-x_1?
Well, is b_2, so what is x_2?
b_1+b_2, right?
And what's x_3?
What do we need there for x_3?
So I'm looking at the third equation.
That'll determine x_3.
When I see it this way, I see those ones and I see it multiplying X 3.
And what do I get?
Yeah, so x_3 minus this guy is b_3, so I have to add in another b_3, right?
I'm doing sort of substitution down as I go.
Once I learned that x_1 was b_1 I used it there to find x_2.
And now I'll use x_2 to find x_3.
And what do I get again?
x_3 is, I'll put the x_2 over there.
I think you've got it.
b_1+b_2+b_3.
So that's the solution.
Not difficult because the matrix was triangular.
But let's think about that solution.
That solution is a matrix times b.
When you look at that, so this is like a good early step in linear algebra.
When I look at that I see a matrix multiplying b.
You take that step up to seeing a matrix.
And you can just read it off.
So let me say, what's the matrix there that's multiplying b to give that answer?
Remember the columns of this matrix-- well, I don't know how you want to read it off.
But one way is the think the columns of that matrix are multiplying b_1, b_2, and b_3 to give this.
So what's the first column of the matrix?
It's whatever I'm reading off, the coefficients really of b_1 here; .
And what's the second column of the matrix?
.
Good.
Zero b_2's, one, one.
And the third is?
.
Good.
Now, so lots of things to comment here.
Let me write up again here, this is x.
That was the answer.
It's a matrix times b.
And what's the name of that matrix?
It's the inverse matrix.
If Ax gives b, the inverse matrix does it the other way around, x is A inverse b.
Let me just put that over here.
If Ax is b, then x should be A inverse b.
So we had inverse, I wrote down inverse this morning but without saying the point, but so you see how that comes?
I mean, if I want to go formally, I multiply both sides by A inverse.
If there is an A inverse.
That's a critical thing as we saw.
Is the matrix invertible?
The answer here is, yes, there is an inverse.
And what does that really mean?
The inverse is the thing that takes us from b back to x.
Think of A as kind of a, multiplying by A is kind of a mapping, mathematicians use the word, or transform.
Transform would be good.
Transform from x to b.
And this is the inverse transform.
So it doesn't happen to be the discrete Fourier transform or a wavelet transform, it's a-- well, actually we could give it a name.
This is kind of a difference transform, right?
That's what A did, took differences.
So what does A inverse do?
It takes sums.
It takes sums.
That's why you see one, one and one, one, one along the rows because it's just adding, and you see it here in fully display.
It's a sum matrix.
I might as well call it S for sum.
So that matrix, that sum matrix is the inverse of the different matrix.
And maybe, since I hit on calculus earlier, you could say that calculus is all about one thing and it's inverse.
The derivative is A, and what's S?
In calculus.
The integral.
The whole subject is about one operation, now admittedly it's not a matrix, it operates on functions instead of just little vectors, but that's the main point.
The fundamental theorem of calculus is telling us that integration's the inverse of differentiation.
So this is good and if I put in B equal one, three, five for example just to put in some numbers, if I put in b equal , what would the x that comes out be?
.
Right?
Because it takes us back.
Here, previously we started, we took differences of , got .
Now if we take sums of <1, 3, 5>, we get .
Now we have a system of linear equations.
Now I want to step back and see what was good about this matrix.
Somehow it has an inverse.
Ax=b has a solution, in other words.
And it has only one solution, right?
Because we worked it out.
We had no choice.
That was it.
So there's just one solution.
There's always one and only one solution.
It's like a perfect transform from the x's to the b's and back again.
Yeah so that's what an invertible matrix is.
It's a perfect map from one set of x's to the x's and you can get back again.
Questions always.
Now I think I'm ready for another example.
There are only two examples.
And actually these two examples are on the 18.06 web page.
If some people asked after class how to get sort of a review of linear algebra, well the 18.06 website would be a definitely a possibility.
Well, I'll put down the open courseware website; mit.edu and then you would look at the linear algebra course or the math one.
What is it?
web.math.edu, is that it?
No, maybe that's an MIT-- so is it math?
I can't live without edu at the end, right?
Is it just edu?
Whatever!
So that website has, well, all the old exams you could ever want if you wanted any.
And it has this example and you click on Starting With Two Matrices.
And this is one of them.
Ok, ready for the other.
So here comes the second matrix, second example that you can contrast.
Second example is going to have the same u.
Let me put, our matrix I'm going to call it, what am I going to call it?
Maybe C. So it'll have the same u.
And the same v. But I'm going to change w. And that's going to make all the difference.
My w, I'm going to make that into w. So now I have three vectors.
I can take their combinations.
I can look at the equation Cx=b .
I can try to solve it.
All the normal stuff with those combinations of those three vectors.
And we'll see a difference.
So now, what happens if I do, could I even like do just a little erase to deal with C now?
How does C differ if I change this multiplication from A to C to this new matrix.
Then what we've done is to put in a minus one there, right?
That's the only change we made.
And what's the change in Cx?
I've changed the first row, so I'm going to change the first row of the answer to what?
x_1-x_3.
You could say again, as I said this morning, you've sort of changed the boundary condition maybe.
You've made this difference equation somehow circular.
That's why I'm using that letter C. Is it different?
Ah, yes!
I didn't get it right here.
Thank you, thank you very much.
Absolutely.
I mean that would have been another matrix that we could think about but it wouldn't have made the point I wanted, so thanks, that's absolutely great.
So now it's correct here and this is correct and I can look at equations but can I solve them?
Can I solve them?
And you're guessing already, no we can't do it.
Right?
So now let me maybe go to a board, work below, because I'd hate to erase, that was so great, that being able to solve it in a nice clear solution and some matrix coming in.
But now, how about this one?
OK. One comment I should have made here.
Suppose the b's were zero.
Suppose I was looking at originally at A times x equal all zeroes, What's x?
If all the b's were zero in this, this was the one that dealt with the matrix A.
If all the b's are zero then the x's are zero.
The only way to get zero right-hand sides, b's, was to have zero x's.
Right?
If you wanted to get zero out, you had to put zero in.
Well, you can always put zero in and get zero out, but here you can put other vectors in and get zero out.
So I want to say there's a solution with zeroes out, coming out of C, but some non zeroes going in.
And of course we know from this morning that that's a signal that it's a different sort of matrix, there won't be an inverse, we've got questions.
Tell me all the solutions.
All the solutions, so actually not just one, well you could tell me one, tell me one first
[UNINELLIGIBLE]
.
OK. Now tell me all
C, C,
C, C>.
Yeah.
That whole line through .
And that would be normal.
So this is a line of solutions.
Right.
A line of a solutions.
I think of  as in some solution space, and then all multiples.
That whole line.
Later I would say it's a subspace.
When I say what that word subspace means it's just this-- linear algebra's done its job beyond just .
OK.
So, again, it's this fact of-- if we only know the differences-- Yeah.
You can see different ways that this has got problems.
So that's C times x.
Now one way to see a problem is to say we can get the answer of all zeroes by putting constants.
All that's saying in words the differences of a constant factor are all zeroes, right?
That's all that happened.
Another way to see a problem if I had this system of equations, how would you see that there's a problem, and how would you see that there is sometimes an answer and even decide when?
I don't know if you can take a quick look.
If I had three equations, x_1-x_3 is b_1, this equals b_2, this equals b_3.
Do you see something that I can do to the left sides that's important somehow?
Suppose I add those left-hand sides.
What do I get?
And I'm allowed to do that, right?
If I've got three equations I'm allowed to add them, and I would get zero, if I add, I get zero equals-- I have to add the right-sides of course-- b_1+b_2+b_3.
I hesitate to say a fourth equation because it's not independent of those three, but it's a consequence of those three.
So actually this is telling me when I could get an answer and when I couldn't.
If I get zero on the left side I have to have zero on the right side or I'm lost.
So I could actually solve this when b_1+b_2+b_3=0.
So I've taken a step there.
I've said that okay, we're in trouble often, but in case the right-side adds up to zero them or not.
And if you'll allow me to jump to a mechanical meaning of this, if these were springs or something, masses, and these were forces on them-- so I'm solving for displacements of masses that we'll see very soon, and these are forces-- what that equation is saying is-- because they're sorta cyclical-- it's somehow saying that if the forces add up to zero, if the resulting force is zero, then you're OK.
The springs and masses don't like take off, or start spinning or whatever.
So there's a physical meaning for that condition that it's OK provided if the b's add up to zero.
But of course, if the b's don't add up to zero we're lost.
Right yeah.
OK.
So Cx=b could be solved sometimes, but not always.
The difficulty with C is showing up several ways.
It's showing up in a C times a vector x giving zero.
That's bad news.
Because no C inverse can bring you back.
I mean it's like you can't come back from zero.
Once you get to zero, C inverse can never bring you back to x, right?
A took x into b up there, and then A inverse brought back x.
But here there's no way to bring back that x because I can't multiply zero by anything and get back to x.
So that's why I see it's got troubles here.
Here I see it's got troubles because if I add the left-sides I get zero.
And therefore the right-sides must add to zero.
So you've got trouble several ways.
Ah, let's see another way, let's see geometrically why were in trouble.
OK, so let me draw a picture to go with that picture.
So there's three-dimensional space.
I didn't change u, I didn't change v, but I changed w to minus one.
What does that mean?
Minus one sort of going this way maybe, zero, one is the z direction, somehow I change it to there.
So this is w* star maybe, a different w. This is the w that gave me problems.
What's the problem?
How does the picture show the problem?
What's the problem with those three vectors, those three columns of C?
Yeah?
[UNINTELLIGIBLE]
There in the same plane.
There in the same plane.
w* gave us nothing new.
We had a combinations of u and v made a plane, and w* happened to fall in that plane.
So this is a plane here somehow, and goes through the origin of course.
What is that plane?
This is all combinations, all combinations of u, v, and the third guy, w*.
Right.
It's a plane, and I drew a triangle, but of course, I should draw the plane goes out to infinity.
But the point is there are lots of b's, lots of right-hand sides not on that plane.
OK. Now if I drew all combinations of u, v, w, the original w, what have I got?
So let me bring that picture back for a moment.
If I took all combinations of those does w lie in the plane of u and v?
No, right?
I would call it independent.
These three vectors are independent.
These three, u, v, and w* I would call dependent.
Because the third guy was a combination of the first two.
OK, so tell me what do I get now?
So now you're really up to 3-D. What do you get if you take all combinations of u, v, and w?
[INAUDIBLE]
Say it again.
The whole space.
If taking all combinations of u, v, w will give you the whole space.
Why is that?
Well we just showed-- when it was A we showed that we could get every b.
We wanted the combination that gave b and we found it.
So in the beginning when we were working with u, v, w, we found-- and this was short hand here-- this said find a combination to give b, and this says that combination will work.
And we wrote out what x was.
Now what's the difference-- OK-- here.
So those were dependent, sorry, those were independent.
I would even called those three vectors a basis for three-dimensional space.
That word basis is a big deal.
So a basis for five-dimensional space is five vectors that are independent.
That's one way to say it.
The second way to say it would be there combinations give the whole 5-dimensional space.
A third way to say it-- see if you can finish this sentence-- this is for the independent, the good guys-- if I put those five vectors into a five by five matrix, that matrix will be-- invertible.
That matrix will be invertible.
So an invertible matrix is one with a basis sitting in it's columns.
It's a transform that has an inverse transform.
This matrix is not invertible, those three vectors are not a basis.
Their combinations are only in a plane.
By the way, a plane as a subspace.
A plane would be a typical subspace.
It's like fill it out.
You took all the combinations, you did your job, but in that case the whole space would count as a subspace too.
That's the way you get subspaces, by taking all combinations.
OK, now I'm even going to push you one more step and then this example is complete.
Can you tell me what vectors do you get?
All combinations of u, v, w. Let me try to write something.
This gives only a plane.
Because we've got two independent vectors but not the third.
OK.
I don't know if I should even ask.
Do we know an equation for that plane?
Well I think we do if we think about it correctly.
All combinations of u, v, w* is the same as saying all vectors C times x, right?
Do you agree that those two are exactly the same thing?
This is the key, because we're moving up to vectors, combinations, and now comes subspaces.
If I take all combinations of u, v, w*, I say that that's the same as all vectors C times x, why's that?
It's what I said in the very first sentence at 4 o'clock.
The combinations of u, v, w*, how do I produce them?
I create the matrix with those columns.
I multiply them by x's, and I get all the combinations.
And this is just C times x.
So what I've said there is just another way of saying how does matrix multiplication work.
Put the guys in it's columns and multiply by a vector.
So we're getting all vectors C times x, and now I was going to stretch it that little bit further.
Can we describe what vectors we get?
So that's my question.
What b's, so this is b equal b_1, b_2, b_3 do we get?
We don't get them all.
Right, we don't get them all.
That's the trouble with C. We only get a plane of them.
And now can you tell me which b's we do get when we look at all combinations of these three dependent vectors.
Well we've done a lot today.
Let me just tell you the answer because it's here.
The b's have to add to zero.
That's the equation that the b's have to satisfy.
Because when we wrote out Cx we notice that the components always added to zero.
Which b's do we get?
We get the ones where the components add to zero.
In other words that's the equation of the plane you could say.
Yeah.
Actually that's a good way to look at it.
All these vectors are on the plane.
Do the components of u add to zero?
look at u.
Yes.
Do the components of v add to zero?
Yes.
Add them up.
Does the components of w*, now that you've fix it correctly, do they add to zero?
Yes.
So all the combinations will add to zero.
That's the plane.
That's the plane.
You see there are so many different ways to C, and none of this is difficult, but it's coming fast because we're seeing the same thing in different languages.
We're seeing it geometrically in a picture of a plane.
We're seeing it as a combination of vectors.
We're seeing it as a multiplication by a matrix.
And we saw it sort of here by operation, operating and simplifying, and getting the key fact out of the equations.
Well OK.
I wanted to give you this example, the two examples, because they bring out so many of the key ideas.
The key idea of a subspace.
Shall I just say a little about what that word means?
A subspace.
What's a subspace?
Well, what's a vector space first of all?
A vector space is a bunch of vectors.
And the rule is you have to be able to take their combinations.
That what linear algebra does.
Takes combinations.
So a vector space is one where you take all combinations.
So if I only took just this triangle that would not be a subspace because one combination would be 2u and it would be out of the triangle.
So a subspace, just think of it as a plane, but then of course it could be in higher dimensions.
You know it could be a 7-dimensional subspace inside a 15-dimensional space.
And I don't know if you're good at visualizing that, I'm not.
Never mind.
You you've got seven vectors, you think OK, their combinations give us seven-dimensional subspace.
Each factor has 15 components.
No problem.
I mean no problem for MATLAB certainly.
It's got what, a matrix with a 105 entries.
It deals with that instantly.
OK, so a subspace is like a vector space inside a bigger one.
That's why the prefix sub is there.
Right?
And mathematics always counts the biggest possibility too, which would be the whole space.
And what's the smallest?
So what's the smallest subspace of R^3?
So I have 3-dimensional space-- you can tell me all the subspaces of R^3.
So there is one, a plane.
Yeah, tell me all the subspaces of R^3.
And then you'll have that word kind of down
[UNINTELLIGIBLE]
A line.
So planes and lines, those you could say, the real, the proper subspaces.
The best, the right ones.
But there are a couple more possibilities which are?
[UNINTELLIGIBLE]
A point.
Which point?
The origin.
Only the origin.
Because if you tried to say that point was a subspace, no way.
Why not?
Because I wouldn't be able to multiply that vector by five and I would be away from the point.
But the zero subspace, the really small subspace that just has the zero vector-- it's got one vector in it.
not empty.
It's got that one point but that's all.
OK, so planes, lines, the origin, and then the other possibility for a subspaces is?
The whole space.
Right.
Right.
So the dimensions could be three for the whole space, two for a plane, one for a line, zero for a point.
It just kicks together.
How are we for time?
Maybe I went more than a half, but now is a chance to just asked me, if you want to, like anything about the course.
Is at all linear algebra?
No.
But I think I can't do anything more helpful to you then to for you to begin to see when you look at a matrix-- begin to see what is it doing.
What is it about.
Right, and of course matrices can be rectangular.
So I'll give you a hint about what's coming in the course itself.
We'll have rectangular matrix A, OK.
They're not in invertible.
They're taking 7-dimensional space to three-dimensional space or something.
You can't invert that.
What comes up every time-- I sort of got the idea finally.
Every time I see a rectangular matrix, maybe seven by three, that would be seven rows by three columns.
Then what comes up with a rectangular matrix A is sooner or later A transpose sticks it's nose in and multiplies that A.
And we couldn't do it for our A here.
Actually if I did it for that original matrix A I would get something you'd recognize.
What I want to say is that the course focuses on A transpose A. I'll just say now that that matrix always comes out square, because this would be three times seven times seven times three, so this would be three by three, and it always comes out symmetric.
That's the nice thing.
And even more.
We'll see more.
That's like a hint.
Watch for A transpose A.
And watch for it in applications of all kinds.
In networks an A will be associated with Kirkoff's voltage law, in A transpose with Kirkoff's current law.
They just teamed up together.
We'll see more.
Alright now let me give you a chance to ask any question.
Whatever.
Homework.
Did I mention homework?
You may have said that's a crazy homework to say three problems 1.1.
I've never done this before so essentially you can get away with anything this week, and indefinitely actually.
How many is this the first day of MIT classes.
Oh wow.
OK. Well welcome to MIT.
I hope you like it.
It's not so high pressure or whatever is associated with MIT.
It's kind of tolerant.
If my advises ask for something I always say yes.
It's easier that way
[LAUGHTER]
And let me just again-- and I'll say it often and in private.
This is like a grown-up course.
I'm figuring you're here to learn, so it's not my job to force it.
My job is to help it, and hope this is some help.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this is the first true review session in 18.085.
The last Wednesday, the first Wedneday afternoon was a brief review of topics in linear algebra.
But now we're into the course.
We've done four lectures on the first four sections of the textbook and one homework problem in and back and a second homework set for next Monday.
So I'm ready for any questions, including questions that are on the homework if necessary.
But anything at all.
Or maybe I'll just ask whether the pace, so this is really informal.
Is the pace of the course, now today's lecture had a lot in it as I realized when I saw that the board with still full of 18.085 and there was a little more still to do because we didn't finish the matrix part.
But are you ok with the sort of speed of the course?
So that'll be one question.
And then, what about specifics?
Somebody start off if you will.
Anybody.
Yes, thanks
Well, actually I had a question about the lecture earlier today
Okay, go ahead
I was just going to look it up in the text, but
That's alright
But haven't had a chance to.
But, ok, so I'm not really sure how you take the initial conditions and apply them to the ramp function to actually get a solution
So that's what luckily happens to be still here on the board, right?
We've got these boundary, I would call them boundary conditions.
So this is definitely, we're in a part of math that's about boundary value problems more than time, isn't in the picture.
I mean later time will get into the picture.
So with this particular example, the general solution is a standard ramp at the point a where things are happening.
Plus the C+Dx, the usual.
So that's a particular solution.
That particular solution has the right behavior at the impulse.
By the right behavior, I mean that the second derivative of the ramp is a delta and when I put the ramp at a, then the delta will show up at a.
And when I put on that minus sign it'll mean that the second derivative is minus the delta.
So the slope will step down.
So that's a particular solution.
But that by itself, what does that equal at zero?
And what does that equal at one if you remember the ramp?
So let me just draw that ramp again.
So the ramp was really based on, centered at the point a.
And I'll put it with a minus sign.
So it came along there and down there.
And now suppose this is one end of our interval and that's the other end.
So is that ramp the answer to our problem?
No.
Well it happens to satisfy this boundary condition, happens to start out at zero.
But it doesn't end up at zero.
So just like every particular solution we need a little more.
We have to include more solutions because that was only one.
All those are equally good solutions.
If I add C+Dx, the second derivative of that is zero, so it doesn't spoil anything at all.
On the contrary, it adds more solutions.
I mean the great thing we're using here is that our equations are linear and when zero's on the right-hand side-- notice there's no arbitrary constant.
I'm not putting an arbitrary constant on that particular solution.
Is one particular solution plus a subspace if I use that language.
A lot of solutions to the, all the solutions to the problem with zero on the right-hand side.
And these are the solutions, these are the null space guys, the ones that have zero on the right-hand side.
Now we need those.
And the effect of those will be to move that ramp.
And effectively, what are they going to do?
They're going to swing that ramp around, it'll stay a ramp.
But instead of being level here, it'll go up.
And instead of going down there, it'll go down there.
It'll be the same ramp, with still that slope dropped by one.
Slope went down by minus one and that's not going to change here.
The slope will still go down by minus one.
But now I've just adjusted it so that it goes through the, it satisfies the boundary conditions.
And in the second problem with the free end, again, here's my ramp.
But now I'm going to adjust it.
And what happened?
It just needed adjustment upwards.
Because this was the zero, this was the u=0 fixed guy.
And now if I'm doing u'(0)=0, the free guy, I can lift the whole thing up.
So you see, I just lifted it up to the point where it came out right there.
So in this case I just needed a C. And in this case I just needed a Dx.
And in other cases I might have needed both.
A little bit of C and a little of Dx.
Anyway that's what was, maybe that's sort of repeating what we did today a little bit.
But mechanically it's just, we've got a particular solution and we've got the complete solution and then we just have to choose C and D and we've got two conditions to do it with, the two boundary conditions.
Could have other boundary conditions.
Now, what about, here's a, yeah.
I guess what will often happen in these review sessions is I get on a roll and I just keep, carry on with it.
You know, you start me with a question and I can't stop.
So I'll go a little longer but then I really will stop, ready for the next one.
So we haven't discussed the free-free.
u(0) u', sorry, free is u'.
That's free at the left and free at the right.
What's up with that?
What's the solution to that?
Again, I'm looking for -u'' equal an impulse.
With those two boundary conditions, free-free.
And do you know what's going to happen?
No solution.
Now it'd be interesting to see why not.
Why no solution?
Well one way to do it is try.
Other right-hand sides could have a solution.
So it's not just that this is something the matter here.
Specifically, that right-hand side and most right-hand sides will fail.
But let's just see it with this one.
Why does that fail?
Well you can see I can't, the slope here is zero and the slope here is minus one.
If I adjust those I can't get, this is asking me to get slope zero at both ends.
I can't do that, right?
Yeah, just not possible for me to.
This will add a straight line but it's the same straight.
I can't get a ramp that comes flat at both ends because there's, once I say what it's doing at the ends, I've got it.
And you see what I mean?
I'm looking for one that starts flat and ends flat and that's not in my family of solutions.
That problem just doesn't have a solution.
So that's one way to do it is look at the solutions and realize you can't satisfy both boundary conditions.
Another way might be this.
This is a little bit deeper way and it leads to something better.
Suppose I take the integral.
So this is an equation that's supposed to hold.
Let me integrate both sides from zero to one.
So there's the idea I'm putting in now.
To go a little further, to discover when this has a solution.
Or let me take more generally, -u''=f(x).
So some other load.
Not necessarily a point load, not necessarily a uniform load, but maybe some other load.
And now my boundary conditions, I'm trying to do free-free.
And usually no solution.
But let's just see why and when there might be a solution.
The key idea is integrate from zero to one.
What do I get on the right-hand side?
Well, I get the integral of f, whatever it is, and I would call that the total load.
Fair enough?
The total load of if it was a delta function, the total load would be one and it would be all in one spot.
If it was uniformly over the whole interval, well I guess that would also integrate to one, so the total load would be one spread out.
But it could be a mixture of the two, could be a few delta functions, whatever.
What happens when I integrate the left side from zero to one?
Can you do that one?
The integral from zero to one.
There's that dopey minus sign.
u''dx.
What do I get?
Why do I say that's a good idea?
If I integrate the second derivative I get the first derivative.
The integral of the second derivative will be the first derivative with the minus.
So it's minus the first derivative.
And what do I do now?
I plug in the end points, right?
You integrate.
I'm integrating zero to one.
So I've found the integral.
I've put zero to one in there.
So what is that?
That's minus the derivative at one plus the derivative at zero.
And now, what's that?
That's zero.
By my boundary conditions, that's zero.
So what have I found?
I've found that if these are the boundary conditions, then when I integrate the left side it's going to give me zero.
So when, what loads could be ok?
What loads f(x) could allow me to solve this equation?
The condition will be I need, what do I need for the total load to be able to solve this equation?
The integral of the left side was zero so the integral of the right side had better be zero.
So that's the condition.
If I have these boundary conditions, then my problem is singular.
Usually no solution.
It's like having a singular matrix.
It's like having this particular singular matrix, of course.
Whoops, not that one.
Let me get the plus sign in the right position.
That's a plus.
-1; -1, 2, -1; -1, 1.
Right?
This is the discrete version with a zero slope at both ends.
It's our T matrix.
No, what matrix is it?
B.
It's our B matrix, both ends.
I'll just come back here and then I'll do the discrete one.
So tell me a load that we could handle?
A load we could handle.
So the integral has to be zero.
So suppose my load has a delta function at a.
Well that integral is one.
So can you fix that, change that load or do something, maybe put on another load to get a total load of zero?
What shall I do?
Add another guy with a minus sign.
In other words, maybe this, a delta function at some other point B.
Well, that would do it.
I believe we could solve that problem.
Even with these bad boundary conditions.
We could solve that problem.
Because the total load would be one from that, minus one from that, the total load would be zero.
In other words, what would are solution look like?
It has to start with zero slope.
So it would buzz alone to a and then after b.
And what does it have to do here?
If I graph the solution to this guy from zero to one it starts with, it's free, so nothing's happening until I get to a.
Then what has to happen?
Let's see, if I'm graphing u, it'll be ramp.
Right?
It'll be a ramp, yeah.
Because I've two derivatives.
And it has to ramp down by one, so it'll ramp whatever it does.
I don't know where it stops.
Where does it?
Wait a minute.
I haven't practiced this.
So I start from the other end.
The other end is flat.
What's up?
They gotta meet here.
Oh, the other end is flat, but not at zero!
Dumb, stupid.
Right.
Ok.
Yes, the other end is flat, right.
And it's, oh yeah, look!
Oh, wonderful.
You see.
That slope dropped by one and the slope there increased by one back to zero.
Slope was zero.
It dropped to minus one because of that load.
Now it increased back to zero because of this load with the minus sign and there's a solution.
So that's a solvable problem.
Well, you say, okay, that was a little surprising to get an answer for a singular problem.
And no, it can happen.
If we have a total load zero it'll happen.
But, there is still a but, that's not the only answer.
That picture is a solution.
But not the only one.
So what my point is going to be, that when the problem is singular, if there's an answer, you say great.
But then something has to go wrong and what goes wrong is too many answers.
So tell me some more, what other graphs would draw solutions to this problem.
I could shift, I could lift the whole thing.
Here I've got a plus C that I haven't used.
I could just do the whole thing higher up.
Any of these.
These would all work.
It's like temperature.
I don't have an absolute temperature here.
All I've got is, I would have to, it's not determined because there's a plus C that, the plus C satisfied everything.
A plus C, a constant has zero slope, it has zero slope, its second derivative is zero.
So it's like, unseen by this equation.
And similarly can I just make the analogy as I always like to do with discrete stuff, so suppose I, tell me a right-hand side that we think would probably, is this going to be the same story for this guy?
Yes.
If I add those, where I integrated there, here I would add and I get zero, zero, zero.
So this has to add to zero if there's a solution.
So let me put for example, .
That would be kind of like our delta function in one direction and our delta function in the other.
I believe I can solve that problem.
So I'm just carrying, because I always want you to see the discrete one as well as the continuous.
Continuous involve this integration.
The finite one just involves adding.
The left side adds to zero so the right-hand side better add to zero.
That right-hand side does.
Tell me a solution.
Well, let me start out with a seven there.
What's the next guy going to be?
See, I want seven.
Whatever I put there, I better have a seven there, right?
Seven, seven, good.
Minus seven, plus 14, oh geez.
I didn't know this was going to happen.
No, I want to get the answer one.
What number goes there?
Six, is it six?
It's six, yeah, good.
Minus seven, 14, minus six is that.
And now my claim is that we'll come out right on the third equation.
So far I've just matched the first two.
Now this one gives minus seven, plus six, that's minus one, good.
So there's a solution.
And I'll leave this problem alone if you tell me the rest, other solutions.
That  was a solution to a singular problem with a right-hand side that had total load zero, so it was ok.
But now that's a solution, but there are more.
Tell me another one.
I can shift it, right?
I could make it .
I could add ten to everything.
Right?
That's the plus C that I could do over there.
That can't change because 17 - 17 is still zero.
-17 + 16 will still be minus one.
all good.
So actually that just like helps our intuition and physically my intuition is this.
That I've got this bar and nothing's holding it.
So if I put a weight on it, nothing to hold it, it'll just, rigid motion will take it out of sight, no good.
But if I put another equal weight on it, no it's not a weight.
What do I call it?
If I lift it at that point, that's the other delta function that's going the other way, then it will sit there.
But it would still be in equilibrium if I just moved it up to there or moved it as I like.
I don't know if that is kind of a dumb picture.
But it's saying what we've said from math.
Well, you see where you're question lead.
Yeah, thanks.
No, the integral, it was-- Watch what we integrated.
We integrated u''.
So that's not the area.
We integrated u'' and got, it's integral was u'.
So that just told us that a difference in slopes at the ends, yeah.
Good, because our intuition automatically is if we're integrating something, we're finding an area.
But here, if it was u , then I'd be finding the area under u, but we integrated the second derivative.
Right, good.
Now let's change the subject.
Yes, please.
Yeah, I guess so.
I'll try.
Let's see.
So my discrete equation was, like -u. Yeah, so let's back up to the beginning.
We've got this minus sign and we're using a second difference.
So second differences have coefficients one, minus two and one.
Now I'm reversing the signs because of my minus.
So I have -u at some point.
Let's take that as the point to the left.
Two u's at what I'll think of as the center point.
-u_(i-1) is the load at that center.
That center point is i times delta x.
That's where I would be looking.
So now I'm using subscript.
It's a little bit of practice then to take subscripts, take this way of writing the equation and convert it to a matrix way.
It's usually clearer once you see it as a matrix.
Now this is happening at all the points.
At i=1, let's say, I have -u_0+2u_1-u_2 is f at, agrees with the load at the first mesh point.
That's the center, the point h, delta x.
And then if I want to back up further, I would have -u_-1, but that doesn't really exist, plus 2u_0-u_1 should match the load at zero.
And so on forward.
But now I want to put in the boundary condition.
That's what you want me to do, right?
Put in this boundary condition.
So what am I going to take as boundary condition?
It has to be some approximation to u'(0)=0.
Maybe I'm never going to get to minus one.
Maybe I don't need minus one.
That's right, yeah, exactly.
We did.
That's what we knew about it.
Sending it forward, we knew about forward difference, so I chose to do it.
But then I think better of it.
I chose to do it because it made the point that we, that at that boundary we were introducing a higher order error, first order error that's going to wreck things.
I mean, it's going to spoil the, this is second order accuracy.
And, but let me do that first order.
So what shall I take?
I'm going to approximate that by u-- Shall I take this one as I did in class?
Yes.
Ah, plus one, thanks, plus one, right, thank you.
Thank you, good.
Okeydoke.
Alright.
This is how we got to that equation.
If I now bring in this boundary condition-- I guess I don't have to, let me take your eye off of that guy for the moment, I think.
We're getting beautiful music here.
Is it coming out of this box or?
No.
Anyway.
so I'm going to use this boundary condition to say, well ok, if u_1 is u_0, I'm going to replace this u_0 by u_1.
This is the direct way.
I replaced that u_0 by u_1 in that first equation.
And then what I have is -u_1 and 2u_1, so that's the one and I have the -u_2.
So do you see that that equation, when I put those together into a one, is going to, if this is u_1, this is u_2, this is u_3 onwards, that first equation is u_1-u_2 and that's what I've got.
This is u_1-u_2.
So I did it.
I got to that matrix.
The matrix is actually quite an important matrix.
But from the point of view of accuracy in solving this differential equation, it's not the greatest.
It's lost accuracy at that point.
But the way to recover it turned out to be just a small adjustment at the boundary, so not a problem.
Thanks.
That's good.
Yes, thanks.
Sorry?
When the boundary-- sorry.
Two boundary conditions at the same point?
That's a good question.
So when would we have two?
So instead of a boundary condition at zero and a boundary condition at one, you're putting them, is that what you mean?
Put both boundary conditions at the end.
Ok.
So that would be, that would happen, I would think that would be more, it would be very typical in a, let me see if there's some space here, yeah.
That would be very typical and we will do it, can I change x to t?
Because that's what, if I have some.
What does this problem look like?
And u(0)=0 and u'(0)=0.
Both at the same, at the start equals zero or whatever.
So what kind of a problem is that?
Now these are, I would say, initial values.
Initial values instead of boundary values, I now have initial values.
And can I solve it?
Yes.
So I'm starting at time zero.
This is t=0.
I'm starting at rest.
No velocity and actually no displacement and just going forward in time.
So I could solve that differential equation.
I'd be interested in the corresponding difference equation.
All fine.
It's a different category of problem.
This is an initial value problem.
It's like tracking some mass that's, some satellite.
So that's what you're doing in tracking a satellite or a planet or something.
Yeah, tracking a planet or a satellite.
You're solving equations like this.
Forward in time.
You know the initial position and you know the forces acting on it.
Probably gravity.
And you go forward in time.
Yes.
What would the matrix be?
Good question.
What would the matrix look like?
So an electrical engineer would call a problem like this, and the kind of matrix that I'm going to write down, I think, would be called causal.
That word just popped into my head, so let me mention it.
You know, part of science and engineering, a big part of it is learning language, learning words.
And you have to learn sort of the math language and the engineering language for whatever you're focusing on.
But it's good to also to know a few other languages.
Electrical engineering languages of filters and causal and other things that we'll see are important.
What would the matrix look like?
Here's what I think it would be.
I've made this a plus there just so I'll have to remember that.
I think, so I'm looking at u_0, u_1, no.
Well, u_0 I actually know, so let me start with u_1, u_2, u_3, u_4.
What would a typical equal sum, right side, f_1, f_2, f_3, f_4.
1, What do you think, what kind of a matrix am I going to get?
Before I put it in there.
This is a good question.
What's the shape of this matrix?
It's going to be triangular.
Instead of being symmetric it's going to be triangular.
I'm going to find, let's see, a typical value would be, say, u_3 because I've used a plus sign, oops!
I can't make myself do it right.
1, -2, 1.
That would say u_3-2u_2+u_1 would be the new force.
This is the kind of thing we're going to get.
One, maybe one something.
I don't know what this is.
This is up in the boundary, in the initial values.
But from now on it'll be below the diagonal.
It'll be 1, -2, 1.
Do you see?
We're marching.
We're marching forward.
We start by knowing these and then the equation tells us the next one.
Then the equation tells us the next one.
That's what initial value problems do.
You're told how you begin and you take a step, you take a step, take a step every time.
And the new value just needs to know the older values.
Do you see the big difference between that and our problems here?
Our problem is looking left and right looking for back and forward.
Back for one condition, forward for another.
We start with one, but we're, it's more of a, it's like a hitting problem.
We start forward, marching forward in our problems, but we have to hit the other end correctly.
We don't know the slope, we don't know the starting slope, we know what we want to hit.
Whereas these problems, we're told how we start and we just follow it in time.
So that's the difference here.
Yeah, sure, okay.
That's true.
So this'll be known.
Yeah, that'll be known.
Yeah.
u_1 will also be known.
Yeah, and really, maybe I should have got, let me put even the other known one.
So we know this, we know this.
So those are sort of not in our, yeah, that shouldn't be in our problem somehow.
No, I think, what would we get in the end?
You're always looking backwards.
That's the point.
Lower triangular matrices are always looking, they only look backwards for earlier values and then they give you the current value.
So that's why lower triangular matrices are so easy to invert.
No problem.
If it's lower triangular, you just, like, march forward.
And if it's upper triangular, which way do you march?
So if you have an upper triangular problem, suppose I gave you the problem, let me make it upper triangular.
So x+y+z=7.
2y+3z=12 and z=17.
So that's upper triangular.
Where do we start in solving that one?
From the bottom.
From the right-hand end, the bottom.
And we march backwards in time.
And what I was saying about A, well L times U, yeah, this is worth seeing.
What I was saying about A=LU, it was, you remember that?
Those letters?
What that was saying was that this matrix that's looking both ways can be written as a product of a matrix L that looks behind for old values and you can go forward with it.
And a matrix U, like this one, this upper triangular, 1, 1, 1, zeroes below that diagonal, that you go backward with.
Somehow that's appealing.
That's like aesthetic to break up a two-way problem into a problem like marches one way and then the other.
And of course, that's what elimination aims for, is this problem that it can solve by, the words would be back substitution.
When you've started with your original problem, got to this one, then you just have a, back substitution, you go backwards.
Oh, so much, I'll mention the Kalman filter.
That's a similar process of going forward, that's called prediction.
Going backward, that's called smoothing.
And so, Kalman had the great idea that he could break these problems that were fundamental in space computations for prediction and smoothing.
Once again, we've got off.
Yes?
Oh, the beam.
Let me help you even more before the question.
I said it's better to draw the beam this way.
I like the beam better this way.
Because the point of the beam problem is loads are acting, and we'll see this, of course later, loads are acting perpendicular to the direction of the beam.
That's why the beam bends.
So it'll bend a little, right?
And that is what leads us, it's bending moments and other stuff.
If you haven't met beams, well, it'll be great to just have a very, half a lecture about, or maybe a lecture about beams.
That gives a fourth order equation that I'll write down again.
Fourth derivative equal the load.
Now, ready.
Yeah, now here I don't have the negative sign.
Because once I've got second derivatives twice, so the second derivative is, in some way, negative.
I'll complete that sentence in a second.
Somehow the second derivative, which is the guy that has the 1, -2, 1, somehow that's a negative thing.
But fourth derivative is second derivative of the second derivative.
Yeah, do you want to tell me what the numbers would be?
As long as we're wildly looking forward to fourth derivatives, just, it helps.
Do you want to guess what will a typical row of the matrix B when I go to finite differences, fourth differences?
Probably you've never seen a fourth difference.
You may not have seen second differences before.
That was a big deal, then, to introduce second differences.
Those 1, -2, 1's.
That was second differences.
Fourth?
Yeah, 1, 4, 6, 4, 1 with minus sign.
1, -4, 6, -4, and 1.
In some way, I would get that by squaring this guy.
So that would be a fourth difference.
Oh, what's the deal with boundary conditions?
What are you figuring on beams, beam problems for a fourth order equation and a matrix that's stretching out further.
What's going to happen at the left-hand boundary?
I guess my specific question is, How many boundary conditions do I now need?
Four.
And the typical is two at each end.
That's the balanced way.
That's the way that would make this matrix sort of symmetric.
So I have maybe at this end I say it's held at zero and maybe it's just sitting on a log there.
Right?
That boundary condition I would call simply supported.
That boundary condition says that u(0)=0.
Because it's sitting there.
And but the slope doesn't have to be zero.
What does have to be zero there?
Yeah, sort of the bending moment.
Nobody's here twisting it, right?
So the other condition in that picture would be second derivative equal zero.
Maybe my point is that now you see what I said before, that the getting the boundary conditions into the problem is often the hardest part.
Because I have to replace u(0)=0, that shouldn't be too hard to do.
But I have to use this other condition somehow, it's going to screw up the 1, -4, 6, -4, 1.
I'll have two boundary rows at the top, two boundary rows at the bottom.
I don't want to go further today.
But I think maybe just mentioning this gives you the picture of sort of the how things fit together.
We would still have some nice constant diagonals in the middle, but now we'll have two boundary rows at each end.
So that's something to come.
Yes, now back to reality which is any questions.
Lower triangular guy, yeah.
What do I mean by marching forward?
So let's see.
I'll replace this.
Let's see it better.
I would replace this by maybe u, I'll use a different letter, n+1 at u_(n+1)-2u_n+u(n-1) is some right-hand side if there's a force acting on my thing.
So f_n maybe.
By marching forward, I just mean that this equation, that I can go in order.
I can start with u_0 and u_1.
They come from the boundary conditions.
Then this equation will tell me u_2.
I use the equation.
With n as one.
This says u_2, some u_1's, some u_0's, f_1's, all that I know.
In other words, once I get started, I'm on a roll.
If I have two boundary conditions to get me started, then the equation tells me u_2.
And then the next time, u_3-2u_2+u_1.
I can find u_3.
So I can get those, I can go forever.
If you give me enough to start on, two things to start, then I march forward.
Whereas in our problems, we've only got one thing to start on and we've got one goal to hit.
And that's why we have to solve the whole system together.
This is, we can solve it step-by-step.
This is way faster of course.
To be able to just go forward in time.
I'll mention that the topic of initial value problems and finite differences for them, we can't get to that.
So we're seeing a little bit here, but it's done properly in 18.086 in the second semester is the initial value problem start part.
And that has it's own interesting questions.
Somehow we've talked about fourth order equations, initial value problems.
But no homework problems.
So I'm ready for, or even related.
But that's fine with me.
Is there a question?
Yeah, thanks.
Or it doesn't have to be a homework question, another question.
Oh, good question.
You mean I should just send the homework out to Natick where MATLAB is.
Do you know that MATLAB is just 15 miles away?
I almost get there, I live 2/3 of the way there.
Yeah, so we could just send the whole thing out there and get it back.
That would save a lot of work.
I suppose, I'm ok. Why should I say no?
Anything MATLAB can do and you can make it do, I'm ok with that.
I don't see that you have to do things by hand if you've got a better way.
That's ok. And then probably the answer gets printed and you can graph it.
So that's fine.
So I mean, somehow a course like this has got two parts to it.
Applied math has two parts to it.
The modeling part, set up the equation, think, what is it you're supposed to do.
And then, step two is do it.
The numerical part, the computing part.
And that's where MATLAB, Python, Fortran, whatever, is going to do a lot of the heavy lifting.
Was there another question?
So that first homework was certainly very general intentionally.
Because I'm hoping you will read the book.
The lecture, you'll be able to match the lectures with the book even later on when they separate a little or separate more.
You'll see what we're doing.
And those, the homework problems, you should look at some of the others just to see.
Do I know how to do that?
Right.
Let's stop here for this first review.
I'm sure we'll have more, questions will build up for the second week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIt OpenCourseWare at ocw.mit.edu
Shall we just start on this review session?
So, any questions on anything from Chapter one, anything from those first seven lectures is very, very welcome.
So this morning finished the serious part of what we'll do in the chapter with positive definite matrices.
And we'll see a lot of those fortunately.
They're the best.
So questions about, I hope you look in the book at other problems in the problem sets as well as the ones I suggest.
And then I can, anyway.
Ready for any questions.
Ok.
Which problem is it?
In section?
Section 1.6, problem 27, what have I done there?
Oh, ok, that's good.
So it's about positive definite matrices.
May I just put on the board what the central question is?
Just put these matrices up.
We're given that H and K are positive definite.
And then the question is, what about these block matrices.
Do I call them M and N?
One is the block matrix that looks like that.
And another one is the block matrix that looks like this.
So those are both symmetric.
We're allowed to ask, are they positive definite or negative definite because they passed the first requirement.
They're symmetric.
We can discuss them.
Because of course H and K each were symmetric.
The transpose of this would bring K transpose down here, but that's K, so all good.
So the question now.
Of these guys to those guys I guess, yes.
Good question.
So this guy has, let's take eigenvalues first.
So this guy has some eigenvalues, say lambda_1 to lambda_n.
And this guy, we'll suppose they're the same size, so they don't have to be.
Maybe I shouldn't, but I will.
This has some other eigenvalues, maybe e_1 to e_n for eigenvalue.
And then the question is, okay, what about the eigenvalues of that combination?
And what about this?
So it's a good question, I think for all of us to practice what just came up in the lecture.
The idea of block matrices.
So looking here at eigenvalues I could also look at pivots.
Pivots would be interesting to look at, too.
Maybe I'll start with pivots.
Can I?
Did you think?
What would be the pivots of M?
If I start elimination on M what will I see for pivots?
Well, I start up in the usual left-hand corner and work down.
So what am I going to see first?
I'm going to see the pivots of H. It won't even know, by the time I had halfway there, it won't even have seen K. And then, that'll be fine.
And then this will be, what's going to happen?
This is all zeroes.
So never get touched, right?
So when I get down to the second half I see all zeroes here.
K is still going to be sitting right there.
Nothing happened.
Because when I did these eliminations nothing changed with K. So the rest of the pivots will be the pivots of K. Good.
Now, we might hope for the same thing with eigenvalues and probably that's going to happen.
This is like a diagonal matrix.
And actually, what words would I use?
Block diagonal.
I'd call that matrix block diagonal.
And those are very nice matrices.
That tells us that the big matrix, for all practical purposes, is breaking up into these smaller blocks.
Actually MATLAB will search for a way to reorder the rows and columns to get that in case it's possible.
So here it's in front of us.
Let's see if we can figure out.
That lambda_1 I believe is also an eigenvalue of M. So it was an eigenvalue of H. So that this, the fact that it has that eigenvalue lambda_1 means what?
That H times this times some vector y is lambda_1*y, right?
If that's an eigenvalue it's got an eigenvector and let's call it y.
Now this is a good question.
I believe this block matrix also has eigenvalue lambda_1 and what's its eigenvector?
What could I multiply M by to get lambda_1 times the same thing?
Can you see what?
Of course I'm thinking that y is going to help but it's grown now.
So what would be the eigenvector here?
When I multiply by M it'll just come out right with the same eigenvalue?
y_1, or y rather, and then?
And then zero, good.
y_0.
Because if I multiply, can I put in what M really is?
The H and K. H there, K there.
When I do that multiplication I get lambda_1*y.
When I do this multiplication, see I've just, that's a zero block, zero, so I got a zero.
Perfect.
So the eigenvectors of H just sit with a zero in the K part and produce an eigenvector of the block matrix with the same lambda_1.
So you can see then, we get the whole picture.
The eigenvalues are just sitting there and the eigenvectors are there.
Now maybe you got all that and wanted-- well I haven't said anything about N, Sorry.
Everybody thinks more about N. So what's the thing with N?
What would you say about N?
If you look at that matrix, suppose I don't even tell you it's positive definite at first, would you say that looks like a invertible or singular matrix?
Everybody's going to say singular.
And why would you say that's singular?
Well, the determinant of a block matrix, this morning I said do whatever you like with block matrices.
But I have to admit that if I had a bunch of general blocks, if I had to take the determinant of that, and of course everybody's remembering Professor Strang doesn't like determinants, if I had to take the determinant, I'd have to do the whole thing.
The separate determinants would not tell me the story, usually.
So determinants are a bit tricky.
But up here the determinant will come out zero.
I guess what I would hope your internal test for a singular matrix is, are the columns independent?
And then the matrix is invertible.
Or are they dependent?
Do you have some columns that are in the same direction as other columns, same direction as combinations of other columns?
If you look at the columns of that, say column one, so column one is the first column of K repeated.
What do you think about the columns of that matrix, that block matrix N?
Do you see that same column showing up again?
Yeah.
That very same column, which is the first column of K, again twice, is going to show up right there, first column of K again.
So this matrix has two identical columns.
No way it could be invertible.
And in fact, you can tell me what vector, I'm always saying are the columns independent?
Here, no, they're dependent.
And then you can tell me an x.
So this is my block matrix N. I want to know an x so that the result is zero.
That's really my same indication.
We found two identical columns.
What would be the x?
Well, you have to tell me more than one, minus one because I've got a big x there.
Yeah I've gotta make it big enough, but essentially it's the one, minus one, thanks.
And enough zeroes in there and enough zeroes in there.
So the fact that that vector gets taken to zero is the same thing as saying that one of this column minus one of this column gives zero.
In other words, the columns are the same.
And of course, by doing this we're seeing the one and minus one could have gone into position two there, position three.
So we've got a whole bunch of vectors.
This matrix N, this [K, K; K, K] has got a whole lot of vectors that it takes to zero.
What I would say it has a large null space.
A large space of vectors that it takes to zero.
So that's a really useful exercise.
I'm delighted you asked it.
Now I'm ready for more.
Could do.
Exactly, row reduction.
I should look to see what would happen in elimination.
Well, elimination would go swimmingly along for the first part because it's only looking here.
But then what would I have after the first half of elimination?
Well I'd have I suppose whatever that K changed to, elimination.
What should we call it?
U or something?
When I did these row steps that matrix turned into this upper triangular matrix.
And maybe you can tell me what will have happened at the same time to the rest?
What will I see sitting here if I just do ordinary elimination and I'm just looking there and using the pivots and so on, I'll see?
It'll be U because whenever I do on the left side I'm doing to the whole row.
And now, the main point is, what will I see?
Now elimination, keep going, keep going.
Do elimination to clear out this column, this whole bunch, right?
Elimination.
And now what am I going to see in that corner?
All zeroes, right.
So that's telling me that the matrix has just got half of the eigenvalues positive, half of the pivots are positive.
The second half all zeroes.
So I guess, here I've found an eigenvector with what eigenvalue?
That's looking like an eigenvector to me if we're thinking eigenvectors.
And what's the eigenvalue that goes with it?
Zero.
Because Nx is 0x.
You can either think of it as Nx=0 if you're thinking about systems of equations.
Or Nx=0x if you're thinking that that guy is an eigenvector with eigenvalues here.
So I'm pretty happy.
I mean many of you will have spotted this.
Probably perhaps all.
But I'm happy that's an example that just shows how you have to think big with block matrices I guess.
Good.
Ok on that?
What else, thanks.
That's true.
And that's really all I've done so far is those four examples.
I think that language of fixed-fixed and fixed-free really comes, I mean I used it early about those four matrices, but it's really going to show up at the next lecture, Friday, when I have a line of springs and the matrices that come out of that.
So Friday we'll finally be on those first four.
A fifth matrix will appear in this course finally.
Of course, it's going to be related to the first ones, naturally but we'll move to, we'll see something new and then we'll see the fixed-free idea again for those.
So if that can wait until Friday, you'll see some different ones.
Good.
Questions, thoughts.
You can ask about anything.
Maybe I can ask.
Any thoughts about the pace of the course?
This is sort of a heavy dose of linear algebra, right?
Of course, the answer maybe depends on how much you had seen before.
So those who haven't seen very much linear algebra at all really got quite a bit quickly here.
Because many courses on linear algebra never reach this key idea of positive definiteness that ties it all together.
So you've seen quite a bit, really.
Of course, we've concentrated on symmetric matrices and there's a whole garden or forest or zoo of matrices of different types.
So what matrices have we seen?
Symmetric matrices and then their eigenvectors were orthogonal and we could say orthonormal.
So that gave us, I don't know if you remember this part, which when we wrote it down I said, big deal.
That's very important.
That's this principal axis theorem.
These Q's, what kind of a matrix is Q?
It's the eigenvector matrix.
And for symmetric matrix, so this is the eigenvector matrix.
And what do we know about it?
In the special case of symmetric K?
What do we know especially about the eigenvectors then?
They're orthogonal.
We can make them orthonormal.
So this will be an orthogonal matrix.
And that was a matrix with Q transpose was the same as Q inverse.
Normally we would see the inverse there, but for these we can put the transpose.
Here's one type of matrix, symmetric, very important.
Here's another type of matrix, orthogonal matrices.
And of course, many, many other varieties.
Well here we have a very nice matrix, so that matrix is diagonal.
Right, that's just the eigenvalues, so that's a diagonal matrix.
And what do we know, if K is positive definite, let's just, this was for any symmetric one.
So what's special if K is positive definite?
Somehow the positive definiteness should show up here.
And where does it show?
Positive eigenvalues, exactly.
The Q could be any, any Q would be fine.
But we would see positive eigenvalues.
Oh, here's a little point about eigenvalues.
Suppose I have my matrix K. And it's got some eigenvalues.
Now let me add four times the identity to it.
What are the eigenvalues now?
What are the eigenvectors now?
What's changed and how and what hasn't changed?
Because that's a pretty easy, the identity matrix is always the easy one for us to know what's happening.
So what is happening to the eigenvalues now?
If K had these eigenvalues lambda, what are the eigenvalues of K+4I?
You add?
You add four, yeah.
The eigenvalues of this are the eigenvalues of K+4.
That is just like shifting the matrix, you could think of it is adding four along the diagonal will add four.
And the eigenvectors would be exactly the same ones.
I would have Kx would agree with lambda*S. And 4Ix would agree with 4x.
So that proves it.
Good to see what you can do, the limited number of things that you're allowed to do without changing the eigenvectors, and therefore you can spot the eigenvalues right away.
The limited things you can invert, you can shift like this, you could square it, cube it, take powers, things like that.
I'm going to look to you now for giving me a lead on something that is interesting or not.
Yes, thanks.
Go ahead.
Oh, I see okay, yes.
I see.
Alright.
So that's page 64 of the book.
Well, so that's a problem that physicists love.
I don't know how much I can say about it here, to tell the truth.
Just to mention.
Do they use a minus sign?
Probably they do.
So their equation is minus the second derivative of u plus (x squared)*u and they are interested in the eigenvalues equal lambda*u.
The case that we've done in class was without this (x squared)*u term, right?
The absolutely most important case is the second derivative of u equal lambda*u.
The eigenvalues were, or what were the eigenvectors in that case?
What were the eigenvectors of the second derivative before there was any (x squared)*u and E potential showing up?
They were just sines and cosines, right?
Sines and cosines have the property that if you take two derivatives you get them back with some factor lambda.
Now let me just look at that problem without saying much about it.
First of all, the first thing I want to know is have I got a linear problem here?
Have I got a linear equation?
Because that's where I talk about eigenvalues.
So in the matrix case, I'd say I have a matrix.
K times an eigenvector.
That matrix represents something linear.
It's just, all the rules of addition work here.
Here it is linear.
It is linear.
What I'm trying to say is, I just call that a variable coefficient and that's what we're going to see in Chapter two.
The material or something could lead to some dependence on x.
But u is still there, just linearly.
In other words, this is a perfectly ok linear operator and am I imagining that it's positive definite?
Let's see.
This part with the minus sign was positive definite, right?
Well, at least semi-definite.
So let me just remember the most important case.
If I look at this equation, d second u/dx squared equals lambda*u.
So that's the eigenvalue, eigenfunction problem for our good friend.
What do I say about the eigenvalues now?
What can you tell me about the eigenvalues of that?
Mostly positive.
Because they were sort of omega squares.
But I mean zero could be an eigenvalue, right?
What would the eigenfunction be for lambda equal zero?
If I wanted to get zero here, if I wanted a zero on the right side, what functions u could give me zero?
Constant function.
Yeah, the constant function is certainly there as a possibility.
But anyway, I would say this is positive semi-definite at least.
And this part?
How do I think about that as a big matrix?
I think of it sort of like a big matrix with x squared running down the diagonal.
With a matrix, you could say walking down the diagonal because it's n steps.
For differential equations, maybe running is the right word.
Because it doesn't jump, it's just bzzz all the way from zero squared to whatever.
Anyway, that would correspond to a diagonal matrix, but not constant diagonal.
Diagonal, but not constant diagonal.
Because this x squared number is changing.
It's like a spring, it's like a bunch of springs in which the first spring maybe has a spring constant of one.
And then we have a tighter spring and then a very tight spring and so on, more and more, higher and higher constants there.
Well, I'm just speaking very roughly here.
Because variable coefficient, variable material properties, springs of different elasticities, we're ready to move to that.
Our problems up to now, the springs were all the same.
The bar, if it was a bar, was uniform.
And now this would be a step forward.
But now, of course, this specific problem just happens to have a solution that physicists love.
It has a meaning to physicists, not to me.
And the eigenfunctions have a meaning and they're famous functions.
It's just glorious.
So you could say that's the special problem, the way we had four special matrices in 18.085, that would be a similar special problem in quantum mechanics.
Let's turn to something entirely different.
Questions about any topic.
Or I can ask some and you can take this, maybe that's one way to review.
Go ahead.
Thanks.
Number 20 of 1.6.
1.6 is a section, oh, no.
That's positive definite notes so I'm okay with that.
I see that I did ask you a question on the homework from 1.7 which I may not get to cover in lecture, but give it a shot anyway.
So what's 20?
Oh, ok, that's good.
Without multiplying out the matrix.
So it's this Q*lambda*Q transpose.
So I'm telling you in that question what Q, lambda, and Q transpose are.
The Q is this [cosine, minus sine; sine, cosine].
The lambda is two and five, I think, in that question.
And the Q transpose of course is [cosine, sine; minus sine, cosine].
And if I've told you that those are the numbers then you could multiply those together to get K. But you can tell me, this is like K exposed.
The matrix is like, we're told more than we would know.
If I multiply it all together, I wouldn't see that the eigenvectors are these guys, that the eigenvalues are these guys.
So what, without looking to see, what are the eigenvalues of this matrix K if we multiplied it all together?
What would the eigenvalues actually be?
Two and five, right, because we built it up that way.
What would the determinant be?
Now what do we know about determinants?
It would be ten is the right answer.
What's the right way to see that?
Well, the determinant is always the product of the eigenvalues, isn't it?
These guys have determinant ten anyway.
And if I hadn't normalized, so this had some bigger determinant, this would have some smaller determinant.
Their inverses, their determinants will give me the 1 and there's the ten.
What else could I ask about or did I ask about for that?
The eigenvectors, ok.
The eigenvectors of the matrix, what are they?
They're these columns that are sitting here for us, they're those two columns, right.
And would you like to just check that if the, I believe that column is an eigenvector.
And which, do you think two or five is it's eigenvalue?
That goes with this first column.
Everybody's going to say two and that's right.
And do you want me to just take that matrix times this proposed eigenvector and just see if it's going to work?
Suppose I just do all and just see, sure enough this will be an eigenvector.
So what do I have at this point?
Can you do this times this first?
What do I get?
c squared plus s squared is one.
And -cs plus cs is zero.
So at that point I have .
Now comes this matrix.
So what do I have after that matrix speaks up?
.
And now I take two times this and what do I get?
Or that matrix times the .
How do you multiply a matrix times that  vector.
Here's the good way to think of it.
It's two times the first column.
And zero times the second.
So the net result of the whole deal was two times that first column.
Which is exactly saying that this is an eigenvector.
When I did all that it came back again.
Scaled by two.
So that's a good example.
And then, is the matrix positive definite?
That connects to today's lecture.
What test would you use to show that the matrix is positive definite?
The eigenvalues, yeah.
The eigenvalues are sitting there.
Two and five, both positive.
If I changed one of those signs, then it would no longer be positive definite.
It would still be symmetric, I'd still have the eigenvectors, but then eigenvalue would have jumped to minus five.
I think this sort of helps out.
I guess I hope that as I'm doing these things, you're ahead of me or with me in the calculation and you just have to do a bunch of these to get confidence that you've got the right thing.
Ok, yes?
1.6, 24.
Is that also a homework problem?
Alright, but you guys are reading the rest of the book, right?
Not only the homework questions.
Ah.
Oh, dear.
24, that's a very good question.
About this, yeah.
Right.
It's a good question.
And if today's lecture had been, well it ran a little late.
But if we ran another 20 minutes late, I could have done this.
I'll just say what's in that problem.
And then we'll see it again.
So what's in that question?
Let me write down what it is.
So I have a positive definite matrix K, right?
And then I've got its energy.
I'm using u rather than x, so let's use u.
So my u transpose Ku, or like x transpose Kx today.
That is this bowl-shaped figure, right?
If I graph this on the u_1, u_2 maybe up to u_n, all in the base.
And now I have the picture.
So I'm in n+1 dimensions.
The other dimension is this one.
Then that's the one where I might get this bowl-shaped guy.
And I've called that energy.
In many, many physical problems there is a factor of 1/2.
And it's going to be nice to have that factor of 1/2.
So that won't change anything, just half as big.
So what is the minimum value of that energy?
And what is the minimum value of this, if I said minimize that, you could do it right away.
It'd be a zero.
Now I'm going to introduce a linear term.
This was a quadratic term and it had u squareds in it.
So the linear term is going to be u transpose f is the shorthand for it.
And of course, we all know that that stands for u_1*f_1, u_2 all minus, u_2*f_2 and so on.
However many dimensions I'm in.
You can imagine I'm in two dimensions.
So it's -u_1*f_1 - u_2*f_2.
So what I'm saying is that minimizing just this was like, too easy, right?
The answer was zero.
Nobody's interested in that for very long.
But now it is much more interesting when I get a linear term in there.
So what happens now?
Well, the effect of that linear term is to shift that bowl sorta over and down a little.
So that instead of sitting where I drew it, let me erase it.
If I know graph this function, this is my function of u, this is still the most important part, but now I have a first order turn.
And the result is, it still goes through here.
Right?
Why does it still go through that same point?
Because if I take u_1 and u_2 to be zero, I get zero.
So I still get zero there.
But the bowl has shifted.
It's more like something here.
And it still has a minimum because this is still the all-important term.
But it's just moved over and down.
So it has the minimum value.
It actually goes below zero, but if I look at it if I'm sitting at the minimum and looking I'm seeing a bowl going up, right.
So I hope that picture shows.
And now, of course, that's the geometry.
In other words, the same geometry just moved the thing over and down.
But the algebra is, where is the minimum?
What is the value of that minimum?
And this problem, 24, is one way to do the minimum.
One way to do it.
But actually, if you doesn't like linear, well I won't say didn't like linear algebra, that's against my religion.
So if you like calculus and you said, wait a minute, if you give me something you want me to minimize, what will I do?
I'll set derivatives to zero.
And can I just jump to the answer?
Oh, what derivatives do I set to zero now, for the minimum here?
It's the first derivatives.
And they're first derivatives with respect to?
I look at df/d what?
You see I've already given it away.
These are going to be partial derivatives.
Why's that?
Because I've got two directions.
So I have a df/du_1=0 and a df/du_2=0.
In other words, when I sit here at the bottom I'm seeing this whole bowl above me.
If I go along the u_2 direction it should go up and if I come along the u_1 direction, goes up.
But it's flat at the bottom both ways.
So what's my point here?
If you like calculus, you'll get to two equations.
And I just want to say what those equations are, because they're all important.
Suppose we only had u_1 and nothing else.
Then this would just be a parabola and the derivative of this would be at 1/2 K*u squared.
Suppose n is one.
I'm only in one.
So what's the derivative of 1/2 K*u squared?
The derivative.
So I'm looking for if this was 1/2 K*u squared and I took the derivative with respect to u, it would be?
It'd be Ku.
And it works here in the matrix case.
And what would be the derivative of u, transpose of u times f, if u was just a number and if u was just one thing and f was a single number, the derivative would be?
f, yeah.
It'd be f. That's the system.
I've jumped to the answer.
That this set of two or n equations in matrix language would just be, and I'll even write it better as Ku=f.
That tells me where the minimum is.
The minimizing guy is, so this is in the base and then the thing is dropping down.
I still have to figure out what's the bottom value.
But I've now identified where the minimum occurs.
So you get two questions about a minimum.
Where is it?
What value of u gives the minimum?
And at that point, at that lowest point, how low is it?
The one thing you've gotta remember is that when you minimize that quadratic, you get that system of equations.
And then, of course, the answer, you have to solve that system.
But this goes back to what I said at the first minute of today.
That we have two ways of looking at a problem.
Usually we go directly to the equations.
Sometimes the problem comes naturally to us as a minimum problem.
Like we have to minimize the cost, we want to build a new school or something.
So we've got some cost function that we minimize that will lead, through calculus or linear algebra, to this.
So I've done everything but answer the question 24.
We only checked the one by one case to see that that's the right equations, derivative equal zero.
And now you could use calculus as I said.
But if I answer that question, well let me just do a little.
The idea of that question 24, so that was what?
1.6, 24, or something.
Is that right?
Yeah.
Is that I could rewrite this to make it clear.
I think it's u minus K inverse f. Transpose K times u minus K inverse f. And then a minus f 1/2, f transpose K inverse 1/2.
Actually, my best friend in China told me this trick.
And I didn't give him credit for it in the book.
But I should have done.
I just think that if you multiply all this out, you'll get this.
It's what I would call an identity.
That just simply means that it's just true for every u.
It's true for everything.
Can I try to multiply some of that out?
Just so you kind of see it.
Yeah, that's what I mean, multiply it out.
You've got it.
This thing would give me four terms.
It'd be this transpose times that times that.
Which is my guy here.
And then I'll have something.
It's just like numbers.
Then this thing times that times this.
And this thing times that times that.
And this thing times that times that.
Let me do that last one.
What happens when I do the 1/2 and this transpose times the K times this.
So I'm using the distributive, whatever, laws.
Let's just do that particular term and see what we're getting.
So I have 1/2 of the minus K inverse f transpose.
So how do I write that?
Shoot.
Well, it's something times something transpose.
So what do I have to do?
Opposite order.
So I have a minus, an F transpose and the K inverse transpose.
You're seeing all this stuff.
And then comes the K and then comes the minus, oh again the minus.
So that'd be a plus, right?
Times K inverse times f. So that's one of the terms.
That's one of the terms that shows up.
And what good is that one?
So that's one.
You could say that's the longest term.
That's the one with the messiest term.
But you can fix it.
What would you do with that?
K times K inverse is?
Identity.
So we can forget that.
And now we're there.
That's 1/2, f transpose, f on this side.
Oh, what's K inverse transpose?
It's the same as K inverse because K is symmetric, so its inverse is symmetric.
So that transpose doesn't change the matrix.
In other words, this term will show up and this term is oh!
Nope, sorry.
I was going to goof here.
I was going to say this is the same as this, but it's not, right?
Why not?
Because it's positive.
And this guy is negative.
Has my good friend Professor Lin messed up?
Nope.
What's going to happen now?
The two that I didn't do, you see, the 1/2 u transpose K u is here.
Then comes this one, which I didn't do, and then another one that I didn't do, and then this one that I did.
They'll all be the same.
So they'll all contribute with their plus sign or minus sign and the net result will be a perfect match, yeah.
So I won't wear out your patience by doing that.
But I do want to make the point.
What was Professor Lin's point in suggesting to write it in this more complicated way?
His point was we could see this is just a constant.
Doesn't depend on u.
And now I can see what value of u would make this as small as possible.
Remember, I'm still trying to minimize.
This part, I can't make it bigger or smaller, it's fixed.
It's u that I can play with.
So what u should I choose to make this part smaller?
Bear with me.
What u will make this big mess as small as I can get it and how small can I get it?
If I take u to B K inverse f, then this is zero, this is zero, I get zero.
And that's my claim, that u equal K inverse f is the best possible, is the minimizer.
And how do I know that I can't make this more negative than the zero?
I can get it down to zero by making that to be the zero vector.
But how do I know I can't make it below zero?
The K is positive definite and I'm sitting here with some x transpose and some x.
The X hax this sort of messy form but it's an x and here's its transpose.
So this is an x transpose, Kx and can't be brought below zero when K is positive definite.
Good.
So we've said a good bit about positive definite here, but happy to think-- Yeah, thanks.
In fact, finally a fifth.
Exactly.
Thanks, perfect question.
And let me answer it clearly.
Each of those five tests completely decides positive definite.
So the five tests are all equivalent.
If a matrix passes one test, it passes all five.
So that's great, right?
So we just do whichever test we want.
Or whichever way we want to understand the matrix.
I was going to add, I didn't say a lot about this one.
Can I just add a note about a MATLAB command?
The command chol(K).
That's the first letters in the name Cholesky.
So chol is the first four letters of this name.
And that's a MATLAB command.
If I've defined a matrix that's positive definite and I use that command, out will pop an A, one particular A that works.
Out will pop on A that makes this work.
It'll be a square A and it'll be upper triangular.
So out will pop, so this command is very, very close to the LU but it's just sort of the appropriate version, symmetrized version of elimination when you have a positive definite symmetric matrix.
If your matrix is not positive definite, the MATLAB will tell you so.
So it produces one particular A.
There are many A's that would work, but there's one particular upper triangular one.
It's just related to the usual u, but yes, thanks.
No, I only even get into that ballpark if the matrix is symmetric.
I don't touch it otherwise.
So my matrix is symmetric before I begin.
So I know good things about it.
And here I'm asking for more.
Here I'm asking are the pivots all positive?
Are the eigenvalues all positive?
So that's more.
But I could think of some interpretation that would, for non-symmetric matrices, but it has problems, so I'd rather just leave it.
Stay with symmetric.
Well that's two hours of lots of linear algebra.
I'm hoping you're going to like the MATLAB problem.
Would you like to see what it'll be?
I'll just tell you what the equation will be.
So it'll be a differential equation.
Oh, dear, what is it?
So it's a differential equation with a -u'' that we know and love.
And what else has it got?
Oh yes, right.
So here's the problem.
Here's the equation.
So it has the -u'', the second derivative and it has a first derivative equal whatever.
In fact, the example will choose a delta function there.
So what am I talking about here?
This would be a diffusion and this would be, anybody met these things before?
That would be a convection.
So that's a first derivative, that's an anti-symmetric.
The MATLAB problem is now going to create the difference matrix for that.
So the symmetric part will be our old friend K. But now we've got the convection term is appearing.
And it's going to be anti-symmetric.
And if v is big, it gets more and more important.
So what happens?
What happens with equations like this?
Really this is like the first time in the course that we've allowed this first derivative term to pop up.
But nevertheless we can see a lot of what's happening.
And how to deal with those equations?
I mean, if you ask a chemical engineer or anybody, they're always dealing with a flow, like the Charles River is flowing along, that's coming from the velocity there, but at the same time stuff is diffusing in it.
It's just a constant problem in true, true applications.
And this is the best model, I think.
So you'll see that and I'm pleased about that.
As you'd see.
Any last question?
I'm always happy.
Well I'll see you Friday then.
Thanks for coming.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational research for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'm open for questions as always on every aspect, or comments on every aspect of the course.
yeah?.
The MATLAB problem, yes.
I better remember what that MATLAB problem is.
So it's solving the equation -u''+Vu' equal some delta function halfway along?
And I chose, boundary conditions like those so that this gets replaced by a matrix K/(delta x squared).
And this term is V times some center difference.
I don't know whether I should use C. We called C earlier in the book, C was a circulant.
This is a centered first difference over delta x equal right-hand side.
All that multiplies the discrete solution u. I didn't get that very well for the camera.
Maybe I'll say it again.
We have the second difference that corresponds to -u''.
And V times the first difference that corresponds to Vu'.
So the physical problem is, what happens is V gets larger.
V grows.
How does the solution change as V increases?
So it's a very practical problem.
The V measures the importance of this convection, the velocity of the fluid.
Then n, which is, well maybe delta x is 1/(n+1), maybe.
So that we would have n interior points and we know the boundary values.
So the question is how do we interpret this increasing n?
I guess we expect more and more accuracy, right?
As n increases, delta x getting smaller.
Our idea is that this should be a better and better approximation to the differential equation.
The question about the eigenvalues of this matrix, I don't know exactly the answer.
So the eigenvalues, did you discover that the eigenvalues suddenly, they change their real, for small values of V and n. But then there's some point at which they become complex.
And a point that involves both V and delta x actually, V and n. It's kind of interesting.
I guess at this moment, interesting is the key word.
So what is the physics going on here?
So I have a 1-D flow and it's blocked by these boundary conditions.
Are sort of blocking it at both ends.
And I'm looking for a steady state.
And I'm feeding in this source here.
So I'm feeding in a source halfway along.
So I think that would be right.
Now what's going to happen roughly?
This term is, I think, flowing to the right.
So it's going to carry, suppose this was smoke or particles or whatever.
This is like an environmental differential equation or a chemical engineering equation.
Just oh, and more and more applications.
So I think of it as this term is carrying the flow with it.
Like, that way.
So I'm expecting your solutions to be larger on this side.
And how do we get any action at all on the left?
That comes from the diffusion.
The particles, when they enter, get carried away, and faster and faster as V increases.
But at the same time they diffuse, they bounce back and forth.
And some bit of it goes this way.
But not a lot.
I'm guessing that the solution would have a profile that would be kind of small.
And then there's whatever jump there has to be here, and then larger there.
Oh, but then I have to get it to zero.
It's a strange situation.
And maybe I'll take this chance to mention it.
What happens as V gets really, really large?
You could say ok, as V gets extremely large, forget that term.
This is the important term.
Vu' equals that.
Which we could easily solve.
But this equation with just this term is only first order, right?
It only has a first derivative.
And how many boundary conditions would we expect to have if I gave you, if this wasn't here and I gave you just a first order equation?
One.
And we've got two.
Which we're imposing.
So this, sort of, limit as V increases, it produces something called a boundary layer.
The solution is forced to satisfy these boundary conditions.
One of them, it's quite happy about.
The other one, it didn't really want to satisfy.
It only satisfied them because, I had two originally with this part.
But as this takes over, the struggle to satisfy that second boundary condition that it really doesn't want to satisfy is resolved by something, a layer that just makes a, sort of, exponential correction at the boundary to get to where it's supposed to go.
So, anyway.
This is a model problem that has boundary layers.
Those are terrifically important in aerodynamics, you have layers around the actual aircraft.
So lots of physics and computational science is hiding in this type of example.
So those are a few words.
But not an answer to your question.
I guess I'm happy if you, for example, let me know where the eigenvalues change from real to complex.
I mean, it happens, but you can probably, if you look at the matrix, you can see what happens at the point where that change takes place.
That's some comments.
And with the MATLAB, pure MATLAB part, I guess I'm hoping, and the graders are hoping, that by providing a code we'll get a pretty systematic set of answers for the requested V equal, what was it, three and 12 maybe, or something.
And the requested n(delta x)'s.
So I'm hoping that everybody's graph is going to look pretty similar for those requests.
And then if you could do a little exploration yourself and make that a fifth page, or really less than a page, about take V larger.
Take n larger.
What happens?
I'm very happy.
Or any direction.
I mean, just think of it as a mini-project to do the requested ones and then to do a little experimentation.
I'll just be interested about anything you observe.
So there's no right answer to that part.
Is that any help with the MATLAB?
Let me just say, since you showed up today, if you have trouble with the MATLAB and you need Peter's help at noon Friday it would be ok to turn in the MATLAB later on Friday.
I shouldn't say that.
But I just did.
So there you are.
Right.
I mean, this course, as you've begun to see, I know that you have lots of demands on your time.
And sometimes you have job interviews.
All sorts of stuff.
Conferences that you have to go to.
We deal with that.
So just give me the homeworks as soon as you can.
If you're ready Friday at class time, that's perfect.
If are stuck on MATLAB and you want to go to Peter's discussion, which follows the Friday class in here, that's fine too.
So that's some thoughts about the MATLAB and it's partly open-ended.
What about lots of other things in this course?
Homework came in.
How was the homework?
Maybe I just take this chance to ask.
Was it a reasonable length?
The homework three, the pset?
Not too bad?
I will be able to post solutions because several people are contributing typed solutions that could go on on the math.mit.edu/cse website.
Or they could also go on our 18.085 website for this semester.
Good.
Ok, help me out with some questions.
Thanks.
So the question is about element matrices.
In lecture 8, I guess it was, where we were assembling this, I used the word that's used by finite element people, we were assembling K. And one way to do it was to multiply the three matrices.
But that's not how it's actually done.
It's actually assembled out of small matrices.
Maybe small k would be the right.
So this would be a k_element out of smaller element matrices.
So for example, what's the element matrix for, one for each spring in this particular problem of springs and masses.
So let me draw some springs, some masses, some springs, another mass, more springs, fixed, not fixed, whatever, maybe fixed-free the way I've drawn it there.
So here's a spring with spring constant c_2.
And we could look at the contribution to the whole matrix coming from this piece of the problem.
This was actually, the finite element method has a wonderful history of people, and it had different names way back, of people seeing the structure as broken in pieces and then connected together.
And then what did a typical piece look like?
So a typical piece there, well, you let me just write down what this matrix is going to be.
The little element matrix is coming from spring two.
So this would be, like element two will be, it'll have a c_2 outside.
I'll put the c_2 outside.
And then inside will be a little this guy.
And we can talk more about why it's that one.
But just to have the element matrix there on the board.
So my claim is that this is a small piece of the big K. So the big K matrix, what's the size of the big K, of K itself?
Three by three in this case, yeah, three masses.
So it'll be three by three.
So I'm thinking that this spring which connects mass one to mass two, so it's only going to be like, a two by two piece, a little local piece, you could say, that that little k fits in this, is assembled into the, I better call it k_2, right?
So it's a little k that sits up in this two by two block and doesn't contribute to the rest.
Then let's just draw the rest of the picture.
So this would produce an element matrix that looks just the same, that's the beauty of this.
That all the elements, apart from change in the spring constant, look the same.
So there'd be a little k_3.
And where will it go?
This is the whole core to the point.
It'll go to the lower right.
Down here?
Overlapping, overlapping.
Because this mass is attached to that spring and to that one.
So these little element matrices, they overlap.
And you just need, if you can imagine the code that's going to do this, you need a list of springs and a list of masses and a list of the connections between them.
And it'll sit in here because it's two by two.
So that's two by two, that's two by two and in this overlap entry will be a c_2 from the upper box.
It'll be a c_2+c_3 as we discovered by direct multiplication.
So that's that spring.
Now what about this first spring?
So there's a little k_1.
Now k_1 should look the same as this.
Except what?
So it's going to be a difference with this spring because?
Because of this fixed.
End.
There's no mass zero.
k_1 would normally sit up here, but actually it's only going to be one by one.
So the k_1 little element matrix would look like c_1[1, -1; -1, 1].
And then the boundary conditions, knock those out.
And an interesting point if you're writing big code you have to decide.
Do I, as I create k_1, this little element matrix, do I pay attention to the boundary conditions?
And then k_1 would just be that single number c_1 which would sit there.
So up there will be c_1 from the k_1 matrix and c_2 from the k_2 matrix.
That's the entry there.
And then, as I say, in coding finite elements, which you guys may do at some point, there's a question.
What's sufficient?
Shall I pay attention to the boundary in creating these element matrices or shall I do it later?
And the rule seems to be, do it later.
So what gets created in finite elements is a, in our language, would be a free-free matrix.
It's a matrix where even this spring has a two by two piece.
So what's the problem with the free-free matrix?
The free-free matrices, those with no supports, those matrices will be singular.
Right, singular.
Because the vector of all ones, if you multiply the K, the free-free matrix times the vector of all ones, you get zero.
The whole thing could shift.
And will have other rigid motions in two dimensions.
Let's just think ahead here.
What are the rigid motions, the null space of K, you could say, for a two-dimensional truss.
So I've got a bunch of bars and springs.
Think of a mattress.
I'm in two dimensions, a mattress is a bunch of springs connected in a 2-D grid.
And say it's free at both ends.
So what can I do to a mattress?
Oh, my gosh!
I didn't expect that to be on videotape.
So it's in the plane.
We're in 2-D here.
What can I do if there are no boundary conditions?
Well I can shift it to the right.
Right?
That's our .
What else can I do?
I can rotate.
And I can shift it the other way.
So there would be three rigid motions for the 2-D problem.
Two translations and a rotation.
And when you get up to three dimensions, do you want to guess what's the number in 3-D?
Six.
Number six.
Three translations and rotations around three axes.
So those, either one rigid motion or three rigid motions or six rigid motions have to get, the boundary conditions eventually have to remove those, have to knock out rows and columns and remove them.
But my point was just that quite efficient to do it later.
The picture is so clear here.
So the actual matrix would be four by four, the unreduced matrix.
And then when we fix node zero there, that would knock out that part and leave the three by three that we want.
So this is just discussion of how to think of this matrix K. So the direct way to think of it was as a product of big matrices.
But in reality it's assembled from these element pieces.
And of course, our goal in talking about finite elements will be to see that.
It'll come up again, of course.
Your good question led me there, but did I answer the question?
And maybe the way I mentioned it in class was to notice that these guys, these element matrices can be thought of this way.
It is matrix multiplication, but done differently.
It's a column of this matrix times the number C here times a row of this.
So it's matrix multiplication, columns times rows.
So you can multiply AB, columns of A times rows of B and then add over from one to n. So column of A times row of B.
So a column, then, is a vector like this.
A row is a vector like that.
And the result is a full size matrix, but if the column is concentrated at a couple of nodes and the row is, then it will have zeroes everywhere except at that.
This is the element matrix with the C included.
That would be the element matrix already blown up to full size by adding zeroes elsewhere.
So, I mean the heart of the element matrix is where the action is on that spring.
And then, when we assemble it, of course, it doesn't contribute down there.
So the technology of finite elements is quite interesting and it fits.
It's a beautiful way to see the discrete problem.
Ready for another question of any sort.
Thank you, good.
Yeah, sorry, it got onto the problem set.
And then I thought-- let me write those words down first, singular value decomposition.
Well I won't write all those words.
That's a wonderful thing.
It's a highlight of matrix theory except for the length of its name.
So SVD is what everybody calls it.
It's only like, every year now people appreciate more and more the importance of this SVD, this singular value decomposition.
So shall I say a few words about it now?
Just a few.
So it's Section 1.7 of the book.
And my thought was, hey we've done so much matrix theory including eigenvalues and positive definiteness, let's get on and use it.
And then I can come back to the SVD in a sort of lighter moment.
Because I'm not thinking you will be responsible for the SVD.
Eigenvalues I hope you're understanding.
Positive definite I hope you're understanding.
That's heart of the course stuff.
But SVD is a key idea in linear algebra, but we can't do everything.
But we can say what it is.
What's the first point?
The first point is that every matrix, even a rectangular matrix, has got a singular value decomposition.
So the matrix A can be m by n. I wouldn't speak about the eigenvalues of a rectangular matrix.
Because if I multiply, you remember, the eigenvalue equation wouldn't make sense.
Ax=lambda*x is no good if A is rectangular.
Right?
The input would be of length n and the output would be of length m and this wouldn't work.
So eigenvectors are not what I'm after.
But somehow the goal of eigenvectors was to diagonalize.
The goal of eigenvectors was to find these special directions in which matrix A acted like a number.
And then, as we saw today in the part that partly still up here, we could solve equations by eigenvectors by looking for these, following these special guys.
What do we do for a rectangular matrix?
What replaces this?
So this is now not good.
The idea is simply we need two sets of vectors.
We need some v's and some u's.
So that's the central equation of the SVD.
Now what can we get?
So now we have more freedom because we're getting a bunch of v's that have, these guys are in our n. They have length n, right to multiply A times v. And the output is, so the n of these, n v's, we're in n dimensional space.
So those are called singular vectors.
They're called right singular vectors because they're sitting to the right of A.
And these guys, these outputs will be in-- these are v's in our n, n dimensions.
I have m of these, m u's in m dimensional space.
And these things are numbers, of course.
And actually, they're all greater or equal to zero.
So we can get, by allowing ourselves the freedom of two right singular vectors and left singular vectors, we can get a lot more, and we can get, here's the punch line.
We can get the v's to be orthogonal, orthonormal.
Just like the eigenvectors of a symmetric matrix, we can get these v's, these singular vectors to be perpendicular to each other and we can get the u's to be perpendicular to each other.
What we can't, what we're not shooting for is the v's to be the same as the u's.
They're not even in the same space now, if the matrix A is rectangular.
And even if the matrix A is square but not symmetric, we wouldn't get this perpendicularity.
But we get it in the SVD by having different v's and u's.
Here's my picture of linear algebra.
This is the big picture of linear algebra.
This over here is n dimensional space.
We have v's.
Think of that as n dimensional space.
That's kind of a puny picture.
But when I multiply by A, I take a vector here, I multiply by A.
So let me just do that.
I multiply by A, I take a vector v, well, already put v, I take Av and I get something over here.
And this will be the space of u's.
Now I have to ask you one thing about linear algebra that I keep hammering away.
If I take any vector and multiple by A, What do I get?
If I take any vector v, like .
Here's A say, [3, 6; 4, 7; 5, 8].
So that's A times v. What can you tell me about A times v that goes a little deeper than just telling me the numbers?
It's a linear combination of the columns.
These are the outputs, the Av's.
So this space is called the column space.
It's all combinations of the columns.
These are all combinations of the columns.
That's the column space.
It's a bunch of vectors.
So the point is that these u's, that I have like, a fantastic choice of axes.
A linear algebra person would use the word, bases.
But geometrically I'm saying they're fantastic axes in these spaces.
So that if I choose the right axes in the two spaces, then one axis will go to that one when I multiply by A.
The next one will go to that one.
The third will go to that one.
It just could not be better.
And that's the statement.
Now maybe I'll just say a word about where it's used or why it's useful.
Oh, in so many applications.
Well most of you are not biologists.
And I'm not certainly.
But we know that there's a lot of interesting math these days in a lot of interesting experiments with genes.
So what happens?
So we've got about 40 people here.
And we measure the, well, we get data.
That's what I'm really going to say.
Somehow we got a giant amount of data.
right?.
Probably 40 columns.
Everybody here is entitled to be a column of the matrix.
And the entries will be like, have you got TB, what height, all this stuff.
So you got enormous amount of data.
And the question is, what's important in that data.
You've got a million entries in a giant matrix and you want to extract the important thing.
Well, it's the SVD that does it.
You take that giant matrix of data, you find the v's and the sigmas and to u's.
And then the largest sigmas are the most important information if things are scaled and statistics is coming into this also.
I could a give sensible, much more mathematical discussion of one word, one name it goes under is principle component analysis.
That's a standard tool for statisticians looking at giant amounts of data.
Is to find the principal components and those will come from these eigenvectors.
Well your question about the SVD set off that discussion.
I'll only add a couple of words.
These v's and these u's are actually eigenvectors.
But they're not eigenvectors of A. v's happen to be the eigenvectors of A transpose A.
And the u's happen to be eigenvectors of A, A transpose.
And the linear algebra comes together to give you this key equation.
And of course, you and I know that if I'm looking at the eigenvectors v of A transpose*A, A transpose A is a symmetric matrix.
In fact, positive definite.
So that the eigenvectors will be orthogonal, the eigenvalues will be positive.
And then this one is coming from the eigenvectors of A A transpose, which is different.
So if the matrix A happened to be square, happened to be symmetric, happened to be positive definite, this would just be Ax=lambda*x. I didn't have to cross it out.
I'll use K for that.
So if the matrix A was one of our favorite matrices, was a K, that would be the case in which the SVD is no different from eigenvalues.
The v's are the same as the u's, the sigmas are the same as the lambdas, all fine.
This is sort of the way to get the beauty of positive definite symmetric matrices when the matrix itself that you start with isn't even square.
Like this one.
More than I wanted to say, more than you wanted to hear about the SVD.
What else?
Yes, thank you.
More about, sure.
Let me repeat the question.
So this refers to this morning's lecture, lecture 9 about time-dependent problems.
And the point was that when I have a differential equation in time there're lots of ways to replace it by difference equations.
So let's take the equation du/dt, let's make it first order is some function of u often and t. That would be a first order.
Yeah, it's going to look at when I write that much down.
What have I got?
I've got a first order differential equation.
Is it linear?
No.
I'm allowing some, this function of u could be sin(u).
It could be u to the tenth power.
It could be e to the u.
So this is how equations really come.
The linear ones are the model problems that we can solve.
This is how linear equations really come.
Euler thought of, let's give Euler credit here, so forward Euler and backward Euler.
And the point will be that this guy is explicit.
So can I write that word so I don't forget to write it?
Explicit.
And that this guy will be implicit.
And this is a big distinction.
And we'll see it.
So this says u_(n+1)-u_n over delta t is the value of the slope at the start of the step.
So that's the most important, most basic, first thing you would think of.
It replaces this by this.
You start with u_0, and from this you get u_1.
And then you know u_1 and from this you get u_2.
And the point is at every step this equation is telling you what u_1 is from u_0.
You just move u_0 over to that side.
You've only got stuff you know and then you know u_1.
Contrast that with backward Euler.
So that's u_(n+1)-u_n over delta t is f at-- Now what am I going to put there?
I'm going to put the end, the point we don't know yet.
So is the slope at u_(n + 1) and the time that goes with it, t_(n+1).
t_n is just a shorthand for n*delta t. n steps forward in time got us to t_n.
This is one more step.
Now why is this called implicit?
How do I find u_(n+1) out of this?
I've got to solve for it.
It's much more expensive.
Because it'll be probably a non-linear system of equations to solve.
We'll take a little time on that.
But of course, there's one outstanding method to solve a system of non-linear equations and that's called Newton's method.
So Newton is coming in.
Newton's method is sort of the first, the good way, if you can do it, to solve.
Well when I say solve, I mean set up an iteration which, after maybe three loops or five loops will be accurate to enough digits that you can say, ok that's u_(n+1).
On to the next step.
Then the next step will have the same equation but with n up one.
So it'd be u_( n+2)-u_(n+1).
So you see the extra work here, but it's more stable.
The way this one spiraled out, this one spiraled in in the model problem.
I hope you look at that Section 2.2 which was about the model problem that we discussed today.
There's more to say.
I mean, this is the central problem of time-dependent, evolving initial value problems.
You're sort of marching forward.
But here, each step of the march takes an inner loop which has to work hard to solve, to find where you march to.
Is that a help, to indicate?
Because this is a very fundamental difference.
And Chapter 6 of the book develops higher order methods.
These are both first order, first order accurate.
The error that you're making is of the order of delta t. That's not great, right?
Because you would have to take many, many small steps to have a reasonable error.
So these higher order methods allow you to get to a great answer with bigger steps.
A lot of thinking has gone into that and somehow it's a pretty basic problem.
Just to mention for MATLAB, ode45 is maybe the workhorse code to solve this type of a problem.
And the method is not Euler.
That would not be good enough.
It's called Runge-Kutta.
Two guys, Runge and Kutta, figured out a formula that got up to fourth order accurate.
So that's the thing.
And then if we looked further about this subject we would distinguish some equations that are called stiff.
So I'll just write that word down.
Some equations, the iteration is particularly difficult.
You have two time scales.
Maybe you've got things happening on a slow scale and a high scale.
Multi-scale computations, that's where the subject is now.
And so there would be separate codes with an S in their names suitable for these tougher problems, stiff equation.
I guess one thing you sometimes get in the review session is a look outside the scope of what I can cover and ask homework problems about.
I'm sure hoping that you're assembling the elements of computational science here.
First, what are the questions?
What are some answers?
What are the issues?
What do you have to balance to make a good decision?
Time for another question if we like.
Yeah, thank you.
Stability, yeah, right.
This here?
First, if we want a matrix then this has to be a vector of unknowns.
I'm thinking now of a system of, this is n equations.
I've got u is a vector with n components.
I've got n equations here.
The notation, I can put a little arrow over it just to remind myself.
And if I want to get to a matrix I better do the linear case.
So I'll do the linear case.
The big picture is that the new values come from the old values.
There has to be a matrix multiplier.
Anytime we have a linear process.
So the new values come from old values by some linear map.
A matrix is doing it.
So there's a sort of growth matrix.
Can I just put down some letters here?
I won't be able to give you a complete answer.
But this'll do it.
So u_(n+1) is some matrix times u_n.
That's what I wrote down this morning for a special case.
It's gotta look like that for a linear problem.
And let's suppose that we have this nice situation as today where it's the same G at every step.
So the problem is not changing in time, it's linear, it's all good.
What is the solution after n times steps compared to the start?
So give me a simple formula for the solution to this step, step, step equation.
If I started at initial value u_0 and I looked to see, well what matrix gets me over n steps, what do I write there?
G to the n, right, G to the n. So now comes the question.
The stability question is whether do the powers of G to the nth grow?
And here's the point.
That the continuous problem that this came from, it's got its own growth or decay or oscillation.
This guy, the discrete one, has got its.
They're going to be close for a step or two.
For one or two steps I'm expecting that this is a reasonable consistent approximation to my problem.
But after I take a thousand steps, this one could still be close or it could have exploded.
And that's the stability thing.
The choice of the difference method can be be stable or not.
And it's going to be the eigenvalues of G that are the best guide.
So in the end people compute the eigenvalues of the growth matrix and look to see are they bigger than one or not.
Let's close with that example because that's a good example.
So it fits this, but not quite, because it wasn't completely forward or completely backward.
And let's just write down what it was in that model problem and find the matrix.
So can you remind me what I wrote today?
So u, was it u first?
U_(n+1) was U_n+delta t*V_n.
And then the new velocity was the old velocity and it should be minus delta t times the u.
Because our equation is, these are representing the equation.
u'=v is the first one.
v'=-u is my second equation.
So and then the point was I could take that because I know it already, I could take it there.
Where is the matrix here?
I want to find the growth matrix that tells me now-- My growth matrix, of course, this is  and this is .
And this is a two by two matrix.
And I hope we can read it off.
Or find it anyway.
Because that's the key.
How could we get a matrix out of these two equations?
Let me just ask you to look at them and think what are we going to do.
That's a good question.
What shall I do?
I would like to know the new U's, U, V from the old.
What'll I do?
Bring stuff onto, a U_(n+1) on this side and n on this side.
This guy I want to get over here.
Can I do that with erasing?
So it's going to come over with a plus sign.
So I guess I see here an implicit matrix acting on the left and an explicit matrix acting on the right.
So I see a .
And I see my explicit matrix.
Shall I call it E?
That's the right sides.
I see a one and a one and a delta t. And now my left side of my equation, the n+1 stuff.
What's my implicit matrix on the left sides of the equation?
Well, a one and a one and a plus delta t is here, right?
And let's call that I for implicit.
Are we close?
I think that looks pretty good.
What's G?
What's the matrix G now?
You just have to begin to develop a little faith that yeah, I can deal with matrices, I can move them around, they're under my control.
I wanted this picture.
I want to move everything to the right.
What do I do to move everything to the right-hand side?
I want that to be over there.
It's an inverse.
It's the inverse.
So G, the matrix G there is, I bring I over here.
It's the inverse of I times the E. And I can figure out what that matrix is.
And I can find its eigenvalues.
And it'll be interesting.
So actually that's the perfect problem.
I mean, if you want to see what's going on, well the book will be a good help I think, but that's the growth matrix for leapfrog.
And I'll tell you the result.
The eigenvalues are right of magnitude one, as you hope, up to a certain value of delta t. And then when delta t passes that stability limit the eigenvalues take off.
So that this method is great provided you're not too ambitious and take too large a delta t. Thanks for that last question, thanks for coming.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, if visit MIT OpenCourseWare at ocw.mit.edu
So this is review session number five, I guess it is.
And it comes before an exam next Tuesday, and actually there'll be a further review session number six on Monday, right before the exam.
So maybe today we would, there's a homework problem set on Chapter 2, mostly the oscillating masses and springs and today's lecture that you see traces of, networks.
And masses and springs also this Saturday case.
So I'm open as always to questions.
Yes please, thank you?
2.2 number 6
2.2, number six.
OK. Yeah.
So this is, and of course you understand that, so I'm happy, that's a good question to discuss.
And maybe number seven people well have something to say about.
Good.
So that just fine, so let me start right in on those.
So, number six is the fact, I mean everybody understands that when energy is conserved, that's an important thing.
And so the question is first when is energy conserved in the differential equations?
In the equation we're trying to solve?
And if it is, then we want to know, we would like to choose difference methods that also conserve energy.
They may not be exactly right, they may not be exactly at a right point on this circle, if we're in that model problem, but still on the circle.
So, and the point is that the trapezoidal method does stay on the circle, and of course the differential equation stays on the circle.
Can I, so, and I put quite a bit into this problem six.
So this is 2.2.6, and and let me try say something about that, OK.
So, first of all, there's the continuous problem.
du/dt=Au.
When does that conserve energy?
And then there's the discrete problem, which we know that Euler's doesn't conserve energy, because we've seen it, the computer shows you right away.
It spirals up from the circle, it spirals in, or forward and backward, or whatever.
But trapezoidal method, is that the one that turns out to be well?
OK, so it refers to equation 24 as the trapezoidal method, and let me try to follow that notation.
Yep.
the trapezoidal method is this one.
Did we get music there for the trapezoidal method?
OK. U_(n+1)=(I+A*delta t/2)U_n.
OK.
Right.
OK.
So that, and actually problem seven, if you maybe want to discuss too, is the question of how accurate this is compared to the differential equation.
Everybody should see that this really came from the original way to look at this was (U_(n+1)-U_n)/delta t, that approximated the derivative equals A and then I'm taking half at U, at the new time and half at the old time.
So it's got that centering.
That we suspect will give us a little extra accuracy.
OK. so two questions then, one was the stability, so problem six was the energy conserved, and problem 2.2.7, if I anticipated, is the order of accuracy.
So these are both topics that are extremely important in choosing a difference method.
We would like to know first when is energy conserved there?
What differential equations have conserved energy?
Physically we kind of can see them coming.
If a physical universe is not being somehow, lots of physical problems we see those masses and springs oscillating, say OK, nothing's coming in from outside, how could energy there would be the sum of the kinetic energy of the masses, and the potential energy in the springs.
So energy passes between kinetic when the mass is zooming past equilibrium, and potential energy when the mass is stretching the spring so we've got two cases.
And we would hope, and this trapezoidal method comes through, that energy is conserved.
So can I just begin with this one?
Maybe I always ought to say, because you guys are also thinking about the quiz.
So, for example, this question about how to find the order of accuracy, I'll speak about that.
but let me just say that's not something that we've done in enough detail that I would expect you to be quick on the quiz and just be able to do it out.
I'll try to choose questions on the exam that you really have had more practice with.
But this is certainly important, so it was that definitely right to put on the homework.
And this is important.
OK, so let me tackle this one.
First the differential equation.
So by energy here I'm meaning just the length of u squared.
So now I'm looking at energy conserved, OK.
So I'm hoping energy conserved would mean that the derivative of u squared was zero.
That what conserving energy would mean.
And my question is which differential equations, what's the condition on A, in other words?
How would I recognize from this matrix A that I have this interesting property?
OK.
So, let me just show you what I would do.
Another way to write u squared is u transposed u.
These are vectors, of course.
So what's the derivative of u transpose u?
My equation is telling me what the derivative of u is here, I've got the thing squared.
OK.
I have a product here, right?
I've u's times u's.
So I'm going to use the standard, they happen to be vectors, so if I want to use like freshman calculus I'd have to get down to the scalar to get down to the numbers, but I absolutely could do that and just follow them along, component by component, or I could try to do it a vector, with a whole column at a time.
And let me try that.
It's going to be the product rule, right?
In some form I'll have this guy times the derivative of this plus the derivative, I'll keep them in order, the derivative of this thing, times this guy.
Right?
That's the product rule.
OK, now what do I know here?
I know that the du/dt is Au, right?
So this is u transposed Au.
And I know du, is that dt meant to be dt?
So du/dt Au, again.
Look, this isn't difficult.
It's Au transpose u.
And the question is, when is this zero, OK?
So those were the two terms from the product rule, and notice they're not exactly the same.
This is u transpose Au, and what's this guy?
u transpose A transpose u.
So if I put them together, I have u transpose times the A and the A transpose times u.
And I'm hoping that this will be zero, for all the solutions that I've come up with.
So the condition is simply that A plus A, we want that to be the zero matrix, A plus A transpose.
In other words, if A transpose is minus A, that's the good one.
If A transpose is minus A, this is zero, that's zero, that's zero, that's zero, energy's conserved.
So the energy is conserved when A transpose is minus A.
It's for the anti-symmetric A's that energy is conserved.
And of course this all makes sense.
What are the special solutions to that differential equation?
The special solutions to this equation are e to the, just, this is connecting now with things that really are basic.
The special solutions, the pure eigen solutions, the ones that follow their own paths, are the e^(lambda*t)x's, right?
Where x is an eigenvector of A, and lambda's an eigenvalue.
Those are the guys and we expect to have n of them, and we expect a combination of those to give us the general solution and to match the boundary conditions.
So these are the, these n of these guys with n different eigenvectors and their eigenvalues are the heart of problems like this.
And of course that's the additional homework problem that wasn't in the book but I added as an additional problem was exactly that, to get you to practice with these eigenvectors and eigenvalues.
OK, now, what's the deal, I want to connect this energy conserving with this picture of solutions.
When would this keep the same energy?
When would this have constant energy, constant length?
The length would be constant, since x is certainly, that's an eigenvector, whatever it is.
This is what's changing.
And now I want to know when does the length not change?
Well, the test would be that this number should have absolute value one, right?
If this keeps absolute value one, then in the eigenvalue picture I have energy staying the same.
OK?
Now, when will this have magnitude one?
Time is running along, this is either the lambda t, so which lambdas?
Zero, certainly, but now there's more, you gotta know the others.
What other lambdas will have, what other lambdas give me this thing stays on the unit circle, absolute value one?
Key question you must know.
lambda could be?
Imaginary.
lambda could be imaginary; right, lambda could be imaginary.
e^(i*omega*t).
That's just like basic fact about complex numbers, that if lambda's imaginary we would have cosine of something t plus i times the sign of something t, cos squared plus sine squared being one, we'd be on the unit circle.
So from this picture we would want the lambdas to be pure imaginary.
And now a little next step, what we'd like for eigenvectors, because the real solution will not be just one of these guys but a combination.
So when we have a combination each one is doing its thing, each one better have lambda imaginary but more than that, we would want the x's to be perpendicular.
Because if the x's interact, then this guy, one of these, you will say there's only one there, but I'm thinking of n of them there.
A combination of say, two of them.
Suppose I have an e^(lambda*1t)*x_1, and an e^(lambda*2t)*x_2, when does that conserve energy?
Well, each one will, but the combination will be fine if the x's are perpendicular.
Because if I have perpendicular vectors, then the length of the whole combination by Pythagoras is just one squared and the other squared and each of those pieces is constant.
Let me say what I'm trying to say.
That the eigenvalue, eigen function picture also tells us that we would like imaginary eigenvalues and perpendicular eigenvectors.
And that is exactly what you get from A transpose equal minus A.
So A transpose equal minus A is exactly, matrices anti-symmetric matrices, have perpendicular eigenvectors, just like symmetric, but the eigenvalues are pure imaginary.
Instead of all being real, they're all pure imaginary.
In other words, that answer and the discussion here came to the same conclusion, that a should be anti-symmetric.
OK. Now let me look at, is that OK, so that's a discussion which is worth knowing about differential equations.
When is energy conserved.
Now I want to do, or the problem asks me to do, what about this difference equation?
When is energy conserved there?
And I believe it will be, this is the requirement for the differential equation to be OK, to conserve energy and so I'm going to expect, I'm going to need that in this one.
And is that enough?
if I have this A transpose equal minus A, anti-symmetric, it was good for this, does it also do the job here?
Is this trapezoidal method just cool so that it will conserve energy, too.
And the answer, I think, is yes, and the problem was to prove it, or to see why.
So what do I now want?
If you don't mind my racing, I'm now going to look at the discrete guide.
So now I'm looking at when is U_(n+1) squared equal U_n squared?
That's what I mean by conserving energy in the discrete case.
At every step, same energy.
So now I want to look at the energy in U_(n+1) compared to the energy in U_n, and I want to see that this holds.
Probably, there's some smart way to do that.
Now we're down to just the math questions.
Math is always looking for some, you just sort of do the right thing, you stand back and poof it works.
OK, so what's the right thing?
Hopefully I have helped you and me by saying what would be a good idea to do here.
Can I just look?
OK, it say, oh, does it say what to do?
Yeah.
It says multiply by U_(n+1)+U_n.
Take the dotted product, that's interesting.
Take the dot product, so why did that work?
Take the dot product of both sides with U_(n+1)-U_n.
Did anybody succeed with this idea?
But that's the idea.
And hopefully we'll get it to work.
That if I multiply both sides by U_n+U_n, maybe better if I look at it this way.
I'm sort of OK to do it that way.
Suppose I multiply both sides.
So now I'm following on this idea, That equation I've rewritten here and without practice I don't know which one is the good one to start with.
But I'm pretty OK with starting with this one.
So what's my idea?
That's my equation, now I'm going to multiply both sides by U_(n+1)+U_n transpose.
Now I don't have room to do it, unfortunately.
I want to stick in here U_(n+ with a plus sign in there, and of course I have to do the same thing here.
OK. are you OK, do you see what I'm doing?
I want to show that this equation, which is the same as this equation, has this property which is a copy of this property.
Here would be another way to do it.
We could do it, the way we're going to do it now sort of compares with the way we started with the derivative of the norm squared.
I could also ask the same question by following eigenvectors.
I could also ask the same question by following eigenvectors.
I'm guessing that here the eigenvalues U_(n+1) is, you see I could do it both ways.
Maybe having just done eigenvectors let me do this one by eigenvectors.
So an eigenvector of A, when it's x itself, is, what happens to an eigenvector.
Suppose U_0 is an eigenvector x of A, What's U_1?
Yeah, you really should see this question.
So U_0 is the eigenvector x, then what is U_1?
Let me just write it here.
Ax equaling lambda*x.
So these are the eigenvalues of A, and we've learned that they're pure imaginary in this case when we're ready to go, and now I'd like to know that we get the good thing here.
OK, so if U_n is an eigenvector, what is U_(n+1)?
OK, so can I just do that, U_(n+1) is, so what do I have on that right hand side?
x and what is Ax?
It's lambda, right?
It's lambda*x.
So all this is one plus lambda delta t on two x but now I've also got to bring this guy over here, it's inverse.
And see what that does.
Now it's the inverse, so it's going to have the same eigenvector and the eigenvalue's going to go in the denominator and it'll be one minus lambda delta t over two.
OK, so that's U_(n+1).
Do you see what's happening here?
The eigenvector x, if we start with that eigenvector x, we come out with a multiple of x.
And this is the multiple.
So each find a different step multiplies by a number just the way each, in the continuous case we were multiplying by either the lambda t and in the discrete step by step case we're multiplying by that number.
Actually, this is why problem seven is important, because if we want to know how accurate the comparison is I want to compare either the lambda t with that number.
So problem six is asking a question about that ratio.
And problem seven is asking another question about that very same ratio.
Now what's the question for problem six?
When will this vector have the same length this x was U_n.
So I started with the U_n, I multiplied by this number to get U_(n+1), when do they have the same length?
When that number has absolute value one.
So if I'm watching eigenvectors, this guy had absolute value one because lambda was imaginary.
Now, what about this guy?
lambda's still that same lambda, imaginary.
What can you tell me about one plus, so lambda is some i, omega, delta t over two and down here I have one minus i omega, that's the lambda delta t over two.
I believe that that does have absolute value one.
Anybody tell me why?
So this is checking that energy is conserved for each eigenvector.
The energy, because the eigenvector is multiplied by that number and that's some number, it's some complex number, but I believe it has absolute value one and I believe you can tell me why.
Yep.
Because they're complex conjugates.
This numerator and the denominator are complex conjugates, in the complex plane here's the one, and I go up and either the i omega delta t over two, or on this one I go down by, but those lengths are the same.
That numerator, the length of the numerator is that guy, the length of the denominator is this guy, and their ratio is one.
So I think that this gives us the point about complex numbers.
That a complex number and its conjugate automatically have ratio of magnitude one.
You see the difference between Euler's method.
So Euler's method, so forward Euler Forward Euler would not have had this stuff on the left side.
It would all have been on the right hand side.
Forward Euler would have been about i plus A delta t. delta t A.
And what are its eigenvalues?
One plus i omega delta t, right?
With no, we're not dividing by anybody.
this part is up top too, so it's one plus i omega delta t. Now, does that have absolute value one?
Well, you know from the way I'm asking the question, what can you tell me about the absolute value of the forward Euler growth factor?
Greater than one.
Because this is the one, and this is the i omega delta t, maybe went up twice as far.
And there was nobody to divide by.
It's bigger than one, so it blows up.
And the backward Euler had only the one over one minus i omega delta t, so the backward was like this, one over it.
And lesson one.
But this balance has absolute value of equal one.
So, OK, that's the sort of heart of what's going on.
Can I, before I tackle the question using the hint there, which would take me on another blackboard, can I discuss question seven?
Were you going to ask me about number seven?
Yeah, I was
You were?
OK. Alright.
We get the answer.
So, question seven is about the accuracy.
So here's the correct number, this is my e^(i*omega*t), that's the correct number that I should be multiplying by.
And the actual number that I'm multiplying by is that much.
Or, in the forward Euler case, it's that one.
And so I'm comparing the one step accuracy.
So let me compare one step accuracy.
So this is the topic now, of order of accuracy.
This is question seven.
And it amounts to comparing the, so what is one delta t step in the continuous case?
So how much does the eigenvector x, what does it get multiplied by if I take a delta t step in the differential equation?
So this is the exact delta t step, what the find a difference won't get exactly right so the exact step delta t?
The differential equation, and of course I'm always looking at Ax=lambda*x, the differential equation multiplies x by what?
What's the exact growth factor, you could say, if my equation is du/dt=Au, that's the differential equation, and I'm supposing that I'm on an eigenvector x, so that the solution is e^(i*omega*t), or e^(i*lambda*x).
Now, what happened over a delta t step?
This is the answer like running along for all time, all I'm asking you to do is if the step is delta t, what's that number?
I mean that number is telling us how much it grew in that delta t step, and of course it's e^(i*omega*t).
That's the exact growth factor, that's G_exact.
In one time step, the eigenvector gets multiplied by that, because that's the amount of time that elapsed.
And what's the approximate growth, the growth factor from trapezoidal is just what we wrote down here.
One plus lambda delta t, maybe I'll stay with lambda rather than i omega.
Just use the lambda delta t, and this was one plus delta t over two.
lambda divided by one minus delta t over two lambda.
So question seven just says compare that with that.
Thinking of delta t as a small time step, if delta t is zero, then of course either the zero is one, if delta t is zero I get one here, they're correct if delta t is zero, that's no big deal.
How do I understand what happens for small delta t?
I'm comparing this exponential for a small delta t with this guy for a small delta t. How do you make comparisons for a small delta t?
Well, that's what Taylor series is all about.
Let's do the Taylor series.
What's the series for the exponential?
If delta t is small, I have e to some little number, tell me, start me out on the exponential.
One, thanks, one plus, lambda delta t plus, this is the exponential series, there are only two series in this world that are worth knowing.
Really, that's literally true.
In calculus you study all these infinite series, there are two that are important, that are worth remembering long after calculus.
And either the x, either the whatever because one of them.
OK, what's the next term?
Over two, lambda delta t squared over two, and then there's a cube guy if you don't mind telling me what's the denominator in that one?
It's three factorial six.
Good.
And onward.
OK.
So that's one of the series that everybody should know.
OK, how we going to deal with this guy?
We want to expand that, so what's my goal?
I want you to expand that in powers of lambda delta t and compare with this.
And see where, they aren't going to be equal, right?
At some point they're going to be different.
But at least they should start out equal.
So so here's the heart of problem seven.
How do I expand this in powers of delta t?
Do you mind if I just, this is just a number let me put it times one over, so this is times one minus delta t over two, lambda inward, right?
I just bring that up as a number.
So it's this guy times one over this guy.
What do I do?
This is, here's the moment when the math tools get used.
And I'm well aware that it's like years since you did calculus or series or whatever, and those tools get rusty.
And the point is that they're really genuine tools that we can now use.
So what do you think?
This is the problem one, this is the one coming from the denominator; this is 1/(1-x).
So I have a 1/(1-x) deal.
And what's the series for that?
I said there were two series worth remembering, and sure enough the exponential was one of them and now we're ready for the other one.
What's the series for that guy?
1+x, good start.
Plus x squared.
Right, x squared plus x cubed and so on.
Real simple.
It's all the same stuff with no factorials.
Those are the two series to know.
The exponential series and the geometric series.
Right, that's the geometric series.
OK, so that's what I've got out of this stuff.
Can I write it below?
I have one plus delta t over two lambda.
Let me just call that x for the moment.
delta t over two lambda is my x.
One plus x, and this is 1/(1-x), which you just told me is one plus x plus x squared plus x cubed and so on.
And now I've got to do that multiplication.
OK, x is, remember this is x, I'm just saving space.
Can you multiply those guys?
So that's one plus x times a lot of stuff here.
What do I have all together?
Well, the one, what's the next term?
Two x's?
Everybody spots the two x's there?
And then the next term, you have to get these terms right because we plan to compare with this guy and see how many we get.
How many x squareds are in there?
Is it two?
Looks like two.
Two x squareds.
And two x cubes, and so on.
Yeah, that looks right, OK. Now I'm ready, what am I ready for?
I'm ready to say what x is, x is this delta t over two lambda.
So what have I got here, one?
What is this guy now?
Two x's is delta t lambda.
Is this good?
Yes, right?
We're pleased.
Because the two x is the, two of these is delta t lambda and that's what we wanted to match.
Absolutely.
delta t lambda, lambda delta t. Now let's keep going.
By the way if this first term hadn't matched we would be extremely surprised.
Because that first matching is only saying that my difference equation is quite consistent, it's a reasonable creation out of the differential equation.
And we knew that.
The question is how much further are we going to get?
Euler will not get any further.
With Euler the next ones will fail.
But I think with trapezoidal the next ones are going to work.
Does it work?
It's like we're holding our breath, right?
Two now, I'm going to put in x squared and see about this term.
x is what?
x is this guy, delta t over two lambda.
delta t lambda over two squared.
And now you get the fun.
Because you're going to compare this term with what?
With this term.
And are they the same?
Yes.
Yes.
So that's the way, you see, that you got the extra accuracy which Euler did not give you, but that's why the trapezoidal rule is a is a second order accurate method.
OK, you may say that I went overboard to say all that.
You may say I didn't ask that question.
But it's the right question to ask about order of accuracy, and it's what problem seven was intending to bring.
Maybe I called it h in problem seven rather than x here.
Well.
Oh gosh, I realize I I'm supposed to come back to this one.
But some people might have other problems that they're interested in.
But let me, because time is pushing along, and the solution to this one will post, let me at least offer the possibility to ask me about something completely not six or seven here but something entirely different, like what's the first question on the quiz or anything.
And that, let me say I'll hope to know by Tuesday.
I love to teach, but making up exams is serious work.
Anyway.
Let me open a board and open to another question of any sort.
Any place, Chapter 1, Chapter 2, whatever.
Is there anything?
So I know that you're in the middle of this homework.
So I can say a little more here about that number six if you want, but I wanted to allow, yep
[INAUDIBLE]
The A, from today's lecture this was the incidence matrix, and this was the a transpose a that's probably still on the board somewhere.
Yep.
Yep.
So this is the A, which you should take in and be able to create if I gave you the graph, and this is the A transpose A, so it's through today's lecture, yeah.
Next lecture I'll be talking about the A transpose by itself, which involves Kirchhoff's current law, it's beautiful.
A transpose w equals zero.
But I think this part was straightforward enough to be able to add this to our list of problems which fit the framework.
So that's what that was about.
It doesn't mean that this will be on but it could be, right.
OK, what else?
You guys are patient, I come on, yeah, thanks
[INAUDIBLE]
Yep
This is only valid when x is less than one?
It's only valid when x is less than one, so that's now the math point that this expansion for e^(x) valid for all x's.
Because you're dividing by these bigger and bigger numbers.
But this one is only valid up to x=1.
At x=1 we're getting one plus one plus one, and we're getting one over one minus one, sort of infinity matches infinity, but then if x goes up to two, yeah what happens if x is two?
It's sort of not good, but you know mathematics, it's never completely crazy, right?
If x is two, what does this say?
What have I got on the left hand side?
Negative one.
And what have I got on the right hand side?
One plus two plus four plus eight.
I should not allow this to be videotaped, but that's actually not so completely crazy.
In some nutty way that could still make some sense.
That's certainly will not be on the.
So you're right that x should be less than one, and of course it will be here because I'm looking at little delta t's.
Little, so my delta t, x was this thing and my delta t, the time step was small and somehow that tells me, actually this is a good indication.
It gives me the units that stability and things going right will depend on lambda delta t. Will depend on lambda delta t, that's the key parameter there.
That's like the dimensionless parameter that we're, or lambda delta t over two, or whatever.
But lambda delta t is the key.
And a highly important key.
It tells us that as lambda gets bigger, as the matrix has bigger eigenvalues, delta t has got to get smaller.
And I mentioned stiff equations.
Stiff equations are equations where the eigenvalues lambda are out of scale.
You know, you might have two eigenvalues, one of size one and the other of size ten to the fourth, because you've got two physical processes going on at the same time.
And those equations are tough, because that ten to the fourth guy is forcing your delta t to be really small.
Whereas the action might, the true, real solution might be controlled by the lambda=1 guy.
So to follow this slow evolution, you're having to take very small steps because on top of that slow evolution with the lambda=1, there's some very fast evolution maybe with lambda equal minus 10,000.
Yeah, there's a lot happening here.
And always you have to think OK, is there some way around that box.
Because forward Euler would not get you through.
OK, thanks for that question, you got another one?
So then if you weren't using small enough time steps, [INAUDIBLE]?
If you weren't using small enough time steps, OK. For trapezoidal, let's say?
I mean, that expansion wouldn't hold if you were using a lambda--
Well, the expansion is really intended for a small delta t. Yeah.
It's not intended, I never added up the whole series.
I just compared a couple of terms to see how am I doing, and I got the extra term to match from trapezoidal that I didn't get from Euler.
So what's to say; if you took delta t too big, what would happen in the trapezoidal method?
Well, you would stay on this circle because the absolute value of this thing is truly one.
Even if lambda is enormous and delta t is way too big, we still had complex conjugates and their ratio was one.
So we would not leave the circle, at least in perfect arithmetic, as everybody says.
If we didn't make any round-off error, we would not leave the circle.
But boy would we skip all over the place on that circle.
So if we took delta t too big, we would be completely inaccurate.
We wouldn't be unstable, for trapezoidal, because it would stay on the circle, but the phase would be completely wrong, yeah.
So it would be a complex number of absolute value one, but it would not be close to the exact growth factor.
Well, so many things to say.
I realize that the course moves along pretty quickly but this topic of numerical methods for differential equations, that's a core part of 18.086.
So I'm like anticipating here in just a couple of days what really takes longer is the stability and the accuracy and the best choices for time-dependent problems.
OK, always good questions.
Anything else that's on your mind of any sort?
Yes, thanks
[INAUDIBLE]
114?
There is a figure 2.7
OK. OK. 114 figure 2.7.
Oh yes, OK. Oh yes
I figure it's about how these shapes [INAUDIBLE].
see.
That has a bunch of figures, so that in order to say for everybody who's not looking at the book, those figures are about the problem we've discussed here with a model problem, where we're on a circle.
So do I have space to draw a circle?
Well, let me just make space here.
OK, so page 114 has that model problem that we've drawn before.
There's the exact solution, here's the phase plane; there's u and there's u', and the u was cos(t), so the u' was minus sin(t), and we travel around the circle.
On the exact solution.
Energy constant, u squared stays one.
u squared plus u prime squared stay one.
Now which figure was it you wanted me to look at?
So
[INAUDIBLE]
Of any of them?
Yeah
OK, that's fine.
Let's see.
Is trapezoidal on that one?
Yeah.
Trapezoidal was the first one.
OK, so figure 2.6 shows the trapezoidal method moving around the circle.
So what happens?
Yeah, thanks, that's a very suitable question.
OK. and I took, in that figure I took, how long does it take for the exact solution to get exactly back where it started?
At t equal what do I come back?
2pi
t equal to 2pi, I'm right back where I was.
Right?
Cosine has period 2pi.
OK, now a single step of size 2pi would be really ridiculous, right?
I mean, I want to now delta t. So in that figure I took delta t to be 2pi divided by 32.
So I'm taking delta t to be the 2pi that would bring me all the way around but I'm dividing by 32.
So, what does that mean?
What what does the exact solution do at those steps?
32 steps?
It goes on the circle, 32 equal steps, 30, 360, 2pi divided by 32 radians every time, comes back exactly there, the exact solution.
And right where I started.
So it's like following a planet.
Now I do it by finding out differences.
So now I'm going to follow the trapezoidal rule, just what we've been talking about.
With that time step, and with the equation, everybody remembers the equation was u, u' equals, do you remember what the matrix was in that equation?
This is the derivative of it and this is u, u'.
Sorry to squeeze this in, but what I'm, u' is u'.
u'' is minus u.
Now we know why that matrix was good, right?
Why is that?
That's my matrix A, why is it good?
Because it's exactly, it fits.
A transpose is minus A.
It's anti-symmetric.
Keeps me right on the circle.
OK now, trapezoidal method keeps me right on the circle, 32 steps.
And so the picture just shows where it goes after 32 steps.
And 32, 32 does it come back there?
Well, not exactly, right?
We don't expect to find a different solution to be exactly in sync with cos(t), the real one.
But it's really close.
I think in that figure I can see that that's sort of a double point there, at 2pi.
I put a little arrow indicating small phase error.
It misses by a little bit.
And actually, roughly what does it miss by?
This was the point of the order of accuracy stuff.
Roughly what size is that little error?
That's what we did over here.
The term that we got wrong was a delta t cubed.
At each step.
Can I just tell you the answer?
The error here is of size delta t squared.
Because over here we match those series and we found the error was delta t cubed.
That's in a single step.
But now we've got one over delta t steps, you see what I'm saying?
That if the error was delta t cubed per step, and I have one over delta t steps, to get somewhere, or 2pi over delta t or whatever, then that gives me delta t squared.
So that little error there is my error of size delta t squared.
And that square tells me I've got a good method.
At least, decent.
Second order accurate.
And the trapezoidal rule is sort of the natural one.
Well, OK, so that's a full hour mostly devoted to two or three things.
Actually the eigenvectors came into it.
And the energy conservation came into it, the stability matching series came into it.
And the picture.
OK, I'll see you Friday for more about these guys, and then Monday evening please ask me everything you want to on Monday evening.
OK.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so this is a review session with open questions on homework.
Open to questions on topics in the exam that's coming tomorrow.
This morning I wrote down what the four questions would be about, and I'm glad I did.
I never should have done this many times before.
So you would know exactly and get down to seeing what those problems are.
And of course the matrices called K, and A transpose C A are going to appear probably more than once.
So, open for any questions.
About any topic whatsoever.
Please.
Yes, thank you
[INAUDIBLE]
The fourth question on the exam?
[INAUDIBLE]
I'm glad you used that word, fun.
Yes.
That's exactly what I mean.
Section 2.4, and they are fun, yeah.
So I drew by hand a little graph with nodes and edges.
And you want to be able to take that first basic step.
So the first step, which is as far as we got by last Wednesday, the first lecture on Section 2.4, was just creating the matrix A, understanding A transpose A, and of course there's more to understand about A transpose A.
Actually, why don't we take one second.
Suppose I have a graph with six nodes, let's say.
Can you imagine a graph with six nodes?
And every node connected to every other node.
So however many edges that would be.
Actually, my grandson just got that question on his exam.
He was told there were l islands with a flight from every island to every other island, and he was asked how many flights that makes.
So I sent him the answer.
But I was very happy with his reply.
He said "that's exactly what I got."
So, what do you know.
It seems to work.
So anyway.
Suppose we had, how many nodes did I say?
six?
OK.
So we have like a six node, so n is six, and it's a complete graph, this is really just to start us off talking about some of these problems.
So the matrix A, so I think it would be 15, where did I come up with that number 15?
And is it right, actually?
Yes.
This is one way, would be the first node has five edges going out and then the second node would have four additional edges, and three and two and one.
And five, four, three, two, one add to 15.
So would be the shape of A in that case?
So it has a row for every edge.
So 15 by six, I think.
OK, and I could create A transpose A just to have a look at it.
So it would be, what shape what A transpose A be?
Six by six, symmetric, of course.
Will it be singular or non singular?
Singular.
Singular, because we haven't grounded any nodes.
We've got all these nodes, all these edges, nothing.
We haven't taken out that column; when I reduce it to five by five, then it'll be invertible.
But six by six, so what will be the diagonal of this?
This'll be now six by six, the the size will be six by six.
And what will go on the diagonal is the degrees of every node.
That means how many edges are coming in, and what number is that?
Five.
So I'll have five down the diagonal, and what else, what will be off the diagonal?
Minus, a whole lot of minus ones, a minus one above and below for every edge.
And since we have a complete graph, how many minus ones have we got?
All of them.
All minus ones.
So all minus ones and all minus ones.
That's fine to cross over if you need to, sure.
I'm not sure what else to say about that matrix.
Well, it's not invertible.
Now let's take the next step which, I'm now going probably beyond the exam part.
Just really to get us started, I ground the six nodes, suppose I ground node number six, that wipes out a row and a column, is that right?
So I'm now left with a five by five matrix.
It still has all minus ones there and there, but now it's five by five, now it is what kind of a matrix, what are its properties?
Square, obviously, symmetric obviously, and now invertible.
Positive definitely, OK.
So it's fives there and now I would have five minus ones.
Let's just write them in here.
Typical row, now in this five by five matrix would have four minus ones and of course more here, and more here, and one there.
And symmetric.
All I want to say is that that matrix, we don't often write down the inverses of matrices, but that one I think we could.
I think we could actually, and it's a little bit interesting to know, for that special matrix, everything about it.
We could find its eigenvalues, its determinate, its pivots, the whole works for that matrix.
And that's one page of the book, maybe at the end of Section 2.4, I think, comes more detail about that matrix.
So in a way that special guy is like our special K matrix -1, 2, -1 for second differences.
Somehow this is taking, all nodes are connected.
Instead of in a line, springs in a line, points in a line, we now have everybody connected to everybody.
So this is sort of the special matrix when everybody is connected to everybody and we could learn all about that particular one.
But then, of course, if some edges are not in then some zeroes will appear off the diagonal in adjacent C matrix part.
The degrees will drop a little if we're missing some edges and the inverse will be not some simple expression.
Anyway, that's to get us started.
So that's really where the last lecture, Friday, brought us to this point.
And I'll take this chance to add in the block matrix just because I think of it as quite important.
So for this case, C is the identity.
So I would have the identity up in that block, A in that block, A transpose in this block.
That would be my mixed method matrix, you could say.
My saddle point matrix.
It starts out very positive, definite.
But it ends up negative definite.
And that's typical of mixed methods, when both unknowns, the currents as well as the potentials, are included in the system.
So A transpose w, that was f, I think, and this is b. I just mentioned that again, it was in Friday's lecture and it's in the book but I would just want to say I often refer to this as the fundamental problem of numerical analysis, is how do you solve that system.
And of course elimination is one way to do it.
When I eliminate w, that will lead me to the equation A transpose Au equals, I think it'll be, there'll be an A transpose b minus f, I think.
C being the identity there.
So that's the mixed method, this is the displacement method, and this is the popular one.
Because it's all at once.
But people think about this one, too.
So that's like saying what was in Friday's lecture and will be used going forward.
OK, that was just to get us started.
Now, please let's have some questions.
We need another question.
Who else?
Yes, thank you
[INAUDIBLE]
The first one in the homework.
What number was that?
Section 2.2, number six?
About the trapezoidal rule?
Yes.
OK, now I did speak about that a little in the last review session, but can I just say a couple words more about it here?
[INAUDIBLE]
What's it asking?
Yes.
People often ask me that about my problems.
I don't know.
You can't read my mind?
You should.
OK, so the point is that for special differential equations, so let me just summarize what we did there.
So this we did before, but I didn't do everything.
So what we did before was point out that the system du/dt=Au, conserves energy.
u squared, u of time, for all time, u of time squared stays constant if A transpose equals minus A. OK, essentially you take the derivative, it's got two terms because we've got a product there, a product rule.
The derivative will be, one term will involve a, the other term will involve A transpose, if out matrix has this antisymmetric property, those terms will cancel; the derivative will be zero, and that'll mean that this is a constant.
OK, so that's the differential equation.
Now, the question was about the difference equation.
So we're taking the trapezoidal rule and we want to show that un squared stays constant for the trapezoidal rule.
And so what that means, in other words is step by step, U_(n+1), and I could write it U_(n+1) squared, but the other way to write that and the way we have to work with it is that is the same.
Now, that was just an identity, that's just the meaning.
Now, I want to show that that's the key.
That's what we would want to prove.
That the trapezoidal rule copies the property of constant energy of the differential equations.
And of course, you know that in oscillating springs when there's no source, no forces coming from outside the total energy will stay constant.
And you could think of many other situations.
You have a spacecraft, where you've turned off the engines.
It's just going there, it's possibly rotating.
So there you've got angular velocity included in the total energy.
Important fact, if energy stays constant you want to know it.
And you're very happy if the finite difference method copies it.
OK, so then it was just a question of here we took derivatives to do that one.
Here we're going to be playing with differences, and my suggestion was that the good way to get it was to take that vector times the trapezoidal equation and show that this turned out to, the trapezoidal equation is something equals zero, and you hope, and it takes a few lines of jiggling around, that when you do that you'll get the difference.
You get exactly this.
You get U_(n+1) transpose U_(n+1) minus U_n transpose U_n.
That's the goal.
We know that the trapezoidal equation, maybe I move everything onto one side so I have something equals zero, then my trick is take that vector equation, multiply by that, play around with those terms and you'll get this.
So, since that is zero, this is zero.
And that's exactly what our goal was to prove.
So it's just in the jiggling around and maybe we don't want to take the full time because I'll post that.
Actually, I may post some of these solutions even before the quiz.
And therefore before the homework is due, just because this particular homework, as I say, is not, the graders are just going to be so busy with all the quizzes.
This is for learning.
Now, here's the one thing you want to learn out of this messy computation.
And also have a term, you'll also find a term U_n, when you just do this mechanically, you'll find a U_(n+1) transpose U_n, and you'll find a U_n transpose U_(n+1), and they'll come in with opposite signs.
That'll be when you've plugged in the fact that A transpose equals minus A, and all I wanted to do is ask you, what does that term amount to?
Because that term will show up.
One way or another.
And what does it equal?
Zero.
Everybody should know that.
That's the one thing, if you have to add to just mechanically computing is the fact that the dot product of that vector with that, v transpose w is the same as w transpose v. So, it's good to just call attention to the easy things that are like that.
Why is that?
That's because both sides, this is equal to what?
v_1*w_1, v_2*w_2, v_3*w_3, component by component.
And this is w_1*v_1.
But we're just talking numbers at that point.
So the numbers of v_1 times w_1 are certainly the same as w_1 times v_1.
Every component by component, they're exactly the same and of course then the dot products are the same.
So that's the fact which for this v and that w, make the term go away, that's still sitting there.
Other terms go away because of this property.
Having written that and recognizing that we have Fourier stuff coming up in the last third of the course, where we have complex numbers.
I have to say that if when I have complex vectors, do you know about those?
The dot product, or the length squared, if this was a vector of complex, with possibly complex numbers, I wouldn't take the length squared just by adding up these squares.
Suppose, my, yes, I'm really anticipating weeks ahead, but suppose my vector was .
What's the length of that particular vector, v?
Well, if I do v transpose v, what do I get?
For v equals , what does v transpose v turn out to be?
Zero.
One squared plus i squared is zero.
No good.
So obviously some rule has to change a little bit to get the correct number.
The correct length squared, I would rather expect two.
The size of that squared plus the size of that squared.
So I don't want to square i, I want to square its absolute value.
And the way to do that is conjugate one of the two things.
Now I'm taking , and on the other side I have and that gives me the two that I want.
So what I'm doing, when I've got complex vectors then I would really do that, and now that is not the same as that.
Right.
yeah.
If in one case if I'm doing the conjugate of v and the other case it's the conjugate of w, then I've got a complex conjugate.
OK, that's a throwaway comment that just is relevant because it keeps us focused for a moment on the real case, where we do have equals.
Now, I don't know if that was sufficient answer?
It wasn't a complete answer because I didn't do the manipulations, but the solutions posted should show you those.
And, of course, you can organize them a little different.
OK, good for that one.
Yes, please
[INAUDIBLE]
The other two terms here?
[INAUDIBLE]
Yes.
You couldn't cancel them.
Well, I recommend just, that's how the best mathematics is done, right?
You want zero, you just x it out.
Anyway.
Let me leave the posted solutions to be a hint and come back to it.
Yeah
[INAUDIBLE]
The next problem was?
[INAUDIBLE]
Oh, yes.
OK, that one I spoke a little bit about, but now let me read from the problem set that I got.
I noticed that was quite brief.
Oh, to find that actual angle?
Somehow that's a little interesting, isn't it?
[INAUDIBLE]
To tell the truth, I meant numerically.
I meant, what's the point of that question.
The point is we're trying to solve, this isn't a big deal.
But it was just if we're using this trapezoidal method, the beauty of that exactly what our thing proves is, here the constant energy surface is the circle.
The point of the trapezoidal method for this simple equation u''+u=0, which amounted to the equation uv' equals something like, our a matrix was antisymmetric.
So it fit perfectly in that problem, and if we started on the circle we stay on the circle.
And if I take 32 steps I come back pretty closely to here, and I just thought it might be fun to figure out numerically, with MATLAB or a calculator or something, we take an angle, theta, is that what the problem asks, and then come around here to 32 theta, and 32 theta will not be exactly 2pi.
But darned close.
Because you could see in the figure in the book it wasn't too far off.
So the question was, what is that theta?
So I think that the formula turned out to be that each step multiplied by this one plus i delta t, or h on two divided by one minus i delta t over two.
And when you plug in delta t to be 2pi over 32, so that's the, what did I say, that's the tangent of theta or something?
Sorry, I've forgotten the way the problem was put.
Oh, it's e to the i theta, yeah.
What's the main point about that complex number?
When you look at that complex number what's the most essential thing you see?
That it, yeah, tell me again
[INAUDIBLE]
Magnitude one, great.
It's a number divided by its complex conjugate, so it's a number of magnitude one.
And now tell me, if you see a complex number of magnitude one, what jumps to mind?
What form do you naturally put it in?
e^(i*theta).
Every complex number of absolute value one is just beautifully written in the form e^(i*theta).
That complex number is some point on the unit circle, so there it is.
Right there, there it is, e^(i*theta).
With that, theta is negative there, because we're going the wrong way.
No big deal.
Maybe here theta's positive.
I've forgotten, so I won't try to go either the clockwise or the counterclockwise way around.
So, if I wanted to figure out what theta was and plugged in these things, let's see.
So that's pi over 32, delta t over two would be pi over 32, and this guy would be its conjugate.
pi over 32, and then in this solution that'll be plugged on the homework this will be, I think maybe, the theta comes out to be, it's kind of cool actually, twice the arc tangent of pi over 32 or something.
I didn't know that.
But that'll be in the solutions for you to check.
So now, why do I like e^(i*theta) so much?
Because now I could tell you what this point is, after you've done it 32 times.
What's angle have you reached?
This is the crunch line of using complex numbers, e^(i*theta), is that they're absolutely great for taking powers.
If I take the 32nd power of x plus iy, I'm lost, right.
x plus iy to the 32nd power starts out x^(32), ends up i^(32), y^(32), with horrible stuff in between.
But what is the 32nd power of e^(i*theta)?
e^(i*32theta).
Just that angle 32 times exactly as we've drawn it.
So that's the point e^(i*32theta).
OK, and therefore if we now know what theta is, so yeah.
So it must be pretty near 2pi, but not exactly.
I guess that's about right.
In fact, having got this far, the tangent of a very small angle is approximately what?
It's approximately the angle, right?
The sine of a very small angle is approximately the angle.
The cosine is approximately one.
The tangent is approximately the angle.
So this, 32 theta, is 32 times two times approximately the angle.
And what answer do you get?
2pi.
Which makes sense.
So you could say what the trapezoidal method has done is to replace the exact angle by the inverse tangent approximately.
That's sort of nice.
In this example you can get as far as that and you could actually find out how near that is.
And, by the way, how near what I expected it to be.
I would expect it, so what do we know about the trapezoidal method without having proved it?
It's second order accurate, right?
If it was first order accurate, I would expect it to miss by something of the size of theta.
Maybe a fraction of theta.
But being second order accurate, I'm expecting it to miss by something of size theta squared.
So it would be pretty near zero, right.
And actually, another way I know it, around, and so the error would be something like, it would have a 32 squared in the denominator.
And I've just thought of another way to see that.
We just said that the first term in the arc tangent of a small angle, theta, of a small angle, alpha, whatever that is, pi over 32, the first term in the arc tangent is?
The angle.
That's what we just said.
Then, do you know what would come next?
Now we're looking at the error.
So that of a very small angle will start theta, and I want to ask you about how many theta squareds and theta cubes.
You're seeing what you can do with paper and pencils type stuff.
Here's my main question, how many theta squareds in there?
You want to make a guess?
A mathematician's favorite number, zero.
Right, there will be no theta squared terms in.
That's an odd function, so I'm expecting only odd powers and therefore I won't be surprised to see theta cubed come up first.
And then when I multiply by the 32, I get the theta squared that I guessed we would have.
OK, once again I'll stop there because that's very narrow path to be following but it shows you how.
You know, there's a lot of room still for what you can do with paper and pencil to understand a model problem.
And then the computer would tell us about a serious problem of following the solar system for a million years.
OK, another totally different question, if I can.
Yes, thank you
[INAUDIBLE]
Yeah, sure
[INAUDIBLE]
2.4.1, right.
A mistake in the book
[INAUDIBLE]
Or in the, yeah.
It's quite possible.
OK, there's a printed error in the graph.
Yeah.
So in numbering the edges, well let's blame it on the printer, right?
Not the author.
OK, so the diagonal edge, that five probably was intended to be a three, yeah.
Thank you.
So we'll catch that in the next printing.
And you recognize that always numbering the edges and nodes is a pretty arbitrary thing, it's just if we number differently that just reorders the rows of the matrix.
If we number the edge differently, it'll reorder the rows and it'll reorder rows and columns of A transpose A.
So it won't make a serious difference in the matrix.
Yeah
[INAUDIBLE]
Do you want to go back to this guy?
So if you have an anti-symmetric matrix, does it follow that the eigenvectors used are perpendicular?
This is a good question.
This guy, way up here, with this property,
The eigenvectors are perpendicular?
The eigenvectors are perpendicular.
Yes, yeah.
So we have, there's this, like, the nobility among matrices are the ones with perpendicular eigenvectors.
So that includes symmetric matrices, this is a good and straightforward point.
So these are the good matrices.
Symmetric matrices.
A transpose equals A.
Their eigenvalues lie on the real line.
And these are all perpendicular eigenvectors.
What about antisymmetric?
OK, that means A transpose is minus A.
They also fall in this noble family of matrices, and where are their eigenvalues?
Pure imaginary, right up here.
Now do you want to know, who else is in this family?
What's the other, this is the complex plane; there's one more piece of the complex plane that you know I'm going to put.
Which is?
What else to make that complex plane look familiar, it's going to have the unit circle.
Every complex plane has got to have the unit circle.
OK, so these guys went with the, and now what do you think goes with the, this will be the matrices.
Can I call them Q instead, because I called them Q this morning.
Q transpose is Q inverse.
Q transpose Q, and they're the orthogonal matrices.
So those matrices again, beautiful matrices in the best class.
And their eigenvalues are on the unit circle.
And that would be z.
Why don't I just show you why?
Because orthogonal matrices, there are not so many that are really worth knowing.
So, let me take Qx=lambda*x, and what is it that I want to prove?
I want to prove that the eigenvalues have absolute value one.
That's the unit circle.
So how do I show that the eigenvalues have absolute value of one?
Let me take the dot product with Qx transpose.
So both sides, I'll do Qx transpose Qx and I'll do lambda*x transpose lambda*x, right?
Only these are complex.
I've got to take complex stuff.
OK.
I just took the length squared of both sides, and kept in mind the possibility that this x and lambda could be, and probably will be, complex numbers.
Now, what do I have on the left?
Do you see it happening?
I get an x bar transpose, what do I get?
Q transpose Qx on the left side.
That's the combination I'm looking for.
For an orthogonal matrix.
Let's imagine the matrix itself is real, otherwise I would just conjugate it.
What nice about that left side?
What fact am I going to use about Q?
Q transpose Q is the identity.
So this thing is nothing but x bar transpose x.
That's the length of x squared.
What have I got on the right side?
I've got the length of x squared times the number.
Lambda bar times lambda squared.
It's there, now.
On the left side I have the length of x squared.
On the right side I have the length of x squared times that number, mod lambda squared.
Therefore, that number has to be one and the eigenvalues are on the unit circle.
So, I've given you the three big important classes of matrices with perpendicular eigenvectors.
I think anybody would wonder, OK, what about other eigenvalues.
What's the condition for perpendicular eigenvectors that includes this.
And includes this.
And includes this, and also allows eigenvalues all over the place.
Would you like to know that condition?
What the heck.
That condition, that includes all these is this, that A transpose times A equals A times A transpose.
That's the test for perpendicular eigenvectors.
A transpose commutes with A.
So this passes, of course.
This passes, of course.
This passes because both sides are the identity, and then some more matrices pass also.
OK. Is that alright?
You asked for some linear algebra and you got it.
Now I'm ready, yes, thanks
[INAUDIBLE]
2.4.19.
Oh, let me look.
2.4.19.
Ah.
OK, yes, sorry and I should have done better with that.
So one graphs that are important are grids like this.
And we'll see them two, three, four, one, two, three, four.
That would be where, these are the nodes.
So this is a grid.
I meant to draw them all in, but I won't.
n squared.
So n is six, and I'd have 36 nodes.
And you can see the edges in there.
So that's the graph I have in mind.
And the reason that problem is there is that last year, I think it was last year or the year before, we spent some time with figuring out resistances and currents and so on for these problems.
And we needed some fast way to generate A, because this matrix A is now, what size is the matrix A?
It's got, I don't know how many edges does it have?
One, two, three, four, five, maybe 30 edges going across and 30 coming down.
It'll be 60 by how many columns in this matrix?
You know the answer now and it's worth knowing, for the quiz of course.
36.
OK.
Anyway, that class rebelled about creating these matrices and working with the matrices, with 2,160 non-zeroes.
People were dropping the course.
So we need a command that would create A pretty quickly.
And so that's what the book, and so this was like the 18.085 command.
After we stumbled around for a while, we discovered that a MATLAB command called kron was a quick way to create the matrix.
We'll see that when we get to that point.
This is an important graph.
And it's closely connected to Laplace's.
You remember Laplace's?
I'll just tell you.
Laplace's equation is this, you have a second x derivative, we know how to deal with those.
But it also has a second y derivative.
So I'm really looking ahead at the most important equation of Chapter 3, Laplace's equation.
And suppose I use finite differences.
i want a matrix K2D that deals with this 2D problem.
And let me just say what it would be.
At a typical point this x derivative is giving me a minus one, a two and a minus one.
And the y derivative is giving me a minus one that moves this guy up to four and minus one.
So instead of -1, 2, -1 along a typical row, we'll now have a four on the diagonal and four minus one in a certain pattern.
Anyway.
You'll see that, it's interesting when we get to it.
That would show up in A transpose A.
So what I've said here is what happens with A transpose A. I guess I'm hoping that you begin to know these matrices, first seeing them occasionally in homeworks and then in the lecture.
Good.
But that's looking ahead.
I needed some questions that are like, close to really on what we're doing or what the quiz would do.
Any - thank you
[INAUDIBLE]
Oh yes.
A little bit.
OK, yeah.
So I wrote down this equation and what I'm writing right there is the new part.
Sort of new, and I guess equals some right hand side f(x).
And the discrete version will be an A transpose C A equal a vector f, maybe there will be a delta x squared here.
OK.
I guess maybe, I don't want to go far but I want you to see that if we have a coefficient C in here it should show up there.
You could actually, it might be reasonable to look at 3.1 just to look slightly ahead to see the parallels but you would get them right without a lecture on it.
Your coefficient shows up in the differential equation, and it shows up on the diagonal of C in the difference equation.
I won't give a whole lecture on that, just to say that correspondence is exactly the one we know.
A is a difference matrix.
Like the derivative.
C will be a diagonal matrix, A transpose will be whatever that comes out to be.
And so you've seen A transpose A, but think again about that difference.
And ask yourselves this.
I suggest, take C to be one, get C out of there.
And just think, again, what is the difference matrix A with a boundary either fixed-fixed or fixed-free, those will be two different A's.
What are the A's for fixed-fixed and for fixed-free?
This is what we were doing at the very beginning of the course.
So A is a first difference matrix, and A transpose A will be the second difference.
So the A transpose A, of course, I was doing A transpose A, then the answer here would be the matrix K and the answer here would be the matrix T. Or, depending which end is free, but we'd have one change.
That's A transpose A, but now think about the a that it came from.
So A will be, A is the matrix that takes differences of the u's, and then A transpose a takes second differences.
Of all the questions asked, this is the one most relevant for the exam and for what we've done so far.
I've gone off onto topics that we look ahead to, but this is where we are.
So that matrix a is a first difference matrix, and then you put in the boundary conditions.
OK.
There was another question.
Yeah
[INAUDIBLE]
Of number four?
Which number four in which?
Oh, in the quiz.
Oh yes, right.
Yes.
Did I tell you what problem four was?
No.
I hope not.
OK problem four in the quiz.
It's about a delta function?
Yeah.
What do I know, what do you want me to tell you?
So the delta, of course, comes in as we've seen it, as the right hand side of a differential equation.
So it might be the right hand side even of this equation.
So if this equation was delta(x), or x-1/2 or something.
I mean, the essential thing is that when delta's on the right side, that gives you a drop in the slope.
Suppose I just have a first order equation like d. I'll call it dz/dx=delta(x-a).
Yeah.
And suppose that I know that z(0) starts at zero.
You've got to be able to solve that equation, so this is a useful prep for that.
That would be a good equation to know the solution to.
And what kind of function is this?
What kind of a function is z(x) there?
I just use the letter z to have a new letter.
z(x) will be a step.
Right.
z(x) will be a step function, yes.
That's right.
OK, so the solution is that at the point a, which I'm presuming is beyond zero, I come along at zero and I step up.
Yep.
OK. That would be a picture of z(x), yeah.
So it's basic delta function material that I'm speaking about here.
So z jumps by one and z is a slope, then the slope jumps by one or drops by one, depending on a plus or a minus sign here.
The things that we've used to deal with delta functions, so that's what Question 4b is about.
The drop in slope, or the jumps, or whatever happens when a delta function shows up on the right side.
Good question.
Yep
[INAUDIBLE]
1.1.27.
Well
[INAUDIBLE]
Oh, and then left a typo
[INAUDIBLE]
Oh.
I'm sorry, OK. 1.1.27 My copy isn't showing it.
Yeah.
I may have to punt on that question.
Or do you want me to look at it?
OK, can you maybe just pass that the book up, alright.
I'll try to read what that question was.
OK. Yeah, maybe this is a question to answer.
This is probably the one new question that got added.
OK, yeah.
Fair enough.
So this is continuing the discussion that you asked me to start here about a, the first difference matrix, OK.
So I'll go a little more over that.
So in the book here, this writes down a matrix A_0 I'll discuss this matrix.
So there's a difference matrix.
You see what I mean by a difference matrix, if were to multiply it by u, <u_0, u_1, u_2, u_3> or something, I would get differences, right?
I'd get u_1-u_0, u_2-u_1, and u_3-u_2.
Good.
So that's A_0 times u. Alright.
So that's a difference matrix.
What graph would that come from?
That's also the incidence matrix of a very simple graph.
This is connecting Chapter 1 with Chapter 2.
It would be a line of springs, it would be a graph.
It's got edges and nodes.
it's got three edges, so I've got three rows.
It's got four nodes so I've got four columns.
Are the columns independent here?
No, they never are.
The vector of all ones would have differences of all zeroes.
So what would that, that would be the difference matrix for fixed?
Free?
Fixed, free, circular what would that difference matrix correspond to?
Everybody's saying it.
Say it a little louder just to.
Free free.
That's a free-free problem, because they're all in there.
We haven't knocked any out.
There are no boundary conditions yet.
That's a free free, so that A_0 would be free free.
OK.
I'll take one more case and then I think we're at time.
Suppose it was fixed fixed?
What would be the difference matrix that would correspond to, how would I change that matrix if my problem became fixed fixed?
So now I'm fixing that u, I'm fixing that u, in the mass spring case I'm adding supports there.
How would that change the matrix?
First and fourth, good.
Say it again?
First and fourth columns would go.
So fixed free would then be three by two.
Free free was three by four.
Yeah, I'm glad this question came up because this is the right thing for you to be thinking about in connection with the recent question you asked.
OK. Maybe that's the right place to stop, because now you've asked questions that are really on target for what we've done, and I hope useful to you.
OK, see you guys tomorrow evening at 7:30 in 54-100, OK.
under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I'm hoping you will ask some questions today.
So we've had the exam.
If you have any questions about grading the two TAs are the people to speak to.
Remy's graded Questions one and two, and Peter graded three and four.
And they control things.
I mean, I can help if there's an emergency, but they would be the right people to speak to.
Because they know how they graded the whole test.
Before you ask questions, can I just say why this truss, you remember this six-sided truss, tied me in a knot and I'm hoping your MATLAB solution will untie that knot.
The knot I was in was to find the mechanisms, to find convenient mechanisms because we have, well, I thought we had six bars.
It looks like we have six bars.
But somebody pointed out after class that that bar six is not very active.
It's connecting two supports, can't do anything, and actually that sixth row of A matrix will be all zeroes.
So our matrix, if we include that, there was no use, it doesn't help.
So our matrix then really comes from the five bars, so A is then five, by two unknowns.
Two, two, two, making altogether eight unknowns.
So three mechanisms and that's what I'm hoping for.
So when I unwisely drew that picture on the board at the end of the truss lecture, I was only looking for two mechanisms because I was thinking we had six edges, six bars, but really this bar, when I knock out the columns that correspond this, that node and that node, there will only be zeroes left in that row.
And it's nothing.
It correctly tells us that the stretching of that bar is zero but we knew that anyway.
OK, I don't know if you've tackled the MATLAB question, and I also don't know whether MATLAB would produce for us.
I mean, you should be able to construct a with a whole lot of square roots of three and over two from sine of 60 degrees and maybe 1/2 from sine of 30 degrees.
a should look pretty nice, but what the solutions to these mechanisms are, of course solutions to Au=0, and.
Anyway, I'm hoping that we learn from this example.
I hadn't intended so it's pretty small MATLAB, it's really just the creation of the matrix A. OK, so that was a comment on that.
Which I added to the homework, now I'm ready for questions about the homework, the exam whatever, yes thank you.
Oh, good
[INAUDIBLE]
For trusses,
[INAUDIBLE]
Yeah.
a TA immediately, couldn't you?
It's not quite so straightforward.
And this is like more realistic.
I mean, yeah.
Ah.
I guess, yeah but remind me what that question was.
This was a 2.7 Problem one?
OK let me just see what I'm asking for.
So, OK, yes.
So I only asked about A transpose A, I only asked you for the shape in that question.
Oh, and then I asked you for the first row, good for me, yes.
I see, the first row so I didn't put you to the agony of writing out the whole thing, but still how do you get the first row, good question
[INAUDIBLE]
Oh, I see.
Now, well it's not going to be so neat.
Let's just think.
We could do the first row of a, and the first column of a probably, yeah.
So what am I looking for in A transpose A, because it didn't say row one.
OK, so row one of A transpose A corresponds to the first, oh, so what is row one?
Yeah, this is worth thinking about.
So this is A transpose A for trusses.
And let's just, maybe we could even take this one as an example.
If I number the nodes, if that's my first node, A transpose A, you remember that's square.
That tells us the edge part, the bar part is built into it but its size is n by n, its size is five by, no, its size is what, eight by eight, right?
It's got to do with the number of, OK.
So its first row, what will its first row be about?
It'll be about u H 1, right?
The first row of this a transpose a matrix will be about, the first node but more than that, that node has got two things, u H and u V, and we're putting H first.
So so I think that the first row of this, let's just draw it.
Maybe I'm not seeing.
So let that be A transpose A, I see yeah.
a transpose a will be multiplying u H 1, u V 1, and so forth.
Right?
OK, good.
So now this is better.
All I'm asking is where do we know there are zeroes in that first row of A transpose A?
Where would we know that there are zeroes for this problem rather than, I won't redraw the one in the book so I'm taking this truss.
So if it's this truss, this is the first node, the bars are numbered like this, and of course your MATLAB construction of A, you might multiply out A transpose A.
Because in MATLAB that would be so quick.
And you could see what it looks like.
And where would we expect to see zeroes or not zeroes?
Let's see, A transpose A, so I'll have to think.
My instinct is that it should only connect two neighbors.
So that I would imagine, but I've got these are double neighbors, right, there's two people living here.
We have to remember.
So I see two people living here and two people at home, so I guess I would imagine four non-zeroes for this problem in in the first row.
I would think that that would have somebody on the diagonal.
That would be what multiplies u H 1 itself, and then maybe u V 1 is involved, and these two guys.
These two and those two would not be involved.
Well, that's only a partial answer, I'm just telling you where the zeroes are, I think.
And you're really asking about the non-zeroes, of course.
So, yeah.
Not so easy.
Yeah --
[INAUDIBLE]
-- maybe have to do it.
OK, yeah.
I think, the first time better to do it, yeah.
But that's an excellent question, and maybe we will find a nice, and if somebody does, let me know.
A nice way to see that.
Somehow I must have felt that was doable when I created that problem.
OK.
The other thing to say though, that while I'm looking at A transpose A, is remember that it can be created by these element matrices.
Yeah, yeah that's important to say.
So there will be five element matrices, right?
Because I've got five bars.
And the element matrix for this bar, or say for a typical bar, the element matrix for this bar will involve all the u's that are connected to this bar.
So two u's there and two u's there, so it'll be four by four, the element matrix here.
So this will contribute four non-zeroes to that to row and to three other rows.
That four by four guy from this part is, you know what I'm speaking about here?
The element matrix which is just, the element matrix is a cosine, sine, minus cosine and minus sine.
Column times row.
Cosine, sine, minus cosine, minus sine.
This is the element matrix for this bar, those are the cosine and sine of 60 degrees, and those positions would be these two and these two.
So there would be actually four zeroes coming after, and after here from those two and those two.
So this bar contributes to a transpose a, sort of stamps into A transpose A.
This four by four of non zeroes plus the rest zeroes, stamps that in up there.
This guy has its own element matrix.
Now, what's different about the element matrix for this guy?
Of course it's got its own c_1, probably, and it had a c_2, this difference constant.
But bigger differences than that.
First of all, it's a different angle, theta.
Here the angle is 120 degrees, and also there's nobody home down there.
So it would be, if we were dealing with A_0, free-free stuff, it would be four by four, like all the others.
But because these two are fixed, those two displacements are fixed, effectively it will only contribute two by two.
A total of four non-zeroes.
But those will stamp in to A transpose A overlapping this guy.
Then the ones from these bars will not touch the first row of A transpose A, so they wouldn't touch the answer to that homework problem.
So you see again another way.
By hand I don't know that you necessarily want to use these element matrices, I'm not sure.
But it gives an excellent check.
So one way to create A transpose A is create a multiplier.
That's one way.
Second way is, create A transpose A element by these five element matrices popped into the right places.
I think this is, so this like the computational science part of the course.
It's the way you would, I'm really speaking about the way you would write the code, not the final result.
So much of the course has been devoted to understanding A transpose A, and the fact that it's positive definite, or in this case only positive semi definite because we have an eight by eight matrix whose rank is only going to be five.
So this matrix will have these same mechanisms, and I'm hoping to learn what they are.
Well I hope that's a little help with that question.
But basically, though, it's not help because I'm sort of saying you're on your own to actually, I don't have a superfragilistic way to construct a matrix and then you kind of have to do it.
But I'm glad you asked.
Yeah
[INAUDIBLE]
More on this problem, OK.
The sign convention
[INAUDIBLE]
Yes.
So let's see.
So I made a speech about sign conventions, right?
And which was that I am too old for this.
And the justification for that is the fact that they wash out in A transpose.
So if you were to use the wrong sign convention in A, you won't see it in A transpose.
So if I change the whole thing.
So sign convention is not really a convention.
Our convention is to decide that that movement that way is plus.
And movement this way is plus.
And here, similarly.
That's a plus movement and that's a plus movement.
That's our sign convention.
If we always do that, then the A matrix is telling us how much this bar stretches.
It's the matrix that gives us stretching from these four displacements.
And then, these, I think in this picture, if this is the later node and this is the thing, then that angle from the horizontal is the right guy to put there
[INAUDIBLE]
Here?
[INAUDIBLE]
Well, I guess I'm, I see, the question is what's the sign convention with this particular picture?
Yeah, if this is node one and that's two, then question is good.
It looks, I think, so I think I've got it not right.
If this is node one down below, and two up above, then moving two positively would stretch the bar.
And I've got minuses there.
So I've got the opposite sides.
I've got there, the signs that would go with this number.
If that was node two and this was node one.
Yeah, you're right, good.
You have to, and that's the only way I do it, is to think will the movement stretch to the bar.
Then that's got a positive entry in there.
Will the movement like that compress the bar, that'll be a negative entry.
And our conventions are if it stretches that's positive.
e positive means the stretching; e negative means the compression.
So it takes a little patience.
Or it takes writing the code correctly once and then of course it would do it for all these problems.
Yeah, I think the TA in an earlier year actually wrote a code to a truss code.
I don't know what happened to it but I could look
[INAUDIBLE]
Yes, yeah.
That's right, and it will come out consistent.
Yeah, if I take horizontal, if I take the same convention that forces that way our plus, that's a plus f H 1 and it's a plus u H 1, then I'll get a will go to A transpose.
Yeah, so if I'm consistent that's plus for the u's, so this displacement is plus for the f forces and the correctly created a will give me the correct A transpose.
But A transpose A has the ability to vary some of those conventions.
OK. Good.
Yes
[INAUDIBLE]
Different.
More.
Oh my God.
OK.
Yes
[INAUDIBLE]
This one, yeah
[INAUDIBLE]
Yeah
[INAUDIBLE]
I wish it would ask about bar three.
Yeah, yeah bar three was good.
Yeah right.
Yeah, I guess here, what's here, well, the angle is, that must be theta, and then this'll be right provided this number is the, is what?
Well, I've got to think it out again.
I'll just think it out.
Maybe, yeah I won't try to, yeah I won't.
I won't.
So I think you, but it's really, you learn the point by thinking that through.
I think we've got it here.
That if that's number one then we like plus sine here because positive displacements will stretch the bar, right.
And here this sign will depend on the angle.
And it will depend on the number, which which numbers, what number that one is and what number that one is.
If I reverse the numbers, then I reverse the side.
Yeah, good.
Well, that's sort of part of computational engineering isn't it?
Like, keep track of the details.
But of course this 18.085 course is also about big picture things.
And that's what I'm hoping comes through even better.
OK, ready for another question on any topic.
Thank you
[INAUDIBLE]
Yes.
Well, no, but let's just try it.
Shall we do Problem one?
Was it Problem one?
in 2.7?
OK, let me just draw that truss.
And let's just talk about it.
So, boy I've put a lot of bars in there.
OK, so this is just one bar down to this guy.
And this is one up to here.
And was that everything?
OK, and then I've numbered them.
Let me just copy the number here, so that we can talk about that.
And I numbered the nodes one, two, one.
Joints, I should say really.
Four.
OK, all right.
Let's just start as I always do by, what's the shape of the matrix A?
A is what, how many bars have I got?
So help me through this now.
Six bars, and how many unknown displacements have I got?
Eight, OK, good.
And have I got any rigid motions here?
No.
I've got two, these are going to prevent all the rigid motion.
So if this matrix, if A has rank six, which we'd probably guess it has, you know that there's an if there.
Then two mechanisms, right?
Eight minus six gives two mechanisms.
OK, and your question is how to find them.
You've got one.
Alright, which one have you got?
[INAUDIBLE]
These guys.
So tilt, so that's a mechanism that only involves u H 1 and u H 2 and none of the other u's.
OK, that looks good.
So that's certainly one.
Now we're looking for a second one.
What if, anybody suggest one?
Let me just give everybody a thought.
If you haven't already started on this homework, you're starting now.
So I'm looking for another mechanism, and of course you might say well, suppose these bars go this way.
Suppose they go to the left, and you know that that's not going to do it right.
And why not?
Why isn't that an OK second answer?
Because it's effectively the same as the first; in fact the mechanism u, how would the u, the one I've drawn and the u for this way, what would we see?
Opposite signs.
If this describes a u 1 H and a u 2 H positive, the other way they're negative, it would just be the opposite sign; nothing new.
OK, so now you've had a look, so tell me.
If somebody sees, to answer your question here, how do you see them?
I don't know.
Just look harder.
Like you know, speaking French.
Just say it louder and maybe it'll work.
Of course we do have the possibility to create the matrix and look for solutions.
But it's more fun to do it this way.
And now who's going to suggest another mechanism, another view?
[INAUDIBLE]
Three and four.
Wait a minute; what was that three?
Oh, these nodes, OK.
These nodes, three and four do what?
Oh yes, sorry.
Three go up this way, ah.
I see you're rotating.
So this guy goes sort of along this way, you're taking that top square and turning it.
OK, so these guys will also turn, right?
Yeah, one and two will move on your mechanism.
So the idea is take that top thing, and I'm allowed to turn it, I'd say I have to turn it, I have to keep this bar, yeah I cannot stretch it.
I can't stretch any bars.
So when I do this turn, I can't just keep it in place and turn.
No.
But I'm going to turn it so that this one goes this way and this one goes this way, and now.
Alright you're, you're responsible for telling me about the other two now.
What do they do?
[INAUDIBLE]
Bar four, Will bar four keep its length?
Ha.
OK, well we're not allowed to use fancy words like four-bar linkage.
Because somebody might say what does that mean.
OK, does it, do you see that?
I think it does.
But it's wonderful.
If that's the original bar and I bring this up a little; a little, remember and this down a little, I think that the new bar has the same length.
You believe that?
No.
Some, yeah, but what happened there?
[INAUDIBLE]
Yeah, this is a good example
[INAUDIBLE]
Sorry, yeah, certainly bringing that up tended to make the bar shorter, but then moving this down tended to make it a little longer
[INAUDIBLE]
Right
[INAUDIBLE]
Yeah, I think so.
See, that was the key.
We went at 90 degrees to five.
And by going at 90 degrees, then the stretch in there was only that one minus cosine that was higher order.
That's the key.
Similarly, here, all these so we're avoiding stretching by this trick.
So maybe this guy is going down this same way.
And this guy is going up that same way.
Yeah, because then that bar is just translating, and this bar is translating and these two are doing the same thing, which I believe is not stretching.
Is this the answer that maybe somebody already got?
And believed in?
Yeah.
So to answer your original question, it wasn't obvious, was it?
But I think that is the right thing.
And actually, you could create the A for this problem.
Well, it's a bit large, 48 entries, but because four of the bars are horizontal or vertical, you will have many, many, many, zeroes in the A matrix.
It would be sort of fun to create the A matrix and then this.
So what is this displacement that I believe in?
This u mechanism that we've talked about.
Let's see, if I look at one that was positive positive.
If I looked at node two, that was positive over but down.
So horizontal but down.
If I look at node three, that's positive positive.
Yeah, one and three is hanging on.
And number four is like number two, one, and minus one.
I think that's the u which should solve Au=0.
Yeah.
OK.
But that's a good example.
OK, so that's trusses.
What else?
[INAUDIBLE]
Thank you
[INAUDIBLE]
Oh, well, let's see.
That's a good question and now I guess we're, ah.
Yeah well, the way I've drawn it at 45 degrees, which is what I wrote here, then I did build in, I did make that a 40, I did make this 45 or 135 or something.
And 45.
To get that, if that goes at 45 degrees, then this had better go at 135, right?
So that has to be a right angle.
If the support was over here then the angle that would have gone off with was changed.
Yeah, so very good point, that the numbers I've written down on the picture I drew required these angles to be those nice numbers.
I hope you like these trusses a little.
I mean, you get some freedom to visualize a little.
Good, yes
[INAUDIBLE]
Maybe.
Let me see.
I thought you were going to say, could I have two and three go inward and one and four go outwards?
You don't like that.
I would go with that.
Is that any good?
Well, OK. Yeah, linear algebra spoke there.
It's a combination of the other two, yes.
If we only got a two dimensional
[INAUDIBLE]
That one
[INAUDIBLE]

[INAUDIBLE]
Yeah
[INAUDIBLE]
Yeah
[INAUDIBLE]
Yeah
[INAUDIBLE]
OK.
But let me, when you suggested one, I overwrote it.
Now, the one you suggested, when you told me, OK, bring these in, I thought, OK, these have to go out
[INAUDIBLE]
They're not allowed to change lengths, so they can only swing around these pin joints.
So the picture of the one that I drew after your question would be, these guys, this guy, let me draw the square, as it was.
And then these guys came in a little.
These guys went out a little.
And we got this
[INAUDIBLE]
No, that's OK.
I think this is legitimate, this is a legitimate one.
And that bar, every bar now is doing the same kind of thing that this one did.
This moved a little, but this compensated and kept the length the same.
Your question has led us to another nifty mechanism.
It's good, a good one to think about
[INAUDIBLE]
Yeah.
It's movement without stretching.
Movement without stretching is the key, yes.
And then where does collapse come?
So I use the word collapse pretty freely.
So the word collapse comes in because since it's can move without stretching there's nothing controlling it, it can move, well, if we stayed linear, then I could make all those ten and the thing would be even worse, of course.
The truth is that if I made those even one or ten I would be out of the linear range.
These all should be 0.01's or something.
But linear we don't know the difference.
OK, I'm glad you've got that suggestion.
And now somebody correctly says that this one must be some combination of that and what was the first movement?
Oh, the upper two.
Which it must be.
Amazing, how many, put a few bars up there and you got it.
Yeah.
And of course, by the way, and don't let me get too far into this discussion, but who is the artist, actually is it Alexander Calder who has, what are they called?
You know, they're, What was it called again?
[INAUDIBLE]
Well, he has those, that wasn't the word I was thinking of.
But he's created these trusses, like with lots of bars and lots of nodes that have some special property.
You know, they just, anyway.
It touches on art, actually this theory of mechanisms.
Yeah it's really quite interesting.
But then linear algebra somehow tells you just from these numbers how many mechanisms you're looking for.
Which is pretty cool.
OK, open for more questions.
You probably haven't looked ahead, I mean today's lecture was a, I hope, and I left up there the central topics for the lecture.
But I'm, I think, correct that you haven't started on those problems.
To know what to ask.
Let me ask you a question, which was not in the lecture.
Suppose I wanted to use finite differences.
It's a little bit like the one on the quiz.
So on the quiz we had c=1's, and then c=2.
And you knew c had to be four by four, so most people correctly got the diagonal as one, one, two, two.
Just by sort of common sense.
But what would be a finite difference, approximation to our equation?
So suppose I didn't go to finite elements, but instead I stayed with finite differences, which would be completely fine in 1-D, completely sensible.
How would you create a finite difference thing for that?
Let me just bring down a blank board and ask.
So you see the equation?
Let me write it again here.
So I want to replace this equation with a very, that has varying c(x).
Well, I won't worry about the right side.
It's F. What's my K matrix for using finite differences for this?
We're going to create a finite element K matrix by the weak form Galerkin trial function, that route.
That's coming that very important, it's coming.
Start it today and it'll be completed Friday.
But now suppose we were back up to finite differences.
What would you take for finite differences there?
When the c wasn't there, what did we do?
Then we just had second difference, right?
How did we get the second difference?
It was a first difference of a first difference.
I guess I would probably, this, I would probably approximate by u_(i+1)-u_i/delta x.
That would be that, and so what should I put for c(x)?
What would you suggest?
The c subscript i, so let's mark.
Here's i, and here's i+1, and here's i-1.
We're going to end up with those three guys involved.
So this takes this difference, and then this one will take that one, and then I'll also have a u_i-u_(i-1)/delta x.
And somehow that'll be the difference of these differences.
But now tell me again, what would be the really cool choice of c?
What's your instinct where what value of c to take?
As sort of average.
I mean you could take it just halfway.
I think I would take c_(i+1/2) there.
Just as being sort of right.
And here I would take c_(i-1/2), halfway along its interval.
And then the second difference takes the difference of it.
So now I've got, I think that's what I would do.
That would be my one typical finite difference equation.
Would be the difference times its c and I took its c to be symmetric.
You could also have taken, if you like, if you wanted to stay at these points, you could do twice as much work and take, let me say, four.
So either c_(i+1/2), or you could average the c_1 and the c_(i+1/2).
And both of those would give you that extra accuracy that you pick up from sort of keeping symmetry where you should.
So I think, I mean, this would be the quicker one to put, that one would be OK too
[INAUDIBLE]
This was just for your entertainment not, not.
Yes, go ahead
[INAUDIBLE]
Ah, when c was a step yes.
When c was a step
[INAUDIBLE]
c_i plus, I have probably the step.
Yeah, yeah.
Yeah.
Let me ask you, what happens in this equation when c jumps?
It jumped on the quiz.
But I didn't require you to solve the differential equation, only to create the finite difference model, to create the A transpose c A, and most people did it fine.
But suppose c jumps.
I have some simple right hand side like one.
It's not a jump in the right hand side I'm interested in.
It's a jump in c from one to two.
So this is a topic that the book does discuss and maybe I might come back to it in the ordinary lecture.
But, while it's in front of us now, the quiz was sort of intended to help you with that.
How do I interpret this equation when c has a jump?
Well, it's actually, if you look at it right there's no difficulty.
That equation is a combination of this equation -dw/dx equals the one.
And the equation of c*du/dx equalling the w. I split it out for you in the last part, Problem 4b in the quiz.
I really helped you to say OK, take these two separate equations.
And now you don't really have a problem.
The equation for w, nothing is out of the ordinary there.
Jump in c is not even seen.
So then you've got the w. Now, you have the c in it with its little jump.
But, so suppose I find w from the first equation.
How do I find u?
I just divide by the c. It integrates.
Yeah.
What I'm saying, let me say it, clearly now.
If there's a jump in c, that's not a bad thing.
Because the point is there's no jump in c*u'.
c*du/dx doesn't jump.
w doesn't jump.
From a jump in c. If c jumps, let it.
There's no jump in w, which is the serious unknown.
There'll be a jump in du/dx, and that was that e thing that you had on the quiz.
There'll be a jump in du/dx to compensate the jump in c, but w is good.
That's the message of, so let me write that down.
Even if c jumps, w does not.
So my point is when you're looking at w, you're looking at the right quantity.
And it deals, this is the sort of general feature that if you look at it right it's not a problem.
If you look at it wrong and try to write out that equation, take the second derivative of this is OK, but then if you just try to take the derivative of the jump, you think, my God there's a delta function sitting in my equation, what am I to do.
Don't do it.
That pair of equations is no trouble.
Do you see that point?
That a jump in c is really OK. c=0 wouldn't be so good, right?
If c went to zero then you have got some problems here, because if c is zero here what's going on?
So your material should have some stiffness, some positive stiffness.
Zero stiffness would be a problem.
Negative stiffness would be really a crazy material, where you add more force and it compresses on you.
OK so that was a point I could make also in the lecture.
Other questions.
I hope you find these review sessions, they give a chance to bring out more points.
Let me say a little bit about Chapter 2 things that we abandoned without including in the lectures.
One very important thing that we didn't touch was the non linear problems.
Where in the end you have to solve a system of non linear equations.
We've only had linear equations, KU=F, where we almost got to.
And we've certainly found enough to discuss there.
But how do you deal with non linear equations?
Well so that's Section 2.6, I guess.
And the answer is Newton's method, or some version, there are many versions of Newton's method.
So all I'm saying is that's an important topic.
How to deal with non linear equations.
And the answer is usually some variation of Newton's method.
Depending on how many equations you've got, how bad the non-linearity is, and things like that.
It's a case where coding is not so simple.
But important.
So that's one thing in Chapter 2 that we've passed, because really this is where we want to be today.
Other topics, questions of any kind?
Yes, thanks
[INAUDIBLE]
This problem,
[INAUDIBLE]
Yes
[INAUDIBLE] Well, OK.
Right, good question.
You're right
[INAUDIBLE]
Yeah, OK.
So now suppose that delta of x was in delta of x-a, the jump at a was there.
What would change in there?
Well, the one would change, right?
The one would now be the delta of x-a.
So what am I seeing now?
What's the jump situation now?
Is there a jump in, does w jump now from this delta function?
Yes.
It did not jump from the jump in c. That did not produce us a jump in w. Why not?
Because again back to the jump in c, you have a bar of copper and you have a bar of steel.
And they have different c's.
But the force at that point, when it's in equilibrium, the bar's not falling apart, right?
The force is the same from above and below; equilibrium holds.
It's just that the force involves a c_1 there, and down here the force involves a c_2 multiplying the du/dx, and this stretching.
So the stretching rule changes.
du/dx, the stretching factor will be different in the two materials.
But the force won't be.
You really have to keep straight what.
Now, you asked about a delta function, OK.
So now if I hang a delta function there, that's like hanging a point load at a point.
What happens at that point?
Well, we don't have a problem with that.
Well, w jumps.
The force pulling up is not the same as the force pulling down because there's this extra point load.
So the balance there and the balance there, those aren't equal.
There's a jump there because of the point load being put there.
So that produces a jump in w, but not a jump in u.
Not a jump in u, the bar is still holding together.
And since time's running out I don't have to ask myself or ask you about the worst possibility that occurs to me, which is suppose they happen at the same place.
Suppose there's a jump in c and a point load.
Do we want to face that, what would happen there?
Suppose, a was the place where this jumped.
Suppose the point load was here.
What's up there?
Well, maybe I will able to do that.
OK, so I'm putting the point load at the place where the copper and the steel meet.
Is w continuous?
Is the force the same above and below that joint?
No.
Because there's these extra terms from the weight.
And then once you know the forces, then above the bar you're solving for u with one c, and then below that point with the other c. So w has a plus and a minus and c has a plus and a minus.
And you're solving this one, let me make them minus plus.
So you're solving this one in one bar, one metal.
And c plus and the w plus in the other method.
So yeah, you can do it.
Shall I just repeat what my overall message is?
w is the right thing to look at.
That combination c*du/dx is the right thing to look at.
And it's what sits there.
In those parentheses.
OK, good, so that's some review topics.
We really have a lot of fun ahead now with finite elements.
provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Okay, shall we start on this review session?
And I'm figuring that the main topic would be finite elements, because the last review session focused on the truss problems.
One note about codes.
So probably you've looked at the CSE page, where some codes that we had earlier are put there, section by section.
Now people are contributing a truss code, so I don't see any reason why that shouldn't eventually go onto the CSE page.
If you have other codes, like finite element codes, for specific problems, think about, if you get them with some comments, if they're appropriate to go on the CSE page, that's certainly a possibility.
And the homework solutions, which are now on the 18.085 page, will migrate to the CSE page, for their permanent status.
So that's just to say, this is a page that's growing, and any help is very, very welcome.
So, I'm ready for questions about finite elements, maybe specifically, or other things, too.
Thanks.
Yeah, lots to say about finite elements.
Yeah?
[UNINTELLIGIBLE]
So this is finite elements for heat transfer.
Okay, which course was that?
[UNINTELLIGIBLE]
Okay.
Right, free or fixed boundary conditions.
Okay, I see.
So the question is, how do we handle boundary conditions which are not something equal zero, but something equals a constant, or something is given.
Right.
Good point.
So, somehow when we're given numbers, they're going to input, source terms, they're going to show up on the right hand side.
Right?
Eventually, they'll be part of the right side vector F, in some appropriate way.
So that we'll be solving KU=F, where U will represent the unknown displacements, whatever they are in the problem.
So let's see, what do I say about getting things correctly onto the right hand side?
My general thought would be, so we create a matrix that I might call K_0.
That's a matrix where we have not knocked out any rows or columns.
It's probably, almost surely singular.
it's got all possible unknowns, even those that are not actually unknown, associated with the columns of that matrix.
So we start with that.
Start with that.
Okay.
Now, suppose let's take things in order then.
Suppose U_7 is set to zero.
So what will that do to our matrix K?
How do we, having created this matrix K, let's suppose we've got it, how do we include in it the boundary condition U_7=0?
Okay, well that column of the matrix, the column number seven, I guess, or the column corresponding to that unknown, is going to go.
Is that right?
Yes.
That's what we've done, we've struck out a column.
And we should also, I guess, strike out a row.
We would really like to end up with symmetric, and normally positive definite, matrices.
So this would be a column row -- I'm speaking roughly, because I didn't have advanced preparation on this question.
So column row removed.
Ok, now suppose U_7 has some value, A.
How do we account for that?
So what do we do?
U_7 isn't still unknown, so we want to get that column out of our matrix.
But something has to happen to account for the A.
So what is that?
So my picture is that in my original K_0, which had my original K_0, and then all the U's equal all the F's.
So this had a column for everybody, including U_7.
So there was the other U's, including U_7, and these F's maybe came from body forces, stuff inside the interval.
Now here comes perhaps a boundary condition.
Oh, I wrote it in the middle anyway.
So what happens when I set U_7 to A?
So I think that becomes, I strike out that unknown, strike out that column, strike out that row, I think.
But what should happen?
How does the a come in?
Well that a multiplied that column, right?
That a multiplied -- this equation expressed equilibrium.
Those equations were all valid.
You could say we haven't imposed the boundary conditions yet.
Now when we do, so that a multiplies that column.
So what happens now, what do you think should I do?
It seems to me that I should bring a times that column over to the other side, with a minus, of course, because I put it on the other side of the equation.
So I would think I do this a times column seven.
I think.
I think that would seem right.
Now you're going to ask about -- you unfortunately did ask about -- a free condition.
Say U'.
So what am I going to do?
I'd better think here.
What's cooking now?
So I could prescribe U', let me say one equal zero, let's just think about that.
What happens with that?
Hm.
So let's see.
Well, so U' was not one of our u's, so this isn't just simply, OK assign a value, zero or A, to AU.
U' being zero.
Well that came in as a natural boundary condition, right?
If I remember my own lectures.
So that U'=0 came in as a natural boundary condition.
And what did that mean?
That meant that I don't have to impose it.
The finite element method should deal with that.
This is really our w. Our dual unknown, w. It's better to think of it as w. Maybe cU' is what it is, but it's good to think of it as w. Okay, so I think of that as something we don't have to impose.
In other words, the way I'm thinking now, I don't have to think about that.
I've got trial functions whose combinations will get me close to the correct answer.
And the equations will naturally do that.
Okay, now you're asking this serious question?
What about w=A?
What about that?
Or B.
Let me make it something different.
Okay.
Let me say right off, without having prepared a good answer to this question, I'm not going to do it as well as I should.
So maybe what my real answer should be, let me think through the best answer and do it another time.
It's going to contribute some term, some new term.
When we integrated by parts to get to the weak form, this guy just showed up.
And up to now, we've just assume that B was zero, and we didn't worry.
But somewhere in that integration by parts, a term is going to appear that I can't just throw away.
Yeah.
So let me think about that more properly.
I mean that's a very good question, and that lectures didn't allow for that.
Okay, if I can put that one off and give you a good answer.
Because it deserves it.
Okay, other questions.
Other discussion.
Yeah, you have one?
I have maybe a silly question
You have a silly question?
Good.
Okay
[UNINTELLIGIBLE]
Compare finite elements with finite differences.
Okay, yeah.
That's not a silly question.
So finite differences is certainly associated with values at a mesh.
Here would be one way to think of it.
So the question is, finite elements, finite element method versus finite difference method.
And of course, there are other methods, too.
Okay, I could think about finite differences.
One way of seeing them together is, I could think about finite differences this way.
My test functions, you remember, which I always integrate against.
Instead of taking the test functions to be the trial functions, which is giving us our standard finite element, K, I could take the test functions to be delta functions.
What would happen if I take the-- so again just think about that.
Suppose my solution, U, is a combination of the phis, as always.
As before.
But my test functions, V, are delta functions at mesh points.
Because up to now, we've always taking the V's to be the same as the phis.
So the V_i, the test functions, and I'll want n of them.
I'll have n mesh points.
My n mesh points, those will give me n delta functions, n test functions.
And what happens then, when I look at the integral?
The finite element deal says, take the weak form.
Well, OK. And you'll see why, I'm going to take the weak form before I take the integration by parts.
Why is that?
Because when I integrated by parts, do you remember what the effect was?
The effect was to take a derivative off of U and throw it onto the test function, V. But now that I'm choosing delta functions for the V's, it's too risky.
I don't want a derivative of a delta function.
I better keep it all over here.
So I'll have -cU'', times V. This is what I had originally.
I just took the equation, you remember how we got these equations?
I just took the strong form, multiplied by V, integrated, and now I usually integrated by parts, but I'm backing off of that.
Because my V is too singular to do it.
So again, what would I do?
My U would be a combination, as always, of these.
And what will I get for these integrals?
What will I get for those integrals?
Well, of course, if that's a delta function at a mesh point, when I integrate something against a delta function, I get -- what do I get when I integrate against a delta function?
If V_i is the delta function at a particular mesh point, i, as always I get the value of that at i.
At i.
So I'll get -cU'' will be whatever it is.
It will be the sum of U_i's, the unknown constant, times the phi_i''.
And what will I get on the right hand side?
When I integrate f times V_i?
I'll get f at that mesh point.
f_i.
In other words, I think what I've got there is essentially a difference equation.
I'm back pretty much to finite differences.
I'm taking the value f, not integrated now, but just it's value at i, because that's what I get when I do a delta function.
And I'm taking this.
Notice, a lot of things are different here.
And there's a name for this.
This is called collocation.
I guess what I'm saying is collocation, this idea, is somewhere in between.
I'm interpolating collocation between the finite element, the Galerkin principle, and finite differences, where I don't even think about trial functions.
In between is something that really looks like -- [SOFT MUSIC] [LAUGHTER] I don't know if that wonderful music comes through on the OpenCourseWare tape, but anyway.
This is MIT.
We get a brass band here.
Okay, that would be a very finite difference looking set of equations to approximate the original differential equation.
The point I want to make, this gives me a chance to make a point here.
Could I use hat functions?
Could I use hat functions in this as my trial functions, if I was not going to integrate by parts?
When are hat functions allowed?
Because hat functions have a jump in the slope.
Right?
Hat functions have a jump in the slope.
Then my answer, you can see it coming, is no.
Hat functions, they don't have good second derivatives.
So I wanted to make that point, also, for beam bending.
And let me make it.
So your question is leading me to another topic that I wanted to mention.
That hat functions, phi' is OK. Phi double prime, second derivatives of hat functions, are basically, let me just say it in a word, not OK.
So this is OK, this is for second order equations.
Well, second order equations have a second derivative.
Looking bad.
But when I integrate by parts after integration by parts.
And that's exactly how we've used them.
We've used them for our second order equation.
We integrated by parts.
We had terms like cU', c*phi' times V'.
First derivatives, no problem.
But not OK for collocation, this method that I've just spoken about.
Because we're not integrating by parts.
Our test function, V, the delta was not smooth enough to move a derivative over there.
And, now my additional not OK, for bending.
So this is just bringing us into today's lecture, but maybe it completes a point.
So in bending we did integrate by parts.
So what did we get?
Our bending weak form, our fourth order equation weak form, say U fourth derivative equal x.
When I integrated by parts twice, you remember, I had U fourth time a V, times a test function, dx.
And my idea, so really we're seeing it here for the first time, was to do two integrations by parts, get two derivatives off of U, put them onto V. And maybe there's a c(x) in here, too.
This was our integral to be done.
That's the basic integral that leads to all the entries of K. But the trouble is -- so that's in the bending problem.
In the fourth order problem, our integrals are going to look like that.
And we have to use functions that are good enough.
So if I use hat functions, they're not good enough.
Why's that?
If you suppose U and V are the same, then I'm integrating the second derivative squared.
And what is the second derivative, what's its square, and what's the integral?
This is really worth recognizing.
So suppose phi is the same as V, and it's the hat function.
And if I want to integrate some constant, or some nice function times phi'', times V'', what do I get?
What's the second derivative?
What's the first derivative of my hat function?
It's up, right?
It's the first derivative.
So let me do phi'*V', is, let me draw its graph.
The first derivative is zero up to there, then the first derivative jumps up to something high, up to the middle.
Then it drops to the negative.
And then it comes back to zero.
Right?
That's my first derivative.
I'm OK integrating that.
I'm OK squaring it and integrating.
But the second derivative, if I graph phi'', which will be the same as V'', what's the second derivative of that function?
It's zero, and then what is it at this point?
The derivative now?
Delta, right?
Spikes up.
Then nothing until this point.
At that point what happens?
Spikes down twice as far.
Really way down.
And then finally there's a spike up.
Okay.
I can integrate spikes times nice functions.
What I can't integrate is a spike times a spike.
So the answer for this integral is what?
It's infinite.
The integral is no good.
If I have a spike times a spike, you see -- delta functions, I'm OK with a nice function, f(x) times a delta, is OK.
But not OK for the integral of delta of x times delta of x.
That's not OK. That one is infinite.
That one is not in our space of functions that we can integrate.
OK.
So this is the difficulty you meet with higher order problems, is that your functions, some of the old functions that worked fine, are no longer OK.
I need two derivatives.
Or I need to be able to integrate the second derivative squared.
It can have a jump, but it can't have a spike.
So what functions have we chosen?
How would we deal with bending problems?
And I guess that'll probably come, maybe quickly, into Friday's lecture.
How would I do finite elements for bending?
I'm not allowed to use hats, so what will I use?
Do you remember that construction, Monday?
Those cubics.
Remember those cubics, those C^1 cubics would be OK. Why's that?
Because they have a continuous derivative.
They're one degree smoother than hat functions.
These were C^0 hat functions.
And they were still C^0 when I put bubbles in there.
I had to get up to that extra degree of smoothness, this continuous slope.
Then I can take the second derivative.
The second derivative may have a jump.
Jumps are OK. A jump in the second derivative there and a jumped there, I can multiply two jump functions and integrate.
So those C^1 cubics that we constructed, you remember at every node I had two unknowns, the height and the slope.
So there were two cubics for each mesh point.
But the key idea was that they had this extra continuity.
And let me just say, looking ahead, these splines that are also coming Friday have one more degree of freedom.
I'm sorry, one more, there's C^2.
So splines, cubic splines -- I'll just put it here because it fits -- C^2 for cubic splines, that's coming Monday.
That's functions that have two continuous derivatives, and the jump is in the third derivative.
So those could be used, well, they could be used for sixth order problems.
Of course we're never going to see a sixth order problem.
I hope.
But they could be used there.
Or they could be used in collocation for fourth order -- well, could they be used for fourth order problems?
I'm not sure.
Anyway, the more smoothness, as I go up in the derivatives, I have to get more smoothness in all the finite element constructions.
And it gets a lot harder.
Actually, I can do it in 1-D, but it gets seriously hard in two dimensions to figure out function surfaces.
This is what CAD is all about, actually.
Is figuring out, how do you get smooth surfaces?
The whole CAD CAM world of NURBS and piecewise, irrational things.
It's a very complicated construction, to get the shape of some engineering piece approximated, including its curved boundary, by smooth piecewise polynomials, or piecewise something.
That whole world of CAD is not trivial, to do that.
Well, OK, that's my speech.
You got me started with your question.
Did I answer your question in some way?
[UNINTELLIGIBLE]
It went beyond your question.
Yes it did.
That's right.
These are things that I may not have time to develop in Friday's lecture, because I really have to get on to two dimensions, and gradients and divergences, all that good stuff has to start Monday.
So we'll see something about splines Friday.
But the whole world of different elements is sort of interesting.
Actually, can I say one more thing about the elements that we've already constructed.
I want to say one more point that occurs to me.
And then I really will stop and answer questions.
This extra point occurred to me.
You remember our example where we had hat functions, and then we also had bubble functions.
Right?
So where did the bubble go?
These were only C^0, hats plus bubbles, plus parabola bubbles, quadratic bubbles.
Parabola bubble functions.
So how did those go?
Between two mesh points -- sorry, I'd better leave myself more space for that bubble.
OK, so one of these guys goes up, down.
One of these guys, another hat goes up and down.
And then between two I threw in a bubble function, like so.
Right?
So those were all in my list of trial functions and test functions.
Two kinds, hats and bubbles.
Now, that's all fine.
And we can do these integrals.
But one point I sort of went past.
So let's call this phi left, phi bubble, and phi right.
OK.
So my trial functions, my approximate solution, U, is going to be some amount of that left hat, some amount of the bubble, some amount of the right hat.
And more functions on other intervals.
Now, my point is just a simple one.
When we only had hats, then U and the coefficient of this hat was exactly the value of U at that mesh point.
Why was that?
Because all the other hats were zero there.
So when I only had hats, these U's, U left and U right, were exactly the values of my approximation at the nodes.
that's why it looked so much like finite differences.
What I want to ask you is, so if I take the halfway point as the sort of mesh point for the bubble, what is the value of U of my approximation, at that halfway point?
I just want to plug in halfway point, evaluate everything at the halfway point.
So I want to find U at this halfway point.
Right there.
Do you see what's happening?
U bubble, what is U bubble, or phi bubble at the halfway point?
One.
The bubble goes up to one at the halfway point.
So the coefficient that multiplies it multiplies one.
And you think, ah.
You might think, all right, that's the value of my approximation at the halfway point.
But that's wrong, correct?
What is the value of my approximation at the halfway point, here?
Well, this guy is not zero at the halfway point.
This one is not zero.
This should have been R, sorry.
These hats are not zero there.
My basis is not quite as beautiful.
It's not quite as beautiful, because my trial functions are not one trial function per mesh point, including halfway points, as before.
No problem.
I can do all the integrals, I can get the values, I can solve KU=f to find the weights.
And then I just have to remember that my basis is not as perfect as before, because at that mesh point, three functions are contributing.
Probably I'm making this sound like a big deal.
It isn't a big deal, but you have to remember that that's the case, that that coefficient of the bubble is not your solution at that point.
You have to remember that the left hat and the right hat are also contributing at that point.
Or I could rethink my function, and make them.
Now that's the cool thing.
So that's the idea.
Just so you see that I could do that.
Here is my left point, here's my halfway point, and here's my right point.
So I'm going to have the same combinations, which are the piecewise parabolas that are continuous at the nodes.
Let me show it to you.
I could change from hats and bubbles to, I'll keep the bubble, because that's one at the halfway point.
That's good.
But I have to fix those hat functions.
If I want the hat functions to be focused -- if I what the phi left function, I call that a hat function, but it's going to change shape, it won't be a conical hat.
If I want a phi function, I want it to be one at that point, its own home point, but what else do I want?
I want it to be zero there.
I don't want it to contribute there, if I want this perfection.
Or there.
So you see my hat, instead of coming down like that and being 1/2 at that point, what shall I do?
I replace that hat by some combination of the hat and the bubble to make a parabola that does that.
And of course, this other way will be the same.
So that's not a hat anymore.
What name should I give it?
Anybody suggest a name?
It's sort of a hat.
Sombrero.
Sombrero.
[LAUGHTER] OK, so I now have sombrero functions, right?
OK. Good.
And there'll be a sombrero function, instead of the hat, it was replaced by a better hat.
Now what about the one here?
Just tell me, how do I draw that one?
What's that function?
I'll do it with dashed lines, it's zero -- it should be zero at its nodes, at the nodes that are not its nodes.
So you see each one is going to come down and back up.
Really, I'm just saying you have the freedom to do this.
By using combinations of those functions, I get exactly the same as combinations of these functions.
The only difference is that with these new sombrero functions-- so now I have a U left sombrero times a phi left sombrero, and a U bubble times its own bubble, I didn't change.
And a U right sombrero times its phi right sombrero.
And the only point is that, at that halfway point, what does this equal at the halfway point?
It equals that.
Right.
I've just done a little change of variables.
I've changed the function, so that would change its coefficients, but it's still the same space of test functions.
I didn't change that.
My space of test functions, these are all still piecewise parabolas.
They always were piecewise parabolas, but some of them were straight parabolas.
Now they're more interesting parabolas.
OK, that may be a point you didn't think about, anyway.
But it shows you that in constructing finite elements, it's really what combinations you get, more than what individual hats or not, or sombreros you might use.
In other words, you could use either of those.
If you use the hat functions, then you could build on the easy integrals.
Those integrals would be a little trickier, not seriously.
And then the solution would be like a finite difference thing.
Every U would link to a mesh point.
OK.
I'll leave that and hope you guys have some questions about homework.
Thanks
[UNINTELLIGIBLE]
Oh, a question about this?
[UNINTELLIGIBLE]
Yeah, when I say better accuracy, when I put the hats and the bubbles together, the improvement would be noticed in between.
Yeah
[UNINTELLIGIBLE]
These guys, you mean?
Well yeah.
I didn't change a thing.
The answer would still be the same.
My capital U that would come out would be no different.
It's just I've chosen a different basis, you could say, to represent it.
But there wouldn't be any difference, and it would still be -- so what would be the accuracy now?
This is like a good review.
How close will U be to the true solution?
Well, I've got up to quadratic, second degree, so I would expect h cubed error.
Because that's the term I'm missing.
And that's the term that the cubics would get.
Right.
Yes?
[UNINTELLIGIBLE]
The question is, why did I pick -- so I had my hats.
I guess essentially, I wanted to throw in some additional functions, the bubbles, that would up the accuracy without serious increase in work.
And the natural construction was that one.
Yeah.
Then I got piecewise parabolas.
You look for a construction of finite elements that kind of comes out neatly, and this one did.
There was another question
[UNINTELLIGIBLE]
Oh, these parabolas would produce.
That's a good question.
Yeah, these parabolas would produce a delta function right there.
You see, they have a jump in slope.
Even if they're parabolas, they don't have smooth slope.
So this could not be used for a bending problem.
Or let me say it a little differently, because people commit crimes.
I call them variational crimes.
People have tried.
You know, it's a heck of a lot of work to construct smoother trial functions.
So people say, well, just ignore that delta function.
Put it in anyway, integrate elsewhere, and just skip the delta function.
So that would be called a nonconforming function.
There's a very famous way to create nonconforming function.
And that's very hot stuff, actually.
Professor Peraire and others in Aero are world experts on that.
Those are called discontinuous Galerkin.
Discontinuous Galerkin.
This is fun.
You see what this world is about.
So what's discontinuous Galerkin?
Those will be ones where there's a jump in the function.
Here I don't need much space to show you discontinuous Galerkin.
So there's a mesh point, there's a mesh point.
Discontinuous Galerkin might do this.
Oh no, not there.
That's continuous, sorry.
Discontinuous Galerkin would use half hats inside the interval.
It would use that one.
And then another one it would use would be one that jumped this way.
In other words, it sticks within an interval, it uses two functions instead of one.
There are two unknowns.
Do you see what's happening now?
My space of trial functions is going to have jumps.
Up to now, all my good hat, my ordinary hat functions were all continuous.
They were all C^0.
Now I'm dropping back to C^-1.
Somehow my functions have jumps.
So this is called DG, discontinuous Galerkin.
And how could it be justified?
I mean, right?
These are functions -- now I've got a function that's not allowed in my second order problem.
Right?
If I looked at my list here, what was OK and what was not OK, if I look at interior half hats, if I look at functions that have a jump, jump functions.
Shall I say, jump functions.
Those would be C^-1, jump functions.
And phi' is not ok.
So I won't tell you, and I wouldn't be able to do it well, how do they rescue that?
However Professor Peraire and others, others, Professor Darmofal, a whole group in aero, computing with functions that have jumps.
And the way they manage to do it is, they have to add some penalty term that penalizes the jumps, that controls the jumps.
If they just took the standard set up, as we have, and tried their functions, it would fail completely.
So they adjust the energy to penalize jumps.
Anyway, somebody might sometime end up working on some project with them, and d g would be very possible.
So you didn't ask me about any homework problems.
Last chance.
Anything to ask about?
You got a lecture about crazy stuff today.
Sorry about that, in a way, but I hope it was all right.
So homework is now posted, with a MATLAB question using the elements that we've talked about and some questions from the text.
And don't forget, then, my original comment that if we end up with some good codes for CSE that could be used by other people, I'm very grateful to get them.
OK. Let's end today a little early, and I'll see you Friday for splines.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Push
So I'm ready for anything, hope I am.
Questions about any topic.
Yes
[INAUDIBLE]
I feel this is like a White House press conference.
I think there's always somebody in the front row who gets to ask the first question, and then gets to say thank you Mr. President at the end, and then I'm off.
Yes.
I'm tempted by the way to ask you, all are you going to vote next Tuesday and of course I'd like to know who you vote for, and I'd like to give you my advice.
But I don't know that that's proper.
If anybody wants advice, they can email.
But please vote.
Please vote.
Yeah.
Alright, question here and then we'll have, well, yeah
[INAUDIBLE]
Oh, OK, so those were just posted.
Like, I see.
3.3 number two
[INAUDIBLE]
OK, so this was a case, yeah, right.
Oh, OK. We could be wrong so this is 3.3 number two, asks you about the flow field.
Which has no flow in the x direction, the velocity in the x direction is zero.
The velocity in the y direction is x. OK, so suppose we just take that as a full field and try to understand, is it a gradient.
I mean, so what are the questions I would ask?
Is it a gradient of anything?
Because we're now thinking v and w are pretty much the same guy.
So is it a gradient, yes or no?
If so, what's the potential?
Is it divergence free, yes or no?
If it is, what's the stream function?
And of course if the answer to both tests was yes, then we would be talking about Laplace's equation.
I suspect for this example the answer at least to one of the two questions will be no.
So we won't have the two pieces coming together into Laplace.
OK, so first of all.
Is it a gradient.
What's the test for, so my two questions are, is v the gradient of some u?
And what's the test for that?
You remember if, is the gradient, and see if I can remember myself.
If it is a gradient, then this is du/dx, and this is du/dy, and the condition that v_1 and v_2 would have to satisfy is that the y derivative of that would have to equal the x derivative of that, because on the right-hand side they are the same.
u_xy is the same as u_yx.
So I would look at at, so let me write that again.
I need the dv_1/dy to equal dv_2/dx, and is that true in this example?
What's dv_1/dy?
Zero.
What's dv_2/dx?
One.
So the answer's no.
OK.
So test, failed.
Alright, the second question is does it possibly sit over in the divergence free world?
Is the divergence of, now I'll call it w, equal zero?
So the answer was no to that question but now I think the answer to this question will be yes.
Because what's the divergence of this thing?
It's the x derivative of that, which is certainly zero, plus the y derivative of that, which is certainly zero.
So the answer is yes.
So there's no potential but there is a stream function, right?
Because the stream function comes in with this test.
So let's remember what, just from today's lecture, what was the stream function from dw_1/dx+dw_2/dy=0, that'll be satisfied if w_1 is the y derivative of the stream function, and w_2 is minus the x derivative.
Because then this matches the x derivative of this plus the y derivative of this, which is the divergence on the right-hand side I would get zero.
So there's got to be an S, and what is it?
Probably not hard to find.
Let's see.
Here w_1 is zero, so that tells me s doesn't depend on y at all.
w_2 is x, so x is supposed to be minus the x derivative of the stream function, so what is the stream function now?
Have I got room to put it here?
Just about.
What will work?
So again, here's w_1.
The y derivative of S is zero.
w_2 tells me that the x derivative of S is minus x.
So all I'm looking for is a function that only depends on x, has no dependence on y, and its derivative should be minus x.
So what's the function?
Minus 1/2 of x squared.
Yeah, so this gives me S equal minus a half of x squared.
Alright, so you're saying that, so there is a stream function, right.
And what does that travel along?
That travels along steam, that means that the flow buzzes along stream lines.
And what are the stream lines?
They're the lines where s is constant.
Equipotentials were the lines where u is constant, but here we don't have a u in this problem.
Stream lines are lines where the S is constant, so minus 1/2 x squared is constant, what's the picture look like?
Picture then, for that, well, and you know what the flow is doing at a typical point here.
Say, x=3, y=1.
Let me draw the little arrow.
With a big chalk Which way is the flow going?
Well, the x component is zero, the y component is x.
So the flow is going up there, right?
Here are the y components x.
This whole line is all traveling up with the same velocity.
If I drop a leaf there, it buzzes up that straight line.
So that's the stream line.
And its velocity is, that's x equal; if, say, the velocity is three then this speed is three.
It's going up that line.
So that's a stream line.
And sure enough, on that line minus 1/2 x squared is a constant.
So you see we're not talking parabolas here because our curve is not y equals minus 1/2 x squared, it's minus 1/2 x squared equal constant.
Yeah, that's what we want.
So, but now having got so far, let me take x=1, say.
What's the flow like on that?
So there's a stream line with S equal constant.
And the velocity on that is zero, so nothing is going in that horizontally.
And now it's one, so the flow is slower up this line.
OK, slower flow.
This was faster flow.
And then the question that's in that homework problem is, is there any rotation in this flow?
We think about rotation, we have an image of rotational flow.
And that could be the next example, we could figure out.
A flow that goes around in circles, right?
Those would be the stream lines.
So this would be like pure rotation, shall I call it.
But I don't have that there.
I just want to draw the other picture, in which the stream lines are circles.
To have another nice, clean, beautiful example.
OK, but here we don't have.
Our streamlines are straight lines, and yet we have rotation.
That's the point here.
Why do I say we have rotation?
Because the test for rotation was that original test of looking at, which I just wrote the answer to be no here.
So if it's not the gradient, the reason is there's some rotation.
Gradient fields don't have any rotation.
The rotation is this thing that comes out, yeah it's this difference.
dv_2/dx, it's the difference between those that tells us the rotation.
And that was not zero, right?
For this example dv_1/dy, was zero, because v_1 is zero.
dv_2/dx was one, because v_2 is x.
So there's some rotation here.
And in other words the test for being a gradient is no rotation.
This fails that test.
But how is it rotating?
How can it be rotating when the all the flow is just traveling vertically?
I guess I give you this example because it meant something to me.
My image of rotation was this simpleminded type of flow.
You know, like a phonograph record or something.
This would be called a sheer flow.
A very important type of flow.
And actually, you'll realize that if x is negative then the flow in the second component, the velocity, is now negative.
So it would be the stream line, the flow would be going down this way.
And this point wouldn't move at all.
This would be, well I don't know if it's a stream line, it's a stagnant stream line, right?
x=0.
On that line, there's no velocity.
.
So this is all just staying there.
These lines are moving, this line moving faster, this line would be moving even faster.
This line's going the other way.
Faster and faster the other way.
It's a important flow.
You know, in earthquakes and things like that.
This happens, when one plate shears with respect to another.
So that's shearing.
Shearing means that a line that was, that line after a while is tilted.
This is going faster than this.
Yes
[INAUDIBLE]
Right, being a constant
[INAUDIBLE] That's true.
Ah, well, OK. Let's see.
Well, how do I fix that?
[INAUDIBLE]
Yes.
That's a good question.
Should I have allowed in my stream function, I mean that's a stream function.
Because it satisfies the equations that stream functions are - I could have thrown in a constant, yeah.
So your pointing out a difficulty makes me think I should have thrown in a constant.
So if I throw in constants when I could get other lines, yeah.
Thanks, that's a good point.
I just want to see, do you see rotation in this flow, in this shear flow?
And I think you do.
If you think about it.
Suppose you put a little leaf, or a little penny or something right there.
OK, is it going to turn?
It'll flow along, but as it flows, is it going to turn?
In other words, is there some is there some difference in the speed on one side compared to the other?
I mean, it's what makes a curveball curve, right?
When the pitcher throws the ball, he imparts a spin to it, and that gives a different pressure on the two sides of the ball, and the ball moves.
I think that's going to happen here.
Maybe you see it, and I'm just talking.
I mean, this side is going faster than this side.
So the net result is that even though the thing is traveling up and down, it's turning.
It's turning because the right-hand side is going faster than the left-hand side.
So it does have a rotation.
This quantity, this difference between dv_1/dy and dv_2/dx, which is the component of the curl, maybe the sign should be the opposite, maybe I think it should be minus this plus this or something.
Point is that it's not zero.
So there is curl, there is rotation.
OK.
I was going to ask about this picture, too.
And then I'll open to more examples.
I just feel examples are good.
Simple velocity fields, like 0x.
Just to think through, OK, what does that mean?
Is it curl free?
Another way of saying is it a gradient field would be to say is it curl free?
Irrotational is the right word here.
Test one, is it irrotational, answer no.
Is it divergence free, is it source free, the answer was yes, for this example.
If we pick another example I could reverse those, or another example I can probably come up with an example here.
Let's see, what if I wanted the stream lines to be circles, what would be a good velocity field that goes in circles?
Let's see.
At a typical point, if I want the velocity to be going that way, here's the vector, the position vector, the radial victor that goes, so what are the components of this vector?
Just, .
So now if I want the velocity field to go other way, what would be a good thing with rotation?
would sound good.
v=.
So are we expecting this to be a gradient of anything?
I'm not.
We've built in rotation here.
I'm expecting the curl of this thing, this quantity, I think I take the x derivative, I look at the y derivative of this and compare it with the x derivative of that.
And they're not the same; in fact one is minus one and the other's plus one.
So I've got rotation here.
I've got sort of two, is the component of the curl.
So let's just write it down.
dv_2/dx, this vorticity that measures the turning speed is one from dv_2/dx, minus one is two.
So it's not a gradient of anything.
If the x derivative of u is minus y, then the y derivative can't be plus x, no good.
OK, what about, is it divergence free?
Do I need a source to keep this flow going?
Well, what's the test?
In other words, is there a stream function for this guy?
I think probably there is.
What's the test to know if there is a stream function?
I take the divergence, I'm over on the right-hand side of my picture now.
I take the divergence.
Divergence of this v is the x derivative of that plus the y derivative of that, good, zero.
So there is a stream function.
And what is it?
Well, I'm pretty sure that these stream lines are circles, I think the stream function is going to be x squared plus y squared.
Yep.
Then, am I right that the y derivative of that will be 2y.
That's not looking too good.
What do I want here?
Here's my v, which is the same as w. And what I'm looking for is to get these guys correct.
So and I should be able to do it.
And what would s be?
I haven't got s quite right.
I think if I multiply by negative 1/2, that might have done it.
Yeah, because now the y derivative is now minus y.
Great.
And the x derivative of S is minus x, and then I should take a minus that, so I should want a plus x, which is what I've got.
So those are the stream lines.
Circles.
So I have circle, the flow is going around in a circle.
I don't have to, I don't need any source to keep it going.
But it's not a gradient.
So this is like a sample test, to take a simple flow field, apply the two tests, and I guess we should complete with an example that passes both tests.
Right?
Let me open to any other question, and then we could cook up an example that passes both tests before we stop.
I'll stop talking first, though.
Just listen for a question on any topic.
Or is it useful just to take fields like this and go through those steps?
It probably is.
It's certainly good for me.
OK, what's a field that will satisfy everybody, that will be a gradient field and also divergence free, so that we'll have solutions to Laplace's equation.
Let's see.
Well we had some solutions to Laplace's equation there.
You know if I make it linear it's real easy.
If I make it quadratic - huh.
Can I anticipate a little what's coming Friday?
I so recommend to come to Friday's lecture, but so what's coming?
What did we do today?
We discovered that we got solutions to Laplace's equation from all, by real and imaginary parts of all these guys.
Those were terrific.
And then we could take combinations of those.
So here's what's coming Friday.
When I take combinations of these guys I get some function of this magic complex, of this magic combination x+iy.
Some function, any function.
Any reasonable function, and we'll say what reasonable means, of x+iy, its real part and its imaginary part are going to be great.
This is like the center of a big, big, part of mathematics.
Functions of x+iy.
And by nice I mean that these series converge.
So that we have really a nice function.
Let me take the first function that comes to mind.
e^(x+iy).
So let me take this to be e^(x+iy).
OK.
Right.
So you remember, I'm aiming to get solutions to Laplace's equation, because that will give me automatically the two pieces both working, so I claim that the real part of that, and the imaginary part, those are my twins, u and S, both solve.
So u is going to be the real part of this function.
And S is going to be the imaginary part of it.
And I claim that those will both solve Laplace's equation.
We can plug it in and see that it does.
And that they will have, they're twinned by the Cauchy-Riemann equations.
So how am I going to simplify that, so that I can identify what's the real part of that thing and what's the imaginary part?
This is actually, that's a good question.
I don't know how much you've run into i, in the past.
Are you happy with something like that?
How could you find the real part of it, how could you simplify it?
How else could I write e to the something?
Exactly.
Think of it as the product of two, so the key fact about exponentials is that that's the same as e^x times e^(iy).
The exponents add, so that's the first thing always to think about as a possibility.
Now, what am I going to do?
I still want to get a real part.
This is clearly all real, right?
e^x is real.
So it's this part that's going to give me the two pieces.
So this is going to be e^x time now, what do I put for e^(iy)?
cos(y), good.
Plus i*sin(y), good.
And now I can read off, no problem.
What is this real part that I was looking for?
e^x*cos(y).
And the imaginary part is just what's multiplying the i, it's the e^x*sin(y).
OK, so what's my claim?
I claim that that function solves Laplace's equation.
And this one too.
And that they're twinned.
And that they give stream lines and equipotentials that meet at right angles, it's another pair.
Plug that into Laplace's equation.
So let me do u_xx+u_yy, just to satisfy that it is going to come out zero.
So what's the xx derivative, the second x derivative of that function?
Take its derivative with respect to x, and then do it again, and what do you have?
Same.
didn't change.
e^x is just, and now what about the second y derivative?
So now e^x is just a constant.
What's the second derivative of cos(y)?
Negative cos(y).
Right.
Because the first derivative is negative sine, the second derivative is negative cosine.
So the second derivative is e^x, it didn't change, times cos(y) with a minus sine.
And you see what, did I write sine?
I meant to write cosine.
Cancel that from the tape.
OK, right.
Yeah.
So the second x derivative was just e^x, e^x*cos(y) didn't move.
Sorry.
That was frightening.
OK, and then now here's the second y derivative.
In other words, it gives zero.
Gives zero, and this one would too.
Now, I don't really have an idea of what the picture is like.
But it's important.
We've got a flow field here, and it's from e^z, e^(x+iy) exponential has gotta be an important function.
So it's got to be somehow interesting.
What do you think, so what would the, what would the equipotential lines looks like?
Oh, boy.
e to the x cos y equal a constant.
My gosh.
e to the x cos y.
So let's see.
I don't know how to draw this picture, but one thing I know is that if I changed y by 2pi, I would get another copy of this curve, right?
If I changed y by every time you see cosine over sine, you think hey, that's periodic.
If I change it by 2pi.
So I'm thinking that y between zero and 2 pi, so here's y=0.
And y=2pi, I'm thinking that my flow probably somehow stays in a strip.
Like that.
And then the whole thing just repeats, and repeats, and repeats.
So I'm thinking really this is flow in an infinite strip.
Infinite height or something like that.
You can imagine that there could be applications.
But I still haven't drawn the curve.
I just think, let's see, what would it look like when y is a little - suppose I'm trying to draw the picture of e^x*cos(y)=1, whatever.
OK, I'll just attempt to draw that curve.
Just, so if y was a little bit off of zero, the cosine would be, yeah, how's it going to go?
If y is just a little off zero, tell me any points on this curve?
I can see that e^x is going to be a big number.
Is (0,0) on the curve?
Good.
Got one point.
Alright.
Now, suppose y is a little bit more than zero.
So suppose y goes up a little bit.
Then what?
(1,0) or something?
Yeah.
I suppose (1,0)?
No, no.
So if y goes up a little, then x would go out a little bit.
So what's happening?
So cos(y), so the cos(y) is going to drop from one to zero, right?
To start with.
Then, if this cos(y) is dropping from one to zero then this e^x has got to climb up, to to keep the product one.
So I'll move out.
So somehow it'll move out.
I I think maybe when y reaches pi/2, then the cosine has got down to zero.
We could work on this for a while.
Or we could let MATLAB draw it.
But I think that we would see these, and I could do better.
I'm feeling pretty humiliated to not have a better picture here.
Suppose y is a little less than zero?
Do we get anything interesting there?
Oh well, the cosine is an even function.
So I think the thing might, is it just going to turn around like that?
So that y and minus y for a certain x, the y value and the minus y will both be on the curve because the cosine doesn't know whether it's the cosine of of a plus or a minus.
Yeah, I think we would get curves of that sort.
And then the other curves, S equal constant, the stream lines will somehow go vertically.
Maybe I'll just not use the whole time to work on that particular curve.
We'd have to prepare it.
The point is, you see how incredibly easily we produce solutions to Laplace's equation that you wouldn't have thought of, and I wouldn't have thought of.
So that would be one way to produce solutions.
I might even repeat this one in class Friday, or I might not.
Let me suggest another couple of possibilities that I will do in class.
Can I just give you a couple of other functions, f. In fact, I'll just erase that one and put in some other ones.
Suppose I took the function 1/(x_iy).
So that's a function of this magic combination, x+iy.
What's its real part and what's its imaginary part?
Do you know how to split that guy into real and imaginary?
There's a little trick, if you remember from learning complex numbers.
Do you remember the trick?
The problem is that this thing is down in the denominator, right?
We don't want it there.
Because we can't split the real and imaginary parts down there.
So I would like to rewrite it in a way that gets something real down in the denominator, moves all the i stuff up in the numerator where I can separate it.
How do I do it?
Multiply both sides by, both top and bottom, by x-iy.
Good.
So what does that put down here now?
That's a number times its conjugate and that's going to produce x squared.
Minus or plus?
Plus y squared, right.
The number times its conjugate is the length squared.
And now we just have x-iy, and now it's obvious what the u is.
This is real now, so the u is just x over x squared plus y squared, and the S is the minus y over x squared plus y squared.
That's a very interesting flow.
That's an interesting flow, and we could do its picture and so on.
And in fact it would be a nicer picture than the one we stopped on.
What should I notice about this flow?
Of course, the flow is automatically irrotational, the curl is zero because there is a potential.
A gradient of a potential, the gradient of a potential is going to be free of rotation.
And there will be stream lines, all those good things.
There's one bad point about the flow, though.
Which is where?
At (0,0).
The whole thing falls apart, at the origin this falls apart.
So this is a great flow except at the origin, it's very problematic.
It's singular at the origin.
So if we drew the pictures we would see you something strange.
This is going to zero at the origin.
So, yeah, we have trouble at the origin but an important flow otherwise, yep.
And I'll just mention the third but I won't do anything with it.
Because it's such a neat one that I have to save it for Friday.
The other natural function to think of is the logarithm.
The logarithm of x+iy.
Split that into u and S. What kind of a thing do we have here?
What kind of singularity?
Yeah, let me just do two moments on this example, and then leave it for Friday because the whole class has to see it.
Is there a singularity for this guy?
Is there a point (x,y) where the logarithm is not great?
At the origin, again.
We'll again have a singularity at the origin.
Something strange is happening at the origin.
And what we'll find is there's a delta function there.
We're feeding in, we have a source right at the origin and then it's flowing out on, I think on radial lines.
I think the stream lines go out from the origin and the equipotentials go around the origin.
Yeah, it's a great example.
So that another one to come.
OK, so examples like these are, I mean generations of thinking went into solutions of Laplace's equation.
And 2-D particularly where we have this special combination.
I wish we had such a combination in 3-D but we simply don't.
We can discuss Laplace's equation in 3-D of course, very important.
But I mean wave equation, this fact that we're talking to each other is got the Laplacian in 3-D but there's no x+iy magic.
OK, that's some u's and s's and v's and w's.
What else is on your mind?
Questions?
I could ask this question, or here's something I did not do in class.
I think I wrote down the divergence theorem.
So can we start by doing that?
Let me write down the divergence theorem, with your help.
And then use it.
So what does the divergence theorem, we're in 2-D.
So this is 2-D, just the similar theorem.
So what does the theorem say, that if I take the divergence of some w, some vector field, then if I integrate that over some region, so I have some region here and at every point there's a flow, w. And I look at the divergence of w and I integrate that, dx/dy, so that's a double integral over a region, I will get, what's the right-hand side?
What is it divergence measure?
So I'm really asking like just memory, what is the divergence, it's an identity.
It's integration by parts in some way, as we'll see.
But what do you remember for the divergence theorem?
You get what?
It measures how much flux out, right?
So when we measure the flux out by integrating around the boundary, how much is getting through the boundary?
And what's the flow through the boundary?
I take w, but that's a vector.
And I'm looking for what component of w?
The normal component, the component of w, w dot n, the component of w that's headed out.
n is defined to be whatever the boundary is, here I've made it look like a circle but I shouldn't have.
Let me make it a little wobblier or something.
So the normal component at any, there, look at that point.
The normal direction through the boundary, down in that crazy point, is this way.
So the normal is going this way here.
Here, it's going over this way.
It's perpendicular to the boundary, OK?
And then we integrate around the boundary.
Alright.
So that's the identity of that, that's the divergence theorem.
Now, let's see.
Could you, yeah.
So we have a minute.
You want to take a particular w and see if this would be correct?
How about w=0x, our first example?
Suppose I tried w=0x.
I just want to see if the divergence, what the flux is through the boundary.
What what region shall I take for the, so the divergence theorem has two inputs.
It has a flow field.
And let me take w to be 0x, just so it's a shear.
And a region.
And of course the integral might not be that much fun to do, unless we make the region nice.
What do you take as a nice region for - actually, it doesn't matter what the region is.
Take any old region.
For the moment.
What's the answer?
For this particular flow, w=0x?
Zero.
That's the cool part.
If the answer's zero then work is suspended.
And why is it zero?
Because the divergence of this particular w, the x derivative of that plus the y derivative of that is zero.
This has divergence.
Everywhere zero.
So integrating is no problem at all, so that would be zero, for this flow field.
For this divergence free field.
Zero for that because div w is zero.
But is that correct?
What does that tell me, these flows are - we saw what the flow is like.
Say there's the origin.
It doesn't have to be a circle, it looks like a circle.
Do you see why the flux is zero?
There is flow through the boundary, right?
Flow is going buzz, buzz, buzz up this line and out.
And it's coming in here.
So there that's what we discovered.
This 0x is vertical flow.
It hasn't gotten any horizontal component.
It's got a vertical component, it's going out.
And would we want to do this right-hand side?
I don't think so, right.
This right-hand side is asking me what, I have to find the normal direction on this whatever curve that is.
I have to take its dot product with the flow 0x, so this is some quantity.
And then I have to do this dS which I haven't even mentioned, dS is arc length around.
I'm integrating around these pieces.
But yet somehow we have some idea from that picture that the total flux is zero.
How would you say it in words, if I say here's the flow field.
There's a region, funny shape.
The flux is zero through that boundary.
And if I asked you why, what would you say?
I mean, a math answer would be use the divergence theorem.
But why from this picture does it look like we have zero flux?
What comes in goes out, yeah.
What's coming in the bottom here is going out the top.
That's basically it.
So we would get zero for that one.
So I think the homework, the suggested homework maybe includes an example where the divergence isn't zero.
And then you actually have to do these integrals.
Just as practice for what do those integrals mean.
Maybe I won't go through one now, but that's good practice.
Take some simple w, but one with a non-zero divergence and then see if you can do either or both of the integrals that are supposed to come out equal.
That's a good one Now, there's one thing I could - any questions, or discussion?
You guys are seeing these examples come up; it's the only way I would know to learn this subject is take simple v's and w's.
And see what you can do with them.
We've got the general principles, but then apply them to specific flows
[INAUDIBLE]
Yes, thanks
[INAUDIBLE]
This theorem?
[INAUDIBLE]
Yeah if it was, well let's draw a funny shape.
See what we think.
I mean, with this flow, right?
Let me just say, if the flow has some difficult divergence and the region is some mess, nobody's going to be able to do it.
I mean, yeah.
So don't think that these can all be done.
The equality, it's like integrals in calculus.
No problem to think of integrations that are just beyond human capacity.
But the formulas still hold.
Suppose my region was even, like this?
Would that still be, do we still see flow in equals flow out for this particular flow?
I think, yeah, the flow's going this way.
So it's coming in here, it's going out again.
That contributes.
Back in again here, and out again here.
Yeah, I think our instinct would be correct there, yeah.
All sorts of examples.
I was going to, well, I'll maybe do it in class.
This divergence theorem is the fundamental theorem of 2-D calculus, you could say.
Or one of them.
And to write these things and see what they lead to, yeah.
I'll tell you what I was going to do.
I was going to apply this to the vector field, u times w. So u is a scalar, w is a vector, and therefore uw is a vector.
It's got two components, uw_1, are you willing to do that one?
So I apply this not to w itself, but to u times w. Which has two components, uw_1 and uw_2.
OK, so I should take the divergence of uw.
I mean, that's a vector field, uw, this guy.
And I'll get the uw dot n. And it just turns out that this is the right way to do it.
To see the fact that gradient and divergence are transposes of each other.
Yeah, yeah.
I maybe I won't do that calculation now, I'll just say that if you take the divergence theorem and you apply it to this guy, , and write out what it means, you get a very interesting formula.
I'll just leave that there.
So I'm ready for a final question if there is one, or otherwise keep going Friday with these Laplace equation solutions.
Play with some vector fields.
That's my best advice.
And I'll see you Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So, shall we start, as always, just open for questions.
About any topic.
I listed again trusses, 1-D finite elements.
Grad div curl, and I should have squeezed in x+iy, too.
As the magic trick for finding solutions to Laplace's equation in 2-D.
So those are all certainly topics that are in this part of the course.
We didn't really get to 3-D, I'm sorry about that.
Where the curl comes in, maybe I can say a few words about curl today.
Anyway, questions.
Discussion.
Yes, thanks
[INAUDIBLE]
Ooh, let's see.
So A^T A for a truss.
That's a good question.
Trusses, A^T A. I guess I don't know any magic tricks either, so 1 way is to construct A, or A transpose, and then just multiply.
A second way to do it would be by the four by four bar element matrices, so go bar by bar.
So four by four bar matrices, four by four.
So those are already in the A transpose A form.
They're little A element, A bar transpose A, jeez, this isn't a great as it should be.
A element, but I don't know if that would be, so and then you pop those into their correct places.
I don't think I know any great idea beyond that.
I think you should really be ready to construct a matrix A, yeah.
For a reasonably small truss, of course.
And of course the other part of trusses, the fun part is to be able to recognize solutions to Au=0.
Possibly by looking at the truss more than by solving Au=0.
Yeah, any particular example?
Of a truss that I should look at just to pin this down?
Any favorite trusses?
There was an exam question, what was it, a complicated truss?
Let's just create a truss.
And just think about it.
Maybe I won't create the whole matrix A.
Here's a truss.
How's that for a truss?
So it's got, and let me put no supports on it.
Just there's a truss to think about.
Probably we won't get to all the gory details but if you look at that truss, what's the shape of A?
Shape of the matrix A. I have a row for every bar.
So one, two, three, four, five.
And how many columns have I got, how many unknown u's, unknown displacements have I got?
Eight, two, four, six, eight.
So I would expect Au=0 would probably have how many independent solutions?
Three.
You don't know the exact rank, that's exactly true.
There could be more than three, right.
So to really pin it down you'd have to be sure you were right about that.
So three, at least.
And I guess here you could tell me the three solutions to Au=0.
Three rigid motions.
I could translate it to the right, I could translate it up, and you would know what the u is, so u translating to the right would be <1, 0, 1, 0, 1, 0, 1, 0>, right?
That's horizontal motion all the same, rigid motion.
And we should certainly discover that if we created A for this truss that Au was zero.
And similarly vertical motion and the third one would be?
Rotation, rotation.
Yeah.
So if I did the rotation around there, for example, this guy would, this u would also be a  here.
This wouldn't move.
So I'm putting in the four pieces that would go into u.
This one, what would be the u for this, the displacement of that corner of the truss?
In a rotation?
So my rotation is just swing this whole thing around.
Zero.
, I think.
Right.
Because it's not going to go out, it's going to go straight down, .
And what do you think this guy is?
, let's see.
It's going to go this way, so it's going to go forward and down.
And I think you're right.
One and negative one, yeah.
I think that would be right.
Yeah, then the truss, we could check each bar.
That bar, for example should not change lanes because the movement is perpendicular to the bar and I'm writing ones, but I really should write some much smaller number like 0.1 everywhere, or something just so that this isn't a very big angle.
And it wouldn't change length to first order.
So that that's maybe an example where we see the motions, but we didn't actually create A, and we should be able to create A, don't let me prevent you from thinking about A. Yeah
[INAUDIBLE] Should those be ones, or should they be root two over twos?
Well, that's a good question and after many years I've figured out that they're ones.
But it's a very good question.
Let's just see why.
Let's look at this bar to be sure it's not stretched.
Right?
So this guy is moving over by one, and this also by one is my claim.
And then this movement down doesn't stretch it to first order, yeah.
So I needed to make those guys the same.
I guess what I figured out is that if you're rotating around here, then somehow it's the x and y, is that.
Anyway.
Whatever.
So that gives us a chance to do a specific example.
OK, but I've dodged the creation of A. Yep, thanks
[INAUDIBLE]
Sorry, the solution to A transpose A?
[INAUDIBLE]
Yeah, I see, OK.
So the reason I stopped here was that A transpose A will be singular.
So I wouldn't, like, go ahead to go forward to A transpose Au=F.
But if I put on some supports, then of course now all good.
So now I have, what's now the shape of A?
For this one.
I now have this bar is now, forget it, right?
This bar is just between two supports.
So if we put it in the matrix it'll just be a row of zeroes.
Nothing will happen, and we're better off to just knock it out.
So I think, now I have, I now have four bars.
And how many unknowns?
Four, two there and two there.
And, do you guess that it's stable?
That truss?
Yeah, that looks stable to me.
So the four by four matrix would be invertable and then I could solve.
Good point.
Then, A would be four by four, A transpose would be four by four, C would be four by four in between.
This is the case that I gave the name for this case.
When I have a square matrix, do you remember the name just for the hell of it?
Statically determinant.
It's determinant because each step determines everything completely.
Normally, if I have another bar there, now it would be five by five, and now I really have to do the A transpose C A to get to a four by four invertable.
By itself, A would not be invertable.
This is the more typical case, where you really have to put all three together.
Right.
I hope you enjoyed the trusses part, though, and continue to enjoy them this evening.
OK, I'll just keep moving to be sure that we cover any other topics.
Yeah, thanks
[INAUDIBLE]
For this matrix?
[INAUDIBLE]
For that truss?
Oh my God.
OK, let me see.
Then can I do one row?
OK, of course, you guys are responsible for much more.
Alright, which row shall I do?
Which bar?
A diagonal bar?
I knew you'd make it like, you could make up quizzes and I could just sit back here.
OK, so let's take this diagonal bar, alright.
And we're going to keep that supported.
Or not, do you want to keep it supported?
OK, so in this case then that end is not moving.
So this will be in this, and this bar corresponds to a row of A.
And how many non-zeroes will I expect in that row?
Just two.
Normally four, but I'm not getting any motion down here.
So it'll just be two and if that's 45 degrees, shall we say, than that row, I think, would be what?
Well, OK. Where do my non-zeroes appear?
This is node number one with an H and a V. So I think we have zeroes.
If that bar stretches, the connection between displacement and stretching of this bar does not involve this guy.
So I think it's zero and zero there.
And now, what else is it?
So now come the real numbers, which I believe to be cosine and sine of that angle.
Because if I, how much does that bar stretch?
I know that I'm looking for a cosine and a sine, and if this goes positively, then the bar does stretch.
If this goes positively, that does stretch the bar so I'm expecting positive numbers there, like the cosine square root of two over two, and the sine square root of two over two.
Well, I dodged the bullet of getting the whole matrix, but maybe that would do it.
Why don't we do this the top one?
Yeah.
Tell me the first, if that's bar one, what would be the first row of the matrix?
OK, it involves both of these nodes.
But the angle is zero.
So that's going to be a little special.
So if this goes out horizontally, I should really start with the first one.
Is this goes horizontally it compresses the bar, I think we get a minus one there.
If it goes vertically, that doesn't do anything.
If this goes horizontally it does do something.
If it goes vertically it doesn't.
I'd say that would be the row of the matrix coming from the top.
That would give me the stretching.
You remember, I'm always going to, I think of multiplying this by u. I think of multiplying that by u 1 H, u 1 V, u 2 H, u 2 V, and this top row should give me you u 2 H minus u 1 H, which is the stretch in bar one.
So that would be a typical one, this would be at least typical of one where I do see a cosine and a sine.
And let me just finally add, suppose this was not supported.
OK, suppose that's not supported, now I've got a couple more columns to squeeze in.
Maybe I can somehow do it here.
Can I squeeze in the two more columns?
So can you complete the top row of the matrix?
Now I've got six columns.
Because here's two, here's two, here's two more.
What goes on the top row of a matrix now?
Zeroes, because this is not affected by bar one.
But it is affected by this bar.
So it's going to show up in this row, and how will it show up.
Two negatives, right.
A negative cosine and a negative sine.
and at 45 degrees I can't tell the difference.
Because if these move forward, that compresses the bar.
So the minus sign.
So again, the rows add up to zero, as we expect when the bar is not touching a support.
This is not touching a support, so it adds up to zero.
OK, we'll have a truss problem this evening, but not a big messy one.
How about finite elements?
You guys, do you like finite elements?
I'm sort of hoping to make them attractive.
I noticed a problem just to give us some specific one to work on, and I don't remember that it was a homework problem.
This is Section 3.1, number 18, asks about the equation u''=0.
Well, we've talked about it in class.
With u(0)=0 but u' of slope equals zero at the other end.
So what's the picture if I use linear elements?
I don't remember how many I used in the problem.
Well, it allows you to use n interior guys, one, two, up to n, and then another.
This will come in.
Or that's the point.
OK, so what's the finite element method, it's finite element matrix K for this.
So I want to do linear elements and I want to construct K. And, yeah, I guess.
Oh, I haven't actually made anything happen to this problem.
All zeroes is kind of slow going.
u will be the solution, will certainly be zero.
So maybe I'd better put in a load here to get some action.
OK, well, yeah.
So I proposed this question but now, is this a question to think about?
I think that's a reasonable example to do.
It's got the two types of boundary conditions.
It's got the right hand side f, it's got linear elements which means it's kind of doable by hand.
And we kind of know what matrix to expect out of it.
What matrix do we expect?
What do I expect out of linear elements, do you remember the point about linear elements on equally spaced meshes?
That just brought back our regular difference matrices.
So I'm expecting this stiffness matrix to be a difference matrix.
Anyway, the point of this question is, OK, I have a hat function, I have a hat function, I've a hat function, a hat function, and is that the end?
Is that the complete list of my trial functions?
One more, right?
Because this condition is, all my trial and test functions don't have to satisfy this.
So I'm allowed, and should have, another guy there.
A half hat for that one.
You may say, don't let that clown into the finite element space but I think it should be.
The solution won't use much of it.
Because the solution is going to aim for zero slope.
But it's going to need a little, but you see why it needs a little bit, something here?
Because this thing has slope down.
So if there's some of that in there, there better be somebody else to cancel it.
If our approximation is going to have about zero slope.
OK, so then.
Can you construct a matrix K?
Let's see, what's the 2, 3 entry so if I call this number one, this number two, oh, I've already numbered.
So number two and number three, so that trial function against that one.
What do I?
What's my formula for the 2, 3 entry of the stiffness matrix?
It's some integral, right?
And what do I integrate?
I integrate, yeah.
And I've got to have to remember.
So I do, yeah, my weak I've integrated by parts.
So my weak form is the integral of u'*v'*dx equals the integral of f*v*dx.
That's my weak form.
I did two integrations by parts and the integrated term will go away.
Because of those zeroes.
OK, so K_2,3 will come from this side when I'm using phi_2 and phi_3, because I'm taking the phis, the phis and the v's both the same hat function.
OK, so what do I get for that?
phi_2' is?
So this is it, and it overlaps this one.
So when it overlaps this phi_2 is coming down and phi_3 is going up.
And the slope is 1/h, let's say.
So I think I'm integrating us a negative slope, is that right?
Times a positive slope.
And I'm really only integrating over one h interval.
The two overlap only here, where this one's coming down and that one's going up.
So I think the x, and the great thing is, of course, we have a constant.
So I have minus one over h squared times one over h, I think minus 1 over h. That would be K_2,3.
That's a simple example.
And then at the end we will see it, we'll see this one, I think we'll get some matrix.
We'll have this 1/h outside, I think we'll have something like two minus one, two minus one, minus one, minus one and then only a one from the hat factor.
I think it would be that matrix that would be K. I think.
Maybe with more, greater size if we have a whole bunch of elements.
But that pattern.
You're pretty much into this?
Yeah, I mean we're doing a lot in this course.
I'm really grateful you guys stay with it, and kept to these new ideas, through doing exercises and so on.
Because there's a lot here.
Well, I thought I'd put an example up, to open up, just to remind you what that language is about there.
And to be ready for any question in that topic.
Or any question whatever.
So I jumped in with finite elements, but I'm ready also to talk about that area of the course
[INAUDIBLE]
Yeah.
x+iy stuff, OK. Basically, any function of x+iy, yeah.
Any function.
So strictly, yeah, I mean a mathematician would say what, any function?
That's, you've open the door to crazy things saying that.
So what I really mean is, we have these powers of x+iy, and then we have combinations of them.
So the only requirement would be that if I want to take an infinite combination it should, the series should, add up to something.
If it has a nice Taylor series then those are the best functions there are.
Functions with nice Taylor series'.
I'll just say it.
Having used those words.
Suppose I take that function.
There's a function, that's a function z is x+iy.
But z is shorter to write.
So it's not a polynomial, obviously.
But it is a function of x+iy.
Well, tell me this.
Where does that function go wrong?
So either the z is a function that never goes wrong, right?
e^z, that series always converges.
Can you tell me the series, if I expand that into a series, what series am I looking at?
This is not on the exam, so to speak.
Do you know one over one plus something, what's the series for that?
Well the constant term, when z is zero is certainly a one.
I think the trick, it's this is geometric series, and because it's a z squared it's that.
That would be the geometric series.
With constant ratio z squared.
If I multiply that by that, 1+ z squared times that, everything will cancel and I'll get the one.
That's it.
So there is a Taylor series for this function.
Now, the reason I chose that example is, you could tell me, it doesn't converge if z is too large, right?
Is this an analytic function?
Where is this a good function and where does it have problems?
If z is less than one, and I really mean magnitude of z, so let me draw the z plane.
Here's the real part of z that you usually call x, and the imaginary part of z that you usually call y, because z is x+iy, and where will this series converge?
It'll converge out as far as this circle.
This is the Taylor series around zero.
Right?
The constant term I found that z=0.
Then that series, this function, is great.
It's an analytic function, everything, it gives us a solution to Laplace's, this'll be.
The real and imaginary parts of that will be the u and the S that solve Laplace's equation.
Out to, at least in this circle.
But something, there's a problem at the edge of the circle.
Now, here's my point.
If I think of 1/1+x^2, look at that for a minute.
That function has no problems at all, right?
1/1+x^2, can let x be anything?
With no trouble.
But 1/1+z^2, when we look in the complex plane, ah, we find a problem.
And where is the problem with this function?
At z equals, so everybody's looking at this guy.
There's a problem with that function at z=i.
And it happens to be not an accident, it's right there on the circle.
It's hiding in the complex-- it's not on the real axis.
So the real person didn't notice it.
But the complex person said ah, that's the problem.
There's a singularity there, and of course it's called a pole, and people in so many parts of science are interested in that.
Is there any other place that there's a problem?
That minus sign.
When z is minus i will also get a problem.
So this is a function with two poles, those two poles and they're the reason that the series couldn't make it.
If going out this way the series doesn't meet any problems.
But the series always goes out in a circle, and the first circle, the first guy, the first problem that hits, the series stops converging.
By the way, let me ask you a question.
Suppose I instead did the Taylor series around this point?
Now, what do I mean by that?
That's the point one, let's say.
What do I mean by that, the Taylor series around one?
I'll rewrite the function as one plus, well now, what do I do?
I want it in z minus one squared.
Oh, gosh.
I'm getting beyond what you will care about.
Again, if if I expanded, if I wrote the power series in powers of z minus one, what would it work in?
And then I'll stop with this example.
The circle would reach out until it hit a pole.
And it can't make it past that pole.
So it would be a circle of radius square root of two, there would be a circle there.
If we were going to discuss, and this is really Chapter 5 of the book.
I mention it, because you've got a book that explains this.
If I thought the center of the universe was there, and then the poles are still here, the circle will make it out to those poles.
So I can do Taylor series, I can sort of hook together Taylor series all over the place.
And they'll all quit when they reach a pole, but when I put all those circles together I can get all the rest of the plane.
OK, so that's something about, I don't know how I got onto that department, but it's amazing.
This, so the real and imaginary parts of that would be a flow, would give me a flow.
I don't know if it'd be easy to compute it or not, maybe I won't tackle that here.
But we could find the real part of that and the imaginary part of that, and we would have a genuine flow field.
Satisfying Laplace's equation with the two orthogonal, the streamlines orthogonal to the equipotentials.
We could totally do that example.
OK, let me, yeah, thanks
You say one test question is based on x+iy?
Yeah.
Well, this sort of stuff.
But, so u would be the, yeah that's right.
Yeah, so a test question would be something like one way or another you would end up with a u, and an S, and the u+iS, if they're a good pair, would be some function of this magic z. Yeah, yeah.
That's right.
So whatever.
We know examples, of course.
For example, this could be x squared minus y squared.
And the S that goes with that is 2xy, and the function that's involved there when I throw in the i is simply z squared.
OK, that would be an example where the real and imaginary parts of this give us the good u, its good friend S, and the picture of stream lines and equipotentials meeting at right angles.
Just, a beautiful picture, all coming out of this function.
So probably the quiz will have some other function.
But you'll still have a u and an S and a function of x+iy.
So if it's not this one, which I don't think it is, it won't be be this one
[INAUDIBLE]
Yes.
Because first of all, I wouldn't have mentioned it if I was.
And secondly, that's a little too messy, I think, to get a good handle of, to take the real and imaginary parts of that.
It's not impossible, of course.
We could completely do it.
There'd be some ratio of two polynomials.
Here we have just simple polynomials.
OK, does that help with that question?
Yeah.
What else is on your mind?
Any thoughts?
Yeah, thanks.
Curl, OK. Well, so I didn't really do three dimensions.
But curl is important.
And we did see in two dimensions the key fact that all this stuff, and let me just write down what the great connections are between these.
Because I can't let the whole semester go without writing down that, what is it, the group?
Is it the curl of a gradient?
The curl of any gradient of u is always zero.
Whatever it is.
Whatever u is.
And this comes from, let me put the other one down and then I'll just say why.
The other one is like the transpose of this one.
So if I transpose, so this is the zero operator.
Curl times gradient gives the zero.
So if I just transpose I still have zero.
So if it's a transpose gradient, I have minus divergence, and actually if I transpose curl I get curl again.
Of any, now I should put in, what should the curl act on?
It acts on a w, I guess is.
No, divergence w, it acts on an S, sorry.
OK, and the minus sign, of course, isn't going to matter because I've got a zero on the right hand side.
So S. Yeah, so if I take any field, I mean this is like, real proper vector calculus.
To check these, I call them identities and maybe sometimes people indicate an identity meaning it's always true for every u, or for every S. They'll use the triple equals sign.
Just to say they're really equal.
OK, so we could define the curl, but you've met it elsewhere and maybe this isn't the time to do that.
What's the key fact, the key little math business that makes all these true?
So there's sort of a formal math fact that makes them true.
And then there's the physical understanding of gradients being directions out with no rotation.
So the physical understanding of that, but the math, the formal math fact is the fact that the second derivative of u over this vector x and y is equal to what?
It's equal to second derivative of this vector y and x, yep.
So you would find if you wrote out all the terms here, or all the terms here, you would find that just by using that fact, they all cancel each other.
And the book, of course, does that.
So we simply didn't do 3-D in the vector calculus section.
So I'll stop there with that, because it's really saying that the curl is tremendously important.
It measures vorticity and flow, and it's being able to.
You know that like, you take the Navier-Stokes equations?
Well, the pressure and the velocity are the, I'd say primary variables or the natural quantities to measure pressure and velocity for a fluid flow.
But you could also use these identities to set up, in terms of other variables.
Just rewrite the equation and you get other things.
Mentioning Navier-Stokes and fluid flow reminds me to say, we keep using the example of Laplace's equation.
And a person could say wait a minute, get beyond that.
Right?
So why are you always, when you teach finite elements, why do you always start with Laplace's equation?
OK, well the main reason is it's the simplest one.
It's the one where you can really see what's happening.
More complicated equations would be for, like, elasticity, plane elasticity, or 3-D elasticity or other boundary value problems could be quite messy.
But Laplace's equation is not totally a waste of time.
First, it comes up when you have these scalar unknowns.
And then it also comes up in numerical methods for Navier-Stokes.
So the standard numerical method for Navier-Stokes, which would come in 18.086, ends up with Laplace's equation for the pressure.
So to have a fast Laplace solver, as in today's lecture, pays off.
So I'm just saying Laplace's equation is important in itself, it has the great advantage of being the simplest example we could possibly think of.
It's the example where an x+iy trick works.
And it actually comes up in serious big computations, because the equation for the pressure comes out to be a Laplace or a Poisson equation.
Now.
I kept going there.
Yeah, thank you
[INAUDIBLE]
We did, as a MATLAB problem
[INAUDIBLE]
Sorry?
And a first order, right
[INAUDIBLE]
Huh.
Yeah.
I would do it the same way but it wouldn't be symmetric, of course.
That was the point about that conviction term.
Is, the diffusion term would be just what we've done, right?
The diffusion with that second derivative.
And what would it look like in, as long as we're close to, what would convection diffusion in 2-D look like?
Just, I mean part of your interest is pass 18.085 and get rid of it, right?
But another part is like, these are problems that if you're in Course 16, Course 2, others, you're going to meet this.
So the diffusion part is going to be, again in 2-D I'll have some minus u, well, I made it simple because I took c(x) to be one.
It could have a c(x) in there.
And what would the convection term look like?
I'd have a velocity, in the x direction, of say a V_x.
And a velocity in the y direction, V_y.
V_y times, and that's just a number.
In the simplest case that would be my river is traveling, or my flow is traveling along, and equals zero
[INAUDIBLE]
Sorry?
Yeah I don't know which way the river's traveling, actually.
So those are just constants.
They could have positive or negative signs.
The V_x and V_y is the constant flow that's carrying, what am I doing here?
The flow is flowing along, and if those are constants it's just flowing stead, steady flow.
But it's diffusing at the same time.
And this would bring in that same difficulties that we met in the MATLAB 1-D.
So the MATLAB 1-D problem just didn't have a y. I don't care, yeah the sign I'm not worried about, it's just is that there, now I'm in 2-D. And what would I expect to see, I'd expect to see some trouble when V is large.
When V is large, convection, this is convection down here.
This is the convection part.
And if V is large, then so that this should be a lower order term, is really fighting against this higher order term.
I expect numerical difficulties, just the way we met.
So anyway, if I did a MATLAB example, stretched it to 2-D we see a whole lot of interesting stuff.
We'd see flow in, flow out, yeah.
But I just can't do everything.
But that would have a weak form, but your question about weak forms, weak form, when you have this anti-symmetric term for odd number of derivatives, is not quite as beautiful.
But you have to deal with it, of course.
Yep.
OK, Ready for whatever.
Any thoughts?
Let's see, just have a look again at the list of problem topics.
To see if anything occurs to you.
I mean, not that it should.
You know we're OK.
I'm happy to call it a day on that, and time for dinner for everybody and see you at 7:30.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, let's start with a review and preview.
I put a P up there because we're really looking into the Fourier part that just started this morning.
And there'll be some homework from these early sections about the Fourier stuff, so we maybe we should just do a few of those problems or discuss here today.
Just in advance.
Can I say one thing about MATLAB and the MATLAB homework first?
And maybe open a conversation about it?
So there's really two different problems that I'm personally quite interested in.
Two model, I'll say model problems because they're there for regular polyons in a circle.
And I'll draw an octagon again, so M sides.
And I'm interested in if M goes to infinity.
And I'm interested in two different problems.
So one of them is our MATLAB problem, minus is Laplace's equation.
What was it, four?
With u=0 on the circle.
OK, so that's our problem, totally open for discussion.
How many have started on that?
Oh, good.
OK. Well, then you all know more about it than I.
And that's great.
I'd be happy to learn.
So have I said everything there?
Yeah, we've got Poisson's equation inside.
We've got u=0 on the circle, so the problem's well defined and the solution should be one minus x squared minus y squared.
So that's the correct solution.
Maybe I can also tell you about the second problem that I'm interested in.
Because it hasn't come up in class but it's very important, too.
It would be the eigenvalue problem.
So this is problem number one, the steady state problem when you've got a source and you want to find out the temperature distribution.
The problem number two would be the eigenvalue problem, -u_xx-u_yy.
I take those minuses so that the eigenvalue will be positive.
So that's what the eigenvalue problem might look like.
And again let me say, with u=0 on the boundary.
On the third one.
OK, so a person would say this is Laplace's eigenvalue problem because we have Laplace's equation.
We've got eigenvalue.
As always, it's not linear because we have two unknowns, lambda's multiplying u.
And we have boundary conditions, and this would describe the normal modes, for example, of a circular drum.
If I had a drum, or a polygon drum.
So maybe I connect to actually build the drum, I might fold in the sides there and have a polygon.
And again, I hope that the eigenvalues of the polygon, this equation in the polygon, which are not known, by the way.
To the best to my knowledge, we know it only for M=3, which would be an equilateral triangle, and M=4, which would be a square.
And those eigenvalues, because of Fourier or something are humanly doable.
But I think five on up is, I may be wrong about six I'm not sure about M=6, a hexagon sometimes gives you enough help.
But beyond that you're on your own.
With finite elements to help you.
So there's a whole sequence of eigenfunctions, u, eigenvalues, lambda, just the way there were in one dimension.
And on the circle they involve Bessel.
That's where Bessel showed up.
He figured out the functions and they're not especially nice functions.
But they're studied for centuries.
Bessel functions come into that.
But, here I have the same question.
I mean, let me just say, for me this could be a UROP project if anybody was an undergraduate, or it could be a project over January or something.
I'd like to know something about what happens as M goes to infinity as the polygon approaches the circle.
So I'm hoping maybe on the homework that comes, if it's not too difficult, and maybe it's not, to let m go up a bit.
There is one thing.
That the code we're working with is linear elements, right?
We're using linear finite elements.
So we're not getting high accuracy.
So I would really like to move up to quadratic elements, at least, you remember quadratic elements would be ones where - well, let me draw the one that we've drawn in class before.
We only have to look at one triangle and then we cut it up into triangular elements by taking some pieces here, taking the points above, which I hope are now correct on the website.
Connecting those edges, and then connecting these.
Is that right?
Is that our mesh?
So that mesh is controlled by N. One, two, N points.
Also, N is going to have to get large too, to give me accuracy.
And another way toward more accuracy is, instead of linear elements, second degree.
So do you remember I wrote those down?
Let me take that little triangle out here as a bigger triangle.
It would look something like that, I guess.
The second degree elements have those six mesh points.
You remember I drew those but we didn't really have time to get further with them.
The trial functions phi, which are one at a typical mesh point and zero at all the others, they are computable.
We're up to second degree, so it's a little, second degree things, then the first derivatives, which come into the integrations, are linear.
And not constant.
So a little bit harder.
But finite elements, linear or quadratic.
Or higher.
Could be used for this problem.
As we know, and for this problem.
What I wanted to add, that I've not mentioned in class, and I think we may just not get a chance to do it, is what does the finite element method look like for an eigenvalue problem?
Because eigenvalues are highly important.
That's the different way to understand.
There's the matrix K and its entries.
But then there eigenvalues.
And you might think that, what do you think is the discrete eigenvalue problem copying this one?
Here's my point.
Your first guess would be, well this is like K, right?
This is like KU, right?
(K2D)U, I should call it, maybe.
Well, I'll call it K, because K2D I have specifically reserved for the Laplace stiffness matrix on a square mesh, square mesh with triangles, the K2D.
That was one specific matrix for one specific mesh, and here we have a different mesh.
So I should just call it K. Ok, I think if anybody was going to make a guess, they would say OK, KU=LAMBDA*U.
Maybe I'll use capital LAMBDA, because I'm using capital U.
Is this the finite element method eigenvalue problem.
And if you answered yes, I would have to say, well that's a reasonable answer.
But it's wrong.
The eigenvalue problem, when I take the differential equation for the Laplace, Laplace's equation, lambda u on the right side, and I go to do finite elements, it produces K. Out of this stuff, out of the weak form, all that stuff.
But it produces another matrix on the right-hand side from the constant term, and we have not really mentioned it, it's the mass matrix.
So this, instead of just the identity here, there's a mass matrix.
So that is the problem that you could do.
I could've made a MATLAB project.
I bet I'd do it next fall.
right?
You guys did the first one, this one.
Or you are doing it now.
And I'm going to pause in a minute for questions about it, or discussion of it.
But this one brings in something called the mass matrix.
So let me just say what those are.
If I write down the entries in the mass matrix, you'll sort of get an idea of why they are.
So what are the entries in the stiffness matrix?
K_ij, you remember, is the integral of the d phi_i/dx, d phi_j/dx.
Plus d phi_i/dy, d phi_j/dy, dxdy, and that's what's you're computing.
And that's what that code is computing.
And when phi is linear, phi linear, then slopes are constant.
So all you have to do, and what that code in the book is doing, is figuring out what are the slopes.
These things are constant, so we just need to know the area of the integration where we're integrating.
The area, triangle by triangle.
Fine.
That's what we're doing.
That's what that code is just set up to do.
Now, I have to tell you what is M_ij, the mass matrix.
I just think you don't want to have - we haven't done too badly with finite that elements in here.
We did it in 1-D, where we got it kind of straight.
And now we're seeing what it looks like 2-D.
But I had not really mentioned a mass matrix.
So here it comes.
The mass matrix will be the integral of phi_i i times phi_j.
dxdy.
It's the zero order, no derivatives, just plain zero order, as you'd expect from the fact that the term in the continuous part is zero order.
So it's this mass matrix that comes in.
And maybe we could just look to see which entries will be zero and which will not.
How sparse is it?
What does the mass matrix look like?
And can we just, let me do 1-D first.
So there's a phi, right?
There's another one.
There's another one.
So, what do you think about the mass matrix, one phi multiplied by another phi and integrated?
Is it diagonal?
No, because each phi overlaps its two neighbors.
So tell me what kind of a matrix m is going to be?
In 1-D. Tri-diagonal.
It'll be tri-diagonal.
Now, so was K. So K and M actually have non-zeroes in the same places.
the same sparsity pattern.
But, of course, not the same numbers in there.
K had minus ones and twos and fours and minus ones.
What can you tell me about this tri-diagonal matrix?
When I integrate that against this, well, again I would do it element by element because this against this, they only overlap here.
Right?
I'll just draw the one place that they overlap.
And what's the point?
They're both positive.
So the mass matrix is, its rows don't add to zero.
Its rows tend to add to one.
But it's not diagonal, that's the difference.
OK, so I just felt I couldn't feel as though I'd done a decent job in describing finite elements if I didn't describe this.
Didn't mention this mass matrix.
And maybe I'd better say where it comes from.
Because eigenvalue problems, it may come number two, but that's pretty high up the list.
So let me tell you where does this mass matrix come from.
First, let me tell you about eigenvalues of a, matrix eigenvalues.
So the answer was, is this the finite element eigenvalue problem?
Only if there's an M there.
And now I want to, OK, first of all, what MATLAB command solves that problem?
Let's just be a little practical for a moment.
What MATLAB command gives me the matrix of eigenvectors, the matrix of eigenvalues would come from eig of what?
I'd call this the generalized eigenvalue problem.
Generalized because it's got somebody over here.
And it's just K,M.
Or of course you get the same answer, well you get the same eigenvalues, I guess the same eigenvectors, yeah, if you or, eig(M^-1,K), of course.
If you want to do it with just one matrix then bring M inverse over here.
But and M inverse, the inverse of this tridiagonal matrix, is full.
No zeroes in the inverse.
So everybody would much prefer this tridiagonal tridiagonal one.
So that's how MATLAB would do it.
And what I want to know is, back in this problem, how close do the finite element guys come, on polygons, come to the correct solution on circles.
I'm hoping that for problem one you can maybe keep M and N equal, or maybe four times M or something.
And let them grow and see.
Well, for example, at the center of the circle, or how quickly do you approach the correct answer one at the center of the circle?
I think it's going to be a good problem.
Let me open to, so I started out just talking there.
What about the MATLAB problem.
You made a start on it, is it going?
Have you got a graph, maybe, or what's reasonable to graph, to give Peter to look at?
Who's done something on that MATLAB problem?
Yeah, go ahead tell us all what to do
I made the triangle pi section
OK, right
[INAUDIBLE] and I found that the [INAUDIBLE] changes to
With M more, I see.
So if you just fixed M like eight, and let n get, it didn't change significantly.
It wouldn't, of course, converge to the right answer.
It'll converge, if it does, to some kind of an answer, for the polygon.
Right.
That's right.
So you know, as I wrote the problem I didn't know whether I dared say let M get increased too, but of course that's the real question.
And what happened then?
Did error shrink?
OK, and now maybe it's possible to see how fast or something that's always--
[INAUDIBLE]
Ah.
OK, at the center.
OK, then I hope for more comment.
Let me say one more thing.
My theory is that the error at the center is quite a bit smaller than the error closer to the boundary.
I would be interested in an error, is it fairly even?
Oh, my theory's wrong.
It wouldn't be the first time.
And maybe because it's linear.
Yeah, my theory is more for better elements, like these.
I'd be interested to know.
Why do I think, why do I have this theory, which you guys are going to prove wrong anyway, but still.
After you've proved it wrong, you won't listen to me if I tell it to you.
So now I'll tell it.
My theory is that the error around the boundary is, there's no error at these vertices, and then there's sort of a going to be an error because the real answer is not zero along here.
It's sort of near zero, but not quite.
You know, there's an error.
So there's errors around here, from getting the boundary wrong.
Squaring it off.
But my theory is that errors, the boundary stuff, drops off quickly as you go inside.
That's why I think, from those, you remember those - well, we'll see them again either today or Friday, those r^n*cos(nx) type things?
That cos(n*theta)?
Yeah, you remember those are the typical solutions to Laplace's equation.
And then so that if, and it has some coefficient, of course, a n. And I look at that, that might be a piece of error.
And it's way bigger when r is one and way smaller when r is zero.
So anyway, that's sort of my theory.
That if you have, like physically.
You have a circular plate and you're maintaining the boundary temperature at some sort of oscillation.
Like, near one but up and down from one.
Then I think further inside, it doesn't know.
It hardly knows about that oscillation.
This is my theory.
That toward the center of the circle it only sees kind of an average boundary temperature and not your little ups and downs.
So when M is big, I expect that part of the up and down part to be not so significant in the center.
Anyway, now that's my theory
[INAUDIBLE]
Ah, good question.
So if we only looked at the center, would it all be the same?
I mean, if we're only looking at that one point where it should be 1 at the center, but along the thing, I don't know.
If you look at both, and see a significant difference in the behavior I'd be interested.
Yeah, yeah.
You know, all these problems are things that there's no single solution to
[INAUDIBLE]
The error between one minus r squared
[INAUDIBLE]
Oh, right, we've got slope error, too.
That's a very significant point.
I see, right.
So the slope error's in there.
Everybody knows, then, everybody in working the problem, I mentioned that the boundary conditions in this piece of pie were zero along here and normal derivative, somehow it got printed du/dh, but that was an accident.
It should've been du/dn, dn is zero.
So Neumann conditions on this thing and then I was a little scared about that point, but I think phooey on it.
It's just, don't worry about it.
But what I was going to say.
How do you, what do you do to take into account this du/dn=0?
This slope condition on these long boundaries?
What should you do in finite elements to take account for that?
And the answer is, in one nice word?
Nothing.
Right, nothing.
Your finite element method should not, you don't impose any condition along these boundaries.
Just use the code as it is with zeroes on this boundary.
And it should work, yeah.
It should work.
Any comments on other people.
Did you get reasonable results, or?
Tell me something.
Because you guys looked at those graphs and I have not.
Any feedback yet?
On those?
I'm happy to get email, too, about.
So all the email, first of all they've corrected the typos in the original coordinate positions.
And now they've pointed out I'd better look at M is very, very welcome.
It doesn't mean that everybody has to do this, if you've completed that MATLAB assignment, you never want to see it again, and you've kept M=8, it's ok.
But if you're interested to see what happens if M goes to 16 or 32, I'm interested also.
Right, yeah.
OK, so anyway that's the problem we're really thinking about.
And that's the problem that is equally important, but it seemed reasonable just to do one of the two.
And we were set up to do, we have the code for the stiffness matrix, we would need a new code to do these integrals.
Because this will be linear times linear, right?
I'll have to compute that one times this one and I would need new formulas that are not there.
I'd need formulas for, this will be linear times linear so I'll be integrating x squared type stuff.
And xy's, because I'm 2-D, and y squareds.
So it would take a little more code, but not much.
I think the math, oh here's a question for you.
Here's a question for you.
Suppose I have my trial functions, phi_i(x).
What do they add up to?
Let me again draw a mesh, so I've got a mesh.
These are you know, I'm sorry, I want to put in some more triangles here.
Lots of triangles, whatever.
Let me get some more vertices, too.
I'm getting in trouble.
OK, whatever.
So phi_i, is the piecewise linear guy that is one at node i.
So I've got all these different nodes.
I need a node there, so I've got one, two, three, there's a node, there's more nodes.
If I add them all up, this is just like in an insights question.
I've got all these, you could add up these hats in 1-D. What's the sum of the hats in one dimension?
One.
Good.
The sum is one.
It's a nice fact that these guys add up to one.
And now why is it still true here in 2-D, that these little pyramids will add to one?
That's an inside question, but it's worth thinking about.
Why do those pyramids add to one?
Let me leave that question.
I'm thinking about, we haven't imposed any boundary conditions yet.
We've got them all.
and I claim that if we add up all the pyramids including the boundary chopped off pyramids from the boundary, that we'll get one throughout the whole, now it'll be phi(x,y).
Because now I'm moving to 2-D, with pyramids.
I think we'll still have one.
Let me give you a minute to think about that one.
And then we could turn to Fourier questions if you would like, we could do some problems from the text.
Any thoughts about this guy?
Why should all those individual pyramids add up to a flat group?
Why did it work here?
Well, it worked because you could see it, right, somehow?
Does it still work if the nodes are not equally spaced?
So we've got a hat function for that guy, and a hat function for this guy, and a hat function for this guy.
And these guys are in there, too.
We haven't imposed anything.
So those one, two, three, four, five functions, five phis, they add up to one and y.
Well, you're going to say it's obvious, but that's what professors are allowed to say.
Things are obvious, you have to actually say why.
Which is not as easy.
So, why do they add to one?
Let me look inside one element.
Why does the sum of these two guys add to a flat top inside that interval?
[INAUDIBLE]
At the end points, you've got it.
Because what's happening at the end points?
This guy, one of the guys, the right guy is one.
And all other guys are zero, right.
And this guy is also at one.
Because it's the right guy.
It has height one and all others zero.
So at the nodes we are at one, because of one person, really, one element.
And then?
[INAUDIBLE]
Right.
But the sum of them is, why is the sum of them always one, why is slope zero?
Yeah.
The slopes cancel, right.
We know that in between it will be a linear function.
That would be one way to look at it.
If I add up a linear function and a linear function the sum is a linear function.
So I'm getting a linear function, which is one at those points, so what is that function?
One.
Right, you know that's the straight line.
So, that idea will work here too.
Look inside some little triangle here.
OK, that's got one, two, three corners, OK. And if I look at this sum, what is it at this point?
If I look at that sum at this corner, one guy is one, the one for that pyramid.
And all others are?
Zero.
So the sum is one there, the sum is one there, the sum is one there, so that blowing up this little triangle, this is at height one, this is at height one, this is at height one, so what's the roof?
Flat.
It's just a nice way to see the nice property of these phis.
That there's a phi for every node, and they add to one.
To that's it.
OK, well I was going to say one more thing and I am, about this eigenvalue problem, just because I'll never have a chance again.
So this is the moment to say something about the eigenvalues.
Lambda.
Eigenvalue.
I'm answering the question where does K come from, where does M come from?
Well, eigenvalue is, boy we really got dramatic music here.
That's a great Gates of Kiev, I think might be.
Mussorgski.
If you like drums and big noise, it's not music actually, but you got a lot of noise out of it.
Well, of course, he'd know more than we do, but still.
OK, so the eigenvalues in the matrix case for Kx=lambda*M*x, the eigenvalue problem, lambda, the lowest eigenvalue, lambda lowest, has a nice property.
It's the minimum of sort of our energy over our other energy.
I just think, well this is something you should see.
This is a quotient here.
It's got a name called the Rayleigh quotient.
And it would appear in the book.
So really, I guess what I'm doing is calling your attention to something that's in the book.
That this a ratio of x transpose K x to x transpose M x, if I look over all vectors x, the lowest one is the eigenvector.
The best x is the eigenvector and the ratio is the eigenvalue.
This is like my point that I wanted to mention the Rayleigh quotient.
Here it is in the matrix case, and there would be similar Rayleigh quotient in the continuous case.
I'll just leave it at that.
That in describing eigenvalues, we can talk about Kx=lambda*M*x, like this.
Or we can get energy into it.
And you remember the whole point about finite elements is, look at the energy.
Look at that the quadratics.
Multiply things by things.
It came from the weak form, it didn't come from the strong form.
In the differential equation here, we just have single terms.
We got to these things through that process of multiplying by u's and integrating.
That's what gave us these products and it works also in the matrix case.
OK, that was a lot of speechmaking about topics that we simply didn't have time for in class.
I'm ready for any question, or I'm ready to maybe do a Fourier example, would you like that?
Because this is where we really are.
I'll even take one that will be on the homework.
Just so you'll have a start.
OK, let me take a square pulse, yeah this is a good one, I think.
In Section 4.1, there's a question for the Fourier series of a square pulse.
OK, so what does the square pulse look like?
Here's minus pi to pi.
Here's zero.
The square pulse goes along here, up square pulse and down.
Actually, let me go to L/2, oh I'll just call it h. Let me find the Fourier series for this function.
It goes along at 0, it jumps up to 1 over a interval of length 2 h, going from minus h to h, and then back down to 0 and then repeat.
So bip bip bip, square pulse.
So that's my function.
Is that function odd, or even, or neither one?
It's even, so I can call that C(x).
And figure that I'm going to use cosines for that one, right?
So tell me a formula for the coefficients, what's the integral that I have to do?
So my C(x) o is going to be some a_0, we have to think what's a_0, then a_1*cos(x), a_2*cos, and so on.
So on.
a_k*cos(kx).
OK, what's the formula for a_k?
Before I plug in that function I would like to get the formula.
So I'm looking for the formula.
It's a formula to remember.
So I'm not wasting your time.
Because you're going to see it on the board and it'll just take a mental photograph of it.
What do you think it's going to be?
How am I going to get it?
I'll multiply both sides of the equation by cos(kx), right?
And I'll integrate.
So and then when I integrate, the cosines are orthogonal.
Just like the sines this morning.
All those terms will go, except for this term.
When I multiply this by cos(kx), I'll have cos(kx) squared.
Here I'll have a cos(kx), and here I'll have a whole lot of cos(kx)'s but when I integrate, all this stuff is going to disappear.
And this will all disappear.
This is it.
So a_k is going to be the integral of my function, times cos(kx)dx.
Divided by what?
Divided by the integral of cos(kx) squared.
Because I haven't normalized things.
So I don't know that that's one, and in fact it isn't one.
So I have to remember to put that number in.
OK, so that's the formula and that number turns out to be pi, again.
If I'm integrating from minus pi to pi, then the average value of the cosine squared is a 1/2, it's sort of as much above 1/2 as it is below 1/2, and so the average of the half, the interval is 2pi, so pi.
OK, that's the formula.
Please just take a mental photograph.
Catch that one.
Alright, now I've got my particular C(x), my square wave, square pulse.
Very, very important.
Very important Fourier series here.
Famous one.
OK, so what do I have?
From minus pi to pi, so what's my integral?
Well, my integral really doesn't go from minus pi to pi because my function is mostly zero.
Where does my integral go?
Negative h to h, right?
And in that region, what is C(x)?
One.
So you see it's going to be nice.
From negative h to h, where this is one, I just have to integrate cos(kx), so what do I get?
sin(kx), over k, and the pi so you see again that that k is showing up in the denominator, and that's going to give me the typical decay rate of 1/k for functions with steps.
For step functions.
And now I have to evaluate this between minus pi and pi.
And no, h. Better be h. I mean, minus h and h. So what do I get for that?
I get sine(kh), right?
At the top, and what do I get at minus?
So I now I want to subtract, what is the sin(-kh)?
It's a negative, right?
So as I expect with an even function like cosine, am I just getting twice?
I could take it from 0 to h, and it would give me one of them and the other one.
Yep, I think so, and divide by k pi.
So those are the Fourier coefficients.
Except for a_0.
a_0 has a slightly different formula, because for a_0, why is a_0 different?
How do you come up with a_0, and what's its meaning?
a_0 has a nice meeting, so this is worth having come this afternoon for.
a_0 will be what?
Well, I could get it the same way.
What will I multiply both sides by?
If I want to pick off a_0?
Just one.
It's not a cosine, it's the cos(0x), it's the one.
And then I integrate.
I'm just going to get the integral from minus pi to pi of C(x) times one, divided by the integral from minus pi of one times one.
Same method.
multiply both sides by one, which was the very first of my orthogonal functions.
Integrate it, all the other integrals went away, right?
The integral of cosine over a whole interval.
Its periodic.
You get the same at the two ends, so the difference is zero.
So we just, the only term left was a constant.
And now what is the integral, row what's the denominator now?
That was the little, slight twist.
2pi.
The denominator's 2pi.
Yeah.
That's that's why it's not, yeah, it's slightly irregular, I have to divide by 2pi.
And now, what word would you use to describe, if I have a function, and integrate it, and I divide by the length, what am I getting?
There's an English word that would describe what this is.
Average.
This is the average.
And it has to be.
This constant term is always the average.
And what will it be for this?
So this was a_k, and what is a_0, then?
So you can now tell me, so everybody's remembering this formula, you integrate the function and divide by the 2pi.
Now we've got a particular function, so what is the integral of that function?
So what does this equal?
For this particular C(x)?
What's the area under that function C(x)?
2h.
Right?
So 2h/2pi cancel twos.
So there's a constant term, a_0 is h/pi and the and the cosine terms are, yeah, actually we're going to get something nice.
A really nice way to complete this will be if I put this together, put this series together.
So now I'm saying that this square pulse is that constant term h/pi plus the next term a_1, you can see all these terms have 2/pi's.
I'm a little surprised that h over, yeah, I guess it's right.
2/pi.
So I've got sin(h), I think.
And now I'm just copying this.
2/pi*sin(h), sin(h), is that what I want?
Over one.
That's the coefficient version of sine, of cos(x).
a_1 was the coefficient of cos(1x).
And then a_2 is the coefficient of cos(2x).
So that will be sin(2h).
k is two, so I have a two down here, cos(2x).
And so on.
Yeah, I think that's the Fourier series.
That would be the Fourier series for the square pulse.
Yeah.
That would be the Fourier series for the square pulse.
Could I test any interesting cases?
Suppose h is all the way out to pi.
Suppose I take that case.
Let h go all the way out to pi, then what's my function?
If h=pi, then what have I got a graph of?
Just one.
It's just a one.
If h is pi, what happens?
That becomes a one, and what about these other things?
What is this thing when h is pi?
Zero.
All the other terms go away.
It's just a sin(2pi) that would go away.
Yeah, so if h is pi, if I go out to the place where I don't have any jumps at all because it's now all the way out there, then these terms all disappear and I just have this.
And I would like to ask you and it's going to come up on Friday, too, what happens if h goes to zero?
Well, let me just take h going to zero.
What happens to this whole thing?
What happens to my function if h goes to zero?
Goes to zero, right, then squeezed it to nothing.
And if h is zero then sin(h) is zero, I get 0=0, that's not interesting enough to mention on Friday But there is one case that is important.
Suppose I make the height, yeah.
Make a guess.
Suppose I make the height higher as I make the base smaller.
I'm going to keep the area as one, so if this has a base of 2h, I'm going to have a height of 1/2h.
So if I keep the area at one, so the height now is 1/2h, so now my square pulse I've divided it by 2h.
I have a 1/2h multiplying everything.
And now if I let h go to zero, something more interesting will happen.
And what?
Just tell me first, what would you expect to happen?
Delta.
Right, delta.
So what I'll see show up will be the Fourier series for the delta function.
When I divide by 2h, so I have sin(h)'s over h's, and of course what's the great fact about sin(h)/h?
As h goes to zero, it goes to, everybody know, that's the big deal.
Yeah.
One.
sin(h) is the same size as h for a very small h, and approaches one.
Yeah so we'll see the delta function Friday.
OK, so you've got a sort of mini-lecture instead of a real chance to ask about homework.
Next Wednesday should be different because there will be Fourier series homework, and I'll be ready to answer questions about it.
OK, thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so review session on the first part of the Fourier chapter, these two topics that we've done and that homework is now coming on.
Fourier series, the classical facts.
Plus, paying attention to the rate of decay of the Fourier coefficients, it's an aspect not always mentioned.
And the energy equality is important.
So it's not just here's the function, find the coefficient.
That's part of it but not all of it.
And then the discrete series, we're doing today.
OK, so those are the two topics for today, and then the next review session, which would be two weeks from now, would focus especially on convolution and Fourier integrals.
OK, so I'm open to questions on the homework, or off the homework.
Always fine.
I didn't know how many questions to ask you on the homework.
I wanted you to have enough practice doing this stuff because the time for this Fourier part of the course is a little shorter.
Thanksgiving comes into it, so needed to do some exercise.
And you've got a good question.
Thanks
[INAUDIBLE]
Number 18 on the homework, OK. Ah yes, OK.
Right, alright.
So the idea of that problem, I'm really asking you to read the two pages, the last two pages of the section.
That use Fourier series to solve the heat equation.
So we've used, briefly, Fourier series to solve Laplace's equation.
So that was just to recall.
So Fourier series to solve Laplace's equation was when the region was a circle.
The function was given, the boundary values were given.
It's 2pi periodic because it is a circle.
And we solved Laplace inside.
Because on the boundary, the perfect thing we needed was the Fourier series to match the path.
Now, I'm taking another, classical, classical application too.
The heat equation.
So I made it heat equation, so this direction is time.
This direction is space.
The heat equation is u_t=u_xx, the coefficient, everybody here knows there would be a c in there.
But let's take it to be one.
Then what are the solutions, and how does a Fourier series help help you to match the initial functions?
So I'm matching, I'm given u(x,0) here.
OK.
Along this is at time zero.
So that says at a time zero, so I have a bar.
I have a conducting bar.
And this is such a classical example that I didn't feel you could miss it completely.
Even though we look beyond formulas.
But here's one where the formula shows us something important.
OK, so what are solutions to this equation?
You look for solutions, so the classical idea of separate variables.
Look for solutions that are of the form, some function of t, maybe I'll try to use the same letters as the text.
Some function of t, times some function of x, OK?
And the text uses, look for solutions.
u(x,t) that are of this form.
Some function of t, and I didn't remember.
Ah yes, it's A(x)B(t).
OK, that's what I mean by separating.
So those will be especially simple solutions.
And when we go to match the initial condition, I'll just plug in t=0 and I'll see the A(x)'s.
Well, what are they?
In this case, so their eigenfunction - oh, I have to tell you about the remaining boundary conditions, don't I?
Because that will decide what the A(x) has to satisfy, and will decide what those eigenfunctions are.
So, let's see.
In this problem I think I picked free conditions.
So I made the interval minus pi to pi, that's a change.
Minus pi to pi just so we have nice Fourier series.
And here I have this boundary is free, du/dx, u'.
At x=-pi.
For all time, so up this line is zero.
And also u' at plus pi, and all time is zero.
So up those lines, heat's going out.
That's what that means.
Is that what that means, or does that mean heat can't go out?
No, so what's happening?
Heat's not going out.
Is that right?
The slope is zero, right?
The slope is the temperature gradient, we're requiring the temperature gradient.
So the ends of the bar are insulated.
So this is insulated.
No passage through, right?
Is that what that means?
Yeah.
It's like there's nobody, it's cut off there.
The rod isn't extended for heat to go further.
OK, so that tells us what the x and the t, what the A(x) and B(t) are.
So here's the point.
You plug that hoped-for solution into the equation, right?
So I plug it in here.
What do I have on the left side, just the time derivative?
So that's A times A(x), B'(t), you see taking the time derivative doesn't touch A(x).
On the right-hand side, I don't touch B(t), but I have a second derivative.
Of the x part.
So far, so good.
Now, a little trick.
If I divide both sides by A and by B, I get B'/B equaling A''/A.
Right?
Just, put the A under here, put the B under here.
Now what?
This is neat, because that function is only depending on time.
This function depends only on x, so that the both sides have to be constant.
One can't actually change with time, because this side is not changing with time.
And this couldn't actually change with x, because that's not changing with x.
So those are both constants.
So both constants.
Let's see, I'll maybe just put a constant.
And various constants are possible.
OK, so now you see the point here.
Now I have two separate equations, I have an equation for the B part, dB/dt, B'.
If I bring the B up there, I have equals, the constant times B.
And I know the solution to that.
B(t) is, as everybody knows, what's the solution to a first order constant coefficient equation?
Just e^(ct)*B(0).
Good, we've got B.
We've got a B(t) that works, and now what's the A that also works?
That has A''.
And I bring the A up, so now I have A''=cA, so the A that goes with it is?
Oh, OK, I've used a c there, so what's the good?
I want two derivatives should bring down a c. Let me change c to minus lambda squared.
How about if I look ahead, change this constant to minus lambda squared, because I want something where two derivatives bring down minus lambda squared, and what will do that?
Any amount of cos(lambda*x), right?
Because two derivatives will bring down a minus lambda squared.
And any amount of sin(lambda*x).
And this is now e to the minus lambda squared t. I'm doing this fast, but actually it's totally simple.
The conclusion is that I now have a bunch of solutions of this special separated form.
Where B(t) could be that and A(x) could be either of those or any combination of those.
And I have to use the same lambda for each, so that the two equations will work in the original problem.
Good.
Now, so far, no boundary conditions.
What I've got so far is just a lot of solutions.
These times that.
With any lambda.
But of course the boundary conditions will tell me the lambdas, first of all.
And how do they tell me?
The only boundary condition I have is this free stuff.
So it's, free the slope should be zero at pi, and zero at?
So that's the x direction.
So that's going to tell me - I've forgotten.
Do I want cosines or sines?
Cosines.
I want the derivative to be zero at pi.
Yeah, so I think I want cosines, good.
And then lambdas can't be anything at all.
Because, should lambda be an integer or something like that?
I think maybe lambda should be an integer, because I want to plug in pi.
So let me take the second derivative.
Is minus lambda squared cos(lambda*x).
And then I want to plug in lambda=x=pi.
And I want this to be zero.
So lambda should be an integer, is that right?
At multiples of pi, the cosine is zero, yes.
Is it?
no.
Did I want sine?
Maybe I wanted sine.
Oh, it's the first derivative I'm looking at, thank you.
Thank you.
OK, good.
OK, now I've got it.
Thank you.
And now I see lambda should be an integer.
Lambda should be, zero is, yeah zero's alright.
That's the constant term, yeah, we need that.
zero, one, two, and so on.
So do you see that I've now got, now I can take, I've got a lot of solutions.
And I have a linear problem.
So I can take any combination.
So finally I have final solution is that u(x,t) is any combination with coefficients, I'm free to choose, of A(x), which is cos(nx), because lambda had to be an n. And n could be anywhere from zero on up.
Times e to the minus, lambda's n, so that's n squared t. Did I get that?
Let me draw a circle and step back.
What's up?
[INAUDIBLE]
I could have negative n's, they wouldn't give me anything new, right?
I mean cos(nx) and cos(-nx) are just, one's just the negative of the other.
So these are the good guys.
I've got a cosine series because I've got free n's, right.
Because of the boundary conditions.
Do you say that?
That's pretty nice.
There's my A(x), there's my B(t), I can take any combination.
Usual stuff.
You get to that solution.
OK, and now I have to meet the initial conditions, right?
Boundary conditions are now built-in because I chose cosine.
Or you did.
Now, this will tell me what the c's are.
I'm going to set t=0.
At t=0, I'm given the initial condition, u(x,0), and I have the same sum of c_n*cos(nx), e^0.
So this will tell me the c's are the cosine coefficients of the given initial conditions.
So I expand, so here's where Fourier series is paid off.
Expand the initial function in a cosine series.
And then go forward in time.
This is just the old e^(lambda*t).
Only the lambda we're talking about here is minus n squared.
And you see what's happening here?
What's going to happen for large time?
So this is a very physical problem.
That I think you cannot take 18.085 without seeing this problem.
You can't learn about Fourier series without using it for the initial value.
And then propagating in time with the usual exponentials.
For each one, and now as n increases what do I see?
Faster and faster decay.
For large n, these are going to zero extremely fast.
So that what you see with a solid bar, which starts with the temperature u in some probably not negative unless it's a really cold bar.
But, anyway, it starts with some initial temperature.
That flattens out fast.
The heat flows, to equilibrate.
What I approach is the constant terms, c_0.
This approaches c_0.
Because all these other n positives, they go to zero.
So the heat distributes itself equally.
OK, and now I guess the particular u(0) in the problem was a delta.
OK, and so the particular u(0) was all, was from that really hot point.
So we know the coefficients.
We know the cosine coefficients for the delta function, we know these c's, and what were they?
1/2pi was c_0.
And the other c's were 1/pi, I think.
Is that right?
Maybe they're all 1/2pi.
Maybe.
Yeah.
Whatever.
They disappear fast.
And this is what we approach.
So the heat from the delta function is, yeah.
So is that everything the problem wanted?
Yeah.
Yeah.
I think we've done it.
We'll put in the c_n's to complete that picture.
Into here.
And then c_0 is the one that survives over time.
Yeah.
I guess you've, once I got rolling I couldn't stop and that's u.
For investing time this afternoon you get a fast look at this classical, classical problem of separating the variables using the Fourier series for the initial function.
And recognizing that we're doing this on a finite interval.
If the bar was infinitely long, then we would be talking about Fourier integrals.
And that's what's coming up a bit later.
We would integrate instead of sum.
Yeah.
But the idea would not be different, if we had infinite bar then we would not be restricted to n equals zero, one, two, three, any n, any cosine, wouldn't have to be an integer at all.
Any number, any frequency would be allowed.
And so we would have to integrate that, yeah.
And that's a classical problem too, again.
It's come up in a modern way, that the famous Black-Scholes equation.
So.
The heat equation is for 18.086.
Here, we brought it up because we could solve it fast.
But the actual, yeah.
The most important solution I could give you to the heat equation would be the one that starts from that point source of heat.
But on the whole line.
The one that would be integrals instead of sums.
Yeah.
So we came pretty close to solving the most important heat equation problem.
But doing the periodic case, with just cosines, and the infinite line case would be the most famous of all.
Yeah.
And it has a beautiful form, and I was going to say that the heat equation's pretty classical.
But let's see, where can I write the magic words, Black-Scholes.
Next to the heat equation, so that's the heat equation but it's also - do you know these names, Black and Scholes?
Anybody in Mathematics of Finance?
So these much-despised option, derivative options, so people on Wall Street, traders, will carry around a little calculator that solves the Black-Scholes equation so they can price the options that they're bidding to buy and sell, so they can price them fast.
And that little calculator does a finite differences, or a Fourier series solution to this Black-Scholes equation, which actually, if you change variables on it, is the heat equation.
So what you see here as is actually important on Wall Street except, it's probably not the right moment to mention it.
Right?
So you can blame the whole meltdown on mathematicians.
But that wouldn't be entirely fair.
They didn't mean it, anyway.
But that's been the biggest source of employment, I guess, apart from teaching, in the last ten years.
People who could work out the partial differential equations, and they get more complicated than the heat equation, you can be sure.
And so the classical one, these guys are economists at MIT and Harvard, or they were.
And I guess maybe the Nobel Prize in Economics came to part of that group.
And also to Merton.
Maybe Black, possibly Black died before the time of the Nobel Prize.
Anyway, they were the first.
And it's a beautiful paper, beautiful paper too.
Just to figure out how do you price the, what's the value of an option to buy that allows you to buy or sell a stock at a later time?
Yeah.
So it's, well of course you have to make assumptions on what's going to happen over that time and that's where Wall Street came to grief, I guess.
If you had to put it in a nutshell, I mean, the options, the standard straightforward options, those are fine.
Using Black-Scholes, and then what's happened is they now price more and more complicated things.
To the point that the banks were buying and selling credit default swaps, insurance swaps, that practically nobody understood what they were.
They just assumed that if there was always a market, for them like insurance, somehow it wouldn't happen.
And you get on.
But it happened.
So, now we're in trouble.
OK, that's not 18.085, fortunately.
Or math, but.
Anyway, a lot of people got involved with things they didn't really know about.
And then were selling them as well as of course the mortgage problems.
Anyway.
Ready for other questions on our homework, or these topics.
Yeah
[INAUDIBLE]

[INAUDIBLE]
OK, right, yeah.
Have an image of waves, so --
[INAUDIBLE]
Yeah.
I suppose if I had to have a picture of the discrete thing, if my picture of the function case was a bunch of sines, and cosines, somehow adding up to my function.
And if time, I'm solving the heat equation then those separate waves are maybe decaying in time.
Here.
So that when I add them up at a later time I get something different.
Or if it was the wave equation, which is probably your image, they're moving in time.
So they add up to different things at different times, because they can move at different speeds.
Yes, so a function is a sum of waves, right?
Then what would the discrete guy be?
I guess I would just have to imagine the function as only having those n values.
And my wave would just be, a wave might be just n values there.
But still, if I have a time- dependent problem maybe that thing is pulsing thing up and down.
It's just that it's only got a fixed number of points.
And I'm not looking at the whole wave on a, yeah.
But I don't think it's essentially different.
And of course, the fast Fourier transform and the discrete case is used to approximate the continuous one.
Yeah.
You can look in numerical recipes for a discussion of approximating the Fourier series by discrete Fourier series.
I mean, that's an important question.
Because of course, Fourier series has got all these integrals.
The coefficients come from an integral formula.
We're not going to do those integrals exactly, so we have to do them some approximate way.
And one way would be to use equally spaced points and do the DFT.
Can you just remind me what that integral formula is?
I don't want you to, it was on the board today.
What's the formula for the coefficient c_k in the Fourier series.
I'm really just asking you this because I think you should have it in some memory cache, you know in fast memory, rather than in the textbook.
OK, so what's the formula for c_k, in the Fourier series case?
Everybody think about it.
It's going to be an integral, right?
And I'll take it over zero to 2pi, I don't mind.
Or minus pi, pi.
And what do I integrate?
As the Fourier coefficients of the function f(x)?
So I take f(x), I remember to divide by 2pi, I'm doing the continuous one, what do I multiply by to get the coefficients?
e^(-ikx).
So I've forgotten whether I signed some of these very early questions.
It just gave you the function and said what's the coefficient.
If I just look at one or two.
Suppose my function is f(x)=x.
I guess in that problem, in Problem 1, I made it minus pi to pi.
Suppose f(x)=x.
So you have to integrate x times e^(-ikx).
Well, you got an integral to do.
OK, it's doable but not instantly doable.
Let me ask you some questions and you tell me about it.
So I draw the function.
The function is x from minus pi to pi.
And tell me about the coefficients, how quickly do they decay?
This is like some constant over k to some power p, and what's p?
What rate of decay are you expecting for the coefficients?
Well you say to yourself, it looks like a pretty nice function, smooth as can be.
But, what's the answer here?
The rate of decay will be, what will that power be?
One.
Because the function jumps.
The function has a jump there, and the Fourier coefficients have got to deal with it.
So the Fourier series for this is going to be, if I took 100 terms it would be really close.
And then it'll go down here to the, but it's got to get there.
And got to start, and pick up below there.
So it's got the same issue that the Gibbs phenomenon, that the square wave had.
It's got that jump.
So it'll go like 1/k.
OK, at coming back.
Could you actually find those numbers?
And do you remember how to do integral like that?
Well, look it up, I guess is the best answer.
But whoever did it the first time, well, it's integration by parts, somehow.
The derivative of this makes it real simple, and this we can integrate really easily, right?
So we integrate that, take the derivative of that.
We get a boundary term, so I don't exactly remember the formula.
But it'll have a couple of terms, but not bad.
And we'll see this 1/k.
OK, and that's the formula.
If I changed x to something else, let's see.
As long as I'm looking at number one, what if I took e^x?
Oh, easy.
Right?
e^x, yeah, let's do e^x, because that's an easy one.
Now, e^x looks like, so what's my graph of e^x?
It's quite small here at minus pi, and pretty large at e^pi.
What's the rate of decay of the Fourier coefficients for that guy?
Same.
Got to jump again.
Drops down here.
So let's find it.
Let's find those Fourier, that integral.
That's e^(x-ik).
Sorry, let's put down what I'm integrating here.
I'm integrating e^x(1-ik).
Right?
That's what I've got to integrate.
The integral is the same thing divided by 1-ik.
I plug in the endpoints, minus pi and pi, and I've got the answer.
And I divide by 2pi.
Yeah.
So that's a totally doable one.
Let's plug it in.
1/2pi.
And then the denominator is this 1-ik, yeah, well there you see it already, right?
You see already the 1/k in the denominator.
That's giving us the slow decay rate.
And now I'm plugging in x=e^pi(1-ik)-e^-pi(1-ik).
So as k gets big, this is slow decay.
Now, what's happening here as k gets large?
Oh, k is multiplied by i, there.
So this e^(pi*ik) is just sitting, it may even be just one or something, or minus one, or whatever.
Is it?
Yeah.
k is just an integer, this is k*pi*i, that's just minus one to something.
So all that numerator is minus 1one to the to the k'th power or something.
Times e^pi.
So e^pi pi there.
e^-ik, at e^(ii*pi*k) is just minus one.
And this is maybe e to the minus pi times the same e^(+i*pi*k).
I think that's minus one to the k. Well, wait a minute.
Maybe they're not both, whatever.
It's a number.
It's a number.
That's just of this size.
There's the big number, there's the small one.
And divided by that, that's the thing that shows is yes, there is this jump.
Right?
OK, that's a couple of sets of Fourier coefficients.
You could ask yourself, because on the quiz there'll probably be one, and I'll try to pick a function that's interesting.
And I mean, I don't plan to pick xe^x or anything.
Yeah.
OK, good.
Yes, thanks
[INAUDIBLE] Fourier series has twice the energy as another, what's that mean?
I don't know.
I guess.
Hm.
Maybe we're talking about power, and things like if we were dealing with electronics, I guess I would interpret that energy in terms of power.
So that's what I'd be seeing.
I'm not thinking of a really good answer to say well why is that energy equality, but it's really useful.
You know, so much of signal processing, and we'll do a bit of signal processing, is simply based on that energy or equality there, and the moving into frequency space.
And convolution.
Actually, they would call it filtering.
So we'll call it filtering when we use convolution.
But it's pure convolution.
Pure linear algebra, for these special bases.
So, OK.
I could try to come up with a better answer, or more focused answer than just to say power or, to use an electric power word is just one step.
Yeah, thanks.
Yes
[INAUDIBLE]
4.3, eight or nine.
Let's just have a look and see what they're about.
OK, just some regular guys.
Yeah so that, OK, let me look at, you want me to do 4.3 eight?
[INAUDIBLE]
Ah, yes.
Good, good question.
OK, so 4.3 eight gave a couple of vectors, a little bit like the ones I did this morning.
I mean here the c and 4.3 eight has a couple of ones, and this morning I just had a .
But no big deal.
Now, yeah.
So if two vectors are orthogonal, are their transforms orthogonal?
I think, yes.
Yes.
The Fourier, so the, yeah.
So maybe this is worth a moment, here.
Yeah.
Because what do we, let me write down the letters for my question, and then answer the question.
OK, so suppose I have two vectors, c and d. And they're orthogonal.
And then I want to ask about their, if I multiply by the Fourier matrices, are those orthogonal?
That's sort of the question.
So here the vectors in frequency space, here they are in physical space.
I don't mind if you started in physical space and went to frequency with F inverse, same question.
Does the Fourier matrix preserve angles?
Does it preserve angles?
Do matrices, and F wouldn't be the only one with this property, do they preserve length.
Right?
If you preserve length, do you preserve angles too?
Preserve length, you're just looking at one vector.
We know that we preserve length, and how do we know that?
Let's just remember that.
So here's the length question.
And this'll be the angle question.
So this is the energy inequality, coming back to that key thing.
This is c bar transpose c, and over here we have (Fc,Fc) and this is what I did this morning.
So that's c bar transpose F bar transpose F c, right?
That's why the bar as well as the transpose because we're doing complex.
OK, let me make a little more space.
And what did we do this morning?
We replaced that by, well I wish it were the identity.
But it's a multiple of the identity, that's what matters.
So this was N, this is the identity with an N, c bar transpose c. So the conclusion was that this thing is just N times this thing.
Well, and times this one.
I'm expecting oh, but over here we got a zero.
So my N is going to wash out.
OK, let's just do the same thing for angle.
This is the key energy equality for length.
And all I want to say is, this Fourier matrix, like other orthogonal matrices, is just rotating the space.
It's sort of amazing to think you have one space, physical space, and you rotate it.
I'll use that word.
Because somehow that's what an orthogonal matrix does.
Well it's complex N-dimensional space.
Sorry, so it's not so easy to visualize rotations in C^N, but it's a rotation.
Angles don't change, and let's just see why?
The inner product of this with this is c bar transpose, F bar transpose, because I have to transpose that.
Times Fd, what do I do now?
This one was c bar transpose d, with zero.
You see you have it?
Again, this fact that this rotation and inverse, and the transpose is the identity, apart from an N. So again, inner products are multiplied by N. Not only the length squared, which is inner product with itself, but all inner products are just multiplied by N. So if they start zero they end up zero.
Yeah.
That was your question, right?
Yep, right.
So if I have a couple of vectors, as I guess that problem proposed, it happened to propose two inputs that happen to be orthogonal, then you should be able to see that the transforms are, too.
Yeah.
Yep.
Can I ask you about question two, was that on the homework?
Problem two in Section 4.3?
Was it?
All I want to say is that's a simple fact.
Let me just write that fact down so we're looking at that fact.
If I look at this F matrix, it's so simple but it leads to lots of good, it's just a key fact.
But if I look at my F matrix, am I looking at rows here?
Yeah, I happened to look at rows, columns would be the same, it's symmetric.
So here's row one, here's row two, I'm sorry that's row zero.
This is row one.
And let me look at row N-1.
This is w, w^(N-1), this was w^2, this is w^2(N-1).
And so on.
Everybody's got the idea of the, and then all these in between rows.
Two, three, et cetera.
And the question asks, show that this and this are complex conjugates.
That that row and that row are the same, except conjugates.
So if we look at the row number one in F, we'll be looking at row number N-1 in F bar.
Now, why is that row the conjugate of that row?
Why is this the conjugate of this?
Why are those two conjugates?
So I'm asking you to explain to me why w bar is the, why is the conjugate of this, this?
So it's just one more neat fact about these important, crucial, numbers w. And so how do I see this?
Well, this is w, this is one way to do it.
This is w^N times w^-1.
And what's w^N?
One.
Everybody knows, w^N is one.
So I have w, so I'm trying to show that.
Well, we know that this is true.
So we know that the conjugate of w, right?
There's w. Here's its conjugate, and it's also the inverse.
This w is some e^(i*theta), this is some e to the the i some angle.
Then always, it's sitting on the unit circle.
So its reciprocal is also on the unit circle.
The reciprocal has e^(-i*theta), so it's just the conjugate.
That's a great fact.
That's a beautiful fact about all these w's.
And their powers.
Conjugate and inverse the same.
Right.
So that was the key to problem two.
I don't think I asked you to go through all the steps of problem three, but just in case you didn't read problem three, let me tell you in one tiny space what happens.
In problem three you discover that the fourth power of F is the identity.
Except there is an N squared, because from two F's we got an N, so from four F's, so that's another fantastic fact about the Fourier matrix.
That its fourth power, it must be somehow, the Fourier matrix is rotating, yeah.
Somehow, I don't know, how do you understand that the Fourier matrix is, its fourth power brings you back to the identity.
Apart from, we just didn't normalize it.
So that we wouldn't, to avoid that N squared, so we had to use it.
Yeah.
That's pretty amazing.
Pretty amazing.
So if I had normalized it right, if I'd took F over the square root of N, that's the exact normalization to give me a to put the N's where they belong, you could say.
Then the fourth power of that matrix is the identity.
Yeah.
New math and new applications keep coming for these things.
I'll tell you, actually.
You could tell me eigenvalues of this matrix.
Yeah, this is a good question.
What are the eigenvalues of a matrix whose fourth power is the identity?
[INAUDIBLE]
Yeah, one.
What else could the eigenvalue be?
If the fourth power of the matrix is i, what are the possible eigenvalues?
So let me, so this matrix is, can I call it M for the, or maybe u for the matrix F?
So if u^4 is the identity, what are the eigenvalues?
What could, well of course u could be the identity, but it's not.
It's a Fourier matrix.
So the eigenvalues could be one, what else could they be?
Minus one is possible.
Because minus the identity, that'd be fine.
i, and minus i.
Four possible eigenvalues.
And when the matrix reaches, I think if you put the four by four Fourier - I don't know.
If you, say, put the four by four Fourier matrix into MATLAB, and see what you get for eigenvalues, I've forgotten whether you get one of each of these.
Or whether somebody's repeated at the level four, but then go up to five or six you'll see these guys start showing up.
Different multiplicity.
Right?
The 1,024 matrix has got these guys.
Some number of times, adding up to a 1,024.
Right, yeah.
So it's quite neat.
Now, here's a question, which I actually just learned a good answer to.
What are the eigenvectors?
What are the eigenvectors?
You could give a sort of half-baked description.
Because once you know the eigenvalues.
But to really get a handle on the eigenvectors, that's has been a problem that was studied in IEEE transactions papers.
But not really the right, not a nice specific description of the eigenvectors.
And somebody was in my office, this Fall, a guy, post-doc at Berkeley who's seen the right way to look at that problem and describe the eigenvectors of the Fourier matrix.
So that's, like, amazing to me that such a fundamental question was waiting.
And turned out to involve quite important, deep ideas.
OK, ready for one more for today?
Anything?
Or not.
OK, so I hope you're getting the good stuff now on the Fourier series.
Fourier discrete transform.
Please come on Friday, because Friday will be the big day for convolution.
And that's the essential thing that we still have to do.
OK, see you Friday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so I feel I should have brought pizza instead of math problems.
I mean, but this is our final, final, hour.
Open for any questions, any discussion about Fourier topics.
Looking toward tomorrow evening's exam.
And, well, hoping you'll find the whole course useful at many times in the future.
So, somebody emailed me that the posted solution, so I just posted a quick solution to the un-collected homework ten, and so I haven't even looked to see what did it say for this question.
But what is the answer?
If I convolve the delta function with the delta function, what do I get?
Delta, right.
If I convolve anything with delta I get delta because I'm over in the other space.
I'm multiplying by one.
Yeah, so if I use the convolution rule to go over to the other space, where this becomes just one times one I'm getting one, and transform back and I've got delta.
Well that question won't be on tomorrow's exam.
But I thought we could deal with that fast.
Now I'm open to questions about any topic.
Homework or not.
Yeah, thanks
[INAUDIBLE]
I could try.
Did I make up the exam?
Probably.
Yeah, I'll recognize it.
OK, alright, I'll read it out.
OK, so this an exam from 2005, not that far back.
OK, so it has some questions.
Shall I tell you what the whole question is but it looks like this Part D is sort of separate from the others.
But why not, this gives us some questions to think about.
So if I just read out the questions.
One was, find the coefficients of, the function is e^(-x).
That would naturally occur to me as a function that you could, so from zero to 2pi.
And then periodic.
Always good to graph it.
So will e to the minus x look like?
It'll come down to, much steeper than that, but that would do it.
So that's e^(-x).
Here it's one, and here it's e^(-2pi).
And then repeat, repeat, repeat.
So find its Fourier coefficients.
I'll just say what the question is and you can ask me.
And then what's the decay rate.
Oh, maybe you can tell me the decay rate.
Even before you tell me the coefficient.
What would the decay rate be if I expressed this as a combination of Fourier terms, harmonics.
How quickly will those coefficients drop off?
So 1/k, only.
That's right, 1/k, because this function has a jump at zero.
Well, at zero and again at 2pi at every one of those points.
It has a jump, I guess, of about this distance, between one and that number.
Which would be the jump that we would see of course every time.
Yeah, so the coefficients would decay like 1/k.
Then it asked to compute the sum of those coefficients squared.
OK, so how can we find the sum of those coefficients squared even before I know what they are?
Well, you know the answer, right?
How am I going to do this problem?
Yeah, so I'm looking for the sum of the squares of coefficients and then there's this magic trick.
What is it that helps me to find that sum of squares?
How do I go about that?
I connected to the integral of the function square.
It can go zero to 2pi of the function square.
And you remember the reason, and I've left a little space for the fudge factor.
2pi, or 1/2pi, we can figure out what it is by taking a simple case.
Yeah, why don't we take a simple case to figure out should there be a 2pi or a 1/2pi or what?
What's a simple case here?
Suppose I take f(x)=1.
Suppose I take the function f(x)=1, then the right side.
So if f(x) is one, the right side would be 2pi, I guess.
The integral of one, and the left side would be what?
What would be the Fourier coefficients of the constant function?
Well just c_0, the only one we would have would be c_0, the constant term.
The c_0*e^0.
And it would be one.
So I'd have a one there, when k is zero, and otherwise I wouldn't have anything.
So this would be one squared, this would be a one.
So I do need to divide by 2pi.
Do you agree with that?
To make it correct I'd better have a 1/2pi to get that.
To be right.
OK, and then this gives me the answer then, I just do this integral, e^(-2x), then f is e^(-x).
So f squared would be e^(-2x), which I can certainly integrate and get some expression.
Yeah, OK.
So that's a straightforward part.
And now no to the Part D, which you asked about which, is oh wow.
OK, so now I'm asking about an equation u, du/dx, plus u(x) equals a train of deltas.
So delta(x) made periodic.
Can I write it that way?
So this is the periodic case.
So I'm looking for a periodic answer.
So I'm in the Fourier series world when I take, I'll be looking for coefficients, not the integral transform.
OK.
So alright, so I'm making it clear.
So how do I proceed now, with such a problem?
So I've mentioned this morning that a problem like this could appear.
So I'm happy to discuss it.
What do I do?
Transform.
Absolutely.
Take Fourier transform.
OK, so what happens when I take Fourier transforms of u?
Well, I guess so we're doing periodic, so we're going to go to coefficients, right?
I'll call them, shall I call, shall I think of u(x), shall I just call its coefficient c?
It's the only function and I have around so I might as well use c for it.
So here's the function, and Fourier takes this to the set of coefficients, OK. And the point is that we can separate - I mean this is the whole reason why Fourier's so great.
That we can do each frequency separately.
So let's see then.
So this has coefficient c_k, and what are the coefficients, what's the k'th coefficient, so this is all the e^(ikx) term.
What's the k'th coefficient of the derivative?
ikc_k, right?
And what's the k'th coefficient of the delta function?
One, right?
Is that right or is there 1/2pi?
Is it 1/2pi?
OK, 1/2pi.
Yeah, I guess that's right.
Because the coefficient, I would have to take the integral times, yeah, the integral with the delta function will give me the one, and then there's a 1/2pi.
Right, OK.
So now I've got the answer.
Now I have this equation at each k. Following each eigenvector, following each separate frequency.
And it's easy to do.
It's just 1/2pi.
And there's one plus ik, I think.
OK, and now those are the c_k's.
So I think maybe this is it.
Sum from minus infinity, I know what the c_k is.
2pi(1+ik), times e^(ikx).
That's my u(x), I think.
Had to be, right?
So I just did the sort of automatic steps.
When I made up this exam, I don't know whether I was thinking I could find out can I actually do that sum to produce some function u(x) that I am familiar with.
I'm not sure.
Anybody reading the question, so the question says solve this differential equation.
Doesn't say what it means by solve.
Anyway, I've done it.
I guess.
Shall we agree that I passed on this question?
If we knew some tricky way to do this addition, but I don't immediately see it.
So that's good.
OK, you're welcome.
Thanks.
OK, so that was a very good question to ask.
Has it got, it's got the method of how do you deal with a differential equation.
By the way, if this could be a difference equation, too.
That could be u(x+h)-u(x-h).
Shall I just emphasize that that wouldn't have been any harder?
u(x+h)-u(x-h)/2h, let me take, for example, plus u equals delta.
This is like I've taken a center difference there, instead of a derivative.
How would you write a solution to that?
Same method.
So what would the method be?
I'd take transforms.
So here this would have coefficient c_k, this would have coefficients 1/2pi.
And what would be the coefficients of these guys?
Well, there's a 1/2h.
So what are the Fourier coefficients of u(x+h)?
Just to see that all these things, I've got constant coefficients here.
Why shouldn't Fourier work?
So if it's u(x+h), so there's an e to the something factor, right?
When I've shifted a function then in the transform I see an e to the i thing.
So let's just see.
If u(x), so if u(x) is the sum of c_k*e^(ikx), then u(x+h) would be the sum of c_k*e^ik(x+h), right?
No.
Nothing surprising there.
So now I'm seeing the factor.
There's c_k, it's multiplied by the factor e^(ikh).
So this is the same c_k, but an e^(ikh), from that one.
And this would be the minus, and this would shift the other way.
So it would be the same c_k, and probably e^(-ikh).
If I did that right.
Yeah.
Anyway, once again we have still linear problem.
This is c_k times something equals 1/2pi.
By the way, this thing is what I would call the transfer function.
You see that language?
We haven't used that language much.
Or at all, it's sort of more of a systems theory, control theory word.
But hey, we're always doing the same correct thing.
That's the thing that transfers the input to the output.
So maybe the transfer function is one over.
Maybe I include the division in the transfer function.
So finally I get c_k is this 1/2pi guy, divided by whatever that was, 1/2h times that number, minus 1/2h of that plus one.
Whatever is multiplying c_k there.
It's a transfer function.
Yeah.
So there's another word which we could have, and should have, used before today.
But here it is.
OK, so that's some thoughts about that question.
Any other direction to go?
[INAUDIBLE]
Yes.
go ahead.
Yeah
[INAUDIBLE]
Lab problem 12, OK. 4.5, Fourier integrals, number 12.
Oh yeah, maybe that was also.
A question was raised about was it right on the posted solution.
So maybe not.
Let's see what we want to do.
Yeah I just thought that was pretty - I could remember the day I thought of this equation.
Integral of u minus derivative of u equal delta(x).
I thought that's kind of a cool looking equation.
But I don't know that I've ever solved it, before.
So integral of u, actually yeah, I look cheerful, but so I'm finishing the fourth edition of the linear algebra book.
Introduction to Linear Algebra.
So I love writing.
I've done all the writing and now comes the only horrible part, solving all those stupid exercises.
So that, you may think of me in sunny Singapore or somewhere, but what I'll be doing is not on the beach.
It's in an office somewhere, doing Problem 1.1.1, and two, three, it's not life.
I mean, it's.
But, OK.
Anyway, that looks like a pretty good equation.
And certainly I'm going to transform it.
So I can see a little question coming up.
So am in the Fourier integral world, yeah.
So I should you use this notation.
u hat of k, right?
Over ik, is that the right thing when I've integrated?
Yeah.
That's made it smoother.
That's dragging the function down.
And I guess the one point that's a little tricky there is always, are you dividing by zero.
Because zero is one of the frequencies.
And so I sort of need u hat of zero to be zero for this.
So it's looking slightly dangerous at k=0, but otherwise it's certainly improving things by speeding up the decay rate.
And then this would be minus ik u hat of k, as we just said.
And this would be the one, maybe it's a one in the Fourier integral world, where the 2pi went in a different point.
So I think that's, yeah.
So I would now just, the usual thing was the transfer function.
1/(ik)-ik, so that's my transfer function.
That's multiplying u hat.
Maybe I can simplify that by putting it all over ik, one minus ik.
Is it plus k?
Does that look good, or does that look bad?
k squared?
Does that look better?
So I'm wondering, so over here is k squared over ik, is that the same as the minus ik?
Yeah.
Yeah, bring it up and the minus i squared is one and I've got k squared.
Yeah, so this looks good.
So that's then dividing by it gives me because, I'm dividing the one.
So u hat k is one over this.
So it's the transfer function.
One plus k squared.
OK.
I don't know whether I had in mind to go any further than this.
Did you get this far, or what?
[INAUDIBLE]
It said that?
Oh jeez.
I bitterly regret saying these things.
Yeah, OK.
I don't know u(x).
Did anybody have any luck?
You did.
Did I say it with a derivative?
Oh one, over one plus k squared, oh yeah.
One over one plus k squared, we know what to do.
And then i k will take its derivative.
So what do we do if it's one over one plus k squared?
That's the one where, that was the guy which was the e^(-x), right?
Oh, it's just one side.
No, two sides, isn't it?
Two sides, yeah.
Yeah, we just got a corner.
Yeah, OK. That's looking good.
And now maybe I need to divide by two.
a is one here but there is a division by two.
Yeah, that looks good.
OK, and now I take - This function, its transform has that.
And now if I want to multiply by ik I have to take the derivative.
So I believe with your help here, that the derivative of that is the same.
Can I just take the derivative while we're looking at it?
I mean e^x is the most - you know, that was created to take its derivative, right?
So its derivative is itself.
The derivative of this is a minus.
So I think that maybe that's the answer.
e^x over two, coming up to 1/2, I guess.
And then - oh, well the picture won't look, because of that minus sign.
Yeah.
Huh.
Good, because that minus sign it's minus e^(-x) over two.
So it starts at minus 1/2.
So there's a jump of one.
Or drop of one.
Is that right, there should be a drop of one in u hat?
Probably.
Yeah, yeah.
The integral will be smooth, at zero where the delta function's hitting us.
I have a minus sign and a derivative.
But yeah, doesn't that look good?
If the derivative has a delta, the function has a jump.
And with that minus sign, and with one delta the jump should be a drop of one.
Which is that.
Yeah.
And the integral minus the derivative should be otherwise zero.
So again this is not unrelated to exam questions.
I did all this stuff, and I really should check, did I get it right?
Does that solve the differential equation?
I mean, you could say of course it does if you took all these steps right.
But it's wise to check.
Plug it in.
OK, it has the drop of one.
So it deals correctly with the delta because, the derivative has a delta.
And things match.
And now what else do I have to check?
I have to check out all the other points, right?
I've just checked that yep, this answer is good at the jump.
The tricky point.
But now what about all the other points?
So where the delta is zero, I should just check that the integral of u, so what is the integral of u?
Let me just check for the points left of the origin.
So I'm just going to look at this part.
Yeah, x negative.
So its integral, what's the integral of e^x over two?
I guess e^x over two, right?
Minus?
And what's the derivative of e^x?
Oh, the derivative is also e^x over two and I get zero, or x negative.
As I want to.
Because the delta is zero in that left side.
And similarly on the right side.
It should be OK. And then the key point was that the jump was right.
Yeah, so thank you.
That's good.
Yes please
[INAUDIBLE]
What, sorry?
Why did I take the derivative?
What will be the derivative of this?
Yes, there is.
That's right.
What would be the derivative of this?
Huh, yeah, good question.
What's the derivative of this answer.
So it's that, on the left.
and then there's a delta function, a minus delta, right, because it's dropped down.
And then the derivative of that is e^(-x).
Yeah, so if I graph the derivative, cool.
I graph the derivative, it looks like the function.
It has a spike going down there.
At the origin, and then the derivative here is positive.
Right?
This function's coming up.
It's e(-x) over two.
It's positive, it's there.
Yeah.
Oh wow, that's a nice graph.
So that's the derivative, this is the graph of du/dx.
Yeah, thanks.
And similarly I could graph the integral, the integral - well, would you want to see the integral?
Probably not.
Maybe, yeah.
The integral would probably look pretty much just like that but without the delta, yeah.
Cool, isn't that nice?
It's artistic.
That's the derivative, and the integral is the same thing without the delta.
And when you subtract, you get the delta.
OK, you sure you guys don't want to come to Singapore and help with these problem solutions?
There's plenty for everybody.
OK. Alright, so another question.
Yeah.
2005
[INAUDIBLE]
OK, alright.
Yeah, which one?
[INAUDIBLE]
OK, I'll just read out the whole question.
Yeah.
So this is 2005, Problem 3.
OK it's a half hat function.
This is Fourier integral, because it's on the whole line.
And it's half a hat for between zero and one.
The function is coming down.
And then otherwise it's zero.
So its graph, done.
Graph its derivative.
OK, let's graph the derivative of that function.
Derivative is certainly zero along there.
Then the derivative is minus one here.
And then the derivative is zero.
So the derivative looks to me like it's that, OK.
So yeah, that function - is that right?
No.
It's not right.
There's a delta.
That was the tricky part of the problem.
Right, there's a delta in the derivative, right here because of that jump.
There's a delta.
OK. Good.
Now, is that better?
That's good.
OK. What's its transform?
The transform of this derivative.
What's the Fourier transform?
Well, I guess we got two parts.
So this is my function u'(x).
The derivative.
Now, what's u' transform?
So it should be a function of k, now.
Alright, what do you think?
So it's the sum of two things.
That delta, which Fourier transforms to one.
And this guy down below between zero and one, which was like the one we did today, we've got the integral from zero to one.
Now it happens to be a minus one e^(-ikx)dx.
And that, of course, we can do.
OK Oh, there's solutions here
[INAUDIBLE]
Yeah.
So OK, so I think we're doing alright.
This expression is familiar.
It's (1-e^(-ik))/ik.
Is that right?
We have a minus.
Is the minus, yeah, I think the minus looks good.
And yeah, at least, that's here and it's probably got the minus signs correct.
So why was it?
I plugged in x=1, and that's why I got an e^(-ik).
And then I plugged in x=0 and that's where that one came from.
OK, so but your question was about - that was it?
Oh, I see.
OK.
Right, and then the next part asked about the transform of the original guy.
The transform of the original guy.
Now, what would be the transform of the original guy?
The original u.
Right, OK, yeah.
The reason I'm sort of stuttering is that if I'm going to integrate - I mean, what are you going to tell me?
What's a quick way to find the transform of the original u now?
Divide by ik.
Divide by ik.
And then I'm all ready to do that except I'm worried that k=0.
But it should come out alright, now.
So I have to hope that this thing comes out to be zero at k=0.
Can you see that it does?
What is, yeah?
This is a good point.
What does u hat at zero represent?
Have you thought about that?
If I look at the Fourier transform, which I've got here.
So let me write what I've got here.
I've got here that Fourier transform of this function.
And if I take the Fourier transform of any function, and I look at zero frequency.
What am I seeing?
The average, right.
I'm saying the average.
What's the average value of that function?
We never thought about that.
But that's fun.
What's the average of this guy?
If I integrate over the whole line.
Do I get zero?
Yes.
Because I get the integral of the delta part gives me a one.
But the integral of this box part gives me a minus one.
So the integral, so this does equal zero at k=0, and I can safely do that division and get, so now u hat of k is this expression divided by ik.
So it's one minus this thing.
I'm just copying.
Minus ik over ik, all that divided by ik.
And I can, of course, maneuver that some more to make it look better.
OK, these are good questions, because it's giving me a few functions to do the standard rules.
Shifting, integrating, differentiating.
Oh, here's a Christmas present.
What does that mean?
Is the convolution of this function with itself the whole hat?
What is that, a Christmas present?
Is the convolution, the convolution of that with itself.
So first point, we don't want to compute a convolution.
You may have noticed, you have not computed convolutions or some things like this.
It's not a lot of fun.
Because you have that integral to do.
And you'd have to separate out the parts where it's this and the part where it's this and the parts where it's that.
It's not nice.
Much better to multiply in the transform domain.
Much better.
OK, so if I multiply in the transform domain, I don't think I get the Fourier transform of the hat.
Of the whole hat.
But can you give me a convincing argument for why - this is here I'm in the transform domain and I'm going to convolve with itself, so I'm just going to square it.
With the decay rate, what's the decay rate as it is now?
The decay rate is, yes.
The decay rate is a good key.
So what is the decay rate right now?
Here's a function with a jump.
So I know that even though has a slightly messy looking transform, I know the decay rate.
It's what, with a jump is 1/k.
OK, so this must be, and I sort of do see a k down here, so that's sensible.
OK, now, if I convolve, then I multiply in this domain.
So that would give me the k squared, as you said.
So could that be the transform of the whole hat?
Does the whole hat have a - ooh, could be.
I'm pretty sure the answer's no.
Yeah, no way.
Yeah, I think that that's the best answer.
I've lost the reason.
Because I'm getting the k squared to k rate.
If I convolve something with itself, that would give me a k squared, and the decay rate for the transform of the hat is a k squared.
But they wouldn't be the same, yeah.
OK, yeah, I won't.
Yes, please
[INAUDIBLE]
Right.
It's just convention.
The difference was that that was the Fourier integral problem.
Where we put, you might have noticed that the 2pi goes on the other - and this was the Fourier series problem.
That was its only difference.
So it's just a convention.
And maybe other people would choose a different convention.
So that is a slight wiggle in the presentation that the 2pi in the Fourier integral section got moved to the transform in the other direction.
Yeah, good.
Yes, thanks
[INAUDIBLE]
Oh, OK.
I'm really just going by this rule that if there is a jump in the function, and nothing worse, then the decay rate is 1/k, and if there's a jump in the derivative, as there would be here.
So that has a jump in slope, then the decay rate is one over k squared and so on.
But those are the main cases.
Yeah, so I'm not using anything deep there.
And by the way, I realize here that one MATLAB problem that I didn't assign this semester but it's quite fun, is to actually see the Gibbs phenomenon.
We know the terms in the Fourier series, with a jump.
And if you computed - make it the periodic case, if you've computed the terms in the Fourier series they would stay near here and then I'd see the famous Gibbs stuff here.
So that - you know, it's quite pleasant to see the printout of the Gibbs phenomenon.
Not getting any shorter, just moving closer to the jump as you increase the number of terms.
Yeah, it's quite interesting.
But couldn't do everything.
Yeah, ready for more.
Any questions?
Well, these are good.
Yes, thanks
[INAUDIBLE]
With cyclic convolutions.
OK, what could I do with cyclic convolutions?
Let's see.
What should I do with cyclic convolutions?
I better make a little space.
Let me make a little space and think.
Anybody got a suggested problem for a cyclic convolution?
I mean, one type of problem would certainly be if I gave you a cyclic convolution and asked for - I mean, the direct way would be to say OK, what's the cyclic convolution of (1, 4, 2), cyclically with (2, 1, 3) or something.
But you could - that's just a calculation, so you would get that.
I won't discuss that further.
Now suppose I ask it with the unknown, the thing that I don't know here?
So let me do that.
Shall I say three, yeah.
(3, 1, 1).
Cyclically convolved with some unknown (x_0, x_1, x_2) equals some right-hand side, what should we take for the right-hand side?
(1, 1, 1), for example?
OK, I'll erase the top one.
OK, how would you go with that?
So that's three by three, three unknowns.
It's linear, so somehow I could write that as three equations in three unknowns.
The question is how do I get some insight into this.
And when I'm seeing a convolution, what's my immediate thought?
Transform.
That's what this month is all about.
November.
So I'll take the DFT of everything.
So what's the discrete Fourier transform of (1, 1, 1)?
Up there is going to be the transform.
So let me call it x_0 hat, x_1 hat, x_2 hat.
Just to emphasize.
So n is three here, in this problem.
Or maybe c_0, c_1, c_2 you might prefer.
Whatever.
And now this is going to be a multiplication, so it's in MATLAB notation will be a dot star, right?
And so now I have to - these are the known ones, so what's the discrete Fourier transform of (1, 1, 1), the cyclic convolution of that?
It's (1, 0, 0) is it?
Or it's (3, 0, 0)?
Is it three - I guess when I asked that question, you're open to two answers.
You could either be going from x space to frequency space or frequency to the other.
Because I haven't told you what space we're in.
OK, let's go with that one.
How did you get to there?
You multiplied by the three by three Fourier matrix, is that what you did?
Aright, so now can you do that here?
So separately I have to do a little multiplication of the three by three Fourier matrix.
One, one - sorry, let me get it right.
So I'm going to transform this.
Make a little space here.
Transform that.
So it's 1, 1, 1, 1, 1 and this is w, w squared, w squared and w to the fourth, which is the same as w. OK, and now I want to multiply that by (3, 1, 1).
And get the answer there.
OK, so what do I have?
Five.
And what is that other one?
Oh, boy.
What is three plus w plus w squared?
Anybody know that one?
Well, I do know what one plus w plus w squared is.
It is zero.
Why is one plus w plus w squared zero?
So I claim that if I have three plus, I think the answer there is two.
And here I think I've also got three, w, w squared, I think if it was a one the answer would be a zero.
But it's a three, so I still have two there.
I think that's probably right.
Can I just remind you why?
There's one, there's w, and there's w squared and they add to zero.
Yeah, yeah.
For many reasons.
That's a very, very, handy property.
That the sum of the nth roots of one add to zero, OK, so now where I am I?
So I got (5, 2, 2).
So I've transformed.
Now what do I do next?
I divide.
So this is telling me that this is three separate equations.
There's three, two, x_1 hat is zero.
Right?
No, x_2 hat.
And 2x_2 hat.
Sorry, one was right.
And 2x_2 hat is zero.
I'm just doing the multiplication as I'm supposed to.
In the other space it's at each, for each component separately.
So now I know x_0 hat is 3/5, and the others are zero.
So now I know what these numbers are.
Oh, yeah.
(3/5, 0, 0).
And now what?
I mean, that statement's clearly true, right?
Point star means five times this gives me three.
Two times zero gives me zero, two times zero gives zero.
So I'm golden, but I'm not finished.
So that's not x, that's its transform.
So what do I have to do?
Go back.
OK, so now can you tell me what this is?
Up to a factor of 1/n, or n, which we may have to figure out separately.
So apart from this possibly appearing, and probably appearing, give me an idea of what's the inverse transform of that?
(3/5, 3/5, 3/5), good.
3/5, all three times.
OK, and now I can check to see if it's right.
And again, I have not attempted to get the 1/n's right, I've just waited to the end.
So now I'll just do this convolution and see if I really do get (1, 1, 1).
So can we do that convolution?
What's the typical, say, this guy.
Where does that come from in the convolution?
It comes from this, times what?
This.
That's the constant, right?
And that's the constant.
Contributes to the constant.
And what other products contribute to the constant?
So we're really seeing - thank you for this question.
So this one multiplies which one?
The answer is, this one multiplies this guy.
You may say don't worry me about it because they're the same.
But it's nice to know.
So this one multiplies this one.
And this guy multiplies this guy.
And we'll remember in a minute why.
And then you add them up, that's what convolution is.
So I'm getting 9/5+3/5+3/5 is 15/5, so that's three.
So what's your conclusion here?
I should divide by three, right?
So I should divide by three.
So that's easy to do.
Those will all be one.
Yeah, OK.
So that's the right answer, right?
And then I check that it's right.
So let me just close by remembering why was it - so I'm doing up just a forward convolution here.
Solving the equation was a deconvolution, figuring out that x.
But now that I've got it and I'm just checking, that's just do it forward and see if you got the right answer.
And so we're just remembering what it means to do a convolution.
So you remember the point about convolutions.
That if I'm convolving (a, b, c) cyclically with (d, e, f), that's a constant term.
Just tell me again, now, what's the constant terms in this?
It's ad plus b times what?
This is the first term in the convolution.
You remember that, the day we brought convolutions into the course?
What does b multiply?
f, why f. Because b is the coefficient of w, if I'm taking polynomials, this is a plus bw plus cw squared, and this corresponds to d plus ew plus fw squared.
And I'm looking for the constant.
So I get an a times ad gives me a constant.
bw times what?
times f w squared gives me the w cubed.
Which is the one so that bf here and then there's a ce.
Because c w squared multiplying ew gives me the w cubed, the one which folds in.
Yeah, OK, so anyway.
That's a quick reminder of cyclic convolutions.
That was a good question to ask, yeah.
You know, I fully realize this especially this third topic, lots of things like this.
Straightforward if you have lots of time, but we had to keep moving.
So remembering what cyclic convolution was is a good chance to do it.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So this is the equation that we-- this inviscid Burgers' equation, no viscosity is the one we've learned about for shocks appear, fans can appear and I want to do more today with exact solutions.
So today is a little bit more analysis than numerical solutions.
So while doing something, sort of PDE rather than numerical PD, well it's natural to include Burgers' equation with viscosity.
So that's got the extra term there and if I can, I would like to say something about two other important equations, nonlinear.
One has a third derivative on the right-hand side, so it's a dispersion rather than a diffusion.
And that equation is named after Korteweg-de Vries, KDV it's often called.
I'll just put that down.
KDV is the shorthand for that equation and it comes up in water waves, one-way water waves actually, along that shallow channel.
It was actually seen by somebody riding along next to the wave and the wave kept its shape and so this equation has had a period of great fame in some way.
It's nonlinear of course, but it was discovered that there was a really clever way to solve it.
Actually, we will solve these equations even though they're nonlinear, that won't be too hard.
The clever ideas that went into solving that one, I can describe a little bit, but not in full detail.
And then we'll connect also to Hamilton-Jacobi type equations, which are very closely related to this in one dimension and in fact, they'll come into it.
OK, so those are the equations and then next time we'll come a little more about finite differences for those equations and a lot more about the level set method, which is more connected to Hamilton-Jacobi and is quite a remarkable idea.
OK, so that's for next time.
This is for this time.
So what I have in mind for this time is to begin, can we solve our equation starting from a source, starting from a delta function, a point source?
What's the solution look like?
And I guess when I thought about it, I can see several ways to get at the answer and it's a very worth it example to do this from a point source.
It has a natural interest and then it comes up in many other ways.
So how we going to get the answer?
Well, I can do it in terms of fans and shocks.
Let me just proceed informally at first.
The delta function is like a step up followed by a step down.
Well, a very big step up and an immediate step down, right?
So it's the extreme case.
And remember that the step up-- for the step up you remember the Riemann problem?
Going from a lower to a higher one.
That's the case where the characteristics separate, leave an open space that has to be filled in with a fan.
So I would expect the solution to start with a fan and then I'm imagining that as the limit of an initial function like this.
So that's step up, we'll start a fan going and then this step down will start a shock going.
Because here I have u is zero and here u is something big and that drop down means that the characteristics from here will immediately overtake the characteristics from the zero, from starting from zero.
Have zero speed, the xt-plane-- they're just going up.
The characteristics that start because of this thing are going to the right, so I'll have a shock.
So I'm expecting a fan and then a shock based on the previous lectures.
OK, and I'm going to figure out what fan it is and what shock it is and where the shock is.
Now what I have on here is an exact formula for the solution.
You might say, why didn't you tell me that before?
And I will plug in the initial, it comes from the initial function and from a minimization process and it's not obvious.
That's a funny way to describe a function of x and t, but of course it does.
x and t appear here, y is the parameter and I'm minimizing with respect to y and that will give us-- well, it better give us the same answer.
So I'm taking this as an important case, which gives us a chance to review all the pieces that came into, all the rules that came into it.
We had the rules for a fan, which was of the form x/t if it started at zero.
We have the rules for a shock, which was that the shock speed should be the jump in f of u-- which is u squared over 2 for Burgers' divided by the jump in u.
And so that's a difference of squares, is a difference of first powers.
When I do that simple division it factors out and I get a u left plus a u right over 2.
The 2 coming from there and the difference going into the difference of squares gives me that.
OK, and then the other condition was the entropy condition that f prime-- remember, this is f prime of u-- so f prime of u, which is here, u.
So u left should be greater-- the shocks, what's the rule on the entropy condition?
That the characteristic should run into the shock.
So coming from the left they should be faster.
We have some shock line that's increasing with a speed, s, which is d-- position of the shock-- dx/dt and coming from the left the shocks should be going faster and run into it.
Coming from the right the shocks, the characteristics should be slower so that the shock line catches them and therefore, these also run into the shock.
So this is f prime at u right.
And remember here, it's just u left bigger than the shock speed, bigger than u right.
Now I'm going to go to the delta function itself, which will be cleaner and neater than this steep square way.
So we could pick any of those as the key to telling us-- this is what I'm expecting for a solution.
Here is zero.
Here is x.
So I'm expecting that-- I guess I'll graph u. u of x at some time, t. OK, so I'm graphing u at some later time, t. OK, I'm expecting the solution to be u equal zero here because it starts at zero-- well, I'm really now when I point to that figure what I really mean now is a delta function.
So this is infinitely thin and infinitely tall.
So characteristics are going to take off and well, who can say-- you might think the whole thing would break down because the delta function is in some sense, infinite at that point.
But it doesn't.
You'll see that a very sensible solution emerges here.
And it's a fan, so u is x/t I guess.
So u starts at zero-- yeah, let me draw u at some later time, t. It's linear and then it's a shock.
So this is what I expect for the answer.
I expect u to be zero, x/t up to some point, up to the shock.
Up to x is the shock position and then after that it has the value zero because the initial function hasn't changed, hasn't got the message.
So this was after x equal x of t. That's what it should look like.
I'm proceeding here sort of on the basis of what we know about the solution.
And maybe I can just mention that the text, the problems at the end of the section on conservation laws in the Introduction to Applied Math text asks about a few other initial functions.
For example, minus the delta function.
Maybe we should think through those questions.
Is the solution with minus the delta function the negative of this solution?
What do you think?
No.
This is a nonlinear equation and that fact shows up right away in the possibility that when you change the sign of the initial function, you don't just change the sign of the solution.
And other things, I could ask well, suppose I have an initial function u zero of x and I add 1 to it.
Does that add 1 to the solution everywhere?
That's a question to think about.
So these are sort of questions that are not finite differences, but differential equations.
All that remains here is to find the position of that shock.
I guess one way to do it-- OK, I can think of several ways to do it.
One way would be to use the fact that the total mass is conserved.
So the integral of u at any time, integral of that function from minus infinity to infinity-- that's the total mass, has to agree with the initial total mass, which is 1 because the initial density distribution was that delta function at the origin and that we know integrates the 1.
So one way to find this position would be the fact that the integral from zero to the place that the fan ends-- that's a capital X-- of x/t dx should be 1.
1 because it was initially 1.
So that's gives us capital X squared over 2-- over 2t as 1.
In fact, I guess I'm thinking now, this approach occurred to me sort of as I was walking to class, I'm thinking now that's unbeatable.
That tells us where the shock position is, capital X squared over 2t is 1, so capital X is square root of 2t.
OK, we could take that as a guess, if you like, and verify that it obeys all the nonlinear rules here.
Right now it just obeys the fact that the total mass is conserved, which is like the minimal fact.
But of course, we've built into this solution an expectation of fan followed by shock, which is now what we really want to verify.
So I guess, what should I check next?
I maybe should check the shock speed.
Does this solution with that shock give me the correct shock speed?
OK, let me check that.
What's the shock speed?
s is the speed of the shock, so it's the derivative of square root of 2t and what's that?
That's square root of 2t to the 1/2.
So that's square root of 2 times t to the 1/2.
The derivative of t to the 1/2 is 1/2 t to the minus 1/2.
So it's 1 over square root of 2t.
That's the shock speed for this proposed solution.
If we decide that the shock should be at that point.
Now what's supposed to happen?
What is it that I want to check?
I want to check that the jump condition is correct for this shock.
That if a shock speed is that, so the question mark is, is this-- so that's the speed of the shock as we're sort of conjecturing it to be located.
And now the question is, whether that shock speed agrees with u on the left plus u on the right over 2.
So what do I have to check?
I have to check what is u on the left of the shock if this is my answer.
Well, the left of the shock is the point when x equals capital X.
That's where the shock happened, so it's capital X over t. Square root of 2t over t. So the value of u square root of 2t over t. That's where the fan is.
That's this height and that height is zero.
So it's the average of this and zero.
So that's the jump condition.
So this is the jump condition that we're checking and is it right?
I hope so.
Otherwise we're in mighty big trouble.
OK yes, I have the same cancellation of square root of 2 into 2 and t to the 1/2 leaves me with-- yes, I'm left with that.
So yes.
Jump condition satisfied by this conjectured solution.
What else?
Well, just having a shock with the right speed doesn't mean I've got the right answer.
Doesn't mean I've got the right answer, can I just make that point because it's easy to slip by.
Let me take an extreme case where I compare.
So these comments are going to be to say that I really need to check the entropy condition.
Because I could have two problems.
One which started from initial value minus 1 jumping to 1.
So that's a Riemann problem.
And the other which starts from 1 and jumps to minus 1.
And I guess in both cases, remember that f of u, which is u squared over 2 is 1/2 on both sides of that line x equals zero.
In other words, could this solution stay at minus 1 and this solution stay at plus 1 and have a jump between them?
And same for this.
This solution could stay at 1, this could stay at minus 1 and have a jump.
And now of course, in this case the jump will be upwards from minus 1 to 1.
In this case it'll be a drop from 1 down to minus 1.
And I guess, I think this one is OK with the entropy condition and this one is not OK. Let me say again what I'm meaning.
The fact I've chosen this simple example with f of u equal u squared so that 1 squared and minus 1 squared give me the same answer.
So the jump across this line is zero, the speed of the shock is zero, the jump condition is OK.
So I'm saying that the jump condition is OK for both.
And it's the right thing to have the jump in this one, but this is wrong.
There should be a fan and I guess I've forgotten how the fan will go.
We do have 1 here and we do have minus 1 there, but the value of u is connected not by a jump at a point, but by a fan that gets us from there to there.
A fan is also known as an expansion way.
That's the right thing to have for that problem.
All that is just to say that I do have to check the entropy condition.
But I hope it's not going to be too difficult.
So my entropy condition, remember for this special case f prime is just u.
So I'm really just checking now ul greater than the shock speed, greater than ur.
ur is zero of course.
On this side of the shock it's zero.
So the shock-- can you get that?
We're going to be OK on the entropy condition.
The shock speed is greater than zero and it's smaller than ul.
Well, how is that?
Is this quantity, the shock speed-- no, greater than is this shock speed, so checking that enropy condition means-- shall I put it up here.
ul bigger than-- so this was the characteristic speeds, the wave speeds on the left of the shock.
They must run into the shock, so they must be going faster than the shock.
OK, so let me just do this comparison.
ul at the left of the shock, on the left of the shock x is capital X, which we decided was root 2t.
This is root 2t over t. That's u left because the shock starts when x is equal to root 2t and that's what u equals.
And the shock speed we figured out here is 1 over.
Now, so the question is, is this true?
1 over root 2t.
I guess I sure hope that it is.
And it is of course.
If I multiply out by this I have 2t over t, which is just 2 is bigger than 1.
So the answer again, is yes.
So that completes checking that our proposed solution of fan then shock is OK.
But now let me come to this formula because it's quite a handy, useful formula.
And what I have here is f of u divided by-- or f of x minus y divided by 2.
That something squared over 2 divided by t. Well, what do I want to say about that going on?
Well, first of all, if I do plug in the delta function for u0, it gives the right answer.
I can easily do this minimization.
What what do I get when I plug in the delta function for u0, just to start on trying this formula to be sure it gives the same answer that we just found?
So the integral of u of x and zero, the integral of u is a step function.
So this will be a step function of 1 for y greater than zero and zero for y less than zero.
Minimization requires a little patience because a step function of y is coming in here, y is there and the only way I would know to do that minimum would be the separate the different cases and do each one.
Each one would be simple.
I don't think we need to take our time to do that.
It's going to give the same answer and the minimization is actually carried out in the applied math book.
But I would like to say where it came from.
Where did this formula-- so I'll go back to this formula.
As a general formula for any starting value, starting function.
And in some way derive the formula.
Figure out where did that come from?
It came from the Burgers' equation with-- so now I want to speak about this guy-- the Burgers' equation with viscosity.
It turns out that that equation and this is something very important.
That equation has an exact solution because I can change variables and make it linear.
I can turn it into the heat equation.
So that was a neat idea, which two people noticed at almost the same time.
That very satisfying change of variables will turn this, will get rid of this nonlinearity and turn it into the heat equation.
So it's a nonlinear change of variables of course, but not that hard to discover once you begin to hope that there is one.
And then once it's a heat equation we know the solution.
The linear heat equation we know how to solve.
So we solve it and then what?
Let the viscosity new go to zero.
Watch what this solution is doing as new goes to zero and of course, it'll be smooth for any positive value of new.
The viscosity keeps things smooth.
The shock appears in the limit.
Maybe I should say something just physically about waves, breaking waves.
If you look at waves in water, they're obeying a nonlinear wave equation, more complicated than ours, but nevertheless the same features.
And a wave forms and as all water skiers know, surfers know, surfers I guess know it best.
The wave in time begins to break.
Now up to that time the viscosity is actually not so important.
But as it begins to break there's a terrific uxx now at the moments before breaking.
And that uxx then, that term produces viscosity.
I mean, there is a viscosity there and suddenly it's important.
And so the viscosity will prevent-- even though it's small, the viscosity will prevent the actual, complete break.
I mean, what actually happens in that wave is of course, a very difficult thing.
The nonlinearity, turbulence-- everything's happening suddenly.
Not easy, but the main point is as in many problems, a term that's small in the case of smooth solutions.
Where uxx has a normal size and new is very small, so that term is small.
Suddenly uxx has a giant size and that term is important, physically important.
So Burgers' inviscid equation, this is the case with inviscid means new equals zero.
So Burgers' equation is dropping that term and it does have a shock as we very well know.
What are the steps that these two people notice that take Burgers' equation and turn it into the heat equation?
Let me see if I can remember the two steps.
So I'm starting with Burgers' equation and I want to make it the heat equation.
OK, so a first step is to introduce the integral of u.
Let me call that capital U of x.
So capital U of x will be the integral so if I write it this way I'm going to change u to capital U.
So I'm introducing a new variable, capital U.
The linearity won't disappear yet, but it'll look a little different.
So if I took this equation and I look at it, well let me just say what's the equation going to look like for capital U.
If I write it down then you'll see that it's right.
dU/dt plus-- instead of d by dx it'll be 1/2 dU/dx squared equals new Uxx.
I claim that that's Burgers' equation written for capital U.
Do you see that?
Let me just see, well, how do I get from there to the equation with a star, Burgers' equation.
I think I just take the x derivative.
If I take the x derivative of every term, I'm taking the x derivative of U, so that's a little u.
So I have d little u dt as I want.
If I'm taking the x derivative of this term I have little uxx, as I want.
And if I take the x derivative of this term it'll produce that.
The x derivative of the 2 will come here, du/dx will be the little u.
The x derivative of that will be the ux.
It's straightforward.
And so now I've got it in a slightly different, but still nonlinear form.
Still quadratic.
Now comes the nonlinear change of variables.
I think it's u is minus 2 new log, so that's the nonlinear part of the new unknown that I finally want, which is w. So that's an exponential change of variable.
Capital U is matching log w, so w is matching either the e to the minus, an exponential of u.
And I won't go through the steps.
But when you introduce that in this equation it turns into the heat equation.
Of course, the initial function is different.
The initial w is different from the initial u.
Well, in fact we could see, at time zero.
So this was at all times.
I didn't write the t explicitly, but at time zero, what is w of zero?
I guess I just take the exponential.
I put this over here and take its exponential.
So w of x and zero will be u of x and zero.
The starting value of u divided by minus 2 nu.
All exponential.
This is the sort of expression, e to the something divided by-- with a minus sign-- divided by 2 nu that we're going to meet.
We meet it in the initial condition and we meet it in the solution.
Well, actually we now know the solution.
If we remember, what's the solution to the heat equation?
So we have the heat equation.
There's a constant in the heat equation, so we just have to remember where does that constant go and that's actually worth doing.
And then we have to start it from this initial condition.
Let me just write over here the solution to the heat equation.
So this board will follow that board and so do you remember the solution to the heat equation?
You remember we know the solution that starts from delta function, that fundamental solution is that e to the minus whatever over 40.
Over 2t is it?
Yeah, so what is w of x and t?
Let's see.
So we have the heat equation.
There is a square root of 4 pi and now it's 4 pi nu t. When nu was 1 we didn't know that, but now essentially this nu I can change the time variable to nu times t. So everywhere I saw t I now will see nu times t. And then do you remember what went in here?
The initial value, w of x and zero times the fundamental solution-- that bell curve that is centered at x.
So it's either the minus, do you remember what it looked like?
I think there's an e to the minus centered at x, so it's x minus t squared over-- was it 2?
And now over 2t it was and now there's a nu t. x minus-- oh, wait a minute.
I can't have x's.
Let me take w of s and zero, e to the x minus s squared.
This is from minus infinity to infinity.
And that finally is going to give me w of x and t. So I integrate ds.
It's actually good to go back to and see something that we derived before but we haven't thought about it for awhile.
This is the fundamental solution starting at the point s. This is the amount of initial function there is at the point s. And I integrate over all those little starts of point sources.
I integrate over all the sources.
All the way from s equal minus infinity to infinity and I get that answer.
And now I know what w of s and zero is.
That's what I just found.
This is e to the minus as we've just said.
Capital U of x and zero.
U zero I'll call it, over 2 nu.
So for this, substitute this.
It's requiring patience, but not genius I'd say to do this.
Now the question is, what happens as nu goes to zero and that the thing that makes this worth presenting.
What happens to integrals like this.
I have an integral of something and the exponentials are both negative.
I've somehow an integral like this.
I have the integral of some e to the minus-- something-- I'll say b over nu.
Do you see that the b involves the u0 over 2?
Well, let me put a 2 nu.
This is really what I have in this formula.
Well, divided by this thing, so let me put that 1 over square root outside.
And remember, my question is what happens as nu goes to zero?
Do you have any idea-- I mean, it's a mess.
We do not want to do this integration.
It's an exact formula, but not one I want to deal with.
What I do want is to know what happens as nu goes to zero.
And the main point is that nu is showing up in the denominator.
We have some quantity in the numerator, what is that quantity?
That quantity is u0 plus this x minus x squared over 2 nu t-- over t because in fact, if we're lucky this b is exactly this quantity.
This is exactly this bracket.
That's I think, my b if I change letter from s to y.
If I'd been like prepared in advance I wouldn't of introduced y and s because they're both just dummy variables and they should have been the same.
You see that this guy, because you remember the change between little u and capital U?
Capital U is just the integral.
So that's u0.
And this is this x minus y squared over 2t.
I'm not sure if I've got a 2 somewhere lost, but we won't worry about that.
The main point that I want to make is, how do you deal with an integral like this?
Because this is like a major topic in a more advanced course on applied math, a more advanced course on the analysis of applied math, not the numerical analysis.
What's happening is nu goes to zero.
Well as nu goes to zero, this thing is getting very large.
e to the minus a very large amount.
So it's as near nothing as you can imagine.
e to the minus a very negative amount.
But still the total is considered somehow and the question is, what's the main leading term in this quantity and the answer is, you look at the point where b is a minimum.
If b at one point is 1 than we have an e to the minus 1 over 2 nu.
And compared to e to the minus 100 over 2 nu the e to the minus 1 over 2 nu is way bigger.
So that in the end the contribution all comes from the point where b is a minimum.
And that's what this formula says we should look for.
And if you figure out what that contribution is in taking into account the fact that we do have a 4 pi nu t here-- without this the whole thing would just be going to zero as nu goes to zero.
There's an e to the minus very negative stuff at zero.
But we have a nu here, so in the end something nonzero emerges, but it emerges totally at this point where b is a minimum and that's what this formula has found.
This is the minimum value of b. OK, I won't track down further.
Do I remember the right word?
Is it stationery phase?
Did anybody meet these words?
I'm speaking now about something in the history of applied math.
Still those tools that were created over the precomputer years are still extremely valuable to understand what's going on and this is just an illustration.
OK that's my comments on the limiting case of Burgers', which is inviscid Burgers and we actually used this form of the equation as the first step in linearization.
As I say, it's quite a nice exercise now to use this formula.
An explicit formula, which we last time did not suspect.
We were tracking down, we knew the problem was nonlinear and were tracking down shocks and fans.
Here's a formula that just plain gives us the answer.
So this gives us a shock or a fan, whichever.
This has the right shock speeds if there's a shock and it has the right-- the entropy condition is satisfied because it was satisfied all along as nu was going to zero.
OK, so now do I want to make some final comments on this Korteweg-de Vries-- KDV-- equation.
I mean that would be a ideal numerical example.
In fact, its importance was discovered numerically.
One could say that this is one of the biggest contributions of numerical simulation to the theory of nonlinear PDEs.
The discovery-- and it just came out of the blue-- the discovery that that particular nonlinear equation was what we might call integrable.
That there is some twist that produce linear equations whose solution give the answer to this nonlinear equation.
Maybe what I ought to say is, first to tell you what those numerical experiments were like and people just like loked at the answers and said, can this be?
OK, what were they like?
So we could find-- without much work-- we could find solutions of the form, of a traveling wave form.
So we could find solutions that have the form some function of x minus ct.
If you plug that in you have then derivatives of f. You have an ordinary differential equation for f and you can actually find shapes; waves that travel following that equation.
So of course, what you can't do is add two of them together and still have a solution because the equation is not linear.
So everybody assumed, OK, people had found these travelling waves before and the computer will-- if you start with the right initial condition the computer will produce a traveling wave.
If you start with the right initial condition the computer will show two travelling waves at different speeds.
This c will be different from one wave to the other.
So what will happen as time goes forward?
One wave will catch the other one because it's going faster.
So the shapes of each wave are known.
I think it's 1 over hyperbolic cosine squared.
So we have one wave going with a different c, going faster than another wave.
We're computing along.
The fast wave catches the slow wave and they interact.
Of course they interact because the problem's nonlinear and you can't see what's happening at the time when the big one catches the small one.
But you can keep computing.
And what happened was that after this jumble the fast wave emerged with the same shape.
And the slow wave emerge behind it with the same shape.
And there was some lag of course, some delay moment, which had been used up in the jumble as the one passed the other.
But they emerged looking the same.
I mean, that's sort of like a linear property.
Not exactly because you couldn't just add them, it wasn't perfect, but the shapes were perfect.
And that suggested that this equation might have hidden linearity that nobody noticed.
And then finally that was discovered.
It was a search.
People found conservation laws, it's easy to show that the integral of u is conserved by this equation.
We've done that a thousand times.
Then other integrals are conserved and if you just like to do algebra a short list of conserved quantities: integral of u, integral of u squared and a few others, not just u cubed and u fourth of course.
People were creating a list.
And then somebody saw how to get an infinite number of integrals.
So that was a suspicion that the problem was integrable.
That if you could find an infinite list of conserved quantities, conserved even under this nonlinear interaction term that somewhere something was linear and then eventually a change of variables-- well, more than a change of variables, a total twist was found integrated and get that explicit answer.
That's more than I can do, but I want to say of course that immediately set out a mad search for other nonlinear problems that were in this sense completely integrable.
And now, am I going to be able to list some of the ones that were discovered?
I think I can-- whoops-- not on that board.
So I'll just end with the completely integrable nonlinear equations.
And I don't know if the list is complete, the discovery of new ones is like a major thing, so we have Korteweg-de Vries.
That was this the first.
Then there was a nonlinear Schroedinger equation.
And the solutions, I have to say what the solutions are called.
Those waves that I mentioned, which they're not quite that bad-- are called solitons, solitairy waves.
And it was the interaction of soltions and the fact that they emerged looking unchanged that set off this horrific effort to figure out what was going on.
So I was going to write nonlinear Schrodinger in here and without saying exactly where the nonlinearity appears, but you take Schrodinger equation.
Then oh, I'm blanking out on other.
There are just a couple more famous ones.
I'll put those names into an e-mail rather than stumble here.
OK, so that's a little history with no details of an incredibly special class of nonlinear PDEs.
Well of course, the Navier-Stokes equations aren't on that list.
And we still don't know, especially in 3D, I guess that's one of the million dollar Clay prizes is to know whether that every Stokes equations have a solution for all time.
So that's one of these 7 famous prizes of which 1 of the 7, geometry analysis prize, apparently that problem has been solved.
So there are 6 to go and 1 of them is whether the Navier-Stokes equations have solutions.
Actually, I'll just take one more minute of your time.
Thank you.
People had or somebody had conjectured some possible blow-up in which maybe that was a case, that was an example of nonexistence.
We could find a particular initial that created such a blow up in a small space that the Navier-Stokes equations couldn't be continued.
But big numerical experiments just reported in the last weeks show that that particular example, that somebody thought might prove nonexistence after a finite time doesn't.
So it's back to the drawing board really.
I don't see how numerically we're ever going to see that it does exist, but this numerical experiment was efficiently clear to say that that particular phenomenon that looked dangerous didn't go wrong.
OK, so next time it's a little more about numerical solutions of these equations and then about the level set method.
And then looking ahead, we have a guest lecture on financial mathematics, the PDs of financial math for everybody who wants to go to New York and make a billion.
And then we're going to very quickly be in the second part of the course on solving large linear systems.
OK, I'll see you Friday then for level sets.
Good.
Thanks for your patience.
Hi
[INAUDIBLE]
Of course I have.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
OK, so I'll start with this.
This is a method for a particular class of problem and maybe moving interfaces, moving curves, curves that are -- or surfaces in higher dimensions that are being transported, and it's their shape that matters.
So it's a sort of topic within shape differential equations rather than particle differential equations.
And some cases of this problem can be studied by a particularly fast method, developed here at MIT in Course 6 actually, called the fast marching method.
It's really thanks to Per-Olof Persson that there's now quite a bit on the 18.086 web page.
The basic equations that'll be in the lecture and also demonstrations, quick demos that show the point.
And I better mention Sethian and Osher, who jointly proposed the method, conceived of this approach in the late '80s, and each has written a book, or coauthored a book, on the level set method.
So those books are, so to speak, the authorities on the method.
OK, so what's the idea?
Well, here is the curve, the shape, the interface that's moving.
And let's suppose -- well, imagine it's sort of -- there's a wall a fire here, and think of it as a moving fire front.
Then that fire front will move, normal to itself, to some later position at time 1.
We could normalize -- in a perfect case when it was homogeneous and the speed of movement didn't change with position, we could imagine that the speed is 1.
So where is this fire front here at time 1?
It contains all the points that are reached from the initial fire in unit time.
And since we're going at speed 1, let's say, this is all the points at distance 1.
This curve, in this best, simplest case, will be the curve that's everywhere a distance 1 away from this curve.
Now, how to describe the curves is the key point here.
Shall I describe the curve by particles?
I could imagine these points as being my unknowns, my positions, and some equation for following them.
So I follow those ordinary differential equations, one for every point, and they reach new positions at time 1.
And then I interpolate the next curve, the new curve at time 1.
Well, that's not what we do.
That's not the idea of the level set method, and I'll explain why that idea of following particles, which is a totally natural one, comes to grief numerically too often.
This picture, of course, was like an ideal case, and the particles separated a little bit, but not too much.
Just to say in the word, the trouble with the particle idea is that particles could separate wildly, so you had a big, big gap to fill in like a fan.
Or they could come very close together so that you had total instability numerically.
You couldn't compute any differences because the distance was too small, and that would be like a shock.
So we're really connecting this type of problem.
It connects well with the conservation laws that we've been studying.
Fans and shocks appear, and I'll show you right away.
Can I just say, what is the approach?
What does this word "level set" mean?
Well, the level set of a function -- I'm saying what this word level set is, and that's going to be the way the curve is described, sort of implicitly.
Instead of saying where the points are that are on the curve, the curve is being given by an equation like phi of x, y equals 0, or phi of x, y equal 1, or phi of x, y equals 7.
Those three curves would be level sets of phi.
Do you see the idea of levels sets?
They're the solutions to the equation phi of x, y equal constant.
Let me take a simple example here on the board that I can -- so the idea of level sets, first of all.
Let me take an easy -- a particularly nice function.
phi of x, y equals, say, x squared plus y squared.
OK, so I want to ask, what are the level sets of that function?
So I set phi of x, y equals a constant to get the c level set, you could say.
And, of course, we recognize that as a circle when c is positive, and there is no set when c is negative.
So this would be -- the level sets would be circles.
So that would be one description.
I mean, what we're really doing is we're introducing a new dimension to the problem.
Here, it's a 2D problem with x square and y squared, but I'm bringing in the third dimension z and thinking of the curve in the x, y plane as being produced by -- so what would be the graph of z equals x squared plus y squared?
That's some surface now in 3D.
So I have x, y, and now I've introduced this third dimension in order to see a whole surface.
And now I cut that surface by the plane z equal to c. So z equal to c cuts through that surface by a plane here.
And that cuts out a circle, which I can drop down to the x, y plane if I want to see it down there, and that's the level set at height c. So that's a way to describe the curve, but it's sort of -- it's implicit.
You'll see that it has major advantages when -- maybe I can just describe them without drawing them.
Suppose I have a problem with two fires, say, two circles of fire, spreading.
What's going to happen as time evolves with these two circles of fire?
Well, having taken circles, they're just going to expand, so after a little time, I've got two bigger circles.
But after a little more time, the circles will meet and form one noncircle, right?
A sort of figure-eight shape.
Now that's a change in topology, and the world of shape calculations is always having to ask how am I going to deal with changes in topology when curves meets, when unconnected things become connected, or when connected things disconnect?
And the one great value of the level set approach is it deals with that easily.
I could imagine a double -- so I don't know that I'll succeed to do it here, but I could imagine a function, which -- OK, maybe this example, this picture, isn't too bad.
So that's intended to be a surface.
Well, let me make the surface go up that way.
OK, so when I cut it at z equals c, that might cut the surface in two different planes.
So the level set at a certain level is two circles.
There's two circles.
But now, let z increase, and what happens?
You see how naturally, as z increases, these two circles are coming closer and closer, and at a certain point, they merge.
And phi of x and y remains a totally nice function.
I mean, I could create some probably fourth-degree function curve that would do that fine.
I could graph it.
I could visualize it.
And the point is that by going up that one dimension and then slicing at different levels, you allow these things to happen smoothly.
OK, so that's one feature of level sets, that description.
OK, so that's the way curves are going to be defined.
So our unknown becomes this function, which will become a function of x, y and t with some initial condition.
OK, so now, let me evolve the curve.
Well, OK, yeah, let me evolve a curve.
So we already took the case of a circle, which just grows, and that's a piece that it.
Let me do a couple more example so that you see -- how does a corner evolve?
Suppose that's a t equals 0.
We have a corner.
We have sort of a fire in here, and it's growing outward.
And we're looking for all the points at distance 1 from this curve.
This is our curve at time 0.
OK, what do the points look like at time 1?
Well, these points here will certainly be a distance 1 away from our curve.
These points here will be a distance 1 away from our curve.
But then there's a little circular arc, right -- maybe I should make that dash line, too -- which are additional points at distance 1.
So in this case, the evolving curve has smoothed out.
That corner, which we might describe as a jump in the slope if we were thinking about jumps from the last week's topic, has -- well, I guess I'd use the word fan.
That jump in slope has sort of the characteristics or some key lines here, and here was an area where none of these lines are matching, but we have distance 1 still.
Somehow, the equation is -- the slope is becoming smooth, which it wasn't initially.
Now the opposite could also happen.
A corner could appear from a smooth start.
So that would correspond, of course, in our thinking.
We immediately think, oh, we started with something smooth, evolved for a little while, and a shock appeared.
So a shock would correspond to the -- it's a shock in the slope, so to speak.
So the curve itself will be continuous.
Let me take it -- well, let me just take it straight.
I mean -- oops!
Well, I was hoping to make it tangent so let me make it more obviously tangent, so a little -- maybe an arc of a circle or something, and then straight again, just as a model that you can see what's happening.
OK, what will happen now, so that's again t equals 0.
So that, you see, has -- these are now -- the fire is moving normal to the surface, normal to the fire front.
And you see what happens now is, after a certain time, say, t equal to 1, we're going to have a corner appear.
This'll get steeper and steeper.
The slope will change faster and faster.
Here it takes a little while to change from that slope.
It curves around and slowly gets to that slope.
But that is going to get squeezed in as we move forward in time and -- well, it's essentially time running in the opposite direction from this box.
So this would be a t equal to 1 with a corner.
OK, and maybe those examples will explain why the idea of a particle method isn't great because if we put particles -- I'll just say that again.
If we tried to follow this, which would be a completely natural idea in discretizing the differential problem, would be to follow particles there.
They travel perpendicular to the fire.
They reach here and here, but nobody reaches there, so we don't have any information there.
And if we did try the particle method here, we'd have a bunch of particles that start following the law of moving -- well, we have to figure out where the tangent direction is, and perpendicular to that is the normal direction.
The particles move along, and you see that all these particles are going to be really close together after a little time, and then there, they meet.
So the particle method just basically fails on a problem like this.
On a problem that stays beautiful and smooth and keeps it shape, probably the particle -- the following ordinary differential equation for each particle and then interpolating to find a curve is reasonable.
But level set method is a different way.
OK, so let me stay with the level set idea, which involves, of course, remembering if this is the curve P of x, y equals 0, then the normal direction is the same direction as the gradient.
I mean, that's the one bit of calculus that we absolutely need, is that the gradient of a function points in the normal direction.
And if we want a unit vector n, the unit normal vector, we have to divide by its length.
So that's the basic formula from calculus of two variables.
And then there's a special case of a function when the gradient is 1.
So we have -- that formula simplifies because the denominator is just a 1 for these functions.
OK, so these are distance functions.
Functions with gradient everywhere 1 are kind of special and nice, very special and nice.
And I guess that if everything is -- so I spoke about distance function here.
So I allowed myself to take -- to imagine that the function phi was the distance, that everything was traveling with speed F equal 1.
So let me introduce that letter F for the speed, and in this case, it's 1.
So add time t. The function that gives the distance has gradient 1.
Maybe an example would serve.
Let me go back to this circle here.
OK, suppose these are my -- I'm not doing this tricky case with two circles, but just back to a single circle.
So if I take the gradient of that function, I get -- it's a vector, right?
The x-derivative and then the y-derivative, it's 2x and 2y.
And that's a vector with some magnitude.
Probably the magnitude is about 2r, the distance from the center.
It's not 1.
This is not a distance function.
The distance function that has the same level curves but has gradient equal 1 would just be the square root of x squared plus y squared.
That's a distance.
OK, so that distance function -- say, and let me measure distance from the center.
OK, so the distance from the origin is 0, when x and y are 0.
I claim that this distance function is -- that that function is this perfect kind for the level set method where its level sets are still those same circles, but the gradient of that thing, the gradient of this -- of course, this is just r. And the gradient of it, if I take the x-derivative of this -- so let me take the gradient of the distance, just in case you were really remembering what we easily mastered in calculus of several variables.
The x-derivative of that is, well, I get 2x, and then I get a half from the square root, and then the square root goes in the denominator.
So that's just x over r, x over the square root, if you like.
x over the square root is the first component.
That's the x-derivative.
The y-derivative of this is 2y from the chain rule, and then the whole square root produces 1/2 square root to the -- square root in the denominator.
In other words, it produces y over the square root.
And sure enough -- so that's the gradient and sure enough, if I take its magnitude, I take this squared plus this squared, square root, but I get 1.
OK, so distance functions are the good ones.
So in working with the level set method, we will follow equations, numerically, of course, and phi of x and y might start out -- phi of x and y and 0 might be a distance function, but later on in time after we've followed phi, it probably changes from a distance function.
And if it changes a lot, if its gradient is getting out of hand, for example, the gradient of this is getting bigger.
And after awhile, we can stop, reinitialize to make the gradient 1 again.
Effectively, we would come back to this function and proceed.
So this reinitialization is part of the numerical computation that I won't say more about.
Really what I now have to do is say what's the key equation?
What's the equation that we solve numerically in x, y, t?
Remember, it's going to be a partial differential equation.
OK, so first we have to know what's the rule for the evolution of a curve.
So this decides the movement of the curve.
This decides the movement of the curve, this velocity vector, velocity field, velocity vector, v, v1, v2.
It's a vector.
All right, then the neat thing is that we get by using this level set description of the curve to this equation.
The convection equation, you could say.
It's like our one-way wave equation.
If v is a constant, it's just our equation, but now we're in two space dimensions, instead where before, we were just in one.
But if v is not a constant, then, for example, v may depend on phi.
The velocity might depend, and it often does depend on the curvature of phi in some nonlinear problem, then our equation becomes nonlinear, but it still has a nice form.
So that expresses exactly what our thinking is that the level sets -- that the function, the evolution of the function, tells us how the level sets evolve, and then, at any time, we read off the position of the curve from the level set.
And then one more step to that equation, which is often easier computationally than this one.
So I'm going to take a simple step between there and there in the case when v is in the normal direction.
So let me look at this v dot grad phi that came into the convection equation.
I'll divide and multiply by the magnitude of the gradient, and then I recognize this as the normal vector.
I recognize that as n. So this is v dot n, and I'm going to give a name to v dot n and call it F. That's the speed, speed in the normal direction, v dot n. And then there's this factor magnitude of grad phi coming along.
OK, so that's the equation.
That's sort of the famous level set equation where if F is a constant, that's the case we're speaking about of a fire moving.
But if F depends on the -- F could be a function of -- if I put parentheses in there, F could be a function of gradient phi, a more complicated function.
For example, I wrote down, because I would otherwise forget, this could be our -- this is a possible F of grad phi.
Actually, it depends on more than the magnitude so I'd better -- yeah.
And certainly not a linear one because of that denominator.
So a big set of examples in this subject, not linear, are curvature shapes.
Actually, the curvature problem is quite interesting.
If the movement depends on the curvature -- so suppose I start with an elliptical-looking shape, the curvature is smaller here, right?
Oh, I'm sorry, smaller curvature there, larger curvature there, and if we choose the right evolution, often it will come -- maybe this is a case where the movement is inward because the curvature is somehow the -- they're probably vectors pointing in.
Here it would move in slowly because the curvature is small.
Here it would move in faster because the curvature is larger.
The curvature is tending to even out.
And, in fact, after a certain -- as time goes on, any shape approaches a circle, and the circle shrinks and shrinks and disappears.
So that this class of -- this sort of nonlinear group of problems has a nice geometric picture, that whatever shape you begin with, you could even begin with a really weird shape like a barbell or something.
So this has zero curvature.
It wouldn't move at the start.
This would move in, so after a little while it would be like this, then the corner would move in.
So it seems a little unlikely it's true.
After awhile, the shape would become nearly circular, get smaller and smaller, and the shape would vanish.
Anyway, that a curvature problem.
OK, so I guess following the main theme of the course would be write down a finite difference method for the level set equation.
That's the natural thing to do.
Write down a finite difference method for the level set equation.
Solve that numerically.
So that would be on the web page.
The difference equation is going to be a sort of upwind equation, but it's a little messier in two variables to write out in detail here.
You'll see it there on the web.
OK, so you solve it, and proceed in time, and reinitialize when necessary and/or when it seems advisable, and you get a great picture of this evolving, moving interface.
So problems of that kind.
Or optimizing the interface.
There are other problems where you might -- ones on the web where you might be look -- the unknown might be a shape.
And you're looking for the shape that maybe makes an eigenvalue as small as possible, subject to some constraint.
So problems for shapes.
That's really the type of problem that level set methods are created for.
OK, and these example show the connection -- I'd say that somehow the -- yeah, it's really that if I tried to connect it to conservation laws, to the previous lectures, this is our earlier guy, would be, in one variable, let's say, this would correspond to the derivative of maybe phi or capital U.
So the conservation laws for little u correspond to equations like this for phi.
Well, or really more rad phi in more dimensions.
OK, well, all right.
So what else remains to say about this topic?
Really I guess my main point is to introduce you to the idea of level sets and level set method for when problems of moving interfaces and shape optimization come up.
I guess I also wanted to say something about this fast marching method.
OK, so what's that?
That's a case where there's a very fast solution to this problem.
And that case occurs when the wave front is always moving the same way, when the F holds its sign.
So it would apply here.
So I could use the fast marching method.
Rather than the level set equation and a finite difference method for it, I could use a fast marching method for this problem because I have no danger of the shape turning, changing direction, coming back in on itself, all the crazy things that can happen, the things that are illustrated on the web.
OK, so the fast marching method, well, maybe I -- so what's the problem in that nice case?
I want to find -- I could put on a mesh.
Yeah, let me take as a problem the computation of the distance function from this curve.
OK, so I could create a mesh.
So I'm going to take that problem and turn it into a -- discretize it in a natural way, and I'm searching for the distance function.
So d of x and y is the distance that I now want to compute.
It's the distance from the curve C. And let me make an assigned distance function so I'll make d negative in here and positive outside of the curve.
OK. All right, so the idea is we could -- this fits our level set picture, but there's a direct way to do it.
And it's a nice -- it's an algorithm that computer scientists probably created and appreciated, that we want to know the distance.
We want to compute d at the mesh points.
Let me put another mesh point.
OK, so how do I -- so the question is, how do I get from -- let me put another y in there, too, and that created some more points.
So how to -- what path -- you know, how to get the distance from the curve to each mesh point?
Well, here's the idea, just in a nutshell.
You keep a list of tentative distances to all the mesh points, and you look for the one that's closest, like it's probably maybe that guy.
So maybe that's right on the curve, let's say.
So the idea is I can -- that will be -- to get to this point, I'm never going to take a path that passes through the others.
To get here, I might take a path that goes through an earlier point.
But the nearest point, the work is done.
So you successively sweep through the mesh points, pick the one whose current distance, tentative distance, is the smallest, and you say that is the distance.
That's the right number.
Then now you know how far it is to get to that point.
So now you could, if you had to, update all the previous -- all the other distances by knowing the correct one, how distant it is, how distant that point is.
OK, so then that one's settled.
Now you look at your list, your updated list.
You pick the smallest one in that, the closest one in that.
You found the best way because things are never going to reverse in sign.
You're never going to travel out and come back, so the closest one, you've got it.
So that might be this or something, and then we would update how far it takes to get to those.
So that fast marching method is a, you might say, fairly natural idea of how to compute distances when everything is measured, when our problem is -- has a fixed sign.
OK, so that's my short comment on the fast marching method.
My longer comment is on the level set method and level set equation where I haven't written out the finite difference approximation that people typically use and note on the web.
All right, so that's level sets, which I probably will not come back to.
It's kind of a one-shot lecture.
So let me take the remaining time to get feedback on the homework.
So were there four problems, possible candidate problems, for homework?
Let me put those down again.
I'm just going to ask first, who did which problem.
So the first problem was the conservation law -- no, was the one way -- OK, remind me.
I don't remember the order I gave.
So was Number 4 the conservation law?
The nonlinear one?
[INAUDIBLE]
Yeah, OK.
So the conservation law.
I'll just write down ut plus uux equals 0.
Oh yeah, Number 1 was the basic question of ut equals cux.
Yeah, that's right.
So that's a very important and not too obvious question, a good numerical question, of what to do.
And Number 2 was the wave equation; was that right?
Schrodinger's
Schrodinger's, Number 2.
Oh, boy.
OK.
So iuxx, which I haven't discussed in class and I had -- the good question came up, how do you deal with these complex numbers because, of course, it's going to get complex right away.
Well, one way that just occurs to me is take the real and imaginary parts so that we get two equations, two real equations
OK, Number 3 was -- what did I have --
The wave equation
Was the wave equation?
OK, second order?
First order
Oh, first order wave equation?
Oh, OK.
So what was I -- OK, first order wave equation like so.
I'm sorry?
[INAUDIBLE]
cux, OK, yep.
All right, so was the question there to go back to the--
Compare
Compare, right.
Compare the methods we've already established.
And then Number 4 was to tackle some case of nonlinear equation where shocks and fans could appear.
OK, maybe I just get a sense of who did what.
Number 1 how many?
Wow!
Number 1 was popular and I think it's justified.
So this is certainly the biggest choice.
So let me come back and get some ideas of what you concluded.
Number 2, did anybody tackle Schrodinger?
Yes.
A couple of complex guys.
Well, I know you asked about it.
Yep, OK, good.
Anybody go back to these?
For this?
You did?
OK, you may have done a little bit of both.
And did anybody venture to nonlinear world?
One.
Good, OK. Nonlinear person.
I'm glad to know that, right.
So this is like -- this homework you could think of us as like prep for a possible project.
Your project could grow out of this homework, or it could grow from other things that are coming.
OK. All right, let's stick with this one then since a lot of people will have input on it.
Let me ask -- well, I'll just ask naive questions because you've actually done the calculation.
Is there a method you'd recommend?
If you had a problem like that or, I mean, like for real, and you had to invest in a code and use it, just give an opinion.
Or did you only try one method?
Tell me something, positive or negative, about your experience with this.
Yeah, thanks
[INAUDIBLE]
Maybe you push -- in theory, there's some mike you push and this will be the first time we've ever done it.
OK, terrific.
Go ahead
[INAUDIBLE]
Centered explicit.
Let me write that down
[INAUDIBLE]
OK, right
[INAUDIBLE]
Yes
[INAUDIBLE]
That's right.
So, OK, the centered explicit one is computing this from three old values.
We certainly need three because we've got a second derivative there.
But then we have a certain amount of that second derivative and a certain amount of this one, and that can make one of those three numbers go negative.
They always add the one because the constant is certainly a solution we want to keep.
OK, so when a coefficient went negative, that doesn't mean instability; is that right?
[INAUDIBLE]
Yeah
[INAUDIBLE]

[INAUDIBLE]
Fine
[INAUDIBLE]

[INAUDIBLE]
Implicit had no trouble with that.
Yeah, OK. Or it didn't matter what the Peclet number, or the cell Peclet number was, or whatever we called it
[INAUDIBLE]
Uh-huh
[INAUDIBLE]
I see
[INAUDIBLE]
OK, if you went -- when you say implicit, it's just the diffusion term that's going implicit?
OK, OK.
So that should be a very stable process, but, of course, now we've got this term affecting it.
So you actually found out that even the implicit one could break?
[INAUDIBLE]
Yeah
[INAUDIBLE]
Yes
[INAUDIBLE]
Really?
[INAUDIBLE]
Huh!
[INAUDIBLE]
Even the implicit one?
[INAUDIBLE]
Yeah, yeah.
So on the implicit part -- so this was method one and this was method two implicit for the diffusion, but you found that that could still be unstable.
OK, yeah.
Anybody else do this comparison?
Can you push your mike button just to -- this is a hotshot room that we haven't taken advantage of, but it's our chance
I actually looked at the -- when I first did the -- looking at just the growth factors of explicit and implicit --
OK, looking at that g --
--on the convection term as well?
Yeah
And I used implicit for all the diffusion.
And I found the most stable using upwind, whereas when both were implicit, the growth factor was actually able to stay below 1
So you moved the first derivative also up to the new time?
OK, and that was -- then you don't have a problem
Yeah, exactly.
So stable for pretty much all big R, little r. But the explicit upwind for the convection term was stable for most as long as the little r was below 0.5
I see.
So the numbers could tell you the--
Sure
--limits on little r and big R?
Yep.
And then I did the explicit convection upwind with the implicit center diffusion, and that seemed to be pretty stable unless I really started cranking the numbers out
I see.
Oh, that's the one that he had also done?
Yep
Yeah, OK. And now so I guess I remember in the figure that isn't yet drawn that might emerge from your calculations a possibility where -- depending on the Peclet number, where it was stable.
It didn't blow up, but oscillations appeared and the calculations -- you wouldn't do it even so --
[INAUDIBLE]
That's what you meant, yeah.
OK, so there are cases where the method isn't good even though it's stable, which is like important to realize that that can happen.
Because up to now, stability has been, you know, the holy grail.
But it's really a good solution that's the holy grail.
Yes?
Go ahead
I did kind of a bigger picture where I basically tried to build the pieces so just to isolate the advection term and the diffusion term and then build their matrices
Yes
And then look at the combination rule.
So when you went to where you had both terms present--
Right
--how the matrices were actually -- came together to form the--
So you took a matrix picture of the -- with all the mesh points, mesh values, at once rather than--
And then I simulated with them just convection, just diffusion and then the coupled problem of both of them to see how that -- just one method may be unstable for a given formulation--
Right
--but yet when you saw the coupled problem with those two pieces, it actually is stable.
So you could look at like the -- when you build the matrices, you can look at the eignevalues and you can make sure that they're--
Let me comment a little on that.
I'll do that, but I don't want to cut off anybody who has another comment to throw in here before I say something about -- follow up on matrices.
Anybody got some urgent thing?
Let me just, in the remaining minute, just say a word because, of course, you'll know that I think of a matrix description quickly and fondly.
So just to write down the sort of matrices that are in here, from the second difference, the second difference matrix is going to be -- it's going to have our -- well, I guess it'll be 1 minus 2 1 1 in a typical row.
We haven't written any matrices down.
It's amazing that -- we'd be at about the, I don't know, eighth or ninth lecture now, and not have seen this matrix, but here it is.
OK, so that's the matrix where I'm at a fixed time level.
I'm taking the second differences of all the points along that level.
And now the main point for the moment is to contrast that with a first difference matrix which, well, that depends whether I take it centered.
Maybe if it's a centered difference, it would be a 1 0 minus 1.
And, of course, there's a 2 delta x in the denominator.
so a minus 1 0 1.
So that would be a centered difference that doesn't do anything, doesn't use the middle value, but takes the difference between the value at the right and the value at the left, or an upwind.
So delta upwind would be more of a 1's forward of the point and minus 1's at ujn and 0's here.
OK.
So what you're saying, am I right in thinking you took some combination of the--
An implicit method for one and explicit for the other, you can do them independently by you can invert the implicit matrix--
Right
--and premultiply the other side and build
So this is a way to see, and actually the next months of our course are going to be about, how do you deal with these large difference matrices.
These are in one dimension so they're tridiagonal, and they're, of course, easy to deal with it.
But problems, serious problems, come up in more dimension.
But here, can I just express one thought here?
The eigenvalues of these matrices and their combinations are what's going to control, as eigenvalues always do, growth of solution.
Now we already had numbers g, growth factors, that we computed by the von Neumann idea of following exponentials, e to the ikx.
So these numbers too were -- their purpose was to track the growth of solution.
And what I want to say is that very often, I mean, that these numbers and the eigenvalues of the matrices, they're very closely related ideas.
In fact, they'll be the same idea if the eigenvector is a pure exponential.
So that's really the link between eigenvalues of matrices and growth factors g that depended on frequency k. If the eigenvector was a pure e to the ikx on the mesh, then that growth factor is identical to the eigenvalue that we get it one step of the difference equation.
So the idea was that the eigenvectors, the details of the eigenvectors, will be affected by boundary conditions.
So von Neumann thought, all right, I'll cut through that Gordian knot and just pretend I have a pure e to the ikx.
I look inside the region, not at the boundary where we may be doing something different, and we get a g, a growth factor g. And again, if the eigenfunction was really an eigenvector, then that g would be the same as lambda.
OK, we're past time.
I always appreciate that you are patient about that.
So I'll collect those that are complete.
And if you wanted to speak to Mr. Cho or have a little more time to do more, Monday is good, too.
OK, thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu
For me this is the third and last major topic of the course.
The first one was initial value problems -- stabililty, accuracy.
Topic two was solving large linear systems by iterative methods and also by direct methods like re-ordering the equations.
Now, topic three is a whole world of optimization.
In reality it means you're minimizing or possibly maximizing some expression.
That expression could be a function of several variables.
We could be in the discrete case, so we've got -- maybe I just emphasized that we have discrete optimization.
So that's in rn, discrete in n dimensions, and that will include some famous areas as single, as small subsets, really, of the big picture.
For example, one subset would be linear programming.
So that's a very special but important case of problems where the cost is linear, the constraints are linear, and has its own special method.
So I think that's worth considering on its own at a later point in a lecture.
Another bigger picture -- for us actually bigger will be quadratic programming where the quantity that's being minimized is a quadratic function.
Now what's good about a quadratic function?
Its derivatives are linear.
So that leads us to linear equations, but always with constraints.
We don't have a free choice of any vector in rn.
We have constraints on the vectors, they have to -- maybe they solve a linear system of their own.
We might be in 100 dimensions and we might have 10 linear equations as the unknowns have to solve.
So in some way we're really in 90 dimensions, but it might -- you know, how should we treat those constraints?
Well, you know that Lebesgue multipliers play a role.
So there's a big area, very big area, of discrete optimization.
Then also there is the continuous problem where the unknown is a function, is a function u of x I'll say, or u of x and y.
It's a function.
That's why I refer to that area as continuous optimization.
Well, first you always want to get an equation, which is in some way going to be derivative equals zero, right.
When we learn about minimization in elementary calculus, somewhere along the line is going to be an equation that has something like derivative equals zero.
But, of course, we have to account for the constraints.
We have to ask what is the derivative of what when our unknown is a function.
I'm just going to write down a topic within mathematics that this is often expressed as.
Calculus -- that would be derivative.
But in the case for functions it's often called the calculus of variations.
So that's just so you see that word.
There are books with that title.
The idea of what is that derivative when our unknown is a function and the objective that we're trying to minimize is the integral of the function and its derivatives -- all sorts of possibilities there.
So that that's a very quick overview of a field that we'll soon know a lot about.
I was trying to think where to start.
I think maybe it better be discrete.
I want to get to the system of equations that you constantly see.
So let me use as an example the most basic problem which comes for -- maybe I'll start over here -- the problem of least squares, which I'll express this way.
I'm given a matrix a, and a right hand side b, and I want to minimize the length squared of au minus b.
So that would be a first problem to which we could apply calculus, because it's a straight minimization and I haven't got any constraints in there yet.
We could also apply linear algebra, and actually linear algebra's going to throw a little extra light.
So, just what am I thinking of here?
I'm thinking of a as being m by n with m larger than n. If a is the square matrix, then this problem is the same as solving au equal b.
Of course, we'll always reduce to that k'th, if m equals n. But to focus on the problems that I'm really thinking about, I'm thinking about the case where this is n is the number of unknowns.
It's the size of u. m, the larger number, is the number of measurements, the number of the data, so it's the number of equations, and it's the size of b.
So we have more equations than unknowns.
You've met these squares before.
I hope that maybe even in these few minutes, a little new light well be shed on these squares.
So here's our problem, and calculus could lead us to the equation for the best u.
So, u stands for u1, u2, up to un.
There are n components in that vector u.
Maybe you know the equation -- I guess I hope you know the equation, because it's such a key to so many applications.
If I just write it, it will sound as if problem over.
Let me write it though.
So that the key equation, and then this comes up in statistics, for example.
Well, that's one of 100 examples.
But in statistics, this is the topic of linear regression in statistics.
Let me write down -- they gave the name normal equation to the equation that gives you [UNINTELLIGIBLE].
Do you remember what it is?
The normal equation?
The equation for the minimizing u, which we could find by calculus?
It involves the key matrix a transpose a.
Let me call u hat the minimizer, the winner in this competition.
The right hand side of the equation is a transpose b.
So I won't directly go back to derivitals, so probably I'm going to end up deriving it, because you can't help approaching that equation from one side or another.
As I say, one way to approach it would just be to write out what that sum of squares is, take its derivative and you would get linear equation.
So again, u hat stands for the u that gives the minimum.
This a transpose a, of course, so I'm putting in a little bit of linear algebra.
This matrix a transpose a is obviously symmetric.
It's important property, beyond the symmetry, is that it's positive-definite.
Well, I have to say positive-definite -- there's always a proviso, of course.
I haven't eliminated the degenerate case yet.
a has m, many, many rows, a smaller number of columns, and let's assume that those columns are linearly independent so that we really do have n unknown.
If those columns were, say if all the columns where the same, then au would just be multiplying that same column and there would really be only one unknown.
So I'm going to say that a has rank n by which I mean n independent columns.
In that case, that's what guarantees that this is positive-definite -- let me try to draw an arrow there -- this is the same statement.
If I say about a that the columns are independent, then I'm saying about a transpose a that it is positive-definite.
That means all its eigenvalues are positive, it's vertible certainly.
All its pivots are positive.
It's the great class of matrices.
But I don't really want to start with that equation.
Here's my point.
Optimization -- a key word that I better get on the board maybe up here to show that it's really important, is plus the idea of duality.
The effect of duality, if I just give a first mention to that word, is that very often optimization problems, there are really two problems.
Two problems that don't look identical, but in some important way they both, each problem is a statement of the task ahead of us.
What are the two problems, the two dual problems in this basic example of these squares.
All right, here's a good picture.
Here's a good picture.
Let me put it on this board so I can recover it.
So, minimize au minus b.
So I think of the vector b as being where it's in m dimension.
So it's a picture, I'm in m dimensions here.
Now what about au?
au -- where will au go in this picture?
So, au is -- all the candidates au is multiply a by any vector u, so that means au is a combination of the columns of a, the possible vectors au lie in a subspace.
So this is a subspace of all possible vectors au.
It's only n dimensional.
This is an n dimensional subspace because I have only n parameters in u, only n columns in a.
So the set of all au's I think of as a, you could say a plane, an n dimensional plane within the bigger space rm.
Another name for that subspace, that plane, is the -- in 1806 I would call it the column space of a, or the range of s is another expression that you--.
All the possible au's and here's b, which isn't one of the possible au's.
So where is u hat?
Where is the best au -- the best au now?
The one that's closest to b is -- now comes another central word in this subject.
If I draw it, I'm going to draw it here.
That will be my best au, which I'm calling au hat.
That's the, if the picture seems reasonable to your eye, this is the vector that's in the plane closest to b.
What's the geometry here?
See, that's what I wanted to see -- a little geometry and a little algebra, not just calculus.
So the geometry is that this vector b, what's the connection between b and that vector -- that's the closest vector, right?
We're minimizing the distance.
This distance here, I might call that the error vector e. This is as small as possible.
That's being minimized.
That's the difference between -- it's a pythagoras here.
Of course, when I say it's pythagoras, I'm already saying the most important point that this a right angle here.
That's a right angle.
The closest au, which is au hat, which is this vector, the way geometrically we know it's closest is that the line from b to the plane, that's where the line from b, perpendicular to the plane.
This line, this error vector e is perpendicular to the plane.
There's a good word that everybody uses for this vector.
Take a vector b that's not on a plane, what's the word to look for the nearest vector in the plane?
Projection
Projection.
So this is the projection orthogonal projection, if I wanted to really emphasize the fact that that's a right angle.
So that would give me a geometric way to see the least squares problem.
Now comes the point to see the dual problem.
The dual problem will be here if I draw the perpendicular subspace.
So that's a subspace of what dimension?
This contains all the vectors perpendicular to the plane.
So I have it as -- if m is 3, so we're in three dimensions, and our plane is an ordinary two dimensional plane, then the dimension is one -- that's the perpendicular line.
But thinking bigger, if we're in m dimensions and this plane is m dimensional, than this is going to have -- the true dimension of this is m minus n, which could be pretty substantial.
But anyway, that's the perpendicular subspace.
If this is the column space of a, I can figure out what vectors are perpendicular to the columns of a.
That's really what I mean.
This contains -- I've drawn it as a line, but I've written up there its dimension so that you see -- I just don't know how to draw like a bigger subspace.
Yet you would have to see that all the vectors in it were perpendicular to all of these vectors.
You see what I'm saying?
If we were stuck in thinking in three dimensions, if I make this a plane I can't make that a plane.
If I make m equal 3, and I make n equal to two, I'm only got a line left to be perpendicular.
But in higher dimensions there are lots of dimensions left.
So what's my dual problem?
My dual problem is find the vector e in this plane closest to b.
In other words, by the same reasoning, what I'm saying is take the vector b, project it over to this plane, project it orthogonally -- that same right angle is going to be there.
This plane -- I haven't said what's in this -- I've said what's in this plane but I haven't written it yet.
But you could tell me already what is this?
What's that vector?
One answer would be it's the projection of b onto this perpendicular.
So you see we're really taking the vector b and we're separating it into two components.
One in the column space, the other perpendicular to the column space.
So just tell me what that vector is.
It is e. Same guy.
In other words, e is the solution to the dual problem -- maybe I call this projection p for the best vector in the plane.
e is the best vector in this subspace, and they add up to--.
So, we're really taking the vector b, and we're splitting it into a part p in this space, and a part e in the perpendicular space.
If I just write down the equations for that, I'll see what's cooking.
Well, I guess what I have to do is remember what are the equations to be in this subspace, to be perpendicular to the column.
So I can't go further without remembering what's in that subspace.
So everything in that subspace is perpendicular to the columns of a.
Let me just write down what that means.
Let me use the letter maybe y for the vectors in that subspace, and e for the winning vector, the projection.
So y will be the vectors in that subspace.
So those vectors are perpendicular -- so this is the subspace of y, all the y's in here.
Now, what's the condition?
So y -- that y in this perpendicular subspace.
What do I mean?
I mean that y is perpendicular to the columns of a.
How shall I write that?
Perpendicular means inner product zero.
So I want to change the columns into rows, take the inner product with y and get zeros.
Zero zero zero.
So, this is column 1 transpose to be a row.
I'm trying to express this requirement in terms of a matrix, so to be perpendicular to the first column, I know that means that the inner product of the first column with y should be zero.
The inner product with the second column -- the second column with y should be zero.
The nth column, it's inner product with y should be zero.
So what matrix have I got here?
What's the condition on y's, simple and beautiful?
What matrix is that?
It's a transpose.
So that perpendicular thing -- this is completely expressed by the equation a transpose y equals zero.
That tells me the y's, and, of course, e is going to be one of the y's.
It's going to be, I could say y hat, but I've already named it e. It's the particular one that's closest to b -- the y's are everybody all along here -- this is the null space of a transpose.
So in words, I would call it that perpendicular thing is the null space of a transpose.
So when you did linear algebra, you'll remember that.
That the null space of a transpose -- let me write it.
It is perpendicular to the column space of a.
The fundamentals theorem of linear algebra right there.
Now we're just using it again to see what are the two dual problems here.
So the prime problem, the one that we stated first, was this one.
So I'll call this the primal -- p for primal.
What is the dual problem?
The dual problem is the problem about the y's.
Not about the u's at all.
That's the beauty of this duality.
One problem is about u's, and it's a problem that ends up projecting on the column space.
The second problem is about y's, it's the problem that ends up projecting onto this perpendicular space, and it was a projection.
So the dual problem is just minimize the distance from b to y, but with the constraint with -- and now I get to use that word constraint -- with a transpose y equals zero.
So there is the other problem.
So I hope your eye can travel between the -- well, let me write underneath it the primal again.
Minimize au minus b square.
This is the one whose solution is e, and this is the one whose solution is, well, u, and the projection u at, and the projection p is au hat, and I guess what I'm trying to say is that somehow there's a very important connection between the two problems.
First of all, the two problems use the same data.
They use the same vector b, they use the same matrix a.
Notice that in one problem it's a, and in the other problem it's the transpose appears.
That's very common -- we'll see that always.
But there's something a little different about the two problems.
This problem was unconstrained, any u was allowed.
This problem was constrained, only a subset of y's, only that subspace of y's was allowed.
This is a problem with n unknown.
This is a problem with m minus n unknown.
m minus n unknown variables, once we've accounted for the constraint.
This is one thing I'm thinking about.
Often, the problem will come with a constraint.
Maybe I'll do a physical example right away.
The problem comes to us with a constraint.
In other words, suppose you were given this problem.
How would you deal with it?
That's like the first question in optimization, or one of the central questions.
How do you deal with a constraint?
If we minimize this, of course, the minimum would be when y equals b.
But that's failing to take into account the constraint on y.
So how do you take constraints into account and end up with an equation?
We can see in this picture that somehow or other we ended up with this normal equation, but actually I would rather end up with a primal dual equation.
I'd like to end up with an equation for the best u and the best y.
So what will that be?
So I need now two equations that will connect the best u and the best y, and probably this is going to be the key.
This b is au hat, right.
So this will be one of my equations, and this will be the other.
Let me see if I -- well, OK.
I don't know what to do now.
Here I've called it y.
Over here it's e. I've got myself in a corner.
Maybe I should call e y hat, would you like that?
We have in mind that it's e, the error in the primal problem.
But just to make the notation for the two problems consistent, let me call the winner here y hat, the winner here u hat.
What's the relation?
So let me just -- here I'll write down the relation between the two.
Well, it's over there.
Let's see, is that right?
Yes?
y hat plus a u hat is b, and a transpose y hat is zero.
That's it.
That's it.
Here we have -- that's the pair of equations that solves, that connects the primal and the dual, solves them both, solves each one, and is really, it's a system -- you could say it's a block equation.
A block matrix being identity a, a transpose zero, the unknown being the y and then u.
The right hand side being the data, which in this case was the b. I guess what I want to do is emphasize in what's coming for the month of April.
The importance of this class of problems.
It's dealing with two -- it's dealing with the primal and the dual at the same time.
It's important for so many reasons I can't say them all on the first day.
That would be a mistake to try to say everything the first day.
But let me just say something -- that linear programming, which is just one example, and it doesn't fit this because it has inequality constraints.
But you maybe know that the number one method to solve linear program is called the simplex method.
Well, it was the number one method for many years.
For many problems it's still the right way to do it.
But a new method called the primal dual -- at least that's part of its name, primal dual.
It is essentially soving the primal and the dual problems at once, and there are inequality constraints, of course.
I'm going to stop there with linear programming and give it its turn later.
In this perfect example here, we have only equations.
How many do we have?
We have m plus n equations, because y is m unknowns, u is n unknowns.
I have all together m plus n equation -- m y's, and n u's, and they come together.
Now, could you just to connected back with what we absolutely know that it's a normal equation, where is this normal equation coming from?
So here's the normal equation.
We know that that's gotta come, right, out of the [?
paper.
?]
How does it come?
Suppose I have a block system, 2 x 2.
How do I solve it?
Well actually, that's, in a way, the big question.
But one way to solve it, the natural way to solve it would be elimination.
Multiply this first row by a suitable matrix.
Subtract from the second row to produce a zero there.
In other words, eliminate y and get an equation for u hat.
So what do I do?
How do I do it?
I multiply -- would you rather look at equations or matrices?
I've tried to keep the two absolutely together.
Let me look at the equation.
What shall I multiply that equation by and subtract from this -- I just want to eliminate.
I want to get y hat out of there and leave just an equation for u hat that we will totally recognize.
So what do I do?
I multiplying that first equation by a transpose, thanks.
So I multiply this first equation by a transpose.
Let me just do it this way.
Now what?
a transpose y is zero.
Now I use the second -- well, this is one way to do it.
a transpose y is zero.
What am I left with?
The normal equation.
Well, it shouldn't be a surprise.
We had to end up with a normal equation.
Maybe you would rather -- and actually, what I intended was to make it sound more like Gaussian and elimination.
Multiply this row by a transpose, subtract from this row.
I still have identity a up here -- when I do that subtraction I get the zero, that was the point.
Here I get a transpose a, subtracted from zero is minus a transpose a, y hat, u hat.
Of course, I had to do the same thing to the right hand side, and when I subtracted this the b was still there, but it was minus a transpose b.
So, now I've got in the second equation -- the second equation only involves u hat, and, of course, when I changed the signs, it's our friend -- a transpose au hat equal a transpose b.
So this is -- maybe you would say the natural way to solve this type of system, but I want to emphasize, throw it away.
I really want to emphasize the importance of these -- let me clean it back up again to what it was -- of these block systems.
Now, they need a name.
We have to give some name to this type of two field problem.
I guess then in the next month I'm going to find examples of it everywhere.
So here I've found the first example of it in ordinary old-fashioned least square.
So what am I going to call this?
I'll call it -- I'll give it a couple of names.
Saddle point equation, saddle point system maybe I should say.
I have to explain why do I call it saddle point system.
In optimization, I could call it the optimality equation -- just meaning it's the equations for the winners.
In the world of optimization, the names of Kuhn and Tucker are associated with these equations -- Kuhn-Tucker equations, and there are other names we'll see.
But let me just say for a moment why saddle point.
Why do I think of this as a saddle point problem?
See, the point about a transpose a was that it was positive-definite.
This is a transpose a.
Now what's the corresponding issue for this matrix?
So this is my matrix that I'm constantly gonna look at.
Matrices of that form are going to show up all the time.
I've said probably in 18085 where these appear but then we don't do much with them.
Now we're ready to do something.
I didn't appreciate their importance until I realized in going to lectures on applied mathematics that if I waited a little while that matrix would appear.
That block matrix.
It just shows up in all these applications.
One of our issues will be how to solve it.
Another issue that comes first is what's the general form of this?
Can I jump to that issue just so we see something more than this single problem here.
Let me point in the matrix as it comes in applications.
Some matrix a, rectangular, right?
It's transpose.
A zero.
Often a zero.
But what's up here is not always the identity.
I want to allow something more general.
I want to allow, for example, weighted least squares.
So weighted least squares -- if you've met least squares it's very important to meet its extension to weighted least squares.
When the equation au equal b are not given the same weight, there's a weighting matrix, often it's a covariance matrix.
I'm going to call the matrix that goes in here c inverse.
So this will be then an important class of application.
This is pretty important already when the identity is there, but many, many applications produce some other matrix that is usually -- it very, very often is symmetric positive-definite in that corner, like the identity is.
But key point, which I have to make today.
Is the whole matrix either this one or this one.
It is symmetric, right?
That matrix is symmetric.
Is it or isn't it positive-definite?
If I do elimination do I get all positive pivots?
It's a matrix of size m plus n. So I'm asking are all its eigenvalues positive, but I don't want to really confuse eigenvalues.
Also, in a lot of cases I would.
Finding the eigenvalues of this matrix would be the lead me to the singular value decomposition, absolutely crucial topic in linear algebra that we'll see.
But let me just take it as a linear system.
If I do elimination, what are the first m pivots?
Let me not be abstract here.
Let me be quite concrete.
Let me put the identity here.
Let me put some matrix -- oh, I want the matrix here to be -- I better put a bigger identity just so we see the picture.
Here I'm going to put to an a transpose, which might be 2, 3, 4, 5, 6, 7 -- when I write numbers like that you'll realize that I've just pick them out of the hat.
Here is the transpose 2, 3, 4, 5, 6, 7.
Here's the zero block.
I'd like to know about that matrix.
It's symmetric.
And it's full rank.
Its invertible.
How do I know that?
It's the invertibility -- of course, the identity part is great.
I guess I see that this is inveritable because I've ended up with a transpose a, and here my a has rank two -- those two columns are independent.
They're not in the same direction -- 2, 3, 4 is not a multiple of 5, 6, 7.
That's an inveritible matrix.
This process finds the inverse.
Elimination finds the inverse.
It's the pivots I want to ask you about.
What are the pivots in this matrix?
So what do I do?
The first pivot is a 1 -- I use it to clean out that column.
The second pivot is a 1 -- I use it to clean out that column.
The third pivot is a 1 -- I use it to clean out that column.
What's the next pivot?
What do I have -- of course, now some stuff has filled in here.
What is actually filled in there?
So this is the identity, if I can write it fast.
This guy is now the zero.
This guy didn't move.
What matrix filled in here?
Well, just what I was doing there.
Elimination is exactly what I'm repeating with numbers, what I did there with letters.
What's in here is minus a transpose a. I could figure out what I could do.
If I was fast enough I could do 2, 3, 4, 5, 6, 7 times 2, 3, 4, 5, 6, 7 and I'd get this little 2 x 2 matrix that sits there.
With a minus sign, and that's the point.
That the pivot -- of course, what's this first number going to be?
4 plus 9 plus 16.
29 is that number.
So a minus 29 sits right there.
That's the next pivot.
The next pivot is a negative number, minus 29, and the fifth pivot is negative.
So what I'm seeing is a matrix, this matrix has three positive pivots, and two negative pivots.
I sort of say that was a saddle point.
Positive pivots describe for me a surface that's going upwards.
This surface is going upward.
I'm a surface in five dimensions here.
It's going upwards in three directions, but it's going downwards in two.
The point at the heart of it, the saddle point, is the solution to our system, is the y hat, u hat.
Well, one conclusion is that I wouldn't be able to use conjugate gradient methods, for example, which we've just learned how powerful they are, on the big matrix because it's not positive-definite.
It's symmetric.
I could use some other available methods.
I couldn't use conjugate gradient.
So if I wanted to use conjugate gradient, I better do this reduction to the definite system.
We're longer than I intended to spend on the simple example.
But if you see that example, then we'll be ready to move it to the wide variety of applications.
So let me just note that one section already up on the web called saddle point systems solves differential equations that are of this kind.
So we'll come to that, and we'll come to matrix problems, too.
It's a very, very central question, how to solve linear systems with matrices of that form.
In fact, I guess I could say it's almost a fundamental problem of numerical linear algebra is to solve systems that fall into that saddle point description.
I'll try to justify that by the importance I'm assigning to this problem in the next week.
OK.
Thanks for today and I'll turn off volume.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
Finite difference methods for initial value problems that we're coming to the end of and the solving large systems that we're coming to the beginning of was to talk today about matrices, because that language is just useful for everything.
So about the homeworks, Mr. Cho put in a long weekend grading the homeworks that were turned in Friday and preparing code and output to go up on the web page, probably late tonight or tomorrow.
So have a look to see and I hope that those codes will be useful for the future too.
Right.
So we'll maybe say more about the outputs and about the projects, which by the way, could grow out of that homework or could go in a totally different direction.
I'm thinking that the right time to say projects due would be after spring break.
So pretty much nearly immediately after the spring break would be -- and we'll talk about it more, but just so you have an idea of what's -- what timetable I had in mind.
So, matrices then.
In particular, finite difference matrices.
So that's the second difference matrix, k, and I'll frequently use that letter k as I did in 18.085 for that very, very important and useful matrix.
So what are its properties?
it's tridiagonal.
That's a very important property, which we'll see because that means that computations solving linear systems are very fast with a tridiagonal matrix.
It's symmetric.
All its Eigen values are positive, so I would say it's symmetric positive definite and those Eigen values and Eigen vectors -- so the Eigen vectors for this matrix turn out to be discrete -- not quite discrete exponentials, discrete sines, discrete sine function.
If I change the boundary conditions, I can get discrete cosines as the Eigen vectors or I could get discrete exponentials as the Eigen vectors by making it periodic.
Maybe I'll mention how to make it periodic and then I'll erase it again.
To make it periodic means that this second different centered at point 1 should look ahead to point 2 and look behind to point 0, but that'll be the same as point n, so I would put a minus 1 in that corner and this one similarly, it looks ahead, which really brings it around again, since we're sort of on a circle, brings it around again here.
So that matrix now with minus 1s added in the corner, I would call c, a circular.
So I'll leave the letter k there, but the right letter is c while these minus 1s are here to make it periodic.
By the way, they mess up that the tridiagonal.
It's no longer tridiagonal.
Also, it certainly got -- it's very sparse.
Mentioning sparse reminds me, if you're coding large matrices that are sparse, you should let MATLAB know that they're sparse.
So MATLAB lab has a whole -- it carries out operations, if you tell it it's a sparse matrix, then it only operates where the non 0s are located and it doesn't waste its time looking at these 0s, these 0s through all the matrix steps.
So using sparse MATLAB is important thing to know about it.
It just typically and s or an sp will appear in MATLAB commands.
For example, I'll just maybe fill it in here.
What's the sparse identity matrix?
The normal identity matrix would be i of n and the sparse identity matrix is sp, speye of n. Similarly, we would create k -- we could use 2 times speye of n as the diagonal of k and the two, the upper and lower diagonals -- and we could tell it these two entries.
Another thing to say -- what I said I wasn't going to do, I'll do because I hate to see a k up there while it's not right.
Two more points about this matrix.
It's singular now.
The determinate is 0.
Now I'm never going to take the determinate of a giant matrix.
That's a bad thing to do.
Much better to recognize that there's a vector, x, let's say and it's the vector of all 1s.
It's the vector of n 1s.
If you imagine multiplying this matrix by the vector of n 1s, what do you get?
You get all 0s.
So that vector of all 1s is in the null space, I would say, of a matrix.
Null space is just the vectors that get wiped out by the matrix.
cx is all 0.
So that vector, this matrix has some non 0 vectors in its null space.
I know right away then, its determinate is 0.
So the determinate of that is 0 and now that would tell me something about the Eigen values of the matrix.
It tells me about one Eigen value.
It's 0.
A matrix, like c, that has cx equals 0 -- I could also say that that cx equals 0 x.
That would actually be better, so that both sides are vectors.
So I'm realizing that that vector in the null space is an Eigen vector and the corresponding Eigen value is 0.
Otherwise, the Eigen values will all still be positive.
So this would be a positive semidefinite.
I would call that matrix positive semidefinite, the semi telling me that it isn't quite definite, that it gets down and has 0.
The matrix is singular, but still it's good to know where all the other n minus 1 Eigen values are.
They're all positive.
About the bandwidth -- let me go back to k now.
Because the bandwidth, strictly speaking, the bandwidth of c is very large.
the bandwidth is, if I'm looking for only one number, that number tells me how many diagonals I have to grow until I reach the last non 0.
So the bandwidth here is large and in general, the operation count, the amount of work to do, is going to grow with a bandwidth.
Of course, not having full bandwidth isn't quite the full story for this matrix, because it's so sparse.
I've got hundreds of 0 diagonals in between and just this one.
Anyway, this is a matrix with a large bandwidth, but a little deceptive.
Now here's the matrix with a -- back to k again.
I call the bandwidth just 1 here.
Really, maybe half bandwidth would be a better word.
The bandwith is the number of diagonals above or below -- take the maximum of the count above and the count below and in this case, both counts are 1.
1 diagonal above, 1 diagonal below, so really, half bandwidth, I would say, is 1.
Just some convention is needed there.
The crucial point is that the bandwidth measures the amount of work to do when you do elimination, as MATLAB will do, of course.
One other -- the thing about MATLAB -- so I'm often referring to MATLAB and I'm thinking of its backslash command.
The backslash, which solves ax equal b by just a backslash b.
So if it doesn't know these matrices are sparse, it will go through all the steps, not taking advantage of the fact that we've got all these 0s here.
If it does know that they're sparse, then it's tremendously fast.
Let me come back to that point.
Maybe actually backslash is smart enough to look to see whether the matrix, whether sparseness is available.
So I shouldn't have said -- I think maybe there's a lot engineered into backslash.
Actually, backslash, also engineered in there is the least squares solution.
If you give it a rectangular problem and, say, too many equations, so that you can't expect to have an exact solution, backslash will pick the least squares solution and much more.
That's probably the most used operation.
So I said something about the Eigen values and everybody sees that if I multiply this matrix by the values of u at successive mesh points, I'll get the second difference that corresponds to uxx.
Now this would cause this matrix -- so that tells me I'm taking a finite difference.
That plus tells me I'm going in the forward direction, so that's 1 at the mesh value to the right minus 1 of the mesh value at the center point.
Of course, this has to be divided by delta x and that by delta x squared.
So these are matrices, along with the backward difference and the center difference, out of which you build the basic finite difference equation, as you've done.
I want to make a comment here though, about now on this topic of Eigen values.
Eigen values for k really tell you the truth about the matrix k. The Eigen values of that matrix k start just a little above 0 and they go to a little before 4.
So the Eigen values for k -- so I put that here, the Eigen values of k are between 0 and 1.
They come very close to 4 and quite close to 0, depending on the size of the matrix of course.
Let me just do -- if I took the one by one case, its Eigen value is 2.
If I took the two by two case, its Eigen values are 1 and 3.
Notice nice properties there.
The Eigen values 1 and 3 are positive, as we said.
This matrix k has positive Eigen values, whatever side.
What's more, the 1 and 3 kind of interlace the 2.
So what I'm saying is, the Eigen value for that is in between the Eigen value for the two by two and the two Eigen values 1 and 3 are in between the three Eigen values that I would get for the three by three case.
Maybe I just write those down.
Those are useful numbers.
So k 2 as lambda equal 1 and 3.
The three by three one, I think the Eigen values are 2 minus root 2, which is smaller than 1, 2 which is in between and 2 plus root 2, which is larger than 3.
Of course, they're all between 0 and 4.
Just a comment.
How do I know they're between 0 and 4?
There's a somewhat handy little rule for getting the location of Eigen values, that's just worth knowing as a sort of general principle, but of course it can't tell you exactly where they are.
First of all, the fact that the matrix is the metric tells us what about the Eigen values?
So we learn a very, very important fact about the Eigen values from just looking at the matrix and observing that it's symmetric.
That tells us that the Eigen values are real.
They're real numbers.
Actually, it tells us something equally important about the Eigen vectors.
The Eigen vectors of the matrix are orthogonal.
The symmetric matrix has real Eigen values, orthogonal Eigen vectors.
That's a little bit of linear algebra to know.
Now why between 0 and 4, though?
There's there's this -- what's his name?
Gershgorin.
Gershgorin pointed out -- and it's a two line proof -- that every Eigen value is in one of his -- one or more of his circles.
So where are the Gershgorin circles?
A typical Gershgorin circle is centered at the number here, 2, that's on the diagonal and its radius is the sum off the diagonal, but you take absolute values.
Gershgorin wasn't doing any of the fine points that really locate the Eigen value.
So in this matrix, all the centers are at 2, the diagonals and the radii are 2, also 2, because that and that and absolute value make 2.
This first row makes a 1 and actually, that's what -- that by a little careful argument, is what gets the Eigen values not actually touching 0 or touching 4.
It takes a little patience.
All you could say from here -- all Gershgorin could say what would be, the Eigen values are in a circle, centered at 2, its radius is 2, so they're between 0 and 4.
Then it's that first and last row -- either the first or the last row would do to say that they're strictly positive.
Of course, the true second difference, I should divide k by delta x squared.
So the Eigen values will be divided by delta x squared divided by a small number, so they will really be much larger.
The ratio of the biggest to the smallest isn't affected by the delta x squared and that's a key number.
Let me put that maybe on the next board for k. So lambda max is about 5.
Lambda min, the smallest Eigen value is close to 0.
How close?
It's of the order of some constant over n squared.
Now I use the word condition number and that -- so let me write that down.
Condition number -- say c of k for condition number is the ratio lambda max over lambda min.
Of course, I'm using here the fact that k is a symmetric matrix.
Symmetric matrices are so beautiful that their Eigen values give you a reliable story here.
So 4 divided by this -- the main point is that it's o of -- it's a border 1 over n -- sorry.
When I divide by lambda min, that puts the n squared up in the numerator.
It's o of n squared, growing like n squared, and that condition number, somehow it measures how close the matrix is to being singular, because it involves this lambda min, which is the smallest Eigen value and would be 0 if it were singular and it's scaled by lambda max.
So if I'm multiply the matrix by 100, what happens to the condition number?
No change.
No change, because I'm doing lambda max over lambda min.
Both lambda max and lambda min would be multiplied by 100, but the ratio wouldn't be different So this is a useful measure and o of n squared, that's a big number if n is large.
A big numbers if n is large and that condition number, where does it come in elimination?
In elimination, the round off error -- roughly, the rule of thumb is that you would -- if the condition number is 10 to the eighth, you might lose eight significant bits in the back slide.
You could.
So this condition number measures how sensitive your matrix is to round off.
So that's a few thoughts about matrices and that matrix k in particular.
Now what about the other matrix, the first differenec?
The point I want to make about that matrix is, what about it's Eigen values?
What are the Eigen values of that upper triangular matrix?
They are, if you remember linear algebra -- but I can just tell you quickly -- the main point for a triangular matrix, the Eigen values are sitting on the diagonal.
So this matrix has the Eigen value minus 1 repeated n time.
That true fact is totally misleading, totally misleading.
The Eigen values for this triangular matrix don't even notice what I've got above the diagonal and somehow they can't give a reasonable picture of what the matrix is actually doing.
So maybe that's my warning here, that for a matrix which is absolutely not symmetric, right?
I mean, not at all symmetric.
For the center difference, which is -- what's the center difference?
I was going to say symmetric, but it's the opposite.
Center difference would be -- let's put delta centered down here.
Center difference would be I'd have a 1.
0s would go on the point that -- for the central value.
1 would multiply the forward value and minus 1 would multiply that and then I'd have 1/2 and then I'd probably have a delta x.
But the main point is, my matrix word look something like this.
Minus 1 and 1s on two diagonals.
Now we could find the Eigen values of that matrix.
Do you know anything about the Eigen values?
This is a chance for me just to speak for a few minutes about useful facts that you can tell about a matrix just by looking at it.
So what do I see when I look at that matrix?
Is it symmetric?
It's the opposite of symmetric, because the symmetric matrix up there, if I transpose it, it doesn't change.
If I transpose this one, I'll get some kind of a backward difference.
If I transpose this one, then the 1s and the minus 1s will reverse.
I'll get the negative.
So the rule for this center difference is -- so shall I -- how am I going to call center different?
Del centered, for the moment.
The transpose of that is minus itself.
So what does that tell me?
First, that matrix is -- it's the opposite of symmetric, but it's actually OK. What I mean by OK is, its Eigen vectors -- we're back to orthogonal Eigen vectors.
I didn't say anything about the Eigen vectors of del plus, but actually that was the biggest problem.
This matrix del plus has one Eigen value repeated n time and it has only one Eigen vector, not -- it doesn't even have a full set of Eigen vectors, much less orthogonal ones.
So that matrix is like -- you don't want to trust the Eigen value picture that you get from a matrix like that.
Here this anti-symmetric matrix can be trusted.
Its Eigen value picture is reliable.
It does tell you what's going on.
The Eigen vectors are orthogonal.
They're complex, actually.
Actually, they'll look a lot like our e to the ikxs.
So we don't panic when we see complex Eigen values.
The Eigen values are -- do you know what the Eigen values looks like for anti-symmetric matrix?
They're pure imaginary, just the way that when we took second differences -- maybe I'll just put here the center difference, the centered first difference, when we applied it to -- I want apply to eo to the ikx to find it's what factor comes out.
So I get e to the ik plus 1 k plus 1 delta x from the plus side.
The is the matrix I'm doing with 1 and minus 1.
So the minus 1 will get me minus e to the ik minus 1 delta x and of course, I factor out of that the e to the ikxs, so I'm left with e to the -- the thing that factors out is e to the i delta x minus e to the e minus i delta x -- and what's that?
That multiplies the eo to the ikx, the Eigen vector.
This is like the Eigen value and what do I say about that quantity?
Of course, it's 2 i sine delta x.
Pure imaginary and I should have divided by the 2, which would take away that 2.
So it's pure imaginary.
In reality and in this Fourier analysis, both are giving this understanding of what i is.
So that when we did this sort of operation, von Neumann's rule of following the exponential, we got something reasonable.
When we do it with this one, it's von Neumann that's reliable and the Eigen values that are not reliable.
So the Eigen values of this being all minus 1s is nonsense, doesn't tell us what the Fourier difference operator is really doing.
But von Neumann tells us what is truly going on.
Of course, that would be the same thing in which this minus 1 wouldn't appear and the 2 wouldn't appear.
So what would we get out of von Neumann?
So this is for delta plus.
I would factor out an e to the i delta x minus 1.
That's what would multiply it e to the ikx.
Oh, e to the i -- sorry.
Yes, that's right.
That would multiple e to the ik delta x.
Sorry, should've had delta x there, but I wasn't paying attention to that.
Here I was paying attention to this and it was pure imaginary.
Here I'm paying attention -- here von Neumann at least is paying attention to this and what's he seeing?
Not pure imaginary or purely real, off in the complex plane and that's really what the rights growth factor or the right number to associate with frequency k for this forward difference.
So I guess I'm saying that von Neumann does -- did the right thing to come up with these pictures for the growth factors.
Eigen values confirm that and really pin it down when they're reliable.
Eigen vectors and Eigen values are reliable when Eigen vectors are orthogonal and that's for matrices that are symmetric or anti-symmetric.
There's a little bit larger class of matrices that include orthogonal matrices, but beyond that -- actually, there's been a lot of discussion over many years of Eigen values and the -- for problems that are not controlled by symmetric or anti-symmetric matrices.
The alternative, the more refined idea of pseudo-Eigen values is now appearing in an important book by Trefethen and Embry, and with many examples -- OK, I won't pursue that.
Right.
So this is some basic fact about those matrices, all of which are one dimensional.
Now this board prepares the way to get into 2D and 3D.
So I just want to ask, what does the -- we didn't really do the heat equation or the wave equation in 2D, but we could have.
The von Neumann test would be straightforward, but now I want to think about the matrices.
What does the two dimensional second difference matrix look like?
What I'm going to do, just to look ahead, I'm going to use MATLAB's operation called kron, short for Kronecker, to create a 2D matrix out of this 1D center difference.
So if I think now about center differences, second differences -- so I'm approximating uxx plus uyy -- or really, I'm approximating minus uxx minus uyy, because that k approximates minus the second difference.
What do I -- what do my matrices look like?
What's their bandwidth?
How expensive is it to invert them?
These are the key questions.
What's the matrix k 2 d that corresponds to -- gives me second differences in the x direction plus second differences in the y direction.
So I'll write k 2 d for that matrix.
Let's get some picture of it.
First of all, let me imagine that I'm on a squred with a square grid.
Delta x in both directions Square grid, let me say n mesh points each way.
So n, I don't know whether I'm counting -- right now, I won't worry whether I'm counting the boundary ones or not.
Probably not.
So n -- I'll say n point, point n, n points -- so n delta x and in this direction n delta x.
So my matrix is a border.
It's of size n squared, the number of unknowns.
Now I will be a little more careful.
Here, let me take the boundary values as given.
They're not unknown.
So n in this picture is 4.
One, two, three, four unknowns there on a typical row.
Now I have to give them a new number -- five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 16 -- and n being 4, n squared is 16 for that particular square.
So my matrix is 16 by 16.
But somehow I want to be able to create it out of 4 by 4 matrices like k. So k 1 d, which I'm just going to call k -- kn will be the 4 by 4 one.
2 minus 1 -- so that's the matrix that gives me second differences along a typical row or down a typical column.
But now what am I looking for?
I'm looking to do both -- second differences in a row and a column.
So if I pick a typical mesh point, like number 11.
Mesh point 11 -- let me blow up this picture here.
It's going to be influenced, mesh point 11, by 10 and 12.
The second difference is in the x direction and by 7 and 15, notice those are not so close to 10 or 11.
The second differences is in the y direction, so let me blow that up.
So here's mesh point number 11 corresponding to row 11 of k 2 d. So I guess I'm asking what role 11 of k 2 d will look like.
So here's mesh point 9, 10 and 12, so I have a second difference in the x direction from uxx but minus uxx -- that means I can put a minus 1 there, a 2 there and a minus 1 there.
Now I have the same in the y direction with 15 and 7, so I have a minus 1 in column 15, a minus 1 in column 7, so I have a 4 minus 1 and then two more for the center gives me a 4.
So a typical row will have -- it'll be sparse, of course -- it'll have a minus 1 in positions 7, a minus 1 in position 10, a 4, a minus 1 and a minus 1 over there in position 15.
That's a typical row of k 2 d. It adds to 0.
If Gershgorin got his hands on this matrix, he would say that since all the rows -- the 4 will be on -- will the 4 be on the diagonal?
Yes, the 4 will be on the diagonal all the way.
I guess let's see a little bit more clearly what the matrix k 2 d looks like.
This is probably the most studied matrix in numerical analysis because his its a model of what stays nice and what gets more difficult as you move into two dimensions.
So some things certainly stay nice.
Symmetries -- so properties of k 2 d, k to d will be -- it'll be symmetric again.
It'll be positive definite again.
So what does k 2 d -- it's 16 by 16 and a typical row looks like that.
That's a typical interior row.
What does maybe -- if I took the first row, what does the very first row look like if I put row one above it?
Let me draw row one.
So what's the difference with row one?
It's these are boundary values.
These are not -- these are going to show up on the right hand side of the equation.
They're known numbers.
They're not -- they don't involve unknown things, so the only neighbors are 2 and 5.
So I'll still have the second difference, minus 12 minus 1 and minus 12 minus 1 -- still this five point molecule with four at the center, but there won't be so many neighbors.
There'll be the neighbor at the -- just to the right.
Neighbor number two and there'll be neighbor number five a little further along and that's it.
It's like the 2 minus 1 boundary row.
It's a boundary row and a boundary row hasn't got as many minus 1s because it hasn't got as many neighboring unknowns.
That boundary row would have three neighbors.
Row two would have a 4 minus 1 minus 1, but not -- nobody there.
So now I'm going to try to draw k 2 d. Let me try to draw k to d. I can do it.
k 2 d will have, from the uxx, the second differences along the rows, it will have -- I'll have a row, row one, it'll have another k on row two.
It'll have a k on -- a k for each row.
These are blocks now.
That's of size n by n. So all of those are, so the whole thing is n squared by n squared.
Actually, I'll stop here.
If I wanted to create that matrix with the same k on -- it somehow the identity is in there and the matrix k is in there and that's -- let's see.
It's one or the other of those.
I guess it's the first one.
So what's this Kronecker product?
Now I'm saying what this construction is.
It's very valuable, because it allows you to create matrices.
If you created k as a sparse matrix and i as a sparse matrix, then the Kronecker product would be automatically dealt with as a sparse matrix.
So what's the rule for a Kronecker product?
You take the first matrix, which is the identity.
The rest 0s.
So that's i.
Then -- so that's 1 d and then you multiple -- each entry in i becomes a block with entry aij times this guy k. This'll be the 0 block, this'll be the k block, this'll be the k block.
I take those 1s and multiply k and those 0s and multiple k and that's what I get.
So you see now the Kronecker product is a bigger guy.
If matrix a was p by p and matrix b was q by q, then the Kronecker product would be p times q by p times q.
It would be square again.
It would be symmetric if a and b are symmetric.
Actually, it would have various nice properties.
Its Eigen values would be the products of the Eigen values of a times the Eigen values of b.
It's a very handy construction and here we saw it in a pretty easy case where a was only the identity.
Let me do this.
What does the Kronecker rule produce here?
So I take this first matrix and I put it in here, 2 minus 1 minus 1, 2 minus 1 minus 12.
So that's the matrix k and now each of those numbers multiplies the second thing, which here is i.
So it's 2 i minus i minus i 2 i minus i minus i minus i minus i 2 i and 2 i.
Now that is the second difference matrix that does all the columns.
When I add those -- so now I'm going to add that to that.
They're both size n squared.
They give me k to d. So the neat construction of k 2 d is Kronecker product kron of ik plus kron of ki.
That's the matrix and let's look at what it actually looks like.
We're seeing it block wise here.
We saw it row wise here and maybe now we can take one more look at it, assemble it.
So k 2 d -- I plan to add that matrix to that matrix to get k 2 d. So it will have -- since they both have 2s on the diagonal, it'll have 4s on the diagonal, so 4s all the way, but it'll be block wise.
Up here, this block is k plus 2 i.
So that block is -- goes down to 4.
Now the k contributed minus 1s next to the diagonal, I guess that's it, right?
k plus 2 i has that block as -- that's the one, one block.
Why is it -- so we're seeing -- did I get it right?
Yes, so we're seeing the neighbors to the right and left, but now let me bring in -- only now comes the neighbor above or below and that comes from this off diagonal block, minus i, which is then minus 1s to minus 1s.
These are all 0 blocks.
Here will be a minus the identity blocked.
Down another block, the diagonal blocks are all the same and another one, another minute the identity.
Now we're getting to interior mesh points, where we see typical rows.
So a typical role has the 4s on the diagonal, the minus 1s left and right and the minus 1s far left and far right -- and of course, what I'm going to ask you is, what's the bandwidth?
We're coming to the key point now.
What's the bandwidth of this matrix?
I only have two non 0s above the diagonal on a typical row, but I have to wait n diagonals before I get to the second one.
So the bandwidth is n because I have to wait that long.
Then the operation count -- if I just do ordinary elimination on this matrix, the operation will be the size of the matrix times a bandwidth squared.
We can easily check that that's the right count of operations for a banded matrix.
This is operations on a banded matrix.
So what do we get?
The size of the matrix is n squared.
The bandwidth is n and it gets squared so we get n to the fourth.
It's getting up there.
If n is 1,000, we've got a serious sized matrix here.
Still a very sparse matrix, so it's not like give up, but the question is, does the matrix stay sparse as we do elimination?
That's the center of the next lecture.
Can we organize elimination?
How closely can we reorganize elimination to preserve all these zillions of 0s that are between here?
They're easy to preserve down here, way up there, but in this intermediate, those diagonals tend to fill in and that's not a happy experience.
And It's even less happy in 3D.
So let me just do this same calculation it for 3D and then we're done for today.
So k 3 d -- you might like to think how k 3 d could be created by the kron operation.
Let me just imagine it.
So what's k 3 d looking like?
I've got three directions, so I have a 6 in the center and 6 minus 1s beside it and so 6 goes on the diagonal of k 3 d. 6 minus 1s go on a typical row and how long do I have to wait until I reach the last one?
So I'm again going to do the size -- which is -- the matrix size is going to be like nq and what's the bandwidth going to be like?
Maybe I'll ask you now.
What do you think for the bandwidth?
How long -- if I just number it in the simplest way - and that'll be the key next time, is there a better numbering?
But if I just number along rows until the rows fill up a plane of rows and then I move up to the next, the z direction to the plane above, then I have to wait -- this was the x and y, so this gave me a bandwidth of n, but that's not the bandwidth because I have to wait until I've finish the whole plane, until I go up to the next plane and catch this chap, so the bandwidth is n squared.
Then the operations, which are size times bandwidth squared, we're up to n 7 and that is a really horrifying exponent to see into the seventh.
That means that even for a moderate -- a problem in 3D with a moderate number, say 1,000 unknowns at each direction, we have -- we're looking, in theory, at a cost that we can't afford.
Of course, there's a lot to do here.
So there's a lot to do if I stay with direct methods and that's what I'll do for the next couple of lectures.
Then it's really in 3D that iterative methods become essential.
I mean, this one, I could do those by direct methods, 2D, by a smarter direct method than any I've tried here, but in 3D, even smarter elimination is facing a serious exponent and loses to iterative methods.
So that's what's coming, actually.
So today's lecture, you see, with the simplest possible matrices, what the central questions are.
Thanks and I've got homeworks coming back from Mr. Cho and I'll collect any that are ready to come in and see you Wednesday.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and OpenCourseWare in general is available at OCW.MIT.edu
OK.
So I'll start a little formally.
That this is 18086 in the spring semester of 2006.
And we're lucky to have it videotaped for OpenCourseWare.
So let me begin by remembering the main topics from what you might call the first half 18085 of fall course.
And then most important to us, so it gets a star, are the main topics for this course.
So 18.085 began with applied linear algebra.
The discreet equations of mechanics, and physics and engineering.
And the type of matrices that involved, so we learned what positive definite matrices are.
Then the center of the course was differential equations, ordinary differential equations.
So that 1D, partial differential equations like LaPlace.
That was 2D, with boundary values.
So that lead to systems of equations.
And then the third big topic was Fourier methods.
All of Fourier ideas: series, integrals discreet transform.
So that's like the first half of a course.
Well that's 18085.
OK. And here we are.
So this course has also three major topics.
The first, the one we start on today is differential equations.
That start from initial values.
So I'm thinking of the wave equation, where we're given the displacement of velocity.
The heat equation.
Black-Scholes equation, which comes from finances is a version of the heat equation.
Ordinary differential equations that's going to be today.
And non-linear equations, Navier-Stokes ultimately.
But not heavily Navier-Stokes.
So this is where we begin.
And then a second big topic is how to solve.
Because this is course is really applied math and scientific computing.
The second big topic is how do you solve a large linear system ax equal b?
Do you do it by direct methods on limitation with renumbering of nodes.
And there are lots of tricks that makes elimination very successful.
Or do you do it by iterative methods?
Multi-grid is one.
So this is really modern scientific computing we're talking about.
We're really at the point where people are computing.
And then the third topic which is a little different is optimization, minimization.
Linear programming would be one example, but among many, many.
So that's a major area in applied math.
OK.
So this is our topic differential equations.
And I was asked before the tape started about my own background.
And my thesis was in this topic.
Because a key question will be stability.
Is the difference method stable.
You'll see that that's a requirement to be any good.
You really have to sort out what, and that usually puts a limitation on the time step.
So we're going to have a time step.
And it may be limited by the requirement of stability.
And the fact that stability and convergence success of the method are so intimately linked.
There was s key paper by Lax and Richtmyer that we'll look at.
It's known now as the Lax Equivalence Theorem, stability and convergence.
We'll see it in detail.
Actually when I was a grad student by good chance there was a seminar, and I was asked to report on some paper and it happened to be that one.
And looking back thank gosh I was lucky.
Because that paper, the Lax-Richtmyer paper with the Lax Equivalence Theorem set the question of stability.
And then years of effort went into finding tests for stability -- how do you decide is the method stable.
So we'll see some of that.
OK.
Since then I've worked on other things well wavelets would be one that relates to Fourier.
And other topics too.
And lots of linear algebra.
But it's a pleasure to look back to the beginning of [UNINTELLIGIBLE].
OK.
So that's the OpenCourseWare site which does have the course outline, and other information about the course.
Some of which I have mentioned informally before the class.
This is our central website with no decimal in 18086, which will have notes coming up, math labs things, problems.
That's our content.
It will be the 086 page.
OK.
So that's the course outline in three lines.
And then web page and a handout will give you half a dozen sub-headings under those.
OK.
So that's the course.
And actually it's the center of scientific computing.
There would be other courses at MIT covering material like that, because it's just so basic you can't go without it.
All right so now I'm ready to start on the first lecture, which will be ordinary differential equations.
And I won't always have things so clearly and beautifully written on the board, but today I'm better organized.
So here's our problem ordinary differential equations.
So we're given initial values u at t equals 0.
OK.
So there's always an initial value here.
The question is what's the equation that makes it evolve?
OK.
So often we'll think in terms of one equation.
The unknown will be u, u of t. Or in reality there are typically N equations.
N might be 6 or 12 or something in a small problem.
But also N could be very large.
We'll see how easily that happens.
OK. And often to discuss stability or understand a method and try it the linear cases the natural one.
So I'll use au with a small a to make us think that we've got a scalar, or a capital A to indicate we're talking about a matrix.
So these are linear.
These are likely to be not linear.
But there's a natural match between the number a their corresponds to the partial of f with respect to u.
And of course we see it.
If that was f then its derivative is indeed a.
And in this matrix case, there would be a Jacobian matrix.
If we have N right hand side, because I have Nu's, and the matrix that comes in is the derivative of right hand side and equation i with respect to unknown j.
So it's a N by N matrix, and of course this is the case where it's a constant.
Yeah I should have said, these are not only linear but constant coefficients.
So we know of course the solution, it's an exponential e to the at times u0 the initial bunch.
So we know everything about it.
Nevertheless, it's the best test case for difference methods.
And let me just comments that we do have to pay attention to the possibility that a could be complex.
Mostly I'll think of a as a real, and often negative.
Often equation will be stable.
A negative a means that e to the at decreases.
A negative definite matrix, negative eigenvalues, mean that the matrix exponential is the case.
And if that matrix is symmetric, those eigenvalues are all real, that's a key case.
Symmetric matrices have real eigenvalues, and they're the most important most attractive class.
And a negative definite matrix might come from a diffusion term like second derivative, d second u, dx square.
But a convection term du dx from the first derivative will not be symmetric.
In fact maybe it'll be anti-symmetric.
It will push the eigenvalues into the complex plane.
And therefore we have to pay attention to the possibility of complex a.
As you know, the whole point of eigenvalues is that we can understand matrices to a very large extent by diagonalizing, by reducing them to N scalar equations with the eigenvalues as the little a's in those scalar equations.
So anyway that's the set up.
Now I want to say something about methods.
I want to introduce some of the names.
What am I thinking here?
This topic, today's topic, of ordinary differential equations is not going to last long in 18086.
I might still be discussing it tomorrow.
The variety I mean.
But not much after that.
We'll soon be on to partial differential equations.
So I want to sort of tell you about ordinary differential equations well in kind of an organized m so that you know the competing methods.
And also so that you know this key distinction.
Nonstiff is a typical equation, u prime equal minus 4u would be a nonstiff still, completely normal equation.
And for that, those are they the relatively easy ones to solve.
We can use explicit differences.
And I'll say what that word explicit means.
What I want to do is just like help you organize in your mind.
So nonstiff are the average every day differential equations.
For those we can use explicit methods that are fast.
And we can make them accurate.
And explicit means -- so this is the first meaning -- that I compute the new U, and I'll use capital U, at time step and n plus 1.
So that's capital U at the new time.
Natural to call that the new time.
Can be computed directly from U at the old time, maybe U at the time before that, maybe U at the times before that.
In EE terms it's causal.
The new U comes explicitly by some formula that we'll construct from the previous computations.
It's fast.
Of course, I not only use Un but I used f of Un.
So I should say from Un itself and from f of Un at time tn, and similarly for earlier times.
You see how fast that is?
Because the only new calculation of f is this one.
The previous step produced Un, and the new step, this step, we plug it in to find out what the slope is.
And that may be the expensive calculation, because f could involve all sorts of functions, all sorts of physical constants.
It can be complicated or not.
but if it is, we only have to do it once here.
that's explicit.
Now what's the opposite?
Implicitly.
So the point about implicit methods is the difference equation, the formula, involves not just Un plus 1, the new value, but also the slope at that new value, at that new unknown value.
So it means that an implicit equation could be non-linear.
Because if f is not linear, then this term could involve cosines, or exponentials or whatever of the unknown Un plus 1.
So that way you can't do it as fast.
So this is definitely slower per stiff.
But the advantage and the reason it gets used for stiff problems is that on stiff problems it will allow a much larger stiff.
So it's this constant trade-off of speed versus stability.
These are less stable as we'll see, and we'll get to see what stability means.
These are slower but more stable implicit methods.
OK.
I didn't get to say what stiff is.
I'll say that in a moment.
OK.
So explicit versus implicit.
Un plus 1 immediately or Un plus 1 only by maybe using Newton's method to solve an equation, because Un plus 1 appears in a complicated formula.
And we have an equation to solve.
OK.
So now this part of the board begins to identify different methods.
And I'll follow the same two column system that non-stiff versus stiff.
So this will be explicit and those will be implicit.
OK.
So that one method that occurs to everybody right away is Euler's method.
And that will be the first method we'll construct of course.
So that's the first idea anybody would have.
It has the minimum accuracy.
It's first order, the order of accuracy, will be an issue for all methods.
And Euler has p equal 1, order p equal 1, first order.
So it's too crude.
I mean if you wanted to track a space station with Euler's method, you would lose it real fast, or you would have to take delta t so small that it would be impossible.
So Euler's method is the first one you think of, but not the one you finish with.
And similarly on the implicit side, backward Euler, we'll see is again the first implicit method you think of.
But in the end you probably want to do that.
OK.
So those are two specific methods that are easy.
The first ones we'll write down.
Now then comes families of methods, especially these Adams families.
So Adams-Bashforth is explicit, Adams-Moulton are implicit, and the coefficients are in books on numerical analysis and will be on the web.
And I'll say more about them of course.
And see the first ones.
So what Adams-Bashforth how does does it differ from Adams-Moulton?
OK.
So those are multi-step methods.
Number two was multi-step methods.
To get more accurate we use more old values.
So Adams-Moulton and Adams-Bashforth, or especially Adams-Bashforth, would use several old values to get the order of accuracy up high.
OK. Now this third category.
Of course actually Euler would be the first of the Adams-Bashforth, and backward Euler might be early in Adams-Moulton.
Or maybe backward Euler is early in backward differences.
OK.
So what I want to say is Runge-Kutta is like a different approach to constructing methods.
You might know what some of these already.
And it's a one-step method.
That's a method that just produces by sort of half-steps you could say.
It gets to Un plus 1.
But it's explicit.
And the code in math lab the code OD45, or maybe I should say the ODE45, is like the work course of ODEs.
I guess I'm hoping you'll try that.
You'll find ODE45 and I'll try to get on to the web.
But you could just easily discover the syntax to call it, and apply it to an equation.
Yeah I'll get examples on the web, and please do the examples.
So the 4 or 5 5 typically means that it use a 4th order Runge-Kutta, so that's pretty accurate.
To get up to a 4th order means the error after a up a time t equal 1 say is delta t to the 4th.
So by cutting dealt t in half, you divide the error by 16.
And then it also uses a fifth order one.
And maybe I can point to these words.
What makes ODE45 4 or 5 good, successful.
I mean how does it work?
It slows down or speeds up.
It speeds up when it can, it slows down when it has to.
And speed up means a bigger delta t, slow down means a smaller delta t for the sake of stability or accuracy.
Yeah.
So if the equation is suddenly doing something, if the solution is suddenly doing something, you know, important and quick, then probably at that period delta t will get reduced automatically by ODE45.
It'll cut it in half, cut in half again, half again.
Every good code is constantly estimating using internal checks to estimate what error it's making.
About the error.
Just a quick word about the error.
So ODE45 it will, unless you tell it otherwise, the default accuracy that it will adjust for is relative.
So ODE45 will have a relative accuracy.
So can I just squeeze in a few words of 10 to the minus 3.
It'll shoot for that.
And an absolute accuracy of 10 to the minus 6.
So weird will try to keep the error.
It will plan to keep the error, and it'll tell you if it has a problem below 10 to the minus 6.
And you could set that differently.
OK.
I hope you experiment a little with ODE45.
And you can either plot the solutions, some exponential.
I mean push it.
Like let delta t be pretty big, and see where it breaks down.
Well I guess ODE45 is engineered not to break down.
If you try delta t, it'll decide on what delta t it can do.
So we'll also code, just quickly, some methods by ourselves, and see for a large delta t what problems could happen.
OK. And then backwards.
This says backward differences.
That's the category of implicit methods for stiff equation, and backward Euler would be the very first.
And ODE15S is the code that is the most used code perhaps for stiff equations.
And it will do the same thing, it will vary delta t. It was will vary the order.
So if has to slow down, it may slow.
And if things are happening too quickly, it may change to a low order, safe, secure method.
When things are, you know, when the satellite is buzzing out in space and it can take giant steps or it can have high order accuracy, say in astronomy of course they would go up to 8th order and high tracking estimating where stars would go or planets, these will do that automatically up to a certain point.
And we could write codes, and of course people have, for Adams-Bashforth and Adams-Moulton varying delta t varying e. I guess here's a comment.
If I had given this lecture yesterday, I would have emphasized number two, Adams-Bashforth and Adams-Moulton, because books do it and I basically learned this subject from the major textbooks on ODEs.
But I had a conversation with one of the computational scientists in the math department, and that changed the lecture.
Because I learned that although books tend to discuss those at the Adams method, in practice these methods three -- Runge-Kutta and backward differences -- are the most used.
And that's reflected in the fact that these two major codes that are available in math lab are in the number three category.
OK.
So now that's a picture with name but not details right.
I And I haven't even said what stiff means.
So somewhere I've written something about stiff.
Yeah.
OK.
So if I had only e to the minus t in the solution, then the solution would decay at that rate either the minus t. That time step would adjust to keep it accurate.
And if I have only e to the minus 99t, then again that of course is a faster decay, much faster decay.
So the time step would adjust to be much smaller, probably about 199th or something.
Anyway much smaller to capture within each step the correct behavior of either the minus 99t and no problem.
So in other words, it's not the minus 99 that makes it stiff.
If it was only the minus 99 -- in other words if I had a equal minus 99 up there, I wouldn't call it a stiff equation.
What makes it stiff is the combination here.
You see what's going to happen as time gets anywhere, this is going to control u of t. The decay of u of t is going to have that time constant 1.
But this is going to control.
So I'll say but this would control delta t for explicit methods.
And I just picked minus 99, but it could have been minus 999 or far worse.
So it could be very stiff.
This word stiff and identifying this class of problems, which now is familiar to everybody who has to compute, didn't come, you know, with Gaussian and so on.
It came much more recently.
But it's become very important now to distinguish stiff from non-stiff.
So where does stiff problems arise?
Well you can see how they arise in chemical processes with very different decay rates, biological processes, control theory, all sorts of cases where there's a big dynamic range of rates.
And they also arise in system, because the eigenvalues could be minus 1 and minus 99.
I can easily create a matrix that has those two eigenvalues.
So it would be a system.
Let me create such a matrix.
I think probably if I put minus 50s on the diagonal, that gives trace.
Sum down the diagonal is minus 100.
And that should equal the sum of the eiganvalue.
So, so far so good.
And now if I want these two particular numbers, I could put 49 here.
So that matrix has those two eigenvalues.
I mean this is the kind of matrix, well you might say ill conditioned would be a word that you hear for a matrix like this.
The condition number is the ratio.
So the condition number of this matrix would be the ratio 99 over 1.
And that condition number is a number that math lab computes all the time every time you give it a matrix problem like this.
It estimate, not computes.
Because to compute the condition number requires solving an eigenvalue problem, and it does not want to take forever to do that exactly.
But it gets an estimate.
So that's only a moderate condition number of 99.
That's not a disaster, but it makes the point.
OK.
So that's what stiff equations are.
OK. Now I got one more board prepared.
I won't have this everyday, but let me use it while I got it.
OK.
It's only prepared in the sense of telling us what we're going to do.
So this is what's coming: a construction of methods, what stability is about and what's convergence.
So let me begin, let me get Euler constructed today.
And backward Euler.
So Euler's method will be the most obvious method, equals f at Un.
So in the model case it's aUn.
So that's Euler.
So that tells us then that we could rewrite it.
Un plus 1 is explicitly 1 plus a dealt t, Un.
Right.
I've moved up the delta t and I moved over the Un, and it's simple like that.
Let me stay with Euler for a moment.
After n steps, every step just multiplied by this growth factor.
Let me refer to that as the growth factor even if it's as I hope smaller than 1.
So Un after and n steps is this thing has multiplied n times Uo.
And what is the stability now?
Stability is suppose a is negative typically.
That means often we'll just think of that model a negative, so that the differential equation is completely stable.
Right.
The e to the 18 with a negative is perfect.
The question is this one perfect?
So stability will be is this number below 1?
That's the test.
Is that we could argue about is probably going to let it equal 1.
Maybe I'll let it equal.
So 1 plus a delta t smaller than 1.
That'll be the stability requirement for forward Euler.
And what's the a delta t?
What's the boundary?
What's the critical value of a delta t?
Remember a negative.
How negative can it be, or how negative can this combination be?
You see it's a combination a delta t. So let me put above here what the stability condition is.
So the stability condition on Euler is going to be -- so do you see what it is?
How negative can a delta t be?
Go ahead you.
Well let's see.
So if a delta t, this is a negative number, so this is dragging us down from 1 always.
Right; a delta t is like subtracting away from 1.
So we're here with the 1.
And a delta t is pulling us this way.
And how far, at what point are we going to meet instability?
Yeah.
When we hit minus 1.
This is the 1 plus a delta t that I graphed.
So it starts at 1 with delta t equals 0.
And as I increase delta t, it move this way.
And eventually it hits there.
And the limit will be at a delta t equal minus 2.
Because if it carries me 2 from 1, that carries me to minus 1.
And if I go further, I'm unstable.
And you could easily check.
That Euler will blow up like crazy if you just go a little beyond that limit.
Because you're taking powers of a number bigger than below minus 1.
OK.
So there's the stability limit for Euler.
And that's a pretty unpleasant restriction.
We will have other stability limits.
We'll have some stability limits over here, but we're looking for numbers better than 2.
This Adams-Bashforth will give us numbers worse than 2.
We'll see those methods first thing next time.
OK.
If I drew a picture then.
Let's see have I got another board here.
Nope.
If I drew a picture of good Euler, so Euler doing its job, the true solution is decaying like that, and what does Euler do?
Forward Euler says, it looks here, it looks at the slope, that's all it knows, it goes maybe that far.
That's a small delta t. That's an OK method.
You see it's not very accurate of course.
It's lost all the curvature in the solution.
So this was the true solution e to the at.
And this is the 1 plus a delta t to the power well t over delta t times.
Well this was only one step, so that would just be 1 plus a delta t. OK. where the bad, the unstable one, if I just try to draw that, the true solution is e to the at.
I started with that value and the right slope.
But I take two big a time step and I'm below minus 1.
So that was a case of a too large delta t. OK. Now I've got one minute left to mention backward Euler.
Well that won't take me long.
I'll just change this to Un plus 1.
So I'm evaluating the slope at the new time.
So this is now backward Euler or implicit Euler.
OK.
So that changes this equation.
Oh let's find out the equation for implicit Euler.
OK.
I only have one more moment, I'll just pop it in this little corner.
So now here's backward Euler.
Alright.
You can tell me what backward Euler is going to be.
So Un plus 1 minus Un over delta t is aU, n plus 1.
So let me collect the n plus 1's.
So I have one of them.
And then minus a delta t. Right.
And when I bring it over to the left side, and here I just have Un.
OK.
So what happens at every step now?
I divide every step as a division by this.
So I could bring this into the denominator.
So Un is that division times U0.
And you see the stability?
What's the size of this growth factor?
Remember I'm thinking of a as a negative.
I'm dealing with a stable differential equation.
And so what's the main point about this growth factor, this ratio in parentheses?
It is?
If a is a negative, what do I know about that number?
Think of a as minus 2 let's say.
So then I have 1 over 1 plus 2 delta t, because it's minus a there.
And it's going to be smaller than 1.
Right.
It's going to be stable.
So Euler is actually absolutely stable, backward Euler.
Backward Euler has this property of a stability.
There is no stability limit on negative a, no problem Or even pure imaginary a, no problem.
Always stable.
So that shows how implicit methods can be greatly much more stable than explicit, just by comparing backward to forward to backward Euler.
But just to remind you the price.
The price is that this equation has to be solved -- well it wasn't any trouble in linear case.
In the linear case we're just inverting a matrix for a linear system.
So backward Euler for linear system, big linear system, is a matrix inversion, well a matrix linear system to solve.
But for a non-linear problem, it's serious work.
And yet you have to do the work if the explicit method gives you such a small delta t that it's impossible to go with.
OK.
Thanks for your patience on this first lecture.
So obviously I'm going to have one more discussion of ODEs, in which we construct these methods, see their stability and their convergence.
And then we move on to the PDs.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
This is my lecture on section 6.1 of the notes.
Really, the central problem of numerical linear algebra is solving ax equal b by elimination.
That's what is used much more often than anything else, but we're talking about sparse matrices, large sparse matrices.
That's what we have in mind for a long time now.
Probably all lectures coming up, thinking of those matrices and this is the prototype of a large sparse matrix.
The main point of the lecture is, if you take the unknowns in whatever order they're given -- here I took them numbering along the rows -- one, two, three, four, five, six, seven, eight and so on -- you got to stop and think or get the computer to look at reorderings, because the reorderings can make an enormous difference in the fill in.
So this is -- actually, the key words for this lecture is and the key concept is the fact that as elimination goes forward, a lot of these 0s become non 0s.
Let me see it happen, first of all.
So this is the whole idea of fill in.
What's going to be the very first entries that are filled in?
Let's see.
I guess that the very first row operation, if you took the matrix like this and did elimination, 4 is the first pivot.
You would add 1/4 of the first row to the second row to make that a 0, right?
Then what would happen?
This would change that number, but this would put a new number in there.
So a new number would appear when we do that first row operation and then also working with this pivot, I would operate on that row to make this a 0 and then this would change, that would fill in, right?
I guess this'll already has something to it.
That won't be new.
So each of those steps, each of those row operations filled in a non 0 and reduced the sparsity of the matrix, and the sparsity is what we're living by.
Now if I continue and continue, we'd need MATLAB, to use spye in MATLAB to see what the picture would look like.
It's very interesting to see it.
Spye shows you the non 0s.
We're very much focused on what's 0 and what isn't 0 in matrix entries.
We actually keep track of that, the structure of a matrix, as well, of course, as separately keeping track of the actual numbers.
One place, we're keeping the actual numbers, but in the decisions on ordering, we're only interested in 0s and non 0s.
So let me look even more at this fill in.
Now you saw it happen up there.
Let me just look at a single fill in step.
So I have a row j and a row i -- and this is going to be the pivot that I'll use to eliminate this, but in this row, there's another non 0 and in this row, the case entry is not non 0.
It's 0.
So this is what we start with prior to elimination, just as we did there.
I just want to see focus on small on the fill in occurring.
Of course, this isn't on the diagonal.
This is in column k, which is somewhere far out.
In that case, some are close by.
In that case -- but we're just -- this is on the diagonal.
That's the pivot, the j'th pivot, that I'm ready to use, so I subtract a multiple of that to remove that.
So what is elimination -- will not change the pivot row and it will produce a 0 there, that's the point.
Then the real point that I'm making is that when I multiple this by the right number to produce 0 and do this subtraction, this 0 disappears, that fills in.
That's some aik new that -- the new value in row i, column k is not 0. so we see that as we saw it for numbers.
Here we're seeing it symbolically.
Let me see it graphically, because everybody sort of -- to visualize the big picture.
I think everybody finds the graph description of the non 0 structure the simplest to imagine.
So the graph for this little problem would be -- so what's the graph that tells me where the non 0s are?
Again, it doesn't tell me what numbers they are, just tells me where they are, which positions.
It's only the positions that I worry about in deciding on the ordering to use.
Then I execute using the actual numbers, but in deciding on the ordering, it's just the position.
So I think of my graph as having a node for every row.
So that's node j for row j.
This is node i for row i.
Somewhere we'll have a node k. So at the start, so those are the nodes.
What are the edges?
There is a non, so the edges come from the non 0s in the matrix.
So there's an edge from j to k, because this is not 0.
There's an edge from i to j or j to i. I'm not worrying about the direct -- this is not a directed graph right now.
So this is the start, but there's no edge here initially, because aik is 0.
Now in this graph world -- it's so small, we can just focus on the part that's going to change a little.
The rest of all the other rows have their nodes and all the other non 0 entries have their edges.
So this is a tiny part of the big graph.
What happens to this tiny part?
So again, I want to see the same elimination step, just this way in the graph world.
So after, what edges are there?
So again, it's j, i and k -- and after I do elimination, j is still connected to k. That's still there.
No change.
j is no longer connected to y, because elimination killed that edge, but elimination created a new edge, i to k. So that's essentially what elimination does.
Maybe we can say it in words.
When this edge disappears, at the same time we create new edges from connecting any two nodes that were both connected to j.
You see that point?
If I'm using j as the pivot row and I look on that row and say, what non 0s are there?
What edges are connected to j and I see one -- and then, when I do elimination, I'm going to introduce, exactly as has happened here, a new edge between any pair that were both connected to j.
So we're kind of fighting against creeping filling.
The question is, what order should we do these eliminations?
So now think of a larger graph.
Actually, I guess we could somehow think of that -- we can try to visualize it, but maybe not succeed.
So the graph for that problem -- could that seve as the graph?
I guess it could, really.
I would remove -- I'd [?
whiz ?]
away the -- if I'm given the values on the outer edges, then I would remove all those, but essentially this is the graph for that matrix, right?
If I number each of the nodes 1 to 16 and I would do the numbering the way I did to produce that matrix -- now the whole question is, if I do some different numbering, will -- that'll shift around.
That'll just be a permutation matrix, right?
If I do a different numbering -- so matrix wise, I have my matrix k to d -- what happens matrix wise for -- if I do a numbering, a different numbering, a reordering?
That just permutes the unknowns and the equations.
So really, what happens is some permutation multiplies k to d on the left and its transpose does the same thing to the columns on the right, because this permutation will change the order of the equations.
This permutation will reorder the columns to come back, to bring the diagonal back onto the diagonal, but in a different order.
We won't know it with all those 4s, but that will have, again, all the 4s on the diagonal, but the minus 1s are moved around.
Of course, none have disappeared or been created by that.
It's just that's giving us a new order.
So that the reodering matrix wise What's the reordering graph wise?
Graph wise would be, instead of this numbering of unknowns -- one, two, three, four, five, six, seven, eight and so on -- nine, ten, eleven, twelve, thirteen, fourteen, fifteen, sixteen -- we have a different order, different numbering.
The computations, the computer science aspect, you could say, was not going to be difficult.
Actually, we should think and maybe we'll think in a minute, from the computer science aspect, the actual execution -- I'm probably not -- certainly not going to write out a permutation matrix.
That's ridiculous.
I'm not going to draw a graph and keep that -- I'll keep some list of positions for non 0s, so we'll say a word about that list later.
Here I'm trying to say, how do we get an idea of what effective reordering is and what a good reordering would be?
So here, this is a specific reordering that I want to mention right away, because it's so straightforward.
You can get the idea of the red-black ordering without any big computations.
This is the -- fundamentally, the winner.
This is what people use all the time and MATLAB's backslash is going to have that available.
Inside this, the codes that are called are the column or symmetric matrix, approximate minimum degree.
So that's what this AMD means -- approximate minimum degree.
So that's the basic idea that I'll come back to as soon as I've just drawn the red-black picture.
It's so important that people don't stop thinking.
so another sort of high level picture of how to do an ordering is look for graph separators.
That's become a big step in the world of computational graph theory, and that leads to something called nested dissection, which is separators of separators of separators, the usual keep going with the same idea that you see in algorithms.
What about red-black?
Red-black, I'm simply going to imagine that this is a checkerboard -- it's the nodes that I'm looking at, not the squares.
So that's red, black, red, black.
So I'm going to number first -- I'm going to take all the reds first, so I'm changing this number now to take all the reds -- one, two, three, four, five, six, seven, eight and now all the blacks -- nine, ten, eleven, twelve and so on.
I'll stop there so your eye will know how to fill in.
What's the point of that?
If I take a red node and look -- suppose this first one is a red node, as it is -- then what do I know about the positions of those minus 1s, the non 0s in the matrix?
They're associated with black, right?
The neighbors of this node number one are black.
The black ones have numbers from nine to sixteen.
So then the black -- everybody sees it.
With this molecule that has a 4 in the center and minus 1s beside it, those minus 1s are associated with nodes of the opposite color.
That's what we see here in the actual numbering.
So what does the matrix become?
What does this -- if I do that permutation, the permutation that's associated with that renumbering -- what will the same matrix -- here's the same matrix, but now I'm permuting it and so what does it look like?
Maybe I can draw it smaller.
I think I probably can.
So in this pretty special situation where the red nodes are not connected to any other red nodes, only to themselves, I think I'll only see the 4 a -- this is red to red.
Those are the eight red nodes, an eight by right identity matrix times 4.
The blacks are only connected to the black and all the minus 1s are in our connections between red and black.
This far off the diagonal, what shall I call it?
b, I'll just pick a letter -- b and transposed probably, because I know the matrix is going to stay symmetric.
If it started symmetric, as it did, then this operation of reordering rows and columns compatibly is going to leave it symmetric and it must look something like this.
Is that matrix more sparse than before?
No.
It's got the same minus 1s.
They're just all pushed up into this corner.
Will it have some fill in as we go forward?
What's the answer on that one?
As I go forward, maybe I should -- I haven't really thought that through.
My instinct is yes.
We'll have fill in.
Will we have more fill in or less fill in than the row by row ordering?
I should know the answer.
I think I'd probably ask the machine to tell me.
My guess is probably not more and we can -- why do I think probably this is better?
Of course, I should know.
In general, it's better to put off the evil day -- that is, this goes through the first half of elimination -- I mean, you know what I'm thinking here.
All the off diagonals have been sort of pushed down in the matrix.
Let me do an example that shows why that's good.
Suppose my matrix has a non 0 diagonal -- and by the way, the first step in the good codes for elimination -- do a permutation exactly to produce a non 0 diagonal, so you're OK to start.
You know where you are.
Now suppose that all non 0s all show up and I'll make the matrix symmetric.
That's the non 0 pattern and my point is, that's a very good non 0 pattern.
Why is it good?
It's good because there's no fill in whatever.
When I subtract some multiple, when I go to make this 0, to eliminate it, I subtract a multiple using the first pivot.
It changes that, but it doesn't fill in anything new.
The only operations required are operating on the last row, which is already full.
So there's an example in which the bandwidth -- see, normally the bandwidth is a good guide -- but that's only for pretty full bands.
Here's a matrix that's not banded, really.
Its bandwidth is the full five.
I have to go out five columns until I get the last, until I get the non 0s.
So that is a big bandwidth.
So bandwidth w, the bandwidth w is large, but it's the perfect ordering, but no fill.
I don't fill anything -- you realize that for banded matrix, we won't have fill in outside the band.
Our concern is for fill in inside the band and here, we don't have any, because the bad row -- and actually, it often happens in some b squares in another problem that you might -- this isn't quite as stupid a matrix as you might think.
These might be blocks rather than single entries.
You often want to think block wise about a matrix.
You might think, why would you have a matrix that was well separated with block diagonal, which is really good, and then a few rows that are full.
That seems sort of unlikely, but I guess I want to say it isn't.
If you had a problem in these squares minimizing some expression involving that matrix, the block diagonal part, but with the constraint that maybe the sum of all the unknowns has to be 1, then when you use a Lagrange multiplier to bring that constraint into the problem, into the matrix, that's going to show like that.
and the beauty of the Lagrange multiplier device is that also that it'll produce a symmetric column there.
We'll see that in the third part of the course, but I'm just -- my point is that this example is not as artificial as it seems.
Now suppose we had taken the bandwidth as our metric, as our goal.
In reducing the bandwidth as our goal, we would have made a bad mistake.
If I wanted to do a permutation that reduces the bandwidth, I better put this up in the middle somewhere.
So this is -- that bad row comes up in the middle and then, I guess, these are maybe whatever -- I guess I need three xs down here.
I'm going to do the same permutation from the right, so that will move this column over to the middle.
This would be a reordered -- this would be a p matrix, p transposed thing, I think.
It's got a better bandwidth, right?
Now we have -- we're totally confident of all these 0s here.
But it will fill in.
Where will we get fill in, in that matrix?
So as we start, we're using elimination to remove those entries, no problem, because that upper left corner is the way this big, the hold matrix was, no fill in up to there, but just as soon as we reach this one as our pivot row and use it to subtract multiples to knock out those, when I make those 0 by using that pivot, it's going to bring the whole -- this will be gone at that time, but this will be non 0 and that will produce new edges in the graph.
Might be fun to look at the graph that is associated with this matrix and its permutations.
But anyway, you see that this will all fill in.
So again, in that small problem, we could really see what was happening.
In the big problem, we can almost see it when we take a very specific, highly structured permutation like red-black, although I haven't pinned down exactly what the count is like now.
I'd be glad if somebody would just run a computer experiment, count non 0s.
So what's the good thing to counts?
How do you see what the fill in finally was when you've done an elimination?
I guess that the lnu factors, which is what elimination produces, that's where the fill in will be.
So maybe I just make that point, so that you could make these experiments that I have in mind and tell me what I ought to know about it.
So if I have the matrix k, whichever it is -- whether it's this one or this one or any other, then the effect of elimination is the factor into a lower times an upper triangular matrix.
I'll just mention a special case while I'm -- special positive definite case -- we can adjust it so that's k is l times l transposed.
This is called Cholesky.
We can arrange the -- it will happen.
We can arrange the scaling so that by putting the square root of pivots with l and the other square root with l traposed -- so Cholesky is a MATLAB command, chol and actually, MATLAB -- if you give MATLAB a symmetric matrix -- if you give MATLAB a matrix to work with, it will first ask -- it will ask, is it symmetric?
If it is symmetric, it will hope that it's positive definitely, so it will begin Cholesky.
It'll begin to look for this -- it'll begin the factorization, which is perfect for positive definite 1 until it succeeds and finishes Cholesky or until it discovers that a negative pivot appeared, which means with signals, that the matrix is not and was not originally positive definite and then MATLAB has to accept, OK, now I may have to do row exchanges, pivoting operations and all the rest.
So MATLAB's sort of optimistic until it -- on a symmetric matrix until it is told otherwise.
But not all matrices are symmetric, so this would be the general case, almost the general case.
I haven't included here the pivoting, the row exchanges that MATLAB does as it executes elimination.
So let me just finish my sentence on l and u. l will have the fill in, in it.
l will tell us and u will, certainly -- I mean, they both will.
u will have these -- that guy is going to be part of u and this one is -- I expect to see something there in l. So l and u will actually exhibit the fill in, so that you could find out how much fill in there is simply by counting the non 0s in the matrices l and u.
This is to encourage you, because this is really a pretty experimental subject.
Nobody -- the best methods, which -- or the favorite method.
I think nobody can say best, but this approximate minimum degree, the favorite method, we could prove some theorem to show that it was better than something else, but to really see, is it successful or not on a wide variety of problems, that's really experimental.
Just try it.
Now what does it mean?
So I looked at that specific one.
Now I'm ready to look at the approximate minimum degree and I'll contrast it with the perfect minimum degree.
There are codes for that and those as become ms.
But it turns out -- and I don't know exactly why it's the case -- that the approximate minimum degree usually, if anything, it's superior.
So these are the ones to use.
So what does minimum degree mean?
Minimum degree means -- let's see -- if I took this problem -- I should draw the graph for this matrix.
So what does the graph look like for this matrix?
I've got six nodes -- let me put six nodes in -- let me put five and the node in the middle will be the one that they're all connected to.
In this ordering, those would be one, two, three, four, five and six -- and all the connections go into six.
So that's the graph for that matrix.
The graph for this one -- so you probably see it better than I do.
What I had originally there -- I guess this is the graph with the numbering one, two, three, four, five, six -- what was there before the fill in happened -- the graph would look the same, but the nodes would have a different numbering, right?
One, two, three, four, five, six -- so shall I just put the bad guy here?
One, two, three, four, five, six and then what's the point?
As soon as I use number three to eliminate four -- three was connected to five, so a new edge is going to appear there.
Three was connected to six, three is connected to four.
When I use it, a new four, six is going to appear.
So those are a couple of the fill ins.
Those are the two fill ins in row four.
Then when I go onto row five -- I'll fill in there.
So those are the three below the diagonal and above the diagonal, non 0s that come from the bad bordering, I think.
I didn't prepare that, so I hope it's right.
Yeah So what's minimum degree?
Minimum degree is simply sort of a greedy type algorithm.
We have no way to look far, far ahead and see what non 0s are coming way, way down the line, so let's just take it where we are, We have to choose what should be the next pivot.
Let's say we take that first pivot.
Good.
So what are the degrees of these ?
Let me redraw the graph before the fill in.
One, two, three, four, five, six.
So I've got six nodes.
That has a degree 1.
It's connected only to one other node.
Actually, all these have degree one and this one has degree five.
Five edges are going out.
So the whole point of minimum degree is, don't pick that one.
At each step, create the new graph.
The graph expresses a fill in and we're not going to be able to avoid some fill in normally -- but at every point, take the graph as it is and choose as the next node to work on -- that means the next row to use an elimination, the next pivot to choose -- choose one that has smallest degree.
Don't choose this one.
So for this very special case, we have five choices for what comes first.
We make a choice like that one.
We do elimination.
In this case, it doesn't fill in anything.
Now I'm looking at the rest of the graph.
I have four nodes there with degree only 1 and a node there with high degree, so I don't choose that.
So I choose 2 and so on.
So of course, minimum degree, approximate or perfect or any dumb idea is going to create the right ordering that produces this matrix.
I hope you see what minimum degree is.
At every stage, you look down the column, just as here.
Actually, what would minimum degree give here?
I would love to have a complete picture.
I think we could at least see at the start.
Let me erase this red-black ordering and prepare to write in a minimum degree order.
So what node do I take first?
Remember, the graph really does not have these pieces.
I might as well erase them now.
Shall I just erase -- these are pieces that connect to known values that are on the right hand side.
So I think the graph doesn't have those connections, those connections -- it's going to look pretty much as it did before, but I think now we have the graph of actual 16, the graph that we should be looking for.
That's the graph of our matrix, the graph of the non 0 structure in the matrix.
So what do you take as the first node?
You take the corners, right?
The corners are going to win, because they have degree 2.
These guys have degree 3.
There's a degree 2, so I might take that -- let's see what happens.
When I take this one, then I have to see when I use that as the pivot, exactly what I did here -- what's going to fill in?
I guess explicitly I know that two new edges fill in and what are they?
Maybe they're the same.
I guess they are.
This is edge number -- on the graph, it's probably easier to see than in the matrix.
So I use this guy.
By elimination, I'm removing the connections, so it maybe sits there, surviving to remind me which node wasn't numbered one, but these connections have been removed by elimination.
That entry became a 0 and that entry became a 0, but there was a fill in here, right?
That's the first step of minimum degree.
The second step, as you say, I'm going to do all the corners first, right?
Because that has degree 2 and these still have degree 3.
No improvement.
It's kind of fun to do.
Two and that filled in.
Number three and that filled in.
Number four and that too.
What next?
There are going to be a bunch of choices now.
As far as I know -- I could be wrong -- I don't know whether the -- I don't know what the choice decision -- how the decision is made when there is a choice, when I have one, two, three, four, five, six, seven, eight nodes, all with degree 3.
Let me just take this one.
So I'll take this as the next one, as number five, I guess.
That'll be number five.
That will kill all those connections and just leave the node, but it will put in that one.
Any others?
Maybe only that one.
Did I do that right?
I mean, it would have put in this one, because those are both connected to this, but that edge is already in there, so nothing new.
Actually, here I've created a node that is also degree 3.
So I'm going to have a lot of degree 3s here.
After awhile -- I think when I get into the center -- I'm guessing that the degree will go up.
I don't think I get away with degree 3 all the way until the finish, I don't think.
Anyway, you see the point that there are choices to make among equal node with equal degrees.
Then there's this calculation to make -- maybe I should -- this is a moment to just mention, if I can do it, how you might keep -- how you might organize the this pass through -- so this pass through the matrix -- it's really a just a pass through the graph.
It's not looking at the actual numbers.
It's just choosing the ordering and then the next pass through we follow that ordering with the numbers, finding the multpliers that go into l and the numbers that are left in matrix u.
So how would I -- how would anybody reasonably write down the original list?
So I have the node numbered one, two, three, four -- up to sixteen.
Those are the node numbers.
Now with node one, I'll list all the -- so this corresponds to row one of the matrix, so I'll list all the nodes that are connected to row one, to node one.
Node one itself is -- I don't know whether to put that in or not.
Maybe I will.
Number two is connected to it and number five above it.
I'm taking the original row ordering.
Then when I look at node two -- this is going to be a long list, but I'll have to have a break there to tell me that these numbers were associated with row one.
Now I want to know those associated with row two.
So row two had the connection back to one, so this little group is going to be all the things connected to row two.
Remember -- sorry, since I've erased all those connections, but you remember them -- two was connected to one, it's connected forward to three and above it to probably six.
Maybe that's one, three, and six and then a slash and then whatever node three -- so I need -- I guess I need one more line there to tell me I need a pointer.
I need a line, a pointer that tells me, number one, start with the first one -- one, two, three.
Number two, start with the fourth one.
Number three, start with the seventh one.
So that number seven is pointing to the beginning of the sub list, the beginning of the short list that's associated to each separate node.
I guess what I'm saying is that this is a pretty compact description of the graph.
This will tell me all the edges in the graph, It tells me what two is connected to and this tells me where that little set lies.
Now if I, as I will, change the ordering, then I just reorganize the pointers.
I'm going to speak roughly and not the details, just in general, in the large, you could imagine that this is a pretty convenient structure to use for keeping the record of what the notes numering is.
The command nnz is so useful one.
That's a MATLAB command that counts the number of non 0s.
So n, n and z suggest exactly what that will do.
So if you pick an ordering, like approximate minimum degree and you pick a matrix like this one and you run -- I've been very interested in that.
You take the matrix, k to d, you take the permutation that comes from approximate minimum degree, that gives you this reordered matrix.
We then do elimination, which factors in l times u -- probably l times l transposed, because it's a positive definite.
Then I would do nnz of l as a measure of fill in.
That would count the fill in in l and with symmetry, of the same in u.
If you have a better idea than approximate minimum degree, that's a measure to use as the criteria.
I've just a little time to speak finally about this idea of graph separators.
So again, I have to get back to a graph.
Let me draw a graph under.
So now I'm ready for this one.
I'll put a little star next to it, to say that it didn't look for a long time is if it could compete with number two, because it'll be a different ordering.
You could, in a way -- it's going to be a block minimum degree order.
Let me say what it is and as I draw the graph, let me say when it might win.
It might win when the size goes way up, 3D.
In 2D, minimum degree is -- on sparse matrices, is quite successful.
So in two dimensions, until the size really gets big, that's going to be fine.
In three dimensions -- I'm still going to draw only two, but in three dimensions or very, very large matrices, I want to think about this graph separator.
So let me -- I'll make it just so I really separate it perfectly -- let me make the number -- so when you look at that graph, you might say, if I -- look at that set of nodes.
So a separator is a set of nodes that separates the rest of the nodes into two groups.
So I have a group of nodes p, a group of nodes q and nothing, no connections between p and q. P is this part.
q is a similar part over there.
s, the separator is the part with the connection.
So p is connected to q only through s. So now if I think of ordering, in what order should I take the p, the q and the s?
Our absolute rule is, everything's connected to s, put it last.
So you see that this is the block form of my small example here.
I didn't particularly have a p or q, but I certainly wanted to put s last.
Maybe I did.
So maybe the separator here was -- so what's the separator?
One possible separator would be like that, as s, p for these two nodes, q for these three nodes and the rule would be -- I haven't said what I'll do within p or within q or within s, but the rule would be put q last.
Of course, that was exactly the right thing to do.
We're following the same rule, just [UNINTELLIGIBLE] So we have the -- so our matrix -- what does our matrix look like when you choose this order?
You have whatever p is, the block of all these ps.
You have the block of all these qs, which might be connected to each other and r, but not the p. So the main point is, you have these 0 blocks and then everything is connected to s and s is connected to itself.
So all you've achieved, but could be considerable, is to take an ordering in which p was -- since p is not connected to q, that produced these large 0 blocks and of course, they'll stay 0 as elimination goes forward.
those 0s there will -- I don't have to do any pivoting operations, any row operations to make those 0.
They're already 0 and so the only operations will be p on to s, q on to s, s onto itself.
In many problems, it's the s onto itself part -- so what will happen then when I do elimination on s?
When I do elimination -- I'll have whatever work is necessary within p, so I'll be factoring -- can I call that p, that part of the matrix?
It's an abuse of notation.
This is the connections of s to p. This is the connections of s to q, s to itself, q to s, p to s -- probably symmetric.
So when I do elimination -- if I don't, if I accept this ordering -- if I accept this ordering and just go for it, elimination will factor p into lu because it'll work up in the corner as it always does -- then there's no connection here, so it'll factor q into its own lu and of course, all that -- all this stuff down here will change, but effectively these will become 0s, because then there's a serious elimination, p onto s, make that 0. q onto s -- q will make that 0.
This will be some -- I'll call it sss.
It's a much fuller matrix -- block, smaller though, hopefully smaller -- but pretty full.
So actually in the graph separator nested dissection world, most of the time is spent on that third block, just working with it, because it's the one that's pretty full.
So how do graph separators work?
First of all, you need an algorithm that finds the separator and a lot of computer science attention has gone on to that.
Give it a graph, cut it in half.
We don't want more nodes in the separator than necessary, because the work is going to end up there.
So we want to cut the graph pretty nearly in half by a short separator.
If we only did the dissection once, that's what we need, but this idea of nested dissection -- you know what's going to come before I say it.
The idea of nested dissection section will be, take this, take the graph -- can I just draw it this way without drawing all these?
Cut it by a separator of s 1.
That gives a p 1 and a q 1, but now what's coming?
You're going to separate p 1.
In order to do the work up in that left corner for the p matrix, that's itself a sparse matrix, so now I'll look for a separator and what separator will I choose for the left side of this graph?
I'll choose the horizontal separator.
This will be the second separator.
It'll separate the two bits of p. The third separator will separate the two bits of q.
Now I have this part to work on, a fourth separator will -- you'll see its a nester.
These separators become nested.
There's s 5 there's s 6 and so on.
I think everybody sees in principle what nested dissection should do and the question is, how does it compare with minimum degree?
You see, it's just like a different -- we're off on a different route here.
You can compute asymptotically for large n, what power of n appears there, what power of n appears in minimum degree -- but I would love to know, does the -- I would be happy if somebody used approximate minimum degree on our model problem and maybe also on some other problem.
We could see what power of n -- that's essentially what we're looking for.
So it's n to what power for fill in and for the operation count, which the number flops, which controls the cost and the time.
So it's what power of n appears there and what power appears in the other method.
There you have it for direct elimination methods.
I'll just mention the name of the -- guy's at University of Florida, and he's the one who contributed the code that MathWorks uses and has just announced a book coming on this exact topic, to come later this year from Sci Am.
So there will be a book that -- from the person I take to be the leader in this algorithmic development.
Just to remind you -- Friday is my guest, former PhD student who is going to speak about numerical methods for finance -- Black-Scholes equations and other problems to find the value of financial derivatives.
See you Friday.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So we started last week on the big topic for the rest of the semester, optimization.
Maybe can you close that door or just part way anyway.
Thanks.
Great.
That's perfect, just like that.
So, since it was a few days ago, I wanted to recap what I did in the first lecture about optimization, which was to pick out the least squares problem as a beautiful model problem.
I followed that through -- the input is a matrix a rectangular, a right hand side b probably measurements.
We would like to get au equal b but we can't.
We got too many measurements.
We do the best possible, which is the solution u hat of this normal equation.
Then I rewrote the normal equations -- well, I also drew the picture that you see over here on the left.
The picture that shows the two optimization problems at the same time.
Here is the vector b.
Here are all possible au's -- this is the column space, this is all au's.
The best au was the projection.
The error e was what we couldn't get right, the part that's perpendicular to the column space we can't help.
It's the solution to another projection problem that e is the same -- of course, it's the same over here -- as projecting b onto this perpendicular line, which is the line of all vectors, a tranpose e says -- what that says in words is e is perpendicular to the columns of a. I've drawn it the best I could as a perpendicular to the columns.
So that we have a plane of columns -- this is like a 3 x 2 matrix.
We have a plane with the two columns and the perpendicular line.
Then just near the end, I said that it would be great to have -- this model would be almost all we need except it should have one more matrix and I call that matrix c. I want just to show you quickly where that comes.
So this is all section 7.1 of the notes, except that these words are not yet typed.
So they'll, as soon as possible, and an updated 7.1 will include this, what I want to do.
Because I think it's the very best introduction I could give to these pair of optimization problems.
You might think who needs optimization.
Your main activity might be solving differential equations.
So, can I just take a time out here because I happened to see the homework upcoming.
I think it's 16930, so it's a course a little bit like this, only the first word in the course title is "Advanced," and our first word is "Introduction," but, of course, it's the same course or same stuff.
This is the homework that I don't think has even been assigned, even been handed out yet.
But I just thought it's a great example of a applied, so that you see where optimization appears and what's involved.
So differential equations are involved, but also you got something that you want to optimize.
So I just wrote it on this board and I simplified it a little over the homework that they'll actually get.
so here's the problem.
We want a certain distribution of heat.
So I could draw a picture.
We want a heat distribution for whatever reason that maybe goes like this over the integral zero to 1.
What do we have at our disposal to get the heat to be that way?
Well, we've got sources of heat.
But we don't have a continuous source, we only have n parameters to play with.
I mean right away you recognize an optimization problem.
We're trying to get this function here u knot of x.
We're trying to match a function, but we've only got n parameters.
Those will be the right hand side.
So what we're allowed to choose, we can put in little space heaters, and we can turn them to the temperatures we want to -- temperatures s1, s2, s3.
That was probably a stupid choice to put an s for down there because I don't even know if negative heat is allowed.
Anyway, we wouldn't want it if we're aiming for that distribution.
Now you understand the s's are not supposed to match that u knot.
The s's are the sources of heat, and the u's are the outputs, the distribution, and they are controlled by a differential equation.
What we control is the right hand side of the equation, but we only control n parameters.
Then we have to solve the equation and find out what distribution that gives.
So that's all like differential equations, we know how to do it.
It's one dimensional in this example, so straightforward.
But now comes the optimization part.
We take this result and we compare it with the desired result.
So the actual result from some source of heat might be something like that.
We want this u of x, the heat distribution that we're actually producing to be as close as possible to u knot.
Close could mean different things, but if we measure closeness in this integral square mean square sense, then we're going to have nice problem.
In fact, we're going to have a linear -- everything's linear right now.
Well, everything's quadratic here, so when we minimize it's going to give us a linear equation for the s's.
I just think that's a good model problem.
A good model of what we might do.
I think actually in the problem to be assigned -- well, it's not a big deal for us.
There is convection as well.
So this is a pure diffusion problem, just the second derivative.
We know very well that if there was a first derivative in there, the stuff's convecting, passing through the region.
So if I put on a convective term, cdudx, how does that change the problem?
Not in a big, big major way, but one thing we can guess is that now that is not a symmetric term, right.
We've seen the difference between first differences and second differences, first derivatives and second derivatives.
So now it's a little trickier and it's not symmetric.
So somehow there will be a primal problem, this one, and there will be a adjoint problem, dual problem, perpendicular problem, whatever name you want to give it, just as there is in our very first model over there.
So that's my sort of quick look to kind of put on the board one example that I didn't invent, that came from applications and gives a sort of typical of what you have to do.
You control an input, you get an output, so that's the analysis problem.
Find the output.
But then comes the optimization problem -- make that output close to something that you wish.
So what's the typical algorithm going to do?
It's going to make a choice of s, it's going to solve the analysis problem for the u, it's going to look and see what the error is, it's going to figure out probably the gradient somehow.
What's the steepest way to make it closer?
That's going to lead us to a change of s. We use a change of s, the new s, solve that, and iterate.
That would be a typical algorithm.
We might be able to shortcut it in a model problem like this.
But that's totally the typical optimization idea, is an analysis problem and then figure out a gradient to get a better source back to the analysis problem with the new source.
What the algebra, the math, has to do is those two steps of figuring out OK, how do we improve the error, how do we reduce the error, what's the steepest direction?
Somehow we got to compute a derivative.
Actually, that's what this month is about.
Derivatives that are not just like the derivative of x cube or something.
I often wondered how many presidents could take the derivative of x cube and I'm not sure.
Anybody occur to you who you could count on being able to take the derivative of x cubed?
I don't think the current president would know what it meant.
But I think Carter could have done it, because he went to the Naval Academy.
Jefferson was probably -- he knew everything.
I don't know.
Anybody else has another candidate they can tell me.
So there is our problem -- finding derivatives that would be definitely beyond the capacity of the White House.
Now I want to stay with this model a little more because it's the perfect model.
So this was the like model 1, unweighted ordinary least squared, and it produced the identity matrix in there.
I mentioned last time what I want to do, the more general model has another symmetric positive-definite matrix in there, but not necessarily i, and it comes from weighted least squares.
So that's what I'm going to talk about.
So what are weighted least squares.
Well, you've got these measurements and you think they maybe don't all, maybe they're not independent, maybe they're not equally reliable.
So you weight them by how reliable they are.
A more reliable one you would put a heavier weight, w, on because you want that to be more important.
So, you change to this problem.
But it looks practically the same.
The only difference is a has become wa, b has become wb.
So the equations will be the same, but a is now wa, b is now wb, and those are the equations.
I guess I should call, just to make clear that u is a different u, that the best u now depends on the choice of weights.
I should really be calling that u -- somehow indicate that it depends on the weight.
Now they key nice thing is that if I write this out as a tranpose -- can you just write this out.
You get the a tranpose, and the a is over here.
But what's in the middle?
What's this matrix c -- I jumped the gun and called that matrix in the middle c, but how is it connected to w?
You can see it here as c is w tranpose w, right?
It's just sitting there in the middle.
That's great that the fact that the combination w tranpose w is all you need to know.
So we can forget w in favor of -- I'll just put it here -- w tranpose w is now given the name c, and this matrix is symmetric positive-definite.
So it's a great matrix and it's exactly the one we want.
This is exactly the equation we want.
So if I go back to writing it in this -- you know, this was the equation.
You remember the point that we could go directly to one equation for the best u, or we could keep our options open and have two equations that led to the same one.
They're totally equivalent.
But the two equations will give us not only u, but the error, b minus au, as an other unknown.
That's what we did up there.
So we had two equations, and if we eliminated e we got to this.
Now I want to have two equations, and if I eliminate e, I get to that.
So let me see what those would be.
Well, here's one of them.
a tranpose c -- e is zero -- you see, that's the only difference really.
That the weighted normal equation, I just took a tranpose c, and then I took the b minus au together, and this is what I'm calling e. So one way to do it is just the new guy is just now e plus auw is still b.
But now a tranpose ce is zero.
Good deal.
I mean that's quite nice.
It isn't absolutely perfect though, because I've lost the symmetry.
I'm putting a c in there and I don't really want it.
I want a c but I don't want it there.
So, I just make a small change.
I'm going to introduce a new unknown that I'll call little w, apologies for the fact that it's also a w, it just happened to fit.
That'll be the ce.
So I'm just calling this a new name here.
So that now my e -- of course, if I just invert, e is c inverse w. So now I'm just going to write these equations with w instead of e because I like it better.
So this equation is now a tranpose w equals zero.
This equation e is disappeared in favor of c inverse w plus au equals b.
So that's the system that I really like.
That's the saddle point system, the Kuhn-Tucker system, the primal dual system, the fundamental system of the whole subject in this linear matrix case.
We haven't got functions, we got vectors, and we've got symmetry, and we've got linearity, and we've got a saddle point matrix that's now the s -- well, let me just change it here.
It's just changed to this, c inverse.
That's the fundamental matrix of the whole subject.
So, s is the saddle point matrix.
So I wanted to get that far.
You see that the whole picture was elementary linear algebra.
Let me come back to the elementary figure that illustrates what's the geometry.
How was the geometry affected by introducing this guy w, this weighting matrix or the c equal w transport w?
Well, here was the picture from last time.
A right angle picture.
This was a right triangle.
This line was perpendicular to that plane, but not anymore now.
The second one it's a tranpose ce is zero.
That's still a line, but it's not any longer perpendicular to the -- this is still all the au's.
This plane is still the column space of all the au's.
We have the same b, but you see the problem is it lost its 90 degree angle, because the projection is now a projection, it's now an oblique projection, it's slanted.
This is the best au, and if I occasionally keep up-to-date I'll put that uw there.
There's still an error e, and this is still a parallelogram but it's not a rectangle anymore.
Forgive my enthusiasm.
I'm sort of happy that the picture and the algebra both come out so neatly.
I totally agree that at this point I'm asking you to follow a model without giving you an application, and that's one reason I threw in this mention of a specific application that came from somewhere else.
But this is the picture there.
So I'll say one more word about the picture.
I said that we lost the right angle.
We lost perpendicularity, and, of course, literally speaking we did.
This is no longer -- this is not a right angle anymore.
This is not a right angle anymore.
It's not a right angle in the usual meaning of right angles.
It is a right angle in the inner product that's associated with c. In other words, right angles here mean x transpose y equals zero.
That's the idea of a right angle, right?
x perpendicular to y, and they have different letters here.
Now over here I still have perpendicular, but I don't -- this is not the right inner product anymore.
It should be a weighted inner product, weighted with this c in the middle.
So that's really what I mean by c orthogonal.
Maybe I'll put those words down.
So this weighted thing is -- if I can squeeze it, I doubt if I can -- is c orthogonality, weighted orthogonality.
So let me circle the whole thing.
The c is the w tranpose w. Just to say that we aren't giving up on dot products and perpendiculars and good equations, we're just changing them by inserting c every time in a product.
What it means is that this is the natural inner product for the particular problem.
This is the natural inner product for Euclid, right?
But then from some specific application like this one or a million others, they have their own natural inner product, and the inner product for that particular problem would be one of these guys with some kind of a matrix c showing up.
So least squares and weighted least squares, that's my example one.
Now I'd like to give a second example, a more mechanic -- will come closer to mechanics.
Because this is least squares, statistics, algebra.
But let me put on the middle board an application out of mechanics.
It will be, say, I'll make it small and just a couple of springs with a mass between them and fixed at both ends.
So this spring extends by some amount.
This spring extends by some amount.
There's a force on this mass.
So there's a force on there mass, maybe just gravity, f. That's the external force from the mass of that's here in between the springs.
Then also acting on that mass are spring forces.
This spring is pulling it up, right?
There's a spring force, w1.
Here, do you want me to draw -- really which direction is this?
I'm going to draw it this way just to show the w2 drawn that way would actually be negative, because I think that this spring would get compressed, right -- this mass is pushing it down.
This spring would be under compression.
It would be pushing the mass back up.
So the w2 in that picture would be negative.
So this w1 will actually be positive and go the way the arrow is showing.
This w2 would actually be negative and go not that way the arrow is showing.
But what's the -- oh, what equations have we got then?
What's our optimization problem?
Well actually, we have a choice.
We could work with equations, period.
Actually, one of the equations is pretty obvious.
This mass is an equilibrium.
So w1 is equal to w2 plus s. So it doesn't move.
Or you might prefer me to write it, I would rather write it, with w's on one side and source terms on the other.
So that's the equilibrium equation.
So what decides, we want to know what these w's are, and these springs are extended.
So that first spring is expanded by an amount e1, and this second spring is extended by an amount e2.
Stretched or compressed.
e1 is probably going to be positive here -- that spring's going to be stretched.
This spring is going to be compressed, so that e2 is probably going to be negative.
What's the mechanics here?
Well, I can state it two ways, as I said.
I can state the mechanics in terms of equations -- force and stretching elastic constant.
That's how we did it in the first semester, 18085.
It's a little clearer because simple equation [UNINTELLIGIBLE PHRASE].
Or I can state the problem as a minimization of energy.
That's what I want to do today in 1806.
So I want to minimize 8.6.
I want to minimize the total energy.
The energy in these springs.
Subject to the constraint -- this constraint, equilibrium.
So that's the optimization statement of the mechanical problem.
Well, I guess all that remains is -- well, I guess what remains?
First of all, I need an expression for this energy.
If the springs are governed by Hooke's law, then it'll be pretty simple.
If they're real springs that don't quite obey Hooke's law then there'll be none -- no, there'll be fourth degree, sixth degree , whatever, terms in the energy.
It's like the energy in the first spring plus the energy in the second spring, anyway.
e in the first spring, and the energy in a second spring, and, of course, the two springs could have different spring constants.
So those e's -- I'll make life easy in solving, if I choose Hooke's law, if I choose the energy to be just a square.
The constraint is linear, so that'll be the model problem.
And actually, that'll be the kind of problem I've got here.
One reason for introducing a new example was to get some mechanics into the lecture, but also to get a problem where we're doing a minimization but we've got a condition on the w's.
The question is how do you find a minimum?
You can't just set derivatives of the energy to zero.
You would discover w1 equal w2 equals zero.
Nothing happening.
That would be the minimum.
But that minimum is ruled out, that solution is ruled out because we have this constraint.
We've got to balance the external force.
So this is the question and you'll maybe have met this question in other courses, but it's essential.
How do you deal with minimizing when there's a constraint?
I guess in some way we had it over here.
We were minimizing something -- well, the minimum would be, take u equal u knot.
But no, there was a constraint that u had to satisfy a certain equilibrium equation.
Here it was a differential equation so that problem is a little harder than this one where the equation is just discrete, one simple equation.
So how are you going to do it?
Well, actually the quickest way would be -- that's such an easy constraint that I could say hey, w2 is w1 minus f. So I could just, if I wanted to really like shortcut this whole lecture, I could say well, w2 is w1 minus f. Now I've accounted for the constraint.
I've removed w2 from the problem.
I have a minimization, an ordinary minimization with an unknown w1.
I take the derivative.
I solve derivative equals zero.
I find w1, then I go back, I get that w2.
That's the fast way.
Of course, gets the right answer.
But there's another way that in the end turns out to be better.
It's not necessarily better for this simple problem, but it's better for the general approach to constrained optimization.
So I'm going to not do the simple deal of solving for w2, but I'm going to keep the constraint around.
It's the idea of Lagrange multipliers.
You've heard those words and probably seen it happen.
So what is Lagrange multiplier?
What is Lagrange's idea?
Lagrange's idea is he constructed a function of w, which is this to work with, which is the same energy.
But he's going to include a multiplier.
Now the next question is what letter shall I use for that multiplier, Lagrange's multiplier.
Books on optimization often call it Lambda -- Lambda's sort of like for Lagrange.
So, Lagrange obviously wasn't Greek, but anyway -- close.
Lambda for us always means eigenvalue -- For me always means eigenvalue.
So I'm reluctant to use Lambda.
Sometimes in books on economics, which is we're doing this all the time, the multiplier's called pi because it turns out to be a price.
But let me use u for the Lagrange multiplier.
So that'll be the Lagrange multiplier.
So what do I do with it?
I multiply this equation by u and I build it in to this.
So this thing is this part -- I'll just copy that down there -- plus or minus, depending what I want, depending which sign I want to end up with, u the multiplier times the constraint -- w1 minus w2 -- let me put it like that.
So the constraint is that this should be zero.
So you can say I haven't added anything.
I've added zero.
But that will only be true at the end when I have a specific w1 and w2.
Right now what I've done is I've built in the constraint into the function.
Lagrange's brilliant idea was that now I've got a function that I can -- my derivatives I can take.
I could set the derivatives to zero.
dldw1 is going to be zero.
dldw2 is going to be zero.
And dldu is going to be zero.
So we got now three equations, three unknown.
Instead of going from two to one, we've gone from two up to three.
But it's so much more systematic that it's the right thing to do.
What is this last equation?
What's the u derivative of an expression?
Well, u doesn't appear here, the derivative is just w1 -- this just leads us to w1 minus w2 minus f, which is the constraint that we wanted.
So the constraint is showing up as this equation.
Just the way the constraint showed up as this equation here.
I guess what I want to say is when I wrote this down, when we did this first example, I didn't say here's the constraint.
I mean that equation kind of came out from the normal equations here.
So what we're doing new here is the constraint equation is sort of part of the mechanics and we're asking the question how do I deal with it.
Here it came out of the geometry, so it wasn't there at the beginning so we didn't have to say how do we deal with this equation.
It just emerged.
Here it's really forced on us right away.
So anyway, we've still got to figure out the derivatives of those, and I guess I do have to finally now say what choice -- yeah.
So this is -- what is this, the derivative of e1 minus u.
If I take the w1 derivative, I've got the derivative of v1 with respect to w1.
Then w1 doesn't appear there but it appears here, so it's minus u.
This would be the derivative of v2 with respect to w2, that second spring minus -- oh maybe plus cu.
Well, those are the three equations.
Let me move now to the linear case, just so we see the beautiful pattern.
So if I make the equations linear, what's the energy in Hooke's law spring here if the extension is e1 and if it produces a force of w1, I want to know what is this e1 here.
So let me just remember Hooke's law.
I think Hooke's law would say that -- well, there's some elastic constant c1, so there's a c1 and a c2 that tell us how hard or soft the springs are.
So these are physical constants.
If I remember right, the energy -- so I'm going to erase a little here just to -- well no.
So what is the energy in that first spring?
You remember, there's a 1/2 of an c1 e1 squared.
That's the energy in a spring with constant c1 and the stretch e1.
But now I really want it not in terms of e1, I want it in terms of w1, just the way I did here.
I've got to do the same thing here.
I want to get to w, because the constraint is in terms of w. What does Hooke's law say?
Hooke's law says w equals ce, that's Hooke, Hooke's law.
The fourth is the elastic constant times the stretch.
So in place of the e I have w over c. So this is 1/2, e1 squared will be w1 squared over c1 squared, and then a c1 cancels -- that's what it looks like.
I guess I'm certainly happy to see that I'm coming up with c inverse.
This c is showing up in the denominator, and that's exactly the -- so this is what I mean, this is what I want for my energy is a 1/2, w1 squared over c1, and a 1/2, w2 squared over c2, and now I was doing a minus sign because that kind of goes well with mechanics where a plus sign would go well with all the other applications.
So now I've got explicit energy.
So now I can say what this thing really -- the derivative with respect to w1.
OK.
The derivative with respect to w1 is just w1 over c1.
Is that what I'm getting?
This is zero.
This says w2 over c2 minus u is zero, and this one says w1 minus w2 equals f. Our three equations.
Yes, so I will help kill these equal signs and just look at -- oh, that's a plus.
We've got our saddle point matrix again.
That's the nice thing here.
This is a problem with a 3 x 3 matrix, with three unknowns, w1, w2, and u.
With right hand side zero zero and f. With what matrix?
That looks like a 1 over c1, zero and a minus 1.
This looks like zero, a 1 over c2 and a plus 1.
This looks like -- oh, wait a minute.
I don't want this.
w1 minus w2 equal f. I can live with 1 and minus 1 there, but it's not really what I wanted.
I wanted the signs to -- you know what I want here.
I want the transpose of that to be here, and zero to be in that block.
So that's what my saddle point matrix would look like.
Well, let me just say that I could live with either.
I was aiming for this one because it's symmetric.
But a lot of people would rather have the opposite signs and have the 1 minus 1 there.
I don't care which sign f has, of course.
Some people these days are liking this form better because then it has a symmetric part and an anti-symmetric part.
I mean the thing is at some point we're going to get problems like this with thousands of unknowns, and we're going to think how do we solve them and maybe some iteration.
So we might want the matrix to be symmetric but indefinite, or we might want a positive-definite symmetric part and an anti-symmetric part.
What we can't have is positive definite-symmetric.
That's like asking for what can't happen here.
The combination of the problems is producing a saddle point and we can play with that sign, but we can't make that zero something -- oh, we could actually.
What I was going to say is we can't make that zero something different, but you could.
That would be a possible way to -- it's another way people thought of of solving these problems is artificially throw in a big number there, or even a small number, and push things towards positive.
Anyway, my purpose today is essentially completed, that we're getting out of physical application after I linearized and it became a linear equation and it had that saddle point form.
So, saddle point form here, saddle point form here, saddle point forms everywhere.
I mean we'll have the saddle point forms for differential equations, as well as for matrix equations.
So those are the examples to sort of hang on to, and it's section 7.1 that has a big part of it.
Ah -- I always have one last thing to say.
What's the meaning of u?
So, u was a Lagrange multiplier.
Lagrange just like helped us out by saying OK, there was a constraint by using one of my multipliers.
But the point is that the multiplier always has a real meaning.
I mention prices before.
Here, what's the meaning of u?
What's the physical meaning of the Lagrange multiplier.
It turns out to be the displacement of the math.
It's the dual variable, so it always has some interpretation.
In this case, with mechanics it's the amount, the mass comes down when the force asks.
What's more, it also has a derivative interpretation.
Turns out -- I'll just put turns out that the Lagrange multiplier u turns out to be the derivative of the minimum energy in the system with respect to the source term.
It's the sensitivity of the problem somehow.
I'll just use sensitivity.
You often want to know -- it's actually this quantity that we need over there.
We want to know how much does the answer depend on the source, and that's what the Lagrange multiplier tells us.
So, if I computed the solution here, so the notes do this -- maybe I'll leave it for the notes.
The notes solve this little problem.
That's easy to do.
They figure out what the energy is in the springs.
It depends on the right hand side, f, which is just a number here.
The energy turns out to be quadratic.
You can take its derivative and you find out that it's u.
So that Lagrange multiplier, that's really a key message, is an important quantity in itself.
Here it happens to mean displacement, which is obviously crucial quantity, and in general it tells us the change in the minimum energy with respect to a change in the input.
That sensitivity's a natural word to use for that.
So that's a final word about -- well, a near to final word about Lagrange multipliers.
So I'll see you Wednesday and by that time I'll know more about the projects and we'll be moving onward with optimization.
Good.
Thanks.
The following content provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MITOpenCourseWare in general is available at ocw.mit.edu
So this is the second 086 lecture and it'll be my second and last lecture on solving ordinary differential equations -- u prime equal f of unt.
So last time, we got Euler's method, both ordinary forward Euler, which is this method for the particular equation.
So I'm using my model problem as always, u prime equal au.
That's the model and I'm using small u for the true solution and capital u for the approximation.
By whatever method we use, and right now, that method is forward Euler.
We looked at the idea of stability, but now let me follow up right away with the point of stability.
Where does stability come in?
So I want to do that for this simplest of all methods, because you'll see how we get to an error equation and we'll get to a similar error equation for much better methods.
So the accuracy of Euler is minimal, but the question of how stability enters and how we show that it converges will be the same also in partial differential equations, so this is the place to see.
So I'll start with that, but then after that topic, will come better methods than Euler and the first step will be, of course, second order methods because Euler and backward Euler are only first order and we'll see -- actually, that'll be the point.
Not only will show when they converge, why they converge, but also how fast they converge.
What's the error?
This estimate of the error is something that every code is going to somehow use internally to control the step size dela t. So I'm just going to go with the model problem, u prime equal au, the Euler method.
So here is the evaluation -- au is f there, so it's really delta t multiplying f, which is just au in this easy case.
Here's the true solution and then here is the new point.
When I plug the true solution in to the difference equation, it, of course, doesn't satisfy exactly.
There's some error term -- this, I would call the discreditzation error -- making the problem discrete.
So you could say the error at a single time step -- it's the new error that's introduced at time step m. Error now is going to be -- e will be u, the true one, minus the approximate, so I just subtract to get an equation.
For the error at time step n plus 1, that subtraction gives ena delta t. That subtraction gives en and this is the inhomogeneous term, you could say, the new error.
So the question is -- first, the question is, what size is that, the local error?
Then the key question is, when I combine and all these local errors, what's the global error?
So first, the local error.
You can see what this is, approximately, because the true solution would be e to the at and we all know that e to the at times u 0 -- e to the at -- there's a 1 plus an a -- from a single time step -- sorry, I should have said the true solution would grow by e to the a delta t over one step.
It would multiply by e to the a delta t, but instead of multiplying by that exponential e to the a delta t, we're only multiplying by the first two terms in the series.
So the error here is going to be something like a half delta t squared -- that's the key point where we see the local error -- times probably a squared -- we needed an a squared and u.
So a squared un, that would really be the second derivative.
I'll put it in as second derivative, because that's what it would look like in a nonlinear case.
So I'm working the linear case for simplicity, but the point is that this is the Euler local error.
That's so typical, that you really want to look at that local error, see what it looks like.
There is some constant and that has a certain effect on the quality of the method.
There's a power of delta t and the question, what is that exponent has a big effect.
So that exponent is the order plus 1.
So for me, Euler's method has accuracy p equal 1.
Over a single delta t step, the error is delta t squared, but we'll see that when I take 1 over delta t steps to get somewhere, to get to some fixed time like 1, then I have one over delta t of these things and the 2 comes back down to the 1, which I expect for Euler.
And finally, this term is out of our control basically.
That's coming from the differential equation.
That would tell us how hard or easy the method is, the equation is.
So if u double prime is real big, we're going to expect to have to reduce delta t to keep this error under control, so that's a typical term -- a constant delta t to a power of p plus 1 and the p plus first derivative, the second derivative matching that to the solution.
So that's -- if I now put this -- think of that as here as the sort of inhomogeneous term or the new source term.
I just want to estimate the end now.
So I've done the local part and now I'm interested in putting it all together.
How do I look at e -- so I really want somehow the solution -- e at some definite time n is what?
I'm asking, really, for the solution to this system, and sort of in words -- so that's a good -- a totally typical error equation.
I think the way to look at it in words is, at each step, some new term, new error is committed, and what happens to that error at later steps?
At later steps, that error committed then gets multiplied by 1 plus a delta t at the next step and 1 plus a delta t at the following step.
So I think we have something like after n steps, we would have 1 plus a delta t to the n times the error that we did -- our first step error.
So that's -- you see what's happening here?
Whatever error we commit grows or decays according to this growth factor.
Then of course, we made an error at step two and we made an error at step k, so at step k, it will have n minus k steps still to grow.
This is de, the error de at step k and all the way through the -- I guess it would end with den.
den would be the last thing that hadn't yet started to grow or decay.
Ok so that's our formula.
It looks a little messy, but it shows everything.
It shows the crucial important of stability and how it enters.
Stability controls -- so there is the error that we made way at the beginning and then this is the growth factor that happened n time, so you see that if 1 plus a delta t has magnitude bigger than 1, what's going to happen?
That will explode and the computer will very quickly show totally out of bounds output.
But if it's stable, if this thing is less than 1, less or equal to 1, then that stability means that this error is not growing and so we get -- so let me put, if we have this stability -- 1 plus a delta t, smaller equal 1, then we get the bound that we want.
So can you see what that looks like?
We've got n terms, right?
To get to point n, we've made n -- capital n errrors -- and then each term, that gives a 1.
That's the critical role of stability and the error is of this order, so I'm getting something like a half delta t squared over 2 times -- let's say some maximum.
I'll use u double prime for a maximum value, let's say, of the second derivative.
That's a typical good satisfactory error estimate.
So what's up here?
So n times delta t times 1 delta t is the time that we reached, right?
We took n steps, delta t each step.
So this is the same as -- this is what I'm wanting and it's going to fit on this board -- there's a one half t -- I should make it t over 2, right?
n delta t is some time -- capital t. We have the half.
We still have 1 delta t. That's the key.
We have this u double prime, which is fixed.
Anyway, we have first order convergence.
So the error after n steps goes down like delta t. If I cut the time step in half, I cut the error.
So that's not a great rate of convergence.
That's, like, the minimal rate of convergence -- just 1 power delta t and that's why we call Euler a first order method.
I think, I hope you've seen and can kind of -- because we'll -- this expressions for the solution is messy but meaningful.
That's a little bit of pure algebra or something that shows how stability is used, where it enters.
So now I'm ready if you are to move to second order methods.
Now we're actually going to get methods that people would really use.
So all those methods are, I think, worth looking at.
Can I say that the notes that are on the web -- you may have spotted chapter five, section one, for example, which is this material -- so I'm -- after the lecture, I always think back, what did I say and improve those notes, so possibly later this afternoon a revised, improved version of this material will go up and then next week starts the real thing, what I think of as the real thing, the wave equation and the heat equation -- partial differential equation -- but it's worth -- here we saw how stability works.
Here we're going to see how accuracy can be increased and in different ways.
In this way -- so what do I notice about the trapezoidal method?
Sometimes it's called Crank-Nickolson.
In PDs, the same idea was proposed by Crank and Nickolson.
So what's their idea or what's the trapezoidal idea in the first place?
Basically, we've centered -- we've recentered the method at what time position?
You can go ahead.
So where -- instead of Euler, which was, like, started at time n or tn or backward Euler that was sort of focused on the new time tn plus 1?
This combination is at time n plus a half delta t, right?
Somehow we've symmetrized everything around the half and the point is that around the midpoint, the accuracy jumps up to 2.
Let me try always u prime equal au.
So f is au.
Then what does this become?
Can I, as you would have to do in coding it, separate the implicit part?
This is implicit, of course.
That stands for this: fn plus 1 means f of the new unknown u at the new time.
So it's implicit.
So let me bring that over to the left side, because it's sort of part of the unknown.
I'll multiply through by delta t and I'll take f to be au.
So I just want to rewrite that for our model problem.
So it will be 1 minus a half when I bring that over minus a delta t over 2 un plus one.
Is that what I get when I multiply through by delta t and bring the implicit part over and now I bring the explicit part to that side, so it's 1 plus a half delta t a delta tun.
So that's what we actually solve at every step.
So first of all, I see that it's pretty centered.
I see that there's something has to be inverted.
That's because the method's implicit.
If this was a matrix, a system of equations, which is very typical -- could be a large system of equations with the matrix, capital a, then u would be a vector and we would be solving this system: matrix times vector equals known right hand side from time n. But here it's a scale or it's just a division, so what is stability going to depend on?
What does stability require?
At every step, we don't want to grow.
So if we take a step -- that's 1 plus a over 2 delta t divided by 1 minus a over 2 delta t, that's the growth factor, the decay factor that multiplies un to give you n plus 1, so the stability question is, is this smaller or equal to 1?
That's stability for this trapezoidal method.
What's the story on that?
Suppose a is very negative.
That's what killed forward Euler, right?
If a delta t -- that's where we look for instability.
If a delta t becomes too negative, we lost stability in forward Euler and we'll lose stability in other, better methods too.
But here, if a delta t is negative, that denominator is good.
It's bigger than the numerator, right?
So we have a -- you'll see it.
If a delta t is less than 0, if it's very much less than 0, then this is approaching maybe minus 1, but it's below 1.
So this message is a, stable.
Absolutely stable.
Stable for all a less than 0.
We're fine.
Delta t can be in principle as large as we like.
So we might ask, why don't I just jump in one step to the time that I want to reach and not bother taking a lot of small steps?
What's the -- in that little paradox?
It would be stable, but what's no good.
So if I jumped in one giant step, this thing would be about minus 1 or something.
So u here would be about minus 1 times u 0 and of course that's not the answer.
So why do we still need a delta t?
Because of the discretization error.
Because this isn't the exact differential equation, so we still need -- we don't have problem with stability, but we still have to get the local errors down to whatever level we want.
Now what are those local errors for this guy?
What are the local errors for that message?
Let's see.
Can we actually see it?
I mean, you know what I'm thinking, right?
I'm thinking that the local error is a border delta t cubed.
That's a second order method for me.
delta t cubed at each step.
1 over delta t steps, so delta t squared overall in that second order.
But do we see why this is?
I mean, instinctively, we say, it's centered.
It should be, but let me expand this to see, are we -- is e to the a delta t -- how close is it to 1 -- so it's approximately 1 plus a over 2 delta t divided by 1 minus a half a delta t. I just want to see that sure enough, this is second order.
So of course I have a denominator here that I want to multiply.
So that's 1 plus a delta t over 2, the explicit part and then everybody knows the 1 -- there are only two series to know, the truth is.
We study infinite series a lot, but the two that everybody needs to know are the exponential series, which is for the left side and the geometric series for the right side, 1 over 1 minus x is 1 plus x plus x squared plus so on.
So that's the -- that this came from the denominator, getting it up in the numerator where I can multiply the other term and get 1 plus a delta t, which is good and what's the coefficient of a squared delta t squared?
Can you see what it is?
If I do that multiplication, what does -- you see it?
Where do I get a delta t squared from this?
I get one here, times 1/4 and I get 1 here times 1/4, so that's two 1/4s is 1/2.
So it's great, right?
It got the next term correct in the exponential series.
so that's why it's one order higher; because it got the right number there, but it won't get the right number at the next step.
What's the -- I'd have to find out what the cube -- this would be a delta t over 2 cubed, so what will it -- the I'll get a wrong coefficient for a delta t cubed.
I'll get -- that would give me 1/8 and this would give me another 1/8, I think, so I think I'm getting two 1/8s out of that.
I'm getting 1/4, which is wrong.
What's right there?
For the exponential series, it should be one over three factorial.
It should be 1/6.
So I got 1/4, not 1/6 and the error is the difference between them, which is 1/12.
So 1/12 would be this number.
The local error, so the local error, de, would be like the missing 1/12, the delta t to the third power now and the third derivative.
That's what would come out if I patiently went through the estimate, just to see, because we'll -- in these weeks, we'll be creating difference methods for partial differential equations and we'll want to know, what's controlling the error?
This is the kind of calculation, the beginning of a Taylor series that answers that question.
So the main point is, local error delta t cubed, stable for every a delta t. So our method would move from Euler to trapezoidal and it would give us a final error e -- sorry, small e. I can say it without writing it.
The error, small e, would be delta t to -- what power?
So squared, delta t squared.
Good.
The same kind of reasoning -- now I want to ask about -- so that takes care of trapezoidal, which is a pretty good method.
I think it's -- in METLAB, it would be ode something small t for trapezoid.
Forgot -- maybe 1, 2.
I should know.
It's used quite a bit.
Of course, we're going to get higher than second order, but second order is often in computational mathematics a good balance.
Newton's method for solving nonlinear equation is second order and it doesn't pay to go to higher.
You could invent a -- you could beat Newton by going, by inventing some third order method for solving a nonlinear equation, but in the end, Newton would be the favorite.
So second order is pretty important.
So let me point to -- so that was implicit, because it involved fn plus 1, but Adams had another idea, which seems like a great idea.
Instead of an implicit using fn plus 1, he used the previous value, fn minus 1, which we already know.
We've already at that previous step substituted un minus 1 n to f. That might have been time consuming, but we sure had to do it once and we only have to do it once with Adams.
We're using something we already know.
This is the one new time that we need it, that we have to compute f and we get explicitly the new u.
These numbers, 3/2 and minus 1/2 were chosen to make it second order and the notes check that, that with a series that it comes out to give us the correct second term.
Good.
Let me mention backward differences.
Now we're just using 1 f and actually the implicit 1, so I'm making it implicit.
So why am I interested in number 3 at all?
Because it's going to have better stability.
By being implicit and by choosing these numbers, 3/2, minus 1/2 and 1/2, I've upped the accuracy to 2, equal to 2 and I've maintained high stability by making it implicit.
So this one competes with this one for importance, for use.
So you're really seeing three pretty good methods that can be written down.
So these numbers were chosen to get that extra accuracy, which we didn't get from backward Euler.
When those numbers were 1 and minus 1, backward Euler was only first order.
Now this goes up to second order and you can guess that we can that every term we allow ourselves, we can choose our coefficients well, so that the order goes up by one more.
Now if you look at that, you might say, why not keep -- you could come back to this.
Why not combine the idea of backward differences, more left hand sides, with the Adams-Bashforth idea of more right hand sides?
Suppose I come back to Adams-Bashforth and create an 1806 method?
That'll be third order, but still will only go back one step.
So it will somehow use some combination like this that goes back -- I'm not writing this method down, for a good reason, of course, but it will use something like this on the left side and something like this on the right side and that gives us enough freedom, enough coefficients to choose -- two coefficients there, three there and enough that we can get third order accuracy.
In other words, the natural idea is use both un minus 1 and fn minus 1 in the formula to get that extra accuracy.
So why isn't that the most famous method of all?
Why isn't that even on the board?
You can guess.
It's violently unstable, so sadly, the most accurate methods, which use both an old value of u and an old value of f of u, use both of this and this -- the numbers that show up here are bad news.
It's a fact of life and so that's why we go backwards.
We have to make a choice: We go backwards with u or Adams goes backwards with f or we think of something way to hype up the accuracy even further within one step method.
So the notes give a table, the beginning of a table.
People could figure out the formulas for any order p, so the notes will give the beginning of a table for p equal 1, which is Euler, p equal 2, which is this.
p equal 3 and p equal 4, which is Adams-Bashforth further back, backward differences further back and those tables show the constants.
So they show the 1/12 or the 1/2 or whatever that is and they show the stability limit and of course, that's critical.
The stability limit is much better for backward differences than it is for Adams-Bashforth.
So actually, I only learned recently that Adams-Bashforth, which we all teach, which all books teach, I should say, isn't that much used way back -- maybe the astronomers might use it or they might use backward differences, which are more stable.
So backward differences has an important role and then one step methods will have an important role.
Backward differences are implicit, so those are great for stiff -- you turn that way for stiff equations and for nonstiff equations, let me show you what the workhorse method is in a moment.
So I have a number 4 here, whose name I better put up here.
Let me put up the name of method 4, which is two names: Runge-Kutta.
So that's a sort of one compound step.
So what do I mean by -- what's the point of -- this looked pretty efficient, but there are two aspects that we didn't really, I didn't really mention and on those two aspects, Runge-Kutta wins by being just a single self-contained step.
So what are the drawbacks of a multistep method like this or like this?
You have to store the old value, no problem, but at the beginning, at t equals 0, what's the problem?
You don't know the old value, so you have to get started somehow with some separate -- you can't use this at t equals 0, because it's calling for a value at minus delta t and you haven't got it.
So you need -- it takes a separate method.
You can deal with that, create -- use Runge-Kutta, for example, to get that first step.
But then there's another problem with these backwards step methods, which again, can be resolved.
So this is a live method.
The other problem is, if you want to change delta t -- suppose you want to cut delta t in half, then this is looking for a value that's a half delta t back in time and you haven't got that either, but you've got values 1 delta t back and 2 delta t back and some interpolation process will produce that half value.
So what am I saying?
I'm just saying that getting started takes a special attention, changing delta t takes a little special attention, but still, it's all doable.
So that this method, which I gave a name to last time -- I've forgotten what it was.
I think it was something like ode 15 s for stiff is much used for stiff equations.
So now I'm left -- let me not only write down Runge-Kutta's name, but the other -- so multistep I've spoken about.
I just -- this is -- I'm just going to write this to remind myself to say one word about it -- dae.
Can I say one word even now?
I'm sorry.
Then I'll say one word later.
So that's a differential algebraic equation.
That's like a situation which comes up in chemical engineering and many other places, in which out of our n given equations, some of them are differential equations, but others are ordinary algebra equations, so there's no d by dt in them.
Nevertheless, we have to solve and there is a d by dt in the others.
So this is a whole little world of daes, which you may never meet.
I have never personally met, but it has to be dealt with and there are codes -- I'll just mention the code dasil by [?
pencil ?]
I'll finish saying the one word.
A model problem here would be some matrix times u prime equal f of u and t, so a sort of implicit differential equation, because I'm not giving u prime, I'm only giving mu prime.
If m is singular -- otherwise I could multiply by m inverse and make it totally normal, but if m and sometimes t or m may depend on u and t, it could go singular.
If it goes singular, then this system is degenerating and we have to think what to do.
this dasil code would be one of the good ones that does.
So that's sort of like my memory to myself to say something about the daes before completing this topic of ordinary differential equations.
Now for Runge-Kutta.
Let me take p equal to 2 first.
The famous Runge-Kutta is the one at p with forth order accuracy.
That will take more of the little internal compounded steps than p equal to 2 takes, but p equal to 2 makes the point.
So this'll be simplified Runge-Kutta -- p equal to 2.
It'll be un plus 1 minus un.
You see, it's just one step, but what goes here?
Do I remember?
The notes will tell me.
I better -- since it's going to be kept forever, I better get it right.
It involves fn and f of f, so Runge-Kutta, here we go.
It has a half of fn, the usual Euler figure out this slope.
But then if the other half, is the same f at -- I take Euler, so doing this part allowed me to taken a forward Euler and now I put that forward Euler step in as like a un plus 1 delta tfn at time -- what's the right time?
Probably -- that's sort of -- this is like un plus 1, but we didn't have to do -- we're not implicit and so I presume that that's a time n plus 1.
So I've written in one line what the code would have to write in two lines.
The first line of code would be compute f at the old un, then take an Euler step, multiply it by delta t and add to un, so this give something that's like un plus 1, but it didn't cost us anything.
So can you compare that with trapezoidal?
You see the comparison with trapezoidal is trapezoidal has the 1/2 fn the same, but it has the 1/2 f n plus 1 at the new step involving the new un plus 1 where this 1 -- here is something like it, but something we already knew from Euler.
So that's the natural idea.
All these ideas actually would occur to all of us if we started thinking about these for a few weeks.
B I guess I'm not planning to spend a few weeks on that, so I'm just jumping ahead to what people have thought of.
So that's, you could say, two step Euler.
There's two evaluations of f within a step.
So what's p equal 4?
The famous Runge-Kutta is four of those.
So you take -- there's an Euler step.
That'll be step one.
Then you'll stick that into something, that's step n plus 1/2.
That'll be a second evaluation of f. That'll give you some output.
You plug that in and that will actually, if it's correctly chosen, be another, better approximation of 1/2, at n plus 1/2.
You put that into the fourth one, which will give you something close to un plus 1 and then you take the right weights.
Here they were 1/2 and 1/2.
Anyway, you can do it.
The algebra begins to get messy beyond p equal 4, but you can go beyond p equal 4.
I mean, there's books on Runge-Kutta methods because the algebra gets -- of these f of f of f. If there's anything that -- any construction in math that looks so easy, just take f of f of f of f of a function, of a number.
That seems so simple, but actually it's extremely hard to understand what happens.
Maybe you know that Mandelbrot's fractal sets and all these problems of chaos come exactly that way and they're extremely hard to see what's happening.
When you do repeated composition, you would call that f of f of f. So Runge-Kutta stops at 4 and then there is actually an implicit Runge-Kutta that people work on, but I don't dare to begin to write that down and I won't even write the details of that one, which are in our notes and in every set of notes.
But what's the point?
First, we get the accuracy, so what's the other thing you have to ask about?
Stability.
So what does our stability calculation give for Runge-Kutta?
The thing that you have to work with, actually, turns out to be quite nice.
It's just the series for e to the at chopped off at the power, however far you're going.
So this is Runge-Kutta stability, p equal to 2.
The growth factor will be from -- if I just do this and I take the model problem, f equal au, you could -- if I took that out of there, the growth factor will be 1 plus a delta t plus 1/2 a delta t squared.
Stop.
That's what this would give for the model problem as the growth factor that multiplies each u to get the next u.
What p equal 4 would give would be the same thing plus -- it's just the right -- it's just the exponential series chopped off at the fourth term.
So of course, you see immediately, what's the discreditization error?
It's still to t of the fifth.
It's the first missing term that Runge-Kutta didn't capture, multiplied by 1 over 5 factorials, so you know right away that the constant is 1 over 120, which is pretty good.
But the question is the stability.
So what do we want to know here?
We want to know -- so that means that a delta tb sum -- because that same number is coming up.
Let me call it z, right?
So the stability test for p equal to 2 is to look at the magnitude of 1 plus z plus 1/2 z squared and we want it to be less than or equal to 1.
Now, the point is, this is something -- I hope you try this this weekend.
Somehow get MATLAB to plot in the complex plane, so you really -- up there now we've just -- at some point z, some negative value of z -- it's going to hit 1 and then after that, it's going to be unstable.
Of course, there's got to be a point where it's unstable.
This is an explicit message.
Runge-Kutta is explicit.
So it can't -- if the problem is too stiff, we're going to be killed, but if it's an ordinary problem, this is a winner.
So p equal 4, this is the code ode 45, which uses -- combines Runge-Kutta with 4 on a higher order formula to get an estimate for the error in that step.
So I could make some comment on predictor, corrector methods, but I'll leave that in the notes.
So what does that region look like?
Then the other one is, what does the region 1 plus the z plus 1/2 c squared plus 1/6 c cubed plus 1/24 v to the fourth, less or equal 1 -- what's that region look like?
I mean, if those regions are small, the method loses.
If those regions are bigger than we get from Adams-Bashforth or backward differences, then the methods in there are good, successful.
I've forgotten exactly what the picture is like there.
Why am I in the complex plane?
Because the number a -- z is a delta t. That number a can be complex.
How can it be complex?
I'm not really thinking that our model equation u prime equal au will be complex, but our model system would be u prime equals a matrix times u and then it's the Eigen values of that matrix that take the place of little a.
So we have n different little as going for a system and the Eigen values of a totally real matrix can be nonreal, can be complex numbers.
Let's just think when that happens and then I'll -- what does the figure looks like for p equals 4?
I sort of remember that the figure even captures a little bit of that plane and then it goes -- why should I mess up the board with -- it's a pretty good sized region.
It's probably not shaped like that at all, but the point is, maybe this reaches out to about e, minus e. I think it does, somewhere around -- I don't know if that's an accurate -- right, reaches out around minus -- and reaches up to around 2.
So in this last second, I was just going to say -- and we'll see it plenty of times -- where do we get complex -- where do we get nice, real, big systems, stiff systems with complex Eigen values in real calculations?
By real by real calculations, I really mean pdes.
So in a partial differential equation -- so this is just a throw away comment that we'll deal with again -- in partial differential equations, a uxx term or a uyy term that get multiply typically by diffusion, those are diffusion terms, those produce symmetric things, symmetric negative definitely, but convection, advection terms, velocity terms that get multiplied by some -- that are first derivatives -- those produce -- those are antisymmetric.
Those produce these complex Eigen values that we have to deal with.
So everybody wants to solve convection diffusion.
Convection has these terms, diffusion has these.
The coefficient may be very small.
The hard -- that's the hard case, when the convection is serious.
It's pushing us up the imaginary axis, where this is bringing us to the left and we're going to spend time thinking of good methods for that.
So my homework -- would you like figure out this region and make a better pictures?
Get MATLAB to do it and try any one of these methods on the model problem.
Try a method or two.
See whether the stability, the theoretical stability limit is serious.
I mean, does the limit of a delta t -- can we -- as I'm driving to MIT, I sometimes exceed the speed limit, the stability limit.
Is that OK or not with finite differences?
I'll see you Monday.
Alright, thanks.
Have a good weekend.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license, and MIT OpenCourseWare in general is available at ocw.mit.edu
Vasily.
Vasily Strela who works now for Morgan Stanley, did his PhD here in the math department, and kindly said he would tell us about financial mathematics.
So, it's all yours
Let me thank Professor Strang for giving this opportunity to talk here, and it feels very good to be back, be back to 18.086.
So, a few more words about myself.
I've been Professor Strang's student in mathematics about ten years ago.
So after receiving my PhD, I taught mathematics for a few years.
Then ended up working for a financial institution for investment bank, Morgan Stanley in particular.
I'm part of an analytic modeling group in fixed income division.
What we are doing, we are doing multiplications in finance and modeling derivatives, fixed income derivatives.
That's actually what I'm going to talk about today.
I want to show how 18.086, it's a wonderful class which I admire a lot, which applications it has in the real world, and in particular in finance and derivatives pricing.
Let's start with a simple example, which actually comes not from finance, but rather from gambling.
Well, let's look at horse racing or cockroach racing, if you prefer.
Suppose there are two horses, and sure enough people bet on them and bookie uses a clever one, very scientific-minded guy and he made a very good research of previous history of these two horses.
He found out that the first horse has 20% chance to win.
The second horse has 80% chance to win.
He is actually right about his knowledge about chances to win.
When that becomes general public, people who bet, they don't have access to all information, and the bets are split slightly differently.
So the 10,000 is placed on the first horse, and 50,000 is placed on the second horse.
Bookie, sticking to his scientific knowledge, places the odds 4:1.
Meaning that if the first horse wins then whoever put on the horse gets his money back and four times his money back on top of it.
Or if the second horse wins then whoever put money on this horse will get the money back and 1/4 on top of it.
So, let's see.
What are chances for bookie to win or lose in this situation?
Well, if the first horse wins then he has to give back 10,000 plus 40,000, 50,000, and he got 60,000, so he gains 10,000.
Well, good, good for him.
On the other hand, if the second horse wins, then he has to give back 50,000 plus 1/4 of 50, which is 12,500, so 62.50 altogether, and he loses $2,500.
After many runs, the expected win or loss of the bookmaker is the probability of the first horse to win times the expected win, plus the probability of second horse to win times the expected loss, which turns out to be exactly zero.
So in each particular run, bookie may lose or win, but in the long run he expects to break even.
On the other hand, if he would put the chances, he would set the odds according to the money bet, 5:1, what would be the outcome?
Well, if the first horse wins he gives back 10,000 plus 50,000, 60,000 exactly the amount he collected.
Or if the second horse wins, again, he gives back 50 plus 1/5 of that 60, he breaks even.
So no matter which horse wins in this scenario, the bookie breaks even.
How bookie operates, well he actually charges a fee for each bet, right.
So the second situation is much more preferable for him.
When he doesn't care which horse wins, he just collects the fee.
Well here, he may lose or gain money.
This is quite beautiful observation, which we will see how it works in derivatives.
So now back to finance, back to derivatives.
So we are actually interested in pricing a few financial derivatives, and what is a financial derivative?
Well, a financial derivative is a contract, pay of which at maturity at some time c, depends on underlying security -- in our case, we always we will be talking about a stock as underlying security, and probably interest rates.
What are the examples of financial derivatives?
Well, the most simple example is where it's probably a forward contract.
Forward contracts is a contract when you agree to purchase the security for a price which is set today -- you've agreed to purchase the security in the future for the price agreed today.
Well, for example, if you needed 1,000 barrel of oil to keep your house, but not today but rather for the next winter, on that account you don't want to take the risks of waiting until the next winter and buying oil then, you would rather agree on the price now and pay it in the future and get the oil.
What the price should be?
What is the fair price for this contract?
Well, we will see how to price it.
Well, the few observation here is that this line represents the payout -- it's always useful to represent the payout graphically.
This is just a straight line because the payout of our contract is f minus k at time c. This actually gives the current price of the contract for all different values of the underlying.
Usually, the price of the forward contract is set such that for the current value of the underlying, the price of the contract is zero.
It costs nothing to enter a forward contract, so that's why it intersects zero here.
What are other common derivatives?
Another common derivative are call, and I put European call input here.
Don't be confused by European or American.
It has nothing to do with Europe or America, it has to do with the structure of the contract.
European basically means that the contract expires at certain time c. American means that's you can exercise this contract at any time between now and future.
We'll be talking only about European contracts.
So European call option is the contract which gives you the right, but not obligation, to purchase the underlying security at set price, k, which is called strike price, at a future, time c, which is expiration time.
So if your security at time c ends up below k, below the strike, then sure enough there is no point of buying the security for a more expensive price.
So the contract expires worthless.
On the other hand, if your stock ends up being greater than k at expiration time c, then you would make money but my purchasing this stock for k dollars and your payout will be f minus k , and this is a graph of your payout.
This line here, as we will see, is the current price of the contract, and we'll see how to obtain this line in a few minutes.
Another common contract is a put.
While call was basically a bet that your stock will grow, right, the put is the bet that your stock will not grow.
So, in this case, the put is the right, but not the obligation to sell the stock for a certain price k. Here is the payout, which is similar to the put, but just flipped.
This is the current price of a put option.
Calls and puts, being very common contracts are traded on exchanges -- Chicago exchange is probably the most common place for the calls and puts on stocks to trade.
I just printed out a Bloomberg screen, which gives the information about a calls and puts on IBM stock.
So I did it on March 8, and the IBM stock was trading at this time at $81.14, and here are descriptions of the contract, they expire on 22nd of April, so it's pretty short-dated contract.
They can go as far as two years from now, usually.
Here is a set of strikes, and here are a set of prices.
As you can see, there is no single price, there is always a bid and ask, and that's how dealers and brokers make their money -- like a bookie, they basically charge you a fee for selling or buying the contract.
That's how the money are made -- they are made on this spread, but not on the price of the contract itself, because as we will see in a second, we actually can price the contract exactly, and there is no uncertainty once the price of the stock is set.
There are plenty of other options.
Slightly more exotics contracts, either digital which pays either zero or one depending on where your stock ends up.
It probably is not exchange-rated, also -- I'm not sure.
There are hundreds, if not thousands, of exotic options where you can say that well, how much would be the right to purchase a stock for the maximum price between today and two years from now.
So it will be past-dependent, depending on how the stock will go, the payout will be defined by this path.
There are American options where you can decide your option any time between now and maturity, and so on and so forth.
So, just before we go into mathematics of pricing, just a few observations and statements.
First of all, it turns out that thanks to developed mathematics, mathematical theory, if you make certain assumptions on the dynamics of the stock, then there is no uncertainty in the price of the option.
You can say exactly how much the option costs now, and that's what provides, and this is a big driver for the market.
So dealers quote these contracts and there is a great agreement on the prices.
The price of the derivative contract is defined completely by the stock price and not by risk preferences of the market participant.
So it doesn't matter what are your views on the growth prospects of the stock.
It will not effect the price of the derivative contract.
As I said, so the mathematical part of it comes into giving the exact price without any uncertainty.
So let's consider a simple example now.
Let's assume that we are in a very simple world.
Well, first of all, in our world there are only three options -- the stock itself, the riskless with money market account, meaning that it is an account where we can either borrow money or invest money at the request rate r, and finally our derivative contract.
Here we are not making any assumptions of what kind of derivative contract it is -- it could be forward, it could be call, it could be put, it can be anything.
Moreover, our world is so simple, but first of all, it's discrete, and second of all, there is only one time step to the expiration of power of contract, dt.
Not only there is only one step left, we actually know exactly what our transition probabilities.
There are only two states at the end, and we know the transition probability.
So with probability p, we move from the state zero to the state one, and with probability of one minus p, we move to the state two.
And I just noticed, because this is riskless money market account, it's the same in both cases.
You just invest money and it grows with risk-free interest rate.
So, what can I say it about the price of our derivative f?
Well it's simple-minded -- well, let's start with the forward contract.
We know what the payout in delta t of our forward contract will be, it will be just the difference between the stock price and our strike.
Well, a simple-minded approach would be -- well, we know the transition probabilities, let's just compute the expected value of our contract, and that's what we would expect to get if there were many such experiments.
Well, you take the probability of going to state one, you multiply by the payoff at stage one, take minus p for probability of going to state two.
Multiply by the payout in that state two.
Sum them up and you get the expression, as I said, the common thing to choose the strikes is that the contract has zero value now, so you get your strike.
Well, in particular you could say that if you research the market well and you know that the stock has equal probability of going up and down, then actually you expect your strike to be an average of end values of the stock.
But as we can imagine, following our bookie example, this is not the right price.
There is actually a definite price which doesn't depend on condition probability.
Here is the reason why there is a definite price.
Well let's just consider a very simple strategy.
Let's borrow just enough to purchase a stock.
So let's borrow f zero dollars right now and buy the stock for this money.
And let's enter the forward contract.
Well, by definition forward contract has price zero now, so we enter the forward contract.
Now, at the time dt when our contract expires, what happens?
Well, we deliver our stock, which we already have in our hand for an exchange of k dollars.
That's our forward contract.
On the other hand, we have to repay our loan, and because it was a loan, it grew.
It grew to s zero times e to the rdt.
Now let's see.
What would happen if k was greater than f times e to the rdt?
Then we know for sure, we know now for sure, that we would make money.
There is no uncertainty about it now.
Similarly, if k is less than this value, then we know that we will lose money.
That's not how the rational market works.
If everybody knew that by setting this price you would make money, people would do it all day long and make infinite money.
So there will be no other side of the market.
So the price had to go down.
So the only choice for k, the only market-implied choice, is that k has to be equal to f times e to the rdt.
As you can see, it doesn't depend on transition probabilities at all.
That's what market implies us.
That's the price of forward contract, and that actually explains why when I was watching the forward contract, current price was just the straight line, it's just discounted payoff.
The payout is linear, so just the parallel to the payoff.
That's the idea basically.
The idea is to try to find such a portfolio of stock and the money market account with such a payout , which will exactly replicate the payoff of our derivative.
If we found such of a portfolio, than we know for sure that the value of this portfolio, the replicating portfolio today is equal to the value of the derivative, because otherwise, you would make or lose money riskless.
That's no-arbitrage condition.
So, can we apply it to our general one step world?
Well, if we have a general payout f, what we want to do, we want to form a replicating portfolio such that at expiration time, it will replicate our payouts.
So we want to choose such constants a and b that such that the combination of stock and money market account in both states will replicate the payout of our option.
Then if we are able to find such constants a and b, then we just look at the current price of the contract and it has to be equal to the current price of our derivative.
Well, but now a particular case, this is easy.
It's just two linear equations with two unknowns, easily solved, and here is current price of our derivative.
No matter what payout is -- I mean you just substitute the payout here, and if you know s1 and s2, and s2 that's it.
A useful way to look at this, just to re-write this equation, is in this form, and then notice that actually the current price of our derivative can be viewed as a discounted expected payout of the derivative, but with very certain probability.
This probability, it doesn't come from statistical properties of the stock or from any research, it actually is defined by the market.
So it's called a risk-neutral probability.
So this probability doesn't depend on the views on the market by the market participants.
An interesting observation is that actually the value of this stock, the discounted value of the stock is actually also is expected value of our outcomes on this risk-neutral probability.
That's basically general idea.
Now let's move one notch up and try to apply these idea to continuous case.
Well, if you live in continuous world now, we need to make some assumptions on the behavior of the stock.
The very common assumption is that the dynamics of the stock is lognormal.
Lognormal meaning that the logrithm of the stock is actually normally distributed.
So, here u is some drift, sigma is the volatility of our stock, and dw is the [INAUDIBLE] process, w's the [INAUDIBLE] of process such that dw is normally distributed with mean zero and variance square root dt.
Our approach would be to find the replicating portfolio.
What that mean, it means that we want to find such constants over time dt.
So we assume that a and b are constant over the next step, dt, such that the change in our derivative is a linear combination with this constant of the change of our underlying security and the change of money market account.
Now we just need to look more closely at this equation.
First of all, let's concentrate on df.
So, f, our derivative is a function of stock value you time.
But unfortunately, our stock value is stochastic, so the f is not that simple, and to write dfo we have to use a famous -- it's a formula from stochastic calculus, which actually is analagous of Taylor's formula for stochastic variables.
Let's see.
If our f will not be stochasticated, it would be completely deterministic and depend only on dt, then there would be no term and differential f is just the standard expression.
On the other hand, if we have dependence on stochastic variables, then we have to have more terms, and why this happens.
Well, in very rough words is that because the order of magnitude of dw is higher than dt's -- its square root of dt.
So we have to make into account more terms, and particularly we have to -- to take in part next order of df squared.
Formally, df square can be written this way, and again, very rough explanation is as follows.
If we would square this equation there will be three terms there.
One would come from the square of this term, and this would be of the order of dt squared, next order of magnitude -- much smaller than dt.
The second term will be cross-product of dw dt.
What order of magnitude that we are talking about, this is dt to the power 3/2, again, much smaller than dt.
On the other hand, the third term will be the square of dw, this is the order of magnitude of dt, so that's what dw is.
And that's what either formula is about.
Now we are basically, now all terms here, and let me stress out that this term, dv it is not stochastic, it's completely deterministic because we know that d grows with the rate r, that's what it is.
So we substitute all those terms into our replicating equation.
We collect the terms.
We get this equation.
And again, there is the deterministic part, there is stochastic part.
So the only way for this equation to hold is this term to be equal to this term, and this term to be equal to this term, and that's what's written out here.
So again, two equations, these two unknowns, and here is answer.
Finally, let's take af to another part.
Notice that this part of our equation is completely deterministic.
So we know how it will grow.
So basically, d of f minus as, which is d times dv, is r times d times dc.
We know all other terms, we substitute them here, take something to the left hand side and we have this equation.
So this is partial differential equation for our derivative f, as a function of f and t, of second order, and this equation is the famous Black-Scholes equation.
It was derived by Fisher Black and Myron Scholes in their famous paper published in 1973.
Myron Scholes and Robert Merton actually received Nobel Prize for deriving and solving this equation in '97.
Black was already dead by the time.
This is really the cornerstone of math finance.
The cornerstone is because using the replicating portfolio, using this reasoning, we were able to find an exact equation for our derivative.
So a few would remarks on Black-Scholes.
So first of all, we make some assumptions on the dynamic of the stock, but we never made any assumptions on our derivative.
Which means that any derivative has to satisfy this equation, and that's very strong result.
So if you assume that our stock was normal, which is not a bad assumption and agrees quite well with the market, then we basically in principle can price any derivative.
We know the equation for any derivative.
The other thing is that our Black-Scholes equation doesn't depend on the actual [UNINTELLIGIBLE] [?
mu ?]
in the dynamics of our stock.
So again, it is the manifest of risk-neutral dynamic.
Not only we wrote down the equation for our derivative, we also found a replicating portfolio.
So in other words, we found a hedge and strategy, meaning that at any given time we can form this portfolio with rates a and b.
If we hold both the derivative and both replicating portfolio all together, this is zero sum gain.
We know that no matter where stock moves, we will not move money or gain money.
So if we just charge bid offer on the derivative, if we charge a few on the contract, we can hedge ourself perfectly, buy the contract or sell the contract, hedge perfectly ourself and just make money on the fee, that's it.
Finally, more mathematical remark is that actually after a few manipulations, a few change of variables, the Black-Scholes equation comes out to be just a heat question, which you already saw in this class.
This is very good news.
Why is this good news?
Well, because heat equation is very well studied.
So the solutions are well-known, and numerical methods, the ways to solve it, and particular the numerical ways to solve it are well-known.
So we are in business.
But as any partial differential equation, the equation itself doesn't make much sense because to find a particular solution we need boundary and initial condition.
And although any derivative which defines Black-Scholes equation, the final and boundary condition will vary from contract to contract.
Here are a few examples of the final and boundary conditions.
Here, an interesting remark that usually we would talk about initial condition, here we are talking about final condition if time goes in reverse -- we know the state of the world at the end, at expiration, not today.
So, here are final and boundary conditions for call and put, and let's look a little bit at the pictures for our call and put to see where they come from.
So for example, for calls, well, this is our final condition, right, this is defined by the payout.
Under the boundary condition, well, what happens, we put them at zero at an infinity, which shows [INAUDIBLE PHRASE], and why, well, because if stock hits zero then it stays in zero.
That's what our dynamics show.
So the value of power of contract at maturity will become zero.
[UNINTELLIGIBLE PHRASE] the stock grows, grows to inifinity, a good assumption to make is that actually it becomes similar to stock itself, so it would just become parallel to the stock, and that's the conditions between both here.
Similar, for the put you can derive these conditions, and again, that's because it is a heat equation.
It turns out that for a simple derivative such that the calls and puts, it is possible to find an exact analytic solution.
Here are exactimated set of solutions for a call, put and the digital contracts.
Well, not surprising again, I mean they're all connected to the error function, so to the normal distribution, basically, as the solutions of heat equation ought to be.
Why do they look exactly the same?
If we have five minutes at the end, we'll probably shed some light on the specific form of equations.
But just trust that we can see that it's discounted, and what I'm saying is expected value of our payout on the [?
risk ?]
[UNINTELLIGIBLE] for measure.
Here is an example -- it's of a particular call option on the same IBM stock.
So I chose the short-dated contract, just avoid the dividend payment.
So it's a contract expiring on March 18.
So there is some based expiration.
The stock, as we saw, the expirations of stock, as we saw, is what's trading at 81.14.
The volatility is somewhere around 14%, estimated either from other options or from historically.
Here is the price of our contract.
I also have a simple Black-Scholes calculator here, and let's see if we can match this price.
So let's see, I believe the volatility was 13%, right, 13.47.
The interest rate, it's already here.
As we all know the Fed just bumped the interest rate, so at 4.75% right now.
The striker of our option was 80 times expiration was actually 10 days, and this should be measured at a fraction of year.
So we divide 10 by 365.
This stock was trading at 81.14, if I'm not mistaken.
Here is the price of our call options contract, which is 150.
Well, it's within the offer.
So maybe our volatility's slightly off and if increase at, let's say, 2.14% it will go slightly up.
152.
Well, in general, let's play a little bit with it.
Well, it is very short-dated option, so the value of our option is very close to the payout.
So if we increase the time to maturity, let's make it three years just to see where, so now what if our option is -- well, that's what it is.
If increase volatility, sure enough, let's make it 30%.
So what do we expect?
We expect if volatility higher, then certain things higher, so the value of our contract should go up, and it sure does.
So basically that's how Black-Scholes works.
[?
Plane ?]
[?
Hills ?]
does contract trade on the market, but unfortunately not all of these contracts are so simple as calls and puts.
First of all, there are many more complicated products with more difficult payout, which will constitute different and [UNINTELLIGIBLE] will discontinue final conditions on our Black-Scholes equation.
Moreover, we made an assumption that the volatility is constant with time, and interest rate is constant with time.
It is certainly not true for the real world.
Volatility probably should be time dependent, and this would make the coefficients in our Black-Scholes equation time dependent.
Unfortunately, these cannot be solve analytically.
So in most of the cases in practice, we will have to use some kind of numerical solution.
Finding different methods is the typical approach for the heat equation.
As you know, both [UNINTELLIGIBLE PHRASE] schemes, and you will discuss some of those in 18.086.
Tree methods.
Tree methods meaning that we go back to our one step tree, and basically assume that our times expiration is many [UNINTELLIGIBLE] time steps away and will grow the tree further, so from this node we have some more nodes, and so and so forth.
That would imply the final condition at the end, and discount back using risk-neutral probabilities, and get the price down.
So those are called tree methods.
One can show that actually those tree methods are equivalent to find a difference -- expect to find a difference scheme.
Those are very popular.
But again, in tree methods, what is very important is to set the probability from your tree the transition probability was the right one.
Since the right one risk-neutral probability.
Probabilities implied by the market, actually.
Another important numerical method is Monte Carlo simulation where you would simulate many different scenarios of the development of your stock after the maturity.
Then basically using this path you will find the expected value of your payout.
But again, in order for this expected value to be the same as the risk-neutral value , as the arbitrage-free value, you have to develop your Monte Carlo simulations is risk-neutral probability.
So, risk-neutral volition is extremely important.
Here is actually the general risk-neutral statement, which one can prove is that actually the value of any derivative is just discounted expected value of the payout of this derivative at maturity, but you have to take this expectation and divide measure.
Using the right measure, meaning that you have to set correctly the transition probability -- you have to make them market-neutral.
Under this measure, actually the dynamics of our stocks looks slightly different, and as you can see, our drift becomes the interest rate.
So either [UNINTELLIGIBLE PHRASE] you measure, everything grows with our risk-free interest rate.
Just to shed a little bit of light on how we go at the solutions for calls and puts, Black Shoals solutions for call and put, well this is the distribution of our stock, the normal distribution of our stock at time t, and if we take this distribution and integrate our payout of our co-option against this distribution -- in other words, find the expected value of payout of our full option under the risk-neutral measure, then sure enough you will get [UNINTELLIGIBLE PHRASE].
This illustrates the best because what is digital -- digital is just the probability to end up at both a strike at time t, right?
So if you integrate this lognormal pdf from the strike k to the infinite, that will be your answer.
This is a good exercise in integration to make sure that it's correct.
So let's see.
To conclude, what we've seen.
So, we have seen that more than derivatives business makes use of quite advanced mathematics, and what kinds of mathematics is used there.
Well, partial differential equations are used heavily.
Numerical methods for the solution of this partial differential equations are naturally used.
In order to get the integrations, we actually need to operate in some stochastic calculus, meaning that we need to know how to deal with either calculus, either formula, Gaussian theorem, and so on and so forth.
The other thing is to be able to build simulations to solve the heat equation and all other equations that you might encounter.
The topic which we didn't touch upon is statistics because, of course, very advanced statistics is used for many, many things for analyzing historical data, which can be quite beautiful for trading strategies and many others.
Besides these five topics, there is much, much more to mathematical finance, which makes it a very, very exciting field to work in.
That's what I wanted to talk about.
Thank you very much for your attention
Maybe I'll ask a firm question about boundary conditions, because you had said that those are different for different contracts and how do you deal with them in the finite differences or the tree model or whatever.
What would be a one typical one?
Well, typical one -- two very typical ones.
So those you basically make a grid of your problem, in particular you build a tree, which is actually a grid of all possible outcomes.
You set them up as the ends, so your tree grows.
So you set your boundaries here at the end, and well you set probably some initial -- this is final condition, so you set some boundary conditions here.
So this is your time t. This is t zero times t -- this is zero, this is 1, this is 2, this is t. So you set your payout here, so it will be maximum of s minus k and zero
How many time steps might you take in this?
Well, you would do like daily for three months -- you do three month options
Maybe 100 steps
Yeah, something like that.
Well, if it's two year option, that you probably would do it weekly or something like that
You don't get into large, what would be scientifically, large-scale calculating
No, in finance you usually don't keep this problem--
In finite differences, do you use like higher order as opposed, where you had second derivatives, would you always use second differences or second order accuracy?
In general, yes.
In general, second order access.
In general you don't go higher.
I mean the precision -- well, it's within tenths, right.
So you can not do better than that.
So it depends -- well, it depends what kind of amount you are dealing with.
If you're actually selling and buying units of stock, you might consider something more precise.
But it's very problem-defined.
So that's how we deal with it
Any questions?
We can put the mic on if you have a question
[UNINTELLIGIBLE PHRASE]
Well, it is market proc-- Yes.
I mean this is just a numerical solution.
So yeah, it is market process.
and basically all stochastic calculus is about market process, continuous market process
Is the mathematics that you get involved with pretty well set now or is there a need for more mathematics, if I can ask the question that way?
Yeah, well in this field it is probably quite well set.
But if you get into more complicated fields, especially into credit modeling, the model for the credits of certain companies, then mathematics is not quite set, because there you start talking about jump processes and [UNINTELLIGIBLE] Weiner processes, not just lognormal processes.
This stochastic differential equations become very hard, but may be analytically practical.
So from this point of view there is -- but it's not a fundamental mathematics, it's not that you are opening a new field, but definitely trying to solve a stochastic differential equation, which usually boils down to solving a partial differential equation analytically.
Can be pretty hard in mathematical problem, view as a mathematical problem
So you showed the example of Black-Scholes solver.
Everybody has that available all the time?
Oh yeah.
On Chicago trading floor, the traders have calculators where they just press a button and it's just hard-wired there
And they're printing out error functions, basically -- a combination of error function, yeah
Well, sure enough, nobody uses just -- I mean this was their approximate example and that's why I chose such short-dated stock that before it pays any dividends, and where we cannot the volatility is constant, and so on and so forth, match the prices.
Otherwise, the prices wouldn't match
Thank you.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
Let's get started.
Could I start first with the announcement of a talk this afternoon.
I know your schedules are full, but the abstract for the talk I think will go up on the top of our web page.
It's a whole range of other applications that I would hope to get to by an expert.
Professor Ronicker, his applications include optimal control, which is certainly a big area of optimization.
Actually, MathWorks, we think of them as doing linear algebra, but their number one customer is control theory world, and it totally connects with everything we're doing.
He's also interested in adaptive meshing for finite element or other methods -- how to refine the mesh where it pays off in arrow problems, all sorts of problems.
The mass behind it is this same saddle point structure, same KK -- when he says KKT equations, that's our two equations.
I've been calling them sometimes Kuhn-Tucker, but after Kuhn and Tucker, it was noticed that a graduate student named Karush, also a K, had written a Masters thesis in which these important equations appeared.
So now, they're often called KKT equations.
I think Ronicker will do that.
Anyway, I don't know if you're free, but it'll be a full talk in this program of computational design and optimization.
I don't know if you know MIT's new Masters degree program in CDO.
So it's mostly engineering, a little optimization that's down in Operations Research, and a couple of guys in math.
So that's a talk this afternoon, which will be right on target for this area and bring up applications that are highly important.
Well, so I thought today my job is going to be pretty straightforward.
I want to do now the continuous problem.
So I have functions as unknowns.
I have integrals as inner products.
But I still have a minimization problem.
So I have a minimization problem and I'll call, it's often a potential energy, so let me use p. Our unknown function is u, so that's a function -- u of x, u of xy, u of xyz.
Instead of inner products we have integrals, so there's a c of x.
This is going to be a pure quadratic, so it's going to lead me to a linear equation.
In between comes something important -- what people now call the weak form of the equation.
So, this is would be the simplest example I could put forward of the calculus of variation.
So that's what we're talking about.
Calculus of variation.
What's the derivative of p with respect to u?
Somehow that's what we have to find and we're going to set it to zero to minimize.
Then why do we know it's a minimum?
Well, that's always the second.
The quadratic terms here are going to be positive.
So we have a positive-definite problem.
Positive-definite means things go this way, convex, and you locate the bottom, the minimum is where the derivative is zero.
But what's the derivative?
That's the question.
It's not even called derivative in this subject, it's called the first variation.
Instead of saying first derivative, I'll say first variation.
Instead of writing dpdu, I'll write -- where can I write it?
So this is key word, so this is going to be the first variation, and I'm going to write it with a different d, a Greek delta, dpdu.
It's just sort of a reminder that we're dealing with functions and integrals of functions and so on.
So it's just change the notation in the name a little as a trigger to the memory.
But this board really has not all the details are here, of course.
This is a summary of what the calculus of variation does.
So it takes a minimization problem.
So I'm looking for a function u of x.
Always, since we're in continuous problems we have boundary conditions.
So as always, those could be -- let me imagine that those are the boundary conditions.
So I would call those essential conditions.
Every function that's allowed into the minimum has to satisfy the boundary condition.
So it's a minimum over all u with the boundary conditions, with those boundary conditions.
There are two kinds of boundary conditions, and maybe I'll postpone thinking about boundary conditions till I get the equations, the differential equation inside the interval.
So all these intervals go from zero to one.
I won't put that.
So we're in 1d.
So, how do you find the function?
u of x -- that stands for dudx.
So the given data for the problem is load some source term, f of x, which is going to show up on the right hand side, and some coefficient c of x, which is going to show up in the equation.
They depend on x, in general.
Many, many, many physical problems looks like this.
Sort of steady state problems, I would say.
I'm not talking about Navier-Stokes fluid flow convection, I'm talking about static problems, first of all.
So here's the general idea of the calculus variation.
It's the same as the general idea of calculus.
How do you identify a minimum in calculus?
If the minimum is at u, you perturb it a little by some delta u that I'm going to call v to have just one letter instead of two here.
You say OK, if I look at that neighboring point, u plus delta u, my quantity is bigger.
The minimum is at u.
So we're remembering calculus -- I guess I'm saying I've written that here.
Compare u with u plus v, which you could think of as u plus delta u.
It's like v you might think of as a small movement away from the best function.
In calculus it's a small movement away from the best point.
So let me draw the calculus.
If you think of this blackboard as being function space instead of just a blackboard, then I'm doing calculus of variation.
But let me just do calculus here.
So there's the minimum of u.
Here is u plus v near it.
Could be on this side or it could be on this side.
Those are both u plus v. Well, you maybe want me to call one of them u minus b.
But the point is v could have either sign.
I'm looking at minimum sort of inside where I can go to the right or the left.
So what's the deal?
Well, that point is then at that point or at that point or at any of these other points, p of u plus v is bigger then what it is at the minimum.
That tells us that u is the winner.
Now how do we get an equation out of that?
Calculus comes in now.
We expand this thing -- this is some small movement away from u.
So we expand it, we look at the leading term -- well, of course, the leading term is p of u.
Then what is the next term?
What's the first order, first variation in p when I vary u to u plus v?
Well, it's the whole point of calculus, actually.
The central point of calculus is that this is some function that we call p prime of u times v, plus order of v squared.
When v is small, v squared is very small.
So what's our equation?
Well, if this has to be bigger than p of u -- I could just cancel p of u -- so this thing has to be bigger equal zero now.
I've squeezed it in a corner, but since it's calculus we kind of remember it.
Also, it's easy to learn calculus and forget the main point.
So this has to be greater equal zero.
Now this is going to be -- if v is small, this is going to be very small, so it won't help.
So this thing had better be zero, right?
That had better be zero.
Because if it isn't zero, I could take v of the right sign to make it positive, and take v small, so that this would dominate this.
Maybe I wanted to take v of the right sign to make that negative anyway.
I need p prime of u to be zero.
So that's what I end up with, of course, as everybody knew.
That p prime of u had to be zero.
So in that tiny picture, I've remembered what we know.
Now let me come back to what we have to do when we have functions.
So what's happened?
I compare p of u -- u is now a function, u of x -- with p of u plus v. So I plus in u plus v, I compare with what I get with only u, and what's the difference?
I look at the difference and the difference will have a linear term from u prime plus v prime squared and then I'm going to take away the u prime squared because I gotta compare the two.
So what's left?
There will be a 2u prime, v prime.
I'm maybe just being lazy here.
I'm asking you to do it mentally and then I'll do it a little bit.
So the difference in this comparison is a 2u prime, v prime times c, and the 2's cancel.
There's the difference right there.
What's the difference over in this term?
Well, I have that term and then I have the same term with u plus v, and then when I subtract I just have the term with v. So this is the dpdu that has to be zero for every v. Now I'm really saying the important thing.
This weak form is like saying this has to be zero for every v. Then, of course, in this scaler case, it was like a very small step to decide well, if this is zero for every v, send that zero, which is the strong form.
Are you with me?
So we have a minimum problem, minimize p, we have a weak form, the first variation, the first derivative, the first order term has to be zero for every v. Then if it's zero for every v, that forces the derivative to be zero and that's the strong form.
Now over here it took more space.
But the ultimate idea is the same.
We looked at p of u plus v compared with p of u, subtracted, looked at the linear term and there it is, the first variation.
That has to be zero for every v. I'll just mentioned boundary conditions now, as long as we're at this weak form.
Don't think of this weak form as just some mathematical nonsense to get to the differential equation, because the weak form is the foundation for the finite element method -- all sorts of discrete methods, discretization methods will begin with the weak form, the weighted integral form, rather than the strong form.
I was just going to say a word about boundary conditions.
What are the boundary conditions on v?
So you could say well, v stands for a virtual displacement.
Virtual meaning kind of we just imagining it, it's a displacement that we can imagine moving by that amount, but the whole point is the nature fix the minimum.
What's the boundary condition?
Well, all the candidates have to satisfy these boundary conditions.
So I have to have u plus v at zero also has to equal a, and u plus v at 1 also has to equal b, if I took these simple boundary conditions.
So, by subtraction, I learn the boundary conditions on v. When I say all v, I mean v of zero has to be what?
Zero.
And v of 1 has to be zero -- when we had those boundary conditions.
Different problems could bring different boundary conditions, of course, but this is easier then.
So when I say all v, I mean every function that starts at zero and ends at zero is a candidate in this weak form.
I have to get the answer zero for all those functions.
Now somehow, I want to get to this point, the differential equation, and why don't we give it the name that everybody -- the two guys' names, Euler-Lagrange -- well, pretty famous names.
This is the Euler-Lagrange equation.
So maybe before where I said Kuhn-Tucker or something, if I'm talking about differential equations, I go back to these guys.
Now, how did I get from weak form to strong form?
That a key.
If you see these two steps from the minimum principle to the weak form, that's just, again, plug in u plus v, subtract and take the linear part.
Then it's true for all v's.
Now how do I get from here to here?
Notice that this form is an integrated form.
This form is at every point.
So it's much stronger and much more demanding.
You could say OK, that gives us the equation -- that's the equation as we usually see them.
I'll do 2d and that'll be Laplace's equation or somebody else's equation, but minimal surface equation, all sorts of equations.
Everything.
All sorts of applications including this afternoon's lecture.
How to get from here to here.
Well, there's one trick in advanced calculus, actually.
The most important trick in advanced calculus is integration by parts.
Well, we use those words, integration by parts, in 1d here.
So I'll do that.
We use maybe somebody else's name -- Green's formula or Green's theorem, or Green-Gauss or the divergent theorem or whatever, in more dimensions.
But 1d, I just integrate by parts.
Do you remember how integration by parts goes?
I want to get v by itself, but I got v prime, because when I plugged in u plus v, here out came -- it's the derivative so I've got a derivative.
So how do I get rid of a derivative?
Integrate by parts.
Take the derivative off a v -- can I just do that this quick way.
Put the derivative onto -- I almost said onto u. I'm doing integration by parts and saying how important it is but not writing out every step.
So I take the derivative off the this and I put it onto this, and it's gotta be -- and a minus sign appears when I do that.
Where the heck am I gonna put that minus sign?
Right in there.
Minus.
Then everybody knows that there's also a boundary term, right?
So I have to squeeze somewhere in this boundary term.
So now that I've done an integration by parts, the boundary term will be the v times the cu prime at the ends of the integral.
I think that's right.
What we hope is that that goes away.
Well, of course, if it doesn't, then we have to -- that there's a good reason that we don't want it to, but here it's nice if it goes away.
You see that it does, because we just decided that the boundary conditions on v were zero at both ends.
So, v being zero at both ends kills that term.
So now, do you see what I have here?
I could write it a little more cleanly.
The whole point is that the v -- I now have v there and I have v there so I can factor v out of this.
Just put it there.
That was a good move.
This minus sign is still in here.
So I now have the integral of some function times v is zero.
That's what I'm looking for.
The integral of some function, some stuff, times v, and v can be anything -- is zero.
What happens now?
This integral has to be zero for every v. So if this stuff had a little bump up, I could take a v to have the same bump and the integral wouldn't be zero.
So this stuff can't bump up, it can't bump down, it can't do anything.
It has to be zero and that's the strong form.
So the strong form is with this minus sign in there, minus the derivative -- see, an extra derivative came onto the cu prime because it came off the v, and the f was just sitting there in the linear, in the no derivative.
So, do you see that pattern?
You may have seen it before, but calculus variations have sort of disappeared as a subject to teach in advanced calculus.
It used to be here in courses that Professor Hildebrand taught.
But actually it comes back because we so much need the weak form in finite elements and other methods.
What I wrote over here is the discrete equivalence.
I can't resist, looking at the matrix form for two reasons.
First, it's simpler and it copies this.
Do you see how this is a copy of that?
That matrix form is supposed to be exactly analogous to this continuous form.
Why is that?
Because u transpose f, that's an inner product of u with f -- that's what this is.
That integral.
au is u prime in this analogy.
So this is u prime times c times u prime with 1/2, and that, again, that transpose is telling us inner product integral.
So if I forget u prime and think of it as a matrix problem, that's my minimum problem for matrix.
I want to find an equation for the winning u.
In the end, this is going to be the equation.
That's the equation that minimizes that.
Half of 18085 was about this problem.
While I concentrated in 18085 on this one, because minimum principles are just that little bit trickier.
So that's 18086.
Then in between, people seldom write about but, of course, it's going to work, is that I change u to u plus v, I multiply it out, I subtract, I look at the term linear in v, and that's it.
That would be if I make that just a minus sign and put it all together the way I did here, that's the same thing.
So this is the weak form.
This is the minimum form, this is the weak form, and this is the strong form.
You see that weak form?
Somehow in the discrete case it's pretty clear that -- let's see.
I could write this as -- you see, it's u transpose, a transpose, cav equal f transpose v. The conclusion is if this holds for every v, then this is the same as this.
If two things have the same inner product with every vector v, they're the same, and that's the strong form.
You'd have to transpose the whole thing, but no problem.
So now I guess I've tried to give the main sequence of logic in the continuous case, and it's parallel in the discrete case for this example.
For this specific example because it's the easiest.
Let me do what Euler and Lagrange did by extending to a larger class of examples.
So now our minimization is still an integral -- I'll still say in 1d, I'll still keep these boundary conditions, but I'm going to allow some more general expression here.
Instead of that pure quadratic, this could be whatever.
Now I'm going to do calculus of variation.
Fill in 1d.
Calculus of variation, minimize the integral of some function of u and u prime with the boundary conditions, and I'll keep those nice so that integral's still zero to 1 and I'll keep these nice boundary conditions just to make my life easy.
So that will lead to v of zero being zero, and v of 1 being zero.
What do I have to do?
Again, I have to plug in u plus v and compare this result with the same thing having u plus v. So essentially I've got to compare f at u plus v, u plus u prime plus v prime with f of u and u prime.
I have to find the leading term in the difference.
So I'll just find out leading term, and then will come the integral.
But the first job is really the leading term, and it's calculus, of course.
Now can we do that one?
It's pure calculus.
I have a function at two variables, the function of u and u prime.
Actually, I did here.
I had a u prime there and a u there.
Once I write it down you're going to say sure, of course, I knew that.
So the function of two variables and I'm looking for a little change.
So a little change in the first argument produces the df -- the derivative of f with respect to that first argument times the delta u, which is what I'm calling v. That's the part that the dependence on u is responsible for.
Now there's also a dependence on u prime.
So I have the derivative of f with respect to u prime times the little movement in u prime, which is v prime.
I can't leave it with an equal there, because that's only the linearized part, but that's all I really care about.
This is order of v squared and v prime squared.
Higher order, which is not going to -- when I think of v as small, v prime as small, then the linear part dominates.
So can you see what dpdu is now?
Now I've got to integrate.
I integrate I this, that very same thing.
This -- dx.
That has to be zero for all v. You don't mind if I lazily don't copy that into the--.
That's the weak form.
This was the minimum form, now I've got to the weak form.
This is the first variation.
The integral of the change in f, which has two components when there's a little change in u. I should be doing an example, but allow me to just keep going here until we get to the strong form.
So this is the weak form for every v. Let me just repeat that the weak form is quite important because in the finite element method we have the v's are the test functions, and we discretize the v's -- you know we have a finite number of test functions.
Well, I can't go entirely.
I'll come back to find that out.
Let me stay with this continuous problem, calculus of variations problem.
v is v of x, and it satisfies these boundary conditions.
That's the only requirement that we need to think about here.
What's the strong form?
Also called the Euler-Lagrange equation.
How do I get to that strong form?
How did I get to it before?
I would like to get this into something times v. Here I've got something times v but I've also got something times v prime.
I wanted to get it up here where was times v. What do I do?
You know what do.
There's only the one idea here.
Integrate by parts.
So I have the integral.
Well, dfduv was no problem -- that had the v that I like.
But it's this other guy that has a v prime and I want v. So, it doesn't take too much thinking.
Integrate by parts, take the derivative off of v, get a minus sign, and put a derivative -- can I do it with a prime, but I'll do better below -- onto this, and then there's a boundary term, but the boundary term goes away because of the boundary condition.
So now I have the v. Can I make it on this board?
There's the dfdu multiplying v, and then there's the minus -- this is the derivative d by dx of dfdu prime.
Now all that is multiplying v and giving me the integral of zero.
I promise to write that bigger now.
But, again, the central point was to get the linear term times v. That's always the main point.
Then what's the conclusion?
What's the Euler-Lagrange equation?
This integral is this quantity times v and v can be anything, and I gotta get zero.
So, what's the equation?
That stuff in brackets is zero.
That's the Euler-Lagrange equation.
Finally, let me just write it down here.
Euler-Lagrange strong form.
In this general problem it would be dfdu minus d by dx of dfdu prime equals zero.
Would you like me to put on what a -- if f happened to depend on u double prime, it would be a plus -- this would be what would happen just so you get the pattern equalling zero.
So I've gone one step further by allowing f to depend on curvature, and writing down what the resulting term would be in the Euler-Lagrange equation.
Could you figure out why it would be that?
Where with this thing have come from?
It would have come from there would have been a dfdu double prime times v double prime in the weak form.
Then I would have done two integrations by parts -- two minus signs making a plus, two derivatives moving off of v and onto the other thing and it would give me that.
All times v, but now I've got everything times v, it's true for every v, so the quantity has to be zero.
That's the Euler-Lagrange equation strong form 1d problem.
Yes?
[INAUDIBLE PHRASE]?
Oh, you're right.
You're right.
If we had this situation then we would be up to -- this would typically be a fourth order equation and we would have two boundary conditions at each end.
Absolutely.
When I slipped in this just to sort of show the pattern, I didn't account for the boundary conditions.
That would be one level up as well.
Exactly.
And nature -- well, fortunately we don't get many equations of higher degree than four.
This would be like the beam equation, the parallels of that would be a beam equation or a plate equation or a shell equation, God forbid.
In shell theory they're incredibly complicated because they're on surfaces.
But the pattern is always this.
Well, and, of course, they're also complicated because they're in 2D.
So maybe I should say a little bit.
Could I maybe write -- I'm trying to do a lot today.
Trying to do kind of the formal stuff today.
So another step in the formal stuff would be to get into 2D, which I haven't done.
What would be the famous 2D problem that leads to a positive equation?
So 2D, what would I minimize, so p of u would be a double integral of dudx squared maybe times a cdudy squared.
I would really like 1/2 on that just to make life good.
Then minus a double integral of f times u. dxdy emphasize these are double integral.
That's a very, very important problem.
Many people have tried to study that.
Euler and Lagrange would produce an equation for it.
You might say OK, solve equation that finishes the problem.
But mathematicians are always worried is there a solution?
Is there a minimum?
So I'm dodging the bullet on that one.
When I say minimize over all functions u, I could create problems where worse and worse and worse functions got closer and closer to a minimum and there was no limiting minimizer.
I won't do that.
This is a problem that works fine.
What happens to it?
Well, should we try to do the same weak form?
We'd have a dpdu, and what do you think it would look like?
It would have the integral, double integral.
What would it have?
It will have a cdudx and there will be a dvdx.
Then integration by parts will take that x derivative off a v and onto this with a minus.
Did it fast, did it way fast.
Then out of this thing will come, if I look at the v term they'll be a dvdy and I take that y derivative off of that and onto this.
Oh, I can't use d anymore, I have to use partials.
D by du of cdudy.
Then the f is just sitting there.
Oh, it's all multiplied by v. dxdu I didn't put yet, equals zero for all v. Again, this is the same thing that we had in ordinary calculus where it was just p prime of uv equals zero.
This is a level of sophistication up.
It's producing differential equations, not scaler equations.
So what's the deal?
I've done the integration by parts, so I've got everything multiplying a v. So what's the strong form of this problem?
Well, if this integral has to be zero for every v, then the conclusion is that this stuff in the brackets is zero.
That's always the same.
That's the strong form.
So that's Laplace's equation or actually Poisson's equation because I have a right hand side, f of x.
Well, those are the mechanics.
Now, what else comes into the calculus of variations.
You've seen the pattern here.
There's one important further possibility that we met last time, which was constraint.
We're dealing here with a pure minimization.
I didn't impose any side conditions on u except maybe the boundary condition.
I'll just close with an example that I'm going to follow-up, and it's going to have a constraint.
So it'll look like the original problem, but well there will be two u's.
So can I try to get it right here?
My minimization, my unknown will have two components, u1 and u2.
And it'll be into 2D actually.
What I'm going to produce here is called the Stokes' problem.
I'll study it next time, so if I run out of time, as I probably will, that's part of the plan.
So it's Stokes and not Navier-Stokes.
All I want to do is to write down a problem in which there is a constraint.
So I write it as a minimization, say, dv1d -- oh, probably all these guys are in here.
dv1dx and dv1dy and dv2dx and dv2dy -- sorry about all this stuff.
Probably an f1v -- oh, I've written v, because my mind is saying that the usual notation, I should be writing u because that would fit with today's lecture.
It's a velocity and that's why many people call it v, and then they have to call the pertubation, some w. f1u1, f2u2, all that stuff.
No problem.
That would lead to Laplace's equation, just the same, Poisson's equation.
But I'm going to add a constraint.
This u1u2 is a velocity, despite the letter.
I want to make the material incompressible.
I have flow here and it's like flow of water, probably incompressible.
So incompressible means that dv1dx plus dv2dy is zero.
So that's a constraint.
How the heck do we deal with it?
So I could do this minimization but with the constraint.
So all this stuff I'm totally cool with now.
That would just be calculus of variation, that would get me the dpdu, but I have to account for the constraint.
So how do you account for a constraint?
You build it into the problem with a Lagrange multiplier.
So I multiply this thing by some Lagrange multiplier, and as I empathized last time, Lagrange multipliers always turn out to mean something physically, and here it's the pressure.
So it's natural to call the Lagrange multiplier p of xy.
I build that in, so I subtract the Lagrange multiplier times this thing that has to be zero.
That gets in the problem.
Now my function now depends on u and the pressure.
I'm not going to push this to the very limit to find the strong form.
But the strong form is the Stokes' equations that we'll study.
So we have a lot to do here to make this into practical calculations where we can compute something.
and finite elements is a powerful way to do it.
So we have to turn these continuous problems into discrete problems.
Then later we have to turn this type of problem, which will be a saddle point problem because it's got this Lagrange multiplier in there into a discrete problem.
Let me just stop by putting the words saddle point there, and just as in the lecture this afternoon, saddle points appear as soon as you have constraints and Lagrange multipliers.
Well, thanks for your patience.
That's a lot of material that will -- quickly now.
Those basic steps will be section 7.2 and we'll go up on the web quickly just as soon as we get them revised.
I'm writing notes on your projects and I hope I'll have them ready for Friday.
I'll aim for Friday because Monday is Patriot's Day and you have to run the marathon.
So I'll see you Friday.
Good.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.MIT.edu
OK.
So I'll start now with the third lecture.
And we're into partial differential equations.
So we had two on ordinary differential equations and the idea of stability came up, and the order of accuracy came up.
And those will be the key questions today -- accuracy and stability.
We do get beyond that actually.
I mean to look at the contour at the output from a difference m once you've decided yes it's stable, yes it's accurate.
You still have to look at the output and see, you know, how good is it, where is it weak, where is it on target.
But first come the accuracy and stability questions.
And beginning with this equation.
So this will be the simplest initial value problem we can think of.
I call it a one way wave equation.
Notice the two way wave equation, which we'll get too, would have second derivatives there.
And that will send waves in both directions.
This we'll see sends a wave only in one direction, so it's a nice scalar problem.
First order constant coefficient, that's the velocity.
Perfect.
The model problem for wave equations.
Because it's just first order, I just begin it with the function at time 0.
So I'm looking for the solution at time t to this equation.
OK.
So that will not be hard to find.
And I want to do it first for pure exponentials.
When u of x0, the initial function, is a pure exponential e to the ikx.
Fourier is always around.
And to see what happens suppose this is given, for example, so this will be my example, e to the ikx.
OK. Now what's special about e to the ikx?
The special thing is that with constant coefficients and no boundaries, the solution will be a multiple of e to the ikx.
In other words, we can separate variables.
Maybe I'll put it here.
I can then look for the solution u to be a function times a number really, but it will depend on the time on the frequency k and the time t times e to the ikx.
You see how that separated variables?
x is separated from t, and the frequency controls what this growth factor will be.
So this growth factors is a key quantity to compute.
And we compute it just by substituting that hope for a solution into the equation.
And getting an equation for g, because e to the ikx will cancel.
That's the key point, that everything remains a pure harmonic with that single frequency k. So if I plug it in, I get dg dt.
I'm taking the time derivative of u, times e to the ikx, right.
That's du dt.
Because t was separate from x, that's what I get when I plug in this u.
So I'm always plugging in this u.
What do I get on the right hand side?
c, the x derivative.
So that's just g. The x derivative will bring down ik, e to the ikx.
No surprise that I can cancel e to the ikx, which is never 0.
And I have a simple equation for g. Linear constant coefficients, but the coefficient depends on k, of course.
The coefficient is ick, and of course the solution is g is equal to -- we're back to our simplest model of an ordinary differential equation.
Now the coefficient a is now ikc, so it's e to the ikct, right.
That shows us what happens to the e to the ikx.
So now I'll put that here, because now we know it.
So we have all together e to the ikct and here's an x.
So let's put those together as x plus ct, because that's a guide to what's coming.
So that's the solution: separated variables, look for the growth factor, tried an exponential.
And we'll do exactly the same thing for difference method.
So this was von Neumann's brilliant idea, watch exponentials, watch every frequency, and see what happens to the multiple of e to the ikx.
Now here is very special, because all frequencies are producing this combination x plus ct.
So when we combine frequency, that's what Fourier said, take combinations of these things.
The solution will be a combination of these things.
And what do we get?
If our original function was u of x and 0, it's a combination of these, e to the ikx's.
And at later time we have the same combination by linearity of these guys.
All that's happening is x is getting changed to x plus ct.
So I'm getting the answer.
Here was the answer for one exponential.
But now I'm going to do the answer for a general u, and it's going to be easy to pick off.
It's just like picking off this x plus ct.
It will be u at x plus ct at time 0.
That's the solution for every u.
Not just for the exponentials, but for all combinations of them which give us any u of x and 0.
So u of x and 0 was a combination of e to the ikx's, and then the solution is the same combination of these guys.
Well, we can understand what that solution is.
It's what I call a one way way.
Let me understand what's happening here.
So here's the key picture now.
I want to a draw a picture in the xt plane.
So I'm not graphing u here.
This is a picture in the xt plane that will show us what this formula means.
Because this is the solution that we then want to approximate by difference method.
OK.
So what does this mean?
This means that the value, for example, u of 0, 0.
So there's the point 0, 0.
Here's what's happening, at a later time, say up here some later time, the value of u, this is the line, x plus ct equals 0; x plus ct equals 0 along that line.
And what's the point on that line, the solution is the same all the way along.
So whatever the initial value is here it travels, so that u at this point is u at this point.
So let me to call that point P, and say what I mean now.
I mean that u of P is the same as u at 0, 0.
Because x plus ct is the same here and here.
Let me take another point.
Let's say x0 another initial value, and again along the line like this.
This will be the line x plus ct equals capital X, because it's the line where x plus ct is a constant, and the constant is chosen so that it starts at this point at t equals 0, X big X.
So now -- let's see -- I better make clear -- that this lines carries this initial value.
This line carries this initial value.
You see it?
The value of u all along this line is the value it started with right there.
And you know the name for those lines?
So these are lines, they happen to be straight here because we got constant coefficients, straight and even parallel.
And they're the lines on which the information travels.
Whatever the information was at times 0.
By information I mean the value of u at 0, 0, or the value of u at this point.
That information travels along that line to give this solution all along the line.
And the name of the line, anybody know it?
Characteristic.
So this is a characteristic line, they all are.
This to.
All these line are characteristic lines.
Can you read characteristic?
I should of have written it a little better.
So this is like a key feature of wave equations.
It's a key feature of wave equations that we will not see for heat equations.
And so let me just say it again, because it will bear also on the stability question for difference equations.
The information, the true information, for the true solution travels along characteristic lines.
We're in one dimension here, and which is a major simplification.
In two dimensions we'll have characteristic cones.
In three dimension certainly we'd call it a cone.
And actually I guess what I'm saying is that if I speak a word, you know, my voice or the sound waves solve the 3D wave equation.
So when I snap my fingers, that sound travels from this point out along characteristics.
And of course it travels in all directions, so we've got a more interesting, more complicated picture in higher dimensions, but we really see it here in one dimension.
OK. Now I should also include a graph of the solution.
What does the solution look like?
Maybe I'll find a spot here for that graph.
Now I'm going to graph u.
This is still x.
So I'll graph u at maybe two different times.
So the starting time.
Suppose we have a wall of water.
So this will be u of x and 0, a step function, maybe at 0.
So u of x and 0 is 1 and then 0.
OK.
So there's a typical starting initial value.
Well, I don't know if it's typical.
Pretty special to have 1 and then 0, a perfect step function.
Now I want to ask what's the solution to the equation?
What is u of x and t?
I want to graph u of x and t. What what happens?
You have to assemble sort of this picture, which is in the xt plane, into a graph here of u of x and t. What happens?
Well all these 0 starting values travel along characteristics, and tell us that u is 0 along all these characteristics -- u is 0.
This here, the starting value is 1.
So along these characteristics we're starting with a 1, we end with a 1.
Do you see what happens?
How do I graph it?
The wave is moving.
The wall of water is moving that way, moving to the left.
Because I'm taking c to be a positive number.
So u of x and t is a wall, or a wave you could say, moving left with speed c. Every solution to the equation is doing this.
And in this particular case it's a wall of water and it stays a wall of water.
That the point.
Shapes are not changed as we go.
There's no dispersion would be the right word I think.
We would expect in a general equation that the shape of the wave would change a little as it travels.
In this equation it doesn't change.
The shape stays the same.
See again I'm thinking of this wave as a combination of pure exponentials.
And they all travel with the same speed c, so the whole wave travels with speed c. And at a later time that's the graph of the solution.
OK.
So that much is all we have to do, all we can do really, in solving the differential equations.
It's a simple model certainly.
But now I come to the difference equation.
Well when I say the difference equation, I should say equations plural.
Because here I've written four possibilities.
Three of them are actually used, and one is a disaster.
But none of the four is perfect.
Most people would say Lax-Wendroff is the best of these four, because it has one extra order of accuracy.
Oh yeah, yeah.
I guess these all have only first order accuracy.
But Lax-Wendroff moves up to second.
So that's a method that everybody knows, because it gets up this way.
But actually upwind is the first one to start with.
Let me start with that one.
OK.
So I've multiplied up by the delta t that should be in the denominator there.
I'm taking the standard forward difference in time -- all these methods are going to be explicit , so they're all going to be solvable in a simple way for u at the new time.
So a forward difference in time for du, dt and c times a forward difference in space for du, da.
So that's the first idea that would occur to you, and it's pretty reasonable.
We can check its accuracy, but you can guess what it would be.
What's your guess on the accuracy of this method?
You're seeing a forward difference in time, a forward difference in space.
The odds are those finite differences give you an accuracy p, if we use p as the measure of accuracy, which would be probably 1, first order accurate.
So we'll see the accuracy as only p equal to 1.
OK. What about the stability?
I'm looking at this first method.
So one comment about the method is that this ratio c delta t over delta x, let me call it r, that's a key number.
it's called the Courant number by some.
So in the literature you might see the phrase Courant number for that ratio r. And maybe my first point is stability is going to put a bound on r. We can tell right away for this explicit method and for any explicit method, that r can't be unlimited.
There's going to be a bound on r, which it means a bound on delta t. It's telling us that delta t has to be less than some constant times delta x.
And stability is the job to find that constant.
How small does delta t have to be?
Now let me sort of understand this forward difference method in this picture.
So I'm going to do the forward difference, delta t of u equals that ratio times the forward difference of the space difference, so time differences is that ratio times this space difference.
OK.
Method 1.
OK.
So I guess somehow you want to see want to see a picture like this for the difference equation, So the difference equation we imagine we've got steps delta x, all these steps are delta x.
Let's suppose that's 0.
OK.
So what happening now, and time going this way and I'm solving this equation?
What's happening here?
Let me copy this method, u at a position j, so this might be the point j delta x, at time n plus 1 is.
I want to write this equation as always in the most convenient way.
So what am I going to do?
I'm going to take this u of x and t and put it on the right hand.
So then my equation looks like that ratio r times u at j plus 1 at time n. That's this.
I'm one position over, but I'm at the given time.
And then what's the coefficient of ujn?
So can you just tell me what do I put in there?
So this will be that nice way to look at our difference equation.
Each new value is a combination of two old values, r times the guy here.
So in other words, here's my little molecule.
This is the new value it comes from r times this plus what, you have to tell me what multiplies this you at this point.
So here's uj n plus 1, and here's uj plus 1n, and here is ujn.
And what goes in parentheses?
1 minus o.
Because I get the 1 when that flips over to the other side.
Good.
So it's just a combination.
The new value is a combination of those two.
And I guess what I want to show is that it could not possibly be stable unless r is less or equal to 1.
So this will be the Courant condition.
The condition on the Courant number.
So the Courant condition or the CFL, that's a really a better condition.
Let me say what it is.
The CFL condition is r less or equal to 1.
That's a condition for stability or convergence that comes to us from comparing the real characteristics with the finite difference picture.
OK.
So what's CFL?
C is Courant still.
So the this came from a really early paper, and I think actually Lewy.
That's Courant, that's Friedrichs, they were close friends.
That's Lewy, all three were friends actually.
And I think it was actually Lewy who spotted what we're just going to do right now, the requirement that r had to be less or equal 1.
Now let me just make clear that this Courant-Friedrichs-Lewy condition is going to be a necessary condition.
It has to hold or there is no hope.
But it's not enough, it's not a sufficient condition.
We can't guarantee that a method stable just because the Courant condition is satisfied.
The Courant condition will tell us something about the position of characteristics, we'll see it in a second.
For example, this centered one is unstable, this guy when the right hand side you might say, oh this is a better method; always better to take a center difference for du dx than a one sided.
But not now.
The one sided one d OK.
The centered one is going to be totally unstable for all r, this centered one.
And we'll identify that not from Courant-Friedrichs-Lewy their paper wouldn't have spotted the difference between one and two, but van Neumann did; van Neumann quickly noticed that if you use an exponential, so we'll do one van Neumann in a moment, if you plug in an exponential the number 2, this center difference, it takes off.
And it's unstable even if delta t is small.
OK. Now what is this CFL, Courant-Friedrichs-Lewy, condition?
Where does that come from?
It comes from just thinking about the information.
I have to write down the thinking now that goes into the Courant-Friedrichs-Lewy condition, the CFL condition.
OK.
It's straightforward.
I have to think at a time t, let's say t equals n delta t. Suppose I've taken n times that, then the true u at, let's say, 0 and that time t is, or x and t because we know the true solution, the true u at x and t is as we know the initial function x plus ct at the point x plus ct.
So that's the correct solution.
And Courant-Friedrichs-Lewy are just saying that you better use this number in finding a difference method or you don't have a chance.
Right.
That makes sense.
So what do I mean by use this number?
OK.
So let me draw.
So u at x and t is some point here.
This is at the point x and t. This use these two values.
They use these three values.
This one use that one, that one, that one.
This one use those two.
Those four use these five values.
Right.
See the difference method is not propagating the information entirely on that line, it's really taking values over that whole interval with some combinations of r and 1 minus r. And in the end after five times step, it's giving us this value up there.
But the point is that if this point x plus ct were out here, then refining delta x an refining dealt t, and using more and more point and filling in and using more of these, but not getting out here, would get you know where.
You wouldn't have a chance.
So that's the Courant-Friedrichs-Lewy condition.
How far out is this?
This is n delta x inside this.
And if this is x, this is n delta x is away, and this is x plus ct. And c is n delta t's.
Right?
t is n delta t. Are you with me?
As I keep delta t over delta x, the ratio r fixed and use more and more points, I'm filling up this interval with points that I've used.
But I'm never going to get to a point there.
So the idea is then unstable, couldn't converge, couldn't work.
If the distance reached here n delta x is smaller than cn delta t, it couldn't work.
OK.
So I'm taking a negative point of view here.
It can't work in this case.
The reason I take that negative point of view is I'm not able to say it does work in the case when delta x is big enough compared to delta t, and I include that point.
I can't be sure.
But I can be sure that if I don't include that neighborhood of where the true u is produce, it can't have a chance.
So let me just cancel n divide by delta x.
And I say fail if 1 is smaller then c delta t over delta x which is r. It fails if r is bigger than 1 by this reasoning of characteristics.
There are two more cases that are worth thinking about.
Suppose the method was down wind.
So this is up wind, because I take a forward difference.
The wind is blowing this way.
Forgive me for incomplete use of the word wind.
I don't know what it means.
But one way or another whatever, the wind is blowing this way.
The values are coming from the up wind direction.
And that's good, because the true value comes from the up wind direction.
That's what we saw on characteristics, the values are blowing down wind.
So what would happen if I use a backward difference here?
So I didn't write that bad idea, but it's important to realizes that's method 1a.
A bad idea is to use a backward difference here.
I should maybe call it 1b for backward.
You see that it would fail?
If I used a backward difference, what will what happen to this picture?
This value would come from stuff on the left.
It would totally have no chance to use the correct initial value.
So a backward difference is an immediate failure.
It will also fail if -- well I was going to say r less than 0.
I don't know whether I can say that.
The way r could be less than 0 would be the delta x being negative, and that's what I mean by the down wind message, where I'm looking for looking for the value here, and of course I'm not finding it.
OK.
So I guess I'm saying that Courant-Friedrichs-Lewy tells us right away that this method will only be possible, And in fact this is the stability condition.
So let me write that fact, but we haven't proved it yet.
It's stable for r between 0 and 1.
That's the stability condition, which limits delta t. Because r is delta t over delta x.
And this says that delta t can't be larger than the multiple of delta x. OK. You might say that sounds like delta t is too small.
Is stability too restrictive?
Well it's restrictive certainly, because it limits delta t. But actually for these methods, accuracy also limits delta t. If I look for a method that allowed a bigger delta t, then the error of that in that method proportional to delta t would be too big.
In other words accuracy as well as stability is keeping delta t of the order of delta x.
Accuracy as well as stability is keeping r within a bound, and stability is keeping it within those bounds.
OK.
So I still have to show that the method is stable.
The Courant-Friedrichs-Lewy condition just helps to eliminate impossible ratios.
OK. One more point, which of course everybody notices in solving the difference equation.
Suppose r is exactly 1?
Think about the case where r is exactly 1.
Remember that's c delta t over delta x.
Suppose r is exactly 1, then what does the method do?
The method takes the new value, r is 1.
This is 0 now.
So the new value is then when r is 1 is that old value.
In this picture, when r is 1 this value is this one.
The values are traveling along a line.
I'm not using these you see, because if r is exactly 1, these are exactly 0.
Now is that good?
Actually it's terrific, because that line when r is 1 is the correct characteristic line.
If r is 1, if c delta t equals delta x, I'm going up exactly with that slope that the true line does.
In other words, the difference equation is gives exactly the solution to the differential equation.
Because it's so simple of course.
Couldn't do it.
Well that's the point, we can't do it for big problems, because in a big problem c is going to be variable.
Maybe the problem we'll receive may depend on the position or the time, or it might depend on the solution u in a non-linear problem.
Those are ahead of us.
But there's possibility in 1D.
It's only a really a one dimensional possibility, where the true information travels on a line, and we could just hope to stay close to that characteristic line.
OK. Now I'm ready to tackle for any method.
Let me get these guys up again.
So first I want to see why is up wind stable, and why is centered unstable.
Should I start with the stability question rather than the accuracy question?
Yeah, Christie we could guess what's first order until we get to Lax-Wendroff.
So accuracy is going to come next time.
We've had Courants take on stability, the CFO condition, but now I'm ready for van Neumann's deeper insight.
OK.
So how does von Neumann study stability?
He watches every e to the ikx.
So von Neumann's is going to be each e to the ikx should have a growth factor in the difference equation smaller then 1.
The growth factor in the differential equation of course was right on.
This has magnitude exactly 1.
So wave equations are not giving us any space to work in.
Because we want the growth factor and the difference equation, we want it to be close to the real 1.
And at the same time it better be not the real 1 having absolute value exactly 1 on the unit circle.
But the difference equation is allowed to go into the unit circle, but not to go out.
OK. That will separate the good ones from the bad.
OK. Let me work here.
Can I copy the equation uj n plus 1 is equal to r uj plus 1n, and 1 minus r ujn.
OK. Now I'm going to do an exponential again.
I'm going to start from e to the ikx.
Sorry, let me say times 0 and position n will be e to the ikn delta x.
Right.
That's my initial, e to the ikx, that's my pure frequency.
I plug that in.
Oh not n but j; j is measuring how many, sorry let's get it right; j is measuring how many delta x's I'm going across; n is measuring how many delta t's I'm going up.
So this is the key, here's von Neumann.
Try a pure exponential, plug it in.
Again it's going to work, because we have constant coefficients.
So what do I get on the right hand side?
I get r times this guy at level j plus 1, e to the ik j plus 1 delta x times the exponential.
Can I save the exponential because it's going to factor out.
Plus a 1 minus 4 times the exponential itself, and here's the exponential: e to the ikj delta x.
This will be one step.
I could say uj1 coming from level 0 to level 1.
Sorry e to the ikj delta x, so I don't need it here.
Sorry I did it wrong.
That e to the ikj plus 1 delta x factors into e to the ik delta x from the 1, and e to the ikj delta x, which is over there.
So I only wanted this much.
OK.
Sorry that's on the permanent tape but maybe it's OK. Because it makes us think through what's the result from a pure exponential?
And again the result is pure exponential in, pure exponential out, but multiply by some finite difference growth factor g, that depends on the frequency.
And it depends delta x and it depends on r. Well actually it depends on k delta x together.
So I could put that together.
What's von Neumann's question now?
He wants to know is this number, it's a complex a number of course because it's got this cosine of k dealt x plus i sign of k delta x, he wants to know is it magnitude.
Does the magnitude get bigger than 1?
if so n steps will give the nth power of the thing and it will blow up, or does the magnitude stay lesser equal to 1, in which case it won't blow up, in which case we have stability.
Of course it would be great if the magnitude was always exactly 1.
Because that's what the true solution has.
But that's only going to happen in this special, special case when r equals 1, and I'm going just right on the characteristic.
When r is exactly 1, that's the case when that's gone, this is a 1, and that has magnitude exactly 1.
But normally r is going to be, I'm going to be on the safe side, r will be less than 1.
And I just want to draw this van Neumann amplification factor.
I'll often use the word growth factor just because it's shorter, but another word is amplification factor.
What does an exponential get amplified by?
Can you identify this number in the complex plane?
Of course the big question is where is it with respect to the unit circle?
OK. Take k equals 0 -- 0 frequency the DC term, the constant start -- what is that number equal when k is 0?
1.
So that's normal that k equals 0.
We're right at the position 1.
That's just telling us that a constant initial value stays unchanged.
The true growth factor and the van Neumann amplification factor both 1.
But now let k be non-0.
Where is this complex number?
And now I'm going to impose the condition that r is between 0 and 1, because I know from Courant that is going to be required.
What's the point here?
OK. Then you see here is a 1 minus r. So there's 1, 1 minus r will be a little bit in here, and then what happens when I add on this number?
So the 1 minus r I've done.
It put me here.
It was real.
But this number is not real.
This just number is r times complex number, but what do I know?
I do something critical here.
What do I know about this number?
I know its absolute value is 1.
So that it gets multiplied by an r. Look, look, look, look.
I start with a 1 minus r. So I go to 1 back to r, and now I add in this part, which is somewhere on a circle, a radius r that's everything in this problem.
It's a circle of radius r around the point 1 minus r. I'm inside the circle, and you could say well you could have seen that clearly by just using the triangle inequality.
The magnitude of this can't be bigger than the magnitude of that, which is what?
r. Plus the magnitude of that, which is what?
1 minus r. So the magnitude by the triangle inequality can't be more than r plus 1 minus r which is 1.
But now wait a minute, where did I use the Courant condition?
The way I said that there sounded like it would always work.
This has magnitude r. What's going on here?
Suppose r is bigger than 1, what's going wrong?
In fact, what's the picture if r is bigger than 1?
If r is bigger than 1, then my 1 minus r is out here somewhere.
So this is the unstable case, 1 minus r, and then a circle of radio r, bad new right?
Every frequency is unstable in fact.
When I'm reaching too far, and that's just telling me I'm not using the information I need.
I don't have a chance of keeping controlling any frequency.
OK.
I've got just enough time to show how bad this second method it is.
So nobody wanted his name associated with that second method.
Why is that?
So what's the van Neumann quantity for the second method?
OK. Just space here to write.
I want to know the number for that second method.
So now the method is ujn plus 1 is ujn from the time difference plus r over 2, uj plus 1n minus uj minus 1n.
This is going to be the bad method.
It looks good because the center difference is more accurate than a one sided difference, but it's also unstable here.
Can you look at that and see what the van Neumann number g is going to be?
Think of an exponential going in.
What exponential comes out?
So g is a 1 from this.
Now this is our guys that's shifted over, so that would be in r over 2 e to the ik delta x just as it was before.
And this one will be a minus r over 2 times e to the -- so it's back one step, so there will be and e to the minus ik delta x.
So what's with this now?
g is 1 plus this quantity, r over 2 times e to the ikx minus d the minus ikx.
What am I seeing there?
I'm seeing 1 plus r, and everybody recognizes this minus this over 2 as i sin, ir sin k delta x.
That's the amplification factor, and is it smaller than 1?
No way.
No way.
Let me raise it up here.
So I don't care whether r is less than 1 or not, I'm lost.
This is a pure imaginary number.
This is a real number.
So I have the sum of squares to get the magnitude, it will be the square root of 1 plus r sin k delta x squared.
So if I draw the bad picture then, what's the bad picture is, now it goes to 1 and then it goes way up the imaginary axis.
Well maybe not way up but up.
Sorry I shouldn't have made it quite as bad as it was.
If I reduce r, I don't go so far up, but no hope.
So do you see why that one is bad, because the amplification factor van Neumann tells us what Courant did not tell us.
that the amplification factor here has magnitude bigger than 1.
It's outside the circle, and every exponential is growing and there's no hope.
OK.
So the second last lecture on this section 52 of the notes will be about Lax-Friedrichs and Lax-Wendroff order of accuracy, stability and actual behavior in practice.
Ok. See you Wednesday.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So I'm thinking of large sparse matrices.
So the point about a -- the kind of matrices that we've been writing down, maybe two dimensional, maybe three dimensional difference matrices.
So it might be five or seven non 0s on a typical row and that means that you don't want to invert such a matrix.
These are big matrices.
The order could easily be a million.
What you can do fast is multiply a by a vector, because multiplying a sparse matrix by a vector, you only have maybe five non 0s times the n -- the vector of length n, so say five n operations -- so that's fast.
I'm thinking that we are at a size where elimination, even with a good ordering, is too expensive in time or in storage.
So I'm going from direct elimination methods to iterative methods.
Iterative means that I never actually get to the perfect answer x, but I get close and what I want to do is get close quickly.
So a lot of thought has gone into creating iterative methods.
I'll write down one key word here.
Part of the decision is to choose a good preconditioner.
I'll call that matrix p. So it's a matrix that's supposed to be -- that we have some freedom to choose and it very much speeds up, so the decisions are, what preconditioner to choose?
A preconditioner that's somehow like the matrix a, but hopefully a lot simpler.
There are a bunch of iterative methods and that's what today's lecture is about.
Multigrid is an idea that just keeps developing.
It really is producing answers to very large problems very fast, so that will be the following lectures and then come these methods -- you may not be familiar with that word Krylov.
Let me mention that in this family, the best known method it is called conjugate gradience.
The conjugate gradient, so I'll just write that down.
Conjugate gradient, so that will come next week.
That's a method that's a giant success for symmetric positive definite problem.
So I don't assume in general that a is symmetric or positive definite.
Often it is if it comes from a finite difference or finite element approximation.
If it is, then conjugate gradience is highly recommended.
I'll start with the general picture.
Let me put the general picture of what an iteration looks like.
So an iteration computes the new x.
Let me call it x-new or xk plus 1 from the known x.
So we start with x 0, any x 0.
In principle, it's not too important for 0 to be close in linear methods.
You could start even at 0.
In nonlinear methods, Newton methods, you do want to be close and a lot of attention goes into a good start.
Here the start isn't too important; it's the steps that you taking.
Let me -- so I guess I'm going to -- I'm trying to solve ax equal b. I'm going to split that equation into -- I'm just going to write the same equation this way.
I could write it as x equals i minus ax plus b.
That would be the same equation, right?
Because x and x cancel.
ax comes over here and I have ax equal b. I've split up the equation and now actually, let me bring -- so that's without preconditioner.
You don't see a matrix p. That's just an ordinary iteration, no preconditioner.
The preconditioner will come.
I'll have p and it'll go here too.
Then of course, I have to change that to a p. So that's still the same equation, right?
px is on both sides, cancels -- and I have ax equal b.
So that's the same equation, but it suggests the iteration that I want to analyze.
On the right hand side is the known approximation.
On the left side is the next approximation, xk plus 1.
I multiply by p, so I -- the idea is p should be close to a.
Of course, if it was exactly a -- if p equalled a, then this wouldn't be here and I would just be solving in one step ax equal b.
So p equal a is one extreme.
I mean, it's totally, perfectly preconditioned, but the point is that if p equals a, we don't want to solve that system.
We're trying to escape from solving that system, but we're willing to solve a related system that is simpler.
Why simpler?
Here are some possible preconditioners.
p equals the diagonal.
Just take the diagonal of a and that choice is named after Jacobi.
That's Jacobi's method.
Actually, it's already a good idea, because the effect of that is somehow to properly scaled the equations.
When I had the identity here, I had no idea whether the identity and a are in the right scaling.
a may be divided by delta x squared or something.
It could be totally different units, but p -- if I take the diagonal of a, at least they're comparable.
Then you see what -- we'll study Jacobi's method.
So that's a very simple choice.
Takes no more work than the identity, really.
Second choice would be -- so that would be p for Jacobi.
A second choice that you would think of pretty fast is the diagonal and the lower triangular part.
So it's the -- I'll say the triangular -- I'm going to say lower triangular, triangular part of a, including the diagonal.
This is associated with Gauss's great name and Seidel.
So that's the Gauss-Seidel choice.
So why triangular?
We'll analyze the effect of this choice.
If p is triangular, then we can solve -- I'm never going to write out an inverse matrix.
That's understood, but I can compute the new x from the old x 1 just by back substitution.
I find the first component of x, k plus 1, then the second, then the third.
It's just because the first equation -- if p is triangular, the first equation will only have one term, one new term on the left side and everything else will be over there.
The second equation will have a diagonal and something that uses the number I just found for the first component and onward.
So in other words, triangular matrices are good.
Then people thought about combinations of these.
A big lot of activity went in something called overrelaxation, where you did a -- you had a weighting of these and it turned out to be if you adjusted the weight, you could speed up the method, sometimes called SOR-- successive overrelaxation.
So when I was in graduate school, that was a very -- that was sort of the beginning of progress beyond Jacobi and Gauss-Seidel.
Then after that came a very natural -- ilu is the next one that I want -- the i stands for incomplete, incomplete lu.
So what are l and u?
That's the letters I always use as a signal for the lower triangular and upper triangular factors of a, the ones that exact elimination would produce.
But we don't want to do exact elimination.
Why?
Because non 0s fill in.
Non 0s appear in those spaces where a itself is 0.
So the factors are much -- have many more non 0s than the original a.
They're not as sparse.
So the idea of incomplete lu is -- and I'll come back to it -- the idea of incomplete lu -- you might say, should have occurred to people earlier -- only keep the non 0s that are non 0s in the original a.
Don't allow fill in or allow a certain amount of fill in.
You could have a tolerance and you would include a non 0 if it was big enough, but the many, many small non 0s that just fill in and make the elimination long, you throw them out.
So p is -- in this case, p is l, is an approximate l times an approximate u.
So it's close to a, but because we've refused some of the fill in, it's not exactly a.
You have a tolerance there where if you set the tolerance high, then these are exact, but you've got all that fill in.
If you set the tolerance to 0, you've got the other [?
instance.
?]
So that would give you an idea of preconditions, of reasonable choices.
Now I'm going to think about -- first, how do -- how to decide whether the xs approach the correct answer?
I need an equation for the error and because my equations are linear, I'm just going to subtract the upper equation from the lower one and I'll call the -- so the error ek will be x minus the true answer.
This is x exact.
Minus x, my case approximation.
So when I subtract that from that, I have ek plus 1 and here I have p minus a. I subtract that from that.
I have ek and when I subtract that from that, 0.
So very simple error equation.
so this is an equation, this would be the error equation.
You see the b has disappeared, because I'm only looking at differences now.
Let me write that over on this board, even slightly differently.
Now I will multiply through by p inverse.
That really shows it just so clearly.
So the new error is, if I multiply by p inverse, p inverse times p is i and I have p inverse aek.
So what does that say?
That says that at every iteration, I multiply the error by that matrix.
Of course, if p equals a, if I pick the perfect preconditioner, then p inverse a is the identity.
This is the 0 matrix and the error in the first step is 0.
I'm solving exactly.
I don't plan to do that.
I plan to have a p that's close to a, so in some way, this should be close to i.
This whole matrix should be close to 0, close in some sense.
Let me give a name for that matrix.
Let 's call that matrix m. So that's the iteration matrix, which we're never going to display.
I mean, it would be madness to display.
We don't want to compute p inverse explicitly or display it, but to understand it -- to understand convergence or not covergence, it's m that controls everything.
So this is what I would call pure iteration or maybe another word that's used instead of pure is stationary iteration.
What does stationary mean?
It means that you do the same thing all the time.
At every step, you just use the same equation, where these methods will be smarter.
They'll adapt.
They'll use more information.
They'll converge faster, but this is the simple one -- and this one plays a part in those.
So what's the story on convergence now?
What about that matrix m?
If I take a starting vector, e 0 -- my initial error could be anything -- I multiply by m to get e 1.
Then I multiply m by m again to get e 2.
Everu step, I multiply by m. So after k steps, I've multiplied k times by m. I want to know -- so the key question is, does that go to 0?
So the question, does it go to 0 and if it does, how fast?
The two parts, does it go to 0 and how fast are pretty much controlled by the same property of m. So what's the answer?
When does -- if I take -- I don't know anything about e 0, so what property of m is going to decide whether its powers get small, go to 0, so that the error goes to 0?
That's convergence.
Maybe wall up or maybe stay away from 0.
What controls it?
One word is the answer.
The eigenvalues.
It's the eigenvalues of m, and in particular, it's the biggest.
If I have an eigenvalue of m, as far as I know, e-not could be the eigenvector and then every step would multiply by that eigenvalue lambda.
So the condition is all eigenvalues of m have to be smaller than 1 and in magnitude of course, because they could be negative, they could be complex.
So that's the total condition, that's exactly the requirement on m. How do we say how fast they go to 0?
How fast -- if all the eigenvalues are below 1, then the slowest convergence is going to come for the eigenvalue that's the closest to 1.
So really it will be -- and this is called the spectral radius and it's usually denoted by a Greek row of m, which is the maximum of these guys.
It's the max of the eigenvalues and that has to be below 1.
So it's this number, that number, because it's that number which really says what I'm multiplying by it every step.
Most likely, e-not, my initial unknown error has some component in the direction of this biggest eigenvalue, in the direction of its eigenvector.
Then that component will multiply at every step by lambda, but that one will be the biggest guy, rho.
So it's the maximum eigenvalue.
That's really what it comes to.
What is the maximum eigenvalue?
Let me take Jacobi.
Can we figure out the maximum eigenvalue for Jacobi?
Of course, if I don't know anything about the matrix, I certainly don't want to compute eigenvalues.
I could run Jacobi's method.
So again, Jacobi's method, I'm taking p to be the diagonal part, but let me make that explicit and let me take the matrix that we know the best.
I'll call it k. So k or a -- this is the a -- is my friend with 2s on the diagonal minus 1s above.
I apologize -- I don't really apologize, but I'll pretend to apologize for bringing this matrix in so often.
Do we remember its eigenvalues?
We computed them last semester, but that's another world.
I'll say what they are.
The eigenvalues of a -- this is an m now -- the eigenvalues of this matrix a -- I see 2s on the diagonal.
So that's a 2.
That accounts for the diagonal part, 2 times the identity.
I have a minus and then the part that comes from these 1s, which are forward and back.
What would von Neumann say?
What would von Neumann do with that matrix?
Of course, he would tested on exponentials and he would get 2 minus e to the ik minus e to the minus ik.
He would put the two exponentials together into cosines and that's the answer.
It's 2 minus 2 cosines of whatever angel's theta.
Those are the eigenvalues.
The j'th eigenvalue is -- the cosine is at some angle theta j.
It's worth since -- maybe just to take again a minute on this matrix.
So the eigenvalues are between 0 and 4.
The eigenvalues never go negative, right?
Because the cosine can't be bigger than 1.
Actually, for this matrix the cosine doesn't reach 1.
So the smallest eigenvalue is 2 minus 2 times that cosine close to 1.
It's too close for comfort, but it is below 1.
The eigenvalues are positive and they're between 0 and 4.
At the other extreme, theta is near pi.
The cosine is near minus 1 and we have something near 4.
So the eigenvalues of that -- if you look at that matrix, you should see.
Special matrix eigenvalues in this interval 0 to 4.
The key point is, how close to 0?
Now what about m -- what about p first?
What's p?
For Jacobi, so p for Jacobi is the diagonal part, which is exactly 2 i.
That's a very simple diagonal part, very simple p. It produces the matrix m. You remember m is the identity matrix minus p inverse a.
That's beautiful.
So p is just 2 i.
So I'm just dividing a by 2, which puts a 1 there and then subtracting from the identity.
So it has 0s on the diagonal.
That's what we expect.
It will be 0s on the diagonal and off the diagonal, what do I have?
Minus -- with that minus, p is 2 so it's 1/2.
It's 1/2 off the diagonal and otherwise, all 0s.
That's the nice matrix that we get for m in one dimension.
I have to say, of course, as you know, we wouldn't be doing any of this for this one dimensional matrix a.
It's tridiagonal.
We can't ask for more than that.
Direct elimination would be top speed, but the 2D case is what we understand that it has a big bandwidth because it's two dimensional, three dimensional even bigger -- and the eigenvalues will come directly from the eigenvalues of this 1D case.
So if we understand the 1D case, we'll knock out the two and three dimensional easily.
So I'll stay with this one dimensional case, where the matrix m -- and I'll even maybe shouldn't write down exactly what the iteration does -- but if I'm looking now at convergence, do I have or don't I have convergence and how fast?
What are the eigenvalues and what's the biggest one?
The eigenvalues of a are this and p is just -- p inverse is just 1/2 times the identity.
So what are the eigenvalues?
Again, I mean, I know the eigenvalues of a, so I divide by 2.
Let me write down what I'm going to get now for the eigenvalues of m, remembering this connection.
So the eigenvalues of m are 1 minus 1/2 times the eigenvalues of a.
Better write that down.
1 minus 1/2 times the eigenvalues of a.
Of course, it's very special that we have this terrific example matrix where we know the eigenvalues and really can see what's happening and we know the matrix.
So what happens?
I take half of that -- that makes it a 1.
When I subtract it from that one, it's gone.
1/2 of that, that 2 is cancelled -- I just get cos theta.
Theta sub whatever that angle was.
It's a cosine and it's less than 1.
So convergence is going to happen and convergence is going to be slow or fast according as this is very near 1 or not -- and unfortunately, it is pretty near 1, because the first, the largest one -- I could tell you what the angles are.
That's -- to complete this information, I should have put in what -- and von Neumann got them right.
That's j -- is there maybe a pi over n plus 1?
I think so.
Those are the angles that come in, just the equally spaced angles in the finite case.
So we have n eigenvalue.
The biggest one is going to be when j is 1.
So the biggest one, rho, the maximum of these guys, of these lambdas will be the cosine of when j is 1 pi over n plus 1.
That's the one that's slowing us down.
Notice the eigenvector that's slowing us down, because it's the eigenvector that goes with that eigenvalue that's the biggest eigenvalue that's going to hang around the longest, decay the slowest.
That eigenvector is very smooth.
It's the low frequency.
It's frequency 1.
So this is j equal to 1.
This is the low frequency, j equal 1, low frequency.
The eigenvector is just samples of the sine, of a discrete sine.
One lowest frequency that the grid can sustain.
Why do I emphasize which eigenvector it is?
Because when you know which eigenvector is slowing you down -- and here it's the eigenvector you would think would be the simplest one to handle -- this is the easiest eigenvector, higher frequency eigenvector, have eigenvalues that start dropping.
Cosine of 2 pi.
Cosine 3 pi over n plus 1 is going to be further away from 1.
Now I have to admit that the very highest frequency, when j is capital n, so when j is capital n, I have to admit it happens to come back with a minus sign, but the magnitude is still just as bad.
The cosine of n pi over n plus 1 is just the negative of that one.
So actually we have two eigenvalues that are both hitting the spectral radius and both of them are the worst, the slowest converging eigenvectors.
This eigenvector looks very smooth.
The eigenvector for that high frequency one is the fastest oscillating.
If I graph the eigenvectors, eigenvalues, I will.
I'll graph the eigenvalues.
You'll see that it's the first and the last that are nearest in magnitude to 1.
I was just going to say, can I anticipate multigrid for a minute?
So multigrid does -- is going to try to get these guys to go faster.
Now this one can be handled, you'll see, by just a simple adjustment of Jacobi.
I just wait Jacobi a little bit.
Instead of 2 i, I take 3 i or 4 i.
3 i would be very good for the preconditioner.
That will have the effect on this high frequency one that'll be well down now, below 1.
This will still be the slowpoke in converging.
That's where multigrid says, that's low frequency.
That's too low.
That's not converging quickly.
Take a different grid on which that same thing looks higher frequency.
We'll see that tomorrow.
So if I stay with Jacobi, what have I learned?
I've learned that this is rho.
For this matrix, the spectral radius is that number and let's just see how big it is.
How big is -- that's a cosine of a small number.
So the cosine of a small number, theta near -- theta near 0, the cosine of theta is 1 minus -- so it's below 1, of course -- 1/2 theta squared.
Theta is pi over n plus 1 squared.
All these simple estimates really tell you what happens when you do Jacobi iteration.
You'll see it on the output from -- I mean, you can -- I hope will -- code up Jacobi's method, taking p to be a multiple of the identity, so very fast and you'll see the convergence rate you can graph.
Here's an interesting point and I'll make it again.
What should you graph in seeing -- to display convergence and rate of convergence?
Let me draw a possible graph here.
Let me draw two possible graphs.
I could graph -- this is k as I take more steps.
I could graph the error, the size of the error.
It's a vector, so I take its length.
I don't know where I start -- at e-not.
This is k equals 0.
This is the initial guess.
If I use -- I'm going to draw a graph, which you'll draw much better by actually getting MATLAB to do it.
Before I draw it, let me mention the other graph we could draw.
This tells us the error in -- this is the error in the solution.
If I think that's the natural thing to draw, it is, but we could also draw the error in the equation, the residual error.
So I'll just put up here what I mean by that.
If I put it in -- so ax equals b exactly.
axk is close.
The residual is -- let me call it r, as everybody does -- is the difference ax minus axk.
It's how close we are to b, so it's b minus axk.
The true ax gets b exactly right.
The wrong ax gets b nearly right.
So you see, because we have this linear problem, it's a times the error.
x minus xk is ek.
So this is the residual and I could grapj that.
That's how close my answer comes to getting b right, how close the equation is.
Where this is how close the xk is.
So that would be the other thing I could graph.
rk is the size of the residual, which is aek.
So now I have to remember what I think these graphs look like, but of course, it's a little absurd for me to draw them when -- I think what we see is that the residual drops pretty fast.
The residual drops pretty fast and keeps going.
Let's say Jacobi's method or in a minute or maybe the next time, another iterative method -- but the error in the solution starts down fast and what's happening here is the high frequency components are getting killed, but those low frequency components are not disappearing, because we figured out that they give the spectral radius the largest eigenvalue.
So somehow it slopes off and it's -- the price of such a simple method is that you don't get great convergence.
Of course, if I put this on a log-log plot, the slope would be exactly connected to the spectral radius.
That's the rate of decay.
It's the -- ek is approximately -- the size of ek is approximately that worst eigenvalue to the kth power.
Do you see how this can happen?
You see, this can be quite small, the residual quite small.
I think it's very important to see the two different quantities.
The residual can be quite small, but then from the residual, if I wanted to get back to the error, I'd have to multiply by a inverse and a inverse is not small.
Our matrix a is pretty -- has eigenvalues close to 0.
So a inverse is pretty big.
a inverse times aek -- this is small but a inverse picks up those smallest eigenvalues of a -- picks up those low frequencies and it's bigger, slower to go to 0 as we see in this picture.
So it's this picture that is our problem.
That shows where Jacobi is not good enough.
So what more to say about Jacobi?
For this matrix k, we know its eigenvalues.
The eigenvalues of m, we know exactly.
The matrix m we know exactly.
You see in that matrix -- now if I said, look at that matrix and tell me something about its eigenvalues, what would you say?
You would say, it's symmetric, so the eigenvalues are real and you would say that every eigenvalue is smaller than 1.
Why?
Because every eigenvalue is -- if you like to remember this business of silly circles, every eigenvalue is in a circle centered at this number - in other words, centered at 0 and its radius is the sum of those, which is 1.
So we know that every eigenvalue is in the unit circle and actually, it comes inside the unit circle, but not very much -- only by 1 over n squared.
What do I learn from this 1 over n squared?
How many iterations do I have to take to reduce the error by, let's say, 10 or e or whatever?
If the eigenvalues are 1 minus a constant over n squared, I'll have to take the n squared power.
So if I have a matrix of order 100, the number of iterations I'm going to need is 100 squared, 10,000 steps.
Every step is fast, especially with the Jacobi choice.
Here's my iteration -- and p is just a multiple of the identity here.
So every step is certainly fast.
Every step involves a matrix multiplication and that's why I began the lecture by saying, matrix multiplications are fast.
a times xk -- that's the work and it's not much.
It's maybe five non 0s and a -- so five n. That's fast, but if I have to do it 10,000 times just to reduce the error by some constant factor that's not good.
So that's why people think Jacobi gives us the idea.
It does knock off the high frequencies quite quickly, but it leaves those low frequencies.
I was going to mention weighed Jacobi.
Weighted Jacobi is to take instead of p -- and put in a weight.
So weighted Jacobi -- let me put this here and we'll come back to it.
So weighted Jacobi simply takes p to be the diagonal over a factor omega.
For example, omega equals -- 2/3 is a good choice.
Let me draw the -- you can imagine that if I -- this is 2 times the identity, so it's just 2 over omega i -- I'm just choosing a different constant.
I'm just choosing a different constant in p and has a simple affect on n. I'll draw the pictures.
The eigenvalues of m are these cosines.
So I'm going to draw those cosines, the eigenvalues of m. So they start -- I'll just draw the picture of cos theta if I can.
What does a graph of cos theta look like?
From 0 to pi, so I'm going to see -- this is theta equals 0.
This is theta equal pi.
If I just draw cos theta -- please tell me if this isn't it.
Is it something like that?
Now where are these theta 1. the bad one.
Theta 2, theta 3, theta 4, theta five, equally bad down here.
So there's cos, there's the biggest one.
That's rho and you see it's pretty near 1.
Here I'm going to hit -- at this frequency, I'm going to hit minus rho.
It'll be, again, near 1.
So the spectral radius is that, that's rho.
The cosine of that smallest angle, theta over n plus 1.
What happens when I bring in these weights?
It turns out that now the eigenvalues of m with the weights will be 1 -- it turns out 1 minus omega plus omega cos theta.
You'll see it in the notes, no problem.
So it's simply influenced by omega.
If omega is 2/3 -- if omega is 2/3, then I have 1 minus omega as 1/3 plus 2/3 cos theta.
Instead of graphing cos theta, which I've done -- let me draw -- let me complete this graph for cos theta.
The eigenvalues unweighted with weight omega equal 1.
Now it's this thing I draw.
So what happens?
What's going on here?
For a small theta, cosine is near 1.
I'm again near 1.
In fact, I'm probably even little worse, probably up here.
I think it goes down like that.
So it starts at near 1, but it ends when theta is pi.
This is 1/3 minus 2/3.
It ends at minus 1/3.
That's a lot smarter.
This one ended at minus 1, that one ended at near minus 1 but this one will end with omega -- this is the omega equal 2/3 of weighted 1.
All you've done is fix the high frequency.
When I say, that was a good thing to do, the high frequencies are no longer a problem.
The low frequencies still are.
So the only way we'll get out of that problem is moving to multigrid.
Can I, in the remaining minute, report on Gauss-Seidel and then I'll come back to it?
But quick report on Gauss-Seidel.
So what's the difference in Gauss-Seidel?
What does Gauss-Seidel do?
It uses the latest information.
As soon as you compute an x, you use it.
Jacobi compueted all -- had to key all these xs until it found all the new xs, but the Gauss-Seidel idea is, as soon as you have a better x, put it in the system.
So the storage is cut in half, because you're not saving a big old vector while you compute the new one.
I'll write down Gauss-Seidel again.
So it's more up to date and the effect is that the Jacobi eigenvalues get squared.
So you're squaring numbers that are less than 1.
So Gauss-Seidel, this gives the Jacobi eigenvalues squared.
The result is that if I draw the same curve for Gauss-Seidel, I'll be below, right?
Essentially a Gauss-Seidel step is worth two Jacobi steps.
Eigenvalues get squared as they would do in two Jacobi steps.
If I do Jacobi twice, I square its eigenvalues.
So it seems like absolutely a smart move.
It's not brilliant, because this becomes cosine squared.
I lose the 1/2, but I'm still very, very close to 1.
It still takes order n squared iterations to get anywhere.
so Jacobi is, you could say, replaced by Gauss-Seidel as being one step better, but it's not good enough and that's why the subject kept going.
This gives rho as 1 minus order of 1 over n first power, where these were 1 minus 1 over n squared.
So this is definitely further from 1 and then this can be way further from 1.
So you can see some of the numerical experiments and analysis that's coming.
I'll see you Wednesday to finish this and move into multigrid, which uses these guys.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.MIT.edu
OK. Ready for today's discussion, which I'm quite sort of happy about.
I hadn't really seen this coming.
First we haven't said anything about how do you estimate the error in a problem.
Because we're often taking continuous problem coming from differential equations typically and discretizing it, and solving it -- solving the discrete problem.
And we want some idea.
It's not just some math question of course, because, you know, in engineering design or analysis an idea of what the error is critical.
And secondly, it gives us an idea of where we can improve it.
You know, if we see what's controlling the error then we know what's going on.
So I'm speaking now about steady state problems.
We discussed error for initial value problems, and we realize there that the error depended on usually finite differences.
So the error we knew we could figure out locally from find a difference that we chose, you know, we chose second order accurate differences frequently, but we could've moved up to a fourth.
And then the other ingredient you need is stability to know that those local errors don't explode as time goes for.
OK.
So we had something to say about error and accuracy at that time about that topic.
And now I'm realizing that many of our other topic fall in this category.
And I put three of the topics here.
In the idea sort of where the underlying discretization is a projection.
It's somehow a projection between a true solution and the discrete computed solution.
Somehow the discrete computed solution is in some finite dimensional family where we can compute, and we're projecting into that family.
And here are three examples.
So you'll see that actually what I'm talking about is central to the whole semester.
Well finite elements actually appeared more in 18085.
So I'll have to recap a little bit there.
But they would be the natural tool for solving that list of continuous problems from calculus variations that I've talked about last time, minimizing some energy, where the energy is a typically an integral and were in the continuous case.
And what's the infinite element central idea?
It is choose these guys, choose some basis function, and look at their combinations.
Maybe choose n basis functions.
I guess I better get a chalk or it's going to be a short lecture.
OK.
So we have a continuous problem, continuous ODE, ordinary differential equation or a partial differential equation with boundary conditions as always that we want to make discrete.
And the point is I am not making a discrete by finite differences.
This is a different route from finite differences.
And in Laplace's equation and these, and many, many problems this is the preferred route.
And finite elements are a particular choice of these guys.
The Galerkin idea, so I'll use his name, for the overall idea is choose these, and look for the best combination, and we have to say what does best mean.
Well best is going to mean in our minimization problems, it will be the combination that gives the minimum.
The combination of these n file functions.
Now the exact solution is not going to be in that little n dimensional space.
Those functions for finite elements might be piecewise linear, or if you want to upgrade it they might be piecewise parabolas, piecewise cubics, whatever.
It was the brilliant finite element idea to choose simple functions.
And then you could take n quite large, but you're still not getting the exact solution of course, and it's to estimate how far off you are.
But then also in multigrid, what was the idea in multigrid?
We started with a system at level h. We started with some problem Ah uh equal fh, which was a big system.
If we did ordinary Jacobi or Gauss-Seidel that was too slow, a multigrid idea was project, there's that magic word project, onto to a course grid, where the problem is smaller.
Here we started with a continuous problem, and we got it down to size n. Here we start with a discrete problem and we cut its size in 1/2 or in 1/4 or an 1/8 again.
So projection is producing a smaller problem, and the question is what are you losing?
And actually conjugate gradients.
You remember these spaces?
The spaces span the combinations of b ab a squared b and so on.
That was the computationally convenient of sub space to project onto.
And so we've got important examples.
Now what I realized, I'm pleased about this, is that they all fit.
So here's my problem, what's the error?
So I'm going to u star for the correct answer.
This is the true.
And I'm going to use U star, capital U star for the approximate, the one that we get by any of those key ideas in numerical analysis.
And I'm trying to estimate the difference.
OK. And I just put up here that I'm dealing with positive definite problems.
In fact, the matrix K the symmetric positive definite problem you have often has this A transposed CA form, or in the continuous case it might K might stand for the A transpose might be minus the derivative, the C might be a variable or constant of the physical coefficient; A would be d by the x, that would be the K thing.
The equation I want to get it Ku equal f. That's the strong form.
Strong form will be Ku equal f. But the whole point of last lecture and this one is that we don't get to the strong form.
This projection starts with a minimum problem, and gets to a weak form.
And it's the minimum problem or the weak form that we want to think about not the strong form.
OK. Now.
Up there I wrote an identity.
If you just multiply that right hand side out, I believe it comes out right.
And it's very valuable for our purposes here.
OK.
So what's on the left side?
This is the quantity that we're minimize, we're minimizing this.
And by just manipulation, we wrote it this way.
So now I can identify what u star.
So I'm going to minimize this over all u.
That certainly looks like a discrete problem right?
So think of u as a vector, f as a vector, K as a positive definite symmetric matrix.
Just think about that.
But I want it to apply very much to the continuous case too, to differential equations.
But here's a point then, what's the winner?
You can't immediately see, well maybe you can, somehow see that when I set the derivative of this thing to 0, I get that equation.
You can kind of believe it, even if it's vectors and matrices.
But here you can see right away that how do I make this small?
This is a constant here.
That's just a constant, so it doesn't depend on u.
So what choice of u makes that term small?
Well of course it's the choice of u is the one that makes this thing 0, because that's a positive definite matrix.
No way is anything is going to get negative here.
This is a something transposed K something.
You remember what positive definite means.
It means that x transpose Kx for every vector x, there's never a negative.
So the best we could do would be to bring this to 0, and of course to bring it to 0 is to bring the x to 0, to bring this thing to 0, so this thing should be 0.
So u minus K in verse f should be 0.
And of course that leads us back to the same conclusion that we reached from this the strong form.
OK. Good.
So that an identity, but now I want to use this identity to answer the question about what if I minimize only on a sub space?
That's the problem for today.
So that's a question.
Maybe I'll write it on this fresh board.
Now I minimize only over some sub space.
Now can I can I use capital U?
I used little u for in here, allowing it to be any vector, and I found a winner.
And let me give the winner a name, u star.
It's a real headache in this subject of just the notation.
What should we call the winner?
So sometimes I call it u hap, that was a familiar notation in estimation theory of these squares.
Today I'm going to call it u star So it's u star is the winner, small u star.
Let me take time out, one minute time out.
I mean optimize is about the problem of minimizing some function f of x. I'm just going to take one minute on the problems of an author.
What do you call the winning function, winning vector, or the winning x?
I mean it could be just a scalar, we could be doing just calculus.
What do I call the x that gives the minimum?
You may have a favorite, you can't call it x right?
I mean Because that's just confusing it with the variable x.
So I'm saying well you could call it x hat, you could call it x star, you call it capital X.
So I'll just write a few of those down - x star would be a possibility, x hat would be a possibility, x minimizing would be a possibility.
I'm doing this just because I want to write down the thing that you often see, which is argmin of F of x.
And I write that down just so if you ever see it, you know what the heck it means.
It has the same meaning as any of these.
I am not a big fan of that.
But what does this mean?
It means the argument that gives the minimum of F of x, right.
Argument is a fancy word for the variable in the function.
So this the argument that gives the minimum of F of x, and that's what we're looking for to name.
But I'm sure not happy about writing that name.
So here it goes, it disappears.
But you'll see it, and now you now want it means.
OK.
So this is now the central issue here.
I want to minimize my same guy 1/2 u transpose Ku minus u transpose f, or which is exactly the same, that same right hand side because the two are equal.
I want to minimize over some capital U's not all U's.
if I minimize over all U's, then I get the exact answer.
But the idea of all these numerical methods is minimize over some finite dimensional, smaller dimensional, sub space of trial functions.
I don't know.
I'll say minimize over U in the trial space, just to write it out in words.
OK. Well now I guess my name for the winner in the trial space is going to be U star.
So the winner in the trial space will be U star.
So that's my finite element solution, my conjugate gradient solution, my multigrid solution.
All these problems are reducing to a smaller trial space, and picking the winner there, computing the winner there.
And then the question of today is how far apart are the two?
OK. And now there's this little formula that gives us a good idea.
This little formula gives us a handle on that.
So now can I look at this formula?
Maybe maybe I'll copy the formula here.
This is the minimum over all these trial, these guys, of 1/2 -- oh but I'm going to write it that way -- 1/2U, that's my trial guy, minus KN verse f, that's my U star now.
Am I OK to give that name to KN verse f?
Yeah, I already gave it I guess.
U star is Kn verse f. So that's cool.
Shorter.
IT's shorter and better.
K, oh it's transposed U minus u star plus a constant.
So I can forget that.
I can forget this constant part.
That's not important to us.
So this simple identity then has expressed our key discretization approach here as finding the U, and we're going to call it U star, that's nearest to u star.
That's the great fact that makes the whole subject pleasant.
OK.
So the winner U star is then, by this since I'm minimizing that expression, it's certainly the nearest in the K norm.
It's the nearest waited by K, somehow that K is important.
That K is reflecting the problem we're solving.
That K is the energy or whatever to U star.
OK. That's the great conclusion.
You could say that's the fundamental theorem of this projection, error estimates and stuff.
So do you see that we got to that?
The thing that minimizes makes this as small as we can.
So nearest is simply a translation of what that says.
Pick the capital U that's closest to u star.
OK. Now how does that help?
That helps because I want to estimate the difference.
So now I estimate the difference of U star minus u star.
So I'm trying to get a handle on how different those are.
How So let me take in this case of, say, Laplace's equation or the 1D case.
again I'm speaking steady state boundary value problem.
So U star is the best combination of these trial function, capital U, and little u star is the exact solution.
Let me draw a picture.
So suppose I'm in 1D, 0 to one.
OK. Yeah, let me pick that model problem again.
So minus the derivative of cdu dx is f of x with boundary conditions.
And I guess if I'm consistent now I should write u star, because it's the winning solution.
And suppose the boundary conditions are 0 at both ends, and it does something.
OK.
So that's u star.
OK.
So that's a continuous case, which I'm drawing a picture to represent what the solution might look like.
OK. Now what about this finite element stuff?
Suppose the phi's are linear pieces.
I'm going to do linear finite element method.
Then any combination of these linear guys is going to be piecewise linear, there's going to be a mesh.
You know this set up.
It might look like that, and have a value there.
It might have a value there, might have a value there, there, there, and there.
OK. And let's suppose this is the winning, doesn't look like a winner to me because I think it could probably do better, but capital U star.
But remember that we're measuring what's the K in norm stuff.
I'm measuring the difference between these two not point-wise, which would be of course pleasant to say, OK, the distance is just, you know, maybe the maximum distance or the mean square error.
That would be quite pleasant.
But here the measure of distance involves this K, which comes with the problem.
So by distance here I mean the distance between U star and u star.
Can I write it with a capital K there to indicate that's the K norm?
That's the norm in which this is small, as small as can be made.
And what is the K norm for this particular problem?
Well it's the integral, and involves the c, and it involves the U star prime minus the u star prime square dx, integrated from 0 to 1.
I'm just picking this example problem so that you get some idea how we're measuring the error.
It's the natural measure for the error.
It's the energy measure.
We're measuring error and energy.
This is an energy expression.
This thing, you know, represents some kind of elastic bar or something, so we're measuring the internal energy here.
And notice in particular, maybe the most important point is not the c of x, which just comes along for the ride, our measure of the error is in the derivative.
We're measuring error and slopes, because those are the stresses in the bar, and that's where energy comes from, internal strain energy.
OK.
So in other words, I have this function u star, this curve guy, and I have this function, which I'm thanking to be the winner.
And again it's the winner in the sense that it minimizes this.
This is the minimum.
This is the expression that we have some handle on, because we know that U star, capital U star, will make that as small as it can.
It does not make small the point-wise error.
It might try accidentally we hope it does of course, get point-wise error right.
But what it is constructed to get right is energy error.
Make that small.
OK.
So now the question is how do we estimate the difference?
OK.
So now here's this key point, how do we estimate the difference.
Again we're looking at the error.
By the way I should of, maybe I did, put a square there.
That was the norm squared.
You realize why I don't want a big square root sign, it's just clumsy.
So I'm looking at the square there.
OK.
So now how do you estimate the thing knowing that this piecewise linear that came out of some finite element calculation or some giant code, is best possible?
Well here's the idea.
Look at a convenient candidate that might not be the winner.
Let me put, because this was as small as possible, this is less than or equal to U minus u star, it's always in the correct measure, for every trial function U.
This just says what I've said now three ways, that u star's the best in the K norm.
Now I to get some bound on this, I can take any u, I can take any u an estimate its difference from u star, and that will give me a bound.
In other words, I don't know what this particular guy happened to be.
Let me just jump to the key idea.
I know that that one is better than for example, I'm just going to pick one piecewise linear trial function that's quite convenient.
Pick the one that interpolates the exact one.
OK. Now for some reason, known only to finite elements, that wasn't the finite element winner.
That wasn't u star, that was another u.
For example, so like I'll put it in blue here, for example take u to be the function that interpolates u star, little u star the function that I've drawn here.
Since our question is how close can we get to the curve guy by piecewise linear, well one choice is how close does the interpolate come.
It doesn't necessarily come the closest.
Probably not.
But it's in the right ballpark.
So now I just ask the question, and let me draw the same picture again.
I have a function and I interpolate it by piecewise linear guys.
So piecewise linear function there.
And so this is a comparison between the u star and its interpolate, which is my candidate U that I'm recommending as a trial.
And now it's a pure approximation question.
You see we no longer have to know all about finite elements, we don't have to know anything about finite elements.
We just asking the question if you give me a function, and you compare it with the piecewise linear interpolate, how far apart are they?
How far apart?
And let me call the step size h, and of course could be an unequal steps.
It could be an structured mesh.
Everything works here.
Now we come to just a basic sort of understanding of calculus.
How close does a curve come from the cord?
Really that's what it's come down to.
How close is curve function from a cord?
I can even blow that up.
So here I have some curve going up, and compare that with the cord.
And this distance here is h. So I'm just looking for something like is the difference of order h, is it of order h square, is it of order e to the h. What the distance between the two?
And you could imagine it's a parabola, because I'm focusing down on just a little piece.
I take a little parabola, a little h piece of it, and I compare it with a cord, what's the distance between the two?
That distance there.
Well your eye probably tells you that it's smaller than h, because h was this big, and I'm only looking this big vertically.
So this distance, that distance there maximum distance, is of order h square.
But that's not the question.
The question is how far apart are these, how far apart of the these in measure?
How far apart are the slopes?
Because the K a norm is dealing with the slopes, and not the function itself.
So more important than the distance is the error in slope, and would you want to guess what that's like?
What's your guess on that, the error in the slope?
Now the slopes are not going to be as good as the function as always.
Slopes are one derivative higher, you loose something, you get order h. So that's the error.
Let's see, I hope I'm right here.
Yeah.
I think that's right.
There I was speaking pointwise, and then I've got to integrate it over the whole interval here, a unit interval.
So if I square it of course I'm going to get h square, and then if I integrated it, I still have h square, and then when I take the square root I have h. So h is the right quantity.
If I can put down here what our conclusion was from this method by taking capital U to be this convenient, not the only choice, but a convenient choice just to get an idea what the error might be.
And then the error from the actual U star is got to be better, our estimate is order of h for this particular application.
This particular application.
That's the error in energy norm in the slopes.
So the theory of finite elements would go ahead to try to show that the error in the distance between them, into displacement you could say, is h square.
But that's not so easy, and I won't be able to do it here.
Why is it not easy to say that the error in the displacement is of order h square?
It's certainly true for this interpolate, right.
There's no question that the interpolate is you could easily see as h square, order h square away from the function.
But the point is that capital U star was not the best in staying near the function, it was only the best in staying near the slope.
So I don't have this crutch to lean on here.
This is in the K norm, and not in the mean square norm.
It's in a norm that deals with slope, but not with just plain displacement distance.
OK. Maybe that's made the point.
And this is the part of finite element theory just to do that.
OK.
If I could take a minute about notation, because the notation that I've used here of K is really a matrix notation.
You don't truly see it in finite element papers.
For me it's clear, right.
I mean that identity was quite clear from vectors and matrices.
But with finite elements I'm really dealing with functions.
So it's not fair to use matrix notation, you know, when it's integrals, and derivatives, and functions that are involve here.
You know, these are functions, and the actual solution is a function.
So I all I want to do is mention the notation that you now see say in 6920.
The engineering course that would be, you know, quite related to this one of finite elements and Galerkin and all sorts of stuff, that we'll touch in the remaining weeks.
How would they write the minimization problem?
The original problem now, going back to the original problem, and just saying wait a minute we didn't really have a good notation for it.
The thing I'm looking for is like this.
This is the kind of thing that I want, c of x u prime square dx.
Well it might also have a first order term a d of x, u square of x, it could have that.
It could have second derivatives.
It could be in two variables.
For Laplace we had minimum of du dx square and du dy square dxdy, well and we had the linear term too.
I've just written the sort of the left hand side, the quadratic term.
And all I want to say is that everybody, well not everybody, but a lot of people now would use the notation a of u,u for the quadratic term.
So that a of u,u in an engineering paper represents the internal strain energy, whether it's this, whether it's this, a combination.
It somehow suggest to us it's quadratic.
And there's a linear term, and that's often written l of u.
So I better put down what l of u typically is.
This l of u might be the integral of f of x u of x dx.
That would be the linear term.
And I think I'd be happier to have the 1/2 there.
So that really it's just a match with this, but somehow it's a little cooler.
This looks so much like vectors and matrices, that that and that is kind of neutral.
OK.
So I'm just speaking about notation here, and I could've mentioned this last time when I was speaking about all these examples from calculus of variations.
OK.
So that's the minimum problem.
If you give me that notation for the minimum problem, what's the weak form in this notation?
So I'm introducing this just because you see it elsewhere.
It's it's exactly what we're doing all the time, so I just want you to recognize it.
OK.
So how do we get the weak form?
Can I recap how you get the weak form?
If us is the winner.
Right.
I'll just think of u as the winner.
Then if I move it by v, move it a little, then this expression should go up.
So what happens if I move it by v, so I'm going to compare the two.
I'm going to compare that with 1/2 a of u plus v, you know, upped a little minus l of u plus v. Maybe I'll erase min and put in less or equal to.
I'm just recapping that this should be less or equal to this one for all v. You see it's the weak stuff?
u is the winner.
I'm now using u and not u star, and I'm using v for the delta u, for the movement away from the winner, which raises the energy.
OK. Now what do I plan to do?
I plan to cancel common terms here, and see what's going on, and find that first variation.
Just what I did last time.
I'm just doing it in this new notation.
So what's the point?
This l of u is linear.
So l of u plus v is the same as l of u and l of v, right.
If I put in u plus v there, it splits into two integrals here they are, so when I subtract these guys go.
Now what about this one?
Well just as last time when I put in u plus v here and expanded everything, now I have something squared.
So this business here is going to be 1/2.
I'll get something from the u alone.
And then I'll get two something canceling the 1/2 from u and v, that's the cross term.
And then I'll get something from the v alone.
I'm dodging a couple of bullets here just going to the main point.
OK.
So the main point is I'm going to subtract, I'm going to at the differences.
So the 0 ordered terms are all gone, and it's this quantity that has to be great or equal 0, right.
Because I was left with that great or equal for all v. OK. Don't let me leave minus l of v there.
OK.
So that's the thing that has to be great or equal to 0.
Now we're just going to repeat the same the discussion that we had last time with different letters.
If this is going to be great or equal 0 for all v, then I'm going to think of small v's, in which case this term is going to be smaller than the others and won't help.
So what has to happen?
What has to happen for this to be great or equal to 0 for all v?
You know like we're taking the derivative in the direction of v. We're moving in the direction of v. And this is the first order term, that's the first variation.
That's what has to be 0 for all of v. I guess I can bring it over here if you eye will follow it.
I have a of uv minus l of v, that's the sort of first order term and then the second order term.
It has to be great or equal 0 all v. And that question is so what?
What do we get out of that?
Well what I'm saying is this stuff has to be 0, because v could have either sign.
So if this was positive or negative, I could switch sign if I want to.
So it has to be 0.
So a of uv has to equal l of v, and that's the weak form.
That's the weak form, that's the form integral of c u prime v prime dx equals integral of fvdx.
That's the l of v, and this is the a of u and v. This is what you see in engineering papers, when they're launching into the finite element method, and they're planning to get some notation.
That's a very familiar notation.
And then the final point is what about this term?
So now we know this is 0.
This is the weak form of the equation, find u, so that this holds for all v. That's the weak form of the equation.
Oh yeah, I better not rush by it.
Is This is the form finite elements come from.
It doesn't solve this equation exactly.
This is the differential equation.
I'm in here.
It's our Euler-Lagrange equation.
The finite element method says, OK take the trial functions, and get it right for guys.
So that give us our N equations, where this is a continuous problem.
Yeah.
So the weak form is what leads you naturally to finite elements.
You take this, and you only make it true on a finite dimensional space.
OK. And the final comment is what about this guy?
Well the whole point is that we assume positive definite, we assume stability, we've made our life easy by guaranteeing by the fact that this is always great or equal 0.
I don't even have to think about this one.
OK. Because if u and v are the same, this is a square, and that material coefficient that better be not negative, right.
OK.
Thank you for your patience to listen to that.
See this notation for the same thing that we did last time for the weak form, and you see how it pays off immediately to give us the finite element.
OK. Good.
So the project ones are all there, and there's just two or three left here.
And I hope you have a super weekend, and do give a thought to project two.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So, ready for the second lecture about -- second to last lecture about the linear one way wave equation, so the model for hyperbolic equation.
That word hyperbolic tells me that signals are traveling with finite speed.
In this case, the speed is c and in this case, the signals are only going to the last.
The two way wave equation will travel left and right, but again, finite speed.
I emphasize finite speed, because for the heat equation, the speed is infinite.
As soon as you change an initial condition, at any later time, you immediately feel the effect -- probably a small effect way, way out.
So diffusion travels with infinite speed.
Waves travel with finite speed.
The different sort of is that the true -- the true growth factor g -- you remember, the true growth factor was, for the wave equation.
That This is the g for the -- that multiplies the exponential e to the ikx and the difference equation -- and g, the true g, is e to the ikct.
So that -- what I was going to say is, that has absolute value 1.
Again, a difference from the heat equation.
For the heat equation with the fusion, g will be smaller than 1.
You're losing energy.
For the wave equation, the energy is constant and the wave is just moving.
So it's a little more delicate, because we're trying to -- this g in the difference equation has to stay -- still be somehow close to that one, which is right on the unit circle and yet, we got to stay -- we can't go outside the unit circle.
So here are two methods.
We had one method last time that worked.
That was one sided differences and of course, you had to take the difference on the good side, the upwind side and that was first order.
The Courant-Freidrichs-Lewy condition was the same as the von Neumann stability condition and it was r between 0 and 1. r is this ratio, c delta t over delta x, it comes in.
It comes into the wave [?
limit.
?]
So now we really have to think about better methods, other methods and here are two: Lax-Freidrichs, which will only be first order.
So when I -- these are the questions I always ask, and then finally, experiment.
I mean, answering these questions will certainly be a guide to what we'll see experimentally, but there's more to learn from actually doing the numbers.
The accuracy, Lax-Freidrichs, will be only first order, whereas Lax-Wendroff will be second order.
So with respect to accuracy, that's certainly better.
Can we see that maybe without too much calculation?
The notes would have the more details.
Can you see -- here's Lax-Wendroff, Lax-Freidrichs.
So what's the idea of Lax-Freidrichs?
You remember the total failure, the complete and absolute failure when we used a time difference there and a space difference there?
You remember that that gave the time difference that produced a 1 ng and this space difference produced an imaginary part ng, so we had g as 1 plus something imaginary and we were outside the circle automatically.
Freidrichs thought of a way to get around that.
Instead of using ujn here, instead of the backward difference, he replaced ujn by the average of that value and that value, so that both sides now involved just two numbers -- uj and uu, this one and this one.
This one is not used now.
Again, for a still first order, you won't be surprised that this approximation, this has got first order error.
But stability now is back with us and easy to check because now each new value depends on what old values -- just these two, and with what coefficients?
So let me just check stability for Lax-Freidrichs.
So uj plus 1 n, the new value, is some multiple of this guy, the one on the left and some multiple of this one, the one on the right.
What are those multiples?
Let's see.
I'm going to move things over to the right hand side of the equation, so I think there's a half and then I think there's an r over -- minus, I guess.
This is the one that comes in with a minus sign -- it's the one on the left and the one on the right, uj plus 1 n. There's a half plus r over 2.
Simple idea.
Simple idea and it has one important advantage over upwind, one sided that we saw last time.
It's not a great method.
It's only the first order here.
If I -- and you see when it's going to be stable?
The coefficients add to 1.
Our coefficients always add to 1, right?
Because if we have a constant state, this next step should be that same state.
It's not going to change, but instead of having the typical 1 in the g, I have -- it's split into a half, so what is g for this guy?
g for this guy is this number: 1 minus r over 2 times e to the minus ik delta x plus this number times e to the plus ik delta x.
You're seeing now the pattern.
So I've sort of jumped to g without plugging in the e to the ikx , taking a step and then factoring out e to the ikx.
I did it without writing down the e to the ikx.
This is what factors out.
Maybe a 1 plus r would be more correct.
So civility requires this complex number, which I could write in different ways.
The real part there would be a cosine.
This would actually be the cosine plus r times the sine -- plus ir times the sine.
This is really -- I could rewrite that as cos k delta x plus ar sine k delta x. I don't know which way you -- here, this shows you the real and imaginary parts.
That makes it nice.
This shows you the two coefficients.
That makes it nice.
Can you see stability -- so what is Courant-Freidrichs-Lewy?
The characteristic discussion, the argument based on characteristics, says what?
Stability is going mean that delta t -- stability is going to happen when r is below 1.
It's just like upwind, except that r could also be coming -- c could have the opposite sine.
Like Freidrichs is prepared for c to be positive or negative, because it's not purely upwind or purely downwind.
It's both winds.
You see that if r is anywhere -- the condition is going to be that minus 1 is -- but r is between minus 1 and 1 -- see what's nice about Freidrichs in that case.
This number is positive.
This number is positive, because the 1 is bigger than the r. So each step -- and they add to 1.
So each step takes the new value as a combination of the two old values and of course, the magnitude can't be bigger than 1.
So under this condition, I have positive coefficients.
That's positive and that's positive.
They multiply things of absolute value 1, so they can't -- the sum can't get beyond 1 -- or you might like it here, that magnitude squared is cosined squared plus r squared sine squared, and if r is below 1 where smaller then cosine squared plus sine squared, which is 1.
So if I draw the -- just for a moment, if I draw the g in the complex plane -- so there's the unit circle.
k equals 0 is always there.
That's at 1, then these points -- I think probably those points fall on an elipse here, where we go up as high as r or magnitude of r and we go left, right as far as -- this is k delta x equal pi as far as minus 1.
So it's stable if this is less than 1.
Nothing hard.
In fact, easy.
Two positive numbers that add to 1.
Now I've added a problem in the problem set.
Not requiring it to it be turned in, but just to mention it here -- that whenever the coefficients are all positive, the accuracy could only be first order, so we can't get too far with positive coefficients.
First order accuracy is the best possible, except in the extreme case when we're right on the characteristic.
So if r was exactly 1 -- you remember that special case when r is exactly 1 and we're following exactly the line that carries the true information -- r is 1.
That's gone and this is just -- so if r is 1, then uj plus 1 n is u -- is this, is that number and we're just carrying that information exactly.
I have to say, again, that the idea of trying to follow that characteristic is a totally natural one and I'm not writing down here a message that tries to do that, but it wouldn't be a bad idea in one dimension.
You couldn't do it, really, well in higher dimensions where these ideas do continue to apply.
So that's Lex-Freidrichs.
That's Lex-Freidrichs.
So now let me turn to Lax-Wendroff.
These are natural ideas, but Lax and Wendroff did it first.
They said, if I use these three values, all three, then certainly I can get the accuracy up one more because I have one more coefficient to choose and to see what they did, the easiest way is to write it as a time difference on the left to see how they match.
All they want to do with match that second term, that delta x squared term in the Taylor series.
Because this is going to give us a d by dt, this will give us a d by dx centered, so we get extra accuracy there and the matching that we need is this term.
Do you recognize that?
What does that term approximate?
Hope I've got it right.
That term approximates uxf, right?
So this is -- they've thrown in a term that approximates uxx and its purpose is to catch the next order of accuracy and hopefully not lose stability, hopefully remain stable.
So what Lax-Wendroff are doing is staying even closer to this curve.
I guess I have forgotten exactly what their curve will be.
If I write down g of, maybe you could ask MATLAB to do that.
Write down the g 4 -- this for Lax-Wendroff.
It's got that extra term and plot absolute value of g. I guess you might plot it along with the Lax-Freidrichs out.
So what am I guessing?
I'm pretty sure that the Lax-Wendroff one will hug the unit circle more closely, but it'll stay inside.
So can I just -- Lax-Wendroff is g closer to 1, at least for a small k, at least for a small k. That's really where the accuracy is checked.
How close does these Taylor series match?
Lax-Wendroff match better and they stay inside.
That takes a little calculation.
Took me awhile yesterday to rewriter the -- write out the g, square the real part, square the imaginary part, add them up and this is what I got.
So for Lax-Wendroff, I got g squared is 1 minus r squared minus r to the 4th times 1 minus cosine k delta x squared.
I put in the notes eventually, much algebra, but it does the job.
Because if r squared -- so what's the stability condition?
It's again this same guy, the same condition -- r squared below 1.
Again, we could be -- we could have a left going wave and a right going wave.
Now you'll say, what's the good of that?
Here we just have one wave, but on Friday, we'll have a wave going both ways.
So Lax-Freidrichs and Lax-Wendroff can both deal with that.
In fact, their stability conditions are the same and in fact, you may be wondering why would anybody use Lax-Freidrichs when Lax-Wendroff gives higher accuracy at no real significant increase in cost?
So Lax-Wendroff is a favorite.
Don't forget, still experiment has to be done to see what it really does in practice, because right now, everything's looking in Lax-Wendroff's favor with one exception.
Lax-Freidrichs, which only used these two coefficients and had them positive.
This is positive.
This is positive, where Lax-Wendroff is going to have a negative.
Let's see.
Where will our negative -- I think there'll be a negative coefficient in here somewhere.
Probably one of the three outside guys.
I know that we couldn't have three positive coefficients and get up to the second order actors.
Now where do we pay the price for that?
Where do we pay the price -- I should finish here.
Suppose r squared's bigger than 1.
If you accept this calculation, which has a nice result, but took awhile -- if r squared's below 1, then what do I know about g?
Suppose r squared's bigger than 1.
What can we say from this formula?
Does that hit you right away?
g -- if r squared -- this is certainly -- absolute g. I've taken the actual g and squared its real parts, squared its imaginary part, did all the algebra I could think of and got this answer.
For the magnitude squared -- so the magnitude squared is never going to go below 0.
I don't have to think of that.
It's just I want to be sure it's below 1.
So all I really want to do is be sure that I'm subtracting something positive.
Am I subtracting something positive?
That's certainly positive.
Is this positive?
r squared'll be bigger than r to the fourth.
If r squared's less than 1, then r to the fourth is even smaller.
So this is positive, this is positive.
I'm subtracting something so g is certainly not bigger than 1.
So that formula really makes it evident and I guess also evident that we're getting some high accuracy.
You see, what's the size of this when k is small?
When k is small, if I'm taking the cosine of a small angle -- what am I looking at there?
If this is a small angle theta, I'm looking at 1 minus cosign theta -- that's of what order?
Theta squared.
Theta squared.
Then that's squared again, so that I'm up to, theta to the fourth.
That's why I'm getting off to a good start.
So stability we've got, two sided we've got, experiment -- can I just try to draw a little graph of what I think the two methods will do for our model problem?
Like this model problem of a wave moving along?
I want to try to draw it at, let's say, t equal to 1, for example.
So at t equal 1, the wave should be here.
I'll draw -- you might say this is totally unsatisfactory, and it is, because I'm just drawing by hand what you can see.
Maybe you should.
This would be a perfect experiment and I really could use for the notes -- the notes could use a proper graph.
I think what you see -- let's see.
At t equal 1, both methods are going to be 0 way out here and they'll be 1 way out here.
Why's that?
Because the signal speed is finite, right?
This way out here, it hasn't heard about this wave of water.
Way out here, it doesn't know that it's not a constant.
The question is, and I guess the thing has moved, so maybe -- can I try roughly?
So the true wall of water would be that.
I think that of what Lax-Wendroff -- Lax-Freidrichs -- so Lax-Freidrichs with positive coefficients will show a smooth profile, but the perfect signal will get spread out.
The perfect signal will get spread out.
Why is that?
Let me just draw a spread out signal.
So apologies to do this by hand.
I think it's probably going to be pretty standard, but it's going to be spread out.
So that, I think, would be Lax-Freidrichs.
That's very rough, right?
But I really -- hearty thanks for anybody who gives me a code that does it and graphs it.
But the main point is, we have a lot of smearing here.
The accuracy is not going to be terrific.
But there isn't any bumpiness.
The reason there's no bumpiness is that we're Lax-Freidrichs with -- has these positive coefficients, so that every value always has to stay between 0 and 1.
The starting values were between 0 and 1.
At every step, the values are between 0 and 1, because we're always taking a combination of a positive multiple of one value and another positive multiple of the other value and the two coefficients adding to one.
We're taking 30% of one and 70% of another and we never -- we're always in between the 0 and the 1.
What about Lax-Wendroff?
What are you expecting for Lax-Wendroff?
Expecting better accuracy, expecting -- if I were on the ball, I'd have another color.
Lax-Wendroff will be a much better profile, sharper profile, but you know what the price is going to be?
There'll be some oscillation.
There'll be some oscillations behind -- I think behind -- oh gosh, I should know.
Forgive me for -- I can fix these if you'll tell me how on the next -- because I'm not sure that I should know whether they're at both or whether the oscillations happens at both ends.
I doubt it.
No, I doubt it.
I don't think it will.
Why do I not think it well?
Because in Lax-Wendroff, the three coefficients -- one of them will be negative and two will be positive.
They'll add to 1, of course.
So the oscillations, which are coming from sometimes a negative coefficient, sometimes a positive, will all be on one side.
So only one of those oscillations [UNINTELLIGIBLE] So this is my picture of -- can I -- of Lax-Wendroff with oscillations behind the wave front.
Alright.
If you've done any computing, you, when you see these oscillations, you immediately think of somebody's name, which is Gibbs.
Of course, Gibbs -- you remember the Gibbs phenomenon that comes when you're approximating a square wave, like there by pure Fourier series?
So the Gibbs, the classical Gibbs phenomenon is, take the 4 a series for a square wave while it's smooth and then truncate that series, cut that series off at 100 terms or 1,000 terms.
Gibbs noticed -- and it's always a problem -- that these oscillations appear right near the front.
So far away, no problem, but here we really see it only on one side, because we're not doing Gibbs Fourier series experiment.
We're doing Lax-Wendroff wave equation.
When I say we, I really hope you'll do it and new lines of MATLAB will be greatly appreciated.
Maybe Mr. Cho can -- you could email to him, for example, a code and the plot.
So what does that tell us?
Tells us Lax-Wendroff has got a lot of potential, but this oscillation isn't good.
How are you going to get rid of it?
How do you get rid of bumpiness?
You introduce diffusion.
So diffusion, heat equation stuff smooth things out.
Wave equations don't smooth things out.
This wall of water in the wave equation stayed a sharp wall.
But we'll see very soon in the heat equation, diffusion, viscosity, all those physical things connect numerically to a smoother solution, smoother profile.
So that is really a key idea, is -- and we'll see it, but it will make the problem not linear.
If we want to stay with a linear equation and increase the accuracy, we're going to get oscillation.
But if we locate the oscillation, introduce some diffusion in the right place, we can reduce the oscillation.
So that's where -- this is what you really have to do if you want good output from numerical methods.
Even on this simple model problem, you'll have to have this idea of numerical viscosity.
Artificial viscosity, numerical diffusion popped in at the right spot.
So we'll see that.
Maybe the right place to see it will be for when the equation itself is already nonlinear.
THat's where it has developed a lot.
So we'll -- when we get to nonlinear wave equations, which isn't far off, that's what we'll see.
So now I guess I have one more topic in this lecture.
that is convergence.
How do you show that -- we know, we sort of know that we need stability.
So I guess I'm going to have to come to the question of convergence.
Alright.
Of course, I have to mention -- more than mention -- the key result in the whole theory, the connection -- so we have methods that are first order or second order -- stable or unstable, and then the key idea is this Lax -- again, he enters -- Lax equivalency.
So this is a little bit of pure math, comes into this and it gives exactly the result we would expect.
Stability implies convergence and convergence implies stability.
So that's the two ideas or equivalent, provided we're talking about a decent problem, for a consistent approximation to well-posed problems.
I've got four words on the board, four keywords words here.
Stability, we kind of know what that means.
Convergence.
These are the new words and I'll just -- and actually -- so consistent means that the method is at least first order and that the difference equation approaches the differential equation at a single step.
Well-posed means that the differential equations is OK.
So, for example, what would be a non well-posed problem?
When we get to the heat equation, if you ran the heat equation backward in time, that would give us an equation dudt as minus uxx.
That would be not well-posed.
That would be not well-posed.
If you tried to solve Laplace's equation equation -- say, utt plus uxx equals 0 from initial values.
That would not be well-posed.
But we're dealing with well-posed problems.
The wave equation, which has the right side and the heat equation with the right side.
So our problems are going to be well-posed.
You can safely think we're only talking about consistent approximations to well-posed problems.
Here is the main point.
Stability holds if and only if the convergence holds.
So there's actually two things to show there.
How does stability give convergence?
Why does convergence requires stability?
So that instability could not [UNINTELLIGIBLE] So that's sort of maybe the fundamental theorem of numerical analysis.
The fact that stability, a balance -- we better say what stability is and what is convergence.
So let me capture these x in any quality, in mathematical terms rather than just words.
So those are the ideas -- maybe I'm going to have to write a little above here, but -- so what does -- what's our methods?
So I need some notation and then I'll lift the board and we'll see what each thing means.
So our method, we're allowing any difference method that moves forward in time.
There's time.
Let me use r for the true operator that takes the function, follows the differential equation for the next time.
So I'm going to say that true u, which is always a small u at delta t is sum r, something r times u of t, where the approximate guy is sum s times u of t. So r is the true solution.
So well-posed.
So of course to take -- if this is n delta t and we've taken r n times, so that the true solution is r to the nth times the initial value and the approximate guy is sn times the initial value and we'll -- we might as well start out with those.
That's what r and s are.
So now we got three ideas here.
What does well-posed -- well-posed means that the true thing is bound.
That r to the nth times u is less than some constant times u.
These inequalities are sort of part of the mathematics of partial differential equations.
I think you've got to take them as not -- I mean, take them as sort of a reasonable statement that the -- this is energy this u typically will measure the energy and in our wave equations, c 1, the constant c 1 will be 1.
The energy doesn't change.
In another problem, we could allow the possibility of c 1 being a little bigger than 1, but think of c 1 as 1.
Then stability will be abound on s to the n, say, c 2.
And again, in our examples, c 2 has been 1.
This is like -- c 2 is like -- I mean, s to the nth on each exponential is g to the nth and g is below 1.
So often c 1 and c 2 will be 1.
Then finally, what is this ideas of consistency?
And this means -- consistent means that -- so what's consistency?
Consistent is comparing the differential equation with the difference equation.
It's comparing r with s and at a single step, they should be close.
So at a single step, r minus su -- that's the -- starting with the same u, I'm just taking one step in the differential equation and comparing it with one step in the difference equation and I would want that to be c 3, maybe delta t to the p plus 1.
Was that -- is that -- times some size of u, I think.
Order delta t to the p plus 1.
Like delta t squared.
This would be the usual situation, that when it's stable.
So at the moment then, I want to say, if these are all true, then I get convergence because what I'm now going to do is show that stability implies convergence.
It's what we did for ordinary differential equations.
This was the local error.
This is the local error that I called de, the discreditization error.
These are saying -- so this is saying that the -- here's really what happens.
So I want to look at the error up here after n steps.
So the error after n steps -- typical term is, I follow the differential equation.
That's u.
Then I have this little error de, which I make at that step, the difference between r and s and then I follow the difference equation up to the remaining time.
What I want to show is that I can track the error and bounce it.
I don't know if you remember the discussion for ordinary differential equations, but it went the same way.
We had -- we looked at the new error de that was produced at each time step k. When we took steps of -- when we got up to this point with the true solution, we plugged the true solution into the difference equation, it had a little error and whatever error there was, that got thrown in with other errors and propogated up to any later time s. Am I bringing back what we did there or just -- you can just see that's what we're going to have here.
Now how will I express that?
Somehow I have to estimate the difference between r to the nth and s to the nth.
That's really my problem.
Actually, it's a nice problem.
Suppose I have control of powers of r and s, and I know that r minus s is small somehow.
How do I show that r to the nth minus s to the nth is small?
That's my job.
This is the error when I applied to a starting value u.
This is the difference between little u and big u.
So how do I -- what I want to do is split it out into this situation where I just have 1 r minus s. I think this'll work.
So suppose I have -- I think I could start with r to the n minus 1 times r minus s, and then in r to the n minus 2 times an r minus s times an s. Maybe I don't -- sorry, this guy is going to carry and I'd rather do -- sorry.
Got to do it right because I want the rs to come first.
I want maybe r minus s times s to the n minus 1 and then -- sorry.
I want rs on the right.
Good.
Finally got it.
So that gives me the r of the nth, but I've subtracted it off so I need an sr minus sr to the n minus 2 and so on.
Sorry for this bit of algebra in the middle of practical things, but do you see that this is the case where I followed the differential equation up and at the last step I made an error?
This is the case where I followed the differential equation almost to the top, I made an error and then I had one more step to go with the difference.
The s doing this part, so a typical term will be a power of s, s to some power, r minus s, r to some power.
I'm breaking the error at n steps into n different terms, each from the error at one step.
This is the error at the last step.
This is the error at the next to last step and so on.
We've got it.
Because I just estimate every term, every term has a power of r, which gives me a c 1, a power of s which gives me a c 2 and the discreditization error, the one step actual error, r minus s, which gives me a c 3 and this delta t to the p plus 1.
You see that I'm just piling up n of these, so I'm getting n of those, delta t to the p plus 1, so the final result will be, I'll have n of these delta t to the p plus 1s and that will be t and delta t, delta t to the p. So that'll be the error.
Since I went to all the trouble of defining a c 1, c 2, and c 3, those three numbers.
So the error -- there'll be a c 1 times c 2 times c 3 constant there in the bound.
Have a look at the notes for that and let me just concludes with Lax's insight on that other part.
The fact that convergence require stability.
So you see, what I've shown is that if we're stable, if I bound these, bound these, bound these, then I have a bound on the error.
That's convergence.
The other way is, if it's unstable, how do you know it couldn't converge?
How do we know that convergence requires stability?
It's a little delicate, so I just -- it's the last 30 seconds of the lecture, just to mention this point, because it's easy to miss a key point here.
We've tracked a different -- now I'm supposing that the method is unstable and I want to show that it will fail, right?
That's my job now.
That's Lax's -- other half of Lax's theorem: If it's unstable, the method will fail.
There'll be some initial value for which it blows up.
Now you might say, that initial value is just e to the iks.
Buw we didn't follow this point, but if I take it e to the iks where g is bigger than 1 -- of course, that is getting amplified a bit, but for that particular e to the ikx, when delta t decreases -- so if I have a -- you see, this is a k delta x.
What I'm trying to say is that for a fixed k, even an unstable method can work.
For a fixed frequency k, as delta t and delta x gets smaller -- these guys, whether they're outside or inside, will be coming down toward 1 and if you're patient, you could see it actually worked.
So it's somehow the initial function that doesn't work, that blows up, is a combination of frequencies.
A single frequency, actually, you move down here and it's not catastrophic in the end, but a combination of frequencies, so that as one frequency moves down, another one is moving into this position, another one into this -- there is a typical initial function and it is disaster.
So that's what Lax had to catch.
The notes will use the single word, uniform boundedness.
Anyway, this is one point where pure math -- a kind of neat result in pure math says that if stability fails, then there is some single starting value on which it will fail.
That starting value won't be a pure e to the ikx, because it, as we said, actually in the end gets to be OK, but there will be a combination of frequencies that'll fail and so convergence will fail.
So that's the key theorem.
I probably won't use the word theorem again, but this is one time when it's justified and when a little sort of abstract functional analysis gives the conclusion that convergence requires stability.
So you'll see that in the notes and next time we move on to the true wave equation, second order.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
This morning's Siam News.
Well, it's an article on something that's not totally up to date because it's a method that was developed 50 years ago called alternating direction and it was exactly developed-- first you did an iteration in the x directions.
Of course, we're tridiagonal along every row, we're one-dimensional.
And then the second half-step is an iteration in the y direction.
Again, tridiagonal, so that iteration's very fast.
You flip-- alternating direction describes it perfectly.
And that method is still used.
It got developed in the oil reservoir simulation world.
That is, Houston.
And the article says the evolution of techniques for oil reservoir simulation has continued.
And then mentions 3-- so it went from this alternating direction to line over relaxation, LSOR over relaxation and I only spoke briefly about that.
What's that word line mean?
That means that we're taking hole rows at once.
Working with blocks and I'll add some notes about that today.
So that was the next step, line SOR.
Then to the Newton-Krylov schemes, that's what's coming.
Conjugate [?
gradients ?]
with ILU type preconditions.
So again, we're talking about preconditioners and one important choice is this incomplete LU.
So notes on Gauss-Seidel and I should maybe add incomplete LU.
By the way, experiments badly needed here too.
Because incomplete LU has this tolerance and as you move the tolerance you get closer or far away to the exact L and U, but you get faster or slower.
So what's the balance?
What's the right tolerance?
How do these methods compare-- and overall, how do these iterative methods that we're speaking about compare with these direct methods?
So we really are in this new chapter solving large systems, facing a whole lot of possible new experiments.
So just to continue the history or the future as described in Siam News, so there was alternating direction, line SOR.
Then to Krylov schemes with ILU preconditioners in use today.
So this is a big use area and the multiscale, multigrid solvers that are quote, now starting to come of age.
So that's a third topic for today's lecture, multigrid.
That of course, it's been around for some time, but it involved, especially at the start more effort in programming.
So it didn't quickly-- I mean, it gave great results on model problems.
On other problems, too, but still it kind of hasn't yet, but is penetrating into production code or the oil reservoir industry or other big industries would use it.
So here you see really a pretty fair picture of where we are today with large solvers.
These are very important.
Minimum degree-- if the size is not overwhelming direct methods are going to be right up there.
And these are the pure iterations, stationary iterations that with this idea, this incomplete LU idea get pretty good.
But the formula's always simple there.
The new x comes from the old x by the same iterations.
Whereas, if we use the ideas there as preconditioners and look to see-- so the point was, and I'll repeat it again, that Gauss-Seidel and Jacobi, the standard iterations were quite satisfactory on high frequencies.
They damped those out quite well in the error, but they don't tackle the low frequency, the smooth part of the error, very efficiently.
And that's what multigrid does incredibly well, so it's a nice mixture of the two.
These are the smoothers, this is the change of scale, change of grid that's coming now and will come.
So all this will be my subject for this week and next, prior to spring break and I think you will have enough to go on to do experiments.
It could be highly interesting.
OK, so that's the Siam News report.
OK, what do I want to say about minimum degree?
And more will go on the web.
Just a few more notes about what we did on Monday after class.
If you take a matrix like that, you take a matrix and how is it described in sparse format?
Sparse format you can see-- normally you don't have to see it, it's usually inside the sparse math lab, but if you want to see what's happening you could ask for i, j, and s. Now what are those?
What will that produce?
i will be the list of nonzeroes, the list of the row numbers of the nonzeroes.
So i will be-- maybe I'll write it as a row so I'm transposing, So the row numbers would be row 1, row 2, not row 3.
Row, 1, 2, 3, row 2, 3.
The column numbers would be-- these are going to be the pairs i,j.
So that's column 1 twice, column 2 three times.
Column 3 twice.
And what's s?
s is the actual numbers in position i,j is the number 2.
In position 2,1 is the number minus 1.
So those numbers minus 1, 2, minus 1 is that column.
And then minus 1, 2 is that column.
So of course, we've got all the information in the matrix.
We know every nonzero position and we know what that entry is.
And of course, our matrices are much bigger than this one, but already we can see one useful point.
Point being that key word that the column numbers-- this I mean, imagine we have a large matrix, so this j is not very efficiently recorded here because what's j?
We're looking at nonzeroes a column at a time, so of course, we'll have a few ones for the nonzeroes in column 1 and then some twos and then some threes and some fours, but all this row is-- the real information in that row is a pointer to-- in other words, I don't have to repeat 2 three times.
So really if I put inside here a compressed version of j, a pointer would be a short factor that just has a 1.
It says look in position 1 for the start of column 1.
Then a 3, see I didn't need that because that just continued column 1.
That just repeated the j I already had, but this tells me that in the third position I start down column 2.
And this would tell me that in position number 4, 5, 6, I guess, I start down column 3.
You see the point of that 6.
That 6 picks out-- yes-- 1, 2, 3, 4, 5, the sixth row number is the beginning of information on column 3.
And then it's conventional to have a pointer, 8, to say finish.
So that's pointing to empty space.
So in other words, j got compressed to 1, 3, 6, 8 and in a large problem it would get seriously compressed.
OK, so that's the form in which the code keeps the matrix and does the reoderings.
Let me just mention that the opposite of this would create the matrix out of these i, j, s. What command would that be?
You could create the matrix a out of the command sparse, would be good.
Sparse of the inputs now would be i, j, s. If I input i, j, and s then that lab creates a matrix.
And sometimes I may want to include the m and n, the shape of the matrix as further parameters, but actually here I wouldn't have to.
So this is the opposite command from this one that we're interested in.
You can imagine.
Suppose by using minimum degree or some other decision, I've eliminated up to a certain point and I want to put the remaining columns in a different order.
I can do that just by playing with the pointers.
So it's a very efficient structure for sparse matrices.
And then comes the question, OK, what's the good order?
I was surprised to realize how open a problem that still is, even from the expert who's developing the key code.
So approximate minimum degree of course by that word you see that it allows freedom.
There are also decisions to be made when minimum degree is a tie between several nodes as it commonly is.
And it's rather nice to get a math lab movie that shows the order in which nodes are eliminated, edges are removed from the graph, from the mesh.
You'll see that.
All right, I'm ready to go ahead now to some comment on Gauss-Seidel.
Maybe I'll put that here.
In fact, maybe I'll just take the same matrix.
So now I want to remember, what is the Gauss-Seidel method for ax equal b.
What's the iteration now?
And again, a will be the same: 2, minus 1, zero minus 1, 2, minus 1.
My favorite, OK.
So all these iterations you remember are splitting of the matrix.
Some of the matrix goes on the left-hand side.
That's the preconditioner.
So it's p, the preconditioner that's sort of close to a in some sense, but easy to work with, multiplies the new thing and on the other side is p minus a-- the rest of the matrix-- that multiplies the old plus right-hand side b.
Let's just remember again, that if we converge to the point where this is the same as this then we have the right answer.
We have the right answer because if that's the same as that, px is the same as px, this comes over on the other side, ax equals b.
So when it converges it converges to the right answer, but you remember that the key equation was that the new error is the old error multiplied by this matrix i minus p inverse a.
That's the iteration matrix you could say.
I just get it when I multiply both sides by p inverse.
And so the problem is choose p so that this is easy to do and at the same time, this has small Eigen values or as small as you can get.
So the Gauss-Seidel is a particular choice, which takes the lower-- p is the lower triangular part here.
So p is, so I just thought I'd better write down explicitly Gauss-Seidel takes that with zeroes there as multiplying-- that's p. And what's p minus a?
It's the rest and it's been moved to the other side so the rest is going to be the strictly upper triangular part.
And because it's moved over to the other side it'll have a plus sign.
It'll be 1 and 1. x sub k plus b. I just write it out so that you see, totally explicitly how being triangular makes the solution step immediate.
Because the first equation will tell us the first component, right?
The first equation, because it's triangular there's only an entry here.
And by the way, all these codes, including Tim Davis' minimum degree codes, their very first step is reorder the equations if necessary to guarantee that the main diagonal has no zeroes.
We want to know that in advance.
So that you just do, let's assume of course, for us it happens automatically.
OK, so that's not zero.
And that first equation gives us the first component then we know the first component so we use it here and the second equation tells us what the second component is.
We use that second component in the third equation to find the third component.
So there's no loss of speed compared to diagonal and actually, it's faster because the storage we're changing-- we're using the new first component to find the second and the new second component to find the third, so we can overwrite x sub k by x sub k plus 1.
Let's see.
I guess if I had space and had prepared I would figure out what this matrix is.
Maybe you could do that.
Figure out what-- here's p. It's invertable.
P inverse is going to be quite simple.
Actually, it would be extremely simple.
You'll be able to see its Eigen values immediately.
We could stop to do it, but I think if you do it it's more valuable.
So we would find that its Eigen values were below 1 and that they were the squares of the Jacobi Eigen values, so the method is twice as fast as Jacobi.
But it has the same good feature of damping the high frequencies and the same bad feature of not very much damping the low, smooth part of the error.
Now we're ready for multigrid, which is intended to solve it.
So multigrid is going to be-- the point is by changing grid, by changing from the fine grid to a coarse grid the smooth oscillation doesn't look so smooth.
On the coarse grid, it's effective frequency is effectively doubled and it's moving over in the direction where the smoother can attack it.
We'll see that happen.
So now, really this begins the lecture on multigrid, which is going to take certainly, today and Friday.
There's a very good book by Briggs and others.
He wrote the first edition of the book and coauthored the second one.
And it's beautifully short and clear and simple and this presentation will follow the same exposition that the book does.
So the key idea is to see what-- how to go from a fine grid problem, so this is on the fine grid with step size a.
So this A sub h is our problem.
A sub h is A.
A sub h, b sub h, we're looking for u sub h on the fine grid.
Fine grid means lots of mesh points, lots of unknowns, big problem.
OK, so the idea is going to be somehow to-- let me just start with two grids.
Well, you see what you do on the fine grid at the start.
You iterate.
You use Gauss-Seidel or Jacobi, whatever.
Maybe three times.
Maybe three iterations, but don't continue to a thousand iterations, it's a waste of time.
Then here is the real-- this is the multigrid part.
The multigrid part is going-- you get an answer after a few iterations, after you've smoothed it.
You compute this residual, which is the amount you're wrong in the equation, the difference between the right side and the left side.
I can put h is here too, to emphasize.
This is residuals being computed with what we have, which is on the fine grid.
Now here's the change of grid.
Number two says, restrict to a coarse grid.
Take that rh, that residual which is probably a bit smooth and move it to the coarse grid.
So the coarse grid we identify by the fact that it's mesh width is twice as big, 2h.
And so we're going to need some restriction, so the input to this multigrid is to decide on a restriction matrix, how shall we take values that are given on a fine grid and restrict them to a coarse grid.
Of course, one restriction is-- and it's not that bad-- is just to take the values of r that are on the coarse grid at every other grid point, use those.
Or we could take into account the neighbors.
You see the question.
So this is coming now in this restriction.
So the restriction is, restriction r, this is fine to course, and we have this question of use neighbors or not?
That will answer it.
Well, the decision on what the matrix r is.
That's the key.
OK, so we choose an r, we restrict, and we have now a half-size problem.
Well, half-size in 1D, quarter-size in 2D, eighth-size in 3D because the mesh width has doubled in every coordinate.
Now I've put solve.
I better put quotes around that.
Well, solve the problem on the coarse mesh.
So the idea is that that's much faster Well, you might say, what problem?
Well, we have to create somehow the matrix on the coarse mesh.
That's not necessarily given to us.
We start the problem with A, I mean, A sub h, the matrix on the fine.
Can I just mention that this is probably, this v-cycle-- that letter v is supposed to suggest going down to the coarse mesh and back up to the fine mesh.
And the standard notation in all the multigrid books is a capital V. And why do I use a small v?
This is my educational contribution.
Capital V is appropriate when you have several meshes.
You go to 2h, you go to 4h, you go to 8h, back to 4h, back to 2h, back to h. So a deeper multigrid.
So it would look more complicated.
I would repeat this idea instead of solve here.
Instead of solve at the 2h level, which of course, I'm not going to do exactly anyway.
But if I was in a big V-cycle I would iterate a few times to smooth here, just as I did here.
I would iterate using weighted Jacobi or something and then go down to the 4h mesh.
So restrict to the 4h mesh.
So you can see that I would stay in this little loop to get to the bottom, 8h and then I would start back up.
So I think it's a good idea to use a small leter v to tell us rather than saying two grid capital V-cycle.
I'm just going to use a small v to signal right away that it's two grids.
Fine, coarse, fine.
OK, so we have some solution, not necessarily exact and actually, there isn't that much point in getting it exact because what we want is to move toward the right answer.
And notice that I'm looking here at the error rather than looking at A 2h, u 2h-- the actual solution.
What I'm computing with is the correction term.
So this is the correction term that I find.
So this is the residual over here.
this is the A 2h that I still have to choose and then solving that will give me a correction, but that correction is on coarse mesh.
I've only had that correction E 2h defined on the 2h mesh.
So now comes the other part of multigrid.
Get back to the fine mesh, climb back up.
I take that E 2h and apply an interpolation.
I need to create an interpolation.
This is going to be coarse to fine.
And I have to decide how to do that.
Once I've decided it that gives me a correction at the fine mesh level, which is where I'm really working, so I make the correction and I have a better answer.
And probably, I iterate a few times-- that would be called a post smoother on that correction.
And maybe that's where I stop or maybe I repeat little v-cycles, but that's not brilliant.
If you're going to do multiple, if you're going to repeat multigrid it's much more efficient to move to more and more coarse meshes because the calculations on these are way faster than on the fine mesh.
You see, rather than-- I much prefer to go way down and back up then a lot of v-cycles.
I mean, that would be a small w-cycle.
And not brilliant.
Much better to use a big V-cycle or a big W-cycle.
There's a place for W-cycles but they're capital W-cycles because you want to get down where it is very fast, very inexpensive and in fact, so efficient that multigrid achieves this holy grail of giving you an answer with whatever accuracy you want-- giving you an answer in o of n operations.
n being the matrix size.
Captial N squared in our example.
So that's the fantastic result from multigrid.
So I'll come back to this, but just say if the size of A is N, which is capital N in 1D, captial N squared, capital N cubed, then the multigrid works in o of in flops, Floating Point Operation.
Not even log n, which we think of for the-- n log n we think of for the FFP, for the Fast Fourier Transform.
But here's it's actually order of n. That really is a goal worth achieving.
So we know what the smoothers might be.
It's the i and the r that are new.
And then the analysis of convergence, why does it converge?
But to use the method, and so by the way, projects using multigrid are totally welcome also.
So let me take the interpolation i.
So what's that?
Coarse to fine.
Let me imagine I'm in 1D, so there's the x direction.
Here's one end of the interval, here's the other end of the interval and suppose my boundary conditions are zero.
So this is the coarse mesh and let me put in here, make it the fine mesh.
So at this point I have an answer on the coarse mesh, which satisfies the boundary condition and let's say it's there, there, and there.
OK, and I need values on the fine mesh at the other points.
Of course, I'm going to use these values when I interpolate.
I mean, that word interpolation means implies, keep what you have on the given points and I'm just going to do linear interpolation.
So halfway there, halfway there, and halfway there.
So ignoring the boundary conditions, the zeroes, it's going to be 5 by 2 I think, this matrix i in this example will be 5 by 2 because it has 2 inputs, these and it has 5 outputs.
And what's the matrix?
Well, for their second and fourth outputs it uses exactly-- this multiplies x or I should say u, u sub 1 and u sub 2.
Follow the notes again.
The notes use v sub 1 and v sub 2 for the coarse mesh values to avoid h 2h, extra subscripts.
So this height is v sub 1.
This height is v sub 2 and I'm going to take-- this'll be the new u sub 1, u sub 3, and u sub 5 And u sub 3 I'm going to save.
I better put i and make this into a true equation, i sub v. This is going to give me the values u sub 1, u sub 2, u sub 3, u sub 4 and u sub 5.
So u sub 2 is just v sub 1. u sub 4 is just v sub 2.
No problem.
And what do I do with the other ones?
Let's see.
I guess u sub 1 is halfway between v1 and 0.
OK, so it's 1/2.
u sub 1 is just 1/2 v1 It doesn't involve v2.
u3, which is sitting here is 1/2 of these so it's 1/2 of this and 1/2 of that where this was nothing.
Then u sub 4 was the same as v sub 2, just take v sub 2.
And what's u sub 5?
u sub 5 is 1/2 of v sub 2 and 1/2 of zero.
So it's a simple matrix.
The interpolation matrix.
It's just reasonable to use the letter I and we all recognize here it's a rectangular matrix, not the identity.
I doesn't stand for identity here, it stands for interpolation.
OK, so I hope you can focus on the matrix.
And if it was much, much bigger, but still in 1D it wouldn't look very different.
Each column would have 1/2, 1, 1/2.
That's a typical part of the interpolation matrix.
It uses the value v sub 1, saves it at the center point, uses 1/2 of it at the previous point, and 1/2 of it at the right-hand point.
And similarly, you see it's sort of a typical rectangular matrix in signal processing as well.
I have to say something about what happens in 2D because our problems are in 2D.
Maybe I just draw a typical 2D mesh.
These are the coarse values, this is the coarse mesh and now of course, we'll save those values.
I will have a 1, so it'll save those values when it goes to the fine mesh, the fine mesh being 1/2.
So I just have to decide what to do there.
So question, what value shall I take?
Oh, I suppose I have to decide there, too.
These are all new points and what shall I do.
I'm just going to stay with linear interpolation, which is quite fast and satisfactory.
This value will be the average of those two.
This value will be the average of those and what will this one be in the center?
It's the average of these four, of course.
And that's what we'll get.
Actually, that Kronecker product business, a Kronecker product of a 1D matrix like that with itself would be a 2D interpolation that would do exactly this.
We can write it out more fully, but it's not fun to write out really fully because if this is 5 by 2 then I 2D would be 25 by 4.
So we're right away, even on this tiny, tiny problem, we're right away up to 25 internal mesh points on the fine mesh.
That's the matrix that I want you to see and the natural choice for R is to transpose.
So one possible choice-- and it will, the answer will be yes, we do use [?
Naver.
?]
So R will be the transpose of I.
Actually, I'll need a factor here.
I think it's 1/2.
Let me just see what it is.
Let me see why we need a little multiple there to make things right.
So there's my matrix I, so I'm just going to transpose it and let me factor 1/2 out of that so you see these matrices without fractions inside.
That would be 1, 2, 1.
So I'm going to transpose that.
1, 2, 1, zero, zero.
Zero, zero, 1, 2, 1.
So I factored 1/2 out of I when I did that.
And then I believe that I need another 1/2 here.
I think I need 1/4.
And what do I mean by needing 1/4?
Of course, the main point is that this is the transpose of this.
And you'll see that that preserves the symmetry that's just sort of the natural thing to pick.
Not the only possibility, but it's very good to connect the restriction with the interpolation.
Of course, you can't take the restriction to be the inverse of the interpolation.
Why not?
I mean that sort of mental idea is that the restriction goes in one direction and the interpolation in the other direction.
So why not just let one be the inverse of the other?
Well of course, that matrix doesn't have an inverse.
I mean it's rank is only 2.
When we do a restriction we're going to lose information, the interpolation can't put it all back because we threw it away when we restrict it to the coarse mesh.
Now why 1/4?
I think it's just if we had all constants, like all ones as our mesh values, we would want to get all ones for the restriction.
So if I multiply this matrix by the vector of all ones-- so assuming you remember where R comes in by just copying this equation, you remember that when I go from the h mesh, it's fine to coarse.
So R times ones on the fine mesh, all ones on the fine mesh gives ones on the coarse mesh.
That's why I wanted that-- you see, if I multiply by all ones then that multiplication gives me 1 plus 2 plus 1 equal 4 and I need to divide by 4 to get back to the 1.
You quickly discover that that just arises because of a change of scale.
It's just a factor of 1/2.
In 2D that 1/2 there would change to 1/4.
In 3D it'd be 1/8.
It's just the scaling to preserve constants.
And now the remaining question is, what's A 2h.
That's the other matrix here that we're not given.
We're given the A sub h, our problem.
We choose an R and we choose an I and if we're smart, ones that transpose of the other.
And then the question is what's A 2h?
And there's a beautiful answer, so let me just say what it is.
A 2h, a coarse mesh matrix is the fine mesh matrix, but I need to do first-- if this is going to apply to the coarse mesh guys, the v's I have to do an interpolation.
You'll see it here.
This is 5 by 5 in my problem.
So I need to first do the interpolation to get from a vector blank 2 up to a vector blank 5.
Then this and then the restriction.
That's terrific.
That's A 2h.
So this is A 2h and we'd better do an example.
I guess it's going to be next time.
So that'll be the first thing for next time, would be to see OK, for the standard second difference A sub h, for these piecewise linear interpolation and restriction, what comes out as A 2h?
Do we get the standard second difference on the coarse mesh, which you really hope we do and we do.
This will be the same difference in our example on the coarse mesh.
So that this middle step of multigrid is exactly what we would have expected.
What we woud've had if we set up the problem originally on the coarse mesh.
But now we're going to get an answer that we take back to the fine mesh.
OK, so I'll do that next time.
Some homeworks here that I held for an extra two days and now I'm returning and some thoughts to go up on to the website about possible projects, but actually, I expressed a lot of them at the beginning of the lecture.
OK, thanks.
Good.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
Ready to go on lecture five, I guess this is.
So this -- we've done the one way wave equation, first order equation and now it's just natural to see what about the ordinary second order wave equation and on this board, I've written it two ways.
I've written it as a second order equation, so second time derivative appears.
That's going to make a change.
It will have second differences in time, so we're going to have more steps, and of the same matching second difference in space.
That's one way.
Then this is another important way to write the same equation as a first order system.
So I can get back to first order systems and I can use all those methods -- Freidrichs, Lax-Freidrichs for example, Lax-Wendroff, whatever improvements we think of.
On this first order system, d by et of a vector is a matrix times d by the x of a vector.
So that's -- you really have a choice here.
Written this way, we'll notice a little bit of a good thing, one good thing here.
The matrix that shows up is symmetric.
That was by the careful choice of putting a c in there.
Then you see that the first equation in this system is utt, is c time cuxx.
So the first equation is the wave equation and what about the second equation?
This is typical when you reduce things to a lower order system, that you get an equation here which is just an identity.
The derivative of c -- the time -- this is cuxt.
It's the derivative xnt from there and we have c times the x derivative of ut, the cross derivative.
So that is an identity because utx is the same as uxt.
We can take time and x and t derivatives in either order for a function of x and t. So we'll see that.
I guess what I want to do in the lecture is not to repeat Lax-Freidrichs or Lax-Wendroff or this system.
That would be unexciting.
Let me just mention, what are the Eigen values of that matrix?
So the Eigen values -- for a system, it's the Eigen values of the matrix that tell us what waves we've got, what speeds they go, where it -- when it was 1 by 1 of course, it was just that single number c, but what are the Eigen values of that matrix?
Now having made it a symmetric matrix, we know right away it has real Eigen value and so it's a wave problem.
For a heat equation, we would see something different, but this'll be a wave equation and this has two Eigen values and they are c and minus c. You could quickly check that that 2 by 2 matrix has Eigen values c and minus c. They add up to 0, which is the trace.
They multiply c times minus c is minus c squared, which is the determinate.
So that's -- they're the right Eigen value.
So that's going to tell us what we'll find every way, that the speeds of the waves are c and minus c, which means a wave going one way was speed c and a wave going the other way was speed c. So it's a two way wave effect and this is the equation of real -- that really happens and I wanted to say just a little bit about it -- just two words about real problems, because ah we can't, in these first days, tackle the difficult problems of electromagnetism, antenna design, all kinds of wave problems.
I guess I'm hoping some of you may be meeting real wave problems in other courses and would make a project that tries our methods on real problems.
I just wanted to write down here one sort of typical real problem.
So in a real problem, we have a variable coefficient x, because maybe our material is not homogeneous, is not uniform and we have a forcing term -- I just put f for the amplitude to emphasize that's forcing term, so f standing for forcing, but it's an oscillating forcing term, but this is what you're going to have.
In electromagnetism, the frequency is going to be very, very high.
So this is a problem that's not easy numerically.
Very, very high frequencies would normally require very, very small wavelengths, very, very small delta x, because you remember there'll be a k delta x in the error.
So we took a big delta x or even an ordinary size delta x and let the frequency go way, way up, high, high frequency oscillation, our k delta x would be big and the accuracy would be poor and the results useless.
So a lot of -- I'm really -- guess I'm thinking here about a course on wave propogation would study this as a first model problem, do some analysis and then go into 2 d and 3 d. That's the reality of wave theory, is you can get asymptotic results for k going to infinity.
You get things -- you're looking in matching powers of k, so to speak.
You're getting plane waves and interesting stuff that we can't do already, right away here.
So I'm going back to the ordinary wave equation, either second or -- start with second ordinance.
utt is c squared uxx.
That's our goal.
As I said, we don't want to see Lax-Freidrichs and Lax-Wendroff again.
We know those.
It's rather -- so one new method is going to show up and it's -- leapfrog is the right name, the natural name to give it.
Leapfrog because in the time direction, we have time n minus 1, n and n plus 1.
There's a leap there, but let me start with semidiscrete.
We haven't done this until today.
Semidiscrete means, as you see, that the time variable stays continuous.
We have ordinary differential equations, nt, but the space variable is what's getting discrete.
so the problem is half discrete -- discrete in x.
There's a delta x, but there's no delta t yet.
So it's natural to analyze this problem.
So what do we have, really?
In the x direction, we have a mesh of with delta x.
So we have unknowns.
If that's 0, j is measuring.
So j delta x is the typical point.
Point number j along there and then in the time direction, we're going continuously.
So I don't have discrete steps, so it's called -- because those are lines in the time direction, this is -- semidiscrete is also referred to as the method of lines.
So the method of lines is what we're doing here and it's -- all I want to do is what I always do.
Take exponentials, look at their growth factors, understand the equation.
Actually, I better do it first with a wave equation itself.
So I guess my organization for today is, first step is follow either the ikx for the wave equation.
So I'll do that here.
utt equals c squared uxx and I'm going to look at u proportional to -- and I'll call this factor g again -- g of t and times the frequency ikx.
So I'm going to look for pure exponentials and will quickly find them.
Then I'm going to do the same for semidiscrete, where that x becomes j delta x and I'm at point j.
Then I'm going to eventually look at the discrete, now fully discrete, discrete in time and discrete in space and the question of stability arises.
Accuracy we can -- you pretty much -- we pretty much know the accuracy for second differences, that's a second order accurate thing because it's centered, but stability is going to come in.
So that's -- this is our first step.
Then I'll just mention what other point remains in this wave equation, is to look at the first order system.
How does that look in space and time?
As I said, the obvious thing to do would be Lax-Freidrichs or Lax-Wendroff, but there's another idea and it brings in a staggered grid.
You'll see it.
Somehow that staggered grid is really an important idea that we haven't met yet.
That some unknowns are on the standard grid, ujn, but the other unknowns are on a grid that's half a delta x and half a delta t different.
It's just right, actually.
So that's a premier method in electromagnetism.
So that's what's coming, but let's start with the second order method, because we haven't seen the second order method.
Second order equation -- plug in g. So I have -- the second time derivative gives me g double prime times either the ikx -- I'm separating variables, of course; t here and x there.
On the other side, I have c squared.
Now I have the x variable, so that brings down an ik -- so that's ik squared times ge to the ikx.
Good.
So simple, but substituting this is just a natural first step to see what's happening.
Then of course, canceling e to the ikx leaves us with the equation for the g. g double prime is -- I'll make that i squared minus c squared k squared g. Of course, the new aspect is it's second order.
So it's got two solutions instead of just one.
It's constant coefficients, of course, and so we look for exponentials and the solution then would be -- two solutions are some -- there's an e to the i -- I guess I'm bringing back the i -- e to the ickt e to the ikx and e to the minus ickt e to the ikx.
Those are the two possible gs.
Those are if we take their second time derivative, we get the same as taking the second x derivative.
We get that factor c is in the right place.
You'll see it.
The second time derivative brings this factor down, just as we wanted, but the point is that there are two of them.
So this I can write as e to the ikx plus ct. That's our old friend.
This one I can write as e to the ik minus ct, which is our new friend, the new way.
The way that's going to the right, so this x plus ct follows the characteristic lines to the left, just the way we had in the one way wave equation.
This x minus ct stays constant on characteristic lines going up to the right, that it travels with speed c and the signal travels along those characteristics, was speed c. So that initial value now -- maybe I draw a picture over here -- so here is t equals this -- is x as always.
This is t equals 0.
I have some initial value u at 00 and I also have an initial velocity, because I'm in -- got a second order equation, but information somehow, pure exponential information travels along characteristics this way -- that's the new one -- and characteristics this way -- that's the old one.
This is time space.
I'm graphing the two characteristic lines now.
Minus ct equals -- if it starts from 0, then it would be 0.
This is the x plus ct equals 0, the one we had before, the two characteristic lines.
Waves are going both ways, right?
Now, what we were able to do with the first order equation -- we even got a completely explicit solution to the one way wave equation and we can do the same for the two way wave equation, so I better write it down.
I better write down the solution.
So let me just continue this same idea.
The general, the complete solutions will be u of x and t is some function of x minus ct and some function of -- call it f 2 of x plus ct. Now I'm taking the jump, the jump from one exponential, a particular k that gave me these particular guys, to all exponentials and assembling all exponentials into functions.
So why is that OK?
It's only OK in this because of this very special situation that all frequencies are behaving the same here.
All frequencies have no dispersion.
Dispersion is going to be different behavior of different frequencies, so that's an important fact, which we don't face here.
There will be just dispersion in the discrete methods, the semidiscrete and the discrete method.
Different frequencies will travel at different speeds and that's what produces the oscillating error.
Whatever shape the error has is because different ks are doing slightly different things, as we'll see when we do the discrete ones.
But in the continuous case, every k, we have the same travelling either with speed c or minus c and therefore, we put them all together and we have a whole function traveling with speed c or minus c. Now we've got to functions and we have to do we have to match two initial conditions.
We have to match a u of x and zero and the time derivative has to match a ut of x and 0, so these are the initial conditions.
You just match them up and you find the answer.
So you discover what these functions really are.
It turns out that, I think, we had -- you remember before, we had a u of x plus ct and 0?
That was everything in the solution to the one way wave equation.
Now we will have u of x minus ct 0 and this divided by 2 -- so there you see a wave's going each way and the division by 2 makes us -- now we're matching the initial condition, so that would be the total answer if the equation, if the body starts from rest with 0 velocity, but I haven't matched -- so I've only matched the udt equal 0 -- think the time derivative of that would be 0, because the time derivative of this would bring out a c, this would bring out a minus c and a t equals 0 -- they've cancelled, so I need another term and it turns out to be -- I'll just write it down.
It's got to depend on x minus the ct and x plus ct and it's this -- it's the integral of this velocity dx.
I won't -- there's no reason to take time to derive that.
I think it's right and could check, [?
right?
?]
So what this means is that the initial velocity is -- it's entering somehow all along -- somehow the initial velocity is affecting things within this cone, this characteristic cone that's defined by the characteristic lines.
In two dimensions, that cone will really look like a cone.
It'll look like an ice cream cone -- or maybe that's in three dimensions.
Whatever.
In other words, if the initial condition was a delta function at the origin, you could quickly see what the solution would be, and if the delta, if the initial condition was a step function at the origin, what would happen then?
Suppose I started with this step function at the origin.
So, like, a wall of water and suppose it's at rest.
So nothing -- at that moment t equals 0, I remove the dam and the water starts -- the water flows.
Then this term is 0 because there was no initial velocity and this term is just half of that wall goes one way and half goes the other way.
The notes draw a picture of that, so that -- so that's -- I've kind of quickly disposed of the solution of the wave equation and of course, that's too easy.
This is -- I mean, that's good to know and good to compare when we meet real problems where c depends on x, where there's a forcing term, where there's a real complication.
But it's just pace to stay with the model problems at the beginning.
So I'm just going to stay with the model problem and move on to discrete, so try to understand -- let me try to understand what happens if I make the space variable discrete, right?
I guess I should say, another step toward reality would be, have some boundaries.
Here, this is on the whole line, x from minus infinity to infinity, so reality would be that there would be -- this picture would have some boundary, maybe there at, say, minus l and l and some boundary conditions there, on that boundary and that -- so now we have a finite problem.
That means the calculations are fine, but it means some new effects are coming in.
Waves can bounce back off the boundary, partly.
they can go out, be lost, go beyond or disappear, depending what the boundary conditions are.
So that's another part of reality, is boundary conditions.
I'm probably suggesting then, some more realistic numerical experiments.
So a realistic numerical experiment would be, put in some boundaries, make this a finite system, so it's at 0.
Let's just have a finite number, stop there.
This would be one boundary, this would be the other.
Then I have just seven unknowns in that problem.
That'd be a seven by seven matrix over there and it would be the second difference matrix that is so fundamental in all of scientific computing.
So we have here a second difference matrix, but it's infinite because I haven't got a boundary right now.
So the fact of not having a boundary means Fourier is ready to go.
So I'm going to plug in -- again, I'm going to try u.
This is capital u now, because I've discretized something.
I better write this in a form where we see the j, so can I write this equation just again for u number j -- so tracking up this line, for example, or this one?
So I'm tracking up this line -- do you see that u is now a system?
In this case -- let's see.
I shouldn't have put this one in so that I'd have 1, 2, 3, 4, 5, 6, 7 -- that's good.
So my unknowns are uj tracking up that line.
Each uj is going in the time direction, so this equation is really the time derivative.
It's an ordinary derivative.
I can write d and not partial derivative, because the only derivatives here are in time of uj is c squared over delta x squared times uj plus 1 minus 2 uj plus uj minus 1.
That's my equation.
My system of equations, sorry.
My system of equations.
For the moment, I'm thinking j goes from minus infinity to infinity.
I'm not going to put in a boundary, but what I will do, of course, is try uj some growth factor that -- j is a space, discounting space steps, so the growth factor is in time and it's continuous in time, because time's continuous, but it's always e to the ikj delta x.
That's the right thing to plug in.
I mean, I guess what we're -- what I'm really -- the reason for really doing this is just to reinforce the idea that trying a pure exponential is a good thing to do.
It's even a good thing to do with variable coefficients and real problems.
It's absolutely a good thing to do with these model problems.
Plus it in and what happens?
So I get g double prime times the exponential, but of course you know in advance, I'm going to cancel that exponential -- is c squared over delta x squared times -- what goes there?
What will cancel the exponential -- so I'm imagining that exponential as far out, but here it is at j plus 1.
So I've incremented j by 1, so that is going to give me an e to the ik delta x, right?
Here I haven't incremented it and here I've incremented it by minus 1, so that will give me an e to the minus ik delta x, all times the same exponential which appeared there and appeared there and got cancelled.
So that's it there.
g double prime -- oh it's got to be a g -- right, there's a g. Right.
When I plug it on the right hand side I got a g that was so uninteresting, I lost it.
What does -- what's g there?
I guess the main point to see is, this quantity, which is divided by delta x squared, which is coming because we have differences instead of derivatives.
When we had derivatives, that quantity was c squared ik squared 4.
In other words, minus c squared k squared.
this was the correct factor and this one is the discrete factor and it's worth just paying a moment of attention to compare the two.
Where am I going to write that comparison?
I guess that's going to be on the board that's hidden, so I'm going to -- can you try to remember that?
Let me focus just on this quantity for a minute or even just on this one, just because we see it so many times, because it's coming directly from second differences.
For the moment, let me take k delta x. I'll just call it theta.
It's some angle.
So can I just do a little bit of trig with this factor?
2 minus 2 cos theta.
Let's see.
I'm going to, I'm going to bring out a minus sign.
So I'm going to put plus signs on those and minus signs on those.
You see the main point of course, that e to the plus something, e to the i theta and e to the minus i theta combine to give 2 cos theta.
Right, so that's the key simplification that's going to make this much neater.
So I have 2 cos theta minus two.
I have 2 cos theta minus 2 and if I bring out a minus sign, I'll have 2 minus 2 cos theta and that's what I want to write.
I want to write 2 minus 2 cos theta.
I want to see what that expression is.
Right, so that brought out a minus sign just to match that minus sign and the c squareds match, but what's different is this k squared is now gone into this 2 minus 2 cos k delta x. I'll write it divided by delta x squared.
You see that this is -- that's the change.
k squared, which is a positive quantity became this, which is also a positive quantity.
2 minus 2 cos theta, but it's not a pure k squared.
We are getting dispersion.
Different frequencies are showing up inside the cosine and that's not a linear function.
So we're seeing different frequencies going their own way in a more interesting way then just what we saw here with k squared.
Of course, when we take the square root as we did here, that produced linear in k and no dispersion and something simple, but here when we take the square root -- let's get ready to take the square.
How can we take the square root of that?
By writing it as a square.
It's never negative, and if I use a little trigonometry -- so I'm just going to write it down.
This is 4 -- I think it's sine squared of theta over 2.
That's just a handy fact.
It's handy because I can take its square root and really see what -- so theta -- remember this -- theta is standing for k delta x.
So then I want to divided it by delta x squared.
Can I bring down that quantity just so you see it again?
So this was the 2 minus 2 cos theta what with the minus sign, and dividing by delta x squared and that's -- so it was g double prime -- let's just write it.
So I have g -- I'm now going to copy what -- g double prime wasn't a minus sign times the c squared times this -- 4 sine squared of k delta x on 2 divided by delta x squared.
Maybe I'll put the delta x squared over there, where it kind of belongs.
Times g, always forgetting g. So we're just following the golden way here, plugging in an exponential and seeing what happens and what happens is pretty nice.
We now know what the solutions are.
There'll be two solutions.
It's a linear problem, so the solutions will be exponentially in t. The solutions will have a plus or minus, because it's a minus sign.
So g -- can I write it this way?
g of the solution to this -- the two exponential solutions will be e to the plus or minus -- now I'm going to take this square root i times c times the square root of this.
Actually, that's not so hard.
That was our point, wasn't it?
Sorry, I forgot about that.
Sine k delta x over 2 divided by delta x -- and look, here's something nice.
Bring the 2 down there.
Make it 1/2 of the denominator.
Do you see what this expression is?
Actually, do you know its name?
It's the sync function.
It's the sync function, right?
For very small delta x -- times t, right?
It just -- we just have an exponential.
Of course we do.
This was a constant coefficient problem in t, so in a second order, I have two exponentials and I just took the square root of that coefficient, put it there, times t. Absolutely straightforward.
This quantity is the is the key one and this is the quantity which, for small k or small delta x, is approximately what?
So if k and delta x are small, or even just delta x is small, what does this sync function look like?
What is that ratio approximately?
1 k is small.
What does the sign of theta look like when theta is a small angle?
Looks like theta, right?
Sine theta is about theta.
So this thing is about k delta x over 2 a divided by delta.
It's k, so it's approximately k for a small delta x, let's say.
Delta x is small.
That's the key thing to know about the sync function, that it begins by approximating the sign function, but then of course, when k delta x gets large, it wanders away.
It's not linear nk.
See, it starts out practically linear nk and that k is exactly the right k. That's the right factor.
That's the k here.
But in the differential equation, it stayed k and in the difference equation for k delta x moving away, it changes and that's why -- of course, that's the source of the error.
We're seeing now why the method is probably second order accuracy.
We know that these second differences are second -- give second order accuracy.
This would be -- if I looked at the next term, what's the next term in sine theta?
Sine theta starts out theta and then -- what's the term after that?
Everybody's Taylor series or -- [UNINTELLIGIBLE] theta cube, it's a theta cube term.
Exactly, it's a minus theta cube over 6 three facotorial, but it's a theta cubed term and so the theta gave us the right thing and then two more powers of theta will give us a second order error, right?
I better move since -- so that's the story for semidiscrete.
Semidiscrete produced a pure exponential, but with a phase factor, you could say, that wasn't linear in k where the real one was.
Now what about -- let me move up to that board above, where now we're going to have -- we have it -- it's discrete in time as well, so we're going to have single steps.
What's going to be the difference?
Again, I'm going to try -- you know, I only have one idea here -- ujn will be sum g. Some growth factor times e to the ijk delta x.
The same -- I'm going to try an e to the ikx and in the space variable, it will be g. At After n steps, it'll be g n times.
So this is the discrete, right?
This is the ansatz, to use a fancy word.
This is what you plug in.
Little crazy to use the word ansatz, which sounds fancy and the word plug, which is far from fancy.
I should say substitute.
Substitute in it.
Alright.
Do we know what the process is going to do?
When I plug it in -- sorry, substitute it in on the left, we get -- and I'm going to cancel this from every term -- so all I'm interested in is the -- maybe I'll cancel g to the n minus 1.
So let me write down what I get.
This is going to give me -- in the time direction, I'll have a g squared minus a 2 g plus a 1 over delta t squared and in the space direction, which is at the centered time, so it's going to multiply a g, it's going to be my same, my very same thing.
I could maybe give it a name here.
I don't know whether -- should I give it a name?
I know what it is anyway, because it's the same thing that I had in the semidiscrete method.
It's this -- let's see what it looked like.
At this point, it was -- look, let's do one thing.
Let me get this delta t squared up there times the c squared divided by the delta x squared.
Let me just clean up those constants, so that when I multiply up by the delta t squared times the c squared divided by the delta x squared, what's that?
c delta t over delta x squared is my old ratio, the Courrant number r squared and then it's this times g. So this will all be just exactly the same and that's what I simplified here.
Let me just leave it as -- with a minus.
Let me remember that I did take a minus so that I could write it as 2 minus 2 cosine of k delta x times g. Is that -- I really think I have kept track of everything here.
The r squared coming from all this stuff, the minus 2 coming here and the plus 2 cosine coming from there.
You OK with that?
But there's one new ingredients here.
Again, g to the n -- I cancelled g to the n minus 1 times this from every term.
So the new ingredient, that we just have five minutes to deal with, is the fact that we have second -- g squared just showed up.
So why did g squared show up?
Because it's a multistep method and therefore, I've got two gs.
Of course, I had two gs in all these other cases because I had second derivatives and now I have second differences.
So I have 2 g, so I have to solve for g and what is stability going to depend on?
Stability is going to be whether g -- both gs, because there are two of them now -- or lesser or equal to 1.
That's what I have to check and that's the equation.
What do we figure?
We figure that if r is pretty big -- if r is big, no way.
If r is big, the Courrant test will fail and stability will fail.
But if r is small, then I expect -- I have a right to hope for stability.
So it's going to be the size of r. So I just have to -- I've got this quadratic equation.
I'm going to bring this thing over here.
I'm going to write this as g squared minus 2 ag plus 1 equals 0, where a is this quantity and minus 2 a, so a is a 1 from there and when I bring this over to the other side, it's going to be a plus r squared -- factoring out -- notice the 2 is everywhere.
So I put the 2 here.
The a comes from the 1 there and from this times the r squared 1 plus r squared minus r squared cos k delta x. I'm just doing algebra and it's coming out nicely.
It's coming out to a nice equation here with a slightly messy expression for a, but if I know that I've named this quantity a, I can solve this.
So what's the solution?
I remember the quadratic formula and I get g as 1, I think, plus or minus the square root of 1 minus a squared.
That's g. Maybe this is an a; probably is.
I guess the quadratic equation takes a little memorizing, but I guess the first point is that that coefficient shows up there and then the square root that we know.
So what's up?
Again, none of this is deep.
I'm just following my one way path.
Check stability.
Check the size of g. Get an equation for g. Get a formula for g and look at it.
So a is some positive number and the key is going to be whether a is bigger than 1 or smaller than 1.
If a is -- is that right, 1 minus a squared or should that be a squared minus 1?
Is it a squared minus 1?
Professors are not responsible for the quadratic formula because we proved that we could do it in third grade and lost it since.
I guess it's right.
Is that right?
Because look, here's the story.
Suppose a is bigger than 1.
Suppose -- I'm really looking at the roots of this equation.
Notice, by the way, the roots of that equation multiply to give 1.
So I'm on the edge here.
Either both roots have size 1 or one of those roots is too big.
The test is going to be, is a greater than 1?
a greater than 1 will be unstable.
a less than 1 will be stable and I'll connect a less than 1 to r less than 1.
That will give me r less than 1.
Alright.
I've given away the result because time is pressing.
Maybe just take the remaining time to see this.
Suppose a is bigger than 1.
I'm just looking at the roots of this equation and here they are.
If a is bigger than 1, like 2, I have 2 plus square root of 3, I'm way up, right?
I'm bigger than 1, no question.
Suppose a is less than 1. suppose a is 1/2.
I have 1/2 plus or minus the square root of -- what?
What's up here?
If a is smaller than 1, this is negative and if it's negative, its square root is imaginary.
So this is good.
I have a real part and then an imaginary part and then on some magic little bit of board, I going to add their squares.
So I just want to take the real parts squared and the imaginary part squared -- now the imagining part is going to be 1 minus a squared because when I bring out an i, then I have a 1 minus a squared there and when I add them, I get 1.
So let me just say again what it is.
I just figured out what is absolute value of g squared as real part squared and imaginary part squared and I got the answer 1.
So the conclusion is, the roots are stable when a is below 1 and I have instability when a is above 1.
Then if I track that down, that reduces to the tests -- it just happens to be the same test on r, whether r is below 1, which is the Corraunt-Freidrichs-Lewy condition that I'm staying inside the characteristics or r is bigger than 1 and I'm taking a delta t that isn't stable.
I'm sorry to go slightly over the time.
Thanks for staying with me.
I'll began next time with this staggered grid.
That won't take the whole lecture, but it's worth knowing because we see staggered grids in many of the best methods.
Thanks.
Have a good weekend and I guess it's Tuesday rather than Monday that I see you next.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So I hope in time for you to consider this material also in thinking about projects because there are lots of experiments to do with these.
I did a MATLAB experiment that I'll mention.
And I guess it was too small because it didn't make multigrid look as good as it is.
So can I remember the steps of multigrid and then you'll remember why we take those steps and then make a sort of basic analysis of how do you decide whether this multiple step process is successful?
How much does it reduce the error?
So you remember the steps.
Oh, thanks.
So the first step is really a little bit outside multigrid.
It uses an ordinary iteration like Jacobi, a few times and so you remember that the matrix that governs this ordinary step, the matrix that's crucial for this is the-- so what multiplies the error at every step.
So we have initial error and if we take a smoothing step with a preconditioner, p, then the error is multiplied by i minus being p inverse A.
So that's the matrix.
That's sort of that one step, one iteration matrix for ordinary iterations where A is of course, we're solving A sub u equal b.
That's our problem.
And matrix p is close to A.
Of course if we took p equal to A then the error would be reduced to zero right away because we would be solving the system one shot.
But now, we're left with some error and this-- steps 2, 3, 4 reduce it further.
And well, our hope is to see that this multigrid operation reduces the low frequency part because this one will act successfully on the high frequency part, the oscillating error, but for smooth error the eigenvalues here are too close to 1 and it's too slow.
OK, so I was just going to track down-- well, first have to remember, what are the pieces here?
So there's a matrix R, restriction matrix.
A matrix A sub Sh, which is original matrix A or A sub h. I could call that.
But the original one is on fine mesh.
So I need a coarse mesh version.
So that's what I solve on the coarse mesh, the smaller problem and then interpolate back with a matrix I back to the fine mesh.
OK, so can I just remember also what those matrices-- we have all the pieces here.
So if I remember what R and I are-- OK, so let me write down all those pieces.
So A-- let me tell you the MATLAB experiment I ran.
So A is our two's and minus one's matrix.
Our second difference matrix in 1D to keep it simple.
And it should be divided by h squared.
Well, I guess we took the mesh where the coarse mesh just had two interior points.
The value at the boundaries was zero and the fine mesh has 5 [INAUDIBLE].
So this will be 5 by 5.
Then we needed the restriction and interpolation matrices.
So this was R, the restriction.
So the restriction, which we're going to use right away will take us from the fine to the coarse.
So that matrix will be-- it'll multiply things with 5 components and produce an output with only 2 components.
Can always find either of those mesh points and their numbers were 1, 2, 1, 0, 0, and 0, 0, 1, 2, 1 and needed to divide by 1/4 because those rows added up to 4.
The idea being that if we restricted all ones we should get all one and we need that 1/4 factor to do it.
And the interpolation went the other way and it was essentially the transpose.
That's the nice connection to have and again, we need a factor-- and maybe the factor is 1/2 there.
Good.
We've now got all the pieces except A sub 2h and I think just at the very last minute last time I said what A sub 2h should be.
And the good choice for A sub 2h-- so that's on the coarse mesh.
So that's going to be small matrix.
So the good choice is I, which puts it up on the fine mesh, A sub h, which is our operator on the fine mesh and then R that restricts back.
So R, A, I, is the right construction, it's sort of the natural construction for the coarse mesh operator.
And notice, I don't know if you appreciate having this, this is the same as R, A, R transposed with some factor so that it'll come out symmetric, it'll come out positive definite, and actually, it'll come out in this example-- it will come out exactly the second difference operator on the coarse mesh.
So this will turn out to be 2, minus 1, minus 1, 2.
The 2 by 2 operator and since we're on the coarse mesh the step size is 2.
I won't do that multiplication, but MATLAB confirmed it.
So that's where we were.
So I've sort of done now a recap of multigrid and I'm ready to tackle a little understanding now of what happens to the error.
So at this stage I end up with some error.
U sub h is not the exact answer.
I have an error e-- eh, which is u minus uh.
That's the error at the end.
Now the question is, what happens to that error?
Can we see how and why it's reduced as we do a v-cycle, as we do a 2, 3, 4 cycle?
So since everything's linear that error is multiplied by some matrix that we just have to track those steps, see what that matrix is, and look at its eigenvalues.
So that's my goal here, is to find the matrix, call it s that takes a step, a multigrid step and what does it do to the error?
OK, so can I just follow that step?
Here's the key matrix.
Maybe I'll write it underneath.
So what happens to the error?
First of all, let me write the steps down.
e sub h then is u minus u sub h. That's entering multigrid.
That's my error.
OK, now I compute this residual, r sub h. I want to say that it's Ae sub h. Can I check that that's true?
Yes.
A times this is A times u-- that's the b minus Ah times u sub h. That's the second term.
So that's OK.
So that's what happened there.
Then it got restricted.
I'll keep a record of what happens to e sub h. So it's multiplied by A to get the residual.
We work with the residual and with b because b is the input we know.
Then at this step two, that residual is restricted by R. Now we've got r sub 2h.
When I multiplied by R I got r sub 2h.
OK, next step.
To get to E sub 2h.
So what do I do there?
I multiply by the inverse.
I solve this system, so of course, I don't actually find an inverse matrix, but that's the formula.
To get this I multiply the inverse matrix by that, so I've got an inverse matrix here.
So I write it as A sub 2h inverse.
This was A sub h. Now when I've multiplied by A sub 2h inverse I'm this far and the final step, multiplies that by I.
So there you go.
It looks pretty awkward, but that's the matrix, s, that produces E sub h. So I'm thinking that my matrix is going to be called s. This matrix here is going to be called s. So let me write down what s is.
All I want to do is remind us what A sub 2h is by this choice there, it's RAI.
And then comes R and then comes A.
A is now A sub h. Well, we've got a matrix.
And for this example I can compute what it is.
Can we say anything about that matrix from its formula?
Looks a little messy.
But of course, it's built out of these pieces.
It's built out of the given matrix A and our R and our I.
And there is a neat property hiding there.
Which you can see if you square that matrix.
So look at s squared.
If we look at s squared, so this is the matrix to understand.
And I don't claim to understand it as well as I would like, but-- so this is the matrix for these three steps.
We would have a full multigrid v-cycle just before I lose the track on that.
A full multigrid v-cycle would do M a few times, say twice.
Two smoothers, then it would do a v-cycle and then smooth again.
Well, I should've said the smooth again would be the one on the left.
This is the original, so there's two smoothers followed by a multigrid cycle, followed by a couple of smoothers.
That would be our total overall matrix, but I want to focus for right now on this matrix, s. And I guess, I hope that anybody who does a project or a homework on this topic, you're going to create I, R, and A and compute s. And it should show what s squared is.
So could I compute, could I look at s squared?
You'll see why.
So I just write s down twice.
RAI inverse RA.
So that's once, now repeat the whole thing.
I, RAI inverse RA.
I have multiplied s by itself.
And what do you see?
Well it's nice.
If you look at this you see that we've got this combination RAI sitting in the middle because when I did it twice the RA came from there and then an I was at the start, so this is-- so what's left?
S. This matrix S is equal to its own square.
S squared equals S. You know, we're able to see this without-- this would apply for this example, but for all examples.
Now I'm ready to think about eigenvalues a little bit.
What does that tell me about the eigenvalues of S?
So this is a small class of matrices with this property that s squared equals s and maybe I say something about those here.
So if s squared equals s, what does that tell us about eigenvalues of that matrix?
It's a square matrix.
It doesn't happen to be symmetric.
Very often we're looking at symmetric matrices, but this is a little lopsided.
We have an A off on the right.
I might be able to make it symmetric by bringing part of that A off to the left, but this is the main point.
What are the possible eigenvalues of that matrix, S?
Well, zero, 1, 1.
The only eigenvalues are zero and 1.
Why is that?
Because if Se equals lambda e, so I'll put that word if in.
If Se equals lambda e I could multiply by S again so I would get S squared e equals lambda Se and that's lambda squared e. I haven't done anything fancy here, all I'm saying is that any matrix, if it has an eigenvalue of lambda and an eigenvector e then S squared has that same eigenvector and it's eigenvalue is lambda squared.
But now S squared is S. So this is the same as Se, which is lambda e. You see that I now have lambda equal lambda squared.
Sorry, I probably made that take more time that it had to.
If S squared equals S then lambda squared equals lambda.
So I have lambda squared equal lambda and the only two possibilities-- that's a quadratic equation that's got two roots and those are the roots.
If lambda squared equals lambda then lambda could be zero or it could be 1.
So what does that tell me?
That tells me that the eigenvectors that have eigenvalue zero, what happens to those in multigrid?
They're killed.
An eigenvector with an eigenvalue of zero means that Se is zero.
So whatever error we started with one cycle through multigrid will remove it.
These are in the null space of S. They're vectors that S completely removes and these are vectors where Se equals e. The eigenvalue is 1 and multigrid does nothing.
Well, one part of the message then is that I could use multigrid-- well, I guess I hadn't thought about this, but one message is don't repeat 2, 3, 4.
That would be a large waste of time.
If you just repeated-- if you just did another v-cycle and didn't do any smoothing or anything in between, then the error that came out of the first v-cycle would survive right through.
Nothing good would happen in the second cycle that didn't already happened in the first.
Now and another message is of course, I could do multigrid forever and I'm not going to get convergance.
Actually, that's sort of interesting because we're totally accustomed to working with matrices M, whose eigenvalues are spread out, but below 1.
This has its eigenvalues not spread out at all, zero or 1.
That's it.
I'd better say quickly that the key matrix here is I minus S. Just the way that it was I minus p inverse A.
So the multigrid matrix-- yeah, maybe I could've messed up.
The matrix is really I minus S, so let me be sure I see where that happens.
I guess I got down here to figure out what E sub h is and then I didn't do anything with it.
I forgot to write in the key step that at the end of step 4 when I have a piece of the error that I add that to Uh.
So add Uh plus this estimated error.
See if I knew E sub h, if I knew small eh-- the exact error-- then when I added I would get U, the answer.
Of course, I don't know that.
What I know is capital E. So that's a cap E, which will agree with e on some of these, but not others.
So all right, I guess I'm seeing the problem.
I may have headed in reverse because it's I minus S that's crucial.
So these are errors.
Now what happens to them?
Are these the good ones?
I think they're the good ones, is that right?
They're the good ones because the errors in this stage, multigrid gets them exactly, so that when I add this in it's right on target.
It's accounted for.
So these are ones that multigrid fixes because of course, everybody recognizes that I minus-- let me just say it, though.
That I minus S times those errors is zero.
So the eigenvectors with eigenvalue 1 are the ones that are perfectly captured-- E sub h has got those components of small e sub h exactly right when I add them on; perfection.
But then there were other components which it hasn't got at all.
And when I add those on, no change, no improvement.
By the way, so these eigenvalues are repeated.
Like here, I've got 5 by 5 matrix.
How many zeroes and how many ones?
You could say it's a projection matrix.
S projects on to these eigenvectors.
I minus S projects onto these eigenvectors and how many zeros and how many ones for this if A is 5 by 5?
Well, what is A?
This A is 2 by 2.
So if I look at my S, look at my nifty formula for S, do you see all this here?
Let's just get the sizes.
A is 5 by 5.
R is 2 by 5.
It shrinks us down to the coarse mesh, which is only where this is only 2 by 2 and this takes us back up.
So the total result, 5 by 2, 2 by 2, 2 by 5, 5 by 5 is 5 by 5.
But what's the rank of S?
Small prize for telling me the rank.
I'm interested in this count.
How many vectors are killed?
How many go through unchanged when I multiply by S?
So what's your guess for the rank of S?
2, exactly.
So S will have 2 of these guys.
2.
This has multiplicity 2 in the 5 by 5 case and this one has multiplicity 3.
So multigrid is only fixing 2-- getting 2 vectors exactly right.
And not getting these right.
Well, I mean, 2 is half of 5.
Well, in fact 2 is exactly half of 5 because if I went up it would be 3 and 7, 4 and 9.
The point is that the number of coarse grid points is about 1/2 the number of fine grid points.
So 2 is about 1/2 of 1, 2 3, 4, 5.
Of course, how could I expect better then getting 2 perfect?
I can't expect more because the problem I ultimately solve here is on the 2 by 2 size.
So when I actually do something useful, I'm only doing it on the coarse grid and that's got dimension 2 and so there will be 2 eigenvectors that get perfectly fixed, but the others don't.
I do have to think which are the eigenvectors?
What eigenvectors are there?
I guess what I hope is, what I believe is that these eigenvectors, which multigrid fixes for me, those E's-- well, the E's have 5 components.
Our matrix S is 5 by 5.
I guess I'm thinking, I'm expecting that the eigenvectors that these 2 are somehow basically on the coarse mesh.
And that these 3 that don't get fixed are somehow basically orthogonal to the coarse mesh I would say.
That these will be low frequency vectors and these will be high frequency vectors.
That's what I expect and that's what somebody-- I mean, there are whole books on multigrid, which where the main step in the analysis is to see that those eigenvectors are as I described, sort of coarse mash vectors that are fixed and fine mesh vectors that don't get fixed by multigrid and those are the ones that a smoother better deal with.
I mentioned that just before class I did a MATLAB experiment and I actually did it on these matrices with a Jacobi smoother.
And I broke down the numbers here somewhere.
and actually I was a little disappointed.
But I came away saying, OK, 5 by 5 just wasn't big enough.
So I'll put down what I've got.
What did I get for the eigenvectors?
eigenvalues of S I got 1 and zero.
That was a good thing.
Now did I write them down?
I sure hope I did.
Yep.
So the question is, what about when M is in there?
So the exercises were find the eigenvalues-- well, it'd be interesting first of all, to know the eigenvalues of M. This is the Jacobi smoother, weighted Jacobi.
Here's one we're really interested in.
We're interested in doing a smoother, doing a multigrid cycle, and doing a smoother.
I'm hoping that those eigenvalues are below this.
So this is one smoother and this is smoothers with multigrid and this would be 2 smoothers and 2 post smoothers.
Repeating Jacobi twice and M cubed would be interesting.
Here is what I got for these things.
What do I know about the eigenvalues of M?
Do you remember that we actually worked those out?
For ordinary Jacobi they're cosines and for weighted Jacobi that weight omega, which I chose to be 2/3 comes into it.
Anyway, so in a 5 by 5 case-- well, it's the biggest one that we all really want to know.
So this is maximum eigenvalue.
Let me just write down the maximum.
So the maximum eigenvalue of M was 0.91.
So that says that ordinary Jacobi on a 5 by 5 problem reduces the error by that factor at every step.
And of course that factor is totally satisfactory.
If we could always do that we wouldn't need to do multigrid.
But that 0.91, if I go to 7 by 7 or 9 by 9 it's going to quickly move up toward 1.
That's an indication that my little experiment here is a bit small because that's not as near 1 as a true [?
vector.
?]
OK, what was the result after this?
I'm sorry to say that it was 0.88.
Disappointing.
So I did this one.
I did 2 smoothers.
Well, of course that's going to give me 0.91 squared in there.
But anyway, I got 0.64.
Which of course is very good.
I'll say very good, but actually for that size I had hoped for better because my impression of multigrid from reading about it is that-- I mean that a good multigrid cycle, but this has an eigenvalue of about 0.1, is the goal.
So how is that achieved?
Well, I don't want to achieve it by just doing a whole lot more smoothing.
Actually it's achieved by doing a whole lot more multigrid instead of doing just a little v-cycle this would be the same for a good, deep v-cycle that went down maybe 4 and maybe a w-cycle; do it twice.
Then of course, this is for a big problem, but really making multigrid work like a w-cycle.
Or even there's something called full multigrid.
Should I draw the picture that corresponds to w for full multigrid?
So full multigrid you start on a coarse mesh.
Go up, back up, up, back, back, maybe twice, up, up, up to this level, back, back, back, and up again.
So that's called full multigrid.
That gives the best numbers.
Anyway, we get numbers like this for when we really do more and it's very much worth it.
I mean, it's completely a success.
I would be interested to see a little-- well, I guess what I'm going to do now is, for this model problem that we worked on I could find these eigenvectors, e. And analyze this model problem.
OK, so how to find these eigenvectors.
Again, for these particular matrices.
Well, let me start with the eigenvectors of A itself.
Do we know the eigenvectors of A?
I guess if I had one important matrix, whose eigenvectors-- and nontrivial.
I can't look at the 5 by 5 there and guess at sight, say what the eigenvectors are.
So everybody sees it's 5 by 5 with 4 minus ones below, 5 two's on the diagonal, 4 minus ones above the diagonal, and the eigenvectors of those special matrices have a special form, which allows us to do everything here.
So what can I do here?
I can do the computation of what S is.
Can I report on that, which will be in the notes.
Let me tell you what S is.
If I take that as a R, that as I, that as A sub 2h I get S to be this matrix.
Column of zeroes, 1/2, 1, 1/2.
Column of zeroes.
1/2, 1, 1/2 and column of zeroes.
So it's rather remarkable in this particular case.
Now what can we recognize from this?
Well can we see eigenvalues of S explicitly from the matrix?
Well, we certainly see that it has rank 2.
It has 2 independent columns so its rank is 2 as we predicted.
So it will have 3 zero eigenvalues and we can see what the eigenvectors are actually.
What are the eigenvectors if I go up above here?
Can I do that?
I want to go up and say something about the eigenvectors with eigenvalue zero.
What's the null space of that matrix?
If I multiple S by vectors I want to get zero vectors.
The eigenvector's e for lambda equals zero.
If I multiply by anything with a first component, with these three components, you see that if I multiply Se is zero there for these e's.
In a way that's exactly what I hoped for.
That these vectors, e are the vectors that live on the fine grid, but with no component on the coarse grid.
And they're the ones which have Se equal zero and multigrid doesn't help.
Multigrid doesn't help.
I minus S times those vectors is those vectors, no improvement.
So they're eigenvectors with eigenvalue zero for S, but with 1 for I minus S. See my general thought is those would be the high frequency eigenvectors, high frequency vectors.
They're sort of high frequency, but they're not low frequency anyway.
Because low frequency these would be a smooth vector.
OK, so what are the eigenvectors with the other two eigenvectors-- the ones that multigrid fixes?
OK, so I'm looking for the other two eigenvectors here and how do I find them?
Let me see if I can even do it for this matrix.
I'm looking for the eigenvectors with eigenvalue 1 now.
So I should subtract the identity from this matrix.
When I subtract the identity the ones become zeroes and the zeroes become minus ones.
That's S minus I now, trying to keep a running equation that's correct.
That's S minus I.
And I'm looking for the null space of that now.
Am I going to find it?
I don't know.
Do see immediately what-- that has a two-dimensional null space.
There are 2 vectors and can you see what they are?
What do you think?
Yeah, we certainly could.
I don't know whether-- anybody see a vector that's in the null space of that matrix?
Let me just start and try to get one.
So I'm looking at that matrix, looking for a vector in its null space.
And that will be an error vector that multigrid totally fixes.
Say if I start with a 1 up in that first component then the first row will force me to have a 2 there.
Is that right?
And then I fixed the first row, the second row is all zeroes already.
Then I have 0.5 of that, which is 1.
Oh, well let me start from the bottom here.
Well, it's probably symmetric.
What do you think?
Maybe something like 1, 2, and what's that middle number?
Maybe 2.
I think that that is in the null space of S minus I because if I multiply by the first row I get minus 1, plus 1 and the third row gives me 1, minus 2, plus 1 and yes-- so that's one of them anyway.
And the point about it is it's pretty smooth.
That's what I anticipate vectors that are smooth, that's the type of vectors we're going to find multigrid fixing.
So I've got one more eigenvector to find.
Maybe I'll leave that as a puzzle, somewhere there's another eigenvector.
Guess I'm not sure where it is, but I think it should be smooth.
Somewhere there is one.
Can I now go away from this numerical example to give you the answer for this problem, but for any size?
Oh, it's because we know the eigenvectors of A. I started to talk about the eigenvectors of A and I didn't finish.
If I want to get formulas for these I need to know the eigenvectors of A.
So A, this matrix of twos and minus ones--- the eigenvectors of that matrix are discreet sine functions.
So an eigenvector would be for example, sine of pi on 6, sine of 2 pi on 6 up to sine of 5 pi on 6.
It's worth knowing that those are eigenvectors.
And other eigenvectors would double the frequency, triple the frequency, four times the frequency and five times.
The eigenvectors of this matrix are what I will call, discrete sines.
Discrete sine vectors.
And why?
Because this matrix builts in the boundary condition since there should be a minus 1 there, but it's chopped off.
That minus 1 if it was there would multiply the zeroes component.
But the zeroes component is removed, so we have to be sure that the zeroes component would have been zero anyway.
And that's the point.
You see if I follow this pattern the zero component that I'm missing would be sine of zero.
The component I'm missing up here would be sine of zero pi over 6, which is zero.
And the component that's missing down at this boundary would be sine 6 pi over 6, which is sine pi, which is zero.
That choice of vectors is the perfect Fourier vector.
I expect Fourier when I see constant coefficients there with the right boundary conditions zero at both ends.
Anyway, those are the eigenvectors.
The k'th eigenvector will have sine k pi, sine k2 pi, down to sine k5 pi, all over 6.
And I'll just, in the remaining moment write down the result of following through those vectors and finding finally, the eigenvectors of S itself.
So this is like multigrid in a nutshell if I can find it.
Multigrid in a nutshell.
All right, better be on this final board.
I minus S times this sine vector , the k'th sine vector that I wrote over there and I minus S times the n minus k'th sine vector.
So these will be low frequencies for small k and these will be high frequencies.
And it turns out that these are 1/2, 1 minus the cosine of k pi over 6 or n plus 1 times yk plus-- sk, sorry.
Sk plus S sub n minus k. And this acting on the high frequencies is 1/2 of 1 plus cosine k pi over 6.
Sk plus S sub n minus 1.
What am I seeing there?
Let's just accept the algebra that went into this and see, does that tell me what the eigenvectors of I minus S are because that's my goal.
Well, it does.
Look, if I add these two equations, let me look first just at this equation.
So this says that if I start with a low frequency, k small, low value of k that it's nearly removed.
If k is very small the cosine of k pi over 6 is somewhere near 1.
Of course, 6 was a little feeble, so let me take n; bigger matrices.
Then the low frequencies are the ones where this cosine is near 1.
This number is near zero and they're practically removed.
Where the high frequencies, not much happes because the cosine is near 1, this number in near 2.
I divide by 2 and it's near 1.
So the overall picture is right.
Now to get to pin it down, if I add that two equations then I get this combination Sk plus S minus k. Oh in fact, I guess I see it.
What happens if I add the two equations?
I get I minus S times Sk plus S sub n minus k and what do I have on the right-hand side?
Well, 1.
These add to 1.
So I get Sk plus S sub n minus k. Those are the eigenvectors of I minus S with eigenvalue 1.
They're the ones I found here.
They're the eigenvalues of S with eigenvalue zero.
These are the Sk's plus S sub n minus k. And they live completely on the mesh points that are in the fine, but not the coarse mesh.
And then if I play around one more step I can find the eigenvectors for the other eigenvalue.
The ones that are basically low frequency and where we have a numerical example.
OK, I've run out of time before I can do that last bit of algebra.
That will be in the notes, which we'll get onto the website, Monday at the latest.
So I hope you have a sort of picture of what multigrid does and the experiments that you might do to check it out.
OK, I'll see Monday and we'll take a little more time to discuss projects because we'll have material on the web to work with.
OK, thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.MIT.edu
OK. Should we go?
Great.
Well I hope you enjoyed Patriots Day, and now we'll go for the final weeks of the course.
And I wanted start with just a thought about scientific computing, and the people who work in it, and the problems.
So just sort of an orientation.
And maybe my point is that in part of the subject, the situation started like just give me the matrix, tell me the equation.
I don't want to know the physics behind it, just the form of the equation, and I'll discuss how it could be solved.
And to contrast that with what you may often be meeting in your thesis even in project, the details, the physical consequences, the actual detail behavior of this solution is what you're looking for.
So somehow there are those two parts of the world, and people who know understand the physical problem are on the second side.
And people who maybe think just about matrices, like that matrix for example -- that saddle point matrix, are more on the first side.
Like just give me the matrix and I'll figure out how to solve the linear system.
If it's a large linear system, I'll invent preconditioners, study stability, whatever you have to do.
But today's lecture is kind of half and half, because that is the framework that we're in.
But the problem comes from a real problem in fluids.
And it's called the Stokes problem, not Navier-Stokes.
So Stokes alone is dealing with this question when there is a diffusion term, and I've normalized the coefficient there, viscosity term, whatever we call it, to be 1.
And there is the constraint of incompressability, constant pressure divergence p equals 0.
I'm sorry not constant, divergence of p equals 0 is a constraint.
First let's do this.
So make some comment on the physical side of this equation.
What we're missing that Navier-Stokes would put in is a convection term.
A term of v dot grad v that takes account of the fact that there's a flow here, things are moving.
We're not in a stationary situation.
And that term has been dropped.
So that's telling us then we are in the case of very, very slow flow, where that term could be neglected.
And certainly neglecting that term makes our problem easier.
It leaves us with this symmetric matrix.
It doesn't make it totally easy as we'll see, but it's a lot easier than when the convection term -- which by the way is not symmetric and pushes outside this framework.
If that's present then of course in computational fluid dynamics, that's the central problem.
You have to deal with it.
So we're taking a model problem that doesn't have the convection, doesn't have that unsymmetric complexity.
But actually it's become more and more important.
It's more than a model problem now, because there are a lot of medical application.
Say probably the flow of your blood pushed by the heart that's moving along at a decent rate.
But flow into tiny capillaries is slow, and Stokes is a reasonable step.
And then actually Stokes is also going to appear in Navier-Stokes calculations, because often when you have two difficult processes you do a kind of splitting method, and you drop one term for half a step.
So you take for example, a Stokes step, and then you do what the fluid forces you to do.
You move that process with the flow.
OK.
So where flowing slowly here.
And the nice thing is that this fits our saddle point framework, that's why I'm calling this block matrix S. And you see really it came from minimization.
The minimize was the problem that leads to the Laplacion, that stands for the Laplace operator with a minus sign to make it positive definite.
And then this constraint gets imposed by a Lagrange multiplier P. So P is the Lagrange multiplier here.
Oh wrong.
Divergence V is 0, incompressive.
Now that's on the tape.
It's now on the tape right.
The vergence V is 0.
OK.
So you remember how our equations looked?
I'm going to write the unknowns here, V and P, So that you see what everything is, and it equals f and 0.
OK.
So this is now S, so I chop off here.
So there you see the equation in the saddle point form.
So A is the gradient, and it breaks on P. And so P is a scalar.
I better say if I put a little arrow here, V is a vector, it has to components say if we're in a SD problem; f has two components.
P the Lagrange multiplier for this constraint, the constraint is just a scalar constraint the diverge is 0.
You remember what divergence means; dv1 dx plus dv2 dy, so those are the two components of V, the vector.
And its divergence is 0.
And these are, well with the minus sign, are adjoints to each other.
OK.
So what's new that makes this problem worth some study?
I guess that up to now, we've taken problems like this and we've done elimination to get to a single equation.
You remember that.
So that single equation is A transpose CA.
Let's see.
We're multiplying that row by A transpose C and subtracting.
Let me do the elimination under here.
I multiply the first equation by A transpose C. Just give me the matrix mode for now.
I multiply by A transpose C and I subtract.
So the first equation is still there, C inverse v plus Ap is f. But the second equation now when I multiply by A transpose C, so I just put it up there, and subtract that produces a 0 there, it produced minus A transpose CAp equals minus A transpose Cf.
It leads us to this crucial matrix.
And in applications up to now that was the way to go.
That was totally satisfactory.
We eliminated v, we got an equation for p, we solved it, and then once we knew p we went back and found v. OK.
So why is this different?
Why don't I want to get to A transpose CA, the stiffness matrix if we were in finite elements.
Why do I want to stay with this two field system?
Which in finite elements is going to be a mixed method of finite elements.
I'm going to keep v and p in the problem for a simple reason, that C in this problem is awkward, C would be the inverse of the Laplacion.
C is the inverse of the Laplacion d second by x square plus d second by the y square.
That's what delta stands for.
And that's an uncomfortable operator to work with, the inverse here.
In matrix language when we discretize, it won't be sparse, right.
The Laplacion is sparse, the Laplacion gives us the five point matrix, the matrix with five non-0's on every row - four on the diagonal and four minus 1's at matrix that we know, the K2D matrix.
But this is its inverse, C is now its inverse.
It's the inverse of that matrix K2D that we've study, and of course it's full.
So taking this step produces a completely full matrix, and all of our efficient algorithms for large problems work for sparse matrices.
So whereas this one is sparse.
I mean this the Laplacion.
So we have the five point matrix sitting here.
Here we have a discreet gradient, and a discrete divergence.
They're all derivatives, and you think of the derivatives as being replaced by differences.
All these matrices are sparse, but if we take that fatal step of eliminating to get here, the matrix is full.
Because of this fact that the inverse of a sparse matrix tends to be a full matrix.
And you don't want to work with that.
So we don't want, that's my message, don't want A transpose A when this is not sparse.
OK.
So that's the point about this example, which in the notes on the website is the section called the saddle point Stokes problem.
OK. All right.
So now we have both a specific example of this Stokes equations, three equations really because this is two equations for v1 and v2 and this is the third equation, the constraint.
OK.
So we have both a specific example and a general case.
So we're now going to work with that matrix as a whole, we're not going to reduce it to A transpose C, although in principle we could.
And the question is one of stability.
So like stability is a question that didn't come up for our minimum principles when there were no constraints.
And maybe I'll just put a little mention why.
So this is the first time we've had to face the question of stability.
What do I mean that S inverse is under control?
I don't mean that it's easy to compute, the inverse of that block matrix.
But I have to know that the matrix is invertible.
And stability is always like one slight step more than just being invertible.
It means it's invertible and you have some control of the inverse.
You know, it's not just on the verge of disaster.
So we want to know about the inverse of that matrix.
That's essentially it, we want to know about the inverse of that matrix.
And in the process, we really want to know about the inverse of this matrix.
So we may not want to compute it, that was my point, but stability is telling us we have here a matrix that's not positive definite.
That the key issue here.
Our saddle point matrix is not positive definite.
It has positive eigenvalues and negative eigenvalues.
Let me just take some examples.
OK.
So here's an S. So I have a positive thing there.
Non-0 stuff there.
There is a simplest example I could think up.
That matrix is not positive definite.
It's determinate is minus 1 actually.
But I mean they're safely away from 0. it's not singular or close to singular either.
So that two by two example no problem.
Well I could study the invertibility of this particular matrix, that would be solving differential equations.
In other words I could study the case RP and V under control if I have a bound on f. Well they are.
But the issue comes when I go to finite elements, when I look at a sub space.
so have to try to make that clear.
What I plan to do to solve a continuous problem is to discretize it.
This is my continuous matrix, block matrix, symbolic of the differential equations.
And it's OK, analysis would show that.
But the question is if I approximate V by some combination of finite elements, let me call it Vh.
If I approximate the pressure by some pressure element Ph, then I'm sort of projecting the problem down into these finite dimensional, finite elements sub spaces.
And of course f will get projected down to fh.
And the matrices, these operators, will turn into finite as far as matrices.
And the question is that projection safe?
So that really it.
It's the stability.
S inverse is under control, this is OK.
But is the projection of S inverse under control?
Is the projection of S. So if I project it, is the inverse of that under control?
That's the question.
What comes up in applications, this is a projection of the continuous problem, the differential equation, onto finite element spaces, finite element trial function.
OK.
So we've spoken a little about finite elements in this course, and more in 18085, and you're probably have seen them elsewhere.
You know that we can approximate velocities.
Here would be a really important example, because it's an especially simple finite elements.
If I take my region, suppose it happened to be that rectangle, I divide that rectangle into small squares, there's a small square.
I'm using squares rather than triangles.
And in each square I make some low degree approximation, low degree trial function, for the velocities and the pressures.
And the question is what functions can I use?
For example, one I would love to be able to use would be, so in each square I would like to be able to use piecewise constant pressures.
Piecewise constant pressure, so ph equal constant in each element.
Element is the square.
So a different constant in each element.
So that would give me my vector of phs.
It would be one component for every little square.
And now what about Vh?
What trial function would I like to use for the velocities?
Constant velocities wouldn't be good, that's further than I'm willing to go.
I'll go piecewise linear in each square.
Now piecewise linear in each triangle meant that it had to form a plus bx plus cy.
Strictly speaking that's what I should mean by piecewise linear with a different coefficient abc in each piece but because we're in squares now, we've got four nodal values, and we want a fourth coefficient.
This was appropriate on triangles, on a square we really include the bilinear term dxy.
OK.
So we have four coefficienct which determine and can be determined from the four values of the node.
You see that I'm describing the whole space.
Now I cut the region into more squares, I've described a away of discretizing the differential difference equation, the weak form of continuous problem.
When I plug in these functions would give me a matrix problem, a discrete problem, that will have this form.
And the question is do I have any control?
So it will give me this projected matrix now, finite dimensional instead of a continuous problem.
And the question is Have I still got any control?
Well the answer for this particular choice of pressure space and velocity space is no.
Unfortunately the matrix S that appears instead, the C inverse h shall I say it would be like our K2D inverse.
That was pretty close of what it would be.
And the Ah and the Ah transpose h and the 0 that matrix has a famous null vector for this specific example.
I'm taking big jumps here, because I just want to say the main point.
The main point here is that a null vector appears in this problem.
S fails to be invertible, and that null vector is like finite element people know its name.
It has a couple of names actually, maybe checkerboard.
The null vector is a vector when the pressure is 1 minus 1,1 minus 1 in a checkerboard pattern over the whole region, and a corresponding velocity.
Well the trouble is Ah times this P is 0.
That's what kills you.
It turns out that A times that particular pressure is 0.
And if you lose control of A that way, you've lost control A transpose CA.
You've lost everything.
So I guess coming back to my opening comments.
Here's a case in which we have a physical problem, but it falls into a pattern, where the mathematics actually decide are we able to use the trial spaces that are most convenient.
I say convenient rather than most accurate of course.
If I wanted to improve the accuracy, I would go up in the degree, I would take pressures to be piecewise linear and velocities to be piecewise quadratic with more points.
And I could check the stability or not, is the matrix invertible or not, and the answer might be better there.
In fact the notes and any book on finite elements has a list.
And I don't plan on making that list here.
But it has a list of press spaces, pressure element and velocity elements, on which the first guy that I've mentioned would be like Q of 0, degree 0 and velocities Q1 degree 1, and the letter Q being used for quadrilateral If I divided it into triangles, I would use the letter p for polynomial in that case.
And this is a fail.
This one fails.
And then another list would be how about Q1, Q1?
I don't know.
I wouldn't bet much on Q1, Q1, and of course I can look at the notes to see what the answer.
Or Q1, Q2 that's probably got a better chance.
You know people are inventing finite elements all the time, and they're having to check the stability or not, the uniform invertibility or not of S. OK.
I just want to say that this one is so attractive, because it's so simple that people don't give up on it.
They want to go with this, this is a fail, but so stabilized.
That one is so attractive and, you know, specifically that it's this checkerboard mode that's in the null space.
So you just think, how am I going to get that mode out of the null?
And the stabilizing method is you stick a matrix here for purely numerical reasons, and it'll be a matrix that's exactly has this checkerboard pattern, so that the checkerboard is not any longer in the null space.
Maybe I call it minus some small number times some E matrix, it's often written E. And alpha you keep as small as you dare.
But the effect is that it takes this checkerboard pressure out of the null space.
You see the problem was that without that the columns of A were not independent.
A times P could be 0.
And if those columns are not independent you're lost.
Let me put the 0 back.
Why was that a bad thing?
I want to now sort of connect you to the matrix theory.
If I have this particular pressure and AhP is 0, so that tells me that those columns of Ah are dependent.
Then the columns of S are dependent, right.
If I have something in the null space of Ah, then I just put a pressure, this pressure, in the null space of Ah, and I just take the v to be there 0, then these guys don't do anything.
And that A times that bad pressure gives 0, so I have vp, a velocity pressure, that's in the null space.
OK.
So the way it gets fixed is to stick in some small stabilizing factor there.
OK. All right now that's the specifics of this particular example.
Well I didn't give all the specific.
If I were really on the ball I would show in detail why that checkerboard appears and kills this choice.
I would check each of these other choices, and beside each one I would put fail or success.
And when it fails, I would try to identify the failure mode, so that if you wanted you could stabilize and get rid of that failure mode.
OK. What I'll do instead is to come to the general question here.
So the general question is to deal with this matrix S. And I write it again C inverse A, A transpose 0.
I want to look if it's invertible?
Can I get some bound on its inverse?
What do I have to know?
And I'll just emphasize here, what's the real problem?
Because we saw it right there.
The real problem is if the column of A are dependent.
In other words i.e.
in other words, AP equals 0 has a non-0 solution.
Then you're wiped out.
I'll just repeat.
If the columns of A are dependent, and then the columns of S are dependent, the matrix is singular, and you're finished.
So somehow we have to get some muscle behind A.
But remember A is a rectangular matrix.
This C is m by m, this A that comes up in the constraint is m by n. It has, we hope, more rows than columns.
But still the columns could be dependent, and in this situation they are.
OK.
So the question is we have to get some muscle behind A.
And this is the famous condition so called in soup condition that you would find in Professor [?
Batiste ?]
finite element book.
But maybe these notes kind of focus on the heart of the idea.
And again in some way it's conditioned on A really, and on the pressure spaces, and the velocity spaces that you choose.
OK.
So these you could really say are Ah, Ph, and Vh, but I'll skip writing h's on everything now.
So I'm down in the discrete case, but I have a sequence of discrete problem which are growing as h gets smaller, and the question is do I have a fix beta that control this?
And it turns out that this the requirement this in soup condition says that for every pressure there is a velocity city, every discrete pressure now, there is a discrete velocity such that this holds.
So it's sort of forcing A to stay away from 0.
This beta is a fix number, and this sort of is the right normalization that goes with it.
So you see why v transpose A and P?
You know if I look at the quadratic form, v transpose P transpose, you know, the quadratic form with the matrix is always multiply by a vector here multiplied by its transpose here, where is the A term?
A will multiply P, and it'll be multiplied by v transpose.
In the quadratic part, in the energy, is this v transpose AP.
And that's what we can't let go to 0.
Because If that part fails us, if we lose these and we have a 0 there, this part on the sub matrix can't save us.
Right.
So before I say anything more about that.
let me just emphasize what stability was in the symmetric case.
So suppose I have a positive definite symmetric matrix.
it's got somebody eigenvalues, lambda 1 to lambda n. OK. What you see on this board is going to explain why stability is automatic if I'm dealing with positive definite things.
So suppose I project?
Let me just project in the direct way.
I'll throw away the last row and column.
Suppose I throw away the last row and column.
That part is now gone, so these are the eigenvalues of K. And of course being positive definite, they're bigger than 0.
OK.
So that was the whole matrix, but now what I'm going to do is I'm going to project on the first n minus 1 rows and columns.
So I'm going to look at the sub matrix, K sub n minus 1.
And it has some eigenvalues.
Are they the same as those?
No way.
And you see I'm not in this situation, this was not positive definite.
So I'm looking here at the positive definite case.
This has some eigenvalues mu1 up to mu n-1.
These will be the eigenvalues of the sub matrix, Kn minus 1.
And linear algebra is just like the right place to look for some information on the connection between the mus and lambdas.
They're certainly not equal.
Our first question is if I take a positive definite matrix with positive eigenvalues, I throw away the last row and column.
Do I still have a positive definite?
Is that sub matrix positive definite?
And the answer is yes.
The eigenvalues are all positive.
And more than that, how about that smallest eigenvalue?
Because the eigenvalue is measuring like how close are we to disaster, to singular.
Can I say anything about that smallest eigenvalue?
And the answer is yes.
The smallest eigenvalue of Kn minus 1 is never below the smallest eigenvalue of the full matrix.
So if the full matrix is OK, all the projections, all the sub spaces, all this sub matricies are even more OK. And up at the other end, at the top, the top eigenvalue in sub matrix never passes the top eigenvalue in the big matrix.
So you see at both ends I have what I want, I have control.
If I have some range of eigenvalues for the big matrix, then all its projections are inside that range.
And therefore the condition number, which would be the ratio of this to this, is smaller than the condition number of the whole matrix.
So the condition number of Kn minus 1 is less or equal the condition number of Kn.
OK. And that's exactly why working with positive definite matrices is so comfortable.
The stability is just automatic.
But here we're not working with positive definite matrices.
The stability is not automatic, and we need this in soup condition.
And I don't know whether to try.
Maybe I'm a little hesitant to go through the steps that lead from there to there, because this lecture is already asking you to put a lot of things together without details.
And the notes do the best job I could do in making those steps simple and clear.
And this is what you actually check.
You check this condition for any proposed finite element, and any different problem.
It wouldn't have to be the Stokes problem.
Other problems would give us a constraint matrix A.
A sort of stiff matrix C inverse.
And we could check the condition again.
OK.
So maybe I'm going to leave that statement for the notes.
OK.
So that's this in soup condition.
And of course, you see right away that if A had dependent columns, which is the big risk, if A had dependent columns that would mean that Ap is 0 for some p. And then there would be no v. No v could possibly be found that would save it.
If A had dependent columns, that's bogey man that we're avoiding above all.
If A had dependent columns, then there would be a solution to Ap equals 0, this left side would be 0, the right side wouldn't, and the in soup condition would fail.
But the in soup condition tells us well what about if there are pressures, that where Ap get smaller and smaller as the mesh size changes.
And the question is can I find a v and so on.
So that's the full condition but not to lose sight of the fact that the central question, the key question is the independence or dependence of the columns of A. OK. Good.
So let's see, maybe that's as much as I want to say about the Stokes example, and what it leads to in a finite element world.
I guess I'm going to stay for another lecture or two with these minimizations.
So maybe I'll just anticipate slightly, the start of the next lecture.
Because I'm quite pleased about getting this next topic into the series of lectures.
Let me say what it is.
So it's going to be on minimization of an Au minus b square.
OK. Just as this lecture is seen, the possibility comes up at the columns of A not independent, that this problem is in trouble.
So I want to regularize this problem.
So this is a major application in which A, maybe A transpose is the easiest thing to say, A transpose a is nearly singular or just plain singular.
I'll say or singular.
How do we rescue the problem?
Because numerically and theoretically it's a disaster as it stands.
OK.
It's ill posed.
So let me introduce that word ill posed problem.
And let me mention, since I'm just using these last few minutes as a kind of getting the big picture, where do ill posed problems come from, where does these things come from?
They come typically from what are called inverse problems.
So the application is to inverse problems.
What's an inverse problem?
An inverse problem for a differential equation is you know some inputs and outputs, you know some solutions, but you don't know the equation.
You don't know the coefficient.
So you're trying to determine what are the material properties by looking to see what the outputs are for experimental inputs.
And very frequently you don't have enough inputs to really determine what these -- so it's inverse problem to find the coefficients of the equation given some information on its solutions.
some measurements.
And of course they're probably noising measurements.
I mean this is an enormous source of applications growing an importance, and how do we deal with it.
Well your regularize it.
So this is typical of this type of problem.
You add on some multiples of something that's safe, like for example u square.
So the results is here that this A transpose a that comes from this problem there's an alpha times the identity coming from this problem.
I mean it's an ordinary least square problem, and it leads us to the matrix A transpose a, comes from the first part.
But I'm just talking about the standard normal equation.
It will produce an A transpose a from here.
But from here it will produce an alpha times the identity.
And of course just the way this minus E worked up there, here I'm doing it even more directly, I'm pushing the problem away from 0.
This alpha, this parameter and the choice of that parameter, is the million dollar question in this whole subject.
But it's purpose is to regularize the problem, to take his nearly singular matrix and push it up.
See this matrix, A transpose a, never has negative eigenvalues, the problem is 0 or very small, small positive.
So by adding on alpha I, I push those away from 0.
Over there we had the problem of eigenvalues being positive or negative, and by pushing in one direction we might just bring a negative eigenvalue right on to 0.
That's not our problem in this upcoming lecture.
In this upcoming lecture we know that nothing is negative, but we need to make it properly positive.
OK.
So that's what's coming.
Can I just say the application that I want to describe is the first one I've given in life sciences, and I hope not the last.
I don't know if any of you can suggest, and if you can an email will be very much appreciated.
Applications of this whole framework in like sciences.
So you'll see a particular application of this ill posed problem in trying to determine gene structure from expressions.
OK.
So that's what's coming Friday.
And I realize here I'm talking about material that probably is not going to go in your projects, because of the timing.
I'm thinking that you're projects are more likely to build on what you had inn project one, and the basic material about solving large systems and basic minimizations.
OK.
Thank you.
See you Friday.
The following content it's provided by MIT OpenCourseWare, under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare, in general, is available at ocw.mit.edu
Several pieces of good news.
Since Brett is back now, all of chapters, five and six of my notes, that is to say, all that we've covered in class, plus a little more, are on the web page now.
So, I was cautious at the beginning of the semester and didn't put that up, and then forgot that it wasn't there.
So now you have something to work with, particularly on any experiments, for example, minimum degree ordering.
Oh and also there's a movie now.
Tim Davis from Florida and [?
Perils ?]
[?
Person ?
], my friend here, created a little movie to show the order that elimination occurs.
And I had a lot of email with Tim Davis.
On that one page handout, it mentions 2d Laplace's equation.
It mentions n cubed for the nested dissection, where you cut the region in half, and half, and half, by separators I believe, and I think experiments will show, that minimum degree is even better.
But the analysis is extremely weak apparently on minimum degree.
So anyway, one possible project either for now or for the final project would be some experiments to see what the actual convergence is like.
And maybe a little understanding of minimum degree.
So we'll talk more about that.
Anyway, there's a movie and a lot more material there on the website.
It'll get updated and corrected as I as I do the edits.
Second bit of good news.
I made a mistake.
That's not normally good news, but this time, I reported a little calculation for multi grid, and I computed the wrong thing.
You remember that I computed m. m was the jacobian matrix.
So m was the jacobian matrix, the iteration matrix with jacobian which is i minus the inverse a.
This is for the 1d second differences.
In fact, this was the 5 by 5 matrix that we've been talking about.
And now this is multiplied by d inverse which is a half, but then with a weighting factor, that becomes a third.
So that's weighted jacobian now.
And this is what we would get for ordinary simple iteration.
And it's not satisfactory.
Remember it had an Eigen value lamda max.
The spectra radius was about point 9.
Then I reported about the Eigen values m s, The multi grid matrix sm and I was a little disappointed in it's Eigen values.
The reason is, I should have been doing i minus sn.
When I do that, I'm not disappointed anymore.
What are the Eigen values of this?
You remember s is the matrix that tells us how much error we're capturing.
s is that matrix that came from the error we captured at the end of multi grid, was s times the error that we entered with.
And so the error that's left, the remaining error, the new error, is the rest.
It's the part we don't get, and that's when I forgot.
That's i minus s e. So that's why I should have been using i minus s. So then the step one of multi grid does it smoother.
Steps two, three, four leave me i minus s. And step five did another smoother.
And this is just with one jacobian step, which you would probably do three.
Anyway the Eigen values of this were, well -- this has two zero Eigen values, and three 1's.
So this matrix has Eigen values 0, 0, 1,1,1.
Because, why's that?
Well, you remember the Eigen values had to be zero or 1, because s squared equaled s, and i minus s squared equals i minus s. So.
We remember that.
Now the question is, what are these?
And the answer is 1/9 is a triple Eigen value.
I was astonished to discover 1/9.
Of course, zero, zero survived.
The rank is 3.
And this is now looking much better.
What I reported last time was some more like 7/8, or something, and I thought oh multi grid, it's not showing it true colors.
But actually it does.
This is a kind Eigen value we expect.
Like 1/10, or a multi grid cycle.
So much better if we just did 3 m's for example, I would get them down the point 9 q, which is more than point 7, where by doing multi grid instead, we're down to point 1.
OK.
So that's the good news.
And, of course, if you experiment a little, you'll find different numbers for different sizes and really see what's happening.
And, in fact, you could do 2d.
I mentioned all this partly for now, if you're wanting to do it now, or partly for eventually later.
Oh, and thinking about projects, just one word to repeat, that the Monday after spring break, I think that's April the 3, no class.
So it'll be Wednesday that I'll see you again.
But Mr. [?
Cho ?]
will be available his office hours.
So that's April 3, if you wanted to discuss issues with your project with Mr. [?
Cho ?]
that would be at class time in his office to 1:30, 1:00.
Otherwise taking extra spring break, and I'll see you Wednesday.
OK.
So that's various bits of good news.
Oh.
One other thing that I guess I didn't finish in that lecture, was to identify the two Eigen values.
Well, Eigen vectors.
Those will be in the notes too.
I guess I found one Eigen vectors, 1,2,22,1.
And glancing at the matrix, I didn't spot the other one, but it has one oscillation.
It's 1,2,0, minus 2 minus 1.
Anyway, this is the main thing.
A much better result.
So that's multi grid, which we now can use as our iteration.
Or we can use it as a preconditioner.
Many of the recommended methods use multi grid as a preconditioner before something even more powerful.
Well, I don't know about even more powerful.
Multi grid people would say not possible.
But some situations we may want to go to a different method but we want a preconditioner, and multi grid excellent.
By multi grid, I include the smoother.
So it'd be that.
That would be a possible preconditioner for something else.
OK.
So where are we now.
We're into the new section starting this moment.
And what's the section about?
It's about things associated with this name Krylov.
So, I'm going to use the letter k for the things that are associated, and I'm always solving au equals b.
There's a Krylov matrix that created, exactly as you see here, this is the js the one with j columns.
b, Ab, A squared, b up to A to the the j minus 1 b.
And the Krylov space is the combinations of those vectors.
That's what this word, span means.
I could erase span, and say, "all combinations of" maybe that even more familiar.
So span by is the same thing is saying all linear combinations of those j vectors.
And, of course, that's the same as saying it's the column space of the matrix, because the column space is all combinations.
OK.
So Why I am I interested?
Why was Krylov interested?
Why is everybody interested in these vectors?
Because actually, that's what an iteration like Jacobi produces.
If I use Jacobi's method, or Gauss-Seidel.
Any of those.
After one step, I've got b.
After two steps, there's a multiplication by a in there, right?
And some combination is taken, depending on the particular method.
So after two steps, I've got a combination of b and ab.
After three steps, Jacobi produces some combination of b, ab, a squared b.
In other words, all of those iterative methods are picking their j'th approximation in this space, kj.
I should put a little j on it.
So these spaces are growing.
They grow by one dimension, by one new basis vector at each iteration.
And the point is Jacobi makes a particular choice of xj or uj the approximation after j steps.
But does it make the best choice?
Probably not.
In fact, pretty definitely not.
And so the idea is let's make the best choice.
Lets not just use as simple iteration that built in a choice that might not be so terrific.
Let's make the best choice.
There are a whole lot of Krylov methods, which all choose xj -- since I think this section we use x, I'm going to change that x.
They'll all the iterative.
They'll all started with xzero as zero say.
And then the first guess will be b maybe.
Or maybe not.
Maybe some multiple of b.
Actually a good Krylov method will take the best multiple of b as the first guess, and not necessarily 1 times b.
And onwards.
So we have this word, best, coming up.
What's the best vector in that space?
And there are different methods depending on what I mean by best.
Oh, let me tell you the name of the method that I'm going to concentrate on first and most.
Will be the will be the conjugate gradient method.
CG.
The conjugate gradient method.
What determines xj?
It it chooses xj in kj, the space that we always look for our approximation in.
Let me not forget to say these vectors, b, ab, a squared b and so on, they're easy to compute, because each one is just a matrix multiplication from the previous one.
And the matrix is, we're assuming we're working with sparce matrixes.
And mostly, and especially, sometimes, especially symmetric ones.
So just let me put in here, before I even finish that sentence, cg, this is for a transpose equal a symmetric.
It's only for those.
And positive definite so symmetric, positive definite.
So it's a limited class of problems, but highly, highly important class.
And you may say what do we do if the matrix is symmetric, but indefinite?
Well, that comes next.
Or what if the matrix is not to symmetric at all.
Well, if you're brave, you might try conjugate gradients anyway.
But if you're cautious, then you would use one of the other methods like min res.
Maybe I'll just mention one more on our eventual list.
Actually that tells us probably what choice it makes.
Min res chooses the xj to minimize the residual.
Minimum r. Remember r is b minus ax.
The residual r will always denote the error in the equation.
And so it's minimum residual.
OK.
So that's a natural choice, but you'll see that this is fantastic method.
Superior, just quicker than min res for a nice reason so it chooses xj in kj and I think the rule it uses is that I think xj should be orthogonal I'll go and check it in my notes to the residual rj.
Let me just be sure if I'm going to write that down, I better be sure I said it correctly.
I actually I didn't say it correctly.
r j is orthoganal to the whole space kj.
It turns out that we can choose xj.
When I make a choice of xj its in kj.
Now when I compute r, there's a multiplication by a to get the residual, so that moves us up to the next space.
And I'm going to make a choice where this is orthogonal to the whole space, kj.
You have to do conjugate gradient.
And I can't start on that for a good reason.
I have to start with something called, arnoldi.
And and what's arnoldi about?
Well.
Let me come back to these vectors.
Quick to compute.
But what's the other property?
Everything in applied numerical mathematics, you're choosing basis vectors, and you're looking for two properties.
And I guess this is like a general rule, whenever you meet a whole new problem, in the end, you're going to construct some basis vectors if you're going to compute.
And what properties are you after?
You're after speed.
So they have to be quick to compute and work with.
Well these are.
Multiplying by a is sparce matrix multiplication.
You can't beat that.
But the other property that you need is some decent independence.
And not just barely independent.
You want the condition number of the basis somehow to be good.
Not in the normal.
And best of all, you would like the basis vectors to be orthonormal.
That's a condition number of one.
That's the best basis you can hope for.
And the point is, that this construction produces a lousy basis from the point of view of condition.
So arnoldi's job is orthoganalize the Krylov basis, which is b, ab, and so on, producing orthonormal basis q1, q2, up to qj.
It's just beautiful.
So arnoldi's taking like a preliminary step Krylov has to make to get something that numerically reasonable to work with.
Then if it's fast, we have to do a multiplication by a, and you'll see one here in the middle of arnoldi.
And then, of course, if you look at those ten lines of MATLAB, and we could go through them a little carefully, but let's just take a first look.
The first step is like Gram Schmidt, right?
It's sort of a Gram Schmidt idea.
You take that first vector b, you accept that direction and the only step remaining is to normalize it.
Divide by its length.
Then you go on to the next.
Then you have some trial vector t, which would in this case be ab, in the direction of ab, which won't be orthoganal to the original b, right.
So this won't be orthogonal to that one.
So, of course, you've got to compute an inner product between them.
And subtract off the right multiple of the previous one from the current t to get the improved t, and then you're going to normalize it again.
So you compute it's length and divide by the length.
You see that overall pattern in arnoldi?
It's the familiar idea of Gram Schmidt of take a new vector, find it's projections on to the ones that are already set, subtract those components off, you're left with a piece that and you find it's length and normalize that to be a unit factor.
It's very familiar.
Its exactly the type of algorithm that you would write down immediately.
And it just involved the one multiplication by a so that it's not going to cost us us a lot to make the vectors orthonormal.
Well.
Wait a minute.
Is it going to be expensive or not?
That's like the key question.
It's certainly going to produce a good result.
Producing orthonormal vectors is going to be a good thing to do.
But is it expensive or not?
That depends.
It's certainly not expensive if the new trial vector t has only components in one or two of the already settled directions.
In other words, if I only have to subtract off a couple of earlier components, then I'm golden.
And that's the case when a is symmetric.
So that will be the key point here.
And I just make it here.
If a is symmetric a transpose equals a, then I only need and it's subtracting off where I'm headed for the new q, I only need to subtract off h j j, multiplying the q j, the q that was just set, and the one before.
hj minus 1j.
I'll call that a short recurrence.
It's short because it only has two terms.
And then there'll be a new h. j plus 1j which is just the length.
So there'll be one of these magic things in so many parts of mathematical analysis, a three term recurrence relation will hold -- in fact, I guess the reason we see three term recurrence relations in all the Legendre polynomials, Chebyshev polynomials, all those classical things.
The reason is actually the same as it is here.
That some thing in the background is symmetric.
I want to put a few numbers on the board so you see what is typical input and output to arnoldi.
And you'll see it symmetric and then you'll see its short recurrence, and then we want to see what?
OK.
So I work out a typical arnoldi example.
Not that one.
Here.
OK.
I think this is a good example to look at.
So matrix a is not only symmetric, its diagonal.
So of course, to find a inverse b is not different.
But we're going to do it through conjugate gradients.
So that means that we don't figure out a inverse, which, of course, we easily could.
We just use a very sparse, here's our right hand side.
Here's our Krylov matrix.
It has b in its first column.
It has a times b in the second column.
Multiply by a again to get the third column.
And a one more time to get the fourth column.
So that's k4, right?
And for a 4 by 4 problem, that's the end.
So actually here I'm carrying Krylov to the end of it.
There's no more to do.
Because once I've got this k j, the combinations of those are all of r4 all of four dimensional space.
Oh, yeah.
That raises an important point about how we're improving on iterations.
If I was using Jacobi, or Gauss-Seidel or one of those others, then, well, I won't say for this particular a, but usually, after four steps, I am by no means finished.
I've taken four steps, I've gotten closer, but I haven't got the exact answer.
But now, in this the world of product methods, after four steps, I will have the exact answer.
Why?
Because Krylov methods take the best x4 out of the space spanned by those four columns.
Well, the whole space, r4 is spanned by those columns and taking the best one must be the answer.
So x4 will actually be a inverse b, the answer we're looking for.
So these methods are on the one hand, direct methods.
They finish after four steps, finish.
You've got the answer.
Now.
That's actually how they were born.
As direct methods that gave the answer from an interesting iteration and they nearly died for that reason.
They were born and died, or were on death's door, as direct method.
Because as direct methods, there are better ones.
Then some years later, people went back to it.
Back to conjugate gradients, and noticed that thought of as just going part way, gave very successful answers.
So we're going to think of Krylov.
We're planning to stop before j equals n. In this picture, j equals n equal 4 here.
But, we plan to stop for j much below n. We're thinking of applications where n is 10 the fifth, and we're going to stop at ten squared.
100 steps maybe.
Or ten steps.
So that meant a total reconsideration reanalysis of conjugate gradients, and it is now back to a big success.
Well its a rather unusual thing that a method that people have put aside gets picked up again and becomes a favorite as conjugate gradients have.
First of all, I was saying that the basis, those basis vectors are not very independent.
That's not a good basis.
Of course, it's a detractive basis and maybe, do you know the name for a matrix that has this particular form?
The columns are constant, then, that's the key column And then it's first powers, second powers, third powers.
Do you remember whose name is associated with that matrix?
It comes up in interpolation, and it starts with a V?
Anybody know?
Vandermon, yeah.
Vandermon.
So I could call it v for vandermonde.
And vandermonde matrixes are not very well conditioned.
So now a little time out to say, because it's so important, how do you judge the good or poor conditioning of a matrix?
Those vectors are linealally independent.
The determinant of v is not zero.
A matrix is not singular, but it's too close to singular.
Suppose you have a basis, as we have here.
And you want to know is it good or not?
So, of course, you always put your bases vectors, -- I don't want to call them v because -- well I guess I could call them v, they come out of Vandermon.
v1, v2, v3, v4.
That's our matrix.
Essentially I want its condition number.
And to find it's condition number, it's not symmetric.
So I can't just take the Eigen value of that.
You might say, look at Lamda max and Lamda min.
The condition number is associated with max divided by min.
But when the matrix isn't symmetric, just taking its Eigen values directly is not cool.
It's not reliable.
I could have a matrix that's badly conditioned, but all it's Eigen values were 1.
I mean a matrix was 1s on the diagonal, and zillions up above the diagonal would be have Eigen values of 1, but it would be badly conditioned.
So the right way to take it is v transpose v. Look at v1 transpose v2 transpose, transpose.
As always, if a matrix isn't symmetric, if the matrix v is not symmetric, good idea to form v transpose v. That is symmetric.
It does have positive Eigen values.
And those Eigen values, the Eigen values of v transpose v, are the singular values, or rather the singular values squared of v. So I guess I'm saying, you can't trust the Eigen values of v. It's the singular values you can trust.
And the way to find singular values is form v transpose v. That give you a symmetric matrix.
Its Eigen values, so the i'th Eigen value would be the i'th singular value squared.
So the condition number, is signma max over cignma min.
And, well, it's not enormous for this 4 by 4 matrix, but if I go up to 10 by 10 or 100 by 100.
100 by 100 would just totally wipe me out.
10 by 10 would already be disasterious.
Completely disastrous actually.
10 by 10 vandermonde matrix, with 1, 2 3, 4, up to 10 as the points it would have entries if that ended in a 10 and I had 10 rows would that be something like 10 to the ninth power.
I mean the dynamics scale would be terrible.
So finally just to v transpose v is called the gram matrix.
So that guy, Gram is coming back in as giving the good measure.
So the point was then, that this measures the dependence or independent of the vs. That ratio.
The bigger that ratio is, the more dependency they are.
What's the ratio if they're orthonormal?
What's the ratio of the q's?
what's the condition number of the qs, if we've arnoldi and got a good basis, it's one.
What's the gram matrix?
If this is q transpose q, with the qs the orthonormal basis in the columns.
Then q transpose q is the identity.
So a grand matrix.
So q transpose q would be the identity, and it's condition number is the best possible one.
Lamda max is lamda min is one.
The condition number is one.
OK.
So that's just some comments about why arnoldi gets brought in to fix the situation.
OK.
So I'll just leave those 10 arnoldi steps on the board.
The notes give 10 comments.
I'm quite proud my MATLAB is unreliable.
Actually you'll find a few errors like after end, I accidentally put a semicolon [INAUDIBLE] But the comments, I'm pleased with, and then the numerical example that runs through one cycle of this with these numbers to see what q2 is.
Why do we have a short recurrence?
This is a key point here.
And in this example, if I work out the hs, I'll discover sure enough h13 will be zero.
h14 will be zero.
Here's the key equation.
Here's arnoldi in matrix language.
Let me see if I can remember arnoldi in matrix language.
Yeah.
So arnoldi in matrix language is going to be this.
It's going to aq equals qh.
I can't write out all of q.
So that's the big equation.
Its a very important equation that we now have all the pieces for.
So a is our original matrix that we were given symmetric, let's say.
q is our basis out of arnoldi and h is the multipliers that gave that basis.
So this qh is a little bit like gram schmidt.
Do you remember gram schmidt is described by q times r. q again is orthonormal.
So its an orthogonal matrix.
In Gram schmidt r is upper triangular.
Here it's not.
Here it's hessenberg so h stands for hessenberg.
I'll write down what the actual h is for these for these numbers.
I won't write down the q. I'll just write down what h turned out to be for those numbers if I did it right.
Five halves.
Oh, interesting.
The lengths all turned out to be five halves.
And this turned out to be root 5 on 2.
This turned out to be the square root of 4 over 5, and this turned out to be the square root of 9 over 20.
And the point is from here, this one is just below the diagonal.
And it will show up as symmetric.
Root 5 over 2, root 4/5 and root 9/20.
OK.
So, what am I seeing from that particular h which somehow can't be an accident?
It must be that it's built in.
It's the fact that h is symmetric and tridiagonal.
And what does that tridiagonal tell us?
It tells us that we have short recurrence.
It's the 3 term recurrence relation is what I'm seeing here in matrix language, because there were 3 non zeros in the columns of h. All right.
I was going to write out, I going to try to understand this one by looking at the first column.
What if I take the first column of both sides?
That'll be a times q1.
q1 is the first column vector in the first basis vector.
And what do I have here when I take qs times hs?
Do you do matrix multiplication a column at a time?
You should.
OK.
So this says take 5 halves of the first column.
And this says that factor times the second column.
And I could track back and see, yes that's what arnoldi has produced.
And then the second one, the next one, would be an aq2, the next would an aq3.
Well, look, here it is.
I want to show that h is symmetric when a is.
Could you do that?
We know what the property of q is here.
We know that q is, because arnoldi made it that way, as q transpose q equal i.
And I can, in one step, show that h is symmetric if a is symmetric.
How do you do that?
I guess we need a formula for h, so I just multiply by q inverse.
So that's good.
And even better is to recognize what q inverse is.
So what is q inverse here?
Its q transpose.
Anytime we see q, that's my letter.
o is for an orthogonal matrix.
So this is q transposed, aq.
And now what?
the argument's finished.
We're here.
If a is symmetric, what can you say about that combination?
If a is a symmetric matrix?
It is symmetric.
Right?
That's how you get the symmetric matrixes.
You start with one.
You multiply on one side by a matrix, and on the other side by its transpose.
The thing has to be symmetric, because if I transfer this whole thing, what will happen?
To transpose things their transposes come in the opposite order.
So q, it's transpose comes first.
a, it's transpose comes in the middle, but what's that?
The transpose of a is a.
We're assuming a to be symmetric.
And then the transpose of q transpose is q.
So we got this back again when we transpose so it's symmetric.
So h is symmetric.
So the conclusion is, h equals h transposed.
And then we know immediately that its tridiagonal, because every h from arnoldi is hessenberg.
We know that these zeros are here.
The arnoldi cycle ended produced h i j in column j but it's stopped one below the diagonal.
So we know these are zero, but now, if we know the matrix is symmetric then we know these are zero.
And, of course, it works out that way.
So the conclusion is that we can orthogonalize the kylov basis, quickly, easily.
And work with that basis either explicitly by computing it or by implicitly by keeping things orthogonal and that's what conjugate gradients will do.
So next time, I'm going to make an effort to describe the conjugate gradients method.
I'll pick the highlights of it.
It has fantastic properties, and to verify those properties in full detail is often more confusing then not.
If you see today's lecture, you're seeing the important points; the role of symmetry of a and the arnoldi algorithm.
OK. so that's our first lecture on the krylov ideas.
And next time, we'll probably complete that topic, and it will be on the web.
So I'll see you Wednesday for the end of krylov.
Thanks.
Good.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at OCW.MIT.edu
Specific problem, and it's linear least squares problem, but it's got two terms.
So we're used to minimizing Au minus b square.
That gives us the least squares solution u hat to linear system.
And usually the reason we have to go to the least square thing is that there's no exact solution.
Probably A has more equations than unknowns.
A is long and thin, and there's no exact solution, so we look for the best solution, and we call it u hat.
OK.
But there are a lot of problems in which a second square appears.
There's also a Bu equal d hiding in the background.
And so we really have like two sets of equations.
And we multiply that second square by some factor alpha and that wise choice of alpha is usually a big part of the problem.
And I want to speak about some of the applications of this area.
So from that point of view of the normal equations, the system that you actually solve, you could say no problem.
If we knew how to do this, then we certainly can do both of them together because instead of A transpose a showing up, we'll now have A transpose a plus alpha B transpose b.
That'll be the positive definite coefficient matrix on the left side.
And then on the right side instead of just the usual A transpose b, this term is also going to give us an alpha B transpose d. All I'm saying is we don't need any new mathematics to reach this normal equation.
With the sort of the two terms normal equation.
And another way to think of exactly the same thing is we're looking at the least squares problem, where the two matrices A and B are multiplying u.
And we have to bits of data being B and d, and all were doing is the usual thing but with a weight in here.
And the weight is the identity matrix for the A part, and it's alpha times the identity matrix for the B part.
So this is our C right.
This is our C, just to say that really the notation that the formulation we have allows us to take this step, so C appears here and A transpose was CB, c appears over here too, just as always.
OK.
But there are important questions.
And of course always the first important question in applied math is what problem are you solving?
Why have we produced this class of problems?
And I have two answer.
Let me just first, so we are sure what the shape of these matrices is.
A as always has more rows than columns.
Of course u is n by 1. it's just a column right.
But A has too many equations, too many rows, for us to get an exact solution; v on the other hand has few rows.
It might even only have one more.
It's very common to add on one constraint or one term in regularizing the situation.
Anyway p is relatively small.
So the total matrix AB has m plus p rows, and the same n columns, and we're ready to go.
But the two parts are different somehow.
They come for different reasons.
And now I wrote down here two places they come from.
And these are big applications of applied math.
And one of them produces small coefficients alpha.
And what's the purpose of the Bu minus d term in that case, with just a small alpha?
The problem is that the A transpose A part is nearly singular or is singular.
So that the usual normal equation without the B would be in trouble, and this of course happens pretty often.
So to the idea of regularization is get some control of the solution by putting in another term that keeps some control over u, and stops it from just taking off as what happened where the or original normal equations would have a very large u hat.
So we're just like adding a little steady part that keeps it a bit under control.
And so the A transpose a is nearly singular in ill posed problems.
It's like giving aspirin to an ill posed problem, right.
You don't fix it, but it can operate.
OK. And where do ill posed problems come from?
And I just wanted to say that I think that fundamental ill posed problems in science is given positions.
Suppose we know that a mass let's say, is in certain positions at certain times, find the velocity.
So we often in applications have some way to know position, and want to know velocity.
And maybe you realize that that problem is not well posed, because velocity take the derivatives.
And if you take the derivative, that's not a good operator to invert.
Taking the derivative makes things very rough.
All sorts of cases we're looking for the velocity, and we only have positions.
One that I think about is from GPS.
So GPS uses space based satellites, as you all know, to give you very accurate positions.
And somehow out of those positions, you get pretty accurate but not of course as accurate as the positions, but you get decent velocities.
And how?
And there is an example where you want to know the motion of course, to ask for the acceleration would be asking for yet another derivative.
You see why the derivative is an ill posed thing?
Let me just say ahead of time, I'm going to make today's lecture about direction number two, not the ill posed problems.
So I'm just like throwing in some comments about the ill posed problem, and then I'll have a weekend to think about those.
And then next week I'll come back to this ill posed problems.
And specifically they often come from inverse problems, is a big source of ill posed problems that need regularization.
It's just a very large class of equations.
I mean I was just going to say about the derivative example.
Why is that so unstable?
Well from the point of your finite differences, if we have positions how do you estimate velocities?
You take a difference quotient, right.
You take the position at this time, the position at a close by time, and you divide by delta t. That's a reasonable start.
But dividing by delta t, that small number, is producing big numbers.
Any errors in the position are multiplied by 1 over delta t and blown up.
And similarly in frequency space, where the functions that we think about are the functions like either DI KT the derivatives brings down the factor K. So high oscillations, that's the point.
Oscillatory functions can be pretty small, but their derivative can be enormous.
So it's that oscillation which is often associated with noise in the measurements.
You know, noisy measurements are jumpy, and when we go to take their derivative or their finite difference, we get big answers.
Anyway for me that's the model ill posed problem to find velocities.
And how to do it?
I mean a lot of thought is going into that.
Let me leave it there, and come back to it.
But I say all this just to emphasize its importance.
Not that we'll completely solve it actually for GPS or for any other thing, it's just all we can do is medicate.
OK. Now this is the one that we can really solve.
So this is a different application entirely.
In this application, this second term Bu equal d, is something important, something that we want to enforce.
It's a constraint you could say.
And one way to enforce it which fits this pattern is to take alpha very large, right.
When we take alpha large, we're putting a really heavy weight on that Bu minus d square, and when we minimize, that weight will forced Bu to be pretty close to d. But of course Bu equal d doesn't determine u.
Everybody's got that picture clear?
From BU equal d has many solutions.
And so the real problem that we're trying to solve is enforce Bu equal d. But among those solutions, pick the one that minimizes the first square, Au minus B square.
So you see the difference?
You're trying to enforce something that the physics or the geometry, or whatever sources, has to be through.
And you can do it.
And you're left with lots of options.
And then the combine problem attempts to pick the right u. OK.
So that's the application number two that I want to speak about today.
And actually I want to give several ways to do it.
It's a very important problem.
And one way will be to actually solve Bu equal d. Find those solution.
And you may say well that's what we learned in linear algebra, that's the very foundation of linear algebra is there a particular solution, right.
Every solution is of this form particular plus null space.
Maybe I'll just point to the start of that approach.
So want to solve Bu equal d. And I'll come back to this method after dealing with the least squares I approached.
But here's really the direct approach.
That if I solved Bu equal d, then there's a particular solution that solves it.
And then you can always add on the general solution which is, sorry add on the null space solution - the solution of Bu equals 0.
And Bu equals 0 has lots of solutions.
So we would have to find them.
OK.
I mean that's what 1806 would naturally do, but actually never, I'm ashamed to say, but I didn't do it in 1806.
I never actually said how I would scientifically compute in a stable way the solutions.
OK.
So I think that will be important.
But that's not the only way to do it.
That's called the null space method.
And sometimes it's the right choice, sometimes not.
This would be called the heavy weight method, right.
Put on a very heavy weight and solve a standard problem.
OK.
So let me follow that one up.
And then they'll be a third method.
And maybe there's going to be space on the middle black word for it.
And what would the third method be?
That will be use Lagrange multiplier.
This thing is a constraint.
I'll enforce it by a Lagrange multiplier.
OK. That's coming next.
The way I'm enforcing it right now is by a heavy weight.
OK. One reason for the popularity of this method is you don't have to do any new thinking.
You just create these equations and solve them.
Where the other methods maybe ask us to think separately about the constraint.
Here we don't have to things separately, we just create this normal equation, we solve it, and we get an answer u hat.
Maybe I should call it u hat alpha, because it depends on the eight alpha certainly, which we hope is near the exact solution.
The exact solution being the one that exactly solves Bu equal d. Because u hat alpha will not exactly solve the Bu equal d. But we can find solutions that do, and then among those we can minimize Au minus b square.
OK.
So just a word about this heavy weight method.
OK. Well first an interesting point.
A point that I think it's sort of interesting.
I want to let alpha go to infinity and see what happens, right.
Everybody figures that as alpha goes to insanity, I'm going to get the right answer.
Because as alpha goes to infinity, it's going to more and more enforce the constraint Bu equal d. And then with that constrain enforced the other part of it will find the best u and that's great.
But let alpha go to infinity in this equation, and what happens?
So this is just like a side comment just to say alpha, you know, taking a limit you got to think about doing it right.
Well let's see, if I let alpha go to infinity as it is, that'll be infinite that infinite, I won't know what's going on.
Let me divide by alpha before I let alpha go to infinity.
So if I just divide everything by alpha -- can I do that with an eraser here?
I'll divide by alpha.
So there's a 1 over alpha here.
I divide this by alpha.
And this has a one over alpha there.
And now if I let alpha go to infinity, I get something sensible.
This goes to 0, right, alpha going to infinity getting bigger and bigger.
This goes to 0.
So what do I get in the limit?
I get that this equals this in the limit.
So shall I put that up here?
Well I'll put it here, because I don't like it frankly.
So I'll just squeeze it in this little spot.
That if I let alpha go to into infinity, so 1 over alpha goes to 0.
I get B transpose B, u hat infinity shall I call it?
Equals b transpose d. And I guess what I want to say is from that I don't learn a whole lot.
because B transpose B is a singular matrix, B transpose B is a matrix of only rank p, it's very singular right.
B had this crazy shape, long and thin.
B transpose B will be tall, B transpose B will be a large matrix, but its rank will only be p. It's an n by n matrix of rank p, and it's singular and who knows what's going on there.
That little side issue was simply to say that you can't just let alpha go to infinity and central equation there, and expect to see what's happened.
OK.
So somehow there's more to it than that.
So let me put alpha back where it belongs, and think again.
OK. And I guess by thinking again, I might as well think in terms of this way of writing it.
Because I recognized this right, this is exactly the framework that we've developed.
So this is the least squares problem.
I just want to write down the saddle point matrix that goes with this least squares problem.
What is the saddle point matrix?
Do you remember?
The saddle point matrix S. So now I've got an A and a B here.
So it's going to be larger than I have my usual 0 block.
And I have my usual A transpose B transpose block.
And what block goes there?
That's the C inverse right.
It's our usual C inverse A, A transpose 0 that we're totally accustomed to.
But now A has grown into AB, 0 is still 0, a transpose is still the transpose, and up here is transpose inverse, and since C was this, C inverse will be the identity, and the identity over alpha.
OK.
So that's my S alpha you could say.
So my equation is alpha, what's written as a block equation.
What are the pieces of it?
u is the guy that I'm looking for, the u hat alpha.
And there was a Lagrange multiplier that came.
You remember that's how we got to a block form from a scalar form.
And I guess I usually call it W, so I'll stay with W for here.
OK.
So that's what multiplies Wu.
And I think it gives a B and it gives a d from this Au and Bu, and I think here if gives a 0, because we didn't have any.
OK. What am I doing here?
I'm just writing the problem in a way where I can let alpha go to 0, and see the limit.
So let alpha go to infinity, this is transpose part.
So this will go to 0.
So this approaches the S infinity, W infinity we could call, u hat infinity is now, well you see what the limit, is that's 0 in that block.
This is A, this is B, this is A transpose, this is B transpose, this is our usual 0 block, multiplying our same W infinity, u hat infinity, equaling our same B, d, and 0.
This is the limiting equation.
And it's great.
This is the equation determines the limit, as alpha goes to infinity that it determines the best u.
This is the problem that we really want to solve.
Maybe that's what I should say.
Do you see the constraint Bu equal d in here from this middle block row?
That say 0, 0, Bu hat is d. So we've introduced the constraint.
The first part is W infinity with an Au infinity, that's the usual error term, the thing that we probably can't make 0.
And then this is the usual Legrange multiplier terms from there.
So I've spoken pretty quickly here, and let me just conclude.
This is the limit equation, is the correct limit equation.
This is the limit equation that we want to solve one way or another.
And taking alpha large is one way to get near the answer, but we'll look at other ways now.
So this is really the correct equations to solve.
The saddle point Lagrange multiplier route.
OK.
So let me summarize what I've done so far.
My problem is when Bu equal d is a constraint that I would like to satisfy, and one way to do it is to take alpha, you know, pretty near the largest number that the machine will hold, say 10 to the 15.
Put a really heavy weight on this.
But of course when you let alpha be 10 to the 15, you can see that there's like some possible problems here.
When you let alpha have an enormous weight, you're really tilting this matrix so strongly, you know, you couldn't let it be 10 to the 20 in single precision or you'd wipe out A transpose A.
So it's a balance here.
So I guess probably a lot of a numerical analysts would say wrong way to do it, the right way is solve this equation or, else do it this other way.
But a lot of people with codes say OK, you know, you're going to be a nervous nelly I'm just going to use my code.
And that's quite normal, quite human response.
OK. And this will frequently succeed.
OK.
So that's one method to do it, not the method that professionals in numerical analysis -- maybe I'm thinking for example the book by Golub/van Loan, if you know that book, that would discuss this problem.
And it would actually discuss this third method, this null space method of solving.
OK. Maybe I'll go to that null space method.
So this was one way.
Another way is solve Bu equal d. And remember again, we only have p equations we have n unknowns, so there's going to be freedom in the solution.
So we have to identify a particular solution, there's a lot of freedom in that particular solution, and then we can add to.
The null space is going to be n minus p dimensions, n minus p degrees of freedom in the null space.
That's the dimension of the null space, n minus p. I'm assuming that b has full rank p, but p is a small number compared to them.
OK.
So how do you find a particular solution?
How do you find the null space solution?
As I said, that's what I should be explaining in 1806.
And of course, we do it in 1806, but we do it with a 3 by 3 matrix, and we practically, you know, we do it by hand, where here we're talking about matrices of order thousands or millions, we don't do those by hand.
And we better not do it in an unstable way.
So the question is what's a good way to do it?
And really the heart of modern numerical analysis is orthogonalize stuff, get orthogonal vectors.
Because if you have orthogonal vectors, they don't get out of scale they.
The numbers involved don't become unstable.
And the standard orthogonalization process is Graham Schmidt, that's right, those are the words we all think of.
If I have a bunch of vectors, I have to make them orthogonal, I want to make them orthogonal, well Graham Schmidt is what we think of.
But actually math lab doesn't use Graham Schmidt, doesn't use the usual Grant Schmidt as Graham Schmidt thought of it.
Mat lab goes a different route to the same conclusion.
So let me just remind you what Graham Schmidt produced.
And let me put in the name of the numeric analyst long after Graham Schmidt, it's Householder, you know, the guy from Tennessee with good ideas.
So he had another way to the same answer, which is this factorization.
So we take our matrix, often it's A in 1806, and we factor it, we want to or orthogonalize its columns.
So the columns of A get orthogonalize into the columns of Q.
So this has the orthogonal columns.
And then of course there's some connection between the original columns and the orthogonal columns, and that connection is by triangular matrix R, upper triangular.
I don't know if you remember that from 1806.
What I typically do is I explain Graham Schmidt as they knew it, and then at the last minute I pull Q and R out as a way to express the result.
OK.
So it's the result we want, and not the particular Graham Schmidt way to get there, and Householder produces a better way to get there.
OK.
But the main point is that if a matrix has independent columns, or even if it hasn't, but if it has independent columns and we know everything about it, we can orthogonalize those columns.
Here's what I'm leading to, this B transpose I'm remembering, has this shape because B had that shape, so it's B transpose that I'm going to do Graham Schmidt, Householder use.
The command in math lab is QR equals, with Graham Schmidt we could have used the letters G and S but since we don't use their actual anymore, we could use the letter HH for Householder or something.
But it's QR of, in this case, B transpose is what we want.
OK.
Very frequently used command in math lab produces is Q and R. And it produces a square matrix Q, where these columns, the columns of the first part Q1 transpose are orthogonalize versions of these columns.
And the R just tells us the connection between them.
Then it also produces, and this is handy as you'll see, the algorithm also produces n minus p more columns, that are orthogonal to these guys.
So it produces a complete orthonormal basis, a complete set of n columns altogether.
Q1 transpose has the column that really are associated with these problems.
And these are going to be associated with the null space.
So out of this I see that actually B transpose is Q1 transpose R. So you can say this is the reduced factorization with only p columns, and this is the full picture with the other n minus p columns that are orthogonal.
And the reason that's handy is they tell us about the null space.
So now I want to identify out of this a particular solution and the general null space solution.
OK.
So what are those?
So particular solution is going to use this part.
So let's see, I want a particular solution.
So B transposing that is R transpose Q1.
OK.
So now I'm prepared to solve.
Step one is the particular solution.
I want to get be Bu particular equal d. OK.
But now I know B is nicely factor.
So this is R transpose Q1 u particular equal d. So now comes the computation the code has to do.
It has to invert that to get Q1 times u particular equals R inverse transpose d. So it had to solve a triangular system, but of course a triangular system is quick to solve.
That's a good part here.
And then the final step to get u particular, I have to put the inverse of that guy over there, because this has orthogonal column, that's just Q1 transpose.
So there we go.
That's the inverse of R. So that's what I should've done in 1806 and never did, and you get to see.
What's a convenient particular solution?
Everybody knows we got a whole selection of particular solutions.
We want to choose one that's nice and stable.
And the reason it's stable is that it works with orthogonal columns, orthonormal even, and triangular matrix for which linear systems are highly active.
OK.
So that's the particular solution.
Now what's the null space solution?
What are the general solutions to null space part?
What are the solution to those?
Well I can just go down the same steps.
This is R transpose Q1, u null space equals 0.
I multiplied both sides, this is a nice square and veritable matrix, and multiplied by its inverse kills that.
So now I have Q1.
Q1 is really the heart of B.
So what vectors are perpendicular to Q1?
I hope I've got this right.
It's easy to mix up a transpose in the process.
So let me just pause to be sure I'm doing it correctly.
OK.
I hope.
Did I check that I get it right?
Yes.
OK.
I could have written of what B is here, since I have B transpose as a product.
B is R0, Q1, Q2.
OK. And I want to multiply by u null and get 0.
OK.
So what should u null be?
This part is giving us a 0, so this is like gone.
So you see the two are the same.
So what vectors are perpendicular to those?
The answer is the u null is a combination of the columns of Q2 transpose.
It's the Q2 part that's telling us about the null space.
It was the Q1 part that gave the particular solution, it's the Q2 part that gives the general solution.
In other words, u null is Q2 transpose times any vector, let me call z, this is any z. OK. Now this has my n minus p degrees of freedom.
Sorry I'm trying to do quite a bit here.
I'm trying to say how you actually solve rectangular systems when they're not determinate.
There are many solutions.
This is a good particular solution to find, and this is a good way to find the general solution, the null space solution.
This is a combination of the other columns.
OK. All right now we're done really, because I now know what u looks like; u looks like this part which I've computed and this part which has the freedom.
Let me put those two parts together.
So now I want to minimize -- so I'm near the end here -- Au minus B, but Au is u particular, and I have u particular here.
Q1 transpose r minus transpose d, that's u particular, plus u null, and that's this.
This u null was also here; u null was any Q2 transpose z, right.
All that is u, AU minus B. OK. Up to possibly screwing up on some transposes, this is the right method.
So this is a fix solution.
I just want to write that as a different way.
Minimize A Q transpose z.
It's now we're minimizing over the z's.
So u had n components, but somehow p degrees of freedom were used up by the constraint Bu equal d. And we have the n minus p true degrees of freedom are in the z.
So there's this minus the B.
This is all known stuff, A Q1 transpose R minus transpose d square.
OK.
I'm there.
So this is a standard minimization problem.
Minimize, shall I call this A tilde z?
And I'll call all this stuff B tilde.
And the solution is found from the normal equations A tilde transpose, A tilde times the best z, I'll put a hat on it to emphasize that it's the great one, is A tilde transpose B tilde.
OK.
Finish that process without leaving myself a lot of time for the other method.
Conclusion here that after you've done the QR stepped, the QR command, and then after you solved a linear system with the r transpose, and you've multiplied by Q's, and you've ended up with this problem with a new matrix A tilde and B tilde, then you just do the normal equation.
The web will have the code that takes those steps, reaches this conclusion, and solves it.
OK.
So that's the null space method.
So z has p minus n components, n minus p components if p is near n, then they're not many z's and this is highly efficient.
OK.
So the null space method is one way to go.
Can I just in the remaining minutes go back to the Lagrange multiplier idea?
so what's the Lagrange multiplier idea?
So let me write the problem again as Lagrange would like it.
Minimize Au minus B square subject to Bu equal d. That's the problem we're solving.
I should of written it earlier.
Let me put a star here, because this is our problem.
OK.
So one way to tackle it was take that constraint give it a heavy weight.
That was method one.
Method two was solve this constraint in full detail, get the z's that remain as degrees of freedom, plug in u particular plus u null space into here, and then you have a problem in the z.
That's method two.
Now, so method three is Lagrange.
So method three would say OK, what does Lagrange do?
L, we call it the Lagrangian, e takes this Au minus b square, and adds to it some Lagrange multiplier, and all the u's are maybe the standard lambda, times Bu minus d, right.
That's Lagrange's idea.
You recognize Lagrange's idea.
Takes the constraint, multiply it by a multiplier.
In fact this is p constraints, so p lambdas.
Lambda's a vector of p multiplied.
Not just a single one, because we've got the p constraints.
And now what does Lagrange do?
He sets the derivative dL d lambda.
Well, so let me do the dL du first.
He sets dL du to 0, and dL d lamdba to 0.
I could've started out with this method, because it's going to lead us to the equations faster.
What equations do we get from dL du equals 0?
What's the gradient with respect to u?
That gives us A transpose Au.
Oh, probably we want a 1/2 here, so that numbers come out right.
We get A transpose Au, and another u part will be the B lambda.
Taking the derivative of u will produce a B transpose lambda out of that.
Yeah a B transpose lambda out of that.
And then in here will be a linear term in u that we might as well put on the right hand side as a transpose B.
Familiar.
OK. And what about dL d lambda?
Well that's just our constraint, Bu equals d right.
Having built in the constraints, when I take the derivative with respect to lambda, the constraint just comes back again.
So this is now method three.
Solve that system?
And I guess what I want say in the remaining 30 seconds is that solving this system is the same as this one.
Those two are exactly the same.
So that's a system with three parts, But maybe I can even get there.
Can you see that if I take this part and I said subtract A transpose those times the top row from the bottom row, what will that give me?
Let me just hope that it works, well I won't actually.
Time is up, it's asking too much to do even this one piece of linear algebra tat can be in the notes.
So this system that we got as the correct limit equation is exactly the same one that Lagrange gets.
So that's one way.
This is a system with n plus p unknowns.
That's the price you pay for going Lagrange's route.
You had add p unknown.
This was a system with n minus p unknowns.
That's because you're using the constraints to reduce the problem.
And the original method one was a method with n unknowns, the unknowns and u.
So you have the choice n plus p that Lagrange would like, and n minus p that Golub/van Loan would prefer.
And usually it's method two or method three is recommended, but method one often used.
OK.
So that's the lecture on this point.
That's today.
And then next week comes the whole class of problems like finding velocities from displacements, where alpha is a small parameter.
And then after that come discussions of the completed project ones, and the upcoming extensions into project two.
OK. See you next week, thanks.
Good.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
Ready to go.
So first of all, to say thank you to several people in the class who sent codes to find these wavefronts when shock, a step function is moving along with velocity c. I had roughly drawn what these three methods produced, but it was much better to see it in Mr. Sekora's movie that's on 18.086 site and sure enough, behind the moving front is an overshoot that reminds us of the Gibbs phenomenon from Lax-Wendroff and I think it's sort of unavoidable if we have a negative because one of the three coefficients at the previous time is negative.
For Lax-Wendroff, that's how it gets its second order accuracy and we see that better accuracy in a much steeper, much steeper profile.
Then Lax-Freidrichs, if you've noticed, has a sort of jagged profile.
Maybe you've realized why that would be.
Somehow Lax-Freidrichs kind of operates on a staggered grid.
We'll see.
Actually, with I've drawn a staggered grid here, so I can say what I mean.
Lax-Freidrichs computes this value from the two earlier values there and then at the next level again and so forth, so that actually, Lax-Freidrichs is using two staggered grids separately.
I think -- normally with Lax-Freidrichs, we call that n plus 1 and n plus 2 and n plus 3, but the picture is still right, so I think that's probably responsible for what you see, that there's two waves, two fronts, right?
1 delta x away from each other and finally, upwind is also smeared out.
I guess I was so pleased to see these actually showing on the website and that makes me think of more questions.
Maybe you too.
First of all, as time goes forward, we see this bump beginning to appear.
So am I right in thinking that it reaches a certain height and stays there?
So I guess I'm thinking about sort of further questions, because every time you do a numerical experiment, good additional question show up.
So I'm wondering, as this wavefront forms, this characteristic profile for these three methods, first of all, does that profile settle down to a steady profile?
Ultimately it'd be great to be able to predict what it would be.
In particular, does that bump reach a certain height, because you can see it just barely forming and then you see it growing to a certain point?
Then when the movie ends, fortunately it plots this picture and you really see the profile and you can actually see if you look closely that it goes above 1 again and probably more.
That would be, does it approach a steady profile?
Does that height settle down?
It'd be fantastic to be able to predict that height.
For example, you always want to know, how do things depend on the parameter?
The one parameter we have in this problem is r. c delta t over delta x.
If I change r, I'm changing all the coefficients.
If I changed r to be exactly 1, then the profile would be perfect, right?
That's the golden ratio, r equal 1.
I would be computing that value exactly from this value at exactly the right slope on the characteristic and perfection.
Now as r goes away from 1, I presume -- it goes down below 1.
Of course, if it goes above 1, catastrophe.
If it goes below 1, I presume the bump begins to appear, goes higher.
I don't know.
Could you see how the dependence on r appears?
Then I can suggest one more thing, which is -- let's put a forth guy onto this picture, which would be leapfrog.
leapfrog for first order equations, so let me just put down below what that would be.
So it would be on that kind of a staggered grid.
A new value at time n plus 1 would come from these at time n, really.
On this one, at time n minus 1.
So leapfrog -- you'd think of it right away.
I use a center difference in time ujn plus 1 minus ujn minus 1 over 2 delta t equals c times a center difference in space uj plus 1 n minus uj minus 1 n over 2 delta x.
So you see right away, the 2s cancel.
The delta x times c -- the delta t times c over delta x is our old ratio r, so this is easily written in terms of r again.
It does have this two time step issue and of course, we have to think about stability.
I'm writing down a fourth candidate for the equation.
Ut equals cux, of course.
Before I come back to the second order equations or systems, I just wanted to put in these comments, partly because I'm thinking about projects that are -- and I hope you are -- so some people will have [?
cici's ?]
or problems from other courses which involve solving PDEs and that'd be perfectly natural to base some project on, but if you don't have some specific application, then here are a whole lot of interesting question.
Just go further with that, so the dependence on r would be a -- the profile dependence on r, c delta t over delta x would be a question.
Let me just emphasize, what's the goal?
When I say that this upwind one is smeared out, I mean that it doesn't satisfy what you really hope for.
You want to capture that shock, that discontinuity -- not perfectly.
You can't expect that, but you want to capture it, maybe in, let's say, 2 delta x.
A good method captures the shock within 2 delta x and does it without much undesirable oscillation, physical oscillaton like that, so these are methods where we're trading off.
A good shot capture versus a smeared one, which is not satisfactory, really, in a lot of applications.
So capturing the shock within 2 delta x roughly is highly desirable and you might have to go add a artificial viscosity that we'll discuss to get there.
So that's past things, but still very much present, if I can say.
Here I've introduced a new leapfrog, a one way equation leapfrog, I'll call this, where the last lecture was about two way leapfrog.
Leapfrog for utt equals c squared uxx, a two way wave equation.
I might just mention about this one way leapfrog, once you put in e to the ikx as we always do, or e to the ikj delta x in the discrete case -- so you would put in an e to the ijj delta x times k and then find the g, the growth factor in -- that'll depend on k and will tell us the time dependence.
It'll be a g to the nth.
I think you'll find that for this guy, the absolute value of g is exactly 1 while it's stable.
Of course, if the Courrant condition is violated, then there's no way it could be stable and gs will grow, be bigger than 1.
But I think in the stable range, you will have -- this g will have absolute value exactly 1.
So it'll be on the unit circle.
You don't have much leeway, right?
We found upwind was, like, well inside, Lax-Freidrichs was well inside, Lax-Wendroff, because it had higher accuracy for low frequencies was closer and I think this guy is right on.
So this is the first order methods.
This would be Lax-Wendroff and this would be this second order method.
So you're really playing with fire there and maybe this is not a favorite, just because it's too close to the edge, in fact, right on the edge.
So those are some comments really generated by your numerical experiments and I think that's the way this course should -- homeworks and projects should develop out of numerical experiment.
Now I'll come back to complete my lecture on the second order wave equation.
So we studied leapfrog for that.
I won't repeat that, but then with second order wave equation, we have another option, which is to reduce a second order equation with a two time derivative to a system of two equations that are first order in time and space.
I've chosen to use the letters h and e as a kind of hint that we're touching a really big application, which is electromagnetism.
So e for the electric field, h for the magnetic field but here in 1 d -- so it allows me just to mention that in 3 d, when e has three components, h has three components, this is really the curl and this is really minus the curl and the notes will show the beautiful Yee mesh, staggered mesh that copies for finite difference methods, the properties of the curl, our interpretation of and the properties of -- that Maxwell discovered the physics of -- you know, current magnetic fields going around a wire that along which one an electric field is going.
But let me stay here with one dimension.
So first of all, can we recover the wave equation from that?
Let's just see that we haven't left behind the wave equation.
So I just take the second derivative -- so the second derivative -- I just want to recover the wave equation so that I haven't lost it here -- so this will be -- if I I'm taking the time derivative of the first equation, so I get d second hdtdx, right?
But bye the useful, extremely useful fact that this is the same as taking the t derivative first and the x derivative second, now I have the x derivative of this equation, so this is c -- now I have the x derivative or c times it, so now I'm up to c squared, the x derivative of the x derivative, right?
So we eliminated h, got back to one equation, second order and it's the wave equation.
I want to show the same thing now for finite differences.
It's just -- it's such a simple and useful device and our point will be that it works for -- just as well for differences.
That allows us to do this staggered grid idea.
So the staggered grid idea is, I'll do e on this grid, at these grid points, h at the half points, e again at the integer points, h again at the half points.
Now, so my difference method, leapfrog, on this staggered mesh, will copy, so the first equation will give e at the new time from the difference in h at this time.
So you see it.
The edt is going to be replaced by that time difference.
The hdx is going to be replaced by this space difference and that will determine e at the new time.
So I need time 0 and time 1/2 to get started.
Maybe I put those down here.
Somewhere I have use the initial condition, the initial velocity to get started on e and h and then I go onwards.
I've got to this point and this point.
Then this guy comes from this one, this one and this one.
Of course, that similarly comes from the same equation shifted over.
Now I know these.
Now I'm ready for this one.
Coming from -- what am I approximating now?
I'm on the staggered part of the grid, the half steps.
I'm approximating this on the half steps, right where h is defined.
So that minus that is the time difference.
This minus this -- which I know these now -- is the space difference in this equation and that's what allows me to compute the new one.
You see, it's a simple idea.
It's close but different from my scale or leapfrog, one way leapfrog here.
Here it was the same u at all the points.
here it is e at the integer time levels and h at the half integer time levels.
It's natural to say I'm calling this leapfrog.
Could I do this for the difference equation?
I just want to see that if I eliminate h -- you saw what happened there.
By using the identity that the second derivative in t and x is always the same as the second derivative in x and t, I eliminated h and got an equation that involved e only.
What's going to be the corresponding identity?
Trivial, but just -- trivial things are not bad things to notice -- for the different method.
This is what I wanted to be sure of, for the h part.
This h -- so the h -- do you mind if I draw yet another symbol?
I want to use this idea to eliminate h. So let me put two hs there.
Is that right?
Yeah.
I think what I'd like to know -- let me just say what I believe.
I believe that the time difference of the space difference of h at mesh points is equal to the space difference of the time difference of h at the same mesh points.
I meant to put these as subsequent.
it's not delta x.
It's delta sub x.
Difference in the time direction, difference in the space direction.
Let's just see what this means.
Let me redraw these four points.
These are h. h at all those points.
So that says, figure out the space differences.
So this, this minus this and this minus this and then take the time difference of those.
Can I see what I'll get if I do that?
So this means I take the space differences, so I have 1 minus 1 and that would be 1 minus 1, but now I'm going to take the time difference of those.
So that'll be this one minus this one and this one plus this one.
Do you see that?
What we have here is h at this point minus h at this point minus h at this point plus h at this point plus 1.
I claim that we have the same thing here, that if I take the time differences first -- can I do that instead of taking the space differences first?
I'll take this one, this time difference, this minus this and I'll subtract -- I'm taking now the space difference -- I'll subtract -- are you with me?
You see, it's just the same four values -- two 1s and minus 1s, whether we go first -- in fact, this actually is a -- I think maybe throws a little light on the calculus identity, that we can exchange the order of the derivatives, because we know we can exchange the order of the differences, because we just got numbers like 1 minus 1 minus 1 and plus 1.
If we let the spacing, the mesh, go to 0 this and divided by the delta x and the delta t, I suppose we would prove the continuous case.
So in a way, this throws light on something that's probably something that we take for granted, but we wouldn't want to be pressed on too closely.
So having that identity allows me to do exactly what I did with the continuous case there and show that this, when I eliminate the half levels, I've got ordinary leapfrog.
So when I eliminate half levels, I will be -- when I eliminate this half level, let's say -- and I'll be connecting -- when I eliminate half levels, I'm going to be connecting these at three levels -- one two three.
It'll be leapfrog.
So since it's been a few days since we wrote down two way leapfrog or second order equation leapfrog, let me just write it again and that's what we would get.
The time difference squared of e, it's equal will be equal to c squared times the space difference squared of e. Of course, here we have a delta x squared and here a delta t squared.
So altogether, an r squared.
That's the formula we know.
That's the formula we know.
So if I simplify this to make it look good and put the delta t squared up there, you see that it's just r squared.
Maybe it's worth realizing that again the magic r equal 1.
The magic ratio r equal 1 gives the exact u, agreeing exactly with you.
This equation, if r is 1, would be satisfied exactly by the continuous solution and so it's the same as this one.
So that's leapfrog.
So can I summarize leapfrog?
Leapfrog was, first of all, that led us to these -- it worked for second order equations.
It led us to the same stability condition that r had to be less or equal to 1 but it took a quadratic equation for g. Remember, we had g squared minus 2 g times some quantity a that involved rs and cosines plus 1 equals 0.
The reason it was a quadratic was there were two time steps involved.
We handled that.
We got stability.
If r was less than or equal to 1, that let us to a.
In magnitude, less or equal 1 and led us to g. I'll just remember those steps: a less or equal 1 and that let us to g. Yes, the notes, including more about Maxwell's equation and Yee's beautiful method m a figure that shows his m will go up -- I mean, every section is getting upgraded and this one was the next one to be upgraded.
Probably by tomorrow, you'll have a good section on second order equations.
For me, that's what I wanted to say about the wave equation alone, with no diffusion and now I'm ready to move to the heat equation.
So what do we have coming up?
Maybe important to say what's coming up.
So we've done one way wave.
Now we've done two way waves.
Next comes the heat equation, heat diffusion and then will come a mixture of the two, convection diffusion -- and by the way, so I'll go straight to these -- in a couple of weeks, we'll have a guest lecturer who will do the applications to financial mathematics, mathematical finance.
Specifically the Black-Scholes equation.
So you probably know that a lot of effort and a lot of money are going into the valuation of options and other financial derivatives and that Black-Scholes equation beautifully produced the heat equation and additional correction terms will produce a convection diffusion equation.
So this is an application that we wouldn't have made some years ago, but now it's worth focusing on that.
I think it's interesting in itself.
So this is what's coming and then you could say one way wave equations, but nonlinear.
So nonlinear wave equations and those are called conservation laws.
that's after these linear cases.
Alright.
So I'm ready to tackle the heat equation, starting then on to the next section of the notes.
So let me tackle it here.
So the heat equation.
Let me take the constant to be 1 ut p equals uxx, because we already see the big difference.
Now there's a second x derivative, but only a first derivative in time.
The dimensions of t and x are no longer the same.
We won't have delta t and delta x comparable.
Now it will be delta t comparable with delta x squared and that's a very significant difference, because that's -- if delta x is small, delta x squared is extremely small.
So when delta t is constrained by delta x squared, we're looking at small time steps.
We're looking at stiff systems, absolutely.
This is going to be a stiff problem, because the low frequencies and the high frequencies differ enormously on the decay rate.
In fact, why don't we see it right away, starting as always with exponentials.
So if I plug in u of x and t is sum g of te to the ikx, separating variables as always.
Look for a pure exponential solution, plug that in and we get dgdt times e to the ikx on the left and now I have to take two x derivatives, so that brings ik down twice.
So that's minus k squared i squared giving the minus 1 times g times e to the ikx.
Now of course, I can cancel that which is never 0 and I've separated out the time part, the g part, the dgt as minus k squared g. So g is e to the minus k squared t. Notice that it's k squared and notice that it's real and negative.
See, that's a very big this difference.
Compare e to the ikct, which it was for the one way wave equation.
The difference is enormous here.
That's magnitude 1 energy conserved.
This is magnitude smaller than 1 energy dissipated.
We had a ratio delta t over delta x.
Now we're going to see delta t over delta x squared because somehow x and t are -- now have -- it's x squared that matches t now.
So now when I put that into here -- let me just do it -- g of t is e to the minus k squared t. So there is the pure exponential.
Simple as always, because we have constant coefficients, but highly informative.
I guess another thing that I read off of that is, what frequencies are decaying fast?
High frequencies?
If k is large, k squared is very large and e to the minus k squared t is decaying very quickly.
So high frequency noise in this system, roughness, discontinuities are going to get smoothed out, strongly by moving forward in time.
Physically, what's happening is we have our wavefront, which stayed a perfect step function in the wave equation, will instantly smear.
So diffusion smears discontinuity.
Heat travels.
If we were back over here, if the temperature starts out as 1 on this side and 0 on this side, then in an instant, some heat flows from right to left.
In fact, it flows with infinite speed.
In a short delta t, there's a little heat all the way out there -- very little, of course, because it's -- I guess we want to see what that behavior is.
So I now want to write down -- staying with the differential equation, what do we want to do?
We did the exponential case and learned a good bit from that, the fast decay of high frequencies.
The next step would be to put different frequencies, different ks together by linearity, we can add these solutions.
We can multiply that by any numbers we want, depending on k. We still have solutions and we can integrate over k, so we will actually.
That that will be the next step.
Let me write down what that does.
I'm going to jump, take a little jump here to put on the board what I just said in words.
I can multiply this, e to the minus k squared t e to the iks -- that's what's happening to the pure frequency k. If I multiply that by the amount of that frequency, which is in the initial condition -- so here is the intiial condition, u 0, and I've taken its Fourier transform.
This tells me how much of frequency k is in the problem, is in the initial problem.
This tells me how much of frequency k is in the initial problem.
That amount multiplies this with this rapid decay as time goes forward, but at any later time, I'm just adding up -- and since k -- we're on the whole line -- k is a continuous -- so we add up from k equal minus infinity to infinity and I think we have to remember the 2 pi that depends on our particular definition of that transform, but I'm going to stay with this one.
That's the answer.
That's u of x and t. So I've said that in words, but actually, we could just check it.
We would like to know that this left hand side solves the heat equation and we would like to know that it has the correct start, the correct u of x and 0.
Let's check that.
At t equals 0 at the start, this is 1.
So at the start, this formula, without that thing there, is just the inverse Fourier transform that takes the Fourier transform, multiplies by this, integrates to recover u 0.
So it does start out correctly.
Should I say that again?
It starts correctly because at t equal 0, this is just the inverse Fourier transform of u 0 so it produces u 0.
It inverts this transform that we took.
Secondlty, does it solve the heat equation?
Sure, because for each k, we've just found that that solves the heat equation and now we're just putting different ks together.
So that's the answer.
So you might say, OK, perfection.
Do we need finite differences?
I guess we do.
This is typical of formulas that give the answer exactly, but in terms that are not numerically convenient.
Because to use his formula, first of all, this this requires an integral, an infinite integral to find -- integral to find for u 0. u 0, u hat.
To use this formula, we'd have to find this transform and then we have to do this integral for the inverse transform, this integral for the answer u.
Two infinite integrals and not to mention the fact that on a finite interval, when x doesn't go all the way from minus infinity to infinity, we have to think all over again.
So we have a nice formula for the answer, but -- it looks nice, but it's not numerically terrific.
We can get a lot of information out of this and there's one other special solution that's also extremely informative.
What if -- so I'm going to do a special case, but it's the big case.
What if the initial conditions is a delta function?
What if the initial condition is a delta function?
An impulse at times 0 x equal 0, an instant source of heat at a point source with finite strength, you could say.
An impulse at times 0.
For the wave equation, what happened?
That was like turning on a flash of light and it went along the characteristics.
For the heat equation, it'll be quite different.
There aren't characteristics here, but this is an extremely important solution.
I think you could call it the fundamental solution to the heat equation.
You may know what it is.
You may have seen this formula.
It's fantastic that we get a formula for this one.
You could say, we've got a formula.
What's the Fourier transform, what's u 0 hat of k for the delta function?
1, right?
The Fourier transform of the delta function is a constant 1.
All frequencies are there in equal amounts.
This is a constant, so I can remove it.
It's 1.
I'm left with a specific integral to do.
The question is, can I do it?
The answer turns out to be yes.
You can do that integral when this is a constant and that will give you the answer without starting value.
Let me just say what that answer is.
u of x and t. This is the fundamental solution.
u fundamental, maybe I should say, because it's such an important case.
is turns out to be -- it's not obvious what that integral is.
By the way, normally -- let's see -- normally it's -- to get the answer that I'm going to write down, we take a kind of end run on the integreal and I'm going to take a serious end run and write down the answer exactly.
Here's the key part of the answer.
It's a solution which starts from a delta function, spreads of course, and the shape as it spreads is a bell.
It's the famous bell shaped Gaussian distribution of e to the minus -- so is it e to the minus x squared, I guess?
Then always here comes -- that scaler tells us the width of the bell.
How far has it spread at time t?
What goes there is 4 t. That's the key observation and notice how x squared and t are coming in.
It's that ratio, x squared to t, that's crucial.
That's the parameter again.
Of course, we need to multiply by a constant.
How do I know I need a constant?
Because the total heat is conserved.
So in other words, the integral of the delta function, the original source of heat, was 1.
The integral from minus infinity to infinity of the delta function, which is all totally concentrated at one point is 1.
So I need to put on whatever constant it takes so that the integral of this thing shall stay 1 for later times.
This is one integral from minus infinity to infinity that we can do and it turns out to need 1 over the square root of 4 pi t. That solves the heat equation.
It's not a lot of fun to plug that into the heat equation, but it's possible, right?
The x derivative, the time derivative is going to be messy.
It'll have two terms.
The x derivative will have two terms and they'll agree.
The notes will give one way to do the integral and get that answer.
You see that that integral, it's -- integrals with e to the minus k squared in them are impossible on a finite interval, but here we're going all the way from minus infinity to infinity, which you might think makes the problem harder, but actually it makes it a great deal easier and we can get an explicit answer, a beautiful answer.
So we learn that the solution to the heat equation is coming from a point source, is a bell shaped Gaussian curve that gets wider and wider as t increases, but I guess you would want to look this see, what happens if t comes down to 0?
Does that really approximate -- I meant to say, does it really converge to -- this should, as t goes to 0, this should go to the delta function, and in some way, it does.
In some way, it does.
It's wonderful.
The delta function, of course, is 0 away from the origin.
So why does this approach 0 away from the origin?
Suppose x is 1.
Suppose x is 1.
Then do you see this thing going to 0 as t goes to 0?
Look what's happening.
As t goes to 0, this part is blowing up, but kind of weekly.
That's not a disastrous blowup.
Compared to this -- e to the minus -- let's say 1 over 4 t. As t goes to 0, that's e to a very negative exponent.
That's going to 0 very fast, much faster than this is going to 0.
If you like to think of l'Hopital's rule or ratio or something, this quantity is going to 0 way faster than this one is and the result is 0, except at the one point, x equals 0, where this is one.
The fact that the total amount of heat stays at 1 tells us that the delta function is the limit there.
Today was the heat equation, continuous case, with these explicit answers.
Then next time is tomorrow I guess, is the heat equation, finite differences.
Again, we'll have several difference methods and to compare them would be terrific.
Please think about that comparison with the heat equation and/or what I was saying at the beginning of class, the deeper look at the comparison for the wave equation.
See you tomorow for difference equations for this problem.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare, in general, is available at ocw.mit.edu
OK. Last time, I started to talk about the heat equation, also known as the diffusion equation.
And what we did then was to solve it.
Give an analytical solution.
Both for any initial value; u of x and zero, and then, specifically for the point source.
So then, we're talking about the impulse response, or the fundamental solution, when we start from a delta function.
And, it was that Gaussian bell shaped curve.
Fantastic fundamental solution.
But the formulas you get involving infinite integrals are not the greatest for actually getting numbers for the whole temperature history.
u of tnx.
We have to go to finite differences.
And, of course, we need finite differences also in more general cases, where non linear terms could appear.
Or even just variation in x and t in the linear co-efficient, which I had just set to be 1.
So, this is the first job.
Difference methods for the heat equation.
And you'll see that we get pushed toward implicit methods.
Because explicit method will require delta t to be that very small sized delta x squared, and that's pretty slow going.
And of course, what I'm saying applies equally to-- we might be in 2D or in 3D diffusion of pollution, for example, in environmental engineering.
Our fundamental solution from the delta function would easily extend to two dimensions or three.
And, our difference methods will extend, but we'll see what comes up.
And then, let me point to this second problem which combines the two.
We already know about one way waves coming from conviction term.
Now we're getting the basic ideas for that the diffusion term and lot of very important, real problems combine the two.
And then you have a situation that I'm just sort of pointing to here, that coefficient, as before, has the units of velocity.
Distance over time.
You see, if I bring time up here, I have time over distance there.
So, I need distance over time from c it's what we've seen always.
And, then the c delta t over delta x the quantity r was dimension.
So, I'm taking three minutes to speak about dimensions.
Dimensional analysis is a very simple but useful thing to do at the beginning of a problem.
So here we had r equals c delta t over delta x.
Which is dimensionless.
And, typically for explicit methods, it had to be bounded by 1, for example.
Now for the diffusion term d. What are the units of d?
Well, here now I have distance squared in the denominator, so d needs to be a distance squared over time.
So it has different dimensions, and the quantity capital R that will meet parallel to this, will be d delta t over delta x squared.
And, again it's dimensionless.
Because if d has dimensions of distance squared over time, that reverses it to give dimensional-less quantity, and we'll see that explicit methods require a bound on capital R, and that's what so forces us a into that small time step.
OK.
So you'll see capital R playing a similar role to little r. And actually while we're talking about dimensional analysis, back to the differential equation, suppose we want, as we certainly do, to compare the importance of the conviction term and the diffusion term.
Well that comparison is somehow the ratio of c to d, So, somehow, it's natural to think of the ratio of c:d as telling us a very important fact.
Is our problem more like a wave problem?
Do we expect to be on the unit circle, or close?
With a term that in free a space is pure imaginary, from ik, or, if d is big, we have then a small ratio, and that tells us that the diffusion term is important, that we've got lots of this dissipation That the stability, at least in the differential equation, is much greater, because this is producing a minus k squared in free a space, and a strongly negative term.
So it's that ratio c:d but of course, my point here in the dimensional stuff is that c:d isn't quite OK as it is, because they have different units.
And we need one more distant.
So we need the ratio of c -- where do I need an l?
I need another l up there.
I need another distance there to get something that is altogether dimensional-less.
And so that's the critical number.
And, it's a known as the Peclet Number Pe, I'll call it.
What is l?
We're talking here only about the differential equation.
l is characteristic length.
If our problem is set on a finite interval, then it's probably the length of the interval.
It's a characteristic length of the problem.
And, then this, quantity cl over d gives us a good measure of the importance of these two.
And, we'll see in the finite difference case, there will be cell peclet number which is entirely different.
A peclet number for the little finite different cell, where this l is changed to like a delta x. OK.
I think we've got it there.
Have I got the numerator and denominator correct there, or should they be reversed?
Is it time over distance?
It is, isn't it?
Time over distance.
That shows that I'm not a dimensional analyst.
Let me fix it.
And, then that will mean fixing this.
So, let me just fix that.
It doesn't sound right as I said it, but I read wrong.
OK.
So c is a time over a distance.
Thanks.
Is that right?
It was right the first time?
In that case, we'll move on.
OK. All right.
OK. Good.
Thanks.
Yeah.
You're right.
OK.
Sorry that's on the tapes.
OK.
So now let's begin with a natural explicit method for the heat equation.
What everybody would think of.
So, this will be explicit for the heat equations, and I'm in 1d.
And, what do you think of naturally, forward difference in time.
So, can I write it this way?
Time difference.
I'll use capital U, as always for the finite difference solution, divided by delta x, and I'm doing the heat equation with c equal to 1.
Just to keep things simple, let's just do the heat equation normalized.
So, here I'm going to take a second difference in the x direction of u divided by delta x squared.
OK. What's the accuracy?
Well, we know the accuracy of each term, the center difference, the error is the discreditization error, the local error, the truncation error, it's also called, will be proportional to -- oh I'm sorry, that's delta t of course.
OK.
So, the discreditization error in a forward difference, when I divide by delta t -- so it's first order in t, and standard difference because of the symmetry is second order in X. OK. And, this would be another problem where I could have begun with semi- discreditization.
I could have begun with keeping this as a derivattives du to t equal this center difference, and the discussion would have been entirely parallel to what we'll have here.
So, I've jump right to the finite difference in x and t. So in other words, what's happening here?
un plus 1, u at that center point, n plus 1, of course, we're our little molecule is computing this backward different, and this or rather forward in time, so this is the time direction.
n plus 1, and this center difference in space.
So if I just write out what that does, it's ujn coming from the time difference.
Then, I'll multiply by delta t, and I have a delta x squared, so that ratio is what I'm calling capital R. Capital R, to distinguish the diffusion ratio from the wave ratio.
Times what?
So it's just going to be uj plus 1 at time n minus 2uj at time n, plus uj minus 1 at time n. We're good at writing that expression down.
We can follow either the ikx, of course.
That's always the idea.
Start with uj zero as e to ikj delta x, and follow it up.
We're looking for growth factor g. The growth factor in a single step, and, of course, the point is, it depends on k. At at every step, the exponential is always with us.
It's the factor that multiplies that exponential that we want, and you see what it is, don't you?
Can you see what g is here?
So, if I put in that, and then ultimately factor out the exponential, it'll be multiplied by a 1, from that part, and an r from this, and now in parentheses, we have now very familiar quantity that comes from either the ik delta x, either the minus ik delta x.
Those combine to give to cosigned k delta x, and the center one is minus 2.
OK. That's our growth factor.
And the question is, does it stay below 1, which is the stability requirement for all k delta x.
And, so you can see the dangerous frequency as often the dangerous frequency is when k delta x is pi, and that cosign becomes minus 1.
And, then we have so stability.
So stability at k delta x equal pi needs -- so this becomes 1 and this is negative 2, and that's negative 2.
We have 1 minus 4r, and that quantity has to be, in absolute value, not exceeding 1.
Well, there's no danger that it's going to go above 1.
The danger is will it go below minus 1, and it's not allowed to.
So we need it to be greater than or equal to minus 1.
And, now, if I put the minus 1 on this side and the 4r on that side, that means 2 is greater or equal to 4r.
And, I get the famous condition r less or equal to half.
That's the stability requirement.
And if there was a coefficient d in heat equation, the coefficient would be there in r. OK.
So, that's the stability condition, and that's the thing we would like to find a way around, because, it says that this delta t -- I'll just put that in, less or equal a half, and that means a very small delta t. OK. What's the way around it?
To go to an explicit method.
So, let me go first to a completely explicit method.
Fully explicit.
I'm sorry.
To go to an implicit method.
Now, implict method -- now I'm going to do -- switch to an implicit method by computing this difference at time and plus 1.
So this is now and plus 1.
And, what's the change in g?
A big change.
OK.
So now I want to recompute g for the growth factor over a single step for the exponential.
When the right hand side, the second difference is taken at the new time.
So this has to move over to the other side of the equation.
That makes the stability better, but, of course, it's going to make the computation more difficult.
Let me just think about the stability for now.
So, instead of this board, this is the explicit case.
Yeah
[INAUDIBLE]
So it has to be, let's draw a little picture here.
So, there's zero.
Here's 1.
Here's minus 1.
And this quantity is starting at 1, and subtracting 4r.
So, the question is, does it stop here?
If it stops there, then in this picture, these would both be negative.
This would be about minus a half, but that is greater than minus 1.
So this would be the OK case when this is true.
And, that's what required r less or equal to half.
So in this case, yeah.
In that case, r is less than but if r equal to half -- everybody sees this -- if r equaled a half, this would be 1 minus 2.
This would bring us all the way here.
And if r is bigger than a half, then that would reverse that inequality, and we would be unstable.
Yeah.
I think it's OK.
Thanks.
It's the difference between looking at the magnitude --if I put magnitude, then I would need less or equal or just realizing that it's just taking it as a number.
1 minus 4.
OK.
I think that's all right.
Now the implicit case.
What happened to g?
What happened to g?
Let me put it here.
Implicit.
So, what's changed?
This term, which came from the second difference is moved over to the n plus 1 side of the equation.
So, we have in Fourier space, we would have 1 from the ujn plus 1, and then minus 2r, and times our expression 2 cosign k delta x minus 2 that's what multiplies e to the i-- do you see that that's showing up on the new side of the equation, multiplying either the ik delta x, and on the old side, the old time step, we only have the 1.
In other words, the growth factor is, this goes down into the denominator, and we can see what's going on.
So the growth factor is 1 over this expression.
1 minus 2r times 2 cosign k delta x minus 2.
We keep running into that quantity from the second difference.
OK.
So if I look at that, what's going on?
The key point of this quantity is that it's always negative, or zero, but never positive.
Because this term is never as large as this one.
So it's negative.
And now, I have another minus sign here, so I have 1 divided by 1 plus something.
And always, for all k and all ratios r it's OK.
Right?
1 over 1 plus something positive.
And you see that everything's real.
That's because we're doing the diffusion equation, where the wave equation took us, we had some imaginary part.
Absolutely stable.
Stable for all delta t. I'll say even for large delta t. That's good.
Of course, we're not going to take delta t equal to 1000.
Even if stability allowed us, the accuracy would be hopeless.
It'd be terrible.
I can't take giant time step and expect to follow the true exponentials for either the ikx.
But, I can take a larger step than this very small delta one half delta x squared step.
So this good.
What's the price that we pay?
The price is that at every time step, we have to solve.
This is now on the other side, so this brings a whole matrix over.
So, this implicit case -- when I bring this second difference matrix over, I've used capital K because the second difference matrix there probably the most interesting difference matrix, I'll say.
One of the most interesting matrixes period.
It has the minus 2s on the diagonal and 1s above the diagonal.
So, actually, with signs exchanged, I called it k. So can I just remember the meaning of this matrix k. I'm going to stay with that letter because it's a standard in finite elements for the stiffness matrix.
So, k by definition, is the matrix with the 2s on the diagonal, minus 1s above the diagonal and minus 1s below the diagonal.
And, what size is it?
On the whole line, where we're working for simplicity, I guess k is infinite in both directions.
2s and minus 1s.
On an interval where we would really compute, k would be a finite matrix.
But, the main point is that this matrix k is minus the second difference matrix.
So that when I bring it over to this side which changes that plus to a minus, I get this equation to solve.
The identity, coming from the 1, plus r, the ratio, times is this matrix k, multiplies this vector of all the values at the new time step, and equals, in this case, all the values at the old time step.
Because, at the old time step, I just have the identity.
Are you OK with this?
This is the problem that we actually have to solve.
I mean the main point about it is, that we have a linear system.
The u's at the new time step are coupled.
Coupled by the second difference.
Coupled by the matrix k, so we have to deal with it.
Well actually, in one dimension, no problem.
Because in one dimension, k is just to tri-diagonal matrix, and we can solve tri-diagonal systems almost as fast as we could work with the identity matrix.
And actually, what we're seeing this ratio are exactly the eigen values of the inverse matrix.
So, if I use a formula, I would say un plus 1 is the inverse matrix times un.
And, the growth factor is just the eigen values.
The growth factor g is simply the eigen values of this inverse matrix.
And, notice again, that all the eigen values of k are greater or equal zero.
It was a positive definite matrix, because I switched sign.
And, the effect of the identity is to push it a little more positive.
Here.
It's making -- this is a positive number there.
That's the eigen values of rk, and then, I making it a little more positive with this 1, so we have a problem that is safely invertible, stable, and in one dimension, quite simple.
So in one dimension we're entirely ready to go implicit.
Let me just, while I'm thinking about it, recall that we had some other matrix in the finite matrix case, we had some other matrixes in the very first section of the original notes, which had different boundary conditions.
In other words, if we're on a finite interval, then this matrix k corresponds to zero boundary conditions.
Zero temperature at the end.
But we might have other boundary conditions, like the derivative of the temperature is zero at one end, or the other end.
That would change this matrix in the first row and the last row to another nice matrix.
So, we might have not k but one of the other tri-diagonal matrixes.
But, here's the real point.
What happens in two dimensions?
Suppose I include now a uyy term in the heat equation, or three dimensions with uzz term.
What's different?
We can follow through the stability test for the explicit method, and I'll have a term from delta x squared, and a term from delta y squared, and a delta z squared.
It'll even slightly tighter on delta t. Or I can follow through the implicit method where now this second difference in the x direction appears also with a second difference in the y direction and a second different in the z direction.
So what's happening?
From the matrix [?
theory ?]
point of view, I still have nice matrixes with 2s and minus 1s or maybe with -- I guess what we would, -- can we just remember what we would have in a 2 dimensional case.
In the two dimensional case, instead of this second difference with a minus 1 a 2 and a minus 1 in the x direction, we also have a second difference coming from the uyy in the y direction so another minus 1 and that 2 moves up to a 4.
So our matrix in the 2d 2 space dimension case, will have a 4s on the diagonal.
So it grows from this one.
It's called the tensor product coming out of this.
Anyway it produces this famous five point molecule with 4s on the diagonal, and 4 minus 1s going down other diagonal.
And the problem is that those diagonal are not -- 2 of the diagonals might be right next to the main diagonal, but the other two will be farther away.
Because we can't number the nodes to keep all these 4 numbered adjacently at all the mesh points.
In other words, we have a serious matrix problem.
That's my comment.
We have a matrix k in 2d.
So what I'm speaking about is this problem.
Identity plus r k2d.
u at the new time equal u at the old time.
And k 2d is by no means so easy to deal with as k1d.
It doesn't have a nice narrow band like this.
The band stretches further, and we'll see in spades in then next part of the course which is about solving large linear systems.
So we have a large system because we've got lots of mesh points.
We have matrix that has a wider band, and so you need new ideas in solving it.
So this problem is what will be a model -- is the connection, really, between this section of the course, difference methods for initial value problems, and the next section which is solving large linear systems.
Because this will be a typical model larger linear system.
So that's a comment of that for implicit equations, the price was low in 1d, but the price is not so low in 2d.
OK. Now, have we got any other candidates between explicit and implicit?
Well, we did way back for ordinary differential equations.
We had a trapezoidal rule.
And what did that do?
That increased the accuracy by centering it at time and plus a half.
Do you remember that?
On the right hand side, we used half the explicit difference and half of the implicit different, so natural to do it again.
It worked then, and it'll work now.
So this will be the trapezoidal method.
But most people in the diffusion application name it after these authors, Crank and Nicelson.
So it's the Crank Nicelson method.
It's half the explicit part plus half the implicit part.
And the order of accuracy is increased by centering it this way.
That's the point of course.
So I guess the point will be we don't have any extra work compared to a fully implicit one, and we get an extra order of accuracy by centering it at a half.
So ujn plus 1 minus ujn over delta t is r. Oh sorry.
We've got delta t there.
So will be the second difference at the old time times a half.
It would just take half of that explicit part plus half of the implicit part.
This is a time and plus one over delta x squared.
You would think of that.
Everybody would.
It's a just a natural idea.
It increases the accuracy.
It's still implicit.
We still have to bring this term over on to this side when ujn moves over on to that side.
And we can look at look at the growth factor.
So that's what we have to do.
So bring that term over so and bring delta t up, so its delta t, divided by delta x squared and r. OK.
So I've got this term.
The 1.
And when I bring that term over, it's a half of what we brought over before, which was the r 2 cosign k delta x minus the 2.
That multiplies g, and on the right side, we have the 1 from the ujn, plus the half, times the r times 2 cosigned k delta x minus 2.
I'm certainly getting to the point where I write these expressions down faster without explicitly writing either the ik delta x and then canceling it.
So, g is a ratio again.
In the explicit case g was all in the numerator.
There was no denominator.
In the fully implicit case, g was all denominator, with just a one in the numerator.
Now we have g as the ratio.
So I'm going to bring this down below.
And, g is a ratio.
So it's a fraction because it's got an explicit part in the numerator and an implicit part in the denominator.
And, of course, the main question is when is g less or equal to one?
Well.
Actually it's cool because g is simply -- look, the numerator is one, and all this stuff is negative.
So this is one minus something.
And, what's going down here?
The sign's just reverse.
It's one plus something.
And this is a positive quantity, and this is the same positive quantity.
So, this is one minus, do you see this, because this is negative, I'll write it as 1 minus something -- something positive.
And here, this is minus and this is negative, so this is 1 plus the same thing.
So what's the answer?
[UNINTELLIGLBEPHRASE] One minus a positive number over 1 plus that positive number can't go wrong.
So, for all r. OK.
In a way, that has to be more attractive than fully implicit, because it didn't take more work, it wasn't any less stable, and it was more accurate.
So Crank Nicelson is sort of a natural idea at this level of analysis.
OK.
So those are the three methods to think about.
Of course actual computations will bring in new questions.
Boundary conditions, non linealarities, and everything.
I'm just staying at this simple level, where I'm comparing the first ideas that would occur to us.
OK. We may want higher accuracy, all sorts of things.
But, we're seeing the contrast with the wave equation case.
But, here we move more toward the implicit side, and the reward is unconditional stability.
No condition on r, but the price is more computation.
OK. Well at that same level of thinking about the most natural methods that occur to you, I want to speak now about the big issue of convection diffusion.
Suppose both types of problem are with us.
So convention diffusion.
What would you do then?
Now I have both convection and diffusion in the equation ut equals cux, plus duxx.
And I guess I want to say that this is something that the world has not come to a decision on.
The computational world is still debating what to do with conviction diffusion.
So it's a very appropriate equation model to experiment with.
Maybe I'll ask you, as the next kind of informal homework, to begin to experiment with that.
And when I say experiment, you can think immediately of the methods you might think of.
The first methods to try.
So the first method of course, will be explicit.
Fully explicit.
What do you expect to have happen there?
Let's just see how the two different terms each give their little push towards the stability requirement.
So explicit is going to be the time difference, forward time difference equals c times-- what shall I chose first for the space difference?
Up wind, maybe?
up wind in space?
So, this is going to be explicit and up wind.
Of course, everybody knows that refers to the windy term in the equation.
The wave term.
OK.
So, of course, I'll get a forward difference in the x direction of u at time n divided delta x, and then I'm going to do the natural one.
A centered second different at time n for the diffusion term.
OK. We'll see what comes out of that.
It's going to involve not only the ratio, it will involve both ratios.
Naturally.
Little r, as soon as I multiply up by delta x, I have a little r there.
My wave equation ratio, and I have a d delta t over delta x squared, which I'm calling capital R. That's the diffusion ratio, and multiplied by all sorts of stuff.
So actually, why don't I write down what the equation really is then?
Explicitly, its ujn plus 1 is ujn so maybe I should collect all the terms here.
Can I collect the terms that multiply uj plus 1n?
So what's going to multiply uj plus 1n?
Here I have a c delta t over delta x. I have an r from this term.
And here I have a d. Right?
No.
Sorry.
I have a capital R. Isn't that right?
When I multiply through by delta t, I have d delta t, over delta x squared.
I have a capital R that's going to multiply the -- does that look right?
And then what multiplies the center guy, the ujn?
Well that's where the time difference comes over.
There's the usual 1.
Then from this difference, this upwind difference, there's a minus at the center term, because it's a forward difference.
So that will be a minus r, and from this center different, well remember, that that's a second difference.
So, there's a coefficient 2 in the middle term, or minus 2, and when I bring the delta t up, it's minus is 2 capital R. And then, the third one, whatever this last coefficient is, is multiplying ujn minus 1, and now where does that come?
It doesn't come from here?
It doesn't come from the upwind difference.
I guess it only comes from there.
And it's probably just r. And I guess I would check any formula like that, to be sure that the coefficients add to 1.
And they do.
And why should they?
Why should those coefficients add to 1?
So that's my little test that I've got it right.
They should add to 1, because a constant initial if uj zero was a constant, if I'm starting from a constant function, it's x derivatives would all be zero.
If I had u equal constant, then the x derivatives would be zeros, so the time derivatives would be zero so that constant would stay forever.
And so I would want to get the same constant out from constants going in.
And so the coefficient should add to one.
Well.
I can see a situation when I know stability will be OK.
Stability can't fail if those three coefficients are all positive.
If those three coefficients are all positive, then my growth factor can't be larger than 1.
Those coefficients, they add to 1.
So the growth factor is this times -- should I write it down?
The growth factor will be this -- so this will be the growth factor --g will be that number times either the ik delta x, from shifting over one.
It'll be this number times 1, and it'll be this number times either the minus ik delta x, and my point is just that the easiest estimate would say, well this has magnitude 1, that has magnitude 1, that has magnitude 1.
If I just add up magnitudes, I'll get this one, and this one and this one.
And if they're all positive, they add to 1, I'm safe.
So what's the stability condition that this simple argument requires?
Well.
That's positive.
That's positive.
The danger is this one.
So I guess I would like that to be positive.
So I would want -- I'm certainly stable if this is positive, which I might as well write as r plus 2 big R being not larger than 1.
Because if that's true, I'm taking a combination with positive coefficients adding to 1, I can't go wrong.
OK. And in fact, that's pretty familiar.
It combines the two conditions we know.
The condition that r should be less or equal 1, small r, the wave Courant-Friedrich-Lewy condition.
And the condition that capital R should be less or equal to half the diffusion condition.
Now they're both.
Now I'm forcing them even more by saying that the sum has to be less or 1.
OK.
So let me leave the explicit up wind at this point, and ask you what other method might you want to try?
Well it would be natural to think of explicit centered, where are you would center this and we could follow that one through.
I guess I'll do that next time very quickly.
On the notes we'll do it.
And I'm not sure which of those two ways is better.
Or, you're going to say go implicit.
And now I'm going to ask, all right but what part of it do you want me to make implicit?
Maybe all I need to make implicit is the heat equation part.
And I can keep this explicit perhaps.
And I could even think of splitting the step.
This is something we haven't done.
I could think of taking a wave step and then a diffusion step.
Whatever.
Putting them together into one step where the diffusion step is changed to implicit because of the great stability that provide.
OK.
I think you're seeing the reality of scientific computation even at this fundamental model level of a model with two quite different physical affects of motion with finite speed, energy preserving, magnitude of g, tending to be right at 1 and diffusion at infinite speed, giving us growth factor that tends to be real and below 1, but the price is it pushes us toward implicit methods.
I think that's this one choice that I've worked through, and the others that I've mention, present a real problem.
I'll say another word about this topic next time and then we'll pretty soon move to a non linear equations and more realistic, more difficult applications.
OK.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu
Up on the web now are lots of sections, including this conjugate gradients, so today is the end of the conjugate gradients discussion, and it's, kind of, a difficult alogorithm.
It's beautiful and slightly tricky, but if I just see what the point is of conjugate gradients, we don't have to check that the exact formulas that I've listed here -- so that's one step of conjugate gradients: going from k minus 1 to k, and we'll look at each of the five steps in the cycle, without patiently checking all those formulas.
We'll just, sort of, see how much work is involved.
But let me begin where we were last time, which was Arnoldi.
So Arnoldi was a Gram-Schmidt-like algorithm, not exactly a Gram-Schmidt that produced an orthogonal basis, and that basis is the basis for the Krylov subspace, so that each new basis vector in the Krylov subspace is found by multiplying by another power of a, but that's a terrible basis.
So the basis that we get from these guys has to be improved, orthogonalized, and that's what Arnoldi does.
And if we look to see what it does matrix-wise, it turns out to be exactly -- have that nice equation, that we're creating a matrix q with the orthogonal vectors, and a matrix h that tells us how they were connected to the column, to the matrix a. OK.
So that's the central equation, and the properties of H are important.
So we saw the key property was, in general, we've got -- it's full up in that upper corner, just the way Gram-Schmidt would be.
But in case a is a symetric matrix, then we can check that h will be symetric from this formula.
And the fact that it's symetric means that those are zeros up there, because we know there are zeros down here.
We know there's only one diagonal below the main diagonal.
And if it's symetric, then there's only one diagonal above, and that's the great case, the symetric case.
So I'm going to take two seconds just to say, here is the best way to find eigenvalues.
We haven't spoken one word about computing eigenvalues, but this is the moment to say that word.
If I have a symetric matrix, eigenvalues are way, way easier for symetric matrices.
Linear systems are also easier, but eigenvalues are an order of magnitude easier for symetric matrices.
And here is a really good way to find the eigenvalues of large matrices.
I assume that you don't want all the eigenvalues.
I mean, that is very unusual to need many, many eigenvalues of a big matrix.
You're looking -- probably those high eigenvalues are not really physically close, good representations of the actual eigenvalues of the continuous problem.
So, we have continuous and discrete that are close for the low eigenvalues, because that's where the error is smooth.
High eigenvalues are not so reliable.
So for the low eigenvalues, here is a good way to do them.
Do Arnoldi, and it gets named also now after Lanczos, because he had this eigenvalue idea.
Run that for awhile.
Say if you want 10 eigenvalues, maybe go up to j equal to 20.
So 20 Arnoldi steps, quite reasonable, and then look at the 20 by 20 sub-matrix of 8, so you're not going to go to the end, to h n. You're going to stop at some point, and you have this nice matrix.
It'll be tri-diagonal, symetric tri-diagonal, and that's the kind of matrix we know how to find the eigenvalues of; so this is symmetric and tri-diagonal, and eigenvalues of that class of matrices are very nice, and they're very meaningful, and they're very close to -- in many cases -- I won't try to give a theory here, in many cases, you get very good approximations.
The eigenvalues of this submatrix are sometimes called Ritz values, and they use the eigenvalues of this submatrix, and those are often called Ritz values.
And so the first 10 eigenvalues of that 20 by 20 matrix, would be in many, many problems very fast, very efficient approximations to the low eigenvalues of the big matrix a that you start with.
OK, the reason is that eigenvalues -- do you all recognize that eigenvalues -- that this is the operation when I bring q inverse over here, that I would call these matrices similar -- two matrices are similar if there's a q that connects them this way.
And similar matrices have the same lambda, the same eigenvalue.
So the full matrix q, if we went all the way to the end, we could find its eigenvalues, but the whole point of all this month is algorithms that you can stop part way, and you still have something really good.
OK, so that's -- I think we can't do everything, so I won't try to do more with eigenvalues except to mention that.
Someday you may want the eigenvalues of a large matrix, and I guess arpack would be the Arnoldi pack, that would be the package to go to for the Arnoldi iteration, and then to use for the eigenvalues.
OK, so that's a side point, just since linear algebra is -- 1/2 of it's linear systems and 1/2 of it's eigenvalue problems, I didn't think I could miss the chance to do that second half, the eigenvalue part.
OK, now I'm going to tackle conjugate gradients, to get the idea.
OK, so the idea, well, of course, a is symetric positive definiteness.
If it's not, you could try these formulas, but you'd be taking a pretty big chance.
If it is, these are just right.
OK, so what's the main point about -- here's the rule that decides what xk will be.
xk will be the vector in the Krylov, of course, xk is in this Krylov space k. So it'll be -- we can create xk recursively, and we should need, just, if we do it right, should need just one multiplication by A at every step, and then some inner product.
OK, so now, what do we know from this?
This vector, since it has this, it starts with -- it has an xk that's in the k'th Krylov subspace, but now it's multipled by a, so that's up to the k plus first.
And b is in k1, it's in all the Krylov subspaces, so this is in, and therefore rk is in, k plus 1, right?
This is in the next space up, because there's a multiplication by a.
And therefore, if we choose to determine it by this rule, we learn that this vector -- I mean, we already know from Arnoldi that qk plus 1 is orthogonal to this space.
This is the new k plus first vector that's in k plus 1, so this is just where this is to be found, so this vector rk is found in the next Krylov subspace, this one is in the next Krylov subspace.
They're both orthogonal to all the previous Krylov subspaces, so one's a multiple of the other.
So, and that will mean that automatically, that the r's have the same property that Arnoldi created for the q's they're orthogonal.
So the residuals are orthogonal.
So, orthogonal residuals.
Orthogonal residuals, good.
I better say, by the way, when we do conjugate gradients, we don't also do Arnoldi, so Arnoldi's finding these q's; we're leaving him behind for awhile, because we're going to go directly to the x's and r's here.
So, I won't know the q's but I know a lot about the q, so it's natural to write this simple fact: that each residual is a multiple of the new q, and therefore, it's orthogonal to all the old ones.
And now I want to -- this is the other property that determines the new x.
So the new x should be -- what do I see here, I see a sort of delta x, the change in x, the correction, which is what I want to compute at step k, that's what I have to find.
And I could look at the corrections at the old steps, and notice there's an a in the middle.
So the corrections to the x's are orthogonal in the a inner products, and that's where the name conjugate comes from.
So this is conjugate directions.
Why do I say directions, and why do I say conjugate, conjugate gradients, even, if I could say.
Why did I -- conjugate directions, I maybe shouldn't have erased that, but I'll also say conjugate gradients.
OK.
So conjugate, all I mean by conjugate, is orthogonal in the natural inner product that goes with the problem, and that's given by a.
So the inner product given by the problem is the inner product of x, and y is x transposed ay.
And that's a good inner product, and it's the one that comes with the problem and so, natural to look at orthogonality in that sense, and that's what we have.
So these are orthogonal in the usual sense, because they have an a already built into them, and these are orthogonal in the a inner product.
OK.
So I wrote down two requirements on the new x.
Well, actually you might say, I wrote down a whole lot of requirements, because I'm saying that this should hold for all the previous i's.
And this should hold for all the previous i's.
And you see indices are getting in here, and I'm asking for your patience.
What's the point that I want to make here?
It's this short recurrence point, this point that we had over here when h was just tri-diagonal, so we only needed, at the new step, the only things we had to find were the -- say at the second step, we just had to find that h and that h, because this one was the same as that.
And then the next one, we -- maybe I didn't say that right.
I guess when we -- let me start again.
At the first step I need to find two numbers.
Now because h is symetric, that's already the same as that.
So at the next step I need these two numbers, and then that number is already that.
So, two numbers at every step and big zeros up here.
That's that's the key point when a is symetric.
So the same will be true over here.
We have two numbers, and they're called alpha and beta, so they're a little different numbers, because we're working with the x's now and not with the q's, so you don't see any q's written down for conjugate gradients.
OK. All right, let me just look at the cycle of five steps, so I have to give it a start, so I start with d0 as b and x0 as 0, and I guess, r0 which, of course, is b minus ax0.
If that's 0, it's also b.
So those are the starts.
So I'm ready to take a step.
I know the ones with subscript zero, and I'm ready for the next step.
I know the ones with subscript k minus 1, and I'm ready for this step, this cycle k r. So, here's a number.
So I'm coming into this -- well let me say maybe how I should say it.
I'm coming into that cycle with a new d, now what is a d?
It's a search direction.
It's the way I'm going to go.
It's the direction in which this delta x is going to go.
This delta x is going to be, probably I'll see it here.
Yeah, there is delta x, is a multiple of d, right?
If I put this on this side, that's delta x, the change in x is a multiple of d, so d is the direction in dimensional space, and I'm looking for the correction, and alpha is the number, is how far I go along d for the best, for the right, delta x.
So alpha will be determined by, probably by that.
Probably by that, yeah, it involves an inner product, product picking off a component.
OK?
So the first step is to find the number; the next step is to take that update, delta x is the right multiple of the search direction that I'm coming in with.
Then correcting r is easy; I'll come back to that; if I know the change in x, I know the change in r right away.
So that's given me, I'm now updated.
But now I have to look ahead.
What direction am I going to travel next?
So the next steps are to find a number beta and then the search direction, which isn't r; it's r again with just one correction.
So it's a beautiful set-up formula, beautiful.
And I could write, you know, because I know x's and r's, I could write the formulas, and there are different expressions for the same alpha and beta, but this is a very satisfactory one.
OK. Have a look at that cycle just to see what how much work is involved.
How much work is involved in that cycle?
OK, well, there's a multiplication by a, no surprise.
It's that multiplication by a that takes us to the next Krylov space, takes us to the next update.
So I see a multiplication by a, and also here, but it's the same multiplication.
So I see one multiplication by a.
So, easy to list, so each cycle there's one multiplication a times the search direction, and because a is a sparse matrix, the cost there is the number of non-zeros in the matrix.
OK. Then, what other computations do you have to do?
I see an inner product there, and I'm going to use it again.
I guess, and here it is at the next step, so I guess per cycle, I have to do one inner product of r with itself, and I have to do one of these inner products.
So I see two inner products, and of course, these are vectors of length 10. so that's 2n multiplications, 2n additions.
OK, this was a multiple of n, depending how far sparse a is, this is 2n adds and 2n multiplies for these two inner products.
OK. Yeah.
I guess you could say take them there.
And then what else do we have to do?
Oh, we have these updates, so I have to multiply a vector by that scalar and add, so and again here, I multiply that vector by that scalar and subtract.
Oh, and do I have to do it again here?
Have I got three?
Somehow, I thought I might only have two.
Oh, yeah, let me say two or three vector updates.
People are much better than I am at spotting the right way to organize those calculations.
But that's really a very small price to pay per cycle, and so if it converges quickly, as it normally does, this is going to be a good algorithm.
Maybe I'll -- maybe having introduced the question of how quickly does it converge, I should maybe say something about that.
So what's the error, after k step?
How is it related to the error at the start?
So, I have to first say, what do I mean by this measure of error, and then I have to say what's the -- I hope I see a factor less than 1 to the k power.
Yeah, so I'm hoping some factor to the k power will be there, and I sure hope it's less than 1.
Let me say what it is.
I think -- so this is maybe not the very best estimate, but it shows the main point.
It's the square root of lambda max minus the square root of lambda min, divided by the square root of that plus the square root of that.
And maybe there's a factor 2 somewhere.
So this is one estimate for the convergence factor, and you see what it depends on, I could divide everything by lambda min there, so that I could write that as the square root of the condition number that I'll divide by that minus 1 divided by the square root of the condition number plus 1 to the k power.
So the condition number has a part in this error estimate, and I won't derive that, even in the notes, I won't, and other people work out other estimates for the error, it's an interesting problem.
And I do have to say that this norm is the natural inner product, ek, ek squared is the inner product of ek with itself, but again with a in the middle.
I suppose that a little part of the message here, and on that middle board is that it's just natural to see the matrix a playing a part in the -- it just comes in naturally.
Conjugate gradients brings it in naturally.
These formulas are just created so that this will work.
OK.
So I suppose I'm thinking there's one word I haven't really justified, and that's the word gradient.
Why do we call it conjugate gradient.
Here I erased conjugate directions I was quite happy with, the directions or the d's, because this is the direction in which we moved from x, so conjugate directions would be just fine, and those words come into this part of numerical mathematics, but where do gradients come in?
OK.
So, let me say, gradient of what?
Well, what is -- so first of all, gradient of what is in these linear problems where ax equal b, those come from the gradient of the energy, so can I say, d of x for the energy, as 1/2x, transpose ax, minus b transpose x?
You might say, where did that come from?
Normally, we've been talking about linear systems, everything's been in terms of ax equal b, and now suddenly, I'm changing to a different part of mathematics.
I'm looking at minimizing -- so optimization, the part that's coming in April, is minimizing energy, minimizing function, and what's the link?
Well, if I minimize that, the gradient of this is how shall I write, Grad
the gradient of e, the list of the derivatives of e, with respect to the xi, and if we work that out, well, why not pretend it's scalar for the moment, so these are just numbers, then at this ordinary derivative of 1/2ax squared is ax, and the derivative of bx is b, and that rule continues into the vector case.
So what have we got here?
We've computed, we've introduced the natural energy, so this is a natural energy for the problem, and the reason it's natural is when we minimize it, set the derivatives to zero, we got the equation ax equal b, so that would be zero at a minimum.
This would equal zero at a min, so what I'm doing right now is pretty serious.
On the other hand, I'll repeat it again in April when we do minimizations, but this is the -- this quadratic energy has a linear gradient, and its minimum is exactly ax equal b.
The minimum point is ax equal b.
In other words, solving the linear equation and minimizing the energy are one and the same, one and the same.
And I'm allowed to use the word min, because a is positive definiteness, so the positive definiteness of a is what means that this is really a minimum and not a maximum or a saddle point.
OK.
So I was just -- I wanted just to think about how do you minimize a function?
I guess any sensible person, given a function and looking for the minimum would figure out the gradient and move that way, move, well, not up the gradient, move down the gradient, so I imagine this minimization problem, I imagine here I'm in x, this is the x1, x2, the x's; here's the e, e of x, it's something like that, maybe it's minimum is there.
So the graph of e of x, and I'm trying to find that point, and I make as first guess, somewhere there.
Somewhere on that surface.
Maybe, so to help your eye, this is like a bowl, and maybe there's a point, so that's on the surface of the bowl.
That takes zero, let's say.
I'm trying to get to the bottom of the bowl, and as I said, any sensible person would figure out, well, what's the steepest way down.
The steepest way down is the direction of the gradient.
So I could call this gradient g, if I want, and then I would go in the direction of minus g, because I want to go down and not up.
I mean, we're climbing, I shouldn't say climbing, we're descending a mountain, and trying to get to the -- or descending a valley, sorry.
We're descending into a valley and trying to get to the bottom.
And I'm assuming that this valley is this nice shape given by just a second degree polynomial.
Of course, when we get to optimization, this will be the simplest problem, but not the only one.
Here it's our problem.
Now so what happens if I move in the direction of -- the direction water would flow.
If I tip a flask of water on the ground, it's going to flow in the steepest direction, and that would be the natural direction to go.
But it's not the best direction.
Maybe the first step is, but so I'm talking now about conjugate gradient seen as a minimization problem, because that's where the word gradient is justified.
This is the gradient, and of course, this is just minus the residual, so this is the direction of the residual.
So that's the question, should I just go in the direction of the residual all the time?
At every step, and this is what the method of steepest descent will do, so let me make the contrast.
Steepest descent is the first thing you would think of, direction is r. That's the gradient direction, or the negative gradient direction.
Follow r until, in that direction, you've hit bottom.
So you'll go down this, you see what happens, you'll be going down one side, and you'll hit bottom, and then you will come up again on a plane that's cutting through the surface.
But you're not going to hit that at first shot.
When you start here, see water would -- the actual steepest direction, the gradient direction, changes as the water flows down to the bottom, but here we've chosen a fixed direction; we're following it as long as we're going down, but then it'll start up again.
So we stop there and find the gradient again there.
Follow that down, until it goes up again; follow that down, until it goes up again, and the trouble is convergence isn't great.
The trouble is that we end up -- even if we're in 100 dimensional space, we end up sort of just, our repeated gradients are not really in good new directions.
We find ourselves doing a lot of work to make a little proress, and the reason is that these r's don't have the property that you're always looking for in computations orthogonality.
Somehow orthogonality tells you that you're really getting something new.
If it's orthogonal to all the old ones.
OK. Now, so this, the direction r is not great.
The better one is to have some orthogonality.
Take a direction d that -- well, here's the point.
This is the right direction; this is the direction conjugate gradients choose.
This is where -- so it's a gradient, but then it removes the component in the direction we just took.
That's what that beta will be.
When we compute that beta, and I'm a little surprised that it isn't a minus sign.
No, it's OK.
It turns out that way, let me just be sure that I wrote the formula down right, at least wrote it down as I have it here, yep.
Again, I'm not checking the formulas, but just describing the ideas.
So the idea is that this formula gives us a new direction, which has this property, too, that we're after.
That the direction, it's not actually orthogonal, it's a orthogonal to the previous direction.
OK, I'm going to draw one picture of what -- woops, not on that -- of these directions, and then that's my best I can do with conjugate gradients.
I want to try to draw this search for the bottom of the valley.
So I'm going to draw that search by drawing the level sets of the function.
I'm going to -- yeah, somehow the contours of my function are, since my function here is quadratic, all the contours, the level sets will be ellipses.
They'll all be, sort of, like similar ellipses, and there is the one I want.
That's the bottom of the valley.
So this is the contour where energy three, this is the contour energy two, this is the countour energy one, and there's the point where the energy is a minimum.
Well, it might be negative, it doesn't, whatever, but it's the smallest.
OK.
So what does -- can I compare steepest descent with conjugate gradient?
Let me draw one more contour, just so I have enough to make this point.
OK, so steepest descent starts somewhere, OK, starts there, let's say, or that's my starting point.
Steepest descent goes, if I draw it right, it'll go perpendicular, right?
It goes perpendicular to the level contour.
You carry along until, as long as the contours are going down, and then if I continue here, I'm climbing back up the valley.
So I'll stop here.
Let's suppose that that happened to be tangent right there, OK. Now, that was the first step of steepest descent.
Now I'm at this point, I look at this contour, and well, if I've drawn it reasonably well, I'm following maybe perpendicular, and now again I'm perpendicular, steepest descent says go perpendicular to the contour, until you've got to the bottom there.
Then down, then up, then do you see this frustrating crawl across the valley.
It wouldn't happen if these contours were circles.
If these contours, if a was the identity matrix, if a was the identity matrix, these would be perfect circles, and you'd go to the center right away.
Well that only says that ax equal b is easy to solve, when a is the identity matrix.
So this is steepest descent, and it's rate of convergence is slow, because the longer and thinner the valley, the worse it is here, OK. What about conjugate gradients, well I'm just going to draw a magic -- I'm going to make it look like magic.
I'm going to start at the same place, the first step, because I'm starting with zero, the best thing I can do is follow the gradients to there, and now, so that was the first step, and now what?
Well, the next direction -- so these are the search directions.
That's the search direction d1, d2, d3, d4.
Here is d1, and what does d2 look like?
May I cheat and go straight to the center.
It's a little bit of a cheat, I'll go near the center.
The next direction is much more like that and then goes to the bottom, which would be somewhere like there inside that level set, and then the next direction would probably be very close.
So this is not 90 degrees unless you're in the a inner product.
So this is a 90 degree angle in the a inner product.
And what does the a inner product do?
In the a inner product, in which level sets are circles, or spheres or whatever, in whatever dimension we want.
Well, I'm not going to be, I don't want to be held to everything here.
If you see this picture, then we've not only made headway with the conjugate gradient method, which is a big deal for solving linear systems, but also we've made headway with the conjugate gradient method for minimizing function.
And if the function wasn't quadratic, and our equations weren't linear, the conjugate gradient idea would still be available.
Non-linear conjugate gradients would somehow follow the same idea of a orthogonal search directions.
So these are not the gradients, they're the search directions here.
OK. That's sort of my speech about the conjugate gradients methods.
Without having checked all the formulas in the cycle, we know how much work is involved, we know what we get, and we see a bit of the geometry.
OK. And we see it as a linear systems solver and also as an energy minimizer, and those are both important ways to think of conjugate gradients.
OK.
I'll just take a few final minutes to say a word about other -- what if our problem is not symetric positive definite.
What if we have an un-symetric problem, or a symetric, but not positive, definite problem?
Well, in that case these denominators could be zero.
I mean, if we don't know that a is positive definite, that could be a zero, it could be anything, and that method is broken.
So I want to suggest a safer, but more expensive method now for -- so this would be min res, minimum residual.
So the idea is minimize, choose xk, then minimize the norm of rk, that's the residual.
So minimize the norm of b minus axk.
So that's my choice of xk, and the question is how quickly can I do it?
So this is min res, OK. And there are other related methods, there's a whole family of methods.
What do I want to say about min res?
Let me just find min res.
Well, yeah, I guess, for this I will start with Arnoldi.
So for this different algorithm, which comes up with a different xk, I'm going to -- and of course, the xk is going to be in the Krylov space kk, and I'm going to use this good basis.
This gives me the great basis of q's.
So I turn my problem, I convert the problem.
I convert this minimization -- I look for x as a combination of the of the q's, right?
Is that how you see x equals qy?
So instead of looking for the vector x, I'm looking for the vector y, which tells me what combination of these columns the q's x is.
In other words, I'm working in the q system, the q basis.
So I'm working with y.
So I would, naturally, write this as a minimum of b minus aqyk.
Same problem, same problem, I'm just deciding, OK, I'm going to find this x as a combination of the q's, because those q's are orthogonal, and that will make this calculation a lot nicer.
OK, what's going to happen here?
Do you know that I'm going to use the property of the q's, that aq is qh?
That was the key point that Arnoldi achieved.
And the net result is going to be, of course, I have to do this at level k, so I'm chopping it off, and I have to patiently, and the notes do that, patiently see, OK, what's happening when we chop it off, chop off these matrices.
We have to watch where the non-zeros appear.
In the end, I get to a minimization problem for y, so we get to a minimum problem for the matrix h that leads me to to the y that I'm after.
So it's a minimum problem for a piece, sorry, I should really say hk, some piece of it, like this piece.
In fact, exactly that piece.
So if I ask -- all right, suppose I have a least squares problem.
This is least squares because my norm is just the square, the norm squared.
I'm just working with l2 norm, and here's my matrix.
It's 3 by 2, so I'm looking at least squares problems, in that matrices of that type -- and they're easy to solve.
The notes will give the algorithm that solves the least squares problem.
I guess what I'm interested in here is just is it expensive or not?
Well, the least squares problem will not be expensive; it's only got one extra row, compared to the number of columns, that's not bad.
What's expensive is the fact that rh is not tri-diagonal anymore, these are not zeros anymore.
I have to compute all these h's, so the hard work is back in Arnoldi.
So it's Arnoldi leading to min res, and we have serious work already in Arnoldi.
We don't have short recurrences, we've got all these h's.
OK, enough.
Because arpack would be a good code to call to do min res.
OK that's my shot at the key Krylov algorithms of numerical linear algebra, and you see that they're more subtle than Jacobi.
Jacobi and Gauss Seidel were just straight simple iterations; these have recurrence that produces orthogonality, and you pay very little, and you get so much.
OK, so that's Krylov methods, and that section of notes is up on the web, and of course, at some point, numerical comparisons of how fast is conjugate gradients, and is it faster than direct elimination?
You know, because that's a big choice you have to make.
Suppose you do have a symetric positive definite matrix?
Do you do Gauss elimination with re-ordering, minimum degree, or do you make the completely different choice of iterative methods like conjugate gradients?
And that's a decision you have to make when you're faced with a large matrix problem, and only experiments can compare two such widely different directions to take.
So I like to end up by emphasizing that there's still a big question.
Do this or do minimum degree?
And only some experiments will show which is the better.
OK, so that's good for today, and next time will be direct solution, using the FFT.
Good, thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons License.
Additional information about our license, and MIT OpenCourseWare in general, is available at ocw.mit.edu
OK. Now where am I with this problem?
Well, last time I spoke about what the situation's like as alpha goes to infinity.
And I want to say also a word about-- more than a word-- about alpha going to 0.
And then the real problems come when alpha is in between the real problem the situations these ill-posed problems that come from inverse problems trying to find out what's inside your brain by taking measurements at the skull.
All sorts of applications involve a finite alpha.
And I'm not quite ready to discuss those topics.
Roughly speaking.
I'll write down a reminder now.
What happened when alpha went to infinity?
When alpha went to infinity, it enforced this part became the important part.
So as alpha went to infinity you the limit was, u infinity, so I'll call it u infinity.
Well so u infinity was a minimizer of this one, u infinity minimized bu minus d squared in fact in my last lecture I was taking bu equal d as an equation that had exact solutions, and saying how did we actually solve bu equal d. So u infinity minimizes bu minus d. But that might leave some freedom, if b doesn't have that many rows, if it's rank is not that big, then this doesn't finish the job.
So among these if there are many and that's what were interested in.
u at infinity.
That limit will minimize the other bit.
au minus b square.
Does that makes sense somehow?
This is in the problem here, for any finite alpha, as alpha gets bigger and bigger we push harder, and harder on this one, so we get a 1 that's a winner for this one, but the trace of this first part is still around and if there are many winners then having this first part in there will give us the, among the winners, the one that does the best on that turn.
And small alpha going to 0 will just be the opposite right?
This finally struck me over the weekend, you know like I could divide this quantity, this whole expression by alpha, so then I have a 1 there, and a 1 over alpha here and now as alpha goes 0 this is the big term.
So now u shall I call this u 0.
There OK, for brilliant notation right, so this is this produces a u alpha.
In that limit it convert is to a u infinity that focuses first on this problem, but in the other limit, when alpha's going to 0 is faster and then this term is biggest.
u 0 minimizes au minus b square and if there are many minimizers among these well you know what I'm going to write u 0, right see I put a little hat there.
Did I?
I don't know I haven't stayed with these hats very much but maybe I'll add them.
u hat minimizes the term that's not so important, bu minus d square.
OK, so today's lecture is still about these limiting cases.
As I said the scientific problems, ill-posed problems.
Especially these inverse problems, give situations in which these limiting problems are really bad, and you don't get to the limit, you don't want to.
The whole point is to have a finite alpha.
And but choosing that alpha correctly is the art, let me just say why, why so I almost, I'm started anticipating what I'm not ready to do properly.
So, I'll say y a finite alpha, on Wednesday, y because noisy data.
Even with, because of the noise at best u is only determined because of the noise up to some order, say order of, some small quantity delta that measures the noise.
This is like a measure of the noise.
Then there's no reason to do what we did last time like forcing bu equal d. Then we don't, there's no point in forcing bu equal d if that equation, if the d in that equation has noise in it, then pushing it all the way to the limit is unreasonable, and may produce a very you know a catastrophic illness.
So that's when, so it's really the presence of noise, the presence of uncertainty in the first place that says OK, a finite alpha is fine, you're not looking for perfection, what you're looking for is some stability, some control of the on the stability.
OK, right.
But now, so that's Wednesday.
Today let me, go I didn't give an example, so today, two topics, one is an example with bu equal d. that was the last lectures and that's the case when alpha goes to infinity, then secondly is something called a pseudoinverse.
You may have seen that expression, the pseudoinverse of a, and sometimes it's written a with a dagger or a with a plus sign.
And that is worth knowing about.
So this is a topic in linear algebra.
It would be in my linear algebra book, but it's a topic that never gets into the 1806 course cause it's sort of a little late and that will appear as alpha goes to 0.
Right so that's what today is about, it's linear algebra because I'm not ready for the noise yet.
But it's the noisy data that we have in reality and that's why, in reality, we alpha will be chosen finite.
OK so part one then is to do a very simple example with bu equal d Here is the example OK.
So this is my sum of squares in which I plan to let alpha to go to infinity.
So a is the identity matrix and b is 0.
So that quantity is simple.
Here I have just one equation so p is 1 p by n is 1 by 2 I've just one equation u1 minus 2 equals 6, and in the limit, as alpha go to infinity, I expect to see that that equation is enforced.
So there's two ways to do it, we can let alpha go to infinity and look at u alpha going toward u infinity, maybe with our little hats.
Or the second method which is the nul space method.
Which is what I spoke about last time, the nul space method solves the constraint bu equal d which is just u1 minus u2 equal six.
OK that's, maybe I'll start with that one, which looks so simple of course just to solve u1 minus 2 equal 6.
I mean there everybody would say OK solve it for u2 equals u1 minus 6 so here is that there's a method any sensible person would use.
But this course doesn't, OK the sensible method would be u2 is u1 minus 6 plug that into the squares and minimize.
So when I plug this in, of course this is exact, and this becomes u, so I'm minimizing, minimizing u1 squared plus, what was it?
u1 minus 6 square.
So let's reduce the problem to one unknown, that this is the nul space method, nul space method is to solve the equation and remove unknowns.
Remove p unknowns coming from the p constraints and here p is 1, OK, and by the way, can we just guess, or not, guess but pretty well be sure what's the what's the minimizer here?
Anybody just tell me what u1 would minimize that?
Make just a guess maybe ?
I'm looking for a number sort of halfway between 0 and 6 somehow.
You won't be surprised that the u1 is 3.
And then, from this equation, I should learn that u2 is mine is 3 u2 not u2 infinity.
Now I've got too many u2 infinity is minus 3, anyway simple calculus if you just set the derivative, this 0 you'll get 3 and then you get minus 3 for u2, so that's the nul space method, except that I didn't follow my complicated qr orthogonalization.
And I just want to do that quickly to reach the same answer.
And to say, why don't I just do this anyway?
This is what?
This would be the row, this would be the standard method in the first month of linear algebra would be to use the row reduced echelon form, which of course is going to be really, really simple for this matrix in fact that's already in row reduce echelon form.
Elimination as row reduction is nothing to do to improve that and then solve and then plug in and then go with it.
OK, well the thing is, that row reduced echelon form of stuff you teach is not for large systems guaranteed stable.
It's not numerically stable and the option of using-- of orthogonalizing is the right one to know for a large system.
So you you'll have to, allow me on this really small example to use a method that I described last time.
And I just want to recap with an example on the small system.
OK, so what was that method?
So this is the nul space method using qr now.
Now MATLAB command qr, so what did we qr of b prime do you remember that we took that the that's the MATLAB command that eventually will, or is actually already in the notes for this section, and those notes will get updated but that's step one in the nul space method.
qrb prime and this gives me a chance to say what's up with this qr out rhythm.
I mean after lu, qr is the most important algorithm in MATLAB and the so what does it do?
b prime the transpose of b is just 1 minus Right?
OK. Now what does Gram Schmidt do to that matrix?
Well the idea of Gram Schmidt is to produce or so normal columns so the most basic Gram Schmidt idea would say, so what would Gram and Schmidt say?
They'd say well we only have one column, and all we would have to do was normalize it So Gram Schmidt would lead you, would produce the normalize saying times square root of 2.
That would be the q, and this would be the r, 1 by 1 in Gram Schmidt.
OK, but here's the point, that the q r algorithm in MATLAB which no longer uses the Gram Schmidt idea, instead uses a householder idea and one nice thing about this is that it produces not just this column, but another one, it produces a column for the, of the, it completes the basis to a full or normal basis.
So it finds a second vector so ordinary Gram Schmidt was just had one column times one number what qr actually does is ends up with two columns.
And well everybody can see what the other column in that's has length 1 of course and is orthogonal to the first column and now that is multiplied by 0.
So this is what qr does after, if we have this 2 by 1 matrix, it produces a 2 by 2 times a 2 by 1.
And you might say, it was wasting it's time, because to find this part because it's multiplied by 0 but what are we learning from the vector?
From this 1, 1 vector or 1 over square root of 2?
1 over square root of 2 vector?
What good can that do us?
It's the nul space of b.
So b was 1 minus 1, So let me just so that's the connection with nul space of b. if I look to if I look at vectors there's my matrix b, and if I'm solving bu equal d, if I'm solving bu equal d, then u is u particular .
And u nul space, and if I want u nul space, then that's where this and these whatever extra columns this might be p columns and then this would be n minus p columns that's what that's good for, and of course that column is tells me about the nul space which for this matrix is one dimensional and easy to find, OK.
So that may be just to, so you know the difference between Gram Schmidt's qr r which stops with if you had one column you end with one column, and the MATLAB house holder qr which finds a full square matrix.
OK, which is good to know and here we've we found a use for it.
Ok, So then the algorithm that I gave last time and I'll give the code in the notes goes through the steps of finding a u particular, and actually the u particular that it would find if I is happens to be 3 minus 3 happens to be the actual winner, and the, therefore, the u nul space that that algorithm would find if I went through all the steps you would see that because I'm in this special case of b being 0 and so on.
That the vector that it would choose this is the basis for the nul space, but it would choose 0 of that basis vector and would come up with that answer.
OK so that's what the algorithm from last time would have done to this problem.
I also over the weekend, thought OK, if it's all true, I should be able to use my first method.
The large alpha method and just find the answer to the original problem and let alpha go to infinity.
Are you willing to do that?
That might take a little more calculation but let me try that.
I'm hoping you know then approaches this answer, this is the answer, I'm looking for.
OK so get your mind just suppose I had to do that minimization again, now I'm not using the nul space method, so I'm not reducing, I'm not getting u2 out of the problem.
I'm not, I'm doing the minimum as it stands, and so what do I get?
Well I've got two variables u1 and u2 .
So I take the derivatives is expect u1.
I'm minimizing, everybody, when I point, I'm pointing at that top line.
So it's 2 u 1 and what do I have here?
2 alpha u 1 minus u 2 minus 6 equaling 0.
Is that-- did I take the u1 derivative correctly?
Now if I take the u2 derivative I get to u2s?
And now the chain rule is going to give me a minus sign, so it would be a minus 2 alpha, u1 minus u2 minus 6 equals 0.
So those two equations will determine u1 and u2 for a finite alpha.
And then I'll let alpha head to infinity and see what happens.
OK, first I'll multiply by a half and get rid of those useless 2s' and then solve this equation.
OK, so what do I have here?
I've got a matrix u1 is what multiplying 1 plus alpha, u2 has a minus alpha.
And this line u1 has a minus alpha, u2 has a 1 minus, minus, plus alpha, am I right?
Times u1 u2 equals-- what's my right hand side?
I guess the right hand side has alphas in it.
6 alpha and minus 6 alpha I think.
OK. two equations, two unknowns, that's what the these are the normal equation for this problem, written out explicitly and probably I can find the solution and let alpha go to infinity.
You could say, what are you doing Professor Strang to this elementary calculation, but there is something sort of satisfying about seeing a small example actually work.
At least to me.
OK so how do I solve those equations?
Well good question.
Should I, with a 2 by 2 matrix.
Can I do the unforgivable and actually find it's inverse?
I mean, it's like not allowed in true linear algebra to find the inverse, but maybe we could do it here.
So u1, u2, is going to be the inverse matrix, which is, so my little recipe for finding inverses is take the entries of, this entry goes up is up here, that entry goes down there, well you couldn't see the difference this entry stays the same but she stays in place those change sign and then I have to divide by the determinant.
So what was the determinant of this 1 plus 2 alpha plus also squared minus alpha squared?
When I get 1 plus 2 alpha and that's the inverse matrix now that's multiplying 6 alpha and minus 6 alpha, OK. And if I can do that multiplication, I have well there's this factor 1 over 1 plus 2 alpha and what do I have?
6 alpha, 6 alpha squared, minus 6 alpha square it I think 6 alpha?
6 alpha squared minus is 6 alpha squared plus minus 6 alpha, I think it's that is 6 alpha and ready for the great moment, let alpha go to infinity, and what do I get?
As alpha goes to infinity the 1 becomes insignificant, the alpha cancels the alpha so that's approaches 3 minus 3 So there you see the large alpha method in practice.
OK. And you see what, well it's something quite important here.
Something quite important and it's connected with the pseudoinverse.
The pseudoinverse so now, I want to, we got this answer.
And what I want to say is that the alpha, the limiting alpha system has produced this pseudoinverse.
So now I have to tell you about the pseudoinverse, and what it means, and basically.
The essential thing that it means is, the pseudoinverse is it gives the solution u which has no nul space component.
That's what the pseudoinverse is about.
I'll draw a picture to say what I'm saying, but it's this fact that means that this part which was this number.
Is the output.
This is the pseudoinverse of b applied to 6, 6.
You see the point?
v hasn't got an inverse.
v is 1 minus 1 for rectangular matrix.
And it's, it's not invertible, in the normal sense.
I can't find a two sided inverse, a b inverse, doesn't exist.
But a pseudoinverse counts.
So just give a MATLAB as long as I've written a MATLAB command here.
Why don't I write the other MATLAB command?
u is the pseudoinverse.
You remember that pseudo starts with a letter p so p i n v of b multiplying d. That's what we got automatically.
And it's what we get, and the reason we got the pseudoinverse.
So let me just say what was special here.
What was special that produced this pseudoinverse-- that I'm going to speak about more-- was this choice.
a equal the identity and b was 0 the fact that we just put the norm of u squared there-- well, the idea is this produces the pseudo inverse.
And if you like-- so can I say a little more about this pseudoinverse before drawing the picture that shows what it's about?
So I took this thing and let alpha go to infinity.
OK, so I could equally well have divided it by alpha the whole thing, if I divide the whole thing by alpha that won't change the minimizer are certainly the same u's will win and now I see while were out we're going to 0 and that's where the pseudoinverse is usually seen we take the given problem which does not completely determine u1 and u2 and we throw in a small amount of norm u squared and find the minimum for that, right.
So yeah.
Let me say it somehow.
I take, I take the b transpose b plus the 1 over alpha i, now 1 over alpha is still going to infinity in my, in this lecture.
So 1 over alpha this is, the whole thing is headed for 0 times the norm of u square.
This is the u1 squared plus u2 squared, OK and that inverse, that quantity inverse approaches the well, once I, I'm not giving the complete formula, but that's is what entering here and it produces leads to, may I see the vague word leads toward the pseudoinverse b plus.
Yeah and I'll do better with that.
OK, I want to go on to the picture.
OK, so right.
Do you know the most important picture of linear algebra?
The whole picture what a matrix is actually doing?
Here we have a great example to draw that picture.
So here's the picture that 1806 is, it's at the center of 1806.
For our 2 by 1 matrix.
So this is our matrix is b equals 1 minus 1 this is the picture for that matrix.
OK, so that matrix has a row space.
The row space is the set of all vectors there are a combinations of the rows, but there's only one row, so the row space is only a line I guess it's probably that line.
So the row space of b of the, my matrix, is all multiples of 1 minus 1, so it's a line.
Let's put the 0 point in.
OK, then the matrix also has a nul space.
The nul space as the side of solutions to bx equals 0, it's a line, in fact it's a perpendicular line.
So this is the nul space of b, and it contains, oh what does it contain?
All the solutions to bu equals 0 which, in this case, are all multiples of 1 1.
And just to come back to my early comment, that's what the qr, that's the extra half of the qr algorithm is telling us, it's giving us and it's a beautiful basis for the nul space.
And so the key point is that the nul space is always perpendicular to the row space, which of course we see here.
This z is what we had to compute when there were, p components and not just 1 and now where is, let's see, what else goes into this picture?
Where are the solutions to my equation bu equal d?
So my equation was my equation was u1 minus u2 equal a particular number 6 and where are the solutions to u1 minus u2 equals 6.
OK, so now I want to draw that.
Where are all the solutions to the u1, u2 plane?
OK, so one solution is take c equal to 3.
3 minus 3-- the combination 3 minus 3-- which is right there is my particular solution, so u particular, or u row space, is 3 minus 3.
That solves the equation and it lies in the row space.
And now if you understand the whole point of linear equations, where are the rest of the solutions?
How do I draw the rest of the solutions?
Well to a particular solution I add on any nul space solution.
The nul space solutions go this way.
So I add on, well this is my whole line of all solutions, so this is the line of all solutions.
And now they key question is which solution has, is the smaller?
When, so this is the idea this pseudoinverse, when there are many solutions pick the smallest one.
Pick the shortest one, it's the most stable somehow.
It's the natural one and which one is it?
OK which, so here is the origin.
What point on that line is closest to the origin?
What point minimizes u1 square plus u2 square?
Everybody can see this guy, that minimi-- so that's the pseudoinverse says wait a minute when you've got a whole line of solutions, just tell me a good one.
Tell me the special one.
And special one is the one in row space.
And that's the one, that the a pseudoinverse picks.
So the pseudoinverse of a matrix, the general rule is, and so part of the lecture was the fact that, as alpha goes to infinity in this problem, the pseudoinverse will do it, or I could say just directly what is the pseudoinverse do?
The pseudoinverse so b plus the pseudoinverse chooses, it chooses up if you like, up that's the b plus, that the solution I can't say b inverse d for everybody knows my equation is bu equal d. So this is my equation bu equal d. And my particular solution, my pseudo solution, my best solution, is going to be b plus d and it's going to be in the row space because it's the smallest solution.
So if you meet the idea of pseudoinverses now you know what it's talking about.
We've got because we don't have a true inverse, we have a whole line of a solutions, would we want to pick one, and this pseudoinverse picks this one.
It's the one in the row space, and it's the shortest because these are orthogonal because these are orthogonal u is up plus un and because those are orthogonal the length of u squared by Pythagoras is the length of up squared plus the length of un squared.
And which one is shortest?
The one that has no un.
That orthogonal component might as well be 0 if you want the shortest.
So all solutions have this, and this is the length of a the shortest one.
OK.
So that tells you what the pseudoinverse is.
At least it tells you what it is for 1 by 2 matrix.
As long as I'm trying to speak about the pseudoinverse let me complete this thought.
But you saw the idea, the thought was there are there two ways to get it again.
The nul space method that goes for it directly, or the big alpha method that we checked actually works.
So that was a point of this board here.
That's a big alpha method, it also produces in the limit as alpha goes to infinity up.
And there's a little-- it doesn't have-- if alpha was a 1000, I wouldn't get exactly the right answer cause this would be 2001 in the denominator.
But as 1,000 becomes a 100,000,000 and alpha goes to infinity, I guess the exact one.
OK, so here I was going to draw the picture, so if I draw row space, can I imagine this is a row space was dimension is the rank of the matrix, perpendicular to it is a nul space who's dimension is the rest the rank, and that was the rank that I always call r and this will have the dimension n minus r the number of this is exactly these are the two things that MATLAB found here.
These where the r vectors in the row space turned into columns, and these were the n minus r, but that was only one vectors in the nul space.
so normally were up in n dimensions not just 2 with two dimensions I just had lines in n dimensions I have a r dimensional sub-space perpendicular to an n minus r dimensional sub-space, and now b.
What does b do?
OK.
So suppose I take a vector un in in the nul space of b.
Then b takes it to 0.
So can I just draw that with an narrow this'll be 0. b u n is o, that's the whole idea.
Ok but a vector in a row space, is not taking to 0, b will take that those are the into the I better draw here column space of b. OK which I'm drawing as a sub-space whose dimension is also, is the same rank r, that's the great fact about matrices, that the number of independent rows equals a number of independent column.
So this guy heads off to some bu row space.
OK and if I've complete the picture, as I really should, there's another sub-space over here, which happen to be the 0 sub-space in this example, but usually it's here it's the nul space of b transpose.
In that example b transpose was 1 minus 1 and that it's column was independent, so there was nul space.
So I had a simple picture and that's why I wanted to draw you a bigger picture with it, it's dimension will be, well not n minus r, but if b is n by n, let's say, then it turns out that this nul space will be have [?
dimension ?]
n-r no problem.
OK now in the last three minutes I want to draw the pseudoinverse.
So what I'm saying is that every matrix b, every rectangular square matrix b, has these four spaces.
Four fundamental sub-spaces they've come to be called.
OK and the nul space is a bunch of is of vectors would be takes to 0 B takes any vector into it's column space, so now let me just draw what happens to u equal u nul space plus u row space.
So this was a guy in the row space.
if I backed up, b, what will b do it multiplies this vector This vector has a part that's in a nul space, and a part that's in the row space, but when I multiply by b what happens to this part?
Gone when I multiply that by b where does it go?
There, so this, all these guys feed into that same point.
bu is also going there.
That's why it's not invertible Of course.
That's why it's not invertible.
Here I guess, yeah, you're up here I, so sorry a yeah.
This was a nul space b. I didn't write in what was the nul space.
OK, so the matrix it couldn't be invertible, and actually cause it has a nul space, and they all send those, so what is a pseudoinverse finally?
last moment this pseudoinverse is the matrix it's like an inverse matrix that comes backwards.
Right?
It reverses what b does.
What it cannot do, is reverse stuff that's appeared at 0.
No matrix could send 0 back to u n right?
by multiply by the 0 vector I'm only going to get the 0 vector.
So the pseudoinverse has to what it can do, it can send this stuff back to this.
This is what the pseudoinverse does.
if I had a different color chalk I would use it now but let use two arrows or even three.
This is what the pseudoinverse does.
It takes the column space and sends it back to the row space.
And because these have the same dimension r, the point is inside, b is this is this r by r matrix that's quick, that's cool.
It's totally invertible.
And b plus inverts it.
So from row space tp columns space goes b from columns back to row space, comes the pseudoinverse but I can't call it a genuine inverse because all this stuff, including 0.
The best I can do is send those all back to 0.
There now I've really wiped out that figure but I'll put the three arrows there that makes it crystal clear.
So this, those three hours are indicating what the pseudoinverse does.
It takes the column space.
It's column space is the row space.
The column space of b plus is the row space of b.
You know they're sort of and these two spaces that's where the pseudoinverses ally and b kills the nul space and b plus kills the nul space-- the other nul space.
The nul space of b transpose.
Anyway that pseudoinverse is at the center of the whole theory here.
If I you know when I take out books from the library about regular rising these squares they begin by explaining the pseudo inverse.
Which as we've seen is arises as alpha goes to infinity or 0 whichever n were at and what I have still to do next time is, what happens if I'm not prepared to go all the way to the pseudoinverse because it's it blows up on me.
And I after and I want to find finite alpha what should that alpha be?
And that alpha will be determined by as I said by the somehow by the noise level in the system, right?
And just to emphasize another example that I'll probably mention you know ct scans, mri all those things that are trying to reconstruct the results from limited number of measurements, measurements that are not really enough to perfect reconstruction, so this is the theory of imperfect reconstruction.
Like if I can invent an expression, having met perfect reconstruction in the world of wavelets and signal processing this is the subject of imperfect reconstruction and I'll hope to do justice to it on Wednesday.
OK thank you.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
So you see that I thought maybe we've got enough good examples in theory where we need some practice.
We need some experiments with some of the finite difference methods that we've spoken about.
So I created this homework in which you would like choose one of these that we've done.
Well, actually, number two, the Schrodinger equation, which has that imaginary i in there we haven't spoken about yet, but you could do the kind of Fourier analysis that we've done for the heat equation.
But, of course, it's going to be different because of that i.
You'll get an e to the minus i k squared t, I guess.
And that means that's a number of absolute value 1 instead of a rapidly decaying coefficient.
OK, so my idea is to choose one of those groups, or number five, if you want.
And say, by next Friday, just give me some experiment with a code.
In this case, you could use -- well, we have four methods and three of them are already in code, and you're welcome to use those or your own.
And I raised questions last time -- and I'll put this on the website with a little more detail.
So just to give you an idea of what's coming.
So this is the problem that we're in the middle of speaking about, near the end, the mixture of convection and diffusion.
And this is Schrodinger, which is new.
This is the advection equation, or the one-way wave equation, which we've started with, and this is what's -- this'll be the subject of today's lecture and next time.
And the best reference I can give, as well as the textbook, the applied math book, has significant discussion of this conservation law.
But also it's very well presented in the 16.920 lecture notes that are on OpenCourseWare, lectures 11 and 12.
So I'm going to use those as a guide in this lecture.
Anyway, I thought I'd just put that up as, you know, time to see what various difference methods will do, And again, it'll be on the website.
OK. Now to say another word about this one, this convection diffusion equation.
So last time I wrote down a difference equation very much like that except what I wrote down last time was upwinded so that was just j.
Now it's j minus 1.
And the choice there is not clear.
This gives us higher accuracy, of course, for that x difference because it's centered at j.
And we still have this centered, of course.
The second difference is centered.
So I wrote down here what the coefficients are of the three old values at j plus 1, at j, and at j minus 1.
But there is one important point here.
The price for that extra accuracy may possibly be a negative coefficient.
This could be negative, or it could be positive.
It depends which is more important: the small r, the convection part, or the big R. And let me follow up.
So you remember the price.
If we see a negative coefficient, then we expect some oscillations to appear.
And actually, they were supposed to appear in Figure 5.12, but it hasn't been done yet.
And if you would choose this one and do Figure 5.12, I'll be incredibly grateful.
So Figure 5.12 was to show the evolution from a u naught for the convection diffusion.
OK, so can I just -- do you see that in comparing R with r over 2, those are dimensionless quantities, right?
Can I take another minute to try?
I'm a total amateur at dimensional analysis.
I did always think, however, that Einstein could have proved -- well, not maybe proved, but could have conjectured that e was mc squared, the most important equation in relativity, just by checking the dimensions.
I don't know if that would -- I mean, right?
That would like -- might have occurred to him to realize that mc squared has the dimensions, if we wrote those out, of energy.
I don't know if that put an idea into the system which produced, of course, the fact that -- and, of course, there would be a dimensionless constant, which amazingly happens to be 1.
OK, so dimensional analysis could like change the world.
This won't quite change the world, but a little touch of something.
OK, so can I just -- you remember that this Peclet number, and I'll write his name down again.
P-E-C-L-E-T.
The Peclet number.
In the differential equation, the Peclet number was a ratio, essentially a ratio, of c to d. But to get the -- I think we needed -- was that what we needed?
We needed a length to make it dimensionless.
c is the coefficient for -- c over d sort of tells us roughly the importance of convection and diffusion, but there's a length scale to be included.
Now the cell Peclet number, which I'll just call P, I'm just going to take to be r over R because those are dimensionless.
And you see that it's going to play a part here.
Let me see, I forgot.
I think I want r over R. Maybe I want a 2 just because I would like that to -- I'd like it to come out simply there.
OK, so I may put in a 2.
So that are would be 2R p. Where did that 2 come from, by the way?
Oh, I guess it came from there, right?
It came from that minus 1/2.
So probably I want that 2 there just so that I have a very simple decision of whether this coefficient is positive or negative, yeah.
So let me put in a 2.
OK, so r over 2 is RP then.
This quantity is R times P, so this is R minus R times P, or it's R, which we know is OK, times 1 minus P. In other words, P smaller than 1, that coefficient -- all the three coefficients are positive.
It's certainly stable.
Not only stable, it can't oscillate.
We can't have -- we're taking a combination of the three old values and oscillations can't show up.
OK, so just to see, what is this?
And you remember that r is c delta t over delta x, and capital R is d delta t over delta x squared.
So just to complete this, the delta t's cancel.
One of the delta x cancels.
The other one comes up.
So it's c over d, and then we have a delta x and a 2.
So that's why it's called this cell Peclet number because it's the cell size, or in my convention, half the cell size that gives the length scale and decides whether P is -- so I'm just going to write down the conclusion that the figure should show.
P smaller than 1.
That means that this coefficient is also positive.
So that's positive coefficients, no oscillation.
And I think you'll see from the output that P bigger than 1 produces oscillation we don't like.
In other words, we really don't want P larger than 1.
And of course you could say, well, look at P. It's got a delta x in it.
That's not going to be a big number.
And the method will work fine as delta x and delta t go to zero, but we're going to do a finite calculation.
And so we're seeing for the first time, like, because we have two terms that are not dimensionally identical, c and d, that in that competition between convection and diffusion, the length scale is crucial, and the mesh size is crucial.
So I guess the answer is we can resolve it by taking delta x small enough, and hopefully, that's not a problem.
But, of course, you know that there are many physical cases in which the diffusion coefficient or the viscosity coefficient is small, is quite small.
And so getting that below 1 might be hard because this d could be a very small number.
OK, so maybe can I also mention on the -- as something that we haven't done here, was to move the second derivative, this guy, to time n plus 1, make it implicit.
In the diffusion part, one often does that.
The diffusion is the term, is the one that's forcing us to take this coefficient R below a half, delta t of order delta x squared, and the way to avoid that is to move the viscosity term, at least half of it, up to the new time.
OK, so that's a third method.
So we've done upwind last time, centered, explicit this time, but then that's a third one that has to be looked at.
OK, so that's where we stand on linear problems.
I'm sort of ready to jump into nonlinear ones, recognizing that -- well, there's always more to say about linear ones.
What are some of the things that we have not properly discussed?
Boundary conditions, above all.
Boundary conditions, and we could do something with those, and the notes will say more about boundary conditions.
So there are several types of boundary conditions.
I guess I'm going to say something about them now, but this'll be a board that kind of gets covered up.
So boundary conditions: So what are the different types?
Well, at a real physical boundary, we might have u equals 0, say, in heat flow.
We might hold the boundary.
It might be the edge of a freezer or something.
We might hold the temperature at zero or at some given value.
We might prescribe u at a boundary, or we might prescribe the derivative.
And just in case we're in several dimensions, I'd better write it as the outgoing derivative.
It could be prescribed, say, 0.
So what are those two types of boundary conditions called?
One's called absorbing, an absorbing boundary, and the other is an insulated boundary where no heat goes through.
To maintain this, heat probably does go through.
If our body is warm and we're holding it at zero on its boundary, then heat is going to travel out.
And in fact, as time goes to infinity, we're going to approach cold -- a uniformly cold.
Here, heat's not allowed to travel out, so what will happen as time goes on is it will approach a constant.
The temperature will approach a constant because it can't leave and it diffuses.
And now I wanted to mention a third type of boundary, which is a purely computational boundary.
We talked last time about Maxwell's equations in free space, and the big application of Maxwell's equations, or one of the big ones historically, has been to -- like the exterior of an airplane, to radar, or other electromagnetic signals.
So what do we do if we have an exterior problem?
See, here, I was always thinking about an interior problem.
You know, we're inside 0 to L or something.
But now, what do we do if a region is the outside, it's free space all the way?
Well, we have to create a computational box of some sort.
So we create a box in which we compute, but the boundaries of this box, the x-boundaries of it, are purely artificial.
They're just because we can't compute to infinity.
So waves travel in this box, and they travel to the -- so you see what I mean?
We're solving it in all this space, except that's too much, so that's meant to be a very large box, not a small one, as large as we can afford.
But waves are still going to hit the boundary of that box.
OK, what do we choose as boundary condition there?
We don't want it to reflect back because it's not a real boundary.
So we have to create -- I'll put ABC for absorbing boundary condition.
Absorbing.
That A is for an absorbing boundary condition.
And the creation of these boundary conditions is an important topic within finite differences.
For Maxwell's equations, a good idea was produced and keeps being developed, called perfectly matched layer.
The idea in that perfectly matched layer -- let me just write PML -- and I haven't seen this so much in other application areas.
The idea of this perfectly matched layer is to create an artificial dielectric, I guess, a dielectric medium it would be in the electromagnetic case.
An artificial thin layer, which has just the right properties to stop the wave and swallow it.
OK, I'll put on the website references to that.
It's a bright idea, but not totally simple, and when we change difference methods, then that perfectly matched layer has to go with it.
OK, that's the end of my thoughts about linear problems for now.
We're coming back, of course, in a few weeks to solving large systems, large linear systems, the kind that we would meet if we use an implicit method, or the kind that we meet if we're solving a steady state problem.
But for now, let me go to that model equation.
And you see right away the difference between that equation and the advection equation that had a constant velocity.
Here we have of a velocity that -- so c is minus u, I guess.
c is minus -- think of c -- this u as being comparable to minus c. So, in other words, if u is positive, I'm now going to have waves going to the right.
So let me just put this down.
u greater than 0 will mean waves going to increasing x to the right.
And the idea will be -- so we have to discuss that problem.
And as always, we would like to solve it analytically.
Find whatever formulas we can, understand what's happening in terms of these characteristic lines because when we have one equation and one space variable, we really can catch the essence of the solution by understanding the characteristic lines.
And then how do we do it numerically?
OK, before class, I wrote just a few equivalent forms of it.
This is an equivalent form, and this is, so to speak, the right form.
This is the form where we see a time derivative of u, as always, and we see a space derivative of u squared over 2, which, of course, produces that.
But somehow this is the quantity, it's called the flux, that's physically meaningful, f of u.
So I've chosen a simple flux function, u squared over 2.
So that's the differential equation.
But you'll see that the differential equation can produce two solutions at the same point, and we have to choose.
In other words, the differential -- the solution can become discontinuous.
A perfectly smooth starting function, after awhile -- this is the essence of the problem now.
I might have a smooth starting function, at least continuous.
The solution is constant along characteristic lines.
But what happens if two characteristic lines run into each other?
I've drawn that possibility here.
And it'll take a little time to see exactly what's happening, but the characteristics we can see here.
So the starting values were 1 up to that point.
So when the starting value is 1 and u is constant along characteristic lines, then u is 1, and those lines, it's just like having c equal to minus 1, I guess, in the one-way wave equation.
We have a one-way wave going to the right now along characteristic lines.
So u is 1 in this part of the x, t plane, right?
We have to come back to the formulas to confirm what I just said.
And then over here on the far right, the starting value is 0, and the characteristics in that case, the c associated with that is 0, so it's standing.
It's not moving.
The characteristics are just straight lines in the time direction.
Nothing's happening, right?
When you think of a characteristic that's going straight up, it's just carrying the value of u without moving left or right, so u stays 0.
So u will be 1 here, u will be 0 here, and now something's happening in here, and we have to figure out what it is.
But maybe I can anticipate the main point before any algebra.
The characteristic lines in this region have a slope somewhere between the slope 1 and the slope 0.
You see, we're starting -- let me graph u0 of x.
So let me graph.
u0 of x is -- the key points are 0 and 1, 0 and 1.
So u0 of x, I start with a constant initial function, a constant profile to the left of 0, and a constant profile to the right of 1, and linear between.
OK, so it's that starting values that I wrote here: 1, 1 minus x, and 0 are these three pieces.
And then here I'm in the x, t plane where here I'm in the x, u plane.
So I drew characteristics, and we still have to track.
We have to say more what characteristics are and understand that these have slopes in between, but you're not surprised because the values of u are in between 1 and 0.
So they go that way and they meet here.
So up to time t equal 1, characteristics tell everything.
Up to t equal 1, I know that u will be 1 along these characteristics, and when I find these characteristics, u is whatever it starts at.
And u is 0 up all of these characteristics, so up to that time, algebra -- the differential equation is going to be totally OK.
But after that time, the differential equation is in difficulty.
Because at that moment t -- so what's happening as t increases from 0 to 1?
This is the picture at t equals 0.
The picture at t equal to 1 -- t equal to half, let's say.
What would be the picture at t equal to half?
Well, this t equal to half, this state 1, has penetrated this far to halfway.
Then in here is this converging characteristic that come from a linear profile.
And over here, it's all zeros.
So that zero is still there, but you see what's happening.
This guy is -- the profile is steepening.
This is at t equal to half.
Steeper.
We still have the algebra to do, but I think the first thing is to see roughly what's happening.
Then at t equal to 1, this is totally steep.
So this is the situation at t equal to 1.
And I could've given you that as the initial condition, but I, in the 16.920 notes, thought, OK, let's have a little peace for awhile, see what those characteristics are doing when they're converging, but they're not crossing.
The trouble is, after t equal to 1, they are crossing.
And now, what's the solution in this crosshatched region?
Do I take the value of 0 that's coming up along these characteristics?
Do I take the value 1 that's coming along these characteristics?
What's happening in here?
So there's a crosshatched region there, which is a big question mark because my rule of following characteristics has led me to two answers.
And we want a physically relevant answer.
We're sort of expecting, and we'll see, that this step function is a step function.
I mean, the answer will be a step function.
It will be 1 and then drop to 0 along some path in the x, t plane, which is not a characteristic.
The characteristics are going at 45 degrees and at 90 degrees.
And actually, so this is the region I'm interested in.
So can I blow up that region?
So if I blow up that region, that's where the shocks -- the shocks are meeting everywhere in this region.
This was the point.
What was that point?
I guess that was the point x equals 0 and t equal to 1 there.
OK, and the question is -- and I know that I have u equals 0 over here, and I know -- oops, u equal 1 over here, and I know that I have u equal 0 here, but the question is where and -- well, what happens in between?
You can think of at least two possible scenarios, and one of those is that there's a shock line that goes, let's say, somewhere in the middle.
And so u stays 1 all the way to that line and drops to 0 after it.
In other words, this wall of water travels to the right.
The shock, the discontinuity, moves along, not necessarily at the speed that this one.
This would be moving along the speed of 1 with it pushing.
Not stationary either, which is what this right side would like, but somewhere between.
So that's a shock line.
I'll just, for short, call it a shock.
Then there's another possibility.
And that will happen in this case, actually.
That will happen in this case.
But there is another possibility that will happen in other cases, and that would be a fan of characteristics.
Let me see that when it comes, but I'm just mentioning it to say that this business of the shock line, which is what we'll pursue now, which because it's what happens with this problem, is not the only -- is not always the way to match, the way to deal with a problem of lack of information coming from the characteristics.
OK, so I guess our question is, what is that shock line?
What's the path?
What's the slope, and what controls it?
And the answer will be that what controls -- we run into problems with the differential equations.
And the way to see what continues to happen is to go to the integrated form, the integral form of the equation, where, of course, by taking an integral, I make everything smoother, and also I preserve the physical law, the conservation law.
So this is really the conservation law.
What is this?
Well, let's first of all see where it comes from.
It just comes from integrating this equation with respect to x.
So I have a time derivative that I keep out here, but my space derivative is going to disappear because I'm integrating with respect to x.
So I can call d by dt or partial d by dt the integral of u with respect to x, and I'm integrating from somewhere -- from some point to some other point.
Instead of a and b, I'll call them -- I really should -- I'm sorry, I should say x on the left and x on the right.
Some point x at the left and some point x at the right.
And now I've integrated this one.
I've called -- this is f of u so I'm now thinking of any flux function.
And the integral, of course, gives me f of u at the upper end minus f of u at the lower end.
OK, so that's the conservation law.
Let me write that word down.
This is the conservation law.
What is it saying?
It's saying that the change in the quantity in the integral is given by -- if there's any change, any time derivative, the rate of change in the interval -- it's the conservation of this integral meaning that any change in that integral happens because there is flow going out the right-hand side of the integral or there's flow coming in the left side of the interval.
So that's why we have a minus sign between them.
So that's a statement of conservation, and -- oh, there's one more example that I'll mention now.
Let me give this particular equation a name, just because you see it.
You often see it written -- called Burger's equation.
But actually, the full name is the inviscid, no viscosity, Burger's equation because it's a 0.
So just one word about Burger's equation.
So Burger allowed a uxx term here.
So he had one of our -- nonlinear convection and a diffusion term, and his convection was this simple expression uux.
It's the kind of thing you see in the Navier-Stokes equations, right?
The nonlinearity in Navier-Stokes is sort of in this term of this general sort, but this is a major simplification.
Such a simplification that Burger could solve, even though the equation was nonlinear.
Well, two people at the same time discovered the change of variable, a simple little trick.
Hopf and Cole: I'll put their names down because they had nice, short last names.
They both discovered -- in the case of a diffusion term, they both discovered a little -- a change of variables that made the equation linear.
Then they could solve the equation.
That's in the textbook in the section on conservation laws.
And then the important thing that Burger could do was let the diffusion coefficient go to zero.
So then in the limit, he got the inviscid Burger's equation, the one we're looking at.
And he could look at the solution and let the limit go to zero.
And that's another way to figure out what's the right solution.
That's the viscosity method, a fundamental method, for finding out when we run into a problem, shocks run into each other.
We don't know what to choose.
One way to find out in this 1D problem and also in much tougher nonlinear multidimensional problems is put a little viscosity in, and let it be small.
Let it even get smaller.
Then with a little viscosity, officially this term -- in some sense, this term, because it's a second derivative, is officially, mathematically speaking, sort of dominates this one as far as -- I mean, this will produce a smooth solution.
This term, as we know doesn't smooth the solution.
It just advects it, just carries it along.
But this term will smooth it.
So this is the dominant term, the second derivative.
No surprise.
So when we introduce that, that smooths everything.
We have no problem to say there is a solution, at least for simpler models, and then we can see what happens when that term approaches zero.
OK, so that's another method.
And, of course, that's kind of what we're doing with finite differences.
Finite differences, if we put in a little viscosity, a little second-space difference, and take delta t small enough for stability, then we can solve it.
And if we take it very small, we should get something close to the inviscid limit.
OK, I was just giving Burger's name to that particular example.
So this isn't the only example, but it's the simplest example.
OK, another example of conservation laws that everybody likes, I think, is the traffic flow, traffic flow in one direction, say, on the Mass.
Pike.
So what's the equation of traffic flow?
Let me put that on a board here, and then I have to come back to this.
Well, I'll put it here.
So Example 2 -- well, I don't know if you can call that Example 1.
We didn't give it a physical meaning.
Example 2 will be traffic flow in 1D, and what's the unknown?
Well, it's the thing I've been calling u, but for traffic flow, the natural unknown is the density of the traffic, the density of the cars.
And everybody uses rho for density.
So can I use rho rather than u?
And then in the equation will be a car -- a velocity that I'll call v. That's not a new unknown.
In fact, the velocity experimentally depends -- oh, there's a relation.
The density and velocity are, by experiments, if you run an experiment on the Mass.
Pike in heavy traffic -- that could be a homework problem, too -- to find the relation between v and rho.
So I'm going to keep this rho was the unknown and imagine that we have some experimental velocity v of rho.
And what would we expect?
I guess we expect that as rho increases -- as the density increases, what will happen?
As the density increases, the traffic is going to slow down, ultimately come to a stop, where if the density is very small, so the density 0 -- I mean, we just have a single driver, the police are not out, so he's going vmax or 2vmax.
But as the road gets crowded, the velocity drops and maybe drops linearly.
That would be a possible and not that terrible relation between v and rho.
So what would be the flux function in this traffic flow problem?
The conservation law is like conservation of cars, right?
If we look at a unit, a piece of the turnpike, then the conservation law says that the total number of cars inside, which is this, changes by cars leaving and by cars entering.
And so the flux function I think will be -- so we have to know like sort of how many cars are leaving, and I think the flux function would be probably v times rho.
I think the flux function of rho now is my unknown.
I think it would be v of rho times rho, the velocity times the density.
That tells me how many cars are going out.
OK, so I guess what I want to say is that we will meet -- that we meet in this traffic flow example exactly these possibilities that -- we meet them mathematically just the way we meet them in actual driving -- of a shock.
So this shock corresponds to -- so what that's?
Like stop-go -- this corresponds to I suppose like meeting a red light?
Coming up to a red light, traffic you're behind normally stops.
It has to slow way down and stop.
Then the light turns green.
The first car takes off.
The cars begin to spread out.
That's the solution that I don't see with this starting function, a sort of fan of cars.
So maybe I could write those words down: shock or a fan.
The shock is what happens when characteristics come together, cars come together.
The fan is when the characteristics fan out.
Actually, I could illustrate a fan here right away.
Suppose I reverse 0 and 1 just to see what -- OK, so here's my -- this is my x again.
So here I'm going to have u0 to be 0 now, u0 to be -- well, I can even -- it's called a Riemann problem when there are just two values here.
u0 is 1.
So the so-called Riemann problem is the cleanest example of the whole conservation law theory when there are just two starting values 0 and 1 or 1 and 0.
And it makes a big difference.
1 and 0, we saw when we reached time 1, we had a 1 and a 0, and after that, a shock formed.
Here 0 and 1, the characteristics here are going as always.
So u is 1 here.
The characteristics here, this is staying at 0.
There's no reason to change.
But here we have now a region, just like the other one, but with a major difference.
That now, it's 0 on the left and 1 on the right, where before, it was 1 on the left and 0 on the right.
And the question is, what happens in here?
Well, you might think a shock, a step up from 0 to 1 at some point.
And I would have to agree that you could do that in such a way that you maintained the conservation law.
But that's not what happens.
This is going to be a fan in here.
We're going to have -- what we'll have is characteristics in here.
This will -- let me draw the profile.
So the profile is 0 there to 0 at some time.
Then over here, where this characteristic is met, it's 1, and this is what happens: Instead of getting worse, less smooth, the solution gets better, more smooth, when the direction is reversed.
So we see that our problem had to be nonlinear.
A linear problem couldn't do these two opposite things in the same thing.
So here we would have a fan and the characteristics go there, and it's reflected there.
OK, so there's a picture of the phenomena that we're looking at and we would like to see numerically, too.
So next time I want to find -- so I can say now what I have to do next time.
One is to locate the shock, the shock line, when there is one, and two, to decide -- how do we decide between shock or fan when from the point of view of pure conservation, we could choose.
But nature makes a choice.
And one way to decide, make that decision, is Burger's way of putting in a little diffusion and letting it disappear, but we need a direct rule.
So we need a direct entropy condition.
So this is going to be a shock speed -- a formula for shock speed s. s equals shock speed we'll find if there is a shock.
And number two is going to be -- this decision is going to be based on a quantity called entropy, which appears only in nonlinear problems, really.
OK, so that's a beginning picture, which these 16.920 notes are an excellent reference for.
The book also and Monday's lecture will pick up there.
So we can see the two situations here, and the question is what do we do at this point.
OK, see you Monday.
Have a good weekend.
And any email questions about the homework, just you could send to me or -- and/or to -- maybe and to Mr. Cho, who will be looking at the homeworks that come in.
Good, thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general, is available at ocw.mit.edu
I thought I would, in this last lecture before the break, speak about one specific topic.
It's often referred to as a Fast Poisson Solver, so what does Poisson mean?
Poisson means Laplace's equation.
So, this is the five-point Laplacian, which could be some other discrete Laplace matrix, but let's take the one we know.
So we're in two dimensions, and you use the word Poission's name, instead of Laplace's name, when there's a non-zero right-hand side.
So otherwise, it's a Laplace Solver, but here Poisson.
OK.
So, remember that right-hand side comes from maybe a right-hand side f of xy in the differential equations, but it also comes from non-zero boundary conditions, because non-zero boundary conditions, when the five-points hit a boundary, that known value was moved over to the right-hand side and becomes part of f. So f comes from the non-zero boundary conditions, as well as the non-zero right-hand side.
OK.
Important problems, but special on a square or a rectangle or a cube or a box, so we're speaking about special geometry.
Today as we've been doing, I'll take the case on a square, and you will see that the whole idea would not fly, if we were on an ellipse or something like that.
But a lot of problems on rectangular domains do appear in applications, or we could use, I hope you always think now about possible preconditioners.
Any time you have something fast, it's a candidate to be a preconditioner for a real problem.
The real problem might not be on a square, or the real problem might not have the constant co-efficient that we have in the Laplacian.
In that case, you're not solving the exact problem, but one that could be reasonably close to it.
OK.
So we've discussed the solution to this problem, and actually I have a little more to say about the movie that is now on the website, because I'm quite excited about that movie.
I'll come to the movie in a second.
Let me just say what today's lecture would be.
The key idea here is that the eigenvalues and eigenvectors of this giant matrix of order n squared by n squared, sizes n squared by n squared.
The eigenvalues and the eigenvectors are known.
First of all, they're known.
The eigenvectors are nice discrete sine functions; they're sines, because I'm assuming they come to zero at the boundary; so that's why I have a sine, rather than a cosine.
First they're known, and second we can work with them very quickly, using the FFT.
So the point is that it's quite exceptional; in fact, I don't think I know any comparable example, in which a linear system is solved by using the eigenvalues and eigenvectors.
Usually eigenvalues and eigenvectors, they pay off for differential equations that are growing in time.
Then it is worth computing them, because then you can just multiply by either the lamda t, and you know what's happening later.
Eigenvectors, eigenvalues have other purposes, but very, very rarely are they used to solve a linear system.
I mean, it's usually far more work.
And of course, it would be incredibly more work if we had to find the eigenvalues and eigenvectors, but for this problem we know them.
And it also would be incredibly more work if the matrix of eigenvectors, the basis matrix, the key matrix that I'll denote by s -- so the eigenvectors go in a matrix s; the eigenvalues go in a matrix capital lambda; so we know lambda.
It's going to be a simple matrix, just diagonal, got those n numbers and n squared numbers, I guess, because we're of size n squared.
But these eigenvectors we know.
So we know them, and we can compute quickly with them, using the FFT, or using, you might want me to say, fast sine transform, discrete sine transform, rather than Fourier transform, which we think of as doing the complex exponential.
So this is a small special fast Fourier world.
It's a special fast Fourier world, in which the FFT and the related sine and cosine give us a quick answer, faster than elimination, because you all know, it's n log n, if n is the number of Fourier components, and that's hard to beat.
Then I'll mention also, for this same problem, there is another way in which it can be simplified.
It's not a Fourier way, just a direct combining neighboring equations, so that'll be number two.
OK, but mostly the lecture is about number one.
The best example I know in which you would use eigenvectors/eigenvalues to solve an ordinary linear system, and I'll say in a word just how you do it, and then what these eigenvectors are.
OK.
Pause.
Time out to say something about the movie.
So that movie that's now on the website does sparse elimination, and the example it takes is k2d, and you can make it 8 squared by 8 squared, 10 squared by 10 squared, 20 squared by 20 squared, because it shows the order that the nodes are eliminated and the graphs of non-zero's in the matrix.
It's a bit slow.
If you do 20 by 20, you have to go away for lunch, well maybe not lunch, but at least coffee before it's done, but when it's done, it counts the number of -- it shows the non-zero's that are in the elimination in the factor l from elimination, and it counts the number of non-zero's, and I was just talking to Persson about also getting it to count the number of actual [?
flops ?].
Well, why am I interested?
I'm interested, because I don't know what power of n those numbers are going there.
I don't know whether the number of non-zero's, and I did 10, 20, 30, and it looked not too far from the power capital n cubed, but the notes say for nested dissection, which would be another ordering n squared log n. So, and of course, what power we get depends on what algorithm we use for ordering, so nested dissection is one ordering, which hopefully we could put into another movie.
The movie now has exact minimum degree, real MMD, which takes as the next node the one with absolutely the minimum degree, not just close to it.
Anyway have a look at that movie, and if you have any interest, see how the count increases as n increases, and also you could change, slightly adapt, the algorithm that creates the ordering, creates the permutation and see what happens there.
There's a lot to do with that, and it's a pretty fundamental problem.
Actually, we're talking there about what MATLAB's backslash operation will do for this equation.
So MATLAB'S backslash operation will use elimination; it won't use fast Poisson solvers, but now let me come to the fast Poisson solver.
OK.
So I guess the main point is I have to say what are the eigenvalues and eigenvectors, and how do they get used.
So let me say, how do they get used.
So how can I use eigenvalues and eigenvectors to solve a problem like that.
Let me just call it k rather than k2d, sorry.
ku equals f, OK, by eigenvectors.
OK, there are three steps.
Step one.
Expand the right-hand side as a combination of the eigenvectors.
OK.
So expand f, this right-hand side vector, as some combination of the first eigenvector, maybe I'm going to call it y1.
Second eigenvector, nth eigenvector, OK, good.
That means what do I mean, I mean you have to find those c's, that's the job there.
Find the co-efficients.
So that's a job, a numerical -- it's a linear system to solve and we'll see what it amounts to.
OK, but suppose the right-hand side is a combination of the eigenvectors, how can you use that?
Well, step two is the real quick step.
Divide each ci by the eigenvector, eigenvalue, lambda.
OK, so it's my eigenvector, so I'm assuming that kyi is lambda iyi and that we know these guys.
So this is known.
And now the question, I'm just saying, how do we assume known?
So my question now is how do we use it?
OK, step one, the idea is going to be write everything in terms of eigenvectors.
Work with the eigenvectors, because if I've got eigenvectors, the step is scaler; I just divide these numbers by those numbers, and then I've got the answer.
And then construct, the correct answer will be u, will be c1 over lambda 1y1 and c2 over lambda 2y2 up to cn over lambda nyn, a combination of those same eigenvectors with the same coefficients, just divided by lambda.
But this is, of course, another numerical job; this is like adding up a Fourier series; this is like finding the Fourier coefficients, this is like reconstructing the input, only because I've divided by lamba i, I'm getting the output here is u, when the input was x.
And do you see that that is the correct answer?
All I have to do is check that ku equals f, so check that this answer from step three is the right answer.
OK, so I multiplied by k. When I multiply y1 by k, a factor lambda 1 appears, the eigenvalue; it cancels that; well that's y divided, so it would cancel, and I have c1y1.
When I multiply this by k, ky2 is lambda 2y2; cancel the lambda 2s, and you're left with c2y2, and so on.
So, when I multiplied by k, I got f. That's it.
So that's the whole idea, written out, but now what actual computations go into steps one and three?
Step two is pretty simple.
Well, actually this is a good way to look it.
I want to write that same algorithm in matrix language.
OK, so in matrix form we have the matrix of eigenvectors, and that's what I'm calling s. And it's got the eigenvectors y1 y2, yn, and it's columns, and the eigenvalue matrix, we need a name for that too, and that we decided to call lambda, and that's got the eigenvalues on its diagonal.
So this is 18.06 linear algebra.
The matrix of eigenvectors if I multiply k by s, then I'm multiplying k by all its little eigenvectors, and k times this gives me lambda 1y1, k times y2 gives me lambda 2y2, kyn is lambda nyn, and if I look to see what this is, this is the same as y1 to yn multiplied by lambda.
If I just multiply [UNINTELLIGIBLE] on the right by lambda, it will take lambda 1 times the first column, lamda 2 times the second lambda n times the third, which is, times the last, which is what we want, so it's s lambda.
This is all n eigenvalues and eigenvectors in one matrix equation, that's all that is.
It's just ks equal s lambda i just says, this for all i at once, all i at the same time.
OK.
So if I use these matrices in describing steps one, two, three, I'll see what's happening matrix language.
OK, let me just do that.
Step one, step one is looking for f as a combination of the columns of s. So step one is just f equals s times c. The vector of coefficients multiplies the columns of s and adds to give f. Then step two, which just divides everything by -- divides by the lambdas.
Step two just creates lambda inverse, sc.
So I took what I had -- let's see, no, the lambda inverse better be multiplying the c. Well, actually I can do it all in -- well step two, that's the easiest step, I should be able to do it.
c is changed to lambda inverse c, so that c becomes lambda inverse c, OK. And then step three uses lambda inverse c to construct u.
So step three is the answer u, what do I have here?
I've got a combination of these vectors, so they're the columns of s, and what are they multiplied by?
They're multiplied by the c's over lambdas, which is what I have here.
That's s lambda inverse c. Those are the three steps.
And what's the work involved?
Here, the work is solving a linear system with the matrix s. Here, the work is taking a combination of the columns of s, doing a matrix multiplication.
Those two steps are usually full-scale matrix operation, and of course, the s , if I just complete this, I'll see that I get the right thing that's s lambda inverse and c is s inverse s. There's the answer.
u is s lambda inverse, that's a lambda inverse there: s inverse f. That's the correct answer in matrix language.
Right.
This is k inverse, that's k inverse.
k is s lamda s inverse, and if I take the inverse of that, I get s lambda inverse, s inverse to multiply f. Well, I doubt if you're much impressed by this lower board, because the upper board was the same thing written out.
It took some indication of what the separate pieces were, but it's pretty clear.
OK, now the million dollar question is, is it fast?
And the answer is, almost certainly no.
But for the particular matrix s, which by good fortune s could also stand for sine, this matrix of eigenvectors for this particular problem are sines.
These are the discrete sines, so this is the discrete sine transform.
That's we're doing, we're doing the discrete sign transform, because those discrete sine vectors are the eigenvectors of k. OK, now let me say what that means.
First I'm thinking of k in 1d.
So this is my 2 2's and minus 1's and minus 1.
Its eigenvectors are discrete sines, if I multiply that by sine kh -- well, let me just take the first one, sine h, sine 2h, sine n minus 1h, that will turn out to be an eigenvector.
So this is ky, ky1, the first eigenvector.
The eigenvectors are -- let me draw them.
The eigenvectors for that second difference matrix k are -- here's the interval, 0 to 1, I chop it up in steps of h, and I plot the sine, which starts at zero and ends at zero, because those are the boundary conditions, and here is sine h, sine 2h, sine 3h, sine 4h, sine 5h, so for the five by five case -- maybe I should just be using n here, or maybe I should even be using capital n, so capital n is 5 in this example.
Good.
What's on the other side of that equal sign?
Some eigenvalue times the same vector, sine h, sine 2h, down to sine nh, and that eigenvalue -- oh, let me just write lambda 1 for it.
We know what it is.
The fact that this is an eigenvector is just trig; you know, I multiply minus 1 of that plus 2 of that minus 1 of sine 3h, and I use a little trig identity.
So minus that, 2 of that, minus that, turns out that to be a multiple of sign 2h.
Well sine 2h give us a 2, and then the minus sign h and the minus sine 3h combine into sign 2h times, I think, it's s cosine of h or something, it's that eigenvector that's near zero.
But the cosine of h is near 1.
Does that look familiar?
That combination of sine h and sine 3h should give us twice sine 2h times some cosine, yep, Elementary trig identity.
OK, so those are eigenvectors.
That's the first one, the next one would have h times the k'th one would have h times k instead of just h itself, it would take jumps of every k sine, and then a cosine hk would show up there, and this would still be an eigenvector.
OK.
I'm making a little bit explicit these vectors, but the main point is they're sines, they're discrete sines at equally spaced points.
That's what the real version of the FFT just lives on, and it would also live on discrete cosines if we had different boundary conditions, we could do those, too.
So this isn't the only -- these zero boundary conditions are associated with the name of Dirichlet, where zero slopes are associated with the name of Neumann, and both, this one gives sines, Neumann gives cosines, the FFT deals with both.
OK.
So, that's the fast solution, and it would take n squared -- well I have to go to 2d.
sorry, I guess I have a little more to say because I have to get from this one-dimensional second difference to the 5.2-dimensional second difference, and that's what's going to happen over here.
I wrote up some stuff about the Kronecker operation, which is the nifty way for these special problems to go from 1d to 2d.
You remember the deal, that k2d, our 2d matrix was this Kronecker product of k and i, that gave us second differences in one direction, and then we have to add in the Kronecker product of i and k to get second differences in the other direction.
And then we better print, because that matrix is going to be large, I don't want to [UNINTELLIGIBLE] it, yeah.
What's the point?
The point is that if I know the eigenvectors of k, then I can find the -- if I know the 1d eigenvectors, I can find the 2d eigenvectors, and you don't have to know Kronecker products to do that.
All you have to do is just make a sensible guess, so the eigenvectors in 2d, have a double index, k and l, and their components are sines in one direction times sines in the other direction.
So what are those sines, there's a kh, I guess, lh.
Those are the first components, I guess I have to tell you what all the components are: k, the seventh component in the x'th direction, there'd be a factor 7, so kmh sine lnh.
This is the mn component of ykl.
It's just what we had in 1d, in 1d there was no l, the components were just sine kmh.
Now we've got two, one in the x direction.
These are the analogs of the -- the continuous case would be sine, k pi x, times sine l pi y.
Those are the eigenvectors as eigenfunctions, functions of x and y.
And the point is that, once again, with these beautiful matrices, I can sample these at the equally spaced points, and I get discrete sines that the FFT is ready to go with, OK.
I'm giving this much detail partly because the continuous case, of course, our operators d second by the dx squared and d second by dy squared, and the whole idea of separating variables, of looking for solutions, u -- here is the eigenvalue problem, the continuous eigenvalue problem.
The Laplacian of u, maybe I do a minus, the Laplacian of u equal lambda u, and that's a partial differential equation, usually it's not easy to solve, but if I'm on a square, and I have zero boundary conditions, then I've solved it, by separation of variables, a function of x times a function of y.
And that function of x times function of y is exactly what Kronecker product is doing for matrices, yep.
I thought maybe this is good to know when the problem is easy, and as I say, the possibility of using the easy case on a square for preconditioning a not so easy case is attractive.
All right, so that's what I wanted to say about number one, that's the main suggestion, and again, the point was just to take these three simple steps, provided we know and like the eigenvector.
Here we know them and we like them very much, because they're those discrete sines.
OK, now just to finish comes, what's up with odd even reduction.
I'll use the same 1d problem first.
It works great in -- that's not good english, but it works very well in 1d odd even reduction, you'll see it, you'll see, oh boy, simple idea.
But of course, don't forget that ordinary elimination is a breeze with a tri-diagonal matrix, so nothing I could do could be faster than that.
But let's see what you can do.
I just want to write down the -- OK, keep your eye on this matrix, so I'm going to write out the equations.
So it'll be minus u, i minus 2 plus 2ui minus 1 minus ui, that would be fi minus 1; that would be equation number -- i minus 1, right with a minus one, two minus 1, centered there; and then the next one will be a minus ui minus 1 plus 2ui -- I better move this guy over further -- minus ui plus 1, that will be fi, right?
That's equation number i, and now I just want to look at equation number, the next equation, minus ui plus 2ui plus 1 minus ui plus 2 is fi plus 1.
So I've written down three consecutive equations, three consecutive rows from my simple matrix: a row, the next row, and the next row.
So the right-hand sides are coming in order, and the diagonals are there, and if I look at that, do I get any idea?
Well, there is an idea here.
I'd like to remove these guys, the ui minus 1's and the ui plus 1's, so that's where this word odd even reduction is coming in.
I'm going to reduce this system by keeping only every second unknown and eliminating those.
How to do that?
Well, you can see how to eliminate, if I'm multiply this equation by 2.
If I multiply that middle equation by 2, that becomes a 4, this becomes a minus 2, this becomes a 2, and now what shall I do?
Add.
If I add the equations together, I get minus u, i minus 2, these cancel, that was the point plus 4 minus a couple, so that's 2 ui's minus this guy, ui minus 2 is that sum fi minus 1, 2 fi's and fi plus 1.
Well, sorry it's squeezed down here, but this is the point.
The main point is look at this.
What do we have?
We've got a typical equation, but now we've removed half the unknown.
The problem is now half-sized.
We've only got the even-numbered unknowns at a small cost in updating the right-hand side, and the problem's cut in half.
So that's this odd even reduction, and it cuts a problem in half, and everybody knows right away what to do next.
The one mantra of computer science, "Do it again."
So that is the same problem on the even, we do it again, so I should really call it cyclic reduction, we just cycle with this reduction, and in the end, we have a 2 by 2 problem.
That seems pretty smart, pretty successful move, and I guess if we do an operation count, well, I haven't thought that through.
What does the operation count look like?
It must be pretty quick, right?
Well, undoubtedly we're solving this linear system in o of n steps.
Well, no big surprise to be able to deal with that matrix in o of n steps, because elimination would take o of n steps, it's size n, but it's bandwith is 1, so 2n or something steps would do it, and maybe, I don't know how many steps we have here.
I guess, when we cut it in half that required us to do that much.
It's order n, it's order n. So the key question is, can we do it 2d?
Can we do the same thing in 2d?
So I want to follow that plan in two dimensions where now u will be a whole row at a time, so I'm doing block 2d, block 2d.
So, can I write down the equations in block 2d for whole rows, so ui is the vector?
So now this is the 2d problem, so it'll be minus the identity, where instead of instead of 1, I have to write the identity, ui minus 2 and 2k, right?
Oh no, what is the middle -- what's on the -- so multiplying ui, ui minus 1.
I wanted to just say the same thing, but I have to write down the -- this is going to be n squared equations, n at a time, a whole row at a time, and what's on the diagonal of k2d?
Not 2k.
It's 2i, is it 2i plus k?
Yeah.
Yeah, k plus 2i.
Isn't that what we have on the diagonal of k2d1 times ui minus 1 minus iui is equal to some -- that's a whole row at a time.
These are all vectors with n components now, right?
The minus i, the 2i, and the minus i are the second differences of one of rows; their difference is in the vertical direction, and this kui the second difference is along the row.
OK, so same equation at the next row.
So the next row is minus i, ui minus 1, k plus 2ui, ui, because that's now the diagonal block minus iui plus 1 is fi plus 1, and it's just taking a second to write this one.
This is k plus 2i, ui plus 1, minus i ui plus 2 is fi plus 2, OK.
Exactly the same, but now a row at a time in 2d.
So the same idea is going to work, right, what do I do?
I want to cancel this, so I multiply that row by k plus 2i.
Before I multiplied it by 2, but now I have to multiply it by k plus 2i times the row, the whole row.
So when I do that this cancels, so I have minus this, this guy wasn't affected.
And this will cancel, the k plus 2i will cancel this one, just as before.
This will not be affected, ui plus 2, and I have fi and k plus 2i, fi plus one and fi plus 2.
That should have been i minus 1, and this should have been i, and this should have been i plus 1, sorry, mis-labeled label the f's, but no big deal.
The point is the left side, and the point is what's in that space.
What goes in that space?
Well, it's minus i, k plus 2i squared, minus i, so this is k plus 2i squared, which before was so easy, and then the minus i, and the minus i is the minus 2i is multiplying the ui.
Yeah, that was [?
my last i.
?]
OK, this is my matrix from odd even reduction.
It's just natural to try the idea, and the idea works, but not so perfectly, because previously in 1d, that just gave us the answer 2.
It was 4 minus 2, but now in 2d, we have a matrix, not surprising, but what we don't like is the fact that the bandwith is growing.
k was tri-diagonal, but when we square it, it will have five diagonal, and when we repeat the odd even cycles.
When we do it again.
Those five diagonals will be nine diagonals, and onwards.
So, I'm getting half-sized problems.
All the odd-numbered rows in my square are eliminated, this just involves the even-numbered rows, but the matrix is not tri-diagonal anymore, it's growing in bandwith.
So, and then you have to keep track of it, so the final conclusion is that this is a pretty good idea, but it's not quite as good as this one.
It's not as good as the FFT-based idea.
Also, if you look to see what is most efficient -- see the eigenvectors are still here, so I could do three steps of this and then go to Fourier, and that probably is about right.
So, if you really wanted to polish off a fast Poisson solver, you could do maybe two steps or three of odd even cyclic reduction, but then your matrix is getting messy and you switch to the fast Poisson solver, so it's not quite Poisson anymore, because it's has a messier matrix, but it still has the same eigenvectors.
As long as we stay with the matrix k, we know it's eigenvectors, and we know that they're sines and that they're quick to work with.
OK.
Anyway, there you go.
That's a fast algorithm for the lecture before the spring break.
So after the break is, first of all discussion of projects.
If your project could include page -- and you could maybe email the whole project to Mr. [?
Cho ?].
Maybe also, could you email to me a sort of summary page that tells me what you did, so I'll save the summary pages, and I'll have for Mr. [?
Cho ?]
the complete project with printout and graph, as far as appropriate, and so we'll spend some time on that, and then move to the big topic of the rest of the course, which is solving optimization problems, minimization, maximization in many variables.
OK, so have a good spring break and see you a week from Wednesday.
Good.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
OK. Good.
So I decided to make today's lecture the one on linear programming and duality, which I'd planned for Friday, and give myself two more days to learn about ill-posed and inverse problems, and then come back to that Friday, so that we've studied the limits in those problems of alpha going to infinity or 0, but the scientific question when there's noise in the system is finite alpha and I want to learn more about applications and examples.
Can I also say I'm very happy to have had volunteers for Monday and Wednesday of next week to present, and if a couple of people might maybe volunteer for Friday, to share Friday, I'll be very grateful.
So you could see me after class, put a hand up now, send me an email -- all those would be very good And again I would be thinking -- since it's just next week I'm talking about -- that it would be essentially a report on your Project One that you would use the overhead projector maybe, if that's preferable.
OK.
So I think you'll like this topic.
It's kind of specific but widely used -- linear programing -- used in business to maximize profits, to minimize costs.
And linear means that the cost function is linear.
That's an inner product -- cx.
c is a row vector, x is a column vector , so that I'm following the conventions of this subject here to take these different shapes, so let me indicate what the shapes are.
But the inputs are -- the data of the problem are -- c and A, an m by n matrix and b.
So A is m by n, b is m by 1, -- right hand side -- and c is 1 by n. And then the unknown -- this is the thing that we're to find -- it's a column vector -- n by 1.
OK. And the point is there are constraints and those are linear too.
So it's rather unusual to have a linear cost function.
Right?
Because when you maximize or minimize some linear function, well, the thing is just going up or it's going down -- or in higher dimensions the same -- and if it's going down, then the minimum is going to be at the right hand end.
Or if it's going up, the minimum will be at the left hand end.
And if I'm in more variables, this idea will still be true that the minimum or maximum will happen at the edges, at the ends of the allowed region and this allowed region, called the feasible set -- so let me give name to this -- these are the allowed [?
axes.
?]
These are the constraints, and that set is called the feasible set, feasible meaning doable.
So those constraints include inequality because we want finite intervals, finite regions in n dimensions.
And I drew a sort of quick picture so that you have a model of this.
So this is a picture with n equals 3 -- 3 dimensions, and so the constraints x greater or equal 0, x sub 1 greater or equal 0, x sub 2 greater or equal 0, x sub 3 greater or equal 0 -- that's what x greater or equal 0 means.
It means all components.
So we're in the quadrant right?
We're in a quarter -- 1/8 sorry -- we're in an octant -- 1/8 of three dimensional space, the positive octant.
And then if I draw maybe just one, just put in one equation, one plane, would cut off a piece of that octant, so that Ax greater or equal b, depending on the signs, but the feasible set could well be the tetrahedron, the little piece of the octant that's cut out by this plane.
Or if our constraint was an equality, the feasible set would be the triangle.
So Ax equal to b would lead to the triangle, and Ax greater or equal b would be, if we pick the signs correctly, would be the pyramid, would include also this corner, because there'd be some volume.
OK.
So the feasible set is a polyhedron.
It's like a polygon only up into n dimensions, so we use the word polyhedron.
And it's got corners, and the whole point of the linear cost, the linear objective function cx, so this is just c sub 1 x sub 1 plus c sub n x sub n. That's what that means.
If I take derivative I get constants.
I don't set derivatives to 0 in this type of problem.
I look at the end points, at the corners.
And that's where the minimum and maximum will occur.
So it's just a question of finding the right corner.
That's the problem: how to find the winning corner.
It's an interesting competition between two quite different approaches: the famous approach -- so let me write these two.
The simplex methods is the best established, best known, approach for solving these problems.
What's the idea of the simplex method?
The simplex method finds a corner.
A corner is a case where we have some equality signs.
A corner is the edge, the limit where maybe this one still has x sub 1 positive but it's down in this plane so it has maybe x sub 3 as 0 for this guy.
So that corner has x sub 3 equals 0 and it also lies right on the plane so it has Ax equal to b.
This corner -- well, I guess that corner has all these guys equal 0: x sub 1 equals 0, x sub 2 equals 0, x sub 3 equals 0, but Ax inequalities over here for this corner that's hiding behind the face.
Anyway, corners are points where some of the constraints are tight or active and others are not.
Well, you might say just check them all, but the trouble is there are lots of corners.
If we're in n dimensions and we have m constraint equations, then the number of corners goes up exponentially.
So there's no way to check all of them.
So the simplex method had a better idea.
The simplex method found one of them -- and already that's a little bit of a job to find a corner, but finds one.
And then what the simplex method does, it stays entirely, it moves along the edges.
So from here it will look to see in which direction would the cost go down, because we're trying to minimize the cost.
So it would check these directions.
Each of those directions we're releasing one equality.
We're allowing one equality to be an inequality.
and that moves us along.
So the simplex method has two steps.
It checks each of these directions to find out which way will the cost drop fastest.
It chooses the direction in which the cost, the gradient, the component of the gradient, you could say, along that edge is the biggest, or maybe the most negative.
And then, once it decides which way to go, it goes -- maybe it takes this direction -- it goes, goes, goes, goes, goes until it hits another corner.
So that's the end of the simples set, when it reaches another corner.
That completes one simplex step.
Then, from this corner, it will look to three ways it could go here.
Well, it's not going to pick this way because in this direction the cost was decreasing or we wouldn't have taken it.
We wouldn't have taken that direction except for the cost went down.
So if we came back, the cost would go up.
No good.
So going down would be one of these two ways.
Maybe it goes down in this direction, so we decide on that direction.
We follow it until we hit a new corner, and eventually we're going to get to the winning corner because there are only a finite number of corners.
And how will we know it's the winning corner?
Well, we'll know that corner is a winner if in every direction the cost goes up.
If the cost goes up in all directions along all the edges out of that corner, then that corner has won.
It's the minimum.
I'm using linearity here -- the fact that you know everything by traveling along, by looking along those edges.
So the simplex method is a big success.
But in reality, in practice, it turns out that the number of edges that you have to travel to get to the winner doesn't grow exponentially.
I mean in principle it could.
People have dreamt up really desperate examples in which following the simplex method you could take a long time, but on average you don't, and in practice you don't.
So it's a very good method and was totally the method of choice.
But a competitor has arrived.
And that competitor goes under the name of interior point method, and you can guess what that method is doing quite different, totally different system.
That method is inside the feasible set.
It finds a point somewhere near the middle maybe.
And then it does a normal gradient type approach from your point.
It figures out which way to move.
It moves, but it doesn't go outside.
It doesn't even reach the boundary of the feasible set because if you reach the boundary of the feasible set you're out of the interior and the method is not going to operate.
Well, that crudest method would be follow the gradient, but we know from several situations that gradient descent can be less than optimal.
So this is more subtle.
Well, Newton's method actually -- I'll explain.
So this is actually the content of my lecture -- this interior point method.
And let me just mention a few names.
People thought of interior point methods long ago, but a big splash came when Karmarkar proposed an interior point method and proved that it converged faster than simplex methods in some problems.
Well, he said all problems, actually.
His advertising of the message was pretty generous The sort of claim that was around was ten times as fast as the simplex method generally, and it was on the front page of the New York Times, and I remember going to a lecture in Boston with lights, TV lights on and everything.
Well, maybe his exact message now isn't so much used, but you have to give him credit for stirring up the whole world of optimization because the result of Karmarkar's method -- and there were others -- and I'll say barrier method, and that's what I'll try to explain.
He stirred up the whole world so that the experts in optimization began looking again at interior point methods, seeing that they did have some merit, improving them.
And now for, I would say particularly for large, sparse problems, these are a way to go.
These are preferred now.
So this is the normal situation in scientific computing: that any message that's good, it's still not good for everything.
It's got it range of problems where it's successful and a range of problems where some competitor wins.
So that's the situation now.
These are methods.
This is certainly not out of date, and I'm sure it's the method of choice and it's carefully coded and well understood, but these are quite effective.
OK.
So my job then is to say something about these interior point methods.
And the beauty of these is that the primal -- this is called the primal problem.
Primal problem.
And often you write a (P) for primal.
It means the given problem.
And over here is the dual problem.
So you put a (D) for dual problem.
[UNINTELLIGIBLE PHRASE] It involves the same data: the same b, the same a, and the same c, but a new variable y.
And [UNINTELLIGIBLE PHRASE] in the original problem for the constraints.
So I won't go at the dual problem exactly that way.
I'm going to ask you just to consider this problem and show you the relation between the two.
So what I want to say is that these two problems -- the primal and the dual, which use the same data a b c, are intimately related, and sort of solving one solves the other one.
Actually is this it?
That applies to the simplex method.
When the simplex method finds the best corner, we could read off of Lagrange multipliers, we could read off y.
We could read off the optimal y.
So my picture was in the primal case, but there's a dual picture in the dual case.
OK.
So we have a minimum problem and a maximum problem, and I'm using this word duality.
So what I want to do is tell you how do we recognize the winning corner in the primal problem, and it's beautiful.
So at the best, so the optimal -- x, let me call it x star and y star -- have min over there equal of max here.
Min of all the cx's, which is cx star, equal to a maximum over all of the y's of the yb's, which is y star b.
So these are equal at the winner.
That's the essence of this duality.
Duality is about two problems that use the same data but they look quite different.
You know they're using the data in different ways.
The cost function there showed up in the constraint here.
The constraint b there showed up in the cost function here.
And even a got flipped, because if I use my usual column vector notation -- if I just transpose this -- this would be a transpose, y transpose, less or equal to c transpose.
If I wanted to stay with column vectors y transpose and c transpose, then it would be the transpose of a that would appear.
So I'll just put transpose with two exclamation marks.
That's typical.
And you often see the word adjoint.
So there are methods in differential equations, in optimization, called adjoint method.
Adjoint is just really another word for transpose.
It's a word that applies in differential equations as well as matrices, so it's kind of a better word, you could say, where transpose we usually apply to matrices, but totally the same idea, identical idea.
OK.
So the wonderful thing is that at the moment of success, at the moment of optimality, these are equal.
A minimum equals a maximum.
And that's one way to recognize that you've succeeded, and that's one way to measure how far you have to go with the duality gap.
So the duality gap would be the difference if you had a particular y that wasn't the winner, a particular x that wasn't the winner, the duality gap would be the difference between cx and yb.
And what I'm saying is that when that duality gap narrows to 0, you've got it.
When this narrows to 0, you've brought cx down as far as you could, you've raised yb up as far as you could.
And if you did it right, if you've got to the optimum, then the duality gap disappeared -- became 0.
So that's a measure of am I at the answer, am I close, do you know if we're going to do an iterative method as I'm planning.
So that's the point, of course.
These interior point methods will be iterative.
We said we never actually allow them to get to the absolute corner until maybe at the last minute.
So here, let me draw a picture of how interior point methods might work.
So here is the feasible set -- some kind of a polyhedron, whatever.
So think of that as a kind of a diamond, a twenty-four caret diamond.
OK?
And start at a point inside.
And somehow find a gradient, decide which way to move And there'll be some barrier here which is going to prevent us from reaching it, so we'll stop.
So that will be one step, and from here we will do the same thing, whatever it is.
It'll be Newton's method, actually.
You'll see.
It's just Newton's method.
The most fundamental way to solve non-linear equations is Newton's method.
And it'll take another direction.
Again it'll stop, and the thing will follow some path.
And then maybe at this point the duality gap is very small.
We'll realize that this is the winner.
So we could at the last minute say OK jump to the winner.
But it's this path through the interior that we're really interested in.
OK, so I'm giving a sort of general picture of it.
And now I'm ready to do two things.
One is the nice little bit of algebra that says that this duality gap is always greater or equal 0.
OK so that's called weak duality.
Weak duality, which is easy to prove, says that always cx, for any feasible x, is greater or equal to yb for any feasible y.
So any x and y that satisfy the constraint.
I should say satisfying the constraint.
So weak duality I'll now prove in one second, And the point is that as I push to bring cx down -- minimize -- as I push to move yb up -- maximize -- they will meet at the winner.
OK, now how do I prove cx greater or equal yb?
Let me try to prove that.
Proof.
OK so look at yb.
OK. Now so I know something about the constraints.
Ax is greater or equal to b.
So this b -- I want to say that this is less or equal to yAx.
Now am I allowed to say that?
First of all, y is feasible; x is feasible.
So they satisfy.
Feasible means that these are satisfied and these are satisfied.
OK. And do you see that that's really all right?
Well you might say no problem.
Ax is greater or b.
It's obvious.
But I have actually used one more point here, haven't I?
If I have an inequality, then I'm multiplying it by y, and it didn't change the direction of the inequality sign, and that was because y is greater or equal 0.
That's where that paid off.
So this used the fact that -- this came from the fact that y was greater or equal 0 and Ax was greater or equal b.
Those two facts meant that I could multiply and preserve the inequality sign.
And now I'm going to go to the next step: yA is less or equal c, and there is x. OK, so you see that I finally got what I want: yb less or equal to cx.
But what went into that step?
Well looking here, I had ya less or equal to c, and I also had x greater or equal 0 by the feasibility of x.
So that inequality I was allowed to multiply by x because x is not negative.
If x had been minus 1, then when you -- right, if I have an inequality like 4 less or equal 7, if I multiply by minus 1, I guess minus 4 and minus 7, and the inequality switches: minus 7 is below minus 4.
But that's not what's happening here because the x is not negative, so this is not what's happening, and I'm OK, so the conclusion was exactly what I wanted -- that the yb was less or equal to cx.
And you see how perfectly it used the four inequality constraints.
OK.
So that's the weak duality where the proof is easy.
Just use what's given.
The duality, without the word weak, is the fact that at the optimum the gap is 0, and actually we can see.
That will tell us a lot.
That will tell us a lot.
When could this gap be 0?
So at the optimum, y star, equality is holding throughout.
So if equality is holding, how can that be?
How can I take these -- of course the inequality, the x star and y star are feasible.
So if I just put stars on all these things, then I would have -- everything would still be totally true.
But when I put stars on them, so I'm picking the optimal guys, then equality is holding.
I still have these inequalities, so what I want to find is the optimality conditions.
How are they related?
If I have y greater or equal 0 and Ax greater or equal b and I multiply, how could I get equality?
Right?
For example, if I have 3 greater than 0 and 5 greater than 2.
If I multiply those, I get 15 greater than 0, I guess, and that's far from equality, right?
So how could equality happen?
Well, the only way is if one or the other of these, if equality holds in one or the other, then I would be OK.
Yes, do you see that?
If equality held -- these are vector inequalities, so I'm going really component by component.
Let me write down the my conclusion and then you'll see what I mean.
So these are called the Kuhn-Tucker conditions.
You've seen their names before.
And they're also called -- well long words -- complementary slackness.
I'm using words that, if you haven't seen the subject, you think OK, who needs the long words.
But the idea of slack variable; slack is the difference.
The slack is c minus yA, or over here the slack is Ax minus b.
These are the slack variables.
w, let's say, is Ax minus b.
And of course it's greater or equal 0 that's the nice thing about slack variables.
You know you fix it so it's greater or equals 0.
Here the slack variable s for slack would be what?
c minus yA greater or equals 0.
And there's no slack when s is 0.
OK, so that's where the word slackness comes in.
Slack is just the amount of give in the inequality.
So what's the point here?
I was looking at this guy, and the only way that I could have equality here when I have inequalities there is for each component I'm going to have to have equality.
And how can I have equality on a component?
Well, I would have it for example if y was 0.
Then when I multiply, I have equality.
Right?
OK. Or I could have equality if I had -- let's see, so what I want to say?
So I want to say either y sub i is 0 or Ax sub i is b sub i.
Equality holds in one or other of the two inequalities, because then if I multiply them together, I have equality.
Right?
You see that.
If one of those holds, say this one holds, if y sub i is 0, then I certainly can multiply the inequality by y and I get 0 equals 0.
Or if Ax is exactly b, then multiplying by y won't change.
OK.
So this is the complementary slackness, one or the other, that has to hold to get equality.
Now what about this guy?
Equality same idea here.
I got the inequality by multiplying these together.
When will I get equality?
Only if either x sub j is 0 or the j component of yA equals the j component of c. Again, the same reasoning: that when I multiply two things, if I get an equality out of two inequalities, then one of those two at least must have been actually an equals; otherwise I'd still have a gap.
OK, so this is pretty important.
These are the conditions -- these are our equations.
That tells us when we've won.
So this actually holds towards the winners.
It doesn't hold for all the other guys, but at the winner because things are equal here.
They had to be equal at every step and therefore the Kuhn-Tucker conditions had hold at the winner.
So they hold at this winning corner when we find it.
So the simplex method chases corners, finally gets to a corner, and it would know it had got there by the fact that it couldn't decrease any more.
And if you look at the algebra, you would see that that tells you that the Kuhn-Tucker conditions are satisfied.
OK, so the only proof I gave was the weak proof, that cx is greater or equal to yb because that's the nice one.
I've proved that for equality we'd need these.
OK, now I guess I'm ready for the method.
I'm ready for the interior point barrier method that tells me how to compute.
OK, so I'm at an interior point.
What do I do?
OK, so here's the method; here's the barrier.
I'll call it a log barrier.
I'll solve the problem of minimizing cx.
I won't solve the exact problem.
I'm going to minimize cx, minus I think some little number times a barrier, which is going to be a sum of the logarithm of the x's.
This alpha is going to be a little bit positive.
I'll take it smaller and smaller because this part is really -- it's that that I really want to minimize.
Right?
That's the original problem.
This is the cost.
I'm adding something to the cost but I'd better just be sure that I've chosen the sign of alpha correctly.
By the way, this is discussed now in the latest version of my Linear Algebra and It's Applications textbook, a fourth edition.
Editions one to three of that book and others have described the simplex method, and now it was just natural to include the interior point barrier method.
OK, so why do I call this a barrier?
Because if x sub i gets to 0, the log blows up.
The log blows down I should say -- blows down to minus infinity.
I'm multiplying by minus alpha, so I get positive.
The combination blows up.
It couldn't be the minimum, so you can see the minimum is never going to make it to x equals 0 because at x equals 0, the thing I have here is plus infinity.
So now I'm just going to use gradient method.
I'm going to solve this problem with the constraints, of course, with the constraints, and set derivatives to 0.
Now I have -- you know it's not linear anymore -- the winner is not at a corner anymore.
It's somewhere in the middle.
Calculus operates.
I can set derivatives to 0.
OK, so I want to do that.
And of course I'm still inside this feasible set, so let me see if I can put down the equations and the constraints.
OK, so I still have the constraints.
Now, forgive me, but I've made a change to Ax equals b. I could have started with that as a constraint.
I've made that change to Ax equals b.
How have I done such a thing?
I'm given the problem with Ax greater or equal b, but I'm also given the slack variable w greater or equal 0.
So I guess it's just a little tricks that's not worth -- you could just take my word for it, a little trick.
My new variable is the x's and the w's.
m plus n variables : the n x's and the m w's.
And now, put that together -- so can I just maybe do this over in the corner here?
Before I start on this, I changed to a new variable that that'll be x's and w's.
And that will be greater or equal 0, right?
Because the x was always greater or equal 0, and the slack says Ax greater or equal b is turned into slack greater or equal 0.
And now that multiplies A minus i to give b, because Ax minus the slack variable is b, which says that Ax minus b is the slack variable w -- bring that over and that over -- and that's what we said was greater or equal 0.
Do you see that I've changed to an equation by introducing more variables?
Putting the x's and the slacks all together in a big variable that I'm now going to call x.
So this is now the sum of the m plus n of these now because this is the new x and this is the new A.
You might say why didn't I just start with equality constraint, and I certainly could have done.
But just to see that inequalities have their place too, and to see that we can get between one and the other.
OK, so now this is the problem with equality constraint.
So my new constraints are Ax equals b and x greater or equals 0.
That's the primal constraint.
And what's the dual constraint?
So the dual constraint is y greater or equals 0.
Right?
And, OK I have to get this right because we're right at the end.
And the slack, let me just write the slack one.
The slack one is s is the slack.
This is s. I'm going to transpose so that I have consistently columns vectored.
So that when I transpose it says that A transpose y plus s is c. Right?
I put that over there with the s and transposed to get column vectors.
I like to have column vectors.
OK, so those are the constraints, but now what's the derivative equals 0 equation?
Derivative equals 0 is the derivative of this equals 0.
So what does that say?
That says that if I set the derivative as a 0 that says that c sub i -- the x remember is, well x has got all these components.
c sub i is alpha and the derivative of log x sub i of course is 1 over x sub i.
So that's the equation for derivative equals 0.
So this is what I'm solving.
OK Equality is here, equality is here, equality is here but non linear.
This is of course non linear.
So Newton's method just says linearize.
Newton's method is just linearized at the point, and that gives you the direction to move.
And you move that direction because you've linearized.
As you move, you're wandering a little away from precision, from perfection, but if you don't take too big a step, Newton is safe.
Maybe since this is a course in scientific computing, I should've written on the very first day in big letters Newton, because that idea of following the gradient is the central method of solving non-linear equations.
And then on the board beneath I would have written in big letters Carefully because the derivative is a local thing.
And if you follow the derivative a long distance, follow the derivative here a long distance out to here, who knows what -- you've lost the safety of Newton's method.
So Newton's method always comes in reality with some kind of a trust region, some region where you can rely on the derivative being a reasonable approximation of the way the function is moving.
OK, so we do that here too.
OK. maybe I won't write out in full notation.
What does Newton's method do actually?
So Newton's method we're at a particular x, y, s, and we've got to move.
So the unknowns are --- the components of x , the components of y, and the components of s. So Newton's method takes steps: a delta x, a delta y, and a delta s, computes what those should be, and then that gives the direction, and if you take them exactly, that's the full Newton set, which you would be very happy to do because that gives terrific conversion, but if it's too big a step, then you have to cut back.
So the equations for these are what you need so there'll be an A delta x will be 0.
Because b isn't changing.
There will be a A transposed delta y, a transposed delta y plus delta s will be 0 because the c isn't changing.
And then we'll get an equation out of this, which is a really significant one, but maybe time is running out on and I'm not going to do justice to.
But that's a non-linear term, where you see if I keep A delta x 0, then my new x is exactly feasible right?
If I'm at an Ax equals b and I move it by a delta x that's in the null space, then I have -- All I'm saying is that when I take that step I will have A x plus delta x still equal to b.
Good.
Constraints still satisfied.
When I take this step, since that's linear, the constraint when I add on the delta y and the delta s and the 0 I still have -- my new point still satisfies that constraint.
But this is of course not exactly satisfied.
If I had the solution to this, I'd be done.
That's my problem.
Anyway, so it's not exactly satisfied.
Newton would tell you a linearization of it, and you would move in that gradient direction to try to make the things -- to try to make equality hold, because our current x doesn't have equality hold.
And of course the c is a transposed y plus s. So that equation -- You see what's going on here?
This is a transposed y plus s, and the x is multiplying those, so there's a product there.
And when I take the derivative it's a product rule.
I get two terms.
Anyway, I get a third equation from here for delta x that connects delta x, delta y, and delta s. I take that step and that's my interior point method.
That's my Newton step.
So maybe I just end by reporting results, so I'll end with just two comments.
First is, is the method any good?
And of course you only know by trying.
And the answer is yeah.
Typically you get the gap -- you get the duality gap down below 10 to the minus 8, which is usually very satisfactory in 20 to 80 steps.
You can never prove a statement like that because you can always create some awful example, but this is the typical performance of the method.
Which is pretty good, regardless of m and n. That's what's wonderful -- that the number of steps doesn't increase with the size of the problem.
Of course, the cost per step does increase with the size of the problem.
OK so that's the results, and that's why the method is popular.
And now I just wanted to not leave duality, which is such a key idea, without going back to our much more familiar problem of quadratics, where there are quadratic terms.
And the best model you remember was projection.
You remember that we had a victor b and we have the line, the null space of A.
This was the column space of A.
This was all Ax's.
And perpendicular to it was the null space of A transpose.
All A transpose y should equal 0.
Do you remember this?
This was the model problem for understanding.
So that the projection of this solved one problem.
The projection in the other direction -- we called that p. This was the projection p equal A times the best x.
The projection in the opposite direction found the e, the error, but it was the solution to the dual problem.
And now I want to say where was duality in this picture?
Well duality was -- let me call it e hat, the winning, the projection, the right guy over here.
Or maybe y hat.
OK where was duality?
Duality came in this case in the fact that it was Pythagoras.
Duality in this simple, beautiful problem was simply the fact that p squared, this winner squared, plus e squared, was b squared.
The winners were the orthogonal projection.
And now where is weak duality?
It's the last second of the lecture.
Weak duality says take something that's allowed, like that and take something that's allowed here, like that.
And then those are not the winner.
Those don't deserve stars or hats.
They're not the winner.
And compute that squared plus that squared.
So this is any Ax.
So any Ax squared and any y -- let's call that y squared.
And what is the inequality that is satisfied by any Ax, like the wrong one here, and any y, like the wrong one there, will satisfy.
Pythagoras won't be quite right.
It'll be Ax squared plus y squared.
What do we know about the sum of those two squares?
It's greater than or equal to b squared.
The only way we get this thing split into two orthogonal parts whose squares add up to b squared is right triangle.
If I replace this by something longer and I replace this -- I should take that error really.
e is really b minus cAx.
That's what I should be putting here.
This thing should be b minus Ax squared.
Anyway, the duality is in the fact of getting an equal sign there and weak duality is the easy inequality that no matter what you do, you get greater than or equal.
So the duality gap is somehow the gap there, and the whole subject of optimization is to bring that gap to 0.
So this is the gap in quadratic problems, of which this is a neat model and this was all about linear programming.
And duality is present for both.
OK, so Friday is the promised lecture on ill-posed problems.
And meanwhile if two people are willing to put up a hand now or email me later and say sure I'll take my turn Friday of next week, that would be terrific.
OK.
Thanks.
I see one hand.
OK.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
All right, so I'll start with this review board, which shows the conservation law.
Scalar, one unknown, one dimension, x, one space variable, and the flux function is f of u.
That's the differential form.
We have seen that at some points that differential form produces a discontinuous solution, a discontinuity.
In that case we are better off with the integrated intgeral form.
So I just integrate that one from a point x left to a point x right from an a to b and that gives me-- I have then the integral.
So if u was density for example, the integral of density will be the mass.
That's the quantity that I'm seeing here, so this is the rate of change of mass in the interval, comes from flux going out minus flux coming in.
So that's the conservation law in its pure statement.
And we learned that solution for this-- while we're looking at the differential problem-- we can follow these characteristics from the initial time zero.
So at time zero we look at a point x and the value at that point just travels-- is sort of safely carried up the characteristic line, which is that equals x0.
And the problem arose when characteristics either collided into each other or separated.
Those are the two difficulties and they have different resolutions.
So this is what we met in the traffic example when a light goes red.
When a light goes red cars are coming up to the light, in front of the light the density is low.
The cars have to brake quickly.
So something happens-- that's a question mark there.
What do you do when characteristics collide?
When the value here cannot be both the initial value that's coming up that characteristic and the initial value that's coming up there because they're different?
So we need a rule for what to do there.
And we also need a rule for what to do in case of an empty space.
When no characteristics-- here are two characteristics are hitting, here are none are touching there.
Characteristics are separating in this case and the question is, what solution do you fill in the middle when you haven't got a characteristic?
It's not coming from the start.
OK, what solution to choose?
The answers will be different.
And let me take the first one, the collision question, first.
So what will happen is a shock will form.
There'll be a shock line so that these characteristics carry information into the shocked, by the way, that's a big deal-- until they hit the shock and these characteristics carry their information.
And across the shock we have different values, u on the left and u on the right.
So that's one possible form of the solution because that will solve the equation in this region where the characteristics from the right are following the formula.
It'll solve it from the left, so everything comes down to the jump condition at the shock.
What's the shock speed?
Where does this shock go?
It's to find that shock in the xt-plane that I have to use this integral form.
I must use the integral form because I've got a discontinuity and the differential form just doesn't make sense.
OK, I wrote down the key idea here or the conclusion, but not the reasoning.
I'm looking for this jump condition at the shock, which is going to locate the shock.
And the key quantity that I have to find is the shock speed, which will tell me where that shock is going, how it's moving up in time.
So time is going that way, x is going this way in this picture.
So this has a speed, s of t. And here's the conclusion.
There's the jump condition.
So what does it mean?
This notation, this bracket notation is the jump.
The shock equation is beautiful.
It says that s, the shock speed, times the jump in u is the jump in f of u.
And let me take this example that is sort of the Burgers' equation example, so this is the example that comes from Burgers' equation.
That's the example where the flux is this nice parabolic function u squared over 2.
Then the change in flux, the jump in the flux function is u right squared minus u left squared over 2 and I divide by the jump in u, which is u right minus u left and that cancels beautifully and leaves me with the 1/2 of u right plus u left.
It tells me that the shock speed is exactly halfway between, it's exactly the average in this Burger's example of the characteristic speed from the right than the characteristics speed from the left.
And notice, it's in between.
That's crucial.
That tells us that this thing here is good.
It tells us that characteristics are entering the shock from both sides.
Characteristic lines are entering the shock.
The shock is somehow in this sort of nonlinear case, it's somehow kind of cutting out bits of the solution, reducing the energy, reducing the entropy, reducing the total variation as we'll see it in a minute.
So that's the answer here and I guess I have to justify, where did this come from?
Well, it came from this integral form.
Suppose I integrate from a little bit to the left of the shock to a little bit to the right of the shock.
Of course, that's where I want to integrate in order to capture the shock.
OK, so if I just take that equation and write it over here and I'll imagine-- because I'm not going to be x left and x right are going to be pretty near the shock-- u up to the shock will pretty much have a constant value, u left.
So approximately that is the shock position.
Let me say that shock is at captial X is the position of the shock.
So I have x minus x left times u on the left of the shock.
I'm doing that integral.
Oh, I need d by dt of all this.
So d by dt.
It's d by dt of the integral of u dx.
Let me just put here what I'm computing.
I'm computing that from x left to x right.
Well, I'm going up to x, up to capital X where the shock is and then onward.
So here I went up to x using the value u left.
And now will be the onward part, which will be x right minus x, the half of the interval that's on the right of the shock times u right.
And then copying this term is exactly our jump in f of u.
Plus the jump in f of u is zero.
Those words, total variation are a reminder to myself for what comes next.
OK, so is that OK?
That will be approximately zero because u wasn't exactly constant on the left and right.
It didn't stay at the value ul, but we're squeezing the interval down where it's essentially constant.
OK, so now just look and see what we've got.
dx dt is the shock speed, so that's s. Multiplying ul and here I have the derivative of that is zero, the derivative of xr is zero.
Here I have a minus, so it's minus ur.
dx dt times that number is e plus the jump in f of u, left to right, is 0.
And that's our equation.
If I put this on the opposite side it becomes u right minus u left.
Shall I do that?
So I'll keep this on this side, got a good sign, but I'll move this onto the other side so it will be the shock speed, d capital X dt times ur minus ul, which is the jump in u.
So that's great.
We now have a way to deal with shocks that collide.
So that was a derivation of this jump condition, which first had a name, two authors [?
Rankin Whogonio ?]
in gas dynamics, but then it was seen as the right condition in all conservation laws.
OK and now I'm ready to comment on the second issue.
What do you do when characteristics separate?
Well, let me say what you don't do.
You could, without wrecking the equation, you could try to put a shock in there and give this the value, u on the left even though a characteristic isn't telling us what's going on in this space or this one.
You could try to put a shock in there.
You could maybe satisfy the jump condition if you put the right shock in, but the characteristics would be coming out and that's wrong.
And it's called the entropy condition, which says that this can't happen.
That the derivative of entropy has a certain sign and it rules this out.
It rules out this possibility of characteristics emerging from the shock and we need a totally different solution.
And it turns out to be a fan in there.
So we put in a fan of characteristics coming from here and that gives a solution that actually will depend on x/t.
This solution will be a simple ratio, x/t enters that.
In other words, that gives us a smooth solution and I guess thinking about the traffic example, that's exactly what happens.
So this is the shock part, now for the fan part, what happens when we have a bunch of cars sitting there, ready to go, the light goes green, the first cars start moving, cars behind them see the car in front moving, they start moving and so the initial profile would be, in this case, for cars, would be high density and then zero density.
So this would be maximum density sitting at the light and zero density in front of the light and then the light goes green now and what happens?
Well, as we all know those first cars pull ahead and the profile is these cars haven't yet started to move.
These have started to move and there's a profile there and then zero density.
So this represents the first car, the lead car.
In front of that car is zero density and that's the fan.
That's the x/t expression.
So a sample linear expression, which will flatten out as time goes on, these cars will be going faster and faster and then eventually up to full speed.
OK, so that's the fan.
Maybe now is my chance to speak about this concept of the variation in a function.
The idea will be you now, how do we find the difference method to do this automatically?
Well, OK, one way to do it is put in a little viscosity.
This rule, the rule that picks out this shock in the one case of colliding characteristics and this fan in the case of separating characteristics, one way to get to those two solution would be to put in a little bit of viscosity and then let epsilon go to zero.
And the result would be these two solutions.
And in a way that's what finite differences is going to do.
Finite difference or finite volume is going to be a numerical viscosity and the epsilon in the Burger's equation would become delta t or a delta x in the numerical method in the finite difference equation.
And then what's the concept?
So we have to find finite differences.
And we want those-- the point is finite differences, sort of takes more care.
We can't just create something that's consistent in the smooth case.
Up till now we've just had no hesitation.
We replaced derivatives by differences without too many worries, really.
And that was fine as long as the solution is smooth.
But now if we're going to replace derivatives by differences we have to do it in such a way that this entropy condition, these solutions emerge automatically.
And really, in a way that our finite difference or our finite volume methods should somehow conserve mass the same way the true equation does.
So those are two different points.
First point is this question of total variation.
What's the total variation in a function?
Well, let me draw a function.
The total variation is-- well, let me see.
Total variation in a function u is going to be, by definition, the integral of the absolute value of du/dx.
OK now we have to understand this concept.
It's beautiful that the analysis has led to a quantity as simple as that, as a guide.
So when we create finite difference methods we will ask, are they total variation diminishing?
So TVD will be total variation diminishing methods.
Methods where this quantity does not increase in time.
Let me just understand this quantity first.
The absolute value is crucial.
If the absolute value was not there then I would just have the integral of the derivative, so it would be u at that point minus u at that.
But here I'm taking absolute values, so for awhile the derivative is positive up to there.
So the integral gives me this value minus this.
Then in this region the derivative is negative, but I'm taking absolute value.
Do you see that that integral will split into 4 separate integrals?
The total variation will be this difference plus this difference plus this plus this.
I'll have 4-- in this example-- have 4 positive contributions to the variation.
And that's the key quantity there.
And what a shock will do-- can I maybe mention in this figure what a shock does, can do, would be to somehow cut out part of the variation.
When a shock occurs it might happen that these things are going into the shock, these are going in from that side, and the shock kind of jumps from this value, u left, to this value, u right following the shock condition when it emerges.
And this additional variation is wiped out.
You see that the variation for the solution that contains a shock is still-- this part is still there, but this part is all downhill, so it's just this.
It's that minus that.
And the middle two parts have been cut out.
OK, so that's the total variation in a function.
Let me just take the most important example that's not total variation diminishing, which is Lax-Wendroff.
The one even in the linear case.
The one experiment we've done with that profile, if you remember we had a wall of water and the exact solution brought the wall to the left with velocity c, with speed c, but Lax-Wendroff, which did quite well staying near the discontinuity near the jump had a little oscillation behind.
That was essentially our one objection to Lax-Wendroff was that oscillation showing up.
And so that's telling us that the variation--see, the variation in the true solution is just that step.
The variation in the Lax-Wendroff answer is this bigger step plus this one plus this one plus this, larger than 1.
Roughly the match is that monotone methods.
So here's the general picture.
We can get monotone or I'll say, all positive coefficients and those will be total variation diminishing.
Great.
Why don't I draw-- recall a monotome example was Lax-Friedrichs or upwind.
Upwind will be our model, so let me draw the upwind one.
The upwind one didn't increase the variation, it stayed at 1.
We had no oscillations, but it was low accuracy.
So first order.
We can't do better then first order accuracy, the order of accuracy I'm measuring in the smooth part of the solution because that's where I can take taylor series and match terms and see which is the first one that doesn't match.
And Lax-Friedrichs and upwind was first order.
Then our problem is that if I allow a negative coefficient then variation increases, ocsillations will come in, but I can get the accuracy up to second order or higher.
And Lax-Wendroff is the example, the key example there that I'll continue with.
OK, that's the idea of total variation.
It just gives us a quantity to work with when we see oscillation.
And of course, I'm interested in oscillations, so this is a linear case with a discontinuity.
I don't think it's quite fair to call it a shock.
At least, the shocks that we meet in conservation laws in the nonlinear problems, they're created-- the ones we saw here are created for nonlinear reasons.
Colliding characteristics, whereas the linear case has characteristics all parallel.
So here we're in the lecture, we're really into the second lecture into the scientific computing part of our problem.
So that's the analysis part, the pd part.
Now comes the experience of quite a lot of people working hard to identify a way to get second order accuracy where it smooths and monotonicity or something like it where the shocks are.
So that's essentially the goal.
Our method is going to turn out to be nonlinear.
It's going to have some-- it'll be called a flux limiter.
The key quantity will be the flux.
And for a smooth function when u right and u left are very close, the flux is small, we don't have a problem.
But at a shock where there's a significant jump in u and especially in f of u, the flux limiter is going to come in, into the discrete method.
Let me began discrete methods with a new word, finite volumes.
And with a new interpretation of capital U.
So the discreet value, capital U jn, which up to now we've been thinking of was that's an approximation to the true solution, j delta xn delta t. Now because we're dealing with discontinuities we better integrate.
We integrate over a delta x interval.
We take the average over a little delta x interval instead of hitting just the point.
So the delta x interval from j minus 1/2 to j plus 1/2, we take the average of u at the time, n and delat t and that's what finite volume-- and let's think of as capital U, the discrete approximation.
The appropriateness of that is in the integral form of the conservation law.
So the integral form of a conservation law leads us to that natural discretization.
d by dt of u, of the average.
You're keeping your eye on the integral form of the conservation law.
Just come back to it again.
The derivative of the mass, the integral of u, so in the discrete case the difference of capital U plus the integrated term, a flux at one end of the integral minus a flux at the other end.
And notice that we are getting for the flux, 1/2.
So a staggered grid is presenting itself naturally.
And so that f j plus 1/2, the fluxes, which is the critical thing is on the j plus 1/2 grid.
All right, now this could be exactly defined as the flux at that grid point.
Capital U squared over 2 in the Burger equation at that grid point.
That would give us a totally straightforward, first order numerical method.
First order, I see first order here in time and it might be better than that in space because in some way that's centered.
But if we want higher accuracy, if we want to include Lax-Wendroff for example, that includes several values of u, we go to a more general flux function.
The whole job here is create a smart flux function.
So a flux function depends not only on u in the center, uj-- at time n, this is all at time n. It can also depend on values to the left of it and values to the right.
It may use those values, it may not, but we allow that possibility then that the flux-- so this is the numerical method that you aim for.
Is to choose a flux function and of course, this should be consistent with the original problem and that simply says that if all the u's were the same that the numerical flux function would give you the flux function in the given equation.
OK, so this is our general pattern.
And now the question is really, maybe the first step is I would like to write upwind and Lax-Wendroff.
I'd like to find their flux functions.
I guess I'm saying that when we look at an equation in this format, we can take methods that we have already used and see, OK, even in the linear case, the one-way wave equation-- what are we picking for the flux function?
OK, now the one distinction is-- the one-way wave equation we had a constant and upwind for example, went in the direction decided by that speed, c. If the speed reversed sign so that the waves were going the other way, the wind is coming the other way, the upwind direction is the other way so our method would change.
Let me just write down the flux function that turns out to come for upwind and then for Lax-Wendroff.
OK, so for upwind the flux turns out to be f, turns out to be 1/2 of-- because it's at this, this is fj plus 1/2, really.
It's at the center point.
The flux function will be, I'll put in the a. a delta t-- sorry, a over 2uj plus 1 plus uj.
Now plus, or rather minus actually, comes absolute value of a/2 times uj plus 1 minus uj.
If we're patient we can track down the two cases, a positive and a negative, and recognize that this form switches direction correctly.
The difference method.
So it keeps us upwind in other words.
If the wind changes direction our flux changes appropriately to follow that direction.
So that's an OK flux function.
It's a monotone one and it's first order.
Now let me put in Lax-Wendroff.
What would Lax-Wendroff be?
Would be the same, the same upwind flux, if I call that up for upwind and then a correction, a Lax-Wendroff term that gets the second order accuracy.
OK, and that turns out to be, it turns out that I multiply this term by that ratio, r. By the absolute value of r. So it turns out that it's the Lax-Wendroff thing, to get that extra accuracy.
You might remember, had an extra delta t over delta x.
And so it's this same thing times a over 2.
In the nonlinear case it'll be f of uj plus 1. f at uj j plus 1 minus f at uj.
Next time I'll come back to this point and pin down this more carefully.
All I want to do in completing today is introduce this idea of a flux limiter.
So what people say when they look at Lax-Wendroff is that this term that's been added on is fine in the case of a smooth solution, but in that case of a jump like our picture here, that term is increasing variation.
It's an anti-diffusion term.
Instead of smoothing things out the way f upwind does, that extra term has the effect of increasing the jump.
It's anti-diffusive and therefore when it's a significant term the flux limiter cuts back on it.
The result will be a numerical method, which is not linear anymore even if we apply it to a linear problem.
The result won't be strictly linear and it will produce a profile that doesn't have that oscillation.
So it has what we want.
It gives us good answers when the solution is smooth, it stays near the shock the way Lax-Wendroff does, but it doesn't oscillate.
OK, so what's the flux limiter?
The flux limiter is, I multiply in here by some function, the limiter function, phi j plus 1 and I multiply in here by phi j.
In other words, there's an extra factor, this flux limiting factor.
So the question is, how to construct that?
What formula do we use for the flux limiter?
So the flux limiter, after a lot of experiments is chosen to depend on-- so we want it to be 1.
So what do we want?
We want it to be 1 or close to 1 and smooth part and we want it to be greater than 1 at a discontinuity.
OK, so our question is, how do you identify that discontinuity?
Well, the convenient way has turned out to be look at the ratio-- look at differences.
Look at the ratio of-- so what comes in here will be the ratio r say j plus 1/2.
That'll be the ratio of u, say j plus 2 minus uj plus 1 to uj plus 1 minus uj.
All that time n. That ratio tells us, that ratio is close to 1.
So the ratio r is close to 1 for smooth and our function, phi of r then, we want our function phi when the ratio is 1 to give the value 1.
So if I try to draw a graph of-- let me draw a graph of peoples flux functions-- of the phi of r. So here's a graph of phi of r. First on the graph I'll put upwind and Lax-Wendroff without anything.
So Lax-Wendroff would take-- phi Lax-Wendroff is identically 1.
This is the ratio r and this is phi.
So Lax-Wendroff before fixing had no flux limiter function and it just had 1.
And actually, if I can keep the same formula, phi upwind would just be zero.
Then it'll turn out that-- let me draw these ratios.
There is a region here and here where the flux limiter is doing the right thing.
So in that region it's acting the right way.
It's limiting the anti-diffusive flux.
It's controlling the variation, it's controlling the oscillation.
So various people have proposed-- let's see, no.
I haven't got this right.
I think it's- because I know that part is wrong for Lax-Wendroff.
So maybe it's there.
No, I've forgotten-- sorry that I'm not, I've lost this because I know that one possibility is this as the flux limiter.
One possibility would be this one and then there's another possibility that is OK that goes through here.
Let me stop for today with this idea of a flux limiter and you see that it's a task to fix the method to do the right thing at smooth parts and also the right thing at discontinuities.
OK, I'll pick up on that next time.
Thanks.
The following content is provided by MIT OpenCourseWare under a Creative Commons license.
Additional information about our license and MIT OpenCourseWare in general is available at ocw.mit.edu
OK. We're ready to go.
So this will be the last lecture that I give maybe, almost, and the last one that will videotape, and then Monday we start on the presentations.
And we'd better get rolling on those because time will run out on us, and I want everybody who would like to to have a chance to talk about work.
OK. Can I begin?
So I promised to speak about inverse problems, ill-posed problems.
It's a giant subject, but there's one thing I didn't do , one beautiful thing that I didn't do well at the end of the last lecture.
And I if you don't mind I'd like to go back to that because the number one example for optimization is b squared, minimizing b minus Ax squared.
Here's b; here are all the Ax's.
Think of it as a line, but of course it could be a n-dimensional sub space.
And then this is the winner.
And the reason it's the winner is that it's a right angle, it's a true projection.
And there's a second problem, which is minimize this thing, the distance to w, where w lies on this perpendicular direction.
And, of course, the closest point w to the b in this direction is there.
So those are the actual winners that get a little star.
And when we're there, the length of this squared plus the length of this squared equals the length of b squared.
We have right angles, and Pythagoras a squared plus b squared equals c squared is right.
But then so what I added just quickly at the end was the key point about suppose I take any allowed Ax and any allowed w. Then I would expect -- and I look at that distance which is larger than this, and I look at this distance which is larger than this, so of course, now I'm not looking at the optimal ones but instead looking at these solid line ones, and then the b minus Ax squared is bigger than it has to be.
The b minus w squared is bigger, so the sum is b squared plus another term.
So that's weak duality that for any choice some inequality holds, since this is never negative -- another way to say weak duality would be to say that this is always greater or equal to.
But what I didn't throw in last time was there's this beautiful expression for what the difference is.
If you don't have the optimum, this is what you miss by.
And one part of the beauty is that that tells us right away how to recognize the optimum.
Duality holds, equality holds, without this term when this term is 0 when b is equal to Ax plus w, and of course that's the good dash line case when that vector plus that vector is exactly b.
Ax and the w make b.
Those are the dash lines that's optimal.
So somehow the optimality condition is this one, and this is a condition that connects the two problems.
See here we had a minimization of this.
w didn't appear.
In the dual problem we had a minimization of this.
x didn't appear.
But when we get both problems right then they connect.
And this is the step that in my three step framework for applied math that dominated 18085.
There was the x, there was the b minus Ax, there was the shift over to w. So matrix A entered there, A matrix A transpose entered here to give A transpose w equals 0.
And here is the bridge.
That's the bridge between them which in this simple model problem is -- I usually write that bridge with a physical constant c, and in this simple model problem c was 1, or c was the identity.
So then this equals this, which is our optimality condition.
So when we have that bridge between one problem and the dual problem we've got them both right.
So I'm happy to find that.
But I have a problem for you.
You may say where did that identity come from?
So that's simply an identity.
And it came from just multiplying it out.
If I multiply out take b minus Ax transposed times b minus Ax.
Take this transposed times itself.
b transposed b This transpose this.
It works.
The terms cancel, and this is an identity.
Now that's a simple identity in geometry, so my puzzle for you is take this -- let me draw the identity here.
So here's my picture.
I'm going up to any w. And here is b in this picture.
So b minus w is there.
This says that some vector squared plus that squared equals that squared plus that squared.
So now I just have to find all those vectors and ask you prove it.
So let me name all these vectors.
So here is b minus Ax.
Right?
That's b minus Ax.
And here's w. OK. Let me get them all right.
I have to get b in there and here is b minus w, and here's b minus Ax.
And what's the fourth one?
The fourth one is this mysterious one.
So I go up b minus Ax w. I'll have to put that here.
I'm going up this same w, straight up.
There is the mysterious fourth guy: b minus Ax minus w is there, and this was the b minus Ax.
And this identity holds.
It's gotta be like Pythagoras could have figured it out but how?
Let me draw this very same picture again.
So it's this line, this line, this line, and this line.
And let me just call those a, b, c, and d. And they connect.
Oh sorry, there's a rather important fact.
So this statement here then says that a squared -- no what does it say -- a squared plus c squared maybe is the first two, and this one is b squared plus d squared and why?
Please submit a proof to get an A. OK. Let me get the picture right.
So this is just dotted lines here.
So that's a rectangle, and there is d. So I have four lines starting at the same point.
Two of the lines go to the corners of a rectangle.
So it's a geometry problem, which struck me early this morning -- too early this morning.
But it must be, of course, it can't be news to the world.
But I think I've got it right: a squared plus c squared equals b squared plus d squared.
We take any point and connect to the four corners of a rectangle, and that holds.
And the question is why?
It holds from algebra, but somehow we also ought to be able to get it out of Pythagoras.
So as far as I know none of these angles are special.
That might look equal to that or something, but it's not, I don't think.
I don't think.
You know, that point is off this d point, it's anywhere.
It's off center, and here's the rectangle that it's connecting to the corners of.
So there you go.
Open problem.
Why is that true?
And I'm sure it is, so this isn't a wild goose chase, but I'm just wondering what proofs can we fight for that.
OK. Can I leave you with that?
I'll leave that on the board and hope you'll listen to my presentation about ill-posed problems, but I hope you copied that little picture and will either email me or hard copy, or whatever.
Anyway let me know a good way to prove it.
OK. Now the lecture begins.
Today's lecture begins.
Oh, I had one other comment about the interior point method with the barrier problem.
I got down at the end to an equation -- you remember I was taking the derivative, solving the barrier problem, which was minimizing cx minus some multiple of the barrier log x sub i.
So I took the derivative of this quantity and set it to 0.
And the derivative in respect to x gave me the equation c equals alpha over x.
Anyway I kind of a lost my nerve, but all I want to say is this is right and it leads to Newton's method that we spoke about.
I won't go back to that.
All right.
Inverse problems.
I think maybe the best thing I can do in one lecture about inverse problems is first of all to get a general picture of what are they.
Secondly to mention areas that we will all know about, where these inverse problems enter.
And then thirdly to look a little bit at the integral equations that often describe inverse problems.
Inverse problems come from many sources, not only integral equations, but integral equations are responsible for quite a few.
But let's think about others.
OK, so really I plan now to list various examples.
And number one I've already spoken about find velocities from positions.
Take the derivative.
So taking the derivative is a process that makes things bigger, so when we go the other way we're -- so the difficulty with the problem is that a small change in the position data may be a very large change in the velocity.
For example, suppose the position is the correct position, say x of t, plus some noise term that's going to be small but small in size, small in amplitude, but not small in derivative.
Maybe like sine of t over epsilon.
So that would be a case in which a small noise term in the position -- so this is the position -- has a big effect on the derivative.
And that's why the problem is ill posed.
The problem is ill posed -- and it goes with our intuition that high frequency.
This 1 over epsilon down below is producing a high frequency oscilation here.
And of course everybody realizes the velocity is the correct, the x dt, plus the derivative of this, which brings out a 1 over epsilon, maybe it's a cosine.
Maybe it's a cosine.
So the 1 over epsilon cancels the epsilon; that's a cosine of t over epsilon.
So this was small.
A small change in position produced an order of 1 change in the velocity.
So if we only know position within epsilon, we're in trouble.
And this is exactly the point of ill-posed problems.
Then our velocity could be that.
I could make this example worse if I increase the frequency further -- put an epsilon squared there.
Then the amplitude would still be a small, but when I take the derivative, there'd be a 1 over epsilon, the amplitude would actually be very large.
So I was just modest to keep epsilon and epsilon there, so that they canceled each other and produced an effect on the derivative.
So that's the problem.
If you have noisy data about position, how can you get velocity?
OK, so that's like example number one.
It's very important, and I actually I have no magic recipe for it.
But let me mention other problems that you'll know about like well seismology.
A typical inverse problem in seismology would be find the density find earth density, say from travel times of waves, from wave travel time, which is what we can measure.
So that's seismology and also, of course, everybody understands that this is what oil exploration depends on.
You set off an explosion on the surface of the earth.
The wave travels into the earth and some part of it bounces back, and in fact maybe several pieces bounce back at different times.
And from those results you have to sort of back project to find the density.
So back projection is a word that comes into several of these applications.
Of course, another one would be the medical ones: CT scans, MRI, PET -- all these ways to take measurements.
And from those measurements you have to find the density, so you're looking for density of tissue because you hope that would allow you to distinguish a tumor from normal tissue.
So that's a giant area of application.
Oh, another one would be find the density of the earth -- let's say another way to find the density of the earth, another bit of information we have from the gravitational field.
You see, that's what we can measure: the effect of gravity, the effect of the earth's density.
We measure the effect, and we want to know the cause.
That's the problem.
And this reminds me that there is a special lecture coming by Professor Wunsch, Carl Wunsch at MIT -- he's outstanding -- and that's Wednesday, May 10 at 4 o'clock.
And in fact his abstract which you might see somewhere -- I can post it on the course website -- his abstract tells, he's solving a very, very large-scale, ill-posed optimization problem.
Least squares problem.
Perfect for this course.
So those are familiar.
The books I've been looking at just list whole lots of examples.
Let me see.
Oh, scattering.
Let me just keep going here.
This is number five.
Scattering.
From scattering data find the shape of the obstacle that's responsible for the scattering.
So that's a giant example of many air force applications and many other applicatios.
But we recognize if you want to identify some object by scattering, by radar data and other scattering data.
Well there are just lots of others.
Oh, and then we have a Laplace or a Poisson equation, which is the divergence of some inhomogeneous material property equals the sum f of xy.
OK, so what we're usually doing in this course and most courses is the direct problem of find u, so the direct problem is find u.
And what's the inverse problem?
The inverse problem is we know u and f and we have to find c. So find the coefficient.
Well, it may not be instantly clear whether it's possible.
In fact, it probably is not possible, and that's what makes the problem ill posed.
Yet if you have enough measurements of inputs and outputs, you could reconstruct the matrix.
Of course, a person like me is going to think about the matrix question.
Suppose I'm looking for the matrix A Can I call this number seven?
And since it's in matrix notation, it doesn't take much space.
Usually I know the matrix, and I know b and I want x.
In the inverse problem I know b and x and I want to know the matrix.
What was the matrix that produced it.
Well obviously one pair bx is not going to be enough to produce the matrix, but enough will.
But then if there's noise -- this is the point, of course, that there's always noise.
So that's what I now have to deal with.
The main thing is how to deal with noise in the data, noise in the measurements, because if the problem is so posed, we saw even in that simple cooked up example that a small amount of noise could produce a big difference in the solution.
OK.
Right.
So those are examples.
Now I wanted because math courses and this one never mention integral equations, I thought I would write one on the board.
And these examples fit in this -- if I describe them mathematically or another whole list of problems that I'm seeing in the books on ill-posed problems -- very often they are integral equations of the first kind.
Again, the direct problem is -- I'd better write it down -- the direct problem -- this is known .
So the direct problem is given the k, which is a bit like c over here, find the u, solve for u.
Solve for the unknown u.
It's a linear problem.
It's an Ax equals b problem, only it's in function space.
And of course one way to solve it will be, probably the way to solve it numerically will be somehow make it discrete, bring it down to a matrix problem.
That's what we would eventually do.
But in function space, integral equations played a very very important historical role in the development of function spaces.
And now then the inverse problem would be given u find the x I guess.
Something like that.
That would be possible.
That would be one possible: inverse number one.
But I wanted to make some comments on integral equations, just so you would have seen them.
The integral could go up to x or it could go up to what.
And Volterra and Fredholm are the names associated with those two possibilities, but these are both -- whether Volterra or Fredholm -- they're both ill posed, whereas if I want to make them well posed.
Well we've seen how to make a problem well posed.
I have this operator A, which has a terrible inverse or no inverse at all, and the way I improve it is add a little multiple of the identity.
I'm supposing that I know about A, that it's not negative, that it's eigenvalues can be very very small or 0 but not negative.
So when I add a little bit, it pushes the eigenvalues away from 0 up at least as far as alpha.
And over here if I add in alpha u of x, that's what it does of course.
I've added alpha times the identity operator here and that's given me an equation of the second kind.
So those three minutes were just to say something about language and to look at an integral equation -- something we don't do enough of.
Integral equations, well some problems on nice domains, you can turn differential equations into integral equations, and it pays off big time to do that.
Professor White in EE, if he taught a course like this it would end up with half a dozen lectures on integral equations because he's an expert in converting the differential equation to an integral equation.
He would convert Laplace's equation to an integral equation, and the Green's function would enter and he would solve it there.
OK, so how does velocity fit?
Well, everybody can see that the integral of velocity -- so example, the velocity example is that the integral from 0 to x of the velocity, that's of course integral from 0 to x or, or 0 to t maybe would be a better -- So suppose velocity is the position dx, ds.
So it's x of t minus x of 0.
So this is position This is velocity.
And in the inverse problem, position is known, say by GPS and velocity is unknown, to find by GPS.
So GPS will give you a measurement of position.
Maybe you know something about GPS.
You know that there are satellites up there whose position is known very exactly, and they have a very very accurate atomic clock on them, so that times are accurately known.
So they send signals down to your little hundred dollar receiver, which of course has a ten dollar clock in it.
So there'll be some errors, partly due to the clock, largely due to the cheap time keeper.
But actually the way you get reall accuracy out of GPS is to have two receivers, and then you can cancel the clock errors and get less than a meter accuracy.
If you take account of all the sources of error, you can get it down to centimeters.
So GPS is giving you -- just your single receiver is still good enough.
It's measuring the travel time and since the signals are coming with the speed of light, that tells us the distance from each satellite.
Right?
Here is your receiver R. Here is satellite number one two three, four up in the sky, and you know these distance.
But you don't know the time very well.
So with four satellites I'm able to find the position of the receiver and somewhere in space and some moment of time that this clock is not good enough to tell us.
So we have to solve for that.
So four receivers sending signals to -- I'm sorry, four satellites sending signals to the receiver, we can solve that problem and find the position and time.
That's the fundamental idea of GPS.
Now of course you get better results if there's a fifth receiver, a fifth satellite, and a sixth.
The more the better.
And of course then it's going to be least squared.
Because you're still looking for four unknowns, but now you have six distances pseudo ranges, so we would use least squared, so by least squared.
OK.
But what about velocity?
Suppose your receiver is moving, as of course it is if you rent a car that has GPS installed to tell you where to turn.
And of course it has to have a map system installed so that it can look up for the map position.
So for many purposes you need velocity, and I'm not an expert on that subject at all.
I just comment that one way to get velocity is to take differences, so that velocity is approximately x of t plus delta t minus x or x of t. Two, let's say: x of t sub 2 minus x of t sub one over t sub 2 minus t sub 1.
But if we want to get velocity near a certain time then these t's better be near that time because the velocity's changing so we better be measuring it at the time we're wanting it.
And then if they're very close, then we're dividing by a small number and the difference -- the noise, the error in measurements, x, is multiplied by that 1 over delta t. And one way to avoid it is to go into the frequency domain.
So this is like an interesting option in a lot of these problems.
Can you operate better in the frequency domain?
Of course the ill-posedness is not going to go away.
It comes as we saw from high frequency.
But if we go into the frequency domain, and if these GPS satellites are sending at a certain frequency and as we move, of course, the Doppler effect of course is the fact that as the receiver moves, the frequency is observed to change a little.
Right?
Just says like the noise of a train going by is the familiar example.
Nobody ever sees a train going by anymore.
But it's the same idea.
But you hear traffic go by.
Actually, I guess that that's how we cross the street, come to think of it, by listening to the noise of cars, and our global internal Doppler says the cars are going away from us, in which case we don't worry, or it says the car's coming fast, in which case we're careful, or it says the car's coming slowly, and we get across first.
So we use Doppler.
I don't know exactly how, how our human audio system builds in Doppler.
Anyway, Doppler would be a change to the frequency domain and a restatement and perhaps an improvement in the in the problem.
OK, so those are examples without math.
Now here's a small bit of math as the lecture ends.
So the only math was this simple example.
So I guess I got one more board over here.
I'm going to put this geometry problem that you've been thinking about out of sight.
OK, so what's the key?
It's this Tikhonov.
Tikhonov Regularization.
Tikhonov Regularization.
And it's adding alpha.
Add alpha to the least squares problem.
And I thought it was amusing to notice, Tikhonov was born in 1906, so this is a hundred years exactly since he proposed this method.
It's one of the about five methods that the books describe.
And it's the one I'll speak about here at the end.
I'll just mention that one which might come up in a project possibly.
Other methods are used in iterative methods, like conjugate gradients, and stop when you're ahead.
See if you push conjugate gradients on and on and on, then eventually it's going to produce your exact ill-posed matrix with the big invert and unrealistic solution.
So you stop.
The same way for an integral equation.
We discretize that so we get a matrix process, which we can solve.
But if we refine the mesh so that the matrix gets bigger, it gets more ill posed.
So the closer we get, the closer the discrete problem gets to the true integral equation, the more [?
sick ?]
it is.
So there's some point at which you are OK, and then if you go too far, you're worse off.
So that happens with conjugate gradients.
Now what's the Tikhonov idea?
So the Tikhonov idea is look at them as, you'll know, it's a minimum.
I'll just call it A again.
Ax minus b squared plus alpha x squared.
That would be the simplest penalty term, regularization term.
So this leads to the normal equation: A transpose A plus alpha I, u hat, which depends on alpha, equals b. OK. so u alpha is the solution to that.
OK, now let's let noise come in.
So noise in b.
Noise yields a b delta, say a delta amount of noise.
A b delta is the measured observation.
See we don't know the exact b.
This is what we measured.
And we can suppose that the size of the noise is -- let's say -- delta.
So delta measures the noise.
OK.
I mean what's my question here?
Always the question is what do you take for alpha?
What should that parameter be?
if you take alpha very small or 0, then your problem is ill posed, and the answer you get is destroyed by the noise.
If you take alpha very large, then you're overriding the real problem -- you're over regularizing it.
You're over smoothing it.
So we don't want to take alpha too large, but we can't take alpha too small either.
And the theory will say that if we use an alpha -- So the theory will say this, essentially this.
It'll say that the u hat alpha, the difference between u hat alpha and u hat alpha with the noise coming from the -- Do you see what I mean by it?
This comes from the true b, which we don't know but it's the answer we would like to know.
This comes from the measured b with noise in it, and it turns out that this is of size delta over square root of alpha.
That's a rather neat result.
It gives us a guide because we want that to be small.
So we're assuming that we have some idea about delta, and it tells us again that if I take alpha very small, then I'm not learning anything.
So you see actually that we want, we need, delta over square root of alpha to go to 0.
Maybe we're reducing the noise.
Maybe we have a sequence of measurements that get better and better.
So delta goes to 0.
We would like to get to the right answer then, but as delta goes to 0, we better not let alpha go to 0 faster.
The message here is -- so I haven't derived this estimate but just written a conclusion, which is that as the noise goes to 0, that means our measurements are getting better and better, we do want to let alpha go to 0 but we can't overdo it.
And a good choice, a good choice is let alpha be delta to the 2/3.
That turns out from these little estimates to be the best choice, because then square root of alpha is delta to the 1/3.
This then is delta divided by delta to the 1/3 third.
This is delta to the 2/3, and we can't do better.
So that's the balancing.
That's the balance when you take that value of alpha.
Then as you reduce the noise, the error gets reduced in the same proportion.
It's not the full reduction in delta, but fraction the 2/3 power.
OK, so that board summarizes, you could say, the theory of regularization.
And there's a lot more to say about that theory, but I think this is perhaps the right point to stop.
OK so that's applications.
One other application, if I can add one last one because it's quite important and it comes up in life sciences.
So in life sciences you might have in genomics you might have a lot of genes acting.
So the action of n genes, n being large, produces some expression, the expression of the gene, and it depends on how much of those genes are present.
But what everybody wants to know is which genes are important, which genes control the blue or brown eyes, or male or female.
Not clear, right?
We have an idea from biological experiments where the important genes lie on the whole genome, where to find them in the chromosome, but this is what we measure.
We observe male or female, and we can change the genes, but there are more unknowns than, more dimensions than sample points.
We're really up against it here.
And we're really up against it because -- so what's going to measure the importance of a gene.
How important is gene number two?
Well, the importance of gene number two is identified by the size of the derivative.
That quantity, if it's big, tells me that the expression depends strongly on gene number two.
If it's small, it says gene number two can be ignored.
That's exactly what the Whitehead and Broad Institutes want to know.
Well, for cancer of course as well.
And my only point here in this lecture is to say that again we're trying to estimate a derivative.
We're estimating a derivative from few samples in high dimension, and it's certainly ill posed.
And it certainly has to be regularized, and it certainly has to be studied and solved.
So that's maybe a seventh example, and with that I'll stop the final lecture and just bring down one last time this little bit of geometry to give you something interesting to do for the weekend.
OK, so I'll see you Monday then with a talk about different methods for solving linear equations and others coming.
And time is, you know, this semester will run out on us.
So please send me emails to volunteer.
And don't think you have to be perfect.
If you've got transparencies, got the topic in, mind got the question in mind, got some numerical results, you're ready.
OK. Good.
Have a good weekend.
Bye.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so the course you're currently sitting in is 18.650.
And it's called Fundamentals of Statistics.
And until last spring, it was still called Statistics for Applications.
It turned out that really, based on the content, "Fundamentals of Statistics" was a more appropriate title.
I'll tell you a little bit about what we're going to be covering in class, what this class is about, what it's not about.
I realize there's several offerings in statistics on campus.
So I want to make sure that you've chosen the right one.
And I also understand that for some of you, it's a matter of scheduling.
I need to actually throw out a disclaimer.
I tend to speak too fast.
I'm aware that.
Someone in the back, just do like that when you have no idea what I'm saying.
Hopefully, I will repeat myself many times.
So if you average over time, you'll see that statistics will tell you that you will get the right message that I was actually trying to stick to send.
All right, so what are the goals of this class?
The first one is basically to give you an introduction.
No one here is expected to have seen statistics before, but as you will see, you are expected to have seen probability.
And usually, you do see some statistics in a probability course.
So I'm sure some of you have some ideas, but I won't expect anything.
And we'll be using mathematics.
Math class, so there's going to be a bunch of equations-- not so much real data and statistical thinking.
We're going to try to provide theoretical guarantees.
We have two estimators that are available for me-- how theory guides me to choose between the best of them, how certain can I be of my guarantees or prediction?
It's one thing to just bid out a number.
It's another thing to put some error bars around.
And we'll see how to build error bars, for example.
You will have your own applications.
I'm happy to answer questions about specific applications.
But rather than trying to tailor applications to an entire institute, I think we're going to work with pretty standard applications, mostly not very serious ones.
And hopefully, you'll be able to take the main principles back with you and apply them to your particular problem.
What I'm hoping that you will get out of this class is that when you have a real-life situation-- and by "real life", I mean mostly at MIT, so some people probably would not call that real life-- their goal is to formulate a statistical problem in mathematical terms.
If I want to say, is a drug effective, that's not in mathematical terms, I have to find out which measure I want to have to call it effective.
Maybe it's over a certain period of time.
So there's a lot of things that you actually need.
And I'm not really going to tell you how to go from the application to the point you need to be.
But I will certainly describe to you what point you need to be at if you want to start applying statistical methodology.
Then once you understand what kind of question you want to answer-- do I want a yes/no answer, do I want a number, do I want error bars, do I want to make predictions five years into future, do I have side information, or do I not have side information, all those things-- based on that, hopefully, you will have a catalog of statistical methods that you're going to be able to use and apply it in the wild.
And also, no statistical method is perfect.
Some of the math people have agreed upon over the years, and people understand that this is the standard.
But I want you to be able to understand what the limitations are, and when you make conclusions based on data, that those conclusions might be erroneous, for example.
All right, more practically, my goal here is to have you ready.
So who has taken, for example, a machine-learning class here?
All right, so many of you, actually-- maybe a third have taken a machine-learning class.
So statistics has somewhat evolved into machine learning in recent years.
And my goal is to take you there.
So machine learning has a strong algorithmic component.
So maybe some of you have taken a machine-learning class that displays mostly the algorithmic component.
But there's also a statistical component.
The machine learns from data.
So this is a statistical track.
And there are some statistical machine-learning classes that you can take here.
They're offered at the graduate level, I believe.
But I want you to be ready to be able to take those classes, having the statistical fundamentals to understand what you're doing.
And then you're going to be able to expand to broader and more sophisticated methods.
Lectures are here from 11:00 to 12:30 on Tuesday and Thursday.
Victor-Emmanuel will also be-- and you can call him Victor-- will also be holding mandatory recitation.
So please go on Stellar and pick your recitation.
It's either 3:00 to 4:00 or 4:00 to 5:00 on Wednesdays.
And it's going to be mostly focused on problem-solving.
They're mandatory in the sense that we're allowed to do this, but they're not going to cover entirely new material.
But they might cover some techniques that might save you some time when it comes to the exam.
So you might get by.
Attendance is not going to be taken or anything like this.
But I highly recommend that you go, because, well, they're mandatory.
So you cannot really complain that something was taught only in recitation.
So please register on Stellar for which of the two recitations you would like to be in.
They're capped at 40, so first come, first served.
Homework will be due weekly.
There's a total of 11 problem sets.
I realize this is a lot.
Hopefully, we'll keep them light.
I just want you to not rush too much.
The 10 best will be kept, and this will count for a total of 30% of the final grade.
There are due Mondays at 8:00 PM on Stellar.
And this is a new thing.
We're not going to use the boxes outside of the math department.
We're going to use only PDF files.
Well, you're always welcome to type them and practice your LaTeX or Word typing.
I also understand that this can be a bit of a strain, so just write them down on a piece of paper, use your iPhone, and take a picture of it.
Dropbox has a nice, new-- so try to find something that puts a lot of contrast, especially if you use pencil, because we're going to check if they're readable.
And this is your responsibility to have a readable file.
I've had over the years-- not at MIT, I must admit-- but I've had students who actually write the doc file and think that converting it to a PDF consists in erasing the extension doc and replacing it by PDF.
This is not how it works.
So I'm sure you will figure it out.
Please try to keep them letter-sized.
This is not a strict requirement, but I don't want to see thumbnails, either.
You are allowed to have two late homeworks.
And by late, I mean 24 hours late.
No questions asked.
You submit them, this will be counted.
You don't have to send an email to warn us or anything like this.
Beyond that, even that you have one slack for one 0 grade and slack for two late homeworks, you're going to have to come up with a very good explanation why you need actually more extensions than that, if you ever do.
And particularly, you're going to have to keep track about why you've used your three options before.
There's going to be two midterms.
One is October 3, and one is November 7.
They're both going to be in class for the duration of the lecture.
When I say they last for an hour and 20 minutes, it does not mean that if you arrive 10 minutes before the end of lecture, you still get an hour and 20 minutes.
It will end at the end of lecture time.
For this as well, no pressure.
Only the best of the two will be kept.
And this grade will count for 30% of the grade.
This will be closed-books and closed-notes.
The purpose is for you to-- yes?
How many midterms did you say there are?
Two
You said the best of the two will be kept?
I said the best of the two will be kept, yes
So both the midterms will be kept?
The best of the two, not the best two
Oh
We will add them, multiply the number by 9, and that will be grade.
No.
I am trying to be nice, there's just a limit to what I can do.
All right, so the goal is for you to learn things and to be familiar with them.
In the final, you will be allowed to have your notes with you.
But the midterms are also a way for you to develop some mechanism so that you don't actually waste too much time on things that you should be able to do without thinking too much.
You will be allowed to cheat sheet, because, well, you can always forget something.
And it will be two-sided letters sheet, and you can practice yourself as writing as small as you want.
And you can put whatever you want on this cheat sheet.
All right, the final will be decided by the register.
It's going to be three hours, and it's going to count for 40%.
You cannot bring books, but you can bring your notes.
Yes
I noticed that the midterm dates aren't dated in the syllabus.
So I wanted to make sure you know
They are not?
Yeah--
Oh, yeah, there's a "1" that's missing on both of them, isn't there?
Yeah, let's figure that out.
The syllabus is the true one.
The slides are so that we can discuss, but the ones that's on the syllabus are the ones that count.
And I think they're also posted on the calendar on Stellar as well.
Any other question?
OK, so the pre-reqs here-- and who has looked at the first problem set already?
OK, so those hands that are raised realize that there is a true prerequisite of probability for this class.
It can be at the level of 18.600 or 604.1.
I should say "B" now.
It's two classes.
I will require you to know some calculus and have some notions of linear algebra, such as, what is a matrix, what is a vector, how do you multiply those things together, some notion of what orthonormal vectors are.
We'll talk about eigenvectors and eigenvalues, but I remind you all of that.
So this is not this strict pre-req.
But if you've taken it, for example, it doesn't hurt to go back to your notes when we get closer to this chapter on principle-component analysis.
The chapters, as they're listed in the syllabus, are in order, so you will see when it actually comes.
There's no required textbook.
And I know you tend to not like that.
You like to have your textbook to know where you're going and what we're doing.
I'm sorry, it's just this class.
Either I would have to go to a mathematical statistics textbook, which is just too much, or to go to a more engineering-type statistics class, which is just too little.
So hopefully, the problems will be enough for you to practice the recitations.
We'll have some problems to solve as well.
And the material will be posted on the slides.
So you should have everything you need.
There's plenty of resources online if you want to expand on a particular topic or read it as said by somebody else.
The book that I recommend in the syllabus is this book called All of Statistics by Wasserman.
Mainly because of the title, I'm guessing it has all of it in it.
It's pretty broad.
There's actually not that many.
It's more of an intro-grad level.
But it's not very deep, but you see a lot of the overview.
Certainly, what we're going to cover will be a subset of what's in there.
The slides will be posted on Stellar before lectures before we start a new chapter and after we're done with the chapter, with the annotations, and also, with the typos corrected, like for the exam.
There will be some video lectures.
Again, the first one will be posted on OCW from last year.
But all of them will be available on Stellar-- of course, module technical problems.
But this is an automated system.
And hopefully, it will work out well for us.
So if you somehow have to miss a lecture, you can always catch it up by watching it.
You can also play at that speed 0.75 in case I end up speaking too fast, but I think I've managed myself so far-- so just last warning.
All right, why should you study statistics?
Well, if you read the news, you will see a lot of statistics.
I mentioned machine learning.
It's built on a lot of statistics.
If I were to teach this class 10 years ago, I would have to explain to you that data collection and making decisions based on data was something that made sense.
But now, it's almost in our life.
We're used to this idea that data helps in making decisions.
And people use data to conduct studies.
So here, I found a bunch of press titles that-- I think the key word I was looking for was "study finds"-- if I want to do this.
So I actually did not bother doing it again this year.
This is all 2016, 2016, 2016.
But the key word that I look for is usually "study find"-- so a new study find-- traffic is bad for your health.
So we had to wait for 2016 for data to tell us that.
And there's a bunch of other slightly more interesting ones.
For example, one that you might find interesting is that this study finds that students benefit from waiting to declare a major.
Now, there's a bunch of press titles.
There one in the MIT News that finds brain connections, key to reading.
And so here, we have an idea of what happened there.
Some data was collected.
Some scientific hypothesis was formulated.
And then the data was here to try to prove or disprove this scientific hypothesis.
That's the usual scientific process.
And we need to understand how the scientific process goes, because some of those things might be actually questionable.
Who is 100% sure that study finds that students-- do you think that you benefit from waiting to declare a major?
Right I would be skeptical about this.
I would be like, I don't want to wait to declare a major.
So what kind of thing can we bring?
Well maybe this study studied people that were different from me.
Or maybe the study finds that this is beneficial for a majority of people.
I'm not a majority.
I'm just one person.
There's a bunch of things that we need to understand what those things actually mean.
And we'll see that those are actually not statements about individuals.
They're not even statements about the cohort of people they've actually looked at.
They're statements about a parameter of a distribution that was used to model the benefit of waiting.
So there's a lot of questions.
And there are a lot of layers that come into this.
And we're going to want to understand what was going on in there and try to peel it off and understand what assumptions have been put in there.
Even though it looks like a totally legit study, out of those studies, statistically, I think there's going to be one that's going to be wrong.
Well, maybe not one.
But if I put a long list of those, there would be a few that would actually be wrong.
If I put 20, there would definitely be one that's wrong.
So you have to see that.
Every time you see 20 studies, one is probably wrong.
When there are studies about drug effects, out of a list of 100, one would be wrong.
So we'll see what that means and what I mean by that.
Of course, not only studies that make discoveries are actually making the press titles.
There's also the press that talks about things that make no sense.
I love this first experiment-- the salmon experiment.
Actually, it was a grad student who came to a neuroscience poster session, pulled out this poster, and explained the scientific experiment that he was conducting, which consisted in taking a previously frozen and thawed salmon, putting it in an MRI, showing it pictures of violent images, and recording its brain activity.
And he was able to discover a few voxels that were activated by those violent images.
And can somebody tell me what happened here?
Was the salmon responding to the violent activity?
Basically, this is just a statistical fluke.
That's just randomness at play.
There's so many voxels that are recorded, and there's so many fluctuations.
There's always a little bit of noise when you're in those things, that some of them, just by chance, got lit up.
And so we need to understand how to correct for that.
In this particular instance, we need to have tools that tell us that, well, finding three voxels that are activated for that many voxels that you can find in the salmon's brain is just too small of a number.
Maybe we need to find a clump of 20 of them, for example.
All right, so we're going to have mathematical tools that help us find those particular numbers.
I don't know if you ever saw this one by John Oliver about phacking.
Or actually, it said p-hacking.
Basically, what John Oliver is saying is actually a full-length-- like there's long segments on this.
And he was explaining how there's a sociology question here about how there's a huge incentive for scientists to publish results.
You're not going to say, you know what?
This year, I found nothing.
And so people are trying to find things.
And just by searching, it's as if they were searching for all the voxels in a brain until they find one that was just lit up by chance.
And so they just run all these studies.
And at some point, one will be right just out of chance.
And so we have to be very careful about doing this.
There's much more complicated problems associated to what's called p-hacking, which consists of violating the basic assumptions, in particular, looking at the data, and then formulating your scientific assumption based on data, and then going back to it.
Your idea doesn't work.
Let's just formulate another one.
And if you are doing this, all bets are off.
The theory that we're going to develop is actually for a very clean use of data, which might be a little unpleasant.
If you've had an army of graduate students collecting genomic data for a year, for example, maybe you don't want to say, well, I had one hypothesis that didn't work.
Let's throw all the data into the trash.
And so we need to find ways to be able to do this.
And there's actually a course been taught at BU.
It's still in its early stages, but something called "adaptive data analysis" that will allow you to do these kind of things.
Questions?
OK, so of course, statistics is not just for you to be able to read the press.
Statistics will probably be used in whatever career path you choose for yourself.
It started in the 10th century in Netherlands for hydrology.
Netherlands is basically under water, under sea level.
And so they wanted to build some dikes.
But once you're going to build a dike, you want to make sure that it's going to sustain some tides and some floods.
And so in particular, they wanted to build dikes that were high enough, but not too high.
You could always say, well, I'm going to build a 500-meter dike, and then I'm going to be safe.
You want something that's based on data.
You want to make sure.
And so in particular, what did they do?
Well, they collected data for previous floods.
And then they just found a dike that was going to cover all these things.
Now, if you look at the data they probably had, maybe it was scarce.
Maybe they had 10 data points.
And so for those data points, then maybe they wanted to sort of interpolate between those points, maybe extrapolate for the larger one.
Based on what they've seen, maybe they have chances of seeing something which is even larger than everything they've seen before.
And that's exactly the goal of statistical modeling-- being able to extrapolate beyond the data that you have, guessing what you have not seen yet might happen.
When you buy insurance for your car, or your apartment, or your phone, there is a premium that you have to pay.
And this premium has been determined based on how much you are, in expectation, going to cost the insurance.
It says, OK, this person has, day a 10% chance of breaking their iPhone.
An iPhone costs that much to repair, so I'm going to charge them that much.
And then I'm going to add an extra dollar for my time.
That's basically how those things are determined.
And so this is using statistics.
This is basically where statistics is probably mostly used.
I was personally trained as an actuary.
And that's me being a statistician at an insurance company.
Clinical trials-- this is also one of the earliest success stories of statistics.
It's actually now widespread.
Every time a new drug is approved for market by the FDA, it requires a very strict regimen of testing with data, and control group, and treatment group, and how many people you need in there, and what kind of significance you need for those things.
In particular, those things look like this, so now it's 5,000 patients.
It depends on what kind of drug it is, but for, say, 100 patients, 56 were cured, and 44 showed no improvement.
Does the FDA consider that this is a good number?
Do they have a table for how many patients were cured?
Is there a placebo effect?
Do I need a control group of people that are actually getting a placebo?
It's not clear, all these things.
And so there's a lot of things to put into place.
And there's a lot of floating parameters.
So hopefully, we're going to be able to use statistical modeling to shrink it down to a small number of parameters to be able to ask very simple questions.
"Is a drug effective" is not a mathematical equation.
But "Is p larger than 0.5?"
is a mathematical question And that's essentially we're going to be doing.
We're going to take this, is a drug effective, to reducing to, is a variable larger than 0.5?
Now, of course genetics are using that.
That's typically actually the same size of data that you would see for FMRI data.
So this is actually a study that I found.
You have about 4,000 cases of Alzheimer's and 8,000 control.
So people without Alzheimer's-- that's what's called a control.
That's something just to make sure that you can see the difference with people that are not affected by either a drug or a disease.
Is the gene APOE associated with Alzheimer's disease?
Everybody can see why this would be an important question.
We now have it crisper.
It's targeted to very specific genes.
If we could edit it, or knock it down, or knock it up, or boost it, maybe we could actually have an impact on that.
So those are very important questions, because we have the technology to target those things.
But we need the answers about what those things are.
And there's a bunch of other questions.
The minute you're going to talk to biologists about say, I can do that.
They're going to say, OK, are there any other genes within the genes, or any particular snips that I can actually look at?
And they're looking at very different questions.
And when you start asking all these questions, you have to be careful, because you're reusing your data again.
And it might lead you to wrong conclusions.
And those are all over the place, those things.
And that's why they go all the way to John Oliver talking about them.
Any questions about those examples?
So this is really a motivation.
Again, we're not going to just take this data set of those cases and look at them in detail.
So what is common to all these examples?
Like, why do we have to use statistics for all those things?
Well, there's the randomness of the data.
There's some effect that we just don't understand-- for example, the randomness associated with the lining up of some voxels.
Or the fact that as far as the insurance is concerned whether you're going to break your iPhone or not is essentially a coin toss.
Fully, it's biased.
But it's a coin toss.
From the perspective of the statistician, those things are actually random events.
And we need to tame this randomness, to understand this randomness.
Is this going to be a lot of randomness?
Or is it going to be a little randomness?
Is it going to be something that's like, out of their people-- let's see, for example, for the floods.
Were the floods that I saw consistently almost the same size?
It was almost a rounding error, or they're just really widespread.
All these things, we need to understand so we can understand how to build those dikes or how to make decisions based on those data.
And we need to understand this randomness.
OK, so the associated questions to randomness were actually hidden in the text.
So we talked about the notion of average.
Right, so as far as the insurance is concerned, they want to know in average with the probability is.
Like, what is your chance of actually breaking your iPhone?
And that's what came in this notion of fair premium.
There's this notion of quantifying chance.
We don't want to talk maybe only about average, maybe you want to cover say 99% percent of the floods.
So we need to know what is the height of a flood that's higher than 99% of the floods.
But maybe there's 1% of them, you know.
When doomsday comes, doomsday comes.
Right, we're not going to pay for it.
All right, so that's most of the floods.
And then there's questions of significance, right?
So you know I give this example, a second ago about clinical trials.
I give you some numbers.
Clearly the drug cured more people than it did not.
But does it mean that it's significantly good, or was this just by chance.
Maybe it's just that these people just recovered.
It's like you know curing a common cold.
And you feel like, oh I got cured.
But it's really you waited five days and then you got cured.
All right, so there's this notion of significance, of variability.
All these things are actually notions that describe randomness and quantify randomness into simple things.
Randomness is a very complicated beast.
But we can summarize it into things that we understand.
Just like I am a complicated object.
I'm made of molecules, and made of genes, and made of very complicated things.
But I can be summarized as my name, my email address, my height and my weight, and maybe for most of you, this is basically enough.
You will recognize me without having to do a biopsy on me every time you see me.
All right, so, to understand randomness you have to go through probability.
Probability is the study of randomness.
That's what it is.
That's what the first sentence that a lecturer in probability will say.
And so that's why I need the pre-requisite, because this is what we're going to use to describe the randomness.
We'll see in a second how it interacts with statistics.
So sometimes, and actually probably most of the time throughout your semester on probability, randomness was very well understood.
When you saw a probability problem, here was the chance of this happening, here was the chance of that happening.
Maybe you had more complicated questions that you had some basic elements to answer.
For example, the probability that I have HBO is this much.
And the probability that I watch Game of Thrones is that much.
And given that I play basketball what is the probability-- you had all these crazy questions, but you were able to build them.
But all the basic numbers were given to you.
Statistics will be about finding those basic numbers.
All right so some examples that you've probably seen were dice, cards, roulette, flipping coins.
All of these things are things that you've seen in a probability class.
And the reason is because it's very easy to describe the probability of each outcome.
For a die we know that each face is going to come with probably 1/6.
Now I'm not going to go into a debate of whether this is pure randomness or this is determinism.
I think as a model for actual randomness a die is a pretty good number, flipping a coin is a pretty good model.
So those are actually a good thing.
So the questions that you would see, for example, in probabilities are the following.
I roll one die.
Alice gets $1 if the number of dots is less than three.
Bob gets $2 if the number of dots is less than two.
Do you want to be Alice or Bob given that your role is actually to make money.
Yeah, you want to be Bob, right?
So let's see why.
So if you look at the expectation of what Alice makes.
So let's call it a.
This is $1, with probability 1/2.
So 3/6, that's 1/2.
And the expectation of what Bob makes, this is $2 with probably 2/6 and that's 2/3.
Which is definitely larger than 1/2.
So Bob's expectations actually a bit higher.
So those are the kind of questions that you may ask with probability.
I described to you exactly, you use the fact that the die would get less than three dots, with probability one half.
We knew that.
And I didn't have to describe to you what was going on there.
You didn't have to collect data about a die.
Same thing, you roll two dice.
You choose a number between 2 and 12 and you win $100 if you choose the sum of the two dice.
Which number do you pick?
What?
7
7.
Why 7?
It's the most likely
That's the most likely one, right?
So your gain here will be $100 times the probability that the sum of the two dice, let's say x plus y, is equal to your little z where a little z is the number you pick.
So 7 is the most likely to happen and that's the one that maximizes this function of z.
And for this you need to study a more complicated function.
But it's a function that enables two die.
But you can compute the probability that x plus y is equal to z, for every z between 2 and 12.
So you know exactly what the probabilities are and that's how you start probability.
So here that's exactly what I said.
You have a very simple process that describes basic events.
Probability 1/6 for each of them.
And then you can build up on that, and understand probably of more complicated events.
You can throw some money in there.
You can be build functions.
You can do very complicated things building on that.
Now if I was a statistician, a statistician would be the guy who just arrived on earth, had never seen a die and needs to understand that a die come up with probably 1/6 on each side.
And the way he would do it is just to roll the die until he get some counts and tries to estimate those.
And maybe that guy would come and say, well, you know, actually, the probability that I get a 1 is 1/6 plus 0.001 and the probability that I get a 2 is 1/6 minus 0.005.
And there would be some fluctuations around this.
And it's going to be his role as a statistician to say, listen, this is too complicated of a model for this thing.
And these should all be the same numbers.
Just looking at data, they should be all the same numbers.
And that's part of the modeling.
You make some simplifying assumptions that essentially make your questions more accurate.
Now, of course, if your model is wrong, if it's not true that all the faces arrive with the same probability, then you have a model error here.
So we will be making model errors.
But that's going to be the price to pay to be able to extract anything from our data.
So for more complicated processes, so of course nobody's going to waste their time rolling dice.
I mean, I'm sure you might have done this in AP stat or something.
But the need is to estimate parameters from data.
All right, so for more complicated things you might want to estimate some density parameter on a particular set of material.
And for this maybe you need to beam something to it, and measure how fast it's coming back.
And you're going to have some measurement errors.
And maybe you need to do that several times and you have a model for the physical process that's actually going on.
And physics is usually a very good way to get models for engineering perspective.
But there's models for sociology where we have no physical system, right.
God knows how people interact.
And maybe I'm going to say that the way I make friends is by first flipping a coin in my pocket.
And with probability 2/3, I'm going to make my friend at work.
And with probability 1/3 I'm going to make my friend at soccer.
And once I make my friends at soccer-- I decide to make my friend soccer.
Then I will face someone who's flipping the same coin with maybe be slightly different parameters.
But those things actually exist.
There's models about how friendships are formed.
And the one I described is called the mixed-membership model.
So those are models that are sort of hypothesized.
And they're more reasonable than taking into account all the things that made you meet that person at that particular time.
So the goal here-- so based on data now, once we have the model is going to be reduced to maybe two, three, four parameters, depending on how complex the model is.
And then your goal will be to estimate those parameters.
So sometimes the randomness we have here is real.
So there's some true randomness in some surveys.
If I pick a random student, as long as I believe that my random number generator that will pick your random ID is actually random, there is something random about you.
The student that I pick at random will be a random student.
The person that I call on the phone is a random person.
So there's some randomness that I can build into my system by drawing something from a random number generator.
A biased coin is a random thing.
It's not a very interesting random thing.
But it is a random thing.
Again, if I wash out the fact that it actually is a deterministic mechanism.
But at a certain accuracy, a certain granularity, this can be thought of as a truly random experiment.
Measurement error for example, if you by some measurement device.
or some optics device, for example.
You will have like standard deviation and things that come on the side of the box.
And it tells you, this will be making some measurement error.
And it's usually thermal noise maybe, or things like this.
And those are very accurately described by some random phenomenon.
But sometimes, and I'd say most times, there's no randomness.
There's no randomness.
It's not like you breaking your iPhone is a random event.
This is just something that we sweep-- randomness is a big rug under which we sweep everything we don't understand.
And we just hope that in average we've captured, the average effect of what's going on.
And the rest of it might fluctuate to the right, might fluctuate to the left.
But what remains is just sort of randomness that can be averaged out.
So, of course, this is where the leap of faith is.
We do not know whether we were correct of doing this.
Maybe we make some huge systematic biases by doing this.
Maybe we forget a very important component.
Right, for example, if I have-- I don't know, let's think of something-- a drug for breast cancer.
All right, and I throw out the fact that my patient is either a man or woman.
I'm going to have some serious model biases.
Right.
So if I say I'm going to collect a random and patient.
And said I'm going to start doing this.
There's some information that I really need, clearly, to build into my model.
And so the model should be complicated enough, but not too complicated.
Right so it should take into account things there will systematically be important.
So, in particular, the simple rule of thumb is, when you have a complicated process, you can think of it as being a simple process and some random noise.
Now, again, the random noise is everything you don't understand about the complicated process.
And the simple process is everything you actually do.
So good modeling, and this is not where we'll be seeing in this class, consistent choosing plausible simple models.
And this requires a tremendous amount of domain knowledge.
And that's why we're not doing it in this class.
This is not something where I can make a blanket statement about making good modeling.
You need to know, if I were a statistician working on a study, I would have to grill the person in front of me, the expert, for two hours to know, but how about this?
How about that?
How does this work?
So it requires to understand a lot of things.
There's this famous statistician to whom this sentence is attributed, and it's probably not his then, but Tukey said that he loves being a statistician, because you get to play in everybody's backyard.
Right, so you get to go and see people.
And you get to understand, at least to a certain extent, what their problems are.
Enough that you can actually build a reasonable model for what they're actually doing.
So you get to do some sociology.
You get to do some biology.
You get to do some engineering.
And you get to do a lot of different things.
Right, so he was actually at some point predicting the presidential election.
So, you see, you get to do a lot of different things.
But it requires a lot of time to understand what problem you're working on.
And if you have a particular application in mind you're the best person to actually understand this.
So I'm just going to give you the basic tools.
So this is the circle of trust.
No, this is really just a simple graphic that tells you what's going on.
When you do probability, you're given the truth.
Somebody tells you what die God is rolling.
So you know exactly what the parameters of the problems are.
And what you're trying to do is to describe what the outcomes are going to be.
You can say, if you're rolling a fair die, you're going to have 1/6 of the time in your data you're going to have one.
1/6 of the time you're going to have to have two.
And so you can describe-- if I told you what the truth is, you could actually go into a computer, either generate some data.
Or you could describe to me some more macro properties of what the data would be like.
Oh, I would see a bunch of numbers that would be centered around 35, if I drew from a Gaussian distribution centered at 35.
Right, you would know this kind of thing.
I would know that it's very unlikely that if my Gaussian has standard deviation-- is centered on 0, say, with standard deviation 3.
It's very unlikely that I will see numbers below minus 10 in above 10, right?
You know this, that you basically will not see them.
So you know from the truth, from the distribution of a random variable that does not have mu or sigmas, really numbers there.
You know what data, you're going to be having.
Statistics is about going backwards.
It's saying, if I have some data, what was the truth that generated it.
And since there are so many possible truths, Modeling says you have to pick one of the simpler possible truths, so that you can average out.
Statistics basically means averaging.
You're averaging when you do statistics.
And averaging means that if I say that I received-- so if I collect all your GPAs, for example.
And my model is that the possible GPAs are any possible numbers.
And anybody can have any possible GPA.
This is going to be a serious problem.
But if I can summarize those GPAs into two numbers, say, mean and standard deviation, than I have a pretty good description of what is going on, rather than having to have to predict the full list.
Right, if I learn a full list of GPAs and I say, well this was the distribution.
Then it's not going to be of any use for me to predict what the GPA would be, or some random student walking in, or something like this.
So just to finish my rant about probability versus statistics, this is a question you would see in a probability-- this is a probabilistic question, and this is a statistical question.
The probabilistic question is, previous studies showed that the drug was 80% effective.
So you know that.
This is the effectiveness of the drug.
It's given to you.
This is how your problem starts.
Then we can anticipate that, for a study on 100 patients, in average, 80 be cured.
And at least 65 will be cured with 99% chances.
So again these are not-- I'm not predicting on 100 patients exactly the number of them they're going to be cured.
And the number of them that are not.
But I'm actually sort of predicting what things are going to look like on average, or some macro properties of what my data sets will look like.
So with 99 percent chances, that means that in 99.99% of the data sets you will draw from this particular draw.
99.99% of the cohort of 100 patients to whom you administer this drug, I will be able to conclude that at least 65 of them will be cured, on 99.99% percent of those data sets.
So that's a pretty accurate prediction of what's going to happen.
Statistics is the opposite.
It says, well, I just know that 78 out of 100 were cured.
I have only one data set.
I cannot make predictions for all data sets.
But I can go back to the probability, make some inference about what my probability will look like, and then say, OK, then I can make those predictions later on.
So when I start with 78/100 then maybe I'm actually, in this case, I just don't know.
My best guess here is that I'm confident I have to add the extra error that I bet you making by predicting that here, the drug is not 80% effective but 78% effective.
And they need some error bars around this, that will hopefully contain 80%, and then based on those error bars I'm going to make slightly less precise predictions for the future.
So, to conclude, so this was, why statistics?
So what is this course about?
It's about understanding the mathematics behind statistical methods.
It's more of a tool.
We're not going to have fun and talk about algebraic geometry just for fun in the middle of it.
So it justifies quantitative statements given some modeling assumptions, that we will, in this class, mostly admit that the modeling assumptions are correct.
| the first part-- in this introduction, we will go through them because it's very easy to forget what the assumptions are actually making.
But this will be a pretty standard thing.
The words you will hear a lot are IID-- independent and identically distributed-- that means that your data is basically all the sams.
And one data point is not impacting another data point.
Hopefully we can describe some interesting mathematics arising in statistics.
You know, if you've taken linear algebra, maybe we can explain to you why.
If you've done some calculus, maybe we can do some interesting calculus.
We'll see how in the spirit of applied math those things answer interesting questions.
And basically we'll try to carve out a math toolbox that's useful for us statistics.
And maybe you can extend it to more sophisticated methods that we did not cover in this class.
In particular in the immersion learning class, hopefully you'll be able to have some statistical intuition about what is going on.
So what this course is not about, it's not about spending a lot of time looking at data sets, and trying to understand some statistical thinking kind of questions.
So this is more of an applied statistical perspective on things, or more modeling.
So I'm going to typically give you the model.
And say this is a model.
And this is how we're going to build an estimator in the framework of this model.
So for example, 18.075, to a certain extent, is called "Statistical Thinking and Data Analysis."
So I'm hoping there is some statistical thinking in there.
We will not talk about software implementation.
Unfortunately, there's just too little time in a semester.
There's other courses that are giving you some overview.
So the main software these days are R is the leading software I'd say in statistics, both in academia and industry, lots of packages, one every day that's probably coming out.
But there's other things, right, so now Python is probably catching up with all these scikit-learn packages that are coming up.
Julia has some statistics in there, but it really if you were to learn a statistical software, let's say you love doing this, this would be the one that would prove most useful for you in the future.
It does not scale super well to high dimensional data.
So there is a class an IDSS that actually uses R. It's called IDS 0.12, I think it's called "Statistics, Computation, and Applications," or something like this.
I'm also preparing, with Peter Kempthorne, a course called "Computational Statistics."
It's going to be offered this Spring as a special topics.
And so Peter Kempthorne will be teaching it.
And this class will actually focus on using R. And even beyond that, it's not just going to be about using.
It's going to be about understanding-- just the same way we we're going to see how math helps you do statistics, it's going to help see how math helps you do algorithims for statistics.
All right, so we'll talk about maximum likelihood estimator.
Will need to maximize some function.
There's an optimization toolbox to do that.
And we'll see how we can have specialized for statistics for that, and what are the principles behind it.
And you know, of course, if you've taken AP stats you probably think that stats is boring to death because it was just a long laundry-list that spent a lot of time on t-test.
I'm pretty sure we're not going to talk about t-test, well, maybe once.
But this is not a matter of saying you're going to do this.
And this is a slight variant of it.
We're going to really try to understand what's going on.
So, admittedly, you have not chosen the simplest way to get an A in statistics on campus.
All right, this is not the easiest class.
It might be challenging at times, but I can promise you that you will maybe suffer.
But you will learn something by the time you're out of this class.
This will not be a waste of your time.
And you will be able to understand, and not having to remember by heart how those things actually work.
Are there any questions?
Anybody want to go to other stats class on campus?
Maybe it's not too late.
OK.
So let's do some statistics.
So I see the time now and it's 11:56, so we have another 30 minutes.
I will typically give you a three, four minute break if you want to stretch, if you want to run to the bathroom, if you want to check your texts or Instagram.
There was very little content in this class, hopefully it was entertaining enough that you don't need the break.
But just in the future, so you know you will have a break.
So statistics, this is how it starts, I'm French, what can I say I need to put some French words.
So this is not how office hours are going to go down.
Anybody know this sculpture by a Rodin, The Kiss.
Maybe probably The Thinker is more famous.
But this is actually a pretty famous one.
But is it really this one, or is it this one.
Anybody knows which one it is?
This one?
Or this one?
The previous
What's that?
This one
It's this one
Final answer
Yeah, who votes for this one.
OK. Who votes for that one?
Thank you.
I love that you do not want to pronounce yourself with no data actually to make any decision.
This is a total coin toss right.
Turns out that there is data, and there is in the very serious journal Nature, someone published a very serious paper which actually looks pretty serious.
If you look at it, it's like "Human Behavior: Adult persistence of head-turning symmetry," is a lot of fancy words in there.
And this, I'm not kidding you, this study is about collecting data of people kissing, and knowing if they bend their head to the right or if they bend they head to the left.
And that's all it is.
And so a neonatal right-side preference makes a surprising romantic reappearance in later life.
There's an explanation for it.
All right, so if we follow this Nature which one is the one.
This one?
Or this one?
This one, right?
Head to the right.
And to be fair, for this class I was like, oh, I'm going to go and show them what Google Images does.
When you Google kissing couple, it's inappropriate after maybe the first picture.
And so I cannot show you this.
But you know you can check for yourself.
Though I would argue, so this person here actually went out in airports and took pictures of strangers kissing and collecting data.
And can somebody guess why did he just not stay home and collect data from Google Images by just googling kissing couples.
What's wrong with this data?
I didn't know actually before I actually went on Google Images
It can be altered?
What was that?
It can be altered
It can be altered.
But, you know, who would want to do this?
I mean there's no particular reason why you would want to flip an image before putting it out there.
I mean, you might, but you know maybe they want to hide the brand of your Gap shirt or something
I guess the people who post pictures of themselves kissing on Google Images are not representative of the general population
Yeah, that's very true.
And actually it's even worse than that.
The people who post pictures of themselves, are not posting pictures of themselves or putting pictures of the people that they took a picture of.
And there usually is a stock watermark on this.
And it's basically stock images.
Those are actors, and so they've been directed to kiss and this is not a natural thing to do.
And actually, if you go to Google Images-- and I encourage you to do this, unless you don't want to see inappropriate pictures, and they're mightily inappropriate.
And basically you will see that this study is actually not working at all.
I mean, I looked briefly.
I didn't actually collect numbers.
But I didn't find a particular tendency to bend right.
If anything, it was actually probably the opposite.
And it's because those people were directed to do it.
They just don't actually think about doing it.
And also because I think you need to justify writing in your paper more than, I sat in front of my computer.
So again, this first sentence here, a neonatal right-side preference-- "is there a right side preference?"
is not a mathematical question.
But we can start saying, let's blah, and put some variables, and ask questions about those variables.
So you know x is actually not a variable that's used very much in statistics for parameters.
But p is one, for parameter.
And so you're going to take your parameter of interest, p, As here is going to be the proportion of couples.
And that's among all couples.
So here, if you talk about statistical thinking, there would be a question about what population this would actually be representative of.
| usually this is a call to your-- sorry, I should not forget this word it's important for you.
OK, I forget this word.
So this is-- OK, So if you look at this proportion, maybe these couples that are in the study might be representative only of couples in airports.
Maybe they actually put on a show for the other passengers.
Who knows?
You know, like, oh, let's just do it as well.
And just like the people in Google Images they are actually doing it.
So maybe you want to just restrict it.
But of course clearly if it's appearing in Nature, it should not be only about couples in airports.
It's supposedly representative of all couples in the world.
And so here let's just keep it vague, but you need to keep in mind what population this is actually making a statement about.
So you have this full population of people in the world.
Right, so those are all the couples.
And this person went ahead and collected data about a bunch of them.
And we know that, in this thing, there's basically a proportion of them, that's like p, and that's the proportion of them that's bending their head to the right.
And so everybody on this side is bending their heads right.
And hopefully we can actually sample this thing you're informing.
That's basically the process that's going on.
So this is the statistical experiment.
We're going to observe n kissing couples.
So here we're going to put as many variables as we can.
So we don't have to stick with numbers.
And then we'll just plug in the numbers.
n kissing couples, and n is also, in statistics, by the way, n is the size of your sample 99.9% of the time.
And collect the value of each outcome.
So we want numbers.
We don't want right or left.
So we're going to code them by 0 and 1, pretty naturally.
And then we're going to estimate p which is unknown.
So p is this area.
And we're going to estimate it simply by the proportion of right So the proportion of crosses that actually fell in the right side.
So in this study what you will find is that the numbers that were collected were 124 couples, and that, out of those 124, 80 of them turned their head to the right.
So, p hat is a proportion.
How do we do it?
Well, you don't need statistics for that.
You're going to see 80 divided by 124.
And you will find that in this particular study 64.5% of the couples were bending their heads to the right.
That's a pretty large number, right?
The question is if I picked another 124 couples, maybe at different airports, different times, would I see same number?
Would this number be all over the place?
Would it be sometimes very close to 120, or sometimes for close to 10?
Or would it be-- is this number actually fluctuating a lot.
And so, hopefully not too much, 64.5 percent is definitely much larger than 50%.
And so there seems to be this preference.
Now we're going to have to quantify how much of this preference.
Is this number significantly larger than 50%?
So if our data, for example, was just three couples.
I'm just going there, I'm going to Logan.
I call it, I do right, left right.
And then I see-- see what's the name of the fish place there?
I go to I go to Wahlburgers at Logan and I'm like, OK, I'm done for the day.
I collect this data.
I go home, and I'm like, wow, 66.7% to the right.
That's a pretty big number.
It's even farther from 50% than this other guy.
So I'm doing even better.
But of course you know that this is not true.
Three people is definitely not representative.
If I stopped at the first one, I would have actually-- at the first two, I would have even 100%.
So the question that statistics is going to help us answer is, how large should the sample be?
For some reason, I don't know if you guys receive this, I'm an affiliate with the Broad Institute, and since then I receive one email per day that says, sample size determination-- how large should your sample be?
Like, I know how large should with my sample be.
I've taken 18.650 multiple times.
And so I know, but the question is-- is 124 a large enough number or not?
Well, the answer is actually, as usual, it depends.
It will depend on the true unknown value of p. But from those particular values that we got, so 120 and-- how many couples was there?
80?
We actually can make some question.
So here we said that 80 was larger than 50-- was allowing us to conclude at 64.5%.
So it could be one reason to say that it was larger than 50%.
50% of 124 is 62.
So the question is, would I be would I be willing to make this conclusion at 63?
Is that a number that would convince you?
Who would be convinced by 63?
who would be convinced by 72?
Who would be convinced by 75?
Hopefully the number of hands that are raised should grow.
Who would be convinced by 80?
All right, so basically those numbers actually don't come from anywhere.
This 72 would be the number that you would need for a study-- most statistical studies would be the number that they would retain.
That's not for 124.
You would need to see 72 that turn their head right to actually make this conclusion.
And then 75-- So we'll see that there's many ways to come to this conclusion because, as you can see, this was published in Nature with 80.
So that was OK.
So 80 is actually a very large number.
This is 99 point-- this 99% -- no, so this is 95% confidence.
This is 99% confidence.
And this is 99.9% percent confidence.
So if you said 80 you're a very conservative person.
Starting at 72, you can start making this conclusion.
To understand this, we need to do our little mathematical kitchen here, and we need to do some modeling.
So we need to understand by modeling-- we need understand what random process we think this data is generating from.
So it's going to have some unknown parameters, unlike in probability.
But we need to have just basically everything written except for the values of the parameters.
When I said a die is coming uniformly with probably 1/6 then I need to have, say maybe with probability-- maybe I should say here are six numbers, and I need to just fill those numbers.
So for i equal 1 to n, I'm going to define Ri to be the indicator.
An indicator is just something that takes value 1 if something is true, and 0 if not.
So it's an indicator that i-th couple turns the head to the right.
So, Ri, so it's indexed by i.
And it's one if the i-th couple turns their head to the right, and 0 if it's-- well actually, I guess they can probably kiss straight, right?
So that would be weird, but they might be able to do this.
So let's say not right.
Then the estimator of p, we said, was p hat.
It was just the ratio of two numbers.
But really what it is is I count, I sum those Ri's.
Since I only add those that take value 1, what this is is-- this sum here is actually just counting the number of 1's.
Which is another way to say it's counting the number of couples that are kissing to the right.
And here I don't even have to tell you anything about the numbers or anything.
I can only keep track of-- first couple is a 0 second couple is a 1, third couple is 0.
The data set-- you can actually find it online-- is actually a sequence of 0's and 1's.
Now clearly for the question that we're asking about this proportion, I don't need to keep track of all this information.
All I need to keep track of is the number of 0's and the number of 1's.
Those are completely interchangeable.
There's no time effect in this.
The first couple is no different than the 15th couple.
So we call this Rn bar.
That's going to be a very standard notation that we use.
R might be replaced by other letters like x-- so xn bar, yn bar.
And this thing essentially means that I average the R's, or the Ri's over n of them.
And the bar means the average.
So I divide by n the total number of 1's.
So here this sum was equal to 80 in our example and n was equal to 124.
Now this is an estimator.
So an estimator is different from an estimate.
An estimate is a number.
My estimate was 64.5.
My estimator is this thing where I keep all the variables free.
And in particular, I keep those variables to be random because I'm going to think of a random couple kissing left to right as the outcome of a random process, just like flipping a coin be getting heads or tails.
And so this thing here is a random variable, Ri.
And this average is, of course, an average of random variables.
It's itself a random variable.
So an estimator is a random variable.
An estimate is the realization of a random variable, or, in other words, is the value that you get for this random variable once you plug in the numbers that you've collected.
So I can talk about the accuracy of an estimator.
Accuracy means what?
Well, what would we want for an estimator?
Maybe we won't want it to fluctuate too much.
It's a random variable.
So I'm talking about the accuracy of a random variable.
So maybe I don't want it to be too volatile.
I could have one estimator which would be-- just throw out 182 couples, keep only 2 and average those two numbers.
That's definitely a worse estimator than keeping all of the 124.
So I need to find a way to say that.
And what I'm going to be able to say is that the number is going to be fluctuating.
If I take another two couples, I'm going to be I'm probably going to get a completely different number.
But if I take another 124 couples two days later, maybe I'm going to have a very number that's very close to 64.5%.
So that's one way.
The other thing we would like about this estimator it's actually-- maybe it's not too volatile-- but also we want it to be close to the number that we're looking for.
Here is an estimator.
It's a beautiful variable.
72%, that's an estimator.
Go out there just do your favorite study about drug performance.
And then they're going to call you, MIT student taking statistics, they say, so how are you going to build your estimator?
We've collected those 5,000 or something like that.
I'm just going to spit out 72%.
Whatever the data says, that's an estimator.
It's a stupid estimator but it is an estimator.
But this is estimator is very not volatile.
Every time you're going to have a new study, even if you change fields, it's still going to be 72%.
This is beautiful.
And the problem is that's probably not very close to the value you're actually trying to estimate.
So we need two things.
We need are estimated to be a random variable.
So think in terms of densities.
We want the density to be pretty narrow.
We want this thing to have very little-- so this is definitely better than this.
But also, we want the number that we're interested in, p, to be very close to this-- to be close to the values that this thing is likely to take.
If p is here, this is not very good for us.
So that's basically the things we're going to be looking at.
The first one is referred to as variance.
The second one is referred to as bias.
Those things come all over in statistics.
So we need to understand a model.
So here's the model that we have for this particular problem.
So we need to make assumptions on the observations that we see.
So we said we're going to assume that the random variable-- that's not too much of a leap of faith.
We're just sweeping under the rug everything thing we don't understand about those couples.
And the assumption that we make is that Ri is a random variable.
This one you will forget very soon.
The second one is that each of the Ri's is-- so it's a random variable that takes value 0 or 1.
Anybody can suggest the distribution for this random variable?
Bernoulli
What?
Bernoulli
Bernoulli, right?
And it's actually beautiful.
This is where you have to do the least statistical modeling.
A random variable that takes value 0 or 1 is always a Bernoulli.
That's the simplest variable you can ever think of.
Any variable that takes only two possible values can be reduced to a Bernoulli.
OK, so this is a Bernoulli.
And here we make the assumption that it actually takes parameter p. And there's an assumption here.
Anybody can tell me what the assumption is?
It's the same
Yeah, it's same, right?
I could have said p i, but it's p. And that's where I'm going to be able to start getting to do some statistics.
It's that I'm going to start to be able to pull information across all my guys.
If I assume that they're all pi's completely uncoupled with each other.
Then I'm in trouble.
There's nothing I can actually get.
And then I'm going to assume that those guys are mutually independent.
And most of the time they will just say independent.
Meaning that, it's not like all these guys called each other and it's actually a flash mob.
And they were like, let's all turn our left side to the left.
And then this is definitely not going to give you a valid conclusion.
So, again.
randomness is a way of modeling lack of information.
Here there is a way to figure it out.
Maybe I could have followed all those guys, and knew exactly what they were-- maybe I could have looked at pictures of them in the womb and guess how they were turning-- by the way that's one of the conclusions, they're guessing that we turn our head to the right because our head is turned to the right in the womb.
So we don't know what goes on in the kissers minds.
And there's, you know, physics, sociology.
There's a lot of things that could help us, but it's just too complicated to keep track of, or too expensive for many instances Now again, the nicest part of this modeling was the fact that Ri's take only two values, which mean that this conclusion that they were Bernoulli was totally free for us.
Once we know it's a random variable, it's a Bernoulli.
Now they could have been, as we said, they could have been a Bernoulli with parameter p i.
For each i, I could have put a different parameter, but I just don't have enough information.
What would I have said?
I would say, well the first couple turned to the right.
p1 has to be one, that's my best guess.
The second couple kiss to the left, well, p2 should be 0, that's my best guess.
And so basically I need to have to be able to average my information.
And the way I do it is by coupling all these guys, pi's to be the same p for all i. OK, does it make sense?
Here what I am assuming is that my population is homogeneous.
Maybe it's not.
Maybe I could actually look at a finer grain, but I'm basically making a statement about a population.
And so maybe you kiss to the left, and then you're not-- I'm not making a statement about a person individually, I'm making a statement about the overall population.
Now independence is probably reasonable, right?
This person just went and know can seriously hope that these couples did not communicate with each other.
Or that you know Tanya did not text that we should all turn our head to the left now.
And there's no external stimulus that forces people to do something different.
OK, so-- sorry about that.
Since we have about less than 10 minutes.
Let's do a little bit of exercises, is that OK with you?
So I just have some exercises so we can see what an exercise going to look like.
This is sort of similar to the exercises you will see with me.
We should do it together, OK?
So now we're going to have-- I have a test.
So that's an exam in probability.
OK. And I'm going to have 15 students in this test.
And hopefully, this should be 15 grades that are representative of the grades of all a large class.
Right, so if you go you know 18.600, it's a large class, there's definitely more than 15 students.
And maybe, just by sampling 15 students at random, I want to have an idea of what my grade distribution will look like.
I'm grading them, I want to make an educated guess.
So I'm going to make some modeling assumptions for those guys.
So here, 15 students and the grades are x1 to x15.
Just like we had R1, R2, all the way to R124.
Those were my Ri's.
And so now I have my xi's.
And I'm going to assume that xi follows a Gaussian or normal distribution with min mu and variance sigma squared.
Now this is modeling, right?
Nobody told me there's no physical process that makes this happen.
We know that there's something called the central limit theorem in the background that says that things tend to be Gaussian, but this is really a matter of convenience.
Actually this is, if you think about it, this is terrible because this puts non-zero probability on negative scores.
I'm definitely not going to get a negative score.
But you know it's good enough because they know the probabilities non-zero but it's probably 10 to the minus 12.
So I would be very unlucky to see a negative score.
So here's the list of grades, so I have 65, 41, 70, 90, 58, 82, 76, 78-- maybe I should have done it with 8 --59, 59-- sitting next to each other --84, 89, 134, 51, and 72.
So those are the scores that I got.
There were clearly some bonus points over there.
And the question is, find estimator for mu.
What is my estimator for mu?
Well, an estimator, again, is something that depends on the random variable.
All right, so mu is the expectation, right?
So a good estimator is definitely the average score, just like we had the average of the Ri's.
Now the xi's no longer need to be 0's and 1's, so it's not going to boil down to being a number of ones divided by the total numbers.
Now if I'm looking for an estimate, well, I need to actually sum those numbers and divide them by 15.
So my estimate is going to be 1/15.
Then I'm going to start summing those numbers-- 65 plus 72.
OK, and I can do it, and it's 67.5.
This is my estimate.
Now if I want to compute a standard deviation-- so let's say estimate for sigma.
You've seen that before, right?
An estimate for sigma is what?
An estimate for sigma, we'll see methods to do this, but sigma squared is the variance, or is the expectation, of x minus expectation of x squared.
And the problem is that I don't know what those expectations are.
And so I'm going to do what 99.9% percent of statistics is.
And what is statistics about?
What's my motto?
Statistics is about replacing expectations with averages.
That's what all of statistics is about.
There's 300 pages in a purple book called All of Statistics that tells you this.
All right, and then you do something fancy.
Maybe you minimize something after you replace the expectation.
Maybe you need to plug in other stuff.
But really, every time you see an expectation, you replace it by an average.
OK let's do this.
So sigma squared hat will be what?
It's going to be 1 over n, sum from i equals 1 to n of xi minus-- well, here I need to replace my expectation by an average, which is really this average.
I'm going to call it mu hat squared.
There, you have replaced my expectation with average.
OK so the golden thing is, take your expectation and replace it with this.
Frame it, get a tattoo, I don't care but that's what it is.
If you remember one thing from this class, that's what it is.
Now you can be fancy, if you look at your calculator, it's going to put an n minus 1 here because it wants to be unbiased.
And those are things we are going to come to.
But let's say right now we stick to this.
And then when I plug in my numbers.
I'm going to get an estimate for sigma, which is the square root of the estimator once I plug in the numbers.
And you can check that the number, you get will be 18.
So those are basic things and if you've taken any AP stats this should be completely standard to you.
Now I have another list, and I don't have time to see it.
It doesn't really matter.
OK, we'll do that next time.
This is fine.
We'll see another list of numbers and see-- we're going to think about modeling assumptions.
The goal of this exercise is not to compute those things, it's really to think about modeling assumptions.
Is it reasonable to think that things are IID?
Is it reasonable to think that they have all the same parameters, that they're independent, et cetera, OK so one thing that I wanted to add is, probably by tonight, so I will try to use-- in the spirit of-- I don't know what's starting to happen.
In the spirit of using my iPad and fancy things, I will try to post some videos of-- for in particular, who has never used a statistical table to read, say, the quantiles of a Gaussian distribution?
OK, so there's several of you.
This is a simple but boring exercise.
I will just post a video on how to do this, and you will be able to find it on Stellar.
It's going to take five minutes, and then you will know everything there is to know about those things but that's something you need for the first problem set.
By the way, so the problem set has 30 exercises in probability.
You need to do 15.
And you only need to turn in 15.
You can turn in all of 30 if you want.
But you need to know, by the time we hit those things, you need to know-- well actually, by next week you need to know what's in there.
So if you don't have time to do all the homework, and then go back to your probability class to figure out how to do it, just do 15 easy that you can do.
And return those things.
But go back to your probability class and make sure that you know how to do all of them.
Those are pretty basic questions, and those are things that I'm not going to slow down on.
So you need to remember that the expectation of the product of independent random variables is a product of the expectations.
Expectation of the sum, is the sum of the expectation.
This kind of thing, which is a little silly, but it just requires you practice.
So, just have fun.
Those are simple exercises.
You will have fun remembering your probability class.
All right, so I'll see you on Tuesday-- or Monday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
--of our limiting distribution, which happen to be Gaussian.
But if the central limit theorem told us that the limiting distribution of some average was something that looked like a Poisson or an [?
exponential, ?]
then we would just have in the same way taken the quintiles of the exponential distribution.
So let's go back to what we had.
So generically if you have a set of observations X1 to Xn.
So remember for the kiss example they were denoted by R1 to Rn, because they were turning the head to the right, but let's just go back.
We say X1 to Xn, and in this case I'm going to assume they're IID, and I'm going to make them Bernoulli with [INAUDIBLE] p, and p is unknown, right?
So what did we do from here?
Well, we said p is the expectation of Xi, and actually we didn't even think about it too much.
We said, well, if I need to estimate the proportion of people who turn their head to the right when they kiss, I just basically I'm going to compute the average.
So our p hat was just Xn bar, which was just 1 over n sum from i over 1 2n of the Xi.
The average of the observations was their estimate.
And then we wanted to build some confidence intervals around this.
So what we wanted to understand is, how much that this p hat fluctuates.
This is a random variable.
It's an average of random variables.
It's a random variable, so we want to know what the distribution is.
And if we know what the distribution is, then we actually know, well, where it fluctuates.
What the expectation is.
Around which value it tends to fluctuate et cetera.
And so what the central limit theorem told us was if I take square root of n times Xn bar minus p, which is its average.
And then I divide it by the standard deviation.
Then this thing here converges as n goes to infinity, and we will say a little bit more about what it means in distribution to some standard normal random variable.
So that was the central limit theorem.
So what it means is that when I think of this as a random variable, when n is large enough it's going to look like this.
And so I understand perfectly its fluctuations.
I know that this thing here has-- I know the probability of being in this zone.
I know that this number here is 0.
I know a bunch of things.
And then, in particular, what I was interested in was that the probability, that's the absolute value of a Gaussian random variable, exceeds q alpha over 2, q alpha over 2.
We said that this was equal to what?
Anybody?
What was that?
[INAUDIBLE]
Alpha, right?
So that's the probability.
That's my random variable.
So this is by definition q alpha over 2 is the number.
So that to the right of it is alpha over 2.
And this is a negative q alpha over 2 by symmetry.
And so the probability that i exceeds-- well, it's not very symmetric, but the probability that i exceeds this value, q alpha over 2, is just the sum of the two gray areas.
All right?
So now I said that this thing was approximately equal, due to the central limit theorem, to the probability, that square root of n. Xn bar minus p divided by square root p 1 minus p. Well, absolute value was larger than q alpha over 2.
Well, then this thing by default is actually approximately equal to alpha, just because of virtue of the central limit theorem.
And then we just said, well, I'll solve for p. Has anyone attempted to solve the degree two equation for p in the homework?
Everybody has tried it?
So essentially, this is going to be an equation in p. Sometimes we don't want to solve it.
Some of the p's we will replace by their worst possible value.
For example, we said one of the tricks we had was that this value here, square root of p 1 minus p, was always less than one half.
Until we could actually get the confidence interval that was larger than all possible confidence intervals for all possible values of p, but we could solve for p. Do we all agree on the principle of what we did?
So that's how you build confidence intervals.
Now let's step back for a second, and see what was important in the building of this confidence interval.
The really key thing is that I didn't tell you why I formed this thing, right?
We started from x bar, and then I took some weird function of x bar that depended on p and n. And the reason is, because when I take this function, the central limit theorem tells me that it converges to something that I know.
But this very important thing about the something that I know is that it does not depend on anything that I don't know.
For example, if I forgot to divide by square root of p 1 minus p, then this thing would have had a variance, which is the p 1 minus p. If I didn't remove this p here, the mean here would have been affected by p. And there's no table for normal p 1.
Yes?
[INAUDIBLE]
Oh, so the square root of n terms come from.
So really you should view this.
So there's a rule and sort of a quiet rule in math that you don't write a divided by b over c, right?
You write c times a divided by b, because it looks nicer.
But the way you want to think about this is that this is x bar minus p divided by the square root of p 1 minus p divided by n. And the reason is, because this is actually the standard deviation of this-- oh sorry, x bar n. This is actually the standard deviation of this guy, and the square root of n comes from the [INAUDIBLE] average.
So the key thing was that this thing, this limiting distribution did not depend on anything I don't know.
And this is actually called a pivotal distribution.
It's pivotal.
I don't need anything.
I don't need to know anything, and I can read it in a table.
Sometimes there's going to be complicated things, but now we have computers.
The beauty about Gaussian is that people have studied them to death, and you can open any stats textbook, and you will see a table again that will tell you for each value of alpha you're interested in, it will tell you what q alpha over 2 is.
But there might be some crazy distributions, but as long as they don't depend on anything, we might actually be able to simulate from them, and in particular compute what q alpha over 2 is for any possible value [INAUDIBLE].. And so that's what we're going to be trying to do.
Finding pivotal distributions.
How do we take this Xn bar, which is a good estimate, and turn it into something which may be exactly or asymptotically does not depend on any unknown parameter.
So here is one way we can actually-- so that's what we did for the kiss example, right?
And here I mentioned, for example, in the extreme case, when n was equal to 3 we would get a different thing, but here the CLT would not be valid.
And what that means is that my pivotal distribution is actually not the normal distribution, but it might be something else.
And I said we can make take exact computations.
Well, let's see what it is, right?
If I have three observations, so I'm going to have X1, X2, X3.
So now I take the average of those guys.
OK, so that's my estimate.
How many values can this guy take?
It's a little bit of counting.
Four values.
How did you get to that number?
OK, so each of these guys can take value 0, 1, right?
So the number of values that it can take, I mean, it's a little annoying, because then I have to sum them, right?
So basically, I have to count the number of 1's.
So how many 1's can I get, right?
Sorry I have to-- yeah, so this is the number of 1's that I-- OK, so let's look at that.
So we get 0, 0, 0.
0, 0, 1.
And then I get basically three of them that have just the one in there, right?
So there's three of them.
How many of them have exactly two 1's?
2.
Sorry, 3, right?
So it's just this guy where I replaced the 0's and the 1.
OK, so now I get-- so here I get three that take the value 1, and one that gets the value 0.
And then I get three that take the value 2, and then one that takes the value 1.
The value [?
0 ?]
1's, right?
OK, so everybody knows what I'm missing here is just the ones here where I replaced the 0's by 1's.
So the number of values that this thing can take is 1, 2, 3, 4.
So someone is counting much faster than me.
And so those numbers, you've probably seen them before, right?
1, 3, 3, 1, remember?
And so essentially those guys, it takes only three values, which are either 1/3, 1.
Sorry, 1/3.
Oh OK, so it's 0, sorry.
1/3, 2/3, and 1.
Those are the four possible values you can take.
And so now-- which is probably much easier to count like that-- and so now all I have to tell you if I want to describe the distribution of this probability of this random variable, is just the probability that it takes each of these values.
So X bar 3 takes the value 0 probability that X bar 3 takes the value 1/3, et cetera.
If I give you each of these possible values, then you will be able to know exactly what the distribution is, and hopefully maybe to turn it into something you can compute.
Now the thing is that those values will actually depend on the unknown p. What is the unknown p here?
What is the probability that X bar 3 is equal to 0 for example?
I'm sorry?
[INAUDIBLE]
Yeah, OK.
So let's write it without making the computation So 1/8 is probably not the right answer, right?
For example, if p is equal to 0, what is this probability?
1.
If p is 1, what is this probability?
0.
So it will depend on p. So the probability that this thing is equal to 0, is just the probability that all three of those guys are equal to 0.
The probability that X1 is equal to 0, and X2 is equal to 0, and X3 is equal to 0.
Now my things are independent, so I do what I actually want to do, which say the probability of the intersection is the product of the probabilities, right?
So it's just the probability that each of them is equal to 0 to the power of 3.
And the probability that each of them, or say one of them is equal to 0, is just 1 minus p. And then for this guy I just get the probability-- well, it's more complicated, because I have to decide which one it is.
But those things are just the probability of some binomial random variables, right?
This is just a binomial, X bar 3.
So if I look at X bar 3, and then I multiply it by 3, it's just this sum of independent Bernoulli's with parameter p. So this is actually a binomial with parameter 3 and p. And there's tables for binomials, and they tell you all this.
Now the thing is I want to invert this guy, right?
Somehow.
This thing depends on p. I don't like it, so I'm going to have to find ways to get this things depending on p, and I could make all these nasty computations, and spend hours doing this.
But there's tricks to go around this.
There's upper bounds.
Just like we just said, well, maybe I don't want to solve the second degree equation in p, because it's just going to capture maybe smaller order terms, right?
Things that maybe won't make a huge difference numerically.
You can check that in your problem set one.
Does it make a huge difference numerically to solve the second degree equation, or to just use the [INAUDIBLE] p 1 minus p or even to plug in p hat instead of p. Those are going to be the-- problem set one is to make sure that you see what magnitude of changes you get by changing from one method to the other.
So what I wanted to go to is something where we can use something, which is just a little more brute force.
So the probability that-- so here is this Hoeffding's inequality.
We saw that.
That's what we've finished on last time.
So Hoeffding's inequality is actually one of the most useful inequalities.
If any one of you is doing anything really to algorithms, you've seen that inequality before.
It's extremely convenient that it tells you something about bounded random variables, and if you do algorithms typically with things bounded.
And that's the case of Bernoulli's random variables, right?
They're bounded between 0 and 1.
And so when I do this thing, when I do Hoeffding's inequality, what this thing is telling me is for any given epsilon here, for any given epsilon, what is the probability that Xn bar goes away from its expectation?
All right, then we saw that it decreases somewhat similarly to the way a Gaussian would look like.
So essentially what Hoeffding's inequality is telling me, is that I have this picture, when I have a Gaussian with mean u, I know it looks like this, right?
What Hoeffding's inequality is telling me is that if I actually take the average of some bounded random variables, then their probability distribution function or maybe math function-- this thing might not even have [INAUDIBLE] the density, but let's think of it as being a density just for simplicity-- it's going to be something that's going to look like this.
It's going to be somewhat-- well, sometimes it's going to have to escape just for the sake of having integral 1.
But it's essentially telling me that those guys stay below those guys.
The probability that Xn bar exceeds mu is bounded by something that decays like to tail of Gaussian.
So really that's the picture you should have in mind.
When I average bounded random variables, I actually have something that might be really rugged.
It might not be smooth like a Gaussian, but I know that it's always bounded by a Gaussian.
And what's nice about it is that when I actually start computing probability that exceeds some number, say alpha over 2, then I know that this I can actually get a number, which is just-- sorry, the probability that it exceeds, yeah.
So this number that I get here is actually going to be somewhat smaller, right?
So that's going to be the q alpha over 2 for the Gaussian, and that's going to be the-- I don't know, r alpha over 2 for this [?
Bernoulli ?]
random variable.
Like q prime or different q.
So I can actually do this without actually taking any limits, right?
This is valid for any n. I don't need to actually go to infinity.
Now this seems a bit magical, right?
I mean, I just said we need n to be, we discussed that we wanted n to be larger than 30 last time for the central limit theorem to kick in, and this one seems to tell me I can do it for any n. Now there will be a price to pay is that I pick up this 2 over b minus alpha squared.
So that's the variance of the Gaussian that I have, right?
Sort of.
That's telling me what the variance should be, and this is actually not as nice.
I pick factor 4 compared to the Gaussian that I would get for that.
So let's try to solve it for our case.
So I just told you try it.
Did anybody try to do it?
So we started from this last time, right?
And the reason was that we could say that the probability that this thing exceeds q alpha over 2 is alpha.
So that was using CLT, so let's just keep it here, and see what we would do differently.
What Hoeffding tells me is that the probability that Xn bar minus-- well, what is mu in this case?
It's p, right?
It's just notation here.
Mu was the average, but we call it p in the case of Bernoulli's, exceeds-- let's just call it epsilon for a second.
So we said that this was bounded by what?
So Hoeffding tells me that this is bounded by 2 times exponential minus 2.
Now the nice thing is that I pick up a factor n here, epsilon squared.
And what is b minus a squared for the Bernoulli's?
1.
So I don't have a denominator here.
And I'm going to do exactly what I did here.
I'm going to set this guy to be equal to alpha.
So that if I get alpha here, then that means that just solving for epsilon, I'm going to have some number, which will play the role of q alpha over 2, and then I'm going to be able to just say that p is between X bar and minus epsilon, and X bar n plus epsilon.
OK, so let's do it.
So we have to solve the equation.
2 exponential minus 2n epsilon squared equals alpha, which means that-- so here I'm going to get, there's a 2 right here.
So that means that I get alpha over 2 here.
Then I take the logs on both sides, and now let me just write it.
And then I want to solve for epsilon.
So that means that epsilon is equal to square root log q over alpha divided by 2n.
Yes?
[INAUDIBLE]
Why is b minus a 1?
Well, let's just look, right?
X lives in the interval b minus a.
So I can take b to be 25, and a to be my negative 42.
But I'm going to try to be as sharp as I can.
All right, so what is the smallest value you can think of such that a Bernoulli random variable is larger than or equal to this value?
What values does a Bernoulli random variable take?
0 and 1.
So it takes values between 0 and 1.
It just maxes the value.
Actually, this is the worst possible case for the Hoeffding inequality.
So now I just get this one, and so now you tell me that I have this thing.
So when I solve this guy over there.
So combining this thing and this thing implies that the probability that p lives between Xn bar minus square root log 2 over alpha divided by 2n and X bar plus the square root log 2 over alpha divided by 2n is equal to?
I mean, is at least.
What is it at least equal to?
Here this controls the probability of them outside of this interval, right?
It tells me the probability that Xn bar is far from p by more than epsilon.
So there's a probability that they're actually outside of the interval that I just wrote.
So it's 1 minus the probability of being in the interval.
So this is at least 1 minus alpha.
So I just use the fact that a probability of the complement is 1 minus the probability of the set.
And since I have an upper bound on the probability of the set, I have a lower bound on the probability of the complement.
So now it's a bit different.
Before, we actually wrote something that was-- so let me get it back.
So if we go back to the example where we took the [INAUDIBLE] over p, we got this guy.
q alpha over square root of-- over 2 square root n. So we had Xn bar plus minus q alpha over 2 square root n. Actually, that was q alpha over 2n, I'm sorry about that.
And so now we have something that replaces this q alpha, and it's essentially square root of 2 log 2 over alpha.
Because if I replace q alpha by square root of 2 log 2 over alpha, I actually get exactly this thing here.
And so the question is, what would you guess?
Is this number, this margin, square root of log 2 over alpha divided by 2n, is it smaller or larger than this guy?
q alpha all over 2/3n.
Yes?
Larger.
Everybody agrees with this?
Just qualitatively?
Right, because we just made a very conservative statement.
We do not use anything.
This is true always.
So it can only be better.
The reason in statistics where you use those assumptions that n is large enough, that you have this independence that you like so much, and so you can actually have the central limit theorem kick in, all these things are for you to have enough assumptions so that you can actually make sharper and sharper decisions.
More and more confident statement.
And that's why there's all this junk science out there, because people make too much assumptions for their own good.
They're saying, well, let's assume that everything is the way I love it, so that I can for sure any minor change, I will be able to say that's because I made an important scientific discovery rather than, well, that was just [INAUDIBLE] OK?
So now here's the fun moment.
And actually let me tell you why we look at this thing.
So there's actually-- who has seen different types of convergence in the probability statistic class?
[INAUDIBLE] students.
And so there's different types of-- in the real numbers there's very simple.
There's one convergence, Xn turns to X.
To start thinking about functions, well, maybe you have uniform convergence, you have pointwise convergence.
So if you've done some real analysis, you know there's different types of convergence you can think of.
And in the convergence of random variables, there's also different types, but for different reasons.
It's just because the question is, what do you do with the randomness?
When you see that something converges to something, it probably means that you're willing to tolerate low probability things happening or where it doesn't happen, and on how you handle those, creates different types of convergence.
So to be fair, in statistics the only convergence we care about is the convergence in distribution.
That's this one.
The one that comes from the central limit theorem.
That's actually the weakest possible you could make.
Which is good, because that means it's going to happen more often.
And so why do we need this thing?
Because the only thing we really need to do is to say that when I start computing probabilities on this random variable, they're going to look like probabilities on that random variable.
All right, so for example, think of the following two random variables, x and minus x.
So this is the same random variable, and this one is negative.
When I look at those two random variables, think of them as a sequence, a constant sequence.
These two constant sequences do not go to the same number, right?
One is plus-- one is x, the other one is minus x.
So unless x is the random variable always equal to 0, those two things are different.
However, when I compute probabilities on this guy, and when I compute probabilities on that guy, they're the same.
Because x and minus x have the same distribution just by symmetry of the gaps in random variables.
And so you can see this is very weak.
I'm not saying anything about the two random variables being close to each other every time I'm going to flip my coin, right?
Maybe I'm going to press my computer and say, what is x?
Well, it's 1.2.
Negative x is going to be negative 1.2.
Those things are far apart, and it doesn't matter, because in average those things are going to have the same probabilities that's happening.
And that's all we care about in statistics.
You need to realize that this is what's important, and why you need to know.
Because you have it really good.
If your problem is you really care more about convergence almost surely, which is probably the strongest you can think of.
So what we're going to do is talk about different types of convergence not to just reflect on the fact on how our life is good.
It's just that the problem is that when the convergence in distribution is so weak that it cannot do anything I want with it.
In particular, I cannot say that if X converges, Xn converges in distribution, and Yn converges in distribution, then Xn plus Yn converge in distribution to the sum of their limits.
I cannot do that.
It's just too weak.
Think of this example and it's preventing you to do quite a lot of things.
So this is converge in distribution to sum n 0, 1.
This is converge in distribution to sum n 0, 1.
But their sum is 0, and it's certainly not-- it doesn't look like the sum of two independent Gaussian random variables, right?
And so what we need is to have stronger conditions here and there, so that we can actually put things together.
And we're going to have more complicated formulas.
One of the formulas, for example, is if I replace p by p hat in this denominator.
We mentioned doing this at some point.
So I would need that p hat goes to p, but I need stronger than n distributions so that this happens.
I actually need this to happen in a stronger sense.
So here are the first two strongest sense in which random variables can converge.
The first one is almost surely.
And who has already seen this notation little omega when they're talking about random variables?
All right, so very few.
So this little omega is-- so what is a random variable?
A random variable is something that you measure on something that's random.
So the example I like to think of is, if you take a ball of snow, and put it in the sun for some time.
You come back.
It's going to have a random shape, right?
It's going to be a random blurb of something.
But there's still a bunch of things you can measure on it.
You can measure its volume.
You can measure its inner temperature.
You can measure its surface area.
All these things are random variables, but the ball itself is omega.
That's the thing on which you make your measurement.
And so a random variable is just a function of those omegas.
Now why do we make all these things fancy?
Because you cannot take any function.
This function has to be what's called measurable, and there's entire courses on measure theory, and not everything is measurable.
And so that's why you have to be a little careful why not everything is measurable, because you need some sort of nice property.
So that the measure of something, the union of two things, is less than the sum of the measures, things like that.
And so almost surely is telling you that for most of the balls, for most of the omegas, that's the right-hand side.
The probability of omega is such that those things converge to each other is actually equal to 1.
So it tells me that for almost all omegas, all the omegas, if I put them together, I get something that has probability of 1.
It might be that there are other ones that have probability 0, but what it's telling is that this thing happens for all possible realization of the underlying thing.
That's very strong.
It essentially says randomness does not matter, because it's happening always.
Now convergence in probability allows you to squeeze a little bit of probability under the rock.
It tells you I want the convergence to hold, but I'm willing to let go of some little epsilon.
So I'm willing to allow Tn to be less than epsilon.
Tn minus T to be-- sorry, to be larger than epsilon.
But the problem is they want this to go to 0 as well as n goes to infinity, but for each n this thing does not have to be 0, which is different from here, right?
So this probability here is fine.
So it's a little weaker, but it's a slightly different one.
I'm not going to ask you to learn and show that one is weaker than the other one.
But just know that these are two different types.
This one is actually much easier to check than this one.
Then there's something called convergence in Lp.
So this one is the fact that it embodies the following fact.
If I give you a random variable with mean 0, and I tell you that its variance is going to 0, right?
You have a sequence of random variables, their mean is 0, their expectation is 0, but their variance is going to 0.
So think of Gaussian random variables with mean 0, and a variance that shrinks to 0.
And this random variable converges to a spike at 0, so it converges to 0, right?
And so what I mean by that is that to have this convergence, all I had to tell you was that the variance was going to 0.
And so in L2 this is really what it's telling you.
It's telling you, well, if the variance is going to 0-- well, it's for any random variable T, so here what I describe was for a deterministic.
So Tn goes to a random variable T. If you look at the square-- the expectation of the square distance, and it goes to 0.
But you don't have to limit yourself to the square.
You can take power of three.
You can take power 67.6, power of 9 pi.
You take whatever power you want, it can be fractional.
It has to be lower than 1, and that's the convergence in Lp.
But we mostly care about integer p. And then here's our star, the convergence in distribution, and that's just the one that tells you that when I start computing probabilities on the Tn, they're going to look very close to the probabilities on the T. So that was our Tn with this guy, for example, and T was this standard Gaussian distribution.
Now here, this is not any probability.
This is just the probability then less than or equal to x.
But if you remember your probability class, if you can compute those probabilities, you can compute any probabilities just by subtracting and just building things together.
Well, I need this for all x's, so I want this for each x, So you fix x, and then you make the limit go to infinity.
You make n go to infinity, and I want this for the point x's at which the cumulative distribution function of T is continuous.
There might be jumps, and that I don't actually care for those.
All right, so here I mentioned it for random variables.
If you're interested, there's also random vectors.
A random vector is just a table of random variables.
You can talk about random matrices.
And you can talk about random whatever you want.
Every time you have an object that's just collecting real numbers, you can just plug random variables in there.
And so there's all these definitions that [?
extend.
?]
So where I see you see an absolute value, we'll see a norm.
Things like this.
So I'm sure this might look scary a little bit, but really what we are going to use is only the last one, which as you can see is just telling you that the probabilities converge to the probabilities.
But I'm going to need the other ones every once in a while.
And the reason is, well, OK, so here I'm actually going to the important characterizations of the convergence in distribution, which is R convergence style.
So i converge in distribution if and only if for any function that's continuous and bounded, when I look at the expectation of f of Tn, this converges to the expectation of f of T. OK, so this is just those two things are actually equivalent.
Sometimes it's easier to check one, easier to check the other, but in this class you won't have to prove that something converges in distribution other than just combining our existing convergence results.
And then the last one which is equivalent to the above two is, anybody knows what the name of this quantity is?
This expectation here?
What is it called?
The characteristic function, right?
And so this i is the complex i, and is the complex number.
And so it's essentially telling me that, well, rather than actually looking at all bounded and continuous but real functions, I can actually look at one specific family of complex functions, which are the functions that maps T to E to the ixT for x and R. That's a much smaller family of functions.
All possible continuous embedded functions has many more elements than just the real element.
And so now I can show that if I limit myself to do it, it's actually sufficient.
So those three things are used all over the literature just to show things.
In particular, if you're interested in deep digging a little more mathematically, the central limit theorem is going to be so important.
Maybe you want to read about how to prove it.
We're not going to prove it in this class.
There's probably at least five different ways of proving it, but the most canonical one, the one that you find in textbooks, is the one that actually uses the third element.
So you just look at the characteristic function of the square root of n Xn bar minus say mu, and you just expand the thing, and this is what you get.
And you will see that in the end, you will get the characteristic function of a Gaussian.
Why a Gaussian?
Why does it kick in?
Well, because what is the characteristic function of a Gaussian?
Does anybody remember the characteristic function of a standard Gaussian?
[INAUDIBLE]
Yeah, well, I mean there's two pi's and stuff that goes away, right?
A Gaussian is a random variable.
A characteristic function is a function, and so it's not really itself.
It looks like itself.
Anybody knows what the actual formula is?
Yeah
[INAUDIBLE]
E to the minus?
E to the minus x squared over 2
Exactly.
E to the minus x squared over 2.
But this x squared over 2 is actually just the second order expansion in the Taylor expansion.
And that's why the Gaussian is so important.
It's just the second order Taylor expansion.
And so you can check it out.
I think Terry Tao has some stuff on his blog, and there's a bunch of different proofs.
But if you want to prove convergence in distribution, you very likely are going to use one this three right here.
So let's move on.
This is when I said that this convergence is weaker than that convergence.
This is what I meant.
If you have convergence in one style, it implies convergence in the other stuff.
So the first [INAUDIBLE] is that if Tn converges almost surely, this a dot s dot means almost surely, then it also converges in probability and actually the two limits, which are this random variable T, are equal almost surely.
Basically what it means is that whatever you measure one is going to be the same that you measure on the other one.
So that's very strong.
So that means that convergence almost surely is stronger than convergence in probability.
If you're converge in Lp then you also converge in Lq for sum q less than p. So if you converge in L2, you'll also converge in L1.
If you converge in L67, you converge in L2.
If you're converge in L infinity, you converge in Lp for anything.
And so, again, limits are equal.
And then when you converge in distribution, when you converge in probability, you also converge in distribution.
OK, so almost surely implies probability.
Lp implies probability.
Probability implies distribution.
And here note that I did not write, and the limits are equal almost surely.
Why?
Because the convergence in distribution is actually not telling you that your random variable is converging to another random variable.
It's telling you that the distribution of your random variable is converging to a distribution.
And think of this, guys.
x and minus x.
The central limit theorem tells me that I'm converging to some standard Gaussian distribution, but am I converging to x or am I converging to minus x?
It's not well identified.
It's any random variable that has this distribution.
So there's no way the limits are equal.
Their distributions are going to be the same, but they're not the same limit.
Is that clear for everyone?
So in a way, convergence in distribution is really not a convergence of a random variable towards another random variable.
It's just telling you the limiting distribution of your random variable [INAUDIBLE] which is enough for us.
And one thing that's actually really nice is this continuous mapping theorem, which essentially tells you that-- so this is one of the theorems that we like, because they tell us you can do what you feel like you want to do.
So if I have Tn that goes to T, f of Tn goes to f of T, and this is true for any of those convergence except for Lp.
But they have to have f, which is continuous, otherwise weird stuff can happen.
So this is going to be convenient, because here I don't have X to n minus p. I have a continuous function.
It's between a linear function of Xn minus p, but I could think of like even crazier stuff to do, and it would still be true.
If I took the square, it would converge to something that looks like its distribution.
It's the same as the distribution of a square Gaussian.
So this is a mouthful, these two slides-- actually this particular slide is a mouthful.
What I have in my head since I was pretty much where you're sitting, is this diagram.
So what it tells me-- so it's actually voluntarily cropped, so you can start from any Lq you want large.
And then as you decrease the index, you are actually implying, implying, implying until you imply convergence in probability.
Convergence almost surely implies convergence in probability, and everything goes to the [?
sync, ?]
that is convergence in distribution.
So everything implies convergence in distribution.
So that's basically rather than remembering those formulas, this is really the diagram you want to remember.
All right, so why do we bother learning about those things.
That's because of this limits and operations.
Operations and limits.
If I have a sequence of real numbers, and I know that Xn converges to X and Yn converges to Y, then I can start doing all my manipulations and things are happy.
I can add stuff.
I can multiply stuff.
But it's not true always for convergence in distribution.
But it is, what's nice, it's actually true for convergence almost surely.
Convergence almost surely everything is true.
It's just impossible to make it fail.
But convergence in probability is not always everything, but at least you can actually add stuff and multiply stuff.
And it will still give you the sum of the n, and the product of the n. You can even take the ratio if V is not 0 of course.
If the limit is not 0, then actually you need Vn to be not 0 as well.
You can actually prove this last statement, right?
Because it's a combination of the first statement of the second one, and the continuous mapping theorem.
Because the function that maps x to 1 over x on everything but 0, is continuous.
And so 1 over Vn converges to 1 over V, and then I can multiply those two things.
So you actually knew that one.
But really this is not what matters, because this is something that you will do whatever happens.
If I don't tell you you cannot do it, well, you will do it.
But in general those things don't apply to convergence in distribution unless the pair itself is known to converge in distribution.
Remember when I said that these things apply to vectors, then you need to actually say that the vector converges in distributions to the limiting factor.
Now this tells you in particular, since the cumulative distribution function is not defined for vectors, I would have to actually use one of the other distributions, one of the other criteria, which is convergence of characteristic functions or convergence of a function of bounded continuous function of the random variable.
0.2 or 0.3, but 0.1 is not going get you anywhere.
But this is something that's going to be too hard for us to deal with, so we're actually going to rely on the fact that we have something that's even better.
There's something that is waiting for us at the end of his lecture, which is called Slutsky's that says that if V, in this case, converges in probability but U converge in distribution, I can actually still do that.
I actually don't need both of them to converge in probability.
I actually need only one of them to converge in probability to make this statement.
But two sum.
So let's go to another example.
So I just want to make sure that we keep on doing statistics.
And every time we're going to just do a little bit too much probability, I'm going to reset the pressure, and start doing statistics again.
All right, so assume you observe the times the inter-arrival time of the T at Kendall.
So this is not the arrival time.
It's not like 7:56, 8:15.
No, it's really the inter-arrival time, right?
So say the next T is arriving in six minutes.
So let's say [INAUDIBLE] bound.
And so you have this inter-arrival time.
So those are numbers say, 3, 4, 5, 4, 3, et cetera.
So I have this sequence of numbers.
So I'm going to observe this, and I'm going to try to infer what is the rate of T's going out of the station from this.
So I'm going to assume that these things are mutually independent.
That's probably not completely true.
Again, it just means that what it would mean is that two consecutive inter-arrival times are independent.
I mean, you can make it independent if you want, but again, this independent assumption is for us to be happy and safe.
Unless someone comes with overwhelming proof that it's not independent and far from being independent, then yes, you have a problem.
But it might be the fact that it's actually-- if you have a T that's one hour late.
If an inter-arrival time is one hour, then the other T, either they fixed it, and it's going to be just 30 seconds behind, or they haven't fixed it, then it's going to be another hour behind.
So they're not exactly independent, but they are when things work well and approximate.
And so now I need to model a random variable that's positive, maybe not upper bounded.
I mean, people complain enough that this thing can be really large.
And so one thing that people like for inter-arrival times is exponential distribution.
So that's a positive random variable.
Looks like an exponential on the right-hand slide, on the positive line.
And so it decays very fast towards 0.
The probability that you have very large values exponentially small, and there's a [INAUDIBLE] lambda that controls how exponential is defined.
It's exponential minus lambda times something.
And so we're going to assume that they have the same distribution, the same random variable.
So they're IID, because they are independent, and they're identically distributed.
They all have this exponential with parameter lambda, and I'm going to try to learn something about lambda.
What is the estimated value of lambda, and can I build a confidence interval for lambda.
So we observe n arrival times.
So as I said, the mutual independence is plausible, but not completely justified.
The fact that they're exponential is actually something that people like in all this what's called queuing theory.
So exponentials arise a lot when you talk about inter-arrival times.
It's not about the bus, but where it's very important is call centers, service, servers where tasks come, and people want to know how long it's going to take to serve a task.
So when I call at a center, nobody knows how long I'm going to stay on the phone with this person.
But it turns out that empirically exponential distributions have been very good at modeling this.
And what it means is that they're actually-- you have this memoryless property.
It's kind of crazy if you think about it.
What does that thing say?
Let's parse it.
That's the probability.
So this is condition on the fact that T1 is larger than T. So T1 is just say the first arrival time.
That means that conditionally on the fact that I've been waiting for the first T, well, the first [INAUDIBLE].
Well, I should probably-- the first subway for more than T conditionally-- so I've been there T minutes already.
Then the probability that I wait for s more minutes.
So that's the probability that T1 is learned, and the time that we've already waited plus x.
Given that I've been waiting for T minutes, really I wait for s more minutes, is actually the probability that I wait for s minutes total.
It's completely memoryless.
It doesn't remember how long have you been waiting.
The probability does not change.
You can have waited for two hours, the probability that it takes another 10 minutes is going to be the same as if you had been waiting for zero minutes.
And that's something that's actually part of your problem set.
Very easy to compute.
This is just an analytical property.
And you just manipulate functions, and you see that this thing just happen to be true, and that's something that people like.
Because that's also something that benefit.
And also what we like is that this thing is positive almost surely, which is good when you model arrival times.
To be fair, we're not going to be that careful.
Because sometimes we are just going to assume that something follows a normal distribution.
And in particular, I mean, I don't know if we're going to go into that details, but a good thing that you can model with a Gaussian distribution are heights of students.
But technically with positive probability, you can have a negative Gaussian random variable, right?
And the probability being it's probably 10 to the minus 25, but it's positive.
But it's good enough for us for our modeling.
So this thing is nice, but this is not going to be required.
When you're modeling positive random variables, you don't always have to use positive distributions that are supported on positive numbers.
You can use distributions like Gaussian.
So now this exponential distribution of T1, Tn they have the same parameter, and that means that in average they have the same inter-arrival time.
So this lambda is actually the expectation.
And what I'm just saying is that they're identically distributed means that I mean some sort of a stationary regime, and it's not always true.
I have to look at a shorter period of time, because at rush hour and 11:00 PM clearly those average inter-arrival times are going to be different So it means that I am really focusing maybe on rush hour.
Sorry, I said it's lambda.
It's actually 1 over lambda.
I always mix the two.
All right, so you have the density of T1.
So f of T is this.
So it's on the positive real line.
The fact that I have strictly positive or larger [INAUDIBLE] to 0 doesn't make any difference.
So this is the density.
So it's lambda E to the minus lambda T. The lambda in front just ensures that when I integrate this function between 0 and infinity, I get 1.
And you can see, it decays like exponential minus lambda T. So if I were to draw it, it would just look like this.
So at 0, what value does it take?
Lambda.
And then I decay like exponential minus lambda T. So this is 0, and this is f of T. So very small probability of being very large.
Of course, it depends on lambda.
Now the expectation, you can compute the expectation of this thing, right?
So you integrate T times f of T. This is part of the little sheet that I gave you last time.
This is one of the things you should be able to do blindfolded.
And then you get the expectation of T1 is 1 over lambda.
That's what comes out.
So as I actually tell many of my students, 99% of statistics is replacing expectations by averages.
And so what you're tempted to do is say, well, if in average I'm supposed to see 1 over lambda, I have 15 observations.
I'm just going to average those observations, and I'm going to see something that should be close to 1 over lambda.
So statistics is about replacing averages, expectations with averages, and that's we do.
So Tn bar here, which is the average of the Ti's, is a pretty good estimator for 1 over lambda.
So if I want an estimate for lambda, then I need to take 1 over Tn bar.
So here is one estimator.
I did it without much principle except that I just want to replace expectations by averages, and then I fixed the problem that I was actually estimating 1 over lambda by lambda.
But you could come up with other estimators, right?
But let's say this is my way of getting to that estimator.
Just like I didn't give you any principled way of getting p hat, which is Xn bar in the kiss example.
But that's the natural way to do it.
Everybody is completely shocked by this approach?
All right, so let's do this.
So what can I say about the properties of this estimator lambda hat?
Well, I know that Tn bar is going to 1 over lambda by the law of large number.
It's an average.
It converges to the expectation both almost surely, and in probability.
So the first one is the strong law of large number, the second one is the weak law of large number.
I can apply the strong one.
I have enough conditions.
And hence, what do I apply so that 1 over Tn bar actually goes to lambda?
So I said hence.
What is hence?
What is it based on?
[INAUDIBLE] PHILIPPE RIGOLLET Yeah, continuous mapping theorem, right?
So I have this function 1 over x. I just apply this function.
So if it was 1 over lambda squared, I would have the same thing that would happen just because the function 1 over x is continuous away from 0.
And now the central limit theorem is also telling me something about lambda.
About Tn bar, right?
It's telling me that if I look at my average, I remove the expectation here.
So if I do Tn bar minus my expectation, rescale by this guy here, then this thing is going to converge to some Gaussian random variable, but here I have this lambda to the negative 1-- to the negative 2 here, and that's because they did not tell you that if you compute the variance-- so from this, you can probably extract.
So if I have X that follows some exponential distribution with parameter lambda.
Well, let's call it T. So we know that T in expectation, the expectation of T is 1 over lambda.
What is the variance of T?
You should be able to read it from the thing here.
1 over lambda squared.
That's what you actually read in the variance, because the central limit theorem is really telling you the distribution goes through this n. But this numbers and this number you can read, right?
If you look at the expectation of this guy it's-- of this guy comes out.
This is 1 over lambda minus 1 over lambda.
That's why you read the 0.
And if you look at the variance of the dot, you get n times the variance of this average.
Variance of the average is picking up a factor 1 over n. So the n cancels.
And then I'm left with only one of the variances, which is 1 over lambda squared.
OK, so we're not going to do that in details, because, again, this is just a pure calculus exercise.
But this is if you compute integral of lambda e to the minus t lambda times t squared.
Actually t minus 1 over lambda squared dt between 0 and infinity.
You will see that this thing is 1 over lambda squared.
How would I do this?
Configuration by [INAUDIBLE] or you know it.
All right.
So this is what the central limit theorem tells me.
So this gives me if I solve this, and I plug in so I can multiply by lambda and solve, it would give me somewhat a confidence interval for 1 over lambda.
If we just think of 1 over lambda as being the p that I had before, this would give me a central limit theorem for-- sorry, a confidence interval for 1 over lambda.
So I'm hiding a little bit under the rug the fact that I have to still define it.
Let's just actually go through this.
I see some of you are uncomfortable with this, so let's just do it.
So what we've just proved by the central limit theorem is that the probability, that's square root of n Tn minus 1 over lambda exceeds q alpha over 2 is approximately equal to alpha, right?
That's just the statement of the central limit theorem, and by approximately equal I mean as n goes to infinity.
Sorry I did not write it correctly.
I still have to divide by square root of 1 over lambda squared, which is the standard deviation, right?
And we said that this is a bit ugly.
So let's just do it the way it should be.
So multiply all these things by lambda.
So that means now that the absolute value, so with probability 1 minus alpha asymptotically, I have that square root of n times lambda Tn minus 1 is less than or equal to q alpha over 2.
So what it means is that, oh, I have negative q alpha over 2 less than square root of n. Let me divide by square root of n here.
lambda Tn minus 1 q alpha over 2.
And so now what I have is that I get that lambda is between-- that's Tn bar-- is between 1 plus q alpha over 2 divided by root n. And the whole thing is divided by Tn bar, and same thing on the other side except I have 1 minus q alpha over 2 divided by root n divided by Tn bar.
So it's kind of a weird shape, but it's still of the form 1 over Tn bar plus or minus something.
But this something depends on Tn bar itself.
And that's actually normal, because Tn bar is not only giving me information about the mean, but it's also giving me information about the variance.
So it should definitely come in the size of my error bars.
And that's the way it comes in this fairly natural way.
Everybody agrees?
So now I have actually built a confidence interval.
But what I want to show you with this example is, can I translate this in a central limit theorem for something that converges to lambda, right?
I know that Tn bar converges to 1 over lambda, but I also know that 1 over Tn bar converges to lambda.
So do I have a central limit theorem for 1 over Tn bar?
Technically no, right?
Central limit theorems are about averages, and 1 over an average is not an average.
But there's something that statisticians like a lot, and it's called the Delta method.
The Delta method is really something that's telling you that you can actually take a function of an average, and let it go to the function of the limit, and you still have a central limit theorem.
And the factor or the price to pay for this is something which depends on the derivative of the function.
And so let's just go through this, and it's, again, just like the proof of the central limit theorem.
And actually in many of those asymptotic statistics results, this is actually just a Taylor expansion, and here it's not even the second order, it's actually the first order, all right?
So I'm just going to do linear approximation of this function.
So let's do it.
So I have that g of Tn bar-- actually let's use the notation of this slide, which is Zn and theta.
So what I know is that Zn minus theta square root of n goes to some Gaussian, this standard Gaussian.
No, not standard.
OK, so that's the assumptions.
And what I want to show is some convergence of g of Zn to g of theta.
So I'm not going to multiply by root n just yet.
So I'm going to do a first order Taylor expansion.
So what it is telling me is that this is equal to Zn minus theta times g prime of, let's call it theta bar where theta bar is somewhere between say Zn and theta, for sum.
OK, so if theta is less than Zn you just permute those two.
So that's what the Taylor first order Taylor expansion tells me.
There exists a theta bar that's between the two values at which I'm expanding so that those two things are equal.
Is everybody shocked?
No?
So that's standard Taylor expansion.
Now I'm going to multiply by root n. And so that's going to be what?
That's going to be root n Zn minus theta.
Ah-ha, that's something I like.
Times g prime of theta bar.
Now the central limit theorem tells me that this goes to what?
Well, this goes to sum n 0 sigma squared, right?
That was the first line over there.
This guy here, well, it's not clear, right?
Actually it is.
Let's start with this guy.
What does theta bar go to?
Well, I know that Zn is going to theta.
Just because, well, that's my law of large numbers.
Zn is going to theta, which means that theta bar is sandwiched between two values that converge to theta.
So that means that theta bar converges to theta itself as n goes to infinity.
That's just the law of large numbers.
Everybody agrees?
Just because it's sandwiched, right?
So I have Zn.
I have theta, and theta bar is somewhere here.
The picture might be reversed.
It might be that Zn end is larger than theta.
But the law of large number tells me that this guy is not moving, but this guy is moving that way.
So you know when n is [INAUDIBLE],, there's very little wiggle room for theta bar, and it can only get to theta.
And I call it the sandwich theorem, or just find your favorite food in there.
So this guy goes to theta, and now I need to make an extra assumption, which is that g prime is continuous.
And if g prime is continuous, then g prime of theta bar goes to g prime of theta.
So this thing goes to g prime of theta.
But I have an issue here.
Is that now I have something that converges in distribution and something that converges in say-- I mean, this converges almost surely or saying probability just to be safe.
And this one converges in distribution.
And I want to combine them.
But I don't have a slide that tells me I'm allowed to take the product of something that converges in distribution, and something that converges in probability.
This does not exist.
Actually, if anything it told me, do not do anything with things that converge in distribution.
And so that gets us to our-- OK, so I'll come back to this in a second.
And that gets us to something called Slutsky's theorem.
And Slutsky's theorem tells us that in very specific cases, you can do just that.
So you have two sequences of random variables, Xn bar, that's Xn that converges to X.
And Yn that converges to Y, but Y is not anything.
Y is not any random variable.
So X converges in this distribution.
Sorry, I forgot to mention, this is very important.
Xn converges in distribution, Y converges in probability.
And we know that in generality we cannot combine those two things, but Slutsky tells us that if the limit of Y is a constant, meaning it's not a random variable, but it's a deterministic number 2, just a fixed number that's not a random variable, then you can combine them.
Then you can sum them, and then you can multiply them.
I mean, actually you can do whatever combination you want, because it actually implies that X, the vector Xn, Yn converges to the vector Xc.
OK, so here I just took two combinations.
They are very convenient for us, the sum and the product so I could do other stuff like the ratio if c is not 0, things like that.
So that's what Slutsky does for us.
So what you're going to have to write a lot in your homework, in your mid-terms, by Slutsky.
I know some people are very generous with their by Slutsky.
They just do numerical applications, mu is equal to 6, and therefore by Slutsky mu square is equal to 36.
All right, so don't do that.
Just use, write Slutsky when you're actually using Slutsky.
But this is something that's very important for us, and it turns out that you're going to feel like you can write by Slutsky all the time, because that's going to work for us all the time.
Everything we're going to see is actually going to be where we're going to have to combine stuff.
Since we only rely on convergence from distribution arising from the central limit theorem, we're actually going to have to rely on something that allows us to combine them, and the only thing we know is Slutsky.
So we better hope that this thing works.
So why Slutsky works for us.
Can somebody tell me why Slutsky works to combine those two guys?
So this one is converging in distribution.
This one is converging in probability, but to a deterministic number.
g prime of theta is a deterministic number.
I don't know what theta is, but it's certainly deterministic.
All right, so I can combine them, multiply them.
So that's just the second line of that in particular.
All right, everybody is with me?
So now I'm allowed to do this.
You can actually-- you will see something like counterexample questions in your problem set just so that you can convince yourself.
It's always a good thing.
I don't like to give them, because I think it's much better for you to actually come to the counterexample yourself.
Like what can go wrong if Y is not a random-- sorry, if Y is not a-- sorry, if c is not the constant, but it's a random variable.
You can figure that out.
All right, so let's go back.
So we have now this Delta method that tells us that now I have a central limit theorem for functions of averages, and not just for averages.
So the only price to pay is this derivative there.
So, for example, if g is just a linear function, then I'm going to have a constant multiplication.
If g is a quadratic function, then I'm going to have theta squared that shows up there.
Things like that.
So just think of what kind of applications you could have for this.
Here are the functions that we're interested in, is x maps to 1 over x.
What is the derivative of this guy?
What is the derivative of 1 over x?
Negative 1 over x squared, right?
That's the thing we're going to have to put in there.
And so this is what we get.
So now when I'm actually going to write this, so if I want to show square root of n lambda hat minus lambda.
That's my application, right?
This is actually 1 over Tn, and this is 1 over 1 over lambda.
So the function g of x is 1 over x in this case.
So now I have this thing.
So I know that by the Delta method-- oh, and I knew that Tn, remember, square root of Tn minus 1 over lambda was going to sum normal with mean 0 and variance 1 over lambda squared, right?
So the sigma square over there is 1 over lambda squared.
So now this thing goes to what?
Sum normal.
What is going to be the mean?
0.
And what is the variance?
So the variance is going-- I'm going to pick up this guy, 1 over lambda squared, and then I'm going to have to take g prime of what?
Of 1 over lambda, right?
That's my theta.
So I have g of theta, which is 1 over theta.
So I'm going to have g prime of 1 over lambda.
And what is g prime of 1 over lambda?
So we said that g prime is 1 over negative 1 over x squared.
So it's negative 1 over 1 over lambda squared-- sorry, squared.
Which is nice, because g can be decreasing.
So that would be annoying to have a negative variance.
And so g prime is negative 1 over, and so what I get eventually is lambda squared up here, but then I square it again.
So this whole thing here becomes what?
Can somebody tell me what the final result is?
Lambda squared right?
So it's lambda 4 divided by lambda 2.
So that's what's written there.
And now I can just do my good old computation for a-- I can do a good computation for a confidence interval.
All right, so let's just go from the second line.
So we know that lambda hat minus lambda is less than, we've done that several times already.
So it's q alpha over 2-- sorry, I should put alpha over 2 over this thing, right?
So that's really the quintile of what our alpha over 2 times lambda divided by square root of n. All right, and so that means that my confidence interval should be this, lambda hat.
Lambda belongs to lambda plus or minus q alpha over 2 lambda divided by root n, right?
So that's my confidence interval.
But again, it's not very suitable, because-- sorry, that's lambda hat.
Because they don't know how to compute it.
So now I'm going to request from the audience some remedies for this.
What do you suggest we do?
What is the laziest thing I can do?
Anybody?
Yeah
[INAUDIBLE] PHILIPPE RIGOLLET Replace lambda by lambda hat.
What justifies for me to do this?
[INAUDIBLE] PHILIPPE RIGOLLET Yeah, and Slutsky tells me I can actually do it, because Slutsky tells me, where does this lambda come from, right?
This lambda comes from here.
That's the one that's here.
So actually I could rewrite this entire thing as square root of n lambda hat minus lambda divided by lambda converges to sum n 0, 1.
Now if I replace this by lambda hat, what I have is that this is actually really the original one times lambda divided by lambda hat.
And this converges to n 0, 1, right?
And now what you're telling me is, well, this guy I know it converges to n 0, 1, and this guy is converging to 1 by the law of large number.
But this one is converging to 1, which happens to be a constant.
It converges in probability, so by Slutsky I can actually take the product and still maintain my conversion to distribution to a standard Gaussian.
So you can always do this.
Every time you replace some p by p hat, as long as their ratio goes to 1, which is going to be guaranteed by the law of large number, you're actually going to be fine.
And that's where we're going to use Slutsky a lot.
When we do plug in, Slutsky is going to be our friend.
OK, so we can do this.
And that's one way.
And then other ways to just solve for lambda like we did before.
So the first one we got is actually-- I don't know if I still have it somewhere.
Yeah, that was the one, right?
So we had 1 over Tn q, and that's exactly the same that we have here.
So your solution is actually giving us exactly this guy when we actually solve for lambda.
So this is what we get.
Lambda hat.
We replace lambda by lambda hat, and we have our asymptotic convergence theorem.
And that's exactly what we did in Slutsky's theorem.
Now we're getting to it at this point is just telling us that we can actually do this.
Are there any questions about what we did here?
So this derivation right here is exactly what I did on the board I showed you.
So let me just show you with a little more space just so that we all understand, right?
So we know that square root of n lambda hat minus lambda divided by lambda, the true lambda defined converges to sum n 0, 1.
So that was CLT plus Delta method.
Applying those two, we got to here.
And we know that lambda hat converges to lambda in probability and almost surely, and that's what?
That was law of large number plus continued mapping theorem, right?
Because we only knew that one of our lambda hat converges to 1 over lambda.
So we had to flip those things around.
And now what I said is that I apply Slutsky, so I write square root of n lambda hat minus lambda divided by lambda hat, which is the suggestion that was made to me.
They said, I want this, but I would want to show that it converges to sum n 0, 1 so I can legitimately use q alpha over 2 in this one though.
And the way we said is like, well, this thing is actually really q divided by lambda times lambda divided by lambda hat.
So this thing that was proposed to me, I can decompose it in the product of those two random variables.
The first one here converges through the Gaussian from the central limit theorem.
And the second one converges to 1 from this guy, but in probability this time.
That was the ratio of two things in probability, we can actually get it.
And so now I apply Slutsky.
And Slutsky tells me that I can actually do that.
But when I take the product of this thing that converges to some standard Gaussian, and this thing that converges in probability to 1, then their product actually converges to still this standard Gaussian [INAUDIBLE] Well, that's exactly what's done here, and I think I'm getting there.
So in our case, OK, so just a remark for Slutsky's theorem.
So that's the last line.
So in the first example we used the problem dependent trick, which was to say, well, turns out that we knew that p is between 0 and 1.
So we have this p 1 minus p that was annoying to us.
We just said, let's just bound it by 1/4, because that's going to be true for any value of p. But here, lambda takes any value between 0 and infinity, so we didn't have such a trick.
It's something like we could see that lambda was less than something.
Maybe we know it, in which case we could use that.
But then in this case, we could actually also have used Slutsky's theorem by doing plug in, right?
So here this is my p 1 minus p that's replaced by p hat 1 minus p hat.
And Slutsky justify, so we did that without really thinking last time.
But Slutsky actually justifies the fact that this is valid, and still allows me to use this q alpha over 2 here.
All right, so that's the end of this lecture.
Tonight I will post the next set of slides, chapter two.
And, well, hopefully the video.
I'm not sure when it's going to come out.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Doesn't to run Flash Player.
So I had to run them on Chrome.
All right, so let's move on to our second chapter.
And hopefully, in this chapter, you will feel a little better if you felt like it was going a bit fast in the first chapter.
And the main reason we actually went fast, especially in terms of confidence intervals.
Some of you came and asked me what you mean by this is a confidence interval?
What does it mean that it's not happening in there with probability 95%, et cetera?
I just went really fast so that you could see why I didn't want to give you a first week doing probability only without understanding what the statistical context for it was.
So hopefully, all these things that we've done in terms of probability, you actually know why we've been doing them.
And so we're basically going to go back to what we're doing, maybe start with some statistical setup.
But the goal of this lecture is really going to go back again to what we've seen from a purely statistical perspective.
All right?
So the first thing we're going to do is explain why we're doing statistical modeling, right?
So in practice, if you have data, if you observe a bunch of points-- in here, I gave you some numbers, for example.
So here's the partial data sets with the number of siblings, including self, that were collected from college students a few years back.
So I was teaching a class like yours and actually asked students to go and fill out some Google form and tell me a bunch of things.
And one of the questions was, including yourself, how many siblings do you have?
And so they gave me this list of numbers, right?
And there's many ways I can think of this list of numbers, right?
I could think of it as being just a discrete distribution on the set of numbers between 1-- I know there's not going to be an answer which is less than 1, unless, well, someone doesn't understand the question.
But all the answers I should get are positive integers-- 1, 2, 3, et cetera.
And there probably is an upper bound, but I don't know it on the top of my head.
So maybe I should say 100.
Maybe I should say 15.
It depends, right?
And so I think the largest number I got for this was 6.
All right?
So here you can see you have pretty standard families, you know, lots of 1s, 2s, and 3s.
What statistical modeling is doing is to try to compress this information that I could actually describe in a very naive way.
So let's start with the basic usual statistical set up, right?
So I will start with many of the boards that look like x1, xn, random variables.
And what I'm going to assume, as we said typically is that those guys are IID.
And they have some distribution, all right?
So they all share the same distribution.
And the fact that their IID is so that I can actually do statistics.
Statistics means looking at the global averaging thing so that I can actually get a sense of what the global behavior is for the population, right?
If I start assuming that those things are not identically distributed-- they all live on their own-- that my sequence of number is your number of siblings-- the shoe size of this person-- the depth of the Charles River, and I start measuring a bunch of stuff.
There's nothing I can actually get together.
I need to have something that's cohesive.
And so here, I collected some data that was cohesive.
And so the goal here-- the first thing is to say what is the distribution that I actually have here, right?
So I could actually be very general.
I could just say at some distribution p. And let's so those are random variables, not random vectors, right?
I could collect entire vectors about students, but let's say those are just random variables.
And so now I can start making assumptions on this distribution p, right?
What can I say about a distribution?
Well, maybe if those numbers are continues, for example, I could assume they have a density-- a probability density function.
That's already an assumption.
Maybe I could start to assume that they're probability density function is smooth.
That's another assumption.
Maybe I could actually assume that it's piecewise constant.
That's even better, right?
And those things make my life simpler and simpler, because what I do by making the successive assumptions is reducing the degrees of freedom of the space in which I am actually searching the distribution.
And so what we actually want is to have something which is small enough so we can actually have some averaging going on.
But we also want something which is big enough that it can actually express.
It has chances of actually containing a distribution that makes sense for us.
So let's start with the simplest possible example, which is when the xi's belong to 0, 1.
And as I said, here, we don't have a choice.
The distribution of those guys has to be Bernoulli.
And since they are IID, they all share the same p. So that's definitely the simplest possible thing I could think of.
They are just Bernoulli p. And so all I would have to figure out in this case is p. And this is the simplest case.
And unsurprisingly, it has the simplest answer, right?
We will come back to this example when we study maximum likelihood estimators or method of moments estimators by method of moments.
But at the end of the day, the things that we did-- the things that we will do are always the naive estimator you would come up with is what is the proportion of 1.
And this will be, in pretty much all respects, the best estimator you can think of.
All right?
So then we're going to try to assess this performance.
And we saw how to do that in the first chapter as well.
So this problem here somehow is completely understood.
We'll come back to it.
But there's nothing fancy that is going to happen.
But now, I could have some more complicated things.
For example, in the example of the students now, my xi's belong to the sequence of integers 1, 2, 3, et cetera, OK, which is also denoted by n, maybe without 0 if you want to put 0 in that, right?
So the positive integers.
Or I could actually just maybe put some prior knowledge about how humans have time to have families.
But maybe some people thought of their college mates as being their brothers and sisters.
And one student would actually put 465 siblings, because we're all good friends.
Or maybe they actually think that all their Facebook contacts are actually their siblings.
And so you never know what's going to happen.
So maybe you want to account for this, but maybe you know that people are reasonable, and they will actually give you something like this.
Now intuitively, maybe you would say, well, why would you bother doing this if you're not really sure about the 20?
But I think that probably all of you intuitively guess that this is probably a good idea to start putting this kind of assumption rather than allowing for any number in the first place, because this eventually will be injected into the precision of our estimators.
If I allow anything, it's going to be more complicated for me to get an accurate estimator.
If I know that the numbers are either 1 or 2, then I'm actually going to be slightly more accurate as well.
All right?
Because I know that, for example, somebody put the 5, I can remove it.
Then it's not going to actually corrupt with my estimator.
All right, so now, let's say we actually agree that we have numbers.
And here I put seven numbers, OK?
So I just said, well, let's assume that the numbers I'm going to get are going to be 1 all the way to say this number that I denote by larger than or equal to 7, which is a placeholder for any numbers larger than 7, OK?
Because I know maybe I don't want to distinguish between people that have 9 or 25 siblings.
OK, and so now, this is a distribution on seven possible values-- the discrete distributions.
And you know from your probability class that the way you describe this distribution is using the probability mass function.
OK, or PMF-- So that's how we describe a discrete distribution.
And the PMF is just a list of numbers, right?
So as I wrote here, you have a list of numbers.
And here, you wrote the possible value that your random variable can take.
And here you rate the probability that your random variable takes this value.
So the possible values being 1, 2, 3 all the way to larger than or equal to 7.
And then I'm trying to estimate those numbers.
Right?
If I give you those numbers, at least up to this you know compression of all numbers that are equal to 7, you have the full description of your distribution.
And that is the ultimate goal of statistics, right?
The ultimate goal of statistics is to say what distribution your data came from, because that's basically the best you're going to be able to do.
Now admittedly, if I started looking at the fraction of 1s, and the fraction of 2s, and the fraction of 3s, et cetera, I would actually eventually get those numbers-- just like looking at the fraction of 1s gave me a good estimate for p in the Bernoulli case, it would do the same in this case, right?
It's a pretty intuitive idea.
It's just the law of large numbers.
Everybody agrees with that?
If I look at the proportion of 1s, the proportion of 2s, the proportion of 3s, that should actually give me something that gets closer and closer, as my sample size increases to what I want.
But the problem is if my sample size is not huge, here I have seven numbers to estimate.
And if I have 20 observations, the ratio is not really in my favor-- 20 observations to estimate seven parameters-- some of them are going to be pretty off, typically the ones with the large values.
If you have only 20 students, look at the list of numbers.
I don't know how many numbers I have, but it probably is close to 20-- maybe 15 or something.
And so if you look at this list, nobody's actually-- nobody has four or more siblings, right?
There's no such person.
So that means that eventually from this data set, my estimates-- so those numbers I denote by say p1, p2, p3, et cetera-- those estimates p4 hat would be equal to what from this data?
0, right?
And p5 hat equal to 0 and p6 hat would be equal to 0.
And p larger than or equal to 7 hat would be equal to 0.
That would be my estimate from this data set.
So maybe this is not-- maybe I want to actually pull some information from the people who have less siblings to try to make a guess, which is probably slightly better for the larger values, right?
It's pretty clear that in average, there is more than 0-- the proportion of the population of households that have four children or more is definitely more than 0, all right?
So it means that my data set is not representative of what I'm going to try to do is to find a model that tries to use the data they have for the smaller values that I can observe and just push it up to the other ones.
And so what we can do is to just reduce those parameters into something that's understood.
And this is part of the modeling that I talked about in the first place.
Now, how do you succinctly describe a number of something?
Well, one thing that you do is the Poisson distribution, right?
Why do Poisson?
There's many reasons.
Again, that's part of statical modeling.
But once you know that you have number of something that can be modeled by a Poisson, why not try a Poisson, right?
You could just fit a Poisson.
And the Poisson is something that looks like this.
And I guess you've all seen it.
But if x follows a Poisson distribution with parameter lambda, than the probability that x is equal to little x is equal to lambda to the x over factorial x e to the minus lambda.
OK?
And if you did the sheet that I gave you on the first day, you can check those numbers.
So this is, of course, for x equals 0, 1, et cetera, right?
So x is in natural integers.
And if you sum from x equals 0 to infinity, this thing you get is e to the lambda.
And so they cancel, and you have some which is equal to 1, which is indeed a PMF.
But what's key about this PMF is that it never takes value 0.
Like this thing is always strictly positive.
So whatever value of lambda I find from this data will give me something that's certainly more interesting than just putting the value 0.
But more importantly, rather than having to estimate seven parameters and, as a consequence, to actually have to estimate 1, 2, 3, 4 of them being equal to 0, I have only one parameter to estimate which is lambda.
The problem with doing this is that now lambda may not be just something as simple as computing the average number.
Right?
In this case, it will.
But in many instances, it's actually not clear that this parametrization with lambda that I chose-- I'm going to be able to estimate lambda just by computing the average number that I get.
It will be the case.
But if it's not, remember this example of the exponential we did in the last lecture-- we could use the delta method and things like that to estimate them.
All right, so here's modeling 101.
So the purpose of modeling is to restrict the base of possible distributions to a subspace that's actually plausible, but much simpler for me to estimate.
So we went from all distributions on seven parameters, which is a large space-- that's a lot of things-- to something which is just one number.
This number is positive.
Any question about the purpose of doing this?
OK, so we're going to have to do a little bit of formalism now.
And so if we want to talk-- this is a statistics classroom.
I'm not going to want to talk about the Poisson model specifically every single time.
I'm going to want to talk about generic models.
And then you're going to build to plug in your favorite word-- Poisson, binomial, exponential, uniform-- all these words that you've seen, you're going to be able to plug in there.
But we're just going to have some generic notations and some generic terminology for a statistical model.
All right?
So here is the formal definition.
So I'm going to go through it with you.
OK, so the definition is that of a statistical model.
OK?
Sorry, that's a statistical experiment, I should say.
So a statistical experiment is actually just a pair-- E. And that's a set-- and a family of distributions P theta, where theta ranges in some set capital theta.
OK?
So I hope you're up to date with your Greek letters.
So the small theta is the capital theta.
And enough of us-- I don't have the handwriting.
So if you don't see something, just ask me.
And so this thing now-- so each of this guy is a probability distribution.
All right?
So for example, this could be a Poisson with parameter theta or a Bernoulli with parameter theta-- OK, or an exponential with parameter-- I don't know-- 1 over theta squared if you want.
OK, but they're just indexed by theta.
But for each theta, this completely describes the distribution.
It could be more complicated.
This theta should be a pair-- could be a pair-- a mu sigma square.
And that could actually give you some n mu sigma squared.
OK so anything where you can actually-- rather than actually giving you a full distribution, I can compress into a parameter.
But it could be worse.
It could be this guy here.
Right?
Theta could be p1-- p larger than or equal to 7.
And my distribution could just be something that has PMF-- p1-- p larger than 7.
That's another parameter.
This one is seven dimensional.
This one is two dimensional.
And all these guys are just one dimensional.
All these guys are parameters.
Is that clear?
What's important here is that once they give you theta, you know exactly all the probabilities associated with this random variable.
You know its distribution perfectly.
So this is the definition.
Is that clear?
Is there a question about this distribution-- about this definition, sorry?
All right.
So really, the key thing is the statistical model associated to a statistical experiments.
OK?
So let's just see some examples.
It's probably just better because, again, the formalism is never really clear.
Actually, that's the next slide.
OK, so there's two things we need to assume.
OK, so the purpose of a statistical model is once I estimate the parameter, I actually know exactly what distribution it has, OK?
So it means that I could potentially have several parameters that give me the same distribution that would still be fine, because I could estimate one guy.
Or I could estimate the other guy.
And I would still recover the underlying distribution of my data.
The problem is that this creates really annoying theoretical problems, like things don't work, the algorithms won't work, the guarantees won't work.
And so what we typically assume is that the model is so-called well-specified.
Sorry, that's not well specified.
I'm jumping ahead of myself.
OK, well-specified means that your data-- the distribution of your data is actually one of those guys.
OK?
So some vocabulary-- so well-specified means that for my observations x, there exists a theta in capital theta such that x follows p sub theta.
I should put a double bar.
OK, so that's what well-specified means.
So that means that the distribution of your actual data is just one of those guys.
This is a bit strong of an assumption.
It's strong in the sense that-- I don't know if you've heard of this sense, which I don't know, I can tell you who it's attributed to, but that probably means that this person did not come up with it.
But I said that all models are wrong, but some of them are useful.
All right, so all models are wrong means that maybe it's not true that this Poisson distribution that I assume for the number of siblings for college students-- maybe that's not perfectly correct.
Maybe there's a spike at three, right?
Maybe there's a spike at one, because you know, maybe those are slightly more educated families.
They have less children.
Maybe this is actually not exactly perfect.
But it's probably good enough for our purposes.
And when we make this assumption, we're actually assuming that the data really comes from a Poisson model.
There is a lot of research that goes on about misspecified models and that tells you how well you're doing in the model that's the closest to the actual distribution.
So that's pretty much it.
Yeah?
[INAUDIBLE]
So my data-- so it's always the way I denote one of the generic observations, right?
So my observations are x1, xn.
And they're IID with distribution p-- always.
So x is just one of those guys.
I don't want to write x5 or x4.
They're IID.
So they all have the same distribution.
So OK-- no, no, no.
They're all IID.
So they all have the same p data.
They'll have the same p, which means they'll have the same p data.
So I can pick any one of them.
So I'd just remove the index just so we're clear.
OK?
So when I write x, I just mean think of x1.
Right they're an idea.
I can pick whichever I want.
I'm not going to write x1.
It's going to be weird.
OK?
Is that clear?
OK.
So this particular theta is called the true parameter.
Sometimes since we're going to want some variable theta, we might denote it by theta star as opposed to theta hat, which is always our estimator.
But I'll keep it to be theta for now.
And so the aim of this physical experiment is to estimate theta so that once I actually plug in theta in the form of my distribution, for example, I could plug in theta here.
So theta here was actually lambda.
So once I estimate this guy, I would plug it in, and I would know the probability that my random variable takes any value, by just putting the lambda hat and the lambda hat here.
OK?
So my goal is going to be to estimate this guy so that I can actually compute those distributions.
But actually, we'll see, for example, when we talk about regression that this parameter actually has a meaning in many instances.
And so just knowing the parameter itself intuitively or say more-- let's say more so than just computing probabilities, will actually tell us something about the process.
For example, we're going to run linear regression.
And when we do linear regression, there's going to be some coefficients in the linear regression.
And the value of this coefficient is actually telling me what is the sensitivity of the response that I'm looking at to this particular input.
All right?
So just knowing if this number is larger or if this number is small is actually going to be useful for us to just look at this guy.
All right?
So there's going to be some instances where it's going to be important.
Sometimes we're going to want to know if this parameter is larger or smaller than something or if it's equal to something or not equal to something.
And those things are also important-- for example, if theta actually measures the true-- right?
So theta is the true unknown parameter-- true efficacy of a drug.
OK?
Let's say I want to know what the true efficacy of a drug is.
And what I'm going to want to know is maybe it's a score.
Maybe I'm going to want to know if theta is larger than 2.
Maybe I want to know if theta is the average number of siblings.
Is this true number larger than 2 or not?
Right?
Maybe I am interested in knowing if college students come from-- so maybe from a sociological perspective, I'm interested in knowing if college students come from households with more than two children.
All right, so those can be the questions that I may ask myself.
I'm going to want to know maybe theta is going to be equal to 1/2 or not.
So maybe for a drug efficacy, is it completely standard-- maybe for elections.
Is the proportion of the population that is going to vote for this particular candidate equal to 0.5?
Or is it different from 0.5?
OK, and I can think of different things.
When I'm talking about the regression, I'm going to want to test if this coefficient is actually 0 or not, because if it's 0, it means that the variable that's in front of it actually goes out.
And so those are things we're testing.
Actually having this very specific yes/no answer is going to give me a huge intuition or huge understanding of what's going on in the phenomenon that I observe.
But actually, since the questions are so precise, it's going to be much more-- I'm going to be much better at answering them rather than giving you an estimate for theta with some confidence around it.
All right, it's sort of the same principle as trying to reduce.
What you're trying to do as a statistician is to inject as much knowledge about the question and about the problem that you can so that the data has to do a minimal job.
And henceforth, you actually need less data.
So from now on, we will always assume-- and this is because this is an intro stats class-- you will always assume that theta-- the subset of parameters is a subset of r to the d. That means that theta is a vector with at most a finite number of coordinates.
Why do I say this?
Well, this is called a parametric model.
So it's called a parametric model or sometimes parametric statistics.
Actually, we don't really talk about parametric statistics.
But we talk a lot about nonparametric statistics or a non-parametric model.
Can somebody think of a model which is non-parametric?
For example, in the siblings example, if I did not cap the number of siblings to 7, but I let this list go to infinity, I would have an infinite number of parameters to estimate.
Very likely, the last ones would be 0.
But still, I would have an infinite number of parameters to estimate.
So this would not be a parametric model if I just let this list of things to be estimated to be infinite.
But there's other classes that are actually infinite and cannot represented by vectors.
For example, function-- right?
If I tell you my model, pf, is just the distribution of x's, the probability distributions, that have density f, right?
So what I know is that the density is non-negative and that it integrates to one, right?
That's all I know about densities.
Well f is not something you're going to be able to describe with a finite number of values, right?
All possible functions is the huge set.
It's certainly not representable by 10 numbers.
And so non-parametric estimation is typically when you actually want to parametrize this by a large class of functions.
And so for example, histograms is the prime tool of non-parametric estimation, because when you fit a histogram to data, you're trying to estimate the density of your data, but you're not trying to represent it as a finite number of points.
That's really-- I mean, effectively, you have to represent it, right?
So you actually truncate somewhere and just say those things are not going to matter.
All right?
But really the key thing is that this is non-parametric where you have a potentially infinite number of parameters.
Whereas we're going to only talk about finites.
And actually finite in the overwhelming majority of cases is going to be 1.
So theta is going to be a subset of r1.
OK, we're going to be interested in estimating one parameter just like the parameter of a Poisson or the parameter of an exponential-- the parameter of Bernoulli.
But for example, really, we're going to be interested in estimating mu and sigma square for the normal.
So here are some statistical models.
All right?
So I'm going to go through them with you.
So if I tell you I observe-- I'm interested in understanding-- I'm still [INAUDIBLE] I'm interested in understanding the proportion of people who kiss by bending their head to the right.
And for that, I collected n observations.
And I'm interested in making some inference in the statistical model.
My question to you is, what is the statistical model?
Well, if you want to read the statistical model, you're going to have to write this E-- oh, sorry, I never told you what E was.
OK, well actually just go to the examples, and then you'll know what E is.
So you're going to have to write to me an E and a p theta, OK?
So let's start with the Bernoulli trials.
So this e here is called the sample space.
And in the normal people's words, it just means the space or the set in which x and-- back to your question, x is just a generic observation lips.
OK, and hopefully, this is the smallest you can think of.
OK, so for example, for Bernoulli trials, I'm going to observe a sequence of 0's and 1's.
So my experiment is going to be-- as written on the board, is going to be 1, 0, 1.
And then the probability distributions are going to be, well, it's just going to be the Bernoulli distributions indexed by p, right?
So rather than writing p sub p, I'm going to write it as Bernoulli p, because it's clear what I mean when I write that.
Is everybody happy?
Actually, I need to tell you something more.
This is a family of distributions.
So I need p. And maybe I don't want to have to p that's a value 0, 1, right?
It doesn't make sense.
I would probably not look at this problem if I anticipated that everybody would kiss to the right.
And everybody would kiss to the left.
So I am going to assume that p is in 0, 1, but does not have 0 and 1.
OK?
So that's the statistical model for a Bernoulli trial.
OK, now the next one, what do we have?
Exponential.
OK?
OK, so when I have exponential distributions, what is the support of the exponential distribution?
What value is it going to take?
0 to infinity, right?
So what I have is that my first space is the value that my random variables can take.
So its-- well, actually I can remove the 0 again-- 0 to plus infinity.
And then the family of distributions that I have are exponential with parameter lambda.
And again, maybe you've seen me switching from p, to lambda, to theta, to mu, to sigma square.
Honestly you can do whatever you want.
But its just that it's customary to have this particular group of letters.
OK?
And so the parameters of an exponential are just positive numbers.
OK?
And that's my exponential model.
What is the third one?
Can somebody tell me?
Poisson, OK?
OK, so Poisson-- is a Poisson random verbal discrete or continuous?
Go back to your probability.
All right, so the answer being the opposite of continuous-- good job.
All right, so it's going to be-- what value can a Poisson take?
All the natural integers, right?
So 0, 1, 2, 3, all the way to infinity.
We don't have any control of this.
So I'm going to write this as n without 0.
I think in the slides, it's n-star maybe.
Actually, no, you can take value 0.
I'm sorry.
This actually takes value 0 quite a lot.
That's typically, in many instances, actually the mode.
So it's n, and then I'm going to write it as Poisson with parameter-- well, here it's again lambda as a parameter.
And lambda can take any positive value.
OK?
And that's where you can actually see that the model that we had for the siblings-- right?
So let me actually just squeeze in the siblings model here.
So that was the bad model that I had in the first place when I actually kept this.
Let's say we just kept it at 7.
Forget about larger than or equal to 7.
We just assumed it was 7.
What was our sample space?
We said 7.
So it's 1, 2, to 7, right?
Those were the possible values that this thing would take.
And then what was my-- what's my parameter space?
So it's going to be a nightmare write.
But I'm going to write it.
OK, so I'm going to write it as something like the probability that x is equal to k is equal to p sub k. OK?
And that's going to be for p. OK, so that's for all k's, right?
Or for k equal 1 to 7.
And here the index is the set of parameters p1 to pk.
And I know a little more about those guys, right?
I know there are going to be non-negative-- PJ non-negative.
And I know that they sum to 1.
OK, so maybe writing this, you start seeing why we like those Poisson, exponential, and short notation, because I actually don't have to write the PMF of a Poisson.
The Poisson is really just this.
But I call it Poisson so I don't have to rewrite this all the time.
And so here, I did not use a particular form.
So I just have this thing, and that's what it is.
The set of parameters is the set of positive numbers of-- p1 to p7, pk-- and sum to 7, right?
And so this as just a list of numbers that are non-negative and sum up to 1.
So that's my parameter space.
OK?
So here that's my theta.
This whole thing here-- this is my capital theta.
OK?
So that's just the set of parameters that theta-- the set of parameters that theta is allowed to take.
OK, and finally, we're going to end with the star of all, and that's the normal distribution.
And in the normal distribution, you still have also some flexibility in terms of choices, because then naturally, the normal distribution is parametrized by-- the normal distribution is parametrized by two parameters, right?
Mean and variance.
So what values can a Gaussian random variable take?
An entire real line, right?
And the set of parameters that it can take it-- so this is going to be n, mu, sigma square.
And mu is going to be positive.
And stigma square is going-- sorry, m is going to be an r. And sigma square is going to be positive.
OK, so again here, that's the way you're supposed to write it.
If you really want to identify what theta is, well, theta formally is the set of mu sigma square such that-- well, in r times 0 infinity, right?
That's just to be formal, but this does the job just fine.
OK?
You don't have to be super formal.
OK, that's not three.
That's like five.
Actually, I just want to write another one.
Let's call it 5-bit.
And 5-bit is just Gaussian with known variants.
And this arises a lot in labs when you have measurement error-- when you actually receive your measurement device.
This thing has been tested by the manufacturer so much that it actually comes in on the side of the box.
It says that the standard deviation of your measurements is going to be 0.23.
OK, and actually why you do this is because we can brag about accuracy, right?
That's how they sell you this particular device.
And so you actually know exactly what sigma square is.
So once you actually get your data in the lab, you actually only have to estimate mu, because stigma comes on the label.
So now, what is your statistical model?
Well, the numbers are collecting still in r. But now, the models that I have is n, mu, sigma squared.
But the parameter space is not mu, and r, and sigma positive.
It's just mu and r. And to be a little more emphatic about this, this is enough to describe it, right?
Because if sigma is the sigma that was specified by the manufacturer, then this is the sigma you want.
But you can actually write sigma is equal to-- sigma square is equal to sigma square manufacturer.
Right?
You can just fix it to be this particular value.
Or maybe you don't want to write that index that's the manufacturer.
And so you just say, well, the sigma-- when I write n squared what I mean is the sigma square from the manufacturer.
Yeah?
[INAUDIBLE]
Yeah.
For a particular measuring device?
You know, you're in a lab, and you have some measuring device.
I don't know-- something that measures tensile strength of something.
And it's just going to measure something.
And it will naturally make errors.
But it's been tested so much by the manufacturer and calibrated by them.
They know it's not going to be perfect.
But they knew exactly what error it was making, because they've actually tried it on things for which they exactly knew what the tensile strength was.
OK?
Yeah
[INAUDIBLE]
This?
[INAUDIBLE]
Oh, like that's pointing to-- 5 prime?
OK?
And we can come up with other examples, right?
So for example, here's another one.
So the names don't really matter, right?
I call it the siblings model.
But you won't find the siblings model in the textbook, right?
So I wouldn't worry too much.
But for example, let's say you have something-- so let's call it 6.
You have-- I don't know-- a truncated-- and that's the name I just came up with.
But it's actually not exactly describing what I want.
But let's say I observe y, which is the indicator of x larger than say 5 when x follows some exponential with parameter lambda.
OK?
This is what I get to observe.
I only observe if my waiting time was more than five minutes, because I see somebody coming out of the Kendall Station being really upset.
And that's all I record is I've been waiting for more than five minutes.
And that's all I get to record.
OK?
That happens a lot.
These are called censored data.
I should probably not call it truncated, but this should be censored.
OK?
You see a lot of censored data when you ask people how much they make.
They say, well, more than five figures.
And that's all they want to tell you.
OK?
And so you see a lot of censored data in survival analysis, right?
You are trying to understand how long your patients are going to live after some surgery, OK?
And maybe you're not going to keep people alive, and you're not going to actually be in touch in their family every day and ask them, is the guy still alive?
And so what you can do is just you ask people maybe five years after your study and say, please, come in.
And you will just happen to have some people say, well, you know, the person is deceased.
And you will only be able to know that the person deceased less than five years ago.
But you only see what happens after that, OK?
And so this is this truncated and censored data.
It happens all the time just because you don't have the ability to do better than that.
So this could happen here.
So what is my physical experiment, right?
So here, I should probably write this like this, because I just told you that my observations are going to be x, but there is some unknown y. I will never get to see this y. I only get to see the x.
What is my statistical experiment?
Please help me.
So is it the real line?
My sample space-- is it the real line?
Sorry, who does not know what this means?
I'm sorry.
OK.
So this is called an indicator.
So I read it as-- if I wrote well, that would be one with a double bar.
You can also write i if you prefer if you don't feel like writing one in double bars.
And it's one of say-- I'm going to write it like that-- 1 of a is equal to 1 if a is true and 0 if a is false.
OK?
So that means that if y is larger than 4, this thing is 1.
And if y is not larger than 5, this thing is 0.
OK.
So that's called an indicator-- indicator function.
It was very useful to just turn anything into a 0, 1.
So now that I'm here, what is my sample space?
0, 1.
Well, whatever this thing I did not tell you was taking value with the thing you should have-- if I end up telling you that is taking value 6 or 7 that would be your sample space, OK?
OK, so it takes values 0, 1.
And then what is the probability here?
What should I write here?
What should you write without even thinking?
Yeah.
So let's assume there's two seconds before the end of the exam.
You're going to write Bernoulli.
And that's where you're going to start checking if I'm going to give you extra time, OK?
So you write Bernoulli without thinking, because it's taking value 0, 1.
So you just write Bernoulli, but you still have to tell me what possible parameters this thing is taking, right?
So I'm going to write it p, because I don't know.
And then p take value-- OK, so sorry.
I could write it like that.
Right?
That would be perfectly valid, but actually no more.
It's not any p. The p is the probability that an exponential lambda is larger than 5.
And maybe I want to have lambda as a parameter.
OK, so what I need to actually compute is, what is the probability that y is larger than 5-- when y is this exponential lambda, which means that what I need to compute is the integral between 5 and infinity of-- what is it?
1 over lambda.
How did I define it in this class?
Did I change it-- what?
[INAUDIBLE]
Yeah, right, right, right.
Yeah.
Lambda e to the minus lambda x dx, right?
So that's what I need to compute.
What is this?
Yeah, so what is the value of this integral?
Can you take appropriate measures?
[INAUDIBLE]
OK?
And again, you can cancel this, right?
So when I'm going to integrate this guy, those guys are going to cancel.
I'm going to get 0 for infinity.
I'm going to get a 5 for this guy.
And well, I know it's going to be positive number, so I'm not really going to bother with the signs, because I know that's what it should be.
OK, so I get e to the minus 5 lambda.
And so that means that I can actually write this like that-- and now parametrize this thing by lambda positive.
OK?
So what I did here is I changed the parametrization from p to lambda.
Why?
Well, because maybe if I know this is happening, maybe I am actually interested in reporting lambda to MBTA, for example.
Maybe I'm actually trying to estimate 1 over lambda, so that I know it is-- well, lambda is actually the intensity of arrival of my Poisson process, right?
I have a Poisson process.
That's how my trains are coming in.
And so I'm interested in lambda.
So I will parametrize things by lambda.
So the thing I get is lambda.
You can play with this, right?
I mean, I could parametrize this by 1 over lambda and put 1 over lambda here if I want it.
But you know, the context of your problem will tell you exactly how to parametrize this.
OK?
So what else did I want to tell you?
OK, let's do a final one.
By the way, are you guys OK with Poisson exponential, Bernoulli's-- I don't know, binomial, normal-- all these things.
I'm not going to go back to it, but I'm going to use them heavily.
So just spend five minutes on Wikipedia if you forgot about what those things are.
Usually, you must have seen them the in your probability class.
So they should not be a crazy name.
And again, I'm not expecting you.
I don't remember what the density of an exponential is.
So it would be pretty unfair of me to actually ask you to remember what it is.
Even for the Gaussian, I don't expect you to remember what it is.
But I want you to remember that if I add 5 to a Gaussian, then I have a Gaussian with me and mu plus 5 if I multiply it by something, right?
You need to know how to operate those things.
But knowing complicated densities is definitely not part of the game.
OK?
So let's do a final one.
I don't know what number I have now.
I'm going to just do uniform.
That's another one.
Everybody knows what uniform is?
So it's uniform, right?
So I'm going to have x, which my observations are going to be uniform on the interval 0 theta, right?
So if I want to define a uniform distribution for a random variable, I have to tell you which interval or which set I want it to be uniform on.
And so here I'm telling you is the interval 0 theta.
And so what is going to be my sample space?
[INAUDIBLE]
I'm sorry?
0 to theta.
And then what is my probability distribution?
My family of parameters?
So well, I can write it like this, right?
Uniform theta, right?
And theta let's say is positive.
Can somebody tell me what's wrong with what I wrote?
This makes no sense.
Tell me why.
Yeah?
Yeah, this set depends on theta, and why is that a problem?
[INAUDIBLE]
There is no theta.
Right now, there's the families of theta.
Which one did you pick here?
Right, this is just something that's indexed by theta, but I could have very well written it as, you know, just not being Greek for a second, I could have just written this as t rather than theta.
That would be the same thing.
And then what the hell is theta?
There's no such thing as theta.
We don't know what the parameter is.
This parameter should move with everyone.
And so that means that I actually am not allowed to pick this theta.
I'm actually-- just for the reason that there is no parameter to put on the left side-- there should not be, right?
So you just said, well, there's a problem because the parameter is on the left-hand side.
But there's not even a parameter.
I'm describing the family of possible parameters.
There is no one that you can actually plug it in.
So this should really be 1.
And I'm going to go back to writing this as theta because that's pretty standard.
Is that clear for everyone.
I cannot just pick one and put it in there and just take the-- before I run my experiments, I could potentially get numbers that are all the way up to 1, because I don't know what theta is going to be ahead of time.
Now, if somebody promised to me that theta was going to be less than 0.5, that would be-- sorry, why do I put 1 here?
I could put theta between 0 and 1.
But if somebody is going to promise me, for example, if theta is going to be less than 1, then you expect to put 0, 1.
All right?
Is that clear?
OK, so now you know how to answer the question-- what is the statistical model?
And again, within the scope of this class, you will not be asked to just come up with a model right that will just tell you.
Poisson would be probably be a good idea here.
And then you would just have to trust me that indeed it would be a good idea.
All right, so what I started talking about 20 minutes ago-- so it's definitely ahead of myself is the notion-- so that's when I was talking about well-specified.
Remember, well-specified says that the true distribution is one of the distributions in this parametric families of distribution.
The true distribution of my siblings is actually a Poisson with some parameters.
And all I need to figure out is what this parameter is.
When I started saying that, I said, well, but then that could be that there are several parameters that give me the same distribution, right?
It could be the case that Poisson 5 and Poisson 17 are exactly the same distributions when I started putting those numbers in the formula which I erased, OK?
So it could be the case that two different numbers would give me exactly the same probabilities.
And in this case, we see that the model is not identifiable.
I mean, the parameter is not identifiable.
I cannot identify the parameter, even if you actually gave me an infinite amount of data, which means that I could actually estimate exactly the PMF.
I might not be able to go back, because there would be several candidates, and I would not be able to tell you which one it was in the first place.
OK?
So what we want is that this function-- theta maps to p theta is injective.
And that really can be fancy.
What I really mean is that if theta is different from theta prime, then p of theta is different from p of theta prime.
Or, if you prefer to think about the contrapositive of this, this is the same as saying that if p theta gives me the same distribution as theta prime, then that implies that theta must be equal to the theta prime.
The logic of those two things are equivalent, right?
So that's what this means.
So this is-- we say that the parameter is identifiable or identified-- it doesn't really matter-- in this model.
And this is something we're going to want.
OK?
So in all the examples that I gave you, those parameters are completely identified.
Right?
If I tell you-- I mean, if those things are in probability box, it means that they were probably thought through, right?
So when I say exponential lambda, I'm really talking about one specific distribution and not-- there's not another lambda going to give you exactly the same distribution.
OK so that was the case.
And you can check that, but it's a little annoying.
So I would probably not do it.
But rather than doing this, let me just give you some examples where it would not be the case.
Again, here's an example, if I take xi-- so now I'm back to just using this indicator function-- but now for a Gaussian.
So what I observe is x is the indicator that y is, what did we say?
Positive.
OK?
So this is a Bernoulli random variable, right?
And it has some parameter p. But p now is going to depend-- sorry, and here y is n mu sigma square.
So the p, the probability that this thing is positive, is actually-- I don't think I put the 0.
Oh, yeah, because I have mu.
OK, so this distribution-- this p the probability that it's positive is just the probably that some Gaussian is positive.
And it will depend on mu and sigma, right?
Because if I draw a 0, and I draw my Gaussian around mu, then the probability of this Bernoulli being 1 is really the area under the curve here.
Right?
And this thing-- well, if mu is very large, it's going to become very large.
If mu is very small, it's going to become very small.
And if sigma changes, it's also going to effect-- is that clear for everyone?
But we can actually compute this, right?
So the parameter p that I'm looking for here as a function of mu and sigma is simply the probability that some y is non-negative, which is the probability that y minus mu divided by sigma is larger than minus mu divided by sigma.
But when you study probability, is that some operation you were used to making?
Removing the mean and dividing by the standard deviation?
What is the effect of doing that on the Gaussian random variable?
Yeah, so you normalize it, right?
And you standardize it.
You make it a standard Gaussian.
You remove the mean.
The mean 0 is Gaussian.
And you remove the variance for it to become 1.
So when you have a Gaussian, remove the mean and divide by the standard deviation, it becomes a standard Gaussian-- which this thing has n , 0, 1 distribution, which is the one you can read the quintiles of at the end of the book.
Right?
And that's exactly what we did.
OK?
So now you have the probability that some standard Gaussian exceeds negative mu over sigma, which I can write in terms of the cumulative distribution function, capital phi-- like we did in the first lecture.
So if I do this cumulative distribution function, what is this probability in terms of phi?
[INAUDIBLE]?
[INAUDIBLE]
Well, that's what your name tag says.
1 minus--
[INAUDIBLE]
1 minus mu of sigma.
What happens with phi in our-- do you think I defined this for fun?
1 minus phi of mu over sigma, right?
Right?
Because this is 1 minus the probability that it's less than this.
And this is exactly the definition of the cumulative distribution function.
So in particular, this thing only depends on mu over sigma.
Agreed?
So in particular, if I had 2 mu over 2 sigma, p would remain unchanged.
If I have 12 mu over 12 sigma, this thing would remain unchanged, which means that p does not change if I scale mu and sigma by the same factor.
So there's no way just by observing x, even an infinite times, so that I can actually get exactly what p is.
I'm never going to be able to get mu and sigma separately.
All I'm going to be able to get is mu over sigma.
So here, we say that mu sigma-- the parameter mu sigma-- or actually each of them individually-- those guys-- they're not identifiable.
But the parameter mu over sigma is identifiable.
So if I wanted to write a statistical model in which the parameter is identifiable-- I would write 0, 1 Bernoulli.
And then I would write 1 minus phi over mu over sigma.
And then I would take two parameters, which are mu and r and sigma squared positive.
So let's write sigma positive.
Right?
No, this is not identifiable.
I cannot write those two guys as being two things different.
Instead, what I want to write is 0, 1, Bernoulli 1 minus-- and now my parameter-- I forgot this-- my parameter is mu over sigma.
Can somebody tell me where mu over sigma lives?
What values can this thing take?
Any real value, right?
OK, so now I've done this definitely out of convenience, right?
Because that was the only thing I was able to identify-- the ratio of mu over sigma.
But it's still something that has some meaning.
It's the normalized mean.
It really tells me what the mean is compared to the standard deviation.
So in some models, in reality, in some real applications, this actually might have a good meaning.
It's just telling me how big the mean is compared to the standard fluctuations of this model.
But I won't be able to get more than that.
Agreed?
All right?
So now that we've set a parametric model, let's try to see what our goals are going to be.
OK?
So now we have a sample and a statistical model.
And we want to estimate the parameter theta, and I could say, well, you know what?
I don't have time for this analysis.
Collecting data is going to take me a while.
So I'm just going to mmm-- and I'm going to say that mu over sigma is 4.
And I'm just going to give it to you.
And maybe you will tell me, yeah, it's not very good, right?
So we need some measure of performance of a given parameter.
We need to be able to evaluate if eyeballing the problem is worse than actually collecting a large amount of theta.
We need to know if even if I come up with an estimator that actually sort of uses the data, does it make an efficient use of the data?
Would I actually need 10 times more observations to achieve the same accuracy?
To be able to answer these questions, well, I need to define what accuracy means.
And accuracy is something that sort of makes sense.
It says, well, I want theta.
I have to be close to theta.
And the theta is a random variable.
So I'm going to have to understand what it means for a random variable to be close to a deterministic number.
And so, what is a parameter estimator, right?
So I have an estimator, and I said it's a random variable.
And the formal definition-- so an estimator is a measurable function of the data.
So when I write theta hat, and that will typically be my notation for an estimator, right?
I should really write theta hat of x1 xn.
OK?
That's what an estimator is.
If you want to know an estimator is, this is a measurable function of the data.
And it's actually also known as a statistic.
And you know, if you're interested in, you know, I see every day I think when I have like, you know, a dinner with normal people.
And they say I'm a statistician.
Oh, yeah, I really like baseball.
And they talk to me about batting averages.
That's not what I do.
But for them, that's what it is, and that's because in a way, that's what a statistic is.
A batting average is a statistic.
OK, and so here are some examples.
You can take the average xn bar.
You can take the maximum of your observation.
That's the statistics.
You can take the first one.
You can take the first one plus log of 1 plus the absolute value of the last one.
You can do whatever you want that will be an estimator.
Some of them are clearly going to be bad.
But that's still a statistic, and you can do this.
Now, when I say measurable, I always have-- so you know, graduate students sometimes ask me like, yeah, how do I know if this estimator is measurable or not.
And usually, my answer is, well, if I give you data, can you compute it.
And they say, yeah, I'm like, well, then it's measurable.
That's a very good rule to check if you can actually-- if something is actually measurable.
When is this thing non-measurable?
It's when it's implicitly defined.
OK, and in particular, the things that give you problems are-- sup or inf.
Anybody knows what a sup or an inf is?
It's like a max or a min.
But it's not always attained.
OK, so if I have x1.
So if I look at the infimum of the function f of x for x on the real line and f of x, sorry, let's say x on the 1 infinity.
And f of x is equal to 1 over x.
Right?
Then the infimum is the smallest value we can take except that it doesn't really take it at 0 right, because 1 over x is going to 0.
But it's never really getting there.
So we just called the inf 0.
But it's not the value that it ever takes.
And these things might actually be complicated to compute.
And so that's when you actually have problems, right?
When the limit is not-- you're not really quite reaching the limit.
You won't have this problem in general, but just so you know, an estimator is not really anything.
It has to actually be measurable.
OK, so the first thing we want to know I mentioned it-- so an estimator is a statistic which does not depend on theta, of course.
So if I give you the data, you have to be able to compute it.
And that probably should not require not knowing any known parameters.
OK, so an estimator is said to be consistent.
When my data-- when I collect more and more data, this thing is getting closer and closer to the true parameter.
All right?
And we said that eyeballing and saying that it's going to be 4 is not really something that's probably going to be consistent.
But they can have things that are consistent but that are converging to theta at different speeds.
OK?
And we know also that this is a random variable.
It converges to something.
And there might be some different notions of convergence that kick in.
And actually there are.
And we say that it's weakly convergent if it converges in probability and strongly convergent if it converges almost [INAUDIBLE].. OK?
And this is just vocabulary.
It won't make a big difference.
OK?
So we will typically say it's consistent with any of the two
[INAUDIBLE]
Well, so in parametric statistics, it's actually a little difficult to come up with.
But in non-parametric ones, I could just say, if I had xi, yi, and I know that yi is f of xi plus noise s1i.
And I know that f belongs to some class of function, let's say-- [INAUDIBLE] class of smooth functions-- it's massive.
And now, I'm going to actually find the following estimator.
I'm going to take the average.
So I'm going to do least squares, right?
So I just check.
I'm trying to minimize the distance of each of my f of xi to my yi.
And now, I want to find the smallest of them.
So if I look at the infimum here, then the question is-- so that could be-- well, that's not really an estimator for f. But it's an estimator for the smallest possible value.
And so for example, this is actually an estimator for the variance of sigma square.
This might not be attained, and this might not be measurable if f is massive?
All right, so that's the infimum over some class f of x. OK?
So those are all voice things that are defined implicitly.
If it's an average, for example, it's completely measurable.
OK?
Any other question?
OK, so we know that the first thing we might want to check, and that's definitely something we want about estimators that is consistent, because all consistency tells us is that just as I collect more and more data, my estimator is going to get closer and closer to the parameter.
There's other things we can look at.
For each possible value of n-- now, right now, I have a finite number of observations-- 25.
And I want to know something about my estimator.
The first thing I want to check is maybe if in average, right?
So this is a random variable.
Is this random variable in average going to be close to theta or not?
And so the difference how far I am from theta is actually called the bias.
So the bias of an estimator is the expectation of theta hat minus the value that I hope it gets, which is theta.
If this thing is equal to 0, we say that theta hat is unbiased.
And unbiased estimators are things that people are looking for in general.
The problem is that there's lots of unbiased estimators.
And so it might be misleading to look for unbiasedness when that's not really the only thing you should be looking for.
OK, so what does it mean to be unbiased?
Maybe for this particular round of data you collected, you're actually pretty far from the true estimator.
But one thing that actually-- what it means is that if I redid this experiment over, and over, and over again, and I averaged all the values of my estimators that I got, then this would actually be the right-- the true parameter.
OK. That's what it means.
If I were to repeat this experiment, in average, I would actually get the right thing.
But you don't get to repeat the experiment.
OK, just a remark about estimators, look at this estimator-- xn bar.
Right?
Think of the kiss example.
I'm looking at the average of my observations.
And I want to know what the expectation of this thing is.
OK?
Now, this guy is by linearity of the expectation, it is this, right?
But my data is identically distributed.
So in particular, all the xi's have the same expectation, right?
Everybody agrees with this.
When it's identically distributed, they'll get the same expectation.
So what it means is that this guy's here-- they're all equal to the expectation of x1.
Right?
So what it means is that these guys-- I have the average of the same number.
So this is actually the expectation of x1.
OK?
And it's true.
In the kiss example, this was p. And this is p-- the probability of turning your head right.
OK?
So those two things are the same.
In particular, that means that xn bar and just x1 have the same bias.
So that should probably illustrate to you that bias is not something that really is telling you the entire picture, Right?
I can take only one of my observations-- Bernoulli 0, 1.
This thing will have the same bias as if I average 1,000 of them.
But the bias is really telling you where I am in average.
But it's really not telling me what fluctuations I'm getting.
And so if you want to start having fluctuations coming into the picture, we actually have to look at the risk or the quadratic risk of the estimator.
And so the quadratic risk is the finest-- the expectation of the square distance between theta hat and theta.
OK?
So let's look at this.
So the quadratic risk-- sometimes it's denoted that people call it the l2 risk of theta hat, of course.
I'm sorry for maintaining such an ugly board.
[INAUDIBLE] this stuff.
OK, so I look at the square distance between theta hat and theta.
This is still-- this is the function of a random variable.
So it's a random variable as well.
And now I'm looking at the expectation of this guy.
That's the definition.
I claimed that when this thing goes to 0, then my estimator is actually going to be consistent.
Everybody agrees with this?
So if it goes to zero as n goes to infinity-- and here, I don't need to tell you what kind of convergence I have, because this is just the number, right?
It's an expectation.
So it's a regular, usual calculus-style convergence.
Then that implies that theta hat is actually weakly consistent.
What did I use to tell you this?
Yeah, this is the convergence in l2.
This actually is strictly equivalent.
This is by definition saying that theta hat converges in l2 to theta.
And we know that convergence in l2 implies convergence in credibility to theta.
That was the picture.
We're going up.
And this is actually equivalent to a consistency by definition-- a weak consistency.
OK, so this is actually telling you a little more because this guy here-- they are both unbiased.
Theta xn bar is unbiased.
X1 is unbiased.
But x1 is certainly not consistent , because the more data I collect, I'm not even doing anything with it.
I'm just taking the first data point you're giving to me.
So they're both unbiased.
But this one is not consistent.
And this one we'll see is actually consistent.
xn bar is consistent.
And actually, we've seen that last time.
And that's because of the?
What guarantees the fact that xn bar is consistent?
The law of large numbers
The law of large numbers, right?
Actually, it's strongly consistent if you have a strong [INAUDIBLE].. OK, so just in the last two minutes, I want to tell you a little bit about how this risk is linked to see, quadratic risk is equal to bias squared plus the variance.
So let's see what I mean by this?
So I'm going to forget about the absolute values that you have a square.
I don't really need them.
If theta hat was unbiased, this thing would be the expectation of theta hat.
It might not be the case.
So let me see how I can actually see-- put the bias in there.
Well, one way to do this is to see that this is equal to the expectation of theta hat minus the expectation of theta hat, plus the expectation of theta hat minus theta.
OK?
I just removed the same and added the same thing.
So I didn't change anything.
Now, this guy is my bias, right?
So now let me expand the square.
So what I get is the expectation of the square of theta hat minus its expectation.
I should put some square brackets-- plus two times the cross-product.
So the cross-product is what expectation of theta hat minus the expectation of theta hat times expectation of theta hat minus theta.
And then I have the last square.
Expectation of theta hat minus theta squared.
OK?
So square, cross-products, square.
Everybody is with me?
now this guy here-- if you pay attention, this thing is the expectation of some random variable.
So it's a deterministic number.
Theta is the true parameter.
It's a deterministic number.
So what I can do is pull out this entire thing out of the expectation like this and compute the expectation only with respect to that part.
But what is the expectation of this thing?
It's zero, right?
The expectation of theta hat minus the expectation of theta hat is 0.
So this entire thing is equal 0.
So now when I actually collect back my quadratic terms-- my two squared terms in this expansion-- what I get is that the expectation of theta hat minus theta squared is equal to the expectation of theta hat minus expectation of theta hat squared plus the square of expectation of theta hat minus theta.
Right?
So those are just the two-- the first and the last term of the previous equality?
Now, here I have the expectation of the square of the difference between a random variable and its expectation.
This is otherwise known as the variance, right?
So this is actually equal to the variance of theta hat.
And well, this was the bias.
We already said that's there.
So this whole thing is the bias square.
OK?
And hence the quadratic term is the sum of the variance and the squared bias.
Why squared bias?
Well, because otherwise, you would add dollars in dollars squared.
So you need to add dollars squared and dollars squared so that this thing is actually homogeneous.
So if x is in dollars, then the bias is in dollars, but the variance is in dollars squared.
OK, and the square here forced you to put everything on the square scale.
All right, so what's nice is that if the quadratic risk goes to 0, then since I have the sum of two positive terms, both of them have to go to 0.
That means that my variance is going to 0-- very little fluctuations.
And my bias is also going to 0, which means that I'm actually going to be on target once I reduce my fluctuations, because it's one thing to reduce the fluctuations.
But if I'm not on target, it's an issue, right?
For example, the estimator for the value 4 has no variance.
Every time I'm going to repeat the experiments, I'm going to get 4, 4, 4, 4-- variance is 0.
But the bias is bad.
The bias is 4 minus theta.
And if theta is far from 4, that's not doing very well.
OK, so next week, we will-- we'll talk about what is a good estimate-- how estimators change if they have high variance or low variance or high bias and low bias.
And we'll talk about confidence intervals as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
--124.
If I were to repeat this 1,000 times, so every one of those 1,000 times they collect 124 data points and then I'd do it again and do it again and again, then in average, the number I should get should be close to the true parameter that I'm looking for.
The fluctuations that are due to the fact that I get different samples every time should somewhat vanish.
And so what I want is to have a small bias, hopefully a 0 bias.
If this thing is 0, then we see that the estimator is unbiased.
So this is definitely a property that we are going to be looking for in an estimator, trying to find them to be unbiased.
But we'll see that it's actually maybe not enough.
So unbiasedness should not be something you lose your sleep over.
Something that's slightly better is the risk, really the quadratics risk, which is expectation of-- so if I have an estimator, theta hat, I'm going to look at the expectation of theta hat n minus theta squared.
And what we showed last time is that we can actually-- by inserting in there, adding and removing the expectation of theta hat, we actually get something where this thing can be decomposed as the square of the bias plus the variance, which is just the expectation of theta hat minus its expectation squared.
That came from the fact that when I added and removed the expectation of theta hat in there, the cross-terms cancel.
All right.
So that was the bias squared, and this is the variance.
And so for example, if the quadratic risk goes to 0, then that means that theta hat converges to theta in the L2 sense.
And here we know that if we want this to go to 0, since it's the sum of two positive terms, we need to have both the bias that goes to 0 and the variance that goes to 0, so we need to control both of those things.
And so there is usually an inherent trade-off between getting a small bias and getting a small variance.
If you reduce one too much, then the variance of the other one is going to-- then the other one is going to increase, or the opposite.
That happens a lot, but not so much, actually, in this class.
So let's just look at a couple of examples.
So am I planning-- yeah.
So examples.
So if I do, for example, X1, Xn, there are iid Bernoulli.
And I'm going to write it theta so that we keep the same notation.
Then theta hat, what is the theta hat that we proposed many times?
It's just X bar, Xn bar, the average of Xi's.
So what is the bias of this guy?
Well, to know the bias, I just have to remove theta from the expectation.
What is the expectation of Xn bar?
Well, by linearity of the expectation, it's just the average of the expectations.
But since all my Xi's are Bernouilli with the same theta, then each of this guy is actually equal to theta.
So this thing is actually theta, which means that this isn't biased, right?
Now, what is the variance of this guy?
So if you forgot the properties of the variance for sum of independent random variables, now it's time to wake up.
So we have the variance of something that looks like 1 over n, the sum from i equal 1 to n of Xi.
So it's of the form variance of a constant times a random variable.
So the first thing I'm going to do is pull out the constant.
But we know that the variance leaves on the square scale, so when I pull out a constant outside of the variance, it comes out with a square.
The variance of a times X is a-squared times the variance of X, so this is equal to 1 over n squared times the variance of the sum.
So now we want to always do what we want to do.
So we have the variance of the sum.
We would like somehow to say that this is the sum of the variances.
And in general, we are not allowed to say that, but we are because my Xi's are actually independent.
So this is actually equal to 1 over n squared sum from i equal 1 to n of the variance of each of the Xi's.
And that's by independence, so this is basic probability.
And now, what is the variance of Xi's where again they're all the same distribution, so the variance of Xi is the same as the variance of X1.
And so each of those guys has variance what?
What is the variance of a Bernoulli?
We've said it once.
It's theta times 1 minus theta.
And so now I'm going to have the sum of n times a constant, so I get n times the constant divided by n squared, so one of the n's is going to cancel.
And so the whole thing here is actually equal to theta, 1 minus theta divided by n. So if I'm interested in the quadratic risk-- and again, I should just say risk, because this is the only risk we're going to be actually looking at.
Yeah.
This parenthesis should really stop here.
I really wanted to put quadratic in parenthesis.
So the risk of this guy is what?
Well, it's the expectation of x bar n minus theta squared.
And we know it's the square of the variance, so it's the square of the bias, which we know is 0, so it's 0 squared plus the variance, which is theta, 1 plus theta-- 1 minus theta divided by n. So it's just theta, 1 minus theta divided by n. So this is just summarizing the performance of an estimator, which is the random variable.
I mean, it's complicated.
If I really wanted to describe it, I would just tell you the entire distribution of this random variable.
But now what I'm doing is I'm saying, well, let's just take this random variable, remove theta from it, and see how small the fluctuations around theta-- the squared fluctuations around theta are in expectation.
So that's what the quadratic risk is doing.
And in a way, this decomposition, as the sum of the bias square and the variance, is really telling you that-- it is really accounting for the bias, which is, well, even if I had an infinite amount of observations, is this thing doing the right thing?
And the other thing is actually the variance, so for finite number of observations, what are the fluctuations?
All right.
Then you can see that those things, bias and variance, are actually very different.
So I don't have any colors here, so you're going to have to really follow the speed-- the order in which I draw those curves.
All right.
So let's find-- I'm going to give you three candidate estimators, so-- estimators for theta.
So the first one is definitely Xn bar.
That will be a good candidate estimator.
The second one is going to be 0.5, because after all, why should I bother if it's actually going to be-- right?
So for example, if I ask you to predict the score of some candidate in some election, then since you know it's going to be very close to 0.5, you might as well just throw 0.5 and you're not going to be very far from reality.
And it's actually going to cost you 0 time and $0 to come up with that.
So sometimes maybe just a good old guess is actually doing the job for you.
Of course, for presidential elections or something like this, it's not very helpful if your prediction is telling you this.
But if it was something different, that would be a good way to generate some close to 1/2.
For a coin, for example, if I give you a coin, you never know.
Maybe it's slightly biased.
But the good guess, just looking at it, inspecting it, maybe there's something crazy happening with the structure of it, you're going to guess that it's 0.5 without trying to collect information.
And let's find another one, which is, well, you know, I have a lot of observations.
But I'm recording couples kissing, but I'm on a budget.
I don't have time to travel all around the world and collect some people.
So really, I'm just going to look at the first couple and go home.
So my other estimator is just going to be X1.
I just take the first observation, 0, 1, and that's it.
So now I'm going-- I want to actually understand what the behavior of those guys is.
All right.
So we know-- and so we know that for this guy, the bias is 0 and the variance is equal to theta, 1 minus theta divided by n. What is the bias of this guy, 0.5?
0.5
0.5 minus theta?
0.5 minus theta, right.
So the bias, 0.5 minus theta.
What is the variance of this guy?
What is the variance of 0.5?
It's 0
0.
Right.
It's just a deterministic number, so there's no fluctuations for this guy.
What is the bias?
Well, X1 is actually-- just for simplicity, I can think of it as being X1 bar, the average of itself, so that wherever I saw an n for this guy, I can replace it by 1 and that will give me my formula.
So the bias is still going to be 0.
And the variance is going to be equal to theta, 1 minus theta.
So now I have those three estimators.
Well, if I compare X1 and Xn bar, then clearly I have 0 bias in both cases.
That's good.
And I have the variance that's actually n times smaller when I use my n observations than when I don't.
So those two guys, on these two fronts, you can actually look at the two numbers and say, well, the first number is the same.
The second number is better for the other guy, so I will definitely go for this guy compared to this guy.
So this guy is gone.
But not this guy.
Well, if I look at the bias, the variance is 0.
It's always beating the variance of this guy.
And if I look at the bias, it's actually really not that bad.
It's 0.5 minus theta.
In particular, if theta is 0.5, then this guy is strictly better.
And so you can actually now look at what the quadratic risk looks like.
So here, what I'm going to do is I'm going to take my true theta-- so it's going to range between 0 and 1.
And we know that those two things are functions of theta, so I can only understand them if I plot them as functions of theta.
And so now I'm going to actually plot-- the y-axis is going to be the risk.
So what is the risk of the estimator of 0.5?
This one is easy.
Well, it's 0 plus the square of 0.5 minus theta.
So we know that at theta, it's actually going to be 0.
And then it's going to be a square.
So at 0, it's going to be 0.25.
And at 1, it's going to be 0.25 as well.
So it looks like this.
Well, actually, sorry.
Let me put the 0.5 where it should be.
OK.
So this here is the risk of 0.5.
And we'll write it like this.
So when theta is very close to 0.5, I'm very happy.
When theta gets farther, it's a little bit annoying.
And then here, I want to plot the risk of this guy.
So now the thing with the risk of this guy is that it will depend on n. So I will just pick some n that I'm happy with just so that I can actually draw a curve.
Otherwise, I'm going to have to plot one curve per value of n. So let's just say, for example, that n is equal to 10.
And so now I need to plot the function theta, 1 minus theta divided by 10.
We know that theta, 1 minus theta is a curve that goes like this.
It takes value at 1/2.
It thinks value 1/4.
That's the maximum.
And then it's 0 at the end.
So really, if n is equal to 1, this is what the variance looks like.
The bias doesn't count in the risk.
Yeah
[INAUDIBLE]
Sure.
Can you move?
All right.
Are you guys good?
All right.
So now I have this picture.
And I know I'm going up to 25.
And there's a place where those curves cross.
So if you're sure-- let's say you're talking about presidential election, you know that those things are going to be really close.
Maybe you're actually better by predicting 0.5 if you know it's not going to go too far.
But that's for one observation, so that's the risk of X1.
But if I look at the risk of Xn, all I'm doing is just crushing this curve down to 0.
So as n increases, it's going to look more and more like this.
It's the same curve divided by n. And so now I can just start to understand that for different values of thetas, now I'm going to have to be very close to theta is equal to 1/2 if I want to start saying that Xn bar is worse than the naive estimator 0.5.
Yeah
Sorry.
I know you explained a little bit before, but can you just-- what is an intuitive definition of risk?
What is it actually describing?
So either you can-- well, when you have an unbiased estimator, it's simple.
It's just telling you it's the variance, because the theta that you have over there is really-- so in the definition of the risk, the theta that you have here if you're unbiased is really the expectation of theta hat.
So that's really just the variance.
So the risk is really telling you how much fluctuations I have around my expectation if unbiased.
But actually here, it's telling you how much fluctuations I have in average around theta.
So if you understand the notion of variance as being--
[INAUDIBLE]
What?
Like variance on average
No
No
It's just like variance
Oh,
So when you-- I mean, if you claim you understand what variance is, it's telling you what is the expected squared fluctuation around the expectation of my random variable.
It's just telling you on average how far I'm going to be.
And you take the square because you want to cancel the signs.
Otherwise, you're going to get 0
Oh,
And here it's saying, well, I really don't care what the expectation of theta hat is.
What I want to get to is theta, so I'm looking at the expectation of the squared fluctuations around theta itself.
If I'm unbiased, it coincides with the variance.
But if I'm biased, then I have to account for the fact that I'm really not computing the--
OK. OK.
Thanks
OK?
All right.
Are there any questions?
So here, what I really want to illustrate is that the risk itself is a function of theta most of the times.
And so for different thetas, some estimators are going to be better than others.
But there's also the entire range of estimators, those that are really biased, but the bias can completely vanish.
And so here, you see you have no bias, but the variance can be large.
Or you have 0 bias-- you have a bias, but the variance is 0.
So you can actually have this trade-off and you can find things that are in the entire range in general.
So those things are actually-- those trade-offs between bias and variance are usually much better illustrated if we're talking about multivariate parameters.
If I actually look at a parameter which is the mean of some multivariate Gaussian, so an entire vector, then the bias is going to-- I can make the bias bigger by, for example, forcing all the coordinates of my estimator to be the same.
So here, I'm going to get some bias, but the variance is actually going to be much better, because I get to average all the coordinates for this guy.
And so really, the bias/variance trade-off is when you have multiple parameters to estimate, so you have a vector of parameters, a multivariate parameter, the bias increases when you're trying to pull more information across the different components to actually have a lower variance.
So the more you average, the lower the variance.
That's exactly what we've illustrated.
As n increases, the variance decreases, like 1 over n or theta, 1 minus theta over n. And so this is how it happens in general.
In this class, it's mostly one-dimensional parameter estimation, so it's going to be a little harder to illustrate that.
But if you do, for example, non-parametric estimation, that's all you do.
There's just bias/variance trade-offs all the time.
And in between, when you have high-dimensional parametric estimation, that happens a lot as well.
OK.
So I'm just going to go quickly through those two remaining slides, because we've actually seen them.
But I just wanted you to have somewhere a formal definition of what a confidence interval is.
And so we fixed a statistical model for n observations, X1 to Xn.
The parameter theta here is one-dimensional.
Theta is a subset of the real line, and that's why I talk about intervals.
An interval is a subset of the line.
If I had a subset of R2, for example, that would no longer be called an interval, but a region, just because-- well, that's just we can say a set, a confidence set.
But people like to say confidence region.
So an interval is just a one-dimensional conference region.
And it has to be an interval as well.
So a confidence interval of level 1 minus alpha-- so we refer to the quality of a confidence interval is actually called it's level.
It takes value 1 minus alpha for some positive alpha.
And so the confidence level-- the level of the confidence interval is between 0 and 1.
The closer to 1 it is, the better the confidence interval.
The closer to 0, the worse it is.
And so for any random interval-- so a confidence interval is a random interval.
The bounds of this interval depends on random data.
Just like we had X bar plus/minus 1 over square root of n, for example, or 2 over square root of n, this X bar was the random thing that would make fluctuate those guys.
And so now I have an interval.
And now I have its boundaries, but now the boundaries are not allowed to depend on my unknown parameter.
Otherwise, it's not a confidence interval, just like an estimator that depends on the unknown parameter is not an estimator.
The confidence interval has to be something that I can compute once I collect data.
And so what I want is that-- so there's this weird notation.
The fact that I write theta-- that's the probability that I contains theta.
You're used to seeing theta belongs to I.
But here, I really want to emphasize that the randomness is in I.
And so the way you actually say it when you read this formula is the probability that I contains theta is at least 1 minus alpha.
So it better be close to 1.
You want 1 minus alpha to be very close to 1, because it's really telling you that whatever random variable I'm giving you, my error bars are actually covering the right theta.
And I want this to be true.
But I want this-- since I don't know what my confidence-- my parameter of theta is, I want this to hold true for all possible values of the parameters that nature may have come up with from.
So I want this-- so there's theta that changes here, so the distribution of the interval is actually changing with theta hopefully.
And theta is changing with this guy.
So regardless of the value of theta that I'm getting, I want that the probability that it contains the theta is actually larger than 1 minus alpha.
So I'll come back to it in a second.
I just want to say that here, we can talk about asymptotic level.
And that's typically when you use central limit theorem to compute this guy.
Then you're not guaranteed that the value is at least 1 minus alpha for every n, but it's actually in the limit larger than 1 minus alpha.
So maybe for each fixed n it's going to be not true.
But for as no goes to infinity, it's actually going to become true.
If you want this to hold for every n, you actually need to use things such as Hoeffding's inequality that we described at some point, that hold for every n. So as a rule of thumb, if you use the central limit theorem, you're dealing with a confidence interval with asymptotic level 1 minus alpha.
And the reason is because you actually want to get the quintiles of the normal-- the Gaussian distribution that comes from the central limit theorem.
And if you want to use Hoeffding's, for example, you might actually get away with a confidence interval that's actually true even non-asymptotically.
It's just the regular confidence interval.
So this is the formal definition.
It's a bit of a mouthful.
But we actually-- the best way to understand them is to build them.
Now, at some point I said-- and I think it was part of the homework-- so here, I really say the probability the true parameter belongs to the confidence interval is actually 1 minus alpha.
And so that's because here, this confidence interval is still a random variable.
Now, if I start plugging in numbers instead of the random variables X1 to Xn, I start putting 1, 0, 0, 1, 0, 0, 1, like I did for the kiss example, then in this case, the random interval is actually going to be 0.42, 0.65.
And this guy, the probability that theta belongs to it is not 1 minus alpha.
It's either 0 if it's not in there or it's 1 if it's in there.
So here is the example that we had.
So just let's look at back into our favorite example, which is the average of Bernoulli random variables, so we studied that maybe that's the third time already.
So the sample average, Xn bar, is a strongly consistent estimator of p. That was one of the properties that we wanted.
Strongly consistent means that as n goes to infinity, it converges almost surely to the true parameter.
That's the strong law of large number.
It is consistent also, because it's strongly consistent, so it also converges in probability, which makes it consistent.
It's unbiased.
We've seen that.
We've actually computed its quadratic risk.
And now what I have is that if I look at-- thanks to the central limit theorem, we actually did this.
We built a confidence interval at level 1 minus alpha-- asymptotic level, sorry, asymptotic level 1 minus alpha.
And so here, this is how we did it.
Let me just go through it again.
So we know from the central limit theorem-- so the central limit theorem tells us that Xn bar minus p divided by square root of p1 minus p, square root of n converges in distribution as n goes to infinity to some standard normal distribution.
So what it means is that if I look at the probability under the true p, that's square root of n, Xn bar minus p divided by square root of p1 minus p, it's less than Q alpha over 2, where this is the definition of the quintile.
Then this guy-- and I'm actually going to use the same notation, limit as n goes to infinity, this is the same thing.
So this is actually going to be equal to 1 minus alpha.
That's exactly what I did last time.
This is by definition of the quintile of a standard Gaussian and of a limit in distribution.
So the probabilities computed on this guy in the limit converges to the probability computed on this guy.
And we know that this is just the probability that the absolute value of sum n 0, 1 is less than Q alpha over 2.
And so in particular, if it's equal, then I can put some larger than or equal to, which guarantees my asymptotic confidence level.
And I just solve for p. So this is equivalent to the limit as n goes to infinity of the probability that theta is between Xn bar minus Q alpha over 2 divided by-- times square root of p1 minus p divided by square root of n, Xn bar plus q alpha over 2, square root of p1 minus p divided by square root of n is larger than or equal to 1 minus alpha.
And so there you go.
I have my confidence interval.
Except that's not, right?
We just said that the bounds of a confidence interval may not depend on the unknown parameter.
And here, they do.
And so we actually came up with two ways of getting rid of this.
Since we only need this thing-- so this thing, as we said, is really equal.
Every time I'm going to make this guy smaller and this guy larger, I'm only going to increase the probability.
And so what we do is we actually just take the largest possible value for p1 minus p, which makes the interval as large as possible.
And so now I have this.
I just do one of the two tricks.
I replace p1 minus p by their upper bound, which is 1/4.
As we said, p1 minus p, the function looks like this.
So I just take the value here at 1/2.
Or, I can use Slutsky and say that if I replace p by Xn bar, that's the same as just replacing p by Xn bar here.
And by Slutsky, we know that this is actually converging also to some standard Gaussian.
We've seen that when we saw Slutsky as an example.
And so those two things-- actually, just because I'm taking the limit and I'm only caring about the asymptotic confidence level, I can actually just plug in consistent quantities in there, such as Xn bar where I don't have a p. And that gives me another confidence interval.
All right.
So this by now, hopefully after doing it three times, you should really, really be comfortable with just creating this confidence interval.
We did it three times in class.
I think you probably did it another couple times in your homework.
So just make sure you're comfortable with this.
All right.
That's one of the basic things you would want to know.
Are there any questions?
Yes
So Slutsky holds for any single response set p. But Xn converges [INAUDIBLE]
So that's not Slutsky, right?
That's [INAUDIBLE]
So Slutsky tells you that if you-- Slutsky's about combining two types of convergence.
So Slutsky tells you that if you actually have one Xn that converges to X in distribution and Yn that converges to Y in probability, then you can actually multiply Xn and Yn and get that the limit in distribution is the product of X and Y, where X is now a constant.
And here we have the constant, which is 1.
But I did that already, right?
Using Slutsky to replace it for the-- to replace P by Xn bar, we've done that last time, maybe a couple of times ago, actually.
Yeah
So I guess these statements are [INAUDIBLE]
That's correct
So could we like figure out [INAUDIBLE] can we set a finite [INAUDIBLE]
So of course, the short answer is no.
So here's how you would go about thinking about which method is better.
So there's always the more conservative method.
The first one, the only thing you're losing is the rate of convergence of the central limit theorem.
So if n is large enough so that the central limit theorem approximation is very good, then that's all you're going to be losing.
Of course, the price you pay is that your confidence interval is wider than it would be if you were to use Slutsky for this particular problem, typically wider.
Actually, it is always wider, because Xn bar-- 1 minus Xn bar is always less than 1/4 as well.
And so that's the first thing you-- so Slutsky basically adds your relying on the central limit-- your relying on the asymptotics again.
Now of course, you don't want to be conservative, because you actually want to squeeze as much from your data as you can.
So it depends on how comfortable and how critical it is for you to put valid error bars.
If they're valid in the asymptotics, then maybe you're actually going to go with Slutsky so it actually gives you slightly narrower confidence intervals and so you feel like you're a little more-- you have a more precise answer.
Now, if you really need to be super-conservative, then you're actually going to go with the P1 minus P. Actually, if you need to be even more conservative, you are going to go with Hoeffding's so you don't even have to rely on the asymptotics level at all.
But then you're confidence interval becomes twice as wide and twice as wide and it becomes wider and wider as you go.
So depends on-- I mean, there's a lot of data in statistics which is gauging how critical it is for you to output valid error bounds or if they're really just here to be indicative of the precision of the estimator you gave from a more qualitative perspective
So the error there is [INAUDIBLE]??
Yeah.
So here, there's basically a bunch of errors.
There's one that's-- so there's a theorem called Berry-Esseen that quantifies how far this probability is from 1 minus alpha, but the constants are terrible.
So it's not very helpful, but it tells you as n grows how smaller this thing grows-- becomes smaller.
And then for Slutsky, again you're multiplying something that converges by something that fluctuates around 1, so you need to understand how this thing fluctuates.
Now, there's something that shows up.
Basically, what is the slope of the function 1 over square root of X1 minus X around the value you're interested in?
And so if this function is super-sharp, then small fluctuations of Xn bar around this expectation are going to lead to really high fluctuations of the function itself.
So if you're looking at-- if you have f of Xn bar and f around say the true P, if f is really sharp like that, then if you move a little bit here, then you're going to move really a lot on the y-axis.
So that's what the function here-- the function you're interested in is 1 over square root of X1 minus X.
So what does this function look like around the point where you think P is the true parameter?
Its derivative really is what matters.
OK?
Any other question.
OK.
So it's important, because now we're going to switch to the real let's do some hardcore computation type of things.
All right.
So in this chapter, we're going to talk about maximum likelihood estimation.
Who has already seen maximum likelihood estimation?
OK. And who knows what a convex function is?
OK.
So we'll do a little bit of reminders on those things.
So those things are when we do maximum likelihood estimation, likelihood is the function, so we need to maximize a function.
That's basically what we need to do.
And if I give you a function, you need to know how to maximize this function.
Sometimes, you have closed-form solutions.
You can take the derivative and set it equal to 0 and solve it.
But sometimes, you actually need to resort to algorithms to do that.
And there's an entire industry doing that.
And we'll briefly touch upon it, but this is definitely not the focus of this class.
OK.
So before diving directly into the definition of the likelihood and what is the definition of the maximum likelihood estimator, what I'm going to try to do is to give you an insight for what we're actually doing when we do maximum likelihood estimation.
So remember, we have a model on a sample space E and some candidate distributions P theta.
And really, your goal is to estimate a true theta star, the one that generated some data, X1 to Xn, in an iid fashion.
But this theta star is really a proxy for us to know that we actually understand the distribution itself.
The goal of knowing theta star is so that you can actually know what P theta star.
Otherwise, it has-- well, sometimes we said it has some meaning itself, but really you want to know what the distribution is.
And so your goal is to actually come up with the distribution-- hopefully that comes from the family P theta-- that's close to P theta star.
So in a way, what does it mean to have two distributions that are close?
It means that when you compute probabilities on one distribution, you should have the same probability on the other distribution pretty much.
So what we can do is say, well, now I have two candidate distributions.
So if theta hat leads to a candidate distribution P theta hat, and this is the true theta star, it leads to the true distribution P theta star according to which my data was drawn.
That's my candidate.
As a statistician, I'm supposed to come up with a good candidate, and this is the truth.
And what I want is that if you actually give me the distribution, then I want when I'm computing probabilities for this guy, I know what the probabilities for the other guys are.
And so really what I want is that if I compute a probability under theta hat of some interval a, b, it should be pretty close to the probability under theta star of a, b.
And more generally, if I want to take the union of two intervals, I want this to be true.
If I take just 1/2 lines, I want this to be true from 0 to infinity, for example, things like this.
I want this to be true for all of them at once.
And so what I do is that I write A for a probability event.
And I want that P hat of A is close to P star of A for any event A in the sample space.
Does that sound like a reasonable goal for a statistician?
So in particular, if I want those to be close, I want the absolute value of their difference to be close to 0.
And this turns out to be-- if I want this to hold for all possible A's, I have all possible events, so I'm going to actually maximize over these events.
And I'm going to look at the worst possible event on which theta hat can depart from theta star.
And so rather than defining it specifically for theta hat and theta star, I'm just going to say, well, if you give me two probability measures, P theta and P theta prime, I want to know how close they are.
Well, if I want to measure how close they are by how they can differ when I measure the probability of some event, I'm just looking at the absolute value of the difference of the probabilities and I'm just maximizing over the worst possible event that might actually make them differ.
Agreed?
That's a pretty strong notion.
So if the total variation between theta and theta prime is small, it means that for all possible A's that you give me, then P theta of A is going to be close to P theta prime of A, because if-- let's say I just found the bound on the total variation distance, which is 0.01.
All right.
So that means that this is going to be larger than the max over A of P theta minus P theta prime of A, which means that for any A-- actually, let me write P theta hat and P theta star, like we said, theta hat and theta star.
And so if I have a bound, say, on the total variation, which is 0.01, that means that P theta hat-- every time I compute a probability on P theta hat, it's basically in the interval P theta star of A, the one that I really wanted to compute, plus or minus 0.01.
This has nothing to do with confidence interval.
This is just telling me how far I am from the value of actually trying to compute.
And that's true for all A.
And that's key.
That's where this max comes into play.
It just says, I want this bound to hold for all possible A's at once.
So this is actually a very well-known distance between probability measures.
It's the total variation distance.
It's extremely central to probabilistic analysis.
And it essentially tells you that every time-- if two probability distributions are close, then it means that every time I compute a probability under P theta but I really actually have data from P theta prime, then the error is no larger than the total variation.
OK.
So this is maybe not the most convenient way of finding a distance.
I mean, how are you going-- in reality, how are you to compute this maximum over all possible events?
I mean, it's just crazy, right?
There's an infinite number of them.
It's much larger than the number of intervals, for example, so it's a bit annoying.
And so there's actually a way to compress it by just looking at the basically function distance or vector distance between probability mass functions or probability density functions.
So I'm going to start with the discrete version of the total variation.
So throughout this chapter, I will make the difference between discrete random variables and continuous random variables.
It really doesn't matter.
All it means is that when I talk about discrete, I will talk about probability mass functions.
And when I talk about continuous, I will talk about probability density functions.
When I talk about probability mass functions, I talk about sums.
When I talk about probability density functions, I talk about integrals.
But they're all the same thing, really.
So let's start with the probability mass function.
Everybody remembers what the probability mass function of a discrete random variable is.
This is the function that tells me for each possible value that it can take, the probability that it takes this value.
So the Probability Mass Function, PMF, is just the function for all x in the sample space tells me the probability that my random variable is equal to this little value.
And I will denote it by P sub theta of X.
So what I want is, of course, that the sum of the probabilities is 1.
And I want them to be non-negative.
Actually, typically we will assume that they are positive.
Otherwise, we can just remove this x from the sample space.
And so then I have the total variation distance, I mean, it's supposed to be the maximum overall sets of-- of subsets of E, such that the probability of A minus probability of theta prime of A-- it's complicated, but really there's this beautiful formula that tells me that if I look at the total variation between P theta and P theta prime, it's actually equal to just 1/2 of the sum for all X in E of the absolute difference between P theta X and P theta prime of X.
So that's something you can compute.
If I give you two probability mass functions, you can compute this immediately.
But if I give you just the densities and the original distribution, the original definition where you have to max over all possible events, it's not clear you're going to be able to do that very quickly.
So this is really the one you can work with.
But the other one is really telling you what it is doing for you.
It's controlling the difference of probabilities you can compute on any event.
But here, it's just telling you, well, if you do it for each simple event, it's little x.
It's actually simple.
Now, if we have continuous random variables-- so by the way, I didn't mention, but discrete means Bernoulli.
Binomial, but not only those that have finite support, like Bernoulli has support of size 2, binomial NP has support of size n-- there's n possible values it can take-- but also Poisson.
Poisson distribution can take an infinite number of values, all the positive integers, non-negative integers.
And so now we have also the continuous ones, such as Gaussian, exponential.
And what characterizes those guys is that they have a probability density.
So the density, remember the way I use my density is when I want to compute the probability of belonging to some event A.
The probability of X falling to some subset of the real line A is simply the integral of the density on this set.
That's the famous area under the curve thing.
So since for each possible value, the probability at X-- so I hope you remember that stuff.
That's just probably something that you must remember from probability.
But essentially, we know that the probability that X is equal to little x is 0 for a continuous random variable, for all possible X's.
There's just none of them that actually gets weight.
So what we have to do is to describe the fact that it's in some little region.
So the probability that it's in some interval, say, a, b, this is the integral between A and B of f theta of X, dx.
So I have this density, such as the Gaussian one.
And the probability that I belong to the interval a, b is just the area under the curve between A and B.
If you don't remember that, please take immediate remedy.
So this function f, just like P, is non-negative.
And rather than summing to 1, it integrates to 1 when I integrate it over the entire sample space E. And now the total variation, well, it takes basically the same form.
I said that you essentially replace sums by integrals when you're dealing with densities.
And here, it's just saying, rather than having 1/2 of the sum of the absolute values, you have 1/2 of the integral of the absolute value of the difference.
Again, if I give you two densities and if you're not too bad at calculus, which you will often be, because there's lots of them you can actually not compute.
But if I gave you, for example, two Gaussian densities, exponential minus x squared, blah, blah, blah, and I say, just compute the total variation distance, you could actually write it as an integral.
Now, whether you can actually reduce this integral to some particular number is another story.
But you could technically do it.
So now, you have actually a handle on this thing and you could technically ask Mathematica, whereas asking Mathematica to take the max over all possible events is going to be difficult.
All right.
So the total variation has some properties.
So let's keep on the board the definition that involves, say, the densities.
So think Gaussian in your mind.
And you have two Gaussians, one with mean theta and one with mean theta prime.
And I'm looking at the total variation between those two guys.
So if I look at P theta minus-- sorry.
TV between P theta and P theta prime, this is equal to 1/2 of the integral between f theta, f theta prime.
And when I don't write it-- so I don't write the X, dx but it's there.
And then I integrate over E. So what is this thing doing for me?
It's just saying, well, if I have-- so think of two Gaussians.
For example, I have one that's here and one that's here.
So this is let's say f theta, f theta prime.
This guy is doing what?
It's computing the absolute value of the difference between f and f theta prime.
You can check for yourself that graphically, this I can represent as an area not under the curve, but between the curves.
So this is this guy.
Now, this guy is really the integral of the absolute value.
So this thing here, this area, this is 2 times the total variation.
The scaling 1/2 really doesn't matter.
It's just if I want to have an actual correspondence between the maximum and the other guy, I have to do this.
So this is what it looks like.
So we have this definition.
And so we have a couple of properties that come into this.
The first one is that it's symmetric.
TV of P theta and P theta prime is the same as the TV between P theta prime and P theta.
Well, that's pretty obvious from this definition.
I just flip those two, I get the same number.
It's actually also true if I take the maximum.
Those things are completely symmetric in theta and theta prime.
You can just flip them.
It's non-negative.
Is that clear to everyone that this thing is non-negative?
I integrate an absolute value, so this thing is going to give me some non-negative number.
And so if I integrate this non-negative number, it's going to be a non-negative number.
The fact also that it's an area tells me that it's going to be non-negative.
The nice thing is that if TV is equal to zero, then the two distributions, the two probabilities are the same.
That means that for every A, P theta of A is equal to P theta prime of A.
Now, there's two ways to see that.
The first one is to say that if this integral is equal to 0, that means that for almost all X, f theta is equal to f theta prime.
The only way I can integrate a non-negative and get 0 is that it's 0 pretty much everywhere.
And so what it means is that the two densities have to be the same pretty much everywhere, which means that the distributions are the same.
But this is not really the way you want to do this, because you have to understand what pretty much everywhere means-- which I should really say almost everywhere.
That's the formal way of saying it.
But let's go to this definition-- which is gone.
Yeah.
That's the one here.
The max of those two guys, if this maximum is equal to 0-- I have a maximum of non-negative numbers, their absolute values.
Their maximum is equal to 0, well, they better be all equal to 0, because if one is not equal to 0, then the maximum is not equal to 0.
So those two guys, for those two things to be-- for the maximum to be equal to 0, then each of the individual absolute values have to be equal to 0, which means that the probability here is equal to this probability here for every event A.
So those two things-- this is nice, right?
That's called definiteness.
The total variation equal to 0 implies that P theta is equal to P theta prime.
So that's really some notion of distance, right?
That's what we want.
If this thing being small implied that P theta could be all over the place compared to P theta prime, that would not help very much.
Now, there's also the triangle inequality that follows immediately from the triangle inequality inside this guy.
If I squeeze in some f theta prime prime in there, I'm going to use the triangle inequality and get the triangle inequality for the whole thing.
Yeah?
The fact that you need two definitions of the [INAUDIBLE],, is it something obvious or is it complete?
I'll do it for you now.
So let's just prove that those two things are actually giving me the same definition.
So what I'm going to do is I'm actually going to start with the second one.
And I'm going to write-- I'm going to start with the density version.
But as an exercise, you can do it for the PMF version if you prefer.
So I'm going to start with the fact that f-- so I'm going to write f of g so I don't have to write f and g. So think of this as being f sub theta, and think of this guy as being f sub theta prime.
I just don't want to have to write indices all the time.
So I'm going to start with this thing, the integral of f of X minus g of X dx.
The first thing I'm going to do is this is an absolute value, so either the number in the absolute value is positive and I actually kept it like that, or it's negative and I flipped its sign.
So let's just split between those two cases.
So this thing is equal to 1/2 the integral of-- so let me actually write the set A star as being the set of X's such that f of X is larger than g of X.
So that's the set on which the difference is going to be positive or the difference is going to be negative.
So this, again, is equivalent to f of X minus g of X is positive.
OK. Everybody agrees?
So this is the set I'm interested in.
So now I'm going to split my integral into two parts, in A, A star, so on A star, f is larger than g, so the absolute value is just the difference itself.
So here I put parenthesis rather than absolute value.
And then I have plus 1/2 of the integral on the complement.
What are you guys used to to write the complement, to the C or the bar?
To the C?
And so here on the complement, then f is less than g, so this is actually really g of X minus f of X, dx.
Everybody's with me here?
So I just said-- I mean, those are just rewriting what the definition of the absolute value is.
OK.
So now there's nice things that I know about f and g. And the two nice things is that the integral of f is equal to 1 and the integral of g is equal to 1.
This implies that the integral of f minus g is equal to what?
0
0.
And so now that means that if I want to just go from the integral here on A complement to the integral on A-- or on A star, complement to the integral of A star, I just have to flip the sign.
So that implies that an integral on A star complement of g of X minus f of X, dx, this is simply equal to the integral on A star of f of X minus g of X, dx.
All right.
So now this guy becomes this guy over there.
So I have 1/2 of this plus 1/2 of the same guy, so that means that 1/2 half of the integral between of f minus g absolute value-- so that was my original definition, this thing is actually equal to the integral on A star of f of X minus g of X, dx.
And this is simply equal to P of A star-- so say Pf of A start minus Pg of A star.
Which one is larger than the other one?
[INAUDIBLE]
It is.
Just look at this board
[INAUDIBLE]
What?
[INAUDIBLE]
The first one has to be larger, because this thing is actually equal to a non-negative number.
So now I have this absolute value of two things, and so I'm closer to the actual definition.
But I still need to show you that this thing is the maximum value.
So this is definitely at most the maximum over A of Pf of A minus Pg of A.
That's certainly true.
Right?
We agree with this?
Because this is just for one specific A, and I'm bounding it by the maximum over all possible A.
So that's clearly true.
So now I have to go the other way around.
I have to show you that the max is actually this guy, A star.
So why would that be true?
Well, let's just inspect this thing over there.
So we want to show that if I take any other A in this integral than this guy A star, it's actually got to decrease its value.
So we have this function.
I'm going to call this function delta.
And what we have is-- so let's say this function looks like this.
Now it's the difference between two densities.
It doesn't have to integrate-- it doesn't have to be non-negative.
But it certainly has to integrate to 0.
And so now I take this thing.
And the A star, what is the set A star here?
The set A star is the set over which the function delta is non-negative.
So that's just the definition.
A star was the set over which f minus g was positive, and f minus g was just called delta.
So what it means is that what I'm really integrating is delta on this set.
So it's this area under the curve, just on the positive things.
Agreed?
So now let's just make some tiny variations around this guy.
If I take A to be larger than A star-- so let me add, for example, this part here.
That means that when I compute my integral, I'm removing this area under the curve.
It's negative.
The integral here is negative.
So if I start adding something to A, the value goes lower.
If I start removing something from A, like say this guy, I'm actually removing this value from the integral.
So there's no way.
I'm actually stuck.
This A star is the one that actually maximizes the integral of this function.
So we used the fact that for any function, say delta, the integral over A of delta is less than the integral over the set of X's such that delta of X is non-negative of delta of X, dx.
And that's an obvious fact, just by picture, say.
And that's true for all A. Yeah?
[INAUDIBLE] could you use like a portion under the axis as like less than or equal to the portion above the axis?
It's actually equal.
We know that the integral of f minus g-- the integral of delta is 0.
So there's actually exactly the same area above and below.
But yeah, you're right.
You could go to the extreme cases.
You're right.
No.
It's actually still be true, even if there was-- if this was a constant, that would still be true.
Here, I never use the fact that the integral is equal to 0.
I could shift this function by 1 so that the integral of delta is equal to 1, and it would still be true that it's maximized when I take A to be the set where it's positive.
Just need to make sure that there is someplace where it is, but that's about it.
Of course, we used this before, when we made this thing.
But just the last argument, this last fact does not require that.
All right.
So now we have this notion of-- I need the-- OK.
So we have this notion of distance between probability measures.
I mean, these things are exactly what-- if I were to be in a formal math class and I said, here are the axioms that a distance should satisfy, those are exactly those things.
If it's not satisfying this thing, it's called pseudo-distance or quasi-distance or just metric or nothing at all, honestly.
So it's a distance.
It's symmetric, non-negative, equal to 0, if and only if the two arguments are equal, then it satisfies the triangle inequality.
And so that means that we have this actual total variation distance between probability distributions.
And here is now a statistical strategy to implement our goal.
Remember, our goal was to spit out a theta hat, which was close such that P theta hat was close to P theta star.
So hopefully, we were trying to minimize the total variation distance between P theta hat and P theta star.
Now, we cannot do that, because just by this fact, this slide, if we wanted to do that directly, we would just take-- well, let's take theta hat equals theta star and that will give me the value 0.
And that's the minimum possible value we can take.
The problem is that we don't know what the total variation is to something that we don't know.
We know how to compute total variations if I give you the two arguments.
But here, one of the arguments is not known.
P theta star is not known to us, so we need to estimate it.
And so here is the strategy.
Just build an estimator of the total variation distance between P theta and P theta star for all candidate theta, all possible theta in capital theta.
Now, if this is a good estimate, then when I minimize it, I should get something that's close to P theta star.
So here's the strategy.
This is my function that maps theta to the total variation between P theta and P theta star.
I know it's minimized at theta star.
That's definitely TV of P-- and the value here, the y-axis should say 0.
And so I don't know this guy, so I'm going to estimate it by some estimator that comes from my data.
Hopefully, the more data I have, the better this estimator is.
And I'm going to try to minimize this estimator now.
And if the two things are close, then the minima should be close.
That's a pretty good estimation strategy.
The problem is that it's very unclear how you would build this estimator of TV, of the Total Variation.
So building estimators, as I said, typically consists in replacing expectations by averages.
But there's no simple way of expressing the total variation distance as the expectations with respect to theta star of anything.
So what we're going to do is we're going to move from total variation distance to another notion of distance that sort of has the same properties and the same feeling and the same motivations as the total variation distance.
But for this guy, we will be able to build an estimate for it, because it's actually going to be of the form expectation of something.
And we're going to be able to replace the expectation by an average and then minimize this average.
So this surrogate for total variation distance is actually called the Kullback-Leibler divergence.
And why we call it divergence is because it's actually not a distance.
It's not going to be symmetric to start with.
So this Kullback-Leibler or even KL divergence-- I will just refer to it as KL-- is actually just more convenient.
But it has some roots coming from information theory, which I will not delve into.
But if any of you is actually a Core 6 student, I'm sure you've seen that in some-- I don't know-- course that has any content on information theory.
All right.
So the KL divergence between two probability measures, P theta and P theta prime-- and here, as I said, it's not going to be the symmetric, so it's very important for you to specify which order you say it is, between P theta and P theta prime.
It's different from saying between P theta prime and P theta.
And so we denote it by KL.
And so remember, before we had either the sum or the integral of 1/2 of the distance-- absolute value of the distance between the PMFs and 1/2 of the absolute values of the distances between the probability density functions.
And then we replace this absolute value of the distance divided by 2 by this weird function.
This function is P theta, log P theta, divided by P theta prime.
That's the function.
That's a weird function.
OK.
So this was what we had.
That's the TV.
And the KL, if I use the same notation, f and g, is integral of f of X, log of f of X over g of X, dx.
It's a bit different.
And I go from discrete to continuous using an integral.
Everybody can read this.
Everybody's fine with this.
Is there any uncertainty about the actual definition here?
So here I go straight to the definition, which is just plugging the functions into some integral and compute.
So I don't bother with maxima or anything.
I mean, there is something like that, but it's certainly not as natural as the total variation.
Yes?
The total variation, [INAUDIBLE]
Yes, just because it's hard to build anything from total variation, because I don't know it.
So it's very difficult.
But if you can actually-- and even computing it between two Gaussians, just try it for yourself.
And please stop doing it after at most six minutes, because you won't be able to do it.
And so it's just very hard to manipulate, like this integral of absolute values of differences between probability density function, at least for the probability density functions we're used to manipulate is actually a nightmare.
And so people prefer KL, because for the Gaussian, this is going to be theta minus theta prime squared.
And then we're going to be happy.
And so those things are much easier to manipulate.
But it's really-- the total variation is telling you how far in the worst case the two probabilities can be.
This is really the intrinsic notion of closeness between probabilities.
So that's really the one-- if we could, that's the one we would go after.
Sometimes people will compute them numerically, so that they can say, oh, here's the total variation distance I have between those two things.
And then you actually know that that means they are close, because the absolute value-- if I tell you total variation is 0.01, like we did here, it has a very specific meaning.
If I tell you the KL divergence is 0.01, it's not clear what it means.
OK.
So what are the properties?
The KL divergence between P theta and P theta prime is different from the KL divergence between P theta prime and P theta in general.
Of course, in general, because if theta is equal to theta prime, then this certainly is true.
So there's cases when it's not true.
The KL divergence is non-negative.
Who knows the Jensen's inequality here?
That should be a subset of the people who raised their hand when I asked what a convex function is.
All right.
So you know what Jensen's inequality is.
This is Jensen's-- the proof is just one step Jensen's inequality, which we will not go into details.
But that's basically an inequality involving expectation of a convex function of a random variable compared to the convex function of the expectation of a random variable.
If you know Jensen, have fun and prove it.
What's really nice is that if the KL is equal to 0, then the two distributions are the same.
And that's something we're looking for.
Everything else we're happy to throw out.
And actually, if you pay attention, we're actually really throwing out everything else.
So they're not symmetric.
It does satisfy the triangle inequality in general.
But it's non-negative and it's 0 if and only if the two distributions are the same.
And that's all we care about.
And that's what we call a divergence rather than a distance, and divergence will be enough for our purposes.
And actually, this asymmetry, the fact that it's not flipping-- the first time I saw it, I was just annoyed.
I was like, can we just like, I don't know, take the average of the KL between P theta and P theta prime and P theta prime and P theta, you would think maybe you could do this.
You just symmatrize it by just taking the average of the two possible values it can take.
The problem is that this will still not satisfy the triangle inequality.
And there's no way basically to turn it into something that is a distance.
But the divergence is doing a pretty good thing for us.
And this is what will allow us to estimate it and basically overcome what we could not do with the total variation.
So the first thing that you want to notice is the total variation distance-- the KL divergence, sorry, is actually an expectation of something.
Look at what it is here.
It's the integral of some function against a density.
That's exactly the definition of an expectation, right?
So this is the expectation of this particular function with respect to this density f. So in particular, if I call this is density f-- if I say, I want the true distribution to be the first argument, this is an expectation with respect to the true distribution from which my data is actually drawn of the log of this ratio.
So ha ha.
I'm a statistician.
Now I have an expectation.
I can replace it by an average, because I have data from this distribution.
And I could actually replace the expectation by an average and try to minimize here.
The problem is that-- actually the star here should be in front of the theta, not of the P, right?
That's P theta star, not P star theta.
But here, I still cannot compute it, because I have this P theta star that shows up.
I don't know what it is.
And that's now where the log plays a role.
If you actually pay attention, I said you can use Jensen to prove all this stuff.
You could actually replace the log by any concave function.
That would be f divergent.
That's called an f divergence.
But the log itself is a very, very specific property, which allows us to say that the log of the ratio is the ratio of the log.
Now, this thing here does not depend on theta.
If I think of this KL divergence as a function of theta, then the first part is actually a constant.
If I change theta, this thing is never going to change.
It depends only on theta star.
So if I look at this function KL-- so if I look at the function, theta maps to KL P theta star, P theta, it's of the form expectation with respect to theta star, log of P theta star of X.
And then I have minus expectation with respect to theta star of log of P theta of x.
Now as I said, this thing here, this second expectation is a function of theta.
When theta changes, this thing is going to change.
And that's a good thing.
We want something that reflects how close theta and theta star are.
But this thing is not going to change.
This is a fixed value.
Actually, it's the negative entropy of P theta star.
And if you've heard of KL, you've probably heard of entropy.
And that's what-- it's basically minus the entropy.
And that's a quantity that just depends on theta star.
But it's just the number.
I could compute this number if I told you this is n theta star 1.
You could compute this.
So now I'm going to try to minimize the estimate of this function.
And minimizing a function or a function plus a constant is the same thing.
I'm just shifting the function here or here, but it's the same minimizer.
OK.
So the function that maps theta to KL of P theta star to P theta is of the form constant minus this expectation of a log of P theta.
Everybody agrees?
Are there any questions about this?
Are there any remarks, including I have no idea what's happening right now?
OK. We're good?
Yeah
So when you're actually employing this method, how do you know which theta to use as theta star and which isn't?
So this is not a method just yet, right?
I'm just describing to you what the KL divergence between two distributions is.
If you really wanted to compute it, you would need to know what P theta star is and what P theta is
Right
And so here, I'm just saying at some point, we still-- so here, you see-- so now let's move onto one step.
I don't know expectation of theta star.
But I have data that comes from distribution P theta star.
So the expectation by the law of large numbers should be close to the average.
And so what I'm doing is I'm replacing any-- I can actually-- this is a very standard estimation method.
You write something as an expectation with respect to the data-generating process of some function.
And then you replace this by the average of this function.
And the law of large numbers tells me that those two quantities should actually be close.
Now, it doesn't mean that's going to be the end of the day, right.
When we did Xn bar, that was the end of the day.
We had an expectation.
We replaced it by an average.
And then we were gone.
But here, we still have to do something, because this is not telling me what theta is.
Now I still have to minimize this average.
So this is now my candidate estimator for KL, KL hat.
And that's the one where I said, well, it's going to be of the form of constant.
And this constant, I don't know.
You're right.
I have no idea what this constant is.
It depends on P theta star.
But then I have minus something that I can completely compute.
If you give me data and theta, I can compute this entire thing.
And now what I claim is that the minimizer of f or f plus-- f of X or f of X plus 4 are the same thing, or say 4 plus f of X. I'm just shifting the plot of my function up and down, but the minimizer stays exactly where it is.
If I have a function-- so now I have a function of theta.
This is KL hat of P theta star, P theta.
And it's of the form-- it's a function like this.
I don't know where this function is.
It might very well be this function or this function.
Every time it's a translation on the y-axis of all these guys.
And the value that I translated by depends on theta star.
I don't know what it is.
But what I claim is that the minimizer is always this guy, regardless of what the value is.
OK?
So when I say constant, it's a constant with respect to theta.
It's an unknown constant.
But it's with respect to theta, so without loss of generality, I can assume that this constant is 0 for my purposes, or 25 if you prefer.
All right.
So we'll just keep going on this property next time.
And we'll see how from here we can move on to-- the likelihood is actually going to come out of this formula.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I'm using a few things here, right?
I'm using the fact that KL is non-negative.
But KL is equal to 0 when I take twice the same argument.
So I know that this function is always non-negative.
So that's theta and that's KL P theta star P theta.
And I know that at theta star, it's equal to 0.
OK?
I could be in the case where I have this happening.
I have two-- let's call it theta star prime.
I have two minimizers.
That could be the case, right?
I'm not saying that-- so K of L-- KL is 0 at the minimum.
That doesn't mean that I have a unit minimum, right?
But it does, actually.
What do I need to use to make sure that I have only one minimum?
So the definiteness is guaranteeing to me that there's a unique P theta star that minimizes it.
But then I need to make sure that there's a unique-- from this unique P theta star, I need to make sure there's a unique theta star that defines this P theta star.
Exactly.
All right, so I combine definiteness and identifiability to make sure that there is a unique minimizer, in this case cannot exist.
OK, so basically, let me write what I just said.
So definiteness, that implies that P theta star is the unique minimizer of P theta maps to KL P theta star P theta.
So definiteness only guarantees that the probability distribution is uniquely identified.
And identifiability implies that theta star is the unique minimizer of theta maps to KL P theta star P theta, OK?
So I'm basically doing the composition of two injective functions.
The first one is the one that maps, say, theta to P theta.
And the second one is the one that maps P theta to the set of minimizers, OK?
So at least morally, you should agree that theta star is the minimizer of this thing.
Whether it's unique or not, you should agree that it's a good one.
So maybe you can think a little longer on this.
So thinking about this being the minimizer, then it says, well, if I actually had a good estimate for this function, I would use the strategy that I described for the total variation, which is, well, I don't know what this function looks like.
It depends on theta star.
But maybe I can find an estimator of this function that fluctuates around this function, and such that when I minimize this estimator of the function, I'm actually not too far, OK?
And this is exactly what drives me to do this, because I can actually construct an estimator.
I can actually construct an estimator such that this estimator is actually-- of the KL is actually close to the KL, all right?
So I define KL hat.
So all we did is just replacing expectation with respect to theta star by averages.
That's what we did.
So if you're a little puzzled by this error, that's all it says.
Replace this guy by this guy.
It has no mathematical meaning.
It just means just replace it by.
And now that actually tells me how to get my estimator.
It just says, well, my estimator, KL hat, is equal to some constant which I don't know.
I mean, it certainly depends on theta star, but I won't care about it when I'm trying to minimize-- minus 1/n sum from i from 1 to n log f theta of x.
So here I'm reading it with the density.
You have it with the PMF on the slides, and so you have the two versions in front of you, OK?
Oh sorry, I forgot the xi.
Now clearly, this function I know how to compute.
If you give me a theta, since I know the form of the density f theta, for each theta that you give me, I can actually compute this quantity, right?
This I don't know, but I don't care.
Because I'm just shifting the value of the function I'm trying to minimize.
The set of minimizers is not going to change.
So now, this is my estimation strategy.
Minimize in theta KL hat P theta star P theta, OK?
So now let's just make sure that we all agree that-- so what we want is the argument of the minimum, right?
arg min means the theta that minimizes this guy, rather than finding the value of the min.
OK, so I'm trying to find the arg min of this thing.
Well, this is equivalent to finding the arg min of, say, a constant minus 1/n sum from i from 1 to n of log f theta of xi.
So that's just-- I don't think it likes me.
No.
OK, so thus minimizing this average, right?
I just plugged in the definition of KL hat.
Now, I claim that taking the arg min of a constant plus a function or the arg min of the function is the same thing.
Is anybody not comfortable with this idea?
OK, so this is the same.
By the way, this I should probably switch to the next slide, because I'm writing the same thing, but better.
And it's with PMF rather than as PF.
OK, now, arg min of the minimum is the same of arg max-- sorry, arg min of the negative thing is the same as arg max without the negative, right?
arg max over theta of 1/n from i equal equal 1 to n log f theta of xi.
Taking the arg min of the average or the arg min of the sum, again, it's not going to make much difference.
Just adding constants OR multiplying by constants does not change the arg min or the arg max.
Now, I have the sum of logs, which is the log of the product.
OK?
It's the arg max of the log of f theta of x1 times f theta of x2, f theta of xn.
But the log is a function that's increasing, so maximizing log of a function or maximizing the function itself is the same thing.
The value is going to change, but the arg max is not going to change.
Everybody agrees with this?
So this is equivalent to arg max over theta of pi from 1 to n of f theta xi.
And that's because x maps to log x is increasing.
So now I've gone from minimizing the KL to minimizing the estimate of the KL to maximizing this product.
Well, this chapter is called maximum likelihood estimation.
The maximum comes from the fact that our original idea was to minimize the negative of a function.
So that's why it's maximum likelihood.
And this function here is called the likelihood.
This function is really just telling me-- they call it likelihood because it's some measure of how likely it is that theta was the parameter that generated the data.
OK, so let's go to the-- well, we'll go to the formal definition in a second.
But actually, let me just give you intuition as to why this is the distribution of the data.
Why this is the likelihood-- sorry.
Why is this making sense as a measure of likelihood?
Let's now think for simplicity of the following model.
So I have-- I'm on the real line and I look at n, say, theta 1 for theta in the real-- do you see that?
OK.
Probably you don't.
Not that you care.
OK, so-- OK, let's look at a simple example.
So here's the model.
As I said, we're looking at observations on the real line.
And they're distributed according to some n theta 1.
So I don't care about the variance.
I know it's 1.
And it's indexed by theta in the real line.
OK, so this is-- the only thing I need to figure out is, what is the mean of those guys, OK?
Now, I have this n observations.
And if you actually remember from your probability class, are you familiar with the concept of joint density?
You have multivariate observations.
The joint density of independent random variables is just a product of their individual densities.
So really, when I look at the product from i equal 1 to n of f theta of xi, this is really the joint density of the vector-- well, let me not use the word vector-- of x1 xn, OK?
So if I take the product of density, is it still a density?
And it's actually-- but this time on the r to the n. And so now what this thing is telling me-- so think of it in r2, right?
So this is the joint density of two Gaussians.
So it's something that looks like some bell-shaped curve in two dimensions.
And it's centered at the value theta theta.
OK, they both have the mean theta.
So let's assume for one second-- it's going to be hard for me to make pictures in n dimensions.
Actually, already in two dimensions, I can promise you that it's not very easy.
So I'm actually just going to assume that n is equal to 1 for the sake of illustration.
OK, so now I have this data.
And now I have one observation, OK?
And I know that the f theta looks like this.
And what I'm doing is I'm actually looking at the value of x theta as my observation.
Let's call it x1.
Now, my principal tells me, just find the theta that makes this guy the most likely.
What is the likelihood of my x1?
Well, it's just the value of the function.
That this value here.
And if I wanted to find the most likely theta that had generated this x1, what I would need to do is to shift this thing and put it here.
And so my estimate, my maximum likelihood estimator here would be theta is equal to x1, OK?
That would be just the observation.
Because if I have only one observation, what else am I going to do?
OK, and so it sort of makes sense.
And if you have more observations, you can think of it this way, as if you had more observations.
So now I have, say, K observations, or n observations.
And what I do is that I look at the value for each of these guys.
So this value, this value, this value, this value.
I take their product and I make this thing large.
OK, why do I take the product?
Well, because I'm trying to maximize their value all together, and I need to just turn it into one number that I can maximize.
And taking the product is the natural way of doing it, either by motivating it by the KL principle or motivating it by maximizing the joint density, rather than just maximizing anything.
OK, so that's why, visually, this is the maximum likelihood.
It just says that if my observations are here, then this guy, this mean theta, is more likely than this guy.
Because now if I look at the value of the function for this guy-- if I look at theta being this thing, then this is a very small value.
Very small value, very small value, very small value.
Everything gets a super small value, right?
That's just the value that it gets in the tail here, which is very close to 0.
But as soon as I start covering all my points with my bell-shaped curve, then all the values go up.
All right, so I just want to make a short break into statistics, and just make sure that the maximum likelihood principle involves maximizing a function.
So I just want to make sure that we're all on par about how do we maximize functions.
In most instances, it's going to be a one-dimensional function, because theta is going to be a one-dimensional parameter.
Like here it's the real line.
So it's going to be easy.
In some cases, it may be a multivariate function and it might be more complicated.
OK, so let's just make this interlude.
So the first thing I want you to notice is that if you open any book on what's called optimization, which basically is the science behind optimizing functions, you will talk mostly-- I mean, I'd say 99.9% of the cases will talk about minimizing functions.
But it doesn't matter, because you can just flip the function and you just put a minus sign, and minimizing h is the same as maximizing minus h and the opposite, OK?
So for this class, since we're only going to talk about maximum likelihood estimation, we will talk about maximizing functions.
But don't be lost if you decide suddenly to open a book on optimization and find only something about minimizing functions.
OK, so maximizing an arbitrary function can actually be fairly difficult.
If I give you a function that has this weird shape, right-- let's think of this polynomial for example-- and I wanted to find the maximum, how would we do it?
So what is the thing you've learned in calculus on how to maximize the function?
Set the derivative equal to 0.
Maybe you want to check the second derivative to make sure it's a maximum and not a minimum.
But the thing is, this is only guaranteeing to you that you have a local one, right?
So if I do it for this function, for example, then this guy is going to satisfy this criterion, this guy is going to satisfy this criterion, this guy is going to satisfy this criterion, this guy here, and this guy satisfies the criterion, but not the second derivative one.
So I have a lot of candidates.
And if my function can be really anything, it's going to be difficult, whether it's analytically by taking derivatives and setting them to 0, or trying to find some algorithms to do this.
Because if my function is very jittery, then my algorithm basically has to check all candidates.
And if there's a lot of them, it might take forever, OK?
So this is-- I have only one, two, three, four, five candidates to check.
But in practice, you might have a million of them to check.
And that might take forever.
OK, so what's nice about statistical models, and one of the things that makes all these models particularly robust, and that we still talk about them 100 years after they've been introduced is that the functions that-- the likelihoods that they lead for us to maximize are actually very simple.
And they all share a nice property, which is that of being concave.
All right, so what is a concave function?
Well, by definition, it's just a function for which-- let's think of it as being twice differentiable.
You can define functions that are not differentiable as being concave, but let's think about it as having a second derivative.
And so if you look at the function that has a second derivative, concave are the functions that have their second derivative that's negative everywhere.
Not just at the maximum, everywhere, OK?
And so if it's strictly concave, this second derivative is actually strictly less than zero.
And particularly if I think of a linear function, y is equal to x, then this function has its second derivative which is equal to zero, OK?
So it is concave.
But it's not strictly concave, OK?
If I look at the function which is negative x squared, what is its second derivative?
Minus 2.
So it's strictly negative everywhere, OK?
So actually, this is a pretty canonical example strictly concave function.
If you want to think of a picture of a strictly concave function, think of negative x squared.
So parabola pointing downwards.
OK, so we can talk about strictly convex functions.
So convex is just happening when the negative of the function is concave.
So that translates into having a second derivative which is either non-negative or positive, depending on whether you're talking about convexity or strict convexity.
But again, those convex functions are convenient when you're trying to minimize something.
And since we're trying to maximize the function, we're looking for concave.
So here are some examples.
Let's just go through them quickly.
OK, so the first one is-- so here I made my life a little uneasy by talking about the functions in theta, right?
I'm talking about likelihoods, right?
So I'm thinking of functions where the parameter is theta.
So I have h of theta.
And so if I start with theta squared, negative theta squared, then as we said, h prime prime of theta, the second derivative is minus 2, which is strictly negative, so this function is strictly concave.
OK, another function is h of theta, which is-- what did we pick-- square root of theta.
What is the first derivative?
1/2 square root of theta.
What is the second derivative?
So that's theta to the negative 1/2.
So I'm just picking up another negative 1/2, so I get negative 1/4.
And then I get theta to the 3/4 downstairs, OK?
Sorry, 3/2.
And that's strictly negative for theta, say, larger than 0.
And I really need to have this thing larger than 0 so that it's well-defined.
But strictly larger than 0 is so that this thing does not blow up to infinity.
And it's true.
If you think about this function, it looks like this.
And already, the first derivative to infinity at 0.
And it's a concave function, OK?
Another one is the log, of course.
What is the derivative of the log?
That's 1 over theta, where h prime of theta is 1 over theta.
And the second derivative negative 1 over theta squared, which again, is negative if theta is strictly positive.
Here I define it as-- I don't need to define it to be strictly positive here, but I need it for the log.
And sine.
OK, so let's just do one more.
So h of theta is sine of theta.
But here I take it only on an interval, because you want to think of this function as pointing always downwards.
And in particular, you don't want this function to have an inflection point.
You don't want it to go down and then up and then down and then up, because this is not concave.
And so sine is certainly going up and down, right?
So what we do is we restrict it to an interval where sine is actually-- so what does the sine function looks like at 0, 0?
And it's going up.
Where is the first maximum of the sine?
[INAUDIBLE]
I'm sorry
Pi over 2
Pi over 2, where it takes value 1.
And then it goes down again.
And then that's at pi.
And then I go down again.
And here you see I actually start changing my inflection.
So what we do is we stop it at pi.
And we look at this function, it certainly looks like a parabola pointing downwards.
And so if you look at the-- you can check that it actually works with the derivatives.
So the derivative of sine is cosine.
And the derivative of cosine is negative sine.
OK, and this thing between 0 and pi is actually positive.
So this entire thing is going to be negative.
OK?
And you know, I can come up with a lot of examples, but let's just stick to those.
There's a linear function, of course.
And the find function is going to be concave, but it's actually going to be convex as well, which means that it's certainly not going to be strictly concave or convex, OK?
So here's your standard picture.
And here, if you look at the dotted line, what it tells me is that a concave function, and the property we're going to be using is that if a strictly concave function has a maximum, which is not always the case, but if it has a maximum, then it actually must be-- sorry, a local maximum, it must be a global maximum.
OK, so just the fact that it goes up and down and not again means that there's only global maximum that can exist.
Now if you looked, for example, at the square root function, look at the entire positive real line, then this thing is never going to attain a maximum.
It's just going to infinity as x goes to infinity.
So if I wanted to find the maximum, I would have to stop somewhere and say that the maximum is attained at the right-hand side.
OK, so that's the beauty about convex functions or concave functions, is that essentially, these functions are easy to maximize.
And if I tell you a function is concave, you take the first derivative, set it equal to 0.
If you find a point that satisfies this, then it must be a global maximum, OK?
What if your set theta was [INAUDIBLE] then couldn't you have a function that, by the definition, is concave, with two upside down parabolas at two disjoint intervals, but yet it has two global maximums?
So you won't get them-- so you want the function to be concave on what?
On the convex cell of the intervals?
Or you want it to be--
[INAUDIBLE] just said that any subset
OK, OK. You're right.
So maybe the definition-- so you're pointing to a weakness in the definition.
Let's just say that theta is a convex set and then you're good, OK?
So you're right.
Since I actually just said that this is true only for theta, I can just take pieces of concave functions, right?
I can do this, and then the next one I can do this, on the next one I can do this.
And then I would have a bunch of them.
But what I want is think of it as a global function on some convex set.
You're right.
So think of theta as being convex for this guy, an interval, if it's a real line.
OK, so as I said, for more generally-- so we can actually define concave functions more generally in higher dimensions.
And that will be useful if theta is not just one parameter but several parameters.
And for that, you need to remind yourself of Calculus II, and you have generalization of the notion of derivative, which is called a gradient, which is basically a vector where each coordinate is just the partial derivative with respect to each coordinate of theta.
And the Hessian is the matrix, which is essentially a generalization of the second derivative.
I denote it by nabla squared, but you can write it the way you want.
And so this matrix here is taking as entry the second partial derivatives of h with respect to theta i and theta j.
And so that's the ij-th entry.
Who has never seen that?
OK.
So now, being concave here is essentially generalizing, saying that a vector is equal to zero.
Well, that's just setting the vector-- sorry.
The first order condition to say that it's a maximum is going to be the same.
Saying that a function has a gradient equal to zero is the same as saying that each of its coordinates are equal to zero.
And that's actually going to be a condition for a global maximum here.
So to check convexity, we need to see that a matrix itself is negative.
Sorry, to check concavity, we need to check that a matrix is negative.
And there is a notion among matrices that compare matrix to zero, and that's exactly this notion.
You pre- and post-multiply by the same x.
So that works for symmetric matrices, which is the case here.
And so you pre-multiply by x, post-multiply by the same x.
So you have your matrix, your Hessian here.
It's a d by d matrix if you have a d-dimensional matrix.
So let's call it-- OK. And then here I pre-multiply by x transpose.
I post-multiply by x.
And this has to be non-positive if I want it to be concave, and strictly negative if I want it to be strictly concave.
OK, that's just a real generalization.
You can check for yourself that this is the same thing.
If I were in dimension 1, this would be the same thing.
Why?
Because in dimension 1, pre- and post-multiplying by x is the same as multiplying by x squared.
Because in dimension 1, I can just move my x's around, right?
And so that would just mean the first condition would mean in dimension 1 that the second derivative times x squared has to be less than or equal to zero.
So here I need this for all x's that are not zero, because I can take x to be zero and make this equal to zero, right?
So this is for x's that are not equal to zero, OK?
And so some examples.
Just look at this function.
So now I have functions that depend on two parameters, theta1 and theta2.
So the first one is-- so if I take theta to be equal to-- now I need two parameters, r squared.
And I look at the function, which is h of theta.
Can somebody tell me what h of theta is?
[INAUDIBLE]
Minus 2 theta2 squared?
OK, so let's compute the gradient of h of theta.
So it's going to be something that has two coordinates.
To get the first coordinate, what do I do?
Well, I take the derivative with respect to theta1, thinking of theta2 as being a constant.
So this thing is going to go away.
And so I get negative 2 theta1.
And when I take the derivative with respect to the second part, thinking of this part as being constant, I get minus 4 theta2.
That clear for everyone?
That's just the definition of partial derivatives.
And then if I want to do the Hessian, so now I'm going to get a 2 by 2 matrix.
The first guy here, I take the first-- so this guy I get by taking the derivative of this guy with respect to theta1.
So that's easy.
So that's just minus 2.
This guy I get by taking derivative of this guy with respect to theta2.
So I get what?
Zero.
I treat this guy as being a constant.
This guy is also going to be zero, because I take the derivative of this guy with respect to theta1.
And then I take the derivative of this guy with respect to theta2, so I get minus 4.
OK, so now I want to check that this matrix satisfies x transpose-- this matrix x is negative.
So what I do is-- so what is x transpose x?
So if I do x transpose delta squared h theta x, what I get is minus 2 x1 squared minus 4 x2 squared.
Because this matrix is diagonal, so all it does is just weights the square of the x's.
So this guy is definitely negative.
This guy is negative.
And actually, if one of the two is non-zero, which means that x is non-zero, then this thing is actually strictly negative.
So this function is actually strictly concave.
And it looks like a parabola that's slightly distorted in one direction.
So well, I know this might have been some time ago.
Maybe for some of you might have been since high school.
So just remind yourself of doing second derivatives and Hessians and things like this.
Here's another one as an exercise.
h is minus theta1 minus theta2 squared.
So this one is going to actually not be diagonal.
The Hessian is not going to be diagonal.
Who would like to do this now in class?
OK, thank you.
This is not a calculus class.
So you can just do it as a calculus exercise.
And you can do it for log as well.
Now, there is a nice recipe for concavity that works for the second one and the third one.
And the thing is, if you look at those particular functions, what I'm doing is taking, first of all, a linear combination of my arguments.
And then I take a concave function of this guy.
And this is always going to work.
This is always going to give me a complete function.
So the computations that I just made, I actually never made them when I prepared those slides because I don't have to.
I know that if I take a linear combination of those things and then I take a concave function of this guy, I'm always going to get a concave function.
OK, so that's an easy way to check this, or at least as a sanity check.
All right, and so as I said, finding maximizers of concave or strictly concave function is the same as it was in the one-dimensional case.
What I do-- sorry, in the one-dimensional case, we just agreed that we just take the derivative and set it to zero.
In the high dimensional case, we take the gradient and set it equal to zero.
Again, that's calculus, all right?
So it turns out that so this is going to give me equations, right?
The first one is an equation in theta.
The second one is an equation in theta1, theta2, theta3, all the way to theta d. And it doesn't mean that because I can write this equation that I can actually solve it.
This equation might be super nasty.
It might be like some polynomial and exponentials and logs equal zero, or some crazy thing.
And so there's actually, for a concave function, since we know there's a unique maximizer, there's this theory of convex optimization, which really, since those books are talking about minimizing, you had to find some sort of direction.
But you can think of it as the theory of concave maximization.
And they allow you to find algorithms to solve this numerically and fairly efficiently.
OK, that means fast.
Even if d is of size 10,000, you're going to wait for one second and it's going to tell you what the maximum is.
And that's what machine learning is about.
If you've taken any class on machine learning, there's a lot of optimization, because they have really, really big problems to solve.
Often in this class, since this is more introductory statistics, we will have a close form.
For the maximum likelihood estimator will be saying theta hat equals, and say x bar, and that will be the maximum likelihood estimator.
So just why-- so has anybody seen convex optimization before?
So let me just give you an intuition why those functions are easy to maximize or to minimize.
In one dimension, it's actually very easy for you to see that.
And the reason is this.
If I want to maximize the concave function, what I need to do is to be able to query a point and get as an answer the derivative of this function, OK?
So now I said this is the function I want to optimize, and I've been running my algorithm for 5/10 of a second.
And it's at this point here.
OK, that's the candidate.
Now, what I can ask is, what is the derivative of my function here?
Well, it's going to give me a value.
And this value is going to be either negative, positive, or zero.
Well, if it's zero, that's great.
That means I'm here and I can just go home.
I've solved my problem.
I know there's a unique maximum, and that's what I wanted to find.
If it's positive, it actually tells me that I'm on the left of the optimizer.
And on the left of the optimal value.
And if it's negative, it means that I'm at the right of the value I'm looking for.
And so most of the convex optimization methods basically tell you, well, if you query the derivative and it's actually positive, move to the right.
And if it's negative, move to the left.
Now, by how much you move is basically, well, why people write books.
And in higher dimension, it's a little more complicated, because in higher dimension, thinks about two dimensions, then I'm only being able to get in a vector.
And the vector is only telling me, well, here is half of the space in which you can move.
Now here, if you tell me move to the right, I know exactly which direction I'm going to have to move.
But in two dimension, you're going to basically tell me, well, move in this global direction.
And so, of course, I know there's a line on the floor I cannot move behind.
But even if you tell me, draw a line on the floor and move only to that side of the line, then there's many directions in that line that I can go to.
And that's also why there's lots of things you can do in optimization.
OK, but still, putting this line on the floor is telling me, do not go backwards.
And that's very important.
It's just telling you which direction I should be going always, OK?
All right, so that's what's behind this notion of gradient descent algorithm, steepest descent.
Or steepest descent, actually, if we're trying to maximize.
OK, so let's move on.
So this course is not about optimization, all right?
So as I said, the likelihood was this guy.
The product of f of the xi's.
And one way you can do this is just basically the joint distribution of my data at the point theta.
So now the likelihood, formerly-- so here I am giving myself the model e theta.
And here I'm going to assume that e is discrete so that I can talk about PMFs.
But everything you're doing, just redo for the sake of yourself by replacing PMFs by PDFs, and everything's going to be fine.
We'll do it in a second.
All right, so the likelihood of the model.
So here I'm not looking at the likelihood of a parameter.
I'm looking at the likelihood of a model.
So it's actually a function of the parameter.
And actually, I'm going to make it even a function of the points x1 to xn.
All right, so I have a function.
And what it takes as input is all the points x1 to xn and a candidate parameter theta.
Not the true one.
A candidate.
And what I'm going to do is I'm going to look at the probability that my random variables under this distribution, p theta, take these exact values, px1, px2, pxn.
Now remember, if my data was independent, then I could actually just say that the probability of this intersection is just a product of the probabilities.
And it would look something like this.
But I can define likelihood even if I don't have independent random variables.
But think of them as being independent, because that's all we're going to encounter in this class, OK?
I just want you to be aware that if I had dependent variables, I could still define the likelihood.
I would have to understand how to compute these probabilities there to be able to compute it.
OK, so think of Bernoullis, for example.
So here is my example of a Bernoulli.
So my parameter is-- so my model is 0,1 Bernoulli p. p is in the interval 0,1.
The probability, just as a side remark, I'm just going to use the fact that I can actually write the PMF of a Bernoulli in a very concise form, right?
If I ask you what the PMF of a Bernoulli is, you could tell me, well, the probability that x-- so under p, the probability that x is equal to 0 is 1 minus p. The probability under p that x is equal to 1 is equal to p. But I can be a bit smart and say that for any X that's either 0 or 1, the probability under p that X is equal to little x, I can write it in a compact form as p to the X, 1 minus p to the 1 minus x.
And you can check that this is the right form because, well, you have to check it only for two values of X, 0 and 1.
And if you plug in 1, you only keep the p. If you plug in 0, you only keep the 1 minus p. And that's just a trick, OK?
I could have gone with many other ways.
Agreed?
I could have said, actually, something like-- another one would be-- which we are not going to use, but we could say, well, it's xp plus and minus x 1 minus p, right?
That's another one.
But this one is going to be convenient.
So forget about this guy for a second.
So now, I said that the likelihood is just this function that's computing the probability that X1 is equal to little x1.
So likelihood is L of X1, Xn.
So let me try to make those calligraphic so you know that I'm talking about smaller values, right?
Small x's.
x1, xn, and then of course p. Sometimes we even put-- I didn't do it, but sometimes you can actually put a semicolon here, semicolon so you know that those two things are treated differently.
And so now you have this thing is equal to what?
Well, it's just the probability under p that X1 is little x1 all the way to Xn is little xn.
OK, that's just the definition.
All right, so now let's start working.
So we write the definition, and then we want to make it look like something we would potentially be able to maximize if I were-- like if I take the derivative of this with respect to p, it's not very helpful because I just don't know.
Just want the algebraic function of p. So this thing is going to be equal to what?
Well, what is the first thing I want to use?
I have a probability of an intersection of events, so it's just the product of the probabilities.
So this is the product from i equal 1 to n of P, small p, Xi is equal to little xi.
That's independence.
OK, now, I'm starting to mean business, because for each P, we have a closed form, right?
I wrote this as this supposedly convenient form.
I still have to reveal to you why it's convenient.
So this thing is equal to-- well, we said that that was p xi for a little xi.
1 minus p to the 1 minus xi, OK?
So that was just what I wrote over there as the probability that Xi is equal to little xi.
And since they all have the same parameter p, just have this p that shows up here.
And so now I'm just taking the products of something to the xi, so it's this thing to the sum of the xi's.
Everybody agrees with this?
So this is equal to p sum of the xi, 1 minus p to the n minus sum of the xi.
If you don't feel comfortable with writing it directly, you can observe that this thing here is actually equal to p over 1 minus p to the xi times 1 minus p, OK?
So now when I take the product, I'm getting the products of those guys.
So it's just this guy to the power of sum and this guy to the power n. And then I can rewrite it like this if I want to And so now-- well, that's what we have here.
And now I am in business because I can still hope to maximize this function.
And how to maximize this function?
All I have to do is to take the derivative.
Do you want to do it?
Let's just take the derivative, OK?
Sorry, I didn't tell you that, well, the maximum likelihood principle is to just maxim-- the idea is to maximize this thing, OK?
But I'm not going to get there right now.
OK, so let's do it maybe for the Poisson model for a second.
So if you want to do it for the Poisson model, let's write the likelihood.
So right now I'm not doing anything.
I'm not maximizing.
I'm just computing the likelihood function.
OK, so the likelihood function for Poisson.
So now I know-- what is my sample space for Poisson?
Positives
Positive integers.
And well, let me write it like this.
Poisson lambda, and I'm going to take lambda to be positive.
And so that means that the probability under lambda that X is equal to little x in the sample space is lambda to the X over factorial x e to the minus lambda.
So that's basically the same as the compact form that I wrote over there.
It's just now a different one.
And so when I want to write my likelihood, again, we said little x's.
This is equal to what?
Well, it's equal to the probability under lambda that X1 is little x1, Xn is little xn, which is equal to the product.
OK?
Just by independence.
And now I can write those guys as being-- each of them being i equal 1 to n. So this guy is just this thing where a plug in Xi.
So I get lambda to the Xi divided by factorial xi times e to the minus lambda, OK?
And now, I mean, this guy is going to be nice.
This guy is not going to be too nice.
But let's write it.
When I'm going to take the product of those guys here, I'm going to pick up lambda to the sum of the xi's.
Here I'm going to pick up exponential minus n times lambda.
And here I'm going to pick up just the product of the factorials.
So x1 factorial all the way to xn factorial.
Then I get lambda, the sum of the xi.
Those are little xi's.
e to the minus xn lambda.
OK?
So that might be freaky at this point, but remember, this is a function we will be maximizing.
And the denominator here does not depend on lambda.
So we knew that maximizing this function with this denominator, or any other denominator, including 1, will give me the same arg max.
So it won't be a problem for me.
As long as it does not depend on lambda, this thing is going to go away.
OK, so in the continuous case, the likelihood I cannot-- right?
So if I would write the likelihood like this in the continuous case, this one would be equal to what?
Zero, right?
So it's not very helpful.
And so what we do is we define the likelihood as the product of the f of theta xi.
Now that would be a jump if I told you, well, just define it like that and go home and don't discuss it.
But we know that this is exactly what's coming from the-- well, actually, I think I erased it.
It was just behind.
So this was exactly what was coming from the KL divergence estimated, right?
The thing that I showed you, if we want to follow this strategy, which consists in estimating the KL divergence and minimizing it, is exactly doing this.
So in the Gaussian case-- well, let's write it.
So in the Gaussian case, let's see what the likelihood looks like.
OK, so if I have a Gaussian experiment here-- did I actually write it?
OK, so I'm going to take mu and sigma as being two parameters.
So that means that my sample space is going to be what?
Well, my sample space is still R. Those are just my observations.
But then I'm going to have a N mu sigma squared.
And the parameters of interest are mu and R. And sigma squared and say 0 infinity.
OK, so that's my Gaussian model.
Yes
[INAUDIBLE]
No, there's no-- I mean, there's no difference
[INAUDIBLE]
Yeah.
I think the all the slides I put the curly bracket, then I'm just being lazy.
I just like those concave parenthesis.
All right, so let's write it.
So the definition, L xi, xn.
And now I have two parameters, mu and sigma squared.
We said, by definition, is the product from i equal 1 to n of f theta of little xi.
Now, think about it.
Here we always had an extra line, right?
The line was to say that the definition was the probability that they were all equal to each other.
That was the joint probability.
And here it could actually have a line that says it's the joint probability distribution of the xi's.
And if it's not independent, it's not going to be the product.
But again, since we're only dealing with independent observations in the scope of this class, this is the only definition we're going to be using.
OK, and actually, from here on, I will literally skip this step when I talk about discrete ones as well, because they are also independent.
Agreed?
So we start with this, which we agreed was the definition for this particular case.
And so now all of you know by heart what the density of a-- sorry, that's not theta.
I should write it mu sigma squared.
And so you need to understand what this density.
And it's product of 1 over sigma square root 2 pi times exponential minus xi minus mu squared divided by 2 sigma squared.
OK, that's the Gaussian density with parameters mu and sigma squared.
I just plugged in this thing which I don't give you, so you just have to trust me.
It's all over any book.
Certainly, I mean, you can find it.
I will give it to you.
And again, you're not expected to know it by heart.
Though, if you do your homework every week without wanting to, you will definitely use some of your brain to remember that thing.
OK, and so now, well, I have this constant in front.
1 over sigma square root 2 pi that I can pull out.
So I get 1 over sigma square root 2 pi to the power n. And then I have the product of exponentials, which we know is the exponential of the sum.
So this is equal to exponential minus.
And here I'm going to put the 1 over 2 sigma squared outside the sum.
And so that's how this guy shows up.
Just the product of the density is evaluated at, respectively, x1 to xn.
OK, any questions about computing those likelihoods?
Yes
Why [INAUDIBLE]
Oh, that's a typo.
Thank you.
Because I just took it from probably the previous thing.
So those are actually-- should be-- OK, thank you for noting that one.
So this line should say for any x1 to xn in R to the n. Thank you, good catch.
All right, so that's really e to the n, right?
My sample space always.
OK, so what is maximum likelihood estimation?
Well again, if you go back to the estimate that we got, the estimation strategy, which consisted in replacing expectation with respect to theta star by average of the data in the KL divergence, we would try to maximize not this guy, but this guy.
The thing that we actually plugged in were not any small xi's.
Were actually-- the random variable is capital Xi.
So the maximum likelihood estimator is actually taking the likelihood, which is a function of little x's, and now the values at which it estimates, if you look at it, is actually-- the capital X is my data.
So it looks at the function, at the data, and at the parameter theta.
That's what the-- so that's the first thing.
And then the maximum likelihood estimator is maximizing this, OK?
So in a way, what it does is it's a function that couples together the data, capital X1 to capital Xn, with the parameter theta and just now tries to maximize it.
So if this is just a little hard for you to get, the likelihood is formally defined as a function of x, right?
Like when I write f of x. f of little x, I define it like that.
But really, the only x arguments we're going to evaluate this function at are always the random variable, which is the data.
So if you want, you can think of it as those guys being not parameters of this function, but really, random variables themselves directly.
Is there any question?
[INAUDIBLE] those random variables [INAUDIBLE]??
So those are going to be known once you have-- so it's always the same thing in stats.
You first design your estimator as a function of random variables.
And then once you get data, you just plug it in.
But we want to think of them as being random variables because we want to understand what the fluctuations are.
So we're going to keep them as random variables for as long as we can.
We're going to spit out the estimator as a function of the random variables.
And then when we want to compute it from data, we're just going to plug it in.
So keep the random variables for as long as you can.
Unless I give you numbers, actual numbers, just those are random variables.
OK, so there might be some confusion if you've seen any stats class, sometimes there's a notation which says, oh, the realization of the random variables are lower case versions of the original random variables.
So lowercase x should be thought as the realization of the upper case X.
This is not the case here.
When I write this, it's the same way as I write f of x is equal to x squared, right?
It's just an argument of a function that I want to define.
So those are just generic x.
So if you correct the typo that I have, this should say that this should be for any x and xn.
I'm just describing a function.
And now the only place at which I'm interested in evaluating that function, at least for those first n arguments, is at the capital N observations random variables that I have.
So there's actually texts, there's actually people doing research on when does the maximum likelihood estimator exist?
And that happens when you have infinite sets, thetas.
And this thing can diverge.
There is no global maximum.
There's crazy things that might happen.
And so we're actually always going to be in a case where this maximum likelihood estimator exists.
And if it doesn't, then it means that you actually need to restrict your parameter space, capital Theta, to something smaller.
Otherwise it won't exist.
OK, so another thing is the log likelihood estimator.
So it is still the likelihood estimator.
We solved before that maximizing a function or maximizing log of this function is the same thing, because the log function is increasing.
So the same thing is maximizing a function or maximizing, I don't know, exponential of this function.
Every time I take an increasing function, it's actually the same thing.
Maximizing a function or maximizing 10 times this function is the same thing.
So the function x maps to 10 times x is increasing.
And so why do we talk about log likelihood rather than likelihood?
So the log of likelihood is really just-- I mean the log likelihood is the log of the likelihood.
And the reason is exactly for this kind of reasons.
Remember, that was my likelihood, right?
And I want to maximize it.
And it turns out that in stats, there's a lot of distributions that look like exponential of something.
So I might as well just remove the exponential by taking the log.
So once I have this guy, I can take the log.
This is something to a power of something.
If I take the log, it's going to look better for me.
I have this thing-- well, I have another one somewhere, I think, where I had the Poisson.
Where was the Poisson?
The Poisson's gone.
So the Poisson was the same thing.
If I took the log, because it had a power, that would make my life easier.
So the log doesn't have any particular intrinsic notion, except that it's just more convenient.
Now, that being said, if you think about maximizing the KL, the original formulation, we actually remove the log.
If we come back to the KL thing-- where is my KL?
Sorry.
That was maximizing the sum of the logs of the pi's.
And so then we worked at it by saying that the sum of the logs was-- maximizing the sum of the logs was the same as maximizing the product.
But here, we're basically-- log likelihood is just going backwards in this chain of equivalences.
And that's just because the original formulation was already convenient.
So we went to find the likelihood and then coming back to our original estimation strategy.
So look at the Poisson.
I want to take log here to make my sum of xi's go down.
OK, so this is my estimator.
So the log of L-- so one thing that you want to notice is that the log of L of x1, xn theta, as we said, is equal to the sum from i equal 1 to n of the log of either p theta of xi, or-- so that's in the discrete case.
And in the continuous case is the sum of the log of f theta of xi.
The beauty of this is that you don't have to really understand the difference between probability mass function and probability distribution function to implement this.
Whatever you get, that's what you plug in.
Any questions so far?
All right, so shall we do some computations and check that, actually, we've introduced all this stuff-- complicate functions, maximizing, KL divergence, lot of things-- so that we can spit out, again, averages?
All right?
That's great.
We're going to able to sleep at night and know that there's a really powerful mechanism called maximum likelihood estimator that was actually driving our intuition without us knowing.
OK, so let's do this so.
Bernoulli trials.
I still have it over there.
OK, so actually, I don't know what-- well, let me write it like that.
So it's P over 1 minus P xi-- sorry, sum of the xi's times 1 minus P is to the n. So now I want to maximize this as a function of P. Well, the first thing we would want to do is to check that this function is concave.
And I'm just going to ask you to trust me on this.
So I don't want-- sorry, sum of the xi's.
I only want to take the derivative and just go home.
So let's just take the derivative of this with respect to P. Actually, no.
This one was more convenient.
I'm sorry.
This one was slightly more convenient, OK?
So now we have-- so now let me take the log.
So if I take the log, what I get is sum of the xi's times log p plus n minus some of the xi's times log 1 minus p. Now I take the derivative with respect to p and set it equal to zero.
So what does that give me?
It tells me that sum of the xi's divided by p minus n sum of the xi's divided by 1 minus p is equal to 0.
So now I need to solve for p. So let's just do it.
So what we get is that 1 minus p sum of the xi's is equal to p n minus sum of the xi's.
So that's p times n minus sum of the xi's plus sum of the xi's.
So let me put it on the right.
So that's p times n is equal to sum of the xi's.
And that's equivalent to p-- actually, I should start by putting p hat from here on, because I'm already solving an equation, right?
And so p hat is equal to syn of the xi's divided by n, which is my xn bar.
Poisson model, as I said, Poisson is gone.
So let me rewrite it quickly.
So Poisson, the likelihood in X1, Xn, and lambda was equal to lambda to the sum of the xi's e to the minus n lambda divided by X1 factorial, all the way to Xn factorial.
So let me take the log likelihood.
That's going to be equal to what?
It's going to tell me.
It's going to be-- well, let me get rid of this guy first.
Minus log of X1 factorial all the way to Xn factorial.
That's a constant with respect to lambda.
So when I'm going to take the derivative, it's going to go.
Then I'm going to have plus sum of the xi's times log lambda.
And then I'm going to have minus n lambda.
So now then, you take the derivative and set it equal to zero.
So log L-- well, partial with respect to lambda of log L, say lambda, equals zero.
This is equivalent to, so this guy goes.
This guy gives me sum of the xi's divided by lambda hat equals n. And so that's equivalent to lambda hat is equal to sum of the xi's divided by n, which is Xn bar.
Take derivative, set it equal to zero, and just solve.
It's a very satisfying exercise, especially when you get the average in the end.
You don't have to think about it forever.
OK, the Gaussian model I'm going to leave to you as an exercise.
Take the log to get rid of the pesky exponential, and then take the derivative and you should be fine.
It's a bit more-- it might be one more line than those guys.
OK, so-- well actually, you need to take the gradient in this case.
Don't check the second derivative right now.
You don't have to really think about it.
What did I want to add?
I think there was something I wanted to say.
Yes.
When I have a function that's concave and I'm on, like, some infinite interval, then it's true that taking the derivative and setting it equal to zero will give me the maximum.
But again, I might have a function that looks like this.
Now, if I'm on some finite interval-- let me go elsewhere.
So if I'm on some finite interval and my function looks like this as a function of theta-- let's say this is my log likelihood as a function of theta-- then, OK, there's no place in this interval-- let's say this is between 0 and 1-- there's no place in this interval where the derivative is equal to 0.
And if you actually try to solve this, you won't find a solution which is not in the interval 0, 1.
And that's actually how you know that you probably should not take the derivative equal to zero.
So don't panic if you get something that says, well, the solution is at infinity, right?
If this function keeps going, you will find that the solution-- you won't be able to find a solution apart from infinity.
You are going to see something like 1 over theta hat is equal to 0, or something like this.
So you know that when you've found this kind of solution, you've probably made a mistake at some point.
And the reason is because the functions that are like this, you don't find the maximum by setting the derivative equal to zero.
You actually just find the maximum by saying, well, it's an increasing function on the interval 0, 1, so the maximum must be attained at 1.
So here in this case, that would mean that my maximum would be 1.
My estimator would be 1, which would be weird.
So typically here, you have a function of the xi's.
So one example that you will see many times is when this guy is the maximum of the xi's.
And in which case, the maximum is attained here, which is the maximum of this.
OK, so just keep in mind-- what I would recommend is every time you're trying to take the maximum of a function, just try to plot the function in your head.
It's not too complicated.
Those things are usually squares, or square roots, or logs.
You know what those functions look like.
Just plug them in your mind and make sure that you will find a maximum which really goes up and then down again.
If you don't, then that means your maximum is achieved at the boundary and you have to think differently to get it.
So the machinery that consists in setting the derivative equal to zero works 80% of the time.
But o you have to be careful.
And from the context, it will be clear that you had to be careful, because you will find some crazy stuff, such as solve 1 over theta hat is equal to zero.
All right, so before we conclude, I just wanted to give you some intuition about how does the maximum likelihood perform?
So there's something called the Fisher information that essentially controls how this thing performs.
And the Fisher information is, essentially, a second derivative or a Hessian.
So if I'm in a one-dimensional parameter case, it's a number, it's a second derivative.
If I'm in a multidimensional case, it's actually a Hessian, it's a matrix.
So I'm going to actually take in notation little curly L of theta to be the log likelihood, OK?
And that's the log likelihood for one observation.
So let's call it x generically, but think of it as being x1, for example.
And I don't care of, like, summing, because I'm actually going to take expectation of this thing.
So it's not going to be a data driven quantity I'm going to play with.
So now I'm going to assume that it is twice differentiable, almost surely, because it's a random function.
And so now I'm going to just sweep under the rug some technical conditions when these things hold.
So typically, when can I permute integral and derivatives and this kind of stuff that you don't want to think about?
OK, the rule of thumb is it always works until it doesn't, in which case, that probably means you're actually solving some sort of calculus problem.
Because in practice, it just doesn't happen.
So the Fisher information is the expectation of the-- that's called the outer product.
So that's the product of this gradient and the gradient transpose.
So that forms a matrix, right?
That's a matrix minus the outer product of the expectations.
So that's really what's called the covariance matrix of this vector, nabla of L theta, which is a random vector.
So I'm forming the covariance matrix of this thing.
And the technical conditions tells me that, actually, this guy, which depends only on the Hessian, is actually equal to negative expectation of the-- sorry.
It depends on the gradient.
Is actually negative expectation of the Hessian.
So I can actually get a quantity that depends on the second derivatives only using first derivatives.
But the expectation is going to play a role here.
And the fact that it's a log.
And lots of things actually show up here.
And so in this case, what I get is that-- so in the one-dimensional case, then this is just the covariance matrix of a one-dimensional thing, which is just a variance of itself.
So the variance of the derivative is actually equal to negative the expectation of the second derivative.
OK, so we'll see that next time.
But what I wanted to emphasize with this is that why do we care about this quantity?
That's called the Fisher information.
Fisher is the founding father of modern statistics.
Why do we give this quantity his name?
Well, it's because this quantity is actually very critical.
What does the second derivative of a function tell me at the maximum?
Well, it's telling me how curved it is, right?
If I have a zero second derivative, I'm basically flat.
And if I have a very high second derivative, I'm very curvy.
And when I'm very curvy, what it means is that I'm very robust to the estimation error.
Remember our estimation strategy, which consisted in replacing expectation by averages?
If I'm extremely curvy, I can move a little bit.
This thing, the maximum, is not going to move much.
And this formula here-- so forget about the matrix version for a second-- is actually telling me exactly-- it's telling me the curvature is basically the variance of the first derivative.
And so the more the first derivative fluctuates, the more your maximum is actually-- your org max is going to move all over the place.
So this is really controlling how flat your likelihood, your log likelihood, is at its maximum.
The flatter it is, the more sensitive to fluctuation the arg max is going to be.
The curvier it is, the less sensitive it is.
And so what we're hoping-- a good model is going to be one that has a large or small value for the Fisher information.
I want this to be-- small?
I want it to be large.
Because this is the curvature, right?
This number is negative, it's concave.
So if I take a negative sign, it's going to be something that's positive.
And the larger this thing, the more curvy it is.
Oh, yeah, because it's the variance.
Again, sorry.
This is what-- OK. Yeah, maybe I should not go into those details because I'm actually out of time.
But just spoiler alert, the asymptotic variance of your-- the variance, basically, as n goes to infinity of the maximum likelihood estimator is going to be 1 over this guy.
So we want it to be large, because the asymptotic variance is going to be very small.
All right, so we're out of time.
We'll see that next week.
And I have your homework with me.
And I will actually turn it in.
I will give it to you outside so we can let the other room come in.
OK, I'll just leave you the--
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So welcome back.
We're going to finish this chapter on maximum likelihood estimation.
And last time, I briefly mentioned something that was called Fisher information.
So Fisher information, in general, is actually a matrix when you have a multivariate parameter theta.
So if theta, for example, is of dimension d, then the Fisher information matrix is going to be a d by d matrix.
You can see that, because it's the outer product.
So it's of the form gradient gradient transpose.
So if it's gradient gradient transpose, the gradient is the d dimensional.
And so gradient times gradient transpose is a d by d matrix.
And so this matrix actually contains-- well, tells you it's called Fisher information matrix.
So it's basically telling you how much information about the theta is in your model.
So for example, if your model is very well-parameterized, then you will have a lot of information.
You will have a higher-- so let's think of it as being a scalar number, just one number now-- so you're going to have a larger information about your parameter in the same probability distribution.
But if start having a weird way to parameterize your model, then the Fisher information is actually going to drop.
So as a concrete example think of, for example, a parameter of interest in a Gaussian model, where the mean is known to be zero.
But what you're interested in is the variance, sigma squared.
If I'm interested in sigma square, I could parameterize my model by sigma, sigma squared, sigma to the fourth, sigma to 24th.
I could parameterize it by whatever I want, then I would have a simple transformation.
Then you could say that some of them are actually more or less informative, and you're going to have different values for your Fisher information.
So let's just review a few well-known computations.
So I will focus primarily on the one dimensional case as usual.
And I claim that there's two definitions.
So if theta is a real valued parameter, then there's basically two definitions that you can think of for your Fisher information.
One involves the first derivative of your log likelihood.
And the second one involves the second derivative.
So the log likelihood here, we're actually going to define it as l of theta.
And what is it?
Well, it's simply the likelihood function for one observation.
So it's l-- and I'm going to write 1 just to make sure that we all know what we're talking about one observation-- of-- which is the order again, I think it's X and theta.
So that's the log likelihood, remember?
So for example, if I have a density, what is it going to be?
It's going to be log of f sub theta of X.
So this guy is a random variable, because it's a function of a random variable.
And that's what you see expectations of this thing.
It's a random function of theta.
If I view this as a function of theta, the function becomes random, because it depends on this random X.
And so I of theta is actually defined as the variance of l prime of theta-- so the variance of the derivative of this function.
And I also claim that it's equal to negative the expectation of the second derivative of theta.
And here, the expectation and the variance are computed, because this function, remember, is random.
So I need to tell you what is the distribution of the X with respect to which I'm computing the expectation and the variance.
And it's the theta itself.
So typically, the theta we're going to be interested in-- so there's a Fisher information for all values of the parameter, but the one typically we're interested in is the true parameter, theta star.
But view this as a function of theta right now.
So now, I need to prove to you-- and this is not a trivial statement-- the variance of the derivative is equal to negative the expectation of the second derivative.
I mean, there's really quite a bit that comes into this right.
And it comes from the fact that this is a log not of anything.
It's the log of a density.
So let's just prove that without having to bother too much ourselves with some technical assumptions.
And the technical assumptions are the assumptions that allow me to permute derivative and integral.
Because when I compute the variances and expectations, I'm actually integrating against the density.
And what I want to do is to make sure that I can always do that.
So my technical assumptions are I can always permute integral and derivatives.
So let's just prove this.
So what I'm going to do is I'm going to assume that X has density f theta.
And I'm actually just going to write-- well, let me write it f theta right now.
Let me try to not be lazy about writing.
And so the thing I'm going to use is the fact that the integral of this density is equal to what?
1.
And this is where I'm going to start doing weird things.
That means that if I take the derivative of this guy, it's equal to 0.
So that means that if I look at the derivative with respect to theta of integral f theta of X dX, this is equal to 0.
And this is where I'm actually making this switch, is that I'm going to say that this is actually equal to the integral of the derivative.
So that's going to be the first thing I'm going to use.
And of course, if it's true for the first derivative, it's going to be true for the second derivative.
So I'm going to actually do it a second time.
And the second thing I'm going to use is the fact the integral of the second derivative is equal to 0.
So let's start from here.
And let me start from, say, the expectation of the second derivative of l prime theta.
So what is l prime prime theta?
Well, it's the second derivative of log of f theta of X.
And we know that the derivative of the log-- sorry-- yeah, so the derivative of the log is 1 over-- well, it's the derivative of f divided by f itself.
Everybody's with me?
Just log of f prime is f prime over f. Here, it's just that f, I view this as a function of theta and not as a function of X.
So now, I need to take another derivative of this thing.
So that's going to be equal to-- well, so we all know the formula for the derivative of the ratio.
So I pick up the second derivative times f theta minus the first derivative squared divided by f theta squared-- basic calculus.
And now, I need to check that negative the expectation of this guy is giving me back what I want.
Well what is negative the expectation of l prime prime of theta?
Well, what we need to do is to do negative integral of this guy against f theta.
So it's minus the integral of-- That's just the definition of the expectation.
I take an integral against f theta.
But here, I have something nice.
What's happening is that those guys are canceling.
And now that those guys are canceling, those guys are canceling too.
So what I have is that the first term-- I'm going to break this difference here.
So I'm going to say that integral of this difference is the difference of the integrals.
So the first term is going to be the integral of d over d theta squared of f theta.
And the second one, the negative signs are going to cancel, and I'm going to be left with this.
Everybody's following?
Anybody found the mistake?
How about the other mistake?
I don't know if there's a mistake.
I'm just trying to get you to check what I'm doing.
With me so far?
So this guy here is the integral of the second the derivative of f of X dX.
What is this?
It's 0
It's 0.
And that's because of this guy, which I will call frowny face.
So frowny face tells me this.
And let's call this guy monkey that hides his eyes.
No, let's just do something simpler.
Let's call it star.
And this guy, we will use later on.
So now, I have to prove that this guy, which I have proved is equal to this, is now equal to the variance of l prime theta.
So now, let's go back to the other way.
We're going to meet halfway.
I'm going to have a series-- I want to prove that this guy is equal to this guy.
And I'm going to have a series of equalities that I'm going to meet halfway.
So let's start from the other end.
We started from the negative l prime prime theta.
Let's start with the variance part.
Variance of l prime of theta, so that's the variance-- so that's the expectation of l prime of theta squared minus the square of the expectation of l prime of theta.
Now, what is the square of the expectation of l prime of theta?
Well, l prime of theta is equal to the partial with respect to theta of log f theta of X, which we know from the first line over there-- that's what's in the bracket on the second line there-- is actually equal to the partial over theta of f theta X divided by f theta X.
That's the derivative of the log.
So when I look at the expectation of this guy, I'm going to have the integral of this against f theta.
And the f thetas are going to cancel again, just like I did here.
So this thing is actually equal to the integral of partial over theta of f theta of X dX.
And what does this equal to?
0, by the monkey hiding is eyes.
So that's star-- tells me that this is equal to 0.
So basically, when I compute the variance, this term is not.
Going to matter.
I only have to complete the first one.
So what is the first one?
Well, the first one is the expectation of l prime squared.
And so that guy is the integral of-- well, what is l prime?
Again, it's partial over partial theta f theta of X divided by f theta of X.
Now, this time, this guy is squared against the density.
So one of the f thetas cancel.
But now, I'm back to what I had before for this guy.
So this guy is now equal to this guy.
There's just the same formula.
So they're the same thing.
And so I've moved both ways.
Starting from the expression that involves the expectation of the second derivative, I've come to this guy.
And starting from the expression that tells me about the variance of the first derivative, I've come to the same guy.
So that completes my proof.
Are there any questions about the proof?
We also have on our way found an explicit formula for the Fisher information as well.
So now that I have this thing, I could actually add that if X has a density, for example, this is also equal to the integral of-- well, the partial over theta of f theta of X squared divided by f theta of X, because I just proved that those two things were actually equal to the same thing, which was this guy.
Now in practice, this is really going to be the useful one.
The other two are going to be useful depending on what case you're in.
So if I ask you to compute the Fisher information, you have now three ways to pick from.
And basically, practice will tell you which one to choose if you want to save five minutes when you're doing your computations.
Maybe you're the guy who likes to take derivatives.
And then you're going to go with the second derivative one.
Maybe you're the guy who likes to extend squares, so you're going to take the one that involves the square of the squared prime.
And maybe you're just a normal person, and you want to use that guy.
Why do I care?
This is the Fisher information.
And I could have defined the [?
Hilbert ?]
information by taking the square root of this guy plus the sine of this thing and just be super happy and have my name in textbooks.
But this thing has a very particular meaning.
When we're doing the maximum likelihood estimation-- so remember the maximum likelihood estimation is just an empirical version of trying to minimize the KL divergence.
So what we're trying to do, maximum likelihood, is really trying to minimize the KL divergence.
And we're trying to minimize this function, remember?
So now what we're going to do is we're going to plot this function.
We said that, let's place ourselves in cases where this KL is convex, so that the inverse is concave.
So it's going to look like this-- U-shaped, that's convex.
So that's the truth thing I'm trying to minimize.
And what I said is that I'm going to actually try to estimate this guy.
So in practice, I'm going to have something that looks like this, but it's not really this.
And we're not going to do this, but you can show that you can control this uniformly over the entire space, that there is no space where it just becomes huge.
In particular, this is not the space where it just becomes super huge, and the minimum of the dotted line becomes really far from this guy.
So if those two functions are close to each other, then this implies that the minimum here of the dotted line is close to the minimum of the solid line.
So we know that this is theta star.
And this is our MLE, estimator, theta hat ml.
So that's basically the principle-- the more data we have, the closer the dotted line is to the solid line.
And so the minimum is closer to the minimum.
But now, this is just one example, where I drew a picture for you.
But there could be some really nasty examples.
Think of this example, where I have a function, which is convex, but it looks more like this.
That's convex, it's U-shaped.
It's just a professional U.
Now, I'm going to put a dotted line around it that has pretty much the same fluctuations.
The bend around it is of this size.
So do we agree that the distance between the solid line and the dotted line is pretty much the same in those two pictures?
Now, here, depending on how I tilt this guy, basically, I can put the minimum theta star wherever I want.
And let's say that here, I actually put it here.
That's pretty much the minimum of this line.
And now, the minimum of the dotted line is this guy.
So they're very far.
The fact that I'm very flat at the bottom makes my requirements for being close to the U-shaped solid curve much more stringent, if I want to stay close.
And so this is the canonical case.
This is the annoying case.
And of course, you have the awesome case-- looks like this.
And then whether you deviate, you can have something that moves pretty far.
It doesn't matter, it's always going to stay close.
Now, what is the quantity that measures how curved I am at a given point-- how curved the function is at a given point?
The secondary derivative.
And so the Fisher information is negative the second derivative.
Why the negative?
Well here-- Yeah, we're looking for a minimum, and this guy is really the-- you should view this as a reverted function.
This is we're trying to maximize the likelihood, which is basically maximizing the negative KL.
So the picture I'm showing you is trying to minimize the KL.
So the truth picture that you should see for this guy is the same, except that it's just flipped over.
But the curvature is the same, whether I flip my sheet or not.
So it's the same thing.
So apart from this negative sign, which is just coming from the fact that we're maximizing instead of minimizing, this is just telling me how curved my likelihood is around the maximum.
And therefore, it's actually telling me how good, how robust my maximum likelihood estimator is.
It's going to tell me how close, actually, my likelihood estimator is going to be-- maximum likelihood is going to be to the true parameter.
So I should be able to see that somewhere.
There should be some statement that tells me that this Fisher information will play a role when assessing the precision of this estimator.
And remember, how do we characterize a good estimator?
Well, we look at its bias, or we look its variance.
And we can combine the two and form the quadratic risk.
So essentially, what we're going to try to say is that one of those guys-- either the bias or the variance or the quadratic risk-- is going to be worse if my function is flatter, meaning that my Fisher information is smaller.
And this is exactly the point of this last theorem.
So let's look at a couple of conditions.
So this is your typical 1950s statistics paper that has like one page of assumptions.
And this was like that in the early days, because people were trying to make theories that would be valid for as many models as possible.
And now, people are sort of abusing this, and they're just making all this lists of assumptions so that their particular method works for their particular problem, because they just want to take shortcuts.
But really, the maximum likelihood estimator is basically as old as modern statistics.
And so this was really necessary conditions.
And we'll just parse that.
The model is identified.
Well, better be, because I'm trying to estimate theta and not P theta.
So this one is good.
For all theta, the support of P theta does not depend on theta.
So that's just something that we need to have.
Otherwise, things become really messy.
And in particular, I'm not going to be able to define likelihood-- Kullback-Leibler divergences.
Then why can I not do that?
Well, because the Kullback-Leibler divergence has a log of the ratio of two densities.
And if one of the support is changing with theta is it might be they have the log of something that's 0 or something that's not 0.
And the log of 0 is a slightly annoying quantity to play with.
And so we're just removing that case.
Nothing depends on theta-- think of it as being basically the entire real line as a support for Gaussian, for example.
Theta star is not on the boundary of theta.
Can anybody tell me why this is important?
We're talking about derivatives.
So when I want to talk about derivatives, I'm talking about fluctuations around a certain point.
And if I'm at the boundary, it's actually really annoying.
I might have the derivative-- remember, I give you this example-- where the maximum likelihood is just obtained at the boundary, because the function cannot grow anymore at the boundary.
But it does not mean that the first order derivative is equal to 0.
It does not mean anything.
So all this picture here is valid only if I'm actually achieving the minimum inside.
Because if my theta space stops here and it's just this guy, I'm going to be here.
And there's no questions about curvature or anything that comes into play.
It's completely different.
So here, it's inside.
Again, think of theta as being the entire real line.
Then everything is inside.
I is invertible.
What does it mean for a positive number, a 1 by 1 matrix to be invertible?
Yep
It'd be equal to its [INAUDIBLE]
A 1 by 1 matrix, that's a number, right?
What is a characteristic-- if I give you a matrix with numbers and ask you if it's invertible, what are you going to do with it?
Check if the determinant is 0
Check if the determinant is 0.
What is the determinant of the 1 by 1 matrix?
It's just the number itself.
So that's basically, you want to check if this number is 0 or not.
So we're going to think in the one dimensional case here.
And in the one dimensional case, that just means that the curvature is not 0.
Well, it better be not 0, because then I'm going to have no guarantees.
If I'm totally flat, if I have no curvature, I'm basically totally flat at the bottom.
And then I'm going to get nasty things.
Now, this is not true.
I could have the curvature which grows like-- so here, it's basically-- the second derivative is telling me-- if I do the Taylor expansion, it's telling me how I grow as a function of, say, x squared.
It's the quadratic term that I'm controlling.
It could be that this guy is 0, and then the term of order, x to the fourth, is picking up.
That could be the first one that's non-zero.
But that would mean that my rate of convergence would not be square root of n. When I'm actually playing central limit theorem, it would become n to the 1/4th.
And if I have all a bunch of 0 until the 16th order, I would have n to the 1/16th, because that's really telling me how flat I am.
So we're going to focus on the case where it's only quadratic terms, and the rates of the central limit theorems kick in.
And then a few other technical conditions-- we just used a couple of them.
So I permuted limit and integral.
And you can check that really what you want is that the integral of a derivative is equal to 0.
Well, it just means that the values at the two ends are actually the same.
So those are slightly different things.
So now, what we have is that the maximum likelihood estimator has the following two properties.
The first one, if I were to say that in words, what would I say, that theta hat is-- Is what?
Yeah, that's what I would say when I-- that's for mathematicians.
But if I'm a statistician, what am I going to say?
It's consistent.
It's a consistent estimator of theta star.
It converges in probability to theta star.
And then we have this sort of central limit theorem statement.
The central limit theorem statement tells me that if this was an average and I remove the expectation of the average-- let's say it's 0, for example-- then square root of n times the average blah goes through some normal distribution.
This is telling me that this is actually true, even if theta hat has nothing to do with an average.
That's remarkable.
Theta hat might not even have a closed form, and I'm still having basically the same properties as an average that would be given to me by a central limit theorem.
And what is the asymptotic variance?
So that's the variance in the n. So here, I'm thinking of having those guys being multivariate.
And so I have the inverse of the covariance matrix that shows up as the variance-covariance matrix asymptotically.
But if you think of just being a one dimensional parameter, it's one over this Fisher information, one over the curvature.
So the curvature is really flat, the variance becomes really big.
If the function is really flat, curvature is low, variance is big.
If the curvature is very high, the variance becomes very low.
And so that illustrates everything that's happening with the pictures that we have.
And if you look, what's amazing here, there is no square root 2 pi, there's no fudge factors going on here.
This is the asymptotic variance, nothing else.
It's all in there, all in the curvature.
Are there any questions about this?
So you can see here that theta star is the true parameter.
And the information matrix is evaluated at theta star.
That's the point that matters.
When I drew this picture, the point that was at the very bottom was always theta star.
It's the one that minimizes the KL divergence, am as long as I'm identified.
Yes
So the higher the curvature, the higher the inverse of the Fisher information?
No, the higher the Fisher information itself.
So the inverse is going to be smaller.
So small variance is good.
So now what it means, actually, if I look at what is the quadratic risk of this guy, asymptotically-- what is asymptotic quadratic risk?
Well, it's 0 actually.
But if I assume that this thing is true, that this thing is pretty much Gaussian, if I look at the quadratic risk, well, it's the expectation of the square of this thing.
And so it's going to scale like the variance divided by n. The bias goes to 0, just by this.
And then the quadratic risk is going to scale like one over Fisher information divided by n. So here, the-- I'm not mentioning the constants.
There must be constants, because everything is asymptotic.
So for each finite n, I'm going to have some constants that show up.
Everybody just got their mind blown by this amazing theorem?
So I mean, if you think about it, the MLE can be anything.
I'm sorry to say to you, in many instances, the MLE is just going to be an average, which is just going to be slightly annoying.
But there are some cases where it's not.
And we have to resort to this theorem rather than actually resorting to the central limit theorem to prove this thing.
And more importantly, even if this was an average, you don't have to even know how to compute the covariance matrix-- sorry, the variance of this thing to plug it into the central limit theorem.
I'm telling you, it's actually given by the Fisher information matrix.
So if it's an average, between you and me, you probably want to go the central limit theorem route if you want to prove this kind of stuff.
But if it's not, then that's your best shot.
But you have to check those conditions.
I will give you for granted the 0.5.
Ready?
Any questions?
We're going to wrap up this chapter four.
So if you have questions, that's the time.
Yes
What was the quadratic risk up there?
You mean the definition?
No, the-- what is was for this
Well, you see the quadratic risk, if I think of it as being one dimensional, the quadratic risk is the expectation of the square of the difference between theta hat and theta star.
So that means that if I think of this as having a normal 0, 1, that's basically computing the expectation of the square of this Gaussian divided by n. I just divided by square root of n on both sides.
So it's the expectation of the square of this Gaussian.
The Gaussian is mean 0, so the expectation of the square is just a variance.
And so I'm left with 1 over the Fisher information divided by n
I see
So let's move on to chapter four.
And this is the method of moments.
So the method of moments is actually maybe a bit older than maximum likelihood.
And maximum likelihood is dated, say, early 20th century, I mean as a systematic thing, because as I said, many of those guys are going to be averages.
So finding an average is probably a little older.
The method of moments, there's some really nice uses.
There's a paper by Pearson in 1904, I believe, or maybe 1894.
I don't know.
And this paper, he was actually studying some species of crab in an island, and he was trying to make some measurements.
That's how he came up with this model of mixtures of Gaussians, because there was actually two different populations in this populations of crab.
And the way he actually fitted the parameters was by doing the method of moments, except that since there were a lot of parameters, he actually had to basically solve six equations with six unknowns.
And that was a complete nightmare.
And the guy did it by hand.
And we don't know how he did it actually.
But that is pretty impressive.
So I want to start-- and this first part is a little brutal.
But this is a Course 18 class, and I do not want to give you-- So let's all agree that this course might be slightly more challenging than AP statistics.
And that means that it's going to be challenging just during class.
I'm not going to ask you about the Weierstrass Approximation Theorem during the exams.
But what I want is to give you mathematical motivations for what we're doing.
And I can promise you that maybe you will have a slightly higher body temperature during the lecture, but you will come out smarter of this class.
And I'm trying to motivate to you for using mathematical tool and show you where interesting mathematical things that you might find dry elsewhere actually work very beautifully in the stats literature.
And one that we saw was using Kullback-Leibler divergence out of motivation for maximum likelihood estimation, for example.
So the Weierstrass Approximation Theorem is something that comes from pure analysis.
So maybe-- I mean, it took me a while before I saw that.
And essentially, what it's telling you is that if you look at a function that is continuous on an interval a, b-- on a segment a, b-- then you can actually approximate it uniformly well by polynomials as long as you're willing to take the degree of this polynomials large enough.
So the formal statement is, for any epsilon, there exists the d that depends on epsilon in a1 to ad-- so if you insist on having an accuracy which is 1/10,000, maybe you're going to need a polynomial of degree 100,000, who knows.
It doesn't tell you anything about this.
But it's telling you that at least you have only a finite number of parameters to approximate those functions that typically require an infinite number of parameters to be described.
So that's actually quite nice.
And that's the basis for many things and many polynomial methods typically.
And so here, it's uniform, so there's this max over x that shows up that's actually nice as well.
That's Weierstrass Approximation Theorem.
Why is that useful to us?
Well, in statistics, I have a sample of X1 to Xn.
I have, say, a unified statistical model.
I'm not always going to remind you that it's identified-- not unified-- identified statistical model.
And I'm going to assume that it has a density.
You could think of it as having a PMF, but think of it as having a density for one second.
Now, what I want is to find the distribution.
I want to find theta.
And finding theta, since it's identified as equivalent to finding P theta, which is equivalent to finding f theta, and knowing a function is the same-- knowing a density is the same as knowing a density against any test function h. So that means that if I want to make sure I know a density-- if I want to check if two densities are the same, all I have to do is to compute their integral against all bounded continuous functions.
You already know that it would be true if I checked for all functions h. But since f is a density, I can actually look only at functions h that are bounded, say, between minus 1 and 1, and that are continuous.
That's enough.
Agreed?
Well, just trust me on this.
Yes, you have a question?
Why is this-- like, why shouldn't you just say that [INAUDIBLE]?
Yeah, I can do that.
I'm just finding a characterization that's going to be useful for me later on.
I can find a bunch of them.
But here, this one is going to be useful.
So all I need to say is that f theta star integrated against X, h of x-- so this implies that f-- if theta is equal to f theta star not everywhere, but almost everywhere.
And that's only true if I guarantee to you that f theta and f theta stars are densities.
This is not true for any function.
So now, that means that, well, if I wanted to estimate theta hat, all I would have to do is to compute the average, right-- so this guy here, the integral-- let me clean up a bit my board.
So my goal is to find theta such that, if I look at f theta and now I integrate it against h of x, then this gives me the same thing as if I were to do it against f theta star.
And I want this for any h, which is continuous and bounded.
So of course, I don't know what this quantity is.
It depends on my unknown theta star.
But I have theta from this.
And I'm going to do the usual-- the good old statistical trick, which is, well, this I can write as the expectation with respect to P theta star of h theta of x.
That's just the integral of a function against something.
And so what I can do is say, well, now I don't know this guy.
But my good old trick from the book is replace expectations by averages.
And what I get-- And that's approximately by the law of large numbers.
So if I can actually find a function f theta such that when I integrate it against h it gives me pretty much the average of the evaluations of h over my data points for all h, then that should be a good candidate.
The problem is that's a lot of functions to try.
Even if we reduced that from all possible functions to bounded and continuous ones, that's still a pretty large infinite number of them.
And so what we can do is to use our Weierstrass Approximation Theorem.
And it says, well, maybe I don't need to test it against all h. Maybe the polynomials are enough for me.
So what I'm going to do is I'm going to look only at functions h that are of the form sum of ak-- so h of x is sum of ak X to the k-th for k equals 0 to d-- only polynomials of degree d. So when I look at the average of my h's, I'm going to get a term like the first one.
So the first one here, this guy, becomes 1/n sum from i equal 1 to n sum from k equal 0 to d of ak Xi to the k-th.
That's just the average of the values of h of Xi.
And now, what I need to do is to check that it's the same thing when I integrate h of this form as well.
I want this to hold for all polynomials of degree d. That's still a lot of them.
There's still an infinite number of polynomials, because there's an infinite number of numbers a0 to ad that describe those polynomials.
But since those guys are polynomials, it's actually enough for me to look only at the terms of the form X to the k-th-- no linear combination, no nothing.
So actually, it's enough to look only at h of x, which is equal to X to the k-th for k equal 0 to d. And now, how many of those guys are there?
Just d plus 1, 0 to d. So that's actually a much easier thing for me to solve.
Now, this quantity, which is the integral of f theta against X to the k-th-- so that the expectation of X to the k-th here-- it's called moment of order k, or k-th moment of P theta.
That's a moment.
A moment is just the expectation of the power.
The mean is which moment?
The first moment.
And variance is not exactly the second moment.
It's the second moment minus the first moment squared.
That's the variance.
It's E of X squared minus E of X squared.
So those are things that you already know.
And then you can go higher.
You can go to E of X cube, E of X blah, blah.
Here, I say go to E of X to the d-th.
Now, as you can see, this is not something you can really put in action right now, because the Weierstrass Approximation Theorem does not tell you what d should be.
Actually, we totally lost track of the epsilon I was even looking for.
I just said approximately equal, approximately equal.
And so all this thing is really just motivation.
But it's essentially telling you that if you go to d large enough, technically you should be able to identify exactly your distribution up to epsilon.
So I should be pretty good, if I go to d large enough.
Now in practice, actually there should be much less than arbitrarily large d. Typically, we are going to need d which is 1 or 2.
So there are some limitations to the Weierstrass Approximation Theorem.
And there's a few.
The first one is that it only works for continuous functions, which is not so much of a problem.
That can be fixed.
Well, we need bounded continuous functions.
It works only on intervals.
That's annoying, because we're going to have random variables that are defined beyond intervals.
So we need something that just goes beyond the intervals.
And you can imagine that if you let your functions be huge, it's going to be very hard for you to have a polynomial approximately [INAUDIBLE] well.
Things are going to start going up and down at the boundary, and it's going to be very hard.
And again, as I said several times, it doesn't tell us what d should be.
And as statisticians, we're looking for methods, not like principles of existence of a method that exists.
So if E is discrete, I can actually get a handle on this d. If E is discrete and actually finite-- I'm going to actually look at a finite E, meaning that I have a PMF on, say, r possible values, x1 and xr.
My random variable, capital X, can take only r possible values.
Let's think of them as being the integer numbers 1 to r. That's the number of success out of r trials that I get, for example.
Binomial rp, that's exactly something like this.
So now, clearly this entire distribution is defined by the PMF, which gives me exactly r numbers.
So it can completely describe this distribution with r numbers.
The question is, do I have an enormous amount of redundancy if I try to describe this distribution using moments?
It might be that I need-- say, r is equal to 10, maybe I have only 10 numbers to describe this thing, but I actually need to compute moments up to the order of 100 before I actually recover entirely the distribution.
Maybe I need to go infinite.
Maybe the Weierstrass Theorem is the only thing that actually saves me here.
And I just cannot recover it exactly.
I can go to epsilon if I'm willing to go to higher and higher polynomials.
Oh, by the way, in the Weierstrass Approximation Theorem, I can promise you that as epsilon goes to 0, d goes to infinity.
So now, really I don't even have actually r parameters.
I have only r minus parameter, because the last one-- because they sum up to 1.
So the last one I can always get by doing 1 minus the sum of the first r minus 1 first.
Agreed?
So each distribution r numbers is described by r minus 1 parameters.
The question is, can I use only r minus moments to describe this guy?
This is something called Gaussian quadrature.
The Gaussian quadrature tells you, yes, moments are actually a good way to reparametrize your distribution in the sense that if I give you the moments or if I give you the probability mass function, I'm basically giving you exactly the same information.
You can recover all the probabilities from there.
So here, I'm going to denote by-- I'm going to drop the notation in theta.
I don't have theta.
Here, I'm talking about any generic distribution.
And so I'm going to call mk the k-th moment.
And I have a PMF, this is really the sum for j equals 1 to r of xj to the k-th times p of xj.
And this is the PMF.
So that's my k-th moment.
So the k-th moment is a linear combination of the numbers that I am interested in.
So that's one equation.
And I have as many equations as I'm actually willing to look at moments.
So if I'm looking at 25 moments, I have 25 equations.
m1 equals blah with this to the power of 1, m2 equals blah with this to the power of 2, et cetera.
And then I also have the equation that 1 is equal to the sum of the p of xj.
That's just the definition of PMF.
So this is r's.
They're ugly, but those are r's.
So now, this is a system of linear equations in p, and I can actually write it in its canonical form, which is of the form a matrix of those guys times my parameters of interest is equal to a right hand side.
The right hand side is the moments.
That means, if I did you the moments, can you come back and find what the PMF, because we know already from probability that the PMF is all I need to know to fully describe my distribution.
Given the moments, that's unclear.
Now, here, I'm going to actually take exactly r minus 1 moment and this extra condition that the sum of those guys should be 1.
So that gives me r equations based on r minus 1 moments.
And how many unknowns do I have?
Well, I have my r unknown parameters for the PMF, the r values of the PMF.
Now, of course, this is going to play a huge role in whether the are many p's that give me the same.
The goal is to find if there are several p's that can give me the same moments.
But if there's only one p that can give me a set of moment, that means that I have a one-to-one correspondence between PMF and moments.
And so if you give me the moments, I can just go back to the PMF.
Now, how do I go back?
Well, by inverting this matrix.
If I multiply this matrix by its inverse, I'm going to get the identity times the vector of p's equal the inverse of the matrix times the m's.
So what we want to do is to say that p is equal to the inverse of this big matrix times the moments that you give me.
And if I can actually talk about the inverse, then I have basically a one-to-one mapping between the m's, the moments, and the matrix.
So what I need to show is that this matrix is invertible.
And we just decided that the way to check if a matrix is invertible is by computing its determinant.
Who has computed a determinant before?
Who was supposed to compute a determinant at least than just to say, no, you know how to do it.
So you know how to compute determinants.
And if you've seen any determinant in class, there's one that shows up in the exercises that professors love.
And it's called the Vandermonde determinant.
And it's the determinant of a matrix that have a very specific form.
It looks like-- so there's basically only r parameters to this r by r matrix.
The first row, or the first column-- sometimes, it's presented like that-- is this vector where each entry is to the power of 1.
And the second one is each entry is to the power of 2, and to the power of 3, and to the power 4, et cetera.
So that's exactly what we have-- x1 to the first, x2 to the first, all the way to xr to the first, and then same thing to the power of 2, all the way to the last one.
And I also need to add the row of all 1's, which you can think of those guys are to the power of 0, if you want.
So I should really put it on top, if I wanted to have a nice ordering.
So that was the matrix that I had.
And I'm not asking you to check it.
You can prove that by induction actually, typically by doing the usual let's eliminate some rows and columns type of tricks that you do for matrices.
So you basically start from the whole matrix.
And then you move onto a matrix that has only one 1's and then 0's here.
And then you have Vandermonde that's just slightly smaller.
And then you just iterate.
Yeah
I feel like there's a loss to either the supra index, or the sub index should have a k somewhere [INAUDIBLE].. [INAUDIBLE] the one I'm talking about?
Yeah, I know, but I don't think the answer to your question is yes.
So k is the general index, right?
So there's no k. k does not exist.
k just is here for me to tell me for k equals 1 to r. So this is an r by r matrix.
And so there is no k there.
So if you wanted the generic term, if I wanted to put 1 in the middle on the j-th row and k-th column, that would be x-- so j-th row would be x sub k to the power of j.
That would be the-- And so now, this is basically the sum-- well, that should not be strictly-- So that would be for j and k between 1 and r. So this is the formula that get when you try to expand this Vandermonde determinant.
You have to do it only once when you're a sophomore typically.
And then you can just go on Wikipedia to do it.
That's what I did.
I actually made a mistake copying it.
The first one should be 1 less than or equal to j.
And the last one should be k less than or equal to r. And now what you have is the product of the differences of xj and xk.
And for this thing to be non-zero, you need all the terms to be non-zero.
And for all the terms to be non-zero, you need to have no xi, xj, and no xj, xk that are identical.
If all those are different numbers, then this product is going to be different from 0.
And those are different numbers, because those are r possible values that your random verbal takes.
You're not going to say that it takes two with probability 1.5-- sorry, two with probability 0.5 and two with probability 0.25.
You're going to say it takes two with probability 0.75 directly.
So those xj's are different.
These are the different values that your random variable can take.
Remember, xj, xk was just the different values x1 to xr-- sorry-- was the different values that your random variable can take.
Nobody in their right mind would write twice the same value in this list.
So my Vandermonde is non-zero.
So I can invert it.
And I have a one-to-one correspondence between my entire PMF and the first r minus 1's moments to which I append the number 1, which is really the moment of order 0 again.
It's E of X to the 0-th, which is 1.
So good news, I only need r minus 1 parameters to describe r minus 1 parameters.
And I can choose either the values of my PMF.
Or I can choose the r minus 1 first moments.
So the moments tell me something.
Here, it tells me that if I have a discrete distribution with r possible values, I only need to compute r minus 1 moments.
So this is better than Weierstrass Approximation Theorem.
This tells me exactly how many moments I need to consider.
And this is for any distribution.
This is not a distribution that's parametrized by one parameter, like the Poisson or the binomial or all this stuff.
This is for any distribution under a finite number.
So hopefully, if I reduce the family of PMFs that I'm looking at to a one-parameter family, I'm actually going to need to compute much less than r minus 1 values.
But this is actually hopeful.
It tells you that the method of moments is going to work for any distribution.
You just have to invert a Vandermonde matrix.
So just the conclusion-- the statistical conclusion-- is that moments contain important information about the PMF and the PDF.
If we can estimate these moments accurately, we can solve for the parameters of the distribution and recover the distribution.
And in a parametric setting, where knowing P theta amounts to knowing theta, which is identifiability-- this is not innocuous-- it is often the case that even less moments are needed.
After all, if theta is a one dimensional parameter, I have one parameter to estimate.
Why would I go and get 25 moments to get this one parameter.
Typically, there is actually-- we will see that the method of moments just says if you have a d dimensional parameter, just compute d moments, and that's it.
But this is only on a case-by-case basis.
I mean, maybe your model will totally screw up its parameters and you actually need to get them.
I mean, think about it, if the function is parameterized just by its 27th moment-- like, that's the only thing that matters in this distribution, I just describe the function, it's just a density, and the only thing that can change from one distribution to another is this 27th moment-- well, then you're going to have to go get the 27th moment.
And that probably means that your modeling step was actually pretty bad.
So the rule of thumb, if theta is in Rd, we need d moments.
So what is the method of moments?
That's just a good old trick.
Replace the expectation by averages.
That's the beauty.
The moments are expectations.
So let's just replace the expectations by averages and then do it with the average version, as if it was the true one.
So for example, I'm going to talk about population moments, when I'm computing them with the true distribution, and I'm going to talk about them empirical moments, when I talk about averages.
So those are the two quantities that I have.
And now, what I hope is that there is.
So this is basically-- everything is here.
That's where all the money is.
I'm going to assume there's a function psi that maps my parameters-- let's say they're in Rd-- to the set of the first d moments.
Well, what I want to do is to come from this guy back to theta.
So it better be that this function is-- invertible.
I want this function to be invertible.
In the Vandermonde case, this function with just a linear function-- multiply a matrix by theta.
Then inverting a linear function is inverting the matrix.
Then this is the same thing.
So now what I'm going to assume-- and that's key for this method to work-- is that this theta-- so this function psi is one to one.
There's only one theta that gets only one set of moments.
And so if it's one to one, I can talk about its inverse.
And so now, I'm going to be able to define theta as the inverse of the moments-- the reciprocal of the moments.
And so now, what I get is that the moment estimator is just the thing where rather than taking the true guys in there, I'm actually going to take the empirical moments in there.
Before we go any further, I'd like to just go back and tell you that this is not completely free.
How well-behaved your function psi is going to play a huge role.
Can somebody tell me what the typical distance-- if I have a sample of size n, what is the typical distance between an average and the expectation?
What is the typical distance?
What is the order of magnitude as a function of n between xn bar and its expectation
1 over square root of n
1 over square root n. That's what the central limit theorem tells us, right?
The central limit theorem tells us that those things are basically a Gaussian, which is of order of 1 divided by its square of n. And so basically, I start with something which is 1 over square root of n away from the true thing.
Now, if my function psi inverse is super steep like this-- that's psi inverse-- then just small fluctuations, even if they're of order 1 square root of n, can translate into giant fluctuations in the y-axis.
And that's going to be controlled by how steep psi inverse is, which is the same as saying how flat psi is-- how flat is psi.
So if you go back to this Vandermonde inverse, what it's telling you is that if this inverse matrix blows up this guy a lot-- so if I start from a small fluctuation of this thing and then they're blowing up by applying the inverse of this matrix, things are not going to go well.
Anybody knows what is the number that I should be looking for?
So that's from, say, numerical linear algebra numerical methods.
When I have a system of linear equations, what is the actual number I should be looking at to know how much I'm blowing up the fluctuations?
Yeah
Condition number?
The condition number, right.
So what's important here is the condition number of this matrix.
If the condition number of this matrix is small, then it's good.
It's not going to blow up much.
But if the condition number is very large, it's just going to blow up a lot.
And the condition number is the ratio of the largest and the smallest eigenvalues.
So you'll have to know what it is.
But this is how all these things get together.
So the numerical stability translates into statistical stability here.
And numerical means just if I had errors in measuring the right hand side, how much would they translate into errors on the left hand side.
So the error here is intrinsic to statistical questions.
So that's my estimator, provided that it exists.
And I said it's a one to one, so it should exist, if I assume that psi is invertible.
So how good is this guy?
That's going to be definitely our question-- how good is this thing.
And as I said, there's chances that if psi is really steep, then it should be not very good-- if psi inverse is very steep, it should not be very good, which means that it's-- well, let's just leave it to that.
So that means that I should probably see the derivative of psi showing up somewhere.
If the derivative of psi inverse, say, is very large, then I should actually have a larger variance in my estimator.
So hopefully, just like we had a theorem that told us that the Fisher information was key in the variance of the maximum likelihood estimator, we should have a theorem that tells us that the derivative of psi inverse is going to have a key role in the method of moments.
So let's do it.
So I'm going to talk to you about matrices.
So now, I have-- So since I have to manipulate d numbers at any given time, I'm just going to concatenate them into a vector.
So I'm going to call capital M theta-- so that's basically the population moment.
And I have M hat, which is just m hat 1 to m hat d. And that's my empirical moment.
And what's going to play a role is what is the variance-covariance of the random vector.
So I have this vector 1-- do I have 1?
No, I don't have 1.
So that's a d dimensional vector.
And here, I take the successive powers.
Remember, that looks very much like a column of my Vandermonde matrix.
So now, I have this random vector.
It's just the successive powers of some random variable X.
And the variance-covariance matrix is the expectation-- so sigma-- of theta.
The theta just means I'm going to take expectations with respect to theta.
That's the expectation with respect to theta of this guy times this guy transpose minus the same thing but with the expectation inside.
Why do I do X, X1.
I have X, X2, X3.
X, X2, Xd times the expectation of X, X2, Xd.
Everybody sees what this is?
So this is a matrix where if I look at the ij-th term of this matrix-- or let's say, jk-th term, so on row j and column k, I have sigma jk of theta.
And it's simply the expectation of X to the j plus k-- well, Xj Xk minus expectation of Xj expectation of Xk.
So I can write this as m j plus k of theta minus mj of theta times mk of theta.
So that's my covariance matrix of this particular vector that I define.
And now, I'm going to assume that psi inverse-- well, if I want to talk about the slope in an analytic fashion, I have to assume that psi is differentiable.
And I will talk about the gradient of psi, which is, if it's one dimensional, it's just the derivative.
And here, that's where notation becomes annoying.
And I'm going to actually just assume that so now I have a vector.
But it's a vector of functions and I want to compute those functions at a particular value.
And the value I'm actually interested in is at the m of theta parameter.
So psi inverse goes from the set of moments to the set of parameters.
So when I look at the gradient of this guy, it should be a function that takes as inputs moments.
And where do I want this function to be evaluated at?
At the true moment-- at the population moment vector.
Just like when I computed my Fisher information, I was computing it at the true parameter.
So now, once they compute this guy-- so now, why is this a d by d gradient matrix?
So I have a gradient vector when I have a function from rd to r. This is the partial derivatives.
But now, I have a function from rd to rd.
So I have to take the derivative with respect to the arrival coordinate and the departure coordinate.
And so that's the gradient matrix.
And now, I have the following properties.
The first one is that the law of large numbers tells me that theta hat is a weakly or strongly consistent estimator.
So either I use the strong law of large numbers or the weak law of large numbers, and I get strong or weak consistency.
So what does that mean?
Why is that true?
Well, because now so I really have the function-- so what is my estimator?
Theta hat this psi inverse of m hat 1 to m hat k. Now, by the law of large numbers, let's look only at the weak one.
Law of large numbers tells me that each of the mj hat is going to converge in probability as n to infinity to the-- so the empirical moments converge to the population moments.
That's what the good old trick is using, the fact that the empirical moments are close to the true moments as n becomes larger.
And that's because, well, just because the m hat j's are averages, and the law of large numbers works for averages.
So now, plus if I look at my continuous mapping theorem, then I have that psi inverse is continuously differentiable.
So it's definitely continuous.
And so what I have is that psi inverse of m hat 1 m hat d converges to psi inverse m1 to md, which is equal to of theta star.
So that's theta star.
By definition, we assumed that that was the unique one that was actually doing this.
Again, this is a very strong assumption.
I mean, it's basically saying, if the method of moment works, it works.
So the fact that psi inverse one to one is really the key to making this guy work.
And then I also have a central limit theorem.
And the central limit theorem is basically telling me that M hat is converging to M even in the multivariate sense.
So if I look at the vector of M hat and the true vector of M, then I actually make them go-- I look at the difference for scale by square root of n. It goes to some Gaussian.
And usually, we would see-- if it was one dimensional, we would see the variance.
Then we see the variance-covariance matrix.
Who has never seen the-- well, nobody answers this question.
Who has already seen the multivariate central limit theorem?
Who was never seen the multivariate central limit theorem?
So the multivariate central limit theorem is basically just the slight extension of the univariate one.
It just says that if I want to think-- so the univariate one would tell me something like this-- and 0.
And then I would have basically the variance of X to the j-th.
So that's what the central limit theorem tells me.
This is an average.
So this is just for averages.
The central limit theorem tells me this.
Just think of X to the j-th as being y.
And that would be true.
Everybody agrees with me?
So now, this is actually telling me what's happening for all these guys individually.
But what happens when those guys start to correlate together?
I'd like to know if they actually correlate the same way asymptotically.
And so if I actually looked at the covariance matrix of this vector-- so now, I need to look at a matrix which is d by d-- then would those univariate central limit theorems tell me-- so let me right like this, double bar.
So that's just the covariance matrix.
This notation, V double bar is the variance-covariance matrix.
So what this thing tells me-- so I know this thing is a matrix, d by d. Those univariate central limit theorems only give me information about the diagonal terms.
But here, I have no idea where the covariance matrix is.
This guy is telling me, for example, that this thing is like variance of X to the j-th.
But what if I want to find off-diagonal elements of this matrix?
Well, I need to use a multivariate central limit theorem.
And really what it's telling me is that you can actually replace this guy here-- so that goes in distribution to some normal mean 0, again.
And now, what I have is just sigma of theta, which is just the covariance matrix of this vector X, X2, X3, X4, all the way to Xd.
And that's it.
So that's a multivariate Gaussian.
Who has never seen a multivariate Gaussian?
Please, just go on Wikipedia or something.
There's not much to know about it.
But I don't have time to redo probability here.
So we're going to have to live with it.
Now, to be fair, if your goal is not to become a statistical savant, we will stick to univariate questions in the scope of homework and exams.
So now, what was the delta method telling me?
It was telling me that if I had a central limit theorem that told me that theta hat was going to theta, or square root of n theta hat minus theta was going to some Gaussian, then I could look at square root of Mg of theta hat minus g of theta.
And this thing was also going to a Gaussian.
But what it had to be is the square of the derivative of g in the variance.
So the delta method, it was just a way to go from square root of n theta hat n minus theta goes to some N, say 0, sigma squared, to-- so delta method was telling me that this was square root Ng of theta hat N minus g of theta was going in distribution to N0 sigma squared g prime squared of theta.
That was the delta method.
Now, here, we have a function of those guys.
The central limit theorem, even the multivariate one, is only guaranteeing something for me regarding the moments.
But now, I need to map the moments back into some theta, so I have a function of the moments.
And there is something called the multivariate delta method, where derivatives are replaced by gradients.
Like, they always are in multivariate calculus.
And rather than multiplying, since things do not compute, rather than choosing which side I want to put the square, I'm actually just going to take half of the square on one side and the other half of the square on the other side.
So the way you should view this, you should think of sigma squared times g prime squared as being g prime of theta times sigma squared times g prime of theta.
And now, this is completely symmetric.
And the multivariate delta method is basically telling you that you get the gradient here.
So you start from something that's like that over there, a sigma-- so that's my sigma squared, think of sigma squared.
And then I premultiply by the gradient and postmultiply by the gradient.
The first one is transposed.
The second one is not.
But that's very straightforward extension.
You don't even have to understand it.
Just think of what would be the natural generalization.
Here, by the way, I wrote explicitly what the gradient of a multivariate function is.
So that's a function that goes from Rd to Rk.
So now, the gradient is a d by k matrix.
And so now, for this guy, we can do it for the method or moments.
And we can see that basically we're going to have this scaling that depends on the gradient of the reciprocal of psi, which is normal.
Because if psi is super steep, if psi inverse is super steep, then the gradient is going to be huge, which is going to translate into having a huge variance for the method of moments.
So this is actually the end.
I would like to encourage you-- and we'll probably do it on Thursday just to start.
But I encourage you do it in one dimension, so that you know how to use the method of moments, you know how to do a bunch of things.
Do it in one dimension and see how you can check those things.
So just as a quick comparison, in terms of the quadratic risk, the maximum likelihood estimator is typically more accurate than the method of moments.
What is pretty good to do is, when you have a non-concave likelihood function, what people like to do is to start with the method of moments as an initialization and then run some algorithm that optimizes locally the likelihood starting from this point, because it's actually likely to be closer.
And then the MLE is going to improve it a little bit by pushing the likelihood a little better.
So of course, the maximum likelihood is sometimes intractable.
Whereas, computing moments is fairly doable.
If the likelihood is concave, as I said, we can use optimization algorithms, such as interior-point methods or gradient descent, I guess, to maximize it.
And if the likelihood is non-concave, we only have local heuristics.
Risk And that's what I meant-- you have only local maxima.
And one trick you can do-- so your likelihood looks like this, and it might be the case that if you have a lot of those peaks, you basically have to start your algorithm in each of those peaks.
But the method of moments can actually start you in the right peak, and then you just move up by doing some local algorithm for maximum likelihood.
So that's not key.
But that's just if you want to think about algorithmically how I would end up doing this and how can I combine the two.
So I'll see you on Thursday.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
So welcome back.
So we are now moving to a new chapter, which is going to have a little more of a statistical flavor when it comes to designing methods, all right?
Because if you think about it, OK-- some of you have probably attempted problem number two in the problem set.
And you realize that maximum likelihood estimators does not give you super trivial estimators, right?
I mean, when you have an n theta theta, then the thing you get is not something you could have guessed before you actually attempted to solve that problem.
And so, in a way, we've seen already sophisticated methods.
However, in many instances, the maximum likelihood estimator was just an average.
And in a way, even if we had this confirmation for maximum likelihood that indeed that was the estimator that maximum likelihood would spit out, and that our intuition was therefore pretty good, most of the statistical analysis or use of the central limit theorems, all these things actually did not come in the building of estimator, in the design of the estimator, but really in the analysis of the estimator.
And you could say, well, if I know already that the best estimator is the average, I'm just going to use the average.
I don't have to, basically, quantify how good it is.
I just know it's the best I can do.
We're going to talk about tests.
And we're going to talk about parametric hypothesis testing.
So you should view this as-- parametric means, well, it's about a parameter, like we did before.
And hypothesis testing is on the same level as estimation.
And on the same level as estimator will be the word "test," OK?
And when we're going to devise a test, we're going to actually need to understand random fluctuations that arise from the central limit theorem better, OK?
It's not just going to be in the analysis.
It's also going to be in the design.
And everything we've been doing before in understanding the behavior of an estimator is actually going to come in and be extremely useful in the actual design of tests, OK?
So as an example, I want to talk to you about some real data.
I will not study this data.
But this data actually exist.
You can find it on R. And so, it's the data from the so-called credit union Cherry Blossom Run, which is a 10 mile race.
It takes place every year in D.C.
It seems that some of the years are pretty nice.
In 2009, there were about 15,000 participants.
Pretty big race.
And the average running time was 103.5 minutes, all right?
So about an hour and a half or a little bit more.
And so, you can ask the following question, right?
This is actual data, right?
103.5 actually averaged the running time for all of 15,000.
Now, this in practice, may not be something very suitable.
And you might want to just sample a few runners and try to understand how they're behaving every year without having to collect the entire data set.
And so, you could ask the question, well, let's say my budget is to ask for maybe 10 runners what their running time was.
I still want to be able to determine whether they were running faster in 2012 than in 2009.
Why do I put 2012, and not 2016?
Well, because the data set for 2012 is also available.
So if you are interested and you know how to use R, just go and have fun with it.
So to answer this question, what we do is we select n runners, right?
So n is a moderate number that's more manageable than 15,000.
From the 2012 race at random.
That's where the random variable is going to come from, right?
That's where we actually inject randomness in our problem.
So remember this is an experience.
So really in a way, the runners are the omegas.
And I'm interested in measurements on those guys.
So this is how I have a random variable.
And this random verbal here is measuring their running time.
OK.
If you look at the data set, you have all sorts of random variables you could measure about those random runners.
Country of origin.
I don't know, height, age, a bunch of things.
OK.
Here, the random variable of interest being the running time.
OK. Everybody understand what the process is?
OK.
So now I'm going to have to make some modeling assumptions.
And here, I'm actually pretty lucky.
I actually have all the data from a past year.
I mean, this is not the data from 2012, which I also have, but I don't use.
But I can actually use past data to try to understand what distribution do I have, right?
I mean, after all, running time is going to be rounded to something.
Maybe I can think of it as a discrete random variable.
Maybe I can think of it as the exponential random variable.
Those are positive numbers.
I mean, there's many kind of running times that could come up to mind.
Many kind of distributions I could think of for this modeling part.
But it turns out that if you actually plug the histogram of those running times for all 15,000 runners in 2009, you actually are pretty happy to see that it really looks like a bell-shaped curve, which suggest that this should be a Gaussian.
So what you go on to do is you estimate the mean from past observations, which was actually 103.5, as we said.
You submit the variance, which was 373.
And you just try to superimpose the curve with this one, which is a Gaussian PDF with mean 103.5 and variants 373.
And you see that they actually look very much alike.
And so here, you're pretty comfortable to say that the running time actually is Gaussian distribution.
All right?
So now I know that the x1 to xn, I'm going to say they're Gaussian, OK?
I still need to specify two parameters.
So what I want to know is, is the distribution the same from past years, right?
So I want to know if the random variable that I'm looking for-- if I, say, pick one.
Say, x1.
Does it have the same distribution in 2012 that it did in 2009?
OK. And so, the question is, is x1 has a Gaussian distribution with mean 103.5 and variance 373?
Is that clear?
OK.
So this question that calls for a yes or no answer is a hypothesis testing problem.
I am testing a hypothesis.
And this is the basis of basically all of data-driven scientific inquiry.
You just ask questions.
You formulate a scientific hypothesis.
Knocking down this gene is going to cure melanoma, is this true?
I'm going to collect.
I'm going to try.
I'm to observe some patients on which I knock down this gene.
I'm going to collect some measurements.
And I'm going to try to answer this yes/no question, OK?
It's different from the question, what is the mean running time for this year?
OK.
So this hypothesis testing is testing if this hypothesis is true.
The hypothesis in common English we just said, were runners running faster?
All right?
Anybody could formulate this hypothesis.
Now, you go to a statistician.
And he's like, oh, what you're really asking me is x1 has a Gaussian distribution with mean less than 103.5 and variance 373, right?
That's really the question that you ask in statistical terms.
And so, if you're asking if this was the same as before, there's many ways it could not be the same as before.
There's basically three ways it could not be the same as before.
It could be the case that x1 is in expectation to 103.5 So the expectation has changed.
Or the variance has changed.
Or the distribution has changed.
I mean, who knows?
Maybe runners are now just all running holding their hands.
And it's like now a point mass at 1 given point.
OK.
So you never know what could [INAUDIBLE].. Now of course, if you allow for any change, you will find change.
And so what you have to do is to factor in as much knowledge as you can.
Make as many modeling assumptions, so that you can let the data speak about your particular question.
Here, your particular question is, are they running faster?
So you're only really asking a question about the expectation.
You really want to know if the expectation has changed.
So as far as you're concerned, you're happy to make the assumption that the rest has been unchanged.
OK. And so, this is the question we're asking.
Is the expectation now less than 103.5?
Because you specifically asked whether runners were going faster this year, right?
They tend to go faster rather than slower, all right?
OK.
So this is the question we're asking in mathematical terms.
So first, when I did that, I need to basically fix the rest.
And fixing the rest is actually part of the modeling assumptions.
So I fixed my variance to be 373.
OK?
I assume that the variance has not changed between 2009 and 2012.
Now, this is an assumption.
It turns out it's wrong.
So if you look at the data from 2012, this is not the correct assumption.
But I'm just going to make it right now for the sake of argument, OK?
And also the fact that it's Gaussian.
Now, this is going to be hard to violate, right?
I mean, where did this bell-shaped curve come from?
Well, it's just natural when you just measure a bunch of things.
The central limit theorem appears in the small things of nature.
I mean, that's the bedtime story you get about the central limit theorem.
And that's why the bell-shaped curve is everywhere in nature.
It's the sum of little independent things that are going on.
And this Gaussian assumption, even if I wanted to relax it, there's not much else I can do.
It is pretty robust across the years.
All right.
So the only thing that we did not fix is the expectation of x1, which now I want to know what it is.
And since I don't know what it is, I'm going to call it mu.
And it's going to be a variable of interest, all right?
So it's just a number mu.
Whatever this is I can try to estimate it, maybe using maximum likelihood estimation.
Probably using the average, because this is Gaussian.
And we know that the maximum likelihood estimator for a Gaussian is just the average.
And now we only want to test if mu is equal to 103.5, like it was in 2009.
Or on the contrary, if mu is not equal to 103.5.
And more specifically, if mu is actually strictly less than 103.5.
That's the question you ask.
Now, why am I in writing mu equal to 103.5 or is less than 103.5 and equal to 103.5 versus not equal to 103.5?
It's because since you asked me a more precise question, I'm going to be able to give you a more precise answer.
And so, if your question is very specific-- are they running faster?
I'm going to factor that in what I write.
If you just ask me, is it the same?
I'm going to have to write, or is it different than 103.5?
And that's less information about what you're looking for, OK?
So by making all these modeling assumptions-- the fact that the variance doesn't change, the fact that it's still Gaussian-- I've actually reduced the number of.
And I put numbers in quotes, because this is still an infinite of them.
But I'm limiting the number of ways the hypothesis can be violated.
The number of possible alternative realities for this hypothesis, all right?
For example, I'm saying there's no way mu can be larger than 103.5.
I've already factored that in, OK?
It could be.
But I'm actually just going to say that if it's larger, all I'm going to be able to tell you is that it's not smaller.
I'm not going to be able to tell you that it's actually larger, OK?
And the only way it can be rejected now.
The only way I can reject my hypothesis is if x belongs to very specific family of distributions.
If it has a distribution which is Gaussian with mean mu and variance of 373 for mu, which is less 103.5.
All right?
So we started with basically was x1-- so that's the reality.
x1 follows n 103.5 373, OK?
And this is everything else, right?
So for example, here is x follows some exponential, 0.1, OK?
This is just another distribution here.
Those are all the possible distributions.
What we said is we said, OK, first of all, let's just keep only those Gaussian distributions, right?
And second, we said, well, among those Gaussian distributions, let's only look at those that have-- well, maybe this one should be at the boundary-- let's only look at the Gaussians here.
So this guy here are all the Gaussians with mean mu and variance 373 for mu less than 103.5, OK?
So when you're going to give me data, I'm going to be able to say, well, am I this guy?
Or am I one of those guys?
Rather than searching through everything.
And the more you search the easier for you to find something that fits better the data, right?
And so, if I allow everything possible, then there's going to be something that just by pure randomness is actually going to look better for the data, OK?
So for example, if I draw 10 random variables, right?
If n is equal to 10.
And let's say they take 10 different values.
Then it's actually more likely that those guys come from a discrete distribution that takes each of these values with probability 1 over 10, than actually some Gaussian random variable, right?
That would be perfect.
I can actually explain it.
If the 10 numbers I got were say-- let's say I collect 3, 90, 95, and 102.
Then the most likely distribution for those guys is the discrete distribution that takes three values, 91 with probability 1/3, 95 with probability 1/3, and 102 with probably 1/3, right?
That's definitely the most likely distribution for this.
So if I allowed this, I would say, oh no.
This is not distributed according to that.
It's distributed according to this very specific distribution, which is somewhere in the realm of all possible distributions, OK?
So now we're just going to try to carve out all this stuff by making our assumptions.
OK.
So here in this particular example, just make a mental note that what we're doing is that I actually-- a little birdie told me that the reference number is 103.5, OK?
That was the thing I'm actually looking for.
In practice, it's actually seldom the case that you have this reference for yourself to think of, right?
Maybe here, I just happen to have a full data set of all the runners of 2009.
But if I really just asked you, I said, were runners faster in 2012 than in 2009?
Here's $10 to perform your statistical analysis.
What you're probably going to do is called maybe 10 runners from 2012, maybe 15 runners from 2009, ask them and try to compare their mean.
There's no standard reference.
You would not be able to come up with this 103.5, because these data maybe is expensive to get or something.
OK.
So this is really more of the standard case, all right?
Where you really compare two things with each other, but there's no actual ground truth number that you're comparing it to.
OK.
So we'll come back to that in a second.
I'll tell you what the other example looks like.
So let's just stick to this example.
I tell you it's 103.5, OK?
Let's try to have our intuition work the same way.
We said, well, averages worked well.
The average, tell me, of over these 10 guys should tell me what the mean should be.
So I can just say, well x bar is going to be close to the true mean by the law of large number.
So I'm going to decide whether x bar is less than 103.5.
And conclude that in this case, indeed mu is less than 103.5, because those two quantities are close, right?
I could do that.
The problem is that this could go pretty wrong.
Because if n is small, then I know that xn bar is not equal to mu.
I know that xn bar is close to mu.
But I also know that there's pretty high chance that it's not equal to mu.
In particular, I know it's going to be somewhere at 1 over root n away from mu, right?
1 over root n being the root coming from what?
CLT, right?
That's the root n that comes from CLT.
In blunt words, CLT tells me the mean is at distance 1 over root n from the expectation, pretty much.
That's what it's telling.
So 1 over root n. If I have 10 people in there, 1 over root 10 is not a huge number, right?
It's like 1/3 pretty much.
So 1/3 103.5.
If the true mean was actually 103.4, but my average was telling me it's 103.4 plus 1/3, I would actually come to two different conclusions, right?
So let's say that mu is equal to 103.4, OK?
So you're not supposed to know this, right?
That's the hidden truth.
OK. Now I have n is equal to 10.
So I know that x bar n minus 103.4 is something of the order of 1 over the square root of 10, which is of the order of, say, 0.3.
OK.
So here, this is all hand wavy, OK?
But that's what the central limit theorem tells me.
What it means is that it is possible that x bar n is actually equal to is actually equal to 103.4 plus 0.3, which is equal to 103.7.
Which means that while the truth is that mu is less than 103.5, then I would conclude that mu is larger than 103.5, OK?
And that's because I have not been very cautious, OK?
So what we want to do is to have a little buffer to account for the fact that xn bar is not a precise value for the true mu.
It's something that's 1 over root n away from you.
And so, what we want is the better heuristic that says, well, if I want to conclude that I'm less than 103.5, maybe I need to be less than 103.5 minus a little buffer that goes to 0 as my sample size goes to infinity.
And actually, that's what the law of large number tells me.
The central limit theorem actually tells me that this should be true, something that goes to 0 as n goes to infinity and the rate 1 over root n, right?
That's basically what the central limit theorem tells me.
So to make this intuition more precise, we need to understand those fluctuations.
We need to actually put in something that's more precise than these little wiggles here, OK?
We need to actually have the central limit theorem come in.
So here is the example of comparing two groups.
So pharmaceutical companies use hypothesis testing to test if a drug is efficient, right?
That's what they do.
They want to know, does my new drug work?
And that's what the Federal Drug Administration office is doing on a daily basis.
They ask for extremely well regulated clinical trials on a thousand people, and check, does this drug make a difference?
Did everybody die?
Does it make no difference?
Should people pay $200 for a pill of sugar, right?
So that's what people are actually asking.
So to do so, of course, there is no ground truth about-- so there's actually a placebo effect.
So it's not like actually giving a drug that does not work is going to have no effect on patients.
It will have a small effect, but it's very hard to quantify.
We know that it's there, but we don't know what it is.
And so rather than saying, oh the ground truth is no improvement, the ground truth is the placebo effect.
And we need to measure what the placebo effect is.
So what we're going to do is we're going to split our patients into two groups.
And there's going to be what's called a test group and a control group.
So the word test here is used in a different way than hypothesis testing.
So we'll just call it typically the drug group.
And so, I will refer to mu drug for this guy, OK?
Now, this let's say this is a cough syrup, OK?
And when you have a cough syrup, the way you measure the efficacy of a cough syrup is to measure how many times you cough per minute, OK?
And so, if I define mu control the number of expectoration per hour.
So just the expected number, right?
This is the number I don't know, because I don't have access to the entire population of people that will ever take this cough syrup.
And so, I will call it mu control for the control group.
So those are the people who have been actually given just like sugar, like maple syrup.
And mu drug are those people who are given the actual syrup, OK?
And you can imagine that maybe maple syrup will have an effect on expectorations per hour just because, well, it's just sweet and it helps, OK?
And so, we don't know what this effect is going to be.
We just want to measure if the drug is actually having just a better impact on expectorations per hour than the just pure maple syrup, OK?
So what we want to know is if mu drug is less than mu control.
That would be enough.
If we had access to all the populations that will ever take the syrup for all ages, then we would just measure, did this have an impact?
And even if it's a slightly ever so small impact, then it's good to release this cough syrup, assuming that it has no side effects or anything like this, because it's just better than maple syrup, OK?
The problem is that we don't have access to this.
And we're going to have to make this decision based on samples that give me imprecise knowledge about mu drug and mu control.
So in this case, unlike the first case where we compared an unknown expected value to have a fixed number, which was one of the 103.5, here, we're just comparing two unknown numbers with each other, OK?
So there's two sources of randomness.
Trying to estimate the first one.
And trying to estimate the second one.
Before I move on, I just wanted to tell you I apologize.
One of the graders was not able to finish grading his problem sets for today.
So for those of you who are here just to pick up their homework, feel free to leave now.
Even if you have a name tag, I will pretend I did not read it.
OK.
So I'm sorry.
You'll get it on Tuesday.
And this will not happen again.
OK.
So for the clinical trials, now I'm going to collect information.
I'm going to collect the data from the control group.
And I'm going to collect data from the test group, all right?
So my control group here.
I don't have to collect the same number of people in the control group than in the drug group.
Actually, for cough syrup, maybe it's not that important.
But you can imagine that if you think you have the cure to a really annoying disease, it's actually hard to tell half of the people you will get a pill of nothing, OK?
People tend to want to try the drug.
They're desperate.
And so, you have to have this sort of imbalance between who is getting the drug and who's not getting the drug.
And people have to qualify for the clinical trials.
There's lots of fluctuations that affect what the final numbers of people who are actually going to get the drug and are going to get the control is going to be.
And so, it's not easy for you to make those two numbers equal.
You'd like to have those numbers equal if you can, but not necessarily.
And by the way, this is all part of some mystical science called "design of experiments."
And in particular, you can imagine that if one of the series had higher variants, you would want to like more people in this group than the other group.
Yeah?
So when we're subtracting [INAUDIBLE] something that [INAUDIBLE] 0 [INAUDIBLE] to be satisfied.
So that's on purpose [INAUDIBLE]
Yeah, that's on purpose.
And I'll come to that in a second, all right?
So basically, we're going to make it if your answer is, is this true?
We're going to make it as hard as possible, but no harder for you to say yes to this answer.
Because, well, we'll see why.
OK, so now we have two set of data, the x's and the y's.
The x's are the ones for the drug.
And the y's are the data that I collected from the people, who were just given a placebo, OK?
And so, they're all IID random variables.
And here, since it's the number of expectorations, I'm making a blunt modeling assumption.
I'm just going to say it's Poisson.
And it's characterized only by the mean mu drug or the mean mu control, OK?
I've just made an assumption here.
It could be something different.
But let's say it's a Poisson distribution.
So now what I want to know is to test whether mu drug is less than mu control.
We said that already.
But the way we said it before was not as mathematical as it is now.
Now we're actually making a test on the parameters of Poisson distribution.
Whereas before, we were just making test on expected numbers, OK?
So the heuristic-- again, if we try to apply the heuristic now.
Rather than comparing mu x bar drug to some fixed number, I'm actually comparing x bar drug to some control.
But now here, I need to have something that accounts for, not only the fluctuations of x bar drug, but also for the fluctuations of x bar control, OK?
And so, now I need something that goes to 0 when all those two things go to infinity.
And typically, it should go to zero with 1 over root of n drug and 1 over square root of n control, OK?
That's what the central limit theorem for both x bar drug and x bar control.
Two central limit theorems are actually telling.
OK. And then we can conclude that this happens.
And as you said, we're trying to make it a bit harder to conclude this.
Because let's face it.
If we were actually using two simple heuristic, right?
For simplicity, right?
So I can rewrite x bar drug less than x bar control minus this something that goes to 0.
I can write it as x bar drug minus x bar control less than something negative, OK?
This little something, OK?
So now let's look at those guys.
This is the difference of two random variables.
From the central limit theorem, they should be approximately Gaussian each.
And actually, we're going to think of them as being independent.
There's no reason why the people in the control group should have any effect on what's happening to the people in the test group.
Those people probably don't even know each other.
And so, when I look at this, this should look like n 0 with some mean and some variants, let's say I don't know what it is, OK?
The mean I actually know.
It's mu drug minus mu control, OK?
So if they were to plot the PDF of this guy, it would look like this.
I would have something which is centered at mu drug minus mu control.
And it would look like this, OK?
Now let's say that mu drug is actually equal to mu control.
That this pharmaceutical company is a huge scam.
And they really are trying to sell bottled corn syrup for $200 a pop, OK?
So this is a huge scam.
And the true things are actually equal to 0.
So this thing is really centered about 0, OK?
Now, if were not to do this, then basically, half of the time I would actually come up with a distribution that's above this value.
And half of the time I would have something that's below this value, which would mean that half of the scams would actually go through FDA if I did not do this.
So what I'm trying to do is to say, well, OK. You have to be here, so that there is actually a very low probability that just by chance you end up being here.
And we'll make all the statements extremely precise later on.
But I think the drug thing makes it interesting to see why you're making it hard, because You don't want to allow people to sell a thing like that.
Before we go more into the statistical thinking associated to tests, let's just see how we would do this quantification, right?
I mean after all, this is what we probably are the most comfortable with at this point.
So let's just try to understand this.
And I'm going to make the statisticians favorite test, which is the thing that obviously you do at home all the time every time you get a new quarter, is testing whether it's a fair coin or not.
All right?
So this test, of course, exists only in textbooks.
And I actually did not write this slide.
I was lazy to just replace all this stuff by the Cherry Blossom Run.
So you have a coin.
Now you have 80 observations, x1 to x80.
So n is equal to 80.
I have x1, xn, IID, Bernoulli p. And I want to know if I have a fair coin.
So in mathematical language, I want to know if p is equal to 1/2.
Let's say this is just the heads, OK?
And a biased coin?
Well, maybe you would potentially be interested whether it's biased one direction or the other.
But not being a fair coin is already somewhat of a discovery, OK?
And so, you just want to know whether p is equal to 1/2 or p is not equal to 1/2, OK?
Now, if I were to apply the very naive first example to not reject this hypothesis.
If I run this thing 80 times, I need to see exactly 40 heads and 40 tales.
Now this is very unlikely to happen exactly.
You're going to have close to 40 heads and close to 40 tails, but how close should those things be?
OK?
And so, the little something is going to be quantified by exactly this, OK?
So now here, let's say that my experiment gave me 54 heads.
That's 54?
Yeah.
Which means that my xn bar is 54 over 80, which is 0.68.
All right?
So I have this estimator.
Looks pretty large, right?
It's much larger than 0.5, so it does look like, and my mom would certainly conclude, that this is a biased coin for sure, because she thinks I'm tricky.
All right.
So the question is, can this be due to chance?
Can this be due to chance alone?
Like what is the likelihood that a fair coin would actually end up being 54 times on heads rather than 40?
OK?
And so, what we do is we say, OK, I need to understand, what is the distribution of the number of times it comes on heads?
And this is going to be a binomial, but it's a little annoying to play with.
So we're going to use the central limit theorem that tells me that xn bar minus p divided by square root of p1 minus p is approximately distributed as an n01.
And here, since n is equal to 80, I'm pretty safe that this is actually going to work.
And I can actually use [INAUDIBLE],, and put xn bar here.
[INAUDIBLE] tells me that this is OK to do.
All right.
So now I'm actually going to compute this.
So here, I know this.
This is square root of 80.
This is a 0.68.
What is this value here?
We'll talk about it.
Well, we're trying to understand what happens if it is a fair coin, right?
So if fair, then p is equal to 0.5, right?
So what I want to know is, what is the likelihood that a fair coin would give me 0.68?
Let me finish.
All right.
What is the likelihood that a fair coin will allow me to do this, so I'm actually allowed to plug-in p to be 0.5 here?
Now, your question is, why do I not plug-in p to be 0.5?
But you can.
All right.
I just want to make you plug-in p at one specific point, but you're absolutely right.
OK. Let's forget about your question for one second.
So now I'm going to have to look at xn bar minus 0.5 divided by xn bar 1 minus xn bar.
Then this thing is approximately Gaussian and 0,1 if the coin is fair.
Otherwise, I'm going to have a mean which is not zero here.
If the coin is something else, whatever I get here, right?
Let's just write it for one second.
Let's do it.
So what is the distribution of this if p-- so that's p is equal to 0.5.
OK?
Now if p is equal to 0.6, then this thing is just, well, I know that this is equal to square root of n xn bar minus 0.6, divided by xn bar 1 minus xn in the bar squared root, plus-- well, now the difference.
Is So square root of n, 0.6 minus 0.5, divided by square root of xn bar 1 minus xn bar, right?
Now if p is equal to 0.6, then this guy is n 0,1, but this guy is something different.
This is just a number that depends on square root of n. It's actually pretty large.
So if I want to use the fact that this guy has a normal distribution, I need to plug-in the true value here.
Now, the implicit question that I got was the following.
It says, well, if you know what p is, then what's actually true is also this.
If p is equal to 0.5, then since I know that root n xn bar minus p divided by square root of p 1 minus p is some n 0, 1, it's also true that square root of n xn bar minus 0.5 divided by square root of 0.5 1 minus 0.5 is n 0,1, right?
I know what p is.
I'm just going to make it appear.
OK. And so, what's actually nice about this particular [INAUDIBLE] experiment is that I can check if my assumption is valid by checking whether I'm actually-- so what I'm going to do right now is check whether this is likely to be a Gaussian or not, right?
And there's two ways I can violate it.
By violating mean, but also by violating the variance.
And here, what I did in the first case, I said, well I'm not allowing you to check whether you've violated the variance.
I'm just plugging whatever variance you're getting.
Whereas here, I'm saying, well, there's two ways you can violate it.
And I'm just going to factor everything in.
So now I can plug-in this number.
So this is 80.
This is 0.68.
So I can compute all this stuff.
I can compute all this stuff here as well.
And what I get in this case, if I put the xn bar 1, I get 3.45, OK?
And now I claim that this makes it reasonable to reject the hypothesis that p is equal to 0.5.
Can somebody tell me why?
It's pretty big
Yeah, 3 is pretty big.
So it's very unlikely.
So this number that I should see should look like the number I would get if I asked a computer to draw one random Gaussian for me.
This number, when I draw one random Gaussian, is actually a number with 99.9% this number will be between negative 3 and 3.
With 78% it's going to be between negative 2 and 2.
68% is between minus 1 and 1.
And with like 90% it's between minus 2 and 2.
So getting a 3.45 when you do this is extremely unlikely to happen, which means that you would have to be extremely unlucky for this to ever happen.
Now, it can happen, right?
It could be the case that you flip 80 coins and 80 of them are heads.
With what probability does this happen?
1 over 2 to the 80, right?
Which is probably better off playing the lottery with this kind of odds, right?
I mean, this is just not going to happen, but it might happen.
So we cannot remove completely the uncertainty, right?
It's still possible that this is due to noise.
But we're just trying to make all the cases that are very unlikely go away, OK?
And so, now I claim that 3.45 is very unlikely for a Gaussian.
So if I were to draw the PDF of a standard Gaussian, right?
So n 0, 1, right?
So that's PDF of n 0, 1.
3.73 is basically here, OK?
So it's just too far in the tails.
Understood?
Now I cannot say that the probability that the Gaussian is equal to 373 is small, right?
I just cannot say that, because it's 0.
And it's also 0 for the probability that it's 0, even though the most likely values are around 0.
It's the continuous random variable.
Any value you give me, it's going to happen with probability zero.
So what we're going to say is, well, the fluctuations are larger than this number.
The probability that I get anything worse than this is actually extremely small, right?
Anything worse than this is just like farther than 3.73.
And this is going to be what we control.
All right?
So in this case, I claim that it's quite reasonable to reject the hypothesis.
Is everybody OK with this?
Everybody find this shocking?
Or everybody has no idea what's going on?
Do you have any questions?
Yeah?
Regarding the case of p, where minus p isn't close to xn.
If you use 1 minus p as 0.5, then you're dividing by a larger number than you would if you used xn.
So it feels like our true number is not 3.45.
It's something a little bit smaller than 3.45 for the distribution to actually be like 1/2.
Because it seems like we're adding an unnecessary extra error by using xn bar.
And we're adding an error that makes it seem that our result was less likely than it actually was
That's correct.
And you're right.
I didn't want to plug-in the p everywhere, but you should plug it in everywhere you can.
That's for sure, OK?
So let's agree on that.
And that's true that it makes the number a little bigger.
You compute how much you would get, we would get if we 0.5 there.
Well, I don't know what the square root of 80 is.
Can somebody compute quickly?
I'm not asking you to do it.
But what I want is two times square root of 80 times 0.18.
3.22 OK.
I can make the same cartoon picture with 3.22.
But you're right.
This is definitely more accurate.
And I should have done this.
I didn't want to get the confused message, OK?
All right.
So now here's a second example that you can think of.
So now I toss it 30 times.
Still in the realm of the central limit theorem.
I get 13 heads rather than 15.
So I'm actually much closer to being exactly at half.
So let's see if this is actually going to give me a plausible value.
So I get 0.33 in average.
If the truth was 0.5, I would get something like 0.77.
And now I claim that 0.77 is a plausible realization for some standard Gaussian, OK?
Now, 0.77 is going to look like it's here.
So that could very well be something that just comes because of randomness.
And again, if you think about it.
If I told you, you were expecting 15, you saw 13, you're happy to put that on the account of randomness.
Now of course, the question is going to be, where do I draw the line?
Right?
Is 12 the right number?
Is 11?
Is 10?
What is it?
So basically, the answer is it's whatever you want to be.
The problem it's hard to think on the scale, right?
What does it mean to think on the scale?
If I can't think in this scale, I'm going to have to think on the scale of 80 of them.
I'm going to have to think on the scale of running 100 coin flips.
And so, this scale is a moving target all the time.
Every time you have a new problem, you have to have a new skill in mind.
And it's very difficult.
The purpose of statistical analysis, and in particular this process that content that takes your x bar and turns it into something that should be standard Gaussian, allows you to map the value of x bar into a scale that is the standard scale of the Gaussian.
All right?
Now, all you need to have in mind is, what is a large number or an unusually large number for a Gaussian?
That's all you need to know.
So here, by the way, 0.77 is not this one, because it was actually negative 0.77.
So this one.
OK.
So I can be on the right or I can be on the left of zero.
But they are still plausible.
So understand you could actually have in mind all the values that are plausible for a Gaussian and those that are not plausible, and draw the line based on what you think is the right number.
So how large should a positive value of a Gaussian to become unreasonable for you?
Is it 1?
Is it 1.5?
Is it 2?
Stop me when I get there.
Is it 2.5?
Is it 3?
I think 2.5 is definitely too big
What?
Doesn't it depend on our prior?
Let's say we already have really good evidence at this point [INAUDIBLE]
Yeah, so this is not Bayesian statistics.
So there's no such thing as a prior right now.
We'll get there.
You'll have your moment during one short chapter.
So there's no prior here, right?
It's really a matter of whether you think is a Gaussian large or not.
It's not a matter of coins.
It's not a matter of anything.
Now I've just reduced it to just one question.
So forget about everything we just said.
And I'm asking you, when do you decide that a number is too large to be reasonably drawn from a Gaussian?
And this number is 2 or 1.96.
And that's basically the number that you get from this quintel.
We've seen the 1.96 before, right?
It's actually q alpha over 2, where alpha is equal to 5%.
That's a quintel of a Gaussian.
So actually, what we do is we map it again.
So are now at the Gaussians.
And then we map it again into some probabilities, which is the probability of being farther than this thing.
And now probabilities, we can think.
Probability is something that quantifies my error.
And the question is what percentage of error am I willing to tolerate.
And if I tell you 5%, that's something you can really envision.
What it means is that if I were to do this test a million times, 5% of the time I would expose myself to making a mistake.
All right.
That's all it would say.
If you said, well, I don't want to account for 5%, maybe I want 1%, then you have to move from 1.94 to 2.5.
And then if you say at I want 0.01%, then you have to move to an even larger number.
So it depends.
But stating this number 1%, 5%, 10% is much easier than seeing those numbers 1.96, 2.5, et cetera.
So we're just putting everything back on the scale.
All right.
To conclude, this, again, as we said, does not suggest that the coin is unfair.
Now, it might be that the coin is unfair.
We just don't have enough evidence to say that.
And that goes back to your question about, why are we siding with the fact that we're making it harder to conclude that the runners were faster?
And this is the same thing.
We're making it harder to conclude that the coin is biased.
Because there is a status quo.
And we're trying to see if we have evidence against the status quo.
The status quo for the runners is they ran the same speed.
The status quo for the coin, we can probably all agree is that the coin is fair.
The status quo for a drug?
I mean, again, unless you prove me that you're actually not a scammer is that the status quo is that this is maple syrup.
There's nothing in there.
Why would you?
I mean, if I let you get away with it, you would put corn syrup.
It's cheaper.
OK.
So now let's move on to math.
All right.
So when I started doing mathematics, I'm going to have to talk about random variables and statistical models.
And here, there is actually a very simple thing, which actually goes back to this picture.
A test is really asking me if my parameter is in some region of the parameter set or another region of the parameter set, right?
Yes/no.
And so, what I'm going to be given is a sample, x1, xn.
I have a model.
And again, those can be braces depending on the day.
And so, now I'm going to give myself theta 0 and theta 1 to this joint subset.
OK.
So capital theta here is the space in which my parameter can live.
To make two disjoint subsets, I could just split this guy in half, right?
I'm going to say, well, maybe it's this guy and this guy.
OK.
So this is theta 0.
And this is theta 1.
What it means when I split those two guys, in test, I'm actually going to focus only on theta 0 or theta 1.
And so, it means that a priori I've already removed all the possibilities of theta being in this region.
What does it mean?
Go back to the example of runners.
This region here for the Cherry Blossom Run is the set of parameters, where mu was larger than 103.5, right?
We removed that.
We didn't even consider this possibility.
We said either it's less-- sorry.
That's mu equal to 103.5.
And this was mu less than 103.5, OK?
But these guys were like if it happens, it happens.
I'm not making any statement about that case.
All right?
So now I take those two subsets.
And now I'm going to give them two different names, because they're going to have an asymmetric role.
h0 is the null hypothesis.
And h1 is the alternative hypothesis.
h0 is the status quo.
h1 is what is considered typically as scientific discovery.
So if you're a regulator, you're going to push towards h0.
If you're a scientist, you're going to push towards h1.
If you're a pharmaceutical company, you're going to push towards h1.
OK?
And so, depending on whether you want to be conservative-- oh, I can find evidence in a lot of data.
As soon as you give me three data points, I'm going to be able to find evidence.
That means I'm going to tend to say, oh, it's h1.
But if you say you need a lot of data before you can actually move away from the status quo, that's age h0, OK?
So think of h0 as being status quo, h1 being some discovery that goes against the status quo.
All right?
So if we believe that the truth theta is either in one of those, what we say is we want to test h0 against h1.
OK.
This is actually wording.
So remember, because this is how your questions are going to be formulated.
And this is how you want to probably communicate as a statistician.
So you're going to say I have the null and I have an alternative.
I want to test h0 against h1.
I want to test the null hypothesis against the alternative hypothesis, OK?
Now, the two hypotheses I forgot to say are actually this.
h0 is that the theta belongs to theta 0.
And h1 is that it theta belongs to theta 1.
OK.
So here, for example, theta was mu.
And that was mu equal to 103.5.
And this was mu less than 103.5.
OK?
So typically, they're not going to look like thetas and things like that.
They're going to look like very simple things, where you take your usual notation for your usual parameter and you just say in mathematical terms what relationship this should be satisfying, right?
For example, in the drug example, that would be mu drug is equal to mu control.
And here, that would be mu drug less than mu control.
The number of expectorations for people who take the drug for the cough syrup is less than the number of expectoration of people who take the corn syrup, OK?
So now what we want to do.
We've set up our hypothesis testing problem.
You're a scientist.
You've set up your problem.
Now what you're going to do is collect data.
And what you're going to try to find on this data is evidence against h0.
And the alternative is going to guide you into which direction you should be looking for evidence against this guy.
All right?
And so, of course, the narrower the alternative, the easier it is for you, because you just have to look at the one possible candidate, right?
But typically, h1 is a big group, like less than.
Nobody tells you it's either it's 103.5 and 103.
People tell you it's either 103.5 or less than 103.5.
OK. And so, what we want to do is to decide whether we reject h0.
So we look for evidence against h0 in the data, OK?
So as I said, h0 and h1 do not play a symmetric role.
It's very important to know which one you're going to place as h0 and which one you're going to place at h1.
If it's a close call, you're always going to side with h0, OK?
So you have to be careful about those.
You have to keep that in mind that if it's a close call, if data does not carry a lot of evidence, you're going to side with h0.
And so, you're actually never saying that h0 is true.
You're just saying I did not find evidence against h0.
You don't say I accept that h0.
You say I failed to reject h0.
OK. And so one of the things that you want to keep in mind when you're doing this is this innocent until proven guilty.
So if you come from a country, like America, there's such a thing.
And in particular, lack of evidence does not mean that you are not guilty, all right?
OJ Simpson was found not guilty.
It was not found innocent, OK?
And so, this is basically what happens is like the prosecutor brings their evidence.
And then the jury has to decide whether they were convinced that this person was guilty of anything.
And the question is, do you have enough evidence?
But if you don't have evidence, it's not the burden of the defender to prove that they're innocent.
Nobody's proving their innocent.
I mean, sometimes it helps.
But you just have to make sure that there's not enough evidence against you, OK?
And that's basically what it's doing.
You're h0 until proven h1.
So how are we going to do this?
Well, as I said, the role of estimators in hypothesis testing is played by something called tests.
And a test is a statistic.
Can somebody remind me what a statistic is?
Yep?
The measure [INAUDIBLE]
Yeah, that's actually just one step more.
So it's a function of the observations.
And we require it to be measurable.
And as a rule of thumb, measurable means if I give you data, you can actually compute it, OK?
If you don't see a [INAUDIBLE] or an [INAUDIBLE],, you don't have to think about it.
All right.
And so, what we do is we just have this test.
But now I'm actually asking only from this test a yes/no answer, which I can code as 0, 1, right?
So as a rule of thumb, you say that, well, the test is equal to 0 then h0.
The test is equal to 1 at h1.
And as we said, is that if the test is equal to 0, it doesn't mean that a 0 is truth.
It means that I feel to rejected h0.
And if the test is equal to 1, I reject h0.
So I have two possibilities.
I look at my data.
I turn it into a yes/no answer.
And yes/no answer is really h0 or h1, OK?
Which one is the most likely basically.
All right.
So in the coin flip example, our test statistic is actually something that takes value 0, 1.
And anything, any function that takes value at 0, 1 is an indicator function, OK?
So an indicator function is just a function.
So there's many ways you can write it.
So it's a 1 with a double bar.
If you aren't comfortable with this, it's totally OK to write i of something, like i of a. OK. And that's what?
So a, here, is a statement, like an inequality, an equality, some mathematical statement, OK?
Or not mathematical.
I mean, "a" can be, you know, my grandma is 20 years old, OK?
And so, this is basically 1 if a is true, and 0 if a is false.
That's the way you want to think about it.
This function takes only two values, and that's it.
So here's the example that we had.
We looked at whether the standardized xn bar, the one that actually is approximately n 0,1 was larger than something in absolute value, either very large or very small, but negative.
I'm going back to this picture.
We wanted to know if this guy was either to the left of something or to the right of something, right?
Was it in these regions?
Now this indicator, I can view this as a function of x bar.
What it does, it really splits the possible values of x bar, which is just a real number, right?
In two groups.
The groups on which they lead to a value, which is 1.
And the groups on which they lead to value, which is 0, right?
So what it does is that I can actually think of it as the real line, x bar.
And there's basically some values here, where I'm going to get a 1.
Maybe I'm going to get a 0 here.
Maybe I'm going to get a 0.
Maybe I'm going to get a 1.
I'm just splitting all possible values of x bar.
And I see whether to spit out the side which is 0 or which is 1.
In this case, it's not clear, right?
I mean, the function is very nonlinear.
It's x bar minus 0.5 divided by the square root of x bar 1 minus x bar.
If we put the p in the denominator, that would be clear.
That would just be exactly something that looks like this.
The function would be like this.
It would be 1 if it's smaller than some value.
Less than 0 if it's in between two values.
And then 1 again.
So that's psi, OK?
So this is 1, right?
This is 1.
And this is 0.
So if x bar is too small or if x bar is too large, then I'm getting a value 1.
But if it's somewhere in between, I'm getting a value 0.
Now, if I have this weird function, it's not clear how this happened.
So the picture here that I get is that I have a weird non-linear function, right?
So that's x bar.
That's square root of n x bar n 0.5 divided by the square root of x bar n 1 minus x bar n, right?
That's this function.
A priori, I have no idea what this function looks like.
We can probably analyze this function, but let's pretend we don't know.
So it's like some crazy stuff like this.
And all I'm asking is whether in absolute value it's larger than c, which means that is this function larger than c or less than minus c?
The intervals on which I'm going to say 1 are this guy, this guy, this guy, and this guy.
OK. And everywhere else, I'm seeing 0.
Everybody agree with this?
This is what I'm doing.
Now of course, it's probably easier for you to just package it into this nice thing that's just either larger than c, an absolute value, or less Than C. I want to have to plot this function.
In practice, you don't have to.
Now, this is where I am actually claiming.
So here, I actually defined to you a test.
And I promised, starting this lecture, by saying, oh, now we're going to do something better than computing the averages.
Now I'm telling you it's just computing an average.
And the thing is the test is not just the specification of this x bar.
It's also the specification of this constant c. All right?
And the constant c was exactly where our belief about what a large value for a Gaussian is.
That's exactly where it came in.
So this choice of c is basically a threshold at which we decide above this threshold this isn't likely to come from a Gaussian.
Below this threshold we decide that it's likely to come from a Gaussian.
So we have to choose what this threshold is based on what we think likely means.
Just a little bit more of those things.
So now we're going to have to characterize what makes a good test, right?
Well, I'll come back to it in a second.
But you could have a test that says reject all the time.
And that's going to be bad test, right?
The FDA is not implementing a test that says, yes all drugs work, now let's just go to Aruba, OK?
So people are trying to have something that tries to work all the time.
Now FDA's not either saying, let's just say that no drugs work, and let's go to Aruba, all right?
They're just trying to say the right thing as often as possible.
And so, we're going to have to measure this.
So the things that are associated to a test are the rejection region.
And if you look at this x in en, such that psi of x is equal to 1, this is exactly this guy that I drew.
So here, I summarized the values of the sample into their average.
But the values of the sample that I collect will lead to a test that says 1.
All right?
So this is the rejection region.
If I collect a data point, technically I have-- so I have e to the n, which is a big space like this.
So that's e to the n. Think of it as being the space of xn bars.
And I have a function that takes only value 0, 1.
So I can decompose it into this part where it takes value 0 and the part where it takes value 1.
And those can be super complicated, right?
Can have a thing like this.
Can have some weird little islands where it takes value 1.
I can have some islands where it's takes value 0.
I can have some weird stuff going on.
But I can always partition it into the value where it takes value 0 and the value where it takes value 1.
And the value where it takes 1, if psi is equal to 1, this is called the rejection region of the plot, OK?
So just the samples that would lead me to rejecting.
And notice that this is the indicator of the rejection region.
The test is the indicator of the rejection region.
So there's two ways you can make an error when there's a test.
Either the truth is in h1, and you're saying actually it's h1.
Or the truth is in h1, and you say it's h0.
And that's how we build-in the asymmetry between h0 and h1.
We control only one of the two errors.
And we hope for the best for the second one.
So the type 1 error is the one that says, well, if it is actually the status quo, but a claim that there is a discovery-- if it's actually h0, but I claim that I'm in h1, then I admit I commit a type I error.
And so the probability of type I error is this function alpha of psi, which is the probability of saying that psi is equal to 1 when theta is in h0.
Now, the problem is that this is not just number, because theta is just like moving all over h0, right?
There's many values that theta can be, right?
So theta is somewhere here.
I erased it, OK. All right.
For simplicity, we're going to think of theta as being mu and 103.5, OK?
And so, I know that this is theta 1.
And just this point here was theta 0, OK?
Agreed?
This is with the Cherry Blossom Run.
Now, here in this case, it's actually easy.
I need to compute this function alpha of psi, which maps theta in theta 0 to p theta of psi equals 1.
So that's the probability that I reject when theta is in h0.
Then there's only one of them to compute, because theta can only take this one value.
So this is really 103.5.
OK.
So that's the probability that I reject when the true mean was 103.5.
Now, if I was testing whether-- if h0 was this entire guy here, all the values larger than 103.5, then I would have to compute this function for all possible values of the theta in there.
And guess what?
The worst case is when it's going to be here.
Because it's so close to the alternative that that's where I'm making the most error possible.
And then there's the type 2 error, which is defined basically in the symmetric ways.
The function that maps theta to the probability.
So that's the probability of type 2 errors.
The probability that I fail to reject h0, right?
If psi is equal to 0, I fail to reject h0.
But that actually came from h1, OK?
So in this example, let's clear.
If I'm here, like if the true mean was 100, I'm looking at the probability that the true mean is actually 100, and I'm actually saying it was 103.5.
Or it's not less than 103.5.
Yeah?
I'm just still confused by the notation.
When you say that [INAUDIBLE] theta sub 1 arrow r, I'm not sure what that notation means
Well, this just means it's a function that maps theta 0 to r. You've seen functions, right?
OK.
So that's just the way you write.
So that means that's a function f that goes from, say, r r, and that maps x to x squared.
OK.
So here, I'm just saying I don't have to consider all possible values.
I'm only considering the values on theta 0.
I put r actually.
I could restrict myself to the interval 0, 1, because those are probabilities.
So it's just telling me where my function comes from and where my function goes to.
And beta is a function, right?
So beta psi of theta is just the probability that theta is equal to 1.
And I could define that for all thetas-- sorry.
If psi is equal to 0 in this case.
And that could define that for all thetas.
But the only ones that lead to an error are the thetas that are in h1.
I mean, I can define this function.
It's just not going to correspond to an error, OK?
And the power of a test is the smallest-- so the power is basically 1 minus an error.
1 minus the probability of an error.
So it's the probability of making a correct decision, OK?
So it's the probability of making a correct decision under h1, that's what the power is.
But again, this could be a function.
Because there's many ways that can be in h1 if h1 is an entire set of numbers.
For example, all the numbers there are less than 103.5.
And so, what I'm doing here when I define the power of a test, I'm looking at the smallest possible of those values, OK?
So I'm looking at this function.
Maybe I should actually expand a little more on this.
OK.
So beta psi of theta is the probability under theta that psi is equal to 0, right?
That's the probability in theta 1, which means then the alternative, that they feel to reject.
And I really should, because theta was actually in theta 1, OK?
So this thing here is the probability of type 2 error.
Now, this is 1 minus the probability that I did reject and I should have rejected.
That's just a little off the complement.
Because if psi is not equal to 0, then it's equal to 1.
So now if I rearrange this, it tells me that the probability that psi is equal to 1-- this is actually 1 minus beta psi of theta.
So that's true for all thetas in theta 1.
And what I'm saying is, well, this is now a good thing, right?
This number being large is a good thing.
It means I should have rejected, and I rejected.
I want this to happen with large probability.
And so, what I'm going to look at is the most conservative choice of this number, right?
Rather than being super optimistic and say, oh, but indeed if theta was actually equal to zero, then I'm always going to conclude that-- I mean, if mu is equal to 0, everybody runs in 0 seconds, then I with high probability I'm actually going to make no mistake.
But really, I should look at the worst possible case, OK?
So what I'm looking at is basically the smallest value it can take on theta one is called power of psi.
Power of the test psi, OK?
So that's the smallest possible value it can take.
All right.
So I'm sorry.
This is a lot of definitions that you have to sink in.
And it's not super pleasant.
But that's what testing is.
There's a lot of jargon.
Those are actually fairly simple things.
Just maybe you should get a sheet for yourself.
And say, these are the new terms that I learned.
What is their test, rejection region?
Probably of type I error, probably of type 2 error, and power.
Just make sure you know what those guys are.
Oh.
And null and alternative hypothesis, OK?
And once you know all these things, you know what I'm talking about.
You know what I'm referring to.
And this is just jargon.
But in the end, those are just probabilities.
I mean, these a natural quantities.
Just for some reason, people have been used to using different terminology.
So just to illustrate.
When do I make a typo 1 error?
And when do I not make a type 1 error?
So I make a type 1 error if h0 is true and I reject h0, right?
So the off diagonal blocks are when I make an error.
When I'm on the diagonal terms, h1 is true and I reject h0, that's a correct decision.
When h0 is true and I fail to reject h0, that's also the correct decision to make.
So I only make errors when I'm in one of the red blocks.
And one block is the type 1 error and the other block is the type 2 error.
That's all it means, OK?
So you just have to know which one we called one.
I mean, this was chosen in a pretty ad hoc way.
So to conclude this lecture, let me ask you a few questions.
If in a US court, the defendant is found either say, let's just say for the sake of discussion, innocent or guilty.
All right?
It's really guilty for not guilty, but let's say innocent or guilty.
When does the jury make a type 1 error?
Yep?
And he's guilty?
And he's innocent, right?
The status quo, everybody is innocent until proven guilty.
So that's our h0 is that the person is innocent.
And so, that means that h0 is innocent.
And so, we're looking at the probably of type 1 error, so that's when we reject the fact that it's innocent.
So conclude that this person is guilty, OK?
So type 1 error is when this person is innocent and we conclude it's guilty.
What is the type 2 error?
Letting a guilty person go free, which actually according to the constitution, is the better of the two.
All right?
So what we're going to try to do is to control the first one, and hope for the best for the second one.
How could the jury make sure that they make no type 1 error ever?
Always let the guy go free, right?
What is the effect on the type 2 error?
Yeah, it's the worst possible, right?
I mean, basically, for every guy that's guilty, you let them go.
That's the worst you can do.
And same thing, right?
How can the jury make sure that there's no type 2 error?
Always convict.
What is the effect on the American budget?
What is the effect on the type 1 error?
Right.
So the effect is that basically the type 1 error is maximized.
So there's this trade off between type 1 and type 2 error that's inherent.
And that's why we have this sort of multi objective thing.
We're trying to minimize two things at the same time.
And I can't find many ad hoc ways, right?
So if you've taken any optimization, trying to optimize two things when one is going up while the other one is going down, the only thing you can do is make ad hoc heuristics.
Maybe you try to minimize the sum of those two guys.
Maybe you try to minimize 1/3 of the first guy plus 2/3 of the second guy.
Maybe you try to minimize the first guy plus the square of the second guy.
You can think of many ways, but none of them is more justified than the other.
However, for statistical hypothesis testing, there's one that's very well justified, which is just constrain your type 1 error to be the smallest, to be at a level that you deem acceptable.
5%.
I want to convict at most 5% of innocent people.
That's what I deem reasonable.
And based on that, I'm going to try to convict as many people as they can, all right?
So that's called the Nieman Pearson paradigm, and we'll talk about it next time.
All right.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're talking about tests.
And to be fair, we spend most of our time talking about new jargon that we're using.
The main goal is to take a binary decision, yes and no.
So just so that we're clear and we make sure that we all speak the same language, let me just remind you what the key words are for tests.
So the first thing is that we split theta in theta 0 and theta 1.
Both are included in theta, and they are disjoint.
So I have my set of possible parameters.
And then I have theta 0 is here, theta 1 is here.
And there might be something that I leave out.
And so what we're doing is, we have two hypotheses.
So here's our hypothesis testing problem.
And it's h0 theta belongs to theta 0 versus h1 theta belongs to theta 1.
This guy was called the null, and this guy was called the alternative.
And why we give them special names is because we saw that they have an asymmetric role.
The null represents the status quo, and data is here to bring evidence against this guy.
And we can really never conclude that h0 is true because all we could conclude is that h1 is not true, or may not be true.
So that was the first thing.
The second thing was the hypothesis.
The third thing is, what is a test?
Well, psi, it's a statistic, and it takes the data, and it maps it into 0 or 1.
And I didn't really mention it, but there's some things such as called randomized tests, which is, well, if I cannot really make a decision, I might as well flip a coin.
That tends to be biased, but that's really-- I mean, think about it in practice.
You probably don't want to make decisions based on flipping a coin.
And so what people typically do-- this is happening, typically, at one specific value.
So rather than flipping a coin for this very specific value, what people typically do is they say, OK, I'm going to side with h0 because that's the most conservative choice I can make.
So in a way, they think of flipping this coin, but always falling on heads, say.
So associated to this test was something called, well, the rejection region r psi, which is just the set of data x1 xn such that psi of x1 xn is equal to 1.
So that means we rejected h0 when the test is 1.
And those are the set of data points that actually are going to lead me to reject the test.
And then the things that we're actually, slightly, a little more important and really peculiar to test, specific to tests, were the type I and type II error.
So the type I error arises when-- so type I error is when you reject, whereas h0 is correct.
And the type II error is the opposite, so it's failed to reject, whereas h1 is correct-- h is correct, yeah.
So those are the two types of errors you can make.
And we quantified their probability of type I error.
So alpha psi is the probability-- so that's the probability of type I error.
So psi is just the probability for theta that psi rejects and that's defined for theta and theta 0, so for different values of theta 0.
So h0 being correct means there exists a theta in theta 0 for which that actually is the right distribution.
So for different values of theta, I might make different errors.
So if you think, for example, about the coin example, I'm testing if the coin is biased towards heads or biased towards tails.
So if I'm testing whether p is larger than 1/2 or less than 1/2, then when the true p-- let's say our h0 is larger than 1/2.
When p is equal to 1, it's actually very difficult for me to make a mistake, because I only see heads.
So when p is getting closer to 1/2, I'm going to start making more and more probability of error.
And so the type II error-- so that's the probability of type II-- is denoted by beta psi.
And it's the function, well, that does the opposite and, this time, is defined for theta in theta 1.
And finally, we define something called the power, pi of psi.
And this time, this is actually a number.
And so this number is equal to the maximum over theta n theta 0.
I mean, that could be a supremum, but think of it as being a maximum of p theta of psi is equal-- sorry, that's n0, right?
Give me one sec.
No, sorry, that's the min.
So this is not making a mistake.
Theta 0 is in theta 2 So if theta is in theta 1 and I conclude 1, so this is a good thing.
I want this number to be large.
And I'm looking at the worst house-- what is the smallest value this number can be?
So what I want to show you a little bit is a picture.
So now I'm going to take theta, and think of it as being a p. So I'm going to take p for some variable in the experiment.
So p can range between 0 and 1, that's for sure.
And what I'm going to try to test is whether p is less than 1/2 or larger than 1/2.
So this is going to be, let's say, theta 0.
And this guy here is theta 1.
Just trying to give you a picture of what those guys are.
So I have my y-axis, and now I'm going to start drawing number.
All these things-- this function, this function, and this number-- are all numbers between 0 and 1.
So now I'm claiming that-- so when I move from left to right, what is my probability of rejecting going to do?
So what I'm going to plot is the probability under theta.
The first thing I want to plot is the probability under theta that psi is equal to 1.
And let's say psi-- think of psi as being just this indicator that square root on n xn bar minus p over square root xn bar 1 minus xn bar is larger than some constant c for a probability chosen c. So what we choose is that c is in such a way that, at 1/2, when we're testing for 1/2, what we wanted was this number to be equal to alpha, basically.
So we fix this alpha number so that this guy-- so if I want alpha of psi of theta less than alpha given in advanced-- so think of it as being equal to, say, 5%.
So I'm fixing this number, and I want this to be controlled for all theta and theta 0.
So if you're going to give me this budget, well, I'm actually going to make it equal where I can.
If you're telling me you can make it equal to alpha, we know that if I increase my type I error, I'm going to decrease my type II error.
If I start putting everyone in jail or if I start letting everyone go free, that's what we were discussing last time.
So since we have this trade-off and you're giving me a budget for one guy, I'm just going to max it out.
And where am I going to max it out?
Exactly at 1/2 at the boundary.
So this is going to be 5%.
So what I know is that since alpha of theta is less than alpha for all theta in theta 0-- sorry, that's for theta 0, that's where alpha is defined.
So for theta and theta 0, I knew that my function is going to look like this.
It's going to be somewhere in this rectangle.
Everybody agrees?
So this function for this guy is going to look like this.
When I'm at 0, when p is equal to 0, which means I only observe 0's, then I know that p is going to be 0, and I will certainly not conclude that p is equal to 1.
This test will never conclude that p is equal to 1-- that p is larger than 1/2, just because xn bar is going to be equal to 0.
Well, this is actually not well-defined, so maybe I need to do something-- put it equal to 0 if xn bar is equal to 0.
So I guess, basically, I get something which is negative, and so it's never going to be larger than what I want.
And so here, I'm actually starting at 0.
So now, this is this function here that increases-- I mean, it should increase smoothly.
This function here is alpha psi of theta-- or alpha psi of p, let's say, because we're talking about p. Then it reaches alpha here.
Now, when I go on the other side, I'm actually looking at beta.
When I'm on theta 1, the function that matters is the probability of type II error, which is beta of psi.
And this beta of psi is actually going to increase.
So beta of psi is what?
Well, beta of psi should also-- sorry, that's the probability of being equal to alpha.
So what I'm going to do is I'm going to look at the probability of rejecting.
So let me draw this functional all the way.
It's going to look like this.
Now here, if I look at this function here or here, this is the probability under theta that psi is equal to 1.
And we just said that, in this region, this function is called alpha of psi.
In that region, it's not called alpha of psi.
It's not called anything.
It's just the probability of rejection.
So it's not any error, it's actually what you should be doing.
What we're looking at in this region is 1 minus this guy.
We're looking at the probability of not rejecting.
So I need to actually, basically, look at the 1 minus this thing, which here is going to be 95%.
So I'm going to do 95%.
And this is my probability.
Ability And I'm just basically drawing the symmetric of this guy.
So this here is the probability under theta that psi is equal to 0, which is 1 minus p theta that psi is equal to 1.
So it's just 1 minus the wide curve.
And it's actually, by definition, equal to beta of psi of theta.
Now, where do I read pi psi?
What is pi psi on this picture?
Is pi psi a number or a function?
Number
It's a number, right?
It's the minimum of a function.
What is this function?
It's the probability under theta that theta is equal to 1.
I drew this entire function for between theta 0 and theta 1.
I drew-- this is this entire white curve.
This is this probability.
Now I'm saying, look at the smallest value this probability can take on the set theta 1.
What is this?
This guy.
This is where my pi-- this thing here is pi psi, and so it's equal to 5%.
So that's for this particular test, because this test has a continuous curve for this psi.
And so if I want to make sure that I'm at 5% when I come to the right of the theta 0, if it touches theta 1, then I'd better have 5% on the other side if the function is continuous.
So basically, if this function is increasing, which will be the case for most tests, and continuous, then what's going to happen is that the level of the test, which is alpha, is actually going to be equal to the power of the test.
Now, there's something I didn't mention, and I'm just mentioning it passing by.
Here, I define the power itself.
This function, this entire white curve here, is actually called the power function-- this thing.
That's the entire white curve.
And what you could have is tests that have the entire curve which is dominated by another test.
So here, if I look at this test-- and let's assume I can build another test that has this curve.
Let's say it's the same here, but then here, it looks like this.
What is the power of this test?
It's the same
It's the same.
It's 5%, because this point touches here exactly at the same point.
However, for any other value than the worst possible, this guy is doing better than this guy.
Can you see that?
Having a curve higher on the right-hand side is a good thing because it means that you tend to reject more when you're actually in h1.
So this guy is definitely better than this guy.
And so what we say, in this case, is that the test with the dashed line is uniformly more powerful than the other tests.
But we're not going to go into those details because, basically, all the tests that we will describe are already the most powerful ones.
In particular, this guy is-- there's no such thing.
All the other guys you can come up with are going to actually be below.
So we saw a couple tests, then we saw how to pick this threshold, and we defined those two things
Question
Yes?
But in that case, the dashed line, if it were also higher in the region of theta 0, do you still consider it better?
Yeah

Because you're given this budget of 5%.
So in this paradigm where you're given the-- actually, if the dashed line was this dashed line, I would still be happy.
I mean, I don't care what this thing does here, as long as it's below 5%.
But here, I'm going to try to discover.
Think about, again, the drug discovery example.
You're trying to find-- let's say you're a scientist and you're trying to prove that your drug works.
What do you want to see?
Well, FDA puts on you this constraint that your probability of type I error should never exceed 5%.
You're going to work under this assumption.
But what you're going to do is, you're going to try to find a test that will make you find something as often as possible.
And so you're going to max this constraint of 5%.
And then you're going to try to make this curve, that means-- this is, basically, this number here, for any point here, is the probability that you publish your paper.
That's the probability that you can release to market your drug.
That's the probability that it works.
And so you want this curve to be as high as possible.
You want to make sure that if there's evidence in the data that h1 is the truth, you want to squeeze as much of this evidence as possible.
And the test that has the highest possible curve is the most powerful one.
Now, you have to also understand that having two curves that are on top of each other completely, everywhere, is a rare phenomenon.
It's not always the case that there is a test that's uniformly more powerful than any other test.
It might be that you have some trade-off, that it might be better here, but then you're losing power here.
Maybe it's-- I mean, things like this.
Well, actually, maybe it should not go down.
But let's say it goes like this, and then, maybe, this guy goes like this.
Then you have to, basically, make an educated guess whether you think that the theta you're going to find is here or is here, and then you pick your test.
Any other question?
Yes?
Can you explain the green curve again?
That's just the type II error?
So the green curve is-- exactly.
So that's beta psi of theta.
So it's really the type II error.
And it's defined only here.
So here, it's not a definition, it's really I'm just mapping it to this point.
So it's defined only here, and it's the probability of type II error.
So here, it's pretty large.
I'm making it, basically, as large as I could because I'm at the boundary, and that means, at the boundary, since the status quo is h0, I'm always going to go for h0 if I don't have any evidence, which means that what's going to pay is the type II error that's going to basically pay this.
Any other question?
So let's move on.
So did we do this?
No, I think we stopped here, right?
I didn't cover that part.
So as I said, in this paradigm, we're going to actually fix this guy to be something.
And this thing is actually called the level of the test.
I'm sorry, this is, again, more words.
Actually, the good news is that we split it into two lectures.
So we have, what is a test?
What is a hypothesis?
What is the null?
What is the alternative?
What is the type I error?
What is the type II error?
And now, I'm telling you there's another thing.
So we define the power, which was some sort of a lower bound on the-- or it's 1 minus the upper bound on the type II error, basically.
And so it's alternative-- so the power is the smallest probability of rejecting when you're in the null.
And it's alternative when you're in theta 1, so that's my power.
I looked here, and I looked at the smallest value.
And I can look at this side and say, well, what is the largest probability that I make a type I error?
Again, this largest probability is the level of the test.
So this is alpha equal, by definition, to the maximum for theta in theta 0 of alpha psi of theta.
So here, I just put the level itself.
As you can see, here, it essentially says that if I'm of level of 5%, I'm also of level 10%, I'm also of level 15%.
So here, it's really an upper bound.
Whatever you guys want to take, this is what it is.
But as we said, if this number is 4.5%, you're losing in your type II error.
So if you're allowed to have-- if this maximum here is 4.5% and FDA told you you can go to 5%, you're losing in your type II error.
So you actually want to make sure that this is the 5% that's given to you.
So the way it works is that you give me the alpha, then I'm going to go back, pick c that depends on alpha here, so that this thing is actually equal to 5%.
And so of course, in many instances, we do not know the probability.
We do not know how to compute the probability of type I error.
This is a maximum value for the probability of type I error.
We don't know how to compute it.
I mean, it might be a very complicated random variable.
Maybe it's a weird binomial.
We could compute it, but it would be painful.
But we know how to compute is its asymptotic value.
Just because of the central limit theorem, convergence and distribution tells me that the probability of type I error is basically going towards the probability that some Gaussian is in some region.
And so we're going to compute, not the level itself, but the asymptotic level.
And that's basically the limit as n goes to infinity of alpha psi of theta.
And then I'm going to make the max here.
So how am I going to compute this?
Well, if I take a test that has rejection region of the form tn-- because it depends on the data, that's tn of x1 xn-- my observation's larger than some number c. Of course, I can almost always write tests like that, except that sometimes, it's going to be an absolute value, which essentially means I'm going away from some value.
Maybe, actually, I'm less than something, but I can always put a negative sign in front of everything.
So this is not without much of generality.
So this includes something that looks like-- something is larger than the constants, so that means-- which is equivalent to-- well, let me write that as tq, because then that means that-- so that's tn.
But this actually encompasses the fact that qn is larger than c or qn is less than c and n minus c. So that includes this guy.
That also includes qn less than c, because this is equivalent to qn is larger than minus c. And minus qn is-- and so that's going to be my tn.
So I can actually encode several type of things-- rejection regions.
So here, in this case, I have a rejection region that looks like this, or a rejection region that looks like this, or a rejection region that looks like this.
And here, I don't really represent it for the whole data, but maybe for the average, for example, or the normalized average.
So if I write this, then-- yeah.
And in this case, this tn that shows up is called test statistic.
I mean, this is not set in stone.
Here, for example, q could be the test statistic.
It doesn't have to be minus q itself that's the test statistic.
So what is the test statistic?
Well, it's what you're going to build from your data and then compare to some fixed value.
So in the example we had here, what is our test statistic?
Well, it's this guy.
This was our test statistic.
And is this thing a statistic?
What are the criteria for a statistic?
What is the statistic?
I know you know the answer
Measurable function
Yeah, it's a measurable function of the data that does not depend on the parameter.
Is this guy a statistic?
It's not
Let's think again.
When I implemented the test, what did I do?
I was able to compute my test.
My test did not depend on some unknown parameter.
How did we do it?
We just plugged in 0.5 here, remember?
That was the value for which we computed it, because under h0, that was the value we're seeing.
And if theta 0 is actually an entire set, I'm just going to take the value that's the closest to h1.
We'll see that in a second.
I mean, I did not guarantee that to you.
But just taking the worst type I error and bounded by alpha is equivalent to taking p and taking the value of p that's the closest to theta 1, which is completely intuitive.
The worst type I error is going to be attained for the p that's the closest to the alternative.
So even if the null is actually just an entire set, it's as if it was just the point that's the closest to the alternative.
So now we can compute this, because there's no unknown parameters that shows up.
We replace p by 0.5.
And so that was our test statistic.
So when you're building a test, you want to first build a test statistic, and then see what threshold you should be getting.
So now, let's go back to our example where we want to have-- we have x1 xn, their IID [INAUDIBLE] p. And I want to test if p is 1/2 versus p not equal to 1/2, which, as I said, is what you want to do if you want to test if a coin is fair.
And so here, I'm going to build a test statistic.
And we concluded last time that-- what do we want for this statistic?
We want it to have a distribution which, under the null, does not depend on the parameters, a distribution that I can actually compute quintiles of.
So what we did is, we said, well, if I look at-- the central limit theorem tells me that square root of n xn bar minus p divided by-- so if I do central limit theorem plus Slutsky, for example, I'm going to have square root.
And we've had this discussion whether we want to use Slutsky or not here.
But let's assume we're taking Slutsky wherever we can.
So this thing tells me that, by the central limit theorem, as n goes to infinity, this thing converges in distribution to some n01.
Now, as we said, this guy is not something we know.
But under the null, we actually know it.
And we can actually replace it by 1/2.
So this thing holds under h0.
When I write under h0, it means when this is the truth.
So now I have something that converges to something that has no dependence on anything I don't know.
And in particular, if you have any statistics textbook, which you don't because I didn't require one-- and you should be thankful, because these things cost $350.
Actually, if you look at the back, you actually have a table for a standard Gaussian.
I could have anything else here.
I could have an exponential distribution.
I could have a-- I don't know-- well, we'll see the chi squared distribution in a minute.
Any distribution from which you can actually see a table that somebody actually computed this thing for which you can actually draw the pdf and start computing whatever probability you want on them, then this is what you want to see at the right-hand side.
This is any distribution.
It's called pivotal.
I think we've mentioned that before.
Pivotal means it does not depend on anything that you don't know.
And maybe it's easy to compute those things.
Probably, typically, you need a computer to simulate them for you because computing probabilities for Gaussians is not an easy thing.
We don't know how to solve those integrals exactly, we have to do it numerically.
So now I want to do this test.
My test statistic will be declared to be what?
Well, I'm going to reject if what is larger than some number?
The absolute value of this guy.
So my test statistic is going to be square root of n minus 0.5 divided by square root of xn bar 1 minus xn bar.
That's my test statistic, absolute value of this guy, because I want to reject either when this guy is too large or when this guy is too small.
I don't know ahead whether I'm going to see p larger than 1/2 or less than 1/2.
So now I need to compute c such that the probability that tn is larger than c. So that's the probability under p, which is unknown.
I want this probability to be less than some level alpha, asymptotically.
So I want the limit of this guy to be less than alpha, and that's the level of my test.
So that's the given level.
So I want this thing to happen.
Now, what I know is that this limit-- actually, I should say given asymptotic level.
So what is this thing?
Well, OK, that's the probability that something that looks like under p. So under p, this guy-- so what I know is that tn is square root of n minus xn bar minus 0.5 divided by square root of xn bar 1 minus xn bar exceeds.
Is this true that as n to infinity, this probability is the same as the probability that the absolute value of a Gaussian exceeds c of a standard Gaussian?
Is this true?
The absolute value of the standard Gaussian
Yeah, the absolute.
So you're saying that this, as n becomes large enough, this should be the probability that some absolute value of n01 exceeds c, right?
Yes
So I claim that this is not correct.
Somebody tell me why
Even in the limit it's not correct?
Even in the limit, it's not correct

So what do you see?
It's because, at the beginning, we picked the worst possible true parameter, 0.5.
So we don't actually know that this 0.5 is the mean
Exactly.
So we pick this 0.5 here, but this is for any p. But what is the only p I can get?
So what I want is that this is true for all p in theta 0.
But the only p that's in theta 0 is actually p is equal to 0.5.
So yes, what you said was true, but it required to specify p to be equal to 0.5.
So this, in general, is not true.
But it happens to be true if p belongs to theta 0, which is strictly equivalent to p is equal to 0.5, because theta 0 is really just this one point, 0.5.
So now, this becomes true.
And so what I need to do is to find c such that this guy is equal to what?
I mean, let's just follow.
So I want this to be less than alpha.
But then we said that this was equal to this, which is equal to this.
So all I want is that this guy is less than alpha.
But we said we might as well just make it equal to alpha if you allow me to make it as big as I want, as long as it's less than alpha
So this is a true statement
So this is a true statement.
But it's under this condition
Exactly
So I'm going to set it equal to alpha, and then I'm going to try to solve for c. So what I'm looking for is a c such that if I draw a standard Gaussian-- so that's pdf of some n01-- I want the probability that the absolute value of my Gaussian exceeding this guy-- so that means being either here or here.
So that's minus c and c. I want the sum of those two things to be equal to alpha.
So I want the sum of these areas to equal alpha.
So by symmetry, each of them should be equal to alpha over 2.
And so what I'm looking for is c such that the probability that my n01 exceeds c, which is just this area to the right, now, equals alpha, which is equivalent to taking c, which is q equals alpha over 2, and that's q alpha over 2 by definition of q alpha over 2.
That's just what q alpha over 2 is.
And that's what the tables at the back of the book give you.
Who has already seen a table for Gaussian probabilities?
What it does, it's just a table.
I mean, it's pretty ancient.
I mean, of course, you can actually ask Google to do it for you now.
I mean, it's basically standard issue.
But back in the day, they actually had to look at tables.
And since the values alphas were pretty standard, the values alpha that people were requesting were typically 1%, 5%, 10%, all you could do is to compute these different values for different values of alpha.
That was it.
So there's really not much to give you.
So for the Gaussian, I can tell you that alpha is equal to-- if alpha is equal to 5%, then q alpha over 2, q 2.5% is equal to 1.96, for example.
So those are just fixed numbers that are functions of the Gaussian.
So everybody agrees?
We've done that before for our confidence intervals.
And so now we know that if I actually plug in this guy to be q alpha over 2, then this limit is actually equal to alpha.
And so now I've actually constrained this.
So q alpha over 2 here for alpha equals 5%, as I said, is 1.96.
So in the example 1, the number that we found was 3.54, I think, or something like that, 3.55 for t. So if we scroll back very quickly, 3.45-- that was example 1.
Example two-- negative 0.77.
So if I look at tn in example 1, tn was just the absolute value of 3.45, which-- don't pull out your calculators-- is equal to 3.45.
Example 2, absolute value of negative 0.77 was equal to 0.77.
And so all I need to check is, is this number larger or smaller than 1.96?
That's what my test ends up being.
So in example 1, 3.45 being larger than 1.96, that means that I reject.
Fairness of my coins, in example 2, 0.77 being smaller than 1.96-- what do I do?
I fail to reject.
So here is a question.
In example 1, for what level alpha would psi alpha-- OK, so here, what's going to happen if I start decreasing my level?
When I decrease my level, I'm actually making this area smaller and smaller, which means that I push this c to the right.
So now I'm asking, what is the smallest c I should pick so that now, I actually do not reject h0?
What is the smallest c I should be taking here?
What is the smallest c?
So c here, in the example I gave you for 5%, was 1.96.
What is the smallest c I should be taking so that now, this inequality is reversed?
3.45.
I ask only trivial questions, don't be worried.
So 3.45 is the smallest c that I'm actually willing to tolerate.
So let's say this was my 5%.
If this was 2.5-- if here, let's say, in this picture, alpha is 5%, that means maybe I need to push here.
And this number should be what?
So this is going to be 1.96.
And this number here is going to be 3.45, clearly to scale.
And so now, what I want to ask you is, well, there's two ways I can understand this number 3.45.
It is the number 3.45, but I can also try to understand what is the area to the right of this guy.
And if I understand what the area to the right of this guy is, this is actually some alpha prime over 2.
And that means that if I actually fix this level alpha prime, that would be exactly the tipping point at which I would go from accepting to rejecting.
So I knew, in terms of absolute thresholds, 3.45 is the trivial answer to the question.
That's the tipping point, because I'm comparing a number to 3.45.
But now, if I try to map this back and understand what level would have been giving me this particular tipping point, that's a number between 0 and 1.
The smaller the number, the larger this number here, which means that the more evidence I have in my data against h0.
And so this number is actually something called the p-value.
And so saying, for example 2, there's the tipping point alpha at which I go from failing to reject to rejecting.
And that's exactly the number, the area under the curve, such that here, I see 0.77.
And this is this alpha prime prime over 2.
Alpha prime prime is clearly larger than 5%.
So what's the advantage of thinking and mapping back these numbers?
Well, now, I'm actually going to spit out some number which is between 0 and 1.
And that should be the only scale you should have in mind.
Remember, we discussed that last time.
I was like, well, if I actually spit out a number which is 3.45, maybe you can try to think, is 3.45 a large number for a Gaussian?
That's a number.
But if I had another random variable that was not Gaussian, maybe it was a double exponential, you would have to have another scale in your mind.
Is 3.45 so large that it's unlikely for it to come from a double exponential.
If I had a gamma distribution-- I can think of any distribution, and then that means, for each distribution, you would have to have scale in mind.
So of course, you can have the Gaussian scale in mind.
I mean, I have the Gaussian scale in mind.
But then, if I map it back into this number between 0 and 1, all the distributions play the same role.
So whether I'm talking about if my limiting distribution is normal or exponential or gamma, or whatever you want, for all these guys, I'm just going to map it into one number between 0 and 1.
Small number means lots of evidence against h1.
Large number means lots of evidence against h0.
Small number means very few evidence against h9.
And this is the only number you need to keep in mind.
And the question is, am I willing to tolerate this number between 5%, 6%, or maybe 10%, 12%?
And this is the only scale you have to have in mind.
And this scale is the scale of p-values.
So the p-value is the tipping point in terms of alpha.
In words, I can make it formal, because tipping point, as far as I know, is not a mathematical term.
So a p-value of a test is the smallest, potentially asymptotic level if I talk about an asymptotic p-value-- and that's what we do when we talk about central theorem-- at which the test rejects h0.
If I were to go any smaller-- sorry, it's the smallest level-- yeah, if I were to go any smaller, I would fail to reject.
The smaller the level, the less likely it is for me to reject.
And if I were to go any smaller, I would start failing to reject.
And so it is a random number.
It depends on what I actually observe.
So here, of course, I instantiated those two numbers, 3.45 and 0.77, as realizations of random variables.
But if you think of those as being the random numbers before I see my data, this was a random number, and therefore, the area under the curve to the right of it is also a random area.
If this thing fluctuates, then the area under the curve fluctuates.
And that's what the p-value is.
That's what-- what is his name?
I forget.
John Oliver talks about when he talks about p-hacking.
And so we talked about this in the first lecture.
So p-hacking is, how do I do-- oh, if I'm a scientist, do I want to see a small p-value or a large p-value?
Small
Small, right?
Scientists want to see small p-values because small p-values equals rejecting, which equals discovery, which equals publications, which equals promotion.
So that's what people want to see.
So people are tempted to see small p-values.
And what's called p-hacking is, well, find a way to cheat.
Maybe look at your data, formulate your hypothesis in such a way that you will actually have a smaller p-value than you should have.
So here, for example, there's one thing I did not insist on because, again, this is not a particular course on statistical thinking, but one thing that we implicitly did was set those theta 0 and theta 1 ahead of time.
I fixed them, and I'm trying to test this.
This is to be contrasted with the following approach.
I draw my data.
So I draw-- I run this experiment, which is probably going to get me a publication in nature.
I'm trying to test if a coin is fair.
And I draw my data, and I see that there's 13 out of 30 of my observations that are heads.
That means that, from this data, it looks like p is less than 1/2.
So if I look at this data and then decide that my alternative is not p not equal to 1/2, but rather p less than 1/2, that's p-hacking.
I'm actually making my p-value strictly smaller by first looking at the data, and then deciding what my alternative is going to be.
And that's cheating, because all the things we did, we're assuming that this 0.5, or the alternative, was actually a fixed-- everything was deterministic.
The only randomness came from the data.
But if I start looking at the data and designing my experiment or my alternatives and null hypothesis based on the data, it's as if I started putting randomness all over the place.
And then I cannot control it because I don't know how it just intermingles with each other.
So that was for the John Oliver moment.
So the p-value is nice.
So maybe I mentioned that, before, my wife works in market research.
And maybe every two years, she seems to run into a statistician in the hallway, and she comes home and says, what is a p-value again?
And for her, a p-value is just the number in an Excel spreadsheet.
And actually, small equals good and large equals bad.
And that's all she needs to know at this point.
Actually, they do the job for her-- small is green, large is red.
And so for her, a p-value is just green or red.
But so what she's really implicitly doing with this color code is just applying the golden rule.
What the statisticians do for her in the Excel spreadsheet is that they take the numbers for the p-values that are less than some fixed level.
So depending on the field in which she works-- so she works for pharmaceutical companies-- so the p-values are typically compared-- the tests are usually performed at level 1%, rather than 5%.
So 5% is maybe your gold standard if you're doing sociology or trying to-- I don't know-- release a new blueberry flavor for your toothpaste.
Something that's not going to change the life of people, maybe you're going to run at 5%.
It's OK to make a mistake.
See, people are just going to feel gross, but that's about it, whereas here, if you have this p-value which is less than 1%, it might be more important for some drug discovery, for example.
And so let's say you run at 1%.
And so what they do in this Excel spreadsheet is that all the numbers that are below 1% show up in green and all the numbers that are above 1% show up in red.
And that's it.
That's just applying the golden rule.
If the number is green, reject.
If the number is red, fail to reject.
Yeah?
So going back to example 2 where the prior example where you want to cheat by looking after beta and then formulating, say, theta 1 to be p less than 1/2
Yeah
So how would you achieve your goal by changing the theta--
By achieving my goal, you mean letting ethics aside, right?
Yeah, yeah
Ah, you want to be published
Yeah
[LAUGHS] So let me teach you how, then.
So well, here, what do you do?
You want to-- at the end of the day, a test is only telling you whether you found evidence in your data that h1 was more likely than h0, basically.
How do you make h1 more likely?
Well, you just basically target h1 to be what it is-- what the data is going to make it more likely to be.
So if, for example, I say h1 can be on both sides, then my data is going to have to take into account fluctuations on both sides, and I'm going to lose a factor or two somewhere because things are not symmetric.
Here is the ultimate way of making this work.
I'm going back to my example of flipping coins.
And now, so here, what I did is, I said, oh, this number 0.43 is actually smaller than 0.5, so I'm just going to test whether I'm 0.5 or I'm less than 0.5.
But here is something that I can promise you I did not make the computation will reject.
So here, this one actually-- yeah, this one fails to reject.
So here is one that will certainly reject.
h0 is 0.5, p is 0.5, h1p is 0.43.
Now, you can try, but I can promise you that your data will tell you that h1 is the right one.
I mean, you can check very quickly that this is really extremely likely to happen.
Actually, what am I-- no, actually, that's not true, because here, the test that I derive that's based on this kind of stuff, here at some point, somewhere under some layers, I assume that all our tests are going to have this form.
But here, this is only when you're trying to test one region versus another region next to it, or one point versus a region around it, or something like this, whereas for this guy, there's another test that could come up with, which is, what is the probability that I get 0.43, and what is the probability that I get 0.5?
Now, what I'm going to do is, I'm going to just conclude it's whichever has the largest probability.
Then maybe I'm going to have to make some adjustments so that the level is actually 5%.
But I can make this happen.
I can make the level be 5% and always conclude this guy, but I would have to use a different test.
Now, the test that I described, again, those tn larger than c are built in to be tests that are resilient to these kind of manipulations because they're oblivious towards what the alternative looks like.
I mean, they're just saying it's either to the left or to the right, but whether it's a point or an entire half-line doesn't matter.
So if you try to look at your data and just put the data itself into your hypothesis testing problem, then you're failing the statistical principle.
And that's what people are doing.
I mean, how can I check?
I mean, of course, here, it's going to be pretty blatant if you publish a paper that looks like this.
But there's ways to do it differently.
For example, one way to do it is to just do mult-- so typically, what people do is they do multiple hypothesis testing.
They're doing 100 tests at a time.
Then you have random fluctuations every time.
And so they just pick the one that has the random fluctuations that go their way.
I mean, sometimes it's going in your way, and sometimes it's going the opposite way, so you just pick the one that works for you.
We'll talk about multiple hypothesis testing soon if you want to increase your publication count.
There's actually papers-- I think it was a big news that some papers, I think, in psychology or psychometrics papers that actually refused to publish p-values now.
Where were we?
Here's the golden rule.
So one thing that I like to show is this thing, just so you know how you apply the golden rule and how you apply the standard tests.
So the standard paradigm is the following.
You have a black box, which is your test.
For my wife, this is the 4th floor of the building.
That's where the statisticians sit.
What she sends there is data-- let's say x1 xn.
And she says, well, this one is about toothpaste, so here's a level-- let's say 5%.
What the 4th floor brings back is that answer-- yes, no, green, red, just an answer.
So that's the standard testing.
You just feed it the data and the level at which you want to perform the test, maybe asymptotic, and it spits out a yes, no answer.
What p-value does, you just feed it the data itself.
And what it spits out is the p-value.
And now it's just up to you.
I mean, hopefully your brain has the computational power of deciding whether a number is larger or smaller than 5% without having to call a statistician for this.
And that's what it does.
So now we're on 1 scale.
Now, I see some of you nodding when I talk about p-hacking, so that means you've seen p-values.
If you've seen more than 100 p-values in your life, you have an entire scale.
A good p-value is less than 10 to the minus 4.
That's the ultimate sweet spot.
Actually, statistical software spits out an output which says less than 10 to the minus 4.
But then maybe you want a p-val-- if you tell me my p-value was 4.65, then I will say, you've been doing some p-hacking until you found a number that was below 5%.
That's typically what people will do.
But if you tell me-- if you're doing the test, if you're saying, I published my result, my test at 5% said yes, that means that maybe you're p-value was 4.99, or you're p-value was 10 to the minus 4, I will never know.
I will never know how much evidence you had against the null.
But if you tell me what the p-value is, I can make my own decision.
I don't have to tell me whether it's a yes, no.
You tell me it's 4.99, I'm going to say, well, maybe yes, but I'm going to take it with a grain of salt.
And so that's why p-values are good numbers to have in mind.
Now, I should, as if it was like an old trick that you start mastering when you're 45 years old.
No, it's just, how small is the number between 0 and 1?
That's really what you need to know.
Maybe on the log scale-- if it's 10 to the minus 1, 10 to the minus 2, 10 to the minus 3, et cetera-- that's probably the extent of the mastery here.
So this traditional standard paradigm that I showed is actually commonly referred to as the Neyman-Pearson paradigm.
So here, it says name Neyman-Pearson's theory, so there's an entire theory that comes with it.
But it's really a paradigm.
It's a way of thinking about hypothesis testing that says, well, if I'm not going to be able to optimize both my type I and type II error, I'm actually going to lock in my type I error below some level and just minimize the type II error under this constraint.
That's what the Neyman-Pearson paradigm is.
And it sort of makes sense for hypothesis testing problems.
Now, if you were doing some other applications with multi-objective optimization, you would maybe come up with something different.
For example, machine learning is not performing typically under Neyman-Pearson paradigm.
So if you do spam filtering, you could say, well, I want to constrain the probability as much as I can of taking somebody's important emails and throwing them out as spam, and under this constraint, not send too much spam to that person.
That sort of makes sense for spams.
Now, if you're labeling cats versus dogs, that's probably not like you want to make sure that no more than 5% of the dogs are labeled cat because, I mean, it doesn't matter.
So what you typically do is, you just sum up the two types of errors you can make, and you minimize the sum without putting any more weight on one or the other.
So here's an example where doing a binary decision, one or two of the errors you can make, you don't have to actually be like that.
So this example here, I did not.
The trivial test psi is equal to 0, what was it in the US trial court example?
What is psi equals 0?
That was concluding always to the null.
What was the null?
Innocent
Innocent, right?
That's the status quo.
So that means that this guy never rejects h0.
Everybody's going away free.
So you're sure you're not actually going against the constitution because alpha is 0%, which is certainly less than 5%.
But the power, the fact that a lot of criminals go back outside in the free world is actually formulated in terms of low power, which, in this case, is actually 0.
Again, the power is the number between 0 and 1.
Close to 0, good.
Close to 1, bad.
Now, what is the definition of the p-value?
That's going to be something-- it's a mouthful.
The definition of the p-value is a mouthful.
It's the tipping point.
It is the smallest level at which blah, blah, blah, blah, blah.
It's complicated to remember it.
Now, I think that my 6th explanation, my wife, after saying, oh, so it's the probability of making an error, I said, yeah, that's the probability of making an error because, of course, she can think probability of making an error small, good, large, bad.
So that's actually a good way to remember.
I'm pretty sure that at least 50% of people who are using p-values out there think that the p-value is the probability of making an error.
Now, for all matters of purposes, if your goal is to just threshold the p-value, this is OK to have this in y.
But when comes, at least until December 22, I would recommend trying to actually memorize the right definition for the p-value.
So the idea, again, is fix the level and try to optimize the power.
So we're going to try to compute some p-values from now on.
How do you compute the p-value?
Well, you can actually see it from this picture over there.
One thing I didn't show on this picture-- so here, it was my q alpha over 2 that had alpha here, alpha over 2 here.
That was my q alpha over 2.
And I said, if tn is to the right of this guy, I'm going to reject.
If tn is to the left of this guy, I'm going to fail to reject.
Pictorially, you can actually represent the p-value.
It's when I replace this guy by tn itself.
Sorry, that's p-value over 2.
No, actually, that's p-value.
So let me just keep it like that and put the absolute value here.
So if you replace the role of q alpha over 2, by your test statistic, the area under the curve is actually the p-value itself up to a scale because of the symmetric thing.
So there's a good way to see, pictorially, what the p-value is.
It's just the probability that some Gaussians-- it's just the probability that some absolute value of n01 exceeds tn.
That's what the p-value is.
Now, this guy has nothing to do with this guy, so this is really just 1 minus the Gaussian cdf of tn, and that's it.
So that's how I would compute p-values.
Now, as I said, the p-value is a beauty because you don't have to understand the fact that your limiting distribution is a Gaussian.
It's already factored in this construction.
The fact that I'm actually looking at this cumulative distribution function of a standard Gaussian makes my p-value automatically adjust to what the limiting distribution is.
And if this was the cumulative distribution function of a exponential, I would just have a different function here denoted by f, for example, and I would just compute a different value.
But in the end, regardless of what the limiting value is, my p-value would still be a number between 0 and 1.
And so to illustrate that, let's look at other weird distributions that we could get in place of the standard Gaussian.
And we're not going to see many, but we'll see one.
And it's not called the chi squared distribution.
It's actually called the Student's distribution, but it involves the chi squared distribution as a building block.
So I don't know if my phonetics are not really right there, so I try to say, well, it's chi squared.
Maybe it's "kee" squared above, in Canada, who knows.
So for a positive integer, so there's only 1 parameter.
So for the Gaussian, you have 2 parameters, which are mu and sigma squared.
Those are real numbers.
Sigma squared's positive.
Here, I have 1 integer parameter.
Then the chi squared distribution with d degrees of freedom-- so the parameter is called a degree of freedom, just like mu is called the expected value and sigma squared is called the variance.
Here, we call it degrees of freedom.
You don't have to really understand why.
So that's the law that you would get-- that's the random variable you would get if you were to sum d squares of independent standard Gaussians.
So I take the square of an independent random Gaussian.
I take another one.
I sum them, and that's a chi squared with 2 degrees of freedom.
That's how you get it.
Now, I could define it using its probability density function.
I mean, after all, this is the sum of positive random variables, so it is a positive random variable.
It has a density on the positive real line.
And the pdf of chi squared with d degrees of freedom is what?
Well, it's fd of x is-- what is it?-- x to the d/2 minus 1 e to the minus x/2.
And then here, I have a gamma of d/2.
And the other one is, I think, 2 to the d/2 minus 1.
No, 2 to the d/2.
That's what it is.
That's the density.
If you are very good at probability, you can make the change of variable and write your Jacobian and do all this stuff and actually check that this is true.
I do not recommend doing that.
So this is the density, but it's better understood like that.
I think it was just something that you built from standard Gaussian.
So for example, an example of a chi squared with 2 degrees of freedom is actually the following thing.
Let's assume I have a target like this.
And I don't aim very well.
And I'm trying to hit the center.
And I'm not going to have, maybe, a deviation, which is standard Gaussian left, right and standard Gaussian north, south.
So I'm throwing, and then I'm here, and I'm claiming that this number here, by Pythagoras theorem, the square distance here is the sum of this square distance here, which is the square of a Gaussian by assumption.
This is plus the square of this distance, which is the square of another independent Gaussian.
I assume those are independent.
And so the square distance from this point to this point is the chi squared with 2 degrees of freedom.
So this guy here is n01 squared.
This is n01 squared.
And so this guy here, this distance here, is chi squared with 2 degrees of freedom.
I mean the square distance.
I'm talking about square distances here.
So now you can see that, actually, Pythagoras is basically why chi squared [?
arrives.
?]
That's why it has its own name.
I mean, I could define this random variable.
I mean, it's actually a gamma distribution.
It's a special case of something called the gamma distribution.
The fact that the special case has its own name is because there's many times what we're going to take sum of squares of independent Gaussians because Gaussians, the sum of squares is really the norm, the Euclidean norm squared, just by Pythagoras theorem.
If I'm in higher dimension, I can start to sum more squared coordinates, and I'm going to measure the norm squared.
So if you want to draw this picture, it looks like this.
Again, it's the sum of positive numbers, so it's going to be on 0 plus infinity.
That's fd.
And so f1 looks like this, f2 looks like this.
So the tails become heavier and heavier as d increases.
And then at [INAUDIBLE] to 3, it starts to have a different shape.
It starts from 0 and it looks like this.
And then, as d increases, it's basically as if you were to push this thing to the right.
It's just like, psh, so it's just falling like a big blob.
Everybody sees what's going on?
So there's just this fat thing that's just going there.
What is the expected value of a chi squared?
So it's the expected value of the sum of Gaussian random variables, squared.
I know I said that
So it's the sum of their second moments, right?
Which is?
Those are n01
It's like-- oh, I see, 1
Yeah
So n times 1 or d times 1
Yeah, which is d. So one thing you can check quickly is that the expected value of a chi squared is d. And so you see, that's why the mass is shifting to the right as d increases.
It's just going there.
Actually, the variance is also increasing.
The variance is 2d.
So this is one thing.
And so why do we care about this?
In basic statistics, it's not like we actually have statistics much about throwing darts at high-dimensional boards.
So what's happening is that if I look at the sample variance, the average of the sum of squared centered by their mean, then I can actually expend this as the sum of the squares minus the average squared It's just the same trick that we have for the variance-- second moment minus first moment square.
And then I claim that Cochran's theorem-- and I will tell you in a second what Cochran's theorem tells me is that this sample variance is actually-- so if I had only this-- look at those guys.
Those guys are Gaussian with mean mu and variance sigma squared.
Think for 1 second mu being 0 and sigma squared being 1.
Now, this part would be a chi squared with n degrees of freedom divided by n. Now I get another thing here, which is the square of something that looks like a Gaussian as well.
So it looks like I have something else here, which looks also like a chi squared.
Now, Cochran's theorem is essentially telling you that those things are independent, and so that in a way, you can think of those guys as being, here, n degrees of freedom minus 1 degree of freedom.
Now, here, as I said, this does not mean 0 and variance 1.
The fact that it's not mean 0 is not a problem because I can remove the mean here and remove the mean here.
And so this thing has the same distribution, regardless of what the actual mean is.
So without loss of generality, I can assume that mu is equal to 0.
Now, the variance, I'm going to have to pay, because if I multiply all these numbers by 10, then this sn is going to multiplied by 100.
So this thing is going to scale with the variance.
And not surprisingly, it's scaling like the square of the variance.
So if I look at sn, it's distributed as sigma squared times the chi squared with n minus 1 degrees of freedom divided by n. And we don't really write that, because a chi squared times sigma squared divided by n is not a distribution, so we put everything to the left, and we say that this is actually a chi squared with n minus 1 degrees of freedom.
So here, I'm actually dropping a fact on you, but you can see the building block.
What is the thing that's fuzzy at this point, but the rest should be crystal clear to you?
The thing that's fuzzy is that removing this squared guy here is actually removing 1 degree of freedom.
That should be weird, but that's what Cochran's theorem tells.
It's essentially stating something about orthogonality of subspaces with the span of the constant vector, something like that.
So you don't have to think about it too much, but that's what it's telling me.
But the rest, if you plug in-- so the scaling in sigma squared and in n, so that should be completely clear to you.
So in particular, if I remove that part, it should be clear to you that this thing, if mean is 0, this thing is actually distributed.
Well, if mu is 0, what is the distribution of this guy?
So I remove that part, just this part.
So I have xi, which are n0 sigma squared.
And I'm asking, what is the distribution of 1/n sum from i equal 1 to n of xi squared?
So it is the sum of their IID.
So it's the sum of independent Gaussians, but not standard.
So the first thing to make them standard is that I divide all of them by sigma squared.
Now, this guy is of the form zi squared where zi is n01.
So now, this thing here has what distribution?
Chi squared n
Chi squared n. And now, sigma squared over n times chi squared n-- so if I have sigma squared divided by n times chi squared-- sorry, so n times n divided by sigma squared.
So if I take this thing and I multiply it by n divided by sigma squared, it means I remove this term, and now I am left with a chi squared with n degrees of freedom.
Now, the effect of centering with the sample mean here is only to lose 1 degree of freedom.
That's it.
So if I want to do a test about variance, since this is supposedly a good estimator of variance, this could be my pivotal distribution.
This could play the role of a Gaussian.
If I want to know if my variance is equal to 1 or larger than 1, I could actually build a test based on this only statement and test if the variance is larger than 1 or not.
Now, this is not asymptotic because I started with the very assumption that my data was Gaussian itself.
Now, just a side remark-- you can check that this chi squared 2, 2 is an exponential with 1/2 degrees of freedom, which is certainly not clear from the fact that z1 squared plus z2 squared is a chi squared with 2 degrees of freedom.
if I give you the sum of the square of 2 independent Gaussian, this is actually an exponential.
That's not super clear, right?
But if you look at what was here-- I don't know if you took notes, but let me rewrite it for you.
So it was x to the d/2 minus 1 e to the minus x/2 divided by 2 to the d/2 gamma of d/2.
So if I plug in d is equal to 2, gamma of 2/2 is gamma of 1, which is 1.
It's factorial of 0.
So it's 1, so this guy goes away.
2 to the d/2 is 2 to 1, so that's just 1.
No, that's just 2.
Then x the d/2 minus 1 is x the 0, goes away.
And so I have x minus x/2 1/2, which is really, indeed, of the form lambda e to the minus lambda x for lambda is equal to 1/2, which was our exponential distribution.
Well, next week is, well, Columbus Day?
So not next Monday-- so next week, we'll talk about Student's distribution.
And so that was discovered by a guy who pretended his name was Student, but was not Student.
And I challenge you to find why in the meantime.
So I'll see you next week.
Your homework is going to be outside so we can release the room.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or give you additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So again, before we start, there is a survey online if you haven't done so.
I would guess at least one of you has not.
Some of you have entered their answers and their thoughts, and I really appreciate this.
It's actually very helpful.
So it seems that the course is going fairly well from what I've read so far.
So if you don't think this is the case, please enter your opinion and tell us how we can make it better.
One of the things that was said is that I speak too fast, which is absolutely true.
I just can't help it.
I get so excited, but I will really do my best.
I will try to.
I think I always start OK.
I just end not so well.
So last time we talked about this chi squared distribution, which is just another distribution that's so common that it deserves its own name.
And this is something that arises when we sum the squares of independent standard Gaussian random variables.
And in particular, why is that relevant?
It's because if I look at the sample variance, then it is a chi square distribution, and the parameter that shows up is also known as the degrees of freedom, is the number of observations of minus one.
And so as I said, this chi squared distribution has an explicit probability density function, and I tried to draw it.
And one of the comments was also about my handwriting, so I will actually not rely on it for detailed things.
So this is what the chi squared with one degree of freedom would look like.
And really, what this is is just the distribution of the square of a standard Gaussian.
I'm summing only one, so that's what it is.
Then when I go to 2, this is what it is-- 3, 4, 5, 6, and 10.
And as I move, you can see this thing is becoming flatter and flatter, and it's pushing to the right.
And that's because I'm summing more and more squares, and in expectation we just get one every time.
So it really means that the mass is moving to infinity.
In particular, a chi squared distribution with n degrees of freedom is going to infinity as n goes to infinity.
Another distribution that I asked you to think about-- anybody looked around about the student t-distribution, what the history of this thing was?
So I'll tell you a little bit.
I understand if you didn't have time.
So the t-distribution is another common distribution that is so common that it will be used and will have its table of quintiles that are drawn at the back of the book.
Now, remember, when I mentioned the Gaussian, I said, well, there are several values for alpha that we're interested in.
And so I wanted to draw a table for the Gaussian.
We had something that looked like this, and I said, well, q alpha over 2 to get alpha over 2 to the right of this number.
And we said that there is a table for this things, for common values of theta.
Well, if you try to envision what this table will look like, it's actually a pretty sad table, because it's basically one list of numbers.
Why would I call it a table?
Because all I need to tell you is something that looks like this.
If I tell you this is alpha and this is q alpha over 2 and then I say, OK, basically the three alphas that I told you I care about are something like 1%, 5%, and 10%, then my table will just give me q alpha over 2.
So that's alpha, and that's q alpha over 2.
And that's going to tell me that-- I don't remember this one, but this guy is 1.96.
This guy is something like 2.45.
I think this one is like 1.65 maybe.
And maybe you can be a little finer, but it's not going to be an entire page at the back of the book.
And the reason is because I only need to draw these things for d1 standard Gaussian when the parameters are 0 for the mean and 1 for the variance.
Now, if I'm actually doing this for the the chi squared, I basically have to give you one table per values of the degrees of freedom, because those things are different.
There is no way I can take-- for Gaussian's, if you give me a different mean, I can substract it and make it back to be a standard Gaussian.
For the chi squared, there is no such thing.
There is no thing that just takes the chi squared with d degrees of freedom, nd, and turns it into, say, a chi square with one degree of freedom.
This just does not happen.
So the word is standardized.
Make it a standard chi squared.
There is no such thing as standard chi squared.
So what it means is that I'm going to need one row like that for each value of the number of degrees of freedom.
So that will certainly fill a page at the back of a book-- maybe even more.
I need one per sample size.
So if I want to go from simple size 1 to 1,000, I need 1,000 rows.
So now the student distribution is one that arises where it looks very much like the Gaussian distribution, and there's a very simple reason for that, is that I take a standard Gaussian and I divide it by something.
That's how I get the student.
What do I divide it with?
Well, I take an independent chi square-- I'm going to call it v-- and I want it to be independent from z.
And I'm going to divide z by root v over d. So I start with a chi squared v. So this guy is chi squared d. I start with z, which is n 0, 1.
I'm going to assume that those guys are independent.
In my t-distribution, I'm going to write a T. Capital T is z divided by the square root of v over d. Why would I want to do this?
Well, because this is exactly what happens when a divide not by the true variance, a Gaussian, but by its empirical variance.
So let's see why in a second.
So I know that if you give me some random variable-- let's call it x, which is N mu sigma squared-- then I can do this.
x minus mu divided by sigma.
I'm going to call this thing z, because this thing actually has some standard Gaussian distribution.
I have standardized x into something that I can read the quintiles at the back of the book.
So that's this process that I want to do.
Now, to be able to do this, I need to know what mu is, and I need to know what sigma is.
Otherwise I'm not going to be able to make this operation.
mu I can sort of get away with, because remember, when we're doing confidence intervals we're actually solving for mu.
So it was good that mu was there.
When we're doing hypothesis testing, we're actually plugging in here the mu that shows up in h0.
So that was good.
We had this thing.
Think of mu as being p, for example.
But this guy here, we don't necessarily know what it is.
I just had to tell you for the entire first chapter, assume you have Gaussian random variables and that you know what the variance is.
And the reason why I said assume you know it-- and I said sometimes you can read it on the side of the box of measuring equipment in the lab.
That was just the way I justified it, but the real reason why I did this is because I would not be able to perform this operation if I actually did not know what sigma was.
But from data, we know that we can form this estimator Sn, which is 1 over n, sum from i equals 1 to n of Xi, minus X bar squared.
And this thing is approximately equal to sigma squared.
That's the sample variance, and it's actually a good estimator just by the law of large number, actually.
This thing, by the law of large number, as n goes to infinity-- well, let's say it in probability goes to sigma squared by the law of large number.
So it's a consistent estimator of sigma squared.
So now, what I want to do is to be able to use this estimator rather than using sigma.
And the way I'm going to do it is I'm going to say, OK, what I want to form is x minus mu divided by Sn this time.
I don't know what the distribution of this guy is.
Sorry, it's square root of Sn.
This is sigma squared.
So this is what I would take.
And I could think of Slutsky, maybe, something like this that would tell me, well, just use that and pretend it's a Gaussian.
And we'll see how actually it's sort of valid to do that, because Slutsky tells us it is valid to do that.
But what we can also do is to say, well, this is actually equal to x minus mu, divided by sigma, which I knew what the distribution of this guy is.
And then what I'm going to do is I'm going to just-- well, I'm going to cancel this effect, sigma over square root Sn.
So I didn't change anything.
I just put the sigma here.
So now what I know what I know is that this is some z, and it has some standard Gaussian distribution.
What is this guy?
Well, I know that Sn-- we wrote this here.
Maybe I shouldn't have put those pictures, because now I keep on skipping before and after.
We know that Sn times n divided by sigma squared is actually chi squared n minus 1.
So what do I have here?
I have that chi squared-- so here I have something that looks like 1 over square root of Sn divided by sigma squared.
This is what this guy is if I just do some more writing.
And maybe I actually want to make my life a little easier.
I'm actually going to plug in my n here, and so I'm going to have to multiply by square root of n here.
Everybody's with me?
So now what I end up with is something that looks like this, where I have-- here I started with x. I should really start with Xn bar minus mu times square root of n. That's what the central limit theorem would tell me.
I need to work with the average rather than just one observation.
So if I start with this, then I pick up a square root of n here.
So if I had the sigma here, I would know that this thing is actually-- Xn bar minus mu divided by sigma times the square root of n would be a standard Gaussian.
So if I put Xn bar here, I really need to put this thing that goes around the Xn bar.
That's just my central limit theorem that says if I average, then my variance has shrunk by a factor 1 over n. Now, I can still do this.
That was still fine.
And now I said that this thing is basically this guy.
So what I know is that this thing is a chi squared with n minus 1 degrees of freedom, so this guy here is chi squared with n minus 1 degrees of freedom.
Let me call this thing v in the spirit of what was used there and in the spirit of what is written here.
So this guy was called v, so I'm going to call this v. So what I can write is that square root of n Xn bar minus mu divided by square root of Sn is equal to z times square root of n divided by square root of v. Everybody's with me here?
Which I can rewrite as z times square root of v divided by n And if you look at what the definition of this thing is, I'm almost there.
What is the only thing that's wrong here?
This is a student distribution, right?
So there's two things.
The first one was that they should be independent, and they actually are independent.
That's what Cochran's theorem tells me, and you just have to count on me for this.
I told you already that Sn was independent of Xn bar.
So those two guys are independent, which implies that the numerator and denominator here are independent.
That's what Cochran's theorem tells us.
But is this exactly what I should be seeing if I wanted to have my sample variance, if I want to have to write this?
Is this actually the definition of a student distribution?
Yes?
No.
So we see z divided by square root of v over d. That looks pretty much like it, except there's a small discrepancy.
What is the discrepancy?
There's just the square root of n minus 1 thing.
So here, v has n minus 1 degrees of freedom.
And in the definition, if the v has d degrees of freedom, I divide it by d, not by d minus 1 or not by d plus 1, actually, in this case.
So I have this extra thing.
Well, there's two ways I can address this.
The first one is by saying, well, this is actually equal to z over square root of v divided by n minus 1 times square root of n over n minus 1.
I can always do that and say for n large enough this thing is actually going to be pretty small, or I can take account for it.
Or for any n you give me, I can compute this number.
And so rather than having a t-distribution, I'm going to have a t-distribution time this deterministic number, which is just a function of my number of observations.
But what I actually want to do instead is probably use a slightly different normalization, which is just to say, well, why do I have to define Sn-- where was my Sn?
Yeah, why do I have to define Sn tend to be divided by n?
Actually, this is a biased estimator, and if I wanted to be unbiased, I can actually just put an n minus 1 here.
You can check that.
You can expend this thing and compute the expectation.
You will see that it's actually not sigma squared, but n over n minus 1 sigma squared.
So you can actually just make it unbiased.
Let's call this guy tilde, and then when I put this tilde here what I actually get is s tilde here and s tilde here.
I need actually to have n minus 1 here to have this s tilde be a chi squared distribution.
Yes?
[INAUDIBLE] defined this way so that you--
So basically, this is what the story did.
So the story was, well, rather than using always the central limit theorem and just pretending that my Sn is actually the true sigma squared, since this is something I'm going to do a lot, I might as well just compute the distribution, like the quintiles for this particular distribution, which clearly does not depend on any unknown parameter.
d is the only parameter that shows up here, and it's completely characterized by the number of observations that you have, which you definitely know.
And so people said, let's just be slightly more accurate.
And in a second, I'll show you how the distribution of the T-- so we know that if the sample size is large enough, this should not have any difference with the Gaussian distribution.
I mean, those two things should be the same because we've actually not paid attention to this discrepancy by using empirical variance rather than true so far.
And so we'll see what the difference is, and this difference actually manifests itself only in small sample sizes.
So those are things that matter mostly if you have less than, say, 50 observations.
Then you might want to be slightly more precise and use t-distribution rather than Gaussian.
So this is just a matter of being slightly more precise.
If you have more than 50 observations, just drop everything and just pretend that this is the true one.
Any other questions?
So now I have this thing, and so I'm on my way to changing this guy.
So here now, I have not root n but root n minus 1.
So I have a z.
So this guy here is S. Yet Where did I get my root n from in the first place?
Yeah, because I wanted this guy.
And so now what I am left with is Xn minus mu divided by Sn tilde, which is the new one, which is now indeed of the form z v root n minus 1, which now I can write it as z v minus 1.
And so now I have exactly what I want, and so this guy is n 0, 1.
And this guy is chi squared with n minus 1 degrees of freedom.
And so now I'm back to what I want.
So rather than using Sn to be the empirical variance where I just divide my normatizations by n, if I use n minus 1, I'm perfect.
Of course, I can still use n and do this multiplying by root n minus 1 over n at the end.
But that just doesn't make as much sense.
Everybody's fine with what this T n distribution is doing and why this last line is correct?
So that's just basically because it's been defined so that this is actually happening.
That was your question, and that's really what happened.
So what is this student t-distribution?
Where does the name come from?
Well, it does not come from Mr. T. And if you know who Mr. T was-- you're probably too young for that-- he was our hero in the 80s.
And it comes from this guy.
His name is Sean William Gosset-- 1908.
So that was back in the day.
And this guy actually worked at the Guinness Brewery in Dublin, Ireland.
And Mr. Guinness back then was a bit of a fascist, and he didn't want him to actually publish papers.
And so what he had to do is to use a fake name to do that.
And he was not very creative, and he used a name "student."
Because I guess he was a student of life.
And so here's the guy, actually.
So back in 1908, it was actually not difficult to put your name or your pen name on a distribution.
So what does this thing look like?
How does it compare to the standard normal distribution?
You think it's going to have heavier or lighter tails compared to the standard distribution, the Gaussian distribution?
Yeah, because they have extra uncertainty in the denominator, so it's actually going to make things wiggle a little wider.
So let's start with a reference, which is the standard normal distribution.
So that's my usual bell-shaped curve.
And this is actually the t-distribution with 50 degrees of freedom.
So right now, that's probably where you should just stand up and leave, because you're like, why are we wasting our time?
Those are actually pretty much the same thing, and it is true.
If you have 50 observations, both the central limit theorem-- so here one of the things that you need to know is that if I want to talk about t-distribution for, say, eight observations, I need those observations to be Gaussian for real.
There's no central limit theorem happening at eight observations.
But really, what this is telling me is not that the central limit theorem kicks in.
It's telling me what are the asymptotics that kick in?
The law of large number, right?
This is exactly this guy.
That's here.
When I write this statement, what this picture is really telling us is that for n is equal to 50, I'm at the limit already almost.
There's virtually no difference between using the left-hand side or using sigma squared.
And now I start reducing.
40, I'm still pretty good.
We can start seeing that this thing is actually losing some mass on top, and that's because it's actually pushing it to the left and to the right in the tails.
And then we keep going, keep going, keep going.
So that's at 10.
When you're at 10, there's not much of a difference.
And so you can start seeing difference when you're at five, for example.
You can see the tails become heavier.
And the effect of this is that when I'm going to build, for example, a confidence interval to put the same amount of mass to the right of some number-- let's say I'm going to look at this q alpha over 2-- I'm going to have to go much farther, which is going to result in much wider confidence intervals to 4, 3, 2, 1.
So that's the t1.
Obviously that's the worst.
And if you ever use the t1 distribution, please ask yourself, why in the world are you doing statistics based on one observation?
But that's basically what it is.
So now that we have this t-distribution, we can define a more sophisticated test than just take your favorite estimator and see if it's far from the value you're currently testing.
That was our rationale to build a test before.
And the first test that's non-trivial is a test that exploits the fact that the maximum likelihood estimator, under some technical condition, has a limit distribution which is Gaussian with mean 0 when properly centered and a covariance matrix given by the Fisher information matrix.
Remember this Fisher information matrix?
And so this is the setup that we have.
So we have, again, an i.i.d.
sample.
Now I'm going to assume that I have a d-dimensional parameter space, theta.
And that's why I talk about Fisher information matrix-- and not just Fisher information.
It's a number.
And I'm going to consider two hypotheses.
So you're going to have h0, theta is equal to theta 0. h1, theta is not equal to theta 0.
And this is basically what we thought when we said, are we testing if a coin is fair or unfair.
So fair was p equals 1/2, and fair was p different from 1/2.
And here I'm just making my life a bit easier.
So now, I have this maximum likelihood estimate that I can construct.
Because let's say I know what p theta is, and so I can build a maximum likelihood estimator.
And I'm going to assume that these technical conditions that ensure that this maximum likelihood properly standardized converges to some Gaussian are actually satisfy, and so this thing is actually true.
So the theorem, the way I stated it-- if you're a little puzzled, this is not the way I stated it.
And the first time, the way we stated it was that theta hat mle minus theta not-- so here I'm going to place myself under the null hypothesis, so here I'm going to say under h0.
And honestly, if you have any exercise on tests, that's the way that it should start.
What is the distribution under h0?
Because otherwise you don't know what this guy should be.
So you have this, and what we showed is that this thing was going in distribution as n goes to infinity to some normal with mean 0 and covariance matrix, which was i of theta, which was here for the true parameter.
But here I'm under h0, so there's only one true parameter, which is theta 0.
This was our limiting central limit theorem for-- I mean, it's not really central limited theorem; limited theorem for the maximum likelihood estimator.
Everybody remembers that part?
The line before said, under technical conditions, I guess.
So now, it's not really stated in the same way.
If you look at what's on the slide, here I don't have the Fisher information matrix, but I really have the identity of rd.
If I have a random variable x, which has some covariance matrix sigma, how do I turn this thing into something that has covariance matrix identity?
So if this was a sigma squared, well, the thing I would do would be divide by sigma, and then I would have a 1, which is also known as the identity matrix of r1.
Now, what is this?
This was root of sigma squared.
So what I'm looking for is the equivalent of taking sigma and dividing by the square root of sigma, which-- obviously those are matrices-- I'm certainly not allowed to do.
And so what I'm going to do is I'm actually going to do the following.
Sigma 1 over root of sigma squared can be written as sigma to the negative 1/2.
And this is actually the same thing here.
So I'm going to write it as sigma to the negative 1/2, and now this guy is actually well-defined.
So this is a positive symmetric matrix, and you can actually define the square root by just taking the square root of its eigenvalues, for example.
And so you get sigma 1/2 equals and follows n0 identity.
And in general, I'm going to see something that looks like sigma 1/2 negative 1/2 sigma sigma negative 1/2.
And I have minus 1/2 plus 1 minus 1/2.
This whole thing collapses to 0, and it's actually the identity.
So that's the actual rule.
So if you're not familiar, this is basic multivariate Gaussian distribution computations.
Take a look at it.
If you feel like you don't need to look at it but you the basic maneuver, it's fine as well.
We're not going to go much deeper into that, but those are part of the thing that are sort of standard manipulations about standard Gaussian vectors.
Because obviously, standard Gaussian vectors arise from this theorem a lot.
So now I pre-multiplied my sigma to minus minus 1/2.
Now of course, I'm doing all of this in the asymptotics, and so I have this effect.
So if I pre-multiply everything by sigma to the 1/2, sigma being the Fisher information matrix at theta 0, then this is actually equivalent to saying that square root of n-- so now i of theta now plays the role of sigma-- times theta hat mle minus theta not goes in distribution as n goes to infinity to some multivariate standard Gaussian and 0 identity of rd.
And here, to make sure that we're talking about a multivariate distribution, I can put a d here-- so just so we know we're talking about the multivariate, though it's pretty clear from the context, since the covariance matrix is actually a matrix and not a number.
Michael?
[INAUDIBLE]
Oh, yeah.
Right.
Thanks.
So Yeah, you're right.
So that's a minus and that's a plus.
Thanks.
So yeah, anybody has a way to remember whether it's inverse Fisher information or Fisher information as a variance other than just learning it?
It is called information, so it's really telling me how much information I have.
So when a variance increases, I'm getting less and less information, and so this thing should actually be 1 over a variance.
The notion of information is 1 over a notion of variance.
So now I just wrote this guy like this, and the reason why I did this is because now everything on the right-hand side does not depend on any known parameter.
There's 0 and identity.
Those two things are just absolute numbers or absolute quantities, which means that this thing-- I call this quantity here-- what was the name that I used?
Started with a "p." Pivotal.
So this is a pivotal quantity, meaning that its distribution, at least asymptotic distribution, does not depend on any unknown parameter.
Moreover, it is indeed a statistic, because I can actually compute it.
I know theta 0 and I know theta hat mle.
One thing that I did, and you should actually complain about this, is on the board I actually used i of theta not.
And on the slides, it says i of theta hat.
And it's exactly the same thing that we did before.
Do I want to use the variance as a way for me to check whether I'm under the right assumption or not?
Or do I actually want to leave that part and just plug in the theta hat mle, which should go to the true one eventually?
Or do I actually want to just plug in the theta 0?
So this is exactly playing the same role as whether I wanted to see square root of Xn bar 1 minus Xn bar in the denominator of my test statistic for p, or if I wanted to see square root of 0.5, 1 minus 0.5 when I was testing if p was equal to 0.5.
So this is really a choice that's left up to you, and that's something you can really choose the two.
And as we said, maybe this guy is slightly more precise, but it's not going to extend to the case where theta 0 is not reduced to one single number.
Any questions?
So now we have our pivotal distribution, so from there this is going to be my test statistic.
I'm going to use this as a test statistic and declare that if this thing is too large, n absolute value-- because this is really a way to quantify how far theta hat is from theta 0.
And since theta hat should be close to the true one, when this thing is large in absolute value, it means that the true theta should be far from theta 0.
So this is my new test statistic.
Now, I said it should be far, but this is a vector.
So if I want a vector to be far, two vectors to be far, I measure their norm.
And so I'm going to form the Euclidean norm of this guy.
So if I look at the Euclidean norm of n-- and Euclidean norm is the one you know-- I'm going to take its square.
Let me now put a 2 here.
So that's just the Euclidean norm, and so the norm of vector x is just x transpose x.
In the slides, the transpose is denoted by prime.
Wow, that's hard to say.
Put prime in quotes.
That's a statistic standard that people do.
They put prime for transpose.
Everybody knows what the transpose is?
So I just make it flat and I do it like this, and then that means that's actually equal to the sum of the coordinates Xi squared.
And that's what you know.
But here, I'm just writing it in terms of vectors.
And so when I run to write this, this is equivalent, this is equal to-- well, the square root of n is going to pick up the square.
So I get square root of n times square root of n. So this guy is just 1/2.
So 1/2 times 1/2 is going to give me 1, and so I get theta hat mle minus theta.
And then I have e of theta not.
And then I get theta hat mle minus theta not.
And so by definition, I'm going to say that this is my test statistic Tn.
And now I'm going to have a test that rejects if Tn is large, because Tn is really measuring the distance between theta hat and theta 0.
So my test now is going to be psi, which rejects.
So it says 1 if Tn is larger than some threshold T. And how do I pick this T?
Well, by controlling my type I error-- sorry, the c by controlling my type I error.
So to choose c, what we have to check is that p under theta not-- so here it's theta not-- that I reject so that psi is equal to 1.
I want this to be equal to alpha, right?
That's how I maximize my type I error under the budget that's actually given to me, which is alpha.
So that's actually equivalent to checking whether p not of Tn is larger than c. And so if I want to find the c, all I need to know is what is the distribution of Tn when theta is equal to theta not?
Whatever this distribution is-- maybe it has some weird density like this-- whatever this distribution is, I'm just going to be able to pick this number, and I'm going to take this quintile alpha, here alpha, and I'm going to reject if I'm larger than alpha-- whatever this guy is.
So to be able to do that, I need to know what is the distribution of Tn when theta is equal to theta 0.
What is this distribution?
What is Tn?
It's the norm squared of this vector.
What is this vector?
What is the asymptotic distribution of this vector?
Yes?
[INAUDIBLE]
Just look one board up.
What is this asymptotic distribution of the vector for which we're taking the norm squared?
It's right here.
It's a standard Gaussian multivariate.
So when I look at the norm squared-- so if z is a standard Gaussian multivariate, then the norm of z squared, by definition of the norm squared, is the sum of the Zi squared.
That's just the definition of the norm.
But what is this distribution?
Chi-squared
That's a chi-square, because those guys are all of variance 1.
That's what the diagonal tells me-- only ones.
And they're independent because they have all these zeros outside of the diagonal.
So really, this follows some chi-squared distribution.
How many degrees of freedom?
Well, the number of them that I sell, d. So now I have found the distribution of Tn under this guy.
And that's true because this is true under h0.
If I was not under h0, again, I would need to take another guy here.
How did I use the fact that theta is equal to theta 0 when I centered by theta 0?
And that was very important.
So now what I know is that this is really equal-- why did I put 0 here?
So this here is actually equal.
So in the end, I need c such that the probability-- and here I'm not going to put a theta 0.
I'm just talking about the possibility of the random variable that I'm going to put in there.
It's a chi-square with d degrees of freedom [INAUDIBLE] is equal to alpha.
I just replaced the fact that this guy, Tn, under the distribution was just a chi-square.
And this distribution here is just really referring to the distribution of a chi-square.
There's no parameters here.
And now, that means that I look at my chi-square distribution.
It sort of looks like this.
And I'm going to pick some alpha here, and I need to read this number q alpha.
And so here what I need to do is to pick this q alpha here, for c. So take c to be q alpha, the quintile of order 1 minus alpha of a chi-squared distribution with this d degree of freedom.
And why do I say 1 minus alpha?
Because again, the quintiles are usually referring to the area that's to the left of them by-- well, actually, it's by a convention.
However, in statistics, we only care about the right tail usually, so it's not very convenient for us.
And that's why rather than calling this guy s sub 1 minus alpha all the time, I write it q alpha.
So now you have this q alpha, which is the 1 minus alpha quintile, or quintile of order 1 minus alpha of chi squared d. And so now I need to use a table.
For each d, this thing is going to take a different value, and this is why I can not just spit out a number to you like I spit out 1.96.
Because if I were able to do that, that would mean that I would remember an entire column of this table for each possible value of d, and that I just don't know.
So you need just to look at tables, and this is what it will tell you.
Often software will do that, too.
You don't have to search through tables.
And so just as a remark is that this test, Wald's test, is also valid when I have this sort of other alternative that I could see quite a lot-- if I actually have what's called a one-sided alternative.
By the way, this is called Wald's test-- so taking Tn to be this thing.
So this is Wald's test.
Abraham Wald was a famous statistician in the early 20th century, who actually was at Columbia for quite some time.
And that was actually at the time where statistics were getting very popular in India, so he was actually traveling all over India in some dinky planes.
And one of them crashed, and that's how he died-- pretty young.
But actually, there's a huge school of statistics now in India thanks to him.
There's the Indian Statistical Institute, which is actually a pretty big thing and trans the best statisticians.
So this is called Wald's test, and it's actually a pretty popular test.
Let's just look back a second.
So you can do the other alternatives, as I said, and for the other alternatives you can actually do this trick where you put theta 0 as well, as long as you take the theta 0 that's the closest to the alternative.
You just basically take the one that's the least favorable to you-- the alternative, I mean.
So what is this thing doing?
If you did not know anything about statistics and I told you here's a vector-- that's the mle vector, theta hat mle.
So let's say this theta hat mle takes the values, say-- so let's say theta hat mle takes values, say, 1.2, 0.9, and 2.1.
And then testing h0, theta is equal to 1, 1, 2, versus theta is not equal to the same number.
That's what I'm testing.
So you compute this thing and you find this.
If you don't know any statistics, what are you going to do?
You're just going to check if this guy goes to that guy, and probably what you're going to do is compute something that looks like the norm squared between those guys-- so the sum.
So you're going to do 1.2 minus 1 squared plus 0.9 minus 1 squared plus 2.1 minus 2 squared and check if this number is large or not.
Maybe you are going to apply some stats to try to understand how those things are, but this is basically what you are going to want to do.
What Wald's test is telling you is that this average is actually not what you should be doing.
It's telling you that you should have some sort of a weighted average.
Actually, it would be a weighted average if I was guaranteed that my Fisher information matrix was diagonal.
If my Fisher information matrix is diagonal, looking at this number minus this guy, transpose i, and then this guy minus this, that would look like I have some weight here, some weight here, and some weight here.
Sorry, that's only three.
So if it has non-zero numbers on all of its nine entries, then what I'm going to see is weird cross-terms.
If I look at some vector pre-multiplying this thing and post-multiplying this thing-- so if I look at something that looks like this, x transpose i of theta not, x transpose-- think of x as being theta hat mle minus theta-- so if I look at what this guy looks like, it's basically a sum over i and j of Xi, Xj, i, theta not Ij.
And so if none of those things are 0, you're not going to see a sum of three terms that are squares, but you're going to see a sum of nine cross-products.
And it's just weird.
This is not something standard.
So what is Wald's test doing for you?
Well, it's saying, I'm actually going to look at all the directions all at once.
Some of those directions are going to have more or less variance, i.e., less or more information.
And so for those guys, I'm actually going to use a different weight.
So what you're really doing is putting a weight on all directions of the space at once.
So what this Wald's test is doing-- by squeezing in the Fisher information matrix, it's placing your problem into the right geometry.
It's a geometry that's distorted and where balls become ellipses that are distorted in some directions and shrunk in others, or depending on if you have more variance or less variance in those directions.
Those directions don't have to be aligned with the axes of your coordinate system.
And if they were, then that would mean you would have a diagonal information matrix, but they might not be.
And so there's this weird geometry that shows up.
There is actually an entire field, admittedly a bit dormant these days, that's called information geometry, and it's really doing differential geometry on spaces that are defined by Fisher information matrices.
And so you can do some pretty hardcore-- something that I certainly cannot do-- differential geometry , just by playing around with statistical models and trying to understand with the geometry of those models are.
What does it mean for two points to be close in some curved space?
So that's basically the idea.
So this thing is basically curving your space.
So again, I always feel satisfied when my estimator on my test does not involve just computing an average and checking if it's big or not.
And that's not what we're doing here.
We know that this theta hat mle can be complicated-- CF problem set, too, I believe.
And we know that this Fisher information matrix can also be pretty complicated.
So here, your test is not going to be trivial at all, and that requires understanding the mathematics behind it.
I mean, it all built upon this theorem that I just erased, I believe, which was that this guy here inside this norm was actually converging to some standard Gaussian.
So there's another test that you can actually use.
So Wald's test is one option, and there's another option.
And just like maximum likelihood estimation and method of moments would sometimes agree and sometimes disagree, those guys are going to sometimes agree and sometimes disagree.
And this test is called the likelihood ratio test.
So let's parse those words-- "likelihood," "ratio," "test."
So at some point, I'm going to have to take the likelihood of something divided by the likelihood of some other thing and then work with this.
And this test is just saying the following.
Here's the simplest principle you can think of.
You're going to have to understand the notion of likelihood in the context of statistics.
You just have to understand the meaning of the word "likelihood."
This test is just saying if I want to test h0, theta is equal to theta 0, versus theta is equal to theta 1, all I have to look at is whether theta 0 is more or less likely than theta 1.
And I have an exact number that spits out.
Given a theta 0 or a theta 1 and given data, I can put in this function called the likelihood, and they tell me exactly how likely those things are.
And so all I have to check is whether one is more likely than the other, and so what I can do is form the likelihood of theta, say, 1 divided by the likelihood of theta 0 and check if this thing is larger than 1.
That would mean that this guy is more likely than that guy.
That's a natural way to proceed.
Now, there's one caveat here, which is that when I do hypothesis testing and I have this asymmetry between h0 and h1, I still need to be able to control what my probably of type I error is.
And here I basically have no knob.
This is something if you give me data in theta 0 and theta 1 I can compute to you and spit out the yes/no answer.
But I have no way of controlling the type II and type I error, so what we do is that we replace this 1 by some number c. And then we calibrate c in such a way that the type I error is exactly at level alpha.
So for example, if I want to make sure that my type I error is always 0, all I have to do is to say that this guy is actually never more likely than that guy, meaning never reject.
And so if I let c go to infinity, then this is actually going to make my type I error go to zero.
But if I let c go to negative infinity, then I'm always going to conclude that h1 is the right one.
So I have this straight off, and I can turn this knob by changing the values of c and get different results.
And I'm going to be interested in the one that maximizes my chances of rejecting the null hypothesis while staying under my alpha budget of type I error.
So this is nice when I have two very simple hypotheses, but to be fair, we've actually not seen any tests that correspond to real-life example.
Where theta 0 was of the form am I equal to, say, 0.5 or am I equal to 0.41, we actually sort of suspected that if somebody asked you to perform this test, they've sort of seen the data before and they're sort of cheating.
So it's typically something am I equal to 0.5 or not equal to 0.5 or am I equal to 0.5 or larger than 0.5.
But it's very rare that you actually get only two points to test-- am I this guy or that guy?
Now, I could go on.
There's actually a nice mathematical theory, something called the Neyman-Pearson lemma that actually tells me that this test, the likelihood ratio test, is the test, given the constraint of type I error, that will have the smallest type II error.
So this is the ultimate test.
No one should ever use anything different.
And we could go on and do this, but in a way, it's completely irrelevant to practice because you will never encounter such tests.
And I actually find students that they took my class as sophomores and then they're still around a couple of years later and they're doing, and they're like, I have this testing problem and I want to use likelihood ratio test, the Neyman-Pearson one, but I just can't because it just never occurs.
This just does not happen.
So here, rather than going into details, let's just look at what building on this principle we can actually make a test that will work.
So now, for simplicity, I'm going to assume that my alternatives-- so now, I still have a d dimensional vector theta.
And what I'm going to assume is that the null hypothesis is actually only testing if the last coefficients from r plus 1 to d are fixed numbers.
So in this example, where I have theta was equal-- so if I have d equals 3, here's an example.
h0 is theta 2 equals 1, and theta 3 equals 2.
That's my h0, but I say I don't actually care about what theta 1 is going to be.
So that's my null hypothesis.
I'm not going to specify right now what the alternative is.
That's what the null is.
And in particular, this null is actually not of this form.
It's not restricting it to one point.
It's actually restricting it to an infinite amount of points.
Those are all the vectors of the form theta 1 1, 2 for all theta 1 in, say, r. That's a lot of vectors, and so it's certainly not like it's equal to one specific vector.
So now, what I'm going to do is I'm actually going to look at the maximum likelihood estimator, and I'm going to say, well, the maximum likelihood estimator, regardless of anything, is going to be close to.
reality.
Now, if you actually tell me ahead of time that the true parameter is of this form, I'm not going to maximize over all three coordinates of theta.
I'm just going to say, well, I might as well just set the second one to 1, the third one to 2, and just optimize for this guy.
So effectively, you can say if you're telling me that this is the reality, I can compute a constrained maximum likelihood estimator which is constrained to look like what you think reality is.
So this is what the maximum likelihood estimator is.
That's the one that's maximizing, say, here the log likelihood over the entire space of candidate vectors, of candidate parameters.
But this partial one, this is the constraint mle.
That's the one that's actually not maximizing our real thetas, but only over the thetas that are plausible under the null hypothesis.
So in particular, if I look at ln of this constraint thing theta hat n c compared to ln, theta hat-- let's say n mle, so we know which one-- which one is bigger?
The first one is bigger.
So why?
[INAUDIBLE]
So the second one is maximized over a larger space.
Right.
So I have this all of theta, which are all the parameters I can take, and let's say theta 0 is this guy.
I'm maximizing a function over all these things.
So if the true maximum is this here, then the two things are equal, but if the maximum is on this side, then the one on the right is actually going to be larger.
They're maximizing over a bigger space, so this guy has to be less than this guy.
So maybe it's not easy to see.
So let's say that this is theta and this is theta 0 and now I have a function.
The maximum over theta 0 is this guy here, but the maximum over the entire space is here.
So the maximum over a larger space has to be larger than the maximum over a smaller space.
It can be equal, but the one in the bigger space can be even bigger.
However, if my true theta actually did belong to theta 0-- if h0 was true-- what would happen?
Well, if theta 0 is true, then theta isn't theta 0, and since the maximum likelihood should be close to theta, it should be the case that those two things should be pretty similar.
I should be in a case not in this kind of thing, but more in this kind of position, where the true maximum is actually attained at theta 0.
And in this case, they're actually of the same size, those two things.
If it's not true, then I'm going to see a discrepancy between the two guys.
So my test is going to be built on this intuition that if h0 is true, the values of the likelihood at theta hat mle and at the constraint mle should be pretty much the same.
But if theta hat-- if it's not true, then the likelihood of the mle should be much larger than the likelihood of the constrained mle.
And this is exactly what this test is doing.
So that's the likelihood ratio test.
So rather than looking at the ratio of the likelihoods, we look at the difference of the log likelihood, which is really the same thing.
And there is some weird normalization factor, too, that shows up here.
And this is what we get.
So if I look at the likelihood ratio test, so it's looking at two times ln of theta hat mle minus ln of theta hat mle constrained.
And this is actually the test statistic.
So we've actually decided that this statistic is what?
It's non-negative, right?
We've also decided that it should be close to zero if h0 is true and of course then maybe far from zero if h0 is not true.
So what should be the natural test based on Tn?
Let me just check that it's-- well, it's already there.
So the natural test is something that looks like indicator that Tn is larger than c. And you should say, well, again?
I mean, we just did that.
I mean, it is basically the same thing that we just did.
Agreed?
But the Tn now is different.
The Tn is the difference of log likelihoods, whereas before the Tn was this theta hat minus theta not transpose identity of Fisher information matrix theta hat minus theta not.
And this, there's no reason why this guy should be of the same form.
Now, if I have a Gaussian model, you can check that those two things are actually exactly the same.
But otherwise, they don't have any reason to be.
And now, what's happening is that under some technical conditions-- if h0 is true, now what happens is that if I want to calibrate c, what I need to do is to look at what is the c such that this guy is equal to alpha?
And that's for the distribution of T under the knob.
But there's not only one.
The null hypothesis here was actually just the family of things.
It was not just one vector.
It was an entire family of vectors, just like in this example.
So if I want my type I error to be constrained over the entire space, what I need to make sure of is that the maximum overall theta n theta not is actually equal to alpha.
Agreed?
Yeah?
[INAUDIBLE]
So not equal.
In this case, it's going to be not equal.
I mean, it can really be anything you want.
It's just you're going to have a different type II error.
I guess here we're sort of stuck in a corner.
We built this T, and it has to be small under the null.
And whatever not the null is, we just hope that it's going to be large.
So even if I tell you what the alternative is, you're not going to change anything about the procedure.
So here, q alpha-- so what I need to know is that if h0 is true, then Tn in this case actually converges to some chi-square distribution.
And now here, the number of degrees of freedom is kind of weird.
But actually, what it should tell you is, oh, finally, I know when you call this parameter degrees of freedom rather than dimension or just d parameter.
It's because here what we did is we actually pinned down everything, but r-- sorry, we pinned down everything but r coordinates of this thing.
And so now I'm actually wondering why-- did I make a mistake here?
I think this should be chi square with r degrees of freedom.
Let me check and send you an update about this, because the number of degrees of freedom, if you talk to normal people they will tell you that here the number of degrees of freedom is r. This is what's allowed to move, and that's what's called degrees of freedom.
The rest is pinned down to being something.
So here, this chi-square should be a chi-squared r. And that's something you just have to believe me.
Anybody guess what theorem is going to tell me this?
In some cases, it's going to be Cochran's theorem-- just something that tells me that thing's [INAUDIBLE].. Now, here, I use the very specific form of the null alternative.
And so for those of you who are sort of familiar with linear algebra, what I did here is h0 consists in saying that theta belongs to an r dimensional linear space.
It's actually here, the r dimensional linear space of vectors, that have the first r coordinates that can move and the last coordinates that are fixed to some number.
Actually, it's undefined space because it doesn't necessarily go through zero.
And so I have this defined space that has dimension r, and if I were to constrain it to any other r dimensional space, that would be exactly the same thing.
And so to do that, essentially what you need to do is to say, if I take any matrix that's say, invertible-- let's call it u-- and then so h0 is going to be something like of the form u times theta and now I look only at the coordinates r plus 1 2d, then I want to fix those guys to some numbers.
So I want to call them theta, so let's call them tau.
So it's going to be tau r plus 1, all the way to tau d. So this is not part of the requirements, but just so you know, it's really not a matter of keeping only some coordinates.
Really, what matters is the dimension in the sense of linear subspaces of the problem, and that's what determines what your degrees of freedom are.
So now that we know what the asymptotic distribution is under the null, then we know basically that we know how which table we need to pick our q alpha from.
And here, again, the table is a chi-squared table, but here, the number of degrees of freedom is this weird d minus r degrees of freedom thing.
I just said it was r. I'm just checking, actually, if I'm-- it's r. It's definitely r. So here we've made tests.
We're testing if r parameter theta was explicitly in some set or not.
By explicitly, I mean we're saying, is theta like this or is theta not like this?
Is theta equal to theta not or is theta not equal to theta not?
Are the last coordinates of theta equal to those fixed numbers, or are they not?
There was something I was stating directly about theta.
But there's going to be some instances where you actually want to test something about a function of data, not data itself.
For example, is the difference between the first coordinate of theta and the second coordinate of theta positive?
That's definitely something you might want to test, because maybe theta 1 is-- let me try to think of some good example.
I don't know.
Maybe theta 1 is your drawing accuracy with the right hand and theta 2 is the drawing accuracy with the left hand, and I'm actually collecting data on young children to be able to test early on whether they're going to be left-handed or right-handed, for example.
And so I want to just compare those two with respect to each other, but I don't necessarily need to know what the absolute score for this handwriting skills are.
So sometimes it's just interesting to look at the difference of things or maybe the sum, say the combined effect.
Maybe this is my two measurements of blood pressure, and I just want to talk about the average blood pressure.
And so I can make a linear combination of those two, and so those things implicitly depend on theta.
And so I can generically encapsule them in some test of the form g of theta is equal to 0 versus g of theta is not equal to 0.
And sometimes, in the first test that we saw, g of theta was just the identity or maybe the identity minus 0.5.
If g of theta is theta minus 0.5, that's exactly what we've been testing.
If g of theta is theta minus 0.5 and theta is p, the parameter of a coin, this is exactly of this form.
So this is a simple one, but then there's more complicated ones we can think of.
So how can I do this?
Well, let's just follow a recipe.
So we traced back.
We were trying to build a test statistic which was pivotal.
We wanted to have this thing that had nothing that depended on the parameter, and the only thing we had for that that we built in our chi-square test one is basically some form of central limit theorem.
Maybe it's for the maximum likelihood estimator.
Maybe it's for the average, but it's basically some form of asymptotic normality of the estimator.
And that's what we started from every single time.
So let's assume that I have this, and I'm going to talk very abstractly.
Let's assume that I start with an estimator.
Doesn't have to be the mle.
It doesn't have to be the average, but it's just something.
And I know that I have the estimator such that this guy converges in distribution to some n0, and I have some covariance matrix theta.
Maybe it's not the Fisher information.
Maybe that's something that's not as good as the mle, meaning that this is going to give me less information than the Fisher information, less accuracy.
And now I can actually just say, OK, if I know this about theta, I can apply the multivariate delta method, which tells me that square root of n, g of theta hat, minus g of theta goes in distribution to some n0.
And then the price to pay in one dimension was multiplying the square root of the derivative, and we know that in multivariate dimensions pre-multiplying by the gradient, post-multiplying by the gradient.
So I'm going to write delta g of theta transpose sigma-- sorry, not delta; nabla-- g of theta-- so gradient.
And here, I assume that g takes values into rk.
That's what's written here.
g takes value from d to k, but think of k as being 1 for now.
So the gradient is really just a vector and not a matrix.
That's your usual gradient for real valid functions.
So effectively, if g takes values in dimension 1, what is the size of this matrix?
I only ask trivial questions.
Remember, that's rule number one.
It's one by one, right?
And you can check it, because on this side those are just the difference between numbers.
And it would be kind of weird if they had a covariance matrix at the end.
I mean, this is a random variable, not a random vector.
So I know that this thing happens.
And now, if I basically divide by the square root of this thing-- so for board I'm working with k is equal to 1 divided by square root of delta g of theta transpose sigma delta nabla-- sorry, g of theta-- then this thing should go to some standard normal random variable, standard normal distribution.
I just divided by square root of the variance here, which is the usual thing.
Now, if you do not have a univariate thing, you do the same thing we did before, which is 3 multiplied by the covariance matrix to the negative 1/2-- so before this role was played by the inverse Fisher information matrix.
That's why we ended up having i of theta to the 1/2, and now we just have this gamma, which is just this function that I wrote up there.
That could be potentially k by k if g takes values into rk.
Yes?
[INAUDIBLE]
Yeah, the gradient of a vector is just the vector with all the derivatives with respect to each component, yes.
So you know the word vector for derivatives, but not for vectors?
I mean, the word gradient you use for one-dimensional?
Yes, derivative in one dimension.
Now, of course, here, you notice there's something-- I actually have a little caveat here.
I want this to have rank k. I want this to be invertible.
I want this matrix to be invertible.
Even for the Fisher information matrix, I sort of need it to be invertible.
Even for the original theorem, that was part of my technical condition, just so that I could actually write Fisher information matrix inverse.
And so here, you can make your life easy and just assume that it's true all the time, because I'm actually writing in a fairly abstract way.
But in practice, we're going to have to check whether this is going to be true for specific distributions.
And we will see an example towards the end of the chapter, the multinomial, where it's actually not the case that Fisher information matrix exists.
The asymptotic covariance matrix, is not invertible, so it's not the inverse of a Fisher information matrix.
Because to be the inverse of someone, you need to be invertible yourself.
And so now what I can do is apply Slutsky.
So here, what I needed to have is theta, the true theta.
So what I can do is just put some theta hat in there, and so that's the gamma of theta hat that I see there.
And if theta is true, then g of theta is equal to 0.
That's what we assume.
That was our h0, was that under h0 g of theta is equal to 0.
So the number I need to plug in here, I don't need to replace theta here.
What I need to replace here is 0.
Now let's go back to what you were saying.
Here you could say, let me try to replace 0 here, but there is no such thing.
There is no g here.
It's only the gradient of g. So this thing that says replace theta by theta 0 wherever you see it could not work here.
If g was invertible, I could just say that theta is equal to g inverse of 0 in the null and then I could plug in that value.
But in general, it doesn't have to be invertible.
And it might be a pain to invert g, even.
I mean, it's not clear how you can invert all functions like that.
And so here you just go with Slutsky, and you say, OK, I'm just going to put theta hat in there.
But this guy, I know I need to check whether it's 0 or not.
Same recipe we did for theta, except we do it for g of theta now.
And now I have my asymptotic thing.
I know this is a pivotal distribution.
This might be a vector.
So rather than looking at the matrix itself, I'm going to actually look at the norm-- rather than looking at the vectors, I'm going to look at their square norm.
That gives me a chi square, and I reject when my test statistic, which is the norm square, exceeds the quintile of a chi square-- same as before, just doing on your own.
Before we part ways, I wanted to just mention one thing, which is look at this thing.
If g was of dimension 1, the Euclidean norm in dimension 1 is just the absolute value of the number, right?
Which means that when I am actually computing this, I'm looking at the square, so it's the square of something.
So it means that this is the square of a Gaussian.
And it's true that, indeed, the chi squared 1 is just the square of a Gaussian.
Sure, this is the tautology, but let's look at this test now.
This test was built using Wald's theory and some pretty heavy stuff.
But now if I start looking at Tn and I think of it as being just the absolute value of this quantity over there, squared, what I'm really doing is I'm looking at whether the square of some Gaussian exceeds the quintile of a chi squared of 1 degree of freedom, which means that this thing is actually equivalent-- completely equivalent-- to the test.
So if k is equal to 1, this is completely equivalent to looking at the absolute value of something and check whether it's larger than, say, q over 2-- well, than q alpha-- well, that's q alpha over 2-- so that the probability of this thing is actually equal to alpha.
And that's exactly what we've been doing before.
When we introduced tests in the first place, we just took absolute values, said, well, is the absolute value of a Gaussian in the limit.
And so it's the same thing.
So this is actually equivalent to the probability that the norm squared is larger so that the chi squared of some normal-- and that's the q alpha of some chi squared with one degree of freedom.
Those are exactly the two same tests.
So in one dimension, those things just collapse into being one little thing, and that's because there's no geometry in one dimension.
It's just one dimension, whereas if I'm in a higher dimension, then things get distorted and things can become weird.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we've been talking about this chi square test.
And the name chi square comes from the fact that we build a test statistic that has asymptotic distribution given by the chi square distribution.
Let's just give it another shot.
OK.
This test.
Who has actually ever encountered the chi square test outside of a stats classroom?
All right.
So some people have.
It's a fairly common test that you might encounter.
And it was essentially to test, if given some data with a fixed probability mass function, so a discrete distribution, you wanted to test if the PMF was equal to a set value, p0, or if it was different from p0.
And the way the chi square arose here was by looking at Wald's test.
And essentially if you write-- so Wald's is the one that has the chi square as the limiting distribution, and if you invert the covariance matrix, the asymptotic covariance matrix, so you compute the Fisher information, which in this particular case does not exist for the multinomial distribution, but we found the trick on how to do this.
We remove the part that forbid it to be invertible, then we found this chi square distribution.
In a way we have this test statistic, which you might have learned as a black box, laundry list, but going through the math which might have been slightly unpleasant, I acknowledge, but really told you why you should do this particular normalization.
So since some of you requested a little more practical examples of how those things work, let me show you a couple.
The first one is you want to answer the question, well, you know, when should I be born to be successful.
Some people believe in zodiac, and so Fortune magazine actually collected the signs of 256 heads of the Fortune 500.
Those were taken randomly.
And they were collected there, and you can see the count of number of CEOs that have a particular zodiac sign.
And if this was completely uniformly distributed, you should actually get a number that's around 256 divided by 12, which in this case is 21.33.
And you can see that there is numbers that are probably in the vicinity, but look at this guy.
Pisces, that's 29.
So who's Pisces here?
All right.
All right, so give me your information and we'll meet again in 10 years.
And so basically you might want to test if actually the fact that it's uniformly distributed is a valid assumption.
Now this is clearly a random variable.
I pick a random CEO and I measure what his zodiac sign is.
And I want to know, so it's a probability over, I don't know, 12 zodiac signs.
And I want to know if it's uniform or not.
Uniform sounds like it should be the status quo, if you're reasonable.
And maybe there's actually something that moves away.
So we could do this, in view of these data is there evidence that one is different.
Here is another example where you might want to apply the chi square test.
So as I said, the benchmark distribution was the uniform distribution for the zodiac sign, and that's usually the one I give you.
1 over k, 1 over k, because well that's sort of the zero, the central point for all distributions.
That's the point, the center of what we call the simplex.
But you can have another benchmark that sort of makes sense.
So for example this is an actual dataset where 275 jurors were identified, racial group were collected, and you actually might want to know if you know juries in this country are actually representative of the actual population.
And so here of course, the population is not uniformly distributed according to racial group.
And the way you actually do it is you actually go on Wikipedia, for example, and you look at the demographics of the United States, and you find that the proportion of white is 72%, black is 7%, Hispanic is 12, and other is about 9%.
So that's a total of 1.
And this is what we actually measured for some jurors.
So for this guy, you can actually run the chi square test.
You have the estimated proportion, which comes from this first line.
You have the tested proportion, p0, that comes from the second line, and you might want to check if those things actually correspond to each other.
OK, so I'm not going to do it for you, but I sort of invite you to do it and test, and see how this compares to the quantiles of the appropriate chi square distribution and see what you can conclude from those two things.
All right.
So this was the multinomial case.
So this is essentially what we did.
We computed the MLE under the right constraint, and that was our test statistic that converges to the chi square distribution.
So if you've seen it before, that's all that was given to you.
Now we know why the normalization here is p0 j and not p0 j squared or square root of p0 j, or even 1.
I mean it's not clear that this should be the right normalization, but we know that's what comes from taking the right normalization, which comes from the Fisher information.
All right?
OK.
The thing I wanted to move onto, so we've basically covered chi square test.
Are there any questions about chi square test?
And for those of you who were not here on Thursday, I'm really just-- do not pretend I just did it.
That's something we did last Thursday.
But are there any questions that arose when you were reading your notes, things that you didn't understand?
Yes
Is there like a formal name?
Before we had talked about how what we call the Fisher information [INAUDIBLE],, still has the same [INAUDIBLE] because it's the same number
So it's not the Fisher.
The Fisher information does not exist in this case.
And so there's no appropriate name for this.
It's the pseudoinverse of the asymptotic covariance matrix, and that's what it is.
I don't know if I mentioned it last time, but there's this entire field that uses-- you know, for people who really aspire to differential geometry but are stuck in the stats department, and there's this thing called information geometry, which is essentially studying the manifolds associated to the Fisher information metric, the metric that's associated to Fisher information.
And so those of course can be lower dimensional manifolds, not only distorts the geometry but forces everything to live on a lower dimension, which is what happens when your Fisher information does not exist.
And so there's a bunch of things that you can study, what this manifold looks like, et cetera.
But no, there's no particular terminology here about going here.
To be fair, within the scope of this class, this is the only case where you-- multinomial case is the only case where you typically see a lack of a Fisher information matrix.
And that's just because we have these extra constraints that the sum of the parameters should be 1.
And if you have an extra constraint that seems like it's actually remove one degree of freedom, this will happen inevitably.
And so maybe what you can do is reparameterize.
So if I actually reparameterize everything function of p1 to p k minus 1, and then 1 minus the sum, this would not have happened.
Because I have only a k-dimensional space.
So there's tricks around this to make it exist if you want it to exist.
Any other question?
All right.
So let's move on to Student's t-test.
We mentioned it last time.
So essentially you've probably done it more even in the homework than you've done it in lectures, but just quickly this is essentially the test.
That's the test when we have an actual data that comes from a normal distribution.
There is no Central Limit Theorem that exists.
This is really to account for the fact that for smaller sample sizes, it might be the case that it's not exactly true that when I look at xn bar minus mu divided by-- so if I look at xn bar minus mu divided by sigma times square root of n, then this thing should have N 0, 1 distribution approximately.
Right?
By the Central Limit Theorem.
So that's for n large.
But if n is small, then it's still true when the data is N mu, sigma squared, then it's true that square root of n-- so here it's approximately.
And this is always true.
But I don't know sigma in practice, right?
Maybe mu, it comes from my, maybe mu comes from my mu 0, maybe something from the test statistic where mu actually is here.
But for this guy I'm going to have inevitably to find an estimator.
And now in this case, for small n, this is no longer true.
And what the t statistic is doing is essentially telling you what the distribution of this guy is.
So what you should say is that now this guy has a t distribution with n minus 1 degrees of freedom.
That's basically the laundry list stats that you would learn.
It says just look at a different table, that's what it is.
But we actually defined what a t distribution was.
And a t distribution is basically something that has the same distribution as some N 0, 1, divided by the square root of a chi square with d degrees of freedom divided by d. And that's a t distribution with d degrees of freedom.
And those two have to be independent.
And so what I need to check is that this guy over there is of this form.
OK?
So let's look at the numerator.
Well, square root of n, xn bar minus mu.
What is the distribution of this thing?
Is it an N 0, 1?
N 0, sigma squared?
N 0, sigma squared, right.
So I'm not going to put it here.
So if I want this guy to be N 0, 1, I need to divide by sigma, that's what we have over there.
So that's my N 0, 1 that's going to play the role of this guy here.
So if I want to go a little further, I need to just say, OK, now I need to have square root of n, and I need to find something here that looks like my square root of chi square divided by-- yeah?
Really quick question.
The equals sign with the d on top, that's just defined as?
No, that's just the distribution.
So, I don't know
Then never mind
Let's just write it like that, if you want.
I mean, that's not really appropriate to have.
Usually you write only one distribution on the right-hand inside of this little thing.
So not just this complicated function of distributions.
This is more like to explain.
OK, and so usually the thing you should say that t is equal to this X divided by square root of Z divided by d where X has normal distribution, Z has chi square distribution with d degrees of freedom.
So what do we need here?
Well I need to have something which looks like my sigma hat, right?
So somehow inevitably I'm going to need to have sigma hat.
Now of course I need to divide this by my sigma so that my sigma goes away.
And so now this thing here-- sorry, I should move on to the right, OK. And so this thing here, so sigma hat is square root of Sn.
And now I'm almost there.
So this thing is actually equal to square root of n. But this thing here is actually not a-- so this thing here follows a distribution which is actually a chi square, square root of a chi square distribution divided by n. Yeah, that's the square root chi square distribution with n minus 1 degrees of freedom divided by n, because sigma hat is equal to 1 over n sum from i equal 1 to n, xi minus x bar squared.
And we just said that this part here was a chi square distribution.
We didn't just say it, we said it a few lectures years back, that this thing was a chi square distribution, and the fact that the presence of this x bar here was actually removing one degree of freedom from this sum.
OK, so this guy here has the same distribution as a chi square n minus 1 divided by n. So I need to actually still arrange this thing a little bit to have a t distribution.
I should not see n here, but I should n minus 1.
The d is the same as this d here.
And so let me make the correction so that this actually happens.
Well, if I actually write this to be equal to-- so if I write square root of n minus 1, as on the slide, times xn bar minus mu divided by-- well let me write it as square root of Sn, which is my sigma hat.
Then what this thing is actually equal to, it follows a N 0, 1, divided by the square root of my chi square distribution with n minus 1 degrees of freedom.
And here the fact that I multiply by square root of n minus 1, and I have the square root of n here, is essentially the same as dividing here by n minus 1.
And that's my tn distribution.
My t distribution with n minus 1 degrees of freedom.
Just by definition of what this thing is.
OK?
All right.
Yes?
Where'd you get the square root from?
This guy?
Oh sorry, that's sigma squared.
Thank you.
That's the estimator of the variance, not the estimator of the standard deviation.
And when I want to divide it I divide by standard deviation.
Thank you.
Any other question or remark?
Shouldn't you divide by sigma squared?
The actual.
The estimator for the variance is equal to sigma squared times chi square, right?
The estimator for the variance.
Oh yes, you're right.
So there's a sigma squared here.
Is that what you're asking?
Yeah
Yes, absolutely.
And that's where, it get cancels here.
It gets canceled here.
OK?
So this is really a sigma squared times chi square.
OK.
So the fact that it's sigma squared is just because I can pull out sigma squared and just think those guys N 0, 1.
All right.
So that's my t distribution.
Now that I actually have a pivotal distribution, what I do is that I form the statistic.
Here I called it Tn tilde.
OK. And what is this thing?
I know that this has a pivotal distribution.
So for example, I know that the probability that Tn tilde in absolute value exceeds some number that I'm going to call q alpha over 2 for the t n minus 1, is equal to alpha.
So that's basically, remember the t distribution has the same shape as the Gaussian distribution.
What I'm finding is, for this t distribution, some number q alpha over 2 of t n minus 1 and minus q alpha over 2 of t minus 1.
So those are different from the Gaussian one.
Such that the area under the curve here is alpha over 2 on each side so that the probability that my absolute value exceeds this number is equal to alpha.
And that's what I'm going to use to reject the test.
So now my test becomes, for H0, say mu is equal to some mu 0, versus H1, mu is not equal to mu 0.
The rejection region is going to be equal to the set on which square root of n minus 1 times xn bar minus mu 0 this time, divided by square root of Sn exceeds, in absolute value, exceeds q-- sorry that's already here-- exceeds q alpha over 2 of t n minus 1.
So I reject when this thing increases.
The same as the Gaussian case, except that rather than reading my quantiles from the Gaussian table I read them from the Student table.
It's just the same thing.
So they're just going to be a little bit farther.
So this guy here is just going to be a little bigger than the one for the Gaussian one, because it's going to require me a little more evidence in my data to be able to reject because I have to account for the fluctuations of sigma hat.
So of course Student's test is used everywhere.
People use only t tests, right?
If you look at any data point, any output, even if you had 500 observations, if you look at the statistical software output it's going to say t test.
And the reason why you see t test is because somehow it's felt like it's not asymptotic.
You don't need to actually do, you know, to be particularly careful.
And anyway, if n is equal to 500, since the two curves are above each other it's basically the same thing.
So it doesn't really change anything.
So why not use the t test?
So it's not asymptotic.
It doesn't require Central Limit Theorem to kick in.
And so in particular it be run if you have 15 observations.
Of course, the drawback of the Student test is that it relies on the assumption that the sample is Gaussian, and that's something we really need to keep in mind.
If you have a small sample size, there is no magic going on.
It's not like Student t test allows you to get rid of this asymptotic normality.
It sort of assumes that it's built in.
It assumes that your data has a Gaussian distribution.
So if you have 15 observations, what are you going to do?
You want to test if the mean is equal to 0 or not equal to 0, but you have only 15 observations.
You have to somehow assume that your data is Gaussian.
But if the data is given to you, this is not math, you actually have to check that it's Gaussian.
And so we're going to have to find a test that, given some data, tells us whether it's Gaussian or not.
If I have 15 observations, 8 of them are equal to plus 1 and 7 of them are equal to minus 1, then it's pretty unlikely that you're going to be able to conclude that your data has a Gaussian distribution.
However, if you see some sort of spread around some value, you form a histogram maybe and it sort of looks like it's a Gaussian, you might want to say it's Gaussian.
And so how do we make this more quantitative?
Well, the sad answer to this question is that there will be some tests that make it quantitative, but here, if you think about it for one second, what is going to be your null hypothesis?
Your null hypothesis, since it's one point, it's going to be that it's Gaussian, and then the alternative is going to be that it's not Gaussian.
So what it means is that, for the first time in your statistician life, you're going to want to conclude that H0 is the true one.
You're definitely not going to want to say that it's not Gaussian, because then everything you know is sort of falling apart.
And so it's kind of a weird thing where you're sort of going to be seeking tests that have no power basically.
You're going to want to test that, and that's the nature.
The amount of alternatives, the number of ways you can be not Gaussian, is so huge that all tests are sort of bound to have very low power.
And so that's why people are pretty happy with the idea that things are Gaussian, because it's very hard to find a test that's going to reject this hypothesis.
And so we're even going to find some tests that are visual, where you're going to be able to say, well, sort of looks Gaussian to me.
It allows you to deal with the borderline cases pretty efficiently.
We'll see actually a particular example.
All right, so this theory of testing whether data comes from a particular distribution is called goodness of fit.
Is this distribution a good fit for my data?
That's the goodness of fit test.
We have just seen a goodness of fit test.
What was it?
Yeah.
The chi square test, right?
The case square test, we were given a candidate PMF and we were testing if this was a good fit for our data.
That was a goodness of fit test.
So of course multinomial is one example, but really what we have in the back of our mind is I want to test if my data is Gaussian.
That's basically the usual thing.
And just like you always see t test as the standard output from statistical software whether you ask for it or not, there will be a test for normality whether you ask it or not from any statistical software app.
All right.
So a goodness of fit test looks as follows.
There's a random variable X and you're given i.i.d.
copies of X, X1 to Xn, they come from the same distribution.
And you're going to ask the following question: does X have a standard normal distribution?
So for t distribution that's definitely the kind of questions you may want to ask.
Does X have a uniform distribution on 0, 1?
That's different from the distribution 1 over k, 1 over k, it's the continuous notion of uniformity.
And for example, you might want to test that-- so there's actually a nice exercise, which is if you look at the p-values.
So we've defined what the p-values were.
And the p-value's a number between 0 and 1, right?
And you could actually ask yourself, what is the distribution of the p-value under the null?
So the p-value is a random number.
It's the probability-- so the p-value-- let's look at the following test.
H0, mu is equal to 0, versus H1, mu is not equal to 0.
And I know that the p-value is-- so I'm going to form what?
I'm going to look at Xn bar minus mu times square root of n divided by-- let's say that we know sigma for one second.
Then the p-value is the probability that this is larger then square root of n little xn bar minus mu, minus 0 actually in this case, divided by sigma, where this guy is the observed.
OK.
So now you could say, well, how is that a random variable?
It's just a number.
It's just a probability of something.
But then I can view this as a function of this guy here when I plug it back to be a random variable.
So what I mean by this is that if I look at this value here, if I say that phi is the CDF of N 0, 1, so the p-value is the probability that it exceeds this.
So that's the probability that I'm either here or here
[INAUDIBLE]
No, it's not, right?
[INAUDIBLE]
This is a big X and this is a small x.
This is just where you plug in your data.
The p-value is the probability that you have more evidence against your null than what you already have.
OK, so now I can write it in terms of cumulative distribution functions.
So this is what?
This is phi of this guy, which is minus this thing here.
Well it's basically 2 times this guy, phi of minus square root of n, Xn bar divided by sigma.
That's my p-value.
If you give me data, I'm going to compute the average and plug it in there, and it can spit out the p-value.
Everybody agrees?
So now I can view this, if I start now looking back I say, well, where does this data come from?
Well, it could be a random variable.
It came from the realization of this thing.
So I can try to, I can think of this value, where now this is a random variable because I just plugged in a random variable in here.
So now I view my p-value as a random variable.
So I keep switching from small x to large X.
Everybody agrees what I'm doing here?
So I just wrote it as a deterministic function of some deterministic number, and now the function stays deterministic but the number becomes random.
And so I can think of this as some statistic of my data.
And I could say, well, what is the distribution of this random variable?
Now if my data is actually normally distributed, so I'm actually under the null, so under the null, that means that Xn bar times square root of n divided by sigma has what distribution?
Normal?
Well it was sigma, I assume I knew it.
So it's N 0, 1, right?
I divided by sigma here.
OK?
So now I have this random variable.
And so my random variable is now 2 phi of minus absolute value of a Gaussian.
And I'm actually interested in the distribution of this thing.
I could ask that.
Anybody has an idea of how you would want to tackle this thing?
If I ask you, what is the distribution of a random variable, how do you tackle this question?
There's basically two ways.
One is to try to find something that looks like the expectation of h of x for all h. And you try to write this using change of variables and something that looks like integral of h of x p of x dx.
And then you say, well, that's the density.
If you can read this for any h, then that's the way you would do it.
But there's a simpler way that does not involve changing variables, et cetera, you just try to compute the cumulative distribution function.
So let's try to compute the probability that 2 phi minus N 0, 1, is less than t. And maybe we can find something we know.
OK. Well that's equal to what?
That's the probability that a minus N 0, well let's say that an N 0, 1-- sorry, N 0, 1 absolute value is greater than minus phi inverse of t over 2.
And that's what?
Well, it's just the same thing that we had before.
It's equal to-- so if I look again, this is the probability that I'm actually on this side or that side of this number.
And this number is what?
It's minus phi of t over 2.
Why do I have a minus here?
That's fine, OK.
So it's actually not this, it's actually the probability that my absolute value-- oh, because phi inverse.
OK. Because phi inverse is-- so I'm going to look at t between 0 and-- so this number is ranging between 0 and 1.
So it means that this number is ranging between 0-- well, the probability that something is less than t should be ranging between the numbers that this guy takes, so that's between 0 and 2.
Because this thing takes values between 0 and 2.
I want to see 0 and 1, though
Negative absolute value is always less than [INAUDIBLE]
Yeah.
You're right, thank you.
So this is always some number which is less than 0, so the probability that the Gaussian is less than this number is always less than the probability it's less than 0, which is 1/2, so t only has to be between 0 and 1.
Thank you.
And so now for t between 0 and 1, then this guy is actually becoming something which is positive, for the same reason as before.
And so that's what?
That's just basically 2 times phi of phi inverse of t over 2.
That's just playing with the symmetry a little bit.
You can look at the areas under the curve.
And so what it means is that those two guys cancel.
This is the identity.
And so this is equal to t. So which distribution has a density-- sorry, which distribution has a cumulative distribution function which is equal to t for t between 0 and 1?
That's the uniform distribution, right?
So it means that this guy follows a uniform distribution on the interval 0, 1.
And you could actually check that.
For any test you're going to come up with, this is going to be the case.
Your p-value under the null will have a distribution which is uniform.
So now if somebody shows up and says, here's my test, it's awesome, it just works great.
I'm not going to explain to you how I built it, it's a complicated statistics that involve moments of order 27.
And I'm like, OK, you know, how am I going to test that your test statistic actually makes sense?
Well one thing I can do is to run a bunch of data, draw a bunch of samples, compute your test statistic, compute the p-value, and check if my p-value has a uniform distribution on the interval 0, 1.
But for that I need to have a test that, given a bunch of observations, can tell me whether they're actually distributed uniformly on the interval 0, 1.
And again one thing I could do is build a histogram and see if it looks like that of a uniform, but I could also try to be slightly more quantitative about this
Why does the [INAUDIBLE] have to be for a [INAUDIBLE]?
For two tests?
For each test.
Why does the p-value have to be normal?
I mean, uniform
It's uniform under the null.
So because my test statistic was built under the null, and so I have to be able to plug in the right value in there, otherwise it's going to shift everything for this particular test
At the beginning while your probabilities were of big Xn, that thing.
That thing is the p-value
That's the p-value, right?
That's the definition of the p-value

So it's the probability that my test statistic exceeds what I've actually observed
So how you run the test is basically you have your observations and plug them into the cumulative distribution function for a normal, and then see if it falls under the given--
Yeah.
So my p-value is just this number when I just plug in the values that I observe here.
That's one number.
For every dataset you're going to give me, it's going to be one number.
Now what I can do is generate a bunch of datasets of size n, like 200 of them.
And then I'm going to have a new sample of say 200, which is just the sample of 200 p-values.
And I want to test if those p-values have a uniform distribution.
OK?
Because that's the distribution they should be having.
All right?
OK.
This one we've already seen.
Does x have a PMF with 30%, 50%, and 20%?
That's something I could try to test.
That looks like your grade point distribution for this class.
Well not exactly, but that looks like it.
So all these things are known as goodness of fit tests.
The goodness of fit test is something that you want to know if the data that you have at hand follows the hypothesized distribution.
So it's not a parametric test.
It's not a test that says, is my mean equal to 25 or not.
Is my proportion of heads larger than 1/2 or not?
It's something that says, my distribution this particular thing.
So I'm going to write them as goodness of fit, G-O-F here.
You don't need to have parametric modeling to do that.
So how do I work?
So if I don't have any parametric modeling, I need to have something which is somewhat non-parametric, something that goes beyond computing the mean and the standard deviation, something that computes some intrinsic non-parametric aspect of my data.
And just like here we made this computation, what we did is we said well, if I actually check that the CDF of my data, that my p-value is uniform, then I know it's uniform.
So it means that the cumulative distribution function has an intrinsic value about it that captures the entire distribution.
Everything I need to know about my distribution is captured by the cumulative distribution function.
Now I have an empirical way of computing, I have a data-driven way of computing an estimate for the cumulative distribution function, which is using the old statistical trick which consists of replacing expectations by averages.
So as I said, the cumulative distribution function for any distribution, for any random variable, is-- so F of t is the probability that X is less than or equal to t, which is equal to the expectation of the indicator that X is less than or equal to t. That's the definition of a probability.
And so here I'm just going to replace expectation by the average.
That's my usual statistical trick.
And so my estimator Fn for-- the distribution is going to be 1 over n sum from i equal 1 to n of these indicators.
And this is called the empirical CDF.
It's just the data version of the CDF.
So I just replaced this expectation here by an average.
Now when I sum indicators, I'm actually counting the number of them that satisfy something.
So if you look at what this guy is, this is the number of X i's that is less than t, right?
And so if I divide by n, it's the proportion of observations I have that are less than t. That's what the empirical distribution is.
That's what's written here, the number of data points that are less than t. And so this is going to be something that's sort of trying to estimate one or the other.
And the law of large number actually tells me that for any given t, if n is large enough, Fn of t should be close to F of t. Because it's an average.
And this entire thing, this entire statistical trick, which consists of replacing expectations by averages, is justified by the law of large number.
Every time we used it, that was because the law of large number sort of guaranteed to us that the average was close to the expectation.
OK.
So law of large numbers tell me that Fn of t converges, so that's the strong law, says that almost surely actually Fn of t goes to F of t. And that's just for any given t. Is there any question about this?
That averages converge to expectation, that's the law of large number.
And almost surely we could say in probability it's the same, that would be the weak law of large number.
Now this is fine.
For any given t, the average converges to the true.
It just happens that this random variable is indexed by t, and I could do it for t equals 1 or 2 or 25, and just check it again.
But I might want to check it for all t's at once.
And that's actually a different result.
That's called a uniform result.
I want this to hold for all t at the same time.
And it may be the case that it works for each t individually but not for all t's at the same time.
What could happen is that for t equals 1 it converges at a certain rate, and for t equals 2 it converges at a bit of a slower rate, and for t equals 3 at a slower rate and slower rate.
And so as t goes to infinity, the rate is going to vanish and nothing is going to converge.
That could happen.
I could make this happen at a finite point.
There's many ways where it could make this happen.
Let's see how that could work.
I could say, well, actually no.
I still need to have this at infinity for some reason.
It turns out that this is still true uniformly, and this is actually a much more complicated result than the law of large number.
It's called Glivenko-Cantelli Theorem.
And the Glivenko-Cantelli Theorem tells me that, for all t's at once, Fn converges to F. So let me just show you quickly why this is just a little bit stronger than the one that we had.
If sup is confusing you, think of max.
It's just the max over an infinite set.
And so what we know is that Fn of t goes to F of t as n goes to infinity.
And that's almost surely.
And that's the law of large numbers.
Which is equivalent to saying that Fn of t minus F of t as n goes to infinity converges almost surely to 0, right?
This is the same thing.
Now I want this to happen for all t's at once.
So what I'm going to do-- oh, and this is actually equivalent to this.
And so what I'm going to do is I'm going to make it a little stronger.
So here the arrow only goes one way.
And this is where the sup for t in R of Fn of t. And you could actually show that this happens also almost surely.
Now maybe almost surely is a bit more difficult to get a grasp on.
Does anybody want to see, like why this statement for this sup is strictly stronger than the one that holds individually for all t's?
You want to see that?
OK, so let's do that.
So forget about it almost surely for one second.
Let's just do it in probability.
The fact that Fn of t converges to F of t for all t, in probability means that this goes to 0 as n goes to infinity for any epsilon.
For any epsilon in t we know we have this.
That's the convergence in probability.
Now what I want is to put a sup here.
The probability that the sup is lower than epsilon, might be actually always larger than, never go to 0 in some cases.
It could be the case that for each given t, I can make n large enough so that this probability becomes small.
But then maybe it's an n of t. So this here means that for any-- maybe I shouldn't put, let me put a delta here.
So for any epsilon, for any t and for any epsilon, there exists n, which could depend on both epsilon and t, such that the probability that Fn t minus F of t exceeding delta is less than epsilon t. There exists an n and a delta.
No, that's for all delta, sorry.
So this is true.
That's what this limit statement actually means.
But it could be the case that now when I take the sup over t, maybe that n of t is something that looks like t. Or maybe, well, integer part of t. It could be, right?
I don't say anything.
It's just an n that depends on t. So if this n is just t, maybe t over epsilon, because I want epsilon.
Something like this.
Well that means that if I want this to hold for all t's at once, I'm going to have to go for the n that works for all t's at once.
But there's no such n that works for all t's at once.
The only n that works is infinity.
And so I cannot make this happen for all of them.
What Glivenko-Cantelli tells you, it's actually this is not something that holds like this.
That the n that depends on t, there's actually one largest n that works for all the t's at once, and that's it.
OK.
So just so you know why this is actually a stronger statement, and that's basically how it works.
Any other question?
Yeah
So what's the position for this to have, because the random variable have a finite mean, finite variance?
No.
Well the random variable does have finite mean and finite variance, because the random variable is an indicator.
So it has everything you want.
This is one of the nicest random variables, this is a Bernoulli random variable.
So here when I say law of large number, that this holds.
Where did I write this?
I think I erased it.
Yeah, the one over there.
This is actually the law of large numbers for Bernoulli random variables.
They have everything you want.
They're bounded.
Yes
So I'm having trouble understanding the first statement.
So it says, for all epsilon and all t, the probability of that--
So you mean this one?
Yeah
For all epsilon and all t. So you fix them now.
Then the probability that, sorry, that was delta.
I changed this epsilon to delta at some point
And then what's the second line?
Oh, so then the second line says that, so I'm just rewriting in terms of epsilon delta what this n goes to infinity means.
So it means that for any a t and delta, so that's the same as this guy here, then here I'm just going back to rewriting this.
It says that for any epsilon there exists an n large enough such that, well, n larger than this thing basically, such that this thing is less than epsilon.
So Glivenko-Cantelli tells us that not only is this thing a good idea pointwise, but it's also a good idea uniformly.
And all it's saying is if you actually were happy with just this result, you should be even happier with that result.
And both of those results only tell you one thing.
They're just telling you that the empirical CDF is a good estimator of the CDF.
Now since those indicators are Bernoulli distributions, I can actually do even more.
So let me get this guy here.
OK so, those guys, Fn of t, this guy is a Bernoulli distribution.
What is the parameter of this Bernoulli distribution?
What is the probability that it takes value 1?
F of t
F of t, right?
It's just the probability that this thing happens, which is F of t. So in particular the variance of this guy is the variance of this Bernoulli.
So it's F of t 1 minus F of t. And I can use that in my Central Limit Theorem.
And Central Limit Theorem is just going to tell me that if I look at the average of random variables, I remove their mean, so I look at square root of n Fn of t, which I could really write as xn bar, right?
That's really just an xn bar.
Minus the expectation, which is F of t, that comes from this guy.
Now if I divide by square root of the variance, that's my square root p1 minus p. Then this guy, by the Central Limit Theorem, goes to some N 0, 1.
Which is the same thing as you see there, except that the variance was put on the other side.
OK. Do I have the same thing uniformly in t?
Can I write something that holds uniformly in t?
Well, if you think about it for one second it's unlikely it's going to go too well.
In the sense that it's unlikely that the supremum of those random variables over t is going to also be a Gaussian.
And the reason is that, well actually the reason is that this thing is actually a stochastic process indexed by t. A stochastic process is just a sequence in random variables that's indexed by, let's say time.
The one that's the most famous is Brownian motion, and it's basically a bunch of Gaussian increments.
So when you go from t to just t a little after that, you have add some Gaussian into the thing.
And here it's basically the same thing that's happening.
And you would sort of expect, since each of this guy is Gaussian, you would expect to see something that looks like a Brownian motion at the end.
But it's not exactly a Brownian motion, it's something that's called the Brownian bridge.
So if you've seen the Brownian motion, if I make it start at 0 for example, so this is the value of my Brownian motion.
Let's write it.
So this is one path, one realization of Brownian motion.
Let's call it w of t as t increases.
So let's say it starts at 0 and looks like something like this.
So that's what Brownian motion looks like.
It's just something that's pretty nasty.
I mean it looks pretty nasty, it's not continuous et cetera, but it's actually very benign in some average way.
So Brownian motion is just something, you should view this as if I sum some random variable that are Gaussian, and then I look at this from farther and farther, it's going to look like this.
And so here I cannot have a Brownian motion in the n, because what is the variance of Fn of t minus F of t at t is equal to 1?
Sorry, at t is equal to infinity
0
It's 0, right?
The variance goes from 0 at t is negative infinity, because at negative infinity F of t is going to 0.
And as t goes to plus infinity, F of t is going to 1, which means that the variance of this guy as t goes from negative infinity to plus infinity is pinned to be 0 on each side.
And so my Brownian motion cannot, when I describe a Brownian motion I'm just adding more and more entropy to the thing and it's going all over the place, but here what I want is that as I go back it should go back to essentially 0.
It should be pinned down to a specific value at the n. And that's actually called the Brownian bridge.
It's a Brownian motion that's conditioned to come back to where it started essentially.
Now you don't need to understand Brownian bridges to understand what I'm going to be telling you.
The only thing I want to communicate to you is that this guy here, when I say a Brownian bridge, I can go to any probabilist and they can tell you all the probability properties of this stochastic process.
It can tell me the probability that it takes any value at any point.
In particular, it can tell me-- the supremum between 0 and 1 of this guy, it could tell me what the cumulative distribution function of this thing is, can tell me what the density of this thing is, can tell me everything.
So it means that if I want to compute probabilities on this object here, which is the maximum value that this guy can take over a certain period of time, which is basically this random variable.
So if I look at the value here, it's a random variable that fluctuates.
It can tell me where it is with hyperability, can tell me the quantiles of this thing, which is useful because I can build a table and use it to compute my quantiles and form tests from it.
So that's what actually is quite nice.
It says that if I look at the square root of n Fn hat minus sup over t, I get something that looks like the sup of these Gaussians, but it's not really sup of Gaussian, it's sup of a Brownian motion.
Now there's something you should be very careful here.
I cheated a little bit.
I mean, I didn't cheat, I can do whatever I want.
But my notation might be a little confusing.
Everybody sees that this t here is not the same as this t here?
Can somebody see that?
Just because, first of all, this guy's between 0 and 1.
And this guy is in all of R. What is this t here?
As a function of this t here?
This guy is F of this guy.
So really, if I want it to be completely transparent and not save the keys of my keyboard, I would read this as sup over t of Fn t minus F of t goes to N distribution as n goes to infinity.
The supremum over t, again in R, so this guy is for t in the entire real line, this guy is for t in the entire real line.
But now I should write b of what?
F of t, exactly.
So really the t here is F of the original one.
And so that's a Brownian bridge, where when t goes to infinity the Brownian bridge goes from 0 to 1 and it looks like this.
A Brownian bridge at 0 is 0, at 1 it's 0.
And it does this.
But it doesn't stray too far because I condition it to come back to this point.
That's what a Brownian bridge is.
OK.
So in particular, I can find a distribution for this guy.
And I can use this to build a test which is called the Kolmogorov-Smirnov test.
The idea is the following.
It says, if I want to test some distribution F0, some distribution that has a particular CDF F0, and I plug it in under the null, then this guy should have pretty much the same distribution as the supremum of Brownian bridge.
And so if I see this to be much larger than it should be when it's the supremum of a Brownian bridge, I'm actually going to reject my hypothesis.
So here's the test.
I want to test whether H0, F is equal to F0, and you will see that most of the goodness of fit tests are formulated mathematically in terms of the cumulative distribution function.
I could formulate them in terms of personality density function, or just write x follows N 0, 1, but that's the way we write it.
We formulate them in terms of cumulative distribution function because that's what we have a handle on through the empirical cumulative distribution function.
And then it's versus H1, F is not equal to F0.
So now I have my empirical CDF.
And I hope that for all t's, Fn of t should be close to F0 of t. Let me write it like this.
I put it on the exponent because otherwise that would be the empirical distribution function based on zero observations.
Now I form the following test statistic.
So my test statistic is tn, which is the supremum over t in the real line of square root of n Fn of t minus F of t, sorry, F0 of t. So I can compute everything.
I know this from the data, and this is the one that comes from my null hypothesis.
As I can compute this thing.
And I know that if this is true, this should actually be the supremum of a Brownian bridge.
Pretty much.
And so the Kolmogorov-Smirnov test is simply, reject if this guy, tn, in absolute value, no actually not in absolute value.
This is just already absolute valued.
Then this guy should be what?
It should be larger than the q alpha over 2 distribution that I have.
But now rather than putting N 0, 1, or Tn, this is here whatever notation I have for supremum of Brownian bridge.
Just like I did for any pivotal distribution.
That was the same recipe every single time.
I formed the test statistic such that the asymptotic distribution did not depend on anything I know, and then I would just reject when this pivotal distribution was larger than something.
Yes?
I'm not really sure why Brownian bridge appears
Do you know what a Brownian bridge is, or?
Only vaguely
OK.
So this thing here, think of it as being a Gaussian.
So for all t you have a Gaussian distribution.
Now a Brownian motion, so if I had a Brownian motion I need to tell you what the-- so it's basically a Brownian motion is something that looks like this.
It's some random variable that's indexed by t. I want, say, the expectation of Xt could be equal to 0 for all t. And what I want is that the increments have a certain distribution.
So what I want is that the expectation of Xt minus Xs follows some distribution which is N 0, t minus s. So the increments are bigger as I go farther, in terms of variability.
And I also want some covariance structure between the two.
So what I want is that the covariance between Xs and Xt is actually equal to the minimum of s and t. Yeah, maybe.
Yeah, that should be there.
So this is, you open a probability book, that's what it's going to look like.
So in particular, you can see, if I put 0 here and X0 is equal to 0, it has 0 variance.
So in particular, it means that Xt, if I look only at the t-th one, it has some normal distribution with variance t. So this is something that just blows up.
So this guy here looks like it's going to be a Brownian motion because when I look at the left-hand side it has a normal distribution.
Now there's a bunch of other things you need to check.
It's the fact that you have this covariance, for example, which I did not tell you.
But it sure look somewhat like that.
And in particular, when I look at the normal with mean 0 and variance here, then it's clear that this guy does not have a variance that's going to go to infinity just like the variance of this guy.
We know that the variance is forced to be back to 0.
And so in particular we have something that has mean 0 always, whose variance has to be 0 at 0, and variance-- sorry, at t equals negative infinity, and variance 1 at t equals plus infinity.
So a variance 0 at t equals plus infinity, and so I have to basically force it to be equal to 0 at each n. So the Brownian motion here tends to just go to infinity somewhere, whereas this guy forces it to come back.
Now everything I described to you is on the scale negative infinity to plus infinity, but since everything depends on F of t, I can actually just put that back into a scale, which is 0 and 1 by a simple change of variable.
It's called change of time for the Brownian motion.
OK?
Yeah
So does a Brownian bridge have a variance at each point that's proportional?
Like it starts at 0 variance and then goes to 1/4 variance in the middle and then goes back to 0 variance?
Like in the same parabolic shape?
Yeah.
I mean, definitely.
I mean by symmetry you can probably infer all the things
Well I can imagine Brownian bridge with a variance that starts at 0 and stays, like, the shape of the variance as you move along
Yeah, so I don't know if-- there is an explicit formula for this, and it's simple.
That's what I can tell you, but I don't know what the explicit, off the top of my head what the explicit formula is
But would it have to match this F of t 1 minus F of t structure?
Or not?
Yeah
Or does the fact that we're taking the supremum--
No.
Well the Brownian bridge, this is the supremum-- you're right.
So this will be this form for the variance for sure, because this is only marginal distributions that don't take-- right, the process is not just what is the distribution at each instant t. It's also how do those distributions interact with each other in terms of covariance.
For the marginal distributions at each instance t, you're right, the variance is F of t 1 minus F of t. We're not going to escape that.
But then the covariance structure between those guys is a little more complicated.
But yes, you're right.
For marginal that's enough.
Yeah?
So the supremum of the Brownian bridge is a number between 0 and 10, let's just say
Yeah, it could be infinity
So it's not symmetrical with respect to 0, so why are we doing all over 2?
OK. Did say raise it?
Yeah.
Because here I didn't say the supremum of the absolute value of a Brownian bridge, I just said the supremum of a Brownian bridge.
But you're right, let's just do this like that.
And then it's probably cleaner.
So yeah, actually well it should be q alpha.
So this is basically, you're right.
So think of it as being one-sided.
And there's actually no symmetry for the supremum.
I mean the supremum is not symmetric around 0, so you're right.
I should not use alpha over 2, thank you.
Any other question?
This should be alpha.
Yeah.
I mean those slides were written with 1 minus alpha and I have not replaced all instances of 1 minus alpha by alpha.
I mean, except this guy, tilde.
Well, depends on how you want to call it.
But this is still, the probability that Z exceeds this guy should be alpha.
OK?
And this can be found in tables.
And we can compute the p-value just like we did before.
But we have to simulate it because it's not going to depend on the cumulative distribution function of a Gaussian, like it did for the usual Gaussian test.
That's something that's more complicated, and typically you don't even try.
You get the statistical software to do it for you.
So just let me skip a few lines.
This is what the table looks like for the Kolmogorov-Smirnov test.
So it just tells you, what is your number of observations, n. Then you want alpha to be equal to 5%, say.
Let's say you have nine observations.
So if square root of n absolute value of Fn of t minus F of t exceeds this thing, you reject.
Well it's pretty clear from this test is that it looks very nice, and I tell you this is how you build it.
But if you think about it for one second, it's actually really an annoying thing to build because you have to take the supremum over t. This depends on computing a supremum, which in practice might be super cumbersome.
I don't want to have to compute this for all values t and then to take the maximum of those guys.
It turns out that that's actually quite nice that we don't have to actually do this.
What does the empirical distribution function look like?
Well, this thing, remember Fn of t by definition was-- so let me go to the slide that's relevant.
So Fn of t looks like this.
So what it means is that when t is between two observations, then this guy is actually keeping the same value.
So if I put my observations on the real line here.
So let's say I have one observation here, one observation here, one observation here, one observation here, and one observation here, for simplicity.
Then this guy is basically, up to this normalization, counting how many observations they have that are less than t. So since I normalize by n, I know that the smallest number here is going to be 0, and the largest number here is going to be 1.
So let's say this looks like this.
This is the value 1.
At the value, since I take it less than or equal to, when I'm at Xi, I'm actually counting it.
So the jump happens at Xi.
So that's the first observation, and then I jump.
By how much do I jump?
Yeah?
One over n, right?
And then this value belongs to the right.
And then I do it again.
I know it's not going to work out for me, but we'll see.
Oh no actually, I did pretty well.
This is what my cumulative distribution looks like.
Now if you look on this slide, there is this weird notation where I start putting now my indices in parentheses.
X parenthesis 1, X parenthesis 2, et cetera.
Those are called the ordered statistic.
It's just because it might be, when my data is given to me I just call the first observation, the one that's on top of the table, but it doesn't have to be the smallest value.
So it might be that this is X1 and that this is X2, and then this is X3, X4, and X5.
These might be my observations.
So what I do is that I call them in such a way that this is actually, I recall this guy X1, which is just really X3.
This is X2, X3, X4, and X5.
These are my reordered observations in such a way that the smallest one is indexed by one and the largest one is indexed by n. So now this is actually quite nice, because what I'm trying to do is to find the largest deviation from this guy to the true cumulative distribution function.
The true cumulative distribution function, let's say it's Gaussian, looks like this.
It's something continuous, for a symmetric distribution it crosses this axis at 1/2, and that's what it looks like.
And the Kolmogorov-Smirnov test is just telling me how far do those two curves get in the worst possible case?
So in particular here, where are they the farthest?
Clearly that's this point.
And so up to rescaling, this is the value I'm going to be interested in.
That's how they get as far as possible from each other.
Here, something just happened, right?
The farthest distance that I got was exactly at one of those dots.
It turns out this is enough to look at those dots.
And the reason is, well because after this dot and until the next jump, this guy does not change, but this guy increases.
And so the only point where they can be the farthest apart is either to the left of a jump or to the right of a jump.
That's the only place where they can be far from each other.
And that means that only one observation.
Everybody sees that?
The farthest points, the points at which those two curves are the farthest from each other, has to be at one of the observations.
And so rather than looking at a sup over all possible t's, really all I need to do is to look at a maximum only at my observations.
I just need to check at each of those points whether they're far.
Now here, notice that you did not, this is not written Fn of Xi.
The reason is because I actually know what Fn of Xi is.
Fn of the i-th order observation is just the number of jumps I've had until this observation.
So here, I know that the value of Fn is 1 over n, here it's 2 over n, 3 over n, 4 over n, 5 over n. So I knew that the values of Fn at my observations, and those are actually the only values that Fn can take, are an integer divided by n. And that's why you see i minus 1 over n, or i over n. This is the difference just before the jump, and this is the difference at the jump.
So here the key message is that this is no longer a supremum over all t's, but it's just the maximum from 1 to n. So I really have only two n values to compute.
This value and this value for each observation, that's 2n total.
I look at the maximum and that's actually the value.
And it's actually equal to tn.
It's not an approximation.
Those things are equal.
That's just the only places where those guys can be maximum.
Yes?
It seems like since the null hypothesis [INAUDIBLE] the entire distribution of theta, this is like strictly more powerful than just doing it [INAUDIBLE]
It's strictly less powerful
Strictly less powerful.
But is there, is that like a big trade-off that we're making when we do that?
Obviously we're not certain in the first place that we want to assume normality.
Does it make sense to [INAUDIBLE],, the Gaussian [INAUDIBLE]
So can you, I'm not sure what question you're asking
So when we're doing a normal test, we're just asking questions about the mus, the means of our distribution.
[INAUDIBLE] This one, it seems like it would be both at the same time.
[INAUDIBLE] Is this decreasing power [INAUDIBLE]??
So remember, here in this test we want to conclude to H0, in the other test we typically want to conclude to H1.
So here we actually don't want power, in a way.
And you have to also assume that doing a test on the mean is probably not the only thing you're going to end up doing on your data after you actually establish that it's normally distributed.
Then you have the dataset, you've sort of established it's normally distributed, and then you can just run the arsenal of statistical studies.
And we're going to see regression and all sorts of predictive things, which are not just tests if the mean is equal to something.
Maybe you want to build a confidence interval for the mean.
Then this is not, confidence interval is not a test.
So you're going to have to first test if it's normal, and then see if you can actually use the quantiles of a Gaussian distribution or a t distribution to build this confidence interval.
So in a way you should do this as like, the flat fee to enter the Gaussian world, and then you can do whatever you want to do in the Gaussian world.
We'll see actually that your question goes back to something that's a little important, is here I said F0 is fully specified.
It's like an N 1, 5.
But I didn't say, is it normally distributed, which is the question that everybody asks.
You're not asking, is it this particular normal distribution with this particular mean and this particular variance.
So how would you do it in practice?
Well you would say, I'm just going to replace the mean by the empirical mean and the variance by the empirical variance.
But by doing that you're making a huge mistake because you are sort of depriving your test of the possibility to reject the Gaussian hypothesis just based on the fact that the mean is wrong or the variance is wrong.
You've already stuck to your data pretty well.
And so you're sort of like already tilting the game in favor of H0 big time.
So there's actually a way to arrange for this.
OK, so this is about pivotal statistic.
We've used this word many times.
And So that's how.
I'm not going to go into this test.
It's really, this is a recipe on how you would actually build the table that I showed you, this table.
This is basically the recipe on how to build it.
There's another recipe to build it, which is just open a book at this page.
That's a little faster.
Or use software.
I just wanted to show you.
So let's just keep in mind, anybody has a good memory?
Let's just keep in mind this number.
This is the threshold for the Kolmogorov-Smirnov statistic.
If I have 10 observations and I want to do it at 5%, it's about 41%.
So that's the number that it should be larger from.
So it turns out that if you want to test if it's normal, and not just the specific normal, this number is going to be different.
Do you think the number I'm going to read in a table that's appropriate for this is going to be larger or smaller?
Who says larger?
Sorry, what was the question?
So the question is, this is the number I should see if my test was, is X, say, N 0, 5.
Right?
That's a specific distribution with a specific F0.
So that's the number, I would build the Kolmogorov-Smirnov statistic from this.
I would perform a test and check if my Kolmogorov-Smirnov statistic tn is larger than this number or not.
If it's larger I'm going to reject.
Now I say, actually, I don't want to test if H0 is N 0, 5, but it's just a mu sigma squared for some mu and sigma squared.
And in particular I'm just going to plugin mu hat and sigma hat into my F0, run the same statistic, but compare it to a different number.
So the larger the number, the more or less likely am I to reject?
The less likely I am to reject, right?
So if I just use that number, let's say this is a large number, I would be more tempted to say it's Gaussian.
And if you look at the table you would get that if you make the appropriate correction at the same number of observations, 10, and the same level, you get 25% as opposed to 41%.
That means that you're actually much more likely if you use the appropriate test to reject the fact that it's normal, which is bad news, because that means you don't have access to the Gaussian arsenal, and nobody wants to do this.
So actually this is a mistake that people do a lot.
They use the Kolmogorov-Smirnov test to test for normality without adjusting for the fact that they've plugged in the estimated mean and the estimated variance.
This leads to rejecting less often, right?
I mean this is almost half of the number that we had.
And then they can be happy and walk home and say, well, I did the test and it was normal.
So this is actually a mistake that I believe that genuinely at least a quarter of the people do make in purpose.
They just say, well I want it to be Gaussian so I'm just going to make my life easier.
So this is the so-called Kolmogorov Lilliefors test.
We'll talk about it, well not today for sure.
There's other statistics that you can test, that you can use.
And the idea is to say, well, we want to know if the empirical distribution function, the empirical CDF, is close to the true CDF.
The way we did it is by forming the difference in looking at the worst possible distance they can be.
That's called a sup norm, or L infinity norm, in functional analysis.
So here, this is what it looked like.
The distance between Fn and F that we measured was just the supremum distance over all t's.
That's one way to measure distance between two functions.
But there's an infinite many ways to measure distance between functions.
One is something we're much more familiar with, which is the squared L2-norm.
This is nice because this has like an inner product, it has some nice properties.
And you could actually just, rather than taking the sup, you could just integrate the squared distance.
And this is what leads to Cramier-Von Mises test.
And then there's another one that says, well, maybe I don't want to integrate without weights.
Maybe I want to put weights that account for the variance.
And this guy is called Anderson-Darling.
For each of these tests you can check that the asymptotic distribution is going to be pivotal, which means that there will be a table at the back of some book that tells you what the statistic, the quantiles of square root of n times this guy are asymptotically, basically.
Yeah?
For the Kolmogorov-Smirnov test, for the table that shows the value it has, it has the value for different n. But I thought we [INAUDIBLE]--
Yeah.
So that's just to show you that asymptotically it's pivotal, and I can point you to one specific thing.
But it turns out that this thing is actually pivotal for each n. And that's why you have this recipe to construct the entire thing, because it's actually not true for all possible n's.
Also there's the n that shows up here.
So no actually, this is something you should have in mind.
So basically, let me strike what I just said.
This thing you can actually, this distribution will not depend on F0 for any particular n. It's just not going to be a Brownian bridge but a finite sample approximation of a Brownian bridge, and you can simulate that just drawing samples from it, building a histogram, and constructing the quantiles for this guy
No one has actually developed a table for Brownian--
Oh, there is one.
That's the table, maybe.
Let's see if we see it at the bottom of the other table.
Yeah.
See?
Over 40, over 30.
So this is not the Kolmogorov-Smirnov, but that's the Kolmogorov Lilliefors.
Those numbers that you see here, they are the numbers for the asymptotic thing which is some sort of Brownian bridge.
Yeah?
Two questions.
If I want to build the Kolmogorov-Smirnov test, it says that F0 is required to be continuous
Yeah
[INAUDIBLE] If we have, like, probability mass of a particular value.
Like some sort of data
So then you won't have this nice picture, right?
This can happen at any point because you're going to have discontinuities in F and those things can happen everywhere.
And then--
Would the supremum still work?
You mean the Brownian bridge?
Yeah.
The Kolmogorov test doesn't say that you have to be able to easily calculate the supremum
No, no, no, but you still need it.
You still need it for-- so there's some finite sample versions of it that you can use that are slightly more conservative, which is in a way good news because you're going to conclude more to H0.
And there's are some, I forget the name, it's Kiefer-Wolfowitz, the Kiefer-Dvoretzky-Wolfowitz, an equality which is basically like Hoeffding's inequality.
So it's basically up to bad constants telling you the same result as the Brownian bridge result, and those are true all the time.
But for the exact asymptotic distribution, you need continuity.
Yes
So just a clarification.
So when we are testing the Kolmogorov, we shouldn't test a particular mu and sigma squared?
Well if you know what they are you can use Kolmogorov-Smirnov, but if you don't know what they are you're going to plug in-- as soon as you're going to estimate the mean and the variance from the data, you should use the one we'll see next time, which is called Kolmogorov Lilliefors.
You don't have to think about it too much.
We'll talk about it on Thursday.
Any other question?
So we're out of time.
So I think we should stop here, and we'll resume on Thursday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're talking about goodness-of-fit tests.
Goodness-of-fit tests are, does my data come from a particular distribution?
And why would we want to know this?
Well, maybe we're interested in, for example, knowing if the zodiac signs of the Fortune 500 CEOs are uniformly distributed.
Or maybe we actually have slightly more-- slightly deeper endeavors, such as understanding if you can actually apply the t-test by testing normality of your sample.
All right?
So we saw that there's the main result-- the main standard test for this.
It's called the Kolmogorov-Smirnov test that people use quite a bit.
It's probably one of the most used tests out there.
And there's other versions of it that I mentioned passing by.
There's the Cramer-von Mises, and there's the Anderson-Darling test.
Now, how would you pick one of such tests?
Well, they're always are going to-- they're always going to have their advantages and disadvantages.
And Kolmogorov-Smirnov is definitely the most widely used because-- well, I guess because it's a natural notion of distance between functions.
You just look for each point how far they can be, and you just look at the farthest they can be everywhere.
Now, Cramer-von Mises involves L2 distance.
So if you're not used to Hilbert spaces or notions of Euclidean spaces, at least it's a little more complicated.
And then Anderson-Darling is definitely even more complicated.
Now, each of these tests is going to be more powerful against other alternatives.
So unless you can really guess which alternative you're expecting to see, which you probably don't, because, again, you're in a case where you want to typically declare H0 to be the correct one, then it's really a matter of tossing a coin.
Maybe you can run all three of them and just sleep better at night, because all three of them have failed to reject, for example.
All right?
So as I mentioned, one of the maybe primary goals to test goodness of fit is to be able to check whether we can apply Student's test, right, and if the Student distribution is actually a valid distribution.
And for that, we need to have normally distributed data.
Now, as I said several times, normally distributed, it's not a specific distribution.
It's a family of distributions that's indexed by means and variances.
And the way I would want to test if a distribution is normally distributed is, well, I would just look at the most natural normal distribution or Gaussian distribution that my data could follow.
That means that's the Gaussian distribution that has the same mean as my data and the same empirical variance as my data, right?
And so I'm going to be given some points x1, xn, and I'm going to be asking, are those Gaussian?
That means this is equivalent to, say, are they N mu sigma square for some mu sigma squared?
And of course, the natural choice is to take mu hat to be-- mu to be equal to mu hat, which is xn bar.
And sigma squared to be sigma squared hat to be, well, Sn hat-- Sn-- what we wrote Sn, which is 1/n sum from i equal 1 to n of xi minus xn bar squared.
OK?
So this is definitely the natural one you would want to test.
And maybe you could actually just close your eyes and just stuff that in a Kolmogorov-Smirnov test.
OK?
So here, there's a few things that don't work.
The first one is that Donsker's theorem does not work anymore, right?
Donsker's theorem was the one that told us that, properly normalized, this thing would actually converge to the supremum of a Brownian bridge, which is not true.
So that's one problem.
But there's actually an even bigger problem is that this distribution, we will check in a second, actually does not-- is pivotal itself, right, the statistic is pivotal.
It does not have a distribution that depends on the known parameters, which is sort of nice, at least under the null.
However, the distribution is not the same as the one that had fixed mu and sigma.
The fact that they come from some random variables is actually distorting the distribution itself.
And in particular, the quantiles are going to be distorted, and we hinted at that last time.
So one other thing I need to tell you, though, is that this thing actually-- so I know there's some-- oh, yeah, that's where there's a word missing.
So we compute the quantiles for this test statistic.
And so what I need to promise to you is that these quantiles do not depend on any unknown parameter, right?
I mean, it's not clear, right?
So I want to test whether my data has some Gaussian distribution.
So under the null, all I know is that my xi's are Gaussian with some mean mu and some variance sigma, which I don't know.
So it could be the case that when I try to understand the distribution of this quantity under the null, it depends on mu and sigma, which I don't know.
So we need to check that this is the case.
And what's actually our redemption here is actually going to be the supremum.
The supremum is going to basically allow us to, say, sup out mu and sigma square.
So let's check that, right?
So what I'm interested in is this quantity, supremum over t and R of the difference between Fn of t and, what I write, phi mu hat sigma squared of t. So phi mu hat sigma hat squared-- sorry, sigma hat squared-- is the CDF of some Gaussian with mean mu hat and variance sigma hat squared.
And so in particular, this thing here, phi hat of mu hat-- sorry, phi hat of mu hat sigma hat squared of t is the probability that some x is less than t, where x follows some N mu hat sigma hat squared.
So what it means is that by just the translation and scaling trig that we typically do for Gaussian to turn it into some standard Gaussian, that implies that there exists some z, which is standard Gaussian this time, so mean 0 and variance 1, such that x is equal to sigma hat x-- sorry, z plus mu hat.
Agreed?
That's basically saying that x has some Gaussian with mean mu and variance sigma squared.
And I'm not going to say the hats every single time, OK?
So OK, so that's what it means.
So in particular, maybe I shouldn't use x here, because x is going to be my actual data.
So let me write y. OK?
So now what is this guy here?
It's basically-- so phi hat.
So this implies that phi mu hat sigma hat squared of t is equal to the probability that sigma hat z plus mu hat is less than t, which is equal to the probability that z is less than t minus mu hat divided by sigma hat, right?
But now when z is the standard normal, this is really just the cumulative distribution function of a standard Gaussian but evaluated at a point which is not t, but t minus mu hat divided by sigma hat.
All right?
So in particular, what I know-- so from this what I get-- well, maybe I'll remove that, it's going to be annoying-- I know that phi mu hat sigma hat squared-- sorry-- phi mu hat sigma hat squared of t is simply phi of, say, 0, 1.
And that's just the notation.
Usually we don't put those, but here it's more convenient.
So it's phi 0, 1 of t minus mu hat divided by sigma hat.
OK?
That's just something you can quickly check.
There's this nice way of writing the cumulative distribution function for any mean and any variance in terms of the cumulative distribution function with mean 0 and variance 1.
All right?
Not too complicated.
All right.
So I know what I'm going to say is that, OK, I have this sup here.
So what I can write is that this thing here is equal to the sup routine R of 1/n.
Let me write what Fn is-- sum from i equal 1 to n of the indicator that xi is less than t minus phi 0, 1 of t minus mu hat divided by sigma hat.
OK?
I actually want to make a change of variable so that this thing I'm going to call mu-- u, sorry.
OK?
And so I'm going to make my life easier, and I'm going to make it appear here.
And so I'm just going to replace this by indicator that xi minus mu hat divided by sigma hat less than t minus mu hat divided by sigma hat, which is sort of useless at this point.
I'm just making my formula more complicated.
But now I see something here that shows up, and I will call it u, and this is another u. OK?
So now what it means is that suping over t, when t ranges from negative infinity to plus infinity, the new range is from negative infinity to plus infinity, right?
So this sup, I can actually write-- this suping t I can write as the sup in u, as the indicator that xi minus mu hat divided by sigma hat is less than u minus phi 0, 1 of u.
Now, let's pause for one second.
Let's see where we're going.
What we're trying to show that this thing does not depend on the unknown parameters, say, mu and sigma, which are the mean and the variance of x under the null.
To do that, we basically need to make only quantities that are sort of invariant under these values.
So I tried to make this thing invariant under anything, and it's just really something that depends on nothing.
It's the CDF.
It doesn't depend on sigma hat and mu hat anymore.
But sigma hat and mu hat will depend on mu and sigma, right?
I mean, they're actually good estimators of those guys, so they should be pretty close to them.
And so I need to make sure that I'm not actually doing anything wrong here.
So the key thing here is going to be to observe that 1/n sum from i equal 1 to n of indicator of xi minus u hat divided by sigma hat less than u, which is the first term that I have in this absolute value, well, this is what-- well, this is equal to 1/n sum from i equal 1 to n of indicator that-- well, now under the null, which is that x follows N mu sigma squared, for some mu and sigma squared that are unknown.
But they are here.
They exist.
I just don't know what they are.
Then xi minus mu can be written as sigma zi plus mu minus mu hat divided by sigma hat, where z is equal to x minus mu divided by sigma, right?
That's just the same trick that I wrote here.
OK?
Everybody agree?
So I just standardize-- sorry, z-- yeah, so zi is xi minus mu i minus mu divided by sigma.
All right?
Just a standardization.
So now once I write this, I can actually divide everybody by sigma.
Right?
So I just divided on top here and in the bottom here.
So now what I need to check is that the distribution of this guy does not depend on mu or sigma.
That's what I claim.
What is the distribution of this indicator?
It's a Bernoulli, right?
And so if I want to understand its distribution, all I need to do is to compute its expectation, which is just the probability that this thing happens.
But the probability that this thing happens is actually now depending on mu and sigma.
And the reason is that mu is what?
Well, it's x bar-- sorry, yeah, so mu hat-- sorry, is xn bar.
So mu hat minus mu, which under the null follows N mu sigma square over n, right?
That's the property of the average.
So when I do mu hat minus mu divided by sigma, this thing is what distribution?
It's still a normal.
It's a linear transformation of a normal.
What are the parameters?
0, 1/n
Yeah, 0, 1/n.
But this does not depend on mu or sigma, right?
Now, I need to check that this guy does not depend on mu or sigma.
What is the distribution of sigma hat over sigma?
It's a chi-square, right?
Yeah, it is a chi-square.
So this is actually-- sorry, sigma hat squared divided by sigma squared is a chi-square with n minus 1 degrees of freedom.
Does not depend on mu or sigma
[INAUDIBLE]
[INAUDIBLE]
Or sigma hat squared over sigma squared?
Yeah, thank you.
So this is actually divided by it.
So maybe this guy.
Let's write it like that.
This is the proper way of writing it.
Thank you.
Right?
So now I have those two things.
Neither of them depends on mu or sigma.
I these two things.
There's just one more thing to check.
What is it?
That they're independent?
That they're independent, right?
Because the dependence in mu and sigma could be hidden in the covariance.
It could be the case that the marginal distribution of mu does not depend on mu or sigma, that the marginal distribution of sigma-- of mu hat does not depend on mu and sigma.
The marginal distribution of sigma hat does not depend on mu or sigma, but their correlation could depend on mu and sigma.
But we also have that if I look at-- so if I look at-- so since mu hat is independent of sigma hat, it means that the joint distribution of mu hat divided by sigma and sigma hat divided by sigma does not depend on blah, blah, blah, on mu and sigma.
OK?
Agree?
It's not in the individual ones, and it's not in the way they interact with each other.
It's nowhere
[INAUDIBLE] independence be [INAUDIBLE] theorem?
Yeah, covariance theorem, right.
So that's something we've been using over and again.
That's all under the null.
If my data is not Gaussian, nothing actually holds.
I just use the fact that under the null I'm Gaussian for some mean mu and variance sigma squared.
But that's all I care about.
When I'm designing a test, I only care about the distribution under the null, at least to control the type I error.
Then to control the type II error, then I cross my fingers pretty hard.
OK?
So now this basically implies what's written on the board, that this distribution, this test statistic, does not depend on any unknown parameters.
It's just something that's pivotal.
In particular, I could go at the back of a book and check if there's a table for the quantiles of these things, and indeed there are.
This is the table that you see.
So actually, this is not even in a book.
This is in Lilliefors original paper, 1967, as you can tell from the typewriting.
And he actually probably was rolling some dice from his office back in the day and was checking that this was-- he simulated it, and this is how he computed those numbers.
And here you also have some limiting distribution, which is not the sup of a Brownian motion over 0, 1 of-- sorry, of a Brownian bridge over 0, 1, which is the one that you would see for the Kolmogorov-Smirnov test, but it's something that's slightly different.
And as I said, these numbers are actually typically much smaller than the numbers you would get, right?
Remember, we got something that was about 0.5, I think, or maybe 0.41, for the Kolmogorov-Smirnov test at the same entrance, which means that using Kolmogorov-Lilliefors test it's going to be harder for you not to reject for the same data.
It might be the case that in one case you reject, and in the other one you fail to reject.
But the ordering is always that if you fail to reject with Kolmogorov-Lilliefors, you will fail to reject with Kolmogorov-Smirnov, right?
There's always one.
So that's why people tend to close their eyes and prefer Kolmogorov-Smirnov because it just makes their life easier.
OK?
So this is called Kolmogorov-Lilliefors.
I think there's actually an E here-- sorry, an I before the E. Doesn't matter too much.
OK?
Are there any questions?
Yes?
Is there like a place you can point to like [INAUDIBLE]
Yeah
[INAUDIBLE]
So the fact that it's actually a different distribution is that here-- so if I actually knew what mu and sigma were, I would do exactly the same thing.
But here, rather than having this average with mu and sigma, I would just have the-- with mu hat and sigma hat, I would just have the average with mu and sigma.
OK?
So what it means is that the key thing is that what I would compare is the 1/n sum of some Bernoullis with parameter.
And the parameter here would be the probability that mu-- xi minus mu over sigma is less than u, which is just the probability that phi-- sorry, it's a Bernoulli with probability F of t. Well, let me write what it is, right?
So that's minus phi 0, 1 of t. OK?
So that's for the K-S test, and then I sup over t, right?
That's what I would have had, because this is actually exactly the right thing.
Here I would remove the true mean.
I would divide by the true standard deviation.
So that would actually end up being a standard Gaussian, and that's why I'm allowed to use phi 0, 1 here.
Agreed?
And these are Bernoullis because they're just indicators.
What happens in the Kolmogorov-Lilliefors test?
Well, here the Bernoulli, the only thing that's going to change is this guy, right?
They still have a Bernoulli.
It's just that the parameters of the Bernoulli are weird.
The parameters of the Bernoulli looks like it's-- it becomes the probability that some N(0, 1) plus some N(0, 1/n), right, divided by some square root of chi-squared n minus 1 divided by n is less than t. And those things are independent, but those guys are not necessarily independent, right?
And so why is this probability changing?
Well, because this denominator is actually fluctuating a lot.
So that actually makes this probability different.
And so that's basically where it comes from, right?
So you could probably convince yourself very quickly that this only makes those guys closer.
And why does it make those guys closer?
No, sorry.
It makes those guys farther, right?
And it makes those guys farther for a very clear reason, is that the expectation of this Bernoulli is exactly that guy.
Here I think it's going to be true as well that the expectation of this Bernoulli is going to be that guy, but the fluctuations are going to be much bigger than just the phi of the Bernoulli.
Because the first thing I do is I have a random parameter from my Bernoulli, and then I flip the Bernoulli.
So fluctuations are going to be bigger than a Bernoulli.
And so when I take the sup, I'm going to have to [INAUDIBLE] them.
So it makes things farther apart, which makes it more likely for you to reject.
Yeah?
You also said that if you compare the same-- if you compare the table and you set at the same level, the Lilliefors is like 0.2, and for the Smirnov is at 0.4
Yeah
OK.
So it means that Lilliefors is harder not to reject?
It means that Lilliefors is harder not to reject, yes, because we reject when we're larger than the number.
So the number being smaller with the same data, we might be, right?
So basically, it looks like this.
What we run-- so here we have the distribution for the-- so let's say this is the density for K-S. And then we have the density for Kolmogorov-Lilliefors, K-L. OK?
And what the density of K-L looks like, it looks like this, right?
And so if I want to squeeze in alpha here, I'm going to have to squeeze in-- and I squeeze in alpha here, then this is the quantile of order 1 minus alp-- well, let's say alpha of the K-L. And this is the quantile alpha of K-S.
So now you give me data, and what I do with it, I check whether they're larger than this number.
So if I apply K-S, I check whether I'm larger or smaller than this thing.
But if I apply Kolmogorov-Lilliefors, I check whether I'm larger or smaller than this thing.
So over this entire range of values for my test statistic-- because it is the same test statistic, I just plugged in mu hat and sigma hat-- for this entire range, the two tests have different outcomes.
And this is a big range in practice, right?
I mean, it's between-- I mean, it's pretty much at scale here.
OK?
Any other-- yeah?
[INAUDIBLE] when n goes to infinity, the two tests become the same now, right?
Hmmm
Looking at that formula--
Yeah, They should become the same very far.
Let me see, though, because-- right.
So here we have 8-- so here we have, say, for 0.5, we get 0.886.
And for-- oh, I don't have it.
Yeah, actually, sorry.
So you're right.
You're totally right.
This is the Brownian bridge values.
Because in the limit by, say, Slutsky-- sorry, I'm lost.
Yeah, these are the values that you get for the Brownian bridge.
Because in the limit by Slutsky, this thing is going to have no fluctuation, and this thing is going to have no fluctuation.
So they're just going to be pinned down, and it's going to look like as if I did not replace anything.
Because in the limit, I know those guys much faster-- the mu hat and sigma hat converge much faster to mu and sigma than the distribution itself, right?
So those are actually going to be negligible.
You're right.
Actually even, I didn't have-- these are actually the numbers I showed you for the bridge, the Brownian bridge, last time, because I didn't have it for the Kolmogorov-Smirnov one.
OK?
So there's actually-- so those are numerical ways of checking things, right?
I give you data.
You just crank the Kolmogorov-Smirnov test.
Usually you press a 5 on MATLAB.
But let's say you actually compute this entire thing, and there's a number that comes out, and you decide whether it's large enough or small enough.
Of course, statistical software is going to make your life even simpler by spitting out a p-value, because you can-- I mean, if you can compute quantiles, you can also when compute p-values.
And so your life is just fairly easy.
You just have red is bad, green is good, and then you can go.
The problem is that those are numbers you want to rely on.
But let's say you actually reject.
Let's say you reject.
Your p-value is actually just like slightly below 5%.
So you can say, well, maybe I'm just going to change my p-value-- my threshold to 1%, but you might want to see what's happening.
And for that you need a visual diagnostic.
Like, how do I check if something departs from being normal, for example?
How do I check if a distribution-- why is a distribution not a uniform distribution?
Why is a distribution not an exponential distribution?
There's many, many, right?
If I have an exponential distribution and half of my values are negative, for example, well, there's like pretty obvious reasons why it should not be exponential.
But it could be the case that it's just the tails are little heavier or there's more concentration at some point.
Maybe it has two modes.
There's things like this.
But the real thing, we don't believe that the Gaussian is so important because it looks like this close to 0.
What we like about the Gaussian is that the tails here decay at this rate-- exponential minus x squared over 2 that we described in the maybe first lecture.
And in particular, if there were like kinks around here, it wouldn't matter too much.
This is not what makes issues for the Gaussian.
And so what we want is to have a visual diagnostic that tells us if the tails of my distribution are comparable to the tails of a Gaussian one, for example.
And those are what's called quantile-quantile plots, and in particular-- or QQ plots.
And the basic QQ plots we're going to be using are the ones that are called normal QQ plots that are comparing your data to a Gaussian distribution, or a normal distribution.
But in general, you could be comparing your data to any distribution you want.
And the way you do this is by comparing the quantiles of your data, the empirical quantiles, to the quantiles of the actual distribution you're trying to compare yourself to.
So this, in a way, is a visual way of performing these goodness-of-fit tests.
And what's nice about visual is that there's room for debate.
You can see something that somebody else cannot see, and you can always-- because you want to say that things are Gaussian.
And we'll see some examples where you can actually say it if you are good at debate, but it's actually going to be clearly not true.
All right.
So this is a quick and easy check.
That's something I do all the time.
You give me data, I'm just going to run this.
One of the first things I do so I can check if I can start entering the Gaussian world without compromising myself too much.
And the idea is to say, well, if F is close to-- if F-- if my data comes from an F, and if I know that Fn is close to F, then rather than computing some norm, some number that tells me how far they are, summarizing how far they are, I could actually plot the two functions and see if they're far apart.
So let's think for one second what this kind of a plot would look like.
Well, I would go between 0 and 1.
That's where everything would happen.
Let's say my distribution is the Gaussian distribution.
So this is the CDF of N(0, 1).
And now I have this guy that shows up, and remember we had this piecewise constant.
Well, OK, let's say we get something like this.
We get a piecewise constant distribution for Fn, right?
Just from this, and even despite my bad skills at drawing, it's clear that it's going to be hard for you to distinguish those two things, even for a fairly large amount of points.
Because the problem is going to happen here, and those guys look pretty much the same everywhere you are here.
You're going to see differences maybe in the middle, but we don't care too much about those differences.
And so what's going to happen is that you're going to want to compare those two things.
And this is basically you have the information you want, but visually it just doesn't render very well because you're not scaling things properly.
And the way we actually do it is by flipping things around.
And rather than comparing the plot of F to the plot of Fn, we compare the plot of Fn inverse to the plot of F inverse.
Now, if F goes from the real line to the interval 0, 1, F inverse goes from 0, 1 to the whole real line.
So what's going to happen is that I'm going to compare things on some intervals, which is the-- which are the entire real line.
And then what values should I be looking at those things at?
Well, technically for F, if F is continuous I could look at F inverse for any value that I please, right?
So I have F. And if I want to look at F inverse, I pick a point here and I look at the value that it gives me, and that's F inverse of, say, u, right, if this is u.
And I could pick any value I want, I'm going to be able to find it.
The problem is that when I start to have this piecewise constant thing, I need to decide what value I assign for anything that's in between two jumps, right?
And so I can choose whatever I want, but in practice it's just going to be things that I myself decide.
Maybe I can decide that this is the value.
Maybe I can decide that the value is here.
But for all these guys, I'm going to pretty much decide always the same value, right?
If I'm in between-- for this value u, for this jump the jump is here.
So for this value, I'm going to be able to decide whether I want to go above or below, but it's always this value that's going to come out.
So rather than picking values that are in between, I might as well just pick only values for which this is the value that it's going to get.
And those values are exactly 1/n, 2/n, 3/n, 4/n.
It's all the way to n/n, right?
That's exactly where the flat parts are.
We know we jump from 1/n every time.
And so that's exactly the recipe.
It says look at those values, 1/n, 2/n, 3/n until, say, n minus 1 over n. And for those values, compute the inverse of both the empiricial CDF and the true CDF.
Now, for the empirical CDF, it's actually easy.
I just told you this is basically where the points-- where the jumps occur.
And the jumps occur where?
Well, exactly at my observations.
Now, remember I need to sort those observations to talk about them.
So the one that occurs for the i-th jump is the i-th largest observation, which we denoted by X sub (i).
Remember?
We had this formula that we said, well, we have x1, xn.
These are my data.
And what I'm going to sort them into is x sub (1), which is less than or equal to x sub (2), which is less than x sub (n).
OK?
So we just ordered them from smallest to largest.
And then now we've done that, we just put this parenthesis notation.
So in particular, Fn inverse of i/n is the location where the i-th jumps occur, which is the i-th largest observation.
OK?
So for this guy, these values, the y-axes are actually fairly easy.
I know it's basically my ordered observations.
The x-values are-- well, that depends on the function F I'm trying to test.
If it's the Gaussian, it's just the quantile of order 1 minus 1/n, right?
It's this Q1 minus 1/n here that I need to compute.
It's the inverse of the cumulative distribution function, which, given the formula for F, you can actually compute or maybe estimate fairly well.
But it's something that you can find in tables.
Those are basically quantiles.
Inverse of CDFs are quantiles, right?
And so that's basically the things we're interested in.
That's why it's called quantile-quantile.
Those are sometimes referred to as theoretical quantiles, the one we're trying to test, and empirical quantiles, the one that corresponds to the empirical CDF.
And so I'm plotting a plot where the x-axis is quantile.
The y-axis is quantile.
And so I call this plot a quantile-quantile plot, or QQ plot, because, well, just say 10 times quantile-quantile, and then you'll see why.
Yeah?
[INAUDIBLE] have to have the [INAUDIBLE]??
Well, that's just-- we're back to the-- we're back to the goodness-of-fit test, right?
So if you look-- so you don't do it yourself.
That's the simple answer.
You don't-- I'm just telling you how those plots are going to be seen spit out from a software are going to look like.
Now, depending on the software, there's a different thing that's happening.
Some softwares are actually plotting F with the right-- let's say you want to do normal, as you asked.
So some software are just going to use F to be with mu hat and sigma hat, and that's fine.
Some software are actually not going to do this.
They're just going to use a Gaussian.
But then they're going to actually have a different reference point.
So what do we want to see here?
What should happen if all these points-- if all my points actually come from F, from a distribution that has CDF F?
What should happen?
What should I see?
Well, since Fn should be close to F, Fn inverse should be close to F inverse, which means that this point should be close to that point.
This point should be close to that point.
So ideally, if I actually pick the right F, I should see a plot that looks like this, something where all my points are very close to the line y is equal to x, right?
And I'm going to have some fluctuations, but something very close to this.
Now, that's if F is exactly the right one.
If F is not exactly the right one, in particular, in the case of a Gaussian one, if I actually plotted here the quantiles-- so if I plotted F 0, 1 of t, right?
So let's say those are the ones I actually plot, but I really don't know what-- mu hat is not 0 and sigma hat is not 0.
And so this is not the one I should be getting.
Since we actually know that phi of mu hat sigma hat squared t is equal to phi 0, 1 of t minus mu hat divided by sigma hat, there's just this change of axis, which is actually very simple.
This change of axis is just a simple translation scaling, which means that this line here is going to be transformed into another line with a different slope and a different intercept.
And so some software will actually decide to go with this curve and just show you what the reference curve should be, rather than actually putting everything back onto the 45-degree curve
So if you get any straight line?
Any straight line, you're happy.
I mean, depending on the software.
Because if the software actually really rescaled this thing to have mu hat and sigma square and you find a different line, a different straight line, this is bad news, which is not going to happen actually.
It's impossible that happens, because you actually-- well, it could.
If it's crazy, it could.
It shouldn't be very crazy.
OK.
So let's see what R does for us, for example.
So here in R, R actually does this funny trick where-- so here I did not actually plot the lines.
I should actually add the lines.
So the command is like qqnorm of my sample, right?
And that's really simple.
I just stack all my data into some vector, say, x.
And I say qqnorm of x, and it just spits this thing out.
OK?
Very simple.
But I could actually add another command, which I can't remember.
I think it's like qqline, and it's just going to add the line on top of it.
But if you see, actually what R does for us, it's actually doing the translation and scaling on the axes themselves.
So it actually changes the x and y-axis in such a way that when you look at your picture and you forget about what the meaning of the axes are, the relevant straight line is actually still the 45-degree line.
It's Because it's actually done the change of units for you.
So you don't have to even see the line.
You know that, in your mind, that this is basically-- the reference line is still 45 degree because that's the way the axes are made.
But if I actually put my axes, right-- so here, for example, it goes from-- let's look at some-- well, OK, those are all square.
Yeah, and that's probably because they actually have-- the samples are actually from a standard normal.
So I did not make my life very easy to illustrate your question, but of course, I didn't know you were going to ask it.
Next time, let's just prepare.
Let's script more.
We'll see another one in the next plot.
But so here what you expect to see is that all the plots should be on the 45-degree line, right?
This should be the right one.
And if you see, when I start having 10,000 samples, this is exactly what's happening.
So this is as good as it gets.
This is an N(0, 1) plotted against the theoretical quantile of an N(0, 1).
As good as it gets.
And if you see, for the second one, which is 50, sample size of size-- sample of size 50, there is some fudge factor, right?
I mean, those things-- doesn't look like there's a straight line, right?
It sort of appears that there are some weird things happening here at the lower tail.
And the reason why this is happening is because we're trying to compare the tails, right?
When I look at this picture, the only thing that goes wrong somehow is always at the tip, because those are sort of rare and extreme values, and they're sort of all over the place.
And so things are never really super smooth and super clean.
So this is what your best shot is.
This is what you will ever hope to get.
So size 10, right, so you have 10 points.
Remember, we actually-- well, I didn't really tell you how to deal with the extreme cases.
Because the problem is that F inverse of 1 for the true F is plus infinity.
So you have to make some sort of weird boundary choices to decide what F inverse of 1 is, and it's something that's like somewhere.
But you still want to put like 10 dots, right?
1, 2, 3, 4, 5, 6, 7, 8, 9, 10 dots.
So I have 10 observations, you will see 10 dots.
I have 50 observations, you will see 50 dots, right, because I have-- there are 1/n, 2/n, 3/n all the way to n/n.
I didn't tell you the last one.
OK.
So this is when things go well, and this is when things should not go well.
OK?
So here, actually, the distribution is a Student's t with 15 degrees of freedom, which should depart somewhat from a Gaussian distribution.
The tails should be heavier.
And what you can see is basically the following, is that for 10 you actually see something that's crazy, right, if I do 10 observations.
But if I do 50 observations, honestly, it's kind of hard to say that it's different from the standard normal.
So you could still be happy with this for 100.
And then this is what's happening for 10,000.
And even here it's not the beautiful straight line, but it feels like you would be still tempted to conclude that it's a beautiful straight line.
So let's try to guess.
So basically, there's-- for each of those sides there's two phenomena.
Either it goes like this or it goes like this, and then it goes like this or it goes like this.
Each side corresponds to the left tail, all the smallest values.
So that's the left side.
And that's the right side-- corresponds to the large values.
OK?
And so basically you can actually think of some sort of a table that tells you what your QQ plot looks like.
And so let's say it looks-- so we have our reference 45-degree line.
So let's say this is the QQ plot.
That could be one thing.
This could be the QQ plot where I have another thing.
Then I can do this guy, and then I do this guy.
So this is like this.
OK?
So those are the four cases.
OK?
And here what's changing is the right tail, and here what's changing is the-- and when I go from here to here, what changes is the left tail.
Is that true?
No, sorry.
What changes here is the right tail, right?
It's this part that changes from top to bottom.
So here it's something about right tail, and here that's something about left tail.
Everybody understands what I mean when I talk about tails?
OK. And so here it's just going to be a question of whether the tails are heavier or lighter than the Gaussian.
Everybody understand what I mean when I say heavy tails and light tails?
OK.
So right, so heavy tails just means that basically here the tails of this guy are heavier than the tails of this guy.
So it means that if I draw them, they're going to be above.
Actually, I'm going to keep this picture because it's going to be very useful for me.
When I plug the quantiles at the same-- so let's look at the right tail, for example.
Right here my picture is for right tails.
When I look at the quantiles of my theoretical distribution-- so here you can see the bottom curve we have the theoretical quantiles, and those are the empirical quantiles.
If I look to the right here, are the theoretical quantiles larger or smaller than the empirical quantiles?
Let me phrase it the other-- are the empirical quantiles larger or smaller than the theoretical quantiles?
This is a graph of quantiles, right?
So if it's [INAUDIBLE] it should be smaller
It should be smaller, right?
On this line, they are equal.
So if I see the empirical quantile showing up here, it means that here the empirical quantile is less than the theoretical quantile.
Agree?
So that means that if I look at this thing-- and that's for the same values, right?
So the quantiles are computed for the same values i/n.
So it means that the empirical quantiles should be looking-- so that should be the empirical quantile, and that should be the theoretical quantile.
Agreed?
Those are the smaller values for the same alpha.
So that implies that the tails-- the right tail, is it heavy or lighter-- heavier or lighter than the Gaussian?
Lighter
Lighter, right?
Because those are the tails of the Gaussian.
Those are my theoretical quantiles.
That means that this is the tail of my empirical distribution.
So they are actually lighter.
OK?
So here, if I look at this thing, this means that the right tail is actually light.
And by light, I mean lighter than Gaussian.
Heavy, I mean heavier than Gaussian.
OK?
OK, now we can probably do the entire thing.
Well, if this is light, this is going to be heavy, right?
That's when I'm above the curve.
Exercise-- is this light or is this heavy, the first column?
And it's OK.
It should take you at least 30 seconds
[INAUDIBLE] different column?
Yeah, this column, right?
So this is something that pertains-- this entire column is going to tell me whether the fact that this guy is above, does this mean that I have lighter or heavier left tails?
Well, on the left, it's heavier
On the left, it's heavier.
OK.
I don't know.
Actually, I need to draw a picture.
You guys are probably faster than I am
[INTERPOSING VOICES]
Actually, let me check how much randomness is-- who says it's lighter?
Who says it's heavier?
Yeah, but we're biased
[INAUDIBLE]
Yeah,
[INAUDIBLE]
All right.
So let's see if it's heavier.
So we're on the left tail, and so we have one looks like this, one looks like that, right?
So we know here that I'm looking at this part here.
So it means that here my empirical quantile is larger than the theoretical quantile.
OK?
So are my tails heavier or lighter?
They're lighter.
That was a bad bias
[INAUDIBLE]
Right?
It's below, so it's lighter.
Because the problem is that larger for the negative ones means that it's smaller [INAUDIBLE],, right?
Yeah?
Sorry but, what exactly are these [INAUDIBLE]??
If this is the inverse-- if this is the inverse CDF, shouldn't everything-- well, if this is the inverse CDF, then you should only be inputting values between 0 and 1 in it.
And--
Oh, did I put the inverse CDF?
Like on the previous slide, I think
No, the inverse CDF, yeah, so I'm inputting--
Oh, you're [INAUDIBLE]
Yeah, so it's a scatter plot, right?
So each point is attached-- each point is attached 1/n, 2/n, 3/n.
Now, for each point I'm plotting, that's my x-value, which maps a number between 0 and 1 back onto the entire real line, and my y-value is the same.
OK?
So what it means is that those two numbers, this is in the-- this lives on the entire real line, not on the interval.
This lives on the entire real line, not in the interval.
And so my QQ plots take values on the entire real line, entire real line, right?
So you think of it as a parameterized curve, where the time steps are 1/n, 2/n, 3/n, and I'm just like putting a dot every time I'm making one step.
OK?
OK, so what did we say?
That was lighter, right?
[INAUDIBLE]
OK?
One of my favorite exercises is, here's a bunch of densities.
Here's a bunch of QQ plots.
Map the correct QQ plot to its own density.
All right?
And there won't be mingled lines that allow you to do that, then you just have to follow, like at the back of cereal boxes.
All right.
Are there any questions?
So one thing-- there's two things I'm trying to communicate here is if you see a QQ plot, now you should understand, one, how it was built, and two, whether it means that you have heavier tails or lighter tails.
Now, let's look at this guy.
What should we see?
We should see heavy on the left and heavy on the right, right?
We know that this should be the case.
So this thing actually looks like this, and it sort of does, right?
If I take this line going through here, I can see that this guy's tipping here, and this guy's dipping here.
But honestly-- actually, I can't remember exactly, but t 15, if I plotted the density on top of the Gaussian, you can see a difference.
But if I just gave it to you, it would be very hard for you to tell me if there's an actual difference between t 15 and Gaussian, right?
Those things are actually very close.
And so in particular, here we're really trying to recognize what the shape is the fact-- right?
So t 15 compared to a standard Gaussian was different, but t 15 compared to a Gaussian with a slightly larger variance is not going to actually-- you're not going to see much of a difference.
So in a way, such distributions are actually not too far from the Gaussian, and it's not too-- it's still pretty benign to conclude that this was actually a Gaussian distribution because you can just use the variance as a little bit of a buffer.
I'm not going to get really into how you would use a t-distribution into a t-test, because it's kind of like Inception, right?
So but you could pretend that your data actually is t-distributed and then build a t-distribution from it, but let's not say that.
Maybe that was a bad example.
But there's like other heavy-tailed distributions like Cauchy distribution, which doesn't even have a-- it's not even integrable because that's as heavy as the tails get.
And this you can really tell it's going to look like this.
It's going to be like pfft.
What does a uniform distribution look like?
Like this?
It's going to be-- it's going to look like a Gaussian one, right?
So a uniform-- so this is my Gaussian.
A uniform is basically going to look like this, one side take the right mean and the right variance, right?
So the tails are definitely lighter.
They're 0.
That's as lighter as it gets.
So the light-light is going to look like this S shape.
So an S-- light-tailed distribution has this S shape.
OK?
What is the exponential going to look like?
So the exponential is positively supported.
It only has positive numbers.
So there's no left tail.
This is also as light as it gets.
But the right tail, is it heavier or lighter than the Gaussian?
Heavier
It's heavier, right?
It's only the case like e of the minus x rather e to the minus x squared.
So it's heavier.
So it means that on the left it's going to be light, and on the right it's going to be heavy.
So it's going to be U-shaped.
OK?
That will be fine.
All right.
Any other question?
Again, two messages, like, more technical, and you can sort of fiddle with it by looking at it.
You can definitely conclude that this is OK enough to be Gaussian for your purposes.
Yeah?
So [INAUDIBLE]
I did not hear the "if" at the beginning of your sentence
I would want to be lighter tail, right, because that'll be-- it's easier to reject?
Is that correct?
So what is your purpose as a--
I want to-- I have some [INAUDIBLE] right?
I want to be able to say I reject H0 [INAUDIBLE]
Yes
So if you wanted to make it easier to reject H0, then--
Yeah, in a way that's true, right?
So once you've actually factored in the mean and the variance, the only thing that actually-- right.
So if you have Gaussian tails or lighter-- even lighter tails, then it's harder for you to explain deviations from randomness only, right?
If you have a uniform distribution and you see something which is-- if you're uniform on 0, 1 plus some number and you see 25, you know this number is not going to be 0, right?
So that's basically as good as it gets.
And there's basically some smooth interpolation if you have lighter tails.
Now, if you start having something that has heavy tails, then it's more likely that pure noise will generate large observations and therefore discovery.
So yes, lighter tails is definitely the better-behaved noise.
Let's put it this way.
The lighter it is, the better behaved it is.
Now, this is good-- this is good for some purposes, but when you want to compute actual quantiles, like exact quantiles, then it is true in general that the quantiles of lighter-tail distributions are going to be dominated by the-- are going to be dominated by the-- let's say on the right tails, are going to be dominated by those of a heavy distribution.
That is true.
But that's not always the case.
And in particular, there's going to be some like sort of weird points where things are actually changing depending on what level you're actually looking at those things, maybe 5% or 10%, in which case things might be changing a little bit.
But if you started going really towards the tail, if you start looking at levels alpha which are 1% or 0.1%, it is true that it's always-- if you can actually-- so if you see something that looks light tail, you definitely do not want to conclude that it's Gaussian.
You want to actually change your modeling so that it makes your life even easier.
And you actually factor in the fact that you can see that the noise is actually more benign than you would like it to be.
OK?
Stretching fingers, that's it?
All right.
OK.
So I want to-- I mentioned at some point that we had this chi-square test that was showing up.
And I do not know what I did-- let's just-- oh, yeah.
So we have this chi-square test that we worked on last time, right?
So the way I introduced the chi-square test is by saying, I am fascinated by this question.
Let's check if it's correct, OK?
Or something maybe slightly deeper-- let's check if juries in this country are representative of racial distribution.
But you could actually-- those numbers here come from a very specific thing.
That was the uniform.
That was our benchmark.
Here's the uniform.
And there was this guy, which was a benchmark, which was the actual benchmark that we need to have for this problem.
And those things basically came out of my hat, right?
Those are numbers that exist.
But in practice, you actually make those numbers yourself.
And the way you do it is by saying, well, if I have a binomial distribution and I want to test if my data comes from a binomial distribution, you could ask this question, right?
You have a bunch of data.
I did not promise to you that this was the sum of independent Bernoullis and [INAUDIBLE].. And then you can actually check that it's a binomial indeed, and you have binomial.
If you think about where you've encountered binomials, it was mostly when you were drawing balls from urns, which you probably don't do that much in practice.
OK?
And so maybe one day you want to model things as a binomial, or maybe you want to model it as a Poisson, as a limiting binomial, right?
People tell you photons arrive-- the rate of a photon hitting some surface is actually a Poisson distribution, right?
That's where they arise a lot in imaging.
So if I have a colleague who's taking pictures of the skies over night, and he's like following stars and it's just like moving around with the rotation of the Earth.
And he has to do this for like eight hours because he needs to get enough photons over this picture to actually arise.
And he knows they arrive at like a Poisson process, and you know, chapter 7 of your probability class, I guess.
And And there's all these distributions outside the classroom you probably want to check that they're actually correct.
And so the first one you might want to check, for example, is a binomial.
So I give you a distribution, a binomial distribution on, say, K trials, and you have some number p. And here, I don't know typically what p should be, but let's say I know it or estimate it from my data.
And here, since we're only going to deal with asymptotics, just like it was the case for the Kolmogorov-Smirnov one, in the asymptotic we're going to be able to think of the estimated p as being a true p, OK, under the null at least.
So therefore, each outcome, I can actually tell you what the probability of a binomial-- is this outcome.
For a given K and a given p, I can tell you exactly what a binomial should give you as the probability for the outcome.
And that's what I actually use to replace the numbers 1/12, 1/12, 1/12, 1/12 or the numbers 0.72, 0.7, 0.12, 0.9.
All these numbers I can actually compute using the probabilities of a binomial, right?
So I know, for example, that the probability that a binomial np is equal to, say, K is n choose K p to the K 1 minus p to the n minus K. OK?
I mean, so these are numbers.
If you give me p and you give me n, I can compute those numbers for all K from 0 to n. And from this I can actually build a table.
All right?
So for each K-- 0.
So K is here, and from 0, 1, et cetera, all the way to n, I can compute the true probability, which is the probability that my binomial np is equal to 0, the probability that my binomial is equal to 1, et cetera, all the way to n. I can compute those numbers.
Those are actually going to be exact numbers, right?
I just plug in the formula that I had.
And then I'm going to have some observed.
So that's going to be p hat, 0, and that's basically the proportion of 0's, right?
So here you have to remember it's not a one-time experiment like you do in probability where you say, I'm going to draw n balls from an urn, and I'm counting how many-- how many I have.
This is statistics.
I need to be able to do this experiment many times so I can actually, in the end, get an idea of what the proportion of p's is.
So you have not just one binomial, but you have n binomials.
Well, maybe I should not use n twice.
So that's why it's the K here, right?
So I have a binomial [INAUDIBLE] at Kp and I just seize n of those guys.
And with this n of those guys, I can actually estimate those probabilities.
And what I'm going to want to check is if those two probabilities are actually close to each other.
But I already know how to do this.
All right?
So here I'm going to test whether P is in some parametric family, for example, binomial or not binomial.
And testing-- if I know that it's a binomial [INAUDIBLE],, and I basically just have to test if P is the right thing.
OK?
Oh, sorry, I'm actually lying to you here.
OK.
I don't want to test if it's binomial.
I want to test the parameter of the binomial here.
OK?
So I know-- no, sorry, [INAUDIBLE] sorry.
OK.
So I want to know if I'm in some family, the family of binomials, or not in the family of binomials.
OK?
Well, that's what I want to do.
And so here H0 is basically equivalent to testing if the pj's are the pj's that come from the binomial.
And the pj's here are the probabilities that I get.
This is the probability that I get j successes.
That's my pj.
That's j's value here.
OK?
So this is the example, and we know how to do this.
We construct p hat, which is the estimated proportion of successes from the observations.
So here now I have n trials.
This is the actual maximum likelihood estimator.
This becomes a multinomial experiment, right?
So it's kind of confusing.
We have a multinomial experiment for a binomial distribution.
The binomial here is just a recipe to create some test probabilities.
That's all it is.
The binomial here doesn't really matter.
It's really to create the test probabilities.
And then I'm going to define this test statistic, which is known as the chi-square statistic, right?
This was the chi-square test.
We just looked at sum of the square root of the differences.
Inverting the covariance matrix or using the Fisher information with removing the part that was not invertible led us to actually use this particular value here, and then we had to multiply by n. OK?
And that, we know, converges to what?
A chi-square distribution.
So I'm not going to go through this again.
I'm just telling you you can use the chi-square that we've seen, where we just came up with the numbers we were testing.
Those numbers that were in this row for the true probabilities, we came up with them out of thin air.
And now I'm telling you you can actually come up with those guys from a binomial distribution or a Poisson distribution or whatever distribution you're happy with.
Any question?
So now I'm creating this thing, and I can apply the entire theory that I have for the chi-square and, in particular, that this thing converges to a chi-square.
But if you see, there's something that's different.
What is different?
The degrees of freedom.
And if you think about it, again, the meaning of degrees of freedom.
What does this word-- these words actually mean?
It means, well, to which extent can I play around with those values?
What are the possible values that I can get?
If I'm not equal to this particular value I'm testing, how many directions can I be different from this guy?
And when we had a given set of values, we could be any other set of values, right?
So here, I had this-- I'm going to represent-- this is the set of all probability distributions of vectors of size K. So here, if I look at one point in this set, this is something that looks like p1 through pK such that their sum-- such that they're non-negative, and the sum p1 through pK is equal to 1.
OK?
So I have all those points here.
OK?
So this is basically the set that I had before.
I was testing whether I was equal to this one guy, or if I was anything else.
And there's many ways I can be anything else.
What matters, of course, is what's around this guy that I could actually confuse myself with.
But there's many ways I can move around this guy.
Agreed?
Now I'm actually just testing something very specific.
I'm saying, well, now the piece that I have had to come from this-- have to be constructed from this formula, this parametric family P of theta.
And there's a fixed way for-- let's say this is theta, so I have a theta here.
There's not that many ways this can actually give me a set of probabilities, right?
I have to move to another theta to actually start being confused.
And so here the number of degrees of freedom is basically, how can I move along this family?
And so here, this is all the points, but there might be just the subset of the points that looks like this, just this curve, not the half of this thing.
And those guys on this curve are the p thetas, and that's for all thetas when theta runs across data.
So in a way, this is just a much smaller dimensional thing.
It's a much smaller object.
Those are only the ones that I can create that are exactly of this very specific parametric form.
And of course, not all are of this form.
Not all probability PMFs are of this form.
And so that is going to have an effect on what my PMF is going to be-- sorry, on what my-- sorry, what my degrees of freedoms are going to be.
Because when this thing is very small, that means when-- that's happening when theta is actually, say, a one-dimensional space, then there's still many ways I can escape, right?
I can be different from this guy in pretty much every other direction, except for those two directions, just when I move from here or when I move in this direction.
But now if this thing becomes bigger, your theta is, say, two dimensional, then when I'm here it's becoming harder for me to not be that guy.
If I want to move away from it, then I have to move away from the board.
And so that means that the bigger the dimension of my theta, the smaller the degrees of freedoms that I have, OK, because moving out of this parametric family is actually very difficult for me.
So if you think, for example, as an extreme case, the parametric family that I have is basically all PMFs, all of them, right?
So that's a stupid parametric family.
I'm indexed by the distribution itself, but it's still finite dimensional.
Then here, I have basically no degrees of freedom.
There's no way I can actually not be that guy, because this is everything I have.
And so you don't have to really understand how the computation comes into the numbers of dimension and what I mean by dimension of this current space.
But really, what's important is that as the dimension of theta becomes bigger, I have less degrees of freedom to be away from this family.
This family becomes big, and it's very hard for me to violate this.
So it's actually shrinking the number of degrees of freedom of my chi-square.
And that's all you need to understand.
When d increases, the number of degrees of freedom decreases.
And I'd like to you to have an idea of why this is somewhat true, and this is basically the picture you should have in mind.
OK.
So now once I have done this, I can just construct.
So here I need to check.
So what is d in the case of the binomial?
1
1, right?
It's just a one-dimensional thing.
And for most of the examples we're going to have it's going to be one dimensional.
So we have this weird thing.
We're going to have K minus 2 degrees of freedom.
So now I have this thing, and I have this asymptotic.
And then I can just basically use a test that has-- that uses the fact that the asymptotic distribution is this.
So I compute my quantiles out of this.
Again, I made the same mistake.
This should be q alpha, and this should be q alpha.
So that's just the tail probability is equal to alpha when I'm on the right of q alpha.
And so those are the tail probability of the appropriate chi-square with the appropriate number of degrees of freedom.
And so I can compute p-values, and I can do whatever I want.
OK?
So then I just like [INAUDIBLE] my testing machinery.
OK?
So now I know how to test if I'm a binomial distribution or not.
Again here, testing if I'm a binomial distribution is not a simple goodness of fit.
It's a composite one where I can actually-- there's many ways I can be a binomial distribution because there's as many as there is theta.
And so I'm actually plugging in the theta hat, which is estimated from the data, right?
And here, since everything's happening in the asymptotics, I'm not claiming that Tn has a pivotal distribution for finite n. That's actually not true.
It's going to depend like crazy on what the actual distribution is.
But asymptotically, I have a chi-square, which obviously does not depend on anything [INAUDIBLE].. OK?
Yeah?
So in general, for the binomial [INAUDIBLE] trials.
But in the general case, the number of-- the size of our PMF is the number of [INAUDIBLE]
Yeah
So let's say that I was also uncertain about what K was so that I don't know how big my [INAUDIBLE] is.
[INAUDIBLE]
That is correct.
And thank you for this beautiful segue into my next slide.
So we can actually deal with the case not only where it's infinite, which would be the case of Poisson.
I mean, nobody believes I'm going to get an infinite number of photons in a finite amount of time.
But we just don't want to have to say there's got to be a-- this is the largest possible number.
We don't want to have to do that.
Because if you start doing this and the probabilities become close to 0, things become degenerate and it's an issue.
So what we do is we bin.
We just bin stuff.
OK?
And so maybe if I have a binomial distribution with, say, 200,000 possible values, then it's actually maybe not the level of precision I want to look at this.
Maybe I want to bin.
Maybe I want to say, let's just think of all things that are between 0 and 100 to be the same thing, between 100 and 200 the same thing, et cetera.
And so in fact, I'm actually going to bin.
I don't even have to think about things that are discrete.
I can even think about continuous cases.
And so if I want to test if I have a Gaussian distribution, for example, I can just approximate that by some, say, piecewise constant function that just says that, well, if I have a Gaussian distribution like this, I'm going to bin it like this.
And I'm going to say, well, the probability that I'm less than this value is this.
The probability that I'm between this and this value is this.
The probability I'm between this and this value is this, and then this and then this, right?
And now I've turned-- I've discretized, effectively, my Gaussian into a PMF.
The value-- this is p1.
The value here is p1.
This is p2.
This is p3.
This is p4.
This is p5 and p6, right?
I have discretized my Gaussian into six possible values.
That's just the probability that they fall into a certain bin.
And we can do this-- if you don't know what K is, just stop at 10.
You look at your data quickly and you say, well, you know, I have so few of them that are-- like I see maybe one 8, one 11, and one 15.
Well, everything that's between 8 and 20 I'm just going to put it in one bin.
Because what else are you going to do?
I mean, you just don't have enough observations.
And so what we do is we just bin everything.
So here I'm going to actually be slightly abstract.
Our bins are going to be intervals Aj.
So here-- they don't even have to be intervals.
I could go crazy and just like call the bin this guy and this guy, right?
That would make no sense, but I could do that.
And then I'm-- and of course, you can do whatever you want, but there's going to be some consequences in the conclusions that you can take, right?
All you're going to be able to say is that my distribution does not look like it could be binned in this way.
That's all you're going to be able to say.
So if you decide to just put all the negative numbers and the positive numbers, then it's going to be very hard for you to distinguish a Gaussian from a random variable that takes values of minus 1 and plus 1 only.
You need to just be reasonable.
OK?
So now I have my pj's become the probability that my random variable falls into bin j.
So that's pj of theta under the parametric distribution.
For the true one, whether it's parametric or not, I have a pj.
And then I have p hat j, which is the proportion of observations that falls in this bin.
All right?
So I have a bunch of observations.
I count how many of them fall in this bin.
I divide by n, and that tells me what my estimated probability for this bin is.
And theta hat, well, it's the same as before.
If I'm in a parametric family, I'm just estimating theta hat, maybe the maximum likelihood estimator, plug it in, and estimate those pj's of theta hat.
From this, I form my chi-square, and I have exactly the same thing as before.
So the answer to your question is, yes, you bin.
And it's the answer to even more questions.
So that's why there you can actually use the chi-square test to test for normality.
Now here it's going to be slightly weaker, because there's only an asymptotic theory, whereas Kolmogorov-Smirnov and Kolmogorov-Lilliefors work actually even for finite samples.
For the chi-square test, it's only asymptotic.
So you just pretend you actually know what the parameters are.
You just stuff them into a theta, a mu hat, and sigma square hat.
And you just go to-- you just cross your finger that n is large enough for everything to have converged by the time you make your decision.
OK?
And then this is a copy/paste, with the same error actually as the previous slide, where you just build your test based on whether you exceed or not some quantile, and you can also compute some p-value.
OK?
The error?
I'm sorry?
What's the error?
What is the error?
You said [INAUDIBLE] copy/paste [INAUDIBLE]
Oh, the error is that this should be q alpha, right?
I've been calling this q alpha.
I mean, that's my personal choice, because I don't want to-- I only use q alpha.
So I only use quantiles where alpha is to the right, so.
That's what statisticians-- probabilists would use this notation.
OK. And so some questions, right?
So of course, in practice you're going to have some issues which translate.
I say, well, how do you pick this guy, this K?
So I gave you some sort of a-- I mean, the way we discussed, right?
You have 8 and 10 and 20, then it's ad hoc.
And so depending on whether you want to stop K at 20 or if you want to bin those guys is really up to you.
And there's going to be some considerations about the particular problem at hand.
I mean, is it coarse-- too coarse for your problem to decide that the observations between 8 and 20 are the same?
It's really up to you.
Maybe that's actually making a huge difference in terms of what phenomenon you're looking at.
The choice of the bins, right?
So here there's actually some sort of rules, which are don't use only one bin and make sure there's actually-- don't use them too small so that there's at least one observation per bin, right?
And it's basically the same kind of rules that you would have to build a histogram.
If you were to build a histogram for your data, you still want to make sure that you bin in an appropriate fashion.
OK?
And there's a bunch of rule of thumbs.
Every time you ask someone, they're going to have a different rule of thumb, so just make your own.
And then there's the computation of pj of theta, which might be a bit complicated because, in this case, I would have to integrate the Gaussian between this number and this number.
So for this case, I could just say, well, it's the difference of the CDF in that value and that value and then be happy with it.
But you can imagine that you have some slightly more crazy distributions.
You're going to have to somewhat compute some integrals that might be unpleasant for you to compute.
OK?
And in particular, I said the difference of the PDF between that value and that value of-- sorry, the CDF between that value and that value, it is true.
But it's not like you actually have tables that compute the CDF at any value you like, right?
You have to sort of-- well, there might be but at some degree, but you are going to have to use a computer typically to do that.
OK?
And so for example, you could do the Poisson.
If I had time, if I had more than one minute, I would actually do it for you.
But it's basically the same.
The Poisson, you are going to have an infinite tail, and you just say, at some point I'm going to cut everything that's larger than some value.
All right?
So you can play around, right?
I say, well, if you have extra knowledge about what you expect to see, maybe you can cut at a certain number and then just fold all the largest values from K minus 1 to infinity so that you actually have-- you have everything into one large bin.
OK?
That's the entire tail.
And that's the way people do it in insurance companies, for example.
They assume that the number of accidents you're going to have is a Poisson distribution.
They have to fit it to you.
They have to know-- or at least to your pool of insurance of injured people.
So they just slice you into what your character-- relevant characteristics are, and then they want to estimate what the Poisson distribution is.
And basically, they can do a chi-square test to check if it's indeed a Poisson distribution.
All right.
So that will be it for today.
And so I'll be-- I'll have your homework--
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MITOpenCourseWare@OCW.MIT.edu
It's because if I was not, this would be basically the last topic we would ever see.
And this is arguably, probably the most important topic in statistics, or at least that's probably the reason why most of you are taking this class.
Because regression implies prediction, and prediction is what people are after to now, right?
You don't need to understand what the model for the financial market is if you actually have a formula to predict what the stock prices are going to be tomorrow.
And regression, in a way, allows us to do that.
And we'll start with a very simple version of regression, which is linear regression, which is the most standard one.
And then we'll move on to slightly more advanced notions such as nonparametric regression.
At least, we're going to see the principles behind it.
And I'll touch upon a little bit of high dimensional regression, which is what people are doing today.
So the goal of regression is to try to predict one variable based on another variable.
All right, so here the notation is very important.
It's extremely standard.
It goes everywhere essentially, and essentially you're trying to explain why as a function of x, which is the usual y equals f of x question-- except that, you know, if you look at a calculus class, people tell you y equals f of x, and they give you a specific form for f, and then you do something.
Here, we're just going to try to estimate what this length function is.
And this is why we often call y the explained variable and x the explanatory variable.
All right, so we're statisticians, so we start with data.
All right, then what does our data look like?
Well, it looks like a bunch of input, output to this relationship.
All right, so we have a bunch of xi, yi.
Those are pairs, and I can do a scatterplot of those guys.
So each point here has a x-coordinate, which is xi, and a y-coordinate, which is yi, and here, I have a bunch of endpoints.
And I just draw them like that.
Now, the functions we're going to be interested in are often function of the form y equals a plus b times x, OK. And that means that this function looks like this.
So if I do x and y, this function looks exactly like a line, and clearly those points are not on the line.
And it will basically never happen that those points are on a line.
There's a famous T-shirt from, I think, U.C.
Berkeley's staff department, that shows this picture and put a line between them like we're going to see it.
And it says, oh, statisticians, so many points, and you still managed to miss all of them.
And so essentially, we don't believe that this relationship y is equal to a plus bx is true, but maybe up to some noise.
And that's where the statistics is going to come into play.
There's going to be some random noise that's going to play out, and hopefully the noise is going to be spread out evenly, so that we can average it if we have enough points.
Average it out, OK. And so this epsilon here is not necessarily due to randomness.
But again, just like we did modeling in the first place, it essentially accounts for everything we don't understand about this relationship.
All right, so for example-- so here, I'm not going to be-- give me one second, so we'll see an example in a second.
But the idea here is that if you have data, and if you believe that it's of the form, a plus b x plus some noise, you're trying to find the line that will explain your data the best, right?
In the terminology we've been using before, this would be the most likely line that explains the data.
So we can see that it's slightly-- we've just added another dimension to our statistical problem.
We don't have just x's, but we have y's, and we're trying to find the most likely explanation of the relationship between y and x.
All right, and so in practice, the way it's going to look like is that we're going to have basically two parameters to find the slope b and the intercept a, and given data, the goal is going to be to try to find the best possible line.
All right?
So what we're going to find is not exactly a and b, the ones that actually generate the data, but some estimators of those parameters, a hat and b hat constructed from the data.
All right, so we'll see that more generally, but we're not going to go too much in the details of this.
There's actually quite a bit that you can understand if you do what's called univariate regression when x is actually a real valued random variable.
So when this happens, this is called univariate regression.
And when x is in rp for p larger than or equal to 2, this is called multivariate regression.
OK, and so here we're just trying to explain y is a plus bx plus epsilon.
And here we're going to have something more complicated.
We're going to have y, which is equal to a plus b1, x1 plus b2, x2 plus bp, xp plus epsilon-- where x is equal to-- the coordinates of x are given by x1, 2xp, rp.
OK, so it's still linear.
Right, they still add all the coordinates of x with a coefficient in front of them, but it's a bit more complicated than just one coefficient for one coordinate of x, OK?
So we'll come back to multivariate regression.
Of course, you can write this as x transpose b, right?
So this entire thing here, this linear combination is of the form x transpose b, where b is the vector that has coordinates b1 to bp.
OK?
Sorry, here, it's in [?
rd, ?]
p is the natural notation.
All right, so our goal here, in the univariate one, is to try to write the model, make sense of this little twiddle here-- essentially, from a statistical modeling question, the question is going to be, what distributional assumptions do you want to put on epsilon?
Are you going to say they're Gaussian?
Are you going to say they're binomial?
OK, are you going to say they're binomial?
Are you going to say they're Bernoulli?
So that's going to be what we we're going to make sense of, and then we're going to try to find a method to estimate a and b.
And then maybe we're going to try to do some inference about a and b-- maybe test if a and b take certain values, if they're less than something, maybe find some confidence regions for a and b, all right?
So why would you want to do this?
Well, I'm sure all of you have an application, if I give you some x, you're trying to predict what y is.
Machine learning is all about doing this, right?
Without maybe trying to even understand the physics behind this, they're saying, well, you give me a bag of words, I want to understand whether it's going to be a spam or not.
You give me a bunch of economic indicators, I want you to tell me how much I should be selling my car for.
You give me a bunch of measurements on some patient, I want you to predict how this person is going to respond to my drug-- and things like this.
All right, and often we actually don't have much modeling intuition about what the relationship between x and y is, and this linear thing is basically the simplest function we can think of.
Arguably, linear functions are the simplest functions that are not trivial.
Otherwise, we would just say, well, let's just predict x of y to be a constant, meaning it does not depend on x.
But if you want it to depend on x, then your functions are basically as simple as it gets.
It turns out, amazingly, this does the trick quite often.
So for example, if you look at economics, you might want to assume that the demand is a linear function of the price.
So if your price is zero, there's going to be a certain demand.
And as the price increases, the demand is going to move.
Do you think b is going to be positive or negative here?
What?
Typically, it's negative unless we're talking about maybe luxury goods, where you know, the more expensive, the more people actually want it.
I mean, if we're talking about actual economic demand, that's probably definitely negative.
It doesn't have to be, you know, clearly linear, so that you can actually make it linear, transform it into something linear.
So for example, you have this like multiplicative relationship, PV equals nRT, which is the Ideal gas law.
If you want to actually write this relationship, if you want to predict what the pressure is going to be as a function of the volume and the temperature-- and well, let's assume that n is the Avogadro constant, and let's assume that the radius is actually fixed.
Then you take the log on each side, so you get PV equals nRT.
So what that means is that log PV is equal to log nRT.
So that means log P plus log V is equal to the log nR plus log T. So we said that R is constant, so this is actually your constant.
I'm going to call it a.
And then that means that log P is equal to minus log V. That log P is equal to a minus log V plus log T. OK?
And so in particular, if I write b equal to negative 1 and c equal to plus 1, this gives me the formula that I have here.
Now again, it might be the case that this is the ideal gas law.
So in practice, if I start recording pressure, and temperature, and volume, I might make measurement errors, there might be slightly different conditions in such a way that I'm not going to get exactly those.
And I'm just going to put this little twiddle to account for the fact that the points that I'm going to be recording for log pressure, log volume, and log temperature are not going to be exactly on one line.
OK, they're going to be close.
Actually, in those physics experiments, usually, they're very close because the conditions are controlled under lab experiments.
So it means that the noise is very small.
But for other cases, like demand and prices, it's not a law of physics, and so this must change.
Even the linear structure is probably not clear, right.
At some points, there's probably going to be some weird curvature happening.
All right, so this slide is just to tell you maybe you don't have, obviously, a linear relationship, but maybe you do if you start taking logs exponentials, squares.
You can sometimes take the product of two variables, things like this, right.
So this is variable transformation, and it's mostly domain-specific, so we're not going to go into more details of this.
Any questions?
All right, so now I'm going to be giving-- so if we start thinking a little more about what these coefficients should be, well, remember-- so everybody's clear why I don't put the little i here?
Right, I don't put the little i because I'm just talking about a generic x and a generic y, but the observations are x1, y1, right.
So typically, on the blackboard I'm often going to write only xy, but the data really is x1, y1, all the way to xn, yn.
So those are those points in this two dimensional plot.
But I think of those as being independent copies of the pair xy.
They have to have-- to contain their relationship.
And so when I talk about distribution of those random variables, I talk about the distribution of xy, and that's the same.
All right, so the first thing you might want to ask is, well, if I have an infinite amount of data, what can I hope to get for a and b?
If my simple size goes to infinity, then I should actually know exactly what the distribution of xy is.
And so there should be an a and a b that captures this linear relationship between y and x.
And so in particular, we're going to try to ask the population, or theoretic, values of a and b, and you can see that you can actually compute them explicitly.
So let's just try to find how.
So as I said, we have a bunch of points on this line close to a line, and I'm trying to find the best fit.
All right, so this guy is not a good fit.
This guy is not a good fit.
And we know that this guy is a good fit somehow.
So we need to mathematically formulate the fact that this line here is better than this line here or better than this line here.
So what we're trying to do is to create a function that has values that are smaller for this curve and larger for these two curves.
And the way we do it is by measuring the fit, and the fit is essentially the aggregate distance of all the points to the curve.
And there's many ways I can measure the distance to a curve.
So if I want to find so-- let's just open a parenthesis.
If I have a point here-- so we're going to do it for one point at a time.
So if I have a point, there's many ways I can measure its distance to the curve, right?
I can measure it like that.
That is one distance to the curve.
I can measure it like that by having a right angle here that is one distance to the curve.
Or I can measure it like that.
That is another distance to the curve, right.
There's many ways I can go for it.
It turns out that one is actually going to be fairly convenient for us, and that's the one that says, let's look at the square of the value of x on the curve.
So if this is the curve, y is equal to a plus bx.
Now, I'm going to think of this point as a random point, capital X, capital Y, so that means that it's going to be x1, y1 or x2, y2, et cetera.
Now, I want to measure the distance.
Can somebody tell me which of the three-- the first one, the second one, or the third one-- this formula, expectation of y minus a minus bx squared is-- which of the three is it representing?
The second one
The second one where I have the right angle?
OK, everybody agrees with this?
Anybody wants to vote for something else?
Yeah?
The third one?
The third one?
Everybody agrees with the third one?
So by default, everybody's on the first one?
Yeah, it is the vertical distance actually.
And the reason is if it was the one with the straight angle, with the right angle, it would actually be a very complicated mathematical formula, so let's just see y, right?
And by y, I mean y. OK, so this means that this is my x, and this is my y.
All right, so that means that this point is xy.
So what I'm measuring is the difference between y minus a plus b times x.
This is the thing I'm going to take the expectation off-- the square and then the expectation-- so a plus b times x, if this is this line, this is this point.
So that's this value here.
This value here is a plus bx, right?
So what I'm really measuring is the difference between y and N plus bx, which is this distance here.
And since I like things like Pythagoras theorem, I'm actually going to put a square here before I take the expectation.
So now this is a random variable.
This is this random variable.
And so I want a number, so I'm going to turn it into a deterministic number.
And the way I do this is by taking expectation.
And if you think expectations should be close to average, this is the same thing as saying, I want that in average, the y's are close to the a plus bx, right?
So we're doing it in expectation, but that's going to translate into doing it in average for all the points.
All right, so this is the thing I want to measure.
So that's this vertical distance.
Yeah?
OK.
This is my fault actually.
Maybe we should close those shades.
OK, I cannot do just one at a time, sorry.
All right, so now that I do those vertical distances, I can ask-- well, now, I have this function, right-- to have a function that takes two parameters a and b, maps it to the expectation of y minus a plus bx squared.
Sorry, the square is here.
And I could ask, well, this is a function that measures the fit of the parameters a and b, right?
This function should be small.
The value of this function here, function of a and b that measures how close the point xy is to the line a plus b times x while y is equal to a plus b times x in expectation.
OK, agreed?
This is what we just said.
Again, if you're not comfortable with the reason why you get expectations, just think about having data points and taking the average value for this guy.
So it's basically an aggregate distance of the points to their line.
OK, everybody agrees this is a legitimate measure?
If all my points were on the line-- if my distribution-- if y was actually equal to a plus bx for some a and b then this function would be equal to 0 for the correct a and b, right?
If they are far-- well, it's going to depend on how much noise I'm getting, but it's still going to be minimized for the best one.
So let's minimize this thing.
So here, I don't make any-- again, sorry.
I don't make an assumption on the distribution of x or y.
Here, I assume, somehow, that the variance of x is not equal to 0.
Can somebody tell me why?
Yeah?
Not really a question-- the slides, you have y minus a minus bx quantity squared expectation of that, and here you've written square of the expectation
No, here I'm actually in the expectation of the square.
If I wanted to write the square of the expectation, I would just do this.
So let's just make it clear.
Right?
Do you want me to put an extra set of parenthesis?
That's what you want me to do?
Yeah, it's just confusing with the [INAUDIBLE]
OK, that's the one that makes sense, so the square of the expectation?
Yeah
Oh, the expectation of the square, sorry.
Yeah, dyslexia.
All right, any question?
Yeah?
Does this assume that the error is Gaussian?
No
I mean, in the sense that like, if we knew that the error was, like, even the minus followed like-- so even the minus x to the fourth distribution, would we want to minimise the expectation of what the fourth power of y minus a equals bx in order to get [?
what the ?]
[?
best is?
?]
Why?
So you know the answers to your question, so I just want you to use the words that-- right, so why would you want to use the fourth power?
Well, because, like, we want to more strongly penalize deviations because we'd expect very large deviations to be very rare, or more rare, than it would with the Gaussian [INAUDIBLE] power
Yeah so, that would be the maximum likely estimator that you're describing to me, right?
I can actually write the likelihood of a pair of numbers ab.
And if I know this, that's actually what's going to come into it because I know that the density is going to come into play when I talk about there.
But here, I'm just talking about-- this is a mechanical tool.
I'm just saying, let's minimize the distance to the curve.
Another thing I could have done is take the absolute value of this thing, for example.
I just decided to take the square root before I did it.
OK, so regardless of what I'm doing, I'm just taking the squares because that's just going to be convenient for me to do my computations for now.
But we don't have any statistical model at this point.
I didn't say anything-- that y follows this.
X follows this.
I'm just doing minimal assumptions as we go, all right?
So the variance of x is not equal to 0?
Could somebody tell me why?
What would my cloud point look like if the variance of x was equal to 0?
Yeah, they would all be at the same point.
So it's going to be hard for me to start fitting in a line, right?
I mean, best case scenario, I have this x.
It has variance, zero, so this is the expectation of x.
And all my points have the same expectation, and so, yes, I could probably fit that line.
But that wouldn't help very much for other x's.
So I need a bit of variance so that things spread out a little bit.
OK, I'm going to have to do this.
I think it's just my-- All right, so I'm going to put a little bit of variance.
And the other thing is here, I don't want to do much more, but I'm actually going to think of x as having means zero.
And the way I do this is as follows.
Let's define x tilde, which is x minus the expectation of x. OK, so definitely the expectation of x tilde is what?
Zero, OK. And so now I want to minimize in ab, expectation of y minus a plus b, x squared.
And the way I'm going to do this is by turning x into x tilde and stuffing the extra-- and putting the extra expectation of x into the a.
So I'm going to write this as an expectation of y minus a plus b expectation of x-- which I'm going to a tilde-- and plus b x tilde.
OK?
And everybody agrees with this?
So now I have two parameters, a tilde and b, and I'm going to pretend that now x tilde-- so now the role of x is played by x tilde, which is now a centered random variable.
OK, so I'm going to call this guy a tilde, but for my computations I'm going to call it a.
So how do I find the minimum of this thing?
Derivative equal to zero, right?
So here it's a quadratic thing.
It's going to be like that.
I take the derivative, set it to zero.
So I'm first going to take the derivative with respect to a and set it equal to zero, so that's equivalent to saying that the expectation of-- well, here, I'm going to pick up a 2-- y minus a plus bx tilde is equal to zero.
And then I also have that the derivative with respect to b is equal to zero, which is equivalent to the expectation of-- well, I have a negative sign somewhere, so let me put it here-- minus 2x tilde, y minus a plus bx tilde.
OK, see that's why I don't want to put too many parenthesis.
OK.
So I just took the derivative with respect to a, which is just basically the square, and then I have a negative 1 that comes out from inside.
And then I take the derivative with respect to b, and since b has x tilde.
In [?
factor, ?]
it comes out as well.
All right, so the minus 2's really won't matter for me.
And so now I have two equations.
The first equation, while it's pretty simple, it's just telling me that the expectation of y minus a is equal to zero.
So what I know is that a is equal to the expectation of y.
And really that was a tilde, which implies that the a I want is actually equal to the expectation of y minus b times the expectation of x. OK?
Just because a tilde is a plus b times the expectation of x.
So that's for my a.
And then for my b, I use the second one.
So the second one tells me that the expectation of x tilde of y is equal to a plus b times the expectation of x tilde which is zero, right?
OK?
But this a is actually a tilde in this problem, so it's actually a plus b expectation of x.
Now, this is the expectation of the product of two random variables, but x tilde is centered, right?
It's x minus expectation of x, so this thing is actually equal to the covariance between x and y by definition of covariance.
So now I have everything I need, right.
How do I just-- I'm sorry about that.
So I have everything I need.
Now, I now have two equations with two unknowns, and all I have to do is to basically plug it in.
So it's essentially telling me that the covariance of xy-- so the first equation tells me that the covariance of xy is equal to a plus b expectation of x, but a is expectation of y minus b expectation of x.
So it's-- well, actually, maybe I should start with b. Oh, sorry.
OK, I forgot one thing.
This is not true, right.
I forgot this term.
x tilde multiplies x tilde here, so what I'm left with is x tilde-- it's minus b times the expectation of x tilde squared.
So that's actually minus b times the variance of x tilde because x tilde is already centered, which is actually the variance of x.
So now I have that this thing is actually a plus b expectation of x minus b variance of x.
And I also have that a is equal to expectation of y minus b expectation of x.
So if I sum the two, those guys are going to cancel.
Those guys are going to cancel.
And so what I'm going to be left with is covariance of xy is equal to expectation of x, expectation of y, and then I'm left with this term here, minus b times the variance of x.
And so that tells me that b-- why do I still have the variance there?
So is the covariance really the expectation of x tilde times y minus expectation of y?
Because y is not centered, correct?
Yeah
OK, but x is still the center
But x is still the center, right.
So you just need to have one that's centered for this to work.
Right, I mean, you can check it.
But basically when you're going to have the product of the expectations, you only need one of the two in the product to be zero.
So the product is zero.
OK, why do I keep my-- so I get a, a, and then the b expectation.
OK, so that's probably earlier that I made a mistake.
So I get-- so this was a tilde.
Let's just be clear about the-- So that tells me that a tilde-- maybe it's not super fair of me to-- yeah, OK, I think I know where I made a mistake.
I should not have centered.
I wanted to make my life easier, and I should not have done that.
And the reason is a tilde depends on b, so when I take the derivative with respect to b, what I'm left with here-- since a tilde depends on b, when I take the derivative of this guy, I actually don't get a tilde here, but I really get-- so again, this was not-- so that's the first one.
This is actually x here-- because when I take the derivative with respect to b.
And so now, what I'm left with is that the expectation-- so yeah, I'm basically left with nothing that helps.
So I'm sorry about.
Let's start from the beginning because this is not getting us anywhere, and a fix is not going to help.
So let's just do it again.
Sorry about that.
So let's not center anything and just do brute force because we're going to-- b x squared.
All right.
Partial, with respect to a, is giving equal zero is equivalent, so my minus 2 is going to cancel, right.
So I'm going to actually forget about this.
So it's actually telling me that the expectation of y minus a plus bx is equal to zero, which is equivalent to a plus b expectation of x, is equal to the expectation of y.
Now, if I take the derivative with respect to b and set it equal to zero, this is telling me that the expectation of-- well, it's the same thing except that this time I'm going to pull out an x.
This guy is equal to zero-- this guy is not here-- and so that implies that the expectation of xy is equal to a times the expectation of x, plus b times the expectation of x square.
OK?
All right, so the first one is actually not giving me much, so I need to actually work with the two of those guys.
So I'm going to take the first-- so let me rewrite those two inequalities that I have.
I have a plus b, e of x is equal to e of y.
And then I have e of xy.
OK, and now what I do is that I multiply this guy.
So I want to cancel one of those things, right?
So what I'm going to-- so I'm going to take this guy, and I'm going to multiply it by e of x and take the difference.
So I do times e of x, and then I take the sum of those two, and then those two terms are going to cancel.
So then that tells me that b times e of x squared, plus the expectation of xy is equal to-- so this guy is the one that cancelled.
Then I get this guy here, expectation of x times the expectation of y, plus the guy that remains here-- which is b times the expectation of x square.
So here I have b expectation of x, the whole thing squared.
And here I have b expectation of x square.
So if I pull this guy here, what do I get?
b times the variance of x, OK?
So I'm going to move here.
And this guy here, when I move this guy here, I get the expectation of x times y, minus the expectation of x times the expectation of y.
So this is actually telling me that the covariance of x and y is equal to b times the variance of x.
And so then that tells me that b is equal to covariance of xy divided by the variance of x.
And that's why I actually need the variance of x to be non-zero because I couldn't do that otherwise.
And because if it was, it would mean that b should be plus infinity, which is what the limit of this guy is when the variance goes to zero or negative infinity.
I can not sort them out.
All right, so I'm sorry about the mess, but that should be more clear.
Then a, of course, you can write it by plugging in the value of b, so you know it's only a function of your distribution, right?
So what are the characteristics of the distribution-- so distribution can have a bunch of things.
It can have movements of order 4, of order 26.
It can have heavy tails or light tails.
But when you compute least squares, the only thing that matters are the variance of x, the expectation of the individual ones-- and really what captures how y changes when you change x, is captured in the covariance.
The rest is really just normalization.
It's just telling you, I want things to cross the y-axis at the right place.
I want things to cross the x-axis at the right place.
But the slope is really captured by how much more covariance you have relative to the variance of x.
So this is essentially setting the scale for the x-axis, and this is telling you for a unit scale, this is the unit of y that you're changing.
OK, so we have explicit forms.
And what I could do, if I wanted to estimate those things, is just say, well again, we have expectations, right?
The expectation of xy minus the product of the expectations, I could replace expectations by averages and get an empirical covariance just like we can replace the expectations for the variance and get a sample covariance.
And this is basically what we're going to be doing.
All right, this is essentially what you want.
The problem is that if you view it that way, you sort of prevent yourself from being able to solve the multivariate problem.
Because it's only in the univariate problem that you have closed form solutions for your problem.
But if you actually go to multivariate, this is not where you want to replace expectations by averages.
You actually want to replace expectation by averages here.
And once you do it here, then you can actually just solve the minimisation problem.
OK, so one thing that arises from this guy is that this is an interesting formula.
All right, think about it.
If I have that y is a plus bx plus some noise.
Things are no longer on something.
I have that y is equal to a bx plus some noise, which is usually denoted by epsilon.
So that's the distribution, right?
If I tell you the distribution of x, and I say y is a plus b epsilon-- I tell you the distribution of y, and if [?
they mean ?]
that those two are independent, you have a distribution on y.
So what happens is that I can actually always say-- well, you know, this is equivalent to saying that epsilon is equal to y minus a plus bx, right?
I can always write this as just-- I mean, as tautology.
But here, for those guys-- this is not for any guy, right.
This is really for the best fit, a and b, those ones that satisfy this gradient is equal to zero thing.
Then what we had is that the expectation of epsilon was equal to expectation of y minus a plus b expectation of x by linearity of the expectation, which was equal to zero.
So for this best fit we have zero.
Now, the covariance between x and y-- Between, sorry, x and epsilon, is what?
Well, it's the covariance between x-- and well, epsilon was y minus a plus bx.
Now, the covariance is bilinear, so what I have is that the covariance of this is the covariance of xn times y-- sorry, of x and y, minus the variance-- well, minus a plus b, covariance of x and x, which is the variance of x?
Covariance of xy minus a plus b variance of x. OK, I didn't write it.
So here I have covariance of xy is equal to b variance of x, right?
Covariance of xy.
Yeah, that's because they cannot do that with the covariance.
Yeah, I have those averages again.
No, because this is centered, right?
Sorry, this is centered, so this is actually equal to the expectation of x times y minus a plus bx.
The covariance is equal to the product just because this insight is actually centered.
So this is the expectation of x times y minus the expectation of a times the expectation of x, plus b minus b times the expectation of x squared.
Well, actually maybe I should not really go too far.
So this is actually the one that I need.
But if I stop here, this is actually equal to zero, right.
Those are the same equations.
OK?
Yeah?
What are we doing right now?
So we're just saying that if I actually believe that this best fit was the one that gave me the right parameters, what would that imply on the noise itself, on this epsilon?
So here we're actually just trying to find some necessary condition for the noise to hold-- for the noise.
And so those conditions are, that first, the expectation is zero.
That's what we've got here.
And then, that the covariance between the noise and x has to be zero as well.
OK, so those are actually conditions that the noise must satisfy.
But the noise was just not really defined as noise itself.
We were just saying, OK, if we're going to put some assumptions on the epsilon, what do we better have?
So the first one is that it's centered, which is good, because otherwise, the noise would shift everything.
So now when you look at a linear regression model-- typically, if you open a book, it doesn't start by saying, let the noise be the difference between y and what I actually want y to be.
It says let y be a plus bx plus epsilon.
So conversely, if we assume that this is the model that we have, then we're going to have to assume that epsilon-- we're going to assume that epsilon is centered, and that the covariance between x and epsilon is zero.
Actually, often, we're going to assume much more.
And one way to ensure that those two things are satisfied is to assume that x is independent of epsilon, for example.
If you assume that x is independent of epsilon, of course the covariance is going to be zero.
Or we might assume that the conditional expectation of epsilon, given x, is equal to zero, then that implies that.
OK, now the fact that it's centered is one thing.
So if we make this assumption, the only thing it's telling us is that those ab's that come-- right, we started from there.
y is equal to a plus bx plus some epsilon for some a, for some b.
What it turns out is that those a's and b's are actually the ones that you would get by solving this expectation of square thing.
All right, so when you asked-- back when you were following-- so when you asked, you know, why don't we take the square, for example, or the power 4, or something like this-- then here, I'm saying, well, if I have y is equal to a plus bx, I don't actually need to put too much assumptions on epsilon.
If epsilon is actually satisfying those two things, expectation is equal to zero and the covariance with x is equal to zero, then the right a and b that I'm looking for are actually the ones that come with the square-- not with power 4 or power 25.
So those are actually pretty weak assumptions.
If we want to do inference, we're going to have to assume slightly more.
If we want to use T-distributions at some point, for example, and we will, we're going to have to assume that epsilon has a Gaussian distribution.
So if you want to start doing more statistics beyond just like doing this least square thing, which is minimizing the square of criterion, you're actually going to have to put more assumptions.
But right now, we did not need them.
We only need that epsilon as mean zero and covariant zero with x. OK, so that was basically probabilistic, right.
If I were to do probability and I were trying to model the relationship between two random variables, x and y, in the form y is a plus bx plus some noise, this is what would come out.
Everything was expectations.
There was no data involved.
So now let's go to the data problem, which is now, I do not know what those expectations are.
In particular, I don't know what the covariance of x and y is, and I don't know with the expectation of x and the expectation of y r. So I have data to do that.
So how am I going to do this?
Well, I'm just going to say, well, if I want x1, y1, xn, yn, and I'm going to assume that they're [?
iid.
?]
And I'm actually going to assume that they have some model, right.
So I'm going to assume that I have that a-- so that Yi follows the same model.
So epsilon i [?
rad, ?]
and I won't say that expectation of epsilon i is zero and covariance of xi, epsilon i is equal to zero.
So I'm going to put the same model on all the data.
So you can see that a is not ai, and b is not bi.
It's the same.
So as my data increases, I should be able to recover the correct things-- as the size of my data increases.
OK, so this is what the statistical problem look like.
You're given the points.
There is a true line from which this point was generated, right.
There was this line.
There was a true ab that I use to draw this plot, and that was the line.
So first I picked an x, say uniformly at on this intervals, 0 to 2.
I said that was this one.
Then I said well, I want y to be a plus bx, so it should be here, but then I'm going to add some noise epsilon to go away again back from this line.
And that's actually me, here, we actually got two points correct on this line.
So there's basically two epsilons that were small enough that the dots actually look like they're on the line.
Everybody's clear about what I'm drawing?
So now of course if you're a statistician, you don't see this.
You only see this.
And you have to recover this guy, and it's going to look like this.
You're going to have an estimated line, which is the red one.
And the blue line, which is the true one, the one that actually generated the data.
And your question is, while this line corresponds to some parameters a hat and b hat, how could I make sure that those two lines-- how far those two lines are?
And one to address this question is to say how far is a from a hat, and how far is b from b hat?
OK?
Another question, of course, that you may ask is, how do you find a hat and b hat?
And as you can see, it's basically the same thing.
Remember, what was a-- so b was the covariance between x and y divided by the variance of x, right?
We check and rewrite this.
The expectation of xy minus expectation of x times the expectation of y, divided by expectation of x squared minus expectation of x.
The whole thing's-- OK?
If you look at the expression for b hat, I basically replaced all the expectations by bars.
So I said, well, this guy I'm going to estimate by an average.
So that's the xy bar, and is 1 over n, [?
sum ?]
from [?
i co ?]
1, to n of Xi, times Yi.
x bar, of course, is just the one that we're used to.
And same for y bar.
X squared bar, the one that's here, is the average of the squares.
And x bar square is the square of the average.
OK, so you just basically replace this guy by x bar, this guy by y bar, this guy by x square bar, and this guy by x bar and no square.
OK, so that's basically one way to do it.
Everywhere you see an expectation, you replace it by an average.
That's the usual statistical hammer.
You can actually be slightly more subtle about this.
And as an exercise, I invite you-- just to make sure that you know how to do this competition, it's going to be exactly the same kind of competitions that we've done.
But as an exercise, you can check that if you actually look at say, well, what I wanted to minimize here, I had an expectation, right?
And I said, let's minimize this thing.
Well, let's replace this by an average first.
And now minimize.
OK, so if I do this, it turns out I'm going to actually get the same result.
The minimum of the average is basically-- when I replace the average by-- sorry, when I replace the expectation by the average and then minimize, it's the same thing as first minimizing and then replacing expectation by averages in this case.
Again, this is a much more general principle because if you don't have a closed form for the minimum like for some, say, likelihood problems, well, you might not actually have a possibility to just look at what the formula looks like-- see where the expectations show up-- and then just plug in the averages instead.
So this is the one you want to keep in mind.
And again, as an exercise.
OK, so here, and then you do expectation replaced by averages.
And then that's the same answer, and I encourage you to solve the exercise.
OK, everybody's clear that this is actually the same expression for a hat and b hat that we had before that we had for a and b when we replaced the expectations by averages?
Here, by the way, I minimize the sum rather than the average.
It's clear to everyone that this is the same thing, right?
Yep?
[INAUDIBLE] sum replacing it [INAUDIBLE] minimize the expectation, I'm assuming it's switched with the derivative on the expectation [INAUDIBLE]
So we did switch the derivative and the expectation before you came, I think.
All right, so indeed, the picture was the one that we said, so visually, this is what we're doing.
We're looking among all the lines.
For each line, we compute this distance.
So if I give you another line there would be another set of arrows.
You look at their length.
You square it.
And then you sum it all, and you find the line that has the minimum sum of squared lengths of the arrows.
All right, and those are the arrows that we're looking at.
But again, you could actually think of other distances, and you would actually get different-- you could actually get different solutions, right.
So there's something called, mean absolute deviation, which rather than minimizing this thing, is actually minimizing the sum from i to co 1 to n of the absolute value of y minus a plus bXi.
And that's not something for which you're going to have a closed form, as you can imagine.
You might have something that's sort of implicit, but you can actually still solve it numerically.
And this is something that people also like to use but way, way less than the least squares one
[INAUDIBLE]
What did I just what?
[INAUDIBLE] The sum of the absolute values of Yi minus a plus bXi.
So it's the same except I don't square here.
OK?
So arguably, you know, predicting a demand based on price is a fairly naive problem.
Typically, what we have is a bunch of data that we've collected, and we're hoping that, together, they can help us do a better prediction.
All right, so maybe I don't have only the price, but maybe I have a bunch of other social indicators.
Maybe I know the competition, the price of the competition.
And maybe I know a bunch of other things that are actually relevant.
And so I'm trying to find a way to combine a bunch of points, a bunch of measures.
There's a nice example that I like, which is people were trying to measure something related to your body mass index, so basically the volume of your-- the density of your body.
And the way you can do this is by just, really, weighing someone and also putting them in some cubic meter of water and see how much overflows.
And then you have both the volume and the mass of this person, and you can start computing density.
But as you can imagine, you know, I would not personally like to go to a gym when the first thing they ask me is to just go in a bucket of water, and so people try to find ways to measure this based on other indicators that are much easier to measure.
For example, I don't know, the length of my forearm, and the circumference of my head, and maybe my belly would probably be more appropriate here.
And so you know, they just try to find something that actually makes sense.
And so there's actually a nice example where you can show that if you measure-- I think one of the most significant was with the circumference of your wrist.
This is actually a very good indicator of your body density.
And it turns out that if you stuff all the bunch of things together, you might actually get a very good formula that explains things.
All right, so what we're going to do is rather than saying we have only one x to explain y's, let's say we have 20 x's that we're trying to combine to explain y.
And again, just like assuming something of the form, y is a plus b times x was the simplest thing we could do, here we're just going to assume that we have y is a plus b1, x1 plus b2, x2, plus b3, x3.
And we can write it in a vector form by writing that Yi is Xi transposed b, which is now a vector plus epsilon i. OK, and here, on the board, I'm going to have a hard time doing boldface, but all these things are vectors except for y, which is a number.
Yi is a number.
It's always the value of my y-axis.
So even if my x-axis lives on-- this is x1, and this is x2, y is really just the real valued function.
And so I'm going to get a bunch of points, x1,y1, and I'm going to see how much they respond.
So for example, my body density is y, and then all the x's are a bunch of other things.
Agreed with that?
So this is an equation that holds on the real line, but this guy here is an r p, and this guy's an rp.
It's actually common to talk to call b, beta, when it's a vector, and that's the usual linear regression notation.
Y is x beta plus epsilon.
So x's are called explanatory variables.
y is called explained variable, or dependent variable, or response variable.
It has a bunch of names.
You can use whatever you feel more comfortable with.
It should actually be explicit, right, so that's all you care about.
Now, what we typically do is that rather-- so you notice here, that there's actually no intercept.
If I actually fold that back down to one dimension, there's actually a is equal to zero, right?
If I go back to p is equal to 1, that would imply that Yi is, well, say, beta times x plus epsilon i.
And that's not good, I want to have an intercept.
And the way I do this, rather than writing a plus this, and you know, just have like an overload of notation, what I am actually doing is that I fold back.
I fold my intercept back into my x.
And so if I measure 20 variables, I'm going to create a 21st variable, which is always equal to 1.
OK, so you should need to think of x as being 1.
And then x1 xp.
And sorry, xp minus 1, I guess.
OK, and now this is an rp.
I'm always going to assume that the first one is 1.
I can always do that.
If I have a table of data-- if my data is given to me in an Excel spreadsheet-- and here I have the density that I measured on my data, and then maybe here I have the height, and here I have the wrist circumference.
And I have all these things.
All I have to do is to create another column here of ones, and I just put 1-1-1-1-1.
OK, that's all I have to do to create this guy.
Agreed?
And now my x is going to be just one of those rows.
So that's this is Xi, this entire row.
And this entry here is Yi.
So now, for my noise coefficients, I'm still going to ask for the same thing except that here, the covariance is not between x-- between one random variable and another random variable.
It's between a random vector and a random variable.
OK, how do I measure the covariance between a vector and a random variable?
[INAUDIBLE]
Yeah, so basically--
[INAUDIBLE]
Yeah, I mean, the covariance vector is equal to 0 is the same thing as [INAUDIBLE] equal to zero, but yeah, this is basically thought of entry-wise.
For each coordinate of x, I want that the covariance between epsilon and this coordinate of x is equal to 0.
So I'm just asking this for all coordinates.
Again, in most instances, we're going to think that epsilon is independent of x, and that's something we can understand without thinking about coordinates.
Yep?
[INAUDIBLE] like what if beta equals alpha [INAUDIBLE]?
I'm sorry, can you repeat the question?
I didn't hear
Is this the parameter of beta, a parameter?
Yeah, beta is the parameter we're looking for, right.
Just like it was the pair ab has become the whole vector of beta now
And what's [INAUDIBLE]??
Well, can you think of an intercept of a function that take-- I mean, there is one actually.
There's the one for which betas-- all the betas that don't correspond to the vector of all ones, so the intercept is really the weight that I put on this guy.
That's the beta that's going to come to this guy, but we don't really talk about intercept.
So if x lives in two dimensions, the way you want to think about this is you take a sheet of paper like that, so now I have points that live in three dimensions.
So let's say one direction here is x1.
This direction is x2, and this direction is y.
And so what's going to happen is that I'm going to have my points that live in this three dimensional space.
And what I'm trying to do when I'm trying to do a linear model for those guys-- when I assume a linear model.
What I assume is that there's a plane in those three dimensions.
So think of this guy as going everywhere, and there's a plane close to which all my points should be.
That's what's happening in two dimensional.
If you see higher dimensions then congratulations to you, but I can't.
But you know, you can definitely formalize that fairly easily mathematically and just talk about vectors.
So now here, if I talk about the least square error estimator, or just the least squares estimator of beta, it's simply the same thing as before.
Just like we said-- so remember, you should think of as beta as being both the pair a b generalized.
So we said, oh, we wanted to minimize the expectation of y minus a plus bx squared, right?
Now, so that's in-- for p is equal to 1.
Now for p lower than or equal to 2, we're just going to write it as y minus x transpose beta squared.
OK, so I'm just trying to minimize this quantity.
Of course, I don't have access to this, so what I'm going to do with them going to replace my expectation by an average.
So here I'm using the notation t because beta is the true one, and I don't want you to just-- so here, I have a variable t that's just moving around.
And so now I'm going to take the square of this thing.
And when I minimize this over all t in rp, the arc min, the minimum is attained at beta hat, which is my estimator.
OK?
So if I want to actually compute-- yeah?
I'm sorry, on the last slide did we require the expectation of [INAUDIBLE] to be zero?
You mean the previous slide?
Yes.
[INAUDIBLE]
So again, I'm just defining an estimator just like I would tell you, just take the estimator that has coordinates for everywhere
So I'm saying like [?
in that sign, ?]
we'll say the noise [?
terms ?]
we want to satisfy the covariance of that [?
side.
?]
We also want them to satisfy expectation of each [?
noise turn ?]
zero?
And so the answer is yes.
I was just trying to think if this was captured.
So it is not captured in this guy because this is just telling me that the expectation of epsilon i minus expectation of some i is equal to zero.
OK, so yes I need to have that epsilon has mean zero-- let's assume that expectation of epsilon is zero for this problem.
And we're going to need something about some sort of question about the variance being not equal to zero, right, but this is going to come up later.
So let's think for one second about doing the same approach as we did before.
Take the partial derivative with respect to the first coordinate of t, with respect to the second coordinate of t, with respect to the third coordinate of t, et cetera.
So that's what we did before.
We had two equations, and we reconciled them because it was fairly easy to solve, right?
But in general, what's going to happen is we're going to have a system of equations.
We're going to have a system of p equations, one for each of the coordinates of t. And we're going to have p unknowns, each coordinate of t. And so we're going to have the system to solve-- actually, i turns out it's going to be a linear system.
But it's not going to be something that we're going to be able to solve coordinate by coordinate.
It's going to be annoying to solve.
You know, you can guess that what's going to happen, right.
Here, it involved the covariance between x and epsilon, right.
That's what it involved to understand-- sorry, the correlation between x and y to understand how the solution of this problem was.
In this case, there's going to be only the covariance between x1 and y, x2 and y, x3, et cetera, all the way to xp and y.
There's also going to be all the cross covariances between xj and xk.
And so this is going to be a nightmare to solve, like, in this system.
And what we do is that we go on to using a matrix notation, so that when we take derivatives, we talk about gradients, and then we can invert matrices and solve linear systems in a somewhat formal manner by just saying that, if I want to solve the system ax equals b-- rather than actually solving this for each coordinate of x individually, I just say that x is equal to a inverse times.
So that's really why we're going to the equation one, because we have a formalism to write that x is the solution of the system.
I'm not telling you that this is going to be easy to solve numerically, but at least I can write it.
And so here's how it goes.
I have a bunch of vectors.
So what are my vectors, right?
So I have x1-- oh, by the way, I didn't actually mention that when I put the lowercase, when I put the subscript, I'm talking about the observation.
And when I put the superscript, I'm talking about the coordinates, right?
So I have x1, which is equal to x1, x1 [?
1, ?]
x 1p, x2, which is 1. x2, 1, x2 p, all the way to xn, which is 1, xn 1, x np.
All right, so those are n observed x's, and then I have another y1, y2, yn, that comes paired with those guys.
OK?
So the first thing is that I'm going to stack those guys into some vector that I'm going to call y.
So maybe I should put an arrow for the purpose of the blackboard, and it's just y1 to yn.
OK, so this is a vector in rn.
Now, if I want to stack those guys together, I can either create a long vector of size n times p, but the problem is that I lose the role of who's a coordinate and who's an observation.
And so it's actually nicer for me to just put those guys next to each other and create one new variable.
And so the way I'm going to do this is-- rather than actually stacking those guys like that, I'm getting their transpose and stack them as rows of a matrix.
OK, so I'm going to create a matrix, which here is denoted typically by-- I'm going to write x double bar.
And here, I'm going to actually just-- so since I'm taking those guys like this, the first column is going to be only ones.
And then I'm going to have-- well, x1, 1, [?
1, ?]
x1, p. And here, I'm going to have x n1, x np.
OK, so here the number of rows is n, and the number of columns is p. One row per observation, one column per coordinate.
And again, I make your life miserable because this really should be p minus 1 because I already used the first one for this guy.
I'm sorry about that.
It's a bit painful.
So usually we don't even write what's in there.
So we don't have to think about it.
Those are just vectors of size p. OK?
So now that I created this thing, I can actually just basically stack up all my models.
So Yi equals Xi transpose beta plus epsilon i for all i equal 1 to n. This transforms into-- this is equivalent to saying that the vector y is equal to the matrix x times beta plus a matrix, plus a vector epsilon, where epsilon is just epsilon 1 to epsilon n, right.
So I have just this system, which I write as a matrix, which really just consists in stacking up all these equations next to each other.
So now that I have this model-- this is the usual least squares model.
And here, when I want to write my least squares criterion in terms of matrices, right?
My least squares criterion, remember, was sum from i equal 1 to n of Yi minus Xi transposed beta squared.
Well, here it's really just the sum of the square of the coefficients of the vector y minus x beta.
So this is actually equal to the norm squared of y minus x beta square.
That's just the square.
Norm is, by definition, the sum of the square of the coordinates.
And so now I can actually talk about minimizing a norm squared, and here it's going to be easier for me to take derivatives.
All right, so we'll do that next time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
[INAUDIBLE] minus xi transpose t. I just pick whatever notation I want from a variable.
And let's say it's t. So that's the least squares estimator.
And it turns out that, as I said last time, it's going to be convenient to think of those things as matrices.
So here, I already have vectors.
I've already gone from one dimension, just real valued random variables through random vectors when I think of each xi, but if I start stacking them together, I'm going to have vectors and matrices that show up.
So the first vector I'm getting is y, which is just a vector where I have y1 to yn.
Then I have-- so that's a boldface vector.
Then I have x, which is a matrix where I have-- well, the first coordinate is always 1.
So I have 1, and then x1 xp minus 1, and that's-- sorry, x1 xp minus 1, and that's for observation 1.
And then I have the same thing all the way down for observation n. OK, everybody understands what this is?
So I'm just basically stacking up all the xi's.
So this i-th row is xi transpose.
I am just stacking them up.
And so if I want to write all these things to be true for each of them, all I need to do is to write a vector epsilon, which is epsilon 1 to epsilon n. And what I'm going to have is that y, the boldface vector, now is equal to the matrix x times the vector beta plus the vector epsilon.
And it's really just exactly saying what's there, because for 2-- so this is a vector, right?
This is a vector.
And what is the dimension of this vector?
n, so this is n observations.
And for all these-- for two vectors to be equal, I need to have all the coordinates to be equal, and that's exactly the same thing as saying that this holds for i equal 1 to n. But now, when I have this, I can actually rewrite the sum for t equals-- sorry, for i equals 1 to n of yi minus xi transpose beta squared, this turns out to be equal to the Euclidean norm of the vector y minus the matrix x times beta squared.
And I'm going to put a 2 here so we know we're talking about the Euclidean norm.
This just means this is the Euclidean norm.
That's the one we've seen before when we talked about chi squared-- that's the square norm is the sum of the square of the coefficients, and then I take a square root, but here I have an extra square.
So it's really just the sum of the square of the coefficients, which is this.
And here are the coefficients.
So then, that I write this thing like that, then minimizing-- so my goal here, now, is going to solve minimum over t in our p of y minus x times t2 squared.
And just like we did for one dimension, we can actually write optimality conditions for this.
I mean, this is a function.
So this is a function from rp to r. And if I want to minimize it, all I have to do is to take its gradient and set it equal to 0.
So minimum, set gradient to 0.
So that's where it becomes a little complicated.
Now I'm going to have to take the gradient of this norm.
It might be a little annoying to do.
But actually, what's nice about those things-- I mean, I remember that it was a bit annoying to learn.
I mean, it's just basically rules of calculus that you don't use that much.
But essentially, you can actually expend this norm.
And you will see that the rules are basically the same as in one dimension, you just have to be careful about the fact that matrices do not commute.
So let's expand this thing.
y minus xt squared-- well, this is equal to the norm of y squared plus the norm of x squared plus 2 times y transpose xt.
That's just expanding the square in more dimensions.
And this, I'm actually going to write as y squared plus-- so here, the norm squared of this guy, I always have that the norm of x squared is equal to x transpose x.
So I'm going to write this as x transpose x, so it's t transpose x transpose xt plus 2 times y transpose xt.
So now, if I'm going to take the gradient with respect to t, I have basically three terms, and each of them has some sort of a different nature.
This term is linear in t, and it's going to differentiate the same way that I differentiate a times x. I'm just going to keep the a.
This guy is quadratic.
t appears twice.
And this guy, I'm going to pick up a 2, and it's going to differentiate just like when I differentiate a times x squared.
It's 2 times ax.
And this guy is a constant with respect to t, so it's going to differentiate to 0.
So when I compute the gradient-- now, of course, all of these rules that I give you you can check by looking at the partial derivative with respect to each coordinate.
But arguably, it's much faster to know the rules of differentiability.
It's like if I gave you the function exponential x and I said, what is the derivative, and you started writing, well, I'm going to write exponential x plus h minus exponential ax divided by h and let h go to 0.
That's a bit painful
Why did you transpose your-- why does x have to be [INAUDIBLE]??
I'm sorry?
I was wondering why you times t times the [INAUDIBLE]?
The transpose of 2ab is b transpose a transpose.
If you're not sure about this, just make a and b have different size, and then you will see that there's some incompatibility.
I mean, there's basically only one way to not screw that one up, so that's easy to remember.
So if I take the gradient, then it's going to be equal to what?
It's going to be 0 plus-- we said here, this is going to differentiate like-- so think a times x squared.
So I'm going to have 2ax.
So here, basically, this guy is going to go to x transpose xt.
Now, I could have made this one go away, but that's the same thing as saying that my gradient is-- I can think of my gradient as being either a horizontal vector or a vertical vector.
So if I remove this guy, I'm thinking of my gradient as being horizontal.
If I remove that guy, I'm thinking of my gradient as being vertical.
And that's what I want to think of, typically-- vertical vectors, column vectors.
And then this guy, well, it's like these guys just think a times x.
So the derivative is just a, so I'm going to keep only that part here.
Sorry, I forgot a minus somewhere-- yeah, here.
Minus 2y transpose x.
And what I want is this thing to be equal to 0.
So t, the optimal t, is called beta hat and satisfies-- well, I can cancel the 2's and put the minus on the other side, and what I get is that x transpose xt is equal to y transpose x. Yeah, that's not working for me.
Yeah, that's because when I took the derivative, I still need to make sure-- so it's the same question of whether I want things to be columns or rows.
So this is not a column.
If I remove that guy, y transpose t is a row.
So I'm just going to take the transpose of this guy to make things work, and this is just going to be x transpose y.
And this guy is x transpose y so that I have columns.
So this is just the linear equation in t. And I have to solve it, so it's of the form some matrix times t is equal to another vector.
And so that's basically in your system.
And the way to solve it, at least formally, is to just take the inverse of the matrix on the left.
So if x transpose x is invertible, then-- sorry, that's beta hat is the t I want.
I get that beta hat is equal to x transpose x inverse x transpose y.
And that's the least squares estimator.
So here, I use this condition.
I want it to be invertible so I can actually write its inverse.
Here, I wrote, rank of x is equal to p. What is the difference?
Well, there's basically no difference.
Basically, here, I have to assume-- what is the size of the matrix x transpose x?
[INTERPOSING VOICES]
Yeah, so what is the size?
p by p
p by p. So this matrix is invertible if it's a rank p, if you know what rank means.
If you don't, that just rank p means that it's invertible.
So it's full rank and it's invertible.
And the rank of x transpose x is actually just the rank of x because this is the same matrix that you apply twice.
And that's all it's saying.
So if you're not comfortable with the notion of rank that you see here, just think of this condition just being the condition that x transpose x is invertible.
And that's all it says.
What it means for it to be invertible-- this was true.
We made no assumption up to this point.
If x is not invertible, it means that there might be multiple solutions to this equation.
In particular, for a matrix to not be invertible, it means that there's some vector v. So if x transpose x is not invertible, then this is equivalent to there exists a vector v, which is not 0, and such that x transpose xv is equal to 0.
That's what it means to not be invertible.
So in particular, if beta hat is a solution-- so this equation is sometimes called score equations, because the gradient is called the score, and so you're just checking if the gradient is equal to 0.
So if beta hat satisfies star, then so does beta hat plus lambda v for all lambda in the real line.
And the reason is because, well, if I start looking at-- what is x transpose x times beta hat plus lambda v?
Well, by linearity, this is just x transpose x beta hat plus lambda x transpose x times v. But this guy is what?
It's 0, just because that's what we assumed.
We assumed that x transpose xv was equal to 0, so we're left only with this part, which, by star, is just x transpose y.
So that means that x transpose x beta hat plus lambda v is actually equal to x transpose y, which means that there's another solution, which is not just beta hat, but any move of beta hat along this direction v by any size.
So that's going to be an issue, because you're looking for one estimator.
And there's not just one, in this case, there's many.
And so this is not going to be well-defined and you're going to have some issues.
So if you want to talk about the least squares estimator, you have to make this assumption.
What does it imply in terms of, can I think of p being too n, for example, in this case?
What happens if p is equal to 2n?
Well, then the rank of your matrix is only p/2
So the rank of your matrix is only p/2, so that means that this is actually not going to happen.
I mean, it's not only p/2, it's at most p/2.
It's at most the smallest of the two dimensions of your matrix.
So if your matrix is n times 2n, it's at most n, which means that it's not going to be full rank, so it's not going to be invertible.
So every time the dimension p is larger than the sample size, your matrix is not invertible, and you cannot talk about the least squares estimator.
So that's something to keep in mind.
And it's actually a very simple thing.
It's essentially saying, well, if p is lower than n, it means that you have more parameters to estimate than you have equations to estimate it.
So you have this linear system.
There's one equation per observation.
Each row, which was each observation, was giving me one equation.
But then the number of unknowns in this linear system is p, and so I cannot solve linear systems that have more unknowns than they have equations.
And so that's basically what's happening.
Now, in practice, if you think about what data sets look like these days, for example, people are trying to express some phenotype.
So phenotype is something you can measure on people-- maybe the color of your eyes, or your height, or whether you have diabetes or not, things like this, so things that are macroscopic.
And then they want to use the genotype to do that.
They want to measure your-- they want to sequence your genome and try to use this to predict whether you're going to be responsive to a drug or whether your r's are going to be blue, or something like this.
Now, the data sets that you can have-- people, maybe, for a given study about some sort of disease.
Maybe you will sequence the genome of maybe 100 people.
n is equal to 100. p is basically the number of genes they're sequencing.
This is of the order of 100,000.
So you can imagine that this is a case where n is much, much smaller than p, and you cannot talk about the least squares estimator.
There's plenty of them.
There's not just one line like that, lambda times v that you can move away.
There's basically an entire space in which you can move, and so it's not well-defined.
So at the end of this class, I will give you a short introduction on how you do this.
This actually represents more and more.
It becomes a more and more preponderant part of the data sets you have to deal with, because people just collect data.
When I do the sequencing, the machine allows me to sequence 100,000 genes.
I'm not going to stop at 100 because doctors are never going to have cohorts of more than 100 patients.
So you just collect everything you can collect.
And this is true for everything.
Cars have sensors all over the place, much more than they actually gather data.
There's data, there's-- we're creating, we're recording everything we can.
And so we need some new techniques for that, and that's what high-dimensional statistics is trying to answer.
So this is way beyond the scope of this class, but towards the end, I will give you some hints about what can be done in this framework because, well, this is the new reality we have to deal with.
So here, we're in a case where p's less than n and typically much smaller than n. So the kind of orders of magnitude you want to have is maybe p's of order 10 and n's of order 100, something like this.
So you can scale that, but maybe 10 times larger.
So maybe you cannot solve this guy b for b hat, but actually, you can talk about x times b hat, even if p is larger than n. And the reason is that x times b hat is actually something that's very well-defined.
So what is x times b hat?
Remember, I started with the model.
So if I look at this definition, essentially, what I had as the original thing was that the vector y was equal to x times beta plus the vector epsilon.
That was my model.
So beta is actually giving me something.
Beta is actually some parameter, some coefficients that are interesting.
But a good estimator for-- so here, it means that the observations that I have are of the form x times beta plus some noise.
So if I want to adjust the noise, remove the noise, a good candidate to do noise is x times beta hat.
x times beta hat is something that should actually be useful to me, which should be close to x times beta.
So in the one-dimensional case, what it means is that if I have-- let's say this is the true line, and these are my x's, so I have-- these are the true points on the real line, and then I have my little epsilon that just give me my observations that move around this line.
So this is one of epsilons, say epsilon i.
Then I can actually either talk-- to say that I recovered the line, I can actually talk about recovering the right intercept or recovering the right slope for this line.
Those are the two parameters that I need to recover.
But I can also say that I've actually found a set of points that's closer to being on the line that are closer to this set of points right here than the original crosses that I observed.
So if we go back to the picture here, for example, what I could do is say, well, for this point here-- there was an x here-- rather than looking at this dot, which was my observation, I can say, well, now that I've estimated the red line, I can actually just say, well, this point should really be here.
And actually, I can move all these dots so that they're actually on the red line.
And this should be a better value, something that has less noise than the original y value that I should see.
It should be close to the true value that I should be seeing without the extra noise.
So that's definitely something that could be of interest.
For example, in imaging, you're not trying to understand-- so when you do imaging, y is basically an image.
So think of a pixel image, and you just stack it into one long vector.
And what you see is something that should look like some linear combination of some feature vectors, maybe.
So there's people created a bunch of features.
They're called, for example, Gabor frames or wavelet transforms-- so just well-known libraries of variables x such that when you take linear combinations of those guys, this should looks like a bunch of images.
And what you want for your image-- you don't care what the coefficients of the image are in these bases that you came up with.
What you care about is the noise in the image.
And so you really want to get x beta.
So if you want to estimate x beta, well, you can use x beta hat.
What is x beta hat?
Well, since beta hat is x transpose x inverse x transpose y, this is x transpose.
That's my estimator for x beta.
Now, this thing, actually, I can define even if I'm not low rank.
So why is this thing interesting?
Well, there's a formula for this estimator, but actually, I can visualize what this thing is.
So let's assume, for the sake of illustration, that n is equal to 3.
So that means that y lives in a three-dimensional space.
And so let's say it's here.
And so I have my, let's say, y's here.
And I also have a plane that's given by the vectors x1 transpose x2 transpose, which is, by the way, 1-- sorry, that's not what I want to do.
I'm going to say that n is equal to 3 and that p is equal to 2.
So I basically have two vectors, 1, 1 and another one, let's assume that it's, for example, abc.
So those are my two vectors.
This is x1, and this is x2.
And those are my three observations for this guy.
So what I want when I minimize this, I'm looking at the point which can be formed as the linear combination of the columns of x, and I'm trying to find the guy that's the closest to y.
So what does it look like?
Well, the two points, 1, 1, 1 is going to be, say, here.
That's the point 1, 1, 1.
And let's say that abc is this point.
So now I have a line that goes through those two guys.
That's not really-- let's say it's going through those two guys.
And this is the line which can be formed by looking only at linear combination.
So this is the line of x times t for t in r2.
That's this entire line that you can get.
Why is it-- yeah, sorry, it's not just a line, I also have to have t, all the 0's thing.
So that actually creates an entire plane, which is going to be really hard for me to represent.
I don't know.
I mean, maybe I shouldn't do it in these dimensions.
So I'm going to do it like that.
So this plane here is the set of xt for t and r2.
So that's a two-dimensional plane, definitely goes to 0, and those are all these things.
So think of a sheet of paper in three dimensions.
Those are the things I can get.
So now, what I'm going to have as y is not necessarily in this plane.
y is actually something in this plane, x beta plus some epsilon.
y is x beta plus epsilon.
So I start from this plane, and then I have this epsilon that pushes me, maybe, outside of this plane.
And what least squares is doing is saying, well, I know that epsilon should be fairly small, so the only thing I'm going to be doing that actually makes sense is to take y and find the point that's on this plane that's the closest to it.
And that corresponds to doing an orthogonal projection of y onto this thing, and that's actually exactly x beta hat.
So in one dimension, just because this is actually a little hard-- in one dimension, so that's if p is equal to 1.
So let's say this is my point.
And then I have y, which is in two dimensions, so this is all on the plane.
What it does, this is my-- the point that's right here is actually x beta hat.
That's how you find x beta hat.
You take your point y and you project it on the linear span of the columns of x.
And that's x beta hat.
This does not tell you exactly what beta should be.
And if you know a little bit of linear algebra, it's pretty clear, because if you want to find beta hat, that means that you should be able to find the coordinates of a point in the system of columns of x.
And if those guys are redundant, there's not going to be unique coordinates for these guys, so that's why it's actually not easy to find.
But x beta hat is uniquely defined.
It's a projection.
Yeah?
And epsilon is the distance between the y and the--
No, epsilon is the vector that goes from-- so there's a true x beta.
That's the true one.
It's not clear.
I mean, x beta hat is unlikely to be exactly equal to x beta.
And then the epsilon is the one that starts from this line.
It's the vector that pushes you away.
So really, this is this vector.
That's epsilon.
So it's not a length.
The lengths of epsilon is the distance, but epsilon is just the actual vector that takes you from one to the other.
So this is all in two dimensions, and it's probably much clearer than what's here.
And so here, I claim that this x beta hat-- so from this picture, I implicitly claim that forming this operator that ticks y and maps it into this vector x times x transpose y, blah, blah, blah, this should actually be equal to the projection of y onto the linear span of the columns of x.
That's what I just drew for you.
And what it means is that this matrix must be the projection matrix.
So of course, anybody-- who knows linear algebra here?
OK, wow.
So what are the conditions that a projection matrix should be satisfying?
Squares through itself
Squares through itself, right.
If I project twice, I'm not moving.
If I keep on iterating projection, once I'm in the space I'm projecting onto, I'm not moving.
What else?
Do they have to be symmetric, maybe?
If it's an orthogonal projection
Yeah, so this is an orthogonal projection.
It has to be symmetric.
And that's pretty much it.
So from those things, you can actually get quite a bit of things.
But what's interesting is that if you actually look at the eigenvalues of this matrix, they should be either 0 or 1, essentially.
And they are 1 if the eigenvector associated is within this space, and 0 otherwise.
And so that's basically what you can check.
This is not an exercise in linear algebra, so I'm not going to go too much into those details.
But this is essentially what you want to keep in mind.
What's associated to orthogonal projections is Pythagoras theorem.
And that's something that's going to be useful for us.
What it's essentially telling is that if I look at this norm squared, it's equal to this norm squared-- sorry, this norm squared plus this norm squared is equal to this norm squared.
And that's something the norm of y squared.
So Pythagoras tells me that the norm of y squared is equal to the norm of x beta hat squared plus the norm of y minus x beta hat squared.
Agreed?
It's just because I have a straight angle here.
So that's this plus this is equal to this.
So now, to define this, I made no assumption.
Epsilon could be as wild.
I was just crossing my fingers that epsilon was actually small enough that it would make sense to project onto the linear span, because I implicitly assumed that epsilon did not take me all the way there, so that actually, it makes sense to project back.
And so for that, I need to somehow make assumptions that epsilon is well-behaved and that it's completely wild, that it's moving uniformly in all directions of the space.
There's no privileged direction where it's always going, otherwise, I'm going to make a systematic error.
And I need that those epsilons are going to average somehow.
So here are the assumptions we're going to be making so that we can actually do some statistical inference.
The first one is that the design matrix is deterministic.
So I started by saying the x-- I have xi, yi, and maybe they're independent.
Here, they are, but the xi's, I want to think as deterministic.
If they're not deterministic, it can condition on them, but otherwise, it's very difficult to think about this thing if I think of those entries as being random, because then I have the inverse of a random matrix, and things become very, very complicated.
So we're to think of those guys as being deterministic.
We're going to think of the model as being homoscedastic.
And actually, let me come back to this in a second.
Homoscedastic-- well, I mean, if you're trying to find the etymology of this word, "homo" means the same, "scedastic" means scaling.
So what I want to say is that the epsilons have the same scaling.
And since my third assumption is that epsilon is Gaussian, then essentially, what I'm going to want is that they all share the same sigma squared.
They're independent, so this is definitely in the identity covariance matrix.
And I want them to be centered, as well.
That means that there's no direction that I'm always privileging when I'm moving away from my plane there.
So these are important conditions.
It depends on how much inference you want to do.
If you want to write t-tests, you need all these assumptions.
But if you only want to write, for example, the fact that your least squares estimator is consistent, you really just need the fact that epsilon has variance sigma squared.
The fact that it's Gaussian won't matter, just like Gaussianity doesn't matter for a large number.
Yeah?
So the first assumption that x has to be deterministic, but I just made up this x1, x2--
x is the matrix
Yeah.
So most are random variables, right?
No, that's the assumption
OK.
So I mean, once we collect the data and put it in the matrix, it becomes deterministic.
So maybe I'm missing something
Yeah.
So this is for the purpose of the analysis.
I can actually assume that-- I look at my data, and I think of this.
So what is the difference between thinking of data as deterministic or thinking of it as random?
When I talked about random data, the only assumptions that I made were about the distribution.
I said, well, if my x is a random variable, I want it to have this variance and I want it to have, maybe, this distribution, things like this.
Here, I'm actually making an assumption on the values that I see.
I'm seeing that the value that you give me is-- the matrix is actually invertible.
x transpose x will be invertible.
So I've never done that before, assuming that some random variable-- assuming that some Gaussian random variable was positive, for example.
We don't do that, because there's always some probability that things don't happen if you make things at random.
And so here, I'm just going to say, OK, forget about-- here, it's basically a little stronger.
I start my assumption by saying, the data that's given to me will actually satisfy those assumptions.
And that means that I don't actually need to make some modeling assumption on this thing, because I'm actually putting directly the assumption I want to see.
So here, either I know sigma squared or I don't know sigma squared.
So is that clear?
So essentially, I'm assuming that I have this model, where this guy, now, is deterministic, and this is some multivariate Gaussian with mean 0 and covariance matrix identity of rn.
That's the model I'm assuming.
And I'm observing this, and I'm given this matrix x.
Where does this make sense?
You could say, well, if I think of my rows as being people and I'm collecting genes, it's a little intense to assume that I actually know, ahead of time, what I'm going to be seeing, and that those things are deterministic.
That's true, but it still does not prevent the analysis to go through, for one.
And second, a better example might be this imaging example that I described, where those x's are actually libraries.
Those are libraries of patterns that people have created, maybe from deep learning nets, or something like this.
But they've created patterns, and they say that all images should be representable as a linear combination of those patterns.
And those patterns are somewhere in books, so they're certainly deterministic.
Everything that's actually written down in a book is as deterministic as it gets.
Any questions about those assumptions?
Those are the things we're going to be working with.
There's only three of them.
One is about x.
Actually, there's really two of them.
I mean, this guy already appears here.
So there's two-- one on the noise, one on the x's.
That's it.
Those things allow us to do quite a bit.
They will allow us to-- well, that's actually-- they allow me to write the distribution of beta hat, which is great, because when I know the distribution of my estimator, I know it's fluctuations.
If it's centered around the true parameter, I know that it's going to be fluctuating around the true parameter.
And it should tell me what kind of distribution the fluctuations are.
I actually know how to build confidence intervals.
I know how to build tests.
I know how to build everything.
It's just like when I told you that asymptotically, the empirical variance was Gaussian with mean theta and standard deviation that depended on n, et cetera, that's basically the only thing I needed.
And this is what I'm actually getting here.
So let me start with this statement.
So remember, beta hat satisfied this, so I'm going to rewrite it here.
So beta hat was equal to x transpose x inverse x transpose y.
That was the definition that we found.
And now, I also know that y was equal to x beta plus epsilon.
So let me just replace y by x beta plus epsilon here.
Yeah?
Isn't it x transpose x inverse x transpose y?
Yes, x transpose.
Thank you.
So I'm going to replace y by x beta plus epsilon.
So that's-- and here comes the magic.
I have an inverse of a matrix, and then I have the true matrix, I have the original matrix.
So this is actually the identity times beta.
And now this guy, well, this is a Gaussian, because this is a Gaussian random vector, and I just multiply it by a deterministic matrix.
So we're going to use the rule that if I have, say, epsilon, which is n0 sigma, then b times epsilon is n0-- can somebody tell me what the covariance matrix of b epsilon is?
What is capital B in this case?
It's just a matrix.
And for any matrix, I mean any matrix that I can premultiply-- that I can postmultiply with epsilon.
Yeah?
b transpose b
b transpose?
Times b
And sigma is gone
Oh, times sigma, sorry
That's the matrix, right?
b transpose sigma b
Almost.
Anybody wants to take a guess at the last one?
I think we've removed all other possibilities.
It's b sigma b transpose.
So if you ever answered to the question, do you know Gaussian random vectors, but you did not know that, there's a gap in your knowledge that you need to fill, because that's probably the most important property of Gaussian vectors.
When you multiply them by matrices, you have a simple rule on how to update the covariance matrix.
So here, sigma is the identity.
And here, this is the matrix b that I had here.
So what this is is, basically, n, some multivariate n, of course.
Then I'm going to have 0.
And so what I need to do is b times the identity times b transpose, which is just b b transpose.
And what is it going to tell me?
It's x transpose x-- sorry, that's inverse-- inverse x transpose, and then the transpose of this guy, which is x x transpose x inverse transpose.
But this matrix is symmetric, so I'm actually not going to make the transpose of this guy.
And again, magic shows up.
Inverse times the matrix of those two guys cancel, and so this is actually equal to beta plus some n0 x transpose x inverse.
Yeah?
I'm a little lost on the [INAUDIBLE]..
So you define that as the b matrix, and what happens?
So I just apply this rule, right?
Yeah
So if I multiply a matrix by a Gaussian, then let's say this Gaussian had mean 0, which is the case of epsilon here, then the covariance matrix that I get is b times the original covariance matrix times b transpose.
So all I did is write this matrix times the identity times this matrix transpose.
And the identity, of course, doesn't play any role, so I can remove it.
It's just this matrix, then the matrix transpose.
And what happened?
So what is the transpose of this matrix?
So I used the fact that if I look at x transpose x inverse x transpose, and now I look at the whole transpose of this thing, that's actually equal 2.
And I use the rule that ab transpose is b transpose a transpose-- let me finish-- and it's x x transpose x inverse.
Yes?
I thought the-- for epsilon, it was sigma squared
Oh, thank you.
There's a sigma squared somewhere.
So this was sigma squared times the identity, so I can just pick up a sigma squared anywhere.
So here, in our case, so for epsilon, this is sigma.
Sigma squared times the identity, that's my covariance matrix.
You seem perplexed
It's just a new idea for me to think of a maximum likelihood estimator as a random variable
Oh, it should not be.
Any estimator is a random variable
Oh, yeah, that's a good point
[LAUGHS] And I have not told you that this was the maximum likelihood estimator just yet.
The estimator is a random variable.
There's a word-- some people use estimate just to differentiate the estimator while you're doing the analysis with random variables and the values when you plug in the numbers in there.
But then, of course, people use estimate because it's shorter, so then it's confusing.
So any questions about this computation?
Did I forget any other Greek letter along the way?
All right, I think we're good.
So one thing that it says-- and actually, thank you for pointing this out-- I said there's actually a little hidden statement there.
By the way, this answers this question.
Beta hat is of the form beta plus something that's centered, so it's indeed of the form Gaussian with mean beta and covariance matrix sigma squared x transpose x inverse.
So that's very nice.
As long as x transpose x is not huge, I'm going to have something that is close to what I want.
Oh, sorry, x transpose x inverse is not huge.
So there's a hidden claim in there, which is that least squares estimator is equal to the maximum likelihood estimator.
Why does the maximum likelihood estimator just enter the picture now?
We've been talking about regression for the past 18 slides.
And we've been talking about estimators.
And I just dumped on you the least squares estimator, but I never really came back to this thing that we know-- maybe the method of moments, or maybe the maximum likelihood estimator.
It turns out that those two things are the same.
But if I want to talk about a maximum likelihood estimator, I need to have a likelihood.
In particular, I need to have a density.
And so if I want a density, I have to make those assumptions, such as the epsilons have this Gaussian distribution.
So why is this the maximum likelihood estimator?
Well, remember, y is x transpose beta plus epsilon.
So I actually have a bunch of data.
So what is my model here?
Well, its the family of Gaussians on n observations with mean x beta, variance sigma squared identity, and beta lives in rp.
Here's my family of distributions.
That's the possible distributions for y.
And so in particular, I can write the density of y.
Well, what is it?
It's something that looks like p of x-- well, p of y, let's say, is equal to 1 over-- so now its going to be a little more complicated, but its sigma squared times 2 pi to the p/2 exponential minus norm of y minus x beta squared divided by 2 sigma squared.
So that's just the multivariate Gaussian density.
I just wrote it.
That's the density of a multivariate Gaussian with mean x beta and covariance matrix sigma squared times the identity.
That's what it is.
So you don't have to learn this by heart, but if you are familiar with the case where p is equal to 1, you can check that you recover what you're familiar with, and this makes sense as an extension.
So now, I can actually write my log likelihood.
How many observations do I have of this vector y?
Do I have n observations of y?
I have just one, right?
Oh, sorry, I shouldn't have said p, this is n. Everything is in dimension n. So I can think of either having n independent observations of each coordinate, or I can think of having just one observation of the vector y.
So when I write my log likelihood, it's just the log of the density at y.
And that's the vector y, which I can write as minus n/2 log sigma squared 2 pi minus 1 over 2 sigma squared norm of y minus x beta.
And that's, again, my boldface y.
And what is my maximum likelihood estimator?
Well, this guy does not depend on beta.
And this is just a constant factor in front of this guy.
So it's the same thing as just minimizing, because I have a minus sign, over all beta and rp.
y minus x beta squared, and that's my least squares estimator.
Is there anything that's unclear on this board?
Any question?
So all I used was-- so I wrote my log likelihood, which is just the log of this expression where y is my observation.
And that's indeed the observation that I have here.
And that was just some constant minus some constant times this quantity that depends on beta.
So maximizing this whole thing is the same thing as minimizing only this thing.
The minimizers are the same.
And so that tells me that I actually just have to minimize the squared norm to get my maximum likelihood estimator.
But this used, heavily, the fact that I could actually write exactly what my density was, and that when I took the log of this thing, I had exactly the square norm that showed up.
If I had a different density, if, for example, I assumed that my coordinates of epsilons were, say, iid double exponential random variables.
So it's just half of an exponential.
And the plus is half of an exponential on the negatives.
So if I said that, then this would not have the square norm that shows up.
This is really idiosyncratic to Gaussians.
If I had something else, I would have, maybe, a different norm here, or something different measures the difference between y and x beta.
And that's how you come up with other maximum likelihood estimators that leads to other estimators that are not the least squares-- maybe the least absolute deviation, for example, or this fourth movement, for example, that you suggested last time.
So I can come up with a bunch of different things, and they might be tied-- maybe I can come up from them from the same perspective that I came from the least squares estimator.
I said, let's just do something smart and check, then, that it's indeed the maximum likelihood estimator.
Or I could just start with the modeling on-- and check, then, what happens-- what was the implicit assumption that I put on my noise.
Or I could start with the assumption of the noise, compute the maximum likelihood estimator and see what it turns into.
So that was the first thing.
I've just proved to you the first line.
And from there, you can get what you want.
So all the other lines are going to follow.
So what is beta hat-- so for example, let's look at the second line, the quadratic risk.
Beta hat minus beta, from this formula, has a distribution, which is n n0, and then I have x transpose x inverse
Wouldn't the dimension be p on the board?
Sorry, the dimension of what?
Oh beta hat minus beta.
Isn't beta only a p dimensional?
Oh, yeah, you're right, you're right.
That was all p dimensional there.
Yeah.
So if b here, the matrix that I'm actually applying, has dimension p times n-- so even if epsilon was an n dimensional Gaussian vector, then b times epsilon is a p dimensional Gaussian vector now.
So that's how I switch from p to n-- from n to p. Thank you.
So you're right, this is beta hat minus beta is this guy.
And so in particular, if I look at the expectation of the norm of beta hat minus beta squared, what is it?
It's the expectation of the norm of some Gaussian vector.
And so it turns out-- so maybe we don't have-- well, that's just also a property of a Gaussian vector.
So if epsilon is n0 sigma, then the expectation of the norm of epsilon squared is just the trace of sigma.
Actually, we can probably check this by saying that this is the sum from j equal 1 to p of the expectation of beta hat j minus beta j squared.
Since beta j squared is the expectation-- beta j is the expectation of beta hat.
This is actually equal to the sum from j equal 1 to p of the variance of beta hat j, just because this is the expectation of beta hat.
And how do I read the variances in a covariance matrix?
There are just the diagonal elements.
So that's really just sigma jj.
And so that's really equal to-- so that's the sum of the diagonal elements of this matrix.
Let's call it sigma.
So that's equal to the trace of x transpose x inverse.
The trace is the sum of the diagonal elements of a matrix.
And I still had something else.
I'm sorry, this was sigma squared.
I forget it all the time.
So the sigma squared comes out.
It's there.
And so the sigma squared comes out because the trace is a linear operator.
If I multiply all the entries of my matrix by the same number, then all the diagonal elements are multiplied by the same number, so when I sum them, the sum is multiplied by the same number.
So that's for the quadratic risk of beta hat.
And now I need to tell you about x beta hat.
x beta hat was something that was actually telling me that that was the point that I reported on the red line that I estimated.
That was my x beta hat.
That was my y minus the noise.
Now, this thing here-- so remember, we had this line, and I had my observation.
And here, I'm really trying to measure this distance squared.
This distance is actually quite important for me because it actually shows up in the Pythagoras theorem.
And so you could actually try to estimate this thing.
So what is the prediction error?
So we said we have y minus x beta hat, so that's the norm of this thing we're trying to compute.
But let's write this for what it is for one second.
So we said that beta hat was x transpose x inverse extra transpose y, and we know that y is x transpose beta plus epsilon.
So let's write this-- x beta plus epsilon plus x.
And actually, maybe I should not write it.
Let me keep the y for what it is now.
So that means that I have, essentially, the identity of rn times y minus this matrix times y.
So I can factor y out, and that's the identity of rn minus x x transpose x inverse x transpose, the whole thing times y.
We call this matrix p because this was the projection matrix onto the linear span of the x's.
So that means that if I take a point x and I apply p times x, I'm projecting onto the linear span of the columns of x.
What happens if I do i minus p times x?
It's x minus px.
So if I look at the point on which-- so this is the point on which I project.
This is x. I project orthogonally to get p times x.
And so what it means is that this operator i minus px is actually giving me this guy, this vector here-- x minus p times x.
Let's say this is 0.
This means that this vector, I can put it here.
It's this vector here.
And that's actually the orthogonal projection of x onto the orthogonal complement of the span of the columns of x.
So if I project x, or if I look of x minus its projection, I'm basically projecting onto two orthogonal spaces.
What I'm trying to say here is that this here is another projection matrix p prime.
That is just the projection matrix onto the orthogonal-- projection onto orthogonal of column span of x. Orthogonal means the set of vectors that's orthogonal to everyone in this linear space.
So now, when I'm doing this, this is exactly what-- I mean, in a way, this is illustrating this Pythagoras theorem.
And so when I want to compute the norm of this guy, the norm squared of this guy, I'm really computing-- if this is my y now, this is px of y, I'm really controlling the norm squared of this thing.
So if I want to compute the norm squared-- so I'm almost there.
So what am I projecting here onto the orthogonal projector?
So here, y, now, I know that y is equal to x beta plus epsilon.
So when I look at this matrix p prime times y, It's actually p prime times x beta plus p prime times epsilon.
What's happening to p prime times x beta?
Let's look at this picture.
So we know that p prime takes any point here and projects it orthogonally on this guy.
But x beta is actually a point that lives here.
It's something that's on the linear span.
So where do all the points that are on this line get projected to?
The origin
The origin, to 0.
They all get projected to 0.
And that's because I'm basically projecting something that's on the column span of x onto its orthogonal.
So that's always 0 that I'm getting here.
So when I apply p prime to y, I'm really just applying p prime to epsilon.
So I know that now, this, actually, is equal to the norm of some multivariate Gaussian.
What is the size of this Gaussian?
What is the size of this matrix?
Well, I actually had it there.
It's i n, so it's n dimensional.
So it's some n dimensional with mean 0.
And what is the covariance matrix of p prime times epsilon?
p p transpose
Yeah, p prime p prime transpose, which we just said p prime transpose is p, so that's p squared.
And we see that when we project twice, it's as if we projected only once.
So here, this is n0 p prime p prime transpose.
That's the formula for the covariance matrix.
But this guy is actually equal to p prime times p prime, which is equal to p prime.
So now, what I'm looking for is the norm squared of the trace.
So that means that this whole thing here is actually equal to the trace.
Oh, did I forget again a sigma squared?
Yeah, I forgot it only here, which is good news.
So I should assume that sigma squared is equal to 1.
So sigma squared's here.
And then what I'm left with is sigma squared times the trace of p prime.
At some point, I mentioned that the eigenvalues of a projection matrix were actually 0 or 1.
The trace is the sum of the eigenvalues.
So that means that the trace is going to be an integer number as the number of non-0 eigenvalues.
And the non-0 eigenvalues are just the dimension of the space onto which I'm projecting.
Now, I'm projecting from something of dimension n onto the orthogonal of a space of dimension p. What is the dimension of the orthogonal of a space of dimension p when thought of space in dimension n?
[?
1.
?]
N minus p-- that's the so-called rank theorem, I guess, as a name.
And so that's how I get this n minus p here.
This is really just equal to n minus p. Yeah?
Here, we're taking the expectation of the whole thing
Yes, you're right.
So that's actually the expectation of this thing that's equal to that.
Absolutely.
But I actually have much better.
I know, even, that the norm that I'm looking at, I know it's going to be this thing.
What is going to be the distribution of this guy?
Norm squared of a Gaussian, chi squared.
So there's going to be some chi squared that shows up.
And the number of degrees of freedom is actually going to be also n minus p. And maybe it's actually somewhere-- yeah, right here-- n minus p times sigma hat squared over sigma squared.
This is my sigma hat squared.
If I multiply n minus p, I'm left only with this thing, and so that means that I get sigma squared times-- because they always forget my sigma squared-- I get sigma squared times this thing.
And it turns out that the square norm of this guy is actually exactly chi squared with n minus b degrees of freedom.
So in particular, so we know that the expectation of this thing is equal to sigma squared times n minus p. So if I divide both sides by n minus p, I'm going to have that something whose expectation is sigma squared.
And this something, I can actually compute.
It depends on y, and x that I know, and beta hat that I've just estimated.
I know what n is.
And pr's are the dimensions of my matrix x.
So I'm actually given an estimator whose expectation is sigma squared.
And so now, I actually have an unbiased estimator of sigma squared.
That's this guy right here.
And it's actually super useful.
So those are called the-- this is the normalized sum of square residuals.
These are called the residuals.
Those are whatever is residual when I project my points onto the line that I've estimated.
And so in a way, those guys-- if you go back to this picture, this was yi and this was xi transpose beta hat.
So if beta hat is close to beta, the difference between yi and xi transpose beta should be close to my epsilon i.
It's some sort of epsilon i hat.
Agreed?
And so that means that if I think of those as being epsilon i hat, they should be close to epsilon i, and so their norm should be giving me something that looks like sigma squared.
And so that's why it actually makes sense.
It's just magical that everything works out together, because I'm not projecting on the right line, I'm actually projecting on the wrong line.
But in the end, things actually work out pretty well.
There's one thing-- so here, the theorem is that this thing not only has the right expectation, but also has a chi squared distribution.
That's what we just discussed.
So here, I'm just telling you this.
But it's not too hard to believe, because it's actually the norm of some vector.
You could make this obvious, but again, I didn't want to bring in too much linear algebra.
So to prove this, you actually have to diagonalize the matrix p. So you have to invoke the eigenvalue decomposition and the fact that the norm is invariant by rotation.
So for those who are familiar with, what I can do is just look at the decomposition of p prime into ud u transpose where this is an orthogonal matrix, and this is a diagonal matrix of eigenvalues.
And when I look at the norm squared of this thing, I mean, I have, basically, the norm squared of p prime times some epsilon.
It's the norm of ud u transpose epsilon squared.
The norm of a rotation of a vector is the same as the norm of the vector, so this guy goes away.
This is not actually-- I mean, you don't have to care about this if you don't understand what I'm saying, so don't freak out.
This is really for those who follow.
What is the distribution of u transpose epsilon?
I take a Gaussian vector that has covariance matrix sigma squared times the [?
identity, ?]
and I basically rotate it.
What is its distribution?
Yeah?
The same
It's the same.
It's completely invariant, because the Gaussian think of all directions as being the same.
So it doesn't really matter if I take a Gaussian or a rotated Gaussian.
So this is also a Gaussian, so I'm going to call it epsilon prime.
And I am left with just the norm of epsilon primes.
So this is the sum of the dj's squared times epsilon j squared.
And we just said that the eigenvalues of p are either 0 or 1, because it's a projector.
And so here, I'm going to get only 0's and 1's.
So I'm really just summing a certain number of epsilon i squared.
So square root of standard Gaussians-- sorry, with a sigma squared somewhere.
And basically, how many am I summing?
Well, the n minus p, the number of non-0 eigenvalues of p prime.
So that's how it shows up.
When you see this, what theorem am I using here?
Cochran's theorem.
This is this magic book.
I'm actually going to dump everything that I'm not going to prove to you and say, oh, this is actually Cochran's.
No, Cochran's theorem is really just telling me something about orthogonality of things, and therefore, independence of things.
And Cochran's theorem was something that I used when I wanted to use what?
That's something I used just one slide before.
Student t-test, right?
I used Cochran's theorem to see that the numerator and the denominator of the student statistic were independent of each other.
And this is exactly what I'm going to do here.
I'm going to actually write a test to test, maybe, if the beta j's are equal to 0.
I'm going to form a numerator, which is beta hat minus beta.
This is normal.
And we know that beta hat has a Gaussian distribution.
I'm going to standardized by something that makes sense to me.
And I'm not going to go into details, because we're out of time.
But there's the sigma hat that shows up.
And then there's a gamma j, which takes into account the fact that my x's-- if I look at the distribution of beta, which is gone, I think-- yeah, beta is gone.
Oh, yeah, that's where it is.
The covariance matrix depends on this matrix x transpose x.
So this will show up in the variance.
In particular, diagonal elements are going to play a role here.
And so that's what my gammas are.
The gammas is the j's diagonal element of this matrix.
So we'll resume that on Tuesday, so don't worry too much if this is going too fast.
I'm not supposed to cover it, but just so you get a hint of why Cochran's theorem actually was useful.
So I don't know if we actually ended up recording.
I have your homework.
And as usual, I will give it to you outside.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So yes, before we start, this chapter will not be part of the midterm.
Everything else will be, so all the way up to goodness of fit tests.
And there will be some practice exams that will be posted in the recitation section of the course.
And that will be-- you will be working on.
So the recitation tomorrow will be a review session for the midterm.
I'll send an announcement by email.
So going back to our estimator, we showed that the least squares estimator in the case where we had some Gaussian observations.
So we had something that looked like this-- y was equal to some matrix x times beta plus some epsilon.
This was an equation that was happening in r to the n for n observations.
And then we wrote the least squares estimator beta hat.
And for the purpose from here on, you see that you have this normal distribution, this Gaussian p variant distribution.
That means that, at some point, we've made the assumption that epsilons were n and dimensional 0 identity of rn times sigma squared, which I kept on forgetting about last time.
I will try not to do that this time.
And so from this, we derived a bunch of properties of this least squares estimator, beta hat.
And in particular, the key thing that everything was built on was that we could write beta hat as the true unknown beta plus some multivariate Gaussian that was centered, but had a weird covariant structure.
So that was definitely p dimensional.
And it was sigma squared times x-- so that's x transpose x.
And that's inverse.
And the way we derived that was by having a lot of-- at least one cancellation between x transpose x and x transpose x inverse.
So this is the basis for inference in linear regression.
So in a way, that's correct, because what happened is that we used the fact that x beta hat-- once we have this beta, x beta hat is really just a projection of y onto the linear span of the columns of x, or the column span of x.
And so in particular, those things-- y minus x beta hats-- are called residuals.
So that's the vector of residuals.
What's the dimension of this vector?
n by 1
n by 1.
So those things, we can write as epsilon hat.
There's an estimate for this epsilon because we just put a hat on beta.
And from this one, we could actually build an unbiased estimator of sigma hat squared, and that was this guy.
And we showed that, indeed, the right normalization for this was n minus p, because y minus x beta hat to norm is actually a chi squared with n minus p degrees of freedom.
And so that's up to this scaling by sigma squared.
So that's what we came up with.
And something I told you, which follows from Cochran's theorem-- we did not go into details about this.
But essentially, since one of them corresponds to projection onto the linear span of the columns of x, and the other one corresponds to projection onto the orthogonal of this guy, and we're in a Gaussian case, things that are orthogonal are actually independent in a Gaussian case.
So from a geometric point of view, you can sort of understand everything.
You think of your subspace of the linear span of the x's, sometimes you project onto this guy, sometimes you project onto its orthogonal.
Beta hat corresponds to projection onto the linear span.
Epsilon hats correspond to a projection onto the orthogonal.
And those things tend to be independent, and that's what you have that beta hat is independent of sigma hat squared.
So it's really just a statement about two linear spaces being orthogonal with respect to each other.
So we left on this slide last time.
And what I claim is that this thing here is actually-- oh, yeah-- the other thing we want to use.
So that's good for beta hat.
But since we don't know what sigma squared is-- if we knew what sigma squared is, that would totally be enough for us.
But we also need this extra thing-- that sigma squared hat squared over sigma squared follows-- and there's an n minus p. This follows a chi squared with n minus p degrees of freedom.
And sigma hat squared is independent of beta hat.
So that's going to be something we need.
So that's useful if sigma squared if unknown.
And again, sometimes it might be known if you're using some sort of measurement device for which it's written on the side of the box.
So from these two things, we're going to be able to do inference And inference, we said there's three pillars to inference.
The first one is estimation, and we've been doing that so far.
We've constructed this least squares estimator, which happens to be the maximum likelihood estimator in the Gaussian case.
The two other things we do in inference are confidence intervals.
And we can do confidence intervals.
We're not going to do much because we're going to talk about their sort of cousin, which are tests.
And that's really where the statistical inference comes into.
And here, we're going to be interested in a very specific kind of test for linear regression.
And those are tests of the form beta j-- so the j-th coefficient of beta is equal to 0, and that's going to be our null hypothesis, versus h1 where beta j is, say, not equal to 0.
And for the purpose of regression, unless you have lots of domain-specific knowledge, it won't be beta j positive or beta j negative.
It's really non-0 that's interesting to you.
So why would I want to do this test?
Well, if I expand this thing where I have y is equal to x beta plus epsilon-- so what happens if I look, for example, at the first coordinates?
So I have that y is actually-- so say, y1 is equal to beta 1 plus beta 2 x 1.
Well, that's actually complicated.
Let me write it like this-- beta 0 plus beta 1 x1 plus beta p minus 1 xp minus 1 plus epsilon.
And that's true for all i's.
So this is beta 1 times 1.
That was our first coordinate.
So that's just expanding this-- going back to the scalar form rather than going to the matrix vector form.
That's what we're doing.
When I write y is equal to x beta plus epsilon, I assume that each of my y's can be represented as a linear combination of the x's, the first one being 1 plus some epsilon i.
Everybody agrees with this?
What does it mean for beta j to be equal to 0?
Yeah?
That xj's not important
Yeah, that xj doesn't even show up in this thing.
So if beta j is equal to 0, that means that, essentially, we can remove the j's coordinate, xj, from all observations.
So for example, I'm a banker, and I'm trying to predict some score-- let's call it y-- without the noise.
So I'm trying to predict what is going to be your score.
And that's something that should be telling me how likely you are to reimburse your loan on time or do you have late payments.
Or actually, maybe these days bankers are actually looking at how much late fees will I be collecting from you.
Maybe that's what they are more after rather than making sure that you reimburse everything.
So they're trying to maximize this number of late fees.
And they collect a bunch of things about you-- definitely your credit score, but maybe your zip code, profession, years of education, family status, a bunch of things.
One might be your shoe size.
And they want to know-- maybe shoe is actually a good explanation for how much fees they're going to be collecting from you.
But as you can imagine, this would be a controversial thing to bring, and people might want to test for their shoe size is a good idea.
And so they would just look at the j corresponding to shoe size and test whether shoe size should appear or not in this formula.
And that's essentially the kind of thing that people are going to do.
Now, if I do genomics and I'm trying to predict the size, the girth, of a pumpkin for a competition based on some available genomic data, then I can test whether gene j, which is called-- I don't know-- pea snap 24-- they always have these crazy names-- appears or not in this formula.
Is the gene pea snap 24 going to be important or not for the size of the final pumpkin?
So those are definitely the important things.
And definitely, we want to put beta j not equal to 0 as the alternative because that's where scientific discovery shows up.
And so to do that, well, we're in a Gaussian set-up, so we know that even if we don't know what sigma hat is, we can actually call for a t-test.
So how did we build the t-test in general?
Well, we had something that looked like-- so before, what we had was something that looked like theta hat was equal to theta plus some n0 and something that depended on n, maybe, something like this-- sigma squared over n. So that's what it looked like.
Now what we have is that beta hat is equal to beta plus some n, but this time, it's p variant, and then x transpose x inverse sigma squared.
So it's actually very similar, except that the matrix x transpose x inverse is now replacing just this number, 1/n, but it's playing the same role.
So in particular, this implies that for every j from 1 to p, what is the distribution of beta hat j?
Well, beta hat j is actually equal to-- so all I have to do-- so this is a system of p equations, and all I have to do is to read the j through.
So it's telling me here, I'm going to read beta hat j.
Here, I'm going to read beta j.
And here, I need to read, what is the distribution of the j-th coordinates of this guy?
So this is a Gaussian vector, so we need to understand what its definition is.
So how do I do this?
Well, the observation that's actually useful for this-- maybe I shouldn't use the word observation in a stats class, so let's call it claim.
The interesting claim is that if I have a vector-- let's call it v-- then vj is equal to v transpose ej where ej is the vector with 0, 0, 0, and then the 1 on the j-th coordinate, and then 0 elsewhere.
That's the j-th coordinate.
So that's the j-th vector of the canonical basis of rp.
So now that I have this form, I can see that, essentially, beta hat j is just ej transpose this np0 sigma squared x transpose x inverse.
And now, I know what the distribution of the inner product between a Gaussian and a deterministic vector is.
What is it?
It's a Gaussian.
So all I have to check is that ej transpose np0 sigma squared x transpose x inverse-- well, this is equal in distribution to what?
Well, this is going to be a one-dimensional thing.
A then your product is just a real number.
So it's going to be some Gaussian.
The mean is going to be 0 in a product with ej, which is 0.
What is the variance of this guy?
We actually used this, except that ej was not a vector, but it was a matrix.
So what we do is we, to see-- so the rule is that v transpose, say, n mu sigma is some n v transpose mu, and then v transpose sigma v. That's the rule for Gaussian vectors.
There's just the property of Gaussian vectors.
So what do we have here?
Well, ej plays the role of v. And sigma squared x transpose x inverse is the role of sigma.
So here, I'm left with ej transpose-- let me pull out the sigma squared here.
But this thing is, what happens if I take a matrix, I premultiply it by this vector ej, and I postmultiply it by this vector ej?
I'm claiming that this corresponds to only one single element of this matrix.
Which one is it?
j
j's diagonal element.
So this thing here is nothing but x transpose x inverse, and then the j-th diagonal element is jj.
Now, I cannot go any further.
x transpose x inverse can be a complicated matrix, and I do not know how to express jj's diagonal element much better than this.
Well, no, actually, I don't.
It involves basically all the coefficients.
Yeah?
[INAUDIBLE] second j come from, so I get why ej transpose [INAUDIBLE].. Where did the--
From this rule?
[INAUDIBLE]
So you always pre- and postmultiply when you talk about the covariance, because if you did not, it would be a vector and not a scalar, for one.
But in general, think of v as a matrix.
It's still true even in v is a matrix that's compatible with the premultiplying by some Gaussian.
Any other question?
Yeah?
When you say claim a vector v, what is vector v?
So for any vector v--

Any other question?
So now we've identified that the j-th coefficient of this Gaussian, which I can represent from the claim as ej transpose this guy, is also a Gaussian that's centered.
And its variance, now, is sigma squared times the j-th diagonal element of x transpose x inverse.
So the conclusion is that beta hat j is equal to beta j plus some n. And I'm going to emphasize the fact that now it's one-dimensional with mean 0 and covariance sigma squared x transpose x inverse inverse jj.
Now, if you look at the last line of the second board and the first line on the first board, those are basically the same thing.
Beta hat j is my theta hat.
Beta j is my theta.
And the variance sigma squared over n is now sigma squared times this [?
jj's ?]
element.
Now, the inverse suggests that it looks like the inverse of n. So those things are going to-- we're going to want to think of those guys as being some sort of 1/n kind of statement.
So from this, the fact that those two things are the same leads us to believe that we are now equipped to perform the task that we're trying to do, because under the null hypothesis, beta j is known it's equal to 0, so I can remove it.
And I have to deal with the sigma squared.
If sigma squared is known, then I can just perform a regular Gaussian test using Gaussian quintiles.
And if sigma squared is unknown, I'm going to just divide by sigma squared and multiply by sigma hat, and then I'm going to basically get my t-test.
Actually, for the purpose of your exam, I really suggest that you understand every single word I'm going to be saying now, because this is exactly the same thing that you're expected to know from other courses, because right now, I'm just going to apply exactly the same technique that we did for the single parameter estimation.
So what do we have now is that under h0, beta j is equal to 0.
Therefore, beta hat j follows some n0 sigma squared.
Just like I do in the slide, I'm going to call this gamma j.
So gamma j is this x transpose x inverse j-th diagonal element.
So that implies that beta hat j over sigma-- oh, was it a square root?
Yeah, sigma square root of gamma j follows some n0 1.
So I can form my test statistic, which to be reject if the absolute value of beta hat j divided by sigma square root gamma j is larger than what?
Can somebody tell me what I want this to be larger than to reject?
q alpha
q alpha.
Everybody agrees?
Of what?
Of this guy, where the standard notation that this is the quintile.
Everybody agrees?
It's alpha over 2 I think.
I think alpha's--
Alpha over 2.
So not everybody should be agreeing.
Thank you, you're the first one to disagree with yourself, which is probably good.
It's alpha over 2 because of the absolute value.
I want to just be away from this guy, and that's because I have-- so the alpha over 2-- the sanity check should be that h1 is beta j not equal to 0.
So that works if sigma is known, because I need to know sigma to be able to build my test.
So if sigma is unknown, well, I can tell you, use this test, but you're going to be like, OK, when I'm going to have to plug in some numbers, I'm going to be stuck.
But if sigma is unknown, we have sigma hat squared as an estimator.
So let me write sigma squared here.
So in particular, beta hat divided by sigma hat squared times square root gamma j-- something I can compute.
Sorry, that's beta hat j. I can compute that thing.
Agreed?
Now I have sigma hat j.
What I need to do is to be able to compute the distribution of this thing.
So I know the distribution of beta hat j over the square root of gamma j.
That's some Gaussian 0, 1.
I don't know exactly what the distribution of sigma hat j squared is, but what I know is that that was actually written, maybe, here is that n minus p sigma hat squared over sigma squared follows some chi squared with n minus p degrees of freedom, and that it's actually independent of beta hat j.
It's independent of beta hat, so it's independent of each of its coordinates.
That was part of your homework where you had to-- some of you were confused by the fact that-- I mean, if you're independent of some big thing, you're independent of all the smaller components of this big thing.
That's basically what you need to know.
And so now I can just write this as-- this is beta hat j divided by-- so now I want to make this guy appear, so it's beta hat j sigma squared over sigma squared-- sigma hat squared over sigma squared times n minus p divided by the square root of gamma j.
So that's what I want to see.
Yeah?
Why do you have to stick the hat in the denominator?
Shouldn't it be sigma?
Yeah, so I write this.
I decide to write this.
I could have put a Mickey Mouse here.
It just wouldn't make sense.
I just decided to take this thing

OK.
So now, let-- so I take this guy, and now, I'm going to rewrite it as something I want, because if you don't know what sigma is-- sorry, that's not sigm-- you mean the square?
Yeah
Oh, thank you.
Yes, that's correct.
[LAUGHS] OK, so you don't know what's sigma is, you replace it by sigma hat.
That's the most natural thing to do.
You just now want to find out what the distribution of this guy is.
So this is not exactly what I had.
To be able to get this, I need to divide by sigma squared-- sorry, I need to--
Square root
I'm sorry
Do we need a square root of the sigma hat [INAUDIBLE]
That's correct now.
And now I have that-- sorry, I should not write it like that.
That's not what I want.
What I want is this.
And to be able to get this guy, what I need is sigma over sigma hat square root.
And then I need to make this thing show up.
So I need to have this n minus p show up in the denominator.
So to be able to get it, I need to multiply the entire thing by the square root of n minus p. So this is just a tautology.
I just squeezed in what I wanted.
But now this whole thing here, this is actually of the form beta hat j divided by sigma over square root gamma j, and then divided by square root of sigma hat squared over sigma squared.
No, I don't want to divide it by square root of minus p, sorry.
And now it's times n minus p divided by n minus p. And what is the distribution of this thing here?
So I'm going to keep going here.
So the distribution of this thing here is what?
Well, this numerator, what is this distribution?
[INAUDIBLE]
Yeah, n0 1.
It's actually still written over there.
So that's our n0 1.
What is the distribution of this guy?
Sorry, I don't think you have color again.
So what is the distribution of this guy?
This is still written on the board
Chi squared
It's the chi squared that I have right here.
So that's a chi squared n minus p divided by n minus p degrees of freedom.
The only thing I need to check is that those two guys are independent, which is also what I have from here.
And so that implies that beta hat j divided by sigma hat square root of gamma j, what is the distribution of this guy?
[INTERPOSING VOICES]
n minus p. Was that crystal clear for everyone?
Was that so simple that it was boring to everyone?
OK, good.
That's where the point at which you should be.
So now I have this, I can read the quintiles of this guy.
So my test statistic becomes-- well, my rejection region, I reject if the absolute value of this new guy exceeds the quintile of order alpha over 2, but this time, of a tn minus p. And now you can actually see that the only difference between this test and that test, apart from replacing sigma by sigma hat, is that now I've moved from the quintiles of a Gaussian to the quintiles of a tn minus p. What's actually interesting, from this perspective, is that the tn minus p, we know, has heavier tails than the Gaussian, but if the number of degrees of freedom reaches, maybe, 30 or 40, they're virtually the same.
And here, the number of degrees of freedom is not given only by n, but it's n minus p. So if I have more and more parameters to estimate, this will result in some heavier, heavier tails, and that's just to account for the fact that it's harder and harder to estimate the variance when I have a lot of parameters.
That's basically where it's coming from.
So now let's move on to-- well, I don't know what because this is not working anymore.
So this is the simplest test.
And actually, if you run any statistical software for least squares, the output in any of them will look like this.
You will have a sequence of rows.
And you're going to have an estimate for beta 0, an estimate for beta 1, et cetera.
Here, you're going to have a bunch of things.
And on this row, you're going to have the value here, so that's going to be what's estimated by least squares.
And then the second line immediately is going to be, well, either the value of this thing-- so let's call it t. And then there's going to be the p value corresponding to this t. This is something that's just routinely coming out because-- oh, and then there's, of course, the last line for people who cannot read numbers that's really just giving you little stars.
They're not stickers, but that's close to it.
And that's just saying, well, I have three stars, I'm very significantly different from 0's.
If I have 2 stars, I'm moderately differently from 0.
And if I have 1 star, it means, well, just give me another $1,000 and I will sign that it's actually different from 0.
So that's basically the kind of outputs.
Everybody sees what I mean by that?
So what I mean, what I'm trying to emphasize here, is that those things are so routine when you run linear aggression, because people stuff in maybe-- even if you have 200 observations, you're going to stuff in maybe 20 variables-- p equals 20.
That's still a big number to interpret what's going on.
And it's nice for you if you can actually trim some fat out.
And so the problem is that when you start doing this, and then this, and then this, and then this, the probability that you make a mistake in your test, the probably that you erroneously reject the null here is 5%.
Here, it's 5%.
Here, it's 5%.
Here, it's 5%.
And at some point, if things happen with 5% chances and you keep on doing them over and over again, they're going to start to happen.
So you can see that basically what's happening is that you actually have an issue is that if you start repeating those tests, you might not be at 5% error at some point.
And so what do you do to prevent from that, if you want to test all those beta j's simultaneously, you have to do what's called the Bonferroni correction.
And the Bonferroni correction follows from what's called a union bound.
A union bound is actually-- so if you're a computer scientist, you're very familiar with it.
If you're a mathematician, that's just, essentially, the third axiom of probability that you see, that the probability of the union is less than the sum of the probabilities.
That's the union bound.
And you, of course, can generalize that to more than 2.
And that's exactly what you're doing here.
So let's see how we would want to perform Bonferroni correction to control the probability that they're all equal to 0 at the same time.
So recall-- so if I want to perform this test over there where I want to test h0, that beta j is equal to 0 for all j in some subset s. So think of s included in 1p.
You can think of it as being all of 1 of p if you want.
It really doesn't matter.
s is something that's given to you.
Maybe you want to test the subset of them, but maybe you want to test all of them.
Versus h1, beta j is not equal to 0 for some j in s. That's a test that tests all these things at once.
And if you actually look at this table all at once, implicitly, you're performing this test for all of the rows, for s equal 1 to p. You will do that.
Whether you like it or not, you will.
So now let's look at what the probability of type I error looks like.
So I want the probability of type 1 error, so that's the probably when h0 is true.
Well, so let me call psi j the indicator that, say, beta j hat over sigma hat square root gamma j exceeds q alpha over 2 of tn minus p. So we know that those are the tests that I perform.
Here, I just add this extra index j to tell me that I'm actually testing the j-th coefficient.
So what I want is the probability that under the null so that those are all equal to 0 that beta j's-- that I will reject to the alternative for one of them.
So that's psi 1 is equal to 1 or psi 2 is equal to 1, all the way to psi-- well, let's just say that this is the entire thing, because it's annoying.
I mean, you can check the slide if you want to do it more generally.
But psi p is equal to-- or, or-- everybody agrees that this is the probability of type I error?
So either I reject this one, or this one, or this one, or this one, or this one.
And that's exactly when I'm going to reject at least one of them.
So this is the probability of type I error.
And what I want is to keep this guy less than alpha.
But what I know is to control the probability that this guy is less than alpha, that this guy is less than alpha, that this guy is less than alpha.
In particular, if all these guys are disjoint, then this could really be the sum of all these probabilities.
So in the worst case, if psi j equals 1 intersected with psi k equals 1 is the empty set, so that means those are called disjoint sets.
You've seen this terminology in probability, right?
So if those sets are disjoint, for all of them, for all j different from k, then this probability-- well, let me write it as star-- then star is equal to, well, the probability under h0 that psi 1 is equal to 1 plus the probability under h0 that psi p is equal to 1.
Now, if I use this test with this alpha here, then this probability is equal to alpha.
This probability is also equal to alpha.
So the probably of type I error is actually not equal to alpha.
It's equal to?
p alpha
p alpha.
So what is the solution here?
Well, it's to run those guys not with alpha, but with alpha over p. And if they do this, then this guy is equal to alpha over p, this guy is equal to alpha over p. And so when I get those things, I get p times alpha over p, which is just alpha.
So all I do is, rather than running each of the tests with probability of error-- so that's a test at level alpha over p. That's actually very stringent.
If you think about it for 1 second, even if you have only 5 variables-- p equals 5-- and you started with the tests, you wanted to do your tests at 5%.
It forces you to do the test at 1% for each of those variables.
If you have 10 variables, I mean, that start to be very stringent.
So it's going to be harder and harder for you to conclude to the alternative.
Now, one thing I need to tell you is that here I said, if they are disjoint, then those probabilities are equal.
But if they are not disjoint, the union bound tells me that the probability of the union is less than the sum of the probabilities.
And so now I'm not exactly equal to alpha, but I'm bounded by alpha.
And that's why Bonferroni correction, people are not super comfortable with, is because, in reality, you never think that those tests are going to be giving you completely disjoint things.
I mean, why would it be?
Why would it be that if this guy is equal to 1, then all the other ones are equal to 0?
Why would it make any sense?
So this is definitely conservative, but the problem is that we don't know how to do much better.
I mean, we have a formula that tells you the probability of the union as some crazy sum that looks at all the intersection and all these little things.
I mean, it's the generalization of p of a or b is equal to p of a plus p of b minus probability of the intersection.
But if you start doing this for more than 2, it's super complicated.
The number of terms grows really fast.
But most importantly, even if you go here, you still need to control the probability of the intersection.
And those tests are not necessarily independent.
If they were independent, then that would be easy.
The probably of the intersection would be the product of the probabilities.
But those things are super correlated, and so it doesn't really help.
And so we'll see, when we talk about high-dimensional stats towards the end, that there's something called false discovery rate, which is essentially saying, listen, if I want to control this thing, if I really define my probability of type I error as this, I want to make sure that I never make this kind of error, I'm doomed.
This is just not going to happen.
But I can revise what my goals are in terms of errors that I make, and then I will actually be able to do.
And what people are looking at is false discovery rate.
And this is called family-wise error rate, which is a stronger thing to control.
So this trick that consists in replacing alpha by alpha over the number of times you're going to be performing your test, or alpha over the number of terms in your union, is actually called the Bonferroni correction.
And that's something you use when you have what's called-- another key word here is multiple testing, when you're trying to do multiple tests simultaneously.
And if s is not of p, well, you just divide by the number of tests that you are actually making.
So if s is of size k for some k less than p, you just divide alpha by k and not by p, of course.
I mean, you can always divide by p, but you're going to make your life harder for no reason.
Any question about Bonferroni correction?
So one thing that is maybe not as obvious as the test beta j equal to 0 versus beta j not equal to 0-- and in particular, what it means is that it's not going to come up as a software output without even you requesting it because this is so standard that it's just coming out.
But there's other tests that you might think of that might be more complicated and more tailored to your particular problem.
And those tests are of the form g times beta is equal to some lambda.
So let's see, the test we've just done, beta j equals 0 versus beta j not equal to 0, is actually equivalent to ej transpose beta equals 0 versus ej transpose beta not equal to 0.
That was our claim.
But now I don't have to stop here.
I don't have to multiply by a vector and test if it's equal to 0.
I can actually replace this by some general matrix g and replace this guy by some general vector lambda.
And I'm not telling you what the dimensions are because they're general.
I can take whatever I want.
Take your favorite matrix, as long as the right side of the matrix can be multiplying beta, and lambda, take it as the number of rows of g, and then you can do that.
I can always formulate this test.
What will this test encompass?
Well, those are kind of weird tests.
So you can think of things like, I want to test if beta 2 plus beta 3 are equal to 0, for example.
Maybe I want to test if beta 5 minus 2 beta 6 is equal to 23.
Well, that's weird.
But why would you want to test if beta 2 plus beta 3 is equal to 0?
Maybe you don't want to know if the-- you know that the effect of some gene is not 0.
Maybe you know that this gene affects this trait, but you want to know if the effect of this gene is canceled by the effect of that gene.
And this is the kind of stuff that you're going to be testing for that.
Now, this guy is much more artificial, and I don't have a bedtime story to tell you around this.
So those things can happen and can be much more complicated.
Now, here, notice that the matrix g has one row for both of the examples.
But if I want to test if those two things happen at the same time, then I actually can take a matrix.
Another matrix that can be useful is g equals the identity of rp and lambda is equal to 0.
What am I doing here in this case?
What is this test testing?
Sorry, this test.
Yeah?
Whether or not beta is 0
Yeah, we're testing if the entire vector beta is equal to 0, because g times beta is equal to beta, and we're asking whether it's equal to 0.
So the thing is, when you want to actually test if beta is equal to 0, you're actually testing if your entire model, everything you're doing in life, is just junk.
This is just telling you, actually, forget about this y is x beta plus epsilon.
y is really just epsilon.
There's nothing.
There's just some big noise with some big variants, and there's nothing else.
So turns out that the statistical software output that I wrote here spits out an answer to this question.
Just the last line, usually, is doing this test.
Does your model even make sense?
And it's probably for people to check whether they actually just mix their two data sets.
Maybe they're actually trying to predict-- I don't know-- some credit score from genomic data, and so just want to make sure, maybe, that's not the right thing.
So it turns out that the machinery is exactly the same as the one we've just taken.
So we actually start from here.
So let me pull this up.
So we start from here.
Beta hat was equal to beta plus this guy.
And the first thing we did was to say, well, beta j is equal to this thing because, well, beta j was just ej times beta.
So rather than taking ej here, let me just take g. Now, we said that for any vector-- well, that was trivial.
So the thing we need to know is, what is this thing?
Well, this thing here, what is this guy?
It's also normal and the mean is 0.
Again, that's just using properties of Gaussian vectors.
And what is the covariance matrix?
Let's call these guys sigma so that you can make an answer, you can formulate an answer.
So what is the distribution of-- what is the covariance of g times some Gaussian 0 sigma?
g sigma g transpose
g sigma g transpose, right?
So that's gx transpose x inverse g transpose.
Now, I'm not going to be able to go much farther.
I mean, I made this very acute observation that ej transpose the matrix times ej is the j-th angle element.
Now, if I have a general matrix, the price to pay is that I cannot just shrink this thing any further because I'm trying to be abstract.
And so I'm almost there.
The only thing that happened last time is that when this was ej under h0, 0, we knew that this was equal to 0 under the null.
But under the null, what is this equal to?
Lambda
Lambda, which I know.
I mean, I wrote my thing.
And in the couple instances I just showed you, including this one over there on top, lambda was equal to 0.
But in general, it can be any lambda.
But what's key about this lambda is that I actually know it.
That's the hypothesis I'm formulating.
So now I'm going to have to be a little more careful when I want to build the distribution of g beta hat.
I need to actually subtract this lambda.
So now we go from this, and we say, well, g beta hat minus lambda follows some np0 sigma squared g x transpose x inverse g transpose.
So that's true.
Let's assume-- let's go straight to the case when we don't know what sigma is.
So what I'm going to be interested in is g beta hat minus lambda divided by sigma hat.
And that's going to follow some Gaussian that has this thing, gx transpose x inverse g transpose.
So now, what did I do last time?
So clearly, the quintiles of this distribution is-- well, OK, what is the size of this distribution?
Well, I need to tell you that g is an-- what did I take here?
1 divided by sigma, not sigma hat
Oh, yeah, you're right.
So let me write it like this.
Well, let me write it like this-- sigma squared over sigma.
So let's forget about the size of g now.
Let's just think of any general g. When g was a vector, what was nice is that this guy was just the scalar number, just one number.
And so if I wanted to get rid of this in the right-hand side, all I had to do was to divide it by this thing.
We called it gamma j.
And we just had to divide by square root of gamma j, and that would be gone.
Now I have a matrix.
So I need to get rid of this matrix somehow because, clearly, the quintiles of this distribution are not going to be written in the back of a book for any value of g and any value of x.
So I need to standardize before I can read anything out of a table.
So how do we do it?
Well, we just form this guy here.
So what we know is that if-- so here's the claim, again, another claim about Gaussian vector.
If x follows some n0 sigma, then x transpose sigma inverse x follows some chi squared.
And here, it's going to depend on what is the dimension here.
So if I make this a k by k, a k-dimensional Gaussian vector, this is x squared k. Where have we used that before?
Yeah?
Wald's test
Wald's test, that's exactly what we used.
Wald's test had a chi squared that was showing up.
And the way we made it show up was by taking the asymptotic variance, taking its inverse, which, in this framework, was called--
Fisher
Fisher information.
And then we pre- and postmultiply by this thing.
So this is the key.
And so now, it tells me exactly, when I start from this guy that has this multivariate Gaussian, it tells me how to turn it into something that has a distribution which is pivotal.
Chi squared k is completely pivotal, does not depend on anything I don't know.
The way I go from here is by saying, well, now, I look at g beta hat minus lambda transpose, and now I need to look at the inverse of the matrix over there.
So it's gx transpose x inverse g inverse g beta hat minus lambda.
This guy is going to follow-- well, here, I need to actually divide by sigma in this case-- if g is k times p. So what I mean here is just that's the same k. The k that shows up is the number of constraints that I have in my tests.
So now, if I go from here to using sigma hat, the key thing to observe is that this guy is actually not a Gaussian.
I'm not going to have a student t-distribution that shows up.
So that implies that if I take the same thing, so now I just go from sigma to sigma hat, then this thing is of the form-- well, this chi squared k divided by the chi squared that shows up in the denominator of the t-distribution, which is square root of-- oh, I should not divide by sigma-- so this is sigma squared, right?
Yeah
So this is sigma squared.
So this is of the form divided by chi squared n minus p divided by n minus p. So that's the same denominator that I saw in my t-test.
The numerator has changed, though.
The numerator is now this chi squared and no longer a Gaussian.
But this distribution is actually pivotal, as long as we can guarantee that there's no hidden parameter in the correlation between the two chi squares.
So again, as all statements of independence in this class, I will just give it to you for free.
Those two things, I claim-- so OK, let's say admit these are independent.
We're almost there.
This could be a distribution that's pivotal.
But there's something that's a little unbalanced with it is that this guy is divided by its number of degrees of freedom, but this guy is not divided by its number of degrees of freedom.
And so we just have to make the extra step that if I divide this guy by k, and this guy is a chi squared divided by k, if I divide this guy by k, then I get this guy divided by k. And now it looks-- I mean, it doesn't change anything.
I've just divided by a fixed number.
But it just looks more elegant-- is the ratio of two independent chi squared that are individually divided by the number of degrees of freedom.
And this has a name, and it's called a Fisher or F-distribution.
So unlike William Gosset, who was not allowed to use his own name and used the name student, Fisher was allowed to use his own name, and that's called the Fisher distribution.
And the Fisher distribution has now 2 parameters, a set of 2 degrees of freedom-- 1 for the numerator and 1 for the denominator.
So F- of Fisher distribution-- so F is equal to the ratio of a chi squared p/p and a chi squared q/q.
So that's Fpq P-q where the 2 chi squareds are independent.
Is that clear what I'm defining here?
So this is basically what plays the role of t-distributions when you're testing more than 1 parameter at a time.
So you basically replace-- the normal that was in the numerator, you replace it by chi squared because you're testing if 2 vectors are simultaneously close.
And the way you do it is by looking at their squared norm.
And that's how the chi squared shows up.
Quick remark-- are those things really very different?
How can I relate a chi squared with a t-distribution?
Well, if t follows, say, a t-- I don't know, let's call it q.
So that means that t, let me look at-- t is some n01 divided by the square root of a chi squared q/q.
That's the distribution of t. So if I look at the square of the-- the distribution of t squared-- let me put it here-- well, that's the square of some n01 divided by chi squared q/q.
Agreed?
I just removed the square root here, and I took the square of the Gaussian.
But what is the distribution of a square of a Gaussian?
Chi squared with 1 degree
Chi squared with 1 degree of freedom.
So this is a chi squared with 1 degree of freedom.
And in particular, it's also a chi squared with 1 degree of freedom divided by 1.
So t-squared, in the end, has an F-distribution with 1 and q degrees of freedom.
So those two things are actually very similar.
The only thing that's going to change is that, since we're actually looking at, typically, absolute values of t when we do our tests, it's going to be exactly the same thing.
These quintiles of one guy are going to be, essentially, the square root of the quintiles of the other guy.
That's all it's going to be.
So if my test is psi is equal to the indicator that t exceeds q alpha over 2 of tq, for example, then it's equal to the indicator that t-squared exceeds q squared alpha over 2 tq, because I had the absolute value here, which is equal to the indicator that t squared is greater than q alpha over 2.
And now this time, it's an F1q.
So in a way, those two things belong to the same family.
They really are a natural generalization of each other.
I mean, at least the F-test is a generalization of the t-test.
And so now I can perform my test just like it's written here.
I just formed this guy, and then I perform against the quintile of an F-test.
Notice, there's no absolute value-- oh, yeah, I forgot, this is actually q alpha because the F-statistic is already positive.
So I'm not going to look between left and right, I'm just going to look whether it's too large or not.
So that's by definition.
So you can check-- if you look at a table for student and you look at a table for F1q, one it just going to-- you're going to have to move from one column to the other because you're going to have to move from alpha over 2 to alpha, but one is going to be squared root of the other one, just like the chi squared is the square of the Gaussian.
I mean, if you look at the chi squared 1 degree of freedom, you will see the same thing as the Gaussians.
So I'm actually going to start with the last one because you've been asking a few questions about why is my design deterministic.
So there's many answers.
Some are philosophical.
But one that's actually-- well, there's the one that says everything you cannot do if you don't have a condition-- if you don't have x, because all of the statements that we made here, for example, just the fact that this is chi squared, if those guys start to be random variables, then it's clearly not going to be a chi squared.
I mean, it cannot be chi squared when those guys are deterministic and when they are random.
I mean, things change.
So that's just maybe [INAUDIBLE] check statement.
But I think the one that really matters is that-- remember when we did the t-test, we had this gamma j that showed up.
Gamma j was playing the role of the variance.
So here, the variance, you never think of-- I mean, we'll talk about this in the Bayesian setup, but so far, we haven't thought of the variance as a random variable.
And so here, your x's really are the parameters of your data.
And the diagonal elements of x transpose x inverse actually tell you what the variance is.
So that's also one reason why you should think of your x as being a deterministic number.
They are, in a way, things that change the geometry of your problem.
They just say, oh, let me look at it from the perspective of x.
Actually, for that matter, we didn't really spend much time commenting on what is the effect of x onto gamma.
So remember, gamma j, so that was the variance parameter.
So we should try to understand what x's lead to big variance and what x's lead to small variance.
That would be nice.
Well, if this is the identity matrix-- let's say identity over n, which is the natural thing to look at, because we want this thing to scale like 1/n-- then this is just 1/n.
We're back to the original case.
Yes?
Shouldn't that be inverse?
Yeah, thank you.
x inverse, yes.
So if this is the identity, then, well, the inverse is-- let's say just this guy here is n times this guy.
So then the inverse is 1/n.
So in this case, that means that gamma j is equal to 1/n and we're back to the theta hat theta case, the basic one-dimensional thing.
What does it mean for a matrix for when I take its-- yeah, so that's of dimension p. But when I take its transpose-- so forget about the scaling by n right now.
This is just a matter of scaling things.
I can always multiply my x's so that I have this thing that shows up.
But when I have a matrix, if I look at x transpose x and I get something which is the identity, how do I call this matrix?
Orthonormal?
Orthogonal, yeah.
Orthonormal or orthogonal.
So you call this thing an orthogonal matrix.
And when it's an orthogonal matrix, what it means is that the-- so this matrix here, if you look at the matrix xx transpose, the entries of this matrix are the inner products between the columns of x.
That's what's happening.
You can write it, and you will see that the entries of this matrix are linear products.
If it's the identity, that means that you get some 1's and a bunch of 0's, it means that all the inner products between 2 different columns is actually 0.
What it means is that this matrix x is an orthonormal basis for your space.
The columns form an orthonormal basis.
So they're basically as far from each other as they can.
Now, if I start making those guys closer and closer, then I'm starting to have some issues.
x transpose x is not going to be the identity.
I'm going to start to have some non-0 entries.
But if they all remain of norm 1, then-- oh, sorry, so that's for the inverse.
So I first start putting some stuff here, which is non-0, by taking my x's.
Rather than having this, I move to this.
Now I'm going to start seeing some non-0 entries.
And when I'm going to take the inverse of this matrix, the diagonal elements are going to start to blow up.
Oh, sorry, the diagonals start to become smaller and smaller.
So when I take the inverse-- no, sorry, the diagonal limits are going to blow up.
And so what it means is that the variance is going to blow up.
And that's essentially telling you that if you get to choose your x's, you want to take them as orthogonal as you can.
But if you don't, then you just have to deal with it, and it will have a significant impact on your estimation performance.
And that's what, also, routinely, statistical software is going to spit out this value here for you.
And you're going to have-- well, actually square root of this value.
And it's going to tell you, essentially-- you're going to know how much randomness, how much variation you have in this particular parameter that you're estimating.
So if gamma j is large, then you're going to have wide confidence intervals and your tests are not going to reject very much.
And that's all captured by x.
That's what's important.
Everything, all of this, is completely captured by x.
Then, of course, there was the sigma squared that showed up here.
Actually, it was here, even in the definition of gamma j. I forgot it.
What is the sigma squared police doing?
And so this thing was here as well, and that's just exogenous.
It comes from the noise itself.
But there was this huge factor that came from the x's itself.
So let's go back, now, to reading this list in a linear fashion.
So I mean, you're MIT students, you've probably heard that correlation does not imply causation many times.
Maybe you don't know what it means.
If you don't, that's OK, you just have to know the sentence.
No, what it means is that it's done because I decided that something was going to be the x and that something else was going to be the y, that whatever thing I'm getting, it means that x implies y.
For example, even if I do genetics, genomics, or whatever, I mean, I implicitly assume that my genes are going to have an effect on my outside look.
I could be the opposite.
I mean, who am I to say?
I'm not a biologist.
I don't know.
I didn't open a biology book in 20 years.
So maybe, if I start hitting my head with a hammer, I'm going to have changing my genetic material.
Probably not, but that's why-- but causation definitely does not come from statistics.
So if you know that that's the different thing, it's actually going to-- it's not coming from there.
So actually, I remember, once, I put an exam to students, and there was an old data set from police expenditures, I think, in Chicago in the '60s.
And they were trying to understand-- no, it was on crime.
It was the crime data set.
And they were trying-- so the y variable was just the rate of crime, and the x's were a bunch of things, and one of them was police expenditures.
And if you rend the regression, you would find that the coefficient in front of police expenditure was a positive number, which means that if you increase police expenditures, that increases the crime.
I mean, that's what it means to have a positive coefficient.
Everybody agrees with this fact?
If beta j is 10, then it means that if I increase by $1 my police expenditure, I [INAUDIBLE] by 10 my crime, everything else being kept equal.
Well, there were, I think, about 80% of the students that were able to explain to me that if you give more money to the police, then the crime is going to raise.
Some people were like, well, police is making too much money, and they don't think about their work, and they become lazy.
And I mean, people were really coming up with some crazy things.
And what it just meant is that, no, it's not causation.
It's just, if you have more crime, you give more money to your police.
That's what's happening.
And that's all there is.
So just be careful when you actually draw some conclusions that causation is a very important thing to keep in mind.
And in practice, unless you have external sources of reason for causality-- for example, genetic material and physical traits, we agree upon what the direction of the arrow of causality is here.
There's places where you might not.
Now, finally, the normality on the noise-- everything we did today required normal Gaussian distribution on the noise.
I mean, it's everywhere.
There's some Gaussian, there's some chi squared.
Everything came out of Gaussian.
And for that, we needed this basic formula for inference, which we derived from the fact that the noise was Gaussian itself.
If we did not have that, the only thing we could write is, beta hat is this number, or this vector.
We would not be able to say, the fluctuations of beta hat are this guy.
We would not be able to do tests.
We would not be able to build, say, confidence regions or anything.
And so this is an important condition that we need, and that's what statistical software assumes by default.
But we now have a recipe on how to do those tests.
We can do it either visually, if we really want to conclude that, yes, this is Gaussian, using our normal Q-Q plots.
And we can also do it using our favorite tests.
What test should I be using to test that?
With two names?
Yeah?
Normal [INAUDIBLE]
Not the 2 Russians.
So I want a Russian and a Scandinavian person for this one.
What's that?
Lillie-something?
Yeah, Lillie-something.
So Kolmogorov Lillie-something test.
And [LAUGHS] so it's the Kolmogorov Lilliefors test.
And because I'm testing if there Gaussian, and I'm actually not really making any-- I don't need to know what the variance is.
The mean is 0.
We saw that at the beginning.
It's 0 by construction, so we actually don't need to think about the mean being 0 itself.
This just happens to be 0.
So we know that it's 0, but the variance, we don't know.
So we just want to know if it belongs to the family of Gaussians, and so we need to Kolmogorov Lilliefors for that.
And that's also one of the thing that's spit out by statistical software by default.
When you run a linear regression, actually, it spits out both Kolmogorov-Smirnov and Kolmogorov Lilliefors, probably contributing to the widespread use of Kolmogorov-Smirnov when you really shouldn't.
So next time, we will talk about more advanced topics on regression.
But I think I'm going to stop here for today.
So again, tomorrow, sometime during the day, at least before the recitation, you will have a list of practice exercises that will be posted.
And if you go to the optional recitation, you will have someone solving them
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MITOpenCourseWare@OCW.MIT.edu
So today WE'LL actually just do a brief chapter on Bayesian statistics.
And there's entire courses on Bayesian statistics, there's entire books on Bayesian statistics, there's entire careers in Bayesian statistics.
So admittedly, I'm not going to be able to do it justice and tell you all the interesting things that are happening in Bayesian statistics.
But I think it's important as a statistician to know what it is, how it works, because it's actually a weapon of choice for many practitioners.
And because it allows them to incorporate their knowledge about a problem in a fairly systematic manner.
So if you look at like, say the Bayesian statistics literature, it's huge.
And so here I give you sort of a range of what you can expect to see in Bayesian statistics from your second edition of a traditional book, something that involves computation, some things that involve risk thinking.
And there's a lot of Bayesian thinking.
There's a lot of things that you know talking about sort of like philosophy of thinking Bayesian.
This book, for example, seems to be one of them.
This book is definitely one of them.
This one represents sort of a wide, a broad literature on Bayesian statistics, for applications for example, in social sciences.
But even in large scale machine learning, there's a lot of Bayesian statistics happening, particular using something called Bayesian parametrics, or hierarchical Bayesian modeling.
So we do have some experts at MIT in the c-cell.
Tamara Broderick for example, is a person who does quite a bit of interesting work on Bayesian parametrics.
And if that's something you want to know more about, I urge you to go and talk to her.
So before we go into more advanced things, we need to start with what is the Bayesian approach.
What do Bayesians do, and how is it different from what we've been doing so far?
So to understand the difference between Bayesians and what we've been doing so far is, we need to first put a name on what we've been doing so far.
It's called frequentist statistics.
Which usually Bayesian versus frequentist statistics, by versus I don't mean that there is naturally in opposition to them.
Actually, often you will see the same method that comes out of both approaches.
So let's see how we did it, right.
The first thing, we had data.
We observed some data.
And we assumed that this data was generated randomly.
The reason we did that is because this would allow us to leverage tools from probability.
So let's say by nature, measurements, you do a survey, you get some data.
Then we made some assumptions on the data generating process.
For example, we assumed they were iid.
That was one of the recurring things.
Sometimes we assume it was Gaussian.
If you wanted to use say, T-test.
Maybe we did some nonparametric statistics.
We assume it was a smooth function or maybe linear regression function.
So those are our modeling.
And this was basically a way to say, well, we're not going to allow for any distributions for the data that we have.
But maybe a small set of distributions that indexed by some small parameters, for example.
Or at least remove some of the possibilities.
Otherwise, there's nothing we can learn.
And so for example, this was associated to some parameter of interest, say data or beta in the regression model.
Then we had this unknown problem and this unknown thing, a known parameter.
And we wanted to find it.
We wanted to either estimate it or test it, or maybe find a confidence interval for the subject.
So, so far I should not have said anything that's new.
But this last sentence is actually what's going to be different from the Bayesian part.
And particular, this unknown but fixed things is what's going to be changing.
In the Bayesian approach, we still assume that we observe some random data.
But the generating process is slightly different.
It's sort of a two later process.
And there's one process that generates the parameter and then one process that, given this parameter generates the data.
So what the first layer does, nobody really believes that there's some random process that's happening, about generating what is going to be the true expected number of people who turn their head to the right when they kiss.
But this is actually going to be something that brings us some easiness for us to incorporate what we call prior belief.
We'll see an example in a second.
But often, you actually have prior belief of what this parameter should be.
When we, say least squares, we looked over all of the vectors in all of R to the p, including the ones that have coefficients equal to 50 million.
Those are things that we might be able to rule out.
We might be able to rule out that on a much smaller scale.
For example, well I'm not an expert on turning your head to the right or to the left.
But maybe you can rule out the fact that almost everybody is turning their head in the same direction, or almost everybody is turning their head to another direction.
So we have this prior belief.
And this belief is going to play say, hopefully less and less important role as we collect more and more data.
But if we have a smaller amount of data, we might want to be able to use this information, rather than just shooting in the dark.
And so the idea is to have this prior belief.
And then, we want to update this prior belief into what's called the posterior belief after we've seen some data.
Maybe I believe that there's something that should be in some range.
But maybe after I see data, it's comforting me in my beliefs.
So I'm actually having maybe a belief that's more.
So belief encompasses basically what you think and how strongly you think about it.
That's what I call belief.
So for example, if I have a belief about some parameter theta, maybe my belief is telling me where theta should be and how strongly I believe in it, in the sense that I have a very narrow region where theta could be.
The posterior beliefs, as well, you see some data.
And maybe you're more confident or less confident about what you've seen.
Maybe you've shifted your belief a little bit.
And so that's what we're going to try to see, and how to do this in a principal manner.
To understand this better, there's nothing better than an example.
So let's talk about another stupid statistical question.
Which is, let's try to understand p. Of course, I'm not going to talk about politics from now on.
So let's talk about p, the proportion of women in the population.
And so what I could do is to collect some data, X1, Xn and assume that they're Bernoulli with some parameter, p unknown.
So p is in 0, 1.
OK, let's assume that those guys are iid.
So this is just an indicator for each of my collected data, whether the person I randomly sample is a woman, I get a one.
If it's a man, I get a zero.
Now the question is, I sample these people randomly.
I do you know their gender.
And the frequentist approach was just saying, OK, let's just estimate p hat being Xn bar.
And then we could do some tests.
So here, there's a test.
I want to test maybe if p is equal to 0.5 or not.
That sounds like a pretty reasonable thing to test.
But we want to also maybe estimate p. But here, this is a case where we definitely prior belief of what p should be.
We are pretty confident that p is not going to be 0.7.
We actually believe that we should be extremely close to one half, but maybe not exactly.
Maybe this population is not the population in the world.
But maybe this is the population of, say some college and we want to understand if this college has half women or not.
Maybe we know it's going to be close to one half, but maybe we're not quite sure.
We're going to want to integrate that knowledge.
So I could integrate it in a blunt manner by saying, discard the data and say that p is equal to one half.
But maybe that's just a little too much.
So how do I do this trade off between adding the data and combining it with this prior knowledge?
In many instances, essentially what's going to happen is this one half is going to act like one new observation.
So if you have five observations, this is just the sixth observation, which will play a role.
If you have a million observations, you're going to have a million and one.
It's not going to play so much of a role.
That's basically how it goes.
But, definitely not always because we'll see that if I take my prior to be a point minus one half here, it's basically as if I was discarding my data.
So essentially, there's also your ability to encompass how strongly you believe in this prior.
And if you believe infinitely more in the prior than you believe in the data you collected, then it's not going to act like one more observation.
The Bayesian approach is a tool to one, include mathematically our prior.
And our prior belief into statistical procedures.
Maybe I have this prior knowledge.
But if I'm a medical doctor, it's not clear to me how I'm going to turn this into some principal way of building estimators.
And the second goal is going to be to update this prior belief into a posterior belief by using the data.
How do I do this?
And at some point, I sort of suggested that there's two layers.
One is where you draw the parameter at random.
And two, once you have the parameter, conditionless parameter, you draw your data.
Nobody believed this actually is happening, that nature is just rolling dice for us and choosing parameters at random.
But what's happening is that, this idea that the parameter comes from some random distribution actually captures, very well, this idea that how you would encompass your prior.
How would you say, my belief is as follows?
Well here's an example about p. I'm 90% sure that p is between 0.4 and 0.6.
And I'm 95% sure that p is between 0.3 and 0.8.
So essentially, I have this possible value of p. And what I know is that, there's 90% here between 0.4 and 0.6.
And then I have 0.3 and 0.8.
And I know that I'm 95% sure that I'm in here.
If you remember, this sort of looks like the kind of pictures that I made when I had some Gaussian, for example.
And I said, oh here we have 90% of the observations.
And here, we have 95% of the observations.
So in a way, if I were able to tell you all those ranges for all possible values, then I would essentially describe a probability distribution for p. And what I'm saying is that, p is going to have this kind of shape.
So of course, if I tell you only two twice this information that there's 90% I'm here, and I'm between here and here.
And 95%, I'm between here and here, then there's many ways I can accomplish that, right.
I could have something that looks like this, maybe.
It could be like this.
There's many ways I can have this.
Some of them are definitely going to be mathematically more convenient than others.
And hopefully, we're going to have things that I can parameterize very well.
Because if I tell you this is this guy, then there's basically one, two three, four, five, six, seven parameters.
So I probably don't want something that has seven parameters.
But maybe I can say, oh, it's a Gaussian and I all I have to do is to tell you where it's centered and what the standard deviation is.
So the idea of using this two layer thing, where we think of the parameter p as being drawn from some distribution, is really just a way for us to capture this information.
Our prior belief being, well there's this percentage of chances that it's there.
But the percentage of this chance, I'm not I'm deliberately not using probability here.
So it's really a way to get close to this.
That's why I say, the true parameter is not random.
But the Bayesian approach does as if it was random.
And then, just spits out a procedure out of this thought process, this thought experiment.
So when you practice Bayesian statistics a lot, you start getting automatisms.
You start getting some things that you do without really thinking about it.
just like when you you're a statistician, the first thing you do is, can I think of this data as being Gaussian for example?
When you're Bayesian you're thinking about, OK I have a set of parameters.
So here, I can describe my parameter as being theta in general, in some big space parameter of theta.
But what spaces did we encounter?
Well, we encountered the real line.
We encountered the interval 0, 1 for Bernoulli's And we encountered some of the positive real line for exponential distributions, etc.
And so what I'm going to need to do, if I want to put some prior on those spaces, I'm going to have to have a usual set of tools for this guy, usual set of tools for this guy, usual sort of tools for this guy.
And by usual set of tools, I mean I'm going to have to have a family of distributions that's supported on this.
So in particular, this is the speed in which my parameter that I usually denote by p for Bernoulli lives.
And so what I need is to find a distribution on the interval 0, 1 just like this guy.
The problem with the Gaussian is that it's not on the interval 0, 1.
It's going to spill out in the end.
And it's not going to be something that works for me.
And so the question is, I need to think about distributions that are probably continuous.
Why would I restrict myself to discrete distributions that are actually convenient and for Bernoulli, one that's actually basically the main tool that everybody is using is the so-called beta distribution.
So the beta distribution has two parameters.
So x follows a beta with parameters a and b if it has a density, f of x is equal to x to the a minus 1.
1 minus x to the b minus 1, if x is in the interval 0, 1 and 0 for all other x's.
OK?
Why is that a good thing?
Well, it's a density that's on the interval 0, 1 for sure.
But now I have these two parameters and a set of shapes that I can get by tweaking those two parameters is incredible.
It's going to be a unimodal distribution.
It's still fairly nice.
It's not going to be something that goes like this and this.
Because if you think about this, what would it mean if your prior distribution of the interval 0, 1 had this shape?
It would mean that, maybe you think that p is here or maybe you think that p is here, or maybe you think that p is here.
Which essentially means that you think that p can come from three different phenomena.
And there's other models that are called mixers for that, that directly account for the fact that maybe there are several phenomena that are aggregated in your data set.
But if you think that your data set is sort of pure, and that everything comes from the same phenomenon, you want something that looks like this, or maybe looks like this, or maybe is sort of symmetric.
You want to get all this stuff.
Maybe you want something that says, well if I'm talking about p being the probability of the proportion of women in the whole world, you want something that's probably really spiked around one half.
Almost the point math, because you know let's agree that 0.5 is the actual number.
So you want something that says, OK maybe I'm wrong.
But I'm sure I'm not going to be really that way off.
So you want something that's really pointy.
But if it's something you've never checked, and again I can not make references at this point, but something where you might have some uncertainty that should be around one half.
Maybe you want something that a little more allows you to say, well, I think there's more around one half.
But there's still some fluctuations that are possible.
And in particular here, I talk about p, where the two parameters a and b are actually the same.
I call them a.
One is called scale.
The other one is called shape.
Oh sorry, this is not a density.
So it actually has to be normalized.
When you integrate this guy, it's going to be some function that depends on a and b, actually depends on this function through the beta function.
Which is this combination of gamma function, so that's why it's called beta distribution.
That's the definition of the beta function when you integrate this thing anyway.
You just have to normalize it.
That's just a number that depends on the a and b.
So here, if you take a equal to b, you have something that essentially is symmetric around one half.
Because what does it look like?
Well, so my density f of x, is going to be what?
It's going to be my constant times x, times one minus x to a minus one.
And this function, x times 1 minus x looks like this.
We've drawn it before.
That was something that showed up as being the variance of my Bernoulli.
So we know it's something that takes its maximum at one half.
And now I'm just taking a power of this guy.
So I'm really just distorting this thing into some fairly symmetric manner.
This distribution that we actually take for p. I assume that p, the parameter, notice that this is kind of weird.
First of all, this is probably the first time in this entire course that something has a distribution when it's actually a lower case letter.
That's something you have to deal with, because we've been using lower case letters for parameters.
And now we want them to have a distribution.
So that's what's going to happen.
This is called the prior distribution.
So really, I should write something like f of p is equal to a constant times p, 1 minus p, to the n minus 1.
Well no, actually I should not because then it's confusing.
One thing in terms of notation that I'm going to write, when I have a constant here and I don't want to make it explicit.
And we'll see in a second why I don't need to make it explicit.
I'm going to write this as f of x is proportional to x 1 minus x to the n minus 1.
That's just to say, equal to some constant that does not depend on x times this thing.
So if we continue with our experiment where I'm drawing this data, X1 to Xn, which is Bernoulli p, if p has some distribution it's not clear what it means to have a Bernoulli with some random parameter.
So what I'm going to do is, then I'm going to first draw my p. Let's say I get a number, 0.52.
And then, I'm going to draw my data conditionally on p. So here comes the first and last flowchart of this class.
So nature first draws p. p follows some data on a, a.
Then I condition on p. And then I draw X1, Xn that are iid, Bernoulli p. Everybody understand the process of generating this data?
So you first draw a parameter, and then you just flip those independent biased coins with this particular p. There's this layered thing.
Now conditionally p, right so here I have this prior about p which was the thing.
So this is just the thought process again, it's not anything that actually happens in practice.
This is my way of thinking about how the data was generated.
And from this, I'm going to try to come up with some procedure.
Just like, if your estimator is the average of the data, you don't have to understand probability to say that my estimator is the average of the data.
Anyone outside this room understands that the average is a good estimator for some average behavior.
And they don't need to think of the data as being a random variable, et cetera.
So same thing, basically.
In this case, you can see that the posterior distribution is still a beta.
What it means is that, I had this thing.
Then, I observed my data.
And then, I continue and here I'm going to update my prior into some posterior distribution, pi.
And here, this guy is actually also a beta.
My posterior distribution, p, is also a beta distribution with the parameters that are on this slide.
And I'll have the space to reproduce them.
So I start the beginning of this flowchart as having p, which is a prior.
I'm going to get some observations and then, I'm going to update what my posterior is.
This posterior is basically something that's, in business statistics was beautiful is as soon as you have this distribution, it's essentially capturing all the information about the data that you want for p. And it's not just the point.
It's not just an average.
It's actually an entire distribution for the possible values of theta.
And it's not the same thing as saying, well if theta hat is equal to Xn bar, in the Gaussian case I know that this is some mean, mu.
And then maybe it has varying sigma squared over n. That's not what I mean by, this is my posterior distribution.
This is not what I mean.
This is going to come from this guy, the Gaussian thing and the central limit theorem.
But what I mean is this guy.
And this came exclusively from the prior distribution.
If I had another prior, I would not necessarily have a beta distribution on the output.
So when I have the same family of distributions at the beginning and at the end of this flowchart, I say that beta is a conjugate prior.
Meaning I put in beta as a prior and I get beta as [INAUDIBLE] And that's why betas are so popular.
Conjugate priors are really nice, because you know that whatever you put in, what you're going to get in the end is a beta.
So all you have to think about is the parameters.
You don't have to check again what the posterior is going to look like, what the PDF of this guy is going to be.
You don't have to think about it.
You just have to check what the parameters are.
And there's families of conjugate priors.
Gaussian gives Gaussian, for example.
There's a bunch of them.
And this is what drives people into using specific priors as opposed to others.
It has nice mathematical properties.
Nobody believes that p is really distributed according to beta.
But it's flexible enough and super convenient mathematically.
Now let's see for one second, before we actually go any further.
I didn't mention A and B are both in here, A and B are both positive numbers.
They can be anything positive.
So here what I did is that, I updated A into a plus the sum of my data, and b into b plus n minus the sum of my data.
So that's essentially, a becomes a plus the number of ones.
Well, that's only when I have a and a.
So the first parameters become itself plus the number of ones.
And the second one becomes itself plus the number of zeros.
And so just as a sanity check, what does this mean?
If a it goes to zero, what is the beta when a goes to 0?
We can actually read this from here.
Actually, let's take a goes to-- no.
Sorry, let's just do this.
I'll do it when we talk about non-informative prior, because it's a little too messy.
How do we do this?
How did I get this posterior distribution, given the prior?
How do I update This well this is called Bayesian statistics.
And you've heard this word, Bayes before.
And the way you've heard it is in the Bayes formula.
What was the Bayes formula?
The Bayes formula was telling you that the probability of A, given B was equal to something that depended on the probability of B, given A.
That's what it was.
You can actually either remember the formula or you can remember the definition.
And this is what p of A and B divided by p of B.
So this is p of B, given A times p of A divided by p of B.
That's what Bayes formula is telling you.
Agree?
So now what I want is to have something that's telling me how this is going to work.
What is going to play the role of those events, A and B?
Well one is going to be, this is going to be the distribution of my parameter of theta, given that I see the data.
And this is going to tell me, what is the distribution of the data, given that I know what my parameter if theta is.
But that part, if this is theta and this is the parameter of theta, this is what we've been doing all along.
The distribution of the data, given the parameter here was n iid Bernoulli p. I knew exactly what their joint probability mass function is.
Then, that was what?
So we said that this is going to be my data and this is going to be my parameter.
So that means that, this is the probability of my data, given the parameter.
This is the probability of the parameter.
What is this?
What did we call this?
This is the prior.
It's just the distribution of my parameter.
Now what is this?
Well, this is just the distribution of the data, itself.
This is essentially the distribution of this, if this was indeed not conditioned on p. So if I don't condition on p, this data is going to be a bunch of iid, Bernoulli with some parameter.
But the perimeter is random, right.
So for different realization of this data set, I'm going to get different parameters for the Bernoulli.
And so that leads to some sort of convolution.
It's not really a convolution in this case, but it's like some sort of composition of distributions.
I have the randomness that comes from here and then, the randomness that comes from realizing the Bernoulli.
That's just the marginal distribution.
It actually might be painful to understand what this is, right.
In a way, it's sort of a mixture and it's not super nice.
But we'll see that this actually won't matter for us.
This is going to be some number.
It's going to be there.
But it will matter for us, what it is.
Because it actually does not depend on the parameter.
And that's all that matters to us.
Let's put some names on those things.
This was very informal.
So let's put some actual names on what we call prior.
So what is the formal definition of a prior, what is the formal definition of a posterior, and what are the rules to update it?
So I'm going to have my data, which is going to be X1, Xn.
Let's say they are iid, but they don't actually have to.
And so I'm going to have given, theta.
And when I say given, it's either given like I did in the first part of this course in all previous chapters, or conditionally on.
If you're thinking like a Bayesian, what I really mean is conditionally on this random parameter.
It's as if it was a fixed number.
They're going to have a distribution, X1, Xn is going to have some distribution.
Let's assume for now it's a PDF, pn of X1, Xn.
I'm going to write theta like this.
So for example, what is this?
Let's say this is a PDF.
It could be a PMF.
Everything I say, I'm going to think of them as being PDF's.
I'm going to combine PDF's with PDF's, but I could combine PDF it PMF, PMF with PDF's or PMF with PMF.
So everywhere you see a D could be an M. Now I have those things.
So what does it mean?
So here is an example.
X1, Xn or iid, and theta 1.
Now I know exactly what the joint PDF of this thing is.
It means that pn of X1, Xn given theta is equal to what?
Well it's 1 over 2pi to the power n e, to the minus sum from i equal 1 to n of xi minus theta squared divided by 2.
So that's just the joint distribution of n iid and theta 1, random variables.
That's my pn given theta.
Now this is what we denoted by f sub theta before.
We had the subscript before, but now we just put a bar in theta because we want to remember that this is actually conditioned on theta.
But this is just notation.
You should just think of this as being, just the usual thing that you get from some statistical model.
Now, that's going to be pn.
Theta has prior distribution, pi.
For example, so think of it as either PDF or PMF again.
For example, pi of theta was what?
Well it was some constant times theta to the a minus 1, 1 minus theta to a minus 1.
So it has some prior distribution, and that's another PMF.
So now I'm given the distribution of my, x is given theta and given the distribution of my theta.
I'm given this guy.
That's this guy.
I'm given that guy, which is my pi.
So that's my pn of X1, Xn given theta.
That's my pi of theta.
Well, this is just the integral of pn of X1, Xn times pi of theta, d theta, over all possible sets of theta.
That's just when I integrate out my theta, or I compute the marginal distribution, I did this by integrating.
That's just basic probability, conditional probabilities.
Then if I had the PMF, I would just sum over the values of thetas.
Now what I want is to find what's called, so that's the prior distribution, and I want to find the posterior distribution.
It's pi of theta, given X1, Xn.
If I use Bayes' rule I know that this is pn of X1, Xn, given theta times pi of theta.
And then it's divided by the distribution of those guys, which I will write as integral over theta of pn, X1, Xn, given theta times pi of theta, d theta.
Everybody's with me, still?
If you're not comfortable with this, it means that you probably need to go read your couple of pages on conditional densities and conditional PMF's from your probably class.
There's really not much there.
It's just a matter of being able to define those quantities, f density of x, given y.
This is just what's called a conditional density.
You need to understand what this object is and how it relates to the joint distribution of x and y, or maybe the distribution of x or the distribution of y.
But it's the same rules.
One way to actually remember this is, this is exactly the same rules as this.
When you see a bar, it's the same thing as the probability of this and this guy.
So for densities, it's just a comma divided by the second the probably the second guy.
That's it.
So if you remember this, you can just do some pattern matching and see what I just wrote here.
Now, I can compute every single one of these guys.
This something I get from my modeling.
So I did not write this.
It's not written in the slides.
But I give a name to this guy that was my prior distribution.
And that was my posterior distribution.
In chapter three, maybe what did we call this guy?
The one that does not have a name and that's in the box.
What did we call it?
[INAUDIBLE]
It is the joint distribution of the Xi's.
And we gave it a name
[INAUDIBLE]
It's the likelihood, right?
This is exactly the likelihood.
This was the likelihood of theta.
And this is something that's very important to remember, and that really reminds you that these things are really not that different.
Maximum likelihood estimation and Bayesian estimation, because your posterior is really just your likelihood times something that's just putting some weights on the thetas, depending on where you think theta should be.
If I had, say a maximum likelihood estimate, and my likelihood and theta looked like this, but my prior and theta looked like this.
I said, oh I really want thetas that are like this.
So what's going to happen is that, I'm going to turn this into some posterior that looks like this.
So I'm just really waiting, this posterior, this is a constant that does not depend on theta right?
Agreed?
I integrated over theta, so theta is gone.
So forget about this guy.
I have basically, that the posterior distribution up to scaling, because it has to be a probability density and not just anything any function that's positive, is the product of this guy.
It's a weighted version of my likelihood.
That's all it is.
I'm just weighing the likelihood, using my prior belief on theta.
And so given this guy a natural estimator, if you follow the maximum likelihood principle, would be the maximum of this posterior.
Agreed?
That would basically be doing exactly what maximum likelihood estimation is telling you.
So it turns out that you can.
It's called Maximum A Posteriori, and I won't talk much about this, or MAP.
That's Maximum a Posteriori.
So it's just the theta hat is the arc max of pi theta, given X1, Xn.
And it sounds like it's OK.
I'll give you a density and you say, OK I have a density for all values of my parameters.
You're asking me to summarize it into one number.
I'm just going to take the most likely number of those guys.
But you could summarize it, otherwise.
You could take the average.
You could take the median.
You could take a bunch of numbers.
And the beauty of Bayesian statistics is that, you don't have to take any number in particular.
You have an entire posterior distribution.
This is not only telling you where theta is, but it's actually telling you the difference if you actually give as something that gives you the posterior.
Now, let's say the theta is p between 0 and 1.
If my posterior distribution looks like this, or my posterior distribution looks like this, then those two guys have one, the same mode.
This is the same value.
And their symmetric, so they'll also have the same mean.
So these two posterior distributions give me the same summary into one number.
However clearly, one is much more confident than the other one.
So I might as well just spit it out as a solution.
You can do even better.
People actually do things, such as drawing a random number from this distribution.
Say, this is my number.
That's kind of dangerous, but you can imagine you could do this.
This is what works.
That's what we went through.
So here, as you notice I don't care so much about this part here.
Because it does not depend on theta.
So I know that given the product of those two things, this thing is only the constant that I need to divide so that when I integrate this thing over theta, it integrates to one.
Because this has to be a probability density on theta.
I can write this and just forget about that part.
And that's what's written on the top of this slide.
This notation, this sort of weird alpha, or I don't know.
Infinity sign propped to the right.
Whatever you want to call this thing is actually just really emphasizing the fact that I don't care.
I write it because I can, but you know what it is.
In some instances, you have to compute the integral.
In some instances, you don't have to compute the integral.
And a lot of Bayesian computation is about saying, OK it's actually really hard to compute this integral, so I'd rather not doing it.
So let me try to find some methods that will allow me to sample from the posterior distribution, without having to compute this.
And that's what's called Monte-Carlo Markov chains, or MCMC, and that's exactly what they're doing.
They're just using only ratios of things, like that for different thetas.
And which means that if you take ratios, the normalizing constant is gone and you don't need to find this integral.
So we won't go into those details at all.
That would be the purpose of an entire course on Bayesian inference.
Actually, even Bayesian computations would be an entire course on its own.
And there's some very interesting things that are going on there, the interface of stats and computation.
So let's go back to our example and see if we can actually compute any of those things.
Because it's very nice to give you some data, some formulas.
Let's see if we can actually do it.
In particular, can I actually recover this claim that the posterior associated to a beta prior with a Bernoulli likelihood is actually giving me a beta again?
What was my prior?
So p was following a beta AA, which means that p, the density.
That was pi of theta.
Well I'm going to write this as pi of p-- was proportional to p to the A minus 1 times 1 minus p to the A minus 1.
So that's the first ingredient I need to complete my posterior.
I really need only two, if I wanted to bound up to constant.
The second one was p hat.
We've computed that many times.
And we had even a nice compact way of writing it, which was that pn of X1, Xn, given the parameter p. So the joint density of my data, given p, that's my likelihood.
The likelihood of p was what?
Well it was p to the sum of Xi's.
1 minus p to the n minus some of the Xi's.
Anybody wants me to parse this more?
Or do you remember seeing that from maximum likelihood estimation?
Yeah?
[INAUDIBLE]
That's what conditioning does
[INAUDIBLE] previous slide.
[INAUDIBLE] bottom there, it says D pi of t. Shouldn't it be dt pi of t?
So D pi of T is a measure theoretic notation, which I used without thinking.
And I should not because I can see it upsets you.
D pi of T is just a natural way to say that I integrate against whatever I'm given for the prior of theta.
In particular, if theta is just the mix of a PDF and a point mass, maybe I say that my p takes value 0.5 with probability 0.5.
And then is uniform on the interval with probability 0.5.
For this, I neither have a PDF nor a PMF.
But I can still talk about integrating with respect to this, right?
It's going to look like, if I take a function f of T, D pi of T is going to be one half of f of one half.
That's the point mass with probability one half, at one half.
Plus one half of the integral between 0 and 1, of f of TDT.
This is just the notation, which is actually funnily enough, interchangeable with pi of DT.
But if you have a density, it's really just the density pi of TDT.
If pi is really a density, but that's when it's when pi is and measure and not a density.
Everybody else, forget about this.
This is not something you should really worry about at this point.
This is more graduate level probability classes.
But yeah, it's called measure theory.
And that's when you think of pi as being a measure in an abstract fashion.
You don't have to worry whether it's a density or not, or whether it has a density.
So everybody is OK with this?
Now I need to compute my posterior.
And as I said, my posterior is really just the product of the likelihood weighted by the prior.
Hopefully, at this stage of your application, you can multiply two functions.
So what's happening is, if I multiply this guy with this guy, p gets this guy to the power this guy plus this guy.
And then 1 minus p gets the power n minus some of Xi's.
So this is always from I equal 1 to n. And then plus A minus 1 as well.
This is up to constant, because I still need to solve this.
And I could try to do it.
But I really don't have to, because I know that if my density has this form, then it's a beta distribution.
And then I can just go on Wikipedia and see what should be the normalization factor.
But I know it's going to be a beta distribution.
It's actually the beta with parameter.
So this is really my beta with parameter, sum of Xi, i equal 1 to n plus A minus 1.
And then the second parameter is n minus sum of the Xi's plus A minus 1.
I just wrote what was here.
What happened to my one?
Oh no, sorry.
Beta has the power minus 1.
So that's the parameter of the beta.
And this is the parameter of the beta.
Beta is over there, right?
So I just replace A by what I see.
A is just becoming this guy plus this guy and this guy plus this guy.
Everybody is comfortable with this computation?
We just agreed that beta priors for Bernoulli observations are certainly convenient.
Because they are just conjugate, and we know that's what is going to come out in the end.
That's going to be a beta as well.
I just claim it was convenient.
It was certainly convenient to compute this, right?
There was certainly some compatibility when I had to multiply this function by that function.
And you can imagine that things could go much more wrong, than just having p to some power and p to some power, 1 minus p to some power, when it might just be some other power.
Things were nice.
Now this is nice, but I can also question the following things.
Why beta, for one?
The beta tells me something.
That's convenient, but then how do I pick A?
I know that A should definitely capture the fact that where I want to have my p most likely located.
But it also actually also captures the variance of my beta.
And so choosing different As is going to have different functions.
If I have A and B, If I started with the beta with parameter.
If I started with a B here, I would just pick up the B here.
Agreed?
And that would just be a symmetric.
But they're going to capture mean and variance of this thing.
And so how do I pick those guys?
If I'm a doctor and you're asking me, what do you think the chances of this drug working in this kind of patients is?
And I have to spit out the parameters of a beta for you, it might be a bit of a complicated thing to do.
So how do you do this, especially for problems?
So by now, people have actually mastered the art of coming up with how to formulate those numbers.
But in new problems that come up, how do you do this?
What happens if you want to use Bayesian methods, but you actually do not know what you expect to see?
To be fair, before we started this class, I hope all of you had no idea whether people tend to bend their head to the right or to the left before kissing.
Because if you did, well you have too much time on your hands and I should double your homework.
So in this case, maybe you still want to use the Bayesian machinery.
Maybe you just want to do something nice.
It's nice right, I mean it worked out pretty well.
What if you want to do?
Well you actually want to use some priors that carry no information, that basically do not prefer any theta to another theta.
Now, you could read this slide or you could look at this formula.
We just said that this pi here was just here to weigh some thetas more than others, depending on their prior belief.
If our prior belief does not want to put any preference towards some thetas than to others, what do I do?
[INAUDIBLE]
Yeah, I remove it.
And the way to remove something we multiply by, is just replace it by one.
That's really what we're doing.
If this was a constant not depending on theta, then that would mean that we're not preferring any theta.
And we're looking at the likelihood.
But not as a function that we're trying to maximize, but it is a function that we normalize in such a way that it's actually a distribution.
So if I have pi, which is not here, this is really just taking the like likelihood, which is a positive function.
It may not integrate to 1, so I normalize it so that it integrates to 1.
And then I just say, well this is my posterior distribution.
Now I could just maximize this thing and spit out my maximum likelihood estimator.
But I can also integrate and find what the expectation of this guy is.
I can find what the median of this guy is.
I can sample data from this guy.
I can build, understand what the variance of this guy is.
Which is something we did not do when we just did maximum likelihood estimation because given a function, all we cared about was the arc max of this function.
These priors are called uninformative.
This is just replacing this number by one or by a constant.
Because it still has to be a density.
If I have a bounded set, I'm just looking for the uniform distribution on this bounded set, the one that puts constant one over the size of this thing.
But if I have an invalid set, what is the density that takes a constant value on the entire real line, for example?
What is this density?
[INAUDIBLE]
Doesn't exist, right?
It just doesn't exist.
The way you can think of it is a Gaussian with the variance going to infinity, maybe, or something like this.
But you can think of it in many ways.
You can think of the limit of the uniform between minus T and T, with T going to infinity.
But this thing is actually zero.
There's nothing there.
You can actually still talk about this.
You could always talk about this thing, where you think of this guy as being a constant, remove this thing from this equation, and just say, well my posterior is just the likelihood divided by the integral of the likelihood over theta.
And if theta is the entire real line, so be it.
As long as this integral converges, you can still talk about this stuff.
This is what's called an improper prior.
An improper prior is just a non-negative function defined in theta, but it does not have to integrate neither to one, nor to anything.
If I integrate the function equal to 1 on the entire real line, what do I get?
Infinity.
It's not a proper prior, and it's called and improper prior.
And those improper priors are usually what you see when you start to want non-informative priors on infinite sets of datas.
That's just the nature of it.
You should think of them as being the uniform distribution of some infinite set, if that thing were to exist.
Let's see some examples about non-informative priors.
If I'm in the interval 0, 1 this is a finite set.
So I can talk about the uniform prior on the interval 0, 1 for a parameter, p of a Bernoulli.
If I want to talk about this, then it means that my prior is p follows some uniform on the interval 0, 1.
So that means that f of x is 1 if x is in 0, 1.
Otherwise, there is actually not even a normalization.
This thing integrates to 1.
And so now if I look at my likelihood, it's still the same thing.
So my posterior becomes theta X1, Xn.
That's my posterior.
I don't write the likelihood again, because we still have it-- well we don't have it here anymore.
The likelihood is given here.
Copy, paste over there.
The posterior is just this thing times 1.
So you will see it in a second.
So it's p to the power sum of the Xi's, one minus p to the power, n minus sum of the Xi's.
And then it's multiplied by 1, and then divided by this integral between 0 and 1 of p, sum of the Xi's.
1 minus p, n minus sum of the Xi's.
Dp, which does not depend on p. And I really don't care what the thing actually is.
That's posterior of p. And now I can see, well what is this?
It's actually just the beta with parameters.
This guy plus 1.
And this guy plus 1.
I didn't tell you what the expectation of a beta was.
We don't know what the expectation of a beta is, agreed?
If I wanted to find say, the expectation of this thing that would be some good estimator, we know that the maximum of this guy-- what is the maximum of this thing?
Well, it's just this thing, it's the average of the Xi's.
That's just the maximum likelihood estimator for Bernoulli.
We know it's the average.
Do you think if I take the expectation of this thing, I'm going to get the average?
So actually, I'm not going to get the average.
I'm going to get this guy plus this guy, divided by n plus 1.
Let's look at what this thing is doing.
It's looking at the number of ones and it's adding one.
And this guy is looking at the number of zeros and it's adding one.
Why is it adding this one?
What's going on here?
This is going to matter mostly when the number of ones is actually zero, or the number of zeros is zero.
Because what it does is just pushes the zero from non-zero.
And why is that something that this Bayesian method actually does for you automatically?
It's because when we put this non-informative prior on p, which was uniform on the interval 0, 1.
In particular, we know that the probability that p is equal to 0 is zero.
And the probability p is equal to 1 is zero.
And so the problem is that if I did not add this 1 with some positive probability, I wouldn't be allowed to spit out something that actually had p hat, which was equal to 0.
If by chance, let's say I have n is equal to 3, and I get only 0, 0, 0, that could happen with probability.
1 over pq, one over 1 minus pq.
That's not something that I want.
And I'm using my priors.
My prior is not informative, but somehow it captures the fact that I don't want to believe p is going to be either equal to 0 or 1.
So that's sort of taken care of here.
So let's move away a little bit from the Bernoulli example, shall we?
I think we've seen enough of it.
And so let's talk about the Gaussian model.
Let's say I want to do Gaussian inference.
I want to do inference in a Gaussian model, using Bayesian methods.
What I want is that Xi, X1, Xn, or say 0, 1 iid.
Sorry, theta 1, iid conditionally on theta.
That means that pn of X1, Xn, given theta is equal to exactly what I wrote before.
So 1 square root to pi, to the n exponential minus one half sum of Xi minus theta squared.
So that's just the joint distribution of my Gaussian with mean data.
And the another question is, what is the posterior distribution?
Well here I said, let's use the uninformative prior, which is an improper prior.
It puts weight on everyone.
That's the so-called uniform on the entire real line.
So that's certainly not a density.
But it can still just use this.
So all I need to do is get this divided by normalizing this thing.
But if you look at this, essentially I want to understand.
So this is proportional to the exponential minus one half sum from I equal 1 to n of Xi minus theta squared.
And now I want to see this thing as a density, not on the Xi's but on theta.
What I want is a density on theta.
So it looks like I have chances of getting something that looks like a Gaussian.
To have a Gaussian, I would need to see minus one half.
And then I would need to see theta minus something here, not just the sum of something minus thetas.
So I need to work a little bit more, to expand the square here.
So this thing here is going to be equal to exponential minus one half sum from I equal 1 to n of Xi squared minus 2Xi theta plus theta squared.
Now what I'm going to do is, everything remember is up to this little sign.
So every time I see a term that does not depend on theta, I can just push it in there and just make it disappear.
Agreed?
This term here, exponential minus one half sum of Xi squared, does it depend on theta?
No.
So I'm just pushing it here.
This guy, yes.
And the other one, yes.
So this is proportional to exponential sum of the Xi.
And then I'm going to pull out my theta, the minus one half canceled with the minus 2.
And then I have minus one half sum from I equal 1 to n of theta squared.
Agreed?
So now what this thing looks like, this looks very much like some theta minus something squared.
This thing here is really just n over 2 times theta.
Sorry, times theta squared.
So now what I need to do is to write this of the form, theta minus something.
Let's call it mu, squared, divided by 2 sigma squared.
I want to turn this into that, maybe up to terms that do not depend on theta.
That's what I'm going to try to do.
So that's called completing the squaring.
That's some exercises you do.
You've done it probably, already in the homework.
And that's something you do a lot when you do Bayesian statistics, in particular.
So let's do this.
What is it going to be the leading term?
Theta squared is going to be multiplied by this thing.
So I'm going to pull out my n over 2.
And then I'm going to write this as minus theta over 2.
And then I'm going to write theta minus something squared.
And this something is going to be one half of what I see in the cross-product.
I need to actually pull this thing out.
So let me write it like that first.
So that's theta squared.
And then I'm going to write it as minus 2 times 1 over n sum from I equal 1 to n of Xi's times theta.
That's exactly just a rewriting of what we had before.
And that should look much more familiar.
A squared minus 2 blap A, and then I missed something.
So this thing, I'm going to be able to rewrite as theta minus Xn bar squared.
But then I need to remove the square of Xn bar.
Because it's not here.
So I just complete the square.
And then I actually really don't care with this thing actually was, because it's going to go again in the little Alpha's sign over there.
So this thing eventually is going to be proportional to exponential of minus n over 2 times theta of minus Xn bar squared.
And so we know that if this is a density that's proportional to this guy, it has to be some n with mean, Xn bar.
And variance, this is supposed to be 1 over sigma squared.
This guy over here, this n. So that's really just 1 over n. So the posterior distribution is a Gaussian centered at the average of my observations.
And with variance, 1 over n. Everybody's with me?
Why I'm saying this, this was the output of some computation.
But it sort of makes sense, right?
It's really telling me that the more observations I have, the more concentrated this posterior is.
Concentrated around what?
Well around this Xn bar.
That looks like something we've sort of seen before.
But it does not have the same meaning, somehow.
This is really just the posterior distribution.
It's sort of a sanity check, that I have this 1 over n when I have Xn bar.
But it's not the same thing as saying that the variance of Xn bar was 1 over n, like we had before.
As an exercise, I would recommend if you don't get it, just try pi of theta to be equal to some n mu 1.
Here, the prior that we used was completely non-informative.
What happens if I take my prior to be some Gaussian, which is centered at mu and it has the same variance as the other guys?
So what's going to happen here is that we're going to put a weight.
And everything that's away from mu is going to actually get less weight.
I want to know how I'm going to be updating this prior into a posterior.
Everybody sees what I'm saying here?
So that means that pi of theta has the density proportional to exponential minus one half theta minus mu squared.
So I need to multiply my posterior with this, and then see.
It's actually going to be a Gaussian.
This is also a conjugate prior.
It's going to spit out another Gaussian.
You're going to have to complete a square again, and just check what it's actually giving you.
And so spoiler alert, it's going to look like you get an extra observation, which is actually equal to mu.
It's going to be the average of n plus 1 observations.
The first n1's being X1 to Xn.
And then, the last one being mu.
And it sort of makes sense.
That's actually a fairly simple exercise.
Rather than going into more computation, this is something you can definitely do when you're in the comfort of your room.
I want to talk about other types of priors.
The first thing I said is, there's this beta prior that I just pulled out of my hat and that was just convenient.
Then there was this non-informative prior.
It was convenient.
It was non-informative, so if you don't know anything else maybe that's what you want to do.
The question is, are there any other priors that are sort of principled and generic, in the sense that the uninformative prior was generic, right?
It was equal to 1, that's as generic as it gets.
So is there anything that's generic as well?
Well, there's this priors that are called Jeffrey's priors.
And Jeffrey's prior, which is proportional to square root of the determinant of the Fisher information of theta.
This is actually a weird thing to do.
It says, look at your model.
Your model is going to have a Fisher information.
Let's say it exists.
Because we know it does not always exist.
For example, in the multinomial model, we didn't have a Fisher information.
The determinant of a matrix is somehow measuring the size of a matrix.
If you don't trust me, just think about the matrix being of size one by one, then the determinant is just the number that you have there.
And so this is really something that looks like the Fisher information.
It's proportional to the amount of information that you have at a certain point.
And so what my prior is saying well, I want to put more weights on those thetas that are going to just extract more information from the data.
You can actually compute those things.
In the first example, Jeffrey's prior is something that looks like this.
In one dimension, Fisher information is essentially one the word variance.
That's just 1 over the square root of the variance, because I have the square root.
And when I have the Jeffrey's prior, when I have the Gaussian case, this is the identity matrix that I would have in the Gaussian case.
The determinant of the identities is 1.
So square root of 1 is 1, and so I would basically get 1.
And that gives me my improper prior, my uninformative prior that I had.
So the uninformative prior 1 is fine.
Clearly, all the thetas carry the same information in the Gaussian model.
Whether I translate it here or here, it's pretty clear none of them is actually better than the other.
But clearly for the Bernoulli case, the p's that are closer to the boundary carry more information.
I sort of like those guys, because they just carry more information.
So what I do is, I take this function.
So p1 minus p. Remember, it's something that looks like this.
On the interval 0, 1.
This guy, 1 over square root of p1 minus p is something that looks like this.
Agreed What it's doing is sort of wants to push towards the piece that actually carry more information.
Whether you want to bias your data that way or not, is something you need to think about.
When you put a prior on your data, on your parameter, you're sort of biasing towards this idea your data.
That's maybe not such a good idea, when you have some p that's actually close to one half, for example.
You're actually saying, no I don't want to see a p that's close to one half.
Just make a decision, one way or another.
But just make a decision.
So it's forcing you to do that.
Jeffrey's prior, I'm running out of time so I don't want to go into too much detail.
We'll probably stop here, actually.
So Jeffrey's priors have this very nice property.
It's that they actually do not care about the parameterization of your space.
If you actually have p and you suddenly decide that p is not the right parameter for Bernoulli, but it's p squared.
You could decide to parameterize this by p squared.
Maybe your doctor is actually much more able to formulate some prior assumption on p squared, rather than p. You never know.
And so what happens is that Jeffrey's priors are an invariant in this.
And the reason is because the information carried by p is the same as the information carried by p squared, somehow.
They're essentially the same thing.
You need to have one to one map.
Where you basically for each parameter, before you have another parameter.
Let's call Eta the new parameters.
The PDF of the new prior indexed by Eta this time is actually also Jeffrey's prior.
But this time, the new Fisher information is not the Fisher information with respect to theta.
But it's this Fisher information associated to this statistical model indexed by Eta.
So essentially, when you change the parameterization of your model, you still get Jeffrey's prior for the new parameterization.
Which is, in a way, a desirable property.
Jeffrey's prior is just an uninformative priors, or priors you want to use when you want a systematic way without really thinking about what to pick for your mile.
I'll finish this next time.
And we'll talk about Bayesian confidence regions.
We'll talk about Bayesian estimation.
Once I have a posterior, what do I get?
And basically, the only message is going to be that you might want to integrate against the posterior.
Find the posterior, the expectation of your posterior distribution.
That's a good point estimator for theta.
We'll just do a couple of computation.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today, we're going to close this chapter, this short chapter, on Bayesian inference.
Again, this was just an overview of what you can do in Bayesian inference.
And last time, we started defining what's called Jeffreys priors.
Right?
So when you do Bayesian inference, you have to introduce a prior on your parameter.
And we said that usually, it's something that encodes your domain knowledge about where the parameter could be.
But there's also some principle way to do it, if you want to do Bayesian inference without really having to think about it.
And for example, one of the natural priors were those non-informative priors, right?
If you were on a compact set, it's a uniform prior of this set.
If you're on an infinite set, you can still think of taking the [?
01s ?]
prior.
And that's called an [INAUDIBLE] That's always equal to 1.
And that's an improper prior if you are an infinite set or proportional to one.
And so another prior that you can think of, in the case where you have a Fisher information, which is well-defined, is something called Jefferys prior.
And this prior is a prior which is proportional to square root of the determinant of the Fisher information matrix.
And if you're in one dimension, it's basically proportional to a square root of the Fisher information coefficient, which we know, for example, is the asymptotic variance of the maximum likelihood estimator.
And it turns out that it's basically.
So square root of this thing is basically one over the standard deviation of the maximum likelihood estimator.
And so you can compute this, right?
So you can compute for the maximum likelihood estimator.
We know that the variance is going to be p1 minus p in the Bernoulli statistical experiment.
So you get this one over the square root of this thing.
And for example, in the Gaussian setting, you actually have the Fisher information, even in the multi-variate one, is actually going to be something like the identity matrix.
So this is proportional to 1.
It's the improper prior that you get, in this case, OK?
Meaning that, for the Gaussian setting, no place where you center your Gaussian is actually better than any other.
All right.
So we basically left on this slide, where we saw that Jeffreys prior satisfy a reparametrization [INAUDIBLE] invariant by transformation of your parameter, which is a desirable property.
And the way, it says that, well, if I have my prior on theta, and then I suddenly decide that theta is not the parameter I want to use to parameterize my problem, actually what I want is phi of theta.
So think, for example, as theta being the mean of a Gaussian, and phi of theta as being mean to the cube.
OK?
This is a one-to-one map phi, right?
So for example, if I want to go from theta to theta cubed, and now I decide that this is the actual parameter that I want, well, then it means that, on this parameter, my original prior is going to induce another prior.
And here, it says, well, this prior is actually also Jeffreys prior.
OK?
So it's essentially telling you that, for this new parametrization, if you take Jeffreys prior, then you actually go back to having exactly something that's of the form's [INAUDIBLE] of determinant of the Fisher information, but this thing with respect to your new parametrization All right.
And so why is this true?
Well, it's just this change of variable theorem.
So it's essentially telling you that, if you call-- let's call p-- well, let's go pi tilde of eta prior over eta.
And you have pi of theta as the prior over theta, than since eta is of the form phi of theta, just by change of variable, so that's essentially a probability result.
It says that pi tilde of eta is equal to pi of eta times d pi of theta times d theta over d eta and-- sorry, is that the one?
Sorry, I'm going to have to write it, because I always forget this.
So if I take a function-- OK.
So what I want is to check.
OK, so I want the function of eta that I can here.
And what I know is that this is h of phi of theta.
All right?
So sorry, eta is phi of theta, right?
Yeah.
So what I'm going to do is I'm going to do the change of variable, theta is phi inverse of eta.
So eta is phi of theta, which means that d eta is equal to d-- well, to phi prime of theta d theta.
So when I'm going to write this, I'm going to get integral of h. Actually, let me write this, as I am more comfortable writing this as e with respect to eta of h of eta.
OK?
So that's just eta according to being drawn from the prior.
And I want to write this as the integral of he of eta times some function, right?
So this is the integral of h of phi of theta pi of theta d theta.
Now, I'm going to do my change of variable.
So this is going to be the integral of h of eta.
And then pi of phi of-- so theta is phi inverse of eta.
And then d theta is phi prime of theta d theta, OK?
And so what is pi of phi theta?
So this thing is proportional.
So we're in, say, dimension 1, so it's proportional of square root of the Fisher information.
And the Fisher information, we know, is the expectation of the square of the derivative of the log likelihood, right?
So this is square root of the expectation of d over d theta of log of-- well, now, I need the density.
Well, let's just call it l of theta.
And I want this to be taken at phi inverse of eta squared.
And then what I pick up is the-- so I'm going to put everything under the square.
So I get phi prime of theta squared d theta.
OK?
So now, I have the expectation of a square.
This does not depend, so this is-- sorry, this is l of theta.
This is the expectation of l of theta of an x, right?
That's for some variable, and the expectation here is with respect to x.
That's just the definition of the Fisher information.
So now I'm going to squeeze this guy into the expectation.
It does not depend on x.
It just acts as a constant.
And so what I have now is that this is actually proportional to the integral of h eta times the square root of the expectation with respect to x of what?
Well, here, I have d over d theta of log of theta.
And here, this guy is really d eta over d theta, right?
Agree?
So now, what I'm really left by-- so I get d over d theta times d-- sorry, times d theta over d eta.
so that's just d over d eta of log of eta x.
And then this guy is now becoming d eta, right?
OK, so this was a mess.
This is a complete mess, because I actually want to use phi.
I should not actually introduce phi at all.
I should just talk about d eta over d theta type of things.
And then that would actually make my life so much easier.
OK.
I'm not going to spend more time on this.
This is really just the idea, right?
You have square root of a square in there.
And then, when you do your change of variable, you just pick up a square.
You just pick up something in here.
And so you just move this thing in there.
You get a square.
It goes inside the square.
And so your derivative of the log likelihood with respect to theta becomes a derivative of the log likelihood with respect to eta.
And that's the only thing that's happening here.
I'm just being super sloppy, for some reason.
OK. And then, of course, now, what you're left with is that this is really just proportional.
Well, this is actually equal.
Everything is proportional, but this is equal to the Fisher information tilde with respect to eta now.
Right?
You're doing this with respect to eta.
And so that's your new prior with respect to eta.
OK.
So one thing that you want to do, once you have-- so remember, when you actually compute your posterior rate, rather than having-- so you start with a prior, and you have some observations, let's say, x1 to xn.
When you do Bayesian inference, rather than spitting out just some theta hat, which is an estimator for theta, you actually spit out an entire posterior distribution-- pi of theta, given x1 xn.
OK?
So there's an entire distribution on the [INAUDIBLE] theta.
And you can actually use this to perform inference, rather than just having one number.
OK?
And so you could actually build confidence regions from this thing.
OK. And so a Bayesian confidence interval-- so if your set of parameters is included in the real line, then you can actually-- it's not even guaranteed to be to be an interval.
So let me call it a confidence region, so a Bayesian confidence region, OK?
So it's just a random subspace.
So let's call it r, is included in theta.
And when you have the deterministic one, we had a definition, which was with respect to the randomness of the data, right?
That's how you actually had a random subset.
So you had a random confidence interval.
Here, it's actually conditioned on the data, but with respect to the randomness that you actually get from your posterior distribution.
OK?
So such that the probability that your theta belongs to this confidence region, given x1 xn is, say, at least 1 minus alpha.
Let's just take it equal to 1 minus alpha.
OK so that's a confidence region at level 1 minus alpha.
OK, so that's one way.
So why would you actually-- when I actually implement Bayesian inference, I'm actually spitting out that entire distribution.
I need to summarize this thing to communicate it, right?
I cannot just say this is this entire function.
I want to know where are the regions of high probability, where my perimeter is supposed to be?
And so here, when I have this thing, what I actually want to have is something that says, well, I want to summarize this thing into some subset of the real line, in which I'm sure that the area under the curve, here, of my posterior is actually 1 minus alpha.
And there's many ways to do this, right?
So one way to do this is to look at level sets.
And so rather than actually-- so let's say my posterior looks like this.
I know, for example, if I have a Gaussian distribution, I can actually take my posterior to be-- my posterior is actually going to be Gaussian.
And what I can do is to try to cut it here on the y-axis so that now, the area under the curve, when I cut here, is actually 1 minus alpha.
OK, so I have some threshold tau.
If tau goes to plus infinity, then I'm going to have that this area under the curve here is going to--
[INAUDIBLE]
Well, no.
So the area under the curve, when tau is going to plus infinity, think of the small, the when tau is just right here
[INAUDIBLE]
So this is actually going to 0, right?
And so I start here.
And then I start going down and down and down and down, until I actually get something which is going down to 1 plus alpha.
And if tau is going down to 0, then my area under the curve is going to-- if tau is here, I'm cutting nowhere.
And so I'm getting 1, right?
Agree?
Think of, when tau is very close to 0, I'm cutting [?
s ?]
s very far here.
And so I'm getting some area under the curve, which is almost everything.
And so it's going to 1-- as tau going down to 0.
Yeah?
Does this only work for [INAUDIBLE]
No, it does not.
I mean-- so this is a picture.
So those two things work for all of them, right?
But when you have a [?
bimodal, ?]
actually, this is actually when things start to become interesting, right?
So when we built a frequentist confidence interval, it was always of the form x bar plus or minus something.
But now, if I start to have a posterior that looks like this, what I'm going to start cutting off, I'm going to have two-- I mean, my confidence region is going to be the union of those two things, right?
And it really reflects the fact that there is this bimodal thing.
It's going to say, well, with hyperbole, I'm actually going to be either here or here.
Now, the meaning here of a Bayesian confidence region and the confidence interval are completely distinct notions, right?
And I'm going to work out on example with you so that we can actually see that sometimes-- I mean, both of them, actually you can come up with some crazy paradoxes.
So since we don't have that much time, I will actually talk to you about why, in some instances, it's actually a good idea to think of Bayesian confidence intervals rather than frequentist ones.
So before we go into more details about what those Bayesian confidence intervals are, let's remind ourselves what does it mean to have a frequentist confidence interval?
Right?
OK.
So when I have a frequentist confidence interval, let's say something like x bar n to minus 1.96 sigma over root n and x bar n plus 1.96 sigma over root n, so that's the confidence interval that you get for the mean of some Gaussian with known variants to be equal to sigma square, OK.
So what we know is that the meaning of this is the probability that theta belongs to this is equal to 95%, right?
And this, more generally, you can think of being q alpha over 2.
And what you're going to get is 1 minus alpha here, OK?
So what does it mean here?
Well, it looks very much like what we have here, except that we're not conditioning on x1 xn.
And we should not.
Because there was a question like that in the midterm-- if I condition on x1 xn, this probability is either 0 or 1.
OK?
Because once I condition-- so here, this probability, actually, here is with respect to the randomness in x1 xn.
So if I condition-- so let's build this thing, r freq, for frequentist.
Well, given x1 xn-- and actually, I don't need to know x1 xn really.
What I need to know is what xn bar is.
Well, this thing now is what?
It's 1, if theta is in r, and it's 0, if theta is not in r, right?
That's all there is.
This is a deterministic confidence interval, once I condition x1 xn.
So I have a number.
The average is maybe 3.
And so I get 3.
Either theta is between 3 minus 0.5 or in 3 plus 0.5, or it's not.
And so there's basically-- I mean, I write it as probability, but it's really not a probalistic statement.
It's either it's true or not.
Agreed?
So what does it mean to have a frequentist confidence interval?
It means that if I were-- and here, where the word frequentist comes from, it says that if I repeat this experiment over and over, meaning that on Monday, I collect a sample of size n, and I build a confidence interval, and then on Tuesday, I collect another sample of size n, and I build a confidence interval, and on Wednesday, I do this again and again, what's going to happen is the following.
I'm going to have my true theta that lives here.
And then on Monday, this is the confidence interval that I build.
OK, so this is the real line.
The true theta is here, and this is the confidence interval I build on Monday.
All right?
So x bar was here, and this is my confidence interval.
On Tuesday, I build this confidence interval maybe.
x bar was closer to theta, but smaller.
But then on Wednesday, I build this confidence interval.
I'm not here.
It's not in there.
And that's this case.
Right?
It happens that it's just not in there.
And then on Thursday, I build another one.
I almost miss it, but I'm in there, et cetera.
Maybe here, Here, I miss again.
And so what it means to have a confidence interval-- so what does it mean to have a confidence interval at 95%?
[INAUDIBLE]
Yeah, so it means that if I repeat this the frequency of times-- hence, the word frequentist-- at which I'm actually going to overlap that, I'm actually going to contain theta, should be 95%.
That's what frequentist means.
So it's just a matter of trusting that.
So on one given thing, one given realization of your data, it's not telling you anything.
[INAUDIBLE] it's there or not.
So it's not really something that's actually something that assesses the confidence of your decision, such as data is in there or not.
It's something that assesses the confidence you have in the method that you're using.
If you were you repeat it over and again, it'd be the same thing.
It would be 95% of the time correct, right?
So for example, we know that we could build a test.
So it's pretty clear that you can actually build a test for whether theta is equal to theta naught or not equal to theta naught, by just checking whether theta naught is in a confidence interval or not.
And what it means is that, if you actually are doing those tests at 5%, that means that 5% of the time, if you do this over and again, 5% of the time you're going to be wrong.
I mentioned my wife does market research.
And she does maybe, I don't know, 100,000 tests a year.
And if they do all of them at 1%, then it means that 1% of the time, which is a lot of time, right?
When you do 100,000 a year, it's 1,000 of them are actually wrong.
OK, I mean, she's actually hedging against the fact that 1% of them that are going to be wrong.
That's 1,000 of them that are going to be wrong.
Just like, if you do this 100,000 times at 95%, 5,000 of those guys are actually not going to be the correct ones.
OK?
So I mean, it's kind of scary.
But that's the way it is.
So that's with the frequentist interpretation of this is.
Now, as I mentioned, when we started this Bayesian chapter, I said, Bayesian statistics converge to-- I mean, Bayesian decisions and Bayesian methods converge to frequentist methods.
When the sample size is large enough, they lead to the same decisions.
And in general, they need not be the same, but they tend to actually, when the sample size is large enough, to have the same behavior.
Think about, for example, the posterior that you have when you have in the Gaussian case, right?
We said that, in the Gaussian case, what you're going to see is that it's as if you had an extra observation which was essentially given by your prior.
OK?
And now, what's going to happen is that, when this just one observation among n plus 1, it's really going to be totally drawn, and you won't see it when the sample size grows larger.
So Bayesian methods are particularly useful when you have a small sample size.
And when you have a small sample size, the effect of the prior is going to be bigger.
But most importantly, you're not going to have to repeat this thing over and again.
And you're going to have a meaning.
You're going to have to have something that has a meaning for this particular data set that you have.
When I said that the probability that theta belongs to r-- and here, I'm going to specify the fact that it's a Bayesian confidence region, like this one-- this is actually conditionally on the data that you've collected.
It says, given this data, given the points that you have-- just put in some numbers, if you want, in there-- it's actually telling you the probability that theta belongs to this Bayesian thing, to this Bayesian confidence region.
Here, since I have conditioned on x1 xn, this probability is really just with respect to theta drawn from the prior, right?
And so now, it has a slightly different meaning.
It's just telling you that when-- it's really making a statement about where the regions of hyperability of your posterior are.
Now, why is that useful?
Well, there's actually an interesting story that goes behind Bayesian methods.
Anybody knows the story of the USS I think it's Scorpion?
Do you know the story?
So that was an American vessel that disappeared.
I think it was close to Bermuda or something.
But you can tell the story of the Malaysian Airlines, except that I don't think it's such a successful story.
But the idea was essentially, we're trying to find where this thing happened.
And of course, this is a one-time thing.
You need something that works once.
You need something that works for this particular vessel.
And you don't care, if you go to the Navy, and you tell them, well, here's a method.
And for 95 out of 100 vessels that you're going to lose, we're going to be able to find it.
And they want this to work for this particular one.
And so they were looking, and they were diving in different places.
And suddenly, they brought in this guy.
I forget his name.
I mean, there's a whole story about this on Wikipedia.
And he started collecting the data that they had from different dives and maybe from currents.
And he started to put everything in.
And he said, OK, what is the posterior distribution of the location of the vessel, given all the things that I've seen?
And what have you seen?
Well, you've seen that it's not here, it's not there, and it's not there.
And you've also seen that the currents were going that way, and the winds were going that way.
And you can actually put some modeling traits to understand this.
Now, given this, for this particular data that you have, you can actually think of having a two-dimensional density that tells you where it's more likely that the vessel is.
And where are you going to be looking for?
Well, if it's a multimodal distribution, you're just going to go to the highest mode first, because that's where it's the most likely to be.
And maybe it's not there, so you're just going to update your posterior, based on the fact that it's not there, and do it again.
And actually, after two dives, I think, he actually found the thing.
And that's exactly where Bayesian statistics start to kick in.
Because you put a lot of knowledge into your model, but you also can actually factor in a bunch of information, right?
The model, he had to build a model that was actually taking into account and currents, and when.
And what you can have as a guarantee is that, when you talk about the probability that this vessel is in this location, given what you've observed in the past, it actually has some sense.
Whereas, if you were to use a frequentist approach, then there's no probability.
Either it's underneath this position or it's not, right?
So that's actually where it start to make sense.
And so you can actually build this.
And there's actually a lot of methods that are based on, for search, that are based on Bayesian methods.
I think, for example, the Higgs boson was based on a lot of Bayesian methods, because this is something you need to find [INAUDIBLE],, right?
I mean, there was a lot of prior that has to be built in.
OK.
So now, you build this confidence interval.
And the nicest way to do it is to use level sets.
But again, just like for Gaussians, I mean, if I had, even in the Gaussian case, I decided to go at x bar plus or minus something, but I could go at something that's completely asymmetric.
So what's happening is that here, this method guarantees that you're going to have the narrowest possible confidence intervals.
That's essentially what it's telling you, OK?
Because every time I'm choosing a point, starting from here, I'm actually putting as much area under the curve as I can.
All right.
So those are called Bayesian confidence [?
interval.
?]
Oh yeah, and I promised you that we're going to work on some example that actually gives a meaning to what I just told you, with actual numbers.
So this is something that's taken from Wasserman's book.
And also, it's coming from a paper, from a stats paper, from [?
Wolpert ?]
and I don't know who, from the '80s.
And essentially, this is how it works.
So assume that you have n equals 2 observations.
And you have y1, so those observations are y1-- no, sorry, let's call them x1, which is theta, plus epsilon 1 and x2, which is theta plus epsilon 2, where epsilon 1 and epsilon 2 are iid.
And the probability that epsilon i is equal to plus 1 is equal to the probability that epsilon i is equal to minus 1 is equal to 1/2.
OK, so it's just the uniform sign plus minus 1, OK?
Now, let's think about so you're trying to do some inference on theta.
Maybe you actually want to find some inference on theta that's actually based on-- and that's based only on the x1 and x2.
OK?
So I'm going to actually build a confidence interval.
But what I really want to build is a-- but let's start thinking about how I would find an estimator for those two things.
Well, what values am I going to be getting, right?
So I'm going to get either theta plus 1 or theta minus 1.
And actually, I can get basically four different observations, right?
Sorry, four different pairs of observations-- plus plus theta minus 1.
Agreed?
Those are the four possible observations that I can get.
Agreed?
Either they're both equal to plus 1, both equal to minus 1, or one of the two is equal to plus 1, the other one to minus 1, or the epsilons.
OK.
So those are the four observations I can get.
So in particular, if they take the same value, and you know it's either theta plus 1 or theta minus 1, and if they take a different value, I know one of them is theta plus 1, and one is actually theta minus 1.
So in particular, if I take the average of those two guys, when they take different values, I know I'm actually getting theta right.
So let's build a confidence region.
OK, so I'm actually going to take a confidence region, which is just a singleton.
And I'm going to say the following.
Well, if x1 is equal to x2, I'm just going to take x1 minus 1, OK?
So I'm just saying, well, I'm never going to able to resolve whether it's plus 1 or minus 1 that actually gives me the best one, so I'm just going to take a dive and say, well, it's just plus 1.
OK?
And then, if they're different, then here, I can do much better.
I'm going to actually just think the average.
OK?
Now, what I claim is that this is a confidence region-- and by default, when I don't mention it, this is a frequentist confidence region-- at level 75%.
OK?
So let's just check that.
To check that this is correct, I need to check that the probability under the realization of x1 and x2, that theta belongs, is one of those two guys, is actually equal to 0.75.
Yes?
What are the [INAUDIBLE]
Well, it's just the frequentist confidence interval that does not need to be an interval.
Actually, in this case, it's going to be an interval.
But that's just what it means.
Yeah, region for Bayesian was just because-- I mean, the confidence intervals, when we're frequentist, we tend to make them intervals, because we want-- but when you're Bayesian, and you're doing this level set thing, you cannot really guarantee, unless its [INAUDIBLE] is going to be an interval.
So region is just a way to not have to say interval, in case it's not.
OK.
So I have this thing.
So what I need to check is the probability that theta is in one of those two things, right?
So what I need to find is the probability that theta is an [INAUDIBLE] Well, x1 minus 1 and x1 is not equal to x2.
And those are disjoint events, so it's plus the probability that theta is in x1 plus x2 over 2 and x1-- sorry, that's equal.
That's different.
OK. And OK, just before we actually finish the computation, why do I have 75%?
75% is 3/4.
So it means that we have four cases.
And essentially, I did not account for one case.
And it's true.
I did not account for this case, when the both of the epsilon i's are equal to minus 1.
Right?
So this is essentially the one I'm not going to be able to account for.
And so we'll see that in a second.
So in this case, we know that everything goes great.
Right?
So in this case, this is-- OK. Well, let's just start from the first line.
So the first line is the probability that theta is equal to x1 minus 1 and those two are equal.
So this is the probability that theta is equal to-- well, this is theta plus epsilon 1 minus 1.
And epsilon 1 is equal to epsilon 2, right?
Because I can remove the theta from here, and I can actually remove the theta from here, so that this guy here is just epsilon 1 is equal to 1.
So when I intersect with this guy, it's actually the same thing as epsilon 1 is equal to 1, as well-- episilon 2 is equal to 1, as well, OK?
So this first thing is actually equal to the probability that epsilon 1 is equal to 1 and epsilon 2 is equal to 1, which is equal to what?
[INAUDIBLE]
Yeah, 1/4, right?
So that's just the first case over there.
They're independent.
Now, I still need to do the second one.
So this case is what?
Well, when those things are equal, x1 plus x2 over 2 is what?
Well, I get theta plus theta over 2.
So that's just equal to the probability that epsilon 1 plus epsilon 2 over 2 is equal to 0 and epsilon 1 is different from epsilon 2.
Agreed?
I just removed the thetas from these equations, because I can.
They're just on both sides every time.
OK. And so that means what?
That means that the second part-- so this thing is actually equal to 1/4 plus the probability that epsilon 1 and epsilon 2 over 2 is equal to 0.
I can remove the 2.
So this is just the probability that one is 1, and the other one is minus 1, right?
So that's equal to the probability that epsilon 1 is equal to 1 and epsilon 2 is equal to minus 1 plus the probability that epsilon 1 is equal to minus 1 and epsilon 2 is equal to plus 1, OK?
Because they're disjoint events.
So I can break them into the sum of the two.
And each of those guys is also one of the atomic part of it.
It's one of the basic things.
And so each of those guys has probability 1/4.
And so here, we can really see that we accounted for everything, except for the case when epsilon 1 was equal to minus 1, and epsilon 2 was equal to minus 1.
So this is 1/4.
This is 1/4.
So the whole thing is equal to 3/4.
So now, what we have is that the probability that epsilon 1 is in-- so the probability that data belongs to this confidence region is equal to 3/4.
And that's very nice.
But the thing is some people are sort of-- I mean, it's not super nice to be able to see this, because, in a way, I know that, if I observe x1 and x2 that are different, I know for sure that theta, that I actually got the right theta, right?
That this confidence interval is actually happening with probability 1.
And the problem is that I do not know-- I cannot make this precise with the notion of frequentist confidence intervals.
OK?
Because frequentist confidence intervals have to account for the fact that, in the future, it might not be the case that x1 and x2 are different.
So Bayesian confidence regions, by definition-- well, they're all gone-- but they are conditioned on the data that I have.
And so that's what I want.
I want to be able to make this statement conditionally and the data that I have.
OK.
So if I want to be able to make this statement, if I want to build a Bayesian confidence region, I'm going to have to put a prior on theta.
So without loss of generality-- I mean, maybe with-- but let's assume that pi is a prior on theta.
And let's assume that pi of j is strictly positive for all integers j equal, say, 0-- well, actually, for all j in the integers, positive or negative.
OK.
So that's a pretty weak assumption on my prior.
I'm just assuming that theta is some integer.
And now, let's build our Bayesian confidence region.
Well, if I want to build a Bayesian confidence region, I need to understand what my posterior is going to be.
OK?
And if I want to understand what my posterior is going to be, I actually need to build a likelihood, right?
So we know that it's the product of the likelihood and of the prior divided by-- OK.
So what is my likelihood?
So my likelihood is the probability of x1 x2, given theta.
Right?
That's what the likelihood should be.
And now let's say that actually, just to make things a little simpler, let us assume that x1 is equal to, I don't know, 5, and x2 is equal to 7.
OK?
So I'm not going to take the case where they're actually equal to each other, because I know that, in this case, x1 and x2 are different.
I know I'm going to actually nail exactly what theta is, by looking at the average of those guys, right?
Here, it must be that theta is equal to 6.
So what I want is to compute the likelihood at 5 and 7, OK?
And what is this likelihood?
Well, if theta is equal to 6, that's just the probability that I will observe 5 and 7, right?
So what is the probability I observe 5 and 7?
Yeah?
1?
1/4
That's 1/4, right?
As the probability, I have minus 1 for the first epsilon 1, right?
So this is infinity 6.
This is the probability that epsilon 1 is equal to minus 1, and epsilon 2 is equal to plus 1, which is equal to 1/4.
So this probability is 1/4.
If theta is different from 6, what is this probability?
So if theta is different from 6, since we know that we've only loaded the integers-- so if theta has to be another integer, what is the probability that I see 5 and 7?
0
0.
So that's my likelihood.
And if I want to know what my posterior is, well, it's just pi of theta times p of 5/6, given theta, divided by the sum over all T's, say, in Z.
Right?
So now, I just need to normalize this thing.
So of pi of T, p of 4/6, given T. Agreed?
That's just the definition of the posterior.
But when I sum these guys, there's only one that counts, because, for those things, we know that this is actually equal to 0 for every T, except for when T is equal to 6.
So this entire sum here is actually equal to pi of 6 times p of 5/6-- sorry, 5/7, of 5/7, given that theta is equal to 6, which we know is equal to 1/4.
And I did not tell you what pi of 6 was.
But it's the same thing here.
The posterior for any theta that's not 6 is actually going to be-- this guy's going to be equal to 0.
So I really don't care what this guy is.
So what it means is that my posterior becomes what?
It becomes the posterior pi of theta, given 5 and 7 is equal to-- well, when theta is not equal to 6, this is actually 0.
So regardless of what I do here, I get something which is 0.
And if theta is equal to 6, what I get is pi of 6 times p of 5/7, given 6, which I've just computed here, which is 1/4 divided by pi of 6 times 1/4.
So it's the ratio of two things that are identical.
So I get 1.
So now, my posterior tells me that, given that I observe 5 and 7, theta has to be 1 with probability-- has to be 6 with probability 1.
So now, I say that this thing here-- so now, this is not something that actually makes sense when I talk about frequentist confidence intervals.
They don't really make sense, to talk about confidence intervals, given something.
And so now, given that I observe 5 and 7, I know that the probability of theta is equal to 1.
And in this sense, the Bayesian confidence interval is actually more meaningful.
So one thing I want to actually say about this Bayesian confidence interval is that it's-- I mean, here, it's equal to the value 1, right?
So it really encompasses the thing that we want.
But the fact that we actually computed it using the Bayesian posterior and the Bayesian rule did not really matter for this argument.
All I just said was that it had a prior.
But just what I want to illustrate is the fact that we can actually give a meaning to the probability that theta is equal to 6, given that I see 5 and 7.
Whereas, we cannot really in the other cases.
And we don't have to be particularly precise in the prior and theta to be able to give theta this-- to give this meaning.
OK?
All right.
So now, as I said, I think the main power of Bayesian inference is that it spits out the posterior distribution, and not just the single number, like frequentists would give you.
Then we can say decorate, or theta hat, or point estimate, with maybe some confidence interval.
Maybe we can do a bunch of tests.
But at the end of the day, we just have, essentially, one number, right?
Then maybe we can understand where the fluctuations of this number are in a frequentist setup.
but the Bayesian framework is essentially giving you a natural method.
And you can interpret it in terms of the probabilities that are associated to the prior.
But you can actually also try to make some-- so a Bayesian, if you give me any prior, you're going to actually build an estimator from this prior, maybe from the posterior.
And maybe it's going to have some frequentist properties.
And that's what's really nice about [?
Bayesians, ?]
is that you can actually try to give some frequentist properties of Bayesian methods, that are built using Bayesian methodology.
But you cannot really go the other way around.
If I give you a frequency methodology, how are you going to say something about the fact that there's a prior going on, et cetera?
And so this is actually one of the things there's actually some research that's going on for this.
They call it Bayesian posterior concentration.
And one of the things-- so there's something called the Bernstein-von Mises theorem.
And those are a class of theorems, and those are essentially methods that tell you, well, if I actually run a Bayesian method, and I look at the posterior that I get-- it's going to be something like this-- but now, I try to study this in a frequentist point of view, there's actually a true parameter of theta somewhere, the true one.
There's no prior for this guy.
This is just one fixed number.
Is it true that as my sample size is going to go to infinity, then this thing is going to concentrate around theta?
And the rate of concentration of this thing, the size of this width, the standard deviation of this thing, is something that should decay maybe like 1 over square root of n, or something like this.
And the rate of posterior concentration, when you characterize it, it's called the Bernstein-von Mises theorem.
And so people are looking at this in some non-parametric cases.
You can do it in pretty much everything we've been doing before.
You can do it for non-parametric regression estimation or density estimation.
You can do it for, of course-- you can do it for sparse estimation, if you want.
OK.
So you can actually compute the procedure and-- yeah.
And so you can think of it as being just a method somehow.
Now, the estimator I'm talking about-- so that's just a general Bayesian posterior concentration.
But you can also try to understand what is the property of something that's extracted from this posterior.
And one thing that we actually describe was, for example, well, given this guy, maybe it's a good idea to think about what the mean of this thing is, right?
So there's going to be some theta hat, which is just the integral of theta pi theta, given x1 xn-- so that's my posterior-- d theta.
Right?
So that's the posterior mean.
That's the expected value with respect to the posterior distribution.
And I want to know how does this thing behave, how close it is to a true theta if I actually am in a frequency setup.
So that's the posterior mean.
But this is not the only thing I can actually spit out, right?
This is definitely uniquely defined.
If you give me a distribution, I can actually spit out its posterior mean.
But I can also think of the posterior median.
But now, if this is not continuous, you might have some uncertainty.
Maybe the median is not uniquely defined, and so maybe that's not something you use as much.
Maybe you can actually talk about the posterior mode.
All right, so for example, if you're posterior density looks like this, then maybe you just want to summarize your posterior with this number.
So clearly, in this case, it's not such a good idea, because you completely forget about this mode.
But maybe that's what you want to do.
Maybe you want to focus on the most peak mode.
And this is actually called maximum a posteriori.
As I said, maybe you want a sample from this posterior distribution.
OK, and so in all these cases, these Bayesian estimators will depend on the prior distribution.
And the hope is that, as the sample size grows, you won't see that again.
OK.
So to conclude, let's just do a couple of experiments.
So if I look at-- did we do this?
Yes.
So for example, so let's focus on the posterior mean.
And we know-- so remember in experiment one-- [INAUDIBLE] example one, what we had was x1 xn that were [?
iid, ?]
Bernoulli p, and the prior I put on p was a beta with parameter aa.
OK?
And if I go back to what we computed, you can actually compute the posterior of this thing.
And we know that it's actually going to be-- sorry, that was uniform?
Where is-- yeah.
So what we get is that the posterior, this thing is actually going to be a beta with parameter a plus the sum, so a plus the number of 1s and a plus the number of 0s.
OK?
And the beta was just something that looked like-- the density was p to the a minus 1, 1 minus p. OK?
So if I want to understand the posterior mean, I need to be able to compute the expectation of a beta, and then maybe plug in a for a plus this guy and minus this guy.
OK.
So actually, let me do this.
OK.
So what is the expectation?
So what I want is something that looks like the integral between 0 and 1 of p times a minus 1-- sorry, p times p a minus 1, 1 minus p, b minus 1.
Do we agree that this-- and then there's a normalizing constant.
Let's call it c. OK?
So this is what I need to compute.
So that's c of a and b.
Do we agree that this is the posterior mean with respect to a beta with parameters a and b?
Right?
I just integrate p against the density.
So what does this thing look like?
Well, I can actually move this guy in here.
And here, I'm going to have a plus 1 minus 1.
OK?
So the problem is that this thing is actually-- the constant is going to play a big role, right?
Because this is essentially equal to c a plus 1b divided by c ab, where ca plus 1b is just the normalizing constant of a beta a plus 1 b.
So I need to know the ratio of those two constants.
And this is not something-- I mean, this is just a calculus exercise.
So in this case, what you get is-- sorry.
In this case, you get-- well, OK, so we get essentially a divided by, I think, it's a plus b. Yeah, it's a plus b.
So that's this quantity.
OK?
And when I plug in a to be this guy and b to be this guy, what I get is a plus sum of the xi.
And then I get a plus this guy, a plus n minus this guy.
So those two guys go away, and I'm left with 2a plus n, which does not work.
No, that actually works.
And so now what I do, I can actually divide and get this thing, over there.
OK.
So what you can see, the reason why this thing has been divided is that you can really see that, as n goes to infinity, then this thing behaves like xn bar, which is our frequentist estimator.
The effect of a is actually going away.
The effect of the prior, which is completely captured by a, is going away as n goes to infinity.
Is there any question?
You guys have a question.
What is it?
Do you have a question?
Yeah, on the board, is that divided by some [INAUDIBLE] stuff?
Is that divided by what?
That a over a plus b, and then you just expanded--
Oh yeah, yeah, then I said that this is equal to this, right.
So that's for a becomes a plus sum of the xi's, and b becomes a plus n minus sum of the xi's.
OK.
So that's just for the posterior one
What's [INAUDIBLE]
This guy?
Yeah
2a
2a.
Oh,
Right.
So I get a plus a plus n. And then those two guys cancel.
OK?
And that's what you have here.
So for a is equal to 1/2-- and I claim that this is Jeffreys prior.
Because remember, Jeffreys was [INAUDIBLE] was square root and was proportional to the square root of p1 minus p, which I can write as p to the 1/2, 1 minus p to the 1/2.
So it's just the case a is equal to 1/2.
OK.
So if I use Jeffreys prior, I just plug in a equals to 1/2, and this is what I get.
OK?
So those things are going to have an impact again when n is moderately large.
For large n, those things, whether you take Jeffreys prior or you take whatever a you prefer, it's going to have no impact whatsoever.
But n is of the order of 10 maybe, then you're going to start to see some impact, depending on what a you want to pick.
OK. And then in the second example, well, here we actually computed the posterior to be this guy.
Well, here, I can just read off what the expectation is, right?
I mean, I don't have to actually compute the expectation of a Gaussian.
It's just that xn bar.
And so in this case, there's actually no-- I mean, when I have a non-informative prior for a Gaussian, then I have basically xn in bar.
As you can see, actually, this is an interesting example.
When I actually look at the posterior, it's not something that cost me a lot to communicate to you, right?
There's one symbol here, one symbol here, and one symbol here.
I tell you the posterior is a Gaussian with mean xn bar and variance 1/n.
When I actually turn that into a poster mean, I'm dropping all this information.
I'm just giving you the first parameter.
So you can see there's actually much more information in the posterior than there is in the posterior mean.
The posterior mean is just a point.
It's not telling me how confident I am in this point.
And this thing is actually very interesting.
OK.
So you can talk about the posterior variance that's associated to it, right?
You can talk about, as an output, you could give the posterior mean and posterior variance.
And those things are actually interesting.
All right.
So I think this is it.
So as I said, in general, just like in this case, the impact of the prior is being washed away as the sample size goes to infinity.
Just well, like here, there's no impact of the prior.
It was an noninvasive one.
But if you actually had an informative one, [?
CF ?]
homework-- yeah?
[INAUDIBLE]
Yeah, so [?
CF ?]
homework, you would actually see an impact of the prior, which, again, would be washed away as your sample size increases.
Here, it goes away.
You just get xn bar over 1.
And actually, in these cases, you see that the posterior distribution converges to-- sorry, the Bayesian estimator is asymptotically normal.
This is different from the distribution of the posterior, right?
This is just the posterior mean, which happens to be asymptotically normal.
But the posterior may not have a-- I mean, here, the posterior is a beta, right?
I mean, it's not normal.
OK, so there's different-- those things are two different things.
Your question?
What was the prior [INAUDIBLE]
All 1, right?
That was the improper prior
OK. And so that would give you the same thing as [INAUDIBLE],, not just the proportion
Well, I mean, yeah.
So it's essentially telling you that-- so we said that, when you have a non-informative prior, essentially, the maximum likelihood is the maximum a posteriori, right?
But in this case, there's so much symmetry, that it just so happens that the maximum in this thing is completely symmetric around its maximum.
So it means that the expectation is equal to the maximum, to [INAUDIBLE] max.
Yeah?
I read somewhere that one of the issues with Bayesian methods is that we choose the wrong prior, and it could mess up your results
Yeah, but hence, do not pick the wrong prior.
I mean, of course, it would.
I mean, it would mess up your res-- of course.
I mean, you're putting extra information.
But you could say the same thing by saying, well, the issue with frequentist method is that, if you mess up the choice of your likelihood, then it's going to mess up your output.
So here, you just have two chances of messing it up, right?
You have the-- well, it's gone.
So you have the product of the likelihood and the prior, and you have one more chance to-- but it's true, if you assume that the model is right, then, of course, finding the wrong prior could completely mess up things if your prior, for example, has no support on the true parameter.
But if your prior has a positive weight on the true parameter as n goes to infinity-- I mean, OK, I cannot speak for all counterexamples in the world.
But I'm sure, under minor technical conditions, you can guarantee that your posterior mean is going to converge to what you need it to converge to.
Any other question?
All right.
So I think this closes the more traditional mathematical-- not mathematical, but traditional statistics part of this class.
And from here on, we'll talk about more multivariate statistics, starting with principal component analysis.
So that's more like when you have multiple data.
We started, in a way, to talk about multivariate statistics when we talked about multivariate regression.
But we'll move on to principal component analysis.
I'll talk a bit about multiple testing.
I haven't made my mind yet about what we'll talk really in December.
But I want to make sure that you have a taste and a flavor of what is being interesting in statistics these days, especially as you go towards more [INAUDIBLE] learning type of questions, where really, the focus is on prediction rather than the modeling itself.
We'll talk about logistic regression, as well, for example, which is generalized linear models, which is just the generalization in the case that y does not take value in the whole real line, maybe 0,1, for example, for regression.
All right.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
--bunch of x's and a bunch of y's.
The y's were univariate, just one real valued random variable.
And the x's were vectors that described a bunch of attributes for each of our individuals or each of our observations.
Let's assume now that we're given essentially only the x's.
This is sometimes referred to as unsupervised learning.
There is just the x's.
Usually, supervision is done by the y's.
And so what you're trying to do is to make sense of this data.
You're going to try to understand this data, represent this data, visualize this data, try to understand something, right?
So, if I give you a d-dimensional random vectors, and you're going to have n independent copies of this individual-- of this random vector, OK?
So you will see that I'm going to have-- I'm going to very quickly run into some limitations about what I can actually draw on the board because I'm using [?
boldface ?]
here.
I'm also going to use the blackboard [?
boldface.
?]
So it's going to be a bit difficult.
So tell me if you're actually a little confused by what is a vector, what is a number, and what is a matrix.
But we'll get there.
So I have X in Rd, and that's a random vector.
And I have X1 to Xn that are IID.
They're independent copies of X. OK, so you can think of those as being-- the realization of these guys are going to be a cloud of n points in R to the d. And we're going to think of d as being fairly large.
And for this to start to make sense, we're going to think of d as being at least 4, OK?
And meaning that you're going to have a hard time visualizing those things.
If it was 3 or 2, you would be able to draw these points.
And that's pretty much as much sense you're going to be making about those guys, just looking at the [INAUDIBLE] All right, so I'm going to write each of those X's, right?
So this vector, X, has d coordinate.
And I'm going to write them as X1, to Xd.
And I'm going to stack them into a matrix, OK?
So once I have those guys, I'm going to have a matrix.
But here, I'm going to use the double bar.
And it's X1 transpose, Xn transpose.
So what it means is that the coordinates of this guy, of course, are X1,1.
Here, I have-- I'm of size d, so I have X1d.
And here, I have Xn1.
Xnd.
And so the i-th, j-th-- i-th row and j-th column is the matrix, Xij, right-- is the entry, Xi to-- sorry.
OK, so each-- so the rows here are the observations.
And the columns are the covariance over attributes.
OK?
So this is an n by d matrix.
All right, this is really just some bookkeeping.
How do we store this data somehow?
And the fact that we use a matrix just like for regression is going to be convenient because we're going to able to talk about projections-- going to be able to talk about things like this.
All right, so everything I'm going to say now is about variances or covariances of those things, which means that I need two moments, OK?
If the variance does not exist, there's nothing I can say about this problem.
So I'm going to assume that the variance exists.
And one way to just put it to say that the two norm of those guys is finite, which is another way to say that each of them is finite.
I mean, you can think of it the way you want.
All right, so now, the mean of X, right?
So I have a random vector.
So I can talk about the expectation of X.
That's a vector that's in Rd.
And that's just taking the expectation entrywise.
Sorry.
X1, Xd.
OK, so I should say it out loud.
For this, the purpose of this class, I will denote by subscripts the indices that corresponds to observations.
And superscripts, the indices that correspond to coordinates of a variable.
And I think that's the same convention that we took for the regression case.
Of course, you could use whatever you want.
If you want to put commas, et cetera, it becomes just a bit more complicated.
All right, and so now, once I have this, so this tells me where my cloud of point is centered, right?
So if I have a bunch of points-- OK, so now I have a distribution on Rd, so maybe I should talk about this-- I'll talk about this when we talk about the empirical version.
But if you think that you have, say, a two-dimensional Gaussian random variable, then you have a center in two dimension, which is where it peaks, basically.
And that's what we're talking about here.
But the other thing we want to know is how much does it spread in every direction, right?
So in every direction of the two dimensional thing, I can then try to understand how much spread I'm getting.
And the way you measure this is by using covariance, right?
So the covariance matrix, sigma-- that's a matrix which is d by d. And it records-- in the j, k-th entry, it records the covariance between the j-th coordinate of X and the k-th coordinate of X, OK?
So with entries-- OK, so I have sigma, which is sigma 1,1, sigma dd, sigma 1d, sigma d1.
OK, and here I have sigma jk And sigma jk is just the covariance between Xj, the j-th coordinate and the k-th coordinate.
OK?
So in particular, it's symmetric because the covariance between Xj and Xk is the same as the covariance between Xk and Xj.
I should not put those parentheses here.
I do not use them in this, OK?
Just the covariance matrix.
So that's just something that records everything.
And so what's nice about the covariance matrix is that if I actually give you X as a vector, you actually can build the matrix just by looking at vectors times vectors transpose, rather than actually thinking about building it coordinate by coordinate.
So for example, if you're used to using MATLAB, that's the way you want to build a covariance matrix because MATLAB is good at manipulating vectors and matrices rather than just entering it entry by entry.
OK, so, right?
So, what is the covariance between Xj and Xk?
Well by definition, it's the expectation of Xj and Xk minus the expectation of Xj times the expectation of Xk, right?
That's the definition of the covariance.
I hope everybody's seeing that.
And so, in particular, I can actually see that this thing can be written as-- sigma can now be written as the expectation of XX transpose minus the expectation of X times the expectation of X transpose.
Why?
Well, let's look at the jk-th coefficient of this guy, right?
So here, if I look at the jk-th coefficient, I see what?
Well, I see that it's the expectation of XX transpose jk, which is equal to the expectation of XX transpose jk.
And what are the entries of XX transpose?
Well, they're of the form, Xj times Xk exactly.
So this is actually equal to the expectation of Xj times Xk.
And this is actually not the way I want to write it.
I want to write it-- OK?
Is that clear?
That when I have a rank 1 matrix of this form, XX transpose, the entries are of this form, right?
Because if I take-- for example, think about x, y, z, and then I multiply by x, y, z.
What I'm getting here is x-- maybe I should actually use indices here.
x1, x2, x3.
x1, x2, x3.
The entries are x1x1, x1x2, x1x3; x2x1, x2x2, x2x3; x3x1, x3x2, x3x3, OK?
So indeed, this is exactly of the form if you look at jk, you get exactly Xj times Xk, OK?
So that's the beauty of those matrices.
So now, once I have this, I can do exactly the same thing, except that here, if I take the jk-th entry, I will get exactly the same thing, except that it's not going to be the expectation of the product, but the product of the expectation, right?
So I get that the jk-th entry of E of X, E of X transpose, is just the j-th entry of E of X times the k-th entry of E of X.
So if I put those two together, it's actually telling me that if I look at the j, k-th entry of sigma, which I called little sigma jk, then this is actually equal to what?
It's equal to the first term minus the second term.
The first term is the expectation of Xj, Xk minus the expectation of Xj, expectation of Xk, which-- oh, by the way, I forgot to say this is actually equal to the expectation of Xj times the expectation of Xk because that's just the definition of the expectation of random vectors.
So my j and my k are now inside.
And that's by definition the covariance between Xj and Xk, OK?
So just if you've seen those manipulations between vectors, hopefully you're bored out of your mind.
And if you have not, then that's something you just need to get comfortable with, right?
So one thing that's going to be useful is to know very quickly what's called the outer product of a vector with itself, which is the vector of times the vector transpose, what the entries of these things are.
And that's what we've been using on this second set of boards.
OK, so everybody agrees now that we've sort of showed that the covariance matrix can be written in this vector form.
So expectation of XX transpose minus expectation of X, expectation of X transpose.
OK, just like the covariance can be written in two ways, right we know that the covariance can also be written as the expectation of Xj minus expectation of Xj times Xk minus expectation of Xk, right?
That's the-- sometimes, this is the original definition of covariance.
This is the second definition of covariance.
Just like you have the variance which is the expectation of the square of X minus c of X, or the expectation X squared minus the expectation of X squared.
It's the same thing for covariance.
And you can actually see this in terms of vectors, right?
So this actually implies that you can also rewrite sigma as the expectation of X minus expectation of X times the same thing transpose.
Right?
And the reason is because if you just distribute those guys, this is just the expectation of XX transpose minus X, expectation of X transpose minus expectation of XX transpose.
And then I have plus expectation of X, expectation of X transpose.
Now, things could go wrong because the main difference between matrices slash vectors and numbers is that multiplication does not commute, right?
So in particular, those two things are not the same thing.
And so that's the main difference that we have before, but it actually does not matter for our problem.
It's because what's happening is that if when I take the expectation of this guy, then it's actually the same as the expectation of this guy, OK?
And so just because the expectation is linear-- so what we have is that sigma now becomes equal to the expectation of XX transpose minus the expectation of X, expectation of X transpose minus expectation of X, expectation of X transpose.
And then I have-- well, really, what I have is this guy.
And then I have plus the expectation of X, expectation of X transpose.
And now, those three things are actually equal to each other just because the expectation of X transpose is the same as the expectation of X transpose.
And so what I'm left with is just the expectation of XX transpose minus the expectation of X, expectation of X transpose, OK?
So same thing that's happening when you want to prove that you can write the covariance either this way or that way.
The same thing happens for matrices, or for vectors, right, or a covariance matrix.
They go together.
Is there any questions so far?
And if you have some, please tell me, because I want to-- I don't know to which extent you guys are comfortable with this at all or not.
OK, so let's move on.
All right, so of course, this is what I'm describing in terms of the distribution right here.
I took expectations.
Covariances are also expectations.
So those depend on some distribution of X, right?
If I wanted to compute that, I would basically need to know what the distribution of X is.
Now, we're doing statistics, so I need to [INAUDIBLE] my question is going to be to say, well, how well can I estimate the covariance matrix itself, or some properties of this covariance matrix based on data?
All right, so if I want to understand what my covariance matrix looks like based on data, I'm going to have to basically form its empirical counterparts, which I can do by doing the age-old statistical trick, which is replace your expectation by an average, all right?
So let's just-- everything that's on the board, you see expectation, just replace it by an average.
OK, so, now I'm going to be given X1, Xn.
So, I'm going to define the empirical mean.
OK so, really, the idea is take your expectation and replace it by 1 over n sum, right?
And so the empirical mean is just 1 over n. Some of the Xi's-- I'm guessing everybody knows how to average vectors.
It's just the average of the coordinates.
So I will write this as X bar.
And the empirical covariance matrix, often called sample covariance matrix, hence the notation, S. Well, this is my covariance matrix, right?
Let's just replace the expectations by averages.
1 over n, sum from i equal 1 to n, of Xi, Xi transpose, minus-- this is the expectation of X. I will replace it by the average, which I just called X bar, X bar transpose, OK?
And that's when I want to use the-- that's when I want to use the notation-- the second definition, but I could actually do exactly the same thing using this definition here.
Sorry, using this definition right here.
So this is actually 1 over n, sum from i equal 1 to n, of Xi minus X bar, Xi minus X bar transpose.
And those are actually-- I mean, in a way, it looks like I could define two different estimators, but you can actually check.
And I do encourage you to do this.
If you're not comfortable making those manipulations, you can actually check that those two things are actually exactly the same, OK?
So now, I'm going to want to talk about matrices, OK?
And remember, we defined this big matrix, X, with the double bar.
And the question is, can I express both X bar and the sample covariance matrix in terms of this big matrix, X?
Because right now, it's still expressed in terms of the vectors.
I'm summing those vectors, vectors transpose.
The question is, can I just do that in a very compact way, in a way that I can actually remove this sum term, all right?
That's going to be the goal.
I mean, that's not a notational goal.
That's really something that we want-- that's going to be convenient for us just like it was convenient to talk about matrices when we did linear regression.
OK, X bar.
We just said it's 1 over n, sum from I equal 1 to n of Xi, right?
Now remember, what does this matrix look like?
We said that X bar-- X is this guy.
So if I look at X transpose, the columns of this guy becomes X1, my first observation, X2, my second observation, all the way to Xn, my last observation, right?
Agreed?
That's what X transpose is.
So if I want to sum those guys, I can multiply by the all-ones vector.
All right, so that's what the definition of the all-ones 1 vector is.
Well, it's just a bunch of 1's in Rn, in this case.
And so when I do X transpose 1, what I get is just the sum from i equal 1 to n of the Xi's.
So if I divide by n, I get my average, OK?
So here, I definitely removed the sum term.
Let's see if with the covariance matrix, we can do the same.
Well, and that's actually a little more difficult to see, I guess.
But let's use this definition for S, OK?
And one thing that's actually going to be-- so, let's see for one second, what-- so it's going to be something that involves X, multiplying X with itself, OK?
And the question is, is it going to be multiplying X with X transpose, or X tranpose with X?
To answer this question, you can go the easy route, which says, well, my covariance matrix is of size, what?
What is the size of S?
d by d
d by d, OK?
X is of size n by d. So if I do X times X transpose, I'm going to have something which is of size n by n. If I do X transpose X, I'm going to have something which is d by d. That's the easy route.
And there's basically one of the two guys.
You can actually open the box a little bit and see what's going on in there.
If you do X transpose X, which we know gives you a d by d, you'll see that X is going to have vectors that are of the form, Xi, and X transpose is going to have vectors that are of the form, Xi transpose, right?
And so, this is actually probably the right way to go.
So let's look at what's X transpose X is giving us.
So I claim that it's actually going to give us what we want, but rather than actually going there, let's-- to actually-- I mean, we could check it entry by entry, but there's actually a nice thing we can do.
Before we go there, let's write X transpose as the following sum of variables, X1 and then just a bunch of 0's everywhere else.
So it's still d by n. So n minus 1 of the columns are equal to 0 here.
Then I'm going to put a 0 and then put X2.
And then just a bunch of 0's, right?
So that's just 0, 0 plus 0, 0, all the way to Xn, OK?
Everybody agrees with it?
See what I'm doing here?
I'm just splitting it into a sum of matrices that only have one nonzero columns.
But clearly, that's true.
Now let's look at the product of this guy with itself.
So, let's call these matrices M1, M2, Mn.
So when I do X transpose X, what I do is the sum of the Mi's for i equal 1 to n, times the sum of the Mi transpose, right?
Now, the sum of the Mi's transpose is just the sum of each of the Mi's transpose, OK?
So now I just have this product of two sums, so I'm just going to re-index the second one by j.
So this is sum for i equal 1 to n, j equal 1 to n of Mi Mj transpose.
OK?
And now what we want to notice is that if i is different from j, what's happening?
Well if i is different from j, let's look at say, M1 times XM2 transpose.
So what is the product between those two matrices?
It's a new entry and [INAUDIBLE]
There's an entry?
Well, it's an entry.
It's like a dot product in that form next to [?
transpose.
?]
You mean a dot product is just getting [INAUDIBLE] number, right?
So I want-- this is going to be a matrix.
It's the product of two matrices, right?
This is a matrix times a matrix.
So this should be a matrix, right, of size d by d. Yeah, I should see a lot of hands that look like this, right?
Because look at this.
So let's multiply the first-- let's look at what's going on in the first column here.
I'm multiplying this column with each of those rows.
The only nonzero coefficient is here, and it only hits this column of 0's.
So every time, this is going to give you 0, 0, 0, 0.
And it's going to be the same for every single one of them.
So this matrix is just full of 0's, right?
They never hit each other when I do the matrix-matrix multiplication.
There's no-- every non-zero hits a 0.
So what it means is-- and this, of course, you can check for every i different from j.
So this means that Mi times Mj transpose is actually equal to 0 when i is different from j, Right?
Everybody is OK with this?
So what that means is that when I do this double sum, really, it's a simple sum.
There's only just the sum from i equal 1 to n of Mi Mi transpose.
Because this is the only terms in this double sum that are not going to be 0 when [INAUDIBLE] [?
M1 ?]
with M1 itself.
Now, let's see what's going on when I do M1 times M1 transpose.
Well, now, if I do Mi times and Mi transpose, now this guy becomes [?
X1 ?]
[INAUDIBLE] it's here.
And so now, I really have X1 times X1 transpose.
So this is really just the sum from i equal 1 to n of Xi Xi transpose, just because Mi Mi transpose is Xi Xi transpose.
There's nothing else there.
So that's the good news, right?
This term here is really just X transpose X divided by n. OK, I can use that guy again, I guess.
Well, no.
Let's just-- OK, so let me rewrite S. All right, that's the definition we have.
And we know that this guy already is equal to 1 over n X transpose X. x bar x bar transpose-- we know that x bar-- we just proved that x bar-- sorry, little x bar was equal to 1 over n X bar transpose times the all-ones vector.
So I'm just going to do that.
So that's just going to be minus.
I'm going to pull my two 1 over n's-- one from this guy, one from this guy.
So I'm going to get 1 over n squared.
And then I'm going to get X bar-- sorry, there's no X bar here.
It's just X. Yeah.
X transpose all ones times X transpose all ones transpose, right?
And X transpose all ones transpose-- right, the rule-- if I have A times B transpose, it's B transpose times A transpose, right?
That's just the rule of transposition.
So this is 1 transpose X transpose.
And so when I put all these guys together, this is actually equal to 1 over n X transpose X minus one over n squared X transpose 1, 1 transpose X.
Because X transpose transposes X, OK?
So now, I can actually-- I have something which is of the form, X transpose X-- [INAUDIBLE] to the left, X transpose; to the right, X.
Here, I have X transpose to the left, X to the right.
So it can factor out whatever's in there.
So I can write S as 1 over n-- sorry, X transpose times 1 over n times the identity of Rd.
And then I have minus 1 over n, 1, 1 transpose X. OK, because if you-- I mean, you can distribute it back, right?
So here, I'm going to get what?
X transpose identity times X, the whole thing divided by n. That's this term.
And then the second one is going to be-- sorry, 1 over n squared.
And then I'm going to get 1 over n squared times X transpose 1, 1 transpose which is this guy, times X, and that's the [?
right ?]
[?
thing, ?]
OK?
So, the way it's written, I factored out one of the 1 over n's.
So I'm just going to do the same thing as on this slide.
So I'm just factoring out this 1 over n here.
So it's 1 over n times X transpose identity of our d divided by n divided by 1 this time, minus 1 over n 1, 1 transpose times X, OK?
So that's just what's on the slides.
What does the matrix, 1, 1 transpose, look like?
All 1's
It's just all 1's, right?
Because the entries are the products of the all-ones-- of the coordinates of the all-ones vectors with the coordinates of the all-ones vectors, so I only get 1's.
So it's a d by d matrix with only 1's.
So this matrix, I can actually write exactly, right?
H, this matrix that I called H which is what's sandwiched in-between this X transpose and X.
By definition, I said this is the definition of H. Then this thing, I can write its coordinates exactly.
We know it's identity divided by n minus-- sorry, I don't know why I keep [INAUDIBLE]..
Minus 1 over n 1, 1 transpose-- so it's this matrix with the only 1's on the diagonals and 0's and elsewhere-- minus a matrix that only has 1 over n everywhere.
OK, so the whole thing is 1 minus 1 over n on the diagonals and then minus 1 over n here, OK?
And now I claim that this matrix is an orthogonal projector.
Now, I'm writing this, but it's completely useless.
This is just a way for you to see that it's actually very convenient now to think about this problem as being a matrix problem, because things are much nicer when you think about the actual form of your matrices, right?
They could tell you, here is the matrix.
I mean, imagine you're sitting at a midterm, and I say, here's the matrix that has 1 minus 1 over n on the diagonals and minus 1 over n on the [INAUDIBLE] diagonal.
Prove to me that it's a projector matrix.
You're going to have to basically take this guy times itself.
It's going to be really complicated, right?
So we know it's symmetric.
That's for sure.
But the fact that it has this particular way of writing it is going to make my life super easy to check this.
That's the definition of a projector.
It has to be symmetric and it has to square to itself because we just said in the chapter on linear regression that once you project, if you apply the projection again, you're not moving because you're already there.
OK, so why is H squared equal to H?
Well let's just write H square.
It's the identity minus 1 over n 1, 1 transpose times the identity minus 1 over n 1, 1 transpose, right?
Let's just expand this now.
This is equal to the identity minus-- well, the identity times 1, 1 transpose is just the identity.
So it's 1, 1 transpose, sorry.
So 1 over n 1, 1 transpose minus 1 over n 1, 1 transpose.
And then there's going to be what makes the deal is that I get this 1 over n squared this time.
And then I get the product of 1 over n trans-- oh, let's write it completely.
I get 1, 1 transpose times 1, 1 transpose, OK?
But this thing here-- what is this?
n, right, is the end product of the all-ones vector with the all-ones vector.
So I'm just summing n times 1 squared, which is n. So this is equal to n. So I pull it out, cancel one of the ends, and I'm back to what I had before.
So I had identity minus 2 over n 1, 1 transpose plus 1 over n 1, 1 transpose which is equal to H. Because one of the 1 over n's cancel, OK?
So it's a projection matrix.
It's projecting onto some linear space, right?
It's taking a matrix.
Sorry, it's taking a vector and it's projecting onto a certain space of vectors.
What is this space?
Right, so, how do you-- so I'm only asking the answer to this question in words, right?
So how would you describe the vectors onto which this matrix is projecting?
Well, if you want to answer this question, the way you would tackle it is first by saying, OK, what does a vector which is of the form, H times something, look like, right?
What can I say about this vector that's going to be definitely giving me something about the space on which it projects?
I need to know a little more to know that it projects exactly onto this.
But one way we can do this is just see how it acts on a vector.
What does it do to a vector to apply H, right?
So I take v. And let's see what taking v and applying H to it looks like.
Well, it's the identity minus something.
So it takes v and it removes something from v. What does it remove?
Well, it's 1 over n times v transpose 1 times the all-ones vector, right?
Agreed?
I just wrote v transpose 1 instead of 1 transpose v, which are the same thing.
What is this thing?
What should I call it in mathematical notation?
v bar, right?
I should all it v bar because this is exactly the average of the entries of v, agreed?
This is summing the entries of v's, and this is dividing by the number of those v's.
Sorry, now v is in our-- sorry, why do I divide by-- I'm just-- OK, I need to check what my dimensions are now.
No, it's in Rd, right?
So why do I divide by n?
So it's not really v bar.
It's the sum of the v's divided by-- right, so it's v bar
[INAUDIBLE] [INTERPOSING VOICES]
Yeah, v has to be [INAUDIBLE]
Oh, yeah.
OK, thank you.
So everywhere I wrote Hd, that was actually Hn.
Oh, man.
I wish I had a computer now.
All right.
So-- yeah, because the-- yeah, right?
So why it's not-- well, why I thought it was this is because I was thinking about the outer dimension of X, really of X transpose, which is really the inner dimension, didn't matter to me, right?
So the thing that I can sandwich between X transpose and X has to be n by n. So this was actually n by n. And so that's actually n by n. Everything is n by n. Sorry about that.
So this is n. This is n. This is-- well, I didn't really tell you what the all-ones vector was, but it's also in our n. Yeah, OK.
Thank you.
And n-- actually, I used the fact that this was of size n here already.
OK, and so that's indeed v bar.
So what is this projection doing to a vector?
It's removing its average on each coordinate, right?
And the effect of this is that v is a vector.
What is the average of Hv?
0
Right, so it's 0.
It's the average of v, which is v bar, minus the average of something that only has v bar's entry, which is v bar.
So this thing is actually 0.
So let me repeat my question.
Onto what subspace does H project?
Onto the subspace of vectors that have mean 0.
A vector that has mean 0 is a vector.
So if you want to talk more linear algebra, v bar-- for a vector you have mean 0, it means that v is orthogonal to the span of the all-ones vector.
That's it.
It projects to this space.
So in words, it projects onto the space of vectors that have 0 mean.
In linear algebra, it says it projects onto the hyperplane which is orthogonal to the all-ones vector, OK?
So that's all.
Can you guys still see the screen?
Are you good over there?
OK. All right, so now, what it means is that, well, I'm doing this weird thing, right?
I'm taking the inner product-- so S is taking X.
And then it's removing its mean of each of the columns of X, right?
When I take H times X, I'm basically applying this projection which consists in removing the mean of all the X's.
And then I multiply by H transpose.
But what's actually nice is that, remember, H is a projector.
Sorry, I don't want to keep that.
Which means that when I look at X transpose HX, it's the same as looking at X transpose H squared X.
But since H is equal to its transpose, this is actually the same as looking at X transpose H transpose HX, which is the same as looking at HX transpose HX, OK?
So what it's doing, it's first applying this projection matrix, H, which removes the mean of each of your columns, and then looks at the inner products between those guys, right?
Each entry of this guy is just the covariance between those centered things.
That's all it's doing.
All right, so those are actually going to be the key statements.
So everything we've done so far is really mainly linear algebra, right?
I mean, looking at expectations and covariances was just-- we just used the fact that the expectation was linear.
We didn't do much.
But now there's a nice thing that's happening.
And that's why we're going to switch from the language of linear algebra to more statistical, because what's happening is that if I look at this quadratic form, right?
So I take sigma.
So I take a vector, u.
And I'm going to look at u-- so let's say, in Rd.
And I'm going to look at u transpose sigma u. OK?
What is this doing?
Well, we know that u transpose sigma u is equal to what?
Well, sigma is the expectation of XX transpose minus the expectation of X expectation of X transpose, right?
So I just substitute in there.
Now, u is deterministic.
So in particular, I can push it inside the expectation here, agreed?
And I can do the same from the right.
So here, when I push u transpose here, and u here, what I'm left with is the expectation of u transpose X times X transpose u. OK?
And now, I can do the same thing for this guy.
And this tells me that this is the expectation of u transpose X times the expectation of X transpose u.
Of course, u transpose X is equal to X transpose u.
And u-- yeah.
So what it means is that this is actually equal to the expectation of u transpose X squared minus the expectation of u transpose X, the whole thing squared.
But this is something that should look familiar.
This is really just the variance of this particular random variable which is of the form, u transpose X, right?
u transpose X is a number.
It involves a random vector, so it's a random variable.
And so it has a variance.
And this variance is exactly given by this formula.
So this is just the variance of u transpose X.
So what we've proved is that if I look at this guy, this is really just the variance of u transpose X, OK?
I can do the same thing for the sample variance.
So let's do this.
And as you can see, spoiler alert, this is going to be the sample variance.
OK, so remember, S is 1 over n, sum of Xi Xi transpose minus X bar X bar transpose.
So when I do u transpose, Su, what it gives me is 1 over n sum from i equal 1 to n of u transpose Xi times Xi transpose u, all right?
So those are two numbers that multiply each other and that happen to be equal to each other, minus u transpose X bar X bar transpose u, which is also the product of two numbers that happen to be equal to each other.
So I can rewrite this with squares.
So we're almost there.
All I need to know to check is that this thing is actually the average of those guys, right?
So u transpose X bar.
What is it?
It's 1 over n sum from i equal 1 to n of u transpose Xi.
So it's really something that I can write as u transpose X bar, right?
That's the average of those random variables of the form, u transpose Xi.
So what it means is that u transpose Su, I can write as 1 over n sum from i equal 1 to n of u transpose Xi squared minus u transpose X bar squared, which is the empirical variance that we need noted by small s squared, right?
So that's the empirical variance of u transpose X1 all the way to u transpose Xn.
OK, and here, same thing.
I use exactly the same thing.
I just use the fact that here, the only thing I use is really the linearity of this guy, of 1 over n sum or the linearity of expectation, that I can push things in there, OK?
So what you have written at the end of that sum for uT Su?
This one?
Yeah
Yeah, I said it's equal to small s, and I want to make a difference between the big S that I'm using here.
So this is equal to small-- I don't know, I'm trying to make it look like a calligraphic s squared.
OK, so this is nice, right?
This covariance matrix-- so let's look at capital sigma itself right now.
This covariance matrix, we know that if we read its entries, what we get is the covariance between the coordinates of the X's, right, of the random vector, X.
And the coordinates, well, by definition, are attached to a coordinate system.
So I only know what the covariance of X in of those two things are, or the covariance of those two things are.
But what if I want to find coordinates between linear combination of the X's?
Sorry, if I want to find covariances between linear combination of those X's.
And that's exactly what this allows me to do.
It says, well, if I pre- and post-multiply by u, this is actually telling me what the variance of X along direction u is, OK?
So there's a lot of information in there, and it's just really exploiting the fact that there is some linearity going on in the covariance.
So, why variance?
Why is variance interesting for us, right?
Why?
I started by saying, here, we're going to be interested in having something to do dimension reduction.
We have-- think of your points as [?
being in a ?]
dimension larger than 4, and we're going to try to reduce the dimension.
So let's just think for one second, what do we want about a dimension reduction procedure?
If I have all my points that live in, say, three dimensions, and I have one point here and one point here and one point here and one point here and one point here, and I decide to project them onto some plane-- that I take a plane that's just like this, what's going to happen is that those points are all going to project to the same point, right?
I'm just going to not see anything.
However, if I take a plane which is like this, they're all going to project into some nice line.
Maybe I can even project them onto a line and they will still be far apart from each other.
So that's what you want.
You want to be able to say, when I take my points and I say I project them onto lower dimensions, I do not want them to collapse into one single point.
I want them to be spread as possible in the direction on which I project.
And this is what we're going to try to do.
And of course, measuring spread between points can be done in many ways, right?
I mean, you could look at, I don't know, sum of pairwise distances between those guys.
You could look at some sort of energy.
You can look at many ways to measure of spread in a direction.
But variance is a good way to measure of spread between points.
If you have a lot of variance between your points, then chances are they're going to be spread.
Now, this is not always the case, right?
If I have a direction in which all my points are clumped onto one big point and one other big point, it's going to choose this because that's the direction that has a lot of variance.
But hopefully, the variance is going to spread things out nicely.
So the idea of principal component analysis is going to try to identify those variances-- those directions along which we have a lot of variance.
Reciprocally, we're going to try to eliminate the directions along which we do not have a lot of variance, OK?
And let's see why.
Well, if-- so here's the first claim.
If you transpose Su is equal to 0, what's happening?
Well, I know that an empirical variance is equal to 0.
What does it mean for an empirical variance to be equal to 0?
So I give you a bunch of points, right?
So those points are those points-- u transpose X1, u transpose-- those are a bunch of numbers.
What does it mean to have the empirical variance of those points being equal to 0?
They're all the same
They're all the same.
So what it means is that when I have my points, right?
So, can you find a direction for those points in which they project to all the same point?
No, right?
There's no such thing.
For this to happen, you have to have your points which are perfectly aligned.
And then when you're going to project onto the orthogonal of this guy, they're going to all project to the same point here, which means that the empirical variance is going to be 0.
Now, this is an extreme case.
This will never happen in practice, because if that happens, well, I mean, you can basically figure that out very quickly.
So in the same way, it's very unlikely that you're going to have u transpose sigma u, which is equal to 0, which means that, essentially, all your points are [INAUDIBLE] or let's say all of them are orthogonal to u, right?
So it's exactly the same thing.
It just says that in the population case, there's no probability that your points deviate from this guy here.
This happens with zero probability, OK?
And that's just because if you look at the variance of this guy, it's going to be 0.
And then that means that there's no deviation.
By the way, I'm using the name projection when I talk about u transpose X, right?
So let's just be clear about this.
If you-- so let's say I have a bunch of points, and u is a vector in this direction.
And let's say that u has the-- so this is 0.
This is u.
And let's say that u has norm, 1, OK?
When I look, what is the coordinate of the projection?
So what is the length of this guy here?
Let's call this guy X1.
What is the length of this guy?
In terms of inner products?
This is exactly u transpose X1.
This length here, if this is X2, this is exactly u transpose X2, OK?
So those-- u transpose X measure exactly the distance to the origin of those-- I mean, it's really-- think of it as being just an x-axis thing.
You just have a bunch of points.
You have an origin.
And it's really just telling you what the coordinate on this axis is going to be, right?
So in particular, if the empirical variance is 0, it means that all these points project to the same point, which means that they have to be orthogonal to this guy.
And you can think of it as being also maybe an entire plane that's orthogonal to this line, OK?
So that's why I talk about projection, because the inner products, u transpose X, is really measuring the coordinates of X when u becomes the x-axis.
Now, if u does not have norm 1, then you just have a change of scale here.
You just have a change of unit, right?
So this is really u times X1.
The coordinates should really be divided by the norm of u. OK, so now, just in the same way-- so we're never going to have exactly 0.
But if we [INAUDIBLE] the other end, if u transpose Su is large, what does it mean?
It means that when I look at my points as projected onto the axis generated by u, they're going to have a lot of variance.
They're going to be far away from each other in average, right?
That's what large variance means, or at least large empirical variance means.
And same thing for u.
So what we're going to try to find is a u that maximizes this.
If I can find a u that maximizes this so I can look in every direction, and suddenly I find a direction in which the spread is massive, then that's a point on which I'm basically the less likely to have my points project onto each other and collide, right?
At least I know they're going to project at least onto two points.
So the idea now is to say, OK, let's try to maximize this spread, right?
So we're going to try to find the maximum over all u's of u transpose Su.
And that's going to be the direction that maximizes the empirical variance.
Now of course, if I read it like that for all u's in Rd, what is the value of this maximum?
It's infinity, right?
Because I can always multiply u by 10, and this entire thing is going to multiplied by 100.
So I'm just going to take u as large as I want, and this thing is going to be as large as I want, and so I need to constrain u.
And as I said, I need to have u of size 1 to talk about coordinates in the system generated by u like this.
So I'm just going to constrain u to have Euclidean norm equal to 1, OK?
So that's going to be my goal-- trying to find the largest possible u transpose Su, or in other words, empirical variance of the points projected onto the direction u when u is of norm 1, which justifies to use the word, "direction," and because there's no magnitude to this u. OK, so how am I going to do this?
I could just fold and say, let's just optimize this thing, right?
Let's just take this problem.
It says maximize a function onto some constraints.
Immediately, the constraint is sort of nasty.
I'm on a sphere, and I'm trying to move points on the sphere.
And I'm maximizing this thing which actually happens to be convex.
And we know we know how to minimize convex functions, but maximize them is a different question.
And so this problem might be super hard.
So I can just say, OK, here's what I want to do, and let me give that to an optimizer and just hope that the optimizer can solve this problem for me.
That's one thing we can do.
Now as you can imagine, PCA is so well spread, right?
Principal component analysis is something that people do constantly.
And so that means that we know how to do this fast.
So that's one thing.
The other thing that you should probably question about why-- if this thing is actually difficult, why in the world would you even choose the variance as a measure of spread if there's so many measures of spread, right?
The variance is one measure of spread.
It's not guaranteed that everything is going to project nicely far apart from each other.
So we could choose the variance, but we could choose something else.
If the variance does not help, why choose it?
Turns out the variance helps.
So this is indeed a non-convex problem.
I'm maximizing, so it's actually the same.
I can make this constraint convex because I'm maximizing a convex function, so it's clear that the maximum is going to be attained at the boundary.
So I can actually just fill this ball into some convex ball.
However, I'm still maximizing, so this is a non-convex problem.
And this turns out to be the fanciest non-convex problem we know how to solve.
And the reason why we know how to solve it is not because of optimization or using gradient-type things or anything of the algorithms that I mentioned during the maximum likelihood.
It's because of linear algebra.
Linear algebra guarantees that we know how to solve this.
And to understand this, we need to go a little deeper in linear algebra, and we need to understand the concept of diagonalization of a matrix.
So who has ever seen the concept of an eigenvalue?
Oh, that's beautiful.
And if you're not raising your hand, you're just playing "Candy Crush," right?
All right, so, OK.
This is great.
Everybody's seen it.
For my live audience of millions, maybe you have not, so I will still go through it.
All right, so one of the basic facts-- and I remember when I learned this in-- I mean, when I was an undergrad, I learned about the spectral decomposition and this diagonalization of matrices.
And for me, it was just a structural property of matrices, but it turns out that it's extremely useful, and it's useful for algorithmic purposes.
And so what this theorem tells you is that if you take a symmetric matrix-- well, with real entries, but that really does not matter so much.
And here, I'm going to actually-- so I take a symmetric matrix, and actually S and sigma are two such symmetric matrices, right?
Then there exists P and D, which are both-- so let's say d by d. Which are both d by d such that P is orthogonal.
That means that P transpose P is equal to PP transpose is equal to the identity.
And D is diagonal.
And sigma, let's say, is equal to PDP transpose, OK?
So it's a diagonalization because it's finding a nice transformation.
P has some nice properties.
It's really just the change of coordinates in which your matrix is diagonal, right?
And the way you want to see this-- and I think it sort of helps to think about this problem as being-- sigma being a covariance matrix.
What does a covariance matrix tell you?
Think of a multivariate Gaussian.
Can everybody visualize a three-dimensional Gaussian density?
Right, so it's going to be some sort of a bell-shaped curve, but it might be more elongated in one direction than another.
And then going to chop it like that, all right?
So I'm going to chop it off.
And I'm going to look at how it bleeds, all right?
So I'm just going to look at where the blood is.
And what it's going to look at-- it's going to look like some sort of ellipsoid, right?
In high dimension, it's just going to be an olive.
And that is just going to be bigger and bigger.
And then I chop it off a little lower, and I get something a little bigger like this.
And so it turns out that sigma is capturing exactly this, right?
The matrix sigma-- so the center of your covariance matrix of your Gaussian is going to be this thing.
And sigma is going to tell you which direction it's elongated.
And so in particular, if you look, if you knew an ellipse, you know there's something called principal axis, right?
So you could actually define something that looks like this, which is this axis, the one along which it's the most elongated.
Then the axis along which is orthogonal to it, along which it's slightly less elongated, and you go again and again along the orthogonal ones.
It turns out that those things here is the new coordinate system in which this transformation, P and P transpose, is putting you into.
And D has entries on the diagonal which are exactly this length and this length, right?
So that's just what it's doing.
It's just telling you, well, if you think of having this Gaussian or this high-dimensional ellipsoid, it's elongated along certain directions.
And these directions are actually maybe not well aligned with your original coordinate system, which might just be the usual one, right-- north, south, and east, west.
Maybe I need to turn it.
And that's exactly what this orthogonal transformation is doing for you, all right?
So, in a way, this is actually telling you even more.
It's telling you that any matrix that's symmetric, you can actually turn it somewhere.
And that'll start to dilate things in the directions that you have, and then turn it back to what you originally had.
And that's actually exactly the effect of applying a symmetric matrix through a vector, right?
And it's pretty impressive.
It says if I take sigma times v. Any sigma that's of this form, what I'm doing is-- that's symmetric.
What I'm really doing to v is I'm changing its coordinate system, so I'm rotating it.
Then I'm changing-- I'm multiplying its coordinates, and then I'm rotating it back.
That's all it's doing, and that's what all symmetric matrices do, which means that this is doing a lot.
All right, so OK.
So, what do I know?
So I'm not going to prove that this is the so-called spectral theorem.
And the diagonal entries of D is of the form, lambda 1, lambda 2, lambda d, 0, 0.
And the lambda j's are called eigenvalues of D. Now in general, those numbers can be positive, negative, or equal to 0.
But here, I know that sigma and S are-- well, they're symmetric for sure, but they are positive semidefinite.
What does it mean?
It means that when I take u transpose sigma u for example, this number is always non-negative.
Why is this true?
What is this number?
It's the variance of-- and actually, I don't even need to finish this sentence.
As soon as I say that this is a variance, well, it has to be non-negative.
We know that a variance is not negative.
And so, that's also a nice way you can use that.
So it's just to say, well, OK, this thing is positive semidefinite because it's a covariance matrix.
So I know it's a variance, OK?
So I get this.
Now, if I had some negative numbers-- so the effect of that is that when I draw this picture, those axes are always positive, which is kind of a weird thing to say.
But what it means is that when I take a vector, v, I rotate it, and then I stretch it in the directions of the coordinate, I cannot flip it.
I can only stretch or shrink, but I cannot flip its sign, all right?
But in general, for any symmetric matrices, I could do this.
But when it's positive symmetric definite, actually what turns out is that all the lambda j's are non-negative.
I cannot flip it, OK?
So all the eigenvalues are non-negative.
That's a property of positive semidef.
So when it's symmetric, you have the eigenvalues.
They can be any number.
And when it's positive semidefinite, in particular that's the case of the covariance matrix and the empirical covariance matrix, right?
Because the empirical covariance matrix is an empirical variance, which itself is non-negative.
And so I get that the eigenvalues are non-negative.
All right, so principal component analysis is saying, OK, I want to find the direction, u, that maximizes u transpose Su, all right?
I've just introduced in one slide something about eigenvalues.
So hopefully, they should help.
So what is it that I'm going to be getting?
Well, let's just see what happens.
Oh, I forgot to mention that-- and I will use this.
So the lambda j's are called eigenvectors.
And then the matrix, P, has columns v1 to vd, OK?
The fact that it's orthogonal-- that P transpose P is equal to the identity-- means that those guys satisfied that vi transpose vj is equal to 0 if i is different from j.
And vi transpose vi is actually equal to 1, right, because the entries of PP transpose are exactly going to be of the form, vi transpose vj, OK?
So those v's are called eigenvectors.
And v1 is attached to lambda 1, and v2 is attached to lambda 2, OK?
So let's see what's happening with those things.
What happens if I take sigma-- so if you know eigenvalues, you know exactly what's going to happen.
If I look at, say, sigma times v1, well, what is sigma?
We know that sigma is PDP transpose v1.
What is P transpose times v1?
Well, P transpose has rows v1 transpose, v2 transpose, all the way to vd transpose.
So when I multiply this by v1, what I'm left with is the first coordinate is going to be equal to 1 and the second coordinate is going to be equal to 0, right?
Because they're orthogonal to each other-- 0 all the way to the end.
So that's when I do P transpose v1.
Now I multiply by D. Well, I'm just multiplying this guy by lambda 1, this guy by lambda 2, and this guy by lambda d, so this is really just lambda 1.
And now I need to post-multiply by P. So what is P times this guy?
Well, P is v1 all the way to vd.
And now I multiply by a vector that only has 0's except lambda 1 on the first guy.
So this is just lambda 1 times v1.
So what we've proved is that sigma times v1 is lambda 1 v1, and that's probably the notion of eigenvalue you're most comfortable with, right?
So just when I multiply by v1, I get v1 back multiplied by something, which is the eigenvalue.
So in particular, if I look at v1, transpose sigma v1, what do I get?
Well, I get lambda 1 v1 transpose v1, which is 1, right?
So this is actually lambda 1 v1 transpose v1, which is lambda 1, OK?
And if I do the same with v2, clearly I'm going to get v2 transpose sigma.
v2 is equal to lambda 2.
So for each of the vj's, I know that if I look at the variance along the vj, it's actually exactly given by those eigenvalues, all right?
Which proves this, because the variance along the eigenvectors is actually equal to the eigenvalues.
So since they're variances, they have to be non-negative.
So now, I'm looking for the one direction that has the most variance, right?
But that's not only among the eigenvectors.
That's also among the other directions that are in-between the eigenvectors.
If I were to look only at the eigenvectors, it would just tell me, well, just pick the eigenvector, vj, that's associated to the largest of the lambda j's.
But it turns out that that's also true for any vector-- that the maximum direction is actually one direction which is among the eigenvectors.
And among the eigenvectors, we know that the one that's the largest-- that carries the largest variance is the one that's associated to the largest eigenvalue, all right?
And so this is what PCA is going to try to do for me.
So in practice, that's what I mentioned already, right?
We're trying to project the point cloud onto a low-dimensional space, D prime, by keeping as much information as possible.
And by "as much information," I mean we do not want points to collide.
And so what PCA is going to do is just going to try to project [?
on two ?]
directions.
So there's going to be a u, and then there's going to be something orthogonal to u, and then the third one, et cetera, so that once we project on those, we're keeping as much of the covariance as possible, OK?
And in particular, those directions that we're going to pick are actually a subset of the vj's that are associated to the largest eigenvalues.
So I'm going to stop here for today.
We'll finish this on Tuesday.
But basically, the idea is it's just the following.
You're just going to-- well, let me skip one more.
Yeah, this is the idea.
You're first going to pick the eigenvector associated to the largest eigenvalue.
Then you're going to pick the direction that orthogonal to the vector that you've picked, and that's carrying the most variance.
And that's actually the second largest-- the eigenvector associated to the second largest eigenvalue.
And you're going to go all the way to the number of them that you actually want to pick, which is in this case, d, OK?
And wherever you choose to chop this process, not going all the way to d, is going to actually give you a lower-dimensional representation in the coordinate system that's given by v1, v2, v3, et cetera, OK?
So we'll see that in more details on Tuesday.
But I don't want to get into it now.
We don't have enough time.
Are there any questions?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We keep on talking about principal component analysis, which we essentially introduced as a way to work with a bunch of data.
So the data that's given to us when we want to do PCA is a bunch of vectors X1 to Xn.
So they are random vectors.
in Rd.
And what we mentioned is that we're going to be using linear algebra-- in particular, the spectral theorem-- that guarantees to us that if I look at the convenience matrix of this guy, or its empirical covariance matrix, since they're symmetric real matrices and they are positive semidefinite, there exists a diagonalization into non-negative eigenvalues.
And so here, those things live in Rd, so it's a really large space.
And what we want to do is to map it down into a space that we can visualize, hopefully a space of size 2 or 3.
Or if not, then we're just going to take more and start looking at subspaces altogether.
So think of the case where d is large but not larger than n. So let's say, you have a large number of points.
The question is, is it possible to project those things onto a lower dimensional space, d prime, which is much less than d-- so think of d prime equals, say, 2 or 3-- and so that you keep as much information about the cloud of points that you had originally.
So again, the example that we could have is that X1 to Xn for, say, Xi for patient i's recording a bunch of body measurements and maybe blood pressure, some symptoms, et cetera.
And then we have a cloud of n patients.
And we're trying to visualize maybe to see if-- If I could see, for example, that there's two groups of patients, maybe I would know that I have two groups of different disease or maybe two groups of different patients that respond differently to a particular disease or drug et cetera.
So visualizing is going to give us quite a bit of insight about what the spatial arrangement of those vectors are.
And so PCA says, well, here, of course, in this question, one thing that's not defined is what is information.
And we said that one thing we might want to do when we project is that points do not collide with each other.
And so that means we're trying to find directions, where after I project, the points are still pretty spread out.
And so I can see what's going on.
And PCA says-- OK, so there's many ways to answer this question.
And PCA says, let's just find a subspace of dimension d prime that keeps as much covariance structure as possible.
And the reason is that those directions are the ones that maximize the variance, which is a proxy for the spread.
There's many, many ways to do this.
There's actually a Google video that was released maybe last week about the data visualization team of Google that shows you something called t-SNE, which is essentially something that tries to do that.
It takes points in very high dimensions and tries to map them in lower dimensions, so that you can actually visualize them.
And t-SNE is some alternative to PCA that gives an other definition for the word information.
I'll talk about this towards the end, how you can actually somewhat automatically extend everything we've said for PCA to an infinite family of procedures.
So how do we do this?
Well, the way we do this is as follows.
So remember, given those guys, we can form something which is called S, which is the sample, or the empirical covariance matrix.
And since from couple of slides ago, we know that S has a eigenvalue decomposition, S is equal to PDP transpose, where P is orthogonal.
So that's where we use our linear algebra results.
So that means that P transpose P is equal to PP transpose, which is the identity.
So remember, S is a d by d matrix.
And so P is also d by d. And d is diagonal.
And I'm actually going to take, without loss of generality, I'm going to assume that d-- so it's going to be diagonal-- and I'm going to have something that looks like lambda 1 to lambda d. Those are called the eigenvalues of S. What we know is that lambda j's are non-negative.
And actually, what I'm going to assume without loss of generalities is lambda 1 is larger than lambda 2, which is larger than lambda d. Because in particular, this decomposition-- the spectrum decomposition-- is not entirely unique.
I could permute the columns of P, and I would still have an orthogonal matrix.
And to balance that, I would also have to permute the entries of d. So there's as many decompositions as there are permutations.
So there's actually quite a bit.
But the bag of eigenvalues is unique.
The set of eigenvalues is unique.
The ordering is certainly not unique.
So here, I'm just going to pick-- I'm going to nail down one particular permutation-- actually, maybe two in case I have equalities.
But let's say, I pick one that satisfies this.
And the reason why I do this is really not very important.
It's just to say, I'm going to want to talk about the largest of those eigenvalues.
So this is just going to be easier for me to say that this one is lambda 1, rather than say it's lambda 7.
So this is just to say that the largest eigenvalue of S is lambda 1.
If I didn't do that, I would just call it maybe lambda max, and you would just know which one I'm talking about.
So what's happening now is that if I look at d, then it turns out that if I start-- so if I do P transpose Xi, I am actually projecting my Xi's-- I'm basically changing the basis for my Xi's.
And now, D is the empirical covariance matrix of those guys.
So let's check that.
So what it means is that if I look at-- so what I claim is that P transpose Xi-- that's a new vector, let's call it Yi, it's also in Rd-- and what I claim is that the covariance matrix of this guy is actually now this diagonal matrix, which means in particular that if they were Gaussian, then they would be independent.
But I also know now that there's no correlation across coordinates of Yi.
So to prove this, let me assume that X bar is equal to 0.
And the reason why I do this is because it's just annoying to carry out all this censuring constantly and I talk about S. So when X bar is equal to 0, that implies that S has a very simple form.
It's of the form sum from i equal 1 to n of Xi Xi transpose.
So that's my S. But what I want is the S of Y-- So OK, that implies also that P times X bar, which is equal to P times X bar is also equal to 0.
So that means that Y bar-- Y has mean 0, if this is 0.
So if I look at the sample covariance matrix of Y, it's just going to be something that looks like the sum of the outer products or the Yi Yi transpose.
And again, the reason why I make this assumption is so that I don't have to write minus X bar X bar transpose.
But you can do it.
And it's going to work exactly the same.
So now, I look at this S prime.
And so what is this S prime?
Well, I'm just going to replace Yi with PXi.
So it's the sum from i equal 1 to n of PXi PXi transpose, which is equal to the sum from-- sorry there's a 1/n.
So it's equal to 1/n sum from i equal 1 to n of PXi Xi transpose P transpose.
Agree?
I just said that the transpose of AB is the transpose of B times the transpose of A.
And so now, I can push the sum in.
P does not depend on i.
So this thing here is equal to PS P transpose, because the sum of the Xi Xi transpose divided by n is S. But what is PS P transpose?
Well, we know that S is equal to-- sorry that's P transpose.
So this was with a P transpose.
I'm sorry, I made an important mistake here.
So Yi is P transpose Xi.
So this is P transpose and P transpose here, which means that this is P transpose and this is double transpose, which is just nothing and that transpose and nothing.
So now, I write S as PD P transpose.
That's the spectral decomposition that I had before.
That's my eigenvalue decomposition, which means that now, if I look at S prime, it's P transpose times PD P transpose P. But now, P transpose P is the identity, P transpose P is the identity.
So this is actually just equal to D. And again, you can check that this also works if you have to center all those guys as you go.
But if you think about it, this is the same thing as saying that I just replaced Xi by Xi minus X bar.
And then it's true that Y bar is also P times Xi minus X bar.
So now, we have that D is the empirical covariance matrix of those guys-- the Yi's, which are P transpose Xi's.
And so in particular, what it means is that if I look at the covariance of Yj Yk-- So that's the covariance of the j-th coordinate of Y and the k-th coordinate of Y. I'm just not putting an index.
But maybe, let's say the first one or something like this-- any of them, their IID.
Then what is this covariance?
It's actually 0 if j is different from k. And the covariance between Yj and Yj, which is just the variance of Yj, is equal to lambda j-- the j-th largest eigenvalue.
So the eigenvalues capture the variance of my observations in this new coordinate system.
And they're completely orthogonal.
So what does that mean?
Well, again, remember, if I chop off the head of my Gaussian in multi dimensions, we said that what we started from was something that looked like this.
And we said, well, there's one direction that's important, that's this guy, and one important that's this guy.
When I applied a transformation P transpose, what I'm doing is that I'm realigning this thing with the new axes.
Or in a way, rather to be fair, I'm not actually realigning the ellipses with the axes.
I'm really realigning the axes with the ellipses.
So really, what I'm doing is I'm saying, after I apply P, I'm just rotating this coordinate system.
So now, it becomes this guy.
And now, my ellipses actually completely align.
And what happens here is that this coordinate is independent of that coordinate.
And that's what we write here, if they are Gaussian.
I didn't really tell this-- I'm only making statements about covariances.
If they are Gaussians, those implied statements about independence.
So as I said, the variance now, lambda 1, is actually the variance of P transpose Xi.
But if I look now at the-- so this is a vector, so I need to look at the first coordinate of this guy.
So it turns out that doing this is actually the same thing as looking at the variance of what?
Well, the first column of P times Xi.
So that's the variance of-- I'm going to call it v1 transpose Xi, where P-- So the v1 vd in Rd are eigenvectors.
And each vi is associated to lambda i.
So that's what we saw when we talked about this eigen decomposition a couple of slides back.
That's the one here.
So if I call the columns of P v1 to vd, this is what's happening.
So when I look at lambda 1, it's just the variance of Xi inner product with v1.
And we made this picture when we said, well, let's say v1 is here and then x1 is here.
And if vi has a unique norm, then the inner product between Xi and v1 is just the length of this guy here.
So that's the variance of the Xi says the length of Xi-- so this is 0-- that's the length of Xi when I project it onto the direction that span by v1.
If v1 has length 2, this is really just twice this length.
If vi has length 3, it's three times this.
But it turns out that since P satisfies P transpose P is equal to the identity-- that's an orthogonal matrix, that's right here-- then this is actually saying the same thing as vj transpose vj, which is really the norm squared of vj, is equal to 1.
And vj transpose vk is equal to 0, if j is different from k. The eigenvectors are orthogonal to each other.
And they're actually all of norm 1.
So now, I know that this is indeed a direction.
And so when I look at v1 transpose Xi, I'm really measuring exactly this length.
And what is this length?
It's the length of the projection of Xi onto this line.
That's the line that's spanned by v1.
So if I had a very high dimensional problem and I started to look at the direction v1-- let's say v1 now is not a eigenvector, it's any direction-- then if I want to do this lower dimensional projection, then I have to understand how those Xi's project onto the line that's spanned by v1, because this is all that I'm going to be keeping at the end of the day about Xi's.
So what we want is to find the direction where those Xi's, those projections, have a lot of variance.
And we know that the variance of Xi on this direction is actually exactly given by lambda 1.
Sorry, that's the empirical var-- yeah, I should call variance hat.
That's the empirical variance.
Everything is in empirical here.
We're talking about the empirical covariance matrix.
And so I also have that lambda 2 is the empirical variance of when I project Xi onto v2, which is the second one, just for exactly this reason.
Any question?
So lambda j's are going to be important for us.
Lambda j measure the spread of the points when I project them onto a line which is a one dimensional space.
And so I'm going to have-- let's say I want to pick only one, I'm going to have to find the one dimensional space that carries the most variance.
And I claim that v1 is the one that actually maximizes the spread.
So the claim-- so for any direction, u in Rd-- and by direction, I really just mean that the norm of u is equal to 1.
I need to play fair-- I'm going to compare myself to other things of lengths one, so I need to play fair and look at directions of length 1.
Now, if I'm interested in the empirical variance of X1 transpose-- sorry, u transpose X1 u transpose Xn, then this thing is maximized for u equals v1, where v1 is the eigenvector associated to lambda 1 and lambda 1 is not any eigenvalues, it's the largest of all those.
So it's the largest eigenvalue.
So why is that true?
Well, there's also a claim that for any direction u-- so that's 1 and 2-- the variance of u transpose X-- now, this is just a random variable, and I'm looking about the true variance-- this is maximized for u equals, let's call it w1, where w1 is the eigenvector of sigma-- Now, I'm talking about the true variance.
Whereas, here, I was talking about the empirical variance.
So the true variance is the eigenvectors of the true sigma associated to the largest eigenvalue of sigma.
So I did not give it a name.
Here, that was lambda 1 for the empirical one.
For the true one, you can give it another name, mu 1 if you want.
But that's just the same thing.
All it's saying is like, wherever I see empirical, I can remove it.
So why is this claim true?
Well, let's look at the second one, for example.
So what is the variance of u transpose X?
So that's what I want to know.
So that's the expectation-- so let's assume that X is 0, again, for same reasons as before.
So what is the variance?
It's just the expectation of the square?
I don't need to remove the expectation.
And the expedition of the square is just the expectation of u transpose X.
And then I'm going to write the other one X transpose u.
And we know that this is deterministic.
So I'm just going to take that this is just u transpose expectation of X X transpose u.
And what is this guy?
That's covariance sigma.
That's just what sigma is.
So the variance I can write as u transpose sigma u.
We've made this computation before.
And now what I want to claim is that this thing is actually less than the largest eigenvalue, which I actually called lambda 1 here.
I should probably not.
And the P is-- well, OK. Let's just pretend everything is not empirical.
So now, I'm going to write sigma as P lambda 1 lambda n P transpose.
That's just the eigendecomposition, where I admittedly reuse the same notation as I did for S. So I should really put some primes everywhere, so you know those are things that are actually different in practice.
So this is just that the decomposition of sigma.
You seem confused, Helen.
You have a question?
Yeah?
What is-- when you talked about the empirical data and--
So OK-- so I can make everything I'm saying, I can talk about either the variance or the empirical variance.
And you can just add the word empirical in front of it whenever you want.
The same thing works.
But just for the sake of removing the confusion, let's just do it again with S. So I'm just going to do everything with S. So I'm going to assume that X bar is equal to 0.
And here, I'm going to talk about the empirical variance, which is just 1/n sum from i equal 1 to n of u transpose Xi squared.
So it's the same thing.
Everywhere you see an expectation, you just put in average.
And then I get 1/n sum from i equal 1 to n of Xi Xi transpose.
And now, I'm going to call this guy S, because that's what it is.
So this is u transpose Su.
But just defined that I could just replace the expectation by averages everywhere, you can tell that the thing is going to work for either one or the other.
So now, this thing was actually-- so now, I don't have any problem with my notation.
This is actually the decomposition of S. That's just the spectral decomposition and it's to its eigenvalues.
And so now, what I have is that when I look at u transpose Su, this is actually equal to P u transpose S Pu.
OK.
There's a transpose somewhere.
That's this guy.
And that's this guy.
Now-- sorry, that's not P, that's D. That's D, that's this diagonal matrix.
Let's look at this thing.
And let's call P transpose u, let's call it b.
So that's also a vector in Rd.
What is it?
It's just, I take a unit vector, and then I apply P transpose to it.
So that's basically what happens to a unit vector when I apply the same change of basis that I did.
So I'm just changing my orthogonal system the same way I did for the other ones.
So what's happening when I write this?
Well, now I have that u transpose Su is b transpose Db.
But now, doing b transpose Db when D is diagonal and b is a vector is a very simple thing.
I can expand it.
This is what?
This is just the sum from j equal 1 to d of lambda j bj squared.
So that's just like matrix vector multiplication.
And in particular, I know that the largest of those guys is lambda 1 and those guys are all non-negative.
So this thing is actually less than lambda 1 times the sum from j equal 1 to d of lambda j squared-- sorry, bj squared.
And this is just the norm of b squared.
So if I want to prove what's on the slide, all I need to check is that b has norm, which is--
1
At most, 1.
It's going to be at most 1.
Why?
Well, because b is really just a change of basis for u.
And so if I take a vector, I'm just changing its basis.
I'm certainly not changing its length-- think of a rotation, and I can also flip it, but think of a rotation-- well, actually, for vector, it's just going to be a rotation.
And so now, what I have I just have to check that the norm of b squared is equal to what?
Well, it's equal to the norm of P transpose u squared, which is equal to u transpose P P transpose u.
But P is orthogonal.
So this thing is actually just the identity.
So that's just u transpose u, which is equal to the norm u squared, which is equal to 1, because I took u to have norm 1 in the first place.
And so this-- you're right-- was actually of norm equal to 1.
I just needed to have it less, but it's equal.
And so what I'm left with is that this thing is actually equal to lambda 1.
So I know that for every u that I pick-- that has norm-- So I'm just reminding you that u here has norm squared equal to 1.
For every u that I pick, this u transpose Su is at mostly lambda 1.
So that's the u transpose Su is at most lambda 1.
And we know that that's the variance, that's the empirical variance, when I project my points onto direction spanned by u.
So now, I have an empirical variance, which is at most lambda 1.
But I also know that if I take u to be something very specific-- I mean, it was on the previous board-- if I take u to be equal to v1, then this thing is actually not an inequality, this is an equality.
And the reason is, when I actually take u to be v1, all of these bj's are going to be 0, except for the one that's b1, which is itself equal to 1.
So I mean, we can briefly check this.
But if I take v-- if u is equal to v1, what I have is that u transpose Su is equal to P transpose v1 D P transpose v1.
But what is P transpose v1?
Well, remember P transpose is just the matrix that has vectors v1 transpose here, v2 transpose here, all the way to vd transpose here.
And we know that when I take vj transpose vk, I get 0, if j is different from k. And if j is equal to k, I get 1.
So P transpose v1 is equal to what?
Take v1 here and multiply it.
So the first coordinate is going to be v1 transpose v1, which is 1.
The second coordinate is going to be v2 transpose v1, which is 0.
And so I get 0's all the way, right?
So that means that this thing here is really just the vector 1, 0, 0.
And here, this is just the vector 1, 0, 0.
So when I multiply it with this guy, I am only picking up the top left element of D, which is lambda 1.
So for every one, it's less lambda 1.
And for v1, it's equal to lambda 1, which means that it's maximized for a equals v1.
And that's where I said that this is the fanciest non-convex problem we know how to solve.
This was a problem that was definitely non-convex.
We were maximizing a convex function over a sphere.
But we know that v1, which is something-- I mean, of course, you still have to believe me that you can compute the spectral decomposition efficiently-- but essentially, if you've taken linear algebra, you know that you can diagonalize a matrix.
And so you get that v1 is just the maximum.
So you can find your maximum just by looking at the spectral decomposition.
You don't have to do any optimization or anything like this.
So let's recap.
Where are we?
We've established that if I start with my empirical covariance matrix, I can diagonalize it and PD P transpose.
And then if I take the eigenvector associated to the largest eigenvalues-- so if I permute the columns of P and of D's in such a way that they are ordered from the largest to the smallest when I look at the diagonal elements of D, then if I pick the first column of P, it's v1.
And v1 is the direction on which, if I project my points, they are going to carry the most empirical variance.
Well, that's a good way.
If I told you, pick one direction along which if you were to project your points they would be as spread out as possible, that's probably the one you would pick.
And so that's exactly what PCA is doing for us.
It says, OK, if you ask me to take d prime equal to 1, I will take v1.
I will just take the direction that's spanned by v1.
And that's just when I come back to this picture that was here before, this is v1.
Of course, here, I only have two of them.
So v2 has to be this guy, or this guy, or I mean or this thing.
I mean, I don't know them up to sine.
But then if I have three-- think of like an olive in three dimensions-- then maybe I have one direction that's slightly more elongated than the other one.
And so I'm going to pick the second one.
And so the procedure is to say, well, first, I'm going to pick v1 the same way I pick v1 in the first place.
So the first direction I am taking is the leading eigenvector.
And then I'm looking for a direction.
Well, if I found one-- the one I'm going to want to find-- if you say you can take d equal 2, you're going to need the basis for this guy.
So the second one has to be orthogonal to the first one you've already picked.
And so the second one you pick is the one that's just, among all those that are orthogonal to v1, maximized the empirical variance when you project onto it.
And it turns out that this is actually exactly v2.
You don't have to redo anything again.
You're eigendecomposition, this is just the second column of P. Clearly, v2 is orthogonal to v1.
We just used it here.
This 0 here just says this v2 is orthogonal to v1.
So they're like this.
And now, what I said-- what this slide tells you extra-- is that v2 among all those directions that are orthogonal-- I mean, there's still d minus 1 of them-- this is the one that maximizes the, say, residual empirical variance-- the one that was not explained by the first direction that you picked.
And you can check that.
I mean, it's becoming a bit more cumbersome to write down, but you can check that.
If you're not convinced, please raise your concern.
I mean, basically, one way you view this to-- I mean, you're not really dropping a coordinate, because v1 is not a coordinate.
But let's assume actually for simplicity that v1 was actually equal to e1, that the direction that carries the most variance is the one that just says, just look at the first coordinate of X.
So if that was the case, then clearly the orthogonal directions are the ones that comprise only of the coordinates 2 to d. So you could actually just drop the first coordinate and do the same thing on a slightly shorter vector of length d minus 1.
And then you would just look at the largest eigenvector of these guys, et cetera, et cetera.
So in a way, that's what's happening, except that you rotate it before you actually do this.
And that's exactly what's happening.
So what we put together here is essentially three things.
One was statistics.
Statistics says, if you won't spread, if you want information, you should be looking at variance.
The second one was optimization.
Optimization said, well, if you want to maximize spread, well, you have to maximize variance in a certain direction.
And that means maximizing over the sphere of vectors that have unique norm.
And that's an optimization problem, which actually turned out to be difficult.
But then the third thing that we use to solve this problem was linear algebra.
Linear algebra said, well, it looks like it's a difficult optimization problem.
But it turns out that the answer comes in almost-- I mean, it's not a closed form, but those things are so used, that it's almost a closed form-- says, just pick the eigenvectors in order of their associated eigenvalues from largest to smallest.
And that's why principal component analysis has been so popular and has gained huge amount of traction since we had computers that were allowed to compute eigenvalues and eigenvectors for matrices of gigantic sizes.
You can actually do that.
If I give you-- I don't know, this Google video, for example, is talking about words.
They want to do just the, say, principal component analysis of words.
So I give you all the words in the dictionary.
And-- sorry, well, you would have to have a representation for words, so it's a little more difficult.
But how do I do this?
Let's say, for example, pages of a book.
I want to understand the pages of a book.
And I need to turn it into a number.
And a page of a book is basically the word count.
So I just count the number of times "the" shows up, the number of times "and" shows up, number of times "dog" shows up.
And so that gives me a vector.
It's in pretty high dimensions.
It's as many dimensions as there are words in the dictionary.
And now, I want to visualize how those pages get together-- are two pages very similar or not.
And so what you would do is essentially just compute the largest eigenvector of this matrix-- maybe the two largest-- and then project this into a plane.
Yeah
Can we assume the number of points was far larger than the dimension?
Yeah, but there's many pages in the world.
There's probably more pages in the world than there's words in the dictionary.
Yeah, so of course, if you are in high dimensions and you don't have enough points, it's going to be clearly an issue.
If you have two points, then the leading eigenvector is going to be just the line that goes through those two points, regardless of what the dimension is.
And clearly, you're not learning anything.
So you have to pick, say, the k largest one.
If you go all the way, you're just reordering your thing, and you're not actually gaining anything.
You start from d and you go too d. So at some point, this procedure has to stop.
And let's say it stops at k. Now, of course, you should ask me a question, which is, how do you choose k?
So that's, of course, a natural question.
Probably the basic answer is just pick k equals 3, because you can actually visualize it.
But what happens if I take k is equal to 4?
If I take is equal to 4, I'm not going to be able to plot points in four dimensions.
Well, I could, I could add color, or I could try to be a little smart about it.
But it's actually quite difficult.
And so what people tend to do, if you have four dimensions, they actually do a bunch of two dimensional plots.
And that's what a computer-- a computer is not very good-- I mean, by default, they don't spit out three dimensional plots.
So let's say they want to plot only two dimensional things.
So they're going to take the first directions of, say, v1, v2.
Let's say you have three, but you want to have only two dimensional plots.
And then it's going to do v1, v3; and then v2, v3.
So really, you take all three of them, but it's really just showing you all choices of pairs of those guys.
So if you were to keep k is equal to 5, you would have five, choose two different plots.
So this is the actual principal component algorithm, how it's implemented.
And it's actually fairly simple.
I mean, it looks like there's lots of steps.
But really, there's only one that's important.
So the first one is the input.
I give you a bunch of points, x1 to xn in d dimensions.
And step two is, well, compute their empirical covariance matrix S. The points themselves, we don't really care.
We care about their empirical covariance matrix.
So it's a d by d matrix.
Now, I'm going to feed that.
And that's where the actual computation starts happening.
I'm going to feed that to something that knows how to diagonalize this matrix.
And you have to trust me, if I want to compute the k largest eigenvalues and my matrix is d by d, it's going to take me about k times d squared operations.
So if I want only three, it's 3 times d squared, which is about-- d squared is the time for me it takes to just even read the matrix sigma.
So that's not too bad.
So what it's going to spit out, of course, is the diagonal matrix D. And those are nice, because they allow me to tell me what is the order in which I should be taking the columns of P. But what's really important to me is v1 to vd, because those are going to be the ones I'm going to be using to draw those plots.
And now, I'm going to say, OK, I need to actually choose some set k. And I'm going to basically truncate and look only at the first k columns of P. Once I have those columns, what I want to do is to project onto the linear span of those columns.
And there's actually a simple way to do this, which is just take this matrix P, which is really the matrix of projection onto the linear span of those k columns.
And you just take Pk transpose.
And then you apply this to every single one of your points.
Now Pk transpose, what is the size of the matrix Pk?
Yeah, [INAUDIBLE]?
[INAUDIBLE]
So Pk is just this matrix.
I take the v1 and I stop at vk-- well--
[INAUDIBLE]
d by k, right?
Each of the column is an eigenvector.
It's of dimension d. I mean, that's a vector in the original space.
So I have this d by k matrix.
So all it is is if I had my-- well, I'm going to talk in a second about Pk transpose.
Pk transpose is just this guy, where I stop at the k-th vector.
So Pk transpose is k by d. So now, when I take Yi, which is Pk transpose Xi, I end up with a point which is in k dimensions.
I have only k coordinates.
So I took every single one of my original points Xi, which had d coordinates, and I turned it into a point that has only k coordinates.
Particularly, I could have k is equal to 2.
This matrix is exactly the one that projects.
If you think about it for one second, this is just the matrix that says-- well, we actually did that several times.
The matrix, so that was this P transpose u that showed up somewhere.
And so that's just the matrix that take your point X in, say, three dimensions, and then just project it down to two dimensions.
And that's just-- it goes to the closest point in the subspace.
Now, here, the floor is flat.
But we can pick any subspace we want, depending on what the lambdas are.
So the lambdas were important for us to be able to identify which columns to pick.
The fact that we assumed that they were ordered tells us that we can pick the first ones.
If they were not ordered, it would be just a subset of the columns, depending on what the size of the eigenvalue is.
So each column is labeled.
And so then, of course, we still have this question of, how do I pick k?
So there's definitely the matter of convenience.
Maybe 2 is convenient.
If it works for 2, you don't have to go any farther.
But you might want to say, well-- originally, I did that to actually keep as much information as possible.
I know that the ultimate thing is to keep as much information, which would be to k is equal d-- that's as much information as you want.
But it's essentially the same question about, well, if I want to compress a JPEG image, how much information should I keep so it's still visible?
And so there's some rules for that.
But none of them is actually really a science.
So it's really a matter of what you think is actually tolerable.
And we're just going to start replacing this choice by maybe another parameter.
So here, we're going to basically replace k by alpha, and so we just do stuff.
So the first one that people do that is probably the most popular one-- OK, the most popular one is definitely take k is equal to 2 or 3, because it's just convenient to visualize.
The second most popular one is the scree plot.
So the scree plot-- remember, I have my values, lambda j's.
And I've chosen the lambda j's to decrease.
So the indices are chosen in such a way that lambda is a decreasing function.
So I have lambda 1, and let's say it's this guy here.
And then I have lambda 2, and let's say it's this guy here.
And then I have lambda 3, and let's say it's this guy here, lambda 4, lambda 5, lambda 6.
And all I care about is that this thing decreases.
The scree plot says something like this-- if there's an inflection point, meaning that you can sort of do something like this and then something like this, you should stop at 3.
That's what the scree plot tells you.
What it's saying in a way is that the percentage of the marginal increment of explained variance that you get starts to decrease after you pass this inflection point.
So let's see why I way this.
Well, here, what I have-- so this ratio that you see there is actually the percentage of explained variance.
So what it means is that, if I look at lambda 1 plus lambda k, and then I divide by lambda 1 plus lambda d, well, what is this?
Well, this lambda 1 plus lambda d is the total amount of variance that I get in my points.
That's the trace of sigma.
So that's the variance in the first direction plus the variance in the second direction plus the variance in the third direction.
That's basically all the variance that I have possible.
Now, this is the variance that I kept in the first direction.
This is the variance that I kept in the second direction, all the way to the variance that I kept in the k-th direction.
So I know that this number is always less than or equal to 1.
And it's larger than 1.
And this is just the proportion, say, of variance explained by v1 to vk, or simply, the proportion of explained variance by my PCA, say.
So now, what this thing is telling me, its says, well, if I look at this thing and I start seeing this inflection point, it's saying, oh, here, you're gaining a lot and lot of variance.
And then at some point, you stop gaining a lot in your proportion of explained variance.
So this will translate in something where when I look at this ratio, lambda 1 plus lambda k divided by lambda 1 plus lambda d, this would translate into a function that would look like this.
And what it's telling you, it says, well, maybe you should stop here, because here every time you add one, you don't get as much as you did before.
You actually get like smaller marginal returns.
So explained variance is the numerator of this ratio.
And the total variance is the denominator.
Those are pretty straightforward terms that you would want to use for this.
So if your goal is to do data visualization-- so why would you take k larger than 2?
Let's say, if you take k larger than 6, you can start to imagine that you're going to have six, choose two, which starts to be annoying.
And if you have k is equal to 10-- because you could start in dimension 50,000-- and then k equal to 10 would be the place where you have this thing that's a lot of plots that you would have to show.
So it's not always for data visualization.
Once I've actually done this, I've actually effectively reduced the dimension of my problem.
And what I could do with what I have is do a regression on those guys.
The v1-- so I forgot to tell you-- why is that called principal component analysis?
Well, the vj's that I keep, v1 to vk are called principal components.
And they effectively act as the summary of my Xi's.
When I mentioned image compression, I started with a point Xi that was d numbers-- let's say 50,000 numbers.
And now, I'm saying, actually, you can throw out those 50,000 numbers.
If you actually know only the k numbers that you need-- the 6 numbers that you need-- you're going to have something that was pretty close to getting what information you had.
So in a way, there is some form of compression that's going on here.
And what you can do is that those principal components, you can actually use now for regression.
If I want to regress Y onto X that's very high dimensional, before I do this, if I don't have enough points, maybe what I can actually do is to do principal component analysis throughout my exercise, replace them by those compressed versions, and do linear aggression on those guys.
And that's called principal component regression, not surprisingly.
And that's something that's pretty popular.
And you can do with k is equal to 10, for example.
So for data visualization, I did not find a Thanksgiving themed picture.
But I found one that has turkey in it.
Get it?
So this is actually a gene data set that was-- so when you see something like this, you can imagine that someone has been preprocessing the hell out of this thing.
This is not like, oh, I collect data on 23andMe and I'm just going to run PCA on this.
It just doesn't happen like that.
And so what happened is that-- so let's assume that this was a bunch of preprocessed data, which are gene expression levels-- so 500,000 genes among 1,400 Europeans.
So here, I actually have less observations than I have samples.
And that's when you use principal component regression most of the time, so it doesn't stop you.
And then what you do is you say, OK, have those 500,000 genes among-- so here, that means that there's 1,400 points here.
And I actually take those 500,000 directions.
So each person has a vector of, say, 500,000 genes that are attached to them.
And I project them onto two dimensions, which should be extremely lossy.
I lose a lot of information.
And indeed, I do, because I'm one of these guys.
And I'm pretty sure I'm very different from this guy, even though probably from an American perspective, we're all the same.
But I think we have like slightly different genomes.
And so the thing is now we have this-- so you see there's lots of Swiss that participate in this.
But actually, those two principal components recover sort of the map of Europe.
I mean, OK, again, this is actually maybe fine-grained for you guys.
But right here, there's Portugal and Spain, which are those colors.
So here is color-coded.
And here is Turkey, of course, which we know has very different genomes.
So Turks are very at the boundary.
So you can see all the greens.
They stay very far apart from everything else.
And then the rest here is pretty mixed.
But it sort of recovers-- if you look at the colors, it sort of recovers that.
So in a way, those two principal components are just the geographic feature.
So if you insist to compress all the genomic information of these people into two numbers, what you're actually going to get is longitude and latitude, which is somewhat surprising, but not so much if you think that's it's been preprocessed.
So what do you do beyond practice?
Well, you could try to actually study those things.
If you think about it for a second, we did not do any statistics.
I talked to you about IID observations, but we never used the fact that they were independent.
The way we typically use independence is to have central limit theorem, maybe.
I mentioned the fact that the covariances of the word Gaussian would actually give me something which is independent.
We didn't care.
This was a data analysis, data mining process that we did.
I give you points, and you just put them through the crank.
There was an algorithm in six steps.
And you just put it through and that's what you got.
Now, of course, there's some work which studies says, OK, if my data is actually generated from some process-- maybe, my points are multivariate Gaussian with some structure on the covariance-- how well am I recovering the covariance structure?
And that's where statistics kicks in.
And that's where we stop.
So this is actually a bit more difficult to study.
But in a way, it's not entirely satisfactory, because we could work for a couple of boards and I would just basically sort of reverse engineer this and find some models under which it's a good idea to do that.
And what are those models?
Well, those are the models that sort of give you sort of prominent directions that you want to find.
And it will say, yes, if you have enough observations, you will find those directions along which your data is elongated.
So that's essentially what you want to do.
So that's exactly what this thing is telling you.
So where does the statistics lie from?
Well, everything, remember-- so actually that's where Alana was confused-- the idea was to say, well, if I have a true covariance matrix sigma and I never really have access to it, I'm just running PCA on the empirical covariance matrix, how do those results relate?
And this is something that you can study.
So for example, if n goes to infinity and the number of points, your dimension, is fixed, then S goes to sigma in any sense you want.
Maybe each entry is going to each entry of sigma, for example.
So S is a good estimator.
We know that the empirical covariance is a consistent as the mater.
And if d is fixed, this is actually not an issue.
So in particular, if you run PCA on the sample covariance matrix, you look at, say, v1, then v1 is going to converge to the largest eigenvector of sigma as n goes to infinity, but for d fixed.
And that's a story that we know since the '60s.
More recently, people have started challenging this.
Because what's happening when you fix the dimension and let the sample size go to infinity, you're certainly not allowing for this.
It's certainly not explaining to you anything about the fact when d is equal to 500,000 and n is equal to 1,400.
Because when d is fixed and n goes to infinity, in particular, n is much larger than d, which is not the case here.
And so when n is much larger than d, things go well.
But if d is less than n, it's not clear what happens.
And particularly, if d is of the order of n, what's happening?
So there's an entire theory in mathematics that's called random matrix theory that studies the behavior of exactly this question-- what is the behavior of the spectrum-- the eigenvalues and eigenvectors-- of a matrix in which I put random numbers and I let-- so the matrix I'm interested in here is the matrix of X's.
When I stack all my X's next to each other, so that's a matrix of size, say, d by n, so each column is of size d, it's one person.
And so I put them.
And when I let the matrix go to infinity, I let both d and n to infinity.
But I want the aspect ratio, d/n, to go to some constant.
That's what they do.
And what's nice is that in the end, you have this constant-- let's call it gamma-- that shows up in all the asymptotics.
And then you can replace it by d/n.
And you know that you still have a handle of both the dimension and the sample size.
Whereas, usually the dimension goes away, as you let n go to infinity without having dimension going to infinity.
And so now, when this happens, as soon as d/n goes to a constant, you can show that essentially there's an angle between the largest eigenvector of sigma and the largest eigenvector of S, as n and d go to infinity.
There is always an angle-- you can actually write it explicitly.
And it's an angle that depends on this ratio, gamma-- the asymptotic ratio of d/n.
And so there's been a lot of understanding how to correct, how to pay attention to this.
This creates some biases that were sort of overlooked before.
In particular, when I do this, this is not the proportion of explained variance, when n and d are similar.
This is an estimated number computed from S. This is computed from S. All these guys are computed from S. So those are actually not exactly where you want them to be.
And there's some nice work that allows you to recalibrate what this ratio should be, how this ratio should be computed, so it's a better representative of what the proportion of explained variance actually is.
So then, of course, there's the question of-- so that's when d/n goes to some constant.
So the best case-- so that was '60s-- d is fixed and it's much larger than d. And then random matrix theory tells you, well, d and n are sort of the same order of magnitude.
When they go to infinity, the ratio goes to some constant.
Think of it as being order 1.
To be fair, if d is 100 times larger than n, it still works.
And it depends on what you think what the infinity is at this point.
But I think the random matrix theory results are very useful.
But then even in this case, I told you that the leading eigenvector of S is actually an angle of the leading eigenvector of-- So what's happening is that-- so let's say that d/n goes to some gamma.
And what I claim is that, if you look at-- so that's v1, that's the v1 of S. And then there's the v1 of-- so this should be of size 1.
So that's the v1 of sigma.
Then those things are going to have an angle, which is some function of gamma.
It's complicated, but there's a function of gamma that you can see there.
And there's some models.
When gamma goes to infinity, which means that d is now much larger than n, this angle is 90 degrees, which means that you're getting nothing.
Yeah
If d is not on your lower plane, so like gamma is 0, is there still angle?
No, but that's consistent-- the fact that it's consistent when-- so the angle is a function--
d is not a constant [INAUDIBLE]??
d is not a constant?
So if d is little of n?
Then gamma goes to 0 and f of gamma goes to 0.
So f of gamma is a function that-- so for example, if f of gamma-- this is the sine of the angle, for example-- then it's a function that starts at 0, and that goes like this.
But as soon as gamma is positive, it goes away from 0.
So now when gamma goes to infinity, then this thing goes to a right angle, which means I'm getting just junk.
So this is not my leading eigenvector.
So how do you do this?
Well, just like everywhere in statistics, you have to just make more assumptions.
You have to assume that you're not looking for the leading eigenvector or the direction that carries the most variance.
But you're looking, maybe, for a special direction.
And that's what sparse PCA is doing.
Sparse PCA is saying, I'm not looking for any direction new that carries the most variance.
I'm only looking for a direction new that is sparse.
Think of it, for example, as having 10 non-zero coordinates.
So that's a lot of directions still to look for.
But once you do this, then you actually have not only-- there's a few things that actually you get from doing this.
The first one is you actually essentially replace d by k, which means that n now just-- I'm sorry, let's say S non-zero coefficients.
You replace d by S, which means that n only has to be much larger than S for this thing to actually work.
Now, of course, you've set your goal weaker.
Your goal is not to find any direction, only a sparse direction.
But there's something very valuable about sparse directions, is that they actually are interpretable.
When I found the v-- let's say that the v that I found before was 0.2, and then 0.9, and then 1.1 minus 3, et cetera.
So that was the coordinates of my leading eigenvector in the original coordinate system.
What does it mean?
Well, it means that if I see a large number, that means that this v is very close-- so that's my original coordinate system.
Let's call it e1 and e2.
So that's just 1, 0; and then 0, 1.
Then clearly, from the coordinates of v, I can tell if my v is like this, or it's like this, or it's like this.
Well, I mean, they should all be of the same size.
So I can tell if it's here or here or here, depending on-- like here, that means I'm going to see something where the Y-coordinate it much larger than the X-coordinate.
Here, I'm going to see something where the X-coordinate is much larger than the Y-coordinate.
And here, I'm going to see something where the X-coordinate is about the same size of the Y-coordinate.
So when things starts to be bigger, you're going to have to make choices.
What does it mean to be bigger-- when d is 100,000, I mean, the sum of the squares of those guys have to be equal to 1.
So they're all very small numbers.
And so it's hard for you to tell which one is a big number and which ones is a small number.
Why would you want to know this?
Because it's actually telling you that if v is very close to e1, then that means that e1-- in the case of the gene example, that would mean that e1 is the gene that's very important.
Maybe there's actually just two genes that explain those two things.
And those are the genes that have been picked up.
There's two genes that I encode geographic location, and that's it.
And so it's very important for you to be able to interpret what v means.
Where it has large values, it means that maybe it has large values for e1, e2, and e3.
And it means that it's a combination of e1, e2, and e3.
And now, you can interpret, because you have only three variables to find.
And so sparse PCA builds that in.
Sparse PCA says, listen, I'm going to want to have at most 10 non-zero coefficients.
And the rest, I want to be 0.
I want to be able to be a combination of at most 10 of my original variables.
And now, I can do interpretation.
So the problem with sparse PCA is that it becomes very difficult numerically to solve this problem.
I can write it.
So the problem is simply maximize the variance u transpose, say, Su subject to-- well, I wanted to have u2 equal to 1.
So that's the original PCA.
But now, I also want that the sum of the indicators of the uj that are not equal to 0 is at most, say, 10.
This constraint is very non-convex.
So I can relax it to a convex one like we did for linear aggression.
But now, I've totally messed up with the fact that I could use linear algebra to solve this problem.
And so now, you have to go through much more complicated optimization techniques, which are called semidefinite programs, which do not scale well in high dimensions.
And so you have to do a bunch of tricks-- numerical tricks.
But there are some packages that implements some heuristics or some other things-- iterative thresholding, all sorts of various numerical tricks that you can do.
But the problem they are trying to solve is exactly this.
Among all directions that I have norm 1, of course, because it's the direction that have at most, say, 10 non-zero coordinates, I want to find the one that maximizes the empirical variance.
Actually, let me let me just so you this.
I wanted to show you an output of PCA where people are actually trying to do directly-- maybe-- there you go.
So right here, you see this is SPSS.
That's a statistical software.
And this is an output that was preprocessed by a professional-- not preprocessed, post-processed.
So that's something where they read PCA.
So what is the data?
This is raw data about you ask doctors what they think of the behavior of a particular sales representative for pharmaceutical companies.
So pharmaceutical companies are trying to improve their sales force.
And they're asking doctors how would they rate-- what do they value about their interaction with a sales representative.
So basically, there's a bunch of questions.
One offers credible point of view on something trends, provides valuable networking opportunities.
This is one question.
Rate this on a scale from 1 to 5.
That was the question.
And they had a bunch of questions like this.
And then they asked 1,000 doctors to make those ratings.
And what they want-- so each doctor now is a vector of ratings.
And they want to know if there's different groups of doctors, what do doctors respond to.
If there's different groups, then maybe they know that they can actually address them separately, et cetera.
And so to do that, of course, there's lots of questions.
And so what you want is to just first project into lower dimensions, so you can actually visualize what's going on.
And this is what was done for this.
So these are the three first principal component that came out.
And even though we ordered the values of the lambdas, there's no reason why the entries of v should be ordered.
And if you look at the values of v here, they look like they're pretty much ordered.
It starts at 0.784, and then you're at 0.3 around here.
There's something that goes up again, and then you go down.
Actually, it's marked in red every time it goes up again.
And so now, what they did is they said, OK, I need to interpret those guys.
I need to tell you what this is.
If you tell me, we found the principal component that really discriminates the doctors in two groups, the drug company is going to come back to you and say, OK, what is this characteristic?
And you say, oh, it's actually a linear combination of 40 characteristics.
And they say, well, we don't need you to do that.
I mean, it cannot be a linear combination of anything you didn't ask.
And so for that, first of all, there's a post-processing of PCA, which says, OK, once I actually, say, found three principal components, that means that I found the dimension three space on which I want to project my points.
In this base, I can pick any direction I want.
So the first thing is that you do some sort of local arrangements, so that those things look like they are increasing and then decreasing.
So you just change, you rotate your coordinate system in this three dimensional space that you've actually isolated.
And so once you do this, the reason to do that is that it sort of makes them big, sharp differences between large and small values of the coordinates of the thing you had.
And why do you want this?
Because now, you can say, well, I'm going to start looking at the ones that have large values.
And what do they say?
They say in-depth knowledge, in-depth knowledge, in-depth knowledge, knowledge about.
This thing is clearly something that actually characterizes the knowledge of my sales representative.
And so that's something that doctors are sensitive to.
That's something that really discriminates the doctors in a way.
There's lots of variance along those things, or at least a lot of variance-- I mean, doctors are separate in terms of their experience with respect to this.
And so what they did is said, OK, all these guys, some of those they have large values, but I don't know how to interpret them.
And so I'm just going to put the first block, and I'm going to call it medical knowledge, because all those things are knowledge about medical stuff.
Then here, I didn't know how to interpret those guys.
But those guys, there's a big clump of large coordinates, and they're about respectful of my time, listens, friendly but courteous.
This is all about the quality of interaction.
So this block was actually called quality of interaction.
And then there was a third block, which you can tell starts to be spreading a little thin.
There's just much less of them.
But this thing was actually called fair and critical opinion.
And so now, you have three discriminating directions.
And you can actually give them a name.
Wouldn't it be beautiful if all the numbers in the gray box came non-zero and all the other numbers came zero-- there was no ad hoc choice.
I mean, this is probably an afternoon of work to like scratch out all these numbers and put all these color codes, et cetera.
Whereas, you could just have something that tells you, OK, here are the non-zeros.
If you can actually make a story around why this group of thing actually makes sense, such as it is medical knowledge, then good for you.
Otherwise, you could just say, I can't.
And that's what sparse PCA does for you.
Sparse PCA outputs something where all those numbers would be zero.
And there would be exactly, say, 10 non-zero coordinates.
And you can turn this knob off 10.
You can make it 9.
Depending on what your major is, maybe you can actually go on with 20 of them and have the ability to tell the story about 20 different variables and how they fit in the same group.
And depending on how you feel, it's easy to rerun the PCA depending on the value that you want here.
And so you could actually just come up with the one you prefer.
And so that's the sparse PCA thing which I'm trying to promote.
I mean, this is not super well-spread.
It's a fairly new idea, maybe at most 10 years old.
And it's not completely well-spread in statistical packages.
But that's clearly what people are trying to emulate currently.
Yes?
So what exactly does it mean that the doctors have a lot of variance in medical knowledge, quality of interaction, and fair and critical opinion?
Like, it was saying that these are like the main things that doctors vary on, some doctors care.
Like we could sort of characterize a doctor by, oh, he cares this much about medical knowledge, this much about the quality of interaction, and this much about critical opinion.
And that says most of the story about what this doctor wants from a drug representative?
Not really.
I mean, OK, let's say you pick only one.
So that means that you would take all your doctors, and you would have one direction, which is quality of interaction.
And there would be just spread out points here.
So there are two things that can happen.
The first one is that there's a clump here, and then there's a clump here.
That still represents a lot of variance.
And if this happens, you probably want to go back in your data and see were these people visited by a different group than these people, or maybe these people have a different specialty.
I mean, you have to look back at your data and try to understand why you would have different groups of people.
And if it's like completely evenly spread out, then all it's saying is that, if you want to have a uniform quality of interaction, you need to take measures on this.
You need to have this to not be discrimination.
But I think really when it's becoming interesting it's not when it's complete spread out.
It's when there's a big group here.
And then there's almost no one here, and then there's a big group here.
And then maybe there's something you can do.
And so those two things actually give you a lot of variance.
So actually, maybe I'll talk about this.
Here, this is sort of a mixture.
You have a mixture of two different populations of doctors.
And it turns out that principal component analysis-- so a mixture is when you have different populations-- think of like two Gaussians that are just centered at two different points, and maybe they're in high dimensions.
And those are clusters of people, and you want to be able to differentiate those guys.
If you're in very high dimensions, it's going to be very difficult.
But one of the first processing tools that people do is to do PCA.
Because if you have one big group here and one big group here, it means that there's a lot of variance along the direction that goes through the centers of those groups.
And that's essentially what happened here.
You could think of this as being two blobs in high dimensions.
But you're really just projecting them into one dimension.
And this dimension, hopefully, goes through the center.
And so as preprocessing-- so I'm going to stop here.
But PCA is not just made for dimension reduction.
It's used for mixtures, for example.
It's also used when you have graphical data.
What is the idea of PCA?
It just says, if you have a matrix that seems to have low rank-- meaning that there's a lot of those lambda i's that are very small-- and then I see that plus noise, then it's a good idea to do PCA on this thing.
And in particular, people use that in networks a lot.
So you take the adjacency matrix of a graph-- well, you sort of preprocess it a little bit, so it looks nice.
And then if you have, for example, two communities in there, it should look like something that is low rank plus some noise.
And low rank means that there's just very few non-zero-- well, low rank means this.
Low rank means that if you do the scree plot, you will see something like this, which means that if you throw out all the smaller ones, it should not really matter in the overall structure.
And so you can use all-- these techniques are used everywhere these days, not just in PCA.
So we call it PCA as statisticians.
But people call it the spectral methods or SVD.
So everyone--
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare and ocw.mit.edu
The chapter is a natural capstone chapter for this entire course.
We'll see some of the things we've seen during maximum likelihood and some of the things we've seen during linear regression, some of the things we've seen in terms of the basic modeling that we've had before.
We're not going to go back to much inference questions.
It's really going to be about modeling.
And in a way, generalized linear models, as the word says, are just a generalization of linear models.
And they're actually extremely useful.
They're often forgotten about and people just jump onto machine learning and sophisticated techniques.
But those things do the job quite well.
So let's see in what sense they are a generalization of the linear models.
So remember, the linear model looked like this.
We said that y was equal to x transpose beta plus epsilon, right?
That was our linear regression model.
And it's-- another way to say this is that if-- and let's assume that those were, say, Gaussian with mean 0 and identity covariance matrix.
Then another way to say this is that the conditional distribution of y given x is equal to-- sorry, I a Gaussian with mean x transpose beta and variance-- well, we had a sigma squared, which I will forget as usual-- x transpose beta and then sigma squared.
OK, so here, we just assumed that-- so what is regression is just saying I'm trying to explain why as a function of x.
Given x, I'm assuming a distribution for the y.
And this x is just going to be here to help me model what the mean of this Gaussian is, right?
I mean, I could have something crazy.
I could have something that looks like y given x is n0 x transpose beta.
And then this could be some other thing which looks like, I don't know, some x transpose gamma squared times, I don't know, x, x transpose plus identity-- some crazy thing that depends on x here, right?
And we deliberately assumed that all the thing that depends on x shows up in the mean, OK?
And so what I have here is that y given x is a Gaussian with a mean that depends on x and covariance matrix sigma square identity.
Now the linear model assumed a very specific form for the mean.
It said I want the mean to be equal to x transpose beta which, remember, was the sum from, say, j equals 1 to p of beta j xj, right?
It's where the xj's are the coordinates of x.
But I could do something also more complicated, right?
I could have something that looks like instead , replace this by, I don't know, sum of beta j log of x to the j divided by x to the j squared or something like this, right?
I could do this as well.
So there's two things that we have assumed.
The first one is that when I look at the conditional distribution of y given x, x affects only the mean.
I also assume that it was Gaussian and that it affects only the mean.
And the mean is affected in a very specific way, which is linear in x, right?
So this is essentially the things we're going to try to relax.
So the first thing that we assume, the fact that y was Gaussian and had only its mean [INAUDIBLE] dependant no x is what's called the random component.
It just says that the response variables, you know, it sort of makes sense to assume that they're Gaussian.
And everything was essentially captured, right?
So there's this property of Gaussians that if you tell me-- if the variance is known, all you need to tell me to understand exactly what the distribution of a Gaussian is, all you need to tell me is its expected value.
All right, so that's this mu of x.
And the second thing is that we have this link that says, well, I need to find a way to use my x's to explain this mu you and the link was exactly mu of x was equal to x transpose beta.
Now we are talking about generalized linear models.
So this part here where mu of x is of the form-- the way I want my beta, my x, to show up is linear, this will never be a question.
In principle, I could add a third point, which is just question this part, the fact that mu of x is x transpose beta.
I could have some more complicated, nonlinear function of x.
And then we'll never do that because we're talking about generalized linear model.
The only thing with generalize are the random component, the conditional distribution of y given x, and the link that just says, well, once you actually tell me that the only thing I need to figure out is the mean, I'm just going to slap it exactly these x transpose beta thing without any transformation of x transpose beta.
So those are the two things.
It will become clear what I mean.
This sounds like a tautology, but let's just see how we could extend that.
So what we're going to do in generalized linear models-- right, so when I talk about GLNs, the first thing I'm going to do with my x is turn it into some x transpose beta.
And that's just the l part, right?
I'm not going to be able to change.
That's the way it works.
I'm not going to do anything non-linear.
But the two things I'm going to change is this random component, which is that y, which used to be some Gaussian with mean mu of x here in sigma squared-- so y given x, sorry-- this is going to become y given x follows some distribution.
And I'm not going to allow any distribution.
I want something that comes from the exponential family.
Who knows what the exponential family of distribution is?
This is not the same thing as the exponential distribution.
It's a family of distributions.
All right, so we'll see that.
It's-- wow.
What can that be?
Oh yeah, that's actually [INAUDIBLE]..
So-- I'm sorry?
[INAUDIBLE]
I'm in presentation mode.
That should not happen.
OK, so hopefully, this is muted.
So essentially, this is going to be a family of distributions.
And what makes them exponential typically is that there's an exponential that shows up in the definition of the density, all right?
We'll see that the Gaussian belongs to the exponential family.
But they're slightly less expected ones because there's this crazy thing that a to the x is exponential x log a, which makes the potential show up without being there.
So if there's an exponential of some power, it's going to show up.
But it's more than that.
So we'll actually come to this particular family of distribution.
Why this particular family?
Because in a way, everything we've done for the linear model with Gaussian is going to extend fairly naturally to this family.
All right, and it actually also, because it encompasses pretty much everything, all the distributions we've discussed before.
All right, so the second thing that I want to question-- right, so before, we just said, well, mu of x was directly equal to this thing.
Mu of x was directly x transpose beta.
So I knew I was going to have an x transpose beta and I said, well, I could do something with this x transpose beta before I used it to explain the expected value.
But I'm actually taking it like that.
Here, we're going to say, let's extend this to some function is equal to this thing.
Now admittedly, this is not the most natural way to think about it.
What you would probably feel more comfortable doing is write something like mu of x is a function.
Let's call it f of x transpose beta.
But here, I decide to call f g inverse.
OK, let's just my g inverse.
Yes
Is this different then just [INAUDIBLE]
Yeah.
I mean, what transformation you want to put on your x's?
[INAUDIBLE]
Oh no, certainly not, right?
I mean, if I give you-- if I force you to work with x1 plus x2, you cannot work with any function of x1 plus any function of x2, right?
So this is different.
All right, so-- yeah.
The transformation would be just the simple part of your linear regression problem where you would take your exes, transform them, and then just apply another linear regression.
This is genuinely new.
Any other question?
All right, so this function g and the reason why I sort of have to, like, stick to this slightly less natural way of defining it is because that's g that gets a name, not g inverse that gets a name.
And the name of g is the link function.
So if I want to give you a generalized linear model, I need to give you two ingredients.
The first one is the random component, which is the distribution of y given x.
And it can be anything in what's called the exponential family of distributions.
So for example, I could say, y given x is Gaussian with mean mu x sigma identity.
But I can also tell you y given x is gamma with shared parameter equal to alpha of x, OK?
I could do some weird things like this.
And the second thing is I need to give you a link function.
And the link function is going to become very clear how you pick a link function.
And the only reason that you actually pick a link function is because of compatibility.
This mu of x, I call it mu because mu of x is always the conditional expectation of y given x, always, which means that let's think of y as being a Bernoulli random variable.
Where does mu of x live?
[INAUDIBLE]
0, 1, right?
That's the expectation of a Bernoulli.
It's just the probability that my coin flip gives me 1.
So it's a number between 0 and 1.
But this guy right here, if my x's are anything, right-- think of any body measurements plus [INAUDIBLE] linear combinations with arbitrarily large coefficients.
This thing can be any real number.
So the link function, what it's effectively going to do is make those two things compatible.
It's going to take my number which, for example, is constrained to be between 0 and 1 and map it into the entire real line.
If I have mu which is forced to be positive, for example, in an exponential distribution, the mean is positive, right?
That's the, say, don't know, inter-arrival time for Poisson process.
This thing is known to be positive for an exponential.
I need to map something that's exponential to the entire real line.
I need a function that takes something positive and [INAUDIBLE] everywhere.
So we'll see.
By the end of this chapter, you will have 100 ways of doing this, but there are some more traditional ones [INAUDIBLE].
So before we go any further, I gave you the example of a Bernoulli random variable.
Let's see a few examples that actually fit there.
Yes
Will it come up later [INAUDIBLE] already know why do we need the transformer [INAUDIBLE] why don't [INAUDIBLE]
Well actually, this will not come up later.
It should be very clear from here because if I actually have a model, I just want it to be plausible, right?
I mean, what happens if I suddenly decide that my-- so this is what's going to happen.
You're going to have only data to fit this model.
Let's say you actually forget about this thing here.
You can always do this, right?
You can always say I'm going to pretend my y's just happen to be the realizations of said Gaussians that happen to be 0 or 1 only.
You can always, like, stuff that in some linear model, right?
You will have some least squares estimated for beta.
And it's going to be fine.
For all the points that you see, it will definitely put some number that's actually between 0 and 1.
So this is what your picture is going to look like.
You're going to have a bunch of values for x.
This is your y.
And for different-- so these are the values of x that you will get.
And for a y, you will see either a 0 or a 1, right?
Right, that's what your Bernoulli dataset would look like with a one dimensional x.
Now if you do least squares on this, you will find this.
And for this guy, this line certainly takes values between 0 and 1.
But let's say now you get an x here.
You're going to actually start pretending that the probability it spits out one conditionally in x is like 1.2, and that's going to be weird.
Any other questions?
All right, so let's start with some examples.
Right, I mean, you get so used to them through this course.
So the first one is-- so all these things are taken.
So there's a few books on generalizing, your models, generalize [INAUDIBLE] models.
And there's tons of applications that you can see.
Those are extremely versatile, and as soon as you want to do modeling to explain some y given x, you sort of need to do that if you want to go beyond linear models.
So this was in the disease occurring rate.
So you have a disease epidemic and you want to basically model the expected number of new cases given-- at a certain time, OK?
So you have time that progresses for each of your reservation.
Each of your reservation is a time stamp-- say, I don't know, 20th day.
And your response is the number of new cases.
And you're going to actually put your model directly on mu, right?
When I looked at this, everything here was on mu itself, on the expected, right?
Mu of x is always the expected-- the conditional expectation of y given x. right?
So all I need to model is this expected value.
So this mu I'm going to actually say-- so I look at some parameters, and it says, well, it increases exponentially.
So I want to say I have some sort of exponential trend.
I can parametrize that in several ways.
And the two parameters I want to slap in is, like, some sort of gamma, which is just the coefficient.
And then there's some rate delta that's in the exponential.
So if I tell you it's exponential, that's a nice family of functions you might want to think about, OK?
So here, mu of x, if I want to keep the notation, x is gamma exponential delta x, right?
Except that here, my x are t1, t2, t3, et cetera.
And I want to find what the parameters gamma and delta are because I want to be able to maybe compare different epidemics and see if they have the same parameter or maybe just do some prediction based on the data that I have without-- to extrapolate in the future.
So here, clearly mu of x is not of the form x transpose beta, right?
That's not x transpose beta at all.
And it's actually not even a function of x transpose data, right?
There's two parameters, gamma and delta, and it's not of the form.
So here we have x, which is 1 and x, right?
I have two parameters.
So what I do here is that I say, well, first, let me transform mu in such a way that I can hope to see something that's linear.
So if I transform mu, I'm going to have log of mu, which is log of this thing, right?
So log of mu of x is equal, well, to log of gamma plus log of exponential delta x, which is delta x.
And now this thing is actually linear in x.
So I have that this guy is my first beta 1.
And so that's beta 1 finds 1.
And this guy is beta 2-- times, sorry that said beta 0-- times 1, and this guy is beta 1 times x. OK, so that looks like a linear model.
I just have to change my parameters-- my parameters beta 1 becomes the log of gamma and beta 2 becomes delta itself.
And the reason why we do this is because, well, the way we put those gamma and those delta was just so that we have some parametrization.
It just so happens that if we want this to be linear, we need to just change the parametrization itself.
This is going to have some effects.
We know that it's going to have some effect in the fissure information.
It's going to have a bunch of effect to change those things.
But that's what needs to be done to have a generalized linear model.
Now here, the function that I took to turn it into something that's linear is simple.
It came directly from some natural thing I would do here, which is taking the log.
And so the function g, the link that I take, is called the log link very creatively.
And it's just the function that I apply to mu so that I see something that's linear and that looks like this.
So now this only tells me how to deal with the link function.
But I still have to deal with 0.1.
And this, again, is just some modeling.
Given some data, some random data, what distribution do you choose to explain the randomness?
And this-- I mean, unless there's no choice, you know, it's just a matter of practice, right?
I mean, why would it be Gaussian and not, you know, doubly exponential?
This is-- there's matters of convenience that come into this, and there's just matter of experience that come into this.
You know, I remember when you chat with engineers, they have a very good notion of what the distribution should be.
They have y bold distributions.
You know, they do optics and things like this.
So there's some distributions that just come up but sometimes just have to work.
Now here what do we have?
The thing we're trying to measure, y-- as we said, so mu is the expectation, the conditional expectation, of y given x.
But y is the number of new cases, right?
Well it's a number of.
And the first thing you should think of when you think about number of, if it were bounded above, you would think binomial, baby.
But here, it's just a number.
So you think Poisson.
That's how insurers think.
I have a number of, you know, claims per year.
This is a Poisson distribution.
And hopefully they can model the conditional distribution of the number of claims given everything that they actually ask you in the surveys that I hear you now fail in 15 minutes.
All right, so now you have this Poisson distribution.
And that's just the modeling assumption.
There's no particular reason why you should do this except that, you know, that might be a good idea.
And the expected value of your Poisson has to be this mu i, OK?
At time i.
Any question about this slide?
OK, so let's switch to another example.
Another example is the so-called pray capture rate.
So here, what you're interested in is the rate capture of preys yi for a given prey.
And you have xy, which is your explanation.
And this is just the density of pray.
So you're trying to explain the rate of captures of preys given the density of the prey, OK?
And so you need to find some sort of relationship between the two.
And here again, you talk to experts and what they tell you is that, well, it's going to be increasing, right?
I mean, animals like predators are going to just eat more if there's more preys.
But at some point, they're just going to level off because they're going to be [INAUDIBLE] full and they're going to stop capturing those prays.
And you're just going to have some phenomenon that looks like this.
So here is a curve that sort of makes sense, right?
As your capture rate goes from 0 to 1, you're increasing, and then you see you have this like [INAUDIBLE] function that says, you know, at some point it levels up.
OK, so here, one way I could-- I mean, there's again many ways I could just model a function that looks like this.
But a simple one that has only two parameters is this one, where mu i is this a function of xi where I have some parameter alpha here and some parameter h here.
OK, so there's clearly-- so this function, there's one that essentially tells you-- so this thing starts at 0 for sure.
And essentially, alpha tells you how sharp this thing is, and h tells you at which points you end here.
Well, it's not exactly what those values are equal to, but that tells you this.
OK, so, you know-- simple, and-- well, no, OK.
Sorry, that's actually alpha, which is the maximum capture.
The rate and h represent the pre-density at which the capture weight is.
So that's the half time.
OK, so there's actual value [INAUDIBLE].. All right, so now I have this function.
It's certainly not a function.
There's no-- I don't see it as a function of x.
So I need to find something that looks like a function of x, OK?
So then here, there's no log.
There's no-- well, I could actually take a log here.
But I would have log of x and log of x plus h. So that would be weird.
So what we propose to do here is to look, rather than looking at mu i, we look 1 over mu i.
Right, and so since your function was mu i, when you take 1 over mu i, you get h plus xi divided by alpha xi, which is h over alpha times one over xi plus 1 over alpha.
And now if I'm willing to make this transformation of variables and say, actually, I don't-- my x, whether it's the density of prey or the inverse density of prey, it really doesn't matter.
I can always make this transformation when the data comes.
Then I'm actually just going to think of this as being some linear function beta 0 plus beta 1, which is this guy, times 1 over xi.
And now my new variable becomes 1 over xi.
And now it's linear.
And the transformation I had to take was this 1 over x, which is called the reciprocal link, OK?
You can probably guess what the exponential link is going to be and things like this, all right?
So we'll talk about other links that have slightly less obvious names.
Now again, modeling, right?
So this was the random component.
This was the easy part.
Now I need to just poor in some domain knowledge about how do I think this function, this y, which is which is the rate of capture of praise, I want to understand how this thing is actually changing what is the randomness of the thing around its mean.
And you know, something that-- so that comes from this textbook.
The standing deviation of capture rate might be approximately proportional to the mean rate.
You need to find a distribution that actually has this property.
And it turns out that this happens for gamma distributions, right?
In gamma distributions, just like say, for Poisson distribution, the-- well, for Poisson, the variance and mean are of the same order.
Here is the standard deviation that's of the same order as the [INAUDIBLE] for gammas.
And it's a positive distribution as well.
So here is a candidate.
Now since we're sort of constrained to work under the exponential family of distributions, then you can just go through your list and just decide which one works best for you.
All right, third example-- so here we have binary response.
Here, essentially the binary response variable indicates the presence or absence of postoperative deforming for kyphosis on children.
And here, rather than having one covariance which was before, in the first example, was time, in the second example was the density, here there's three ways that you measure on children.
The first one is age of the child and the second one is the number of vertebrae involved in the operation.
And the third one is the start of the range, right-- so where it is on the spine.
OK, so the response variable here is, you know, did it work or not, right?
I mean, that's very simple.
And so here, it's nice because the random component is the easiest one.
As I said, any random variable that takes only two outcomes must be a Bernoulli, right?
So that's nice there's no modeling going on here.
So you know that y given x is going to be Bernoulli, but of course, all your efforts are going to try to understand what the conditional mean of your Bernoulli, what the conditional probability of being 1 is going to be, OK?
And so in particular-- so I'm just-- here, I'm spelling it out before we close those examples.
I cannot say that mu of x is x transpose data for exactly this picture that I drew for you here, right?
There's just no way here-- the goal of doing this is certainly to be able to extrapolate for yet unseen children whether this is something that we should be doing.
And maybe the range of x is actually going to be slightly out.
And so, OK I don't want to see that have a negative probability of outcome or a positive one-- sorry, or one that's lower than one.
So I need to make this transformation.
So what I need to do is to transform mu, which is, we know only a number.
All we know is a number between 0 and 1.
And we need to transform it in such a way that it maps the entire real line or reciprocally to say that-- or inversely, I should say-- that f of x transpose beta should be a number between 0 and 1.
I need to find a function that takes any real number and maps it into 0 and 1.
And we'll see that again, but you have an army of functions that do that for you.
What are those functions?
[INAUDIBLE]
I'm sorry?
[INAUDIBLE]
Trait?
[INAUDIBLE]
Oh
[INAUDIBLE]
Yeah, I want them to be invertible, right?
[INAUDIBLE]
I have an army of function.
I'm not asking for one soldier in this army.
I want the name of this army
[INAUDIBLE]
Well, they're not really invertible either, right?
So they're actually in [INAUDIBLE] textbook.
Because remember, statisticians don't know how to integrate functions, but they know how to turn a function into a Gaussian integral.
So we know it integrates to 1 and things like this.
Same thing here-- we don't know how to build functions that are invertible and map the entire real line to 0, 1, but there's all the cumulative distribution functions that do that for us.
So I can you any of those guys, and that's what I'm going to be doing, actually.
All right, so just to recap what I just said as we were speaking, so normal linear model is not appropriate for these examples if only because the response variable is not necessarily Gaussian and also because the linear model has to be-- the mean has to be transformed before I can actually apply a linear model for all these plausible nonlinear models that I actually came up with.
OK, so the family we're going to go for is the exponential family of distributions.
And we're going to be able to show-- so one of the nice part of this is to actually compute maximum likelihood estimaters for those right?
In the linear model, maximum-- like, in the Gauss linear model, maximum likelihood was as nice as it gets, right?
This actually was the least squares estimator.
We had a close form.
x transpose x inverse x transpose y, and that was it, OK?
We had to just take one derivative.
Here, we're going to have a generally concave likelihood.
We're not going to be able to actually solve this thing directly in close form unless it's Gaussian, but we will have-- we'll see actually how this is not just a black box optimization of a concave function.
We have a lot of properties of this concave function, and we will be able to show some iterative algorithms.
We'll basically see how, when you opened the box of convex optimization, you will actually be able to see how things work and actually implement it using least squares.
So each iteration of this iterative algorithm will essentially be a least squares, and that's actually quite [INAUDIBLE]..
So, very demonstrative of statisticians being pretty ingenious so that they don't have to call in some statistical software but just can repeatedly call their least squares Oracle within a statistical software.
OK, so what is the exponential family, right?
I promised to do the exponential family.
Before we go into this, let me just tell you something about exponential families, and what's the only thing to differentiate an exponential family from all possible distributions?
An exponential family has two parameters, right?
And those are not really parameters, but there's this theta parameter of my distribution, OK?
So it's going to be indexed by some parameter.
Here, I'm only talking about the distribution of, say, some random variable or some random vector, OK?
So here in this slide, you see that the parameter theta that indexed those distribution is k dimensional and the space of the x's that I'm looking at-- so that should really be y, right?
What I'm going to plug in here is the conditional distribution of y given x and theta is going to depend on x.
But this really is the y.
That's their distribution of the response variable.
And so this is on q, right?
So I'm going to assume that y takes-- q dimensional-- is q dimensional.
Clearly soon, q is going to be equal to 1, but I can define those things generally.
OK, so I have this.
I have to tell you what this looks like.
And let's assume that this is a probability density function.
So this, right this notation, the fact that I just put my theta in subscript, is just for me to remember that this is the variable that indicates the random variable, and this is just the parameter.
But I could just write it as a function of theta and x, right?
This is just going to be-- right, if you were in calc, in multivariable calc, you would have two parameter of theta and x and you would need to give me a function.
Now think of all-- think of x and theta as being one dimensional at this point.
Think of all the functions that can be depending on theta and x.
There's many of them.
And in particular, there's many ways theta and x can interact.
What the exponential family does for you is that it restricts the way these things can actually interact with each other.
It's essentially saying the following.
It's saying this is going to be of the form exponential-- so this exponential is really not much because I could put a log next to it.
But what I want is that the way theta and x interact has to be of the form theta times x in an exponential, OK?
So that's the simplest-- that's one of the ways you can think of them interacting is you just the product of the two.
Now clearly, this is not a very rich family.
So what I'm allowing myself is to just slap on some terms that depend only on theta and depend only on x.
So let's just call this thing, I don't know, f of x, g of theta.
OK, so here, I've restricted the way theta and x can interact.
So I have something that depends only on x, something that depends only on theta.
And here, I have this very specific interaction.
And that's all that exponential families are doing for you, OK?
So if we go back to this slide, this is much more general, right?
if I want to go from theta and x in r to theta and x theta in r-- to theta in r k and x in rq, I cannot take the product of theta and x. I cannot even take the inner product between theta and x because they're not even of compatible dimensions.
But what I can do is to first map my theta into something and map my x into something so that I actually end up having the same dimensions.
And then I can take the inner product.
That's the natural generalization of this simple product.
OK, so what I have is-- right, so if I want to go from theta to x, when I'm going to first do is I'm going to take theta, eta of theta-- so let's say eta1 of theta to eta k of theta.
And then I'm going to actually take x becomes t1 of x all the way to tk of x.
And what I'm going to do is take the inner product-- so let's call this eta and let's call this t. And I'm going to take the inner product of eta and t, which is just the sum from j equal 1 to k of eta j of theta times tj of x. OK, so that's just a way to say I want this simple interaction but in higher dimension.
The simplest way I can actually make those things happen is just by taking inner product.
OK, and so now what it's telling me is that the distribution-- so I want the exponential times something that depends only on theta and something that depends only on x.
And so what it tells me is that when I'm going to take p of theta x, it's just going to be something which is exponential times the sum from j equal 1 to k of eta j theta tj of x.
And then I'm going to have a function that depends only-- so let me read it for now like c of theta and then a function that depends only on x.
Let me call it h of x.
And for convenience, there's no particular reason why I do that.
I'm taking this function c of theta and I'm just actually pushing it in there.
So I can write c of theta as exponential minus log of 1 over c of theta, right?
And now I have exponential times exponential.
So I push it in, and this thing actually looks like exponential sum from j equal 1 to k of eta j theta tj of x minus log 1 over c of theta times h of x.
And this thing here, log 1 over c of theta, I call actually b of theta Because c, I called it c. But I can actually directly call this guy b, and I don't actually care about c itself.
Now why don't I put back also h of x in there?
Because h of x is really here to just-- how to put it-- OK, h of x and b of theta don't play the same role.
B of theta in many ways is a normalizing constant, right?
I want this density to integrate to 1.
If I did not have this guy, I'm not guaranteed that this thing integrates to 1.
But by tweaking this function b of theta or c of theta-- they're equivalent-- I can actually ensure that this thing integrates to 1.
So b of theta is just a normalizing constant.
H of x is something that's going to be funny for us.
It's going to be something that allows us to be able to treat both discrete and continuous variables within the framework of exponential families.
So for those that are familiar with this, this is essentially saying that that h of x is really just a change of measure.
When I actually look at the density of p of theta-- this is with respect to some measure-- the fact that I just multiplied by a function of x just means that I'm not looking-- that this guy here without h of theta is not the density with respect to the original measure, but it's the density with respect to the distribution that has h as a density.
That's all I'm saying, right?
So I can first transform my x's and then take the density with respect to that.
If you don't want to think about densities or measures, you don't have to.
This is just the way-- this is just the definition.
Is there any question about this definition?
All right, so it looks complicated, but it's actually essentially the simplest way you could think about it.
You want to be able to have x and theta interact and you just say, I want the interaction to be of the form exponential x times theta.
And if they're higher dimensions, I'm going to take the exponential of the function of x inner product with a function of theta.
All right, so I claimed since the beginning that the Gaussian was such an example.
So let's just do it.
So is the Gaussian of the-- is the interaction between theta and x in a Gaussian of the form in the product?
And the answer is yes.
Actually, whether I know or not what the variance is, OK?
So let's start for the case where I actually do not know what the variance is.
So here, I have x is n mu sigma squared.
This is all one dimensional.
And here, I'm going to assume that my parameter is both mu and sigma square.
OK, so what I need to do is to have some function of mu, some function of stigma square, and take an inner product of some function of x and some other function of x.
So I want to show that-- so p theta of x is what?
Well, it's one over square root sigma 2 pi exponential minus x minus mu squared over 2 sigma squared, right?
So that's just my Gaussian density.
And I want to say that this thing here-- so clearly, the exponential shows up already.
I want to show that this is something that looks like, you know, eta 1 of-- sorry, so that was-- yeah, eta 1 of, say, mu sigma squared.
So I have only two of those guys, so I'm going to need only two etas, right?
So I want it to be eta 1 of mu and sigma times t1 of x plus eta 2 mu 1 mu sigma squared times t2 of x, right?
So I want to have something like that that shows up, and the only things that are left, I want them to depend either only on theta or only on x.
So to find that out, we just need to expand.
OK, so I'm going to first put everything into my exponential and expand this guy.
So the first term here is going to be minus x squared over 2 sigma square.
The second term is going to be minus mu squared over two sigma squared.
And then the cross term is going to be plus x mu divided by sigma squared.
And then I'm going to put this guy here.
So I have a minus log sigma over 2 pi, OK?
OK, is this-- so this term here contains an interaction between X and the parameters.
This term here contains an interaction between X and the parameters.
So let me try to write them in a way that I want.
This guy only depends on the parameters, this guy only depends on the parameter.
So I'm going to rearrange things.
And so I claim that this is of the form x squared.
Well, let's say-- do-- who's getting the minus?
Eta, OK.
So it's x squared times minus 1 over 2 sigma squared plus x times mu over sigma squared, right?
So that's this term here.
That's this term here.
Now I need to get this guy here, and that's minus.
So I'm going to write it like this-- minus, and now I have mu squared over 2 sigma squared plus log sigma square root 2 pi.
And now this thing is definitely of the form t of x times-- did I call them the right way or not?
Of course not.
OK, so that's going to be t2 of x times eta 2 of x eta 2 of theta.
This guy is going to be t1 of x times eta 1 of theta.
All right, so just a function of theta times a function of x-- just a function of theta times a function of x.
And the way combined is just by sending them.
And this is going to be my d of theta.
What is h of x?
1
1.
There's one thing I can actually play with, and this is something you're going to have some three choices, right?
This is not actually completely determined here is that-- for example, so when I write the log sigma square root 2 pi, this is just log of sigma plus log square root 2 pi.
So I have two choices here.
Either my b becomes this guy, or-- so either I have b of theta, which is mu squared over 2 sigma squared plus log sigma square root 2 pi and h of x is equal to 1, or I have that b of theta is mu square over 2 sigma squared plus log sigma.
And h of x is equal to what?
Well, I can just push this guy out, right?
I can push it out of the exponential.
And so it's just square root of 2 pi, which is a function of x, technically.
I mean, it's a constant function of x, but it's a function.
So you can see that it's not completely clear how you're going to do the trade off, right?
So the constant terms can go either in b or in h. But you know, why bother with tracking down b and h when you can actually stuff everything into one and just call h one and call it a day?
Right, so you can just forget about h. You know it's one and think about the right.
H won't matter actually for estimation purposes or anything like this.
All right, so that's basically everything that's written.
When stigma square is known, what's happening is that this guy here is no longer a function of theta, right?
Agreed?
This is no longer a parameter.
When sigma square is known, then theta is equal to mu only.
There's no sigma square going on.
So this-- everything depends on sigma square can be thought of as a constant.
Think one.
So in particular, this term here does not belong in the interaction between x and theta.
It belongs to h, right?
So if sigma is known, then this guy is only a function of h-- of x.
So h of x becomes exponential x squared minus x squared over 2 sigma squared, right?
That's just a function of x.
Is that clear?
So if you complete this computation, what you're going to get is that your new one parameter thing is that p theta x is not equal to exponential x times mu over sigma squared minus-- well, it's still the same thing.
And then you have your h of x that comes out-- x squared over 2 sigma squared.
OK, so that's my h of x.
That's still my b of theta.
And this is my t1 of x.
And this is my eta one of theta.
And remember, theta is just equal to mu in this case.
So if I ask you prove that this distribution belongs to an exponential family, you just have to work it out.
Typically, it's expanding what's in the exponential and see what's-- and just write it in this term and identify all the components, right?
So here, notice those guys don't even get an index anymore because there's just one of them.
So I wrote eta 1 and t1, but it's really just eta and t. Oh sorry, this guy also goes.
This is also a constant, right?
So it can actually just put sigma divided by sigma square root 2 pi.
So h of x is what, actually?
Is it the density of--
Standard [INAUDIBLE]
It's not standard.
It's centered.
It has mean 0.
But it variance sigma squared, right?
But it's the density of a Gaussian.
And this is what I meant when I said h of x is really just telling you with respect to which distribution, which measure you're taking the density.
And so this thing here is really telling you the density of my Gaussian with mean mu is equal to-- is this with respect to a centered Gaussian is this guy, right?
That's what it means.
If this thing ends up being a density, it just means that now you just have a new measure, which is this density.
So it's just saying that the density of the Gaussian with mean mu with respect to the Gaussian with mean 0 is just this [INAUDIBLE] here.
All right, so let's move on.
So here, as I said, you could actually do all these computations and forget about the fact that x is continuous.
You can actually do it with PMFs and do it for x is discrete.
This actually also tells you if you can actually get the same form for your density, which is of the form exponential times the product of the the interaction between theta and x is just taking this product, then a function only of theta and of function only of x, for the PMF, it also works.
OK, so I claim that the Bernoulli belongs to this family.
So the PMF of a Bernoulli-- we say parameter p is p to the x 1 minus p to the 1 minus x, right?
Because we know so that's only for x equals 0 or 1.
And the reason is because when x is equal to 0, this is 1 minus p. When x is equal to 1, this is minus 0.
OK, we've seen that when we're looking at likelihoods for Bernoullis.
OK, this is not clear this is going to look like this at all.
But let's do it.
OK, so what does this thing look like?
Well, the first thing I want to do is to make an exponential show up.
So what I'm going to write is I'm going to write p to the x as exponential x log p, right?
And so I'm going to do that for the other one.
So this thing here-- so I'm going to get exponential x log p plus 1 minus x log 1 minus p. So what I need to do is to collect my terms in x and my terms in whatever parameters I have, see here if theta is equal to p. So if I do this, what I end up having is equal to exponential-- so determine x is log p minus log 1 minus p. So that's x times log p over 1 minus p. And then the term that rest is just-- that stays is just 1 times log 1 minus p. But I want to see this as a minus something, right?
It was minus b of theta.
So I'm going to write it as minus-- well, I can just keep the plus, and I'm going to do-- and that's all [INAUDIBLE].
A-ha!
Well, this is of the form exponential-- something that depends only on x times something that depends only on theta-- minus a function that depends only on theta.
And then h of x is equal to 1 again.
OK, so let's see.
So I have t1 of x is equal to x.
That's this guy.
Eta 1 of theta is equal to log p1 minus p. And b of theta is equal to log 1 over 1 minus p, OK?
And h of x is equal to 1, all right?
You guys want to do Poisson, or do you want to have any homework?
It's a dilemma because that's an easy homework versus no homework at all but maybe something more difficult.
OK, who wants to do it now?
Who does not want to raise their hand now?
Who wants to raise their hand now?
All right, so let's move on.
I'll just do-- do you want to do the gammas instead in the homework?
That's going to be fun.
I'm not even going to propose to do the gammas.
And so this is the gamma distribution.
It's brilliantly called gamma because it has the gamma function just like the beta distribution had the beta function in there.
They look very similar.
One is defined over r plus, the positive real line.
And remember, the beta was defined over the interval 0, 1.
And it's of the form x to some power times exponential of minus x to some-- times something, right?
So there's a function of polynomial [INAUDIBLE] x where the exponent depends on the parameter.
And then there's the exponential minus x times something depends on the parameters.
So this is going to also look like some function of x-- sorry, like some exponential distribution.
Can somebody guess what is going to be t2 of x?
Oh, those are the functions of x that show up in this product, right?
Remember when we have this-- we just need to take some transformations of x so it looks linear in those things and not in x itself.
Remember, we had x squared and x, for example, in the Gaussian case.
I don't know if it's still there.
Yeah, it's still there, right?
t2 was x squared.
What do you think x is going-- t2 of x here.
So here's a hint.
t1 is going to be x
[INAUDIBLE]
Yeah, [INAUDIBLE],, what is going to be t1?
Yeah, you can-- this one is taken.
This one is taken.
What?
Log x, right?
Because this x to the a minus 1, I'm going to write that as exponential a minus 1 log x.
So basically, eta 1 is going to be a minus 1.
Eta 2 is going to be minus 1 over b-- well, actually the opposite.
And then you're going to have-- but this is actually not too complicated.
All right, then those parameters get names.
a is the shape parameter, b is the scale parameter.
It doesn't really matter.
You have other things that are called the inverse gamma distribution, which has this form.
The difference is that the parameter alpha shows negatively there and then the inverse Gaussian distribution.
You know, just densities you can come up with and they just happened to fall in this family.
And there's other ones that you can actually put in there that we've seen before.
The chi-square is actually part of this family.
The beta distribution is part of this family.
The binomial distribution is part of this family.
Well, that's easy because the Bernoulli was.
The negative binomial, which is some stopping time-- the first time you hit a certain number of successes when you flip some Bernoulli coins.
So you can check for all of those, and you will see that you can actually write them as part of the exponential family.
So the main goal of this slide is to convince you that this is actually a pretty broad range of distributions because it basically includes everything we've seen but not anything there-- sorry, plus more, OK?
Yeah
Is there any example of a distribution that comes up pretty often that's not in the exponential family?
Yeah, like uniform
Oh, OK, so maybe a bit more complicated than [INAUDIBLE].
Anything Anything that has a support that depends on the parameter is not going to fall-- is not going to fit in there.
Right, and you can actually convince yourself why anything that has the support that does not-- that depends on the parameter is not going to be part of this guy.
It's kind of a hard thing to-- in fact, you proved that it's not and you prove this rule.
That's kind of a little difficult, but the way you can convince yourself is that remember, the only interaction between x and theta that I allowed was taking the product of those guys and then the exponential, right?
If you have something that depends on some parameter-- let's say you're going to see something that looks like this.
Right, for uniform, it looks like this.
Well, this is not of the form exponential x times theta.
There's an interaction between x and theta here, but it's actually certainly not of the form x exponential x times theta.
So this is definitely not going to be part of the exponential family.
And every time you start doing things like that, it's just not going to happen.
Actually, to be fair, I'm not even sure that all these guys, when you allow them to have all their parameters free, are actually going to be part of this.
For example-- the beta probably is, but I'm not actually entirely convinced.
There's books on experiential families.
All right, so let's go back.
So here, we've put a lot of effort understanding how big, how much wider than the Gaussian distribution can we think of for the conditional distribution of our response y given x.
So let's go back to the generalized linear models, right?
So [INAUDIBLE] said, OK, the random component?
y has to be part of some exponential family distribution-- check.
We know what this means.
So now I have to understand two things.
I have to understand what is the expectation, right?
Because that's actually what I model, right?
I take the expectation, the conditional expectation, of y given x.
So I need to understand given this guy, it would be nice if you had some simple rules that would tell me exactly what the expectation is rather than having to do it over and over again, right?
If I told you, here's a Gaussian, compute the expectation, every time you had to use that would be slightly painful.
So hopefully, this thing being simple enough-- we've actually selected a class that's simple enough so that we can have rules.
Whereas as soon as they give you those parameters t1, t2, eta 1, eta 2, b and h, you can actually have some simple rules to compute the mean and variance and all those things.
And so in particular, I'm interested in the mean, and I'm going to have to actually say, well, you know, this mean has to be mapped into the whole real line.
So I can actually talk about modeling this function of the mean as x transpose beta.
And we saw that for the [INAUDIBLE] dataset or whatever other data sets.
You actually can-- you can actually do this using the log of the reciprocal or for the-- oh, actually, we didn't do it for the Bernoulli.
We'll come to this.
This is the most important one, and that's called a logit it or a logistic link.
But before we go there, this was actually a very broad family, right?
When I wrote this thing on the bottom board-- it's gone now, but when I wrote it in the first place, the only thing that I wrote is I wanted x times theta.
Wouldn't it be nice if you have some distribution that was just x times theta, not some function of x times some function of theta?
The functions seem to be here so that they actually make things a little-- so the functions were here so that I can actually put a lot of functions there.
But first of all, if I actually decide to re-parametrize my problem, I can always assume-- if I'm one dimensional, I can always assume that eta 1 of theta becomes my new theta, right?
So this thing-- here for example, I could say, well, this is actually the parameter of my Bernoulli.
Let me call this guy theta, right?
I could do that.
Then I could say, well, here I have x that shows up here.
And here since I'm talking about the response, I cannot really make any transformations.
So here, I'm going to actually talk about a specific family for which this guy is not x square or square root of x or log of x or anything I want.
I'm just going to actually look at distributions for which this is x.
This exponential families are called a canonical exponential family.
So in the canonical exponential family, what I have is that I have my x times theta.
I'm going to allow myself some normalization factor phi, and we'll see, for example, that it's very convenient when I talk about the Gaussian, right?
Because even if I know-- yeah, even if I know this guy, which I actually pull into my-- oh, that's over here, right?
Right, I know sigma squared.
But I don't want to change my parameter to be mu over sigma squared.
It's kind of painful.
So I just take mu, and I'm going to keep this guy as being this phi over there.
And it's called the dispersion parameter from a clear analogy with the Gaussian, right?
That's the variance and that's measuring dispersion.
OK, so here, what I want is I'm going to think throughout this class-- so phi may be known or not.
And depending-- when it's not known, this actually might turn into some exponential family or it might not.
And the main reason is because this b of theta over phi is not necessarily a function of theta over phi, right?
If I actually have phi unknown, then y theta over phi has to be-- this guy has to be my new parameter.
And b might not be a function of this new parameter.
OK, so in a way, it may or may not, but this is not really a concern that we're going to have because throughout this class, we're going to assume that phi is known, OK?
Phi is going to be known all the time, which means that this is always an exponential family.
And it's just the simplest one you could think of-- one dimensional parameter, one dimensional response, and I just have-- the product is just y times or, we used to call it x.
Now I've switched to y, but y times theta divided by phi, OK?
Should I write this or this is clear to everyone what this is?
Let me write it somewhere so we actually keep track of it toward the [INAUDIBLE].
OK, so this is-- remember, we had all the distributions.
And then here we had the exponential family.
And now we have the canonical exponential family.
It's actually much, much smaller.
Well, actually, it's probably sort of a good picture.
And what I have is that my density or my PMF is just exponential y times theta minus b of theta divided by phi.
And I have plus phi of-- oh, yeah, plus phi of y phi, which means that this is really-- if phi is known, h of y is just exponential c of y phi, agreed?
Actually, this is the reason why it's not necessarily a canonical family.
It might not be that this depends only on y.
It could depend on y and phi in some annoying way and I may not be able to break it.
OK, but if phi is known, this is just a function that depends on y, agreed?
In particular, I think you need-- I hope you can convince yourself that this is just a subcase of everything we've seen before.
So for example, the Gaussian when the variance is known is indeed of this form, right?
So we still have it on the board.
So here is my y, right?
So then let me write this as f theta of y.
So every x is replaceable with y, blah, blah, blah.
This is this guy.
And now what I have is that this is going to be my phi.
This is my parameter of theta.
So I'm definitely of the form y times theta divided by phi.
And then here I have a function b that depends only on theta over phi again.
So b of theta is mu squared divided by 2.
OK, then it's divided by 6 sigma square.
And then I have this extra stuff.
But I really don't care what it is for now.
It's just something that depends only on y and known stuff.
So it was just a function of y just like my h. I stuff everything in there.
The b, though, this thing here, this is actually what's important because in the canonical family, if you think about it, when you know phi-- sorry-- right, this is just y times theta scaled by a known constant-- sorry, y times theta scaled by a known constant is the first term.
The second term is b of theta scaled by some known constant.
But b of theta is what's going to make the difference between the Gaussian and Bernoullis and gammas and betas-- this is all in this b of theta.
b of theta contains everything that's idiosyncratic to this particular distribution.
And so this is going to be important.
And we will see that b of theta is going to capture information about the mean, about the variance, about likelihood, about everything.
Should I go through this computation?
I mean, it's the same.
We've just done it, right?
So maybe it's probably better if you can redo it on your own.
All right, so the canonical exponential family also has other distributions, right?
So there's the Gaussian and there's the Poisson and there's the Bernoulli.
But the other ones may not be part of this, right?
In particular, think about the gamma distribution.
We had this-- log x was one of the things that showed up.
I mean, I cannot get rid of this log x. I mean, that's part of it except if a is equal to 1 and I know it for sure, right?
So if a is equal to 1, then I'm going to have a minus 1, which is equal to 0.
So I'm going to have a minus 1 times log x, which is going to be just 0.
So log x is going to vanish from here.
But if a is equal to 1, then this distribution is actually much nicer, and it actually does not even deserve the name gamma.
What is it if a is equal to 1?
It's an exponential, right?
Gamma 1 is equal to 1. x to the a minus 1 is equal to 1. b-- so I have exponential x over b divided by b.
So 1 over b-- call it lambda.
And this is just an exponential distribution.
And so every time you're going to see something-- so all these guys that don't make it to this table, they could be part of those guys, but they're just more-- they're just to-- they just have another name in this thing.
All right, so you could compute the value of theta for different values, right?
So again, you still have some continuous or discrete ones.
This is my b of theta.
And I said this is actually really what captures my theta.
This b is actually called cumulant generating function, OK?
I don't have time.
I could write five slides to explain to you, but it would just only tell you why it's called cumulant generating function.
It's also known as the log of the moment generating function.
And the way it's called cumulant generating function is because if I start taking successive derivatives and evaluating them at 0, I get the successive cumulance of this distribution, which are some transformation of the moments
What are you talking about again?
The function b
[INAUDIBLE]
So this is just normalization.
So this is just to tell you I can compute this, but I really don't care.
And obviously I don't care about stuff that's complicated.
This is actually cute, and this is what completes everything.
And the rest is just like some general description.
You only need to tell you that the range of y is 0 to infinity, right?
And that is essentially telling me this is going to give me some hints as to which link function I should be using, right?
Because the range of y tells me what the range of expectation of y is going to be.
All right, so here, it tells me that the range of y is between 0 and 1.
OK, so what I want to show you is that this captures a variety of different ranges that you can have.
OK, so I'm going to want to go into the likelihood.
And the likelihood I'm actually going to use to compute the expectations.
But since I actually don't have time to do this now, let's just go quickly through this and give you spoiler alert to make sure that you all wake up on Thursday and really, really want to think about coming here immediately.
All right, so the thing I'm going to want to do, as I said, is it would be nice if, at least for this canonical family, when I give you b, you would be able to say, oh, here is a simple computation of b that would actually give me the mean and the variance.
The mean and the variance are also known as moments.
b is called cumulant generating function.
So it sounds like moments being related to cumulance, I might have a path to finding those, right?
And it might involve taking derivatives of b, as we'll see.
The way we're going to prove this by using this thing that we've used several times.
So this property we use when we're computing, remember, the fisher information, right?
We had two formulas for the fisher information.
One was the expectation of the second derivative of the log likelihood, and one was negative expectation of the square-- sorry, expectation of the square, and the other one was negative the expectation of the second derivative, right?
The log likelihood is concave, so this number is negative, this number is positive.
And the way we did this is by just permuting some derivative and integral here.
And there was just-- we used the fact that something that looked like this, right?
The log likelihood is log of f theta.
And when I take the derivative of this guy with respect to theta, then I have something that looks like the derivative divided by f theta.
And if I start taking the integral against f theta of this thing, so the expectation of this thing, those things would cancel.
And then I had just the integral of a derivative, which I would make a leap of faith and say that it's actually the derivative of the integral.
But this was equal to 1.
So this derivative was actually equal to 0.
And so that's how you got that the expectation of the derivative of the log likelihood is equal to 0.
And you do it once again and you get this guy.
It's just some nice things that happen with the [INAUDIBLE] taking derivative of the log.
We've done that, we'll do that again.
But once you do this, you can actually apply it.
And-- missing a parenthesis over there.
So when you write the log likelihood, it's just log of an exponential.
Huh, that's actually pretty nice.
Just like the least squares came naturally, the least squares [INAUDIBLE] came naturally when we took the log likelihood of the Gaussians, we're going to have the same thing that happens when I take the log of the density.
The exponential is going to go away, and then I'm going to use this formula.
But this formula is going to actually give me an equation directly-- oh, that's where it was.
So that's the one that's missing up there.
And so the expectation minus this thing is going to be equal to 0, which tells me that the expectation is just the derivative.
Right, so it's still a function of theta, but it's just a derivative of b.
And the variance is just going to be the second derivative of b.
But remember, this was some sort of a scaling, right?
It's called the dispersion parameter.
So if I had a Gaussian and the variance of the Gaussian did not depend on the sigma squared which I stuffed in this phi, that would be certainly weird.
And it cannot depend only on mu, and so this will-- for the Gaussian, this is definitely going to be equal to 1.
And this is just going to be equal to my variance.
So this is just by taking the second derivative.
So basically, the take-home message is that this function b captures-- by taking one derivative of the expectation and by taking two derivatives captures the variance.
Another thing that's actually cool and we'll come back to this and I want to think about is if this second derivative is the variance, what can I say about this thing?
What do I know about a variance?
[INAUDIBLE]
Yeah, that's positive.
So I know that this is positive.
So what does that tell me?
Positive?
That's convex, right?
A function that has positive second derivative is convex.
So we're going to use that as well, all right?
So yeah, I'll see you on Thursday.
I have your homework.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We're talking about generalized linear models.
And in generalized linear models, we generalize linear models in two ways.
The first one is to allow for a different distribution for the response variables.
And the distributions that we wanted was the exponential family.
And this is a family that can be generalized over random variables that are defined on c or q in general, with parameters rk.
But we're going to focus in a very specific case when y is a real valued response variable, which is the one you're used to when you're doing linear regression.
And the parameter theta also lives in r. And so we're going to talk about the canonical case.
So that's the canonical exponential family, where you have a density, theta of x, which is of the form, exponential plus.
And then, we have y, which interacts with theta only by taking a product.
Then, there's a term that depends only on theta, some dispersion parameter phi.
And then, we have some normalization factor.
Let's call it c of y phi.
So it really should not matter too much, so it's c of y phi, and that's really just the normal position factor.
And here, we're going to assume that phi is known.
I have no idea what I write.
I don't know if you guys can read.
I don't know what chalk has been used today, but I just can't see it.
That's not my fault.
All right, so we're going to assume that phi is known.
And so we saw that several distributions that we know well, including the Gaussian for example, belong to this family.
And there's other ones, such as Poisson-- Poisson and Bernoulli.
So if the PMF has this form, if you have a discrete random variable, this is also valid.
And the reason why we introduced this family is because there are going to be some properties that we know that this thing here, this function, b of theta, is essentially what completely characterizes your distribution.
So if phi is fixed, we know that the interaction is the form.
And this really just comes from the fact that we want the function to integrate to one.
So this b here in the canonical form encodes everything we want to know.
If I tell you what b of theta is-- and of course, I tell you what phi is, but let's say for a second that phi is equal to one.
If I tell you this b of theta, you know exactly what distribution I'm talking about.
So it should encode everything that's specific to this distribution, such as mean, variance, all the moments that you would want.
And we'll see how we can compute from this thing the mean and the variance, for example.
So today, we're going to talk about likelihood, and we're going to start with the likelihood function or the log likelihood for one observation.
From this, we're going to do some computations, and then, we'll move on to the actual log likelihood based on n independent observations.
And here, as we will see, the observations are not going to be identically distributed, because we're going to want each of them, conditionally on x to be a different function of x, where theta is just a different function of x for each of the observation.
So remember, the log likelihood-- and this is for one observation-- is just the log of the density, right?
And we have this identity that I mentioned at the end of the class on Tuesday.
And this identity is just that the expectation of the derivative of this guy with respect to theta is equal to 0.
So let's see why.
So if I take the derivative with respect to theta, of log f, theta of x, what I get is the derivative with respect to theta of f, theta of x, divided by f theta of x.
Now, if I take the expectation of this guy, with respect to this theta as well, what I get is that this thing-- what is the expectation?
Well, it's just the integral against f theta.
Or if I'm in a discrete case, I just have the sum against f theta, if it's a pmf.
Just the definition, the expectation of x, is either the integral-- well, let's say of h of x-- is integral of h of x. F theta of x-- if this is discrete or is just the sum of h of x, f theta of x.
If x is discrete-- so if it's continuous, you put this soft sum.
This guy is the same thing, right?
So I'm just going to illustrate the case when it's continuous.
So this is what?
Well, this is the integral of partial derivative with respect to theta, of f theta of x, divided by f theta of x, all time f theta of x-- dx.
And now, this f theta is canceled, so I'm actually left with the integral of the derivative, which I'm going to write as the derivative of the integral.
But f theta being density for any value of theta that I can take, this is the function.
As a function of theta, this function is constantly equal to 1.
For any theta that I take it, it takes value of 1.
So this is constantly equal to 1.
I put three bars to see that for any value of theta, this is 1, which actually tells me that the derivative is equal to 0.
OK, yes?
What is the first [INAUDIBLE] that you wrote on the board?
That's just the definition of the derivative of the log of a function?
Log of f prime is f prime over f. That's a log, yeah.
Just by elimination
[INAUDIBLE]
I'm sorry
When you write a squiggle that starts with an l, I assume it's lambda
And you do good, because that's probably how my mind processes.
And so I'm like, yeah, l. Here is enough information.
OK, everybody is good with this?
So that was convenient.
So it just said that the expectation of the derivative of the log likelihood is equal to 0.
That's going to be our first identity.
Let's move onto the second identity, using exactly the same trick, which is let's hope that at some point, we have the integral of this function that's constantly equal to 1 as a function of theta, and then use the fact that its derivative is equal to 0.
So if I start taking the second derivative of the log of f theta, so what is this?
Well, it's the derivative of this guy here, so I'm going to go straight to it.
So it's second derivative, f theta of x, times f theta of x, minus first derivative of f theta of x, times first derivative of f theta of x.
Here is some super important stuff-- no, I'm kidding.
So you can still see that guy over there?
So it's just the square.
And then, I divide by f theta of x squared.
So here I have the second derivative, times f itself.
And here, I have the product of the first derivative with itself.
So that's the square.
So now, I'm going to integrate this guy.
So if I take the expectation of this thing here, what I get is the integral.
So here, the only thing that's going to happen when I'm going to take my integral is that one of those squares is going to cancel against f theta, right?
So I'm going to get the second derivative minus the second derivative squared.
And then, I'm divided by f theta.
And I know that this thing is equal to 0.
Now, one of these guys here-- sorry, why do I have-- so I have this guy here.
So this guy here is going to cancel.
So this is what this is equal to-- the integral of the partial, so the second derivative of f theta of x, because those two guys cancel-- minus the integral of the second derivative.
And this is telling me what?
Yeah, I'm losing one, because I have some weird sequences.
Thank you.
OK, this is still positive.
I want to say that this thing is actually equal to 0.
But then, it gives me some weird things, which are that I have an integral of a positive function, which is equal to 0.
Yeah, that's what I'm thinking of doing.
But I'm going to get 0 for this entire integral, which means that I have the integral of a positive function, which is equal to 0, which means that this function is equal to 0, which sounds a little bad-- basically, tells me that this function, f theta, is linear.
So I went a little too far, I believe, because I only want to prove that the expectation of the second derivative-- Yes, so I want to pull this out.
So let's see, if I keep rolling with this, I'm going to get-- well, no because the fact that it's divided by f theta, means that, indeed, the second derivative is equal to 0.
So it cannot do this here
[INAUDIBLE]
OK, but let's write it like this.
You're right, so this is what?
This is the expectation of the partial with respect to theta of f theta of x, divided by f theta of x squared.
And this is exactly the derivative of the log, right?
So indeed, this thing is equal to the expectation with respect to theta of the partial of l-- of log f theta, divided by partial theta.
All right, so this is one of the guys that I want squared.
This is one of the guys that I want.
And this is actually equal, so this will be equal to the expectation--
[INAUDIBLE]
Oh, right, so this term should be equal to 0.
This was not 0.
You're absolutely right.
So at some point, I got confused, because I thought putting this equal to 0 would mean that this is 0.
But this thing is not equal to 0.
So this thing, you're right.
I take the same trick as before, and this is actually equal to 0, which means that now I have what's on the left-hand side, which is equal to what's on the right-hand side.
And if I recap, I get that e theta of the second derivative of the log of f theta is equal to minus-- because I had a minus sign here-- to the expectation with respect to theta, of log of f theta, divided by theta squared.
Thank you for being on watch when I'm falling apart.
All right, so this is exactly what you have here, except that both terms have been put on the same side.
All right, so those things are going to be useful to us, so maybe, we should write them somewhere here.
And then, we have that the expectation of the second derivative of the log is equal to minus the expectation of the square of the first derivative.
And this is, indeed, my Fisher information.
This is just telling me what is the second derivative of my log likelihood at theta, right?
So everything is with respect to theta when I take these expectations.
And so it tells me that the expectation of the second derivative-- at least first of all, what it's telling me is that it's concave, because the second derivative of this thing, which is the second derivative of kl divergence, is actually minus something which is must be non-negative.
And so it's telling me that it's concave here at this [INAUDIBLE].
And in particular, it's also telling me that it has to be strictly positive, unless the derivative of f is equal to 0.
Unless f is constant, then it's not going to change.
All right, do you have a question?
So now, let's use this.
So what does my log likelihood look like when I actually compute it for this canonical exponential family.
We have this exponential function, so taking the log should make my life much easier, and indeed, it does.
So if I look at the canonical, what I have is that the log of f theta of x, it's equal simply to y theta minus b of theta, divided by phi, plus this function that does not depend on theta.
So let's see what this tells me.
Let's just plug-in those equalities in there.
I can take the derivative of the right-hand side and just say that in expectation, it's equal to 0.
So if I start looking at the derivative, this is equal to what?
Well, here I'm going to pick up only y.
Sorry, this is a function of y. I was talking about likelihood, so I actually need to put the random variable here.
So I get y minus the derivative of b of theta.
Since it's only a function of theta, I'm just going to write b prime, is at OK-- rather than having the partial with respect to theta.
Then, this is a constant.
This does not depend on theta, so it goes away.
So if I start taking the expectation of this guy, I get the expectation of this guy, which is the expectation of y, minus-- well, this does not depend on y, so it's just itself-- b prime of theta.
And the whole thing is divided by phi.
But from my first equality over there, I know that this thing is actually equal to 0.
We just proved that.
So in particular, it means that since phi is non-zero, it means that this guy must be equal to this guy-- or phi is not infinity.
And so that implies that the expectation with respect to theta of y is equal to b prime of theta.
I'm sorry, you're not registered in this class.
I'm going to have to ask you to leave.
I'm not kidding
[INAUDIBLE]
You are?
I've never seen you here.
I saw you for the first lecture.
OK. All right, so e theta of y is equal to b prime of theta.
Everybody agrees with that?
So this is actually nice, because if I give you an exponential family, the only thing I really need to tell you is what b theta is.
And if I give you b of theta, then computing a derivative is actually much easier than having to integrate y against the density itself.
You could really have fun and try to compute this, which you would be able to do, right?
And then, there's the plus c of y phi, blah, blah, blah-- dy.
And that's the way you would actually compute this thing.
Sorry, this guy is here.
That would be painful.
I don't know what this normalization looks like, so it would have to also explicit that, so I can actually compute this thing.
And you know, just the same way, if you want to compute the expectation of a Gaussian-- well, the expectation of a Gaussian is not the most difficult one.
But even if you compute the expectation of a Poisson, you start to have to work in a little bit.
There's a few things that you have to work through.
Here, I'm just telling you, all you have to know is what b of theta is, and then, you can just take the derivative.
Let's see what the second equality is going to give us.
OK, so what is the second equality?
It's telling me that if I look at the second derivative, and then I take its expectation, I'm going to have something which is equal to negative this guy squared.
Sorry, that was the log, right?
We've already computed this first derivative of the likelihood.
It's just the expectation of the square of this thing here.
So expectation of the derivative, with respect to theta of log, f theta of x, divided by partial theta squared.
This is equal to the expectation of the square of y, minus b theta, divided by phi squared-- b prime, theta squared.
OK, sorry, I'm actually going to move on with the-- so if I start computing, what is this thing?
Well, we just agreed that this was what?
The expectation of theta, right?
So that's just the expectation of y.
We just computed it here
[INAUDIBLE]
Yeah, that's b prime.
There's a derivative here.
So now, this is what?
This is simply-- anyone?
I'm sorry?
Variance of y, but you're scaling by phi squared.
OK, so this is negative of the right-hand side of our inequality.
And now, I just have to take one more derivative to this guy.
So now, if I look at the left-hand side now, I have that the second derivative of log, of f theta of y, divided by partial of theta squared.
So this thing is equal to-- well, no, I'm not left with much.
The white part is going to go away, and I'm left only with the second derivative of theta, minus the second derivative theta, divided by phi.
So if I take expectation-- well, it just doesn't change.
This is deterministic.
So now, what I've established is that this guy is equal to negative this guy.
So those two things, the signs are going to go away.
And so this implies that the variance of y is equal to b prime prime theta.
And then, I have a phi square in denominator that cancels only one of the phi squares, so it's time phi.
So now, I have that my second derivative-- since I know phi is completely determining the variance.
So basically, that's why b is called the cumulant generating function.
It's not generating moments, but cumulants.
But cumulants, in this case, correspond, basically, to the moments, at least for the first two.
If I start going farther, I'm going to have more combinations of the expectation of y3, y2, and y itself.
But as we know, those are the ones that are usually the most useful, at least if we're interested in asymptotic performance.
The central limit theorem tells us that all that matters are the first two moments, and then, the rest is just going to go and say well, it doesn't matter.
It's all going to [INAUDIBLE] anyway.
So let's go to a Poisson for example.
So if I had a Poisson distribution-- so this is a discrete distribution.
And what I know is that f-- let me call mu the parameter of y.
So it's mu to the y, divided by y factorial, exponential minus mu.
OK so mu is usually called lambda, and y is usually called x, that's why it takes me to a little bit of time.
But it usually it's lambda to the x over factorial x, exponential minus lambda.
Since this is just the series expansion of the exponential when I sum those things from 0 to infinity, this thing sums to 1.
But then, if I wanted to start understanding what the expectation of this thing is-- so if I want to understand the expectation with respect to mu of y, then, I would have to compute the sum from k equals 0 to infinity of k, times mu to the k, over factorial of k, exponential minus mu-- which means that I would, essentially, have to take the derivative of my series in the end.
So I can do this.
This is a standard exercise.
You've probably done it when you took probability.
But let's see if we can actually just read it off from the first derivative of b.
So to do that, we need to write this in the form of an exponential, where there is one parameter that captures mu, that interacts with y, just doing this parameter times y, and then something that depends only on y, and then something that depends only on mu.
That's the important one.
That's going to be our B.
And then, there's going to be something that depends only on y.
So let's write this and check that this f mu, indeed, belongs to this canonical exponential family.
So I definitely have an exponential that comes from this guy.
So I have minus mu.
And then, this thing is going to give me what?
It's going to give me plus y log mu.
And then, I'm going to have minus log of y factorial.
So clearly, I have a term that depends only on mu, terms that depend only on y, and I have a product of y and something that depends on mu.
If I want to be canonical, I must have this to be exactly the parameter theta itself.
So I'm going to call this guy theta.
So theta is log mu, which means that mu is equal to e to the theta.
And so wherever I see mu, I'm going to replace it by [INAUDIBLE] the theta, because my new parameter now, is theta.
So this is what?
This is equal to exponential y times theta.
And then, I'm going to have minus e of theta.
And then, who cares, something that depends only on mu.
So this is my c of y, and phi is equal to 1 in this case.
So that's all I care about.
So let's use it.
So this is my canonical exponential family.
Y interacts with theta exactly like this.
And then, I have this function.
So this function here must be b of theta.
So from this function, exponential theta, I'm supposed to be able to read what the mean is
[INAUDIBLE]
Because since in this course I always know what the dispersion is, I can actually always absorb it into theta from one.
But here, it's really of the form y times something divided by 1, right?
If it was like log of mu divided by phi, that would be the question of whether I want to call phi my dispersion, or if I want to just have it in there.
This makes no difference in practice.
But the real thing is it's never going to happen that this thing, this version, is going to be an exact number.
If it's an actual numerical number, this just means that this number should be absorbed in the definition of theta.
But if it's something that is called sigma, say, and I will assume that sigma is known, then it's probably preferable to keep it in the dispersion, so you can see that there's this parameter here that you can, essentially, play with.
It doesn't make any difference when you know phi.
So now, if I look at the expectation of some y-- so now, I'm going to have y, which follows my Poisson mu.
I'm going to look at the expectation, and I know that the expectation is b prime of theta.
Agreed?
That's what I just erased, I think.
Agreed with this, the derivative?
So what is this?
Well, it's the derivative of e to the theta, which is e to the theta, which is mu.
So my Poisson is parametrized by its mean.
I can also compute the variance, which is equal to minus the second derivative of-- no, it's equal to the second derivative of b. Dispersion is equal to 1.
Again, if I took phi elsewhere, I would see it here as well.
So if I just absorbed phi here, I would see it divided here, so it would not make any difference.
And what is the second derivative of the exponential?
Still the exponential-- so it's still equal to mu.
So that certainly makes our life easier.
Just one quick from remark-- here's the function.
I am giving you problem-- can the b function-- can it ever be equal to log of theta?
Who says yes?
Who says no?
Why?
[INAUDIBLE]
Yeah, so what I've learned from this-- it's sort of completely analytic, right?
So we just took derivatives, and this thing just happened.
This thing actually allowed us to relate the second derivative of b to the variance.
And one thing that we know about a variance is that this is non-negative.
And in particular, it's always positive.
If they give you a canonical exponential family that has zero variance, trust me, you will see it.
That means that this thing is not going to look like something that's finite, and it's going to have a point mass.
It's going to take value infinity at one point.
So this will, basically, never happen.
This thing is, actually, strictly positive, which means that this thing is always strictly concave.
It means that the second derivative of this function, b, has to be strictly positive, and so that the function is convex.
So this is concave, so this is definitely not working.
I need to have something that looks like this when I talk about my b.
So f theta squared-- we'll see a bunch of exponential theta.
And there's a bunch of them.
But if you started writing something, and you find b-- try to think of the plot of b in your mind, and you find that b looks like it's going to become concave, you've made a sign mistake somewhere.
All right, so we've done a pretty big parenthesis to try to characterize what the distribution of y was going to be.
We wanted to extend from, say, Gaussian to something else.
But when we're doing regression, which means generalized linear models, we are not interested in the distribution of y but really the conditional distribution of y given x.
So I need now to couple those back together.
So what I know is that this same mu, in this case, which is the expectation-- what I want to say is that the conditional expectation of y given x-- this is some mu of x.
When we did linear models, we said well, this thing was some x transpose beta for linear models.
And the whole premise of this chapter is to say well, this might make no sense, because x transpose beta can take the entire range of real values.
Whereas, this mu can take only a partial range.
So even if you actually focus on the Poisson, for example, we know that the expectation of a Poisson has to be a non-negative number-- actually, a positive number as soon as you have a little bit of variance.
It's mu itself-- mu is a positive number.
And so it's not going to make any sense to assume that mu of x is equal to x transpose beta, because you might find some x's for which this value ends up being negative.
And so we're going to need, what we call, the link function that relates, that transforms mu, maps onto the real line, so that you can now express it of the form x transpose beta.
So we're going to take not this, but we're going to assume that g of mu of x is not equal to x transpose beta, and that's the generalized linear models.
So as I said, it's weird to transform x transpose beta-- a mu to make it take the real line.
At least to me, it feels a bit more natural to take x transpose beta and make it fit to the particular distribution that I want.
And so I'm going to want to talk about g and g inverse at the same time.
So I'm going to actually take always g. So g is my link function, and I'm going to want g to be continuous differentiable.
OK, let's say that it has a derivative, and its derivative is continuous.
And I'm going to want g to be strictly increasing.
And that actually implies that g inverse exists.
Actually, that's not true.
What I'm also going to want is that g of mu spans-- how do I do this?
So I want the g, as I arrange for all possible values of mu, whether they're all positive values, or whether they're values that are limited between the intervals 0, 1, I want those to span the entire real line, so that when I want to talk about g inverses define over the entire real line, I know where I started.
So this implies that gene inverse exists.
What else does it imply about g inverse?
So for a function to be invertible, I only need for it to be strictly monotone.
I don't need it to be strictly increasing.
So in particular, the fact that I picked increasing implies that this guy is actually increasing
[INAUDIBLE]
That's the image.
So this is my link function, and this slide is just telling me I want my function to be invertible, so I can talk about g inverse.
I'm going to switch between the two.
So what link functions am I going to get?
So for linear models, we just said there's no link function, which is the same as saying that the link function is identity, which certainly satisfies all these conditions.
It's invertible.
It has all these nice properties, but might as well not talk about it.
For Poisson data, when we assume that the conditional distribution of y given x is Poisson, the mu, as I just said, is required to be positive.
So I need a g that goes from the interval 0 infinity to the entire real line.
I need a function that starts from one end and just takes-- not only the positive values are split between positive and negative values.
And here, for example, I could take the log link.
So the log is defined on this entire interval.
And as I range from 0 to plus infinity, the log is ranging from negative infinity to plus infinity.
You can probably think of other functions that do that, like 2 times log.
That's another one.
But there's many others you can think of.
But let's say the log is one of them that you might want to think about.
It is unnatural in the sense that it's one of the first function we can think of.
We will see, also, that it has another canonical property that makes it a natural choice.
The other one is the other example, where we had an even stronger condition on what mu could be.
Mu could only be a number between 0 and 1, that was the probability of success of a coin flip-- probability of success of a Bernoulli random variable.
And now, I need g to map 0, 1 to the entire real line.
And so here are a bunch of things that you can come up with, because now you start to have maybe-- I will soon claim that this one, log of mu, divided by 1 minus mu is the most natural one.
But maybe, if you had never thought of this, that might not be the first function you would come up with, right?
You mentioned trigonometric functions, for example, so maybe, you can come up with something that comes from hyperbolic trigonometry or something.
So what does this function do?
Well, we'll see a picture, but this function does map the interval 0, 1 to the entire real line.
We also discuss the fact that if we think reciprocally-- what I want if I want to think about g inverse, I want a function that maps the entire real line into the unit interval.
And as we said, if I'm not a very creative statistician or probabilist, I can just pick my favorite continuous, strictly increasing cumulative distribution function, which as we know, will arise as soon as I have a density that has support on the entire real line.
If I have support everywhere, then it means that my-- it is strictly positive everywhere, then, it means that my community distribution function has to be strictly increasing.
And of course, it has to go from 0 to 1, because that's just the nature of those things.
And so for example, I can take the Gaussian, that's one such function.
But I could also take the double exponential that looks like an exponential on one end, and then an exponential on the other end.
And basically, if you take capital phi, which is the standard Gaussian cumulative distribution function, it does work for you, and you can take its inverse.
And in this case, we don't talk about, so this guy is called logit or logit.
And this guy is called probit.
And you see it, usually, every time you have a package on generalized linear models.
You are trying to implement.
You have this choice.
And for what's called logistic regression-- so it's funny that it's called logistic regression, but you can actually use the probit link, which in this case, is called probit regression.
But those things are essentially equivalent, and it's really a matter of taste.
Maybe of communities-- some communities might prefer one over the other.
We'll see that again, as I claimed before, the logistic, the logit one has a slightly more compelling argument for its reason to exist.
I guess this one, the compelling argument is that it involved the standard Gaussian, which of course, is something that should show up everywhere.
And then, you can think about crazy stuff.
Even crazy gets name-- complimentary log, log, which is the log of minus, log 1 minus.
Why not?
So I guess you can iterate that thing.
You can just put a log 1 minus in front of this thing, and it's still going to go.
So that's not true.
I have to put a minus and take-- no, that's not true.
So you can think of whatever you want.
So I claimed that the logit link is the natural choice, so here's a picture.
I should have actually plotted the other one, so we can actually compare it.
To be fair, I don't even remember how it would actually fit into those two functions.
So the blue one, which is this one, for those of you don't see the difference between blue and red-- sorry about that.
So this the blue one is the logistic one.
So this guy is the function that does e to the x, over 1 plus e to the x.
As you can see, this is a function that's supposed to map the entire real line into the intervals, 0, 1.
So that's supposed to be the inverse of your function, and I claimed that this is the inverse of the logistic of the logit function.
And the blue one, well, this is the Gaussian CDF, so you know it's clearly the inverse of the inverse of the Gaussian CDF.
And that's the red one.
That's the one that goes here.
I would guess that the complimentary log, log is something that's probably going above here, and for which the slope is, actually, even a little flatter as you cross 0.
So of course, this is not our link functions.
These are the inverse of our link function.
So what do they look like when actually, basically, flip my thing like this?
So this is what I see.
And so I can see that in blue, this is my logistic link.
So it crosses 0 with a slightly faster rate.
Remember, if we could use the identity, that would be very nice to us.
We would just want to take the identity.
The problem is that if I start having the identity that goes here, it's going to start being a problem.
And this is the probit link, the phi verse that you see here.
It's a little flatter.
You can compute the derivative at zero of those guys.
What is the derivative of the-- So I'm taking the derivative of log of x over 1 minus x.
So it's 1 over x, minus 1 over 1 minus x.
So if I look at 0.5-- sorry, this is the interval 0, 1.
So I'm interested in the slope at 0.5.
Yes, it's plus, thank you.
So at 0.5, what I get is 2 plus 2.
Yeah, so that's the slope that we get, and if you compute for the derivative-- what is the derivative of a phi inverse?
Well, it's a little phi of x, divided by little phi of capital phi, inverse of x.
So little phi at 1/2-- I don't know.
Yeah, I guess I can probably compute the derivative of the capital phi at 0, which is going to be just that.
1 over square root of 2 pi, and then just say well, the slope has to be 1 over that.
Square root 2 pi.
So that's just a comparison, but again, so far, we do not have any reason to prefer one to the other.
And so now, I'm going to start giving you some reasons to prefer one to the other.
And one of those two-- and actually for each canonical family, there is something which is called the canonical link.
And when you don't have any other reason to choose anything else, why not choose the canonical one?
And the canonical link is the one that says OK, what I want is g to map mu onto the real line.
But mu is not the parameter of my canonical family.
Here for example, mu is e of theta, but the canonical parameter is theta.
But the parameter of a canonical exponential family is something that lives in the entire real line.
It was defined for all thetas.
And so in particular, I can just take theta to be the one that's x transpose beta.
And so in particular, I'm just going to try to find the link that just says OK, when I take g of mu, I'm going to map, so that's what's going to be.
So I know that g of mu is going to be equal to x beta.
And now, what I'm going to say is OK, let's just take the g that makes this guy equal to theta, so that this is theta that actually model, like x transpose beta.
Feels pretty canonical, right?
What else?
What other central, easy choice would you take?
This was pretty easy.
There is a natural parameter for this canonical family, and it takes value on the entire real line.
I have a function that maps mu onto the entire real line, so let's just map it to the actual parameter.
So now, OK, why do I have this?
Well, we've already figured that out.
The canonical link function is strictly increasing.
Sorry, so I said that now I want this guy-- so I want g of mu to be equal to theta, which is equivalent to saying that I want mu to be equal to g inverse of theta.
But we know that mu is what-- b prime of theta.
So that means that b prime is the same function as g inverse.
And I claimed that this is actually giving me, indeed, a function that has the properties that I want, because before I said, just pick any function that has these properties.
And now, I'm giving you a very hard rule to pick this, though you need still to check that it satisfies those conditions and particular, that it's increasing and invertible.
And so for this to be increasing and invertible, strictly increasing and invertible, really what I need is that the inverse is strictly increasing and invertible, which is the case here, because b prime as we said-- well, b prime is the derivative of a strictly convex function.
A strictly convex function has a second derivative that's strictly positive.
We just figured that out using the fact that the variance was strictly positive.
And if phi is strictly positive, then this thing has to be strictly positive.
So if b prime, prime is strictly positive-- this is the derivative of a function called b prime.
If your derivative is strictly positive, you are strictly increasing.
And so we know that b prime is, indeed, strictly increasing.
And what I need also to check-- well, I guess this is already checked on its own, because b prime is actually mapping all of our into the possible values.
When theta ranges on the entire real line, then b prime ranges in the entire interval of the mean values that it can take.
And so now, I have this thing that's completely defined.
B prime inverse is a valid link.
And it's called a canonical link.
OK, so again, if I give you an exponential family, which is another way of saying I'll give you a convex function, b, which gives you some exponential family, then if you just take b prime inverse, this gives you the associated canonical link for this canonical exponential family.
So clearly there's an advantage of doing this, which is I don't have to actually think about which one to pick if I don't want to think about it.
But there's other advantages that come to it, and we'll see that in the representations.
There's, basically, going to be some light cancellations that show up.
So before we go there, let's just compute the canonical link for the Bernoulli distribution.
So remember, the Bernoulli distribution has a PMF, which is part of the canonical exponential family.
So the PMF of the Bernoulli is f theta of x.
Let me just write it like this.
So it's p to the y, let's say-- one minus p to the 1 minus y, which I will write as exponential y log p, plus 1 minus y, log 1 minus p. OK, we've done that last time.
Now, I'm going to group my terms in y to see how y interacts with this parameter p. And what I'm getting is y, which is times log p divided by 1 minus p. And then, the only term that remains is log, 1 minus p. Now, I want this to be a canonical exponential family, which means that I just need to call this guy, so it is part of the exponential family.
You can read that.
If I want it to be canonical, this guy must be theta itself.
So I have that theta is equal to log p, 1 minus p. If I invert this thing, it tells me that p is e to the theta, divided by 1, plus e to the theta.
It's just inverting this function.
In particular, it means that log, 1 minus p, is equal to log, 1 minus this thing.
So the exponential thetas go away.
So in the numerator, this is what I get.
That's the log 1 minus this guy, which is equal to minus log 1, plus e to the theta.
So I'm going a bit too fast, but these are very elementary manipulations-- maybe, it requires one more line to convince yourself.
But just do it in the comfort of your room.
And then, what you have is the exponential of y times theta, and then, I have minus log, 1 plus e theta.
So this is the representation of the p and f of a Bernoulli distribution, as part of a member of the canonical exponential family.
And it tells me that b of theta is equal to log 1, plus e of theta.
That's what I have there.
From there, I can compute the expectation, which hopefully, I'm going to get p as the mean and p times 1, minus p as the variance.
Otherwise, that would be weird.
So let's just do this.
B prime of theta should give me the mean.
And indeed, b prime of theta is e to the theta, divided by 1, plus e to the theta, which is exactly this p that I had there.
OK just for fun-- well, I don't know.
Maybe, that's not part of it.
Yeah, let's not compute the second derivative.
That's probably going to be on your homework at some point-- if not, on the final.
So b prime now-- oh, I erased it, of course.
G, the canonical link is b prime inverse.
And I claim that this is going to give me the logit function, log of mu, over 1 minus mu.
So let's check that.
So b prime is this thing, so now, I want to find the inverse.
Well, I should really call my inverse a function of p. And I've done it before-- all I have to do is to solve this equation, which I've actually just done it, that's where I'm actually coming from.
So it's actually telling me that the solution of this thing is equal to log of p over 1 minus p. We just solve this thing both ways.
And this is, indeed, logit of p by definition of logit.
So b prime inverse, this function that seemed to come out of nowhere, is really just the inverse of b prime, which we know is the canonical link.
And canonical is some sort of ad hoc choices that we've made by saying let's just take the link, such that d of mu is giving me the actual canonical parameter of theta.
Yeah?
[INAUDIBLE]
You're right.
Now, of course, I'm going through all this trouble, but you could see it immediately.
I know this is going to be theta.
We also have prior knowledge, hopefully, that the expectation of a Bernoulli is p itself.
So right at this step, when I say that I'm going to take theta to be this guy, already knew that the canonical link was the logit-- because I just said oh, here's theta.
And it's just this function of mu [INAUDIBLE].. OK, so you can do that for a bunch of examples, and this is what they're going to give you.
So the Gaussian case, b of theta-- we've actually computed it, actually, once.
This is theta squared over 2.
So the derivative of this thing is really just theta, which means that g or g inverse is actually equal to the identity.
And again, sanity check-- when I'm in the Gaussian case, there's nothing general about general linear models if you don't have a link.
The Poisson case-- you can actually check.
Did we do this, actually?
Yes we did.
So that's when we had this e of theta.
And so b is e of theta, which means that the natural link is the inverse, which is log, which is the inverse of exponential.
And so that's logarithm link, which as I said, I used the word natural.
You can also use the word canonical if you want to describe this function as being the right function to map the positive real line to the entire real line.
The Bernoulli-- we just did it.
So b-- the cumulative enduring function is log of 1, plus e of theta, which is log of mu over 1 minus mu.
And gamma function-- where you have the thing you're going to see is minus log of minus [INAUDIBLE].. You see the reciprocal link is the link that actually shows up, so minus 1 over mu.
That maps.
So are there any questions about the canonical links, canonical families?
I use the word canonical a lot.
But is everything fitting together right now?
So we have this function.
We have canonical exponential family, by assumption.
It has a function, b, which contains every information we want.
At the beginning of the lecture, we established that it has information about the mean in the first derivative, about the variance in the second derivative.
And it's also giving us a canonical link.
So just cherish this b once you've found it, because it's everything you need.
Yeah?
[INAUDIBLE]
I don't know, a political preference?
I don't know, honestly.
If I were a serious practitioner, I probably would have a better answer for you.
At this point, I just don't.
I think it's a matter of practice and actual preferences.
You can also try both.
We didn't mention it, but there's this idea of cross-validation-- well, we mentioned it without going too much into detail.
But you could try both and see which one performs best on a yet unseen data set.
In terms of prediction, just say I prefer this one of the two, because this actually comes as part of your modeling assumption, right?
Not only did you decide to model the image of mu through the link function as a linear model, but really what you're saying-- your model is saying well, you have two pieces of [INAUDIBLE],, the distribution of y.
But you also have the fact that mu is modeled as g inverse of x transpose beta.
And for different g's, this is just different modeling assumptions, right?
So why should this be linear-- I don't know.
My authority as a person who has not examined the [INAUDIBLE] data sets for both things would be that the changes are fairly minor.
OK, so this was all for one observation.
We just, basically, did probability.
We described some density, some properties of the densities, how to compute expectations.
That was really just probability.
There was no data involved at any point.
We did a bit of modeling, but it was all for one observation.
What we're going to try to do now is given the reverse engineering to probability that is statistics, given data, what can I infer about my model?
Now remember, there's three parameters that are floating around in this model.
There is one that was theta.
There is one that was mu, and there is one that is beta.
OK, so those are the three parameters that are floating around.
What we said is that the expectation of y, given x, is mu of x.
So if I estimate mu, I know the conditional expectation of y, given x, which definitely gives me theta of x.
How do I go from mu of x to theta of x?
The inverse of what-- of the arrow?
Yeah, sure, but how do I go from this guy to this guy?
So theta as a function of mu is?
[INAUDIBLE]
Yeah, so we just computed that mu was b prime of theta.
So it means that theta is just b prime inverse of mu.
So those two things are the same as far as we're concerned, because we know that b prime is strictly increasing it's invertible.
So it's just a matter of re-parametrization, and we just can switch from one to the other whenever we want.
But why we go through mu, because so far for the entire semester I told you there was one parameter that's theta.
It does not have to be the mean, and that's the parameter that we care about.
It's the one on which we want to do interference.
That's the one for which we're going to compute the Fisher information.
This was the parameter that was our object of worship.
And now, I'm saying oh, I'm going to have mu that's coming around.
And why we have mu, because this is the mu that we use to go to beta.
So I can go freely from theta to mu using b prime or b prime inverse.
And now, I can go from mu to beta, because I have that g of mu of x is beta transpose x.
So in the end, now, this is going to be my object of worship.
This is going to be the parameter that matters.
Because once I set beta, I set everything else through this chain.
So the question is if I start stacking up this pile of parameters-- so I start with my beta, which in turns give me a mu, which in turn, gives me a theta-- can I just have a long, streamlined-- what is the outcome when I actually start writing my likelihood, not as a function of theta, not as a function of mu, but as a function of beta, which is the one at the end of the chain?
And hopefully, things are going to happen nicely, and they might no.
Yeah?
[INAUDIBLE]
Is G-- that's my link.
G of mu of x-- now, its mu is a function of x, because its conditional on x.
So this is really theta of x, mu of x, but b is not a function of x, because it's just something to tells me what the function of x is
[INAUDIBLE]
Mu is the conditional expectation of y, given x.
It has, actually, a fancy name in the statistics literature.
It's called-- anybody knows the name of the function, mu of x, which is a conditional expectation of y, given x?
[INAUDIBLE]
That's the regression function.
That's actual definition.
If I tell you what is the definition of the regression function, that's just the conditional expectation of why, given x.
And I could look at any property of the conditional distribution of y given x. I could look at the conditional 95th percentile.
I can look at the conditional median.
I can look at the conditional [INAUDIBLE] range.
I can look at the conditional variance.
But I decide to look at the conditional expectation, which is called the regression function.
Yes?
[INAUDIBLE]
Oh, there's no transpose here.
Actually, only Victor-Emmanuel used this prime for transpose, and I found it confusing with the derivatives.
So primes here is only a derivative
[INAUDIBLE]
Oh, yeah, sorry, beta transpose x.
So you said what?
I said that g of mu of x is beta transpose x?
[INAUDIBLE]
Isn't that the same thing?
X is a vector here, right?
Yeah
So x transpose beta, and beta transpose x are of the same thing
[INAUDIBLE]
So beta looks like this.
X looks like this.
It's just a simple number.
Yeah, you're right.
I'm going to start to look at matrices.
I'm going to have to be slightly more careful when I do this.
OK so let's do the reverse engineering.
I'm giving you data.
From this data, hopefully, you should be able to get what the conditional-- if I give you an infinite amount of data, you would know exactly, of pairs xy, what the conditional distribution of y given x is.
And in particular, you would know what the conditional expectation of y given x is, which means that you would know mu, which means that you would know theta, which means that you would know beta.
Now, when I have a finite number of observations, I'm going to try to estimate mu of x.
But really, I'm going to go the other way around.
Because the fact that I assume, specifically, that mu of x is of the form g of mu of x is x transpose beta, then that means that I only have to estimate beta, which is a much simpler object than the entire regression function.
So that's what I'm going to go for.
I'm going to try to represent the likelihood, the log likelihood, of my data as a function, not of theta, not of mu, but of beta-- and then, maximize that guy.
So now, rather than thinking of just one observation, I'm going to have a bunch of observations.
So this might actually look a little confusing, but let's just make sure that we understand each other before we go any further.
So I'm going to have observations, x1, y1, all the way to xn, yn, just like in a natural regression problem, except that here my y's might be 0 one valued.
They might be positive valued.
They might be exponential.
They might be anything in the canonical exponential family.
OK so I have this thing, and now, what I have is that my observations are x1, y1, xn, yn.
And what I want is that I'm going to assume that the conditional expectation of yi, given-- the conditional distribution of yi, given xi, is something that has density.
Did I put an i on y-- yeah.
I'm not going to deal with the phi and the c now.
And why do I have theta i and not theta is because theta i is really a function of xi.
So it's really theta i of xi.
But what do I know about theta i of xi, it's actually equal to b-- I did this error twice-- b prime inverse of mu of xi.
And I'm going to assume that this is of the form beta transpose xi.
And this is why I have theta i-- is because this theta i is a function of xi, and I'm going to assume a very simple form for this thing.
Sorry, sorry, sorry, sorry-- I should not write it like this.
This is only when I have the canonical link.
So this is actually equal to b prime inverse of g, of xi transpose beta.
Sorry, g inverse-- those two things are actually canceling each other.
So as before, I'm going to stack everything into some-- well, actually, I'm not going to stack anything for the moment.
I'm just going to give you a peek at what's happening next week, rather than just manipulating the data.
So here is how we're going to proceed at this point.
Well now, I want to write my likelihood function, not as a function of theta, but as a function of beta, because that's the parameter I'm actually trying to maximize.
So if I have a link-- so this thing that matters here, I'm going to call h. By definition, this is going to be h of xi transpose beta.
Helena, you have a question?
Uh, no [INAUDIBLE]
So this is just all the things that we know.
Theta is just the, by definition of the fact that mu is b prime of theta, the mean is b prime of theta-- it means that theta is b prime inverse of mu.
And then, mu is modeled from the systematic component.
G of mu is xi transpose beta, so this is g inverse of xi transpose beta.
So I want to have b prime inverse of g inverse.
This function is a bit annoying to say, so I'm just going to call it h. And when I do the composition of two inverses, the inverse of the composition of those two things in the reverse order-- so h is really the inverse of g, composed with b prime, g of b prime inverse.
And now, if I have the canonical link, since I know that g is b prime inverse, this is really just the identity.
As you can imagine, this entire thing, which is actually quite complicated-- would just say oh, this thing, actually, does not show up when I have the canonical link.
I really just have that theta can be replaced by xi of beta.
So think about going back to this guy here.
Now, theta becomes only xi transpose beta.
That's going to be much more simple to optimize, because remember, when I'm going to log likelihood, this thing is going to go away.
I'm going to sum those guys.
And so what I'm going to have is something which is essentially linear in beta.
And then, I'm going to have this minus b, which is just minus the sum of convex functions of beta.
And so I'm going to have to bring in the tools of convex optimization.
Now, it's not just going to be take the gradient, set it to 0.
It's going to be more complicated to do that.
I'm going to have to do that in an iterative fashion.
And so that's what I'm telling you, when you look at your log likelihood for all those functions.
You sum, the exponential goes away because you had the log, and then, you have all these things here.
I kept the b. I kept the h. But if h is the identity, this is the linear function, the linear part, yi times xi transpose beta, minus b of my theta, which is now only xi transpose beta.
And that's the function I want to maximize in beta.
It's a convex function.
When I know what b is, I have an explicit formula for this, and I want to just bring in some optimization.
And that's what we're going to do, and we're going to see three different methods, which are really, basically, the same method.
It's just an adaptation or specialization of the so-called Newton-Raphson method, which is essentially telling you do iterative local quadratic approximations through your function-- so second order [INAUDIBLE] expansion, minimize this guy, and then do it again from where you were.
And we'll see that this can be, actually, implemented using what's called iteratively re-weighted least squares, which means that every step-- since it's just a quadratic, it's going to be just squares in there-- can actually be solved by using a weighted least squares version of the problem.
So I'm going to stop here for today.
So we'll continue and probably not finish this chapter, but finish next week.
And then, I think there's only one lecture.
Actually, for the last lecture, what do you guys want to do?
Do you want to have doughnuts and cider?
Do you want to just have some more outlooking lecture on what's happening post 1975 in statistics?
Do you want to have a review for the final exam-- pragmatic people
[INAUDIBLE] interesting, advanced topics
You want to do interesting, advanced-- for the last lecture?
Something that we haven't thought of yet
Yeah, that's, basically, what I'm asking, right-- interesting advanced topics, versus ask me any question you want.
Those questions can be about interesting, advanced topics, though.
Like, what are interesting, advanced topics?
I'm sorry?
Interesting with doughnuts-- is that OK?
Yeah, we can always do the doughnuts.
[LAUGHTER]
As long as there are doughnuts
All right, so we'll do that.
So you guys have a good weekend.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at oct.mit.edu
All right.
So let's continue talking about maximum likelihood estimation in the context of generalized linear models, all right?
So in those generalized linear models, what we spent most of the past lectures working on is the conditional distribution of Y given X.
And we're going to assume that this follows some distribution in the exponential family.
OK. And so what it means is that if we look at the density, say-- or the PMF, but let's talk about density to make things clearer-- we're going to assume that Y given X has distribution.
So X is now fixed, because we're conditioning on it.
And it has a density, which is of this form, c of Yi phi.
OK.
So this c, again, we don't really need to think about it.
This is something that's going to come up naturally as soon as you need normalization factor.
And so here what it means, if this is the distribution of Y given Xi, so that's the density of Yi given Xi is equal to little xi.
So if it's the conditional distribution of Yi given Xi, it should depend on xi somehow.
And it does not appear to depend on Xi.
And here, the model is going to be on theta i, which is just a function, theta i of Xi.
And we're going to take a very specific one.
It's going to be a function of a linear form of the Xi.
So really we're going to take something which is of the form theta I, which is really just-- as theta does not depend on Xi-- of Xi transposed from beta.
OK?
So all these parts here, this is really some modeling assumptions that we're making once we've agreed on what distribution we want.
OK.
So to do that, our goal, of course, is going to try to understand what this beta is.
There's one beta here.
What's important is that this beta does not depend on i.
So if they observe pairs Xi, Yi-- let's say I observe n of them, i equals 1 to n-- the hope is that as a accumulate more and more pairs of this form where there's always the same parameter that links Xi to Yi, that's this parameter beta, that I should have a better and better estimation of this beta.
Because it's always the same.
And that's essentially with couples all of our distribution.
If I did not assume this, then I could have a different distribution for each pair Xi given YI.
And I would not be able to do any statistics.
Nothing would average in the end.
But here I have the same beta, which means that I can hope to do statistics and average errors in them.
OK.
So I'm going to collect, so I'll come back to this.
But as usual in the linear regression model, we're going to collect all our observations Yi.
So I'm going to assume that they're real valued and that my Xi's takes value in Rp just like in the regression model.
And I'm going to collect all my Yi's into one big vector of Y in our n and all my X's into one big matrix in Rn times p just like for the linear regression model.
All right, so, again, what I'm interested in here is the conditional distribution of Yi given Xi.
OK.
I said this is this distribution.
When we're talking about regression, I defined last time what the definition of regression function was.
It's just one particular aspect of this conventional distribution.
It's the conditional expectation of Yi given Xi.
OK. And so this conditional expectation, I will denote it by-- so I talk about the conditional, I'm going to call it, say, mu i, which is the conditional expectation of Yi given Xi equals some little xi, say.
You can forget about this part if you find it confusing.
It really doesn't matter.
It's just that this means that this is a function of little xi.
But if I only had the expectation of Yi given big Xi, this would be just a function of big Xi.
So it really doesn't change anything.
It's just a matter of notation.
OK.
So just forget about this part.
But I'll just do it like that here.
OK.
So this is just the conditional expectation of Yi given Xi.
It just depends on Xi, so I think it depends on i, and so I will call it mu i.
But I know that since in a canonical exponential family, then I know that mu i is actually B prime of theta i. OK.
So there's a 1 to 1 link between the canonical parameter of my exponential family and the mean mu i, the conditional expectation.
And the modeling assumption we're going to make is not directly-- remember, that was the second aspect of the generalized linear model.
We're not going to assume that theta i itself directly depends on Xi.
We're going to assume that mu i has a particular dependence on Xi through the link function.
So, again, we're back to modeling.
So we have a link function g. And we assume that mu i depends on Xi as follows.
g of mu i-- and remember, all g does for us is really map the space in which mu i lives, which could be just the interval 0, 1 to the entire real line, all right?
And we're going to assume that this thing that lives in the real line is just Xi transpose beta.
I should maybe put a small one, Xi transpose beta.
OK?
So we're making, indeed, some modeling assumption.
But compared to in the linear regression model, we only assume that mu i was Xi transpose beta.
So if you want to make a parallel between generalized linear models and linear model is the only difference is that g is not the identity necessarily in this case.
And all the g does for us is to just make this thing compatible, that those two things on the left and the right of equality live in the same space.
So in a way, we're not making a much bigger leap of faith by assuming a linear model.
The linear link is already here.
We're just making things compatible, all right?
And so it's always the same link function.
So now if I want to go back to beta-- right, because I'm going to want to express my likelihood-- if I were to express my likelihood from this, it would just be a function of theta, right?
And so if I want to maximize my likelihood, I don't want to maximize it in theta.
I want to maximize it in beta.
So if I can write my density as a function of beta, then I will be able to write my likelihood as a function of beta, and then talk about my maximum likelihood estimator.
And so all they need to do is to just say, OK, how do I replace theta by-- I know that theta is a function of beta, right?
I wrote it here.
So the question is, what is this function?
And I actually have access to all of this.
So what I know is that theta-- right, so mu is b prime of theta, which means that theta i is b prime inverse of mu i. OK.
So that's what we've got from this derivative of the log likelihood equal to 0.
That give us this guy inverted.
And now I know that mu i is g inverse of Xi beta.
So this composition of b prime inverse and g inverse is actually just the composition of g with b prime.
Everybody's comfortable with this notation, the little circle?
Any question about this?
It just means that I first applied b prime.
Well, actually, it's D inverse.
But if I look at a function g composed with b prime, I first applied the g b prime of x, is just g of b prime of x. OK. And then I take the inverse of this function, which is first take g inverse, and then take b prime inverse.
OK.
So now I have everywhere I saw theta, now I see this function of beta.
So I could technically plug that in.
Of course, it's a little painful to have to write g circle beta prime all the time.
So I'm going to give this guy a name.
And so you're just going to define h, which is g b prime inverse so that theta i is simply h of Xi transpose beta.
OK.
I could give it a name, you know.
But let's just call that the h function.
And something which is nice about this h function is that if g is the canonical link-- what is the canonical link?
So what is it canonical to?
A canonical link, it's canonical to a particular distribution in the canonical exponential family, right?
A canonical exponential family is completely characterized by the function b.
Which means that if I want to talk about the canonical link, all I need to tell you is how it depends on b.
So what is g as a function of b?
[INAUDIBLE]
b inverse.
b prime inverse, right?
So this is g is equal to b prime inverse, which means that if g is composed with b prime that means that this is just the identity.
So h is the identity.
So h of Xi transpose beta is simply Xi transpose beta.
And it's true that the way we introduce the canonical link was just the function for which we model directly theta i as Xi transpose beta, which we can read off here, right?
So theta i is simply Xi transpose beta.
So now, for example, if I go back to my log-likelihood, so if I look log-likelihood, the log-likelihood is sum of the log of the densities.
So it's sum from i equal 1 to n of log of exponential Yi theta i minus b theta i divided by phi plus c of Yi phi.
So this term does not depend on theta.
So I have two things.
First of all, the log and the exponential are going to cancel each other.
And second, I actually know that theta is just a function of beta.
And it has this form.
Theta i is h of Xi transpose beta.
And that's my modeling assumption.
So this is actually equal to the sum from i equal 1 to n of Yi.
And then here I'm going to write h of Xi transpose beta minus b of h of Xi transpose beta divided by phi.
And then I have, again, this function c of Yi phi, which again won't matter.
Because when I'm going to try to maximize this thing, this is just playing the role of a constant that's shifting the entire function.
In particular, your max is going to be exactly what it was.
OK?
So this thing is really not going to matter for me.
I'm keeping track of it.
And actually, if you look here, it's gone, right?
It's gone, because it does not matter.
So let's just pretend it's not here, because it won't matter when I'm trying to maximize the likelihood.
OK?
While it's here up to constant term, it says.
That's the constant term.
All right, any question?
All I'm doing here is replacing my likelihood as a function of theta i's.
So if I had one theta i per observation, again, this would not help me very much.
But if I assume that they are all linked together by saying that theta i is of the form Xi transpose beta or h of Xi transpose beta if I'm not using the canonical link, then I can hope to make some estimation.
And so, again, if I have the canonical link, h is the identity.
So I'm left only with Yi times Xi transpose beta.
And then I have b of Xi transpose beta and not b composed with h, because h is the identity, which is fairly simple, right?
Why is it simple?
Well, let's actually focus on this guy for one second.
So let me write it down, so we know what we're talking about.
So we just showed that the log-likelihood when I use the canonical link-- so that h is equal to the identity, the log-likelihood actually takes the form ln.
And it depends on a bunch of stuff.
But let's just make it depend only on the parameter that we care about, which is beta, all right?
So this is of the form l of beta and that's equal to what?
It's the sum from i equal 1 to n of Yi Xi transpose beta minus-- let me put the phi here.
And then I'm going to have minus b of Xi transpose beta.
OK. And phi we know is some known positive term.
So again, optimizing a function plus some constant or optimizing of function time as a constant, that's not going to change much either.
So it won't really matter to think about whether this phi is here or not.
But let's just think about what this function looks like.
I'm trying to maximize a function.
I'm trying to maximize a log-likelihood.
If it looked like this, that would be a serious problem.
But we can do like a basic, you know, back of the envelope guess of what the variations of this function is.
This first term here is-- as a function of beta, what kind of function is it?
Linear
It's linear, right?
This is just Xi transpose beta.
If I multiply beta by 2, I get twice.
If I add something to beta, it just gets added, so it's a linear function of beta.
And so this thing is both convex and concave.
In the one-dimensional case-- so think about p as being one-dimensional-- so if beta is a one-dimensional thing, those are just the function that looks like this, right?
Those are linear functions.
They are both convex and concave.
So this is not going to matter when it comes to the convexity of my overall function, because I'm just adding something which is just a line.
And so if I started with convex, it's going to stay convex.
If I started with concave, it's going to stay concave.
And if I started with something which is both, it's going to stay both, meaning neither.
It cannot be both.
Yeah.
So if you're neither convex or concave, adding this linear-- so this will not really matter.
If I want to understand when my function looks like, I need to understand would b of Xi transpose beta does.
Begin, the Xi transpose beta-- no impact.
It's a linear function.
In terms of convexity, it's not going to play any role.
So I really need to understand what my function b looks like.
What do we know about b again?
So we know that b prime of theta is equal to mu, right?
Well, the mean of a random variable in a canonical exponential family can be a positive or negative number.
This really does not tell me anything.
That can be really anything.
However, if I look at the second the derivative of b, I know that this is what?
This is the variance of Y divided by phi.
That was my dispersion parameter.
The variance was equal to phi times b prime prime.
So we know that if theta is not degenerate, meaning that the density does not take value infinity at only one point, this thing is actually positive.
And clearly, when you have something that looks like this, unless you have some crazy stuff happening with phi being equal to 0 or anything that's not normal, then you will see that you're not degenerate.
So this think is strictly positive.
And we've said several times that if b prime prime is positive, then that means that's the derivative of b prime, meaning that b prime is increasing.
And b prime is increasing is just the same thing as saying that b is convex, All right?
So that implies that b is strictly convex.
And the strictly comes from the fact that this is a strict sign.
Well, I should not do that, because now it's no longer.
So it's just a strict sign, meaning that the function, this is not strictly convex, because it's linear.
Strictly convex means there's always some curvature everywhere.
So now I have this thing that's linear minus something that's convex.
Something that's negative, something convex, is concave.
So this thing is linear plus concave.
So it is concave.
So I know just by looking at this that ln of beta, which, of course, is something that lives in Rp, but if I saw it living in R1 it would look like this.
And if I saw it living in R2, it would look like a dome like this.
And the fact that it's strict is also telling me that it is actually a unique maximizer.
So there's unique maximizer in Xi transpose beta, but not in beta necessarily.
We're going to need extra assumptions for this.
OK.
So this is what I say here.
The log-likelihood is strictly concave.
And so as a consequence under extra assumptions on the Xi is because, of course, if the Xi's are all the same, right?
So if the entries of Xi's-- so if Xi is equal to 1, 1, 1, 1, 1, then Xi transpose beta is just the sum of the betas.
And of the beta i's, I will be strictly concaving those guys, but certainly not in the individual entries.
OK.
So I need extra thing on my Xi, so that this happens, just like we needed the matrix capital X in the linear regression case to be a full rank, so we could actually identify would beta was.
OK.
It's going to be exactly the same thing.
So here, this is when we have this very specific parametrization.
And the question is-- but it may not be the case if we change the parameter beta into something else.
OK.
So here, the fact that we use the canonical link, et cetera, everything actually works really to our advantage, so that everything becomes strictly concave.
And we know exactly what's happening.
All right, so I understand I went a bit fast on playing with convex and concave functions.
This is not the purpose.
You know, I could spend a lecture telling you, oh, if I add two concave functions, then the result remains concave.
If I had a concave and a strictly concave, then the result still remains strictly concave.
And we could spend time proving this.
This was just for you to get an intuition as to why this is correct.
But we don't really have time to go into too much detail.
One thing you can do-- a strictly concave function, if it's in one dimension, all I need to have is that the second derivative is strictly negative, right?
That's a strictly concave function.
That was the analytic definition we had for strict concavity.
So if this was in one dimension, it would look like this, Yi times Xi times beta.
Now, beta is just one number.
And then I would have minus beta Xi times b.
And this is all over phi.
You take second derivatives.
The fact that this is linear in beta, this is going to go away.
And here, I'm just going to be left with minus-- so if I take the second derivative with respect to beta, this is going to be equal to minus b prime prime Xi beta times Xi squared divided by phi.
So this is clearly positive.
If Xi is 0, this is degenerate, so I would not get it.
Then I have the second derivative of b prime, which I know is positive, because of the variance thing that I have here, divided by phi.
And so that would all be fine.
That's for one dimension.
If I wanted to do this in higher dimensions, I would have to say that the Hessian is a positive definite matrix.
And that's maybe a bit beyond what this course is.
So in the rest of this chapter, I will do what I did not do when we talked about maximum likelihood.
And what we're going to do is we're going to actually show how to do this maximization, right?
So here, we know that the function is concave.
But what it looks like specifically depends on what b is.
And for different b's, I'm going to have different things to do, Just like when I was talking about maximum likelihood estimation, if it had a concave log-likelihood function, I could optimize it.
But depending on what the function is, I would actually need some algorithms that may be working better on some functions than others.
Now, here I don't have random things.
I have the b is the cumulant generating function of a canonical exponential family.
And there is a way for me to sort of leverage that.
So not only is there the b part, but there's also the linear part.
And if I start trying to use that, I'm actually going to be able to devise very specific optimization algorithms.
And the way I'm going to be able to do this is by thinking of simple black box optimization to which I can actually technically feed any function.
But it's going to turn out that the iterations of this iterative algorithms are going to look very familiar when we just plug in the particular values of b, of the log-likelihood that we have for this problem.
And so the three methods we're going to talk about going from more black box-- meaning you can basically stuff in any function that's going to work, any concave function that's going to work, all the way to this is working specifically for generalized linear models-- are Newton-Raphson method.
Who's already heard about the Newton-Raphson method?
So there's probably some people actually learned this algorithm without even knowing the word algorithm, right?
It's a function.
Typically, it's supposed to be finding roots of functions.
But finding the root of a function of the derivative is the same as finding the minimum of a function.
So that's the first black box method.
I mean, it's pretty old.
And then there's something that's very specific to what we're doing, which is called-- so this Newton-Raphson method is going to involve the Hessian of our log-likelihood.
And since we know something about the Hessian for a particular problem, we're going to be able move one to Fisher-scoring.
And the word Fisher here is actually exactly coming from Fisher information.
So the Hessian is going to involve the Fisher information.
And finally, we will talk about iteratively re-weighted least squares.
And that's not for any function.
It's really when we're trying to use the fact that there is this linear dependence on the Xi.
And this is essentially going to tell us, well, you know, you can use least squares for linear regression.
Here, you can use least squares, but locally, and you have to iterate.
OK. And this last part is essentially a trick by statisticians to be able to solve the Newton-Raphson updates without actually having a dedicated software for this, but just being able to reuse some least squares software.
OK.
So you know, we've talked about this many times.
I just want to make sure that we're all on the same page here.
We have a function f. We're going to assume that it has two derivatives.
And it's a function from Rm to R. So it's fist derivative is called gradient.
That's the vector that collects all the partial derivatives with respect to each of the coordinates.
It's dimension m, of course.
And the second derivative is an m by m matrix.
It's called the Hessian.
And ith row and jth column, you see the second partial derivative with respect to the ith component and the jth component.
OK. We've seen that several times.
This is just multi-variable calculus.
But really the point here is to maybe the notation is slightly different, because I want to keep track of f. So when I write the gradient, I write nabla sub f. And when I write Hessian, I write nabla H sub f. And as I said, if f is strictly concave, then Hf of x is negative definite.
What it means is that if I take any x in Rm, then x transpose Hf, well, that's for any X0 X, this is actually strictly negative.
That's what it means to be negative definite.
OK?
So every time I do x transpose-- so this is like a quadratic form.
And I want it to be negative for all values of X0 and X, both of them.
That's very strong, clearly.
But for us, actually, this is what happens just because of the properties of b.
Well, at least the fact that it's negative, less than or equal to, if I want it to be strictly less I need some properties on X.
And then I will call the Hessian map the function that maps X to this matrix Hf of X.
So that's just the second derivative at x. Yeah
When you what are [INAUDIBLE]??
Where do [INAUDIBLE]??
Oh, yeah.
I mean, you know, you need to be able to apply Schwarz lemma.
Let's say two continue derivatives that's smooth
[INAUDIBLE]
No, that's fine.
OK.
So how does the Newton-Raphson method work?
Well, what it does is that it forms a quadratic approximation to your function.
And that's the one it optimizes at every single point.
OK. And the reason is because we have a closed-form solution to defining the minimum of a quadratic function.
So if I give you a function that's of the form ax squared plus b x plus c, you know exactly a closed form for its minimum.
But if I give you any function or, let's say-- yeah, yeah.
So here, it's all about maximum.
I'm sorry.
If you're confused with me using the word minimum, just assume that it was the word maximum.
So this is how it works.
OK.
If I give you a function which is concave, that's quadratic.
OK.
So it's going to look like this.
So that's of the form ax squared-- where a is negative, of course-- plus bx plus c. Then you can solve your whatever.
You can take the derivative of this guy, set it equal to 0, and you will have an exact equation into what the value of x is that it realizes this maximum.
If I give you any function that's concave, that's all clear, right?
I mean, if I tell you the function that we have here is that the form ax minus b of x, then I'm just going to have something that inverts b prime.
But how do I do that exactly?
It's not clear.
And so what we do is we do a quadratic approximation, which should be true approximately everywhere, right?
So I'm at this point here, I'm going to say, oh, I'm close to being that function.
And if I'm at this point here, I'm going to be close to being that function.
And for this function, I can actually optimize.
And so if I'm not moving too far from one to the other, I should actually get something.
So here's how the quadratic approximation works.
I'm going to write the second order Taylor expansion.
OK. And so that's just going to be my quadratic approximation.
It's going to say, oh, f of x, when x is close to some point x0, is going to close to f of x0 plus the gradient of f at x0 transpose x minus x0.
And then I'm going to have plus 1/2x minus x0 transpose Hf at 0x x minus x transpose, right-- x minus x0.
So that's just my second order Taylor expansion multi-variate 1.
And let's say x0 is this guy.
Now, what I'm going to do is say, OK, if I wanted to set this derivative of this guy equal to 0, I would just have to solve, well, you know, f prime of x equals 0, meaning that X has to be f prime inverse of 0.
And really apart from like being some notation manipulation, this is really not helping me.
OK. Because I don't know what f prime inverse of 0 is in many instances.
However if f has a very specific form which is something that depends on x in a very specific way, there's just a linear term and then a quadratic term, then I can actually do something.
So let's forget about this approach.
And rather than minimizing f, let's just minimize the right-hand side.
OK.
So sorry- maximize.
So maximize the right-hand side.
And so how do I get this?
Well, I just set the gradient equal to 0.
So what is the gradient?
The first term does not depend on x.
So that means that this is going to be 0 plus-- what is the gradient of this thing, of the gradient of f at x0 transpose x minus x0?
What is the gradient of this guy?
So I have a function of the form b transpose x.
What is the gradient of this thing?
[INAUDIBLE]
I'm sorry?
[INAUDIBLE]
I'm writing everything in two-column form, right?
So it's just b. OK.
So here, what is b?
Well, it's gradient of f at x0.
OK. And this term here gradient of f at x0 transpose x0 is just a constant.
This thing is going away as well.
And then I'm looking at the derivative of this guy here.
And this is like a quadratic term.
It's like H times x minus x0 squared.
So when I'm going to take the derivative, I'm going to have a factor 2 that's going to pop out and cancel this one half.
And then I'm going to be left only with this part times this part.
OK.
So that's plus Hf x minus x0.
OK.
So that's just a gradient.
And I want it to be equal to 0.
So I'm just going to solve this equal to 0.
OK?
So that means that if I want to find the minimum, this is just going to be the x* that satisfies this.
So that's actually equivalent to Hf times x* is equal to Hf x0 minus gradient f at x0.
Now, this is a much easier thing to solve.
What is this?
This is just a system of linear equations, right?
I just need to find the x* such that when I pre-multiply it by a matrix I get this vector on the right-hand side.
This is just something of the form ax equals b.
And I have many ways I can do this.
I could do Gaussian elimination, or I could use Spielman's fast Laplacian solvers if I had some particular properties of H. I mean, there's huge activity in terms of how to solve those systems.
But let's say I have some time.
It's not a huge problem.
I can actually just use linear algebra.
And linear algebra just tells me that x* is equal to Hf inverse times this guy, which those two guys are going to cancel.
So this is actually equal to x0 minus Hf inverse gradient f at x0.
And that's just what's called a Newton iteration.
I started at some x0.
I'm at some x0, where I make my approximation.
And it's telling me starting from this x0, I wanted to fully optimize a quadratic approximation, I would just have to take the x*.
That's this guy.
And then I could just use this guy as my x0 and do it again, and again, and again, and again.
And those are called Newton iterations.
And they're basically the workhorse of interior point methods, for example, a lot of optimization algorithms.
And that's what you can see here.
x* is equal to x0 minus the inverse Hessian times the gradient.
We briefly mentioned gradient descent.
We briefly mentioned gradient decent, at some point, to optimize the convex function, right?
And if I wanted to use gradient descent, again, H is a matrix.
But if I wanted to think of H as being a scalar, would it be a positive or negative number?
Yeah
[INAUDIBLE]
Why?
[INAUDIBLE]
Yeah.
So that would be this.
So I want to move against the gradient to do what?
[INAUDIBLE]
To minimize.
But I'm maximizing here, right?
Everything is maximized, right?
So I know that H is actually negative definite.
So it's a negative number.
So you have the same confusions they do.
We're maximizing a concave function here.
So H is negative.
So this is something of the form x0 plus something times the gradient.
And this is what your gradient ascent, rather than descent, would look like.
And all it's saying, Newton is telling you don't take the gradient for granted as a direction in which you want to go.
It says, do a slight change of coordinates before you do this according to what your Hessian looks like, all right?
And those are called second order methods that require knowing what the Hessian is.
But those are actually much more powerful than the gradient descent, because they're using all of the local geometry of the problem.
All of the local geometry of your function is completely encoded in this Hessian.
And in particular it implies that it tells you where to switch and not to go slower in some places or go faster in other places.
Now, this in practice for, say, modern large scale machine learning problems, inverting this matrix H is extremely painful.
It takes too much time.
The matrix is too big, and computers cannot do it.
And people resort to what's called pseudo-Newton method, which essentially tries to emulate what this guy is.
And there's many ways you can do this.
Some of them is by using gradients that you've collected in the past.
Some of them just say, well let's just pretend H is diagonal.
There's a lot of things you can do to just play around this and not actually have to invert this matrix.
OK?
So once you have this, you started from edge 0.
It tells you which H* you can get as a maximizer of the local quadratic approximation to your function.
You can actually just iterate that, all right?
So you start at some x0 somewhere.
And then once you get to some xk, you just do the iteration which is described, which is just find a k plus 1, which is the maximizer of the local quadratic approximation to your function at xk and repeat until convergence.
OK.
So if this was an optimization class, we would prove that convergence actually, eventually, happens for a strictly concave function.
This is a stats class, so you're just going to have to trust me that this is the case.
And it's globally convergent, meaning that you can start wherever you want, and it's going to work for under minor conditions on f. And in particular, those conditions are satisfied for the log-likelihood functions we have in mind.
OK. And it converges at an extremely fast rate.
Usually it's quadratic convergence, which means that every time you make one step, you improve the accuracy of your solution by two digits.
If that's something you're vaguely interested in, I highly recommend that you take a class on them in your optimization.
It's a fascinating topic.
Unfortunately, we don't have much time, but it starts being more and more intertwined with high dimensional statistics and machine learning.
I mean, it's an algorithms class, typically.
But it's very much more principled.
It's not a bunch of algorithms that solve a bunch of problems.
There's basically one basic idea, which is if I have a convex function, I can actually minimize it.
If I have a concave function, I can maximize it.
And it evolves around a similar thing.
So let's stare at this iterative step for a second and pause.
And let me know if you have any questions.
OK.
So, of course, in a second we will plug in for the log-likelihood.
This is just a general thing for a general function f. But then in a second, f is going to be ln.
OK.
So if I wanted to implement that for real, I would have to compute the gradient of ln at a point xk.
And I would have to compute the Hessian at a given point and invert it.
OK.
So this is just the basic algorithm.
And this, as you can tell, used in no place the fact that ln was the log-likelihood associated to some canonical exponential family in a generalized linear model.
This never showed up.
So can we use that somehow?
Optimization for longest time was about making your problems as general as possible accumulating maybe in the interior point method theory in Koenig programming in the mid-'90s.
And now what optimization is doing is that it's [INAUDIBLE] very general.
It' says, OK, if I want to start to go fast, I need to exploit as much structure about my problem as I can.
And the beauty is that as statisticians are a machine learning people, we do have a bunch of very specific problem that we want optimizers to solve.
And they can make things run much faster.
But this did not require to wait until the 21st century.
Problems with very specific structure arose already in this generalized linear model.
So what do we know?
Well, we know that this log-likelihood is really one thing that comes when we're trying to replace an expectation by an average, and then doing something fancy, right?
That was our statistical hammer.
And remember when we introduced likelihood maximization we just said, what do we really want to do is to minimize the KL, right?
That's the thing we wanted to minimize, the KL divergence between two distributions, the true one and the one that's parameterized by some unknown theta.
And we're trying to minimize that over theta.
And we said, well, I don't know what this is, because it's an expectation with respect to some known distribution.
So let me just replace the expectation with respect to my unknown distribution by an average over my data points.
And that's how we justified the existence of the log-likelihood maximization problem.
But here, actually, I might be able to compute this expectation, at least partially where I need it.
And what we're going to do is we're going to say, OK, since at a given point xk, say, let me call it here theta, I'm trying to find the inverse of the Hessian of my log-likelihood, right?
So if you look at the previous one, as I said, we're going to have to compute the Hessian H sub l n of xk, and then invert it.
But let's forget about the inversion step for a second.
We have to compute the Hessian.
This is the Hessian of the function we're trying to minimize.
But if I could actually replace it not by the function I'm trying to minimize to maximize or the log-likelihood, but really by the function I wish I was actually minimizing, which is the KL, right?
Then that would be really nice.
And what happens is that since I'm actually trying to find this at a given xk, I can always pretend that this xk that I have in my current iteration is the true one and compute my expectation with respect to that guy.
And what happens is that I know that when I compute the expectation of the Hessian of the log-likelihood at a given theta and when I take the expectation with respect to the same theta, what I get out is negative Fisher information.
The Fisher information was defined in two ways-- as the expectation of the square of the derivative or negative of the expectation of the second derivative of the log-likelihood.
And so now, I'm doing some sort of a leap of faith here.
Because there's no way the theta, which is the current xk, that's the current theta at which I'm actually doing this optimization-- I'm actually pretending that this is the right one.
But what's going to change by doing this is that it's going to make my life easier.
Because when I take expectations, we'll see that when we look at the Hessian, the Hessian as essentially the derivative of, say, a product is going to be the sum of two terms, right?
The derivative of u times v is u prime v plus uv prime.
One of those two terms is actually going to have expectation 0.
And that's going to make my life very easy when I take expectations and basically just have one term that's going to go away.
And so in particular, my formula, just by the virtue of taking this expectation before inverting the Hessian, is going to just shrink the size of my formulas by half.
OK.
So let's see how this works.
You don't have to believe me.
Is there any question about this slide?
You guys remember when we were doing maximum estimation and Fisher information and the KL divergence, et cetera?
Yeah
[INAUDIBLE]
Because that's what we're really trying to minimize
[INAUDIBLE]
Yeah.
So there's something you need to trust me with, which is that the expectation of H of ln is actually H of the expectation of ln, all right?
Yeah, it's true, right?
Because taking derivative is a linear operator.
And we actually used that several times when we said expectation of partial of l with respect to theta is equal to 0.
Remember we did that?
That's basically what we used, right?
ln is the likelihood
It's the log-likelihood
Log-likelihood, [INAUDIBLE] OK.
When we did Fisher [INAUDIBLE],, we did the likelihood of [INAUDIBLE] observation
Yeah
Why is it ln in this case?
So actually, ln is typically not normalized.
So I really should talk about ln over n. OK.
But let's see that, OK?
So if I have IID observations, that should be pretty obvious.
OK.
So if I have IID x1, xn, with density f theta and if I look at log f theta of Xi, sum from i equal 1 to n, as I said, I need to actually have a 1 over n here.
When I look at the expectation, they all have the same expectation, right?
So this is actually, indeed, equal to negative KL plus a constant.
OK?
And negative KL is because this-- sorry, if I look at the expectation.
So the expectation of this guy is just the expectation of one of them, all right?
So I just do expectation theta.
OK?
Agree?
Remember, the KL was expectation theta log f theta divided by f. So that's between p theta and p theta prime.
Well, no, sorry.
That's the true p. And let's call it f. p theta, right?
So that's what showed up, which is, indeed, equal to minus expectation theta log f theta plus log of f, which is just a constant with respect to theta.
It's just the thing that's up doesn't matter.
OK?
So this is what shows up here.
And just the fact that I have this 1 over n doesn't change, because they're IID.
Now, when I have things that are not IID-- because what I really had was Y1 Yn, and Yi at density f theta i, which is just the conditional density given Xi, then I could still write this.
And now when I look at the expectation of this guy, what I'm going to be left with is just 1 over n sum from i equal 1 to n of the expectation of log f theta i of Yi.
And it's basically the same thing, except that I have a 1 over n expectation in front.
And I didn't tell you this, because I only showed you what the KL divergence was for between two distributions.
But here, I'm telling you what the KL is between two products of distributions that are independent, but not necessarily identically distributed.
But that's what's going to show up, just because it's a product of things.
So when you have the log, it's just going to be a sum.
Other questions?
All right, so what do we do here?
Well, as I said, now we know that the expectation of H is negative Fisher information.
So rather than putting H inverse in my iterates for Newton-Raphson, I'm just going to put the inverse Fisher information.
And remember, it had a minus sign in front.
So I'm just going to pick up a plus sign now, just because i is negative, the expectation of the Hessian.
And this guy has, essentially, the same convergence properties.
And it just happens that it's easier to compute the i than H Ln.
And that's it.
That's really why you want to do this.
Now, you might say that, well, if I use more information, I should do better, right?
But it's actually not necessarily true for several reasons.
But let's say that one is probably the fact that I did not use more information.
Every step when I was computing this thing at xk, I actually pretended that at theta k the true distribution was the one distributed according to theta k. And that was not true.
This is only true when theta k becomes close to the true theta.
And so in a way, what I gained I lost again by making this thing.
It's just really a matter of simple computation.
So let's just see it on a particular example.
Actually, in this example, it's not going to look much simpler.
It's actually going to be the same.
All right, so I'm going to have the Bernoulli example.
All right, so we know that Bernoulli belongs to the canonical exponential family.
And essentially, all I need to tell you what b is.
And b of theta for Bernoulli is log 1 plus e theta, right?
We computed that.
OK. And so when I look at my log-likelihood, it is going to look like the sum from i equal 1 to n of Yi of-- OK, so here I'm going to actually use the canonical link.
So it's going to be Xi transpose beta minus log 1 plus exponential Xi transpose beta.
And phi for this guy is equal to 1.
Is it clear for everyone what I did?
OK.
So remember the density, so that was really just-- so the PMF was exponential Y theta minus log 1 plus e theta.
There was actually no normalization.
That's just the density of a Bernoulli.
And the theta is actually log p over 1 minus p. And so that's what actually gives me what my-- since p is the expectation, this is actually giving me also my canonical link, which is the log [?
at link.
?]
We saw that last time.
And so if I start taking the log of this guy and summing over n and replacing theta by Xi transpose beta, which is what the canonical link tells me to do, I get this guy.
Is that clear for everyone?
If it's not, please redo this step on your own.
OK.
So I want to maximize this function.
Sorry.
So I want to maximize this function over there on the first line as a function of beta.
And so to do this, I want to use either Newton-Raphson or what I call Fisher-scoring.
So Fisher-scoring is the second one, when you replace the Hessian by negative Fisher information.
So I replace these two things.
And so I first take the gradient.
OK.
So let's take the gradient of ln.
So the gradient of ln is going to be, well, sum-- so here, this is of the form Yi, which is a scalar, times a vector, Xi beta.
That's what I erased from here.
The gradient of b transpose x is just b.
So here, I have just Yi Xi.
So that's of the form Yi, which is a scalar, times Xi, which is a vector.
Now, what about this guy?
Well, here I have a function.
So I'm going to have just the usual rule, the chain rule, right?
So that's just going to be 1 over this guy.
And then I need to find the Hessian of this thing.
So the 1 is going away.
And then I apply the chain rule again.
So I get e of Xi transpose beta, and then the Hessian of this thing, which is Xi.
So my Hessian-- my radiant, sorry, I can actually factor out all my Xi's.
And it's going to look like this.
My gradient is a weighted average or weighted sum of the Xi's.
This will always happen when you have a generalized linear model.
And that's pretty clear.
Where did the Xi show up?
Whether it's from this guy or that guy, the Xi came from the fact that when I take the gradient of Xi transpose beta, I have this vector Xi that comes out.
It's always going to be the thing that comes out.
So I will always have something that looks like with some sum with some weights here of the Xi's.
Now, when I look at the second derivative-- so same thing, I'm just going to take the derivative this guy.
Since nothing depends on beta here or here, I'm just going to have to take the derivative of this thing.
And so it's going to be equal.
So if I look now at the Hessian ln as a function of beta, I'm going to have sum from i equal 1 to n of, well, Yi-- what is a derivative of Yi with respect to beta?
[INAUDIBLE]
What?
[INAUDIBLE]
Yeah.
0.
OK?
It doesn't depend on data.
I mean, this distribution does.
But Y itself is just a number, right?
So this is 0.
So I'm going to get the minus.
And then I'm going to have, again, the chain rule that shows up.
So I need to find the derivative of x over 1 plus x.
What is the derivative of x over 1 plus x.
Actually don't even know.
So that gives me-- OK.
So that's 1 over 1 plus x squared.
So that's minus e Xi transpose beta-- sorry, 1 divided by 1 plus e Xi transpose beta squared times the derivative of the exponential, which is e Xi transpose beta and again, Xi.
And then I have this Xi that shows up.
But since I'm looking for a matrix, I'm going to have Xi, Xi transpose, right?
OK?
[INAUDIBLE]
So I know I'm going to need something that looks like a matrix in the end.
And so one way you want to think about it is this is going to spit out an Xi.
There's already an Xi here.
So I'm going to have something that looks like Xi.
And I'm going to have to multiply by it another vector Xi.
And I want it to form a matrix.
And so what you need to do is to take an outer product.
And that's it.
So now as a result, the updating rule is this.
Honestly, this is not a result of anything.
I actually rewrote everything that I had before with a theta replaced by beta, because it's just painful to rewrite this entire thing, put some big parenthesis and put minus 1 here.
And then I would have to put the gradient, which is this thing here.
So as you can imagine, this is not super nice.
Actually, what's interesting is at some point I mentioned there's a pseudo-Newton method.
They're actually doing exactly this.
They're saying, oh, at each iteration, I'm actually going to just take those guys.
If I'm at iteration k, I'm actually just going to sum those guys up to k rather than going all the way to n and look at every one.
So you're just looking at your observations one at a time based on where you were before.
OK.
So you have a matrix.
You need to invert it.
So if you want to be able to invert it, you need to make sure that the sum with those weights of Xi outer, Xi, or Xi, Xi transpose is invertible.
So that's a condition that you need to have.
And well, you don't have to, because technically you don't need to invert.
You just need to solve the linear system.
But that's actually guaranteed in most of the cases if n is large enough.
All right, so everybody sees what we're doing here?
OK.
So that's for the Newton-Raphson.
If I wanted to actually do the Fisher-scoring, all I would need to do is to replace the Hessian here by its expectation when I pretend that the beta have, iteration k, is the true one.
What is the expectation of this thing?
And when I say expectation here, I'm always talking about conditional expectation of Y given X.
The only distributions that matter, that have mattered in this entire chapter, are conditional expectation of Y given X.
The conditional expectation of this thing given X is what?
It's itself.
It does not depend on Y.
It only depends on the X's.
So conditionally on x, this thing as far as we're concerned, is completely deterministic.
So it's actually equal to it's expectation.
And so in this particular example, there's no difference between Fisher-scoring and Newton-Raphson.
And the reason is because the gradient no longer depends on Yi-- I'm sorry.
The Hessian no longer depends on Yi.
OK?
This slide is just repeating some stuff that I've said.
OK.
So I think this is probably-- OK, let's go through this actually.
At some point, I said that Newton-Raphson-- do you have a question?
Yeah.
When would the gradient-- sorry, the Hessian ever depend on Yi?
Because it seems like Yi is just-- or at least when you have a canonical link, that the log-likelihood is just [INAUDIBLE] to Yi Xi [INAUDIBLE] theta and that's the only place Y shows up.
So [INAUDIBLE] derivative [INAUDIBLE] never depend on Y?
Not when you have a canonical link
So if it's not a [INAUDIBLE] there's is no difference between--
No

Yeah.
So yeah, maybe I wanted you to figure that out for yourself.
OK.
So Yi times Xi transpose beta.
So essentially, when I have a general family, what he's referring to is that this is just b of Xi transpose beta.
So I'm going to take some derivatives.
And there's going to be something complicated coming out of this.
But I'm certainly not going to have some Yi showing up.
The only place where Yi shows up is here.
Now, if I take two derivatives, this thing is gone, because it's linear.
The first one is going to keep on like this guy.
And the second one is going to make it go on.
The only way this actually shows up is when to have an H here.
And if I have an H, then I can take second derivatives.
And this thing is not going to be completely independent of beta.
Sorry.
Yeah, this thing is still going to depend on beta, which means that this Yi term is not going to disappear.
I believe we'll see an example of that, or maybe I removed it.
I'm not sure, actually.
I think we will see an example.
So let us do a Iteratively Re-weighted Least Squares, or IRLS, which I've actually recently learned is a term that even though it was defined in the '50s, people still feel free to use to define to call their new algorithms which have nothing to do with this.
This is really something where you actually do iteratively re-weighted least squares.
OK. Let's just actually go through this quickly what is going to be iteratively re-weighted least squares.
The way the steps that we had here showed up-- let's say those guys, x*-- is this, is when were actually solving this linear system, right?
That was the linear system we were trying to solve.
But solving a linear system can be done by just trying to minimize, right?
If I have x a and b, it's the same as minimizing the norm of ax minus b squared over x.
If I can actually find an x for which it's 0, it means that I've actually solved my problem.
And so that means that I can solve linear systems by solving least square problems.
And least square problems are things that statisticians are comfortable solving.
And so all I have to do is to rephrase this as at least square problem.
OK?
And you know, I could just write it directly like this.
But there's a way to streamline it a little bit.
And that's actually by using weights.
OK.
So I've come in the weights-- well, not today, actually, but very soon, all right?
So this is just a reminder of what we had.
We have that's Yi give Xi as a distribution distributed according to some distribution in the canonical exponential family.
So that means that the log-likelihood looks like this.
Again, this does not matter to us.
This is the form that matters.
And we have a bunch of relationships that we actually spent some time computing.
The first one is that mu is b prime of theta i.
The second one is that if I take g of mu i, I get this systematic component, Xi transpose beta that's modeling.
Now, if I look at the derivative mu i with respect to theta i, this is the derivative of b prime of theta i with respect to theta i.
So that's the second derivative.
And I'm going to call it Vi.
If phi is equal to 1, this is actually the variance.
And then I have this function H, which allows me to bypass altogether the existence of this parameter mu, which says if I want to go from Xi transpose beta all the way to theta i, I have to first do g inverse, and then b prime inverse.
If I stopped here, I would just have mu.
OK?
OK.
So now what I'm going to do is I'm going to apply the chain rule.
And I'm going to try to compute the derivative of my log-likelihood with respect to beta.
So, again, the log-likelihood is much nicer when I read it as a function of theta than a function of beta, but it's basically what we've been doing by hand.
You can write it as a derivative with respect to theta first, and then multiply by the derivative of theta with respect to beta.
OK. And we know that theta depends on beta as H of Xi transpose beta.
OK?
I mean, that's basically what we've been doing for the Bernoulli case.
I mean, we used the chain rule without actually saying it.
But this is going to be convenient to actually make it explicitly show up.
OK.
So when I first take the derivative of my log-likelihood with respect to theta, I'm going to use the fact that my canonical family is super simple.
OK.
So what I have is that my log-likelihood ln is the sum from i equal 1 to n of Yi theta i minus b of theta i divided by phi plus some constant, which will go away as soon as I'm going to take my first derivative.
So if I take the derivative with respect to theta i of this guy, this is actually going to be equal to Yi minus b prime theta i divided by phi.
And then I need to multiply by the derivative of theta i with respect to beta.
Remember, theta is H of Xi transpose beta.
So the derivative of theta i with respect to beta j, this is equal to H prime of Xi transpose beta.
And then I have the derivative of this guy.
Actually, let me just do the gradient of theta I at beta, right?
That's what we did.
I'm just thinking of theta i as being a function of theta.
So what should I add here?
It was just the vector Xi, which is just the chain rule again.
That's Hi prime, right?
You don't see it, but there's a prime here that's derivative.
OK. We've done that without saying it explicitly.
So now if I multiply those two things I have this Yi minus b prime of the theta i, which I call by its good name, which is mu i. b prime of theta i is the expectation of Yi conditionally on Xi.
And then I multiply by this thing here.
So here, this thing is written coordinate by coordinate.
But I can write it as a big vector when I stack them together.
And so what I claim is that this thing here is of the form Y minus mu.
But here I put some tildes.
Because what I did is that first I multiplied everything by g prime of mu for each mu.
OK.
So why not?
OK. Actually, on this slide it will make no sense why I do this.
I basically multiply by g prime on one side and divide by g prime on the other side.
So what I write so far is that the gradient of ln with respect to beta is the sum from i equal 1 to n of Yi minus mu i, let's call it, divide by phi times H prime of Xi transpose beta Xi.
OK.
So I just stacked everything that's here.
And now I'm going to start calling things.
The first thing I'm going to do is I'm going to divide.
So this guy here I'm going to push here.
Now, this guy here I'm actually going to multiply by g prime of mu i.
And this guy I'm going to divide by g prime of mu i.
So there's really nothing that happened here.
I just took g prime and multiply and divide it by g prime.
Why do I do this?
Well, that's actually going to be clear when we talk about iteratively re-weighted least squares.
But now, essentially I have a new mu, a Y which is-- so this thing now is going to be Y tilde minus mu tilde, so i, i.
Now, this guy here I'm going to call Wi.
And I have the Xi that's there, which means that now the thing that I have here I can write as follows.
Gradient ln of beta is equal to what?
Well, I'm going to write it in matrix forms.
So I have the sum over i of something multiplied by Xi.
So I'm going to write it as x transpose.
Then I'm going to have this matrix W1 Wn, and then 0 elsewhere.
And then I'm going to have my Y tilde minus mu.
And remember, X is the matrix with-- sorry, it should be a bit [INAUDIBLE].
I have n, and then p. And here I have my Xi j in this matrix on row i and column j.
And this is just a matrix that has the Wi's on the diagonal.
And then I have Y tilde minus mu.
So this is just the matrix we're writing of this formula.
All right.
So it's just saying that if I look at the sum of weighted things of my columns of Xi, it's basically the same thing.
When I'm going to multiply this by my matrix, I'm going to get exactly those terms, right?
Yi minus mu i tilde times Wi.
And then when I actually take this Xi transpose times this guy, I'm really just getting the sum of the columns with the weights, right?
Agree?
If I look at this thing here, this is a vector that has S coordinates, Wi times Yi tilde minus mu i tilde.
And I have n of them.
So when I multiply X transpose by this guy, I'm just looking at a weighted sum of the columns of X transpose, which is a weighted sum of the rows of X, which are exactly my Xi's.
All right, and that's this weighted sum of the Xi's.
OK.
So here, as I said, the fact that we decided to put this g prime of mu i here and g prime of mu i here, we could have not done this, right?
We could have just said, I forget about the tilde and just call it Yi minus mu i.
And here, I just put everything I don't know into some Wi.
And so why do I do this?
Well, it's because when I actually start looking at the Hessian, what's going to happen?
[INAUDIBLE]
Yeah.
We'll do that next time.
But let's just look quickly at the outcome of the computation of my Hessian.
So I compute a bunch of second derivatives.
And here, I have two terms, right?
Well, he's gone.
So I have two terms.
And when I take the expectation now, it's going to actually change, right?
This thing is actually going to depend on Yi.
Because I have an H which is not the identity.
Oh, no, you're here, sorry.
So when I start looking at the expectation, so I look at the conditional expectation given Xi.
The first term here has a Yi minus expectation.
So when I take the conditional expectation, this is going to be 0.
The first term is going away when I take the conditional expectation.
But this was actually gone already if we had the canonical term, because the second derivative of H when H is the identity is 0.
But if H is not the identity, H prime prime may not be 0.
And so I need that part to remove that term.
And so now, you know, I work a little bit, and I get this term.
That's not very surprising.
In the second derivative, I see I have terms in b prime prime.
I have term in H prime, but squared.
And then I have my Xi outer Xi, Xi, Xi transpose, which we know we would see.
OK.
So we'll go through those things next time.
But what I want to show you is that now once I compute this, I can actually show that if I look at this product that showed up, I had b prime prime times H prime squared.
One of those terms is actually 1 over g prime.
And so I can rewrite it as one of the H primes, because I had a square, divided by g prime.
And now, I have this Xi Xi transpose.
So if I did not put the g prime in the W that I put here completely artificially, I would not be able to call this guy Wi, which is exactly what it is from this board.
And now that this guy is Wi, I can actually write this thing here as X transpose WX.
OK?
And that's why I really wanted my W to have this g prime of mu i in the denominator.
Because now I can actually write a term that depends on W. Now, you might say, how do I reconcile those two things?
What the hell are you doing?
And what the hell I'm doing is essentially that I'm saying that if you write beta k according to the Fisher-scoring iterations, you can actually write it as just this term here, which is of the form X transpose X inverse X transpose Y.
But I actually squeezed in these W's.
And that's actually a weighted least square.
And it's applied to this particular guy.
So we'll talk about those weighted least squares.
But remember, least squares is of the form-- beta hat is X transpose X inverse X transpose Y.
And here it's basically the same thing, except that I squeeze in some W after my X transpose.
OK.
So that's how we're going to solve it.
I don't want to go into the details now, mostly because we're running out of time.
Are there any questions?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I apologize.
My voice is not 100%.
So if you don't understand what I'm saying, please ask me.
So we're going to be analyzing-- actually, not really analyzing.
We described a second-order method to optimize the log likelihood in a generalized linear model, when the parameter of interest was beta.
So here, I'm going to rewrite the whole thing as a beta.
So that's the equation you see.
But we really have this beta.
And at iteration k plus 1, beta is given by beta k. And then I have a plus sign.
And the plus, if you think of the Fisher information at beta k as being some number-- if you were to say whether it's a positive or a negative number, it's actually going to be a positive number, because it's a positive semi-definite matrix.
So since we're doing gradient ascent, we have a plus sign here.
And then the direction is basically gradient ln at beta k. OK?
So this is the iterations that we're trying to implement.
And we could just do this.
At each iteration, we compute the Fisher information, and then we do it again and again.
All right.
That's called the Fisher-scoring algorithm.
And I told you that this was going to converge.
And what we're going to try to do in this lecture is to show how we can re-implement this, using iteratively re-weighted least squares, so that each step of this algorithm consists simply of solving a weighted least square problem.
All right.
So let's go back quickly and remind ourselves that we are in the Gaussian-- sorry, we're in the exponential family.
So if I look at the log likelihood for one observation, so here it's ln-- sorry.
This is the sum from i equal 1 to n of yi minus-- OK, so it's yi times theta i, sorry, minus b of theta i.
Then there's going to be some parameter.
And then I have plus c of yi phi.
OK.
So just the exponential went away when I took the log of the likelihood.
And I have n observations, so I'm summing over all n observations.
All right.
Then we had a bunch of formulas that we came up to be.
So if I look at the expectation of yi-- so that's really the conditional of yi, given xi.
But like here, it really doesn't matter.
It's just going to be different for each i.
This is denoted by mu i.
And we showed that this was beta prime of theta i.
Then the other equation that we found was that.
And so what we want to model is this thing.
We want it to be equal to xi transpose beta- sorry g of this thing.
All right.
So that's our model.
And then we had that the variance was also given by the second derivative.
I'm not going to go into it.
What's actually interesting is to see, if we want to express theta i as a function of xi, what we get, going from xi to mu i by g inverse, and then to theta i by b inverse, we get that theta i is equal to h of xi transpose beta, h of xi transpose beta, where h is the inverse-- so which order is --this?
Is the inverse of g, and then the compose would be prime.
OK?
So we remember that last time, those are all computations that we've made, but they're going to be useful in our derivation.
And the first thing we did last time is to show that, if I look now at the derivative of the log likelihood with respect to one coordinate of beta, which is going to give me the gradient if I do that for all the coordinates, what we ended up finding is that we can rewrite it in this form, some of yi tilde minus mu tilde.
So let's remind ourselves that-- so y tilde is just y divided-- well, OK y tilde i is yi-- is it times or divided-- times g prime of mu i. Mu tilde i is mu i times g prime of mu i.
And then that was just an artificial thing, so that we could actually divide the weights by g prime.
But the real thing that built the weights are this h prime.
And there's this normalization factor.
And so if we read it like that-- so if I also write that wi is h prime of xi transpose beta divided by g prime of mu i times phi, then I could actually rewrite my gradient, which is a vector, in the following matrix form, the gradient ln at beta.
So the gradient of my log likelihood of beta took the following form.
It was x transpose w, and then y tilde minus mu tilde.
And here, w was just the matrix with w1, w2, all the way to wn on the diagonal and 0 on of the up diagonals.
OK?
So that was just taking the derivative and doing a slight manipulations that said, well, let's just divide whatever is here by g prime and multiply whatever is here by g prime.
So today, we'll see why we make this division and multiplication by g prime, which seems to make no sense, but it actually comes from the Hessian computations.
So the Hessian computations are going to be a little more annoying.
Actually, let me start directly with the coordinate y's derivative, right?
So to build this gradient, what we used, in the end, was that the partial derivative of ln with respect to the gth coordinate of beta was equal to the sum over i of yi tilde minus mu i tilde times wi times the gth coordinate of xi.
OK?
So now, let's just take another derivative, and that's going to give us the entries of the Hessian.
OK, so we're going to the second derivative.
So what I want to compute is the derivative with respect to beta j and beta k. OK.
So where does beta j-- so here, I already took the derivative with respect to beta j.
So this is just the derivative with respect to beta k of the derivative with respect to beta j.
So what I need to do is to take the derivative of this guy with respect to beta k. Where does beta k show up here?
It's set in, in two places
In the y's?
No, it's not in the y's.
The y's are my data, right?
But I mean, it's in the y tildes.
Yeah, because it's in mu, right?
Mu depends on beta.
Mu is g inverse of xi transpose beta.
And it's also in the wi's.
Actually, everything that you see is directly-- well, OK, w depends on mu n on beta explicitly.
But the rest depends only on mu.
And so we might want to be a little-- well, we can actually use the-- did I use the chain rule already?
Yeah, it's here.
But OK, well, let's go for it.
Oh yeah, OK.
Sorry, I should not write it like that, because that was actually-- right, so I make my life miserable by just multiplying and dividing by this g prime of mu.
I should not do this, right?
So what I should just write is say that this guy here-- I'm actually going to remove the g prime of mu, because I just make something that depends on theta appear when it really should not.
So let's just look at the last but one equality.
OK.
So that's the one over there, and then I have xi j. OK, so here, it make my life much more simple, because yi does not depend on beta, but this guy depends on beta, and this guy depends on beta.
All right.
So when I take the derivative, I'm going to have to be a little more careful now.
But I just have a derivative of a product, nothing more complicated.
So this is what?
Well, the sum is going to be linear, so it's going to come out.
Then I'm going to have to take the derivative of this term.
So it's just going to be 1 over psi.
Then the derivative of mu i with respect to beta k, which I will just write like this, times h prime of xi transpose beta xi j.
And then I'm going to have the other one, which is yi minus mu i over 5 times the second derivative of h of xi transpose beta.
And then I'm going to take the derivative of this guy with respect to beta j with beta k, which is just xi k. So I have xi j times xi k. OK.
So I still need to compute this guy.
So what is the partial derivative with respect to beta k of g?
So mu is g of-- worry, it's g inverse of xi transpose beta.
OK?
So what do I get?
Well, I'm going to get definitely the second derivative of g. Well, OK, that's actually not a bad idea.
Well, no, that's OK.
I can make the second-- what makes my life easier, actually?
Give me one second.
Well, there's no one that actually makes my life so much easier.
Let's just write it.
Let's go with this guy.
So it's going to be g prime prime of xi transpose beta times xi k. OK?
So now, what do I have if I collect my terms?
I have that this whole thing here, the second derivative is, well, I have the sum from 1 equal 1 to n. Then I have terms that I can factor out, right?
Both of these guys have xi j, and this guy pulls out an xi k. And it's also here, xi j times xi k, right?
So everybody here is xi j xi k. And now, I just have to take the terms that I have here.
The 1 over phi, I can actually pull out in front.
And I'm left with the second derivative of g times the first derivative of h, both taken at xi transpose beta.
And then, I have this yi minus mu i times the second derivative of h, taken at xi transpose beta.
OK.
But here, I'm looking at Fisher scoring.
I'm not looking at Newton's method, which means that I can actually take the expectation of the second derivative.
So when I start taking the expectation, what's going to happen-- so if I take the expectation of this whole thing here, well, this guy, it's not-- and when I say expectation, it's always conditionally on xi.
So let's write it-- x1 xn.
So I take conditional.
This is just deterministic.
But what is the conditional expectation of yi minus mu i times this guy, conditionally on xi?
0, right?
Because this is just the conditional expectation of yi, and everything else depends on xi only, so I can push it out of the conditional expectation.
So I'm left only with this term.
OK.
So now I need to-- sorry, and I have xi xj, xi j xi j. OK So now, I want to go to something that's slightly more convenient for me.
So maybe we can skip that part here, because this is not going to be convenient for me anyway.
So I just want to go back to something that looks eventually like this.
OK, that's what I'm going to want.
So I need to have my xi show up with some weight somehow.
And the weight should involve h prime divided by g prime.
Again, the reason why I want to see g prime coming back is because I had g prime coming in the original w. This is actually the same definition as the w that I used when I was computing the gradient.
Those are exactly these w's, those guys.
So I need to have g prime that shows up.
And that's where I'm going to have to make a little bit of computation here.
And it's coming from this kind of consideration.
OK?
So this thing here-- well, actually, I'm missing the phi over there, right?
There should be a phi here.
OK.
So we have exactly this thing, because this tells me that, if I look at the Hessian-- so this was entry-wise, and this is exactly the form of something of the form of the k. This is exactly the jth kth entry of xi xi transpose.
Right?
We've used that before.
So if I want to write this in a vector form, this is just going to be the sum of something that depends on i times xi xi transpose.
So this is 1 over phi sum from i equal 1 to n of g prime prime xi transpose beta h prime xi transpose beta xi xi transpose.
OK?
And that's for the entire matrix.
Here, that was just the j kth entries of this matrix.
And you can just check that, if I take this matrix, the j kth entry is just the product of the jth coordinate and the kth coordinate of xi.
All right.
So now I need to do my rewriting.
Can I write this?
So I'm missing something here, right?
Oh, I know where it's coming from.
Mu is not g prime of x beta.
Mu is g inverse of x beta, right?
So the derivative of x prime is not g prime prime.
It's like this guy-- no, 1 over this, right?
Yeah.
OK?
The derivative of g inverse is 1 over g prime of gene inverse.
I need you guys, OK?
All right.
So now, I'm going to have to rewrite this.
This guy is still going to go away.
It doesn't matter, but now this thing is becoming h prime over g prime of g inverse of xi transpose beta, which is the same here, which is the same here.
OK?
Everybody approves?
All right.
Well, now, it's actually much nicer.
What is g inverse of xi transpose beta?
Well, that was exactly the mistake that I just made, right?
It's mu i itself.
So this guy is really g prime of mu i.
Sorry, just the bottom, right?
So now, I have something which looks like a sum from i equal 1 to n of h prime of xi transpose beta, divided by g prime of mu i phi times xi xi transpose, which I can certainly write in matrix form as x transpose wx, where w is exactly the same as before.
So it's w1 wn.
And wi is h prime of xi transpose beta divided by g prime of mu i.
There's a prime here times phi, which is the same that we had here.
And it's supposed to be the same that we have here, except the phi is in white.
That's why it's not there.
OK. All right?
So it's actually simpler than what's on the slides, I guess.
All right.
So now, if you pay attention, I actually never force this g prime of mu i to be here.
Actually, I even tried to make a mistake to not have it.
And so this g prime of mu i shows up completely naturally.
If I had started with this, you would have never questioned why I actually didn't multiply by g prime and divided by g prime completely artificially here.
It just shows up naturally in the weights.
But it's just more natural for me to compute the first derivative first than the second derivative second, OK?
And so we just did it the other way around.
But now, let's assume we forgot about everything.
We have this.
This is a natural way of writing it, x transpose wx.
If I want something that involves some weights, I have to force them in by dividing by g prime of mu i and therefore, multiplying yi n mu i by this wi.
OK?
So now, if we recap what we've actually found, we got that-- let me write it here.
We also have that the expectation of H ln of beta x transpose xw.
So if I go back to my iterations over there, I should actually update beta k plus 1 to be equal to beta k plus the inverse.
So that's actually equal to negative i of beta k-- well, yeah.
That's negative i of beta, I guess.
Oh, and beta here shows up in w, right?
w depends on beta.
So that's going to be beta k. So let me call it wk.
So that's the diagonal of H prime xi transpose beta k, this time, divided by g prime of mu i k phi.
OK?
So this beta k induces a mu by looking at g inverse of xi transpose beta k. All right.
So mu i k is g inverse of xi transpose beta k. So that's 2 to the-- sorry, that's an iteration.
And so now, if I actually write these things together, I get minus x transpose wx inverse.
So that's wk.
And then I have my gradient here that I have to apply at k, which is x transpose wk.
And then I have y tilde k minus mu tilde k, where, again, the indices-- I mean the superscript k are pretty natural.
y tilde k just means that-- so that's just yi.
So that's just yi times g prime of mu i k. And mu tilde k is, if I look at the i coordinate, it's just going to be mu i times g prime of mu i. OK?
So I just add superscripts k to everything.
So I know that those things get updated real time, right?
Every time I make one iteration, I get a new value for beta, I get a new value for mu, and therefore, I get a new value for w. Yes?
[INAUDIBLE] the Fisher equation [INAUDIBLE]??
Yeah, that's a good point.
So that's definitely a plus, because this is a positive, semi-definite matrix.
So this is a plus.
And well, that's probably where I erased it.
OK. Let's see where I made my mistake.
So there should be a minus here.
There should be a minus here.
There should be a minus even at the beginning, I believe.
So that means that what is my-- oh, yeah, yeah.
So you see, when we go back to the first, so what I erased was basically this thing here, yi minus mu i.
And when I took the first derivative-- so it was the derivative with respect to H prime.
So the derivative with respect to the second term-- I mean, the derivative of the second term was actually killed, because we took the expectation of this guy.
But when we took the derivative of the first term, which is the only one that stayed, this guy went away.
But there was a negative sign from this guy, because that's the thing we took the negative off.
So it's really, when I take my second derivative, I should carry out the minus signs everywhere.
OK?
So it's just I forget this minus throughout.
You see the first term went away, on the first line there.
The first term went away, because the conditional expectation of yi, given xi 0.
And then I had this minus sign in front of everyone, and I forgot it.
All right.
Any other mistake that I made?
We're good?
All right.
So now, this is what we have, that xk-- sorry, that beta k plus 1 is equal to beta k plus this thing.
OK?
And if you look at this thing, it sort of reminds us of something.
Remember the least squares estimator.
So here, I'm going to actually deviate slightly from the slides.
And I will tell you how.
The slides take beta k and put it in here, which is one way to go.
And just think of this as a big least square solution.
Or you can keep the beta k, solve another least squares, and then add it to the beta k that you have.
It's the same thing.
So I will take the different routes.
So you have the two options, all right?
OK.
So when we did the least squares-- so parenthesis least squares-- we had y equals x beta plus epsilon.
And our estimator beta hat was x transpose x inverse x transpose y, right?
And that was just solving the first order condition, and that's what we found.
Now look at this-- x transpose bleep x inverse, x transpose bleep something.
OK?
So this looks like, if this is the same as the left board, if wk is equal to the identity matrix, meaning we don't see it, and y is equal to y tilde k minus mu tilde k-- so those similarities, the fact that we just squeeze in-- so the fact that the response variable is different is really not a problem.
We just have to pretend that this is equal to y tilde minus mu tilde.
I mean, that's just the least squares.
When you call a software that does least squares for you, you just tell it what y is, you tell it with x is, and it makes the computation.
So you would just lie to it and say all the actual y I want is this thing.
And then we need to somehow incorporate those weights.
And so the question is, is that easy to do?
And the answer is yes, because this is a setup where this would actually arise.
So one of the things that's very specific to what we did here and with least squares, we assume that epsilon, when we did at least the inference, we assumed that epsilon was normal 0 and the covariance matrix was the identity, right?
What if the covariance matrix is not the identity?
If the covariance matrix is not the identity, then your maximum likelihood is not exactly these least squares.
If the covariance matrix is any matrix you have another solution, which involves the inverse of the covariance matrix that you have, but if your covariance matrix, in particular, is diagonal-- which would mean that each observation that you get in this system of equations is still independent, but the variances can change from one line to another, from one observation to another-- then it's called heteroscedastic.
"Hetero" means "not the same."
"Scedastic" is "scale."
And a heteroscedastic case, you would have something slightly different.
And it makes sense that, if you know that some observations have much less variance than others, you might want to give them more weight.
OK?
So if you think about your usual drawing, and maybe you have something like this, but the actual line is really-- OK, let's say you have this guy as well, so just a few here.
If you start drawing this thing, if you do least squares, you're going to see something that looks like this on those points.
But now, if I tell you that, on this side, the variance is equal to 100, meaning that those points are actually really far from the true one, and here on this side, the variance is equal to 1, meaning that those points are actually close to the line you're looking for, then the line you should be fitting is probably this guy, meaning do not trust the guys that have a lot of variance.
And so you need somehow to incorporate that.
If you know that those things have much more variance than these guys, you want to weight this.
And the way you do it is by using weighted least squares.
OK.
So we're going to open in parentheses on weighted least squares.
It's not a fundamental statistical question, but it's useful for us, because this is exactly what's going to spit out-- something that looks like this with this matrix w in there.
OK.
So let's go back in time for a second.
Assume we're still covering least squares regression.
So now, I'm going to assume that y is x beta plus epsilon, but this time, epsilon is a multivariate Gaussian in, say, p dimensions with mean 0.
And covariance matrix, I will write it as w inverse, because w is going to be the one that's going to show up.
OK?
So this is the so-called heteroscedastic.
That's how it's spelled, and yet another name that you can pick for your soccer team or a capella group.
All right.
So the maximum likelihood, in this case-- so actually, let's compute the maximum likelihood for this problem, right?
So the log likelihood is what?
Well, we're going to have the term that tells us that it's going to be-- so OK. What is the density of a multivariate Gaussian?
So it's going to be a multivariate Gaussian in p dimension with mean x beta and covariance matrix w inverse, right?
So that's the density that we want.
Well, it's of the form 1 over determinant of w inverse times 2 pi to the p/2.
OK?
And times exponential, and now, what I have is x minus x beta transpose w-- so that's the inverse of w inverse-- x minus x beta divided by 2.
OK?
So this is x minus mu transpose sigma inverse x minus mu divided by 2.
And if you want a sanity check, just assume that sigma-- yeah?
Is it x minus x beta or y?
Well, you know, if you want this to be y, then this is y, right?
Sure.
Yeah, maybe it's less confusing.
So if you should do p is equal to 1, then what does it mean?
It means that you have this mean here.
So let's forget about what it is.
But this guy is going to be just 1 sigma squared, right?
So what you see here is the inverse of sigma squared.
So that's going to be 2 over 2 sigma squared, like we usually see it.
The determinant of w inverse is just the product of the entry of the 1 by 1 matrix, which is just sigma square.
OK?
So that should be actually-- yeah, no, that's actually-- yeah, that's sigma square.
And then I have this 2 pi.
So square root of this, because p is equal to 1, I get sigma square root 2 pi, which is the normalization that I get.
This is not going to matter, because, when I look at the log likelihood as a function of beta-- so I'm assuming that w is known-- what I get is something which is a constant.
So it's minus p minus n times p/2 times log that w inverse times 2 pi.
OK?
So this is just going to be a constant.
It won't matter when I do the maximum likelihood.
And then I'm going to have what?
I'm going to have plus 1/2 of y minus x beta transpose w y minus x beta.
So if I want to take the maximum of this guy-- sorry, there's a minus here.
So if I want to take the maximum of this guy, I'm going to have to take the minimum of this thing.
And the minimum of this thing, if you take the derivative, you get to see-- so that's what we have, right?
We need to compute the minimum of y minus x beta transpose w minus y minus x beta.
And the solution that you get-- I mean, you can actually check this for yourself.
The way you can see this is by doing the following.
If you're lazy and you don't want to redo the entire thing-- maybe I should keep that guy.
W is diagonal, right?
I'm going to assume that so w inverse is diagonal, and I'm going to assume that no variance is equal to 0 and no variance is equal to infinity, so that both w inverse and w have only positive entries on the diagonal.
All right?
So in particular, I can talk about the square root of w, which is just the matrix, the diagonal matrix, with the square roots on the diagonal.
OK?
And so I want to minimize in beta y minus x beta transpose w y minus x beta.
So I'm going to write w as square root of w times square root of w, which I can, because w-- and it's just the simplest thing, right?
If w is w1 wn, so that's my w, then the square root of w is just square root of w1 square root of wn, and then 0 is elsewhere.
OK?
So the product of those two matrices gives me definitely back what I want, and that's the usual matrix product.
Now, what I'm going to do is I'm going to push one on one side and push the other one on the other side.
So that gives me that this is really the minimum over beta of-- well, here I have this transposed, so I have to put it on the other side.
w is clearly symmetric and so is square root of w. So the transpose doesn't matter.
And so what I'm left with is square root of wy minus square root of wx beta transpose, and then times itself.
So that's square root wy minus square root w-- oh, I don't have enough space-- x beta.
OK, and that stops here.
But this is the same thing that we've been doing before.
This is a new y.
Let's call it y prime.
This is a new x.
Let's call it x prime.
And now, this is just the least squares estimator associated to a response y prime and a design matrix x prime.
So I know that the solution is x prime transpose x prime inverse x prime transpose y prime.
And now, I'm just going to substitute again what my x prime is in terms of x and what my y prime is in terms of y.
And that gives me exactly x square root w square root w x inverse.
And then I have x transpose square root w for this guy.
And then I have square root wy for that guy.
And that's exactly what I wanted.
I'm left with x transpose wx inverse x transpose wy.
OK?
So that's a simple way to take into account the w that we had before.
And you could actually do it with any matrix that's positive semi-definite, because you can actually talk about the square root of those matrices.
And it's just the square root of a matrix is just a matrix such that, when you multiply it by itself, it gives you the original matrix.
OK?
So here, that was just a shortcut that consisted in saying, OK, maybe I don't want to recompute the gradient of this quantity, set it equal to 0, and see what beta hat had should be.
Instead, I am going to assume that I already know that, if I did not have the w, I would know how to solve it.
And that's exactly what I did.
I said, well, I know that this is the minimum of something that looks like this, when I have the primes.
And then I just substitute back my w in there.
All right.
So that' just the lazy computation.
But again, if you don't like it, you can always take the gradient of this guy.
Yes?
Why is the solution written in the slides different?
Because there's a mistake.
Yeah, there's a mistake on the slides.
How did I make that one?
I'm actually trying to parse it back.
I mean, it's clearly wrong, right?
Oh, no, it's not.
No, it is.
So it's not clearly wrong.
Actually, it is clearly wrong.
Because if I put the identity here, those are still associative, right?
So this product is actually not compatible.
So it's wrong, but there's just this extra thing that I probably copy-pasted from some place.
Since this is one of my latest slide, I'll just color it in white.
But yeah, sorry, there's a mis-- this parenthesis is not here.
Thank you
[INAUDIBLE]
Yeah.
OK?
So why not square root [INAUDIBLE]??
Because I have two of them.
I have one that comes from the x prime that's here, this guy.
And then I have one that comes from this guy here.
OK, so the solution-- let's write it in some place that's actually legible-- which is the correction for this thing is x transpose wx inverse x transpose wy.
OK?
So you just squeeze in this w in there.
And that's exactly what we had before, x transpose wx inverse x transpose w some y. OK?
And what I claim is that this is routinely implemented.
As you can imagine, heteroscedastic linear regression is something that's very common.
So every time you a least squares formula, you also have a way to put in some weights.
You don't have to put diagonal weights, but here, that's all we need.
So here on the slides, again, I took the beta k, and I put it in there, so that I have only one least square solution to formulate.
But let's do it slightly differently.
What I'm going to do here now is I'm going to say, OK, let's feed it to some least squares.
So let's do weighted least squares on a response, y being y tilde k minus mu tilde k, and design matrix being, well, just the x itself.
So that doesn't change.
And the weights-- so the weights are what?
The weights are the wk that I had here.
So wki is h prime of xi transpose beta k divided by g prime of mu i at time k times phi.
OK, and so this, if I solve it, will spit out something that I will call a solution.
I will call it u hat k plus 1.
And to get beta hat k plus 1, all I need to do is to do beta k plus u hat k plus 1-- sorry, beta-- yeah.
OK?
And that's because-- so here, that's not clear.
But I started from there, remember?
I started from this guy here.
So I'm just solving a least squares, a weighted least square that's going to give me this thing.
That's what I called u hat k plus 1.
And then I add it to beta k, and that gives me beta k minus 1.
So I just have this intermediate step, which is removed in the slides.
OK?
So then you can repeat until convergence.
What does it mean to repeat until convergence?
[INAUDIBLE]?
Yeah, exactly.
So you just set some threshold and you say, I promise you that this will converge, right?
So you know that at some point, you're going to be there.
You're going to go there, but you're never going to be exactly there.
And so you just say, OK, I want this accuracy on my data.
Actually, the machine is a little strong.
Especially if you have 10 observations to start with, you know you're going to have something that's going to have some statistical error.
So that should actually guide you into what kind of error you want to be making.
So for example, a good rule of thumb is that if you have n observations, you just take some within-- if you want the L2 distance between the beta-- the two consecutive beta to be less than 1/n, you should be good enough.
It doesn't have to be that machine precision.
And so it's clear how we do this, right?
So here, I just have to maintain a bunch of things, right?
So remember, when I want to recompute-- at every step, I have to recompute a bunch of things.
So I have to recompute the weights.
But if I want to recompute the weights, not only do I need to previous iterate, but I need to know how the previous iterate impacts my means.
So at each step, I have to recalculate mu i k by doing g prime, rate?
Remember mu i k was just g inverse of xi transpose beta k, right?
So I have to recompute that.
And then I use this to compute my weights.
I also use this to compute my y, right?
so my y depends also on g prime of mu i k. I feed that to my weighted least squares engine.
It spits out the u hat k, that I add to my previous beta k. And that gives me my new beta k plus 1.
OK.
So here's the pseudocode, if you want to take some time to parse it.
All right.
So here again, the trick is not much.
It's just saying, if you don't feel like implementing Fisher scoring or inverting your Hessian at every step, then a weighted least squares is actually going to do it for you automatically.
All right.
Then that's just a numerical trick.
There's nothing really statistical about this, except the fact that this calls for a solution for each of the step reminded us of sum of the squares, except that there was some extra weights.
OK.
So to conclude, we'll need to know, of course, xy, the link function.
Why do we need the variance function?
I'm not sure we actually need the variance function.
No, I don't know why I say that.
You need phi, not the variance function.
So where do you start actually, right?
So clearly, if you start very close to your solution, you're actually going to do much better.
And one good way to start-- so for the beta itself, it's not clear what it's going to be.
But you can actually get a good idea of what beta is by just having a good idea of what mu is.
Because mu is g inverse of xi transpose beta.
And so what you could do is to try to set mu to be the actual observations that you have, because that's the best guess that you have for their expected value.
And then you just say, OK, once I have my mu, I know that my mu is a function of this thing.
So I can write g of mu and solve it, using your least squares estimator, right?
So g of mu is of the form x beta.
So you just solve for-- once you have your mu, you pass it through g, and then you solve for the beta that you want.
And then that's the beta that you initialize with.
OK?
And actually, this was your question from last time.
As soon as I use the canonical link, Fisher scoring and Newton-Raphson are the same thing, because the Hessian is actually deterministic in that case, just because when you use the canonical link, H is the identity, which means that its second derivative is equal to 0.
So this term goes away even without taking the expectation.
So remember, the term that went away was of the form yi minus mu i divided by phi times h prime prime of xi transpose beta, right?
That's the term that we said, oh, the conditional expectation of this guy is 0.
But if h prime prime is already equal to 0, then there's nothing that changes.
There's nothing that goes away.
It was already equal to 0.
And that always happens when you have the canonical link, because h is g b prime inverse.
And the canonical link is b prime inverse, so this thing is the identity.
So the second derivative of f of x is equal to x is 0.
OK. My screen says end of show.
So we can start with some questions
I just wanted to clarify.
So iterative-- what is it say for iterative--
Reweighted least squares
Reweighted least squares is an implementation of the Fisher scoring [INAUDIBLE]??
That's an implementation that's just making calls to weighted least squares oracles.
It's called an oracle sometimes.
An oracle is what you assume the machine can do easily for you.
So if you assume that your machine is very good at multiplying by the inverse of a matrix, you might as well just do Fisher scoring yourself, right?
It's just a way so that you don't have to actually do it.
And usually, those things are implemented-- and I just said routinely-- in statistical software.
But they're implemented very efficiently in statistical software.
So this is going to be one of the fastest ways you're going to have to solve, to do this step, especially for large-scale problems
So the thing that computers can do well is the multiplier [INAUDIBLE].
What's the thing that the computers can do fast and what's the thing that [INAUDIBLE]??
So if you were to do this in the simplest possible way, your iterations for, say, Fisher scoring is just multiply by the inverse of the Fisher information, right?
So finding that inverse is slow?
Yeah, so it takes a bit of time.
Whereas, since you know you're going to multiply directly by something, if you just say-- those things are not as optimized as solving least squares.
Actually, the way it's typically done is by doing some least squares.
So you might as well just do least squares that you like.
And there's also less-- well, no, there's no-- well, there is less recalculation, right?
Here, your Fisher, you would have to recompute the entire matrix of Fisher information.
Whereas here, you don't have to.
Right?
You really just have to compute some vectors and the vector of weights, right?
So the Fisher information matrix has, say, n choose two entries that you need to compute, right?
It's symmetric, so it's order n squared entries.
But here, the only things you update, if you think about it, are this weight matrix.
So there is only the diagonal elements that you need to update, and these vectors in there also.
There's two inverses n squared.
So that's much less thing to actually put in there.
It does it for you somehow.
Any other question?
Yeah?
So if I have a data set [INAUDIBLE],, then I can always try to model it with least squares, right?
Yeah, you can
And so this is like setting my weight equal to 1-- the identity, essentially, right?
Well, not exactly, because the g also shows up in this correction that you have here, right?
Yeah
I mean, I don't know what you mean by--
I'm just trying to say, are there ever situations where I'm trying to model a data set and I would want to pick my weights in a particular way?
Yeah

I mean--
[INAUDIBLE] example [INAUDIBLE]
Well, OK, there's the heteroscedastic case for sure.
So if you're going to actually compute those things-- and more generally, I don't think you should think of those as being weights.
You should really think of those as being matrices that you invert.
And don't think of it as being diagonal, but really think of them as being full matrices.
So if you have-- when we wrote weighted least squares here, this was really-- the w, I said, is diagonal.
But all the computations really never really use the fact that it's diagonal.
So what shows up here is just the inverse of your covariance matrix.
And so if you have data that's correlated, this is where it's going to show up.
Hi there.
My name's Andrea Derosier, and I'm a Sloan fellow, as I mentioned earlier.
And my pitch idea is actually having to do with something that I actually know something a little bit about, that many high school or middle schoolers also struggle with, and that is, having braces.
I just recently got my braces off, and I also worked for a orthodontics company before I came here to MIT.
So it's something that's both near and dear to my heart, and also I think I've seen a lot of high schoolers and middle schoolers struggle with this whole process.
I know I didn't know a lot about it.
So basically, that would be my pitch, is to teach kids about what's happening with their teeth, as they're having braces on.
How the teeth move; how osteoblasts and osteoclasts work, to deposit and dissolve some of their bone, which is actually one of the reasons why it hurts so much; and also why your mouth gets dry.
All those kinds of scientific aspects is something that these kids really deal with on a daily basis during a very awkward time of their lives.
So that's my mini-pitch.
Hi.
What if I told you that people actually pay thousands of dollars to have a part of their body intentionally modified-- and it's one that we can see every day?
In other words, people modify their teeth to be straight.
This is called orthodontic braces.
I'm sure many people are already familiar with them from adolescence, but what actually happens in your mouth while your teeth are being moved-- there's a process called bony remodeling that's happening, and it's really pretty fascinating.
As your teeth are moved, your jaw actually partially dissolves around the tooth to accommodate its new position.
But then instead of floating away, there's another process in your body mediated by cells called osteoblasts that actually build up your bone around the new tooth's position.
This process happens throughout our entire bodies, and scientists and engineers are exploiting these really fascinating properties of our bodies to build better implants for when, for example, parts of our body or parts of our skeleton are wearing out as we age.
So, I'm trying out some voice recording here on my iPhone.
So this is Andrea's blog entry for, I guess that was day three yesterday.
It was a-- kind of a tough day to take on all the criticism and-- but it was always-- as always, it was pretty good.
I'm glad I waited for a good night's sleep before I decided to-- to make any comments about it, because yesterday I was not feeling terribly good about myself.
So the orthodontic idea seems to be a little difficult for a lot of people.
Yesterday, I thought about abandoning it, but, in fact, I think I'll go ahead and stick with it for various reasons.
A.
It happens to be something I do know a lot about and there are some intriguing hooks.
I'm hoping that in class today, some people can give me some ideas, if they've thought about it, of what my initial intro or hook can be.
Until then, I probably won't be redoing my pitch until I really think through the best way to do that.
Hi there.
This is Andrea's v-log for the 13th of January in regards to the class yesterday, which I guess would have been class six, where basically, we sat down and did a table read of most of the scripts.
And it was a fairly good day.
I gained an appreciation for how difficult it is to distill all of your ideas down into a small, really good, compact, and compelling story.
And so I thought it was really useful.
I hope some of the other students also thought it was pretty useful.
I know it's really difficult to stand in the line of fire of six, seven, eight other people who are all critiquing you but I think you guys all did a really wonderful job.
And yes, I would want to watch all of your movies.
I think they would be all awesome.
So I actually have a couple of confessions to make.
First of all, I'm feeling much better today.
So I'm much more hopeful about being able to complete the assignment of actually doing some filming in the next few days since I have most of my voice back and I just had to basically sleep all night last night.
But my confession is actually that I've considered dropping or at least reducing down to listener status in this class seven or eight times.
It's been probably one of the most difficult classes I've ever taken and that's saying something, since I was a chemistry major and I'm also in business school.
But I think it highlights that we have a lot of potential psychological blocks and it's really instructive to try to get over these mental blocks.
And one of them, of course, for me is being on video and being photographed.
That's one of them but another one was sort of shifting my mindset from sort of being a product manager, which is what I was prior to coming to school.
I was very business-oriented mindset and getting back into the educational mindset.
And I think that that has changed-- I don't want to say, oh my gosh, it's changed my life.
But in a sense, it's why I decided to step back from business and to sort of recapture my love of science, of making people successful in science, whether it's small children or other adults who are starting their own businesses.
That's really why I came to business school.
And I think this dovetails actually really nicely in with this whole mission.
So I'm glad I haven't dropped yet and I'm really hopeful that I can continue my momentum and basically complete everything.
And I know it's not going to be perfect.
That's another big mental block that sometimes we cling on a lot to perfection.
And I know it's going to be a pretty rough cut by next week but it'll be a lot of fun.
So cheers, everyone.
Yay, breakfast blogging.
Or vlogging, I guess.
Here I am at Clover.
Actually sort of testing out the sound levels, I know it's really always pretty noisy in here.
I didn't know if maybe at some point, I might want to film in here.
Yesterday's class I really enjoyed.
It's a lot of stuff.
I'm really excited about music.
My husband does a lot with weird music in fact, so I wouldn't be surprised if he helps me out a little bit with this.
But we'll see how far we get.
I think I have some pretty good ideas for some of the shots that I want to take.
I'll be meeting my partner, Nathan, coming up here in just a few minutes.
We're hopefully going to grab some breakfast.
So I'm pretty nervous to start shooting.
Don't know how that's quite going to go for me personally.
I'll try to be a really good partner, but we'll see how that goes.
Eating is one of the great pleasures and necessities of life.
And to enjoy everything from energy bars to apples we rely on one part of our bodies to do an important job, our teeth.
Teeth are the hardest substances in our bodies.
They're harder than our bones and they're even harder than iron or steel.
While we chew our teeth experience forces of up to 225 pounds.
So why doesn't our jaw just crumble under all of those forces?
Between your tooth and your jaw bone there's a specialized piece of tissue called the periodontal ligament, or a PDL for short.
The PDL can easily absorb the normal forces that a tooth experiences when we chew, say, an apple, cushioning or protecting our jaw bone from our teeth.
And inside the PDL there are all kinds of cells.
One type called mechanoreceptors sense forces of movement or pressure applied to the tooth.
Teeth sound like they're already perfectly designed but sometimes we really need to force them in a certain direction, like with braces.
As the braces slowly force the teeth to move the PDL gets squeezed in one direction and stretched in the other, kind of like a rubber band.
Here's where it gets interesting.
To make room, the mechanoreceptors in the PDL trigger cells called osteoclasts that actually come in and dissolve part of your jaw to make extra room.
The mechanoreceptors also trigger another kind of cell called an osteoblast, which comes in and builds up part of the jaw bone.
This allows the PDL to get back into its regular cushioning shape, thus holding the tooth securely in position.
So if braces use osteoblasts to physically reposition teeth for cosmetic reasons, what if we want to use them to replace things in our bodies?
Dental implants replace teeth that are damaged or missing to restore chewing function.
And MIT engineers are using the properties of osteoblasts and osteoclasts that are already in our bodies to create a chemical coding for these implants.
Just like in a mouth with braces, this coding helps create natural bone to help lock the implant into place.
Your jaw isn't the only place where these osteoblasts and osteoclasts are altering your bone structure.
In fact, this boney remodeling process is happening throughout your entire body.
And these implants aren't just limited to teeth.
Doctors can replace knees, hips, and even spinal disks.
Right now these implants are designed to have the exact same functionality as the parts that they're replacing but, in the future, scientists may even use implants to improve our brains.
Just like with braces, we could modify our bodies to be more perfect.
Once we've tasted the forbidden fruit of perfection, will we still be human?
Eating is one of the great pleasures and necessities of life.
And to enjoy everything from energy bars to apples, we rely on one part of our bodies to do an important job-- our teeth.
[TEETH CHATTERING] Teeth are the hardest substances in our bodies.
They're harder than our bones.
And they're even harder than iron or steel.
So why doesn't our jaw just crumble under all of those forces?
Between your tooth and your jawbone, there is a specialized piece of tissue called the Periodontal Ligament, or PDL for short.
The PDL can easily absorb the normal forces that a tooth experiences when we chew, say, an apple, cushioning or protecting our jawbone from our teeth.
Teeth sound like they're already perfectly designed.
But sometimes, we really need to force them in a certain direction, like with braces.
As the braces slowly force the teeth to move, the PDL is squeezed in one direction and stretched in the other.
To make room, the mechanoreceptors in the PDL trigger cells called osteoclasts that actually come in and dissolve part of your jaw to make extra room.
The mechanoreceptors also trigger another kind of cell called an osteoblast, which comes in and builds up part of the jawbone.
This allows the PDL to get back into its regular cushioning shape, thus holding the tooth securely in position.
So if braces use osteoblasts to physically reposition teeth for cosmetic reasons, what if we want to use them to replace things in our bodies?
Dental implants replace teeth that are damaged or missing to restore chewing function.
Your jaw isn't the only place where these osteoblasts and osteoclasts are altering your bone structure.
In fact, this bony remodeling process is happening throughout your entire body.
And these implants aren't just limited to teeth.
Doctors can replace knees, hips, even spinal discs.
And MIT engineers are using the properties of osteoblasts and osteoclasts that are already in our bodies to create a chemical coating for these implants.
Just like in a mouth with braces, this coating helps create natural bone to help lock the implant into place.
Right now, these implants are designed to have the exact same functionality as the parts that they're replacing.
But scientists are already developing implants to improve the performance of our bodies and brains.
So at that point, would we be more machine than human?
Science Out Loud.
Every day, we rely on a substance that's harder than iron or steel, our teeth.
[MUSIC PLAYING] So if teeth are harder than steel, they must also be harder than bone.
And if they're harder than bone, then why does your jaw, which is made of bone, not crumble under all that pressure?
There's a bit of tissue called the periodontal ligament, or PDL, around your teeth under your gums.
The PDL is a shock absorber, cushioning your jawbone from all the chewing forces.
OK, so far, so good.
But what if your teeth come in all funky?
Sure, it looks kind of goofy.
But that's not the only reason that somebody might want to fix it.
Funky teeth can interfere with the way you talk and the way you eat.
So how do we fix this?
Well, we basically break our mouths with braces, except it's actually our bodies that do the breaking.
The PDL has these cells called mechanoreceptors.
And when these cells detect a force on your teeth that's too big, like if you accidentally bite into your fork, they signal the brain to stop biting down before you hurt yourself.
Braces tether your teeth, pulling them together or pushing them apart.
Either way, they're applying a steady force to your teeth.
And when mechanoreceptors in the PDL sense this kind of smaller but sustained force, they signal cells called osteoclasts to the area, which spew out acid and proteins to dissolve parts of your jawbone.
Then the mechanoreceptors signal osteoblasts to come.
And those cells deposit minerals that make bone.
Osteoblasts rebuild the jawbone in a new shape that lets the PDL hold teeth in the new position.
So braces basically force your body to dissolve itself and then rebuild itself according to their evil whims.
Sounds barbaric, right?
But your body is actually breaking down and rebuilding bone using osteoclasts and osteoblasts all the time, not just if you have braces.
Bone remodeling is just the way our body grows.
The infant body replaces almost all of its original skeleton by the time it's a year old.
And it happens throughout our entire life.
10% of my bone material is technically new since last year.
We can manipulate the bone remodeling process to not only get straighter teeth, but also to treat diseases, like osteoporosis, which make your bones very brittle.
By keeping overactive osteoclasts from dissolving the bone so much, or by boosting osteoblasts to produce more bone, drugs can prevent bones in those patients from breaking so easily.
People with severe bone injury have to rely on bone transplants, where they take bone from other parts of their body and move it to the damaged area, which is sometimes not even possible and is always painful.
So instead, Paula Hammond's group at MIT has created a new material-- --that very slowly releases proteins.
And these proteins cause osteoblasts to go right to the site where the injury happened and generate new bone.
Now, this was a really big deal for us, because it's really hard to generate something that very slowly releases the protein.
Most of the time, the protein comes out very rapidly and gets swept away in the body, so that it no longer has any effect.
It's easy to write braces off as a form of medieval torture.
But it's kind of amazing that this mouth torture actually works.
And the technology that makes it possible is not just in your braces, it's in your bones.
Hi, my name is Andrea, and thank you for watching this episode of Science Out Loud.
If you liked this episode, why don't you check out some of these other videos.
And visit our website.
And visit our website.
[LAUGHING] They're separate things?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Assessment for this class was probably the hardest thing for me, at least.
How to assign a fair grade to students, because at the end of the day, they're a grade for this.
And it wasn't going to be an A for effort type thing.
It was actually really hard because anything that requires creative effort-- I talked to you about this for a long time-- you don't want to crap on people's creative efforts, even if it's just not very good
It's very hard
Jamie and I talked a lot about this.
But initially, the rubric was very plastic elements based.
Like, is this well lit, can you hear the audio OK?
And we just kind of scrapped that, and really went with-- there were four sort of principles to the class that I talked about on the first day, which was spark, does it foster curiosity, and that was like a big theme that went through every single assignment, and how it manifested change assignment to assignment.
You can see on the rubric.
Kill your darlings.
Like, is it concise enough.
There are a couple others.
So that's how we approached grading.
And we assigned them raw numerical scores, and we didn't have like, an A is 20, a B is 10, or 15.
It was in progress was from 0 to 5, and steps moving forward was like five to 10, and then mastery was like 10 to 15, for instance.
So they would get numerical grade feedback.
We gave a lot of weight to their daily reflections so that not everyone in the class would end up with a
I have a question.
Do you think grading should have been A for effort?
Because in real life, and I didn't tell this to you when we were planning the course, and I'm sorry.
But in real life, it almost is A for effort.
If I show up to do voice over and I'm late and I'm sloppy, and I don't do a very good job, I'm not going to get hired again.
And part of that is my technical ability to do voice over.
And part of it is showing up 15 minutes early, being professional, all that sort of stuff, which does kind of seem like A for effort.
I don't know.
What do you think?
Well.
So I've had to think about this a lot for the course I teach at MIT.
And it's a huge, huge issue because, so who should get the better grade?
The person who has a huge amount of talent and experience as a filmmaker and video maker, sluffs it, just like lays back throughout the whole course, but does a really nice piece at the end.
Or the person who never has done it before, doesn't necessarily have a whole lot on the ball, when it comes to media production, but just works and works and works and comes out with something that's kind of mediocre.
Just because by the nature of it having never done it before.
That's really the issue.
When you're dealing with non-professionals-- because I completely agree with him, it's the Woody Allen thing right?
Like most of life is showing up.
And it's totally true.
But when you're dealing with this huge, huge learning curve, it becomes very, very tricky.
So what we do, we don't do quite A for effort.
But we do kind of allow an out if your final project, which is a big part of your grade, goes completely haywire and just falls apart on you, you can write a fairly substantial paper that describes what your intents were or what your thought process was, what went wrong, why it went wrong, what you would have done differently.
And so, essentially, you do have a way supplementing or making up for this innate problem you had, which was that you didn't know what the hell you we're doing.
Which is a problem
It's fairy innate for me
For all of us.
Well, two of the course values were challenge, and a lot of that manifested in the rubric in the final project and the graded assignments, as is their improvement from the previous iteration?
Is this person going outside of their comfort zone?
And then thoughtfulness.
Can you tell that this person has integrated some of the things we talked about in class, whether it's through their blog posts, or just like office hour reflections, things like that.
So that was built into the final grades, but then I would also say that the graded sort of quality projects, and the daily reflections were weighted pretty equally so that if you had a student, which we did, who had the skills, but didn't do the daily blogs, their grade would suffer a little bit more.
Just as much as someone who did the daily blogs, and just could not host for instance.
In the end it worked out by doing it that way.
All the students who I thought deserved great grades, because I knew they worked really hard, got good grades.
And all the students who were very technically skilled still got decent grades, but it wasn't like they got 100 or anything.
It's always a tough challenge, though
My math teacher used to say-- we'd be like, how are you going to grade this test.
He'd be like, well I'm going to blindfold myself, turn around and throw darts behind my back.
And we were like, good.
That's good
I just which I could have written a paper in chemistry class in college to make up for the--
My experiment went south.
And here's what happened
It's hard because MIT students, they want to know what do I need to do to--
Exactly.
People are very focused on achievement and doing well
Which is why I was very conscious of by creating as specific of a rubric as I can
That makes lot of sense, especially as you said before, because this is a form of art, and there is no right or wrong answer, so how do you--
But you can tell someone, look you didn't change anything from your first draft.
So you're going to get docked for points on challenge, as the rubric says.
Or you straight up didn't do a blog reflection so you're getting a zero on that section.
But at least I do think that there has to be something tangible to go back to that you can't just sort of arbitrarily assign a grade.
I will say that, at the end of the day, an assessment, a great assessment, should be a reflection.
Or should be there to help students improve.
And I do think that the value in assessment and feedback came from just sort of one on one, when you all worked with the students directly.
And that's something that's not captured on the OpenCourseWare class.
But I would stay with Jamie and our TA Siri, for at least two hours every day after class, just talking to students.
And you work with them.
And when you and John came to do the workshop, you'd just have direct interaction with the students.
I really think that's where the most value came from.
I don't know how you all feel about that
I thought that workshop was great, by the way.
I didn't get a chance to tell you, it was really nice
I mean, I thought it was just great to interact with the students.
I mean I wasn't immediately sure how engaged they were, but I had some sense of it when I would actually just sort of sit down with them and they would directly ask me questions and there was that sort of that direct engagement.
And I guess one would question how this course could be done electronically, or via some kind of distance learning, or that kind of thing.
Because that seemed invaluable to me.
That kind of personal contact
If you are teacher in Missouri, for instance, how can you teach this course without a Chris, without a George, and without a Josh
I mean, there are versions of us in Missouri
There are
I have an alter ego in every state
You said something nice about my home state.
It's on camera, too.
But really, again, you can watch the OpenCourseWare stuff to get a sense of just, I mean, they're just fundamental facts, I guess, that you should become aware of.
But I gave a midpoint survey to the students of what lectures have you found most helpful, and they were all like, all of them.
Like, Chris' was really helpful because I got his perspective as a filmmaker, like as someone who was, and still is, in the industry.
And then working with Planet Nutshell, they were like, oh I had no idea how to storyboard.
And it was waving more helpful than just watching a video of someone do it.
We were actually interacting with them.
Your hosting workshop, like how do you give hosting feedback not in real time, not live.
I don't know
I just remember sitting down with a couple students and I think students were running their ideas by both of us.
Maybe me individually, and John, just giving sort of direct creative feedback.
Well, I think that could work, but have you thought about doing this?
And then that would spur them to further sort of creative thinking.
That kind of interaction, that relationship, seemed invaluable
And the one on one interaction.
I've thought about that too.
I mean maybe the equivalent of crowd sourcing or something.
But most of the valuable experiences I had as a student, and that I think I've had with students as a teacher, are really not even in the classroom, but one on one
Yeah, I mean, most of the experiences I remember most powerfully are being at office hours and having a teacher asks me like, so do you really think you're going to be an English major?
And particularly in media or video, it's like you have this work, it's a piece of crap, and someone is going to sit there and help you turn it into something else.
And just to learn what can be done to a script, to something that's been shot and not edited the way it could be.
It's a really powerful experience
Yeah definitely.
And so I wonder, sorry to interrupt you, I just I just wonder if the balance-- I don't have a complete picture of how the course was balanced, because I wasn't there the whole time, but I wonder if there could be more opportunity for that kind of a one on one interaction
It's very short.
That's the challenge
Yeah
Yeah
It's such a short class
Like some kind of critique or workshop that maybe we all attended and gave thoughts on?
And for the students who worked with us in production week, that was a lot of one on one.
A lot of one on one, or like, three on one, in that case.
But there again, for production week it was like you don't spend two hours like, here here's why your hosting was not optimal.
You just tell them what to do to fix it quickly so you can get the next shot.
But that still is valuable because you still see how a professional does it.
You're there and you're doing it.
You're not just sitting there watching.
Without you, the show doesn't run.
Without the student, the show doesn't run.
So they have to do it
A couple of the students who did Science Out Loud, they waited to write their final blog reflection until after they shot.
So they were like, I wanted to wait to see how different it would be from the class.
And even in that one day, I learned so much.
Like, more than I did in the two weeks of the class.
Just like being in that production setting.
Sorry go ahead
That was it
Something I wanted to mention, again, this kind of thing isn't scalable.
We can't spend one on one time with 20 students.
It was manageable because there were only seven or eight of them at any given time, thank goodness.
Initially were like, yeah we're going to blow it up, make it like 24 students.
Oh my gosh, no.
It was a struggle to keep up even though our teaching team is huge, just because it is such an intensive process to work with them one on one.
But what I did want to make use of their peer-to-peer learning and support.
And somehow, I don't know exactly know how this got fostered, but we really had a nice class environment.
The students were very supportive of each other, which was really nice, maybe because it was such a small class.
But during the second week, they would go off in groups to shoot each other, and they would give advice to each other.
Which was awesome.
I would overhear are some really great directorial advice, but I would also overhear some advice that would be like, no.
Don't make them have that intonation.
I wouldn't say anything because I think it's important for their learning experience.
But that's what I mean by there's only so much you can do with people in the same sort of learning sphere teaching each other
It's a very short-- I mean so much of what this class is like, is characterized by the fact that it's so short.
Because the cohort thing is so important.
And, I mean, as someone who, I'll confess, went to art school.
I know, sorry MIT people.
So I went to NYU, the graduate film program, the three year program.
So the first year of that program, is kind of like the first couple of lectures in your class.
It's all class based, you feel like your head is exploding with all this stuff.
And you're hanging on your professor's every word, and it's like kind of transformation.
After that, it's kind of pointless to go to class.
It doesn't really matter what happens in the class.
It's all about the cohort, and it's all about what you do outside of class, and the help and support you give each other as you make these iterative projects.
So it kind of takes over and becomes the thing, the only thing.
The relationship of the students with each other, critiquing each other's work, working on each others projects.
The rest of it, once you kind of have the basics, the basic conceptual framework--
It's like learn by doing
Yeah, it's all learn by doing after that
The problem is because it's a two week, or three week, class.
Or even a semester, or even a year.
At MIT, I mean, these students are nowhere near experts.
Not saying that we are, right, there's only so much that they can be aware of with each other, that it was a weird balance to strike.
So hi, I'm Elizabeth Choe.
By day, I run the MIT K-12 videos program in the Office of Digital Learning at MIT.
And I was the lead instructor/creator 20.219
Hi, I'm Joshua Gunn, and I'm the co-founder of Planet Nutshell a Cambridge, Massachusetts, based animation studio.
And my role in the course was simply to do a workshop on storyboarding, as well as a little instruction on animation in the context of producing educational media
My name is George Zaidan I am a freelance science video host and writer and producer.
And for 20.219 I did a workshop on writing and hosting
I'm Chris Boebel.
I'm media development director for MIT's Office of Digital Learning and for the course I was basically a guest lecturer.
I came in for a day or two to talk about visual storytelling
So the way this class stated, I would say, is that George and I were working on Science Out Loud, which is this educational web series that's hosted by MIT students.
And it sort of noticed that there was this big need to help students not only know how to articulate themselves on camera-- articulate well on camera-- but just understand digital media in general.
And that's a conversation that Chris and I have had since I was your student in undergrad basically
Not that many years ago
So, actually not really.
Yeah, but with Science Out Loud we would do these, like, crash courses on scripting and writing these short videos, but at the end of the day like we would still intervene quite a bit to make the videos good.
And at the end of the day, the priority was on making a great video, and not necessarily on the students' learning experience.
So, I mean, this was a space for them to develop these skills sort of lower states, and not have to worry about creating something that was going to be on a web series at the end
The class was supposed to be lower stakes
Yeah
Because we had-- Science Out Loud was a lot of pressure.
We were shooting long days.
If they felt like they didn't get to take right, they were nervous
Or, if the script just wasn't good by the time we had to shoot, we would just fix it
And one of things you and I've talked about a ton over the last couple of years is that tension between creating a consistently good product and teaching.
Because if you're in a teaching environment, you're going to fail over and over and over again, which does not lead to a particularly, consistently good product, which is what you're going for with Science Out Loud.
So this class is kind of a way, I think, to find a space in between where you can actually have a little bit of a chance to experiment and learn by doing and fail, bluntly.
But also not actually impact your show, which needs to be consistently good
Was there any desire to sort to create an awareness for students of other sort of creative opportunities outside of the science and math, or their current studies?
I mean, I think it's a really interesting way this course is sort of offered to students who are engaged in stuff that's really quite different
I mean, interesting because MIT has this communication requirement, but it's not really taken seriously by students, because it's like, I need to take a class where I'm required to write 20 pages of paper.
And you and I were talking about one at the bioengineering communication fellows, and he said something like, I'm doing this program because I don't see a difference between science and science communication.
But I think like that was inadvertently was what this class is about.
Because it's not really about how to host a science show, or how to write a five minute script.
I don't care if they never make a five minute video again, but hopefully the skills that they learned and picked up were things that they need to have, regardless of what they end up going to.
Just the ability to articulate passionately and clearly some very technical topics is not something you get trained on enough at
I think I think the MIT communication requirement is an interesting situation, too, because it is all focused on writing, and writing papers, not even just writing scripts, writing papers.
And, for instance, the course that I teach that you took, we initially had trouble getting it listed as fulfilling the communication requirement, even though students consistently said, well this is all this course is about, communication.
Because the writing requirement was just a little bit shy of the 20 page, or whatever it is, requirement.
So there's a whole other kind of communication, which is, as you say, clear conceptualization, visualization, organization, which is related to writing, and writing kind of traditional papers, but it's in 2015.
It's not just writing journal papers, or kind of really more traditional pieces of text
Actually, before I put the class together, the first thing I did was just survey all of the medial making classes on campus.
So anything related to video production or like script production.
And there are actually a lot of classes on campus that are about how do you edit, how do you produce video?
But there wasn't one on really hosting, actually.
And being on camera, and being comfortable talking in front of people was something that we found to be very important through Science Out Loud and something that most students were the most comfortable with, would you say?
Yeah I mean they didn't even know they were uncomfortable with it.
They'd just be like, oh yeah, no problem, I'll just talk, and then they get in front of a camera and it's like Jack Donaghy in that episode of 30 Rock, where he can't-- you know.
You don't understand why your body doesn't quite work the way you thought it was, or your voice is a little too high and you get nervous, and then you screw up, and then you screw up again.
And then if they get and then they realize how hard it actually is to stand in front of a camera and say things.
So yeah
It's not just about standing in front of a camera, though
Any audience, right?
I feel like I wanted them to learn to be self aware of who they were as a presenter, you know.
Like, we weren't trying to make them become a persona.
Like we called it becoming the next Bill Nye, but it really wasn't.
That was like the first thing I talked about was that we're not actually turning you into Bill Nye.
That wherever you go, whether it's a job interview, or you're giving a Ted Talk or formal presentation, I mean, you need all these skills and you can't assume some sort of platonic personality When you do these things
It's not acting, it's just being themselves
But just a clear, polished, version of yourself
Kind of close to what the real thing is, but maybe not entirely there yet
Awareness of an audience.
So on my way back from class, I had time to reflect on today's lessons, I guess.
And to demonstrate how you can vlog about it instead of writing a written post and to practice being on camera, I decided to make a vlog instead.
Loosely edited, because I still forget what I'm going to say sometimes, but I'll keep it mostly raw footage.
So the first main thing I was thinking about after talking with Elizabeth and a couple of you after class and re-reading the comments on the annotated articles, especially those on the Hank Green one, is that I was thinking about how online video is a thing now.
And people don't necessarily always understand how much of a thing online video is.
A lot of you in the class right now seem to be coming from the perspective of scientists learning how to communicate better.
And video is one method of communicating ideas to a larger population.
And it's great, and it's wonderful, and I'm clearly very in favor of using videos to spread information.
A lot of people who are getting into online video in the more modern day are people who have worked on productions, like TVs and film, and come from a different medium background.
And they're trying to transition over to online video and taking practices with them from those mediums, like how to shoot correctly, how to edit, how to compose a script, how to write a script, how to edit a script, how to engage an audience from those other mediums, which is all fine and good, because it's helping online video evolve.
But then there are people like myself, who watched online video, and grew up with it, and started watching YouTube and the cheesy vloggers as soon as they started.
And my editing practices-- the first time I ever filmed and edited a vlog, it was filled with all these jump cuts and pauses for comedic timing just because this is the way that people like Charlie McDonald, like the original Vlog Brothers channels, would frame and put together their videos.
You would never see a movie edited with jump cuts like these or a professional documentary like this, but this editing style has evolved because of online video, which is kind of cool and kind of crazy.
It just become a way for people to start thinking about things like editing and how to frame yourself within whatever-- a webcam or a slightly high quality camera can hold.
And thinking about your background as your room, as something that demonstrates who you are-- it just so happens that my desk faces opposite from my wall posters.
And because people are coming from all these different perspectives, it makes establishing a best practices type thing even harder.
Because online video, maybe five years ago, no one would think to use it for education, specifically for education, and channels like Sci Show and Minute Physics and the Brain Scoop wouldn't exist.
But now they're being watched and things like that.
And so it's constantly evolving.
People are constantly trying to find ways to make it more interesting.
And so it's hard to try and describe how to do something when everyone's making it up as they go.
The second thing I wanted to talk about was getting overwhelmed at the beginning of the class.
This class can seem scary at first, especially because we're throwing a lot of new things at you.
A lot of you have not uploaded videos before, or are relatively unfamiliar with video making and relatively unfamiliar with platforms like Tumblr and things like that.
And all of a sudden, you're tossing around ideas, and it's really scary to pick an idea that you could conceivably commit to for an entire month or three weeks of your life developing and fleshing out and building upon.
And especially today, it's fun to bounce around ideas off of sixth graders, to have conversations, to see what excites people, and excites your peers, and excites them.
But ultimately, you have to start somewhere, to pick an idea, run with it, make something.
And even if that thing is kind of crappy-- even if your first draft of your script is kind of crappy, as scripts tend to be the first time you write one ever, and that's OK. Because it's all about revision.
That's one of the wonderful things about digital media, is that you can always re-record, you can always re-dub, you can always rewrite and re-film.
So I think the biggest thing going in tomorrow is just getting something you can work with, getting ideas concretely down on paper, which sometimes can be a really hard step in itself.
Because you want your idea to be perfect, and you know what good content looks like.
But it's very different knowing what good content looks like and getting to that level where you can produce good content.
And that gap is really frustrating sometimes.
Yeah.
That's all I have to say.
So today's class reminded me of two main points.
For one, the hourglass video on storytelling reminded me of a separate hourglass video on the creative process that I really like watching when I am in a creative slump or I feel like what I'm doing isn't good enough, which is how I feel like you guys might feel about your scripts at some point or your videos at some point during this process.
I'll put a link to it below, but basically it's the idea that you want to make videos, and you can recognize what is good within a video.
So you have this good taste.
But there's a large gap.
There's a large learning gap and a lot of trial and error and being able to produce content that you're proud of.
And so you're going to go through all these stages of making essentially crap.
And knowing, because of your taste in videos, that what you're producing isn't what was in your head, isn't what the ideal video should look like.
And that can be really frustrating.
But no one instantly jumped to success.
And it's hard to remember that sometimes.
The second thing is a lot of George's and Elizabeth's talk about hosting reminded me of a lot of theater exercises that I've experienced.
I don't act, but I've stage managed.
I've produced.
I've done tech, sets, light, sound.
Basically anything behind the scenes in theater that's possible, I have dabbled in at some point.
And the things that they were getting you to do, where in saying the lines over and over again, thinking about your intonation, is exactly what actors do in rehearsals when they're reading scripts to put on a play.
And so one thing I thought might be interesting to watch, or I went back and watched them partially because I like Shakespeare and partially because I like David Tennant, is a bunch of Hamlet's soliloquies.
And there's a YouTube playlist of the to be or not to be speech, pretty famous, with six different versions done by six different professional actors.
And they all have slightly different takes on the lines.
And they still imbue meaning.
And the meaning of the monologue or the soliloquy changes depending on their inflections on the words.
And so it's really interesting to think of yourself as not only just hosts, not only just creators of these videos, but also as actors of sorts.
And this might bring around reservations.
I know it would for me because it's like, I'm not an actor.
I prefer to be behind the scenes.
I prefer my face not to be on camera.
But if you think of yourself as a developing actor, as someone who wants to practice these things, it's not a bad idea to do so.
For example, memorizing lines helps.
Once actors have their lines memorized, it means that they can focus on the nuances of the performance.
They can focus on the emotions that they want to convey.
They can focus on their blocking.
They can focus on all different numbers of things.
So that carries from video to stage acting.
If you know the words that you're going to say, you can start thinking about what those words really mean, which is the point that George was making earlier.
Yeah, just thinking about the different emotions you can portray and trying all the different ways you can say a similar sentence and imbue different emotions into it.
And like the exercises they were doing today where you yell it, you say it in a weird accent, and then what we're ultimately trying to do is different than the goal of an actor.
We're not trying to become someone else.
We're trying to figure out how to become ourselves on screen instead of acting like someone else.
And so by doing all these extravagant things, and by doing all these acting exercises, you automatically revert back to your natural, which is, I guess, what we were saying.
So I don't know if I have anything new to add.
But I don't know.
If you're curious about acting exercises, feel free to ask me.
I've not personally done a lot, but I have seen actors go from reading a text aloud like popcorn reading in third grade to being in a stage production.
And I'm no professional director, but I can definitely share some tips that I at least have observed.
Yeah, this video has gone on longer than I thought it would, but that's all I have to say.
Let me know if you need any help with anything, as always.
So I really appreciated today's lecture and meeting the guys from Planet Nutshell, partially because I'm really a visual learner.
And historically that's been true-- if there's an image or an animation or something associated with a concept, then I remember it a lot better.
But I'm also really a visual explainer of things.
I took a lot of art classes growing up.
I used to draw for fun all the time.
And if there is something that I can express by drawing in addition to words, then it usually comes out better than if I'm just rambling on.
But that's a personal thing to work on as well.
Regarding animation-y things, I have very, very, very little experience with Flash.
I've watched tutorials.
I haven't experimented it with myself.
But I would be willing to learn if people would like to learn alongside me or had an interest in utilizing it.
Because then I could learn it and troubleshoot if you guys have questions.
But on the flip side of that, I'm currently learning how to do animation using Adobe software from one of the people I'm working with on MITx, the molecular biology course.
And that's just a really simple workflow from creating vector art in Adobe Illustrator, importing into Adobe After Effects for the animation part, and then importing that into Premiere, which is where I do all my video editing as opposed to importing back in After Effects, for example.
So if you guys are curious about very, very simple things, anything that can be drawn with vectors or solid shapes and moving those around the screen, I can definitely show you what I know about that and how to apply it to your videos.
Normally, I use it for-- if you need to animate a protein moving, or you want text in the shape of your handwriting to appear on the screen as though you were writing it, things like that, which can be cool.
For me, they're a lot more for the explaining type videos, like what is the cell cycle as opposed to the videos that you guys are making, the Science Out Loud type videos, the style of videos that we're making in class.
But you may find a use for it, and I'd be willing to help.
The second thing that I wanted to talk about was my own bad habits regarding storyboarding.
Even though I'm a very visual person, a lot of my intro into the video making world was through vlogs-- like daily vlogs or channels where people just talk to a camera.
And it may be scripted, but it's ultimately them talking to a camera.
And these did not have varied backgrounds.
Sometimes, they had varied shots for comedic timing or had little sketches within it.
But I never thought about storyboarding when I first made videos.
And so it's my really bad habit to leave it out.
It would be probably the last thing on my mind when it comes to making a video, which is a really crappy habit when you're trying to make a video like this where it can be really complicated with a lot of visuals.
And so I'm trying to break that right now, especially with my other video projects where a lot of them are animation.
And it's so, so important to have an idea of what you're going to draw, what's going to be on screen at every single moment when it's not something as simple as one of these daily reflections where I'm just talking to a camera.
I have some talking points written down, but mostly, it's free form.
And people are watching it, if you watch it, for the content, the verbal content and the haphazardness of it as opposed to the produced video.
So I don't know.
It was really humbling to see people who draw so well for a living, and storyboard and stuff for a living, because I'm interested in the media industry, and it's such a big part of it.
But I'm also really lazy when it comes to this part, this really critical part of making produced media as opposed to something casual like this.
So everyone has a lot to learn, including myself, and I'm really happy I'm in this class.
That's all.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so everyone is here for 20.219, correct, Becoming the Next Bill Nye?
No one's in the wrong room?
All right.
OK, well, welcome to the class.
I'm Elizabeth, I run the K12 videos program.
Maybe we can do a quick intro with the staff over here.
So Jamie, you want to--
Hi, I'm Jaime Goldstein.
I run the communication lab in Biological Engineering, and I oversee it in Nuclear Science and Engineering as well
I'm Ceri.
I am the TA for this class.
I'm a junior in Comparative Media Studies and Biology
I'm Josh.
I run an animation studio here in Cambridge called Planet Nutshell
I'm Chris Boebel.
I'm Media Development Director at MITx, and I also teach a class here at MIT called DV Lab on documentary, and I'll be guest lecturing for a couple of sessions
Come on in, take a seat.
So just logistics stuff.
All the information about this class is on our Tumblr, which I'll email it out to you guys, but it's mit219.tumblr.com.
And you'll find the class syllabus, all the links, all the videos that I'll be showing you during class, and basically everything you need to know about just homework assignments and things like that.
So we're just going to go ahead and dive right in.
This class is about writing, hosting, and producing video, short-form video, but in the context of this sort of scary, amorphous blob that people talk about a lot, the problems in science, technology, and engineering and math education.
And the specific problems that we're going to be trying to be thinking about in the context of making these videos are these three problems, which is that the culture that we have right now, especially with video and science, is that it kind of implies that the door to science and engineering, whether or not you want to be a scientist or whether or not you want to study it, is only open to certain people.
We also have this issue of maintaining a love of lifelong learning.
Josh I were just talking about the dark ages, when you quit playing with LEGOs.
But kids are very curious creatures, and somehow our educational system squelches that curiosity.
And science and engineering are these topics that I'm sure you guys are very excited about.
I'm very excited about it too.
But somehow it becomes this sort of boring subject, and especially in high school, people think of chemistry as their least favorite class.
It was my least favorite class in high school.
And then you have this civic responsibility as scientists and engineers to make sure that you're promoting STEM literacy among the public.
We don't want everyone in our culture to become a scientist or an engineer.
That would be terrible, it would be super boring.
It wouldn't really be helping our society at all.
But at the same time, science and engineering affects your daily life.
Whether or not you're an artist or you're an author, you're voting on issues that require a certain knowledge of science and engineering.
And as scientists and engineers, we need to be making sure that we're promoting a basic level of literacy among the voting public.
So it's a practical thing for us, as scientists and engineers.
It's also sharing that excitement and love of learning with other people that maybe they lost in a boring class or something like that.
So the idea of this class is to bring together the best practices in education, and some of these problems that I'm talking about with STEM education, and bringing it together with some of the tools that we have in entertainment.
So how can we use video, how can we leverage video to address that big, amorphous blob.
So we're going to try to occupy this middle space.
But the problem is that the best practices in this middle space have not really been established.
Part of that is because there aren't really experts in edutainment, necessarily.
It's not a super academic field.
It's also pretty new.
The advent of science entertainment on the web, especially now it's exploding, but it's all very new, so it's hard to establish the best practices.
And on top of that, the best practices in education and the best practices in entertainment don't always align.
A lot of times they conflict with each other.
So a lot of this class will be piecing together what's great about these two disciplines, and it'll also be sort of discovering on our own what are the pieces that we want to take and implement ourselves.
At this point, I wanted you guys to think about this question, partially to inform us and help us help you guys over the course of this month.
But I wanted to get a sense of why you all signed up for this class.
Maybe it was because you needed six extra units to graduate, which is totally fine.
And also what you wanted to get out of this class.
If you're coming into the class and you want to learn how to record video, we want to know if that's your objective.
If you came in because you thought it sounded interesting or anything like that, we want to know, too.
So you guys can think about it, maybe share.
If you don't want to share right now, that's totally fine.
But I will mention this at the end of class, but every day you're going to be writing blog posts and daily reflections, so hopefully this is something that you can incorporate into your reflection today
Elizabeth, why don't you start?
Why are you here?
Why am I here?
Well, I'm going to talk about why I'm here after this, so I don't want to spoil it.
But Jaime, why are you here?
Oh, OK. Really good question.
I run the communication lab in Biological Engineering, and that was it.
I oversee Nuclear Science as well.
And our goal is really to help scientists learn to communicate more effectively.
And that's what I do now.
I mostly train graduate students to talk to each other about their science, and to undergraduates, about how do you tell people about what you're doing.
And when Elizabeth approached me-- she was an MBE originally, so she knew about our lab.
This was a perfect match for me personally, this course, because prior to my life here I taught middle school for many years, among the many other things.
I was a magazine editor for a little while.
So this to me is a perfect intersection of things that I'm passionate about, because your videos are targeted, that you're going to be making for that middle school population.
And that's one that I know very well and that I care about a lot.
So I'm looking forward to sharing with you what I know about the work that I do now blended with the work that I used to do.
So I'm excited to tap into your creativity
Anyone else want to share why they're here?
No wrong answers
My name's Paul Folino.
I came here to, kind of on what you were saying, just try to be able to convey technical concepts in a simple way, so I can explain to someone, whether it's one of my friends or someone like my niece, what I do
Awesome.
Anyone else?
You will have to stay it in your blog, though
I want to hear from everyone
I'll probably go first
Sure
And what's your name?
Sorry
My name is Joshua
Joshua
You can call me Josh.
I come from Singapore.
So we're here on a really short exchange for about a month.
We come from Singapore University of Technology and Design.
It's a university that was started in collaboration with MIT.
Well, I came to this class because we study OCW a lot, and that's because we are probably the first [?
special ?]
university and most of our curriculum comes from MIT.
And honestly, I feel like close to 25% of my education comes from online.
And I'm very comfortable with it, but-- my mom is a chemistry teacher, and she tells me that students don't necessarily like stuff like Khan Academy, because it might look too boring for them.
And that's kind of my curiosity right now, how do I convey something that people might not find really interesting but very important for them, and make it very interactive
It's Yuliya, right?
Mm-hm.
So I was very interested in education in high school, and I noticed that kids don't like STEM subjects, generally.
And so I just wanted to, as a career, pursue promotion of STEM subjects.
That's one of the things I'd like to do
Yes
Hi, I'm Jonathan
Did you say John?
Josh?
Jonathan
John
John is fine, too.
So I noticed there were a number of videos on YouTube popping up, a number of educational videos in range from exciting to pretty boring.
And I think this is a very powerful medium to reach kids who wouldn't normally be interested in science, or math.
And exposure, and how you receive and where you get information, usually tends to tip the scales as far as who's into something and who's not.
So I think this is a really cool project.
And I like to be on the other side of the camera as well
I guess I'll go next.
My name is Andrea.
I'm a Sloan Fellow here at MIT.
So I'm significantly older than maybe everybody.
What I've been doing for the past 10 years of my life has been working in regulatory, so doing medical device submissions to the FDA.
So it's a lot of communicating highly technical information to people who aren't exactly laypeople, but they're not going to be as familiar with the topic as, say, I am.
So that's what I've been doing.
And one of the things that I'd really like to introduce is using video as another tool to convey this information
Hi, I'm Nathan.
I think why I'm taking this class is [INAUDIBLE] spread scientific information, too.
At least for me, I think, not necessarily people who adore the science, but just people so they can be scientifically literate now.
I don't know
Hi, I'm Kenneth, from SUTD as well.
Back when I first graduated from junior college, I actually went to have a teaching internship for a period in Singapore, and that's when I realized that most students in Singapore do not really like blackboard or whiteboard teaching.
I realized that a little-- like halfway into my first week of doing the internship, so that's when I revised everything that I had prepared, started creating multimedia tools and animations.
It's been like two, three years since then.
But the whole idea of using multimedia as well as video tools has still stuck around.
I still regularly, when I'm bored and I don't feel like doing my homework, I'll go on YouTube and watch all these random videos, because at least I learn something; not relevant, but something.
So this has always been an interest
Does anyone have any particular goals out of this month or something that you want to get out of taking this class?
That's OK. We'll give you goals, don't worry.
All right, well, I won't torture you any further
I'll just mention that I'm excited about the class.
I run an animation studio, as I said, and we do this work every day.
We're engaged in helping biotechs and technology companies and educators explain very complex things, and sometimes not-so-complex things, to very young kids.
But I certainly don't have the market cornered on how to do this.
And as you said, this isn't really an academic field, and there's no grand unified theory of how to do this most effectively.
I think a lot of things are out there right now, but I'm just excited to be engaged in a conversation about techniques and strategies for doing this, because they'll help me in my own work, and help me understand the perspective that you guys have as well.
So thanks
Great.
As I said in the beginning, I run the K12 videos program, which is a fairly new initiative out of the Office of Digital Learning here.
So it's the same group that oversees MITx and OpenCourseWare.
I produce a series called Science Out Loud, which is essentially trying to mix education and entertainment, having students like you guys be the face of science and the face of engineering.
And to get people not only educated and informed, but also inspired as well.
So I've had an interest in the intersection of all that stuff, I guess, since I took Chris' class.
When I was an undergrad I took his DV Lab class.
But I think it's a very fascinating area.
And like Josh said, it's this market that I don't think anyone really has cornered or really-- no one really has a good grasp on what the best way to practice in it is.
So I think it's a really interesting problem, especially for people at MIT to think about, as well.
As some of you guys said, videos have exploded in the last decade or so.
These are all science YouTube channels that have over a million subscribers each.
And it's not just video exploding in a certain area.
You have YouTube entertainment slash educational shows, and then you have things that are very lucrative in our pop culture.
Cosmos was raking in like five million viewers a week.
Which isn't a whole lot compared to some of the other primetime network shows, but thinking about the fact that a science-based show aired on the FOX Network during a primetime slot and was raking in those views is pretty amazing.
And then you have movies like Interstellar, which are addressing a lot of real science topics, and is also making like $50 million during opening weekend.
And then you have things like edX and OpenCourseWare, and all these open education resources that are video-based, and are also reaching this incredible audience and have a huge group of people investing in it.
And these are all very, very different styles of video, but they're all unified by this theme that they're using video and they're accumulating a huge database of users.
So something must be happening with video that's making it compelling for people to use to learn and to entertain.
What can video do?
What is it about video that makes it so compelling to use as a tool?
Educationally, you have stuff like OpenCourseWare that's really great, because people can pause the video whenever they want, they can rewind and rewatch an explanation.
And it's also super scalable, so it means that you don't have to be physically in Cambridge to hear Eric Lander talk about genetics in the 7.012 class.
You can be anywhere in the world.
You can be in Singapore.
You don't have to be restricted to the confines of room 26-100, right?
It's a very scalable tool.
And then in terms of movies, I mean the thing about movies is we have this culture where people say that the attention span for video is like less than three minutes, yet you and I-- or at least I spend $10 to sit for two, three hours in a dark room.
And it's because movies are these completely immersive experiences that give us a chance to sort of escape into this other world.
And you don't see people paying $10 to sit in a room to listen to a radio podcast, right?
There's something about the integration of the visual with the storytelling and the acting that totally immerses you in this experience, and that's a very, very powerful element that video can do.
I also think it's a window into a world that you don't necessarily have access to.
So whether it's the world of 26-100 where Eric Lander is teaching or if it's the world of Middle Earth-- I just went and saw Hobbit III-- you get this chance to look into this world that you don't have access to, and that's what video can do.
And then, finally, especially in nonfiction video, there's this idea of the trusted guide, which is a term that PBS Kids use to think about new shows that they pitch.
They think about the person who's going to be the host, that it's more than just the person who's going to stand and transfer information to you.
It's someone who's relatable, somebody who's a role model, someone who you really trust, that you end up really growing up with people like Bill Nye.
I don't know if you guys have shows or show hosts that you remember growing up as kids.
Mine was Jeff Corwin.
I'm like obsessed with Jeff Corwin.
Does anyone have a figure that they really loved watching growing up?
Did anyone watch the old Cosmos with Carl Sagan?
Yeah?
I feel like that's a big one for people here.
So the trusted guide is a really powerful thing that video can leverage.
And I think that's a really big reason why we're here, at least the reason why I'm here, is because when you look at the landscape of trusted guides right now on especially science and technical TV, and not just TV but Hank Green hosts SciShow on YouTube.
And the landscape of trusted guide looks very homogeneous.
And I'm not saying that it's a bad thing to be a guy at all, but I do think that we have this opportunity with video to showcase who the scientists and engineers really are.
And this landscape is building a group of advocates who are going to be sort of being the ambassadors to the public about what you're learning about and what you're studying.
And we have this opportunity to create a landscape that's just as diverse as the group of people it's trying to represent.
And it's an opportunity that I think hasn't necessarily been capitalized upon.
Maybe it's because there are political things and it's hard to invest in a figure that looks different than the science hosts that we're used to seeing.
But we have a chance here at MIT where the people are so diverse, not just in ethnicity and gender, but in background and personalities and interests.
We have this huge landscape here that we can tap into and start contributing to a picture that looks a little bit more like what science and engineering really is.
And the other thing is literacy in digital media, which is the whole idea of understanding your video product as something more than the thing that you just hit play and watch, or understanding making video as something more than just hitting certain buttons to record the present moment, is something that is going to become increasingly relevant.
I don't know you guys UROP in labs or anything, but a lot of labs right now have their own cameras.
Many of you will probably end up having to give like a TED Talk or some sort of talk to the public where it's going to be more than just conveying information.
Like people were saying, you want to learn more about how to engage an audience in a way that is going to inspire and engage them.
So at the end of the day, this class is not about becoming the next Bill Nye.
That title is just totally clickbait to get people to sign up for the class.
And it's not even really about making videos, like I was saying.
The eventual deliverable of this class, the thing that we're going to be working towards every single day of this month, is to eventually create around a three-minute episode that you host, that you're on camera for, where you talk about a science or technology or engineering/math subject or topic that you're interested in.
So that's the deliverable, it's a video.
But it's not necessarily about what are the technical steps I need to get there.
Those are important, but eventually it's about taking those skills of being a producer and transferring them into your everyday life.
So maybe some of you won't make a video after this month is over, and that's OK, but hopefully the skills that you've picked up along the way to becoming an effective video producer will help you become a great advocate for science and engineering later on.
Every video that you guys will make in every assignment that you make, and every video that's online right now, has different objectives.
Some of you may want to do a video on physics, some of you may want to talk about math.
But the thing that should be the overarching objective of everything you make during this month should be to address these three problems in that amorphous STEM blob that I was talking about earlier, that you want to open the door to science and engineering, and really love of learning to everyone.
So a lot of that will translate to making sure that your script isn't condescending to your audience, making sure that you're really meeting and knowing your audience where they are, encouraging this lifelong learning.
We want curiosity to drive a lot of what your objectives should be.
You want your audience to become curious.
And then, of course, we want your science to be legit so you're educating the public, you're spreading STEM literacy.
What makes a good video?
Now, this is a quote from one of the optional readings that's listed on the syllabus.
It's actually really great, we just don't have time in class to cover it.
But I love this quote.
"Learning how to make really effective video is like learning to speak a second language.
You have to learn not just what to say but how what you say will be received by others.
We understand video better than any humans that have ever lived.
Most of us just don't speak it well."
We consume video all the time, like you were saying you watch YouTube videos.
But that doesn't necessarily transfer into us knowing how to make videos.
It's the argument of, oh, teaching can't be that hard because I've been a student this whole time, right?
It's very hard to extrapolate what it is about the practice of the video that makes it good when all you are used to doing is being a consumer.
So what makes a great video?
Let's just take this piece by piece and look at best practices in established video realm.
So educational videos.
There was this task force that MITx had a couple years ago on what makes an effective educational video, and the four traits that they came up with were, one, that they were short, that they were digestible modules, usually under 10 minutes.
Another is that the learner could interact with the speaker of the video, so maybe the speaker would mention a topic and it would be something that the viewer could end up looking up on their own and interacting with.
The other thing was that the host had them engaged, so it wasn't just this passive recording of them rambling on for an hour, the host would actually tell the viewer, OK, now try this problem at home, for instance.
And then the host would acknowledge the viewers, so it wasn't just, here's this math problem sent out to this vague audience, it was, you try this at home.
And this is also what a lot of what I call explainer videos do on YouTube.
I don't know-- you guys must be familiar with Khan Academy, but there are also things like MIT BLOSSOMS, which is a set of videos that's developed by a group here.
There's Bozeman Science, who's this high school teacher out in Montana, and he does tutorial videos on all the AP science subjects.
Tyler DeWitt, a former grad student here, he does chemistry tutorial videos.
And they're all very, very successful because they take an audience that's already motivated in learning something and they're sort of one-stop shops.
So how do I calculate the molar mass of this compound?
You can just go to one of Tyler's videos and learn how to do it.
It's very misleading sometimes, though, because people see it and they often mistake it for it being a direct recording of an instruction.
And so they say, oh, these explainer videos are much easier to do than big budget productions.
And that's not necessarily true.
They still spend weeks and weeks writing a script and developing exactly what they're going to say.
And they're so effective because it's a very talented host, a very talented teacher who is engaging their audience in a way that helps them achieve mastery of the subject.
So it gives them learning confidence.
People love Sal Khan because they really feel connected to him beyond sort of this robotic voice that's telling them how to calculate the derivative of something, and that's a very intentional thing I think that he does.
Now, for entertainment, even within the world of YouTuber or within the world of entertaining videos, there are subcategories that have their own best practices.
But in general, for entertainment, the thing that is most important, at least I think, is building an emotional connection with your audience.
And that's something that is repeated by some of the folks that I'll be quoting in a little bit.
And in terms of viral video, the way to best establish that emotional connection is through authenticity.
By authenticity I mean when you see a video that goes viral on YouTube, it's not necessarily something that has a super corporate sheen.
It's not really polished.
A lot of times it just looks like a very janky video that someone took with their iPhone.
And that in some ways helps people feel really connected to the material that they're watching.
And you have to be careful with it.
But I did want to show a couple clips of two science videos that have each raked-- I think the first one has over six million and the second one has almost 10 million views.
And we'll just watch part of each
All right, here is what we need.
Empty soda cans and put a little bit of water.
Just a little bit, so it will boil down, you know?
And then turn on the stove.
While this gets ready we're going to make a nice bowl.
So right here we've got our ice water already.
And you see how water is boiling and steam coming out.
I don't know if you can see it, but this steam is coming out right now.
So we're going to get it and drop it.
Use the tongs, don't use hands.
Get it and drop it upside-down.
That kind of fell.
Let's start with this one.
So are you guys ready?
You see?
They called this implosion.
I want you to tell me why that happens.
Why does it implode like that?
So we've got this pan.
Here you go, a pan.
And just we're going to turn this on.
But remember, safety is first.
Kids, you're going to need adults' permission.
So and there we're going to leave it for a minute.
We're going to put a little bit of water.
Do you see how water is boiling now because it's too hot?
So all the water is gone, boiled out.
And let's leave it for another two minutes.
So it's been about two minutes.
You see how hot it is?
Let's put just a little bit of water, just little drops.
You see how all these drops just like spinning around and not boiling out?
I want you to comment and tell me why.
Let's drop some more
So the thing is like 10 minutes long so I'm not going to show it.
But this video, which I personally cannot stand watching, this thing has gotten over five million views on YouTube.
And Crazy Russian Hacker, who's the user who created it, he has millions and millions of subscribers.
I have the co-worker whose son is 12 years old and he loves Crazy Russian Hacker.
And I don't know how you guys feel about the video.
Were there any certain elements that you noticed about it that either you liked or you understood why people would like it so much?
Yes, John
He asked the audience to explain what was going on, so that encouraged feedback and more posts by commenting on the video
Yeah, so there's that element of not only acknowledging your audience but engaging it.
And that's something that a lot of YouTube videos do.
Maybe it's to get more comments, but it certainly works a lot of the time.
Yes
That video seemed like it was kind of-- it wasn't necessarily scripted, so it kind of shows that he really knows what he's talking about, he's not robotically following a script.
Shows he knows a little bit about what he's doing
And he's quite a character.
I mean, Crazy Russian Hacker has a huge brand, right?
Like as soon as he starts talking you know that it's a Crazy Russian Hacker video.
And he's kind of outrageous.
Which is why it's hard to repeat that.
It would be hard-- if I replicated that exact same video, it would probably not go viral.
So his whole persona and the way that he has this sort of unpolished look about his videos is actually maybe one of the reasons why it's compelling.
Were you going to say something, Yuliya?
Yeah, I think it's also relatable.
He dropped the can the first time, so that tells the viewers that they can also make mistakes
A lot of online videos have this single-take format.
So instead of having two cameras like these guys set up where you're switching back and forth between angles, he kind of just held his iPhone where he was, right, or whatever camera, and followed along.
So you actually feel like you're right there with him.
You feel like you're in the kitchen with him.
There are no cuts back and forth to him talking.
It's just one long, continuous take with just a couple mess-ups taken out in between
Even though he asked comments, I'm very sure these comments are not all legitimate comments.
I've very sure there will be probably one or two people who purposely are trolls and probably say, oh, this is black magic or something, and then someone will comment, and the virality probably gives him the extra few million views
Yeah, yeah.
Logan Smalley, who's the head of TED-Ed-- you guys have seen TED Talks, right?
Have you ever seen TED-Ed videos?
They do a spin-off where they have animators and writers partner up.
He's the one who directs TED-Ed, and he had this talk where he was saying how a video online has a beginning, middle, and end, but the beginning, middle, and end isn't what's in the video itself.
The beginning is the tweet that announces the video, the middle is the video itself, and the end it is all the conversation that takes place afterwards in the comments online and social media.
And so with things like Crazy Russian Hacker, the end, the conversation, is almost just as important or just as much a part of the experience as the 10-minute video itself.
I wanted to show you one other video.
This is from Smarter Every Day, and this has also over five million views
Hey, it's me, Destin.
Welcome back to Smarter Every Day.
So you've probably observed that cats almost always land on their feet.
Today's question is why.
Like most simple questions, there's a very complex answer.
For instance, let me reword this question.
How does a cat go from feet up to feet down in a falling reference frame without violating the conservation of angular momentum?
Now, I've studied free-falling bodies-- my own, in fact-- in several different environments.
And once I get my angular rotation started in one direction, I can't stop it.
Today we're going to use a high-speed camera.
We're not going to use Ally because this is my daughter's cat, I don't want to hurt it.
We're going to use a stunt cat.
Let me introduce you to Gigi, the stunt cat.
I'll just flip the video vertical and then motion-track the cat.
It's just going to take a lot more effort in post.
We're going to try to do it in a way that doesn't make anybody mad.
That's pretty hard to do.
You got to drop a cat.
Ready, Gigi?
Checking out the high-speed data there, Gigi?
OK, the first thing a cat does when it's falling is try to figure out which way is up.
It does this either with a gyro in the ear or with its eyes.
Ready to talk cat physics?
All right.
So check out this footage I captured with the Phantom Miro while Gigi goes to get a drink of water.
So here's what's interesting about this to me.
If you'll notice, at the beginning of the drop the cat is not rotating.
Halfway through the drop the cat is rotating, and then at the very end Gigi somehow stops rotating.
Newton's first law says that an object at rest will stay at rest unless acted on by an external force.
I see no external forces on this cat.
So what's happening here?
It's not making sense to me.
OK, so in order to really get the right data, we're going to have to drop her 90 degrees out of phase.
Ready, girl?
This time watch her tail.
Three, two, one.
OK, so you think you figured it out?
Check this out.
You probably noticed that when the cat was falling her tail was rotating in the direction opposite of where her body was rotating.
What's interesting about that is that that's not how it works.
In fact, even bobtail cats can do this.
It's called the cat righting reflex.
I'll prove it to you.
I came across some video from the '60s when the Air Force was researching microgravity for future astronauts.
Turns out they took some cats up on parabolic flights.
He turns to rotate his tail to flip over, but it doesn't work.
He just ends up nutating wildly.
Then he does something interesting.
He takes his back and he bends it.
And when he bends his back and then creates motion, something interesting happens.
Ahhh, now we're getting somewhere.
So let me show you one more cat flip with the Miro and we'll figure this out.
OK, the arched back ends up being pretty important.
What he does is he divides his body up into two separate rotational axes that are tilted from one another.
When he's released, he pulls his front paws in and does the ice skater trick.
He decreases his moment of inertia in the front so he can spin fast up there.
But in the back he pushes his legs away from him, increasing his moment of inertia.
So a really large twist in the front equals a really small twist in the back in the opposite direction, and the torques equal out.
So as soon as he gets his front paws in under him, all he has to do is extend those legs back out to increase that moment of inertia and stop the front twist, and extend his back legs along that rear axis.
That allows him to twist those around really fast.
And then all he has to do is pull them back in under his body and then extend all four legs, and brace for impact
All right, so very different from Crazy Russian Hacker but just as popular.
I personally really like Smarter Every Day.
I think he's one of the best hosts, period.
You can disagree with me, that's totally fine.
But the thing I like about him is even though he's maybe a little bit more scripted-- he doesn't say um a lot, he doesn't really mess up as much-- I still feel like Destin is talking to me, that it's just this random guy in his backyard.
He happens to be a rocket scientist down at NASA or some similar engineering firm in Alabama, so he obviously knows his stuff very well.
But there's something about him that's still very relatable, very natural.
He seems like who he is on screen is who he is in real life.
Did anyone have any particular aspects about that video that they liked or disliked?
Or anything that they noticed?
I think he thought about how [INAUDIBLE] tail was what affected the cat's movement, allowed the cat to-- but he mentioned that it wasn't the tail at all, so then that made us more interested in wanting to finish the rest of the video
Mm-hm.
He anticipated the misconception [INAUDIBLE] the cat.
And that's actually a really big tool that you can use in your videos to hook people in.
And Ceri, you take that summer MIT course, right?
Where you were specifically addressing misconceptions in biology?
Yeah, I actually took a summer class at MIT, and I personally made a video.
But our whole objective of the class was to choose a topic in science that people often have misconceptions about.
So some people did videos on you catch a cold if you're outside in the cold.
And then mine was your deoxygenated blood in diagrams is always shown as blue and so people think our blood is blue sometimes, but it's always different shades of red.
Yeah, and so our entire goal was to figure out not only the real science behind the topic, but how to explain it in a way that debunks the myths or the misconceptions that people have about them
And I don't know what it is about misconceptions, but it does seem to be something that not necessary predicates but can help launch a video into being something more engaging.
And maybe it is because it taps into the audience itself and it becomes more relatable, because it's acknowledging, hey, this is what you actually think.
And it becomes surprising and maybe more memorable in that way.
That you're more likely to remember that a cat doesn't use its tail to sort of out-balance the rotation that it feels when it's flipping but actually scrunches up.
I mean, I'm going to remember that way more than I'm going to remember a single one of my 801 lectures on rotational physics.
So that's just something to think about.
Now, something I do want to mention, and this is a conversation that Chris and I have a lot, which is that people often confuse correlation for causation, right?
They think, oh, well, all these videos did really well so I just need to make a crappy video and do really bad lighting and just be really terrible on camera, and it'll go viral, it'll be great.
It'll be relatable and authentic.
And that's not necessarily true, right?
Low production value is not equal to and does not predict a viral video.
If any of us tried to recreate Crazy Russian Hacker it would probably not go viral for the same reasons that I talked about earlier, which is that there are a lot of elements that go in play into creating something beyond what you see on screen when you're watching it.
That it's OK to have good production.
Smarter Every Day is actually pretty decent, and when you see his newer videos, his camera work, it's pretty good.
And you can tell that he's gotten new equipment and things like that.
And it doesn't take away at all from the experience, from the authenticity of the video.
That authenticity starts at the core of the host, at the way they're creating it, the reasons why they're creating the video.
And it's not created by the conditions of the production itself, necessarily.
It's just that the conditions don't hinder the authenticity.
But please don't mistake.
Bad video production does not help you get to viral video.
It just doesn't stop you from getting there.
Now, for YouTube-- and we sort of previewed this a little bit with the two videos that we watched-- there are a couple traits about YouTube videos that I think make them really great for learning and for educating, and for successfully engaging an audience.
The first being just sort of the format in which they're written.
A lot of viral YouTube videos or videos that are well known, or a lot of SciShow videos, for instance, they follow this format of asking the why, what, and how questions.
So you'll notice that the Smarter Every Day video was not titled gravitational physics, Newton's second law, or rotational dynamics, right?
It was why does a cat fall on its feet when it falls down, right?
It's contextualizing the question.
It's making it relatable.
It's sort of tapping into the innate curiosity that people have, even if they hate science.
People who watch that video are not physics majors.
And it makes them more sharable.
It makes them more memorable.
And I think it makes it a little bit more conducive to this idea of retrieval learning, that people aren't going to just end the video right there, right?
They're going to maybe share it with their friends or they're going to talk about it with their teacher at school.
Which is such a powerful thing to happen with learning.
I think that's what really everyone's objective is as a teacher, or a lot of teachers have this objective, that they don't want the lessons that they're teaching to just stop once people leave the room.
They want people to keep thinking about it and thinking about it in the context of their life outside of the classroom.
So this format really aids in doing that.
And I have a couple of videos I want to show you guys, the first being from Asap
Ahhh, sleep.
You can never have enough of it, it seems.
In fact, sometimes it literally feels like you aren't getting enough.
But what if you stopped sleeping altogether?
Strangely, science understands relatively little about why we sleep or how it evolved in the first place.
After all, laying unconscious and dormant for hours on end while predators lurk hardly seems advantageous or smart.
But we have discovered a few correlations.
For example, adults who sleep between six to eight hours a night tend to live longer.
Excessive sleep, however, can lead to medical problems, including cardiovascular disease and diabetes.
Similarly, chronic sleep deprivation has been linked to aspects of cardiovascular disease, obesity, depression, and even brain damage.
But what if you stopped sleeping right now?
Well, after your first sleepless night, your mesolimbic system becomes stimulated and dopamine runs rampant.
And this may actually trigger some extra energy, motivation, positivity, and even sex drive.
Sounds appealing, but it's a slippery slope.
Your brain slowly begins to shut off the regions responsible for planning and evaluating decisions, leading to more impulsive behavior.
Once exhaustion sets in, you'll find yourself with slower reaction times, and reduced perceptual and cognitive functions.
After a day or two of no sleep, the body loses its ability to properly metabolize glucose, and the immune system stops working as well.
In some cases, three days of no sleep has led to hallucinations.
Care about how you look?
Studies have shown a direct correlation between sleep deprivation and a person's perceived beauty.
That is to say, sleep-deprived individuals appeared less healthy and less attractive than when they were well-rested.
The longest scientifically documented case of being awake was 264 hours, or 11 days.
And while they did develop problems with concentration, perception, and irritability, the surprising truth is that they suffered no serious long-term health effects.
In fact, no individuals under these documented conditions experienced medical, physiological, neurological, or psychiatric problems.
But there are limited studies, and this doesn't mean permanent damage couldn't be inflicted with more time.
Sleep deprivation experiments on rats, for example, generally lead to death after about two weeks.
But scientists aren't totally sure if they're dying from the lack of sleep or from the stress of constantly being woken up.
Perhaps we should look at fatal familial insomnia for an answer, a rare genetic disease of the brain which causes progressively worsening insomnia, or sleeplessness, leading to hallucinations, dementia, and ultimately death.
This disease has only affected around 100 people in the world, but their average survival span was around 18 months.
Over time, the lack of sleep becomes worse and the body's organs begin to shut down.
So while lack of sleep won't necessarily kill you quickly, continual sleep deprivation will have a negative effect on your body.
Sleep tight.
But not too much.
Got a--
So were there any elements of that video that you found striking, that you liked, that you disliked?
Yes, Yuliya
I really liked the movement and that he changed his materials
Oh, like from drawing to construction paper and things like that.
What did you think about the animation?
Did you like it or not?
Oh, are you asking me?
Yeah
Yeah, I think it's effective.
I think one of the things about it that's nice is that the animation allowed him to change to appeal to I think a current need for a constant stream of images that are different.
This kind of-- I think we've been conditioned to be sort of bombarded with images, and animation is great for that
I mean, I personally am not a huge fan of AsapSCIENCE style, but I also recognize why it's super effective and why other people would like it.
Because like you said, it's very engaging, if only for just the visual--
It's also really well written.
I actually think that it's better written than it is-- I mean, I think there are any number of ways it could have been performed.
But I think it's really well written.
It's very concise
Right.
It's very shareable.
It's sort of written in this listicle form that is dominating web media in general.
It's like basically taking a BuzzFeed article and putting it into a video.
Yet with the visual elements, there is a motivation for you to watch versus just see it on text form, in BuzzFeed, for instance
I actually thought that the cat video lost some of its engagement for me when he was-- there was this sort of stream of scientific information and all you're doing is just watching this cat in slow motion.
And for me, it lost that kind of key component of what makes something engaging, which is the tight cohesion between what you're seeing and what's being said.
And also the streamlining of scripting
And that's always going to be a push-and-pull that everyone's going to experience, too, with-- at some point, are you explaining too much to the viewer?
Should you just rely on the visual?
Because you don't want to be too jargony and end up alienating an audience, but at the same time you have to cover-- you can't cover everything, right?
You can't go in it in detail and explain everything.
And that's a conflict that we'll hit in later lectures, how you can use animation to help you with that, some of the best practices in that realm.
But that is something to think about, and I think that's a struggle that everyone has with video, for sure.
I have one more video.
Did anyone else want to say anything about what would happen if you didn't sleep?
Yes
You know, I was watching and I was thinking because so much of it is being shown just sort of from Gary of these pictures and these words, the thing that Crazy Russian Hacker had and the cat guy had that this guy doesn't is that you know they're not making anything up because you can see the cat and you can see the bubbles, whereas there are no references, so how do I know that everything that is being said is based on fact upon fact upon fact.
He could be making all of this up, for all I know.
And as I was watching it, I was thinking, how do I know he's not-- like where's the truth in here?
I don't know
Yeah, there is some of that element of like the web has a lot of information but what of it is validated, or where does it come from
If there were a little subtext somewhere even just referencing any of the ideas, I'd feel more like, all right, I can believe this guy.
But as it is, it's hard, because you just have a whiteboard and a marker, and you think, well, how do I know they've done their research
Right.
And I mean, with YouTube channels, a lot of them put in the sources in their descriptions.
But I think the thing that maybe I enjoy less about AsapSCIENCE is you could switch the creator out for any other person and have any-- you could have Siri from your phone narrate the video for you and it wouldn't be a completely different experience.
I mean, it has its branding because it was really one of the first channels to do that style of hand-drawn animation, so that's its thing.
But it doesn't have the compelling sort of authenticity that Crazy Russian Hacker does, or Smarter Every Day, or SciShow.
And I think a lot of it is because you don't see someone on screen, you don't see a person on screen.
And maybe that's just my personal taste, but I do think that makes a really big difference.
This video is from Vsauce.
Has anyone ever seen that channel before?
Yeah.
So I actually haven't watched all of this video by myself because I was scared to, but this is why I think it's creepy
Hey, Vsauce, Michael here.
Fear gives us life.
Being afraid of the right things kept our ancestors alive.
It makes sense to be afraid of poisonous insects or hungry tigers.
But what about fear when there is no clear and obvious danger?
For instance, a teddy bear with a full set of human teeth
I can't
A smile.jpg.
There's something a little off about these images.
Too much mystery and strangeness.
But no obvious threat the way there is with a gun or a falling rock.
But yet they still incite fear because they are creepy.
But why?
What gives us the creeps?
What causes something to be creepy?
We are now in my bedroom, the bedroom I grew up in, in Kansas.
Like a lot of children my age, I was terrified of scary stories to tell in the dark.
But the very first book that ever scared me was the Curse of the Squirrel.
To this day I still haven't finished the book, but that's just me.
Psychologist James Geer developed the Fear Survey Schedule-II, which he used to find out what scared us the most.
Combined with the results of a more recent Gallup poll, these are the things that scare most of us the most.
All of these things are scary, but are they creepy?
Let's get more specific.
I love the way Stephen King delineates three types of scary stuff.
The first is the gross-out.
This is something disgusting, morbid, diseased.
The second is horror.
Horror, to King, is the unnatural; a giant spider or being grabbed in the dark when you thought you were alone.
The third, terror, is different, creepier.
He says terror is coming home to find that everything that you own has been replaced with an exact copy.
Terror is feeling something behind you, its breath on your neck, knowing that you will be grabbed, but then turning around to find that there was never anything there in the first place.
Not a lot of research has been done on that feeling, the creeps.
But many theories and ideas involve vagueness, ambiguity.
For instance, masks, and why clowns are creepy.
Claude Levi-Strauss wrote that the facial disguise temporarily eliminates from social intercourse the part of the body which reveals personal feelings and attitudes.
Part of the reason even a neutral or happy mask can be creepy may have to do with ambiguity.
A mask hides the true emotions and intentions of the person underneath.
I don't know if the person wearing that mask is a threat or not.
Vagueness is creep when it comes to the human form.
This is the famous uncanny valley.
On a chart of humanness, there's a zone where something can be almost entirely human but off by just a little.
Not so wrong that it's clear fake or funny, or so good that it's indistinguishable.
Instead, it's just troubling.
The creepiness of the uncanny valley is wonderfully demonstrated by John Bergeron's singing androids.
Watch these videos when you're alone.
A similar uneasy feeling comes from Shaye Saint John, a character created by Eric Fournier.
Funny to some, nightmare fuel to others.
Uncanny humanoids, like all creepy things, straddle a line between two regions that we can understand and explain with language.
Francis T. McAndrew and Sara Koehnke describe being creeped out as an adaptive human response to the ambiguity of threats from others.
Creepy things are kind of a threat maybe, but they're also kind of not, so our brains don't know what to do.
Some parts respond with fear while other parts don't, and they don't know why.
So instead of achieving a typical fear response, horror, we simply feel uneasy.
Terror.
Creeped out.
Between the mountains of safety and danger, there is a valley of creepiness, where the limits of our knowledge and trust and security aren't very clear.
Will looking at this cause you to die one week later?
Impossible, right?
Maybe.
That's the terror of ambiguity.
We don't do well with ambiguity.
When it involves our own intentions, it can make us lie.
And when it involves danger--
Can I stop or you guys want to keep watching?
[INAUDIBLE] recognizable threat, it can make us think and feel some pretty weird--
So he goes on and talks more about like the actual psychology behind this.
The whole pop-up thing is his trademark move.
He does that between all of his transitions.
It's like a little kitschy but it works since he was the first to do it.
But what did you guys think about this one, especially in comparison to AsapSCIENCE?
Very similar format, it's the why-what-how, but executed very differently.
Or maybe was there something about this video that you liked better than AsapSCIENCE or that you disliked compared to AsapSCIENCE.
Yep
I think I got the point maybe couple minutes in, but it seemed like the next three or four minutes, he was just expounding on that point that I already got
Yeah, so it dragged a little bit more
Right, it never really answered-- I mean, you didn't finish the video, but it never seemed to answer the question [INAUDIBLE]
Yeah.
I mean, he does sort of eventually, but it does take him a little bit longer to get there.
Yes
It still kept my attention because he was very close to the audience and he was an interesting character, so I was interested in knowing what he would say next, [INAUDIBLE]
Yeah, Michael Stevens definitely is a very memorable persona on screen.
And this is something that we'll emphasize throughout the course.
We don't want you to feel like you have to exaggerate yourself or sort of because this persona on screen that you're not naturally.
It's not about you trying to adapt into a Bill Nye personality or into a Michael Stevens personality.
A lot of people find him super annoying, and I don't blame them.
It's really about how to best capture who you are in real life and just maintain that as much as possible in front of a camera, because maybe he is a little bit different in real life, but it doesn't seem super unnatural when you watch, right?
You're not like, this guy is acting just like totally over the top.
It's just part of his brand, part of his persona to get his message across.
I do think that something that was effective about that video is it took all these concepts that you learn in cognitive psychology, but again, instead of doing a video on here's the definition of uncanny valley or things like that, he tapped into something that resonates-- at least it resonates a lot with me, because I was super creeped out by this video.
But it taps into something that we experience every day, that it's a very relatable video, that it's contextual, that I don't feel like I'm watching a lecture, necessarily, on it.
I don't feel like I'm even watching a Khan Academy video.
That I just want to sit there and learn more.
I wanted to learn more for the first two minutes, at least.
I don't know if you guys felt that way.
Yes, no?
Yeah
I felt like I got pretty much what-- after he showed me the graph of the thing, I didn't really understand the graph, but he kind of explained it with like, oh, you need to be in between two points to be creepy.
Then after that, it's all about trying to figure it myself.
He does this connect-the-dots thing.
He doesn't really give you the right answers.
He seems to be seeding you
Yeah.
The payoff happened, but it takes a long time to get there.
And maybe he loses interest along the way.
Again, that's something that you're going to have to figure out how to balance, how much tension do you want to build up for the audience.
Maybe you'll do it too much, to the point where you lose them.
Maybe you do it to just create the buildup to the final reveal of your video.
In addition to being these digestible, listicle type chunks that are shareable and contextual, YouTube videos are also super searchable, right?
So you can just go on YouTube and search for literally anything you want, and you'll have suggested videos that pop up at the end, so there's this sort of inherent-- what was the thing I put?
There's an inherent vetting process that happens on YouTube, that you have a compendium of knowledge and compendium of resources available to you, and you have things like comments and suggested views and subscriptions to help you decide what to watch.
And everything's at your fingertips.
You can get whatever you want, whenever you want it.
Then there's this whole trusted guide element that is really-- it's just as present on YouTube as it is on TV.
You have people like Vsauce and Michael Stevens.
Even with AsapSCIENCE, you don't see the guy, but it has a really strong brand.
And it's one of those things where if you replaced a successful YouTube video with another person or another narrator-- a litmus test that I like to think of is if you replace it and it's just not really a noticeable change, then maybe there's something with your video that isn't very engaging.
That if you swapped out any of your classmates for it and it still sort of felt like the same video, maybe there's more of your inherent personality that you can highlight more in the video.
And then, with YouTube, there's no invisible college, right?
There's no gatekeeper of knowledge.
You don't have to pay $100 to access a lecture course, right?
Anybody can access OpenCourseWare.
Anybody can go watch a Vsauce video.
Anyone can comment, so theoretically, anyone can contribute to that community and landscape, and its products as well.
I think we're starting to see more of it.
You see more female creators, you see a little bit more diversity in the YouTube environment than you do on TV, certainly.
But again, there's a lot of room to grow in that regard.
And then finally, for TV, TV is very different from online video, which is something that I didn't actually realize myself until fairly recently.
I always kind of thought if I want to make a good video online then I should just sort of apply the best practices of making a good video on TV and just shorten it to a five-minute online video.
But that's not necessarily the case, because the priority of TV, first of all, is to hold your attention.
When you're watching TV, there are going to be commercials, you can flip the channel.
I guess you can do that with online stuff as well, but the distractions aren't necessarily as huge.
When you go to a commercial, you're going to want to change the channel.
So they really have to hook your attention, and they do that a lot with some of the production styles.
On TV you'll tend to see a lot more multi-camera type videos.
You'll see a lot of quick cuts, a lot of effects to sort of keep you engaged.
I noticed that about the Hobbit, there was like an explosion every other second.
And they do that because they have to have this really big hold on you for a much longer period of time.
Whereas on online videos, stuff like Smarter Every Day, it's just kind of the same camera set up on a tripod that's shooting you the whole time.
And TV is very much about narrative, about telling a story.
And I think that that is something that you should do in an online video too, but it's not as much of a necessity.
Things like-- I don't know, any viral video that's not necessarily science-related, most of them don't really have a story.
It's more just like, here's a naked celebrity.
I guess you could create a story out of it, but it's not narrative-driven, necessarily, whereas TV very much is, and movies are as well.
I wanted to show you another video, and this is from Connections, which I never heard of until Chris showed us this show in his class.
But it's awesome.
It's like one of the first science TV shows in the UK.
And this is from Connections 2, and it's a whodunnit episode
I suppose a detective catches a crook because he follows a trail from one uniquely relevant event or person to another, until he finds a unique piece of evidence that points to the only person in the world who could've done the deed.
And strangely enough, the story I'm about to tell you about why modern detectives are able to do that at all follows exactly the same kind of tail, from one unique character to another through history.
Here's my first unique character, Steve Davis, one of the best snooker players in the world
Ah, don't look at this.
So that is a show that aired-- I don't know, was that like the '70s?
It's the late '70s [INAUDIBLE]
Late '70s?
And this is Connections 2, so maybe it's early '80s.
I don't know about you, I felt like that was so endless.
It took him two whole minutes to even just introduce the first character of what he was talking about, and it felt--
Isn't it amazing how the pacing has changed, and our expectations?
Right, it felt agonizing, almost.
And I'm sure if I watched it on TV and I saw the rest of it, it would be fine.
But I mean, compare that to AsapSCIENCE, for instance.
The priority of what Connections producers were doing with that video is very different than AsapSCIENCE, and you have a very different effect.
I actually really enjoyed Connections, I really liked the show too.
So I'm not saying it's a bad thing, necessarily.
But that's sort of what happens when you're applying this best practices in TV or movies and you think you can just move it over to video on the web and just shorten it down.
That's not necessarily going to be the best approach, because some of the best practices aren't going to work, necessarily.
So what I would like you guys to do right now, if you can access-- does everyone have a laptop with them or can share a laptop with someone?
If you go onto our class site, it's mit219.tumblr.com.
You'll see a post that I just put up earlier today, and it has a link to a Google doc that should look like this.
Should look like this.
And what I'd like you guys to do for maybe the next like 20-25 minutes-- and feel free to get up and stretch during this time if you want.
If you need to use the restroom, feel free to head out
Can you write the URL on the board?
Oh, yeah.
Can everyone see on this board?
There should only be one post, and it'll have a link to the class syllabus that, again, has all the videos that we've watched today and the assignments for all the days.
And then it should have a link to this Google doc.
And what I'd like for you guys to do is to watch the videos that I've listed here, and then record the exact time code in which you want to stop watching the video, and just be completely honest.
And then afterwards we'll talk about maybe some of the reasons why you thought certain videos were more engaging.
Think about why you might still think that a video is good even if you wanted to turn it off.
And we'll convene at the end.
But again, feel free-- if you guys need to get up and stretch or go to the bathroom during this time, feel free to do that too.
And you guys can talk to each other as well.
All right, let's go ahead and talk.
I know you guys are finishing up.
Just so you know, every day you'll do blog reflections, and if you want to talk a little bit more about your thoughts on these videos in your blog today, that's totally fine.
I know we're not going to get everything.
But did anyone have a favorite video?
Yes
The Veritasium one
Oh, Veritasium, you like that one the best?
Yeah, actually, that one's my favorite too
And Asap
And Asap.
What about those two videos did you like, or maybe can you hit upon one thing that you found particularly--
Veritasium, very charismatic host, and kept the pace not going so quickly, but not going too slowly.
Then for AsapSCIENCE, it was very crisp and conveyed the point very succinctly and clearly
That's interesting because I actually thought that those were the two videos that I personally watched most of myself.
And I'm not necessarily the biggest fan of AsapSCIENCE, but I did want to watch the whole thing.
The thing about Veritasium is interesting, because he's not using super fancy equipment, per se.
His stuff looks a lot better than a lot of what's on YouTube, but he's using just basic DSLR cameras, I think.
But his background, he's a physicist, that's his PhD.
But he does physics education as well, so he's very cognizant of the role of video in education, and that's going to be one of your reading assignments tonight, is watching one of his videos.
But I do think that he does a very interesting job and a very good job of being very natural on screen, charismatic, like you were saying.
There's something about his delivery that is-- he talks the way that you would think he would just talk in real life, which is actually really hard to do on video.
Did anyone else-- let's see what you guys thought about the Veritasium video.
It looks like a lot of people actually finished that one.
And I didn't mention this before, but maybe you noticed.
All these videos are talking about evolution or natural selection in some way or another, but they're doing it very different ways.
What about anyone else, let's see.
Nathan, you finished SciShow.
What was it about SciShow that you liked?
I'd say I liked that there's a little bit of humor in his video.
I think that's something that tends to keep my attention better
Sorry, that you what?
Just when you have a little bit of humor inserted, that keeps my attention better
And Hank Green is a very neurotic host, right?
He talks like this and he has quick cuts and he's very hyper all the time, and it works for him because that's just who he is in real life.
So again, being Hank Green doesn't make you successful, it's that he's able to just be himself on camera that works.
Did anyone have a least favorite video or one-- let's see, a lot of people ended the TED-Ed one, or maybe a couple people ended that one early, and the Khan Academy one early.
Any thoughts about those?
What is it that made you-- I keep doing that.
What is it about those videos that made you turn them off early?
Paul, you turned that one off at 17 seconds.
That's pretty early
What one?
The Khan Academy one
Oh, yeah.
I just thought that was a bad way of portraying evolution.
I usually use Khan Academy for math or something like that, because he is going through all the hand motions and how to do an equation
Right
But I think something like evolution that's kind of nebulous to begin with, I'd like to see pictures, not really-- I don't know.
That's just my personal opinion
Well, yeah, I personally agree with that as well.
I mean, I also feel like Khan Academy works for a very certain type of video, and that tends to be a numbers and graph-driven one.
And we'll talk about this more in a little bit, but when you think about why you want to make a video, you have to consider quite a bit about the visuals that you're going to use and how that's going to motivate the story that you're going to tell, or trying to convey the message that you're trying to convey.
And I know-- I wish we could talk a little bit more about this, but we are running a little bit short on time and I want to leave enough room for Chris.
So if you can respond a little bit more in your blog reflection tonight about some of these videos, that would be awesome.
So we've hit upon some qualities of what makes a good video, a good educational video, a good YouTube video, a good TV video.
But the question of what is good, right?
Because ultimately we have to figure out what we're going to define as success in this class, because otherwise there's going to be no impetus to improve.
It's a really hard question, because what is good is really intangible.
It's really inaccessible, at least in the field of video, or whatever we're going to try to do, because what is good is very much a matter of personal taste, right?
Like I personally don't really like the AsapSCIENCE videos, but I recognize them as being successful in what they're doing.
They're engaging people, they're spreading STEM literacy, they're getting people to be curious.
I'm not sure how much they're opening the door to science, but I mean they're doing good things even if I don't personally enjoy them.
And a lot of people love Hank Green from SciShow and a lot of people think he's super annoying.
So again, it's really hard to define what is good.
We often try to approximate what is good by other measures, so a lot of times we'll look at virality, how many views it has, how many subscribers they have.
But again, virality is something that you can sort of assess in retrospect but it's not necessarily the most predictive measure, because so much of it rests on confounding variables like what are the current events that are happening at the time that the video comes out, what's the personality like.
Have you guys ever seen the OK Go music videos?
So they're this popular alternative band, and their big breakout video was them doing this huge choreographed dance on four treadmills.
And ever since then they've sort of tried to do this similar thing of a single take shot.
I didn't actually show any of the videos, but I can send you guys the links later.
They do these single cam, continuous shots of just like outrageous dances, and I don't even know how to describe them.
But the first one was the one where they struck gold.
And they repeated the same thing, it was the same people, but their later ones, which are successful, aren't nearly as successful as that treadmill one.
So there are qualities about viral video that are universal, things like authenticity.
A naked celebrity is always going to go viral.
But it's very hard to build a predictive model based off of a regression that you're looking at in the past.
And then you have things like learning objectives that you can use to qualify something as good.
You can say, oh, this video hit upon all to the standards that seventh graders in the US need to know about biology so it's a good video, and maybe that's why Bozeman Science-- maybe that's why he makes good videos.
But it only tells so much of the picture.
So what is going to be our definition of good?
And this is a question that I still am struggling with and I am totally open to us discussing as the class progresses.
But I do think that there are going to be certain qualities that we should strive for, regardless of if your taste is in really rapid delivery or if it's in very sort of philosophical, grandiose type videos.
This is a quote from a reading that you guys are going to annotate on tonight for your homework, from Hank Green, who's the host of SciShow.
But he says, "People who are new to the medium are starting to think that online video is not 'Just a little bit better than everything else on YouTube' but 'Just a little bit worse than everything that's on TV.'"
And, "That perspective is a super dangerous road to go down on."
That's sort of hitting upon what I was saying earlier about you can't take necessarily the best practices in the realms that we're trying to combine and transfer them over and expect great results, right?
You can't try to make a video like you would TV show, necessarily.
So what is good?
If good isn't the combined practices of TV, education, YouTube, what is good going to be?
So this is a viral video manifesto that's, again, optional reading.
It's actually pretty good.
It's from the guys who made the Diet Coke and Mentos video?
Have you ever seen it, where they combine Mentos and Diet Coke and there's this huge, choreographed explosion?
They talk about four points.
Be true, be authentic.
Don't waste our time.
On an online video, you can't afford to do what Connections did and take two whole minutes to set up the premise, because that's sometimes the entirety of a video.
Be unforgettable, show us something that we've never seen before.
And that honestly might be the hardest thing for us here, at least it was the hardest thing for a lot of the students that we worked with on the first two seasons of Science Out Loud, that it's very hard to take yourself out of the perspective of being an MIT student, or being a student involved in science and engineering.
And remember that some of the things that you take for granted and are part of your everyday life are actually really awesome and really cool, and things that most people don't see every day.
You may work at the nuclear reactor here and think that it's just another everyday thing, but that's a window that you can create for people who would have never, ever had access to it.
And then ultimately, it's about humanity, about the emotional connection.
A lot of that comes from the authenticity, that if we can swap you out with just the computer voice and there's not that much of a difference, then really rethink how you're structuring and writing and producing your video.
So there are going to be four course values that'll guide all of the assignments and all the things that we do.
And the rubric is listed in the syllabus that's on the website.
We don't really assign point values to did you do correct lighting, did you do this, did you set up your camera the right way, because we don't want you guys to get bogged down in the minutia of it.
Instead, there are going to be these four overarching values that'll dictate what is good, I guess, as we move along in the class.
The first one is Spark.
There's this quote, and I think it's by Fellini, but something along the lines of it's not what's inside the camera that matters, so much as what's in front of and behind it.
So are you promoting curiosity?
Is there a perceived love of learning in your product?
Right, is there a perceived love of learning in your product?
Is your passion and joy evident in your delivery?
This can be seen in your script, in your final video.
This doesn't mean that you need to go over the top.
It doesn't mean that you have to pretend to be someone that you're naturally not or you don't feel comfortable being.
But I'm sure all of us in this room have something that they're excited about, so can we perceive that excitement in your product?
Clarity.
So "Kill all of your darlings" is one of my favorite sayings and something that dictates a lot of the work that I do.
But this will be learning how to navigate the material you have and getting rid of stuff, maybe getting rid of explanations that don't need to be there.
Editing is going to be a lot about this, and the last lecture is a lot about this quote.
But it's also about can you convey your message in the most clear manner?
Is your script tightly written?
Is your delivery engaging?
Have you realized your vision as effectively as possible?
Then we have thoughtfulness, which is that every decision matters.
Whether it's deciding what background you're going to stand in front of to deliver one of your scenes, or deciding what facts to include in your final script, every single decision matters.
And it's thoughtfulness about what you decide to put in front of the camera, but also how you decide to talk to your audience.
And again, it's outlined more in the rubric.
And then go big or go home.
This is a hard one for people, understandably.
This is probably the hardest one for me.
But learning how to step outside your comfort zone and getting creative.
We do you want to reward creative risks, so don't feel like you can't try something new.
Again, hopefully it's an educated risk.
But these are all qualities that we're going to assess, not only through your final products but through your daily reflections, through a lot of the iterative processes that will be happening on the way to your final products.
And this is where we're going to integrate things like are things clearly lit, but in the context of was this a clear message.
Does that makes sense to everyone?
So if you want to make a video, where do you start?
The first thing is deciding if you should make a video.
And I think this list of questions is one that everyone should ask themselves before they make a video.
Why am I making a video in the first place, versus a BuzzFeed article or versus a radio piece?
Who is going to watch this video?
This will make a very big difference in how you decide to write your script, how you decide to deliver.
For the most part, our assignments will be assuming sort of a middle school type science background.
What visuals will I show?
Again, this makes a really big difference.
As Paul was saying, the Khan Academy video didn't really have the visuals that motivated the lesson that he was wanting to learn, so thinking about what visuals you're going to show are super important.
What is the fact or piece of information that the audience is really going to remember the most or surprise them?
In the Smarter Every Day video about the cat flipping, it's just that whole concept of a cat can seemingly defy the laws of gravity, but it's really just a modification of their torque or something that actually makes them able to flip the way to do.
And the biggest one is what is the point of this video?
Now, the guys who wrote the viral video manifesto, they used to be circus performers, so in their book they talk about how there's a slideshow circus pitch, which is when you have the ringleader around the side and he's like step right up, step right up, come see the world's smallest person next to the world's giant man or something like that, right?
There should be some sort of circus sideshow pitch to your video idea.
So one of the episodes we made was on farts, which maybe our motivation was just that we wanted to make a video about farts.
But on the other hand, there's this amazing biochemical process that's associated with all of that, and it really reflects just this incredible diversity of microbiomes that exist in your body.
And so it's not just step right up, step right up, come watch a video about farts.
It's like step right up, step right up-- what's the thing I wrote?
Witness your body's countless armies of lowly bacteria squeeze every drop of energy from your gut and create horrendous sounds and smells.
So there should be some sort of circus sideshow pitch to your video, and Chris is going to help us realize that in the actual way of how do I actually make that video, how do I actually create that vision.
So that's what we're going to do for the last part of class.
But be thinking about maybe what your circus sideshow pitch is going to be.
And eventually tonight everyone is going to need to create a very short, maybe two-minute video, one to two-minute video, a science, technology, engineering, math topic that you're basically going to pitch through this video.
It's going to be sort of a trailer for your final project.
All right, Chris, I will let you take over.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So anyway, I'm Chris Boebel and, as I said, I'm the Media Development Director at the Office of Digital Learning, which is what the K-12 video project is part of.
As well as MITx, and OCW, and a bunch of other initiatives on campus, I also teach a documentary video class-- a science documentary class-- here at MIT that Elizabeth took a few years ago.
And what I'm going to talk about is storytelling.
And specifically, visual storytelling, and how that can relate-- or hopefully will relate-- to the work you do in this class and to online video in general.
I'm going to start out just by taking maybe-- I don't know if it's exception-- but just having a slightly different perspective than something that Elizabeth said.
I would say that almost all videos that are successful, almost all viral videos, have some element of a story to them.
Even a five second Vine video.
Just about everything.
And I actually was thinking about that, and I went looking for what is probably one of the dumbest viral videos to come out recently.
Just a few days ago.
You may have seen it.
This is a fan at Ohio State who was caught on the Jumbotron basically scratching her boyfriend's head and then picking her nose.
And this thing got-- I just, it's ridiculous how much play this got in about a 12 hour, 20 hour period.
Very ephemeral, you know.
It's absolutely not going to last.
It's not going to change anyone's life, but the reason it took off is that it has a story.
And the story was created and invented by the people who watched it.
You know, it's not really-- yeah, go ahead
I thought the point of that video-- I saw the same thing that she was with-- that's like the guy she was seeing on the side--
Exactly
--that's not her boyfriend
That's actually not true.
That was the story that was completely created by people who watched it.
They've been together for a long time, and she didn't even know she was on the Jumbotron actually, apparently.
But it just happens to be this kind of little moment that tells a story, and because it told a story that people related to, suddenly it turned into a giant thing.
One thing that's hilarious is somebody actually reversed it, so she picks her nose and then sort of leaves a deposit on his neck, and that became another story
There's something disturbing about her facial expression
Yeah exactly.
She looks guilty.
She looks guilty, and that's why that's part of the story.
Like, oh my god, I've been caught rubbing his neck.
So anyway, I think this has got to be the dumbest viral video of all time, but I think it's also a story.
Never think you're not telling a story.
If you forget everything else that I say in the next 30, 40 minutes, I would remember this.
There's some element of a story in what you do.
I don't care if you're making an FDA approval video, as one of you mentioned you are interested in doing-- something I've actually done personally.
I don't care if you're making a 30 second TV commercial, a 10 second TV commercial, a 12 minute physics demonstration.
There is an element of story to it.
And it may not be the sort of conventional, traditional, "television" story that we would see on an HBO show.
It may not be a Hollywood movie story.
But there is an element of story, and one of the things that you want to do here is find what those elements are.
Now, let's unpack that a little bit.
Let's talk about what is the story, and how might it relate to what you're actually going to be doing here.
And then secondly, we'll also talk about what are some of the specific techniques of visual storytelling that you might be able to employ.
So first of all, what is a story?
Full confession.
I went to film school and I worked in that sort of ancient medium that Elizabeth talked to, television and film.
And the classic and traditional way of talking about a story is that it's got these things.
It's got a protagonist, a hero.
It's got an inciting incident.
It's got something that happens that sets everything in motion.
Then there are barriers along the way, problems that the protagonist has to overcome, and then there's a resolution.
So you do that in a kind of traditional, classic way, and you end up with connections, or worse, right?
It's not going to work, obviously, in the genre that you're talking about.
Another way of looking at this traditional way is that you've got act one, which is the set up, you've got act two, the barriers, the plot.
You've got act three, the resolution.
The three-act structure is kind of the classic screenwriter mode that you see over and over again.
So how does that relate to what we're actually doing?
Well, this is going to be somewhat repetitive to some of the things Elizabeth talked about.
First thing, clearly, that you need to do in this very short form is engage your audience immediately.
I mean you've got maybe, like, five, six seconds, maybe eight or ten, to engage them on some level.
That's the beginning of your story.
That's the inciting incident.
And then this is something that I find very interesting, and you know when I talk about web video, this is something that is a paradox, that I think is really, really true.
You want to surprise your audience, while also showing them exactly what they want to see.
And this can be really a challenge.
Surprise your audience.
In other words, you want to take them somewhere, show them something that's fresh, that's new, that they haven't seen before-- something Elizabeth just mentioned.
But at the same time, you really want to-- you need to sort of feed into their expectations, because otherwise they're going to shut down.
And that is why a lot of the YouTube channels, and a lot of the people who do this work for a living, have a genre that they stick to.
Smarter Every Day, ASAP, Vsauce.
You know-- hey, I always jump up from beneath the frame.
That's my thing, you know?
So you're showing people what they want.
At the same time, you're surprising them and taking them somewhere they've never been.
And it's kind of a paradox, but I think it's kind of a key paradox.
And then the last thing is, leave your audience wanting more.
Shortness, brevity, is a very important thing, particularly for this.
I mean, this is a series, Science Out Loud.
The nicest thing for Elizabeth would be if somebody said, oh my god, that was so great.
Click, next one.
Now if you think about it, this is very similar to a three-act structure, it's just a really compressed three-act structure.
Engage your audience-- the inciting incident.
Surprise your audience, and also showing them what they want to see.
This is sort of the development, which might only take 30 seconds.
And then, leave your audience wanting more-- some kind of resolution that takes them to the next thing.
The other element of all of this that's very important to just briefly mention-- and Elizabeth mentioned this earlier-- is that in online video, just as we saw with the Ohio State clip, a lot of the storytelling happens outside the frame of your video.
You want to, basically, be able to leverage that.
So people's expectations before they see your video play into how they perceive your story, their experience during, and then what they do after.
All of those are part of the story, as well.
So, you know, the naked celebrity picture-- well, that's kind of a story too, because you have an experience of Brad Pitt in one context, and then suddenly, wow, is that his butt?
That's a story.
Not a very good one, necessarily, but.
I also want to talk a little bit about protagonist.
I started out talking about one of the things you need is a protagonist.
Well, on one level, the protagonist is you, if you are the host of this video.
And if you think about the sort of videos Elizabeth showed, all of them, pretty much on one level or another, have a protagonist.
It's a very strong character, a very strong protagonist, whether it's crazy Russian science guy, or the Smarter Every Day sort of down-home but super-smart science guy, or Vsauce.
Love him or hate him, he's a very strong character.
So having a protagonist is really, really, really important.
That protects is going to be you in this particular instance, but there's also another protagonist that we should talk about, and that's your audience.
Because what you're doing is taking people on a journey through this learning, and some of the barriers that are going to be thrown up are barriers to their understanding.
Think about Smarter Every Day and the cat clip.
The whole thing is structured around, oh hey, now we know, it's the tail.
Oh-- no it's not.
No it's not, it's something else.
And that happens two or three times over the course of the clip.
That is, basically, a story.
That's storytelling.
There's a barrier to understanding-- oh, we think we've solved it-- no, we haven't.
It's like a car chase.
It's like breaking into a safe.
It's like a lot of things that you would see in a conventional movie.
So there are actually kind of-- in some ways, you can think of it as a dual protagonist.
You, the trusted guide, are taking your buddy, the audience, on this journey.
On a story.
To return to what I said earlier, I think of web videos-- great web videos are like great snack food.
They're very familiar, yet they're also sort of spicy and enticing, and after you have one you just want to have another.
And then you have the second one, and you just think, maybe I should have a third.
Hopefully the videos you make won't leave people with heartburn, which is what happens when you eat too many of these, but that is, to me, kind of an interesting analogy.
And if Interstellar, or some three hour opus that you pay $10 to sit in the dark to watch, is a seven course meal at a four star restaurant, you're making something that's just got to be really tasty and delightful for just a moment, and leave people wanting another one.
Just exactly like that, but somehow subtly different, too.
I'm not going to show any video, because I'm going to try to just talk about the videos you've already seen, rather than show.
So let's unpack stories just a little bit more.
I've made the case that you're telling a story, and you should never forget that you're telling a story when you do this kind of work.
Well, what are stories actually built on?
There's a whole list of things, and Elizabeth has mentioned a couple of them.
Emotion.
Now, I'm not necessarily talking about Gone With the Wind emotion, but certainly curiosity, identification, humor, which is an emotion.
All of those things really are going to play into what you do and are very, very important.
Character.
We've talked about character.
Conflict.
Again, not necessarily high drama, but there's often some kind of conflict.
The conflict may be, oh, you thought the cat was using his tail to spin, that's not the case.
But some kind of conflict, barriers to understanding, I think are very, very important.
Spectacle.
Again, you're not going to do Interstellar, you're not going to spend $100 million on CGI, you're not going to do The Hobbit.
But often-- it's another word for taking people a place they've never been.
Maybe you can get into the nuclear reactor here.
Intellectual engagement.
I think that's kind of the obvious one, the easy one that we can all think about when we're talking about educational videos.
In and of itself, I don't think it's enough, but it's certainly an important one.
And there are plenty of videos out there-- plenty of lecture videos-- that basically just depend on that.
And then the last two are two of my favorite ones.
The unfamiliar becoming familiar, which is kind of the classic science video.
But the familiar becoming unfamiliar, as well.
In other words, you think you know farts, but you don't, despite all you know about farts.
And in one context, you don't know that much, and here's why.
So I think both of those are really interesting structures for you to use to hang a story on.
Again, the first one, obviously.
I'm going to take this thing that you don't understand, and I'm going to make it completely-- I'm going to demystify it.
But a lot of science education is also about kind of taking something that seems really simple and showing just how complex, and intriguing, and unfamiliar it really is.
Now let's talk a little bit about visual storytelling, because that's obviously what you want to do here.
Lecture videos-- again, intellectual engagement certainly one way to go.
If you use OCW, as a couple of you mentioned, you know very well that lecture videos can be effective.
And in fact, lectures can be great stories.
We all have had experiences of people who are great lecturers, who tell great stories.
And those stories may or may not be visual, but I think there's certain things that distinguish visual storytelling that are worth talking about.
And this is a concept that I often use to just kind of get into to this whole idea, which Elizabeth will probably remember.
This is what we sometimes call the plastic elements of video or a film.
And sometimes I use film and video indistinguishably because-- it betrays my age-- video is film, film is video now, at this point.
What I mean by the plastic elements of video are, just simply, all of the things that we have control over when we make a video.
And that could be the casting of the actors, it could be where we put the camera-- that's kind of the obvious one-- the lighting, there's an endless list of things.
And, in fact, here are just a few of them.
Again, most people would list the camera right away, since that's kind of the "paint brush", quote unquote.
The lighting, how we choose to light something, or whether we're actually literally using lights or just using the sun or existing room lights.
Editing, sound, music, characters, the actors, locations, props, costumes, the list is really endless.
It goes back to the decisions.
Elizabeth mentioned decision-making.
All of these things are things you need to think about and you can control.
And they can work for you, or they can work against you.
Think about in Smarter Every Day, props.
The cat itself, of course, is a great prop.
Think about the stuffed cat at the beginning, which is that sort of fun moment of, oh my god, he's really going to drop a real cat.
That was a choice.
That was a choice, to add that stuffed cat, and it sort of made a fun little moment that brought you in for just a second.
Locations.
Very, very important.
I mean, the sort of lowest common denominator location is a chalk board or a white board for something like an educational video, but there are all sorts of other possibilities.
Costumes.
Again, doesn't have to be clown costumes or silly hats, or things like that-- although it could be.
But all of the other things that you can, or might be able to, bring to your video by making these choices.
Camera, of course, is a crucial one, and I'm going to unpack that just a little bit as well, although we could spend hours talking about film grammar.
So I'm going to just go through a few other tips to think about.
The first one-- and this is kind of like the bible of visual storytelling-- show, don't tell.
And it sounds kind of obvious, but it isn't necessarily that easy to do.
If we think about the sleep video, I think one of the things that I responded to in that video, that I thought could have been better, is that it's actually not showing us much.
It's telling us.
Elizabeth talked about the fact that the voiceover just read, like, a web article.
Like, a sort of "10 Facts About Sleep", and that's basically the case.
I mean, there is some really cool animation, and the animation is fun, and it helps, and it definitely supports the words, but together, it's really just this thing-- you know, I could have listened to it on my iPod.
I could have listen to it in my car.
The visuals helped a little bit, but it didn't really add anything to it.
The other way to do it, of course, is Smarter Every Day.
Which again, you may or may not have liked that.
You may have thought that it dragged on too long, and I agree with that certainly.
I think there were things that could've been improved, but there's nothing like dropping a cat, and showing it in super-slow motion, to get my attention.
So I think that was a great example of showing something.
This is just something that I wanted to mention.
Having worked in television, there was a saying that went around a lot when people were talking about a script, which was, "Oh, this is kind of a wolfpack deal."
And what was meant by that-- this was sort of an infamous situation from, I don't know, God knows when.
Somebody who's scripting a nature documentary had written this long, very beautiful, kind of flowery narration about "the ravenous pack, lopped across the frozen tundra in search of its prey"-- blah blah blah.
Like two pages of sort of beautiful prose, and at some point in the editing process, somebody was like, because they had that.
They didn't need to say all that stuff.
This picture was worth endless amounts of stuff.
So avoid your pack moments.
If you don't need to say something, you can show it.
There may be plenty of things to say, and I'm not saying that it should be all visual, but you don't need to say certain things if you're already showing those things.
Use your language, use your words, which are very important to do other things.
To reinforce, to bring your own perspective, to embellish the story, to bring people into your own mind, your own character.
You don't need to say the obvious stuff.
You don't need to do the wolfpack description.
So that's show, don't tell.
The other thing that's just sort of a fundamental tip is, video loves motion.
Moving pictures.
And Josh is going to talk about storyboarding and animation later in the week.
Clearly, animated pictures, moving things, things that reveal one thing or another in animation, are really important.
Same with cameras.
If you look at Science Out Loud, some of the other episodes, you'll see some really terrific moves with the camera, dollies.
All of those things add visual interest and improve your storytelling.
Locking down a tripod can also be very effective, but again, think of it as a plastic element that you can control.
Don't do it just because you didn't think about it, do it after thinking about what you're doing by either moving or not moving.
So the camera loves motion.
This is kind of the corollary to that.
And I don't know if people have a lot of shooting experience or not, and this isn't really strictly a class on the production techniques, but this is something I really wanted to mention.
Motion for the sake of motion is also not terrific, not fantastic.
Hosing down the room is that tendency that your uncle or your grandfather has when you give them a camera to just do this all the time.
And you're going, oh, hey, I'm getting everything.
I'm just getting everything here.
This is fantastic.
And in fact, what you get is absolutely nothing, because there's no intentionality.
There's no sort of thought to what's happening with the camera.
All you're doing is making everybody seasick and really pissed that you're not focusing on anything.
Motion is actually-- I think a better way to talk about it, in story terms, is reveals.
When you're moving things, you're revealing the next story element.
So if I pan from our camera person over to Josh, and Josh is looking over at the camera person, is going like, why is he doing what he's doing?
That's a story element.
Whereas if I do this, I'm not telling a story at all.
Same with animation, and I'm sure Josh thinks about this night and day.
I mean, how do you reveal your next thing from the thing you've got?
How you go from this idea to that idea?
How do you reveal using motion?
Because it is moving pictures.
So that's maybe a more pointed way of talking about motion in story terms.
Revealing things.
I'm going to talk just a little tiny bit about grammar, and framing, and stuff like that, very quickly.
This is just a still from an interview that I did a couple of years ago here at MIT.
Joel Dawson, a professor here.
And just a couple of things to just point out about it in terms of, sort of, just the way it's visually set up.
First of all, you'll see that the background and the foreground are very distinct.
We shot it so that the background is completely out of focus, and that's so that you know what to focus on.
That's the other thing.
So we talked about reveal.
The other sort of key crucial thing to think about when you're storyboarding, or creating animation, or directing is, how do you get people to look at what you want them to look at in the frame?
One way is to foreground the foreground and let the background drop off, either through lighting or through the focus, in this case.
Or through art direction-- there are all sorts of ways to do it, but doing it is really important.
The other thing is that this is a close up.
Sounds kind of simple, sounds kind of obvious, but it's making a choice to go in and to find out about this guy, this person.
It's not about the lab.
It's not just sort of this wishy-washy, moving shot of him wandering around his lab doing something.
We want to hear what this person is saying, what he's thinking, right now.
Just another quick thing to just talk about.
When you're shooting in this particular format, 16x9, there are a couple of really powerful parts of the frame that you can use, whether you're doing animation or shooting, and they tend to be where the thirds cross each other.
So that's a very powerful part of the frame.
That's also powerful.
These are powerful in slightly different ways.
And you'll see his eyes, like, dead-on in the crosshairs.
If we're people who read left to right, in sort of the Euro, European tradition of writing, this tends to be the most powerful point, because we read frames left to right, because we're used to our eyes used to going that way.
Not saying you have to follow this rule.
You can break it, but it's just an interesting thing for you to know.
When you're framing things, when you're creating animations, there's a tendency when you start to think oh, dead-center will be really great.
But it's actually not the case.
A little off-center and in sort of the thirds of the frame is a good way to go.
The other thing just to talk about quickly, in terms of making choices, think about-- and this will really come up when you do your storyboarding exercises with Josh-- think about what the kinds of shots you choose are doing.
Wide shots say one thing, close-ups say other.
Starting with a wide shot and coming in, that's a reveal of something.
But starting in a close-up and going wide could also be a reveal of something else.
I think we can imagine what kinds of stories might come out of these two frames, and how the story might be different if we started here and went here, and if we started hear and came out.
So make choices, and think about what your choices are actually going to do in terms of the story.
Another example of that.
Camera angles framing.
Low angles looking up, that gives you a certain feeling, it gives you a certain kind of power.
A sort of master of my domain feel.
The opposite, looking down, kind of the opposite.
These are just two examples, and I'm not going to go on and on and on and talk about others, although we may get into it later in the week.
The point is that the camera is one of the key plastic elements that you have at your disposal to tell your story.
And the medium shot, just kind of center frame, doing your thing, is kind of the fall-back.
You don't have to do that.
There are any number of other choices you can make.
And when you start to go and choose locations, and choose props, and choose elements like that, how you shoot those things and how you shoot your characters becomes very, very, very important.
And then the last thing that I'm going to talk about-- and I'm kind of racing through this because I want to just make sure we have time for questions and stuff-- I'm just going to be a little counter-intuitive.
We've been talking about visual storytelling, but just don't short change your sound.
Sound is often more important than the picture.
And one of the other things that I think-- Elizabeth talked a little bit about production technique, and quote unquote "bad technique" or rough videos, versus slicker TV-style videos.
One thing that's very, very, very hard to overcome is poor sound.
And even if your video is a little out of focus and its shaky, if it's really compelling, that's fine, as long as the sound is good.
But as soon as the sound is bad and hard to listen to, you're going to have a really hard time pulling people back.
So even though this is mostly about visual storytelling, I just want to point that out, that that doesn't mean that the sound is not important.
In fact, the sound is extraordinarily important.
And it's both a technical thing but also a plastic element that you can use.
Whether it's music, or sound effects, or the way you treat dialogue, all of those things are really important.
So just to contextualize this a little bit, I know that Elizabeth's co-instructor, George, is going to be doing a scripting workshop with you, and he's going to be focusing a lot on finding your voice.
Which is really, really important, and goes back to the idea of character, and the protagonist, and authenticity, and a lot of the things Elizabeth was talking about.
But as you do that, as you do all of those things, remember that ultimately you're telling a visual story.
And that once you, sort of, hone your story, hone your voice, you want to put that in a visual context.
You don't have to have the words carry everything.
And in fact, the words may end up doing other things than conveying the core of what you need to do.
So that's it.
I kind of tried to race through it, because I didn't want to keep people for too long.
First of all, any questions or thoughts?
I just had a thought, just a question.
It was interesting, you mentioned that the-- I'm sorry, the name's escaping me-- the whiteboard video that you were critiquing earlier--
ASAP Science
Oh, the ASAP Science, yeah.
I liked it, I didn't mean to dump on it
No, I think it was really important and a really interesting point.
So then I was thinking, okay, what would have been more effective?
How could that have been turned into some sort of more storytelling exercise, or something that was more engaging?
And I think it would be an interesting question, just, how could that be better?
And my feeling was, would it be better to do something where you actually had someone not sleep for overnight, or for two days--
Well I think that's-- yeah, no, go ahead, yeah
Something like Supersize Me, where you have someone actually engaged in this thing, and it's more experiential, as opposed to being didactic I guess would be the distinction there.
So that was just one idea
I think that's a really good point.
I mean, that's kind of the Smarter Every Day approach, which I respond to personally better.
On the other hand I think that there are graphics that tell amazing stories.
I think data visualizations that I've seen recently, in particular, just can be extraordinary storytelling devices.
So it's not so much that I think that it was animated, and somehow that was not as effective.
It's that I think the process was write script, and then come up with stuff to fill in space.
So yeah, I personally respond more to the ones that are more experiential, where you have people-- that may also be supported by animation-- but there is someone taking you through the experience.
But I do think that there are lots and lots of ways to tell really, really dynamic and really interesting, good stories with animation
That's good to hear
Oh no, totally.
And you know, we've all been there.
I've been there, too.
You have something that you have to make come alive.
I think they did a great job of making it come alive.
I think the animation was really effective.
It's just that I think the approach was to write a script without necessarily thinking about the visual medium, and then to take the visuals and layer them on top as a separate part of the process.
So anybody else have any-- I mean again, I know I kind of raced through it.
How many people-- I mean, do you have video experience?
How many people have some kind of video experience?
That's always a really good question to ask.
No?
Nobody?
Well that's cool.
That's great
You have bad habits
Absolutely.
Absolutely.
I think, just to go back a little bit, I think that whole relationship between web video and TV is a really interesting one.
It's a really interesting one to unpack, because again, the sort of conventional storytelling techniques that we're used to seeing on TV are not going to be effective for all sorts of reasons.
It's also worth mentioning that of course TV has changed too, radically, since connections.
Game of Thrones doesn't open that way, two minutes of somebody wandering around a house.
But at the same time, it is a different genre, and really a crucial thing is the thing that Elizabeth alluded to, and I mentioned at the beginning, that I think you're telling a story, but your story extends beyond the bounds of your video, or can extend beyond the bounds of your video.
It basically can begin with what people's preconceptions are before they start watching, and it continues on after the video is done.
And I think that is a really crucial difference.
One suggestion that I would make, whenever you write something this week-- and you're going to be writing treatments, and pitches, and scripts-- think about it in purely visual terms.
One of the early exercises that I had to do, when I was in grad school, were silent films, and films or versions of scripts that had no dialogue, had no sound at all.
And I think that's-- I'm not saying that you should do that as a technique in your finished piece, but it's a way of bringing up stuff that you can then apply to your video later.
So if you were going to do something that was purely visual, that was going to convey what wanted to convey, how would you do it?
And what might you create, or invent, or think about, through that process, that you can then use to inform your script.
So just a way of thinking about what you're going to be doing this week.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we have been talking a lot about, sort of, the theory of things, but I know you guys want to get into the practice.
At the end of class, we have a lot of plastic elements in the back that Ceri will help you guys check out.
Some cameras, mics, headphones, tripods.
If you guys want to just use your iPhones for tonight's assignment, that's totally fine as well.
Chris brought up a lot of really good points of just framing, practicalities of filming.
I do want you to consider, though, that web video breaks a lot of these rules as well.
So you'll notice that Sideshow doesn't do the rule of thirds framing at all.
Hankering just--
All rules are made to be broken
Yes, yes.
But he does that very intentionally.
A lot of it is because-- the way his videos go, he'll look to the side and a little image will pop up.
So I think part of what he does that is to leave room for some of those visuals.
And Vsauce does it because he's just super up-close.
It's a little weird to do a rule of thirds when you're so up-close because you don't have a background to get rid of so much.
So don't feel like you have to adhere to a rule just because we said, you have to do this, you have to do this.
Thoughtfulness is one of the values, so as long as you justify it in your reflection or in your assignments somehow, it's totally fine.
So for today's assignment, in addition to the two readings that are online-- and don't worry, they're pretty short and they're fun-- we're going to have you guys make a video on, basically, a circus sideshow pitch about a video that you're eventually going to make for your final project.
And it's OK if this ends up changing, it's just an exercise to get you guys started, to get you familiar with the camera.
Don't worry about doing crazy camera moves or anything.
If all you want to do tonight is to set up your tripod and mic up, and deliver your pitch, that's totally fine.
We just want you guys to get familiar with all the equipment.
But, basically, it is going to be selling an idea or topic about science, technology, engineering, or math that should convince someone to want to watch your eventual video.
And again, it's OK if you change the idea.
I do want you guys to brainstorm three ideas in addition to the pitch video that you make.
You don't have to post them online or write them down.
Just have them in your head for class tomorrow because we're going to have some very special guests speakers first thing tomorrow, who will help you with these pitches.
So to give you an idea of the type of thing that we're looking for.
Again, don't necessarily copy this video exactly.
[VIDEO PLAYBACK] -What is the most astounding fact that you can share with us about the universe?
-The most astounding fact-- -The most astounding fact.
- --is the knowledge that the atoms that comprise life on earth.
The atoms that make up the human body are traceable to the crucibles that cooked light elements into heavy elements in their core, under extreme temperatures and pressures.
These stars, the high mass ones among them, went unstable in their later years.
They collapsed and then exploded, scattering their enriched guts across the galaxy.
Guts made of carbon, nitrogen, oxygen, and all the fundamental ingredients of life itself.
These ingredients become part of gas clouds that condense.
Collapse.
Form the next generation of solar systems, stars with orbiting planets.
And those planets now have the ingredients for life itself.
So when I look up at the night sky-- and I know that yes, we are part of this universe.
We are in this universe, but perhaps more important than both of those facts, is that the universe is in us.
When I reflect on that fact, I look up-- many people feel small, because they're small and the universe is big-- but I feel big.
Because my atoms came from those stars.
[END PLAYBACK]
So obviously you don't have to do a super dramatic thing like Neil deGrasse Tyson did.
Again, don't feel like you have to go home and contemplate on life's mysteries, and how evolution is just this beautiful thing that happens on earth, and it's an amazing thing or whatever.
But the reason why I wanted to show you that video is because the actual plastic elements of making that were pretty straightforward.
Pretty simple, just two cameras on a tripod.
Nothing too crazy.
But he still, hopefully, manages to engage you somewhat.
I mean, I personally wanted to keep watching, keep learning about it.
That's partially because of his persona, which some people find off-putting.
So again, it's a matter of taste.
What he does here is just, sort of, rattle off this theme of-- I think the thesis of this whole conversation that they have is that you take this misconception that we all think that we're part of the universe, but really the universe, and all the things that make it up, are inside of us.
He has this sort of fact, or nugget, that he builds upon to create this bigger narrative.
All right, so for Science Out Loud, George and I just did an episode on snot.
And the way we started it way was-- one of my former professors here, Katharina Ribbeck.
Her lab studies mucus, and I was reading about her in Popular Science, because she just got this big feature in it.
And one of the facts they talked about was that your body produces over a gallon of mucus a day, which I thought was, like-- how does that happen?
That's disgusting.
We think of snot as the stuff that oozes out of us when we're sick, and the stuff that we blow out into tissues and throw away.
But actually it's so important that our body makes over a gallon of it a day.
And that it does more than just come out of our nose when we're sick or when we're crying.
That it's doing things, like providing moisture for our bodies, which doesn't sound like a big deal, right?
Until you think about your body without mucus.
That your eyes would dry out, and you wouldn't be able to swallow because your esophagus would get ripped up every time you tried to eat something.
And then you think of mucus of being more than just this sort of bodily waste that comes out of you, but really this amazing thing that takes care of you in a way that's so important that you make over gallon of it a day.
And that sort of became the pitch the circus sideshow, step right up type of thing that informed the rest of our video.
And so that's the kind of thing I want you guys to be thinking about tonight, and want you guys to record.
Now we very well could have had you guys just write a pitch, but I think it would be more fun to make a video.
I mean, I think it's good for you guys to get used to using the cameras and the plastic elements.
So again don't worry about-- are my camera movements good enough?
You don't even have to move the camera if you don't want to.
Feel free to play around with it if you want.
But really focus on clarity for this assignment.
So set up the camera in such a way, and set up the lighting in such a way that you're very clearly visible.
We are going to give you these little lavalier mics, so make sure that your audio's very clear.
Those are going to be the priorities for this assignment.
And then make your pitch very engaging, and think of three backup ideas just in case.
Does that sound good?
I had one more example, too.
We can show it.
[VIDEO PLAYBACK] [MUSIC PLAYING] -Hello, this is Hank Green, and welcome to SciShow.
You might have noticed I talk a lot about how great plants are.
And why shouldn't I?
They're autotrophs.
They can harness the raw, untamed power of the sun and use it to make food for themselves.
Pretty impressive.
So why can't animals do it?
Well oddly enough, some of them do.
And in just the past few years, researchers have been discovering animals that seem to be able to harvest sunlight for energy.
Take for instance, the eastern emerald elysia, a sea slug that looks like a big floating leaf.
It lives along the East Coast of the United States, where it just hangs out eating a specific kind of yellow-green al-- [END PLAYBACK]
So I just wanted to show you that as an example of a style and delivery.
It's very different from the Neil deGrasse Tyson, but again, same concept.
So don't feel like you have to be Neil deGrasse Tyson.
Don't feel like you have to be Hank Green and talk like this, and jump back and forth between, like, closeup and wide screen.
But hopefully that gives you an idea of what you guys need to make tonight.
If you guys have any questions, I'll stick around after class.
You can always email me too.
So for tonight, stem pitch video.
Ceri's going to come and talk now about what that process exactly looks like.
But basically, always look at the online version of the syllabus, because I might tweak some things day to day, but the stuff for day one is totally up-to-date right now.
You'll post yours stem pitch to Tumblr with the hashtag "dayonepitch" and a hashtag of your Kerberos ID, so that's-- all the SUTD students, you guys have an MIT ID, right?
OK, so it's just your username.
And then you'll post a daily blog, and that's going to happen every day.
Just reflections of what happened in class.
If you want to take pictures during class, if you want to record video tomorrow, feel free to.
Tomorrow will be a great day to record video, just saying.
And then there are two readings that I'd like you guys to annotate on Annotation Studio, which Ceri's going to show you right now, as well.
And then just bring three other pitch ideas to class tomorrow.
Did everyone get the email about class timing shifted up tomorrow?
So if you guys want to show up at 12:10, that would be good, because our guest lecturers are coming at 12:15.
And we'll end class at 3:15 tomorrow
This is Annotation Studio.
It's on annotationstudio.org and it's where you're going to be doing your reading assignments.
So what you need to do is just go to the website.
Click sign up for sign in.
And it's a really easy registration form.
First name, last name, email, I use my MIT email.
Affiliation, just put MIT.
And a password.
And the most important thing, so you can get the assignments in the first place, is to type in the class.
So you just need to do 20219, press Enter, and it'll be blue, to like, confirm that you're in it.
Read and accept the terms of service, check that box, and register.
And so I'll show you what it looks like.
So this is what you'll see when you sign in.
You won't have any created documents, but you'll have assigned documents.
The two that we want you to read and annotate are the first two-- "You Can't Make It on YouTube Anymore," and "This Will Revolutionize Education."
You'll click on it, and it'll bring you to the document itself.
So for example, this is the first one.
And you'll be able to read along and highlight it, and add in your thoughts as you go along.
So for example, you're reading it.
You're reading about Hank Green, and how he was invited to New TeeVee Live.
I was starting my own online media conference, and maybe you're thinking, well what is that?
So you can say, does anyone-- [TYPING] And then add some tags.
I don't know what tags really do, but then everyone can see that annotation and who did it.
And I'm not sure, because we haven't played around with this software too much, but you probably will be able to comment on each other's annotations.
So if for some reason someone poses a question like this, and someone knows the answer, it's Vid Con.
Then they can respond.
You can add pictures, you can add links.
And it's just really handy because then you can see people's idea flows as they go through the document
Just so you guys know, we're grading the annotation just on a three point scale.
If you put effort into it, you put marginal, if you did it at all.
Don't feel like that you have to spend hours and hours on this.
These readings are pretty fun, I mean, one was posted on Medium.
It's not a super dry academic platform.
And the other one's a YouTube video.
So just respond honestly.
Don't feel like we're expecting huge analyses of these, but we do want to see you guys creating it and taking [INAUDIBLE]
OK, the other thing I was going to show you guys.
How many of you have used Tumblr before as a platform?
OK, so a couple.
So that's going to be our main blog, I guess, for this class.
I've sent you guys all an email invitation to join the website and be able to use it.
This is what the main website looks like, you should have already looked at it earlier today.
The main things are that there are links on this top bar to the stellar, which we probably won't use, Annotation Studio, and an ask box.
And so I'll be on the Tumblr monitoring it all the time.
So you can ask questions there.
There's lists of all our Kerberoses, so we'll organize our posts with your MIT ID.
Then it's also organized so you can look at specific assignments, like day one, daily blogs, and things like that, which are all sorted and dependent on you guys tagging your posts correctly.
And then there's also reference material.
This bottom tab on the left side bar is probably most useful for you guys.
I can't scroll down right now because there aren't enough posts, but it has things like a list of all the assignments that Elizabeth will post.
Answered questions, and a how to use Tumblr and upload videos on YouTube, which I'll go through, hopefully, really quickly, so you guys don't have to stay too long.
For those of you who haven't used Tumblr, when you login, when you accept the invitation to join the blog, it'll look like this.
This is your dashboard.
I have multiple accounts, and so if you have multiple accounts, you have to go up here, and choose MIT 219.
And this is going to be exactly what your dashboard is going to look like.
And so, from here, you can make different types of posts.
We're probably going to use two most often, text and a video.
And so for a text post, you just click Text post.
You can give it a title.
Put something in the body.
And we request that you tag things with specific names depending on the assignments.
So Elizabeth gave examples of what we want you to do for your daily blogs, which is "day one", or whatever data is and your MIT Kerberos.
And so for example, if this was my daily blog for today, I would type in "day one", enter, and then my Kerberos, which is ceriley.
And it'll be different for your video that you end up uploading to the site.
And you can include images in this.
You can include other things, like links if need be, if you find that necessary to your blog.
And click Post.
And then you should be able to see it right here on your dashboard, and also on the website if you go click there.
OK so if you're making a video post-- how many of you have-- have any of you uploaded a video to YouTube before?
OK, this is more familiar
Is there a way to just upload directly your files to Tumblr, right?
There is, but there's a five minute limit per day
OK.
So for tonight's assignment, if you just want to upload your raw video file to Tumblr, that's totally fine.
But eventually, for some of the drafts, you may need to upload it to YouTube
So I'm just going to go through this really briefly.
YouTube.com.
If you have a Gmail account, you have a YouTube account automatically.
They're linked because Google owns everything.
Just click Upload in the upper right hand corner.
Select files to upload.
I have an example video that I shot this morning.
OK, so here you have basic information.
You can choose a title.
A description.
And this is-- you can tag it with anything you want.
This is the key part, I think, to YouTube, is where-- how you want to upload it.
So public means anyone can search it through the search bar, through Google searches.
It'll be available to the public.
Private means you have to enter certain email addresses, and those are the people who can access it using their Gmail accounts.
I like doing unlisted personally because then, if you have the link to the video, you can access it.
But if you don't have the link to the video, you can't see it.
It keeps, like, my draft videos like this one, no one will want to watch it.
Eight seconds of me talking to a phone camera.
Private enough from, like, the rest of the world.
But you can still post the link places, like the Tumblr, and have it be accessible.
And so I like unlisted, but you can do whatever you want with your videos.
So you saw that the video uploaded, and then it takes a little time to process.
And that's just YouTube doing things so that it can be played.
That's my video.
Now it's available, the processing is really quick because it was eight seconds.
And to upload on Tumblr, just copy the URL or the embed code.
Make a video post.
Paste it in here.
There's no thumbnail yet because I haven't chosen one.
YouTube sets one automatically after a couple minutes.
I can tag it with day one, and my username, and set a caption.
And post it.
And it will appear on my dashboard, and on the website.
So this is just like the general process.
Under the References tab on the Tumblr, there's more detailed if you forget any of these steps.
Or are having any trouble, or feel free to email me at any time and I could help troubleshoot any steps of this process.
If you prefer Vimeo, you can also upload it there and just paste in the URL.
If you already have Vimeo account, or something like that.
So I know we held you guys to the last possible minute, so we'll let you go ahead and go
If you need any equipment tonight, Ceri and I will stick around and check out stuff to you that you can keep for the month.
If you want to just use your iPhone tonight, that's also fine.
Just make sure you're lit well enough, and that we can hear you well enough for tomorrow.
And if you have any questions, we'll stick around after class.
So see you tomorrow at 12:10.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hey guys.
So, these are our special guest lecturers for today.
These are sixth graders from [INAUDIBLE].
Do you guys want to go ahead and find a table to sit at?
[INAUDIBLE] And everyone will rotate.
[INAUDIBLE]
Hi.
Music matters.
It really does.
Are you a musician?
Yep
What do you play?
I play the snare drum, the bass drum, the tambourine, pretty much all the percussion instruments
The triangle
Triangle
That's really cool
All right, guys so, before we get started, the way this is going to work is we'll give you five minutes at each station with your consultants.
And during this five minutes-- this is for the MIT students-- go ahead and introduce yourselves.
Tell them what you study, maybe what year you are, what you're interested in.
And then, if you feel comfortable showing them the video pitch that you made last night-- [LOUD RUMBLING]
We have this to deal with, so just know that's out of our control.
And we're really sorry.
They're doing some construction over there, and it's going to be a little noisy.
Sorry
So go ahead and show them your pitch if you're comfortable with it.
If not, you can just tell them your idea.
And then workshop some of the backup ideas that you thought about last night.
And the Cummings Middle School students know exactly what criteria you're looking for, right?
So you can use the rubrics that we gave you.
And at the end of the five minutes, we're going to have the MIT students rotate.
So Nathan, you'll go to the back.
Paul, you'll scoot here.
Everyone will just shift that way.
Does that make sense to everyone?
We'll try it out for the first time.
And if anyone has any questions, you can always come talk to one of the instructors.
OK?
Does anyone have any questions about how this is going to work before we get started, or what your job is?
Does anyone have a question about that, either?
Your job, it's a very important job.
You're critics, right?
All right, then it looks like everyone knows what to do
So, I will go ahead and start the timer for the first five minutes.
And if at any point you have a question about what you're doing or just a clarification thing, you can come find one of us.
Have fun
So, introduction, I study computer science.
Anyone know what that is?
It's kind of like programming kind of stuff.
And so, the kind of stuff I like to do is I make games.
So this is kind of like [INAUDIBLE].
It's kind of like Fruit Ninja.
Have you ever heard of Fruit Ninja?
Except, because snowing today, you'd slash the snowflakes.
It's kind of the same, but this time you get a lot of combos.
And if you slash the red one, you lose time and lose points.
And you eventually lose the game.
That's the kinds of things I do
Did you make that?
Yeah.
I made that
Oh.
I didn't--
I've got Fruit Ninja on my phone
It's kind of like Fruit Ninja except don't slash the red snowflakes.
And it's OK if the snowflakes fall off.
Oh, and the blue ones give you these mega explosions
Oh.
That's so cool
Yeah.
So I have this idea.
It's just, without programming, how can you make computer games?
That's what I was thinking of off my head.
What do you think of that?
I don't know
Do you play games a lot on your phone?
Yeah
Oh yeah
What's your favorite?
I like a lot of them.
Plus social media
Ah, social media
Yeah, that kind of thing
I don't really have a favorite
Oh, I see
I play Madden Mobile and Clash of Clans
Ah, Clash of Clans.
Yes.
Totally addictive.
Yeah, so it'll probably be about, like, how can you come up ideas to make games?
Because, honestly, the people who are trying to make games, I'm probably not as smart as any one of them
Yeah
Because you play the games.
You know what people like to play.
And it's not me, it's you.
You know what you like to play.
So if you could make the games that people of all ages could play, that would be great, great right?
Yeah.
OK, well, just another curious-cat question.
Do you know what a hacker is?
What?
No?
Hacker?
Oh, hacker?
Yeah, hacker
Oh, yeah.
Someone who has to invade the firewall thingy.
Or get through the security bars
Yeah.
Would you want to know about what is a hacker and what they do?
No, I don't want to
No.
OK. Then let's just scrap that.
How about a curious question.
Everyone knows Google, right?
Yeah
But what do they do?
Search engine?
OK, yeah.
And then, what does that make them?
How are they so important in our lives?
Well, because, when we do our homework, we kind of look up on Google a lot.
Like to [INAUDIBLE] stuff.
Like history
OK.
Cool, cool
Helpful.
Very helpful
Yeah, OK. You can also ask Siri on your phone or something, like your little button.
I'm actually not from the US, I'm actually from a little country in Asia called Singapore.
So I'm just curious.
Do you all study programming at your age?
Like any kinds?
No
Oh, OK. Me, neither.
I have never studied programming until I came to university
Yeah
So what do you think programmers do?
They create things?
Yeah, they help build games and code and stuff
If you were to be curious about a programmer or what can he do, what would interest you, games?
What else?
Anything else?
I pretty much just play games.
I can't call on mine, because it's an iPod.
I don't have an iPhone.
I only have an iPod
Yeah
All right, you have two more minutes with your consultants before we move tables
So out of the random ideas I threw at you, which one would you be interested in in a possible video?
Football games, pretty much
Football games
I'd say, yeah
I don't know.
I think something that has to do with.
What's the nearest thing you have in your mind that always nags you about computers, I guess?
It slow sometimes
It takes too long
It's too slow
Yeah, too slow
Like, even when you have Wi-Fi in range and you're near the router, it's always really slow
Oh,
Oh yeah, and Wi-Fi.
When I'm in my car, I can't play on the antenna and stuff.
I can play games that don't need antenna.
I don't like Wi-Fi.
They should have built-in Wi-Fi on everything.
So that would be cool
OK. You know Pixar.
You like the movies?
Yeah
I love The Incredibles But that's really old, but I still love it.
And have you ever wondered how they do it?
It kind of looks different from the normal--
Yeah
Yeah, so they actually use some computer software to do it.
It's called 3D animation, like three-dimensional.
Is that interesting?
Yeah, I would be interested in that
Yeah, like how do you draw, but on the computers?
Yeah, like do they have to draw all the little pictures of every movement?
Ah, that's the key.
No, actually the answer's no
Oh, I thought they did
It's pretty cool, right?
What they do is they make a teddy bear in the computer, or some kind of 3D image of it
Yeah
And they put bones in.
Just like how you put bones in humans that he was programming, you put logical bones.
And then you can shift the bones, and they're like puppets, you know?
Yeah
Yeah.
This is kind of like what they do all the time
All right, guys, let's go ahead and shift over
OK, so which one of the ideas--
I like the last one
The last one
Yeah
Yeah
Yes
The last one the most?
All right.
Thank you, guys, you're so amazing
What are your names?
I'm Ronald
And I'm Michael
I'm going to sit in with you guys, too.
My name's Sari
Hi
Hi
I'm Nathan.
I study something that's a mix of chemical engineering and geology
Geology?
Mhm.
I'm a second year.
I have two ideas.
It looks like my main idea has been scrapped at this point, but I'll still mention it to you in case one of you is fervently interested in it.
I was interested in possibly doing something on how and why fruit decomposes.
But I guess my backup idea is a bit more interesting to most people, which is how you can tell if there is magma lava under a surface and how you can detect and map it.
And then you might be able to say oh is this place in this scope having some sort of eruption at some point or not.
So I have to work on the hook.
But I don't know.
I wanna hear from listeners
Which one of those ideas do you think is more interesting?
What was the first one?
How and why food rots and decomposes
I would probably do the lava stuff
Yeah
It's more cool
Yeah
You could save lives
Yeah.
So I don't know exactly what to say about that
What would you be expecting?
If you wanted to watch a video about lava, what would you want to be?
A diagram of the Earth, probably

And have the lithosphere and asthenosphere and the layers.
And then having probably, like you said how you can detect if there's lava under a surface, probably having a place that already does have lava under the surface.
And like showing the magma chamber, and then stuff
It sounds like you guys have already done a unit on volcanoes pretty recently.
Is there any questions that you still have after that that you might want to have answered?
Like, really cool facts about volcanoes, or things like that?
I know it's kind of a hard question to answer.
Or the coolest part of the it unit that you remember?
How it looks.
I've seen videos of volcanoes erupting at night, and that was pretty
The question I have-- there's some volcanoes that are big but they have small eruptions, and then they do a big one.
And then there's other volcanoes that just have huge eruptions every time they go.
Why would some volcanoes have big eruptions and some have small, small and then a big eruption?
All right, so, here's what I would say.
Of the volcanoes that can have smaller eruptions, it's actually based a lot on what the makeup of the lava from the volcano is.
I don't know if you covered the difference between andesitic and [INAUDIBLE] volcanoes.
Basically, glass is made of something called silicon, silicate.
It's an element, and volcanoes that have a lot more silica in it tend to be a little less dense.
And ones that have lower content are a lot heavier, the magma.
And so, as you might expect, the ones that are heavier, when they erupt, it doesn't just shoot out everywhere.
It just kind of bubbles over the side and it's more viscous and it flows slowly.
And so those ones often will be the ones, like the big mountains that kind of spread out this way a lot
Sort of like a shield volcano?
Yeah.
Those are the shield volcanoes.
And so the lava type causes it to have a lot more smaller eruptions, because it takes a lot and build up of fresh air if you want to have a big eruption from that.
Whereas, when you have a higher silica content, which means it weighs less, it's going to explode more.
And it's going to have more violent eruptions.
And there's other factors as well, like water content.
And at what depth the magma chamber underneath the volcano is
Yeah.
Another question.
Let me see
If you don't have one, I have one
All right
All right, guys, let's rotate one more time
The amount of-- which one was dormant for a long time and then blew it's top and it was crazy?
Mount Saint Helens?
Yes
Yeah
If volcanoes are dormant for a longer period, when they erupt, will they have a violenter eruption than ones that are dormant shorter?
I can't actually answer that question.
I know what causes volcanoes to go more dormant, what might make them not dormant.
But I can't really say if that would effect it
One thing about math that you have really enjoyed or really hated.
Or two things.
So we'll start.
Go left to right
Wait, what did you say again?
So name, favorite subject, and things about math that you you have really loved or really hated
OK. My name is Brian.
My favorite subject is gym.
And I hate math, because it's boring
OK. Actually, I guess it's a kind of new direction, but a lot of people are actually studying the math of sports.
So they go out and do really weird things with sports, like throwing basketballs at, like, places where you shoot an arrow, and they do math with it.
So that's kind of a connection, gym and math getting closer together
My name's Matt.
I like math.
You hear that, Brian?
And what was the third thing that I have to say?
Some things that you really enjoyed it about it
I enjoy solving variables and equations and graphing things.
That's good
You're ready for high school
Burn
My name is John, and my favorite subject is--
History
History.
And I personally do not like math
Hi, guys
What?
If I had to say what I like about it, I'm really into fractions, kind of.
I just like the idea
OK. Do you know what about the idea?
I've never heard anybody say I like the idea of fractions, so that's really interesting to me
Like, things like this table being broken up into thousands of pieces.
I don't know
Now, really wacky things that mathematicians do later on is that they don't just break the table into a thousand pieces.
That's too easy.
They break it into an infinite number of pieces.
And then they see what they can do with that.
And, kind of related to that, mathematicians have proved that you can take a ball.
You know, just volume one.
One-gallon ball.
And you can cut it in such a way that you can make two one-gallon balls.
So you can make two out of one using math
That's cool
So that's some other idea
You like math now, boys?
I mean, if you think about it, math people do really cool things.
There was a study once where they analyzed the language of King George and Kanye West.
And they compared them.
And this was an actual study, like they put time into it.
So I thought that was pretty cool.
But anyway, that's not what my pitch is about.
And I'll just say it real quick.
Not going to show the video, because the sound is terrible.
What I want to talk about in this video is just the question of is math real and true?
Because what mathematicians learned in the 20th century, there was a young man named Kurt Godel who proved that--
One more minute, guys, before we need to switch again
--you can never prove if math is true or not.
So the arithmetic we learned in school, we think it's true, because it's worked, but it may not be.
There's no way to say for sure
Wait, is Kurt Colonel the guy who made Kentucky Fried Chicken?
No, it's Kurt Godel
Oh
I mean, I think it would be really cool if he did math and Kentucky Fried Chicken.
That would be the best
On Family Guy, he went to Kentucky to get chicken.
And he's like, is whatever his name is that sounded like what you said.
And he's like no, he's dead.
And he's like the something.
It was so funny
I'll have to look and see now.
Did any mathematicians open fast food chains?
That's probably an interesting question.
I don't think anyone's asked it before.
But, that said, any thoughts on the idea?
I'll get a thumbs up, thumbs down, or thumbs middle
Rotate
All right.
Thanks, guys.
It was nice meeting you and enjoy playing or not playing
I want an aha moment, a eureka
I think you want to get away from instruction and more of concept.
Right?
So see if you can think of a little narrower focus than in these big, big, big concepts.
And that might help you with your next group
So some of them are pretty similar, like the robots?
Yeah.
So we're teaching them.
They're scientists, and we're teaching them how to make-- have you ever looked on cool videos?
Have you seen Charlie the Unicorn, or you've seen--
Oh, I've seen Charlie the Unicorn
Charlie the Unicorn.
Or maybe you've seen-- I used to work with kids your age, and they loved that one.
It was about this little snail, Marcel the Shell.
Have you seen that one?
No?
Well, you see these little YouTube videos, and everyone's like, oh, that's cool.
You've got to watch that too
Oh, like the 21 kid?
Which one?
The 21 kid that says 21.
Everybody in America says 21 now
Oh really?
Is that the it one you have to watch?
21?
It's a Vine
And then they make it into a video.
Or there's a series of shows on YouTube called ASDF.
It's about this little stick figure
Right, so you guys see them, and then you're like hey friend, you've got to see this too.
And then you get everyone watch it, right?
We're trying to make those really cool, short videos about science topics.
Right?
And so we're going to make them free, so that it also teaches people something while it's also cool and fun.
And so what he's trying to figure out is what in his world of computer science would be cool enough that you'd be, like hey friend, you've got to check this out.
It's really awesome
I think the hacker one would be good for people out age
To understand how someone hacked your account?
Yeah, but then you don't want kids to hack people
Well, that's the fear, right?
You don't want to end up giving kids instructions on how to hack, because I don't think the government or anyone else would probably like that.
But that's a fear, right?
It's just making sure that we're not actually giving instructional videos on how to destroy other people's credit cards
Google might be good
Because then you could just hack this ATM and get all the money
How to protect yourself, like make sure your money's safe would be interesting to you?
Like how to protect yourself from hackers, how to protect yourself
How to protect yourself, instead of destroying someone else's something?
Yeah
Yeah
That's a nice way of turning it around
I have a question.
So say if someone lost their wallet, and they had their credit card in there, and it can access to the bank.
If someone found it, and the person to access, do they end up giving them the money?
Sometimes, and that's why credit cards are really good at-- do you know anything about this in your work, or no?
Credit cards?
Protecting credit card identity and all that?
But don't they have your picture or something, so they know it's you?
But online, it's really easy to use another person's credit card if you know all the information
I get called often from my bank saying, did you make this purchase, or did someone else?
But that's an interesting thing for people who do a lot of shopping online.
How do you protect yourself?
If somebody else took your money, would you get that back or no?
That's why, if you catch it quickly with your bank, sometimes they can quickly stop it before it ends up getting charged.
to you.
But that's an interesting idea.
How, if you're buying stuff on the internet, what's actually happening to protect yourself?
What if someone did catch the person, and the person already used all the money, and you can't return it after that, did the bank end up giving their money to the person?
Do you ever wonder what happens when you type in the credit card number, and how the money actually gets where it needs to go?
Oh, they track them down?
Have you ever wondered about how that happens?
No, not really?
You just care about getting your package in the mail?
Does the ATM make its own money, or do they put it in?
Do they just put the money in there?
Every country's different, but at least here, there is a central office that makes money.
And then they just distribute it into different places.
The ATM doesn't make the money.
That's a good question.
But, an interesting thing for you to think you think about is what within these topics more specifically?
Because the hacking is a cool topic, but you're right It's a dangerous one.
And we'd want to make sure we had the right angle, so that we're not accidentally giving sixth graders the tools to be able to hack into whatever account, right?
That would be scary
Google might be a good one
Google?
Could be, right?
How does Google actually work?
How does it find your result?
Like, if you want to learn how zebras have stripes, how does it actually give you an answer?
Or how does something spread, like a virus?
That's a cool idea, too.
Something to think about.
Keep thinking.
You're on it.
Good, good, good
Virus
And you are?
Anama
Those are some really cool names
Thank you
And?
Gianna
Gianna
So, we're going to have to move a little quickly, because we've got two teams to talk about, so--
You can go first
All right, introduction first.
I'm a first year here at MIT.
I like to study math, physics, and education.
I've said it too much.
And I'm going to just pitch the idea right away.
Normally, I would ask you guys what you like.
But the question that I would like to discuss in the video is whether math is actually real and true.
And the thing is that mathematicians have proved that they can never prove that.
So we can never know for sure if math is true.
Even arithmetic, you know, it's worked for us so well.
Two plus two is always four, it seems.
But that may not be true.
I guess that's kind of the question I wanted to address.
And out of that it also comes, you know, can machines be smarter than humans?
And can robots overtake the world?
Just things like that, so any thoughts?
So those were three topics, right?
So that was one topic.
All those questions kind of come out of that
Those are big questions
Are we supposed to say something?
What do you guys respond?
Do you think any of those ideas that she mentioned are things that you would be like, hey friend, this is really cool.
You've got to check out this video
Mhm
Yeah
Which one?
The robot thing
Would robots take over the world?
They're all related, you know?
I like the one that's like, is it true?
Like, four plus four is eight.
Is that true?
So, I'll tell you a secret.
Even though we cannot tell that math is true, that's sad, that also means that robots cannot overtake the world.
In case you were wondering
It's still possible
It's not possible
I changed my answer.
I'm changing it to math
The math one?
But you'll get to the math one through the robots one
You know the answer?
Yeah
Right?
So do you have any other ideas, or was this the big ideas?
That was the big one
So you think if she talks about the idea of could robots overtake the world, and through that she answers some of the questions about math being true or not true, that would be really cool?
Yeah
I don't think robots could take over the world
You don't think so?
They couldn't
Wait, what about that weird robot with the face that's creepy?
I mean, you can smash them and they can break down
But what if they try to smash you?
So, eventually we'll be smarter.
Because that's kind of what a mathematician proved, is that a machine can never be as smart or do as many things that we can do
I like the could robots take over the world idea with all this other stuff connected to it.
I think that's a cool idea
Do you know the show Lab Rats?
No, I don't.
Well, there's these three humans who are also a robot thing.
And they have all kinds of powers, because there was this guy that wanted to make them evil and take over the world
They don't want Disney Channel
It's not Disney Channel, it's [INAUDIBLE].
And then this person stole them to make them become heroes.
But what if someone that invents a robot gives them all kinds of--
Powers?
Yeah, like a laser
That's what she was saying, that robots can be really, really smart
But not as smart as us.
So I guess, eventually, we'll win if robots did decide to break down on us
I want to make sure Andrea gets some time, so let's hear your ideas
Do you guys have any tissues?
I'll go get you some.
You keep going.
OK?
I'm sorry about that
That's
So, I already know the answer for you.
Are you going to have to have braces someday?
Do you know?
No.
They're cool.
They look cool
You want them?
No
Kind of
No you don't.
No you don't
So how long have you had your braces?
Beginning of fifth grade
So you've had them a while?
Yeah
I don't even know if I want braces.
Because you can't eat gum or anything like that
Let me see your teeth.
You need braces
No, I don't
Yes, you do
So are you curious about what they do to your teeth?
Just make them straight, I guess
Do you know how they do that?
Yeah, they put them together
But I have two teeth up here
And they pulled them down, right?
Yeah
You had two teeth that were flipped up?
I had a palate expander, too
Tissue
And it made me talk like that, and I couldn't talk for a while.
And it widened my jaw.
I had to key it every night.
It kind of hurt.
You don't want braces
What was it?
I'm sorry, I missed it.
What was your idea?
To talk about braces, basically what they're doing to your teeth
Oh, that's cool
Are they bad for your teeth also?
Are they bad for your teeth?
That's a good question
And why don't more teeth move back?
Why do my teeth move, and what happens when stuff gets in.
I just got my braces off in October
Good for you
Yeah
My dad has instructed me to get them changed every month, but it's been like three months
So what's actually going on with your braces?
That's a cool, very age-relevant relevant topic
But instead of getting braces, can't you get that plastic thing?
Yeah, the aligners
Do you have to wear a retainer?
Have to wear a retainer
I know one of my friends who had them, still wears this retainer
Do you have to wear it every day?
If you don't wear it, they can get crooked again
Every day for the rest of your life?
I am proof of that, actually
For the rest of your life?
That if you don't wear your retainer, your teeth do go back
Do you have to wear it in school?
I had braces twice
See?
I still wear retainers at night
And they give kids braces so young, now.
You can be in third grade and have braces, whereas, when I was a kid, you didn't get them until high school.
Third grade?
Yeah.
I've seen lots of little kids with braces.
And you're like, your teeth are so little and you--
You have to wear retainers for the rest of your life
For a while
What about that thing that goes--
The headgear?
The headgear
Oh, did you have to wear like that?
That's so gross
Willy Wonka?
It looks weird
Charlie and the Chocolate Factory, where he wears [INAUDIBLE]
It makes you talk weird, too
We're actually really good on time
OK, everybody, it's time to stop what you're doing
I heard some amazing conversations.
And you sixth-graders asked fantastic questions.
We're just wrapping up.
I'd love to hear from a couple groups of students, the sixth-graders.
What were some of the best ideas that you heard all day, that you're like, I can't wait for the video to come out.
I really want to watch it.
And maybe hear any ideas that you're really exited about.
Yeah?
The navy one
The navy one?
What about it?
What did he--
With the torpedoes
Torpedo.
Interesting
[INAUDIBLE]
The idea of watching things blow up is exciting to you?
Yes.
What else?
I liked the idea with the robot and [INAUDIBLE] talk and Wi-Fi and stuff and you can play with it and run around with it.
I'd like to have seen it move
So you're excited about learning more about robotics and how they work?
Yeah
Anyone else have one?
Gianna?
Yeah
I have three
Pick one.
Pick your best
I like the braces
So you have braces, understanding what braces do to your teeth and how they work.
That's cool, if you've got braces, so that would be interesting
Someone said something about volcanoes
The volcanoes, understanding if there's a volcano underneath your house.
That was a cool idea.
Is there lava under my house, right?
Yikes.
Sixth grade, any advice to our MIT students before they take your good feedback and turn them into videos?
Anything you want to make sure that they know to do or not to do as they take your good ideas and over the next month develop them into actual movies?
Use picture ideas instead of talking to them, like show examples so they can memorize it
So we talked about that, MIT students, yesterday, the idea of showing, don't telling, which I'm sure you guys talk about in your writing a lot, right?
Showing, don't telling.
Cool
Don't ask too many questions, like when we watch one video and it asks a ton of questions.
And we didn't really understand it
You mean asking the audience question?
Yeah, like saying, what does this do?
What does this do?
If they're just saying one thing and then show it
So asking your audience too many questions is annoying
Yeah
You first, and then you
Be original
Be original, so try and have a new idea.
And what else?
While you explain how things work, try and make some jokes while you do it, so your audience laughs at it and they understand what you're talking about
Awesome.
Last one
Don't use a monotonous pitch
Awesome.
Change your voice around and make it more interesting.
MIT students, I'm curious from you what was valuable that you got from the sixth-graders today, to give them some feedback about this experience.
Did anyone get really good feedback?
Yeah, well, I found that-- and I should have realized this-- is that having robots stomping around and destroying a city is way more fun than showing old people doing math
Robots are more than old people doing math.
Huh
I, too, learned that my tastes are different, that lava and volcanoes are a lot more interesting than food decomposing
He wanted to show food rotting, and everyone thought that was not as fun as volcanoes exploding.
Any other thing from the MIT students that you found helpful?
Yeah
I was kind of-- it helped me to know how much people in sixth grade know about some of the really complicated subjects that we're using, learning here
Awesome.
We had to do a conversation about sinking and floating and how do boats even work.
It was awesome.
Well, we can't thank you sixth-graders enough for coming here today.
So helpful.
Thank you.
[APPLAUSE] And then, we'll be in touch with you guys to let you know how we get our ideas developed
And the videos that you guys made with me last month are almost finished.
Carrie is almost finished with them today.
So I will send those over to your teachers and you can watch them some time this week
Any directions from teachers before they leave the room?
And coming up next?
Nope, just make one long single line so I can count before you leave the building.
And then I'll count you on the bus again
Check for jackets and all of those things.
But have a safe trip back home
Did you say thank you?
Thank you
Thank you so much.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So we are among friends, please call me Natalie, please interrupt me as I go.
Elizabeth has given a very nice framework for-- the high level questions about this next little bit of time are things about creating a scalable educational experience, and things like that.
But truth be told, this is really going to be me, my dirty laundry, telling you all the mistakes I've made in developing something that I thought seemed like a good idea at the time.
How's that as the sort of tag line for this talk?
It seemed like a good idea at the time.
But maybe, just so that you don't go running from the room and think this is going to be all disaster, I will start by showing you where this has ended up.
Because where all this meandering and mistake-making has brought me is to something that I'm actually pretty proud of, and that seems to be reasonably useful.
I don't think this is the final form of what this will take, but let's start with where it ends up, and then we'll rewind the tape and go from the beginning to get it back to here.
So BioBuilder is now a nonprofit organization that I am founder of, I have a wonderful board of directors that help protect me and my time, and help me make good decisions about protecting just thinking through the business questions, and the strategic questions, and the wisdom questions that are involved in running a nonprofit organization.
It came about because I thought that the teaching I was doing here at MIT, in the Department of Biological Engineering, was such a great way to teach and to learn.
I was having so much fun drawing on current research questions from the department, and using the questions that existed to teach the engineering, and using the engineering to teach the science behind it.
So I'm a scientist by training, not an engineer.
In fact, I didn't really even appreciate the distinction between science and engineering until I got to MIT 12 years ago, or something like that.
I really like learning about how the world works.
What I discovered when I got here and started teaching this way to the undergrads was that, by building stuff made from biology, you learn a lot about how it works.
And so I really just wanted to more broadly distribute that content to students who wanted to learn this way, and to teachers who wanted to teach this way.
Pretty simple-- not that wise, maybe, but that's what I've organized this around.
And so there is a nonprofit organization that is called the BioBuilder Educational Foundation which has, as I say, a board of directors, partners, we have publications and newsletters and things like that.
We are a 501(c)3, and we were founded in 2011.
The funding for this organization came from the National Science Foundation.
They awarded MIT, Harvard, Stanford, UC Berkeley, and UCSF-- a consortium of five institutions-- 10 years' worth of funding.
From the NSF, that's really unheard of.
And I thought, as the educational director on the East Coast for this, with 10 years of funding we could do something really, really cool.
And so what that started with were animations, which is why I'm here talking to you today.
What those animations have been connected into is a curriculum for students that has a number of hands-on laboratory activities.
So for example, this one is for students.
These bubbles that are here link to the animations, but the content that's on the website is some introductory content, laboratory procedures, lab report information, and then a portal for sharing your data once you've collected it.
There's also a teacher's component to this website, so that teachers, if they wanted to start teaching this content, would have classroom materials like PowerPoints to introduce the content, videos about introducing synthetic biology-- which is what the focus of these labs are-- and then components for the lab introductory PowerPoints and things like that.
So this is a website that has information about the nonprofit organization, it has curricular content for the students-- which are mostly high school students and some college students-- and then it also has a portal for teachers.
We run teacher workshops in the summers to train teachers how to teach this in the classroom, and how to teach other teachers.
Yes?
So one of the things we talked about yesterday was this notion of a video being something more than the experience of just pressing play and finishing it, that a video is meant to spark an ending, that is the conversation that happens on comments on YouTube-- we focused a lot on YouTube and social-media based video.
What is, I guess, the ending or the goal-- or what's your vision what an ending looks like for things like BioBuilder?
And what's your user pipeline?
Because for us, we're talking a lot about people who are exploring the internet and finding things that spark their curiosity or love of learning.
I would imagine that something like BioBuilder, you have a very focused pipeline that's maybe more looking at classrooms?
So the distinction you just made between the focused pipeline and the user exploring through the internet, is a distinction that I did not know when I started this project, but that has become really clear.
So the pipeline for this is really intended to be formal education setting-- so I don't know if when you took high school biology, you took advanced placement biology or something like that, there are this series of lessons and labs.
And these are intended to fit squarely into formal education settings, where teachers have content that they need to cover, hands-on labs that teach techniques as well as investigative questions, and then writing and points to communicate their work.
Originally-- and maybe this is a good segue into to how this really started-- this was my notion.
I don't know if you've ever seen this website-- I'm sure many of you have-- BrainPOP.
This was what I thought the animations were going to fit into.
This is kind of the explore the internet depending on what you're curious about kind of sense of the animations that are in BioBuilder.
So cellular life in genetics-- and maybe in watching cellular life in genetics, a person would get interested in something in particular-- cloning, or whatever.
And so this is a very rich library of animations that a person can self-navigate through, and learn content that they're particularly interested in.
And I imagined, when I started this project, that I would build a library of videos that looked like this.
That is not where I have ended up.
That was, in fact, really misguided.
Yeah?
My question is-- you're in the kind of structure and giving this to students and teachers, how do you choose what to pick as a--
As a topic?
Right, so I work with high school teachers every summer to decide on the topics.
The very first thing that the high school teachers I've worked with told me was that they did not have more time in their day to teach other stuff, but that they did feel that they could teach what they were teaching in a better way, in a more effective way.
And they thought engineering was a great lens to teach biology, in particular, and these current research questions
And they're giving you the concept that has to be covered, and you're coming up with the example that-- are they saying, it would be nice if we could use this example for--
So it's more of a partnership than that, because what I'll do is I'll say, here's a really interesting system that has an ongoing question associated with it-- how do you reliably engineer a cell?
How do you improve gain in a system?
Something like that.
And they'll say, oh, well, you know, I don't really know much about that topic, but I have always had trouble teaching photosynthesis.
So is there a way that we could teach that question and cover photosynthesis.
So I'll say, well, in a plant cell, we could do it this way.
So it's more of a give and take-- what they have challenges teaching, and what are the ongoing research questions in this field.
So this kind of BrainPOP kind of meandering through a library of videos is what I imagined I would be creating at the beginning of this 10-year NSF funding.
What I had not appreciated was how long it takes to make each video, how hard it is to make each video, and how bad I am at making each video.
So let me just walk you through the very first video that I made.
So in preparation for this conversation, I went back to some of the earlier stuff that I had done.
And I'll tell you that the characters that are in these videos started with the characters that were-- these are all animated videos, they're all animations.
And the characters who are in them started with this comic strip, which appeared in the journal, Nature, in 2005.
So this was the launch of some of the interest around the field of synthetic biology.
There were some splashy papers in Nature, as well as this comic strip, and it shows the adventures of a woman in the lab named System Sally, and a boy who's just super curious named Device Dude, and Buddy, who is this little green blob in the middle there.
And it's a really fun comic-- I like comics a lot, I grew up with Spider-man.
I love comic books.
And I like this one until it gets to the really hard content.
So right at the fold here, right at the midpoint, it starts to talk about engineered genetic devices.
And I'll pass this around.
There's a lot of content here, Boolean logic gates, ribosome binding sites-- you have to be an expert to understand what was going on in that comic.
So my naive thought was, I would take these characters-- because they were very popular, people really liked them-- and I would have dialogue between them to explain these ideas, rather than have them on the page with just bubbles and question marks and things like that.
So in a very lucky way, It turns out that my college roommate's husband runs an animation company.
So that's how I got connected with animated storyboards.
They gave me a very nice discount on what they would do in terms of these animations-- it was still expensive.
But this was the transition that was made.
So the comic book turned into this website, which was BioBuilder.
This was the very first iteration of the website.
And it had, on the left hand panel, the idea was there would be science animations, engineering animations, technology animations, things like that.
And like BrainPOP site, you would enter that, and there would be a whole library of animations that you could explore.
In the end, I don't have enough years to live to populate the site.
It takes so long to make each of these animations-- it would take me forever to really populate the site for visitors to self-navigate through the topic areas.
So this was the original idea.
Then I started working with a teacher, and the website changed to look more like this-- where the animations were put into a formal context of activities, there was also a link where you could go directly to the animations, if you just wanted to watch the animations.
So this was sort of the transition period, where there was both a library of animations and activities that you could do, if you wanted to teach with these animations.
There were also single-page comic strips-- so returning to what was originally the unsuccessful version-- single-page comic strips that set up each of the laboratory activities.
And this was the transition to, then, the third version of the website, which I showed you at the beginning.
So let me show you, since you're working on animations of your own, let me show you some of the detail, some of the things that went into building that very first-- well, let me show you the first animation I did.
How about that?
I'll show you what these animations look like, and then I'll show you some of the fun and the choices that went into making this animation in particular.
Does that sound like a reasonable plan?
This would be a great review, because on Thursday, Josh from [INAUDIBLE]
Great, great.
[VIDEO PLAYBACK] [BUBBLES] -Dude , no running in the laboratory.
-Sorry, Sally-- but have you heard about this competition?
It's called iGem, and I think my bacterial bubble could totally win this year.
-I thought you were done with bacterial bubbles.
And what do you know about iGem?
-Um, not much, except that there's going to be a bunch of losers and me.
-I don't think you understand the nature of this competition.
iGem-- the International Genetically Engineered Machine Competition is a way to get young scientists and engineers working together to engineer biological systems.
-Working together?
Where's the competitive spirit in that?
The dude works alone, that way the dude gets all the credit.
-You need to be a member of a team to join iGem, and you need a professor to lead it.
-But I heard it was a student competition.
-Well, yes, the competition started in 2004, based on an undergraduate class developed at MIT in 2003 for their short winter session.
And it continues to be an undergraduate experience-- but not without guidance and support.
Last year, there were more than 30 teams who competed from all over the world.
-So that's my competition?
That's a lot of people.
-A lot of people all asking the same question-- can simple biological systems be built from standard interchangeable parts, and operate in living cells?
Or is biology simply too complicated to be engineered in this way?
What do you think?
-Biology is not too complicated for me.
-The goals of this competition are to enable systematic engineering of biology, promote open and transparent development of tools for engineering biology, and help construct a society that can productively apply biological technology.
-OK, I got a lot of work ahead of me.
If we're not going to use my bubble idea, what else is possible?
-Maybe a better question would be, what isn't possible?
-OK, so first off, what's your standard interchangeable part be?
Actually, what parts exist?
Do we have to make those?
-You'll probably need more than one, but the registry of standard biological parts is a great resource, with lots of parts already designed.
And if we make parts of our own, we should add them to the registry, in case other teams can use them, too.
-And help other teams?
What kind of competition is that?
Of is that how so many teams got cool projects going last year?
Pleasant-smelling bacteria, a bacteria nightlight, a DNA drug delivery system-- I heard one team even made up bacterial freeze tag.
To beat those, we should get started right now.
-All right, well let's brainstorm for a little while before I have to get back to work.
How does that sound?
-Great.
Can you show me this registry?
Maybe we can find some good parts to use.
-Good idea, Dude.
Let's start there.
This is where you'll be able to search.
-Oh, wow!
[END VIDEO PLAYBACK]
All right, so that, for better or worse, is probably the best video that's on my website.
And you know, it was done several years ago, it still holds true.
So in some ways it-- it still very accurately describes iGem.
It has some old screen shots of the registry and things like that, but you know, it was done five years ago and I think it holds up reasonably well.
It was fun to make, so let me show you some of the fun things that were involved in making it.
One thing was the script.
So the script was written by a former undergrad of mine in biological engineering, and she did a summer UROP with me, and wrote some of these scripts.
And then it was edited by a woman who's on my board who is also a screenwriter.
And this just shows you some of the edits that were involved in that short script.
"Dude, no running in the lab.
Sorry, Sally, but have you heard about this competition?
It's called iGem.
I think my bacterial bubble could totally win this year."
And she deleted, "I don't much about it, but someone gave me a flyer for it."
So there were a lot of deletions and edits on the text, itself.
And if you like writing, and if you like this kind of sentence level work, this is actually pretty fun and creative.
I'm not very good at it, but I think it's fun.
The other part that I would say was pretty fun was-- oh, before we did the storyboarding actually, we did-- where are those pictures?
Maybe it's here.
Yeah, so the company that did these animations sent me pictures of people who were posing in ways that they thought they would be using in this animation.
And I got to choose which little boy I wanted to be Dude in these comics.
So I don't remember which one I chose, but these are all shots of actors that they have who come in and pose in ways that they would think might fit the script, and then you get to choose which one they're going to base the image on.
I also got to choose voices, and I have an early read-through here that-- [AUDIO PLAYBACK] -My bacteria bubble could totally win this year.
-I thought you were done with bacterial bubbles.
What do you know about iGem?
-Not much, except that there's going to be a bunch of losers, and me.
[END AUDIO PLAYBACK]
So that was a quick read of one of the candidates for the voices and stuff like that.
So that was kind of fun to work with them to choose actors who were doing the still shots, and the voices, and things like that.
And then they did the storyboarding for this animation, which I had never done before.
They were very talented at-- they knew where to break it up and which shots they'd use.
And in watching that video now, and honestly I don't watch these videos regularly, so it's interesting to see a couple things that I think still work reasonably well.
I think it's short enough, it doesn't go on and on-- so could be a little bit shorter, but I think it's pretty interesting.
I think it zooms in and out of a reality scene and a non-real scene.
And it covers the questions they have.
There's a little bit of lingo and secrecy to it that's not as easy to understand, if you don't already know iGem a little bit.
But I think it answers most questions that people would have about iGem.
Yes?
Why did you choose to do animations rather than film a real woman with a boy?
I think I was misinformed, that I thought animations would be quicker and easier.
And then I also had that BrainPOP idea in mind, and this comic strip in mind, which had Buddy, this sort of bacteria thing, as one of the characters.
And so I didn't think a puppet or something would make sense.
But that does lead me to a mistake that I made.
And again, if I knew then what I know now-- who's my audience for this?
I mean, would high school students watch this?
I don't think so.
I think it's too young for high school students, and too old for middle school students.
It kind of doesn't have-- frankly, I show it to my undergrads, so I was like, I'm OK with that.
So maybe it's OK for MIT undergrads, but then it doesn't really have a great audience mark.
And the high school teachers who are now using them, it's pretty rare.
I mean, they may assign them to people to watch the night before they come in and start the discussion on synthetic biology, or start their iGem team, but it's not something they show in the classroom.
Yeah?
So what was the intent behind making it?
Because we talk a lot about what is going to be your what, why, and how questions before you make a video.
Was it to make something that was instructional for people who were already interested in iGem?
And if so, what you made you want to do a video over a comic book, for instance?
What was that initial interest?
I thought-- why are there all the words listed in the dictionary?
My goal was to actually make like a Wikipedia of synthetic biology, covering the relevant topics through animation.
So I wasn't being selective-- I had to choose something to start with.
iGem was my first, because it seemed to really capture some of the unique features of synthetic biology, and was explainable in a way.
It's a discrete entity, not too technical or anything like that, that I could cover.
But it was really intended to be encyclopedic, not targeted to an area that I wanted to cover-- except synthetic biology.
When I started this project, people were coming to me for learning materials about synthetic biology, and some of those people were people in government and policy, some of them were biologists wanting to learn engineering, and some of them were engineers wanting to learn biology.
And I was not going to make a comment book, or a different way for each of those populations to learn.
I guess my thinking was that through self-navigation through animations, people could learn what they wanted to learn.
But what it requires is a really rich library of animations to look through.
And this took me a full summer to do, and I just don't have that much time.
I mean, I would need hundreds of animations, and I just didn't have time.
Then, to top it off, when the teacher finally came to me and said, I can't teach this way.
There's no narrative to using these.
I need a hook, I need a something that introduce to my students.
It has to fit into a framework.
That's when starting with the research questions in the field, and using these animations to support the background that you learn to do the investigation, just seemed to make a lot more sense.
So if we look back to the website, and we go to the student side, we can look at this one-- what a colorful world.
So now the bacterial growth curves animation is here.
I did, I will say, introduce another character, Izzie the iGemmer-- she's Latina, somebody said you should have a little diversity.
Anyway, again, it has all kinds of probably rookie mistakes that you guys will never make, thanks to your course that you're taking now at IAP.
But this is the idea, is that now these animations don't stand on their own the way like BrainPOP does, and you sort of meander through them.
Now they're part of the background material you can look at and read and understand, in order to do some biodesign, or some experiments about transformation of bacteria to different colors, and things like that.
Were there are other-- tell me what would be relevant for your work here.
-OK, I was thinking that since you work so well with the high school population, if you could orient these guys a little bit, since you work so well with teachers and the students, understanding that age range.
Right before you work with them, which is our target audience, what are some science concepts that those kids-- like what do they know?
It wasn't that long ago that you were that age, but you've done a lot since then.
You probably forget.
Meeting with the sixth graders would probably be very informative to think about, oh, what do they know, what don't they know?
But maybe it would be helpful, since you have a better understanding of what's taught when, for them to have a sense of-- say sixth grade is kind of our target, what do those kids know, and what don't they know about science?
Sorry, if I can add one more thing, it's really important because even if-- I mean, I had you guys meet with the sixth graders today, but that doesn't mean that your videos should be talking to a ten-year-old, right?
Because that's really the basic knowledge base of the average public view, that they're not really going to remember the science that they learned beyond middle school or high school.
So don't mistake understanding an audience of sixth graders as thinking that it's an audience of ten-year-olds, that you're talking to them like they're 10.
Because the person who watches Myth Busters, and the person who goes to vote on certain policies, they're going to share a knowledge base.
So that's why I think it's important to orient yourself to that audience
So in terms of what folks know and don't know, where this content currently is positioned is to fit into the formal frameworks that most high schools teach for biology.
In particular, the advanced placement biology classes.
And the reason for that is that, because their laboratory based, you really kind of need particular equipment, you need time in your day to do this kind of stuff with them-- if it's a hands-on lab activity-- and so the advanced placement curriculum has that built in, and you can make some presumptions about what's available, and what isn't, what sorts of basic knowledge they have.
At the high school level, generally speaking, there's good abstract thinking.
You can think about things that you can't see and understand what's going on with them, without actually having to either manipulate them directly, or visualize them, have them right in front of you.
At the middle school level, I don't know that that's true.
And I will say there is a BioBuilder junior site that's quite fledgling, and not as robust as this, but that there's great interest in it.
I mean, you can see the in sixth-- they're awesome, these middle schoolers.
They're so curious, and they really want to do stuff, and they're not, oh, I don't like science, and oh, I never got that.
Or they're not already turned off in some ways, they're very open to new ideas.
What's hard, I think, is the abstract thinking.
So when I tell, in this lab in particular, that's here, which is an experiment which brings DNA into a cell and turns the cell different colors.
And the scientific question here is, does it matter what cell type you're using to get different levels of color expression?
So like the chassis in which you're running this genetic program, what's the impact of that?
It's a very important question in the field of biological engineering-- what cell are you going to choose to run your program in, and how do you choose it?
This is a typical transformation experiment that many schools do with their students, but usually schools are doing it just as a demonstration.
What are you going to do with a middle schooler when you say to them, well, when you add salt to the cell, the membrane gets porous, there are tiny holes in it, and the DNA can sneak through?
They have to buy it.
So some of the middle school teachers are doing acting out-- they have students stand in a circle and hold hands, and say, OK, now salt's added, and have some of the kids let go.
And so I think working with teachers who know their audience has been, for me, critically important in making this any kind of success that it is.
And I continue to really value their advice.
They know what works with their students and what doesn't.
And these animations, some of them use them, but I think they mostly use them as homeworks, not as ways to have students discover what they want to know about synthetic biology
That's going to be really important, because some of you guys have authentically more concrete topics than others.
Because ships are something that you can see.
Computer programming is not.
So you guys, some of you will have more abstract concepts, are going to have to really work hard to figure out, how do you make that concrete enough so that a kid, who developmentally can't grasp an abstract concept as easily as someone a bit older-- how do you make that with metaphor, with acting, with all these different things?
How do you use your communication tools to make something that is maybe a reach outside of their developmental conceptual ability, but is still accessible to them
And I will say it's that, as a case study it's very interesting.
So when the MIT K12 Videos program started, it was originally allowing students to create full videos on their own without production.
And I'll have to show you this slide, maybe after Natalie's talk, but when you let MIT students decide what they want to make, just let them do it free-reign, almost 70% of the videos were physics.
It was alarming, the overwhelming majority.
And it's because physics is a very visual, very tangible topic.
And when you go online, a lot of science videos and a few are physics-based.
Maybe one was on electrical engineering.
And the electrical engineering ones were about circuits, so again, you could see the parts.
When we started producing Science out Loud, the one video that went as popular-- in order of magnitude, at least-- and the view count [INAUDIBLE], that's a different discussion.
But the one that did the best on its own was the one that I marketed the least.
I literally just posted it to YouTube, and sent one Tweet about it.
And it was one on how computers work.
And I don't know if you've been in the Stata Center, but there's this machine called the [INAUDIBLE] which explains how switches count in binary, as the fundamental component of the computer.
And people loved that video, and it showed this desire and this innate need for a way to visualize these super abstract concepts.
And the fact that things that fulfill that need just don't exist right now.
It's very hard to find a good video on programming.
It's very hard to find a good video on biology, actually.
Anything microscopic-- it's hard to create something that's visually engaging, to where you really justify making a video out of it.
If you ask yourself, why would I make a video about microbiology, when it's so much more clear and so much more engaging to write a blog post about it?
And so are the challenges that come along with it.
And these are very good questions.
They're very good mindsets to have when you approach those ideas
And Elizabeth had a really-- we had a really interesting conversation with those girls about what computer programming concept was actually most interesting to them.
What we realized was that all of these ideas were really big ideas.
The binary is such a narrow topic, that is allows you to flesh it out in great detail to show-- for the different types of learners who are out there-- different ways to access that information.
So the challenge is to pick a topic that's narrow enough, that in five minutes, you can really unpack it and pack it back up again.
And really targeting our audience to figure out what do they know, and what don't they know, so that we're hitting that sweet spot where it's interesting, complicated, but not so complicated that I can't-- it's too much for me
Jaime's original pitch was almost six minutes long.
And it was about how does a computer work?
That was his original idea.
And so it hit upon switches.
It hit upon talking about binary and on semi-conductors and all these things.
The switches part is literally less than a sentence of his pitch, and that is what we ended up exploring for five minutes.
We had so much to say about it.
I do think the more focused a video is, it can be a lot more powerful.
Somehow it makes it more dense.
Having worked with this age group myself, I remember sixth grades learning basic biology-- just so you guys can orient yourselves.
Seventh grade is learning basic structures, typically, like how to build a bridge, and how to keep something physically standing.
Fifth grade is-- I'm trying to think backwards.
Eighth is-- chemistry.
So just so that you guys have a sense of where they are in their learning, they're really sticking with physical things until they get a little bit bigger, and can conceptualize chemistry.
They're not even ready, yet, to think about that until about eighth grade-- is like 14
So maybe in the last just minute or so, I'll show you another video that's not very good.
But that does really speak to this question of how I'm trying to visualize something that is microscopic, and what we did with animation to try to accomplish that.
And we won't look at all of this.
But we'll get to the microscopic part.
[VIDEO PLAYBACK] -Dude, I like what you built there.
What's it do?
-Oh, hi, Iz.
This is a cool bug-- I made it using in an electronics building kit that Sally got me.
She wanted to get me a genetics kit, but nothing's available yet.
Eesh-- I don't get it.
Why is it so easy to turn on and off the lights for this electronic bug, but so hard to turn things on and off in biology?
-What do you mean, Dude?
The arsenic sensor that we built in bacteria a few years ago works pretty well.
If there's arsenic in the water being tested, the sensor turns on.
And when arsenic isn't in the water, the sensor is off.
And we built that sensor with natural biological parts.
-So arsenic works like this switch?
How is that possible?
How does the cell know?
-We used parts from the ArsR operon to program arsenic inducible expression.
-Hold up, arsenic inducible expression?
What's that mean?
All I know is that arsenic is toxic.
Don't the cells just die when you poison them?
-Cells have to come up with a bunch of clever ways to survive.
In the case of arsenic, the cells make a pump to get the poison out as soon as any comes inside the cell.
-So asser is the pump.
-Dude, be careful how you say it.
The protein is called ArsR, since it's the arsenic repressor protein.
And ArsR isn't the pump-- it just tells the cell when to make the pump.
The operon has a promoter, which is where the cell's RNA polymerase binds.
And at least three genes, ArsR gene, which encodes the repressor protein, and the ArsB gene which encodes the pump, and the ArsC gene, which encodes an enzyme that reduces arsenic in the cell, so it can be pumped out.
-Oh, OK, right.
[END VIDEO PLAYBACK]
OK, so you get the idea.
It just goes on from there, and it talks about how you build an inverter by moving the repressor and the promoter in different spots, and manipulate the DNA.
But you can see with this one, you know there was some physical object that we were trying to use to illustrate what a switch does.
But then when you get down to that DNA level, and you're talking about proteins binding to DNA, and names of open reading frames and stuff, it's just deadly.
I mean, I can't even watch it and I know this stuff.
So I guess it's a real challenge-- it's a hard thing to think about making tangible, making evident, things that are not.
But I do think there's a great power through storytelling, through animation, through video that you have, and when it's done well it's really very, very effective, and very memorable, I think is the big part of it.
Are there any other quick questions?
Yes?
What is the criteria element of the idea is too big?
When you're brainstorming, trying to find out ideas.
Is there communication of when-- because I think, after today, we won't get to see the sixth graders anymore
Well, you have us and each other
I think the litmus test that I used when producing Science Out Loud-- I didn't mention this yesterday, but it was probably the first thing I should've said.
Because there are no best practices established in this field, take everything that we present to you in class with a grain of salt.
And [INAUDIBLE].
So what I've found with distilling a big idea is that when I'm scripting with a student, and we start to go down a rabbit hole, so we're saying, well what is a computer made out of?
Well, I have to talk about a switch.
Then I have to talk about binary.
Then I have to do this.
And we have somehow found ourselves down a rabbit hole that is tangential to the core topic at hand.
That's always been an indication for me that the core topic is too big.
And you examine the branches that are possible from that core topic, and start looking at, well, what can I do from a branch, itself.
We can talk about this more-- because I know Natalie has to leave.
Does anyone have any final questions for her?
Her email address is on the syllabus
I'd be happy to talk
Yeah, I was curious, I guess you worked with the sixth grade, and many different teachers.
Are they conflicting advice that is being given you?
Yes, but I think that's because teachers teach in many different settings.
And so what is true in one setting is false in another.
There are some schools that won't show YouTube, that have that site blocked.
There are other teachers that teach every day with YouTube.
So it's just every teaching setting is really-- there's a wide range of--
Somehow you never get to make a decision--
Standards are the same
Yes, right, so what they have to reach
It may be different where you are, but at least here, in public settings, and in some private settings, the teachers have uniform goals that they each have to achieve in each academic year for each student.
And it's a national standard in public schools that all sixth graders must do these big topics.
And so within that, how to teach those, that's the common thread in that
Yes, right.
But in terms of using animation-- so the question I thought was more related to the tone of the video that you're making, and that is hard, because teachers have very different students, and teachers are very different people, one from the other.
Some are very comfortable teaching with animated characters, some think that it undermines their authority.
And I've heard both.
So it really-- for better or worse, I tend to just make a decision and go with it.
This goes back to if I knew then what I know now.
I think small, smart steps is really the only way to figure this out.
Anyway, and with that, I really do have to go.
And I'm sorry, but I'd be happy to talk with you guys offline, OK?
Thank you
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today was a little different than yesterday.
So I thought it would be good for us to just sort of recap what happened.
Let's go back to the sixth graders visit.
Thank you, Jamie, for prepping them for it.
I woke up this morning.
I was like, I didn't prep them for it.
If I were them, I would have been super overwhelmed.
I'm a terrible person.
Was that helpful to you?
What did you guys think about that whole experience?
I immediately knew that whatever ideas I had yesterday was completely off.
And wrong.
180 degrees in the wrong direction
I mean, do take what they say with a grain of salt.
Sixth graders don't always know how to articulate what they want.
In a pitch, robots always sound cooler than food decomposing.
But there is a way to make that awesome, if that's really what you're interested in.
So as with everything in this class, the feedback is ultimately for you to decide what to do with it.
Was there anything surprising that came out of the first hour?
I know it's hard to put yourself back into that perspective.
That's what I meant yesterday with the whole thoughtfulness quality.
One of the values of the class is being able to take yourself out of the perspective that you have here.
We were shooting science out loud during the first season with two grad students who are in AeroAstro.
They were making a video about how engines work.
And we were discussing one of the scenes that they had planned was to describe a turbine engine.
And I'll go into this more during Thursday's lecture.
But one of the students was trying to describe that it's the turbine blades that push the air forward.
And we he was thinking to do an animation with it.
And we thought, this is really complicated.
I couldn't understand what he was trying to say.
And finally, the other student who was there was like, oh, well there's one Boeing 777 in the hangar in that building over there.
Like, is it worth filming with it?
And just sort of that mentality of like, you have access to an actual turbine engine.
And it didn't cross your mind to film it.
And they were awesome to work with.
And you know, that the whole day of shooting was actually really fun.
But the ability to take yourself out of the things that your experience all the time, or take for granted.
Creating that window for people who don't necessarily have access to the world that you live in is surprisingly difficult to do.
And hopefully the first hour of classes sort of helped you orient into that new perspective.
But I know that's a challenge for me a lot of times is, I don't think people are going to think this is super interesting.
And it's not the content that's not interesting to people.
It's just the delivery.
How you package it up
I think that getting the right angle for-- like, some of you had really broad topics.
Right?
And the thing that's really hard is that there are so many things within that topic that could be really interesting.
And like the food decomposing thing.
Actually think there are a billion really cool topics in there that could be really fun.
Like cheese is technically mold.
And why is some mold good, and some mold bad.
Right?
And there's some really cool things like that can do about that.
And getting back to microbiome things, or bacteria, or there's some really cool stuff in there.
And just like every one of your topics that you guys brought up.
There's some really cool little pieces of truths inside of there.
And the hard part is figuring out how to give-- I mean, this is what writers do, is having the right angle to your story.
Right?
And so, many-- I think all of what I heard, I heard many your ideas.
I'm like we're in the right zip code, but we haven't quite narrowed down the right building yet.
Right?
Or the right street.
And for the hard part is figuring out within the topic of shipbuilding, or within the idea of hacking, or building a game, what is that little cool angle that you're are going to take to make it really exciting and interesting?
And snare enough and focused enough and that kind of thing that makes it really interesting.
Because I do think actually that lava sounds cooler to like a sixth grader because they're like, lava!
That's cook.
Right?
But the food one.
I'm not ready to give up on that yet.
Like, I think there's a lot of cool stuff in there.
And the hard part is figuring out, really narrowing down your story.
And then figuring out which one has more meat to it
I mean, I think it's a very beautiful story, actually.
The whole idea of material objects on this earth transforming in a way that's tangible to us.
And maybe that's just like super romantic sounding.
But I do think that there is-- I loved what Chris said yesterday about making not just the unfamiliar familiar, but making the familiar unfamiliar.
And I think that's a lot of what of what VSauce does.
And I think that's a lot of what Veritasium does is it's taking the things that you see, and taking the world that you're used to seeing, and showing you a new lens at looking at it.
And I know that sounds very abstract.
And I'm sure that we've thrown out a lot of things to think about that you're not maybe quite sure how to implement on a practical level.
And I promise that that's what we're going to hit next.
And once you guys write a script tonight, or just try writing it, it'll help you a lot to sort of understand how to implement these things practically speaking.
So worry.
I am aware that that might be a frustration that you have
I wonder if we have a little time today to give them more time to flesh out their ideas a little more
Oh, yeah.
Absolutely.
So I was originally planning on showing you guys a couple hosting case studies.
But if you would find it more helpful to just maybe brainstorm script ideas--
I feel like that's where they are right now.
If you think I'm wrong.
But I feel like today made you think a little more.
And now before you go home and have to really implement some idea, maybe it's worth really making sure each of you know what you want to do before.
I don't know.
What do you guys think?
I'm feeling some anxiety right now that like it would help to flesh out your ideas out a little more.
Yes
OK. We will totally do that.
I did want to show you something.
Because for me personally, it's hard for me to hear abstract level ideas.
And I just need to see an example to help understand that.
So.
And again, I'm not saying that the way I necessarily do things or the way George or any of the teaching staff do things is necessarily the best way.
Because again, best is not really been defined.
But this was a video we did during second season.
And it was the first biology video that any of us had worked on, which I was really excited about.
But it did present these challenges of like how do you create a visually interesting video that's worth making as a video.
Because again, I mean, I think asking yourself, like why do I need to make an animation about synthetic biology when I could just create a cartoon about it?
As Chris was saying, visual elements, just pictorial representations of things are very, very engaging.
We really were asking ourselves, like what's going to be the visual purpose of her explaining what basically became a little bit about systems biology, actually.
And one of the things that we really fixated on, which is an idea of hers, was a metaphor.
And so this is about a four minute long episode.
Would you guys mind if showed it?
And then we can workshops scripts.
But let me turn off this.
Is the light already off?
You guys can see it OK, right?
OK.
So I'll show this to you guys and maybe we can un-pack what you've think is successful or not successful about our attempt.
[VIDEO PLAYBACK] -Paclitaxel is a compound that can treat cancer-- Paclitaxel is a compound that can treat cancer.
salicylic acid reduces headaches and fevers.
Carotenoids can turn your skin orange.
And miraculin changes your sense of taste.
What do all of these awesome compounds have in common?
They all come from plants!
[MUSIC PLAYING] More than 100,000 natural compounds occur in plants and we barely explore them.
These small molecules are called metabolites.
Just like how all the DNA in an organism forms the genome, all of the metabolites form the metabolome.
Even though it's metabolite can be made from only six elements, there are so many possibilities that it would take scientists thousands of years to make each one try and figure out its usefulness.
Luckily, plants have already done this for us.
Plants have the disadvantage of being rooted to the ground.
So over time they've trial and errored making lots of compounds to see which ones help them survive and thrive best.
And because they've been interacting with other species like us for 100s of thousands of years, some of their chemicals turn out to be really useful, both inside and outside our bodies.
But to use plants to their full potential, we have to know what chemicals they make and how to make them.
Instead of studying every chemical one by one, what if we could study all of them at once?
We could start by mapping the huge network that connects metabolites.
In any living organism, molecules are always on the move.
Being converted and shuttled, decomposed and built back up again and re-used.
It's just like a subway system!
Except in biology, the people are the chemicals and the train is the enzyme that converts and moves them.
If you look at a city from above, how could you map the whole subway system?
Similarly, if we look at a plant, how could you figure out the entire metabolome network?
To figure out a path and a system, what we actually need to do is break it.
If we mutate or disrupt a pathway and see how the metabolite quantities change, we could figure out the connections between them.
Let's say the train from Central to MIT breaks.
We wouldn't see students arriving at MIT, and would instead see them building up at Central.
But not only that.
Anyone else traveling along the red line would also be affected.
So it's the redistribution of people which reveals the red line subway path, and tells us where the train broke.
We can use the systems thinking to uncover the plant metabolite network.
For example, we know about a compound sinapoyl malate which protects the plant from UV damage by interacting with UV light, making the plant grow green.
But without it, the plant would glow red.
So if we see a red plant, it's like seeing no people at the sinapoyl malate station.
But we wouldn't yet know where the train broke, or what other stations are along the route.
To do that, we can mutate a lot of seeds, plant them, and choose the reds ones.
[MUSIC PLAYING] Now we can analyze these samples by using the mass spectrometer.
It measures how much of each metabolite is present in the sample.
Then we can use a program to see which compounds were effected and map that part of the network.
It's like reviewing the red line.
Once we figure out how the entire metabolome works, we can use it to engineer plants to create new bio materials, medicines, and clean energy.
We might even discover that plants have the secret to living forever.
We just need to unlock their chemical mysteries.
[MUSIC PLAYING] [END PLAYBACK]
So Anastasia's topic was-- it was really hard to visualize.
It mean, if you can imagine, our first meeting she was talking about basically mapping chemical pathways in plants.
And it's a hard concept to describe verbally.
And I think that there are some things that we did well with that video.
I still feel like sometimes it's easy to get lost in the metaphor that it doesn't really do as precise of a job as it could.
One of the things I did want to hit though because someone had asked, how do you know if your idea is too-- was it you who asked, how do you know your idea is too big, or not?
You're not going to appeal to everyone.
And you're not going to be the perfect video for every viewer.
And that's OK.
I think understanding your niche is one of the biggest lessons that I learned in producing Science Out Loud.
At the beginning, our intention and our goal was to be something like SciShow or VSauce.
Something that kids just sort of watch on their own.
And we quickly realized that that won't necessarily be the case for us and that's OK. And that we serve a very powerful function being coupled in classrooms, or after school programs, or parents watching them with their kids.
And that'll dictate what concessions you might make.
I mean, in this video, we kind of just said whoever is going to watch it is going to understand a little bit about chemistry and biology.
And that is what allowed us to sort of gloss over some of the details about chemical pathways.
I think that video worked for us, that there was a reason that we made that video partially because Anastasia I think is really good on screen.
And that if you didn't have her, and if you had just a narrated video or an animation, it would be a very different visual experience.
I don't know if you guys agree with that or not.
But that's sort of how I felt.
The other thing is, she naturally as a person is really like that.
She's really jazzed about biology.
It's OK if your personality is not like that.
But the way she wrote her script-- I mean, she was probably one of the best during the day of shoot just remembering her lines.
Because the way she wrote her script was the way that she just sort of talks about that stuff in real life.
And that was a big tip that I wanted to give you guys before you started scripting, was right things the way that you would say it.
And George will talk about this a lot tomorrow, as well.
It's a very, very obvious statement that is very, very difficult to implement.
I don't know if you guys were pay attention to that in some of the BioBuilder videos.
their purpose is to be very instructional like Natalie was saying.
It's like a very encyclopedic type of product.
And it works well in schools, but most people don't talk in their everyday conversation like those characters did.
And that's OK for their product.
But if your intent is to engage someone like Hank Green does, then you have to think about the way that you're writing your script.
And again that may sound like a very abstract tip.
So what I would say is, if you're struggling with that in your script, literally read your words aloud.
And then put your script away so that you can't see it.
And just explain your sentence sort of ad libbed to someone next to you.
So this happens a lot to me.
I actually went through all the exercises that you guys are going to do this month myself to sort of see if what I was saying made sense.
And George and I created an episode about snot.
And the writing process for me was a lot harder for me than I thought it would be, even though I've coached so many people through it.
And it was because I fell into this habit of being super newscastery.
And that's my personal habit that-- I'll say things like, you think this would do this.
But turns out, it doesn't.
Right?
And I would say, this sounds awful.
And George would take my script away and he would say, Elizabeth, what is so cool about snot?
And I'd say, well it's awesome because you think that it's just like this crap that flows out of your body.
But it turns out that has like all these really amazing things about it.
And he would say, well you should just say that.
So as you script, or as you try to think of ideas, turn to your partner.
And just tell them as you would tell anyone what it is you're trying to convey, what it is you're excited about.
Do you need to do-- want to add anything to that?
Well, I think right now we've got about 35 minutes before the end of class.
And I suspect we have some wrap up that we need to do to talk about what to do tonight and what to do to get ready for tomorrow.
So that really puts us more at 20, 25 minutes.
Right?
I feel like maybe what we need to do is old school actually have each of one of you think with paper and pen and brainstorm a little.
And then come back together as a group a little bit, so that you have some time to really conceptualized.
Like if I were to do this on my own, I'm just saying like, I would actually do an old school spiderweb to get my ideas out.
Which is-- put this guy down here.
I would actually probably put my topic in the middle and start brainstorming everything that I can about it to just keep me thinking about what, like-- and then to maybe realize if shipbuilding is in the middle here and we talked about ships, like what specifically within ships am I thinking about?
Did we want to talk about sea sickness?
All right, then if I wanted to talk sea sickness, what would I put in my video?
Right?
And to really visualize for yourself, all the topics, concept, things you would want to film.
And then start getting a sense for yourself of whether or not this is a doable concept.
And you may realize that actually this is the branch I want to keep exploring, as opposed to this whole big thing.
And if this were my brain, that's how this would work.
But if there's a different way that your brain works, we want you-- I would think that you should use this time a little bit solo.
And then we'll have a chance to maybe bounce some ideas off.
So I feel like people, at least what I heard from people was that this morning was really helpful.
But it made you realize you really need to narrow your topic and really think it through a little bit.
So maybe what we do is we spend about 10 minutes quietly working on our own to really flesh out our concepts.
And then we get back together as a group.
Do you need paper.
I have paper.
Does anyone need-- you need some paper and pen?
I'll go grab them in my office.
I'll go get some
Real quick.
I mean, there are so many strategies.
And some people have different preferences of beginning to script.
I wasn't going to share this until tonight.
But maybe it would help to do it right now.
So when I was sitting down to script the snot episode-- and again, I'm not saying that this is the best way to do it.
Nor am I saying it's the right way to do it.
But I was fixated on the idea that your body makes a gallon of snot a day.
That was really the seed that began the idea of the episode.
So George, who you'll meet tomorrow, said, well just list every amazing thing about snot that you want to talk about.
So I didn't even start with an idea of a story or anything.
All I did was list every sort of fact about mucus that I was interested in.
And this is sort of the same concept of having a web.
I just put in a list.
But it doesn't really matter.
And as I was going through-- and some of this was stuff that I had learned in a class I took as an undergrad.
So I was looking through my old notes, basically.
There was a lot of stuff in here.
And I ended up taking out maybe 2/3 of it for the final script.
But this is what got me started.
Mucus as a problem.
Mucus as a solution to things.
Just like random, cool facts about mucus.
The research that's happening.
And just the process of going through all of these things helped me discover a story.
And just the process did.
So if you would find it helpful to do sort of a brain dump idea in the next 10 minutes, feel free to do that.
Again, we're not saying that you have to do any of these methods.
But this is what's helped me personally in the practice of doing this.
So feel free to do something like this.
I had some ideas of demos I could do.
Ted Ed.
When they recruit people to write scripts, they have a requirement that you have a sharable fact in your video.
So what's the fact in the video that's going to go viral?
So I was brainstorming, what could be some shareable facts in the script?
And then I was thinking about what's the point?
Mucus is alive.
that actually ended up not making the final video at all.
But the exercise and the act of doing that helped me figure out what the point of the video was going to be
So let's maybe spend like 5 to 10 minutes of just quiet time for you to really think about your topic.
And then maybe we can like slowly check in with people, and then maybe come back together and see if people are struggling with anything.
I assume that from you're head nods that sounds like a good plan?
Yes.
OK. [NO SPEECH]
Do you know of omnivores?
What is it?
Not omnivores.
What's it called?
Opportunivores
No
No.
They're people who like running food.
It's like a--
They discover?
So I have this great story I'll have to give you if this is what you decide to do.
A non-fiction piece that was written by a journalist exploring people who actually live off of people's decomposing food scraps
Oh, so like freegans?
I guess.
Is that another work for it?
Like I know-- I actually do know some people who their diets is basically composed of free food
Right?
And like what is fermenting?
Fermenting if like beer and cheese.
And like fermenting is a whole other concept in here, right?
Which is actually a really cool concept.
I know.
I kind of was like there's a lot of things like beer and cheese.
And I was like, what isn't there anything of that I'm interested in?
I guess the conflict I have right now is that is seems people are more interested in volcanoes.
And that's easier because I actually know a lot about this.
This would be something I'd have to learn about a lot

But that's kind of what I want to do more

Is I want to learn a topic as opposed to just going in with what I know
What makes you excited about this topic?
I don't know anything about it.
And I think that's something that happens a lot.
Whether it's because my dorm hall's terrible at maintaining our fridge.
And there's always things in there
That's rotting?
That's rotting.
And I'm looking at it, and it's like wow.
This is the-- how did this beautiful broccoli turn into this black muck?
And different things like that.
That's actually something that fascinates me a lot.
Like the transformation
And like why do things--
And like different things.
Why things-- some things decompose in different ways like where if meats starts to rot, then even if you cook it, it's not going-- that's not good.
You just can't eat it.
But some things you can kind of like say, well, if you cook it well enough, it'll be fine.
And also like the whole concept of how you prevent this.
you know like these-- I was looking at it and there's like all these different ways that you could prevent-- just like they have the three topics of like killing things.
How to like bacteria and fungi that cause it.
And it's like attacking like preventing like their functions.
I think they're like their enzymes.
And then there's more like controlling a growth
So maybe-- I mean I love this story that you picked.
PS remind me sometime to tell you about the story I wrote for Backpacker magazine where I took a whole backpacks worth of food and saw what rotted over the course of a week to see what the best foods are to take hiking.
So I did a piece on this in my earlier days and it was really fun.
But I actually found that there were maggots that showed up in the completely sealed bologna.
And it grossed me out entirely.
Because you like, they must have been there from the start.
That's terrifying
Yeah, that's-- Yeah.
The things that like start and they-- with meat and like things that the animal has--
Right
Go through the entire process and they survive
Right.
But the story that I love that you just told me about was you going into your dorm and seeing that broccoli and being like, why did this happen?
And what does it smell so bad?
And you can make your whole five minute piece about that
Yeah
Do you know what I mean?
Like just that little anecdote of you looking in your fridge and seeing this nasty broccoli and being like, why does this happen?
That's your story.
You know?
And I thought--
And then all this comes from that
It's just like a very big topic.
Because I made my video and I looked and I don't really go that far into anything.
And it's five minutes already.
You know?
And I've--
But if you were to really narrowly look at what's going on what that broccoli, that's actually a really great five minute piece
Because like, I mean, I looked on YouTube.
They have this time-lapse of things decaying.
But there's nothing ever explains why, or how, or what does it
Right
Like, basically the only thing that there is on the internet in like video form is watching things.
But there's no explanation
Of like why does it become so darn smelly?
Right?
You know, I can go and read academic papers.
But academic papers are academic papers
Yeah.
But also the question-- I mean, if you think about this, if your center becomes instead of food decomposition, but broccoli.
The broccoli.
Right?
Looking at that broccoli.
So many questions come out of that.
Like when is it not OK to eat it?
It looks nasty.
But is it still safe to eat it?
I'm like what's happening in there?
And how long can I expect my broccoli to last?
There's this cool people you might end up wanting to talk to you.
That I went to an innovation conference last year the-- I'll have to look up the name.
These people made this device to be able to tell you when you're food is rotting in your fridge.
And it's an innovation that's happening on campus that you could actually go talk to those people.
That they've actually really, really thought deeply about food waste.
And created a device that attach to your fridge to tell you like bananas are going bad in three days.
You know?
And they are people on campus.
So I have to think about their name, but there's this really cool lab that's actually really exploring this topic that you could talk to
And then I actually just had a random idea that was completely unrelated to everything I had before.
Which is the concept of a world without decomposition, and what that would look like
Why do things die?
Or sort of--
A world in which things did not decompose
What would that looked like?
Yeah
Which better gets to the question, why do things die?
Yeah
Right?
Well, I mean things would die but then they would just sit there
Right?
Why do things decompose?
And if they had to be broken down by other means than just bacteria and fungi
The decomposition cycle.
Right?
Without a need for carbon and nitrogen cycles
That's much conceptually harder than a concrete broccoli.
But you can allude to this in the exploration of that broccoli.
Do you know what I mean?
Yeah
By looking at one thing really, really deeply, you actually allow yourself the ability to abstract from it.
That-- think for a few minutes about that broccoli.
A little more.
Cool How are you doing?
After talking to [INAUDIBLE], I realized probably my initial idea is I want to go into talking about how we go from [INAUDIBLE]
Time?
Yeah.
I will think of that.
And then another topic, closely related topic [INAUDIBLE].
The one I submitted was the pre-conceptual time traveling
Is time travel even possible?
Is that the question?
No.
Not really.
More along the lines of problems with time traveling.
Like for example, there's the grandfather paradox.
And then the idea and theory of the multiverse universe.
And multiverse theory [INAUDIBLE]
But I mean, even like the idea of like is time travel possible is a really cool concept
But it's really broad.
Right?
That's why I decided not to delve into time travel, if it's possible.
Rather looking at what happened-- why concepts for then.
I think maybe it's--
I think you're still too broad
Yeah
What-- to tell me more about what you actually do in your studies
That's what you think--
But what about-- like, there's got to be something in there that's really exciting to you that you could explore
I mean, yeah.
But I really doubt kids would even really want to see how a computer does information.
Because mostly I do AI.
So, it's lot-- It's to do a lot with like filtering of information and deciding what we do with our information
But I mean, this came up with a different group.
Which is when type-- filtering information's actually really interesting.
Like when I Google something, how does my computer know what to send me?
That's actually a really interesting question.
Right?
If I'm using Google Shopper, how does it know--
Identify what you--
You know?
And explaining that like a system.
That's actually interesting
Bringing that layman terms can be challenging
But important.
Right?
That would be quite hard
I mean, is it possible to pick something that you do that's conceptually hard but to really flesh it out to challenge yourself to think about how to describe that or show that.
Like the train concept.
A little bit hard.
But I kind of got it.
Right?
Is there something that you could-- something in your work that you're passionate about and you know a lot about that you think is actually really useful for the public to know, as well
That's really hard to think about.
I honestly don't think anything's that relevant to the public
What do you want to do with your degree?
Probably robotics, itself
OK. Robots are really cool topics.
So what about-- what about robots?
They contain information.
Probably how they input-- how they take input and how they decide what to do
Great this is an awesome topic.
Can you think of one very specific question that you could answer, ask a robot, and how it would be able to figure out the answer it?
Yes or no?
Have you ever design a simple robot or a simple machine?
Really rudimentary
Or like is there any way of very simply showing how a machine makes a decision?
I'm trying-- you know what I mean?
What I'm trying to push you to do is think about it robotics were exciting to you, is there one teeny--
The concept behind them are quite abstract.
That's the thing.
They're really like mathematically behaving.
That's why I'm thinking of it.
I'm not sure how really it can be brought across.
Or whether is should be brought across in that space.
So I was actually exploring something maybe a bit more.
Maybe more on like a pain.
I was a medic back in the military.
So dealt with a lot.
Like every day we see people coming in and looking for pain killers.
So maybe a bit more like for people to understand better
Like do I take and Advil or an aspirin?
Like do I take a--
They're roughly the same thing.
Yeah.
Something like that.
What is the difference in two kinds of medication?
And why do we take one more than the other?
But we just assume more useful than the other.
I don't think so.
It's more like-- it's like doctor, MD.
You know?
OK. A bit more explanatory around there.
So I was thinking of something like that
But I'd hate to use to not talk about robots when that's what you're studying, and that's what you're passionate about, just because it's hard
I don't think it's because its hard.
I mean, if it were to be an instructional video for someone at my level, maybe it's not un-doable
But is there anything within robots that you think is a cool idea.
Simple.
A simple, simple, simple something.
So not everything about how a robot works.
But even how to get-- just something very simple.
I don't know your world old enough to be able to help you think about what is that thing.
But I challenge you to think for a few minutes really deeply about what within the robots is a simple enough concept?
Do you have an idea?
I actually-- just listening in and I know something I think is really cool
What?
I very basic on-- but this might still be too complex.
But probability based maps
Probability-based maps as in based on the probability of what-- something happening that you predict what that is?
Like when the robot has to construct a map of an area it doesn't know
Oh, mapping.
Yuck.
Not my favorite project.
Done before.
Yeah
But if you can think-- maybe you brainstorm for a second about robots to see if you can get narrow enough that it's a concept
I mean, I think that there's a seedling here.
Right?
But like what is the relate-ability and the drama that's hooking people in?
Right?
Because I personally think that it's fascinating that we live in the 21st century with all these super computers.
Yet this seemingly basic problem of you have constraints x, y, z.
You can only work four hours in the day.
You need to make chairs and you need to make tables.
What's the optimal number of chairs and tables to make?
Like, that seems like a question that we would be able to solve.
And the fact that we theoretically actually can't is interesting.
Except there's not enough at stake in that example.
Right?
Like people will say, oh, we lose a couple cents on making chairs.
Like, that's no big deal.
But if you say something like, because integer programs can't be solved right now, this is what's preventing us from knowing-- like this is the reason, or one of the reasons why companies can only achieve like certain profit margins, for instance.
What would happen if we can solve integer programs?
Like what would happen to humanity, I guess?
You know?
Hm
Do you know?
I mean, I don't know
It's just means that-- when they say you can't solve a problem, just saying that you can't solve is easier than the-- you actually can solve it.
By the way to solve it is to just go through every single point on the graph
So it's very inefficient
It's just complete--
So like it limits us to the types of problems we can solve
Yes
You can solve it for like very small variables.
Like in this case, it's two variables.
So you can solve it
Yeah.
But you can't solve until you proof the problems
The more variables than-- actually I was mentioning this before.
I think it's called it n problem
What?
You know how you were saying as long as the number of variables increased, the number of times you must try exponentially that you can't really solve the problem
There's a concept called NPI.
Or NP.
So it's about categorizing a problem.
How difficult a problem is.
And if it's harder than this problem--
I think that's very--
Like problem in the world can be made into a type of problem called a-- I'm forgetting like--
You should write this.
I think like maybe this might be specific enough
Yeah.
This sounds really-- because-- [INTERPOSING VOICES]
You don't have to baby anything down for the audience.
You know what I mean?
Like you don't have to talk to them like they're ten
Because that helps encourage the conversations afterward, too.
Because like when I watch a video about something that I'm unfamiliar with, if I want to learn more I'll go out and search the words that I don't know.
Like as you with the sixth graders, maybe they kept asking questions.
If they're curious about something they'll keep wanting to know about things.
And so-- and they're are a lot more intel-- you think sixth grade was way too long ago.
But they were very like they're actually intelligent.
They're actually curious.
So you don't want to take out things that you're saying just because you think it's too intelligent for them, or it's too far above their level.
Because if you transition well into it, if you script it properly, and do the proper build up-- here's your introduction.
Now these are the details I'm going to talk about.
Then they'll remember some of those details, and look up the things that they still want to know
Is it OK if we just use the vocabulary, but instead of the-- because to say what a problem can become two separate [INAUDIBLE].
Like a lot of--
Yeah.
So for tonight's assignment, we're going to have you guys just write a rough draft of a script.
And at this point, I think it's OK if you want to err on the side of having too much jargon.
That's OK. Because George and I are going to workshop all the scripts with you tomorrow.
And we'll-- It's always a fine balance.
Because you don't want to sit and define everything.
Right?
Like, it's OK to challenge your audience.
You don't want to alienate them by talking in a language that they don't understand.
Right?
So it's a balance between that.
I think for tonight, we want you just like get something on paper.
And so don't get too stressed about that.
Really focus on what the overarching story's going to be
This is day two of the class and scripts take like a long time to write and refine and--
So we're not expecting perfection or anything
I'm just concerned about like something like this because there's no like visual elements like what you were saying
So do you want me to show you the computer-- Or have you see the computer?
Yeah.
I've seen-- You're talking about the one where he puts like these pin balls
With the balls.
Yeah.
I don't know any videos on top of my head.
But what you might want to try doing is just looking on YouTube for maybe like VSauce of Veritasium videos that were about computer engineering and see what they used visually.
Sometimes it's just their persona like on screen.
VSauce does that a lot because he talks a lot about psychological concepts that are sort of hard to visualize.
And it's just him on screen talking.
He's just visually engaging that way.
I do agree.
This is going to be hard to-- it's going to be hard to visualize.
But maybe this is a place where animations could help.
And I always think animation should be used sparingly.
People have a tendency to rely on animation because think it's like cheaper, or easier to make.
And when Josh come in, maybe he and I and you can talk a little bit more about how you would implement that exactly.
But having a person with just an animated overlay like you saw at the beginning of that video, that can be a really simple solution to a way to engage people visually.
But I think the idea is-- personally, I think the idea is very fascinating.
The notion that we live in a world where we think we've achieved so much technological prowess.
But a problem that seems so basic like that understanding how to solve four, five, six things is actually incredibly challenging.
And this is why.
This is the thing that we can conceptualize in our minds as human.
But when you try to get a computer to do it, like it's actually impossible.
It's impossible.
I went to this talk awhile back given by one of the guys who runs the Humanoid Robots Group at MIT.
And he was saying how it's so hard to program a humanoid robot, a bi-pedal robot to walk like a human behaves.
And it's because we walk every day.
Right?
And we don't think anything of it.
But to model the movement of your knee bending and putting your foot down, it's actually possible to mathematically model with a precision of reality.
And I don't remember the exact math about it.
But I remember hearing this notion that like there's no way in math for us to model something that we do in reality every day.
Right?
So even if we create like the world's biggest super computer, a robot is never going come close to what the human body can achieve in their sleep.
You know?
And I thought that was such an interesting concept that humans are still like so complex, even with all the advances we've made in robotics.
And that was a theme that we tried hit in one of our episodes, actually.
Does that help a little bit?
I don't know.
Maybe you just have more questions.
So at the end of the whole treatise, we each will come with a video and post it
Mhm

Yep.
It'll be great
Can I just picture--
Sure
So I mean, I have one of an algorithm.
And like the simplest algorithm I can think of this is one search.
So I was just thinking like wal-- You know, wal-- or whatever you call it.
Where is Waldo?
Oh, Where is Waldo?
Yeah.
So like just-- it would probably be like just some guy just hiding in the building.
And then he'll be like, what's the best way to find Waldo?
Yeah
Yeah.
Then we'll just be running around.
Row
Yeah
Yeah, like-- so first wave, open every door in the university and you just keep going that way
Right
Then there's another way where you keep going up.
Yeah but then I start to realize that the energy might break down somewhere half way.
Because the algorithm on the [INAUDIBLE] was a binary search which is just like, you cut half.
And then you see whether, is Waldo on the right side or the left side?
And then you cut half again.
And less, and less
I think that's very interesting.
What is the final application of search algorithms?
Google
Like Google?
Yeah
Yeah.
I think as long as you relate it back to the big picture of like why that's such an awesome thing.
Because what you don't want happening is people to think like, OK, cool.
I get that.
It's a binary search.
He split things in half and half.
So what?
Like why should I care?
Right?
Yeah
You should care because it's like what drives the most used website in the world.
And I think relating it back to that-- like that's what we tried to do with the switches video.
Is like, OK I get that the billiard ball hit with the switch on or off.
But we tied it back at the very end to say like this toy has 20 switches but most computer have like billions of switches.
And that's what makes a semi-conductor.
And that's what makes every modern electronic possible.
As long as you relate back to the big picture, to the thing that people relate to, the thing that people use every day.
I think that can be powerful

I mean, I also-- I like this path in GPS thing.
Like how exactly does a GPS work?
How do you find your way?
You know?
Knowing what-- knowing your location in GPS?
I know but I was-- like mapping a path, that's a different problem

Yeah.
GPS--
It's like how does Google Maps know?
Oh, I see what you mean
We are about out of time.
Maybe we should get everyone back together
Oh, yeah.
Sorry.
Thank you, Danny
I don't have too much to say.
Except for just the daily assignment.
So another daily blog.
Thank you all for posting your stuff on Tumblr yesterday.
Keep in mind that you don't have to do a text post like I did.
I just did that as an example.
But if you guys want to do a video blog.
If you want to just vlog on your laptop webcam, that's totally fine.
But if you end up taking pictures of any of the lectures, or taking videos during the lecture, you're free to post those, too.
And just post them with a hashtag day two and your Kerberos ID.
All the stuff that is on Tumblr.
I'd also do the day two thing.
And then a quick 200 or 300 rough script based on some of the stuff we talked about now.
Don't worry about being too jargony in this.
This is really about pulling the overarching story.
We're going to work out the details of what's the best wording to use.
We're going to work out some of the minutiae a bit tomorrow during our scripting workshop.
For tonight, think about what's the point of the video, and what the overarching story of the video is.
Those are really the things I want you to think about tonight.
If you get so stuck, and you just can't think of something to write, write down some facts like I was showing earlier.
Just come to class with something that we can work with tomorrow.
Because tomorrow's going to be a really informal day.
[INAUDIBLE] and I are just going to workshop and then table read some of this stuff that we've written.
And figure out how to implement the theoretical things that we were talking about in the last two days in a more practical sense.
So does anyone have any questions about those assignments?
Yes
Can you also bring back the canvas tomorrow?
No.
Tomorrow will just workshopping.
You will need them back at least on Friday.
I'll send you guys an email the night before and the day we meet in the final class
When I-- just as something to think about.
When I walked around and led to some of you guys more deeply, I feel like a lot of you are sort of struggling with this same concept of broad versus detailed.
And my two cents on it is that by asking a broad question but going very narrowly in with one example, you actually end up being able to bring people back up to that broader framework.
So if you pick something-- like we talked about food composition.
And we started exploring further.
Maybe talking about why a piece of broccoli rots.
And when is it too rotten for me to eat that I get sick?
Is actually the perfect focus for him to ask all these other questions about food composition.
And that by focusing really narrowly on that broccoli, we actually get to ask a lot more questions.
Right?
Like with shipbuilding.
Why does something really sink or float?
But that question diving really narrowly into one scenario of one ship sinking.
And why does it do this?
It allows us to be able to ask some broader questions.
Why-- the braces.
The opposite.
Right?
I want to talk about braces and what they do to your teeth.
But all right, let's think about if that's a specific example, what's the question that we're trying to think through?
And I feel all into a kind of in that place right now where you're trying to figure out what specific story do I tell in order to get these bigger, bigger concepts?
The narrower you are in that microcosm, the easier it is to extrapolate out.
Because it lets you really dive deeply into that topic.
So if you have a really, really broad topic still, see if you can get it really narrow.
And then from that it allows you to get broad again.
If that makes sense
And that's why I personally like going with the whole just trying to list out as many facts as I can with this snot thing.
Because it was just sort of, snot is awesome.
OK. Like, you know, why is it awesome?
What does that even mean?
And so I'm listing out all these specific things that people study about it.
And that helped me delve into the tangible examples that I gave.
That allowed me, at the end, to go back out to the story of this material that protects you every day, and is amazing and awesome
And I know we're just about-- we're actually over time, so I apologize.
But I realize that some of you may not know how to implement Elizabeth's task of asking you to actually write your script.
And that may be an overwhelming task.
Is that feeling like that right now?
Like sitting down and actually putting something on paper, you don't know where to start?
Or do you feel like you know to at least dump something down?
Sort of like you know, Paul, colon, says this.
Picture of something in here.
Like just jot it down.
Don't worry about format.
Don't worry about any of that
We're going to go over all the details tomorrow.
We just need an idea
Yeah.
Just dump-- so when she says script, that word may be intimidating right now, because a script can look certain ways in different genres.
Like a film script looks very different from a theater script, versus this kind of transcript.
Don't worry about anything other than getting some ideas on paper.
And we'll deal with all of the other stuff later.
If that makes sense
And I'll stick around afterwards, if anyone wants to talk specifically.
Yeah?
And just upload all that stuff to Tumblr
And we're both available.
And we're all available via email.
So if any of you are struggling tonight, and you need bounce an idea off of one of us, don't hesitate to do that.
I do not check email usually until after 8:00 PM when my son goes to bed.
But I will respond.
So if you have a question or anything, don't hesitate to bounce ideas off.
And I do really get back to you guys, as does-- do all of us.
So that's all for today.
You guys can pack up and leave.
But really just get ideas down more than anything.
We will help you with everything else tomorrow
Even if drawing is the way that helps you.
Just get something down
But do get words down.
We need to have like some words.
That would be great
Is the script supposed to go on a tablet, or?
Yeah.
Yeah
Good work, guys
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So this is our plan.
I'm going to talk a bit about writing.
We're going to practice some writing with the examples that you uploaded and then some other examples that are not yours.
And then I'll talk a little bit about hosting.
And then if we have time-- I don't know if we will-- at the end, we'll do a little bit of hosting practice.
But part of the hosting practice will come with the writing practice.
It's hard to separate them.
I'm going to talk for like maybe 10 minutes about writing.
And then we'll just get right into the practice.
I'm going to overload you with a bunch of guidelines or rules or however you want to think about it.
Don't worry about remembering them or writing them down.
But we'll come back to them all as we do the practice, because that's really how you learn, is by doing.
This is one way to write.
It's kind of how I approach writing.
You start with an idea.
You're like, hey, it'd be great have a video about whatever, x.
And then the explosion phase is the, I'm going to think about everything that could possibly relate to this idea.
I'm going to write it all down.
I think you saw an example of that with Elizabeth's snot brainstorm.
Brainstorm is another word for it.
And then from there, you sort of go outward and then you distill it down to the essence, like what is the actual point of the video, who, what, how, why.
Where is sometimes important, but not usually.
And then the stuff that we're going to talk about today, this is the writing piece of it.
So when you write a script, you want to pay attention to structure.
So what do you talk about first?
What's in the middle?
How do you end it?
That kind of thing.
Tone-- are you going for funny?
Are you going for dry?
Are you going for informal?
Are you going for formal?
Jargon-- you want to generally try to avoid jargon, especially if you're shooting for high school, middle school, that range.
By the way, it's ironic that I'm using the word jargon.
Jargon is a technical language, which jargon is itself a technical word.
So that's sort of an ironic thing there.
Visuals-- you always want to write the script, because video is a weird medium.
I mean, you've got the image, and then you've got the soundtrack, and you want to make sure that they are related and don't duplicate each other.
So that is a trick that we'll get into.
I was reading some of your examples and there are particular sentences where this is going to be really important.
So if I don't talk about that on my own, remind me.
Metaphors and jokes-- it's really hard to be funny.
So if you think you can, and you want to try, then I would say go for it.
But generally, I try to avoid jokes.
Metaphors are similar, it's kind of hard to write a good metaphor.
Do we all know what a metaphor is, by the way?
I showed them the [INAUDIBLE] video yesterday
OK, cool.
So those are sort of-- the stuff on the left is in every video, you've got to pay attention to.
The stuff on the right, the metaphors and the jokes, are sort of advanced.
Anyway, then once you sort of get your first draft together, you explode, you distill that into your essence, then you condense it, and you write your script.
And this doesn't happen once.
This happens many, many times.
Typically, a script will go through between 4 and like 12 revisions before we actually shoot it, up to and including-- we're rewriting five minutes before we shoot.
Because you say it out loud, and something just doesn't sound right, and you feel like you have to change it.
So this is generally how my writing process works.
Other people do it completely differently.
This is sort of an MIT approach.
It's very sort of structured and logical, but it doesn't have to be this way.
If you don't want to write like this, you absolutely don't have to.
The first writing guideline, this is in addition to the flow chart, is-- and I think you guys talked about this a little bit already-- know your audience and write for them.
And I have this note down there, not necessarily to them.
So most of the videos that you're going to be working for are aimed at people younger than you than you, and sometimes people a lot younger than you.
I mean, maybe even down to like 6th, 7th grade, middle school.
So you want to write to their level of education, but not necessarily talk as if you were talking to a middle schooler.
Does anyone know what I mean by that distinction?
I'll cold call if people don't speak up
Like, don't baby talk?
Exactly.
Yeah, exactly.
Don't baby talk.
So give me an example of like-- pick a sentence, any sentence.
You can read it off the intellectual property form in front of you if you want.
But give me the non-- just the regular version.
And give me the baby talk version, just so we have a sense
[INAUDIBLE]
No, that's exactly right.
And you guys notice that it's funny when he does that.
There's just something ridiculous about-- you call it baby talk or talking down to your audience or something-- that you immediately don't pay attention to what he's saying.
You pay attention to how he's saying it.
And you've lost your audience immediately.
So that's a really important distinction.
By the way, this holds true if you're just interacting with kids anyway.
Most kids are smarter than we give them credit for.
And people baby talk kids all the time.
But if you just straight talk to them, you'll be surprised at the kind of responses that you get.
Rule number two-- or guideline, sorry, number two-- is write it like you would say it, because you will be saying it.
What I mean here is your script is a draft.
And it will always be a draft.
You will never have a final script, because nobody's going to log on to the internet and read your scripts.
What they're going to do is hear you say the words that are on your script.
So even if a sentence is great on paper and you read it in your head and it sounds awesome-- if when you say it out loud it, it sounds weird or it's hard to understand or whatever, then it doesn't matter how great it looked on paper.
It's not an effective sentence.
This is the most important thing we'll talk about today.
We'll come back to that when we do the practice.
This is another version of the previous slide.
When you-- and you guys are all know this because you read scientific journal articles and stuff-- when you read that kind of text, it's very dense.
It's extremely well structured and logically laid out.
It has a lot of jargon.
It's very precise.
There's very little ambiguity.
And it's understated, and it's elegant.
Meaning like-- usually people take the fewest possible words to say something, and that means using long and complex words.
And it is beautiful to read.
It's great English.
But if you try and read it out loud and you do it on video, again, you'll lose your audience instantly.
It's just as bad as baby talk.
When you talk-- people talk in-- people are very repetitive.
They say, "uh."
They say, "um."
They-- uh-- like I just did.
They don't really use fancy words.
If their audience doesn't understand, they stop and they try another approach.
It's a lot more free-form, a lot more inconsistent, a lot fluffier.
And that's what you want to shoot for, because you will be on screen talking to the camera as if that camera is another person.
And so that's the write it like you say it.
That's what I mean when I say write it like you would say it.
Another-- and I think this is the last guideline-- is you really want to be accurate.
Because, obviously, you don't want to say things that are wrong in your videos.
But accuracy is not the same thing as detail.
So this is the difference between going back to the last slide.
Being very, very, very precise usually requires a lot of detail.
If you want to say exactly what you mean, it probably will take you a couple sentences to say it.
But if you want to be accurate, that is going to-- and especially if you want to be accurate for a short five minute video-- that's going to mean leaving information out.
And that's OK.
The test that I like to subject scripts to is, can an eighth grader understand it?
If so, great.
And can a PhD In the subject matter of that script read the script.
And if their reaction is, wow, this is surprisingly accurate, then you've done a good job.
And that's all.
That's it for my rules.
Hopefully that was less than 10 minutes.
What I want to do now is put these rules in to practice.
We're going to start with a quote that is not one of yours.
And then we'll move into your scripts.
So I'm going to go ahead and-- let me pick one.
Let's do the first one.
OK.
So let's see.
Everyone take a second and just read this.
So you'll notice the citation.
This is from a paper.
It's written English.
Can someone just shout out some of the rules that would make this bad for speaking and reading aloud?
In fact, why don't we-- let's actually read this aloud just to see how it sounds.
So anthropogenic CO2 emissions are contributing to global climate change and could negatively impact our way of life if serious action is further delayed.
OK.
So what rules are we breaking, or what guidelines are we breaking?
I think just hearing it out loud, it seems kind of-- life if you were presenting it to someone, it'd be a long sentence.
And it's very statement.
It's not conversational.
It's not like you're-- you know, it's not written like you would speak
Exactly.
What else?
The word anthropogenic is difficult to hear and remember
Exactly.
It's also jargon.
It means human-made
It's a very passive sentence
It is a very passive-- what you mean by that?
Explain
Well, if further delayed-- I mean, what you really mean is this could be troublesome.
This is bad.
Right?
Or it could be bad.
Right?
But their way of stating as further delayed isn't very-- it's not a very active way of saying the same thing
So here's another interesting thing.
Negatively impact-- those are not difficult words.
It's easy to understand what they mean, but let's think about that for a second.
So what doesn't negatively impact our way of life mean in colloquial language?
What would you say to someone if you were going to say, in 100 years climate change will do what?
[INAUDIBLE]
Yeah.
Will it negatively impact our way of life?
No, it's going to kill us all.
That's sort of the real-life subtext of this sentence.
What else?
The are a lot of complicated grammar structures-- are contributed, is delayed
Mm-hmm.
You have to-- I mean, you wait till the end of the sentence before you get the payoff.
Right?
You don't understand what the point of the sentence is until you get to "negatively impact our way of life if serious action is further delayed."
And really, you don't understand-- I mean, the point is we need to act now.
That's what this sentence is saying.
If we act later, we're screwed.
And you don't get that until, "if serious action is further delayed."
So yeah, the payoff really comes at the very, very end
It's a fun exercise to eliminate all of the adjectives and look at the nouns and verbs.
Right?
Just take a second and see if you can do that.
It actually makes it really simple.
Emissions are contributing to change and impact-- like picking out all of the things that suddenly simplifies everything-- [INAUDIBLE].
There's a lot of descriptors in there that are very distracting.
So now that we've done this, let's rephrase it as if we were saying it to someone on camera
Or what if just to someone in general?
OK, yeah, to someone in general.
Say it to me.
Anyone
If we don't act now, [INAUDIBLE]
Right.
Another version
Man-made CO2 emissions are hurting us
OK. Another.
Yeah
Humans create CO2 emissions.
We must act now or suffer
I like that.
We must act now or suffer.
That's good.
Anyone else?
Someone who hasn't said anything yet
If we don't stop emitting CO2, life will look very different soon
Good.
OK.
So the point I'm trying to make by having five or six of you say it each a different way is that there is no one right way to write a script.
There are lots of interesting, engaging, dramatic ways to say the same thing.
And we are going to get into that now with your examples, which we're going to spend considerably more time on
Before you move on, George
Yes?
So it's always an Interesting challenge balancing avoiding being too jargon-y and ending up baiting your audience.
Right?
Like, there's no reason why you can't say that words, climate change.
Even though those are technical terms.
[INAUDIBLE] What was it?
Anthro--
Anthropogenic
Anthropogenic.
I mean, we talk about this with [INAUDIBLE] scripts all the time.
The way you deliver a word, for instance, can help a lot with conveying.
What was the word-- [INAUDIBLE]
I mean if you want to-- if you're going to use that same word four or five times throughout the script, and it's an important word, and it relates to a demo or something-- if there's some reason for you to use jargon, it's OK to define it.
As long as you just define it like you would to another person.
Like you say anthropogenic means human-made.
And then you go on and use it later on.
That's fine.
You don't want to do that like six times in the video and define six different words, because then you overload your audience.
But that's true.
There is-- if you find yourself having to say human-made, man-made, woman-made.
Like if you are stretching to try and find alternatives to word five or six times that you could just define and get it over with, that's fine.
I wouldn't limit that probably to like one or two words a video, or one or two words per like three, four minute video
And the other reason why it's really important to actually do the exercise of reading your script out lout, or even just flipping your script over and not looking at it and trying to describe the contents to the person next to you, is that-- a very bad habit of mine, which I am constantly finding I am annoyed toward it a lot-- is that I get very newscaster-y.
So I may end up having a script that sounds good when you say it out loud.
But it sounds like you're a reporter talking about the latest news update [INAUDIBLE].
CO2 emission-- can you go back to that quote?
Yeah, like the middle part isn't actually that crazy to say out loud.
It's contributing to global climate change and could negatively impact our way of life.
Right?
Like that doesn't sound too crazy to say out loud.
But it sounds like someone is saying that in the context of, I'm here at the MIT Course One lab where researchers are finding a way to prevent global climate change from negatively impacting our life.
Right?
It doesn't sound like I'm just talking [INAUDIBLE] like a lot of men in admissions are harming [INAUDIBLE] life or something like that.
I also-- [INAUDIBLE]
Yeah.
The way you write influences the way you're going to naturally say something.
And you could say this sentence both ways.
So you could say it, you know, anthropogenic CO2 emissions are contributing to global climate change and could negatively impact our way of life if serious action is further delayed.
That sounds like the 7 o'clock news, right?
You could say you could say it like this.
Look, anthropogenic CO2 emissions are contributing to global climate change.
And that could really negatively impact our life if we don't do something right now.
I mean, it's basically the same line, but it's said two completely different ways.
And it's much harder to say a line like this naturally, but it's possible.
So the way you write can impact how you end up saying something.
And that's important too.
We will-- whoever of you ends up shooting with us in the last week of January, we are going to guaranteed say, no, try that again.
It sounds too new newscaster-y.
And this is a style choice, but for Science Out Loud, it's not news.
We don't want it to come across like, here at MIT, we're talking to you about whatever.
Because MIT already has a news office.
They do that well.
Our goal is different.
And so making that style choice consciously from the beginning is important.
But if you want to write like a newscaster-- I mean, if you're writing for a news show, if the point of the show is news, then that could be perfectly fine.
Good?
Were you going to show the Ira Glass clip?
Later for hosting.
Yep.
It might actually be fine to show that now while we're talking about it.
Let we find it.
Yep, here we go
[INAUDIBLE]
Oh, yeah.
There we go
[INAUDIBLE]
Do you guys know who Ira Glass is-- This American Life?
He's a radio personality.
He's been doing this for many, many years
[INAUDIBLE]
I'm going to play you a clip of tape from my eighth year.
[MUSIC PLAYING]
[ON RECORDING] It's not such a long way from the local grocery store to the international debate over whether sorghum and meat production are causing corn to decline in Latin America.
There's a general air of prosperity here, partly thanks to Mexican imports of US grains, which helps boost our farm economy.
Mexico is now one of our biggest grain customers, buying a half billion to $1 billion worth every year, including corn to feed its people and sorghum to feed its livestock.
Like what am I talking about?
Like, I don't even understand-- like, I wrote this.
I don't even understand what it is.
OK, also, like every part of this is ill conceived.
OK?
The writing is horrible.
You can't even follow what I'm talking about.
And then the performance like-- OK, just a little tip if you're performing for broadcast, you don't underline every third word for emphasis, because it sounds really unnatural.
What you want to do is you want to talk the way people normally talk.
[ON RECORDING] This helps cut our own trade deficit and benefits everyone in the US economy.
But in Mexico, this policy has lead to fewer tortillas for the poor and unappetizing tortillas for everyone else.
Again, like this is the most moronic kind of like-- it doesn't mean anything.
And it's hard.
It's actually kind of an interesting story, which I'll say to you in a sentence.
Which is, because Mexico produces a lot of stuff that they ship to the United States-- tomatoes and all sorts of really wonderful food that we eat here-- they don't make enough corn for their own people.
That's the story.
So for us to get really great tomatoes, or semi-great tomatoes, year round, basically, Mexicans eat worse.
That's the story.
And it's kind of an interesting idea.
Right?
Like that's actually sort of like a cool idea executed in the worst possible way.
So he's talking about two things there.
The first is how he wrote the story, and the second is how he delivered the story.
Right?
So in the second part of the clip where he tells you the point of the story in like two sentences and you can actually understand what he's saying, that's a writing issue.
And then the point that he makes about, don't underline every third word, once you hear that-- like, if you listen to the local news or even the national news-- I mean, once you hear that, you can't un-hear that.
It's incredible how pervasive that tone is.
Because it makes people sound very important and official when they choose certain words to underline and deliver correctly.
And, you know, that is something that is fine for the news, because it has a purpose.
But it's just-- I mean, I think it's terrible.
And Ira Glass does too.
If you listen to-- his show is called This American Life.
And if you could just listen to the first like three, four minutes of any show, it's a great example of really good hosting.
Marketplace is another show that has a similar style and feel to it that's good to listen to.
And then everything-- I think eight minutes past the hour or something-- NPR-- or every hour-- NPR goes back to their local news stations.
So you can hear the difference between newscaster-y delivery and natural delivery.
It's a really good exercise.
Like I say, once you hear it, you can't un-hear it.
So let's go back to writing.
What we're going to do now is I'm going to have-- I picked, I think, three or four scripts at random.
And I'm going to have whoever wrote the script come up here and read it.
And then we're going to do a group critique of the script.
This is a humbling experience.
We are going to say-- we're going to point out ways that your script has been better.
But that's OK. That's how you get better.
And it's an experience that we will go through again and again.
Well, I won't-- because I won't be here, but you will with Elizabeth again and again throughout the rest of the course, and especially in January.
Because we are going to be very anal about, does this make sense, does this convey the idea we wanted to convey, does it do it effectively?
So this is, I think, going to be your first experience of that.
So who wrote that?
And by the way, I picked random-- all right, come on up-- I picked random sections of the script to talk through.
Do you want him to put on a mic?
Do you want him to wear a mic?
Yes?
Does it matter?
It doesn't matter
All right, come on.
What's your name again?
David
David.
Nice to meet you.
I'm sorry you're going first.
All right.
So stand up there.
You can read the text here.
Deliver it like you would like you would deliver it if you were being recorded, which you are being recorded.
And I'm going to sit in your place.
And we're all going to have a conversation about it
[INAUDIBLE] I was sitting next to my roommate.
So I was actually kind of shy about reading out loud.
So actually, I tried to read out loud.
But I read it semi-internally.
So maybe that's why some of this would sound a bit odd
That's OK. No judgment yet.
Let's get it out there
Yet?
No judgement ever.
This is a safe space
OK.
So my topic's about why do some people handle cold better than others.
So why I was interested is that, why is it that some people are more fearful about the cold, that they'd rather die than be caught outside without all the winter gear, mask and all, while others can wear one layer for morning jog.
Actually, I say yesterday morning.
And what makes all the difference?
So the understand this, we first need to find out how our body reacts to cold.
The first point is that the changes that way it burns energy more efficient.
We actually burn more carbohydrates through our metabolic system to generate more heat.
Imagine a giant furnace within our bodies.
As winter starts to come, we throw more bits of carbohydrates and start energy to burn, and create more heat to warm our bodies.
Then we cue pictures of furnace and burning stuff
OK, great.
OK. You can come back and sit down.
Thank you.
Excellent.
I want to point out something that actually has nothing to do with writing.
When you were-- very similar to how there is a newscaster-y tone to deliver things, especially when you're reading off a teleprompter, there is also a reading tone to deliver things, versus reading something out loud versus saying it to someone.
And when you got to the end of your first paragraph-- so you said, without all the winter gear on, mask and all, while others can wear one layer for a morning jog-- I saw that yesterday actually.
Did anyone else notice how when he switched from reading to talking, you instantly snapped to attention?
Because, I mean, here his focus is there.
And then when he switches over to, I saw that yesterday, by the way, he looks at you all.
He makes that connection with you, and you respond instantly.
Right?
So that's something that is maybe better when we when we talk about hosting, but since you did it right there, I wanted to point it out.
OK, so now in one sentence, someone summarize the sort of main point of this part of the script for me.
Without re-reading it-- sorry that was a crucial piece of information.
Don't read it again
Carbohydrates generate heat
OK, how?
Carbohydrates are the fuel to generate heat
OK, that's a little more specific.
Does someone else have a different sentence they were thinking of?
Yeah
Our bodies process carbohydrates differently, so we react to differently to the cold
OK, cool.
What technique is used here to explain how we process carbohydrates?
Metaphor?
Explain.
Where?
The giant furnace within our bodies
Does it work?
Yeah.
I think it makes you understand something that's technical and scientific in a more relatable way
What do you mean by relatable?
Well if you have an oil furnace at home and you're used to that thing cranking out in the winter, then you can kind of relate to that
Right.
So your body is an incredibly complex piece of machinery.
It is basically unlike anything else that we're used to in our world.
But a furnace, an oil furnace, or even a fireplace, or whatever, something that burns stuff, is something that your audience pretty much you can count on will have seen that.
So relating this very complex piece of machinery to something that's much simpler is a great way to clue your audience in immediately on what's going on.
We can talk about metaphors more later and what the tricky parts of metaphors are, but what else here makes this effective?
What else makes you want to listen?
Makes you interested to know more?
What techniques?
Asking the question?
Yeah.
So the first paragraph is all questions.
That's all it is.
it's just sentences of questions.
So why is that useful?
Why does that help?
[INAUDIBLE] they have to know the answer.
They [INAUDIBLE] so when the [INAUDIBLE] speaks, they will want to continue
Yeah, exactly, and that's a really common technique.
You start with the question with-- you start off the video, excuse me, with either one or more questions.
What else is effective here?
Do you want to say something?
I mean the questions themselves work.
It's not that pose-- you could have posed much bigger questions like why do I get cold?
That's not a really compelling question, but it's a-- the second question, why is it that some people are so fearful in the cold that they would rather be caught dead than go outside?
And when you said like I just say this outside, I see it outside all the time too, so the questions themselves immediately jump into relating to the viewer
And it's something that is a common-- we didn't talk about-- your idea should be good, right?
But having a good idea is an important part of making an interesting and readable script, and this is something that we've all experienced.
I mean you walk around MIT, and you see people walking around with shorts in the weather outside.
It's like 12 degrees outside or whatever.
And you think, my god, they must be insane.
But no, maybe they actually don't feel cold like we do.
And that is something that is one of those things that you see a lot, and you don't really process it, but when you really stop to think about it, you're like wow, that is really bizarre.
I want to know more.
So you're right.
It's not just asking questions in and of themselves, but it's what are those questions, and do they point you somewhere interesting?
What else is effective here?
There's a "we" right there that I feel its absolutely essential
Which one?
Right in the very middle of the page, there's-- middle, middle, middle
To understand this, we first need to find out how our body reacts to the cold
That "we" I think is so essential, because if it were a you or a me it wouldn't work, but the "we" is very appropriately engaging
What does it do?
I means that you and I together are going to figure this out.
I'm not going to tell you, but we're going to explore this together.
And that's the perfect tone for this piece
You are leading your audience on an adventure of understanding.
You're the guide, because you're on camera, but you're bringing them with you-- versus I'm just going to open your head and dump this information in your head, and then you're going to know it.
That's not what this.
This is an odyssey of exploration that we're going to go on together.
Anything else that we want to point out is effective?
OK. What could be improved?
I think the pacing makes it easier for the host to read, so just have less commas but more maybe periods, and just split it up into shorter sentences.
So you don't have to keep [INAUDIBLE] for a long time
Yeah the sentences do get kind of long.
The last paragraph is just two sentences and then the middle question in the first paragraph is a very long sentence.
So I agree.
That's one of the default patterns that you fall into when you're writing something.
When you're thinking of something in your head, you tend to think in sort of long, unbroken streams of thought, and that tends to manifest itself on the page.
Whereas when you talk, it may not be sentences in the traditional sense of-- OK, I'm going to stop talking now-- there's a period.
And now I'm going to start again.
But you do have pauses in your speech that let your audience understand what's going on, and you can count those as sentences.
What else?
Maybe putting the metaphor before the scientific-- picturing the giant furnace [INAUDIBLE]
So give us a version of how you would imagine that would go
Imagine a giant furnace in the winter, burning coal and charcoal to generate heat.
Now imagine the body, the metabolic system as that giant furnace.
And then the coal and charcoal as the carbohydrates in your food
Yeah, great.
That's an excellent point.
So the way this script is written, the simple technical answer that has no metaphor comes first, and then you have the metaphor.
But the way that-- remind me of your name?
Kenneth
Kenneth.
The way that Kenneth said it is, OK-- we're going to start simple.
We're going to start with something you know, a giant furnace.
Got that in your head?
Great.
Now imagine that your body is that furnace, and instead of coal, you've got sugar, et cetera, et cetera, et cetera.
And you the metaphor like that, and that's sort of a principle of start small, start easy, start with something you know, and then build layer by layer on that.
And that is something that is not one of my rules, but easily could be.
Everywhere except the very first sentence of the video, it's a great technique to make sure that your audience is caught up with you all the time.
And I say everywhere except the first bit, because the exception is that if you want to start with a compelling hook, you might start completely out of order, and then come back to something later in the video just for the purposes of hooking your audience and getting them through the first 10 seconds.
But other than that, that sort of expansion-- I sort of look at it as a funnel.
You start really simple, and then work your way out to complicated.
What else?
I'm not really sure whether you need the middle sentence
Which one?
This one?
Yeah
Let's try it.
So give us a read-- read the last few lines here, and then why don't you go into Kenneth's version of it.
So read the last couple of lines here, and then go into the furnace, and we'll see how it sounds
Well Alice can wear one layer or more layers for the morning job-- what makes all the difference?
[INAUDIBLE] What makes all the difference?
Imagine a giant furnace within out bodies.
Our bodies changes and burns energy [INAUDIBLE]
Great.
That's perfect.
So I would agree with you-- I think you're absolutely right.
I don't think we need the middle sentence, because you're asking a question, and then you go right into this image.
And it's clear that you're not immediately answering the question, because imagine a giant furnace is not the answer to what makes the difference, right?
But it's clear that, from your tone, the way you said it-- so you say-- what's the difference?
Well, imagine your body is a giant furnace.
So the way that you say something can-- essentially, you're saying this sentence in the tone of your voice.
So you don't actually need to say that sentence out loud great.
Anything else we can eliminate by the way?
What can we cut?
I think you already mentioned it, but the second sentence-- I think you can say that a little less wordy
Give it a shot
So why do people hate the cold?
That they'd rather die than be caught outside without all their winter clothes on
So the way you're doing it is to sort of piecemeal try and chop out certain words, right?
Yeah
But take a step back, look at the meaning of the sentence, reformulate it, and just say as if you were saying it to me.
Actually, do say it to me
So why do we hate the cold?
Why do some people have to bundle up and others don't have to wear as much clothes, or something like that
Yeah, I think you're on to something.
You walk around on a cold winter's day-- why are some people bundled up and some people wearing shorts?
What's the deal?
That's a much shorter way of saying, essentially, that.
There is a theatricality in the way that the first paragraph is written that you lose by changing it to the shorter version, but that comes from the benefit of you're getting into the material faster.
But there is a trade-off there
[INAUDIBLE] You can also accomplish all you say and include the extra detail with your images.
So if you show an example of someone bundled up with all the layers, and then a jogger with barely anything other than shorts, then you have all the content that was in that sentence, but you're using the rest of the medium that you haven't worked with
You reminded me of the thing that I was going to talk about.
Thank you.
That's exactly what I wanted to talk about.
Yeah, so this is really really, really easy to forget when you're writing a script-- is that there's going to be a whole other layer of information that is available to your audience in the final product.
So to the extent that you can, I would absolutely encourage use of visuals that are complimentary to the script.
So the way that I-- I don't have the slides for this, but let's say that you wanted to show a synapse.
You guys know what a synapse is?
The part in your brain where neurotransmitters go?
So you've got this, this, and then-- OK.
So this is one nerve.
This is another nerve.
This is just the space in between the nerves.
And these dots are chemicals that this nerve is transmitting to this nerve.
These sort of like Poseidon spears are receptors for these chemicals.
Now the way that I said that-- if you were to write down what I just said-- these dots, this nerve, this space, but this area is the space between the nerve, you'd have absolutely no idea what I was talking about without the image.
Whereas if you didn't have this image, and you were just writing a script trying to explain what a synapse is, you would have to say, a synapse is the space between two nerves.
One nerve releases chemicals that cross that space and get captured by receptors on the other nerve, and that's how data is transmitted from one nerve to another.
So if you were writing a script without the visuals in mind, you can very easily fall into the trap of doing what I just did in writing a script that stands alone.
Whereas if you're thinking images, it's going to be mostly like, look at this.
Check this out.
This spaghetti shaped thing, those dots, that kind of language, and that's totally OK and even encouraged when you're doing a video.
And that's exactly what you said.
Another example is to basically talk in general terms and put specific visuals that compliment the script.
You were going to say something?
Yeah, just that it's not just that someone can wear less clothes, it's that they're comfortable wearing less clothes.
And to me that's an important concept that we can't use in that first paragraph.
Because my husband's like this, where he can be wearing half the clothes, and he's happy that way.
It's one thing to wear less clothes and be miserable, but it's another thing to be comfortable with that choice, right?
I see students on campus without shoes, and I'm like, what's going on?
But the fact that they-- it's one thing to wear less clothes and be shivering, and cold, and miserable, but it's another to have less clothes and actually be OK with that
And following that even further-- I just had a question in my mind.
Let's say you took two people.
One who can wear shorts in the dead of winter, and one who has to be completely bundled up.
If you transport them into the Antarctic or the Arctic where it's-- I don't know.
Pick a really cold temperature, subzero.
And just let them stand there naked for like 20 days-- who dies first?
Is it the person's comfort, or are they actually better adapted to survive in the cold?
Which is a really interesting distinction, because if someone's comfortable in the cold, but they're still going to die at the exact same time, then yeah, comfort is great, but we haven't, as a species, adapted to better overcome the cold.
Whereas if it's really your body is substantially different in a way that could help the human race survive the next Ice Age, that's cool.
So that's a really interesting point
And that's taking the familiar and making [INAUDIBLE].
It's taking an instance that you see and a question that maybe you very fleetingly have, which is how is that some people don't have to put on a huge winter jacket and can walk outside in the snow fine?
Whereas I have to layer up?
It's taking that familiar question and slowly building out to this pretty epic concept of-- are certain people within our human race better adapted for total catastrophe?
[LAUGHTER]
I'm not saying that you have to go that route, but I think that that's why watch some of these viral videos, because it takes the familiar and makes it unfamiliar
It also-- I mean, not to totally go a different route-- but a question that this could go into is are skinny people versus fat people-- does it have anything to do with your physical being?
Or is it your chemical being that makes you be able to handle the situation better, and can you change that?
Or is it just how you're built?
But if you tighten this first part up, it leaves you the room to explore the really cool questions.
We could easily cut all this text down into half it's length.
Why do some people handle the cold better than others?
How is it that this person who's wearing shorts in winter is fine, whereas this person is wearing five jackets?
What's making the difference?
Well, imagine a giant furnace.
That furnace is actually us.
You're taking out the whole middle chunk.
I also think you can highlight the unexpected nature of the furnace metaphor right?
You're taking the familiar ones, and you're entering the unfamiliar.
Imagine a furnace-- everyone can picture that-- but the crazy thing is that our bodies are furnaces.
We kind of did that with the [INAUDIBLE].
And then the whole end-- and winter starts to come and we throw more bits of carbohydrates and stored energy.
Carbohydrates and stored energy are similar enough things that you can distill it down into one phrase
Sugars
Sugars, yeah.
But by tightening all of this intro stuff up into two concepts, it leaves you the rest of the episode to explore the cooler questions
And then you can go with-- the metaphor, you can take as far as you want.
You can say, OK, well, are some people's furnaces more efficient?
Do some burn different types of sugar?
Whatever the answer ends up being-- I don't actually know what the answer is.
I really want to know, but whatever that is, you can develop the metaphor or not, depending on whether it fits and really explore this question that's fundamental to the nature of our survival, because there will be another Ice Age.
It's coming.
We just don't know when.
So we took this experience that we've all had, and we turned it into an existential crisis, which is great.
So thank you.
Next, I was going to do two examples from each script, but I think we should just move on, because we're running short on time.
Whose script is this?
All right, come on up.
All right, so you know the drill.
Give us a read
Once upon a time, in the year 2009 actually, Stephen Hawking held a party with all the usual stuff-- wines, hors d'oeuvres, and yet no one turned up!
And you'd have thought he'd be upset, but in fact, he was actually quite happy, because it just proven his own belief that it was impossible to travel back in time.
He had, in fact, sent out the invitations for the party to time travelers after the party had-- I already mentioned it.
A lack of partygoers actually meant that time travel was probably impossible, or just that people thought that Stephen Hawking was lame and didn't really want to go to his party.
So was time travel really not possible?
Yeah
Great, come on back.
So what is effective in this script?
Introduction, really-- It's not a full script.
Yeah, you have to raise your hand.
Just yell it out
Well he introduces a character who we know, but he's doing this party thing that's just like us
So we all love Stephen Hawking.
If you don't, leave immediately.
So right off the bat, Stephen Hawking is invoked, and that's great.
Now we know exactly who we're talking about.
We're all on the same page, and we all love Stephen Hawking.
At least your audience for this video is most likely going to know and love Stephen Hawking?
Not everyone does
And even if they don't, it's still a cool enough premise.
This one's doing the opposite-- taking something unfamiliar, and you're going to spend the adventure familiarizing yourself with it.
That someone had a party intending to have no one show up, because he was proving time travel.
It's like this pop culture fascination.
It's very Hollywood science.
It's the reason why so many of the special effects that you have on TV and so much of the science that does end up broadcasting on major networks.
They're all about astrophysics stuff, and it's because it's commercializable?
It's just cool.
Who here has watched Cosmos?
New or old?
New?
Both?
At one point in one of the episodes, either Carl Sagan, or Neil deGrasse Tyson, or probably both say that we have this inbuilt fascination looking up at the stars and wondering what our place is in the universe, and that is absolutely true.
And shows and movies capitalize on that all the time.
Interstellar, Gravity, are two of the most recent ones, but we've always looked up, and wondered, and written stories about it.
What else works?
So what's instantly different about this script?
This introduction versus the last introduction?
Something visual that [INAUDIBLE] something very visual.
In my mind, I'm thinking of this sad face, smiley face.
It's going to be quite funny
Take that even further.
It's visual, but it's also-- When Kenneth got up here and started talking to you, what was he doing?
He was [INAUDIBLE] face?
He was what, sorry?
Yeah, he was expressive, right?
He used his [INAUDIBLE] and [INAUDIBLE]
But even before that.
"Once upon a time," what's that?
Once upon a time.
It's a story.
You're telling a story.
And eventually, in this intro-- this is also a story that you're telling, but once upon a time is right off the bat, everybody knows that's story time.
And just like we're hard-wired to sort of look up at the stars and wonder where we came from, we're hard-wired to love stories.
When you start a sentence with once upon a time, you're listening
But that doesn't necessarily mean you should.
[LAUGHTER]
Again, take everything we say with a grain of salt.
It's not the tools that you use that make things work, it's the reason why you use the tools
So you don't have to say once upon a time here to know that it's a story.
Because even if you didn't say, you'd say, in 2009, Stephen Hawking held a party.
No one showed up, but he was thrilled.
Why?
Because he'd just proved time travel.
Let me explain to you how that works.
Blah, blah, blah, blah.
I didn't say once upon a time, but even though I didn't say it, my first two sentences are still this like, click-- I'm going to tell you a story.
What else is effective?
Actually effective or what you would improve
I think the last sentence kind of throws a loop in.
I'm unsure about what the body of it is going to be, because you take this really smart guy like Stephen Hawking.
He just proved that you can't travel back in time.
So is this going to explain why that's true?
Or it's just the next breakthrough where it actually is true.
I'm kind of unsure what's going to happen next
Anybody else a little bit-- are you saying that as something that works or something that could be improved?
To clarify, i guess
Was anybody else a little bit hazy on where the script is going, or what's been said already?
Yeah?
So when I read this, I was a little bit confused.
So what's the answer to this question?
Tell me in a couple sentences
And what happens after?
Summarize your script for me in two or three sentences
After that, we talk about what time travel is and how it's maybe possible to time travel, and why, theoretically, it's not possible to do it yet.
[INAUDIBLE] Maybe we just haven't explored enough yet.
We don't understand enough of the [INAUDIBLE] time travel
So the bottom line, main point of the video is we're not sure if time travel is possible?
That currently it points to the fact that we are unable
But most science says that it's not going to be possible?
Yeah
So we think it's not going to be possible ever?
But yet we're still asking.
We're questioning [INAUDIBLE]
I think to fix that, instead of maybe prove by-- I don't know-- you could just take a less qualifying word like the possibility that he has disproven it, or something that gives him some sort of air that there is the possibility that it isn't true.
I don't know
So let me ask you-- without going back to this intro, tell me again the underlying theory behind the party, and what it proves that no one showed up?
Or doesn't prove, or we think, or whatever
Prior to this, Hawking has said [INAUDIBLE], but he believes that it's not possible.
So he set up this extra experiment
He believes that time travel is not possible?
So he set up this extra experiment.
He hosted the party.
It fails on [INAUDIBLE].
Then he sent out invitations after the party to time travellers.
So he even included specific coordinates and specific date and time.
So when no one showed up, [INAUDIBLE] so it's not possible
I see, OK. Now I get it.
So when you read the script the first time-- whose understanding of the script changed based on hearing that explanation.
I must say that I actually know this point, but reading this I didn't understand, until I [INAUDIBLE].
And it actually happens to be a Big Bang Theory joke.
A Big Bang Theory joke where they sat together, and they said, let's swear to this day that we will come back to this exact same moment if we do invent a time machine.
And then they wrote this letter and swore on it, and they waiter for one second and no one came.
And they were really sad
So that's another version of the same?
So now I totally get this.
Let's think about other creative ways to--
I don't
You don't.
I still don't get it.
I'm sorry, but I still don't get it.
I don't understand the concept that he sent invitations after the party and no one showed up?
So someone else explain it in a different way than has already been explained
So it's like if I were to swear to myself right now that in the future, if I do invent a time machine, I will come back to the time when I'm writing this right now.
So theoretically speaking, that would mean that as I'm writing this, my future self will come right now
But actually, now that you say that, that brings up a really interesting point.
Your version just proves that you're not going to invent a time machine.
But Kenneth's version proves that no one--
All the time travelers possible that he sent would not be able to--
Sorry.
So Kenneth's version proves that everyone that Stephen Hawking sends an invite to will not invent a time machine.
It doesn't necessarily prove that no one will ever invent a time machine.
It's just an interesting point
There was also a time travel [INAUDIBLE] in the '90s on East Campus
So this happened?
At MIT.
So that's just another thing that you should note.
Other people have tried to do it, and that's [INAUDIBLE].
But also, it's nicer that MIT students [INAUDIBLE].
[LAUGHTER]
That's a good example, but we're still missing what the big-- we're coming up with examples and accessories when we don't have the core point of the video.
And I think I really feel like taking the last question, completely scrapping it, and trying to figure out how to best set up the rest of the episode is what's going to help you the most.
When you explained all that stuff to me, I Totally read your intro completely different.
And the question that I came up with in my head before you asked your question-- so you read through the whole thing.
You said, or maybe Stephen Hawking was too lame-- my initial question as an audience member was, OK, well then how did he prove that?
Like how did that party prove it?
And if it did prove it, why are we still asking ourselves if time travel is possible?
I guess that's how I initially reacted, and that sets up a very, very different episode than the question that's up there right now
I also want to give-- so this kind of discussion is exactly the kind of discussion that happens with every script that we do for Science Out Loud.
And I know it seems like we're piling on, but I really want to give you props for taking a chance and doing this sort of creative intro, because we're going with this, I think is really interesting.
So your two questions at the end are-- what were they again?
It was?
It was how did this actually prove it?
And if it's proven, why are we still asking ourselves?
Yeah.
So if we could have an infinite party, where we send out infinity invitations to everybody on the planet in the future, would that prove-- and then no one shows up at this party-- would that prove that time travel is impossible?
So let me back up.
I'm going from the Big Bang example, which is-- or no, actually your example.
Which is you proving to yourself.
You say, all right, if I invent a time machine then I swear to myself I'm going to come back to this moment.
I don't come back, so I didn't invent a time machine.
Then to Kenneth's version-- Stephen Hawking sends an invitation to let's say 20 people.
No one shows up?
That means those 20 people didn't come back in time
Actually, it was more like an open invitation to the future.
It's [INAUDIBLE].
The fact that he had the party.
So in the future, [INAUDIBLE] time travel anyone who would [INAUDIBLE] know of this party
But conceivably, you could-- I get what you're saying.
I think, as a thought experiment-- you're going to miss somebody in your invite, if you send out a finite number of invitations, no matter how broad your spectrum is.
But I mean you could pick something in-between where you send out a million invites, but there are still 7 billion people on the planet.
So then the question is, if you send out if you've sent an invitation to not just everybody who's alive right now, but everybody who will ever be born from this moment forward-- everybody, whatever number of billion or hundreds of billion of people that is.
Theoretically, if someone invents a time machine and decides to go to your party, they would come back in time and go to your party.
So then if you set that story up with the audience-- and that's a long set up.
You've got three separate scenarios-- between one and three, you could do two scenarios.
It's a long set up, but the story element of it means that your audience will probably stick with you, especially because it's an unusual story.
It's like a weird thing, and there's confusion at the beginning.
You're saying, like, hold on, I didn't get it.
Your audience is going to be in that space for the first 30 seconds of the video.
And there's a fine line between I don't get it-- I'm going to turn off this video, and I don't get it-- I'm interested; I want to keep watching.
But if you can stay on the line of I'm interested; I'm going to keep watching, the I don't get it can be an OK thing to do.
And then after that, you get into Elizabeth's question of what does this thought experiment actually mean?
And does it does it prove anything and how do we know, which I think is a fascinating video.
Conceptually, I think it's really hard, the double negative.
To either choose to say is time travel possible or not the opposite, which is time travel not-- it's just harder to conceptualize.
So when you do go down this road, I would just see if you could take the affirmative approach, which is time travel possible
I would wait to ask that question though, because that's actually not the question that's relevant.
The question is how did that prove it?
Then the next question is why are we still looking, and then the immediate question that [INAUDIBLE] is time travel possible?
That sets the core question, but it doesn't show up at-- it's too big to [INAUDIBLE]
This is great.
This is a fascinating discussion.
Good job.
Anything else before we move on here?
So there was a bit of discussion where there was a detail on how many people Stephen Hawking actually sent out an invitation to, but that kind of detail doesn't really affect the knowledge that you're trying to disseminate.
Is it OK if we were to-- because it's not really an official history point, so I'm not sure whether I'll be allowed to deviate from it
That's a great point.
In writing your scripts, you will generate more material than you'll use.
And probably going through-- I had the three scenarios.
Probably the middle scenario is just not worth doing, because it takes up a lot of time, there are a lot of subtleties to it-- how many invitations exactly, who gets the invitations, et cetera, et cetera that is not important for the subject matter of the video.
So being able to edit yourself or edit others on that stuff is really good.
So yeah, absolutely, great point
You can also gloss over it pretty fast.
You can be like, OK, so when you're explaining the actual details of the party, you say he sent out an open invitation, no one came, so the conclusion was that no one invented a time machine.
Well I guess you could say maybe someone in Antarctica wasn't aware of the open invitation so that's why they didn't come.
So what if we could ensure that every single human being on will ever exist in the future somehow got this invitation?
Then will the argument still hold up?
Then you're glossing over each incremental example and addressing the crux of the loophole that's in that scenario
Also, I wanted to mention humor.
So saying that Hawking was too lame-- while the other humorous parts are very effective, that one may be kind of edgy, especially for people who don't know who Hawking is.
Maybe like, uh-oh, this person is lame, and people who do know him will--
So that reminds me of your earlier point-- which is that this actually happened at MIT.
[INAUDIBLE] time [INAUDIBLE] going.
And one of the things that we try and do with the Science Out Loud videos is establish a relevant MIT connection.
And so that would be a great way to do that, and then you get around the whole Stephen Hawking thing
I like the humor point, though
I do too
The word lame maybe is not as
Yeah, I completely forgot
But the whole concept of what other possibility--
But maybe Stephen Hawking's rival just didn't want to show up to spite him.
Maybe he's like, well, I've invented a time machine, but I'm not going to let you know about it, because I'm too cool for you or whatever.
There are ways to get that exact same humor thing without using lame.
But I like it.
I like it too.
Anything else?
Cool.
Whose is this?
All right, come on up
So as long as people have lived on this planet, there have been boats.
Many of the model boats shown in this museum-- this is at the ship museum at MIT-- are over 100 years old, but evidence suggests that the oldest ones date back to log boats, almost 10,000 years ago.
But now, when we look onto the hardware, it is clear that we have advanced since the times of hollowed trees.
Today, there are all types of ships, such as container ships, oil tankers, and cruise ships, among others
Great, thanks.
OK, what works?
Actually, we're not going to-- just comments in general?
Good, bad, whatever.
Yes?
It feels a lot like a documentary
What do you mean by that?
The image I get walking through a museum and look around us-- I don't know.
That's the feel I get
[INAUDIBLE]
Uh, what's you're question?
So what three was-- it kind of gets to that.
So I think you're right.
There is a lot of words before I get to what I'm presenting
Tell us what it is
So it's about subdivision in ships.
So if you take something that floats, like a shoebox, and you put a hole in it, it'll sink.
But if you divide that shoebox into watertight sections, this one compartment might not cause it to sink.
And how we've been asked to-- the time where a ship can be extensively damaged and can still stay afloat and people won't die
What you just did right there is a fantastic pitch for a video.
Like the way that you explained it to us right now.
You get into the subject matter quickly.
Use you use something that we're all familiar with, shoebox.
You start simple and you expand out.
You start with a shoebox, expand out to a ship.
And you have the potential for an example that is super relatable, like that Italian cruise-- was it an Italian cruise ship that didn't sink, but it turned sideways-- you could use something like that.
I'm sure there's a more appropriate example.
But you just add that, and you have the perfect pitch and also a great introduction to a video too
I also really liked what he says in the second part where he says he's a coast guard marine inspector and naval architect, a ship designer, and he [INAUDIBLE] sleep over the subject.
So that to me sounds pretty cool.
First of all, those are awesome jobs, and also, you are saying that you have an emotional connection.
You're an expert in [INAUDIBLE]
And the other thing is that we don't-- Science Out Loud doesn't put random people on camera.
Whoever we put on camera is going to have some connection to the material, whether it's this is my lab that we're working in, or I am a member of the Coast Guard or whatever it is.
And highlighting that in a way that is not newscaster-y.
That's hard to do, but highlighting that can really draw the audience into the material
A lot better than poached eggs
[INAUDIBLE]
So honestly, if I were you, I would cut the intro and go with what you just said.
The other nice thing about what you said is that you can do it with a shoe box in front-- it's a demo you can do.
So it has a visual that goes along with it really easily
And since you're actually doing that, there's potential for a really big visual reveal, where we have you frame with the shoebox, and then cut you to wide shot, and reveal you in front of an actual ship.
So there's a lot of potential to play with the visuals there.
Where you don't have to say all the boats here-- we don't even have to go in the museum, we can actually go to wherever you work.
But I agree with George that-- when you were talking about the shoebox stuff, I didn't even realize that that was a question that I had, but then you made me think, just like how David made me think about the whole why do some people feel colder than others.
How does a giant ship not sink if there's a little hole in it?
Whereas on a toy boat, if there's a hole, then it's just catastrophe
Another question I have about ships is that-- you see a giant cruise liner, and a, what percentage of it is underwater?
Is it 10% or is it 50%?
And b, how the hell does that thing not just tip over?
We actually just asked that yesterday.
So you're wondering?
Yeah there is a lot of draft on it, so the distance below the waterline is substantial, so it's not uncommon to be 40 to 60 feet under the water.
But they fin stabilizers which they use when a lot of people are on board.
They're just pretty much wings that go on the side of a ship.
And with that, they're able to adjust their attitude to provide a more comfortable draft
Are they like wings like as big as a airplane wing?
Or is it shorter?
They're a little bit shorter, and they're kind of stubbier, but they have pretty good--
So if you took a cruise ship out of the water, it would look like a deformed penguin kind of?
I guess you could think like [INAUDIBLE].
Most have-- newer ones all have them
This is again that idea explosion phase, where you're talking about compartmentalization, and that's the core of the video.
But there may be other things that-- I don't know if this would actually make it into your video or not, but that is something that you wonder when you look at a cruise ship.
Like how does that thing possibly stay afloat?
Is most of the mass underwater?
I can't really say quantitatively.
I'm not sure, but I know that that's one of the big parts of it, is having appendages that are able to reduce the roll.
So if you have a cylinder, and it's water, and you're trying to pitch it or just have it roll, it's going to go forever, but if you put these two wings on, the inertia of actually rolling it is a lot harder, because it has these arms sticking out
I think you've actually got material for two separate videos in there.
I think the compartmentalization is one really-- it could be a short video.
It doesn't have to be long.
And then there's this whole other-- boats, and ship building, and how do these things, how do these giant-- because they've gotten big enough now that you can have 2,000 or 3,000 people on a ship.
How do these things stay afloat?
It'd be really cool, I think
From an educational context point of view.
With the K-12 videos program in its old iteration, where people made their own episodes, there were like 10 episodes on buoyancy.
I don't know why people are so obsessed with explaining it, but I think it's because it's a topic that maybe people struggled with in school
No, it's a framework test.
So, second graders-- it's the essential question that is asked in the curriculum
Buoyancy?
Yes, sink or float.
What sinks, what floats, and why?
And it's a questions that's intended to get them thinking scientifically.
And so that big buoyancy question is-- when I was learning to be a teacher, that was the core question, even in schools that had no science curriculum whatsoever.
At that age, that's when they begin asking scientific questions, and that experiment is something you need to do, no matter where you are.
You have a small bucket.
Any kid can do it, regardless of what class.
I taught in some inner city schools where there no resources, but you can still do that experiment
Do they figure out at that age-- do they why a sinkable object like steel would float, or something?
Well the idea is that students can construct themselves with themselves by experiment.
So the teacher, if it's a great teacher, will put a bucket of water and a whole bunch of objects.
And encourage students to eventually come up with some truths themselves about what sinks and what floats through hands-on experimentation.
And that's sort the crux of the most early science exploration in our curriculum, which is why there's so much on sink and float, and that's why the kids were all-- they all were interested in that when you talk to them, because it was drilled into them, at least in our educational system, the idea of sinking and floating and a very elementary concept that is the beginning of their scientific roots
But the reason why I bring it up is that there are 100 videos on sinking and floating, and they're all the exact same thing.
And every lesson on buoyancy-- you learn it in more detail in high school.
It's still on the same thing.
And this is presenting it in a way that's very, very different.
And I hesitated to bring it up at all, because I don't want you to look at those videos, and I don't want you to look at the standards and accidentally default to that style.
I think you should totally keep going the way you're going.
I only wanted to bring it up, because I think that there is definitely an appetite for it in an educational context.
I also wanted to bring up that-- it's funny-- I wish we could have the raw footage from all the classes just to [INAUDIBLE], because when you explain the technical components of the wings and stuff-- we could've used that footage.
And it's very different from when you read this aloud.
And the script also doesn't sound like you at all, when you talk.
And it's a very difficult thing to do.
I sympathise with you guys completely.
It's really hard to become self-aware of your natural talking style.
And so some of it will be just-- maybe you want to record a conversation that you have with someone else.
That might not be the best thing for everyone
One of the things you could do is instead of recording yourself on camera, record just audio.
Because the visual part of it-- like if you're conscious that there's a lens looking at you, it's very hard to be natural
Yeah, and I'm not very good in front of a lens or anything
Getting rid of that misconception is a whole other thing.
We'll get into it in a second, but it's much easier to just be yourself if you just have a mic, and you're not really-- you sort of forget that the mic is there.
You're not looking into a lens or whatever.
So having that, it's easier to record yourself and hear what you really sound like if you just use audio, versus video as well
So say you're trying to simplify something, a complicated concept, so you're talking to [INAUDIBLE], and there's a lot of different technical reasons that you have these fin stabilizers.
So just being able to say that this is stopping the ship from rolling, it's true, but there's other aspects to it.
So this gets into the accuracy portion.
Like you could have a naval architect go on K through 12 and be like, yeah, but there's more to it than that.
So at what point are you oversimplifying
I think there's no harm in alluding to the fact that there are that you are just at the tip of the iceberg of information.
So you could say something like-- the way you said it was if you take a cylinder, put it in water, and spin it, it'll spin.
You take a cylinder, attach some wings, put it in water-- it's much harder to spin it.
That is all 100% accurate and true, but you don't get into any of the technical reasons as to why that's the case.
That's one way to do it.
The other way to do it is if you want to start talking about whys.
And I don't know what the whys are, so you have to help me, but you can start off your senses with-- look, this is a really simple way of explaining it, but basically, blah, blah, blah, blah, blah, blah, blah.
So you just sort of allude to the fact that I'm giving you the super simple version of it.
It's much more complicated than this in real life.
And you can even explain what those complications are.
So if you had wings with a bunch of holes in them, they wouldn't be as effective as solid wings, because x, y, or z.
Or if your ship's wings need to be able to do this and that for whatever reason-- you can allude to stuff without actually explaining it, and that's OK, because you do that in real life.
If you're sitting across the table from someone and explaining something to them, the first time you do it, you're not going to be like-- roman numeral I, part a, subsection b-- you're not going to go super in-depth.
You're going to be like look, here's the important thing to get right now.
Here are some of the details.
We can about the details later, but this is the crucial thing
So more of the result and not everything that got to that result?
I don't know that I would--
Or if it's a little too tricky, then mention it, but focus on what the main reason why they're there
Yeah.
I think just being conscious of the fact that what you're presenting isn't all of the information, it is an accurate portion of the information
And that's actually what we want with Science Out Loud.
And keep in mind, it's a very different style than some of the other videos that i was talking about on the first day of class.
But we want to pique curiosity.
We want people to finish watching the video and ask themselves those questions and try to figure it out on their own and explore the other resources.
And that's a very different objective than say a lecture video or a technical video.
So just keep that in mind.
Again, whatever best practices we're establishing here aren't necessarily ones that you'll carry over to every single educational video.
We encompass such a huge variety, right?
But I think that's true of even the most technical video.
And again, this goes back to the whole rabbit hole witness test.
Once you find yourself clarifying more and more to the point where you found yourself on a topic that is tangential to the main point of your video or the main point of your lesson, then it's time to backtrack a little bit, and that's OK. No video under an hour.
No semester-long class is going to answer every single question that's related to the topic of the class
But this gets back to what we were talking about the [INAUDIBLE].
By going deeply and narrowing your focus actually allows you to allude to a lot of much larger topics superficially.
And people get it more quickly the more narrow you are in your examples.
And think if there were a truth in here, that's the truth to sort of apply-- is that the more specific and narrow you can be without getting [INAUDIBLE] in that extra information, the easier it is to allude to that extra information without it-- like a story flow, you want it always more forward.
And if this information is so little that it doesn't prevent the forward motion of your storyline, then you know you're doing a good job of including that.
What you don't want is for this information to take you backwards or sideways
I mean I guess I should rephrase that it's not necessarily a rabbit hole.
You don't want to branch out too much.
You want accuracy, specificity to a certain degree.
You could very well do a video on why do boats float, but that wouldn't really be a good video.
It wouldn't be a very compelling video.
So you're going to make a video about the wings on a boat.
You're going to hit the bigger topic of how things float, but that's not the thing that's driving your video forward
And that's why your topic is perfectly narrow of imagining three differently smooshed shoeboxes.
Why does this one versus this one-- and it's a narrow enough topic, and concrete enough that it will allow you to get to some of those bigger sink or float buoyancy design issues.
But using this very small way of getting access to that
By the way, I wanted to come back to something that is a tad tangential, ironically.
You said, maybe it's better then to just tell the results and leave out the process.
And I said no, maybe not.
The reason that I said no is because sometimes, taking your audience through the experimentation that was done to get to a result is just as interesting or even more interesting as the result itself.
So the Stephen Hawking example-- the time travel experiment and the intricacies of the time travel experiment are just as interesting, to me, anyway, as the result of that experiment, whatever it is.
So that's why I would just caution against using that.
Sometimes-- how did people first invent compartmentalizing boats?
I'm not saying that is an interesting answer.
It may not be.
But sometimes that process is just as interesting as the fact that there are compartmentalized boats and why they work
One thing that I mentioned-- I don't know if it's worth exploring is the whole reason why [INAUDIBLE] was like I know that floating is obviously a big teaching point, but if you open with why the Titanic sank, which has exactly to do with compartmentalization, is that something that would be a better hook?
I honestly like your shoebox better.
I'll tell you why-- because it's more visual.
Sorry, it's more visual because it has you in it.
There's a demo.
There's a thing that someone can look at, and it's directly accessible.
With the Titanic, yeah, it's an example that everyone knows, and you could maybe use stock footage of the movie Titanic or whatever.
But it's one degree of complexity-- it starts at a higher level of complexity than your shoebox example
Well I'm saying you could do--
First shoebox, then Titanic?
No, how the Titanic did their compartmentalization with a shoebox.
So they didn't finish their [INAUDIBLE] to the deck, which you could actually easily build with a shoebox, and then just see how it progressively floats
So the problem with the Titanic was that their compartments did not extend all the way through the hull?
All the way up the [INAUDIBLE]
Why?
That seems dumb
I have no idea
But then the video becomes about why did the Titanic sink, instead of about how do I prevent boats from sinking as a Coast Guard
I think it's distracting.
I think it's distracting, and I think that the simplicity of this shoebox allows you to gain access to the concepts more easily, without getting the story of the Titanic entangled into it
The other thing is, if you are not at MIT studying naval architecture or whatever, and if you weren't actually an engineer, just maybe Vsauce or hand Hank Green, then I'd say, yeah, talk about why the Titanic sunk.
But it's a story that's already been told.
And it's a story that can be told by anyone.
You have this opportunity to tell us a story from a perspective that not many very people can, and that's what I was trying to get out on the first day about being able to take yourself out of the perspective that you're used to having.
I think that is an opportunity to capitalize upon, for sure.
So it's not that I think that the Titanic thing is a bad hook, necessarily-- just that the one you have is so much better.
And one that not other people can do
Can I also say that you had mentioned about process.
In my lab, the work that I see predominately-- I would actually argue that it's extremely hard for scientists to share a process in an interesting way.
That scientists, typically-- and I am generalizing, and this is not really fair, but because we spend so much time day-to-day going through a process, you want to share it chronologically, and that's where your default is to go from start to finish, telling people what you did in painstaking detail, and that's boring.
And so that's really, really hard-- if any of you end up doing a process where you're sharing your process, it's really hard to figure out the bigger picture within the process so that an outsider actually finds your process interesting, as opposed to just sharing the I did this, then I did this, then I did this.
Explaining why I did what I did each way is what people care about
Well the interesting part is the moment of ingenuity that led to an experiment designed the way it was designed.
Not the day-to-day--
Exactly, and I feel like because people in their labs are like someone needs to listen to me.
I did this.
It didn't work, and I spent hours and hours working on this and it didn't work.
They feel like they want to share that with people, but people don't actually care about that.
They care about the why you designed the experiment that way, and why it didn't work, as opposed to the what you actually did, the who cares kind of concept
Actually, I agree with you on the Titanic idea, but you mentioned that you could build these compartments out of shoeboxes, and I think that at least the kids that we talked to yesterday-- they really like the idea of destruction.
Something burning, being crushed.
So if you could kind of talk about those concepts and in the end, you sink some random ship.
That would kind of show, first of all, that it's cool, but also it shows that what you;re talking about is important
Don't get me wrong.
You can totally bring up the Titanic at the end as the bigger picture application.
That's awesome.
It's just the hook isn't the place for it, I think personally
It's been done before.
I agree with you
Great.
Anything else on this script?
Going once, going twice--
Before you go onto the next one, I do just want to mention-- sorry
I'm just kidding, go ahead
I do want to mention that you guys came up with really fascinating topics.
I mean George and I were talking about it this morning.
So I mean even though we're critiquing things, the topic at the core, I think there's a lot of potential for it.
So just a word of encouragement
Yeah, I would totally watch all of these videos
I don't know where we're going quite yet, but we've been going for a while.
I suspect people might need a bathroom break
Oh yeah, good idea
Yeah, I need a bathroom break
So maybe we should, before we dive into the next one give people a chance to stretch and do that
Yeah, OK, who wrote this?
I'm cold
I'm warm
I'm pretty warm, yeah
I think it's where you're sitting partly.
The first day I was under a vent and it was freezing
OK, whenever you're ready
OK. Few things strike more dread than hearing you have to get braces.
Whether it's worrying about how much it will hurt, what happens if a wire pops out, how you're going to talk, or how much your friends will make fun of you, it's kind of an awkward experience.
So why even get braces in the first place?
Cut to pictures of famous stars who have had braces with a voiceover.
Should I keep--
Yep, just the next sentence
Studies have shown that having a beautiful smile with even white teeth conveys health and youth, and makes people like you even a little bit more
Cool, thanks.
Who were the famous stars that you are thinking of here?
[INTERPOSING VOICES]
But this video's about you
Well the way that I structured it is to have as little of me in it as possible, which is why I had so much of the [INAUDIBLE].
There would only be one or two choices
Just out of curiosity, why?
Why did you structure it that way?
I like the animation aspect.
I also know I don't look good on camera
No one knows that they don't look at my camera
I feel like this is the fear that every person comes into a Science Out Loud episode with is like, we want to animate this, because I don't want to explain it on a camera
Or I can't say this long of a paragraph without stumbling
Or we'll take their script away, and they'll be like I need to have it.
I can't do it.
That is so not true.
You can't actually prove that you can't do it.
And we want your faces to be on camera, and we can talk about this more, but I think that if you do too much of the pictures and the voiceovers and the animations, then we go back to that Siri litmus test.
Or if we swap you out for anyone else to do this video, it's not going to make a big difference.
And I don't know if you're OK with that, but I certainly feel like you have compelling enough of a perspective.
Didn't you say you used to work for an orthodontics company?
Yeah, that is a really unique perspective, both from the industry-- and my mom's a dentist.
So I used to work at her office, and I had braces twice.
And I know for sure that there's so much fascinating information about that that you would know from that perspective that most people don't.
You can make a video that a lot of people can't.
And I would hesitate to let you go on thinking that your video should be that way
And that's not to say that like it's just going to be you on camera or any of you.
That's not to say the entire video is going to be from here to here, 100% of the time, talking.
We know that that doesn't make for compelling video.
You want to have the person doing something, interacting with their environment, being in an interesting place, whatever.
There are lots of ways to compliment the fact that-- let me back up a second.
The human-to-human-- ideally, you want to transmit information.
One person sits down and talks to another, that's like the idealized way to engage someone, convey information, talk about an interesting subject, whatever it is.
Obviously, video gives you the power to do that to a wide audience at once, and it's remote, but there is something about seeing someone's face, and expressions, and intonations, and something that really draws you into the material.
But that having been said, you don't want to just stay on that person's face for the whole video and not show anything else.
But that's our job, as producers and directors, is to figure out where it makes sense to just do a simple head shot of someone talking versus where it makes sense to cut away for an animation.
If you're talking about like teeth moving, that might be a perfect place to do it.
Versus where to do a demo or whatever it is to keep visual variety and interest throughout the video.
But yes?
That was a tangent-- an important one
No, I think it's actually really important
Well, it's also a segue into [INAUDIBLE]
Yeah.
I have all these graduate fellows who work for me.
We do workshops all over campus and our biggest credibility, and why we can give our workshops,and why people care, and listen to them is because we give workshops on how to succeed with the National Science Foundation awards from people who have actually earned them.
We're authentic.
I did this.
I'm going to tell you what I did to be successful.
That's what you have that's so powerful.
I'm a student.
I'm studying this with this really great institute, and I'm going to share with you how I think about something.
That's like the most authentic you can be, is being you.
It would be a shame to go to someone who's inauthentic, like someone we don't know like a movie star, when you have this incredible credibility by just being you.
That's so powerful

Sorry, you guys are so cool
No, you're absolutely right.
That is correct
But now it's about how to structure the script in which you can feel most comfortable being that
Yeah There was something I wanted to say about this, but I forgot.
Maybe it'll come back to me.
Anyway, comments on the script before we get into the hosting side?
I like the first paragraph especially
What do you like about it?
I felt that whe she was saying it's as if-- I imagine her saying this to a bunch of girls in front with braces.
Telling them a story about her life and saying, oh, I used to have braces, and I didn't like it.
So why do you want to get braces?
[INAUDIBLE] as to why it's important or why not
Yeah, she gives very specific examples about realistic fears that people will have about braces, and that makes it relatable.
It's going to hurt.
You know you've got these like metal wires in your mouth.
What happens if they poke you in the gum?
How you going to talk with all this stuff in your face?
People are going to make fun of you.
Those are actual, real fears that you get when you've had braces.
So, pointing them out like that is effective.
What else?
Just to check, is the content of the question or the video why you get braces or is it [INAUDIBLE]?
The actual video itself is about what happens when the braces move your teeth on a cellular level
What happens [INAUDIBLE]?
As the force is applied to your tooth, there are two different types of cells, one of which actually dissolves some of the bone in your jaw to make a little bit more room, and another one that actually builds the bone to be able to hold the bone in place
Wow.
I had no idea.
That's cool
And that's like the shoebox thing
That's the reaction you want from an audience.
What I just did-- I wasn't faking it.
That's an authentic reaction.
That's the reaction that you want
Yeah, I mean later on in the script I say it's actually dissolving your jaw
I had a question-- so the braces are only attached to your teeth, right?
So normally when you want to move something-- like I want to move this table-- my foot's on the floor, and I push against my foot to pull the table, whereas with braces, I'm not holding some external object to pull teeth in a certain direction, so how is that by--
Headgear
Huh?
Headgear
Headgear, yeah.
But most people don't-- I don't know.
Correct me if I'm wrong, but I think people-- when you get braces, usually it's just the metal and the wire on your tooth.
But there's no external support there.
So how does it move your teeth?
So that's a physics question, and we went over this yesterday.
Because there are different aspects.
There's the sort of what's happening on the cellular level, which is one approach.
You can also do it from the material science-- basically what moves your teeth the wire itself and its shape that's constantly applying pressure
Is that why you get your wire changed?
Yes
I mean imagine being one person who has two boxes on either side of them, and they want to bring them closer together.
The external force is the person itself pulling them together
Yeah, but the tension in the wire-- that's the--
Right
[INAUDIBLE] when I had braces they had my molars-- I don't know if they're using that now to the points or not, but you had the metal-- [INTERPOSING VOICES]
They put a whole band around your-- they don't do that anymore
They don't-- back in my day.
[LAUGHTER]
They have much better adhesives now, so that was another-- the chemistry of the epoxy that actually-- I mean there's a lot of science that goes into your braces
And when you think about it-- I would have this discussion with my mother all the time, because I had to have braces twice.
And I was like, mom.
Braces, when you think about it, they're like barbaric.
I can't believe we're living in the 21st century, and I pay money for a person to glue metal to my mouth and take wires to yank my teeth together because I have too much gap in-between.
That's so barbaric when you think about it, that your jaw is dissolving.
But barbaric in a way that's fascinating to watch on video, and I feel like that's such a more high-stakes drama-driven hook than you're worrying if braces will hurt or how you'll look
It's interesting too, because the bottom question gets at the anthropological question of what is beauty.
But really, I think it's much more.
If we're going to get back to the evolutionary question of what makes someone cold versus hot, and will they die soon or not-- if all of us has evolved to the point where 80% of tweens end up getting braces, what's wrong with our society, and why do we have to keep fixing these teeth?
Is it actually going to make us die sooner if you don't fix your teeth, or is it just because we look prettier?
Is it cosmetic?
Yeah.
I know that I will at some point have to deal with some more significant dental issues that are actually about my functioning.
But proportion of this is actually-- I don't know.
And I don't know if that's a question you're even going to get into, but why do so many teenagers end up getting braces
So that and Elizabeth's point are related.
When she was describing the barbarity of you get these metal blocks and a wire in your mouth and all sorts of stuff-- what technique is that?
What is that?
It's not a metaphor.
What is she doing there?
Yeah, so she's inducing emotional pain by asking you to put yourself in her shoes and getting this.
But specifically when she says, OK, you get braces.
You're going to go and have someone-- you're going to pay them to glue metal stuff to every one of your teeth, then take a wire that is not in the shape of your mouth, bend it so that it fits your mouth, and then it forces your teeth to move around.
Like what is that that she's doing?
Who here has seen Robin Williams' comedy sketch on golf?
Anyone?
Oh, you have?
It has a lot of bad language in it, but it is hilarious.
And I can't do it here for that reason, but you should watch it tonight.
He basically says, look, golf-- you take a stick with a really tiny metal thing on the end of it, and you have a small ball.
And the goal is, without touching the ball, to beat it with the stick until you get close enough to a tiny hole in the ground that you can take another stick and then gently tap the ball into the hole, not once, but 18 times.
That's the same technique as Elizabeth's using, what is that?
Why is that interesting?
Why does that make us listen?
Yeah?
Well it's was humorous, because you get used to the jargon and different terms, but it's the sort of thing if you're trying to describe it to someone who has no clue what you're talking about.
it just sounds so utterly ridiculous
Exactly.
So the humor, actually, I'd argue is a secondary component.
Really, the crux of it is what you said about being used to something.
It is taking something that's familiar and describing it from a completely different vantage point, and you can only do that with things that are familiar enough that people are so used to them that they just don't think about it-- braces, oh yeah.
Braces, whatever.
But when you shift that perspective, you do inject humor.
You can inject drama.
You can inject-- mainly humor and drama.
Humor and drama.
And you force people to sort of step out of their comfort zone and consider something that-- they just look at it in a totally different way, which is an alternative opening.
So that's another thing that you could consider.
If I were to tell you-- and the way to do it would be to do it as a reveal.
If I were to tell you that you're going to go to someone's place of business, pay them money to fill your mouth with a bunch of metal and plastic and chemicals, would you do it?
And then like, no.
But people do that every day.
It's called getting braces.
The way I just did it was not very good, but you get the general idea.
Other comments?
I also liked-- I was actually with Andrea when she was pitching her idea to sixth graders.
And her first question was do you have braces?
And this one girl who did-- she was really excited about that.
Like yes, this is about me.
And then the next question was, well, do you know what they do?
And she started pitching all these ideas.
This is what I think they do, but then the answer was very different
Sorry-- the girl thought they did something completely different than they actually did?
What did she think they did?
She just kind of said pushing teeth, whatever-- just a really basic answer.
But then Andrea was saying there's so much they actually do.
And I thought that was a really good way to get into it, because they got really excited, and they started talking about their experience with braces.
So here it's kind of-- if you say maybe instead of whether it's worrying, you could say do you worry about how much it will hurt?
All these things.
Get the audience thinking maybe offering their own ideas, but also it connects them to-- this is something that I have
Yeah.
So that approach-- generally, I would call it the preconceived notion approach.
So you basically guess or know what your audience thinks about something, and usually that is either entirely wrong, or parts of it are wrong, or not complete, or whatever.
And you say you start the video by saying you probably think that x, but actually y.
Or z or whatever.
And so that is another-- it's very different than the fill your mouth with a bunch of metal and wires and stuff, but it can be just as engaging and interesting, depending on what the subject matter is.
The only difference-- and this is where you've got to be careful-- is that when you're actually in a room interacting with people, going back and forth with the Socratic method is great, and it is a perfect way to get into the topic, but you can't do that with video.
And so you what you don't want to do is make that two dimensional conversation one dimension by sort of just stripping the away the kids' answers and just putting in the questions.
It's kind of like trying to fit a square peg into a round hole, or the other way around in the sense that it's designed for one medium, and you're trying to use it in another.
[?
SEAM ?]
And I know that's not a great way of explaining why it doesn't fully work, and I don't necessarily have a better way of explaining
Isn't that why Dora and why Blues Clues are held the way that they are, where it's like a rhetorical question, but they leave space for the kids to-- my son watches and answers the television
And that's the educational best practices for educational videos that I was talking about on the first day-- that you acknowledge your audience, that you give room for interaction.
I will say, going out of your comment, and sort of addressing the kids where they are-- don't forget about George's point of talking for the audience, not to them.
Because the overall point I would say about the wording and the set-up of this opening is that a sixth grade kid would be interested in this, but I'm not sure that their parent would necessarily be relate.
And maybe it's just the way the questions are posed, or the types of questions that are asked, but there's a risk of falling into this tone of like after-school special.
I don't know.
George, do you have tangible feedback that you can give about that?
I can't identify exactly what it is
Let's see
Maybe the question of why even get braces-- how that follows this list of examples of how much it'll hurt and depending on the delivery, it can really change how this reads.
So why even get braces leads me to think that this is the type of video that they'll show at a dentist's office
I feel maybe what you're getting back to is the-- this is the first example that we've seen where she's directly talking to you, whereas the other ones, it's more of either the voice is different in all the other ones.
This is the first one where she's talking directly to the audience, and it's abstract.
You were using yourself to say, this is what I do.
This is a boat, and this is back to me, my voice.
And you were talking about time travel and Stephen Hawking as this third voice.
But this is the first time we've seen sort of an abstract you in there that makes it, I think, a little fuzzy in that way
And I'm not necessarily sure that it would help to just switch out the you with a we, because it is a we, but it's also sort of a royal-feeling we rather than a-- we're going to do this.
Royal, by the way, I don't want to-- I realize I'm lacing this with jargon, but royal we means basically just I.
You say we, but you mean I.
So I don't know.
I think this is a really interesting problem.
Does anyone have thoughts on how to reward without going the sort of dramatic dentist route?
Because I think this is interesting.
Yeah?
We mentioned the personal aspect of it.
If you had braces, and you suffered from this and that kind of [INAUDIBLE].
So we can say how [INAUDIBLE], because I experienced it just as you might be experiencing it right now
I kind of think I'd be even better if-- I know you don't have braces now, but I would almost want the host of the video to either actually have braces or I don't know if it's even possible to like put fake braces in your mouth.
Like temporary braces that you could just have in there for a day and to just start with something like--
They did it on Ugly Betty for all their seasons
Did they really?
Yeah, they did
I thought she actually had braces.
I don't know.
Anyway.
But to start with something like see the metal stuff in my teeth?
This is called braces.
And you have this-- it's a piece of metal.
It's glued to my teeth.
This wire is bent, and it's forcing my teeth around.
So you're kind of doing this sort of dramatic dentist approach, but not as storified.
And you are sort of sideways hitting these points.
Like I've got all the stuff in my mouth-- is it going to hurt?
What happens if this wire pops out?
Clearly I have a bit of trouble talking or whatever the issues are, but you do it in a way that is-- because you're pointing at something, because it's kind of a demo, it's more conversational, more relatable.
i don't know.
That's my thought on it.
What do you guys think?
I was thinking maybe-- you know how infomercials always ask you have you ever had this problem, and then they go into the details.
So maybe start off with have you ever had braces [INTERPOSING VOICES]
How much it hurts when [INAUDIBLE].
So it relates back to them
Were the braces your second time having them or your first-- it was your first time
You could show crazy teeth
David Bowie's teeth?
Any English person.
If think maybe we've all seen the bubba teeth enough times that we know what they are when we see them.
But maybe many Europe creators haven't seen then, and you could start talking with this horrible mouth of teeth and then pull them out
I feel like this backing out a lot from what we were talking about.
But my big question is why we even have the first section, because to my understanding, the video is about how braces work, but in this first section.
It's more about why would you get them, but I kind of get the feeling that the video isn't really about why you get them, it's just how they do that.
I don't know
Going off that, I was trying to do figure out what my main concerns with this were, and I didn't have a mental grasp on it until you mentioned [INAUDIBLE].
Because I haven't had braces.
I don't plan on getting braces, so I felt really, alienated by the you in this, because I would probably, realistically, turn this video off because I'm not interested in why people get graces, because that's not relevant to my life.
But if you posed the question have you ever thought about exactly what braces do, that's a much more interesting to me and to probably a broader audience.
And [INAUDIBLE] your tone with whatever intro you decide to [INAUDIBLE], because that seems to be what your video's going to be about, especially if you phrase it like braces are much more scientific than we think they are.
They're not just magically mushing your teeth into the correct place-- there's actually a lot of things going on in your mouth that people don't think about.
And thousands of people have braces every day
And the other thing is I don't actually think that the point of your video should be how braces work, beside that's an instructional video that Invisalign could make and has made, so many times.
I've seen them so many times.
There's no spark or wonder or curiosity in that whatsoever.
And I'm not saying that every video has to be Neil deGrasse Tyson contemplating on the philosophies of our existence or whatever.
But when you think about it, you can be born with bubba teeth, and we have the ability to know how to craft bones in our bodies-- your teeth are the only exposed bone in your entire body.
And we have figured out a way to alter that, in such a way that we can still continue to use them.
So it's not like you break a bone and try to align a crooked finger or something.
We've loop-holed around bubba teeth and the bones that we have been born with, and can actually dissolve the osteoblast or osteoplast or get them to dissolve bone and restructure it.
And we can manipulate, essentially--
I think what you're saying is braces shouldn't work, just by reasonable--
Or not even that they shouldn't work, but we shouldn't be able to do the things that we can do with braces.
Braces shouldn't exist
Well I don't know that they should exist.
The fact that you can basically blunt trauma move teeth around and then your body gets them to stay where you put them is really weird.
That's something that is-- I don't know about if you were to break an arm and what the difference is between a braces and a cast, for example.
That might be another analogy, or simile, or whatever.
But it is bizarre.
When you said the thing about one type of cell dissolves the bone, and then the other one re-bonifies it in the right place.
To me, that's the weirdest and coolest part of the braces thing.
And so incidentally, why does your body even do that?
I mean it's not something that your body came up with in response to braces.
This is an ability that it has
Yeah, this is in your entire skeleton
So what happens in our natural experience where our mouth would need to do that?
Get a tooth knocked out or what?
Why can we do that?
Is it only gums or is it elsewhere in our body too?
It's in your entire body, because you very rarely have two hard substances butting up against each other.
They have usually some covering that's mediating that relationship.
And then if something happens to permanently mean that these two pieces of hard substance now have to be in this orientation as opposed to this one.
How do you get it to stay that way
So this is more about how the body heals itself than it is about braces
Yeah.
The word isn't even heal.
It's more like how your body deals with something it can't change anymore.
Because healing implies that it puts it back the way it was, which is not what it's doing.
It's like we are brute-changing our bodies, and it's adapting to what we're doing.
Did you raise your hand?
Actually, I think that some experiment [INAUDIBLE] basically to make yourself taller, what they do is they break the bones in your legs
Of your [INAUDIBLE]?
To make yourself taller
So maybe for one year, they keep breaking the bone in your leg.
Then they stretch it slightly, like a few millimeters
How many inches can they [INAUDIBLE] out of you that way?
I think it's like an inch or an inch and a half
Is it really worth one inch?
It's really painful
So they keep breaking it, and then you have to [INAUDIBLE] the distance apart [INAUDIBLE]
Sorry.
The point of the video though.
This is going back to Jamie's point of you hone in on something very specific like braces, but the point of the video is not really just this is what happens when you get braces-- it points to this bigger thing that we're actually manipulating natural processes that happen in our bodies that are important.
Like your bones have to break down and remodel all the time.
That is actually an essential process of dissolving old bone and osteoblasts forming new bone.
If that doesn't happen, your body can't exist.
That's actually an essential process, and with braces, you're manipulating that.
You're tricking your body in a way to adapt to these blunt forces that you otherwise wouldn't be able to do
We had no idea when we invented braces that that's what was happening.
This was discovered afterwards?
Or did we know?
And I think taking those-- what you were saying earlier about how can we do this and what you were just saying.
We're manipulating a natural thing.
I think part of what you can do is like-- I was talking to you about how your teeth slip.
You teeth want to go back to its natural state.
They don't want to be straight.
They want to be whatever you're born with.
And how that-- I don't know if it's possible.
You might be able to take that angle-- we're made as how we're made and we're to manipulate this, but in the end, we just want to go back to the [INAUDIBLE].
And how this is process of using braces as kind of-- at first it can seem like it's barbaric, but it's trying to manipulate this process.
I don't know
This is such an interesting challenge, because we were talking about this yesterday-- how Andrea is struggling with the opposite issue that almost all the rest of your scripts are struggling with which is she has the very concrete thing, but she's not sure about the big question is.
That this is the example for that big question.
And we're struggling right now to help her figure out was is that question that allows her to use what she knows about braces as a cool way of illustrating some very cool concept.
And I feel like most of you are actually struggling with the opposite-- you have some cool concept, and you're trying to figure out what's the specific story I'm going to tell to illustrate that.
And so we've actually got like a reverse engineering problem here, which is we don't know what the question is that she wants to ask that allows her to tell this cool story about braces in a way that that example is a perfect example for this larger concept.
And the problem right now is that there are a bunch of different questions we could ask
But it's a good exercise to throw-out all these bigger questions, because that gets you out of the after-school special.
That gets you out of why do we get braces?
You put up with this trauma so that you have a healthy, shiny smile, and then thumbs up and end credits.
You don't want to have a video that's like that
To play devil's advocate
You don't want to have a video!
I agree.
No, no, I agree.
I agree with you.
I agree with you.
But you can play off that expectation in fun and interesting ways, like I would be perfectly happy to start a video as a parody of an infomercial
Like a parody is different
Parody of an infomercial, yes.
Yes, yes, yes.
I mean to do the infomercial and then reveal it's a parody
But you wouldn't actually go out with the intention of the infomercial as your tone.
No, but we'd shoot it and deliver it as an honest-to-God infomercial for the first 20 seconds or whatever, until you establish that you're joking
No more than 20 seconds or else you'll bore the crap out of your audience
Yeah.
And you actually have to go a bit over the top of what the normal tone would be to sort of signal that you're-- yeah.
Yeah?
So when I first heard Andrea say that your jaw dissolves.
That kind of brings up a lot of images.
There are so many movies where there's a focus on teeth destroying mankind.
So there's Jaws, there's Piranhas, there's just random people with teeth walking around, and that's kind of their main feature
Zombie movies
Yeah, and we watched a video about what makes things creepy, and one of the images was just a teddy bear with teeth, and that was horrifying.
So I think the idea-- so it's just making the familiar thing unfamiliar.
So you can even get away from the idea of getting braces.
This is something we do, but rather start with this monstrous thing that we dissolve our jaw.
And that's a really cool image.
And maybe you put it on a teddy bear.
You don't even have to associate it with human kind.
And then at the end, you come back to the familiar.
This is what we do, let the viewer think back and say, oh, wow.
This is me doing really well cool things inside me body
I think figuring out other examples where essentially bone is dissolved and basically listing all of those out-- seeing what the differences are, what the commonalities are.
And then from there, I think you will have a better sense of what the overall story or big question of the video is, whether it's infomercial parody, horror at the dentist, bub teeth, teddy bear with a creepy smile.
Whatever it ends up being, I think you're absolutely right.
The challenge is you have one concrete example.
One really interesting, good, concrete example.
And the question is how to get to it in a way that is authentic and engaging
The body morphing concept is very interesting
Oh, that's the other one.
Breaking your legs to get taller
And because it's a everyday example of something that's actually really bizarre of us physically altering our own image for one reason or another.
And people have experimented with-- in certain countries, big feet are not attractive.
And trying to swaddle their feet in order to protect women's feet from getting too big.
And thinking of what are the ways we have either for beauty or for function found ways to manipulate our physical appearance at a structural or cellular level.
Not just on a going to the plastic surgeon and getting them to fix my nose [INAUDIBLE]
In a weird way, some of the body modifications that we associate with extremism are the easiest.
You get a piercing, you're just driving a nail through your ear, so what, big deal.
Braces, which everybody thinks are like-- everyone gets braces-- are actually this incredibly complicated biochemical process that's happening that's a team effort between the hunk of metal in your teeth and your body to reshape the only exposed bones that you have on you
And it's more than just brute force pulling them together, but you're actually affecting the biochemical interactions that are happening within your body
Maybe start off with that, like say the common misconception of how braces work
But until you know where you're going, that doesn't help.
So I wonder if, since it kind of feels like we're circling, that maybe what we need to do is either have Andrea or all of us come up with a whole couple of our questions that the braces could be the answer to
I think also coming up with a list of other examples that are similar and simple
And then maybe tomorrow we circle back and see if we can better explore this, because I don't know that right now we're going to get there
I mean we have a lot of options that you can toy with.
The point of the conversation is that the point of the video shouldn't be this is how braces work.
It's a tool that you use to reach the bigger point.
And the bigger point has not been established yet, but there are lots of possibilities that hopefully we've been throwing out.
And this thread of body modification is a different thread than the old making the familiar unfamiliar, and I'm not sure that you can address all of them in a single video
No, you can't
But they're all possible options They're all equally viable options.
I would say that on a practical level, how that relates to the script that you've written here-- I mean the studies have shown that having a beautiful smile-- that's a sentence that's explaining why we get braces, but that's not necessarily a point that I think you should eliminate from the video, it's just a point that can come much, much later
I'd actually argue that that's its own video, because the whole-- this is kind of this like philosophical point that raises so many interesting questions like well how important is a smile to the rest of your face.
Is symmetry important?
Does it really actually make people like you more if your teeth are whiter-- so that, for me--
It's almost like a psychology video
It's a whole separate video.
[INTERPOSING VOICES]
Time has done a whole bunch of different-- I think Time Magazine has done a whole series on beauty and the physical nature of beauty and all that.
I think tons of popular magazine have really tried to explore this topic.
I always get nervous when we say studies too, because the critical scientist in all of us is like who?
How legit are they?
And are they at Harvard, or are they at Podunk somewhere and the only person in the universe who thinks this.
So that's just something that I always think about
But just saying that that sentence-- you can neither get rid of it entirely in the open-- I would get rid of it entirely in the opening
I think that if it were me at this point, I would want to actually-- this is what I tell a lot of people who come in my lab for a variety of a different communication tasks-- is to actually minimize this and not not look at it, and see if you can come up with an idea that this fits under, and then come back to this idea and see if you can somehow repurpose it.
But the idea of trying to repurpose this without that bigger concept is not really going to allow your mind to get there
Yeah, actually.
And that leads me to another point, which is everyone who's gone today and almost certainly the people who have not been subjected to this horrifying treatment-- you will all need to completely rewrite your next draft from scratch.
So when we say redraft a script, we're not saying like change a few words here or there.
We're saying start with a blank word document and start from scratch, because otherwise you're going to end up with all the same issues that we talked about
But if you end up loving a phrase that you used, you can do what you call lifting a line, which is to just copy and paste that line once you've got your new structure in place.
And that keeps you from sticking to something that doesn't really work, and trying to step it into something that works a little better.
It keeps your mind open that way
If there's a line that really sticks with you, oftentimes you will just, of your own will, rewrite it.
You don't even need to copy and paste it
We won't get a chance to hit everyone's scripts today, by George can stay until five, and I can stick around as well.
It's about 3:30, so we should probably do a hosting
I have to apologize, because I have to leave every day at 3:30, and so I am sorry.
This is my moment to leave.
But if any of you need to reach me or want to connect with me, email.
I will totally get back to you tonight.
But I am sorry that this always the moment of the day where I have to leave.
So awesome work
This is also a totally normal process.
George, your script was terrible.
No, we went back and forth actually three or four times where she kept being like this is all fascinating information, but what is the point of your script?
And as the writer, you are the worst person to like objectively assess what the point of your script is.
So having other people talk about it is helpful
And it's a lot easier to edit someone else's work that exists than to come up with it on your own.
So it's totally normal
I think we did eight revisions of my script, and I'm a professional writer
What about defining the point before you get started.
Particularly if you're doing a series, you probably want some common theme, and then each episode has a particular point.
Or I don't know
Oftentimes you have a point in your mind when you start writing, but then as the script comes out, maybe there's a better point.
Maybe discussions like these you discover that you know what?
Actually, this point that I was thinking of when I wrote the script isn't what I want to talk about.
This other point really is.
And that usually happens, no matter how diligent you are about picking the best point that you can before you start writing.
And again, it's a slightly different process for like purely technical instructional video-- like Lynda videos.
Because I remember you had emailed me about that.
I don't know if you guys have checked it out yet, but we're not going to technical how to use Final Cut Pro, how to use iMovie-- but you can go on Lynda.com and all MIT students have access to it.
And they're just purely technical videos.
I listed all the helpful ones in the syllabus.
But the objective of that is basically open your brain, dump the information in, and close the brain.
And I mean that in a very positive way.
They are very, very good videos, but that's a totally different objective than maybe doing a whole series on time travel, because there's so much you can talk about that right.
The objective of a series on time travel to pique curiosity among people can also include informing them and transferring knowledge to them, but it's going to be a very different scripting process.
I hesitate to say a very different scripting process, but the priorities that you're going to have are going to be a little bit different than a Lynda series for example.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
We're going to talk about hosting.
I'm going to go through rules quickly, guidelines just the same as we did before.
Then I'm going to read a short section of one of your scripts.
You're going to critique my hosting.
And then we'll do the same thing for some of you.
Again, we won't do everyone.
This we covered already.
Don't read it.
Tell it.
This is the Hourglass video from before.
This is a subtle point.
You can start off by imitating someone else's hosting style, but you'll invariably just get caught, as like that's the only thing that you can do.
And since it's not your own personality, it's just sort of weird and unnatural, and doesn't work.
I did this when I was first starting out.
It did not end well.
These are my words of wisdom to you.
And no more needs to be said about that.
This is a really hard one.
So there's this device called interview something.
I don't know.
There's a famous director, who basically has a silver mirror that he puts in front of the camera.
And there's an image below.
It's like a teleprompter, except, instead of text, you see a person's face.
You see the interviewer's face.
So the interviewee is looking straight into the camera lens, but that person is actually seeing the face of the person who's interviewing them.
So who's seen the movie The Fog of War?
No one else?
OK. You should watch it.
It's really interesting.
It's a documentary.
It's an interview movie.
And not Richard Nixon.
The Secretary of War for the Vietnam War was McNamara.
So he was interviewed on that movie using this technique.
And it's a really interesting technique.
But the point is when you look into the camera-- we don't have a fancy device like that, unfortunately.
But when you look into a camera, see a person's face, preferably someone that you know, whether it's like a friend, husband, wife, son, daughter, whatever it is.
Pick someone, and see their face on camera.
That is the quickest and easiest way to make your hosting more natural without really thinking about it.
And that's very hard to do, because you're thinking of your script.
You're thinking of 18,000 other things.
But that's the most important rule.
Usually, the camera tones you down.
So if you are talking like you would normally talk, you will kind of seem like this on camera.
It just tones you down and slows you down, and lowers your energy level, unless it's right in your face.
Then it's different.
But usually it's not right in your face.
So you need to compensate by-- I really hesitate to say, up your energy level, because when people say that the initial reaction is to be like, and then, you need to talk like this, because that's how-- and that just doesn't work either.
So you want to be a more energetic version of your natural self.
So think of if you were just really excited about something, how would you actually be?
That's how you should be on camera
And just to have physical metrics, higher energy doesn't mean higher pitch and tone.
That's the biggest thing that happens when they say, now, with more energy.
Then people talk with a higher tone, like this.
And that's the last thing you want to do, generally
OK.
So now what I'm going to do is I'm going to read this.
And you all are going to critique my hosting style.
And I'm going to read it as if-- I'm going to deliver it as if-- oh, wait.
No, most of this is a quote.
So let me do something else.
I'll do this.
I'm going to deliver this as though it's not my job.
OK. Ready?
Here we go.
Why do some people handle cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside with all the winter gear on, mask and all, while others can wear one layer for a morning jog?
What makes all the difference?
OK.
Critique.
What was wrong with that?
Yeah?
I felt like you could do more
In what way?
So you were restrained by the fact that you were looking at somebody else's work
OK. That is true.
So if I was reading something that I had written myself, I'd probably do a better job.
But in this case-- you will be reading your own work.
So that won't be quite the same.
Yeah?
So I think just how you read it, it seemed like you were asking a question, but there's no, oh yeah, I've never thought about it.
It's like, why do some people handle cold better than others?
Maybe if you changed it to like, have you ever wondered why some people-- and try to-- I don't know if it's the structure of sentence or just how you said it
And this is part of the exercise, as I'm reading it word for word.
So let's assume that I can't change the wording.
I mean, in real life, you can.
But what could I do just with intonation of my voice, or what else?
Maybe slow down a little bit, and really hit that question hard
OK. You were going to-- no, you had your hand up
I'm going to think a bit more

The intonation, when you ask a question, you probably want to raise it at the end.
And if you were to ask a very [INAUDIBLE], because there are a couple of questions there.
So I guess you can pick the one that you really want them to think and give them pause.
So get them to think before you [INAUDIBLE]
That's a good point, that you can create suspension in your tone of voice.
But a lot of people make the mistake of assuming that every question has to end with an uptone.
Right?
A powerful question can be posed with the tone ending downwards.
Why do some people handle cold better than others?
That is still a strong question even though I didn't say, why do some people handle the cold better than others?
That actually is--
Well, you didn't end upward on either of those.
The ending upward is like, why do some people handle cold better than others?
That's the traditional way that people asks questions.
But I agree with you.
The way that I read it, all the questions were flat.
All had the same intonation.
What I really want to do is vary.
So something like, why do some people handle cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without all the winter gear, all the mask and all that stuff, where other people can just where one layer for a morning jog?
What's the difference?
So each question I asked there in a totally different way.
What else?
So a big piece of advice that we give to students is to think about the words that you're saying, which is also the hardest thing to do when you're trying to remember everything that you're saying.
You have a camera in front of you.
But that's also why we often take the script away from the students.
It's easy to get caught up in, oh shoot, I have to remember all the words.
And I'm very, very guilty of this as well.
So I totally understand how hard it is to do.
But when you are asking the question genuinely, why do some people get cold while other people don't?
The way I said that was very different than why do some people handle cold better than others?
When I'm actually thinking about the words that I'm saying, which is a very obvious suggestion to make, but, again, very hard to implement yourself, the intonation and the delivery become a lot more natural and genuine.
I don't know if you want to add anything to that
Yeah, and that's another rule.
Like when you look into the camera, see someone's face, that's something that you can do that doesn't-- I mean, it requires some preparation.
But it doesn't require thought while you're actually doing it.
So when you look in the camera, see someone's face.
And actually think about the words that you're saying is helpful too
You want to avoid monotony not just in the intonation.
So you don't want to say, why do some people handle cold better than the others?
Why is it that some are so fearful-- right, that's monotonous in tone, but it's also monotonous in rhythm.
All right.
So when George said it the second time, he rushed through some of the words in the middle that were in a list, which is very natural.
When we're like, this morning I have to go to work, and I have to do this, and I have to do this, and this, you rush through things that you list.
Whereas, if you're just reading like you would read, you say it with the same rhythmic pacing.
You make the pauses where the punctuation is.
You sound very even with the way you break up your words.
And that's super unnatural.
That's why it sounds robotic.
So when you mix up not only the intonation, but also the speed at which you're delivering some of the words-- I don't know.
You can do it again and maybe pay attention to how he does it.
That's kind of also why it feels a lot more natural.
So I'm going to do it again, this time completely different, and get your critique as well.
Why do some people handle cold better than others?
Why is it that some are so fearful of the cold that they would rather die than be caught outside without all the winter gear on, mask and all, while others can where one layer for a morning jog?
What's the difference?
Was that better or worse than the first time I did it?
It sounds like a Ted Ed video
It sounds like a Ted Ed video?
Oh man, then I'm not making very good Ted Ed videos
Better than the first, not as good as the second
OK. Why?
What was inauthentic about that?
You sounded too intentionally energized
Yeah.
So normally, when we say to someone, up your energy, or when anyone says that, that's what comes out.
You get this weird, I have to really try to be excited about this, because I don't really know what I'm reading.
But I'm going to just emphasize random words and say them louder.
And that makes it exciting.
No, it doesn't make it.
And especially, if you mix that in with a bit of newscastery, then you're just dead in the water, because you end up with the most unwatchable video ever
And you sound like you're trying to talk to a 2-year-old too.
I think that's the final effect, which we want to avoid
OK. Now, I'm going to read it again.
And this is going to be a bit more subtle.
Here we go.
Why do some people handle cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can wear one layer and be fine?
What makes the difference?
What does that sound like to you?
It sounds more natural, like you're talking to someone
It definitely sounds more natural.
Any critique of it?
I think it sounded a little disinterested at parts
Uh-huh.
Why?
It was kind of like a little bit of a mumble, like when I talk, and I'm going on like this.
[MUMBLING]
It's like, yeah, [MUMBLING].
It is very natural.
But there is a lack of wonder.
This is a topic that I am sort of taking for granted.
And I sort of expect that you'll take it for granted too, so you do take it for granted.
All right.
I've sort of run out of different ways to do that.
So let's have you guys try.
Who did not have their script analyzed?
Come on up.
So take my hot seat there.
And give us a read
Why do some people handle the cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can wear one layer for a morning jog?
What makes all the difference?
One more time.
Differently.
Do it differently.
By the way, those of you that will do production with me and Elizabeth, that is the phrase that you will hear most throughout the week is, one more time
Why do some people handle the cold better than others?
Why is it that some people are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can where one layer for a morning jog?
What makes all the difference?
Do you have the first sentence memorized?
Yeah
Deliver it without looking at the script
Why do some people handle the cold better than others?
Now, look at Elizabeth, and just ask that question
Why do some people handle the cold better than others?
Now, don't have the line in your head.
Just ask her
Why do some people handle the cold better than others?
Now, ask me as if I were Eric Lander
I've never seen him.
Why do some people handle the cold better than others?
Elizabeth, sit closer
OK.
I'm a professor.
And you're at my office hours.
And I've given you the piece at question relating to hypothermia
Why do some people handle the cold better than others?
Now, ask the question in your own words.
Don't use the words that are written on the script.
Ask Elizabeth
Why are some people so immune to the cold?
So do people see how hard this is?
This is really not easy.
Right?
Thanks.
You're good.
Come on back.
All right.
Who's next?
Your turn.
Same paragraph
All right.
Why do some people handle cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can wear one layer for a morning jog?
What makes all the difference?
One more time
Why do some people handle the cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can wear one layer for a morning jog?
What makes all the difference?
Now, do the first time without reading it
I have a horrible memory
Don't worry about the words.
You can say it however you want
I guess, it's just--
What's the video about?
What's the video about?
Dealing with the cold weather
Say who is dealing with it
OK.
I have it.
Why are some people better at dealing with cold than others?
OK. You can sit down.
Anyone else want to go?
Anyone else want to give it a shot?
Yeah, come on up.
Same paragraph
Why do some people handle the cold better than others?
Why is it that some people are so fearful of the cold that they'd rather die than be caught outside without all the winter gear on, mask and all, while others can wear one layer in the morning jog?
What makes all the difference?
One more time
Why do some people handle the cold better than others?
Why is it that some are so fearful of the cold that they'd rather die than be caught outside without the winter gear on, mask and all, while others can wear one layer in the morning jog?
What makes all the difference?
Great, thanks.
What time is it?
Do we have time?
You only have like 10 minutes
OK. What did you guys notice about all of those, all three deliveries?
Were they different or similar?
Both.
Their personalities came through
OK.
But-- it sounds like there was a but
But they had a really hard time doing something different than what they did the first time
I don't know why this is.
I don't have any cognitive theory as to why it's so hard to do something so differently-- when you have the same words in front of you, why it's very difficult to read them in different ways.
But this just happens with everyone.
And especially, if you've written the script yourself, you know the words so well, you've looked at it a million times, when it comes time to actually be on camera and perform that script and we say like, OK, well, that was great, but do it a different way.
You're like, well, I wrote it this one way.
Why should I do it a different way?
Maybe that's what your brain is thinking.
And so you just don't know how to do it a different way.
Let's try something a little weird and creative.
Julia, right?
Do the first line, but try and do it as different from your first reading as you possibly can.
And I mean, if you want to exaggerate-- don't think of this as I'm hosting a line.
Just think of it as I'm going to say these words in the most different way I can think of
It's OK if it doesn't make any sense
Yeah
Like, why do something handle cold better than others?
So valley girl.
OK, good.
That was very different than your first delivery.
Great.
Remind me of your name again
Joshua
Joshua.
Go ahead.
Do the first line totally different than the way you did it, and not valley girl either
Why do people some-- OK, let me try again
Yeah, yeah
Hey.
Why do people sometimes handle cold better than others?
OK. That was an accent.
So great, very different.
Remind me of your name again
Nathan
Nathan, your turn
Why do some people handle cold better than others?
Julia, yell the line
[YELLING] Why do some people handle cold better than others?
OK, now, close your eyes.
And just say it.
Ask the question
Why do some people handle cold better than others?
OK, that was your best delivery.
I don't know why.
Sometimes getting people to step way outside their comfort zone and do a line in a totally ridiculous way that you would never actually use resets something in your mental circuitry.
I have no explanation for why this works.
But you get out of your normal, and then you just sort of do it the way that you would normally do it if you were talking to someone.
That's a technique that we'll also use when we're in production.
We'll be like, OK-- actually, we might even say, do this line as a valley girl.
Do it with an accent.
Yell it out loud.
Do it like a robot.
[TALKING LIKE A ROBOT] Why do some people handle cold better than others?
And then do it in your normal-- why do some people handle cold better than others?
I mean, that was different than how I've done it every single time.
What other hosting tech-- I mean, this is just one of those things that practice-- you need to practice.
And you should do it, like I was saying earlier, with an audio recorder.
You have them doing vlogs.
Right?
Mhm
It might be good to change one of those assignments into an audio vlog versus a--
Yeah, actually, for your daily reflections-- I know Siri did a video just to make sure that you guys knew.
But you don't have to do text reflections.
You can do an audio recording if you want.
You can do a video blog if you want.
And I think that's a great opportunity to practice some of this stuff.
Your reflection last night, it was really good.
It was really authentic.
You weren't trying to host.
But I thought you did a pretty good job on camera.
And it was very different from your pitch.
Right?
I would suggest that you go back and watch yourself on both your pitch and on your blog last night.
And you'll see a pretty surprising difference.
And it happens to all of us
When you listen to-- that's the other part of this is that once you do your vlog, reflection, or audio vlog, or whatever, listen to it, and see what you sound like.
Do one where you're just talking off the cuff.
You're like, today was great.
We had this guy.
He was terrible.
But he had some good points, whatever, whatever.
And then do one where you actually read-- you can read even this same paragraph if you want, or pick something that you've written.
And listen to the difference.
I think it's only by sort of really listening to yourself that you will learn to do this.
Yeah?
Have you ever tried to have people do practice dry-runs, and maybe tell them that it's not running--
Yes, we do that all the time.
We'll say, oh, this is a rehearsal.
We're not rolling, but go ahead and deliver the line anyway.
Sometimes it works.
It usually doesn't, because you've still got the camera there.
It's still--
And the other thing is even though we want you to sound as natural as possible, when it comes to the day of shooting, the more you've practiced your lines, it just takes out one more confounding variable that will stress you out during the day of shoot.
The students, you can really tell if they've rehearsed their lines.
Even if we end up changing a lot of the wording, they're just much more comfortable, because they're not worrying about remembering things.
They actually are thinking about the words that they're saying.
For tonight's assignment, it will just be to recreate your episode pitch.
I would suggest that you redraft your script.
We are going to do a table read next Monday.
And there's no official assignment for another draft of a script until then.
If you want feedback on any material that you write, which I would recommend, I am happy to give feedback.
George--
Yep, I'm going to write my email right now
--is happy to give-- his email's also on the syllabus.
We're all happy to give feedback.
We will have traditional office hours today, tomorrow, Friday.
So there's lots of opportunity for feedback before the table read.
But just so you guys know for tonight, it's just a repitching of an idea for your episode
That's all the material that I had.
Does anyone have any questions in the two minutes remaining?
And I'm going to stick around.
So if you guys want to chat with me individually, that's fine too
So basically, to host, just be yourself
Yeah.
To the most extent that you can, be yourself
And I mean there are exceptions to every single rule.
I know that when I host, I'm very different than I am in real life
Not really
Really?
No, you're pretty similar
You said that my hosting persona was a lot different than you were expecting
I was just messing with you
But for the most part, be yourself.
And we mean that very literally.
Right?
I don't know if you guys noticed, but that time you asked that question at the very end, it was how you would say it, and it was the best delivery.
It really was
So less of like a pitch to get people excited, but just kind of explain what you're doing?
Because a pitch, just [INAUDIBLE] kind of like excited
Well, when you think of pitch, you think of Shark Tank.
And you think of, I have 30 seconds to buy into my idea.
So I'm just going to be as loud and exaggerated as possible.
So please don't mistake pitch for that.
Keep it under a minute and a half.
A lot of your videos were five and a half minutes.
That's longer than your entire episode is going to be.
Keep it under a minute and a half.
Not just as a challenge, but that'll really help structure the way you deliver.
And that should be the point of the video.
So maybe don't think about it as a pitch as it is a point, like a thesis almost
Think of it as explaining your video to someone else
So it's not the actual stuff that you wrote?
It's like a trailer
Remember-- Paul, right?
Remember when Paul was doing his shoebox thing.
It's that
I don't think I have a shoebox, but--
You can just mime one
We've got a budget for a cardboard box.
Let me tell you
We've got about $1.50.
OK, thanks, guys
All right.
We'll stick around for a while if you guys have questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So this is Josh and John.
They're both from Planet Nutshell.
Josh is the founder and John is an animator?
Yes
And they're going to talk today about storyboarding and continuing our discussion on visual storytelling.
I'm super excited about today's session.
It's going to be a lot of not only lecture, but workshopping some of the scripts and ideas that you guys have been thinking about so far.
Anything else?
Oh!
Josh's background is in creative writing.
So you can also get feedback from him on your scripts.
And they make short explainer videos that engage people, not only in diseminating information, but entertaining them as well
Yeah.
So we in lot of ways do many of the things that, I mean, I guess directly that you guys are working on.
So this is a good fit for us to be talking to you.
So, yeah.
So my job is to sort of rally all of the people who work at Planet Nutshell in interesting, creative directions.
And John is a really talented illustrator and animator.
And he also does our storyboard artwork.
So today, after I give a brief little talk about animation and storyboarding, we'd like to just open things up and share with you a script that we produced a while ago.
And we'll all storyboard it together.
And I would love to see what we come up with.
We've produced this script, but what's really great about what we do is there's no one right way to do it.
So what will be interesting is to see what we come up with collectively here.
As opposed to what we produced.
And I have looked at your scripts, the things that you're working on.
They're all really interesting.
So I would like to open things up to any questions you have about what you're working on a little later.
And yeah, I write a lot of scripts.
So if you need help with that, I'm very willing to give you any guidance I can.
So I'll get started with this short lecture portion.
Somebody made this spinning thing for you guys today.
We thought that would be a nice touch.
Ding!
All right.
Let's see.
Yeah, so we're going to talk about animation and storyboarding.
And specifically, animation in the context of what you guys are doing.
I mean, animation is a huge topic.
I'm not going to talk about techniques or theory or any of that kind of stuff.
But I do want to talk about when it's a good idea to use it maybe in your own work.
And there may be more reasons why it might be good.
But these are the ones that I came up with for today.
So animation is good when synthesizing, reinforcing, or recapping information in a simple, visual way.
Cows plus farts equals global warming.
So taking abstract ideas or summarizing information and presenting it very simply is something that animation is really good at.
So when thinking about developing your own scripts, you can think about moments where the visual story could be augmented with some sort of encapsulation or summary that is worthy of animation.
And here's one that's pretty obvious.
The subject cannot be seen, it's microscopic, cosmic, or theoretical and abstract.
So those are often good times to think about animation.
And we have some examples of where this sort of thing is employed later.
John came up with this one.
Actually, John helped with all these.
Because I was really busy this week.
So I really appreciate it.
Footage is not easily attainable.
So say you want to do a video about-- well, in your case, you know you're doing navel engineering.
Maybe you want to shoot something about, like, the bottom of the ship.
Or something, you're going to have to hire a team of scuba divers to get down there and shoot the hull of the ship.
So that might be a nice opportunity for visualization that can't be attained through live action.
And kind of the catch-all category is that animation adds visual engagement.
The story is somehow made better by using it.
And that's sort of an abstract, subjective thing to say.
But I think the best way to demonstrate that is just to show examples.
Which I'll do in a bit.
So animation is sort of silly or not a great choice-- we use this term a lot.
Like, is this eye candy, or is this really a functional part of the story?
Again, is it eye candy or is the story made better?
So just using animation because it's cool or it's neat can actually distance the viewer and disengage them from the experience.
Because, I don't know if you guys watch-- you know, you've seen a movie like Transformers.
Or, god, what are some of these ridiculously visually opulent movies, you know?
And a lot of times when I watch those sorts of things, I begin to emotionally disengaged from them.
Because all I'm doing is being bombarded by visual information.
There's no story that's really compelling or engaging me.
So that's something to think about.
Live action is really great that at a number of things.
And these are some of the things I came up with.
The notion of personality, the human face, the human experience.
The host, is Elizabeth talked about on Monday.
Location, creating-- animation can create a lot of interesting locations.
But why would you draw the MIT campus when you have it right in front of you, for instance?
And again, the notion of human interaction.
Humans talking to each other and engaging with each other is something that live action is really good at.
And the final, catch-all category is you just can't think of anything better to do.
So, oh, I'm just going to animate that part.
I don't really want to think about how I'm going to present this.
So I'm just going to do some animation.
And that's not the best choice.
You should be making choices based on what the strengths of animation are and the strength of live action
And just to add to that, that's also the easiest choice to make.
And the one that happens most often.
Or the one that happened most often when we were making seasons one and two is that we'd hit a point in the script that looks a little bit complex in terms of visual execution.
And we'd say, well, what are you going to do here?
And the host or the cast would say, oh, well, we're just going to animate it.
As if, like, that was going to solve all their problems.
And I think that these points may sound pretty reasonable and obvious.
But it's very, very hard to thoughtfully integrate that into your visual script.
So I really encourage you guys to think about especially the third point, not defaulting to animation because you don't want to figure out how to visually integrate your script into the video
Yeah.
OK, so here are some examples of videos that integrate both live action and animation.
There are many examples.
I thought these were especially strong ones.
I don't know what you guys-- you're going to talk more about how you made this one.
But the composition of this one works.
It was obviously designed around position of the live action character and the animated elements.
Who has seen this Story of Stuff series?
It's quite interesting
Do you want to play just a little of it?
Yeah, I'll play a little bit of it
So next, the-- Do you have one of these?
I got a little obsessed with mine.
In fact, I got a little obsessed with all my stuff.
Have you ever wondered where all the stuff we buy comes from?
And where it goes when we throw it out?
I couldn't stop wondering about that.
So I looked it up.
And what the textbook said is that stuff moves through a system.
From extraction, to production, to distribution, to consumption, to disposal.
All together, it's called the materials economy.
Well, I looked into it a little bit more.
In fact, I spent 10 years traveling the world, tracking where our stuff comes from and where it goes And you know what I found out?
That is not the whole story.
There is a lot missing from this explanation.
For one thing, this system looks like it's fine.
No problem!
But the truth is it's a system in crisis
Just a storytelling component there that I really like is you may already sort of know-- we talked it about on Monday.
Here's a paradigm or here's a situation that you probably already understand, but hang on.
There's a problem.
There's more complexity here than you ever could have imagined.
And I'm going to tell you about it.
And that is a really cool storytelling component on top of the integration of animation and live action
An the reason it's a system in crisis is it's a linear system.
And we live on a finite planet.
And you cannot run a linear system on a finite planet indefinitely.
Every step along the way, this system is interacting with the real world.
In real life, it's not happening on a blank white page.
It's interacting with societies, cultures, economies, the environment.
And all along the way, it's bumping up against limits.
Limits we don't see here because the diagram is incomplete.
So let's go back through.
Let's fill in some of the blanks and see what's missing.
Well, one of the most important things it's missing is people.
Yes, people!
People live and work all along this system.
And some people in this system matter a little more than others.
Some have a little more say
OK, so you get the idea.
I mean, it's a long video.
But I think it retains engagement because of the quality of the writing.
And because of the interest that's created from the art work.
I encourage you to watch the whole thing if you get a chance.
Let's see--
[INAUDIBLE] all of the videos that Josh is going to play today are on the syllabus
Oh, OK.
Cool.
You mind if I play some of this?
Or have you guys already seen this video?
I've got a little clip of the portion that I'm going to talk about, but you can play it
OK, I'll just play the beginning.
This has George in it, who I assume you saw yesterday
You know what's more like life than a box of chocolates?
Farts.
You really never know what you're going to get.
Skunk, rotten eggs, garbage dumpster?
The possibilities are, unfortunately, endless.
But why do we fart?
And how does your body make so many different smells?
The answer is fermentation.
This is a giant bioreactor.
It's also called a fermenter.
And in here, a fungus known as brewer's yeast is transforming water and dead plants to beer.
So you know that thing where you eat food and breathe in oxygen to get your energy?
That's called respiration.
Yeast also need to get energy from food, but there's no oxygen in those tanks.
So they have to use a different process called fermentation.
Now, both your cells and yeast cells break food down to get energy.
But your main byproducts are carbon dioxide and water.
And yeasts main byproducts are carbon dioxide and alcohol.
That's how this stuff gets to be beer
OK, I think that-- similar concept, the integration of animation and live action.
And the way that the animation somehow augments or improves your understanding of what he's talking about.
So let's see.
So let's switch back here.
So that's a very brief overview of how you can use animation in your own work.
How to do it, technically and artistically is another matter.
I'm happy to talk about that separately.
I don't know how much training is in this course for animation
Not whole lot.
We do have the equipment if you want to learn more.
We have about 10 of these similar tablets if you guys are interested in learning about this.
But we're not going to go super in depth
Yeah, this might be an opportunity for you to talk about that segment that you did.
And from what I understand, the animation that you do is fairly rudimentary.
And very effective.
And it might be something that these guys could pull off
All right, so I'm going to talk a little bit about one of the scenes from the farts video.
This was actually something that George was going to talk about yesterday in his script writing workshop.
So this will tie in pretty nicely if I can get this to work.
All right, so take a second to read this passage.
So when you were writing the farts script, this was a piece of information that George had come across.
Which he found very fascinating.
It was one of the shareable facts that he latched onto.
The fact that the microbial population in your gut is about 10 times greater than all the somatic germ cells in your body.
Now if he had said this line in the video it would've been a disaster, right?
But what is this passage essentially saying?
Does anyone want to take a guess?
[INAUDIBLE]
Not just that.
So somatic and germ cells basically refers to all of your human cells
There's more stuff that's not you in you than there is pieces of you
Yeah, yeah basically
Mostly [INAUDIBLE]
Right, right.
So the way George decided to distill this down there 10 times as many microbes living in your gut as there are human cells in your body.
I mean, that's basically what this entire passage taken from a journal article is saying.
But the problem with saying just this is that, at least for me and for George, one of the questions that we thought of as an audience member was well, how does that exactly work?
Like, how can you fit 10 times as many bacteria in you, yet you're not walking around just oozing bacterial cells completely outside of your body.
Because I think a misconception that people often have is that they're the same size, right?
So it was kind of this paradox that wasn't essential to the story of the video to explain, yet was a potential point of confusion.
So this was a reason why we decided to use an animation in this particular spot.
Because As Josh was saying earlier, we felt like the story needed to be clarified.
We felt like the story needed to be advanced a little bit.
But we also felt like if we took the time to say there are 10 times as many microbes in your gut as there are human cells in your body.
Which doesn't make sense, because you would think that you'd be oozing out bacteria, but it actually is fine because bacteria are a lot smaller than human cells, right?
That actually is, like, two or three seconds of very, very precious time wasted.
And sort of loses the momentum of the video.
So we decided to do an animation to explain all of that instead.
So what we did was we had shot everything without the animations.
And our storyboarding process is incredibly janky.
But it essentially works.
And hopefully after today your storyboards will be a lot better than the one we used.
But what I literally did was I got the rough cut of the video.
And I got all of the footage.
And I specifically was looking at this portion of the video where this animation would need to go.
And I took screenshots of the video at various time points where I knew the information needed to go.
So this is setting the line in context of the section that he was delivering.
So the first line [INAUDIBLE] that we carry a fermenter right here with us in our gut.
And this was the still that he said that, so I was telling the editor, at this point I want a fermenter to show up.
And he actually motions during the scene.
And then there was this second line, "Except instead yeast, my gut uses gut bacteria."
So I was trying to convey to the editor to take my drawing of the yeast and swap it out for drawing of a bacteria.
Again, this is incredibly rudimentary.
You don't have to be an amazing artist to pull off the clarity that you need for a storyboard.
Then he says, "A lot of them," and I wanted a ton of bacteria to show up.
And then we get to the line, "There are 10 times as many gut bacteria as there are human cells in your body."
At which point I wanted lots of bacteria to show up and then show it in comparison to the human cell.
This was going to be the key frame to explain the concept of how you can have so many bacteria without oozing it outside of your body.
So essentially we were explaining the whole difference in scale through one frame
This reminds me a bit of the flipping cat video we saw on Monday.
And there's a segment in that video that I think is kind of weak.
Where he is explaining a lot of technical information.
I just didn't absorb it.
I couldn't follow it.
And I was thinking, you know, this would have been a lot better if he'd maybe done some better script writing.
But also maybe some animation, you know?
I think animation can be really good at moments like these when you've got this sophisticated or complex information and you need to maintain engagement with the viewer.
But also encapsulate it.
Yeah, anyway, sorry to highjack
No, no, no.
That's fine.
[INAUDIBLE].
And then the whole way we wrap up that scene is he explains that instead of just one species of yeast like they use when you brew beer-- that's the whole reason why we were at the brewery-- my gut uses hundreds.
And if we just had that text, I think that it would bring up a lot it questions or maybe points of clarification for the viewer that, again, we just didn't have the time to explain in the script.
Which is why we relied on the animation to do it for us.
So John's going to show you some animations that are awesome and very artistic.
For our animations, I literally just take a Sharpie and a piece of paper, and I draw out the doodles
They're nice!
Thank you!
But it's nothing crazy.
And I basically scan them in to our editor and had named each picture.
And then in the storyboard I had explained, OK, here's my drawing of the fermenter.
Please refer to fermenter.jpeg for the image I want you to use.
And then he went into After Effects and inverted all of the drawings and moved them around.
And I think you're going to talk about After Effects just a little bit
We can talk about After Effects
OK. And again, if you want to play with any of this on your own, there are really good tutorials on Lynda.com.
And as MIT students, you guys have free access to the entire library.
So if you want a tutorial on how to use After Effects, which you all have access to as well if you're interested.
You can just come talk to me during class.
So we storyboarded everything out.
And then I drew the doodles.
And the key frame here was this picture that sort of explains the whole concept of how so many bacteria can fit inside of you.
But it just takes one picture instead of 10 words.
So this is what that scene ended up looking like
Weird.
Turns out, we all carry a fermenter with us right here in our gut.
Except instead of yeast, my guy uses bacteria, and a lot of them.
There are 10 times as many bacteria in my intestine as there are human cells in my entire body.
And instead of just one species, my gut uses--
My gut uses hundreds.
Sorry, it got cut off.
So, I mean, there are many ways that we could have done that.
Like Josh was saying, there's no right way when it comes to animations, when it comes to video in general.
But this was sort of our way of working around the problem of not having access to the footage of real bacteria necessarily.
It was just a world that we couldn't access, or didn't have time to access, or would have been too distracting to access in this video.
The other thing I wanted to mention really quickly before Josh continues is when we shot, we shot knowing that we were going to do the animation overlay in mind.
You may not know that ahead of time.
So what I would suggest, if you're interested in doing something like this, is to shoot thinking that you're not going to do an animation overlay.
And then just do another take.
Just in case you want to do something like this.
So the way we framed the shot, we tried to make sure that the background was dark enough to where the animations could appear.
This is not an ideal framing in a not an ideal background.
So I'm sorry to use this as an example.
This is probably one that we could have improved.
But I'll show you another one
These small molecules are called metabolites.
Just like how all the DNA in an organism forms the genome, all of the metabolites form the metabolome.
Even though each metabolite can be made from only six elements, there are so many possibilities that it would take scientists thousands of years to make each one and figure out--
And configure out its usefulness.
So we had the exact same storyboarding process to create this.
But when we shot Anastasia that day I knew that this is what I wanted to do for that scene.
So we told her to go like this with her hand.
And we were like, we're going to put stuff there, don't worry.
And we framed her wide enough.
And we gave her enough grass space outside of her that we could fit some of these animations in.
So just anticipating that-- and I can talk more about this tomorrow, when I talk about producing-- is a good thing to do if you want to do animations.
And I'll show you one last
Those animals get eaten by slightly bigger animals, which get eaten by slightly bigger animals.
Which get eaten by animals at the top of the food chain.
Animals like us
So that's the raw footage from our video on engineering river cleanups.
And again, this is a spot that we knew we wanted to do an animation there.
So we framed her in such a way that there is enough background space to accommodate for the animation.
And we actually told her, OK, we're going to put fish along here in post.
So pretend like you're looking at stuff, point to them as if there are going be fish there eventually.
I think we also did it take where we just had her delivering it normally.
So we had her framed with not as much extra space behind her.
Just in case we weren't going to use that animation.
But again, sort of anticipating using that if you need.
So this is what the final scene looked like
Those animals get eaten by slightly bigger animals.
Which get eaten by slightly bigger animals, which get eaten by animals at the top of the food chain.
Animals like us
So again, it wasn't incredibly vital to have these animations.
One of the reasons why we did use it is that, in the context of the video-- which I'm going to show tomorrow-- we felt like the visual interest was getting a little bit static.
So we used animation to move the story forward in that we felt like it was getting a little visually monotonous by the time this hit.
And the other thing is the food web stuff is OK to explain.
I think that if you didn't have the animation it would still be OK.
But it just added I guess a little moment of whimsy to it.
And so that was sort of our justification of putting it there.
So I just wanted to show you guys those examples.
And Josh, you can continue on
OK!
I can switch here
Yep
When John is helping us out with the workshopping portion, he can show you some projects in After Effects.
I don't think After Effects-- it's a complex program with a lot of capability.
But I think some basic compositing like this would be quite easy in it.
So we can show you some of those, just some basics about After Effects.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so let's turn the page to storyboarding, which, for us, no matter what sort of video you're doing-- live action, animation, or some sort of combination-- is vital to the process.
And here's why.
It allows you, basically, a lot of freedom to not have to make any really high stakes creative decisions and just be creative at a very early stage.
You don't have to commit to any sort of production at that point.
That's why we do it.
And it's why we share with clients before we commit to doing any animation, because it's very time consuming for us to animate.
And we want the client to know what we're doing before we commit to doing any of that really labor-intensive work.
It serves as a blueprint for timing and visual flow.
So you can get a sense of the pacing of the video, whether or not, as Elizabeth was saying, things are going to get stale at a certain point.
It can help you make those sorts of creative decisions without having to commit a lot of resources.
The storyboard can reveal where you're going to need to shoot, what props you're going to need, the types of shots you're going to employ.
All that decision making gets made in the storyboarding process.
And another, really crucial thing is, so often when we're doing a storyboard, we realize that the script needs to be different.
You write something-- maybe you have some visual sense when you're writing a script.
And that's good, because-- and I've done it a lot.
So a lot of times I am thinking visually.
But I'm not always thinking visually.
And I'm not-- it's not my strong suit all the time when I'm script writing.
So a lot of times, we'll be in a storyboard meeting.
And we'll say, well, this line is just sort of dumb.
Why don't we say this so that we can show this?
So that integration of visual and oral-- the voice-- is a really crucial part of storyboarding.
So that's why we storyboard.
And let me just show you an example of one of our storyboards.
There are many different ways to storyboard.
But I'll just show you one example.
The first is-- well, I'll show you the storyboard for this video we did for NASA, actually.
And it was produced by WNET in New York.
And they asked us to create a series of videos about different physics topics.
And this one is on centripetal force.
So I'll just switch over to-- let's see, episode-- So this is essentially-- this is our storyboard method.
We do an Excel spreadsheet with three columns-- shot, narrator, and notes.
Very simple-- email me, and I'll send you this storyboard.
And you can use it in your own work if you want.
Imagine you've got a tennis ball.
So we start the shot on the tennis ball.
We either pull back-- we'll watch the video.
I can't remember if the camera pulls back or we jump cut or something-- attached to a piece of string.
And we pull back further.
And you start swinging it.
And then-- when did we have this idea for the dog?
I mean, probably this came in the storyboard meeting.
Like, OK, we'll have this nice dog element to the experience.
In circles-- and so gradually, the camera's getting further back.
And now we can see the background.
"Whoa now, watch out for your buddy."
This is not for adults, by the way.
I'm sorry.
I should've mentioned that.
This is for, like, elementary and middle schoolers.
So if you find this a little facile and dumb, just imagine you're 13 years old.
"Watch out for your buddy."
So the guy on the skateboard ducks down.
And then he goes past and waves.
Let's take-- so we start with-- my attitude about this, in a lot of cases, is to start with action.
Like, start with something that's really compelling.
Like, some-- you don't have to start, right away, explaining things.
Just sort of start with imagining something or doing something.
There's that old comment, I think-- this is Hemingway said this, he talked about "in medias res."
Start in the middle of things.
So don't start with elaborate explanations.
Just start in the middle of the action.
So something needs to be happening at the beginning of a video.
So that's why we chose to start-- you got this ball spinning.
There's something going on.
There's this dog.
There's the kid skateboarding by.
You're starting with a high level of engagement and interest.
Let's take a closer look at the fascinating physics of this simple action.
And then we go into the more detailed analysis of what's really going on.
We talk about f equals ma-- Newton's second law-- and how it relates to objects moving.
So I'm not going to show you the whole storyboard.
But you start to get a sense of how we put this thing together visually in a very methodical way before we did any work-- I mean, this is work-- but before we did any animation or illustration.
And I think John did all the artwork for this.
And it's very basic.
She's basically a stick figure.
John makes really nice stick figures.
So he has that talent.
So let me just show you the video.
And it came out of that process.
Hold on a second.
There we go.
Whoops, that's not the right one.
Here we go.
This is a minute and 46.
So our challenge was to do these topics in about a minute and a half, a minute and 45 seconds.
[VIDEO PLAYBACK] -Imagine you've got a tennis ball attached to a piece of string.
And you start swinging it in circles.
Whoa, now!
Watch out for your buddy!
Let's take a closer look at the fascinating physics of this simple action.
You've probably heard of Newton's second law of motion as it relates to objects moving in a straight line.
Turns out, Newton's second law also tells us about the net force on an object as it moves in a circle.
When an object moves in a circle, its velocity changes direction.
Since the velocity's changing, that means we have an acceleration.
And we know from Newton's second law that if we have an acceleration, we have a net force.
Now back to our spinning ball.
For objects undergoing circular motion, the net force on the object is called the "centripetal force."
It's a weird sounding word, but it's Latin for the phrase "center-seeking."
It's named centripetal because the centripetal force always points to the center of the circle.
The ball has a tangential velocity.
So if you suddenly cut the string, it would fly off in whatever the tangential direction was at the moment you made the cut.
But assuming you don't cut the string, and you keep swinging the ball, the ball's tangential velocity is constantly changing directions.
This means the ball is accelerating.
And the reason it's accelerating is due to the centripetal force.
The centripetal force acting on the ball is constantly directing the ball towards the center of the circle, your hand.
This, combined with the ball's velocity, keeps the ball swinging in a circle.
The same centripetal force that keeps the tennis ball swinging in a circle also keeps roller coasters on the track as they go through a loop-de-loop.
So the next time you're having a blast on a roller coaster, you can thank the centripetal force.
[END PLAYBACK]
OK, so that's a pretty short little ditty.
But you can-- I just wanted to show you how we built it.
And this next thing is-- I'm not going to show the whole thing.
But it's interesting.
This is a video, a sort of an insider look at how Pixar does their storyboarding.
And I'll just play a little bit of it.
And then perhaps you can watch the whole thing on your own time.
[VIDEO PLAYBACK] -All right, so this section is the storyboarding process.
And this is where, instead of writing with words, we write with drawings.
And we put them on little pieces of paper.
And it's sort of seeing the movie in comic book form.
And we'll do a section of the movie at a time and we'll pin them all up on boards.
And then, when we get a certain chunk done, we'll show them to the directors.
Or they will show them to us.
And, I don't know, it's kind of complicated thing to explain-- -Well, you know, that was a really good description of about how better to describe-- -Something better than that.
-Yeah.
-Well, we could show it.
-Yeah, let's show it.
-With storyboards.
-With storyboards.
OK, this is the storyboarding process, as illustrated by Joe Ranft, our head of story.
-Yes, so there's the little, cute, Mr. Storyboard Artist working very diligently and happily in January.
-And he has a deadline.
He's working hard to do his sequence.
Every time he does a drawing, he pins it up on the board.
And there, he stands back when he gets done with his sequence.
And he says, hmmm, did I do it the best way?
And storyboarding actually balances, like, composition, cutting, staging, acting, writing, everything that goes through the storyboard artist's mind as he's doing it.
-So he'll try another pass at it a couple more times until he feels he's got it just right.
-Just right.
-It's perfect.
-So he calls in the story department-- -And the directors.
- --and, of course, the director.
-There you can see John Lasseter.
Looks just like him.
-Yeah, it does, doesn't it?
Thanks, Joe.
-And then Joe-- or whoever's pitching, the little storyboard artist, in this case-- will pitch the sequence.
And in the middle of the pitch, John gets an idea.
-We all get ideas.
And that's one of the great things about storyboarding here at Pixar is it is a free-for-all once we get in this.
And everybody is out, coming up with ideas to make the sequence better, to make the film better.
And when the pitch session is done, inevitably there is lots of ideas to make the sequence better.
The storyboard artist kind of wakes up from it.
-Right, and we're infamous for putting all our notes on little post-its.
-Yeah.
-That's why you see all the little yellow squares there.
And so we'll all leave, all excited about the great new ideas we've got.
And then the storyboard artist-- -Mr. Storyboard Artist goes back and thinks about it and starts over with all his notes that he has to do.
Joe Ranft often calls it-- the storyboarding process-- not storyboarding, but "story-reboarding" because we work on it, redo it, redo it, redo it, to make each sequence, to make the entire film as best as we possibly can be.
In the next section, you're going to see Joe Ranft, our head of story on A Bug's Life pitching to Co-director Andrew Stanton.
-Me.
-And I think Bob's in there, too.
-I think Bob Peterson, one of our story guys.
-He's pitching the circus sequence from A Bug's Life.
Enjoy.
-OK, so Slim and Francis are hit by the spotlight.
They're not too excited.
And Slim says, tra la la la la, spring is in the air.
And I'm a flower with nothing interesting to say.
And then they go, ah, look, a bee.
And then Heimlich comes running forward, I am a cute, little bumblebee.
Here I come!
I want to catch up with your flowers!
-Oh, oh, oh, I've got it, I've got it.
What if, like, he just can't catch up?
He's like-- it's just so heavy, it's just hard for him to catch up to Slim.
-Slow down, slow down!
So he's like-- [PANTING] you flowers, slow down, slow down!
-Right, right.
-Yeah, right.
That would be great.
You flowers are too fast!
You're running too fast.
Slow down, you guys!
-That's great.
-OK, that'd be good.
-OK, so then he's chasing after them and Heimlich runs by the stands.
And there's a little kid eating candy corn.
And he stops.
And he turns around and he goes, oh, candy corn, here!
I, oh-- you have to let me eat it!
Oh!
And his stomach is grumbling.
And he can go, let me help you, let me help you to finish it!
-Wait a minute.
Is a candy corn funny?
-We were thinking of red vines.
-Would red vines be funny?
That's a little bit tough for a fly to hold.
-There's those little sombrero jujubes.
Those are kind of funny.
-The question is, what would a fly like as far as candy goes?
-Well, dots.
But we already have a character-- -Candy corn is pretty funny, though, because it's really, like-- it's a big triangle.
I think that actually-- -OK. Yeah, that's why-- -More identifiable.
-I think we should go with candy corns.
Like, is it too Halloween?
-Nah.
-No, OK.
So he goes, Oh, candy corn, here.
Let help you to finish it.
-Great.
That's great.
-Tra la la la la.
Spring is in the air.
And I am a flower with nothing interesting to say.
-Ah!
-A bee!
-I am a cute little bumblebee!
-Ah!
-Here I come!
-Ah!
-Slow down, you flowers!
Oh!
Candy corn!
Here, let me help you to finish it!
[END PLAYBACK]
So they obviously-- they went with the candy corn
Before you move on--
Go ahead
--share the storyboard that came from Science Out Loud?
Oh sure, yeah
Sorry, I totally forgot that I had this

Not to--
No problem
Sorry, Josh
No problem, here
Thanks.
We've been seeing all these very gorgeous storyboards.
Like, even-- you guys talk about how it's not when the real work starts.
But obviously that takes a lot of time and talent.
So this was a storyboard that Ashley did for our very first episode of Science Out Loud, just to show you how basic it can be and still be effective.
So she did an episode on exoplanets.
Ashley's K-12 video-- ooh!
And what we discovered after she storyboarded this-- she's talking about a very abstract, cosmic topic.
So obviously we can't go into space, which made deciding what the visuals were going to be surprisingly difficult.
Or maybe it should have been unsurprising.
But we were newbs at the time.
So it was hard.
But she-- literally just drawing stick figures, aligning them with her text.
And when we were looking at the storyboard, we were like, wow.
There's not really a whole lot going on, except for her just standing there.
And basically half of the video was going to be an animation without her on screen.
And there would have been no way for us to really anticipate the extent to which she was missing from screen until she storyboarded this out for us.
And so she tweaked not only her visuals, but she ended up tweaking her script a lot after she did this to make room for scenes in which she could be more present, where it could be more than a fully animated video.
But in case you wanted to see, we actually kept the first six frames pretty similar to her original plan.
And we just tried to alter the animations to make it a little more engaging.
So I'll show you the first part of her video.
[AUDIO  PLAYBACK] -This rock can absorb-- [END AUDIO PLAYBACK] So this was-- this animation was actually done by our first editor.
[VIDEO PLAYBACK] -25 years ago, most people would have told you that there are only nine planets in the universe.
Since then, we've lost one.
Sorry, Pluto.
But we've discovered thousands of others.
So what happened?
Did astronomers suddenly get a new pair of glasses and now we're meeting all of our new neighbors?
Well, no.
It turns that we could've seen these exoplanets all along.
But we only recently figured out exactly where and how to look.
We call these new planets exoplanets, or-- [END PLAYBACK]
So, because we knew what she wanted to do with the animations ahead of time because of her story board, that allowed us to direct her while we were shooting to put her hands down, to look up at the ceiling, and all that stuff.
And I'll show you one more storyboard that was done a little bit differently.
And then I'll let Josh talk again.
But Alex actually just did pictures for some of them.
So he did an episode on how carbon nanotubes are made.
And he had this sort of "imagine a blank" opening.
And so he just took pictures of the locations and types of places that he wanted to go.
And then, from there, he literally just drew stick figures.
So you don't have to draw anything crazy beautiful-- no offense to Alex.
But this was actually very helpful for us as well to see how the pacing of his demos were going to go.
So we altered his script again after he made this, because we realized that it was a lot of just him sitting at a table demoing things.
And we decided that there needed to be something a little more visually engaging besides him just at a table.
But he's talking about how a thread made out of carbon nanotubes could lift up something as heavy as an elephant.
We quickly realized that we wouldn't have footage for that.
But this is again, really basic-- nothing too crazy-- but helped us a lot in the planning of the video.
So I'll show you the first couple bits of his.
[VIDEO PLAYBACK] -Imagine a material that would let you use your phone for weeks without a charge or would let you build an elevator to space.
These elevators and batteries can one day exist with a material called "carbon nanotubes."
Carbon nanotubes are tiny.
Take this human hair, for example.
You need 10,000 carbon nanotubes to make a rope as big as this hair.
Despite being small, they're really strong.
This cotton string holds up this lego car fine.
But if it was made from carbon nanotubes, it could hold up a real car.
[END PLAYBACK] So originally he was going to talk about this model of carbon nanotubes at the very beginning of the video.
But after we saw the storyboard, we thought that we should wait until later on to explain it and just hit them with this example first.
So I just wanted to show you guys that so you knew that your storyboards don't have to be anything super complicated
Cool, thanks.
OK, let's see.
All right, so just some-- we're almost done with this, by the way.
But some quick tips for storyboarding-- and I guess this was something that Elizabeth was just saying.
It doesn't have to be a work of art.
Stick figures work.
Focus on details that matter.
We don't draw a lot of backgrounds.
We just draw the bare minimum of elements that will get across the intention of the scene or the shot, rather.
It should include a composition and camera angles.
Is this a wide shot?
Is it a close-up?
What is the cam-- you might make a note about what the camera's doing.
Is it pulling back?
Is it on a dolly?
Is it tracking something?
And just so you guys know, since we haven't really talked about that stuff yet, that's a lot of stuff that Chris is going to talk about tomorrow.
So I would suggest that you try to keep that stuff in mind and then maybe revisit it again after Chris talks about it tomorrow, too
Yeah
And you can also ask him while he's here, too
Right
You know I'm not supposed to--
It's OK if you have, like, spoilers in the class--
Yeah.
And Chris-- I mean, Chris knows the grammar of this stuff better than I do.
Admittedly, I didn't go to film school.
So we use terms of art in our studio that probably aren't appropriate.
But the camera's close-up on this guy.
The camera's far back from this guy.
The camera's moving while this guy's talking.
That sort of-- any of that stuff.
Don't be constrained by--
Too many technical terms
Right
What about aspect ratio of, like, the paper that you're using in the shot that you're gonna use.
Eight and a half by 11 turned sideways, like--
Yeah
Or is there another shortcut that's helpful?
Yeah, well a note card is a very-- turned sideways is perfect, is a good 16 by 9 approximation.
The template that I'm going to give you guys is widescreen.
But yeah, definitely have to think about those sorts of things.
You don't want to be doing it in a 4 by 3 box.
You want to think about the full frame.
And this is sort of stupid.
I don't why I put this.
Make them easy to read visually and understand.
So I think one thing I wanted to really stress is that, for us-- and I think, for most people, as you saw on the Pixar video-- storyboarding is a collaborative process.
So you can't-- this is a process.
This kind of art is something that comes out of a lot of different minds working together creatively.
And so the story-- one reason to make the storyboard really easy to understand is that somebody else is going to have to use it and work on it.
It's not just what's inside your head.
This is a part of a process that collaborators are all working on.
So it has to communicate very effectively
And I haven't mentioned this formally yet, but obviously, if you're going to be hosting your video, you can't simultaneously be shooting it as well.
So the way that's going to work is we'll partner you guys up, or we'll figure something out to where basically, you'll be shooting someone else's video.
And someone else will be shooting yours.
And the storyboard is going to be really important for that reason, because you have to somehow give the instructions to whoever's shooting for you exactly how you want things framed, how you want your shot to look like and things like that.
So I agree completely.
It's a very collaborative and necessary thing
Yeah.
If there's time, make a few boards.
Like, a lot of times people in our office will make some boards independently of each other.
And then we'll bring them all together and see what we've all got.
Sometimes that's part of our workflow.
If you have time, storyboard the same script in a couple of different ways.
You might get different ideas with different perspectives.
And as I said before, revise the script as needed.
Sometimes a great visual idea doesn't line up with the script.
Focus on visual story, not adherence to your script.
So if you come up with a cool idea, and you may-- you know this is hard to-- I can say this.
But you just have to experience it to really understand it.
You might come up with a cool idea, but the script doesn't quite match what you're saying.
So don't feel like you're locked into that script.
And finally, the storyboard is only a guide.
Changes are often made in the production and editing process.
So all this stuff is very plastic.
You're moving it.
You're changing it as you go.
All right, so how to make a storyboard.
We brought some paper templates.
You can use note cards, We don't use note cards.
But it's actually an interesting idea, which is that-- then, if you put one scene or one shot on each card, you can start moving the cards around on the wall, which might be interesting, an interesting way to do it.
And each frame should include the script line and basic direction notes.
We don't always do that.
But we, at minimum, we include the script line that's coordinated with the shot.
So let's workshop.
And there's a couple different ways we can do this.
Well, first let me go over what I think this basic format.
John and I will sit up here and storyboard a script.
And what that script is, I want to talk to you about a minute.
And then, once we've done some of it-- maybe over 30 or 45 minutes-- we'll break you into small groups and you'll continue the same script, storyboarding it.
You'll do a few-- some frames on your own.
We'll walk around and help you.
And then you guys will present what you have.
And we'll just and share and discuss them.
Yeah?
Josh, how do you feel about using PowerPoint for storyboarding?
That works.
That would work great
In my brain, that would be a better tool--
Yeah, sure
That's what Ashley did.
She made hers on PowerPoint
Yeah, that would work great-- Keynote, PowerPoint, that would be great.
And you don't have to use these paper sheets.
But unless you have a Wacom with you, it can be difficult to draw on your computer.
So the script that we're going to storyboard, we can either do a script that we've already done-- that we've already written and storyboarded and produced.
But it would be-- it would be wonderful to storyboard it again with all of your input.
Or we can take one script that you guys are currently working on and just get started on it with you.
So either someone-- is-- someone is either a victim or very lucky or some combination of the two in this.
Who votes for storyboarding one of our scripts?
OK, who votes-- OK, so fine.
So it sounds like you want to storyboard one of your scripts
They might not raise their hand for that, either.
We've got a quiet crew
All right, who wants to storyboard one of their scripts?
Oh, come on, guys.
This is absurd
[INAUDIBLE]
I find mine is trivial, but the one I showed you yesterday-- is that--
Oh, you mean the hotel one
Yeah, because it's shot there
Yeah, do you have-- should we just storyboard the latest script that you uploaded to the Tumblr?
It's the one I showed you yesterday
So he's got an idea that he hasn't fully scripted yet
Oh, OK. Yeah
It might be easier if we have actual, like, text to a line
Yeah, we need a script for sure
Just for logistical purposes.
Does anyone have-- it's not going to be your entire script.
It's just going to be a portion, a couple scenes from it.
Does anyone feel comfortable?
And it's OK if you end up changing it completely, too
Could we do enough of one that you've already done?
Because if mine has to be-- I'm just thinking, from a learning perspective, that it might be really interesting also to match that with--what you are-- actually ended up doing?
Yeah, and I thought that, too.
That was kind of what we were planning to do.
But I didn't want to make it all about us.
But if you think it's instructive, then--
I think that might be nice to have that framework for everyone, since everyone else is copying off of it as well
OK, cool.
So I do have a script here
Do you guys all have pencils?
Or should I grab some--
Do you want them to do pen or pencil?
It doesn't matter.
It doesn't matter.
Pencil's nice because you can erase it.
Yeah, so I think what we'll do is we'll start off with John working.
And then we'll all chime in.
And then you can see how John works.
And then you can do it yourself.
Does that make sense?
Yourselves.
I have the script, actually.
So you can see what I was hoping we do.
This is a video, another one of those WNET episodes on mass and force.
I hope I printed out enough of these.
I may not have
We can share
OK.
Some people are going to have to share, unfortunately.
OK, so John-- we use Flash a lot, just for illustration and for storyboarding and for some animation components.
I won't go into all that.
But what John has here is a Wacom tablet.
And it allows us to just get our ideas in the computer, basically.
So I need the script
Yeah, let me help.
We can share this one
All right, so let's just focus on the first paragraph.
And in order for this work, please chime in and participate in this process because what'll end up happening is we'll end up with exactly the same thing we already did.
And I don't want to do that.
"Mass is a measure of an object's ability to resist being accelerated by force.
That means the more massive an object is, the harder it is to make it move if it's at rest or to stop it if it's moving."
So thinking about the need to make the beginning of the video very interesting and to immediately engage the viewer, how do we want to start this one?
Maybe somebody being pulled by a force and they're trying really hard to resist?
OK. "Mass is a measure of an object."
So we've got an object.
What kind of object do we want?
Well, a fat man--
Huh?
A fat man
A fat man.
Something big.
Something iconic.
Something that really symbolizes mass
A building, a monument or something?
A building, a monument.
How about-- what were you gonna say, Josh?
I was thinking of just the but in some kind of like stone, like a big stone that says "mass."
OK, that could work
And it falls down in the dust

A brass weight
A brass weight, yeah
An anvil?
An anvil-- those are iconic things, yeah
A one-ton thing that gets dropped on someone's head
Yeah, that would be the Looney Tunes approach for sure.
It OK, so what do you think, John?
Should we go with something big that falls?
Is a good-- do we need the word "mass" on the screen?
Not really

Something big that falls
Stick with beer
So beer could work
Why don't we do the-- let's go with-- let's try out this idea of the word "mass" being made out of stone.
That's kind of interesting.
It falls and, like, crumbles, like, creates this sort of cloud of dust and stuff.
That sounds good
I have a question
Yeah
How many is number of storyboards do you need for every one of your ideas?
Like, to what detail do we need to go?
Yeah, that's a good question.
What you think about that, John?
I think that's up to you.
I mean, I'm a very visual person.
So for me, I'd like to draw maybe three or four frames of this mass dropping down.
But for you, maybe you could just make one drawing.
And next to the storyboard, explain the action.
Explain what's going on
Yeah.
A lot of times, we draw a red arrows to indicate movement.
So you could draw the word "mass" at the bottom of the screen with the clouds and have a red arrow facing down
I mean, there's no right answer.
This is really for your benefit or the benefit of whoever's gonna be filming it for you
Yeah
So we definitely go the more rudimentary route in general
Yeah
But whatever would make you feel most comfortable.
I think a rule of thumb-- like, a bare minimum-- is that any time you're changing the scene, like you're changing the camera angle or you're changing the location, you definitely want to do another board for that
And John is drawing in, like, just a horizon here.
So you have a sense of the scene, just at a very basic level, where the background is going to be and where the word is going to land.
And I think that's enough for the first line
And this is just the beginning, too.
A lot times with our storyboards, we'll just draw out our ideas.
And then we'll come to the end of the whole story board and take a look at it.
And we'll say, well, we have this ending.
But it doesn't really match with the beginning.
So then we go back and maybe we change the beginning to more match the end.
We're just getting all our ideas out.
But you're not locked into this idea now that we've drawn it
Right, OK.
So, next line-- "That means the more massive an object is, the harder it is to make it move if it's at rest or to stop it if it's moving."
And that-- yeah, so what do you think about that one?
We've got this object that's fallen onto the screen and now we're talking about moving it and stopping it
I think of a person who's, like, kind of doing, like, the old pushing against it
Yeah
And their legs are moving, but they're not going anywhere
I think that's a natural, very, very iconic way of doing that.
So we have a character now who comes in.
And, you know, like this-- "oh god!"
-- is trying to push this thing
I think that, for the second portion, when the thing slides and the mass will come sliding off the guy, he has to try stop it by [INAUDIBLE].
So he's, like, pinned to the wall by the mass
OK. What if this little guy tries to push it and then maybe an elephant comes on the other side of the word and pushes it-- and is effective at pushing it towards him?
And then he has to, like, go this to get it to-- he's like, oh my god.
And he can't stop it.
And he gets pushed off the screen.
Something like that, would that--
How about the screen tilts?
Oh, that's a good idea
I like that, yeah
So then it's-- gravity is working on the word, then.
That could work
I really like that idea
Yeah, yeah.
And he tries to get out of the way.
OK, great.
So that covers the first paragraph for me.
Are you guys-- do you think we're-- this is a good start?
So let's start with the next sentence.
Are you all set?
OK. "The nifty thing about mass is that it doesn't necessarily have to do with size.
Think of a big pile of rocks and a big pile of feathers.
Both piles are the same size or volume.
But it would take a lot more effort or force to move the pile of rocks than it would to move the pile of feathers.
The reason?
Rocks have more mass than feathers because they're more dense."
All right, so how do we tackle this one?
You could, like, camera this big mass thing and just [INAUDIBLE] into a bunch of rocks
OK, cool.
Maybe--
I don't know if you want to go away from the mass thing, but if you want to keep it, then just-- I don't know
That's nice-- I like the idea of making it cohesive, like it's all-- it's all working together.
So maybe the mass slides down the ramp and hits something and breaks apart into a pile of rocks.
So there's a pile of rocks on one side of the screen
The birds
What's that?
And it hits the birds, and then you get the feathers
Oh, that's a good idea.
That's an interesting idea
And you have a pile of feathers and a pile of rocks
Oh, so this character is a bird?
Yeah
I like that idea
Right?
Now we've got organically our rocks and our feathers
Yeah, so--
Oh, I have a clarification question.
So, in the script, is it that the guy is running from the mass?
Or should he push back on the mass when he starts to fall?
Oh, I guess he's doing it on the first frame
He's pushing against it.
And then it starts-- then things tilt.
And it starts to slide forward
And then he gives up because--
Yeah, but is somebody suggesting that this guy's a bird?
Yes
OK, so how are we going to do this in a kind of fun, nonviolent way, that this bird, like--
Tell us that he had feathers, right?
If we're not gonna kill a bird, like, there's pillows, right?
Yeah
How else could you get a big pile of feathers?
Well, I mean they could just sort of divinely appear, like they could just fall from the top of the screen
Someone could walk on screen with a wheelbarrow full of rock, dump it, and then carry on a bag of feathers that's about the same volume
Yeah, yeah.
And let me back up for a second, because let's think about this in terms of-- So this first scene-- obviously, we're talking about animation here.
Like, we're not going to do this live action, right?
But let's imagine that the next scene-- the second paragraph-- is live action.
So then we're talking about more of like an idea like you have, where you have some actual people moving rocks around.
And you have maybe somebody dumping out feathers.
Is that what we want to do-- is kind of a combo here?
I know I may be--
Well, maybe bringing it to [INAUDIBLE] the mass where it falls into his hand, crushes it.
And he can see, maybe it's like he puts it ont he table and [INAUDIBLE]
Oh, OK, so you're talking about a nice transition between-- that could be interesting
Or you could do overlay to transition--
Yeah, with the overlay
That person in the middle and the mass falls down next to him or her and creates a rock pile.
And then--
Yeah, so then we have a--
And then feathers pop up or something
Yeah, so then we have a host at the-- throughout-- we start with a host, a talking head.
And then we have the mass fall down beside him.
You see how this gets complicated, right?
There's lots of different ways to do things.
Well, for the sake of simplicity, let's continue as if we're just going to animate this video, OK?
Is that OK with you, Elizabeth?
Oh, absolutely not.
No

Of course it should be that way
Just so that we-- you guys get the sense of the process.
But you can already see that there are so many options.
There's so many different ways to do things.
OK, so back to our feathers.
"The nifty thing mass is that it doesn't necessarily have to do with size."
All right, so we've got this mass that's rolled down the hill.
And I like the idea of hitting something and turning into a pile of rocks.
Are we going to go with that idea?
And maybe it goes off a cliff?
OK, yeah, it goes off a cliff
It could hit a pillow factory.
You could have a--
A pillow factory.
[LAUGHTER] That's funny.
Yeah, that's funny.
All right, so here we have our cliff.
And this action will cover the nifty thing about mass.
And then, down in the valley-- or the canyon-- is the Acme Pillow Company?
I've seen this video.
So it's really interesting.
Yeah, it's different, this one.
I really like this one
Pillows 'R' Us?
That's
Fun
So we've got to do-- think of a big pile of rocks and a big pile of feathers.
Boom!
So then maybe we cut--
Then we cut to a pile of rocks and a pile of feathers.
So this is a jump cut to a close-up?
Just imagine these piles
"Both piles are the same size or volume.
But it would take a lot more effort-- or force-- to move the pile of rocks than it would to move the pile of feathers."
So maybe our protagonist, our little guy who was holding the mass thing, needs to come back in here.
What should he do now?
When [INAUDIBLE]
Yeah, we could do it that way.
When is--
Could he try to clean it up, with a broom or something?
Potentially, yeah.
Yeah.
Or maybe he-- well, any other ideas?
If you want, you could add this-- you could take away with, like, a leafblower.
Something that just blows off and thunders off the screen
Yeah--
--blow the rocks
That could work, yeah
By adding onto that, maybe he tries to blow the rocks and nothing happens.
And he kicks it.
And he injures his own toe
Yeah.
So what do we have?
A leaf blower, we have a cleaning up scenario.
You want to try the leaf blower?
Sure
Or anything else?
Is there any way we can connect this with the beginning?
In the beginning, he's trying to move this word, "mass."
Is there anything there?
Is he, maybe, the janitor?
And this mass is in his way.
He doesn't like it there.
And then he's trying to do the same thing here.
He's trying to clean up this mess.
I don't know, just spitballing here.
Is there a way we can tie the two things together is the question
Yeah
I don't know.
I was thinking that he's running-- because he's running away from the mass.
So he runs along with the mass.
And he crashes and booms.
And he lands in the feathers.
He's like getting away from the feathers.
This clears the feathers.
Then he tries to get it off
That's interesting.
So it's what does it say here?
"Think of a big pile of rocks and big pile of feathers.
Both files are the same size or volume.
But it would take a lot more effort-- or force-- to move the pile of rocks than it would to move the pile of feathers."
So I think we're back to a scenario where we need to have some sort of-- we need to show force being exerted.
So "It would take a lot more effort-- or force-- to move the pile of rocks."
So what sort of-- how can we show that?
We almost already showed it in a way
Maybe a cleaning up crew trying to move them off the frame?
Yeah, or maybe just our character trying to push this pile of rocks again.
It's like, oh man, not again.
I got to push something else
I'm wondering if he should have a backpack, so he could--
What's that?
We're heading towards having a student, right?
Yeah, yeah
So maybe he's our student
Right, right
And that we identify with the beginning as some thing that makes him obviously a student
He's on his way to school or something.
And this adventure happens
A kid would obviously, like--
Yeah, yeah.
And the audience is kids.
This is middle schoolers.
So we could add a note to that first scene, like, kid with backpack.
So we agree that-- so this kid is going to try to move these rocks in some way?
He's trying to push them or gets his back up against them and tries to get them out of the way?
I like the idea of there's a clean-up crew.
So there could be two different kinds of [INAUDIBLE] on the right.
Here are the feathers.
Or someone sweeping-- one of them.
And the rocks would be just like this construction truck-- beep, beep, beep.
Then he comes in and picks it up
Yeah.
Does that really show the notion of differing amounts of force being required?
That's a good question
Maybe less likely
Yeah
Maybe you use the heavy machinery to move the rocks.
And then we just move our little character.
And he can move the feathers
Yeah, that could work.
Like, a big backhoe has to move the rocks.
But the character is able to just either blow on the feathers or move them out of the way really easily
Can you see a little berm here?
OK, yeah, that's nice.
That work?
Does that "sell it," as we say?
So we need to show that the-- do we need to show the backhoe first?
"It would take a lot more effort-- or force-- to move the pile of rocks."
And then we have another shot-- "than it would to move the pile of feathers."
We see the rocks gone in the next scene
Right here they're gone?
Yeah.
So he gets through-- so the feathers are gone.
And then we've got another line here.
"The reason rocks have more mass than feathers because they're more dense."
Maybe you could, like, have a scale with, like, the backhoe with rocks on it.
And you've got feathers on it or something.
[INAUDIBLE]
Is it-- yeah
That's why I was, like, it's not a type of scale
Right, right.
I'm not a scientist-- and you guys can help me with this-- but does density really equate to weight?
If we just show them on a scale, does that really show that one is more dense than the other?
Yeah, I guess not
I mean, it's indirect.
So maybe a better way would be to address density more directly?
Yeah.
Could we do something, like-- just very simple, where we just-- the character extends two hands.
In one hand is a rock.
And in the other is a feather.
And we just-- and we put the words more dense above the hand with the rock in it or something like that?
How do you visually say something is more dense?
Like, how do you show that visually?
The tractor, or whatever that heavy machine was, it could be throwing the rocks off into the ocean.
And you see the rocks sinking.
And then the feathers are just floating in the air, maybe just landing on the surface
That's an interesting idea
It definitely shows that.
And I'm not sure how familiar the kids will be with the concept of density.
But having the water-- sinking, floating, maybe-- makes the point a bit better
You already introduced volume.
So I feel like to introduce density wouldn't really be doing, like, what came before, which is volume.
I don't think it would do it justice.
Because density is volume and mass.
So if you're able to have volume and then go to mass after that, I think that's where they'd connect, that the mass could be different even though the volume is the same.
And you would still have different densities
I mean, the other thing is that the video isn't really about density.
The video is about mass and acceleration
Yeah
So it may not even be-- and, like, this is sacrilege-- but it may not even be that important to really define what density is in this particular video, which is why maybe it's OK to say one's more dense than the other.
It's like, a couple minute video?
Yeah
The point is really about how things that have more mass require more force.
If you start going into density, I'm afraid that people would also want to talk about what the difference between weight and mass are, too
Yeah, true
So again, it's like, at what point are you branching off too much from the point of the video and the main point of the story?
That's a good point, yeah.
Well, "The reason rocks have more mass than feathers because they're more dense."
Yeah, so maybe-- I don't remember what we did in the video we actually ended up producing.
I think I would go with the hand displaying the two things and think about how they're essentially the same size.
But one is very light and one is this rock.
It's dense
Can you-- is it possible to zoom in on the rock?
Because density is really that there's more stuff per space, right?
So somehow have the rock bulge a little bit to show that frame is like stuffed with as much stuff as possible
Yeah
And then show the feathers and show like a lot more air and space in between the feathers.
And you can still put the label "more dense."
But maybe there'd be some sort of visual cue associated with that as well
Yeah, potentially.
This is a toughie
So you're saying, maybe, we cut to a new shot where we see a rock and a feather side by side?
And then maybe we see like, a view of each one under a microscope?
Almost, but again, I don't know if that's distracting too much from the main point of the story
Well, I like that idea-- the microscope image.
Like, we show like the crystalline structure of the rock is really, it's like a lot of stuff packed in there
What if he was poking?
So you can't poke a rock but maybe he takes out a pencil and pokes through the feather.
We can do that [INAUDIBLE]
I think it's too complicated to go with this concept.
Like, I don't know.
The word "dense," like, I don't think we have to-- I don't know if we have to explain that right now
Yeah, well, let's think of another strategy here.
How necessary do you think this line is in the script?
I mean, is it vital to the script itself, I mean, to the whole story?
I don't know, I haven't given you guys a chance to read the whole thing.
But maybe this is one of those moments where we take this line out.
And we end the paragraph with moving the feathers
Well, it's far easier
Yeah, it is easier.
And it might be one of those moments where this line is a distraction from the-- I think that's where we're struggling right now
Yeah, yeah.
So let's say, OK, we're going to remove this line for now.
And some client is gonna come to you and say, well, why did you guys take that line out?
We really need that.
And then you have to argue for that.
That's all part of the process, right?
Like, well, we talked about this for about an hour.
And we realized that this wasn't really important to the process.
You know, and that--
How often do you feel like you have to, like, you have an artistic vision of what direction it should go?
It happens all the time.
Yeah,
And then you have to comment?
In my professional opinion-- and that's when I have to sort of fake this professional persona-- we don't think you should have this line in the script.
And in retrospect, this line is in the video we-- but we didn't have 15 people to talk about it with.
We only had four
But do you see how-- like, are people OK with taking that line out now?
Like, you see why maybe that's a good choice?
Would you have reacted that positively to the idea of taking this out maybe five days ago, before we had gone through this class?
Because I feel that is the reaction of a lot of scientists.
Like, wait, wait, wait-- this is accurate, but it's not precise, like you were just talking about.
You have to talk about density.
And that's sort of-- I mean, I want to tie this back to the work that you guys are doing now.
You guys see how this process is really making you examine what is truly necessary and what isn't to the project.
Because this is going to happen to your scripts, too.
You're gonna sit down and storyboard.
And you're gonna rack your brain over one sentence that you can't think of an appropriate visual for.
And maybe that's a key for you to reconsider whether or not that part is truly vital.
But you are not forced to do that until you think about things like storyboarding and think about things like an overall narrative to your video.
Because otherwise you're going to do what most scientists do.
And I-- I mean, I say this as a former amateur would.
But the tendency is to just throw as much information out there to cover your grounds, right?
Yeah, and I think-- if I remember correctly-- this script was a lot more complicated.
We received a kind of rough script from the client.
And I had to edit it down and get it into something that worked for the medium.
And we had some discussions about well, like, why can't we say that?
I mean, exactly what you're talking about happened.
And I think that this one just sort of slipped through.
This one got through.
And, in retrospect, maybe we shouldn't have produced that line.
But I love that we're thinking about this.
I think this is a really cool-- this is a really important part of the process
This goes back to the core concept.
I mean, the core, number one thing is know your audience.
Think about your audience.
I was actually earlier today helping one of my graduate fellows apply for the master's in journalism program here.
And she has to write a 500-word piece on a science topic.
And she was trying her hardest to consolidate an experiment.
And she was trying to think about what the lay audience cares about.
And it was so hard for her.
And she distilled it down to two and a half sentences.
But I told her even that was too much for a New York Times piece, that no one cares about which mice were chosen and why for this experiment.
And then-- that this is the crux of, like, what I think is one of the hardest things
Right.
And there was-- I think on Monday we heard this phrase, "kill your darlings."
There's that--
And that'll come back [INAUDIBLE].
It's my favorite
Yeah, exactly.
And I think, in retrospect, introducing the notion of density into this is a real distraction.
It's not what this is about.
So yeah, I would say let's strike that line-- that we have the luxury of doing that.
So I want to stop there with our portion of it.
And then kind of let you guys break up into groups and do maybe the next two paragraphs.
If you make it further, that's fine.
And then we'll walk around and to you about your ideas.
But-- so how many of us are there?
1-2-- there's seven?
There are eight
Eight?
OK, so that is nice-- two groups of four.
Yeah, I'll let you divide up however you want.
If you just want to go with you four over here and then you four over here is fine.
You guys from this-- are you guys, I would say not all three of you guys together because you know each other and all that.
But yeah, that's let's get started with that
And if you guys need to take a break to go to the bathroom, this is a wonderful opportunity to do that since we're--
That's a good idea.
Yeah, yeah.
Let's take a break before you start.
Thank you
--the student just standing there.
So you could have a teacher come in
Second teacher--
Yeah.
And I guess they sit down and it's like--
You could actually use Elizabeth and me.
That would be--
-gotten behind a word that you're like, oh, that's why we use that word, right?
I don't know
They sort of like blow up
There were people--
--stuff them inside their shirt.
And now they're really big
These people write a press release about a product that hasn't been invented yet.
So what do you guys got?
Is this thing on?
Sorry.
Hot mic.
All right
So we've, for the third paragraph-- what it takes to show how-- "Picture a student and a teacher."
So we have a student and a teacher.
So the student is like, really happy.
And then the teacher is like-- [MAKING A FROWNING FACE] Plus he has to do this exercise with you, I guess probably to find [INAUDIBLE] And then he zooms out.
And you see him there, sitting on a chair.
So then you show this shot with him sitting on a chair.
And I haven't drawn the last one here, but, it's supposed to show from, like, the corner angle where you see the student being very small.
And then the teacher will be, like, massive to show the difference in this
So you do the classic thing where you have an above shot for the student and a-- from the bottom for the teacher
Yeah
And they're sitting in the chairs
Yes
OK, cool
I guess that stopped with the third paragraph
Yeah, I mean that's enough.
I mean it sounds like you guys went further than that
Yeah, we went further than that
OK, go ahead and share it.
I'd be curious to see
Where are my-- Ah!
So then he-- so he just pushes him off.
I guess he wants a little bit of comical-- so you can see from the side.
But because he does rolling chairs with this and things, so the kid will go "whee."
Now there's physical distance.
But there's this little line they put over to show there's a difference between [INAUDIBLE]
OK, cool.
I like where that's going.
That sounds cool.
Did you guys have any thoughts or feelings about the process, like-- does this give you a sense of what it's really like to do this stuff and have to run these scenarios through your brain and visualize them?
I think, like, the big thing we found was you have this text but, like, knowing how to read it.
So like, we just-- with that first student teacher-- we read it like, "picture a student and teacher."
Like that's just-- if we have a frame of a student [INAUDIBLE] a teacher, that would slow that down.
So we had to be like, picture a student.
Now picture a teacher
Right, right.
So you're thinking about directorial sort of things, like how you're going to-- how the voiceover is going to change as a result of these visual decisions you're making.
Yeah, that's cool.
You guys?
It sounded like there was less of a hum going on over here.
Did you guys-- what do you have for that third paragraph?
Well, basically we have the kid that's standing there from the last part, you know, where the feathers have been swept away.
And the teacher walks up with two rolling chairs
OK, cool
They sit down.
There's a poof of smoke.
And that's where it transitions to live action
OK, I like that.
That's a cool idea
And the teacher grows also.
That's how we're sure she's massive
Oh, the teacher grows.
In live-- How are you going to grow the teacher?
We stuff her with pillows
Oh, that's funny.
So, like, maybe a stop action, where you take a frame.
And then you put one more pillow in her.
You take another frame.
And you get like-- and then you do between frames.
And you've got, like, maybe 30 or 60 frames of the student-- of this teacher getting bigger and bigger.
That could be really funny.
Good idea.
Any thoughts about the process?
What was the hardest part?
[INAUDIBLE]?
How to make it visually engaging, I guess.
So have enough interesting shots but not too much
Yeah, true.
It's like that fine line, right?
You don't want to overdo it.
You want to-- it really, really boils down to this kind of empathy for the viewer, for their ability to understand things.
And maintaining this balance between your desire to have them engaged and their need to understand-- and kind of threading that line
We'll talk about this a little bit more during our editing lecture.
And Chris will talk about it tomorrow.
But visual pacing is another thing that storyboarding can show you.
So again, I'm not, like, balancing between the sides.
If you have so many shots that are just changing too quickly, then that can be a very nauseating experience for the viewer.
And I'm gonna show you guys a rough cut of one of the videos that we made where that happened.
And it just felt too rapid.
So we slowed it down
OK, yeah, that's really good point.
Two last really small things.
We didn't get to talk about After Effects.
But it's essentially a piece of software that lets you layer video and artwork on top of each other in time.
And I'm sure Elizabeth will talk a little bit about it.
But if you want to email me, and you have a specific thing you want to animate, and you want some advice about how to do it, feel free.
You can come by the studio and say hey, I've got this problem.
How would I animate this?
We can give you some tips about how you would go about doing it
And just so you guys know, again, you have free access to Lynda.
But there's a computer lab next to 26100, the new media center.
And all the Macs are outfitted with FinalCutPro, After Effects, the entire Adobe creative suite.
And you guys have access to it anytime.
So if you're interested in playing with that software, or going on a media experience, let me know
Is After Effects different from IS&T?
It is if you're installing it on a machine at MIT homes.
I can ask about maybe getting a license for you guys just for this class
You can also just sign up on Adobe and get a month's free trial of all the Adobe Creative Suite things.
So you can just sign up with any email and download the program suite
And then the last thing I wanted to do is to show you the video that we actually ended up producing for this script.
So here we go.
[VIDEO PLAYBACK] -Mass a measure of an object's ability to resist being accelerated by a force.
That means the more massive an object is, the harder it is to make it move if it's at rest or to stop it if it's moving.
The nifty thing about mass is that doesn't necessarily have to do with size.
Think of a big pile of rocks and a big pile of feathers.
Both piles are the same size, or volume.
But it would take a lot more effort-- or force-- to move the pile of rocks than it would to move the pile of feathers.
The reason?
Rocks have more mass than feathers because they're more dense.
To show how it's harder to accelerate something with more mass, picture a student and a teacher each sitting at rest in a rolling chair facing each other.
Let's also say the teacher is more massive than the student.
When they both push off each other, they go rolling backwards.
Notice how the student goes back much farther and faster than the teacher.
The teacher accelerates less, since she is the more massive of the two objects.
And it takes more force to accelerate her than it does the student.
All of this has to do with Newton's Second Law of Motion-- f equals ma.
It says, for a set acceleration, an object with more mass requires more force to get it moving at that set acceleration.
As mass increases, the force required to accelerate or decelerate that mass increases as well.
Now don't you feel massively smarter?
-Mm-hmm.
Whoa!
[END PLAYBACK]
All right, so that's what we ended up doing.
So you can see how the timing allowed us to sort of cheat a few lines, like the density line.
We just sort of let it play through.
So there's some moments here where you're allowing the viewer to take in what's going on.
There's not constant action going on, you'll notice.
But the action is at key moments where we need to keep things-- This is so abstract.
But again, there are no right answers.
But there some-- if you do it long enough, you'll sort of start to get a sense of what decisions to make.
Yeah, question
How long did that video take start to finish?
Like, from storyboarding
I would say, John, you can help me remember all this.
But you know, the script was maybe a couple days, revisions and so forth.
And then we probably sat down for at least four to six hours-- almost a whole day-- storyboarding it, every scene.
It was a complex thing.
We're not physicists.
So we had to understand this stuff.
We had to do some background research.
About a day storyboarding, and then about a total of three people doing the animation and illustration.
And that took maybe three weeks.
So I'd say about a month to do the whole thing
Real quick, because you guys are going to stay on for another class?
Sure, I can do that
I mean, if you need to go, that's fine
That's OK. We can hang out
So thanks guys, thanks for listening
Thank you.
[APPLAUSE]
Josh gave you guys all his info
Yeah, feel free to get in touch
For today's assignment, you guys will be storyboarding one scene from your script, which is why it'll be a good idea to have some semblance of a script together.
I know that you're still revising all that stuff, too, which is why we're only having you storyboard one scene.
And then tomorrow, Chris will help you shoot that scene.
So as far as the storyboard goes, you can use these sheets, you can use PowerPoint, you can use note cards.
But just upload them as images onto Tumblr.
And then all the assignment details are on the syllabus.
I wanted to mention-- this was something that I was going to show you guys during the editing portion.
But you can actually storyboard in post-production after you've shot things.
So this was an episode on snot that I'd filmed with George.
And basically, after we got all of the raw footage from the first takes, these are sort of like stills from each major scene of the video.
There was a portion where I did a demo of this, like, fake snot that I made.
But I had filmed it in Katharina Ribbeck's lab and the sun had set.
And my delivery was terrible.
I was super tired that night.
So there were a couple things that jumped out to us after we watched everything.
One being that I am lit very dramatically.
And I wanted to have more of a casual feel to my parts.
And I felt like we should only dramatically light Katharina Ribbeck.
The other was that-- I thought, well, how bad was that decision really?
Like, is it really worth shooting again?
So we shot it again because we had the time.
And then it was a matter of deciding which set of footage to use.
So what we're doing right now is we just put together a cut of the video using both clips.
But another thing that you can do is storyboard the stills from the footage that you take day of.
So here I can see how that scene looks in the context of everything else we shot.
So I see that, like, I'm wearing a lot of blue.
The whole video looks fairly blue.
I look a lot more like Katharina Ribbeck in this set of footage than I do the rest of me, right?
Like the way I'm lit in all the other scenes is really different from this.
So that's what the re-shot footage looks like.
One-- the snot demo went a lot better the second time we did it.
My delivery was also a lot better.
But I'm also lit a lot more flat.
I actually think that the footage that we had before-- it looks a lot better, like the lighting is a lot more flattering.
But it's a lot more formal and traditional.
And it's hard to know how much that jumps out in the context of the rest of your video until you line up all these stills and you sort of storyboard this way.
So we haven't actually finished this video.
I'm in the process right now.
Like, this is exactly where I am with this video.
I thought I would show it to you today since we've been talking about storyboarding to show how you can do it not just in pre-production, but also after you've shot material.
So we have about 20 minutes left in class.
I thought that it would be good to give you guys a chance to get started on storyboarding your scene while we're around.
So if you guys have questions, I'll be here until the end of class and a little bit later.
You can ask Josh as long as he's around.
But does anyone have any questions right now about what they need to do tonight, about storyboarding, writing?
How are your scripts, like, what shape are they in?
Are you happy with them?
Do they still need more work?
Yeah, we can also talk about scripts right now, too, because take advantage of Josh's background in writing as well
How long are the scripts?
What are you shooting for, three minutes, two minutes?
Like three to four minutes.
Five is the absolute max
We do 150 words per minute.
I don't know if that's something you guys have already talked about
How much is one scene?
So what does that entail?
Well, I think of scenes as sort of locations almost.
And so let me go back to-- sorry, my lap-- bye, Jamie.
A scene kind of encompasses a general statement.
So that the way I broke up the storyboard is I basically just did stills of each scene.
You can kind of consider the whole top row as, like, a general scene, one which is how I originally scripted it.
It was just sort of like the visual introduction to the video.
But breaking it up into scenes is nice, because then you'll notice if one scene has a lot of text.
And then you have a scene right afterwards that's very short.
And you just have a series of really quick scenes.
Maybe you can think about restructuring or spreading out the material a little bit more.
Because, like I said, what you don't want is either really long shots or really long scenes where you're not really changing that much, or if you're doing too many, really quick changes.
But I don't know.
I don't have a good definition for what a scene technically is
It's kind of a unit of information.
When I think about writing, I think about paragraphs.
This is kind of a discrete unit that's imparting a certain part of the narrative.
And that's-- for me, that's a scene
Yeah.
I think the way I'd scripted it, those four shots are all different locations.
But I listed it as scene one.
Like, where you see Professor Ribbeck and then I'm right there in front of her, I also chunk that together as one scene.
But then, when we were editing and sort of taking note of all the clips, we denoted one as like scene five and then the one after as scene five A.
But, in general, like it's the same sort of idea.
We're in her lab in that portion.
Does anyone have any other questions?
What was, like, the motivation in using professor to--
So, in this particular one, I wanted to highlight the research that was going on here.
And I also wanted to get her involved because I knew her.
And I really like her.
And it was sort of like a quirky breaking down the fourth wall moment.
So what happens in this scene is-- I set it up in the previous scene as saying snot is more than just this.
And then it jumps to her.
And she finishes my sentence.
And she goes, it's actually xyz.
And then it cuts to a wider shot, revealing that I'm in front of her, interviewing.
And you see the camera interviewing her.
And I'm, like-- [GESTURING TOWARDS PROFESSOR] And then I go, she would know.
And then her name sort of pops up.
Partially because we wanted to have her in the video, but we didn't want it to be really formal.
Because that shot is a very traditional formal interview.
So we wanted to make it quirky.
And we also wanted to sort of like informalize it.
So that's why we broke out of that.
And I did the whole like, she would now.
And then we could make that setup a little more fun
How do you think about the authority of the host, right?
Because you can say things that are easily verifiable.
But then if you're going to say something about snot that, like, most people don't know, then on whose authority do you say this?
I've watched like an Alton Brown thing.
And I was like, all right, that's wrong.
And like, but you are talking about all the things I want to know.
And now I don't know when you're lying to me
Yeah, well, I mean, part of it is, that's the responsibility that you have as the host, is that the viewer has, by default, placed their trust in you.
I think with us, we're not going to have as much of a problem to prove because everyone knows that all the hosts are students at MIT.
And in our YouTube descriptions, we say that all the scripts were fact-checked by postdoc researchers.
For this, I mean, that was one of the reasons why we used her is because I could explain her research.
Or I could just have her say it, which I thought was like a cooler window into the world here, which is what I was talking on the first day.
How do you create the window into the world that people don't have access to?
I also studied bioengineering for my undergrad.
So all the material for the script came from the class I took from Professor Ribbeck.
But I think you don't take the time in your video to introduce yourself necessarily.
And maybe you should.
That's always something we've considered, is that something that we need to introduce into Science Out Loud.
The host saying at the beginning, like, I'm Jack and I work at MIT's D-lab.
And that's sort of what we did with Lindsey's episode just a tiny bit.
But there are ways of establishing authority where you don't have to vocally do it.
So in Lindsey's episode, we filmed at MIT is D-lab.
And instead of saying, like, we're at MIT's D-lab, we just spent a few seconds doing shots of the sign that said D-lab, basically.
If you guys want to see it, I can show you later.
But I don't have a good answer for that, because I think that, even if you're Stephen Hawking, people can question your authority to say the things you're saying.
And the point of the video isn't necessarily to establish that-- or at least the point of these videos.
Your material should be accurate, though
You showed us the next slide, right?
And you have this [INAUDIBLE]?
Oh, yeah.
So George made me wear the goggles, partially because I think he was trying to torture me.
But there are some environmental health safety standards.
And since we were technically in a lab, even though I wasn't working with anything biohazard related, the lab itself required goggles.
So that's why I wore gloves and a lab coat and goggles.
If you guys plan on filming in any sort of environment where that might be an issue, we'll talk about it more during producing.
But just keep that in mind, too.
We took shots of me with and without the goggles, just in case.
Any other questions?
OK, so I'll let you guys work.
And we'll just sort of circulate through and see if you need any help.
But again, I just wanted to give you some time to make the most of the resources here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Thank you all for uploading all your stuff last night.
We do watch everything, just so you know.
And it's a really, really helpful for us.
So I really appreciate you guys doing that.
I thought we'd start off today by doing just a quick exercise where you say the point of your video in one to two sentences.
And I think about this last night.
Because it's actually a really nuanced thing, understanding the difference between what the point of your video is and what your videos does.
I was trying to think of a good way to explain it last night and this morning.
And I think that might be the best way for me to understand it.
And that's why I gave that example of the snot video, right?
Because the point of the video that I made about snot wasn't to explain what snot is.
That's what my video does.
But the point of it is to show that this material that you take for granted, that's in your body, actually does all these xyz amazing things.
So does the difference between those two things make sense to you guys?
Or do you recognize the difference between those two things?
So for instance, Joshua.
Your video maybe about what a binary search is, right?
But that's not the point of your video.
That's what your video does.
In achieving the point of your video, explaining why binary search is, is sort of a happy afterthought, right?
But it's not what drives the video forward.
So if we could just go around and pitch the point of your video in one to two sentences.
And the more specific it is the better, right?
Because we can generalize things like, the point of my video is to explain how amazing snot is.
But that's actually not that different from the point of my videos to explain what snot is, right?
I've just thrown in an adjective, basically.
So does that sound OK to everyone?
And I think this will be helpful because once you really think of what the thesis of your video is it's going to help you a lot in scripting.
Because you'll always be able to go back to that big point, right?
If you're scripting with the objective of explaining what a fractal is, that's what Ulea's video is about it, it's really easy to get lost in the details.
So that's the whole point of this exercise.
Does anyone want to go first?
Otherwise we'll just go in order
I'll go first
Awesome, thanks Josh
So you were just highlighting that my video, it was about the binary search algorithm.
But I guess the theme here is why can't find my stuff?
And how can I find it there?
That's what the theme is about.
And the question is can Google teach me how to do my own search in my life better
I think it's a really good start, right?
Because we were talking about yesterday the power of questions.
But again, the point of your video isn't the question.
The point of your video is the answer to the question.
So what if you go from there and try to go for two sentences that try to answer those questions
Do you have an example?
Well, wait.
Say the questions again?
Why can't I find my stuff?
And how can I find it better?
OK.
So Josh, how can you find your stuff better?
If we keep a list of, like, where is it located?
Or better yet, if they're positioned in a way that has a relationship among them.
So it's kind of like, if you put stuff in the toilet, everything you need to use in the toilet, you put them in that location.
So you avoid putting things in places where they don't need to be there.
But you place the things where the relationship is natural
So basically what you're saying is that the way you organize objects isn't just for aesthetic purposes.
It can actually change the time and efficiency in which you find things
Yes
Which doesn't sound like a big deal until you realize that that's the fundamental way that something as huge as Google uses.
Is that correct?
Yes
Does that feel like more of a point than this is how a search engine works?
It's more of a point
That's more of a point?
That sounds like something more than you're writing down.
I mean, for me, at least, maybe.
I'm a visual person.
But, like, that's a really important concept that we just figured out.
Right?
It's like, it's a super nuanced thing.
And very difficult to figure out.
It's a lot easier to figure out someone else's point than your own, actually.
Which is why I wanted you guys to come up with a script sometime tomorrow.
It's a lot easier for me, at least, as an editor to give productive feedback when I actually have something tangible in front of me.
So your questions were a really great starting point.
And the script that you wrote is also a really great starting point.
Because there's a lot of material to work with.
But I wouldn't have been able to figure out that that was your point unless I had had those materials, right?
The questions are a super good starting point.
Oftentimes the point of your video is going to be the answer to those questions.
So the questions set up the video.
But the point of the video is not the questions
It's like the difference between a symptom and a disease from a doctor's perspective, right?
Like, that's a different way of thinking about it
Yeah
You keep asking, like, why is something itching?
Why is something itching?
And then you finally understand the core reason to why?
I think that's what you're trying to get across?
Yeah.
And I think it's a very compelling way to set up not just video, but scientific papers, talks.
Any time you're trying to sell an idea to someone, grant proposals.
Oftentimes you'll ask somebody you're going to the conference, what are you going to do?
And their answer is well, I'm going to talk about the research that I did.
And that's not really helpful.
Obviously, you're going to talk about the research you did.
What I want to know is what is your research?
I know that you're going to talk about binary search, probably from the title of your video.
What I want to know is, what is it, right?
So let's do this one more time.
Can you describe the point of your video in two sentences or less?
The way we could find things better is the way we put-- [INAUDIBLE] right now
That's a good start.
That's good.
So the way we can find things better, you're trying to relate a concept or an everyday experience that we have, like organizing things.
So the way we organize things
Yeah.
The way we find things better is the way Google could find our results better
Yeah.
Yeah.
That's a much stronger point than the questions you had initially.
Than saying the point is to explain what binary search is.
And you can flesh that out a little bit more.
But do you see where we're going with this?
Ulea, we're going around doing one to two sentence explanations of what our videos are

Let's have other people go since you just got here.
Paul, do you mind going next?
Sure.
So mine's about using principles that kids are already familiar with, like floating and sinking to show an application of navel architecture.
Specifically, how to prevent ships from sinking using subdivision and Archimedes's Law
OK.
So that is very informative.
That's, like, a perfect description of what your video's going to be.
That maybe verbatim what you put as the summary in the YouTube video.
Like, that's going to be super helpful to a teacher who's looking through the videos and wanting to find out what videos are about.
So I guess my question would be, what are the actual principles that kids learn that would help them understand the ship and the subdivision?
So kids are familiar with floating and sinking because they go through second grade or something like that
And not just kids, but everyone
Yeah, right.
Right
So you could start off "Most people are familiar with the concepts of flooding and sinking."
Right, but how you can be able to divide your ship so that it can float when it's [INAUDIBLE].
I think I'm kind of going in circles
No, no, no.
That's a great start.
And oftentimes what I personally like a lot-- and maybe this is something that we've been repeating over and over again-- but the whole concept of the unexpected.
The familiar becoming unfamiliar, as Chris was talking about.
The fact that a huge piece of technical equipment, like a ship, can be protected from sinking using just the basic principles that even second graders know is actually really amazing.
All right, so maybe that's something that you can sort of play off of in the point of your video.
That the concepts that we learn as children, like why things float and why things sink, are actually the building blocks of knowing how to protect machinery like submarines from sinking even when they get damaged.
And the wording of that obviously isn't super great
But yeah.
That an aircraft carrier, which is one of the most complicated machines ever built-- the fact that it can even float is based on this very simple principle that you learn in, you know, first grade or second grade
Yeah, I think that's a very strong point to the video, actually.
I personally love the idea of complexity from simplicity.
That the things that people often think of as being really complicated, like naval architecture or systems biology, are actually built off of really simple concepts and really simple ideas.
Does that makes sense to people?
OK. David, how about we go with you next?
The point of my video is that some people are naturally predisposed to better survive the next Ice Age
Sorry, I didn't hear-- some people are better disposed to what--
Some people are better predisposed to better survive the next Ice Age
The next Ice Age?
Yeah, and they could be predisposed by genetically or physically, or maybe they just do meditation
OK. Yeah, actually that is a point.
That's a thesis, that's an argument.
And I think that's a very interesting one.
But notice that the pitch wasn't why are some people more predisposed to this?
Or why do I feel cold but my friend doesn't?
Right?
The point is an answer to those questions.
Does anyone have feedback or suggestions on maybe ways that David could alter the point at all?
And again, no one is ever going to read your point, right?
No one is ever going to see your thesis written out before they watch your video.
The idea is that they're going to watch the video, and they're going to come to the same conclusion that you've written.
So it's OK if the wording's not perfect.
Because no one's ever going to read it.
They're just going to watch your video.
But I guess if you were going to publish it, I would open up with have things like genetic makeup or physiological differences among people are what might predispose certain humans to survive the next Ice Age.
Kenneth, do you want to go next?
Sure.
The more succinct point I can think about is letting them know that time travel right now is not doable yet.
Yeah, that's the most succinct point I can think of.
If you want me to elaborate a bit more on that, it's to let you know that we can possibly travel forward in time faster.
But traveling back in time requires too much energy
Too much energy.
OK. , Again nice distilled point.
I would say that you could flesh that out a little bit more for the point.
Because that's, like, time travel is not possible is the ultimate argument that you're making.
But it's a super broad one.
So you can include maybe why it's not possible.
Again, because it takes too much energy.
I don't exactly know what that means.
But maybe your video is going to cover that?
OK. Andrea, do you feel comfortable going?
I'll throw something out there.
Even though I think it's more what it does
That's OK.
But see, when you throw that out there, it makes it a lot easier for us to give feedback.
It gives us something to work off of
So the video illustrates how engineers in medicine use natural processes in our bodies to actually repair or improve the human body
So I actually think the point is in there if you just take out the first three words.
Engineers and-- what was it-- scientists use natural processes in our bodies to what?
Enhance it.
To improve or repair
Yeah, I actually think the point is in there if you just get rid of the "my video explains."
Which is kind of awesome, considering where she started two days ago, right?
Like, the braces, you totally done the [INAUDIBLE].
It's awesome
That's what you asked me to do
Yeah, it was awesome.
I'm saying it was awesome!
Way to go
Nathan, you want to go?
Yeah.
We aren't the only ones who eat our food.
And life without them would be a lot different
That's pretty good!
That's pretty good!
Nathan had an aha moment last night, if you heard his reflection.
And I'm really excited to see where you go with that.
But very, very different point and answer to my question than the one you gave, like, yesterday even.
So I asked him what the point of his video was yesterday.
And he was like, to explain what decomposition is.
But do you see how this is, like, a completely different point?
And a much more engaging one?
And in explaining and getting to that point, he is going to cover what decomposition is.
All right, Ulea
Mine is repeating patterns called fractals are present in every facet of our existence and are indispensable in nature, research, medicine, and technology.
In that way, mathematics helps us explore and understand the world around us
Nice.
Nice.
Any feedback on that one?
Can you repeat it?
Yeah.
Repeating patterns called fractals are present in every facet of our existence and are indispensable in nature, research, medicine, and technology.
In that way, math helps us explore and understand the world around us
That's also a super different point than the one you had given yesterday, right?
Yesterday the point of your video was-- it wasn't even like, what is a fractal?
It was like, what is the philosophy of math, right?
No, it was was math even real, right?
Yeah, that was two days, yeah
Yeah, but you've come so far in narrowing this concept, it's awesome
Yeah, so hopefully with these points it's going to help you script a tighter set of text.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So I'm going to try to run through things pretty quickly, and if you don't have any sort of video experience or still photography experience, a lot of this will sound foreign to you.
If you do, some of it will be very familiar.
But either way, if what I'm saying is confusing because of the density of it, I would just sort of encourage you not to be too freaked out.
Because the takeaways are actually fairly simple.
It's kind of like-- again, there's a lot of complicated technology and a lot of complicated technique, but the base is pretty simple.
And what I'm going to do-- partly what I want to do today is just expose you to some concepts and terms that you actually may not have to really fully grasp or have mastery of in order to make your video.
But it's worth it to at least kind of know what they are I think
And I can condense my [INAUDIBLE]
Yeah.
But I do want to-- I'm very cognizant of not wanting to spend more than like 30 minutes.
So can you tell me-- if I'm like maybe 10 to 2:00, just kind of give me a wave or something.
And I'll condense even more.
OK.
So visual language, there's sort of a question here-- what do we mean by visual language?
Because it's kind of a contradiction in terms.
Sometimes we think of language and the visual as being at odds with each other.
We have text.
We have speech.
And we talk about, oh, well, we've got to make a visual.
So what is a visual language?
It goes back to something that I talked about in the first class.
And in some ways what I'm going to do today is try to build on some of those things as I sort of race through them.
We have all of these plastic elements of film that we can manipulate.
And today I'm going to focus on a handful of them, in particular camera.
Because the camera is kind of the speech device that allows us to speak in this visual language.
And Elizabeth made a really interesting point, or quoted a very interesting kind of way of thinking about it, which was all of us understand this language better than any generation that's ever lived, Because we just have consumed it our entire lives.
But that doesn't mean we can speak it, and understanding that is a great first step in speaking it.
But you really have to kind of unpack it and think about it consciously in order to be able to kind of reproduce it or change it and make it what you want it to be.
Anyway, so I'm going to race through some of these fairly quickly, focused mostly on camera, and I'm going to dispense with sound right away.
Because I just want to say about sound-- don't dispense with it right away.
Don't do what I'm doing.
It is super, super important.
I think I mentioned this the other day, getting good sound is absolutely critical.
And sometimes sound can be more important than pitch and all this stuff I'm going to talk about.
It's also sort of the boring part sometimes.
Well, you know, is the microphone close enough to me?
I don't know.
Who's listening to me?
Am I OK?
Good.
All right, because it's really important.
Because if you don't hear me-- or even if you can sort of hear me, but it's a camera mic and it's six feet away, there's a huge barrier between your audience and your material.
So do not skimp on sound.
The environmental sound is important.
It's important to be in a quiet room.
It's important to have good mic placement.
It's important not to have a signal be too soft or too loud.
Listen with your ears.
And then creatively you can do so much with sound too with music and sound effects.
But I don't really have time to talk about that stuff
I'll go over that stuff
Yeah.
Yeah.
OK, good.
I just want to sort of make the pitch to not forget about sound in all of this.
But let's go on and talk about camera.
And I don't know if there are any theater people here, or people who have done theater, but I want to start with this concept of the proscenium.
Film and video, film at least, has been around for over 100 years-- 125 years or something like that.
The sort of first Edison experiments or the Lumiere brothers in France I think it was the late 19th century.
And for the first 20 years of the technology, the way people kind of accessed it or thought about it and the kinds of things that people did were based on theater.
Because that, of course, is thousands of years old.
In a theater, you sit in an audience, and there's a frame, which is the proscenium.
And then stuff happens in the frame.
And a lot of the plastic elements that we ran through the other day are very, very relevant.
You've got lighting.
You've got staging.
You've got a frame, a sort of placement of people.
You've got costumes.
You've got sound effects, music even, all sorts of things-- very, very similar.
But it all happens in this kind of static frame right there in the middle of your field of view.
And it was really literally about 20 years before people started going, oh, with a camera we can do something a little bit different than that.
The first films-- and these were wildly popular films.
It was already a big industry.
Very quickly it turned into a major, major entertainment and communication industry.
But people would set up a camera, and then people would do their thing in front of the camera.
And then they would project the thing, and people would be like, wow, this is really cool.
This is like theater, except we can watch it again and again.
And this is very relevant to educational video, because that is essentially what a lecture is.
A lecture is a proscenium presentation of material.
And the question which you're asking yourselves in this class is-- OK, we've got that.
We get that.
When we watch media, whether it's online or on television or in a theater, obviously we have progressed way beyond that visually, in terms of visual language, and the tools and techniques that we use to do that are the things that you need to do in order to become conversant in this quote unquote "visual language," not just a consumer of it but a speaker of it.
So I'm going to go over something that is sort of like a convenient bucket to put things into, and it's also easy to remember.
When I think about camera, I think about the four F's.
And some of them are F because I make them be F. It just is easier.
So F-stop or exposure-- and if you know anything about photography or film, F-stop is the aperture of the camera lens, and you can let more or less light in.
But what I mean by that actually is not just the F-stop or the exposure of the lens, but how you control the light.
It could be lighting.
It could be the shutter speed.
There are a million different ways of controlling the exposure of what you're filming.
The second thing is focus.
The first thing about focus is it's a good idea to have it.
It's kind of one of those non-negotiable things.
You really do want something to be in focus in your frame.
Sometimes you might not, but it's better to do that in post.
Always make sure something's in focus.
You may be working with cameras that take care of this for you-- both of these-- the exposure and the focus.
I'm not sure.
Is that true?
Yeah.
So they're all working with the [INAUDIBLE] or your iPhones.
The equipment that you guys will be working with, you won't have to necessarily-- it's not an element that you can control necessarily on the device itself
Or you can control it only by fooling the device, which is something you might want to do if you get more sort of comfortable with it
To do focus on these cameras, you just literally touch the touchscreen with your finger where you want that to be in focus.
And exposure, we're going to have to tweak with the external lighting and stuff
Yeah.
Yeah.
99 times out of 100 it's pretty straightforward.
And the auto systems on newer cameras are really pretty good about it.
So I'm not going to spend a lot of time dwelling on that.
But they're kind of non-negotiable.
You want to get a good exposure, and you want to get something in focus.
And 9 times out of 10 that something is someone's face, the person who you're filming-- not always but usually.
The third one is focal length.
And that is basically how many millimeters you are shooting at from wide angle to telephoto.
And you're not going to be changing lenses on the camera, but you can zoom in and out.
Focal length is not the same thing as wide shot or close up.
People often think that that's true, but it's not.
You can go completely wide angle on a camera, and then walk up-- I can walk right up to Elizabeth and get a close up.
And similarly, I can go all the way telephoto and walk back 50 feet and get a wide shot.
But they have very, very different kinds of resonances and feelings and emotions.
So I'll try to talk a little bit about that in a minute.
You might want to do everything at a very long lens.
You might want to do everything at a wide angle.
It will mean something different.
It will feel different.
Framing-- this is the big one, and this is kind of like the catch all that I throw all of this sort of real thinking about visual language into.
It's not just framing as I talked about the other day, like, oh, gee, do I center somebody?
Do I put them on the right or on the left?
It's the entire structure of your video.
How am I going to shoot this?
Where I'm going to put the camera?
How is it going to edit together?
Is the camera going to move?
Is it not going to move?
So framing is the big one, and that's mostly what I'm going to talk about today in that broader context.
Quickly, so exposure-- just to get this out of the way-- this is what an aperture looks like and the lens.
Again, it's automated for you.
A big number means not a lot of light getting through, a little number means a lot of light is getting through.
This affects the exposure in a way that I'm not going to get into.
But basically you have a lot more depth of field, a lot more will be in focus the smaller the hole.
Because the photons that get through are more directed, they're less scattered.
And we talked about this.
Just as a sort of an example of classic elements of film-- so these are two stills from Science Out Loud videos.
I think we saw a little bit of the Exoplanets video the other day.
I don't know if you've seen this one on plants
I think they did just a little bit
OK. What I thought was really interesting about these two frames is if I was storyboarding this, what would I draw?
I would draw something like this-- medium shot, presenting to camera.
But that doesn't capture anything about what is compelling about these shots.
It's all the other stuff.
Here we are launching into space, deep space, to look at planets.
And here we are-- bright, welcoming the world of plants.
It's kind of an obvious point.
But this tells us nothing about that, unless you then add a lot of other information.
And that's kind of what storyboarding is about.
It's not just about what does my shot look like, but it's about-- what is all the stuff in the shot that's going to really make it compelling?
And it's really a great moment to start thinking about locations-- a field of flowers.
Wow.
I guess I need a field of flowers.
Where am I going to get that?
Or what if everything was black?
What if it's a black screen?
And how are we going to shoot that?
Lighting-- I don't know if that's even an option here.
I'm not going to go into it
We have a light kit if people are interested in doing lighting.
But come talk to either one of us during office hours or after class or something
The only thing I'm going to say about lighting is it's kind of like being a doctor, like, first do no harm.
If you bring the lights out, and you don't know what you're doing, and you aren't trained on the kit, you can break the kit.
You can hurt yourself or someone else, or you can ruin your video.
On the other hand, it's really, really an important tool and something that-- you can't shoot certain kinds of things without some kind of lighting.
I'm not necessarily recommending that you try to use artificial lights or become a lighting expert.
It's a little beyond the scope.
But if there's something like-- to go back to this example-- you have to light to get a look like that.
Some kind of lighting has to be done-- maybe not film lighting or video lighting, but some kind of lighting set up has to be created in order to get a look like that
[INAUDIBLE]
I'm sorry
[INAUDIBLE]
[INAUDIBLE]
Oh, you can improve it, but you have to begin with something that's been filmed with that intention
If you have an effect that you want to do in your video, you can come talk to me about it.
Because we achieve some of these things with jankier props.
I've done-- I'm not proud to admit this in front of Chris
No, that's
I've done a shoot--
Oh, you don't even know janky till you've been on some of the shoots I've done
I've done a few with desk lamps that I've covered with white tissue paper to help diffuse the light.
You don't need a super fancy light kit to achieve some of the basics
No, absolutely.
And that's exactly what I'm about to get into.
To do sort of like a darkroom thing, you don't need a fancy video light kit, but you do need to light the person.
And yeah, it could be a desk lamp with paper over it to sort of smooth it out, to diffuse the light, but you will need to do something.
It's not going to just magically happen.
And that's the point of this-- lighting doesn't necessarily mean a video light kit.
It means being aware of the environment and the available light that you're shooting in and taking that into account either by manipulating that thing, like a light kit, or by taking advantage of the best light in the world, which is the sun.
So for example, if I was shooting in this room, and I didn't have any lights-- I might love the background out there.
But shooting somebody in front of that window is not going to be possible if I want to get the background.
Because by the time I expose for the person's face, the background is going to completely blown out.
And if I expose for the background, the person is silhouetted.
So unless I want that look, I really can't have it unless I bring in lights.
And it would actually take quite a bit of lighting.
Fluorescents-- you can do a lot with fluorescent overheads.
The difference between me standing here-- look at my face here, and look at my face here.
Where do you want your person to stand?
And you can manipulate that kind of thing even in a lab or a classroom or some other setting.
You can make choices based on the available light or the ambient light that will help you.
And the other thing to think about in terms of exposure-- going back to exposure-- is what do I want to be properly exposed?
In that face floating in blackness, the face is properly exposed.
Everything else is very, very dark.
But the face is properly exposed.
In the flower shot, the whole thing is properly exposed.
It's more even lighting.
There's some nice modeling on the face.
But the whole point of that is that you want to see the environment, you want to see the flowers, you want to have this very vibrant, bright scene.
So being very sensitive to the light that's available is part of lighting.
It's not necessarily setting up a bunch of light stands and video lights and spending two hours when you should have been shooting your scene.
Because it can also be very time consuming
Just a quick tip on lighting-- since most of you will probably be filming indoors and relying a lot on fluorescent overhead lights, the only caveat to that is that oftentimes-- there are only two people in here-- but for me, the glasses is a big issue.
You'll get like a weird shadow underneath on people's faces
Or a glare in the lens
Yeah, or a glare.
So you can compensate for that with maybe a desk lamp that's also shone on your face from a little bit below to hide some of those shadows
Particularly, as Elizabeth mentioned, if you diffuse that light with some kind of tissue or something in front of it-- because one thing about lighting-- the first do no harm-- when you're lighting faces, harsh, direct light, whether it's sunlight or desk lamps or video lights, are very unflattering.
So that's absolutely a great tip.
But I think it's always a good idea to sort of try to diffuse it.
If it's not a hot light, you can put like a tissue or a white paper towel in front of it, and it will give this very nice soft light
But if you're shooting something indoors, and you just feel like the person looks a little funky on the screen, it's probably because it's an overhead light that's casting shadow.
So just experiment with a desk lamp or something
And I'm not going to go into focus too much.
Because again, I know you're going to be working with autofocus.
But just to say-- make sure that what you want to be in focus is in focus.
There are all sorts of things you can do to mess with the depth of field, which is like how much of the plane of your image is in focus.
So if you have a very-- if you're very wide open, if there's not a lot of light, then suddenly you have very shallow focus.
Which means that if you were filming me, my face might be in focus, and the background would be out of focus.
If you are stopping way, way down because there's a ton of light, suddenly it's a very deep focus situation where everything's in focus.
And that may or may not be a good thing.
You may not want more focus.
There's something very nice about just what you want to be in focus and everything else being out of focus, because that makes people's eyes go to what you want them to see.
It also creates depth in the frame
Does everyone understand what depth of field in the frame means?
Here I am.
I don't know if you've seen this Science Out Loud video, Engineering Trash into Treasure.
I won't show it.
But it's got a really nice opening, which uses depth of field really effectively.
So this is a person talking about the creation of charcoal.
And in the foreground are the props, and she's out of focus.
And it's kind of counter-intuitive, because your presenter is out of focus and the props are in focus in the foreground.
But it's actually quite engaging and interesting.
And then what they do-- and this is something that you will have a hard time doing if you are just doing autofocus-- is they shift the focus to reveal her.
And that's a reveal.
That's something we talked about the other day.
Revealing things is part of the visual language that you want to think about-- whether it's a cut, with an edit, a change of the angle, a pan, a camera move.
Revealing your next thing is really just kind of a key visual element.
So here it's like the burning-- I forget what kind of--
Corn husks
Yeah.
The burning corn husks come into frame, and then the charcoal comes into frame, and then the focus shifts and you reveal the person who's been talking to you.
It's cool.
It's quite a nice little opening
It's really hard to achieve depth of field with these camcorders.
One way you can sort of trick it is if you stand really far in front of your background.
That's the closest way you're going to get that blurry background effect.
So if you wanted to film in this classroom for instance, and you didn't want the stuff on the chalkboard to be really visible, you would have Chris move up all the way up to maybe where David is at least and then shoot from the very back of the room
Right.
And so that's what this is.
So again, you have a zoom lens.
If you go all the way wide on the lens, you have much deeper focus than if you zoom all the way in.
So if I walk up and shoot Elizabeth with a very wide angle, everything behind her will be in focus.
If I go way back there, and I get the same kind of close up of Elizabeth with a longer part of the zoom, everything behind her will be out of focus.
Which might be better if we don't care about the background, or if it's messy.
I've often been in labs that just look horrible.
And the best way to handle that is to zoom way in, go way back, and have the background be way out of focus.
Or you can, as Elizabeth mentioned, you can bring the person forward so that there's a lot of distance between the person and the background.
So focus, focal length, all of these things relate to each other.
And again, we're going to move a little quickly
It's about 10 till
No, I'm doing fine, I think.
I think I'm doing OK.
This is an extreme wide angle.
You can see how deep the focus is.
It's infinity.
And this is a telephoto, so it's a close up, but even the leaf behind the bird is out of focus.
Just a word about zooms-- so there are two kinds of zoom, and there's one kind of danger-- well, two kinds of danger actually.
Optical zoom, which is if you zoom in and out with the zoom control on your camera.
That is the way you change the focal length.
Then there's digital zoom, which is usually on these cameras-- I'm assuming these are like this.
If you go past a certain point, you're not changing the lens anymore, you're just zooming in on the image itself digitally.
The problem with that is that your resolution goes down immediately.
It looks really bad.
It starts to pixelate and just fall apart.
So I would recommend that you not use the digital zoom.
It's kind one of those situations where people say, oh, well, when I can use it?
I'm like, well, when the UFO is landing in Killian Court, and you're just too far away to get the shot, definitely use the digital zoom.
But if you have time to do something different or plan a little differently then stick with the optical
And also realize that you can zoom in on a frame and post when you're editing.
So it's generally better to just capture something wider.
So say you want to do like a traditional interview set up, and you're not sure how close you should be into your subject.
Go ahead and err on the side of zooming out as much as possible.
You can always play with how much you zoom when you're editing later.
And the other thing about zooming-- maybe this is just a matter of personal preference.
But generally, I don't like it when I see the camera zooming in live time.
I prefer seeing just a clean cut from a wider angle to the zooming angle
And that's the other part of the danger for sure.
Yeah.
Zooms in general-- unless you're doing it because you know what you're doing-- all these rules are made to be broken.
That's something I should mention right up front.
They're often kind of the lazy person's way of doing something.
And in general, I would even go so far as to say even if you're not planning to show the zoom-- if I was, again, filming Elizabeth, and I wanted to get a wide shot, so I shoot her here.
And then I want a close up.
And I zoom in on Elizabeth, and I get the close up.
There's something kind of unpleasing about going from a wide angle shot to a zoomed in telephoto shot of somebody.
The more sort of smart way to do it is to pick a focal length that makes Elizabeth look great and gives you what you want and then move the camera in.
Don't change the focal length of the lens when you change the size of the shot.
So maybe I end up on a medium shot, in terms of the focal length.
And then when I want to get the close up, I pick up the camera, and I walk in and get the close up.
And that's going to cut better, it's going to look better, it's going to feel better.
It's going to feel like just more controlled and like you sort of knew what you were doing.
So zooms-- very useful, dangerous as well.
All right, so back to the proscenium.
I just want to say a little bit about framing.
What shots do you actually need to tell your story?
Well, the way you figure that out is exactly the way you do it with animation or anything else, you do a storyboard.
And this is a storyboard I did a few years ago for a narrative scene.
But you'll see it contains a lot of the things that we've talked about in this class.
Who's our main character?
This kid.
How do we know that?
Well, the next shot is a point of view of what this person is seeing.
There's a moving shot up, and the next thing that you see is his grandfather's reveal.
You see what they're looking at.
And only then do you actually see a wide shot, which is a home-built airplane being pushed down a field.
So this is kind of visual storytelling-- a very simple thing.
Who's our character?
What do they see?
Who's around them?
And then revealing the whole scene-- it could have been done completely differently.
This could have been the first shot.
It could have been-- let's establish the scene.
Let's see all the people.
Then let's kind of cut in and see our main character, or maybe it's revealed later.
There's no right or wrong answer here.
But the point is is that what you want to do is think through your scene just as you do with animation.
That whole process that Josh took you through is completely relevant to absolutely everything that you're going to shoot.
Just a few notes about what different shots do.
Wide shots establish the location, the environment, the spatial relationships.
They add production value.
I don't know about you, but if I could go to the top of this bridge, I would shoot this shot.
I wouldn't not, because this is awesome.
And that's happened.
I've seen that happen.
People get an incredible location, an incredible view, and they don't get a shot that shows where they were.
That's what we call production value.
Similarly, I think this is an awesome picture.
This was actually taken in 1911.
This is a pre-revolution Russian nobleman in a tiny village in Russia.
It was taken in 1911.
It's an amazing picture.
But I would feel like I missed a huge amount if it was just his face.
You learn so much about the environment, you learn so much about the person, by having a wide shot.
If this was a video, I might want to also have a close up when I started to hear about him and learn about who he was.
But boy, this is great.
How could you miss that?
So again, showing environments, getting production value, establishing characters and relationships, relationships and space.
Closer shots-- well, they kind of bring you into someone's world.
They reveal details-- closer shots of a face, closer shots of what the person is looking at.
If you're explaining something, obviously, really good close shots that show the angle, show from a good angle what you're trying to describe are very, very important.
So again, it's kind of obvious, but just make sure to get those as well.
You want to get the wide shot.
But if you're actually describing a process that's like a physical process that somebody is doing, make sure you document it well, and that you don't shoot it from the angle where the person's hand is covering the key thing.
Just look at your frame and see if you're actually communicating what you want to communicate.
Moving shots-- endless kinds of moving shots.
You can pan the camera on a tripod.
You can walk with a camera.
You can move on a dolly.
They have all sorts of good great uses.
They reveal story beats.
If, again, to keep picking on Elizabeth, if I want to introduce Elizabeth, and I dolly or pan over to Elizabeth, people want to know what the end of that shot is going to be.
What am I going to see?
Oh, this person, who's a character that I am supposed to pay attention to.
Or you can follow a person, it creates a very strong point of view.
If someone's walking and talking and you're following them, then we pay attention to that person, because the camera is picking them out of the background to pay attention to.
So that's part of this visual language.
Similarly, we could do a point of view of that person's walking, and then we're suddenly into their world.
So kind of a classic situation is someone's walking, we're following them, and then you cut to the reverse, and you see what they see.
And now we're in their head.
So it's another great thing.
And then there's just kind of the classic one of we have reptile brains-- this is the same reason we edit it quickly.
Here's a new thing, you can either cut to a new thing, or you can pan or dolly to a new thing.
Let's see-- just a quick word about locations.
You're going to talk about this too, but always scout your locations.
It's smart producing, which Elizabeth is going to talk about, but it's also very smart creatively.
I have been in so many situations where I've gone to a location a few days in advance, and I have completely rewritten my script based on the location.
Either because I saw a problem that I wasn't anticipating-- like a classic one is, OK, well, we're going to film all this stuff at the electron microscope, but there's a column here, and I can't shoot anything from this angle.
Oh, shit.
It's really good to find that out a few days in advance, because you can't move the microscope.
So what are you going to do?
Or because the location is really cool, and there's amazing stuff.
And you get very excited, because there's all sorts of possibilities that open up that you couldn't have ever imagined just sitting, doing your own story board in your room or writing your script.
But suddenly because this place is amazing, you see all sorts of things that you could do.
So I always suggest trying to visit your location.
One, for just kind of practical scouting reasons, like is there a chainsaw factory next door?
But two, because it can completely affect your creative vision.
And try to do it enough in advance so that you can incorporate what you learn.
So I'm going to just wrap up with a couple words about editing.
This is where all of the stuff comes together and where you find out whether your plan worked-- your storyboard, your visual language, your script.
And it's also where you fix all the things that didn't work.
And I guarantee you that plenty of things will not work.
Just a word about pacing, B-roll, and transitions.
So pacing is kind of like how much do I edit.
And this is kind of the big question, like, gee, what's the big thing?
What's the secret?
What's my pacing?
What should it be like?
And I don't think there is an answer to this.
When I was in film school 20-plus years ago, I was being taught by people who were trained in the 1950s.
And people who were trained in the 1950s would tell us, well, you really can't edit faster than four seconds per shot.
Because the human brain just can't comprehend an image that lasts shorter than four seconds.
And I remember dutifully editing my very first piece and following that rule and watching it and going, this sucks.
This is terrible.
What's wrong with this?
This is awful.
And it's because four seconds is a really long time in some cases, in a lot of cases, in many cases, in most cases.
And that has increased-- has only increased.
My idea of a fast edit may be completely different than your idea of a fast edit because I'm older.
I don't know.
And there's also the difference between genres.
I think Elizabeth mentioned this briefly, television-- there's this sort of like bombard everybody.
If you want to show an establishing shot of a house in a reality TV show, you can't just show one shot or two shots.
You have to show eight shots.
They're each 12 frames long, like, boom, boom, boom, boom, boom, with drums going.
Just to make the reptile brain go, oh, yeah, this is important somehow.
I don't know.
And it gets kind of absurd at a certain point, and it can be, obviously, overediting.
But the more typical experience that people who are just starting out have is that they don't shoot enough, not that they shoot too much.
You think, oh, I've got three shots, and this is going to cover like 30 seconds of my video.
But 30 seconds can be an eternity.
And if your shots aren't telling a story, that could be pretty skimpy.
Which brings us to B-roll, I'm going to talk about that in one second with an example.
The other thing, though, to think about is transitions.
How do I get from this to that?
Obviously, you can just cut.
But is there an issue-- like I'm cutting from a medium shot of Chris talking, talking, talking, to a medium shot of Elizabeth talking, talking, talking.
That's not necessarily super compelling.
Do I need some other way of transitioning?
Do I need a wide shot of the classroom?
Do I need people listening?
What other shots do I need to tell my story and to transition from scene to scene?
The B-roll I just wanted to mention.
Have you guys talked about what that even means at all?
OK.
I was going to show this in the PowerPoint.
But B-roll is a term that goes back, way, way back, like 80 years ago, when people edited on film.
The B-roll was the second roll of film that they would then composite over the first roll of film.
And it was all the stuff that wasn't included in the sort of main narrative.
And I'll show you what I mean in a minute.
This is just a little snippet from a video that we shot not too long ago with Jeff Hoffman, who's a professor here in AeroAstro, a former astronaut.
And we had done a storyboard and kind of a script plan.
So we knew when he was going to be on camera and when not.
[VIDEO PLAYBACK] -The design, construction, and operation of aerospace vehicles have allowed us to leave the surface of the earth and fly through our planet's atmosphere and out into space.
I'm Jeff Hoffman.
I'm a professor in the Department of Aero-- [END PLAYBACK]
So anyway, we knew that he wasn't going to be on camera in the first part.
So we were able to cut between takes.
We didn't even bother to really have him be compelling on camera at that point.
But here we knew he would be on camera, so we filmed him that way.
And the first string out is just literally putting together your script in a very linear way, the way you wrote it, without any kind of fancy stuff.
And then I'll just show you what it looked like after.
I'll just show the first 10 seconds, and then I'll be done.
[VIDEO PLAYBACK] [MUSIC PLAYING] -Commencing sequence start-- 6, 5, 4, 3, 2, 1, 0.
All engines running.
Lift off.
We have a lift off.
32 minutes past the hour, lift off on Apollo 11.
-Have you ever wondered how rockets work?
Have you ever looked out of the window of an airplane and wondered how something so big and heavy can actually fly?
Have you ever wondered what happens to the human body in space, or how a spacesuit keeps astronauts alive and able to work in the vacuum of space?
These topics are all part of aerospace engineering.
Aerospace engineering is giving humans the freedom to move in three dimensions.
The design, construction, and operation of aerospace vehicles have allowed us to leave the surface of the earth and fly through our planet's atmosphere and out into space.
-Mission specialist Jeff Hoffman.
-Hello, Jeff?
-I'm Jeff Hoffman.
I'm a professor in the Department of Aeronautics and Astronautics at MIT.
Before coming to MIT, I spent 19 years at-- [END PLAYBACK]
[AUDIO OUT] We were able to shoot with him very, very quickly, because we knew what we needed on camera.
And we knew that we were going to be able to cover and fill and sort of explain things with other imagery.
In this case, a lot of it's obviously archival stuff, but it could be stuff that we shot.
And later in the video there is stuff that we shot, but it was all the same plan.
Anyway, I know that's a lot of stuff thrown at you, but I'm also cognizant of the time.
So we can talk about this after class, too, if you would like.
Thanks
So just to give another example of B-roll maybe on a more likely level for you guys, because I assume you won't be going out into outer space
Well, neither did we.
I called NASA.
NASA is a great resource
For Anastasia's video-- [VIDEO PLAYBACK] -What do these awesome compounds have in common?
They all come from plants!
[MUSIC PLAYING] [END PLAYBACK]
You notice that she's not talking over any of it.
We put the B-roll in because we needed to give the video a little bit of space to breathe.
Otherwise it would have been just her talking constantly and saying facts.
So we wanted to help with the pacing of the video, which is what Chris was talking about.
So that's why we shot B-roll.
So if you're-- especially you, Paul, because I imagine you're going to shoot some stuff maybe with a boat, sort of like you have access to certain equipment.
Whoever you're filming with, you're not only going to shoot, Paul, but I would also spend some time maybe just doing a pan of your environment, shooting some of the equipment that's around.
It may not be super relevant to what you're talking about exactly in those moments, but you can always have that footage to intersperse like we did here, if that makes sense.
And as Chris was saying, you're not always going to shoot B-roll yourself.
NASA, I guess, is what you guys used.
And all of their stuff is-- they're not copyrighted or licensed.
So you can use that footage.
There's a lot of-- there a lot of photos and pictures that you can look up.
I'm going to talk a little bit more about music that you can use, and I can hit graphics as well next week during the editing lecture.
But there are resources out there, if you want certain footage that you just can't shoot yourself.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to talk a little bit about producing and realizing your visions, and give an example of some of the things that Chris was talking about.
So we shot a video on how engines worked, and there is a part of the video where the two creators were talking about how a turbine engine works.
And I think I might have told this story before, about how we ended up finding like an actual Boeing engine.
But there are a lot of ways that we could have filmed this scene.
We could have animated it, we could have done a wide frame shot with animation overlay, like I showed you with George's [INAUDIBLE] videos.
We could have done B-roll, so we could have just done like close up shots of the engine while you heard Luke explaining how it worked, with his voiceover.
Or we could have done a live explanation, which is what we went with.
So this is how George shot Luke.
We went with the traditional thirds framing.
Again, this isn't necessarily a rule that you have to go by, it just worked really well for this scene.
And we went with this because it also gave us the ability to do sort of an interesting point of view shot.
The point of view isn't from the narrator, it's actually from the viewer.
So I'll show you the clip in a second.
But essentially, we did one long take where George moved his camera as Luke was explaining the parts of the engine.
So we wanted it to feel like the camera was basically where a person would be if Luke was explaining how a turbine engine works to a person standing there
--won't work
What you really need is one engine that can produce enough horsepower to fly the plane.
A turbine engine is capable of much more horse power than a piston engine because of the way it uses giant fans to compress the air
At the front of the turbine engine is an intake fan which spins and brings a huge amount of air into the front of the engine.
As we move backwards through the engine, a series of compressor blades makes that air more and more compressed, until we get here.
And here, fuel is introduced and burned.
And that hot, high pressure flow gets sent out the back of the engine.
And that makes the thrust of the engine.
But at the same time, that flow passes through a series of four or five turbine blades, and those blades are connected directly to a shaft at the middle of the engine.
That shaft runs all the way back to the front and spins the intake fan.
That's what keeps the engine running
So that shot worked because Luke was very comfortable explaining that in one long take.
He actually ad libbed that whole thing.
And I can see that it's a lot of information at once, so maybe it would have been better to do an animation.
We certainly could've done it a lot of different ways.
But what I like about that shot is that if you were just standing there, and Luke was explaining it to you, the camera would basically be your eyes, right?
Like you would walk along with him, and you would look down at the engine when he pointed to it, you would look back up at him.
So that's the type of movement that Chris was talking about that you can introduce to your video.
It would be really different if we had just set up a tripod and had Luke explain the whole thing, right?
Like it would have been a very different experience
And it's not random movement, it's obviously very directed, which is the key to many of these kinds of things.
It's not [INAUDIBLE]
Right.
Like we wanted to really follow what the natural person would do if they were just standing there talking to Luke.
I want to talk about the lighting real quick.
So we were in an airplane hangar, and it was like one in the morning, so it was super dark outside.
I mean, we mostly relied on the overhead lights.
I don't actually know if we lit this with anything else.
Because it's super bright in there.
So you don't need super fancy equipment to achieve the videos that you guys are going to do.
So to talk a little bit more about producing, when you think of directing I feel like that's what most people think of when making a video.
And directing is a lot about thinking in the present.
So you're coaching the host, you're coaching the talent, telling them what to do right there in that moment.
You're thinking about what the framing looks like in that moment.
Producing is a lot about thinking about the future, anticipating what's coming next, the mistakes that you're going to make.
So on a practical level it's a lot of check boxes, basically.
You're thinking about where am I going to film, what are my backups going to be, where am I going to record voice over?
My experience filming at MIT is that it is incredibly difficult to find a quiet place on campus to record clean voiceover.
And if you're having trouble finding a spot, come talk to me at any time.
I can help you with that
What do you mean that?
Why do you mean that?
So take this classroom for example, most of the spaces on campus have this hum overhead.
And most of you guys will be relying on the lavalier mic, so it's not as bad.
But there's just a lot of background noise that is on campus.
So if you're just recording plain voice over that you want to use in your video, it would be more ideal if you could get something that's quieter than this room, for example.
So you're thinking in checklists, and some of the boxes are a little more inane, like did we shoot all of the scenes?
Which sounds stupid, but I actually missed that on my own video.
I forgot to film one of the scenes.
And then, did we get clean audio?
So a tip that I would recommend for you guys is, even after you film all of your scenes, I would sit down and just record voiceover of your entire script in a separate audio file.
Because you never know, looking back on the footage, there may be sections where the audio isn't very clean.
So you may have to rely on your clean voiceover for that section.
And maybe use B-roll to fill in.
If that makes sense to people.
Does anyone have questions so far?
Yes
Like you are filming, and then you have a voice--
Yeah
--then you want to use a voiceover separately.
Then wouldn't the pacing off?
Yeah.
So for example we shot a video, and part of it-- I'll talk more about it during the editing lecture next week, but if you can find a clean place to transition between the live footage and your voiceover, that's like the best way to do that if you find that some of your footage just has really bad live audio.
For every place that you shoot, I recommend that you get room tone, which is basically the natural sound of the room.
So for example, if you were going to shoot in this room, you would shoot the scene, and then you would take your mic and just record 30 seconds where everyone's quiet.
And it's just going to record the background noise of the room.
And you do that because if there is a case where you're relying on voiceover that you shot later, or you're cutting bits and pieces from the footage that you shot in this room, you're going to have this gap or a hiccup where the hum doesn't exist, and it's going to sound really awkward in the video.
So you layer the room tone underneath when you're editing so that you don't have those hiccups between the cuts that you make
It's kind of a decision that you have to make, too, based on the room.
So like with Jeff, we actually did the voiceover in the room.
Because it had a hum, so we could have done a better job of the voiceover.
But what we decided was that it would be more jarring if there was a major audio quality difference between the voice over and the on camera, you know, had we done it in a studio or in a better space later.
So, you know, it was clean enough, we accepted it.
And that's why we had him do everything on the spot in the same place, so that it wouldn't have that drop in and out
Yeah.
Does that make sense to people?
Like if we took the footage, like this footage where I'm talking here, and then we cut to something where I continue to talk about this class, but I was doing it over a voiceover, and we had some B-roll of you guys working.
The first scene, you would hear the hum, and you'd hear me talking.
But in the second scene, you would just hear like very clean dead space while I'm talking.
And you would see you guys working.
And it would sound and look really weird.
It would feel really weird to watch
So we just did another video not unlike the [INAUDIBLE] video, except that the person is in Kendall Square talking.
All the on camera stuff was in Kendall Square.
And that's really noisy, so we decided in that case to do the opposite.
We didn't have him record everything in Kendall Square, because then we would be dealing with problems throughout.
But what we did is we recorded the equivalent of room tone.
We just got audio of Kendall Square, and then we were able to sort of slowly fade it out after the person was on camera, so that it was more imperceptible.
And then slowly fade it back in before an on camera arrived-- you know, comes back
There's no way you can filter it out?
Just kind of--
Well you can filter audio.
What I always tell people is don't.
Especially like, if you're not going to spend thousands of dollars doing it, don't think you can do a lot.
There are a lot of consumer tools that are available.
They generally really degrade the voice.
Everyone ends up sounding like I'm Stephen Hawking
Because the frequencies in the room tone--
Very similar, yeah
--some of the frequencies in the voice
So very, very high end, very time consuming tools are out there to do it.
But they're out of reach
Music can help hide that a little bit, and I'm going to show you guys examples of that next week.
But in general like it literally takes 30 seconds for you to record the room tone.
So that's--
Yeah, it's definitely worth it
--It's worth it.
So in general, as a producer you're doing a lot of anticipatory thinking, which is to help you ultimately realize your big vision for your video.
So you're doing all of these checklists, you're thinking ahead of time.
Even in pre-production, which is where we are right now, where you're scripting.
As a producer you're going to be thinking about the big picture the entire time.
So producing is really looking at the big picture.
You're looking at how the microscopic decisions like, where you're going to shoot for instance, or where are you going to record the room tone, how that all fits into your macroscopic plan.
So as Chris said, it's really important to scout locations.
And when you go scout locations, the two big things to look at are what's the lighting like at that place, and what's the sound like?
And when you're looking at lighting, not only look at the natural light that's available in the room, if it's indoors for example, but how the sunlight affects the room.
So when we were pre-producing this OpenCourseWare class, we came and scouted this classroom.
And we noticed that there are all these big windows, and it was an afternoon class.
And I knew that the lighting in this room would be really different at 1 o'clock than it would be at 4 o'clock.
So we decided that it actually wouldn't be too big of a deal, because people watching the video wouldn't be like, oh my gosh it's dark in that classroom now, that doesn't make any sense.
Because obviously we know it's a three hour class
You'll notice Billy's on that side of the room, not on this side of the room
Yeah
That's because if Elizabeth was teaching in that corner, we'd have a problem with the windows
And we thought about if we should just lower the shades the entire time so we have uniform lighting.
So those are the kind of questions that you'll ask yourself during pre-production.
And sound, again, take room tone.
The biggest sound issues if you're outdoors, the wind makes a big difference.
Most of you guys are going to just rely on your lavaliers, and they have the little cap.
And that does a generally good job, but just take note of it when you go scout.
My personal tip is look for outlets.
That's always been like the biggest thing for us is, especially if it's an outdoor location, scouting ahead of time and seeing how many outlets there are helps you anticipate how many extension cords you need to bring.
Which sounds like a silly thing, but actually makes a huge difference.
So for example we shot this video with Jamie in the Stata Center on computer switches.
And we were with the Digi-Comp.
Or the Digi-Cart?
Digi-Comp.
Which we couldn't move, so we had to shoot in this location.
And it was in Stata so we knew it would be really noisy.
So we shot at like 6:00 in the morning on Sunday, when no one would be around.
But the thing that we didn't anticipate, because I didn't scout that location, which is bad, is that there's a skylight right above the Digi-Comp-- I always forget what it's called.
So noon hit, and we had to stop filming because there was sunlight directly overhead that completely changed the shadows on the prop, and on Jamie.
So we basically lost two hours of shooting time because we were waiting for the sun to pass overhead.
So that's why you should scout your locations ahead of time.
A quick note on locations, also look at backgrounds.
And with backgrounds you should really be deliberate.
And go big or go home.
So for example, this is a background that I think a lot of people default to when they don't really have anything else to shoot, because they just sit-in their office and they're like, I need to record something really fast, I'll just sit here.
Sitting in front of a window that has blinds on it is maybe the worst possible background that you could use.
And again, this is a matter of personal taste, but I cringe when I see this for two reasons.
One, it's really back-lit, which is what Chris was talking about earlier.
So you'll have so much outdoor light that it makes your subjects look really dark.
And having all the blinds in the background, I think having all those lines is really distracting as well.
This is what happened when George tried to change the exposure setting on the camera to light for me.
So I'm lit well, but now the background is completely blown out.
So in general, avoid filming right in front of a window where there is direct sunlight coming in.
And please don't film in front of blinds.
If anyone is interested in filming in front of a green screen, it's possible, and I'm happy to help you.
I think you should be very deliberate with that decision.
Sometimes it works really well.
This is an example
So in order to turn red, neurons from the squid's brain sends signals to these muscles, which pull open all of his chromatophores at the same time
So-- oh, this was the wrong clip.
Sorry guys.
But there is one example where she says, take note, when I walk next to the squid, he turns red.
And so you actually see her walking on screen, and then the squid turns red.
And so we used a green screen to film this because we like the actual visual of seeing some sort of object stimulating the squid to turn colors.
So that's an example of a green screen working well, I think in my opinion.
If you want to do something like that, I'm happy to help you.
But again, you don't have to do it.
So more production tips.
Days before, make sure you scout your locations.
A word about props.
So props don't have to be expensive.
And I'm not just talking about props on screen.
I'm also talking about the tools that you can use to film.
So we're giving you a tripod, and that's basically your only production prop.
But as Chris was saying, like you can use desk lamps with paper over them.
Let me show you this shot
Let me show you how
So those are all dolly shots basically, of us establishing the location of this video, which is in D lab, which is where Jack works.
And they're all very smooth pans of looking around at the locations.
And we shot that with a cart, like a hand cart that I just sat in, and George pushed me around.
So you could buy a super expensive dolly, or you can just borrow a cart from our department, and sit-in it, and have someone push you around.
The dolly is always nicer, but we don't have the budget for that
Great job with steady camerawork by the way.
It can be an issue if you're sitting on a cart, if you start getting a lot of this, it takes away from the niceness of the shot.
But in this case it was great
We took a tripod and spread the legs out all the way, so that it was like really, really firm in the cart.
And then I held it just to add more stability.
So it's possible this is another one
To figure out how the entire metabolome works, we can use it to engineer plants to create new bio-materials, medicines, and clean energy.
We might even discover that plants have the secret to living forever.
We just need to unlock their chemical mysteries
So that's actually one of my favorite shots from Science Out Loud, ever.
And that was all George's idea, so I have to give him credit.
But I like it because it's a reveal, like Chris was talking about.
It's the way we ended the video.
We start off on the plants, focused on it, and you reveal Anastasia at the end.
And that's actually-- to get that smooth movement is actually really hard.
You might be able to do it with these tripods because the cameras aren't really heavy.
And you could probably get enough friction on them to get it to work.
But what we did was, George was using a camera like that one.
So he took the tripod, flipped the top head of it upside down, and then he took a boom pole, which is basically this long pole, and put a weight on one end of it, and put the camera on the other, and just slowly let the camera come up from the weight.
So I'm all about MacGyvering props if you need to.
There are creative ways to get around things and achieve nice effects with cheaper equipment.
The other thing about days before is that you should make a shot list, which is something that I would like you guys to submit.
I think originally I had specified you to do that tonight.
But it's not necessary to do it and post it until next week.
So I'll update you with another email.
But a shot list is basically what it says.
It's a list of all the shots that you need.
So it doesn't have to look a certain way.
It's for your benefit only.
So organize it however you think is best for you.
But this was my shot list for the reshoots that we did for the snots video.
This is the first page of it.
So I listed out all the locations that we were going to, what time approximately we were going to use it, so that I could get a good sense of what the lighting would look like in these locations.
And then I listed out exactly what lines I needed to say, and what shots we needed to get at each location.
And then at the bottom I specified the wardrobe.
So this is a super helpful organizational tool to use when you're planning things.
When you are thinking about locations, definitely go big or go home.
You can e-mail anyone and just tell them you're doing a project for a class at MIT, and you'd be surprised what you could get.
We shot at a horse farm in Acton.
We shot in an airplane.
We shot at a skydiving zone.
We shot at the butterfly exhibit at the Museum of Science.
Most of the labs here, especially since you're students, are going to be super open to having you.
And again that's like your opportunity to create a window into a world that most people don't have access to.
So definitely take advantage of that.
If you have any issues getting into a location, just let me know.
The day before, check all your equipment.
Check the batteries, that's the biggest thing.
Bring spare batteries.
If you need spare batteries let me know.
Double check your locations.
If you're meeting people, make sure that they know that you're coming.
And check the weather.
And then day of.
So to help keep you organized I recommend that you use camera reports or log sheets.
And what that looks like, and again you can set this up however you think is most useful to you to use.
These are the camera reports that we use for Science Out Loud.
But basically it's to help us keep track of what video file is what.
And when you're shooting like for six days straight, this is really important.
It may not be as vital to you, but it'll be really useful when you go into edit.
But it's basically saying, this video file was scene two, take two.
I thought it was OK.
There were two good takes in there.
The last one was the best.
Make sense to everyone?
Let me know if you have any questions.
Yeah.
That's pretty straightforward.
And I would also recommend looking at the dailies.
So I guess dailies refers to all the footage that you take during a day.
Like maybe during a movie production or a TV production.
And the dailies for you guys would be like, at the end of the day look through all the video footage that you have on your camera and just take note of what you shot.
Make sure that it's good.
What you don't want is to shoot your entire episode and realize that the camera settings had somehow muted you and so you didn't have any audio.
So that's why looking at the dailies is important.
Also back up and label all your files like right after you get back from a shoot.
So the biggest production piece of advice, or like heads up that I would give you guys is that it takes a lot of time.
Even if the locations are scouted and everything's sort of organized perfectly the day of, you're going to take-- you're going to do a lot of takes.
Even if the person says the line perfectly, try a bunch of deliveries.
So remember how George was saying like, say it really loud!
Now say it like however you want, right?
And you got a different style.
Try a bunch of deliveries.
Try a different bunch of point of views with the camera.
So maybe you shoot someone straight on, then you try something from the side.
You try a different movement.
And then like try different types of framing.
And Chris's piece of advice, which was like, most people tend to not get enough footage than get too much, is really true.
Shoot like you're going to use everything, and then when you edit, you're going to edit it like you're not going to keep anything.
But when you're shooting, you always want to have more to work with, because the worst feeling is when you're sitting, editing, and you're like, I wish I had shot x y z.
It's the worst feeling.
So it's going to take a lot of time, because you want to create as many options for you as you can.
Any questions about that?
No?
OK.
So I'll show you an example of this.
No, one more thing.
A note about clothing, because sometimes this comes up.
So in general you don't want to have really fine print.
So I always try to avoid stripes.
This is probably like the closest or the widest difference in your stripes as I would recommend getting.
Generally like solid colors work the best on screen.
And then the material, bring a couple different sets of shirts with you, because sometimes like the silkier material brushes up against the mic a little bit more, and you hear that actually on the audio.
I will show you some raw footage from the snots video.
This is super embarrassing for me, so.
But I think it's good.
And maybe will just sort of set the tone that this is a safe space where you can feel like you can just be yourself.
But this was for B-roll at the beginning of the video that was literally going to last-- it's going to last one second in the final product.
I literally mean I think it's going to be a one second clip.
And this is the amount of footage we shot for it
--little weight is so funny
OK, thank you.
And go.
Put it down and get the little weight.
Both hands.
Do the thing where you like, you're bringing it up with your right hand, and then you're sort of like spotting it with your left a little bit.
OK. No, no just two fingers on the other hand.
On your hand, on your hand, the back of your hand.
Like
This?
Higher.
Turn the weight more.
Yeah.
Just go like this.
Put-- yes, and then like, as if you're helping it up.
MALE IN VIDEO 2: [INAUDIBLE] just like, supporting it
Yeah, like that, but don't take your face out of frame.
Slower.
Struggle a little bit more.
All right, good
So the point of making you guys endure that is that the final clip is going to be just, right?
Literally a second.
This is at least two minutes worth of footage.
This was like the third take.
So this is what I mean by get as much as possible.
Try as many different things as you can.
Because we could have used the first one where I just like lifted the weight, but the last one was really the best take out of all of them.
And it wasn't-- we didn't achieve that.
Or we achieved that because George made me do it for like 10 minutes.
But don't tell him I said that.
So these are all the day of things.
Now for the day after, take a step back for now.
It's going to be hard to go in and edit your stuff like right after you finish shooting it.
You want to give yourself a little bit of time to sort of clear your perspective.
And I'll talk more about this during the editing lecture.
But even when you do the camera reports and you're logging sort of what the best takes are the day of, you might watch it a couple days later.
And I recommend re-logging everything and going through footage again later.
And you might find that the thing that you liked during the day of, maybe you liked it because you were starting to get tired, and it's going to look a little bit different later.
So that's why I just recommend the day after, just taking a step back.
Let's see.
Oh.
With the whole multiple takes thing, I forgot to mention this before, but I thought of it when you were doing your lecture.
I would also recommend doing takes from varying degrees of separation from your subject.
So say your host says, I'm going to say this line, and you shoot it from where Chris is standing.
Even if the host like nails the line perfectly, tell them to do it again and shoot it from maybe from where Yulia's standing.
And tell them to do it again, and shoot it from like right here.
It's just good to have those varying degrees of separation.
It'll be easier to do the zooming and everything in post.
But again, having those options to work with.
You're not going to know which one you're going to use necessarily in the moment that you're filming.
So that's why it's good to have a couple options
The term is-- the technical term, not that it matters, is called, coverage.
So basically, you cover yourself.
I don't really know how I'm going to edit this together, so I'm going to give myself as many choices as possible in the edit room.
Because even the best directors don't know exactly what they want.
I mean there's Alfred Hitchcock supposedly only filmed exactly what he wanted.
And his editors went insane because they could only do one way.
But pretty much everyone else who's ever worked at any level in film and video does coverage
Does anyone have questions?
The rest of class, we're going to let you go actually film a scene from your script.
And maybe this is a good time to take a break, but Chris and I will stick around if you have questions about producing your specific scene, or actually using any of the equipment, we'll be around to help you with that stuff
Can I make one other suggestion, or just reinforce one thing about producing.
This comes up over and over again, and it was a very hard lesson for me to learn initially.
And that's being realistic about your schedule.
You know there's a tendency to want to pack a ton of stuff in and to be very ambitious.
And what happens is that you have to give enough time for setting things up, breaking things down, moving from this lab to that lab, or this location of that location, eating something, figuring something out.
So if you go like, well, OK.
This is a 10 minute scene, this will take me 20 minutes to edit.
And then we'll be over at Stata, and we'll shoot that two minute scene in five minutes.
You know, very, very quickly get behind schedule, and then the panic sets in.
It's far better to have a schedule that's realistic, where you really think about, not just what it's going to take to shoot, but what it's going to take to set up, breakdown, figure stuff out, and move.
And then execute the schedule in a realistic way so you're not panicking, you're not freaking out, you're not calling people and saying, I know I said I would be at your lab at 3:00 but I can't get there till 5:00, can you stick around?
You know, whatever it may be.
So just be very realistic
For the Science Out Loud episodes, we allot at least 12 hours per five minute episode in terms of filming time with the students.
Now that's a huge number, mostly because the thing that's most time consuming for us is setting up the lights, which is something that you're not going to have to worry about.
The lighting actually-- I mean, it's by far the most time consuming thing.
But it gives you an idea of-- that's a really great point I'm glad that you made it-- that it's more than just the amount of time it takes for you to say a line.
Hopefully you won't have that panic, because the way this class is structured, it's very front heavy.
We've had a lot of lectures so far but after the editing lecture, that's actually the very last one.
And the rest of the class period will mostly be for you to work on stuff during class time.
So don't worry too much about that.
We tried to anticipate that for you guys.
But just to let you know.
I forgot had a couple examples for you for some of the stuff that Chris was mentioning.
And I'll play them real quick
What is the most awesome place in the world?
Seems like that would be pretty much the most subjective question ever.
But this week, scientists used a set of objective criteria to come up with a list of the world's most biologically important places.
There might be--
So that's an example of having zooms, but not actually showing you zooming with the camera, right?
And I think they do it for a couple reasons.
He probably-- he might have messed up his line in between, and so that was a way to sort of cut in between and not make the cuts like super awkward, by having a zoom in and a zoom out.
And it's sort of like a rhythmic visual pacing thing, too.
It cuts in like after he says--
It's kind of equivalent to, oh, different angle, pay attention.
It's a different convention than television, but it comes from a very similar place.
You know, if he was just standing there, delivering without edits, there'd be this-- there's this terror that you might turn away, and might not watch it
To achieve that, though, if you wanted to do something like that, I would say, again, you could even film this with a tripod and film yourself.
But I would shoot yourself-- that sounds weird-- I would take the camera and put it on tripod like that far away, deliver your whole sentence, and then move it forward and deliver the whole thing again.
Because you don't know exactly where you're going to cut into that
And that's clearly what most people-- I mean, this is a very conventional thing you see on the web all the time.
And that's how they film.
You know, a bunch of takes of wide, a bunch of takes of close, medium, whatever.
Editing
But the point is you're going to be saying the same line multiple times, even if you nail it.
This is where a zoom might work.
[VIDEO PLAYING]
So I say, or is it.
But we use the zoom there intentionally to make it funny.
Which hopefully it'll read funny in the actual thing.
So again, there aren't like hard and fast rules were you should never ever use a zoom, or you should never ever center the person, or you should never ever--
It's interesting, I mean, you know, I was thinking about that whole centering thing after the other day, Monday.
And a lot of times when people break those kind of traditional rules, is for comedic effect.
And center frame stuff tends to read funny.
And you see that in movies all the time.
If you start paying attention to this.
If you go to a comedy, things tend to be more center frame than if it's a drama.
And I think one of the reasons that vlogs are often center framed is that, even if they're not funny, that's the affect.
It's like this is amusing, this is light, this is not scary.
And that's why people center frame.
And you know, you can, too.
If you want to
All right so, at this point I will let you guys free.
Oh, sorry I have another thing to say.
The film that you've seen today-- the scene that you film today won't be due until at least Tuesday.
If you need more time to work on it, we can push it to Wednesday, too.
And that's because I really want your focus to be on honing the script.
Because Monday will be our table read, where everyone reads their script aloud to each other.
And you're really going to get the most out of the table read when you come to it with as complete as a script as you can.
I'm not saying that what you have on Monday should be as close to final product as possible, but it's just really hard to give concrete feedback to something that isn't fully fleshed out.
So do try to come on Monday with as complete of a script as you can, because Monday is going to be the most useful to you if you do.
So that being said, some people have already uploaded second drafts onto Tumblr.
If you could upload stuff by tomorrow at noon, I can give you feedback on it so that you can incorporate that before Monday.
And the best way to get feedback from me is to double post to Tumblr, and then also write a Google doc, and just email it to me, and give me commenting permissions.
That's how we actually edit all the scripts from Science Out Loud.
So I'll show you, real quick.
Season two.
Let's go to this one.
Yeah, so this is eventually how we asked the students to format their scripts.
You don't have to do that for Monday, but eventually you're going to want to do it.
I think it makes organizing things a lot easier.
So it's just like a four column format, where you specify what scene number it is, the location.
So OS just means on screen.
It means like you're not going to B-roll, you're just going to be delivering it live.
Just a description of what the video looks like, and what the script is itself.
And if you share it to me on Google Docs, it just makes it easier for me to comment for you.
And George also said that we will give script feedbacks this weekend too.
So if you can e-mail us this stuff by Saturday at noon, we can give you lots of feedback before Monday
I have another tip that I usually forget about because I'm not used to talking for long periods of time, and you guys will probably get tired at some point while filming.
Bring a water bottle and probably snacks too, if you get hungry easily.
Just because when your voice gets really dry and starts cracking, that might ruin a take that otherwise went really, really well.
And it helps with the consistency of your voice.
And you can edit together something that you did later with something that you did earlier if your voice sounds like consistently fresh throughout it.
Because I've had problems trying to edit audio where at the end of it, my voice is really, really raspy.
And at the beginning it's still good
Yeah, that's a great point.
I'm glad you brought that up.
So we try to cram everything in as much as we can, which is purely because of logistical issues, and because of our budget.
But you're going to have the time this month to where you're not going to have to shoot your entire video in one day.
So really try to spread it out over the course of several days.
And that's built into the class time.
But I will just say from personal experience, it is so incredibly difficult to film for more than five or six hours straight.
You just get really, really exhausted by the end of it.
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So for today we're just going to break you guys up into groups.
So Kenneth and Andrea, you guys can work together today for the rest of class.
Paul, Ulea, and David.
Since we have an odd numbers, there's going to be a group of three.
And then Nathan and Joshua, you guys can film each other.
And I know that you're going to be a little bit limited with locations today.
So if you wanted to film on location, you might not be able to do that for today.
But just get comfortable with, like, the movements of the camera, incorporating the stuff Chris talked about earlier.
That's what we want you guys to do today.
And break!
Go team!
[MANY STUDENTS TALKING]
In the year 2009, famous astrophysicist Stephen Hawking and actually hosted a [INAUDIBLE]
[INAUDIBLE]
So, like, just shoot
The goal I'm trying to get you and the-- You're grabbing that, right?
No, because this should be in
Through all sorts of [INAUDIBLE].
Even [INAUDIBLE].
Even [INAUDIBLE].
But absolutely no one came
Just go [INAUDIBLE]
Oh, that sucks
So that was all in my shot
I mean this could be had if they're going [INAUDIBLE] half a second shock response
Or in the couch or where they are
[INAUDIBLE]
Oh, I'm sorry.
Where are they
Since it should be [INAUDIBLE].
You would grab some water, grab one, pour the water, [INAUDIBLE]
So like, I'm kind of like-- [INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today, we are having a table read, which basically means that we're all going to go around and read the scripts where you have them standing as if now.
The way it'll work is that say, Siri goes first.
She would read the script aloud.
We would all keep our laptops closed for the first reading.
So just listen to her script as it flows.
And then the second time she reads it, we will follow along on her script on annotation studio and make notes in real time.
And then afterwards we'll talk about any feedback that we have for her.
And then we'll just go around the class.
Do you want to say anything before we start?
I think not really, other than the fact that, I'm sure you guys know this, but feedback is best when it's very specific.
So as we give feedback to each other, and you've been doing this for the most part, is to just really as much mentioning the things that really work as it does the things that don't work.
Because I'm sure you all maybe felt this way when we were work-shopping the other day, but as a writer, it's very hard to put yourself out there and have people just give you critical feedback.
So if we can also remember to just give some positive critical feedback as well, about things that we liked and things that worked, that helps the person in the hot seat feel a little bit better
Yeah, for sure.
And I also wanted to say, don't be afraid to repeat comments that other people have made, especially on annotation studio.
If you notice that someone has made the same note that you were going to make, just that tag a reply to it saying like ditto or something.
Because that'll draw people's attention to a space that's worth looking at a little bit more
Andrea had something to say.
I just want to make sure I don't lose it
Oh, yeah.
Sorry
Just one of the techniques that I learned, actually from an entrepreneurship class, was the technique of doing the yes/and, as a way of giving positive feedback where you're sort of riffing a little bit or adding something to what somebody just said.
And I think you guys did that, actually, in your comments.
And that was received very positively, because it's like yes, you get it, and you're adding more to it
And I will say that looking over the scripts that were sent in this weekend, I think everyone's in a really good spot as far as ideas go.
Like the topics of your videos are actually very, very fascinating.
So it's a really good space to be in.
I think that the common sort of room for improvement that people have right now is connecting specific examples to the bigger picture.
And I think that you get that in your head, but making it a little more explicit in your script is where your focus should be on next.
And I tried to give some of that feedback online.
And maybe today I can explain some more of the comments if you didn't understand them as well.
But connecting the facts that make up a tutorial video into a bigger picture, like this is why you should care, is a very tricky thing to do.
But hopefully today, with the feedback of everyone in class, you can have a better sense of how to do it.
And I'll also say that at the end of the day this is your video, and the final decisions are up to you.
So it is OK if people offer different pieces of advice, conflicting opinions.
In a lot of my feedback I was saying this as a matter of personal taste.
I don't really find this necessary in this part.
And that's OK.
Final decisions are up to you as long as you sort of justify why you've made the choices that you have.
And that I don't want you to feel like you need to come away from this table read with a checklist of things that you need to do.
And then once you do them, like, the script is magically finished.
Because that's not going to happen.
And unfortunately, that's sort of how a lot of us approach these sort of things.
At least that's how I first approached them.
So it's OK to repeat people.
Constructive feedback is good.
The yes/and thing is a good strategy.
And I don't know if there's anything else
[INAUDIBLE] I'm the only one who doesn't know how to get onto the annotation software.
Does everyone know how to do that?
No?
Yes?
OK, then maybe I'm the only one who doesn't know how to do this
Here's the homework on the first day
Oh
There's a link on the Tumblr
OK. We should probably, in order to make this run smoothly, all be ready so that there isn't a big gap in between the first person.
[INAUDIBLE]
Oh, this will also be an opportunity to practice hosting.
So whenever you're reading the script, it's OK if things aren't memorized completely.
But try to deliver it as you would on screen and we'll give you feedback on the hosting as well.
So for this first reading, Julia, just go ahead and read it start to finish.
If you have queues for like b-roll or something like that, just say this is the part where I'm going to have some b-roll
OK, and should I also describe the animations?
Um, yeah.
Just like briefly, briefly go over that.
And so for this first read, everyone just close the computers.
If you have to make notes, do it analog.
Whenever you're ready
What do a cellphone, a river, and a cancer cell have in common?
The answer is fractals.
Fractals in mathematics are never ending patterns.
Scientists can program these infinite patterns by repeating a simple mathematical process over and over.
So if you zoom in, you'll see the shape-- the same shape, again and again and again.
And here I would show an animation of that on a computer.
Similarly, a tree grows by repetitive branching.
Just like a fractal, a tree extends its branches one smaller than the other, but similar.
A tree can't grow as far and precisely as a truly mathematical fractal, but we can still study nature in terms of fractals.
In fact, so many things in nature have these pattern properties-- and show animation of all those things-- it sometimes feels like the world itself is one giant fractal.
Rivers of the planet flow like the blood vessels in our bodies.
Lightning bolts become electrifying rivers of the sky.
And just look at this honey-- show fractal honey-- here's something even wackier, a brain-fractal shaped forest.
One way to explain this abundance of patterns is the fact that nature is just great at reusing efficient mechanisms.
How and why that happens, we can't really tell.
But although the existence of fractals remains a mystery, mathematicians have found a way to study the wacky structures.
Clouds are not spheres and bark is not smooth, but with fractal geometry we can mathematically explore them.
In the 1970s, a mathematician, named Benoit Mandelbrot, was hired to investigate noise in telephone lines.
Now Mandelbrot loved connecting images with numbers, so he immediately graphed the data he collected.
And he came up with this-- show Mandelbrot fractal, and also the equation that comes along with it.
At first, the image didn't look too special.
In fact, it kind of resembled a turtle with a giant head.
It wasn't until night time that Mandelbrot looked closer.
He zoomed in once and found a smaller turtle etched onto the original one, and an even smaller turtle on that one.
Mandelbrot kept zooming and zooming and the turtles kept shrinking and shrinking, but they were still all the same shape.
Mandelbrot was convinced he'd seen a nightmare.
But when the shape remained on the screen the next day, Mandelbrot knew he was onto something huge, a simple equation applied repeatedly, carried incredible properties.
What if, thought he, you could create such expressions for other natural phenomena?
And that's exactly what mathematicians do today.
Fractal geometry allows them to model, say, mountain ranges, and then use the models to study earthquakes or creed realistic special effects for our favorite movies.
So you would show animating a mountain range from fractal triangles, and then a scene from Star Wars where that was used, or other movie.
In healthier news, fractals may also help doctors diagnose cancer faster and more accurately.
They can study the edges of various cells in our bodies using fractal geometry.
Here the cell on the right is more jagged and repeating than the one on the left, which means it's the more aggressive, faster-growing cancer cell.
This way of discovering cancer can be that 10 times more effective than the current methods.
So that's how cancer cells and rivers relate.
But what about cell phones?
They aren't really part of nature.
Well, in the '90s a radio astronomer by the name of Nathan Cohen was having troubles with this landlord.
The man wouldn't let him put a radio antenna on the roof.
So Cohen decided to make a more compact fractal radio antenna instead.
The landlord didn't notice it.
And it worked better than the ones before.
Working further, Cohen designed a new version, this time using a shape called the menger sponge.
The fractal's infinite sponginess allowed the antenna to receive multiple different signals.
The menger sponge is not really the sponge you'd be scrubbing your back with, but you can still think of it like that.
Imagine both water and soap getting through sponges holes, except the water is wifi and the soap is, say, Bluetooth.
Without Cohen's sponge, your cellphone would have to look something like a giraffe to receive both those signals.
Not quite as handy, is it?
Fractals are already very common, yet we're still searching for more applications, asking questions, building new patterns, and exploring nature's best, here at MIT and everywhere in the world.
Look around you.
What beautiful patterns do you see?
[APPLAUSE]
All right
Can I ask a question before we start?
Absolutely
Is the reason why you have us not having our laptops up so that we don't want to start writing.
Or from your perspective, as someone who doesn't process information auditorily, like with my ears, that was a really hard activity for me.
Is it possible to still accomplish your goal while looking if we don't type?
So the reason why George and I don't have people look at the text the first time around is because when you're watching the video, ultimately you're not reading it with the script in front of you.
And you'll notice that there may be moments where you tune out when you don't have a script in front of you following along, which is going to be the experience that a viewer is going to have.
And I know that that's very difficult to do, because we don't have the visuals, which is why we also do it again with the script.
But the first time is more to just like experience it as close to an experience as the viewer would have of the video, because they're not going to be following along with notes.
They're not going to know what's coming next.
So whatever confusion you may feel in this first read-through is probably more similar to what an actual viewer of the video will feel
That helps me understand that.
I can deal with my discomfort now
So how about for this next part, let's all go to annotation studio and look at the fractals document.
Sorry, my internet's slow.
Would you guys find it more helpful if, for the second time through, everyone got to read through the script at their own pace and then offered feedback?
Would you rather do it that way?
Yes?
OK. Then how about everyone take a look through Julia's script.
Take a couple minutes to do that.
And if you have feedback while you're reading through, go ahead and make notes.
And then we'll talk about it afterwards.
All right, has everyone gotten a chance to at least read through the whole script?
OK. Let's go ahead and talk about it now, just in the interest of time.
Julia, did you want to talk a little bit about some of the edits you were already thinking about making or some of the things that you're struggling with right now?
Yeah, so what the instructors have suggested is to take out some of the stories and examples and focus on one story, which is the one that has to do with the cellphone, because that is an example of taking a mathematical concept and applying it to real life.
So that way I could delete or shorten the Mandelbrot story.
But also some of the examples of fractals in nature, I would focus on them less and kind of put them at the end, maybe describe them less.
But instead, explain more what a fractal is and some equations that come along with it, which to me is the hardest part, because the equations of fractals, even though they are-- So the equation for Mandelbrot, for example, is z equals z squared plus c. But the z's in there are complex numbers.
And you just keep iterating the equation.
So even though it could be simple to show what's happening in just a quick animation kind of plugging in numbers and see what happens, I cringe at the idea of simplifying it to that, because that is not necessarily the accurate representation.
So that's why originally I did not include the equation.
I kind of wanted to, maybe, animate just the z equals z squared plus c at some point just to show what it looks like, but let the reader kind of explore that on their own if they wanted to.
So I guess right now my biggest problem is can I and should I include more math in this or not?
I mean I don't think it's necessarily about the math.
I think that the script has a good specific example.
And it has a really great big picture.
Like the pitch is awesome.
I think that's a really great point to work from.
It's just that the connecting piece seems a little bit-- like there's not as much connective tissue there.
So when I'm reading through this, and I saw Nathan kind of had a similar comment, too, I don't quite get how all the specific examples actually relate to that big picture.
And for me, personally, it was understanding what exactly-- I don't fully understand what a fractal is yet, so I can't make that jump with you to all the applications of fractals.
So I understand what you're saying how the math itself is maybe a little complicated to explain, but I don't understand how a fractal is math, I guess.
Does that make sense to other people?
I don't know if anyone else felt that same confusion But it was like, I get the turtle thing, and I get it's a repeating thing, but I don't understand how math describes that
OK, I guess that's an easy fix because you can just say there's an equation that goes along with all of these fractals that we can program
I had a similar-- like to me, that's the big piece that's missing with this.
That being said, let's pause and take a moment, and like you're big idea at the very beginning was to talk about how math isn't real, right?
And this is so amazing that you've come so far.
And these stories-- that I just commented-- that they make it real in this way that when we were first sketching on the board that first day, you were really struggling to figure out what were those anchors that were going to help you talk about that abstract idea.
And I feel like you really are doing a fantastic job at grounding this big idea in these stories.
That being said, this is when being an expert or thinking deeply about a topic is so hard, being able to separate and to figure out what the rest of the world does or doesn't know about this thing, right?
To you, what is probably very obvious, and I'm going to be very honest that I'm not quantitative person, right?
So I'm probably at a sixth grade math level for this kind of a video.
And I completely had a gap for me being like, I faintly recall having studied fractals in my early middle school years.
I remembered that it's a pattern and I remember kind of what it looks like.
I have no idea how a mathematician goes from seeing a tree to being able to break that into an equation, and what that would even look like, or how it would-- that to me-- and how that becomes into a fractal.
And to me, that's a big mystery to me, that I don't want you to explain everything because that would probably be really boring, honestly
[INAUDIBLE]
Totally.
But if there's some way, on a high level, explain what a mathematician is doing in there, why would I study that?
Like, the big questions for me are like, why would I want to study that tree?
How would I go about doing it in such a way that would bring me to some sort of mathematical-- the why and the how is very interesting to me in terms of what the mathematician's actually doing and why they would do it and then how you would apply that to other things.
So I don't need you to go into the whole equation, but I need to understand the process that a mathematician would go through.
I don't know if you guys feel similarly
I was thinking with that, what I was thinking the whole time, I know it's not fractal geometry or anything like that, but if you were to take-- you mentioned mountains, and just to describe how they're doing this-- because they know y equals mx plus b in sixth, seventh grade.
I think that's when they start to learn it.
So you could just put like a slope of this line.
It's described by this equation.
And it described this slope of a mountain.
And that's kind of what it does, just to a lot higher degree.
I think that's something that they would understand what they already know with the math behind that.
And it's just a little bit more complex.
And one part where I thought that you could put that in is in scene three.
So you just described fractals as never-ending patterns.
So I think there's room there where you could kind of go into-- there's not even much rearranging you have to do.
You could probably just insert something right there
OK. Good to hear
I thought there was really good examples that describe-- I thought it was pretty good
Nathan brought up a point that I also had when I read this script for the first time about having so many examples.
Do you want to explain that further?
Well, just at the beginning I thought that there are a lot of different examples in a row that were really cool examples.
But all of them, I was like, each one I tried to stop and think, well how is that a thing?
How does that work.
But then there's another example.
I just kind of got a bit overloaded
OK.
I think the animation might help with that because where I talk about rivers, blood vessels, and lightning bolts, and honey, if you look at them from above, like if you have pictures, they look very, very similar.
So the idea is kind of to show the similarity.
I guess somebody mentioned that the brain-shaped forest was kind of overkill with that.
So I can definitely understand that
I think that they're good examples.
But what might be an issue here is that it's taking too long to get to the bulk of the video.
And this is something that George and I were talking about, too, that you have really three themes in this video, cancer cells, nature, and cellphone lines.
And I like how you open it with what do all these things have in common.
But there's honestly so much to explore in each facet, that I'm wondering if you should just really focus on one, like you had said earlier.
Just because the thing that I kept thinking about-- which is along the lines of Nathan's thinking, and I mentioned it in the comments, too-- but it really reads like a BuzzFeed article almost at the beginning, with all these examples.
And I keep thinking about how awesome of an article it would be with like GIFs showing all these different things.
But there's not enough of a compelling reason to make a video out of it, right?
The whole story line of what's his name, Nathan Cohen, is actually, I thought that was really interesting.
I don't know what example you guys thought was the most compelling to listen to or read.
What did you guys feel about the cancer cells or--
From a biologist's standpoint, exactly what your concerns are with reducing the equations and math, that's what I felt you did unintentionally with cancer cells, in that you have to this really punchy statement, like this way of discovering cancer can be about 10 times more effective than the current methods.
But coming from a biology background, you cut out so much in explaining how cancer is detected, what are the current methods and comparing it to the fact that the edges of the blubbing cells look like fractals in the first place.
And so for me this was your weakest example.
And I enjoyed reading your script.
And the other examples, I feel like just because you spent more time explaining them and fleshing them out, and you have, I think from your background, a better understanding of the Nathan Cohen story.
They just sounded like stronger examples.
So in my opinion, cut out the cancer even though it's cool and cancer is a very hot topic right now in all fields of study.
I think people will get more out of your video and more focused point if you choose fewer stories to tell
And by doing so, you actually allow yourself to be able to allude to the cancer topic without diving into it.
If you pick the Nathan story, for instance, and you go more deeply into explaining more of the meat that we're talking about, it doesn't mean that you don't get to mention how fractals could have an impact on cancer, right?
It doesn't give you the same depth, but it's easier to apply the concept when you understand something deeply in that way.
If that makes sense
Yeah, it does make sense
I think in general, and again this is sort of something that I saw with a lot of scripts, it's a lot of breadth and not enough depth.
So lots of examples, but sort of shallowly sitting on top of them.
I think this is an opportunity to really challenge your audience and teach them something really substantive that they're not going to get in school.
And you can do that through the example.
And I think there's a lot that you can cut out.
And again, we've noted them on the annotation so you can see exactly what we're talking about.
But if you really rely on showing not telling, for example scene nine where you're describing rivers of the planet, the brain-shape forest, like you don't even have to say any of those things if you have the images pop up.
Like there are lots of pattern fractal patterns in nature, like in rivers, anatomy, the sky, honey, right?
Like you've reduced an entire scene down to basically five words there.
The other thing I wanted to mentioned, because this is sort of a tool that anyone can use, we talk a lot about the reveal, right?
So Chris was talking about how you can use the camera to make a reveal.
You have a really good set up for a reveal in the first scene.
What do x, y, z have in common.
That's pretty much exactly how the plants video was set up.
What do all these chemical compounds have in common?
They all come from plants.
And the reveal is that all these unfamiliar things come from a familiar thing.
Here you've got sort of the reverse set up.
You have all of these familiar things come from an unfamiliar thing.
And I don't think a reveal works as well, necessarily, when you've got that set up, because people don't know what a fractal is.
And so if you maybe switch scenes three and two, or maybe get rid of scene two, and just go straight to seem three, the answer are never ending patterns seen in nature and math called fractals.
That's like a subtle nuance, but a reveal set up to where you lead with a set of familiar, and then you reveal what the punchline of an unfamiliar.
It doesn't work as well, necessarily, because the response is going to be "what" instead of oh, whoa, plants.
You know?
Can I add one thing?
Yeah, of course.
We talked about how the math stuff was really complicated.
I feel like this-- maybe I'm way out in left field with this one-- but I feel like as soon as you feel like you're in that zone of being like, oh, I know a lot about this and it would be really hard for me to figure out how to share this with my audience, then to me, that's our negative truth is with these videos, is figuring out how to share those complicated ideas with the lay-audience.
As soon as you're in that uncomfortable zone and being like, I know a lot about this and it's complicated, that's where the truth is.
That's it.
That's our gift that we're offering the world, is how can you simplify a complicated idea for the public.
So if you're in that discomfort place, I fell like we need to live in there.
That's our zone.
That's just the way I see it
Would it help if like-- because I feel like I'm very curious of all the fractals [INAUDIBLE], would it help if [INAUDIBLE] explained it.
When she said [INAUDIBLE], I just couldn't stop thinking about it.
[INAUDIBLE]
Yeah, it's up to Julia to figure out what style helps her the best, whether talking it out or writing it out or what is your tool that's going to help you to get that complicated idea shared.
And that's each of your challenges to figure out your own learning.
To figure out how do you go into that challenging place and figure out how to simplify it.
And if it's talking it out, then that's great.
But maybe it's not for Julia, I don't know
I also wanted to use your script as an example to talk about a broader thing that everyone can use, which is how you set up things with your intonation.
So people generally associate emphasizing things with emphasizing your voice.
At first-- this is scene 12-- at first the image didn't look too special.
In fact, it kind of resembled a turtle with a giant head.
But it wasn't until nighttime that he looked closer and he zoomed in.
And there was a smaller one and a smaller one.
And he thought they were all the same shape.
And he had a nightmare.
Like that's sort of the default way that we try to bring focus onto a natural subject when we're hosting.
But you can play a lot with emphasizing with the smaller voice, if that makes sense.
So instead of saying he zoomed in on a smaller one, and a smaller one, and a smaller one.
And then he had a nightmare.
You can emphasize the weirdness by pulling back at an unexpected point.
He zoomed in and he saw a smaller one, and a smaller one, and then a smaller one.
Those are two different types of deliveries that you can use.
But you don't always have to rely on using your volume and sort of the brashness of your voice to emphasize certain topics, because it gets a little repetitive over the course of the entire episode.
So you can do that for certain sentences, but for others, really play up the power of actually a quiet volume and an unexpected pull back that also draws attention too.
Does that makes sense to people?
OK, does that give you enough to work with?
Yes
I think it's a really good start.
And again, let me know if you guys disagree, but I think that if you jump into the meat of your episode a little bit earlier, really dive into the Nathan Cohen story, you can use the anecdotes to actually explain some of the things instead of having to take whole scenes to go over concepts with an analogy that you don't need necessarily
Yeah, I definitely hear that.
That makes a lot of sense.
[INAUDIBLE]
And just so you know, I timed it.
And took you five minutes to read through the script.
And I would give yourselves about a minute buffer room on top of however long it takes you to read the script and even describe the scenes.
Because with b-roll and with cuts, you're going to have a little more time that your video is going to last, in addition to the time it takes for you to read the script.
All right, does anyone want to go next?
How about, the top one on the list.
Why do some people handle the cold better than others?
So this is David.
And did you change the script at all since last time I saw it?
So my idea was-- I added some more research, to make it more [INAUDIBLE].
But I also think that maybe the wording in it is not ideal
Wording is definitely the easiest thing to address in editing.
So why don't you just go ahead and read the script aloud to us.
And if everyone can pull their laptops down
So why do people handle cold-- why do some people handle cold better than others?
Why do some people need to where lots of layers, where others feel fine in running shorts.
What makes all the difference?
Imagine a giant furnace.
To generate more heat, we need to burn more coal.
Now imagine your body as this giant furnace.
Our metabolism is the fire.
And sugars, which are broken down carbohydrates [INAUDIBLE], are the coals.
Inside your cells, the sugars are burned by mitochondria to produce heat and ATP, a molecule that starts and releases energy as required by the cell.
To generate more heat for one, our bodies burn more sugars.
This the first way we deal with cold.
The second way we react to cold is that our blood is restricted by the other-- our blood is restricted through the other organs.
The blood circulatory system exits highways to the different organs.
Imagine [INAUDIBLE] trucks carrying oxygen and heat to the organs.
As the speed of the trucks is higher, more heat falls out and is lost to the surroundings, hence our body slows down the flow of blood by tightening the blood vessels.
It is the same way as squeezing a lane on the highway.
The third way is during more extreme case of cold.
Our body results to quick skeletal muscle contractions, called shivering, in an intent to create warmth by expending energy.
It turns out that our bodies aren't always equally created.
A team of California geneticists, led by Doctor Douglas C. Wallace of the University of California, has found that many of the world's people are genetically adapted to the cold because their ancestors lived in northern climates during the ice age.
This is a very big chuck [INAUDIBLE].
The genetic change affects basic body metabolism.
The genetic adaptation is still carried by many northern Europeans, East Asians, and American Indians, most of whose ancestors once lived Serbia.
But is absent from people native to Africa.
The genetic change affects the mitochondria, causing it to generate more heat and less chemical energy, which was very helpful to early ancestors trying to survive the cold.
Other than our genetic make up affecting how much we can withstand the cold, our physical makeup also plays a part in our resist to cold.
We lose heat to the environment through convection of the air surrounding our bodies.
When there's a greater difference in temperature, or more surface area exposed, there's a greater heat loss.
We can slow down this process by reducing the surface area in contact with the cold surface or by increasing thermal resistance by insulation.
In 1877, American biologist [INAUDIBLE] showed that the length of one's limbs affected the amount of heat lost to the environment.
Bodies with stockier frames and shorter arms mean less surface area exposed to the cold.
This is so men with smaller bodies, which have more surface area to volume, lose heat more rapidly.
Fat, which acts as an insulation, helps increase thermal resistance, making one lose heat at a slower rate.
Thus more people with-- [INAUDIBLE] people with a healthy [INAUDIBLE] will be able to withstand the cold.
Other than our genetic and physical makeup, there's one more way to resist the cold, meditation.
Introducing Wim Hof from the Netherlands.
In 2009 he completed a full marathon in temperatures below minus 30 degrees Celsius, dressed in nothing but his shorts.
Wim Hof is aptly named Iceman for his ability to withstand extreme cold conditions by turning up internal thermostat of his mind.
Wim Hof practices geothermal meditation that allows his body to produce more heat than the average person.
Now this sounds wishy-washy and non-scientific, but a team of researchers, led by associate professor Maria [INAUDIBLE], from the department of psychology at NUS, faculty of arts and social science, showed for the first time that it is possible for the core body temperature to be controlled by the brain.
The scientists found that core body temperature increases can be achieved using certain meditation techniques.
A second study was conducted with Western participants who use a breathing technique of the geothermal meditative practice.
They were able to increase their core body temperature within limits.
Now that we understand our genetic and physical makeup, and how it affects resistance the cold, as well as how to combat the cold, maybe if the next Ice Age were to come, you will better be able to withstand and survive it
Now, so we'll all go look at his script and-- actually, maybe this time we can talk more of the feedback and you can take notes, because you took a little longer than I thought it would.
So I just want to leave enough time for everyone.
But a quick note, when you guys read your scripts during the table read, read it like you would actually host it.
You clocked in at exactly five minutes, but you read a lot faster than you talk normally.
And when people read, they often have the exact same intonation structure when they read a sentence.
Which is to go, this is how I'm reading this sentence.
I'm reading it, and my tone goes down at the end.
I'm reading the next sentence, and it's going down at the end.
And so every single sentence sounds the same.
Does that makes sense?
And when you do that-- and I know that you're just reading off of the script, but it's a habit that everyone falls into.
When you do that, it makes it very, very evident that you're working off of a script.
When you talk in real life, you have different intonations in your voice.
You go up.
You take kind of pauses.
Your voice goes down at the end of some sentences.
Sometimes it goes up if you're asking a question.
But when you're reading verbatim from text, your intonation tends to fall into the habit of having the same intonation structure.
So just be mindful of that.
And for people who are going next, really try to read your script as you would present in the video.
So that was just a quick note.
Now how about everyone just read through the script.
Don't worry about commenting.
And then we'll all just sort of do live comment feedback on this one.
OK, so I think this script has similar strengths as Julia's in that you have a lot of very specific examples.
But I think it also struggles with having a hodgepodge of anecdotes, and not necessarily a real, unifying, noticeable theme.
My big question about this script is, is your theme more about why some people feel more comfortable in the cold or why some people would survive better in the cold?
Because that's a point that Jamie brought up in the last class
The second
The second.
Because what I don't understand is how these specific examples actually imply better survival in the cold, because they're all about you burn more energy to produce more heat.
But are some people's genetic makeup actually-- does it make them more prone to surviving in the cold?
Or does it just make them more comfortable in the cold?
Some people's genetic makeup allows them to create more heat, and therefore better survive the cold.
Which is what makes them more comfortable.
[INAUDIBLE]
But the way you open up the video, it kind of implies why do some people wear a lot of layers while other people can go running.
Are you implying that the people who go running in shorts could survive being out in the cold longer?
[INAUDIBLE].
They are able to do that because-- maybe because of the reasons below, maybe they're genetic makeup is predisposed to better withstand the cold.
Or maybe they have a physical makeup that is just there that helps them to do
But is that scientifically proven?
Yeah.
Based on the-- I cited the documents, the scientific study
Because to me, I don't know how you guys feel, but to me, that's actually not established with the facts that you've given.
Like if you took the person who was wearing shorts running and the person who was wearing a coat and you stuck them in Antarctica, would the person who was wearing shorts live longer?
I mean the person who was wearing shorts wouldn't complain as soon, right?
The person who bundles up, they'd complain about being cold, but would they necessarily die first?
Yeah, [INAUDIBLE] while you were talking I put on my jacket
Like I have seen stuff.
I mean-- and I know whatever I've seen there's probably a million things out there to debate it, but if people are in hypothermic conditions-- and this kind of alludes to the metaphysical, I guess, like that aspect of you talking about with the marathon runner-- some people are able to make their core temperature warmer than it would be in just like ambient temperature, if they're actually in a colder environment.
I have seen stuff like that.
What I thought would be cool with that piece, though, was if at the beginning you say why do some people handle the cold better than others, well, it's all these questions.
But if you introduced with like, in 2009, whoever ran this marathon and he was wearing shorts.
Like how was he able to do that.
That way I think you're taking out that first chunk and you're replacing it with what you already have in there.
So you're shortening it a little bit.
And I think it's more of an interesting introduction.
Because that's the stuff that I thought was really interesting, is like how are people able to do this and scientists study them and whatever your facts are
Yeah, I agree.
I think it's a much more compelling opening than why do some people wear shorts outside.
So basically moving scene 3C to the beginning.
Go ahead, Julia
The way that the scene 3C is currently placed it almost seems like the odd one out because it talks about meditation.
So if you did start with it, that would be really cool.
But also maybe replacing the word meditation with you're controlling something with your brain, just so it seems more like a scientifically-related concept.
And the also, another thing is you name a lot of scientists and dates.
So maybe you could talk about those experiments but not, name, people or locations.
That might make it easier because then you don't have to think about California and Northern Europe in the same sentence that mean different things
Nathan, were you going to say something?
I actually think it's more of the second scene.
I feel like the second scene is kind of explaining on a cellular level and anatomic level how we stay warm.
And it doesn't really connect to well to either the intro or the third scene.
I think it's actually really interesting, but right now it's kind of like-- it doesn't really talk about why some people do any of these better than others.
And then there's only like one part in the third scene where mitochondria is mentioned.
I think that just needs a little bit better connected to everything else
How vital is scene two?
Basically, my thought process of how this is going to go is introduction, and then how our body handles, and what is the difference between people
It's a very classic five paragraph essay form, right?
Intro, background knowledge, question, example, right?
Which is a very logical flow of ideas, which is why so many people write essays that way.
But I think it's also why it's reading a little bit like a news story right now.
It's reading a little bit like a part of a textbook, almost.
And it really-- the stuff at the end is really the more interesting part, to me at least, the reason why maybe it should be a video over an article.
And I think PJ's idea of moving the example from the guy from Iceland as your opening will help with that.
And I also agree with Nathan that you may not even need most of scene two, right, because the point of the video isn't to explain how we stay warm.
The point of the video is to explain why some people are more predisposed to handling incredibly harsh environments than others.
Like I think the concept of hypothermia is fascinating, but that word isn't mentioned that all in this video.
And let me know if you guys disagree, because this is a totally personal taste thing, but I feel like that would be a much more compelling example to use than to say you stay warm by shivering.
And you stay warm by your blood vessels contracting.
Like those are all very true facts that don't take away, necessarily, from understanding how people are more predisposed, it's just that you spend such a long amount of time on that.
And it takes a while to get the bulk of your body.
It's like the same thing that was happening with Julia's.
And it's very counter-intuitive because it's very different from typical science communication.
That's not how you write a journal article.
With a journal article, you span like five pages with an intro and set up.
But with the video, I don't think you need the second thing.
I think you need bits and pieces from it to explain the point, but what do you guys think?
I think may be to just like identify second scene.
So it's on a cellular level.
And then we shiver.
So skeletal.
Maybe just identifying it but not really expanding on it might be something to do
Jamie, were you going to something?
Yeah, I'm just processing.
This is one by going deep it allows you to go shallow as well, like with the Julia with diving deeply [INAUDIBLE] to something else.
I'm with you now, about if we start with the idea of this runner who was able to beat the cold with his him, through telling that story, you can actually tell me about what's happening on a cellular level and how he was able to beat it, right?
So one thing that I think needs to be said somewhere in this is the idea that if you strip away some of the variables like what people ate or-- as someone who used to be a serious hiker, if someone has to go to the bathroom versus doesn't, that actually makes a really big difference on staying warm.
Because if you think about it, if you have all this liquid and you need to go to the bathroom, your body's warming all of that up.
And if you just went to the bathroom you'd suddenly get a lot warmer.
So taking those variables out, I'd say all things being the same, these two fundamental bodies, what's going on in a cellular level?
That helps you understand the mind breaking, the cool extreme example we have of this guy beating the system somehow.
But in order to understand how cool it is that he's broken and beat the system, we kind of need to understand what the norm is for the system.
So by using him as your freak example, it allows you to also tell the story of how things should work in a normal-- do you see what I'm saying?
And then suddenly his story becomes your streamline, that forward motion throughout your video.
And the backward telling of the story of the facts about how a person breaks down glucose or whatever, ends up being part of that forward motion because you're trying to explain this cool, weird freak thing of someone can actually use their mind to change, or at least to appear to change their temperature.
So I really like this re-framing.
And I think it allows you to go deeper and to make it more interesting by doing less
I think you also need to bring the point of the video earlier on, because right now it comes in scene four at the very end.
Maybe if the Ice Age were to come, we would better be able to withstand and survive it.
It's really random right now, because you haven't really set that up as an argument.
So I would say put that at the beginning, too.
Like there was this guy a long time ago.
And he ran this marathon half naked.
Why didn't he die?
Is there something about him that is genetically-- don't say genetically superior because that's very un-PC.
But you know, are there genetic traits about him that would equip him to not only withstand cold temperatures, but maybe survive the next Ice Age.
Is there something about him, about humans like him, that are different from other people.
That's what I'm saying about it needs to get into the meat and the bulk of the video sooner, because right now it's a lot of intro, intro, intro.
And the example doesn't even come until scene 3C, which is over halfway through the video.
Does that makes sense to people, what Jamie just said, about using a specific anecdote to describe a lot of the core concepts instead of just describing the core concept separately?
Does that make sense?
You can have a lot of fun with that as it being your main story.
Like imagine you running half naked across the football field with obvious snow out, right?
That's a really fun intro.
I mean, there are really fun things you can do that also make it concrete.
Because you've done all the hard work already, which was coming up with the science behind it.
You've already made it into a concrete story.
Here are three specific things that might be happening.
You just need a hook and a story line to help you tell that
The application is what's going to differentiate it from a textbook as well.
Because right now, scene two is a textbook moved to video.
And that's fine, but it's not going to be interesting, I think, has all your anecdotes.
I mean I have the concern that this-- what's his name-- coldman, Iceman.
I had a concern that is wasn't scientific enough or robust.
But since you added the stuff about the research from NUS, I do think that's interesting.
Expand on that more.
And cut out stuff from scene two to give you time to do that.
So I would say open with Iceman.
Open with the big picture question, what are the fundamental qualities about him that differentiate him from other humans like me.
Then you can talk about what exactly happens when people die of the cold, essentially.
Like why is that happening, it's because your body can't keep up with burning enough fuel.
So you're talking about the whole furnace concept in that reason, but you're not taking the time to separately explain it to people.
And then-- so you have the example.
You have the huge question.
You go into the details of explaining the core concepts.
Then go back out to the research question again.
Does that makes sense?
I think the really critical part of what [INAUDIBLE] just said that's not in your video right now is explaining what happens when people die of the cold.
Because that'll really drive home your big question, how will we survive the next Ice Age.
How people died in the cold in the first place.
So what happens when these systems that generate heat fail.
And what about them fails
And this may be too much.
So as author, totally feel free to ignore this next idea, but at the very beginning you talk about putting clothes on as a way of combating that cycle.
What are the ways that we-- and I don't want you to dive deeply into this, but it might be a nice beginning of a wrap up.
What are the ways that we can fight the cold given our current cellular structure.
So I'm born this way.
This is me in the world.
What can I do right now to fight the cold.
Obviously I can put on another jacket.
What's my jacket doing, fundamentally
And you talk about that right now in scene two, but do you see that makes much more sense in the context of Jamie's example?
So you don't have to take the time to say, let me define shivering.
Let me define blood constriction.
Let me define metabolises.
Don't take the time to define those things separately from a context.
Just seamlessly integrate that into the examples you're already talking about
Or would eating a big spoon of peanut butter help me.
I would want to probably know some concrete things at the end, just wrapping up, along with the scientific.
As someone who used to do some pretty extreme outdoor stuff, I know that eating a spoonful of peanut butter actually will help, but why?
And just bringing it back on that very concrete level to your audience, to be like, because of carbohydrates here.
So next time you're about to stand out and watch a football game in the cold, eat a huge bar of chocolate or put on an extra layer, or something concrete.
And not a lot of time.
Don't waste a lot of time.
That might tie it back to the concrete for your audience a little bit
I wouldn't end it that way, though, because that's a little too simplistic for what you could do.
So end it by tying it back to the big question, for sure.
And that's what I meant when I said in your comments, like what type of research is going on.
Because there must be people out there who are studying ways to sort of trick our bodies against the way we're naturally hardwired beyond just putting on another jacket.
So if you can talk about those things, I mean that's another opportunity to differentiate it from a traditional textbook content.
So right, now the way your script is structured, is simple question, facts, background facts, a bunch of research questions, and then really good question.
That's like a very standard five paragraph format.
I would say restructure it so that you open with one of the anecdotes, Iceman.
Open with the big question.
Go into the details in the context of the example.
Then go back out to the research question.
Then you can do what Jamie says, which is like well, if some people, like Iceman, just are better off, like what about for the rest of us?
What are researchers doing or what can we practically do to sort of trick nature.
And then point it back to the big question again.
Does that make sense?
It might be worth it to-- hibernation's like a hot topic right now with all the space travel and stuff that they're thinking of.
So that's like-- I mean that's addressing eating a lot, low metabolic, it kind of, I think, gets all of that.
[INAUDIBLE]
But also they're doing that for space research, too, right?
There are a lot of resources at your disposal, so you can talk about things that have never been talked about before.
That's the whole point of these videos.
That's what I mean by don't leave them as instructional tutorial videos, because anyone can make those, but we have current research at our disposal.
There's so many things that we can talk about that just haven't been talked about yet, so I really want you to spend most of the time of your videos on that
And you might be able to find-- I mean I have no idea who does this on campus-- but I bet we could probably do some research to figure out is anyone in [INAUDIBLE].
I don't know, someone.
There might be someone on campus that you could talk to that's doing some research on this [INAUDIBLE] thing.
If you could add a little of that it would be cool.
What's the pacing that you're planning for this?
Pacing for today?
Like how many people are we hoping to get?
I was going to try to get as many people as we could.
Tomorrow I'm going to do my last lecture to give you guys more time to work on your video than we had originally planned.
Yeah, so just as many as we can get through today.
Whatever we don't finish we'll do tomorrow
We should probably give people a chance to break soon
Yes, absolutely.
Let's do one more and then take a break.
David, do you have any questions about that?
About all of our feedback?
[INAUDIBLE]
So for example, one section I think could be condensed to a single sentence, like 3A.
So you have this really big paragraph.
It's like a paragraph in your essay.
You have a citation and everything.
But that section can just be reduced to some people have different mitochondria, which generates more hear and less chemical energy.
Like that's it.
That was the essence of section 3
[INAUDIBLE]
You are-- you are a reliable narrator, inherently, in your video.
So this is a citation to something, as something you can include in the video description below.
And oftentimes a lot of educational channels do.
It's like, if you want to do further reading on this, you can include it in the description.
But if you say a statement like that, that because of genetics, because of the hard-wiring of our DNA, some people's mitochondria is more efficient in this way, less efficient in this way.
They're accepting this.
And I think because you're a scientist and because you're making this at MIT, that adds to your reliability.
And the fact that, for your more dubious claims, which in my opinion are the mediation stuff, because psychology is inherently a more dubious field, like anything with people's brains, we have a very, very loose understanding of brains.
Because that's where if any citations go on your video, that's where you should start referencing researchers, because that's where people are going to go, hm, this kind of sounds like pseudo science.
But people have heard of mitochondria.
People know what mitochondria do.
So if you make a claim that some people's mitochondria, not everyone's, acts differently, then that's an acceptable fact
David, I know this is not what you want to hear, but what I actually recommend you do is write a script from scratch.
Write your next draft.
Because you have all this knowledge in your mind already, so it's not like you need to look at it to rewrite it.
But approach it from this completely fresh, new structure, of opening with an anecdote, connecting that to the big question.
Explain the anecdote.
And in the process of explaining it, you can hit the definitions of why people-- or how your body shivers, or how many Iceman, when he meditated, maybe his blood vessels constricted.
I'm totally making this up.
You have to fact check all this.
Then go back out to how this could affect our survival rate in the Ice Age.
If you approach it from-- because this is going to take a completely different structuring.
So I almost don't want you to even look at this when you try your next draft, because you're going to have the habit of thinking about it in the same five paragraph structure that you did before.
Send the next draft that way.
I think in the process, you will end up not including things just naturally.
But if you try to look at this draft and say what do I need to take out.
What do I need to put in.
What do I need to move around.
I think it might be a little confusing to you, does that make sense.
All right.
And I will stick around after class today, if people want to talk more about their scripts.
But I wanted to give everyone a chance to read it out loud to people.
Oh, when you are reading also, this is a tendency that a lot of people have.
And I just noticed it because you read last.
But carry your words through.
You're going to want to talk a little bit slower than you usually do naturally.
And you don't want to drop the ends of words off like this.
Because the effect that that has is you sound robotic on camera, and it's really hard to understand the ends of words.
So that's it for everyone, just make sure that you carry all of your words out to the end of the sentence.
All right, let's do one more and then take a break.
How's everyone--
[INAUDIBLE]
You want to take a break right now?
[INAUDIBLE]
All right, let's do a five minute break and come back
And I need to make a quick phone call, so I'm going to be late coming back in.
[INAUDIBLE]
Go for it.
All right, Nathan, take it away.
And remember, read it like you would actually try to host it
What does food in your fridge start to smell, and where does that icky black liquid come from?
Wouldn't life just be easier if we didn't have to worry about that?
Well, to answer these questions, let's talk about decomposition.
Wikipedia says that decomposition is the process by which organic substances are broken down into much simpler form of matter.
Well, what does that mean?
Basically, in your fridge, on a forest floor, almost everywhere, there are fungi and bacteria that live entirely by eating dead things.
These dead things that make up food can be more or less divided into three categories, carbohydrates, which are like things like sugars and starches, lipids, like fats, and proteins, like meat.
All of these are chemically different compounds, so they each get digested in a different way.
Enzymes like amylases, which yields us starches, and cellulases, which deal with cellulose, the main component of plants, create sugar from complex carbohydrates, which cells can easily get energy from.
Lipase has split lipids into two parts, glycerone and fatty acids, both of which can produce energy.
Lastly, proteases break protein into the amino acids that they're made of, which both releases energy and provides a bacteria and fungi with crucial building blocks.
So how does a-- so how does a perfectly nice broccoli floret start giving off this foul black liquid?
Well, fruits and vegetables are almost entirely water, so one the most basic level, you could say a plant cell is an extremely complex water balloon.
The elastic outside is the cellulose cell wall, and the [INAUDIBLE] inside is the intracellular fluid.
When a bacteria or fungi use a cellulases to break down the exterior of a cell, it's like if I were to pop the balloon.
But liquid and other materials that come are what cause the muck you see in your fridge.
And so what about that smell?
It depends on the food type.
Basically, if a meat starts to get rancid and smell bad, it's because when lipases break down the fat and the meat, a lot of the fatty acids you end up with aren't too pleasant.
If it's a fruit or vegetable, [INAUDIBLE] all the time, the smell happens after that icky liquid forms.
Other bacteria that weren't involved in initial breakdown, move in and start to stink everything up.
So looking at this all, wouldn't it be easy if we didn't have to deal with decomposition, if things just lasted forever?
Well, maybe but probably not.
The USDS found that the average American household wastes about 25% of its food, and a lot of restaurants are even worse.
Sure, that number could go down if there's no decomposition.
But other factors can affect how good a food is, too, like moisture and exposure to air.
And if we didn't have decomposition, what would happen to all that food we throw away?
Food items in landfills are dealt with by decomposes, and allow the nutrients in food to return to the soil and eventually other living things.
If things just sat idle in landfills, we'd end up with a crisis on our hands, eventually.
Aside from just food, bacteria, fungi, and protists are responsible for breaking down other dead things, like trees.
In fact, they're more or less the only things that can do it.
So while your house would never have to worry about termites, who rely protists in their stomachs, the forest would pretty soon be flooded with dead stuff.
So decomposition, while maybe annoying when you can't eat that now fuzzy peach in your fridge, is essential to the continuation of our world as you know it.
And it's pretty darn interesting, too
Just so you know, that was also exactly five minutes.
but keep in mind, the pacing of your reading is a lot faster when you're reading off of a script.
And then you got to account for all the b-roll that you're going to put in there.
So it's clocking in at a little bit over five minutes.
For this one, let's look at the script altogether.
And I'm going to go ahead and make my first point in the interest of time, if I can find my cursor, which is that this script has the same issue that I was talking about with David, which is that I think the coolest part about your episode is this whole part right here, these two paragraphs.
What would happen if we didn't have decomposition, and why it's actually super vital.
But it doesn't come until you've gone through all this stuff.
And I wonder how much of this stuff you actually need.
Like this whole sentence, and actually this one, I think you could take out completely.
Go from why does the food in your fridge start to smell, wouldn't life be easier if we didn't have that to worry about.
You could just say, well, in your fridge, or on a forest floor, there are fungi and bacteria that survive entirely by eating dead things.
And they're responsible for turning your broccoli into the black mold.
This stuff is informative.
I don't think anyone here suffers from the inability to be informative.
This is all very factual.
But I personally don't care about it as much as the stuff at the end.
How do you guys feel?
Yeah
I was kind of zoning out when there was the amylases
Yeah.
It's OK to use technical terminology.
But I think this gets a little too jargony to the point where people are going to have a hard time understanding why they need to know what a protease is.
This is what I meant by David's structure.
We're so used to writing stuff in this is form, where you have the question, and you have the background.
Then you usually go into an example or an anecdote.
Add then you have some sort of conclusion where you say-- your conclusion is, so that's why decomposition is important.
In a video, this is your money shot, usually.
This is the thing that looks cool.
Have you guys ever seen that-- it's a commercial for Sprint where they have the people cutting their bills in half.
Have you guys ever seen that before?
I find that a very weird commercial, because it takes them such a long time to get-- they interview people about how much they're spending on their monthly wireless bill.
It's not until halfway through the commercial that you get the part where they're actually cutting all their bills in half.
And I'm like, it they just opened the commercial with those shots, then they could go into the interview stuff.
But whoever produced that commercial was like, no, no.
We have to establish that people hate their current cellphone plan.
And we have to get them to talk about who they are before we can show that stuff.
That's not true of video.
You can start with the example.
You can start with the big question.
And instead of having a conclusion, you can have a connecting-- instead of this conclusion, oftentimes you revisit the question instead.
So I think you should move your example up.
You should move it up right after the question.
And in David's case, I think you should actually do the example, and then the question.
You should talk about Iceman and then say what is it about certain people that give them the advantage.
For you, I think you should say why does food in your fridge start to smell?
Because it's like a nice, familiar intro into the topic.
But then I think you go into what would happen if we didn't have decomposition.
And that's not exactly an example, but do you see what I mean by just diving straight into the meat in the bulk of your video?
Were you going to say something, Andrea?
Well I just remember when you did your one line or two line summary, which was so amazingly awesome.
And I'd love to see that in the script, of well, it turns out that there are lots of other things that like to eat your food, too.
And that's the hook
Yeah, I think that is where the video exists.
I don't think you actually need most of this stuff, which again I know is like super disheartening for people to hear that the majority of what you wrote isn't actually necessary.
But I think that in the act of writing this, you've discovered what the really compelling part of your script and your content it, which is this
I'd said the big problem is that, for me, it's accepting that in the end what I really wanted to make a video about is how it happens.
Because that's the part that I look at and when I want to figure it out on YouTube, there's nothing that told me how it happens.
And I don't know, that's the problem for me.
Because that's more-- I guess that's the issue, is that it's not as compelling what I want to make a video, as what is probably a better video as what I'm not as interested in as myself
Then I think simply by moving these two paragraphs, basically, to the beginning, prefacing all this stuff, you can have the best of both worlds.
And I actually think that, you're right, you do want to explain what's happening, because that's like substantive science that people can be learning about.
So this is a really compelling hook.
What if we just didn't have the thing that seemingly inconveniences us.
What if things could just last indefinitely?
Is this crisis unrelated to all the stuff with the enzymes that you were talking about earlier in your script?
I think the crisis is more the-- I tried the very in a couple of like-- it's like nutrient cycling and nitrogen are very complex topics.
So basically I tried to very succinctly, sum it up that basically you need to have things returning
Then what if you just simply say you need to have things returning to the soil just like you have in this sentence.
And then you could say, so what averts this crisis, basically.
And then you can go into this stuff, that you have bacteria that break the outside walls of plants.
And all the liquid comes out.
And that's like the muck.
That's what rot is, basically.
But in doing so, drive home the point that this stuff is important and desired.
So you set it up with well, what would happen if we didn't have decomposition.
You wouldn't have stuff returning to the soil.
You'd have just endless material piling up on each other.
So what averts this crisis?
Well, bacteria do this.
And by allowing it to rot and allowing things to go back to the soil, go back to the question of that's why stuff rots in the first place.
So actually, maybe it just needs like a little bit of restructuring more than anything.
What do you think you, Julia?
I really enjoy your informal language, so saying things like pretty darn interesting.
So I think also if you can put the science-y jargon into that informal language, that would make it more fun.
Because that's kind of how you talk, and so if you can explain it that way, that would be great.
And also, you can add images.
So for example, for you saying how does a perfectly nice broccoli florets start giving off this foul black liquid, you could show this black liquid instead of mentioning it or describing it and just say, how does it end up like this.
[INAUDIBLE]
Yeah, that's a great point.
It'll save you a lot of time, too.
I actually don't think you need this sentence, personally, because by then you've established that it is interesting, and you sort of highlight it through your tone.
And in general, again it's totally a personal taste thing, but I try to avoid observations that a viewer could come to on their own, or ones that I could say quickly in passing in the context of a sentence.
I rarely try to have an observation like that merit its own sentence or its own thought
Do you want to touch the concept of recycling at all, like the idea that some things that may seem like waste or dead stuff is actually really useful to other things.
I don't want to complicate this
I actually don't know what you mean by that
Like for instance, mushrooms are decomposers.
Their job is to help speed up the cycle.
And there are other creatures and microorganisms out there that have an important role in this part of the materials cycle, that I'm not sure whether it's worth mentioning that that ooze and that black stuff is actually someone's delicacy
I think that's what you're trying to get at with the whole having the nitrates and everything return to the soil.
And maybe that's something that you can hit in just a sentence.
Like it goes back-- it allows nutrients in food to return to the soil that's vital to the organisms that live in that environment, and eventually other living things
And this is one where maybe having a visual of the food chain cycle, like that whole-- really hammers this down visually for people while you're talking.
As the sun creates-- like just a visual map of just the food chain cycle, starting from the sun and going back to you.
And maybe that's all you need with this to hone that biological concept down for people, that life cycle
Yeah
Going off of what Julia was saying a little bit earlier, so right now, the part where you discuss enzymes sounds a lot like the videos I'm making right now for MITx.
It's super factual and really concise, which is good for those kind of videos.
For these kind of videos, I feel like you need to integrate this information in the greater structure of the story like we were talking about.
So jumping off of what Andrea was saying earlier, if you structure it as, we are not the only people who eat our food.
Bacteria and protists and things like that also eat our food.
They use enzymes like proteases to break the protein in our food to do this.
As opposed to introducing them as list, like these are different enzymes.
If you can figure out a way where the outer story is, they also are eating our food and breaking it down.
And then bringing up these enzymes when it's necessary to define them instead of just giving blanket definitions of all of them at the beginning, it helps keep that viewer engaged, because then they're still getting the same amount of information overall, but they're getting it in smaller doses, and when that information is relevant.
So they're more likely to remember it later on, because they'll see a picture of the meet.
And be like, oh, the bacteria uses proteases to break the proteins down in the meat because they use those as amino acids for themselves, or something like that.
As opposed to giving this list of enzymes
Yeah.
That's a really good tip for everyone.
I think you Julia, that's definitely relevant in your script.
David, for sure.
Maybe a litmus test is if you find yourself with an entire sentence or an entire paragraph that is just a definition on its own, really rethink how that's being done.
Enzymes like amylases, which deal with starches, create sugar from complex carbohydrates, which cells can easily get energy from.
Like that's a textbook definition of what enzymes and amylases do.
And you can explain that stuff in your examples of well, it turns out we're not the only things that eat our food.
Bacteria do that, too.
I don't think it's super accurate to say that enzymes eat our food.
But you know what I mean, never leave an island of a definition out on its own.
I guess.
Like don't have this random-- don't leave random islands of facts just hanging out in the middle of your script, for the purposes of this video.
I think this is an important idea.
I think you can integrate in the context of an example, though.
Does that make sense?
Is that making sense to other people?
Yeah, what were you going to say, Joshua?
[INAUDIBLE] I had the question in my mind that the topic of smell wasn't addressed
Oh, right about here.
Yeah, right now you set up sight and smell as two unrelated things.
But I think you don't have to separate the smell out.
You can say, when the bacteria break down the exterior of the cell, and the crap oozes out of it, it also is leaking out chemicals that cause rancid smells.
So you don't necessarily have to differentiate that it is a separate topic.
One more thing, again this is a personal thing, but I just think the word rot is a lot more visceral and a lot more the familiar to people.
So I know that you talk a lot about decomposition, but maybe if you open it with why do things rot in the first place.
What would happen if things didn't.
That might be a little bit more welcoming to viewers.
Any final thoughts about this one?
Again, I think in general people are in really good shape with their ideas, and the concepts, and the topics of their scripts.
It's really a matter of restructuring it, escaping away from the traditional five paragraph structure and really hitting into money shots right away.
That's going to help your videos the most.
Andrea, would you mind reading next?
Can you pass the mic over?
Everyone gets to hear my nice bronchitis voice.
I'm going to mime through most of this, too.
Mmm, delicious.
Eating foods is one of the great pleasures of life.
And to enjoy foods from apples to candy bars, we rely on one part of our bodies, our teeth.
Teeth are the hardest substances in our bodies-- little animation holding a tooth-- harder than our bones and even harder than iron or steel.
While we chew, our teeth actually experience forces up to 225 pounds.
That's like having a mountain goat jump up and down on our teeth hundreds of times each day-- little animation of a mountain goat.
So why doesn't our jar just crumble under all of those forces-- explanation graphic.
Between your tooth--
Are you guys following along on your script?
Between your tooth and your jawbone there's a specialized piece of tissue called the periodontal ligament, or PDL for short.
The PDL can easily absorb the normal forces that a tooth experiences while we chew, say, an apple, cushioning or protecting our jaw bone from our teeth.
And inside the PDL there are all different kinds of cells.
One type, called mechanoreceptors, sense forces of movement or pressure applied to the tooth.
If the force is large enough, such as biting into an apple seed, these receptors tell your brain to stop biting down-- graphic.
But what if we want to force teeth in a certain direction, like with braces?
As the braces pull on the tooth, the PDL is squeezed in one direction and stretched in the other, kind of like a rubber band.
And here's where it gets really interesting.
To make room for the PDL, cells called osteoclasts come in and dissolve a little bit of the bone in your jaw.
Another type of cell called an osteoblast comes in and then builds up the jaw bone so the PDL cushion can get back to its proper shape holding the tooth in its new position.
Cut to image of the entire body with the skeleton highlighted.
Your jaw isn't the only place where these osteoclasts and osteoblasts alter your bone structure.
In fact, this bony remodeling process is happening throughout your body all the time.
When braces-- where braces use osteoblasts to physically move things around that are already in our bodies, what if we try to use them to replace things in our bodies, like implants?
Implants are kind of like spare parts for our bodies.
And MIT engineers are using the properties of osteoblasts and osteoclasts that are already in our bodies to create a chemical coating for these implants.
Just like in a mouth with braces, this osteoblast coating will create natural bone to help lock the implant in place.
And this is a new section.
Right now, implants are designed to have the same functionality as the body parts they are replacing.
But in the future, scientists could make implants that work better than the original body parts, using bone remodeling to make us lighter, stronger, and faster.
And then, at a certain point, like at the end, I'm actually wearing this weird cyber thing over my eye.
That's the end.
[APPLAUSE]
So you're clocking in at about four minutes, which is good.
And I thought the delivery was also a really nice, too
I got so much more out of it than reading it, which is funny because I don't usually process things that way.
Yeah, but the way that you did it was so slow and dramatic, like the way that you emphasized, it was helping me process it.
It was really good
I like the way that you emphasize the word same.
Lot of people would do the same thing.
They would just sort of accent the word the same.
But you sort of linger on in it, it's same thing.
And I actually really like that.
I think it's a delivery that is not necessarily intuitive to a lot of people, but it works really nicely in that context.
All right, so this has changed quite a bit from your original idea, which was the first on how braces work to more about bone remodeling and rebuilding.
Does anyone have comments off the bat about the script?
Yeah, Julia
I really enjoyed it, especially at the beginning when you're talking about our teeth are stronger than steel.
And you kind of demand the attention.
And also when you mentioned this is where it gets interesting, so I kind of, oh, I should listen to this is the reaction.
The only thing I kind of wanted to learn more about was how can you withstand the mountain goat on your teeth.
That was something very-- it had a large wow factor.
And to me that was like, oh, how does that actually happen.
That's the only thing I wanted to know more about.
But other than that I enjoyed the information.
And it made a lot of sense because it wasn't very scientific, as in didn't have a lot of jargon
I did get great feedback from Elizabeth, and [INAUDIBLE], and Jamie, and George.
I think the crumbling job was Elizabeth
I agree that you did a really nice job.
You don't start with a money shot, necessarily.
Like you don't start with the big picture question, but I think it works in the script because she has so many shareable facts off the beginning.
And it's not a traditional five paragraph set up where she talks a lot about defining what the PDL was, which is kind of what your earlier iteration of the script did.
But I think this is a great example of not necessarily defining every single fact that you need to know about dentistry to get the point.
I want you to follow along because you're setting up sort of-- it's sort of establishing characters.
Like the PDL is a character in the story.
And what all of these shareable facts do is that it creates the persona of the PDL as being something really strong, and something really robust.
And the fact that we're now, as humans, going to try to manipulate that, it establishes the challenge in that, which I think is a really nice thing that it does.
This section is not as related to it, but I think the video is short enough to where you can include it, because it's just kind of a cool fact.
Do you guys find that distracting?
Or do you think it's OK to leave in there?
[INAUDIBLE]
I have a different question to go with that.
So there's something that's kind of missing at the end, which is the four minute question.
I don't know that this is actually a four minute one, because if you add on my comment, it might add some [SOUND] to the end.
I feel like the end ends kind of abruptly for me.
And so I think we get back to your question once we know where we're ending.
But part of me wants-- you asked some really good questions at the very end about what if we manipulate the body to be better, stronger, lighter, all of those things.
Then it asks all these moral questions, of like, is that OK.
I mean, what about the Olympics.
What does that do for people who have replacement arms and now they can throw faster.
There's all this world of questions that have nothing to do with the actual physicality of pulling it off, but it makes you wonder about this super human race that could be created by replacing a lot of our functional parts with things that are better, faster, stronger.
All of the above.
But aside from even going there, I feel like I want you to tie it back to the apple at the beginning somehow, and teeth.
So I don't know if it's possible to allude to all of the moral questions that this might bring up without having to go deeply into it, because we don't have to answer that.
We just have to get people thinking.
And maybe tying it, somehow, back to the teeth, somehow, because that's what we started with.
I'm not sure what that direct [INAUDIBLE] is yet.
But that, to me, would feel-- so you start with-- and the apple is very central and the teeth are a very central part of the story.
And then we sort of go forward.
But forget why we started there at the end.
And I'm wondering maybe you guys have a clever way to tie it back to that concept.
But I feel a little unsatisfied leaving that apple at the front and not coming back at all
Maybe if you added [INAUDIBLE], not just in the future can scientists do that.
But they're already kind of doing that with braces.
And then it goes back to the teeth
There you go.
Right.
And just bring it back to the idea that they're doing all these crazy things.
Think about your arms, your legs, your nose, or whatever piece of your body that you might want to trade out and upgrade.
But every day people around you are doing that with their teeth and we take it for granted.
Just something that-- just a quick tie back to that
Also, and that kind of goes along with the goat thing.
The fact that these implants work better make people want to replace the bones in their body.
Just I want stronger arm bones, or something like that.
So I think, for me, that's another question that I had, what are the implications of these implants for our society?
Are they, right now, are they prohibitively expensive.
Are they-- you don't have to go totally out there.
But I think you're right.
And as an audience member you're wondering, oh, I love running.
Maybe I could run faster if I just traded my legs for different ones.
Like why couldn't I do that?
Are we ten years away from that?
Ten thousand years away from being able to do that?
The only thing is I'm worried that this is opening up a huge can of warms at the end of your video.
So maybe a quick way to sort of gloss over it, and maybe this is a little cheap, but what I would do if I were trying to fix or rework some of the ending-- and endings are so hard to write.
I don't know if you guys have experienced this before, but I've always found the ending to be the hardest part to write in the script.
But what if you went, your jaw isn't the only place where these osteoclasts and osteoblasts alter your bone structure.
In fact, this bony remodeling processes happening throughout the entire body.
And where braces use osteoblasts to physically move things that are already in our bodies, Engineer use the properties of these cells to replace things in our bodies.
Now the idea of building ourselves as bionicle people and replacing our bones with stronger ones may sound like something out of science fiction book.
But when you think about it, that's what the things that most teenagers have to use do as well.
Then add an ending sentence that I can't think of right now, crunch, into your apple.
So that way you've kind of reduced--
Yeah, that's what we need.
It doesn't need to be long, but it needs to tie those concepts.
Yeah, I like that a lot
And I can talk to you more about it afterwards because I can't ad lib a great script impromptu.
But that way you're sort of condensing all this down, because I really love the example of the implants.
But I want your focus of the video to be on braces, basically, so that way you're just sort of hinting at it and then tying it back to the beginning with the apple
Do you-- people know that if your braces come off and you don't wear your retainers they go right back.
Which I just wonder if someone who's listening to this is like, well, if you follows Andrea logic, my teeth have permanently rebuilt their cellular structure in this new space.
Why would I need to do that?
That's a good point
Like if understand what you're sharing in this from a cellular level is actually going on, I shouldn't have to wear a retainer, because my cells have-- things have ground together.
And a change has actually happened in my mouth.
It's not just the movement, but it's physically changing my mouth, right?
So why should I have to wear a retainer if my mouth is actually changed because of my braces?
Can you just add a line in there after you say that the PDL cushion gets back into its proper shape.
And then having a retainer make-- because bone remodeling and rebuilding happens constantly.
I think that's the one fact that isn't in there that might be the crucial point here, that even after you get braces your osteoclasts and osteoblasts are continually remodeling your jaw.
That a retainer make sure that the osteoblasts and clasts reformation is happening in the right place.
You can just throw in a quick sentence there, or a quick afterthought to that sentence.
And I think that clarifies that
I think another thing to point out is that your descriptions of the science that goes behind it don't seem very heavy.
So if you wanted to add more science, maybe, I think that would be OK in this video.
I wasn't distracted by the science that was there.
So if necessary, to add more about the PDL, that could be done
Well, I originally had a lot more about the PDL and I ended up cutting it out
I mean the reason why I suggested that you take some of the stuff about the PDL out is because the video was really starting to seem more about the PDL and less about bone remodeling.
So I agree that the way you've worded things, it's not too jargony.
So if you wanted to add more about remodeling and rebuilding, and more about the osteoblasts and osteoclasts, maybe how that bone remodeling happens to you in your leg bones all the time when you grow.
I think that's a perfectly nice place for that addition.
If you want to make room for it, I do think that-- what is it-- the mechanoreceptors part.
It's a neat thing that I'm OK with having in there.
But if you need to make room for more science about the osteoblasts and osteoclasts, I think it's OK to take out, too.
But I would hesitate to talk more about the PDL, because then you're really getting into a different type of video.
Which if that's what you want to pursue, that's totally fine.
But I think to me the video is really more about bone remodeling than it is about the periodontal ligament.
So that's my only hesitation with adding more facts about the PDL.
Does that sound good?
All right, any last thoughts Andrea's?
No.
All right, Paul, do you want to go next.
Or PJ.
Paul is actually a PJ, as discovered last week at the end of class
My parents told me the other day that I didn't have a first name for the first three days I was born
Wait, so you're formally PJ?
No, I'm formally Paul.
But it took them three days to [INAUDIBLE]
But everyone calls you PJ?
Uh, it's 50-50
OK, because I was like, gosh, why didn't we know this sooner
Yeah, this was my reaction last week, too, at the end of class.
Oh yeah, the doc is gone.
Siri's updating it-- or Sari.
Sorry, I don't know why I keep doing that.
Wait, so your parents knew that you were going to be a PJ, they just didn't--
No, no, they didn't know what my name was going to be
Oh, ok
It's amazing how many people do that.
Did they know you were going to be a boy?
No.
They did it old school.
[INAUDIBLE] No, I don't.
I'm the youngest [INAUDIBLE]
You run out of names
Is it up now?
Refresh.
Yeah, it's still showing up blank for me.
Do you mind reading it from your Google doc while Sari figures that out?
All right
All right, laptops down.
And remember, read it like you would host it
Let's take the small foam box, Orca One, and cut a hole in it.
Now, let's put this box in water and see what happens.
As you can see, the box unfortunately sinks due to the weight of the added water.
Now what if that box contained cargo, or oil, or even people.
That would make for a very bad.
Now let's take Orca Two and do the same thing.
You can see that Orca Two do not sink, all though it is sitting at an angle.
So why did Orca Two not sink?
As easy as it sounds, this simple demonstration is essential to the design of huge, complex ships, ships that are responsible for safely transporting 90% of all our stuff.
As naval architects, how do we design ships carrying our stuff to make it into port safely and not sink?
Well, let's take a look inside.
Here we have Orca One and Orca To from before.
Even though these boxes do not engage in international trade, they behave just as 1,000 foot steel cargo ship would.
If we remove the top the boxes and take a look inside, we see that Orca Two is divided into compartments by these walls, called transverse bulkheads, while Orca One is not.
These compartments are water tight, meaning even if damage-- [INAUDIBLE].
So these compartments are water tight.
Even if damage occurs in this part of the ship, water rushing in won't go into other compartments because the damage is isolated.
We refer to Orca Two as being subdivided.
It is unclear when subdivision started being used in boat building, but accounts of Chinese trade ships as far back as the fifth century indicate that water would enter ships without sinking.
So how exactly does this work?
Well, when we divide the ship into water compartments, we are limiting the amount of water that can enter the vessel.
If we divide a ship into 10 equal watertight compartments, and one compartment sprung a leak, only that compartment would take on water.
This would likely cause the ship to heel and trim, but it would not cause a complete loss of shipping cargo.
And the ship would be able to hobble back into port and get repairs.
But why is it so important, you might be asking.
Because ships are huge and they carry a ton of stuff.
A ship the size of more the four football fields can carry 715 million bananas, that is about one for every European.
Also, US takes in almost $2 trillion worth of goods every year through ships.
So by subdividing ships, we're ensuring the safe delivery of our stuff and the health of international World Trade.
But sadly, even with subdivision vessels still can sink.
It is both expensive and impractical to design a ship that can withstand any amount of damage, so Naval architects consider the likelihood of damage in writing ship design regulations.
Also, the use of computers in naval architecture allows us to simulate a likely damage scenarios, so we can better prevent them from happening.
So the next time you use your cellphone or eat a banana, remember the amount of engineering that went into safely delivering it to you
You are good on time.
It was about four minutes.
It's up now?
So you guys should be able to access the script now.
Does anyone have thoughts about this off the bat?
PJ slash Paul, there have been moments in class when I feel like your personality has come out and I've your eyes sparkle and your voice-- I feel like there's two versions of you.
There's probably serious coastguard man.
And there's like [INAUDIBLE].
And I think the thing that we need to make sure we tap into is not super serious coast guard man, because when you read that just now, I was like, he's talking like this and I'm kind of board
That was me inflecting
Remember that you have to exaggerate in front of the camera
You're smiling.
You're dynamic.
Your voice is alive.
I know it's in there because I've seen it.
And when we talked about giving fifth graders pornography, it was definitely there
Wait, what?
Teaching faux pas that I had back in the day.
So I know that that personality in you is totally there, regardless of the content.
I feel like you need to find a zen way to channel that part of you, being whatever video content you do, because there's something innately very fascinating about this.
Like content's really cool.
The fact that you have this is really awesome experience that none of us have and I totally think is amazing.
But if you gave it the way that you gave it right now, you would kill us.
So we've got to figure out how to get that energy in your being
I agree with you
What was your two line summary of the whole
So my initial one was wasn't that great.
But like it's pretty much taking something simple that people know how to do, like floating and sinking, and how there's a lot of complex things that are designed and built to carry a lot of really important things that we depend, based off very elementary principles
I will show you your two sentence pitch.
[VIDEO PLAYBACK] -Yes.
-It feels a lot like a document.
-What do you mean by that?
-Kind of like the image I get like kind of walking through a museum.
[INAUDIBLE] -Tell us what it is.
-So it's about subdivisions, subdivision in ships.
So if you take something that floats, like a shoe box, and you put a hole it in, it'll sink.
But if you divide that shoe box into watertight sections, this one compartment might not cause it to sink.
And how we've advanced to a time where a ship can be extensively damaged and still stay afloat.
And people won't die.
[END PLAYBACK]
Sort of like the first little section that you gave about it, it's almost like I'd like you to say that and then say hey, we can just do this experiment live right now.
Why don't we just do this, right now.
Let's take this-- and here's a small foam box, that I've decided to call Orca One
Right
And I think again, I mean the showing not telling is really hard to script when you haven't actually worked with the video stuff that much.
And maybe as you shoot, you'll realize, oh, I don't actually need to say, as you can see.
I can just say the box sinks.
And you don't need to say-- you don't put any contractions in your script, which I think is interesting.
And when you deliver the lines, you're not going to say do not.
You're just going to say it doesn't sink
I say let's right there
Oh, OK. One apostrophe
Doing this, you're probably going to ad lib based on what happens, just like the crazy Russian hacker guy who's like, OK, that didn't [INAUDIBLE].
And that's what gives it, kind of, your personality will come out as this is happening
So with the introduction I was-- Elizabeth and Sari's comments kind of allude to there's got to be some sort of like-- just to know what we're talking about.
So I did that.
And then that turned out to be bad.
It was a bad introduction.
So the George said, just get right into.
Just get right into the box and how this-- so I'm kind of confused, though, what--
What you just said was perfect
With?
We all know that some things sink and some things float.
But it's actually not that simple.
And this fairly rudimentary concept is something that naval architects use every day
Wait, but that was the original-- do you mind if I pull up the Google doc that you sent me before?
I just felt like it started really abruptly.
And you needed just a little, little, little bit
Yeah.
Even just having a mock Navy ship smashed and then watching it sink or float.
Because like, what you're talking about is the grey zone in between sinking and floating.
And you want us to get in that grey zone quickly with you
Yeah, I like that you're starting a right away.
Like I know that we had talked about-- Josh had mentioned that during his workshop, like to start in the action.
But I think that your intro itself is visually interesting enough and topically foreign enough to most people, to where that in itself is a bit of a hook, too.
And by setup, we don't mean like an entire paragraph.
I think like that one sentences is fine, actually.
And when you guys shoot, I really challenge you guys not to use a script, which sounds terrifying.
And often times that'll just mean, what line do I have to say here.
I'm going to say this line.
And you put down your script.
And you deliver it ad lib.
But it's going to come out so much more natural that way.
When you talk in real life, you turn your head a lot, that's what makes you seem like a human, versus a humanoid.
Not saying that the delivery was this time.
But there are a lot of things that people do naturally that they don't realize.
And so I think it's fun to point it out to people, because when you're shooting in front of a camera, it's very hard to remember those things.
It's really hard to be natural in front of a camera, basically.
But like all of you guys were saying-- or some of you guys were saying, like I'm not good in front of the camera.
And I watched some of the raw footage from class.
And you guys are all great.
And it's technically in front of a camera, right?
So you're just not looking and not aware of it.
So it's somehow being able to extrapolate how you are in this classroom and carrying that over to when there's a camera right in front of your face.
And I do think that it's the little things, like sometimes you tilt your head.
Sometimes you look off and not directly in front of the camera.
Blinking is a big thing.
I don't know if I mentioned that in earlier classes.
But if you look at your footage and you're like I seem off and I don't know why.
I feel like my voice is fine.
I feel like my pacing is fine.
Try to notice how many times you blink.
A lot of times people will keep their eyes for a really long period of time.
And they don't blink where it makes sense in their speech.
And so they're just talking like this
[INAUDIBLE] who does this.
When he gives presentations it's totally creepy
Yeah.
Yeah so--
I'm like, dude, you have to blink
Blinking, head tilts, looking off to the side like you would normally, those are little techniques that you can think about when you're hosting.
And it's sort of counter intuitive to force yourself to be natural, but sometimes that's what you have to do
I really recommend, if you haven't already-- I think most of you have already.
But for your reflections, think about doing a vlog, because vlogging is like the basest form of video creation in that you like literally stare at your computer.
And if you want to get comfortable with the idea of a camera, you can set up your camera as though you were recording yourself for your video, or something like that.
And just talking to it.
Because we have no expectations for your daily reflections.
You can be like, I was really scared going into class today.
And then you can stop thinking.
And you can look off camera.
And you can say um.
And we won't judge you at all.
And it gives a good reference point for when you do have a polished video.
It's like do I have even some of the quirks that I showed my vlog
Yeah
And it just gets you more-- I don't know, I was really uncomfortable in front of the camera.
And I still kind of am.
But vlogging and doing that process and seeing how, oh, if I make a mistake, like forgetting what I'm going to say, I can just cut that out and seal it together.
It really gets you a lot more comfortable with just seeing your face on the screen, which is a really foreign experience for people
I think the overall format of the script is pretty strong.
I don't think you need to change a whole lot in terms of the format.
Adding a little bit of context at the beginning would help.
Stuff like this sentence, though, I don't think you would ever say these two sentences in real life, in your speech, right?
That's what I said.
I think while he's doing it, he's going to be ad libbing
And again, you don't have to script it.
It's OK if you deviate from the script a little bit when you actually deliver your lines.
But this sentence, that would make for very bad day, or the one that at the end of your video, you know, that's why we can't enjoy-- or decomposition is awesome.
You don't necessarily have to say that out loud to people, they're just going to kind of know already.
The one comment I wanted to make was that the context-- I think introducing a little bit of context at the beginning will help with this issue, because right now without it this jump seems really huge all of the sudden.
Talking about sinking and then talking about economy and money, all of the sudden.
I don't know if any of you guys felt that way, too.
But it seems a little like, oh man, first we were talking about cardboard boxes, now we're talking about trillions of dollars.
And I think it's a good point to make.
Because you do want to take people out to the bigger context.
Yeah, but it is a little abrupt.
And instead of saying, but sadly even with this subdivision, I think you can set this up as a question.
So we have the subdivision, why do ships still sink?
And last comment, then I have to hit the road.
But if any of you didn't get feedback from me and you want it, because I have to leave a little early.
Just email me.
My [INAUDIBLE] address is not the one that grants me access to Google Docs.
So if you want, just email me and I'll give you my gmail account so that I can actually read it.
The naval, ship is such a cool thread throughout this, that I don't think you necessarily-- like if you can use the cardboard box as Navy ships, and not have them me be just abstract boxes.
And if the cargo ship carrying bananas is carrying military personnel.
You don't have to go outside of that genre in order for this to be a really cool video.
In fact, I think it's cooler if you stick with the military theme, because that's something that's actually-- especially to middle school boys who like blowing up things-- that's even more authentically cool than bananas.
And plus, if you can even make Orca, the box, look a little bit like--
I already made them
You made them?
But even if you just [INAUDIBLE] and put some stars on them.
It doesn't have to be crazy fancy to still get the idea that this is like a Navy ship, right?
But I think that if that Navy theme hums throughout the whole thing, then it's like that becomes a really cool theme
Yeah
I gotta run.
I'm gonna miss my train.
But you guys are awesome.
[INAUDIBLE]
Julia, do you want to say something?
Thanks, Jamie
When the football fields and bananas were introduced, that was kind of taking me out of the video and making me think of monkeys.
And then at the end, also, you're mentioning cell phones and bananas, but you never mentioned cell phones before that.
So I do like the idea of kind of keeping with the military theme.
Also some of the-- there was a part where you had to questions-- a question at the end of the paragraph, and that was for two consecutive paragraphs
Yeah, right here.
[INTERPOSING VOICES]
That's a good point
Taking out of the video.
But other than that, I really enjoyed the content.
This is very different from the original draft, but I would very much enjoy watching this video at this point
Yeah, originally you had talked about Archimedes' principle and buoyancy, but I think this is a lot stronger because you're actually getting into why the compartments itself
So just like pretty much fix those questions and take out the whole merchant marine-type thing
Yeah, I think the point that you're trying to make with those examples is basically just the sheer importance of making sure your ships stay afloat.
And I think you can do that with other examples that are a little more unified to the theme.
I will say that this stuff at the end, like this sentence right here, no, this portion right here.
If you want to expand on that a little bit, I think it would be interesting, because it's more of like a current event engineering problem.
And again, like that makes the video more unique than a typical engineering textbook.
You're talking about what are the actual problems that I myself as a naval architect am dealing with.
And you can talk about the-- I don't think you need an also right here, because this is really a continuation of this explanation.
Ships sink because they're expensive to design.
So you have trade offs in the engineering of it.
But we use computers to simulate the damage, so we know exactly how to subdivide them and what the best way to design those subdivisions are.
And then instead of this sentence, I mean this point is a little bit different than the point that you're intending, which is that very simple principles are what allow huge, seemingly complicated machines to work.
So maybe you can somehow revisit that point at the end instead of this sentence.
I think that would end it a lot stronger
So was your advice generally to drop the cargo ship
That wasn't even in there.
I added all that.
It was more, I'd say, the equations and stuff like that
[INAUDIBLE] the draft that he sent us, and there was a big part about Archimedes principle and buoyancy.
Which again, is super informative, very accurate.
It's the type of material that they're going to learn in school.
But it isn't necessarily vital to-- this is more of an engineering video than it is a physics video.
And I think that's very valuable because there's actually not a lot of material on this out there.
So I think it's more unique [INAUDIBLE].
I'm going to give him a chance to-- like someone who's teaching Archimedes principle may find this video really [INAUDIBLE] in class because it gives them a different context.
So you took out this big chunk, but I think that you replaced it with something a lot better for the video
I'm looking at simple demonstration is essential to the design of huge, complex ships, ships that are responsible for safely transporting 90% of all our stuff
Wait, where is this?
It says new angle way up at the top
Yeah
I mean-- I kind of thought the container ships are sort of a theme in the entire thing
Yeah, I don't think cargo-- I'm not saying get rid of the cargo stuff at all
OK, because I mean, that's my bread and butter is merchant ships and not-- because I'm not really in the Navy or anything
And then the tie-in with the cell phone thing is that safely transporting 90% of all of our stuff.
If you take like a zoom in on a cell and flip it over and it says "Made in China" that's sort of like how did this get from China into my hand
I don't think that you should get rid of the cargo stuff completely.
Sorry if it sounds like we're having competing opinions.
I think the thing that throws us off is the connections to the examples you make aren't necessarily immediate.
So I hear four football fields can carry 750 million bananas.
Wait, why is he talking about bananas all of the sudden
I was just trying to say like how-- yeah, I got it.
I'll try to revisit that
And maybe it's as simple as adding a single sentence right here, about they carry a ton of stuff, a lot of products that we have to import from other places.
Things like bananas.
And a ship the size of this can carry 750 million bananas.
So it just sets up the example a little bit more clearly, if that makes sense.
Any other thoughts for PJ?
So I think that we're at a good stopping point here.
I don't want to hold you guys over today.
So we have Kenneth and Joshua left, right?
So tomorrow we will finish up the table reads with their stuff.
And then I'm going to give my final talk on post production.
So this is everything from editing, to music, to thinking about what you do with the footage that you take.
We'll also tell you tomorrow what groups you're going to film in.
Was anyone planning on filming stuff tonight?
No?
That's fine.
Just use tonight to rework your scripts.
As far as deliverables and assignments, I think really reworking your scripts tonight and tomorrow should be the main focus.
So I'm not going to have anything particularly due.
How about all that footage that you shot last week, I'll let you guys tinker around with the editing of that after I give my editing lecture.
Does that makes sense to people?
So for tonight, rework your scripts.
And for people who haven't, I mean Kenneth and Joshua, if you want to update stuff based off of the things we talked about today, that's fine.
And you can just send Sari an update.
Make sure to do your daily blogs, though.
That'll be really helpful.
And then just come to class ready tomorrow to finish up the two reads and then do some editing.
Does that sound good?
The first cut of your video is going to be due this Friday.
Do you think that is feasible and reasonable?
So this would be the first draft of your video, basically.
Like you'd have to film everything for it this week
Is there a day where we're submitting the shots--
The shot list?
Yeah.
I would like to see that by Wednesday, if possible.
But that's really more for your benefit over mine
And Wednesday, Thursday and Friday will be in class workdays
Friday's the screening.
What I could do-- so this weekend is a three day weekend.
We don't have class next Monday.
If it would help people to use that weekend to film more, I could have the rough cuts due-- or we could screen the rough cuts on Tuesday instead of Friday.
Would that help people?
My question is, I guess, what are like the things you generally would do between a rough cut and a final cut that would like--
So, I will talk a little bit about that tomorrow.
I'll actually show you the rough cuts of our old videos.
But a rough cut takes all the footage that you've shot and puts it together into a draft that doesn't have music, and may be missing some scenes, or may require reshooots of scenes.
So usually what happens between a rough cut and a final cut, in addition to music and maybe better editing, is you almost always end up reshooting something because it doesn't look that good in the context of all your other footage.
So the only reason why I had a rough cut due so early is because I wanted to leave you guys enough time to reshoot if you needed to.
But if it's going to be too much work to get the rough in place in the first place, I'd rather you really try to get as good of a rough cut as you can than try to rush one together
[INAUDIBLE]
Not necessarily.
So if it-- are we on consensus?
You guys want rough cut due on Tuesday instead of Friday?
Because that totally works for me.
Basically, after tomorrow, and aside from the screening of the rough cut, every other day of class is work time for the projects
And that next week it's Tuesday, Wednesday, or Tuesday, Wednesday, Thursday?
Because doesn't the class end--
The class ends-- so today is the 12th.
Yeah, so the class ends that Thursday.
It was sort of optimizing like leaving enough time to get the rough cut, but also leaving enough time to give you time for reshoots.
Our screening is the evening of the 22nd.
Yeah
But then what happens, though, the last week [INAUDIBLE]
So the last week of IEP we'll pick scripts to produce for season three of Science Out Loud.
And if you guys are interested and if it fits into the casting, then we would produce your video, basically.
I'm I'm sorry to the SUTD guys, you have to go back.
Yeah.
You could stay another week.
So that's why we end so early, is because you guys have to go back and then we're producing season three in parallel.
So that's all I have.
I'll make the rough cut due Tuesday.
If you finish stuff before Tuesday and want me to look over it, I'm happy to do so.
Because I know that the turnaround is insanely quick for this.
And I know that we're cramming a ton of stuff into these three weeks
Do we need to meet up with our groups to film over the weekend, then?
No, and tomorrow we were going to hand out assignments for who your groups would be.
If you can shoot stuff on your own, like b-roll, feel free to do that.
Like you don't have to just use class time to work.
If it's more convenient for you to just catch some stuff on your own, do a tripod selfie type shot, go ahead and do that.
But no, you don't have to just only work in class.
Everyone has their own camera and their own equipment, right?
OK, yeah.
So tomorrow we'll do editing, finish the table reads.
You'll have the rest of the week to work.
We will screen rough cuts on Tuesday the 20th, then.
But that means that you won't get feedback from your peers until Tuesday the 20th.
That OK?
So basically your rough cuts are going to be so good that you're not going to have to reshoot anything.
No, I'm just kidding.
It'll be fine
How good does our video need to be in terms of the visuals
I mean I totally understand that you have a week to do this, basically.
So if you can't, you know, do a full-fledged animation, you can't fully realize those things, I completely understand that.
This is going to sound stupid, but just try your best.
Because that's why I have-- creating the rubric for this class was really hard, actually, because I knew that all these constraints were here.
And so I couldn't raw scale points, like 10 points it's lit correctly, because that's just unreasonable.
But what I really want is the intent to be there, the thoughtfulness, the understanding and self awareness of maybe what's lacking, what you'd like to develop further.
If you can't-- if a film location falls through at the last minute and you can't film there, stuff happens.
I totally understand.
If you can just maybe write what you wanted to do instead, that's totally fine.
But--
Same thing with animations.
I'm sure some of you have very-- especially with the fractals.
Zooming in on fractals, that's a difficult animation to accomplish, especially if you haven't done animation before.
Think of using still images, because it's really easy to overlay a still image, or even something you draw yourself.
And it would just kind of like-- what Science Out Loud videos do.
As opposed to making this really complex, like multiple moving pieces.
The more moving pieces you have, the harder it's going to be to do faster
We will be around during class every day.
So if people want specific help, like Sari's done some animations herself, too.
So we can help you with that stuff if you have specific questions.
But shoot for the best you can.
Definitely don't be like, well, I'm going to leave this out because I'm just going to justify to Elizabeth later that I couldn't get it.
Try to do the best you can.
But I understand the constraints that you're working under for sure.
So don't stress out about that
And feel free to email us, because some things like after Elizabeth's editing lecture, there might be something like, you're in the editing program, and it might take you three or four hours to figure out where this tool is and how to do this one thing.
Like my audio disappeared, what?
But if you bring it to one of us, chances are we've encountered it before it and can fix it in a couple minutes, hopefully
Sari's probably a lot better at that than I am, just to be honest.
Does that sound OK to everyone?
I do apologize for the quick turnaround.
I know it's a lot to cram into three weeks.
So I understand the limitations that we're working with.
All right, well I'll stick around after class if people want to talk about their scripts more, or have other questions.
But you guys are free to go.
I'll see you tomorrow.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, today's the last official day of lecture.
We'll finish up Joshua and Kenneth [?
Reids ?
], and then we'll jump right into post production.
Whenever you're ready.
Laptops down
The science of search.
Have you ever found it difficult to find a particular item in your house?
Let's say you have lost a pair of gloves.
And you spend an entire afternoon looking for it, but you just can't find it.
You have encountered the same problem that companies like Google or Yahoo try to solve every single day, the problem of search.
Just like a house, which stores about 1,000 different items, Google stores 45 billion different index pages of information.
If each page of information was a sheet of paper and we stack them high altogether, we would create a tower 610 times taller than Mount Everest.
How can a search engine like Google find your results so quickly, when we find it so difficult to find a pair of gloves?
Well, it turns out, that searching on the internet is kind of like looking for a person in a hotel room.
Let's suppose we're looking for a guest called James.
To find James, the simplest way would be to run to every room nearest to you until you find him.
But that would take very long.
There's a better way known as binary search.
Let's say the people were arranged in alphabetical order in increasing numbers of the hotel rooms.
We could run to the room in the middle and check if the name of the person is James.
If the person's name starts with the letter before J, we head to the right.
If not, we would go to the left.
We then head off to the middle room of the newly sectioned area, and we rinse and repeat.
Eventually we'll find James, just like the first method.
But we found him much faster than using the first method.
How much faster would that be?
Well that depends on the people staying in the hotel-- the number of people staying in the hotel.
Let's say it takes ten seconds to knock on each hotel door, and that's 500 people.
It will take about 80 minutes for the first method, and one and a half minutes for binary search.
But if there were 1,000 people in the hotel room, it would take 160 minutes for the first method, but only 1.6 minutes for binary search.
Now that's a whole lot of difference.
A name like James is just a word, but companies like Google taking searches for long combination of words, making it a little bit more complicated.
Just like identifying the word James from its first letter, Google identifies unique characteristics of search phrases using over 200 different factors.
Also, notice how the better method depends on prearrange the people in alphabetical order?
Company scientists are always actively looking for better methods to sort, manage, and eventually retrieve the data.
In the same way, simply by placing your home items in location where they have a natural relationship to make it easier for us to find them.
For example, the TV remote goes near the TV.
The shoes goes near the shoe rack.
The coats go near the cupboard, and the winter gloves goes in the winter jacket.
Ah-ha, so that's where my gloves are.
And that's the story of search
You're clocking in around three to four minutes, which is a really good place to be.
All right, Joshua's script is up on annotation studio.
Let me turn this on
Turn to this [INAUDIBLE]
Oh, sorry.
Joshua, this script has come along a really, really far way.
This is really tight, very good.
I think the flow of it is very natural.
I also thought your delivery was very natural.
I don't know if you've been practicing or maybe you've just felt a little more at ease today, but I think the way you pause at certain words, and just sort of the natural feel of it is-- it's nice.
It sounds like you're really just talking to us
I think I learned a lot of Andrea
From Andrea - yeah
[INAUDIBLE] yesterday
When she said-- what was it that you said during class?
Remember--
It was basically just observing how she presented that I learned how [INAUDIBLE]
Oh, you were sort of copying her style a little bit.
Let's see, here we go.
I think something that works really well for this script is that you build complexity.
Which is a topic that we sort of talked about earlier in the class, but I think is hard to really understand unless you see a specific example of it.
But he starts off with a very simple example of losing your gloves, and then he clearly relates it to how Google searches for something that is far greater in number and complexity.
But he connects it very clearly to the glove analogy, which is why I think it works.
I think this explanation of binary search is very clear, because you're relating it to the context of the hotel analogy.
I know in earlier iterations of your script, you were just describing binary search as a concept where people take a set of data, and then they split in half and then they search the data, and then they split in half again.
I think that's the tendency that a lot of us would have in defining concepts and topics.
But for me, personally, I think that this definition works really well in the context of this video.
Did anyone have any thoughts about the script up until this point, up until right here?
This is purely cosmetic.
I thought the script-- I definitely understand the basics of binary search after listening to you, not even reading the script.
I think that's really good.
Just something I do, and this is-- you could take it or you can leave it, but whenever I do 610-- I don't know, sometimes if I did hundreds of times of the Mount Everest, that kind of-- it could be 100 or can be up to 1,000 so it scales it a little better, but that's very nick-picky I think.
And the other one was over a 1,000 different items, maybe just thousands of items in your house or something.
That was it, though.
I thought the rest was really good
I thought it was really beautifully written.
Those layers of complexity, the way that they-- I'm not someone who processes particularly well just by hearing something.
But the way that you brought me from a simple concept, to making it more complicated and more complicated, and then bringing it back to the simple.
It was a beautifully designed piece, really well
But keep in mind this has happened over several iterations.
And I wouldn't feel discouraged if maybe you feel like your script isn't in as tight as a place.
Because if you look at your earlier iterations, I mean it was a very, very different idea, a very different script.
And we talked a lot after class.
Pretty much every day last week.
But I think-- do you feel happy with how it is right now?
Yeah, [INAUDIBLE]
You feel happy.
I think the challenge is going to be in realizing this visually.
Because the script itself is very tightly written, but executing the whole hotel analogy, I think, will be really key.
And it's nice that you actually have access to a hotel since you're staying in the Hyatt right now.
But scaling back far enough so that people understand, oh, the key to the reason why this works is that people are arranged alphabetically in ascending, numerical hotel rooms is a really vital piece of information that you can use the visuals to convey really well.
Let's see, what about the second half of the script?
I don't think you need this last thing right here.
I mean that's what I was trying to say yesterday.
That oftentimes we feel like we have to continue the ending longer than it needs to be, like this is why this is awesome.
And that's the story of search, and this is the definition of prime numbers.
It reads a little-- at the end.
I personally just don't think it's as necessary.
But I like how you tied it back to your opening.
I think that's very strong.
Does anyone have any comments about this at all?
On the 80 minutes for the first method and then 1 1/2 minutes for binary search, you explain how you derive the 80?
Because it's 10 seconds and there are 500 people.
But then how do you get the 1 and 1/2?
And I guess my point is you might not need the let's say it takes 10 seconds to knock on each door
Yeah, that's an interesting point, because you could explain the calculations you did to get to the binary search estimate.
But, again, I don't think it's super necessary to the point of the video
You don't need the other calculation either and that'll make it a little cleaner.
Because I really think the 1 1/2 minutes is very dramatic in the comparison.
And I started to picture little pods pop up
Because I was also thinking about they could be instead of 500 to 1,000, it could be 500 to 5,000, but I don't know how-- I'm not so sure whether how realistic that would picture be of 5,000 people in a hotel
But, I mean, I understand why you brought up the specific numbers, because you're trying to show sense of scale.
That the difference between the two methods isn't as exaggerated when you're only looking through a few hundred items.
But when you up that to thousands, or even close to what Google searches, than the difference in the time becomes super, super dramatic.
In that sense, I mean, I personally am totally OK with having those numbers in there, because I think that the script is written naturally enough to where I'm not overwhelmed or distracted too much by the numbers
I had one thought.
And I know the hotel concept works, because when I think of a hotel, I think of a very simple box with a very linear line through it and rooms going off in different directions, which is why it works.
But I wondered if having instead of it being a hotel, it's looking for something at MIT.
Because then instead of it being grounded at the hotel, it grounds the film on campus
Yeah, it could work with-- but I'm not so familiar of the campus.
Is there a place where--
Totally
--it's very uniform--
--very The infinite corridor, down the middle hallway, is a very, very standard place with doors.
That was the only thing that fundamentally felt like if we could shift it, it would help ground this, instead of being at the Hyatt, to being here on campus
Yeah, that's a nice suggestion.
And there are a couple fun ways that you could shoot that if you wanted to--
Getting 5,000 people at MIT is very easy
If you wanted to shoot centered like this and start-- whoever is filming you, start relatively close.
And you have the classrooms branching off this way.
Does this drawing make sense to people?
Yeah
OK, instead-- you'd have to find some sort of dolly, maybe take a rolling chair.
But then slowly pan back as he says, but if there were like 5,000 people, and then it's a nice reveal and it's an exaggerated angle too
[INAUDIBLE] knocking on every single door
Yeah, yeah, that'd be a nice B-roll to put there
Or even all-- I mean having a little fun.
It's not about [INAUDIBLE] looking for days and then all these funny buildings on campus.
I mean not that it takes up all our time, but it's a fun way to ground this really and being about an MIT experience
Do you want to say something?
I think a really good place for that, and that's where I imagined, is Building 4 if you go on the first floor.
That it's just a standard hallway, and all of the rooms I'm pretty sure are classrooms so they're available if you want to enter them.
And then also there's an exit to [?
Killian ?]
Court.
That may be a nice location, because there's the dome and grass and all that
Any final thoughts about this script?
Again, I think it's come a super long way.
This is pretty much ready to film, I think.
All right, Kenneth, you are next.
Do you mind giving your mic to him?
Do you want to just do some of this?
Not the whole thing.
Yeah--
It's exactly talking [INAUDIBLE]--
Yeah, but I don't know.
I just sent-- I just checked the link it has the doc-- you know what, Google Doc has everything.
There's still a few paragraphs below that.
It's OK, I'll just read what I have
When we read together, I'll just pull up the Google Doc
Yeah, sure.
OK, how do you feel if you organize a party and no attends?
Pretty much like this one right now.
And then it'll show me at the head of the table with lots of food and no one.
And I say, oh right, I forgot to send the invites out.
And then they'll cut to a shot of me dropping the-- sticks the envelope into the mailbox.
And I say, hmm, wait a minute.
I guess the only way that anyone could attend the party right now is if they actually travelled back in time.
But if they had received my invite, they would have traveled back in time and I would have seen them, and that means I may not send out this invitations.
Oh, right.
And then after that would be-- sorry.
This is actually what we call a time paradox, but let me just go in and talk about-- OK, never mind.
I'm ad-libbing right now.
I'm sorry
That's OK.
In your real thing, you'll probably be ad-libbing
Yeah, then somehow there's a transition.
There will be a transition so just [INAUDIBLE].
But wait, could we actually travel through time?
This was in fact an experiment conducted by Stephen Hawking in 2009 to show that time travel was probably improbable to be invented, even in the future.
But wouldn't being able to travel through time be cool?
We could submit that homework that was due yesterday but only completed today or head to the really cool party that I just organized.
But how could we time travel and is it actually possible?
A lot of what we know about time travel, all what we think we know, comes from Einstein's theory of special and general relativity.
Now this is all part of what we call modern physics and is still being explored and tested as we speak.
It doesn't mean that all of if is wrong.
It just means that we're so unfamiliar with much of it, as we are with the universe around us.
Now back to time traveling.
When you think about it, we are all actually time traveling, but not in the manner sci-fi movies has often depicted them.
We are traveling through time at a pace of one hour per hour.
In other words, as our bodies experience an hour of time, the wall around us has also experienced an hour of time.
It sounds simple, but stay with me.
This is where it gets interesting.
Within Einstein's theory of special relativity, there is a phenomenon he describes, which is known as time dilation.
And there will be a subtext that says, dilation means to expend.
Now, the gist of this phenomenon states that the faster you travel, the slower time passes around us.
In other words, at high speeds, we could be traveling at a higher than one hour per hour.
This is, however, only noticeable traveling at really, really high speeds, near the speed of light itself.
And then the c equals the 3.0 times 10 above 8 meters per second will appear overhead.
And say I invent a spaceship that could travel at 90% the speed of light and I have a pair of newborn twins.
And I'll have two babies in my hands, cartoon babies.
And one of them, I'll put into this spaceship, and he will be in a spaceship.
And I'll send this spaceship off to space for a year.
After the baby comes back for a year, the baby will have aged a full one year, while the one that remains on the earth will have aged approximately 2 and 1/2 years old.
We can see that they have experienced times at different rates and the baby on earth had experienced times about a 2.3-- has experienced time at a rate of now 2.3 years per year where we travel at [?
.049 ?]
c. If my spaceship was capable of traveling at even higher speeds, now I fling the baby's to one side.
This rate would increase exponentially, as shown in this particular graph as you travel toward the speed of light, which is-- there will be a graph overlay.
As you travel towards the speed of light, the magnitude of the time would change dramatically and exponentially.
We could potentially travel forward in time, we just have to travel extremely high speeds to do so.
But, I think the question on everyone's mind is, how do we travel back in time?
I mean surely isn't that a lot more interesting and a lot-- sorry, I need to scroll back.
Isn't that a lot more interesting than travelling forward in time?
Do we have run backwards really, really quickly?
And then we'll show the flash running backwards across the screen.
All this change is probably just the direction we're facing as we travel, unfortunately, and that doesn't seem to be helping our case.
Now looking back at our previous graph, we can see that the amount of time that passes seems to be asymptote to the speed of light.
What happens on the other side of the graph then?
Scientists think that the idea to-- think that the secret to time traveling at negative rates, and subtext negative 3 equals time travel backwards, lies on the other side.
Rephrasing that, we would have to travel faster than the speed of light to maybe be able to travel back in time.
And this is where the dicey part comes in.
In general relativity, Einstein describes another phenomenon known as relativistic mass.
When you have an object traveling at high enough speeds, the mass of the object would increase severely.
To move an object with higher mass at the same velocity, one would also require a lot more energy.
Einstein proposes that as we put more energy into an object, both the speed and mass would increase.
However, as we approach the speed of light, it is more likely that the energy put in increases the mass of the object, rather than the speed of the object.
And thus, to be able put in enough energy to push an object beyond the speed of light, that will require possibly an infinite amount of energy, and thus rendering faster than light motion highly improbable.
But why are we so obsessed with time travel?
Time travel often come with it's own set of problems.
Referring back to my initial problem, and say I do have friends in the future who are capable of traveling back in time.
If they received invitations after the party and travelled back in time to attend it, I wouldn't have had an empty party.
Since I don't have an empty party, I wouldn't have to send all my invitations after it, and if I had not sent out my invitations, they may not have received it, and they may not have traveled back in time.
And this is where the whole confusing part comes in, and this is what we call a paradox.
In fact, this particular paradox is known as the grandfather paradox, in which two conflicting series of events occurs, based on whether I choose to time travel or not to time travel.
This is just one of the three theories of time travel and more details can be seen at a link below, and then they'll be a link to a YouTube description bar.
Because if not, the thing gets really, really long.
It seems our current understanding of modern physics-- based on our current understanding of modern physics, time travel might not be possible for us to achieve yet.
That's not to say that things might not change over the next decade or the next century as we are slowly figuring out the world around us better.
But for now, if they're going to be planning a party, be sure to send out the invitation before it, and don't count on your friends on being able to travel back in time just to attend it
Nice.
All right
And I know this one goes way--
Yeah, no, it's OK.
This is great stuff to work with, and actually I think that with just a couple fixes, you will also be pretty ready to film as well.
This is not your script, you--
Yeah, if you could open Google doc-- I just-- I'm not sure.
Because I'm updating it in Word, it's of-- no, I'm Kenneth
Is it this one?
No, I'm Kenneth
Oh wait, oh, I'm so sorry
It's okay.
Yeah, yeah, this should be it
I feel like I have a fairly easy fix for the time-- speaking of time.
I feel like the whole party invite thing is too confusing
Too confusing, yeah, that's what I felt when I was writing it.
But then, the grandfather paradox is--
Yeah, I actually-- I like how you explain the paradox at the end because it answers the question, well, if Stephen Hawking proved that time travel wasn't possible, then why are we still thinking about it now?
Like what sort of loop holes exist?
I think that--
Maybe not mention [INAUDIBLE]
I know that you had restructured this.
And I'm guessing you restructured it, because George had said earlier, it would be nicer if you could actually get Stephen Hawking in this video as you were talking about the video or as you were talking about the party.
But all this can go away.
I think that you can explain the set up with just what you had before
Why do we need-- I mean I get that it's a cool story, but I feel like you explaining this idea of-- I must admit that I have not yet once in my life thought about time travel ever, before this moment.
It's not something I think about.
In fact, I don't even think that I was capable of thinking about time travel, because it seems like this very abstract idea that my brain probably doesn't think about or capable of thinking about.
You did an incredibly good job at helping me even understand how someone would even think about that idea, which I think is the beauty of your piece.
That's like the essence of what you've done is you've simplified really complicated set of theories and all of that into something that I could now follow and be like, oh, I now understand how people devote their life into studying this idea.
I would have never, ever thought about this before.
The party, to me, I don't need it all in this to me.
Not at the beginning, not at the end.
The idea is time travel even possible is interesting enough without that story, from my perspective.
That I think the party piece is just-- it's got so many double negatives in it that it's just really complicated to wrap my head around.
I just wonder what it would look like if we took the party thing out entirely and focused on--
I was actually thinking an alternate was maybe a B-roll of footage from Hollywood films of people traveling back in time like in Harry Potter or in Star Trek
I don't think you need it, I mean, personally
Then there's like a whole IP issue, which we're going to talk about it an hour.
I mean here's the thing-- and again, this is the part where like personal taste comes into play.
I think that the story is nice as a storytelling element.
That you end the video nicely, because you reference the beginning, where you're just like just send out your annotations.
But I do agree that it takes a really long time to explain it, and I wonder if you can cut this intro down to a sentence.
I think this is an OK start, held a party, but you can take out all this stuff
I guess I get lost with the he sent the invitations, no one came.
How does that-- I don't know.
It's really-- maybe I'm less adept at thinking this way than the rest of this room, but I really struggle with this
Yeah, PJ?
One thing I found when I was redoing my script was-- mine was pretty wordy to begin with and I think the story-- I think it's good, and I think you get a lot of those words based off what we were shooting.
There's no one there, and the fact that you're describing again, I don't think you have to do it because they're going to get all that.
I like the Stephen Hawking story, but like it was with [INAUDIBLE] doing a sentence based on all the visual queues they're going to get from the video
Because I like that you talk about the grandfather paradox.
And I actually didn't super understand the grandfather paradox until I heard you read this paragraph aloud in the context of the party.
But again, Paul is totally right.
You can complete rely on the visuals to tell a lot of these words for you.
So you could say, back in 2009, Stephen Hawking held a party that no one came to you, and he smiled at himself, because he had just proven that time travel was possible.
All right, so you're reducing everything down into a sentence
Not possible
Yeah, is not possible.
The reason is that he sent his invitations out after the party had finished, thinking that if people in the future were capable of time travel, they'd receive those invitations after the party had ended and come.
So, because the party was empty, time travel is not possible.
You know I mean?
Really, really condense it down to two sentences.
Really challenge yourself to explain the story in two sentences
And I might need a-- I mean, for me, I might actually want an animation in this part too.
I don't know.
I don't know why I find it really complicated
It is, it's a complicated-- it's complicated because you don't really explain it until the very end, which is OK.
I agree that it's not vital to your content, that the content is fascinating enough.
But it's used really well, right now, as the storytelling device to open and close the video, and then in the middle, you bring it up again.
I think that there's a lot of stuff in the video that you can take out that might give you some time to play around with.
For example, this whole chunk can be cut down a lot.
Let me change this to-- I'll just resolve this.
I think the question that you can set yourself up with is, so time travel-- if you disprove time travel, why are we still fascinated by-- or why do physicists still think about it?
Well, what exactly is time travel?
A lot of this--
In otherwords--
Here, I don't I think you necessarily need these two sentences.
I also wonder if you need this paragraph here.
Yeah, I mean it's fun.
It's going to be a little hard to execute visually given the amount of time you have left.
So, just from a production standpoint, it'll be tricky
Actually, this really worked for me
It did?
Yeah, at least for me.
When you-- I mean with the babies, you don't actually need real babies.
You can have dolls, right?
No, I was thinking of just cartoon babies
Yeah, or whatever-- plus, as a mother-- oh, there's no way I would lend you my [INAUDIBLE] for this film.
No offense, but there is no way I would lend a child for this film.
But I think that this example actually was very concrete and I think [INAUDIBLE]
Yes
I'm just going to say that example, I thought, was really good too.
But I think you can-- not the sentence before but the paragraph before, you could turn that into one thought, because you're explaining how you travel in hour per hour.
I think if you were able to just maybe get a sentence out of that and just get rid of the rest and then blend it with that now I take a pair of newborn twins' paragraph, that'd be good
Yeah, I think the only issue with it is that it takes so long to getting to explain to how we travel backwards in time, and I don't know from the beginning of the script, that you're planning on explaining that.
And then I'm just sort of waiting to see where you'll go next.
Maybe if you preface I'm going to explain what time traveling is.
You can say, well, first-- in your own words, you can say first I'm going to talk about how we travel forward in time, and then backward in time or somehow indicate that to the viewer.
I don't know if anyone else felt that confusion.
Sorry, Andrea
I almost wonder if you shouldn't you lead off with the baby example.
As like let's do this experiment, let's do this side experiment where we do this.
And it might-- I think it'll help with your pacing.
It was difficult for me to listen to.
It was pretty wordy.
I think if you actually like just grabbed two props as though you're acting it out and ad-lib it and record yourself doing it, you'll get something that flows a lot better, because you'll be able to cut out a lot of words.
Because you'd be like, here's a one-year-old baby and here's a 2 and 1/2 year-old toddler
Yeah
That's what would happen
There are a lot of just little phrases like this like we are with the universe around us that are very poetic.
They sound really nice in an essay.
But if you actually strip some of this away and really just try to distill it down to the essential facts, I think that's going to save you a lot of time too.
But, overall, at this point it seems more like distilling ideas sound.
Do people agree with that?
I'm glad that you flushed it out, though, because before it was sort of missing some of the essential explanations, and I know you were worried about it getting too long because of it.
But I think the overall structure of it is very sound.
Yes, Julia?
Well, I was going to say that all of the topics involved in the script kind of to me don't seem necessarily connected, because the paradoxes are more in the field of philosophy, and then relativity is physics, and then the twin paradox is a thought experiment so that's closer to philosophy.
I think it will be nice to separate the two to mention this is actually scientifically proven and this is just something that people think about.
Also, the most compelling part to me was, and that's something that answers the question of time travel the best, is the part where he talks about traveling forward in time and travelling backwards in time.
And the backward part was not very flushed out whereas there was a lot of time spent on the twin paradox that doesn't necessarily have to do that much with time travel.
Do you think there's a way to restructure it?
Or focus on something other than time travel that would be paradoxes that arise when you work with relativity and travelling at the speed of light?
That would be the twin paradox and the grandfather paradox, and then you could explain them in more detail.
Especially for the grandfather paradox that would help, because you could split up the explanation into different sentences so you wouldn't use would of and had multiple times
Yeah, I mean the only thing is that you are working with a very tight time constraint.
And I think all of this information is nice and it's good and, it helps me understand it better.
And would work really well if this were a blog post, for instance.
But you do only have four or five minutes.
And I actually like the structure that you have waiting until the end to address the paradox, because you're kind of going back to the beginning.
And you're saying, why isn't time travel possible, aside from the fact that we can't go beyond the speed of light.
How else are you going to support Stephen Hawking's claim that time travel isn't possible?
I like that the way you've split up putting the paradox at the end.
I don't know how you guys feel about it.
How do you guys feel about the overall structure of the script?
I think that he has to-- I think a paradox should be at the end.
I think time travelling backwards should be after time travelling forward.
I think that he-- correct me if I'm wrong, but I think that you don't have enough information for time travelling backwards because nobody knows how to do it.
Nobody knows the [INAUDIBLE] travel backwards in time, so you can't give a really concrete example for it
Right, like you can't have an analogous example
Yeah, because we don't really know
[INAUDIBLE]
Nathan, did--
I think [INAUDIBLE]
Yeah, Nathan did you want to say something?
Oh, I just thought that Chris saying the structure-- I actually really liked how everything was laid out.
And that I feel like most of the work is just shortening individual parts
Yeah, I agree with Nathan on that.
And honestly, the thing about the traveling backwards in time, for me personally, it's just that you don't get to it soon enough, so that I feel like we're wading through a lot of intro and a lot of explanation of moving forward, which are important.
I wouldn't say take them out completely or anything.
But that I had no idea what any of this was actually before I read this.
And I'm glad that you put it in there, because that's really what the meat of it is.
What exactly does it mean to travel back in time?
And at first I was like, why is he putting graphs up?
This is being really physicsy, but now I understand why you're talking about asymptotes and everything.
It's just that it took a really long time to get there.
And--
I think, if I hear everyone, we just be ruthless going through paragraphs and cut it-- information that you don't think is essential.
Because the overall structure works, but there's just too much
Yeah, I think this paragraph is very vital and I would shorten that unless you're like going to take out fluffy words.
If anything, I would take out from this stuff before it.
And this stuff is super interesting, and I don't want you to take it out because it's a further explanation of why time travel isn't possible.
But it gets really dense here.
This is the part when you were talking where I was like what is happening
That's another video
Yeah, this is OK.
But this paragraph, can you just say that would-- it would also need a lot more energy, actually an infinite amount of energy to move it.
Traveling faster than the speed of light is highly improbable
I'm just afraid that it condenses the whole chunk on traveling back in time--
--too much
Yeah, that's why I decided to talk a little more
Yeah.
No, and I'm glad that you flush it out.
I know that you were hesitant to add more stuff, because you were aware of the time constraint
And mention e equals mc squared
Not exactly.
It's, effectively, he goes--
[INAUDIBLE] e equals mc squared then as he increase e-- as e increases-- normally you'd think that m stays the same and c increases--
Actually, e equals the [INAUDIBLE] mc squared.
That's why I didn't really want to bring it in
Yeah.
Yeah, I don't think you should add any equations to this.
Aside from the fact that like e equals mc squared is like the characteristic Einstein equation that people--
But the thing is, it doesn't really fully explain this.
That's why I didn't really feel like putting the other equation in
I agree with Jamie, overall.
The structure, I think, is there.
The substance is there.
I don't think you necessarily need to add anything except for maybe a transitioner or so.
But at this point, I really think it's just, again, being ruthless about taking words out, taking a sentence out or two.
You're only over by a few minutes, which actually is an eternity
[INAUDIBLE] fast
Yeah
And I will say that slowing down your pace will help people--
Yeah, I think I started getting lost around the Einstein, because you were talking a lot faster in that section too.
But I'd say at this point it's really just paring it down to the essentials.
And we can help you with that during office hours if you want.
Any final thoughts for Kenneth?
Yeah, Joshua?
I do agree with your point just now about [INAUDIBLE].
I think it's helpful.
It's been a bit confusing the [INAUDIBLE]
Yeah, I was thinking whether I should even mention a paradox straight off or not to mention it at the top.
That's why I was just playing around.
And, yeah, I think no
Is there a to phrase it in the positive?
You're saying because the double negatives are really confusing?
Yes.
I was thinking perhaps it might be something like he was happy because he proved time travelers wrong, and they could not come back in time.
So he makes it very clear that once that-- perhaps it's like once [INAUDIBLE] proved time travel wrong
Yeah, maybe just a little wording thing like that
He might lose a level of complexity that you would want people-- I guess it's not the intent to help them understand this part.
The intent is to tell them that Stephen Hawking proved time travel is wrong?
Is that [INAUDIBLE]?
Then you might be OK to hide the complexity away.
[INAUDIBLE] Just keep thinking about why is that, why is that-- how did that prove it?
How did that prove [INAUDIBLE]?
And you talk about it at the end.
I think the whole grandfather paradox part is nice, because it is an expansion of why maybe his proof wasn't really a proof and why people are still thinking about time travel even though he supposedly proved that it wasn't possible
What if you take the structured approach that Josh took in his whole piece in that little part?
Make it really simple at first.
Stephen Hawking hosted a party.
No one showed up.
It proved that time travel wasn't possible.
Actually, it's a little more complicated, and explain it a little further
Actually, you could open the video like that
[INAUDIBLE]
Just really tight and then it allows you to use that story throughout, but not confusing people with all the details
Yeah
Right?
PJ were you going to say something?
OK. All right, any final thoughts?
I mean this is in good shape too.
Again, it's much easier to-- I think it's easier to pair down from something that has too much content then the other way around for sure.
And the delivery again will help too
These have come so far
They really have.
I mean I don't know if you guys have looked at your pitches that you made last week.
But they really have-- I mean I'm really excited to watch these videos.
Yes
I'm just going to say that I really liked how you were able to explain this complex phenomenon in words that are understandable
[INAUDIBLE]
Yeah, I think that's really cool
Yeah, and that's the whole building learning confidence with the viewer too that I talked about on the first day of-- simple explainer videos are so compelling to people even though they're quote unquote boring, because people feel good when they feel like they've mastered a seemingly complex subject.
And this is a video that is going to help people understand special relativity.
I think that it's going to be a very compelling thing to watch for people.
They're going to want to keep watching because they're understanding what you're saying.
All right
You seem so smart when you get something complicated.
I'm like, oh wow, I just got a really complicated [INAUDIBLE]
Yeah, you're helping them achieve mastery, which is nice.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so for the rest of class, I'm going to talk about post-production.
Do you guys want to take a five minute break before I jump into that?
OK.
I will do that when you guys come back.
All right, so last lecture-- should I stand up for this since I stood for all the other ones?
OK. Feel free to interrupt me at anytime.
So I'm going to start off with my favorite quote of all time, at least for this class, which is, "In writing, you must kill all your darlings."
And that is what today's lecture is about.
Post-production 101 or Killing All of Your Darlings.
So when we think of production, we think of the journey that we have taken in the last week of this class.
We start with ideation, we script, we storyboard, shoot, edit, and then we get to this product magically.
And we have these three concrete steps of pre-production, production, which is what you're about to do, and post-production.
Let me know when I can go to the next slide.
The reality is that it's not like that at all.
And maybe you've seen that.
Hopefully, you've seen that in the course in the last week-- that you don't do production in this nice linear, sequential format, but you ideate and you script.
And when you script, you go back to your idea.
When you're storyboarding, it affects your original idea.
And that's how I'd like you guys to approach editing as well.
At least, I used to think of editing as something that happened after I shot everything.
And once I got to editing, that was sort of equivalent to just throwing what you have together, but that that's not necessarily the case.
In post-production you're going to find a lot of things that maybe help you re-shoot things.
And thinking about post-production before you've even started production can help you think of ways to shape what you're going to do.
So that is why I'm talking about editing before you have even started to film.
So when we think of editing, we think of taking or creating something that is representative of continuous time.
We think about putting together a video that has just the bad bits removed.
So has anyone done any video editing whatsoever?
Kenneth, you have.
Yeah, so especially with the webcam videos, what you're basically doing is recording a continuous event and taking out the parts where you mess up your line.
That's what we think of.
And we think of the bad bits as being things that maybe as a take that was out of focus, maybe your light was overexposed, maybe your delivery wasn't so good.
But in this simplest sense, most people think of editing as presenting something that is of continuous time.
But the problem is that sometimes the bad bits are not as obvious as things like overexposed light, and sometimes, you're not necessarily creating a product that represents continuous time.
So I love this book.
It's on the syllabus.
It's written by the editor of Godfather III, Apocalypse Now.
I think he did The English Patient.
He did a bunch of movies that won a lot of Academy Awards, but he says that editing is not so much a putting together of a path, as it is the discovery of one-- or something like that.
So the question is, how do you best discover this path?
So the nice thing about editing right now-- we're in the digital age, so we have the files that we shot available to us in these chunks and it's up to us to tape pieces together.
But back when film was actually on film, you would have to watch the entire reel through to know exactly what was on the physical tape itself, which was very time consuming, but it also allowed you to see everything in your fresh perspective after you've shot it.
And I think that's the first step that you need to do when you edit-- to review the pieces that you've shot.
And I've mentioned this before in one of the earlier lectures I think, but after you shoot, take a day or take an afternoon.
Don't look at your stuff unless you're just checking to make sure that all of the file's OK.
But when you sit down to edit, I encourage everyone to re-watch the footage all over again because when you're shooting stuff, or when you're really tired and it's been an entire day of shooting, the way you look at your footage may be a little bit different.
So this is one of the camera reports from when we shot Lindsey's video in D-Lab actually.
And I also encourage you to do this while you're shooting to keep track of your files, keep track of what's a good take.
But then afterwards, I would suggest that you re-log everything.
And you don't necessarily have to do it completely like this.
But this was a log that I did after we shot Snot.
I could not review the footage the day of because I was so exhausted so I looked at it a couple days later.
And I was able to see like, oh, you know, this take wasn't really good and it's not worth keeping.
And I'm not just saying that because I'm tired.
So editing is really about looking at the pieces to see which ones are worth keeping and which ones are not worth keeping.
But as I said, the bad bits are not necessarily as obvious as this scene was really poorly lit.
Bless you.
What you're really looking for are things that don't advance the video.
And that is also not a super tangible quality, and I'm going to give you a bunch of examples that will hopefully help you understand what is worth keeping and what isn't.
But to understand that, you have to ask yourself the basic question-- why should I keep this scene?
And this is a very seemingly stupid and inane question, but I didn't ask myself that the first couple videos that we edited for Science Out Loud and it made for a very painful editing experience.
So I'm going to give you an example.
We shot a video on how engines work-- and I showed you guys clips of that earlier.
And we went out to a horse farm in Acton.
And it was kind of a drive, and it was a little bit complicated to get all that set up, so I really wanted to get great footage of the farm, and we did.
We got awesome footage of Luke with one of the horses
We would need 230 horses, like AJ here, to power our Cessna.
But we need 290,000 to power our Boeing 777
So it was this really cool shot, and it took us an extra couple hours to shoot with this horse, AJ.
And we were talking about horsepower, so that's why we had the visual.
But in the context of the rough cut, which I will show you later on in this lecture-- it really didn't make a lot of sense.
It was just like, we're learning about engines, and now here's all this gratuitous B-roll of a horse prancing around.
And I think that I was really set on keeping it because I had remembered how much extra time it took us and how much extra planning it took us.
But when you go to the edit-- when you shoot, you're going to shoot like you're going to keep everything.
And what Chris was saying about take multiple angles, take shots, take things that you think you may not need.
But when you edit, you have no ties to anything.
They are no longer your darlings.
You can kill any of them, and really have a discerning eye for that.
So that's just quick tip number one.
When you edit, try to see what's on the screen and not so much the circumstances in which you got the material.
So if you were really tired that day, don't think about how much effort it took for you to get that shot.
And I will admit that I partially showed you that because it broke my heart to take that out and I just feel like it needs to see the light of day.
So like I said, most people think about editing as having a continuous linear representation of an event.
This is what Slideshow does.
This is what OpenCourseWare does.
Like these lecture videos-- when you watch them online, it's pretty much start to end, nothing too fancy edited in between.
But remember that there are lots of ways that you can play with this, either in the editing, or in the scripting and in the production.
So you have continuous linear.
Josh also talked about this idea of In Medias Res, which is starting in the middle of things.
I think it's a hard concept to really understand so I'm going to show you an example of it.
This is Lindsey's video that we did in D-Lab.
And the way we start it is with an opening that we revisit halfway through the video again
Corn husks burn quickly and turn to ash.
These lumps burn hot like firewood, longer than charcoal, but are cleaner, greener, and less expensive than both.
This is actually made from this, and I'm going to show you how.
[MUSIC PLAYING] About 3 billion people in the world use charcoal today to heat their homes and cook their food.
But charcoal can be expensive.
It produces a lot of smoke when burned and it's made from wood, which contributes to deforestation.
In fact, 98% of Haiti's forests have been cut down for fuel.
What we need is an alternative fuel source.
Let's take a look at how charcoal is made.
We have to first cut down trees for wood, which we then place in a closed vessel and deprive of oxygen.
It carbonizes into charcoal instead of burning to ash.
We then mix the charcoal with a binding solution and compress that into lumps called briquettes.
This process creates a fuel source that works, but has its problems.
So let's hack the charcoal production process to make everything about it better.
[MUSIC PLAYING] Part of what makes the charcoal production process so expensive is the equipment.
Instead of a fancy vessel, you can make a simple kiln out of an oil drum by cutting a large hole in the top and several smaller ones in the bottom.
Gather a couple large rocks for the kiln to rest on, a lid for the top, and you have yourself a $15 kiln.
We also need some kind of tool to mold the charcoal material from the kiln into briquettes.
This is one of D Lab's earlier designs for the hand press.
We pour a mixture of charcoal and binder into the spout here, hammer down the piston, slide open the panel at the bottom, and the briquette pops right out.
This design costs about $20 to manufacture, but the engineers at D-Lab wanted to do better.
They changed the shape of the mold into a square to make it easier to manufacture and significantly cut down on the amount of material used.
This time, the mold itself is used to scoop the charcoal.
Again, we hammer the piston.
And the briquette pops right out.
This design is easy to manufacture.
It creates briquettes faster and it only costs about $2.00 to make.
Now that we've made the equipment less expensive, what can we do about making the fuel cleaner?
If we can get all the smoke to burn off while the charcoal is being made instead of being used, we can decrease the amount of smoke released in people's homes.
Leaving the kiln open for about five minutes at the beginning of the process will allow the smoke to burn off outside.
So that's why we cut a hole in the top of the oil drum.
All right, final challenge.
What can we do to make this fuel more sustainable?
What if we replaced the wood with something else?
Something like dried corn husks, or dried corn puffs, dried bean stocks, or even dried banana peels?
These agricultural waste products, or ag waste, would normally be discarded as trash.
But we can use them to make charcoal-- and poof!
Your trash becomes a precious resource.
And that's how you turn trash into treasure.
Less expensive, cleaner burning, more sustainable, treasure
So in watching this, it's actually a little more Bookend than In Medias Res, but the concept is that you open immediately with the thing that you're not going to talk about until later on in the video.
And we could have opened without her doing the, this is actually made from this.
We could have just opened with her at D-Lab and it would've been fine.
It would've been the continuous linear thing, but we kind of used it as a hook.
And I think a lot of you are actually going with this format of starting in the action.
Julia's is kind of like this.
Let's see, who else kind of has-- sort of in the middle of things?
I think Paul's is a little bit like this, too.
Just like you're in the action immediately without a bunch set up.
So that is one way you can do it.
Again, this was a little more of an example of bookending stuff, too, which is what Joshua's script does.
Kenneth, yours does this right now, too.
Which is also a nice way to encapsulate a video experience in a tidy package.
So sort of opening the same way you close.
For George's video on farts, we did this quite literally.
We had him in the exact same location and set that we opened the video with.
And that's how we did my snot video as well.
You don't have to do it that way, but it's a very explicit way of bookending your video
You know what's more like life than a box of chocolates?
Farts.
You really never know what you're going to get.
A skunk?
Rotten eggs?
And then-- oh, shoot.
Uh-oh.
I wanted to skip to the end of the video, so let me just do that really quickly.
So he opens in that location.
And then--
We're at the very beginning of understanding the incredible, smellable science in your gut.
And until we do, happy farting.
[MUSIC PLAYING]
So that's just something to think about if you're getting ready to film and you're deciding on the location of where you're going to open and close your video.
The other thing editing does is it allows you to figure out what's best to keep.
So this is an example of where cutting out the bad bits isn't as obvious for us.
In this section, it was cutting out the bits that don't work the best.
So I showed you guys this before, but basically taking stills of all of your scenes.
And then seeing the shots in the context of everything else can help during editing as well.
The next concept I wanted to talk about, which is also in Walter Murch's book-- is the idea of referred pain when you're at the doctor.
You go to the doctor and you have shoulder pain-- or like elbow pain.
And if your doctor says, OK, we're going to go in and operate on your elbow, then you should leave because maybe this person is not very good doctor.
What a good doctor would do is ask you about what kinds of activities you've been doing.
He or she would check out the rest of your body, and then maybe find out that the problem isn't your elbow, but the problem is a pinched nerve somewhere upstream.
So this is the analogy he makes.
Same thing with your videos.
If you're finding that there's a certain scene that's not working, most of the time it will be because you need to change that scene.
But a lot of times, it may be because you've set it up in not the greatest way.
Or maybe the scenes preceding it don't make a lot of sense.
So I will show you an example of this.
And this is super, super nuanced, and it might just be because we were being super nit-picky and getting really OCD about it.
But this is a scene from the fifth draft of Jamie's video
This Digi-Comp has 31 switches and can count up to 127.
But modern computer chips have over a billion switches.
They're made from wafers, such as this one, where each square represents a chip.
They're made from semiconductor switches called transistors, which have the advantage of being solid state, meaning they have no moving parts.
This allows engineers, like me, to make them smaller, faster, and more energy efficient.
We can make them over a billion times faster than the Digi-Comp.
These semiconductor switches make modern electronics possible
So the whole video is about how big, complicated machines, like computers, are built out of actually very simple parts, like a switch.
And so the majority of his video is with the Digi-Comp.
And he's talking about how the switches are calculating in binary, and then at the very end of the video, he goes, these switches are what make modern electronics possible, which is true and it ties it into the bigger context.
But when we were watching the rough cut, our editor [INAUDIBLE] and George and I were all like-- we felt like the ending came super abruptly and seemed a little bit random-- that we're talking about basic switches the whole video and then all of a sudden, he's like, and that's what makes things possible.
So we thought about re-recording the end, but then we realized-- and again, maybe this is a personal taste thing.
Maybe this video isn't actually super different from the one we ended up publishing, but this is an example of us troubleshooting by looking at previous scenes and how we set up the final shot.
And we felt that the gap between him talking about semiconductors, and then his final sentence was a little bit too short.
So we thought maybe we could make the pause a little bit longer to signify that his final shot is the encompassing idea.
So that's what we did
Big the numbers can be.
This Digi-Comp has 31 switches and can count up to 127.
But modern computer chips have over a billion switches.
They're made from wafers, such as this one where each square represents a chip.
They're made from semiconductor switches called transistors, which have the advantage of being solid state, meaning they have no moving parts.
This allows engineers, like me, to make them smaller, faster, and more energy efficient.
We can make them over a billion times faster than the Digi-Comp.
These semiconductor switches make modern electronics possible
I mean, it's a super, super nuanced thing.
I'm not sure if you guys noticed it, or even if it was worth us doing.
But I wanted to show it as an example of the thought process that we had in troubleshooting the final scene.
And I think it makes a little more sense in the context of watching the entire video, but in the interest of time, I didn't want to show you 10 minutes of the same video.
Edits can change a lot more than just taking out the bad, though.
They can completely restructure things.
And I'll show you a little bit from the rough cut of the engines video.
Someone was asking exactly what a rough cut should be-- if we need everything in there, if we don't.
So this is an example of what the first draft of a video can look like.
We're missing a couple scenes.
A lot of things change
Hundreds of years ago, James Watt decided that my friend, Helga, could lift 550 pounds a foot in the air in one second.
And that's the definition of a horsepower
This lawn mower has 6 horsepower
This car has 110 horsepower
This Cessna has 230 horsepower
And this jet has 85,000 horsepower
But they do it in different ways
A car uses a piston engine.
This is a piston engine that's been cut into a half so you can look into it.
A mixture of fuel and air enters through these valves into this metal cylinder.
It's compressed when the piston pushes up onto it, and then this mixture is burned through a spark plug, causing the mixture to expand.
And the piston comes back down, and this happens over and over again.
There are multiple pistons in a car, and they're connected through a series of linkages to the wheel, causing them to turn
So I wanted to show this example to highlight that editing is a super, super collaborative process.
So after you guys put together your rough cuts and we screen-- for the second half of that class, we'll randomly assign you editing partners.
And they're going to look through your video and annotate on a live software that we have at MITx.
The editor who put together that rough cut did not hate that font as much as I did.
And she did not hate that music as much as I did.
All right, so if you're the only person looking at your video-- to this person, maybe they really liked the style, they like the pacing of it.
But to me, and to George, and to the second editor that we had, we often have multiple editors look through our stuff, even if one person's the main one doing the editing.
You get feedback that you otherwise wouldn't have, which is a very obvious statement, but I want you to see how much it changed by the end.
I'll show you the opening again
Hundreds of years ago, James Watt decided that my friend, Helga, could lift 550 pounds a foot in the air in one second.
And that's the definition of a horsepower
This lawn mower has 6 horsepower
This car has 130 horsepower
This Cessna has 230 horsepower
And this jet has 85,000 horsepower.
All of these engines produce power by forcing a mixture of fuel and air into a tight space, and then burning it
But they do it in different ways
A car uses a piston engine.
This is a piston engine that's been cut into a half so you can look into it.
A mixture of fuel and air enters through these valves into this metal cylinder.
It's compressed when the piston pushes up onto it, and then mixture is burned through a spark plug, causing the mixture to expand.
And the piston comes back down, and this happens over and over again.
There are multiple pistons in a car, and they're connected through a series of linkages to the wheel, causing them to turn
So we changed a couple things, changed out the music.
I'm still not a huge fan of it, but I thought it was a lot better than what was there before.
We added a couple labels, put in some stock footage.
I mean, to me, I think the feel of it changed a lot after having the collaboration between four or five of us.
And that happens with every video, too.
I wanted to mention the five words that you should never say, which I often say, too-- but the notion of, we'll fix it in post.
Like oh, the sound isn't super great for this take, but that's OK.
I'll fix it in post-production.
Thinking about post is really good to set you up for creating as high quality materials to work with as you can possible.
So really try not to lean on post-production and editing as a space where you will have a bunch of band-aids to cover up material that isn't as good.
You will save so, so, so much time doing just that extra take to get the good audio than you will if you tried to clean it up in editing afterwards.
So don't say these words.
So we have to kill our darlings.
We have to figure out what things are worth keeping and what aren't.
What is the way that we actually kill the pieces.
And that way is with the cut.
So the cut is going to be the spaces in which we decide to break the footage that we have and align it with something new-- a transition.
Now this notion of seams is an interesting one when you're editing.
A seam is basically like when you notice that the location changes or that the scene changes.
Being aware of where your cuts are and what your scenes look like are going to be important for the overall feel of the video.
So in an earlier lecture, I think I'd mentioned storyboarding is great because it gives you a sense of what the visual pacing of your video is going to be.
When you're doing cuts in editing, watch through sections of a portion that you've edited-- maybe 30 seconds of it, and basically see how many seams there are and if they're really noticeable.
What I personally like-- this is my taste, and I think it's the taste of most people-- is that I think your seam should be as deliberate and clean as possible so that they're not very noticeable.
What you don't want to end up happening is someone to watch your video, and notice every single time you take a cut.
You don't want their reaction to be, oh, she cut there.
Oh, she cut there too.
And for that reason, I favor just clean cuts.
No funky transitions, no ripple effects.
Do you guys know what I'm talking about?
No spins.
I think we did one push transition that I fought against.
But that's my personal taste.
But in general, the cut should be as hidden as possible, I think.
Because that's not what your video is about.
Your video is about the content.
It's not about the transitions in between the scenes.
When you do clean cuts, though, especially for these videos, it has to be deliberate enough to where it makes sense.
And I'll explain this further.
So in this example-- [VIDEO PLAYBACK] -You can count up to 127.
[END PLAYBACK]
That is the wrong example.
Never mind.
I'm not going to talk about clean cuts.
Apparently I'm going to talk about transitions.
So this is one transition where something that wasn't a clean cut worked for us, I think.
Because we were being a little tongue-in-cheek about the wipe.
The way we achieved this is, George actually loosened up his tripod.
And when Jamie was in Stata, at the end of his shot, he just flung the tripod up so that the camera actually swung up.
And then we used that to move into the transition effect that was in Final Cut.
It just makes the video look a little more clean and deliberate, I think.
Otherwise, you have still frame, wipe up, wipe down, to still frame.
It looks a little bit different.
What was I going to talk about?
So yes, sometimes it's OK not to use clean cuts.
This is another example.
[VIDEO PLAYBACK] [TICKING] [BEEPING]
All those were clean cuts, by the way.
-Radiation is frightening.
At least, certain types of it are.
My Geiger counter doesn't go off near my mobile phone, or the Wi-Fi router, or my microwave.
That's because a Geiger counter only measures ionizing radiation, that is, radiation with enough energy to rip electrons off atoms.
And it's measured in units called sieverts.
If you're exposed to more than two sieverts all at once, you'll probably die shortly after that.
But we're exposed to low levels of ionizing radiation-- [END PLAYBACK]
So that was a wipe.
But I don't feel like that's super cheesy, because he only uses it very sparingly.
Did you notice how many cuts there were in that 30 seconds of clip?
He must have cut 20 times.
But as a viewer, I'm not noticing, oh, he cut there, oh, he cut there, oh, he cut there, right?
So that's what I mean by keeping your seams hidden.
Yes
In a way, it seems intuitive.
But when you're editing, and you're looking, and you're cringing at your own speech, it's a practice that you might want to do.
You never cut in the middle of a phrase, or in the middle of your talking.
Because that's really jarring, like seeing your head in one place, and then seeing your head in a slightly different place.
It's always really jarring.
But whenever he cut, he cut to something that was other than his face.
When he was walking through with the Geiger counter, he was cutting to different angles of that same shot.
Or when he was just talking to the camera, he cut away from his face, and then cut back to it.
So you don't see a little twitch where he was saying one phrase and he did another take, and he was in a different spot.
It's something you don't notice until you actually get into the editing process.
And you see, whoa, that's really weird.
But it's something good to keep in mind.
So if you aren't going to finish a two sentence phrase, think of a way to transition between that, rather than just doing a hard cut between them, because it won't look clean
I have some examples of that, that I was going to show you.
But that's a super, super important point.
Yes, Joshua
I wanted to say a sentence, but I remember the plans video.
What did they have in common, [INAUDIBLE]?
Yeah.
So for that, we are jumping ahead.
I have this one slide, and then I'm going to get to it, I promise.
So this is going off of the whole hide your seams type of thing.
I like to keep labels really simple.
Sometimes you'll have a need to label things.
Again, this may be a matter of personal taste.
But I think the cleaner the labels are-- no funky Word Art, I think those other things look a little amateurish.
If it's good content, it doesn't need the frills, necessarily, to be engaging.
So you were asking, cutting, how do you do the thing that Sari was saying?
For plants, we had her really close up for the beginning, right?
And then she goes, what do all these chemical compounds have in common?
And then we cut out to her really wide, with her saying, plants.
Right?
So that is an example of making your cuts deliberate.
You want to cut between two scenes that look different enough, to where the cut looks intentional.
So this is an example from Jamie's video, and I think this is the right one.
[VIDEO  PLAYBACK] -Even though we think of computers as super complicated high-tech machines with very tiny parts, they can also be huge, wooden, and mechanical, just like this computer here.
And even though they-- [END PLAYBACK]
So right there, I think in that take, he messed up.
Or he took too long of a pause in between the two words that he was delivering.
So we wanted to cut that pause out.
But it would have been weird if we just cut that pause out, and then didn't do anything else to the footage.
So basically, he would have been like, computers, blah blah blah.
And then he would have moved slightly to the left.
And then you would have seen him keep talking.
Do you guys know what I mean?
I'll show you an example.
So to get around that, we had his first part of the sentence delivered the way we filmed it.
And then when we cut, we cut to a digital zoom of him.
So he was a lot more close up.
And that's what Hank Green does a lot in SciShow.
He cuts back and forth, between a wide shot and a close up shot, to make the cuts a little less jarring.
I'll just show you the example.
[VIDEO PLAYBACK] -What it is the most awesome place in the world?
Seems like that would be pretty much the most subjective question ever.
But this week, scientists use a set of objective criteria to come up with a list of the world's most biologically important places.
There might be one near you.
[END PLAYBACK]
He also cuts at the ends of phrases and sentences.
So it would feel a little bit more awkward if he was cutting in between, this is my sentence.
And then he cut to a close up, and I'm going to keep talking about this.
And then he didn't cut, and he just kept talking about the next idea.
And then he cut back to this wide shot.
So again, when you're filming and you mess up, keep going.
But try to do as many continuous thought takes as possible.
Because again, don't say, I'm going to fix it in post.
Because sometimes you're going to look at it, and be like, there's no way around this.
It's OK if you can't get an entire scene perfect.
But at least try to get chunks of sentences, or chunks of thoughts continuously, because that'll be a lot easier to work with in post.
So this is the pitch that I recorded for you guys.
And I made it this way because I wanted to show you that it does look weird when you don't cut to something different enough.
[VIDEO PLAYBACK] -When we think of snot, we usually think of the stuff that oozes out of us when we're sick, right?
[SNEEZE] It's just more gross bodily fluid that we seem to excrete for whatever reason.
But snot is more than just the stuff that hangs out of the noses of most two-year-olds, and some 20-year-olds.
It's filled with all of these cells, that do everything from fighting infection, to keeping your esophagus from ripping up every time you try to eat something.
[CHOKING] And that junk actually can keep stuff out of your body.
And then it knows what to keep out, what to let in, and when to do it all.
What?
That is so crazy.
Snot and mucus is so important that your body makes a gallon.
[END PLAYBACK]
It's visually very hiccupy, right?
Because I'm only moving slightly to the left and right.
And I think it's appropriate for webcam YouTube videos.
That's just kind of the style of it.
You're centered, which is also breaking a rule of framing in thirds.
So stylistically, I think it's more acceptable in YouTube webcam videos.
But for these, it will look out of place.
Because you have like a very formalized script, and it's going to jar with a very amateurish style of editing.
Does that make sense to people?
When you did that, did you actually have it in your head that you were going to cut to those spots, or was it just what you put together after you had everything?
So I did that in one long continuous take.
I just hit Photo Booth on Mac and I said my whole thing.
And there were lots of spots where I'd messed up in between.
So I did the style of editing that I first talked about, which is just cutting out the bad bits.
So I wasn't intentionally doing that.
But that's why the movements are just a little off-kilter.
I'm not trying to move or anything.
It's just that I move when I talk.
But it reads a little bit weird and jarring
And you'd recommend just leaving record on for however long it takes?
Like in between scenes, maybe?
I personally have a really hard time remembering what I'm going to say.
So I would feel better, when I was shooting, if I knew that I only have to say two or three sentences.
But when you record, always record the preceding sentence, and the sentence that comes after the one that you're trying to record.
Because your intonation will sound different.
Otherwise, every single sentence will sound like this.
It will sound like you are talking in the same style of sentence.
It will sound like you were just reading a bunch of bullet points.
So that's why, if you're shooting scene 2, shoot the last sentence of scene 1, and the second sentence of scene 2.
Make sense?
And so this is also from Walter Murch's book.
But this is sort of a tongue-in-cheek thing.
Thing But he talks about what makes a good cut, and the order of priority.
A cut that maintains the emotion of the scene is something that's really important.
Which sounds very strange.
But if you have, say, a very dramatic scene, with two people, like Sari and I are getting into an argument, right?
And we have the camera that's focused on me, and a camera that's focused on her.
And you want to show both of us.
So actually, you would show Sari getting all mad at me.
And then you'd show me getting all mad at her.
Maybe you'd show her again.
If you did super rapid cuts of her and me, back and forth, back and forth, back and forth, it's going to read kind of funny, right?
It'll read like Western parody.
You guys know what I mean?
But if you wanted to highlight the gravitas of the situation, you would cut from her face to maybe a wide shot of the two of us.
And you wouldn't make the cuts as rapid.
Does that make sense to people?
So I think that's what he means by maintaining the emotion of a cut.
I think for the purposes of your video, the main thing will be just making sure that the cuts aren't happening too rapidly, or too slowly.
I think you'll know.
You'll watch the thing, and you'll feel like something's a little off kilter.
And if that's the case, I would encourage you to adjust how long you're making the clips, and how much dead space you're leaving it between the clips.
This is a way to inject-- [VIDEO PLAYBACK] -OK, so this might be we our weak point.
Or is it?
[END PLAYBACK]
So that's not a cut.
But that was a type of zoom that is emotionally appropriate for this scene, because it's trying to be funny.
But if you did that for you talking about cargo ships not sinking carrying lots of people, it would be totally inappropriate, right?
The other thing he talks about is, does it advance the story?
And I will show you an example of this.
[VIDEO PLAYBACK] -And all those bacteria care about is staying alive.
Our intestine actually helps them do that, because the temperature is warm and constant.
They're sheltered from the environment.
And best of all, we provide the food.
[END PLAYBACK]
So does anyone want to guess how that particular cut advanced the story?
Or did you notice where the cut was?
Can you show it again?
Sure.
So try to pay attention to where the cut happens.
[VIDEO PLAYBACK] -All those bacteria care about is staying alive.
Our intestine actually helps them do that, because the temperature is warm and constant.
They're sheltered from the environment.
And best of all, we provide the food.
[END PLAYBACK]
So there are actually two cuts in this.
But the first cut is of him walking.
He's like, and best of all.
And then it cuts to him arriving to the place where he was walking.
So in those five seconds or so, the story was about George walking to the hood to do his demo.
That sounds like a dumb story, but that's essentially what the story of those five seconds were.
So a cut that often works is, if you're doing like a walk and talk, for instance, and you're saying blah blah blah.
Navy ships, people might sink.
And then you cut to the person arriving at their station, like where the boats or whatever are.
Does that make sense to people?
So how do you ensure that your voice doesn't sound funny?
Yeah, like if your delivery of the sentence in one location is too different from the other
Do you remember how you spoke it in the shoot?
So to shoot that scene, for instance, he would say, we shelter them.
And best of all-- we didn't cut there.
We made him finish the line, when he's saying, best of all, we provide the food.
And we took a complete cut of him walking and saying the whole line.
And then, when we shot the part of him walking toward the hood, it wasn't like we said action, and he said, we provide the food.
We said action, and he says, and best of all we provide the food.
So that's what I mean by, say the lines that precede and follow the line that you're actually trying to capture
And do multiple takes
Yes
And say the line a little bit differently each time.
Because maybe you'll find that your third take for the initial shot, and your second take for your second shot, are the most cohesive
Yeah, that's very good advice.
And as the directing partner of whoever you're shooting, this is the thing that you need to do to help your partner.
If you notice that their delivery in the second location is a lot more subdued and tired, which is going to be the tendency for most people, you can be like, you said that the beginning of that line really energetically when we were in that other place, so you need to match that
Even if you do the take right, do it again.
It's really easy to say, oh I did it right this time, I'm done.
I memorized my lines.
But even if it's that struggle to remember the last word in the sentence, it's worth it to get another complete usable take rather than be stuck with one
And you're going to have to work within reason.
Obviously you're not going to do 100 takes of each scene perfectly.
So this is up to your discretion, and the amount of time that you have.
But I would say, at least try to get three good takes.
I lost my train of thought completely, but that was a very important point.
Thank you.
And then the other stuff is rhythm.
And this is what I mean by if your cuts are too rapid.
Oh, I remembered what I was going to say.
Sorry, I'm going all over the place today.
So in the engines video, that was the latest night that we had worked.
I think I got home at 3:00 in the morning that day.
And at the end of the night, obviously the two talents, their energy levels were super depleted.
And so that's what we ended up re-shooting.
And that's what re-shoots are for.
Even if you end up getting all the shots that you need, oftentimes you won't notice that someone's delivery was really different until you're in the editing room, and you cut everything together.
So that's why I wanted to leave some room for you guys have time to re-shoot if you needed to.
I don't anticipate that you're going to not get shots that you need to get.
But that buffer room, we always use it for Science Out Loud, regardless of how perfect the host is.
Just because stuff like that happens.
[VIDEO PLAYBACK] -This rock can absorb heat from the sun during the day, and then release that heat hours later, after the sun has set.
This is fine when you're dealing with rocks in the sun, but unfortunately it also happens with bodies of water and toxins.
Rivers, lakes, streams, and oceans are still releasing contaminants that we dumped into them up to 100 years ago.
From 1947 to 1977, humans dumped hundreds of thousands of pounds of polychlorinated biphenyls into the Hudson River.
Those PCBs attach themselves to sediment particles and settle to the bottom.
[END PLAYBACK]
So for that video, we had a sense, when we saw the rough cut, that it was kind of a frantic video.
There's a lot of information being thrown at you.
I'm not really following, even though I've helped her edit the script.
If I'm tuning out, then the audience is probably going to tune out.
So rhythmically, the cuts maybe were happening a little too fast.
And she's also, in real life, a naturally fast talker.
We had to slow her down a lot.
But she's just an animated person.
So we knew that the tendency of this video is going to just feel super, super fast.
And if you are naturally a fast talker, this is something to keep in mind, that you can help the pacing, and the franticness, of maybe how you might read, with how you cut the scenes.
So what we did instead was, we allowed for a lot of time in between her scenes to fill with B-roll or music.
We tried to make the cuts not happen as rapidly, basically.
[VIDEO PLAYBACK] -This rock can absorb heat from the sun during the day, and then release it hours later, after the sun has set.
This is fine when you're dealing with rocks in the sun, but unfortunately it also happens with bodies of water and toxins.
Rivers, lakes, streams, and oceans are still releasing contaminants that we dumped into them up to 100 years ago.
From 1947 to 1977, humans dumped hundreds of thousands of pounds of polychlorinated biphenyls into the Hudson River.
[END PLAYBACK]
So again, it's a super, super nuanced thing.
I think the music helped with it a lot.
We tried to use very calm music.
We used a still of that old photograph of people cleaning up the Hudson River, just to break up how many times we were changing back and forth between all the locations.
Is this making sense to people?
Sometimes I feel like I'm saying it.
And then I watch it, and I'm like, that doesn't look super different from the rough cut.
But it helped me sleep that night.
And this is the other stuff he talks about, which is not as important.
But the eye trace, like does it make sense for the audience's focus of interest within the frame?
So are you cutting from something where you're in the top corner, to all of a sudden, you're cutting to something in the middle, centered, to all of a sudden, you're cutting to something where you're on the left side?
Are you cutting so rapidly, and changing your position, to where it's not making sense to the viewer?
The 2D plan on screen, are you are you consistent with the stage line, basically?
So I don't think this is going to be as relevant for you guys.
But if someone's walking, for example, and then you cut to a scene where they're continuing to walk, but they're in the corner of the screen, then it doesn't make as much sense.
And then the 3D space of action.
If you have multiple people in the same room, or multiple objects, and you cut from one angle to another angle, make sure the objects are still the same relationship to each other.
And the reason why he lists this out is, Walter Murch's theory is, you should never compromise emotion before you would compromise the eye trace with a cut.
So if a cut satisfies the eye trace, but it doesn't make sense emotionally, then you shouldn't do it.
Practically speaking, I think what this translates for you guys, is that if something feels weird when you see the rough cut of your video, most of the time, it will be because you cut it in a weird place.
You either cut your speech too short, or you didn't cut to a wide enough shot from a close up.
You are cutting too much, too quickly, or not cutting enough.
Does that make sense?
I'm going to talk a little bit about music, and some of the Creative Commons resources that will be available to you, if you want to use footage that you don't have access to.
Before I go into that, does anyone have further questions about editing?
Sari is good with Adobe, right?
You use Premier?
Yeah, I used Premier.
I used iMovie before that, so just give them those.
Final Cut Pro, I haven't looked at it, but professional editing softwares usually have some weird rules.
So, if that's what you choose to use, I can definitely go and work in the computer lab and figure it out
So Building 26, I'll show you at the end of class the website.
But I've reserved it for the rest of this class period, and it has all the software on it if you want to use it.
I have used Final Cut Pro and Final Cut Express.
I'm not super, super good at it, though.
But in the remaining class time, when you basically have time to work on your projects, we will be around if you want to schedule a certain time to work on a particular problem that you're having in your editing software.
Just come talk to us.
All the tutorials for a bunch of these software, they're all listed on the syllabus.
So when it comes to music, it's a bit like lighting, as Chris said.
First, do no harm.
So I would much prefer a video that does not have music to a video that has music that is super distracting, or an entirely different tone from the video that you're trying to convey.
So be careful with music.
But music is a really, really great tool, because I think it establishes emotion better than any other tool that you have in your video kit.
That's my opinion.
I'll show you an example of it.
Do you guys remember the video of Neil deGrasse Tyson that I showed you on the first day?
The 10 questions?
[VIDEO PLAYBACK] -What is the most astounding fact you can share with us about the universe?
-The most astounding fact, the most astounding fact, is the knowledge that the atoms that comprise life on earth, the atoms that make up the human body, are traceable to the crucibles that cooked light elements into heavy elements in their core.
Under extreme temperatures and pressures, these stars, the high mass ones among them, went unstable in their later years.
They collapsed, and then exploded, scattering their enriched guts across the galaxy.
Guts made of carbon nitrogen, oxygen, and all the fundamental ingredients of life itself.
These ingredients become part of gas clouds that condense, collapse, form the next generation of solar systems, stars with orbiting planets.
And those planets now have the ingredients for life itself.
So that when I look up at the night sky, and I know that yes, we are part of this universe, we are in this universe.
But perhaps more important than both of those facts, is that the universe is in us.
When I reflect on that fact, I look up.
Many people feel small, because they're small, and the universe is big.
But I feel big, because my atoms came from those stars.
[END PLAYBACK]
So this video is effective for a couple of different reasons.
You feel like you're sitting there with him.
You feel like you're the person who's sitting in the third chair, watching an interview happen.
Say what you will about Neil deGrasse Tyson, but he's a very engaging speaker.
And I think that this video encapsulates the wonder that he has for astrophysics.
I'm going to show you another video that took the audio from this interview.
And this user Max Schlickenmeyer put together a fan video that highlights certain elements of this experience in a very different way.
[VIDEO PLAYBACK] -What is the most astounding fact you can share with us about the universe?
-The most astounding fact.
The most astounding fact is the knowledge that the atoms that comprise life on earth, the atoms that make up the human body, are traceable to the crucibles that cooked light elements into heavy elements in their core.
Under extreme temperatures and pressures, these stars, the high mass ones among them, went unstable in their later years.
They collapsed, and then exploded, scattering their enriched guts across the galaxy.
Guts made of carbon, nitrogen, oxygen, and all the fundamental ingredients of life itself.
These ingredients become part of gas clouds that condense, collapse, form the next generation of solar systems, stars with orbiting planets.
And those planets now have the ingredients for life itself.
So that when I look up at the night sky, and I know that yes, we are part of this universe, we are in this universe.
But perhaps more important than both of those facts is that the universe is in us.
When I reflect on that fact, I look up.
Many people feel small, because they're small and the universe is big.
Because my atoms came from those stars.
There's a level of connectivity.
That's really what you want in life.
You want to feel connected, want to feel relevant.
You want to feel like you're a participant in the goings-on of activities and events around you.
That's precisely what we are, just by being alive.
[END PLAYBACK]
So, a very different experience, right?
And did you recognize that song?
I don't, but I thought it was particularly terrible when there was a different set of words
Yes.
So we won't question the scientific accuracy of the video that he made.
And my mom watched it.
And she was like, Elizabeth, this is super cheesy.
But it elicits emotion, right?
It's a very different experience.
I also wanted to point out, how would it have felt if all the images Max used were bombarding you with cut, cut, cut, cut, universe, microbes, a tree of life, right?
Would that have been emotionally appropriate for this video?
No.
So that's what I mean.
Visual pacing can change the emotion of a video.
Music, when intentionally done, can be very effective.
And when I look at seven or eight Science Out Loud videos, it usually takes me about a week to get the music for it.
It's actually super time-consuming.
I'll basically take you through two or three episodes where I looked for music, and did a soundtrack for it, and sort of the workflow that I had, and the reasoning behind why I chose what I did.
Any comments so far, questions?
Everyone's OK?
So, the farts video.
George was very adamant that he didn't want this to read like dad humor.
So I wanted to put a bunch of fart jokes in it.
He was like, Elizabeth, I'm not doing that.
That's not who I am.
That's not me.
And I was like, OK, I recognize that.
I recognize that I'm a different person.
So he gave his delivery very straight.
He gave it very dry.
He didn't make it very outrageous.
This is farts, without music.
[VIDEO PLAYBACK] -You know what's more like life than a box of chocolates?
Farts.
You really never know what you're going to get.
Skunk, rotten eggs, garbage dumpster-- the possibilities are, unfortunately, endless.
But why do we fart, and how does your body make so many different smells?
The answer is fermentation.
This is a giant bioreactor.
It's also called a fermenter.
And in here, a fungus known as brewer's yeast is transforming water and dead plants into beer.
[END PLAYBACK]
So it feels really weird, right?
And part of it is that you don't necessarily use the audio that you record with the B-roll.
You usually put music underneath.
So we didn't want it to read as dad humor.
So we use a couple of more serious tracks at the beginning.
But it was a little bit weird, because he is inherently talking about a subject that most sixth grade boys, and myself, laugh at.
So the fact that we weren't acknowledging the humor at all whatsoever, and we're being totally straight and deadpan about it, it was a little funny, unintentionally.
So we wanted to introduce the humor through the music that we used.
So I will play you the rough cut.
Let me mirror this.
And this was a personal decision of mine.
I'm sure that there are people out there who would have done it very differently.
And that's OK.
But let's play this.
Let's see.
I will show you a track that went a little overboard with the humor.
This is a track that the editor found, called Grandpa Is Scheming Again.
Oh, sorry, it's the wrong one.
Let me show you the one before it, Ukulele.
This is ukulele comedy swing.
[VIDEO PLAYBACK] -You know what's more like life than a box of chocolates?
Farts.
You really never know what you're going to get.
Skunk, rotten eggs, garbage dumpster.
[END PLAYBACK]
But when I told [INAUDIBLE], the editor, I think we've got to inject a little bit of humor into it with the music.
I'm sorry.
There's too much tuba.
I think the comment I wrote was, it was in all caps, "LOL TOO MUCH."
But that's an example of music.
[LAUGHS] So this is Grandpa Is Scheming Again.
[VIDEO PLAYBACK] -You know what's more like life than a box of chocolates?
Farts
I felt that this one had little bit of potential, right?
-Skunk, rotten eggs
But I thought it was a little too frantic for the beginning.
-The possibilities are, unfortunately, endless.
[END PLAYBACK]
I'll show you one more, Sharp Mind.
[VIDEO PLAYBACK] -You know what's more like life than a box of chocolates?
We wanted to highlight the wonder of your digestive system.
So I said, [INAUDIBLE], find me something that evokes amazing wonder, curiosity.
-The possibilities are, unfortunately, endless.
But why do we fart?
[END PLAYBACK]
But it reminded me a little bit too much of Willy Wonka and Harry Potter, which is why we ultimately settled on-- [VIDEO PLAYBACK] -You know what's more like a life than a box of chocolates?
Farts.
You really never know what you're going to get.
Skunk, rotten eggs, garbage dumpster-- the possibilities are, unfortunately, endless.
But why do we fart?
And how does your body make so many different smells?
The answer is fermentation.
This is a giant bioreactor.
It's also called a fermenter.
And in here, a fungus known as brewers yeast--
So it's casual enough.
A super, super important part about music is that you don't want it to be too distracting from your audio.
-So you know that thing where you eat food?
[END PLAYBACK]
You should stay away from music that is super high tempo, a lot of noise.
You want something that's more uniform.
So there are a couple of tracks where it was OK.
But then, every now and then, there'd be this random cymbal hit, or a guitar riff that would come out of nowhere.
And it would be really, really evident while the person was talking.
So that's very distracting.
Music, like everything in post, I think, should be as hidden as possible.
And by hidden, I don't mean that you shouldn't be able to hear it.
But you shouldn't be able to notice it too much.
Yes?
Do you cut the scenes in accordance with the beat of the song, or can it be right in between?
No.
It can be in between.
And I was going to talk about this later.
But all the examples I'm going to show you do this.
Sometimes, if you want to end a track before your actual video ends-- say you're talking and you want some music in the background.
But then you cut to a scene where it's OK not to have music, or you want to change the track, you can do what's called ducking.
Which is when you take the audio, and instead of just cutting the audio cleanly and just making the music end all of a sudden, in the editing software, you can pull this cursor down to where the music slowly fades.
And it slowly becomes quiet.
You can duck in and out of scenes.
So, go ahead
If you have music and B-roll, that's a good way to help with the pacing of B-roll, because otherwise it's kind of ambiguous.
Or it can kind of ambiguous, how long you've spent on a single object.
But music can help you, if you're having trouble finding the pacing of that.
Because then if you have a music track, you can time with the beats.
[INAUDIBLE]
I'll show you an example where we intentionally lined up the beats of the music to the visuals with B-roll, actually.
[VIDEO PLAYBACK]
Like when he does this thing like this.
The editor did that one [INAUDIBLE].
[END PLAYBACK]
So that was more of a stylistic thing.
We wanted it to read like bad-ass, like these skydivers are going to go.
Yeah, that was a day where we were getting as much footage as possible, and didn't end up using half of it.
Yeah, go big or go home, if you have a skydiving license.
Go film, and go skydiving.
And shoot that.
This is an interesting example.
Because the computers one, when I first saw the script-- I thought this is going to be the hardest one to produce.
Because computer science videos inherently are just really hard to do, because of the visuals.
And that's why there aren't that many of them.
But this explanation, for lack of a better term, it just kept reading incredibly boring.
Because it's essentially about how you add in binary, right?
Which is not the most riveting material.
[VIDEO PLAYBACK] -Just like a light switch, these switches have two states.
Either left and right, off and on, or zero and one.
What's really cool here is that instead of programming this computer by writing code, we program it by physically setting the position of several different switches.
I'll set this computer to count the number of balls on the top tray by setting the count switch to on.
The sum is given by this bank of switches here.
We'll set the sum to zero before we start counting
I think Jamie's doing a good job of hosting, but it just gets tedious, a little bit.
Because it's the same thing over and over again.
You count the balls, you set the switches.
[END PLAYBACK]
So my initial thought with this was, I had a list of brainstormed tracks.
And I was like, let's go with something boopy, sort of robotic and electronic-y.
And this is what I mean by be careful of how distracting your music could be, because this is what happened.
Universal Fluff Theory.
[VIDEO PLAYBACK] -Just like a light switch, these switches have two states.
Either left and right, off and on, or zero and one.
What's really cool here is that instead of programming this computer by writing code--
It's just a little much, right?
-by physically setting the position-- [END PLAYBACK]
And we fell into a stereotype a little bit, that a video about computers has to be with electronic music.
And everything has to be very robotic.
And the track that [INAUDIBLE] ultimately found, I really loved, because it was very Zen, actually.
Which wasn't the initial response that I had to the video.
[VIDEO PLAYBACK] -has switches with a mechanical input and mechanical output.
Just like a light switch, these switches have two states.
Either left and right, off and on, or zero and one.
What's really cool here is that instead of programming this computer by writing code, we program it by physically setting the position of several different switches.
I'll now set this computer to count the number of balls in the top trade by setting the count switch to on.
The sum is given by this bank of switches here.
We'll set the sum to zero before we start counting.
Now let's start counting by pressing this lever.
[END PLAYBACK]
So I actually personally like that track, because there's something about it that makes the scene very inviting.
He seems really friendly.
You have something that's as scary as programming a computer, but then the person is super friendly.
Jamie's just a really friendly person in real life.
We were trying to make computer programming as not scary as possible.
And I think that having a track that was a little more Zen, it was a little quieter.
Someone was saying that it's like a very mesmerizing experience to watch it.
Yeah?
[INAUDIBLE] how do you know if the volume of the song is too much?
Or the beats themselves are too much?
Again, a lot of this is personal taste, which is not the answer that you guys probably want to hear.
But I would choose what you like.
And hopefully I'm giving you some pointers as to maybe what works, and what doesn't, as well.
But that's also where the other three people that you'll have looking at your rough cuts will come in handy, quite a bit.
You'll have the opinions of at least five or six people.
Which, ideally you'd have more.
But that would give you some sense.
And this sounds dumb, but if you can't clearly make out what the person is saying because the music is too loud or too distracting, then don't use it.
But sometimes you don't know, and that's why you ask other people.
The last example I wanted to show you-- did anyone have questions besides that?
Everyone's good?
So the last example I wanted to show you was of Anastasia.
Now, I was very conscious of who she was as a person when we were making this video, because she was just this very bubbly, blonde girl.
But she does intense, legit systems biology.
And I didn't want to undermine her intelligence or her persona as this very inspiring STEM figure by playing into the ditzy trope.
So at first, I was putting really hard rock music into her video to fight this.
She's doing intense science, give that vibe.
It did not work for obvious reasons, now.
[VIDEO PLAYBACK] [END PLAYBACK]
And what I realized was that Anastasia as a person is super bubbly.
And she loves science.
And that doesn't undermine her abilities as a scientist, or her authority at all.
And that if we fought that too much, it read really weird, because that's just not who she is.
So we did go with a track that was more capturing the wonder that she had for plants.
[VIDEO PLAYBACK] -Paclitaxel is a compound that can treat cancer.
Salicylic acid reduces headaches and fevers.
Carotenoids can turn your skin orange.
And miraculin changes your sense of taste.
What do all of these awesome compounds have in common?
They all come from plants.
[END PLAYBACK]
So music is really good for evoking emotion, establishing things like wonder, creepiness.
This is an example of one.
And it can also help if you just have some really funky audio going on from the environment that you're filming in.
I hate to say this, but sometimes it can be a little bit of a Band-Aid to cover up weird background noise.
When we were shooting in the brewery for George's video with all the vats, it was just super noisy in there.
And there's nothing that we could do about it.
So he just talked really loud, and we put in the track.
And I think it helps distract from that a little bit.
But again, don't say we'll fix it in post.
Last example, I lied about the other one.
[VIDEO PLAYBACK] -These patterns can seem pretty complicated.
Some people even think octopuses use patterns like these to mesmerize their prey.
But these intricate displays emerge on the squid skin even after it's been disconnected from the brain.
So what's going on?
First I wanted to measure how the chromatophores-- [END PLAYBACK]
So when the music changes, we ducked from the previous track.
This is what I was talking about.
Hopefully it's not super jarring or obvious to you that we were changing music tracks in the background, because we ducked one out and ducked one in.
But the second track, where the question mark comes, I liked it because it was sort of eerie.
And it emphasizes, this is super weird that the squid skin does that.
How exactly does it do it?
And it's a way that we implied that without her explicitly saying it in the script.
So, when a lot of you guys end your scripts, or have a line in there that's like, this is awesome.
Or, and that's how you do that.
You don't have to say that explicitly.
Because a lot of the time, the music can do that for you.
All right.
So I talked a lot about what kind of music you should use, and you are probably wondering, well, where am I going to get this music?
So I wanted to talk a little bit about Creative Commons, which is the license that MIT usually uses.
It's what OpenCourseWare uses.
It's what Science Out Loud uses.
And basically, it's a type of license or copyright.
And in particular, we use CC BY-NC-SA, which I will explain, which allows you to freely share and adapt materials.
So because this class, and this lecture, is being recorded under Creative Commons, under this license if someone watches this video later on, and they want to take a portion of this lecture, and remix it into a video about best practices of music, they can do it.
And I won't sue them.
Because it's totally allowed.
And I want them to take the material and remix it, as long as they attribute.
And this is true of any music that you use for your videos.
So if you go and search for Creative Commons music, you will find a ton of resources.
I've listed the websites that we have personally used in the day 7 lecture on the syllabus.
Things like ccMixter, Jamendo-- thanks to Billy, we know about that one now-- Free Music Archive.
Unfortunately music is not organized online in the best way possible.
So a lot of it is sitting and wading through tracks, and just listening to a bunch of them.
And that's why it's so time consuming.
But a quick note about Creative Commons.
So your videos will be released under this license, meaning that the music that you use, and any stock footage or video that you find online, has to also be Creative Commons.
And if you have trouble with this, let me know, because I have dealt with issues like that a lot.
But basically, when you upload your final projects onto YouTube, it's totally fine for you to use other music, as long as, in the description, you say we used this track.
Here's a link to the original site.
This is the person who made it.
That's just best practices in general.
Ideally, you would cite images that you use in your video itself.
You'll notice that Science Out Loud doesn't annotate with citations on the actual videos themselves, because that was a stylistic decision of ours.
But we put all the citations in our YouTube descriptions.
And everywhere the video is online for us, we put all of the citations.
So for our license, it means that is free to share and adapt, as long as you attribute, and it's not used for commercial purposes, and that you're releasing your work under the same terms.
So if someone took this lecture and then used a portion of it to record a lecture for The Great Courses, for instance, which you have to pay to use, because it's a commercial company, that's not allowed.
On the syllabus, you'll also find links to stock music.
Good music costs usually around $30 to $40 per track.
So sometimes, we use things like PremiumBeat, which was where all the tracks for the examples I showed you were found.
And so sometimes, if you just can't find something that's free, it's really worth, I think, the money to do music the right way.
And then one more note about video.
So maybe you have the Hudson River, for example.
We obviously couldn't film at the Hudson River.
So I found a picture of it on the Internet Archive, which is where you can find a lot of videos, music.
They're all under Creative Commons.
I'll put links to all this in the syllabus, and I'll post something on Tumblr later today that consolidates all these resources.
For biology videos, the NIH has a lot, actually, under Creative Commons.
A lot of open source publications, you can use their figures, as long as you cite.
And Wikipedia, all of those images are released under Creative Commons.
The US government websites are actually really, really great resources, because they release everything under public domain.
So it's not even Creative Commons.
You can just use whatever you want.
NASA has awesome footage.
NIH, I'm trying to think of others.
But that's actually a go-to place for me if I'm searching for footage that we just can't shoot.
So that's all I had planned for today.
Does anyone have questions off the bat?
Yes
I have a question about working with the music that you had.
I've been doing stock music before.
Some of them are like a one [INAUDIBLE] thing, and some are repetitive [INAUDIBLE]
Yeah, it loops the same tune over and over
Do you use those?
Yeah.
The looped ones are nice for us, because sometimes the longer ones are like big orchestral arrangements that take awhile to get to the point of the song.
Whereas, for a lot of the music, you just want something that's present, that establishes a general sound and atmosphere.
But you're not looking for a Mozart composition, right?
You just want something to fill in the background a little bit.
So having something that's looped is actually preferred for us
Have you done a video where there was [INAUDIBLE]
Yeah, some of our earlier videos don't have as much music.
And part of it was that I wasn't savvy enough to do it.
And I watch some of them now.
I'm like, man, this video just feels super long.
And it seems like they're talking for a long time.
And I think part of it's because of the music.
You don't need music everywhere, though.
And like I said, during the presentation, first do no harm.
So sometimes, if you just can't find music for it, leave it out.
I'm trying to think of an example where we said we specifically do not want music here.
And I can't come up with any off the top of my head.
But maybe after class, I can look for something
Elizabeth, can you talk a little bit about how the next couple days are going to work?
Absolutely.
Thank you for the reminder.
So this is it for formal lectures between now and next Thursday, which is when the final presentations will be.
You will have class time to work.
So we are going to pair you up, or put you into groups for filming partners.
And you will have class time tomorrow, the day after, and Friday to work.
We will have a screening of the rough cuts next Tuesday.
So that gives you the weekend to edit together the footage that you shoot this week.
We'll screen everyone's rough cuts.
We'll also assign you editing partners.
And I'll tell you more about this on Tuesday as well.
But you'll watch the videos.
And we have a video annotation software that you can use.
So you'll make comments on at least two or three other people's videos during class time.
And then you'll have until Thursday, where we don't have normal class time.
Our final screening is going to be in Simmons Hall at 6:30 instead of at 1:00 PM.
It'll be like a red carpet event.
But you'll have all day that day to finish whatever you need.
I wouldn't wait until 6:00 PM to have your video finished.
But if it comes down to that, it's not like we have to know.
Yeah, so it's pretty much work time from here.
The teaching staff will be available to help you.
So it's basically like office hours from here on out, except for the rough cut screening next Tuesday.
And if you have specific questions on editing, we'll send out a Google spreadsheet so you can sign up for specific people, or if you have certain requests to go over things
Or, if it's in the evening and you're editing, and you find a problem that you just don't know how to solve, feel free to email us.
Shoot out an email before you start tackling it for two to four hours.
Because if it's a simple solution that we can relate over email, that'll save you so much time and frustration
And I'll send out an email to where everyone's emails are CC'd.
You can ask your peers questions as well, because I'm sure you're going to run into very similar problems once you start editing and shooting.
So use each other as a resource as well.
And honestly, I can't feign expertise over any of this.
So oftentimes, I just Google.
Why won't my clip snap to the previous one?
But, I will send out shooting assignments.
Actually, I have them with me
Elizabeth, we'll be in here during class time as a home base.
So if you have any questions, and you need help, we'll be physically in this room for you to swing by.
You don't have to come into this room physically.
You can use the entire class time to be out and about.
But just so that you know that you can, at any point, during that 1:00 to 4:00 time frame.
If you need help with anything, I will as much as I physically can be--
Yeah, I will definitely be here
They will definitely be in this room, to be here if you want physical help with anything.
And if you want to, just to make sure, you can feel free to email a specific point person to say, I am coming in at this time for sure.
If you know ahead of time that you need help with something, just so that we know we should reserve time to deal with this issue with you
Yeah.
And if you need Chris's help or Natalie's help on anything, just email them directly.
Because they are available, although they're a little bit busy.
So as long as they have enough of a heads up, we can make that work.
I did want to let you guys know about 26 139.
Do guys know where the 801 TEAL classrooms are?
Oh, I guess you guys would know.
It is basically in that building on the first floor, actually, right next to us.
It says Building 26.
It's on the first floor.
You need key card access.
You just swipe your MIT ID.
And I have that room reserved every day during class time until the final day.
So if you want to use their software, you can.
I also have nicer headphones if anyone needs to borrow those.
I think everyone has a kit.
I feel like I'm forgetting something.
Oh, I'm forgetting to tell you the shooting assignments.
So David and Joshua, we'll put you guys together.
[?
Yulia, ?]
Kenneth, and Paul, you guys will be a group.
And then Andrea and Nathan, you guys will be shooting partners
That wasn't random.
We really tried to think about who you are, and your scripts, and all of that.
Because sometimes, if you have a great partner that can really complement you, that can really help your project.
We really did put a fair amount of thought into those partnerships.
It's not just like a random division of labor
If you have any issues during filming whatsoever, about anything, if you can't get into a location, if you need help clearing something, if you're not sure something is allowed to be filmed, if things are just not going well, if Murphy's Law is happening, email me.
Or come find me during class.
Or email any of us.
We are definitely here to help you guys for the home stretch
And I think I've showed this to you, but when you're walking to the bathroom, there's a big sign that says communication lab.
That's my office.
So if ever during normal business hours, outside from class time, you literally need help with anything, that is where I am.
So you can always knock and interrupt anything, and just say, I need help with something, and it'll be fine
And my office is a little far away from campus.
But if you want to meet in the morning, or sometime outside of class, if you just email me ahead of time, I can definitely meet with you guys
And one last thing is the "kill your darlings" thing.
As someone who has been writing for quite a long time, and who has edited people professionally, it's really hard.
The more you get invested in this, the harder it is to be able to do that.
And that's when as a partner, that's a great point, when you're not emotionally connected to the work.
This is just advice.
Instead of pointing out that it's not working, maybe think about framing some questions for your partner, to help them realize that this isn't working.
And that's a really great job you have as a partner, to help your colleague realize that this piece is awesome, it's really cool, but it doesn't work.
And that can be really, really hard feedback.
So thinking about how to help your partner see that sometimes is a really important task
And part of your grade will come from how great of a partner you are.
So ha ha, you have to do it.
I don't remember the exact percentages, but 60% of the grade comes from your writing and hosting.
40% of it comes from the producing that you do for your partner.
And for the group of three, figure it out how you want.
But I was thinking one person does for person 2 which does for person 3 which does for person 1.
Does anyone have questions, concerns?
You say the assignment's on the quality.
So what is that [INAUDIBLE]
She means just that these are your partners to do the project.
The people that she just mentioned are your partners for producing your project.
It's a phrase that I'm sure you use [INAUDIBLE]
Yeah, sorry.
And also, don't feel limited to only working during class time, if you and your partner would rather meet in the morning.
Or if you want to just shoot some stuff on your own, like B-roll, that's totally fine.
I will put this link up on the Tumblr.
And I'll email it out to you guys today, too.
I would love it if sometime, before the end of the weekend, you filled out this super, super short midpoint survey, just to give us a good idea of what is working best for you all.
What type of feedback is working best?
Because I want to make sure that for the last week, it's going to be a lot of project time.
So I just want to make sure that the time is as valuable to you as possible.
So it's really quick.
It's super painless.
It's just telling me what your favorite day was, basically.
So I'll send out a link to you guys.
Just any time between now and the weekend would be awesome
[INAUDIBLE]
No, for most of the videos, we have at least two or three.
I think we have around three in all of them
[INAUDIBLE]
I don't have an algorithm for you, unfortunately.
A lot of it is just what feels right to me
And you can ask for second opinions
Of course
This is a great time to ask your group, or other people in the class.
Or come in and ask us, too.
If it's like, I found this music, I'm not sure how I feel about it.
How do you guys feel about it?
Yeah.
And if you're in the editing room on Friday during class time, and you want us to look at it, just send me an email and be like, hey, can you meet me in 26 at 2 o'clock to help me with this?
And I will do it.
So, at the beginning it's funny.
[VIDEO PLAYBACK] -A skunk, rotten eggs, garbage dumpster-- the possibilities are, unfortunately, endless
And he's talking about how funny it is.
-Break food down to get energy.
But your main byproducts are carbon dioxide and water, and yeast's main byproducts are carbon dioxide and alcohol.
That's how this stuff gets to be beer.
And here's where things get weird.
[END PLAYBACK]
So that's a new break in thought.
It was a logical transition point for us, because he opens with farts are like a box of chocolates.
But this is the part where you're starting to really get into the science.
And he wants to highlight the misconception of, the thing that you thought was funny is actually really weird and super interesting.
So we introduced this track that I felt conveyed that sense of, this is weird.
There's something interesting about this.
But yeah, unfortunately, I don't have a recipe.
I think once you start doing it on your own, you're going to get a sense of it.
And once you start seeing other people's projects, you'll get a sense of it
There's always the mom test.
My mom, I feel like she's so like that-- oh, I don't understand, what are you doing with this?
I can't hear.
I feel like my mom is like, if I can show it to her, and she'll be like, oh, I can hear everything, and your music's not distracting me.
Or, can you make it louder, honey, I can't hear that part.
If you have someone in your life who plays that role, that's a great person to check in with
I'm happy to be that person for you guys
Can you talk louder on this part?
But that's why audio is so important to get right when you're producing.
Because if you record too soft, there's only so much you can do in the editing.
And if you record too loud, you're going to peak the microphone.
And it's going to sound really scratchy.
So when you're filming someone, watch a take or two, just to make sure that the audio's sounding clean and
Do you have the GoPros here, or is that somewhere else?
Oh, I have them in my office.
I can actually go there after class right now, if you're planning on using them before tomorrow's class
Probably Thursday
Thursday?
OK, then I'll just bring them with me to class tomorrow, and I'll leave them in Jamie's office.
They've never been opened before
What's that?
They've never been opened before.
They're brand new, just for you guys
And if you need access to something that's In my office, and there's a code, and if you email me, and I'm not there, don't steal anything, obviously.
But if you need access and I'm not around, I can give you the code to get into that room
You guys can come to my office, but it's all the way out in Tech Square.
So I would imagine most people wouldn't want to
And if you see lots of strange students in my office, that's just probably my fellows.
So, say hey
Does anyone have certain props that they want to use?
Does anyone need a hand cart?
Yes
I'll need simple things, like a wire that I can bend.
So maybe a couple, if it doesn't work out the first time
So for those props, I'm going to have you guys get them.
For bigger things, like a hand cart, or if you need another tripod, or if you need a light kit, or something
I think that may be [INAUDIBLE]
Yeah.
If it's simple, like under $20, I'm going to have you guys go ahead and coordinate that, though
Just a quick question.
What font is that?
What font is it?
[INAUDIBLE]
Actually, that font was very, very intentional.
For a while, I tried to make my own font.
But then I was like, that's silly, just use an existing one.
I can't remember what the name of it is off the top of my head.
But we use it to brand Science Out Loud.
But I can send it to you once I find out.
I think it's a Google Font, actually
The O's are different, and the H's are different
Oh, no, that's just stuff I drew.
But I think he's talking about for the text font.
I think it's actually Amatic SC.
It's a Google font
[INAUDIBLE]
I will put it on the Tumblr.
I've got to remember to do that, though
So how exactly do you do that, like draw something on the computer or would [INAUDIBLE]
I don't have any of the examples with me, but I just draw on a piece of paper with Sharpie.
And that's how I do all these presentations, too.
So for day 9, I did this at the airport.
This is basically all it is.
And then I take a picture with my iPhone.
And then I blow out the exposure in Keynote.
It's nothing fancy.
And Sari knows how to animate the things in After Effects.
But basically you take a still image, and you draw a path for the computer to wiggle it around for you
Before I knew how to animate in After Effects, you can just use Illustrator, which is a really good vector art program.
Do you have a tablet?
I have 10 Wacom tablets, in case anyone wants to use them
The best way to translate your handwriting to digitally, so if you want your handwriting to just be digital, then you open up a file in Illustrator that's the same size of your screen, or your video, so, 1928 by 800.
Or something like that.
And you just draw in it, and you can change the width of your lines.
You can redo the shapes that you do, because it's just a readable document.
And then you can export that as an image file.
It'll be the exact same size as your video clips, when you've edited it.
And you can make backgrounds transparent then too.
So you can just have your vector art
You don't have to make it black and white.
I just tend to do it because it's easier for me.
But if you're trying to draw the PDL or something, she's much better at that
[INAUDIBLE]
You what?
I thought that was your font
Oh really?
No, I made it at the airport
So how can you avoid giving [INAUDIBLE]?
Well, I will show you exactly what I did for this slide.
So I said, I'm going to insert this image.
And this is Keynote.
I think you can do in PowerPoint.
You can definitely do it in Photoshop.
This is me just being newb extraordinaire.
I went to Image, and first I flipped it.
And I basically adjusted the levels, so I blew out the background first.
And then I inverted it.
I'm basically taking all the whites to the extreme.
Of course it's not working now.
I promise it does work.
OK, let's try this again.
Sometimes it's finicky.
Does anyone have other questions while I try to get this to work?
I'm going to try a different picture
It makes us all feel better that it's not [INAUDIBLE].
For me, it makes me feel better
No, it is.
I'll go with this one.
And I'm sure there is a very streamlined way that kills fewer trees.
Oh my goodness, I don't know why it's not working.
So if you're doing a black image on a white background, this is all you would do.
This is not clearing the background, so the background is going to be black.
So you would have to do what Sari's talking about as far as basically creating a vector art that has a transparent background
And it's really easy.
The hardest part is getting used to the tablet, because it's weird writing, and seeing your writing up here, somewhere else than what's on the page
Ha, I got it.
That is all I did.
And I cropped it
I've done stuff like this in PowerPoint.
You could do it just as easily in PowerPoint, yeah.
They give you cool stuff to make it look like teeth, or there's all these cool-- and then just black and white.
They're really easy.
But that looks really cool
But for animations, labels, overlay in general, I think simple is better because clarity is most important for graphics, and anything that's not the video.
The video itself also needs to be clear too.
Clarity is very important
Can we use the Wacoms in your office on our own computers, or do we have to use that in the lab?
No, they're are my office.
It's a USB hookup
[INAUDIBLE]
And they're kind of old.
Yeah, they're not the newest kind.
I think they're five or six years old.
But I have a bunch of them.
I didn't bring them with me because I wasn't sure if people would be interested in them.
But yes, please take them, because they're just sitting in a cabinet in my office
[INAUDIBLE]
Yeah.
If you guys want them tonight, I'm happy to go over and get them now.
But I can also just bring them to class tomorrow
Can you overlay the video with animations in iMovie so I can [INAUDIBLE]?
Yeah, iMovie is great, because there's a very low learning curve.
And so, if you're doing simpler things like cutting out the bad bits, people might say iMovie is a newb thing to use.
But I use it for really simple videos that I don't need to do a whole lot of work with.
Unfortunately, for the animations, I don't think it's possible, and that you would need to use something like After Effects.
Or you would need to use Final Cut to incorporate it
So the problem with iMovie is because it's so simple, it only has one video track and one audio track.
And in order to overlay an animation, you need at least two video tracks, which comes in Final Cut or Premier
Something that you can do with iMovie, I'm not sure if some of you guys would do this-- but say you are standing in the side of the frame like this.
And you want another movie to play on the inside.
You can do that in iMovie.
You can do an inset.
And that's something that's pretty straightforward to do
And you can have words show up over you.
It's very easy to do in iMovie.
If it's not an animation, but just a word that you want next to you, that's really easy to do, and have text show up
Or an image.
That would be [INAUDIBLE]
Yeah
I'll show you the video later because I can't find it right now.
You can do insets, really basic things on iMovie.
But animations, you can't.
What you could do, is you could work with Sari in After Effects to get the video clip where you want the animation, and export it as its own file.
And then import it into iMovie.
So you're basically using a polished animation.
That way, if you want to keep working in iMovie for the rest of your project, you can do that
And we need to go to the [INAUDIBLE] lab to use the After Effects?
Yeah, because they only are in installed computers on campus.
And it's literally in that building
And if you end up wanting to work at home, it takes usually an hour, but you can install all the Adobe software for free for a month.
And they don't automatically charge you, because you don't put in credit card information, or anything.
You just sign up with an email address.
And then you can download them for a month trial.
Did teams want to spend the last little bit on a little game plan time?
Don't forget, a daily blog for today.
I think I put this in the email.
The syllabus is updated with all the assignments.
Basically, from now until the end, it's just the video itself that's due.
And daily blogs are just for today and the rough screening day.
But if you take behind the scenes pictures, which I love to do, behind the scenes stuff, feel free to upload that to the Tumblr
So there's no shot list, or anything like that?
Yeah.
The scripts, I think everyone has sent me, for the most part, updated scripts.
The shot list and all that, it's really for your benefit.
I want you to feel as ready as possible to shoot.
Actually, I would like you to upload the shot list.
But I won't set a due date for it.
I'll change the syllabus with that.
But do upload your shot list.
I would like to see it up there by Thursday, at least.
Because if you're not thinking about shooting by then-- does that sound OK to everyone?
Break, go team.
Yay, people.
It'll be awesome.
I can't wait to see your videos.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This is going to be really informal and you can try and follow along if you want.
I'm just going to show you my basic workflow from Illustrator to After Effects to Premiere and some important, I guess basic tools that you might use.
This is Elizabeth.
OK.
I just started.
We finally got everything set up.
I don't know how to make it any better.
OK.
So first thing is creating your new canvas.
So if you open up Illustrator, it's just File New.
You can name it whatever you want.
And then the profile is going to be automatically set to one of the options, but I adjust the width and the height to match the screen size of the video that we capture.
Which is usually 1920 by 1080 pixels.
So you want to make sure the width is 1920, the height is 1080, and the units are pixels.
And you can keep the color mode and the pixel grid the same.
And you press OK. And your screen should look something like this.
And this is just the drawing surface on which you're going to work.
Do you guys want to follow along or should I just keep going from here?
Nod
How about-- sorry, can you go through just like drawing a single shape.
And then maybe draw a person's face and then export it out of Illustrator and then animate it in After Effects and save the movie?
That's what-- yeah, OK. Yeah
Oh, is that what you're going to do?
Yeah
OK.
Sorry.
Go ahead
Yeah.
No worries.
So this is just going over some basic tools.
I like expanding the right tool panel just so I can see everything, especially the layers.
That's a tool I use a lot.
Basic tools that you're going to use are the paint brush.
This is how you mimic writing on your tablet so you can mimic your handwriting.
You can draw basic shapes with this, but they appear as lines.
So everything in Illustrator is a vector.
So this is a solid object.
And then if you want to make shapes, there's a rectangle tool over here and you can hold down on it and find other shapes.
And then you can just drag to place it.
If you want to change the border color, or the color of an individual line, you can click on this.
This is stroke color.
Double click and it'll just give you a full-color spectrum.
You can select anything.
And if you want to change the fill-- right now, the fill, the white box with the red line through it means that it's transparent.
But you can choose any color you want and change the shape.
If you want to draw a free-form shape, the pencil tool will connect lines to make a potentially solid object.
So if you want to make an irregular polygon or some other shape, the pencil tool automatically connects them even if you don't completely draw them together.
You can also redraw lines sometimes if you draw them in the same direction that you originally did.
So you can adjust the border of a shape slightly.
What else?
The black selection tool can help you select entire objects.
You highlight--
I'm sorry
No.
No worries.
Is it not recording?
No, it was not
Do you want me to do a Quicktime recording just as backup?
Yes, as a backup
OK.
So the black selection arrow can help you select entire objects or you can drag to select groups of objects.
The white selection arrow, you can select individual points on objects.
So you can select a point on the path and then move it around.
So you can warp a line that already exists.
And change its direction and things like that.
The eyedropper tool is useful if you-- let's say I took a paint brush and I had a specific stroke weight.
I had a specific color that I was using to draw a circle or a spiral.
And then I reverted back to black at one point, but I want to go back to that size and color.
The eyedropper tool will hold both those informations.
So if I just eyedropper that, then it will turn blue and in 8-point brush.
But if I eye-drop the block, then it'll go back to 1-point in black.
So it just saves you some time if you want to completely recreate something.
The last thing that's important in Illustrator is layers.
I usually divide up my project based on types of objects.
So if I was doing this from scratch, I would probably put all the stuff I drew with a paint brush on one layer and all the shapes on a separate layer.
You create a new layer just by clicking this new little page here.
And it's basically just a way to group objects and manipulate them independently from each other.
So, for example, I will move this square into layer one.
And I'll those animate using layer two.
And you can toggle a lock on each layer.
So if I put a lock on layer one, that means I can't do anything to these objects, like I can't manipulate them accidentally.
And the eyeball triggers whether you can see it or not.
So sometimes you want to draw something in one layer and then you create a couple more on top of it, and you don't want to see the original one.
And I usually name my layers something to remind me what they are for the purposes of After Effects.
Just so it's less confusing when I get there.
So this would be text.
And layer two will be rectangle.
That's pretty much all there is to know about Illustrator.
You can save the files.
It'll save as a .ai, which is important when you animate something.
Save it somewhere where you're going to remember where it is.
And you can just keep the default settings.
So here's where it's going to branch off if you want a still transparent image over your video or an animation.
If you just want a still transparent image, you can draw it in Illustrator like this.
Just click File, Export.
And it'll automatically go to PNG.
Click Use Artboards because that preserves the screen dimensions that you specified in the beginning.
And you click Export.
And it'll give you this pop up window and here is where you can choose whether you want a white background, a black background, or a transparent.
Which is really nice if you just want a text overlay on your video because you automatically have an image with vector art in it, or vector words, and a transparent background.
And you can just click OK, and then your image will be exported.
I think mine's in my downloads folder.
Yeah.
Just as a PNG file, which you can import into iMovie or Premiere or Final Cut Pro.
And overlay it just like another video track or an image.
That is everything for Illustrator.
So I will close that.
Now, if you want to animate, animate in After Effects most likely.
You can also learn Flash animation, but that's a different concept behind it.
So the way to start a composition in After Effects and import your images that you started in Illustrator.
You exit out of the initial window when it asks you to create a new composition because we're going to just import our files from Illustrator as a new composition.
So you just double click in the project window and it'll give you a pop-up window that says Import File.
I'm sure there's another way to do this, but this is just how I was taught.
You can probably go File Open or something like that.
Go to wherever you saved your Illustrator file, enable all acceptable files, and then import it as a composition.
That just means it will process each layer in your Illustrator file as a different layer in your composition, which will be important when it comes to animating.
And then from here, you can alter your Composition Settings.
And you can name your composition.
It will automatically adopt the name that you named your Illustrator file.
You can customize the width and the height, which should be the same dimensions that you specified before.
You want it to be in square pixels.
Sometimes it defaults to a different size, and I think rectangular pixels are silly and they mess up how your video's going to look eventually.
Or how your animations relate to your video.
The frame rate is important because it allows you to match up your animation exactly to your video.
In Adobe Premiere the frame rate is 30.
The default for After Effects is 25.
And that messed me up on my first project because I was trying to time everything and then it was off by five frames per second, which is really frustrating.
I don't know what they are for iMovie or Final Cut, but I would check when you go in before you start your project
[INAUDIBLE]
OK.
I set the background color to white just so I can see everything because I usually draw with a white background in Illustrator.
If you want to set the default background in Illustrator to black and do white text, and set that as transparent in front of your video, then a black background would make more sense.
This background color does not export with the video, though, so your animations are still transparent.
It's just so you can see things in After Effects.
And usually what I do before I start an After Effects project is I look at how long the audio is for the portion that I want to animate.
So for example, I was working with Andrea yesterday and she wanted a goat during one of her lines when she says, and the force of a mountain goat jumping on your teeth is some amount of newtons.
Or whatever a line is.
And you look at the duration of how long it takes you to say that line in your editing software.
And that's how long I set my duration of my animation to be.
Just so it's really easy.
It's really modular.
I'm animating for this part of the video and I have this much space to do it in.
And because I don't have an audio track right now, I'm just going to set it to 15 seconds.
So that's good.
Then you can double click on your composition to open it.
And this is going to be your display window.
This is your timeline.
It's where I guess the animation is taking place and you can see different things.
And here is where you're going to be doing a lot of the animation stuff.
This little Options window, I guess.
You can still toggle the visibility just like you did in Illustrator.
And so let's say-- I'm going to show you how to animate this rectangle, for example.
So the first thing you want to do is right click on it and Create Shapes from the vector layer.
Basically this just allows After Effects to take the vector shapes, or objects that you created in Illustrator, and translate it into a manipulable format for After Effects.
So then you have this new item-- object-- I don't know, called Rectangle Outlines.
And you can look inside in their contents.
If you had multiple objects in the same layer, you'd have multiple groups, but I just have the one rectangle so I only have group one.
And so in order to move this, what I'm going to do is open up group one.
And I have different options.
And you guys are going to be doing a lot of Transform animations.
So these are things like if you want something to move around on the screen, if you want something to change in size or rotate, or things like that.
So what you're going to do for if you wanted-- let's say I wanted this rectangle to move across the screen and grow bigger.
The things that I will-- and in After Effects you want to think in key frames.
So what does it look like at one point in the animation, and then what do you want it to look like in two seconds?
So right now this is where I want it to start.
So I'll click a little stopwatch on position and scale, and these will set key frames for both position and scale.
So now After Effects remembers where the rectangle is at this point in time.
Then I'll set it forward to let's say two seconds.
In two seconds I want my rectangle to move.
And then it's as easy as dragging the rectangle to where you want it to be and then changing its size.
So let's say I want it to be really big like that.
And then once you make those changes, After Effects automatically sets another key frame for you in your timeline.
And then you can review what you want the animation to do by dragging this back and pushing Space, which is the same thing as Play.
And then you can watch how over the two seconds, it transitions from one point in space to another point in space and one size to another size.
You can edit and delete key frames.
You can add more in the middle.
Let's say I wanted it to go up and left before I wanted it to go up and right.
You can just set it to the middle of this animation and drag it up and left to where you would want it to be.
And then it'll keep scaling the size at the same time, but then it will just adjust its position path in the new animation.
And that's the bare bones of animation.
The one other thing you guys might use is-- let's see.
If you want to group multiple items and have them move at the same time, for example-- I'm going to take my text layer, Create Shapes from Vector Layer, and now I have another layer of objects.
Except this one has many contents.
Many groups of things that I can manipulate.
So let's say I want both these spirals to move at the same time.
I haven't found a better way to do this, but-- so I learned that the spirals are groups two and three.
I can click on Contents just to highlight everything, Add a New Empty Group, and it's just labeled whatever the highest number-- one after the highest number.
You can rename it to something like Spirals so you remember it.
You can highlight both of these and just drag it into the new group.
And that way you can transform spirals as a unit.
Let's say I wanted them to rotate.
And so then-- OK. That's kind of rotating.
And so it set the key frames for rotations, but instead of just manipulating one spiral I have two grouped together.
And then the rectangle animation is still going from before
Can you [INAUDIBLE]?
What do you mean?
With the point they pivot on?
Pivot point?
I think so.
I don't usually rotate things in my animations.
I think it's probably the anchor point
See the cross-hair?
Yeah.
So that's the pivot point.
I don't know how to change that necessarily.
OK.
There you go.
You can just adjust it in relation to the anchor point.
So you can change this value.
Yeah.
A lot of this is just playing around and seeing what you want to do.
You can ask me questions afterward and I'd be happy to troubleshoot it with you.
Yeah.
So I guess that's how you pivot.
And so it'll maintain the rotation just around the smaller anchor point now.
Let's see.
This is might be useful.
So if you go in and you have a lot of work-- so for example, in a different project what happened was I had the outline of a cell and I was showing contents of the nucleus.
And when I screened it to my project advisor, she said it's great but we're not focusing on the nucleus of the cell.
And so I can't really see what's going on.
So I had already done all this animation with individual squiggles of DNA and proteins moving around in the nucleus, and I wanted to preserve that animation, but I needed to just zoom in on everything.
And you can do this with a thing called Null Object.
Let's see if I remember how to do this.
OK. Layer New and then a Null Object.
And so it'll appear in here as just Null One.
And basically it's a way to group multiple animations at the same time.
You can just leave that named Null One I guess.
So let's say I want the spiral and the rectangle to both be smaller because they're both way too big.
So then you can you can set the Null Object as the parent of these layers.
It says Parent and you can have a drop-down menu.
And so you can set both these layers to have their parent as Null One, and then you can animate Null one as though it were an independent object.
So let's say I just want the scale of everything to go down.
So because those are both groups that are under the Null Layer, then the scale goes down.
Or you can animate it independently.
So let's say I want the scale at 100%, but as the rectangle grows I want everything to shrink because it got too big.
So then you can turn the scale down of everything.
And so when I run the animation, the rectangle is going to grow, the spirals are going to rotate, but then everything, because it's all grouped under the Null Layer, is going to shrink.
And so I guess those are the basics of it.
Once you have your After Effects file, I would save it as something just so you don't lose anything.
And then you can do one of two things.
So if you want to edit in Premiere, you can start-- Adobe products are really nice where they auto-update to each other.
So if you change the Illustrator files that make up the composition in After Effects, then your After Effects will change to reflect those changes.
So if you messed up drawing something, or you want a shape to look a little bit different, if you update the Illustrator file your After Effects project will also update.
Same thing with your After Effects Project and Premiere.
So if I have a Premiere project, you can drag your After Effects composition into Premiere just to import media.
And then I have my test animation and I can drop it here to create a sequence.
And so right now, because it's transparent, I can't see anything.
But if I were to add a color match in light blue-- this is going to be really garish-- and add it on the video layer below the animation, then it will appear.
So the animation itself is transparent.
And so now if I make changes in After Effects, it'll automatically update those changes in Premiere, which I find really handy as I'm video editing and switching back and forth.
If you don't want to edit in Premiere, that is totally fine, and you just need to export your composition.
I was just playing around with this earlier.
Usually I don't export, so I don't necessarily have the ideal settings for it, but we can play around and see what works.
So File Export to Adobe Media Encoder.
And then it adds it automatically to the queue.
And you can choose preset export settings over here.
I created a preset for the videos that I'm working on and so I usually just select that.
And then once you do all that, then you can just push play.
And then the media encoder does its thing.
I don't know exactly what it does.
I think it's a combination of rendering and just making sure-- changing the file format from the After Effects Project to, in this case, an MP4 file
So, for a lot of yours, I'm imagining you want to do Animation Overlay, right?
So you can't do that in stuff like iMovie, so what you will have to do, which is a super techy workaround, is you create the animation in After Effects, you go in Premier and that's where you overlay it on top of your live action footage, and then you export that little chunk into your Add Movie Project.
Does that make sense to you guys?
So would you mind showing real quick how to do it in Premiere where you overlay an animation on top of your live action footage?
Sure.
So it's basically just doing what I was doing and just using something besides the color mat.
I will see if I have any movies on my computer.
I don't know if I do
So if you are holding your hands out like this, what's the best way to time and place the animations?
Oh,
Because you basically have to draw them without seeing a background in After Effects, right?
Yeah.
OK.
I have a video.
So the way I would animate in After Effects is to use the time codes.
So Premiere has a time code, and it's right here, and it matches up with where the movie is.
So right now the animation is overlaid for my movie, and the formatting is weird because I didn't do the movie first.
What I would do is, let's say for example I wanted at five seconds something to happen, like something to pop up above my hand, then I would take note of the time code or whenever you raise your hand in the video.
So let's say it was at 11 seconds and 24 frames.
Then you go back into Adobe After Effects and you can scroll over to 11 frames and make sure the time code here is listed here.
And so you just switch back and forth and adjust the time code of your animations to the time code where you want it to appear in the video.
And you can drag key frames to adjust that.
So if I say, well the rectangle moved too fast in this video, I actually want the animation to end at 3:29 instead of two seconds, I can drag my cursor to 3:29, and then just drag the key frame to match it.
That's usually how I do it just because it's a lot easier to visualize.
And then it will automatically update in here and the animation just got a lot longer.
And so now when I push play, instead of taking two seconds it's still going until 3:29.
It's a lot of playing around.
I don't have a really, really concrete method of doing it.
And I'm still learning animation too, so I don't have the perfect answer, but usually it's a lot of switching between windows, which is why when you have two monitors it's really nice.
But a lot of just checking the time code exactly and making little adjustments at a time.
It's kind of like how you edit a movie.
You start with a rough cut.
You start with a rough animation of what you want, and then you can adjust the key frames to time it to the video that you're working with
So I think what will be most productive for you guys is if you attempt to do your own animation and then if you run into any problems, email [INAUDIBLE], or you can email me.
We'll be around tomorrow if people are working, and I'll be on campus tomorrow actually if you need help.
But I think the most efficient route is if you try a little bit on your own first and then when specific questions pop up you can come ask us
Yeah.
And if you have anything specific in mind that you don't think you can accomplish with what I've taught you so far, then you can definitely come talk to me or I can come over and talk to you.
And we can talk through what your picture was for animating something and how you can best do that either using the tools that I just taught you or using other things.
Like I know how to mimic handwriting to write out a word.
And I don't know if that's necessary for any of you, but that's a thing that you can do in After Effects.
It's just a little more complicated than what I've shown you so far
I have two questions.
One of them is I have used Adobe Premiere before, but in the sizing, of course I'm not sure when I'm doing [INAUDIBLE] it looks like the sizing is correct.
However, when I export it into external video, the sizing was wrong.
So is this a thing?
For exporting from Premiere?
I mean like, if I see the screen like this, is this everything that's on the screen that will be shown in the actually video?
Yes.
So what this is-- I recorded this movie on my camera on my computer, which is lower quality than your camcorders, which should record in 1920 by 1080.
So right now, this in the upper right hand corner is what's going to be exported.
So if I exported it right now, it would export with this weird black border because my video is smaller than this composition
So it's preferable to expand the screen?
Can we expand the video?
Yeah, you can individually-- you can double click, and this is how you digitally zoom.
Basically.
You can manipulate it just like any other object and make it bigger so it fills the screen
[INAUDIBLE]
What do you mean?
So this [INAUDIBLE] would be the one that--
Yeah.
So whatever you see in this preview screen is what's going to come out when you export it.
And then you can change export settings.
File Export Media.
And you can change the file type that you want it to be and the video format.
I use the settings that MITx needs to upload usually just because it works and it produces a high enough quality video.
But if you know things about audio and video and things like that, you can add a copyright encoded into the video itself if you really want to protect it and things like that.
But less necessary
If you're going to check the camera we're given, it's how many frames per second?
I think it's 30.
I'll double-check it
And if I want to get a-- let's say I have video.
I took a video.
But at the moment I want to cut that frame and extend that frame to make it like a picture.
[INAUDIBLE] explain the concept.
So how do I do that?
So this is something-- I don't know how to do it in Final Cut.
I know how to do it in Premiere.
So hypothetically in Premiere-- that's a bad frame pause.
Let's say I want to pause this frame, there's a button here that just says Export Frame.
I click it and then what do I want it to be?
Still Face.
I just wanted a JPEG, I guess.
And then this box just means that you import it into the project automatically.
So no matter where it saves it-- in this case it's saving it in my Premiere folder-- you click OK and then it automatically appears as an image in your media browser area.
And so then if you want to extend this frame, what I would personally do is use the Razor tool.
Cut the clip into two.
Use the Selection tool and drag this over so I have some space to include it.
And then include the still image.
And then just seal that gap back up.
So now, instead of just seeing me talk, I'll talk, I'll talk, I'll talk, and then the still frame will happen for however many seconds you set it, and you can adjust the size and timing of that, and then the video will resume when you're ready for it to do that
I don't exactly know what the frame rate of the [INAUDIBLE] is because it records both in 24 and 30, but if you open the video up in Quicktime and you do Command I, which is what it was recorded in.
And I just found that out by Googling it, so don't underestimate the power of Google
Are there any other questions about animating or editing or Adobe?
I mean, we're just going to stick around for the rest of today so if you just want to hang out here and work or you want to go back to the classroom and work, we'll be around to answer your questions.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Basically, you're not going to able to use Beatles clip, unfortunately.
But can I hear what the song is?
Yes
So the reason why this is under Creative Commons-- which, it's a little sketchy.
I'm not sure if they ligitimately could have done that.
But it's because they've transformed it so much, to the extent that it's, like, no longer the original work, right?
So if you revert it to the original work, then you're also reverting-- you, like, taking away the Creative Commons work basically.
[MUSIC PLAYING] And you like this instrumental?
Uh huh
You could find something like this on ccMixter or a different music archive.
It'll take a while, and unfortunately I don't have a short cut.
That's a problem with music, is that it's not catalogued very well.
So it's time consuming, because you just have to listen to everything
What would be, like, a good mood to search for that?
I wouldn't necessarily search by mood.
I would search by instrumentation

So, have you looked at any of those links that were up on the syllabus?
STUDENT 1: Uh huh
I mean, this is sort of, like, chill rock basically.
STUDENT 1:
And that may not actually narrow it down.
Go ahead and try to explore other genres, too.
I-- unfortunately, there's no quick algorithm for finding good music.
That's why it's so time consuming.
But let me see what you've got cut so far.
STUDENT 1: OK, the video?
All right
Do you mind if I bump up your brightness a little bit so I can see it better?
STUDENT 1: Oh yeah.
And I'll delete this song for now.
[VIDEO PLAYBACK] -Hi, what do snowflakes and cell phones have in common?
Well, let me start by drawing a snowflake first.
First, I'm draw an equilateral triangle, divide each side into three parts, and draw another equilateral triangle on top of each one.
Then, take out the middle and repeat the process.
This time with one, two, three, four, times three, which is 12 sides.
Eventually, the shape will start looking something like this.
In mathematics this is called a Koch snowflake.
If I repeated my process again and again, I would see this same pattern anywhere I looked.
This is a Koch snowflake, this time-- STUDENT 1: And this would be an animation.
I just haven't inserted it
Sure.
- --that on any scale, on any level of zoom look roughly the same, are called fractals.
Computer scientists can program these infinite patterns by repeating an often simple mathematical process over and over.
In the 1990s, a radio astronomer named Nathan Cohen, used fractals to revolutionize telecommunications At the time, Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So Cohen designed a more compact, fractal radio antenna instead.
The landlord didn't notice it, and it worked better than the ones before.
Working further, Cohen designed a new antenna.
This time, using the fractal called the Menger sponge.
The Menger sponge is not really the sponge you'd be scrubbing your bathroom with.
But you can still think of it like that.
Imagine both water and soap getting through your sponge's holes.
Except the water is Wi-Fi and the soap is, say, Bluetooth.
The fractal's infinite sponginess allows it to receive many different frequencies simultaneously.
Before Cohen's invention, antennas had to be cut for one frequency.
And that was the only frequency they could operate at.
So without Cohen's sponge, your cell phone would have to look a hedgehog to receive multiple signals, including the one your friends use when they call.
Cohen later proved that only fractal shapes could work with such a wide range of frequencies.
Today, millions of wireless communication devices-- [END PLAYBACK] STUDENT 1: Oh, sorry my bad
Oh, was that the music?
STUDENT 1: I was testing something
OK, I was like, what's happening when you guys were recording.
STUDENT 1: Yeah, I'm going to delete that.
Come on
Here, let's go-- STUDENT 1: It was like-- earlier.
Somewhere up there-- this.
[VIDEO PLAYBACK] - --genuis invention, however, was not the first application of fractals in the w-- [END PLAYBACK]
Why is it doing this.
STUDENT 1: Yeah, you can just do this.
And it's around here.
I guess it's-- [VIDEO PLAYBACK] - --including laptops and bar code scanners used Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
It tuns out, nature has been doing it the whole time, and not just with snowflakes.
Natural selection favors the most efficient systems and organisms, often of a fractal shape.
For example, the spiral fractal is present broccoli, seashells, and hurricanes.
The fractal tree is relatively easy to program, and can be used to study river systems, blood vessels, and lightening bolts.
So many natural systems previously thought off limits to mathematicians can now be explained in terms of fractals.
Plus, mathematics allows us to learn nature's best practices, and then apply them to solve world problems.
Much like Cohen's antenna revolutionized the field of telecommunications, other fractal research is changing medicine, weather prediction, and building design-- here at MIT, and everywhere in the world.
Look around you.
What beautiful patterns do you see?
[END PLAYBACK]
All right, that's a good start.
Good start.
So I think in general, the transitions could be a little bit tighter.
So, like right here.
When you walk off-- well, this hiccup up is really noticeable.
So either hide it by having an extreme close up before you leave, or just get rid of this entirely.
Although, I know that you did that to show the walk off, so that you could show walking in.
STUDENT 1: And I can connect-- this is actually one video I separated so I could speed this up.
And I guess I was just playing around with that
I almost think this happens too quickly, though.
STUDENT 1: Yeah, it's at 140 right now.
So I was just like--
Yeah, I would slow it back down, yeah.
So let's go scene by scene.
[MUSIC PLAYING] Is this track Creative Commons?
STUDENT 1: No, it's Aerosmith.
I would take that out.
I was just looking for an interem--
There's probably an iMovie sound effect, actually, that you could use.
STUDENT 1:
For this part, where you're just, like, walking around, I would digitally zoom in close up.
Because otherwise, you jump to talking all of a sudden.
But your movement wasn't exaggerated enough to where you, like, get the joke that you're talking.
So you can exaggerate it by changing the actual framing of the scene.
STUDENT 1: So would I zoom in when I start talking?
I would stay zoomed in when you're talking, and then clean cut to a wider shot of you afterwards.
So basically, I would cut, like, right there.
I don't remember how-- my version's a little but different than yours.
But I would cut right there.
And for the preceding clip, do you know how to crop and zoom on clips?
STUDENT 1: Yeah, I can
You just go like that, basically.
Yeah, for that.
Now, for this I would also-- this is basically one, really long shot that is framed the exact same way.
And it's very lectury, because you're literally at a blackboard.
Although I think that that was a good set up for you guys, given the complexity of explaining this.
But I think you can avoid making it sound too chalkboard lectury by cutting in and out of your frame.
STUDENT 1:
Or, of your framing.
So this is fine.
Like right there, I would save your audio but just cut to a close up of your hand drawing it.
Does that make sense?
STUDENT 1: So kind of like with the phone?
When I'm showing--
Yeah, yeah-- I would do more of that.
I think in general, you have very static shots that last for a long time.
So you can break that up by keeping the same track from your one long take, but mixing the extent to which you've zoomed in.
STUDENT 1: OK, yeah, that's what I did with the phone, so I can definitely work on that
I think that would help.
Because this section is pretty long.
Like, if you just even look at your project, this file is 32 seconds.
STUDENT 1: Which, it's-- the video is about three minutes and 40 seconds right now
OK. STUDENT 1: And then, like, this part is because I inserted the song
Oh, OK, great.
[VIDEO PLAYBACK] - --repeated my process again and again, I would see this same-- [END PLAYBACK] STUDENT 1: So I guess you could zoom in at "this same shape."
I actually-- I feel like you're revealing this image too soon in your sentence.
"If I repeated this process over and over again, I would see this image."
And I almost don't want to see the image until you say "this image."
STUDENT 1:
So maybe if you can zoom in, and frame, and crop out the fractal zoom in until you say "this."
And then at "this" you can either pan to the drawing, or you can cut to a wide shot.
STUDENT 1: OK. [VIDEO PLAYBACK] -If I repeated my process again and again, I would see this same pattern anywhere I looked.
[END PLAYBACK]
And anywhere you looked-- I mean, is there a way to, like, shoot footage of just close ups of the snowflake?
Of you, like, going around the edge of the periphery of the snowflake?
STUDENT 1: So that would this part
That would be that part, OK. STUDENT 1: Yeah.
[VIDEO PLAYBACK] -This is Koch snowflake-- [END PLAYBACK] STUDENT 1: Which, this is a software-- a fractal drawing software.
Am I allowed to, like, film the zoom in on that?
And then use--
It depends on what license the software is released under, and who created that image.
STUDENT 1: OK.
So, I mean, I didn't have to pay for it or anything
Oh, you created the image.
STUDENT 1: No.
Well-- so, kind of.
It's just a software.
And then you can do-- let's say you draw a Koch snowflake, and then you can just zoom in on it.
And then I just--
And that is are free, open source software?
STUDENT 1: Yeah, it was free
You know, what you should do is?
You should just take a QuickTime recording of your screen.
Like what we were doing.
STUDENT 1: That's what I was going to do
And just, like, do a QuickTime recording of you zooming in.
STUDENT 1: And that's OK to use?
Yeah.
All STUDENT 1: Right
Oh wow, this is so trippy.
How do I make it stop?
STUDENT 1: I don't actually know.
I haven't worked too much-- there is actually a video of somebody doing that, which is well done.
So I may use part of that
You-- ideally, you would make your own video.
STUDENT 1: Yeah, I'll work on that.
Maybe I'll just, like, track their movements at least
Yeah, because I don't know what the details of that would be.
If you're actually allowed to use a video.
Unless they've specifically released it under Creative Commons.
All right, so you're here.
[VIDEO PLAYBACK] -This is a Koch snowflake.
This time, drawn on a computer.
Such never ending patterns that on any scale, on any level of zoom look roughly the same are called fractals.
[END PLAYBACK]
OK, this is where you change your wardrobe all of a sudden.
Ideally, that wouldn't happen.
Wardrobe changes are OK if it makes sense.
But here it seems, like, a little arbitrary.
But it's not the end of the world.
Does that makes sense why that seems a little bit weird, though?
Because it's like, in your script there is nothing that sort of signifies or identifies that you've shot, like, at a different day.
Or that you're in a location that requires a different wardrobe, you know?
[VIDEO PLAYBACK] -This is a Koch snowflake.
This time, drawn on a computer.
Such never ending patters that, on any scale, on any level of zoom look roughly the same are called fractals.
[END PLAYBACK]
OK, I'm just going to say it.
How do you pronounce Koch snowflake.
STUDENT 2: I think it's "coke," but I don't know
I'm pretty sure-- it's K-O-C-H, right?
Just cause I would imagine that it's educators watching this video, but if there are little children watching this one, there are going to be some, ha, ha-- she said cock snowflake comments.
STUDENT 1: I was actually thinking of doing-- because the sound changes so much, I was actually thinking of doing a voiceover of this.
Because what I found-- I recorded this part indoors and outdoors, and I could match up the sound perfectly for what I was moving.
So if I, like, do the same thing but somewhere else, I can probably match up the sound and change the snowflake part
Helge von Koch.
Yeah, I would just look up the pronunciation of that, just to-- STUDENT 1: OK, I will do that
Just be careful.
Yeah, and especially-- this is just video stuff anyway, so you can easily re-record that.
So wait for the reveal here.
Basically, do a little more with cutting in and out of close ups and wide shots.
OK, right here.
[VIDEO PLAYBACK] - --are called fractals.
[END PLAYBACK]
So this reveal is a little awkward, because you're not looking at the audience when you're saying the big reveal.
And then you have this sort of unnatural pause before you start the next sentence.
STUDENT 1: OK, I was actually originally intending to say fractals here, but I cut it--
Oh, OK, I see.
STUDENT 1: I thought, oh, well I'll just say that.
But originally, it was going to be I finished saying fractals, and then I type on the computer and say--
OK, that will work a lot better.
STUDENT 1: So I can just redo that
OK, cool.
I mean, I think it's also-- I think ideally, you would say, fractals, and then you'd look down at your computer.
But what you're saying now, that'll work too.
STUDENT 1: And that's how we filmed it, so--
Yeah, because this, like, gap in where you take a breath-- it sounds fine, but it looks a little awkward.
[VIDEO PLAYBACK] -Computer scientists can program these infinite patterns-- [END PLAYBACK]
So this is super overexposed in the background.
So I would try to see if you can bring the exposure down just a little bit.
STUDENT 1:
I don't remember how to do that on iMovie.
STUDENT 1: It's on this.
I actually increased the exposure, because there were some shadows.
But I can play around with that
I mean, the main thing is the whites, right?
Like, you're super blown out in the window.
[VIDEO PLAYBACK] - --infinite patterns by repeating-- [END PLAYBACK] STUDENT 1: No, it was different in the original one
I don't know if there's a history on iMovie, but if you can somehow just, like, increase the blacks but don't blow out the whites, that might help if you have weird shadows going on.
STUDENT 1: I think-- so it looked like this originally I think
Yeah, that's better.
STUDENT 1: Yeah, I don't know.
I was playing around with that.
But that's not the final version
OK, cool.
STUDENT 1: But that was the original
Got it.
Oh, yeah, I remember-- this one also feels pretty long to me.
[VIDEO PLAYBACK] -Computer scientists can program these infinite patterns my repeating an often simple mathematical process over and over.
[END PLAYBACK]
So, for that line, do you have footage of you on MATLAB graphing a fractal?
STUDENT 1: I can do that.
Because that would probably be on the fractal software.
I can just--
Yeah, even that.
Like, just to break it up a little bit and give them, like, a visual association.
Because this is one of those times where if you're like, again, in static place saying a big chunk of words to them, it's sort of easier to tune it out a little bit.
[VIDEO PLAYBACK] - --fractals.
Computer scientists can program these infinite patterns by repeating an often simple mathematical process over and over.
In the 1990s--
Yeah, and then cut back out to you when you say-- I mean, because that's, like, a very separate thought.
- --a radio astronomer named Nathan Cohen--
Is there a picture of Nathan Cohen that could, like, pop up on the side?
STUDENT 1: I can probably find that, yeah.
There is one, actually.
- --to revolutionize telecommunications.
[END PLAYBACK]
Oh, OK.
So half way through the video, you switch to these, like, cube turn transitions.
I mean, this is a personal preference.
I don't know how you fell about transitions.
But I feel like, either keep it all the same or use, like, this a little quirkier transitions for a very pointed reason.
STUDENT 1: OK, for this one, because it was such different locations, I used these to tell the story.
So this is me introducing the story and telling the story.
So just for that story I'm using those transitions
Yeah, I think if you're going to do that, I much prefer a push rather than a cube turn.
Because we've done that once before, and-- again, personal preference.
George had to really sell me on the idea, because I was very opposed to it.
But instead of doing a rotation, you just-- it's a wipe, basically.
One frame pushes off and reveals the next one.
I think that looks a little bit better.
STUDENT 2: Basically, I prefer as subtle of a transition as possible.
And the animations are cool, but they remind people of, like, really flashy homemade videos as opposed to something professional
Yeah, like this is a very iMovie transition.
STUDENT 2: Yeah.
[VIDEO PLAYBACK] -At ths time, Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So--
There's some-- wait, let me listen to this again.
-Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So, Cohen designed a more compact fract--
There I would zoom in.
Cut to a closer framing up that that's, like, just the wire.
- --radio antenna instead.
The landlord didn't notice it, and it worked better than the ones before.
Working further--
And then cut come back out.
- --designed a new antenna.
This time, using the fractal called the Menger sponge.
[END PLAYBACK]
And again, I don't think it's necessary to do a rotation.
STUDENT 1: OK, and I was thinking here, while-- maybe have the Menger sponge increase in size, and then curtain as a background.
So would that be another transition, to kind of have the Menger sponge expand and then have me wipe it off?
STUDENT 2: Are you drawing a Menger sponge?
STUDENT 1: Going to have a model of it, hopefully.
STUDENT 2: So you are-- sorry, are you going to re-film the part with, like, a model?
STUDENT 1: No, just like and image.
STUDENT 2: OK,
So it would pop up in your hand right there.
So the way this is framed-- I mean, the subject is really here-- is going to be here-- and you're so biased.
I mean, this is beautifully framed in terms of, like, rule of thirds and everything.
But because you held your hand out this way, it's going to look really fun-- like, the visual balance of that frame is going to look really funny when you're like, Menger sponge, and then all of a sudden this huge thing gets cut off on the side.
So either zoom in a little bit-- ideally, you would have held out this hand so that it would have been in the middle.
Although the background is really distracting, so I don't know if-- [VIDEO PLAYBACK] -Menger sponge.
The Menger sponge-- [END PLAYBACK]
And are you going to hold-- so, I'm assuming the Menger sponge picture will be floating somewhere.
STUDENT 1: It would be here, yeah.
But then I would get rid of that when I'm talking about the regular sponge
Well here it would still be floating.
Because I think when you say, the Menger sponge isn't exactly like a real sponge, right?
But I think the Menger sponge needs to be shown.
STUDENT 1:
If you're going to show something, show it long enough to where the viewer can process it a little bit.
Otherwise, it's like this blip that might distract them.
STUDENT 1:
Did you cut right here, or is there a little bit more?
STUDENT 1: I think there's a little bit more
OK, yeah.
Hold it just a little bit, and maybe cut to-- again, cut to a close up shot.
That can be sort of a workaround to the weird framing issue.
Otherwise it's like, it's really nicely done.
[VIDEO PLAYBACK] -The Menger sponge--
Yeah, you can say, the Menger sponge, and then pop up right there.
- --is not really the sponge you'd be scrubbing your bathroom with, but you can still think of it like that.
Imagine both water and soap getting through your sponge's holes, except the water is Wi-Fi and the soap is, say, Bluetooth
Yeah, so here the timing of you turning on the water is a little bit weird.
Because you turn it on, like, way after you say water and soap.
And I wonder, were you planning on doing an animation for this section?
[VIDEO PLAYBACK] -Imagine both water and soap getting through your sponge's holes.
[END PLAYBACK]
Because I feel like this is a spot where it actually would be beneficial to not even have an overlay, but a completely separate animation of, like, a sponge with holes and, like, H2O and soap with arrows pointing.
Like, going into the sponge holes.
What do you think, Zoe?
STUDENT 2: Yeah, or just to, like, demonstrate that concept.
Because it's easier to understand when you read it than when you hear it in a couple of seconds.
STUDENT 1: Yeah
And I think, like, the visuals of this-- again, it's a pretty long-- it's, what, 14 seconds?
It's actually kind of long to have the same sort of still framing.
STUDENT 1: If you like how it's framed, it's another-- you could also just overlay like, a little arrow pointing in.
Because you hold it pretty still.
[VIDEO PLAYBACK] -But you can still think of it like that.
Imagine both water and soap-- [END PLAYBACK]
Yeah, like, you could just have H2O and soap with arrows pointing into the sponge, and then coming out of the sponge.
[VIDEO PLAYBACK] -Except the water is Wi-Fi and the soap is, say, Bluetooth.
[END PLAYBACK]
Yeah but I would almost-- you can leave it in for the rough cut and we can see what other people say.
But the movement of that is almost distracting, because it has nothing to do with what you're saying when you're saying it, right?
Do you have another take of this?
STUDENT 1: All of them are kind of like
Oh, the exact same.
OK. STUDENT 1: There might be one when I finish talking and then turn on the water, so I can use that
Yeah, because if you cut it right here, and you didn't turn on the water, then that would have been better.
STUDENT 1: I think there are versions like that that I can use
And again, this one-- like, I wouldn't even do it a push at all.
I really think it looks a lot better when you do as clean transitions as you can.
[VIDEO PLAYBACK] -The fractal's infinite sponginess allows it--
I would shave off, maybe, like a half second.
Because you take a breath, and then it takes just a hair before you actually start talking.
So I'd just snip the tiniest bit off of the beginning.
Are you planning on doing any animations to the dies?
- --allows it to receive-- [END PLAYBACK] STUDENT 1: I probably can put them in the sponge, there.
And, like, signals coming inside
Yeah, because otherwise-- again, like, nice rule of thirds.
But you have a bath-- is that a bathroom stall?
STUDENT 1: Yeah
Yeah, so it's like a really awkward visual.
But if-- STUDENT 2: It's a good background
It's a good background if you want to put, like, a picture of a Menger sponge receiving different signal.
STUDENT 1: All right, I can see that.
[VIDEO PLAYBACK] -Before Cohen's invention, antennas had to be cut for one frequency
Yeah, so show an old antenna.
-And that was the only frequency they could operate at.
So--
And like, right here.
-So without Cohen's sponge, your cell phone would have to look something like a hedgehog.
[END PLAYBACK]
I would-- there's like, a bit of a hesitation before you say, "without Cohen's sponge."
And I don't even think you need the so.
So I would just cut out the pause and the so, and cut straight to, "without Cohen's sponge."
And then to hide that cut, you can either zoom in little bit more-- STUDENT 1: OK, that makes sense
Yeah, and like, for that, since you're not going to have the animation, I would frame it and zoom in to where it's just you.
And the resolution is going to look a little funky, but I think it's worth the compromise to do that.
STUDENT 2: And it helps lead to start leading the viewer into seeing your phone.
Because it's like, you're starting out with a wide shot, and now you're on you, and then eventually it will be your phone.
And so it's like-- it just sort of helps the narrative along
And again, I wouldn't ripple.
I would just go straight to the cut into that.
[VIDEO PLAYBACK] - --multiple signals, including the one your friends use when they call.
[END PLAYBACK] STUDENT 1: So I'll take out that transition
This one you could-- because it's such a different-- wait, is that the end?
STUDENT 1: The end of the story
Yeah, maybe you can do a push.
[VIDEO PLAYBACK] -Cohen later proved that only fractal-- [END PLAYBACK]
But it's still part of the same story, so I would say just do a clean cut there, too.
STUDENT 1: OK. STUDENT 2: I like that background
But it is a little-- it's still a little overexposed here, too.
STUDENT 1: Yeah, that was another one I edited as well
Yeah, so clean transition-- I would cut out this gap, too.
It's just like the slightest-- it's too long by just the slightest amount of time.
[VIDEO PLAYBACK] -Cohen later proved that only fractal shapes could work-- [END PLAYBACK]
And that's actually what makes it read sort of newscastery.
You know, like, have you ever seen the special segments, and the reporter is on their couch, and they're like, da dah da dah da dah.
And then this person did this.
So it's just, like, those tiny little gaps that make it read that way.
But if you get rid of it, then it will be a lot more natural.
[VIDEO PLAYBACK] - --such a wide range of frequencies.
Today, millions of wireless communication devices, including laptops and bar code scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
[END PLAYBACK] STUDENT 1: So I'm going to change that audio as well, because I think I was too grand.
And I have other versions that may work better
OK, I like this a camera slow pan off, actually.
STUDENT 1: We did another version which had also this pan.
It wasn't as good, but I said the next sentence after that.
So it sounds more natural.
So I'm going to see if I can trade off the visual for better--
Do you need something?
OK, sorry.
OK, so this is why this transition is a little bit funny here.
The speed at which you're moving the camera when you pan off is way faster than when you pan down onto you.
So what you could do is cut this clip into two, and speed up the first part.
Like you did earlier.
Also, is this broccoli?
STUDENT 1: It is broccoli.
There's a mural, and stata
I didn't know that was broccoli.
It looked like bags of seeds or something.
STUDENT 1: And maybe it looked like broccoli enough
It looks like cauliflower.
[VIDEO PLAYBACK] -Turns out, nature has-- [END PLAYBACK] STUDENT 1: It actually may be cauliflower.
I think I confused the words when I was filming it.
So yeah--
IMovie, I hate you.
OK. STUDENT 1: So if you just press here
I mean, it's indistinguishable.
It looks like bags of stuff almost.
STUDENT 1: I can still insert just an image of broccoli, so I don't-- if it's not broccoli.
[VIDEO PLAYBACK] - --devices, including laptops and bar code scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
[END PLAYBACK]
Yeah, the pan just needs to happen sooner.
I would even speed the pan off, and then speed up the pan down.
Right?
Cause it's like you're waiting for the camera to move off of you.
[VIDEO PLAYBACK] -Cohen's genius invention, however, was not the first application of fractals in the world.
[END PLAYBACK]
Are you planning on re-shooting any of this for sure?
[VIDEO PLAYBACK] -Turns out, nature has been doing it the whole time.
[END PLAYBACK] STUDENT 1: Some scenes.
I can re-shoot this one, and the one in the classroom.
The ones in east campus, I don't think that I can do that
Yeah.
[VIDEO PLAYBACK] - --organisms, often of a fractal shape.
For example, the spiral fractal is present in broccoli, seashells, and-- [END PLAYBACK]
Yeah, because I think especially with this one, go ahead.
STUDENT 2: Oh, I know-- I think the broccoli-- the spiral fractals are most present, it's like romanesco broccoli or something like that.
And so like,
It's a specific type.
STUDENT 2: And so people might be confused, and like look at their normal broccoli, and like--
I would show a picture of the romanesco broccoli.
Especially because I don't think this is-- I think this is bags of barley, or cotton almost.
STUDENT 1: That would make more sense.
It's also why the person is up there
Yeah, and I think, also, it just like doesn't really look like a fractal.
Whereas the broccoli actually-- there are visualizations of it where it actually looks like, oh, I can identify this as a fractal.
STUDENT 2: And so it will just be reassuring for your viewer when they can see the picture of the broccoli and be like, oh, I can totally see how that's a fractal.
Rather than, like, squinting at your background and being like, is that-- did I learn anything from this video type thing
If you're going to re-shoot this part-- and if you can't get it this weekend, that's OK.
Remember, you still have Wednesday to do it.
I would be careful about the way you're emphasizing words.
It's kind of falling a little into the trap of what George was saying about how news presenters emphasize every third word, just a sound sort of-- they're emphasizing important thoughts, but they don't really make sense where they're emphasizing the words.
So I would just try a few different types of deliveries.
One where you're actually emphasizing every other word on purpose.
And then do something, like, completely exaggerated.
And then try one where you're sort of more natural tone talking.
STUDENT 1: I think that's how I usually present.
Like, even in real life.
[VIDEO PLAYBACK] - --the whole time.
And not just with snowf-- fractals in the world.
[END PLAYBACK]
Remember that eighth grade PHD Thing.
You're talking for eighth graders, but you're not talking necessarily to a 10-year-old, right?
So how would you say this line-- [VIDEO PLAYBACK] -Turns out, nature has been doing it the whole time.
[END PLAYBACK]
How would you say that line to, like, your professor?
STUDENT 1: I mean, I wouldn't say this line to my professor
How would you say that line to Jamie?
STUDENT 1: Turns out, nature has been doing it the whole time
Yeah, that's a little bit different.
Like, turns out nature's been doing it the whole time.
It's such a subtle nuance, but it's just something about, like-- [VIDEO PLAYBACK] - -- bar code scanners use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
Turns out, nature has been doing it the whole time.
[END PLAYBACK]
I think it's the way you say time.
Nature has been doing it the whole time.
I mean it's OK to say the whole time, but it reads like a little Reading Rainbow when the intonation is like that.
But again, maybe that's just my personal taste.
In general, I think your enthusiasm is really, really good.
Like, you're taking a subject that's inherently a little bit hard to visualize.
It's a little bit tedious.
And because you have this enthusiastic delivery, it's a lot easier to follow along with you, which is good.
So don't lose any of that energy, for sure.
[VIDEO PLAYBACK] -And not just with snowflake.
Natural selection favors the most efficient systems and organism, often of a fractal shape.
For example, the spiral fractal is present in broccoli, seashells, and hurricanes.
[END PLAYBACK]
If you re-shoot this, just make sure that you're framed more in the center, and there's enough room above you to accommodate for the animations.
And this is actually something that you could, if you had to, you could do on your own.
You just set up the tripod and flip the screen around you so you could see.
You wouldn't have the pan down, but that's OK. [VIDEO PLAYBACK] -The fractal tree is relatively easy to program, and can be used to study river systems, blood vessels, and lightening bolts.
So many-- [END PLAYBACK] STUDENT 1: Yeah, I definitely don't think there is enough space there.
But I wasn't the one directing
The other thing is, like, if you end up filming this part on your own you don't necessarily have to have a pan down.
You could just do a push down transition.
That's where I feel like a transition would be OK, because you've got the camera tilt up previously.
If you do this again, I would just look up instead of going like this.
Because it's a little bit awkward to go like that.
[VIDEO PLAYBACK] - --natural systems previously thought off limits to mathematicians can now be explained in terms of fractals.
[END PLAYBACK]
I wouldn't move off of this scene so quickly.
Like, give it one more beat before you transition.
[VIDEO PLAYBACK] - --and can be used to study river systems, blood vessels, and lightening bolts.
So many natural systems previously thought off lim-- tree it is relatively easy to program, and can be used to study river systems, blood vessels, and lightening bolts.
[END PLAYBACK] STUDENT 1: And I would re-shoot this completely.
[VIDEO PLAYBACK] -So many natural systems previously thought off limits to mathematicians can now be explained in terms of fractals.
[END PLAYBACK]
Yeah, and right here I would just cut to, immediately, you talking.
Because there's, like, a little bit too long of a pause before you start.
And bump up the-- STUDENT 1: Yeah, I'll do a voice over for this one, because there's a lot of wind.
[VIDEO PLAYBACK] - --learn nature's best practices, and then apply to solve real world problems.
Much like Cohen's antenna revolutionized the field of telecommunications, other fractal research is changing medicine, weather prediction, and building design-- here at MIT and everywhere in the world.
Look around you-- [END PLAYBACK]
When you say, "look around you," I would actually cut back to a close up of you.
Look around you-- [VIDEO PLAYBACK] -What beautiful patterns do you see?
[END PLAYBACK]
And just end it a little bit sooner.
It's a good start-- really good start.
STUDENT 2: A good rule of thumb-- we never mentioned this in the editing lecture, but, like, you want to give the end of a scene a little room to breathe so people can process, like, what's going on.
And then at the beginning of a scene, you want it to be, like, one or two frames before you start talking.
So it's, like, almost immediately your voice
Yeah, that's actually a very good point.
And I should have brought that up during the editing lecture.
But I feel like the tendency is to do the opposite with what you've done with the scenes here, where you end too soon, and then you wait too long before you start the next scene.
STUDENT 2: It's like-- it's counter intuitive, but it makes it flow better
But yeah, you're in really good shape in terms of, like, pacing what you need to do before next week.
So I think, at this point, it's a matter of getting the animations in and the music.
STUDENT 1: OK, think you.
STUDENT 2: You're definitely, like, well on your way, on the right track to be done
This chair is bad news bears.
I lean back to stretch, and I'm still leaning.
I'm still leaning.
STUDENT 2: This entire room is kind of--
It's a little haphazard.
How are you guys doing?
STUDENT 1: Thank you
Yeah, good luck.
You're still cutting and pasting the script?
STUDENT 3: No--
Oh, your scenes together.
You guys finished filming everything, right?
STUDENT 4: Yeah
This is nice framing.
STUDENT 3: Thank you See, you should--
Now, if you did it again, one cool thing that you could do-- and I'm not saying that you need to, because this is fine.
But if you had moved him to the center of that picture, and you moved close enough, then this image could basically fill the entire background and you wouldn't have this weird gap, right?
STUDENT 3: Oh,
I don't know if the image was big enough to do that, compared to the size of his body.
STUDENT 3: The image was big enough.
I didn't want to do that because I didn't want him to look like he was in green screen
That's a good point, actually.
That is a good point.
STUDENT 3: That's the reason why.
I have a bit OCD, so I can't stand what's on the right, actually.
But I'm fine
It's OK.
It's OK. STUDENT 2: Yeah it kind of-- it's blurry enough that's it's not distracting, because I'm not trying to read the words.
Because I don't think they're, like, readable
Yeah, I wish I could have given you guys lights, because that's what happens when you rely on the fluorescent.
You get some weird shadows on your face from overhead.
But it looks nice.
Do you guys have any specific questions right now, or should we just let you work?
STUDENT 3: I'm planning to do animations all night, so--
I didn't see you guys film anything.
I guess I saw you guys film that stuff.
STUDENT 3: Like, I plan to put animation here
Yeah, that's definitely-- it's good.
That's a good surface.
As long as you keep the pictures small enough.
STUDENT 2: And you can adjust the size, like, even after you've done it After Effects-- and you've exported it or transferred over to Premiere-- you can change the size of it.
[VIDEO PLAYBACK] -How much faster would that be?
Did you guys use the VIXIA for that one?
Did you use the VIXIA in the hotel?
STUDENT 3: What is the VIXIA?
The Canon camera.
STUDENT 3: Yeah
That's like, nicely lit.
Surprisingly nicely lit.
So it's like a lot of natural light, then.
Yeah, it looks really nice.
STUDENT 2: I think the white bed sheets help, because they're like a natural reflector
Yeah.
STUDENT 3: So they're all natural light.
And there was a lot of stuff here.
That we threw away.
I was like, oh, I can't stand it.
It's too much
And the other thing is, you might want to bring the exposure down just a tiny bit, because the whites here are really peaking pretty high.
But everything from here is so nicely lit that I'm almost inclined to just, like, forget it.
Just keep it the way it is.
STUDENT 4: How do you do the exposure on the--
In the editing software there should be a way to just, like, bring down the exposure.
What are you using for yours?
STUDENT 3: I think the simplest way would be to tap the bright area, and then it would be-- STUDENT 4: You mean on the video camera itself?
STUDENT 3: Like, on the video itself-- on the touch screen-- you just tap your shirt.
Then you will have dim down
Nice job, guys.
STUDENT 2: You just kind of disappeared for a couple days, but you came back with really good footage
I'll let-- you guys can just work.
We'll stay out of the way.
STUDENT 2: If you have any questions, then-- and I'll be able to answer, like, any of the editing questions you have in Premiere
This one, I would zoom in on you a little bit more.
Because you're a little low.
So I would crop in like that, to where the top of the frame is just above your head.
STUDENT 2: Usually do the rule of thirds with, like, eyes rather than the top of head.
And I think.
And if you look at your eyes.
They're kind of in the middle so you want to adjust it so they're equal.
Yeah, like that
We'll let you guys work.
STUDENT 2: If that's all let me [INAUDIBLE] for today, actually.
STUDENT 3: What time is it?
STUDENT 2: It's 10 till 3:00.
STUDENT 3: OK. STUDENT 2: OK, so now what-- the problem is, illustrator defaults to a brush.
Which we don't want, because that will mess up when we redraw it in After Effects.
So we select the arrow, we want to change Brush Definition to basic.
Go into Stroke-- let's see, why are there not more options?
There we go.
Change the caps and corners to round, so it's, like, nice and smooth.
Like, it looks like a brush.
Basically, we're making it look like a brush, while doing a basic shape.
And then change it to, like, whatever weight you want.
So you want it about that.
And you can play around.
Like, do you want it super thick, or do you want it not so thick?
And then the last thing you need to do before going into After Effects, is go into this layer-- and so, see, it has the different parts of the arrow.
You want to highlight everything, click this drop down menu, and click Reverse Order.
Because it stacks it from the last thing you drew to the first thing you drew, and you want to be first thing you drew to last thing you drew.
Then you want to save this.
We'll just save it in Documents for now.
OK, and then you open up After Effects
That's a nice pan, Joshua.
This is nice camera work.
STUDENT 3: Oh yeah, I held the tripod to the right.
I looked really stupid, though.
STUDENT 2: But it's really stead.
STUDENT 3: I didn't actually hold it like that.
I knew that it would be bad.
STUDENT 2: If After Effects would open, and not give us the color wheel of death, then I can show you.
I can get it.
I have my computer.
It's, like, open.
It's about half an hour of video.
OK, After Effects-- hurray!
OK, so we're going to import our composition as a composition.
All right, double click it.
I'm going to set it to a white background, just so we can see things.
And depending on-- if you do it on the black portion, you might want to set the background to black and have the arrow be in white.
Like, depending on how you want it to look.
OK, so this is the fun part.
So we're going to turn the arrow layer into shapes.
And then what we're going to do is, we're going to a add a Trim Paths.
And basically what that means is, it's going to draw the line for you.
And then under Trim Paths we're going to trim multiple shapes individually.
Because that means-- so, if you do it simultaneously it'll draw the arrow-- like, both lines at the same time.
So it'll start drawing the long arrow and, like, the arrow head at the same time.
And we don't want that.
So change it individually.
And then wherever you want it to start you turn the end down to zero.
Set your key frame.
And, say, we want it to draw in two seconds.
You turn the end to 100.
And you go back to beginning and you press space.
And it will draw the arrow is as much time.
Is that like what you wanted to do?
STUDENT 3: Yeah, I wanted to do arrows.
STUDENT 2: Yeah, and then you can change it.
Like there's some lines-- like, I like writing out handwriting a lot in animations in my videos.
And so, like, for example, if you want it to pause on the store, you can set it-- OK, so it's like a 35%.
So you can set a key frame there at 35%.
And then you can set another one later on, also at 35%
And then what it does is it, like, pauses at the door, holds it there, and then keeps going in remaining time.
STUDENT 2: OK, thank you
Yeah, I don't know if you can remember all that.
But if you forget the rest of it, write me and I'll redo it.
STUDENT 3: OK, thanks.
STUDENT 2: Yeah, let me know if there's anything else.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today is the first screening of your rough cuts.
And the way this will work is I will show one of the videos, and we'll all watch it.
You guys can take notes on it while you watch.
And after each screening we will give general high-level feedback to the creator.
And that's what-- we'll just go through the whole class and do that for the first half.
Now for the second half, there are a couple options that we could proceed.
The first being what I usually do with Science Out Loud, which is to basically do Google Docs for detailed comments.
And this is really useful because it's sort of like a check box when you're going back through and looking at feedback from another person when you edit.
So this was for George's farts video.
And I made some notes about where we should add some animations, where we should add overlays.
So this is one option that we could do.
Basically break you guys up to where you will end up commenting on at least two people's videos, and we would know who you would be assigned to.
The other option is that we could just go ahead and pair you guys up or group you guys up, and instead of doing the Google Docs feedback, you can just talk to each other one on one.
Do you people have preferences on how we do this?
Google Docs
Google Docs.
Anyone else?
All right.
So we'll do Google Docs.
And I will give you the details on how that's going to happen during the second half of class.
Sounds good?
So, as it was for the table reads-- remember that repetitive comments are OK.
It's OK if you feel the same sort of feedback that another person has already said because it'll draw attention to the creator and what they need to work on.
Also remember that at the end of the day this is your video, so the feedback is for you to take or leave-- hey, Nathan.
And this is probably going to be the most valuable class time the whole month because this is where you're really going to have concrete advice on implementing the stuff that we've talked about so far.
So really take this as an opportunity to give as constructive and meaningful feedback as possible.
All right?
Does anyone have questions or comments before we start watching?
No?
Are you excited?
We need popcorn
I thought about making pop-- but then I thought that if people were taking notes on their computers and they had, like, butter on their fingers.
We'll pop for another screening
--later for a snack
Yeah
That's fine.
Maybe we should close the big--
Yeah, I'll close the door.
Does anyone want to go first?
No?
Does that show up on screen OK if I turn these lights off?
OK. OK, well, then we will just go down the line.
How about that?
So lucky you, Nathan.
Do you want to say anything before we screen, or do you want to just go ahead and watch and then talk about your process?
I'm going to be honest, and I made a pretty bad mistake in putting off filming until yesterday.
And so I only got to get in the editing this morning.
And it really shows
Well, let's see what you have.
And there is time, that's what tomorrow's for
I couldn't get the animations to work
OK. And do keep in mind that all day tomorrow and all day Thursday is open for you to incorporate feedback.
After today's class ends, we're not meeting again until the screening on Thursday at 6:30.
So as long as we get videos in by, like-- 6:00 PM's the latest.
Hopefully you're not working up until 6:00 PM.
You have the rest of the day to work
But if you need our help, then e-mail us.
And we can meet up if you want
Yes.
We will be in the classroom tomorrow and Thursday if people need help.
[VIDEO PLAYBACK] -Why does your-- why does your fridge start to smell?
[END PLAYBACK]
The sound sounds good.
[VIDEO PLAYBACK] -Why does your fridge start to smell?
And where does that icky black liquid come from?
Wouldn't life just be easier if things didn't rot, if things lasted forever?
Well, maybe, but probably not.
According to the USDA, the average American household wastes 25% of its food.
And a lot of restaurants are even worse.
So if we didn't have decomposition, what would happen to the food we throw away?
Food in landfills normally gets dealt with by bacteria, fungi, and protists that allow the nutrients in food to return to the soil and, eventually, other living things.
These decomposers also break down other dead stuff, like trees.
In fact, they're pretty much the only thing that can eat wood.
So in a world without rot, while your home may not have to worry about being eaten by termites, who rely on protists in their stomach, pretty soon forests and landfills would be flooded with a lot of dead stuff.
So how do we avoid this problem?
Well, basically, in your fridge, on a forest floor, in a dumpster, almost anywhere, there are fungi, bacteria, and protists that live entirely by eating dead stuff.
These dead things can be more or less divided into three categories.
Carbohydrates-- sugar and starches.
Lipids-- think fats.
And proteins, like meats.
All of these are chemically different, so they each get broken down by different enzymes in different ways before being absorbed by decomposers.
For example, proteases break down proteins into amino acids, the cells building blocks.
Lipids rely on lipases.
And carbohydrates on things like amylase and cellulases.
So how does a perfectly nice broccoli floret start giving off this foul black liquid?
Fruits and vegetables are almost entirely made of water.
So on the most basic level, you could say their cells are like extremely complex water balloons.
The exterior-- a cell wall-- is made of cellulose, a complex carbohydrate that gets broken down by enzymes into small sugars the cell can get energy from.
When bacteria or fungi uses cellulase to eat the exterior of a cell, it's like if I were to pop the balloon.
That's the muck you see in your fridge.
And that's how you get that icky black liquid.
What about that stink?
For fruits and vegetables a lot of time the smell happens after the icky black liquid forms.
Other bacteria that weren't involved in the initial colonization move in and start to stink everything up.
Meat gets smelly when lipases break down fat in the meat into glycerol and fatty acids-- two energy sources.
And fatty acids are kind of gross.
And so while your food rotting may smell absolutely terrible, because of it the environment's able to recycle crucial nutrients it needs.
And, well, that's why we have all this.
[END PLAYBACK]
Are you kidding?
That was awesome
All right.
General feedback?
Well, I want to hear what you think before we give you feedback
Like what are the things that you like about what you did, and what are the things that you already know you want to change or add?
The audio
The audio
The lack of animation.
The main things
What was the hardest part about making this video?
Actually going and doing the shooting.
I kind of put that off as long as I really feasibly could have because I really didn't want to do it
Was it as painful as you thought it would be once you started?
Yeah
Yeah.
Well, I will say that the script is very, very tight in this video.
And it has come a very long way from your initial brainstorming.
And if you come to the screening, you'll get to actually see all the pictures.
So, just a plug to come to the screening if you want to.
But the script is super tight.
And I know that you spent a long time honing that, and I think that really paid off.
It's two thirty-three, which is a great time.
So I think that as far as the content goes it's really strong, and I can tell that you spent a lot of time developing that.
Does anyone want to add some more feedback?
I mean, I can talk to-- yes, Yuliya
I like the transitions.
There weren't too many of them, but it was nice to see a change of background and scene
Yeah.
And clean cuts.
No funky swirls, no fades.
It's nice
Yeah, definitely.
I was going to say something else, but I forgot
That's OK. We can come back if you remember.
Anyone else?
Yes, Andrea.
Sorry
I really like the pacing of it.
It's a little rushed in a couple of places, but that's-- that happens.
But I like the pacing in general.
And I also like the water balloon example and the different camera angles, and I thought it was just really artistically very well done, very well composed
What was so terrible about actually shooting?
Was it being on camera?
Yeah
Yeah
And also, each one took a lot of takes
Yeah.
I will say-- because I know that a lot of you expressed sort of concern and hesitation about being on camera, and I feel like a lot of this class has been like actually no, you're really great on camera.
The opening shot is delivered really, really well.
And there are spots where, like Andrea said, you're rushing your words a little bit.
And I'm sure you'll see it when you go back through.
And whenever you guys do your Google Doc comments, point out spaces where maybe his delivery changes.
It is a little inconsistent between scenes, but there are some scenes where it is delivered so perfectly and so clearly, and you seem really, really comfortable on screen.
Yeah
I will say-- did you shoot it by yourself?
I did shoot it mostly by myself, since I put it off
Because I found that it was a lot easier to have someone shoot it than, like, when we were doing those first blogs and stuff at the beginning, I thought that was a lot harder doing it by myself than having someone there.
I felt less comfortable being alone than with someone.
Which is kind of-- I thought it'd be the other way around when I first started the project.
Other than that, like, how I think he used-- went to Star Market.
So representing a local grocer too
Yeah, yeah.
I agree.
I like the part where he talked about the smell.
You know?
Yeah
And I like the cut on the fridge
The visual, yeah
What do you guys think about the visuals in general?
Because it's always hard to do a chemistry or biology video because the visualizations are always somewhat impossible to do.
Yeah, Yuliya
I think it would be nice to show some of the scientific vocabulary, so we can see it and not just hear it
Yeah, like a label on top or something
And then for the backgrounds, like, the plants backgrounds worked really well and, like, the fridge and all that.
But there was a part where there was just a random penguin or random pictures on the wall that maybe could be overlaid with animation.
[VIDEO PLAYBACK] [END PLAYBACK]
Like this shot.
I like that you have the kitchen space as sort of your home base.
Because your centered on the question of why things rot in your fridge, and that's what allows you to explore decomposition at greater length.
But this is doing that whole thing that a lot of web videos do, where you center the subject.
And I was talking to Chris [INAUDIBLE] about why it's so common that SciShow does it and Vsauce does it.
And it sort of gives off an air of it being not as formal.
But it's also a little bit comical, too.
And I think that if you just-- even if you zoomed in just a little bit and got rid of, like, this spray thing on the side and the cooking things on that side, it might help with this one.
Also remember to tuck your mic in under your shirt if you can.
Let's see.
But you were talking about this penguin here, right?
Yeah, so this-- I mean, you could play around with keeping this footage, if you like, and just zoom in a little bit.
See if you can digitally zoom in on the editing software.
If the resolution kind of goes kaput, you may have to re-film that part.
But again, I think Yuliya is right in that be careful about having visual distractors when you don't really need them there.
I do agree with-- was it Andrea?
Who said something about the pacing?
Yeah, I think the pacing of your cuts are pretty good.
Like, nothing lasts a little too long, nothing's too short.
If you have B-roll, it might give you a little bit of breathing room in between shots where you're talking really fast and you have a lot of content.
But yeah, like stuff like this-- I don't remember what you're talking about here.
[VIDEO PLAYBACK] - --while your home may not have to worry about being eaten by-- [END PLAYBACK]
Yeah, so--
Comical note on that, I filmed in front of someone's house, and they actually came out and asked me to stop.
And so I kind of--
Oh, no
--gave up on actually filming at home
Filming in front of a house.
I mean, is this in your dorm room?
Yeah

That's what my backup idea was, since it's my home
Yeah, actually, I think that it works fine.
And I would say for this one, do a wider shot for this one because it doesn't quite establish that you're in your own place.
Like, if you can see you sitting on your actual bed, that might help a little bit more.
So again, it's not a matter of using wide or close-up shots for the sake of using them on principle, but like what the background actually reveals and sets for the audience
I was wondering, can you go back a little bit?
Because there was-- I was wondering what your vision was for this
Right there.
Yes.
I'm still missing stock footage

Oh, so this is like a placeholder for the rough cut.
I figured that was the case
--what you were planning to do here.
I was like, I'm not sure what I'm looking at
Yeah.
Oh, the dumpster shot.
Yeah, I wanted to say something about this, too.
If this is something that you decide to re-shoot, I would also maybe scoot back a little bit as well.
Because it's hard to tell that it's really a dumpster until you actually say it.
Where was this?
It's down at the loading docks
Oh,
I had a lot of trouble with the dumpsters I'd planned on going to.
By Walker is a bit of a wind tunnel.
And the Stata Center, out behind W20 was really noisy
Which one's W20?
The student center
Oh, OK.
I mean, there is an optimal dumps-- if you guys are ever in need of an optimal dumpster to fill up, let me tell you.
Right behind the MIT Museum-- by D-Lab actually-- there are some prime dumpster spots for shooting.
And that's where we shot George's farts one, too.
So if you need to.
But in this one, I mean, I think it's really matter of the camera angle.
Did you shoot this on a tripod?
Yeah
OK.
Were you limited by how far you could let your mic cable go?
Yeah
OK. Yeah, maybe think about this one as a potential one to re-shoot.
Because I think the visual background makes a lot of sense and is a really good transition-- it's a really good change from the kitchen setting.
So I would really love to see that it's actually a dumpster
And if you are refilming-- only if you are refilming-- something that I noticed is that when you're talking you bounce a little.
So just-- like in some of the film when your up close we see that.
And it's just maybe something that if you can be aware of that while you're filming.
It was just one moment, I can't remember exactly where it was.
Where is it?
When you were talking, a little towards the end.
And I noticed it a little more, this sort of gentle bounce while you talk.
That you don't do all the time.
[VIDEO PLAYBACK] [END PLAYBACK]
Maybe we can find the--
I'll look at it more carefully, because there was a point where I was like--
I think it was the end of [INAUDIBLE]
Do you know what I'm talking about?
Maybe we can note that in the Google Docs, whoever's commenting on Nathan's.
And again, stuff like this, I think it's great, but maybe just like crop this framing a little bit more so it's a little more rule of thirds.
Because the rule of thirds works really well for this shot.
And [INAUDIBLE] pointed out a really good tip.
When you do rule of thirds, use the eyes as sort of the point instead of, like, your face.
Because there's just a little too much room on the top and right-hand side of this framing.
How did you film the opening?
Like, were you by yourself?
Was it the first thing that you filmed?
I was by myself, and that was the-- no, it wasn't.
It was like second place I went
OK. Because whatever conditions you were filming under for this scene, I think you should really try to remember what it was that you did.
Because this opening-- [VIDEO PLAYBACK] - --start to smell?
And where does that icky black liquid come from?
Wouldn't life just be easier if things didn't rot?
[END PLAYBACK]
Like that question is posed actually quite naturally.
I think the tendency for a lot of people when they're starting would be to sort of overly cheese that question up.
And that's sort of what happened in early readings that a lot of you guys did.
But this one just sounds very natural, like how you would actually pose that question in real life.
So whatever conditions that you were under filming this one, really try to apply that to whatever you re-shoot.
Were you there with him for this one by chance?
Hmm.
Maybe it's an energy level thing.
Like--
I think it was I was inside
You were inside, so you weren't as nervous about filming outside.
Yeah, actually, that does a lot to people's deliveries.
But yeah, just something to keep in mind.
Does anyone have final thoughts before we move on to the next video?
Do you know who we had paired up to comment on Nathan's?
Andrea and Kenneth
OK.
So basically, for the rough cut thing, we will pair you up with two people.
One of those people is going to be the person who helped film your video, and the other person is just someone completely new.
So Kenneth and Andrea are commenting on Nathan's.
Andrea, do you want to say anything about yours before we screen?
There are lots of placeholders for stuff, like similar.
It's really boring
We will be the judge of that.
All right.
This is Andrea's.
[VIDEO PLAYBACK] -Eating is one of the great pleasures and necessities of life.
And to enjoy everything from energy bars to apples, we rely on one part of our bodies to do an important job-- our teeth.
Teeth are the hardest substances in our bodies.
They're harder than our bones, and they're even harder than iron or steel.
While we chew our teeth experience forces of up to 225 pounds.
Hmm.
So why doesn't our jaw just crumble under all of those forces?
Between your tooth and your jawbone, there is a specialized piece of tissue called the periodontal ligament, or PDL for short.
The PDL can easily absorb the normal forces that a tooth experiences when we chew, say, an apple, cushioning or protecting our jawbone from our teeth.
And inside the PDL there are all kinds of cells.
One type, called mechanoreceptors, sense forces of movement or pressure applied to the tooth.
Teeth sound like they're already perfectly designed.
But sometimes we really need to force them in a certain direction.
Like with braces.
As the braces slowly force the teeth to move, the PDL gets squeezed in one direction and stretched in the other, kind of like a rubber band.
Here's where it gets interesting.
To make room, the mechanoreceptors in the PDL trigger cells called osteoclasts that actually come in and dissolve part of your jaw to make extra room.
The mechanoreceptors also trigger another kind of cell, called an osteoblast, which comes in and builds up part of the jawbone.
This allows the PDL to get back into its regular cushioning shape, thus holding the tooth securely in position.
So if braces use osteoblasts to physically reposition teeth for cosmetic reasons, what if we want to use them to replace things in our bodies?
Dental implants replace teeth that are damaged or missing to restore chewing function.
And MIT engineers are using the properties of osteoblasts and osteoclasts that are already in our bodies to create a chemical coding for these implants.
Just like in a mouth with braces, this coding helps create natural bone to help lock the implant into place.
Your jaw isn't the only place where these osteoblasts and osteoclasts are altering your bone structure.
In fact, this bony remodeling process is happening throughout your entire body.
And these implants aren't just limited to teeth.
Doctors can replace knees, hips, and even spinal disks.
Right now, these implants are designed to have the exact same functionality as the parts that they're replacing.
But in the future, scientists may even use implants to improve our brains.
Just like with braces, we could modify our bodies to be more perfect.
Once we've tasted the forbidden fruit of perfection, will we still be human?
[END PLAYBACK]
All right.
So, Andrea, why do you think your video is boring?
It's really dense with information.
It also doesn't have very good settings in it, except for the office.
And even in the office, the lighting-- there was just nothing I could do about the lighting
I think that one of the quickest fixes that you can do, and one that'll make the biggest difference, is that it's very slow moving.
And I think it's because you are so conscious of fighting the tendency that most people have to deliver their lines too quickly that you're talking a lot slower than you usually do in real life, in a way that is clearly not you.
And you could actually just play with speeding up how fast the video plays.
We did that for one of the people who did Science Out Loud, actually.
The same thing happened to her.
As soon as the camera started rolling, she would talk just way slower than she normally does, and it just felt really unnaturally paced.
And once we bumped up the speed to, like, 115%, it read a lot more normally and flowed a lot better.
And I think in general, look at where you're cutting into new spaces.
I think that you erred on the right side for your rough cut, which is to leave a little bit too much room at the beginning of each of your clips.
But when you go in, really, you can shave off like a good second, half a second off of some of the clips that you're cutting, too.
Instead of having a beat and then your action starting, just go straight into the action already happening.
So I think that happen a couple times.
[SKIPPING THROUGH VIDEO] For this opening shot I mean, just start it like half a beat before you say, eating is one of life's great necessities.
I think you take these nice long pauses in between some of your thoughts to really let the viewer gather or comprehend what you've said.
But some of the gaps are just really long.
So you can either cut those out and maybe cut to a close up of you talking, or cut to B-roll in those cases.
But I think that'll help a lot with just the sense of the movement of the overall video.
Did anyone have other comments?
Yeah, Kenneth
I remember in there you were a little scared to start filming.
But I thought you felt pretty natural in front of us, and it didn't feel you were nervous or anything.
Apart from the slow speed, I think actually you look quite natural
I was going to say that too
Yeah, David?
I was thinking that one way to cut that down is when you look at a clip or you look at a song, so you cut right before the song.
And then it's like a straight line
Wait, sorry.
I didn't hear that
Plus, when you put the video as well as the sound
The sound?
The sound wave
Oh, yeah.
The sound wave.
Yeah, the sound wave.
That's a very good tip
Then I was thinking that I like the transitions.
I find the transition very meaningful, and they help you continue the story
What were you having planned on this big section here?
Was it supposed to be an animated section?
I've been looking for images
This is when we chew, say, an apple
I think one part that will help that felt really slow-- but maybe you put in there for a visual pause-- is the middle section when you were talking about putting on braces, and you did the whole sequence of you like putting on gloves like an orthodontist would.
And it's good for comedic effect, but it kind of interrupted the video for me.
And I think this is an instance of killing your darlings, where I don't know how long it took to film this
So should cut out everything in the office?
No, I like-- like when we all laughed, that was a great reveal.
And I think you can almost cut from this shot to the one with you with braces-- like when you say, with braces.
Because you don't even say anything over this part.
It was just kind of silent
I think she could keep it if she tightened it to like one second.
We can get enough out of you just putting on-- starting to put on the glove and then transitioning to the next one.
I think it's fun.
I think that I agree with you that it's too long, but I think you could tighten it, really, and it would still get the impact
Comedy is really hard to pull off, like George was saying in the earlier workshop.
But a lot of comedy that works in these types of videos, and that you'll see in SciShow or Veritasium or pretty much any popular YouTube channel, is that they're very quick.
They're quick insertions of someone making a face or sort of an unexpected B-roll shot.
And I think that I agree with Jaime.
And you can play around with this.
Play around with taking this part out completely, play around with shortening it.
But it has a funnier effect when it's a lot shorter, because one, it doesn't interrupt the flow of the video as much.
And also just gives a chance for the viewer to be like, wait, what just actually happened?
It keeps them on their toes a little bit more
For that, I really like that part.
The only thing I would suggest is that at the end when you reveal the braces to kind of show them for half a second longer so the viewers can notice that it's braces
Yeah, I think the braces shot was actually a little too short
And like right here--
How did you do that?
The braces shot?
Did they just put pretend braces on you for a minute?
It was like a denture
Oh,
And this is an example of shaving off some of the beginnings of a clip.
That is exactly where it starts, it's a good half a second.
Which doesn't seem like much, but when you're watching a video it seems like an eternity-- half second that you can just take out completely
So how close to when the sound starts?
Should you do it right there?
I don't have a best practice of do it like 0.2 seconds before it starts.
But I think that, in general, having it as close to when you start talking is best.
Because otherwise, it reads a little bit newscastery.
Because you'll notice that on a Dateline special, the host will go, and I'll start talking.
And that's what gives it the newscastery feel, versus a lot of YouTube channels and SciShow, they just go immediately as close to the soundwave starting as possible
Bring us two frames back
Two frames back?
I usually go like two and three frames back.
And then if that doesn't sound good, then adjust it from there
And who's doing Google Docs for Andrea?
Nathan and Yuliya
Nathan and Yuliya.
Any last comments for Andrea?
I do want to emphasize-- and for the people who are just here for today-- it is a crazy amount of work that they're doing in such a small amount of time.
We basically spent the first week not really touching cameras whatsoever.
I mean, they were doing assignments them, but pre-production takes up so much time.
So I think that the scripts that you guys have are really, really tight and really good.
And I know that you're working under limited time and very, very limited resources.
Like you were saying, you didn't have lights.
And that makes a huge difference in what your final product looks like, so I'm really happy to see that people can identify-- this is what I think it's off about this video, I think this is what it would have improved.
That's the stuff that really is important.
And I think the scripts, and you guys really constructing compelling narratives around the topics that you were talking about, they've come such a long way from the first day.
So I'm really happy to see that
It's ready, but I think you have to refresh the page
Oh, OK.
This one?
All right.
Any words before we screen?
No disclaimers
Yeah, it's good.
It's fine, I hope.
I just didn't like how I cut it, but I think it still is quite well.
I haven't finished the animations, but yeah, some of it's in.
[VIDEO PLAYBACK] -Imagine holding a party, and then sending out invitations after the party.
Wait a minute-- that doesn't make any sense.
Are my friends are going to be able to time travel back just so it's at my party?
In 2009, Stephen Hawking actually conducted such an experiment to disprove time travel.
What exactly is time travel, and how does it work?
It actually has a lot to do with speed.
A lot of what we know about time traveling actually comes from Einstein's theory of special and general relativity.
We are actually time traveling right now, but not in the manner that sci-fi phi films have depicted them.
We are, in fact, time traveling at a pace of one hour per hour.
In other words, every hour I experience, the world around me experiences a singular hour too.
Now, it may sound simple and trivial, but stay with me.
This is where it gets interesting.
Now, the lat part of Einstein's theory of special relativity is actually a phenomenon known as time dilation.
Effectively, this means that the faster you travel, the slower time passes around you.
It potentially means that I could travel at maybe more than one hour per hour.
This phenomenon, however, is only noticeable at really, really high speeds-- speeds near the speed of light.
This rocket here, not even close.
Say I do a spaceship now that travels at 90% the speed of light.
And I have a pair newborn twins.
I bring one of them with me on this journey for 10 years.
When I return, one of them will have turned 23 years old while the one with me will have only turned 10 years old.
While we have experienced 10 years on a spaceship, 23 years have actually passed on Earth.
This amount of time actually increases exponentially as you get closer to the speed of light.
So we could potentially travel forward in time.
But what about backwards in time?
That's a whole other issue.
Now, remember our previous graph.
The time that passed on Earth was asymptopic to the speed of light.
Looks like we're seeing maybe the secret to time travel lies on the other side of the line.
In other words, we may have to travel faster than the speed of light just to be able to travel back in time.
Now, when you do travel near the speed of light, something called relativistic mass comes into play.
In other words, as your speed goes up, your mass increases.
As your mass increases, you require more energy to move the same object.
This might mean that you need an infinite amount of energy just to move an object beyond the speed of light.
The more energy you put into an object, the more likely you are to increase its mass rather than to increase its speed.
Therefore, to get an object past the speed of light, quite impossible at this point in time.
So why are we still so obsessed with time travel?
Time travel does come with its own set of problems.
Take for example my party before.
If I were to send out my invitations after the party, my friends would have come back in time to attend it.
And if they did come back in time to attend it, I wouldn't have sent out my invitations.
And if I didn't send out my invitations, they wouldn't be at my party.
It doesn't make any sense at all that they are both at my party and not at my party.
This is what we call the grandfather paradox.
It's one of three time travel problems that we currently have.
So it seems that we may not be able to time travel yet, based on what physics tells us, at least.
For now, if you are planning a party, be sure to send out the invites.
Your friends can't time travel yet.
[END PLAYBACK] [APPLAUSE]
All right, any comments from Kenneth first?
OK, I didn't know that the warp stabilizer on YouTube was like that
Oh, you tried the stabilizer on YouTube
Because I new I had to stabilize some of my footage, and they were taking a very long time on Premier.
So I just popped it on to YouTube and just let YouTube stabilized it.
But I didn't know that it did it very liberally.
So I realize a lot of my shots got like this warpy effect, and some of my text got cut out, and what not.
But yeah, and I'm sorry that the audio for this whole chunk just kind of got warped because I had to use my other camera, which I didn't want to
Oh, you used your DSLR for the middle part, right?
I mean it's a trade-off, because it looks way better than the camcorders that we have.
But the camera that he has doesn't have an input for the mic.
Yeah, it's really hard to understand that part.
Yeah, PJ?
With these scenes, though, we were talking about doing this walking down.
It was just tough to have a ten foot tether to do some of these.
And I thought that came out really good-- that video part
Could you try a voiceover?
Yeah, I was thinking about putting that in
Because it's far away enough where I don't think it matters
Yeah, you could try lip dubbing basically.
I have one pair of wireless mics, if you do want to try shooting it.
I mean, it may be a little too much to throw in at the last minute, because I do agree that this part is-- [SKIPPING THROUGH VIDEO] It's hard, but I mean, I also think that having that wide of an angle is important in that shot.
So I understand using the mic was sort of not really an option there.
But I do think that voiceover is a good option.
And especially this part.
It's a long walk and talk, and--
--example, my party before.
If I were to send out my message--
And the audio is so distractingly unclear, and this is such a crucial sort of inherently hard to understand portion anyway, that I think that it's really important to have clear audio.
And I don't know if you can just shoot B-roll to do voiceover for this.
Because it's a little-- the setting and what you're saying don't necessarily match up to me right away.
But I also like the feel that you were going for with this walk and talk.
I can tell that you were drawing a lot of Veritasium influences with the beginning, and you did the whole Michael Stevens Vsauce head up, which I think are fun.
Because you're taking something that is typically a very robust science topic and sort of presenting it in a very casual, like, I'm just your friend talking about time travel type of way that makes it a lot less intimidating.
Which I think is really nice.
It's a nice feel for the video.
Any comments from anyone else?
Yeah
It's sounded really well in its complete form.
Even though-- like the script, there was a lot of technical detail.
It didn't read like that at all.
The only part here was that when you were walking and describing the grandfather paradox, I know we also shot the party
Yeah, I was thinking of putting that in split screen, but I--
Oh, you actually shot a fake party?
Yeah
I think that it would be a really good option to try playing around with that footage and just using voiceover there
I only put it at half a screen, because I'm keeping to the left of the screen so I have it on the right side
Yeah.
Is it super important that you maintain this footage of you talking?
Or can you just cut to straight B-roll of the party?
I thought it would be a nice transitions to walk into the last scene
Oh, yeah.
I mean, I can tell that you were very conscious of how your cuts were being made
--the grandfather paradox--
I think that this opening one and some of your pans are really slow.
So you could cut the clip right before it lands on you and just speed up the pan.
This part especially.
It just feels very nice
--sci-fi films have depicted them.
We are, in fact, time traveling at a pace of one hour per hour
The script is a lot better here.
It's really good
--speeds near the--
And this too is a really good visual reference there
I really liked whatever you did to animate the graph.
I was like, mind blown.
I don't know if I could do that.
But it looked really nice because it followed your finger and everything
Yeah, how did you do that?
I kept a screenshot there, and I imported for the shot that I drew everything over.
This amount of time actually increases--
But YouTube cut it off right at the bottom.
OK, that's what I figured what happened
Yeah, it was supposed to be nice frame.
I don't know why they even warped this
Yeah, I mean, take the stabilizer out and just do a closeup shot of you.
Because that's really nice.
Are you going to do it for this part too?
I was thinking that I should
I think you should, just because like you're throwing in the graph and you have the asymptote, and it's just such a key part.
Something that I didn't know until you read the script out loud to us-- really cool, the idea of looking at the other side of the graph.
So, I know it's more work for you, but I think it would look cool.
Who's commenting on Kennith's?
Yuliya and Andrea
Yuliya and Andrea, great
I really like the ending
Yeah, the beginning and ending, you book ended it very nicely.
Very good.
I'm very happy with how people have taken their scripts and really created a point and a story around it.
I don't think any of these drafts are instructional videos whatsoever.
You've taken all these concepts-- and we talked a lot at the beginning about tying specific examples and allowing a specific instance to let you hit these bigger points.
And I think everyone's really done a nice job of doing that with their scripts.
All right, next up we have Joshua.
You ready?
Yeah
Any comments?
I haven't put in the animations yet, so there will be a part where there's a blank space.
But the blank table is supposed to be the animations.
[VIDEO PLAYBACK] -Hi, have you found it particularly difficult to find a specific item in your house?
Let's say you're looking for a pair of gloves, but you just can't find it and you spend your entire afternoon looking for it.
Well, you've just encountered the same problem that companies like Google or Microsoft encounter every single day.
And that's the problem of search.
Just like a house, which stores thousands of different items, Google stores 45 billion index pages of information.
When if every page was a sheet of paper and we stick them up real high, we will create a tower 600 times taller than Mount Everest.
Well can Google find my results so quickly when I find so difficult to find a pair of gloves?
Well, searching on Google is kind of looking for a person in a big school.
Let's say you're looking for James in the row of classrooms.
One of the easiest method would be to go to every classroom nearest to you until you find James.
There's a better method known as binary search.
Let's say the students were arranged from A to Z in an increasing number of the classrooms.
And let's say we head to the middle room first, and if the person in the middle room isn't James, but his name starts with a letter before j, we head to the right.
If not, we head to the left.
We then approach the middle room in the newly sectioned area.
We rinse and repeat.
Eventually we will find James just like the first method, but we find him in a much faster way with the second method.
How much faster would that be?
Well, that depends on the number of students in the school.
Let's say there are 500 students and we're looking for one, it would take about 80 minutes in the first manner, but one and a half minutes with binary search.
But let's say they are 1,000 students in the school, it would take 160 minutes with the first method but 1.6 minutes with the second method.
Now, that's a whole lot of difference.
So a name is just a word.
But Google searches a combination of words, making it a little bit more complicated.
So just like how we identify the first letter of each alphabet of the name, Google identifies 200 unique factors making your search terms faster.
If you recall, the effectiveness of binary search depends on the prearrangement of data.
And that's why computer scientist are actively looking for ways to sort, manage, and eventually retrieve data fast and better.
In the same way, the TV remote goes near the TV, the shoes go to the shoe rack, the coats go into the cupboard and the winter gloves go into the winter jacket.
Aha!
So that's where my gloves are.
[END PLAYBACK] [APPLAUSE]
Joshua, what are the things that you like about your cut, and what are the things that you don't like about it?
I guess I like the opening.
But I realize-- there were times where I realized I have a better take, but it's because they clicked it but I started earlier
So there were takes where you got cut off just at the beginning or something like that
So I realize I should have much, much more takes, because you never know what happens.
And I had to scrap out an entire scene because I was in a location where they had just started preparing for some event, and then everything became very noisy
Yeah, so there were like last minute things that came up in your locations
I do recognize that the one where I'm supposed to do animations the sound is off, and the visuals are not really there yet
You mean the one where you're looking at the desk?
Yeah, at the black-- [SKIPPING THROUGH VIDEO]
Yeah, there's supposed to be animations here, but I guess this is not probably the best one
You were going to have your animations come up here with your hotel room-- or your--
If I have to redo a scene, this scene is the most inconsistent among the rest
This was one where you were filming later in the day
Yes, this one was my first day, but later in the evening.
And then the next day the rest.
Everything else in the next day except this
I think in this piece, because A, we don't have the animation, but B, your eyes are looking at the animation as opposed to still talking to the screen-- and I think, from my perspective, I feel like there's a little bit more of balance at play there of connecting with the audience a little with your eyes.
Like if you're going to reshoot it, for me, I think balancing looking down and still looking back instead of just looking down during the animation and then looking back at the end
Looking at the animation is good.
It's kind of fun thing to do.
But again, look and acknowledge the audience as well.
I think this one, it's really just a matter of energy level here.
Overall, your delivery is very nice.
It's a very natural.
I love the script.
Again, you're taking something that's very hard to visualize, and that's why-- You know, we were trying to figure out what topic to do when you were brainstorming, and there were just no videos on binary search, which was shocking to me.
Or there aren't very many good ones.
And there aren't very many videos on how Google works, period.
And it's because it's very hard to visualize.
And I think you've done a nice job.
The only thing is this last scene.
The delivery is great.
But I almost would like to see this done in a computer lab or something instead of in front of the Microsoft Office.
I know that you were trying to work with sort of that Kendall Tech area, but it reads like a little maybe product placement-y.
So if you want to reshoot that one if you have time, I would say there are computer labs here that I can help get you into if you want.
Were you going to have animations in this scene?
This would be just a number that pops up and then a number below
I think that sparing use of labels will also help this video too, like when you first introduce the term binary search, have come up.
When you talk about the numbers.
Numbers are hard to keep track of in a video, so just having those pop up
So it would just be first measure 80 minutes and binary search 1.5
Yeah, something like that.
Something really simple.
And you've got a good background for it, so I would go and take advantage of it
You have such a smiley presence that's so engaging on the camera.
You have such positive energy up there.
It's like you kind of want to smile while you're watching it
And again, you're taking something that is a seemingly intimidating subject, which again is why it's very hard to find videos on computer science, and even more rare to find a live hosted video on computer science.
All the ones that are out there are usually some sort of voiceover animated tutorial, and I think that having you on screen being the guide for the audience with this is very, very nice.
It's very powerful
Could you go back to the beginning of that, where you're in front of the desk and about to point at the animation?
--find James.
There's a better method known as binary--
I feel like someone just told you a joke and we missed the joke.
And you're like kind of about to die laughing.
I don't know if you guys felt that same way when you saw it.
Did someone tell you something really funny?
I couldn't figure it out.
I don't know why I behaved like that.
I do remember a joke.
I did remember a joke
I wanted to be like, what was so funny?
There's something funny going on here?
I must have saw something that he was doing.
Maybe he was in an awkward position
I mean, I think you're going to refilm this one anyway, but if you do-- it was funny because in this moment I was thinking, there was something funny that must have just happened, and I don't know what it is
What was I going to say?
For the shots that you did at the Koch Institute, I think you can just center this.
Center it close enough.
Because I know you're trying rule of thirds here, but there's sort of like-- it's non-homogeneous distribution of this pattern in the background, so it looks a little weird to have all this extra space.
So I would go ahead and just center this.
What do you guys think about the music in this video?
It's good?
It's a little light-hearted.
It doesn't play into the whole computer science trope, which is nice.
I would bring-- go ahead
I thought it was like 1950s suburbia, middle America thing
I would bring down the levels just a little bit.
In the opening it's fine, but as it progressed, I would either up your recorded levels for your own audio or lower the music.
Once the bass starts driving, it gets a little distracting.
Any final comments?
Who's doing the Google Docs for Joshua?
David and Nathan.
Also, just as a comment, I think you did a really, really excellent job with the pacing of your cuts.
We never explicitly stated it, but I think Elizabeth and I said it in the computer lab last week.
You did a really good job leaving a little bit of space at the end of something that you said for people to absorb.
But then when you starting your new scene, you cut it right before you dialog.
And so the video itself just felt really, really well-paced because of that.
There was never any point of it where I was waiting for something to happen.
It was all just like--
If I were to add, they have that actually I didn't feel-- I cut once, and I felt it was awkward.
And then I cut again, until it reached so near to the audio.
So it was a few times I had to cut to realize that it awkward.
Like even for 0.3 seconds, it would be awkward
You have really good instincts to do that
And I know that I have said this ad nauseum in the emails, but I feel like there's this sense of fear.
And maybe it's just the training that we have in our traditional educational system, but we all want to measure like 20 times because heaven forbid we cut more than once, right?
And here you're literally cutting.
But it's through the process of cutting and trying different things that you learn what feels right and what makes sense in the video.
And I don't know of anyone who has ever made a video on their first try.
Even if they've perfected the shot list, gotten the best script ever, have their production date down to the second, something will ultimately end up happening that makes you reconsider how you're going to cut things.
And it should.
It really should.
I mean, I am really happy to hear that that's what you discovered, because I think that that goes maybe a little against how we traditionally go through school.
So I'm glad to hear that you try different things.
It's OK, do not be discouraged by having to try different cuts.
It's a productive process that moves you forward.
OK, any thoughts from PJ before we screen?
So there's parts of the audio that I'm moving so I can get the--
Oh, you hear a little bit.
Yeah
So yeah, after we do that.
And then it was really hard trying to keep it-- there's no-- after watching it, it's like two videos.
It's hard because it's in one setting, and they complete their form.
And I couldn't find like--
Music is hard.
It's so much harder than you initially think it will be.
You think, oh you just like slap something on in the background, it'll be fine.
But it's very time consuming.
So it's very like-- I don't know.
It seemed very inspirational at the beginning
Like inappropriately inspirational
Friday Night Lights kind of thing
All right.
Well, let's see.
[VIDEO PLAYBACK] -When we take complex things and break them into smaller pieces, we find out that we know a lot more about things than we think.
Now let's take this box-- ORCA I-- and create damage below the water line, which I've indicated right here.
As we can see, we've damaged ORCA I right here.
Now let's put ORCA I on the water and see what happens.
[GURGLING] The ORCA I sank due to the weight of the added water.
What if the ORCA I contained cargo, or oil, or even people?
Now let's take ORCA II and do the same thing.
So we can see that ORCA II did not sink, although it is sitting on an angle towards the bow.
So why didn't ORCA II sink?
As easy as it sounds, this simple demonstration is essential to the design of huge complex ships-- ships that are responsible for carrying about 90% of all our stuff.
As naval architects, how do we design ships carrying our stuff to make it into port safely and not sink?
Well, why don't we find out?
Here we have ORCA I and ORCA II from before.
Although ORCA I and ORCA II don't engage in international trade, they behave just as a 1,000-foot container ship would.
Now, let's take a look in ORCA I.
We can see that there's nothing in it.
It's just a box.
But if we look at ORCA II, we can see that it's subdivided into these watertight compartments by these transverse watertight bulkheads.
Now, what that means is that if we were to damage this ship right here, water would only flow into this compartment.
It would not go into this one, this one, or this one.
That would cause the ship to be angled or trimmed in the water, but it would not cause the ship to sink completely.
We refer to ORCA II as being subdivided, and we can see subdivision in many of these ship's plans.
It is unclear when subdivision first started being used in ships, but accounts of 5th century Chinese trade ships indicate that water would be able to enter the vessel without sinking.
So let's find out why this happens.
Let's imagine a barge divided into 10 equal compartments.
One of them springs a leak from damage.
Since the ship is subdivided, only the first compartment floods, and the ship remains afloat, protecting both its people and cargo.
Although the added water causes a ship to trim, it still has enough buoyancy to return to port for repairs.
Ships still sink, though.
It's both expensive and impractical to try to design the unsinkable ship, especially when these ships will never see that amount of damage.
That's why, as naval architects, we use computer programs to help us out with subdivision.
Computers make it easy to simulate certain damage cases in practically no time.
With different software, we can damage certain compartments, and see how the ship responds to it.
This gives the naval architect a good idea of what parts to improve on the ship, if any.
So even though ships seem like these intricate, complex things, they're really just based on principles that we all already know.
[APPLAUSE] [END PLAYBACK]
Just out of curiosity, did you shoot this video in sequence?
Like, you shot the first scenes first, second--
So the first scenes were second.
The second scenes were first
Really?
Yeah.
That wasn't out of design, it was just because the the only time we could go there was day
Yeah, it's a location practicality there.
Any comments for PJ?
I didn't feel the change of music, as in, it was natural.
I didn't feel like there was a sudden change

I liked the second song.
I agree that the first one is a little Friday Night Lights.
[LAUGHTER]
The thing with-- just like comedy, pulling off drama is also really hard to do in YouTube videos, because you're not working with, like, a sweeping epic or a documentary.
It's like a little YouTube blurb, and we had trouble with that with Farts when he talks about how Clostridium difficile kills some thousand Americans a year.
We didn't want to have grandpa's banjo playing in the background when he says that, obviously.
But it's not like we were going to insert music from Gone with the Wind in the background either
Well, one thing that I wanted to try music because a lot of the places I was in, there was-- you can see there's a lot of cuts, so where it goes to, and if there's not anything underneath, it's very pronounced.
And I tried editing a lot of it out
Let's see.
So I know that you were hesitant about being in front of the camera too
So natural
And I'm surprised that you said you filmed them in the rivers, because I feel like the stuff that you have in the end is really, really natural, and I thought that you would have maybe done that second.
It's hard to host while you're doing a demo.
You just have a million things to think about.
But in general, the hosting is really nice.
It's really, really natural, especially this part.
I can tell that Yuliya and-- who was it-- David was filming with you?
Kenneth
Kenneth was filming with you.
They probably tried to make you laugh in some of them and smile
Yeah, they were really good at doing that
Yeah.
I think this scene is your strongest hosting.
Just, like, going back and forth between the computer, this scene just plays out very nicely.
And again, taking a seemingly intimidating subject, and welcoming and inviting the viewers in.
It's very nice.
I would say, in general, the cutting advice that [INAUDIBLE] gave, which is to leave the room at the end of cuts-- or at the end of scenes-- and cut very close to when you start talking at the beginning of scenes, might help with this.
There was one--
Is it the moment where Yuliya's in the picture with--
Oh, yeah.
That's, like, a weird insert thing
I was, like, who is she?
What is she doing here?
She needs to get cut out
In general, I love the speed up walk, and how you open the door into the lab.
I love seeing the background.
It's very cool.
I think you can speed up you damaging

And I think, you say something like--
Let's take this box-- ORCA I-- and create damage below the water line, which I've indicated right here
Yeah.
I think you can actually cut that part out completely, because after you cut it, you say, as you can see, we've damaged ORCA I, so I think that's enough information.
And I get what you're saying with the water line, but I don't think it's super relevant to the point of the video, so I think it's OK if you cut that out entirely.
Oh, yeah, when you go from this scene--
--in the weight of the added water.
What if the ORCA I contained cargo, or oil, or even people?
Yeah.
The music here is, like, it's like trying to pull out my heartstrings
Now let's take ORCA II and do the same thing.
We can see that ORCA II did not sink-- ORCA II sink.
As easy as it sounds, this simple demonstration is essential to the design of huge complex ships.
So why didn't ORCA II sink?
So this is an example-- it happened one other time where you leave a gap in between some of your sentences, and the gap is a little long.
So you can cut to a digital zoom of the same shot to clean up the gap a little bit.
Because I think that you were doing it to help you host, and, like, help you get through the lines, which is totally OK and understandable, but I would just hide that gap a little
I stuttered here, and I kept looking back to the boat, so it was weird, like I was using that almost as my crutch, that I had this prop that I could kind of like square up
I think you actually were-- your delivery was fine here.
I wasn't distracted by it
--II did not sink, although it is sitting an angle towards the bow.
So why didn't ORCA II sink?
I like the way you ask the question.
That's, like, how you would normally
As easy as it sounds--
And see how that gap feels just a little bit too long
--this simple demonstration is essential to the design of huge complex ships-- ships that are responsible for carrying about 90%--
Do you have B-roll or pictures of some of these ships?
That was the one thing I was thinking throughout this video was, like, I would love to see what the things look like in real life
I could probably do something like that
I think this would be a great place to cut to B-roll or pictures.
You can just keep the audio, but that would help mix up the visuals a lot too
--of all our stuff is--
OK.
So this cut was a little-- this was what I was talking about with the line of sight when I gave the editing lecture.
In the one scene, you're kind of sitting like this, right?
And you have, like-- this is a terrible drawing.
But you have your boat in the water, and it's sort of, like, a weird semi-bird's-eye.
And then you cut, and you're not quite in a different enough position, so it feels just, like, slightly jarring.
It feels like someone just tipped the camera over a little bit.
So either cut to a closer shot of you like that, to hide that a little bit, or do B-roll there.
If you're coming out of B-roll here, then this transition won't be as weird, and my comment is totally irrelevant, but just something to think about
I mean, so-- sorry, I'm, like, 10 steps ahead of my head, where I am in actual reality.
You never put your Coast Guard outfit on
Yeah.
I felt like if I wanted to try to make it seem like less, like, me talking, that would have just put on a whole different persona.
So I kind of, like, I'm wearing boat shoes, so that's hard to do
OK.
I just wonder if at the very, very last scene, where you have your finale comment about a very simple concept as floating, if ever there were a moment to put that hat on, and have your awesome Coast Guard outfit on, that would be it
If you don't want to shoot it now, we're going to make you do it next week when we shoot season 3, OK?
Yeah.
Got to go to the dry cleaners
Yeah, that's a good comment
The thing that you have that's so cool is, I mean, we talked about the fact that you actually, legitimately, are out there in the Coast Guard.
And you say that, but showing us that in your film is really powerful.
And even if, at just the very end, you tie that real life application back, I think that would be really, incredibly cool
Just for the last sentence too.
And again, like, time permitting, right?
Because I know that you guys don't have a ton of time to incorporate all this stuff.
You've already gotten plenty of footage of you talking in this setting, so it wouldn't be very jarring to have you say the last sentence, like, out by the sea, or by the boats
Or even if when you say, like, I'm a Coast Guard, whatever, even if you quickly have a picture of even photos of you in uniform, to just put-- I don't know.
I don't know how that would look.
But just something that shows us, and doesn't tell us this totally authentic side to yourself would be cool
So do you have access to the base here?
To the one in North End?
Yeah
Yeah
Because one of our--
They have shirts there
Yeah.
I mean, one of the Sloan fellows is also an officer
OK. [INAUDIBLE]?
[INAUDIBLE]
No.
He's based in there
So again, I know it's short notice-- you'd have to do it between today and tomorrow, basically, so it's OK if you can't do it, but something to think about, if you want to do it for Science Out Loud.
All right.
Yuliya, you ready?
Yeah.
So there are-- the beginning and the end, we filmed outside.
There's a lot of wind, and I couldn't connect the mike, because I was standing far away.
So I tried to do voiceover.
I don't know how it sounds, because honestly, I was just tired of hearing my voice at that point.
And then, there is this, actually, like the thumbnail, this is where the animation is supposed to be.
So the white screen is animations
Yeah?
David and Kenneth are the ones who are Google Doc commenting on this
Thank you.
And we'll go through it again if you guys forget.
[VIDEO PLAYBACK] [MUSIC PLAYING] -Hi.
What do snowflakes and cellphones have in common?
Well, let me start by drawing a snowflake first.
I draw an equilateral triangle, divide each side into three parts, and draw another equilateral triangle on top of each one.
Then take out the middle, and repeat the process, this time with 1, 2, 3, 4 times 3, which is 12 sides.
Eventually, the shape will start looking something like this.
In mathematics, this is called a Koch snowflake.
If I repeated my process again and again, I would see this same pattern anywhere I looked.
This is a Koch snowflake, this time drawn on a computer.
Such neverending patterns, that on any scale, on any level of zoom, look roughly the same, are called fractals.
Computer scientists can program these patterns by repeating an often simple mathematical process over and over.
In the 1990s, a radio astronomer by the name of Nathan Cohen used fractals to revolutionize wireless communications.
At the time, Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So Cohen designed a more compact fractal radio antenna instead.
The landlord didn't notice it, and it worked better than the ones before.
Working further, Cohen designed a new antenna, this time using a fractal called the Menger sponge.
The Menger sponge is not really the sponge you'd be scrubbing your back with, but you can still think of it like that.
Imagine both water and soap getting through your sponge's holes, except the water is Wi-Fi, and the soap is, say, Bluetooth.
The fractal's infinite sponginess allows it to receive many different frequencies simultaneously.
Before Cohen's invention, antennas had to be cut for one frequency, and that was the only frequency they could operate at.
Without Cohen's sponge, your cell phone would have to look something like a [INAUDIBLE] to receive multiple signals, including the one your friends use when they call.
Cohen later proved that only fractal shapes could work with such a wide range of frequencies.
Today, millions of wireless devices, such as laptops and bar code scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
Nature has been doing it the whole time, and not just the snowflakes.
Natural selection favors the most efficient systems and organisms, often of a fractal form.
The spiral fractal, for example, is present in seashells, broccoli, and hurricanes.
The fractal tree is relatively easy to program, and can be used to study river systems, blood vessels, and lightning bolts.
So many natural systems previously thought off limits to mathematicians can now be explained in terms of fractals.
Mathematics allows us to learn nature's best practices and then apply them to solve real world problems.
Much like Cohen's antenna revolutionized the field of telecommunications, other fractal research is changing medicine, weather prediction, and building design here at MIT and everywhere in the world.
Look around you.
What beautiful patterns do you see?
[END PLAYBACK] [APPLAUSE]
All right.
Any thoughts from anyone?
Should we dive right in?
Yuliya, what are your favorite and least favorite parts about this cut?
Let's see.
There was some scenes that I liked better than others.
That's mostly because of the lighting and the sound.
I think the one in the classroom had a nice sound and nice lighting.
So I guess the indoor scenes were a lot easier to shoot
So I think, in general, again, very nice script.
This came such a long way from the initial ideation phase, and I think it is, like computer science-- I mean, my favorite thing about all the videos that you guys have done is the topics are so unique, and so different than what people normally get in school.
And they're taking all these concepts that they've learned in school, but just putting it in a very different context.
And it's hopefully because you're talking about the things that you're excited about.
And so I'm really happy with how you've transformed, like, math, which is, again, a very hard video to do if you don't do it in the style of Vi Hart, and putting your face and hosting it.
It's done nicely.
I think, in general, the video has a very fast pace to it, and I don't know if it's because the cuts are happening so frequently, or you're naturally a very animated person, which is nice, but leave a little bit of room, like, a little bit of breathing space to compensate for that.
I think maybe the reason why it was hard for me to follow a little bit was the opening seemed a little too short.
And I know that you had edited it from the last time I had seen it, but let's-- [VIDEO PLAYBACK] [MUSIC PLAYING] -Hi.
What do snowflakes and cell phones have in common?
Well, let me start by drawing a snowflake first.
[END PLAYBACK]
Yeah.
It seems like it's so overly condensed.
And I know you had brought it down from something that was a little bit bigger, but I think that it's just so, like, it comes out of nowhere, like, hi.
What do these have in common?
It's like, wait, what?
Like, I wouldn't have had that question.
And even getting rid of the hi helps with that.
It's a nice question.
What do snowflakes and cell phone antennas have in common?
But there's not enough of a lead into you going into describing what a fractal is, yet fractals is on the title.
I think that maybe you could do a little bit with the transition there, and I'll brainstorm with you, because I can't think of anything off the top my head.
Do you guys have any suggestions for that opening?
Yeah, Andrea
Maybe like, I don't know exactly how you could do it, but almost like the first shot, where you're saying, what do snowflakes-- and then you see, like, sort of an image of a snowflake and cell phones, and then it sort of zooms out, and you realize that the picture is on the cell phone
Maybe
Sort of have in common.
And then, you're-- and then you
I mean, I think it's OK to introduce the idea of fractals, as long as you set it up a little bit more.
So something like, what do snowflakes and cell phones have in common?
Neverending repeating patterns called fractals.
And then do a little bit of a slower transition into you at the blackboard here.
Something a little bit more like that.
I think the visuals help a lot.
I know that you had filmed up in Stata, because eventually you talk about how it affects architecture, but I don't think that this location is super vital to the opening, and I think you can get cleaner audio if you do VO, or if you just re-shoot it with you on the table, and then you have a picture of a snowflake that comes up in a picture of a cell phone, or something like Andrea said.
But I think having the audience onboard from the get-go will help with the pacing of the video as it continues.
I know that you had struggled with the music too, and I think-- I don't know-- so much of music is personal taste.
To me, all I can think about when I hear this music is, like, old Korean soap operas.
And so I don't know if it is, like, the most appropriate tone, and maybe I feel that way because of experiences of my parents watching them, but I think you could play around with some other pieces.
I like the general tempo of the music, and how it's sort of this same melody on loop in the background, so it's not super distracting, and it's a good volume.
But I would play around with a couple other songs, and I can help you with that, if you want to run some options by me
I wanted to use this one because it was kind of calming, so kind of to offset the general fast pace and animated talking to something soothing in the background
Yeah So I think it's a good tone to start with, and again, maybe it's just my personal preference, but something about it made me fixate on the music more than on the text.
Did anyone else have-- maybe it really is, just, like I can't get over how much it reminds me of Korean soap operas
It stood out for me at the start, like, especially for this scene, it gave me the feeling like it was going to something romantic.
I don't know
It's very sentimental, like the music evokes something very sentimental, just like the opening of the ships one evokes a mood that's very different from what you're about to go into.
And especially if you open with something like that, if you'd open with a banjo thing-- or not the banjo-- whatever you had for the second piece, I think it would have been fine.
But something about opening with music that sets a tone that's very different from what you're intending to achieve, it threw me off a little bit.
But luckily, it's something that it's easily fixable.
You can easily play around with different types of music
Because for me, you give us, because there was a Digicom video, and there was the idea that you could use something calming, so I tried to use something calming, but as you see, I didn't choose that, because I realized the calming one will make my video feel very monotonous.
But when I observe the Digicom audio closer, I realized there was some percussion notes, so that helped.
So maybe you could find something similar to what you have right now, but with a little bit-- not too much-- percussion here would help with the pacing
That's really good.
And again, with music, there's no hard and fast rules with what works and what doesn't.
I mean, a lot of it is just, like, your gut feeling when you hear it, if it makes sense, or if it doesn't.
And obviously, my gut feeling was very different from everyone else's.
Yeah, Jamie?
First of all, compared to where we started, this is isn't, like, oh my goodness.
This is way, way more awesome than I ever thought was possible, with our first conversation about it's not real, so this is like-- and what I loved with the fractal piece at the beginning, where you really explained, because we gave you that feedback in the last time, was you need to set us up for success in explaining what a fractal actually is, and I thought you did a really fantastic job.
One think that you may want to think about is you have a very sing-songy voice.
Your voice is high, and it's kind of sing-songy.
And at the beginning, you set up this-- I don't know if you've ever seen Mrs. Doubtfire, but it's this movie where Robin Williams is dressed up as this old grandmother, and is just kind of this hilarious figure.
And I think there's an unintended humor to your sing-songy-ness at the very beginning that makes me feel less credible wanting to watch you.
And I think there's a very simple fix, which is just grounding your voice in a lower register.
When you talk with your voice later on, just listen to that as you watch yourself.
I notice that the times when your voice is a little lower is when I don't notice that as much, and so since you are thinking of reworking this first scene, you may want to think about consciously grounding your voice a little bit more in your chest, so that we don't, like, unintentionally think it's funny when it's not intended to be
And I know George had said the camera calms you down, and in general it does, and you compensate with higher energy, but again, this is what I saying before, and oftentimes, hopefully, this stuff that we were saying earlier in the class makes a little more sense after you guys have gone through it, but sometimes the tendency is to compensate or equate higher energy to higher pitch, higher volume, like, higher everything, and it's really not that at all.
A lot of it is just the mentality of, like, I'm going to be really present here, right now, and really give it 100% here
There's actually a really good TED Talk that's about how to speak to be listened to.
And it's basically, you do these little vocal exorcises, and they both relax you, and they'll drop your register down, like, just going through them.
And I had to do those prior to doing-- I learned something similar when I was first learning public speaking
It's a great thing to think about not just with making these videos, but when you're giving talks later on.
Again, like, when you're nervous or excited, your tendency is to talk a lot with your hands, which I know I do, and talk really high pitched.
So it's just a good thing to think about, in general, after this class
Number two
At the beginning of those voiceovers, I was trying to match the video in my room.
And it did feel a little bit off as well.
I wasn't sure how to--
I think you could do the opening, like, just you set up a tripod, and sitting in your room or something, just something really simple.
What do this and this have in common?
Well, it's actually neverending repeating patterns called fractals, and then you go into fractals
On a very practical level, like, I do a lot of helping of people get ready for talks in my lab, and just, like, grounding your feet wherever you are, feeling like you're grounded to the earth, and then getting a really deep breath, like way from in here before you speak, that will help you fight that tendency to go way up high
And I think the animations here will look really good.
Again, like, visual cues, anything that you're going to add, even if it's just still pictures will make a big difference.
And I would play around with the ending too.
I like that you do the review, but again, the audio is hard when you're that far away.
And you may be able to shoot this in the same room that you shoot the opening in-- or you re-shoot it, and then, I don't know, have a picture of Stata just pop up next to you.
I think that's a way to compensate pretty easily.
What is the world background for?
It matches the world at the very end
Yeah.
And it was just a nice background for titles
Oh.
I would just go black, honestly.
Just a black background.
In general, I think simple reads more professional.
Oh, I don't know.
Professional's not the right word, but simple is better, I think.
And I didn't make the connection of, like, look at the world when I saw the background.
It's such a minor thing, though.
I mean, it's just your credit, so it's OK, but just something to think about.
Any final comments for Yuliya?
I thought that one scene in the dorm was really good, where you inadvertently had all the stuff behind us.
I didn't know that it was going to turn out like that
Oh, the one in the lounge
Yeah
Yeah.
And I think it was the best lighting, too, of all of them
That's great
All right.
So who's going to comment on Yuliya's?
PJ and Josh
All right.
Last one, and then we'll take a break.
David, do you want to say anything before we screen?
Your video will just speak for itself.
[VIDEO PLAYBACK] [MUSIC PLAYING] -If I went out for a run now, dressed like this, you'd probably call me nuts, right?
Because it's so cold outside that I'd probably die of hypothermia.
However, in 2009, Wim Hof managed to run a full marathon in minus 20 degrees Celsius, wearing nothing but shorts.
While most people would probably die, Wim Hof has survived.
So what makes Wim Hof able to survive where others can't?
The research team were suggesting that they subjected him to an icy bath for almost two hours.
While most people would have their core body temperature drop to below 35 degrees, causing hypothermia to kick in, Wim Hof's body temperature dropped to a merely 37.4 degrees Celsius, staying surprisingly warm.
And the funny thing is that the researchers couldn't find exactly what had happened.
Maybe it's his genetic ancestry that saves him.
In a sample study, studies have found that individuals with cold-climate ancestry, have been found to have mitochodria that can produce more heat and less chemical energy.
Mitochondria is like a mini-furnace that burns the sugars from the food you eat into chemical and heat energy.
Individuals like this are better able to survive the cold, however, scientists haven't fully studied Wim Hof's genome, so we don't really know.
Wim Hof practices g-tummo, a Tibetan form of meditation and breathing, which allows him to double his metabolism.
Remember those mitochondria I mentioned before?
Wim Hop can produce enough heat through these mitochondria with special breathing, and massive contractions that can produce enough heat to body warm.
Well, you might be tempted to believe that this is wishy-washy nonsense.
However, participants in a study at NUS have been shown to increase their core body temperature using g-tummo, and they had no prior experience.
Now, why is it so?
Strong increases in brain waves were noticed, and many have theorized that this enables the body to effectively heat the center as well as distribute it to the extremities.
We do not fully understand Wim Hof's methods.
Although some have tried to follow in his footsteps, you would be unwise to do the same.
And Wim Hof even claims that he can control other parts of his body.
Now, if this is true, and scientifically a fact, who knows what doors could open.
And since science hasn't figured it out yet, and I, myself, don't practice these methods, maybe I'd better be off donning more jackets.
[END PLAYBACK]
David, what was the hardest part about filming this video?
I think it was following the script
Following the script.
Just, like, remembering it when you were shooting
Yeah.
I really think that was the hardest, because trying to say everything while looking at a camera felt very, very real
Yep.
That makes a lot of sense.
And again, like, using your partner to the fullest extent when you're filming can really help with that sort of humanizing the mechanical element of production, trying to talk to the person behind the camera.
I think that you didn't look uncomfortable on camera.
The only thing is, in general, you talk really fast, and with the music that was in it, I think that you can take some of the music out, or drop the volume a ton, or play with, like, less busy music.
But having someone who talks naturally fast, and then, like, very upbeat music in the background is just, like, exacerbate, it's, like, constructive interference
[INAUDIBLE] calm music, but I felt like I put calm music in
Yeah.
I mean, you can also experiment with not using music in certain parts of the video.
You guys don't need to have music the whole way through.
It's OK. And I think because you talk so fast naturally, the music can be-- remember, with music, first do no harm, right?
It can be hard to understand just the sheer amount of content that you're saying when the music is so upbeat and so loud
I think the music was, at least for me, personally, it had the same effect as when Elizabeth was showing the choices for the heart video, where at the beginning, it was a little bit too much for me, and I got the giggles.
I don't know what about you putting on running shoes, and, like, just the loudness of that music, it just seemed a little bit ridiculous, that I wasn't mentally ready for some serious facts at that point
I wanted to make it upbeat.
I wanted it to be like I was going to run in the cold.
[VIDEO PLAYBACK] [MUSIC PLAYING] [END PLAYBACK]
It's good, like, until the point where you ask the question.
From, like, instead of being, like, oh, OK, it's OK to be, like, oh, this is me when I'm running in the cold.
But then as soon as you switch to actual explanation--
Yeah.
I would just duck it out after you ask the question.
[VIDEO PLAYBACK] -If I went out for a run now dressed like this, you'd probably call me nuts, right?
Because it's so cold outside that I'd probably die of hypothermia.
[END PLAYBACK]
And also there, I mean, everyone's done this, where they take pauses in weird parts of their sentences-- I mean, if you wanted to make it funny, I don't know if you do
She's making--
I am so sorry
I think it's one of my favorite things--
The last time this happened, I was thinking, oh my gosh, this happens to be more than I would like to admit, but I was, like, I hope this doesn't happen during the class, and it ends up on the internet, but now it's going to anyway.
[LAUGHTER] If you wanted to make it funny, you could zoom in-- as you-- nevermind.
Don't make it funny
So if you wanted-- I was going to say-- every time you pause, cut out that pause, and then maybe a little bit closer to your face, like, what if I die?
And then, like, zoom in, I'm cold.
That's really comedic
It's kind of like the thing that we did, and it's not very good, like the extreme zoom, and you could make it, like-- with comedy, going big or going home works the best, because you really have to commit to it, so if you want it to have a huge comedic effect, you would, like, zoom in really close, and you would have, like, a weird sound effect.
I'm so sorry, you guys
I think it's really funny to you
But if you don't want to make it funny, which I would recommend, either cut out the gaps and alternate between zooming in and out like Hank Green does on SciShow.
There's one other time that happens, like, right here.
In general, like, if your gaps are happening in places that are not at the ends of scenes, really be conscious of how long those last.
And having two other people look at your video today will help, because sometimes it's hard to tell when you're watching yourself if something looks like it's lasting too long or not.
But if you have gaps that are too long that are in the middle of your scenes, you can use the trick of digital zooming in and out to hide the cuts.
Does anyone have general comments?
Yes, Yuliya
I like the delivery.
It was really natural.
And the music was sometimes distracting.
Another thing that had that effect were some of the backgrounds, so like the one where you're holding a trashcan, and there's a person, and then there's a trashcan.
So I wasn't sure, like, did that mean the person was in the trashcan, like, what happened, and, like, yeah, like, this scene.
And then there were some others, like the lab.
I wasn't sure why the lab was there.
Even though it was really cool
I know that you held up the trashcan because you were trying to say, like, he was in a bucket.
But again, with the music, it's hard to hear that you're explaining the bucket thing, so I would just cut to the picture of him before you hold up the trashcan.
[VIDEO PLAYBACK] - --team were suggesting that they subjected him to an icy bath for almost two hours.
[END PLAYBACK]
Yeah.
I mean, in general, that line is spoken so quickly that it's hard to follow what you're doing
And it's not just the speed.
I think you need to put a little more diction in between your words, so that, like, what I hear happening in they're kind of slurring all together, so it's not just a matter of speed, but it's that I'm having trouble actually separating this word from this word, and therefore, there are chunks that feel like one big long word, that they're actually made up of four or five words.
I think there's a piece of slowing down, but then also, forcing yourself to use over-diction to clarify your words.
That'll help a lot too, more so than you think, because it will help clean up those little spaces in between the words to then provide me with a little more, like, OK, I'm with you every single step of the way, if that makes sense.
So if you are going to do any retaking of videos, try really hard to really tighten up the way that you say the words, in addition to slowing down a little bit.
I think that'll help people
I think if you do that, that'll inherently help you slow down, because you're going to think about each word at a time.
I like the use of pictures here.
It's really helpful, because this is an instance where we just don't have access to shoot any of this.
I would play around with, in general, zooming in and out, when you're doing long takes.
I forgot to mention this, but I thought that was a really nice thing that Yuliya did in her video, was-- because we had talked about this last week-- how to clean up some of the gaps-- and she does some nice work with cutting into, just like her hand drawing the fractals, and I think that, in general, people can take more advantage of that technique.
Oh.
So here you're cutting in between closeups, but this is another thing with the iTrace, where in the first scene you're on the left, and on the second scene, you're on the right, and you're also not in the same 3D plane of field as you were before.
So in the previous scene, you were close to the bench, and here, did you scoot up forward?
So it's not terrible-- it's not like the end of the world or anything, but it's, like, a little jarring, and I guess I'm subconsciously taking time to adjust my sense of space and where you are when that cut happens.
I don't necessarily think you have to re-shoot it, but I would either-- let's watch it
And they had no prior experience.
Now, why is it so?
Strong increases in brain waves were noticed, and many have theorized that this enables the body to effectively heat the center as well as distribute it to the extremities
Is there a B-roll that you could use there?
Because I think that's a really good opportunity to-- it's a key point that you're connecting what someone is doing with their mind, and how that's legitimately connecting to the rest of their body
[INAUDIBLE]
Could you even just use pictures of the brain, and then, like, the human body?
Again, this is a really hard thing to visualize, and I know you're not working with a ton of resources, but I think that's an opportunity to maybe try voiceover with the technique that Jamie was talking about, because it's such a crucial bit to the video, actually.
That's where you're actually tying the science into something that seems a little pseudo science
Keeping that in mind, as you watch your video-- and whoever's talking over with you, really connecting it and grounding it in the real science too.
Because I liked your furnace metaphor, but then I was distracted by the burning exploding boxes, and if I've never heard of mitochondria before, or cellular respiration type things, that wouldn't be very distanced.
I would think explosions rather than this is grounded in my biology type things.
And so maybe think of places, where you can include images like the circulatory system, things like that.
Just, like, keep grounding it
When you cut from the guy doing meditation to you sitting cross-legged, was that purposeful?
Yes.
I'm supposed to be doing meditation
OK. That's what I thought.
Because I was like, not 100% sure, but I was pretty sure that was a choice that you made.
There's no comment.
I just had a question
Any other thoughts for David?
So you guys want to do Google Docs versus just talking one on one with people?
Is that the consensus?
Could we do both?
Yeah.
You can totally do both.
So the rest of class time is for feedback for each other, and this is a super, super crucial, critical part of the production process.
During the break, I will post a link onto the Tumblr that has all the Google Docs that I will finish making for everyone.
So for example, Yuliya, your two commenters are Joshua and PJ.
So Josh and PJ, just go into your corresponding page, and on the time code, just leave your comments.
I will comment on everyone's rough cuts as well.
And again, don't worry if you're repeating the same remarks as other people are doing.
How about we spend-- after the break-- doing the Google Docs stuff, and then whenever you guys finish, I guess, in the order of people finishing, group up, and give one on one feedback as well.
And we can help facilitate that too.
Does that sound good to everyone?
The hardest part is over.
The hardest part is just getting the first cut.
It's getting the first script out there.
And hopefully, moving forward, going to Thursday, it'll be easier.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Well thanks everyone for coming.
I know that this is kind of a funky time of day.
So I really appreciate everyone coming out to support the students of 2219.
This is the very first time we ever offered this class.
This is the very first time I've ever taught, so I-- it was an awesome experience for me.
And hopefully it was a great experience for all the students.
I am going to pull you guys up on stage at some point to talk about what it was like.
Just as a heads up.
So this class was initially called, Becoming the Next Bill Nye.
But as many of the students might tell you, it's not exactly about becoming the next Bill Nye.
What it is about is trying to figure out how to occupy the space that's in between education and entertainment-- a space that's not very well defined, a space that doesn't have very good best practices or best practices at all.
But the things that are important in this space are to create materials that leave a door open to the world of science, technology, engineering and math to everyone, to create materials that foster a lifelong love of learning, and to create materials that promote stem literacy within the public.
And these are all super broad sweeping ideas, but that is sort of what I think at least defines the space that's in between education and entertainment.
These were the four class values that we had.
And sorry guys, I know you've heard this already.
But this is for people who have never been to the class.
Because we couldn't really grade things like is this scene well lit, because honestly we were asking them to do so much within the span of two or three weeks, these were the values that were important to us.
Do the materials that the students make exhibit a sense of spark?
Is their enthusiasm for the topic matter very evident?
And this doesn't mean that they're talking with their hands, waving all the time.
It doesn't mean that they're acting like Hank Green with like super hyper personality.
It just means that their genuine love of the material is evident.
Is it clear-- "kill your darlings" is the phrase I used over and over again-- but are you able to really determine and distill what your message is very clearly?
Are you thoughtful?
Is every decision that you go into making-- what your scene looks like, what your line is, what your location is-- is that clear?
And then are you pushing yourself outside of your comfort zones?
I think there was only one student in this class who'd ever had experience with cameras or film.
And I think most of the students were a little nervous about being on camera, which is totally understandable.
It's very, very intimidating to be in front of the camera.
And I think that over the course of the two, three weeks, all these projects exhibit these values which I'm very happy about.
So what exactly happened in the last month?
Well, we basically took production, which is all of the stuff, which I think is actually structured the wrong way.
This is really what production is.
And they had to learn all of it, in basically two weeks, because they spent the last week making their videos.
And this isn't just how to turn on a camera.
It's how to write an effective script, how to engage people with your texts and your body language, and the way you're presenting things.
And then on top of that, they had to learn how to use all the equipment.
And then they had to learn how to edit sort of on their own, and learn how to plan things, and learn how all of these things interact with each other.
So what you're about to see are things that they basically put together in like two or three days, which is kind of nuts, and with very, very limited resources.
We had these crappy little canon camcorders.
So it's pretty impressive when you think about the constraints that they had.
One of the things that we talked about was storyboarding.
So I storyboarded the class for you guys.
Day one, if you don't remember, we talked about intro to the class.
Chris Babel gave a guest lecture on sort of introduction to using the camera.
Then on day two, I made the terrible decision of not telling people that we were going to have 20 sixth graders come to the classroom.
But luckily Jamie took care of that before I got there.
And they basically had to pitch ideas for videos to 20 sixth graders from Winthrop.
And then Natalie Kuldell came and talked a bit about her experiences launching BioBuilder, which is this educational nonprofit.
And then George came in and did a hosting workshop, if you guys remember this.
This is only like two weeks ago, which is crazy.
And then Josh and John from Planet Nutshell gave a workshop on storyboarding.
And then we had our table read.
And then you guys shot for the first time.
And then you had another table read.
Wow, did I mess up my pictures?
Sorry.
And then we talked about post production.
You guys literally learned the philosophy of editing in two hours.
And then you went off to film your episodes.
And then we had one screening.
And then what-- two more days passed.
One day passed.
And then we're here at today.
So it was a very condensed, very expedited version of all that stuff in the web that I showed earlier.
And so speaking of storyboards, we are at our first project.
This was Yuliya's first storyboard.
And yes, I combed through the Tumblr, and I'm pulling up all the embarrassing first drafts of everything that people posted.
Yuliya's first storyboard-- and I'm bummed because the text is super small and you can't see it-- but it's her first version of her script.
And it's basically her first attempt at planning what her video was going to be.
And the intro was what do a snowflake and cellphone have in common.
And her plan was to go out by the Charles, maybe while it was snowing.
And then she's going to have all these things about fractals, and how fractals are everywhere.
Fractals are crazy.
They're in broccoli.
Oh my gosh, so many fractals!
And then she starts talking about fractals.
Yuliya, do you mind coming on up here?
Pin this to yourself.
What was the process like-- hang on, I'm going to give this to you too-- what was the process like going from this storyboard to the video that we're about to see?
Well I mostly changed the script since this storyboard.
But this kind of helped me think through what are some possible locations I'm going to film at, what animations I can use.
So this was very important for me to kind of imagine what was happening, but not just in my script, but also how it would look on film.
And then I noticed that there were some issues with the storyboard, and then changed the script afterwards
And how-- your script probably changed the most.
Well, there were some drastic changes that happened.
But from your very first idea from day one, your thought what to do something about math, but how math wasn't real.
It was like this very philosophical idea.
And it was awesome.
But we had no idea how she would tell it in five minutes.
So how did it evolve over the course or what were some of the challenges that you had revising it?
So I started with talking about Godel's incompleteness theorem, which asks the question of is math real?
And then I pitched that idea to sixth graders.
And I came back, and I was really shocked that I ever thought that I could explain that to sixth graders.
So then I have to think, well what else can I tell about math?
And I like theoretical math, so all of the concepts that I line in math are abstract, and don't really have a concrete application.
So I start freaking out.
And then eventually I came up with this idea of fractals which were doodles, and they were present everywhere nature.
So it could be made relatable.
It could be made concrete, even though it was this very abstract math concept.
So that was the third script that I wrote.
That was about fractals and chaos theory.
Then I wrote another one, which didn't talk about chaos theory, because that was also complex.
So the issue for me was coming up with a topic that I could enjoy talking about, but also sixth graders would enjoy hearing about.
And in the process I also realized that I need more help than I thought, and more practice than I thought.
So it was a very humbling experience, especially the table reads
And how do you feel about your final video?
Again, what they're working with are these small cameras.
Most of them had never touched a camera really before.
None of them had used editing software except for maybe one or two.
So how do you feel about the final product that we're about to see?
Well, I'm not very happy with the quality of the video.
And I think part of it is I had to transfer between computers, and work with software.
There are definitely some scenes I would re-film if I had more time to get better lighting and better sound.
And it's only three and a half minutes long.
So I could definitely add more math to it.
But I think that yesterday when I uploaded it, I was pretty happy with what I got in the time that I had
Awesome.
OK, well we'll watch your video.
[VIDEO PLAYBACK] -What do snowflakes and cell phones have in common?
The answer is never ending patterns called fractals.
To explain, let me start by drawing a snowflake.
First, I draw an equilateral triangle, divide each side into three parts.
I draw another equilateral triangle on top of each one.
Then take out the middle and repeat the process, this time with one, two, three, four, times three, which is 12 sides.
Eventually, the shape will start looking something like this.
In mathematics, this is called a cox snowflake.
If I repeated my process again and again, I would see this same pattern anywhere I looked.
This is a cox snowflake, this time drawn on your computer.
Such neverending patterns that are under any scale, on any level of zoom, look roughly the same, are called fractals.
Computer scientists can program these patterns by repeating an often simple mathematical process over and over.
And in 1990s, a regular astronomer, by the name of Nathan Cohen, used fractals to revolutionize wireless communications.
At the time, Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So Cohen designed a more compact, fractal radio antenna instead.
The landlord didn't notice it.
And it worked better than the ones before.
Working further, Cohen designed a new antenna, this time using a fractal called the menger sponge.
The menger sponge is not the name of the sponge you'd be scrubbing your back with.
But you can still think of it like that.
Imagine with water and soap getting through the sponge's holes, except the water is Wi-Fi and soap is safe Bluetooth.
The fractal's infinite sponginess allows it to receive many different frequencies simultaneously.
Before Cohen's invention, antennas had to be cut for one frequency.
And that was the only frequency they could offer.
Alcon sponge, your cellphone would have to look something like a hedgehog to receive multiple signals, including the one your friends use when they call.
Cohen later proved that only fractal shapes could work in such a wide range of frequencies.
Today millions of wireless devices, such as laptops and barcode scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
Nature has been doing it the whole time, and not just with snowflakes.
Natural selection gave us the most efficient systems of organisms, all in the fractal form.
The spiral fractal, for example, is present in sea shells, broccoli and hurricanes.
The fractal tree is relatively easy to program, and can be used to study river systems, blood vessels and lightning holes.
So many natural systems, previously thought off limits to mathematicians, can now be understood in terms of fractals.
Mathematics allows us to learn each of those practices, and then apply them to solve real-world problems.
Much like Cohen's antenna, revolutionized telecommunications, on the fractal research, is changing medicine, weather prediction and art, here at MIT and everywhere in the world.
Look around you.
What beautiful patterns do you see?
[MUSIC PLAYING] [APPLAUSE] [END PLAYBACK]
I forgot to mention-- so obviously, if a student's on screen hosting, they are not filming their own project.
So what we did was paired people up into groups, and they filmed each other.
So that piece was filmed by Kenneth and PJ, and they all worked together to help each other out and direct each other, and it was awesome.
Next up, we have PJ.
Now PJ, for you-- you can come on up.
I took your second pitch from the Tumblr posts--
All right
--where you had come up with the idea of talking about ships.
His first-- yes?
Oh, thank you, sorry.
You had come up with the idea of talking about the science behind poached eggs?
Yeah
And--
Yeah, you shouldn't show that one
How did--
I don't even know how to poach an egg, so--
How did that turn from the science of poached eggs to what ended up being a video about how ships basically don't sink?
Well, honestly I was trying to look for something easy.
And poaching eggs isn't actually that easy.
So I wanted to learn how to do it.
And I thought it'd be some where I'd be able to present on something, and also learn something myself but-- also I could just do it in my kitchen at home.
So it seemed very accessible.
But yeah, I decided to ditch that because it's a lot harder than people-- or I think people know it's hard to poach an egg so--
OK, so let's watch your pitch and then we can talk about it, and how it changed to your final video.
Oh, woops.
Can everyone see that OK?
Can I leave the back light on?
Let us assume for a second that this cardboard box I hold in front of you is a ship.
Let's say it's a box barge.
It kind of looks like a barge.
Now what I've done for you is drawn the water line.
And what that is is that is the level at which the barge would float on top of the water, assuming that it's going to float.
Now if I take these scissors, and create damage to the bow section of this barge, what do you think would happen?
You'd probably be right in saying that water is going to go through this hole that I created, fill up in the barge, and that the barge is going to sink.
Now let's take that same scenario and do it again.
So you want to damage this bow portion right here.
Now what do you think would happen?
You'd probably say that well, you just did that so it's going to sink.
Water's going to climb in that hole, and it's going to sink the ship.
But you'd be wrong.
What I did is I took these cardboard pieces right here, and subdivided the vessel.
And what that means is that when water comes in through this hole, it's just going to fill up this compartment.
Now what that's going to do, it's going to cause a trim condition in the barge, but it's not going sink.
And people on board are not going to be injured as a result of the vessel sinking.
Now I show you this example as a segway into what naval architects do.
One of the most important features of designing a ship is making sure it's not going to sink, as easy as it sounds.
And the way that we do that it through subdivision
So I showed that because basically the difference between this and the video that you're about to see in terms of content isn't super, super different.
The science, the basic science behind it isn't incredibly different.
So what happened between you making this to you making your final video?
Well I kind of knew the resources that I had at MIT to do a pretty decent job trying to show it.
So that was tough trying to organize that, like getting the labs I had to go to because I don't actually work in these labs.
So it's-- it was tough to schedule that.
But I knew it would be possible.
So I just tried to do it, and had lot of fun int he process so--
I don't think that this is actually a bad video at all.
I think that you'll go on YouTube, and you'll find a ton of videos just like this one where it's someone that just set up their camera back at home, and they're talking a little bit about a physics concept.
Which one do you like better-- this one or the one that you made for the project?
I liked the first one better.
No, the second one's definitely a little bit more in depth, and kind of goes into the science a little bit more so--
What was the hardest part about getting to this final video?
I'd say-- I think it was probably organizing all of it because just the different shots that had to take and everything-- it was pretty tough.
But it was good having Kenneth and Yuliya.
They definitely helped to-- I think my storyboard was kind of all over the place.
But they were like, no, you should do it from this angle and you should try it from this angle.
So I got a lot good stuff that I could edit
Yeah, and you actually said that it was easier for you to be on camera when you were around other people than when you were by yourself
Yeah
Well, let's have a look at your final project
You want me to sit down?
[VIDEO PLAYBACK] [INAUDIBLE] -When we take complex things, and break them into smaller pieces, we find out that we know a lot more about things than we think.
[MUSIC PLAYING] Now let's this box Orca One, and create damage below the waterline which I've indicated right here.
[MUSIC PLAYING] Now let's take Orca One on the water and see what happens.
[MUSIC PLAYING] The Orca One sank due to the weight of the added water.
What if the Orca One contained cargo or oil or even people?
Now let's take Orca Two, and do the same thing.
[MUSIC PLAYING] [BELL RINGING] So we can see that Orca Two did not sink, although it is sitting at an angle towards the bow.
So why didn't Orca Two sink?
As easy as it sounds, this simple demonstration is essential to the design of huge, complex ships-- ships that are responsible for carrying about 90% of all our stuff.
If [INAUDIBLE], how do we [INAUDIBLE] ships carrying our stuff to make into ports safely and not sink?
[MUSIC PLAYING] He we have Orca One and Orca Two from before.
Although Orca One and Orca Two don't engage in international trade, they behave just as 1,000 foot container ship would.
Now let's take a look into Orca One.
We can see that there's nothing in it.
It's just a box.
But if you look at Orca Two, we can see that it's subdivided into these watertight compartments by these tranverse watertight bulkheads.
Now what that means is that if we were to damage a ship right here, water would only flow into this compartment.
It would not go into this one, this one or this one.
That would cause the ship to be angled or trimmed in the water, but it would not cause the ship to sink completely.
We refer to Orca Two as being subdivided.
And we can see subdivision in many of these ships' plans.
It is unclear when subdivision first started being used in ships, but accounts of fifth century Chinese trade ships indicated that water would be able to enter the vessel without sinking.
So let's find out why this happens.
Let's imagine a barge divided into ten equal compartments.
One of them springs a leak from damage.
Since the ship is subdivided, only the first compartment floats and the ship remains afloat protecting both its people and cargo.
Although the added water causes the ship to trim, is still has enough buoyancy to return to port for repairs.
Ships still sink, though.
It's both expensive and impractical to try to design an unsinkable ship, especially when these ships will never see that amount of damage.
That's why, as naval architects, we use computer programs to help us out with subdivision.
Computers make it easy to see certain damage cases in practically no time.
With different software, we can damage certain compartments and see how the ship responds to it.
This gives the naval architect a good idea of what parts to improve on the ship, if any.
So even though ships seem like these intricate, complex things, they're really just based on principles that we all already know.
[APPLAUSE] [END PLAYBACK]
All right, so [INAUDIBLE], I don't know if you saw me mess up, but you're up next.
I accidentally fast forwarded through all the slides.
So I'm also going to show-- that's this just the sound that's there.
You can't see?
Shouldn't have done that.
Here we go.
So Andrea, you can come on up.
So I'm going to show the second pitch that you did.
I guess my question for you is, you knew pretty early on where you were going to write about.
I think your topic stayed pretty much the same, but the way that you were saying it changed a lot over the course of the class.
So, I guess, what was the most important thing that you learned in the past few weeks?
Oh gosh, the most important thing
Or maybe the most practical thing.
Here you go, sorry
OK, two things-- persistence and iteration.
And the persistence was by far the harder thing to really do, because I'd experience a block at something.
My first one was that I was too much in, like, businesswoman presenting information mode, and it wasn't very authentic.
And I didn't think-- I was like, I'm not trying to sell orthodontics to the sixth graders.
That's kind of ridiculous
[INAUDIBLE]
I'm trying to teach them about science.
So it was just a series of blocks that I really just had to work through, much like life
What type of advice would you give anyone who's going to give a talk, who's going to make a video?
Just anyone who's trying to communicate their technical ideas to lay audience.
Because you come from a very interesting background.
So, Andrea is a Sloan fellow, so she has a sense of presentation and communication minus the video.
So, I guess drawing on all these experiences, what type of advice would you give to folks?
Well, I think the first time you ever try to present information, you think you're going to get through a lot more information than you actually will, and that was the first thing that I learned when I taught.
I went with a lesson plan of like 20 things, and I only got through, like, maybe four.
So I think that that's your first lesson is, keep it simple initially
Awesome, do you-- well, let's watch your pitch first.
[VIDEO PLAYBACK] -My name's Andrea [INAUDIBLE], and I'm a Sloan fellow, as I mentioned earlier.
And my pitch idea is actually having to do with something that I actually know something a little bit about, that many middle schoolers also struggle with.
And that is having braces.
I just recently got my braces off, and I also worked for an orthodontics company before I came here to MIT.
So it's something that's both near and dear to my heart, and also I think I've seen a lot of high schoolers and middle schoolers struggle with this whole process.
I know I didn't know a whole lot about it.
So basically, that would be my pitch.
To teach kids about what's happening with their teeth as they're having braces on.
How the teeth move, how osteoblasts and osteoclasts work, and to deposit and dissolve some of their bone, which is actually one of the reasons why it hurts so much.
And also why your mouth gets dry-- all those kinds of scientific aspects of something that these kids really deal with on a daily basis during a very awkward time of their life.
So that's my main pitch.
[END PLAYBACK]
So before we see your video, what's one thing that you're really proud of, and one thing that you want to change if you had more time or more resources?
OK, that's a really hard question.
I'd change the whole thing.
But the one thing that I liked about it is that I tried to keep my sense of humor in the video, and the one thing that I wish I could change is learning more about lighting and editing, and just more of the technical details
Awesome.
Well, let's go ahead and watch it
Do you want me to stay up here?
You can move down.
[VIDEO PLAYBACK] -Eating is one of the great pleasures and necessities of life, and to enjoy everything from energy bars to apples, we rely on one part of our bodies to do an important job, our teeth.
Teeth are the hardest substances in our bodies.
They are harder than our bones, and they're even harder than iron or steel.
So, why doesn't our jaw just crumble under all of those forces?
Between your tooth and your jawbone, there's a specialized piece of tissue called the periodontal ligament, or PDL for short.
The PDL can easily absorb the normal forces that a tooth experiences when we chew, say, an apple, cushioning or protecting our jawbone from our teeth.
Teeth sound like they're already perfectly designed, but sometimes we really need to force them in a certain direction, like with braces.
As the braces slowly force the teeth to move, the PDL is squeezed in one direction and stretched in the other.
To make room, the mechanoreceptors in the PDL trigger cells called osteoclasts that actually come in and dissolve part of jaw to make extra room.
The mechanoreceptors also trigger another kind of cell called and osteoblast, which comes in and builds up part of the jaw bone.
This allows the PDL to get back into its regular, cushioning shape, thus, holding the tooth securely in position.
So if braces use osteoblasts to physically reposition teeth for cosmetic reasons, what if wanted to use them to replace things in our bodies?
Dental implants replace teeth that are damaged or missing to restore chewing function.
Your jaw isn't the only place where these osteoblasts and osteoclasts are altering your bone structure.
In fact, this bony remodeling process is happening throughout your entire body.
And these implants aren't just limited to teeth.
Doctors can replace knees, hips, even spinal discs, and MIT engineers are using the properties of osteoblasts and osteoclasts that are already in our bodies to create a chemical coding for these implants.
Just like in a mouth with braces, this coding helps create natural bone to help lock the implant into place.
Right now, these implants are designed to have the exact same functionality as the parts that they're replacing.
But scientists are already developing implants to improve the performance of our bodies and brains.
So at that point, will we be more machine than human?
[APPLAUSE] [END PLAYBACK]
All right, so Nathan, why don't you come on up.
Every day, the students had to write a daily reflection on our class blog, which is open to anyone.
Any one of you guys can go see it, if you want.
And this is in addition to doing all the assignments that they had to do.
But they would just talk about what the day was like, what they learned, what was challenging-- just any thoughts.
And they could either post pictures, or write things, or make video blogs.
And Nathan, by far, had the most entertaining blogs.
And I'm just going to show you it, and then we talk about it
All right, second day down
Oh wait, Nathan, can you scoot over?
Whoops.
[VIDEO PLAYBACK] -All right, second day down.
No, third-- third day-- so, third day-- all right, so, third day reflection.
A lot happening.
So much that's good, actually, so I guess I'll start with the first thing that pops into mind, which is shooting.
I've just got done finally finishing my trailer pitch thing.
And one thing I guess you always know, but you don't appreciate, is how many tries at shooting something it takes to get something you're satisfied with.
Because, oh my gosh, I don't still really like what I've come out with, but I spent a good hour and a half repeating the same thing over and over again, and I don't know.
I am OK with what came of it, because I know it's still very, very rough, and that's the point of it.
So-- [END PLAYBACK]
So that's part of his day three blog.
So the reason why I said sort of becoming the next Bill Nye is that what I didn't want to end up happening was people thinking that this class was about molding them into some science celebrity that was becoming a next Bill Nye, or becoming the next Neil deGrasse Tyson, or becoming the Hank Green, because what I think is so awesome about what is here in the student body, and what everyone in this class brought, was their own individual personalities.
And I would have hated it if Nathan was trying to be Bill Nye, because he's so much better as Nathan.
And that's why his blogs are so great.
So Nathan, what was-- I know that you had said that it was hard to be on camera, and it was hard to do production, but your blogs are so effortless, and your video blogs-- they're very you.
So what was so hard about doing something when you had a line to say?
You can just clip it to yourself if you want
I think we made it so hard was, if I'm just doing a blog sort of thing, I think of it in a way of, like, I'm just talking to someone.
But when I'm trying to go with the script it becomes a lot more formal in my mind, and I just get nervous
How do feel about how the video that we're about to watch turned out?
How do you feel about your hosting?
What was it like shooting that?
I think there's still a lot that I think could be better about it, but if I look back at the very first pitch video I did, and the second one, and even the rough cut, it's definitely improved.
So, I'll take what I can
What are you most proud of in your final project?
Probably that, I remember when we did the [INAUDIBLE], the first table reads as scripts, mine was, one, a lot different from what it is now, but also very, very long.
And now, it took like about a little over five minutes to read through just on the table, which is, in theory, faster than it would be in a video.
And the whole kill your darlings thing happened, and I ended up with about two and a half minute long video.
So I think that's what I like most
OK, well, let's see what you got.
[VIDEO PLAYBACK] -Why does a fridge start to smell, and where does that icky black liquid come from?
Wouldn't it just be easier if things didn't rot?
If things lasted forever?
Well maybe, but probably not.
The average American household wasts about 25% of its food, and lot of restaurants are even worse.
So if we didn't have decomposition, what would happen to all of the food we throw away?
Food and microbes usually get settled by bacteria, protists, and fungi that allow nutrients in the food to return to the soil and, eventually, other living things.
These decomposers also break down other dead stuff, like trees.
In fact, they're pretty much the only thing that can eat wood.
So even if in a world without rot, where you don't have to worry about your house getting eaten by termites, who align protists in their stomachs, pretty soon forests and landfills would be flooded with a lot of dead stuff.
How do we avoid this problem?
Well, basically in your fridge, on the forest floor, in the dumpster, almost anywhere there are fungi, bacteria, and protists that live entirely by eating dead stuff.
These dead things can be more or less divided into three categories.
Carbohydrates-- sugar and starches-- lipids-- think fats-- and proteins like meats.
All of these are chemically different, so they each get broken down by different enzymes in different ways before being absorbed by decomposers.
For example, proteases break down proteins into amino acids, the cells building blocks.
Lipids rely on lipases, and carbohydrates on things like amylase and cellulases.
So how does a perfectly nice broccoli floret start giving off this foul black liquid?
Fruits and vegetables are almost entirely made of water, so on the most basic level you can see the cells are like extremely complex water balloons.
The exterior of the cell wall is made of cellulose, a complex carbohydrate that gets broken down by enzymes that the smaller cell can get energy from.
The bacteria or fungi uses cellulase to eat the exterior of the cell.
It's like if I were to pop the balloon.
That's the muck you see in your fridge.
What about that stink?
For fruits and vegetables, a lot of times the smell happens after the icky black liquid forms.
Other bacterial that weren't involved in the initial colonization move in, and start to stink everything up.
Meat gets smelly when lipases break down fat in the meat into glycerol and fatty acids, two energy sources, and fatty acids are kind of gross.
And so while your food rotting may smell absolutely terrible, because of it the environment's able to recycle crucial nutrients it needs.
And, well, that's why we have all this.
[APPLAUSE] [END PLAYBACK]
All right, so David, you can come on up.
As I was saying, the students all have to post daily blogs, and after the first day I was really curious to read what everyone thought about the class, how they were feeling, and David's was one of the first ones I read.
I don't know if you can see it, but it says, "Today was the first day in the class of MIT 219.
I felt pretty much overwhelmed with the amount of course work required from the get go, but also pleasantly surprised by the amount of enthusiasm from students and teachers alike in the course.
This will be a day of many firsts, then.
First time making a blog, first time using Tumblr, first time making a video describing something myself.
Previously, we had done in a group setting and it wasn't such a good experience."
So I read that and I was like, oh no, I totally miss-gauged everyone's starting level.
I gave too much to do.
This is going to be an interesting month.
How do you feel now that you not only have mastered Tumblr, but you're uploading videos, you're making videos on your own-- was it as bad of an experience as the first one?
Actually, I felt it was much better.
Really, a lot much better, and I'm very grateful for all the help that we've been getting from all the teaching staff.
And at first I was really-- because the first three times that we made video I was quite demoralized, because the video came out quite badly.
And then when I watched it I was, like, cringing at the videos.
So because of-- but I feel it is because when we did the video shoot most of the time I was doing it by myself, so I couldn't have another person to shoot it against, to let a person tell me what is good, what it bad about a video.
However, in this class, every day we have opinions given to us as to what we can improve, what we can take out, what is better, what to cut.
And so I think through this process we improve our videos much more than we could have alone
And I forgot to mention, so the last three videos that I'm going to show are from SUTD students.
So we got to host a few students who are from the Singapore University of Technology and Design, and it was super interesting having all of your guys' perspectives.
I was thinking before the class started, like, how's this going to work?
Culturally, are people going to have the same sort of digital media literacy?
And I feel like you guys were like, oh yeah, we've seen Slideshow, we've seen Veritasium.
We already know about these YouTube channels.
So I was like, OK, this is going to be totally fine.
But David, you guys are going back to Singapore tomorrow morning.
Do you think that you will take any of the things that we did in class?
Either maybe making more videos, on in your talks, or what types of things were most useful to you?
I don't think that I'll be making more videos because I think it's really quite hard.
And to do it, really you need a lot of people to do it with you.
But one thing I will take back is how to give effective feedback.
Because I actually feel that in this class, many times the feedback was very effective, and it came across very pleasantly.
So I'm very thankful for that
All right, well let's see your final project.
[VIDEO PLAYBACK] [MUSIC PLAYING] -If I went out front right now, dressed like this, you'd probably call me nuts, right?
Because it's so cold outside that I'd probably die of hypothermia.
However, in 2009, Wim Hof managed to run a full marathon in -20 degrees Celsius wearing nothing but shorts.
While most people would probably die, Wim Hof has managed to survive.
So, what makes Wim Hof able to survive while others can't?
The research team [INAUDIBLE] icy bath for almost two hours.
Well, most people who had their core body temp to below 35 degrees causing hypothermia to kick in, Wim Hof's body temperature dropped to nearly 37.4 degrees Celsius, staying slightly warm.
And the funny thing is that the researchers couldn't find exactly what had happened.
Maybe it's a genetic adaptation.
In a separate study, studies have found that individuals with [INAUDIBLE] ancestry have mitochondria that can produce more heat and lactic energy.
Mitochondria is like a mini furnace that burns the sugars from food we eat into chemical and heat energy.
Individuals like get better able to survive the cold.
However, scientists haven't fully studied Wim Hof's genome, so we don't really know.
Wim Hof practices g-tummo as a form of meditation and [INAUDIBLE] which allows him to double his metabolism.
Regardless of condition before, Wim Hof can produce enough heat through his mitochondria with specific breathing and muscle contractions.
They can produce enough heat to keep his body warm.
Well, you might believe that this is not scientific.
However, studies at NUS have shown that diversity increase the core temp of the individual using g-tummo, and the inner prior medication experience.
Now, why could this be so?
Increases in alpha and beta waves are notices, and many have theorized that this has enabled the body to effectively heat the center as far as the and the knees.
We do not fully understand Wim Hof's methods, and while some have tried following his footsteps, it would be unwise to do the same.
And Wim Hof, you don't hear him saying he can control all parts of his body.
Now, if this were true, and scientifically backed, who knows what doors could open?
While science has already audited and I myself don't practice Wim Hof's methods, maybe I better be off only one jacket.
[END PLAYBACK]
Oh wait, sorry, did I cut it off too early?
Actually, I'll show your pitch first, then we can talk about it.
[VIDEO PLAYBACK] -Everyone has pretty much seen a science fiction movie in this day in age, and I'm pretty sure everyone has seen at least one that involves time travelling.
Maybe it's Star Trek, Interstellar, or more recently, Predestination.
And it's such a widely [INAUDIBLE] topic in science fiction, the whole genre of time travelling.
But many times, Hollywood films always have these little, little loopholes in their storyline and their plot lines, and they often use theories and science to explain themselves, which are just like got from machine solutions-- totally not efficient.
However, scientists have actually put forward three main theories when it comes to time travelling.
First theory-- the fixed timeline theory.
Say, for example, you're trying to prevent World War II, and you travel back in time after inventing a time machine.
And after killing the baby that was supposed to be Hitler, you put-- I don't know, some other baby from an orphanage, and put it in the same cot.
However, when you travel back in the future, you realize that the baby that you had replaced it with was the one that turn out to be Hitler, and there is no changing of the past, effectively, or the future, or the present.
It's all in a fixed timeline.
And this particular motions are actually often seen in movies such as Harry Potter.
It was seen in Interstellar as well.
It was also seen in Predestination.
It's a good film.
You have to catch it.
The second theory is the multiverse theory, which states that every time you go back in time and you alter the history-- for example, you step on a butterfly and that butterfly could actually maybe be the prevention of, I don't know, a hurricane that sweeps through Orlando right now-- you spawn a new set of events which happens in a separate universe.
Which, in other words means that you have actually got infinite number of universes after you have all these different decisions and different chain of events occurring And this particular timeline concept has legitimately explored many times as well.
It's been featured in the recent regular Star Trek movies.
It's also been featured in recent Fringe series, which kind of I hate because JJ Abrams right now remind me of something.
And yeah, this multiverse series is actually quite popular because it's the easiest for producers to just show the solution, and then they'll say, oh, we actually spawned off a new timeline.
So that's when you get sloppy and lazy.
The third theory you have is what we call the dynamic timeline theory.
It's also the theory which induces all the paradoxes.
Which, you often hear about these paradoxes, such as the grandfather paradox, where you go back in time and you kill the grandfather, and because you kill the grandfather, you won't exist.
And because you won't exist you can't go back in time and save the grandfather, and that's where the whole paradox comes in.
Who kills who?
God knows.
The third theory we have is called the dynamic timeline theory, which is also where all your grandfather paradoxes come in.
Let's say your grandfather is a really, really evil man-- Hitler-- and you have to go back in time to kill him.
And you go back, you do the deed, and then you realize that, oh no, after I kill him I can't exist.
And because you can't exist, you can't go back in time to kill him.
And because of that, he comes back.
And then after you have to come back, and you have to go back in time again [INAUDIBLE].
That's where the paradox starts.
It's an infinite circle of never ending events, and that's how the grandfather paradox came about to be in the first place.
And I think you may have seen it in Back to the Future.
And I can't really think of any other film right now.
So, this is a brief history of time travel and the three theories associated with it, as well as a quick touch on the grandfather paradox.
[END PLAYBACK]
All right, come on up, Kenneth.
So of the four values-- I don't know if you all remember, but there's spark, there's clarity, there is thoughtfulness, and there's challenge, right?
Kenneth obviously is not lacking in spark whatsoever, which is awesome.
As soon as you see this video-- and this was the very first thing you did-- and you have some video experience, right?
Yeah, I have.
And if I had known this would be shown I would have put on more clothes.
Yeah, I have had some
So the video technique was there, and the enthusiasm for the subject matter was definitely there.
So how did you go from this pitch, which is basically it's own video-- it's kind of like a mini SciShow video, basically, on all the theories of time travel.
How did you go from this to the video we're about to see?
So this was talking about the three main theories of time travel, and this happened right before we saw the sixth graders.
And when I started talking to the six graders, I realized that most of them haven't seen Interstellar or all of these new Star Trek movies.
Funnily enough, they have seen Back to the Future.
So yeah, I realized what really intrigued them about time travel wasn't all these paradoxes, or all these theories, or the timeline.
They were more interested in how is it possible-- is it even possible, and other questions.
So it wasn't-- they were interested in time travel, just not my scope that I initially planned.
So I decided to change it up a bit.
Maybe talk more about the science of time travel itself, which is where I went to at the end
I mean, something to emphasize too-- we talked about on the first day that this space is so hard to work in because what is good is not defined by anyone.
So just because the sixth graders wanted you to talk about the exact science of time travel, and we encourage you to do that, that doesn't necessarily mean that this video is bad, or that people wouldn't really be into a video about the science behind Interstellar.
That's not true at all.
So part of the challenge of working in this space is knowing who to sacrifice-- what audience to sacrifice when you decide to make your video.
What were some of the new or unexpected things that you had learned that you didn't know about production?
Sound is important.
Sound is really, really important.
Yeah, if I had more time I would re-shoot some of the scenes because the sound was just so bad.
Other things-- I realize my writing and my speaking is really, really, very different.
My initial script when we first wrote it out, I think it was about seven-- six to seven minutes long.
My final cut is short of four minutes.
So I think while filming it's really a big difference to how we bring things-- how I brought things across.
It feels a lot more natural to speak just ad lib to the camera
Awesome.
Any final thoughts before we show yours?
Be forgiving
Thanks, Kenneth.
[VIDEO PLAYBACK] -Imagine holding a party and then sending out invitations after the party.
Wait a minute, that doesn't make any sense.
Are my friends going to be able to time travel back just to attend my party?
In 2009, Steven Hawking actually conducted such an experiment to disprove time travel.
But what exactly is time travel, and how does it work?
It actually has a lot to do with speed.
A lot of what we know about time traveling actually comes from Einstein's theory of special and general relativity.
We are actually time travelling right now, but not in the manner that sci-fi films have depicted it.
We are, in fact, time travelling at a pace of one hour per hour.
In other words, every hour I experience, the world around me experiences a singular hour, too.
Now, it may sound simple and trivial, but stay with me.
This is where it gets interesting.
Now, a [INAUDIBLE] of Einstein's theory of special relativity is actually a phenomenon known as time dilation.
Effectively, this means that the faster you travel the slower time passes around you.
It potentially means that I could travel at maybe more than one hour per hour.
This phenomenon, however, is only noticeable at really, really high speeds-- speeds near the speed of light.
This rocket here, not even close.
Say I do have a spaceship now that travels at 90% the speed of light, and I have a pair of newborn twins.
I bring one of them with me on this journey for 10 years.
When I return, one of them will have turned 23 years old, while the one will me will have only tuned 10 years old.
While we have experienced 10 years on the spaceship, 23 years have actually passed on earth.
This amount of time actually increases exponentially as we get closer to the speed of light.
So, we could have potentially travelled forward in time, but what about backwards in time?
That's a loaded issue.
Now, remember our previous draft.
The time that it took [INAUDIBLE] to the speed of light.
Some scientists think maybe the secret to time travel lies on the other side of the line.
In other words, we may have to travel faster than the speed of light just to be able to travel back in time.
Now, when you do travel at speeds near the speed of light, something called relativistic mass comes into play.
In other words, as your speed goes up your mass increases.
As your masses increases, you require more energy to move the same object.
This might mean that we need an infinite amount of energy just to move [INAUDIBLE] beyond the speed of light.
However, the more energy you put into an object, the more likely you are to increase its mass rather than to increase its speed.
Therefore, to gather [INAUDIBLE] quite impossible at this point in time.
So, why are we still so obsessed with time travel?
Time travel does come with it's own set of problems.
Take, for example, my party before.
If I were to send out my invitations after the party, my friends would have come back in time to attend.
And if they did come back in time to attend, I wouldn't have to have sent out my invitations.
And if I didn't send out my invitations they wouldn't be at my party.
Doesn't make any sense at all that they are both at my party and not at my party.
This is what we call the grandfather paradox.
It's one of three time travel theories that we currently have.
Now, so it seems that we may not be able to time travel yet, based on what physics tells us at least.
For now, if you are planning a party, be sure to send out the invites first.
Your friends can't time travel yet.
[APPLAUSE] [END PLAYBACK]
We have one more video.
Joshua, I'm going to show your day three blog.
This was the day after that we had our hosting and script writing workshop.
[VIDEO PLAYBACK] -I thought what really stood out for me was the one of looking at a member of your audience like your family member or a professor behind the camera, so it's not just a camera.
So yeah, but the funny thing is, speaking of this tip, I kind of forgot it and I've been looking at the camera thinking of the camera.
So it's a bit ironic.
I guess many if these things require a lot of practice, and that's kind of like why you do, and get out of the zone of thinking I'm in front of a camera.
Usually, every time you press a camera there's this red button.
Sometimes I think maybe if that red button was eliminated that a lot of things would be different.
Yeah, so I guess one of my questions I would like to ask as well is, what kind of face-- what would be a perfect member of the audience that I would like to put there?
I mean, if I put someone who is very respectable, like my professor, then I guess that the whole tonality in my voice, the whole choice of my vocabulary would try to be more serious.
By the same time, I guess if I put someone like a two-year-old in my imagination in front of the camera, I'll probably not talk in the same tonality.
So I guess, what would an appropriate member of the audience be in front of the camera, or some of-- [END PLAYBACK]
So that's a very hard question.
You can come on up.
And the thing that's hard when you're hosting is that there's a difference between talking to and for your audience, which is a point that George brought up during the workshop.
But being able to actually implement that, as you mentioned, it's really hard.
So what did you think of when you were hosting?
Because I think that the way you talk in this blog is totally appropriate for sixth grader, for your professor.
But being able to distill like who you are as an individual, and having the self-awareness to process, like, this is how I really-- this is who I am at my core.
That's actually a really hard thing to do.
So how are you able to do in your final video?
I guess the most important thing is remembering a script.
So you at least you know what you're going to speak, and then you probably just want to focus on being yourself.
Because at the end of the day, we don't want to be Bill Nye.
Still we want to be ourselves, but we must be clear.
So you have to balance those two objectives.
And this is just me just talking off my head, but I had no objective coming into these reflections.
Yeah the question was interesting, because it was just-- it was the day after-- am I right to say it was the day after the students came in?
And if you saw my first day picture it was extremely embarrassing.
Thank God for not showing it
It's not bad-- it's really not that bad
I was explaining like hashing, and it was like some security concept that's so hazy that--
It's actually-- it was quite a good video.
Very clear and succinct.
I now know that my password is safe on Facebook just as long as I update it every now and then
Yeah, but I've now understood my audience.
And so the point of this video-- and hopefully what I wanted to show was something very accessible, something you use everyday.
And it goes deeper and deeper until it goes to a concept that you're not very familiar with.
But because you started off at the very familiar point, it lead you there.
And so I guess that's the exciting part of the K-12 videos
Awesome.
That was a point Chris [?
Babel ?]
brought up a lot, was taking the familiar and making it unfamiliar, or taking the unfamiliar and making it familiar.
And I think this video that we're about to see does a great job of taking something that's familiar to all of us and showing sort of the unfamiliar side of it.
Any other final thoughts before we show it?
Ugh.
Don't cringe, but I better hide under the table
No, no
OK. [VIDEO PLAYBACK] -Hi.
Do you find it particularly difficult to find a specific item in your house?
Let's say you're looking for a pair of gloves, but you just can't find it and you spend your entire afternoon looking for it.
Well, you've just encountered the same problem that companies like Google or Microsoft encounter every single day, and that's the problem of search.
Just like your house, which stores a thousand different items, Google stores 45 billion index pages of information.
What if every page was a sheet of paper, and we stacked them up real high?
We would create a tower 600 times taller than Mount Everest.
Well, how can Google find a result so quickly when I find it so difficult to find a pair of gloves.
Well, searching on Google is kind of like looking for a person in a big school.
Let's say I'm looking for James in a row of classrooms.
One of the easiest method would be to go to every classroom next to you until you find James.
There is a better method called binary search.
Now, let's say the students were arranges from A to Z, one in each classroom.
We would then go to the middle room first and see if James is there, and if James isn't there we would look at the first letter in the name, and if the letter is before J, we head to the right.
If not, we head to the left.
We would then approach the middle room in the newly sectioned area.
Eventually, we will repeat the process over and over again until we find James.
Now, this is just the first method, but it's at a much, much faster rate.
How much faster will that be?
Well, that depends on the number of students in the school.
Let's say there are 500 students and we are looking for one.
It will take about 80 minutes in the first method, but one and a half minutes with binary search.
But let's say there are 1,000 students in the school.
It will take 160 minutes with the first method, but 1.6 minutes with the second method.
Now, that's a whole lot of difference.
So a name is just a word, but Google searches a combination of words, making it a little bit more complicated.
So just like how we identified the first letter of each alphabet of the name, Google identifies 200 unique factors making the search terms faster.
Well, if you recall, the effectiveness of binary search depends on the pre-arrangement of data, and that's why computer scientists are actively looking for ways to sort, manage, and eventually retrieve data faster and better.
In the same way, the TV remote goes near the TV, the shoes go to the shoe rack, the coats go into the cupboard, and the winter gloves go into the winter jacket.
Aha!
So that's where my gloves are.
[APPLAUSE] [END PLAYBACK]
OK, so like I said, given the incredibly small amount of time and the small amount of resources they had, I think that these videos are pretty impressive.
It's pretty impressive what they were all able to achieve in such a short amount of time.
We didn't even really hit music, which is like an entirely separate, time consuming thing.
But well done everyone.
[APPLAUSE] I wanted to thank all the teaching staff who made this class possible.
Sarah, you're an awesome TA.
Jamie, the co-instructor.
We had George [?
Ziden ?]
come and teach hosting.
We had the guys from Planet Nutshell, Josh and John, come and teach script writing.
Chris [?
Babel ?]
taught a lot of how to make the camera techniques advance your narrative, so it was really a collaborative effort to make this class happen.
Just like hopefully you guys realize how much of a collaborative effort it takes to make a video.
So big thank you to everywhere who made this class possible.
So all of these videos and everyone's stuff is up on our Tumblr if you guys are interested.
I don't know when we're going to offer this class again.
We'll see what happens.
But in the meantime, best wishes to all the students from Singapore who are going back tomorrow, and then the remaining students-- I've talked to most of you guys but waiting to hear back from one person-- we'll be producing their videos for Science Out Loud season three next week, in addition to a few more that we've been developing in parallel to this class.
So maybe when the spring semester rolls out you'll not only be able to see their progress from pitch, to script, to second script, to rough cut, to final project, you'll also be able to see their final product in the videos that we produce for them, which is going to be really exciting.
And that big thanks to Opencourseware too, for helping document this class.
It will be interesting to see that roll out this summer as well.
But feel free to take-- if there's food left, feel free to take it.
There's plenty of salad left, which is unsurprising.
But other than that, thanks so much for coming out.
Feel free to-- I'm going to volunteer the students, if you want to ask them about their experiences, you can.
But other than that, have a great evening, and thanks again, everyone.
How you feeling, Paul?
I'm feeling well, thank you
How does it feel to be finished with this class?
It's bittersweet.
It's a little bit of a relief, but I definitely learned a lot
What was the most unexpected thing that you learned in this class?
I guess how much audio is important to making a video instead of the actual video.
I mean, the video is important too
Do you think that there is anything that you learned in this class that you would apply to stuff outside the video or anything moving forward?
Yeah, definitely.
I learned how to communicate better.
So I think I'd be able to explain things that are a little bit more technical in a simplistic way
What was the most helpful thing that happened in class?
I'd have to say working with the groups and having the feedback from multiple people.
I think that was a good method to hone in on the script and the video.
So all the feedbacks was spot on too
What about the hardest thing?
So maybe the hardest assignment?
I think the hardest part of it was taking a script and trying to translate it into different camera shots and angles and how many times to change an angle and all that stuff.
It's kind of hard to do that, so that took a while
What was it like being on camera?
It was nerve-wracking at first.
I think I was more nervous being on camera by myself, shooting at home
Really?
Yeah
Because I was going to say you sound very natural right now
It might have just been beaten out of me
How would you explain this class to someone who's never taken it before?
It's a-- that's a tough one
Or who should take this class?
I think anyone who wants to-- I wouldn't even say anything to do with videography or anything like that.
It's just if you want to learn how to describe things in a clear way and be-- I don't know.
That's all I'd say.
There's definitely a big technical aspect to it, but I'd say that's kind of just a byproduct of the actual learning
Thank you for your time
You're welcome
Next question
Are you nervous?
Are you excited?
How do you feel about the--
I'm excited to see everyone's videos.
Hey, Kenneth
Hi
So how do you feel with it being over?
A little nervous because I'm having my friends from Singapore come over.
Some of them will be seeing it, and I'm not sure how they really would like it or not like it.
So yeah, I'm a little nervous
How was this class different from the ones that you've taken at SUTD?
I think it's a lot more hands-on, definitely.
We have a similar course back in SUTD, where it's called Film Studies, but I think that's a more film-related module.
This feels a lot more like a good mix between education and entertainment.
It's really, really nice to see all this stuff that you always wanted to do or you always see around.
I see so many people making YouTube videos and you're kind of learning the theory behind it all.
And usually, it was just, oh, let's just set up a camera and just put it on, but there is actually a lot more going into it.
And yeah, probably that's what's the difference between all the good videos you see and some people just teaching how to derive some equations on a blackboard, I guess
What is the most useful thing that you learned in the class?
I think the small class size really, really helps and the fact that we just help each other.
We give each other feedback.
The conventional way would just be turn in your ideas and one teacher or maybe two teachers would just grade it.
But this, you get the whole class's feedback, which is a lot more useful
Do you think it would work if it were more than ten people
I think it might be a bit hard because things and people may not be to gel that well to be able to be comfortable to give too much comments
OK.
Thank you very much
Thank you.
I'm doing very well.
I'm very excited
Are you excited, nervous?
Yeah, I'm very excited.
It's my first red carpet premier, which is what I think it is
What was the hardest thing about this class?
The beginning was difficult because there was a lot of work you had to do, and for me it kind of made me realize what I know and don't know.
So just getting rid of all the misconceptions I had about filmmaking, script writing, and all else was difficult
Do you feel like you want to keep doing this, or are you going to try to take the things in the class and apply it to non-video stuff?
I'm definitely interested in educational resources.
So video, I think, is one of the best mediums to convey excitement about science and mathematics.
So this is definitely something that will be important in life
[INAUDIBLE].
[LAUGHTER] How would you describe this class to someone who's never taken it before?
You have a month to learn how to make an educational video, and then you get to write, produce, direct, and create it all
Any words of advice for people who have never done it before?
Don't think it's going to be easy.
Just always think that it's going to be harder than you expected.
And put work into it
Thank you very much
Thank you.
INTERVIEWER 1: You recorded your videos together, sort of are partners
Yeah.
INTERVIEWER 1: Are you excited, nervous?
You're excited.
I can tell.
He was all bouncy when he came in.
INTERVIEWER 1: Really?
What was the hardest part about the class?
Or maybe about making the projects
So I'll go first.
For me, it was a lot of psychological barriers.
First of all, being on camera.
And I think you have shared that as well.
And then just channeling the rusty, disused, artistic part of my brain, which really only comes to the fore when I'm under extreme pressure and I know I have 24 hours to get something done.
But that's when I felt the most creative.
It had to be a lot of pressure though.
INTERVIEWER 1: Nathan, how did it feel doing the second cut this time?
The second cut I thought I had a better chance to fix the little things and work on putting the finishing touches, I guess.
INTERVIEWER 1: Do you feel different-- better, worse-- about being in front of the camera?
Same?
Probably the same.
INTERVIEWER 1: Probably the same
I don't think it got any worse, at the very least.
INTERVIEWER 1: That's good.
INTERVIEWER 2: How are you feeling right now?
INTERVIEWER 1: Yeah, like you guys are being super natural right now, totally on camera
I have my nervous leg [LAUGHTER] shake.
INTERVIEWER 1: But, I don't know.
I feel like a lot of people were selling themselves short, thinking that they were not good on camera.
But you guys were on camera the entire time the class was going on.
And everyone's very compelling.
Everyone's very natural.
What piece of advice would you give to people who would be interested in making educational videos?
I guess it would be to try not to worry too much about the actual part where you're on camera because it's painful, but if you keep doing it enough times, eventually you'll get a cut where you say something all right.
INTERVIEWER 1: So this class was super high-paced, very condensed.
Do you think it would work as a semester-long class?
Or do you prefer it as sort of this [INAUDIBLE]?
Oh, I would love it as a semester-long class, in particular because learning editing, more advanced editing techniques, learning more about animation, which I think for an educational video is really critical to convey the ideas that you want to.
And I know that's one thing I didn't really have time to do.
And that made me sad.
Oh well.
INTERVIEWER 1: How about you, Nathan?
More or less the same feeling.
I feel like there's things that could be expanded on.
But I also do like the idea that you just have three weeks, and at the end you've got something out of it, which is also cool.
Really good really good.
Yeah.
Very, very excited.
I have like three most useful things, so I don't know what that counts as.
But I think you have first thing you have preparation.
I learned a lot about the preparation process in the video, which I previously didn't actually consider.
I used to think of people with their shaky cameras.
Then actually have an idea of what they're doing.
But most of the time they probably had a storyboard.
And it's not like by accident.
And then eventually it becomes the idea of iteration.
So you have to keep trying different scenes.
Many, many takes.
It never came to my mind.
And lastly, you have to have the passion for this.
Because you would do a lot of takes.
You will be pretty sleepy after a while, but you love what you've done.
And I'm kind of excited to show what I have.
I guess the hardest part is there's so much content, and there's a lot of technical stuff there.
I wish I knew more about the Creative Suite and the software.
So I guess I was limited in that department.
But other than that, I think the preparation part gives me the background.
It helps me to do a lot of different things.
Yeah.
Thank you.
OK.
So this is the day one confessional.
Lecture ended a couple hours ago.
Just been trying to do the homework assignments along with the students.
Holy cow.
It is really exhausting to teach.
And three hours goes by way faster when you're talking than it does when you are listening.
I I think overall, it could have definitely been a lot more catastrophic than it was, so that's always good.
I think the hardest thing was just getting people to be engaged in discussion.
It was just so quiet.
Overall, it was just hard to get them engaged in discussion.
Jamie was awesome, but it's funny because she's taught middle school, so she clearly has a very good handle on facilitating these kind of things.
Once she got people to talk, some of the stuff that they were saying was great-- the conversations that they were having about their experiences with online video.
I'm really glad that she prodded them a little bit more, whereas I just wanted to give up after no one said anything.
Yeah, the time goes by really fast.
I was definitely pressed for time, which shouldn't have surprised me but still did.
And just making sure to leave breathing room for students.
I mean, I did try to pace out the video stuff so that they'd have sort of a breather from hearing someone just talk to them for three hours.
But I do feel like it was just a super dense lecture today.
I think passing it off with Chris at the end helped a lot, but I would have liked to have more time to just have students get up and use the cameras and maybe work on the STEM videos during class.
I know a teacher who teaches high school sociology, and his philosophy is just stay one day ahead on this syllabus from the class.
And I've been feeling a little bit like that in terms of the materials, but maybe it's just the adrenaline from day one.
But I do feel like it was OK. A little nervous for tomorrow with all the sixth graders.
A lot nervous about tomorrow.
But I've already met the kids and they're good kids.
It's just a matter of trying to balance everything I need to do for work aside from lecturing.
Now I get it.
Now I get why professors don't always want to prioritize lecturing.
It's not that they don't want to.
It's that there just aren't enough hours in the day.
I'm going to sleep like a rock tonight.
The best part about today I would say is how smoothly everything went with the sixth graders.
I was stressed about it.
They actually had pretty constructive feedback, so I'm happy about how that went.
The worst part about today, I feel like people feel lost.
They kind of had a deer in the headlights look on their faces sometimes, because we're really talking at a more abstract conceptual level, and they haven't had a chance to put stuff into practice yet.
And I know that'll change tonight, and I know that the workshops and stuff will help tomorrow and the rest of this week, but at some point, I just felt like I was saying words that weren't super useful to them, and I couldn't stop.
I should've just shut up, but I didn't.
Reconfiguring our furniture arrangement in this U shape definitely made the class feel more intimate, and I think that maybe that contributed to them talking a lot more compared to yesterday.
I mean, there are a lot of compounding variables with that, but they did seem to talk a lot more.
If I could do anything different today-- well, maybe leaving more time for scripting.
I really was going to rely on tomorrow to be the big scripting day, and I realize now that that probably wasn't the greatest idea because people don't know how to script.
And of course they don't, because who does if you haven't ever done it before.
I'm excited about tomorrow because it's a more calm day.
It's more relaxed.
It's more informal.
It's more normal.
So today was the lecture on scripting and hosting.
It's the end of day three.
This is probably the lecture that I was the most prepared for in terms of practice ahead of time, ahead of this class.
Because it's kind of what George and I do for most of the production process that happens for Science Out Loud.
So whenever we work with our students in the individual seasons, this is basically what we do with them, just a little bit more formalized and for a longer period of time.
It is so hard to teach good hosting because it's one of those things that you can sit and read about.
And you can watch a bunch of case studies.
And it just doesn't matter until you try to adopt it yourself.
I had planned all this material where it would be like, there were all these videos of various hosts.
And it's going to be sort of a similar exercise to day one, to all those YouTube videos where it would go through and they would write some of the qualities that they liked and try to see what they wanted to take on for themselves.
But we just totally scrapped it.
And I think, honestly, it was much better that we did.
The hosting thing is hard in helping the students really be comfortable with their own voice and their own persona on screen.
It's really important.
So I don't want to distract from that at all.
One of the students wanted to minimize her presence on screen.
I don't want the students here to sell themselves short on the power that they have as a practitioner of some of the science and engineering to default to a type of video that a lot of other people can make.
These students are not the best hosts.
Finding like a Neil deGrasse Tyson at MIT, you can't bank on that.
And I don't mean that in a mean way.
Perhaps I should rephrase this.
Maybe they can cut this out.
It is an imprudent thing to assume that we're going to find the next Bill Nye at MIT.
We can't expect the students to become like Hank Green and to make the next SciShow.
That's an unfair expectation to have.
But the value proposition that they offer here, which I think is so powerful, which I think is the most important thing, that someone like Michael Stevens on Vsauce can't do, and SciShow can't do, is that they are the scientists.
They are the people who worked in the industry.
They are the actual Naval engineers.
That is something, for me, personally is so important to highlight.
So to make themselves evident, I think, is really important.
To make themselves the hosts, to make themselves visible, to make their persona and their actual personal stories part of the video, I think, is important.
Whether you're in front of the camera, or you're on a stage, or you're on a podium giving a talk, the habit of falling back into this monotonous robotic or perceived persona that you think you have to take on totally happens all the time.
So I think it's a relevant lesson that people can take with them after this class is over.
I'm definitely rambling at this point.
Maybe it's because George did most of the talking today.
But I think it went OK.
Today is the end of day four, it's Thursday.
Today is the day that I'm feeling the class sort of catching up to my general well-being.
So today's the day that has made me empathize a lot with professors.
And it's made me think of how the entire infrastructure of teaching roles was created in the first place.
It actually doesn't seem very intuitive to me to pair a researcher and an instructor and put it together in the same role, because they're really different people.
I'm really sensing a lot of trepidation and hesitation in just trying creative ideas, which I completely relate to.
But it's just so incredibly striking to me how everyone has this mentality-- and not just students in this class, but every single student I've worked with for Science Out Loud-- that you have to measure 20, 30 times before you cut once, and that as long as you measure 20, 30 times, you only have to cut once.
But the whole paradigm for this production process is almost the opposite.
You measure like three or four times, and you cut 20 times.
You come up with like five ideas and you write 20 scripts.
And there's just this crippling hesitation, this overwhelming paralysis to just attempt to write a script that is so pervasive through every single student I've worked with, including myself.
I'm also realizing that the thing that students need is concrete examples, that people can recognize big concepts and not be able to implement them themselves successfully.
It takes a lot to become self aware of your work, essentially.
But I think today was good because it gave the students a chance to look at a script that was written and storyboard it and sort of unpack what about it worked and what didn't, and sort of tie the bigger concepts that we've been talking about into a concrete example.
The next step that they need to make is being able to implement that into their own work.
So we have the concepts that we've established, we give examples of what sort of fit into those concepts and what don't, and then there needs to be a third level of using their work specifically, to explain if their work fits into those concepts or not.
I feel like each day goes by so quickly, but it's like a massive amount of time that's passed.
Today is the end of week one.
Day five is down.
This class is very front-loaded.
So I think the students feel it, and I definitely felt it yesterday.
Yesterday was probably the peak of just the exhaustion of the class, and getting used to doing that in addition to my day job.
I was frustrated because I wanted the students to see what I was seeing, and they weren't.
And I felt like it was because of an inability to clearly articulate what I was trying to get them to see.
Today was so different.
We started class with having them pitch the idea, or the point, of their videos.
And there's this nuance between an idea of the video, the point of the video, and what a video does.
And it wasn't until this morning that I was-- I finally was able to articulate in my head what was going on with the students, which was the inability to distinguish a point from an action.
So a lot of them were describing what their videos did, and using that description as the point of the video.
And I feel like we finally all make a breakthrough.
Today was when it sort of became real to everyone because they got to take their cameras and actually go shoot stuff.
And it was just incredibly gratifying to overhear them shoot, and hear them offer each other the same pieces of advice I would.
And See them take creative risks, and experiment with the equipment.
And to also see they really are a good class group.
They're just a really good group of students I front-loaded the class, and I'm realizing that it really was a lot work for them.
I wanted to equip the students as best I could with the skills they needed to have so that the rest of the month could be more about them trying stuff on their own.
And I don't know how I would re-structure that.
It's something that I have to think about a lot.
I thought about that a lot coming into the class, but it was still a lot of work for them this week.
In short, today was excellent.
I saw smiles on people's faces, and witnessed them integrating things that we had taught them this week.
Just wrapped up the end of day six.
Today was the table read where everyone showcased their scripts for the first time.
Table reads are always so much more awkward then I would anticipate.
It's just a lot of dead space and silence.
I'm really feeling the time crunch with this class now.
The panic seems to sort of be setting into the students' mind.
At the end of class, they were like, our stuff is due next Thursday, basically.
I do think that their scripts are in much, much better form than they were last week.
I guess I wonder if it's more valuable for them for me to just give them the feedback online, which is what I did for half of them, versus talk during these group table reads.
But I also think it's really important for them to be able to read the scripts aloud in front of the class, and to get some of the input from their classmates.
I mean, Andrea brought up really good points to PJ-- or Paul-- at the end.
And that didn't happen when I was just sending him comments through Google Docs.
There's a fine line between constructive criticism and just doing the work for them.
I was super tempted over the weekend to just say, change this word to this, sort of be their editor.
And I had to fight that, because I think it's more valuable for them to sort of discover that on their own.
But the problem is that takes so much time, and it's time that we don't have.
It's been a challenge to balance that so far, and I think it will continue to be a challenge.
Adjusting assignments on the fly, and adjusting lecture materials on the fly, and taking stuff out-- a lot of stuff out, actually-- has been not the easiest thing to do.
I had a very interesting discussion with [?
Sari ?]
after class about if this class could really be feasible in a semester, in the style of the semester.
I can't really break up a lecture into 45-minute chunks.
Josh's storyboarding lecture wouldn't have made any sense in only 45 minutes.
Maybe it could have, but it would've been so different.
And it's really inefficient to just shoot for 45 minutes at a time.
All in all I feel antsy.
I feel very antsy.
I feel antsy for the students.
It's going to be a sprint to the end.
We are approaching the end of day eight.
Students are still working.
Today is the first day that they're off on their own.
I didn't record a confessional last night, because I sort of forgot to.
But yesterday was the last official day of lecture.
It was on post-production.
That's been a challenge to balance context, and sort of the more bigger picture digital literacy, understanding the medium type things with just the practical things that they need to know to do their project.
And given a small amount of time, there's always been a compromise of something.
Yesterday was great.
I think when I heard Josh read his script, it was one of my favorite moments in class.
First of all, his idea had come such a long way.
And his delivery was very good, too, and he said that he had watched Andrea from the day before, and tried to pick up tips from her.
I'm happy with this community of students that have come together for this class.
And I think, in part, a lot of that has happened because it's small enough.
Seeing them pair off to work on their projects, they have really helped each other a lot.
And certain personalities have come through that were super quiet in class.
Jamie and I were very deliberate with how we grouped people up for their final projects.
It's been hard to balance the practical skills with context.
I sent out this midpoint survey to the students yesterday, and someone had said it was really, really packed the first week, and all of a sudden, the pressure's been released.
If I could find a better way to spread out all that stuff, I think it'd be more ideal.
Oh, the other thing is, thinking about how this could be a scalable experience, I can't imagine this class going over ten people.
And I think the thing that's been most valuable is individual feedback.
And I might have mentioned this before, but you can easily teach best practices.
You can easily talk about techniques.
You can even show the examples.
You can go from concept to example, and people can acknowledge that and people can critique other peoples' work.
But then transferring it to the final layer of being self-aware enough and self-reflective enough to view your work with that same critical eye, and, in some sense, like being able to cultivate taste among people is really hard.
I don't know if I'm really doing it.
And who am I to say that I have the best taste in anything?
A lot of the times questions are posed, opening with, Is it OK?
Is it OK if I do this?
Is it OK if I change this?
Is it OK if I go to this location?
I know that what they're trying to ask is, do you recommend this?
But it's just, it's striking the way that's so ubiquitously posed.
I don't want the mentality to be that I am sort of the end-all be-all.
I think, more than anything, I'd want the self-awareness though.
If someone made a terrible video, but they could say, I think this is bad, and I think maybe this is why, I think that that's ultimately where we need to be.
More self-awareness in science communication.
End of day 11.
Today was the rough cut screening of everyone's projects.
I think that this class really is more about pre-production and everything.
And I think that's the thing that gets a little overlooked by most people, at least who are trying to come from a technical side-- that they view production as when the camera comes on, that you just show up and sit down and do your interview, and you leave.
And they overlook how much work it takes leading up to that point and of course, coming afterwards.
And what I was most pleased about today was that people's scripts in these videos had really come a long way.
It's really funny, because people-- and I know I've said this before, but it's easy to look at something that someone else has made and point out all the flaws.
And it's so hard to be able to learn from that and implement that in your own work.
Some of them are excellent critics, and you see their work, and they're making the same mistakes that they're pointing out with other people.
That's partially because it's very hard to execute some of these things well, and it's also hard to see that you're not doing it.
The other thing is when I first started planning this class, I was so obsessed with every day being perfect.
And I would spend hours and hours just on the presentation.
I think the aesthetic of the presentation was really important to me-- how I would say things, how I'd lay them out, that there would be sort of this branding in the class.
And today, I feel like things came together at the last minute.
And it was OK.
It turned out fine.
For the rough cut editing we had spent, I guess it's been months now, trying to get this video annotation software to work.
We were emailing back and forth with the developers hours before the class started, and at the last minute we were creating Google spreadsheets to use those instead.
And it worked out fine.
Doing what's best for the class and not using technology that exists just for the sake of using them is something I'm learning to do.
I caught myself in the editing spreadsheets, I would say, try this if you have time.
And I would almost feel bad bringing stuff up of what they should reshoot, but then I think like, this is part of the experience.
This is why we gave them two whole extra days to do reshoots is to reshoot things.
And I shouldn't feel bad about making them do more work.
But it's a fine line that I'm trying to toe really well, because I do sympathize with the demands of being a student here.
So I'm not really sure if I'm doing that super well, and I'm basically relying on their individual feedback to get a sense of that.
We were talking about like, what is digital media literacy exactly, because we talked about this a lot.
And I think what it really is, is not so much being proficient in scripting and filming and editing, it's just the self awareness to know what you're good at, and what you're not good at.
And just the appreciation for what goes into things, and the knowledge of who to tap into to help you create a project or help you communicate something.
Go ahead
I just have a question because I'm not plugged into the MIT community, or even into the scientific community, for that matter.
But is there a culture of sort of individuality wherein this notion that basically, my intellectual work is to be done sort of in a silo, versus the kind of, like, I know I'm not good at this, so I need to build a team where we can build
At MIT, it's pretty interdisciplinary.
What I would say, is not siloed, but there's an ingrained culture of is this idea that real science-- and I'm doing it now by saying real science-- that science and engineering is work, and communication is not work
Oh, I see
Or even that it a separate thing
Yeah
Yeah.
CHRIS BOEBELThat's a pretty good way of putting it
So that's-- it's not that people are like siloed within themselves, it's that communicators don't have street cred with scientists, period
And that part will just happen, it will take care of itself.
I'm not going to worry about it too much
I see, so that sounds like the real meat of what you guys are trying to work on, in a way
Yeah, I guess.
I mean it's interesting because I gave a survey to the students at the end of the class, and they were all like, I probably wouldn't make another video again.
A couple of them said they would, but most of them are like, I probably won't make a video, but, I have so much more appreciation for what goes into it.
Now I can't watch a movie without thinking about how much, you know, how much work it took to just light the scene.
So even just having the appreciation for what goes into it, I think, is a big step.
But, again, with the whole literacy thing, just the awareness of how important these skills are.
Because, like you said, I mean, it's just crazy that we think of them as separate things.
There's science and there's communication, and you're trained separately on these things, if you happen to be a science communicator
One of things that I think is really interesting about media is that video or media, films, they're really interesting in that when they're well made, they're so accessible.
They're so immediately graspable by those of us in the last 100 years who's really literate at unpacking video and understanding the medium.
They're so kind of intuitively open that people who don't have any appreciation or awareness of how they're made assume that there isn't any kind of effort behind them.
That they just kind of happen.
And I can remember as a kid kind of thinking that.
Never really thinking about movies and how they were made until, I was probably, God, in college.
I just kind of assumed that they just kind of happened out there
Our clients think you just hit a button
Exactly.
Yeah, exactly.
And that's because if things are well made, it's like, didn't that just always exist that way?
You mean somebody actually had to create that?
It seems so natural.
And I think that one of the things, a key of media literacy, is breaking that myth
Right.
Or, gasp, it would cost money to hire people
And take time
You would have to hire professionals to do it that are trained and pay them
So the biggest thing in teaching this literacy, though, I think, is making them go through the experience themselves.
Because it's one thing if you watch like a series of lectures.
Like it's one thing if you watch this on OpenCourseWare, and you can take the notes on, like, this is good framing if you use the thirds rule.
Write a script like you would say it.
We go through all these sort of tangible, take home points.
These sort of plastic elements, like you mentioned.
But I don't think you'll have the same, you won't draw the same value from it as if you actually go through and try to do the projects that these students did.
And that's a big issue, I think, especially in media teaching.
That it's easy to first establish a sort of theoretical principles.
Like, this is sort of what makes a good story.
And then it's another thing to actually give hard, tangible examples to students.
Like, look at this script.
They're able to point out issues in the scripts.
They were able to critique them, but then it's a whole other step for them to be able to integrate it themselves into a script.
So you'd have a student who understood the principles, they took the notes, they could reiterate everything you said to them.
They were able to look at the examples you gave them and say, oh you know, this is unclear here, this is awkward wording.
And then they would write the script, and you'd be like, there's this disconnect to the third step.
And it wasn't until they tried it, and then we were able to critique them directly that they were able to come to this final self awareness conclusion.
Hi.
I just came from the icy cold weather of Boston where I felt like my toes were going to freeze and drop off.
And the question I want to ask you is, what makes the difference between one person who can withstand the cold very well and one person who cannot?
And what is the difference between these two people?
Sorry, part of my speaking fast is because it's really cold outside.
So the question is, how can one person withstand the cold really well and go out jogging in the open without wearing much clothing?
They can just wear shorts and shoes and just go running.
Whereas people like me have to wear two, three, layers, and they still feel very cold.
We still feel like our appendages are going to drop off.
So what makes the difference?
Could it be that some are more fit than others?
Or some have better tolerance to the cold?
Or could it be some have a high metabolism that they can burn more energy faster so that they can get heat into their bodies faster?
Or is it that some people have shorter arms and shorter legs so they lose heat less and therefore can withstand the cold better?
So what answers all these questions?
Watch my video to see.
If I went out for a run now dressed like this, you would probably call me nuts, right?
Because it's so cold outside that I'd probably dye of hypothermia.
However, in 2009 Wim Hof managed to run a full marathon in minus 30 degrees Celsius wearing nothing but shorts.
While most people would probably die, Wim Hof has been able to survive.
So, what makes Wim Hof able to survive where others can't?
To research him, researchers in Netherlands subject him to icy bath for almost two hours.
While most people would have their core body temperature drop to below 35 degrees causing hypothermia to kick in.
Wim Hof's Body temperature drop to merely 37.4 degrees Celsius, staying surprisingly warm.
And the funny thing is that the researchers couldn't find exactly what had happened.
Maybe it's his genetic ancestry that saves him.
In a separate study, scientists have found that individuals with [?
Cocam ?]
ancestry have mitochondria that can produce more heat and less chemical energy.
Mitochondria is like a mini furnace that burns the sugars from the food we eat into chemical and heat energy.
Individuals like this would have been better able to survive the cold.
However, scientists haven't fully studied Wim Hof's genome, so we don't really know.
Wim Hof practices g-tummo, a Tibetan form of meditation and breathing which allows him to double his metabolism.
Remember those mitochondria I mentioned before?
Wim Hof can produce enough heat through his mitochondria with specific breathing and muscle contractions that can produce enough heat to keep his body warm.
Well, you might believe that this is nonscientific.
However, studies at NUS have shown these participants increased their core body temperature using g-tummo, and they had no prior meditation experience.
Now, why could this be so?
Increases in alpha and beta waves are noticed, and many have theorized that this has enabled the body to effectively heat the center as well as distribute heat to the extremities.
We do not fully understand Wim Hof's methods.
And while some have tried to follow in his footsteps, it would be unwise to do the same.
And Wim Hof even claims that he can control other parts of his body.
Now if this is true, and scientifically backed, who knows what doors it could open?
While science hasn't explained it all yet, and I myself don't buy these Wim Hof's methods.
Maybe I'd better be off donning more jackets.
When we think of snot, we usually think of the stuff that oozes out of us when we're sick, right?
[SNEEZE] It's just more gross bodily fluid that we seem to excrete for whatever reason.
But snot is more than just the stuff that hangs out of the noses of most two-year-olds, and some 20-year-olds.
It's filled with all of these cells that do everything from fighting infection to keeping your esophagus from ripping up every time you try to eat something.
[CHOKING] And that "junk" actually can keep stuff out of your body.
And then it knows what to keep out, what to let in, and when to do it all.
What?
That is so crazy.
Snot and mucus is so important that your body makes a gallon of it a day, which is disgusting.
But then I think about it more, and I actually would rather keep all that around.
All right-- so end of day three.
This was probably maybe the more challenging lecture for you guys, which is funny because it's probably the one that I was most comfortable with coming into the class because George and I had done it before.
That's basically what happens during pre-production of "Science Out Loud" is we work with the students one-on-one honing their scripts and their ideas.
But I do recognize that it's very uncomfortable from the other end and very challenging to think outside of your perspective, to anticipate an audience, to learn how to host-- hosting is really hard and maybe you have a better sense of it now.
But I'm going to try to figure out how to get some of the raw footage from Open CourseWare up today.
But it's really stunning to see the difference when you guys are just talking during class, which is inherently very engaging.
For anyone who thinks that they're not good on camera, I challenge you to watch some of the footage from Open CourseWare because that's just not sure.
You guys are very engaging on camera when you're not aware that you're on camera.
But the difference between that and when you're just reading out loud-- I don't know what it is about just human nature that you sort of inhabit this completely different persona and completely different talking voice.
I do hope that you found today productive, even if it was a lot of getting rid of stuff that you had written last night or a lot of stuff that you had made beforehand.
I do believe that this is collectively a process that propels forward.
And its very normal.
For George's draft, we literally went back and forth 10, 15 times-- same for mine.
And it's what makes it fun.
I feel really good about everyone's ideas for their episodes, at least.
It's really just a matter at this point of figuring out the best way to structure things and the best way to phrase and delivers things.
And that's really what's going to make the big difference in the end but when it comes to the core ideas and the topics, they're all quite fascinating and they're all videos that I would watch, that George would watch.
So that's a really good place to be.
I can see that people are being challenged outside of their traditional modes of thinking or maybe their comfort zones, which I think is a really great thing.
I also totally empathize with it and know how uncomfortable it can feel sometimes.
But I'm really happy with where we are right now.
There's definitely a lot of room to grow from here but that's sort of the point.
So hopefully, we're supporting you enough on this.
Let us know if there's anything we can do.
I loved the Hourglass piece that George showed today just because it's funny to see that a person like Hourglass-- who's really great on audio, a really great host-- even he struggles with delivery sometimes.
In taking his sort of jumbled mess of words and hearing him deliver the point of it and seeing the big difference between the two, I hope that you guys see that that's sort of what we're trying to get you all to the point, of being able to distill this cloud of ideas into a message that is very concrete and very compelling.
All right, end of day 11, rough cut screening.
The most important thing about this class, and I know I've said before, but I'm going to repeat it, is not so much what you achieve in your final product as it is the ability to be able to look at it and identify its strengths and its shortcomings, and to have the self awareness to be able to look upon your work, and really be able to say, this is where I could have been better and this is what I did well.
And I think that I see those elements of being able to critique and edit and knowing what makes a quote-on-quote good video when I look at your Google Docs and the feedback that you give to your peers.
All of that looked excellent, and a lot of you said the same things I was going to say myself.
So that is always really awesome.
See, the scripts are excellent.
Pre-production is something that I think is super overlooked by lot of people to try to make technical videos.
They don't really see how much work goes into the video before the actual cameras come out.
And so to the pre-production process, I know, was very intensive, but I think it really paid off, because these scripts are really, really good.
I know that after you put together a rough cut, the last thing you want to do is re-shoot things and redo things or add more things, and it can seem discouraging.
And I myself felt really guilty while I was giving feedback on some of your videos, because the last thing I want to do, after this super intense two, three week experience, is to ask you guys, hey, shoot a little bit more or try redoing things.
And at the end of the day, it's your call what you want to do, as long as you are able to justify those decisions.
I understand that people are working under limited time and limited resources, but at same time, I don't want you to be discouraged just because you have to redo things, because as I said, cutting more than once, literally, is unavoidable.
It really is an essential part of the creative process and discovering what path you want your video to take, and it really is what is crucial to move people forward.
So, second cuts are just-- they're part of the process to everyone, to professionals.
So I wouldn't be so scared or apprehensive to do them.
Don't be discouraged if you have to do them.
We're around for the rest of the week if you have any questions.
Until then, I'm super excited to see where these go.
Hi, I'm Joshua and have you wondered, can my Facebook password be stolen from Facebook's databases?
For example, if a hacker were to come into the Facebook network and get into the administrator's account, would he be able to steal my passwords, your passwords, everyone's passwords?
Mua-ha-ha-ha.
Would that be the end of the world?
Well, it turns out, no, because companies like Facebook or Google store your passwords not as passwords.
In fact, this comes to one of the most important concepts in computer security and that is the concept of hashing.
Well, so if companies like Facebook and Google don't store your information like passwords as passwords, what do they store in MS?
Well, turns out they store in MS things called "hashers."
And hashers are kind of like a little snapshot of the actual password.
So let's say this coffee is the password.
So I just take a little snapshot and all I see is a 2D image of the password.
So a hash is simply a representation of the password which proves that you know what the password is.
But you'll probably tell me, well, I can understand the idea of a photo but what about it in computer science?
What about it in actual programming?
How does it actually look?
Well, the formal definition of a hash is kind of like what you understand as a one-way mathematical function.
This is why we need to understand what a mathematical function is in the first place.
And so a mathematical function is a relationship between the inputs and the outputs.
For example, f x equals to 2x and if x is 2, then 2x would be 4.
It's just a mathematical relationship from the left side to the right side.
But at the same time, you'll probably be wondering, well, that is not one-way.
So what "one-way" means-- well, "one-way" means that the input on the left-hand side cannot well determine what is on the right-hand side.
But knowing the results on the right-hand side doesn't mean you know what the actual inputs were.
For this case, if you see 2x and you see 4, you know for sure, oh, it's got to be 2 as an input.
So how do computers enable a one-way function?
In other words, knowing what the inputs are to determine the outputs but just having the outputs would not be enough to determine the inputs?
Well, it comes to the concept of the modulus operator.
So what's a modulus operator?
A modulus operator is an operator the returns the remainder of a division.
So an operator is just like a plus or a minus.
You have, for example, 7 divided by 3.
It would give you the answer of 2 with the remainder of 1.
And so 7 modulus 3 would just be the answer 1, which happens to be just the remainder of a division operator.
Well, then you might say, how is that useful to become a hash function?
Well, notice that 7 is not the only number that you can modulus by 3 to give you 1.
You could use 4.
You could use 1.
In fact, there are many numbers that result in the modulus function returning 1.
So just by telling you the answer is 1, you probably don't know what the input is.
And hence, you can determine what the outputs are with the input but you cannot determine what the inputs are just with the output.
And so it seems that, is your password safe?
Well, it's quite safe because most of your passwords are stored in hashers.
And all the company just needs to do is to take your password whenever you log in and convert it into that same snapshot and to compare the snapshots to see whether the snapshots are equivalent.
If they are, you logged in.
If they're not, too bad for you.
So in that way, you are able to be secure.
At the same time, if any hacker steals any of the hashers, they cannot determine what the passwords are in the first place.
So in that sense, your passwords are safe.
But however, in the same sense, a hacker could still brute force his way through trying any number of passwords and will eventually find your password some day.
So point or story-- is your password safe?
Yes and not so because eventually, someday, someone will be able, if they try really hard, to find your password.
So stay safe and change your passwords regularly.
But other than that, your password is still safe with any company you put it with.
Thank you.
Hi.
Do you know that Google stores 45 billion web pages on the internet?
And if every single page was a sheet of paper and we stacked them up real high, that would be a tower 610 times taller than Mount Everest?
Whoa.
And if that's not mind blowing enough, Google is able to give you your search results in a split second.
So how can Google do that?
I mean, that's like finding a needle in a haystack.
And so what I was thinking of is showing the analogy of a search algorithm, like this, through a hotel.
So imagine I have a friend, James, and I'm looking for James in the hotel and he's hiding in one of the doors of the hotel rooms.
And assume that the hotel rooms are arranged in a numerical order and the people in the hotel are arranged in alphabetical order to the numbers behind them.
And so the easiest way, I guess, you could think of to find James is to just run through the doors, opening every single one, and say, is James here?
Is James here?
Is James here?
And so on.
But the fastest way to do it is to run to the middle door and to open the door and say, is James here?
And if James isn't there, you ask him, who are you?
Well, this other person says, well, my name is Daniel.
Then you would instantly know, oh, the people on the left of this door are people whose names start with A, B, and C. And the people on the right-hand of the door will be people from D onwards, D, E, F, H. And then you realize, oh, so if I'm looking for James, James is on the right-hand side.
And so you will run to the door right in the middle of the section you just cut on the right-hand side, and then you will ask again, is James here?
If James isn't there, you would try to figure out, is f on the left-hand side of the grid or on the right-side of the grid, and so on.
And that way you get to your results real fast instead of just trying it one at a time.
Now, I guess what I wanted to get out of this video to the people out there is to blow their minds on the simplicity of some of these algorithms and to say, whoa, if I can understand that-- and to say that just knowing that something is a little bit better actually makes a humongous difference in the real world.
And that's what I want to get out of the video.
Hi.
Joshua here.
So excited to do is vlog.
This is kind of like my first time doing a vlog in front of a real public audience, and so some of the things I really liked about yesterday was-- I mean, honestly, yesterday was one of the most mind-blowing sessions.
I got so many simple, yet I can see how very effective videoing doing, and being a talk show host techniques.
And I see how that can connect to my life.
Yeah, the first one I thought that really stood out for me was the one on looking at a member of your audience like your family member or a professor behind the camera, so it's not just a camera.
So I mean, yeah.
But the funny thing is speaking of this tip, I kind of forgot it, and I'm looking at the camera thinking of the camera, so it's a bit ironic.
I guess many of these things require a lot of practice, and that's kind of like what I need to do and get out of the zone of thinking that I'm in front of a camera.
I mean, usually, every time you press a camera, there's this red button.
Sometimes I think maybe if that red button was eliminated, a lot of things would be different.
Yeah, so I mean, I guess one of my questions I would like to ask as well is what kind of face, what would be an appropriate member of the audience that I would like to put there?
I mean, if I put someone who is very respectable like my professor, then I guess the whole tone, tonality of my voice, the whole choice of my vocabulary would try to be more serious.
But at the same time, I guess if I put someone like a two-year-old in my imagination in front of camera, I probably would not talk in the same tonality.
So I guess what would an appropriate member of the audience be in front of the camera?
Well, some of the other tips I got that really stood out for me was-- yeah.
For example, the idea of doing it in a ridiculous way, so basically, George had us come in front of the camera-- not the camera but in front of the screen-- and read off the script, and some interesting things is he tried his best to tell us to say it in many different ways.
But although we tried, we kind of like said it almost the same way.
And I kind of realized it myself.
And the funny thing about that is, well, I guess somehow wired in our brain to think that there is probably the best way of saying it, and then after we have adapted that way of saying it, we find it very difficult to draw away from that.
But the key element in this process that stood out for me is saying repeat.
Oh, that's good.
Oh, you recorded that well.
But can we do that again?
And you would do it like over and over and over again, but every time we do it, you want to be different.
And so stepping out of your zone and doing something crazy would help you and aid you in the process of coming up with variations of ways to say it.
And I think the beauty of recording is probably that you could get the best version and use the best version in your edits, rather than in a live performance when you only have one shot, and it's kind of like if you don't make it, it's over.
So I guess it's a bit different then.
But at the same time, it gives me that understanding of like you have to keep repeating.
It's kind of like doing many things.
If you do well at, you probably have to repeat many times to get it done right.
And I guess it's no different here.
Yeah, and overall, I think it's a very, very good learning experience that I have.
I'm actually super excited, and I've always come to ask myself, why did I choose to make videos?
Why video rather than like anything else?
And I've thought that, well, videos in education are not just videos to educate, but it's also to excite.
And that's what I want to bring.
And my view is I don't see that it should just be about, oh, I learned something.
Oh, I'm just becoming smart.
But rather, it should be something that, oh, my goodness, I didn't believe I didn't know this.
It's so simple, but now that I've watched it, I used to think it was difficult, but now I know it's so simple.
And I know that I can do it.
It's kind of like being enlightened in some sense.
Or rather, in another sense, it could be seeing something really normal and turning it extraordinary or seeing something really extraordinary, which I always thought was super complicated, and only a small group of people could do-- taking that and turning it into something that I can understand.
I feel that that's quite empowering for anyone.
And that's kind of I want to achieve here, and I hope I do with my pictures and my video eventually.
I guess one of the hardest things I have with the script and with the way I want to share is that programming and software algorithms are very conceptual in nature, and they don't have a physical form.
So usually, I have to choose like analogies that would be physical in nature that represent them.
And one of the key points I learned about how do I avoid sounding like I'm dumbing down my audience, and I found that the tips that Josh gave was to give them a sort of a reply, something like preparing them to understand that this is the really simple way of saying it.
So you would say something like, this the really simple way of saying it, but basically, blah, blah, blah.
So that kind of like helps you prep the audience to understand that you're giving the simpler picture instead of something more complicated, I guess.
And I look forward to the next few lessons, and it will be interesting.
Hi.
It's Josh here again.
And what happened today for animation storyboarding was really awesome.
Josh and his team gave really interesting points on how we should do our storyboarding.
And one of the main takeaways I got from it is that storyboarding is, yet again, one of the main parts of the iterative process, where you just keep going on and on, and you keep improving on your script.
So even though you might think that you have the perfect script, but when combined with the visual elements and using a storyboard, you can start to see how things will look like onscreen, even before you start filming.
And that is quite modern.
And I guess sometimes in some of my concepts I would find that there will be times where I have topics or I have things that I would find it difficult to put on camera.
But if I had to draw it out, it'd be easier.
Or if there was some way to show it through animation, it would be better.
And I guess that applies for most of the concepts that are very conceptual in nature and very abstract in nature, but not very physical, something like an algorithm or a software concept.
I guess this is something that is probably not physical in nature.
So it will be perfect to actually use animation.
But that being said, I'm quite concerned about to what degree the animation needs to be.
And so I saw what Elizabeth showed us on how she used very simple animations.
And they were drawn by hand.
I thought that that was very effective.
And That's probably the way it could possibly go.
I really liked the fact that they had a way of arranging the shots, narration, endnotes in the same Excel file, as kind of like a template to go through.
I'm also quite fascinated by the idea that we have to keep thinking of unique ways to make the script better.
There will be times when you draw a certain storyboard for a particular line, and you will realize after a while that the line isn't important, especially when you find it super hard to fit a storyboard that would make sense with the rest of the storyboards.
And I found some of his advice on 150 words per minute really works.
If you want to write a script, then you will roughly be able to know how long it will take as well.
So yeah, I'm really looking forward to tomorrow, when we talk about more of the traditional camera angles and camera shots.
And I'm looking forward to that and looking forward to the traditional concepts of filmmaking.
All right.
See you tomorrow.
Hi.
Have you found it particularly difficult to find a specific item in your house?
Let's say you're looking for a pair of gloves, but you just can't find it, and you spend the entire afternoon looking for it.
Well, you've just encountered the same problem that companies like Google or Microsoft encounter every single day.
And that's the problem of search.
Just like a house, where you store thousands of different items, Google stores 45 billion index pages of information.
And if every page was a sheet of paper and we stacked them up real high, we'd create a tower 600 times taller than Mount Everest.
Well, how can Google find my results so quickly when I find it so difficult to find a pair of gloves?
Well, searching on Google is kind of like looking for a person in a big school.
Let's say you're looking for James in a row of classrooms.
Well, the easiest method would be to go to every classroom nearest to you until you find James.
There's a better method when it's binary search.
Let's say the students were arranged from A to Z in the increasing number of the classrooms.
And let's say we head to the middle room first.
And if the person in the middle room isn't James, but his name starts with the letter before J, we head to the right.
If not, we head to the left.
We then approach the middle room in the newly sectioned area.
We rinse and repeat.
Eventually, we will find James, just like the first method.
But we find him in a much faster way with the second method.
How much faster would that be?
Well, that depends on the number of students in the school.
Let's say there are 500 students, and we're looking for one.
It will take about 80 minutes in the first method, but 1 and 1/2 minutes with binary search.
Let's say there are 1,000 students in the school.
It would take 160 minutes with the first method, but 1.6 minutes with the second method.
Now, that's a whole lot of difference.
So a name is just a word.
But Google searches a combination of words, making it a little bit more complicated.
So just like how we identified the first letter of each alphabet of the name, Google identifies 200 unique factors, making your search terms faster.
If you recall, the effectiveness of binary search depends on the prearrangement of data.
And that's why computer scientists are actively looking for ways to sort, manage, and eventually retrieve data faster and better.
In the same way, the TV remote goes near the TV, the shoes go to the shoe rack, the coats go into the cupboard, and the winter gloves go into the winter jacket.
Aha.
So that's where my gloves are.
Hi, have you found it particularly difficult to find a specific item in your house?
Let's say you're looking for a pair of gloves, but you just can't find it, and you spend your entire afternoon looking for it.
Well, you've just encountered the same problem that companies like Google or Microsoft encounter every single day.
And that's the problem of search.
Just like a house which stores thousands of different items, Google stores 45 billion index pages of information.
If every page was a sheet of paper and we stack them up real high, we'll create a tower 600 times taller than Mount Everest.
Well, how can Google find my results so quickly when I find it so difficult to find a pair of gloves?
Well, searching on Google is kind of like looking for person in a big school.
Let's say you're looking for James in a row of classrooms.
One of the easiest method would be to go to every classroom nearest to you until you find James.
There's a better method called binary search.
Now, let's say the students were arranged from a to z, one in each classroom.
We would then go to the middle room first and see if James is there.
And if James isn't there we will look at the first letter in the name, and if the letter is before j, we head to the right, if not, we head to the left.
We would then approach the middle room in the newly sectioned area.
Eventually we will repeat the process over and over again until we find James.
Now, this is just like the first method, but it's at a much, much faster rate.
How much faster would that be?
Well, that depends on the number of students in the school.
Let's say there are 500 students and we're looking for one.
It will take about 80 minutes in the first method, But one and a half minutes with binary search.
But let's say there are 1,000 students in the school.
It would take 160 minutes with the first method, but 1.6 minutes with the second method.
Now that's a whole lot of difference.
So a name is just a word.
But Google searches a combination of words, making it a little bit more complicated.
So just like how we identified the first letter of each alphabet of the name, Google identifies 200 unique factors making your search terms faster.
Well, if you recall, the effectiveness of binary search depends on the prearrangement of data.
And that's why computer scientists are actively looking for ways to sort, manage, and eventually retrieve data faster and better.
In the same way, the TV remote goes near the TV, the shoes go to the shoe rack, the coats go into the cupboard, and the winter gloves go into the winter jacket.
Aha, so that's where my gloves are.
Everyone has pretty much seen a science fiction movie in this day and age, and I'm pretty sure everyone has seen at least one that involves time traveling, maybe Star Trek, Interstellar, or more recently, Predestination.
And it's such a widely-explored topic in science fiction-- the whole genre of time traveling.
But many times, Hollywood films always have these little, little loopholes in their story line and their plot lines.
And they often use weird theories and science to explain themselves, which are just like god-from-the-machine solutions.
Totally not efficient.
However, scientists have actually put forward three main theories when it comes to time traveling.
The first theory-- the fixed timeline theory.
Say, for example, you're trying to prevent World War II, and you travel back in time after inventing a time machine.
And after killing the baby that was supposed to be Hitler, you put, I don't know, some other baby you found in an orphanage and put it in the same cot.
However, when you travel back in the future, you realize that the baby that you had replaced it with was still one that turned out to be Hitler.
And there is no changing of the past effectively, or the future or the present-- it's all on a fixed timeline.
And these particular motions are actually often seen in movies such as Harry Potter.
It was seen in Interstellar as well.
It was also seen in Predestination.
It's a good film-- you have to catch it.
The second theory is the multiverse theory, which says that every time you go back in time and you alter history-- for example, you step on a butterfly and that butterfly could actually maybe be the prevention of, I don't know, a hurricane that sweeps through Orlando right now-- you spawn a new set of events which happens in a separate universe.
Which, in other words, means that you have actually got infinite number of universes after you have all these different decisions and different chain of events occurring.
And this particular timeline concept has actually been explored many times as well.
It's been featured in the recent reboot of Star Trek movies.
It's also been featured in the recent Fringe series, which kind of died because J.J. Abrams ran out of money or something.
And, yeah.
This multiverse theory is actually quite popular because it's the easiest for producers to just throw this illusion, and then they say, oh, we actually spawned off a new timeline.
So that's when you get sloppy and lazy.
The third theory we have is what we call the dynamic timeline theory.
It's also a theory which induces all the paradoxes, which you often hear about as [INAUDIBLE], such as the grandfather paradox, where you go back in time, and you kill your grandfather.
And because you killed your grandfather, you won't exist.
And because you won't exist, you can't go back in time to kill your grandfather.
And that's where the whole paradox comes in.
Who kills who?
God knows.
The third theory we have is called the dynamic timeline theory, which is also where all your grandfather paradoxes come in.
Let's say your grandfather is a really, really evil man-- Hitler-- and you have to go back in time to kill him.
And you go back, you do the deed, and then you realize that, oh no, after I killed him, I can't exist.
And because you can't exist, you can't go back in time to kill him.
And because of that, he comes back, and then you have to come back, and you have to go back in time again and kill him.
And that's where the paradox starts.
It's an infinite circle of never-ending events.
And that's how the grandfather paradox came about to be in the first place.
And I think you may have seen it in, like, Back to the Future.
And I can't really think of any other films right now.
So this is a brief history of time travel and the three theories associated with it, as well as a quick touch on the paradox of time-- grandfather paradox.
All right.
So effectively we'll start off by talking about why people think that time travel is not possible, all the supposed "theoretical" experiments that proved that it's not possible.
And then we'll explore a little bit more into, like, what's the definition of time travel, because we can talk about time traveling, like we're traveling through time right now, it's just at, like, a pace of one second per second.
So we can talk about time dilation and explore how we can travel at periods of maybe seven seconds per second.
And then after that, we go a little more in-depth in a bit more layman terms of how time dilation works.
And then we ask, like, so if we can travel forward in time, can we travel backward in time?
And that brings in the whole concept of, like, how you need more-- when you increase the speed of an object, the mass of the object increases, and then you also require more energy.
And when you put in more energy, you may not necessarily increase the speed, and therefore it's still a big question whether time travel is actually really possible.
Imagine holding a party and then sending out invitations after the party.
Wait a minute, that doesn't make any sense.
Are my friends going to be able to time travel back just to attend my party?
In 2009, Stephen Hawking actually conducted such an experiment to disprove time travel.
What exactly is time travel, and how does it work?
It actually has a lot to with speed.
A lot of what we know about time traveling actually comes from Einstein's Theory of Special and General Relativity.
We are actually time traveling right now but not in the manner that sci-fi films have depicted them.
We are, in fact, time traveling at a pace one hour per hour.
In other words, every hour I experience, the world around me experiences a singular hour too.
Now, it may sound simple and trivial, but stay with me.
This is where it gets interesting.
Now, the last part of Einstein's Theory of Special Relativity is actually a phenomenon known as time dilation.
Effectively, this means that the faster you travel, the slower time passes around you.
It potentially means that I could travel at maybe more than one hour per hour.
This phenomenon, however, is only noticeable at really, really high speeds, speeds near the speed of light.
This rocket here, not even close.
Say I do have a spaceship now that travels at 90% the speed of light, and I have a pair of newborn twins.
I bring one of them with me on this journey for 10 years.
When I return, one of them will have turned 23 years old, while the one with me will have only turned 10 years old.
While we have experienced 10 years on a spaceship, 23 years have actually passed on earth.
This amount of time actually increases exponentially as you get closer to the speed of light.
So we could potentially travel forward in time, but what about backwards in time?
That's a whole other issue.
Now, remember our previous graph.
The time that it took the [INAUDIBLE] to the speed of light.
Looks like this thing may be the secret to time travel lies on the other side of the line.
In other words, we may have to travel faster than the speed of light just to be able to travel back in time.
Now, when you do travel at speeds near the speed of light, something called relativistic mass comes into play.
In other words, as your speed goes up, your mass increases.
As your mass increases, you require more energy to move the same object.
This might mean that we need an infinite amount of energy just to move an object beyond the speed of light.
However, the more energy you put into an object, the more likely you are to increase its mass rather than to increase its speed.
Therefore, to get an object past the speed of light, quite impossible at this point in time.
So why are we still so obsessed with time traveling?
Time travel does come with its own set of problems.
Take for example, my party before.
If I were to send out my invitations after the party, my friends would have come back in time to attend it.
And if they did come back in time to attend it, I wouldn't have to have sent out my invitations.
And if I didn't send out my invitations, they wouldn't be at my party.
It doesn't make any sense at all that they are both at my party and not at my party.
This is what we call the grandfather paradox.
It's one of three time travel theories that we currently have.
Now, so it seems that we may not be able to time travel yet based on what physics tells us at least.
For now, if you are planning a party, be sure to send out the invites first.
Your friends can't time travel yet.
Imagine holding a party and then sending out the invitations after the party.
Wait a minute.
That doesn't make any sense.
Are my friends going to be able to time travel back just to attend my party?
In 2009, Stephen Hawking actually conducted such an experiment to disprove time travel.
What exactly is time travel?
And how does it work?
It actually has a lot to do with-- speed.
A lot of what we know about time traveling actually comes from Einstein's theories of special and general relativity.
We are actually time traveling right now, but not in the manner that sci-fi films have depicted them.
We are, in fact, time traveling at a pace of one hour per hour.
In other words, every hour I experience, the world around me experiences a singular hour, too.
Now, it may sound simple and trivial, but stay with me.
This is where it gets interesting.
Now, the last part of Einstein's theory of special relativity is actually a phenomenon known as time dilation.
Effectively this means that the faster you travel, the slower time passes around you.
It potentially means that I could travel at, maybe, more than one hour per hour.
This phenomenon, however, is only noticeable at really, really high speeds, speeds near the speed of light.
This rocket here?
Not even close.
Say I do have a spaceship now that travels at 90% of the speed of light, and I have a pair of newborn twins.
I bring one of them with me on this journey for 10 years.
When I return, one of them will have turned 23 years old while the one with me will have only turned 10 years old.
While we have experience 10 years on a spaceship, 23 years have actually passed on earth.
This amount of time actually increases exponentially as we get closer to the speed of light.
So we could potentially travel forward in time.
But what about backwards in time?
That's a whole other issue.
Now, remember our previous graph.
The time that it took [INAUDIBLE] to the speed of light.
[INAUDIBLE] may be the secret to time travel lies on the other side of the line.
In other words, we may have to travel faster than the speed of light just to be able to travel back in time.
Now, when you do travel at speeds near the speed of light, something called relativistic mass comes into play.
In other words, as your speed goes up, your mass increases.
As your mass increases, you require more energy to move the same object.
This might mean that you need an infinite amount of energy just to move an object beyond the speed of light.
However, the more energy you put into an object, the more likely you are to increase its mass rather than to increase its speed.
Therefore, to get an object past the speed of light is quite impossible at this point in time.
So why are we still so obsessed with time travel?
Time travel does come with its own set of problems.
Take, for example, my party before.
If I were to send out my invitations after the party, my friends would have come back in time to attend it, and if they did come back in time to attend it, I wouldn't have time to send out my invitations.
And if I didn't send out my invitations, they wouldn't be at my party.
It doesn't make any sense at all that they are both at my party and not at my party.
This is what we call the grandfather paradox.
It's one of three time travel theories that we currently have.
So it seems that we may not be able to time travel yet, based on what physics tells us, at least.
For now, if you are planning a party, be sure to send out the invites first.
Your friends can't time travel, yet.
Yo.
My name is Nathan.
Today I'm going to talk to you about food preservation.
So everyone has food.
We eat it.
Got to survive, and you can't eat it forever.
It's going to go bad.
And so I've got these two examples.
I got this nice orange, pretty vibrant, but it's a bit squishy.
I don't think I'd want to eat it.
I've also got these crackers.
They were left out for a couple weeks, but I don't know.
They're still pretty crunchy.
And I think I'd eat them.
In fact, I did.
So what exactly is it that causes food to go bad?
And the answer to that is usually fungus and bacteria.
So first off, let's think about what we're up against.
Where are fungus and bacteria?
And the answer is everywhere.
So when we're talking about fungus and bacteria, they can enter the food's system at pretty much any point along the way.
You know, when you have a nice animal sitting along munching its grass, it can get in at that point.
And they'll be carried along to the meat if you're eating meat.
It can get on when you're processing food, when it's on a con-- you know, in large factory group settings.
And of course, when you're preparing food or eating it, it's going to get off your grubby, little hands.
So that's what it is.
How does it survive?
What does it need?
And that's going to be same as us.
It just needs food, water, and oxygen.
Pretty simple, so how do you deal with that?
There's no sort of like special thing to survive that you can easily take advantage of.
Well, there's two basic ideas.
First, you want to slow down and then try to stop its beh-- you know, its activity.
And the second is you want to kill the fungus or bacteria.
And there's a number of ways to do these.
So you can refrigerate and freeze an item.
It slows down activity.
You generally want to keep it in the fridge about-- just like a smudge around like 35 degrees Fahrenheit.
And if you-- or in the freezer, you want it just below freezing.
You can dehydrate or freeze dry something, and that does a pretty good job of inhibiting activity.
You can salt something, which can, in fact, both kill and inhibit activity.
You can pickle something.
You can pasteurize it.
You can can it.
You can radiate it, which works fantastically.
But people don't like nuclear stuff.
You can ferment it.
You can carbonate it.
But by and large, in American society, we like chemicals.
So most of the food you're going to eat isn't, like fresh produce, is going to, in fact, have chemical preserves, preservation in it.
So what are the-- there is three main ways to do this.
You can have an antimicrobial approach.
You can have an antioxidant.
And you can working its enzymes.
The first one is pretty simple.
You're just going to try to kill the small microbes, the bacteria, the fungus.
The second one, you're going to, as the name would imply, antioxidant.
You're basically going to try to prevent these things from breaking down fats and lipids by having a reducing agent.
And last thing is when you're talking about going against enzymes is basically in some things like food in particular, when an item is ripe, these different enzymes start coming up, like an ascorbic acid in apples, and basically what you try to do is counteract these.
So within things that perform these functions is four main things that use-- get used a lot.
First, you have sulfides, which do everything perfectly, or pretty well.
But unfortunately, some people are allergic, so you don't see it in everything.
Most commonly you're going to see them in dried fruit , kind of potatoes, starchy things.
You also have propionates, which tend to be more for like things like bread.
Or they also feed it to a lot of animals.
So to kill-- prevent them from getting infections and all these things.
And what that mainly does is kills bacteria and especially fungus and can keep things like bread more fresh.
You have benzoates, which are mainly for-- which basically just kill the fungi or bacteria.
But they tend to work at a lower pH, so when it's acidic.
And so often you'll see those in things like fruit juices or carbonated beverages or sparkling things or-- and vinegar-based solutions.
So that's where you see that.
The last one, which is getting to the meat of the topic, are nitrates and nitrites.
And mainly you see those as antimicrobials and as antioxidants.
And you will see them in hams, cured items.
You know, if you're getting any sort of frozen meat, that's going to have nitrates, which are wonderful except when they react with amino acid.
They can possibly sometimes lead to cancer.
They have some carcinogenic links.
But those are the main ways that you're going to be able to have a nice, fresh cracker versus a squishy old orange that had nothing done to it.
If you're interested, the crackers have in them something that's called sodium bisulfite, and it makes it nice and tasty.
All right.
So day two-- yeah, a lot happened today.
So we had the sixth graders come in and critique our ideas.
And I'd say the main thing that came out of that was I realized a lot about what I wanted to do.
Kind of the moment that you said, you're going to have to tell your ideas to sixth graders, I had this dread because I knew my main idea just wasn't somewhere that was going to be of any interest to them.
And talking to them, it wasn't just at all.
There was nothing that interested them at all.
And I know it's part of the issue of framing and the issue's so big.
And I didn't have a narrative that made it better and this contrasts one of my backup ideas.
The nuggets was something I did this past semester on how to find magma chambers underground, which is really cool and yeah-- but honestly wasn't what I wanted to do a video on.
And so I'm not exactly sure what I'm doing yet but I think I'm going to stick with the decomposition of food.
I'm going to make it-- I think that it has a lot of potential, especially once you get to the visual aspect because there are a lot of people who look at these videos of McDonald's decomposing or I think-- there's just a lot of really high-quality film time lapses of things decomposing on YouTube but there's not an explanation or anything.
And I think that's why there's room for it.
And if I really work on how to do it, I think I can make a good video out of decomposition and rotting.
But it definitely caught me off guard when they said we're going to have to talk to sixth graders because that really just made me realize that it's very important what we want to do here to make something.
And that something-- it's going to be geared to a very different audience than myself.
I would be probably OK watching an educational video about all of the ways decomposition and the specifics but I guess that's not for everyone in the most basic sense.
So you have to pitch it in a way and I'll have figure out how to do that.
So yeah, that was today.
I thought that Natalie's Intelius talk was actually really interesting because like you said, she talked about a lot of the mistakes she made and I can see how some of those might come up for me.
And aside from the fact that Parabola just seems like something awesome that actually one of my teachers are in touch with still and they're probably forwarding along to him because he's always looking for cool things to do in the class.
But yeah, all in all, a pretty good day-- lots of work to do on my script since I have no script.
So yeah, bye.
I look in that fridge and it's pretty disgusting.
There are things that are oozing black liquid at the bottom.
There's shrivelled up things in the corner.
And there's a smell that I don't really want to ask where it came from.
But where does all of that come from?
And the answer is decomposition.
Because it turns out we aren't the only ones that eat food out of our fridges.
There's bacteria and fungi in there that break down organic material and that's how they survive.
And so that's how you end up with broccoli that has its cell walls broken down.
And so all of its water and nutrients and other liquidy things flow out and make this disgusting black ooze.
And that's why you get the fats in meat being broken down, releasing this just terrible rancid smell.
And that's why the inside of a peach gets eaten away and so there's nothing but the exterior, which collapses on itself, shrivelling up like I do in a social situation.
And I guess if you want to know how that happens, we can find out.
All right, second day down.
Wait, no.
Third.
Third day-- so third day.
All right, so third day reflection.
A lot happening.
So much to talk about, actually.
So I guess I'll start with the first thing that pops into mind, which is shooting.
I just got done finally finishing my trailer pitch thing, and one thing I guess you always know, but you don't appreciate, is how many tries at shooting something it takes to get something you're satisfied with, because, oh, my gosh, I don't still really like what I've come out with, but I spent a good hour and a half repeating the same thing over and over again.
And I don't know.
I am OK with what came of it, because I know it's still very, very rough, and that's the point of it.
So, yeah, that's first.
Other things that I'm coming to appreciate, which is there's a YouTuber named CGP Grey who makes videos, and everyone rags on him because he has a release schedule that is very, very spacious.
You know, order of, like, 68 weeks between videos, and I've never gotten, you know, particularly annoyed at this, but a lot of people seem to.
I can understand how you could just spend so long pouring over every little detail because-- I mean, not even just that, but just the paralysis you get when you're trying to think about things.
So I'm appreciating that.
And, yeah.
So that's my thoughts more considering what I'm doing on that level.
And I've got some other concerns, I guess, that have come up, how I feel about my video as a whole.
I guess in lecture there's been a lot mentioned in talking about why, as MIT students, we kind of have this different perspective and access to these different things that give us this unique ability to make cool science videos.
And I watched the other Science Out Loud videos, and, you know, they're really cool.
You know, the computer one, the access so they have to, you know, [INAUDIBLE] has this random thing inside of it that's a computer.
And, you know, the Boeing turbine that the students are explaining how jet engines work, they could just have access to that.
And, you know, the plant labs and all of that stuff.
But, honestly, if I think about the science videos and education videos on YouTube I like, they tend to be more like the style of Hank Green or, you know, Vsauce or-- just other people.
People who are definitely educated and have this background, but they're more, like, tackling the subject.
They don't necessarily have, like, this great degree of expertise in themselves, but they're going to research it and learn about it and then they're going to kind of relate it to you.
And that's kind of what I want to do with this project, I guess, is I want to learn about something and then turn around and be able to make a video about it.
But I'm not really sure if that's what I'm supposed to be doing.
If I am supposed to take something cool that I know and that I've done and then make a video on that, which would be the more magma thing, but in a way, I think I'm committed to doing the decomposition idea for better or worse.
Yeah.
That's my feelings about that.
I thought the workshops were especially helpful today.
I've kind of had that simultaneous, like, oof, that I didn't have to be the one with my script up there, but at the same time, I realize that that would probably have been very, very helpful to have gone through that.
So I know.
And the hosting thing is def-- you know, more.
It's like the sort of thing you kind of know, and there's some good tips, but it's just going to be a lot of practice, I think, and coaching and being able to throw myself off in order to catch myself more in a natural state.
Yeah.
I don't really have much else to say.
I'm still enjoying the class, so thumbs up on that.
And I shall probably now finish this up, you know, upload things, and go to sleep.
So, yay.
All right, day 4.
Today, I'm doing an audio recording to see what's the difference when I'm talking versus in my video.
So moving on-- day 4, story boarding.
First off, I apologize to you guys I seemed a bit out of it during class.
When I made my last reflection, I said I was going to go to bed.
And then some things happened that caused me to have a bit of a thing to deal with that I couldn't go to bed until I got to a point where I would have been sleeping through class if I had tried to sleep, because I don't like getting up.
Anyway, day 4.
Story boarding was really interesting.
It's really made me think about how I'm going to present my material, not just in making sure the content is interesting, but how to keep everything visually engaging, because I can't really just sit and talk and word vomit all the facts.
And I knew this.
But I hadn't really thought about how are you going to go about this.
So it's kind of difficult because I still don't have my script.
I'm working on that.
And I have not very much at the moment-- just a lot of fragments.
And I'm going to try to piece together.
But I also had a bit of an aha moment this morning.
It mainly gave me some ideas on what I can do for my story moving forward.
Particularly, what I guess I'll just say is one, I've been looking at decomposition in just the context of food in the fridge.
But I think one question I get asked that could be the title of the video even is how's your fridge like a forest floor?
Because the cool thing with decomposition is that there is basically nothing on Earth that eats lignin.
Lignin is the other main polysaccharide or sugar-like carbohydrate thing that makes up wood.
It's cellulose in a matrix of lignin.
And people think it's cellulose, but not think it's lignin.
Termites eat it.
And then protists in their stomach digest it.
And then there is fungi and bacteria that can digest lignin.
And that's a really cool thing that decomposers do that the world really actually needs, because if you imagine-- and this is the second idea for a video that would really be turning everything on is imagining a world without decomposition and what that would be like.
That could be another thing.
Both ideas that I think would make great videos.
And it would be I think a good place to go with where I am, because right now, I'm just going through an explanation video of decomposition.
And, yeah, those are the two aha.
What would life without decomposition be?
Or talking about how your fridge is like a forest floor.
And you could even say like your stomach and how the word decomposition-- but I probably won't do the stomach.
Aha moment, storyboarding, I think I've come up-- at least for the part I did storyboard is actually a pretty good way to present the material.
So I'm excited for that.
And yeah, I really, really liked the presentation by Planet Nutshell.
I thought they did a great job.
And they're super helpful and getting me to understand what the process is.
Yeah, wow!
Makes me appreciate also in the end, it was about a month working on that animation.
So I think that animation is far beyond what I would expect myself to produce.
But given I have less than a month-- less than professional skills.
I feel a little less pressure, at least if I want to animate something or add these things, then that I'm not going to have to do some great production out of it.
See you.
So, day five, wow.
I guess also just week one reflection.
Doing another audio one because-- I don't know.
I feel like it.
Don't feel like doing camera.
What to say?
Well, day five-- I thought the talk was very quick.
Yes, it was prefaced I was going to be [INAUDIBLE].
But just about things to know.
And it was a very useful talk.
I don't think it necessarily was mind-changing like some of the other ones.
But I don't think that was the point.
It was just to give a lot of the practical things to know about how to go about everything.
When you're getting ready to film, to film, and just what to do, best practices-- so that was helpful.
Working on the script still.
Not done.
Hopefully I will get that done soon.
I know there's a schedule.
Not much really to say.
The filming, the scene, I don't think it's going to be a final take, partly because, one, I didn't actually have a balloon.
But I think the script probably is going to be cutting down the time on that part a little bit.
But I think it was really fun to film and really getting a feel for-- it's kind of satisfying to film something that I wrote, a little bit.
And also just getting to work with someone else, you could get the gist of what someone else's piece is going to look like.
I was working with [?
Josh ?]
so I saw a bit about his.
I don't know.
It was pretty fun.
So this is a very short reflection, I guess, compared to the rest.
But I'm just going to continue working on my script.
All right.
So day six.
Another audio thing.
Yeah, my script needs a lot of work.
I've done about the first round of revisions.
Last night I was supposed to get people on my hall to give me feedback, but I got distracted, and that never happened.
So this morning, hopefully before class, I can get some feedback from people who are, I guess, random people.
I guess MIT students are a bit higher than the educational bar that I might be aiming, but they should still, I think, at least be able to give some sort of valuable feedback.
Not just, I guess, on the content, but also on like-- I guess you forget that some MIT people also are better at writing than others.
And what I mean by that is better than me.
So some feedback from them, and then in class, hopefully a bit more.
Yeah.
Just, yesterday was helpful.
Told me things that I don't know if I wanted to hear, but things that I needed to hear.
So yeah.
And some of them were-- as we were going through other people's, I knew, OK.
This is going to be a flaw that's going to come up with mine.
So yeah.
I think that taught me a lot.
And seeing examples of people who have better scripts in there I think also helped.
So I don't know.
That's where I am right now, is just-- you know, you're definitely at crunch time.
The shooting date got changed.
I don't know how I feel about it.
Mainly because of that it's a lot shorter time between the rough and final cut.
And yeah.
I mean, I guess a three-day weekend is kind of annoying, in a sense, because I feel like if we had this class on Monday, actually, that would be a pretty good time to have a rough cut done, and then a couple of days in between that and the final cut.
But eh.
I don't know.
I'm not necessarily going to complain about a day of no cl-- where I don't have class or anything.
But I think the timing kind of has made it a little bit more difficult by that.
But I don't know.
Just move on.
Keep on chugging.
And yeah.
See everyone in class.
Why does your fridge start to smell, and where does that icky black liquid come from?
Wouldn't life just be easier if things didn't rot?
If things lasted forever?
Well, maybe, but probably not.
According to the USDA, the average American household wastes 25% of its food, and a lot of restaurants are even worse.
So if we didn't have decomposition, what would happen to the food we throw away?
Food in landfills normally gets dealt with by bacteria, fungi, and protists that allow the nutrients in food to return to the soil, and eventually, other living things.
These decomposers also break down other dead stuff, like trees.
In fact, they're pretty much the only thing that can eat wood.
So in a world without rot, while your home may not have to worry about being eaten by termites who rely on protists in their stomach, pretty soon, forests and landfills would be flooded with a lot of dead stuff.
How do we avoid this problem?
Well, basically, in your fridge, on a forest floor, in a dumpster, almost anywhere, there are fungi, bacteria, and protists that live entirely by eating dead stuff.
These dead things can be more or less divided into three categories.
Carbohydrates-- sugar and starches.
Lipids-- think fats.
And proteins, like meats.
All of these are chemically different, so they each get broken down by different enzymes in different ways before being absorbed by decomposers.
For example, proteases break down proteins into amino acids, the cell's building blocks.
Lipids rely on lipases, and carbohydrates on things like amylase and cellulases.
So how does a perfectly nice broccoli floret start giving off this foul black liquid?
Fruits and vegetables are almost entirely made of water.
So, on the most basic level, you could say that cells are like extremely complex water balloons.
The exterior, or cell wall, is made of cellulose, a complex carbohydrate that gets broken down by enzymes into small sugars the cell can get energy from.
When a bacteria or fungi or uses cellulase to eat the exterior of a cell, it's like if I were to pop the balloon.
That's the muck you see in your fridge.
That's how you get that icky black liquid.
What about that stink?
For fruits and vegetables, a lot of time, the smell happens after the icky black liquid forms.
Other bacteria that weren't involved in the initial colonization move in and start to stink everything up.
Meat get smelly when lipases break down fat in the meat into glycerol and fatty acids to energy sources.
And fatty acids are kind of gross.
And so while your food rotting may smells absolutely terrible, because of it, the environment is able to recycle crucial nutrients it needs.
And, well, that's why we have all this.
Why does your fridge start to smell, and where does that icky black liquid come from?
Wouldn't life just be easier if things didn't rot, if things lasted forever?
Well, maybe, but probably not.
The average American household wastes about 25% of its food, and a lot of restaurants are even worse.
So if we didn't have decomposition, what would happen to all the food we throw away?
Food in landfills usually gets dealt with by bacteria, protease, and fungi, who allow nutrients in the food to return to the soil and eventually other living things.
These decompositions also break down other dead stuff like trees.
In fact, they're pretty much the only thing that can eat wood.
So even if in a world without rot, you don't have to worry about your house getting eaten by termites who rely on protease in their stomachs.
Pretty soon forests and landfills will be flooded with a lot of dead stuff.
How do we avoid this problem?
Well, basically in your fridge, on a forest floor, in a dumpster, almost anywhere, there are fungi, bacteria, and protease that live entirely by eating stuff.
These dead things can be more or less divided into three categories-- carbohydrates, sugar and starches; lipids, think fats; and proteins like meat.
All of these are chemically different, so they each get broken down by different enzymes in different ways before being absorbed by decomposers.
For example, proteases break down proteins into amino acids, the cell's building blocks.
Lipids rely on lipases and carbohydrates on things like amylase and cellulases.
So how does a perfectly nice broccoli floret start giving off this foul black liquid?
Fruits and vegetables are almost entirely made of water.
So on the most basic level, you could say the cells are like extremely complex water balloons.
The exterior, or cell wall, is made of cellulose, a complex carbohydrate that gets broken down by enzymes into small sugars the cell can get energy from.
The bacteria or fungi uses cellulase to eat the exterior of a cell, it's like if I were to pop the balloon.
That's the muck you see in your fridge.
What about that stink?
For fruits and vegetables, a lot of time the smell happens after the icky black liquid forms.
Other bacteria that weren't involved in the initial colonization move in and start to stink everything up.
Meat get's smelly when lipases break down fat in the meat into glycerol and fatty acids to energy sources, and fatty acids are kind of gross.
And so while food rotting may smell absolutely terrible, because of it, the environment is able to recycle crucial nutrients it needs.
And, well, that's why we have all this.
[MUSIC PLAYING]
We all know that breakfast is the most important meal of the day.
And our staple for breakfast is the egg.
There's omlettes, there's scrambled, there's hard boiled, but there's one type of egg that I don't know anyone that knows how to make it, and what is that?
That's the poached egg.
Oh, making a poached egg is easy, you know, I'll just go on to Google and see how to make a poached egg, it's not a big deal.
But it's not that easy.
And there's actually a lot of science that goes into poached eggs.
The basic principle behind poaching eggs involves protein.
Protein is made up of amino acids.
When those amino acids are subject to heat, they lose their form and they solidify.
This is commonly what you see in over easy or over hard eggs.
That white crust that forms on the yolk, that's called ovalbumin, or ovalbumin, and that's the main protein that is in the egg.
Now, what happens when you poach an egg is you have to have this process, this denaturing occur very rapidly or else the egg will dissipate into the water and you won't get a good poached egg.
Now, to do that you have to have a very high temperature.
And it is known that ovalbumin denatures at a very high acidity.
So what you do is you'd have to add some sort of acid to the water that you're boiling the egg in.
So that can be either lemon juice or it could be vinegar, anything that will lower the acidity of the water, which is typically around 7.0.
That's what's going to give you the best poached egg.
Let us assume for a second that this cardboard box I hold in front of you is a ship.
Let's say it's a box barge.
It kind of looks like a barge.
Now what I've done for you is drawn a water line.
And what that is, is that is the level at which the barge will float on top of the water, assuming that it's going to float.
Now, if I take these scissors and create damage to the ballast section of this barge, what do you think would happen?
You'd probably be right in saying that water is going to go through this hole that I've created, fill up in the barge, and that the barge is going to sink.
Now let's take that same scenario and do it again.
So we're going to damage this bow portion right here.
Now what do you think would happen?
You'd probably say that, well, you just did that, so it's going to sink.
Water's going to come in that hole, and it's going to sink the ship.
But you'd be wrong.
What I did is I took these cardboard pieces right here and subdivided the vessel.
And what that means is that when water comes in through this hole, it's just going to fill up this compartment.
Now, what that's going to do is it's going to cause a trim condition in the barge, but it's not going to sink.
And people on board are not going to be injured as a result of the vessel sinking.
Now, I show you this example as a segue into what naval architects do.
One of the most important features of designing a ship is making sure it's not going to sink as easy as it sounds.
And the way that you do that is through subdivision.
Hi.
I'm taking [INAUDIBLE] advice and trying to desensitize myself to speaking on camera.
So that's why I'm doing a vlog instead of blog.
I thought today was really a good class.
It was good having George come and share some of his wisdom with us and also being able to give us some sort of critique on our scripts and what we should change about them.
Some of things I wrote down that I thought were particularly important that I'm going to take away from it is that a lot is making sure that you're able to relate with your audience.
And, with that, being able to kind of be personal with them and create an environment where it's not just you presenting or talking to the students but something that you're trying to figure this out-- you and them.
It's not just-- you get my drift.
Also another thing that I got out of it was how it takes 8 to 12 scripts, and not just revising scripts, but actually doing the entire thing over again.
And I was jotting down some notes about how I would change mine, and I was looking at specific sentences.
Then I shied away from that, and I actually tried to rewrite some of it.
And I can see what you're saying, that you definitely-- you don't have any bias, you're not thinking of what you said and what you're trying to force into place from your previous script.
You're just able to have a clear mind and to present it in a new way now.
So that's what I did with my pitch.
I'm going to post it right after this video.
But the pitch was definitely a lot easier.
I'm speaking to a subject that I am personally familiar with, so I don't have to fake it as much.
And the other part that I thought was good with pitching something that I know is, I'm somewhat a little bit more excited about it, even though I think naturally I don't project a very excited tone, I guess.
So that's something I have to work on.
And actually the piece of advice that George gave about trying to step up your video, because the video naturally makes you a little bit more boring.
That's something I really have to work on, and the video I'm going to post, I'm not perfect at it yet.
I still sound pretty monotone, but I think that will come with time.
So I'm excited to see what tomorrow's class has in store.
And, yeah, I think I'm going in the right direction.
So I'm excited about that.
All right.
Bye.
When we take complex things and break them into smaller pieces, we find out that we know a lot more about things then we think.
[MUSIC PLAYING] Now let's take this box, ORCA I and create damage below the waterline which I've indicated right here.
[MUSIC PLAYING] As we can see, we've damaged ORCA I right here.
Now let's put ORCA I in the water and see what happens.
[MUSIC PLAYING] [WATER GLUGGING] The ORCA I sank due to the weight of the added water.
But what if the ORCA I contained cargo or oil or even people?
Now let's take ORCA II and do the same thing.
[MUSIC PLAYING] So, we can see that ORCA II did not sink, although it is sitting at an angle towards the bow.
So why didn't ORCA II sink?
As easy as it sounds, this simple demonstration is essential to the design of huge, complex ships.
Ships that are responsible for carrying about 90% of all our stuff.
As naval architects, how do we design ships carrying our stuff to make it into port safely and not sink?
Well, why don't we find out?
[MUSIC PLAYING] Here we have ORCA I and ORCA II from before.
Although ORCA I and ORCA II don't engage in international trade, they behave just as a 1,000 foot container ship would.
Now let's take a look into ORCA I.
We can see that there's nothing in it, it's just a box.
But if we look at ORCA II we can see that it's subdivided into these watertight compartments by these transverse watertight bulkheads.
Now what that means, is if we were to damage this ship right here, water would only flow into this compartment.
It would not go into this one, this one, or this one.
That would cause the ship to be angled or trimmed in the water, but it would not cause the ship to sink completely.
We refer to ORCA II as being subdivided.
And we can see subdivision in many of these ships' plans.
It is unclear when subdivision first started being used in ships.
But accounts of 5th century Chinese trade ships indicate that water would be able to enter the vessel, without sinking.
So let's find out why this happens.
Let's imagine a barge divided into 10 equal compartments.
One of them springs a leak from damage.
Since the ship is subdivided, only the first compartment floods and the ship remains afloat, protecting both its people and cargo.
Although the added water causes the ship to trim, it still has enough buoyancy to return to port for repairs.
[MUSIC PLAYING] Ships still sink, though.
It's both expensive and impractical to try to design the unsinkable ship, especially when these ships will never see that amount of damage.
That's why as naval architects we use computer programs to help us out with subdivision.
Computers make it easy to simulate certain damage cases in practically no time.
With different software, we can damage certain compartments and see how the ship responds to it.
This gives the naval architect a good idea of what parts to improve on the ship, if any.
So even though ships seem like these intricate, complex things, they're really just based on principles that we all already know.
[MUSIC PLAYING]
When we take complex things and break them into smaller pieces, we find out that we know a lot more about things than we think.
[MUSIC PLAYING] Now let's take this box ORCA I and can create damage below the water line, which I've indicated right here.
[MUSIC PLAYING] Now let's put ORCA I in the water and see what happens.
[MUSIC PLAYING] [WATER FLOWING] [FOG HORN] The ORCA I sank due to the weight of the added water.
What if the ORCA I contained cargo or oil or even people?
Now let's take ORCA II and do the same thing.
[MUSIC PLAYING] [WATER FLOWING] [BELL RINGING] So we can see that ORCA II did not sink, although it is sitting at an angle towards the bow.
So why didn't ORCA II sink?
As easy as it sounds, this simple demonstration is essential to design a huge, complex ships.
Ships that are responsible for carrying about 90% of all our stuff.
As Naval architects, how do we design ships carrying our stuff to make it into port safely and not sink?
[MUSIC PLAYING] Here have ORCA I and ORCA II from before.
Although ORCA I and ORCA II don't engage in international trade, they behave just as a 1,000 foot container ship would.
Now let's take a look in ORCA I.
We can see that there's nothing in it, it's just a box.
But if we look at ORCA II, we can see that it's subdivided into these watertight compartments by these transverse water type bulkheads.
Now what that means is that if we were to damage the ship right here, water would only flow into this compartment.
It would not going to this one, this one, or this one.
That would cause the ship to be angled or trimmed in the water, but it would not cause the ship to sink completely.
We refer to ORCA II as being subdivided.
And we can see subdivision in many of these ships' plans.
It Is unclear when subdivision first started being used in ships.
But accounts of 5th century Chinese trade ships indicate that water would be able to enter the vessel without sinking.
So let's find out why this happens.
Let's imagine a barge divided into 10 equal compartments.
One of them springs a leak from damage.
Since the ship is subdivided, only the first compartment floods and the ship remains afloat, protecting both its people and cargo.
Although the added water causes the ship to trim, it still has enough buoyancy to return to port for repairs.
[MUSIC PLAYING] Ships still sink though.
It's both expensive and impractical to try to design the unsinkable ship, especially when these ships will never see that amount of damage.
That's why as naval architects we use computer programs to help us out with subdivision.
Computers make it easy to simulate certain damage cases in practically no time.
With different software, we can damage certain compartments and see how the ship responds to it.
This gives a naval architect a good idea of what parts to improve on the ship, if any.
So even though ships seem like these intricate, complex things, they're really just based on principles that we all already know.
[MUSIC PLAYING]
Science Out Loud.
So if I take this box and just obliterate it in the corner right here and then put it in the water, the weight of the water rushing in will cause the box to sink.
If I take this box and do the same thing, it tips over a little bit, but it still floats.
This box uses the same kind of design that naval architects use to prevent ships 1,000 times bigger from sinking.
What's so special about this box that prevents it from sinking, even with the big hole in its side?
Let's talk about why ships sink in the first place.
You probably know that things will float in water if they're less dense than water and sink if they're more dense.
The metal that this ship is made out of is way more dense than water, so you might think it would sink.
But that metal is shaped so that it traps a lot of air, which is less dense than water, inside it.
So the average density of the hull of the ship is actually much lower than that of water, so the ship floats.
But if the hull springs a leak and fills up with water, there's no more air to lower the average density of the hull, so the ship sinks.
The design of this box basically prevents too much water from getting into it.
This box's hull was divided up into watertight compartments with these walls called bulkheads.
So when I made a hole right here, only this compartment filled up with water.
The rest stayed dry and were able to keep the box afloat.
This box is subdivided, and so is this ship.
Say this ship is divided into ten watertight compartments.
If one area of this ship got damaged, only that compartment would flood, but not any of the others.
The added weight of the water here would cause the ship to tip over a little, to be angled or trimmed in the water, but it wouldn't completely sink, and could still be taken to a port for repairs.
So even with subdivision, why do ships still sink?
Well, it's impractical and expensive to design an unsinkable ship, especially because most of the time, ships just don't see that much damage.
What naval architects do now is try to predict what kind of damage is most likely to happen when designing a ship.
Now, ships are more complicated than this box.
Naval architects have to think about where to subdivide the ship, the shape of the hull, and equipment that goes into the compartments.
We don't know when people first start subdividing ships, but the caps of Chinese trade ships as far back as the fifth century indicate that water would be able to enter the vessel without causing it to sink.
It's pretty crazy that technology that existed so long ago is still being used today.
Hello, I'm Paul.
Thank you for watching Science Out Loud.
For more information, please visit our website.
[LAUGHTER] See what they had me do?
One thing I thought was effective was the way that Chris, I think, on the first day I attended and watched, you kind of laid out a sort of more theoretical framework you know about storytelling and that sort of thing.
And I think that was nice because, for me, I think it's effective.
I'm just one kind of learner, but sort of have a sort of theory laid out first, and then practice later.
I got the big idea of the why and then it felt to me like the course dove into some of the details of how.
So it was was like a theory practice kind of structure, which would have been quite useful to me had I taken the whole course
One of the big challenges, I think, with any kind of media course or film, video, is it that the technical stuff is so overwhelming, potentially, for people who are new.
There's such a big learning curve that you can get lost in those details really fast.
And you go, oh my God, I'll never learn Final Cut and why does everything I shoot suck?
Oh, this is terrible, I'm a failure.
And if you kind of go, well, those aren't really the learning objectives, which you very clearly articulated, then it's kind of a bigger un-clench.
It's like, OK, what I'm really learning here is to be a communicator, be a storyteller, carry myself, understand this medium, and maybe someday I'll be a terrific editor, or maybe I won't.
But either waw--
Maybe you'll plug into the processes in some other way
In some other way.
Exactly, yeah.
But you have to understand the whole process
I remember when I first proposed like-- because you were like, what do you want me to do?
Because I was just like, can you be involved in this somehow?
And it was sort of like, well, maybe you could cover the actual basics of shooting and everything.
You said something like, they seem like a technical things.
And I was like, oh shoot, this is terrible.
I'm doing the thing that I tell people not to do.
So we made a very conscious decision to take all this technical elements.
We did not teach how to edit on Final Cut or anything.
Instead, MIT has free Window licensing.
And honestly, their videos teach it better than I do.
I don't know Final Cut well enough to teach it.
And not all the students used it, so we said, for all the technical learning stuff, it was all on digital blended learning.
Flip Classroom.
Thank you.
And class time was really about teaching big picture stuff, and then work shopping specific examples, and specific things they came up with
Because time is limited and it's so daunting to become a director, on camera host, writer, editor, camera person, all in a few weeks.
And it's just not going to happen
Just the expectation that they come out with the notion of storytelling would be sufficient
It really ended up being a class on pre-production.
It was like two weeks of intense scripting and planning and the rest just sort of happened in the last couple days.
The just had to figure out how to film
That's great, I mean the technical execution gets pushed to the margin, and the more creative stuff takes precedence
I mean it's a different class.
There were a couple people who dropped because they came in thinking, like, this is going to be a class on how to make a video.
And it's not exactly about that.
Because there are other classes like that that already exist on campus, so there's no reason for us to redo that
And you learn stuff by like shooting your own stuff and screwing up
Yeah
Yeah, exactly
But if someone's going to watch this class on OpenCourseWare, for instance, they're not necessarily going to pick up the practical skills that they might be expecting to
We don't teach you how to light, sorry
But like a teacher who might be watching this might be like, how do I teach a video production class?
And I'm not sure this is necessarily exactly that.
Sorry
Oops.
But it is hopefully maybe a conceptual framework that you could then plug-in production modules into
Or a larger course
Yeah, it was so short.
It felt like a sprint
Yeah what can you do?
What can you accomplish in that time?
I was impressed by how ambitious it was
It was very ambitious, I thought.
Pulling together this teaching team, I kind of felt like Nick Fury from The Avengers.
Tapping into all these people that I came across either as a student, or working with.
Video production, that was something that we covered in the class.
It's not a solo effort, and I think teaching video production very much is not a solo effort.
So Chris and I had been talking about this for several years, I feel like.
Just going through your class, and then talking about the program, and the videos that we make.
We'd just been tossing around these ideas.
And George and I have worked together extensively, she said with a heavy sigh, on Science Out Loud.
Josh and I met at a conference purely by accident.
And I knew you guys were doing awesome animation work, and that was something that I didn't really feel like much expertise on.
Just like storyboarding in general, which is really important for students, and for everyone really.
And then we had a couple other guest speakers here and there.
So it was really-- so teaching something that doesn't have an objective answer, in physics there's a right answer to a problem set, for instance.
In media production or creative writing, or anything like this, you can't just be like, well this is what makes a good answer, necessarily.
I feel like something that helps solidify, or helps you come to a good feedback mechanism for students is to have people who are good practitioners of whatever media you're making.
So, you know, if you're teaching creative writing, hopefully you'd be a pretty good creative writer.
If you're teaching media production, or storyboarding, hopefully you're a good animator, or story boarder.
Hopefully you're a decent host
Fair to middling
I know, right?
Tolerable
So that was why from the beginning I felt like it had to be team effort to teach this class
I think you had a skeleton outline for the course, and then you filled in the parts, the learning objectives for parts that you were going to do, and then each of us filled in our parts, and then we all commented on the other people's parts.
Which was a good way to do it
It's tricky because, its not like a linear sort of knowledge comprehension format where first you learn scripting, and then you can learn storyboarding, and then you can learn hosting.
That's actually how we first approached it, and it got really hairy because there's so much overlap
Why?
Why did we approach it that way?
No.
Why did it get hairy?
Well, because I think Chris, initially you wanted--
I caused all the trouble
What did you do?
I'm like, well what should I talk about?
Yeah, there was a lot of coordination that we-- Josh and I, especially, I think, we felt the need to coordinate.
Because there are things that are similar when you're thinking about kind of telling a story in animation and in live action, but there are also things that are different.
And we wanted to make sure that we weren't overlapping each other, and that we were kind of complementary and not repetitive or contradictory.
So I think we ended up kind of restructuring a little bit the first few sessions of the course.
I'm trying to remember
It was interesting because in terms of the big picture learning objectives, I worked with Jaime a lot, the other course instructor.
And she has never done video production.
She wasn't familiar with the content.
But she's more familiar with just instructional design in general.
So she would ask me all these questions that I wouldn't necessarily think to ask or answer, like at the end of the day, what are the skills that you want people to have?
And then what are the sort of learning objectives that you want them to aspire to?
And those are two very different things, right?
Because I don't really care if they were masters of Final Cut by the end, because that doesn't really matter.
The objective is can you articulate a clear story?
But she and I would meet to sort of figure out what are the big picture learning objectives that we want, and then it was a matter of how can we take all these details and all these very important plastic elements, and pick at the most relevant ones to match to the objectives.
So I think the idea of a workshop-based class, a hands-on class, that's what's becoming trendy in education.
At least at MIT right now.
It's bringing it back to mind and hand.
They won't work if students don't bring anything to workshop or have anything ready.
So that's why the first two or three days of class were very, very information intense to bring them up to the point where they had a script that they could actually read aloud at the table read.
Or that they would have a scene they could storyboard.
How was it leading the workshops?
Because I could kind of let you guys take over for those
Yeah, I mean I think it was great.
I mean, I think we couldn't have done what we did if they hadn't come to the class with a portion of a script that they could storyboard.
Or do some sort of storyboard work on.
I think maybe in the future you might think about having-- well, I don't have.
Setting it up so that there's instruction about say, storyboarding, and examples and kind of models.
Then have them do that and then have them return for the critique.
As opposed to trying to compress all of that into one class.
But again, that requires more time
My only workshop-- workshops are great because it means that you rely a lot more on stuff that you sort of just know and can talk off the cuff about and don't have to prepare long lectures, which is fantastic
But that requires that you actually are just natively an expert in that
But the nice thing about that is that you can reduce the time you spend speaking and increase the time that they spend working and speaking.
And I think the earlier that you get the students talking and doing things things in the workshop, the more quickly they realize that it's not a lecture.
And that they're not going to sit there and absorb.
That they have to actually do something
So you had people reading their scripts from the beginning of your workshop.
There was no easing into it.
It was pretty much just like all right, this is the script that so and so submitted last night OK, come up and read it for us
Yeah.
And then they get over their nerves and they just do it.
And then you roll into it.
The hard thing about doing that is-- giving feedback in a workshop is tough because you can't just be like great job.
Go sit down.
That was great that you did it, but we're not going to give you any constructive criticism.
You've got to say OK, here's what I would do differently.
But you have to do it in such a way that it's not ego destroying.
And that applies for any of the stuff.
It's hard to do, especially for hosting, because there you're commenting on their tone of voice, their delivery style--
Their existence
The way they talk.
You know?
So it's a tricky thing to get.
You sort of err on the side of being a little bit softer than you probably would be
It's also-- sorry.
It's also interesting to get them-- I think just from my basic pedagogical experience, because I did teach creative writing for a while.
It's nice to get them-- to get students, as you said, doing peer work where they're critiquing each other so there isn't just this authority figure that has all the answers.
There's also this innate sense that we all have, and students have, of what's working and what's not.
Because they are audience members too and they have that audience sensibility
Actually a really important thing was to establish that the medium in which we are working had no established best practices, and that was something that I tried to emphasize a lot and say, you know, I have very strong opinions on what I think is good hosting and what isn't.
But at the end of the day, you do what you want as long as you can justify it to me.
You can take or leave my advice on how I think you should edit this section, but if you want to stick with what you have and you're able to write out why--
You're going to get an
But an A for effort.
That's fine as long as I see that they're consciously processing their own personal taste.
What was the thing you just said?
I'm saying that you don't want to just be the only person critiquing, which is really important
It's really, really important and I think one of the things about that too is trying to make the classroom I guess kind of a safe space where people can be really honest.
And it's like, you know what?
Someone is probably going to go at you and your work at some point, but you need to be OK with that.
And if you feel that you need to be honest with somebody about their work, that's OK too.
As long as you're not doing it on some kind of illegitimate grounds.
If it's a legitimate critique, you should feel OK about saying it
Yeah, that's a good point.
I think too, it can sometimes be helpful to just kind of lay the ground rules of critique.
Like what's personal.
I mean, sometimes it's not obvious to students what's a personal attack.
What's constructive criticism of the work
Sometimes people need instruction on peer review
Yeah, exactly.
You can't expect that people know the rules
Luckily, just having Jamie at every single class was really helpful.
I mean, it's hard.
In an ideal world, every class would have a staff of five people who are all experts on different facets of the industry.
And then a dedicated instructional designer facilitating group discussions every day
Two instructors per student
Let's recommend that
Right.
But you know, that's what it takes.
It really does.
Because if Jamie wasn't there, I don't know how to facilitate classroom discussion.
She would be able to interject and sort of coach these things subliminally without having to establish guys, these are the classroom rules.
No picking on people.
It was just all done very organically.
And that's actually very hard.
This is such a cop-out answer to be like, it's important to foster a good classroom space.
Go do it.
You know?
But I don't really know how to give actionable items to people who are trying to do this in their schools.
I think one thing I tried to do was, and this was off of Jamie's suggestion, was to do every single assignment that I assigned to the students.
So on day one when they had to record a selfie pitch, I did it and I was like this is not perfect.
And the next day I pointed out things that I didn't like about it or things that could have been better about my own work.
So I think having something physical to be able to critique, to sort of show people how to critique it, you know?
It's not even just show people what's wrong, but show people the active critiquing is really important.
Because if you don't have the physical examples, it's just like saying lighting has to be done well.
Well, if you don't show them an example of a well-lit scene, they're not going to be able to do it
The things you get from students who aren't used to giving that kind of critique tend to be either oh, it was good or I didn't like it.
And neither one of those is particularly helpful
Yeah, or in creative writing class students will be like, this is really deep
Yeah.
But then to be able to be there as an instructor and say, OK. Why do you think that?
Yeah, exactly.
Why is it good?
Why is it deep?
Why didn't you like it?
Questions that drive them to give a more tangible answer.
That's really important in the live discussion.
And it's important, I think, to have it in real time.
Which is why I don't know how you can scale this experience up.
If you offer this as an online class, for instance, I guess you could have forums where students critique each other.
But again, without the guidance of someone who's practiced it.
That limits you.
And then someone who responds like, why do you think this was deep?
The next day, you sort of lose that momentum
I mean, the really great thing is when this sort of stuff happens on set.
Like when we were shooting the episode about the exploding wire and our student host got to see you, me, and the director of photography get into a fight about why we thought the framing was wrong.
And I mean, it was literally like, what do you think of this framing?
I hate it.
Why?
Because it looks like an episode of Home Improvement?
Like an actual episode of Home Improvement or the show that they were parodying in Home Improvement?
No, the crappy version.
OK. Well, let's figure out how to fix it.
Like that's the kind of discussion that happens on set
And that is so much more helpful for someone to see than sometimes people struggle with framing.
Here are some tips.
Try to put the people in this framing.
It might be more engaging.
That's sort of how media things are presented right now.
But to see like George and me fight, is one, way more entertaining
I'm sure it is
But two, I think like she probably took a lot more out of that than watching a lecture series.
Guess who likes to live dangerously?
That's right.
Set theorists.
Or, more generally, mathematicians.
To this day, we have no standard definition of what they do.
The 20th century mathematician Bertrand Russell said that math is the subject where we don't really know what we're doing nor whether it is true.
And actually, that's not that much of an exaggeration.
Mathematics is highly ambiguous and mysterious still.
Take, for example, the supported and yet unanswered question-- Is mathematics real?
Is the arithmetic we learn in elementary school true?
We believe that it's true, because, well, it's worked so well for us.
But, as young logician Kurt Godel proved, we can never know for sure.
In short, math is messy.
There's no way around that.
So what exactly does this mean for us?
Why do set theorists continue to live on the edge of this shaky mathematical precipice?
Could machines really be smarter than humans?
And could The Matrix scenario really happen to us?
Luckily, Godel answers that and so much more.
Oh, hi.
Don't mind me.
I'm just trying to cause a tornado in Brazil, although it could happen in Texas.
We can't really tell for sure.
The reason why is a story that began with a cup of coffee at MIT, continued with doodles, in the end with curing cancer.
It is a mathematical journey through nature, filled with chaos and more funky doodles.
So after perhaps the fourth draft of my script, the world is beginning to look like a fractal, which is in a way cool, because even the creation of a script, if you think about it, is a fractal.
I've been creating all these drafts that are very similar, and it seems that the script writing process has been endless and infinite.
But hopefully it will and.
I'm actually enjoying it at this point, even though it has been a lot of the same thing.
But every day, I come back and I do more research on fractals, and I learn many cool things, which then makes the decision of what to put in the video and what to take out a lot more difficult.
But I managed to cut it down to about 600 words at this point, which clocks at around four minutes and 10 seconds in a fast rate, so that does give me a bit of leeway.
I can't really think of other things I could add, though I can certainly see some things I could take out that I wouldn't want to take out, but that's always been the biggest struggle, because there's so much cool associated with fractals.
And there is a reason why I started seeing the whole world as a fractal, because there are so many patterns and repetitions in everything.
Right now, I'm looking at my corkboard, and it's a fractal.
And the walls of east campus look pretty random, but I'm sure there's some repetitions there.
And this makes math really cool, because essentially, it is the study of patterns, and as long as you can discover those patterns, you can then apply them to solve real-world problems, which is what I wanted to convey to the viewer because that's not the side of mathematics people usually learn.
And when you tell them that, in math, we study patterns and we use them to solve problems, that's often surprising because we are taught so many tools in school, that we don't even think about how to apply them usually.
And I think that's very sad, because it's the applications that make math really cool, or otherwise the concepts that are not really offered at school, like infinity.
So that's about the video.
Today's class was a lot of fun.
I know I mentioned that I was excited for the critique, but there was still a bit of a concern that I wouldn't handle it very well, that it would be somehow a saddening or worrisome experience, which really wasn't like that at all.
I appreciated all the things that were said.
I suspected more criticism, because that may have been more helpful, but in the end, I came back home, I looked over all of the comments from constructors, and they made a lot more sense after the session when people actually said them out loud.
And I also looked at the suggestions by my classmates.
And even though the structure of the video and most of the words are different from what it was at the beginning of the day, a lot of those suggestions are still a part of it.
And hopefully I was able to incorporate them all.
I checked, but like I said, the script changed so much that it's difficult to pinpoint the minor changes.
And I guess my biggest concern was the overall structure, and that's something I felt was a bit off when I originally wrote the script, but I also thought that I could improve that by showing a lot of cool images and animations, which isn't how it works.
So, I am happy that I got to cut it down.
That's something I've always enjoyed with every piece of writing I've done-- cutting out the unnecessary words, to me, has always seemed like the most necessary thing to do.
At least that's the way I was taught, because five years ago, I would write sentences that had three or four adjectives in a row that I thought were absolutely essential to the story, and if you introduced a character, you had to describe all of his or her clothing.
So I learned the hard way that that really wasn't the way to do things, and I thought I learned that lesson, but I'm still learning five years later to cut things out and sacrifice what I think is absolutely essential.
Except this time it's not four adjectives but it's a ton of fun math facts that to me are absolutely amazing, but to others are probably overwhelming, because they have five minutes to learn about this stuff, and I had hours.
But, hours were spent to cut things down, and they have been cut.
I'm really glad with the way the class has made me feel about my writing abilities.
I guess it was nice to see my weaknesses and my strengths, because now I understand better how to describe mathematical concepts.
I always felt-- I always went for the really abstract, shock value kind of math facts, but that's not what I've been trying to do with the video, and that is really nice.
And I always go back to the conversations that I had with the sixth graders and just remember them and the things they were interested and the vocabulary they knew.
So that was an essential learning experiences-- an essential learning experience.
That said, I don't think I have anything to add at this point.
Looking forward to more comments on the script.
Hopefully there are fewer of them this time.
Although, I can probably think of ways to improve the script, and I can definitely do that again tomorrow, if necessary.
With the absence of the final statement, I end this.
Farewell.
Look around you.
What beautiful patterns do you see?
I have just finished practicing my fractal script.
Seem to have memorized it, so that's very exciting.
And now I guess I'm practicing to speak on camera, which I'm trying to do in a faster manner than yesterday, when I paused a lot and that was because I didn't think through what I was going to say.
And today I did.
On the left, I have this white board that has a incomplete Koch's snowflake, as well as a doodle of the pattern that repeats in the Koch's snowflake.
I have perfected at this point, as well as I could, the are of drawing equilateral triangles as I talk about them, so that I don't slow down when I talk, but also the equilateral triangles turnout sort of OK. And I'm really glad I practiced before this shooting because that was a skill to learn.
As to the class today, it was kind of scary and also kind of soothing because that was it.
The last lecture.
And part of me wished that we continued listening to lectures and hearing your script because that was easy part.
That was the part that I was kind of, sort of familiar with.
Taking notes, asking the important questions, such as why show [INAUDIBLE] when talking about engines?
And then coming back and writing stuff about that.
And now, I'm really venturing into an unfamiliar world, that I've seen in movies, I've seen in YouTube videos.
And actually, I have consumed a lot of YouTube videos in the past couple days.
Most of which, allegedly were for research, but then there's of course, a lot of procrastination going on YouTube.
And even those extra videos, really pushed me to ask the question of why I clicked on this video, why I liked this video, and why I continued to watch it.
Which is a question I still continue to ask.
And I don't think it's a possible question because I remember when during the first class, we did the activity marking down where we stopped for different videos.
And some people really liked certain hosts and didn't like others.
For example, I really like Hank Green because I enjoy how he talks.
He's fast, he's animated, and he shoots a lot of information at you, which is awesome because that doesn't leave me a space to stop.
On the other hand, I didn't really enjoy Bill Nye's video because it had a slow transition, it had an introduction, and all those other things that I didn't want to sit through.
But some people didn't like Hank Green and some people really liked Bill Nye.
So, I guess I'll never be able to appeal to all audiences, which I'm OK with.
But back to the post production, it sounds interesting because it's a challenge.
And challenges are always exciting.
But also I wonder how well I can finish up the video because well, either I spent a lot of time on it and get to a shadow of its professional self, or I do a rough draft because that's the only thing the time allows.
And I really don't know.
I don't know what I'm signing up for.
I have a visual of how the video is supposed to look in my head, but it doesn't seem to be the visual possible in real life.
And certainly not something that I can create in a week and find the music, find the images, and do all that in-- well, a very short time.
But I'll see filming goes tomorrow.
It doesn't feel real which maybe is good because then I'm not worried about what's going to happen.
And I hope it becomes real soon.
I'm really excited to film other people's videos and help them because-- well, it's fun to work with other people, but also it puts my mind away from my video.
And it'll allows me to learn from them, which is really awesome.
I think this class has really been the class where I learned to interact with people and collaborate and learn from their ideas.
People had some wonderful suggestions.
And everybody had a different perspective.
I think it's awesome that our class had a range of ages, locations, disciplines, and styles of speech, writing, and-- there were so many awesome ideas that I would love to see fulfilled.
So, if I watched a YouTube show based on what the class has created, that would totally be my favorite show.
And I hope I can contribute to that.
I'm going to try my best.
I'm very excited for it.
And from what I learned, if you are excited, that will make you try harder and eventually you'll put together something that is decent, good.
That said, I'm going to go scout some fractal looking locations on campus.
And after that, I'll get some sleep so I look my best for the filming tomorrow.
[MUSIC PLAYING]
Hi.
What do snowflakes and cell phones have in common?
Well, let me start by drawing a snowflake first.
I draw an equilateral triangle, divide each side into three parts, and draw another equilateral triangle on top of each one.
Then take out the middle and repeat the process, this time with one, two, three, four, times three, which is 12 sides.
Eventually, this shape will start looking something like this.
In mathematics, this is called a Koch snowflake.
If I repeated my process again and again, I would see this same pattern anywhere I looked.
This is a Koch snowflake, this time drawn on a computer.
Such never ending patterns, that on any scale on any level of zoom look roughly the same, are called fractals.
Computer scientists can program these patterns by repeating an often simple mathematical process over and over.
In the 1990s, a radio astronomer by the name of Nathan Cohen used fractals to revolutionize wireless communications.
At the time, Cohen was having troubles with his landlord-- the man wouldn't let him put an antenna on his roof.
So Cohen designed a more compact fractal radio antenna instead.
The landlord didn't notice it, and it worked better than the ones before.
Working further, Cohen designed a new antenna, this time using a fractal called the Menger sponge.
The Menger sponge is not really the sponge you'd be scrubbing your back with, but you can still think of it like that.
Imagine both water and soap getting through your sponges holes, except the water is Wi-Fi and the soap is, say, Bluetooth.
The fractal's infinite sponginess allows it to receive many different frequencies simultaneously.
Before Cohen's invention, antennas had to be cut for one frequency, and that was the only frequency they could operate at.
Without Cohen's sponge, your cell phone would have to look something like a hedgehog to receive multiple signals, including the one your friends use when they call.
Cohen later proved that only fractal shapes could work with such a wide range of frequencies.
Today, millions of wireless devices, such as laptops and bar code scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
Nature has been doing it the whole time, and not just with snowflakes.
Natural selection favors the most efficient systems in organisms, often of a fractal form.
The spiral fractal, for example, is present in sea shells, broccoli, and hurricanes.
The fractal tree is relatively easy to program, and can be used to study river systems, blood vessels, and lightning bolts.
So many natural systems previously thought off-limits to mathematicians can now be explained in terms of fractals.
Mathematics allows us to learn nature's best practices and then apply them to solve real world problems.
Much like Cohen's antenna revolutionized the field of telecommunications, other fractal research is changing medicine, weather prediction, and building design here at MIT and everywhere in the world.
Look around you.
What beautiful patterns do you see?
What do snowflakes and cell phones have in common?
The answer is never-ending patterns called fractals.
To explain, let me start by drawing a snowflake.
First I draw an equilateral triangle, divide each side into three parts, and draw another equilateral triangle on top of each one.
Then take out the middle and repeat the process, this time with one, two, three, four times three, which is 12 sides.
Eventually, the shape will start looking something like this.
In mathematics, this is called a Koch snowflake.
If I repeated my process again and again, I would see this same pattern anywhere I looked.
This is a Koch snowflake, this time drawn on a computer.
Such never-ending patterns that on any scale, on any level of zoom, look roughly the same, are called fractals.
Computer scientists can program these patterns by repeating an often simple mathematical process over and over.
In the 1990s, a radio astronomer by the name of Nathan Cohen used fractals to revolutionize wireless communications.
At the time, Cohen was having troubles with his landlord.
The man wouldn't let him put an antenna on his roof.
So Cohen designed a more compact fractal radio antenna instead.
The landlord didn't notice it, and it worked better than the ones before.
Working further, Cohen designed a new antenna, this time using a fractal called the Menger sponge.
The Menger sponge is not really the sponge you'd be scrubbing your back with, but you can still think of it like that.
Imagine both water and soap getting through your sponge's holes, except the water is Wi-Fi and the soap is, say, Bluetooth.
The fractal's infinite sponginess allows it to receive many different frequencies simultaneously.
Before Cohen's invention, antennas had to be cut for one frequency and that was the only frequency they could operate at.
Without Cohen's sponge, your cell phone would have to look something like a hedgehog to receive multiple signals, including the one your friends use when they call.
Cohen later proved that only fractal shapes could work with such a wide range of frequencies.
Today, millions of wireless devices, such as laptops and barcode scanners, use Cohen's fractal antenna.
Cohen's genius invention, however, was not the first application of fractals in the world.
Nature has been doing it the whole time, and not just with snowflakes.
Natural selection favors the most efficient systems and organisms, often of a fractal form.
The spiral fractal, for example, is present in sea shells, broccoli, and hurricanes.
The fractal tree is relatively easy to program and can be used to study river systems, blood vessels and lightning bolts.
So many natural systems previously thought off-limits to mathematicians can now be understood in terms of fractals.
Mathematics allows us to learn nature's best practices and then apply them to solve real-world problems.
Much like Cohen's antenna revolutionized telecommunications, other fractal research is changing medicine, weather prediction and art, here at MIT and everywhere in the world.
Look around you.
What beautiful patterns do you see?
[JAZZ MUSIC]
[MUSIC PLAYING] [SINGING] Science out loud.
What do snowflakes and cellphones have in common?
The answer is never ending patterns called fractals.
Let me draw a snowflake.
I'll start with an equilateral triangle.
Then I'll draw another equilateral triangle on the middle of each side.
Pull out the middle and repeat the process, this time with 1, 2, 3, 4 times 3, which is 12 sides.
If I do this over and over, the shape will look something like this.
This is called a Koch snowflake, and it has a special property.
No matter where I look or how much I zoom in, I will see the same pattern over and over.
Never ending patterns like this that on any scale, on any level of zoom look roughly the same are called fractals.
We can actually draw a Koch snowflake on the computer by having it repeatedly graph a mathematical equation.
Each time we add a triangle, one side of the Koch snowflake will turn into four.
After the first repetition, we'll get three times four to the first, or 12 sides.
After the second repetition, we'll get three times four to the second, or 48 sides.
After repetition number n, we'll have three times four to the n sides.
If we do this an infinite number of times, we'll get infinitely many sides.
So the perimeter of the Koch snowflake will be infinite.
But the area of the Koch snowflake wouldn't be infinite.
If I draw a circle with a finite area around the snowflake, it will fit completely inside no matter how many times we increase the number of sides.
So the Koch fractal has an infinite perimeter, but a finite area.
In the 1990s, a radio astronomer named Nathan Cohen used the fractal antenna to rethink wireless communications.
At the time, Cohen's landlord wouldn't let him put a radio antenna on his roof, so Cohen decided to make a more compact, fractal like radio antenna instead.
But it didn't just hide the antenna from the landlord.
It also seemed to work better than the regular ones.
Regular antennas have to be cut for one type of signal, and they usually work best when their lengths are certain multiples of their signals' wavelengths.
So FM radio antennas can only pick up FM radio stations, TV antennas can only pick up TV channels, and so on.
But fractal antennas are different.
As the fractal repeats itself more and more, the fractal antenna can pick up more and more signals, not just one.
And because the perimeter of the Koch snowflake grows way faster than its area, the fractal antenna only takes up a quarter of the usual space.
But Cohen didn't stop there.
He designed a new antenna, this time using a fractal called the Menger sponge.
The Menger sponge is kind of like a 3D version of the Koch snowflake and has infinite surface area but finite volume.
The Menger sponge is sometimes used in cellphone antennas.
It can receive all kinds of signals while taking up even less area than a Koch snowflake.
Now, these antennas aren't perfect.
They're smaller, but they're also very intricate, so they're harder and more expensive to make.
And though low fractal antennas can receive many different types of signals, they can't always receive each signal as well as an antenna that was cut for it.
Cohen's invention was not the first application of fractals.
Nature has been doing it forever, and not just with snowflakes.
You can see fractals in river systems, lightning bolts, seashells, and even whole galaxies.
So many natural systems previously thought off limits to mathematicians can now be explained in terms of fractals, and by applying nature's best practices, we can then solve real world problems.
Fractal research is changing fields such as biology.
For example, MIT scientists discovered that chromatin is a fractal, and that keeps DNA from getting tangled.
Look around you.
What beautiful patterns do you see?
Hi.
I'm Yulia.
Thanks for watching "Science Out Loud."
Check out these other awesome videos, and visit our website.
Good.
Now wait for it.
OK. That's it.
I've thought a lot about what it means to lecture to undergraduates.
One of the reasons I thought a lot about it was that the most inspiring teachers I had as a student were lecturers.
And as I become-- Both at Yale and at MIT I taught lecture courses.
And I became, I think, good at them.
But I was never as good as I wanted to be.
And I think maybe one thing that made me a good lecturer was I was always nervous.
And even to this day, I've been lecturing in the film course now I think 38 or 39 years.
I still get stage fright.
But I'm glad I do it because it makes me live for the students.
I hated bad lectures.
I thought they were the pits of higher education.
And I swore to myself that if I became a lecturer I would never bore my students.
That was a worse sin then almost any other a teacher could commit was to be boring.
And I have also felt that, in recent years especially, although it's always been an undercurrent in education, there's been a kind of growing negative feeling about lectures as if they're herding students into our herd experience.
And it doesn't give them hands on experience.
And I think, intellectually, that's a foolish perspective.
Professors no more than students.
It's appropriate for students to listen to professors sometime.
Professors should take the responsibility of when they talk to students very seriously.
If they can present the information to students in ways that are more effective on paper, or on film, or in some other format, on video disks, or on computer disks they should do so.
But if they take-- And in a small classroom they should certainly encourage discussion and interaction.
But I think surely there are times and places, especially in the humanities, when it's perfectly appropriate for a professor who knows an immense amount more than the students to present material that has been synthesized by the years of study and understanding that he or she has.
And the greatest experiences, the most inspiring intellectual experiences I had as an undergraduate in college were from lecturers.
And I taught-- I was very fortunate.
I went to a wonderful university.
Princeton.
They had very small classes.
I was in some tutorials that, they were called preceptorials, that were seven students and a full professor.
So I got the benefits of intense one on one or close to one on one attention.
And I loved those classes.
But the classes I had in which they were 100 or 75 or 150 students listening to a Professor of History, or a Professor of Philosophy, or a Professor of Literature talk in the ways they did about ideas, about the history of ideas, about the power of texts, taught me what literature is and taught me what intellectual engagement with literature should be.
And I swore to myself that if I became a teacher I would try to be a good lecturer rather than a bad one.
So it seemed to me as-- First of all, as I thought about the most inspiring lectures that I had as an undergraduate.
It was one professor I had especially.
He's now dead.
But he was a wonderfully inspiring man.
Also became an inspiring scholar who didn't produce a lot of work but his scholarship was very productive.
And he was a department chair who generated a lot of colleagues work.
He was a great man.
His name was Lawrence Holland.
His book was called, it was on Henry James, it was called The Expense of Spirit.
his lecture was on American literature.
And a number of us in that course became Professors of Literature.
I'm thinking now who I've state in touch with and they're also teaching or still teaching around the country.
So Holland affected other students as well as me.
And his strategy in lectures I've tried to copy.
Or even plagiarize.
One thing he always did was he presented an outline.
It was not the text of the lecture.
But it was three or four central digestible points that organized the lecture in sequence.
He would give out mimeographed sheet to each of his students and you would follow along.
And sometimes under each rubric there might be one or two other sub facts to help the student.
It didn't capture the whole lecture.
But you could use that sheet to organize your understanding of the lecture.
I thought that was a wonderfully helpful.
I found it wonderfully helpful.
Both for review.
But also because what it showed me was that Professor Holland had planned what he was doing.
That he was presenting material to me that he had thought long about that he organized in a serious way.
But the second great thing about Larry Holland's lectures was that they were not completely written out.
There were parts of them that were written out.
Very eloquent moments often at the beginning at the end.
And I've sometimes imitated that.
But the greatest part of his lectures were his lectures seemed spontaneous.
Even though he was following his lecture it seemed as if he was not.
He was speaking to me.
And I tried to capture.
I've never been as good as he was.
Another professor who did this, but who read his lectures and was theatrically very powerful, taught with me at Yale.
And it was Bart Giamatti the man who became the President of Baseball.
First the President of the American League and then the Commissioner of Baseball.
And he was a great teacher, too.
And a great lecturer.
But he wrote most of his out.
And he used to practice his readings.
And I felt, even though his lectures were great, that they didn't match what Holland did, my undergraduate mentor, because Holland seemed really to be talking to me.
And yet when I, as a student, looked at his outline I realized that he'd given me a brilliantly coherent-- That was my ideal.
I'm not sure I actually fully ever realized the ideal.
And that may be part of the cost you pay for not writing it out.
Because what has happened-- I mean, I always give an outline to my students.
And I try to follow the outline.
But I also try to leave-- and I also plan and I pick my details.
When I'm lecturing in the film course I create clips that I show that I've sort of planned when I want to use the clips.
And they interrupt the flow of my discourse.
So this first sequence, it comes from Cops.
And in many ways I think it embodies that, kind of, a version of the trajectory , or crescendo joke I was talking about earlier.
It's a small version of the cannon joke that you see in, or jokes plural, because there's another joke with a cannon in the second half of the film that I've talked about earlier.
Let's show it Greg.
And of course this is the moment where Keaton is in flight from the police.
The cops are chasing him.
He runs up this teeter totter, right?
He runs up a ladder it turns out to be like a seesaw.
But when I am at my best I think I'm doing something that can't be captured in printed form.
Or couldn't be captured if you gave students a transcript of what you had done.
Which is that I'm modeling thinking.
I'm modeling how you make connections.
I'm even modeling discovery in a certain way.
Intellectual discovery.
Because by not actually writing the words out I sometimes bring the material together in a different way.
And I do make discoveries while I'm talking.
And that energizes me and makes me more powerful.
And I think that was another thing that I got from Larry Holland.
I realized that another aspect of what he was showing me was how not just intellectually engaged but how passionately engaged, how emotionally valuable studying literature could be.
And I tried in my lectures, not exactly consciously because I think I absorbed some of this unconscious.
Only in recent years that I've thought about it abstractly and I'm able to articulate it.
But I see now that Lawrence Holland was a great-- I had other great lecturers at Princeton.
He wasn't the only one.
But Holland was the one I remember best and I admire and I sometimes, when I remember to do this in the last lecture of the film course, I pay respects to him.
I'm getting sentimental about him.
I haven't seen him or talked to him in 30, 40 years.
In a certain sense that what I try to tell the students about what I do for the film courses is this.
I want them to watch-- the movie is the primary text.
And I see myself, in some sense, as playing the role of a kind of synthesizer or channel, which tries to give a kind of brief but extraordinarily concise account or summary of the best that has been said about this particular film.
Or the best that has been said about the historical processes that lead up to the making of the film.
In other words, because I don't expect my students to become film scholars or literary scholars.
In my literature courses I have the same ambition.
I imagine that the kind of student I want to reach is that of a literate citizen.
And I don't expect them to have the kind of knowledge that a scholar what have.
I try to play the role of someone who synthesizes aspects of the scholarship for them.
And that's also another reason why the lecture is what is important.
Because I could have the students read five scholarly books and synthesize them.
But I do that for them so that they have the time to look at the film and think about it seriously.
The environment of the classroom or the lecture room is a unique kind of community.
It depends on certain kinds of conventions that everyone accepts.
For example.
In the lecture students know they're not supposed to raise and interrupt you in the middle of your lecture.
And it's only a very aggressive or arrogant student who will do so.
A good lecturer will sometimes say, if you have questions I'll make time at the end.
Or in my classes what I try to do is say, look.
I'm going to give the first 15 minutes of my lecture next week to questions you might have about anything.
And I urge you to bring them up.
You need to do that, I think, if you're lecturing.
You shouldn't ever-- In my courses, of course, there's also, and I think is critical.
I don't believe in courses that are only lecture.
Even my course, which has two hours of lecture a week, has one hour of discussion.
Recitation.
I think it's very important for the students to take possession of the ideas that you talk about.
I think that students in a good class, I think you do create a kind of community of learning.
A community of inquiry.
It can be especially exciting in a discussion class where you actually talk about-- I try to tell students, you know, this is one of the few places in which you can get angry in a certain way.
And passionate in disagreement with someone and not offend them.
And that's the most exciting kind of class when you say, you know, you're completely wrong about that passage.
Knowing that the person you're saying that to won't be offended.
You're not attacking the person.
You're part of the passionate discourse of trying to understand the excitement of trying to understand the text.
Trying to understand what's some writer or artist has done.
That can be so enlivening.
And it can be truly a deeply communal experience.
It doesn't even work if it's not, in some sense, communal.
So I think even in a lecture there's a sense of interaction between you and your audience.
And if your audience is bored, if your audience is not listening, if they go online and start doing their homework.
Right?
Or their email.
You're a bad teacher and you shouldn't be doing it.
If you're arrogant enough to think that you want the students to pay attention to you for 50 minutes without interruption you better be organized and you better be good.
And I think a lot of professors are not.
And I think that they dishonor my profession when they do that.
I think we shouldn't have lecturers who don't really do their job properly.
But I think lecturers who do, do a uniquely valuable job.
They model things for students that they can't get themselves in discussion.
They show what real learning can be.
And what passionate engagement with learning can also be.
And that's the kind of defense I would give of the medium of the live lecture.
One of the central arguments of my film course-- and this is one of the ways it's different from a traditional film course in a university, in a cinema studies program, or even in an English department-- is that it sees the advent of television as a critical factor in the history of the movies, much more central and critical than most contemporary histories acknowledge.
Because what I try to show is that the function that the movies had in American society before the advent of television was the function that the novel had in the 19th Century in Europe, and the function that Shakespeare's public theater had in Shakespeare's day, a form of popular narrative that's articulated a kind of assumed or imagine consensus of values for the whole of the society.
That made it a culturally, and anthropologically, and socially much more important medium then a mere artistic medium, even though it's artistic quality remains very important.
So that one of the ironies of entertainment is it can become a space, especially public forms of entertainment in cultures, like Shakespeare's public theater, or like the public theater of the American movie system in the studio era, these public spaces can become spaces in which the body politic, the political and social community entertains ideas about its own nature, entertains, considers, speculates, holds in its mind accounts of its origins, stories of its values And what we can say then is that the space of entertainment becomes from in a certain angle in virtually all societies, a space of discourse, a space in which-- exactly because it's a space recognized as not real, as make believe, is therefore licensed or allowed to explore possibilities that might be too dangerous or too disturbing to explore in other way.
And I taught both television and film in this way.
In an ironic way, it's a deeply literary way to teach the medium.
So the argument ends up being that the movies after television, the movies after about 1970, are profoundly a different animal.
The old studio system is dead.
Movies have become much more single, individual deal-making events using great stars, and particularly bankable scriptwriters, and scripts, and so forth, nothing like the great factory for the making of stories, the story machine that the movies had been until the advent of television, and that television became as it took on that consensus function.
I think it's a powerful and central insight that helps to explain the tremendous changes that overtake American movies after, certainly, after 1970 and beyond.
Another way in which I think my film course is not fully a film course, even though every text we study is a movie, all the scholarship site is movie schol-- not all.
But some is movie scholarship.
And I try to teach them about the history of film in ways that I think are helpful to them.
But there is a profoundly literary dimension to my teaching.
And I think of as deeply as a literature course.
It's a course that, if a student takes, serves as a prerequisite for advanced subjects in literature as well as in media.
And my reasons for that is that I treat the problems one confronts in a film from the perspective of a professor of literature, from the perspective of one who's a specialist in narrative.
Remember, many of the narratives that I have taught all my life we're not printed texts like novels.
They wear all narratives, like Homer's Odyssey.
Or, they were dramatic presentations.
Like Shakespeare's plays.
Literature has never meant simply printed text.
And I think my field is narrative of all sorts.
And what I try to do is show them, among other things, certain kinds of linkages or continuities across different forms of narrative.
And I certainly try to make them see that movies in a certain era, television in another era, were like the literature of earlier eras.
And it makes my film course a much more centrally literary and humanistic subject than would be the case if it were taught in a more conventional kind of cinema studies sequence.
The film experience is, as I said, began many years ago, 38 or 39 years ago in the version that I teach.
And it has undergone almost a continuous process of transformation.
One of the ironies is that it began its career in an era before there was any technology available to support film study.
It made things actually very awkward.
But it was also in some way convenient for the professor.
What we used to have to do was show the film.
We used to-- and in fact, MIT belong to a group called the University Film Study Center.
A group of institutions, included Harvard and Yale.
And what we did was we had a library of 16 millimeter prints of classic films.
And we circulated them.
So you had to alter your syllabus so that the guy at Harvard was also able to do Keaton because there was only one copy of The General available.
So that was very complicated.
And but we all managed to do it.
And then at while I was at MIT, that system became moribund because videotape became available, and films became available on videotape.
We also begin to be able to project them on videotape.
So we didn't have the problem of trying to rent the movie at great expense, or to belong to the Film Study Center, and try to keep these fraying 16 millimeter prints in operation.
So that was a great boon.
It also allowed us for the first time to stop and start the text.
So it allowed for forms of close reading of the media that had never existed before.
And one of the things it exposed was the errors of older scholars and older critics, who were working from memory in describing films, because in order to write about a film in the old days, in the days when I began to teach film, you actually had to project a movie.
And they had scholarly versions of 16 millimeter and other kinds of projectors that would allow you to stop and start things so you could study things.
They were fairly expensive.
And you had to go to an archive that had the material film.
The advent of videotape transformed media study and transformed film study in ways that are still not fully clear because every film scholar, any place in the world, was able to get his hands on important text.
And that process is, of course, been elaborated an almost infinite number of times, a tremendous number of times over the past decade, so that today, people could go online and virtually get access, certainly to clips, and to many older films that were never available before.
And every university, and every film class has access to this material, and can even set up-- first, as we've finally done at MIT-- a system in which students can actually videostream films to their own laptops at will.
And that's another wrinkle because that's a very recent addition to what the film course did.
So in the beginning, we would have a communal viewing.
And it was the only time the students could look at the film.
They couldn't even get a second look at it.
Then we moved to a phase in which we continued to have the communal viewing.
But we made videotape versions of the films available on reserve for students who wanted to write papers or to review for exams.
And then we moved to a phase in which when DVDs were available, we would sometimes not only project the film from DVDs, but we would also make both tapes and video DVDs available to students for use after class.
And we would also allow students to borrow the DVDs and watch them in their own time, and not attend the communal viewing if they were unable to attend.
And, of course, that immediately set in a process, began a process of attrition, in which fewer and fewer students found time to come to the communal viewings.
This is partly a function of the scheduling crisis at MIT, and the busyness of our students.
If they have an option to do it in their own time, they will often do so, so that what had been a kind of sacred space in the course, the 8 to 10 evening slot that I often used to show the film experience text became less and less well attended.
And this had the also troubling effect of a slight-- ans I say maybe more than slight-- but at least a slight decline in the attendance at the evening lecture.
Because the way I taught the course was I had one lecture at 4 o'clock.
From 4:00 to 5:00, they would break for dinner or their sports activities.
At 7 o'clock, I would give an introductory lecture.
And at 8 o'clock, the communal screening would begin.
That communal moment before the advent of streaming and before it was easy for the students to get access to the text on their own, was a much more powerful moment than it has become.
And so we haven't figured out what to do.
We haven't exactly figured how the course can evolve in ways that can maximize the students opportunity to look at the material in their own time, but also maintain a sense that there are certain spaces ad times during the day that are devoted to our activity, just as there are for their physics lab, and just as there are for their chemistry lecture.
We've got to do that to maintain our parody with the sciences.
When I begin the film experience, I asked the students to do something they find increasingly difficult.
In the old days, this request wasn't necessary in quite the same way.
In the old days, before the proliferation of smartphones, and visual media all over the place, and everyone being able to make his own video and so forth, and the ubiquity of DVDs, before those days, I was able to tell the students, look, we live in an audio visual age.
We live in a televisual age.
I want you to think that stuff away.
I want you to think yourself back to an age before people understood that there is such a thing as a film.
And think about the original meaning of movie is.
Go back to the root meaning.
It's a metaphor that is a dead metaphor for us.
But it actually-- it tells you what the most fundamental aspect of the movies were for the original audiences, and why they were so fascinated by the capacity of this new technology to capture motion.
It seemed like, and it was, a tremendous new advance in technologies of the representation of human experience, because it captured humans in motion.
It captured the movements of the world.
And the first films, as I try to show my students, were so preoccupied by this wondrous capacity of the medium, that it's almost the only thing they filmed.
And you can see the evolution of the medium embodied in the way in which, after an initial profound fascination with the novelty of movement begins to wear off, other properties of the film medium begin to be discovered.
So I tell my students this principle.
And I ask them to try to think their way back into the attitude of these first viewers, who didn't know about the visual media, to play a kind of thought experiment.
Today, it's much harder to make them play that experiment because I have to tell them, as I do in that first lecture, think away your iPads.
Imagine a world without iPads.
Imagine a world without smartphones.
Imagine a world without instant communication.
And of course, this is much harder for the students to do.
And of course, at one level they're much more capable of processing audio visual information than I am, and then my parents' generation, and all of the generations older than they are, because they've grown up in an environment in which essentially they're surrounded 24/7 by audio visual stimuli, by audio visual signals.
Many people might see this as dangerous.
Perhaps it is.
But it develops new capacities.
And one can see this in the history of television, in which at a certain point, there are certain kinds of television programs that become too complex for older generations.
And only the younger-- the cutoff comes around 1980 with a show called Hill Street Blues, which older fans, the fans that belong to my parents' generation, couldn't watch because it had too many subplots.
And it jumped around too much.
But the younger generation that was used to going to the movies, and had been watching television, and had been watching MTV, or the origins of MTV, were able to understand this very easily.
And of course, that process continued on television very intensely sot that today, there are forms of television whose audio visual complexity is beyond the imagination of the people who were making programs for the medium in the 1950s.
One irony of this situation is that even though the students are very adept at processing this information, they are even more ignorant about the history of the medium, especially the history of movies, than ever.
And it's even harder today to find students who even remember the environment in which people had television sets that required you to change the channels, or who had television sets in which were confined to a single room, and which in order to watch it, you had to move into that room.
So the physical environment is changing so much.
And of course, the original space of the movies, the great communal theater, that's a dying space.
You have to try to remind the students about that.
And so the part of teaching the film course has become much more historical.
I'm David Thorburn.
I'm a Professor of Literature and Comparative Media.
What else?
I've taught the film experience for, I guess, 38 or 39 years at MIT.
And I've taught a lot of literature subjects as well.
The story of what made me into a film scholar or a media scholar is at least interesting to me and still surprises me.
I've told it a few times.
But it was actually a somewhat transforming experience in my life.
I had been teaching at Yale for seven or eight years.
And I was offered a chance to do a visitation at the University of California in Santa Barbara.
And for the first time in my life, I was asked to teach students who weren't the product of very elaborate private school educations or very powerful public school educations.
They were smart kids, but they were less well-read in some ways.
So I was trying to illustrate-- it was a literature course, Introduction to Literature, or Comedy, or something like that.
And I was trying to introduce the students to the idea of conventions, what it means when you begin a story, "once upon a time," and every story begins that way.
Or what it means when you say, "they lived happily ever after," to take the simplest example of what a convention is.
I was trying to illustrate how, in fact, we internalize these things, and we understand the narratives of our own time, in our blood, without even thinking about it.
And I was trying to illustrate this principle.
So what I needed to illustrate the principle were shared texts.
So I began by asking them-- this was in the 1970s, in the early 1970s.
And I asked the students there-- these are bright California public school students-- how many of you have ever seen a John Wayne movie?
Maybe 10% of the students-- in 1970.
It's not 2016 now.
Maybe 15% of the students raise their hand.
Then I said, well, how many of you have read Huckleberry Finn?
Maybe 30%.
I tried other novels, classic novels.
I tried Shakespeare plays.
Finally, I said, how many of you have seen All in the Family?
For those in the audience too young to know this, All in the Family was a television program-- a classic and transforming one-- that appeared in 1971 on American television and involved a working-class hero who had never-- such characters had never appeared on television before.
It was a very important program.
And, of course, 100% of the class had seen it.
And I began to take my examples from television programs.
I talked about the conventions of situation comedy as a way of explaining.
And I began to think about this.
And this wore on me-- it wore in on me as the term went on.
And what I realized was that the literature of our own day, the story forms that belong to my students and to the generation of kids I was teaching, were visual, not literary-- or at least, many of them were.
And the ones that were shared 100%, shared by everyone, was certainly televisual.
So I became interested in television as a narrative medium.
And then I came to realize, as I began to think more seriously about the role of television in American life-- remember, by 1970, American television was deeply embedded in American life.
It was the dominant form of entertainment and narrative in American life.
And in fact, this initial understanding, out of which my whole interest in media developed, has shaped-- I realize now, just in talking about this-- my way of teaching the course, the film experience-- and it's one of the things I think that makes the film experience unique.
Because I began thinking about television.
I was not a film scholar.
I was not a media scholar.
I was a literary scholar.
In fact, ironically, I was a specialist in the high modern period, a period of Joyce and Virginia Woolf and Picasso, right?
And I knew about what we might think of as-- I was an expert in art-- in forms of art-- that could be thought to be elitist, or at least non-accessible to the ordinary people.
And here I was confronting texts that everyone could understand and everyone shared.
We get to think about them more deeply and think more deeply about my own literary education, which was narrow in some sense.
As everyone in my generation was so educated, we were cut off from the historical implications of the texts we studied.
So I began to fill that in as well, as I began to learn more about television.
And what I realized, thinking about television, was I couldn't understand television without the movies.
So I then set on a sort of personal course of education, and I read very deeply in movie history and movie scholarship.
And I amplified my own knowledge of movies tremendously by constant viewing of movies, mostly because I was trying to understand television, ironically.
And then I began to develop a recognition of what a remarkable cultural phenomenon the movies were.
I began to generate great respect for the great movie scholars.
It was a great era, the late '60s and early '70s, for movie scholarship, because it had become mature by that time.
And the people writing movie criticism in the United States, certainly in the 1970s, were very interesting critics.
Critics and scholars, the equivalent of the literary scholarship that I was used to, that I aspired to write myself, and so forth.
So it was very exciting, intellectually, for me to have this experience.
When I got back to Yale, after my year's visitation, I introduced a course called Literature and Popular Culture in which I explore these ideas in greater depth and with greater intensity.
I read some bestsellers.
I ended the course by looking at The Honeymooners.
And I remember I was teaching the course in a room at Yale College called Linsly-Chittenden Hall, in which the great Shakespeare lecture course had been taught for generations.
And my course, Literature in Popular Culture was very popular with the students.
I had a couple of hundred students.
And I was lecturing from the same podium that legendary figures had lectured on Shakespeare about.
And one of these legendary professors, the man who had actually hired me at Yale, Maynard Mack, a great literary scholar, poked his head in while I was teaching The Honeymooners.
And I had actually dragooned the medial people at Yale to give me a reel-to-reel tape recorder so I could tape the stuff off the air.
Mack saw this and later called me into his office and expressed great disappointment over the fact that I was polluting this environment.
So we then had a serious conversation.
And I said to him, what did he actually think Shakespeare's environment was?
And that was when it struck me how narrow my education had been.
Because he really had not-- even though he was a great man-- he knew the history.
But he somehow had managed to disconnect the literary power of Shakespeare from the fact that it was done on this incredibly popular stage, and it was so sensational, and it was so much more like television or the movies than it was like some process of sitting in a library in wood paneled splendor reading words in silence-- that's not what Shakespeare was.
And that began to lead me into recognitions of the importance of popular culture, the fact that most high culture was a form of popular culture.
I became much less apologetic than most teachers of popular culture are.
And so when I came to MIT in the mid-- I came to MIT in 1976.
And I was already in the throes of this.
But it was, I think, fortunate for me and fortunate for MIT, in one sense, because MIT's literature program was much more open than Yale's had been.
And they were interested in my work on television and media.
That was one reason they wanted me to come here.
And it was much less hidebound about literary and historical study here.
So it gave me a chance to-- and I introduced a course in television.
And that was one of the very first courses in an the American Academy that studied television programs for their content.
Now, that's common.
It's thought to be-- of course, everybody takes it for granted.
But at that time, it was a very rare thing.
And what I also did was take a course that I inherited from my wonderful colleague-- still teaching here in his 80's-- Alvin Kibel, a course called the film experience which was of a course, that was not intended for all students but a small one I transformed it into a general introduction to film and I've taught it almost every year since.
One of the things I aim for in the course, is to try to show students, in ways as concrete and rich as I can, what the authority, or power, of movies in a cultural moment might be.
And I try to give them some examples of that.
In my first lecture, I sometimes read to them from a passage from James Agee's novel, A Death in the Family.
In the very beginning of the novel he describes, beautifully, an excursion between a father and son, going to see a Chaplin comedy in 1915 in some southern city.
And what he describes is what a social occasion it was, what a moment of bonding it was for father and son, what a cultural moment it was more broadly for the whole of the population that went into the theater to laugh uproariously at Charlie's shenanigans.
"And then the screen was filled with a city, and with a sidewalk of a side street of a city, and a long line of palms.
And there was Charlie.
Everyone laughed the minute they saw him squattily walking, with his toes out and his knees apart, as if he were chafed.
Rufus's father laughed, and Rufus laughed, too.
This time, Charlie stole a whole bag."
This time.
What does that imply about the audience?
An intimate familiarity with the previous adventures of this character, right?
An ongoing, routine connection.
"This time he caught a sight of-- this time Charlie stole a whole bag of eggs.
And when a cop came along, he hid them in the seat of his pants."
What it showed was the cultural embeddedness of movies, even in their earliest days, in the society, and in the life of individuals.
So, part of what I mean by the power of movies, and what I try to encourage the students to see, is something of this anthropological energy, as well as its artistic power as a representation of human experience.
That the way these kinds of stories permeated the society helped to shape America's-- Americans understanding of their natures.
Of what the social fabric was like.
Of what masculinity was.
Of what femininity was.
Of what families were.
Of what the relations among the races were.
Of what American history was.
About what the founding story of America is.
About what the central, organizing values of our culture are.
All of those things are dramatized, either explicitly or implicitly, in movies.
And because the movies were so central through the 20th century, they were one of the central ways in which the belief system, the values, of American society were promulgated, dramatized, rehearsed, and in some ways, altered and changed.
A related matter that connected again to film, in part as an anthropological or sociological phenomenon, not simply an artistic one, as a social practice in a way, has to do with the power and role of genre in movies, which is a recurring theme in my course.
Because we use genre in American society.
I think all cultures do this, but we've made particularly rich and cunning use of genre in American society.
In many media, but especially in film.
And if you look at particular long live genre, like the Western, or the detective story, or the domestic melodrama, over time what you of course can recognize are changing cultural and social attitudes.
So, the Western has become a screen on which America's anxieties about the Vietnam War have been projected.
Why would the Western work so well for this?
One answer is what I've been saying all along about the power of genre, and the power of repetition.
Look if a thing is repeated again, and again, and again, it looks familiar.
When it's so familiar, what happens?
It licenses something disturbing Because the genre, it seems on the surface, to contain so many familiar, reassuring elements, those very elements of reassurance enable the exploration of disturbing, or uncertain, or problematic materials.
And of course, because-- not any one of these films would have nothing like the power they actually have if they existed individually.
But it's because they're part of this long conversation that goes back to the earliest days of movies, that they have the power that they have.
One of the functions of genre is one of the deep subjects of my course.
And I want students to see how centrally variations in genre forms can reflect social and historical realities.
And you can feel this in every film.
You can feel this in every television program.
If you look at American television from the 1950s to the 1990s, let's say you've restricted yourself to situation comedy, what do you think you would see?
You would see a drama in which the American family undergoes profound change.
What happens?
Women become more important.
Patriarchal values become less, and less powerful.
Children become more respected-- are treated less like appendages to the family.
And in some-- by the '80s and '90s, sometimes the children have more dominant roles than the adults, and are often even shown to be wiser than their parents.
Right?
Something that would have been impossible in the early 1950s, a more patriarchal era, when the classic television program about families was called, Father Knows Best.
Another spine question, or organizing key question in the course, is deeply literary for me.
In other words, what I try to do here, is encourage the students to develop what I'll call an aesthetic sense.
To begin to have confidence when they say, that's a good film, or that's a bad film.
I think that's a good movie.
I think that's a bad movie.
I think that's a good-- and what I try to tell them is that that judgment, good or bad, isn't merely subjective.
It isn't just what you feel, right?
That there may be potentially objective standards, by which we can judge the excellence of texts.
Part of it may remain subjective, but we can still judge excellence when we see acting.
We can judge excellence when we ask how well the film is edited.
We can judge the excellence of various aspects of the film.
And teaching students how to do that, and to distinguish between effective and ineffective uses of the medium, turns out to be not simply a skill that you give to students who are watching movies, it's a skill you can give to students who read poems, who read books, who go to plays.
So one of the central principles I try to teach them about this, how they can distinguish what I call a work of art from a mere entertainment, is what I call the multiplicity principle.
And what I mean by the multiplicity principle is fairly clear.
And I give many examples in the course.
Essentially, it's the idea that what happens in a particular moment will have multiple functions.
You won't just look at the scene in order to have the plot extended.
The scene will also dramatize character.
It will also tell you something about the atmosphere in which the character lives.
Do you remember the moment in the passage I showed you, from the De Sica movie, Umberto D.?
That very brief moment, where we see a man get on a tram, a bus, and sit next to someone on the bus and the bus moves.
It's a completely wordless sequence.
And there's a character we've never seen in the field before, who sits down next to the protagonist.
No words exchanged.
When our protagonist gets up, and gets off the bus, the camera lingers briefly on the man he left behind.
And we see that men go like this.
And what I suggested to you, again, I suggested that this was a moment of richness and complexity of multiplicity, in a way, because one implication of that scene was that the man, this stranger that we'll never meet again, that we only saw for one brief instant in the course of the film, might have concealed a story as deep, as rich, and as moving, as that of our hero, Humberto D. When a text contains-- when a moment contains multiplicity, when it many things simultaneously, you're in the presence of a kind of excellence.
That's only one principle, but it's a very valuable principle.
And you can see it's worth if you compare, let's say, a trivial or relatively banal form of action entertainment, on the one hand, with a movie that has complex characters in it.
And you could see the different kinds of demands.
Or, even that a serious action adventure film, a film that incorporates action adventure elements in it, but also has a serious subject matter, to ones that don't.
Let's say, Kurosawa's samurai movies, against the trivial, merely entertaining samurai movies that weren't interested in history or character.
What you try to do is give the students a greater and greater sensitivity, not just to the particular text that exhibits such excellence, but you want to arm them, so that when they encounter things on their own, they'll be able to make this judgment.
A second aspect of this excellence, what I call artistic quality, is the quality of organic form.
I talk about this again, and again in the course, and try to show them instances of it.
At its simplest level, the term, of course, derives from Coleridge, who borrowed it from the German romantics.
But essentially, it's the idea that every text should generate a form that is unique to its needs.
Form should be-- in other words, you should not have a prior form into which content is poured, right?
The form of the text, the shape of the text, should take it nature from the meaning of the text.
From what the text is saying.
And if you think about the structure of modern times, you'll see that it's a marvelously distilled instance-- a marvelously clear instance of the principle of organic form.
What happens to Charlie serially is more meaningful, in a way, than what happens to him in a single episode.
And the fact that this-- that we have this sense that things are repeating, that Charlie is on a treadmill, that he lives a cyclical life, that he'll always have moments of hope and hopelessness, that he'll have momentary times when he's free of-- when he finds a place to live, or finds sufficient food, but that it will always be temporary.
That his life is always on the road.
That his life is always in process.
And, that at the same time, he never lets himself completely despair.
There's always a moment where his resilience reasserts itself.
Well, those elements are embedded, in some sense, in the very structure, the very organization, of the film.
There should be a symbiotic relationship, a marriage, a linkage, between form and content.
Between the way, the shape of the text.
Between the strategies and techniques that are used to embody the story, and what the story says.
There should be a marriage between form and content.
That's what organic form is.
If you can show students moments of organic form, and if they can really recognize it, they are proofed for life against banal art.
In thinking about how the course has to evolve given the availability of all this new technology, one thing I've especially thought about-- worried about, actually-- is how much of my own lecture material, which I now have on video to make available to them, as against my live performance, and whether my live performance is even necessary at all.
How does it differ from-- and I still haven't worked that out.
For two semesters now, I have taught the course with a combination of live lectures and video lectures.
And I've asked the students to tell me in commentaries whether they saw much difference, and whether they preferred one to the other, or whether they had strong feelings.
And so far, the testimony has been, meh.
They don't seem too interested in the question, which makes me a little nervous, actually.
So my feeling is that it's still an open question.
One thing I definitely have decided I will be able to do with this video material is using it in other courses.
If I an in advanced course in film, and students haven't taken my earlier film course, I can send them to this material and have them use it.
So it's immensely valuable as a teaching tool.
But how to integrate it into the existing course remains a conundrum I haven't quite figured out.
As I've begun to work on this, now it's been some years since I've started to have some video lectures available.
I think I realize that I was naive in imagining that the transition would be easy, that moving into a video format was simply a matter of transferring what I say, because in fact, the live environment turns out to be more central and more important.
And my feelings as I'm in the life environment turn out to be more essential and more important than they might otherwise be.
There's something about being in a live environment that affects me, and I think other teachers, maybe not all teachers, but other teachers very deeply.
I used to have the fantasy when I was in front of a large audience of students, maybe as many as 100 or 150, I had the delusion or illusion that I was in touch with every student in the class, and that every single one of them was paying attention, and than when they didn't, I could feel it, and that I could see there was a student who would stop paying attention.
I would somehow walk in that direction of the class or get that student's attention.
Now, I'm sure it was a fantasy because how could you be aware of 100 students?
But I had the illusion that I was aware of them.
I was certainly aware of many of them.
I was aware of them as individuals.
I was aware of whethet-- and I felt I had them wrapped.
And when I felt I was losing them, I would change pace.
Maybe it has something to do with a theatrical aspect to me that really doesn't help much with being a good teacher.
But maybe it does help with being a good teacher, and that maybe good teachers do have to have a bit of the ham in them and like being in front of audiences.
This is a special problem for the Western, because it celebrates American individualism so powerfully.
The iconic scene of the Western in which two men wearing gun square off against each other, and the quicker one survives, that scene of the quick draw, and of the [?
dif ?]
and the dusty street, can be said to be a kind of embodiment of these values in which you have to rely on yourself.
You can't rely on someone else.
You can't rely on the government.
You can't rely on the Sheriff.
You can't rely on the police.
You can't rely on your neighbors.
You have to trust yourself.
And in fact, one of the ways you can see the power of these Western mythologies operating, as it seems to me in some sense, one of the great divides in the States today over the question of guns and gun control is partly grounded in these myths of individualism, that the Western mythology has been promulgating since before the advent of the movies.
And I think part of what makes my teaching good is that I'm in a kind of trance when I'm in front of a class, in the sense that I am utterly oblivious to what I look like, I know for one thing.
I have done teachers nightmare kinds of things, like having my shirt unbuttoned without realizing it, or coming in with my hair incredibly askew without knowing about it.
Once I gave a lecture at Yale.
I had young children at the time.
And I had a child.
My wife was with the two other children.
I had to take a child to a doctor.
We took him to the doctor.
But I had to go to give my lecture.
So I took him into the lecture hall.
And I sat him down in the corner of the lecture.
And he was about seven years old.
And he sat nicely in the seat for a little while.
I wasn't paying attention to him.
Then he climbed up.
I was lecturing on a stage.
And he climbed up on the stage.
And he went behind me.
And he sat in a big chair behind me.
And he made faces.
And the students begin to laugh.
And I couldn't understand what was going on.
But it taught me a great lesson in the theatrical reality of the lecture hall.
So I've come more fully to understand that things that are hard to articulate or that I at least had not much thought about, certainly have not been written about in pedagogical literature, about the ambiance of the live classroom has some effect on lecturing.
I mean it should have effect in smaller classrooms even more fully, I think.
But for lectures, I think it's important too.
And the transition to purely video lectures loses that.
How big a loss that is, or whether that loss can be compensated for in other ways is a good question.
One of the ways that the study of literature and the study of film differs from the study of technical things, and the reason I teach it, is that it belongs to everyone, that it's valuable for everyone.
Not everyone needs to know about quantum mechanics.
But I believe everyone should.
And if they don't, I feel they're impoverished.
Know how to read a good story, enjoy plays, know how to enjoy the movies.
So I feel that what I'm giving my students is, in some sense, something even more valuable to them because it will be something that they'll have with them for their whole lives and their whole careers.
One of the things MIT students often do not realize, is that a very large number of them do not end up making their living in the areas in which they majored.
But all of them end up wanting to go to the movies.
All of them end up with the capacity to read and enjoy literature.
And I hope that coming out of my classes, they'll do those things with greater joy and with greater intelligence.
One of the things I hope all of you will have when you come out of this course, is a much more confident sense of how to look at movies, not to spoil your enjoyment of kicking back and looking at a stupid Schwarzenegger entertainment if you want to, not to make you feel guilty about that, but to make you be able to tell the difference between something like that and what I'll call a work of art.
Implicit in my film course is an admiration for a certain kind of achievement, and admiration for a certain kind of artist, and artist like Jean Renoir, the great filmmaker.
If I were deprived of the pleasure of seeing Boudu again for the rest of my days, I would never forget that grass, that dust, and their relationship to the liberty of a tramp.
The point of this exercise is to remind you of the immense power, the potency, of even a single camera move.
Think what that 180 degree pan suggests as Bazin brilliantly argues for us.
So the conclusion then is that the visual style of a film, or of certain films anyway, can express a moral vision.
And by moral vision, I don't mean moralistic, what's didactic right and wrong, but a vision of having to do with the values and assumptions you make about the nature of the world.
There's a moral vision implicit in the tentativeness, the hesitancy, the retarding impulse to dwell and linger on things in Renoir's camera, and in the basic habits of poetic realism that you will see brilliantly embodied in the film you're going to watch tonight, Grand Illusion.
Because I want to set up, as a candidate for their admiration, and alternative to Wall Street, an alternative to entrepreneurial genius.
I admire Jean Renoir's genius, Or Orson Welle's genius, or James Joyce's genius universes more than I admire what entrepreneurs-- I respect what entrepreneurs do.
But I am in awe of what great artists do, and of what great doctors do, while we're on the subject.
Which, I'm uneasy about the extent to which even within MIT's culture, we've become so preoccupied by what we might call commercial or financial success, instead of the kind of lasting success that great art, or the practice of medicine, or the practice of nursing, or dare I say it, the practice of teaching, might also embody.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality, educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to begin by asking what seems an obvious question, what is film?
I used to sometimes present it by saying, film as, dot, dot, dot.
Film as what?
And it may be surprising to you, but one way we could think about film is as chemistry.
Now how could that make sense?
Why would it make sense to think of film as a form of chemistry?
What this film got to do with chemistry?
In fact it's a very fundamental relation.
This is also true of still photography, as well as movies.
But what's the process by which they're made?
Yes
Film really comes together from a--
Speak loudly so everybody can
Film is made up of a lot of different components.
You have your lighting, your scene, your character.
And that all has to come together to make film.
Without even on component of it, you're losing part of the experience
You're right about that.
But that's a more general answer than I wanted.
There's something much more dramatically fundamental about the way, about the connection between chemistry and movies.
What is it?
The interaction between the audience and
It doesn't have to do with the experience of movies.
Come on.
It's technical
Chemistry had to be developed before you could have it
It depends on chemistry.
Film is a form of applied chemistry.
Why?
Of course you're right
[INAUDIBLE]
Yes.
What is a film?
There are certain emulsions that are put on piece of celluloid.
Light actually has to normally-- it can be any light, but sunlight is best-- act, interacting with those emulsions, causes the image to appear.
The actual, fundamental process by which film is physically created is a chemical process.
And if you reflect on for a moment, one of the things this suggests is that then when we think about film in a much larger sense, in the film as we experience in theaters, film, as an engine of economic development, as a provider of jobs, and careers, and so forth.
What we could say is that it is a form of applied chemistry that is among the most profound uses of chemistry that humankind has ever found.
Because if you think about the impact of movies on human life, it is now a global phenomenon.
Is there any culture that is free of movies?
Maybe there are Taliban cultures that dream of being free of movies.
But to my knowledge there's no culture in the world now that's completely oblivious to film.
It's become a global phenomenon.
And it's more than a century old.
It is the distinctive, narrative form of the 20th century, the signature form of storytelling for the 20th century.
All of it derives from this chemical reaction, when the emulsions are subjected to light, the image appears on the celluloid.
There are even theoreticians of movies who have suggested that there's a fundamental break of a kind that is subliminal, unless obvious to many people.
But it's fundamental to our experience of text when we moved from real film to digital forms of filmmaking.
Because nature is eliminated in digital form.
There's something natural, and in fact slow, about the way when light works on those emulsions to bring the images up.
And those of you who are amateur photographers will know that you can control the clarity or the blurriness of the image, the darkest of the lightness of the image, by how long you leave the film paper in the emulsions.
You can control it, and still photographers and creative movie directors actually use those use that chemical principle in order to create certain kinds of effects.
So one way to think about movies is to think of it as a form of applied chemistry.
And one of the most profound uses of chemistry that we could imagine in terms of its impact on society, in terms of the vast number of people who have been affected, and continue to be affected by this invention.
So film is a form of chemistry.
What I'm suggesting, these different framings of what film is, these different frameworks for understanding film.
One thing I'm trying to do is to suggest some of the ways in which we might understand film apart from what we're going to be doing in this course.
Now I don't know if one could justify persuading a professor of chemistry to teach a course in film.
That might be going too far.
But certain broad principles of photography, and how they are linked to other photochemical processes, might very well make a quite exciting and complicated course in the chemistry department.
One can also think of film simply in a historical sense, certainly, as a kind of novelty.
When film first appeared in the world, and especially in the United States, it was seen as a novelty that caused its first appearance to take place in places like penny arcades.
Where people went to experience other kinds of public novelties as well.
there would be machines in these penny arcades that would guess you're weight.
And you put a penny in, if the machine was right it kept your penny, if the machine was wrong would give your penny back.
There were fortune telling machines in these penny arcades.
In some of the more sleazy ones there were live peep shows.
Strip shows of various moderate kinds.
And of course, even at the very early stages, film begin to replicate those live performances.
There were very trivial forms of burlesque began to-- women stripping-- I don't think there were any male strippers in this late Victorian era-- began to appear in the penny arcades as well.
So one could say that film in its earliest stages was also just a kind of novelty item, like a PEZ dispenser or some equivalent kind of silly thing, or baseball cards, or football cards, that kind of thing.
More profoundly of course, we could think of film from another [INAUDIBLE], a manufactured object.
And this identity of film is incredibly important.
It's again, easy for us to forget, because when we go to the movies today, we see these complex and overwhelming-- we have these complex and overwhelming audiovisual experiences.
And we might tend to forget what in fact is the sort of industrial base on which movies were made at a relatively early stage.
Part of what we want at least to be aware of in our course, even though we won't study it systematically, is the fact that the movies, the film, is one of the first significant commodities to become a mass-produced item.
And in fact, the same principles that led to another manufacturing miracle that we associate with the late 19th and early 20th century-- the automobile-- the same principles that went to the production of the automobile also worked in the production of film.
And in fact, both film and the automobile could be seen as prototypical instances of this fundamentally defining industrial capitalist behavior, capitalist activity, which is mass production.
And especially, what does mass production depend upon?
The specialization of labor.
The rationalizing of the production process into smaller, and smaller units.
So that particular people can do it quickly, and you can create essentially an assembly line production.
You can create mass production.
I still find it very inspiring and important, significant, the notion that film was created on an assembly line, just like toasters or automobiles.
Seems a shocking and important insight.
Because they're still in some fundamental way produced like this.
I don't mean that the same movie studios are churning out 500 movies a year, which is what was churned out during the great era of the Hollywood Studios, from around 1930 through the end of the 1940s.
But the fact is the production of movies, the manufacture of movies still depends on these principles of the specialization of labor.
And I'm not simply talking about the way in which we have actors, and directors, and cinematographers, and grips, and best boys, and set dressers, and makeup people, and script writers, and so forth.
All are relevant to this.
But I'm also talking about the way in which movies, still to this day, are divided in their production principles in three stages, a pre-production phase, a production phase, and a post-production phase.
And there are specialists at each level, on each phase.
And a vast army of specialist is hired to handle the problems that are connected to the production of every single film.
So we can think of films as a really distinctive, signature, instance of what mass production is capable of.
OK.
So we can say that the film is a manufactured object.
And not just a manufactured object, but a product of mass production, a product of essentially, assembly line principles.
And what makes this so remarkable to me, still an idea that I have trouble fully absorbing is that the mass-produced item that we're talking about, unlike a toaster or even an automobile, managed so fully to permeate our society and our world, that it's infiltrated ourselves even into our dreams and our fantasy life.
And finally, another way to think about film, and I'm going to sort of enlarge in that.
And this is a way we'll be talking about quite a lot in the course of our discussions in this semester.
We can say that film, after it's elaborated, and established itself in culture, it becomes a fundamental social form, a fundamental social formation.
And experienced, widely practiced, widely indulged in by a vast number of people in the society.
So that one could say for example, toasters are important, but they don't generate the kind of social rituals that are involved in going to the movies, and of identifying with movie stars, and of generating fans surround movies, and ancillary, complex activities that we associate with movie going.
And in fact, one might say that the great era of movie going is already gone.
That it was really in the era of the Hollywood Studios when they were before the internet and before television.
So we could also think of the film as a social form.
Not when it first appears, when it's just a novelty, but after it goes through various phases.
When it embeds itself into the society the way the movies did, it becomes a kind of social form.
And one might almost argue that it becomes one of the most important social forms in the society because it's so widely shared.
Most social activities in the society are relatively limited in the circle of people they involve.
Even the number of automobile drivers is controlled in a way that is contained or demarcated in a way that's less true.
Movies appeal to children as well as adults.
And from there very beginning this has been true of movies, especially in the United States.
They've appealed across lines of social stratification, across differences of gender, across differences of age, across differences of race.
There's one book on the film, a rather overly optimistic one that simplifies the pernicious or sinister aspect of movies called "Film -- The Democratic Art.
And you can understand, even though I think it's a simplification, why that's an interesting way to think about movies.
Because it reached so widely across so many social barriers.
In that sense, film was the narrative form that reached a wider audience than any other narrative system that had been invented by human beings before it.
What's one explanation for why film would be so appealing, be even more appealing than printed narrative?
It also is connected to why the movies grew so quickly in their infancy.
Why they went from being a novelty to being an embedded social form so quickly.
You probably know what the answer is
That more people could see than could read
Yes, that's the real answer, isn't it.
That it's mostly a visual medium.
It doesn't depend on language to the same degree.
And especially silent film, which did not depend-- which it did depend on language, it used intertitles and things, but it depended on language minimally.
Why was this important?
Because when the film was in its infancy, there was also-- and this was a fundamental, enabling condition for the development of movies-- there was also in the United States a vast and growing immigrant population in all the major cities, but especially Chicago, New York, some of the other larger, industrial cities.
And this new immigrant population, which many of them didn't know English at all, had only a little bit of money, disposable income, but they needed entertainment.
And the silent film was the perfect answer to this.
The film, relatively quickly becomes a profoundly embedded social form.
To give you some sense of how monumental, and how central, how important this was in American society, what I can just simply remind you of is that for most of the period in which the Hollywood Studios were operating at full power, roughly the period from the advent of sound film in the late '20s, until the late '40s when television intervened, even though there's a period when television is around when the movie studios retain something of their old character, but they begin to decline without fully realizing if, that occurs in the mid '50s sometime.
So in the period from roughly 1930 to say, 1955, to be crude about it.
In this period, the vast majority of Americans went to the movies every single week.
Think about that, every single week.
This was before television, which supplanted that quality of movies.
In 1947 or 1948, 80 million Americans went to the movies every week, every week.
That was like 2/3 of the population at that time.
So it was near to being a universal experience.
And not just the universal experience that occurred occasionally, but a routine experience.
An experience that families, and individuals, and young, and old, had regularly, as a fundamental part of their life, as a part of their ordinary experience.
That's what I mean by an embedded social form.
That's an immensely important fact about the movies, and especially about the classic movies.
And when that feature, the idea of the movies as something routine in people's lives, something they did regularly, not occasionally.
When that feature disappears, it disappears in part because of the impact of television on society.
Television's in the house, so it's a lot easier to retain, to establish an habitual relation to television than it is to the movie.
So the advent of this new technology changed movies relation to its audience.
And this is the fact that to which again we will return again, and again.
So we could also frame movies in other ways.
But these framings I think, are helpful to us in part to remind us of some things that I'm not going to do.
One could certainly imagine a course in the Department of Economics that looks simply at the film as an economic engine, at the number of jobs created by movies.
Not just the immediate jobs, the people who are actually producing the film, but another kind of a accounting that would take account of all the ancillary jobs.
The theater owners, the popcorn sellers, the people who create the publicity for movies.
The whole entourage of hangers-on-- you're thinking of the TV show, aren't you?
The whole entourage of hangers-on that follow the movie stars around.
It's an unbelievable engine of economic development and economic growth.
And it is arguable, given the fact that the movies had been a dominant industry in the Western world, and especially in the United States, since the early 20th century, one could make an argument that it's one of the most productive and central engines of economic growth that capitalism has ever developed.
And one could teach a course in the movies that simply emphasized its economic aspects, its power as an employer, its role as a generator of wealth, as a generator of resources.
And I mention this partly to clarify the extent to which, in our course, we're focusing on only aspects of what film might be.
But also, in order to remind you that we need to be aware of this as a backdrop to the more cultural and aesthetic concerns about the content of movies, and about the way they developed, the way they evolved, that will be the central energies we will be committed to in our course.
Well, one thing you need to do in order to experience these first weeks of this course in a really effective way is to try to in a certain sense, get outside of your own head, get outside of your own skin.
We live in such a visually saturated environment.
In which audio/visual messages are beamed at us constantly.
Some of us are connected to apparatus all the time.
We're connected to our cell phones.
We're connected to our computers.
It's almost as if we have audio visual signals bombarding us 24/7.
It's almost as if you were entering a cave, imagine that you're entering a cave.
What I want to do is think away your iPods.
Think away your cell phone.
Literally, think them away.
Imagine a world without them.
Imagine a world without movies, a world without television, a world without radio.
I want you to imaginatively put yourself back into the era when the first films began to appear.
And try to recover some of the excitement and wonder that those earliest audiences must have felt when they saw some of these images.
The most important thing in some sense would be that they were immensely amazed, they were taken aback by the simple, shocking, wonder of movement captured on film.
It was if movement itself, something they associate with reality, could suddenly be recaptured in film.
Can we show some examples of this, Greg?
Some of you have seen one example of this.
Remember in the recitation section, when you saw The Great Train Robbery.
Do you remember the moment in The Great Train Robbery-- which seems unconnected to the story-- in some prints of the film it comes at the end.
In some prints it came at the beginning.
In some prints it didn't occur at all.
Where was it in the film you saw?
Is it the guy?
Yes, yes.
When was that?
It was at the very end
That was at the very end.
It's the moment where the guy pulls out his gun, and points it at the camera and shoots him.
Did any of you find that odd?
I mean, I think you should have.
One reason is that it had nothing to do with the narrative.
This is important film.
It used to be thought to be the very first story film, the very first systematic narrative on film.
It's not that, there are earlier examples.
But it's one of the very first, and that's why I wanted you see it.
It's one of the earliest story films.
It's one of the earliest film to tell a sequential story.
Although it seems very primitive to you guys, it's actually a very sophisticated item.
And there has been a lot of filmmaking going on before this film was made 1902.
We'll talk a little bit about that in a moment.
So this moment where he shoots the gun at the camera, why is that disturbing to us or strange to us?
I'll answer my own question.
One, it's disturbing to us because it breaks the narrative.
It seems unconnected to the narrative.
Why would they do it?
And then second, what's going on there?
Why is he doing it?
Does the filmmaker not like his audience?
What's the reason for it?
Why do you think it's there?
Who has an idea?
Why would be there?
Again enter the cave.
Think yourself back to an era before movie.
When people had never seen movies before.
What's the answer?
He couldn't engage the audience in the film
Well, he does engage them, but what else what does it do?
What does it call attention to?
It puts him into a situation that they normally would not encounter, and probably would not survive
OK, that's right.
Yes, a way that's right.
And in fact, there are a lot of accounts of early films playing these kinds of tricks, and audiences not yet sure of what films were, reacting as if they were looking at something real.
So there are at least reports of people seeing this film when the guy shot the gun, people screaming and ducking down under their seats.
And there are many stories like this about early films where people would come to the-- and that's why they showed it.
They showed it because it's a very dramatic way of dramatizing, of crystallizing the difference between reality and movies.
And also, how realistic movies can be.
The movie's name's the most fundamental feature of the movies.
It's a dead metaphor for us.
We don't even think about it when we say movie.
But think what it means, it means movement.
Films capture movement.
What don't you just show this in sequence while I talk, Greg, OK.
These are a sequence of early films.
And you can see that all of them have in common is a fascination with motion.
In other words, the novelty of motion was so great in the beginning that the earliest film simply did this.
There's an important principle here about the way all media developed.
In their infancy, the first thing that happens is that no one really much knows how a particular medium we should be or could be developed.
And part of the reason for these early weeks in the film for the first two or three weeks in this course is to put you back into that situation.
To try to in a very crystallized and distilled way, because there are thousands of films made in this era, but in a very distilled and crystallized way, I'm trying to recapture some of that excitement for you.
And in the case of both Chaplin and Keaton, what I've done is choose some short films they made earlier, and then show you a feature film.
So if you watch the Chaplain, Keaton films in sequence, two shorts and then a feature film, you'll see enacted in a kind of small compass within the terms of a single director's career this larger process that I'm saying was also enacted by movies themselves.
It was this period of the silent era, was a period in which the movies discovered their identity, or such identity as they have.
And I want to talk a bit more about that.
While all of these early, relatively primitive films show us is a kind of-- this is one of the earliest-- some people call this the first comedy film.
The first joke in the movies.
The simplicity of it seems to us weird.
But if you think yourself back, if you go into the cave, try to imagine a universe without audio/visual stimuli, you could begin to understand why some of this stuff was so interesting.
Motion itself captivated early audiences and filmmakers.
Waves on the shore for example.
There are a number of films that just show waves lapping on the shore.
Many films of trains coming into stations, and of course, this Fred Ott Sneeze.
Have we shown the kiss, or the electrocute?
Can we do that?
There were also risque or anarchic elements that showed up in early film that I want you see.
Here is one of the most famous and scandalous of early films, something called The Kiss.
And it was an unbelievable scandal when it came out.
It films a scene from a stage play.
Look how short it was.
It was banned in many cities.
It was thought to be scandalous.
Do you have the electrocute?
Many people would call this the first snuff film.
And this is a film called The Electrocution of an Elephant.
And think again what's going on here.
Part of it has to do with this wonder that the film can capture actuality, in a way.
There's something bizarre and gross about this in some way.
This was a rogue elephant that was about to be put to death
She had apparently stepped on a couple of handlers and killed them
So they were going to electrocute the elephant, and they brought a camera there to witness the electrocution.
And of course it's so dark, because this fragment of film survives from over 100 years ago, more than 100 years.
He's supposed to collapse not to stand up, isn't he?
So there it is.
[AUDIENCE GASPS] Grotesque, isn't it?
But also a part of early film.
This idea that film could capture reality was itself so-- the novelty of this was so powerful, that in the early stages this was enough to cause great excitement for audiences.
One way to think about this problem, and to think about-- the way I have for encapsulating, or dramatizing in a kind of a distilled way, all the elaborate processes that went into the development or the evolution of film is by-- the way I do this in part is by reference to old Fred.
This film used to be thought to be the very first film, it's not actually.
But it is one of the earliest films made and patented in Edison's movie studio, the very first movie studio in the United States in East Orange, New Jersey.
Probably, this made in 1894, at least the copyright I think is of Fred Ott's Sneeze is in 1894.
And think of how simple it is, how ridiculous it is.
A camera sets up, they were still working on the technology of the motion picture camera at this stage, and they were testing it out.
And Fred Ott was an employee of the Edison Company.
And they said different, OK, Fred, stand in front of the camera and take snuff.
And that's what he's doing.
We might think of Fred Ott's Sneeze as, theoretically, symbolically, the first film, even though there are earlier films.
Well, think of how unbelievably simple it is.
Shown it once more, Greg.
Can you freeze it?
Oh, yeah.
Hold on.
I'll get it
So that Fred stays on the screen while we're talking about this.
Look how short it is.
What is it?
It's two seconds long.
Think of this.
So this film is made in whatever it is, 1894, 1895, this two-second long film was made in 1894, 1895, by the 1920s, astonishingly complex narrative films are being made, great works of art are being made.
What I call the Fred Ott principle is this whole complex, social, and technical, and technological, and artistic process, the swirling of all of these energies going together.
Including the demographic fact of so many immigrants audiences creating an environment that made early film a very profitable activity, despite how crude it was.
What happens in this period between 1900 and the end of the '20s is that film goes from being a novelty separated off, sharing a space in the penny arcades with other forms of novelty, to being one of the dominant industries, and one of the dominant social experiences of the American population.
This principle is replicated in some other European societies as well, but not in all.
It's an advanced capitalist event.
And it occurs in other societies less fully industrialized at later stages.
But there is an equivalent history in some of the European cultures.
So in this period of fewer than 30 years, film goes from being the most trivial and simplified kind of novelty, to being one of the most complex narrative forms human beings have ever devised.
What I mean by the Fred Ott principle is that whole complex process that we can go from something so simple to something so complex, from something so marginal in society to something so central in society in such a short time.
And I want to at least remind you of what that principle involves.
I'm talking about all the technological, cultural, demographic, and economic currents that swirl together to create the movie industry that emerges-- really by the mid teens the movie industry essentially is in place.
Variations will occur, new studios will appear, but by 1915/1916 American movies have been established on an assembly line basis.
Have been established on an assembly line basis, and millions and millions of people are now making it a regular habit to go to the movies.
And the movies are elaborating themselves in a complex way that has to do with the way in which they were industrialized.
There are three phases, I think, to what we might call-- let's go back to the outline, Greg.
There are three phases to what we might call media evolution.
And this is another way of sort of dramatizing what I mean by the Fred Ott principle.
The first phase is a phase of imitation and patent warfare.
A new technology appears.
Nobody yet knows how it would be applied.
How people will want to do.
Everything about the new technologies up for grabs.
So there are competitors who want to sort of claim patents on the technology.
Questions about how long should films be?
Where she feels be shown?
How should films be distributed?
All of that's up for grabs.
All of that is rationalized and decided-- I put "decided" in quotes because no one sits down and makes a decision.
It's really a function of the marketplace, and of certain economic opportunities.
There's no question, for example, that the movies would never have developed as they did were not for those immigrant populations in cities like Chicago, and New York, and Los Angeles.
And many other cities on the East Coast especially, where they were very large immigrant populations who needed.
And it was that financial infusion that caused the immense amount of experimentation and development to take place so quickly.
So in this first phase of imitation and patent warfare many of these questions are not-- even such simple questions as how long a film should be?
Some of the question of the length of the film are technological, the very first films had to be only 10 minutes long, because that was as long as the film cartridges in which they put in the cameras were capable of doing.
Later they were able to make two reelers, and three reelers.
That happens over a relatively short space of time.
So in this phase of imitation, one the most important things that happens, and this is the part I want you to become attentive to, is that all of the ancestor systems that lie behind the new technology are potential influences on the new technology.
And in new films that you've seen already, the silent films you saw in recitation, you saw some examples of this.
For example, do you remember-- maybe we could show this, Greg.
The deaths in The Great Train Robbery.
Remember the deaths of The Great Train Robbery?
Why are you smiling?
Because there was a dummy
Because they were still noticeably what, false, fake?
Where do they come from?
The guy shoots, and the guy goes "ooh."
And he staggers around, and then he falls.
What's going on there?
Where does that tradition of performance come from?
Theater
Yes, yes.
I'll let it play behind me.
Yes, it comes from theater.
Makes sense that one of the deep influences on early film would be theater.
But now in the Fred Ott principle, the most important sub idea in this theory, and this label, the Fred Ott principle is the fact that this process of development involves, among other things, the capacity of filmmakers to discover those features of the new medium that are unique or special to the medium.
What makes the medium different from its ancestors?
Clearly, what was appropriate for books, or newspapers, or theater, won't be perfectly appropriate for the new medium.
But nobody knows what the new medium's capable of until things have been tried out.
So that's why I call it a phase of imitation.
And what one can say here especially is that certain theatrical styles of acting are dominant.
Why our theatrical styles of acting so broad?
So unbelievable.
Why are they like that?
Because you're far away
Yes, in the theater you're far away, and you're in a fixed space.
But what's one genius of the movies?
One way the medium of the movies is different from theater is that the camera can achieve a lover's closeness to the action.
Or it can achieve a long view that's much longer and further away than what.
Well, it's going to take time before this kind of thing is figured out.
And one of the most decisive things we see an early films is first, an acting style that seems derived from theater.
But some of you may have noticed that this begins to change relatively quickly.
Can we show the fragment from A Beast at Bay?
Remember that silly film, A Beast at Bay, that I had to watching it along with The Great Train Robbery?
Remember the ending of A Beast at Bay?
Something odd happens in this ending.
There's the monstrous, drooling, rapist-like figure.
He's always a convict or a low-life.
There are all kinds of social hierarchies and established social prejudices that get imported into early films.
And now we're going to have the rescue.
And so far it seems a conventional sort of sleazy melodrama, in which there is at least a hint of something morally disturbing in the imminence of the rape.
It's as if the film makes a kind of sleazy appeal to its audiences.
Here's a perfect example of such a moment.
It's interesting in fact, how one of the recurring subjects of films always seems to be not just rape, but violence against women.
It tells you something about the patriarchal societies in which films emerge that those mythologies are replicated in movies.
So he's going to be rescued here.
And what I want you to notice is what happens at the very end of this sequence.
And can some of you remember what it is?
None of you remember?
It was so fast.
But it actually is very significant.
I think it shows us the emergence, the beginnings of the emergence of a new style of acting more appropriate to movies.
And also, something else, the development of a total complexity that had not been in movies before.
Now the date of this film is early, 1907, 1908, something like that.
You're going to see one other silent film by D.W. Griffith tonight, The Lonedale Operator, made in 1912.
And I hope you'll be attentive to how much more complex that film is.
I'll say a few words about that tonight.
I wish we were really just at the end, Greg.
I didn't really want to show the whole film.
OK, so here we're at the conclusion.
Now this is the part I wanted you to notice.
Look at this.
What's happening here?
The film has shifted over into a kind of silly comedy, a kind of gentle comedy.
The woman said, kiss me hear, kiss me hear.
And in fact, if you notice she's not she's not saying, kiss me here!
In other words, her gestures are more modulated to the nature of movies.
What's beginning to happen there is two important things.
First, total complexity is entering in.
A melodrama has turned comic.
And acting style's beginning to.
Can we do the very end of Musketeers?
Here is the very end of a film by D.W. Griffith called The Musketeers of Pig Alley.
Some people see it as one of the first urban crime films.
But what I want you to watch is this small actor here .
Watch his performance.
This is the emergence of a new kind of acting, a non-theatrical kind of acting.
And it's happening very early, this film appeared in 1912.
That's Lillian Gish, the famous silent film star.
This gangster is socked that this woman would choose his rival over him.
He says, you're nuts, lady.
I can't get it, but OK. Look at this strutting peacock of a man.
Watch this.
He's a gangster, he's about to be arrested, but the good guys will save him.
Oh, we lost it
[INAUDIBLE]
One good turn deserves another.
But you see how much more restrained the performances are here?
I mean, Lillian Gish became one of the great silent-screen stars.
But I think it's this strutting peacock of an actor who really begins to show what you can do on film.
And there are many film critics who have seen this as a precursor of Edward G. Robinson's performances.
He's also a diminutive actor, often plays gangsters.
But what you see, I think, in this moment, and in this man's performance is the emergence, the beginnings of an acting style that's more modulated, that's appropriate to the nature of the film So that kind of thing that's happening is not just with acting, it has to do with all kinds of things.
It has to do with where you place the camera, and such matters.
So what I mean then, by the Fred Ott principle is this process of evolution that I want you to be aware of.
And really, in a certain sense experience in distilled form in the short films I've asked you to watch in this first week of classes.
The first phase is a phase of imitation and patent warfare.
In which all kinds of questions are up in the air.
How will the film be distributed?
How will it be exhibited?
How will it be produced?
There's nothing inherent in the nature of the technology that requires the economic arrangements that developed in the United States, and then we're replicated elsewhere, for the distribution of movies.
In fact, Thomas Edison had a different idea for how movies would be developed.
When he first conceived the apparatus, he actually thought that movie projectors would be owned by each individual.
In fact, what he was imagining was the camcorder that occurs in a much later generation.
And there's nothing inherent in the technology that would not permit that.
So one of the things we need to be aware of-- and I'll return to this matter either tonight or in a later lecture.
We need to be aware of fact that the shape the technology takes is not the only shape it might take.
It's not the technology itself that drives its development so much as economic, and social, and demographic factors.
And that's dramatically the case with the system of distribution and access that was developed for the movies.
It's certainly theoretically possible for the apparatus to be developed in a way that would be sold to every individual.
What instead happened was a system in which a professional elite becomes the production arm.
And these productions become very expensive, and they are pumped out into the society for screening at public theaters, which people will pay money to attend.
That economic structure, and that basic industrial structure is not necessary to the technology.
It's not required by the technology.
The shape of film is at least as much cultural as it is technological.
A very important point.
So the second phase after the phase of imitation is the phase I call technical advance.
This phase occurs after some of the warfare is concluded.
Limited monopolies are established.
Some companies are more powerful than others, or come up with a more successful product than others.
And they begin to dominate the marketplace.
They drive competitors out.
And what essentially happens is a kind of stability is introduced in which the basic system is put in place.
Here's how we'll manufacture the item.
Here's how we'll distribute it.
Here's how long it will be.
That sort of thing.
And in this period of technical advance what then happens once the stability sets in, the particular, unique features of the medium begin to be explored.
And these early films that I'm asking you to look at, what I hope you'll watch for are moments like the ones I was just pointing out this afternoon.
In which you see the emergence of a recognition of something that is distinct or special in the nature of movie making.
I'll return to some of these matters again tonight.
But I want you to be aware of them.
So in the second phase, the phase of technical advance, the system begins to learn what is unique about it.
How are movies different from their ancestors?
What does it mean that the camera can move close to the object is photographing, or very far away from it?
What does it mean that the camera itself doesn't have to be stable, that it can move?
You can see some of the early Griffith films really experiment with a camera that's mounted on something that's moving.
And then the final phase is the phase I call maturity.
And that's the phase that occurs really in silent film, in the 1920s.
When the technical advances that are accomplished in phase two are married to a serious subject matter.
The phase I call maturity is the phase in which feature films are made, and in which some films become works of art.
And all films become more complex forms of narrative.
In which particular genre forms begin to emerge.
And audiences begin to choose particular kinds of films that matter to them.
In other words, the system really elaborates itself in a way that suggests an immense variety of appeal to a range of audience.
That's the phase of maturity.
Why doesn't it go on forever?
What explains why the process doesn't continue?
What stopped this process?
So the Fred Ott principle encapsule means going from Fred Ott's Sneeze to going to Chaplin's Modern Times.
This immensely rich, complex narrative film that we'll be looking at next week.
What explains why that moment of maturity doesn't extend forever?
Why is this system not stable forever?
The simplest answer is capitalism never allows for stability.
But the more exact answer is new inventions, new technologies subvert the stability.
What happens at the end of the '20s to subvert the confidence and stability of the system?
The introduction of sound
The advent of sound.
Yes.
And the sound film doesn't completely revolutionize movies, but it profoundly alters them.
It changes the nature of film.
And it changes the nature of the kinds of performers that you need in film.
It profoundly enlarges and complicates what film is.
And then, something of the same principles that I've talked about before happen in the sound era.
I'm sorry I'm running a little over, but I promise you I'm almost done.
And normally, I will never run over even by a minute.
I promise.
I'll work for this.
Why doesn't the final phase continue?
Because of new technologies.
Because of new possibilities.
What happens at the end of the studio system?
The advent of television overturns the stabilities that had been created and the old studio era.
And the intimate routine connection to movies that had been established in the studio era is finally obliterated by the presence of television.
And movies change their character after television comes in.
We will return to this.
We will return to this matter.
, Well one way I can suggest for you to capture this imaginative linkage, this imaginative connection to the world of early film is by reminding you of a wonderful passage from the Pulitzer Prize winning novel by the great film critic James Agee, which appeared posthumously in 1957.
And it distills for me what I mean-- and I hope for you-- what I mean in part by film as a social form, film as a socially embedded formation.
It helps to explain in less abstract terms what I mean when I say that filled permeated American life.
Listen to the beginning of, this is from the first chapter.
And we're finished after this is over.
From the first chapter of A Death in the Family.
"At supper that night as many times before, his father said, well, suppose we go to the picture show.
Oh, J, his mother said, that horrid, little man.
What's wrong with him?
His father asked.
Not because he didn't know what she would say, but so she would say it.
He's so nasty, she said as she always did, so vulgar.
With his nasty little cane hooking up skirts and things, and that nasty little walk."
Who is he talking about?
Chaplin, Charlie Chaplin, of course.
And this story takes place before Chaplin was making feature films.
So the story takes place probably around 1915.
When comedy was still being made as shorts.
Comedies didn't become feature length until sometime late in the '20s.
"His father laughed as he always did.
And Rufus felt that he had become rather an empty joke.
But as always, the laughter also cheered him.
He felt that the laughter enclosed him with his father.
They walked downtown in the light of mother of pearl to the majestic," nice name for a theater-- "and found their seats by the light of the screen in the exhilarating smell of tobacco, rank sweat, perfume, and dirty drawers.
While the piano played fast music," right, because violent films were never silent.
There was always music accompanying them.
"And galloping horses raised a grandiose flag of dust.
And there was William S. Hart--" the passage then goes on to describe a western film with William S. Hart, a silent film.
"And then the screen was filled with the city, and with the sidewalk of a side street of a city, and a long line of palms, and there was Charlie.
Everyone laughed the minute they saw him squatly walking with his toes out and his knees apart as if he were chafed.
Rufus's father laughed, and Rufus laughed, too.
This time Charlie stole a whole bag--" this time.
What does that imply about the audience?
And intimate familiarity with previous adventures of this character, an ongoing routine connection.
"This time Charlie stole a whole bag of eggs.
And when a cop came along, he hid them in the seat of his pants.
Then, he caught sight of a pretty woman, and he began to squat and twirl his cane, and make silly faces."
I'm going to skip it.
It's magnificent prose that captures the essence of the film very wonderfully.
But I don't want to keep you longer than I already have.
And it shows Charlie sort of flirting with the girl.
And then finally, he flirts with her so much that she pushes him and he falls back down.
"Then he walked back and forth behind her, laughing and squatting a little.
And while he walked very quietly, everybody laughed again.
Then, he flicked hold of the straight end of his cane, and with the crooked end hooked up her skirt at the knee in exactly the way that disgusted momma.
Looking very eagerly at her legs, and everybody laughed very loudly.
And she pretended she had not noticed.
And then she pushes him over.
And there was Charlie, flat on his bottom on the sidewalk.
And the way he looked, kind of sickly and disgusted, you could see that he suddenly remembered those eggs.
And suddenly you remembered them, too.
The way his face looked with his lip wrinkled off the teeth and a little sickly smile, it made you feel just the way those broken eggs must feel against your seat, as queer and awful as that time in the white PK suit."
He's also dramatizing how personal our relation to film can be.
"When it ran down out of the pants' legs, and showed all over your stockings, and you had to walk home that way with everyone looking.
And Rufus's father nearly tore his head off laughing, and so did everybody else.
But Rufus was sorry for Charlie, having so recently been in a similar predicament.
But the contagion of laughter was too much for him, and he laughed, too."
And the passage goes on to describe the intimacy and the complexity of the relations between audiences in 1915 and one of the iconic figures of the movies.
In this idea, that Charlie mobilizes this anarchic, liberating laughter, and mobilizes this father-son relationship.
In that idea we recapture something of what I mean by the notion that film was an embedded social experience, an embedded social form.
I'll see you tonight.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
In this evening lecture what I'd like to do is continue a bit to expand on and complicate and maybe also deepen this idea I've been suggesting to you that I'm labeling the Fred Ott principle.
Really a shorthand way of describing the immensely complex-- and from an intellectual standpoint, a historical standpoint-- immensely exciting and astonishing process whereby film went in an incredibly short time from being a mere novelty to being an embedded social formation in the United States and in other industrialized societies.
And not just only a embedded social formation, becoming by the end of the 1920s one of the central media for the expression of art, and an artistic medium.
So that he went from being a mere novelty, from Fred Ott's Sneeze in 20 years or so, 25 years, to being a medium of astonishingly rich and complex narrative art.
That progress is part of what I mean by the Fred Ott principle.
And it might be helpful if I concretize that a little bit more by mentioning a bit more about what I mean about what I was implying or referring to when I spoke about that moment of imitation.
Remember I said that there were essentially-- that the silent film-- and incidentally, I take it as a model for the way in which most new media systems develop in culture.
You can apply this same basic schema to other forms of popular entertainment, including-- shocking as it may seem-- Shakespeare's public theater.
And I'll come back to the implications of that analogy.
The idea that Shakespeare was the movie of his time.
That Shakespeare's theater was the Hollywood studio system of the Elizabethan era.
It's a very complicated one.
It's a very inspiring one.
And it's complicated because it makes us reassess our inherited notions about what art is, and where art comes from.
One of the subtexts in this course is precisely that, it has to do with what I call the enabling conditions of art.
And I'll come back to that again as a theme, again and again in the course.
So part of what I mean by the Fred Ott principle is this roiling complex, partly unpredictable process, whereby an enlarging population incredibly hungry for the novelties being produced by this new technology create a kind of symbiotic relationship between the audience and the emerging medium.
So that as the medium grows more complicated, in part because the audience-- OK, after 5, or 10, or 15 visits to the Nickelodeon, or-- after the movies moved out of the penny arcades, and began to take their own space as some of you know, and as it's recounted in our reading for these early weeks, the movies first moved out into what were called Nickelodeons.
They were storefronts for the most part.
And admission was a nickel, that's where they were called Nickelodeons.
And you sat in many of these on benches without backs, next to strangers.
You didn't have private seats.
And you would sit in there and see a series of short films.
But the fact that the Nickelodeons emerged so quickly is a mark of how popular a film became, eve in its relatively primitive early form.
And then, of course, within a few years of the Nickelodeons appearing, what else began to happen?
Theaters made specially for the showing of motion pictures that were longer than the early shorts, because film began to expand its length all through the period from 1910 through 1920.
What's the great moment when the feature film is born in the United States?
1915. Who's the director?
D.W. Griffith.
What's the film?
Birth of a Nation.
Birth of a Nation.
A very complicated example, because it's content is disturbing in some ways.
It's a very reactionary, and in some respects racist film.
It absorbs and carries forward what we might think of as the racial prejudices that were very widespread in American society at the turn of the century, and especially in the South.
From which Griffith himself was a southerner, he was born in the South.
And his film was deeply influenced by a bestselling novel called The Clansman, which was a celebration of the Ku Klux Klan.
So the content of Birth of a Nation is very unsettling and disturbing.
And even when it was first released to the public-- to great acclaim because it expanded the possibilities of movies in all kinds of ways.
From a technical standpoint it is an astonishingly important film.
And from a content standpoint it's a very disturbing film.
It's a wonderful reminder of the fact that this progress I'm describing to you is not an unalloyed triumphal story.
Even as the movies became more technically complicated, and more demanding, and longer, and more interested in character, they also nonetheless carried the lies, and prejudices, and hierarchical assumptions that were embedded in the society from which they arose.
How could it be otherwise?
Every media form does this.
But it's important to make this point as a way of reminding us that the story we're telling is not, in some simple sense, just a kind of progress myth in which we're celebrating development and genius.
We're identifying and locating something more complicated than that-- the process whereby these cultural myths, and these stories that are drawn from inherited older stories, and from the lore of the society more broadly, more generally, are transformed into this new medium.
And they have a tremendous technical interest.
But very often they also have a kind of cultural or sociological interest of a negative kind in the sense that what they reveal are the prejudices, the lies, the limitations of the society, the mythologies that sustain the society.
Again this is a matter to which we'll return.
One of the deep cultural functions of American movies, especially, was to promulgate a kind of mythology of America.
And we'll talk more about this when we reach certain genre forms in the second segment of the course.
I said something earlier this morning that I want to make explicit, because I think another way of qualifying this triumphal story.
It's hard because I'm enthusiastic about what I'm looking at.
And there is some material that's so exciting here.
In a way, what we're watching is we're watching the birth of the movies.
We're watching the discovery of the language of cinema in these early films.
And that's why even though from an artistic standpoint, some of these shorts are not very rich, I hope they're interesting enough-- that I've made them interesting to you from a historical standpoint.
If you look at them closely, you can see the movies being born.
You can see a language, a syntax for speaking in pictures, for new visual language, this new visual medium being developed.
And if I have time this evening, I'll give you a few more concrete examples of that.
But another way in which one can qualify this apparently triumphal story, it's not a triumphal story even though it is a story of refinement, development, and evolution of a kind that involves increasing complexity and technical perfection, technical mastery.
It's not merely that, or not even primarily that.
It's very important to recognize among other things, as I implied this afternoon, that not all the possibilities that are inherent in the nascent medium are necessarily exploited in a particular cultural moment.
Or maybe ever exploited, depending on how things develop.
That is to say.
I mentioned this afternoon how for instance, when Edison first conceive the apparatus before it was actually invented, of the motion picture projector, and the motion picture camera.
His first idea was that he would create an item that would be a consumer item-- what we would today call a consumer item-- that would be sold to individual families.
And the field would become an equivalent or a kind of a photo album, although it would have motion it.
And as I suggested this afternoon, there's no reason why given the nature of the technology itself, that that vision of how film might develop was impossible.
In fact, it wasn't impossible, it was just incredibly ahead of its time.
It took half a century before something like that actually became available in society.
But there was no reason in terms of the possibilities of the technology that that needed to happen.
What I'm calling your attention to is what is a very widespread myth.
It's especially pernicious and widespread at MIT.
For reasons that are obvious and understandable we at MIT love to believe the technology will solve all problems.
The primary thing I'm suggesting to you is that the evolution of the movies that I want you to be aware of, this process of increasing complexity and compression in which the movies become a more and more independent form of expression.
In which the movies begin to discover the unique characteristics of the motion picture camera and of the environment of the movie theater, that the movies begin to explore.
Those qualities in this new medium that are unique and special.
That process is important, and I want you to be aware of it.
But I don't want you to embrace that principle to uncritically.
Because I want you to recognize that the technology itself does not explain this process.
The processes explained by cultural factors, and social factors.
And even sometimes individual's psychological factors.
What we're talking about here is the myth of technological determinism.
The myth that technology drives culture.
The myth that a new invention obliterates old inventions.
The truth is much more complicated than that.
One of the most remarkable things about this evolutionary process I've been describing, as I said earlier, is how swift it is.
How fast it is.
How we go from being a mere novelty, to becoming a significant social form by 1910 or so.
And to becoming a virtually universal aesthetic and entertainment experience for the majority of the population in the country by 1920.
So when something like 20 or 25 years the movies go from being an absolutely unknown or trivial novelty, sharing space with fortune tellers, and strip shows in the penny arcades, to becoming not just an embedded social form, but one of the dominant economic engines of the society, employing tens of thousands of people in various direct and ancillary positions.
And mobilizing virtually the entire population of the country in a regular routine, a habitual experience to which they return again and again.
And because the audience is returning again and again-- remember I said there's a kind of symbiotic relation between the audience and the-- because the audience is returning again and again, what happens?
That's also one aspect of the resources that are available to the medium.
What begins then to develop are forms of storytelling that rely on the audience's prior memory of other shows.
So that one thing that emerges very quickly is what we might call a star system.
In which particular actors become identified with particular kinds of roles.
And audiences want to come back to see those stars again and again.
What also begins to happen is the establishment of a very rigorous kind of genre system.
What's a genre?
A category of story.
The Western is a genre.
The detective story is a genre.
The elegy is a poetic genre.
And the typical, the fundamental movie genres begin to emerge relatively early.
There's a Western adventure story.
There's a historical spectacle.
There are certain forms of urban, crime narrative that became our detective and crime films at a later stage.
And there emerge especially certain forms of satire and comedy.
So what genre system begins to emerge as well?
And there are reasons of course why this kind of thing is useful to a mass production system, to an assembly line system.
If the same movie studio makes seven Westerns, they can use the same horses again and again.
They can use the same set.
They can use the same hats.
And this in fact happened, they might eve be able to use the same clip of film that shows a cattle stampede in seven different movies, because nobody would actually see it.
So economies of scale become possible as the system begins to regularize itself.
Once it stabilizes after that period of imitation and patent warfare.
And there's nothing noble, or artistic, or conscious about this development.
That's part of what makes it so remarkable.
And it puts out in touch with the paradox that's at the very heart of what I hope to make you aware of in this course.
It's a paradox that-- I guess it animates my transition many years ago now, from a traditional literary scholar into someone interested in film and media.
And it's this paradox.
Based on what I've been telling you about this evolution, this evolution of the medium, of the film medium.
It's the paradox that capitalist greed-- the crassest of alliances between commerce and modern technology-- might be the enabling conditions of a complex narrative art.
Where does art come from?
It's not just about art though, eve about sort of social.
Even if we forget the question of art, and just talk about the social and cultural importance of movie-going in modern societies.
Leave out the question of whether it's an aesthetically valuable experience.
It's so central to human experience, remains so central to human experience, that there's a tremendous paradox in this, that the crassest of alliances between commerce and technology.
Nothing noble or admirable about it.
Nobody's sitting down thinking, I want to make a new, great medium for art.
Nonetheless this alliance becomes the enabling conditions for a complex form of social experience and of narrative art.
Well, that paradox at the heart of a lot of what we'll be looking at in this course.
I hope you'll keep in mind as we go on.
I'd like to say a couple of words, not too many, a couple of words about The Great Train Robbery.
The very first film that you've seen.
The Great Train Robbery is discussed in some helpful detail in David Cook's History of Narrative Film.
And I urge you to look closely at his account of it.
But I want to remind you about a couple of highlights.
One is that although The Great Train Robbery seems very primitive to us, it's important to realize that even The Great Train Robbery is already a refined text.
Most of the most fundamental principles of what we might call the syntax or grammar of the movies is already embedded in The Great Train Robbery.
And what that means is that the five or six year period before that, when there was all this practicing with non-narrative forms, lies behind The Great Train Robbery.
And what also lies behind The Great Train Robbery is this experience of audiences going in first to the penny arcades, then to the Nickelodeons that begin to open up by the turn of the 20th century.
And becoming impatient with repeated scenes that simply show jokes, hoses going off in people's faces, or trains coming into a [INAUDIBLE].
After a while the novelty wears off.
But of course, the people who are making films are also interested in what they're doing.
You can feel especially, if you look at D.W. Griffith's shorts, and you will have looked at two of them in this course-- A Beast at Bay, from last time, and The Lonedale Operator, which you'll see tonight before we turn to the Keaton films.
And you'll see these are just a sampling of his early work.
But if you look closely at those where you can feel Griffith's own excitement at discovering the possibilities of the cinema.
In many cases Griffith is doing some things with the camera, doing some things with the shaping of narrative in cinema that had never been done before.
And you can feel his own excitement at the possibilities of this astonishing technology.
Well, the most important discovery that The Great Train Robbery shows us-- and again it seems so obvious and so embedded that we wouldn't even think about it today.
We absorb these principles so deeply into our DNA that we don't even think.
We learn how to watch movies in this way from the cradle, in some ways.
Because by the time most people today are five or six years old, they've been subjected probably to more audio/visual images than anyone alive in 1900.
And we're just cognitively different in terms of our capacity to process audio/visual information.
But imagine an audience that's completely illiterate in these things, that has to learn this system as the system itself is evolving.
That's part of the excitement of looking at these early films.
And the most important discovery in The Great Train Robbery is the discovery that the shot, the single uncut shot of film, however long it lasts, and it's dependent on the decision you make about when to edit it.
So it can go on for a long time, or it can be very short.
And the length or the shortness of cuts also affects sort of the rhythm of your experience in the film.
And this is something that you can see Griffith practicing with, even as early as A Beast at Bay.
If you think about the great climax of the Beast at Bay where you have the woman being threatened, but you also have this race in the automobile to try to get there on time.
And then the race to rescue her by a variety of people.
What Griffith does is he cuts back and forth.
He creates the principle of parallel action.
And what he realized, what he discovered was that the audience will recognize this kind of cutting as showing you simultaneity.
Showing you things that are going on at the same time.
What he discovered were in a certain sense how we would cognitively process certain kinds of gestures on film.
And what he discovered of course, is that if he cuts, if he edits more quickly, if he creates a fast rhythm he creates excitement.
And what he began to discover was something that-- in later, directors like Hitchcock would carry, to an almost evil pitch-- a capacity to manipulate and control the audience's reaction by the speed with which particular images appear or disappear.
So even at this early stage, you could see Griffith discovering important principles.
The most fundamental principle in a Great Train Robbery, which is not a Griffith film, it proceeds Griffith, is the discovery that the shot is the basic unit of action.
Why is that a great discovery?
Well, if you think of the movies as a device or as a medium that grows out of especially a theatrical dispensation, grows out of other things as well, novels feed into it, visual art feeds into the movies.
But it's certainly true that most of the major ways in which directors first conceive how to think about the experience of what the camera is looking at, they would have been influenced by their own experience of theater.
And if you think about it for a moment, what you can see, even in The Great Train Robbery, an impulse to treat the camera as a stable thing.
There are panning shots in which the camera will go like this.
But the camera will not track.
It will not move back and forth, or sideways, off, in The Great Train Robbery.
That's a discovery that Griffith is going to make and will exploit.
And you can see it in A Beast at Bay.
But in The Great Train Robbery, we haven't got that far yet, but what Edwin Porter did discover was that the shot is the basic unit of action.
And what this means is it's not simply what the camera sees, but then how the shots are edited afterwards that is so crucial.
So what he came to understand was that the making of any movie is always a two-stage process.
There's the shooting of the film, and then there's the editing the film.
And that itself is a fundamental discovery.
But the idea that the shot is the unit of action instead of the scene was hard for movie makers to get.
What they would do, they would set the camera up, and then actors would walk in front of the camera, they would do what they had to do, and they would walk off.
What's going on there?
They're conceiving of cinematic space in theatrical terms.
As soon as the camera is liberated, taken off its tripod, moved around in some way, allowed to backup or come close, even allowed to move laterally.
What we're beginning to do was to explore more systematically what the camera's unique features are.
How the theater is different from the movies.
How certain effects are possible in the movies that would not be possible in the theater.
When you watch the Keaton films that you're going to see tonight, watch for the ways in which Keaton repeatedly sets up situations such that when you see a particular gag of his, a joke, what you realize is that the joke could not have happened in the theater.
It could not have been told to you verbally.
The joke is visual.
The joke involves motion.
The joke involves a recognition of the powers of cinematic representation to tell stories.
You'll see many examples of this in the Keaton materials that you'll be looking at tonight.
So one of the very first discoveries that's made is the discovery that the shot is the basic unit of action.
A second discovery is the recognition that you have to cut between scenes.
You do you're editing between shots.
Mostly what you do is you let scenes play out, even though you might abruptly cut a scene before the action is completed.
That's another discovery that was made, if you cut the scene-- a person starts to walk toward the door-- the first, earliest films, they would show the person walking all the way to the end, walking out of the door.
Then, filmmakers discovered, wait a minute.
There's a way of creating a kind of quickness here.
Because if I show Thorburn walking toward the door, starting, and I cut.
And he knocks his glasses off.
And then I cut, and I wait till he gets to the door, and we see him leaving the door.
I don't have to watch him take those 17 steps.
So what's been discovered is a basic principle of cinematic syntax.
It seems so obvious to us, but of course it's really a great discovery.
The properties of the medium have to be understood before you can really begin to explain them.
And that's why I've asked you to look at some of these short films.
None of the short films I've shown you are completely without some technical interest.
And even though of course they're the only films I'm showing you that I wouldn't sort of make arguments to defend them as works of art.
But I certainly would make arguments for them as technically fascinating documents.
And I hope you'll look at them in that light.
So the shot is a basic unit of action cutting between scenes.
And a couple of other things that I think are important in The Great Train Robbery.
One is that you can see in The Great Train Robbery, they're beginning to discover the value of camera placement.
What happens when you place the camera in a particular position.
Especially if action's coming toward the camera, that you might not want to place the camera directly in front of the action that's coming towards you, that something happens if you place it slightly forward it, so it could see it at a slight angle.
It creates a greater illusion of depth in the image, for example.
And it does other kinds of things.
It let's the action get closer to you before the scene has to be cut.
So camera placement is something that you can see is beginning to be learned, beginning to be mastered in the great trick in The Great Train Robbery, or camera position.
And finally, one might mention one other thing that a again establishes a principle that then is continuous being useful 50 or 75 years.
And that's the principle of the process shot.
You can see in The Great Train Robbery that some effects have been created by tricks.
That there's back projection used in that.
It looks crude to us, we can see through it.
But the early audiences would not have recognized that.
We're so used to location filming that we can pick out-- we today can see through the process shots of earlier films.
This in fact damages some of Hitchcock's films very badly today.
Because Hitchcock used a lot of process shot effects in his films.
When he made them the audiences were used to it.
There was a convention of film going.
Today because cameras are so much more mobile and light, and so much has been learned about how to-- even about filming in low light and so forth.
That there's much less use of process shots.
A new kind of process shot, I suppose, is emerging with digital animation.
And one could say it's sort of a return to the idea of manipulating the image in illegitimate ways.
But in The Great Train Robbery we can even see the beginnings of the use of process shots.
Of recognizing that you can create effects in the movies that are unique and special, that no other medium would allow you to do.
And what's embedded there is science fiction.
What's embedded they are all the kinds of imaginative, not just science fictional.
All the kinds of imaginative movies that have been made that maybe take you inside people's nightmares or dreams.
Where what the film is showing you now-- what is embedded in that process shot?
Even more fully embedded in Melies', the French director's films, like Trip to the Moon, where he explicitly goes into a realm of fantasy.
It's not only that the film is the most profoundly realistic medium that was ever invented.
It may also be the most expressive medium for the rendering of non-realistic states of dream, of nightmare, of fantasy.
Because of the freedom that the film medium allows.
And even these implications are at least embedded in the use of the process shots in Porter's The Great Train Robbery.
The other short that you're going to see tonight that is not by Keaton is called The Lonedale Operator.
It was made in 1911.
In a way it's the most advanced film of its time.
And what I hope when you look at it, you'll watch especially for how much more complex it is than, let's say, The Great Train Robbery.
And there'll be a kind of small measure of this Fred Ott principle that I've been talking about.
Let me mention a couple of things about The Lonedale Operator.
Between 1907 and 1911, most films consisted of no more than 24 separate shots.
Between 1908 and 1913, led by D.W. Griffith and his company, The Biograph Company, many more shots were introduced into a single reel of film.
And in The Lonedale Operator, there are 98 shots, six intertitles, one main title, and two written inserts.
All within a 17 minutely length.
But think of just the access of complexity that has occurred just in less than 10 years.
It's such a complicated film that one of the things we realized, even looking at it if we look closely is that it actually required a kind of attentiveness that earlier films did not.
As filmmakers began to make these things, and began to make them longer, and began to make them more complicated.
They realized that they couldn't just bring the camera out and start shooting.
They began to develop the formal rationalizing strategies that led to the making of movies as we understand them.
There was a four day shooting schedule created for The Lonedale Operator.
It was one of the first films to have this format.
And of course it's the format that is followed today with virtually every film in different aspects.
And this also required a preplanned shooting script.
They had to preplan all of their shooting, all of their shots, all of their camera angles.
They need a shooting script, a written script.
Films didn't have written scripts at first, but Lonedale did.
And it also required in order for them to make their schedule, out of order shooting.
That is to say they had to shoot certain things not in the sequential order.
And that's one of the most basic principles of filmmaking.
Because once you set up the camera for a shot of a particular environment, if that environment shows up later in the film, it's unbelievably more expensive and time consuming to reconstitute that whole set.
So almost all films to this very day are still shot out of sequence in order to maximize the time you have, in order to work at your most efficient.
It's not an efficient way to make a movie to shoot it sequentially, to follow the to follow the script in its chronological sequence.
And The Lonedale Operator is maybe the first film have to have all of these features that we would associate with a modern or contemporary film.
You might notice in it's an immensely complex films, even in terms of its content.
And I won't talk about that much now except to say this, watch the way in which the film tries to deal with what we might call hierarchies of gender.
The film is partly about how a woman takes on a man's job.
I think it's guess it's her father, isn't it?
Her father is ill. And she has to take over running the train station.
And so there's a certain sense in which one of the things that's dramatized socially is a woman moving into a masculine realm.
Taking on a role that's more aggressive than is appropriate for the gender ideologies that are dominant at the end of the 19th century in the United States.
But the film was pretty clever about this, like many such popular texts.
Because although it does grant this woman a certain kind of masculine authority and competence, it also has other things in the film that balance that.
That remind us that she is still nonetheless a weak vessel.
That she is still a damsel in stress like the typical Victorian construction of women.
And what you can see is that the movie exhibits an anxiety about gender roles that's also an anxiety that's resident in the society itself.
And I won't talk more about this now.
We will develop this aspect of the movies as a reflection of social anxieties and social problems as the course goes on.
But I do want to call your attention to certain technical things that you can watch for, to at least one technical item in the film.
It contains what might be called rhyming scenes.
In which at one point in the film you'll see a sequence of scenes in a particular shot.
And then, later in the film, almost exactly the same shot will reappear.
And the effect of this, of course, is to create a sense of coherence and a sense of familiarity in the audience.
It's actually a very subtle and careful strategy that Griffith himself devised.
It creates a kind of continuity and clarity to the whole film.
Watch, for example, how the doorway to the Lonedale station is photographed when you first see it.
How this image anchors the film's spatial relationships.
And how the film has a certain trajectory of movement that is repeated as the film goes on.
So that for example, early in the film, we see the heroine go from the exterior to the interior of the ticket office, and then to the inner office of the telegraph operator.
We see her do this.
And then, this sets up a basic location and basic action for the entire film.
It's as if what happens in that moment sets up, creates a familiarity for the audience with it, the geography of the movie.
Because a lot of the action then will depend on our understanding about geography.
And then there at least four different times that Griffith repeats that basic three-shot trajectory, from the exterior to the interior of the ticket office, and then to the inner office of the telegraph operator.
It happens four times in the film.
So the film has a kind of visual coherence.
By the time you come to it the third or fourth time, it's as if you were intimate with the physical geography of the movie.
This is actually a structural feature that is easy to miss it, but it's actually very artful.
And we can see it's a mark of how far film has come, even by 1911, that this level of attentiveness is available to us in the film.
There's much more that could be said about The Lonedale Operator, but I will leave it to your intelligence and your attentiveness to try to pick out more things.
And again we're only talking about a 17-minute film.
Well, Buster Keaton's career could be said, in some degree, to enact in small the larger processes I've been describing.
He comes into film late.
And in fact, if we were working perfectly chronologically in terms of when Chaplin became a director, he would precede Keaton.
But I've organized the film according to the dates of the feature films, and the general is much of a decade earlier than modern times.
Which is the last, quote, "silent film."
It's not really a silent film, but it's the last silent film in a deep sense that was ever made.
It was made in the sound era, in 1936.
And next week we'll talk more about the implications of that, and why that was a very daring thing for Chaplin to do.
Something like almost a decade after the advent of sound he made a movie that essentially had no dialogue in it.
So Keaton came a little later.
Keaton comes into movie making a little few years after Chaplin.
But his career has a similar trajectory.
He began his career in legitimate theater, in vaudeville and on the performance circuit.
He was an Acrobat from a very young age, from the age of two or three was in his parents stage act.
And in fact, he learned a lot of his acrobatic feats on his parents stage act.
One very famous passage in his-- his father used to take him, he was just a young kid, three or four years old, five years old, he would take him by his legs and swing him back and forth, like this.
He had very long hair.
And he would sweep the floor with his hair.
And then, swing him two or three times-- this would happen life-- he would swing it like this and let him go.
And the kid, five years old, would fly against the wall and fall down.
It would look terrible.
And later in his life, Keaton talked about this.
There were actually some cities in which the societies the prevention of cruelty to children came in, shut down the act, wouldn't allow the act to go on.
Not stupid, because his father was a drunkard and sometimes performed drunk.
And Keaton was sometimes injured.
He was also injured when he made his own movies, because he was such a daring acrobat.
And Keaton recounts that when he first began to perform in this, he would make faces, faces that indicated pain, or happiness, or relief, when his father was manipulating this way.
But he found that the audience responded much more comically, much more immediately, if he just kept his face expressionless.
And that's where they're developed, the probably mistaken label for Keaton, of the great stone face.
And in his films very often you'll see this young man facing monumental catastrophe.
His face would look out at things without ever changing expression, and there is something kind of comical about that.
You'll see some examples of this in the general.
Well, you'll see in the two short films that Keaton began to make that he enters film at already a relatively sophisticated time.
And the shorts I'm asking you to look at are very interesting, remarkable films in their own way.
And one of the things they teach us, and one of the reasons I want you look closely at Keaton is they show us yet again another aspect of what I mean when I talk about the Fred Ott principle, this development of evolution and increasing complexity.
Because if you look at Cops, or if you look at One Week, what you will see Keaton doing in those films is learning how to link jokes together in some sort of a coherent way.
And there are wonderful coherent sequences in these films.
But still one doesn't feel that there's a real character in them, that there's a sustained, coherent, fictional universe being dramatized.
Maybe in Cops one does feel this.
This much more sort of textual coherence in Cops.
But what we can see Keaton doing is beginning to move from the idea of a single joke to a sequence of jokes.
And then, by the time we get to the general, moving from a single joke to a sequence of jokes to a sustained narrative, which incorporates within it various kinds of comical jokes, and also sequence jokes.
Because Keaton's best jokes are not single jokes at all, they're one liners, but the joke will be told again and again.
Or really, what will happen is he will do something, and you'll think how could.
And then, the next step will top what he's done before.
Then, the next step will top.
You think that he got to a point where it'd be impossible to add another comic complication to what he's done when he, of course, does it.
And this becomes one of the deep principles of Keaton's comedy.
There are numerous examples of it in both the two shorts, and in the longer film The General.
But let me mention an example from The General.
Very early in The General, there's a sequence-- it's probably the quintessential Keaton sequence, and I won't go into such detail as to spoil the film for you.
But let me just say, it involves a cannon, and it involves Keaton.
Keaton plays the engineer of a locomotive, a locomotive engineer.
The general of the title is not a military general, it's a locomotive, the locomotive is called the General.
And the film recounts a historically true story of a time when the Union Army came into Confederate territory, and stole a locomotive belonging to the Confederacy.
And Keaton, in a comic vein retells this story.
When you watch this cannon sequence, you'll see exactly what I mean.
And this sequences at the heart of James Agee's essay on the silent film comedy.
When you read that essay, you'll see even more clearly what I mean by a trajectory joke or a sequence joke.
Which builds, and builds, and builds, and keeps topping itself.
Well, this cannon sequence is a perfect example of that.
And it's a perfect example of it not only because we see the joke getting more complicated as it goes on, but as the joke becomes more complicated something else happens.
The joke becomes more than a joke, it becomes a kind of comment on experience.
It becomes a kind of comment on what life is like.
That is to say, we move from something that simply makes us smile to something that makes us think about the world, that comments on the world, that imagines the world in a coherent way.
I'll try to be more explicit about this in a moment.
So in Keaton's career, we see him doing astonishing things as an Acrobat.
And we would admire him just for his acrobatic performances in these films.
In The General for example, the way in which when he's piloting his locomotive, chasing the people who have stolen his original locomotive, how he's the only one on the train.
And he crawls all over the train.
While the train is running he'll crawl up to the front of it.
He'll run to the back of it to put more fuel in.
And also, you can see this stuff happening, really happening.
It's not faked in any way, because there's a camera following the train as it's moving.
You can see it's really moving, because you can see it.
You can see it moving on the track, and you can see the world behind it.
And you can tell that Keaton is not taking these effects.
Well, this cannon sequence is a sublimely rich example of this complex process I've been describing.
And it has to do in part with what I call the multiplicity principle, Which I will define in a moment.
So we see Keaton as an acrobat actor.
As a man, much more than Chaplin, interested in the technology of motion pictures, and of cinema.
And you'll see a couple of examples in the films you going to look at tonight in which we see Keaton exploring in a somewhat systematic way the nature of the illusion of motion pictures.
There's one remarkable comical moment in One Week.
In which the heroine gets into a bathtub, and it looks like she's sitting in the bathtub.
She's starting to get up, and looks like you're actually about to see some unmentionable parts of her body.
And just as that's about to happen, a hand comes in front of the camera and blocks you.
And it's actually a very disturbing moment in a way, because it interrupts the narrative flow.
It's an intervention from the outside.
Whose hand did it?
It's not a character in the film.
It's the director's.
Very bold moment, because what it does is it reminds you of the film as an artifact.
It reminds you that the film is an artifact.
In other words, it's a moment of self-reflexiveness self-consciousness that you would never seen a Chaplin film, but shows Keaton's astonishing interest in the technology of motion pictures.
I had many examples that Greg and I had ready, but I don't have time to show to you.
Maybe you could put up, while I'm talking, The Playhouse.
One of his most famous examples of this is a film called The Playhouse, in which he ends up playing every role in the movie.
And the way he did was incredibly interesting.
He would shoot the film, but he put masking tape over the lens of the camera.
And only one little, tiny part of the lens of the camera was what would expose the film.
And he would shoot that, then he would rewind the film.
And then, if he were over in this corner for that shot, then he'd move over, and sit in the other corner.
He'd put a new kind of tape on the camera, film it again.
And the film you're going to see will show this.
And you can see, they're all Buster.
Again what it shows is he's a technician.
He's interested in the technology of the motion picture camera, and what kinds of manipulations are possible.
He's the conductor.
As it turns out, he's going to also end up being the only person in the audience.
And then finally, what I've already implied is that Keaton's jokes and his technical interest also end up making him a metaphysician, a philosopher, an artist.
Now Keaton himself could never articulate his own ambitions in this way.
If you said, Mr. Keaton, your vision of the absurd universe is more complex and rich than Jean-Paul Sartre.
As in fact, it is incidentally.
He would laugh.
He wouldn't even understand what you were talking about.
Not only because he'd never read Sartre, since he died before.
He was alive live when Sartre was alive, actually.
But the universe that he projects has a kind of content or a meaning that's a function of how intelligent he is about exploiting his medium, and telling his stories, as if his jokes express a vision of life, a sense of experience.
What you're looking at is sort of minor Keaton here, but it shows his technical interest.
So let me explain what I mean basically by the multiplicity principle.
This process of accretion and complexity I've described-- at a certain point you get to a point where what began as a gag or as a mere entertainment takes on a density or a texture that makes you think, wait a second, the vision of experience that's projected in this text is intrinsically interesting.
That is to say, there's an understanding of life that's embedded in the story.
When you reach the point where the text itself embodies some sort of understanding of life, and an understanding of life that's coherently presented, powerfully presented, what you're looking at is what I call art.
You might just call it more intelligent entertainment.
But this is a way of distinguishing between entertainment and art.
At which there's a certain point at which entertainment becomes so complicated that it's more than mere entertainment.
When does that occur?
Well, one way to tell when it occurs is to watch for I call the multiplicity principle.
When events in the film play more than one function, do more than one job.
When they forward the story, but also declare for character.
When they make you laugh, but also express a vision of life.
And at the same time are consistent with the character's psychological nature.
That multiplicity, that density, that texture is what we associate with works of art.
A bad example would be you are familiar, I hope, with the kind of bad soap opera line in which a character comes into something like this, oh, hello, John, my long-lost brother who just got out of the insane asylum after 15 years of unfair misery.
What's wrong with that line?
Why is that a stupid line?
Although we hear them all the time.
What's wrong with it?
Sounds excessive, and awkward exposition
It's very awkward exposition, but why is it so awkward?
What makes it so awkward?
Its intention is clear.
It wants to present information to the audience.
But in order to do that, what has it violated?
It ceases to be something that follows the nature of the character
It ceases to be something that honors the character's nature.
Exactly right.
That's a very good answer.
What's your name?
Michael
Michael.
Thanks, Michael.
That's very good.
I want to give information to the audience.
In order to do that I'll compromise the character's basic nature.
No sister talking to a brother would recount the brother's history to the brother.
That just wouldn't happen.
So in order to accomplish one task, the text has compromised itself on another task.
But imagine a story in which this information emerges in dramatic ways.
Two characters are talking, and Mary says to John, oh, John, my brother's coming back from the asylum.
And she says, really?
I didn't even know he was in the asylum.
And Mary pulls back and says, I don't know if I should tell you.
OK.
So we have a dramatic scene, in which the information that the audience is supposed to have is presented in a way that maybe tells us something about Mary's nature.
And maybe tells us something about John's nature.
That is multiplicity.
That's what I mean by the multiplicity principle.
You will see many, many instances of what I'm calling the multiplicity principle in Keaton's work.
And you will especially see it in these astonishing, repeated sequences, and I'm talking now in a way about The General, in which we see the Keaton hero.
Who sort of is steadfast but also a muddler.
He keeps doing the same things.
They're not really likely to help them.
And then, he gets help from the outside.
Keaton's jokes repeatedly depend upon the idea of accident in a certain way.
And I'll come back to that in a second.
So think about the multiplicity principle, the difference between entertainment and art.
Again this is a principle I'll come back to, and try to illustrate with greater concreteness and greater leisure in later lectures.
But I want you to be aware of these issues.
So The General can be understood as a kind of culminating text in at least two ways.
It's a culmination of Keaton's career.
That is to say the jokes and gags that are enacted in the film recall many earlier Keaton films.
And the basic character of The General is the basic character of the Keaton hero.
A steadfast, unremarkable young man who perseveres in the face of catastrophic difficulties with mostly comic expectations, very small expectations of success.
Who often models through almost despite himself.
But I say, almost despite himself.
And this is where the complexity of Keaton's vision of life comes in.
Why I'm saying that Keaton moves from being merely a jokester, or being merely a great entertainer, to being an artist.
His jokes embody an idea of what experience is like, and it's a very powerful, mordant, mature idea of experience.
Because what happens again, and again in the small jokes, the sequence jokes in Keaton's work, especially in The General, and in the larger film, what happens again, and again, is an adventure in which a steadfast, relentless, young, hero does everything he can to accomplish his goal.
And it's obvious that he's up against forces far beyond himself, that he could never-- there's an unbelievable miss mismatch between this minuscule, tiny, figure and his powers, and the forces he's fighting against.
His one Keaton film called The Navigator, in which the Keaton character-- even more than with the locomotive in The General-- has to navigate an ocean liner.
And he's alone on the ocean liner.
And you see running up and down into the engine room, back up to the bridge, running all over the ocean liner in great difficulties.
There's a wonderful film, late film made in 1928, just his last silent film called Steamboat Bill Jr.
In which he dramatizes an astonishing hurricane.
And a long, extended sequence in which we see him trying to escape the hurricane, trying to contend against this natural force that's so gigantic.
So The General is a culmination in the sense that it gives us another version, and the richest version so far of the adventures of this kind of Keaton hero.
But what makes it metaphysical or philosophically interesting is that what this joke repeatedly does, and what the structure of the whole film repeatedly says is something like this, we get through, we make it through life.
We get through life mostly not because of anything we've done but because of accident.
We live in a contingent universe.
The idea of contingency is at the heart of Keaton's jokes.
We could call them jokes about contingency.
The word contingent is interesting.
Remember it means libel, but not certain to occur.
It means possible.
It means dependent upon, subject to, conditioned by.
In law you can say that something that is contingent is dependent for its effect on something that may or may not occur.
As in a contingent estate or legacy.
In logic, the same thing, dependent upon some condition, or upon the truth of something else.
Not true a priori, not necessary.
So what Keaton's jokes constantly do is they show us two things.
They show us a world in which human agency matters.
Because if this should schmendrick didn't keep-- that's a Jewish word for schlemazel, clown-- If this schmendrick did not keep on so steadfastly, he wouldn't survive.
But he's keeping on so steadfastly still wouldn't make him survive if there weren't the intervention of mere accidents, stuff he has no control over.
This one magnificent moment in One Week, for example, where he's standing at a certain place, and the house behind him falls down.
And he's happens, just by accident, to be standing by the open window, and the house falls, so he doesn't get hurt.
He used this joke more than once in other films because it's such a great one.
But it's a perfect example of what I mean, a partial example of what I mean.
Because he is not responsible for the fact that he escaped.
On the other hand, because he was trying to get away from difficulties, maybe he was in that particular position because of acts of his own.
So what this joke and vision says about the world is something that's true.
We all love the idea that we've gotten to where we are by our own acts, by our own conscious control.
The truth of the matter is that life is much more contingent than that.
It's partly accident, it's partly luck.
And something deeper than that.
Keaton's vision of the universe is one which suggests that while the universe may screw you up sometimes, it won't always do that.
The universe in Keaton's world is not angry at us, it's indifferent to us.
It doesn't give a damn about us.
It's not aware of us.
It's too big for us.
You can't come away from Keaton's films without a simultaneous sense on the importance of human action, of human agency.
But also of the inadequacy of human agency.
Of how contingent every experience, every adventure, every personality is.
And that and that meaning, that idea of the world is a profound one, I think.
So that even though Keaton is telling a joke, embedded in the joke is a kind of interpretation of life.
And embedded in the film is a kind of interpretation of life.
When you're looking at The General, I think you will see multiple examples of this kind of thing.
Let me say one last thing, I haven't mentioned anything about the structure of the film, I'm a little out of the way there.
Watch how the film is structured.
It's a much more complex variation on what I said about Lonedale Operator.
The film begins with him going up the track.
And then, the second half of the film has him going back down the track.
You may not be aware of it, but actually the film in a certain sense has its tail its mouth.
When he goes up the track, he has a series of adventures.
When he comes back down, he has a series of adventures that in a certain sense replicate what's happened before.
The film mirrors itself in a way, repeats itself.
It's incredibly elegant structure.
And the elegance of the structure reminds us also-- its own way, also reminds us of this principle of contingency that's at the heart of Keaton's vision of the world.
Let me conclude by reminding you again what I said earlier.
When I call Keaton a philosopher, and a metaphysician and a wise person about the world, I do not at all mean to suggest that if ask Keaton to explain how we thought about the world he could do this.
That that's the job of professors and literary and film critics.
What I'm saying to you is that Keaton, like many great artists, had an intuitive understanding of the world.
Which he expressed with greater, and greater complexity in the medium of the cinema.
And in The General, we have what is ultimately a culminating instance of that.
One last point about the film as a culminating example.
It's not just the culmination of Keaton's career.
It's a culmination of silent film as a whole.
Keaton would have expected his audience to recognize the degree to which this film was a parity or a mockery of famous historical films made by D.W. Griffith.
It puts down the pretentiousness and the martial valor that's celebrated in so many of the historical spectacles that Griffith and other serious directors were making at the time.
And it also goes back in a certain way to the very roots of film.
What was the very first narrative film?
Or one of the very first?
Also a film about a train robbery, called The Great Train Robbery.
Well, this is about a train robbery, too.
And who is the central character in this one?
Really, a train is the central object, the central-- more interesting in some ways than the character himself.
And there are many other ways in which the particular features of the general allude to, and in some sense summarize, the history of silent film.
And all of the original audience coming to see The General would have been, of course alert to these connections.
Would have been literate in these connections.
Wouldn't have needed footnotes or professors to tell them about it.
Because they would have lived through the history.
They would have been through the history of the previous 20 years.
They would have seen other Keaton films.
They would have seen D.W. Griffith.
They would have the history of the medium of movies in their heads when they sat down to watch The General.
So that's another way in which The General is a profoundly culminating text.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
Hello people.
Welcome back to the second week of our course.
I'm happy to have this wonderful live audience.
But I also would like very briefly to address our video audience, who will no doubt be watching this long after I'm dead on MIT's OpenCourseWare.
So think of this as a voice from the grave.
You and the video audience will sometimes notice that as I'm lecturing to this lively group, I will turn this way and look up.
You should realize that what I'm turning toward and looking up at is this projection screen, which we will show you.
And it sometimes has this outline on it, which hopefully guides us through each lecture and sometimes has other information and material on it.
Sometimes it isn't worth shifting over to show the video audience that image.
But that's what I'm looking at when I look so strange.
I'd like to begin today by taking a step back, by reminding you that the order in which we're watching the movies in this course is a little bit arbitrary because I wanted to use the feature films as the defining date.
And that's the reason that we made Keaton come first because The General is an earlier film.
It was almost a decade earlier than Modern Times.
But the truth is Chaplin entered film before Keaton.
And Keaton had certain advantages by the time he entered the movies, in part because of Chaplin's example, although also because of the fairly complicated way in which silent film had already elaborated itself after the first 10 years or so of the 20th century.
So I want to take a step back into that early phase, just before Chaplain becomes a filmmaker really in 1911, I think is when he joins the Keystone Studios.
Just to remind you of a story that is told systematically in David Cook's History of Narrative Film, which is part of our required reading in the course.
And I hope you will attend especially to the details Cook offers about those early years in Hollywood.
If you read those chapters from Cook about the early years of Hollywood before Chaplin, the movies before Chaplin, you'll recall that one of the most fundamental things that happens before Chaplin even appears is the establishment essentially of the industry I was describing last time.
The migration to the West Coast is worth mentioning.
Because movies, of course, begin their career in the United States on the East Coast.
Edison's first movie studio was in East Orange, New Jersey.
The Biograph Company was originally located on the East Coast.
And almost all the original earliest films, even the earliest films that were put into production and shown commercially, were made on the East Coast.
Why did people move to the West Coast?
Why did Hollywood become the sort of center of the movie industry?
What explains it?
It happens very early.
One might think of it as a very important moment in the history of culture, of popular culture, because for all the changes that the movie industry has undergone, Hollywood remains the kind of center of what is now a kind of global enterprise of movie making.
What helps to explain the move to the West Coast?
Nothing fancy.
In the back, yes
Film year around
They could film the year around.
Why is that important, especially in the early days?
You can make movies whenever.
You don't have to produce them in the summer
Yes.
You're not as dependent on the weather.
The Sun always shines in California.
If it's overcast, you can still film.
And there's no snow.
There's virtually no rain.
The weather is very conducive to the making of film.
And especially in this early phase, when film stock was relatively primitive compared to the complexity of film stock we have-- today, the film stock is so subtle that you can film in the dark.
You can film by fire light.
Even amateur folks owning camcorders can film virtually in the dark.
They can film by electric light inside a room.
Or tape, they can videotape in that way.
In the earliest days of film, you needed a tremendous amount of light to expose the film.
And what it meant that you were dependent on the weather.
So that was one major reason.
But there's another one that's even more important in some ways, even though that would seem a sufficient reason for explaining why the migration to the West Coast occurred.
There's another even more powerful one.
It has to do with the idea of patent warfare.
Yes.
What's your suggestion?
You don't have to give the right answer.
Give a--
Different types of terrains fall into those--
Oh, that's also true.
Yes.
The environment of Southern California is particularly conducive to a variety of film projects.
There are mountains nearby.
There's ocean nearby.
There's desert nearby.
And, of course, that's also something that makes it a uniquely valuable environment.
Maybe not a unique environment, but a particularly valuable environment.
And that's a good answer.
And you're right about that.
But there's something else.
And it has to do with conditions in the East and Thomas Edison.
Remember, Edison in a certain way wanted to lay claim to virtually every invention in the early part of the 20th century.
And he was primed to do so.
He was already recognized as a great genius, as the premier entrepreneurial inventor of his era.
And he immediately, as he had done with other projects of his, began to take out patents to try to control the use of movie technology.
He wanted to become the Edison monopoly.
He was fantasizing that he would control the movie-- whatever the movie industry would become, Edison's company fantasized it was going to take control of that.
And, in fact, the Edison Company hired law enforcement officers and, according to some historians, even in some places hired goons, thugs, to go and break up the sets and the shooting of rival producers who were, according to Edison, infringing on Edison copyrights.
So the other reason-- another reason for the move to the West Coast was a lot of the entrepreneurial movie makers wanted to get free of the patent police.
They wanted to get out from under the scrutiny of Edison and his police.
And by running to the West Coast, they were able to do so.
And, in fact, Edison was unsuccessful in controlling the technology, as Cook and other historians have made clear to us.
So the move to the West Coast is a very important one.
And I will return to the cultural, the historical, and ideological or intellectual implications of that move to the West Coast in a few lectures, maybe next week if I get the opportunity.
Because I want to talk a little bit more about what it meant for American movies that film was made on the farthest western verge of the society, away from cultural centers, unlike the way in which movies developed in Europe.
And I'll talk about that implication a bit later.
Probably next week, when we talk about the great German silent film that concludes our segment on silent films.
So even before Chaplin enters, the rudiments-- or more than the rudiments, the basic system of mass production, of studio-based production, of manufacture of the films in one place and then the distribution system that developed in which theaters essentially rented or purchased films from the producers was already in place.
And, of course, the two great figures here is Thomas Ince-- and Cook writes very well about Ince and his contribution to the movies-- probably the person to whom we could attribute the invention of the movie studio, I suppose.
And Mack Sennett, who developed the Keystone Studios, out of which so many important early comedies emerged and the studio which Chaplin joined when he came to the United States to become a movie maker in, I think, 1911 or 1912.
So Ince and Mack Sennett are important.
And one might say one further thing about this moment.
As you know if you've read Cook, Sennett created what came to be called the Keystone Studios.
And Keystone Studios were named, I think-- or maybe it was the reverse.
I'm not sure-- was associated with their prime product in the early days.
And they were called the Keystone Kops Komedies.
We're going to show you a fragment of those in a moment.
They were frenzied action, frenzied physical action.
So this is the era.
This is the moment, the period in the first 10 years of the movie industry in the United States, when the basic system is being, by trial and error, worked out.
And by the time Chaplin comes to join the system after the first decade or so of the 20th century, the system essentially is in place.
There are already a number of famous performers, both in comedy and in drama.
There are emerging certain particular studios who have refined the mass production technique to produce their movies on a regular basis.
So an expanding market for the movies is being met by a systematic manufacturing process that Ince and Sennett begin to work out in systematic and long-lasting ways.
In ways that in a certain sense continue in residual form, in vestigial form, to dominate the production of movies even today.
When we were talking about the primitive and the simplicity of early film, there was one film I had on the stocks we didn't have time to show you last week.
And I thought I would show this to you as a way of illustrating again how relatively simple the early films are.
[VIDEO PLAYBACK] See how influential that first sneeze was.
The name of this film is The Whole Dam Family.
There's Mr. Dam.
There Mrs. Helen Dam.
[MUSIC PLAYING] There's their son, Jimmy Dam.
The simplicity of these movements are part of what's interesting to us.
I mean this is still an era in which people are just fascinated by the idea that movies capture movies.
But what's the joke here?
The joke is embedded I suppose in the title, The Whole Dam Family.
There's Miss.
I.P.
Dam.
There's Baby Dam.
What an actor.
You see how it's framed as if it's a painting, how stationary the camera is.
There's the Dam Cook.
What's the joke here?
Finally, we're going to come to the Dam Dog.
What's going on here?
It's kind of risque.
Remember, we're talking about late Victorian, post-Victorian era.
What's the joke here?
It's the play on the word "damn."
It doesn't strike you as shocking.
But, in fact, if you said, damn you in 1890 or in 1900 in certain quarters, people stepped back and said, oh, my goodness.
There's a kind of semi-vulgarity there.
In other words, it's not an accident that this is not the Thorburn family or the Smith family.
It's significant that it's the Dam family.
Why?
Because they want to make a joke.
What's wrong with the joke?
Why is it a stupid joke for a movie?
Because it's not a visual joke.
What does it depend on it?
It depends on language.
In other words, it's almost as if the whole point of this film depends on a joke that depends not on anything that you can experience visually, but that is resident in a verbal trick.
And it's a trivial joke.
It's not much of a joke, but the film depends upon it.
And it's evidence in a very modest, but at the same time dramatic and obvious way of this process whereby the new medium, this new visual medium, has to emerge out of media, some of which are not visual at all, but are verbal.
And it's as if early filmmakers haven't yet learned fully how to exploit the qualities of the medium they're working in.
So it's a verbal joke, rather than a visual joke.
Much more powerful and much more successful were the Keystone Kops Komedies themselves.
And we have one example of that to show you.
I'm not sure that this is a characteristic example.
But it's close enough to give you as a sort of sense of what the Keystone Komedies were like before Chaplin entered the game.
[VIDEO PLAYBACK] [MUSIC PLAYING] The essential joke of the Keystone Kops Komedies-- I don't even know how many there were, perhaps hundreds.
There were certainly dozens and dozens.
They were immensely popular early short films.
Some film scholars have speculated that one reason for their popularity was that they made fun of policemen.
That was an implicit sort of mildly anarchic impulse in these early comedies because authority figures, people in uniform, were being shown to be clownish in some way.
The primary element, of course, is to find ways to exploit the movies' capacity to show motion.
And one reason I want you to see this relatively crude example is that there's a certain sense in which the short film that you saw last week from Keaton, the film Cops, is a sort of reprise of this tradition of movie making.
Although you can see how much more coherent and character oriented Keaton's short film was, but the way in which especially the cops are mocked and shown to be completely clownish figures.
Almost all of the Keystone Kops Komedies to my knowledge, involved incompetent policeman, who engaged in mad active chases, which often ended badly for the policeman.
Sometimes they would end up all flying into a river, or an ocean, or some other natural disaster.
And I think you can also see-- that was actually a very dramatic effect for an early film.
So there's an awful lot about this that would have interested early-- seeing that car turn over, some people in the audience might never have seen that in the movies before.
And one could say that these films in a certain way anticipate a great deal of the ridiculous action adventure activity that we see in films even today.
But this is typical of those early films, where they'll make incredibly implausible jokes just for the sake of making the joke.
And I'm not sure-- all right, a Mack Sennett comedy.
This is the kind of comedy then that was being made by the Sennett Studio before Chaplain arrived.
They were also certain figures-- Fatty Arbuckle, the fat comedian who worked with Buster Keaton, is an example.
Who had already begun to establish comic personae, something like the persona that Chaplin and Keaton created.
So they were not the first to do this.
So there was a subtler and already somewhat more character-oriented comedy beginning to emerge before Chaplin appears.
But Chaplin's entry into American movies is also a turning point, a fundamental turning point, as many scholars have recognized.
And it's a turning point primarily because what Chaplin introduces-- I mean introduces many, many things to comedy.
But he introduces one thing especially-- well, two things especially.
The first thing that he introduces is what we might call the principal of the slow down.
Instead of frenzied motion all the time in Chaplin comedies, there are moments of quiet.
And in these moments of quiet-- and this is why they're important-- what begins to happen is an attention is thrown away from violent, frenzied physical action toward character.
What Chaplin introduces into movies is a somewhat more systematic and serious interesting character than had ever occurred before.
And what follows from that introduction, from slowing down the action, creating moments of human interaction in which the interest is essentially psychological and individual, rather than an interest simply in frenzied motion and jokes, or frenzied motion and suspense, which is what you would get from a Griffith melodrama, what you're getting is-- so this introduces several things.
First of all, it introduces a tonal variation.
It means that in the middle of comedies, there could be quiet, serious moments.
And, in fact, it becomes the hallmark of Chaplin's comedy that it really is often not funny or at least that it's based on unfunny materials.
And that there are poignant and moving moments.
It's as if Chaplin combines comic and melodramatic elements, especially in his best work.
I don't mean that when he came into the films he began to do this.
In fact, he discovered how to do this in the course of his career, about which I will speak in a moment.
So Chaplin brings a new interest in character and in psychological nuance.
And, because of that introduces tonal variation, a complexity of tone.
We think of the Tramp figure as indelible and unchanging in a certain way.
His costume always the same, his face always the same, the kinds of problems he gets into always similar.
And that's partly true.
But the Tramp was not born fully dressed in Chaplin's brain.
He evolved the Champ over a series of short films in the period from 1911 through about 1913 or '14.
And one way I can dramatize this for you is to show you how Charlie looked in the very first film he appeared in.
He was not a director in this film.
He was just an actor.
And he was a walk-on actor.
He was not the star.
The camera had an immediate affinity for him.
And after people saw this film-- this is the first film he appeared in.
This is a still from a film called Kid Auto Races at Venice.
It's a kind of documentary about soapbox racing.
And the Chaplain character had just been hired by Mack Sennett to join the Keystone Studios.
And he made an appearance in the film.
He really didn't have anything to do.
But his appearance in the film caused all kinds of interest in the audience.
The audience wanted to know who that character was, why he was so interesting, and what was interesting about him.
You can see the beginnings of the Tramp costume there, but not the complete Tramp.
Here's another early version of Chaplin, wearing a long coat and not looking very nice.
Charlie is supposed to be drunk in this picture.
That's one reason he looks nasty.
It's in a film called Mabel's Married Life.
And he's actually in conversation with a dummy in this sequence.
But you can see here that the long coat violates our idea of who the Tramp was.
So it took awhile before the Chaplain persona completely emerged.
And it was partly a matter of trial and error, although very quick trial and error.
I want to say a word about Chaplin's career, to give you a fuller sense of how swiftly he became an icon and how significant his example is for our understanding of the power of movies.
Chaplain's dates are 1889 to 1977.
He was born in England.
And his early childhood is actually the stuff of Dickensian melodrama, both terms I choose with some intentionality because there are parallels between Charlie Chaplin's early life and the victimization that Dickens dramatizes so powerfully in many of his famous novels.
And there are parallels between Dickens' own childhood and Chaplain's childhood.
Dickens, the great 19th century novelist, who, like Chaplin, was interested in victims, laid emphasis on the problems of vulnerable people, of women, of children, who wrote variations on what we might call the mortgage melodrama all his career.
His greatest works could be said to be forms of melodrama.
I'm talking about Charles Dickens, the great 19th century English novelist.
And Chaplin grew up in an environment that was steeped, not only in Dickens, but in the traditions of stage and novelistic melodrama that permeated European culture and especially English culture of this time.
But his life was a very difficult one in some ways as well.
His father died when he was eight.
His mother was in and out of the institutions all of her life.
There were periods in Chaplin's life, when he was just a little child, when he was literally completely abandoned.
And he and his older brother spent time living in doorways on the streets of London.
He was literally a homeless urchin as a child.
He began to act on the stage from the age of seven and almost immediately was recognized as having astonishing gifts.
He worked from 1906 through 1913 for a theatrical company in England called the Karno Company, K-A-R-N-O.
And it was a company that put on a certain kind of theatrical anthology, in which there would be skits, comic skits and serious skits acted out.
Sometimes when the skits were over, there would be song and dance numbers, sometimes just straight singing.
A kind of British variety show, influenced especially by the traditions of the British musical.
And the Karno Company was a traveling company.
And first, Chaplain worked with the company all around the British Isles.
And then, in I think 1909 or 1910, the Karno Company made a triumphant tour of the United States.
By this time, Chaplin was one of the leading actors in this.
One of the great features of the Karno Company performances were pantomime performances.
And Chaplin was a particularly brilliant pantomimist.
So he had a very significant career.
He was already a successful performer on the stage in the Karno Company and drew on traditions of performance.
Chaplin's father was a popular singer.
His songs were published and he sang in the musicals for a certain period.
And Chaplin's mother had visions or fantasies of being a performer herself and used to perform privately to Charlie.
So Charlie was steeped in performance, steeped in theater.
And as a young boy, from the age of seven on, was already sort of acting on stage.
Almost continuously after the age of eight or nine, made his living as an actor.
So as a relatively young man, he made a trip to the United States with the Karno Company.
The Karno Company toured various cities in the United States, was a great success.
A second tour almost immediately followed, I think within six months.
And on that second tour, Mack Sennett saw Chaplain perform and hired him away from the Karno Company.
Saw that Chaplin had certain gifts as a performer that would work in the infant medium of the movies and hired him away.
And it was an inspired decision.
When Chaplain first joined the Keystone Studios in '19-- I think it's 1911.
The date is in Cook.
So be careful.
My date may be a year or two off.
But around 1911, he joins the Karno Company.
His first few films, his first three or four films, he's just a performer in.
But he has an immediate affinity for the camera as well.
And within half a dozen films, he's already beginning to direct as well as to act.
And within a relatively short time, he gets to be in control of every aspect of the making of his films.
We can get some sense of how richly and symbiotically the audience reacted to the character of the Tramp that Chaplin created by talking in monetary terms about the development of his career, about the progress of his career.
He officially joined the Keystone Company in 1914.
He was hired in 1911 and 1912, came over to the United States.
And from 1914 and 1915, just that one year, he worked for the Keystone Company and made 35 films in that one-year period.
He received what was then a princely salary of $150 a week.
The final films of those 35, the last 15 or 20 or so of them, featured the Tramp figure, who had now been fully elaborated in various adventures that became characteristic of Chaplin's vision of the Tramp.
And they were unbelievably popular, by far the most popular shorts that had ever been sent out of any movie production company.
And they made Chaplain a household name, first in the United States, and then fairly quickly all over the world.
So successful were those early films that in 1915 he was hired away by another company, the Essanay Company, E-S-S-A-N-A-Y.
And for another year 1915, 1916, he worked just for the Essanay Company.
But his salary went up from $150 a week to $1,250 a week in one year.
And that's a gigantic-- more than a thousand dollars a week in 1915 is an unbelievable fortune, of course.
He makes only 14 films in that period.
So he goes from making 35 films in a year to making only 14 films.
And those 14 films are more highly finished.
He spends more time with them.
We can see in this period, we can see Chaplain learning.
We can see Chaplin enacting in small the Fred Ott principle.
We could see Chaplin enacting in small the principal I talked about last week, in which we see him learning to develop performance styles that are appropriate to the camera, appropriate to cinematic experience, developing techniques as a director to emphasize certain aspects of the Tramp's character, experimenting with tonal variation in his films, experimenting with various kinds of comic devices, some of which he imports from his performances with the Karno Company and many of which he refines and changes because he's working in the movies.
One famous example of this migration from stage to screen is a bit that Chaplain was especially famous for when he worked for the Karno Company.
He did a bit them, a pantomime, called the Inebriate Swell, the inebriated-- the drunk aristocrat.
That's what a swell is, a rich aristocrat, a man about town, the inebriated-- the drunken aristocrat.
And it was one of the most popular pieces that he performed on stage for the Karno Company.
He remade a version of it, a very famous film.
It's one of the very few short Chaplins where Chaplin doesn't play the Tramp, where he's dressed up in a tuxedo and he plays the inebriated swell.
And it's a film called One A.M. And in the entire film, Chaplin plays a man who is drunk and who keeps encountering objects that misunderstands.
So as an example, he comes across a clothes tree in his house.
And instead of hanging his clothes on it, he embraces it as if it's his wife, that sort of thing.
And almost the entire length of One A.M. has only one performer in it, Chaplain himself.
And he kind of recreates and modifies what he had done on stage in that performance.
It is in many ways a very brilliant performance and shows how imaginatively the Chaplain character can interact with objects, a theme to which I'll return tonight when I talk more systematically about Modern Times and about certain of Chaplin's favorite strategies for establishing character.
So he works for the Essanay Company.
He makes 14 films, including in 1915 a film called The Tramp, fragments of which I'm going to show you in a few minutes.
Then he's so successful again.
And the 14 films he makes starring the Tramp become international-- I don't know what to call them-- international phenomena, best sellers.
And the Tramp becomes the most sought after and the most memorable character in the movies.
So he's hired away again by the Mutual Company, one year later.
And from 1916 to 1917, he works for the Mutual Company, in which he makes this time $1 million for that year.
And he makes only eight films in that year.
So he goes from $150 a week, to $1,250 a week, to $1 million a year within three years.
And each move gives him greater creative freedom, gives him more autonomy.
He begins to sort of really refine his art.
And we can see the process happening in these shorts.
In the films you're going to see tonight, both of the short films, Easy Street, 1917, and The Immigrant, also made in 1917, come from this Mutual period.
And I mention this to you because they're already in some sense sophisticated films, even though they're not feature-length films.
And you can see that Chaplain is bringing all the resources of his narrative skills, of his skills in creating character to bear on those films.
They're already in some sense sophisticated works of narrative.
And there is an apprentice period that occurs earlier, which I have just described to you.
Finally, in his career, he moves, in 1918, to a company called First National, for which-- those are the people who pay him a million dollars.
I'm sorry.
I made a mistake.
His Mutual salary was a mere $670,000.
He gets $1 million when he goes to work for First National in 1918, when he makes, among other films, one of my favorite Chaplin shorts, a film in which we see the Tramp character playing a GI in the first World War.
It's called Shoulder Arms.
And it's a characteristic Chaplain short, except that now what Charlie is doing is contending with the Kaiser's army.
He's fighting World War I.
In earlier films, we'd seen him fighting ice storms.
We'd seen him fighting against hunger.
We'd see him literally fighting in the boxing ring in a film called The Champion.
We'd seen him working as an assistant to a pawn broker in a film called The Pawn Broker-- a whole series of jobs.
And by the time we get to 1918, it's almost as if the Tramp has held virtually every job you could imagine.
He can't hold a job for long because he's a bum.
He's a tramp.
He's footloose.
Again, a theme to which I'll return when we talk about Modern Times this evening.
And then finally, in 1919, Chaplain has become such a gigantic figure, such a international icon, that he joins with certain other icons, D.W. Griffith, the greatest, most famous director of the silent era; and Mary Pickford and Douglas Fairbanks, the two most famous actors after Chaplin in the silent era.
The four of them get together and they form United Artists.
And they formed it in part to create their own company so that they would be able to control their artistic output.
And Chaplain, working for United Artists, beginning in 1919, Chaplain goes to work on his feature films.
And he makes the great film, the great silent features, that are at the heart of his achievement.
He makes The Gold Rush in 1925; a film called The Circus in 1928, a more imperfect and broken-back film in some way, but full of some of his most imaginative comic performances; one of his great films in 1931, called City Lights.
And then finally, in 1936, the film you're going to see tonight, already deeply into the sound era, a film that remains in some fundamental way a silent film, Modern Times.
Chaplin then goes on, after the sound era is over, to make a series of movies in the sound era.
And they all have some modest interest for us because they're Chaplain films.
Some images from them that remain very famous to this day.
In fact, one of them was in the New York Times today.
Did any of you notice it?
There was an article about a scholar of globes, who had found evidence for the idea that a particular globe in a German museum, which was said to have been Hitler's globe, was not Hitler's globe.
But in the middle of the story, what they-- any time a person of a certain age hears the word "globe" and "Hitler," they think of Charlie's film, The Great Dictator.
You're smiling.
Why?
Do you know the image?
Have you seen The Great Dictator?
[INAUDIBLE]
What is the image they're talking about?
Playing with a globe
Yes.
There's a moment in the film-- it's a moment of great-- it's not really a very good film.
And Chaplain's sound films are mostly of historical interest, despite what some Chaplain buffs would say.
His great work is-- he's a silent artist.
And the sound film caused problems for him that he couldn't fully adjustable to.
But every one of his sound films has interest and has brilliant moments.
And the one you're remembering is one of the most remarkable and astonishing symbolic moments in Chaplin's work.
Yeah.
It's an incredibly ambitious film.
It tells us how ambitious a figure Chaplin was because he makes a satire of Hitler and Mussolini in an era when Hitler and Mussolini are not yet recognized for the egregious, monstrous, evil figures that we know them to have been.
They were taken seriously.
They had been voted into office.
Millions and millions of people thought they were admirable leaders.
So when Chaplain made The Great Dictator in '19-- what- '40 or '41, he was doing something incredibly daring.
And he was actually showing that he thought the movies were an art form capable of taking on the largest political subjects.
So even though there are problems in the film, and it's not an absolutely great film, not memorable artistically in its own way, there are moments in it that are remarkable.
And it's also a significant document because it reminds us of Chaplin's ambitions as an artist.
Chaplin was one of the very first filmmakers, along with D.W. Griffith I suppose, to realize that the movies had an aesthetic possibility, latent aesthetic energies and powers that if they were tapped, would make the movies the equivalent of traditional art forms, like theater and the novel.
And, of course, he was right about that.
And he even made some films that achieved that level of seriousness and resonance.
The moment that you're remembering, what happens in it?
He's like dancing with a globe
Yes.
It's a moment of sort of manic glee, in which the Hitler character, played by Charlie Chaplin, takes a globe out and it's like a balloon.
And he pops it up and down.
And we see him throwing it up and kicking it with his hat, like that.
And it's a kind of fantasy of world domination.
And it's supposed to show what a ridiculous and infantile figure this mock Hitler is.
And it is the most memorable image from that film.
So in this article, that really had nothing to do with Chaplain, there was a picture of Charlie playing Hitler, kicking the balloon up and down.
And one reason that I think it's significant, that it's a mark of Chaplin's greatness as an artist, that even in his imperfect texts, even in the films that really do not belong to the category, to the most important category of his most lasting and valuable art, images from those works are still part of our common cultural inheritance.
He's a very great artist, the first great artist of the movies.
And a figure who's imaginative creations remain vital and alive today in a way that very few artists from that era remain, similarly accessible and important to people.
And it's a mark of the reach of Chaplin's imagination that this is so, of his importance as a figure.
Well, there's much more one could say about the development of his career.
But I want to cut this short because I want to show you the moment-- I don't mean that there's only a single moment.
But I want to show you fragments from the film in which we can see Chaplin really discovering certain aspects of what he wants to do and also where we can see Chaplin screwing up.
We can see Chaplin at a stage before he becomes the self-confident, totally autonomous artist he becomes relatively soon.
Can we put up fragments from The Champ-- from the Tramp?
This is a film appropriately called The Tramp.
[VIDEO PLAYBACK] [MUSIC PLAYING] And it's a fairly characteristic Chaplin film.
But I think it's the film in which, among other things, Chaplin discovers that he can marry melodrama and comedy.
It's the film in which he discovers that he can do effects that are pathetic, and poignant, and sentimental and mix them with comedy in ways that are interesting.
So here's a scene where Charlie is rescuing a damsel.
And he does it in a kind of comic way.
He's a kind of bum himself.
But he's always a kind of chivalric figure.
Can you jump ahead?
Let's just show the next one, Greg.
I'm just going to show you a bad moment here.
He rescues this girl.
She turns out to be the daughter of a farmer.
He's then hired by the farmer in gratitude.
And then we get this bit.
And I'm showing you this bit partly because I want you see that it isn't really all that funny or that it's just kind of easy slapstick.
I don't mean that it's without interest.
It obviously has some interest.
But this incompetence is inconsistent in some respects with the coherent character that he's going to begin to create.
It's never that the Tramp becomes this fully competent character.
But this kind of joke here, look at this.
That's his foolish joke.
That trivializes him in a way.
Can you think of examples of that in Keaton?
Remember in Cops, where, for example, when he's riding along in his car, he devises that mechanism that ends up punching the cop.
That would be an example of a kind of-- it's a joke.
But it sort of violates the spirit of the movie.
It seems a piece of debris that is forced into the film, that may have a comic moment, but makes the film less coherent.
That kind of thing is characteristic of the Tramp.
A bit later in The Tramp-- I don't have time to show you this passage-- a bit later in The Tramp, we see Charlie confronting the woman that he had rescued and discovering for the first time-- is it a long sequence, Greg?
No.
It's only a half minute
Well, put it up while I'm talking, if you can find it.
You can see in this sequence, which I'll let run behind me, while you pay attention to my words-- --and while the camera looks at it and then comes back to me.
What you can see happening in this sequence is Chaplin discovering how he can be, not just comical, but sentimental.
How he can elicit sympathy from his audience, create sentimental effects.
In other words, what Charlie has just done here in this image is he's meeting the girl's boyfriend.
He didn't know the girl had a boyfriend.
He was falling in love with her.
Watch his expression here.
You see, he's very upset by what he's learned.
In other words, this is not a funny moment.
And this is the moment, I think, in Chaplin's career, where he realizes the films he's going to make should not be thought of as simply comedies at all, but as something more complicated than that, as comedies with heart, as comedies devoted to character, as comedies that are interested in human interactions.
And that discovery tremendously enlarged movie comedy, tremendously enlarged the movies more generally.
And then we can see Charlie saying-- Charlie says goodbye.
And he writes this little sad message and he leaves her.
So it's the moment when pathos, sentiment, enters Chaplin's work.
And in the films you'll see tonight, you'll understand why, after he made this discovery, he began to meld these two elements more and more fully together.
So this is a way of reinforcing what I called last week, the multiplicity principal.
What we can see is that in those jokes with the pitchfork, there was no multiplicity.
They were just funny because Charlie didn't know how to use the pitchfork.
But later moments in this film, and in many later films, we see Chaplain beginning to create effects which have more than one consequence, more than one significance for the audience.
One of the most remarkable things about the chase scenes that Charlie Chaplin develops and begins to perform in his films-- and you'll see wonderful examples of them tonight-- is that the chase scenes themselves are tremendously moving.
They're tremendously comical.
They're full of comic bits that are just as imaginative as any comic sequence one could ask for.
But they do something else, in addition to amuse us.
They declare for Charlie's character.
When Charlie runs away from someone, what we can see him doing is using his intelligence, using his improvisatory capacity to figure out how to get away.
So that even when he's running, he is also expressing his nature.
And when the audience sees the imaginative, or clever, or daring way in which he makes his escape in a particular chase sequence, they're not just being amused by a joke.
They are also understanding his character in a deep way.
He's defining his character, even as he's also amusing us.
So we can feel the multiplicity of that, the density, the texture of that, always a mark of superior entertainment, always a mark, I would say, of art itself.
Well, one way to clarify some of this, and especially to clarify Chaplin's remarkable popularity, is to talk about the Tramp as a myth, as a mythic figure.
And if you think about his costume, you'll understand partly what I mean.
He wears a coat and vest that are too tight and pants and shoes that are too big.
So the principle of mismatch or of ambiguity is built into his costume.
And then he carries this cane and this hat.
Now, he shows a kind of elegance.
But it's a kind of shabby, down-market elegance.
But the question is, is Charlie downwardly mobile?
Is he an aristocrat who's fallen on hard times?
Or is he a poor person, who has put on airs and aspires to the world of tuxedos and fancy clothes?
This question is never answered.
And that ambiguity in his character may be part of why the audience is able to identify with him.
There's an uncertainty about where he belongs, a kind of mismatch or an ambiguity inherent in his costume.
His body itself is a source of his mythological status.
It's small, dexterous, energetic, incredibly graceful.
He's one of the great acrobats and physical dancers, physical performers, that you'll ever see.
And in the films you'll see tonight, you'll see many examples of this, including examples of Chaplain on skates, of Chaplain doing the equivalent of dances, and of his dexterousness in situations involving running or escape, just as powerfully as anyone.
We'll see dramatizations of this, just as powerful as anyone could ever imagine, ever hope for.
And then his face.
So his costume, his body, and then his face are part of the myth as well.
He has an unbelievably expressive face.
His face is a theater in itself.
And it was Chaplain, more than any other early director, who explored the implications of the closeup.
And I'll show you some examples tonight of the way in which Chaplin manages to use the closeup in silent film for incredibly-- to do so.
And to be, at the same time, what we might call deeply, incredibly eloquent, a silence that is eloquent because of the qualities in his face.
And the moral qualities of the Tramp are also appropriately ambiguous.
He has chivalric qualities.
He wants to defend children and vulnerable women.
He's always a kind of defender of the weak.
But at the same time, he's an opportunist and also a survivor.
And these qualities don't fit perfectly together.
And we see both elements in the Tramp character, again and again.
Again, tonight, in The Immigrant especially, you'll see both sides of the Chaplain character, both his opportunistic side and also his chivalric side.
So these qualities created a kind of mythic aura, a mythic authority for Chaplin.
And there's one other thing that contributed to the myth.
It was that there's a historical reality to the hobo figure.
Long before Charlie Chaplin began to perform as a kind of tramp figure in the movies, they were real tramps in the society.
And the existence of a genuine population of homeless people-- they often rode the rails, and they were present actors in most large and small cities in the country.
The fact was there was a population of homeless people that the Tramp figure alluded to and reminded the audience of.
And that also helped to strengthen his mystic power because it was as if the audience was recognizing that in some sense Charlie was a kind of embodiment or a symbol of this social actuality.
So he had a kind of mythic power that communicated itself to his audience.
And it meant that every repetition of a Tramp film, which might very well go through the same motions, but put the Tramp in a slightly different environment or a slightly different job, would be pleasing to the audience, some principle of repetition and variation.
Well, finally, the paradox of Chaplin's work.
And the paradox I want you to think about tonight-- and we'll end with this point-- is that Chaplin's world is full of comedy.
It's a comedian's world, although it's more than just comic, as I've suggested.
But the irony of Chaplin's world is that the comedy is so fundamentally based on things that are so deeply unfunny.
Chaplin's the most deeply social and in some ways even political of directors.
And I'll discuss this aspect of his work tonight, when we talk about Modern Times and some of his other films.
But the crucial point here is that Chaplain's world is a world of elemental themes.
Virtually every single Chaplin movie is about hunger, aggression, victimization, confinement.
I mean the subject matter of Chaplin's comedy is the furthest from comedy.
And part of his genius, part of what makes him such a remarkable and important artist, is that this immensely unpromising material, this disturbing and in many ways frightening material, dramatized sometimes with full frightening effect-- Chaplin often, because of his diminutiveness, puts himself up against large figures, giant figures, to emphasize his vulnerability and smallness.
Again and again, the substance of Chaplin's comedy is the substance of victimization, aggression, misery, inequality, inequity.
In other words, the subject matter of Chaplin's work is the furthest from comedy.
And yet he makes enduring comedy, enduring melodrama, out of those unpromising materials.
That paradox about Chaplin's world is at the heart of his work.
And it's at the heart of why, of all silent directors, his very best films remain interesting today intrinsically, not just because they're wonderfully interesting artifacts.
But intrinsically because they remain serious, valuable films.
All of the Chaplain films you're going to see tonight, the two shorts and Modern Times, fit that description, as I hope you will agree.
See you this evening.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This lecture tonight is our last chance to pay attention to this process I've been calling the Fred Ott principal.
And forgive me for appearing slightly repetitious about this.
But to me, it's an essential aspect of our appreciation of these early weeks, to recognize that films like The General, or Modern Times, or the film we're going to see next week, The Last Laugh, Murnau's remarkable silent film from Germany, to recognize that these astonishing, complex narratives grew out of something so primitive and grew out of something so primitive in such a relatively short time, is part of what makes these early films so interesting.
And the viewing you're going to see tonight is your last opportunity to, in a concrete way, experience something of that process, something of that evolution for yourself.
Because the two Chaplain shorts that you're going to see, while they are, as I indicated this afternoon, not from the earliest stages of Chaplin's work, they're three or four years advanced beyond that.
He's already mastered his medium in certain ways.
And these shorts are, in themselves, very coherent and interesting works.
And I hope you'll look at them for their own intrinsic value.
But I also want you to recognize, as you think about those short films, remarkable as they are, how much richer, how much more complex, how much more demanding and rewarding in many ways Modern Times is.
And I think the same thing could be said about all of Chaplin's features.
There's an astonishing distance between even the best shorts or between most of the best shorts and the features.
Some of the very best shorts have a kind of elegance and purity, as well as a level of comic inventiveness that makes them in their own way almost the equal of the features.
But, of course, even the best of them don't have quite the reach or the resonance of a film like Modern Times or The Gold Rush.
One way we can perhaps clarify some of what I've been saying, and maybe also do a bit more justice to poor Buster Keaton, who I think in many ways is an equally remarkable artist and in a technical sense an even more interesting director than Chaplain himself, is to begin with a comparison of the two.
Not so much because the comparison will really illuminate weaknesses in one or the other, but the contrast between them I think will help to clarify what are some of the essential qualities in each of these director's work in films.
And I hope that you'll sort of think back to the Keaton material you saw last week as I'm making these comparisons.
One way to think about the differences between Chaplin and Keaton, and also to think about what is essential about both of them, is to talk about the way they deal with objects and the role that objects have in their films.
In Keaton's case, we might say that the basic objects of interest are usually massive and gigantic things, whole houses, locomotives, as in The General, an ocean liner as in The Navigator.
And Keaton was interested in intricate systems and in the intricacy with which these systems operated.
And he liked to pose his Buster character against these massive systems, to see whether Buster could survive them and to show us certain aspects, both of Buster's resourcefulness and power to muddle through, but also his comic inadequacy and comic failings as well.
And a number of the most dramatic and famous bits from The General could be said to crystallize this principle.
Think of that magnificent joke.
It's more than a joke.
That cosmic joke, that vision of experience, that existential mockery, but also affectionate mockery of human nature, that's embedded in the cannon sequence at the very beginning of Keaton's most important film.
You remember how that works.
It's very funny at every level.
But as Keaton first fails to fire the cannon, then the second time reloads the cannon with 10 times as much powder, then gets stuck in front of the cannon and it looks as if he's going to be shot.
But every move in that sequence is full of amusing comic business.
But think of how as the joke builds, as it keeps topping itself, something else begins to happen.
What begins to emerge, as I suggested last time, is almost a kind of vision of life, a vision of life in which human agency matters.
You do have to struggle and do what you can to try to save yourself or accomplish your ends.
But then when your ends are accomplished, even when things work out almost exactly as you had planned, remember what happen.
The cannon does fire.
It doesn't shot at poor Buster, who runs away from it and hides in the cowcatcher of the engine to avoid it.
But why doesn't it go?
Because at a certain moment, the train just happens to go around the bend and geography and physics collaborate with the Keaton character in order to create a shot that actually does almost hit the engine he's pursuing.
And it certainly persuades the people he's pursuing that they are being chased by more than one man.
And, in fact, remember when the cannon fires and we see that the outcome that results is pretty much the outcome that the Buster character intended, there's a double comedy there, isn't there?
And it's a metaphysical or an existential comedy.
Because what that joke is saying is, of course, what the whole film also says and what many other individual crescendo jokes, we might call them, also say in the film.
Which is that we get through in life by muddling through, by working hard, by engaging in all kinds of sometimes almost obsessive labor in order to accomplish our ends.
But in the end, when we do accomplish our ends, it's not entirely because of us.
It's because of accident.
And when we don't accomplish our ends, it's also because of accident often, as well as because of our human frailty.
So there really is a kind of complex understanding of the world implicit in the kinds of jokes that Keaton manages to tell us, as if there's a sort of understanding or interpretation of life embedded in the best moments of The General.
And this kind of vision of experience wouldn't really be possible if Keaton was as interested in individual encounters with small objects as Chaplain is.
Because the distinction between Keaton and Chaplain in terms of the way they use objects is that Chaplain is in love not with large, gigantic structures, but with tiny ones.
He wants to see how Charlie manipulates his cane.
He wants to watch Charlie interact with a hamburger or with a shoe that he is pretending is a turkey dinner, in a fragment from a late film of his that I'm going to show you I hope in a few moments.
So if objects in Keaton are massive and systemic in a certain way, the objects that are most characteristic of Chaplin's world are small and manageable.
And the interactions between the Chaplain character and these objects are often an occasion for the exploration of character.
When the Tramp character encounters a particular object, one of the things he is characteristically tempted to do is to transform its use, is to make it useful some other purpose.
As if in the contest between Charlie and the world, Charlie has some transcendent power to allow his optimism to transform a recalcitrant reality, to make the reality kind of bend to him.
And it becomes especially poignant, as you'll see in the passage from The Gold Rush I'm going to show you in a little while.
It becomes especially poignant when we understand how minimal are the constellations that Charlie finds.
When he transforms this shoe into a turkey dinner, he's actually starving.
And although it may psychologically help him out, we can see his resilience facing hunger and making the shoe do for a meal.
So what is expressed there is something of the character's imaginativeness, but also something of his resilience and optimism.
Because when he encounters the world, his relation to it is one of a magician or a transformer.
He's always trying to impose his imagination on the world.
And if you watch Chaplin's films-- this is true of his shorts as well as his longer films-- if you watch Chaplin's interactions with objects closely, you will see a continual drama in which the imaginative world of the Tramp is in some sense in a kind of conflict, or a kind of collision, or at least a kind of angry conversation with the world in which the objects of the world are, although they may resist, are constantly under the pressure of the transformative power of Charlie's optimism, resilience, and imagination.
So it's small objects in Chaplain that matter.
And they declare for character, rather than for some cosmic understanding of the world.
We can get at another fundamental difference and some of the strengths and the alternate kinds of strengths of both Keaton and Chaplin by talking about the different protagonists or heroes that are characteristic of their works.
In Chaplin's case, let's start with Chaplain, we often have heroes who have grand visions, or schemes, or hopes.
They tend to be chivalric characters.
So sometimes the project they take on is rescue the damsel, protect the woman.
In the first feature-length film or nearly feature-length film that Chaplin made-- it was a film called The Kid.
And it actually involves a child.
The kid of the title is a young boy, an urchin, very much like the real Chaplain in his childhood in London.
And the Tramp character, who is himself starving, encounters the kid and has to protect him.
So it's actually a story of maternal-- there's a maternal quality to Charlie in this.
He's much more like a mother than he's like a father.
But anyway, it's in a certain sense a kind of sentimental parenting fable, in which we see starving, miserable, down-at-the-heels Charley defending a person who's even more vulnerable and even weaker, even less able to take care of himself than Charlie is.
And although it is a deeply sentimental film in many ways, it also was a film that provides another occasion in which we can see the Tramp character's imaginativenes operating.
Because as I suggested this afternoon, one way you can also see it operating in chase sequences because he makes such amazing split second decisions about how to elude his pursuers.
And often those decisions reveal his intelligence and his improvisatory quickness.
So almost everything that happens in a Chaplain film returns in some sense to our sense of the Tramp's character.
And we feel that especially strongly I think in The Kid because one of the things that happen-- I think The Kid is 1921.
And one of the things that happens in The Kid is that we see Charlie exceeding himself in imaginative resourcefulness because he now has to protect a child.
And what you feel is that his compassion and his protective instincts mobilize a resourcefulness and an intelligence that you can't help but recognize and you can't help but see.
And so that in a certain sense, there's a grand scheme here, protect a child, save a child's life, or protect an orphan, or protect a young damsel in distress from bandits and outlaws, that sort of thing.
And then Chaplain himself, the character himself, often has grandiose visions of wealth or of success.
Some of his films, as in Modern Times, include dream sequences or fantasy sequences, very, very clever, intelligent sequences.
Because you can tell that they are taking place in a subjective realm, itself a kind of thematic and technical advance in the making of movies.
That Chaplin found ways to introduce images that the audience would recognize instantly as taking place, not in the real world, but in the subjective life of the characters as part of his achievement as much as a mature director.
So the Chaplain hero is often an ambitious character, although he often fails.
He has large ambitions.
He has chivalric tendencies.
He's hopeful, resilient, in many ways grandiose and sentimental.
And the Keaton character couldn't be more opposite in some ways.
He has sometimes been called wrongly the great stone face.
But he's a character who sort of muddles through without any grand schemes.
The Keaton character usually isn't trying to do anything more.
The largest thing he ever tries to do is impress a woman, as he does in Cops.
Mostly what he's just trying to do is survive, get through the day, get around the corner, keep this ship from sinking, keep this locomotive on the track.
One might even say that in The General, which is his most grandiose plan in all of Keaton's films, what the Buster character does in The General is the most ambitious and remarkable.
He's going to recapture his train.
He's chasing the Union army as if he's fighting the war on his own.
But it's important to realize how we're supposed to understand that in a deeply comic way because the Keaton character is so wedded to his engine that's really not thinking about any more marshal victory or winning the war.
What he wants to do is recover his engine.
So even there, there's a kind of smallness, a kind of lack of ambition, a lack of grandiosity to what the Keaton character is doing.
So he wants to just survive and do a task, again a very sharp contrast.
The treatment of women, a third category, might be a way of seeing them in sharp distinction to each other.
Chaplin's treatment of women is deeply sentimental and chivalric.
They're almost always, with the partial exception of Modern Times-- and I'll talk about that briefly in a moment-- they're almost always treated sentimentally and as characters who need protection, who are weak and vulnerable, as well as beautiful and stand for some kind of purity.
They are an inheritance or a vestige of the Victorian ideal of women.
They were put on pedestals because they were too pure for the world.
And they were also too weak to open doors for themselves.
They had to have men do that for them.
So Chaplain's female characters, with the partial exception, dramatically partial exception of the gamin in the film you're going to see tonight, tend to be of this sort, Victorian stereotypes.
And his treatment of them is deeply sentimental.
If you think about the treatment of the heroine in The General, you'll see how dramatically Keaton diverges from this view.
Remember, Keaton is not making fun of women in that.
He's making fun of the Victorian stereotype of women that appears in so many movies.
So when the Keaton character picks up that woman when she's hiding in the potato sack and throws over his shoulders, and throws her onto the pile as if she's another sack of potatoes, or later in the film when the Keaton heroin begins to do such stupid things that you begin to realize that she is one of the great airheads in the history of movies.
Do you remember some of the tricks she pulls?
Like she ties a willow tree across the tracks.
She tries a rope across the tracks to stop the next train from catching up with them.
Or she sweeps out-- do you remember this business-- where she sweeps out in the cabin of the engine while poor Buster is struggling to keep the engine going?
She doesn't like a little bit of soot on the floor.
Keaton makes fun of this figure.
But he's not making fun of women.
He's not making fun of actual living females.
He's making fun of the stereotype of women that appears not only in the movies, but also appears in sort of the popular conception of what women are.
In any case, we can see how deeply mocking and unsentimental Keaton's treatment of women is.
One final contrast that I think is helpful and interesting might be the contrast that we might generate if we tried to describe the visual style of each director.
Chaplin favors close shots and emphasizes character.
He aims for a totally realistic style in which you're oblivious to the camera's presence.
You're not supposed to think about the camera.
He wants to create a camera that is as transparent as possible.
You're supposed to just focus on the action.
Chaplain's idea is that his camera is a window on reality.
And he doesn't want the window or the window pane to be part of your understanding of what you're watching.
There's something social and character-oriented in Chaplin's visual style and especially in his preference for closeups which reveal the psychology of character.
The contrast with Keaton is very dramatic.
Keaton is a more interesting director in a technical sense.
He'll mix his shots much more often.
He'll use closeups.
But he also loves long shots and middle shots, depending on the effect he's looking for.
If you think about what a master he is of the long shot, think of the moment, for example, in The General where the camera backs up far enough so that we can see Buster working furiously on the car that carries the wood, the wood carrier on his train, working furiously with his head down, while we see the train move across the battle line.
So that he ends up in enemy territory and he doesn't even know it.
So there's a kind of cosmic joke.
And again, it's a joke that's only possible in the movies.
I mean it's a joke that partly depends upon the camera's position, its power to photograph motion, and the fact that it can encompass a much larger mise-en-scene that is ever possible in a live theater.
So it's a technically interesting achievement.
But it's also a morally interesting achievement because what does it dramatize?
The usual trick, the usual thing we know about the Buster character.
He's mostly oblivious to the dangers around him.
And often the comedy in Keaton comes from the fact that he's oblivious to his narrow escapes, as we've said before.
So Keaton uses mixed shots.
He's much more interested in creating an aura of authenticity in his filming.
The General especially alludes to documentary photography made during the Civil War.
And some of the images are actually recreations of Matthew Brady photographs, very famous Matthew Brady photographs.
It was very important for Keaton to create these realistic effects, this sense of realism, because so many of his most magnificent jokes depend on your recognition that it's actually happening.
That he's not using process shots.
Think of, for example, of the unbelievable engineering feat that was involved in the final crash of the train, when the bridge burns at the end of The General.
Remember, there's no digital animation then.
When Keaton did that, if he didn't get on the camera the first time, that was it.
And, in fact, it's incredibly beautifully done, It happens perfectly.
There's something almost graceful about that catastrophic sequence.
It's so well done, in fact, it even holds up today in an era when we are so used to special effects of a much more dramatic and astonishing kind.
There are no special effects in that sequence.
It depended on a mastery of knowing where to put the camera; a kind of engineer's mastery of various individuals and machines that were in motion at the same time; perfect timing, so that the bridge had to actually burn down at just the right moment for the train to come over it.
There was no margin for error there.
Again and again, Keaton's comedy shows us this level of technical, as well as intellectual or thematic mastery.
Because, of course, that moment when the train crashes into the canyon isn't just a magnificent visual moment and a magnificently comic thwarting of the expectations of the generals who are running the thing.
But that moment is also completely consistent with the mock heroic interests, the mock heroic themes of the movie as a whole.
And what I'd like to do now is concretize and embody the arguments I've been making very briefly and inadequately by having you look at three scenes, one from Keaton, two from Chaplain, that can help ground what I've been saying in particular images, in particular moments.
The first scene is a scene that I wanted to show you last week and didn't have time.
I think of it in a certain sense as a particularly clear embodiment of what we might call the Keaton vision.
And it's a sequence that violates one of Keaton's deepest principles at the very end.
And I'm a little sorry it does.
And the sequence is harmed by that.
There is a special effect in it.
And it's too bad that there is.
But it's one of the very rare such moments in Keaton.
Keaton tried never to do that.
And as I mentioned to you, sometimes very dangerous things happen to Keaton.
And they happen in reality.
And he was often begged, when he became a director of features, to not do his own stunts.
But he always refused.
And he always did his own stunts.
He broke his neck when he was making The General.
And any continued filming with a broken neck for part of the time.
He didn't quite realize how seriously he was injured when that happened.
If you think back to The General, you might actually be able to figure out-- it happens relatively early in the film when he hurt himself.
When he's on the hand car, when he's chasing on the hand car and he falls off the hand car.
And he actually did real harm to himself in that sequence.
So this first sequence, it comes from Cops.
And in many ways, I think it embodies that kind of aversion of the trajectory or crescendo joke I was talking about earlier.
It's a small version of the cannon joke that you see in-- or jokes plural, because there's another joke with the cannon in the second half of the film-- that I've talked about earlier.
Let's show it, Greg.
And, of course, this is the moment where Keaton is in flight from the police.
The cops are chasing him.
He runs up this teeter-totter.
He runs up a ladder.
It turns out to be like a seesaw.
And there we see the Buster character trying to elude cops who come up on either side of the teeter-totter.
You see his acrobatic qualities.
But part of what makes it good is that we know he's improvising here.
He's in trouble.
People on both ends now are after him.
How can you escape this?
Teeter-totter up and down, it seems as if there's no escape.
It's almost as if every cop in the city is now after poor Buster, massive forces arrayed against the lone hero, a key to his comedy.
And, of course, that's the process shot.
But it is a very witty way to end.
I mean what does it depend on?
Among other things, an engineer's understanding of the laws of motion.
Many of Keaton's jokes depend on a kind of collaboration with gravity and call attention to that.
So it seems as if there's no escape.
And he does not escape because of anything that he himself has really done.
It's contingent.
It's accidental.
But he does escape, right.
It's as if the universe sometimes , not always, conspires to help us.
If we try hard, sometimes it'll help us.
Sometimes it won't.
We're mostly foolish characters muddling through.
The jokes say that again and again.
It's a vision of life.
Now, I want to contrast this kind of joking and this kind of style.
Again, for that effect to work, the camera had to be pretty far back.
It depended on your seeing the seesaw going up and down.
It depended on your seeing the cops on either side of the ladder.
So in that sense, it's a characteristic Keaton moment, a characteristic Keaton passage.
I want to show you two passages from feature films of Chaplin's that I think are those characteristic moments of Chaplin.
And you'll see how they fit into what I've already said.
First, a scene from The Gold Rush.
And let me set it up for you.
One of the delightful and important things about the Tramp's character, as I mentioned this afternoon, is after a while-- Chaplain made a total of 81 films.
The vast majority of them were silent films.
After the first couple of films that he made, the Tramp character began to elaborate itself and proceed by a principle of accretion.
That's very important for us to be aware of.
Because think of what this means.
If the audience for Modern Times had seen 50 Chaplin shorts or most of Chaplin's movies-- and this would have been true.
In 1936, the vast majority of the movie audience would have lived through the silent era and certainly would have known a great many Chaplin films.
So when they came to see Modern Times, they had in their memory banks all these other stories about the Tramp.
And, in fact, they didn't have to wait for Modern Times for this effect to occur.
As the Tramp begin to generate a kind of reputation and more and more people came to watch him, what began to happen was that each subsequent adventure of the Tramp got richer, not because of anything inherent in the new adventure, but because the previous adventures lay behind it and were part of the Tramp's ongoing identity.
So as the audience became more familiar with the Tramp and as Chaplin's gifts for dramatizing character and for making movies enlarged, so did the resonance of the Tramp as a figure.
And I should have said this afternoon because it's the deepest explanation for why the Tramp became a mythic figure, why the Tramp became such a memorable and powerful icon.
And in this sequence, you're going to see one of the classic instances of the Tramp's resilient imaginativeness in the face of difficulty and danger.
In The Gold Rush, the identity that Charlie takes on-- by the time he comes to make his feature films, there are dozens and dozens, 30, 40 short films in the past.
And all of those identities, all of those adventures, feed into and inform the more recent ones.
When the audience then comes to watch Modern Times and The Gold Rush, it has that history behind it.
In this particular film, Charlie takes on the identity of a '49er, not a football player but a miner, who's gone to Alaska for the Alaska gold rush.
We've seen him as a waiter.
We've seen him as a ship builder.
We've seen him as a pawn broker's assistant.
We've seen him in a whole range of other jobs.
And until Modern Times, each film, even the feature-length films, only show Charlie in one occupation.
It's only in Modern Times where we see him in multiple occupations.
It's one of the things that marks Modern Time's greater complexity from the earlier film.
So what's happened, he goes to Alaska.
He finds himself stuck in a cabin after a terrible avalanche and snowstorm.
He's stuck in the cabin without food, with a massive, gigantic fellow, one of his favorite antagonists.
Mack Swain, there's a name.
And Chaplain certainly chose him because he was so humongously large that when he was juxtaposed against Charlie, he looked even more menacing.
Well, in the sequence you're about to see Charlie is confined in a cabin, in a horrible snowstorm, starving, with this gigantic fellow, who's beginning to look at Charlie as if he might be a tasty morsel.
And it's Thanksgiving.
It's Thanksgiving.
They're about to have Thanksgiving dinner.
What are they going to do?
Watch Charlie's imaginative response.
OK, Greg.
[MUSIC PLAYING] Even before this, we've seen the Mack Swain character look at Charlie with a kind of hungry eye.
And you'll see him do it again here.
Watch how Charlie interacts with particular objects, with small objects.
What he's done is he's cooked a boot, one of their shoes.
The shoelaces become spaghetti.
Did you see the subtlety of that moment where Charlie look frightened?
Watch the range of emotions that he's able to express in his face.
Now, you see how he uses closeups, how it's a drama of character.
You see now, he lacks Charlie's imaginativeness.
So he doesn't survive as well.
It's almost like mind over matter.
It's that quality in the Chaplain character, I think, more than any other, that explains why people loved him.
You see, if you act like you're eating good food, you almost are.
All right.
That's enough Greg.
You get the idea.
It's a remarkable film.
And you get an idea.
Now, the next sequence I want to show you is in some sense even more powerful.
It's less comic, although there are comic elements in it.
It's Chaplain at his most dramatic.
And it's the final sequence of the film City Lights, the film that precedes Modern Times.
Five years separate them.
But it's the last film he made before Modern Times.
And it's essentially an urban drama, in which the Tramp character finds himself protecting a vulnerable woman.
And the woman he's protecting is a blind flower girl.
She's blind.
And so she's even more vulnerable and deserving of protection than a sighted flower girl would be.
So Charlie sort of takes her under his wing.
And because she's blind, she doesn't realize-- there's a kind of trick in the beginning, a visual trick, that makes the blind flower girl believe that Charlie is a wealthy man.
And Charlie allows her to believe this.
He's the Tramp.
But he allows her to believe that he's wealthy.
And he visits with her.
And she has an aged mother, something like the character you see in The Immigrant, who dies in the course of the film.
And the second subplot in the film has Charlie befriending a very wealthy man, who is incredibly nice to him when he's drunk and kicks him out of his life when he's sober.
And his irrationality is part of Chaplin's critique of capitalism, I think, because he's a rich man.
But when he's drunk, he loves Charlie.
And he gives Charlie money and so forth.
But Charley uses this to get a fortune.
And he sees an article in a newspaper about a Swiss doctor who can fix blindness.
And he sends the flower girl to Switzerland or someplace.
And she has the operation.
She has her sight regained.
And the Tramp character has had his usual misadventures.
And he's just getting out of prison at the very end of the film.
So the encounter he's going to have with the now sighted flower girl, which is the very climax of City Lights-- this encounter that he's going to have with her is an encounter in which he and the audience know perfectly well who all the characters are.
But she has never set eyes on Charlie before.
And she has the idea that her benefactor, the man who sent her for the operation and restored her sight, is a wealthy and handsome fellow.
Here is the ending of City Lights.
[MUSIC PLAYING] So for the audience, this moment has tremendous poignance because the audience is aware of things that the female character is not aware of it.
That there's her benefactor, this shabby little tramp.
So her condescension to him in this sequence has tremendous poignance for the audience.
And look at the closeup on Charley here, Charley sees who it is.
Charlie knows who it is.
The only thing that's wrong with this ending is the titles are unnecessary.
The faces are so eloquent, they're unnecessary.
The words aren't needed.
In fact, the words even simplify.
Can you see how Charley's expression involves pride, as well as affection and a kind of love.
He's incredibly excited and happy to see her and to see her sighted.
See the power of a closeup.
She suddenly realizes who it is from touch.
The least necessary intertitle in the history of movies is about to appear.
But look at the closeups here.
Look at the art of these closeups.
Another unnecessarily line.
There are people who have said that that last closeup is the most eloquent closeup in the history of movies.
It might be true.
Because the complexity of emotion that's playing across Charlie's face there-- it's hard for people who haven't seen the whole film to fully grasp it, although I tried to set it up.
It's partly that he knows there can never be anything between them.
It's partly that he's full of joy over the fact that he's been able to help her.
But there's also a sense of their inevitable separation, of the differences between them.
And that's part of what's reflected in that closeup at the end.
And another thing I would call your attention to, part of Chaplin's subtly, is look how he doesn't hold it too long.
A truly sentimental director would have held his final shot, Charley's face, twice as long, three times as long.
But do you notice how wasn't there for very long?
His hand was over here and then the screen goes blank.
He doesn't milk the moment, even though it's an immensely complicated moment.
But the most important thing to say about this scene is that it's a scene that's played out by the drama of the face.
It shows how Chaplin had reached a point by the time he reached his maturity in the 1920s, had reached a point where he could create films in which closeups on expressive actors' faces, and especially closeups on the Tramp's face, could reveal worlds of meaning.
The range of contradictory feelings and meanings that run across the theater of Charley's face in these final sequences of City Lights are proof of the eloquence of the silent film and are a particularly distinctive signature instance of Chaplin's art.
Well, in the time I have remaining, what I'd like to do is talk very quickly about Modern Times itself, the great masterpiece that you're going to be seeing in a few minutes, as a way of sort of setting up some of the things for you to watch for.
But I feel badly that I couldn't do this fully enough with The General, which is an equally complex film.
And I only have a few minutes here.
So I can hardly do full justice to the astonishing richness and complexity of Modern Times.
But I want to call your attention to certain features in it and count on your being as attentive and generous in your viewing as you possibly can.
I'm sure you'll pick up other things as well.
First, the context of the film.
I've already mentioned this.
So I don't have to spend a lot of time on it.
But it's very interesting and significant that the film was made in 1936, released in 1936, something like seven or eight years into the sound era.
There hadn't been any silent films made.
Now, I'm not saying that Modern Times is actually a truly silent film.
It's not.
It has a soundtrack.
And the soundtrack is complex and rich.
And I'll say at least a word about it before I'm finished here.
But in every fundamental respect, Modern Times wants its viewers to think of it as a silent film.
It invokes the tradition of silent film and especially the tradition of Chaplin films, to which it systematically alludes in virtually every frame.
So the context for the film is first is that it is in a certain way a kind of silent film.
There is no synchronous dialog in it.
There are sound effects-- and, of course, there's music-- a soundtrack the Chaplain arranged very complexly.
And it's a very rich soundtrack.
But there's no synchronous dialogue until the very end.
At the very end, there is synchronous dialogue.
And it's Charley who speaks it.
In fact, the film was advertised as follows in many newspapers.
It was, "The Tramp speaks."
Well, watch for what happens when the Tramp speaks.
It's one of the film's great and serious jokes.
He does speak at the end.
And it's the only moment.
You do hear dialogue elsewhere in the film.
I'm getting ahead of myself.
This is what I wanted to say about sound.
But you don't see synchronous dialogue.
You don't see people moving their lips and words coming out of there.
You see people on a screen talking or you hear people on a radio talking.
But you don't see someone on screen actually moving his lips and words, sound coming out, until the very end, when Charlie does his song.
He sings a song.
He plays a waiter near the end of the film.
And he sings a song.
And he tells a story with the song.
Watch for what happens.
So the Tramp does talk.
But it's a wonderful comic joke.
Another aspect of the context of Modern Times is it takes place during the Depression.
But the subject matter of Modern Times was Chaplin's typical subject matter.
It was about the Depression.
It was about hunger, about misery, about someone who can't find a job, about two homeless people, about the conflict between capital and labor, about the irrationality of both sides.
It was literally about the turmoil and trouble that was going on during the Great Depression.
Many people predicted, not only that the film would fail because it was really essentially a silent film in an era when no one was interested in silent film anymore, but also that it would fail because it was rubbing the audience's noses in the very experiences they were trying to escape when they came to the movies, the escapist theory of why people attend films.
Of course, both predictions were mistaken.
Modern Times was a significant commercial success, partly because Chaplin was very clever in the way he did his exhibition of the film.
He released it first, not in mass release, but to elite feeders, which charged quite a great deal of money, and then slowly allowed the film to reach larger and larger theaters.
He anticipated certain modern marketing methods with the release of Modern Times.
And, of course, by 1936, Charlie Chaplin was an internationally famous figure, of such stature.
He was about the only figure, maybe apart from D.W. Griffith, about whom people had begun to write essays calling him an artist.
So Chaplain occupied a unique place.
And that was also part of why Modern Times was waited for so fully.
So the context of Modern Times is important.
There's something brave and surprising about the fact that it was about the miseries of the Depression during the Depression, that it was essentially a silent film in the middle of the sound era.
Both very bold decisions on Chaplin's part, both characteristic of his ambitiousness as an artist.
Like Keaton's The General, we could certainly call this film also a culminating text, a summarizing text, in several senses.
It's a culminating text in terms of Chaplin's work.
And I've already talked about that.
Every single person coming to see Modern Times, who had any experience with the movies before, would recognize in Modern Times situations and even physical spaces that had occurred in earlier Chaplin films.
So there was almost a sense in which Modern Times was a reprise of the whole career of the Tramp.
And I mentioned that Modern Times shows us the Tramp for the first time in multiple jobs Having multiple sources of-- and, in fact, multiple misadventures.
He goes to jail more than once in the film.
We find him in a factory more than once in the film and so forth.
So there's a sense in which the various separate episodes of Modern Times, taken together, dramatize, in small, the whole of the Tramp's career.
And there are many specific bits of business and scenes in Modern Times that are little repetitions, or echoes, or reprises of earlier Chaplin films, which have caused a shock of recognition in the audience, a happy kind of recognition.
There are many examples of that.
There are many earlier films which deal with restaurants.
You'll see some of them from the shorts that you've looked at, certain echoes.
I won't talk about any of them beyond mentioning that there's a remarkable sequence in the film-- if we have time, I'll show it to you.
But we probably won't have time-- in which we see Charlie on skates.
And that scene alludes to a whole short that Chaplain had made, in which he plays a waiter who has to serve people on skates.
And you'll see that he has to do that again at the end of the film.
He sort of reprises that role.
And then there's an earlier moment in the film where we see Chaplin states as well.
But the main point is that virtually every moment in Modern Times has some counterpart in an earlier Chaplain short.
So there would have been a sense of the audience coming back to Charlie, returning to the world that Charlie stood for, when they came to watch Modern Times.
So it's a culmination of Chaplin's career.
But it's also in a deep sense, like The General, a culmination of silent film.
And it would have had that nostalgic affect.
It would have had that nostalgic effect on the original audience.
Most of the original audience would have recognized the film as an embodiment of a phase of movie going that had passed.
And we should watch it in that way as well.
I want to say a word about the Gamin.
That's the character, the female character in the film because she represents an advance over any previous Chaplin movie.
As I suggested earlier, in most Chaplin films, women are relatively stereotyped and don't play very active roles.
This is the first film in which there's a woman who seems to be in some sense the Tramp's equal, in many ways more than his equal.
She has more energy.
She's a bit younger.
You might watch for the ambiguity in their relationship.
Is it a sexual relationship, a romantic relation?
Probably not.
Charlie's a lot older than the Gamin.
In real life, he was having an affair with her, I might mention to you.
In fact, he was famous for having relations with young women, especially his young actresses.
And there were scandals connected to Chaplin's name partly because of this.
And Paulette Goddard, the actress who plays the Gamin in this film, entered on a career of great fame in the movies, partly because of this role.
And you could see how expressive and remarkable her face is.
This is the first female character who has energy, will, who has a kind of aggressive capacity to be resilient and care for herself.
And that represents a tremendous advance in Chaplin's work.
She's the first sort of semi-independent female in Chaplin's work.
And you can see in a certain way, she's a kind of partner for Chaplin.
When this film ends with Charlie and the Gamin walking down the road toward an unknown future, the only difference between that ending and the ending we normally see in a Chaplain short is that Chaplain is usually alone when he walks down the road toward the mountains in the distance.
And in this film, he is has partner.
So there's a slight suggestion of, if not exactly hopefulness, at least the resilience, of the capacity to survive.
You have someone to help you.
And you might watch the way the Paulette Goddard character is treated in the film.
Because she actually represents something new in Chaplin's work, a real advance in his understanding of how to deal with female characters.
A new kind of respect for women show up in this film.
I've mentioned the soundtrack before.
But I want to quickly call your attention to two features of it.
One is that Chaplin arranged the sound track very carefully.
And what you'll notice if you pay attention is that the soundtrack has what we might call a quality in which particular themes recur when certain characters appear on screen.
And you begin to associate certain melodies with certain characters.
And you should watch the way this unfolds.
I mean the melody that's played behind the Chaplain character was actually a popular song of the day, entitled, although the words are never sung in the film, "Hallelujah, I'm a Tramp."
And it was a song about being free, and not having to worry about working, and riding the rails, and so forth.
Hurray, I don't have a job.
Hurray I'm a free, in those ways.
But each of the characters, each of the major characters in the film, has a musical theme associated with him or her.
And you should watch the way they sort of meld in and blend together.
Chaplain took the sound in the film very seriously.
I've mentioned the talking theme.
Watch the mocking way in which talk itself, speech, is dealt with in the soundtrack.
You might want to talk about this in your recitations.
But ask yourself what's going on there?
Why would Chaplain, in the end, denigrate talk?
I think the answer is obvious.
The Tramp is a silent character.
It's as if this is the revenge of silent film on talkies.
It's as if the film is saying talk is crap, talk is silly, talk is ridiculous.
Who needs talk?
Watch how the film repeatedly makes fun of talking or shows talking as inadequate or unnecessary.
And also the way the film associates talkies with something else that the film is very hostile to, which is mechanized industrialization.
The film is a critique of the excesses of capitalism and the excesses especially, the tedium of factory work.
And talk and talking films are associated with what the film identifies as evil or dangerous, as you'll see.
But there's also a moment in the film that-- I won't show it to you.
But I'll mention it and ask you to watch for it-- where we can see another way.
It's only one of the ways in which Chaplin manipulates sound.
And it's an interesting moment because it's easy to miss, easy not to register what's happening there.
There's a moment in the center of the film where Charlie and the Gamin escape their difficulties briefly.
Charlie gets a job as a night watchman in a department store.
And the department store is like capitalist heaven.
They've been living hand to mouth, nearly starving.
And they find themselves in the department store.
There's food.
There are beds.
There are beautiful warm clothes.
There are toys.
There are even roller skates.
And there's a moment when Charlie and the Gamin exploring the store, having this exciting experience with all these goods, which they are deprived of.
It's not an accident that that scene is at the very center of the film.
Almost all the scenes on either side of the department store scene rhyme with each in certain ways, as an aspect of the structure of the film.
And I'll come to that in second.
I'll conclude with that point.
But the department store scene is the only one that doesn't have any counterpart.
It's right at the very center of the film.
And what one can feel is the way in which that dramatizes the injustice, the sense of haves and have nots that is at the center of the film's meaning.
The film is always in some sense interested in that topic.
So there's a moment when the Gamin and Charlie find themselves in this store.
Charlie, in his exuberance and high spirits, straps on roller skates and begins skating around.
And if you listen to the soundtrack, you'll see the soundtrack is very beautiful and gentle.
And you know, dah.
It's very calming and exciting.
And as Charlie is skating, the camera shows that, in fact, he's skating in a very dangerous area.
The store is partly under construction.
And on one point, while he's skating, he puts a blindfold on.
And he's still skating very calmly, and beautifully, and gracefully.
And the soundtrack under it is very graceful and appropriate to his feelings.
But what the camera shows, what you see, is him coming near the edge of an abyss.
Actually, it's a platform.
He might fall three stories down into the lower story.
And he's not aware of it because he's blindfolded.
And the sound-- and then, of course, there's a certain moment where he takes the blindfold off and he sees that he's about to fall.
And he gets nervous.
And he suddenly is no longer so graceful.
It's kind of a minor joke.
But what's really interesting about the moment is to think about what Chaplain is doing with the soundtrack.
Because when you reflect on it, what you realize is that what Chaplain has done with that soundtrack is have it reflect not the external reality that we're all watching, but the inner feelings that Charlie is feeling.
What he's done is he's discovered a way-- it's a simple point, but it's a brilliant discovery.
He's discovered a way to use the soundtrack to express a subjective state.
And what he's also discovered, of course, is that-- although other people had discovered this principle too.
But Chaplin's using it with great intelligence.
He's discovered that the soundtrack and the visual track don't have to absolutely coincide.
That they can report different emotions or cause different kinds of reaction.
And that what you get when you have this disjunction is something much more complex.
We see Charlie is skating in danger.
But his obliviousness to the danger is partly registered by the music, which tells us not about what's happening outwardly, but what's happening inside Charlie.
And there are other moments like this in the film, not necessarily with the soundtrack, but in other ways, in which the subjective states of the characters are explored.
Pay attention to those because they're very remarkable and interesting moments, in which Chaplain is expanding the repertoire of film.
The structure of the film is especially interesting.
And I won't say more about it, than to say if you watch closely what you will see is the structure is a profound structure of repetition.
As in The General, the film's first half is revisited in the film's second half.
It's as if the film is based on a series of rhyming scenes.
In the first half of the film, there's a fantasy sequence in which we see Charlie and the Gamin inside a kind of cabined environment, inside a kind of domestic scene.
It's actually a fantasy scene, in which Charlie imagines a suburban heaven for him and the Gamin.
In the second half of the film, there's an actual cabin in the mud flats that they live in.
And those two scenes are sort of in conversation with each other.
There are two extended sequences in factories and involving the malfunctioning of machinery.
And what you'll notice, if you pay close attention to the film, is that this principle of rhyming scenes creates a situation in which in the second half of the film you partly feel that you're revisiting scenes and experiences that you've had before.
And one of the reasons that this is a useful and an important idea is that if you think about Charlie's life and you think about the previous history of Chaplin's career as a director and as an actor, what you realize is that in a deep sense what Modern Times is doing is replicating the history of Charlie's career.
Because Charlie has held a series of jobs.
Charlie has been in 15, or 20, or 40, or a hundred different environments.
And in virtually every case or in every case, the ending has been the same.
He has been released into an ambiguous kind of homelessness and freedom at the end of the film.
There's something footloose, homeless, and unfinished about Charlie's life.
And after a while, you come to realize you this would have been true by 1920, if you had paid attention to the 30 or so other Chaplin shorts that already existed.
The fact is that it's an endless process.
So another thing that the structure of Modern Times manages to dramatize, maybe more deeply and powerfully than any prior Chaplin film has been able to do exactly because of this mirror structure, is that this principle of repetition never ends, that Charlie is on a kind of treadmill.
And, of course, Charlie revisits scenes and repeats actions that we'd seen earlier in the film because his life is a series of repetitions.
Because the film's vision of social life for a character like Charlie is one that's so mordant.
What he's suggesting is things are hard, things are tough.
This man's homelessness, this man's hunger, will never abate.
He's going to have a struggle for his whole life.
But we don't despair because the Chaplain character himself doesn't despair.
So pay attention to the way the structure embodies the meaning.
The point I'm making now, and I'll come back to this later in the course, the point I'm making now is a way of talking about what could be called the film's commitment to the principle of organic form.
When the structure of a text is organic, what is meant by that is that the structure itself embodies meaning.
That the way the text is organized carries the themes of the text.
And if you think about the structure of Modern Times, you'll see that it's a marvelously distilled instance, a marvelously clear instance, of the principle of organic form.
What happens to Charlie serially is more meaningful in a way than what happens to him in a single episode.
And the fact that we have this sense that things are repeating, that Charlie is on a treadmill, that he lives a cyclical life, that he'll always have moments of hope and hopelessness, that he'll have momentary times when he finds a place to live or find sufficient food.
But that it will always be temporary.
That his life is always on the road.
That his life is always in process.
And that, at the same time, he never lets himself completely despair.
There's always a moment where his resilience reasserts itself.
Where those elements are embedded in some sense in the very structure, the very organization of the film.
And I urge you to watch it.
What's been embedded or implied in a good deal of what I've been saying tonight also has to do broadly with what might be called as the director Chaplin's complexity.
Almost every moment in Chaplin, and certainly almost every moment in Modern Times, has the kind of multiplicity, or density, or texture that I've been describing to you as the mark of a serious film.
The mark of what I want to call a work of art.
And if you pay attention, one of the things you'll find is that even the moments that seem on the surface to be the simplest ones, are not.
Or even the implications in the film that seem the simplest are not.
Just one example, the film's critique of capitalism, of a certain kind of excessive form of industrial capitalism, is obvious and very deep.
I mean the film is deeply hostile to the dehumanization that work on the assembly line generates for people.
And Chaplain works wonderfully comic variations on this idea.
And yet if you think of the film in a simple way, if you think in the film in a simple way as a kind of Marxist or left-wing screed, you're really mistaken.
And some people did, in fact.
And Chaplain was hounded out of the United States later in his career, in part because he was associated with what we thought to be left-wing causes.
But Chaplin's socialism was of a particularly sentimental variety.
And he was certainly no systematic communist.
But the most important thing about Chaplain is his complexity.
Because if you look closely at the film, one of things you'll find that it isn't just capital that's criticized.
It's also labor.
There's a wonderful moment in the film when Charlie finally, after times of difficulty, reads in the newspaper that the factory is hiring again.
And he's very excited.
And he leaves the Gamin.
They'd been living in their little sort of idyllic cabin in the mud flats in the second half of the film.
And he runs out to the factory.
And he gets hired.
And just as he's about to start his work, the workers go on strike and his life is destroyed again.
And, in fact, the truth of the matter is that Chaplain is an equal opportunity insulter.
He thinks that labor is just as stupid.
Organized labor is just as foolish, just as misguided as capital.
Something of the complexity of his social vision is reflected there.
And something of the complexity of his moral and psychological vision is reflected in the way the Tramp interacts with the Gamin.
And in the way in which both the Tramp and the Gamin together, at different moments in the film, try to buck each other up.
So we see that there are moments in which each character falls into a kind of despair in the face of the difficulties that they encounter and in which they are helped by their partner.
So there's a kind of complexity or maturity in Chaplin's way of understanding even the social difficulty, the social evils, the social problems that he recurrently dramatizes in his film.
For those of you who have never seen a Chaplin film before, and especially those of you who have never see Modern Times, let me conclude by saying I'm a little bit jealous of the opportunity you have to see this film for the first time.
It's one of the very few silent films I think-- I think The General is also one-- that actually stands on its own.
That's worth watching, even without any arguments about its artifactual value.
I wish you the joy of this remarkable film.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This afternoon, by welcoming our virtual audience, the audience that's looking at this lecture on MIT'S OpenCourseWare, some of you attentive viewers may notice what the students here would not notice-- that seven years have elapsed.
There's no podium-- some of you may have gotten that-- and a much older professor.
I hope that our completion of these lectures seven years later will not result in a reduced or less energetic performance.
I'll do my best.
We come now to the end of our first segment in the course on silent film.
And I thought it would be helpful to use today's lecture in part to create some perspectives on both the silent film, the idea of the silent film-- not just the particular films we've looked at, but more generally the phenomenon of silent film, the whole phenomenon-- and some perspectives that will also help us look forward to what will follow, to the sound films that will follow this week.
I'd like in a certain way to do this by complicating an idea I've already suggested to you about the notion of the film as a cultural form.
What does it actually mean to say that a film is a cultural form?
What, in a concrete sense, does this phrase signify?
Well, one answer I think I can offer by drawing on your own experience.
My guess is that all of you have watched older films, films from 20 or 30 or 40 years ago, and immediately been struck as soon as you began to watch the film by certain kinds of differences that the original filmmakers would have been oblivious to.
And I'm talking about things like the hairdos of people, the clothing that they wear, the way automobiles look, or even a world in which there are no automobiles, the physical environment that is shown.
One of the things that this reminds us of is that always, even the most surreal and imaginative and science-fictiony films, always inevitably in some deep way, in some essential ways, reflect the society from which they come.
They may reflect more than that, and they may be influenced by other factors as well, but they are expressions of the culture that gave rise to them in certain really essential ways.
And one of the things this means, among other significance, one of the most interesting aspects of this recognition is the fact that films get richer over time.
They become artifacts of immense anthropological interest, even if they're terrible films, because they show us what the world of 50 or 25 or 30 years ago actually looked like and how people walked and how people combed their hair and what kind of makeup they wore, all of the things, many of the things, which in many respects, the people making the original film would simply have taken for granted as part of the reality they were trying to dramatize.
So one way of thinking about film as a cultural form is to recognize that as films grow older, they create meaning.
They become more interesting.
They become richer, and a corollary implication of this idea that films become richer is that the meaning of any individual artifact, cultural artifact, especially cultural artifacts as complex as films, is always in process.
But the meaning is never fully fixed or finished, that new significance and new meanings emerge from these texts with the passage of time, as if the texts themselves undergo a kind of transformation.
One final point about this, just to sort of tweak your broader understanding of these kinds of questions-- one of the kinds of transitions that occurs with particular artifacts is they sometimes move or make a kind of transition from being recognized as merely ordinary and uninteresting parts of the society from which they grow, from which they emerge, simply ordinary routine aspects of the experience of society.
Later ages may value these routine objects as profoundly valuable works of art.
And in a certain sense, one could say that the film in the United States underwent a transition of that kind, that at a certain point in the history of our understanding of movies, American culture began to recognize that movies were actually works of art, that they deserved comparison with novels and plays and poems and so forth, probably an idea that all of you folks take for granted.
Many of members of your generation admire movie directors more than they do novelists and poets-- a radical mistake it seems to me, but that's my literary bias showing through.
I certainly admire great directors certainly as much as I do good novelists.
But the fact is that this is really not the case.
This recognition of the film as an artistic object, as I've suggested earlier in the course, is not some fixed or stable identity that the film has had from the beginning.
It's an identity that the film has garnered, that has been laid on the film later as cultural changes have occurred and as other forms of expression have emerged that have put the film in a kind of different position hierarchically from other kinds of imaginative expressions.
And as I've already suggested many times in this course, we'll come back to this principle, because it's such a central historical fact about the nature, the content of American movies especially.
It's the advent of television that is partly responsible for the transformation, although it takes some time for the transformation in American attitudes toward what movies are, because television became the throwaway item, the routine item, the thing Americans experienced every day.
And the consequence of that was to change our understanding of what the film was.
Now of course, the Europeans had an insight like this long before the Americans did, and that's something I'll talk about a bit later today and also at other times in our course.
So that's one way of thinking about what it means to say that a film is a cultural form.
It means that it's unstable in the sense that its meanings are not fixed, and the way in which a culture categorizes and understands a particular artifact is also something that's unstable, that undergoes change over time.
But there are other ways to think about this problem of film as a cultural formation, as an expression of society, and I want to tease out some of those meanings for you as well.
One way to come at this problem is to think of a kind of tension or even contention between our recognition that film is a global form-- that is to say that because the movies are watched across national boundaries, movies that are made in the United States can influence movies that are made in Europe and vice versa.
So in one sense, the film, especially after film got going within the first 10 years of its life, it had become an international phenomenon, and American films were watched in Europe, and European films influenced American directors, even at very early stages so that we begin to get certain kinds of films that certainly appealed across national boundaries.
And so there is a kind of global dimension to what film might be.
And there's another way of thinking about what it means to talk about film as a global phenomenon, not as a merely national expression.
And that has to do specifically with the way in which particular directors and films in particular societies can influence world cinema.
And from the very earliest days of cinema, as I suggested, this has been a reality.
As David Cook's History of Narrative Film informs you, and I hope you'll read the assigned chapters on Russian film closely, because I can only skim these topics in my lecture.
What you'll discover among other things is that the great American director, DW Griffith, had a profound impact on Russian films and that, in fact, at a certain point in the history of Russian films, there was a workshop run by a man named Kuleshov, who actually took DW Griffith's movies and disassembled them shot by shot and studied the editing rhythms in his workshop.
This had a profound impact not only on Russian cinema, but Griffith's practices had a profound impact on virtually all filmmakers.
And there's a kind of reverse influence, because certain Russian directors, Eisenstein especially, but also Dziga Vertov, their work had a profound impact on the films from Western Europe and from the United States.
So it's a two-way process.
It's too simple to say that particular films are only an expression of French culture or only an expression of Russian culture or only an expression of American culture.
They are also global phenomena, and they were global phenomena from almost the earliest stages.
So it's important to recognize this tension or this balance.
There are dimensions of film that reach across national boundaries.
And as we've already suggested, one of the explanations for the success of American movies in the United States was in part a function of the fact that they did not require language in nearly the same degree.
They were visual experiences, and an immigrant population coming into the large cities of the United States at the turn of the century was one of the primary factors that helps to explain the phenomenal quick growth of the movies from a novelty into a profound embedded cultural experience.
So it is a global phenomenon in a certain way and reaches across national boundaries.
But there's also-- and we need to acknowledge this side of the equation too-- there's also a profound, a really deep fundamental sense in which films, at least until very recently, are an expression of the individual national cultures from which they come.
I say until very recently, because some of you must be aware of the fact that a new kind of film is being made now by which I mean a film that seems to appeal across all national boundaries, that doesn't seem to have a decisive national identity.
At least some films like that.
I think the Bollywood people are making films like this.
Americans are certainly making films like this now.
And sometimes if you think of some of the action adventure films that will have a cast that is drawn from different cultures, a sort of multiethnic and multilingual cast, all of them dubbed into whatever language the film is being exhibited in, you'll see an example.
What's begun to emerge now in our 21st century world is a kind of movie that already conceives of itself as belonging to a kind of global culture.
So far I'm not sure these movies have as much artistic interest as one would like, but it's a new phenomenon, and the globalizing tendencies of digital technology are certainly encouraging new ways to think about the origins or the central sources of movies.
But until very recently, it is still the case that virtually every film made in any society reflected in deep and fundamental ways aspects of that society.
And one of the reasons that this is such an important thing to recognize is it means that, especially in cultures like the European societies and those in the United States, the movies are profoundly illuminating source of cultural and social history.
Even if they had no artistic interest, they would be worth teaching and studying.
And the fact that some of them are luminous works of art makes teaching them a particular pleasure, a particular joy, a real vocation.
So if we talk about films as a national expression, what we're talking about here is the extent to which the assumptions about personal relationships and the assumptions about the way society operates are going to be grounded in culturally, socially specific phenomena, socially specific practices.
And we're also talking not just about the content of movies, but also about the structure of the industries which end up providing movies to the public.
And part of what I want to at least allude to today in the lectures and materials that we're looking at today is to crystallize or concretize this idea that the variations that are possible within the broad universe of the cinema so that, for example, the individual and atomistic system that developed in the United States for the production of movies, the capitalist arrangements that developed in the United States for the development of movies, are in many ways radically different from the systems that were developed in some European societies or in the Soviet Union.
And there's a particular contrast with the Soviet Union, which developed movies in a quite different way and had a quite different notion about them.
The emergence of the movies coincides in some degree with the turmoil in the Soviet Union.
The Russian Revolution is 1917.
Movies become a central source of information and propaganda for the emerging of Soviet culture.
Lenin called movies our greatest art form, because he understood how important they were in promulgating certain ideals and embedding those ideals in the society.
And in fact, there were not in Russia a series of independent companies that produced films.
There was a top-down arrangement in which the government controlled filmmaking.
It doesn't mean that they didn't make remarkable and interesting films, but it was a different system.
It was a top-down system.
We had central government financing in which the genres in Soviet films could be said to have had what we might call rhetorical sources.
For example, a revolution story is one genre of Russian film, celebrating the heroic struggle of the people.
There were even sort of genres that we might call building genres or creating genres, and they were about creating a farm or building a skyscraper.
And the film was put in the service by the Soviet state, was put in the service of this emerging society.
It was understood as a system that would mobilize mass social forces for the betterment of society.
And these differences in attitudes and in the ways films are financed and who makes the decisions about what films will go forward, of course, has a profound impact on the nature of those movies.
Our demonstration instance today will be one of the most famous passages from Eisenstein's Potemkin to demonstrate some of the, in a much more concrete way, some of the implications of this difference between American and Russian film that I'm suggesting.
There also profound differences, and I'll develop this argument a little more fully to this evening when we shift over to the great German silent film that we're going to look at tonight.
There are profound differences between the American and German systems of moviemaking and attitudes toward the making of movies.
And I'll elaborate on some of those notions later today in the evening lecture.
But for the moment then, suffice it to say that virtually all movies are going to reveal or are going to embody the values and assumptions of the culture from which they come, that that makes them anthropological artifacts of profound significance and distinguishes French film from British film from American film in ways that continue to be illuminating and significant.
But there's certain other contrasts or potential tensions in this notion of film as a cultural form that I'd also like to develop or spend a little bit more time on.
One of them is the notion that there's a profound, even a fundamental difference, more broadly, not just between French and American cinema, but between all forms of European cinema and the American version.
And this is a principle we'll talk about more this evening, but I want to allude to it now.
One of the ways to crystallize this is to remind you of something we've already talked about briefly in the course, which is the migration of filmmaking from the east coast to the west coast in the early days of filmmaking in the United States, the flight of filmmakers to California.
And we've talked a little bit about why that's a significant transformation and a significant move.
But perhaps, the most important aspect of this historical fact, the migration of the movies to the west coast, is that what this meant is that the movies in the United States were able to develop in a culture whose intellectual and artistic and cultural authorities were on the east coast, as far away as possible from where movies were developing.
In other words, the American movie is much more fundamentally in its emergence a popular form, a form that has no consciousness of itself as a work of art.
It knows that what it's trying to do is make money and entertain people, and the earliest-- very early, there were some directors like DW Griffith who recognized the artistic importance of movies.
I don't mean there weren't directors who recognized it.
Chaplin surely thought of himself as making works of art, especially later in his career.
But the fact is the American movies begin on the farthest Western verge of the society.
Nothing developed there.
New York is the cultural center.
Boston is a cultural center.
Maybe we could even say some of the great Midwestern cities have some kind of cultural authority, but there's nothing on the west coast.
And what that means is that all the writers, all the dramatists, all the actors, all the theater actors, all the poets, all the musicians-- they were in the East.
They lived in New York, and there was a kind of freedom that this imparted to American movies.
And this is a very sharp contrast with the development of almost all forms of European cinema, partly because the cultures are literally geographically more limited, unlike the vast expanse of the United States.
But also because of the much stronger traditions in these European societies of high culture, the much stronger respect in these societies for theater and for poetry and for prose narrative.
In the European societies, and this was especially true in Germany, but it was true in some degree in every European society, including the Soviet Union, there was a sense that the movies were emerging in the shadow of older art forms whose greatness and grandeur shadowed, menaced this emerging form.
And in a way, the distinction I'm mentioning, the difference I'm mentioning, accounts both for the limitations and for the glories of both kinds of film, because if the European film was more static-- and we'll talk much more.
I'll give you some examples of this tonight.
If the European film was more static, it was less cinematic in a way in its early years, because it thought of itself as emerging from literature, from theater, from poetry.
And in fact, some of the important early German filmmakers especially were people who came from theater, and they had theatrical notions of what art was.
We'll talk more about this this evening.
So because that was true, the glory of the early European cinema was its recognition that it could be artistically powerful, its sense that it was talking about important subjects.
But of course, the limitations were that it was often very boring visually, that it was serious, but not a movie, that it didn't exploit the properties of the medium nearly as quickly.
It didn't try to explore the unique properties of the medium nearly as quickly, in part because it was so in thrall to inherited ideas of artistic value and artistic expression.
This isn't entirely a disadvantage, as I said, because it also imparted to European filmmakers a sense of dignity and the importance of their enterprise that served them well in certain ways and made them pick ambitious subjects.
And you'll see the outcome, the final outcome, once the European film was liberated into a greater cinematic freedom.
And I'll show you an example or two tonight of that.
It became something immensely rich in part because it had this legacy of high art behind it and high artistic ambitions.
The United States' story is almost the opposite.
In the United States, there was a kind of glorious sense of having no responsibility toward older art forms.
There was something exuberant, experimental, joyous, unembarrassed about early American films.
They didn't think of themselves as artwork, so it gave them a kind of freedom.
They were also vulgar as hell.
They were often trivial and silly.
They often had limited artistic ambitions, but they explored the nature of the medium in a way that became the legacy of movies and a legacy that was communicated to other societies as well.
Well, this distinction then between American and European cinema is something I'll develop a little bit more fully with examples this evening.
But it's a crucial distinction.
It's a crucial difference, and it tells us a lot about both forms of filmmaking.
There's one final tension that I want to mention here.
We'll return to it again when we come to look at Singing in the Rain later in the course, which dramatizes this subject among others.
There's another kind of tension implicit in what I've already said, which is the tension between what we might call popular culture, notions of culture that are enjoyed by the masses, by everyone as against high culture like opera and poetry and theater, which only the educated people go to.
And this tension is especially important-- it's important in many films-- but it's an especially important tension in American movies.
And one of things that we will come back to in different ways as we think about these American films is the way in which very often, American films position themselves as the antagonist of high culture.
And there are many films that actually do that, and some of the Marx Brothers films systematically dismantle the objects of high culture.
There's one Marx Brothers film called A Night at the Opera, which takes place in an opera, and the whole set comes crashing down.
The whole place falls apart in the course of the film, acting out a kind of aggression against the older art form.
And this is a tension also that we will see played out in some of the films we're going to be looking at a bit later in the course.
So this notion of Hollywood as the embodiment of a certain kind of demotic vigor and populist energy is a helpful way of thinking about how, especially in the early years, American film was somewhat different from European film, and how it also very aggressively was happy to distinguish itself from established art forms.
I want to take a quick, what will appear to be a digression, but actually isn't.
I want to talk a bit now about two crucial terms that will be useful in our discussions of the matters I've already raised and some other matters that will come up later in the course, and then return after clarifying these terms to an example from Battleship Potemkin, Eisenstein's most famous film, to demonstrate something of what I mean by the principles of top-down organization and film as propaganda that I was talking about earlier, as well as calling your attention to some of the artistic innovations that we still attribute to Sergei Eisenstein.
The two terms I want to discuss are the terms montage and mise en scene.
They're contrasting elements of what is in all movies.
In a certain way, the term montage and the term mise en scene describe the most essential features of what movies are.
Mise en scene, a term drawn from the theater, which literally, it's a French word.
It literally means what is put or placed in the scene, what is in the scene.
Mise en scene refers to the single shot, to what goes on within the single continuous unedited shot of film, the frame of film, however long it lasts.
And the mise en scene of that shot is virtually everything inside that frame.
In other words, even how the actors move in the frame is part of the mise en scene, but especially, the mise en scene emphasizes what is the environment like, what's the furniture like, what's the relation between the foreground, the middle ground, and the background.
And in mise en scene, the emphasis is on the composition within the frame, and sometimes very great directors will compose their frames with such subtlety that if you freeze them, they look like paintings.
They're balanced or unbalanced if that's the artist's intention in particularly artistic and complex ways.
So we can think of this in some sense almost as having a kind of painterly equivalent.
What goes on in the scene within?
The other great term, montage, which is also a French term, comes from the verb the French verb monter, which means to assemble or to put together.
And a montage means what is put together, what is edited, what is linked together.
So a montage means the editing of continuous shots in a sequence.
So the montage of a film is the rhythm of its editing.
So all films have both elements in them, and in fact, we need to be aware of both of them.
When we look at a film, it's often very helpful to ask yourself questions about the rhythm of the editing, to pay attention to how long the shots are held, to the way the film is edited.
Again, the Eisenstein example we're going to look at in a minute will give you some dramatic instances of why manipulating the editing and the montage can be so dramatic and so signifying.
So there's a kind of convention that has developed, and though ti radically simplifies in some ways, it's a simplification that's immensely instructive.
One way you can talk about directors is to categorize them as montage directors or mise en scene directors.
Mise en scene directors-- I'm oversimplifying, remember, because there's montage in every film.
So a mise en scene director can be a master of editing too, and a director that we identify as a montage director certainly has to know how to manipulate his mise en scene.
So it's not as if one kind of director doesn't do the other thing, but what it does try to signify, what it does try to indicate is that directors we call montage directors are directors whose effects come in a central way from the way they edit the film, from the quickness of their editing, from the way their editing manipulates or controls meaning in some sense.
And we therefore would think of montage directors-- Eisenstein is a classic example.
Hitchcock is probably the contemporary example, near contemporary example, that most of you might have in mind, in which the editing of the film, the quickness with which the shots develop, the way the music is superimposed on the editing rhythm, to increase your emotional response to the film.
What we would say is that that's what a montage director embodies.
So if we say that Hitchcock is a montage director, what we mean is that most of his most profound meanings come from the way in which he edits his film.
And a contrast would be, let's say, with a director like the director we're going to see in a few weeks later in the term, Jean Renoir, a realistic director who might be called much more fully a mise en scene director, because he does edit.
His editing rhythms are subtle, but he's interested in long takes.
Montage directors like short takes, shots that last only a short time.
In the most dramatic segments of the segment from Battleship Potemkin that I'm going to show you this afternoon in a few minutes, sometimes the edits are so brief that they don't even last a second and a half.
The average number of shots in the film as a whole, in Battleship Potemkin as a whole, the shots last four seconds.
That's not very long.
In a Renoir film, they might last 10, 15 seconds, sometimes much longer than that.
That's a very long time for a shot to be held, and if a shot is held that long, it means the camera will move, action will occur in it, but it'll still be a single shot.
And can you see that if you hold the shot for that time, and the camera moves like this, what is it encouraging?
It's encouraging you to think about the relation between characters and the environment.
It's encouraging a kind of realistic response to what the film is showing you, whereas if you're looking at a film in which the cuts occur every two seconds, you don't have time to sort of take in what's the relation between the actor and the furniture.
You're disoriented, and in fact, Hitchcock often brought his editing to a point just below the threshold of disorientation.
When Eisenstein was theorizing about the power of editing-- he was one of the first great film theorists-- he talked about the way in which you could control an audience physiologically by manipulating montage.
And it's true.
You can, as you will know, and something that fascist societies are fully aware of and make use of.
So this distinction between montage and mise en scene is immensely useful, and in some degree, if you apply the terms generously and tactfully, you can learn something about every film you look at by thinking about how these elements work in the film.
I want to turn now to arguably, certainly one of the most famous films in the history of cinema and to a particular fragment from the film or an extended one, which I think embodies and will help clarify many of the abstract ideas I've just been suggesting to you.
Let me say a word about the film.
The film Battleship Potemkin was produced in 1925 at a point when Eisenstein was now at the height of his power and authority.
And it commemorates a moment in an abortive revolution of 1905 so that by the time Eisenstein came to make the film, Battleship Potemkin was kind of like a founding story, or at least, it was about an abortive founding that would then occur years later.
What it dramatized was a historical fact.
There was a rebellion by the crew of the Battleship Potemkin against its officers, and it the battleship sailed into the port of Odessa, and its mutineers were welcomed by the people in the port of Odessa.
And then the czar, angry that his Navy and his Naval officers had been mutinied against, sent soldiers to Odessa to decimate not just the mutineers, but the population of Odessa.
And so the film, it was understood in a way, it was a revolutionary document or an attempt to sort of create a kind of founding myth for Russian society, because everyone watching the film would have known that the real revolution occurred only whatever it was, 12 or 13 years later, and that this was a kind of rehearsal.
And so the film would have had a kind of patriotic aura for its audience.
So the passage I'm going to show you is the famous passage.
I think David Cook calls this the most famous montage sequence in the history of cinema.
It was certainly profoundly influential, and as we're watching it, I may interrupt it to say a few things as you're watching, but I'll try not to do too much interruption.
What I want you to watch for especially is not only-- I will have to make some commentary-- as you're watching it, among other things, watch for the way in which the length of the shots or the time between shots varies.
And as this passage begins to increase in intensity and terror, the cuts become even briefer.
And then watch also the way in which certain other strategies of Eisenstein's reinforce these montage strategies.
For example, where the camera is positioned.
Is it looking up at a character, or is it looking down?
And very different thing.
If you look up, you enlarge, and you mythify.
If you look down, you humiliate and minimize.
Watch how he does that sort of thing.
You'll find it, I think, very illuminating and significant.
The sequence is often seen today, and rightfully, I suppose, as deeply heavy-handed, because you're not allowed when you're looking at this film to have an alternative view of things.
The film doesn't leave you room.
Eisenstein's strategies don't leave you room for independent judgment.
You're immersed in a spectacle so emotional and so wrenching that you don't have time to sort of sit back and think and come to conclusions.
And one could say that this is one of the great differences between montage directors and mise en scene directors.
Not an accident most horror movies, all horror movies really, are a form of montage, because your feelings are being manipulated.
You're not supposed to be allowed to sit back and say how ridiculously implausible these events are.
If that happened, it would spoil the film.
We'll come back to these things.
So here is the Odessa step sequence from the Battleship Potemkin.
[MUSIC PLAYING] These are the Odessans welcoming the mutineers.
One of the things that Eisenstein was fond of was a theory of montage that was based on two principles.
One he called typage, typage-- T-Y-P-A-G-E. And what he meant by typage was the idea that there were ethnic, very racist in a way, that there were ethnic and social types that could be recognized visually.
So he would type.
So he felt, if I show you this face, you'll know he's a working class character.
If I show you a woman with a parasol, you'll know that she belongs to the upper classes.
And in fact, he's probably right about that.
Here are the czar's forces, come to punish the mutineers and the city of Odessa.
So the soldiers are on top, and they're forcing people down the steps, and they are presumably shooting them.
.
Kristen, freeze it for a second.
I don't want to distract you by talking while it's running, so let me interrupt it for a second and say something else about the way the film works.
One of the things Eisenstein understood was-- and it's actually a brilliant discovery.
He realized that he could create through his strategies, especially of dramatic editing, he could create a situation in which the actual time of the experience that you're watching was not real time, but was what might be called emotional time.
That is to say what's happening here, it's probably in the film taking longer than it took in reality, because in moments of horror, the horror is extended.
And watch how those kinds of rhythms operate in the film.
OK.
Seems like a naive hope.
Freeze it again, Kristen.
One other quick observation.
I hope you recognize how artful this is, even if you're not moved in the way the original audiences would have been.
I think contemporary audiences often feel it's too heavy-handed.
They resist the extent to which the film is manipulating them, but think back to the earliest days of film.
What an unbelievable, shocking, incredibly exciting experience it must have been for early film-goers to have an experience that-- certainly for the Russian audience, but I think for every audience-- that was so intense and so emotionally powerful, so full of fear and violence that can be evoked by the rhythms of the editing, by the music, by how close-- I hope you noticed the way he mixes in closeups in incredibly powerful ways trying to create certain effects.
Again, you're not given a choice about how to feel about this.
You can descend from it by withdrawing your interest, but you can't say, oh, I really love those soldiers who were doing the shooting.
Let's make a case for them.
The film won't allow you to do that, will it?
In that sense, it's manipulating you, but it's telling us a story about the creation of a revolutionary society.
Finally, remember, I said that this is a question about emotional time as against real time.
Think how long this has been going on.
You think that this massacre is over, but in fact, it's only half over, as you'll see.
There's going to be a moment when horse-mounted cossacks, horsemen, show up at the bottom of the steps and get them in a pincher.
Go on.
I don't think this soundtrack is the original soundtrack.
It's very good though.
This is a brilliant moment.
I don't know whether we can attribute this to Eisenstein or not when suddenly the music stops.
There should be sound now.
Maybe something wrong with our print.
I wanted to wait at least until you saw this, because some of you may recognize this moment as something that's been copied in recent American movies, a kind of allusion or a reference to this scene.
The moment I wanted you to think about is this baby carriage.
OK.
Thanks, Kristen.
Blood in the eyeglasses-- can you think of a movie in which you've seen that recently?
Maybe not that recently.
It's actually an ancient film now by your standards.
How about The Godfather?
There's a wonderful scene in The Godfather where a guy looks up from a massage table, and he's shot through the eyeglasses.
Very memorable moment.
It's surely an allusion to this movie, but how about the carriage going down?
There have been several films that actually recreate that moment, but the one I'm thinking of-- Who is it?
The Untouchables
Yes.
From The Untouchables.
Who's the director?
Do you remember?
Yes.
Brian De Palma's film, The Untouchables, has a moment just like that.
And De Palma, of course, is a kind of historian of movies.
Virtually every scene in a De Palma film is a reference or an allusion to an earlier film.
And part of the importance of Battleship Potemkin is that it is still a fruitful and fructifying source of imagery for contemporary filmmakers.
So let me conclude then by simply reminding you that, as Cook suggests in his book, this is the single most influential montage sequence in cinema history, and that it's a wonderful instance for us, I think, of the way in which film in a different kind of culture, in an authoritarian culture, in a revolutionary culture, full of moral fervor, would be conceived both as an apparatus, as an engine of social transformation by a society that controlled film in a way fundamentally different from the way in which film developed, let's say, in the United States.
We will continue these arguments, and I hope complicate them this evening.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This afternoon I talked a bit about Russian film and suggested that among other things that the cultural environment in which the Russian film emerged was an environment of revolution and of ideological turmoil.
And we could say, therefore, in some degree, if Russian film served revolutionary ideals and the ideology of the Bolsheviks, the German film had its own sort of boss, its own authorities that it was following.
But it was not in a narrow sense political or ideological as the Russian example was.
But it was something similar.
The German film I want to suggest for a good part of life, a silent film, was in thrall to established culture.
It's an example of the sort of thing I was saying this afternoon about the difference between European cinema and American cinema.
American cinema grew up in an atmosphere of wild populist freedom, almost untouched by the cultural authorities who mostly on the east coast of the country dominated literature, drama, and so forth-- music.
In contrast, in Germany, the film grew up in the cultural centers of the society, and the dominant cultural figures, the major writers, the major dramatists, the major poets and musicians, the established theater actors and directors all migrated into the film.
And it gave German film a tremendous intellectual ambition and authority from its various earliest stages, and I'll show you some examples of this.
It made the German film, especially the German film in the period between the end of the First World War and the advent of sound that sometimes is called-- I don't know if this is correct-- but some people call it the golden age of German silent film, the golden age of German film, because an extraordinary number of very important and influential films were made in this period, roughly from about 1916 or '17 through the '20s, through about 1926 or '27 before the advent of sound.
And it certainly is a very distinctive and influential moment in the history of cinema, but as you will see in the examples I'm going to give you, influential as many of these films were, they were influential in what we might say-- they were influential in part not because they were wonderful examples of exploratory innovative cinema, but because of the grandeur or the disturbing authority of the themes they encompassed.
In other words, the German film, as I suggested this afternoon, because it was guided by cultural authorities who were adept at earlier media, at older media, they already came to the making of film, of their films, with high artistic intentions.
I'm not so sure they realized those intentions, because they didn't explore the full nature of film as much as might have been done.
But they were interesting and deeply influential films.
And I should also admit that my perspective is, perhaps, biased.
There are certainly many, many people who have written magnificently and seriously on this era of German film, who would not be negative as I am about what I'm calling the sort of uncinematic or unvisual qualities of some of the masterpieces, early masterpieces in the category.
It's a contested argument, and I don't want you to simply accept my word for this.
Take it as a perspective that could be qualified or even challenged by other folks, and especially by certain film scholars who see this moment in German cinema as a really fruitful and generative moment.
Much of the basic understanding that we have of German silent film comes from one remarkable book by a woman named Lottie Eisner, and I wanted to mention it here.
I borrowed a tremendous amount of what I'm going to say from her book, learned a great deal from it.
Eisner was a wonderfully important historical figure, worked as a film historian and annotator and film archivist for a good part of her career.
Some of you who are film buffs may know that Wim Wender, as the great contemporary experimental German director, dedicated his film Paris, Texas to Lottie Eisner.
She worked for a long time in the French Biblioteque, the first great archive of movies in the Western world.
She also wrote books, separate books about Murnau and Fritz Lang, two of the great figures in this tradition we're looking at briefly this evening.
And her arguments in The Haunted Screen are still widely accepted.
The book focuses on the Weimar period, the period before the rise of Hitler, after the First World War, in which German culture was an astonishingly rich, fermenting, stew of innovation and anxiety.
It's the era of the great cabarets, and you will return to this era in an imaginative way when you come to see the film Cabaret later in the term, which tries to reimagine that moment.
And it actually dramatizes the era in which the German films I'm talking about this evening were made.
One way I can begin to identify or explain the distinctive qualities of these German films is to talk briefly about the idea of expressionism.
The broad word expressionism refers to movement in modernist art that involves principles of distortion and surreal exaggeration.
So an expressionist work of art, either a painting or a poem or a film, is in some degree interested in finding equivalents for the inner life, dramatizing not the external world, but the world within us, and especially the tormented world within us-- our anxieties, our fears, our sexual fantasies, our murderous impulses.
There's a dark, disturbing side to expressionism, especially to German expressionism.
There are expressionist artists who appear in other traditions, and expressionism is a movement that goes far beyond Germany.
But one might say that German expressionism is the expression of modernism in German culture.
It's almost indistinguishable.
If we talked about modernism more generally, we would say that not every aspect of modernism involves expressionism.
But in Germany, we would say the modernist impulse was embodied in the expressionist tendencies of the art forms that appear in that period.
Let me talk a bit about them.
They involve the external representation of the inner life, as I've said.
And they're interested, the expressionists, in elemental emotions like fear, love, hatred, anxiety.
They're interested in our aggressive and appetitive impulses.
The sources of this expressionist movement are in German romanticism.
This is the literary movement that gave rise to important poets and painters in the earlier part of the 19th century.
And also in economics, in the historical realities of German culture, because the period we're talking about when these films were made, the Weimar period was a period of-- was a unstable and turbulent period.
There were power outages all the time, for example, frequently, even in the major cities like Berlin.
And the visual style of the expressionists-- there were expressionist painters, expressionist poets, expressionist musicians, expressionist dramatists.
Virtually all the art forms in Germany in some degree participated in this interest in the irrational and in the hidden and in the turbulent night side of our personalities.
And I can, perhaps, very quickly capture some of the essence of this movement by showing you a couple of paintings that are examples of this expressionist movement.
Let me show you one image first, and then I'll do a slideshow while I talk a bit more, and you can watch the slideshow.
This is a painting by the German painter Beckmann, and it's called "The Family."
It sometimes has been used as the illustration on the cover of Kafka's The Metamorphosis.
And we might say incidentally, if you're looking for examples of what expressionism would be like, how many of you have read Kafka?
Only a couple.
OK. How many of you remember the first sentence of The Metamorphosis?
All right.
I'll tell it to you in some translations.
Gregor Samsa awoke one morning after a night of uneasy dreams to find himself transformed into a gigantic insect.
Maybe the most famous first sentence in modern literature.
But think about that.
And then the whole story proceeds from that shocking transformation.
So one of the things that's characteristic of these forms of expression of are principles of exaggeration that take you into surrealism and into modes of exaggeration that attempt to illuminate reality not by giving you a realistic picture of it, but by exaggerating certain elements in reality so intensely that you understand them more deeply and fully than you might have otherwise.
It's important to recognize that surreal impulses like this or expressionist impulses like this do not mean that the stories or art forms that are committed to these strategies are unrealistic in the broadest sense.
That is to say they still talk about reality.
It's just that the realities they talk about do not allow themselves to be represented in straightforward ways, because they're either hidden, or the intensity that they represent is so hidden within us that if you represent them externally, you need to exaggerate.
You need to highlight.
You need to enlarge.
So we might think of Kafka as a literary embodiment of this principle of expressionism, and we might think of paintings like this as the painterly embodiment of it.
And look at this.
We won't have to spend much time on this, but look closely at it.
Look at how the whole family is crammed together in an incredibly tight space, the ceiling pressing down on them.
Look how grotesque each of them is.
But what's the most obvious thing about the painting?
Can you see what it looks like?
None of the characters are looking at each other.
There's no connection between them.
They're all isolated in their own little space.
Not only are they physically grotesque and a little weird, but they're all caught in their own narcissistic or, perhaps, drug-induced stupor.
Calling it "The Family?"
Think of what an unromantic unsentimental view of the family this is.
But it's the principles of distortion and surreal exaggeration that you can see in this image that we might say are the essence of what is meant by expressionism.
Do the slideshow now.
While I'm talking, I'll show you a series of images by other painters, and we can put this on so they'll see what's what.
So just play this, Kristen, while I continue.
So one of the things that-- so this expressionist movement was central to all the high arts of German society at the time.
And it's, therefore, not an accident, not at all, not even remotely an accident that such energies would express themselves in the new medium of the movies, because so much of the German art establishment was already embracing principles of this kind.
And some very remarkable, of course, and significant, immensely influential films grew out of this movement.
They emphasized atmospheric qualities like lighting and set design, and they were interested, as I've already implied, in morbid states and in distortion, in inner fears and anxieties.
And what one of the things that the expressionist moment in German cinema certainly did was it tremendously furthered our sense of the power of the cinema to explore human subjectivity.
Even the films that, for me at least, looked more like historical artifacts than living works of art have this fascination for us, for me.
We might also very quickly acknowledge how profoundly influential the movement was.
Can you think-- any of you know what famous directors after the-- not even German directors-- are linked repeatedly with German expressionism, have connections to them?
One of them is Alfred Hitchcock, who actually, his very first film was a co-production with German producers.
And he carried it-- if you think of Hitchcock's work, Hitchcock carries the some of the darkness and the distortion and the interest in our inner convulsive lives that is present in German expressionism.
But there are other examples as well.
The American gangster film shows the influence of German expressionism very powerfully, and the great American director Orson Welles was steeped in German expressionism, and some of his greatest effects and the atmospheres of his films could be said to be a direct outgrowth of expressionist directing, of expressionist movies.
So it's a critical and powerful movement.
So let me now turn to a couple of examples to give you a sense in film of how this expressed itself.
And my first example comes from the film, the very famous film directed by Robert Wayne, called The Cabinet of Dr. Caligari.
It's a story about a murderer, who when he goes to sleep, he turns into a monster, and he wakes up and he walks around like a somnambulist ready to kill people.
And it may be there were, of course, lurid stories in the public press of these kinds of all the time.
So there may have been, in the watching of the film, there may have also been a kind of [INAUDIBLE] in the audience of feeling that what they were watching potentially was possibly real or reflected certain forms of certain possibilities in reality.
But as you'll see, the film is tremendously stylized.
It's tremendously planned in a way.
One of the most interesting things about the film is that the sets were designed and painted by prominent expressionist artists, and then the fact is it was as much an artwork as it was a movie.
Let me show you part of it, and you could see for yourself the consequences of an artistic movement that grows out of traditional art being transferred to the movies.
Here's a scene from the film.
Do we have sound?
No sound, I guess.
Sorry.
There would have been music, of course, and very scary music.
Maybe you can hear some of it.
I think their soundtrack is also a new one.
It was superimposed and made for the film later.
But I suppose in a certain way, this has a kind of interest.
But do you see how static it is, how uncinematic it is, how stable the camera is, how the emphasis is on-- I don't know.
What could we call it?
On set design, on surreal angles, on this strange form of lighting?
This chiaroscuro black and white lighting, very great emphasis on contrast between black and white, of course, becomes a trademark element of expressionistic films and then is carried over in many other film traditions, including the American tradition called film noir-- dark film, black film.
So here's this somnambulist walking over to the sleeping woman.
We might note parenthetically, isn't interesting how long one might say that this is one of the earliest horror films?
How long has the horror film used as one of its central elements violence against women, and especially sexual violence against women?
Look at this.
Do you see how he looks like an actor on a stage?
Now surely it was intentional that they put those two men who were woken up by the noise in a space that seems without space.
There's a surreal ungrounded element in the film.
All right.
So they're looking for her now, and they're going to be [INAUDIBLE].
Can you speed up, Kristen, so we don't watch the whole thing in slow motion, in full, but we get near the end?
Is that possible to do?
So they're-- keep-- let it speed up a little bit more.
OK. Now we're near the ending here.
Again, see how static it is, how that's impressive, the building of the sets might have been impressive.
But you see how in some sense how uncinematic this is in certain ways?
Seeing the camera stable in one spot, active moving toward it, moving in and out of the camera's range.
This is a theatrical idea of what the camera should do.
It's sitting in the audience watching a set, and in fact, you can feel the theatrical authority here.
Watch this image now.
And then it's something clever if you're interested in silhouettes, creating an image of darkness and horror, I guess.
But you can also see the limitations of it.
I want to show you one other example, a more distinguished example, an example that does begin to manipulate the visual image in somewhat more rich and significant ways, a film even more influential than Caligari, what is called so often the first science-fiction film.
It was the very first feature length science-fiction film in the world.
Its title is Metropolis, directed by Fritz Lang.
And at the time, many film scholars claim it cost something like five million Reichsmarks to make.
I don't know if Reichsmarks were the equivalent of dollars, but let's assume they were.
In 1926, this was the most expensive film that had ever been made up to that point, and it is a measure of the authority and the prestige that movies had in German society, that so much money would be invested in a film.
This would never have happened in the United States.
They weren't spending that level of money on American movies.
They didn't have to.
They couldn't.
And in some sense, it shows you the authority and the centrality, but it also explains why we had to wait before the German film fully discovered its cinematic or its filmic essence.
And I'm going to show you what I think is the moment or one of the moments in which this discovery was made.
And the director we're looking at tonight, FW Murnau, is the director who made this discovery and freed the camera from the constraints of high art, from the constraints of the inherited attitudes toward art that the established culture in Europe and especially in Germany imposed upon the making.
So here are a couple of scenes from Metropolis.
Still, I think we're not fully there yet, but it's a significant film.
It's much admired by certain film scholars, although not by me.
[MUSIC PLAYING] Some of you may notice how Chaplin borrowed from this introduction.
And you could certainly see how this film influenced Modern Times.
Feel a lot of Chaplin there, I hope.
You think there's an idea or an attitude toward what labor means in these images?
These are the workers of the city.
Interesting that such a politically radical film would be so heavily financed.
That's enough of this.
That's plenty of this, too much of it, in fact, I think.
But can we speed up a little, Kristen?
I think we can come to one more image from a different part of the film.
So I won't talk about the plot of the film, which is rather ridiculous I think, although it's also a kind of science-fiction fantasy, a dystopian fantasy.
But there is a sequence later in the film.
Let's go further.
So here's another image, another moment from the film.
Visually more exciting, but it also shows you in some sense how literary the film is.
And also, I want you see it also, because it remind you, again, of how much Chaplin borrowed from this movie when he made Modern Times.
Now there are many imaginative effects here, but still, you see how stable the camera is, how stationary it is?
The kinetic power of the camera had not yet been discovered, not because it was undiscoverable, because the Americans had discovered it a decade earlier, but because the power of older forms was inhibiting the people making the movies.
Again, in the early days, you can imagine that some of these effects would have been very dramatic-- the falling bodies.
Still, watch what happens here.
We've turned symbolic-- Moloch, the devil, the devil's henchman.
That's what modern industrial society is, a maw that will swallow us up.
You see how weird and disturbing it is.
Thank you, Kristen.
A serious, an interesting film, but still in certain ways, I hope you could see both deeply melodramatic, and the acting styles are very broad.
Where do they come from?
Theater.
They're theatrical acting still.
They haven't fully adjusted despite the power of these films, and in intellectual and thematic ways, they haven't fully adjusted to the idea that they're movies.
And the man who solved this problem or the German director who understood more fully than the others what it meant to make a movie was FW Murnau.
His dates are on the board.
You could see he died in his early 40s.
It was a tragic death.
He was just coming into his own as a great, great director.
He had emigrated to the United States in the late '20s, and the film I've listed on the board, Sunrise in 1927, was made in the United States.
And many people think it is the greatest silent non-comedy made in the United States.
It's a bit sentimental, but there are brilliant, brilliant, brilliant things in it.
It's a very, very rich artifact.
And then with the famous documentary director, Robert Flaherty, his last film Tabu, which he co-directed with Robert Flaherty, became a tremendous success.
But he died in 1931 just before the film was released, just on the verge of a kind of a claim that perhaps no other director in the world at that time had.
It's tragic, a tragic story.
The film in which we can see Murnau freeing the camera is the film called Nosferatu made in 1922.
And some of you will perhaps recognize it as the origin film for a tremendous number of horror movies.
Many people would call it the first horror movie.
That may be an exaggeration or inaccurate, but it is among-- it is certainly the most influential and significant early horror film.
It's based on the Dracula story, and like Dracula itself, it tells a very strange sort of story about vampires who survived for centuries and then come into the world, and by their little sort of suggestive sexual bite in the neck of a woman, transform the women into vampires as well.
And it's now a kind of commonplace established idea that this energy, that the kind of Gothic fantasy that is embodied in the story of Dracula and in so many similar ones, has its origins in various forms of sexual fantasy.
And the Dracula figure, when he kisses the woman's neck, is symbolically engaged in a sexual act.
And the woman's resistance but then her transformation into a vampire is presumably the woman's transformation to sexual activeness in some way.
At least these are the kinds of symbolic or underlying associations that many scholars bring to bear when they talk about why such stories were so fascinating and so popular at the end of the 19th century.
So one thing to remember about Nosferatu is it is an adaptation of a novel already very widely known.
And in fact, there were copyright questions about Nosferatu.
The copyright owners from the book sued the filmmakers, and I'm not sure what the outcome was.
But the film did survive in any case.
So here are a couple of scenes.
Here is the beginning of Nosferatu.
And the subject matter seems to me not that interesting, but again, my view is not necessarily the only view.
There are people who love these films.
I don't know why they do, but I don't want to simply impose my view on you.
If you're interested in this, or if you share a love of horror film-- one of my problems is I don't like horror films today, even though they've reached a very sophisticated level.
I think they're silly.
I don't get them.
So I'm not a good judge of them, and my negative judgments, take them with a grain of salt.
They're partly colored by my own preoccupations.
There are ways I could defend my position, but I don't feel I should impose my views on you.
I do think it's a silly story, and it seems to me that reality is complex enough without sort of inventing horrors that hide underground for thousands of years and emerge with magical powers.
The actual horrors of reality seem to me sufficient and not adequately studied.
So that's one of my objections to these forms of fantasy.
But in any case, that's personal.
But in any case, what you can see in Nosferatu, even people who are not as interested in the subject matter, they can see what a technically innovative film it is.
Watch this.
Here's the beginning.
[MUSIC PLAYING] In, fact freeze it already.
Does this seem more realistic to you?
Do you see already-- no fake sets?
We don't feel we're watching a theatrical piece, that a camera is filming a theater piece?
What's one of the reasons for that?
The natural sunlight.
That's natural sunlight.
Go ahead.
The texture of his bedding also is a factor.
And of course, the sunlight is a crucial motif in the film, because Dracula lives by darkness.
And the film depends, quite brilliantly, on a series of juxtapositions of light and dark, of shadow and brightness.
Now look at this.
Freeze it, Kristen.
Freeze it for a second.
Look at this.
We're really outside in a real world.
Look at these animals.
And you can feel the film-- at first, we're disoriented.
We don't even know why are we looking here?
But the film's joy-- we can feel almost the film's excitement at showing us what the real world looks like.
I actually find this visually and intellectually much more exciting than a ridiculous set in which a symbolically dressed monster staggers around in a way that no one who ever walked the earth moved.
I find this much more compelling.
But what was important at the time was how liberating it was, because what Murnau was showing is look what the camera can actually do.
He's freeing the camera.
Go ahead.
OK. That's enough of this.
Maybe I'll give you one more quick scene.
Do we have a separate clip?
Let's go to the other clip.
Now here's something from the end of the film.
Can you freeze it a second?
I'll sort of briefly set the [INAUDIBLE].
This guy is Dracula's weird assistant, and he's escaped from jail.
I find the story idiotic, ridiculous, totally unserious.
But look at the realism visually here.
This is maybe the first time that these narrow alleyways were captured with such precision.
It's almost as if, at least for me, the subject matter is irrelevant, and what the film is showing about how it can render reality is so exciting that it's as if a new universe has opened up.
And I must admit that even though there were American directors who were working with a freed camera long before this, no one ever used the camera outdoors more effectively up to this time than Murnau.
And I'll show you a little more of this, even though the plot is stupid, because you can see how realistic and persuasive, compelling the visual experience is.
See the freedom of the camera to shoot down, how that changes around you?
The quickness of the editing.
There he is up on top of his roof.
If you didn't know the context, you would see this as a form of oppression and mistreatment.
Why are they going after this poor guy?
And I think that is an implication of this, even though he's the source of evil.
But look at the realism of this.
Look at the texture of the bricks, the sense you have that this is a real place, that he really climbed down a roof.
The movies.
If the movies don't do this, they have to give us a sense of that gross reality, which the movies have the power to do.
They have to have very good reasons for giving up that power.
It's a very rare film that can really give it up.
Some animations can do it.
But even surreal films, even films that exaggerate or that create alternative worlds that are not like our real worlds today have the kind of kinetic camera and sense of the textured quality of the external world that we get in those images.
So Murnau was critical to us and crucial to us for this reason, and his great masterpiece is one of the two films that I've asked you to watch tonight.
We're showing both Nosferatu and The Last Laugh.
Let's do it in the order of The Last Laugh first.
I think Nosferatu's a less interesting film, but still from a technical standpoint, fascinating.
And those of you who are interested in the history of horror films, you should pay attention, because there's hardly a frame in Nosferatu that has not been alluded to or echoed in later horror films.
And it is the origins of what is now a dominant movie genre, both a literary and a cinematic genre.
So it's a very, very important film.
But The Last Laugh is a true masterpiece, and one of the reasons I think it's such a remarkable film is that it's freed in some degree from the pretentiousness, the surreal pretentiousness of expressionism.
There are expressionist elements in the film as you'll see.
Very cleverly done.
There's one immensely grotesque and powerful moment in which, for example, there are a series-- we are given an image of some gossiping women, who are trading gossip about the primary character.
And we'll see how interestingly Murnau handles that sequence.
And there's an element of distortion or grotesquery in that.
There are also elements in the film, moments in the film, brilliant moments in the film where what is shown externally is not what's happening externally, but what is going on in the mind of the primary character.
In other words, it carries the principle of finding ways to create a subjective camera even further than any previous film had done.
There's one remarkable moment in this film-- watch for it.
It's very astonishing-- in which the director of photography, Karl Freund, was asked to sort of figure this out.
There's a moment when the protagonist gets drunk.
He's very disappointed about something, and he gets drunk, and he staggers around.
And they were trying to figure out a way of dramatizing this.
They played around with the camera, and they strapped the camera to the actor's waist, and when he staggered around, the world you saw was staggering too.
And it created an immensely powerful sort of subjective experience for the viewer.
And there are a number of other places in the film where you can see that what's being dramatized is the inner life of the characters.
So there are expressionist elements in the film, but it transcends expressionism.
It becomes a film truly about reality without distortion, and of course, its basic subject is what happens to our old people when they get too old to function.
This is a theme I'm beginning to find more and more sympathetic as I age.
But it's one of the most powerful treatments of this subject.
And basically, the story is about-- as you'll see-- it's about a porter in a hotel whose job is to help carry baggage in.
But he's an immense big man played by one of the most famous theater actors of the day, Emil Jannings.
And at a certain point in the film, as you'll see, he's demoted.
He loses his job.
They take his uniform away.
His uniform is very important to him.
And they demote him.
He has to go and work in the men's room as a men's room attendant.
So the film in a certain way does several things.
First, it shows us what Murnau himself came to call the unchanged camera, the free camera.
And over and over again, you can see how immensely innovative the camera work in this film is.
The very opening scene of the film is a tour de force you might miss because it happened so quickly.
The camera follows a person down an elevator, a glass elevator, and then follows it across the room.
The camera was actually mounted on a bicycle so they could move it that way.
It happens so quickly you might not even notice it, although filmmakers were overwhelmed by it, because he was doing things with the camera that no one else had done before.
And you'll see other moments in the film where his use of the camera is complex and innovative.
I've already talked a bit about the themes in the film, but let me continue this.
So I've said in a way it's a working class tragedy.
And the very fact that it's about an ordinary man, but it tries to treat him with the kind of dignity that in the classical times kings and high born characters would receive is very significant and is a powerful aspect of the subject matter.
What also develops is a deep sense of the character of the protagonist.
The way in which his identity is connected to his job and how when his job, and especially his uniform, which is the symbol of his authority, is taken away, how he's unmanned, unhousled, undone, incapable of functioning.
So it's a very powerful, socially aware movie.
It dramatizes character with a kind of subtlety that's very rare in silent film.
You really have a sense of this man's character.
It's quite a brilliant performance, even though Jannings was its theater actor, and there is something somewhat broad about his performance, you can see him beginning to adapt to the camera and to the more nuanced things that the camera can do.
But even the parts of his role that are a little sort of exaggerated make sense because there's something pretentious and pompous about his character.
One of the subtlest things about the film, in fact, is the way the character has this complexity.
A character for whom the audience's sympathy is constantly solicited is also imperfect, in some ways, an annoying old man, not a nice person, selfish, proud of his superiority to the other people with whom he lives.
One of the ways you could see this operating is the film really operates in two different spaces-- the space of the great hotel and the space where the character played by Jannings lives.
He lives in a working class ghetto, a working class part of the city.
And the two environments, the hotel and the working class part of the city, are juxtaposed, and we learn about each of them by their juxtaposition.
The working class part of the city is interesting for a number of reasons, but one of the best is the pride he takes in his uniform.
He may be only a hotel porter, But in his neighborhood, he's a big man.
He's a big shot, and he's very proud of himself.
You can even see him boring the children and giving them instructions and things.
The most important point I'm trying to make is that he's a complex character that we learn this without dialogue through the genius of silent film, because he's both flawed and also an object of sympathy and affection.
The film doesn't simplify his character.
He's a spiky, unpleasant man in some ways, but we also still feel for him and recognize how terrible his tragedy is.
One of the most important characteristics of the film and the mark of what a cinematic achievement it is is that up until very close to the very end of the film, there is not a single title card, not a single use of words.
In other words, the film depends upon its visual power.
And you understand even complex things like the fact that the character is drunk, whether he's not thinking straight, or that what he's seeing is not reality, but some distorted version of reality.
No title cords explain this to you.
The film has a fluency, a visual eloquence that's very remark-- that's the mark of a movie maker.
But there's an interesting tip-off here.
An interesting problem here is presented, and I'll end with this point.
It has to do with the ending of the film, and I want you to think about this as you reflect on the film after you've watched it.
I've described the basic plot of the film, and as you'll discover, this catastrophe that descends upon the protagonist of the film is reversed at the end of the movie.
And see, he wins a lottery, and great good fortune descends upon him.
Don't be angry that I've given you the plot.
You can't talk about film seriously without doing it, so don't be disappointed.
I won't spoil movies by doing real spoilers.
If I think you shouldn't know this for the experience of the film, I won't mention it.
But don't be annoyed if I do this, because I need to explain this stuff to you.
So there's this reversal at the end, but it's a fake reversal.
You don't really believe it.
It seems inconsistent.
Why do you think it's there?
The film was such a downer.
The original cut was such a downer that the people, the producers paying for the film said, no, no, no, you've got to put a happy ending on the movie.
So Murnau and his colleagues-- it was a collaborative process-- the screenwriter, Carl Mayer, and the director of photography Carl [?
Freund, ?]
as well as the actors were all collaborating in the making of the film.
They said, OK, well, what should we do?
So they tacked on this happy ending.
One of the things that marks the happy ending is a title card, and the appearance of the title card is a confession or a secret signal to the real members of the audience-- this film is over.
Forget this crap that's coming at the end.
At least that's how I want to read it.
But what you should ask yourself as you're watching the film are two things.
First-- not ask but meditate on-- first of all, this poor ending or this ending that seems to reverse.
The reason the ending is bad is that the whole momentum of the film leads toward this catastrophe at the end, and to reverse it by an act of fortune like that is to deny the social and political implications and psychological implications of the story.
It's a very unartistic thing to do.
It violates the momentum of the text, of the trajectory of the character.
So it's a bad thing.
But it is also an embodiment of an enduring issue in movie making, which is the conflict between art and commerce.
Movies are commercial objects.
This is hardly the only film in the history of movies to be distorted in this way.
And in fact, that tension is not always negative.
Sometimes forcing directors to sort of communicate with their audiences is helpful.
It isn't always the case that these commercial imperatives are negative, although they often are.
But this is one of those classic instances that can you think of other famous examples?
Orson Welles, the American director, was often-- suffered tremendously from this kind of pressure, because his films were thought to be unwatchable, and one of his films was cut against his will, and he lost control of his films.
And he made very many fewer films than he might have made had he not been thought of as a commercial disaster by the Hollywood money people.
And there are many, many other examples of this in the history of movies.
One could say that this is one of the originating moments in which the conflict between art and commerce in the movies is dramatized for us.
But there's something else I want you to think about as well as you're thinking about the ending.
What is the moment at the end of the film that really would end the film?
Because I think there is one.
It's a visually very powerful moment, but it's so bleak.
It's so sad that you can understand why the money men wouldn't have liked it.
So as you're watching the film, think about all of these matters, but especially think about the way in which Murnau discovered more fully than any director up to his time what we might call the kinetic powers of the camera.
He unchained the camera.
He [?
fliberated ?]
the camera, and German film was-- world cinema, but German film especially, was never the same after that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome people.
I'm glad to see you this afternoon.
We finally make our transition to sound film.
I hope that you endured the silent era with good spirits.
I think you'll find, as the semester goes on, that even some of the silent films you found less compelling when you began will resonate for you and become more valuable.
And my expectation is that some of you will decide, after the fact, that Keaton and Chaplain, and perhaps even for some of you Murnau, presented you with films that were at least as memorable as the most memorable sound films.
But we now move into a segment of the course in which every object has more than merely artifactual a art value, although the artifactual value of these remarkable films is very great.
And as I indicated earlier in the semester, my hope is to toggle back and forth in some ways, in each of our conversations about these films on our syllabus.
To toggle back and forth between what we might think of as an aesthetic perspective, talking about the qualities in these texts that make them artful, memorable, long lasting, worth something even beyond the eras in which they were created on the one side.
And then on the other side, more an anthropological kind of perspective rather than an aesthetic one.
What do these texts tell us about the culture from which they come?
In what ways do the social, and moral, and political mythologies of the society find complex articulation in these texts?
So those are the two sort of perspectives between which we oscillate.
We now turn to the kind of film that emerges after the advent of sound in 1927.
And we can say that the period, roughly between 1927, 1928-- when the first sound films began to be made-- and the era into the 1950s, briefly after the advent of television, that era is often thought of as-- often called by some film historians the golden age of Hollywood.
And it was in some respects, maybe in many respects, a golden age.
It was certainly a golden age in terms of the size and loyalty of the audience for American movies.
And I'll come back to that in a moment.
What happened of course in the sound era was that an industry emerged.
A more neutral term for the golden age of Hollywood might just be the studio era.
Because what happened in the sound era was a fortification and extension of the system that had already emerged, and which we've discussed-- and which David Cook catalogs very fully in the required reading-- a fortification, an extension of the system that had emerged in the silent era.
The mass production system that was already in place in the silent era was perfected and extended.
The principle of the specialization of labor became even more complex and effectively deployed in the production of motion pictures.
And in this period, in the period between 1928 and 1930 for example, Hollywood spent what was then a tremendous amount of money-- $500 million-- in gearing up for the transition to sound.
It was almost as if silent films became obsolete almost instantly.
Almost instantly.
And the famous line from-- what's the title of the first sound film?
The Jazz Singer.
Who's the star?
The theater singer Al Jolson, right.
And there isn't very much dialogue in that film, although there are some 8 interpolated songs.
There's very little synchronous dialogue, but there's a little.
And one of the most famous lines-- maybe the most famous line in the film-- comes at a certain point where someone says "You ain't heard nothing yet", and the implication is you ain't seen nothing yet either.
And it is certainly the case that within three years there were no sound films really being made.
As I've suggested earlier, a comic and still illuminating perspective on this process of Hollywood adjusting to the advent of the new technology of sound is dramatized for us in the wonderful musical we're going to see in a few weeks, Singing In The Rain.
So in this era then, after the advent of sound, the movies become America's dominant entertainment form.
And in the period between 1930 and 1945, 7,500 films are made by the Hollywood production system, approximately-- almost exactly in fact-- 500 films a year.
Think about the confidence that this production system needed to have in its capacity to get a return on the number of films that it turned out.
And think about what it meant that the entire society was now geared toward this mass form of entertainment.
We can get some measure of the tremendous popularity of the movies, the centrality that the movies assumed by the end of the silent era and into the sound era, if we think of some numbers.
In 1938, for example, what we might think of as the central, the height, the apex of the studio system, of the sound era, there were a total of 80 million admissions per week in the American movies.
That comes to something like 67% of the population.
Now I'm sure some people were going to the movies more than once a week, so it isn't literally as if 70% of the population was attending the movies every week, but it's something close to that.
Far more than half the population had an intimate, weekly, habitual relationship to the movies.
And the form that the Hollywood system took on was surely, in part, a reflection of this increasingly intimate and knowledgeable connection between a mass audience and the kinds of movies, the kinds of stories that Hollywood was pumping out.
Can you think, for example, why the broad attributes of the system that I've already described, can you think about why those broad attributes would encourage the development of what we would call a system based on stars, in which, movie stars, the names of particular actresses and actresses become so well known that their names carry the film.
And people go to the movies to see Gable, or Colbert, or whatever famous performer, famous actor.
And there were dozens of such films, in a tiering system in which there were great stars at the top, B level stars below them, C level actors and actresses below them.
And the border between those were somewhat permeable.
And an extraordinary number of people in the society were familiar not only with these stars and the particular films they had made, and would follow them from film to film, but were all so familiar with what was a partly mythological and fictional story about their private lives that the Hollywood publicity system kept pumping out about the stars and performers.
And what was essentially created was a kind of mythological system in which the stars were literally-- for the audience-- characters or creatures who are literally larger than life.
And something, some vestigial form of that star system still survives today in the great celebrity stars, rock stars, and some of the great movie stars of today, but it's on very different scale.
And the number of such recognizable figures is much smaller today than it was in the heyday of the Hollywood era, when there were dozens and dozens of actors and actresses whose entire careers were known intimately by the audience and whose work from film to film was something that particular members of the audience would follow religiously, would follow as if they were members of a kind of fan club.
And then the second aspect of the star system was-- coupled to it and linked to it-- was what we might call a genre system.
What's a genre?
A category or a kind of story.
So the elegy is a poetic genre, and the western is a narrative genre.
And as soon as we mention western, or elegy, or situation comedy-- to talk about a dominant television genre-- as soon as we say that, a set of expectations immediately enters our mind, right?
We know that there are certain features that westerns have.
We immediately know when we hear the word western what historical era the story is set in.
We know what the costumes will be like.
We can predict what kinds of animals we're likely to see in the film.
We can even, if we know the western genre well, we could even predict pretty accurately what kind of story we're likely to be told, and what the rhythms or trajectory of that story might be.
Not because we know the particular western, but because we know the category.
And every western that calls itself a western is in some sense in conversation with every previous western.
What we're talking about here, in part, is a new form of literacy.
Or at least not exactly a new form, but a central form of literacy.
A form of literacy that emerges in popular art when a mass audience becomes more and more knowledgeable about the conventions, and operations, and formulas on which the system depends.
By the middle of the 1930s, virtually the whole history of the silent era and all the new genres that were elaborated and fulfilling themselves in the sound era, were now intimately known by segments, if not the entire American audience.
So to come back to my question, why would a system that emphasized the repetition of stars and the repetition of genres across experiences be a particularly appropriate or useful one for the kind of system I've described?
What's the answer?
If it's a mass production system.
The alternative would be, to clarify my question, why would we not have a system in which every movie was a unique experience?
Did not make gestures toward a whole class of other stories to which it belongs in, which it wants the audience to compare it.
Or that a series of stars would emerge, so that every time you see a Clark Gable movie, or a Claudette Colbert movie, or a Barbara Stanwyck movie, you associate those performances in those films with all the earlier performances in which you've see-- why does that system work?
What's the answer?
All right.
That's right.
Because it familiar, you know what to expect.
It's a branding.
And in fact, if you think about it, if you want people to shell out money every single week, you can understand why the notion that they should be getting something familiar might work, might be important.
It's possible that you could get people to spend all of a significant amount of money, or at least a small amount of money-- in the early days admissions we're not that expensive-- for one experience, but if you expect them to do it week after week after week, it's natural that you would expect that the system that would develop would offer the audience certain familiar landmarks.
In other words, the answer-- and I think it's a very good answer-- the answer is that a mass production system lends itself to principles of repetition and economies of scale.
What's an obvious thing, if you're a studio and you make a lot of Westerns, how can you save money?
Do you have to hire the horses every time?
Maybe you have your own stable.
Not only that, maybe if you have a scene of a stampede, you could use it in more than one movie, you don't have to film it again.
And in fact, a lot of early Westerns have stock footage that were put in.
They would film a tremendously exciting cattle stampede and they would use it in four or five different movies that needed stampedes.
The same with the sets.
They don't have to tear down the Western town and rebuild it for every movie.
So in other words, one could say that there are economic incentives that encourage this system.
But this is another example of that magnificent paradox that I spoke to you about earlier in the term, the paradox that the crassest of alliances between commerce and technology should be the enabling conditions of a narrative art.
Why?
Because even though the origins of these strategies may very well have been totally commercial, as they begin to elaborate themselves, they inevitably become more complex.
The audiences begin to generate expectations.
The people who are repeating themselves begin to repeat themselves with variation.
Even if they never intended to get better, they would get better.
And of course some of them are excellent professionals and they do want to do a better and better job.
So what happens is the system begins to refine itself, even if it begins in the crassest of intentions.
Figure out a way to save money and make as much money as we can, by giving the audience the same thing again and again.
If the audience would buy the same story over and over again, no doubt Hollywood would give it to them.
But of course, they have to introduce variations just to make a living.
Just to make people come back.
And then as these variations begin to elaborate themselves, something complex and rich begins to emerge.
And this knowledge that the audience has becomes what we might call a form of literacy, a form of popular literacy.
So there's much more one could say about this, and I'm being very abrupt and overly brief about the way this system elaborated.
Suffice it to say that there emerged in this era a series of recognizable separate categories of movie making.
Westerns, gangster films, various forms of comedy, various forms of social realism, certain forms of historical spectacle.
And other smaller genres like the newspaper picture, which is a kind of smaller sub genre.
We're going to see one example of that tonight.
And so that what I'm trying to do over the next couple of weeks, and especially in this week, is trying to demonstrate these principles in small, by talking today especially about certain aspects of screwball comedy.
And by placing that particular central Hollywood genre in the larger context of this manufacturing system, this mythological system that we've been describing.
One way of explaining what I'm trying to get out when I talk about these stars and genres, and the purely commercial origins of what becomes a rich set of narrative expectations, is to say that what I'm really implicitly dealing with is a problem.
It will seem less of a problem to your generation than it did to generations earlier.
But it took a long time before the American Society began to recognize that the movies were a signature art form of American society, and that if American society was going to be remembered for its artistic achievements in the 20th century, the movies were going to be one of the prime instances of its artistic identity as a society.
It took a long time for Americans to recognize this.
In fact, the Europeans recognized it before the Americans did.
And one of the fundamental reasons for that is that our ideas about what art is were inherited notions that came essentially from the traditions of romanticism and modernism, which lead tremendous emphasis on pure originality.
On the idea that the work of art has to be sui generis, unique to itself.
That it has to grow out only of the individual genius of the artist.
Well, this notion of art is actually a very time-bound one.
It's not really a universal notion of art at all.
The notion of art that I want to suggest to you was a more populist one.
It goes back to Homer and to the various forms of Epic Performance, and this notion of artisan work-- a communal one, in which the artist is a singer or a speaker to the main parts of the community, to the whole of the community, or to a large part of the community.
And in order to do that kind of singing, to do that kind of address, the singer, the poet, the storyteller needs to use the categories and the story forms, and the language that is known by the majority of the community.
The aesthetic for this kind of art is an aesthetic of familiarity, in which meaning and significance is generated not by a shock of total recognition of everything in the text being new or innovative, but by something else.
By the fact that certain very familiar strategies are suddenly very, very slightly.
And it's the slight variation from the familiar that generates the significance.
And I'm going to try to demonstrate these principles.
In fact, all of the clips and films that you're going to see over the next few weeks-- and especially in today's viewing and lectures-- are embodiments or instances of this principle, as I hope you will see.
What we're talking about here, to borrow a phrase from the film scholar Leo Brody, whose essay An Aesthetics of Connection is required reading in the course, we might say that what we're trying to identify here is an aesthetics of connection, in which the links and connections amongst different texts or amongst different performances, are emphasized.
In which the familiar is a central element in artistic experience.
Most people love to recognize the familiar.
It is not the case that art always must continually surprise you.
In fact, art that is constantly surprising, that makes no assumptions that you can share, or that allude to or refer to earlier forms of the experience, probably such a text doesn't even exist.
Even if the allusion or reference to ancestors is one of antagonism in which the text is saying look how different I am from my ancestors, it still needs its ancestors in some way.
So the notion that a text can be entirely creatively original is itself a kind of modernist and romantic fallacy.
In any case, the artistic principles on which the Hollywood film is based are artistic principles we could identify as populist or democratic.
Democratist.
And they're based on the idea that the familiar and the every day can be transmuted into something more complex by good performances, by good writing, by the creation of a compelling story.
And that meaning is created not by a single unique creative act, but essentially by a conversation on communal process, in which a particular perception or sense of the world is juxtaposed against older instances of the same thing.
And by comparison, one learns something about where one is in the particular new text.
An Aesthetic of Connection.
It's one measure of the complexity and social centrality of the movies in the 1930s, '40s, and early '50s, that we can find in the system three separate kinds of comedy.
It's a measure of its maturity, by not just one but three fairly robust separate strains of comedy.
And this even excludes musical comedy, the musical film, which could be said to be even yet a fourth strain.
Not just one form then, but three.
And the form of comedy we're focusing on today is screwball comedy, but I want to say a word about the other two strains to give you at least some sense of where we're coming from.
The first significant strain that I would like to mention to you is what I call anarchic comedy.
It mobilizes something that's elemental.
An elemental impulse in many comedies and maybe in the comic impulse itself at some very deep level, which is an impulse toward anarchic destruction.
There's always, hidden in comedy, at least a kind of skepticism or suspicion or hostility toward the established and the respectable.
And one form that that can take in comedies is a kind of systematic anarchy, in which the large respected structures of a culture are systematically-- or sometimes unsystematically-- disabled or dismembered.
And over and over again, in the comedies of the Marx brothers, something like this happens.
I won't mention titles, but there are a number of titles-- I think I mentioned one of them last week or the week before-- A Night at the Opera, which concludes with the entire set of the opera falling apart, breaking down.
And that kind of destructive-- there's another film that the Marx brothers made in which they are on a football team, and we see one of the things that Groucho does in that film is he gets the ball and he runs the wrong way.
And he scores a touchdown for the other team, but then he keeps running, and he starts running out of the stadium.
It was a sense that, the rules of this games are silly, let's ignore them.
And so this sense of the dissolution of boundaries and the anarchic chaos that follows from the breaking down of established categories and edifices is part of the pleasure of these Marx brothers' comedy.
And it even extends, in some way, to a kind of deconstructive attitude toward language itself.
And many times the dialogue in a Marx brothers film will go nowhere, it will be able end up making no sense.
So even language itself is reduced to a kind of anarchic nonsense at times, in the Marx brothers films.
An even cruder version of this would be in The Three Stooges.
The Three Stooges are a version of this kind of anarchic comedy.
And there's a final version I should mention, a rather distinguished one in it's own way, films that are associated with the actor W.C. Fields, a gravelly voiced comedian who often portrayed people who were half drunk and were very hostile toward children, very anti-sentimental.
He would kick children, or spit at them, act as if children were smelly, annoying things who should get out of his way so we could keep drinking his scotch, that sort of thing.
So there was both an anarchic and a kind of subversive element to this kind of comedy.
A second strain of comedy, even more powerful and artistically more valuable.
A very, very interesting one, which continues to be influential.
Films from this tradition have been remade recently in American cinema.
And this is the strain of comedy we would call worldly, that other scholars have called, it's not my label.
And one thing that the label intends to acknowledge is that many of these directors in this tradition were Europeans.
And they brought a European attitude, especially toward sexuality.
They were much more amused than shocked by sexual shenanigans.
So one measure of worldly comedy was it had a more generous and a less puritanical attitude toward infidelity, toward the body itself, toward sexual allusion, and sexual by-play.
They still were not explicit by today's standards.
You didn't see nudity in these films.
But they were more open about the fact that men and women actually engage in that behavior, and even sometimes take pleasure in it.
An acknowledgment that was more difficult for non-worldly films to make.
And the central director of this tradition, although there were others, was Ernst Lubitsch.
I put his name up on the board there.
His dates are 1892 to 1947, and he made a series of films, especially in the '30s through the early '40s, that are still regarded as classics of this kind of worldly comedy.
Trouble In Paradise, Design For Living, a film called Ninotchka in 1939.
A film called To Be or Not To Be in 1942, an immensely bold film in which Jack Benny the comedian plays an actor who impersonates Hitler.
The film is set in Berlin, and he actually plays as a double of Hitler.
A very bold film, on a par-- and intellectually, I think, and artistically an even more powerful film than Chaplin's The Great Dictator, although it shares the same ambition.
And these worldly comedies often told infidelity stories, or stories, as I've suggested, that took a much more benign and amused attitude toward our sexual crimes and misdemeanors.
Both these strains of comedy are robust and deeply popular in the era.
But the form of comedy that I want us to focus on, that we're centering on in this week, is the most important of all of these strains of comedy, it's what has come to be called screwball comedy.
And let me say a few things about screwball comedy as a category, describe it a little bit, and then I want to show you two clips that are intended to illustrate many of the principles I'm trying to articulate here.
And we'll come back to this idea of star and genre in these clips as well because those elements are also at issue or embedded in what I want you to look at.
Maybe I should've mentioned, if you were curious about the later incarnations of worldly comedy, I meant to mention that for example in 1998, Nora Ephron made a film called You've Got Mail.
How many of you have seen it?
At least a couple.
That's a remake of one of Lubitsch's films.
And in fact, the Lubitsch film, A Shop Around The Corner, is a much more wonderful film.
And it's a measure, I think, in part of the power and authority of Lubitsch's imagination and the elegance of his comedies that they continue to inspire imitation in that way.
What screwball comedy is, in some ways, a distinctive, a signature form of American movies.
It's a unique equation of Hollywood in the depression years.
It's derived in part from Broadway farces of the 1920's, and also from the slap stick physical comedy of the silent era.
It's a kind of marriage or hybrid.
And remember-- we've already said this but let me remind you-- one measure of a mature medium or of a medium that's coming into maturity-- an expressive medium, a communications medium-- one measure of its maturity or of its emerging maturity is that forms that used to be thought to be separate, marry.
Become hybridized.
One of the things I was suggesting about Chaplin's work is it suggests a new maturity in the film, in the cinema as an institution because it's marrying comedy and melodrama, because Chaplin bring seriousness into comedy.
He slows it down, it's not merely just slapstick anymore, it's also psychologically resonant.
Not only can it make you laugh, it can move you.
That marriage or hybridization of two essentially separate forms-- melodrama, comedy-- makes for a richer film.
I'm saying the same kind of thing about this particular form of comedy, this merging of ancestor forms, of the slapstick comedy of the silent era-- and of course the slapstick comedy of the silent era has its ancestors in stage slapstick-- and fast dialogue and witty repartee that was especially characteristic of a series of Broadway farces, made in 19-teens and the 1920's.
But the particular historical moment in which this form emerges is also a shaping factor in the kind of comedy and the subject matter of these comedies.
The key qualities of screwball comedy were a kind of irreverent humor, something of the anarchic subversive skepticism that's characteristic of many comic perspectives, coupled with very fast paced vernacular dialogue.
One way to think about screwball comedy is to say it's one of the first forms, along with singing movies, movies that have singing in-- I don't want to call the musicals yet because sometimes they weren't fully musicals, although musicals do emerge at a very early time-- but any movie that had singing in it, like some westerns.
Early western's had singing in it.
Why?
Because sound had just come in, it was a novelty.
The screwball comedy is one of the forms that exploits sound in a really expressive and powerful way, compelling way, because it's full of dialogue.
It's full of witty, rapid dialogue, most often arguments or conflicts or quarrels between men and women, dramatizing certain kinds of conflicts of gender in the course of the film.
So, key qualities.
Irreverent humor, a kind of vernacular fast-paced dialogue, a fast physical pace as well, full of slapstick.
So a physical quick pace and a verbal quick pace that are often in competition with each other to see which can be more irreverent and quick and fast and abrupt.
Eccentric, often high born characters, who are made fun of and mocked.
A lot of the screwball comedy deals with the upper social orders, but they're seen as clownish or foolish in some way.
And in many screwball comedies, the wealthy characters need to be rescued by their servants.
It's a very modest form of class warfare, in which-- again, remember it's the American capitalist system that's making these stories, so we're not going to have a Marxist comedy that says that the government should be overthrown by violence-- but the comedy is constrained in certain ways, what the story's able to say, what the films are able to say.
This is a matter to which I'll return tonight and talk a little bit more fully about the ideological and political limits that are imposed on the Hollywood system because it has to appeal to everyone.
I think what I have to say will clarify these questions in ways that you'll find very helpful.
But for now, let's stick with screwball comedy itself.
These high born characters then, they're criticized and seen as clownish, but they're never seen as morally evil in the way that a Marxist or a social historian might regard them.
The stories are often very deeply improbable ones that are set among the irresponsible wealthy, the irresponsibly wealthy people.
Often the aristocrats are very spoiled and hard drinking.
In one of the most famous of these, a film made in 1937 titled My Man Godfrey, directed by Gregory La Cava, Carole Lombard and William Powell, two of the central to screwball comedy performers, are the stars.
And there is a scene in a shantytown by the river, in which the poor are located.
And the film seems to make some sort of an acknowledgement of the fact that the poor are here.
It's acknowledging the horrors of the Depression.
But it solves the horrors of the Depression in the end by having the wealthy people build a nightclub on the river banks to create employment for all the unemployed.
And it's a very imposed-- and rather foolish in some sense-- happy ending.
So that film acknowledges more fully than some screwball comedies the actual existence of shanty towns and places where homeless people, or people who have to live under canvas live.
Many screwball comedies didn't even go that far, but it solves the social problem in a characteristically mythological and magical, socially incredible way.
Another aspect of these films is that they foregrounded women, and this is a very important aspect of it.
This is partly a function of the fact that in the 1920s, the era of the flappers, a new kind of sexually liberated woman begins to appear.
Some of that is an outgrowth of the fact that, during the First World War when so many American men were in uniform, women got new jobs and were able to work, enter the workforce for the first time.
And that created moments of freedom for women that had never existed in American society before.
And some of that is reflected in screwball comedy, and in other films as well, but especially in the screwball comedies of the era.
So it foregrounds women.
And it shows women who often have a great deal more energy and freedom than the females in other films of the era.
Very often in these films, marriage is seen for the first time as a partnership of equals.
And as you'll see in It Happened One Night, that's the ideal of marriage that's dramatized in that film, one of the two screwball comedies we'll be looking at this evening.
So these films also reflected the social disorientation of the '30s, the uncertain values that were represented during the Depression.
The fear, in some sense, that the culture was breaking down or that the fabric of society was tearing in some ways.
And so these films were ironically simultaneously escapist, because they were screwball comedies and they did zany things, and they often had ridiculous or improbable plots and improbable characters, and yet at the same time the anxiety that they also dramatize, the anxiety of things falling apart, of the center not holding, of a kind danger to the established structures and belief systems of the society, underlay even the most apparently escapist of these films.
So there's a subtext of strong women, of quarreling and manipulative men.
And especially, of course, one of the things this dramatized for us were intelligent and witty women, like Katharine Hepburn in Bringing Up Baby, or Claudette Colbert in the film you're going to see tonight, It Happened One Night, or Rosalind Russell, in the second screwball comedy you're going to see tonight, His Girl Friday.
And then finally, implicit in what I've been saying is that there are implicit in these stories, sometimes rising to the surface, sometimes just beneath the surface, but there are as a kind of hint or anxiety, are what we might call first some form of class conflict.
Some sense that the class structure of the United States is a problem, that the Depression is causing difficulties.
And second, a form of sexual warfare, in which the inequality between men and women, and the unequal relations that men and women have in the society are put in question, are quarreled with.
In some sense, even mocked.
And in many of these films, the actress is the dominant figure and the male character is the secondary figure.
And even in some of the films where the male characters are even more powerful, the best one can say is that they're equals.
And that in itself, regardless of what the plot does, was in some sense, not exactly revolutionary, but transformative.
Potentially transformative.
Well, what I want to do with the time I have left is show you two clips, and they're intended to dramatize many of the principles I've been talking about.
So let me rehearse them very quickly to you.
I'm going to show you two clips from two separate screwball comedies.
One, directed by Preston Sturges in 1941, called The Lady Eve.
A second, directed by Howard Hawks, the director of the second film we're going to see tonight, Ball of Fire.
Hawks is probably, of all the directors, the one most identified with screwball comedy, although as you'll see tonight when I talk a bit more about him, he was a master of other genres as well.
But the reason I want you look at these two clips is-- I have multiple reasons.
It's the multiplicity principle in criticism.
What I want you to recognize as you're looking at this, is first the power of the star system.
Because here's Barbara Stanwyck.
At this point in her career, she's one of the dominant female stars in the society, one of the biggest moneymakers in the society.
And here we see her in two films that are in the same genre.
Two different versions of screwball comedy.
As you're watching, think about the way in which the persona that she established, the kind of character she stood for in film to film, is both stable but also changing.
That is to say, the pleasure you take in watching the two different versions of Stanwyck is an embodiment of the principle I was talking about before, when I said that we need an aesthetic of connection.
We need to recognize that it's the familiar, rather than the completely strange and new, that can generate profound, complex sentence of artistic achievement.
Because the Stanwyck we see in these two films is in many ways a radically different character, but we can also see certain continuities across her characters.
And we can take pleasure both in what is the same about her, but even more we can take pleasure in the variations she's working on this persona.
So one of things that is embedded here is a notion of acting that emerges in the studio era that establishes American acting as something different from the classical British theory of what an actor is.
The classical British theory of what an actor is is that the actor submerges herself in her role and becomes indistinguishable from the role.
And that you shouldn't even be able to recognize the actor from role to role.
The American system that emerges from Hollywood and from movies is an opposite one, in which you are not only are you supposed to recognize the actor, you're supposed to take pleasure in the way the actor-- it's not really Barbara Stanwyck but the character that Stanwyck plays, the persona Stanwyck established is in film after film after film, how does that persona evolve?
Change in nuances, when that persona is presented with different sets of questions, different sets of problems.
And what you can feel-- and this isn't just true of the actresses of course, it's even truer for the male stars who had more freedom, there were more rules available to them.
So people who are interested in Clark Gable or Humphrey Bogart could watch these actors across a range of films, each time not getting the same experience, but watching a variation on the persona that the actor was setting up.
So a new idea of acting, a different conception of what an actors job might be and of how the audience might have a kind of literacy in performance that would enhance and complicate their sense of any individual performance, emerges in this era as well.
And the Stanwyck clips demonstrate that.
The second thing I want you to see, again, same principle, having to do with screwball comedy itself.
Here are two different screwball comedies, look at the differences in tone.
They belong to the same genre, we can feel how much they belong to the same category, but also how much variation there is between them.
So the point is, it isn't a simple form of repetition at all.
It's a form of very complex elaboration and complication, in which the relation between one text in the category and another text is far more complex, artistically satisfying, and rich than we might imagine if we described this system in too abstract a way.
So here's the first scene from The Lady Eve.
And let me set the scene.
In this scene, Barbara Stanwyck plays a con lady.
She's on a ship loaded with wealthy aristocrats, wealthy American businessmen, and she and her partner are trying to goal them out of money.
And she's sitting at dinner trying to pick out a guy that is going to become her mark, played by the great American comic and serious actor Henry Fonda.
[VIDEO PLAYBACK]
He's very wealthy.
Everybody knows he's very wealthy.
He's the son of a beer magnate, an ale magnate.
He's sitting down in the dining hall of this cruise ship.
And all the fortune hunting women who are there wanting to marry rich men are staring at him.
-Not good enough.
-What did you say?
-I said they're not good enough for him.
Every Jane is giving him the thermometer, and he feels they're just a waste of time.
He's returning to his book, he's deeply immersed in it.
He sees no one except-- watch his head turn when that kid goes by.
Won't do you any good, dear.
He's a bookworm, but swing them anyway.
Oh, now how about this one?
How would you like that hanging on your Christmas tree?
Oh, you wouldn't?
Well, what is your weakness brother?
So Stanwyck has become the narrator of this scene.
But see what intelligence she shows, what knowledge.
-Look at that girl over to his left.
Look over to your left, bookworm.
There's a girl pining for you.
A little further.
Just a little further.
There.
Wasn't that worth looking for?
See those nice store teeth, all beaming at you.
Well, she recognises you.
She up, she's down, she can't make up her mind.
She's up again
I hope you see how brilliant this is because look in her hand here, we're looking at this through the mirror as she's narrating it to us.
-Went to manual training school with in Louisville?
Oh, you're not?
Well, you certainly look exactly like him, it's certainly a remarkable resemblance.
But if you're not going to ask me to sit down, I suppose you're not going to ask me to sit down.
I'm very sorry, I certainly hope I haven't caused you any embarrassment, you so-and-so.
I wonder if my tie's on straight.
I certainly upset them, don't I?
Now who else is after me.
Ah, the lady champion wrestler.
Wouldn't she make a houseful?
Oh, you don't like her either?
Well, what are going to do about it?
Oh, you just can't stand it anymore.
You're leaving, these women don't give you a moment's peace, do they?
Well, go ahead.
Go sulk in your cabin.
Go soak your head and see if I care.
-Very sorry, sir.
-Why don't you look where you're going?
-Why don't I look?
-Look what you did to my shoe, you knocked the heel off.
-Oh, I did?
Well, I'm certainly sorry.
-You did, and you can just take me right down to my cabin for another pair of slippers.
-Oh, well certainly.
It's the least I can do.
By the way, my name's Pike.
-Oh, everybody knows that.
Nobody's talking about anything else.
This is my father Colonel Harrington.
My name is Gene, it's really Eugenia.
Come on.
[END PLAYBACK]
I hope you recognize the tremendous intelligence and wit that was embedded in that dialogue.
That was a little too low.
What is revealed about that character, dramatized about that character, in that opening scene, tells us almost everything we need to know about her.
Not everything, but almost everything.
And you can feel the energy, the wit.
And of course, who's in control of that scene?
Not the man, but the woman.
Now let's look at another scene from Howard Hawks film, Ball of Fire.
The basic situation is Gary Cooper, the man in the middle here, this fellow Gary Cooper is the head of a consortium of scholars who are compiling a dictionary of the English language, with special emphasis on colloquial or slang English.
And earlier in the film, Gary Cooper went to a nightclub.
And he saw the Barbara Stanwyck character, who is actually the mistress of a mob boss, do a song and dance number.
He spoke to her, he interviewed her, and he became very excited by what she represented.
By the linguistic resources, the verbal realities of the American language that he'd been oblivious to.
And he wants her to contribute to his project.
She's not interested.
When she shows up at the lair of the men of the scholars, when she invades the nest of scholars, she does it because-- although you don't know that from seeing this clip, but viewers of the film would know it-- because she's really on the run.
She's afraid that her boyfriend, the mob boss, is going to bump her off.
So she needs to hide out.
And that's really why she's come here.
All right.
So here is this scene from Ball of Fire.
[VIDEO PLAYBACK] -Have I missed anything?
-No, I was just about to explain.
You see the word puss means face.
As for instance, sour puss, pickle puss.
Sugar puss implies a certain sweetness in her appearance.
-Never mind the etymology, was she-- -Was she blonde or brunette?
-Yes, which?
-That I don't know, I didn't notice.
But her vocabulary, even an ordinary conversation-- -Oh, you spoke to her?
-Yes.
In her dressing room-- -In her dressing room?
-Backstage?
-Yes.
Unfortunately, she disclaimed any interest in our project, in words so bizarre they made my mouth water.
Shove in your clutch, for instance.
-It's amazing.
-Potts, could you tell us, what was it like backstage?
-Very vivacious, I imagine.
-Perhaps ballerinas giggling up and down iron stair cases?
-Possibly wearing tights?
I hope you're picking up the subtext that these old men are revealing.
-It's getting a bit late gentlemen.
Perhaps we had better get to bed.
-Someday I'll tell you my experiences with a young actress named Lillian Russel.
-Did you know her?
-I stood in the snow four hours to get-- -Is that our doorbell?
But it's 12:25!
-Oh, must be the statistics on San Salvador saltpeter.
I asked for it to be sent--
Did anyone pick up on the joke there?
Why is there a joke in reference to saltpeter?
It's used in the army to keep men from getting too horny.
It damps down your sexual energies, and he's writing an article about saltpeter.
-Hi-DE-ho!
Don't tell me I'm too late for class.
-Oh my goodness gracious me.
-What is that?
I don't know what scholar came up with this brilliant description, but there's a kind of convention now among Capra's fans, and fans of this movie, to refer to this scene as the Snow White and Seven Dwarfs scene.
-Those are my colleagues.
-Oh!
-I must apologize for their lack of costume.
-Oh, that's all right professor.
-I haven't got my tie on.
-Oh, you know once I watched my big brother shave.
-Would you come in?
-Why not?
-Frankly, your coming here was the last thing I expected.
Your no was so explicit.
-Well, I got thinking it over, and I said to myself, who am I to give science the brush?
-And I take it you've reconsidered?
-Yeah, that's the big idea.
-Oh dear, oh dear.
-Look out.
-That was Professor Oddly.
-Anymore of them around?
-I hope not.
-Hey, who decorated this place?
The mug that shot Lincoln?
-This is our work room.
The living quarters are upstairs.
-Whee, that's a lot of books.
All of them different?
-I trust.
May I have your coat?
-Yeah, thanks.
Oh, Greek philosophy.
I've got a set like this with a radio inside.
-Are you sure you don't want your coat?
-No, I'm fine.
Except I've got a run in this stocking.
Well how do we start, professor?
You see, this is the first time anybody has moved in on my brain Have you got some kind of a machine, an x-ray or a vacuum cleaner maybe that's out the words you want?
What's your method, professor?
-Well, it's quite simple.
If you'll be here tomorrow morning, not later than 9:30-- -Tomorrow morning?
-Well, yes.
I've arranged a roundtable discussion with a few people of various backgrounds.
-You, you don't think we could begin the begin right now?
-Well, it's nearly one o'clock Miss-- -Oh, fool professor.
Let's get ourselves a couple of drinks, light the fire maybe, and you can start working on me right way.
-I wouldn't think of imposing upon you at this hour.
-Listen, I figured I'm working all night.
-Any hasty, random discussion would be of no scientific value.
You see, I have to have my notes thoroughly prepared for the seminar tomorrow.
-OK. Where do I sleep?
-I don't know, where do you live?
-Up on riverside, but I'm going to sleep here.
-Here?
Oh, you don't understand Ms. O'Shea.
We're all bachelors, with the exception of Professor Oddly who is a widower.
Why, no woman ever-- even Ms Bragg, who takes care of our needs-- goes home every night at 7:30.
-If you want me tomorrow morning at 9:30-- -Oh, I do Ms. O'Shea.
But even the most free-thinking people must respect the-- -All right.
Feel that.
Go on, feel that foot.
OK, tootsie bell, what do you say?
-It's cold.
-It's cold and it's wet.
Now, come here.
Come here.
Closer.
Closer.
Oh, come on, give.
Hello kids.
Look down my throat.
Go on, look down.
All right.
[MUMBLING] -I don't know what to look for.
-There is possibly a slight rosiness in the laryngeal region.
-Slight rosiness?
It is as red as the Daily Worker and just as sore.
Who are you?
The Daily Worker.
Red as the Daily Worker.
Did you get that?
No?
The Daily Worker is a communist newspaper, so when she says "it's as red as The Daily Worker, she's making a rather witty joke, especially for a gunman's maul.
-Oh.
-Professor Robinson, Law.
Professor-- -Ah, ah, ah.
Not so fast.
Just let it creep up on me.
I'll get to know them.
Come here physiology, for all I know I've got a fever.
Feel.
-It's possible.
-Certainly.
And he wants to throw me out on my tin.
There'll be no 9:30 for me if you make me go out in the rain now.
-Naturally not.
-With the streets cold, and the subway hot and full of germs?
-Oh.
I'm a pushover for streptococcus.
Can I have this now, kid?
-We'll call you a taxi, and furnish you with woolen socks and warm slippers.
-Really, I don't understand you Potts.
Why take chance with valuable material?
-Think of your article, Potts.
-See.
They get the point.
-If I might venture a suggestion, why couldn't the young lady sleep in my room?
-Professor Peagram!
-I can bunk in with Professor Robinson, I sometimes do when there's an electric storm.
-Yes, he's afraid of thunder.
-Well then, it's all settled.
Well, I guess I'll turn in.
Can I have my coat?
-Thank you.
Hi-DE-ho, fella!
-I'll show you to my room.
-Yes, we all will.
-I know where my own room is, thank you, without any help from you.
-I'll find it, don't bother.
Just rough out the directions.
-The top of the stairs, and the third door on the left.
-Gentlemen.
Just a moment, please.
Gentlemen, this is all highly irregular.
What if this should come to the attention of the foundation?
And what about Ms. Bragg tomorrow?
-What are you talking about?
This is research, isn't it?
-Yes.
-Certainly.
Who was that guy that learned so much from watching an apple drop?
-Isaac Newton, 1642 to 1727, the law of gravity.
Yeah, that's him.
And I want you to look at me as another apple, Professor Potts.
Just another apple.
[END PLAYBACK]
All right.
So you can get an idea not only for how richly comic these films can be, but also how rich the subtext, the sexual subtext can become.
Even though, of course, as you all should know if you've been reading Cook, there was a production code in force after 1934, and there were all kinds of very elaborate rules imposed on American movies.
A form of self censorship because the movies feared censorship from the government, so they came up with their own system.
And until sometime in the, I guess it was the 1960's, the code held and it had all kinds of ridiculous elements in it.
One of the primary ones that people always laugh about, rightly I think, is that if it showed a woman on a bed, at least one of her feet had to be on the floor.
You couldn't show a woman lying with both feet on the bed, it was too suggestive.
But you can see how clever filmmakers and powerful performers were able to make a mockery of those kinds of constraints, because subtle and complex and witty films are able to do many, many things that are in a suggestive way, that keep them from crossing the boundaries set up by the censors, but nonetheless have a powerful energy and sexuality.
So the similarities between those two scenes, but also the differences between them, tell you something about the way in which the aesthetic of connection, this aesthetic of familiarity, this popular aesthetic operated in Hollywood movies.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
The aesthetic of connection, this aesthetic of familiarity, this popular aesthetic operated in Hollywood movies.
And I want to expand a bit on this idea by talking about more explicitly what might be called an anthropological view of this matter, what could be called the cultural work of movies.
One way I can make you think about the larger and more significant implications of what the movies are, of how to think about the movies, is to ask you to think about the true implications of certain words that we just take for granted.
For example, take the word entertainment or the verb entertain, to entertain.
We think of it as-- we think of it as a relatively straightforward term, as it is.
And we think of entertainment as some form of diversion that takes us away from the serious things in life.
But I want to suggest to you that if we look closely, even if we look at sort of the root of the word and the way the word is used, and the variations on the kinds of meanings that are associated with the word entertainment or the verb to entertain, we can see deeper implications.
For one thing, for example, let's just take a standard dictionary definition.
There are three different senses in which almost all dictionaries in English would iden-- would define the verb to entertain.
They would say, to hold the attention of someone is to entertain them.
A second meaning would be to extend hospitality to someone.
That makes sense, right?
But think when you extend hospi-- you're engaged in a communal, in a hostly act, right?
It's an act of generosity, an act of inclusion, right?
So forms of entertainment are potentially community creating, gestures towards a community, acts of hosting, right?
And they also, of course, are a bid for your attention.
And then there's a third meaning, which is the most interesting.
To entertain, almost all dictionaries will tell you also means, to hold in the mind.
I'm entertaining this idea.
I'm entertaining this thought.
I'm entertaining this possibility.
In other words, the word enter-- the verb to entertain means to meditate, to think, to speculate, to consider possibilities, right?
And when you attend an entertainment in a certain sense, that's exactly what happens.
Exactly because the space of entertainment is marked out by the society, by social convention, as a place of diversion, as a place of relaxation, as a place that is theoretically not a place of work.
Certain things are licensed or permitted because of that, or encouraged.
Because you know you're kicking back, it's not a serious place, what is likely to happen?
You're allowed, or at least by implication, or permitted to entertainment possibilities that in actuality might be scary or disturbing or unacceptable.
So that one of the ironies of entertainment is that it can become a space, especially public forms of entertainment in cultures, like Shakespeare's public theater or likely public theater of the American movie system in the studio era, these public spaces can become spaces in which the body politic, the political and social community, entertains ideas about its own nature.
Entertains, considers, speculates, holds in its mind accounts of its origins, stories of its values, right?
And what we can say, then, is that the space of entertainment becomes from, in a certain angle, in virtually all societies, a space of discourse, a space in which exactly because it's a space recognized as not real, as make believe, is therefore licensed or allowed to explore possibilities that might be too dangerous or too disturbing to explore in other ways.
And we can see that certain features of the entertainment systems of most countries and of most societies, and certainly the entertainment system we're calling the Hollywood studio movie, are systems in which other features of the-- other features of the formats and organizational principles of the films reinforce this idea that you're escaping into a space that is also a space of exploration and discovery.
That you're not escaping and running away exactly into forms of diversion and hiding from the world, but in fact, you're escaping into a space of freedom in which you can consider possibilities that might otherwise be too disturbing to think about.
And one way you can see how the formats encourage that comes back to the argument I was making about genres and stars.
Because think about, one of the most fundamental things about the conventions of genre are that audiences understand not only sort of the costumes and the setting and so forth, but also usually understand the basic-- what do we call it-- the basic segments, a story force, the segmentation of the story, the stages through which the story is likely to go.
And they're likely, in many cases, to have a pretty good idea about the nature of the ending of a particular genre form.
They may not be absolutely certain about the details of how it will work out, but they have some pretty good ideas about-- for example, the notion is that various forms of comedy end on some sort of an affirming or a happy note, even if the difficulties that are dramatized in the text from the beginning are very serious and troubling, right?
Not that every single comedy ends on a totally happy note, but that's the expectation you have when you come to see a comedy.
You expect an adventure with many difficulties that are-- that will finally be resolved in some form.
And in fact, traditionally, one of the most fundamental ways in which comic narratives and comic dramas are resolved is in the act of marriage.
Because of course, marriage is a kind of reconciliation of potential opposites.
It is a space of celebration in which different families and different groups and, in many forms of comedy, different social classes are brought together.
So that the marriage itself becomes a social space that replicates, if it's dramatized on stage or in the movie, it replicates in some sense the communal space of sharing that the audience is engaged in when it's watching the text, right?
So you have expectations about the characters, about the plots, about the basic formats, and especially about the endings of genre forms.
Now what follows from the fact that you know that genre forms usually end happily?
And indeed, that most-- that most Hollywood movies, leave aside the question of comedy, will end on a reassuring, or if not a happy, at least a reassuring or a positive note.
You simply know that going in.
Even though that rule is sometimes violated, it's not usually violated.
And when it's violated, it's often violated in ways that are instructive and interesting.
But you have that expectation.
Well, the expectation that when you get into the story, the expectation you have as an audience member when you enter the story, knowing that you are going to come to a happy ending, what does that do?
It's a form of reassurance before you start.
There are theorists of folk tales and fairy tales who argue that that's one of the functions of fairy tales.
That when a folktale or a fairytale begins, once upon a time, many, many years ago, in a land far away, and the story ends, and they all lived happily ever after, what are the-- those are ritualized or conventional beginnings and endings.
What do they contain?
They may contain terror.
They may contain Little Red Riding Hood being raped or eaten alive, right?
But it's OK in a certain sense, because when you come to the ending, you know that you'll be reassured.
So because you know that the ending will reassure you, it enables an exploration of stuff that would be scary and frightening.
There are child psychologists, for example, who say that one of the important aspects of storytelling to children is precisely this.
That the ritualized reassurance of once upon a time and they all lived happily ever after are enabling conditions that permit the exploration, that permit the story to go into territories that would otherwise be frightening and terrifying.
What I'm suggesting to you is that that's a way of understanding adult entertainment as well.
And over and over again, the Hollywood film will prove this principle out.
It will explore disturbing and frightening materials, but in ways that are some sense legitimated or permitted by the very fact that it belongs in an entertainment space, by the very fact that it marks itself as something that is a diversion, a mere entertainment.
But I'd like to eliminate the word mere because it's an inadequate and derogatory way of describing an experience that often can be among the most creative and valuable that people in a society can have, right?
So one way then of crystallizing all of this is to say that the Hollywood film for me, in the studio era especially, represents a particular form of what I would call consensus narrative.
What I mean by this term is something rather elaborate, but I think very helpful.
What I mean by the term is this, that in most societies-- perhaps not in all, but in most societies-- all societies have many different sites of storytelling, right?
People tell jokes to each other.
We go to the opera.
We watch-- we play video games that are deep forms of narrative today, right?
So that's the coming-- probably the new form of narrative that's emerging in our society today, are video games.
We hear stories in theaters.
We can hear stories in popular songs.
We read stories in novels, right?
There are stories all over.
And in fact, we get non-fictional stories out of our newspapers and from our politicians.
These are usually even more outlandish, ridiculous, and false than the fictional stories that we're fed, although we often don't recognize that, right?
So we live in a sea, an ocean of narratives, right?
All societies have many different sites of it.
But in many societies, if not all, there usually is a single site that by general acceptance or consent is recognized as the space in which the stories that are being told are intended for everyone, right?
The difference between what goes on in a wrestling match or in a baseball game, also forms of narrative in a way, or in an opera or in a theatrical piece and a movie of the Hollywood era, or a television program in the great era of television before, say, 2000 when new systems so atomized and divided the television system that it began to-- it began to imitate what happened to its ancestors, the films, a generation earlier, losing its hold on the majority of the population-- the idea is then that the idea of a consensus story is the story-- it's the story form or the story system that the culture recognizes as the one that reaches the largest number of people.
And that by common agreement or consent-- not by law, but by something much richer and larger than that, by a kind of communal agreement, this space is recognized as the place in which the stories that are told are meant to appeal to the largest number of people, cutting across social class, cutting cross age and gender differences, cutting across ethnic differences.
Virtually all societies have some space in which some story system of one kind or another that wants to appeal to virtually everyone exists.
Now it's important to understand that I don't mean it's a real consensus, an authentic one.
That is to say, I don't mean that the stories that are told there really embody the values universally of everyone in the society.
Not at all.
Because these societies are limited in certain ways.
They have exclusions.
They have caste structures, and so on.
So the consensus narrative in 1920 was a consensus narrative that sort of saw African Americans as second class citizens, right?
There was a lot of prejudice then.
Or it certainly was a patriarchal consensus, which assumed that women belonged in the home, and so forth.
I don't mean that the consensus is universally embraced by everyone.
What I do mean, though, is that everyone in the society recognizes that this is the consensus.
And even the people who descend from it recognize that this is the space in which the central values of the society are articulated, dramatized, and diffused.
And what makes this space-- these spaces-- so that another example, I mentioned one earlier, in Shakespeare's day, the public theater of Shakespeare's time was the equivalent of what I'm saying Hollywood was in the studio era.
In the Middle Ages, it would be the Bible, right?
The central story promulgated through all-- virtually everyone in the society had-- not only had access to the Bible, when they went to church, they received further discussion and elaborate propaganda about how important and central the Bible was to their life.
So the consensus narrative system was the one the church sustained in the Middle Ages, right?
In fact, one of the interesting things is to watch in Western culture the way in which the consensus story becomes more and more secularized.
And not only secularized, not only freed from religious dispensation, but also freed, in some sense, in many societies from government control.
Right?
So in some societies, there is no such freedom.
So for example, the consensus narrative in the Soviet Union was the story that was approved by the government.
It was-- and in fact, probably the majority of the population dissented from that consensus, right?
So the consensus I'm talking about is always a constructed one, right?
But that construction is still interesting.
And this is one of the exam-- one of the reasons, for example, that today people will still scream bloody murder if they see that on television, the representation of minorities or women is prejudicial.
But you don't see anybody complaining about the fact that there are such representations in books or in theater any longer.
And the reason for that is that it's the consensus system that matters to people.
It's the consensus space in which it is assumed the values of the society can be challenged or discussed, OK?
So what I want to suggest, then, is we might think-- so we can think of the Hollywood film then, all Hollywood movies, as belonging-- as participating in the creation of what we can call the consensus story, the most widely shared story about American society.
And it has many sides, right?
Part of the story has to do with how you define masculinity, right?
And some of that burden is carried in every film, but there are certain movies like war movies or Westerns in which masculinity, tough guy-ness is the main issue.
And then there's some-- then we could say the norms about family life.
Where do they come in?
Well, more often in comedies and in melodramas that focus on families, right?
In other words, different parts of the consensus are dramatized more fully in different genres.
But all the films together can be said broadly to participate in a process whereby the central values and assumptions of the society are rehearsed and repeated and tested.
The central fact about this process is that it's never finished, just like culture itself.
Culture's never a fixed, finished thing.
This is one of the deep insights of modern sociology and anthropology.
I attribute it primarily-- specifically to the great English sociologist and historian Raymond Williams from whom I'm borrowing some of the ideas on-- although the word consensus narrative is mine, not his-- many of these ideas are drawn from Raymond Williams' account of culture.
What he says is essentially that culture is constantly making and remaking itself.
That every time you watch-- think about-- some of you can see this example more fully with television than with movies.
Remember, the argument of the course is that at a certain point, sometime in the 1950s or early '60s, television supplanted the movies as the central form of narrative in the American society.
Took over the job of being the consensus, the purveyor of the consensus, the purveyor of the culture's central values.
And in many ways, that was a liberating moment for the movies.
Our course dramatizes that.
This week, we're looking at two screwball comedies from inside the studio era.
Hereafter when we juxtapose two films, one will be from the studio era.
One will be from the era after the movies no longer carry the consensus, the responsibility to be the consensus medium.
That's been taken over by TV.
And the differences are dramatic.
The differences are stark and exciting.
Both forms are interesting.
The post-consensus forms have great claims on us because they're much freer.
They're not limited or constrained by what we might call the central pieties of the culture.
The most obvious way you could see this is in nudity, right?
They can show-- attitudes-- and dirty language, right?
Think about-- you can see the stupidity and the arbitrariness of these systems if you think about how television worked today.
If you put on an American network show, even a late night show by 10:00, 11 o'clock, at 9:00 or 10:00 a night, what you will get on those shows-- you might get things very different from what you would have gotten 10 or 15 years ago, but you still won't get full frontal nudity.
You won't get women taking off their clothes.
You won't see sexual activity.
You won't see very much dirty language.
You used to see none of it.
But if you click the channel one over and you go to HBO, you could see the most disgraceful pornography that you can ever imagine, right?
And you could hear people using the F word so often that it runs out of your ears, right?
And they're right next to each other on the dial.
What explains the difference?
HBO doesn't belong to the consensus system.
The networks are still a vestigial survival of the old consensus system.
And if you think about the difference between network television, even today, and the movies or network television and what's on HBO or Showtime, you will see this distinction with great clarity, right?
So one of the things that this means is that these systems of consensus narrative have a tremendous claim on our attention.
They're more important, in some sense, in a sociological or anthropological sense than other forms that don't bear the burden of having to tell the story, the consensus story of the society.
You remember I said culture's unfinished.
So are these stories, right?
The great thing about them is that they change, right?
That over time, they'll alter.
I'll come back to this question next time when we talk about the Western.
We talk about how the Western alters over time in ways that are tremendously responsive to problems in the outer society.
A very clear example of the way in which genre constraints can enable an exploration of serious subjects.
So we'll return to that next time.
For our purposes tonight, the primary point is this.
What the advantage of these consensus-- one of the advantages of these consensus systems is that they can review a great deal to us about the societies they serve.
We can say in a certain sense that the work or the job, the anthropological function of these consensus systems is to help the society figure out what it is.
So that one of the things that happens in these stories is that-- and here I'm borrowing directly on Raymond Williams-- we can think of a consensus narrative as a discourse space, as a space for conversation, right?
Thinking of also remembering that genre, in one way or another, could be thought to carry some sub-category or some sub-element of the larger consensus.
So masculinity and warrior behavior, and certain forms of honor and courage are dramatized in war movies and Westerns.
But certain aspects of family life and of the relation between siblings and of the relationship between parents and children are dramatized in forms of melodrama and comedy, right?
And detective stories sometimes explore the social and political implications of particular societies more systematically and fully than other forms.
But so take it all together.
The different genre forms may be said to lay different emphases on the consensus, but they all contribute to the consensus.
And one way they contribute is by constantly showing us fissures and disagreements.
They don't all tell the same story.
What they tend to-- and this helps to explain why movies are often either incoherent or partly contradictory, right?
Movies are not simple, stable, unitary things at all.
One reason is they're made by so many people, right?
There's no single author.
There's a director, a writer, actors.
There's an army of subsidiary people who help.
Films are profoundly collaborative form.
And one of the effects of this is that in some forms, the director's control may be imperfect, and the star may distort the film in some way, or take it-- or in some films, the writer may be so powerful and so imaginative, and the director's relatively weak that a different kind of movie emerges, right?
So movies are different in that way.
And then, of course, they're also different because they occur at different moments in time.
And those moments-- when they occur in different moments in time, even if they belong to a particular genre, they may be responsive to later problems in the society from ones that differ from the problems that may have been the case 10 years earlier when the genre-- when a different version of the same genre story would handle the characters or the story in a somewhat different way.
So what we can say, among other things, is there's a tremendous value to the idea, to the recognition that not only is culture always unfinished or always sort of struggling to figure out certain kinds of conflicts and difficulties, but that our consensus narrative systems are also a space that accommodates that kind of disagreement.
And one way to conceptualize this is to see that in almost all films, and certainly the whole system, could be said to dramatize or give voice to three different strains or perspectives on society.
One is what might be called a traditional voice.
That is to say, these are old-fashioned ideas that are still powerful, but they really belong to an older era.
The second kind of voice you hear is what we could call the dominant voice.
That is to say, the central values that the majority of the people in the society now believe, right?
A little different from traditional voices that are more old-fashioned that are becoming vestigial.
And then on the other side, emergent voices, right?
New ideas, new attitudes that are coming through.
And you can feel this in every film.
You can feel this in every television program.
If you look at American television from the 1950s to the 19-- say '90s-- let's say you've restricted yourself to situation comedy.
What do you think you would see?
You would see a drama in which the American family undergoes profound change.
What happens?
Women become more important, patriarchal values become less and less powerful, children become more respected, are treated less like appendages to the family.
And by the '80s and '90s, sometimes the children have more dominant roles than the adults and are often even shown to be wiser than their parents.
Something that would have been impossible in the early 1950s, a more patriarchal era, when the classic television program about families was called Father Knows Best, right?
Whereas in later episodes, the father not only knows-- doesn't know best, he knows least.
Or sometimes he's completely gone, and it's women who are raising the children, right?
Well, what's going on in that process has to do with the way in which the feminist movement and a reconfiguration of our idea of gender, our ideas of gender and family-- what's happening is the situation comedy is the space in which those problems are being dramatized over time.
And if you juxtapose a sitcom from the '50s and a sitcom from the '60s, and a sitcom from the '70s and one from the '80s, you can actually see that drama being enacted in tremendously powerful ways.
I choose examples from television because I want to choose examples more of you are likely to have your own experience with.
But I could offer such instances from the history of movies as well.
But almost none of you would have sufficient experience with the movies to understand the examples.
So that's why I've chosen examples from television.
I hope some of you at least-- well, I know many of you can't, because you didn't grow up on American television either.
But some of you at least can get the idea that I'm talking about here.
So the notion that-- so this is a way of explaining, for beyond what the intentions of any particular movie maker or producer would have, it's a way of explaining and clarifying the profound historical and anthropological importance of the movies in American society.
And by implication, in other societies as well because the argument applies in different ways to those societies as well.
The notion is not that movies are unimportant today.
But they occupy a different niche in our consciousness and in our daily lives than they did in the Hollywood era, in the era of the Hollywood studios.
OK. Capra and Hawks can be said to be two characteristic instances, two characteristic examples of really journeyman directors from the studio era.
I had intended-- let's put up the other-- I actually had intended and had planned to give you a much more elaborate, sort of brief biographical accounts of each director.
But I don't have time to do that.
And what I'm putting up instead here are just a list of some of their films.
You can see what a journeyman Hawks was.
He made very important screwball comedies.
Probably the dominant figure in screwball comedy.
Classic Westerns.
I've listed some of them there.
And he was also be a great director in other genres.
He worked-- it was very interesting that he could work across genres in this way.
He may be the most diverse of all the classic Hollywood journeyman professional directors because he worked so well across so many categories.
There are recurring themes that show up across genre in Howard Hawks' work.
And I'll mentioned some of them in a little while when I try to introduce His Girl Friday, one of his great screwball comedies.
Capra operated in a slightly more-- a slightly narrower range.
But like Hawks, he was an immensely successful, immensely popular director during an era when the studios were essentially operating like gigantic factories in which the studio heads where the primary bosses, and in which each studio, especially the five major studios as well as many of the smaller ones, were essentially small cities, small factory cities for the manufacture of movies, right?
Not just that they employed dozens, if not hundreds of-- well, they employed hundreds, even thousands of workers of different-- but they employed dozens of writers and dozens of directors, and they had dozens of stars and actors on contract, and they owned great lots.
And of course, they had an army of technicians, of grips and best boys and camera people, and post-production people and publicity people, and various other factotums who took care of people.
And they probably had a more elaborate encyclopedia of gophers than any other industry in history.
But the point is, in a sense that each studio was a kind of manufacturing hub, was like a great factory, the equivalent of General Motors or Chrysler.
But instead of making automobiles or toasters, they were making these things that infiltrate our dreams.
A product I think much more powerful, disturbing, and interesting, even than the most remarkable automobile or toaster.
Two other things I should mention about both Capra and Hawks, because they share this quality, and they share it with Hitchcock as well.
They began their careers in the silent era.
They made a transition to the sound era.
Capra was a more successful silent director than Hawks, but both had experience in the silent era.
Unlike some directors, were able to make a very successful transition, along-- as Hitchcock also did, and became major directors in the sound era.
The other significant thing to mention about both of them, they both spent time in the Army.
They both kicked around.
They were not sure they were-- it wasn't as if when they were born they said, I want to be a movie maker.
They had all kinds of uncertainty in their lives.
They almost fell into movie making.
But there's one thing I think all of you will find interesting.
And I love to mention this to my MIT students.
Both Capra and Howard Hawks studied engineering.
And Capra, in fact, got an engineering degree from the University of Southern California.
When he couldn't get work as a mechanical engineer, he began to cast around for other things, and he ended up in the movies.
Howard Hawks didn't graduate, but he studied engineering at Cornell.
I think he never graduated from Cornell.
But he studied engineering seriously.
And I don't think it's an accident that these directors had engineering experience.
Because there's an organizational intelligence, a practical intelligence, even genius that every good director-- not to say great director needs-- that is very will served by an engineering education.
So I encourage all of you, when you finish your degrees, to go to Hollywood and become directors.
I won't say more about Capra and Hawks now except to say again that they represented a certain sense, the center of what a Hollywood director could represent in the era of the studios, in the era when the movies where the dominant form of entertainment in American society.
I want to say a couple of things about each of the movies that you're seeing tonight, the two different screwball comedies.
One by Hawks, one by Capra, each representing a characteristic tone of each director.
As I said earlier, you can think of Capra as a slightly more-- as a somewhat more sentimental and optimistic director.
If you want a quintessential-- a scene that sort of distills this sentimental populism-- a sentimental populism is a good way of describing much of Capra's work-- there's a scene in It Happened One Night that captures this quality in him.
You can feel that-- you can feel its optimism and perhaps its fakery.
But you want it to be true, even though you know he may be imagining a nicer humanity than actually exists.
There's a scene on a bus in It Happened One Night.
A good deal of the film or a good part of the film takes place as people are traveling.
It's a road film, in a certain way.
And there's a scene on a bus, a number of scenes on the bus.
But there's one very powerful scene in which all of a sudden, after a certain dialogue occurs, the people on the bus sort of rise up and begin to sort of create a little musical number themselves.
Watch how it happens.
And it's almost as if these strangers suddenly become members of a musical troupe, and they begin to interact with each other.
It's a fantasy of community that has all kinds of sentimental, affectionate assumptions in it that is no doubt an exaggeration, but it tells us something about the sentimental populism that's at the heart of Frank Capra's vision of life.
Let me say quickly a couple of words about each film now.
It Happened One Night is not the first screwball comedy, but is the film that established screwball comedy as a dominant genre.
There are films going back probably to 1930, if not earlier, that could be identified as screwball comedies.
And there are certainly films-- full screwball comedies that existed before It Happened One Night.
But the success of It Happened One Night nailed the genre, made the genre a popular one.
And I want to mention a couple of production notes that are kind of interesting.
One of them is this.
Both the stars in It Happened One Night, Claudette Colbert and Clark Gable, had had significant careers before they came to make It Happened One Night.
Gable had been in movies for over a decade.
He'd been in a whole number of films.
But he was a second rate-- a secondary player.
And he was a difficult man.
He was resisting assignments that he was being-- he was under contract to the MGM Studio, which was the most elaborate and wealthy of all the studios.
And the head of MGM, Samuel Goldwyn, was annoyed with Gable, because he was uncooperative.
They were sending him scripts he didn't want to do, even though he was under contract to them and theoretically had to do what they told him.
And because he annoyed Goldwyn, Goldwyn decided to discipline him.
And the way he disciplined him is we said, all right.
I'm assigning you to Frank Capra who works for Columbia Studios.
You have to make a film that he's working on.
You have no choice.
You're under contract to me.
Gable was incredibly angry about this.
One of the reasons is Colombia was not a major studio.
It was a low-budget studio.
Capra was not yet a widely known director, even though he had been directing since the silent era.
Gable felt that he was being imposed upon.
He was very reluctant and angry about the job.
The star, the female star, Claudette Colbert, was also reluctant to take the job.
She had been in-- she was a stage star, and she had been successful on Broadway.
But she'd had a checkered career in the movies, some bad luck in the movies.
She'd made a film that was panned.
She was very reluctant to make any more movies.
And when she was asked, she resisted.
And she finally agreed only when they doubled her fee.
She normally would get $25,000 for a film.
This time she got $50,000.
But even when she accepted this, she still was reluctant.
She said, OK.
I will do this, but I have to be free to go to spend Christmas with my friends and family in Palm Beach.
The shooting has to stop no-- I will not work late-- longer than December 23.
And so they had four very intense weeks of shooting in which they-- in which they did their best to finish the film.
And they didn't quite finish it.
But Claudette walked off the set and said, I'm done.
My contract-- I fulfilled my contract, and she left.
Capra hadn't yet filmed the final scene of the film.
When you watch the brilliant final scene of the film, which Capra had to do without his stars because Colbert had walked off the set, you will see a magnificent example of what we might call necessity being the mother of invention.
Because it's a brilliant, rich scene, but he had to do it because he didn't have his actress there in order to make the scene.
So watch for it when you come to it.
Another point about-- when Colbert left the film, she was said to say to friends, I've just completed-- I've just finished the worst movie of my life.
This is a horrible film.
Gable was angry about the film, and thought it was a piece of garbage.
In the advent, the film was the first film to win Academy Awards in every major category.
It won for best film.
It won for best actor, best actress, best director, best screenplay.
It was the first film to notch all five victories, right?
Like, it's better than a hat trick in hockey, right?
Like a perfect game for a pitcher, right?
The best-- very few films have ever done this.
And it turned every one of the major participants in the film into major Hollywood players.
After this film, Gable became what-- came to be known as the King of Hollywood.
And for a decade and a half, he was the top-grossing actor in Hollywood.
And all of his great work follows after this film.
He was in a series of classic Hollywood films to the very end of his life in his last film with Marilyn Monroe.
Anyone know what it-- written by Arthur Miller?
A film called The Misfits, his very last film.
He was dying when he was making that film.
It's very moving film.
You can see that he's dying of cancer in the film if you look closely.
But he had a magnificent career.
He was one of the characteristic, central, iconic actors of the studio era.
And this is the film that made him.
There's a moment in the film when you see Gable and Colbert discussing-- they're forced to share a motel room, and this is very risque in the 1930s.
Man and woman together, unmarried in the same room.
And in fact, they have separate beds, but it's a very small room.
And she's nervous about it.
And one of the things he does is he puts up a line across the room of, like, a clothesline, throws a blanket over it, and he says, OK. We'll stay on either side of the wall of Jericho.
And you'll see how the wall of Jericho works out in the final scene when it comes down.
And there are moments where you could see the Colbert character very nervous about the fact that she's alone with a strange man in a motel room, and she's at his mercy, in some sense.
And there is a kind of [INAUDIBLE] about this.
And there's a moment where we see them getting ready for bed.
Gable takes off his shirt, and he's not wearing an undershirt.
He's only wearing an outer shirt.
And it is said-- it has been said so often I can't resist mentioning this to you, even though I can't confirm it absolutely.
But it's mentioned so often in the scholarship that I want it to be true.
It is said that the sales of undershirts in the United States plummeted after it was discovered that this actor wasn't wearing an undershirt.
(GRUFF VOICE) Because that's what men do, OK?
But there is a larger context for the film.
The film takes place during the Depression.
And even though it's a comic film about two apparently mismatched creatures who will come together in the end-- can you guess from the scene in Ball of Fire what's going to happen to Gary Cooper and that sexy woman by the end of the film?
Of course you know, don't you?
Why do you know it without ever having seen the film?
Because that's what comedy does, right?
The conventions of comedy do that.
And part of the pleasure in watching that is to say, look how different they are.
Look how frightened he is.
Look how aggressive she is.
How could they ever get together?
But of course you know that they will.
And part of the pleasure in watching the film is watching them get over those barriers.
Well, the same thing happens, as you'll see, in tonight's film.
But the gap between the Gable character and the Colbert character is not just gender male and female.
It's also a class difference.
She belongs to the upper classes.
She has no experience with the outer world at all.
She's lived an incredibly protected life.
So that when she comes to terms with Gable, she's coming to terms not only with certain inherited forms of class prejudice.
She's also overcoming a certain kind of innocence and naivete about her relationship to men, and her relationship maybe even to her own sexuality.
So the film is a bildungsroman.
You know that word?
Novel about growing up?
About this woman who lived a very-- lives a very sheltered life.
She's a very spoiled rich girl.
And the drama of the film is she becomes, in a certain sense, begins to discover her own real nature, begins to discover something about her own-- about her own character and about her own capacity to judge the outer world.
She's very limited and foolish earlier.
The context of the film, then, is a dramatization of this relationship.
But the background of the film is the Depression.
And as you'll see, this sexual and marital adventure, this comedy of marriage, or eventual marriage, takes place against a backdrop of suffering and unhappiness and misery.
And it imparts a resonance and an importance to the film that it wouldn't otherwise have.
Because there are scenes in these trailer homes where the Colbert character, completely unused to having to live a life like that of ordinary people, is suddenly forced to confront what such lives are-- what such a life is like, and it's part of her education.
So the context of the Depression is important.
A second fundamental thing, as I've already implied by my anecdote about the undershirt, is that the Gable character came to stand for an ideal of American masculinity.
In the scene where he is first introduced, he's half drunk.
And it's important for you to understand the context here, because it's not exactly celebrating drunkenness.
Although it partly is, I mean, he's a man's man, right?
We see him get drunk and talk on the telephone to his editor boss, who fires him, right?
And he staggers out of the phone booth, and he's talking to his friends, and he says, I'm the king.
This is where the word-- where the term the King of Hollywood originates, in the lines that you see in the film.
But it's important for you to understand that part of what's going on here-- it's 1934.
Prohibition has just been lifted, right?
And what's being dramatized there, it's illegal for him to drink for the first time in whatever it is, a decade.
And the film is partly celebrating that, right?
So you're not supposed to think of him as some ridiculous roustabout drunkard.
You're supposed to recognize that what's partly being dramatized there is the new freedom that has emerged in American society now that Prohibition has been repealed.
And in that scene you can see something-- because his half drunkenness, we do see some of his inhibitions.
You can see something of his true nature there.
And what you discover is-- and you can see this come out elsewhere in the film too-- something that the Colbert character at first can't recognize.
That he has a heart of gold, that he's a decent, admirable person, that he would never treat anyone badly.
But he's also very self-reliant, and no one can push them around.
He's a particular kind of-- represents a particular kind of masculine ideal, right?
He can't be intimidated, but he'll never mistreat someone.
He has a kind of moral compass that guides his behavior, even though it's implicit, right?
He's a bit vulgar.
He's suspicious of wealth and pretension.
The Gable character came to stand for a particular kind of American masculinity that was celebrated over and over again in American movies, a kind of-- a type of the American male.
I've already said that one of the things the film does also is dramatize a romance across social class.
And you might want to think about the way in which the-- in almost all comedy, and certainly comedies that involve-- comedies about marriage, comedies about men and women, almost all such comedies involve some notion of barriers of some sort that are placed in the way of the consummation in which the text wants to end, right?
And those barriers can take many forms.
But one of the most fundamental forms they take over and over again is the barrier of class.
And one of the things that the Colbert character and the Gable character both have to do is get over their prejudices, get over their blind spots about the social classes they have less understanding of.
One of the most moving and powerful aspects of the film, and I think you'll see this really powerfully, is the way in which the audience comes to have an understanding of the characters that exceeds that of the characters.
And that there are many moments-- there are many moments in the film in which you will understand how the characters are feeling before they are able to acknowledge it themselves.
You can see them falling in love, for example, without them being able to admit it to themselves.
And it's immensely-- think of what an immensely subtle trick that is, to put the viewer in possession of information or knowledge that the characters themselves don't have, and what a rich kind of knowledge and ironic perspective-- a rich kind of irony that generates for the viewing of the film.
"A marriage of true minds" is a famous sonnet by Shakespeare that begins, "let me not to the marriage of true minds admit impediments."
The impediments to the marriage of true minds in this film have to do with the characters-- the Colbert character's naivete with the prejudices and self-reliance of the Gable character, with the social realities of the Depression.
And these barriers have to be overcome.
But "a marriage of true minds," what could the phrase mean?
A marriage of equals, a marriage of people who are not different.
The deepest thing I love about this movie, and the reason that I've come-- that I treasure this film, even though there are many old-fashioned qualities in it that make it feel old and distant, in some sense, from contemporary society is its understanding of-- it's the implicit, idealized understanding of marriage.
Because what the film really wants to suggest is not that-- first of all, it suggests that these people are drawn to each other sexually.
It's not embarrassed by the fact that they're both physically beautiful people and that they feel attraction for each other that they can't themselves fully acknowledge or deal with.
But the second kind of attraction they have is a mental attraction.
They realize that they're both deeply intelligent people, that they're both people who are deeply verbal, that they both have improvisatory powers, intellectual powers, that distinguish them from other people.
That they're made for each other, essentially.
There's one scene that shows this.
Before they realize it, we realize it.
It's a wonderful scene.
They're hiding out in a motel.
And at a certain point, she's on the run.
Her rich father is searching for her.
And the Gable character's a reporter.
Says, I'll protect you and help you if you give me your-- if you let me have an exclusive story, because she's such a rich heiress that she's big news.
So she agrees, and he's escorting her north, hiding from the people who are searching for her, right?
And at a certain point, they're in a motel.
Some private cops knock on the door of the hotel, and they come in, and they're searching for her.
And watch what happens.
What Gable and the Colbert character do is they immediate-- Gable says, follow my lead.
And then when they come in, they begin to play husband and wife having a tremendous quarrel.
And you can watch-- you can see that they're improvising it.
And you can see they're enjoying it.
And you can see how well they play with each other.
So in other words, it's a moment in which you can see that it is a marriage of true minds, long before the characters themselves fully recognize how they are meant for each other.
That's a wonderful moment in which their intimacy and there appropriateness for each other is dramatized.
It's not only a physical or a sexual thing.
It's also an intellectual and a moral and mental thing, a psychological thing.
They are-- it will be a marriage of true minds if they can come together.
I want to end-- just finally mention-- I've mentioned the ending already.
And I'll just say one more time that, watch how the ending works out.
And you'll see a magnificent example of a director dealing with contingencies he could not possibly have anticipated.
And it's a wonderful example of a kind of inventiveness.
Seems to me, the film ends as well as it could end.
And it's better that the actors aren't there.
But he certainly-- when this happened, I'm sure that Capra thought it was the end of his film and that he was-- didn't know what he was going to do to try to finish his film without his star actress.
I have to be even briefer about His Girl Friday, but I think it's a more straightforward film in some respects.
It's a much less sentimental film, and I hope you'll recognize how deeply-- how much more deeply, unsentimental, even cynical, or at least deeply tough minded and suspicious, skeptical Howard Hawks is about all of the operations of the world.
Not only about individual human motives, but about all human institutions.
Virtually everyone in His Girl Friday is a cynical, self-seeking creep.
Even the primary characters, the Ros Russell character and the Cary Grant character have intriguish, manipulative elements in their natures.
And you can see especially that the Cary Grant character is, in some respects, a very nasty piece of work.
He does terrible, terrible things to poor Ralph Bellamy, the man that Ros Russell is supposed to marry, her fiance.
He plays a sort of straight, sweet, straightforward kind of person, very much in love with his girlfriend, but completely-- how can we put it?
But he's a dummy.
He's stupid.
And even though he's very nice, you don't really like him.
You think he's a jerk.
The film makes you mean or nasty, in some sense, because it makes you-- it makes you see things from the perspective of the Cary Grant character.
It encourages you to be tough minded and skeptical.
And you do come away from the film thinking, oh, gee.
I've participated in the humiliation of this poor, decent guy.
But anyway, he's deserved it.
He's such a stupid mark, right?
You can't-- you almost can't help that kind of reaction.
The film is based on what had been a very successful play on Broadway, some generation earlier.
And that play was made into a film in 1930 that was quite successful, directed by Lewis Milestone.
But the Hawks film is the classic version of it.
And Hawks does one thing that's very important, that changes the original, changes the play in a fundamental way.
In the original play, the character played by the Rosalind Russell figure in the movie is a male character.
And the great discovery or the great transformation that was wrought in the film by Howard Hawks was his decision to turn that character, a guy named Johnson, from a male Johnson into Hildegard Johnson.
Hildy, Johnson, the character played by Rosalind Russell.
And as soon as he turned her into a woman, he created his screwball comedy.
He created a wonderful kind of antagonist for the Cary Grant character.
I've already indicated to you how many films in different genres Hawks directed.
Remember that list I put up there was not a complete list at all.
It was a selected list.
But it was some of the most important films by each director.
And one of the things that's really interesting about Hawks is that he directs Westerns and screwball comedies and private eye films and war movies, and a range of other lesser, sort of no-genre films as well.
And yet in all of them, there are certain throughlines.
You can recognize certain recurring themes across these radically different films.
And one of the most fun-- and one of the way-- one the most fundamental impulses in a Hawks film, whether it's a Western or a war movie or a screwball comedy, is an impulse to slow things-- well, not to slow things down exactly, because his dialogue is so rapid-fire.
But to sort of stop the plot, to interrupt the story with a scene of argument.
He loves scenes of quarrel.
He loves scenes of discussion, of talk.
He loves talk.
And especially, he loves intelligent people quarreling with each other.
And there's never been a director better at eliciting from his actors quick, rapid-fire dialogue.
In fact, he developed a strategy for doing this.
He used to sit right next to the camera.
He didn't stand behind the camera.
He had a cameraman.
But he sat right next to the camera, and he had his actors play to him, only sometimes very close to him as they were acting.
And if you watch, especially some of the scenes in this film, you will see what astonishingly realistic performances he got from his actors by this strategy.
He even encouraged certain forms of improvisation.
He would run through rehearsals first, and then he would encourage, when he began to film, scenes-- he would encourages his actors to improvise and to improvise to his face.
And he creates a kind of electric authenticity in his dialogue scenes that has a tremendous authority.
So the character plot I'm talking about is this impulse to slow the plot down or to stop the plot for scenes of argument and conversation.
And what constantly happens, whether he's making a Western or a screwball comedy, is his characters stop the action and say, oh, what do you think is going to happen next?
If I do this, what will happen then?
As if the films begin to generate a kind of meta-commentary about their own nature, right?
So they're incredibly talky films, and the dialogue is fundamental to your-- the rapidity of the dialogue is fundamental to your understanding of the characters and to your understanding of the larger film.
So his central scene then is a scene of quarreling intimacy.
You feel that they're quarreling, but you also feel that no people could quarrel exactly in this way if they weren't also intimate with each other.
And you'll see fully how it works out when you see the film itself.
So finally, I want to say at least a word to remind you how it actually-- although he's tremendously entertaining line by line and scene by scene.
And never calls-- such a professional.
He never calls attention to his camera.
He's a very quiet filmmaker in that sense.
A real journeyman, a real professional.
He doesn't want you to think about him or his camera.
He wants you to think about what you're seeing.
There are limits to this strategy, and we'll talk about what the limits are when we look at certain other directors who behave in other ways.
But there are also great strengths to this, as you'll see when you watch tonight's film.
And something of his complexity is embedded here.
Because what you can find is he creates scenes of immense comic energy, really generates real laughter.
But beneath the laughter, there's a deep layer, a deep substrate of cynicism, of skepticism, about human motives, about the virtue of human institutions, about the capacity of human institutions and individual human beings for corruption and misdirection.
There's a kind of deep political and social cynicism in the film that doubles the comedy or complicates or deepens or roughens the comedy, sours the comedy, in a way that makes for a particular kind of complexity.
It's possible to argue, which some critics have, that this film, more than any other that Hawks made, combines laughter and cynicism in unique measures.
And I think together, these two films and these two directors can mark at least a certain pole, certain poles in the tradition of Hollywood filmmaking, and especially in the tradition of screwball comedy.
You can feel the pull toward a kind of sentimental, affirming impulse in Frank Capra, and the pull toward a kind of cynical, distanced, complex, psychologically deep, politically informed laughter, even mockery in Howard Hawks.
And they represent two aspects, two elements, almost poles within the larger story of the Hollywood studio film.
Enjoy the films.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We turn today to Alfred Hitchcock, one of the most remarkable, if not the most remarkable and significant, of what we might call the great studio directors, the people who worked with great ease and success inside the studio system that developed in the United States-- began to develop in the United States in more than embryonic form, in early systematic form in the silent era and then was fortified and extended into the great mass manufacturing center for dreams and movies, the Hollywood system of the studio era.
And I wanted to begin by saying a word about Hitchcock in relation to that great system.
If we think about a famous quotation from the great film scholar, pioneering film critic Andre Bazin, to whom we will return later in the semester.
Bazin, among his other an influential comments, this one is a particularly powerful and significant one.
"The American cinema," he wrote in 1957, "is a classical art," and it's unclear exactly what he meant by that, but most critics assume that what he meant by that is that it's a system that works according to essentially classical genre forms, that these genre forms have origins behind the movies, and that the system works in a kind of a generic way, and that at least that's a part of what he meant when he called it a classical art.
The American cinema is a classical art, but why not then admire it in what is most admirable, not only the talent of this or that filmmaker, but the genius of the system.
And I think I alluded to this phrase or used this phrase earlier in the term, and I want to do justice to it by acknowledging its origins in Andre Bazin.
There's also a wonderful book by a film scholar from Texas named Tom Schatz called The Genius of the System, and it borrows, it uses this quotation as its inspiration, and it's a systematic analysis of the Hollywood studio system in which it talks about the interaction of individual agency-- the writers, the directors, the camera people, and so forth, the customers-- the interaction of those creative or semi-creative figures with the manufacturing and publicity practices and rituals that surrounded the production of movies in which he tries to provide a kind of integrated sense of how the system worked to explore what Bazin apparently meant when he talked about the genius of the system.
Well, one way of understanding the genius of the system is to recognize that it creates an environment, first of all, of stability in which particular filmmakers or particular writers or directors can have confidence-- sometimes overwhelming confidence because they're ordered to do the job by the studio head-- can have confidence that the genre stories they're creating have an audience.
And of course, that's been sort of established by the, essentially, assembly line system that develops, and the elaborate system of distribution and access that developed in the United States.
The major studios actually, although they only controlled something like 16% of the resources they-- of movie-making-- actually controlled a much larger percentage of the theaters, because many of the theaters either were owned outright by studios or until a certain point when a Supreme Court decision divested the studios from-- forced the studios to divest themselves from their theatrical holdings.
But when the system was in place, it was a monopoly system in which the rich got richer, in some sense, and the major studios controlled, owned many theaters themselves and by a system of what was called block booking, they forced theaters not just to book particular films and particular stars that they might like, but MGM would say to independent theater owners, if you want the MGM films, you have to take my whole card.
You can't just choose the Clark Gable movies, you have to choose the whole thing.
And this stability created an environment in which each major studio was confident that, in effect, it had a captive market for its product, and it created an immense sense of stability and confidence when the system was working at its best.
And of course, again, because it was a system that was committed to entertaining the largest number of people it could reach as possible, it meant that there were certain parameters that were established within which the entire system was forced operate.
Whatever genre was involved, there were certain kinds of limits.
It's what I was talking about earlier, trying to get at this aspect of the system earlier, when I talked about the idea of consensus narrative.
If the Hollywood film is reaching out to the whole of the society and it's telling, essentially, a story that appeals to or is supposed to embody the consensus view, the largest general view of what the belief system of the society is, what that means is that there are limits, very sharp constraints-- political and moral constraints-- within which the text operates.
And as you know, especially if you've been reading your [?
Cook, ?]
the introduction of a particular sensory system introduced by Hollywood itself in order to avoid other kinds of censorship perhaps from the government, further constrained the kinds of stories that could be told.
Well, within those constraints, of course, as I hope I've already begun to show you in our discussion of screwball comedy, there can be an immense range of difference, but it's still within certain controlled parameters.
And one of the ways to understand Hitchcock's immense success is to recognize that he had the kind of sensibility, the kind of artistic impulses, and maybe even the kinds of limitations that made the studio system a kind of perfect environment for him.
Anyone who's looked at a Hitchcock-- more than one or two Hitchcock films, is aware of the fact that there's something obsessional, something deeply disturbing about Hitchcock's imagination.
He's drawn again, and again, and again to the same kinds of situations, to scenes of violence against women, to scenes of confinement, but the studio system was a kind of perfect environment for these kinds of preoccupations.
On the one hand, the constraints of the system did not allow Hitchcock, even if he had the impulse to do so, to press so far into the perversity and disturbance that is the general subject matter he is looking at, as to actually fall into, say, pornography or to fall into something that will deeply offend some segment of the population.
We can see that these were impulses in Hitchcock's imagination though, because after the-- Hitchcock lasted long enough, lived for such a long time-- he was a successful director in the silent era.
Then he worked in Britain during the sound era and was the most well known and successful of all British directors in the early sound era.
Then in 1940, he came to the United States for what as most people recognize as his American phase.
He emigrated to the United States and that's a separate kind of distinct part of his career, as I'll describe in a little more detail in a moment.
Well, by the end of his career, Hollywood itself was undergoing profound changes for reasons we've already begun to talk about in this course, the most important of them being the advent of television and the way in which, through the 1950s, television began to leech away the consensus audience and the consensus function that Hollywood had played in American life.
And the effect this was to, in some sense, liberate Hollywood, and we'll be talking about what this ambiguous liberation later in the course when we look at some of the great films that emerge in the 1970s that are free of the constraints of a consensus system.
Films like McCabe & Mrs. Miller or Cabaret, the two films that we're going to be looking at later that embody these sort of post studio era ideals and principles.
Well, one of the ways in which you can see the damage that was-- the help that was given to Hitchcock's own imagination by what I'm calling the genius of the system, is that you can see that Hitchcock's-- that the latest films that Hitchcock makes, the films that Hitchcock makes at the very end of his career, and especially a film-- two films, a film called Frenzy starring Michael Caine in 1972, and the last film the Hitchcock made in 1976 called Family Plot.
The same basic materials, but there's something gratuitous about the nudity in these films.
He wasn't allowed to show nudity in the studio era, and it was good for his imagination.
And when his imagination-- and any honorable-- any honest viewer of Hitchcock watching Frenzy, or especially Family Plot, and watch-- there's a scene in Frenzy in which a murderer strangles a woman.
Now, there have been-- strangulation is one of Hitchcock's favorite forms of murder, and there've been many characters strangled in Hitchcock's movie, but this strangulation has a pornographic dimension to it that none of the early scenes like that did.
And it has to do with the fact Hitchcock is now working in a film environment that is not telling him that he can't go too far, and he does go too far.
And there's a terrible scene in which a murderer takes off his tie, and he strangles this woman with a tie.
The woman is naked from the waist up, and you see her breasts bobbling around on the screen while she's being strangled, and you can see that the camera is enjoying looking at those breasts, even though it's a scene of murder.
It's a very, very horrible scene, a very disturbing scene.
It's a scene that-- it almost is a scene that makes you believe in censorship, to think that censorship ought to be-- certainly, you wouldn't want young people to see this scene.
Now, the only reason that Hitchcock was free to create this scene is that there were no longer the same constraints being imposed upon him by what I want to call the constraining genius of the system.
So my point is that Hitchcock was a man of obsessional genius of a certain sort, but he had the great good fortune to work within a system that also limited his liabilities, that didn't even allow the full expression of his obsessions in ways that might become deeply disturbing to audiences.
The fact of the matter is, when you look at that scene and then you think back at many earlier Hitchcock movies, you can see many equivalents of it.
Violence and damage to women is a recurring obsession in Hitchcock.
Hitchcock is a sick man in many ways.
But he's not a sick artist, he's a sick person.
He turns his sickness into art.
He turns his sickness into use by dramatizing it and reminding us of the-- so insofar as he's a good filmmaker, he's not a-- he's not a horrible perv, right?
But there are perverted dimensions to Hitchcock.
And in fact, part of why we find him interesting is his films are always hovering on the brink of awakening in us feelings that are disturbing and unsettling, and that touch on deep taboos in the ways in which our culture sort of understands how we should behave, and especially in our attitudes towards sexuality, and towards seeing, toward the act of seeing, which in Hitchcock becomes a kind of voyeurism.
So one way to understand Hitchcock is to understand his genius, his greatness as a director as being directly connected to the fact that for the most part of his career, he worked in systems that constrained him.
He worked in systems that had a very sharp boundaries, that didn't allow him to do certain kinds of things, and those limitations were turned into artful and valuable gestures.
An anecdote that Hitchcock told about himself many times, this deeply revealing anecdote.
It may not be true.
We're not really sure, but he told it so many times that it's true even if it didn't happen.
That's a line-- it's true even if it didn't happen-- it's a line from Ken Kesey's wonderful novel, One Flew Over the Cuckoo's Nest.
And the character, the narrator in the book, who was sort of a crazy man, or we think, he's in an insane asylum, he's in a loony bin when he starts narrating the story, tells us a story of his confinement in a lunatic asylum, and he says, you think the man who's saying this is raving and crazing?
My god, this is true even if it didn't happen.
And what he means is there's a kind of emotional truth, even if the actual facts aren't true.
And I think that's true of the Hitchcock anecdote as well, and this is the anecdote he repeatedly told about himself.
He said, when he was a young boy, five or six years-old, he committed some indiscretion that his father disapproved of, and his father, in response, without saying anything to the boy, wrote a note, sealed it in an envelope, gave it to the boy, and told him to take it down to the local constable-- the policeman in the police station around the corner from where they lived in London.
So the young boy dutifully took the note to him, and then gave it to the constable, and the constable opened it, read it, and locked the boy in a prison cell, and kept him in there for a certain short amount of time, and then finally released him, and said, this is what happens to bad boys.
And Hitchcock has said, ever since then, I have gone to any length to avoid arrest and to avoid confinement in confining spaces.
And in fact, if you think about it for a second, over and over again in Hitchcock's films, a basic story, not the only story, but a basic story he tells is the story of someone who is wrongly accused.
We might call it the wrong man theme, right?
And someone who was on the run, who looked at the circumstantial evidence against him is-- or her, usually him-- is overwhelming, right?
Absolutely looks it.
And all the authorities, all the legitimate authorities of the culture, think the protagonist of the Hitchcock chase film is guilty, right?
And his obligation is to somehow not only escape the authorities, who are much more powerful than he, of course, and have at their fingertips all kinds of modern systems for searching, and following, and capturing people, and the fugitive is a lone fugitive on his own without resources and without allies.
So he's up against tremendously difficult forces and these are the forces of legitimate authority.
So the theme of the wrong man, who's wrongly accused.
The audience knows that he's wrongly accused.
Often, Hitchcock will show the real murderer and will show us how circumstantially persuasive but also falsely evidence is, so through most of the movie, we identify with the fugitive, with the person who's running.
We know he is innocent.
So part of it is-- so many of his films then sort of dramatize a kind of massive principle of injustice that happens again and again in his movies.
Authorities are almost always after the wrong man.
And then many of his films, an overwhelmingly disturbing recurring element in his films, is confinement in tight spaces, is the sense of being caught.
And you'll see one of the great, great instances of this, one of his most artistic accountings of this impulse, this fear of confinement, but this fascination with confinement, in the great film you're going to see tonight, Rear Window, which takes place entirely in a single confined space, in a room, because the man who-- the protagonist has a broken leg at the beginning of the film and he's literally unable to move out of his apartment.
So the entire film is confined there, and, of course, many of Hitchcock's films love this.
So what an interesting anecdote to be told.
And think, by implication, what it says about his father, not to mention what it says about Hitchcock himself to tell a story like this so many times over the whole of his career.
But the idea that a world that seems perfectly benign and protective could suddenly turn menacing and terrible, that behind any door or window lurks some monstrosity, that the ordinary world is just an illusion, that what you think of as ordinary, and plain, and prosaic can suddenly erupt in violence, or in terror, or in some form of unpredictable assault is a constant feeling you have in Hitchcock's films.
His films dance along an edge in which the whole of the universe could be said to be, in some sense, endangered.
The basic laws of our experience, of even sometimes the physical laws of our experience are upended, or denied, or suspended in Hitchcock's world.
Can you think of one example where the laws of nature itself suddenly go cuckoo?
One of his most famous films
The Birds
The Birds, of course, the late film The Birds.
What happens in The Birds?
All the birds start attacking [INAUDIBLE]
Yes, the most benign and beautiful parts of nature suddenly began to attack human beings.
He gives some partial explanation in the film, where you see there's a scene in a restaurant where you see pictures of fried chicken, and people talking about how they love roast pheasant, and so forth.
But in all of Hitchcock's films, there is never an adequate, a truly adequate motive for the madness, terror, disorder that is released in the movies.
There's often an attempt to explain them, but nobody takes the explanations very seriously, because the point is, his vision of life is a vision in which the world is an incredibly dangerous, wholly unpredictable, monstrously fearsome place.
Maybe his dominant emotion is the emotion of fear.
He dramatizes fear over and over and over again in his movies.
Here is a brief selection.
He made over 50 films, and I think 53 total films, silent and sound films in his career.
The list I put up there is just a very brief selection, but let me say a few things about the trajectory of that career and then I'll turn to a bit more about his technical genius and about the central themes of his work, which I've already begun to elaborate and won't repeat myself too much, I hope.
But let's talk a bit about his career, because like Howard Hawks and Frank Capra, Hitchcock is, even more than those two, a dominant figure of the studio era, maybe the dominant figure.
Certainly, the director whose work has been most influential, has lasted the longest.
One of the most interesting things about Hitchcock's career, as a whole, is that even after film came to be recognized as something more than mere entertainment, Hitchcock was always admired as a great entertainer and was a successful director from the silent era from the time he began to work in Great Britain.
When he made his move to the United States in 1939, he was Britain's most famous and most admired director.
In fact, when he came to the United States, there was a kind of negative reaction in Britain, because they felt it was unpatriotic of this great director to desert his native homeland in the middle of the war, among other things.
And he did feel guilt over this, in many ways, and some of that guilt is said, by some scholars, to express itself in some aspects of the other film you're going to see this evening, an earlier film, made in 1943 shortly after Hitchcock had come to the United States.
It wasn't his first American film.
While he was making that film, he was receiving news from England about his mother, who was very ill, and she actually died while he was in the United States filming Shadow of a Doubt.
And there are some scholars who say that his familial feelings, his guilt over leaving England, his guilt about deserting his mother come out in various ways in Shadow of a Doubt.
I'm not so sure about that, because it's a pretty cynical, and tough-minded, and anti-sentimental film in its own way, but there are some elements of family life in it that perhaps recover or allude to aspects of Hitchcock's own career.
One significant thing, as I've already suggested, is that he was successful at every phase in the history of movies that, like Hawks and Capra, he began in the silent era, did distinguished work there, moved into the sound era and did distinguished work in the sound era.
He has something else in common with Hawks and Capra, and I only recently discovered this.
It's not as systematic as in the case of Hawks and Capra, but he, too, studied engineering.
So there must be something in this.
You guys should maybe reconsider what you're up to here, because three of the most remarkable and technically adept directors in the history of the studio era all had partial training as engineers.
So he begins in the silent era, and in fact, let's go behind-- let me say just a little bit about his background.
He was an outsider, in a certain way, even though he was an Englishman, because he was a Jesuit.
His parents were Catholic in Protestant England.
And he felt himself, all his life, I think in some ways, to be a kind of outsider, someone who didn't exactly fit in traditional society.
He went to work after his schooling in the advertising department of a telegraph company, began to write the title cards for silent films as early as 1921, and then began for this telegraph company, began to work on certain feature films that were co-produced in Germany.
And this is, of course, the great period of German Expressionism, when the great German silent directors are creating their science fiction and Expressionist works.
And Hitchcock, in his early life, is immersed in that stuff, learns that stuff, goes to school in that, and you can feel the Expressionist impulse in the darkness and the disturbance that's a central part of almost every Hitchcock film.
He makes something like, I think, a total of six or seven silent films of which the-- do we have the list up there, yes-- of which the most important is-- The Pleasure Gardens was a co-production, the first film that he worked on systematically, and it was a German co-production.
He wasn't the prime director in it.
The Lodger is probably his most-- almost surely his most important and most Hitchcockian silent film.
Can you guess the topic?
It's Jack the Ripper.
It's a silent film.
It's a story about a landlady who thinks fearfully, nervously that she may have rented a room to Jack the Ripper, the famous killer.
So he's a successful silent director already, an admired director, makes the transition to sound, and in fact-- historically he'd be famous just for this one fact-- Blackmail, a British film he made in 1929, is the first British talkie.
And he almost immediately began to figure out how to integrate-- was very interested in all the technical aspects of movie-making, especially interested in the way you integrate sound with image.
And as anyone who's watched the shower scene in Psycho, for example, would be a famous example, anyone who's watch these scenes like that in Hitchcock will recognize the tremendous importance of music in his films.
The way he uses music to deepen and complicate the moods he's creating, and especially how he can use music to enhance your sense of terror and fear as he does in certain frenzied, powerful scenes in his most remarkable films.
And then he goes on to really an immensely successful career as the director of action adventure mystery films of which I've listed the most famous and significant ones, films that are still interesting to people, that are still watched today for their own intrinsic excitement, even though many of them also feel a bit old-fashioned in their behavior.
The Man Who Knew Too Much, The 39 Steps, and The Lady Vanishes, and there are other films he made in this era, I'm only listing a selection, as I've said.
Then he makes the transition to the United States.
He's lured here by David Selznick, the head of a great studio.
Hitchcock is especially drawn to coming to the United States because he envies the technical resources that are available to American directors, who, because they have much more budget, they can do much more when they have enough money to add cameras, and to have adequate crews, and so forth.
And the first film he makes in the United States, Rebecca, a remake of a famous novel, won an Academy Award as best film of the year, although Hitchcock did not win a director's award.
But it's the least Hitchcockian of all of his films, and perhaps he was restraining himself a little bit in an attempt to establish himself in the American audience.
It's a very interesting film, and you can still see that it, in some broad way, fits Hitchcock.
It's a Gothic story, the classic English novels about governesses who go off in the country to strange mansions, and they are both attracted to and repelled by the handsome, sometimes scarred stranger who runs the place, right?
I'm talking especially about what novel?
Jane Eyre, Charlotte Bronte's great novel.
And of course, that pattern-- that [INAUDIBLE] that pattern repeats itself again and again in the movies, and Rebecca is a version of that kind of story.
And then Hitchcock goes on in the '40s to make a series of films in which his own interest in the technology of motion pictures and his own obsession with setting problems for himself that are difficult to solve, his own sort of engineer's obsession with the technology of motion picture begins to become clearer and clearer.
Another reason that The Lodger is an important Hitchcock film is that it's the first film in which Hitchcock himself makes a brief cameo appearance, and after that it becomes a kind of signature feature of his movies that at some point in the film, a character-- Alfred Hitchcock will appear briefly.
He won't be identified, he won't be noticed in the credits, but as he became more and more well known, audiences began to watch for this.
This bothered him in some way, because it meant that it distracted people from the movies themselves, because they were waiting to see when Hitch would emerge, so he began to do it earlier and earlier in the films in order to avoid distracting audiences.
But think what it means, having to devise a way for him to be in the movie.
In many cases, it's easy.
OK, a man's getting on a bus, so he can be in the crowd of the-- but what happens when he makes films that are arbitrarily restricted, as he does in some cases?
For example, the film Lifeboat, in 1944, some of you know about it.
It takes place entirely on a lifeboat.
There are eight or nine characters.
They've escaped a shipwreck.
It's a World War II parable about Nazism, and they're all stuck in the lifeboat.
And for the entire film, the film is confined to that lifeboat.
It presents all kind-- it's not one of his best films, but the technical challenge presented is really interesting, isn't it?
Vertigo was an even more interesting instance, and a much more successful instance of the same desire or the same weird impulse in Hitchcock himself to create confining situations and then see what that confinement allows him to do, what grows out of these arbitrary limitation.
But just as a kind of minor instance of this kind of thing, if all the characters in Lifeboat are on a boat in the middle of the ocean, how can Hitchcock appear?
Because he's not one of the main characters.
Does anyone know how he did it?
There's a newspaper lying on the floor of the boat, and you pick up the newspaper, and there's a picture of Hitchcock in the newspaper.
It's a trivial thing, but it [INAUDIBLE], but its triviality is exactly what makes it so interesting.
In other words, what Hitchcock loved, this kind of problem.
When he took these problems in a serious way and when these problems sort of led him into a kind of exploration of our darker, and more disturbed, and more uncivilized sides, he made remarkable films that are memorable and haunting.
And one more example of Hitchcock setting himself these difficult technical feats.
The film Rope, well titled in a way.
It's a murder mystery, but Hitchcock set it up in a way, it's actually, such that, it creates the illusion that the entire film-- I don't remember how long it is.
Let's say it's a standard feature length film.
Let's say it's 89 minutes long.
It's a standard feature length film.
It feels as if it's all unraveled in a single take, as if there are no cuts, no edits in the movie.
Now, it's not exactly true, because the film was not capable, technically, of doing that.
What Hitchcock did was, he had cassettes of film that would last, say, 10 minutes.
So he would make 10-minute long takes, but then he'd have to change the cassette, put in a new one, but he disguised the takes.
He disguised the cuts.
And in fact, it's an unbelievably tedious film, because when you're watching it, you feel you can't look away.
One of the things that you discover when you're watching this film is that cuts are actually a relaxation.
They let you relax.
They break the rhythm in some way.
And if you sit there for 89 minutes watching something that seems to be unfolding-- and film, of course, unfolds in real time, also, therefore, right?
So he's playing both with another kind of confinement, which is the duration of the movie.
He's saying, I'm going to make the duration of the viewing the same duration as the action of the movie, and I'm never going to cut, or I'm going to create the illusion that there's not going to be a single edit in my film.
When you're watching it, you actually, yourself, feel trapped.
You feel guilty if you look away, or you, but you constantly want to blink or look down, because it's almost as if the absence of edits creates a kind of continuous stream of imagery, and you feel you're trapped within it, like a wall that doesn't let you escape from it.
Well, that kind of-- I don't know that Hitchcock actually intended such a reaction, but is not inconsistent with Hitchcock's desire to manipulate your feelings as you're watching a film.
And he became a master at manipulating your reactions as you watch the film.
So there are many-- and I could cite many other examples of this sort of technical obsession, but let me just say a couple of things about the major films.
And he successfully, in every phase, but he really comes into his own, most folks critics would say, at the end of the 1940s with a series of-- maybe the early '50s-- with a series of films that run from '51 through, maybe, most people would say through The Birds in 1963.
And I haven't listed all the films he made in that era, but I've listed the most important ones, and you can judge for yourself how significant this is, why?
Because look at how many titles you recognize.
How many titles by directors so many years ago would you actually have seen?
It's one measure of what an important figure, what a successful figure Hitchcock is that he's still so widely known, that he's more famous today than he was when he was making his movies.
As I started to say earlier, one of the most distinctive things or interesting and revealing things about Hitchcock's career is that through the mass of his career, even though he was one of the most popular directors, and his films made money, and he was a totally bankable director, no, maybe the most bankable director in Hollywood during the studio era, he didn't get much respect.
Some people have suggested that one of the reasons for it is that he gave so much pleasure that there was a feeling that anyone who gave this much pleasure couldn't be an artist, that it worked against him that he was such a great entertainer.
And there may be something to that.
But steadily over time, and especially since his death, his reputation has increased, and increased, and increased.
And in fact, recently, his film Vertigo was voted the best film of all time, the best film ever made, displacing Orson Welles' Citizen Kane, which had been at the top of this list for something like 20 years.
And this ascension in which Hitchcock was sort of denigrated as a mere entertainer to now being recognized as one of the greatest artists in the movies of the 20th century is a very interesting development, and it's a development that's partly a function of the fact that all of that happened after the movies were no longer the central narrative experience in the modern world in which they had already been displaced as a system.
So in this period between Strangers on a Train through The Birds in 1963-- some people might include Marnie in '64-- he made a series of masterpieces that are among the most significant and most powerful films ever made, two of which we're talking about tonight.
Hitchcock the technician.
I've already implied something about this, but it's worth emphasizing this.
Hitchcock became famous, especially in the industry, for the unbelievably fastidious, almost bored way in which he carried out his direction of a movie.
By the time Hitchcock got to the set of the film, he had so obsessively planned out every aspect of the movie that he acted as if everything was all finished.
He often annoyed his actors.
Some actors disliked him a lot, because he gave very little instruction to his actors, and some actors like this, but many resented it a great deal.
See, he's been quoted a number of times as showing a kind of disregard for actors.
At one point, he said actors are just cattle to be moved around on the set like animals.
But it was more of the fact that the actors had trouble with him because he seemed to have worked everything out before he got to the set.
He was always tremendously calm, he sat in his director's chair, would give directions, and he had storyboarded-- he had drawn out every single scene and every single camera angle before he even came to the set, so that it was as if he had worked everything out before he reached the set.
And what that meant, of course, is that no improvisation on the set.
He would be very unhappy if people tried to deviate, never allowed people to deviate from the plans that had been set before.
This is in shocking contrast to the way in which, let's say, a director like Jean Renoir or sometimes Orson Welles might operate in making their films.
Quite the-- looking for the contingent and the accidental that happens while you're at work creatively.
Hitchcock would have none of that.
Hitchcock came into the process of directing with the film finished, completed in his head, completely, fastidiously blocked, angled, and so forth, and there was no deviation from it.
And he actually did often act bored when he did it.
How interesting that is.
But he's also, of course, a supreme technician of the movies, a master of camera angle, a master of montage, a master of the mingling of music and image, right?
And you'll see many, many examples of this in the two films we are going to see this evening.
How interesting, then, that he is this incredibly fastidious, granular technician who plans out every single camera angle before he comes on the set.
What does this tell us?
Why would he do that?
What's interesting about it?
What's most revealing about it?
Why would someone do this?
Maybe if the subject matter you're trying to deal with is so unruly and so frightening that the only way you could handle it is by surrounding it with this pretense of fastidious coherence and control.
And I think that there's no question at all that his cool demeanor, his mocking demeanor, and especially his sense when he comes to actually make the film, that all the problems have been solved, that everything has already been done, is an attempt to compensate for or defend himself against the roiling irrational power of the subject matter that his films mobilize.
What a wonderful revealing paradox, right?
He's this fastidious technician, but his themes are the themes of craziness, of madness, of murder, of voyeurism, of violence, of rape, of strangulation, of fear, of pursuit, imprisonment, confinement, injustice.
One the most interesting things about Hitchcock, and those of you who have seen more than two or three of his films will feel this much more strongly than those of you who've only seen a few of them, is how essentially passive, deeply unaggressive, and acted upon most of his protagonists are.
They're being pursued.
They're on the run.
They're confined in small places.
They're full of fear.
One of Hitchcock's recurring moments, a recurring scene, is the protagonist dangling from a height.
And you'll see one magnificent instance of this at the end of Rear Window, where Jimmy Stewart is hanging like this above a void, and you see his face, and you can see his face register abject fear, right?
We could call that one of Hitchcock's iconic moments.
It occurs again and again in his movies, of characters hanging over a void, terrified, terrified of what is about to happen, and often they fall into the void, and some of the dream sequences in Hitchcock show us characters falling through voids into nothingness.
So the subject matter of Hitchcock's films couldn't be more shocking and disturbing in some sense.
They mobilize problematic subjects that are terrifying, and they obviously were so terrifying to Hitchcock that his only way of dealing with them was to surround them with all this appearance of control.
Another thing that follows from this, and it's a very-- on a final point to make about Hitchcock's films, is how often his reassuring or happy endings don't reassure or make us happy.
And the reason, of course, is that he doesn't mean the happy endings.
I'll come back to this again.
But very often, even though his films fit perfectly within-- or imperfectly, but well enough-- into the convention that the endings in a consensus system, and the endings of these stories have to, in some sense, restore normality and reassure us, Hitchcock goes through the motions of doing that, but again and again, the endings of his films are morally ambiguous, and provide us with kind of subtext which put in question the reassurance that we have superficially been offered, as if there's a level of irony, and cynicism, and deconstructive contempt beneath, undermining the reassurance that the endings offer us.
And that's also a part of this idea that the themes that Hitchcock mobilizes are the dark elements of our subconscious and of our unconscious, and the fear that people have, fear of authority, fear of disorder, fear of disturbance, a sense that the ordinary world is full of menace and terror.
But again and again in Hitchcock's films, he doubles his characters, and you'll have a good character and an evil character, and the idea-- he grew up in the late Victorian era of Robert Louis Stevenson, Dr. Jekyll and Mr. Hyde, and he actually thought that was a version of human psychology, that there's a kind of dark or savage aspect in ourselves that's hidden or damped down by civilization, and if we let it come out, we could turn monstrous, we could turn horrible, right?
And again and again in Hitchcock's films, we have a kind of doubling in which one character interacts with a more villainous or murderous character, who we come to recognize as a kind of double of the protagonist, or represents the darker, more disturbing energies within the protagonist's nature, right?
And in Strangers on a Train, what happens, a tennis star, who's having trouble with his wife, is on a train, and a guy pops up next to him and says, I know you're having trouble with your wife.
I read about it in the newspaper.
Let's make a deal.
Nobody could track us.
It'll be the perfect murder.
I'm having trouble with my wife.
You murder my wife and I'll murder yours.
Then the guy said, get thee behind me, Satan, right?
Don't ever talk to me again.
I'm very nervous about it.
A few weeks later, his wife is murdered, right?
And then he starts getting messages from this guy, when are you going to fulfill your part of the bargain?
That's what the film is about.
And it's a wonderful, interesting movie.
The scene I want you to see now dramatizes some of this.
It comes at end of the film, at the great climax of the film, and one reason I want you to see it is that it also shows another aspect of his what we might call the thematic world of Hitchcock's films, and it's his recurring interest in the subject of entertainment itself.
What he became aware of and what his films often dramatize is the illicit and disturbing dimension of our desire to go to the movies, or of our desire to have the kinds of excitements we get when we go to amusement parks.
What Hitchcock understood was that these experiences have an illicit dimension.
We sit in the dark, and what do we do when we sit in the dark?
We watch people take their clothes off.
We watch them murder each other.
We're in the dark.
We're safe.
Nobody knows.
We're solitary.
We're anonymous.
But we're, what are we?
We're voyeurs.
And what he understood was that voyeurism was at the heart of going to the movies, was at the heart of the movie experience, and that there-- we'll come back to this.
We'll come back to this scene.
So at the end of Strangers on a Train, there's a particularly wonderful and dramatic example of Hitchcock looking at the space of entertainment as a space that can turn into an environment of menace and disturbance.
And this is character-- and again, exactly because-- remember, I said one of his deep themes is what we might call the menace of the ordinary.
Well, what could be more ordinary and reassuring than a child's merry-go-round?
Well, here's the great climax of Strangers on a Train in which the good and the bad, the good and evil sides have to engage in a kind of contest or wrestling match.
And as you're watching, note the way in which, characteristic of Hitchcock, characteristic of him, there's a combination of terror and comedy.
He unsettles us, also, because often the most terrifying scenes are leavened with a kind of macabre comedy that unsettles us even more.
Here's the scene.
[VIDEO PLAYBACK] [MERRY-GO-ROUND MUSIC] Good guy, bad guy.
He finally sees his double.
These are very helpful FBI people who do more harm than good.
They shoot at the wrong person to begin with, and they accidentally shoot the person who's controlling the merry-go-round, so it goes out of control.
-Everyone stand back.
Stand back, please.
-[INAUDIBLE], sir.
That's him.
-He's the one.
He's the one who killed him.
-[INAUDIBLE].
We know that
Part of the comedy here is they're wrong.
They don't know who they're talking about.
They have the wrong man, but we in the audience know it.
-Hey, [INAUDIBLE], stop.
-Or do you want to do it yourself?
-No.
I guess you could make [INAUDIBLE].
-My little boy [INAUDIBLE].
My little boy.
Please
See, only Hitch-- Hitchcock had such confidence that he could introduce a comic moment like that in the middle of what is a terrifying climax.
The only director who would do that regularly.
[LAUGHTER] Now you know who the bad guy is, right?
So the hero rescues the kid.
[SCREAMING] Is this a montage or a mise en scene effect?
A montage effect, right?
See the close up on the animal, the way you get very-- the camera itself doesn't allow you to sort of look back and get a sense of the environment?
Your emotions are controlled by the tightness of the shot, by the quickness of the editing.
Comic version of a hero, right?
And this is quintessential Hitchcock.
Are we saved?
Very helpful.
All right, that's enough.
You get the idea.
[END PLAYBACK] And so the fact that this space of entertainment-- can you get lights up?
The fact that this space of entertainment could become a-- one more minute, people-- could become a space of terror is the point.
So finally, what would we say about Hitchcock?
He's a double man.
To borrow from the great essayist Montaigne, who said, we human beings are, I know not how, doubling ourselves, so that what we believe, we disbelieve, and cannot rid ourselves of what we condemn.
What Hitchcock tries or condemn or what Hitchcock thinks he hates keeps coming back in his films.
Images of strangulation, of damage to women, images of fear and terror in the face of irrational authority.
These themes keep returning again and again in his film with the power of obsession, but he controls the obsession with that fastidious technical distance that allows these obsessions to reach us in a way that disturbs and enlightens.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
My intention tonight is primarily to provide you with some suggestions, framings that will help in your viewing of these two films.
And we'll also, at the same time, reinforce and extend some of the things I was saying this afternoon about Hitchcock's work more generally.
And about the recurring preoccupations, themes, obsessions, strategies that are characteristic of his work.
And I want to begin by adding one note to what I was saying this afternoon.
You remember I began this afternoon by talking about the relationship between Hitchcock and what I call the "Genius of the System"-- borrowing on that famous passage from Andre Bazin, who said Hollywood is very interesting, and there are interesting directors, but why don't we talk about the genius of the system-- the way the system itself encourages or enables complex forms of narrative.
And we've already talked several times earlier in the course about this topic.
But there's another way in which one can think about this issue with regard to Hitchcock.
And I think it's helpful in clarifying, so I want to mention it again.
One way to think about Hitchcock is to think of the Hitchcock film as a kind of genre unto itself.
Right?
The Hitchcock.
Just like the Western, or the screwball comedy.
Broadly, Hitchcock belongs to the broad category of what might be-- well, of two categories-- the horror film-- although, he's a strange director of horror.
He certainly is incredibly influential.
In fact, when he made Psycho, he publicly announced, "This is my first horror film."
Even though there are elements of horror earlier.
And it made something of a publicity splash when he said that.
So one way to understand Hitchcock is to think of him as participating in at least two categories of genre or subgenre that are more and more widespread than Hitchcock's work itself.
And we might call that the horror film and the mystery film.
And even that's a bit of an oversimplification, because within the category of mystery, you can have different sort of subcategories.
One of which is the sort of romantic adventure of the sort that North by Northwest.
The chase film starring Cary Grant that Hitchcock made that ends with characters climbing across the face of Mount Rushmore.
So that's one sort of action-adventure version of a mystery story that Hitchcock makes.
Another version-- other versions are examples of the films that we're seeing tonight.
But my point is that although one could fit Hitchcock's film into some sort of larger category-- of mystery or mystery and adventure, something of that kind-- it's also the case that once you begin to look closely at Hitchcock's films, you can see that there's such a profound, almost obsessional coherence in them.
They keep returning again and again to those dominant themes I talked about this afternoon.
That one can see that they, in some sense, constitute a kind of category of their own.
If we think of-- so if we think of the Hitchcock as a kind of genre, even within that category we can discover immense variation.
So that Hitchcock's example itself becomes a kind of crystallized, or distilled example of the principle I've been talking about all semester concerning the way in which a system like the Hollywood system could engender variation, even as it is also committed to giving its audience something that seems familiar, that they recognize, that they're willing to attend to week after week after week.
I think the primary point is this-- in a system in which you have any habitual, a regular, a daily, or a weekly relationship to the material, there have to be elements of familiarity, of repetition.
Of a return to shared subjects and themes.
Or you wouldn't do it.
You would find it intolerable every time you made what was a routine or habitual connection to a text, if what it gave you was a completely new experience which forced you in some sense to begin at the beginning to figure everything out for your-- you would find that you would resist such an experience.
It's one thing to go to a great overwhelming theatrical experience once or twice a year, or once or twice a month.
It's quite another thing to try to watch King Lear every day, or even every week.
So part of the difference has to do with the need to what it means to have a habitual, rather than an occasional, connection to the artwork.
And one of the fundamental-- well, maybe the most important thing about the Hollywood studio film, and later the most important thing about American television narrative forms, is the is the intimate, regular, routinized connection that the viewer has with the material.
And in the case of Hitchcock, what we can say is, really from the very beginning of his work to the end, there are these fundamental, emotional, and strategic, or technical coherences, resemblances, that made virtually every Hitchcock film a kind of commentary on an earlier one.
And the two films you are seeing tonight are a particularly rich example of this.
So I hope you'll think hard about the way in which they are both profoundly different from each other, and also at the same time, strangely, mysteriously, profoundly similar.
How they circle back on the same kinds of obsessions and the same kinds of problems.
And part of our pleasure in looking at the two films is the simultaneous recognition of the difference and similarity.
Well, that is an example in small compass of the larger principle of popular aesthetics that I was saying needs to apply to the way in which we understand the audience's relation to the classic Hollywood film.
So as you're watching these two films and thinking about them, think both about resemblances and differences.
And you'll get some sense of how much remarkable variation is possible within what remains certain familiar conventions, expectations-- not only about the content of the text.
But also about the way the text is organized, about its formal features.
So what I'd like to do then is say a couple of introductory things about each of the films you're going to see.
And if I'm able to keep the time right, maybe we'll pause for one clip.
Maybe we won't, depending on how concise I'm able to be in my arguments.
You'll notice that Shadow of a Doubt and Rear Window are 11 years apart.
And I think that's significant because one of the most interesting ways to think about comparing the two films is to think about the way in which Rear Window seizes on many of the preoccupations and themes that are at the center of Shadow of a Doubt.
But deals with them in ways that one might say are even more disturbing and profound, that press more deeply into the implications that are mobilized in the earlier film.
And maybe that sounds too general and abstract and mysterious.
Let me be specific about what I mean.
One example is-- one of the most interesting aspects of Shadow of a Doubt, if you look down to my next to the last item, is what I call the subplot.
And the subplot in Shadow of a Doubt involves essentially a kind of subsidiary story about two very ineffectual characters.
In this subplot, the husband of the family has a kind of-- his main preoccupation when he's not working at his bank-- is he has an ongoing conversation and sort of reading group with his friend, played by Hume Cronyn.
Also as a diminutive man, also a man who seems to lack masculine force.
He's short, he's not aggressive.
And they spend their time reading murder mysteries.
And they spend their time trying to devise the perfect murder.
So part of what's funny about this, but also macabre in its own way, is that while these two sort of capon like older men-- who stand for a kind of impotent maturity, a kind of useless, unattractive maturity-- they spend their time and their diversion is to try to invent or imagine murders and mayhem.
When in fact a real murderer is living at the heart of the family.
And Young Charlie herself is actually in danger, even though her father and and no one around her in the world understands that that's the case.
Although the viewer comes to understand it fairly early.
And even Young Charlie comes to a recognition of it before the film is over.
So in this subplot, what's going on is we have two sort of characters playing at the diversion of murder, and finding stories about murder a kind of escape.
And in fact, one of them is actually trying to devise the way to kill someone and get away with it.
And what we're supposed to feel is that this subplot is a reminder to us about a number of things.
But most disturbingly and subversively, it should be a reminder to the audience of what the audience is doing sitting and watching Hitchcock's film.
Because the equivalent of the audience in the movie is-- one equivalent of the audience in the movie are these two sort of foolish people who, immersed in, and in some degree surrounded by evil and authentic danger are living out their silly diversions by reading silly magazines about murder stories.
So it's a variation on Hitchcock's obsession, or deep, deep recurring interest in the space of entertainment.
And what it means to want to go to entertainments.
And there's a certain sense in which every single Hitchcock film enables or is grounded upon this experience of sitting in the dark-- actually what sort of equivalent to what the Jimmy Stewart character does in Rear Window-- sitting in the dark, anonymous peeping watching the lives of other people.
And the idea is not only that you're anonymous, but in some sense there's something sort of illicit about it.
As if in the process of doing it, you're indulging what are base desires or at least not totally respectable impulses.
But you're doing it in a space of comparative safety and a place of anonymity.
So the subplot in Shadow of a Doubt, in a certain sense, raises self conscious questions about the nature of the entertainment that we ourselves are watching as audiences for the film.
All right?
And it's clearly a major dimension of Shadow of a Doubt.
It's one of the things that make Shadow of a Doubt a rich movie.
There's a kind of cunning, cynical complexity in it that careless viewers, or very superficial viewers won't pick up on.
And it's really possible, I could really imagine, that they were viewers who came away from Shadow of a Doubt fully satisfied.
Thinking, oh yes, evil has been purged.
What a nice mystery story!
Never fully picking up on the dramatized elements of the text, which put in question the ease with which we embrace these forms of entertainment, the ease with which we accept murder and mayhem as a form of diversion.
And some of the implications of that for human nature and for our ideas of entertainment itself.
So that moment when in Strangers on the Train when the merry-go-round goes mad and becomes an instrument of terror and murder is another example of Hitchcock's interest in the complexity and problematic nature of our connection to what we call entertainment.
And one of the things you can see when you shift to Rear Window which you can see how these preoccupations have been distilled and clarified.
Because the entire field becomes a meditation on the nature of our voyeuristic impulses.
The entire film becomes a story about seeing and not being seen, about following-- about shadowing people, about hidden impulses or potential doubles of our own respectable nature.
And the way in which Rear Window is self-reflexive or self-conscious is much more systematic and much more deeply embedded in the experience of watching the film than was the case with Shadow of a Doubt.
The same theme, the same self-consciousness, the same awareness of the problematic nature of entertainment, but now much more systematically and fully integrated into the formal and thematic nature of the text.
So I love Shadow of a Doubt.
In many ways it's one of my favorite Hitchcocks.
But it's early Hitchcock, and you can feel the much greater rigor with Hitchcock has begun to seize on the implications of his own problematic relation to his material.
And his audience's compromised, complicit relationship to the material that Hitchcock mobilizes in his film.
So Rear Window was more systematic in some sense, more coherently conscious of this element of its meaning.
It's almost as if by the time we get to the mature films of the '50s and beyond-- the period between, let's say, 1951 in the early '60s, when Hitchcock did all of his most powerful work-- it's almost as if we can say that the films of this era have a kind of double significance.
That all of these films-- all the films after Shadow of a Doubt are partly about filmmaking.
They're partly self-conscious attempts to be meditations on the nature of movies.
And it's almost as if there's a kind of separate story, a separate plot.
There's the traditional kind of mystery story that, is characteristic of many films.
But then there's this meta story, this separate story.
It's the story of the filming.
The story of the making of the film.
The story or the drama of watching-- the drama of the watching, we might call it.
The literary version of this would be the drama of the telling.
Right?
There are literary-- if you think of a novel like-- a novella like Conrad's Heart of Darkness.
I think I've mentioned this before in the class because it's such a clear example.
But the story of Heart of Darkness is really two stories.
It's the story of the narrator's trip up the Congo River when he was a young man.
But it's even more of the story of his struggles to tell the story 20 years later.
And when you read Heart of Darkness, naive readers hate it, can't figure out why it's taking so long.
And why it's taking the narrator so long to get to the point.
But of course, the story is not primarily even about the adventure he's trying to talk about.
It's about the struggle to tell the adventure.
So that there's a kind of double plot in that novella as there is in a lot of great modernist fiction.
As there is in another way in a lot of great modernist paintings, where you are aware not only of the subject of the painting, but of the materials that go into the making of the painting.
I'm thinking of the great impressionist and post-impressionist painters, right?
Or of the painters who would paint-- like Monet who would paint 50 versions of a haystack in different lights.
Or 32 versions of Rouen Cathedral in different lights at different times of the day.
Why?
Well, one of the reasons is, he's trying to say that reality is not stable.
That Rouen Cathedral changes depending upon the light, depending upon the weather, depending upon my emotion when I look at it, right?
In other words, reality is much more complicated.
And so that when we look at the Rouen Cathedral paintings, we're aware of paint.
We're aware not just of the fact that a cathedral is being represented.
We're aware of the process whereby Monet went through the terrific labor of capturing the image.
And you can see there's a lot of examples even on painting of the thickness of the paint.
When you look at Monet's paintings, very often the paint is so thick that if you see in reality you want to reach out and touch it.
So you're aware of the materials that go in-- again, another kind of self-consciousness or self-reflexiveness.
Well, Hitchcock belongs in that tradition, I think.
And a great many of his mature films have this double plot.
Right?
The second plot is what we might call the drama of the watching, the drama of watching.
And Rear Window was a particularly austere ingredient instance of this topic.
Well, let me start over again with Shadow of a Doubt, set the context.
It's World War II was going on.
I already mentioned that Hitchcock had already felt ambivalent about leaving England at the end of the 1930s.
And now, by 1943, this anxiety-- because the war was on, England was being attacked.
Hitchcock felt ambivalence about that.
His mother was tremendously ill, he kept receiving information.
And she couldn't get back to England because the war was going on.
You couldn't travel there.
And it was said that he was afflicted by a kind of a really deep guilt and unhappiness during the filming of Shadow of a Doubt.
And during the filming of Shadow of a Doubt, his mother died.
Now there are some film critics who say, Oh well, the port-- well there's one critic who writes that the portrait of the mother in Shadow of a Doubt is Hitchcock's last benign portrait of a mother.
But if this is a benign portrait, I'm very nervous about what is a mean portrait.
And it's a very bad mistake.
I mean, it can't be that Hitchcock was trying to memorialize his mother in the primary character in this film.
Because as you'll see, she's a magnificent airhead.
The only dumber woman in film that I know is Annabelle Lee in The General, who sweeps out the engine while they're trying to escape their enemies.
And as you'll see, she's a figure-- she's a benign figure in the sense that she has not a mean bone in her body.
But she's really a dullard.
She's really silly.
And she silly in ways the film is aware of.
But we'll come back to that.
So there are elements in the film of autobiography, but there are skewed by Hitchcock's own sour, cynical, disturbed nature.
And this portrait of a family is very far from being a celebratory portrait, as will say in a moment.
So Hitchcock's in exile.
He's feeling a bit sour, he's feeling alienated.
So he makes a film in which-- really his first film, or one of his first films-- in which an American family is at the very center of the story, right?
But he treats this family with an odd kind of comic irony or distance that careful viewers immediately see and lazy viewers might not see.
But the careful viewers definitely see.
And one way to pick up on how subversive his portrait of the family is, is to compare him to his great contemporary Frank Capra.
We've seen a Capra film already, of course, that standard and establishing screwball comedy.
But the more characteristic Capra films-- all of these kinds of film were also coming out at the same time-- are even more fundamentally celebrations of American life.
And maybe the quintessential instances of this as a film, that anyone who lives in the United States for a few months around Christmas time becomes sick of, what's the film?
Starring Jimmy Stewart, who was also the star of one of tonight's Hitchcock film, Rear Window.
What's the film?
It's a Wonderful Life, yes.
And in fact the title tells-- Capra-- although there is a dark side to Capra, too.
But Capra's the great American sentimental and optimist.
He celebrates small town virtues, celebrates-- and the villains in Capra's films are Big Business and people who are hostile to the values of small town America.
Hitchcock-- and one way of understanding what Hitchcock is doing in Shadow-- so I feel like it's A Wonderful Life-- also, incidentally, these films often star Jimmy Stewart-- James Stewart.
But when Hitchcock uses James Stewart, this upstanding, perfect example of middle American decency and American honesty is turned into a much more fearful, damaged, impotent sometimes sexually perverted or implicitly sexually damaged adult.
And it was Hitchcock's version of Jimmy-- the Jimmy Stewart that Hitchcock draws out is a much more disturbed and damaged character.
And part of what makes Stewart's performances in Hitchcock's films so powerful-- there's some people say Hitchcock and Stewart's collaboration was one of the great collaborations between a director and an actor in the history of movies.
And you could see one great example of it in Rear Window.
But the primary thing to recognize is that what Hitchcock is doing something so subversive, because Jimmy Stewart was already a kind of American icon.
He stood when he-- except in Hitchcock's films-- he stood for a form of American decency and openness and honesty and sincerity, goodness and even heroism.
Which Hitchcock undermines in all kinds of-- I don't mean that he makes Jimmy Stewart an evil character, he doesn't!
And he him his outwardly heroic lineaments.
But then we discover that there are things wrong with him, as you'll see in Rear Window.
And he does this again in Vertigo, an even more dramatic example of Jimmy Stewart playing an impaired or a damaged character.
In Rear Window, Jimmy Stewart has one leg that's broken, right?
In a cast for all the film.
As you'll see.
At the very end of the film-- one of the outcomes of the film is what-- in the denouement-- we see Jimmy Stewart after the action has occurred and now he has two broken legs.
So he's back where he was in the beginning.
He's even more damaged, and impaired, and immobilized than he was before!
And we'll come back to this, because there's something very disturbing about this ending in a certain way.
Even though it purports to be a kind of survival tale and in that sense, a kind of happy ending.
But the important point for the moment is that Hitchcock was certainly fully aware of the fact that what he was doing was juxtaposing his much more sour, unsentimental, even cynical vision of American life against Capra's more sentimental one, right?
So it's as if he's against Capra.
Or he's against Capricorn.
The corniness.
Get the pun?
I'm stealing it, I'm stealing it.
But the corniness, the sentimentality of a characteristic Capra film.
And the characteristic attitude in the Capra film towards small town American life.
And one of the ways to recognize this is to recognize a kind of recurring trope of his that I've mentioned before.
Let's show it, Kristen.
So here is the opening Shadow of a Doubt.
I'm giving you a chance to watch it and then savor it the second time when you see it again, in 20 minutes, half an hour or so.
And it dramatizes a scene, a version of which occurs in many Hitchcock films.
Watch what happens.
In which the effect of the opening is in some sense to implicate the viewer right at the beginning in an act of voyeurism.
[MUSIC PLAYING]
See?
A benign scene.
A benign cityscape.
[MUSIC PLAYING]
We go up to the window.
In Psycho, it's even more dramatic.
The film actually crawls under a window shade that's almost all the way down.
But you see what the camera did?
[KNOCK ON THE DOOR]
Come in
OK, stop.
When we first see Young Charlie in the film many, many scenes later, what we've just looked at will be replicated, shot by shot.
It's an example of what I call rhyming scenes.
And I'll come back to that in a moment.
So behind any door or window is potential evil.
And the way that we go out in the street, that then the camera looks up at a window, then it goes implicitly through the window.
And as I say, in some Hitchcock films, the slithering through the window is actually even more explicit.
And you actually feel that the camera has slid in, as if it's a kind of unwelcome and unobserved witness to the event.
And the implication is that you are watching something that you shouldn't be watching, or that you have access to worlds that normally you would never have access to.
So behind any door window in Hitchcock's world can be evil, danger, fright.
An American town, an American family.
I've already mentioned that one of the things that's going on in this film is a portrait of an American family that has a kind of ironic undercutting built into it.
We can feel almost from the very beginning when we first go to Santa Rosa.
Our first shots of Santa Rosa-- which will occur shortly after the scene you've just seen.
You'll see, there will be a more elaborated scene in which we learn a little bit more about Uncle Charlie, and then we shift to Santa Rosa.
And the very opening scene in Santa Rosa shows a sort of fat, jolly policeman in bright sunlight directing traffic.
As if nothing could be more wonderful than the job of directing traffic in a small American town in sunny California.
What could be more noble?
But there's something excessive about it.
I mean, you almost feel, even before you fully realize that what the film is going to be doing, you sort of feel that this is too much.
And this is characteristic of Hitchcock as well.
Almost as if there's an excess of brightness and an excessive jolliness.
As if it's more than we could possibly believe.
And then, as the family's nature begins to unfold to us, what we begin to see is these children are horrible, insipid children.
They're hardly admirable.
The daughter's a nasty little bitch who is more interested in reading her books that in speaking civilly to her parents.
Not long Charlie, but Charlie's younger sister.
There are three children the family.
Charlie's the oldest, right?
And then, she has a sister and a younger brother.
And the younger brother and the sister are insipid, nasty little children.
The boy when we see him is constantly saying, what-- when he first sees Uncle Charlie, he says, "What have you got?
What present have you gotten for me?"
And if we don't think that the children are very attractive, wait till we look more closely at the fact the parents.
I've already mentioned the father, who spends most of his most of his time with Hume Cronyn trying to talk about ridiculous diversions in murder mysteries.
Meanwhile, his family is in danger around him and he doesn't have any sense of it.
But the wife, as I've mentioned also, comes across as one of the supreme airheads in American movies.
There's a certain moment when Uncle Charlie is under suspicion, so some FBI agents come to town in disguise.
And they suspect him but they don't have evidence yet.
So they claim they're from a magazine, and they come into the family and they say, "What we want to do is we want to do a story about a typical American family."
And the mother of the family is very proud.
"Oh, isn't it nice to be a typical representative American family?"
Well, who would want to be the mean, right?
We're the mean, we're the norm, right?
We are the typical-- but she loves this idea.
And then, not only does she love the idea, she's really rather a silly person.
So at one point when they're getting ready to sort of-- I don't know whether they're getting ready to do still photography in the house-- she insists on completing her cooking because she wants to finish the pie before the photographer gets there.
As if completing her cooking has something to do with what the photographer would find.
And you find repeatedly that she's kind of a little askew.
She certainly has no possible intimation that her brother is a murderer and in fact a cynical nihilist that has no respect for anything in life.
And she never learns any better, and that's part of what makes the film so disturbing.
So what happens in the course of the film essentially is that this American family is systematically shown to be sort of silly and foolish, banal in its ordinariness.
Hardly something that we might fully-- I don't mean to say that we come away in horror thinking, Oh!
How disgusting this family is!
Just that we think how ordinary, banal, silly, foolish it is.
One of the ways I've talked already about the way in which Hitchcock is obsessed by doublings.
And by the idea of a self divided.
And one of his richest treatments of that theme is in the treatment of Uncle Charlie and Young Charlie.
And you'll see how explicit the film is about that.
One of the ways it's explicit about it is that Hitchcock uses particular visual strategies to reinforce the connection.
And I've already mentioned the rhyming scene in that opening scene.
Watch it, because it's quite exciting.
Later, in our first introduction to Young Charlie, the camera behaves identically to the way it behaved when we saw Uncle Charlie.
And so even before Uncle Charlie has entered the story, even before we're fully aware of what the relationship between Uncle Charlie and Young Charlie is, the camera has told us that they are linked.
And we can say then that in the doubling Young Charlie represents a kind of innocence and part of her night-- one way to understand the film is to see it as a Bildungsroman, as a growing up, as an initiation story in which Young Charlie learns something about the evil and difficulty of the world.
She has a naive and simplified view of the world, and before she's done, she comes face to face, not just with murder, but with the possibility of murder.
And in fact, Uncle Charlie actually tries to murder her, because he realizes she knows the truth about him.
So she's in danger for part of the film, and knows she's in danger.
And part of the issue is that she can't really talk about it.
Why not?
Well, it's her uncle!
Have you ever heard this kind of thing?
There are actually critics who say that although that it's never made explicit in the film, that one of the ways to understand the film, and at least in the subtextual way is to think about the relation between Uncle Charlie and Young Charlie as that between the sort of perverted uncle man who manhandles his young niece when his family isn't watching, right?
And there actually is in the film what is a scene sometimes called a betrothal scene in which we see Young Charlie come down a staircase, like a traditional bride down a staircase, and Uncle Charlie gives her a ring.
So that's the film does insinu-- and that's really perverted.
The film doesn't actually say anything sexual is going on between uncle and niece.
That's not-- but there are these suggestions of illicit or disturbing connections that are part of the subtext with which we experience the film and with which we recognize the tangled, disturbing quality of the connection between Young Charlie and her uncle.
So the naivete on Young Charlie's part is her failure to understand or to even to be able to imagine the extent of evil and duplicity in the world.
And we could say that what happens as she comes to recognize what Uncle Charlie really is, and what Uncle Charlie really stands for, she is educated into the ways of the world.
But it is in many ways a very terrible and very serious initiation.
And one of the climactic instances of this occurs toward the end of the film when Young Charlie and her uncle meet together in a bar called the-- think it's called the Till Two because it closes at 2:00, which is very late.
In the universe of the film, 2:00 AM is very late in Santa Rosa, California.
They meet in this place, and this is where Uncle Charlie realizes he can no longer hide the truth.
And he talks openly to Young Charlie, and he says at one point this.
"We're old friends, Charlie.
More than that, we're like twins.
You said so yourself.
You think you're the clever little girl who knows something.
There's so much you don't know, so much.
You're just an ordinary little girl living in an ordinary little town--" You hear the contempt for ordinary there?
Hitchcock shares some of this, even though his villain is saying it, right?
"You go through your ordinary little day at night and you sleep your untroubled, ordinary little sleep, filled with peaceful, stupid dreams.
You live in a dream--" This is a direct quotation.
"You live in a dream.
You're a sleepwalker.
Blind.
How do you know what the world is like?
Do you know the world is a foul sty?
Do know that if you rip the fronts off houses you'd find swine?
The world's a hell.
What does it matter what happens in it?"
Now what an extreme nihilistic statement.
That's what Uncle Charlie stands for.
That's the full ugliness and disturbance that Young Charlie-- and, of course, a murder is speaking to her.
This is also implicitly a confession to her.
"Yes, I have killed people."
So it's a tremendously disturbing moment in the film, and it represents in a certain way Young Charlie's coming of age.
But the irony, or one tremendous irony of this tale, of this story, is that when we get to the end of the film, there is a profound kind of ambiguity.
Yes, the evil has been purged.
I won't go into the details of the plot so as not to spoil your experience of the film, but Uncle Charlie is eliminated from the scene in a-- and Young Charlie survives.
It's a kind of reassuring or happy ending.
But what undercuts it?
As all, remember all, of Hitchcock's endings are undercut, they're never really happy.
They're never really reassuring.
They go through the motions.
They're conventionally reassuring, but they're not.
The moral ambiguity on which this film ends is maybe it's most disturbing aspect.
Because what it says-- because Young Charlie never tells her mother, never tells her father, never reveals to the community that Uncle Charlie was this evil figure, was the Merry Widow murder.
Why not?
Well, maybe she sparing her mother.
Uncle Charlie's no longer there, what's the difference?
There's no discussion of this.
But what does it mean?
It means that when the film is over, the idea that evil might lurk at the heart of the American family remains a reality that no one has acknowledged.
In other words, it's as if the implications of the story are denied bye Charlie's silence.
And we're left in the situation at the end of the film in which the implications or meaning of Uncle Charlie's life has been suppressed.
And therefore, the significance of his pervasive nihilism and of what it means that such nihilism could exist at the heart of a typical American family is never acknowledged or known by anyone inside the film.
Even though it's a disturbing knowledge that we take away from the film.
So to conclude on this film, one of the things to think about as you're watching the film is how remarkably Hitchcock is able to give us what might be thought in many ways to be, and what is in many ways, a light entertainment.
A humorous and very well put together mystery story about an American family full of the wit and scene by scene richness and complexity.
A film that even naive film goers can enjoy fully.
And yet the full implications of it, the moral ambiguity with which the film ends, and the moral ambiguity that's a really implicit all the way through the film, the sense of the ominous and the sense of disturbance remains with the viewer after the film is over.
It's a characteristic way for Hitchcock to end films.
Rear Window, as I've suggested, is an even more rigorous exploration of some of Hitchcock's deepest preoccupations and obsessions.
Let me remind you to really savor the opening scene of the film.
It starts under the titles, and you get the title sequence, and onto the title-- it's a sequence that lasts three or almost four minutes.
And it's completely wordless.
There is no dialogue.
Watch how much what happens.
It's a complicated moment in a certain way.
But if you are attentive to it, you can see what's happening.
Essentially, what we see the camera do-- first, we see the blinds go up on two open windows.
And then we realize that they're open windows, they're encasement windows that are open unscreened.
And after the blinds-- the camera is stable, it doesn't move-- three separate blinds go up one time.
They reveal the other side of an apartment complex being seen through the window of another apartment.
And then, what we see-- the camera then goes on and sort of elaborate journey.
You're caught in the camera's gaze, as a viewer.
Once the shades go up, then the camera begins to move.
It moves toward the window, then it moves out the window, and then it looks down and it does a kind of circuit of the-- essentially, it shows you what can be seen from this window.
It does a kind of circuit of the physical spaces that are open it if it looked at this world from this window.
And you see it might make a kind of circle, it's not a 360 degree turn but it's more than 180.
The camera turns like this, it pans, it turns around, and then it comes back into the room.
And when it comes back into the room-- it goes out of the room and it looks around, and then it comes back into the room, through the same windows, backs up further, and then you see Jimmy Stewart lying there, sweating in the heat.
The camera then looks up a wall at a thermometer, and you see the thermometer is hovering in the mid 90s someplace, explaining why Jimmy Stewart is sweating so much.
But then, the camera moves outside again and does another circle of the space that you just looked at.
It's almost as if it wants you to look at a second time, to remind you of the space of the action in the film.
And of course, we don't fully realize it at the time, but this is a rehearsal for the whole movie, because we are going to be confined in exactly this way for the entire film.
We're never going to get out of this room.
Everything we see will be seen through this window.
Then the camera come pulls back in again, we see Jimmy Stewart again-- so it's almost as if that opening sequence does the same thing twice.
What it mostly does, of course, is trap the viewer in an act of voyeurism.
It traps the viewer completely in an act of spying.
And then, almost the very first words of the film, the camera pulls back, it looks at a few items to tell us a little more about the fact that Jimmy Stewart is a photographer who works for a magazine, who was injured taking one of his action photos.
He can't wait to till he's out of his cast, but he has another week to go and he's going stir crazy.
And then he has a conversation on the telephone with his editor.
And at the same time engages in various acts of voyeurism, including especially a young woman who is scantily clad across the courtyard in her room wearing shorts and a very revealing brassiere doing sort of exercises and so forth while she's eating her breakfast.
And you see the Jimmy Stewart character voyeuristically savoring her as he's talking to his friend.
And then he hangs up the phone, a knock comes on the door, a woman enters.
It's his cleaning lady played by the great actress-- what's her name?-- Thelma Ritter.
And Thelma's almost first words are, "We've become a nation of peeping Toms."
And I mention this just to show you how aware the film is, how much the film wants us to be aware of the fact, that voyeurism is one of its deep subjects, right?
Why do we have this desire to watch the illicit?
What is it about us that wants to make us sort of look through key holes and peep at people and watch the intimate lives of people?
So they saw the opening scene is a scene that dramatizes the confinement and the voyeurism that are at the very heart of what the movie is about.
We can say that the whole essay, the whole film in some sense, is a kind of essay on seeing.
And in that three or four minute sequence in the beginning without dialogue, the camera is the star, in a certain sense.
We become aware of the camera, as Hitchcock wants us to be, in a way we would never be in another kind of film.
There are other forms of elegance in the film.
And I hope you'll watch the way the camera behaves, because it's a marvel of complexity and intelligence.
But there's another way in which there's an elegance of structure in the film.
As the camera in the very beginning makes its tour of the building, what begins to emerge in a very elementary way at first, and then as the film precedes becomes more fleshed out, is that there are three or four basic stories about people who live in the apartments that Jimmy Stewart is looking into.
And each of them tells a separate kind of story.
There's a Miss Lonely Heart story about a woman who was pining for a boyfriend and who feels very lonely and maybe toying with the idea of suicide.
Then there's a story about a composer who's having trouble with his work.
They're a series of sort of subplots, mostly trivial ones.
Except for the primary one-- what's the primary one?
A murder may have been committed in one of the apartments to Jimmy Stewart has been prying into.
And he begins to investigate the murder and that puts him in danger himself.
And the people involved with him, including his girlfriend Grace Kelly, get involved in the danger as well because he's a immobile.
And Grace has to be the one who walks across the way, and does-- So the elegance of the structure of the film has partly to do with the way in which these subplots work themselves out.
And at the end of the film, each of these little subplots that we'd seen being played out in these different apartments-- only watching them through the windows of the apartments-- each of them is resolved in some way.
And I want to suggest that their resolution, while from one angle is very satisfying-- there's a kind of elegance of structure.
We're very satisfied by this sense of completion.
But at the same time, I think the very fact that these stories are all tied up in such neat knots makes us feel that there's something a little too neat, a little too perfect about it.
In other words, the very perfection with which these subplots are tied up at the end suggests a kind of excess that Hitchcock wants us to be suspicious of.
As if of this is just too perfect.
And of course, there are other ways in which the ending of this film is deeply ambiguous.
And I'll come back to that in a moment.
I also ought to mention the way it which, in a quiet, subtle way, this film is an exploration of both class and gender issues.
Because there's a kind of little sort of contest going on.
Grace Kelly, who's this elegant upper class woman who gets her clothes at the fanciest fashion stores and has food catered from the fanciest restaurants in the city and who reads magazines like Vogue and so forth-- she stands for sort of moneyed elegance.
And there's a kind of tension between them.
She wants to marry Jimmy Stewart and Jimmy Stewart's resisting.
One reason is that he doesn't belonged to her class.
He's more of a working class type, right?
He's a down to earth sort of guy, and he's worried about that.
And he's also nervous because like many American males, especially in the movies, he's nervous about being entrapped by marriage, as if he isn't already entrapped in all kinds of ways.
And in an odd kind of way, we could say that the Thorwald marriage-- that's the name of the murderer and the wife he supposedly murders and chops up, at least that's what Jimmy Stewart comes to suspect.
We could say that the Thorwald marriage is a kind of double of the marriage that the Jimmy Stewart character fears he will have.
That his resistance to Grace Kelly is partly a class resistance.
He feels uneasy around this aristocratic woman because her tastes seemed to him banal and trivial.
In any case, they seem to him to involve values and commitments that are at odds with a sort of rough and tumble life of a traveling photographer who goes to war zones and other dangerous places.
But that's his image of himself.
So his resistance to her is partly a desire to maintain his freedom, right?
And it's partly a class issue.
So that both issues of class and gender are involved.
The gender issue has to do with how men and women, especially in the stereotyped way of the '50s and '60s, regarded the question of marriage.
And there are other marriages or relationships that are alluded to in the film, and we can say that those marriages and relationships also double the relationship between Stewart and Kelly.
So there's a kind of low level, not exactly hostility, but a low level badinage and argument going on between Kelly and Stuart all the way through the film.
Which is in some sense resolved, but not fully resolved.
Or resolved in a typically unsatisfying Hitchcockian way at the end of the film.
So the very ending of the film.
There is the tradition-- the murderer is caught, is exposed.
The danger to which the Jimmy Stewart character was exposed is purged by the end.
Again I won't go into details if you've not seen the film, but I need to say this much in order to explain the ending.
And at the end of the film, as I said, there's a moment where we see him-- the murder finds out that he's watching him.
And after a while, he comes to Jimmy Stewart's apartment apparently to try to murder him.
Because he thinks he's going to reveal that he's a murderer himself.
And it's a terrifying scene.
The great climax of the scene-- Stewart-- his physical limitations some critics have seen as a symbolic embodiment of what might be seen as a form of impotence.
Both a sexual impotence and sort of broader kind of impotence, an incompetence because he's crippled, he can't fight, he can't defend himself.
He can't behave like a traditional man in a traditional movie.
And again, remember, this is characteristic of Hitchcock.
His men are almost always like this.
It's amazing, in fact, how weak, vulnerable, damaged, fearful, frightened his protagonists are.
So there's this climactic moment at the end where Thorwald, the murderer, is trying to kill Jimmy Stewart, and he pushes him.
And Jimmy Stewart is holding on.
He's on the second story window.
He's holding on like this, looking up.
The camera's looking down at him.
You see the fear and horror in his face.
He lets go.
There's a critic, a brilliant critic, a French critic, who talks about the idea of letting go, of losing your grip as central action in Hitchcock's films.
You lose your gripping.
You fall into a void.
And that moment that is, as I said this afternoon, could be said to be an iconic moment in Hitchcock's movie.
So when Jimmy Stewart actually lets go and falls.
You see his face as if he's falling to his-- you think he might be falling to his death.
But of course, he's not.
He just breaks his other leg.
And at the end of the film then we have a final scene.
And this is where the ambiguity comes in.
We see him with both legs, now.
He's back where he was.
But the Grace Kelly character is sitting happily in the room.
She pretends that she's reading material that would please Jimmy Stewart.
But in fact, if you look closely, she has a copy of Vogue hidden inside the material that she's reading.
And there's a very deep sense that in this gender conflict between Grace Kelly-- and also we discover at the end but they're now engaged.
So Jimmy Stewart has become engaged to this woman, right?
So it's almost as if he's sort of lost the battle.
Or at least he's agreed to marry.
But what we feel that the end is that his agreement to marry her is still full of the same ambiguities that were present before.
And many people watching the film have said, you know the Grace Kelly character is very beautiful and very elegant, but she's not a very attractive person morally.
She tries to manipulate the Jimmy Stewart character.
She doesn't seem fully to appreciate him.
She doesn't see why he wants to run around the world in corduroys and sneakers instead of living the high life in New York and go and going to Le Cirque for dinner.
She can't really understand that.
So the conflict between them is a kind of lifestyle conflict.
At the end of the film, none of those things have been resolved.
And there's the sense that he's more entrapped than ever at the end of the movie.
Now, I don't want to make too much of this.
This is a light entertainment at one level.
But these subjects that I've been talking about, the sense of loosing your grip, of being caught in a confined space and being terrified by forces far larger than you, that you're too weak or damage to resist.
Those moments of horror and terror are dramatized with profound authority in this remarkable film.
So what Hitchcock repeatedly does, and especially in these two films, and if you can see them in a conversation with each other you'll understand more fully, the fascination of Hitchcock's career and the fascination of his work.
Because he keeps returning with these remarkable variations again and again to the same set of problems and of subjects.
So there's even more ambiguity and uncertainty at the end of Rear Window than there was when it began.
And one final point and then I'll stop.
One of the things that we notice in the course of the film is there's one moment in the movie where the Jimmy Stewart character responds to Grace Kelly with a kind of excitement and allure.
It's almost the moment where you see suddenly really desires her, really sees her as a possible partner.
And it's the moment in which she has entered the voyeuristic drama.
It's the moment in which he's watching her through his lens when she's gone across to try to do some detective work in the murderer's apartment.
And there's a kind of excitement and adventure that goes on there.
And it's actually the moment when they bond.
When they realize-- and there is some sense that they're both engaged in a morally exciting, in a mutually exciting adventure.
But there's also the sense, very disturbing in a way when you reflect on it later, that the moment in which Grace Kelly became truly attractive to the Jimmy Stewart character was the moment in which he had entered the voyeuristic drama.
In which Kelly was now distant from, which he was attracted to her because he was seeing her as if he were a peeping Tom.
So now this is a subtext.
I don't mean that I don't mean that the film is using a megaphone to remind us of these implications.
But the number of them build for the course of the film.
And as with Shadow of a Doubt, what seems on the surface to be a kind of light diversion, once we begin to press its details becomes a much more disturbing and powerful meditation on our human impulse to watch what we're not supposed to watch, to participate in illicit activities without being punished for it, especially if we, like Jimmy Stewart, are able to do so in the dark.
And in that sense, when we step away from Rear Window, one of the things we might recognize is that we in the audience have been engaging in the same activity that Jimmy Stewart is engaging in the movie.
That we are complicit as viewers in the immorality and in the murderous tendencies of these forms of entertainment.
So it's not possible if you're an alert and self-conscious film goer to watch these apparent light entertainments without recognizing their deeper and more disturbing implications.
[NO SPEECH]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
One of the ways to think about the American musical, generally, is to recognize that it's a distinctive, maybe even a unique American form and with the Western movie, the two most distinctive contributions that Hollywood has made to the art of movie making.
Listen to this entry on musicals, the very beginning of the entry on musicals from The Oxford Companion to Film.
"The history of the screen musical is essentially that of the American musical.
The outstanding examples of the genre have been made in Hollywood, and only the American industry has consistently produced musicals throughout the sound era."
The book was published in the early '70s, just as Hollywood movies were undergoing the transformation I've alluded to.
"Indeed with the Western, the musical could be claimed as Hollywood's outstanding contribution to the history of cinema."
So that's a good context for thinking both about the Westerns we'll be talking about in the next two weeks and about the musicals we're thinking about this week.
Perhaps before we go any further, it would be useful for me to give a very quick talk about the problem of definition because there is music in almost all films.
How do we distinguish a musical, a true musical, from a film that contains music?
And the simplest answer is also the most obvious one.
We would embrace a definition, I think, that would say that a film in which song and dance are so essential that to remove them would leave virtually nothing means you're watching a musical.
And that would distinguish it then from many, many films in which musical elements may be present, even musical performances may be present, but they are not at the very essence of what the film is.
And the film is not entirely dependent upon those elements.
So a simple, straightforward definition.
One way we can measure the significance, the centrality, of the musical and also of the Western, because the numbers are somewhat similar for the Western-- the centrality of the musical to the studio system is to look at the numbers.
And between 1927 and 1947, just over 900 musical films, were made.
That's approximately 48 films a year.
That's roughly 10% of the total production of movies during that period were musicals.
And one of the things it shows, of course, is how steady the audience for musicals were, how stable the system became relatively quickly, allowing for the elaboration and repetition of the fuller exploration of this particular genre of movie.
And you can recognize the decline, too.
There are people who would argue that the musical, in its exuberance, and the American musical especially, in its exuberance and optimism, represented essentially a pre-war value.
And there are people who say, well, after the war, the musical began to decline.
There are other explanations for why the musical began its decline, and the primary one is the one I've already implied.
The studio system was breaking down, and the captive audience of millions and millions of people who went to the movies every week was disappearing.
But in the period between 1949 and 1958, a much briefer period, only 23 films a year were made.
They were still a significant proportion of the Hollywood output, but you can understand-- you can see the tremendous decline.
And then in the period between '59 and 1980, only seven films that could be named musicals a year.
The genre is virtually-- not exactly extinct, but it's now a specialty item.
And musicals continued to be made, but they are hardly part of the regular fare of moviegoers.
So we can think of the musical as one of the distinctive expressions of the Hollywood studio era.
And in their optimism and in their-- maybe in even in their relative reluctance to engage directly with political and social questions, they express a certain value or a certain perspective that particularly illuminates the remarkable entertainment industry that Hollywood became.
I want to just say a very few words about the dominant themes in these films.
They're what you might expect from this combination of music and comedy.
One recurring topic is the theme of show business.
Many musical films have as their plot-- there's a kind of subgenre of musical films devoted to the question of putting on a show.
And the drama of this subcategory of musicals is you begin at the beginning with the cast being assembled, and you watch the difficulties, the internal and public difficulties, that the show has getting on the boards.
And one very common-- this is derived, in some sense, from forms of musical performance that existed on Broadway before they migrated to Hollywood, before they were adopted and adapted by Hollywood.
And what's embedded in this show business theme, for example, are larger themes.
And one of them is the theme of community.
In other words, you'll see a very concrete and dramatic articulation or dramatization of these principles in Singin' in the Rain this evening.
All of the themes I'm talking about now you will find, I think, expressed, dramatized with great authority and wit in Singin' in the Rain.
So the idea is that this putting on of a show means that you have a community of people who have to get along together, and what is discovered in the tensions and problems that begin to develop?
There are things that have to do, if we jump down in my categories to the next one, not to high culture popular but to the next one, to class or position-- you can see that, in some sense, the theme of community leads also then, at least implicitly, into an interest in the relative questions of hierarchy, as against questions of talent.
In other words, I'm the important singer, but my understudy has the better voice.
What's the tension there?
What's going to happen there?
And you'll see how this theme, with wonderful comic energy, plays itself out in Singin' in the Rain.
But what I want you to recognize as embedded in that theme is what we-- not only in what I'm calling the theme of community, or the theme of class, social class or social hierarchy or social position, and the tension between individual talent and social position, which repeatedly expresses itself, at least implicitly, in these music-- what does this say?
That the musicals do have, at some level, a kind of implicit political subject matter, that they're often about democracy or about how to get along.
And in fact, during the most recent crisis of partisan disagreement in Congress, I was thinking, gee, if I could send them a couple of musicals, and if they pay attention to them, maybe they could get along, too.
Probably too hopeful an idea.
But there is a political dimension to the drama, even to the confined drama of putting on a show, that some very good musicals bring to the surface.
Another central theme is the one I skipped over-- the conflict, often a tension or a conflict between high and popular culture, with Hollywood and the musical film almost always standing on the side of the popular as against the moribund and pretentiously sclerotic, old-fashioned older forms.
And they often create a kind of cartoon of the older forms in order to celebrate their own energy.
But this tension is a recurring one in musicals, and you'll see versions of these themes in both the musicals that are on our roster for this evening.
Finally, implicit in what I've been saying is the theme of convention and restraint versus spontaneity, energy, what we think of as "the natural."
And I'll say more about this tonight when we talk about the dance sequences in Singin' in the Rain.
But at the moment, just let me emphasize the extent to which this principle of a kind of conflict between what is handed down, what is conventional, what is expected and what is dictated by your spontaneous needs, desires, a sense of joy is a constant source of tension and energy in many, many musicals and is one of the central organizing principles of Singin' in the Rain.
And we'll come back to that tonight when I talk more fully about that film.
Well, what I'd like to do with the time I have left is try to illustrate in a certain way two different things.
And they're partly in conflict, and therefore, in some sense, I'm simplifying, but it's an intellectually justifiable simplification.
What I want to do simultaneously is give you a kind of brief sense of the history of the musical but also a sense of the different flavors within the broad category of the musical that might be possible.
This is obviously an impossible task in 30 minutes or so, but we're going to accomplish it anyway in a very reduced and crystallized way, I think.
As you might imagine, the central impetus for the musical occurred technologically with the advent of sound.
As soon as sound became possible, people were thinking, ah, we can sing in the movies.
Music is going to be important in the movies.
And of course, that was in fact the case.
The very first sound film, The Jazz Singer, was about singing.
It had ten interpolated songs in it and only two lines of dialogue.
In it, Al Jolson, the famous blackface performer, played a performer in blackface, a Jew actually, a white man, who played a black, most famously, on stage.
In this particular film, he plays a religious family's-- the son of a religious family, of a Jewish family, who wants to sing jazz, not to be a cantor in the synagogue.
And his father's very angry at him because he thinks he's betraying-- and can you see embedded there is the theme of, in a certain sense, high religious culture versus popular culture?
In the very first film that interpolated music, that theme is already present.
And in 1928, Jolson again appeared in a film called The Singing Fool that made a great success.
And what then followed in 1929 were a whole series of musical films, of films that incorporated music, including, in 1929, a film called The Broadway Melody of 1929.
And it contained a backstage story, like the one I've already described, about two sisters from vaudeville who tried to reach Broadway and star on Broadway.
And therefore, the putting on of a show is one of the implicit subjects.
But this form of early musical was derived from Broadway in a very direct way.
And it often did not have a coherent plot.
It often involved a series of singing and dancing numbers interspersed with comic dialogue of some kind-- literally a revue, not a play.
And then some versions of the revue began to have more systematic plots, animated, I think, generated by this theme of putting on a play.
And The Broadway Melody of 1929 was an astonishing success.
It was partly because it was the most fully success-- most fully musicalized film up to that time.
And it was so popular, it was-- incidentally, the songs written for it were written by Herb Nacio Brown and Arthur Freed, the two people who are at the center of Singin' in the Rain.
Nacio Brown's songs are in Singin' in the Rain, and Arthur Freed, of course, is the great executive producer responsible for more of the great musicals in the Hollywood era than any other figure.
So they begin even that early, at the very earliest stages of the medium's history.
And so then Broadway Melody was so popular and so successful that in 1930, 70 new musicals were produced, mostly in imitation.
And the film won an Oscar for Best Film.
So it immediately established musical films, films with music, song, and singing and dancing in them as a central category.
Most early musicals were revues of this sort, very early ones.
They combined-- they used Broadway, Hollywood, vaudeville, and radio stars, incorporated them in lavish musical numbers interspersed with sketches.
And again, they were not necessarily coherent plots, and most of them didn't even attempt to present a coherent plot.
This kind of film, more coherently plotted, remained a subcategory of musicals all the way through the history of musicals.
And that's why I want to qualify the idea that I'm giving you a history.
I am giving you a kind of history, but it's a complicated history because the forms I'm talking about don't completely disappear and get supplanted by the more advanced forms.
They live together simultaneously.
And the question of whether one is a more advanced form than another might be debatable, although I think there are distinctions, as you'll see, that do suggest that there's a kind of progression, an increasing understanding of what it would mean to make a musical film that was truly coherent, that wasn't simply presenting us with the joy of song and dance in a way unconnected to story or character.
But it remained a subcategory.
This revue remained a subcategory of musical films all the way through the history of the form.
And in 1946, one of the most remarkable versions of this was made.
It was called Ziegfeld Follies.
And again, it was based on-- it was a film about-- a nostalgic film about a Broadway phenomenon, but it was a fully integrated film.
--of a form that was popular in the United States but even much more popular in Europe-- the stage operetta.
It wasn't a full opera.
It was often a kind of smaller comic drama that involves singing and dancing.
And they were called operettas because they were understood to be smaller versions of opera, more amusing, less demanding versions of opera, I suppose, less pretentious and tragic versions of opera.
And there were a number of European directors who, when they immigrated to the United States, brought their distinctive feel to the making of early musicals.
And some of the most powerful, artistically powerful early musicals fell into the category not of the revue but of the operetta.
And I want to show you one version of such a film now, one of my favorite passages from the great director Rouben Mamoulian in 1932, a scene from Love Me Tonight, made in 1932.
It's a long sequence.
I may have to skip some of it, but let me set it up for you.
In this film, Maurice Chevalier, whose English is so terrible that you'll have trouble understanding him-- his French accent is so deep at this point.
He became much better at English because, as some of you, I hope, realize, he became a great star in the English-speaking world as well as in France.
But in this film-- this is one of his earliest films, if not his first English film, and he plays, even though he's very well-dressed here-- he plays a working man, a middle class-- a tailor.
He plays a tailor.
And he owns a tailor shop in Paris.
And in this opening scene, one of the men that he-- one of the aristocrats whose clothing he makes comes in to be fitted out for his wedding outfit, and Maurice Chevalier does so.
The plot of the film depends upon the fact that Maurice Chevalier, this middle class tailor who's trying to make a living in Paris in his shop, is brought into the ambit of an aristocratic family, of a royal family.
And because there is an aristocrat for whom Maurice Chevalier has made a great many suits who owes him money, and Maurice chases him to his castle in the country, trying to find, trying to recover his money, and romantic entanglements ensue.
And the larger part of the film actually involves a romance across social class between this Parisian tailor and a princess played by Jeanette MacDonald, one of the great musical stars of the studio era.
So here is a scene from very early in Love Me Tonight.
[VIDEO PLAYBACK] -All right.
[WHISTLING] [MUSIC - FELIX MENDELSSOHN, "WEDDING MARCH"] -Maurice, it's beautiful
Can you make it any louder?
Can you hear it?
-You make a work of art.
-Work of art for your sweetheart.
-Oh, it's like poetry in a book.
Oh, a beautiful large book.
-[INAUDIBLE].
The romance of the season-- -So clear like wind.
Oh, me.
[LAUGHING] -Oh.
Oh [INAUDIBLE].
-Oh, you're a magician.
-Isn't it romantic?
-(SINGING) My face is glowing.
I'm energetic.
The art of sewing I find poetic.
My needle punctuates the rhythm of romance.
I don't give a stitch if I don't get rich.
A custom tailor who has no custom is like a sailor.
No one will trust him.
But there is magic in the music of my shears.
I shed no tears.
Lend me your ears.
Isn't it romantic?
Soon I will have found some girl that I adore.
Isn't it romantic?
While I sit around, my love can scrub the floor.
She'll kiss me every hour, or she'll get the sack.
And when I take a shower, she can scrub my back.
Isn't it romantic?
On a moonlight night, she cook me onion soup.
Kiddies are romantic--
Would you freeze it one second?
Very quick observation.
I can't interrupt too often because I have a lot of clips to show you, and they're more important than my commentary.
But let me remind you that even at this very early stage in the history of the musical, look at how visually sophisticated Mamoulian, and pay attention as the film, as the sequence goes on.
But look at how he's working with the mirrors here and how it's doubling-- what it does to your sense of perspective and your sense of the environment of the shot.
This is a visually sophisticated, very thoughtful, artistically manipulated camera, and it's Mamoulian's work.
So very early in the history of the musical, we're already getting visually subtle effects.
-(SINGING)-- we'll have a troupe.
We'll help the population.
It's a duty that we owe to France.
Isn't it romance?
-Isn't it romantic?
Da, da, da, da.
A very catchy strain.
Isn't it romantic?
Da, da, da.
Oh, I forgot my cane.
Oh, thank you very much.
-I better fix your tie.
-Da, da, da, da, da, da, da, da-- -Goodbye.
Isn't it romantic?
Da, da, da, da, da.
-Taxi.
-Oh no, I need some air.
Isn't it romantic?
--[WHISTLING] At last I've got a fare.
-Railroad station!
-[WHISTLING] -Not too fast.
I hate to take a chance.
-Isn't it romance?
Isn't it romantic?
[HONKING] --[WHISTLING] Drive around the town.
-Isn't it romantic?
[HONKING] -Da, da, da, da-- I think I'll take that down.
-[WHISTLING] -A, B, A, G, F, E, D, C, C, A, B-flat.
Isn't it romantic?
Da, da, da, da.
I like the noise as well.
Isn't it romantic?
[SINGING CONTINUES] Isn't it romance?
Isn't it romantic?
[SINGING CONTINUES] -Isn't it romance?
-Isn't it romantic?
That's Jeannette MacDonald.
-Music in the night, a dream that can be heard.
[END PLAYBACK]
OK. Can we have the lights also?
Thank you.
All right.
I hope you enjoyed that lovely sequence.
That device is called a "pass-along song."
I've written it on the board.
And I won't say more about how witty and wonderful the film is, except to suggest that apart from its wonderfully amusing and witty qualities, I hope you implicitly recognized why I took the time to describe the plot to you because what does the music do?
What does this song do in this early part of the film?
It predicts the plot.
It predicts the story.
It carries the story into the countryside to the very castle where the plot will take Maurice from Paris later in the story.
And not only that.
What is dramatized by the way in which this song moved from tailor to aristocratic well-dressed man to composer, artist to cab driver to soldier to gypsy to Princess Jeanette?
And also that it covers physical territory as well as-- it crosses social barriers.
It crosses particular social classes.
It crosses differences of gender.
It crosses ethnic difficult-- what does it say about music?
Music creates community.
Music is democratic.
Music can overcome barriers, which is going to be ultimately the message of the film itself.
So it's an immensely subtle, crystallized moment in the film, and it has all kinds of powerful thematic implications for later in the film, apart from the fact that it's a wonderfully rich introduction to the primary character and to the joys of music.
Reality is, however, that until the advent in something like 1933 of work by Busby Berkeley, the great choreographer and director working, at this stage, primarily for the Warner Brothers Studio did we begin to see forms of the movie, forms of film, forms of cinema that were uniquely adapted to the nature of music itself.
That is to say, Busby Berkeley became a legend in Hollywood for the immense extravagance and elaborateness of his dance numbers and of his production numbers.
They were an elaboration of something that was popular in the Broadway theater before the advent of movies.
But what Berkeley did was take this tendency of the Broadway theater and cinematize it in a certain way.
And we can get some sense of what Berkeley was like, what Berkeley's work is like, by talking about the way in which he created a new attitude toward film spaces.
He loved to create receding arches and doorways and even waterfalls and to destroy what we might call "theatrical space" by exploring the camera's powers coupled with the power of editing, the power of editing now linked to rhythms of music.
So he created, essentially, a choreography for the camera.
No seated audience could have had the views and experiences that the Busby Berkeley production numbers began to provide.
And there's a moment in Singin' in the Rain tonight that explicitly recalls this tradition of the musical.
There's a moment in which a character in the film has a megaphone, and the front of the megaphone dissolves into a circle of costumed dancers in a sequence called "Beautiful Girls."
That's the name of the number.
And that whole number is an homage to this Busby Berkeley tradition.
Well, there were certain disadvantages to the Berkeley strategy.
One of them was that especially in the extremity with which he began to pursue his expressive aims, he began to sort of dematerialize his women.
He began to objectify them.
And many of his films, many of his production numbers, literally objectify parts of a woman's body, especially breasts and bottom and legs.
And there's almost, in a certain sense, a kind of dehumanizing impulse in which the human figures, especially when he used-- he often would use cameras that would rise very high.
He invented a shot called the "Berkeley top shot."
Often cut a hole in the ceiling of the places in which they were filming so he could move up higher.
And he would film from so far above that the individual lineaments of bodies would disappear, and only the patterns that they made appeared.
What he anticipated was MTV and forms of expressive filmmaking that severs a musical experience, or an audiovisual experience animated by music, driven by music, from subject, from plot, from story.
In other words, there's something in one sense very visionary about what Busby Berkeley was doing.
But of course, he was doing this often in the middle-- always in the middle of films that had stories and characters.
And one of the issues in his films were the production numbers were far more lavish, interesting, and witty and aesthetically energized than the stories that carried them.
And in a certain sense, the stories were really the platform to generate these kinds of performances.
So the films were not as aesthetically coherent as one would like, but they were full of these astonishingly expressive musical numbers, audiovisual experiences.
And let me just show you one that will give you a sense of his extravagance, of his extremism, in a certain sense, although that's part of the pleasure we take in how far he can go.
There's something deeply camp, deeply campy about Busby Berkeley's choreography.
And he reveled, he reveled in these kinds of gestures.
But here is one of the most famous of the-- although not by far the most extreme or ridiculous, of these kinds of gestures from a film Berkeley made in 1933 called 42nd Street.
[VIDEO PLAYBACK] -(SINGING) Say I know a bundle of humanity.
She's about so high.
And I'm just driven to insanity when she passes by.
She's a snooty little cutie.
She's been so hard to kiss
See how we started a theatrical space.
-And then I'll tell her--
Dick Powell, one of the great performers of the studio era, both musical and stage.
- --healthy, and you've got charms.
It'd really be a sin not to have you in my arms.
I'm young, healthy, and so are you.
When the moon is in the sky, tell me what am I to do?
Say-- If I could hate ya, I'd keep away.
But that ain't my nature.
I'm full of vitamin A, say!
I'm young and healthy, so let's be bold.
In a year or two or three, maybe we will be too old.
I'm young and healthy, and you've got charms.
It'd really be a sin not to have you in my arms.
I'm young and healthy, and so are you.
When the moon is in the sky, tell me what am I to do?
If I could hate ya, I'd keep away.
But that ain't my nature.
I'm full of vitamin A, say!
I'm young and healthy, so let's be bold.
In a year or two or three, maybe we will be too old.
-I'm young and healthy, and you've got charms.
It would really be a sin not to have you in my arms, dadum.
-I'm young and healthy, and so are you.
When the moon is in the sky, tell me what am I to do?
-If I could hate you, honey, I'd keep away.
But that ain't my nature.
I'm full of vitamin A. Whoa-- I'm young and healthy, so let's be bold.
In a year or two or three, maybe we will be too old
The Berkeley top shot.
[MUSIC PLAYING] See how he's moved to a kind of abstraction here?
-If I could hate ya, I'd keep away.
But that ain't my nature.
I'm full of vitamin A, say!
I'm young and healthy, so let's be bold.
In a year or two or three, maybe we will be too old.
[END PLAYBACK]
You think there might be something almost grotesque about those final images?
On the other hand, can you also see how in a system that is censored, in which women are not allowed to be seen on a bed without one foot at least on the floor, in which there is a kind of sexual energy and a kind of testing of boundaries running through this kind of display as well.
We're running out of time, and I have another even more magnificent sequence to show you to finish out this attempt to give you at least a quick flavor of the varieties of experience available in the musical before we look at a fully integrated musical this evening with Singin' in the Rain.
Let me say very quickly then, to jump to the bottom, the final aspect of this is what Arthur Freed and his cohorts articulated as the idea of a musical that would integrate sufficiently music and dance with character and story in a way that would be coherent.
And the musicals made under Arthur Freed's direction, often called the "Freed Unit" at MGM, tried to satisfy this criterion.
They were, in many ways, more coherent in terms of the relation between the singing and the dancing and the story than these earlier forms were, although they were often less imaginatively expressive in some cases, although this is not true, I think, of Singin' in the Rain.
But there is another stage to mention in this development or evolution, and it's one of the most important in the history of the Hollywood musical.
And it has to do with the partnership between Fred Astaire and Ginger Rogers, mostly at the RKO studios.
They made nine films together between 1933 in 1939 at RKO.
And they introduced another element into the musical film that is then, I think, incorporated in a certain degree in the so-called integrated musical of MGM.
So my last example this afternoon comes from the great Astaire-Rogers musical, Top Hat.
And let me set the stage for you.
Get ready to show it because we're running out of time.
The situation is this-- and it dramatizes even more fully the idea that there are potentially sexual tensions, sexual themes embedded in the experience of watching a song and dance musical.
What the Astaire-Rogers musicals brought to the table that had not been there before was, in some sense, a kind of psychological element.
Even though in the Astaire-Rogers films, the singing and dancing was still in some sense much more central to the film than the rest of the story, and the stories were often thin, what they did introduce was an individuation.
In the Busby Berkeley numbers, you can see the women are-- all the characters in fact are-- hard to differentiate from the backdrop or from the curtains.
They are objectified in a certain way.
Especially when he gets going, they become parts of an abstract pattern.
In the Astaire-Rogers films, Astaire and Roger's characters are deeply, sharply individuated, and certain kinds of psychological themes begin to enter in.
In this clip you're going to see now, a fairly characteristic one and probably the single most famous dance number in the history of the studio era, a dance number based on the song "Cheek to Cheek," danced by Astaire and Rogers-- in this sequence, we are in a setting that's worth mentioning to you.
We're on board a cruise ship.
Wealthy Americans are there.
It's wealthy people, aristocrats.
The Astaire and Rogers characters have met earlier, and they haven't really hit it off.
And the Rogers character, the woman, thinks that Fred Astaire is an immoral man, that he commits infidelities against his wife.
And she's under the mistaken impression that the woman he's with is his wife.
So what happens in the sequence, essentially, is that in front of what she believes to be his wife, he asks her to dance-- he asks Rogers to dance with her, and then, to her surprise, Astaire's apparent wife says, oh, go ahead.
I'd love to see you two together.
And she says, well, I'm-- and so when the dance begins, she's sort of resistant.
She thinking, ah, this is disgus-- and she goes along with it, but she feels there's something wrong about it.
And then as the dance goes on, you can see her resistance begin to break down.
And you can see the dance becoming more and more mutual.
And you can see her enthusiasm for the dance increase.
And think about what the dance, first of all, how profound and brilliant this dance is as an embodiment of mutuality and collaboration because one of the reasons Rogers was so remarkable was that she was capable of dancing at the same complex, astonishing level as Fred Astaire himself.
And it's an image of mutuality and of collaborative artistry that's very remarkable.
But it's also a metaphor for other kinds of mutual interactions that are physical.
And I hope you will pay attention to that subtext as you're watching this remarkable sequence.
The fact that Rogers begins in an act-- hostilely, resistant, and yields slowly is a key to the energy we see here.
OK. [VIDEO PLAYBACK] -Nice to see you again, Miss Tremont
Ginger Rogers.
-You've robbed me of the pleasure of introducing you two.
You've already met
Now, remember, Ginger is mistaken about this.
Fred's not really married to her.
And of course that's a typical Hollywood trick because we're able to enjoy the [FRENCH] of infidelity and of the idea of straying from your wife, but it isn't really true.
-I don't know what I'd do without her.
-Oh, that's sweet of you, darling.
But you two run along and dance, and don't give me another thought.
-That's what I'm afraid of.
I think Madge is a very brave person.
-Yes, I have tremendous admiration for her
You see how the misunderstanding adds tremendous sexual energy, wit to the scene.
-Well, if Madge doesn't care, I certainly don't.
-Neither do I.
All I know is that it's-- (SINGING) heaven.
I'm in heaven.
And my heart beats so that I can hardly speak.
And I seem to find the happiness I seek when we're out together dancing cheek to cheek.
Heaven, I'm in heaven
Kristen, can you put the sound off?
And pay attention-- [INAUDIBLE] I'm going to do the terrible damage to this.
Keep watching as I'm talking, and imagine you hearing the song, all right?
But because we're running out of time, I don't want to lose our energy, and I don't want to keep you over.
So this dance continues.
As you'll see, it's quite a long sequence, and he's singing to her in his weedy-- really not a very good singer, is he?
It doesn't seem to matter.
But one of the things you should watch is the way they move out of a crowded space into a more private space and how the intimacy of their-- and how the dance partly stops for a moment.
Then it resumes.
And they move-- as you'll see in the sequence, they move into a space that's completely private, and her-- and if you watch her expression, you can also see that she's beginning to yield, that she's beginning to become enthusiastic.
OK.
Put the sound back on.
-(SINGING) --charm about you will carry me through to heaven.
I'm in heaven.
And my heart beats so that I can hardly speak.
And I seem to find the happiness I seek when we're out together dancing cheek to cheek
--the sound for a second.
My final comment-- we're coming to near the end of this sequence, although it will extend.
I want you to hear the music, too.
But watch especially the way the intensity of this picks up and then subsides.
Ask yourself what rhythms are being implicitly dramatized here and then why, at the very end, it's time for a cigarette.
OK. [MUSIC PLAYING] [END PLAYBACK]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I want to continue our discourse about musicals, especially as a way of trying to lead into the two remarkable, in many ways landmark films we're going to juxtapose this evening.
Remember that our plan here is to juxtapose one film-- a classic instance, even a culminating instance of the studio era and of the tradition of the musical in the studio era-- against another kind of musical that appears after the advent of television, and after the studio system has begun to break apart, and especially after the fundamental transformation that the audience underwent, migrating from an habitual, a regular, a weekly relationship to Hollywood films, to essentially a blockbuster relationship, in which the vast majority of Americans would attend the movies only once or twice a year.
And if you think about the implications of that, that fact alone will explain how profoundly the movie industry was transformed.
Incidentally, I should mention to you that this is still not an idea that has been fully understood by what I'll call traditional film historians, who are so locked into an idea that film must resemble the kind of thing that I watched when I grew up as a boy, or as a young girl.
And now that a tradition of scholarship has talked about, that is to say, those films that never were shown on television, or were never made for television, it causes a problem.
Because what these scholars tend to do is they ignore the social context within which the function of the movies began to be transformed.
Partly there was a kind of hostility on the part of many older film scholars toward the notion that television might be a worthy place to look for a valuable cinematic narrative.
And it took a long time before the community of film scholars-- even a part of it-- was ready to acknowledge that television was also an environment in which cinema-- what we might call cinema audiovisual narrative-- could be look at, taken seriously, and so forth.
And of course, today it's taken for granted by many commentators, many of them no longer even with a cultural memory of the studio era, that what goes on on television-- that shows like Breaking Bad or The Sopranos are the supreme forms of audiovisual entertainment in our society.
This too is radically oversimplified.
And I will return to these questions a bit later when I talk about the Westerns, and about film in the '70s, in two weeks.
I want to continue our discourse about the musical then, first by giving you a very brief prospective drawn from the film scholar Leo Braudy, whose essay "The Aesthetics of Connection" is required reading I think for this week-- in any case, for this week or next week.
And he talks in that essay, among other things, very richly about the musical, and about ways of thinking about the musical.
And I'm borrowing from an argument he makes in that remarkable essay.
He makes a distinction in that essay, or a series of comparisons between Astaire and Kelly, the two great male leads we associate with the American film musical.
And he sees them very helpfully as contrasting figures illustrating what we might call contrasting elements, or contrasting tendencies within the musical form.
If you think about Astaire-- and you saw him in one of his classic performances at the end of our last lecture, in that sequence "Cheek to Cheek," if you think about Astaire in this way, from that sequence you can get something of what Braudy had in mind.
He reminds us that Astaire almost always dances-- and this is a partial exception in the film you saw tonight-- only a partial one.
But almost always we see Astaire dancing in places that seem intended for performance.
That is to say we won't see Astaire dancing in the middle of a corn field, or dancing up the side of the Empire State Building, as Frank Sinatra and Gene Kelly do in a musical-- it's called On the Town.
And it's one of the wonderful MGM musicals.
Kelly and Sinatra are dancing in spaces that are inappropriate for dance.
But almost all of Astaire's dances take place either in literal performance spaces or in public environments that seem as if they're quasi-stage or dance environments.
In other words, he dances in spaces that we expect for dance.
And he doesn't violate those spaces in some way.
He often in his films plays either a professional dancer, or a professional performer of some kind.
So when he segues into his performances, there's a kind of natural connection.
He's almost always very fancily dressed, as you saw in tonight's sequence, sometimes very often dancing in a tux and tails.
But in any case, in formal dress of some kind.
So he dances in space is appropriate for dance, dressed up in a kind of formal way, often playing a professional dancer or a performer.
But there's an immense grace and energy in his dancing.
He's one of the great dancers-- many would say the greatest dancer who ever performed in American movies.
And one way to talk about the attractions of what he's doing is we can say that what he embodies is the principle of grace under constraint, like a great baseball pitcher.
Look.
A lot of people can throw the ball 97 miles an hour.
But only a very small number of pitchers who make a lot of money can throw a ball 97 miles an hour and control where it goes.
And we can say that that kind of athletic power-- a lot of athletics involves this, where you're trying to control tremendous physical energy.
And the physical energy has to be constrained in certain ways.
And part of what we admire about the expression of that physical energy is how constrained or graceful it can become.
There are many people who have made analogies between Astaire's dancing and what great basketball players do, showing tremendous physical energy, but also a kind of gracefulness within the constraints that are imposed upon them by the lines on the court, by the demands of the basket, by the claims of gravity when they jump, and so forth.
And I make allusion to these non-dance analogies, because so many people seem to respond more fully to those than to dance itself, even though those of you who have danced, or are dancers, or know serious dancers, will understand that dancers themselves are much greater athletes than most athletes, and that there is a profound form of athleticism, of physical energy, grace, and physical control involved in dance.
And we see that in Astaire's performances.
It's as if he stands for a form of grace under constraint, of strength and power constrained but not undermined by the rules and boundaries within which it must operate.
It's a form of classical art.
And what we can say is that Kelly stands for classicism.
I'm sorry.
Astaire stands for classicism, and Gene Kelly, who is a kind of opposite in this way in his dances, stands for the romantic.
The classic and the romantic tendencies or energies.
I'm oversimplifying here, as Braudy does.
But again, it's a useful and illuminating simplification.
Because we see Kelly for example, mostly he doesn't play professional performers, or professional dancers.
He almost always wears ordinary casual clothing-- open shirts, jeans.
And this casualness of appearance is matched by the fact that the dances he engages in often take place in spaces that are not appropriate for dance, or almost hostile to dance.
And one of the deepest principles of Kelly's performances-- of Kelly's dances-- in a certain sense is this contest between his exuberance and energy, and the constraints that surround him.
But unlike Astaire, he does not reveal the power and grace of his dance by acknowledging the constraints and living within them.
The energy of his dance expresses itself in the moments when he over-goes boundaries, when he violates borders of various kinds.
And there are numerous examples of this in Singin' in the Rain.
I'll mention some of them in a little while.
So the contrast between Kelly and Astaire in this way-- and look, in both cases I've simplified, because they also resemble each other in certain ways.
But these tendencies distill quality.
This argument distills qualities in them that clarify certain energies or tensions that can be said to be characteristic, not only of two dance styles and the kinds of movies these two stars are in, but also in the musical genre as a whole.
I'd like to say a couple of things about each of the two films we're going to see tonight, by way of framing some of the most important issues for you to think about.
And these are very rich, complex texts.
And 20 minutes of conversation, no matter how pointed, can never do full justice to them.
I don't intend to do full justice to them.
But I want to suggest some ways for you to think about aspects of these films as you're viewing them.
And my hope and expectation is that these perspectives will energize and enliven the film for you, and cue you in to aspects of the film that you might otherwise not see as deeply, or might not occur to you until you reflected on the film after the case.
One way to think about Singin' in the Rain, then-- our example of a Hollywood musical, or a culminating instance of a Hollywood musical-- is to recognize that it belongs to a subcategory of musicals-- to several subcategories of musicals, one of which I've already mentioned-- the putting on the show, the backstage story.
But here the backstage story is the story of a movie, instead of a play.
But it's the same basic situation.
But it is also sometimes called a song catalog movie, because there's so many songs in it.
And that's kind of a kind of catalog, or an encyclopedic object.
An encyclopedia of film, of music, and of song.
And this is almost literally true of Singin' in the Rain.
Arthur Freed, the executive producer of so many major films-- over 40 MGM musicals between something like 1939-- what was the great musical in 1939?
The Wizard of Oz-- the first musical that is thought to be a musical in color.
And think how organically The Wizard of Oz uses color.
Arthur Freed was the executive producer of that film.
From that point until sometime in the 1960s, he was the executive producer or the supervisor of something like 40-plus MGM musicals, of which Singin' in the Rain is a classic instance.
And so Freed had worked as a lyricist.
Arthur Freed had worked as a lyricist at MGM with Nacio Herb Brown-- the songwriter-- many years earlier.
And most of the songs in Singin' in the Rain, 1952, had been introduced in MGM musicals from 1929 on.
So in a literal sense, the film is a kind of encyclopedia of older songs.
And especially two key songs in the film-- "Broadway Melody" and "You Were Meant for Me," were both featured in that early and signature musical, founding musical that I mentioned this afternoon, The Broadway Melody of 1929.
So part of the pleasure for older members of the audience in 1952 would have been to hear and see these numbers in the new, souped-up arrangements that were created for this film, and in such artful performances.
There's one song in the film that my colleague Martin Marks-- M-A-R-K-S-- he sometimes teaches this course in the spring.
He's a professor of music.
And he sometimes-- and I encourage all of you to consider taking it-- teaches an advanced course in the American musical.
It's a wonderfully interesting course.
And he knows more about the musical, I think, than almost any living human being.
And I'm stealing from some of his teaching notes here when I say this.
One song that Marty Marks loves to talk about is a minor song in the film.
And I'm mentioning it because you might miss it otherwise.
It's called "Would You?"
And it's a song that chose the process of dubbing.
And it's a song that actually has a kind of a plot, and historical function within the film.
It comes near the end of the film.
And when it was sung in this moment, it was already old-fashioned in its original day, when it was first written and sung.
And by 1952 it's much more outmoded, Professor Marks tells us.
Although it's a simple and graceful waltz, it uses an antiquated diction, for example.
And there are other signals-- even the fact that it's a waltz-- that show it to be kind of old-fashioned in a way.
But in Singin' in the Rain, this old-fashioned song is used to telescope the process of recording for films shot-by-shot.
Watch how it happens.
It's a very exciting moment.
And so what happens then is that the old-fashioned song becomes the cornerstone of the new technology of sound recording.
Watch how it happens in the film.
And what this calls our attention to is something I've talked about earlier, and will return to in a moment-- the idea that in the most remarkable works of art we are surely aware of the fact that we're in the presence of what the literary critics and the art critics sometimes call organic form, in which the form of the text contributes to its meaning, in which form and content are married, can't really be separated.
That the formal behavior of the text enacts or embodies or articulates or dramatizes what the film is about, what the text means.
I'll come back to this point in a minute with even more dramatic examples, I hope.
And an added point about this "Would You?"
song, just this one sort of minor sequence-- I'm mentioning this partly to show you how deeply you could actually dig into particular sequences and particular songs in the film, and find meaning, relevance, and complexity there.
When Debbie Reynolds, who sings the song, looks point blank at Gene Kelly as she sings it, she drives home the question of whether he'll be brave enough to love her, So it even implicates itself in the love story that's at the heart of the film.
Her singing voice, we might note ironically and comically, in this scene is actually dubbed for this song.
And you'll understand if you haven't seen the film why that's a comic and ironic commentary.
She had the wrong kind of voice.
It was too flat and nasal for this operetta-like number.
So they dubbed her voice.
So there's a double irony here, because she's playing a dubber when she's singing the song.
And you might compare the voice in "Good Morning," another number in the film, where Debbie Reynolds is really singing.
And you can see the difference between the two voices.
So the implicit point that I made in talking about this one number, the "Would You?"
song number, it has to do with what we might call a central element or feature of Singin' in the Rain, its profound self-consciousness, the profound way in which it shows itself to be aware of itself-- to be aware of itself as a movie, to be aware of itself as a bearer of traditions of singing, of dancing, and performance.
And so it isn't simply an encyclopedia of musical history.
It's almost an encyclopedia of almost all forms of performance.
I don't want to go too far.
But there's many, many forms of performance are incorporated in the film's grasp, in the film's embrace.
All right?
One way we can think of it as an encyclopedic recapitulation of the history of musicals, and of films in general.
As you'll notice when you look at the film, Singin' in the Rain begins by being about the problem of changing a silent film into a sound film, because the silent has been made just at that cusp moment when the transition to sound is occurring.
And the film in that sense is also a wonderful sort of historical reprise for our course, even though it wasn't made for that purpose, alas.
So in the film, then, we also get an allusion to and a representation of silent movies.
But not just some generic idea of silent movies-- a very particular kind of swashbuckler adventure film is dramatized there.
But then later, in a scene I mentioned earlier today, in the audition scene, the beautiful girls sequence in Singin' in the Rain where a megaphone suddenly turns into a circle of dancers, alluding to the Busby Berkeley tradition, we have a very specific reference to another kind of movie musical.
So, juxtaposed against each other are two different kinds of movies, And I've only mentioned a couple of examples.
As you watch the film, you'll see dozens and dozens of instances in which the film capaciously reaches back to older forms of performance, and tries to incorporate them in some sense into its own structure, into its own energies.
And there are many older performance forms in the film.
There are forms of slapstick, and Keaton-like comedy-- especially in Donald O'Connor's wonderful dance and song sequence "Make 'em Laugh."
There are allusions and references and dramatized scenes from vaudeville, from jazz, from what we might call forms of love melodrama, all the way through the film.
And I haven't really cataloged even remotely all the kinds of performance spaces and performance types that the film incorporates and utilizes in the course of its length.
Nor have I even begun to list the number of musical styles-- particular songs and particular dance styles-- that are also incorporated into the film.
But I've said enough to suggest that Singin' in the Rain has what might be called a structural or a formal exuberance, a kind of excitement, as if every time it turns it wants to make an allusion to another kind of text, as if it feels itself in some sense with a kind of excitement and exuberant energy constantly in touch with its ancestry, and within ancestry that it tries to define very generously and broadly.
And so what we're saying is that formally, structurally, the film is expressing a special kind of intellectual energy, excitement, exuberance.
And of course what's interesting and fundamental about that is that those energies are also replicated, centrally reenacted in the actual dance sequences that we see in the film.
It's a profoundly rich example of what I called a moment ago organic form.
The structure or the shape of the film contributes to our sense not only that it has a kind of encyclopedic, self-reflexive interest in its own history, and that it's recapitulating its own history for us as it proceeds.
We also have a deep sense of the extent to which the film understands that many of the older forms it dramatizes are now simply being archived, as if the film understands itself not only as a dramatic entertainment, but also as a kind of history book, or a kind of encyclopedia-- literally a kind of archive of all the forms of performance and song and dance that lie behind the making of such a film.
And of course that this formal exuberance, as I said, is repeated again and again in an actualized and concrete way in the story, in the plot, every time the characters start singing or dancing.
And as you'll see, there's a particular kind of exuberance and energy in many of the dances.
Key themes.
Well I've already implicitly been talking about some of the themes.
But let me mention just two that seem to me especially significant.
One I've mentioned much earlier.
And I simply want to remind you again that this tension between what might be called high art, and the pretentious expectations of a cultural establishment, are set up for parody and mockery in this film repeatedly.
The film is on the side of the popular and the energetic, and the unpretentious.
And one can feel that tension again and again in many, many of the sequences in the movie in the film.
Let me give you just two obvious examples.
In the very opening sequence of the film, one of the classic moments in the history of American movies replayed again and again in compilation anthologies of great moments in Hollywood, great moments of Hollywood film, is the introduction to the film.
And the introduction to Singin' in the Rain recreates an historical actuality that I need to remind today's audiences of.
Shows you how old this course is.
Because when I first began to teach this course, these practices were still going on, and the students still understood, and in fact had their own experience of exhibition movies, and of going to theaters that were very lavish and, in many ways, even pretentious.
And what they were especially aware of is something I have to remind you of now, which is that very often Hollywood had a strategy of rolling out its great blockbuster, what it hoped would be great blockbuster films, with very elaborate inaugural events.
The debut of a film was often a major event.
And it would be covered by newspapers.
And many Hollywood stars would come and attend.
And they would turn the event into a publicity event partly in order to get publicity for the release of the film.
And Singin' in the Rain opens with such a moment, in which a film is opening.
And it dramatizes the comic hypocrisies that attend such publicity gestures.
And as the film opens then, we see a sequence in which the stars of the film we're about to see, Gene Kelly, is interviewed.
The star of the film is interviewed by a reporter.
And he then tells sort of the story of his career.
But the story that he tells is fake-- is a lie.
It's very pretentious.
Oh, we went to the best conservatories.
We were trained by classical musicians and classical actors-- "ac-TORS."
And he acts very pretentious.
But that's what he says.
But what we see, what is shown to us is the actual history of how Don Lockwood and his partner, his sidekick, played by the brilliant performer Donald O'Connor, really got their start.
And so while he says, we were trained in the best conservatories, what we see are Donald O'Connor and Gene Kelly dancing in a burlesque show.
And so that there's this very obvious, even in some ways crude mockery or laughter at the hypocrisies of Hollywood.
And part of what's being celebrated there is the exuberance and energy, of unsanctioned forms of performance, like the dances that people do in burlesque shows, as against what you do when you go to the museum, or when you come to the university and have your professor declaim ancient Greek to you in syllables you have no understanding of.
So there's one example of how that theme plays out.
But you can find it in dozens and dozens of other places in the film.
And I hope you'll watch for such moments.
So I won't give you my second example.
I'm running out of time.
Let me turn to the next large theme I want you to be aware of.
I call it the theme of outer versus inner.
It's really a variation on or an extension of the theme of high versus popular art, in the sense that it's also about the difference between a kind of pretentious and constraining and old-fashioned view of things, and a more spontaneous and we might even say a more spontaneous and popular or democratic view of things.
And the term I give it is the difference between outer and inner, or the difference between appearance and reality.
And in fact the example I just gave you, in which we have Don Lockwood telling the reporter that he was educated in the best conservatories while we see that he's dancing in a burlesque show, is an example of that kind of thing as well.
But there are many other very dramatic examples.
And I want to mention one of them now.
When the Debbie Reynolds character-- the love interest of our protagonist-- first meets the Don Lockwood character-- the character played by Gene Kelly-- she pretends not to recognize him, even though he's a famous movie star.
And she says, oh, she spends her time watching Shakespeare, but she doesn't spend much time watching the movies.
And Don Lockwood is kind of offended by this.
It later turns out-- not even that much later-- that of course the Debbie Reynolds character herself is deeply immersed in the popular culture.
Right after she makes this comment to Don Lockwood, or a few minutes after that, we find her jumping out of the cake as a dancer at a party.
And then as more information emerges later, we come to recognize, of course, that she knows a great deal about Don Lockwood.
She's always been a tremendous fan of the movies.
But she's been covering it up.
And again and again in the film we see this kind of tension playing itself out, this tension between a kind of outer facade.
And the notion is that what is shown on the outside, the exterior of things is often fake, a mere cover for what is authentic and within, And you can understand this in a number of ways.
One of the ways it plays out in the movie is the question of the person who is in front of the audience and singing, but it's not really her voice.
It's the voice of Debbie Reynolds, who's behind the curtain.
And so that again, this theme of the artificial versus the authentic, or the outer versus the inner, plays itself out in many, many ways in the film.
That's part of what makes the film so coherent, so dramatically effective and energetic.
And so another way of thinking about this sequence, this idea of the outer person versus the inner, authentic person; the outer appearance, the illusion that film makes, but what really goes on behind the scenes.
And as you'll discover, Singin' in the Rain constantly takes us behind the scenes to show us how the illusion of movies is put together.
And that's a part of this same theme.
So one of the things I'm trying to call your attention to again is the way in which the central ideas of the film permeate both its form and its content in ways that make it very difficult to separate.
Finally, one last fundamental issue connected to what I've been saying before, the way the film meditates on or dramatizes an idea about the place of song and dance in human life.
You will have noticed, of course, that I myself feel-- and if you've seen any of the performances in this film, you'll know yourselves-- that a principle of energy, vitality, playfulness, exuberance is almost always present in the dances in Singin' in the Rain.
Song and dance is seen in Singin' in the Rain as a morally and psychologically revolutionary experience, I think.
And I use the word "revolution" in a dangerous way.
I don't mean a politically revolutionary way, but I mean in a morally, a psychologically revolutionary way.
And so what's going on in some sense is the sense that constraint and convention are constantly threatened by the exuberance, spontaneity, energy, excitement, vitality of the dance.
And this expresses itself in all kinds of ways in the film.
Let me just mention a couple to call your attention to.
You'll discover many, many other examples.
One is the wonderful sequence, the Moses supposes sequence.
This is a sequence in which Don Lockwood and his partner Donald O'Connor have to be trained, because they're making the transition to sound.
And the studio calls in an elocution expert, who's dressed up like a very pretentious old-line professor.
And he's going to teach them how to pronounce things correctly.
So he comes in, and he says, I want you to repeat after me-- "Moses supposes his toses are roses.
But Moses supposes erroneously."
He's also ugly.
His teeth are disgusting.
So you're already offended by the guy before Don and his comrade begin to respond.
And also, of course, what's ridiculous about this?
It's meaningless talk.
He's so interested in making them pronounce the "O" vowel properly that he creates meaningless sentences.
But their meaninglessness amuses Don Lockwood and his partner.
And essentially what they do is they take the Moses supposes lines, and they begin to sort of make fun of them.
And they begin to make fun of their interlocutor, their teacher.
They end up taking a garbage can and emptying it out just has paper in it-- putting it on his head.
They end up dancing in a ridiculous way.
So they end up dancing on top of desks, or up the side of walls.
Again, what are they doing?
Their dances are violating the normal spaces in which we expect people to be walking or standing upright.
The very way in which the dance violates normal boundaries creates this sense of exuberance and of freedom.
So what they're doing is they're rebelling against the pretentiousness, the constraints of this kind of high-culture expectation.
And the song and dance in the Moses supposes sequence create a kind of exuberant chaos in which they dance on the tables and make fun of the very elements that they're supposed to be learning.
The "Good Morning" sequence-- one of the great sequences in the history of movies, I think-- shows the same thing.
It takes place at breakfast in an ordinary domestic space, which is transfigured or transformed, including the props, in something we might imagine Chaplin doing such imaginative things, in which the ordinary props of the kitchen become partners in the dance, become elements in the dance-- what we feel is the imaginative power of love and of performance transforming an ordinary space.
And we can see this principle operating with an even greater force in the title sequence of the film-- probably the most famous sequence of dance in the history of Hollywood, the Singin' in the Rain sequence itself.
We see the rain itself.
What a funny place to be dancing, There's a contradiction.
But your exuberance, your energy, your sense of love is so great that even rain doesn't matter.
And watch what happens.
He has an umbrella, but he doesn't use the umbrella to protect him from the rain, violating our expectation.
There's one wonderful moment in the film where he hands the umbrella in the midst of pouring rain to someone else, to a pedestrian who goes by, There's a moment when he's dancing in the rain.
He puts the umbrella down.
The rain's coming.
But he doesn't have enough rain pouring on him.
He steps under a downspout to have even more rain come down on him.
And when we experience this, one of the reasons we experience this as joyful, and as assertive, and as individualizing, is the sense we have that these are exactly not things that people normally do.
He's over-going or violating certain conventions or literal boundaries.
And there's one moment in the dance, very dramatic.
And I know this incidentally, not because I've discovered it myself, but because 20 years ago or 15 years ago in this course a student who later went on to work in Hollywood-- and he still works there.
He's a sound genius.
He was an MIT engineer who loved this course, and was a musician as well.
He wrote a paper for me about that sequence.
And he showed me this.
And I'm sure that some smart critics also saw it, but I learned this from an MIT student in this course about 15 years ago.
He analyzed the dance sequence very closely.
And one of the things he showed was that the decisive moment in the dance occurs-- or a decisive moment in the dance occurs-- when the Kelly character suddenly-- he's dancing on the sidewalk-- jumps off the sidewalk across the curb into the street.
And psychologically it has a tremendously profound effect, because you don't walk in the street, much less dance in the street.
The street is for cars.
But that violation of the boundary expresses the sense of exuberance, of freedom that is the essential meaning of what these dances represent.
So energy, spontaneous feeling, performing and over-going limits is one of the things that's being expressed in the exuberance of the film's formal structure, in its encyclopedic reaching out to other forms which it was to incorporate into itself, and in its particular dance and song sequences.
And this energy, this celebration of transformative energy, of the over-going of boundaries, strikes me as particularly American, as a uniquely interesting expression of what people have sometimes called American optimism, and even of the democratic spirit, of the openness that we associate with our ideals-- not the reality, but our ideals-- of democratic society.
What these dances and songs tell us then about the place of dance and song in our lives, the idea that you would begin to dance in a space unused to dance, hostile to dance, transforms the space.
So what this film then says about these things is that our speech can be nudged into song, our way of walking can be edged into a dance, and that the ordinary objects in our domestic and public spaces can become props in an improvised ballet.
When we turn to Cabaret, we're looking at a very different kind of object.
And it's important to recognize what I mentioned earlier, and I will develop more fully in a couple of weeks, the fact that now we have gone almost beyond genre.
There are people who would argue-- and there are purists, scholars of the musical film who want to argue that, well, Cabaret really isn't a musical for a variety of reasons.
It seems to me a very strange and unhelpful argument.
But I think if you watch the film closely, you'll understand why people who are partisans of the great Hollywood musical tradition, or even the older operatic tradition represented by value directors like Rouben Mamoulian would have problems fitting Cabaret into the category of musical.
I think it's a misunderstanding of how we should think about musicals.
But for the moment, let me just encourage you to watch the film and think about how different it is from traditional musicals-- and especially its tone, its moral perspective.
But also it's in its structure.
Think about that, and you'll understand more fully why it's easy to see that this film would be very unlikely to have been made in the Hollywood era, that it required the kind of freedom that came to film after film was no longer constrained by the necessity to speak in a consensus voice.
And as I said, these are matters I will try to clarify and conclude in two weeks when I return to this question, when we talk about film in the '70s, and the great '70s Western-- almost contemporary with Cabaret-- Robert Altman's McCabe and Mrs. Miller.
I had hoped to say a number of things about Fosse's career, but I don't have time.
And I'm just going to encourage you-- look him up.
He's a very, very remarkable figure.
I call him "The Inheritor" because he was both a Broadway director and choreographer and a dancer both on Broadway and in films.
And he performed and directed both on Broadway and in films.
He was first a choreographer in films before he became a director.
And in this career as a performer and choreographer, he linked up with almost all the other great names of the American musical stage-- both the Broadway stage and the Hollywood tradition.
And in that sense, we can say that Bob Fosse is the inheritor of the great tradition of the American musical.
And I wish I could go into detail about this and show you how many affiliations and specific connections he has with the great figures of the classical age of American movies, and especially the classical age of the American musical.
But suffice it to say that it's a very rich set of connections.
And his career is a very remarkable career, a very honored career.
Let me mention one fact about his career, perhaps the most remarkable one, easy to remember.
Tells you something about his diversity as a director, his gifts, his ambition.
In 1973, Bob Fosse I think became the only man in history to win an Oscar, a Tony, and an Emmy-- an Oscar for Cabaret as best director, and a Tony for-- or maybe he either he won as best director or the play that he directed on Broadway-- Pippin won as best play on Broadway.
And he also won a television Emmy for a show starring Liza Minnelli, the star in Cabaret, called Liza with a 'Z.'
So And I think he's the only man in history for whom those three awards came in the same year for three different texts.
He directed one, two, three, four, five films, of which Cabaret was the second.
The last four are all very remarkable and interesting texts.
I won't talk about them here, except to encourage you to look him up and think about them.
The theme of performance and the idea of the characters who are themselves performers are always at the center of his films.
Not all of his films involve dance.
He made a film about the life of Lenny Bruce called Lenny starring Dustin Hoffman, for example, that doesn't have musical numbers in it.
But it has many performance numbers in it.
And it's a very rich, complex, sophisticated film.
So an interesting and profoundly serious director.
I wish I could say more about his career.
I want to say a word about the context of Cabaret, so you'll understand that aspect of the film more fully, too.
It's set in Weimar Germany, that curious time in the 1920s during the rise of Nazism, but long before Hitler has come to power, in which the Weimar Republic, a vulnerable and unstable democracy, but a democracy in which an enormous amount of artistic and literary experimentation was taking place.
It's the period of German expressionism.
It's the period of the great German silent films that we talked about earlier.
And so our film Cabaret is set in that era, and in that driven era, in that divided era in which the rise of Nazism was coupled with or joined to and connected to the fact that German society was under immense constraints, because it had lost the first World War.
Conditions under which they had been forced to surrender were very onerous.
There was very high levels of unemployment.
It was a schizophrenically divided society, a society of great artistic energies, but of great political divisions and difficulties, and nightmare possibilities that were lurking in many of its practices.
And there's a link-- or at least a partial link-- between the instability and the excitement, but also the dividedness of Weimar Germany, and the actual condition of the United States in the era when the movie came out, Because this is the period at the end of the '60s, after the late '60s and early '70s, probably the most divided time in American history after the Second World War before our current moment.
And in fact, the divisions then were more legitimate.
By more legitimate, I mean they had a more sensible cause.
There were terrible wars going on.
There were better reasons for the partisanship that occurred in the late '60s and early '70s than there are for the weird partisanship we're suffering in the United States today.
And in any case, there was a much greater sense in the late '60s and early '70s that the fabric of American society was under attack, that it was imperfect, not only because the Vietnam War had so divided the country against itself, but because of other matters as well, including the increasing militancy of the civil rights movement, which had gone from a sort of a Martin Luther King type commitment to nonviolent protest into much more disturbing and sometimes violent confrontations with authorities, and with some theorists and speakers actually calling for violence.
And this racial tension was then coupled by another kind of political tension coming from the people, from the young men and women-- who were mostly young men and women-- who were opposed to the Vietnam War.
So it was a very driven and disturbed time.
There were then parallels between the instability and the disturbance of the Weimar period and the period in which the film appeared.
And the original audiences, I think, would have been highly aware of this connection, would have seen that although the film was set in the past, it spoke to the contemporary moment in a very powerful way.
I want also remind you that there are certain aspects of the style of Cabaret that are worth keeping in mind.
And one way to think about this is to recognize that both mis en scene style and montage style are embodied in the film, and are carried to extremes in certain moments of the film.
That is to say, in many of the performance sequences in the film, the camera behaves in an unbelievably frenetic and frantic way-- changes your eye angle, jumps around a lot, disorients you sometimes-- almost disorients you in the style of a Hitchcock.
And yet in other sequences in the film, especially those outside the cabaret that take place in ordinary life, not in a performance space, the camera and the editing slow down and become more like a kind of mise en scene style.
And the combination of the two is especially rich and interesting.
You might want to watch for the way these rhythms alter your sense of what's happening in the film, the way the film slows down, and speeds up at certain point.
And I mean this literally in terms of the number of images that go by quickly, but also in terms of the way the physical bodies of the performers move across the screen.
And of course the way in which the camera itself behaves, whether it's jittery, oriented toward constant movement; or whether it's more stable and gives you a chance to look at the environment this scene is set in.
One way I could give you a sense of the richness of the film is to remind you that there's a kind of texture or multiplicity in Cabaret in almost every moment, that if you watch for it, and you're attentive to it, you will always be rewarded.
Let me just give you two very quick examples of what I mean by this.
Remember I suggested to you that this principle of multiplicity is one of the principles we turn to if we want to recognize a work of art.
The difference between a work of art and an entertainment in my terms is that a work of art is just more intelligent.
That is to say, it makes more complex use of its materials.
So that in a entertainment, a particular event or gesture may have only one function.
But in a work of art, you will have multiple functions.
It will perform several jobs simultaneously.
I'll give you two examples of this.
Think about what takes place on the periphery of what you're watching, and you'll begin to get some sense of one way in which the texture of this film brings the material alive.
There's one moment I think fairly early in the film.
It's a street scene.
We're not paying much attention to what's going on.
We see, I think, the protagonist walking in the street.
And around on the periphery-- certainly not in the center of the frame, nothing that is commented on by the characters-- a cripple slides by on one of those wheeled platforms that people without legs sometimes used in major cities before-- maybe they still happen.
But they were very common in the first half of the 20th century.
And we see this guy.
Obviously he's a wounded veteran, and he's an amputee.
And he pushes himself along on this cart.
And we just see him go by.
But what's interesting about that is that this cripple on the platform has a little flag on his platform.
Guess what kind of a flag it is?
It's a Nazi flag.
And this is early in the film, before the Nazis have become a powerful force.
The rise of Nazism is in the background of film, as you will see.
The Nazis moved from the periphery to the center of the frame before the film is over.
And that's the fundamental historical motion of the film.
But all of that's taking place in the background.
The foreground are the central characters, and the comic and innocent-- and ultimately not so comic, disturbing love affairs and difficulties that they encounter in this environment.
They're too naive or innocent to understand the historical forces that are surrounding them.
So that example of the Nazi flag and this crippled figure-- because the crippled figure is in fact an allusion to the onerous demands that were put upon Germany by France and the other allies at the end of the war.
And he's crippled.
So the assumption is he's probably a war veteran.
I don't remember if he's actually wearing a uniform, which would confirm this.
And it would be helpful if he were.
But I don't remember.
Watch for it, and you'll see.
But in any case, he refers to that.
Anyone attentive to the firm can recognize that the historical texture of the film is carried by such moments.
There's another such moment, also so minor that you might not notice it.
The foreground is the love affair, or not yet a love affair, but the acquaintance, the deepening friendship between the two protagonists-- Sally Bowles and her boyfriend.
And they go out for a walk in Berlin.
And Sally pretends that she's this immensely sophisticated woman.
Turns out that she's fooling herself and fooling us.
She's not at all, even though she pretends to a great kind of sophistication.
And she sings dirty songs in a cabaret, and she performs in a cabaret where they make jokes about everything, including all forms of sexual perversion.
Nonetheless, she's an innocent.
She has no way of recognizing the depth and complexity of the moral evil and moral terror that's building around her.
And she's not alone in this.
Most of the other characters in the foreground of the film perform this same naive ballet, although Sally Bowles is the most powerful instance of it.
It is certainly Liza Minnelli's greatest role.
Certainly the role that she'll be remembered for long after she's dead.
Long after we're all in the ground, Liza Minnelli's performance in this film will still be treasured and savored, I think.
So another instance of the multiplicity.
When we find the two characters are walking in the street, and Sally says, oh, I want to show you something I do.
And she pulls this guy along.
She leans up against the side of an elevated train stanchion.
And she waits until the train comes by.
And as the train comes by, she screams.
And she says, see?
Whenever you wait for the noise, nobody will hear you scream.
And she's just expressing her exuberance.
She's showing him how she commands this space, how she knows how to use this urban space.
But there's an irony she's unaware of.
When she leans up against the wall behind her, what we see are political posters.
And the political posters behind her are defaced.
They're both Nazi and communist posters.
And the communists and the Nazis are defacing one another's posters.
And what is implicit then again in the background against the foreground of these two relatively naive characters carrying on is the historical reality that's building behind them.
Another example of what I mean by the multiplicity, the texture of this film.
Next to finally, the musical numbers-- penultimately the musical numbers.
Another kind of integration is going on in this film, when we talk about the integrated musical as Arthur Freed and his comrades during the studio era articulated this idea.
We're not talking about the form of integration that we see in Cabaret.
And this in fact is one of the reasons that some people argue that Cabaret is not a musical.
Because with one exception, none of the musical performances in Cabaret take place out of a performance environment.
They all take place in the cabaret.
The film is about a cabaret performer, and we see many performances in the cabaret.
Now when we watch those performances, we see them from angles that no one in the audience could ever see, because Fosse is a genius with the camera, and creates visual effects drawing on the tradition of the Hollywood musical that could never be replicated in a theater.
Nonetheless, the performances themselves are theatrical performances.
They all are real performances.
And when we see the characters behaving like real characters, they don't suddenly burst into song and dance.
Because there's something theoretically so implausible about that, that Cabaret refuses that convention.
It refuses to embrace that basic convention of the American musical, which is that one way that people express their exuberance-- I love you.
I love you.
Now let me sing a song telling you how much I love you-- whereas in real life we don't do that.
Now I don't think that this convention is a convention that undermines the musical.
It's a convention, or a rule, or a habit, a protocol you need to embrace in order to experience these films appropriately.
And it's no more implausible in one sense than the implausibility of opera singers bursting into song at moments of great passion.
The convention requires it.
It's as if we're in a form of artistic experience in which art is crystallizing experience for us.
So I don't mean to undermine this convention at all, or cast doubt on it.
But Cabaret refuses that convention-- with one exception.
And I'm calling attention to that exception.
I'm not going to tell you what it is, but I want you to watch for it.
There's one moment in the film when there is a song sung outside by ordinary people.
It happens spontaneously.
It may be the most morally and historically frightening moment in the whole of the film.
And it's completely believable, totally plausible that it would happen.
But that's the only moment in the film when there's any singing that doesn't take place inside a space intended for singing and dancing.
So one of the interesting things about Cabaret is that there is a kind of true integration of song and dance, because none of that happens except in spaces where it's appropriate-- realistically appropriate.
And so you might watch for that.
And the complexity that this decision on the part of Bob Fosse and the people making the film, the impact of that decision on your understanding of the movie as a whole.
Finally, the central themes of the film.
I've already talked about them.
The theme of innocence-- the way the central characters in the film are too innocent to understand the historical forces around them, underestimate them.
This isn't just true of the central figures.
It's true of the secondary characters as well, some of whom even more poignantly embody this theme.
And then the theme of history.
What history is.
How the force of history can overwhelm individual desires, individual hopes, love itself.
And of course, also the limits of satire.
The cabaret is a space of satire, a space of mockery, a space of exuberant intellectual distance and disdain.
Much of the energy of the cabaret comes from an impulse of parity, mockery, satire.
And in the very beginning of the film, there's actually a moment where you can see the Joel Grey character, who plays this ominous, strange figure of the emcee, the interlocutor, in the cabaret.
He sort of is the master of ceremonies.
And there's one moment early in the film when he, in a rather disgusting scene in which they replicate a scene of mud wrestling.
And he's emceeing it.
And he reaches down, and he picks up a piece of mud.
And he rubs it on his upper lip.
And he makes himself look like Hitler.
And he's mocking Hitler in that.
Well if you compare that early moment in which Hitler and Nazism are objects of mockery, there's also an early scene in which we see a Nazi being pulled out of the audience of the cabaret and pushed outside, kicked out of the cabaret, because Nazis are thought to be unacceptable.
They're thought to be vulgarians.
This is early in the film.
By the time the film is over, by the time you come to the very end of the film, one of the very final images of the film shows us the audience of the cabaret filled with Nazi officers, filled with Nazi uniforms.
The Nazis have moved from the periphery to the center of the movie.
And that's part of the terror that the film dramatizes.
And it also therefore of course dramatizes the limits of satire, because the cabaret itself, like the characters, imagines itself to be exempted from or apart from history.
It can maintain its mockery, its distance.
Satire can protect me.
And of course the cabaret is just as naive as the characters themselves.
It's overwhelmed by the forces of history in a way that the characters themselves also are.
Both of these films in their own way have generated a tremendously interesting and serious volume of discourse, both in terms of their musical qualities and the qualities of their choreography, and in terms of their content.
They're really such complex films that watching them together makes an immense intellectual demand on everyone.
I know you're capable of that, but I hope that you'll take the time to let the implications of these films settle into your mind for days, and maybe even weeks after you've watched them.
And I hope, of course, that this first viewing will only be the first of many.
I wish you the joy of these wonderful movies.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We continue this wee, next week, with our conversation-- it's really our final conversation-- about the genre forms that are at the center of the Hollywood era with a consideration of the Western.
And that perhaps we should begin by acknowledging how central the Western is as a movie form in the history of American cinema.
Before the advent of television in the late '40s-- the first American television broadcast was 1946.
And we can say the television is actually established is a social experience in American life certainly by the late '40s, by let's say 1949 or 1950.
And it's through the '50s that television spreads through the society and begins to supplant the movies.
So that before television, let's say that before 1950, nearly 30% percent of every movie made in the United States was a Western all the way through the Hollywood era.
And in fact, in the silent era, as I will describe briefly again this evening when I do a kind of mini history of the Western movie, even in the silent era, it's a central form.
And even after the 1950s, even after the advent of television, nearly half of all movies that we might identify as set in the past, as historical movies, were Westerns.
And of course, as I suggested when I talked about the musical, the Western can be understood along with the musical as a uniquely American form, even though at a certain point in the mid and late '1960s, certain gifted Italian directors begin to take over the Western and darken it and throw a kind of European cynicism into the tradition of the Western.
By that time, the Western genre evolved and transformed in ways that made this cynicism not completely unique, not totally European, because a kind of subversive perspective had begun to enter the American version of the Western as well.
Nonetheless, the Western form underwent profound transformations.
And it's those transformations that I want to briefly to talk about here in an introductory way.
And I'll return to that sort of what we might call the trajectory or the career of the Western briefly this evening when I introduce some comments I want to make about John Ford and about his classic Western, the film we're watching tonight, "The Searchers."
I want to talk a little bit more, not only about the Western, but about why the Western might be seen as being the central American form, why it is so central not only to the History of American cinema, but in a certain sense, central to the cultural history of American society.
One way to see this is to reframe the idea I've been suggesting all through the term this notion that at a certain point, the movies became a form of central storytelling in the society, the form of storytelling that more Americans than any other form participated in and experienced, a form that I've come to label consensus narrative, a kind of story system that appeals across social barriers and across age that intends in some sense to express what might be called the dominant or the central values of a society.
We talked about this idea-- I've talked about this idea already at some length and I don't want to spend too much time on it here.
But I want to complicate those ideas one more time by reminding you that one way to think about or one way to frame this notion that the movies and the Western form in particular embody this notion of a story form that tends to reach the whole society or to speak for the whole society or to embody the central values of the whole society is one way of understanding the implications of that is to recognize, first of all, that the Hollywood system which is so committed to forms of genre representation is particularly served by the principles of repetition that are inherent in the conventions that surround the Western.
Even the Western-- the content of Westerns or the attitudes within Westerns might undergo great transformations, the basic formulas remain recognizable from the earliest days.
We think about the costumes, the horses, the guns, and so forth, and even certain basic plots and subplots that keep returning in the Western.
As with the musical, the genre is so robust that it has many sort of some sub within it so that one can talk about Westerns that involve battles between sheep herders and ranchers.
Or one can talk about the Westerns that are so focused on the laying of the transcontinental railroad.
Or one could focus on Westerns that focus on the conflict between white settlers and Indians.
Now, many Westerns combine a number of these elements.
And there are other sub genres we could describe.
The very fact that there are these sub genres is a measure of how robust the system became.
And one way to think, therefore, about this idea that I've labeled, consensus narrative, or another way to think about what I've earlier in the course called the cultural work of the movies or the cultural, the anthropological job, that genre forms in general during the Hollywood era perform, is to recognize that we can understand the specific genre as a distilled instance of the larger process of the movies, right, as a smaller clarifying instance, we can understand in a certain sense as a forum, a space of conversation, a space of discourse in which the culture engages in a kind of ongoing conversation.
You remember the terms I used earlier borrowing from the English critic Raymond Williams.
I said that we can understand that these genre reforms or these consensus forms could be said to mobilize three different strains of attitude-- an emergent, a dominant, and a traditional set of voices-- and that one of the explanations for why genre texts and the Hollywood movie specifically and the Western in particular are often so conflicted or divided is precisely that they embody or incorporate some attitudes that are traditional and old fashioned, some attitudes that are emerging and challenge traditional ideas, and also what we might central or dominant values, and that not every text will balance those elements in exactly the same way.
The second point about this is to recognize that one of the beauties of genre, one of the reasons they serve such a fundamental cultural or anthropological function in a society, is precisely that they go on forever, that in other words we can think of genre forms as a kind of ongoing continuing conversation centered on the same formats and the same conventions.
And if the genre is stable and if the society produces these genre forms over decades, the genre form itself becomes a mirror in which we can see reflected the changing values and assumptions of the culture.
Now, this might be true of any kind of a particular genre form in any medium.
But the reason it has special force when we're talking about the movies in the 20th century in the United States is precisely because the form is so central to the society, because so much of the society participates in it.
So the picture of the society that is the ambivalent and conflicted picture of the society that emerges from studying these genre forms is likely to be far more illuminating about the larger culture than, let's say, a study of a cultural form, let's say, like opera.
Not that the opera isn't worth studying and doesn't reflect certain cultural values too, but because the opera appeals to a particular subculture, the values are going to be narrower.
The kinds of assumptions the people writing opera and attending opera make about the larger culture and about aesthetic experience are profoundly different from the much more dispersed and democratic attitudes and accessible notions that animate the create of a consensus form, of a form that's trying to reach the whole of a culture.
So this is a way of reminding you of what we might call the sociological and anthropological importance of the study of popular forms in general and of American movies in particular, the Western genre even more especially.
We can think, therefore, of the Western almost as a form of kind of national theater, a kind of repertory company in which the same relatively small number, recognizably small number, of actors and actresses, of directors and other sub specialists whose names are never remembered, collaborate to create the film.
We can think of the movies as embodying, in a certain sense, what I'm calling the theater of a nation.
And I put the phrase in quotation marks not because it's a fancy phrase, but because I want to call attention to the fact that I'm actually borrowing a phrase from a great literary historian who wrote a book about Shakespeare's audience and about Shakespeare's plays and about the public theater of Shakespeare's day and called that theater the theater of a nation.
And in a certain sense, we could say that the Hollywood system was a similar kind of theater.
It doesn't from this that every text created there is a noble work of art.
And it's very important to recognize that the distinction I'm making between and anthropological perspective and an artistic one must hold here.
And it's another reason for emphasizing this.
Because the fact of the matter is it's very hard in a course like this, especially-- because some of these films are so wonderful, my own enthusiasm encourages you to like them and I want you to like them.
Nonetheless, it's very important to recognize that I'm not trying to tell you a merely triumphalist story about how magnificently American movies began to sort of realize their possibilities and dramatize experience.
Because in a certain fundamental sense, these consensus values are not necessarily noble ones.
And the western is a particularly clear instance of this.
And what I'm talking about in a certain sense is the extent to which these genre reforms carry the lies, the prejudices, the nasty qualities of a society as well as their ideals and values.
And we can feel those elements, even though the films themselves want to tell uplifting stories, the values embedded in them may not be so attractive at all.
And the Western is a particularly clear instance of this-- a reminder, again, that we don't just look at films for aesthetic satisfaction.
We don't just look at films for-- we don't need to just look at films for mere enjoyment.
They are anthropological artifacts of the greatest importance exactly because they existed at such a central, at such a fundamental space in the larger society.
Why would the Western-- why would the Western be such a central form?
Why, in some sense, could we think of the movies, with its vast, wide, giant screen, as the ultimate destiny of the Western?
I think one explanation for the power of the Western film, because there were Western stories and other Western formats before the advent of film-- and I'll talk about them in a moment.
Why would the film be such a decisive space for the Western?
And I think if you think about it for a moment, you'll recognize instantly why that is the case.
The format of the movies is particularly conducive to the sense of grandeur and largeness, spaciousness, that's characteristic of Western spaces and of the parables of heroism and social create, society creation, so the equation of a particular culture or environment in the wilderness, the establishment of civilization as a recurring theme in the western.
This kind of recurring subject is at the very heart of what Westerns do and turn the movies, I think, into a particularly appropriate environment for the largeness and grandeur that Westerns represent.
But that itself isn't quite enough to explain why the Westerns occupy such a central place.
And of course, there are historical and cultural explanations as well.
The first explanation is that the movies are born at the end of a period in which we could say the real west actually existed.
In other words, the movies come in just at the moment-- the movies actually are still around when the experience of what we call the West was still historically actual.
It wasn't quite in the past.
The Western genre is focused really on a 30 year period, that what some critics have called the real west, in the period from about 1860 through 1890.
And this is the period of the Civil War, of a whole variety of Indian wars, of the laying of the transcontinental railroad, of the California and Dakota gold rushes, the period of the range wars I mentioned a little while ago between settlers and great ranchers who wanted to keep spaces open for their cattle.
And of course, we now realize it's the period, although the early Westerns didn't want to acknowledge this and told a kind of opposite, it's also of course the era in which we exterminate, virtually-- the era of genocide against Native Americans, the extermination of Indians and of the Buffalo that roamed the Western plains.
And before this real west moment was over, even before it became history, it had begun to be mythologized.
It was mythologized in the popular culture simultaneously with its actual existence.
And there were a whole series of forms of publication and forms of drama that drew on the west and stories of the west and begin to establish what we might call the basic conventions of the Western mythology or the parable America likes to tell itself about the nature of the settling the West.
One of these forms were the dime novels that appeared in the late 19th century and into the early 20th century, available usually in paperback at news stands.
And some of them were immensely popular.
Here's one title, a very famous one, the equivalent of a best seller, I guess, of the day published in 1882-- The Life, Times, and Treacherous Death of Jesse James, 1882.
And of course, what these dime novels did was they took what were quasi historical events, sometimes real events, and immensely elaborated them, fictionized them into heroic fables of various kinds, often celebrating people like Jesse James who may well have been outlaws and disreputable people in actuality.
Another title-- The Illustrated Lives and Adventures of Frank and Jesse James and the Younger Brothers.
And you can imagine what that kind of thing would have done.
In addition to these dime novels, there will also stage performances, plays.
They were often some form of melodrama set in the Old West and involving the basic costumes and accoutrements of a Western, right?
Pistols and cowboy hats and school marms from the east and so forth.
But perhaps even more important than these two sources, and they were very powerful and important forms of popular culture before films to help establish what we might call the mythology of the west, were the Buffalo Bill shows, the wild west shows.
These were unbelievably popular and successful.
Here is a picture of a kind of flyer that would be distributed in small towns or sometimes in larger towns as the Wild West show and-- the Wild West show came to town.
The star of the Wild West show was William Cody, Buffalo Bill.
He had a very modest actual career in the west.
But it was immensely elaborated by him and his publicist into something much grander and more remarkable.
Buffalo Bill made his first age appearance in 1872 in a stage melodrama called "Scout of the Planes," which went on a two year four, right?
And then in 1882, 10 years later, he formed his own wild west show, which became immensely popular, traveled all over the country, and then made trips to Europe.
He even did one Wild West show before the Queen of England.
And it was an immensely successful public enactment.
The basic elements of the wild west show contained the essential mythology that then moved into the movies.
They often dramatized-- in the wild west show, they often dramatize a battle between white men and Indians in which the Indians were very treacherous and were finally defeated.
They sometimes, in fact, in some of the forms of the wild west show, they reenacted the Battle of Little Bighorn in which at the last minute, there was a rescue.
They reversed history in it sometimes in order to dramatize an outcome that was more appealing to the white audiences.
The wild west show staged battles between cowboys and Indians.
And in fact, one of the great historical ironies, the historical irony that the director we're looking at next week, Robert Altman, actually picks up on in one of his films called "Buffalo Bill and the Indians," a film he made in the 1970s-- 1975, I think.
One of the shocking things is that at a certain point, at least for two years, I think, one of the participants in the Wild West shows was Sitting Bull himself, the defeated Indian, the defeated Native American leader who was now domesticated and actually became part of the show.
And in Altman's fictionalized version of this episode, a drunken Paul Newman plays Buffalo Bill.
And he's mocked and made fun of in some sense by a character who represents Chief Sitting Bull, who has all the dignity-- he's almost the only character in the film with real dignity.
It's a subversive post Hollywood era Western, as you might guess.
So the Wild West shows, the Buffalo Bill wild west shows, were essentially an early rehearsal for what the movies became.
And many of the elements of the wild west show-- fancy roping, fancy shooting, conflicts between cowboys and Indians, the performance of various horse riding tricks, herding of cattle and so forth-- all sort of contained the elements that then moved into the movies in an even more systematic and powerful way.
And then I might mention one final source-- the novel itself.
At the beginning of the 20th century, in 1902, one of the best selling novels of all time in English was published.
It was a book called The Virginian by a writer named Owen Worcester.
And it remains one of the bestselling books in the history of the United States.
It still sells thousands of books every year.
I once found, in a used bookstore, an addition of The Virginian that had been published I think 15 years after its original publication.
And that addition was, I think, the 50th addition of the book.
It went through an incredible number of-- and in fact, the addition I had also included beautiful illustrations that obviously were added to later additions once the popularity of the book became known.
So what we find is that the popular culture before the advent of movies had already, in some sense, created a full fledged mythology of the Western, had familiarized the society with the basic attitudes, assumptions, costumes, and fundamental archetypal characters who organize the Western.
And there's one further thing to emphasize.
This may be the most important thing of all, what we might call the intellectual culture.
It was-- not only was there a real historical background to the Western and not only was there a profound form of popular culture in theater, in public performances, like these wild west shows, and in dime novels as well as in literary novels-- a tremendous compost of discourse that already sort of establishes the genre even before the movies arrive.
There was also a final touch.
And this is the thing that helps to explain, I think, maybe more powerfully than the other examples I've given you why the Western had such a hold and continues in some ways to have such a vestigial hold on the American imagination and on America's sense of itself.
And that is what happens in what I call here the intellectual culture.
In 1893, one of the most famous historians, Frederick Jackson Turner, published an essay in the American Historical Soci-- or delivered a lecture and then it was later published as an essay-- delivered a lecture to the American Historical Society.
And the title of the essay was "The Significance of the Frontier in American History."
it's probably the single most influential historical essay about the American past.
It's been attacked and undermined and qualified and its limitations have been widely discussed over the last 25 years.
But it still remains, in some respects, a seminal essay, even when historians are reacting against it-- one of the most important central essays about the nature of American culture.
And this article-- in this article, Frederick Jackson Turner argued essentially that-- he was trying to make a case for what makes the United States unique.
He was making a case for what we now call American exceptionalism, a dubious proposition.
But let's listen to his argument, which has a good deal of power and interest.
He said essentially that the existence of free land, of a continuously receding settlement, of a continuously receding frontier, the advance of settlement westward with European settlers always pressing against that frontier, that the existence of such a reality in American life had altered inherited notions of social class and aristocratic government and had created a tendency toward what Turner implied were or said were the American ideals of individualism and classlessness, of democracy itself.
In other words, the United States, according to Turner's argument, had a particularly open and democratic feel because the existence of this frontier allowed people to escape into a new life, to find possibilities, to escape the confinements of social class, even perhaps in some small degree of gender, although feminist historians have certainly shown how confining the life of women in the west actually was.
Nonetheless, a sense that the existence of the frontier made the United States a uniquely individualistic and a uniquely free society.
What Turner had done, of course, was define a central historical myth for European Americans, for the white society in the United States.
And we've come to recognize that limitations of the argument.
What's the most profound, obvious limitation that occurs to you instantly?
How about the people who already here, right?
In other words, one of the things that's wrong with the Turner thesis is that it's oblivious to the existence of the Native Americans who were pushed aside by the white settlers.
It's a very complex story and I don't want to oversimplify it.
I don't mean that the heroic white people who got in the Conestoga wagons and who settled the west, about whom there are dozens if not hundreds if not thousands of-- certainly thousands if we count stories in all formats-- of movies and books and short stories and plays and so forth.
I don't want to suggest that there was nothing heroic or remarkable about that behavior.
There was.
The United States is not wrong-- American culture is not wrong in some sense to celebrate the heroism of those people.
But of course, as we've come to realize, and as many of our most recent Westerns openly acknowledge, this original mythology which sort of celebrated unambiguously the white settlement of the West was actually also in some sense an invasion and even a genocide.
And the Turner thesis doesn't even imagine that.
It sees it as a triumphal reality.
So there were limits, terrible limits, to the argument.
But there's also something profoundly helpful about the argument.
There is a sense of spaciousness, a sense of new possibilities, a greater hostility to social class and inherited wealth in the United States than there are in other places.
And for both good and ill, the United States celebrates forms of individualism as against communal behavior with a kind of intensity and compelling power that many, many, not just Europeans, that many, many non Americans think is excessive and think has been reflected in some sense in various forms of national behavior which we needn't discuss in detail now.
But you could all imagine your own examples.
But to come back to the primary point, then, one of the reasons that-- it's as if the movies were sort of waiting for the Western, as if the Western form already in some sense, in fundamental ways, fully formed comes into the movies just as the movies are born, as if the format of the movies and the content of the Western were made for each other in some sense.
And it becomes one of the dominant, one of the signature features of American cinema.
One of the fundamental aspects of what the Western is, if we think of it as a kind of cultural mythology, is that it tells a founding story.
It tells the story again and again.
It repeats it every time you see a Western in some sense.
Every Western is implicitly a story about the foundation of a society, about the equation of a community.
It's always implicitly about it.
In many Westerns, this is an explicit topic.
And what we can say is that this means that the Western is the United States' form of a kind of story that every culture has, right?
The ancient Greeks had a founding mythology grounded in The Iliad and The Odyssey, right?
And Tudor England had a founding mythology partly promulgated by Shakespeare in the history plays that were put on in the public theater in the 16th century.
And what we can say about these founding stories as they are always, in some sense, fantasies or fictions.
But they articulate values and assumptions that the culture, in some sense, would like to believe about itself.
And by their reiteration, by their repetition, by telling them again and again, they instantiate these values, these assumptions about the society, in the culture as a whole.
Their very repetition is important.
Because one of the fundamental things about a myth or a mythology, one of the most fundamental things about it, is that it's a collective-- it's a collective story.
It's not made by a single person.
And one way to think about all Westerns, although of course many of the best Westerns have the trace of-- more than the traces-- have the imprint of individual contributions.
John Ford-- John Ford is a great director.
And there are aspects of John Ford's personality in every one of his Westerns.
Nonetheless, even the most profoundly Fordian Western is still a collaborative form, not only because Ford as a director-- because the movies are inherently collaborative because they have so many different people who contribute to the creation of the whole.
No one, even the most powerful director, can completely control everything.
The greatest directors know how to collaborate with their performance and their writers.
But in any case, it's a collaborative form to begin with, right?
But there's another kind of collaboration, several other kinds.
What are they?
One of them is that every Western is implicitly in a conversation with every previous Western.
And in that since, there's no author.
It's a collective-- the story of the American Western, in that sense, is a collective story.
It's a creation of the culture.
And in fact, every new version of the Western adds to the mythology in some sense, right?
So in that sense, we can think of genre reforms generally, the Western in particular, as a particular kind of ongoing, recurring discourse.
And if you come back again to the arguments I was making at the beginning of our class about the ways in which genre forms and especially the Western can be seen to sort of dramatize cultural change or the way in which a society's attitudes may alter over time exactly because so many elements of a genre are stable and recognizable, those that are different are easy to recognize, are easy to identify and speculate about.
So one of the things we're recognizing here is that in a certain sense, the very fact that the mythology is open, is unfinished, reflects something that's fundamental about society too.
And this is part of Raymond Williams' great insight into culture when he said-- what he said was culture is a constant mix of emergent, traditional, and dominant voices.
It's never over.
What culture is is a conversation.
It's never any one fixed thing because culture is always in process.
It's always changing.
And what we can say-- the value of these genre reforms, and especially because the movies could reach so widely, even across linguistic barriers, the value of the movies or the power of the movies as a form of this kind of discourse is overwhelmingly clear exactly because we can imagine this process, this conversation, going on repeatedly over time-- the recognizable elements juxtaposed against the elements that are not recognizable, allowing for a kind of constant process of reevaluation, of rehearsal, and of rethinking, right?
And we can say, in fact, that that's one of the deep functions that genre forms in general play and it's one of the central functions that the Hollywood movie plays.
And I'll try to concretize this for you this evening in a kind of-- it's a kind of comic attempt, but I'm going to give you a kind of capsule history of the trajectory of the Western from its earliest days, from "The Great Train Robbery," to "Unforgiven."
So the Western is a mythology.
It's a mythology.
The fact that mythologies themselves are open ended, are spaces in which a society defines and redefines its identity and its central values, makes the Western movie a particularly powerful and central anthropological space.
It enacts, in different ways over and over again, a story about the founding of society.
And at the end, it is grounded therefore also in a series of complex dichotomies that are worth making clear, even though they're very obvious when you think about what all Westerns do.
But think about them.
One of the dichotomies is the dichotomy between what is thought to be civilized and what is thought to be savage-- between civilization and savagery.
Another is often between East and West in which the east represents civilization and the west the sa-- but there's an ambivalence.
Because the east is also a place that is thought to be dessicated and lacking vitality.
And the west is thought to be an authentic place that tests us in our heroic, largest possibilities, right?
And another kind of dichotomy that is present in virtually all Westerns is a dichotomy between what we might call the claims of community, the claims of society on the one side, and the claims of the individual on the other side, right?
This is a special problem for the Western because it celebrates American individualism so powerfully.
The iconic scenes of the Western in which two men wearing guns square off against each other and the quicker one survives-- that scene of the quick draw one and of the death in the dusty street could be sent to be a kind of embodiment of these values in which you have to rely on yourself.
You can't rely on someone else.
You can't rely on the government.
You can't rely on the Sheriff.
You can't rely on the police.
You can't rely on your neighbors.
You have to trust yourself.
And in fact, one of the ways you can see the power of these Western mythologies operating, as it seems to me in some sense, one of the great divides in the United States today over the question of guns and gun control is partly grounded in these myths of individualism that the Western mythology has been promulgating since before the advent of the movies.
In other words, the attitudes-- and in fact, if you listen to even some of the most extreme forms of gun rights advocates actually talk in a language that we can recognize from Westerns, from Western movies.
Not an accident that one of our recent presidents was a star in Western films and actually remembered some films he'd been in as if they'd been actual history.
And I don't say that in a way to sort of disparage poor Ronald Reagan at all, but as a way of reminding us the confluence or the confusion of actual history with these mythological accounts of the west is a characteristic feature of the American imagination.
How could it be otherwise?
We have been so inundated in these stories about the west.
So another dichotomy, then, is that between society and the individual, maybe finally we can say a dichotomy between culture and nature, right?
Which is the story that's enacted again and again in the great epic stories that every culture sort of generates for itself.
One of the most distinctive features of these dichotomies is the divided hero, a character you'll see embodied with great complexity in tonight's film played by John Wayne, a character who is-- the Western hero is often seen very ambivalently as both a savior and a savage, as someone who has the power and authority to bring a kind of decency and lawfulness to a community, but who is himself so touched by, so tainted by the very savagery that he tried to tame, that he's often excluded from the community once it's been created.
And you'll see a wonderful instance, a complex instance of that, in tonight's film in the way the John Wayne character, at the very end of the film, is in a certain sense excluded from the community that he has helped to defend.
One way to see many of these things or to clarify many of these things is to juxtapose-- or maybe as a way of crystallizing this process I'm talking about whereby a genre form can, in some sense, dramatize changing values.
I want to show you one scene.
Maybe I'll just end with the scene.
This is a scene from "McCabe and Mrs. Miller", the film you're going to see in two weeks, a Western that comes after the breakdown of the Hollywood system, after the movies no longer have the responsibility or the obligation-- no longer saw themselves as having to speak to everyone-- a moment of great, in many ways, extraordinary artistic freedom, the period of the late '60s and through a good part of the 1970s.
Many people would say it was a moment in American cinema unrivaled for its complexity and its richness.
And one of the reasons it was such a rich moment was that the movies had been liberated from their responsibility to talk to the whole culture.
There was an irony connected to that because they have smaller audience.
But it gave them a kind of political freedom, a kind of moral freedom that they hadn't had before.
And I want to show you one scene.
One reason I'm choosing it is because it will resonate for you when we come to "McCabe and Mrs.
Miller."
But it requires that you know one thing.
The scene, the iconic scene that I mentioned earlier where two heroes or one hero and a bad guy square off against each other on a dusty street to draw a gun might be said to be the most recurring or the most-- the quintessential Western scene, although there are others we might nominate as almost as important.
And in order to understand the scene you're about to see, you need to recognize that what lies behind this very subversive version of the shoot out are dozens, if not hundreds, of earlier scenes in which heroic characters that we greatly admire shoot down bad guys in the street, right?
Well, here's a scene from "McCabe and Mrs. Miller" of this iconic moment.
Remember, "McCabe and Mrs. Miller" comes after the consensus dispensation.
It's free not to be a consensus form.
But one reason I want to show you this is I hope you can recognize that what gives this scene its power is not-- it's very brilliantly photographed and dramatized.
But what gives it its power has very little to do with what it itself does.
What grants its enormous authority are the dozens, if not hundreds, of prior scenes with which it's in conversation, against which it is contrasting itself.
Because this scene of confrontation between two individuals-- who has the faster gun, right-- is, as I've suggested, the iconic moment in the Western.
And it is, in the heroic Western, in the traditional Western, it's a heroic moment of individual self assertion in which good conquers evil.
Watch Robert Altman's version of this
Careful.
Come back.
[VIDEO PLAYBACK] [?
-There's ?]
shorty!
[INAUDIBLE] [DOG WHINING] [CLINKING] [GUNSHOT] -I wasn't trying to hit it.
The trick is not to hit it, but to make it float
He's lying.
You're supposed to realize he's lying here.
He meant to hit it.
Here comes the cowboy who's just [INAUDIBLE].
All the prostitutes loved you.
-Hey, hold it, sonny.
-What?
-Hold up on your target practice a minute.
I don't want to get shot.
-Well, then get off the bridge, you saddle tramp.
-I want to buy some socks.
I got a long ride ahead of me.
-What's wrong with the socks you got on?
-I wore them out running around half-naked in that whorehouse over there.
Actually quite a place.
You been there yet?
-Take off your boots and show me.
-You're joshing me.
-I said, take off your boots, and show me, eh, sucker?
-I ain't going to do that.
-What are you wearing that gun for?
-[INAUDIBLE] I just wear it.
Can't hit nothing with it.
-Well, that don't make no sense.
What kind of a gun is it?
-Colt.
-Them's big guns.
That's what I got.
Must be something wrong with it.
-Naw, it's me.
I just can't shoot good.
-Well, let me see it.
Come on.
Maybe I can fix it for you.
-
OK. [END PLAYBACK]
So what the scene-- what the scene does is it turns on its head what we might say are the central values or assumptions that have lain behind such a seen in dozens, if not hundreds, of prior Westerns.
And that's a distilled example of this process of a conversation that goes on-- if it goes on for a long enough time, the content of the conversation can virtually reverse itself even though the genre remains stable.
And that's one of the-- we might call that the genius of genre, the genius of a genre form.
It has to do with-- it's a version of what I mean when I talk about the cultural work or the anthropological function that movies have.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
What I'd like to do very quickly is give you a brief account-- much to skeletal and simplified-- of this conversational drama-- the way in which the Western film becomes a screen on which American values are projected, and on which American values are tested, and in which we see American values or assumptions undergoing certain kinds of transformation-- certain kinds of change.
I don't expect you to remember these titles or all of these titles although the ones I talk about you might think about.
And this is very far from a complete list of films.
It's just a list of highlights.
But I want to give you a sense of, first-- that the Western film is at the center of movies at every phase of the history of the medium.
And as we've noted, one might argue that one of the very first films-- some people have called it the first narrative film although that's a slight exaggeration-- but one of the most fundamental founding documents in the history of movies is, of course, a Western.
If not its first story-- film-- one of the earliest narrative films was a Western.
And we might note, in fact, that Western, The Great Train Robbery, was also based itself, not on-- although it has mythologized and fictionized-- The Great Train Robbery is based on a historical actuality-- on a robbery that was very widely covered in the press about which dime novels were written and which became a site of many films-- the first one The Great Train Robbery.
A more recent one that you may have heard of if you haven't actually seen it-- Butch Cassidy and the Sundance Kid which tells the same story.
It's based on the same gang.
It doesn't tell exactly the same story, but it's based on the same gang that was said to have committed the great train robbery.
And that calls our attention to another factor-- another aspect-- of this-- what we might call the story of the mythological aspect of American cinema, and especially of the Western, because it calls our attention to the fact that the same events may be narrated again, and again, and again.
One of the points about myth-- about collective stories-- is they're literally told repeatedly.
One way to think about the Western-- say, OK.
There are variations but, basically, it's the same story.
Of course, that's a tremendous simplification because it's thematic, and moral implications can change radically-- but, basically, the same set of characters, the same setting, the same materials told over and over again, obsessively.
Why would a society do this?
What is driven-- what drives us to do this?
Well, one answer is there are pleasures in familiarity.
One of the deepest lessons of the history of literature is that the romantic and modernest idea of the artist as a unique individual creating art ab novo-- sui generis-- with no connections to the past is a fantasy and a delusion-- a particular outgrowth of the romantic and modern period.
But that throughout history and throughout most cultures, art, and especially storytelling, has been much more like the activities of these popular formats in which the same kinds of stories are repeated again and again with variations.
And one could say that one of the profoundest resemblances between Shakespeare's time and our own is the dominance of genre forms in a popular theatrical environment.
In Shakespeare's day, it was the popular theater that had a similar effect of trying to define in some sense our notion of national identity.
And that's why I quoted the Renaissance scholar this afternoon who called Shakespeare's theater "the theater of a nation."
He was trying to make the same kind of argument about the importance of the centrality of the public theater in Shakespeare's day that I'm making about the movies in the United States in the 20th century.
So The Great Train Robbery is a kind of originating text both for film itself and for the genre of the Western.
Griffith's The Battle of Elderbush Gulch was one of the first longer films.
I think it was a half an hour or 45 minutes long.
It was a much longer creation than earlier films, and it was one of the first major examples of large crowds of people-- mounted people-- engaged in battle.
And that was one of the most astonishing things.
It was a tremendously influential film.
It dramatized battles between cowboys and Indians in a way that showed the power of the medium of film at a very early stage.
In 1914, the first movie dramatization of that novel by Owen Wister that I mentioned earlier-- that was also one of the founding texts in the Western genre.
I've mentioned a couple of other films here.
One of them-- one of John Ford's earliest films to demonstrate that-- I've got two Ford films listed there to try to give you a sense of his importance in the silent era.
I list The Covered Wagon as a way of showing you that the genre of the pioneer story was already present in the silent era.
I list The Iron Horse by Ford as a way of reminding you of how central Ford is but also a way of reminding you that that story of the laying of the railroad-- the Intercontinental Railroad-- one of the central sub stories-- recurring stories in the Western begins in the silent era.
And I mentioned the Keaton partly to show you that the genre had become so widespread that it was now so fully-fledged-- so fully recognizable-- even in the silent era that Keaton was able to make a comic parody of Westerns called Go West.
With the advent of sound, the first decade of the Western movie is, from an artistic standpoint, not very distinguished.
Can you guess why one primary reason is?
If you look, I say singing cowboys-- because it was sound.
The very first thing said, oh, well.
We're so enamored of sound.
The novelty of sound is so remarkable.
The first thing we'll do is we'll introduce cowboys who sing.
And there were primitive stories that they told, but the stories were relatively superficial.
I mentioned In Old Arizona, the first sound Western, in part because what was amazing about that film, especially to audiences, were literally the sounds-- the crackling of the fire, the creaking of the saddles.
The noises of the West were fascinating to people.
So the '30s, from an artistic standpoint, were not a very interesting one in terms of the Western's development.
Some scholars have argued that the silent Western was more interesting than the Western of the '30s.
But they're worth mentioning because they were very popular, and many Westerns in the '30s were identified as what were called B-films.
That is to say, as you may know the-- I wish I had had time in this course to spend some energy talking about what might be called the movie house Culture-- how incredibly the whole society after about 1930 or in the early '30s was permeated by the movies-- how every major city had dozens-- maybe even more than that-- of movie houses showing films that-- many levels of films-- recent films and many older films as well.
And as I think I mentioned in an earlier class, it was possible to go to third and forth-run movie houses showing movies that were several years old and to pay something like a quarter in the morning and go in and watch five or six movies.
And I did this as a child in the late '40s in Brooklyn, New York where I would go into a theater and come out-- I would go in in the bright morning.
I would come out-- it would be dark.
They were lousy movies, but I was a kid.
I didn't know any better, and it was an astonishing kind of culture.
I should also mention, I call it the movie house culture because, of course-- again, I haven't had time to develop this.
But if you google "movie houses," what will immediately appear are a series of magnificent photographs or images of movie palaces.
You could even google "movie palaces" and find the same answer-- that now mostly fallen into disuse or destroyed that were all over the country-- very exotic and remarkable.
Even the physical structures of these theaters were exotic and exciting.
These exhibition spaces were themselves meant to excite you and take you into another world.
And partly because of that, what they always offered was a really robust ticket of offerings.
You didn't just go to the movies and watch one movie.
You saw at least a double feature.
The second feature was often what was called a B-movie-- a second rate movie-- lower budget.
Many of them were Westerns.
One reason for that was because the accouterments of the Western are so interchangeable.
As I mentioned in an earlier class, one of the things they could do was use stock footage to put these Westerns together very quickly.
So they could have a stampede that might have appeared in many different movies but will do the job in this B Western and so forth.
And they could use the same props, the same town, the same horses, and the same costumes, and so forth.
So for a lot of the '30s, the Western movie was a kind of back-up-- a second banana to the primary film.
In addition, there began to develop, especially on Saturday-- in this movie house culture, Saturday mornings were especially for children-- even Saturday afternoons.
They began to develop what they called matinees which were intentionally devoted toward trying to get younger people-- children-- into the theaters.
And they had a series of sort of B or C Westerns some of which were serials and would continue from one Saturday to the next.
I want to mention my favorite childhood Western hero whom I still remember.
And I remember I loved this character when I was seven and eight years old and would go to the movies on Saturdays and see-- sometimes the serials would last several weeks.
Part of it was to try to get the kids to come back and get their parents to pay for them to come back every Saturday.
So they would always end with a cliffhanger-- with a moment of great tension-- and you wonder whether the hero and heroin would survive into the next week.
And there were a whole series of such films.
My favorite such film was a film where a series of adventures starring a character named Lash LaRue-- can you guess what Lash LaRue's weapon of choice was?
A whip
Yes, the whip-- a bullwhip.
He never carried a gun.
But he had this long bullwhip, and he would take this whip and he'd go-- [CRACK] --like that.
And it would snake out across, and it would flip a gun right out of a person's hand.
Or he could go-- [CRACK] --like this, and a little tiny spot of blood would appear on a person's nose.
It was a unbelievably, ridiculous fantasy.
And the lash made a tremendous beautiful crack when he brought it back.
He was obviously a figure of childhood fantasy, but it was a tremendously, memorable film.
And, again, there was no pretensions toward psychological seriousness or even thematic density in these Westerns, but they permeated the culture.
They were part of the experience of childhood.
So children in the society would grow up watching these children's versions of Westerns, and then they would graduate to the more adult Westerns.
I don't want to call it a form of brainwashing because that's much too negative a way of describing the way in which these story forms shape our understandings or help to shape our understandings.
And I don't mean that human beings are completely programmable creatures-- that they go and look at a movie, and they become robots following what the movie said.
But what I am saying is that the way these kinds of stories permeated the society helped to shape Americans understanding of their natures-- of what the social fabric was like, of what masculinity was, of what femininity was, of what families were, of what the relations among the races were, of what American history was, about what the founding story of America is, about what the central organizing values of our culture are.
All of those things are dramatized either explicitly or implicitly in movies.
And because the movies were so central through the 20th century, they were one of the central ways in which the belief system-- the values-- of American society were promulgated, dramatized, rehearsed, and in some ways altered and changed.
The classical age of the Western film is often said to be the period of the 1940s, and this is really the moment when the Western as an artistic form begins to come into its own.
Although, it also still carries ideological or thematic baggage we might not want to embrace or celebrate.
So remember, it's an ambivalent thing here.
I'm asking you to respect the artistry of these films, but I'm also asking you to do something mature and grown-up which is be skeptical about the meanings that are embodied in what are sometimes very artful movies.
And we call it the classical age, in part, because a certain kind of much more ambitious intellectually powerful and artistically organized Western begins to appear, dominated-- but not completely-- by the great director John Ford-- one of whose masterpieces you're going to see in a few minutes.
The age probably begins in 1939-- the same year that the integrated musical begins with-- what was the film?
We mentioned it last week.
The Wizard of Oz in which color is introduced for the first time.
Well, here the genre of the Western comes into its own, really, roughly in the same year in a film that stars John Wayne, who was one of the iconic stars of the American Western, and directed by John Ford-- his favorite director-- a director who did other things but who was noted, especially, for his remarkable Westerns.
Wayne had had a career in the movies 10 years before that, but in Stagecoach, for the first time, he became a major star.
The film made him a star.
And Stagecoach is an interesting film because it articulates or formulates one of the great sub genres within the category of the Western.
And this is-- I don't know exactly what to call it.
It's a story in which you have a group of unlike people who are traveling, usually by stagecoach or in some other-- sometimes it can be by train-- who are beset unexpectedly by some kind of danger-- usually an attack by Indians.
And then the uneasy community that is forced upon them when they have to defend themselves reveals various aspects of human nature and of human society.
And the film Stagecoach is a classic instance of that in which the John Wayne character gets on the stagecoach as it's making a journey across a bleak Western landscape through Indian territory.
And he's a, kind of, ambiguous figure in this film-- the Wayne character-- perhaps an outlaw.
And the respectable people who are in the stagecoach-- a banker-- a character who's actually embezzling money and is trying to get away although no one knows.
He hasn't revealed it at the time.
It comes out in the course of the film-- and a series of other sort of characters with interesting backgrounds that emerge when the catastrophe descends upon them.
And, of course, what happens is the ambiguity of the Wayne character-- the fact that he has a kind of connection to lawlessness turns out to be one of the great virtues that he has because, of course, he becomes the primary defender of the group when they are beset by savage Indians.
And then I've listed-- again, let me remind you.
Again, I've just picked-- these are just highlights.
Remember what we said.
Nearly 30% of all films made during the studio era were Western movies, and these are among the highlights.
From the titles you can see the recurring sub genres that occur.
Billy the Kid in 1941-- the film directed by David Miller-- this preoccupation with outlaws.
There are a number of films about Billy the Kid, a number of films about Jesse James, and about the gun fight at the O.K.
Corral.
The 1946 film My Darling Clementine-- Ford's famous film starring Henry Fonda-- is one of six movies made about the gun fight at the O.K.
Corral.
And think of what that says to us.
What does a myth do?
It tells the same story again, and again, and again.
Each of those six films is in many ways different from-- in significant ways, each film is different from the other film even though the basic information, or at least the basic situation, is the same.
And, in fact, in some versions of the film-- in some versions of that account-- the account of the O.K.
Corral-- Wyatt Earp and the Earp brothers fight against the Clanton brothers.
In the classic versions-- in the version My Darling Clementine in 1946 by Ford, the Earps are heroes.
And Wyatt Earp is a great American-- is a noble American hero who defends a town against anarchy and evil.
The Clantons are entirely evil.
But in later revisionist versions of the same story, Earp and his brothers are seen as arrogant officials who are trying to control gambling and prostitution in the town of Tombstone.
And the Clantons are not seen as noble but are seen, essentially, as rivals of the Earps-- not necessarily as inferior, evil characters.
And such a perspective, of course, profoundly changes our understanding of what the gun fight at the O.K.
Corral represented-- interesting and deeply revealing that so many Westerns have been repeated.
I mean, the same story told again and again.
And so the fact that the gunfighter at the O.K.
Corral was repeated at least six times in feature length films-- there were also television versions of the O.K.
Corral event, adding to the number-- is a dramatic example of what we might call this mythological function or this forum function-- this discourse space function that genre forms in general and the Western movie, in particular, have.
I've already mentioned Red River in my earlier discourse about Howard Hawks.
And I want to just remind you that that great director of screwball comedy was also one of the great directors of masculine adventure films including what many people think is one of the great Westerns of all time-- Red River.
And Red River is an interesting sub genre as well.
It's a cattle drive Western.
Many Westerns are about cattle drive, but this one also adds a very interesting complexity because there's a kind of oedipal drama and acted out between the John Wayne figure-- recurring figure in so many Westerns-- and a younger man played by the great actor Montgomery Clift who plays a surrogate son to the John Wayne character.
And, of course, conflict develops and the rivalry between father and son is a fundamental subplot in that remarkable Western.
One other thing I should say about the '40s period-- I call it the classical age primarily for this reason-- I sort of didn't name this.
Forgive me for not mentioning this earlier.
Again, we're talking about the period just before, during, and after the Second World War.
The United States feels itself to be beleaguered.
The world is beleaguered in a way.
That outer reality also helps to explain why the Western of the 1940s would be, essentially, a celebration of American power, a celebration of American empire, a celebration of American know-how and competence in the wilderness because, by analogy, the Western is a story of American success-- of American conquest.
One might think of it as a story that American needed to tell itself during the period.
So the war-- the Second World War is a great reinforcement to the way in which-- in the classical age of the American Western, the Western film is basically telling an heroic story, is celebrating the conquest of the West, and is seeing the white man's settlement of the West in relatively unambiguously, admirable terms, celebrating the courage, and the resilience, and the physical prowess of the pioneers and the great, noble gun fighters who tamed the West for us.
But by the 1950s, this confidence has begun to wane.
Part of it is, look-- the war has been over for a significant time.
American society is in a peacetime boom which is going to build through the '50s.
And it's also a period in which a certain kind of-- this was, of course, the period of my childhood.
I was 10 years old in 1950.
And I remember it vividly.
In fact, saw some of the films that are listed here in the theaters for the first time.
So I mention that to you just to remind you that, in fact, what we're talking about is not ancient history at all because I actually experienced some of the films on this list in my own actual lifetime when they were originally shown in the theaters.
And although I am infinitely old compared to you guys, I'm still a vigorous, functioning human being.
So within my lifetime's memory, many of these films were contemporary.
And it's important to realize how close we are, in fact, to these times.
But one of things that begin to appear in the 1950s was a kind of skepticism about the social arrangements in the society.
At first they were very modest and mild.
There was in the 1950s, for example, what came to be called a generation gap in which older and younger-- in which there was thought to be a distance between older and younger people.
And there was a great concern in the 1950s for alienated youth and juvenile delinquency which was a great catch phrase.
And some of the films of the 1950s dramatized that.
For example, the film The Left-Handed Gun in 1958 starring Paul Newman-- directed by Arthur Penn-- is a story really about-- it's a Billy the Kid story.
Newman plays a Billy the Kid type character, but he plays a young boy.
He plays, really, a post adolescent.
And you can feel that what's partly being dramatized there are teenage angst.
There are-- and what also begins-- in 1952, the film High Noon-- a fragment of which I will show in next week's lecture when we talk about McCabe and Mrs. Miller to do a contrast with a classical Western and a post-classical Western-- that film introduces certain kinds of contemporary political concerns.
Many people see High Noon as a parable about the McCarthy era-- Senator Joseph McCarthy and the fear of-- the paranoia about communism and the anti-communist hysteria of the late '40s and early 1950s when the Cold War was just getting powerfully started.
And High Noon is a kind of parable about a good man who turns to-- a sheriff played by Gary Cooper-- one of the great iconic heroes of masculinity of the classical Hollywood era along with John Wayne.
And Cooper plays a sheriff who has brought peace to the town.
He's been a sheriff for many years.
And on his wedding day, he's marrying a Quaker-- a Quaker woman.
He's giving up his gun because a Quaker woman, played by Grace Kelly-- you'll remember her from the Hitchcock film although here she plays a Western heroin.
She's a Quaker, and he's putting down his gun to join his wife.
And on his wedding day a man that he sent to jail is coming back by the noonday train to take his revenge.
So the Sheriff spends the early part of the film going around to the townspeople-- a town that owes its life to this great, heroic sheriff saying, help me.
I'm going to be attacked by Frank Miller and his gang.
No one will help him.
He's left on his own.
He has to stand by himself because the town is too cowardly.
And it's a parable of the cowardice of people who deserted-- it's in some sense a, kind of, left wing argument.
It's a parable about the cowardice of people who wouldn't stand up for folks who were attacked by Senator McCarthy and other anti-communists for being disloyal.
So certain kinds of political meanings begin to-- skeptical political meanings begin to creep in.
And the society that's dramatized in High Noon is hardly an admirable society.
The Western culture may have created a town, and the town has a church.
And they have a meeting in the church in the film.
He goes-- it's a Sunday, and you can hear the church bells.
And he goes to the church, and he asks for help from the congregation.
And they won't do it.
They're too cowardly.
And so it's a parable about cowardice, and about political dysfunction, and about communal disloyalty.
Also, already by the 1950s, before the Hollywood era has completely concluded or been obliterated-- Hollywood dominance is still significant in the 1950s-- the Western has begun to change.
It's begun to reflect social changes as well as thematic or ideological changes that have to do with the way the society is altering after the Second World War.
And a number of the films that I've listed there do that kind of thing.
And one can also see in this listing of so-called adult Westerns a greater emphasis on psychological themes-- on the conflict between individuals.
One very powerful and interesting instance of that is a film in 1967 directed by Martin Ritt, starring the great Paul Newman, in which Newman plays a half white, half Spanish, or half Indian character.
I think he's an Indian-- a Native American.
And it's a parable about racism.
The Newman character is angry at both the white and the Native American society.
He doesn't fit in any place.
But it's especially a parable about his exclusion, about the Paul Newman character's exclusion from white society, and about the power of racism.
By the time we get to the end of the 1960s, we're beginning to get films that have a kind of violence quotient that's different, and bloodier, and more disturbing than the violence we saw during the classical age.
And we especially associate this form of violence with the director Sam Peckinpah whose film in 1969, The Wild Bunch, takes a number of actors who had been stars in classic Westerns and brings them back in a Western that is much darker and shows these older men as-- first of all, their age is emphasized.
They're sort of doddering around.
They're my age.
They're in their late 60s-- early 70s, and they don't seem as vigorous as they should be, for one thing.
I think there are even scenes in which their nearsightedness of one of the characters is dramatized.
But their age and trembliness is only a part of their unheroic dimensions.
And we can see that the film, although it celebrates these heroes in a certain way-- the celebration is cankered.
It's damaged.
It's a damaged kind of celebration.
There's a sense that these men are past their prime, that they're sleeping with women who are far too young with them, and there's something kind of disgusting about that.
We see that one of the actors-- an older figure-- Robert Ryan-- a very gifted actor who was in many, many Westerns and other films in the classical age of Hollywood-- plays an agent and very effective gunfighter.
There's a terrible scene in a brothel where he's-- she's not a teenager, but there's a woman surely 30 years younger than he who becomes his bedmate.
And there's something sour about the scene intended, I think, by the director.
I mean, you can feel it.
It's as if the Western values have begun to congeal and sicken even before we get to the-- what I'm calling the-- anti-Western.
And, of course, in the late '60s there also begins to emerge the phenomenon I talked about earlier this afternoon-- how the Italians start making Westerns-- putting them in Italian and they find locations in Europe that replicate American deserts and so forth.
And the Italian Westerns, especially those that are associated with Sergio Leone, have a dark, existential, almost nihilistic flavor.
And one of the things that Leone did that was very subversive.
He took characters-- American actors-- who were associated with great, heroic, idealistic roles in American Westerns of the '40s and '50s, and he put them into Westerns in which they played evil villains-- murderers criminals who had no compassion or pity for any of their victims.
And there's something shocking about seeing noble Henry Fonda who played young Abe Lincoln in John Ford's movie and played so many wonderful, heroic figures including Wyatt Earp in My Darling Clementine suddenly turned into a stone killer.
And that kind of reversal anticipates what happens in the 1970s.
And, of course, what happens in the '70s-- partly, again, a consequence of transformations in American society.
I'll talk a bit about the cultural history of the '70s next week when I talk about film in the '70s in a systematic way.
But suffice it to say for now, that what goes on there is a series of Westerns begin to appear that are permeated by the anti-establishment values of the counterculture-- that are permeated by the anti-war movement-- because the era of the Vietnam War-- of tremendous fissures in American society.
I'll talk more about this next week, so I don't want to repeat myself.
So it's a period in which American culture is-- it's conflicted and divided in a horrific way.
And the Westerns will begin to reflect that.
And, in fact, the old heroic enterprise of the classic Westerns of the 1940s and early '50s is suddenly transformed in a series of anti-Westerns or dissenting Westerns.
In 1970, Ralph Nelson makes a film called Soldier Blue in which the American cavalry, the heroes of so many films including a trilogy of films by John Ford made in the late '40s called the Cavalry Trilogy in which-- John Wayne is in one of those films.
At least one of those films celebrates the American-- the blue-suited cavalry of the Western era.
In Ralph Nelson's film, the American soldiers are the enemy.
It's almost-- and they commit atrocities that some scholars have associated with the My Lai atrocities in Vietnam.
And so the Western has become a screen on which America's anxieties about the Vietnam War have been projected.
Why would the Western work so well for this?
One answer is what I've been saying all along about the power of genre and the power of repetition.
Look, if a thing is repeated again, and again, and again, it looks familiar.
When it's so familiar, what happens?
It licenses something disturbing.
Because the genre seems on the surface to contain so many familiar, reassuring elements, those very elements of reassurance enable the exploration of disturbing, or uncertain, or problematic materials.
And that's one of the-- and, of course, because any one of these films would have nothing like the power they actually have if they existed individually.
But it's because they're part of this long conversation that goes back to the earliest days of movies that they have the power that they have.
In 1971, the film starring Dustin Hoffman, Little Big Man, is another such film in which it's the Indians-- the Native Americans-- who are the heroes.
And the little, big man of the title is played by Dustin Hoffman.
He's a Native American who has survived Indian massacres and lives to tell the story.
And, of course, in 1971-- the film you'll see next week that will embody these principles-- Robert Altman's McCabe and Mrs. Miller.
The Western doesn't disappear.
It doesn't completely die, but it falls away as time continues in part, I think, because our increasingly urbanized society after the 1950s became less and less amenable.
The Western became less and less credible or valuable.
It became harder and harder for most Americans to identify with aspects of the Western.
That's one reason.
And other genres begin to take over, especially the science fiction genre.
And one of the things you might think about if you think about the rise of science fiction from the '50s and '60s, and then especially from the '70s and beyond, and especially the rise of the science fiction film is you can begin to see a hybridization going on in which many of the features of the Western are superimposed on a science fiction format.
And, man, I'm sure you can think of many such examples of science fiction films they borrow from and utilize Western conventions.
So that's one feature-- that certain kinds of hybridization in which some of the features of a Western begin to appear in other genres.
But, of course, the Western-- because of the resonance of the form, it continues to be a space in which gifted directors are able to make statements about the nature of American society.
And exactly, again, because of the familiarity of it and because of the long history of the themes and discourses we associate with the Western, every new Western is always implicitly in a conversation with its ancestors.
And we have in the 1990s a series of interesting films.
Some of you may have seen them-- Clint Eastwood's Unforgiven, Kevin Costner's Dances with Wolves, two more versions of the Wyatt Earp story-- Tombstone and Wyatt Earp.
Just a couple of titles from more recent times-- in 2007, 3:10 to Yuma-- a remake of an earlier film that I listed earlier-- a 1957 film.
If you compare those two films, you can see how much more violent, how much more anarchic, how much less idealizing and romanticizing the Western has become.
The Assassination of Jesse James by the Coward Robert Ford-- a wonderful title which replicates exactly the title of one of those dime novels appeared in 2007.
And let me mention two astonishingly rich and interesting examples from television.
Television by the '80s and '90s is already a rival to the movies-- has really replaced the movies in many ways in terms of complexity and richness as well as having replaced the movies somewhat earlier as our consensus form-- as our central form.
But two films, especially-- one, the well-known miniseries Lonesome Dove made from the Larry McMurtry novel-- one of the best television-- one of the best Westerns ever made-- a wonderful multipart story utilizing the serial nature of television to tell a Western with an expansiveness that the movie screen can allow but that the conditions of exhibition in the movies do not allow.
You can't go back to the movie.
You can't watch a movie for six hours or eight hours.
And it's much less convenient to go once a week for two hours for seven or eight weeks to see the same story.
Television's a perfect environment for that in that sense.
The reduced visual scale of television is what keeps the television from being really as powerful a medium for the Western as the movies are.
But still, great Westerns are made.
And the greatest of all television Westerns-- one of the great Westerns-- one of the greatest Westerns ever made-- is the ongoing series-- David Milch's great series, Deadwood, that appeared on HBO from 2004 to 2006 was abruptly cut-off by the HBO bosses.
In my view, it's the equivalent in the history of television to the story of what happened to the Last Laugh, in which the producers damaged a work of art.
But even in its damaged form, Deadwood is one of the most remarkable Westerns ever made.
And it is deeply, deeply, deeply anarchic, dissenting, skeptical about human values and about motives.
It's the opposite of an idealizing Western.
It sees the West as emerging-- and Western society as emerging from human greed, from entrepreneuring violence and viciousness, from fear.
From a historical and intellectual standpoint, it's a far more persuasive understanding of how history evolves, I think, than earlier idealizing forms-- a deeply disturbing and powerful text.
So the Western remains even in its vestigial condition-- even though it's now faded.
It's no longer as central a format for an urbanized and technologized society.
Remember, the United States before the Second World War was mostly a rural society.
We only became a suburban and a fully urban society after the Second World War.
And the growth of the suburbs is the great event, of course, after the Second World War-- so that more and more of the population found rural spaces alien, and didn't grow up in big spaces, and couldn't identify in the same way with the Western story.
And the conditions of experience-- the conditions of life in the United States-- underwent a transformation that made the Western much more of a traditional-- a vestigial voice-- a vestigial form than a central one.
And we can trace the progress of that-- of the Western genre's movement-- from the center to the more marginal or peripheral parts of the society in this history that I've briefly shown you.
I have a few things to say about John Ford.
Can you put him up?
I won't spend a lot of time on this, but I want to mention some things about him.
This is a list of some of his significant films.
Not all that I've listed here are Westerns, but you can see a number of them are.
He begins in the silent era.
He continues robustly through the sound era-- one of the great directors of the sound era.
And you can see a film like Young Mr. Lincoln is not quite a Western.
Drums Along the Mohawk-- half a Western-- but he made other films that were not Westerns.
But the Western was his signature form.
He was a very unpretentious man.
And when he was introduced to people he often said, hello, I'm John Ford.
I direct Westerns.
I make Westerns.
He didn't say, I'm a film director.
He didn't say, I'm a cineaste.
He didn't say, I'm an artist in the movies.
He said, I make Westerns.
And I like the unpretentious of that quality in him.
He was the 13th child of Irish immigrants-- born in Maine.
His father was a saloon owner.
And at the age of 18, in 1913 after high school, he followed his brother Jack to Hollywood where his brother was a writer and a director.
He began to work in his brothers films as a prop man, and a stunt man, and a bit actor.
He played a Ku Klux Klansman in The Birth of a Nation.
So he's a man who's the first-- why is Birth of a Nation important?
The first feature-length film in the United States-- D.W. Griffith.
And it shows Ford's connection to the history-- centrality to the history of the American film.
And he directed his first film in 1917 which was a bank robbery Western.
And in that year he directed his first feature-- the film I listed earlier on our little history of the Western, Straight Shooting.
And then a series of films through his career-- 30 silent films by 1921 nearly all of which have been lost.
So he was a very prolific director in the silent era.
And, finally, at the end of his career he had directed more than 100 films-- over 60 sound features, 14 Westerns of which at least five to seven are among the greatest Westerns ever made.
I won't single out any particular films here except to indicate that by the time we get to the late '50s-- to The Searchers, The Man Who Shot Liberty Valance, Cheyenne Autumn, Seven Women-- a new kind of skepticism has entered into the idealizing and heroic tendencies that were characteristic of Ford's work in his earlier Westerns.
And one of the reasons that the film you're going to see tonight is so central is that it may be Ford's most profoundly ambivalent Western-- the Western in which his impulse to celebrate the heroic qualities of these pioneers and the resilience and survival qualities that were required to survive in these environments are very central.
But he's begun to recognize-- begun to acknowledge-- what his earlier films had not acknowledged-- the moral complexities of the idea that the whites should come and take land away from the Indians.
He's become to realize that the settlement of the West is not an unambiguous, triumphal story but a more complex and morally ambiguous tale.
And that moral ambiguity is at the heart of the film you're going to see tonight-- The Searchers.
It has a damaged hero as you'll recognize.
It's probably John Wayne's greatest role.
John Wayne-- he plays a character named Ethan in the film.
And it's interesting-- maybe Wayne didn't fully understand how ambivalently the central character is treated because he named his firstborn son Ethan.
It's a strange thing because the Ethan that John Wayne plays here is not really an attractive person.
He has qualities that we admire and that John Ford surely admires.
He's tremendously good with a gun.
He's deeply knowledgeable about how to survive in the West.
He knows a tremendous amount about the Native Americans.
He can ride a horse like a rodeo rider.
He's brave beyond belief, but he's also, as we discover, a racist.
There's a danger all the way through the film that he may murder the young girl he's trying to find.
He's searching for a young girl.
There's some question all the way through the film on what his motive is-- whether he wants to find her to kill her or whether he wants to find her to rescue her.
Why does he want to kill her?
Part of his damage-- the implication of the early part of the film is that he-- not just the implication-- the fact is he-- in the early part of the film, we discover that he fought on the Confederate side of the war, so he's coming home a loser.
And there's an implication very early, as you'll see, that he committed various kinds of crime-- that he rode with outlaws.
And that some of the loot that he has taken from the war may be ill-gained-- that he's a bitter and unhappy war veteran who feels that his side lost.
So he enters the film bitter-- as an embittered and damaged person.
As the film continues, we recognize that, in fact, this family he's returned to-- his brother's family and his brother's wife and his nieces and nephews-- there's a complexity there because of the looks that he exchanges with his brother's wife.
You realize, not that they've ever had an affair or anything, but they had loved each other first and that the wife maybe still loves the John Wayne character even though she'll never act on it.
Watch how that happens because not a word of dialogue is spoken, but Ford dramatizes this almost heavy-handedly.
It's not all ambiguous even though the character playing John Wayne's brother is completely oblivious to this, but the audience knows this.
So he's an embittered, damaged-- a partial figure even from the beginning.
And then as the story goes on, we discover other qualities in him that make us nervous.
I said, for example, that he may be a racist.
And we don't know whether he'll kill or save the person he's after.
The film involves an Indian attack in which the John Wayne character's niece is taken by Indians.
And most of the film is the John Wayne character and his sidekick's attempt to recover the stolen woman-- his niece.
She's stolen as a child, but it takes many years before they recover her.
And by the time they recover her, she's a young woman.
And she's been the wife of an Indian brave-- a character named Scar who was really, ultimately, a kind of double of the hero in certain ways as we come to recognize when we look closely at the film.
And his search for this girl is ambiguous to us because he's so against-- he hates Indian so much.
It's a form of-- remember when the film is taking place-- during the civil rights era.
So we can read John Wayne's hatred of Indians as a form of the racism-- the anti-African American prejudice-- that is being dramatized-- that is being fought over in the civil rights movement in the United States in the '50s and 1960s.
So it's a kind of metaphor for what's happening there.
And we're not sure whether the John Wayne character is actually going to save her or kill her when he finds her because she's been polluted by her connection to an Indian.
And this theme-- it's not hidden because the John Wayne character's sidekick is really worried about this and even accuses him of this at some point.
So he's a damaged and ambiguous character.
There are things about him we admire, but there are things about him that disturb us.
And at the very end of the film in what is one of the most famous images in John Ford's work, we get a shot of John Wayne from the back.
He's been excluded from the community in a way.
We see him from the back.
He's standing like this.
We see him through an aperture through a doorway.
And he grabs his-- he makes this kind of a gesture-- a gesture of vulnerability in which we-- and he's standing isolated and alone.
There's a sense in which he's still-- this extraordinary hero has been, in some sense, excluded from the very community that he helped create, in part, because he doesn't deserve to be part of it.
He's damaged.
He's imperfect.
But also because of the natural-- the mythic logic of the Western in which the savior figure is so tainted by the savagery he saved us from that he can't be included in the community that he helped to create.
So it's a very clear example of this Western pattern, but it's morally complicated because it comes at such a late stage and because, by this time, Ford himself has begun to reflect on the implications of the much more simplistic stories he had told earlier in his career.
The setting of the film is important to talk about because it is filmed in a place that came to be called John Wayne's own special theater-- his own stage set-- Monument Valley in Arizona.
Other directors have worked in that place, but it always feels as if they're plagiarizing John Ford.
He made at least nine films in Monument Valley.
And you'll see the power of that space-- the looming rock formations-- the sense you have all the way through the film that the human habitations are minimal structures that could be blown away in a second and that are dwarfed in comparison to the rock formations and the grandeur of the gigantic natural phenomena that we see around us.
The human being seems small, and the human habitations seem marginal and far from permanent in this world of grandeur-- of what we might call inhuman grandeur.
So the setting's important.
The plot-- very important to realize that it embodies two separate, important kinds of stories.
The first-- the founding story-- what I've talked about earlier today.
The idea that this is how a culture is founded.
You create a society against warring elements, rescue it from the wilderness, and from savage-- not just savage nature but from savage antagonists-- the American Indians who don't want to give up their land and who are-- until we get to this film-- in John Ford's films mostly-- had been figured as anarchic figures of pure evil.
They hadn't been humanized.
They've been seen as part of nature.
In this film, not so.
There's a sympathy for the Indians that enters this film that helps to explain its complexity and its richness.
The second great story is the captive's tale-- one of the central and earliest forms of American narrative.
The captives tale that is told here is based on a true story.
A woman named Cynthia Ann Parker was abducted at the age of nine by Comanches in Texas in 1836.
A novel was written about that event by Alan Le May much after the event.
And Ford's film is based on the novel.
One of the ironies is that the real Cynthia Ann chose her Comanche husband over her Anglo relatives when she was rescued.
She didn't want to leave unlike the novel and unlike what happens in the film.
And I mention this, in part, to remind you of the romanticizing tendencies of the American Western.
But still, captives' tales-- stories of white people, especially white women, captured by Indians and raised by Indians were a staple of the popular cultures from long before the history of movies.
And this movie, like other movies, dramatizes a version of that captive's tale.
The structure of the film is worth calling attention to, in part, because it's ambiguous.
One way to organize the plot-- I mean, the structure of the film-- is to recognize that there are seven moments in the film that we might think of as chapters or bookmarks.
And if you watch for these moments-- it's also a structurally significant device.
If you watch for these moments, you'll be able to follow the-- you'll recognize that the film is progressing because its timeframe is intentionally ambiguous for reasons I'll hint at in a moment.
And these seven moments all involve a similar kind of shot.
The camera looks through a doorway or an opening out into a rectangle of receding light.
The very opening of the film has one such shot.
And at seven different moments in the film, we see a shot from inside an interior space-- from a dark interior space-- and we look out through a doorway.
You might ask yourself why in a film that celebrates and that dramatizes the immense expansiveness of the Western environment, Ford would give us images-- such self-conscious obvious images-- that, first of all, refer to each other so you think about when they occur in the film.
But why he would so intentionally restrict the visual range of what's available-- make you feel confinement.
Why does he do that?
One possible-- and I might leave this for your section but think about why he does it.
One effect is surely to make you even more aware of how expansive the outside is because you are in the outside for a lot of the time.
By confining your vision in this way, it's a reminder of how much bigness is out there.
But there are other reasons, as well, for why the perspective would be restricted to these aperture shots.
In any case, there are seven of them through the film, and they mark the progress of the film.
The reason that the film is so hard to follow is, in part, that the timeframe is weird.
And I'll come to that in a second.
There is one turning point in the film that I want to mention to you that you should watch for.
It's a moral turning point or a psychologically important turning point because it's a moment in the film where the sympathy of the viewer is shifted-- is radically transformed.
It occurs about midway through the film where the John Wayne character and his sidekick, Martin Pawley, come riding down a-- not quite a mountain but a promontory through heavy snow.
The weather in the film is interesting and powerful.
That's one of the ways you can feel time passing.
Their winters and summers go by.
You can feel the seasons passing.
But you're not clear how much time goes by.
By the end of the film, you know many years has gone by-- at least 10 years-- because the young girl who was stolen away in the beginning is a young woman at the end of the film-- the adult played by the actress Natalie Wood-- a very successful and important actress of the '50s and '60s.
So she's a grown woman by the end, so we know that a significant amount of time has gone by.
But it isn't clear as the story's going on exactly how much time has gone by, and that ambiguity's important.
The turning point occurs as they're coming down this hill.
The horses are almost up to their shoulders in snow.
And they come riding down this hill into a village-- an Indian village-- a Native American village.
And it's a Native American village that's in smoking ruins.
It has been decimated by American cavalry.
And they go into one of the teepees, and they find-- dead in the teepee-- a Native American woman-- a squaw named Look who had been following them.
And we've gotten to know her.
And the Martin Pawley character-- the secondary character-- the sidekick-- says to the John Wayne character, why did they have to kill Look?
She didn't do anything.
And it's a moment in which the butchery of the white soldiers is dramatized in which your sympathy turns entirely away from the official American government account of the West towards something else.
And in the very next scene, something amazing happens.
It's easy to miss because it happens so quickly, but it's very important, morally, to the film.
We have a scene in which we see three Indian chieftains standing in-- they're captives-- standing in blankets looking on as another group of Native Americans-- Indians-- are being, essentially, herded into a compound-- into a hospital-type building.
And they're being herded as if they're cattle, or sheep, or not fully human.
And you can see-- there's no dialogue, but you can see the sadness and the unhappiness and the regret in the eyes of these sturdy chiefs as they look upon that.
And what happens in that moment is that the film's perspective has become that of the Native Americans.
And the audience's perspective has become that as well.
And so the effect of these moments coupled with many other things in the film-- some of which I've mentioned, some of which I've not-- about the ambiguity of the John Wayne character's nature, come together here.
And this is a moment in which something of the complexity of the Western story is being acknowledged by the text.
No longer are we getting a kind of simple celebration of these Western values.
Something much more ambiguous and morally complex is happening here.
A critique of those values is being embedded in this apparently classic Western.
And it's one of the reasons it's such a remarkable film.
I'll conclude now.
I apologize for running over.
The title itself clarifies some of the ambiguity of the movie.
What are they searching for?
I said before that the timeframe of the film is confusing, and it's not just that.
Even the setting of the film-- the Monument Valley setting-- comes to seem almost like a lunar landscape before we're finished with the film.
There's a sense that the quest goes on forever-- that they begin to lose a sense of why they're doing it.
After all, so much time has gone by.
She's lived with the Native Americans all this time.
What's the point of the rescue even-- we might think.
The fact that the quest has gone on for such a long time, but it becomes more and more unclear why the search is going on-- what the purpose of the search is.
There's a kind of ambiguity in it that is deeply disturbing and deeply troubling.
And, of course, the sense we have at the end of the film that the search-- it is concluded, but almost until the very end we're not sure whether the John Wayne character is going to save or murder the object of his quest.
And there's also the sense that even after he's done, what has been gained?
So there's a deep ambiguity at the end of the movie as there is a deep ambiguity surrounding the alleged heroism of the John Wayne character.
So one of the reasons I emphasize these ambiguities and complexities is as a way of stressing the fact that even before we get to the era of what I'll call the anti-Western or the dissenting Western, the classic Western itself has reached a level of complexity and maturity, embodied especially in this film and others that we might cite from the 1960s, in which a kind of self-consciousness about the assumptions that lie behind and animate this central American form have already come to the surface and are being confronted by our best directors in our best films.
Goodnight.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good afternoon, people.
We begin this afternoon our-- if I'm right about this, I think I am, as the syllabus shows-- that this is the last American film we'll be looking at in the term.
So we're sort of finishing of the Central American phase of our course with tonight's film, Robert Altman's McCabe & Mrs. Miller.
And I want to use this afternoon's lecture to contextualize that moment of the early '70s-- the late '60s and early '70s-- in American film history.
And to try to draw out some of the implications of the astonishing transformation that overtook American film in this period.
We've been talking about this topic in different ways all semester.
So by now I think the narrative that I've been implicitly following in the course should already be overt for all of you.
The film we're going to see tonight, Robert Altman's McCabe & Mrs. Miller, can be understood as are particularly clear example of the kind of transformation and even subversion that becomes a dominant note of American movie making in the 1970s.
And part of the lecture today will be devoted to speculating about why that transformation, why those forms of subversion, occurred.
And of course, if you've been paying attention in the course, you know the answer already and it's the answer that's already embedded in the lecture outline.
A new form of what I've been calling consensus narrative emerges really decisively by the middle and end of the 1960s and into the 1970s.
It's a period that takes time but, essentially, television supplants the movies as the central story system of the society.
Not that movies become trivial or unimportant, but they begin to occupy a new niche in the media ecology of the country.
And I'll return to this matter briefly in a few moments.
But let me begin by clarifying something about the importance of the presence in our syllabus of McCabe & Mrs. Miller and more generally, the prominence we've given in our course to the genre of the Western film.
So I ask you to remember the sorts of things we were saying about genre forms generally.
About the aptness-- even one one might think of as the economic necessity genre forms for a system that is based on the principle of mass production.
And I also want to suggest to you that the particular forms of genre story, the particular kind of story that is told in a particular society and certainly the ones that are told in the generic forms that are chosen by American society, are going to very powerfully reflect aspects of the culture.
And we made a special emphasis on the importance in our previous classes of two particular genres that emerge with great decisiveness in the sound era.
Both identified by some scholars as America's most distinctive contribution to world cinema.
One form was the screwball comedy and the elaborations on comedy that developed through the '30s and 1940s.
And the second form, the second distinctive American contribution to film-- has some scholars have sais-- is the Western.
And we've talked a bit about the nature of the Western and how the Western as a form is rooted in the assumptions and preoccupations of American culture.
How it grows out of an historical actuality-- the period roughly from 1860 to 1890-- when the Civil War occurred, when the range wars occurred, when the when the great gold discoveries were made at the California Gold Rush, the laying of the transcontinental railroad.
All of these historical realities that become mythic elements in the founding story that becomes the epic account of American identity.
The epic account of America's founding.
And I've suggested that the creation of that mythology, expressed so powerfully and complexly often-- and disturbingly too because there's a racism, as we've said and imperialism implicit in certain aspects of the Western mythology-- that fundamental energy of the genre, as we've suggested, is not only deeply rooted in American culture, and grows out of the particular individualistic preoccupations of American culture as we suggested last time.
But also, then becomes a particularly dramatic instance of what we might say the movies as a whole have been in American society.
Or were in American society, until the advent of television.
And that is, each genre, and then the Western genre especially, can be understood as kind of a small inversion of the larger processes that operates in this consensus system.
And if you recall the kinds of things we were suggesting last week about the evolution of the Western, what we were confronting in a certain sense was the way in which this astonishingly protean shift-changing mythology can be altered depending on the preoccupations of the society that wants to mobilize the mythology.
Another way of putting this is to say, the mythology, like the larger culture, is never fixed.
It's always in process, right.
So that when one Western appears, a kind of conversation is going on with all previous Westerns.
And maybe with other forms of discourse as well, about the central preoccupations that are part of the Western.
I just want to remind you of the sort of historical scheme that I sketched in last time about the Western, so you can recognize the point at which McCabe & Mrs. Miller sort of fits our picture.
Remember that I suggested in some degree that when the Western as a form emerges after the '30s, when it's mainly are sort of novelty item, which are singing cowboys in it and that sort of thing-- When the classic Western emerges in 1939 with John Ford's film-- what's the name of the film-- Stagecoach, the first film that Ford filmed in Monument Valley-- when the age of the so-called classical age of the Western begins in the 1940s, I suggested in that lecture that the Western's of that period had a kind of imperial dimension.
That those were the Westerns that most decisively and fully celebrated the white settlers idea of the manifest destiny of the American experiment.
Which was translated of course into exterminate the Indians.
The Indians are sort of in the way.
The Westward expansion of the United States was, in many ways, clearly an imperial and even a racist project.
But in the period of the 1940s, the disturbing elements of the mythology were suppressed or ignored, even though they were implicit.
Why would that have been.
Well one reason was America itself was at war in the 1940s.
And the Western was a particularly appropriate.
It was not the only vehicle, but it was one of the vehicles in which imperial ideas of American destiny and American identity would be articulated or dramatized.
And I suggested in the period of the 1950s, this Western form begin to complicate itself, that it became less imperial.
And that there are many Western films of the 1950s that actually reflect many of the domestic concerns of that era.
The generation gap, the discontent of young people growing up into a society that seem to have no work for them-- this is the early 1950s.
And as the decade goes on and leads into the 1960s, an increasing awareness of racial turmoil in the United States-- so Westerns begin to be partly about juvenile delinquency, or about social problems.
Always filtered, of course, through the historical lens of the Western.
And I suggested to you that the great John Ford classic Western that we saw last week, The Searchers, had already in the late '50s began to incorporate certain kinds of subversive questions.
So the '50s is a time when the Western begins to show social problems.
There's a kind of inward turning, in which the psychological problems of characters become important in a new way.
And then in the '60s, this questioning of the genre proceeds and expands even more fully.
And as we suggested, there are a series of films of Westerns in the '60s that, even more substantially than The Searchers, begin to sort of raise doubts about some of the most fundamental questions of the Western.
And of course, as the decade wears on, there are increasing encouragements or incitements from the larger culture-- from the social history of the larger culture-- to make the Western genre begin to question itself.
And what are those things that begin to happen as the '60s wears on.
They were embedded in the '50s but they become much more serious.
The first is racial turmoil, right.
The Civil Rights Movement becomes more and more militant through the 1960s.
And there is real conflict in the society emerging very powerfully.
And that racial turmoil becomes more and more militant, more and more polarizing as the decade goes on.
But what's the other great terrible fact of the 1960s, that explains why American society was so divided.
And so uncertain of itself.
The Vietnam War of course, right.
By the end of the decade, America's deeply immersed in Vietnam.
The college-aged generation is alienated from its elders.
There are demonstrations on campuses in 1967, '68, right.
And then in 1968, two really horrific assassinations occurred to help mark the 1960s as a decisively disturbed time in American history.
Who are the two great figures who die and were assassinated in 1968.
One is Martin Luther King, the great hero of a peaceful, Ghandi-like Civil Rights Movement is assassinated.
And that gives it even stronger impetus to the more militant arguments for civil rights that unlike Doctor King's position, did not so it really embrace nonviolence.
So the combination of the Vietnam War and the civil rights movement combining together created a tremendous turmoil in American society.
And there are many film historians who would say that the totally subversive genre films, the anti-genre films that emerge in the 1970s-- of which the most decisive example in our course is McCabe & Mrs. Miller, so it completes our argument.
There are many who would say that a sufficient explanation for the subversion of the genre, for the fact that the Westerns of the '70s-- that some of the Westerns of the '70s and beyond-- in some sense, could be identified as anti Western.
So hostile are they-- so hostile are they to the basic assumptions of their ancestors, so much that they turn the heroic values of their ancestors on their head.
And you'll and you'll see a wonderfully decisive example of this in tonight's film.
There's some social and film historians who would say, well, the difficulties in the outer society-- the war, the Civil Rights Movement, the polarization of the culture is sufficient to explain why these genres, and especially this centrally American genre the Western, would suddenly sort of turn on itself and attack what it had previously celebrated.
The notions of individualistic achievement.
Notions of violent masculinity.
One of the best books on the Old West and on the Western film, has the wonderful title Regeneration through Violence.
Regeneration through Violence.
And it does describe what the classic Western seems to believe.
It's actually a scary idea.
You might think about how often this notion has been translated into sort of foreign policy behavior by the United States.
Again, reinforcing the importance of these mythologies.
But the social history of the country is not really sufficient fully, to explain why the movies would become so subversive.
And what's the answer.
It's implied up here in number three, right.
It has to do with the new consensus medium.
So let me try to make these arguments in a slightly more systematic way.
And first, by talking more concretely about the ways in which the movies were different in the 1970s.
Wherever we look at central elements of the movies, we can see a profound change in atmosphere and tone and feel.
The first, if we think about the sort of characteristic actors or acting styles of the period, and especially the characteristic stars and contrast them to the great stars of the studio era, we can get a feel for how much more subversive, problematic, ironic, anti-heroic, this new period in American film is.
But there are many, many people who would actually say that this moment in American film is actually one of the-- and I think I might say this-- is a moment of great artistic achievement.
Whatever we would say about the early '70s and the whole decade of the '70s, we could call it a moment or a period of profound transition.
A great artistic achievement.
But also, an achievement gotten at a profound cost.
A fundamental, irreversible shift in the relation of American movies to their audience, OK.
So that's the basic idea.
But what did this shift look like, what did it feel like.
Well, I've suggested that it's grungier, more ironic, deeply anti-heroic.
And one way I can capture this is by taking a scene from one of the characteristic, really defining actors of the 1970s, of what we might call the post-consensus era, the post-Hollywood era, Jack Nicholson.
I want to show you what is, in some sense, his signature scene.
And people who haven't seen Five Easy Pieces, where this fragment comes from, know about this scene.
What I want to propose to you as you look at this scene is think about Jack Nicholson's behavior in the scene.
And ask yourself, is it possible to imagine one of the classic, much more dignified heroes-- male protagonists== movie stars-- from an earlier era behaving in the same way.
Could we imagine John Wayne behaving this way.
Or could we imagine Jimmy Stewart behaving this way or Henry Fonda behaving.
And of course the answer-- I'm sure as you'll recognize-- is no.
Here's the scene then, from Five Easy Pieces.
Some people would call it Jack Nicholson's signature moment.
[VIDEO PLAYBACK] -I'd like a, uh, plain omelette, no potatoes, tomatoes instead, cup of coffee, and wheat toast.
-No substitutions.
-What do you mean?
You don't have any tomatoes?
-Only what's on the menu.
You can have a number two-- a plain omelette.
It comes with cottage fries and rolls.
-Yeah I know what it comes with.
But it's not what I want.
-Well I'll come back when you make up your mind.
-Wait a minute.
I have made up my mind.
I'd like a plain omelet, no potatoes on the plate, a cup of coffee, and a side order of wheat toast.
-I'm sorry we don't have any side orders of toast.
I'll give you an English muffin or a coffee roll.
-What do you mean you don't make side orders of toast?
You make sandwiches, don't you?
-Would you like to talk to the manager?
-Hey Mac-- -Shut up.
You've got bread and toaster of some kind?
-I don't make the rules.
-OK I'll make it as easy for you as I can.
I'd like an omelet, plain, and a chicken salad sandwich on wheat toast.
No mayonnaise, no butter, no lettuce.
And a cup of coffee.
-A number two, chicken sal san, hold the butter, the lettuce, the mayonnaise.
And a cup of coffee.
Anything else?
-Yeah, now all you have to do is hold the chicken, bring me the toast, give me a check for the chicken salad sandwich, and you haven't broken any rules.
-You want me to hold the chicken, huh?
-I want you to hold it between your knees.
-You see that sign, sir?
Yes you'll all have to leave.
I'm not taking any more of your smartness and sarcasm?
-Do you see this sign?
-[INAUDIBLE] [END PLAYBACK]
Now, you've seen Nicholson in far more frenetic moments in, let's say the Batman movie, right.
But the essence of the Nicholson character, he's actually calmer and nicer there than he normally is.
His persona has not been fully established at this early stage in his career.
But you can feel the energy there.
And the hostility to-- I think we're supposed to be on Nicholson's side in that film-- but when he knocks that stuff off the table.
He's doing something that we would never imagine a classic Hollywood hero to have done.
And I don't want to put too great of an emphasis on a single scene, but if you think about Nicholson's incarnations after that movie, you'll understand how decisively and fully a new kind of actor has emerged in the 1970s.
And you might talk about some of the other actors-- you can do the other sheet there, Greg-- that emerge in this period.
Warren Beatty.
Elliot Gould.
Dustin Hoffman.
Robert De Niro.
Fewer female stars emerge in this period.
But they're important.
some of them did begin their careers around this time.
And even the actresses standing in the same sort of ironizing and anti-heroic relation to their ancestors as the male actors.
We might think about Julie Christie, who is the co-star of tonight's film in McCabe & Mrs. Miller.
Or Jane Fonda.
Or the actress Faye Dunaway, who does such a brilliant job in the characteristic early '70s film, Chinatown.
And each of these performers, has, as I suggested, a sort of grungy, ironized, non-heroic relationship to the iconic figures of the Hollywood past.
And something of the same thing is true about the kinds of directors and the kinds of films that emerged in this period, as well.
It's the period in which some directors who then became dominant figures over the next 30 years, Martin Scorsese, Roman Polanski, Alan J. Pakula, Francis Ford Coppola, Sam Peckinpah, Altman himself, Stanley Kubrick.
All of these really distinctive, auteurist directors emerge decisively in this period.
And again, the films that they make have a kind of personal stamp that would not have been possible in the studio era.
And of course, what I've left out of the equation in talking about the transformation of American movies, is the reminder that it was in the period from 1950 through the late '60s or early '70s, that the apparatus of the old Hollywood system began to break down.
Movies continued to be made, but the old studios by the end of the '60s all really dissolved.
The studios no longer held contracts for dozens, if not hundreds, of actors and armies of technicians and so forth.
The mass production system that had been characteristic of Hollywood for most of the 20th century had broken down by the end of the '70s.
And what movie is really had become, essentially, were deals.
You've got a bankable star to agree to a script.
You get the screenwriter to agree to do the work.
You go to the money man and you say, I've got Nicholson and I've got this screenwriter, how about if we go ahead on the movie.
Very different from the days in which the studio bosses would sit down and say, we have to produce x number of films this year.
Let's decide how many Westerns, how many comedies we want to make.
Let's decide how we want to use Barbara Stanwyck this year, that sort of thing.
That game is all over.
So that from the economic circumstances and the physical ways in which movies are made have undergone a transformation, as well.
Every movie becomes a kind of major product.
And even though the studio names survive beyond the 1960s, they're just names.
They're just labels now.
They actually no longer represent what they had in the studio era.
Not only a particular number of films that each studio would pump out each year, but holdings in land and major movie studios with all the supporting material and personnel necessary to run a systematic operation for the mass production of movies.
That game is over really by the time the '70s began.
And if you actually look at the physical appearance of the actors of this period, you could see that they look less grand.
They look less large.
They literally are often not as tall.
Dustin Hoffman's a shrimp, is even shorter than I am.
When he acts with some female stars, they have to put him on a box so it doesn't look like he's too small.
They don't photograph the box, of course.
And there's something, not exactly wimpish, but ironic or outlawish, an unwholesomeness over a nervousness or a sense of explosive anger-- the Nicholson character is a wonderful instance of that-- might emerge.
These qualities, the instability of the idea of the hero is part of what is emerging in these periods.
And I have another clip I want to show you that I think and capture another part of this.
It will also illustrate the new sort of grungy type hero and movie star who emerges in this period.
But it will also tell you something else about the way in which these transformations and subversions were expressed in films, themselves.
And this is a clip from the end of another Robert Altman movie, made roughly around the same time.
It's also a genre film.
And like McCabe & Mrs. Miller, it's a kind of anti-genre film.
It uses the trappings of a traditional genre, but he turns those trappings askew in ways that make the audience nervous, in ways that depart, from not just the standard expectations of the genre, but even the moral assumptions that are made by the genre.
In this case, the moral assumption that is at most decisively undercut, as you'll see, has to do with the moral rectitude of the hero, himself.
And this expresses itself all the way through many of these stories and many of these films, but it often expresses itself with particular clarity in the endings of films.
So watch for the kind of actor that Elliot Gould is, because he's the star, he plays the detective in this sequence.
I want you to look now, at the final, really sort of the climactic sequence-- the dramatic ending-- of an Altman detective film.
A remake of a film by the same title, based on a novel by the same title, going back into the '20s.
It's a film entitled The Long Goodbye.
And it stars Raymond Chandler's most famous private eye figure.
This private eye figure had been played in the past, by a series of really, sort of tough guy, Hollywood leads.
And in Altman's version of The Long Goodbye, I think Robert Mitchum, in fact, a very hulk, bulky kind of tough, studly kind of actor had played the role at least once, previously.
In Altman's version of the film, Elliot Gould plays the roll.
And Eliot Gould is a kind of-- as some of you know if you've seen his acting-- is an actor for the modern age in some ways.
He talks to himself a lot.
He always a kind of anti-heroic demeanor.
So, remember we've said also that the endings of genre forms are often very conventionalized.
Think about the endings of situation comedy, as an example.
Or the way in which, at the end of certain forms of melodrama, you expect certain forms of reassurance.
That things will work out, or have worked out, right.
One of the fundamental functions of the happy ending of most genre forms in the classic age of Hollywood was precisely to return us to that form of reassurance.
All right, so here is the ending of a detective film called The Long Goodbye, starring Elliot Gould.
And perhaps the only thing I need to do to introduce the scene, is to simply say that the detective's quest has led him, finally, at the end of the film, to sort of solve the mystery.
And he's found the man who has committed the horrendous crimes he's been investigating through the film.
And the man turns out to have been a friend of his, someone he has known.
And in fact, someone who had taken advantage of him.
And here's the very end of the film.
[VIDEO PLAYBACK] -How you doing, Terry?
-Marlowe?
I guess if anybody's going to track me down, it'd be you.
Want a drink or something?
-No, I don't want no drink.
-You get a kick out of that medicine I sent ya?
-Yeah, I got a big kick out of it.
So you murdered your wife, huh Terry?
-Well I killed her, but you can't call it murder.
Wade told her about Eileen and me, she started screaming.
She was gonna tell the cops.
She knew I was carrying money for Augustine.
She was gonna turn me in.
I hit her.
I didn't try to kill her, I hit her.
I didn't mean it!
-I saw the photographs, boy.
You bashed your face in.
She didn't give me any choice!
-You didn't have much choice, huh?
So you used me.
-The hell, that's what friends are for, I was in a jam.
Come on, have a drink.
I had a dead wife, $350,000 that doesn't belong to me, I had to get out.
It's as simple as that.
-Simple as that, huh?
-Goddamn simple.
Cops have me legally dead.
Augustine's got his money, he's not looking for me anymore.
I got a girl that loves me.
I got more money than Sylvia and Augustine put together.
What the hell, nobody cares.
-Yeah, nobody cares but me.
-Well, that's you Marlowe.
You'll never learn, you're a born loser.
-Yeah, I even lost my cat.
[END PLAYBACK]
Pretty shocking.
The hero of the movie-- the protagonist-- the guy we identify with all the way through commits cold-blooded murder.
He's a vigilante-- commits cold blooded murder at the end of the film.
And that is the end of the film.
He gets away with it.
So there is a kind of justice done at the end of the film in a kind of narrow way, because the murderer is caught and punished.
And incidentally, this violates what happens in the original novel.
It violates what happens in the first film version of the novel.
But think what happens to genre forms, not only when the hero or the protagonist becomes grungy ironic and anti-heroic in his behavior and demeanor.
And then when the story itself begins to undermine the deepest assumptions about heroism and righteous behavior that seem to justify or undergird these endings.
But in a way, one could say it's a happy ending if one has a very austere notion of justice.
An almost inhuman notion of justice.
But what about this protagonist who does his own vigilante justice.
And when the film first appeared-- it's impossible to exaggerate the effect this shocking and had.
One reason it's so shocking of course, is that the generic expectations run so totally in the opposite direction-- that the most cold blooded murder in the film should be the character with whom you've identified all the way through.
Who is supposed to be solving the crime and bringing evil to justice should turn out to be evil himself in such a shocking and brutal way undermines some of the most up the basic assumptions you make about the kind of safety that the genre of the detective story seems to afford to its viewer.
And this is not an isolated instance, as you'll see in tonight's film.
Altman does almost exactly the same thing with the Western that he does in The Long Goodbye to the detective story.
I'll return tonight to say a little bit more about the particular ways in which McCabe & Mrs. Miller embodies this notion I've been suggesting to you.
Embodies the anti-generic or subversive principles of the kinds of movies that begin to emerge at the end of the '60s and in the 1970s.
Well I've already talked to you about the social history of the '60s and '70s.
And I don't want to repeat myself, but there were some things that I left out in my quick account earlier.
So let's return briefly to remember the sequence of presidents.
Because of course, the sequence of presidents-- Kennedy, who was assassinated in the early '60s, right.
Already, the social fabric in the United States was frayed to the point of breaking by the end of the '60s.
Those of us who lived through it will always remember it as an astonishingly polarized time.
And in the progression from Kennedy to Johnson to Nixon, there is a social history in which that change in the society is of course, implied.
And of course, it concludes in a way, with the terrible event of the Watergate scandal-- the Watergate investigations-- in which for the first time in our history, a sitting president resigned.
And so you can see that the outward social-- and of course, the Watergate scandal raised doubts about the probity and decency of the highest officers of our government.
And while that might not seem so surprising in the age of-- I have to avoid offending some of you-- in our current situation, where it isn't that difficult to doubt what we're told by our government leaders, it's important to recognize that-- not that this was a new experience for Americans, governments always lie-- but there really was a sense of disillusionment with the fundamental structures of the society that was emerging.
Because of the Civil Rights Movement.
Then because of the well because of the war in Vietnam.
And then also, as a kind of icing on the cake in a certain sense, a kind of terrible addition to this weakening the country, but strengthening this sense of polarization and disturbances, if nothing can be trusted.
As if our structures are falling apart or are endangered in some way was the fact the president, in effect resigned, rather than face impeachment.
Because of his behavior in the Watergate business.
So the highest levels of government and the fundamental structures of government were in play.
And it would obviously, in such a time, it's not inappropriate.
You would expect the central story forms of a culture to reflect some disturbance.
But why is it not sufficient simply to urge this social history argument?
To explain what happened to movies?
The simplest answer is, if you look at what was on television at the same time, you can certainly see television becoming more complex.
In fact, the late '60s and early '70s are a transforming moment for television, as well.
Partly because that's the moment when television consciously realized that it has a more central place in the society than the movies did.
It had, for awhile, already achieved this.
But no one on either side of the divide fully recognized it.
But by the late '60s and early '70s, American television now knows itself to be the central medium of consensus narrative.
The society recognizes that the changes occur.
I don't mean that legislation is passed, I mean this understanding slowly dawns over a period of time, right.
And there was a great sense among filmmakers of liberation and freedom.
And they're not wrong about that.
I mean, one might argue that the '70s is the greatest decade of all in the history of American movies.
So many remarkable films were made in that time.
And surely, one of the reasons for it was film was liberated from the necessity to embrace what we're ultimately, although very generous principles, they were still in the end, conservative principles.
About how a story in a consensus medium can be told, right.
But the reason that the social history argument is insufficient for explaining the way movies changed, is that if you look at television you could state a story system that's telling stories that are more complex, and more ironized, and do raise certain kinds of questions about the social and political order that might be thought to be disturbing, and even potentially subversive.
But on television, the center holds.
On television, the endings remain happy, or at least reassuring, mostly.
On television, some of the very same genre forms, especially detective and cop shows, are all over the place.
The fables of law and order.
Fables of injustice.
And the stories that are being told on television do not have the subversive, completely, anti-generic energies that we're being told on television.
Why not?
And again, my answer is the movies have been supplanted by television.
That consensus role is now being played by television.
And one, quite, I think, truly dramatic way in which we can recognize this development is to talk about the differences between the movie version and the television version of another Robert Altman text.
One of his distinctive films of the 1970s, the movie, MASH.
How many of you have seen Altman's MASH?
Only two, three, not very many.
But it's a significant film.
Not as remarkable, I think, in its full shape as the film you're going to see tonight.
But a very, very remarkable film.
How many of you have seem the television show, MASH.
A much larger number.
But still, for an American environment, a very small number.
Especially because the show was on for, in its original run, for something-- I forgot-- like 11 years.
It's one of the longest running shows American history.
And then of course, it was in re-run for more than a decade in virtually every market in the United States.
So there's hardly anyone over the age of probably 15 or 20 in the United States who hasn't seen many episodes of MASH.
So it's somewhat surprising that only a minority of you have seen it.
But we can take the difference between the television version of that story and the movie version as a kind of emblem for this transformation that I've been talking about.
And you don't have to seen either of the films fully to see what I have in mind.
Elliot Gould, the character that you saw just a moment to go in the clip from The Long Goodbye, is one of the central doctors in the movie.
And the other doctor, the partner doctor, characters that are played by Alan Alda and a comrade, who changes over the course of the full run of the show in the series.
Elliot Gould plays that character in the movies and his partner-- also varies in the film-- but his most decisive partner is Donald Sutherland.
And Donald Sutherland is another of these sort of ironic, anti-heroic actors.
They're often actors whose diction is odd.
They mumble and they swallow their words.
They're sometimes literally harder to hear than their elocutionarily superior-- bad adverb-- than their better spoken ancestors in the classic age of Hollywood.
The movie MASH came out in 1970 and it was a kind of disguised Vietnam fable, it was officially based on a novel describing a MASH unit-- a medical unit-- in the Korean War.
And the movie dealt with the Korean War too, ostensibly.
But it was clearly a metaphor for the current war that we were in.
And the doctors, especially the two central characters played by Elliot Gould and Donald Sutherland, are unbelievably subversive figures.
They take their energies more from the counterculture than they do from any other source.
They have no respect for authority at all.
They engage in forms of subversive monkey business, all the way through the Altman version of MASH.
When the movie is translated a year or two later, still while the Vietnam War is on-- onto American television in 1972, I guess-- And the series are oddly quite a lot different.
It does still mobilize certain anti-war sentiments, of course.
But, the television program-- which I think in an artistic sense is probably over the long run of its series better than the movie for all kinds of reasons, partly because the characters evolve and change over time in the series in a way that could not in the movie-- but the most decisive thing about it is that the television series retains some of the qualities that we associate with an anti-genre form.
There are subversive implications in it, for sure.
But the center holds in the television version.
Why?
When I say the center holds, what I mean is that the remains in the film are respect for the larger structures of authority and order in a society, however antic and subversive the doctors might behave in any particular instance.
The literal behavior of the characters is often less grungy and doesn't work with pornography on the television show as it does in the movie.
I don't mean that the movie is pornographic, but it's much more overt about sexual matters than the more than the television series is.
And one of the best ways one can see the contrast is to contrast an actor like Alan Alda with an actor like Elliot Gould.
There's something dark and grungy and self-reflective, self-aware about Elliot Gould that one doesn't feel about Alda.
Alda is much easier in his skin than Elliot Gould.
One feels about Elliot Gould as about many of the actors that emerges-- dominant stars in the '70s and beyond-- is that they're afflicted by a kind of cosmic anxiety.
That they're anxious figures, that they're figures of damage, in some way.
Warren Beatty is such an actor in many ways.
He's the star of tonight's film, McCabe & Mrs. Miller.
There's a particularly wonderful film of the 1970s that I also want to mention to you after I complete my discourse about MASH.
So these two versions of MASH then, sort of dramatize for us, not that television becomes-- especially by 1970-- simply a reactionary medium that just repeats what the ruling class in the United States wants to say.
It's a much more complicated medium than that.
But it is a consensus medium.
It understands that it operates according to certain unwritten rules that the society, as a whole, has accepted.
And one of those rules is that in a consensus medium, you can mobilize all kinds doubts, and dubieties, and ironic perspectives on the larger culture are on the central values of the culture.
But you can't repudiate them completely.
You can't let people come out of a consensus medium thinking that the society is totally corrupt.
That there's no hope for the world.
Consensus systems won't allow that.
They have a comes a kind of aesthetic and political conservatism built into them.
And the movie MASH shows how free it is of those consensus values.
The television show MASH, on the contrary, demonstrates how much pressure a particular genre may receive and still remain, in some sense, a consensus form.
And it seems to me, there are many arguments to be made for the idea that MASH is one of the most distinctive artistic achievements of American television.
That it represents the culmination, in some sense of series traditions that go back almost 30 years by the time MASH appears.
But what gives it its real authority and power is that incorporates into itself a subject matter that, before the mid '60s, anyway had never been on television before.
So it's not that the television that appears in the late '60s early '70s is trivial or foolish, it's just that it is constrained by these consensus principles that no longer constrain the American movie.
And when we look at tonight's film, you'll see very decisive examples of this.
And I'm excited about how clearly McCabe & Mrs. Miller will crystallize the kinds of things I've been saying to you this afternoon.
Now I also want to mention this other example.
A film I meant to mention earlier when I was going through these notions about the way in which we have different kinds of directors, different kinds of actors, different kinds of endings, right.
Which means that the style of the movies is often different, as well.
The visual texture of the film is-- they favor jump cuts and discontinuity tricks much more than the classic Hollywood movie does.
And the film I wanted to tell you about is a film-- I meant to mention earlier and I'll conclude today's lecture with sort of another example, like the examples I've given so far-- in which what we get is it a profoundly subversive film.
Whose presence would have been impossible five years earlier, or 10 years earlier, when the movies were still performing their consensus function in American society.
The film I'm thinking about as a film by the director, Alan J Pakula, some people call him Alan Peculiar, called The Parallax View.
Any of you seen The Parallax View?
It's one of the great films of this early '70s period.
And it's a conspiracy fantasy film.
Warren Beatty plays the central character in the film, a drunk.
And fact that he's a drunkard and radically imperfect person, is also characteristic of the kinds of heroes-- a protagonist-- that we begin to see in the '70s and beyond.
Radically imperfect protagonists begin to become much more the norm in American movies after the Hollywood system has declined.
Well in this film, Warren Beatty plays a recovered alcoholic journalist, who has been fired from a series of important jobs and is now working for a minor newspaper in Seattle, Washington.
His name is Frady.
F-R-A-I-D-Y, I think, or maybe it's F-R-A-D-Y, but it's supposed to echo afraid.
He's like a fraidy cat.
you're supposed to sort of partly feel in the film.
And with the help of a newspaper editor he works for, he stumbles upon what turns out to be a vast political conspiracy.
He's actually present in the beginning of the film, at what looks like a random killing of a political candidate.
Begins to investigate, he discovers in fact, that there was probably conspiracy.
The candidate was assassinated.
And then he goes undercover and he engages in a long investigation.
The shape of the film, and the energies of the film, are very like dozens of films we've seen earlier, especially during the Hollywood era.
In which a lone hero, discovering terrible facts that are dangerous to his community or his society, shows the courage and the fortitude and the end resilience, and the focus, to follow through and solve the mystery and then save the world.
In the most elaborate versions of this fantasy, what's at stake is the fate of the world or the fate of a society.
In smaller versions, it might be something somewhat less.
And that's what it turns out here.
He penetrates into this conspiracy and there are allusions and references that make you-- especially at the end of the film-- associate the conspiracy with the Kennedy assassination.
There is, in the film then, a kind of ongoing process in which the full nature of the conspiracy begins to emerge.
And in fact, it is an unbelievable.
If you actually saw the world in that way you probably couldn't even get a job in the White House.
I'm sorry.
But said it's really an unbelievably conspiratorial film, in which it turns out that there are a sort of network of corporate controllers who are sort of, in effect, taking over the government of the United States.
And they have unbelievable power nobody knows-- but anyway.
This detective has finally uncovered the plot and watch him do it.
And as the film comes to its conclusion, what do we expect will happen?
What's the normal expectation in such a film?
That his revelations will save the day, right.
Well in the very final sequences of this film, we see what looks like a kind of revelation scene, in which our character, Frady, is in a convention hall where they're about to hold the political nominating convention in fact.
It's very resonant because the conspirators are trying to control that space as well.
And essentially, he starts chasing a bad guy, being chased by a bad guy, without going into any detail-- I'll spoil the film for you a little in a way, I must-- guess what happens.
Our hero is killed at the end.
And the film ends with the idea that the conspiracy is enforced.
That there's been no change.
So, it would have been a disturbing film, even if it had ended with the requisite happy ending and a reassuring ending.
And in and how many films does the great hero, the character you've been following all the way through, who's supposed to be one in which you've invested all your sympathy and respect.
In how many films does this guy get blown away at the end.
But it's a mark of how subversive, how morally turbulent, and politically and morally engaged movies had become by the early '70s, that's such an was possible.
So, The Parallax View is another kind of anti-genre in a way.
It's like a spy film-- in some ways it's like one of those Hitchcock romantic thrillers, in which Cary Grant and a beautiful woman travel through beautiful locales and finally, even though they look as if they're in great danger, win the day and are able to marry at the end then also incidentally, have saved the United States government.
The film fits that kind of category, but it explodes all of our expectations about the characters and about the atmosphere of such films.
And in that sense, The Parallax View, like McCabe & Mrs. Miller, like Altman's film MASH, like his film Nashville, like the films of Martin Scorsese that appear-- Mean Streets, his first film and maybe his greatest film appears in this period as well-- have given you a much grungier, more disturbed, more damaged, more hopeless view of urban violence than any film before it had done.
Again, mobilizing the genres of the gangster film and the urban crime film.
But mobilizing it in a way that leaves the viewer unsettled and disturbed.
So the '70s is an astonishing, remarkable moment in the history of American film.
It's remarkable, not only because films became more courageous and more dangerous in the thematic and moral sense, but also because it was an era in which that happened in consequence of the fact that the movies had surrendered their central place in American society to this new medium of television.
I'll talk about some of these matters tonight in a much more concrete way by looking closely at McCabe & Mrs. Miller.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good evening, people.
I want to continue our discussion of the end of the studio era, the advent of the kind of much freer, more independent, more politically and even morally liberated movies that appear after the Hollywood dispensation has subsided.
But remember that I've also been suggesting to you that even though there's a certain sense in which a kind of artistic freedom was now granted to directors and to American films, and on balance I think most scholars and film buffs looking at the 1970s would certainly share my sense that it's a very remarkable decade.
But I think they might also share my sense that the films had given up more than they realized, certainly more than they were conscious of, that the shift in what we might call the media ecology of the society, which enabled this level of freedom for movies, also carried with it a tremendous cost.
And remember how decisive the numbers are.
I've mentioned them a few times in this course.
At the height of the studio era, 2/3 of the country is going to the movies every week.
It's an habitual experience for most Americans, both adults and children, really.
By the time this system comes to an end, by the time the movies have been, as I've suggested several times in the course, supplanted by television as the new central storytelling system of the society, by the time that occurs at the end of the '60s, most Americans are going to the movies only once a year.
And it's only particular subcultures-- teenagers, for example, who make a regular habit of going to the movies.
And a good deal of the triviality and superficiality of movies of many of the box office successes of the '80s and '90s from the American film industry is a direct function of the fact that the audience had dwindled from the whole society to particular subsets of the society.
And the consequences for the nature of-- the movie's sense of its connection to the society are powerful, are profound.
Well, we're going to explore aspects, finally-- our final exploration of aspects of this topic comes in tonight's lecture, and especially in the exemplary early 1970s film McCabe and Mrs. Miller that we're going to screen for you after my lecture.
I want to begin by talking a little bit about Robert Altman's career.
And I'll actually look at certain specific films and say a few things about them in a minute.
But first let me talk about what I take to be some of his-- not only the only ones, but some of his defining qualities as a director-- signature features of his work.
The first and most obvious to me is what we might call his moral skepticism.
He really is in some respects temperamentally an outsider.
And it's interesting to speculate, I think, about what might have happened to Robert Altman, an immensely gifted natural filmmaker, had he been born a generation earlier and had to actually struggle with the requirements and responsibilities, the confinements of the studio system.
But luckily for Altman, he emerges in the time when the tyranny of the studios and also the creative input of the studios has disappeared, has gone away.
And in that sense, he represents a new kind of independent spirit in filmmaking.
And he resembles some of the other directors who come of age at roughly the same time, directors some of whom I mentioned this afternoon, like Scorsese or Coppola or Alan Pakula.
So one of his defining qualities is a kind of moral skepticism, a kind of sympathy, we might call it, which is connected to what we might call his sympathy for the marginal.
Over and over again in Altman's films, we focus on either marginal figures who are reluctantly or inadequately part of a larger community, or we focus on a community that itself is so dubious as a community, so damaged as a community, so enfeebled and unlike the traditional communities that it's as if the community he treats itself, the societies or groupings that he finds interesting, are also in some deep way non-traditional communities.
And in fact, in McCabe and Mrs. Miller, most of the supporting cast were literally members of a commune.
They were members of the counterculture.
The ballad that is sung by the great balladeer, American singer Leonard Cohen in the beginning of the film, would have struck knowledgeable viewers as a deeply subversive gesture.
Because think how often the Western begins with a ballad, with someone saying a ballad, as if the stories of the West have become so mythological that they're part of our song tradition, our musical tradition-- which is true-- as well as our narrative tradition.
And many, many classic Westerns begin with a ballad of some kind.
The Searchers has such a ballad, in fact, as I hope you noticed.
Well, McCabe and Mrs. Miller begins with a ballad, as you'll see.
But it's a subversive ballad.
It's sung by a man, Leonard Cohen, who's deeply associated with marijuana, with the counterculture, with striking out against authority, with writing anti-war songs.
And there's a kind of countercultural ambience, a kind of countercultural skepticism and sympathy for the margins in nearly all of Robert Altman's work.
So his moral skepticism then could be said to have at least two dimensions.
One is what we might call an aesthetic dimension, in which he is skeptical about the inherited strategies, belief systems, and even structures of earlier movies.
And he actually is in that sense aesthetically subversive, although very subtly so, as I hope you'll see in tonight's film.
He'll very often give you a film that seems on the surface to sustain and to recreate all the classic conventions of the genre in which he's working.
And it's only when you look a little bit more closely that you realize that although the structural elements are there, they have been perverted or inverted so that they have in some sense enacted some kind of moral migration and stand for virtually alternative beliefs, assumptions, or values from what they normally are taken to stand for in the classic genre films on which Altman is borrowing, or against which he is engaged in a kind of mockery or at least hostile action of some kind.
And I think that McCabe and Mrs. Miller demonstrates this very dramatically, although most of Altman's films have these qualities in some degree.
The second quality of his moral skepticism is what would be not now related to his aesthetic principles, to his filmmaking practice, but would be more broadly associated with politics, with his general ideological view of the world.
And even there, especially there maybe, Altman styled himself a rebel.
Even when he was successful in Hollywood in his early career when he was so-- MASH was a box office success.
And the decade of 1970 through '80 is probably the richest decade of Altman's career.
And in that time, even though Altman was very successful, later he fell into disfavor with the Hollywood bosses and he spent some period of his career underground in a way, either not making movies at all-- directing plays in various places or making documentary films or private films that were not released commercially.
Then he has a resurgence later, before the end of his life.
The last 10 or 15 years of his life again involve successes at the box office.
So the second aspect of his moral skepticism might be a kind of political skepticism, and a political skepticism both against the Hollywood system and then against the larger political structures of the society.
Now remember, his political subversiveness comes at a time when many honorable and normal people, including your professor, were subversive in some sense, because it was a time when the country was at war in a war that many people in the country thought was unreasonable and wrong.
Many people went to jail or protested the war.
And Altman was on the side of the anti-war forces.
He was on the morally skeptical side about wars of choice in foreign environments.
So he was both politically and culturally a skeptic and aesthetically a skeptic.
And both qualities show up in his films in all kinds of remarkable ways.
I've talked about how that leads to a kind of sympathy for marginality, for the condition of marginality and for characters who belong on the margins of the society, who live on the margins.
And he loved to make films often about certain kinds of human communities, subsets, small versions of human society comprised entirely of misfits and marginal characters.
And he even has a sympathy for what we might think of as unconventional appearance, so that many of the actors and especially the secondary background actors in his films might actually be in ads for grotesquerie.
He loves the grotesque.
And at least his background characters often have a sort of grotesque element.
There's a love of the weird and the strange in Altman that you can feel in his films, in his casting and in his subject matter.
Another thing that I would identify as a kind of defining quality for Robert Altman is the relation in his films, especially his mature films, between plot and character.
There's a tendency-- it's an aspect of his skepticism.
He doesn't like structures that control him.
And from one angle, if you think about storytelling or narrative, plot is a kind of controlling feature, especially if you allow the plot to take the foreground of the-- to assume a position in the foreground of the text, the plot can become a kind of iron hand, controlling the choices the director makes, controlling what can happen to the characters.
And certain forms of action adventure-- what happens next, the sequence of events and the visual excitement of watching those events, is virtually the only interest you have in film.
Those totally plot-oriented films tend to have stereotyped or flat characters in them.
But Altman, even when he embraces certain genre principles that require what looks like a conventional linear plot, often tries to subvert the linearity of his plot.
There's a kind of digressive or dissenting, nonlinear tendency in Altman's work.
You can sense it in McCabe and Mrs. Miller, although it doesn't emerge totally fully until later in his career, a little bit later in his career.
The fullest embodiment of this non-plot principle, of this preference for character over plot, I think is probably the film Nashville in 1975, in which Altman displays an immense range of characters to us and skips from one character to another to another.
There are certain centers of interest.
They seem to have no connection to each other, and you sit there in the film-- when the film first came out, it actually elicited a number of negative reviews from reviewers who were angry about the fact that the film took virtually its whole length to make clear that in fact there was a connection between these characters.
The complaint against the film was it was plotless.
Well, that wasn't exactly true.
It wasn't like plotless.
In fact, all the lines of the plot do come together toward the end of the film.
But the truth is that Altman is at least as interested in digression as he is in linear progression.
And there is a sense in which his films like to pause and savor what's going on around them.
And as you're watching McCabe and Mrs. Miller, one of the things you might ask yourself at a certain level is what's the throughline of the story?
In fact, it feels when you're watching McCabe and Mrs. Miller as if it doesn't have at least a conventional Western plot.
As the film goes on, you begin to realize, yes, the story here is about the growth of this town.
In that sense, the film really is a founding myth, an ironic, mocking, founding myth.
But while you're watching the film, you're interested in McCabe and you're interested in some of the other minor characters in the town.
But it's hard to identify what the actual subject of the film is, at first.
Altman likes to leave his audiences in a little bit of confusion of that sort.
He doesn't like to satisfy the narrow desire to have a clear, linear plot.
He prefers some principles of digression.
And one of the reasons that he does, apart from the fact that he wants to savor the experience of the world and he's attentive to so many aspects of experience that he wants to often bring all of them into the film-- he's very interested in sound, for example, in the way sound operates in the world.
And he does odd things with his soundtrack, as you'll see in McCabe and Mrs. Miller.
I'll come back to this in a minute.
His preoccupation, then, is with trying to create a situation in which character unfolds.
And he'll always sacrifice what a more action-oriented director might think was the most important thing in order to explore character more fully.
Now, one very broad way of dividing narratives, as I may have suggested earlier in the semester to you, is a distinction between plot-driven and character-driven stories.
And one of the things one can see happening in Altman's film-- again, it's another marker of the fact that Altman is operating in a cinematic environment that no longer has the conservative aesthetic constraints that are placed upon a consensus system.
So can explore these kinds of possibilities more fully than-- or can explore them at all, in fact.
In some cases, he couldn't have even looked down these territories, looked down these pathways if he'd been a conventional studio director.
So a distinctive feature, then, of the way he works is this tension between plot and character and the emphasis that Altman will almost always put on character, the way he'll even subvert your interest in what is going to happen next some time in the film.
And as you're watching McCabe you might sort of think about that.
You don't ever feel that the film is flagging.
You always are interested in what happens.
But after a while, you suddenly realize that you're not being carried along by the momentum of story itself in the way you are in a normal film, or what you would think of as a conventional film.
And McCabe is only a minor version of this.
As I say, Nashville is a much more dramatic instance of this.
Finally, another distinctive-- not unique, but distinctive feature of Altman's work-- and this occurs, especially powerfully as his career continues.
You can sense it earlier, but it actually becomes a dominant topic later in his career after he's very successful.
He has great confidence in himself.
By the last decade of his life, he is conceived of, I think, as a master movie maker.
And he finds it very easy to get people to appear in his films.
And that's where this principle comes in most fully.
He's interested in blurring the distinction between fiction and reality.
And in fact, in some of his films, I think Nashville is the first one in which this happens.
And then he returns to this trick or this strategy much later in his career in a whole series of films.
In Nashville there are some performers who are playing themselves.
They're not fictional characters.
Some of them are famous singers and so forth.
And they appear as themselves.
Nashville is about the American music scene, and also about the turmoil of American society in the 1970s.
And so many of the people who appear in the film are playing musicians, singers, songwriters, and performers.
But there are actually some real performers who are recognized by the audience because they're relatively famous, who play themselves.
This blurring of the distinction of the difference between fiction and reality, this mingling of fictional with real characters becomes a very distinctive feature of Altman's work later in his career.
And very quickly then, what I'd like to do is show you a list of some of his films.
You can put that up now, Greg.
I want to say something about all of them.
This is not a full list of his films.
And you can see, there's a long interregnum of almost a decade where he makes some films.
I just haven't bothered to list them.
This is not a complete listing of his films, although I think it is a complete listing of his films made in the 1970s.
And I did that partly because I wanted you to see what a prolific and productive decade this was for Altman, but also because I thought you might be interested in seeing how many titles register with you.
Even if you're not seen the films, you've probably have heard of some of these films, because they are among the most innovative and important American films of the second half of the 20th century.
I want to say a couple of things about some of these films.
You've already seen the ending of The Long Goodbye and you know what a deeply subversive version of the detective genre that film is.
I want to say a word about California Split, because in some ways it's a film that most fully embodies this principle of being interested in the marginal.
California Split is a gambler's movie.
And Eliot Gould, that sort of shuckin' and jivin' figure of anti-heroic irony that you saw at the end of The Long Goodbye is the star of California Split as well.
And almost the entire film-- a large part of the film actually takes place at the race track and in gambling environments.
The climax of the film is a trip to Las Vegas, where our hero has a tremendous run of luck, nearly breaks the bank.
And California Split follows the fortunes of these two gamblers.
It's hard even fully to follow the chronology of the film.
Sometimes you lose sense of how much time has passed.
There are many things about the characters that are vague and unclear.
It's also a film-- parenthetically-- this is a trivial detail, but it's always struck me as a mark of Altman's perversity-- every woman in the film is named Barbara.
I especially notice this because my wife is named Barbara.
I think the perversity had to do with Altman's idea that I'm creating a kind of slice of life movie.
Thieves Like Us is a particularly characteristic Altman film, not only because it has a kind of plotless amorphousness to it when you first look at it, even though it has great excitement in it and a great emphasis on character and a great emphasis on marginal, weird people who sort of are not solid citizens in any way.
But it also carries another signature feature of Altman's work, which I did not list on my outline.
And that is, he was very prone to allow his actors to improvise when they got in from the camera.
Or not literally when he was shooting, but they would improvise, work something out.
He didn't work from a set script, or he would allow departures from his script very easily and quickly.
One of the reasons he loved to work with Eliot Gould was Eliot Gould was a particularly gifted, improvisational actor.
And there's a kind of authenticity in the performances in most of Altman's films that partly come from the fact that he was deeply interested in trying to create an illusion of authenticity that was much more intense and much more compelling than what the traditional Hollywood movie offers.
He really wasn't interested in the fancy lighting and the very elaborate patterning of the Hollywood film.
He liked grit and dirt and imperfection.
And you can feel these elements in his work.
And you can especially feel it often in the performances of his actors.
The ones who liked working with him really loved it, because it gave them a significant creative part in the film itself.
So Thieves Like Us is a relatively minor Altman film in certain ways, but it shows his interest in character, his lack of interest in plot, and his love of the marginal, and of course of improvisational performance.
I've mentioned Nashville already.
Maybe I should say a little bit more about it, simply because it really was an unbelievably influential film.
The film seemed to desert conventional plot, and almost in an arbitrary way for its first half, simply to look at various characters who had musical ambitions and were bound for Nashville.
That was sort of the only throughline in the film.
Everyone's going to Nashville.
And you really wondered as you were watching the film, what could these characters possibly have to do with each other?
Will they ever come together?
Should they come together?
It's a very artful film in part because of the way it plays with the reader's frustration about a linear plot.
And in fact, one of the things the movie does is it makes you aware as a viewer of the extent to which a linear plot is a kind of crutch you lean on when you're watching a movie.
It helps you sort of get through the film.
If you're confused about other stuff, you always have that throughline to grasp onto.
And when Altman takes that away from you, or seems to take it away from you, he immerses you in the moral texture of his movie, in the psychological and political meanings of his movies in a way that would not be true if you were fixated on what happens next, what happens next, what happens next.
I want to say a couple of words.
I've mentioned Buffalo Bill and the Indians to you earlier.
And it's one of his most subversive Westerns.
It's interesting that he went back to the Western genre in this very creative period of his career.
And I think one of the reasons was he understood that there were aspects of the genre he hadn't made enough fun of.
He hadn't quite ruined all the sacred truths, to use a phrase that I'm going to refer to later that I have on my outline for McCabe and Mrs. Miller.
So he wanted to return again.
And that of course is the story about those Wild West shows I spoke about last week.
And the episode he dramatizes actually involves a comical and also very disturbing confrontation or interaction between Great Chief Sitting Bull and drunken Buffalo Bill, and an absolutely corrupt, drunken morally muddled Indian hero, played by incidentally Paul Newman, who had played upright heroes frequently before.
And let me jump down a little bit and say one word.
The films I've listed from '88 on are more recent films, all of them relative successes.
My view is that his greatest work is in the '70s.
But these later films I've listed are interesting.
But I want to say a word about Tanner and Tanner on Tanner, because they show how imaginative and experimental he remained, even in his later years as a director.
As some of you may know, Tanner is not a movie.
It was a television series.
And it was a very interesting television series in which we could see Robert Altman further developing this interest he had in this confusion between reality and fiction.
I think he was right about this in some ways.
Partly, I think, he felt that because we live in such a media-saturated culture and because of the power that celebrity seems to exercise in our culture, this celebrity was often one of Robert Altman's subplots.
It's a key subject in Nashville and he returns to it in a number of his later films as well.
The trick of Tanner is that it purports to be a documentary following a Democratic presidential candidate in the Democratic primaries of 1988.
And you've got an actor, one of his favorite actors-- this actor-- I think his name is Michael Murphy, he has a role in McCabe and Mrs. Miller.
He's the younger company representative who comes near the end of the film to negotiate with McCabe about selling his interest in the town.
And he's a wonderful actor, this guy Michael Murphy.
And like many of the actors who appear in Altman's films, you have a sense, if you watch Altman's films over the whole of his career, that there was a repertory company that kept coming back to work with him again and again.
Eliot Gould must be in six or seven of his movies, Michael Murphy in at least three or four.
He has favorite actresses too, who love to work with him, and Julie Christie is one, the star of tonight's film.
Warren Beatty is another of his favorite actors, although Beatty was such a big star that could generate his own films.
He didn't need Altman as much as some of these other actors did.
In any case, the trick of Tanner, the strategy of Tanner, the really memorable thing about Tanner was, I think, a very significant television event, is that the fictional Tanner, played by this guy Michael Murphy, treated with complete seriousness.
We mostly saw him through the eyes of his press managers, his press handlers.
So there was a great cynicism about the political process and the gossip of journalists about the presidential candidate.
But the real genius of Tanner was that we saw Tanner on the stump with the real presidential candidates.
And they cooperated.
The real presidential candidates cooperated in the making of the film.
And there are actually scenes in which the fictional Tanner gets into a conversation with Michael Dukakis, who was also running to be President.
And then there's an interview with Michael Dukakis in which Michael Dukakis assesses the danger that Tanner represents to his candidacy.
So when he returned to Tanner on Tanner in 2004, he elaborated this theory.
Now Tanner is a retired politician, and he's helping his daughter, who's a documentary filmmaker, do a new political film.
But now it's about the contemporary.
It was about the last election, the nomination.
I guess it was the first Bush election was what it was about.
And it's very interesting.
He brings back many of the characters.
Well, both of these texts, like some of his other films, are really preoccupied with the question of the blurred boundaries between the fictional worlds we live in and the actual worlds we eat and breathe in, and the extent to which our media-saturated culture has created a kind of virtual reality that we watch on television.
And now, of course, we watch on our computer screens that competes with the actual reality we live every day.
Well, Altman loved playing games with that question and raising questions about it.
And the first Tanner series is, among other things, a powerful meditation on the mendacity and the stupidity of the political process in our country.
It showed people trying to raise money, candidates scrambling to get five minutes in front of the news cameras just at the moment when the evening news is going to break, that sort of thing.
It's very wonderfully accurate, cynical account of American politics.
Well, let me turn now to McCabe and Mrs. Miller and try to concretize some of what I've been saying with specific reference to this remarkable film.
"Ruin the sacred truths," a line that I have plagiarized from the Yale literary critic and scholar Harold Bloom, who uses it to describe the enterprise of certain romantic poets whose aim was said, in Bloom's terms, to ruin the inherited truths of the society, to attack them, to assault them.
And in a certain sense, one could say that that's Altman's ambition as well, to ruin, to expose, to undermine, to drench in irony the secret truths that have come down to us from cinematic tradition, and maybe from more political and moral culture as well.
And we can see this in many different aspects of Altman's movie.
In many of his films we feel these elements, but especially powerfully and especially clearly, I think, in McCabe and Mrs. Miller.
One might say, in fact, that the whole energy of McCabe and Mrs. Miller, the whole energy of the film, results in a kind of tension or clash between the conventional expectations that we have about Westerns, and even the conventional expectations as they have been complicated by the serious and enlarging and already ironic Westerns of the late '50s and early '60s, a kind of tension or clash between the expectations we have about traditional Westerns, no matter how ironic they might have been, and the gritty, comical, uglified, and humanized reality that Altman actually presents us with.
And almost every element of the film has this kind of shock of recognition in it.
Let's take the question-- watch for example, how the film treats both the soundtrack and what you visually see.
When the film first came out, some reviewers complained that they had trouble making out the dialogue.
And they weren't lying about this.
It's because when Altman takes his camera inside dingy saloon in which some of the action early in the film begins and to which it returns periodically as the film goes on, Altman does not tamp down the ambient noise in the bar.
You hear the conversation of other people.
You hear bodies moving.
You hear various kinds of noise.
And sometimes, if the people who are in the foreground of the frame are talking in very low voices, their voices are blotted out by the sounds around them.
And in fact, Altman's treatment of sound all the way through the film has an energetic realism in it.
And it's a strange and ultimately, I think, very satisfying aspect of the film, the way in which Altman seems to try to create a grittier, more accurate account of the way sound operates in our world.
The fact is, when we're in actuality and we address someone in a crowded room, it's not quiet.
You can hear a lot of other things.
Altman wants to sort of recreate that.
And sometimes it almost seems as if that tendency, that digressive tendency that I've said is characteristic of Altman's imagination may even be expressing itself in the way he uses the soundtrack, because it's almost as if the soundtrack may sometimes be more interested in the words or sounds coming from the margins of the frame rather than from the center of the frame.
So there are these elements of potential abstraction that you always have to contend with when you're watching Altman's films.
The same thing from another angle could also be said to be true in some degree of the way the images work in the film.
Watch especially for the color palette that Altman uses.
He plays around with light and dark, with a certain kind of cold, dark, damp feeling.
And he contrasts that with a kind of lighting that suggests not incredibly bright sunlight, but a kind of nostalgic, golden, sunset kind of imagery.
And if you watch the film, in the very beginning of the film, one of the things that happens is that sometimes this golden sort of light appears indoors, and the outdoors seem gloomy and dank and dark.
And then later in the film, this division between light and dark undergoes an alteration.
You might ask yourself why.
In one of the most dramatic moments toward the end of the film, there is a group of murderers, assassins ride into the town.
And they ride in accompanied with the sun behind them, accompanied by this sort of golden haze that Altman has ironically associated with comfort and pleasure earlier in the film.
So both in terms of the sound and the image, we're engaged in something new here.
In considering the word "image," I should also mention Altman's montage, Altman's habits of editing.
He's capable of the most fluid and undistracted editing if he wants to do it.
But he is actually fonder of forms of editing that are abrupt, that are disconcerting, that are mildly disorienting.
And that style is part-- I don't mean does this at all moments of the film.
But that style, I think, is part-- sometimes he'll use abrupt cuts.
That style is part of his art, I think, but it's also part of this desire not to have things too smooth, not to have things too neat.
His love of the apparently imperfect is part of what's operating there.
Even a third element of what I'm calling his special or his new realism might have to do even with the way he treats not just the physical spaces that you see, but even the weather.
Watch how lousy the weather is in his film and how much damage it does to people, what an unpleasant thing the weather actually is.
Snow really matters in this film.
Bad weather really matters in this film.
It affects the outcome of the plot in certain ways, as you'll see.
So this messy realism I'm talking about even extends to the way in which the film treats the weather, as you'll see as the film proceeds.
The central features of the traditional Western-- the hero savior figure, the protagonist figure, the gal from the East who is often a schoolmarm who falls in love with the hero, and the idea that the Western as a text, as I've said many times already, is always a kind of founding story about the equation of a community.
All of those conventional structures remain in McCabe and Mrs. Miller, but they're reverted.
They're weird.
Let's take just a quick example.
The character played by Warren Beatty, he really is in some sense the founder of the town.
There's a miserable mining camp-- it just has a saloon in it when the film opens.
And McCabe arrives.
He's the great gambler.
He's actually a visionary entrepreneur in a certain way, although in a very ironic sense.
The story is an ironic entrepreneur's story, a Horatio Alger story.
This gambler, McCabe, comes to town, looks the place over, gambles for awhile with these poor schlubs who are suffering on the very margins of civilization and then goes away.
When he comes back, he's bringing prostitutes with him.
This town is not built by noble intentions.
It's built by a whoremonger who brings a bunch of prostitutes into the town and sets up the town's first house of prostitution.
And in fact McCabe becomes the biggest landowner and the richest man in town, not because he has the entrepreneurial vision of an Andrew Carnegie-- although actually, if we look closely at Andrew Carnegie's life, we might find he more resembles McCabe than his biographers have acknowledged.
In other words, it's as if what Altman is doing is he's telling a kind of ironic or mocking or antiheroic Horatio Alger story.
And as the story unfolds, certain lovely things begin to happen.
One thing that begins to happen is we begin to see a soft or a vulnerable side to the McCabe character.
And when the schoolmarm East figure arrives, she's really from the East.
She's not even from the East of the United States.
She has an English accent.
It's Julie Christie.
But she's not a schoolmarm.
She's a retired whore who is now a manager of whores.
She's a whore mistress, an expert in running whores.
And in fact, as soon as she arrives in town and partners with McCabe, McCabe really begins to make money.
What we begin to realize is McCabe really doesn't know what he's doing.
McCabe had a kind of vision, but he doesn't really-- and there's a wonderful scene over a meal in which the Julie Christie character who wolfs down her food like some John Wayne character, while the McCabe character looks very askance at her.
There's a scene in which Julie Christie in effect explains to McCabe how little he knows about how to manage women.
And then what develops is a kind of love affair between McCabe and this prostitute.
And what one begins to realize is first of all, the woman is much smarter than the man.
And you can see her despair being generated as the story unfolds and McCabe is manipulated by other people.
She knows the powerful forces that McCabe will be up against by the end of the film.
And she tries to discourage him and help him, but he's too caught up in his macho idea of what a man is to allow himself to see the wisdom of what she's saying.
And he's actually sometimes embarrassed by the fact that she seems to know more than he does.
Well, the relation between McCabe and Mrs. Miller inverts the normal love affair relationship.
It shows us a woman who's stronger and smarter, although also much more cynical and despairing than her man.
But McCabe's optimism and sense of self-confidence, which he loses pretty quickly, is shown in the film to be a function of his ignorance and his innocence.
So the treatment of the love story and the treatment of the hero savior figure in the film is deeply humanizing and deeply ironic.
They're actually more credible, psychologically complex characters, these two, McCabe and Mrs. Miller, than most of their predecessors in earlier Westerns.
But they're also deeply ironic embodiments of their much nobler ancestors.
And the same thing that I've just been saying about the hero and the heroine, about the love story and savior story is also true of the founding myth itself.
The name of this town is Presbyterian Church.
Very good name, isn't it?
And it takes its name from the steeple of an unfinished church.
And I think the allusion to unfinished churches in earlier traditional Westerns is certainly intentional.
It takes its name from the unfinished church in the town.
And in fact as you will see, the church is never finished.
You might watch how in the plot that works out, and why it works out, and what role the church and especially the minister who is hanging out at the church plays in the outcome of the story.
Does the church, and implicitly therefore does religion play a constructive or a destructive role in the film?
I'm not giving too much away to tell you it's obvious which answer is appropriate.
It plays a destructive role.
Just as it's hostile to certain political myths about the founding of the West, the film is also hostile to our pieties and complacencies about the role of religious institutions, and especially in the founding of our society.
So watch how it works itself out and how that traditional icon of the conventional Western has its meanings and its significance inverted and subverted in this film.
There's one crucial film, conventional Western, to which this film directly alludes, I think, or at least implicitly directly alludes.
It alludes to other films as well, but there's one especially.
And I want to show you a clip from it, because it has its own distinction.
And it reminds you not only about the sort of ballad dimensions of the Western, but how stylized and heroic in a certain way the conventional Western is.
It'll be a nice contrast with the antiheroic qualities of McCabe and Mrs. Miller.
And this is a scene from the very beginning of a film made in the early '50s called High Noon, starring the first or second most iconic of Western heroes.
If John Wayne was the first, then Gary Cooper was the second.
And this stars Gary Cooper.
But the main reason I want you to see the opening of the film is because it's very important to McCabe and Mrs. Miller.
And there's no doubt that Altman expects his audience not only to be thinking about many prior Westerns to contrast with what happens in this Western, but he certainly expects his audience to remember specifically the beginning of High Noon and the way High Noon plays itself out.
And perhaps also certain other Westerns play maybe almost as significant a role in the film, but High Noon especially.
And the main reason that High Noon is so important to McCabe and Mrs. Miller is that it's also a drama that takes place in the physical spaces of the town.
High Noon begins brilliantly with bad guys riding into town.
And we hardly see the hero until the final images of the sequence I'm going to show you, the Gary Cooper figure.
A man's been released from prison who's going to take vengeance on the town marshal.
It's the day of the town marshal's wedding when the bad guys are coming into town to take their revenge.
The film itself, incidentally, is a parable of the McCarthy era of the early 1950s.
Like many Westerns-- and we've talked about this-- like many Westerns, it's a disguised form of social and political commentary on the present.
And in this case, the marshal's endangered.
A lot of bad guys are coming after him.
He goes around the town asking for help to set up a posse to help him defend the town.
Nobody wants to help him.
He's left on his own, and he has to fight against these guys on his own.
And the drama of the film has to do with this.
And as you'll see, McCabe and Mrs. Miller ends on a similar note, in which McCabe is up against a force of men far larger than he, and also professional killers.
And in that sense, the end of McCabe, the whole last section of McCabe, which involves McCabe defending himself against assassins who have been dispatched by a corporation that has tried to buy McCabe out.
And when he's refused to sell, they've said, OK, we'll take another route.
We'll eliminate him.
So Altman's telling the story of capitalism at its most aggressively evil, instead of the kind of fairy tales that were told about how the West was made by certain mythological, optimistically mythological Westerns.
So when McCabe defends himself at the very end against these larger forces, one of the things that gives him an advantage-- and one of the most interesting passages in the film occurs here-- has to do with his knowledge of the town.
He helped build the town.
So he knows every nook and cranny of the town.
And he's able in part to hold his own against four other men who are actually much better with violence than he is, because of his knowledge of the town.
Something of that same kind of thing happens at the end of High Noon.
The beginning of High Noon is exciting because what it shows is the film is exploring the territory that it will later return to.
So I'm showing it to you because I think in part it's interesting.
It's a nice contrast.
There's no question that Altman expects us to have this film in our minds when we're watching McCabe.
[VIDEO PLAYBACK] Do any of you recognize this actor?
His name is Lee Van Cleef.
He played bad guys for most of his career, and then spaghetti westerns came in.
He was reversed.
He began to play less evil characters.
But he's an actor who's associated with evil roles in a series of Westerns.
The singer is Tex Ritter.
I also wanted you to see this beginning so you could hear this ballad and contrast it with Leonard Cohen's stoned aria.
Leonard Cohen sounds like he's singing out of the counterculture.
And watch the different subject matter.
Only someone who had never seen a movie would fail to recognize that these are bad guys.
Now how do you know they're bad guys?
They're unshaven.
They have black hats.
There's a kind of implicit symbolism that the conventional Western, even subtle ones like this one, which is a grownup Western, it's an adult Western, still make use of.
Altman will throw all those kinds of markers out.
He thinks they're foolish.
Another church.
[CHURCH BELLS RINGING] Think that's heavy-handed?
So you see how what's happening here is a kind of visit through the town?
It isn't until later in the film that you realize that every space you go by should have registered with you, because the later action of the film will take place on these streets, in these barns, on these balconies.
[INAUDIBLE] Now, watch this shot.
And there's Gary Cooper, who will be our hero.
And in fact, he's marrying Grace Kelly, who in this film is a Quaker.
And guess what one of the climactic events in the film is?
The Quaker lady fires a gun to save her beloved, violates her moral principles and kills someone to save her man.
OK. [END PLAYBACK] Now, I don't want you to think that I'm suggesting that High Noon is not a remarkable movie.
It is.
It's one of the very, very good Westerns.
It has great subtlety and power in its own way.
But I think that Altman definitely wanted us to think about the radical differences, between the universe imagined in McCabe and Mrs. Miller and the world as it's imagined in even subtle, adult '50s Westerns.
So many of these things are crystallized for us in the remarkable conclusion of the film.
And I hope that you'll pay a lot of attention to that.
It's an extended sequence.
And it operates according to a principle of parallel action.
The church I mentioned earlier catches fire.
Watch why it catches fire.
It's connected to the plot.
And while the church is burning, most of the town gets together in a kind of comic bucket brigade to put the fire out.
They are partly aided by a new fire engine, a mark of the town's growth that arrives at a certain point in the history of the town.
The fire engine, I believe, arrives at the same time that Mrs. Miller arrives in town.
They're both equally important historical events for the town's future.
But the arrival of the engine suggests the coming of civilization.
And there's a certain sense in which the fighting of the fire, it's actually almost treated like a slapstick event.
Real film buffs would actually recognize visual allusions or references to the Keystone Kops era.
I don't want to exaggerate this, but you'll see it.
I mean, there actually is someone carrying a bucket that has a hole in it, water's falling, that kind of thing.
And they're incredibly ineffectual in their trying to put out the fire.
I don't think they succeed very well.
The church essentially is destroyed, or at least the steeple is destroyed.
But what happens nonetheless is the town has come together.
And it's actually a moment in which you actually feel that the town's identity is being forged as these disparate people-- the whores and the barbers and the respectable people and the saloon keeper all get together to try to put the fire out.
So it is a moment in which the town is really aborning.
While that is happening, McCabe is being stalked by the assassins.
The creator of the town, the founder of the town, is in a fight for his life while this slapstick fire fighting is going on.
Watch how that works itself out.
And then pay special attention to the profound, disturbing ambiguities of the ending.
One of the reasons I admire McCabe and Mrs. Miller so much is that it seems in many ways to satisfy the desire one has for the trappings of a conventional Western.
All of the elements that you think should belong to a serious Western are present.
But they're present in such ironized, such damaged, such inverted form, that they make you rethink not only those particular conventions, but the larger mythological and ideological work that the Western mythology has performed in American society up to this time.
So it makes McCabe and Mrs. Miller not just a wonderfully entertaining film.
It's full of comic moments and brilliant performances.
It is in its own right a great work of imaginative intelligence.
But it is an even larger thing, it seems to me, because of the intelligence with which it engages in this systematic conversation with its aesthetic and moral ancestry in the great founding story of the Western, which I suppose must be identified as America's most distinctive and in some ways most disturbingly revealing story about itself.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We make a transition now to the final segment of our course in which we-- and this transition always induces a kind of guilt in me, a kind of deep ambivalence and disappointment, and awareness of the simplifications and reductions that are embedded in all syllabi and in all curricula.
And it's a good opportunity for me to remind you that even the primary emphasis of our course, which is on certain forms of American film, is itself highly selective and simplified and reductive as you-- as I told you.
They were-- in the Hollywood system, they were making 500 films a year.
Think of the small number we've actually seen, right?
It's the tiniest sampling of what's actually available.
And what I've tried to do is give you a kind of taste of what that phenomenon of the Hollywood film was like without trying to pretend at all that I was coming anything close-- not just couldn't be exhaustive-- we couldn't even be extensive or mildly serious about trying to do justice to the complexity of that body of material.
And that's even truer of course in this moment-- in these final moments of the course, where I try to do sort of a brief acknowledgment of the fact that there are profound and powerful traditions of cinema outside the United States-- in fact some more profound or at least as profound and some more powerful ones many people would say than the achievement of American cinema-- than the achievements of American cinema.
And perhaps the strongest example of that-- the most dramatic example of that-- is French film.
It has a kind of parallel history.
At virtually every stage of the history of French film-- of American film, you can find a kind of counterpart story in French film as if-- it's almost as if-- and in fact, there have been historians who have tried to argue about the priority.
Who invented film?
Were the French more advanced than the Americans?
Who first developed forms of serial movies?
Who first developed forms of the chase-- of the comic chase film in the early silent era?
And so forth.
And I think these arguments are not very helpful about priority.
But it is very important to remind you that the history of other national cinemas, and most especially the histories-- the history of French cinema has the same complexity, nuance, detail that American film has.
We can speak-- and just let me very quickly remind you.
I've asked you to read a chapter from David Cook's history of narrative film that deals with the complex achievements of French film in both the silent and in the sound era.
And I hope you read those chapters and reread those chapters closely, and think about them.
Just as a reminder, however, we-- there are arguments, just as Edison and-- the earliest films that were made in the United States are associated with the Edison company.
So the same kind of thing is true for the Lumiere films that were developed in the 1890s in France.
And of course, just as they were pioneering directors who explored and the nature of cinema, the possibilities of this new medium in the United States, an identical activity was taking place in France.
And some of you of course know about the essential contribution of the early pioneering director, Melies, who talks extensively about him in the chapter I asked you to read-- the man who in fact invented the science fiction movie, in the film A Trip to the Moon, and did so many other early experiments with for-- with combining the reality of film with certain forms of surrealist imagination.
The tradition of discourse about film in France probably is deeper and has a longer life than in any other society.
And many people would say that along with Eisenstein, the great Russian director, it's the French who elaborated the first systematic forms of film theory.
And there were in the silent era and in the '20s, in France, a series of developments that encouraged a systematic kind of theoretical approach to film.
I won't mention names here or particular or particular theorists-- simply acknowledging the fact that the French tradition of discussion about film, and in the equation of film, is at least as rich as that of any other society.
I say this because making John Renoir stand for this whole rich tradition is unfair, even to so remarkable and influential director as Jean Renoir.
But having made that basic apology, let me now make my transition to Renoir himself.
Some of you will recognize Renoir's name I hope.
He's the second son of the great impressionist painter, Auguste Renoir.
And in fact you can-- he was himself a subject of his father's paintings.
There's the young Renoir with blonde reddish hair as he appears in one of his father's paintings in the period 1895 to 1899, when young Jean appeared in a series of paintings.
In some of the paintings his gender is very ambiguous.
He looks-- not just that he has long hair, but he even seems dressed in a kind of girly way.
And he's sometimes mistaken for a girl when people look at the images.
But isn't it interesting that this great film artist is himself in a series of-- is already in a series of immortal works of art created by his father?
Jean Renoir grew up in an atmosphere of avant garde-- almost avant garde frenzy-- many of the most distinguished artists and performers of the day were regulars in his household because his father by the-- he was born late in his-- very late in his father's life.
And his father was already a very famous painter by the time Jean was born.
When he grew to adulthood, he-- around the time of the First World War-- he served first in the cavalry in 1913 very briefly, and was actually injured-- he was kicked by an animal, I think a horse or a mule kicked him, and he was laid up.
Then he re-enlisted in the infantry where he was wounded and he said finally-- so this is very rare in the first World War-- he ended up serving also in the Air Force.
And he was briefly a pilot in the French Air Force.
So he served in all three branches of the French-- First World War.
And the film you're going to see tonight, the great masterpiece, such-- the great influential, deeply influential film, Grand Illusion, is a war film of a certain Renoirish sort as you'll see this evening.
He also served briefly as a prisoner of war during the First World War.
And that experience of course, in some degree, in some central way, informs the story and the experience of Grand Illusion, which is also a story about-- a film about prisoners of war during wartime.
You can get some sense of-- do we have some pictures of his father's paintings?
Let me just get-- these are just of course among some of Renoir, Pierre's most famous paintings.
And I thought that it would be-- you would be interested to see them to get some sense of-- to remind you of the importance of his father's art, and of the asking you-- and also as a way of encouraging you to imagine what the impact might have been on the young boy to have grown up in an environment like this.
I don't know, that does look like Jean.
Maybe there's another picture of Jean.
He had kind of a jowly face.
And that may be among the oldest images of him that he-- him at the oldest age that his father painted him at.
He worked on his first-- after he emerged from the First World War, he had conflicting ambitions.
But he moved into film fairly soon.
He wrote a script in 1924.
He began to make his own movies around the same time.
I have a very brief list of some of his films.
Would you put those up?
I want to talk very quickly about a couple of his films, and then turn to certain other important matters.
But the most important thing to say about Renoir's-- Jean Renoir's career-- is that like some important American directors, his career spans both the silent and the sound era.
And he made some significant experimental silent films of which the most well known in terms of experiment is the film The Little Match Girl, based on a Hans Christian Andersen story.
And it uses certain kinds of surreal imagery that's very surprising, especially in a silent film.
And it-- and I mention that he made a silent version of a novel by Zola, Nana, as a way of simply also reminding you from a very early stage, Renoir understood that the movies were a form that could sustain the most ambitious kind of artistic act-- aims.
And it's a measure of his sense of that, that he would adapt, even during the silent era, a film by one of France-- by a classic French author.
But his mostly-- but his really significant work begins in the sound era, and in what's often called his French period.
And the films I've listed between 31 and 39-- not a complete list of his films, this is a very selected list-- but it's a list of his most-- of some of his most significant-- all of his most-- among his most significant films for sure.
No really important title is missing, although there are many other interesting films that he made.
And it's the period between 31 and 39 when he left France, after Rules of the Game, while-- when the Second World War was imminent, came to the United States.
And the films of the '40s that I've listed there were made in Hollywood.
And he was very welcomed in Hollywood.
And there are people who-- film scholars who are great fans of these three films, these three titles, that he made in the United States.
Then he left the United States.
The film, The River, which many Renoir buffs love as much as his primary films, although I find it a much slower and less powerful film, although visually incredibly beautiful.
It's set in India.
And the river is the Ganges.
And it's a very, very remarkable meditation on the power of place in society.
But his central achievement are the films of his French period running through the 1930s.
And I want to say a word about some of these.
Well, one of them, La Chienne, stars the same character who stars in Boudu Saved From Drowning-- and you're going to see a clip from that film in a moment-- Michel Simon.
And La Chienne announced a kind of complexity and ambitiousness in Renoir's work that-- would deeply significant.
It had a kind of moral or political claim.
It was a film set in the slums.
And it was about an aborted, or an abortive or tragic love affair, between a working man played by a proletarian, played by Michel Simon, and a prostitute who is unkind to him at the end, and throws him over at the end.
And the film's interest in the life of the low-- in the lives and circumstances of the lower social orders was especially significant.
And Michel Simon's immensely powerful physical performance was also memorable.
And you'll see a version of that in a moment.
In the following year, he made what many people call his first great masterpiece, a film called Boudu Saved From Drowning.
It's a-- the French title is [FRENCH], saved from the waters, [FRENCH], the waters, plural.
And the French title is a little better as I'll try to say in a moment.
He made-- I mention the Madame Bovary partly again to show you something about his ambition.
He's trying to say in effect, look, the film is an equivalent art form to the great novels of our past.
In 1935, he made a film that many identify as a forerunner of Italian neorealism, the movement we'll be studying next week.
And I talk a bit about Toni in next week's lecture.
It's a film, very experimental in certain ways, it uses a lot of non-professional actors.
And again, it's about the circumstances of the working class in some ways.
It's about French Quarry workers who were treated with a kind of clarity and attentiveness that previous films had not-- had rarely granted to members of the-- not only the lower social orders, but here, immigrants-- not even natives.
His two masterpieces come in '37 and '39.
Again, remember I've left out a number of titles there-- this is a selected list-- the film you'll see tonight, Grand Illusion, and what many people think of as his greatest film, a deep, complex, satire on contemporary French life in the-- just before the war, called Rules of the Game.
The primary label that's attached to Jean Renoir's work is that of poetic realism.
And it's a way of trying to distinguish a form of movie making that emerges, of which Renoir is the most dramatic and powerful exemplar.
But there were many other examples in French cinema.
And it's worth talking a little bit about a key forerunner to the tradition of poetic realism, a director named Jean Vigo, who died tragically young, as you can see from his dates, who made the three films-- three titles I've listed there.
And these films had an immense influence on later filmmakers.
Zero for Conduct is a film set in a-- it has surreal elements.
And its plot is hard to follow.
But essentially, it tells the story of children going back to school.
It's a-- and following them once they get into a kind of boarding school, and rebel against their house masters.
And there's a sort of comic element in the film.
And there's also an element that might be called an impulse toward lyric retardation, by which I mean, the retarding is the retarding of the plot.
I don't mean mental retardate-- right?
There's a kind of lyric impulse to celebrate what's going on right in front of you at the expense of the plot, as if the story almost stops moving at a certain point, while the camera and the-- well the camera itself sort of indulges in witnessing a spectacle so intrinsically interesting in itself that it seems to lose interest in the ongoing story.
So there's this tension in Vigo's films from the very beginning.
And what some scholars have seen a tension between denotation and connotation, between the denotation being the simple realism, the ongoing story.
Right?
And the connotation being the poetic or lyrical impulse of the film to sort of celebrate life in its complexity and its nuance, without any interference from the demand that you follow a story.
And this tension is a fundamental element, not only in poetic realism, but as you'll see, in the Italian form of this, called neorealism that emerges out of it.
And it's somewhat of a grittier and more historically politically engaged kind of drama-- kind of film, even though it's a direct outgrowth of the kind of thing we're saying about poetic realism.
And in both forms, in both kinds of film, there is this retarding impulse, this lyric impulse, in which the story's desire to get on with itself is sometimes in conflict with the camera's desire to look at what it sees, to revel in what it wants to look at.
And so Jean Vido, partly indeed because of his tragic life-- short life-- and partly because he was the son of an anarchist, of a political anarchist, of a serious anarchist, who wrote about theories of anarchism as a political movement-- and he himself was very hostile to authority, Jean Vigo.
And his films animate a kind of anarchic, anti-establishmentarianism, that is very distinctive.
He's almost always on the side of the weak and the powerless.
Let me say a few words about what we might call the key features of neorealism.
One of the most fundamental elements of neorealism is what could be called a mise en scene style, that is to say, it's committed to long takes.
The poetic realism is interested in the external world, and in the relation between characters and the outer world.
And although it will use abrupt cuts of various kinds, it won't do so at the expense of your experience of the outer world.
It wants the audience to take in what it sees, and to make judgments about it.
It doesn't try to-- in the way that certain other kinds of styles might do-- to manipulate your response in quite the-- in so dramatic a way.
And what follows from this mise en scene style is what some scholars have called in camera editing.
because what's going on is because the camera, the tape, because the camera is going-- well the style is committed to long takes, what will happen is the camera will alter its focal length-- its depth of field-- or it will simply move and change its-- the object of its gaze while it is in operation.
So that it's-- instead of a series of cuts that are made in the editing room, it's the camera man working the camera who's making certain decisions that in another kind of film would be made by a film editor after the fact.
And this creates an effect of not exactly improvisation, but an effect of-- an effect that creates the impression that there's a kind of-- sometimes at its most powerful or compelling moments, as if the camera is actually almost a living witness to what it's seeing.
And it's responding in very nuanced, and sometimes very abrupt ways, to what it sees and hears.
So there are sometimes moments in both poetic realist films, and in Italian neorealist films, in which a noise will occur, and you'll suddenly see the camera turn to find out what the noise is, as if the camera is humanized in this kind of scene.
And what follows from that is the camera is all-- not always, but usually set at eye level.
It does not give a lot of fantastic distorting angles of vision because it's not interested in that kind of surreal or expressionist representation.
So mise en scene style in camera editing.
A second fundamental feature would be filming on location, right?
No more in studio.
There's a kind of gross fundamental realism to these kinds of films because they use natural lighting.
They insist or try to insist on themselves as actually photographing a world you recognize as real, not a world that has been artificially constructed in a studio-- in a studio space.
What follows also from the location filming, is a commitment to what might be called true light and sound, right?
It's very rare in these kinds of films to have external sound.
The sound, even the music that comes up in many of these, in most of these films-- this is a wonderful habit of Renoir himself.
Sometimes you actually think the music is not a part of the drama.
But then you discover it is.
Almost all the music in Renoir's films is part of the dramatic texture.
It's not imposed from the outside.
It's not a soundtrack that doesn't grow out of the action.
So if you hear music in a Renoir film-- I'll show you an example of it in a moment from Boudu-- it turns out that the music is being played by characters in the film.
And you actually end up seeing them with their instruments.
So true light and sound.
And what that also means is sometimes sound is obscured.
Sometimes sound is overlapped with other noises.
Sometimes dialogue is hard to hear because there are noises in the real world.
I noted that this was an aspect of a number of American directors in the post studio era, and especially in a film like McCabe where Altman does this kind of thing.
And I think that it's very probable, even though I don't know this for certain, that Altman himself went to school on these films and knew the experiments with sound or the treatment of sound that was characteristic of Renoir's films, and other forms of Ita-- of both French and Italian forms of realism-- what was called realism.
And then two other features-- one already implied in what I was saying-- a camera that's very fluid in motion.
Not in gigantic motion, but very subtle kinds of motions.
And if you watch Renoir's camera, you'll see that it's almost constantly making these minute adjustments to what it's looking at or to what it wants to see.
And then finally, something I've already said, there's a great emphasis on what could be called depth of field.
That is to say, you're aware of the dimensionality of the images you're looking at.
So there's a background, a middle ground and a foreground.
And very often, you'll see the camera change its focal length, and what will-- if the foreground was in focus, you'll actually experience the camera shifting its focus, bringing in the background that had been out of focus before.
And the effect of this while it-- because it takes place within your experience of viewing it-- doesn't take place hidden behind edits-- is to make you part of the process in some sense.
Again, to make you feel that the camera's behavior is a part of your experience of the film, that you're experiencing the film through a medium that has a kind of immense sensitivity to what it's looking at, an immense respect for what it's looking at.
Well, so, key features, let me just summarize them again-- a mise en scene style, which means in camera editing, especially filming on location.
Commitment to true light and sound.
A fluid moving camera.
Commitment to depth of field.
What we might call, again, creating this tension between a kind of lyrical impulse, a kind of celebratory or a poetic impulse as some people have said, and an impulsive simply to deal with the world, to capture the world, to describe, or to dramatize the world fully.
Well, I think I can show you or dramatize these principles more fully for you, if I give you some examples.
But as a way of introducing this, I want to talk very quickly about one formulation immensely influential formulation about Renoir that I think you'll find helpful and interesting.
And these are lines that have been written about Renoir by the great critic, Andre Bazin, who was a great champion of Renoir's work-- also a great champion of Italian neorealism, and one of the great theorists of the cinema.
And we can capture-- I think when-- I think that Bazin captured something of Renoir's importance in these passages from his book titled, John Renoir, published in 1971.
"No one has grasped the true nature of the screen better than Renoir," so wrote Bazin.
No one has more successfully rid it of the equivocal-- of equivocal analogies with painting and the theater.
Plastically the screen is most often made to conform to the limits of the canvas.
And dramatically, it's modeled on the stage.
You see what he-- Bazin is saying, is he's reminding us that when a new technology emerges, old habits are so deeply embedded that it's very hard to free oneself from them, even though there's nothing inherent in the new technology that requires you to see the world in the ways that the older systems did.
And then there's a period of transition that's involved, a period of experiment and discovery.
What Bazin is saying here is that Renoir is one of the great pioneering discoverers, right?
Because with these two traditional references in mind, directors have conceived their images as boxed within a rectangle.
As do the painter and the stage director.
And in fact this still happens today with lazy or foolish directors.
Renoir on the other hand understands that the screen is not a simple rectangle, but rather the surface of the viewfinder of his camera.
It is the opposite of a frame, right?
There's stuff that goes on outside the frame.
If it's look-- this is a brilliant piece of writing I think.
Technically, this conception of the screen assumes what I, Bazin, shall call lateral depth of field.
Lateral depth of field-- that is to-- but it's an oxymoron.
Again-- contradiction, but how could there be a lateral?
But what he means is that you become aware of things going on on the margins of the visual field, even outside of what you can actually see.
Since what we are shown is only significant in terms of what is hidden from us, and since the value of what we see is therefore continually threatened, the mise en scene, right, what is put in the scene, right, the mise en scene cannot limit itself to what is presented on the screen.
The rest of the scene, while effectively hidden, should not cease to exist.
The action is not bounded by the screen, but merely passes through it.
And a person who enters the camera's field of vision is coming from other areas of the action, and not from some limbo or some from imaginary backstage.
You get this?
Think how exciting this is.
What a fundamental insight it is into the way a film can become compelling, and why it is that so many films might seem factitious to us despite the gross reality of the cinematic image.
Likewise, the camera should be able to spin suddenly.
Renoir is full of moments in which the camera will make a wide swa-- sometimes 360 degree circle, sometimes 180 degree turns, in order to remind us of what has not been in the frame, of what is on the margins of the frame.
So the camera should be able to spin suddenly without picking up any holes or dead spots in the action.
Well I want to give you two examples of this, which are, among other things, instances of what I call-- put my outline back up-- instances of what I call visual style as moral vision.
And I'll hopefully be explaining that term as we look at it.
So the first clip I want to show you is from the film you're going to see tonight.
And it's a way of alerting you to qualities not just in this scene, but elsewhere in the film that I hope you'll become much more attentive to, because we're taking the time to single them out now.
So here is a scene from-- are we ready?
From Grand Illusion, from relatively early in the film.
Now, as I've told you, the film is about prisoner-- French prisoners of war.
So here's a scene.
These are prisoners.
But it's the First World War.
They're allowed to receive packages from their-- in the mail, and what people-- that relatives and friends send them is food very often.
All right.
Can you make it a little louder, not that it really matters?
Now, watch the camera's behavior.
Can you freeze it one second?
Look at the window there.
Why is that significant?
What do you notice in the window already?
Movement, right?
Do you see how-- and the way the scene began, we know where that character has come from?
Even though these two characters are in confrontation and talking to each other, we're aware of activity in the window behind them.
We're aware of other people in the room.
It actually may seem as if it's very easy to create this effect.
But of course it's profoundly difficult.
And Renoir is the pioneer in creating these kinds of effects.
How fluid the camera is, how-- that's what I want you to watch for.
Your awareness of the fact of action that's going on partly outside the frame.
All right, continue.
And the man at the head table is the wealthiest of the prisoners.
And he gets the best food.
So he shares it with his comrades.
It's a tremendously good meal for prisoners.
This is the least violent war movie ever made.
Are you beginning to notice how each of the characters is individuated in this scene?
I don't have time to talk about this.
Watch it tonight.
Maybe I'll say a bit about it tonight when I introduce the film.
Look, each one of them we can almost-- we can tell what social class they belong to from their dress, from their mode of speech.
You see how quietly, but complexly, the camera examines the space?
And that's Jean Gabin, one of the central characters, and one of the great French actors.
All right, we have to stop this because-- I'm sorry I have to cut this short.
It's a brilliant scene.
Watch for it tonight.
Comes fairly early in the film.
Watch how each of the characters is individuated as this scene goes on.
How crowded the scene is in one sense.
How you're aware of activities, and even conversations, that are taking place outside the frame of the camera itself.
And when the camera shifts over to one set of characters, you're aware of other conversations that are ongoing.
In other words, the sense of reality that this style creates is profound and compelling it seems to me-- powerful.
But I want to show you another and even more dramatic clip, partly because it comes from a film you've not seen, and because it captures certain other qualities in Renoir's work.
And this is the famous ending of Boudu Saved From Drowning.
Let me set the scene for you while Kristen's getting it up.
In this scene-- this comes at the very end of the film.
The film is a satire.
And Boudu is a bum, a [FRENCH], of whom there were tens of-- there were thousands, if not tens of thousands living in Paris during the era when the film was made.
A kind of tramp, like the character that the Chaplin figure plays in the American films.
Although Michel Simon is a much more massive figure than Chaplin.
And in the very beginning of the film, he plays this-- he seems to try-- he jumps into the river to try to commit suicide because he's lost his beloved dog.
He's a figure of despair.
And a middle class bookseller, named Lestingois, spying him through his spy glasses, looking out of his book-- out of the windows of his bookshop, sees this bum.
And at first he goes, oh, what a perfect bum, he thinks to himself.
What a perfect embodiment of what a tramp is.
And then he becomes upset when he sees the guy jump into the river, and he runs out, he dives into the river to save him.
And it turns into a kind of comic scene of saving.
People gather on the bridge and look down and so forth.
He pulls him out, brings him into his home.
His wife is very resentful of it.
But eventually, she adapts.
And he moves into the middle class bookseller's home.
And of course he wreaks havoc there because he stands for nature itself.
He can't be civilized or tamed, right?
He's Boudu soul, saved from the waters, right?
So he's saved from the waters at the end.
But as you'll see, at the end-- at the beginning.
But as you'll see at the end of the film, he's back to the waters.
So here-- so what happens essentially is among other things in the course of the story, this figure of nature, turns out that he can't quite be civilized.
One of the things he does is he manages to have a love affair or to seduce-- I shouldn't call it a love affair-- both Lestingois's mistress, a young maid who works in his house, and his wife.
And a kind of semi scandal occurs in which far-- in which the tramp character, played by Michel Simon, wins the lottery and becomes very rich.
So he then-- a marriage of convenience is arranged.
And he's going to marry the maid that had been the bookseller's mistress.
And this sort of straightens everything out because there had been a kind of scandal brewing.
And in the final scene of the film is the wedding party, in which Boudu, his new bride, the bookseller, and his wife are in a boat together celebrating the wedding.
[MUSIC ONSCREEN] And of course, these images would have invoked Renoir, Pierre's paintings very powerfully as well.
[MUSIC ONSCREEN] Many of Jean Renoir's treatment-- films set in nature are said to have been influenced in some ways by his father's paintings.
There's Michel Simon and his new bride.
You see how there's music, but it's coming from source inside the story.
[MUSIC ONSCREEN] That's the bookseller in the back, with the white hat.
Everything looks wonderful.
[MUSIC ONSCREEN] One thing I think, one reason I wanted you to see this is look how leisurely this is.
I mean, where is it going?
We're at the end of the movie yet, now, and yet look how the film sort of, almost as if it doesn't, even at this stage, doesn't want to end-- doesn't want to move forward.
[SILENCE] All right, he reaches for the flower.
[SPLASHING] [SCREAMING AND CRYING] Now remember, he's just been married.
What's happening to him?
Where's he going?
[SCREAMING ON FILM] Often when I show this to my classes, because I'm always worried about time, I get impatient here.
But in fact, it's a bad reaction, because the film wants us to savor what it's doing.
Right?
Can you feel the sort of lyrical tendency just to sort of look at the world in its beauty?
As if the impulse to tell a story and the impulse to photograph the world are in some degree in conflict, because the world keeps resisting the categories you put it in.
The camera keeps discovering new things to look at.
So here Boudu comes ashore.
Think of how anarchic this vision of life is, because he's just deserted his new bride, and his new wedding, and all that.
Right?
Doesn't seem to matter.
He gets up on shore.
He's still wearing his wedding clothes.
And this becomes then a highly symbolic moment.
[MUSIC ONSCREEN] He sees a scarecrow.
Now he's going to shed his middle class identity by changing clothes with it.
[MUSIC ONSCREEN] I suppose one fundamental aspect of the satire is-- what the film is saying is look, nature is always more untamed and difficult than middle class romantics imagine.
It's not a tamable thing.
Oh here he comes, eating is bread, newly free again.
Right?
Identifies with the goat, shares his food with the goat.
[SILENCE] [SINGING ONSCREEN] [GOAT BLEATING] [SINGING AND MUMBLING ONSCREEN] Pay attention to the camera.
[MUSIC ONSCREEN] OK, shut it off.
At one point in Bazin's book on Renoir, he talks about this sequence.
And I want to remind you that he talks about this sequence in an era before it was possible to easily replay the movies, right?
It was an era long before video tape, long before television actually.
And when you wanted to watch a movie again, you have to get projectors and show them.
So very often when Bazin was writing about his film, he was working from memory.
And he makes some technical mistakes.
But he gets the essence of the scene right.
Listen to this.
He says, "Renoir the moralist is also the most realistic of filmmakers, sacrificing reality as little as possible to the thrust of his message.
The last scenes from Boudu could serve as an epigraph to all of Renoir's French work.
Boudu, newlywed, throws himself into the water"-- not exactly, it's an accident.
It's important that it's an accident.
It's not like he was planning his escape, right?
Because he's nature.
He doesn't think.
He stands for the natural, right?
But by accident, the boat overturns, and Boudu makes his escape because it's possible to do so.
It's not as if it's premeditated.
That's important.
"Dramatic or psychological logic would demand that such an act have a precise meaning.
Is it despair or suicide?
No it's not at all.
In fact, it's just an accident."
Boudu is-- without maybe even fully realizing it is trying to flee the chains of a bourgeois marriage.
"Renoir like his character, forgets the act in favor of the fact, and the true object of the scene ceases gradually to be what Boudu intends."
Right?
What is Boudu up to?
Why is he doing this?
I don't even think Bazin's right to even raise the question because what we think he is doing is just enjoying himself, right?
Then it occurs to him, when he gets on land, that he's free of his marriage.
"Renoir, like his character, forgets the act in favor of the fact.
And the true object of the scene ceases to be Boudu's intentions, and becomes the spectacle of his pleasure.
And by extension, the enjoyment that Renoir derives, or we as viewers derive, from the antics of his hero.
The water is no longer water, but more specifically the water of the Marne-- a tributary of the Seine-- in August.
Michele Simon floats on.
It turns over, sprays like a seal.
And as he plays, we begin to perceive the depth, quality, and even the tepid warmth of the water.
When he comes up on the bank, an extraordinary slow--" He calls it a 360 degree pan.
Remember that final move the camera?
But it's probably 180 degrees.
He's remembering it imperfectly. "
--shows us the countryside he sees before him."
The world he sees before him.
"But this effect by nature descriptive, which could indicate space and liberty regained, is of unequal poetry because what moves us is not the fact that this countryside is once more again Boudu's domain," although that is one effect of it.
Boudu is free.
And the camera shows the beauty and freedom of the world in that 180 degree pan.
"but that the banks of the Marne and all their richness of detail are intrinsically beautiful.
And you're aware of that, aren't you?
At the end of the pan, the camera picks up a bit of grass, where in close up one can see the dust that the heat and the wind have lifted from the path.
One can almost feel it between one's fingers.
If I were deprived of the pleasure of seeing Boudu again for the rest of my days," I remember the first time I read those lines, and I realized what it meant to be a movie critic or a literary critic.
Think of what's implied by that.
"If I were deprived of the pleasure of seeing Boudu again for the rest of my life" as if this would be a deprivation too horrible to contemplate.
Right?
It made me realize my own vocation in a way.
"If I were deprived of the pleasure of seeing Boudu again for the rest of my days, I would never forget that grass, that dust, and their relationship to the liberty of a tramp."
The point of this exercise is to remind you of the immense power, the potency, of even a single camera move.
Think what that 180 degree pan suggests, as Bazin brilliantly argues for us.
So the conclusion then is that the visual style of a film, over a certain films anyway, can express a moral vision.
And by moral vision, I don't mean moralistic-- what's didactically right and wrong-- but a vision of having to do with the values and assumptions you make about the nature of the world.
There's a moral vision implicit in the tentativeness, the hesitancy, the retarding impulse to dwell and linger on things, in Renoir's camera, and in the basic habits of poetic realism that you will see brilliantly embodied in the film you're going to watch tonight, Grand Illusion.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good evening people.
I place so much emphasis on Renoir's camera because I think it's so fundamentally expresses his sense of experience.
And if you become attentive to the really quiet and subtle ways in which Renoir's camera is almost always in action, is a part of the story in some sense, you can begin to capture something of his importance in the history of cinema.
Because no director or apprentice director, whoever studied, looked at Renoir's films came away from it the same, and his influence is almost impossible to fully measure.
there are no cinematic traditions, after Renoir, that haven't, at least in some degree, even if not directly, then indirectly through other people who've been influenced by him have been shaped by Renoir's example.
And I wanted to give you one more instance of Renoir's camera.
This is from a film I've mentioned, but not shown you.
It's from the last of his great French films.
Some people, many people would say it's his greatest film, one of the great masterpieces in the history of movies, the rules of the game.
The only reason I don't show it in our course is that it's, in many ways, a more difficult, a more complex film then Grand Illusion.
there's a grand clarity, or simplicity of a certain kind.
not a simplicity of character, or a vision, or a theme, but a simplicity of intention that becomes clear, at least after the fact in Grand Illusion.
It's a more accessible film, especially for Americans, I think.
The Rules of The Game, much more profoundly, is a meditation on the theme that is also present in Grand Illusion, as you'll see this evening, the theme of historical transition.
But it's a much more complex and extended meditation on what was happening to French society in this immense transition that took place at the beginning of the 20th century.
Essentially, it's a transition, as you'll see dramatized in the film we're showing tonight, Grand Illusion, It's a transition from an aristocratic to a middle class culture.
From an aristocratic dispensation, to a quasi democratic, or more democratic dispensation.
And this was a topic which preoccupied many directors, Renoir especially, and is a kind of central topic in both Rules of the Game and Grand Illusion.
This scene from Rules of the Game is maybe his least subtle in some ways.
Or rather, the subtlety is harder to see if you're just looking at a clip.
The subtlety is involved in the characters themselves, who have already, in some degree, been established in a kind of individuality, which then further expresses itself in the scene you're going to see.
The basic situation of the Rules of the Game is a classically powerful one.
it's essentially a weekend of festivities.
A marquee, a grand figure, presumably who owns a remarkable chateau, invites some friends for a weekend of games and shooting, hunting, at his great chateau in the country.
And the film is a kind of series of tableau, or a series of festivities, all of which are cankered, or damaged in some way.
It's as if we have a sense that each of the festivities or festivals that are dramatized in the Rules of the Game are diminished versions of something that, in the past, had great authority, power, and cultural centrality, but have become now diminished, and even parodically reduced versions of what they had been when they were part of a coherent culture.
it's almost as if these festivals and rituals have become-- are recognized by the film, although not necessarily by the characters inside the film, as vestiges of a time that has now passed.
And the most dramatic instance of this subject, or the embodiment of this theme, is the scene you're going to see.
It's the hunting scene.
And I want you to, sort of, at least get a sense of it, not only because it's an important film and I'd like to fix it in your mind.
Maybe give you a taste of it so you'll go out and look at the film yourself.
these two films, Grand Illusion and Rules of the Game are certainly the pinnacle of Renoir's art, and people who know these films deeply know something important about film, know something important about what works of art are.
So I want to see for that reason.
But I also want you to see to get one more experience, outside of Grand Illusion, of the way Renoir's camera behaves.
especially the way it establishes a relationship between character and ground, between character and environment.
And here, of course, the camera's job is not only to tell us something about the relations among the characters, and to show us their interactions in as economically powerful a way as possible, but also to give us a sense of this grand ritual gone bad.
OK. [VIDEO PLAYBACK] [HORN BLOWS] -[INAUDIBLE] [END PLAYBACK]
Freeze it one second.
That little character who's giving instructions, you will recognize him.
The actor's name is Dalio, and he's the one who plays Rosenthal in Grand Illusion.
In this film, he plays the marquis.
[VIDEO PLAYBACK] -[SPEAKING FRENCH]
You see how powerful it is not to have fake music?
What, just the effect of that is on the film?
I shouldn't say fake music, non-diegetic music, music that's not part of the scene itself.
There's Dalio.
-[SPEAKING FRENCH] -[SPEAKING FRENCH] [HORN BLOWING] [CLATTERING] [END PLAYBACK]
Freeze it for a second, please.
I wish we had time to dwell on this, and I won't do this too much because I'll run out of time for what I need to tell you, but tell me what you see here quickly.
Make some conclusions about-- what about this great ritual?
Immediately, what would we think if we're watching this?
What's one conclusion you draw from this?
Is this hunting the activity of brave pioneering souls who are confronting the dangers of the wilderness?
[LAUGHS] Right?
It's an unbelievably controlled, and in many ways, murderous environment isn't it?
What have we begun to feel about these creatures that are being flushed out?
Tremendous sympathy for them, right?
We're hostile to the adults, right?
We're hostile to the human beings in the scene.
We think there's something the matter with this.
And of course in the olden days, presumably in the days when such hunts occurred in a more coherent, and less forced way, you need beaters to go around in other words, making noise to scare the few vestiges of rabbits and other game out into the open so people with giant guns could kill them.
right?
in the 16th or the 15th century, or the 18th century, these fields were teeming with game.
Now, in the 20th century, you have to beat the sticks to get them to come out.
What a parody of what an actual hunt would be.
Go ahead.
[VIDEO PLAYBACK] [CLATTERING]
When this film was made-- I forgot the number, I have it in my notes, but I couldn't find them there, I left them in my office.
A certain number of these animals were actually killed in the making of the film.
I forgot the number, but it's a significant number.
[GUNSHOTS]
If today, they made such a film, there are rules that don't allow them to kill animals.
[GUNSHOTS] [BIRD CHIRP] [GUNSHOTS] [GUNSHOTS] [HORN BLOWS] [END PLAYBACK]
OK, you get some sense of how powerful a satirist, but also how quiet a satirist, he can be.
It's not as if the film-- it's not as if there's someone there saying, isn't this horrible.
He's leaving it to the audience to figure out, although, it doesn't take too much to figure it out, does it?
What you can pick up from this, it makes, maybe, the scene that I've shown you may make the scene seem cruder than it is.
Virtually all the characters, faces that you've seen.
most of them, anyway, are major characters in the film.
And even by this time in the film, you've gotten to know some of them quite well.
So they've been humanized and individuated, and that makes the brutality, to which they are mostly oblivious, of what's happening even more powerful.
You don't hate the characters who are doing the shooting.
you know them for the comic, and damaged, and ambitious, and idealistic, and foolish characters they've already shown themselves to be by the time they get into this scene.
It's almost as if they, themselves, are also victims of this ritual they're part of.
So there's a kind of subtlety that you can't pick up on just watching the clip, that is present for any audience member.
But I wanted you to see that this scene, also in part, again, to show you how powerfully Renoir's camera is able to create emotional and intellectual responses in the audience, in a way that is, in some sense, diametrically opposed to what we might call expressionist directors like Hitchcock or Eisenstein might do.
By manipulating our attitudes, by constant, by very rapid editing, and by giving us high or low angle shots that control our emotional response because of the way we're seeing things.
Renoir's strategies can be just as powerful in terms of emotional and moral reaction, but Renoir's strategy is quite different.
Present the evidence, in some sense, to the audience.
Keep the camera at a level that allows the full action to be taken in.
I hope you realize that there were relatively few cuts in that scene, and in most of Renoir's work the number of edits is relatively small compared to the number of scenes that are extended, the amount of long takes that we have.
And there are certain moments in the film where he will vary this strategy.
For example, and you might want to watch for this in tonight's film, there's one moment in tonight's film, an interview between two of the central characters, the German prison warden played by Erik Von Stroheim, and the French aristocrat named du Boeldieu, played by an actor named de Fresnay.
And in this scene between the German and the Frenchman, one of the things we recognize is, because they belong to the aristocratic class, they have a lot in common.
They speak English, for example, because they both know English, even though one's a German and one's a Frenchman.
And there is a sense in the film, a very powerful one that's established, that they have, in some sense, a greater connection to each other than they have with other soldiers in their own armies.
And the film meditates on these distinctions, and especially on this conundrum of social class, in rather a complex way as I'll suggest.
Well, in the very first scene between Boeldieu and Rauffenstein, the German and the Frenchman, Renoir uses a fairly standard procedure, much more common to other directors, it's sometimes called a shot, counter shot style.
it's especially common on American television, but it's common in the movies too, in which, when you see characters in conversation, you'll get relatively rapid cutting.
And you look at one face, then the camera will shift, and you look back at the other face.
They'll be a cut, and you'll look-- right?
And you'll constantly shift back and forth, this shot, counter shot style.
But he, Renoir uses this kind of style rarely, because first of all, it involves the idea of cutting.
It involves the idea of separation.
Why use shot/ counter shot, Renoir would say, when I can move my camera like this and show the two characters together?
And move it around in another way, and show the back of one character's head, and the way the other is reacting.
In other words, if the camera's part of the action, a fluid part of the action, I can show you more about character, Renoir would say.
But in that particular scene, he doesn't do it in this first extended interview.
It lasts about two minutes and 45 seconds, and it uses 21 shots.
21 separate shots in two minutes and 45 seconds.
For Renoir that's a tremendous number of edits.
and it why would he do it there?
Because it's very unusual for him to use this shot, counter shot style in the film.
Why would he do it?
He wants to emphasize their separation, their isolation from each other, their distance from each other.
And there is a later see in the film, near the very end of the film, again between Boeldieu and another character.
But this one between the Boeldieu and Marechal, the French working class figure who's on the French side, just before there's an attempted escape.
And that scene is almost as long, two minutes and 20 seconds long, but that scene only has three separate edits in it.
And you might compare it.
The farewell scene is a very obvious one, and you'll recognize it when you see it.
and, of course, the reason is that he wants to emphasize, insofar as it's possible, that they are together.
He wants to minimize the sense of isolation in that sequence.
Again, I mention this because I want you to become as attentive as possible, without destroying your enjoyment of the film, to the camera's way of behaving.
to summarize what I'm trying to say about the way the camera works in Renoir, I want to remind you that, even in his darkest films, his camera work carries an undercurrent of excitement, even sometimes a kind of joy at the sheer particularness, the particularity of the world.
you can sense his awareness of human ambiguity and his wonder at the texture of the visible world all through his films of his so-called French period.
And especially in the film you're going to see tonight, which is in many ways in environments that are very unpoetic right?
The entire film takes place in a series of prison camps.
The entire film is about these French prisoners of war.
There is an introduction or prologue before they're captured, and then after they're captured we see them in a series of prison camps.
it's quite an amazing experience, in some ways, to watch Renoir's camera discover the complexity of a world that seems, on the surface, to be so unpromising, and so uninteresting, so lacking in texture.
we can think of Renoir's camera as shy, timid in a sense.
Capable of embarrassment, even.
There's one moment in the Rules of the Game which partly involves a force dimension, so that when all the guests in the great estate or castle go to sleep at night.
They sort of come out of their rooms, and start changing bed partners and things like that.
they have various liaisons.
And there's a moment when the camera is in this long hallway, and one of the lovers comes out to meet another lover, and it looks as if the camera actually backs away in embarrassment, and looks away as if it says, oh, that's inappropriate for me to be looking at.
The camera is humanized in a very subtle, quiet way all the way through Renoir's work.
So we think of his camera as shy, as embarrassed, reluctant to overhear, or to intrude.
There are even moments when it is distracted, or drawn toward an apparently irrelevant element.
Times when the camera hesitates, or even appears to change its mind.
An uncertain camera, then, always adjusting to a world that is, itself, always shifting and changing.
And of course, that's the deepest argument for why the camera is so restless.
remember, it's not obtrusive.
it's not like to feel the camera swinging wildly, not at all.
You have to pay attention, sometimes you won't even be aware of the fact of how quietly and simply the camera-- but it's as if the camera is always making adjustments.
As if it's always alive to the changing character of what it's looking at, and that very aliveness reminds you of how complex reality is, because reality itself is always shifting and changing.
This describes, I think, something of what is meant by Renoir's realism.
But there's a kind of lyric element, as I've mentioned, even a kind of tranquility, or pleasure, or joy.
Celebratory impulse in his camera's gaze that qualifies, or sweeteners, his realism of space and character, and we need to be aware of that too.
One way to crystallize this is to suggest that once we become attuned to the camera's forms of attention-- that's a good phrase, the camera's forms of attention.
There's a wonderful book about poetry called forms of attention.
Lyric poetry is what constitutes a special kind of attention.
We can say that Renoir's camera is engaged constantly in these forms of attention.
Once we become aware of that, of the camera's interest in that, we become aware of a quiet continuous struggle, a loving contest almost, between the multiple points of interest in the scene, and the camera's reluctant gently hesitant choices.
The menu of things to see and hear, Renoir's films keep telling us, is too rich to be fully captured, even by the most eye.
And that's what I mean when I say that visual style can be understood as a form of moral style.
I want to say a few more words about Grand Illusion, itself, to help frame your experience of it.
It's not a complex film, but it's a deep and beautiful one, and I think you'll all find it very memorable.
First a word about the actors.
or maybe I should say one other word about Renoir himself, because it bears on his actors.
As I implied, or I think, suggested this afternoon, one of the most distinctive features of Renoir's practice-- if you think about the way is camera behaves, you can see how logically connected these elements are-- was that he was an improvisational director.
He collaborated with his actors, with this performers.
He didn't come in with an absolutely fixed script.
he's like the anti-Hitchcock.
Hitchcock has solved all his problems before he's come in to do is shooting, Renoir has just begun the process of making his movie.
he has an idea of the subject matter, He knows what he's doing.
I don't mean that he's working completely blind, but he allows discovery to take place in the course of the making of his movies, and he's very deeply involved in collaborating with this performers.
So that, for example, when he had already begun working on the movie-- I don't think they'd begun shooting it yet-- but they were very far along in creating the movie, and in casting the movie, when they discovered that Erich Von Strohein, who was that already by the time this film was made, a very well known actor and director.
Much, in fact, incredibly admired director --both an actor and a director, agreed to join the film.
And when he agreed to join the film, he and Renoir began to collaborate on expanding the role that Von Strohein plays.
And you'll understand what an immensely helpful contribution to, what could we call it, to the ecology of the movie, to the sense of the movie Von Strohein made, because he's one of the dominant characters.
if we try to imagine Grand Illusion with a much reduced role for the Von Strohein character, it's a much lesser film.
So you can recognize what an extraordinary contribution Von Strohein made.
Now, he not only made contributions to questions about the dialogue, what the character would say, expanded the role, but even made suggestions about how the character would look.
How the character would be costumed.
I think it was Von Strohein who emphasized, even more fully than Renoir had originally intended, that the Von Strohein character is a war casualty.
That he has ram rod up is back, he has to wear gloves because his hands are horribly burned.
He's a cripple, and that's why he's now running prison camps.
Because he's not useful for anything else, right?
And there's the sense that he's a damaged, wounded figure even from the very beginning of the film, when we first meet him.
So that's important in itself, but there's a second thing about Von Strohein that I want to emphasize to you.
it shows you how subtly Renoir was aware of how he could utilize all the elements available to him as a filmmaker.
Von Strohein was especially well known as an actor, for roles in which he would play-- this was a special true in roles he played in the United States-- in which he would play a dangerous, scary martinet.
There were even hints of sadomasochism about the role sometimes.
So he would play brutal prison camp wardens, right?
brutal police types, militarist types, violent husbands.
And, in fact, he was understood, also, as a villain.
He often played roles that were recognizably evil roles.
He came to be known as the man you love to hate.
That was like a sort of motto that was attached to his name.
And, when you want your Grand Illusion, one of the things I hope you'll think about is the complex way in which that inherited idea of the Von Strohein character is complicated, undermined, and humanized.
Because, in many ways, he still is the villain in the piece, but he's a humanized villain.
he becomes a complex character, as much a victim of history as any of the other characters in the film.
So there's a sense in which, what Renoir and Von Strohein, together, are doing is taking the persona that Von Strohein had established, and undercutting it, undermining it in some slight way, creating a much greater resonance for the character, exactly because of the ways in which he first that appears to fulfill the expectations that he's an evil villain, and then as the film goes on, this sense is undermined, complicated, and you realize he's a human being with powers of generosity, although also certain limitations.
he's a German, he's prejudiced, he doesn't really like the French, especially doesn't like working class people, most especially doesn't like Jews.
he's an imperfect character, but you see this he's deeply humanized.
And something of the same kind of thing happens in the film's treatment of the Jean Gabin character.
He's the character who plays Marechal, the actor who plays Marechal, the working class Frenchman who, along with de Boeldieu and the Jew played by Dalio, a character called Rosenthal.
These three are the central French characters, and we see them moving from prison camp to prison camp through the film.
that group is the central group of the film.
We follow them as the film goes on.
The Gabin character too, was, in many ways, the most famous actor, if not in Europe, certainly and France of the late '20s and '30s.
major, major figure.
and he was especially associated with working class roles.
, And, most especially associated with roles-- he was like an early Jack Nicholson.
An even more powerful version of it, because he became famous for these moments in the films when he would go nuts, when he would have a fit, when he would start screaming, or he would lose it.
He would rant or rave.
In some films he was in he would have a climactic scene-- this may be, in fact, where the stereotype began-- he would have a climactic scene.
Often he would be a suicidal proletarian, a suicidal worker up against the forces of capitalism, or the forces of the police.
The Gabin character had frequent roles like this, in which he would essentially lose all rational control, and rant, and rave, and scream, sometimes die.
he was so famous for this, and this was such a fundamental part of his persona, that he had written into most of his contracts that he had to have a scene which he did this.
He had to have at least one scene in the film in which he sort of went Cucamonga, in which he lost it.
Now, one of the most subtle things about this film is, there is such a moment in this film.
Watch for it, because it's so quiet compared to what you might expect, that you might miss it.
But it's really there.
And in fact, the way in which it both fulfills, but also undermines, our expectations about this character, again, tells us something about the way in which Renoir and his collaborating actors were able to create immensely nuanced meanings, possibilities, out on the raw material, not only of the story of the film, and of the setting of the film, but even the past history of the performers.
I want to say just a couple of things about some of the dominant themes in the film to help organize your experience of it.
And I'm borrowing here, or deeply influenced here, by two critics especially, or perhaps I should say parenthetically.
You should probably assume that almost anything I say has been said by someone else at some time in the past.
the job of a teacher, as I conceive it, is not to come up with totally original arguments every time, because no one can be so original as to say everything that's wonderful about particular texts, or about a whole phenomenon like the movies.
So a good teacher draws on that.
When I really am drawing on a prior scholar, who I know has shaped the arguments I'm making, or has given me the categories I'm using, I mention them by name.
But, of course, I've been teaching for so long that much of this must not be unconsciously absorbed into my psyche, and I don't always know this, so I'm not saying that those moments where I haven't made this acknowledgement are free of such influence.
Quite the opposite.
But I do want to mention two scholars who have especially influenced me on Renoir, because virtually all the things I've told you about Renoir, though I've made the ideas my own, come from these.
One is a Frenchman named Alexander Sesonske.
And he wrote a wonderful book, I think a several volume book, on Renoir that taught me a tremendous amount, especially about his French period.
And the other is the, now California scholar, Leo Braudy, one of whose essays I've asked you to read in this course.
He's written a wonderful book on Renoir.
And, although I have read the book, I learned more about Renoir from Braudy in conversation, because he was my colleague many years ago when we were teaching together in the English department at Yale, and he's one of the people who turned me into a person interested and movies, interested in film.
So, Leo Braudy, B-R-A-U-D-Y, Alexander Sesonske, S-E-S-O-N-S-K-E, the two principal scholars and critics behind what I've been saying, both this afternoon and this evening.
Dominant themes.
One.
The prison camp as a microcosm.
There's a wonderful passage, at one point in the movie, where one of the character's said "everyone would die of the disease of his class, if the war did not reconcile all microbes".
think about the implications of that.
the idea is that one of the things that makes this prison camp such a fascinating place is that it is, in some sense, a microcosm of the larger world.
and especially if you think of the prison camp beyond just the grouping of the French soldiers, the French prisoners who we follow primarily, what you can recognize if you back away, because we are told about, or we meet, and we see briefly some of the other prisoners.
There are Russian prisoners and there's other kinds of prisoners as well.
There's a sense that the prison camp is a kind of microcosm, at least of Europe, if not of the world as a whole.
And, more specifically, the way in which the prison camp is a microcosm for the social striations that organize outer society.
There's a working class, there seems to be a middle class, you'll notice that there's a scholar who has a copy of Pindar, the great ancient poet, who treasures his work on Pindar-- he's one of my favorite minor characters in the film because of his commitment to literature despite the war, and despite the miseries of prison camp.
And what you can see is, in a certain sense, that Renoir is delighted by the idea that a kind of microcosm is created.
And if you watch the variations amongst the Frenchmen, and again, if you think about that fragment of a scene I showed you this afternoon, of these characters at dinner table in the in the prison camp, you'll begin to see more fully how powerfully and interestingly, complexly, he differentiates his characters from each other.
Now That fragment that you saw, I hope you realized one of the characters was an actor.
It becomes clearer if you watch the whole scene, but he's a very histrionic and theatrical character, he bursts into song all the time, right?
And then there's the aristocrat who's looking down his monocle very, very austerely and his lip seems curled in a permanent gesture of condescension.
And so forth.
Watch how richly, how complexly, the characters are distinguished from each other by social class, by profession, and so forth.
I don't want to romanticize or idealize this argument, because I'm not suggesting that when he says it's a microcosm of the world, he wants to celebrate that microcosm, although he's interested in it.
Renoir is also aware of the limitations.
it's not as if he's suggesting that this is a perfect community at all.
One reason is, they're forced together, but they're being forced together makes them share, forces them to come out of their limitations in certain ways, to overcome certain of their prejudices, although night never fully.
But what it also does is reveal them.
And we can and we can see how both processes-- the process of adjusting to people different from yourself, and maybe overcoming some of your class or racial prejudices-- is a part of the experience that the characters in the film have, but another part of the experience is that they come up against the differences between them, some of which can never be breached.
Let me mention one scene which dramatizes this ambiguity very deeply.
There's one moment in the film, very powerful moment I think, in which the character played by Gabin, the Marechal character is wounded in a early part of the film.
in fact, in the dinner scene that I showed you this afternoon, you'll see that, at a certain point, his comrade sitting at the table cuts his meat for him because his hand is injured.
And, I don't know if it's in this scene or subsequent scene, there's a moment when another character in the film washes Marechal's feet.
Why is foot washing instantly a symbolic thing?
Who knows the answer?
Cause, like, Mary washed Jesus' feet
All right.
There's a scene in the Bible, in the 12th chapter of John, in which Mary or Martha, I can't remember which, washes Christ's feet.
And then, in the very next chapter, at The Last Supper Christ, himself, washes the feet of his disciples.
And in fact, the washing of feet is, in Catholic and in certain other Christian rituals, an actual official ritual.
It used to be much more common than it is today, but it's still done in certain Catholic countries at the advent of Holy Week.
so it's a deeply symbolic act in which Christ, himself, is washing the feet of his disciples, and the message he gives is everyone, you should now do this to your Fellows.
The implication being no one is better than anyone else.
a radically democratic invasion, in some sense, of God's love.
And where has this practice been recently, very dramatically, and publicly revived?
Who's been doing it?
The Pope, the new pope.
Haven't you read about this?
The new pope has been doing this, washing the feet of parishioners, a symbolic gesture in which God's emissary on earth is behaving with the humility that Jesus Christ, himself would.
So the foot washing scene in our film has this sort of resonance to it, even though there's no openly religious reference.
but it has a similar kind of significance.
And they are carrying on a kind of conversation as the feet are being washed.
And what you feel is, boy, what a moving scene.
the man who's doing the washing doesn't feel that he's being condescended to, or that he's being diminished in any way by doing it.
The man whose feet are being washed is not embarrassed to have it happen.
What a scene of solidarity and community.
And you could wax very poetic about what a scene of community, and yet, watch how the scene ends.
There's a moment at which, as he's drying his feet, Marechal asks the foot washer, what do you do for a living?
at some point during their conversation he says I am a cadastre.
What that means is something like city planning.
He's a city planner.
And so the Marechal character hears this and they go on, at the very end of the scene, Marechal turns to his foot washer and he says, what does that mean, cadastre?
The implication being that they're vocabularies aren't even the same.
what it dramatizes is the distance between them.
He's been sitting there having this conversation with him, but he didn't even say, gee I have no idea what that word means, I certainly have no idea what you do for a living.
It's a way of reminding us, yes they've come together, yes they're sharing space, yes they're doing something that dramatizes human solidarity, and yet also how separated they are.
Again and again in the film we have this sort of double experience, in which we see characters make contact with each other, or even sacrifice for each other, and at the same time we're aware of their differences.
Watch for this.
It's very powerful and moving.
It's a form of moral realism, in which the film refuses to simplify what it wants to say about human experience.
Very powerful example of this in the relationship between the Jew, Rosenthal, who's proud of his wealth, he's the heir of a department store fortune.
but because he's a Jew, and because he's nouveau rich, he can never be part of the aristocracy, his relation to the Marechal character's very complex.
Marechal, as I mentioned this afternoon, retains a kind of French working class characteristic antisemitism.
And there are moments when you can feel the hostility between them.
When the two of them finally try to make an escape from the prison, and they actually are away from one of the prisons for a while, and you see the Rosenthal character hurts his foot.
And there's a moment in which it looks like, gee, if the Marechal character stays with him, he's putting himself in danger as well.
And they get into a fight.
And Rosenthal says, go away, leave me alone!
And Marechal says, all right I will!
it's a very wonderful scene, because they start singing at each other.
it's a form of singing hostility.
So there's a comic dimension to it.
it's really rich.
But what is also being dramatized there are the limitations of the characters.
Marechal's antisemitism rises, comes to the surface.
Their hostility toward each other comes to the surface at that moment, although of course, they also overcome it.
And there are many other such moments in the film.
The relationship between de Boeldieu and his German counterpart, the aristocratic German, is also a version of this kind of thing.
as I've already implied, a second central theme of the film has to do with what we might call barriers and boundaries.
There's the barrier of social class, which I've mentioned already.
There's the barrier of language and culture, right?
The Germans and the French are at war.
The Russians don't speak the same language as the Germans, the English don't speak the same language as the French.
again and again what separates people is dramatized for us, although there's also a longing the film to transcend these boundaries, they can't always be transcended.
Sometimes these boundaries are shown to be foolish and trivial, or at best, unhelpful.
there are many examples in the film, both of language and of geography, as well as of social class, but let me mention just two of them, There's one wonderful moment in the film.
The French prisoners have been digging a tunnel, an escape tunnel.
And they've almost completed the tunnel when the Germans come and say, all right we're moving you out of your barracks and we're putting these English prisoners in.
and so the French are very unhappy about this, because they've almost completed their tunnel.
So, at great danger to himself, the Marechal character breaks ranks, does a kind of trick where he drops a suitcase, I think, and he bends over to try to pick it up to give them an excuse to talk to the English prisoners who are going to change places with him.
He wants to tell me English prisoners, there's a m you could escape.
But it turns out he's not able to give them the message, because the English don't understand French and he can't speak English.
So the English Prisoners end up going into the-- without ever knowing that there's a tunnel waiting for them that's almost ready to be completed.
An example of the barrier, the way language separates people, that in many ways can be said to be frustrating.
One more example.
At the very end of the film, there's a moment when the two characters, Marechal and Rosenthal are making their escape across a great field, a great white field of snow.
And they're running across the field, and the camera backs up, and we see them being watched by German soldiers.
One of the soldiers picks up his rifle, he says they're making an escape, and starts to aim at them.
Then, all of a sudden, he stops.
watch for that how it happens.
I think his partner says, wait a minute, hold off.
Why doesn't he shoot?
And then we see Marechal and Rosenthal disappear across an invisible boundary, and the German says, well they've crossed the border.
They're no longer in Germany, I can't shoot them, they're now in Belgium.
But the fact is there's no boundary there, it's all snow.
It's a snowy field.
How arbitrary that boundary is.
But again and again the film we have this theme of barriers and boundaries.
What separates us?
And also, one last thing about the question of social class.
Maybe the most disturbing and powerful insight that the film has about social class, one that Americans tend to resist.
But I encourage you not to resist it.
Those of you not from the United States will, perhaps, understand this more easily, although Americans need to understand it, is the extent to which one's social class shapes one's identity.
That is to say, one of the things we feel in this film is that de Boeldieu's nature is a function of his having been raised as an aristocrat.
There are wonderful scenes between de Boeldieu and Marechal, where Marechal says to him, there's a wall between us.
And you're so undemonstrative.
Marechal doesn't trust him because he belongs to a different class, even though, of course, they're Frenchmen after all.
And in the end, nationality trumps class in this film, as you'll see.
But in this sort of distinction or difference between Marechal and de Boeldieu, we have another expression of this sense, of this idea, of this recognition that one's personality, itself, what one likes and dislikes, ones fundamental nature is in part shaped by the class into which one is born.
We're not born utterly free.
We don't have absolute choice in what we become.
Our characters are shaped, controlled, by the circumstances in which we live, and the film profoundly understands that.
It understands our desire to breach those limitations, to go beyond them, and it celebrates our impulse to do so.
But it also recognizes the degree to which we remain confined within personalities that are partly controlled or shaped by forces far beyond our individual control, and that are especially located in the social class to which we belong.
Finally, as I mentioned this afternoon, the theme of historical transition is at the heart of this remarkable film.
And you'll understand very clearly what is going on there.
Again, dramatized even more complexly and fully in Rules of the Game, it's a central topic here.
aristocratic traditions are giving way to modernity in this film.
And we're aware of them in all kinds of ways.
Even in the nostalgic way in which de Boeldieu and Rauffenstein carry on their conversations.
they have experiences in common because they belong to the aristocracy, a class that transcended, in some sense, at least in part, transcended national origins.
That aristocratic dispensation is disappearing.
In fact, this first World War completely obliterated it, and Renoir's not the only one who said this.
apart from scholarship, there are a number of wonderful novels, for example, that deal with the transformation of European society because of the First World War, and Renoir's film is one such text.
Renoir's maturity.
This is another way of talking about the complexity, or the richness of his films.
I simply want to remind you, at the end, that one way to think about what Renoir is doing is to recognize that his visual style is, in some sense, an embodiment of the ambition not to simplify.
To be as fluid and as attentive to the world as it's possible to be.
And one could say that that visual, that commitment to the way the camera acts toward the world, is a version of a larger kind of maturity or complexity that's revealed in the content of his work, as well.
Take the characters.
I've already mentioned the key point here.
The way in which the characters are both flawed, full of weaknesses, but also attractive in certain ways.
The way in which a Rosenthal is marred by his pride in his family's wealth.
The way in which the Gabin character is marred by his antisemitism and his lack of education.
The way in which the de Boeldieu character is damaged, or limited, by his aristocratic reserve, and his impulse toward a kind of condescending superiority, which you could see articulated in the very first moment you see him, at the very beginning of the film, where he's tsk tsking about the incompetence of his own troops.
So there's a complexity of character that reflects the maturity of the film's vision of life, and there's a complexity in the story, because think of it for a minute.
This is a war story, but it's a war story without battles.
Some people have said it's the happiest war story they ever saw.
One way of organizing the film is to see it as a series of meals, in which character sit down at table and share convivial thoughts.
It's not a sufficient way of thinking about it, but it's a partial way of thinking about it.
so, although it's as film about war, about the consequences of war, it doesn't show shooting.
And in fact, although it's a war story, it's about most deeply, human community and human solidarity, and about historical transition.
Well, one way to crystallize all of this, or to summarize it, is to remind you of the complexity of the title-- Grand Illusion.
the title seems to me to have at least four separate implications, or four separate ways of understanding.
It's a measure of what I've called multiplicity, or complexity.
what would the title mean?
What is the grand illusion to which the title refers?
Well, one is surely this.
Remember, the film was made on the eve of World War II even though it is ostensibly set in World War I.
And, in fact, it was recognized as an anti-war film.
It was banned in Germany and in Italy.
President Roosevelt in the United States said anyone who loves freedom should see this film.
I think he was right.
The Title.
One of the things about that war, that first World War, some of you will know, was it was identified especially by the American President Woodrow Wilson as the war to end wars.
So one deep grand, but horrible, illusion is that this is the war to end wars.
That there won't be anymore wars after this one.
A second possible meaning of the title is this.
one of the central events in the film, I haven't mentioned yet, and it's, maybe, the most memorable interlude in the film is, after Marechal and Rosenthal make their escape from one of the prison camps, they're on the run.
They're in Germany, and it's the middle of winter.
And they managed to find safety in the home of a German woman whose husband is away at the war, perhaps dead.
And she is there trying to manage a farm with her young daughter Elsa, and Marechal and Rosenthal spend the winter with them, and it's an idle in the winter.
It's very moving.
And, again, the barriers of language are present there.
At the very end of their idle in the woods, we have Marechal say to the child, --I called her Elsa, I think it's Lottie.
He looks at the girl and, in bad German, he says [GERMAN].
He says blue eyes.
But he's very proud of himself for having been able to articulate two German words.
But the fact that he's done that is a mark of his having reached out.
And the implication, more than the implication, is he becomes the woman's lover.
while they are what while they spend the winter in this way-- and the German woman, at a certain point, when they first take refuge in her place, a German soldier searching for these guys, comes to the door, comes to the window and speaks to the window.
And the widow protects them.
So a second grand illusion may very well be, look, this is a very lovely story, but how believable is it?
French soldiers on the run from a prison camp find a farmhouse in which a German widow takes them in and protects them from the German soldiers.
It's a nice story, but is it a grand illu-- if it is an illusion, maybe it's a grand one, a wonderful one.
Not just a large one, but a valuable one.
So that's a second way in which the title resonates for me.
The third grand illusion has to do with the community of the prisoners, the solidarity and community that's dramatized in the film in the end.
The de Boeldieu character sacrifices himself for the good of the other, something that we might find surprising at first, given his distance from the other characters.
and there is a sense all the way through the film, that a kind of community or solidarity has been established amongst the men, especially amongst the Frenchmen.
And especially that a kind of friendship between Rosenthal and Marechal has been forged.
Well, again, maybe that's a grand illusion.
The idea that you can transcend social class, that you can transcend the ethnic prejudices into which you were born, and with which you've been brainwashed.
So that's another kind of potential grand illusion.
And then finally, the fourth example I would give.
the fourth nuance that one could add to the title, the grand illusion of movies themselves.
What are movies but grand illusions, right?
They're two dimensional, not three dimensional, but when we watch them we feel we've entered into a real world.
The grand illusion of movies themselves is part of what the title alludes to.
Well, something of the complexity and maturity of Renoir's vision of experience, sense of human character, and sense of history is embodied, then, in the complexity of the title of his movie.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Our topic today is the influential movement called Italian neorealism, a movement of filmmakers that arose in Italy at the end of the Second World War and had-- although it was a very short lived movement, lasted only six or seven years before it kind of dissolved of its own way.
In part, incidentally, because although the films were very successful with critics and several of the neorealist films won Academy Awards as the best foreign film of the year in 1946 and then again in 1948 when the film you're going to see tonight, Bicycle Thieves, won the reward.
Despite that acclaim and even global success, the movement sort of petered out, although its influence as I've suggested was very great.
One reason the movement petered out was that forces in Italian society, chambers of commerce types, people who are interested in encouraging the rebuilding of the devastated cities of Italy, a function, a consequence of the Second World War, those folks began to feel that the messages coming from neorealist films were too negative and were discouraging foreign investors.
So although it was a very remarkable movement, it lasted only, as I say, a short time.
But its influence ramified very, very, very broadly.
We can trace its influence to virtually every national cinema that came after it, and there are certain very particular influences that one can trace from the neorealist movement to certain American directors of film noir in the '40s and early '50s, to certain very interesting and historically curious forms of television that emerged in the early years of American television that were deeply imitative of and influenced by Italian neorealist movies.
The man that many would identify as the greatest of all Indian directors, Satyajit Ray, was directly inspired to make movies by The Bicycle Thief, and he made a series of three films that are essentially a form of Italian neorealism moved to India, a wonderful trilogy of films, some of them actually about the untouchable classes in India, called The Apu Trilogy.
And they remain among the most treasured forms of Indian film.
The Italian neorealist had a profound influence on other European cinemas as well, and most particularly can be said to be the originating movement behind the French Nouvelle Vague, The New Wave movement, that in France that emerges at the end of the '50s and the early '60s and it involved such now recognized, such directors who are now recognized as major figures in the history of film as Jean-Luc Godard and Francois Truffaut.
So it was although a short-lived movement, profoundly influential one, and it produced some enduring, and beautiful, and I guess I would say immortal films.
The most famous and perhaps in some ways the most perfect of those films is the one you're going to see tonight, Bicycle Thieves.
And I'll talk a bit about that film in my introduction to the lecture this evening.
I want to begin, though, by giving you a sense of how neorealism feels, a kind of distilled version, or a distilled embodiment of many of the deepest principles of neorealism, and in order to do that, I want to show you a clip from the very beginning of the film you're going to see tonight.
So you'll see this beginning twice.
Hopefully, the second time you see it, you'll be alive, more alive to some of its implications.
So here is the very opening of Bicycle Thieves.
And one of the things I want to ask you to watch for is first how much information, how much visual information, dramatic information, becomes available to you under the titles, and then how fluently and fluidly the film moves out of the title sequence into a kind of continuation of the drama that's articulated in that title sequence.
On the surface, this little anecdote at the very beginning of the film seems very modest and one might even think of it as simply expository, that is providing information to the audience so that they can get on with the story.
As I hope you'll recognize, and as I'll try to show you briefly after I've shown you the clip, much more is going on than that, and it's a particularly wonderful example of the way in which the best neorealist films generate a kind of texture, a kind of density, what I've sometimes called a multiplicity that we have suggested in this course is one of the fundamental ways in which you can distinguish between entertainment and serious entertainment or between entertainment and art.
When items and objects begin to have this multiplicity, when they begin to serve multiple functions, not just a single function, you're in the presence of something not only complex, but something that acknowledges the complexity of the world in some respects.
And this opening sequence of Bicycle Thieves does this with remarkable economy.
So here's the beginning of the film.
[SPEAKING ITALIAN]
Great.
Can you freeze it for a second?
One thing I should mention while you're watching this, to interrupt the sequence for a moment.
The activity that you see taking place under the titles is supposed to be confusing at first.
Although, after a while it begins to clarify.
Perhaps you're not fully clear what is going on until the titles disappear and you see these men assembling around that staircase there where a man with a list is speaking to them.
What is going on here is what in the United States in the '40s and '50s on the docks in New York, they called the shape up.
And a shape up is a place where men who are eager for work sort of come and hang around and wait for the boss, the foreman, or the hiring boss to pick them out of the crowd.
Shape ups are always unpleasant experiences because there are many people who are excluded, and the people who are chosen are often-- this was especially true on the New York waterfront in the days which I know about because my father was a longshoreman for a while.
It was a particularly bad thing in the New York waterfront because you had to know somebody to get work.
I don't know if that's an implication here or not.
Although, you'll see as the men complain that some of them have that suspicion.
In any case, what's going on here is what is called a shape up.
Men are gathered around, men who are unemployed, looking for work.
[SPEAKING ITALIAN]
I wish I could let the scene run on.
But I hope you, I hope you-- I can tell some of you are responding to what you saw there.
Make some observations about what you saw there.
What in that introduction what do we learn as an audience?
You have to be fast.
I only have a few minutes here.
Yes?
Is there a way for [INAUDIBLE]?
All right.
One thing we know is that there's a lot of unemployment in this country, a lot, right?
And there's a lot of discontent.
And then the minute he says, I don't have a bicycle, virtually everyone else in the crowd says, I have a bicycle.
Everyone is desperate for a job, not just Ricci.
That's right.
So we learn how grateful he is, but also how troubled he is.
What's the evidence in the film that he is really very unhappy, very, very troubled by the fact that he doesn't have a bicycle to get this job after so long?
What's the most dramatic piece of evidence we see that he's distracted by his anxiety?
Help?
He doesn't help her with the water
He doesn't help his wife with the water.
Yes.
It's a very dramatic scene, isn't it?
She's walking behind him, carrying these two heavy-- and he doesn't ignore because he's a nasty patriarch who thinks that women should carry water, he's so distracted he doesn't notice it.
And, in fact, he walks down that little incline and looks back and sees her struggling, goes back, and as soon as he recognizes it, he takes one from her.
And it's significant, isn't it, that he takes only one from her.
They're really partners.
And what else is significant?
She doesn't yell at him.
She doesn't say, my god, Ricci, what kind of a pig are you, making me carry all this stuff by myself?
because she shares his anxiety and his concern, doesn't she?
So on the surface, the scenes certainly dramatizes Ricci's desperate need for a job and the fact that he's one of many people in a similar situation.
But what does it tell us about his marriage?
On the basis of these details, what you just saw in this beginning, do you think he has a good marriage, that he has a good relationship with his wife, that it's a mutually trusting relationship?
Of course, of course, right?
And how do we know that?
We know that from that detail.
Now, the detail does not advance the plot in the sense of telling us anything further about his need for the bicycle or his unemployment, but it does reveal to us something about their marriage, about the kind of man he is, about the kind of woman she is.
So that it's a moment in which his anxiety about his job is maybe the primary thing we see, but what is also dramatized for us is the nature of their relationship.
And it's dramatized for us without anyone coming out and saying, let me show you a mutually engaged couple who depend on each other and who survive in part because they share each other's troubles.
People whose respect for each other is so total that even when one of them commits a really mean or at least an unpleasant act by ignoring his wife struggles, she doesn't even think to complain to him about it, because she shares his anxiety.
It's an example of what I mean by a moment of multiplicity in which something about their relationship, something about their marriage emerges even as we are also learning something about the economic circumstances in which these characters live.
Say a little bit more about the scene you saw.
How about the place where she's drawing water?
What about that?
Why would-- They're in a city.
They're walking, in fact, to a high rise apartment house in which they live.
What does it mean that she's carrying two buckets of water to an apartment and then she's going to climb up four flights or three flights to get to their apartment?
What does that tell us about Mussolini's Italy?
There's no running water
There's no running water in the building, in the high rise apartments.
And not just that, is the water precious?
How do you know it is?
She doesn't leave it there
Say?
She doesn't leave it there
Yes.
But what else?
Where do they get the water?
Is the water-- do they go to a pop in the open square?
No.
It's in a barbed wire compound.
Again, these details may not register instantly as deeply significant, but think what they tell us about the conditions of urban life in Rome in 1947 or 1948 when the film is set.
And you can even go further.
Look more closely at the setting.
Did anything grow there?
It looked like they were walking across a wasteland of mud, except for these high rise tenement buildings.
So we have high rise tenement buildings set down in a kind of desert, a place in-- buildings that don't even have running water in which the main water pump has to be protected by barbed wire.
So that it's almost as if in this brief introduction to the film, the whole socioeconomic environment of the story has been established for us without anyone giving us lectures about it or calling our attention to it.
That's what I mean by multiplicity, when the details of the film serve character and serve plot and serve our sense of the environment.
So it's a rich moment.
It's a moment, it's a moment of art.
It's a moment of complexity.
It's a moment of human truthfulness that resonates for us as the film goes on.
Well, there are many, many moments like this.
Virtually every moment in Bicycle Thieves has this kind of density or texture, has this kind of implication built into the particular details.
I want to very quickly set the kind of the context for-- I've already implied the context for the neorealist movement.
The basic context is the end of World War II.
Most historians trace the beginnings of the neorealist movement to films that were made under Mussolini while the war was still going on, and some people actually date the very beginning of the movement to a film made in 1942 by Luchino Visconti called Obsession.
It's actually an adaptation of a very popular American pulp novel called The Postman Always Rings Twice.
It's been made twice into an English language American movie, most recently starring Jack Nicholson.
But the Italian version may be the most important and interesting of the films made from the book.
The book itself is a rather tawdry noirish story about the owner of a diner who's an ugly middle-aged pug who's married to a beautiful and somewhat younger woman who very unhappy with her husband begins an affair with a drifter who comes by, and ultimately encourages the drifter to kill her husband.
And it's actually a rather heavy-handed male chauvinist fantasy about, partly about, how cunning and dangerous women are.
The Italian version is much more interesting, in part because the working class dimensions of the story are given much greater emphasis in Luchino Visconti's version.
And the film partly becomes a kind of meditation on the fate of undocumented workers and nonunion workers in Italian society.
In any case, the emphasis on ordinary people and the emphasis on socioeconomic circumstances in that film by Visconti came to be defining features, among the defining features of Italian neorealist filmmaking.
But the movement really doesn't get going until after the war.
And the context of the war is very important.
We need to recognize that the film is really set in a devastated society, a society in which, I'm talking now about all of Europe after the Second World War, unemployment reached 25% or 30% higher in some societies.
There was economic turmoil in virtually every European society, profound terrible pressures from black market activities of all kinds.
The war was so devastating that something on the order of 35% of all the permanent dwellings in Western Europe were destroyed during the war.
25% of the Polish population was killed.
A tremendous proportion, virtually every-- Well, a very large number of Italian cities were damaged very badly, including, of course, Rome itself.
And these abject and terrifying economic conditions were, of course, in Italy doubled by the problem that Italy was on the losing side of the war and therefore faced even more difficulties in the recovery than a culture like the French culture or English society.
So the devastation of World War II and the turmoil and the dislocation caused by World War II is the fundamental fact.
What is still operative in Italian society, what institutions still work, is one of the implicit questions that the film explores.
And the film suggests that a lot of the official institutions, like the police and especially the religious institutions, have broken down and are not really helping people very much, as you'll see when you watch tonight's film.
But there are other forms of help and aid that people get, mostly in some sense from their, the film suggests, from their local communities and from their immediate families.
Well, to this basic sort of setting of devastation and turmoil and economic uncertainty, we need to add two other contextual factors, both of them having to do with the film traditions against which neorealism set itself, against which it understood itself to be in a kind of profound argument.
The first tradition was the tradition of fascist film under Mussolini.
These were almost all studio films.
They were shot inside studios.
Neorealist films are shot outdoors.
And they were largely escapist fare focusing more often than not on the upper social orders.
In fact, there was a genre of films so devoted to the romantic entanglements of aristocrats that they came to be known as white telephone films.
The reason for this is that in the old days before cellphones and before multicolored phones, all telephones were black.
And it was a mark of astonishing aristocratic privilege that the people in some of these films had white telephones.
They were such unusual objects for the time that the film's came to be labeled as white telephone films.
And they were largely, as I said, escapist movies with beautifully made up actors and actresses in artificial light, performing what were essentially artificial stories.
The second tradition against which the neorealists are responding, against which they are reacting, is the tradition of Hollywood glamor.
There's a wonderful moment in The Bicycle Thief as you'll see tonight when Ricci finally goes off to get his job.
He recovers his bicycle from hock.
He's pawned it.
And that's why he's so nervous.
He knows where the bicycle is, but he doesn't know how they're going to get out of pawn.
And you'll see how his wife solves that problem in the very next sequence beyond the one I've just shown to you.
When Ricci finally goes to work for the first time, he has someone with him giving him instructions, and what he is a bill sticker.
He puts up posters, great pos-- In fact, the poster that he's putting up is a poster for an American movie.
And it has a great American, a famous American movie star of the '40s and '50s, Rita Hayworth, on it.
And as he's putting the picture up-- you put paste on the board, then you-- he's given these stupid instructions.
Anyone could do this job, but the man giving him instructions acts as if he's explaining something really difficult like rocket science, when all he's doing is telling him how to paste a poster up on a board.
But his instructions include systematic remarks about being sure not to leave any wrinkles in it.
And I've taken that myself to be a kind of implicit complaint against Hollywood.
They'll say the Hollywood female has no wrinkles.
The Hollywood female, the Hollywood star, the Hollywood movie is so glamorized that you never see wrinkles or defects.
And this couldn't contrast more shortly with the ordinary faces of the actors that we see in neorealist films.
So two traditions of glamor and escapism are essentially the antagonist that help to explain how neorealism defines itself and understands itself.
It's worth briefly mentioning that neorealism has distinguished origins in at least three different national cinemas.
The first is in Italian cinema itself.
There was a movement during the silent era, a kind of documentary movement, that wanted to talk-- a quasi-documentary movement.
They were still fiction films.
But they wanted to talk more openly and honestly about contemporary events in contemporary society.
And the movement was called the verismo movement, the truth movement.
And it occurred between 1914 and 1916 in Italy.
And this-- When the neorealists began to articulate their own manifestos and to explain theoretically what they were up to, they sometimes reached back to the verismo movement, as a way of saying we're actually traditional as well as new.
A second fundamental influence on neorealism, a kind of earlier version of neorealism, occurred in the German tradition at the end of the silent era in a series of films made in the 1920s by the director GW Pabst, P-A-B-S-T.
The most famous of these films is a film produced in 1925 called The Joyless Street.
It partly deals with prostitutes, and it talks, it tries to dramatize in realistic ways the conditions of the lower social orders.
And of course the final originating source, this also explains the richness of neorealism that there are other film traditions that are feeding in to it and enriching it, is French, and it's the kind of French film that we associate with Jean Renoir and with the traditions of poetic realism.
Some of the neorealist, at least one of the neorealist directors Visconti, had worked as an assistant director for Jean Renoir at one point in his career.
And I believe, I'm not sure this is accurate, so don't take this as absolute fact, but I believe that it was Renoir himself who gave a copy of The Postman Always Rings Twice to Visconti, which led to the creation of what many people would identify as the very first neorealist film.
The basic practices and traditions of poetic realism are carried over in neorealism.
And in one sense, one can see neorealism as an extension of poetic realism.
The difference is that neorealism has a more political and social orientation.
It's more conscious of having a political and social ambition.
It wants to-- And sometimes neorealist films can seem a bit preachy because of that.
There's clearly a kind of sentiment deeply sympathetic to the lower social orders, deeply suspicious of the hierarchies of capitalism.
Some of the neorealists were Marxists or Marxist sympathizers, and that helps to explain the political orientation of neorealist films.
But the basic features of poetic realism, the use of an outdoor camera, the interest in a mise en scene style, and in a film made by, which I mentioned I think last time, made by Renoir in 1935, a film called Toni T-O-N-I.
We have a French version of neorealism, because that film, as I mentioned last time, is about Italian quarry workers in the south of France, and it has a lot of nonprofessional actors in it, something that became a key feature of neorealism.
So the neorealist movement then has this powerful context which creates a kind of moral and aesthetic energy that helps to explain the power and the excitement of these films.
It has two very clear antagonists, the glamor of Hollywood and the false and aristocratic glamor of the film under Mussolini.
And it also draws as well on older traditions of film in Italian, in German, and in French that set earlier examples for the kind of thing that the neorealists wanted to do.
I've already talked about the key features of neorealism just a second ago when I spoke about poetic realism.
But let me mention some of those key features in a little bit more detail.
One way to think about neorealist films is to recognize that their emphasis on reality is one which insists on trying to create an illusion of a plausible actuality.
And what this meant to them was, listen, people have warts on their noses.
People are not beautiful the way they are in Hollywood.
So one thing we need to do is we need to roughen up the look of our performers.
And in fact, professional actors, it was felt, often bring a kind of smoothness or a fluidity to their performance that is inherently artificial.
So many neorealist films have no or virtually no professional actors in them, or sometimes only professional actors in the major roles.
In the film you're going to see tonight, the star is a nonprofessional actor.
He actually ended up having a tragic life.
The film won the Academy Award as the best foreign film of the year in America in 1948, and the man who played Ricci became an international star, but he fell into hard times and ended up not really making much money from his very brief stardom, and my understanding is that he never again made a film as remarkable as The Bicycle Thief.
But one of the things that is absolutely astonishing about the performance is that this man is not an actor.
You would never guess it.
I mean, it's an absolute-- or maybe you would, but you would guess it only after having been told that.
Because he performs with such naturalness and such grace.
So the use of nonprofessional actors is one of the key features of neorealism.
A second key feature is a rejection of studio filming.
All neorealist films, like almost all poetic realist films, were shot outdoors in natural light.
Partly because they wanted natural light, and partly because studios lie about the texture of the world.
We want the texture of the world in our films, the neo-- Our films are really about reality, so we can't shoot indoors.
A third-- nonprofessional actors, outdoor camera, a third feature would be what we might call a fundamental mise en scene style, taken directly from the stylistic habits established by Jean Renoir and other French directors who worked in the same style.
Relatively modest forms of editing.
The forms of montage that you see are not disruptive.
You won't get jump cuts or disorienting moments.
The camera, the camera is, as with Renoir, constantly restless and in motion, but often in very gentle ways.
There's a wonderful moment in the scene I mentioned a moment ago where we see Ricci putting up his first poster when the camera is watching them, and he's getting these instructions from his comrade about how to put the thing up, and it's really boring.
Well, one reason that you know it's boring-- well, obviously as a viewer, you begin to probably feel it without registering that it's boring, but the reason that you know it is is that the camera suddenly looks away.
What happens is the camera discovers something interesting happening on the street and while the conversation continues on the soundtrack, the camera shifts away from Ricci and his instructor, and follows a wealthy man and two children who are walking behind him.
And they're really beggars trying to get money from him.
And the camera just found it interesting, so it looked at it, a very Jean Renoir-ish move.
There are other moves like this in the film, as if the camera, like Jean Renoir's camera, is so interested in the world that it's capable of being distracted.
There's a sense that the world's complexity is always threatening the coherences and meanings that the film is releasing for us.
So nonprofessional actors, outdoor filming, a mise en scene, a profound serious mise en scene style influenced deeply by Jean Renoir, and finally, we might maybe another-- not finally, one more, a fourth feature, we might say is what I was trying to indicate when I talked about the social or political bias of these films.
One might say that these films have a more profoundly documentary flavor.
I don't mean that we feel we're watching documentaries.
But we do feel we're watching a slice of life, that the economic and moral and social and personal circumstances of the characters we see in neorealist films are a version of what we would find if we walked into Rome not in a movie and just looked for people of this class, a documentary flavor.
So that's four features.
And a final feature, I suppose I would want to add, has to do with the relation between plot and character.
It's not quite true to say that these films are plotless, they're not, and one of the most magnificent things about The Bicycle Thief is the economy of its plot, and I'll talk a bit about that tonight, but it's nonetheless true that in neorealist films, you do not feel that action drives events.
What you feel is that character drives events in some sense, what the plot aims for is a natural unfolding.
You're never supposed to feel in a neorealist film that some event in the plot has been introduced arbitrarily.
You're never supposed to step back and say, oh, that could never happen, or how implausible is that, or I guess they did that because they wanted to show x.
The kind of response you often have watching certain forms of Hollywood entertainment and other forms of merely escapist entertainment fare.
But you don't ever have that feeling in a good neorealist film.
What you feel is that the events of the plot unfold naturally out of the economic and social realities the characters live in and out of their characters.
And you can test this generalization against tonight's film, and I'm sure that you'll find that it's a compelling one, that it really describes this movement's respect for the complexity of character over plot, for the complexity of character and therefore for its commitment to a kind of movie that is more interested in character than it is in story or event in a narrow sense.
The central figures of neorealism are important to mention.
And maybe especially the fellow that I have first on my list here, Cesare Zavattini, because he's the least well-known of the names I've listed there.
The other three names are all famous directors.
And of course De Sica is the director of Bicycle Thieves, and I'll talk about his career this evening.
But Zavattini was the theoretical genius behind neorealism and a figure for whom I still retain a tremendous sympathy and affection.
He was a novelist and an essay writer as well as a screenwriter.
He collaborated with Vittorio De Sica on almost every one of De Sica's significant films.
Not on every one, but on virtually every one that is truly outstanding, and there are a number of them.
He's the scriptwriter on.
And he had a particularly symbiotic relation to De Sica.
Zavattini also articulated what came to be understood as the manifesto of neorealism.
And in one such essay, I don't know if it was titled Manifesto of Neorealism, but that was its effect.
He said this to try to summarize what he thought the ambitions of neorealism were.
He said, a good film ought to be focused on the life of an ordinary man to whom nothing happens.
Now, he didn't really that nothing would happen.
It's not an early version of the Seinfeld program.
What he's trying to get at is this idea that plot does not dominate, that character dominates, that the events of the story grow organically out of the materials of the text, both out of character, and out of social realities.
So Zavattini was in many ways a visionary figure, an overt Marxist for a good part of his life, and probably the central intellectual influence on the neorealist movement.
Visconti, the director whose dates I've put there is sometimes suggested-- it is often said that not all of Visconti's work fits neatly, maybe none of it fits neatly into the category of neorealism.
And this partly has to do with a kind of flamboyance in Visconti's visual style, and maybe also a fondness for decadence, and for elaboration that does not comport with the social meanings that are implicit in neorealism.
Of the main neorealists, he's the one who fits least well, but he's often said to be the best of them.
And there are many historians of film and scholars of film, including David Cook, who believe that The Earth Trembles, the film he made in 1948, La Terra Trema, The Earth Trembles, is the greatest of all neorealist movies.
I have sometimes been tempted to show it, but it's less characteristic of neorealism, so I don't show it.
But it is a very remarkable film, and Visconti is a very important figure in this movement, even though he probably could be said in his later films to move beyond neorealist concerns.
Roberto Rossellini who also had a very rich long career as a filmmaker, and I've only listed one film there, because I want to show you a clip from it.
Part of a trilogy of films he made at the very end of the war called, it was called his Roman Trilogy or his Rome Trilogy, and the first one was-- or his World War II Trilogy-- the first one was called Rome, Open City, and then there were two other.
The third one was called Germany, Year Zero.
And they all had sort of a political and social ambitions.
They wanted to dramatize the postwar conditions and show the devastation that followed aftermath, that was the aftermath of the Second World War.
Open City had a tremendous impact what it was first released, especially in Italy, because it told an anti-fascist story.
It's a somewhat melodramatic film that ends in a very melodramatic death, in a very melodramatic murder, or an accidental murder.
And partly for that reason, it's a less powerful and less compelling film totally than something like Bicycle Thieves.
But its importance can't be overestimated.
It established the basic principles that I've been talking about.
It was actually begun before the war was over when it was illegal to-- when the German authorities, the fascist authorities were still controlling Rome and Italian society.
And you weren't supposed to-- you couldn't film without a license, and some of the early scenes from Rome, Open City were actually shot illicitly on very bad film stock from behind doorways and around corners, so the people behind the camera wouldn't be caught by the authorities.
And then the film was completed after peace came, after the defeat of the Axis forces.
I'm going to show you a clip from that sequence, from that film, from the very beginning of that film to give you a feel for what the film was like as well.
And I think that that clip will also show you why the film was so successful.
But I'll partly explain it now.
I think the Italians love this movie because it told a useful myth.
It made it appear that very few Italians collaborated with the Nazis and that most Italians were really on the side of the saboteurs who were trying to blow up fascist waterworks.
This is probably not entirely historically accurate since Mussolini was during much of his career a very popular figure in Italy, but you can certainly understand why it would have hit a sympathetic chord among Italians, many of whom, I'm sure, felt terrible ambivalence and shame over the events of the Second World War.
The Italians were always much more reluctant anti-Semites than the Hitler fascists were, and the story of Italian anti-Semitism is much more complicated than the story of German anti-Semitism in this period especially.
So I think that one reason for the success of the film had to do with the way it told a kind of anti-fascist fable that worked particularly well for this postwar society that wanted to put fascism behind it, that wanted to repudiate that past.
that felt-- There were certainly segments of the population that had never gone along with that past anyway, so the film sort of mobilized a new kind of feeling of Italian patriotism, and maybe even in some sense a kind of, despite the catastrophe in which it ends, a kind of hope for the future, because the film takes place entirely within the period when the fascists were still in control of Italy.
And the catastrophe that occurs at the end is an act of fascist murder, an assassination, an indifference to humanity.
So Rosselini went on to make many, many films, and I feel a little guilty only mentioning this one not totally characteristic film of his, but it's important in the history of neorealism.
Rosselini is a director I much admire, partly because he moved away from fiction.
And at the end of his career, he was making what he thought of as educational or instructional films.
But with the dazzling technique of a great fiction filmmaker.
So he's a very interesting figure to study, and I hope some of you will look at more neorealist films and consider looking at Rosselini.
De Sica is the final figure that I want to mention as a fundamental energy in neorealism, and I'll only say about him that he's probably the most various and gifted of all the neorealist figures, partly because he had been a very successful actor before he became a director.
And I'll talk a bit more about De Sica this evening.
What I want to illustrate now in the time we have left is an aspect of the neorealist aesthetic, if we could use that term, that I've talked about before when we discussed Renoir and that I implied in earlier comments that I made this afternoon.
It's what I call the neorealist counter plot.
And by counter plot, what I really want to call attention to are all those elements in the film that seem not to take part in the forward momentum of the story.
We might even think of them as retarding elements in which these films sort of pause to smell the roses.
A version of what I mean, of what I mean by this neorealist counter plot, might be said to be embodied in that astonishing camera move we studied at the end of Boudu Saved from Drowning last week where the camera makes 180 degree swerve, becomes interested in the beauty of the river, becomes interested in the scenery, almost as if it's forgotten its character.
Now, what I mean by the counter plot is this impulse in these films and especially in these Italian films to complicate the social and political and even the psychological processes of the story simply in order to give us a sense of what the experience it's describing is like, as if it's a version of what I'm interested in when I talk about how the camera can distract itself.
One way I can illustrate-- So what I mean by counter plot then are energies that retard or slow down the basic story or that seem not to directly contribute to the basic story, but remain part of the film's desire to represent the world, to represent reality and something of its complexity and something of its nuance, even in something of its contradictory incoherence.
In other words, I don't really want to call these moments incoherent moments, they're not, because they build a much larger and more complete picture of the whole world you're looking at.
And in that sense, they're not incoherent.
But they are a kind of counter to the ongoing pressure to find out what happens next, what happens next, what happens next.
Well there are many, many examples of this principle in The Bicycle Thief and in other neorealist films.
But I want to show you one very dramatic one, also to give you a flavor for an earlier neorealist film other than the one you're going to see tonight.
This is from the very beginning of Open City.
[SPEAKING ITALIAN]
These residents of Rome.
They've heard an explosion.
There's sabotage going on in Rome now.
It's near the end of war, and the war's not over.
And now the saboteurs return home, and look who they turn out to be.
And of course, that's why, Anna Magnani was the actress, was worried about where the children were, because they suspected.
[SPEAKING ITALIAN]
There's already a counter plot operating here.
I hope you see.
Look at these little chil-- innocent faces.
So these saboteurs have now come home.
Watch.
[SPEAKING ITALIAN]
All right.
Stop.
I'm stopping it a little early.
This scene continues.
You actually go into another room in a moment, and you see children in a kind of dormitory like room in which there's one little child sitting on a potty.
I think it's the first potty scene in Western film.
They do something very disturbing too.
They pick him off the potty and put him back in bed without wiping him.
But leave it.
Maybe there was no toilet paper in postwar Italy.
But your laughter indicated to me that you sort of got the point.
But talk to me a little bit about what you see there.
It's a wonderful example of what I was trying to illustrate at the beginning as well, this principle of multiplicity.
What's happening in this sequence?
What do we learn?
What about living conditions in Rome, what do we learn?
Nobody gives a lecture about it.
You think there is a housing crisis in Rome?
Seven families living together in the same room, an old man in bed, lying in a bed, in the parlor, in the space, in the living space of the place.
That's clearly where he stays.
He's very ill. What else?
The deviation into comedy.
The extent to which a kind of a series of comic encounters occur after what, after a moment of political sabotage.
And the incongruity of these little children who are the ones who are the saboteurs is also part of the comedy even before we see them fully return to the condition of children when their parents start slapping them in the face and say, how dare you stay out so late.
I didn't mean to suggest that the parents actually know that the children are saboteurs, although, some may.
But I think the point is that being outside at night is very dangerous for little children in this environment, and that's why they're so concerned But the fact that even the children are partisan anti-fascist fighters becomes a part of what the film is about.
Now, you can't tell from the very beginning, from this beginning, what the real sort of plot energy of the film is yet, can you?
And it will take a bit longer before it finally turns out.
The man that she's about to marry is a secret agent, and he's under-- he's being hunted by the fascist authorities, and the film is about how the good Italians try to hide him.
But we don't learn that yet.
What I want to suggest to you is something of life's variety, something of life's comedy, something of life's incongruity emerges for us in those moments.
It's what I want to call a counter plot, because we take delight in those moments.
We don't get impatient and say, come on, film, tell me where you're going.
Because what the film teaches us is that it's not in that narrow sense about plot at all, but it's about life in its infinite complexity.
The greatest neorealist films embrace this principle as well as any films I've ever known, and, in addition, they have a kind of moral authority that's very rare.
I'm very jealous of those of you in this course who have never seen The Bicycle Thief, because it is a remarkable film and I'm sure you'll remember for the rest of your lives.
I'll see you tonight.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Good evening, people.
We continue our discussion of neorealism by looking closely tonight at one classic instance of the neorealist movement, the single most famous film from that school, Bicycle Thieves.
Before I turn to talking a bit about aspects of Bicycle Thieves itself, which I hope will frame your viewing in a way that will help you get a lot out of the first viewing of the film, I want to say a few words about Vittorio De Sica himself.
I mentioned this afternoon that the neorealist movement, although it was immensely influential and continues to be influential, in fact, on realist filmmakers and on documentarians, and it was especially in its earliest phases, hailed globally as a very remarkable movement, it incited criticism almost from the beginning inside Italy.
And I suggested it this afternoon that some complaints against movie essentially came from conservative forces in the society that felt that the portrait of decay and breakdown and dislocation that was at the center of neorealist films-- after all, they were about the contemporary world and about the broken universe, the broken planet, at least a broken Europe that we encountered after the war.
There were many kinds of conservatives in Italy who felt that it promulgated a bad image for the society, that made investment more difficult, that it was a kind of downer.
And this criticism was very powerful.
But what really undermined the movement in terms of getting financing, the reason that the movement sort of ended after about the time Umberto D, the last of the truly neorealist films that De Sica made, was produced in 1952.
It's life was very, very brief.
Many people would say that was the end of neorealism although there are some films that come after that have neorealist dimensions and elements in them including some by Fellini.
Still the idea that the movement had a very short life despite its immense influences it is an accurate one.
But what I had left out this afternoon is an important qualification or addition which is that the films also incited criticism from the left.
When you see Bicycle Thieves you'll see why that might be a surprising fact, because the sympathies of the film are so totally on the side of the underprivileged and the underclasses of the culture.
It's difficult from our contemporary vantage point to see what would've offended leftists.
But there was a very vigorous and theoretically sophisticated communist movement in Italy in this period as there continues to be today in some degree.
And many people on the left, especially communists, but socialists as well, also complained about neorealist films.
In particular about De Sica's films because their complaint was not that the difficulties of the World War were exposed.
They were perfectly happy with that.
But they were angry that there wasn't a program for renovation.
What they were really angry about was that there wasn't a Marxist or a socialist program built into the films.
And it's always struck me as a very revealing and interesting fact about the movement that it couldn't satisfy either left or right, even though these films are among the most powerful and compassionate films about the lives of ordinary-- not just ordinary people, but people without power, of people without control over their lives that have ever been made.
So it's a curious fact.
It bothered De Sica all his life that he seemed unable to please either side in that political argument.
And at one point in an interview he said something-- if I could find my notes, I'd like to quote to you.
He said in response to a question that asked him why do you think the socialists and the communists were somewhat-- or at least some socialists, so remember Zavattini was a socialist, so he collaborated deeply and loved De Sica and collaborated with him on his most important film.
So I don't mean that all socialists thought ill of neorealism.
They did not.
But some did and certainly people on the far left definitely did.
And here was De Sica's response to why-- he expressed disappointment about this.
But his response really in a way is a critique of the programmatic demands that these people were making on his films.
He said, "My films are a struggle against the absence of human solidarity."
You should think about this quotation as you're watching tonight's film because as you'll see in the film, the official institutions of the society seem to have broken down or at least to have become not effective.
They're overburdened.
They're somewhat indifferent.
It's mostly that there overburdened, that the problems of the society are too great for the institutional structures that are in place to handle the problems.
But in any case as you're watching the film, I think you'll notice not only that clearly this breakdown of institutions are very-- that we might expect to be helpful to Ricci in his quest.
But what we will also discover is that there are other forms of solidarity that emerge in the film.
One of them is the solidarity of husband and wife.
Another is the solidarity very much tested in the film.
It's the great central theme of the film, the solidarity of father and son.
And then there's also the solidarity of what we might call neighbors or neighborhoods.
And you'll see how that element works toward the end of the film in its remarkable and surprising conclusion.
Now here's the quotation.
"My films are a struggle against the absence of human solidarity against the indifference of society towards suffering.
They are a word in favor of the poor and the unhappy."
One reason I like the statement is it so, in a certain sense, unsystematic.
It shows-- they seem to me to be the words of an artist not a political pamphleteer.
And they partly explain the richness, the resonance, what I've been calling the multiplicity of Bicycle Thieves.
There's a moment in Bicycle Thieves about which Andre Bazin has written very beautifully.
The same critic that was a champion of French poetic realism and of genre and of war of course becomes one of the great champions of Italian neorealism.
He can see that the Italian form of realism is an extension and elaboration and in some ways a deepening of the implications of poetic realism.
He's written brilliantly about the neorealists and especially about De Sica.
There's one place in the film that he particularly focuses on one of his essays.
It's-- I have to give away a small part of the plot in order to explain this but it really won't trouble you because the plot engine gets going so early in the film that it's hardly a surprise.
In fact, you really know what's going to happen almost from the opening dialogue in the film where people are arguing about who has bicycles and who does not.
The film is organized in a very elegant, relatively simple way.
It covers essentially three days of time.
The first day is the day in which Ricci gets the commission for the job.
And then goes with his wife to get his bicycle out a hock.
And you'll see that they have to go through certain kinds of tricks to find the money to do this.
And then once he has his bicycle he goes to work.
On his first day of work, guess what happens?
Someone guess who hasn't seen the film.
What do you think might get stolen?
OK, right.
His bicycle is stolen on the first day of work.
Now the fact is you're really not surprised when it happens because almost from that first scene you're aware that bicycles are this incredibly precious object in the world and yet also incredibly common.
And that's part of why it's such a resonant symbol in the film.
But there at the center of the film, the most fundamental and most memorable parts of the film involve what I'll call sort of the third segment.
I don't know that the segments are actually equal in proportion in terms of time.
I think the third is the longer one.
But the third sequence is the Odyssey or the quest in which Ricci and his young son embark upon in order to recover his bicycle.
So he spends the whole-- I think he gets his job, gets his bike, starts his job on Friday, has as bike stolen on Friday.
Or maybe it's on Saturday.
But in any case, he then gets his bike stolen.
He then asks for help from various sources and the following day he spends the entire day walking through the city of Rome looking for the thief trying to find his bike.
He actually finds the thief as you'll see in the grand climax of the film.
Although the thief turns out to be as miserable and at least as much without prospects as poor Ricci himself.
Something very interesting happens in that moment in the film because your sympathies shift in some way to the thief in an odd way.
And you watch as his neighborhood gathers around the thief to protect him in certain respects.
So the heart of the film is this search that the father and the son go on through the third day of the plot looking for the bicycle.
As the day wears on, the father becomes more and more nervous, more and more uneasy, more and more filled with anxiety.
The bike seems to him the solution to all difficulties.
And sometimes the son slows them down and there are scenes in which the father becomes annoyed or angry with the son but I'm getting a bit ahead of myself.
So I won't repeat this again.
But I'll come back to these matters in the theme of fathers and sons.
In any case there's a moment in this Odyssey where the father and son are engaged in the search for the bicycle when it's just after-- I think it's exactly at the moment when there's a gigantic downpour, they're caught in a terrible rainstorm.
I think in fact the rainstorm caused this.
The boy has to pee very badly.
And so he interrupts the quest to go and pee.
I mean he has- you have to go, you have to go.
But the father is so upset that the boy has slowed him down that the slaps him.
And you get a sense that it's maybe the first moment that this man has ever struck his son.
It's a tremendously fraught moment.
This is how Bazin talks about it.
And this is why I mention this moment now.
It's a way of illustrating this principle of texture or openness to experience-- that's part of the De Sica's vision of the world.
He is a socially conscious director but he's not a programmatic or a politically preachy director.
So Bazin focuses on that moment I've just described to you.
He says, the boy needs to pee.
A downpour interrupt the chase.
Then he says quote, "The events are not necessarily signs of something of the truth of which we are to be convinced.
They all carry their own weight."
The rain, the need to pee, the father's anger, right?
"They all carry their own weight.
Their complete uniqueness.
They all have," Bazin so finely, "They all have that ambiguity that characterizes any fact."
When a film is able to capture that ambiguity that characterizes any fact, it's a work of art.
It can be programmatic if it does that.
And in fact the moment I'm describing is really in some sense, a version of what I've been calling the neorealist counter plot.
What constantly happens in good neorealist films and what happens again and again the Bicycle Thief is that the ongoing quest for the bicycle is retarded or interrupted by various kinds of accident disturbance, strange encounter.
So that the search for the bicycle becomes something much larger than that and every particular moment in the film has, as Bazin says, its own uniqueness, its own-- you don't feel that anything is in the film to further the plot or to illuminate the character or to tell you something about what political attitudes you should have.
Although by the time you're finished with the film you do have a very coherent sense of how the society is breaking down, of how people are sustaining themselves, about how difficult life is in postwar Italy.
You're aware of all of these things.
And aware of them with a concreteness and the clarity that you wouldn't have if you read an abstract essay.
You've seen them embodied in a totally believable fiction.
But what gives the fiction it's authority, what gives it its power is precisely that you don't feel you're watching a sermon.
You don't feel you're watching something in which some political moralist is beating you over the head and telling you, oh you better love this poor guy, you better.
And in fact as the film goes on, one of the things you discover is that there are nasty or bad sides to poor Ricci.
We understand that he's driven by economic anxiety that's very deep and partly and certainly mitigates his behavior.
But he behaves crudely and cruelly to his son, never intending to.
Then of course there are moments when he recovers and tries to make up with his son.
I'll come back to this in a moment.
So this passage from Bazin also calls attention to what I think of as a signal feature in these films and especially in the Bicycle Thief, that power that any moment in the film has to express a kind of complete uniqueness, the ambiguity that characterizes reality itself.
Well, what I'd like to briefly do is give you a partial account, a modest account of De Sica's own career and then say some things about the Bicycle Thieves.
De Sica was a stage actor.
He gave regimental performances when he served in the Army in the first World War.
He became a stage hand right after the war and an actor in 1923.
He joined a theater company in 1925, a touring company, and he became a tremendous success in a series of performances.
In 1927 he appeared on stage in Rome in something called [?
Sabun ?]
Reviews and I think it involved singing and dancing.
This was an unbelievable success and turned him into a kind of a matinee idle or at least a theater icon.
He made his first film in that same year in 1922 as an actor.
And his first success came-- he continued to act in films intermittently for the next decade but he didn't become a recognizably serious film actor until his first major success in 1932.
Then for 10 years, he's a kind of matinee idle Italian movies.
He made over 40 films in the period as an actor between 1932 and 1942-- a very handsome man as some of you may know if you've seen him and capable of great subtlety as an actor.
This may help to explain why he was such a great director of nonprofessional actors, because he knew intuitively I think, how to bring out the performance from someone who was not trained in performing, in acting.
He begins to direct in 1940 and makes a series of moments in interesting but forgettable films I suppose.
His first really significant film from one angle anyway, especially if we're interested in the development of neorealism, is a film he makes in 1942, co-screenwriter Cesare Zavattini, it's entitled The Children Are Watching Us.
It's-- I've seen the film.
It's not a great film.
But it's an interesting film.
And there are some remarkable moments it.
The title tells you something about the neorealist commitment to focus on the under privileged and the dis-empowered in society.
Many neorealist films focused directly on children.
And The Children Are Watching Us is one such film.
And it almost announces the idea that children are among the victims of society because they are subjected not only to all the political and social authorities of the society, but also to the arbitrary authority of their own parents and their own immediate family environment.
So The Children Are Watching Us announces this interest in the disenfranchised and especially in children.
It's a film about the breakup of a middle class marriage mostly told through the eyes of children.
So there are certain things, certain behaviors on the part of the adults are never fully explained because you witness and experience the film through not just the eyes but through the perceptions the understanding, the sensibility of children who are somewhat confused about what they're seeing.
The film that established him as an international figure and established neorealism as an international movement was one that to De Sica and Zavattini collaborated on in 1946.
It was entitled Shoeshine.
I sometimes have been tempted to show Shoeshine in this course, even though it's a much less perfect film than Bicycle Thieves because in its own way, it's so powerful and moving.
One of the most interesting things about neorealism, apart from the fact that it was so short lived, is how relevant the films still feel today.
If you watched Shoeshine or another of the films that I've listed here on Umberto D which is about the problems that people who have gone into-- of the old, people who are retired, pensioners, and the problems they face when there is horrific inflation.
De Sica dedicated the film to his father.
It's about an old man who was very proud.
He used to be-- he dresses very well.
But he's tremendously isolated.
He's terribly poor.
He's almost reduced to begging in the course of the film.
There's even a moment near the end of the film where he contemplates and then pulls back from suicide.
But watching that film today, it feels much less dated than some films that are much later than-- were produced 10, 15, 20, 25 years after it because the theme is so relevant.
Because we know that in society today there are so many-- in all of a Western societies but especially in the United States-- there are pensioners who can't make in ends meet.
The same thing is true, I think, in some degree of the role of the fate of children in modern society.
So Shoeshine, although it has a very melodramatic and violent ending, it's an immensely powerful film.
Essentially it's a story about two boys, two Roman street urchins who are taking advantage of in some sense by their own relatives who are trafficking in black market goods.
It's set in the same moment at the end of the war.
In fact, the film opens with these urchins trying to make a living as shoeshine boys.
That's why where the title comes from.
And these shoeshine boys are wandering the streets of Rome, going up to American soldiers who are dominating the streets, it's still an occupied city, asking soldiers if they want shoe shines.
The children are unbelievable victims.
They're very close friends.
They end up buying a racehorse, a broken down old racehorse that can't race anymore and they stable it someplace.
They are very excited about their possession of this horse.
There's some early scenes in which you see them riding on the horse with tremendous joy and so forth.
But of course this is the dawn before the miserable night.
Because what then happens is the kids are taken advantage of by one their-- by a close relative.
They're involved in black market activities.
The adults use the children because they know that children are less vulnerable than they would be if they're caught.
The children are caught.
They're sent to reform school.
Most of the film takes place in the reform school.
You see these two bright-eyed, beautiful young children, maybe 10 years old, who are the fastest and closest of friends when they go into jail, slowly breaking down.
You see their humanity being beaten out of them by the conditions in the reform-- in the children's prison.
It's a terrible disturbing film in part because you could see that the people who are running the prison actually mean well.
They're not vicious, evil people who want to harm children.
The system seems designed simply to cause more harm than good and to destroy the vitality and optimism.
And when I say the film ends melodramatically, one child strikes out at his erstwhile best friend and knocks him down a bank and he dies.
So he in effect accidentally murders a person who had been his dearest friend.
I think it's a failure in the film.
I think that you didn't need the murder for the meanings of the film to be available.
I think that De Sica learned something from that.
When you watch the ending of the Bicycle Thief, one of the things I hope you'll ask yourself is why is it so apparently undramatic?
And why does that undramatic ending in the end seems so profound, so moving, so meaningful?
It's as if De Sica sort of learned his lesson.
It's as if he drove his point home too heavily at the end of Shoeshine.
He realized that to have a truly organic text that did what Bazin wants text to do, reflect the unique the unique ambiguity of every experience.
The ending of the Bicycle Thieves does that much more effectively as I hope you'll watch for when you you're watching the film.
Shoeshine made a tremendous impact and it won the first of four Academy Awards that De Sica won as best director.
This was for the best foreign film of the year in 1946 and it put new realism on the map.
Then two years later Zavattini and De Sica followed with what I suppose we could call their masterpiece, Bicycle Thieves.
I want to say that this is not a complete-- remotely complete list of De Sica's films.
I've just listed some highlights here because I wanted to give you a sense of his work.
I mentioned Umberto D already, the last neorealist film according to many people, and a very moving one about an old pensioner.
The film Two Women stars Sophia Loren, one of the great sort of pulchritudinous Italian movie stars of the '50s and '60s.
It got a lot of attention.
There were many other interesting films that he made.
But one of his great triumphs was the very last we made.
That's what I want to mention it.
Zavattini returned to do an uncredited coscreenplay work on Finzi-Continis and you can feel Zavattini's hand in many moments in the film.
It's an adaptation of a novel.
As some of you may know, it's really a Holocaust film.
It's about the buildup to the Holocaust.
The Finzi-Continis are very wealthy Italian Jews.
The story is about a poor young Jewish scholar who is, because of the anti-Jewish laws that are developing in fascist Italy, is barred from using the public library.
He's a very gifted young scholar.
He's in his early '20s.
So he finds out that this very wealthy aristocratic family, also Jewish, the Finzi-Continis have a magnificent library.
The Finzi-Continis open their library to him since he's barred from going into the public library.
Of course what happens is, you might guess this too, it's a wonderful novel although the film is even more beautiful I think.
He falls in love with the Finzi-Continis' daughter.
Right, so it's a class issue.
The poor boy, rich girl.
So the foreground of the film is this cross class romance.
But in the background of the film it's something like cabaret but not as dramatic, not as overt.
Because what's going on in the background of the film is the growth of fascism.
The end of the film ends with many of the characters we've gotten to know very well, both young and old, all of them Jews, some very aristocratic and some at the bottom of the social hierarchy, all being herded together being ready to be shipped off to the camps.
You don't actually go-- the final scenes of the film have them in the railroad station being prepared for their final journey.
The film is also full of comedy, just as some of the passages that I showed you in clips from this afternoon mix comedy with socially serious themes.
This is even truer of The Garden of the Finzi-Continis One of the reasons I like the film so much is that it's a return to the energies and the moral passion that had marked De Sica's earlier career.
And I think that my own feeling is the The Garden of the Finzi-Continis is one of the great European films.
It certainly would be on my list of the best 15 or 20 European films of the sound era.
I think many of you would find it very instructive and moving.
But I think especially so after you saw some of the truly neorealist films, the early neorealist films.
One can't really call The Garden of the Finzi-Continis a neorealist film although it has many elements in common with those earlier texts.
So De Sica was a remarkable man, a significant actor, a great director.
He directed 25 films in his career, won four Oscars for best foreign film.
He acted in more than 150 films, mostly light, comic social matters kind of roles.
But there's one exception I perhaps should mention.
Playing in a film for his friend, Roberto Rossellini in 1959, Rossellini's greatest film, a film called General Della Rovere, De Sica plays the lead and it's probably his most memorable performance.
It's a very, very subtle and good film about political ambiguities and conflicts of loyalty, human frailty, and human inadequacies in a way that's characteristic of the best neorealist work.
So those are all films I recommend or urge you to consider watching in your leisure time.
Let me say a few words now about Bicycle Thieves.
I've already implied or actually said a part of what I want to say so what I think I can be briefer than I had originally planned.
Let me remind you again about the structure.
I mentioned to you that the structure of the film is elegantly simple, that the amount of time the passes is very compressed.
And that's helpful because the time compression makes you aware of the urgency of the quest that the father and the son are engaged in when they're searching for the bicycle.
So one way-- and as the story unfolds, a number of things begin to happen.
The quest for the bicycle becomes a kind of Odyssey in which several things happen.
One is the film takes you on an exploration of Rome.
It's like one of the great city films.
It visits the city.
It explores the city.
You see different-- it's not just that you see police stations and religious institutions and the catacombs underneath the city.
Where the unions meet, and where Ricci first goes to get help from Union members who promised him that they'll find his bike, very easily, and they give him a plan and he's very excited and then the next morning when they follow the instructions of the union, the boss doesn't help.
So the unions are breaking down as well as the church, as well as certain kinds of welfare organizations in the state and so on.
So one thing the film becomes then is a kind of travelogue, a kind of exploration of the city, the city's geography as well as the city's make up in some way.
You go into different neighborhoods and you hear-- I think Italians can pick up the different neighborhoods speech patterns in the film, at least I've been told this made by native Italians.
So it's a kind of travelogue in one sense.
It explores this place.
And part of the exploration of course involves certain ancillary or other themes that are organically linked to the story of going through the city looking for the bicycle.
One of course is what we might call the social themes of the film.
Because as the protagonists make their way through the city searching for the bicycle what you encounter are little vignettes that tell you something about the social hierarchies of the society, about the efficacy or in efficacy of its institutions.
I won't go into great detail here.
I hope you'll draw your own conclusions and name some of the specific examples that reinforce this general idea.
But I do want to mention at least one moment in this Odyssey.
One of the most dramatic and disturbingly comic moments in the film occurs when Ricci and his son actually think they've seen the thief.
They think they recognize him and they follow him.
And they get to essentially a mission where relatively well-off, self-satisfied Christians are serving meals to the indigent and the poor, right?
Ricci gets into this institution not because he sort of wants the meal or because he needs the religious uplift but because he's chasing his thief.
And he finds once he's in there that he can't get.
It turns out that in this institution that not only do the poor have to pay for their meal by agreeing to all kinds of religious behaviors, prayers, and other kinds of things.
But they're also locked into the building.
They're not allowed eat, they're not allowed to leave at least until they finish eating.
Maybe they're not allowed to leave until they can be checked to make sure they're not stealing the soup bowls.
But in any case, it's a very interesting moment because you can see that even the people who would intend to do well in society often do more harm than good.
And of course the implication is that the traditional religious institutions and traditional religious arrangements are not sufficient to take care of or adequately to respect the difficulties of the poor.
That's just one such moment.
There are number of others.
In fact there is one other moment I should mention as a kind of contrast.
I've said that we see many instances in which we understand that the larger institutions of society like the police or the religious system or the union system are breaking down or not really helping people.
But I will but I've also said that there are some other sort of unofficial, and non-institutionalized sources of solidarity and comfort that emerge in the course of the film and I hope you'll watch for them and think about what their significance is.
But there's one moment which complicates this argument because we can see the social institutions of society not actually functioning effectively but at least showing some compassion.
It's an astonishingly vivid, meaningful moment in the film, another one of these moments so full of multiplicity that the meanings are so deeply embedded in what you dramatically see that sometimes you don't even step back to generalize on the meaning of what you've seen.
This is a scene very early in the film where Ricci actually goes to retrieve his bicycle.
His wife has figured out a way-- basically what she figures out is they sell her dowry.
She has a couple of fancy silk sheets.
They take them to the hockshop, and they pawn the sheets in order to get enough money to get the bicycle out of hock.
It means they're going to be sleeping on mattresses and but so what, this is more important right?
You see Ricci and his wife go to the pawn shop which is clearly run by the state.
Is not like a private pawn-- it's obviously a city institution of some kind.
And they bring the linen up to the guy and he looks at them and he says, OK, I'll give you this much money.
And they need a little bit more because they need to get the bike out of hock so they ask him for more money.
And he looks at it, he says, oh, it's not really worth it but, OK. And it's an act of generosity on his part because he didn't have to do it.
He gives them just enough money for this stuff so that they can get his bike out of hock.
And then, completely silent, you see him pick this material up, this pawned bedding, this pawned linen, and he turns around-- I think he does-- he may give it to another character.
I haven't looked at this scene a year or two so I don't remember the exact-- you'll recognize the scene.
I think it's the same man who does it or he may give it to someone else.
The material-- the three or four sheets that are involved are taken by someone.
He walks to the back of a room, and he begins to climb a ladder.
As he climbs you realize that what he's climbing up is-- he's climbing a ladder which is leaning against a series of shelves that seem to go on endlessly.
They seem to be in an auditorium the size of the city of Pittsburgh.
And they go up endlessly high and every single shelf, every single inch of space on the shelf is filled with bedding as if the entire city of Rome has pawned its bedding, as if everybody in Rome is sleeping on mattresses.
Now the film never makes a comment about this.
No one in the film says anything about it.
But it's an unbelievably resonant and rich moment.
And it's also-- it does further the sort of social themes of the film.
But it occurs in a moment in which you will also see the pawn broker showing a moment of compassion and generosity.
And if you did not show that generosity they wouldn't have been able to get the bicycle.
So it's a complicated moment.
But if all this linen has been pawned, there's some question about whether or not the city services are doing much good for anybody.
The city's in obvious trouble if nobody has any bed linen any longer.
The implication is this is an unbelievably impoverished and damaged environment.
But it's done in such a quiet way there's even something almost comic about the sense you have of the endless number of these things.
Even the idea that you have to distinguish one from the other seems from our vantage point a bit silly, even though each one has a ticket on it, each one belongs to an individual family.
So watch for that moment.
It has some of that ambiguity that Bazin talks about.
So the structure of the film is organic in the sense that nothing that happens in the film doesn't seem to arise naturally out of the needs of the characters.
We have to find our new bike.
And in the course of their travels other things happen.
The boy has to pee, they get hungry.
They have to eat.
Or sometimes it rains, they have to get out of the rain.
Right?
And that becomes part of the ongoing sort of organic development of the story.
You never once feel, I think, in this film that any event has been imposed from the outside.
Every event seems to arise naturally out of the circumstances and nature of the characters.
And that's why I call it organic.
It's a deeply organic way of thinking about the material.
But it's organic in another way too course because it's deeply coherent.
When you finally finished the film, when you step back, you say, OK, yes the film slowed down here.
And yes, the film seemed to generate what Thorburn calls a counter plot here.
Nonetheless, taken as a whole, there's a profound moral and thematic coherence in this organically fluid movie.
So the structure of the film is part of its essence.
And although I haven't put this on the outline because we've talked about this so often already, pay attention to the Renoir-ish type camera behavior.
The film's visual style is like Jean Renoir's.
There's a kind of elegant restlessness to the camera that's very much a part of what the film achieves.
So in talking about the social themes all I meant to call your attention to was the extent to which in a variety of ways, the film calls attention to the miserably endangered circumstances of ordinary people in this broken down, war torn environment.
It also looks to those, or seems to find without even looking, to dramatize or reveal those sources of support and sustenance that get us through bad times.
As you're watching the ending of the film, ask yourself what these things mean.
I won't actually tell you whether he recovers the bike or not, let you see that for yourself.
But ask yourself, whether or not he recovers the bike, would it have made any difference?
Does it matter given what the film is finally saying to us in those final images?
Well, character.
This is one of the great films about character, I think, especially about the relation between the father of the son.
They're at the very center of the film.
I've already said in a way that this Odyssey that they go on is a travelogue.
It teaches us about the geography, social circumstances, social arrangements, and class system and institutional systems of Rome.
It does that.
But it's also an Odyssey in which the father of the son get to know each other in a deeper and in many ways, much more disturbing way than has ever happened to them before.
The most poignant and terrible thing about this film in fact, is that the Odyssey I've been describing through Rome with his son, by Ricci, is also an initiation story, a story about growing up.
But what happens to Ricci's son in this film doesn't happen to most young boys until they're much older.
And it happens gradually, not all at once.
But what Ricci's son discovers in the course of this day are his father's flaws.
He discovers things about his father that no boy really wants to know even if he's 67 years old like your professor.
And he discovers these things about his father in the most stressful circumstances.
His father's violent toward him, his father is indifferent toward him.
Then there are moments when after his father feels terrible anxiety and grief over the way he's treated him where his father tries to make up with him and you can see the boy not wanting to let them do it.
It's unbelievably rich.
And I don't think anyone who pays attention to this will fail to recognize the interactions between the father and son as unbelievably true, marvelously accurate to the way in which loving parents and loving children often interact with each other.
Because they have many moments where they're really not happy with each other at all.
What the boy discovers about his father are some things that no child ever wants to know about his father.
Not just that he's flawed, but he's deeply, deeply flawed, that he's capable of really terrible behavior.
And the boy is-- something happens in this Odyssey.
It's one of the subtlest things about the film.
The basic relations between fathers and sons are reversed.
We find that the son, this little child, is twice in the film in the position of having to rescue his father.
I'm simplify certain things because I don't want to give away the plot.
But you'll see how this works out in the film.
I mean there are two moments in the film in which Ricci fate would be far more dire than it is because of his son's presence.
His son, in a sense, is his Savior.
The fathers are supposed to protect their sons not the reverse.
Part of the meaning of this adventure is that Ricci has been reduced to such a miserable level that even has to rely on his son's help in order to get by.
Both of them, the father and son experience this reversal.
So it's a psychologically traumatic and deeply moving experience.
Although don't misunderstand.
I'm not suggesting that the child doesn't come to terms-- he does come to terms with what he learns.
I think is quite remarkable.
I mean I would have had trouble with what that boy comes to terms with as an adult but he does it as a child.
But I think of course it's also believable as you'll see.
So the film is profound and serious about the social circumstances of postwar Italy.
Is profound and serious about the indifference of certain kinds of institutional arrangements to the real miseries of human experience.
And it is a powerful and deep exploration of human character and especially, centrally, the relation between Ricci and his son.
The relation between fathers and sons-- becomes a kind of one of the great parables of the relations between adults and children.
It's one of the great father and son stories of all time.
Let me conclude by saying a couple of words about the title.
I mean there's much more in the film I wish I can talk about.
But by this time in the course if you can't get this stuff yourself, I haven't done my job.
I hope all of you are watching films much more attentively and carefully than you had before.
I hope that your enjoyment has been increased rather than decreased by what you've learned in this course.
Let the Bicycle Thief be a test.
Because in many ways it's not kind of film most of you guys would go out to watch.
First of all it's in black and white.
Second it's more than 50 years old.
But I think you'll I think you'll recognize that it hasn't aged, that the film is as compelling now as it was when it was-- maybe even more compelling now than when it was made because some of the political turmoil has disappeared.
There's no longer the taint of fascism hanging over Italian artistic productions as they were at the end of the war.
There's a historical sense of the importance of this movement.
So maybe we can look at these neorealist films with a more objective eye than the original audiences could.
In any case the title is an interesting one.
When it was first translated into English, and in fact it's still widely known, you might have noticed a slips of the tongue on my part where I started to call it the Bicycle Thief.
And the reason is, it's still often called that.
When it was originally brought into the United States and translated both in Great Britain and in the United States, it was called The Bicycle Thief.
Now the Italian is in the plural.
It's Ladri di Bicilette.
Thieves of the bicycle would be the literal translation.
Right now the question is why the plural?
I think you could imagine the answer just from that opening sequence that I showed you this afternoon.
Why would it be plural?
Why would the title be plural?
One of the reasons is from the very opening sequence, we see that everybody-- that the bicycle is a very ambiguous and complicated symbol in the film.
Symbol is a bad word because it's such an actuality that it seems pretentious to call it a symbol.
You see hundreds of maybe thousands of bicycles in the film.
That's part of what makes Ricci's loss of the bicycle so poignant.
You have the sense as he's walking around the city, oh my God, everybody has a bicycle.
There are 10 million bicycles in this city.
Poor Ricci can't find one, can find-- there's something horrific about the idea that an item so common could still for Ricci you be so precious, as to constitute his livelihood or threaten against his livelihood if he can't find the bicycle.
You can guess now based on what I've already said one reason for why the title is plural.
Why?
As Ricci's walking around looking for his bicycle, walking through a city full of hundreds and thousands of bicycles, what might he think?
What might you think if you needed the bicycle desperately?
Why not pinch one?
Why not thieve?
So one of the reasons that the title is plural is that Ricci is one of the thieves.
I'm only suggesting that he's a thief by sort of psychological inclination.
He looks lovingly-- as you'll see, the plot makes this even more explicit.
I don't want to give away the plot.
I will.
He actually does in a certain sense become a thief.
But long before he becomes a literal thief and actually puts his hands on someone else's bicycle, you and I might even think some people in the audience may get the idea before Ricci does.
When you're watching this guy, so terrible, looking every place for his bicycle, he's surrounded by bicycles, sees inundated by bicycles.
After he recovers the bicycle, before he goes to work, on his wife asks him to take into a special location.
She wants to thank someone.
It turns out, that he had-- and they get on the bike, she rides behind him and he rides over on the bike.
And he waits outside this place while his wife goes up.
His wife is taking a long time.
So he leans his bike against the wall.
He starts to go up the stairs.
He looks, and sees his bike is vulnerable so he walks down, he finds a kid, gives the kid, I think he gives the kid a coin, and tells the kid to guard the bike.
And then he starts walking up the stairs again.
This is very early in the film.
And think how clever this is.
You're aware of how vulnerable the bike is from that moment.
This is a day before it's stolen, right?
You're aware of how precious is and as he ascends the stairs you actually become afraid that his bicycle might be stolen.
So one of the reasons the title is ambiguous is I think in the end, the title encompasses not just the thief who steals Ricci's bike and not just Ricci himself who contemplates becoming a thief in his desperation, but also the audience which becomes complicit Ricci's impulse to criminality because we are ourselves think God, steal a goddamn bike.
Take the bike.
Stop worrying about it.
Get it.
Get away.
There's a bike you could take.
And I think anyone watching the film has this feeling after a relatively short time so that we are bicycle thieves as well.
So part of the subtlety of the title-- and you'll see as the film works itself out, I've already implied this.
When they finally catch up the thief, alleged thief, the guy they think is the thief, he turns out to be an even more miserable specimen than poor Ricci.
He's an epileptic.
He's out of work.
He has to be protected by this community.
His mother's very protective of him.
There's some modest minor possibility that he's acting, that he's faking his mental problems and maybe faking everything.
But I don't think so.
You could judge yourself whether what is in any case, any case surely true, is that the neighborhood coalesces around him because Ricci and his son are strangers and are attacking someone from their neighborhood.
So when this happens, one of the things you'll feel is not just that the thief Ricci was going after deserves some sympathy too.
You begin to feel some sense that his difficulties are-- not that you excuse his thievery-- but you understand it in a deep and generous way I think.
That complicates your sense of who the thieves are and what thieves are like.
And then you've come to recognize that Ricci is a potential thief.
And you care about Ricci.
You see what a loving father he is, how desperate he is to take care of his family.
So you sympathize with him.
And then, you realize if you've been thinking about it, that of course you've been covering bicycles yourself.
So the audience and the real thief and Ricci the would-be thief are all implicated together as if what he's saying is that our common humanity is what bicycle thieves suggest.
And that's one of the reasons that the title needs to be plural because the film is not about a single bicycle thief.
It's about the bicycle thief in all of us.
Enjoy the film.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
There are various kinds of neorealisms, flavors of neorealism-- an Italian flavor, a French flavor.
There is a kind of tonal difference that's worth paying attention to.
And I maybe I can capture it over-simply in two short clips for you.
So the first clip I want to show you is from an Italian neorealist film, the last really powerful fully neorealist film that De Sica himself directed, a film called Umberto D, made in 1952.
[VIDEO PLAYBACK] -[SPEAKING ITALIAN]
And this is our hero, of course.
-[SPEAKING ITALIAN] - - - - - - [MUSIC PLAYING]
And compare this-- think about this scene and then compare t-- to the opening scene of The 400 Blows and you'll feel a difference in mood, I think.
Even the music is a part of it.
So he's in great despair here, and we in the audience know that.
[MUSIC PLAYING] -[SPEAKING ITALIAN] -
Random dialogue-- we don't even know who said that.
[MUSIC PLAYING] Did you see how Umberto looked at the man who sat down next to him?
I think the purpose of that close look was to make the audience look at him too, pay attention to him, even though he never says a word.
It's possible that these scenes might make some members of the audience think that he's thinking about jumping off a building.
All right, freeze it, James.
Back it up two seconds.
This is the last moment in the scene that I want you to see.
Look what happened here.
Why are you smiling?
It's because he was facing--
Speak up so everyone can hear
It's because he was facing towards the middle of the bus.
And then, Umberto, he got up-- [END PLAYBACK] --and he turned right forward
Yes, as if he had more space
Yes
But then what else?
What about the gesture?
What do we feel about this man?
Never says a word.
Sits down next to him Umberto.
When he goes like this, what do we feel about him?
He has some horrible story too.
He maybe is in much despair as Umberto.
The point, of course, is remember I talked quite a bit in earlier lectures about what I called the retarding impulse in certain neorealist films and also in certain films of Renoir-- the extent to which what suddenly happens, as the camera is looking at the world, it finds a locus of interest that may be distracting in some sense.
Now it doesn't truly distract in the sense that it takes you off completely on a digressive course.
But what it reminds you of is the complexity of the world.
What it reminds you of-- this particular scene especially-- is there's plenty of despair to go around.
What it reminds you of is that the story that has not been told about this man-- this silent, suffering, working man who sat down next to old Umberto-- probably has a story, just as poignant, just as terrible as Umberto himself.
And it's one of the ways that the film has of enlarging the implications of the individuated story that it tells.
But the most significant thing about this moment-- so it's a moment in which the camera doesn't actually swerve, but even though our hero, the protagonist, moves off the bus, the camera stays briefly on that man with the hat.
Can we watch it-- just the very end, James-- one more time, just to get another look at it?
[VIDEO PLAYBACK] See, that's a gesture of, what, despair?
[END PLAYBACK] Misery?
Sadness?
Sorrow?
Reflection over what terrible things are about to happen?
And the idea that that story has not been told is as significant for our understanding of the film as the story that is told.
So there's a similar-- I don't want to call it a digressive impulse, but let's call it an impulse toward noticing what is not at the center of the story.
We might call it an impulse toward a partial kind of abstraction, an attentiveness to the world that is always threatened or challenged by new options that it might want to look at.
And not all of these impulses lead to misery or despair, even in the neorealist tradition.
But the point I want to make is that I think the neorealist tradition lays more emphasis on social problems.
Its stories are often parables of impoverishment or parables about disempowerment.
Now the film you're going to see tonight, The 400 Blows, could in many ways be said to fit that description.
But there's a lightness, a lyricism in it, that doesn't sustain the heavy mood, the mood of disturbance, the mood of-- if not tragedy-- of misery, or at least of very alert social awareness that permeates neorealist films.
So here's a moment from the film you're going to see tonight.
But it's a very distracting moment.
You won't forget it, when you see it.
And I'm doing a little harm to your experience of the film by calling your attention to it, because it's such a strange moment in the film.
The hero is one of these two boys, the boy on the left.
And the guy on the right-- if you're facing the film-- to your left is the hero, the protagonist.
And the boy on the right is his best friend.
The girl in the middle is a sister.
And they're accompanying this girl-- this is in the middle of the film.
We're not even told where they're going when it happens, but they're going to the park to watch a puppet show.
And watch what happens.
We're in the middle of the film.
We already know a lot about the boys.
The boys have played hooky, neither of them get along very well with their parents.
[VIDEO PLAYBACK] [MUSIC PLAYING] -[SPEAKING FRENCH] [YELLING] - - - - -
Freeze it second.
They're talking about stealing a typewriter to get money, right.
So all right, continue.
-[SPEAKING FRENCH] [YELLING] - [YELLING] - - - - - [END PLAYBACK]
One thing I should say immediately is that it is that this scene alone would give you a very false sense of the film, if this was all you ever saw of the film.
And the fact is it comes as a tremendous shock when you're watching it.
Although you enjoy it when you're watching it, you're wondering, why has Truffaut spent so much time on these faces, so much time on these children?
We never go back to such a scene.
None of the little children's faces that we see there do we ever see again in the film.
Something happens to the movie here.
It's almost as if it gets sidetracked.
It's interested in the boy and in his friend.
And when we're watching this scene, it certainly occurs to us in the film, my goodness, the scene might be useful, but why has it gone on so long.
And I think the simple answer is that Truffaut, the camera, became preoccupied by the astonishing variety and vivid individuating complexity of those children's faces, that the drama of that became so interesting to the camera, to the filmmaker, that he couldn't quite desert it without giving it its just due, even though from a narrative standpoint it doesn't really advance the story very well.
But that moment, when we do cut back to the two older boys sitting in the back, talking about stealing something, does justify this scene.
Why?
What does it show?
That they're separated from innocence.
That they're too old for this.
They're sitting there, everybody else in the audience is absolutely rapt.
Now they're a little younger.
The implication is these boys can't participate in the joy of innocence any longer, that they're already too old for it in some sense, and almost because of their own behavior, the choices they've made.
So the scene does further a central element in the film, clarifying our sense that especially the protagonist of the film is being wrenched too quickly out of childhood, is being forced out of childlike circumstances into circumstances that no child should have to should have to face.
So it does in some sense dramatize that.
But it dramatizes that in a very imperfect way if that were its primary purpose.
And what we have to say is, no, that's not its only purpose.
The film is stopping here and watching this joy in these children for the same reason that Jean Renoir stopped to watch Boudu do his tricks in the water-- not because it was furthering any plot, but because the spectacle of Boudu's pleasure became what was interesting that the film, to the filmmaker, and to the camera.
And so one way to understand the difference between these forms of new realism, these kinds of neorealism-- the Italian and the French-- is to say that the French is more open to this lyrical joy, that there's a lighter tone, that the social and political dimensions of the story are not obliterated in Nouvelle Vague films.
But they play a lesser role, they're less central.
There is no programmatic social message in most of the Nouvelle Vague films, even though there is by implication-- as you'll see in The 400 Blows-- certainly a powerful critique of adult society and the ways in which disempowered creatures like children are treated in modern society.
And that theme is there, it's at the center of it, and there's no question that it's significant.
But it is surrounded by other complicating attitudes and things that make the version of new realism that the French develop in the decade beginning around 1959 different in tone from its origins.
Well, let me very quickly say a little bit about the origins of the Nouvelle Vague.
You're prepared for this.
And if I showed you this film without any background, and I said, where does this come from, my hope is that most of you would say, oh, I can see Italian neorealist elements here, I can see Renoirish elements here.
A really acute and perceptive student might even see that this line goes back to Chaplin.
And Chaplin's films are committed to forms of realistic representation that is in a line, I think, with these later, more complex films, more visually complex films.
And in fact, there's an allusion to Chaplin in tonight's film.
I hope you'll watch for it.
One of the teachers actually plays a Chaplin-like role at a certain point.
And there's no question that the film intends to invoke the joy and pleasure of the Chaplin character in the memory of its audience when those allusions occur.
So the origins of the Nouvelle Vague-- first in Jean Vigo and in French poetic realism, in the practices that are developed by John Renoir and other directors of the time, that kind of lyrical, improvisatory open camera of Renoir, and then the way in which those are adapted by the Italians, very much influenced by the French.
So those traditions are at the center of the Nouvelle Vague.
And the primary practitioners of the Nouvelle Vague, directors like Truffaut and Jean-Luc Godard, and Alain Resnais, and Jacques Rivette, and a number of other directors-- Louis Malle, a little older, one who would make magnificent films.
It's a wonderful, wonderful group of really significant directors.
Most of them still alive, but in their dotage now-- even older than I am, if you can believe that-- and with magnificent careers behind them, all starting in this moment of the late '50s when French film took a new energy from the critical discourses of the critics who were working for Cahiers du Cinema, which I'll talk about in a second.
So one central source for the Nouvelle Vague are these earlier film traditions.
And the directors who made the Nouvelle Vague were quite self-conscious about this-- many of them had spent a decade before they became directors writing for Cahiers du Cinema, the great magazine founded in 1951 by Andre Bazin, the critic I've talked about earlier in the course.
And probably Truffaut was the most well-known of the critics who worked for the magazine.
Truffaut spent at least seven or eight years working for Cahiers du Cinema before he made his first film.
And in their time working for Cahiers du Cinema, they actually articulated certain theories of movie making.
One of the theories that they articulated, or an aspect of their overriding theory, was articulated in an article-- a very famous article-- written by Truffaut himself before he became a director, before he became an actual director.
And the title translated went something like this-- A Certain Tendency of French Cinema.
He published this essay in Cahiers du Cinema in 1954.
And it was an attack on what he called the tradition of quality, in quotes the "tradition of quality" in French movies.
It attacked contemporary French movies for being stiff, for being too literary, for being too much of the establishment.
And the other thing that article, and many other articles in Cahiers du Cinema did, was they began to write very favorably about American studio directors, the directors who worked under the Hollywood system most effectively, among them especially Hitchcock.
And in fact Truffaut did a series of interviews with Hitchcock later published as a book, Truffaut on Hitchcock, or Truffaut/Hitchcock.
I forgot the exact title in English.
It has a different title in French.
It's a very interesting series of interviews.
And those of you who are interested in this might want to look at it.
Because one of the comical things that happens in the interviews is you constantly see Truffaut, who's much more articulate and theoretically subtle than Hitchcock is-- abstractly, I don't mean Hitchcock's not a great theorist, but he's a practicing theorist-- and you constantly see Truffaut trying to turn Hitchcock into an intellectual and Hitchcock refusing, Hitchcock saying, no, no.
It's almost as if Truffaut is disappointed that Hitchcock can't articulate the theories that Truffaut wants him to articulate.
So Truffaut is constantly trying to make Hitchcock into a great director, and Hitchcock is often in the interview saying, I make genre movies, leave me alone, I just want to entertain people.
So it's comical.
And in fact I do think that the French inflated-- Truffaut especially-- inflated Hitchcock's reputation, maybe even beyond what it actually deserved.
But in any case, it was subversive of the Cahiers du Cinema critics to write about American directors, studio directors-- because the American studios were held to be the essence of commercial popular art, nothing artistic in it at all.
And in fact it was these French critics who first began to write about American directors in a way that showed respect, recognized their complexity.
It's a great irony that the French recognized the artistic value of American movies before the Americans did.
And at the very time that the Americans were swooning over European art cinema, this is the period when the Nouvelle Vague appears, when I was in college-- in the late '50s and early '60s.
And it was an extraordinarily exciting time in the United States for people interested in movies, not because of the American movies that were being made, but because there was this astonishing-- what seemed like an astonishing-- flower of European and Asian cinema.
It's also the moment of Kurasawa, as I'll mention next week when we talk about Rashomon, and Kurosawa's intervention in world cinema occurring earlier in the 1950s, at the same time as the Italian neorealists.
So it was an incredibly exciting time.
And educated Americans for the first time began to realize that the movies weren't just popular entertainment, but were serious works of art.
The irony is they recognized this about European films but not about their own films.
And it took another generation really before a serious attention to American movies was paid by American scholars and American critics.
Interesting irony.
A further articulation of some of the theoretical underpinnings of the Nouvelle Vague came from the director and theorist Alexandre Astruc, born in 1923.
He was good director, admired director.
But he's most well-known as a critic, a theorist.
And his central theory, then elaborated and embraced by generations later including Bazin and the writers for Cahiers du Cinema, was focused on the term camera-stylo, camera-style, camera hyphen pen, as if what he imagined was that the camera should be wielded by the director with the same subtlety as the writer wields his pen.
And It was an idea that the visual style of a film could have the suggestiveness and subtlety of a literary work.
And when Astruc articulated this, it was rather a revolutionary idea.
And it was elaborated, further picked up.
And the directors of Cahiers du Cinema were very powerfully in favor of what came to be called the auteur theory-- author, that's French for author.
And the auteur theory essentially is that the director of the film is an author who leaves his signature in every frame of the film.
It's a radical over-simplification of course, because films are such collaborative enterprises.
But it is still nonetheless true that it grants to the director a kind of respected authority that is certainly justified in the case of many, many films, where the director is the dominant and central creative energy.
And this of course has been especially true in Europe, and even more especially true in France, perhaps, than in any other society.
So auteur was a very important underpinning, encouragement to the Nouvelle Vague.
So when these critics-turned-movie makers began to make movies, they had already made theory for why they were doing what they were doing.
They were hostile to certain forms of pretentiousness in contemporary French films, they wanted some of the energy and power to entertain that they associated with the most gifted American directors, but they also wanted films that expressed a signature individuality.
They thought films ought to be and were the expression of the sensibility of their director.
And the film you're going to see tonight certainly embodies those values.
One way to think about the moment at which the Nouvelle Vague emerges in both French and in global culture is to think of the year 1955-60-- I used to say just '59, but there's some dispute about when Jean-Luc Godard's film actually first appeared, whether it was '59 or '60, so I've added 1960.
But these three films appeared within a very short time of each other in 1959-60.
And all three of them are thought to be sort of the origins of the Nouvelle Vague, the moment when the Nouvelle Vague declared itself as a major film movement.
But Hiroshima, Mon Amour was a dramatically important film when it first appeared.
In the foreground of the film is a story of a love affair between a Japanese architect and a French woman.
And the love affair is in the foreground.
But in the background is Hiroshima and the dropping of the bomb on Hiroshima.
And the story of the lovers-- I don't mean that it takes place while Hiroshima is going on.
Hiroshima is remembered and invoked in the course of the film.
So the foreground of the film are the lovers.
The background of the film is this horrific event, the atomic bombing of Hiroshima.
And the film plays with our sense of time.
Its sense of the passage of time is confused in some sense.
It's not linear in the traditional sense of an ordinary well-made film.
And there's not exactly a surreal quality in it, but there's a quality in it that we might say that one of the things that happens is that time is sometimes treated subjectively rather than objectively in the film.
And so the time frames you're in are often mixed or unclear in the film.
The film was also much more explicit with its nudity than American audiences were used to, and it caused quite a scandal when it came to the United States.
I remember the people who were in college with me at the time were very excited to see it, partly for that reason.
Nudity was much rarer in the popular culture when I was your age than it is today.
An even more dramatic and radical movie was Jean-Luc Godard's Breathless, which tells the story of a third-rate criminal played with astonishing panache by Jean-Paul Belmondo with a cigarette stuck in his lips through most of the film.
And there's one moment in the film where Belmondo walks up to a movie marquee, I think, outside a movie theater.
And he sees a poster advertising a film with Humphrey Bogart.
And he looks at the film and he tries to imitate a Bogart move-- he goes like goes, puts his arm, Bogart had a move like this.
And he says something like this, Bogie he says in his French accent.
And you can see in a certain way that he's patterning himself on Bogart, the Bogart who played ambiguous, sometimes criminal, characters in the movies.
And it's one of the earliest examples of something that becomes really common in America, for example in the great television series The Sopranos, where all the gangsters are constantly invoking The Godfather and other fictional gangsters, and modeling themselves on them, or quoting scenes from them.
Well, Belmondo's character in Breathless is one of the first gangsters to pattern himself on gangsters out of popular culture, instead of being a real gangster.
And there's a moral ambiguity in the film that is never fully resolved, because the protagonist of the film is this-- we're not going to show that, James, we don't have time for it.
I was going to show you a clip, but there's no time.
It's quite a remarkable film.
Some of you might want to look at it-- it's available from the film office.
And the central ambiguity is that we identify in some degree with the protagonist-- we follow him all the way through the film.
Very early in the film he actually commits a murder-- it's an accident, he doesn't intend it, it just sort of happens.
But he still is a murderer.
He steals a car in the very opening scene.
And then he ends up murdering someone.
He's on the run-- and for the rest of the film, he's on the run.
He takes up with an American girl played by Jean Seberg who ends up betraying him to the cops.
And in the very final scene in the film, he's shot down by the police.
And we see him running down a street with a wound in his back, finally falling in a very dramatic way with his girlfriend coming and looking over him.
And the ending of the ending of the film is very strange, because it leaves the audience uncertain about what its attitudes toward what has happened ought to be.
Should we resent this betraying girlfriend as an evil character because she's sold her boyfriend out to the cop?
On the other hand, her boyfriend is an amoral murderer and thief who doesn't seem to care about anyone, much less her.
And yet there's a kind of charm and an erotic energy about the Belmondo character.
So the film's moral and psychological grounding is unclear.
And so it was very powerful and significant for that reason.
And then finally The 400 Blows, even more successful film in a popular sense than either of those two, appears in the same time frame.
And it won the best director's prize at the Cannes Film Festival in 1960.
And it put Truffaut on the map.
It made Truffaut a famous director.
And he never really looked back after that.
He went on to have quite a remarkable career.
Let me say a couple of things about the style and tone of the Nouvelle Vague.
Most of it applies to what we've already said about neorealism and about French poetic realism.
So there's no reason to go through a lot of it.
But let's jump to the bottom, to the items at the end of the list.
First, the improvisatory aspect of, let's say, Renoir's practice becomes even more pointed and central in many of the Nouvelle Vague films.
And there really is a sense in which they often would begin filming without having a full script, letting the story itself play out, discover itself as they were filming.
Tremendous amount of collaboration between performer and director.
So both plot and dialogue are often the result of a kind of inspired improvisation.
Even more significant is the commitment in the Nouvelle Vague-- even more powerful, much more powerfully then than in the earlier realisms-- toward use of a style we might call a discontinuous style, a jumpy style, an edgy style.
What's a jump cut?
A jump cut is an edit that takes place in the middle of an action.
A normal cut, or what we might call a classic cut.
If a character is speaking and making a gesture-- say the camera's on me-- and he says, I insist on this idea more powerfully than any other-- like this, right.
Well in a classic sense, you would wait until my fist came down before you cut.
But if you did a jump cut, it would say something like, I insist on this idea more than any other.
Or maybe not even let me get the word "other" out and it would cut to another scene.
It creates a sense of discontinuity or of jaggedness that reminds you that life itself is not so smooth.
And the other thing that the jump cut reminds you of is something else-- it reminds you of the presence of the editor.
It reminds you that you are watching a film.
It's a very quiet way of doing this, And these films have other, more explicit, ways of reminding you that you're watching a film.
But there is in these films not only the use of discontinuous editing and what David Cook in his A History of Narrative Film calls elliptical editing, in which things are much more compressed, much more elliptical.
Things are not always perfectly worked out.
Remember I've talked about this earlier, how in classical Hollywood style if I were walking toward the door they would show me walking.
When I got here, they might cut it, and then they might pick up the scene as I reached the door.
But in a Nouvelle Vague film, they might show me going like this, and the next cut I'd be in the next room or in bed with my girlfriend.
They wouldn't bother to show-- in my case, it would be my wife, please, important.
But they wouldn't bother to show that progress, because it's silly, it's unnecessary.
That's what is meant by elliptical editing.
And it makes a greater demand on the audience.
But it also has the effect of creating a continuous, quiet sense of what I call self-reflexiveness or self-awareness in the film.
That is to say, you are aware at almost every moment of these Nouvelle Vague films that it is an artifact, that the film has been made by human hands, that various strategies and gestures have created these effects.
And that distances you from the material in some sense.
But it opens out a new topic.
And that new topic is film itself.
And many of the Nouvelle Vague films-- and especially the films that come after the Nouvelle Vague, but influenced by it-- turn out to be films that one of their topics is the making of movies themselves, films about film.
And there are a couple of wonderful examples of this in Truffaut's own corpus of work, including a very lovely, gentle film called-- a light film, not as profound as his most powerful films, but a beautiful, beautiful film-- the French title is La Nuit Americaine.
But it was translated in the English-speaking world as Day for Night.
If you want to create the illusion that you're filming at night, you can put a filter on the camera.
And even though you're filming during the day, it will look nighty, it will look dark.
It's a strategy of directors of photography.
And it's an allusion to that.
It's called The American Night because this was a strategy that was especially associated with American movies, thought by the French-- especially by the Nouvelle Vague directors-- to be shocking and ridiculous, because you should film on location.
You should minimize the falsity that is involved in setting up filters for your film-- faking that it's night when it's really not, or faking that it's day when it's right, that kind of thing.
But the film, La Nuit Americaine Is about the making of a film.
And in it Truffaut himself I think appears.
He plays the director who is having trouble making a film.
And is the film is interesting about character.
It studies the psychological circumstances of its central characters with great subtlety and interest.
It says powerful and interesting things about the erotic and personal connections among serious adults.
And it also says something about what is involved in the making of a film.
It's about making movies.
Its subject matter is the making of a film.
That level of self-reflexiveness or self-consciousness is always present in at least some degree in these films.
And sometimes it can become an explicit central topic.
These films, as I've already indicated, are full of allusions and references to earlier films.
And I mentioned the references to Chaplin-- you'll see other references like this, I think, in the film.
And one thing that happens in this film, for example, is that the characters go to the movies.
There's a moment when the family goes to the movies.
And the film that's playing is the title of a film by a friend of Truffaut's, Jacques Rivette, a film called Paris Belongs to Us.
And it's a real film.
So it's a kind of allusion in a way, to another film, to a friend.
But again, there's a level of self-reflexiveness in which that sequence in the film is also a meditation on the role of movies in social life.
I'll come back to that in a second when I talk a little bit about The 400 Blows itself.
So let me say a word about Francois Truffaut.
I hope you'll read more deeply about him in Wikipedia and in other places.
His biography is very rich and interesting.
One decisive thing to say about it is that in some sense The 400 Blows is a deeply autobiographical film.
That is to say, Truffaut himself was born out-of-wedlock, lived with his grandmother.
Not all the details are exactly the same in the film, but these details I'm giving you will show you how closely the film mirrors the reality of Truffaut's own life.
He was raised by his grandmother until he was eight years old.
Then he lived with his mother and his stepfather, who gave him his name.
And you'll see that that's an issue in this film too.
The father of Antoine Doinel, the central character, is his stepfather, not his real father.
Lived with his mother and his stepfather until he was 14.
Was constantly truant from school.
His father ended up turning him in, and he spent time in a reformatory.
To escape his parents and other miseries, he joined the army at the age of 18, but he hated the army.
He was constantly looking for a chance to desert.
He finally did desert the army, and he was arrested.
He spent time in prison for desertion.
And he was really in big trouble.
But he had always loved film, even as a kid.
And as a 14-year-old in Paris, he had found his way.
You'll see that this is replicated in The 400 Blows, because the attraction of Parisian movies is one of the great escapes for this boy and his friend.
So in his real life he loved movies, and he went to the French bibliotheque to watch films.
And there he met Andre Bazin, who was already an eminent critic and was the co-founder of Cahiers du Cinema.
Bazin intervened in his case, apparently used his influence to get him either released from prisoner or have his sentence reduced-- I don't know the exact details.
He went to work for Cahiers du Cinema.
And the rest is history.
He worked for something like eight years as a critic for Cahiers du Cinema and then began to make his own films.
And one of the most significant things about his life as a critic is that he was famous for being a nasty, incredibly unsympathetic critic.
He was famous for the viciousness and unforgiving quality of his reviews.
And think what it means that such a person should then, after eight years of doing this and making enemies all over the French film world, should take the risk of beginning to make a film on his own.
In many ways very bold of him.
But it's also true that the Cahiers du Cinema crew, as we might call it, including the great eminence of Bazin himself and a whole bunch of other ambitious young critics would-be directors, they constituted a kind of critical mass of folks with shared ambitions.
So I don't want to make it appear that what Truffaut was doing was shockingly brave, or courageous, or self-destructive.
But it was a dangerous thing, because he was certainly opening himself to revenge reviews by other people.
And he was really infamous for being nasty to other directors, especially to French directors.
You'll see on the outline that I've listed some of his most significant films.
And the films that have asterisks next to them, those five films, are all about the same character that you're going to see in tonight's film.
They're all about Antoine Doinel.
And it's a unique film record.
It's a series of autobiographical films, played by the same actor, who is a kind of Truffaut stand-in.
And the second film that I have on the list, or rather the third film, Antoine and Colette, I have it in quotations because it's not a feature-length film.
It's an episode from an anthology film that appeared in 1962 called Love at Twenty.
It was a very flattering thing for the young director to be asked to contribute to that.
And so that's another installment of the story of Antoine Doinel.
And then the next one takes place in 1968, Stolen Kisses, again in 1970-- Bread and Board.
And then finally, nine years later, the final Doinel film, Love on the Run.
It ends in an ambiguous way, like all of the films.
And one of the wonderful things, Jean-Pierre Leaud-- L-E-A-U-D, I don't know how to say it, Leaud, Leaud-- who plays Doinel, you can see him aging through these films.
It's a wonderful sequence.
And if you like The 400 Blows, you might want to watch the other films.
The 400 Blows is probably the best of all of them, but they're beautiful films.
And what they show is, in some sense, Truffaut returning to this autobiographical theme at different stages in his life.
There are even some scholars who have suggested that there's a kind of analogy, or a mirror relationship, between the young Jean-Pierre Leaud, the 14-year-old who plays the central character in The 400 Blows, that there's an analogy between his relationship to Truffaut and Truffaut's relationship to Bazin, as if what's happened is that Truffaut with his new young actor is re-enacting-- but now on the other side-- the mentor-mentee relationship that Bazin and Truffaut apparently enjoyed as well.
I don't have time to talk about this remarkable list of films.
The only film on this list that isn't wonderful, I think-- certainly worth looking at closely, maybe two that are not absolutely first-rate-- is the one in 1966, Fahrenheit 451.
I mention it to you because it's such an interesting example.
Based on the Ray Bradbury.
And it is his worst film.
It's his first film in English, he didn't know English very well when he made it.
It's an oddly heavy-handed and wooden film.
But I still have some kind of affection for it, even though it's not a very good adaptation.
Because he was attentive to Ray Bradbury, attentive to the ambitions.
It's a film that's theoretically much more interesting than it is in practice.
But all the other films are very remarkable films and among the most significant films of their day.
And let me just say one final word about the last film I've listed there, The Last Metro, which starred Catherine Deneuve and Gerard Depardieu.
In a way it's a World War II film.
It tells the story of a Jewish director who has to hide in the hidden basement of his theater while his wife, who is not Jewish, is able to run the theater above ground.
And it tells the story of the importance of theater even during the war years, when Paris was occupied by the Germans.
It's a very powerful and moving love story, as well as a story that celebrates the power of theater and the power of art in hard times.
And it's a lovely, poignant, powerful film.
I want to say a few words about the film itself.
It's fairly straightforward, and I think you'll absolutely enjoy it.
First the title, weird title, The 400 Blows.
It comes from a French idiom.
And the idiom means something like to sow your wild oats, to raise hell.
[SPEAKING FRENCH] To do or to make the 400 blows means essentially to raise hell, to do a walkabout, something that young people do.
And of course the title is very resonant.
You end without any certainty about what the resolution of this wild behavior is going to be.
The second important thing to say about the film, something I've implied earlier, is that it's a film that loves Paris.
And in a certain sense the boy's odyssey through the film is an exploration of Paris and of Parisian delights.
And again, think of what it says about the city, about this great city, in the time that the film was made, that boys of this age could wander around so safely and engage in so many remarkable adventures in the course of an ordinary day in Paris.
So the film is attentive to the feel, the texture, of Paris, in something of the way that some of the Italian neorealist films were attentive to the physical texture of Rome, or of other Italian environments.
The family romance at the heart of this film is one of Truffaut's subtlest achievements.
And I just want to call your attention to how subtle it actually is.
Watch how the story unfolds, and how in a very brief and elliptical way we come to understand the motivations and the unhappinesses of each of the major characters-- the mother, the father, and the son.
What we discover fairly early is that the mother isn't that happy with her husband.
She feels that she's losing her beauty, she feels trapped in an unhappy marriage.
The husband is to her a kind of boring character.
He's the boy's stepfather.
Early in the film, we get a wonderful sequence.
Remember, watch for the principle of multiplicity there, because so many things are going on simultaneously.
The physical, and social, and economic environment within which the family lives is dramatized in those opening scenes.
The boy's relation to his family is dramatized in those opening scenes.
And the father's relation to the mother is dramatized in those opening scenes as well.
And what we discover, of course, is that the father is actually-- even though he's the stepfather-- actually feels great affection for the boy.
And they get along very well, much better than the boy gets along with his mother, who is actually his blood relative.
And the mother's resentment partly has to do with the fact that she must feel that the boy has trapped her.
And she's entering middle age, she thinks she's losing her beauty, she thinks she's chained to a child and to a marriage.
And you can feel this.
And there are certain moments in the film when the mother, for various reasons, is embarrassed over the fact that she seems to have been a bad mother.
And she uses various strategies to reach out to her son.
And some of them are kind of creepy, as you'll discover.
She's a very complex character, and the least sympathetic character in the film, because she behaves in many ways so shockingly toward her son.
But even she, the least sympathetic character in the film, is treated in the film with a kind of complexity that we expect from adults.
In other words, we don't think that she's an evil villainess who would like to suck the blood of her child.
Not at all, not at all.
What you feel is that she's driven to her hostilities and her unhappinesses by the confined and unhappy circumstances of her life.
Not that she's totally forgiven, but she's understood by the film.
Something of the same kind of thing is true for the husband, who loves his wife and knows that his wife is probably dissatisfied with him, may be unfaithful to him.
And that colors his relation to his son, with whom he often feels a great connection.
So there's a great poignant sense that this dysfunctional family is less happy than it needs to be, if only it could understand its circumstances.
There's one wonderful moment in the film where some of the hostility disappears.
And it comes at a moment that I mentioned earlier where the husband and the wife go off to the movies together, taking their son with them.
It's like a family outing.
That's when they go to see the Jacques Rivette film.
And it's the one moment in the film where all three of the central characters in the film are happy together.
And you might ask, why?
Well, the father is happy because the wife is paying attention to him and is doing something with the family.
And the boy is happy because the family, the father and mother, aren't squabbling with each other and they're including him.
And the mother is at least partly happy because she can see both of the men are delighted by what's going on and because she likes movies too.
So one of the things that the film dramatizes is the importance of film in human life, is how film is a kind of escape or a respite from misery in certain ways.
And so it's a kind of commentary on the centrality of the movies as a source of information, insight, and consolation.
Pay attention to the way the film is structured.
It's not unlike what I've said about the organic form of certain earlier Italian and French movies.
The structure unfolds in a natural way.
We don't feel that plot is driving the story, we feel that what happens in the film happens out of a kind of natural unfolding that is a function of both the environment and the characters.
There's so much that I've left out here that I feel guilty about not having mentioned.
I should mention one parenthetic remark, just because it's such a vivid detail.
The film was made for a relatively small amount of money, even in those days.
It cost $75,000 to make this movie.
And in the United States alone it earned over $500,000, a gigantic amount of money in those days.
So it was an astonishing success in that way.
Let me end by saying one final word about the astonishing conclusion of the movie.
The ending is beautiful in its own right.
I mean there's a kind of visual beauty that's constantly competing for your attention as you watch.
The boy, as you'll see in the story, ends up in a kind of reform school.
His crimes are so modest and that's part of what makes the film so poignant.
He's actually caught stealing-- not stealing-- the typewriter that he was talking about stealing in that scene you saw.
He's caught when he's trying to return the typewriter.
He has an attack of conscience, and he thinks I shouldn't do this.
And he's caught returning the typewriter, and that's his big crime.
And you'll see, one of the things that happens to him when he's kicked out-- he runs away from his family at one point-- he spends nights in Paris on his own, this 14-year-old child, boy.
Very interesting again, about the way the film celebrates the city of Paris as a nurturing, adventurous, exciting, not really dangerous place.
But let me say one final word about the ending.
At the end very end of the movie, he kind of escapes.
He makes a kind of escape from this reform school that he's in.
And it's an immensely beautiful sequence, in which we see him running through a countryside.
And it's a tracking shot, an immense long tracking shot.
And the camera follows him as he's running-- running, running, running.
And it goes on, in some sense, something like the scene that I showed you, where the puppet show seems to be going on forever.
When you're watching the film, you'll see that it's strange, because the puppet show intervenes in the movie without explanation and disappears from the movie without explanation.
Something of the same kind of thing is true here, where the beauty of the countryside starts to compete for your attention.
But in any case, finally at the end of his run he comes he comes to a verge, to an end.
He comes to the water, he comes to the sea.
He always said he wanted to see the sea, but this is a very disappointing version of the sea.
And he runs up against it.
Watch how the music works.
There's a beautiful use of a stringed instrumen-- I'm not sure what instrument, maybe a guitar.
But anyway, a beautiful stringed instrument plays a melody that you've heard earlier in the film.
And we see him running up to this verge.
And then there's an astonishing moment in which he comes to a stop.
And he's looking at the water.
And the camera freezes on him.
There's a freeze frame at the very end.
It's the most famous freeze frame in the history of movies.
And after this film, the freeze frame became a cheapo trick.
You began to find it in television commercials.
But at the time that he used this frieze frame, it was an immensely dramatic effect.
I don't mean nobody had ever used such strategies before.
But it is one of the subtlest uses of a freeze frame in the history of movies.
And one of the reasons I want you to watch for it and pay attention to it, is listen to the music as it's going on.
Because the tune is finally unfinished at the end.
In other words, where one final note that isn't played.
But you've heard the note before, so you keep expecting it.
And that's part of the power of that freeze frame at the end.
I think he's looking at us when the freeze frame occurred.
But the point is, where is he going, what comes next?
The film doesn't really have an answer.
We don't know where the film is going.
We don't know what's going to happen to this boy.
So it ends on a note of open-endedness and ambiguity that's powerful and profound.
But what we also know is that this has not been an apocalyptic ending.
It's not a catastrophic ending.
The boy is isolated, the boy is alone, he doesn't know what's going to happen to him, he doesn't know whether he'll be recaptured, he doesn't know where he's going to go.
But I don't think we have any sense that he's going to die.
We don't have any sense that this is the end of his life.
We have a sense, maybe, that this is the beginning of his life.
So that there's a profound uncertainty, a kind of beautiful uncertainty, a beautiful ambiguity at the end of the movie, that is partly captured in the complex relation between sound and image at the end of this movie.
Those of you who have never seen The 400 Blows are in for a really great treat.
The final films in this course are among the greatest films ever made.
I hope you enjoy it as much as I did.
I still remember the place in which I saw this film and the exhilaration I felt when I came out of watching it, with the recognition that movies could be what great novels had been.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Kurosawa's Rashomon is a particularly dramatic example of a film that understands itself to have the kind of claim on its audience that the greatest art has always imagined itself to have on its audience.
So I want to begin by talking very briefly about what I call the moment of Rashomon.
There's a bit of confusion, or at least chronological confusion, or inconsistency in the principle that we end the course with a film that was made and shown internationally before the last two films that we've seen in our course.
My reasons for that, as I partly explained in an earlier lecture, had to do with my desire to show a certain continuity amongst forms of European cinema and the link between Jean Renoir, and the Italian neorealists, and the French nouvelle vague is so intimate that it seemed to me important to show you that progression in sequence.
But if we had been going by strict chronological order, we would have introduced this Kurosawa film a bit earlier, because it was made in 1950.
And in 1951, it won an important international prize, The Golden Lion, the highest prize available at the Venice Film Festival in 1951.
And this had a seismic effect on movies around the world.
The dramatic and powerful subject matter of Kurosawa's film of course riveted attention.
But even more than that, the freedom and imaginative energy of his stylistic innovations in the film had a profound impact on filmmakers around the world.
And when the film was shown at Venice in 1951, another effect it had when it won the prize was to introduce Japanese cinema to a wider world.
It was the first significant Japanese film, Kurosawa, the first important Japanese director to gain a reputation outside of Japan itself.
In fact, there are many film buffs, and especially specialists in Japanese film, who are somewhat resentful of Kurosawa's eminence, even though no one denies that he is an eminent director, because there are other directors.
The two I've listed under item 2 in our outline are the most dramatic examples, Mizoguchi and Ozu, who are often thought to be his superior, even greater directors than Kurosawa.
This is a debate of nuances.
All three of these directors are major artists.
But it is true, I think, and it is widely recognized that Kurosawa was the director who crossed that barrier more immediately, more dramatically than any other, and opened the world, not just to Japanese cinema, in some degree, but opened the world in some longer sense to Asian cinema more generally, that the so-called Western world, the European and American cinema universes had been fairly oblivious to Asian cinema and certainly to Japanese cinema prior to this.
And the appearance of Rashomon, its enormous impact in 1951, began to change that.
So that what was demonstrated in moment when Rashomon won this reward, won The Golden Lion at the Venice Film Festival, was a reinforcement of a principle I've been discussing throughout the semester, the notion of film as an international medium, the notion that directors from different national cinemas were now being deeply influenced by directors from other nations, and that film itself was in some deep way, a global phenomenon, even an international form.
And I think it was in the '50s and early '60s that this idea began to become more widely embraced by film goers in the United States and in Europe, but perhaps especially in the United States.
And one mark of this, the emergence of cinema as a fully recognized independent art form.
Obviously people had thought this, and many directors had achieved artistic distinction before this.
But I'm talking about the public understanding of movies, the way people in different cultures actually recognized and thought about movies.
It was as if this is the moment in which movies were understood to enter the museum in a certain way, to earn in a public sense, the status that more traditional art forms had had.
And one of the explanations for why this would have been so, why it would have had such a powerful impact-- now, I think I mentioned last time that this insight was partial in the United States-- especially, that is to say, in the '50s and early '60s, it began to dawn on movie critics and scholars of whom there were only a few at that time and then movie audiences that European films and Asian films, especially Japanese films, might have great artistic value.
But it was a longer time before Americans began to realize that their own native forms of films had had a similar kind of authority.
So this moment, in the early 1950s, was a deeply significant one.
Let's remember historically what it represented in Europe and in the United States.
It's the moment of the emergence of Italian neorealism, which itself begins to establish a kind of very powerful claim on people's attention.
One irony of Rashomon's success was that it was not very successful in Japan when it was released in 1950.
And the producer, the production company responsible for the film was very dubious about entering it in the competition, didn't think it was a significant film, even though it transformed Kurosawa's career because of the immense recognition it finally got.
And Kurosawa himself recognized-- he'd been making films for almost a decade before that, but Rashomon was his most ambitious film to that point, and it also incorporated more innovative strategy, visual strategies than any he had tried before.
It established him as an international director.
And I mentioned the names of two other directors just from different traditions as a way of reminding you of another feature of this phenomenon, another reason, as I began to say earlier, for why this moment was such a significant one.
And the term I use here is modernism, modernist cinema.
Remember, one of the ways to understand this idea is to recognize that a great revolution in the arts had occurred at the turn of the 20th century, the end of the 19th, and at the turn of the 20th century.
We've talked about this earlier.
It's the movement we call modernism.
It's the moment of Picasso.
It's the moment of James Joyce, and it was a kind of revolution in both visual art, literature, music took place in this period.
And among the characteristics of this modernist movement was a newly complicated and self-conscious attitude toward narrative itself, toward storytelling.
So modernism in literature and in art involved, among other things if not a hostility or antagonism, at least a kind of skepticism about inherited traditional categories and ways of doing things.
And one form this took in narrative was to dislocate or disorient the narrative line.
Instead of telling a story in a chronological sequence, a lot of the great works of fiction of the modernist era, books by writers like Joseph Conrad, or Proust, the great French novelist who was so preoccupied by memory and human subjectivity, or the great German novelist, Thomas Mann, a number of other great figures that we could mention began to construct stories in which chronological order was profoundly disrupted.
And they also began to create stories in which there were multiple narrators.
And the effect of multiple narrators begins-- even if you do nothing more than have multiple narrators, you begin to raise questions about the veracity, the truthfulness of any single perspective.
And you will understand when you look at Rashomon why this movie embodies many of these same modernist principles.
But the point is that cinema, as a narrative form, lag behind these more traditional arts.
And it really wasn't until the 1950s, and partly because of films like Rashomon, that it began to be recognized that the movies too could embrace and embody the principles of modernism.
So one way to understand what happened in the 1950s is to recognize that directors like Kurosawa and Ingmar Bergman, the great Swedish director, and Fellini, the great Italian director, and the inheritor and expander of the neorealist tradition, going far beyond a narrow realism, that directors like that began to create films that in a formal sense, in a structural sense, and also in terms of their content had the kind of complexity, nuance, and skepticism, and even the philosophic self-awareness that was characteristic of high modernism at the turn of the 20th century.
So it's as if what was going on was the movies themselves were now asserting themselves as a modernist art.
I don't mean as a contemporary art.
I'm referring specifically to the modernist movement, and to the dislocated, and much more demanding kinds of narrative strategies that are characteristic of the modernist movement.
So Rashomon played a fundamental role in this sort of transformation of what we might call the cultural understanding of movies among ordinary people, as well as among scholars, critics, and other filmmakers.
I want to mention one other point.
I'll give you a kind of note to clarify some of what I've been implying, some of what I implied when I talked about Mizoguchi and Ozu as directors who were often even more highly regarded than Kurosawa.
I'll leave that to each individual film goer.
All three directors are astonishing and remarkable.
But it wouldn't be appropriate to talk, even about this single film, Rashomon, without paying respects to those two great directors whose dates I've put on your outline.
I won't talk about individual films by these directors, but I urge you all to look them up, read about them in David Cook's history of narrative film, and think about experimenting by extending your knowledge of Japanese cinema by trying films by these two remarkable directors.
One of the things that's characteristic of all three of these directors, of Kurosawa, even more fully of Mizoguchi and Ozu, Ozu most fundamentally of all, is that their films are marked by a kind of impulse toward stylization, toward fabular, fable-like equations that distinguish them in some ways from Western, from European, and American films.
And I think that one explanation for this has to do with the longer artistic traditions of Japanese society.
Japanese film grows out of theatrical traditions, like kabuki theater, or Noh drama, N-O-H drama, both of which have profoundly stylized and fable like qualities.
They're anti-narrative, in some sense, and any of you who have ever had even a minimal experience with either of these two theatrical traditions will understand what I'm discussing.
These are theaters of gesture and of very decisive, symbolic representation.
What we would think of as sort of realistic characters or realistic stories are not a part of these very ancient traditions.
These theatrical traditions go back hundreds, even thousands of years.
So there's a tradition in Japan of a kind of stylized, of symbolic representation.
And you'll see, I think, how in Russia, how powerfully this principle operates in Rashomon.
Even when film itself emerged in Japan in the silent era, it emerged in a slightly different way.
And one of the most interesting features of silent film tradition in Japan was the appearance of a character who has no counterpart in Western cinema, a character called a benshi, B-E-N-S-H-I.
Any of you heard of it?
None.
Well, he essentially was a narrator and explainer, and he stood next to the movies in a way and gave explanations.
He said now, we will introduce the villain.
Now, we will introduce-- he was like a kind of intermediary, a narrator or a concierge who mediated between the audience and the text, who gave the audience information.
Again in one sense, we might think of it as an anti-narrative tradition, as a tradition in which things are presented or spoken rather than literally acted out, and certainly one in which the details of a story are less important than its general outline.
So when we talk about stylization, one of the things we're talking about is an impulse toward what we might think of as generalized argument instead of specific argument, an impulse to have one moment stand symbolically for many other moments, and what we might think of as a simplification or a distillation of reality into certain symbolic moments that are thought to be emblematic in certain ways, but don't necessarily have a realistic feel.
And you'll see almost instantly when this film begins, there's a kind of prologue.
And then when the film makes a transition into the first sequence that takes place in the forest, you'll begin to see what I mean when I say that the film seems to enter into a kind of symbolic realm in which your sense of reality is in some sense undermined, as if you're entering into a dream or a symbolic space.
Kurosawa, talking about that astonishing sequence at the beginning of Rashomon, said that camera's complex movements and the movements of a character himself-- everything is in motion in that remarkable opening sequence.
Some people have called it the most visually poetic sequence in the history of movies.
Kurosawa called this moment a moment in which the camera was shown to be penetrating into a space where the heart loses its way, as if you're penetrating into an ancestral space, into a space that's dreamlike in fundamental ways.
So the very opening of the film, or almost the very opening of the film establishes this kind of complexity.
I don't want to exactly call it an ambiguity, but this complexity about the nature of the reality that you're watching.
And this is even before the film proceeds to present essentially four different accounts of the same event, these four different accounts conflicting with each other in a variety of ways.
So these abstracting, or symbolizing, or stylizing narrative and dramatic traditions lie behind and shape the movies in Japan, even movies like Kurosawa's, which embrace the camera's freedom in a way that's much more characteristic of Western directors than of Eastern ones.
Ozu, the second of the two directors I've listed on your outline, is especially famous for holding his camera almost stationary for a tremendously long time.
And in fact, he's sometimes called a director who tries to create a zen aesthetic, because the camera is so quiet, and so stationary, and relatively inactive.
It's a style that lays tremendous emphasis on the nuances of facial expression and vocal tone.
And both Mizoguchi and Ozu do, in some sense, have an even greater sense of stylization in many of their films than Kurosawa does.
But I don't want to oversimplify, because they are also capable of very great, realistic moments, and they have a moral realism that's at least as powerful in their films as Kurosawa himself.
Kurosawa's career is a very remarkable one.
And I wish I had time to talk about it in detail.
Organizational structure of Japanese cinema was not unlike the structures that developed in Western societies in the United States or in France.
There were essentially monopolies of not a small number, but a relatively larger number of film production companies operating at different levels of significance.
So they were second rate, and then they were second level and third level production companies, as well.
But all of them operated in a similar way.
The director was a more dominant than major figure in this system, and surrounding each director were a group of workers and a group of creative people, including usually performers who went with a director from film to film, as well as his technical people.
They would often use the same people to write their music, and the same crew to work on the film-- if they could succeed, get the same cinematographer.
And Kurosawa's-- so Kurosawa's group was called the Kurosawa gumi, G-U-M-I.
It means the group, or cadre.
The Kurosawa group worked on a series of films.
I don't mean it was always identical.
There were changes, but it was a stable group unified especially by Kurosawa's vision and supervision.
And I've listed here a few of his most famous and fundamental films besides Rashomon.
Ikiru, maybe his greatest film, a realistic film set in the modern world.
The title means to live, and it's about a man who discovers that he has only a few months to live.
And it stars the actor Takashi Shimura, who plays the woodcutter in Rashomon.
The other actor that you'll see in Rashomon that is one of Kurosawa's favorites and appears again and again in Kurosawa's films is the actor Toshiro Mifuni.
Rashomon, he plays the bandit.
You'll see what a remarkable figure he is.
So I've only listed a few of his films here, but among his most important, Rashomon, Ikiru, Seven Samurai-- many people would say the greatest of all samurai movies, and probably the greatest of all Western movies, because it puts most American Westerns to shame.
It's influenced by American Westerns, as Kurosawa himself acknowledged.
And it was itself, that film, made in 1954, remade as an American film some years later under the title, The Magnificent Seven.
And it was so successful that a sequel was made, something like The Magnificent Seven Return.
And in fact, one of the deep features of Kurosawa's work is that many of his films have been remade by other directors, both American and European directors.
Rashomon was made 14 years later, remade 14 years later, with Kurosawa given screenplay credit in a film directed by Martin Ritt in the United States called The Outrage.
And it retells the story that's at the heart of Kurosawa's film.
It starred Paul Newman among others, and Edward G. Robinson, among other significant American actors.
Throne of Blood I mentioned, because many people see it as the most successful of all adaptations of Shakespeare.
It's a Japanese kabuki-ized version of Macbeth starring Toshiro Mifune.
And many people think of it as the greatest of all Shakespearean adaptations.
Yojimbo is a samurai film, a much more straightforward samurai film in many ways than Seven Samurai, also stars Mifune, and it has brilliant, brilliant sword fight sequences in it that anticipate the kind of thing that is now common in Asian cinema, but much less trivially done in Kurosawa's than in many of these later films that merely seem to want to entertain us by their sword play and the physical grace of their actors, but don't connect nearly so powerfully as Kurosawa's films do to a profound and serious historical setting and story.
Yojimbo was also made into an American movie called Last Man Standing, in 1966.
I mentioned Kagemusha, only because it's a later film, and many people admire it, because it shows that Kurosawa was working effectively, even in old age.
He made another film in 1985, one of his final films called Ran, R-A-N, which is a remake of King Lear.
And these two older films, later films, Kagamusha and Ran, show Kurosawa's visual sense, visual imagination to great effect, but they feel stylized in the way that they're-- stylized may not be the right word.
They feel abstract in a way that earlier, Kurosawa's films do not.
They are extraordinary spectacles, but they don't have the same interest in character, the same focus on character that his earlier films, despite their stylisation, seem to do.
I've saved most of my time to talk about Rashomon itself, because it's such a central and significant film.
And I hope when you watch it, you'll not be impatient, and especially that you watch for the ways in which from sequence to sequence, the visual style alters.
It's a very demanding film, in that sense.
Let's begin by talking a little bit about the problem of rape in cultural stories, because I think that one of the problems with responding fully to Rashomon is that we, especially in the Western world, are newly struggling with notions of gender identity and of the legacy of patriarchy that put us in a fraught and complex position in relation to stories like that of this film.
And I want to confront it right in the beginning.
As some of you may know, the story of Rashomon is the story of a rape.
There are four different-- a rape occurs at the center of the film, and there are four different accounts of what happened, of how the rape occurred.
And the film is partly a meditation on what motives do the different tellers have for putting this particular spin on the story?
And part of what's subtle and disturbing about the movie is that when the first testimony is given, it's not fully clear yet to us that we should be skeptical of the testimony.
And I think the first time-- one of the people whose testimony we heard is the murderer himself, or the rapist himself, the Mifune character.
He's the first one to testify.
And as he's testifying, it begins to dawn on an attentive viewer that maybe his testimony is self-serving in certain ways, that there's certain things he's saying that maybe we shouldn't fully accept.
And then, when the next account comes, our sense of skepticism is reinforced and fortified.
We begin to worry.
And then the film itself reminds us of the fact that these tales are problematic, because the film's structure is so interesting.
Roughly every 10 minutes or so, I've timed most of them-- a little less than 10 minutes in some cases, a little longer than some-- you'll have an extended narrative sequence which will last about 10 minutes.
Usually it's the testimony of one of the people appearing before the court.
And then after that happens, the film sort of shifts into another mode.
And the way you can tell is that it shifts back to the scene with which the film opens.
All the way through the scene, it's marked by this.
The structure of the film is marked by this return to a scene at Rashomon gate, which I'll explain in a moment.
So one point that I'm trying to get to here is the idea that as we watched the film and we begin to weigh the accounts that different people give of this rape, many of us are likely to feel uneasy and disturbed, because one of the things that disturbs me in the film is the woman's reaction to her rape.
She feels terrible shame.
It's as if she felt-- and there seems to be an impulse in the film that certainly some people have certainly gotten there.
At least they perceive an impulse in the film or an impulse in the narrative to blame the victim.
In some sense, what I'm suggesting is not that that response is inappropriate, but that it's a little bit off key, off center, because if you recall the idea that the film is deeply stylized, and it's set in an ancestral past, in a medieval Japan, in a moment of terrible social breakdown in which vestigial or ancestral attitudes towards sexuality and gender are being mobilized or awakened.
And if we understand it in that way, we can begin to recognize that our own discomfort with the subject matter is a discomfort that the film itself may even be aware of and may even be encouraging.
And as you're watching the film, watch how, in some sense, especially in one moment where the victim of rape makes an appeal to her husband right after-- there's a sequence where we see her embracing her husband and looking into his face.
Now the problem is she's giving this testimony, and there's some reason.
It's after the fact, and there's some reason to doubt what she's saying, especially as the film goes on.
Nonetheless, it's a moment of great power.
And that moment at least mobilizes a sympathy for the victim of rape.
That is very significant, because you hear so little of it elsewhere in the film.
Not that the woman is treated badly, but she's subjected to the same suspicions as the other central characters.
But there's a larger thing to think about, a larger way in which we can accommodate ourselves to the slight discomfort we might feel at turning a story of rape into a philosophic discourse as this film does.
And here's how we might do that.
Let me just remind you that stories about rape are at the heart of many cultures.
How many of you have heard of the story of The Rape of Europa?
It's a Greek myth.
None of you?
In many ways, it's the story of the foundation of Europe.
Zeus disguised as a white bull, the great Greek god, the god of all gods.
One of Zeus's best habits or the most remarkable habits in these mythological stories is that when he gets a yen for a human female, he will disguise himself as a creature of the earth and go down and rape her.
And he does this with Europa.
The rape of Europa is a kind of symbolic story which later Europeans actually took as one of the founding tails of how Europe itself was founded.
Can you think of another story in which Zeus was a rapist?
How many of you know the story of Leda and the Swan, about which Yeats wrote such beautiful poems?
Again, Zeus, the god of gods, disguises himself as a great spawn and swoops down on lead of this beautiful woman and rapes her in the guise of a swan.
And there's a brilliant, almost pornographically powerful poem by W. B. Yeats in which he describes this terrible moment of rape.
It's one of the great poems of the Western world, and it's about this rape.
So what I'm reminding you of is that the misogyny, that you may sense there is a misogyny that's embedded in culture.
It's a misogyny that's embedded in all the stories that human beings tell, in many of the stories that human beings tell themselves about the world, about the relations of men and women, and often, especially about the foundations of society, so that this meditation on human frailty and human deceit focused on a rape from that perspective is one of many such stories.
Not a unique object at all.
And it seems to me that that's one of the ways in which we can recognize that what Kurosawa is doing is part of a long, and complex, and in many ways, very disturbing habit of mind that many, many cultures share.
The title-- Rashomon.
Western students are often puzzled by it.
It's a reference to the name of the gate, but the word gate is complicated too, because it's not an American gate that just opens and closes.
It's a great, massive entrance to the city of Kyoto in the southern part of Japan in the late 11th or early 12th century.
It's a period of complete disillusion and destructive poverty, political chaos.
And the broken down condition of the gate, which you get long shots of, you see this massive structure.
There's a terrible rainstorm going on, under which certain people come to get shelter from the rain.
And that is Rashomon gate.
And it's broken down condition symbolizes the broken down condition politically and socially of the society that is represented there.
And again and again, the characters gathered beneath the gate to protect themselves from the weather and gauge in conversation about human nature.
Are human beings innately evil?
Do they always lie?
Can we never trust them?
And one of the characters who carries on this discourse is a priest who has an idealizing tendency, which another of the character's a commoner.
He's called the commoner.
He's an ordinary man is constantly mocking and arguing against.
It's almost a kind of argument that reminds me in some ways of the argument between spirit and flesh in Cervante's Don Quixote, in which Sancho Panza is constantly reminding the idealizing Quixote of the miserable actuality of the world.
Look, when you get stabbed, you bleed.
When you haven't eaten, you're hungry.
The world is real in a way and miserable in some respects in a way that idealists don't like.
And so that's a kind of argument that runs through these interludes as the film goes on.
So the title refers to the Rashomon Gate, and Rashomon Gate is itself a massive symbol for the breakdown of order for the miserable circumstances that individuals find themselves in.
And one of the things you'll see is that it's chilly.
It's cold.
It's raining like mad, a tremendous torrent, a downpour incidentally created partly by fire trucks.
In his autobiography, Kurosawa talks about how difficult it was to create this sense of an immense ongoing, almost a tsunami of rain, and he talked about the technical difficulties of doing so.
Very impressive rain, the most impressive rainstorm in the history of movies, I think.
So these people are gathered beneath the gate in order to protect themselves, and the gate's symbolic significance is important.
We will notice that one of the things they do when they get cold is they go over to certain parts of the building.
It's a wooden structure already half broken down and in decay.
And they'll break off banisters or other pieces of wood, and break them up, and burn them up.
And the implication is if things go on like this, pretty soon the whole gate will have been consumed by people who have tried to take shelter under it.
So it's a symbol of the breakdown of social order and of the society.
I've already mentioned the medium.
The Japanese word is miko.
And I mentioned it here, just because I wanted to be sure all of you understood what was going on there.
The husband is dead when the testimony begins.
He's a samurai who was the husband of the rape victim.
And as the story unfolds, you'll get the basic facts, but even when the film is over, there are many fundamental things you won't be able to have decided.
And I think that's certainly part of Kurosawa's point.
So the medium is just this clairvoyant type apparently real characters believed in and socially recognizable in late medieval Japan, a character who claims to have access to the words and beliefs of dead people.
So the dead man testifies.
And another way of reminding you that we're looking at a very stylised, a story that isn't in a narrow sense, realistic at all.
The visual style of a film is especially remarkable and astounding, in some ways.
It's almost as if each form of testimony has its own style.
And you might want to watch the way in which Kurosawa builds his eclectic and dynamic way in which Kurosawa's editing camerawork use of music combine to a kind of almost constant visual excitement.
One of the most remarkable things about the film is how many sequences in it are without dialogue-- extended, wordless sequences, truly entirely cinematic.
The opening sequence-- almost the opening sequence-- the first extended sequence in a forest, which comes after the sort of introduction, which I've described earlier, is a magnificently clear example of that process.
And one of the things that you may notice in that sequence especially is the way in which you become increasingly disoriented about the direction in which the woodcutter is going.
He's apparently narrating the story, and his narration sort of segues into a visual experience, as happens again and again in the film.
And the visual experience we have shows him going into the woods, walking, and then discovering first the woman's hat, and then discovering other things, and discovering a body, and then running away in fear.
And as he penetrates into the woods, one of the things that happens is the camera is always moving.
And the camera becomes as interested in the forest itself, in this densely wooded forest and in the play of light and dark, because the sunlight comes through the wooded canopy in odd and profoundly visually powerful ways.
You begin to have a sense that the camera is at least as interested in the woods and in the play of sunlight as it is in the motions of the woodcutter.
And the whole sequence has a kind of profoundly lyrical, but also in some degree, disorienting sense that as Kurosawa said, you're entering a space that's dreamlike, that's dangerous, a place where the heart will lose its way, as if you're entering a symbolic space, not a realistic space-- a stylized space in some deep way.
And there are a couple of specific strategies that Kurosawa uses in the film to reinforce, I think, our sense that he's engaging every element of his cinematic palate in order to create his effects.
One thing he does, he violate certain rules, especially at the time where it would have been tremendously shocking to professional directors.
One thing he does in the film was he points the camera at the sun, and he creates sun effects.
That was a no-no.
It was a sort of a rule that directors should never do that.
Kurosawa does it.
And you watch how he does it.
It's very powerful.
It also has a disorienting effect, the effect of making us understand more deeply what it's like to work our way through the incredible dense forest in which the crime occurs, as if the forest itself is a space so complex and so private, so cut off from the outer world that almost anything could happen there-- a space of dream, a space of terror, a space of symbolic fable.
And there are a couple of other things I wanted to mention about the way his camera behaved.
One is that Kurosawa uses here a device at certain points in the film, a very interesting device.
The official name for it, the fancy name for it, the technical name for it is he makes what is called an axial cut, A-X-I-A-L.
It's really a form of a jump cut-- that is to say, an abrupt edit which you're not fully prepared for.
A jump card, as you know, breaks the action in mid-stride, or in mid-action, and then jumps to something else in a way that's slightly disorienting that eliminates.
It's elliptical.
It eliminates connection or transitions.
But the axial cut does this in a very dramatic way that also calls attention to the apparatus of the movies.
The most dramatic places in which this occurs in the film are certain scenes in which you see the samurai husband tied up, sitting on the ground, tied up like this, kneeling on the ground.
And the camera's at some distance from him and moved toward him, but it doesn't move toward him in a smooth trucking motion characteristic of most films.
What it does is it moves forward, and it stops, and then it jumps.
It moves forward.
And what you feel is it leaps forward.
And what's happening, of course, is that he stops the cameras forward movement, moves it further, makes a cut.
So the effect is the camera moves-- not that the camera's jerky, but it's as if it's speeded up in some sense.
We can feel that the camera is becoming elliptical.
So say this fellow in the front is the person I'm focusing on.
I'll be here.
You'll see this shot.
And then you'll see this shot, and the effect is very abrupt.
Watch how it happens.
One effect, one consequence of this kind of a shot is that watching it, you can feel how mechanical it is.
You begin to think to yourself well, how could that have been created?
You're aware of its mechanical qualities.
That is to say, you become partly aware of the apparatus behind the making of the movie.
It's a moment of self-consciousness that other elements on the film also reinforce.
So the visual style is profoundly eclectic and dynamic.
I've mentioned the axial code in pointing the camera at the sun.
Maybe I'll mention one other device.
One of the other technically intricate, and at the time, revolutionary thing that Kurosawa did was he violates what's called the 180 degree rule.
And the 180 degree rule essentially has to do with your sense of spatial orientation within the frame.
Essentially the 180 degree rule holds that if you're showing characters moving in this direction, so you're showing a character moving this way, you won't suddenly, if you're still going in the same direction, show him walking this way, because it disorients the viewer.
In our film, in Rashomon, there are certain moments.
There are hints of it in that opening sequence, that lyrical, first sequence in the forest that I mentioned in which you can see that the camera's own movements complicate, and in some sense, confuse our sense of where the woodcutter is going.
And it's in that sequence and some other places in the film as well, where the 180 degree rule is violated.
And the effect again is to disorient us, is to feel gee, I don't know whether I'm coming or going.
This guy doesn't know whether he's coming or going.
What kind of a space is he in?
Again, violating certain conventions of traditional filmmaking in order to create new effect.
And the consequence of these choices, the impact of these choices in 1951, when the film won its prize, was profound.
I want to say one other thing, another aspect of the film's structure, which I've described him perfectly.
And I apologize for being so tongue tied about it.
But as I tried to describe earlier, the basic structure of the film becomes fairly clear.
What happens is you get testimony.
Then there are interruptions in which you-- essentially, all four of the primary pieces of testimony takes place in the past.
So what we have are flashbacks, but competing flashbacks.
And at various points, the film returns to our scene of reign at Rashomon Gate in which the people under the gate, the three people under the gate-- two of them are actually partial participants.
The third, the commoner, is just a kind of listener to the story, although a profound commutator on it.
The Sancho Panza type who says, look.
The world is miserable.
Why should you believe anyone?
And the priest is constantly resisting him.
Well when we return to these moments-- so we return to Rashomon Gate several times, many times in the film.
And every time we return to that spot, where are we?
We're in the present time of the film.
So one of the things the film does, it creates what I call a drama of the telling of the story in which the conversation that's going on underneath Rashomon Gate is a kind of metacommentary on the story that we're watching.
The characters inside the film comment on well, can we believe her?
Is this credible?
Why did she say this?
And the effect of this metacommentary is to create essentially a separate story.
What's the separate story?
It's a philosophic topic.
The topic is the telling of stories.
In other words, this interruption creates a new kind of moral and thematic complexity in the film, something that's characteristic of the great novels and fiction works I mentioned earlier in the lecture that appeared at the turn of the 20th century.
Not is the principle of unreliable narration being introduced, and the principle of competing flashbacks being introduced, and the principle of dislocated chronology being introduced-- all of those things are operating.
But what is even more important about it is that these moments of conversation amongst those three characters at Rashomon Gate also constitute a kind of philosophic meditation on the nature of storytelling and the nature of truth.
And they actually say oh, how can you believe a person?
Or what is truth?
How can we believe what anybody says?
So the film calls attention not only to the profound subjectivity of human responses and the profoundly unreliable nature of memory, but also the extent to which individuals themselves have reasons developing from their egos to distort and tell stories that are more flattering to themselves and so that by the time we come to the end of the film, it isn't clear at all, when the film is over, whether there is a truth, whether there is any final truth that we can embrace.
The issues are not finally resolved.
But what is resolved for us is the idea that human reality is immensely complex, that human beings are endlessly deceitful, that the stories they tell about themselves and others may not be trustworthy.
So in other words, the film opens out into a kind of philosophic profundity that's partly a function of its structure.
So it's another example, one of the most remarkable examples that we've seen in our course, of what I call organic form, of a text whose structure helps us understand what it's about and whose structure is part of what it means, whose structure is essential to its meaning.
We couldn't imagine this film as a straightforward, chronological sequence.
It wouldn't be able to do what it does.
So what I mean by the drama of the telling of the story is literally that.
That is to say, there's a second story, a second subject matter in these interludes-- let's call them interludes in these interruptions in which we return to the present time, get out of the past.
And those interludes are an extended, philosophic, and moral conversation about human nature, about the nature of our human capacity to understand the world, and our capacity to talk about it, to narrate it accurately and fully.
So this drama of the telling of the story, this drama of the screening, this drama of the making of the story is as important a dimension of the film as its actual story, as the actual story that it wants to tell.
I have two other points to make about this remarkable film, and I'll be done.
The first is that one of the things I think you'll notice, as the story goes on, and as different people give different accounts of what happened is that the actual physical conflict between the two male characters, which one would expect to be grand and heroic, is almost always clownish and unheroic.
We expect this great-- he's a samurai warrior, after all, and the man he's doing battle with is a very famous or infamous bandit, criminal-- so gifted a criminal that he's famous.
And one realizes in retrospect that when the criminal, the Toshiro Mifune character, gives his testimony.
In the beginning, in the early part of the film, he's exaggerating his own martial genius.
Although we don't fully realize that at first, but it becomes clearer and clearer to us as the film goes on that he has a motive to exaggerate his heroic stature, and his strength, and so forth.
Not to mention, a motive to exaggerate and maybe to lie about the woman's reaction to his forced attentions.
So all of that is an essential part of our understanding of what is at stake, I guess, when we think about the various subject that Rashomon gestures toward.
So the clownish, unheroic behavior of these fighters is something to note, because there's a deep skepticism in the film itself about all forms of human aggrandizement.
There's a skepticism that the film shares with the commoner who maybe is too negative about human nature, who thinks human beings are completely abject, and that this is the justification for the most selfish kind of behavior, because no one can behave well.
I have hardly exhausted the film, but I hope I've said some things that will be valuable and useful to you on your first viewing.
But let me end by talking about the ending, because the ending of Rashomon presents us with a problem similar to the problem that we confronted in a film like The Last Laugh, Der Letzte Mann, in which there seems to be a kind of optimistic or reassuring ending to this film.
The film has been very dark and rainy.
And in fact, one of the ways you can tell that the film has changed registers is that the rain finally disappears.
Well as you're watching the ending, which is quite explicit, even heavy handed about its attempt to return us to a sort of more hopeful view of mankind, you should ask yourself, does it deserve to be deleted?
All through the '40s, when Kurosawa was first learning his trade, he began to direct early in the '40s.
And this film in 1950 is his first real masterwork.
He's become more and more confident and ambitious as a director during this period, but he hadn't displayed his full capacities as a director until this point, most accounts of his career suggest.
But during this period, in the 1940s, Kurosawa took up part of himself what he regarded as a social project, which was to try to help renovate Japan after the devastations of the war.
And his films of the '40s almost always try to suggest various forms various ways in which people could behave decently and heroically-- if not heroically, at least decently in an effort to renovate and reconstitute a damaged society, a broken society.
One of the reasons that the breakdown of ancient Japan is so powerful in Rashomon, no question, is that Kurosawa and his cast believed that in some sense, there was a symbolic analogy to be made between conditions in Japan, actual Japan in the 1940s and early '50s, and the broken, terrifying conditions of society in the 11th and 12th centuries in the past parable that the narrative is telling us.
And this film continues that tradition.
But I think many, many viewers, I among them, have the feeling that this is more wish fulfillment on Kurosawa's part than reality.
And one could say from an artistic standpoint then, one might conclude that it's a weakness in the film.
I think I might say that the film might be more powerful, more truthful to itself, that the ending that's tacked on may undermine its deepest energies in disturbing ways.
So it's another example in which commercial and social imperatives may be interfering with the artistic integrity of the text.
But it's significant, important to understand that this was a tendency that was present in Kurosawa's work all the way through the '40s, and that therefore, it's a kind of expression of a moral sense that the director had that begins to become less powerful after Rashomon, although he remains a deeply moral director.
So the ending is a question, and you might want to ask yourself how you would respond to the question of the relevance of the ending to the rest to the rest of the film.
Let me end with a reminder about maybe what is, in some ways, the most powerful aspect of what happens when you're watching Rashomon.
I've said that you feel that you've entered into if not a dream, into a kind of uniquely stylized space in which what happens resembles what happens in real life, but also distills what happens in real life, highlights it in a way that isn't true of actuality.
And I think you feel this mythic tendency all the way through the film.
In a certain sense, one way of capturing what I'm saying is to say that there is a tension in the film between this impulse to be mythic, to tell a story that it understands to have a fable-like significance and its sense of the complexity and concreteness of actuality.
That is to say, so there's this wonderful, constant tension in the film between the enormous persuasiveness of the individual images that you see.
But you sense also that you're in a world that's not totally real.
So the tension I'm trying to get you to feel, you can feel it in the dialogue.
But especially you can feel it in the visual images, in the visual texture of the film.
You can feel a kind of tension between an impulse to mythologize, and to fable-ize, and an impulse to show the world in its deepest and most concrete elements, in its most authentic actuality.
And the tension between the two-- gee, this is so real.
Gee, this is so unreal.
This is so fable-like, is part of the secret of the movie.
And one way you can feel it with an immensely intense power is sometimes when you see the way the film deals with human flesh, there are certain scenes, for example, where a woman's hand will be on a man's body-- talk about how you can be erotic without offending anyone.
There's a moment where you can see the woman's fingers pressing into the man's flesh.
It's an immensely erotic and powerfully concretizing moment.
It reminds you of flesh.
It reminds you of film's power to capture actuality with a vividness that goes far beyond what words can ever do, the visual power of movie.
And that's what I mean when I say that there's this constant tension in the film between the mythologizing tendencies of the story, and of Kurosawa's imagination, and what we might call the breaking tendency, the concretizing tendency of the film medium, which has this capacity to register the gross, concrete reality of our experiences with a detail and a power that no other medium can.
Now this sense of tension between a story that wants to be a fable and a story that wants to persuade you of its concrete reality is part of what makes the film so memorable and so significant.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I wanted to use this final time-- hour we have together to summarize a few of what I take to be the central themes of course, the organizing ideas that, at least have seemed to me, to structure our argument and our progress through these, through this almost 100 years of movies.
The first idea, one that we've come back to, I think, again and again, has to do with what we might call film in its anthropological or sociological aspect.
Film as a cultural form, as an expression of a culture, as something that grows out of a culture, that expresses the values and prejudices and well weirdness of a culture.
From this angle, questions of aesthetic value, questions of artistic excellence, are irrelevant in some sense.
From this angle, we're thinking about film simply as a central expression, as a central habit our culture.
Itself and we might recognize that in the 20th century, where film was the dominant narrative form, the film performed functions in the 20th century that earlier forms of storytelling, of narrative end drama, had played in earlier cultures.
And one of my deep desires in the course, one of my implicit ambitions all the way through, was to encourage you to think about movies in this way, as an expression of the values and assumptions on the 20th century cultures, which might be in earlier times have been expressed in other media.
And it was important for me to have you recognize that our 20th and 21st century, although it's somewhat different in the 21st century, because films are-- film is undergoing such a profound transformation in the digital age.
Nonetheless, in the 20th and 21st century, audio, visual narrative is a dominant experience for virtually everyone in the world.
In earlier societies, we didn't have this option, but it didn't mean that the functions played by the movies weren't played by earlier forms.
So that's been one of the deep assumptions, one of the recurring principles to which I've recurred through the course.
And another aspect of this idea, something I've come back to, I think several times, has been the relationship between storytelling and culture.
And drawing especially on certain anthropological and historical sources, I've tried to suggest to you that one way of understanding culture itself, is to understand culture through its stories, is to understand that the stories a culture tells reveal the culture in basic ways.
Not necessarily reveal the culture at its best, although sometimes they may do so, but they reveal the culture in powerful and central ways.
And some of our work in this course, as I hope you recognize, has been attempt in some ways to recover what might be called the cultural, or the anthropological, or the sociological assumptions and beliefs that are embedded in our movies.
One of the most important aspects of this principle of storytelling as intimately connected to the progress of culture has to do with an idea I suggested early in the term, the idea that culture itself-- and this is what I think, one of the great insights of 20th century cultural history, cultural analysis-- is the recognition the culture itself is a completely unfinished thing, that it's never a single fixed item.
That culture itself is, society itself is always in the process of evolving and changing.
That it's idea of itself is always in flux.
This is an immensely productive way of understanding what society is, because what it allows us to do is recognize that the various expressive and other-- especially the expressive forms of a culture, play a central role in what we might call this ongoing attempt of a society to define and renew itself according to various values and assumptions, and historical inheritances.
But also that the culture uses these stories as a way to react to disturbances and challenges to the social fabric.
And remember, borrowing on the great English cultural historian Raymond Williams, I suggested in an earlier course, in earlier class, that one of the ways to think about this process is to imagine culture itself as consisting of sort of three strands or voices.
An emerging voice, which represents new ideas, the future, right?
Emergent voices.
Dominant or central voices, where the central values of a society are embedded and articulated.
And on the other side, what might be quote traditional or vestigial voices.
The voices or energies that are losing force, losing power, as society changes, that belong to a past, but are still part of the mix.
And once we understand that culture is this constant sort of our ongoing, mixed conversation amongst traditional, dominant, and emerging voices, we begin to understand much more about the complexity of how a culture proceeds.
And of course, what this also does, once we make the link between the stories a culture tells and the cultures' idea of itself, we can recognize that this same process can work for understanding stories.
It's immensely liberating, immensely illuminating, to recognize that of course films, like other forms of narrative, are going to do contain elements that are not completely coherent, not completely in agreement.
That they actually may sustain contradictions.
That films, like other forms of narrative, may very well articulate both emergent and traditional voices, and those two voices may be in conflict, or at least not in perfect coherence in a particular text.
So, one of the advantages of this perspective is to allow us to recognize something of the complexity, even the dynamic and potentially partly contradictory elements that we may find in particular films.
So the idea that culture is always unfinished, never a fixed or finished thing, and that the stories a culture tells are a part of this ongoing conversation, this dynamic, always unfinished conversation about the value he was and assumptions the society lives by.
I suggested a number of times that one way to think about the central or the dominant narrative forms in particular cultures, film in the 20th century, the novel in the 18th and 19th century in certain European and in American society, the public theater of Shakespeare's day, the oral formulaic narrative in the ancient world of Homer.
And what I've suggested is that these narrative forms might be labeled, can usefully be labeled as consensus forms, because they understand themselves to occupy a central space in the society.
That they recognize, not necessarily that they're expressing a true consensus, no society has a perfect consensus, but that what they recognize is that they're expressing what is the imagined, or the accepted, or the presumed consensus of the culture.
That consensus of values may not be bought into by everyone, certainly won't be.
Nonetheless, even the people who don't accept the consensus values recognize what the dominant values of the culture are, and recognize those sites, like the movies in the 20th century, that are the carriers of what we might call that official or consensus meaning.
And one of things I tried to suggest to you in the course of our discussions is the way in which, in the 20th century form of film, the central role that genre forms play in this process.
And can you see why that would be true?
One reason is that there's an implicit principle of repetition within variation in genre forms, because they're so familiar.
Different as a late Western and an early Western might be, they're still Westerns and they share a great deal.
They share certain fundamental qualities that every viewer recognizes.
The more literate the viewer or the reader is in the conventions of a particular genre, the richer the uses of that genre become, especially as the genre elaborates itself over time.
And one of the things I've tried to show you by talking, especially about the Western in American society, was to suggest that we can actually understand some of these genre forms as in some sense mirroring the debate a society may very well be having about its own nature.
And that the way the Western changes over time in the history of film is a kind of distillation this process, whereby the stories of a culture help the story, help the culture to accommodate itself to changes, help the help of culture figure out how its past and its present can be brought into some kind of relation, help but help the society get used to change and dynamic challenges to its coherence.
One implication of the kind of argument I've been making about consensus narrative, as I've suggested a number of times, is that a consensus narrative always-- what I'm calling consensus narrative.
We might use a simpler and less aggressive term, we might say various forms of popular culture share the same qualities with the particular form of popular culture I'm calling consensus narrative.
And remember, one of the exciting things about this concept is that it shifts from culture to culture, or from era to era.
The consensus narrative system of one society won't be the same as that of another society, but the systems will share certain-- in one case it might be the novel, in another case it might be the film, but both system will-- both of these forms, when there are consensus system, consensus forms, will share certain qualities.
And I've suggested that the central qualities they share are that they are always in some deep way conservative, right?
They're both morally conservative, or conservative in terms of content, because they're trying to reach the whole of the society and they have to be careful about that, they can't be as advanced or as radical as the most radical, or avant garde parts of the culture, because certain elements of the culture would be alienated by it, and it's the cultural work of consensus narrative to address the center of society.
So they're conservative in content and the moral values.
And there are also conservative artistically.
That is to say, they don't do a lot of sort of daring and shocking, technical or structural experiments.
Again, because they want to be, because they aim to be understood by a large proportion of the population.
They aim to be readable by everyone.
And that brings us to do a second item that's an aspect of, always a fundamental aspect of consensus narrative, they're accessible.
That they are written or created in a language according to archetypes, or stereotypes, or shared tropes and conventions that are accessible, readable to the whole of the society.
And a final feature that these texts always have is that they are collaborative.
And I've suggested earlier in the course that one way to think about how collaborative the movies are is to recognize in every movie is itself a collaboration.
That you can't make a movie-- one person can never make a movie.
That a movie is always a collaborative form.
But, I mean this idea of collaboration in a deeper way as well.
Not only is the text, are the text of movies literally collaboratively created by many, many creators, but they're collaborative it another way, in a deep way.
Of course the most obvious one is the way in which you all genre forms collaborate with themselves, the way they're in a kind of conversation with all prior instances of the genre.
And what that means is the genre forms are collaborating with their history, their collaborating with their-- and they're also implicitly collaborating with the audience's literate knowledge of what these genre forms are like, how they behave, what the expectations in them are.
And their collaborative in yet another way, and this aspect of their collaborative characters is one that I haven't emphasized too much, and I want to do it today in our final, in our farewell lecture.
They're collaborative also because these text, in some deep way, are collaborating with what we might call the collective wisdom or values of the culture.
And this is implicit in what I've been saying about the way these story forms draw on the histories, and stories, and conventions that are most widely shared in a culture.
But I want to clarify what I mean by-- so there's that kind of collaboration, in which they're constantly borrowing forms, and arguments, and principles, and symbols from the vast body of shared knowledge that a culture might have.
But what this might actually lead to-- one thing this leads to is the recognition that these cultural forms are not always noble or wonderful, that they carry prejudices, that they carry racism, that they carry misogyny, that they're going to embody, that the study of these texts is not necessarily, never should be a totally celebratory experience, even though many of these texts are what valuable, and lasting, and speak to us in deep and powerful and useful ways.
It's also true that these text carried the lies and prejudices of the cultures from which they come, as I've said a number of times in the course.
Perhaps too many times.
But I mentioned again in this last lecture, because I want to also mention that there's a way in which, not just the evils and the limitations of a society that get expressed in the collaborative consensus forms that emerge from them, but also sometimes the wisdom of society does too.
In other words, what I want to suggest to you is that the collaboration has to do not only with inherited values, and inherited beliefs, and inherited story forms, they also have also have to do with the kind of inheritance of wisdom.
And I have a particularly exciting way of distilling this for you.
There's a passage, one of my favorite passages, from Shakespeare, from his from his great late play Cymbeline.
From Act four, scene two, in which a group of characters gather around a body.
They think the body actually comes alive later in the play, because there's a lot of wonderful magic in the play, it's a play about redemption and rebirth.
But at this moment in the play, they think this person is dead, and they single dirge.
It's one of the most-- a dirge, a song, a funeral song, a song or a poem to be sung or recited at a graveside.
And there's, maybe the greatest of all such dirges is in English, is in Cymbeline.
And this is the first stanza and the greatest stanza, fear no more-- right there, people are standing over corpse.
Fear no more the heat of the sun; Nor the furious winter's rages, Thou thy worldly task hath done, Home art gone and taken thy wages.
Very interesting to image, to embrace a metaphor of working, a working class metaphor.
You're dying, and the metaphor is, you've taken your last wages, you finished your work.
Home art gone and taken they wages.
And here comes the couplet that I think is most powerful.
Golden lads and girls all must, as chimney sweepers, come to dust.
Let me say it again so you get the whole stanza, it's quite beautiful, I think.
Fear no more the heat of the sun; Nor the furious winters' rages, Thou thy worldly task hath done, Home art gone and taken thy wages.
Golden lads and girls all must, As chimney sweepers come to dust.
Now think about those last lines for a minute.
They're very powerful, even if you don't exactly get them.
Golden lads and girls, the implication is high born, wealthy, golden in their youth, and in their energy and in their vitality.
No matter how golden, whether in wealth, or a vitality, even golden lads and girls end up in the grave like all of us.
And there's, the implication of course, is there's a kind of constellation in the fact that it's-- deaths is our shared fate.
It's a consoling, an attempt to console us.
Although the older I get, the less consolation I find myself taking from this insight.
Nonetheless, it is a great poem, and this stanza is a particularly remarkable poem because of one element in it, the phrase, golden lads.
For those of you know-- very few Americans know this, but some Brits know this.
In Shakespeare's native Warwick shire, the phrase golden lad referred to something very specific.
It referred to the condition of the common ordinary flower that we call the dandelion.
And it's a flower that has a gold cup when it's young, and then when it goes to seed, and you've all seen them, because they're native to North America as well as England.
The top of the cap turns into a kind of rounded series of filaments, a beautiful round cap and when you-- a lot that children pick them and blow them, and the seeds blow all over.
One of the reasons that these common weedy flowers are so common, is that they're so brilliantly designed by nature to propagate, because the wind blows their seeds all over the place.
Do you all have an image of this?
So when you're young, the flowers have these golden caps, and then when they go to seed, they become these, they have these rounded filaments on the top.
And these rounded filaments, in Shakespeare's day, were called golden lads when they-- were actually called, these flowers were called golden lads when they were in bloom, and they were called chimney sweeps when they went to seed, because they looked exactly like a chimney sweepers broom.
Chimney sweepers we take these rounded brooms and put them up in chimneys to clean them.
So when Shakespeare writes in the end there, he says, golden lads and girls all must, as chimney sweepers, come to dust.
He's describing literally the phases of life in a flower.
Why is that wonderful?
Can you see why that's wonderful?
Why that's what art is?
Because of course, what's happened is that, what Shakespeare did was create a brilliant metaphor out of something that's profoundly concrete.
So the phrase golden lad and chimney sweeper actually refers to the life cycle of a flower.
Well think of how much more consoling, how profoundly brilliant the lines become, when actually Shakespeare's metaphor is literally linked to the passages of nature, to the progress of nature.
So what I'm calling your attention to is a brilliant piece of poetry, a piece of poetry that has a kind of richness and complexity that almost couldn't be better.
And even when we don't know that about the flower, we get the meaning of the lines, but think how the lines come alive once we recognize that the progress of human life in those couplets is being explicitly associated with the life cycle of a natural object of a flower.
Now this is a great act of poetic geniuses, isn't it?
But where does the genius lie?
Where does it come-- Shakespeare is collaborating with the wisdom of his culture.
It takes Shakespeare to figure it out, I mean I'm not saying that it is an act of genius for Shakespeare to realize the metaphoric implications of this, he did, but the metaphor is embedded in the way ordinary people named that flower for hundreds and perhaps thousands of years.
Shakespeare is collaborating there with the wisdom of his culture, with the shared understanding of his society, in order to make that brilliant line of poetry.
So we can take that idea that, the notion of golden lads, as a kind of emblem for the collaborative energies that are sometimes released when artists collaborate with the traditions of their culture, with the traditions of naming, with the traditions of understanding, that are embedded-- and so therefore the collaboration is a deep one.
It's an intellectual, and an expressive, and a creative collaboration between the societies understanding of things, the cultures understanding of things, and the artists interaction with those.
And that's an aspect of, an even deeper aspect of what I mean when I say, that consensus narrative, and that movies are a collaborative art form.
I've also tried to suggest, again and again in the course, that there are films and many of, most of the films we've seen in the semester, also deserve the designation art.
And I don't want, I've not really wanted to turn the term art into a kind of elitist category from which we must sort of retreat in awe, and sort of before it.
I want us to think of art in a simpler and, I think more useful way, as a form of intelligence and competence.
In other words, when things are done well in a work of art, when a movie as well made, when a poem is well made, we admire it and we respect it, and we don't have to get mystical about the qualities of art in order to recognize that intelligence and competence are qualities that we value in other aspects of experience, as well as an art itself.
What do I mean specifically by intelligence incompetence?
The most profound way of understanding this argument has to do with what I've spoken about a number of times in the course, I've sometimes used the phrase organic form to explain what I meant in this regard.
But essentially the idea has to do with the sense, with a recognition that a serious artist will use instruments that further his or her intentions effectively and fully.
That is to say, that there's a connection between the formal or stylistic choices that an artist makes, and the content that is part of the art work.
That is to say that there's a link or a connection between the formal choices that a writer or a filmmaker makes and the content intended to be communicated.
And we can think of the term competence as having to do simply with the idea of technical mastery.
In other words, a competent poet is a master of language, is a master of metaphor, is a master of grammar.
You can't be a great poet if you have no understanding of grammar whatever, because you be you'd be throwing away one of the great technical resources of the language, which is the complexity that grows out of a mastery of grammar.
By the same token, you couldn't have a serious filmmaker who didn't have a deep profound sense of what it meant to create a visual story, what it meant to try to embody meaning in audio, visual ways.
You couldn't have a great filmmaker who is oblivious entirely to the visual nature of his experience.
The filmmaker would-- any serious filmmaker has to be always aware of the special qualities and properties of the medium in which he or she is working.
So if we think or art then, not as a mystical category, but as an expression of intelligence and mastery, intelligence and competence, perhaps we've demystified it a bit, and given us grounds for actually making judgments about particular films, and particular passages, and moments in film.
And let me remind you again, that a second version of this argument about the relationship, at least in part about the relationship between art and what I had sometimes called entertainment, the greater complexity or richness of art, can be embodied also, not only in this first principle of a connection between the stylistic and formal choices and the content.
But in this second way, in the second category or second perspective as well, in terms of what I've called texture or multiplicity.
The multiplicity principles.
Let me just quickly remind you of a couple of examples we've looked at in the course, just as a way of giving you some instances from which you can-- from which you can extrapolate your own responses to other texts.
Do you remember for example the scene of that, that we looked at fairly closely, in McCabe and Mrs. Miller, that disturbing ironic shoot out on the bridge?
In which had me murderous young killer goads a young former into taking the gun out of his holster, and shoots him down in cold blood, right?
Let me simply remind you about the way in which, part of what gave that scene it's power had to do with the fact that there was a psychological dimension to it.
The young murderer who was committing the murder had been frustrated because he tried to show off earlier and not succeeded.
And that imparted a terrible, horrible irony to the scene, because you realize that this murder is completely fortuitous, it's completely accidental.
If that little murderer had succeeded earlier in shooting the bottle that was floating around on the frozen pond, he probably would not have challenged, he definitely would not have challenged this young farmer to try to get back his mojo and show the people around him that he knew how to handle a gun.
So what made this-- and of course, and then that already created a kind of complexity and texture.
But then, when we add in the fact that this scene itself is one of the central iconic scenes in all Westerns, two males facing off against each other, ready to shoot we, come to realize that the various anti-heroic and ironic ways in which this shoot out is handled, throws a profoundly critical and disturbing light on dozens, and perhaps hundreds of earlier such scenes that have occurred in Westerns.
That the very basis of the Western's embrace of violence, it's notion that you could achieve regeneration through violence, is put in question by that scene.
So there's a texture or multiplicity the scene.
It forwards the story, it shows a violent act, it tells us something about the psychology of the murderer, but even more than that, it resonates through the whole history of the Western, because the shoot out itself has been shown in such an anti-heroic, such an ironic light.
Texture, multiplicity.
Remember the moment in Cabaret where the Joel Gray character, the interlocutor, at one point kneels down while he's talking during the mud wrestling scene, picks up a piece of mud on his finger, rubs it on his face like this, and gives himself with Hitler mustache.
I mean it's a rich comic moment, but is more than that, especially in the context of, as the film proceeds.
We understand from that moment that the Nazis are objects of fun and mockery.
Not that dangerous, it would seem, at least at that the cabaret feels superior to it.
And of course, when we think about the course of the film, when we think about how the Nazis moved from the periphery to the center of the movie, we come to recognize that that moment, especially in retrospect, that moment has a particular kind of ironic original richness, that's partly a function of the obliviousness that everyone in the cabaret, and even the Joel Gray character, despite his apparent cynicism and worldliness seems to have about-- they're innocent, they're naive about the power and danger of the rise of the Nazis.
And let me give you one last example, although there are many others that we mentioned.
Do you remember the moment in the passage I showed you from the De Sica movie, Umberto D?
That very brief moment where we see a man get on a tram, a bus, and sit next to someone on the bus, and the bus moves.
It's a completely wordless sequence, and there's a character we've never seen in the film before who sits down next to the protagonist, no words exchanged.
When our protagonist gets up and gets off the bus, the camera lingers briefly on the man he left behind, and we see that man go like this.
And what I suggested to you, again, I suggested that this was a moment of richness and complexity, of multiplicity in a way.
Because one implication of that scene is that the man, this stranger that we'll never meet again, that we only saw for one brief instant in the course of the film, might have concealed the story as deep as rich and as moving as that of our hero Umberto D., who is now leaving the tram, and we will shortly be following him.
Again, a moment of complexity or richness that embodies central meanings, helps us to see central meanings in the text.
Well I have a final example of this kind of texture or multiplicity.
It's an example I had hoped to show you what we talked about Kurosawa last week, but I didn't quite make enough time in my lecture, and I want to show it here, as a kind of final homage to that great director, and as a way of reminding you of the variety of performances that the great actor Toshiro Mifune, who plays the rapist in Rashomon, was capable of.
So here is a scene, a very minor scene, actually, from the film, from the Kurosawa film, Seven Samurai, which I mentioned last time.
Let me set the context for you, before I show it to you.
It's a very brief scene.
Seven Samurai, which I've sometimes shown in this course, I actually toyed with the idea of having to watch both Rashomon and Seven Samurai, but Seven Samurai is a very long movie, and I sometimes warn students about it when I show it in the class.
I decided not to do it, but I hope all of you over your Christmas break, soon, will look at Seven Samurai.
It will resonate, especially for you, since you've just seen an earlier Kurosawa film.
Well, this immensely long film has a wonderful epic amplitude.
And the long first section of the film involves, essentially, the assembling of a group of samurai who are going to go to a village to defend this village of farmers from the deprivation of bandits.
When the film opens, some farmers aren't make their way to a great city, trying to find samurai who will come and defend them.
And the first section of the film shows this group of samurai slowly coming together, but only six of them are in the group, because there was a seventh who tried to join them, the Toshiro Mifune character, but he was rejected in the earlier part of the film that we haven't yet seen.
That we have seen, but when you look at the clip, you will not seen yet, because he's not really a samurai, he's is a commoner who was trying to fake that he was a samurai.
He grew up top knot, he can handle a sword, but they could tell in various ways, he didn't know the poetry that samurai knew, a lot of ways in which they figured out that the Mifune character really was not a samurai.
So they didn't let them join them.
And in this scene we're going to see, it's two or two and a half minutes long, and this is one reason I wanted you to look at it, because it's so brief, and its basic function is so simple.
The point of the scene is to get the samurai to the village where the central aspect, where the central adventure of the film will take place.
So we could almost think of part one as a kind of prologue, or at least an introductory preparation, and the film really isn't going to get going in one sense until after this transition moment.
So the sequence we're looking at has a very simple narrative function, it's to get the samurai from the city to the rural village, and we see them making this trip.
But as you'll see, the scene has a much greater multiplicity, a much greater significance than simply this simple narrative significance of moving the actors to a new location.
Here we go.
And see he's following them in the back there and they say go away.
[VIDEO PLAYBACK] [MUSIC PLAYING] [LAUGHTER] [WHOOPING] [WHOOP] [LAUGHTER] [SPEAKING JAPANESE] [LAUGHTER] [WHOOPING] [SPEAKING JAPANESE] [END PLAYBACK]
Alright, so a scene of unexpected comedy, but what else?
Can you see how much is going on there?
If we think about the scene for a moment, we can see, OK, the first function of the scene is to get them to the village.
But it's a wonderful example of the multiplicity principle, it seems to me.
It forwards the story in some simple sense, but what does it also do?
It defines the samurai in some basic way.
First of all, the legitimate samurai have a sense of humor.
But even more than that, as we're watching this scene, what do we see, among other things?
A wonderful very brief dramatization of the conflict between man and nature.
Think of a raw those natural environments are in that sequence.
And the extent to which we also-- the film itself is in many ways-- a sub-theme in the film is the relationship between human beings and natural forces, this scene implicitly sets that them up without doing anything fancy or calling our attention to it.
What else does it do?
It defines the Mifune character really richly and fully.
He's something of a clown, isn't he?
He likes attention, there's something deeply theatrical about him.
See the way he, he knew the other samurai were watching him when he caught the-- What else did we learn about him?
He's also unbelievably competent.
He jumps in that freezing water and he catches a fish with his bare hands.
And one implication of this scene may very well be that one the things to the Mifune character is trying to do is show the samurai how competent is.
And you see, he ends up leading them to the village.
And of course, you might guess, do you think he remains excluded?
No of course not, he ends up joining.
He is the seventh samurai, and becomes the seventh as the film goes on.
So we, Mifune's nature is defined for us in action more than in words, we have the theme of nature versus man, we have the broader definition of the nature of the samurai group itself, and we get an action that also moves the story to a new location.
And one reason that I find this example such a powerful one, is that many people watching this movie never even remember this scene, because it's hardly-- even though it is very dramatic scene, isn't it?
The fact is there's so many astonishing, dramatic, visually memorable scenes in the film, that most viewers who don't have the advantage of having this scene singled out for them would never even remember it.
And that's exactly why it's so valuable, it seems to me, because if it plays a relatively minor role in the film.
Its role in the film is fairly simple and structural, but even in that simple two and a half minute sequence, a great deal is going on.
It's a wonderful instance, it seems to me, of what we mean by texture or multiplicity.
If you are aware of this principle when you look at other forms of art, not just films, I think you'll find your sense of what is going on in a particular text, or a particular expressive form, will be immensely enlarged.
And I hope you apply these ideas not just to movies in the future, but to other but to other forms of expressive experience as well.
I also, at the end of this course, as at the end of every semester, am afflicted by a sense of gratitude that I can't always fully express to my students.
I've been doing this for a long time, I was thinking as the class opened this afternoon looking out at your faces, I realized that one of the most amazing things about being a professor is that your basic clients stay young, and they keep you young, they keep you young, even though they also, in some ways make you more and more conscious of the gap between you and your students in some respects.
I have no doubt that part of what has made my career a satisfying one, and certainly what has made me a much, much better teacher than I might otherwise have been, has been the enthusiasm and welcoming interest that I've felt from virtually every student I've ever taught.
And that's especially true in this course, and it's been especially true in the semester.
So I want to end then, by saying thank you.
In a way, symbolically, I'd like to say thank you to the students I've been teaching for more than 35 years at MIT, for whom you are stand ins.
But most of all, I want to thank you.
You make me, teaching makes me, feel serious.
Being able to introduce students to these astonishing works of art has always seemed to me to be a privilege for which I was unable to express, fully, my gratitude .
So thank you very much.
Have a wonderful life.
[BLANK AUDIO]
This class has shaped how I approach communicating with others in showing me how differently languages that I previously understood to be invalid, including Creole, Southern American English after I hear anything, I'm like-- Stereotypical languages like that are actually very systematic, and should be regarded as languages in and of themselves that should be given validity inside the US, whether that be through court room interpreters for people like Rachel Jeantel, who spoke primarily African-American English, or even just inside of Haiti teaching classes in Creole, as a means to connect with students better
Yeah.
Also I think how this class shaped my communication is it allowed me to discuss more difficult topics without feeling hindered.
I didn't have to feel like I was going to offend anyone because it was in the context of learning.
And I think people were very open with answering questions that might be seen as like, why don't you know this, or stuff like that.
And further, also with the presentations and with the essays, I feel like you had to develop your communication skills in a bunch of different ways, not only speaking but how we present now, and then also how do you write an essay.
I think Dr. Nora Jackson was really great in helping me develop my writing style.
And yeah, I thought I was going to say something else, but-- [LAUGHING]
I think the conversations that we had were really interesting all the time.
Everyone has some sort of opinion.
Some people are more willing to speak about them than others, and personally I feel that oftentimes in conversations I'm more reticent.
But this class felt like such a safe space that it was possible to speak about personal experiences or opinions on readings and all of these things.
And I really appreciated the presentations that we have to do as well, because as a communication intensive course we have to have some sort of oral component and present things, and those freak me out all the time.
But it's important to have this type of practice and to be able to have a safe space in order to practice and then be able to do it better and better each time
Yeah.
And, you know, for me, starting this summer I'm going to be beginning my M.D.
PhD in anthropology.
And trained as an anthropologist, I've always had this idea that you don't really need interject that much personal experience into what you write, especially if it's an academic work.
And from the get-go in this class, we were expected to draw equal parts from the theory with which we engaged and equal parts from our own experiences and present that in the form of a narrative, that was in conversation with the theory.
And I was uncomfortable with that.
I thought that-- I had this notion that doing so would make my writing less professional, less objective, and perhaps less valuable, because it wasn't primarily rooted in theory.
So much of it was narrative, so much of it was now subjective.
But I found that interrogating my own experiences as an academic, who's engaging with the theory, helped me to kind of deconstruct stereotypes like the ones Colin articulated, and these preconceived notions that we inherently carry in when we're trying to do objective analyzes in a way that I had never had the opportunity to do before.
And so getting out of my comfort zone by writing these academic papers that were equal parts personal narrative was something that I think will prepare me well as an anthropologist to ensure that, to a greater extent, I'm able to understand where I stand in terms of a particular issue and perhaps do more justice when I do produce a work
And on top of all this, one way that I would like to see the course improve-- which Colin mentioned in class and has to do with communication-- is having difficult conversations with someone who doesn't have your view.
Something we talked about in this class was that everyone here is kind of like-minded.
We all sought out this class for a reason.
And I think I didn't get to challenge myself in communicating to someone who didn't agree with me.
And that could be a way that educators can improve
Definitely.
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
I don't want</b> <b>to make anyone jealous who</b> <b>came before, but I think</b> <b>we can say pretty safely</b> <b>that we saved one of the</b> <b>best for last, right?</b> <b>I won't say the best.</b> <b>OK?</b> <b>All right.</b> <b>So as you can see,</b> <b>throughout the course</b> <b>we looked at these issues about</b> <b>how language, race, ethnicity,</b> <b>gender, how they can be used</b> <b>to create various hierarchies.</b> <b>And as you can see now</b> <b>after the semester,</b> <b>these are tools, really.</b> <b>They're not inherently</b> <b>part of who we are,</b> <b>but they are tools that</b> <b>various pools of power</b> <b>use to keep control,</b> <b>to create hierarchies.</b> <b>And I think last week, you guys</b> <b>you had a very good discussion</b> <b>with Dr. Aleman about how</b> <b>sometimes these hierarchies can</b> <b>be internalized.</b> <b>And there was a very</b> <b>good discussion,</b> <b>as I can tell, from the video</b> <b>that she took for me about how</b> <b>you, yourself, we need to</b> <b>do some work inside of us</b> <b>to go beyond these threats that</b> <b>these terror attacks impose</b> <b>on us.</b> <b>So we went very micro.</b> <b>So we went very macro,</b> <b>then went micro.</b> <b>And today, with my friend</b> <b>and colleague, Noam Chomsky,</b> <b>we're going to go</b> <b>my macro again.</b> <b>But I guess you've</b> <b>read the papers.</b> <b>I read the papers.</b> <b>And I must say, Noam, I was</b> <b>very disturbed by what I read.</b> <b>But the fact is that</b> <b>those are the data.</b> <b>So this is MIT.</b> <b>We are very much involved in</b> <b>trying to understand knowledge.</b> <b>And many of you</b> <b>throughout the semester</b> <b>made a case that we have all</b> <b>this knowledge here at MIT.</b> <b>But often, this knowledge</b> <b>does not translate into action</b> <b>into the real world.</b> <b>And in a way, I think</b> <b>with Noam's entire life,</b> <b>we have an example of</b> <b>how that can happen.</b> <b>How we can take very</b> <b>abstract, technical knowledge</b> <b>and make it available</b> <b>to the world</b> <b>and try to make it better.</b> <b>And the theme of this</b> <b>course throughout</b> <b>was how to build bridges, how to</b> <b>make change and build bridges.</b> <b>So we hope that with</b> <b>Noam today, we'll</b> <b>get a clear sense of</b> <b>how that can happen.</b> <b>And perhaps, how we</b> <b>might, with some luck,</b> <b>save the world, right?</b> <b>
I think the</b> <b>best way to proceed is--</b> <b>in a court seminar is for you to</b> <b>say what you're interested in.</b> <b>I'll see if I have some</b> <b>way of reacting to it.</b> <b>Lots of things I</b> <b>could talk about.</b> <b>I mean, one thing we</b> <b>could talk about related</b> <b>is right in the headlines.</b> <b>So there are things which is an</b> <b>opening for things we can do.</b> <b>That's the plan that was</b> <b>just made public yesterday</b> <b>to deport Haitians back</b> <b>to Haiti from Boston.</b> <b>That's a very live issue.</b> <b>A lot of lives depend on it.</b> <b>We can do something about</b> <b>it right here if we want.</b> <b>It turns out-- I</b> <b>didn't know this--</b> <b>that the Haitian</b> <b>Ambassador, Michel just</b> <b>told me, who is pleading</b> <b>with the government</b> <b>not to carry out this</b> <b>onerous and destructive act,</b> <b>is actually a former</b> <b>MIT student, who</b> <b>wrote about these</b> <b>topics in his thesis.</b> <b>So that's one of many things</b> <b>that could be discussed.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
So first of all,</b> <b>thank you for being here.</b> <b>We've talked about</b> <b>immigration in the past.</b> <b>And with regard to</b> <b>the Haitian refugees,</b> <b>I think it also applies.</b> <b>And one of the</b> <b>themes we studied,</b> <b>I believe with</b> <b>Professor Tormund was,</b> <b>what's the actual reason that</b> <b>people do not want immigrants</b> <b>in the country versus the</b> <b>reasons that they state?</b> <b>So for example, one of the</b> <b>reasons that people would state</b> <b>would be that immigrants</b> <b>are ruining the economy.</b> <b>But if you look at--</b> <b>
One of the</b> <b>reasons why people stay?</b> <b>
Oh, sorry.</b> <b>One of the reasons that</b> <b>people state that they don't</b> <b>want immigrants in the country.</b> <b>
Oh,</b> <b>state that they--</b> <b>
Yeah.</b> <b>Is because they</b> <b>say that immigrants</b> <b>will ruin the economy.</b> <b>But then, Professor Tormund</b> <b>pointed out the fact</b> <b>that Mayor Bloomberg</b> <b>of New York,</b> <b>or the former Mayor</b> <b>Bloomberg of New York,</b> <b>said that New York could</b> <b>not run without immigrants.</b> <b>So what do you think are the</b> <b>actual reasons that people</b> <b>do not want immigrants</b> <b>in the country?</b> <b>
Well, first of</b> <b>all, it's not a new phenomenon.</b> <b>And it's not the United States.</b> <b>So for example, in</b> <b>Europe, a majority</b> <b>of the population</b> <b>of Europe, majority,</b> <b>thinks that Europe should not</b> <b>permit any Muslim immigrants.</b> <b>Europe has a certain history.</b> <b>It's called crematoria,</b> <b>for example.</b> <b>That's part of the background.</b> <b>And that's part of the</b> <b>immigration background</b> <b>here, too.</b> <b>I can talk about it from a</b> <b>very personal point of view.</b> <b>My father happened to be able</b> <b>to come to the United States</b> <b>before fleeing from</b> <b>Czarist Russia.</b> <b>Horrible conditions,</b> <b>especially for Jews.</b> <b>He made it before the</b> <b>Immigration Act of 1924.</b> <b>In 1924, Immigration Act for</b> <b>the first time barred Europeans.</b> <b>Prior to that, there were</b> <b>racist immigration acts,</b> <b>like it kept out Orientals.</b> <b>But Europeans were admitted.</b> <b>And there was a very</b> <b>simple reason for that.</b> <b>We were exterminating the</b> <b>indigenous population.</b> <b>So there was a</b> <b>lot of open space.</b> <b>That's what you see</b> <b>glorified in stories</b> <b>about covered wagons going</b> <b>to the West-- the pioneers.</b> <b>Of course, they were</b> <b>displacing some people</b> <b>who happened to live there,</b> <b>but it's another story.</b> <b>Anyway, there was an open--</b> <b>the continent was</b> <b>open after you've</b> <b>exterminated the population.</b> <b>And you want the right</b> <b>kind of people to come in.</b> <b>And if you go back, the question</b> <b>of the right kind of people</b> <b>was quite delicate.</b> <b>So for example,</b> <b>Benjamin Franklin,</b> <b>who was the leading figure of</b> <b>the Enlightenment in the United</b> <b>States, he thought we should</b> <b>have discriminatory immigration</b> <b>rules to keep out the</b> <b>wrong kind of people.</b> <b>People who were too dark.</b> <b>And the people who were too</b> <b>dark were Swedes and Germans.</b> <b>That's Benjamin Franklin.</b> <b>So we got to watch out.</b> <b>Keep out Germans and Swedes.</b> <b>They're too siwrly.</b> <b>But that was finally relaxed.</b> <b>Anyway, they needed</b> <b>Europeans to come in.</b> <b>By the beginning of</b> <b>the 20th century,</b> <b>they didn't need immigrants</b> <b>any longer to fill the country.</b> <b>And the constant racism emerged.</b> <b>So the 1924 Act was--</b> <b>in reality, it was based</b> <b>against Jews and Italians.</b> <b>What they said is East</b> <b>and Southern Europeans.</b> <b>But that meant</b> <b>Jews and Italians.</b> <b>Well, OK.</b> <b>What happened to</b> <b>the Jews who were</b> <b>unable to come until the 1960s?</b> <b>They ended up in concentration</b> <b>camps and crematoria.</b> <b>So it's a serious issue.</b> <b>That's most of my</b> <b>family, incidentally.</b> <b>The ones who didn't make it.</b> <b>So the story goes way back.</b> <b>It's not something new.</b> <b>And it's not just</b> <b>the United States.</b> <b>Western society generally is</b> <b>infected by very deep racism.</b> <b>And it's even more extreme</b> <b>in many ways in Europe</b> <b>than in the United States.</b> <b>In fact, in Europe, it's</b> <b>even more frightening</b> <b>than the United States.</b> <b>So take this</b> <b>morning's newspaper,</b> <b>if you happened to look at it.</b> <b>The government of Austria--</b> <b>there was a</b> <b>coalition government,</b> <b>but it wasn't able to</b> <b>have an agreement, so they</b> <b>have to have an election.</b> <b>And the concern is that the</b> <b>most popular figure in Austria</b> <b>might win the election.</b> <b>He happens to be a neo-Nazi.</b> <b>That's Austria.</b> <b>There's a little bit</b> <b>of history in Austria.</b> <b>Well, that's scary.</b> <b>And it's happening all over.</b> <b>Here, it has a special character</b> <b>because of white supremacy.</b> <b>White supremacy is very deeply</b> <b>rooted in the United States.</b> <b>There are comparative</b> <b>studies of white supremacy.</b> <b>The major ones by</b> <b>George Fredrickson,</b> <b>a good sociologist who</b> <b>did a comparative study</b> <b>of white supremacy</b> <b>in many countries.</b> <b>It's more extreme in the</b> <b>United States than any country</b> <b>that he studied, including</b> <b>South Africa under apartheid.</b> <b>And you see it all</b> <b>over the place.</b> <b>I mean, when the Nazis</b> <b>in the early '30s</b> <b>were trying to find models</b> <b>for the Nuremberg laws,</b> <b>the racist laws, they</b> <b>looked around the world.</b> <b>And the only country where</b> <b>they could find good models</b> <b>was the United States.</b> <b>They took US models.</b> <b>The US had miscegenation rules--</b> <b>rules against</b> <b>interracial marriage--</b> <b>which lasted, in fact,</b> <b>until they were finally</b> <b>struck down in the 1960s.</b> <b>The Nazis didn't</b> <b>entirely take the US laws</b> <b>because they regarded</b> <b>the US laws as too harsh.</b> <b>So the US laws were based on the</b> <b>concept of one drop of blood.</b> <b>None of this means anything</b> <b>from a biological point of view.</b> <b>It's all totally meaningless.</b> <b>But these are socially</b> <b>constructed categories.</b> <b>And one drop of</b> <b>blood meant if you--</b> <b>by whatever category</b> <b>was concocted,</b> <b>you had one drop of Negro</b> <b>blood, you're black.</b> <b>And then, you can't</b> <b>marry a white.</b> <b>Those were laws that were</b> <b>not struck down, I think,</b> <b>until 1967.</b> <b>And you see it all over</b> <b>the place right now.</b> <b>I mean, it's right in</b> <b>front of us all over.</b> <b>In fact, if you look at</b> <b>Afro American history</b> <b>in the United States,</b> <b>it's pretty remarkable.</b> <b>In fact, that's something</b> <b>that really should be studied.</b> <b>The two topics that</b> <b>should be studied</b> <b>are, what happened to</b> <b>the native population</b> <b>and what happened to</b> <b>the black population?</b> <b>Anything else is secondary</b> <b>for the United States.</b> <b>Without going</b> <b>through the details--</b> <b>I presume you know them.</b> <b>But the first slaves</b> <b>were brought here</b> <b>in Massachusetts 400 years ago.</b> <b>That's 400 years.</b> <b>You look over those</b> <b>400 years, there</b> <b>have been literally</b> <b>a few decades.</b> <b>Maybe 30 years scattered in</b> <b>which African Americans had</b> <b>kind of a small opportunity</b> <b>to enter the general society.</b> <b>That's quite a record.</b> <b>And, of course,</b> <b>it has a residue.</b> <b>The residue shows in things</b> <b>like mortality, wealth.</b> <b>The wealth of African American</b> <b>families is virtually zero.</b> <b>What there was, was pretty</b> <b>much destroyed by the crash.</b> <b>It shows up in, why are</b> <b>there murders in ghettos</b> <b>in Chicago and Baltimore?</b> <b>It's not the genes.</b> <b>Everywhere you look, you see it.</b> <b>And it shows up.</b> <b>Going back to your</b> <b>question, it shows up</b> <b>in the racist immigration</b> <b>laws, which are bad enough now.</b> <b>But the ones that</b> <b>were enacted in 1924</b> <b>and maintained right</b> <b>through the Holocaust,</b> <b>those were pretty serious.</b> <b>In fact, you probably</b> <b>know the stories.</b> <b>But ships were turned</b> <b>away bringing refugees</b> <b>who thought they could make it.</b> <b>There's some strange</b> <b>stories if you look at it.</b> <b>They're kind of suppressed.</b> <b>But one quite</b> <b>interesting story--</b> <b>it's worth looking at if</b> <b>you're interested-- has</b> <b>to do with Japan.</b> <b>The Japanese fascists</b> <b>were kind of--</b> <b>only had limited understanding</b> <b>of the Western world.</b> <b>Japan had been pretty isolated.</b> <b>So the fascist leaders, the</b> <b>generals in Japan in the 1930s,</b> <b>were pretty much</b> <b>influenced by Germany.</b> <b>And they read German</b> <b>propaganda, Nazi propaganda,</b> <b>about how Jews are</b> <b>running the world.</b> <b>And they were very eager to get</b> <b>the Americans off their back.</b> <b>They didn't want a conflict</b> <b>with the United States.</b> <b>So they concluded</b> <b>that the way to get</b> <b>the Americans off</b> <b>their back is by doing</b> <b>something nice for the Jews.</b> <b>And this, in fact, fit</b> <b>with Japanese history.</b> <b>Because in the early</b> <b>part of the 20th century,</b> <b>there was a Russo-Japanese war.</b> <b>And wealthy Jews who</b> <b>were very anti-Russian--</b> <b>because Russia was</b> <b>very anti-Semitic--</b> <b>were funding and</b> <b>supporting the Japanese.</b> <b>Jacob Schiff and others.</b> <b>Japan considered that that</b> <b>was part of the reason why</b> <b>they were able to win the war.</b> <b>So they had their</b> <b>own connections</b> <b>with the international</b> <b>Jewish conspiracy</b> <b>that runs the world that</b> <b>they were picking up</b> <b>from Nazi propaganda.</b> <b>So they concluded that the way</b> <b>to mollify the United States</b> <b>was to bring Jews from</b> <b>Eastern Europe to Japan.</b> <b>And there was a period</b> <b>of the Nazi Russian pact</b> <b>where people could move.</b> <b>And in fact, there was</b> <b>a considerable number</b> <b>of Polish Jews, very</b> <b>orthodox Polish Jews.</b> <b>The kind you see with</b> <b>black hats and white shirts</b> <b>and you know, all of this.</b> <b>They were brought</b> <b>from Poland to Japan,</b> <b>which was a feudal society.</b> <b>The cultural contact</b> <b>was incredible.</b> <b>There's all sorts of</b> <b>interesting stories about it.</b> <b>But the point here</b> <b>is that the Japanese</b> <b>assumed that if they brought</b> <b>Polish Jews to Japan,</b> <b>the American Jewish community</b> <b>would welcome them here.</b> <b>They didn't want them to</b> <b>stay in Japan, of course.</b> <b>They thought they'd go</b> <b>on to the United States.</b> <b>They stayed in Japan.</b> <b>Nobody wanted them here,</b> <b>including the Jewish community.</b> <b>And there's plenty</b> <b>of stories like that.</b> <b>As soon as you start to</b> <b>lift the veil very slightly,</b> <b>you find it everywhere.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
So you think</b> <b>that the ultimate reason</b> <b>for these policies is racism?</b> <b>So rasicm on its own</b> <b>is the driving factor?</b> <b>Because there's been arguments</b> <b>that perhaps the racism</b> <b>itself is a result of other</b> <b>factors, like economic control.</b> <b>Think in the case</b> <b>of Haiti, right?</b> <b>Or the Caribbean in general.</b> <b>So what came first?</b> <b>Was it the racism first, or</b> <b>then the need for slaves,</b> <b>which in turn triggered racism?</b> <b>
It's</b> <b>a complicated story.</b> <b>I mean, racism grew partly</b> <b>out of the Enlightenment.</b> <b>If you go back to the 16th</b> <b>century, say in Europe.</b> <b>We're talking about Europe now.</b> <b>There was the beginnings</b> <b>of the age of exploration,</b> <b>exploring the world.</b> <b>The Spanish going to</b> <b>South America and so on.</b> <b>One of the things that was</b> <b>happening was that Europeans--</b> <b>recall that you go back a little</b> <b>further, to say St. Augustine.</b> <b>It was widely believed in</b> <b>Europe that nobody could</b> <b>live south of the equator.</b> <b>For one thing, they would</b> <b>be standing upside down.</b> <b>That would be impossible.</b> <b>So the idea that</b> <b>there were actually</b> <b>people there was kind</b> <b>of novel, anyhow.</b> <b>But as the age of</b> <b>exploration continued,</b> <b>Europeans were finding</b> <b>creatures of a kind</b> <b>that they had never seen</b> <b>before, like orangutans.</b> <b>And Africans-- blacks and so on.</b> <b>And there was a</b> <b>crucial question.</b> <b>Remember, these were</b> <b>deeply religious societies.</b> <b>You either had a soul or</b> <b>you didn't have a soul.</b> <b>That's critical.</b> <b>So the question arose, which</b> <b>of these creatures have souls?</b> <b>Can be converted to</b> <b>Christianity and saved?</b> <b>That was the dominant issue.</b> <b>And was there a distinction</b> <b>between, say, orangutans</b> <b>and black natives who they saw?</b> <b>How do you distinguish them?</b> <b>Can you distinguish them?</b> <b>And there were all kind of</b> <b>debates, including language.</b> <b>In fact, one of my favorite</b> <b>comments was by Louis Racine.</b> <b>He's the son of the</b> <b>famous dramatist who</b> <b>argued that apes</b> <b>did have language</b> <b>and they were more</b> <b>intelligent than humans.</b> <b>And he had a proof, actually.</b> <b>He said the proof</b> <b>is they don't speak.</b> <b>And it's a good argument.</b> <b>He said, they don't</b> <b>speak because they</b> <b>know that if they spoke, we</b> <b>would turn them into slaves.</b> <b>
Wow.</b> <b>
So</b> <b>therefore, they keep quiet,</b> <b>but they're really</b> <b>smarter than we are.</b> <b>But that's the kind of argument</b> <b>that was taken seriously.</b> <b>So what came first?</b> <b>What does it mean?</b> <b>The all integrate--</b> <b>Take right now the</b> <b>racism that's surfacing</b> <b>is in considerable</b> <b>measure a reaction</b> <b>to the neoliberal policies</b> <b>of the last generation,</b> <b>which have simply</b> <b>undermined a very large part</b> <b>of the population.</b> <b>In fact, for a majority</b> <b>of the population,</b> <b>conditions have</b> <b>stagnated or declined</b> <b>during the period that</b> <b>began basically with Reagan.</b> <b>So just take a look</b> <b>at the numbers.</b> <b>So in 2007-- that's</b> <b>right before the crash.</b> <b>And if you go back</b> <b>to that time, there</b> <b>was great euphoria among</b> <b>economists and intellectuals</b> <b>about the wonderful</b> <b>economy that we had.</b> <b>Everything had been solved.</b> <b>There's no more problems.</b> <b>Great moderation,</b> <b>perfect, and so on.</b> <b>1979.</b> <b>Real wages for American</b> <b>workers were actually lower</b> <b>than they had been in 1979</b> <b>before this grand experiment</b> <b>was initiated.</b> <b>And you see the same in Europe.</b> <b>Now, that's a large part of</b> <b>the reason for the anger,</b> <b>the contempt for institutions.</b> <b>The fear, and so on.</b> <b>And a lot of it</b> <b>shows up in racism.</b> <b>The racism is kind of an</b> <b>underlying phenomenon,</b> <b>but it surfaces when you</b> <b>have to blame somebody.</b> <b>And there's very</b> <b>interesting work on this.</b> <b>If you haven't seen it,</b> <b>there's a really fine book</b> <b>by a sociologist named</b> <b>Arlie Hochschild.</b> <b>I forget exactly</b> <b>what it's called,</b> <b>but she's a Berkeley University</b> <b>of California sociologist</b> <b>comes from a liberal background.</b> <b>She really wanted to</b> <b>understand, to see</b> <b>if she could understand</b> <b>what has always</b> <b>been regarded as</b> <b>a strange paradox,</b> <b>that the people</b> <b>who are suffering</b> <b>the most from these</b> <b>neoliberal policies</b> <b>are the ones who</b> <b>most support them.</b> <b>It's very striking</b> <b>when you look.</b> <b>We saw it in the last</b> <b>election, in fact.</b> <b>The people who voted for Trump--</b> <b>the working people-- are</b> <b>voting for their class</b> <b>enemy who's kicking them in</b> <b>the face at every opportunity.</b> <b>And the more he kicks</b> <b>them in the face,</b> <b>the more they support him.</b> <b>So what's going on?</b> <b>And it's very widespread.</b> <b>She went and lived and she</b> <b>picked an ultra-red county</b> <b>in Mississippi.</b> <b>The Bayou area of Mississippi.</b> <b>And she went to live there for,</b> <b>I think, five or six years.</b> <b>And just integrated</b> <b>herself into society.</b> <b>Got to know the people.</b> <b>Had respect for them.</b> <b>And she should,</b> <b>instead of contempt.</b> <b>She became part of the society.</b> <b>And she got to</b> <b>understand the way</b> <b>they were looking at things.</b> <b>And the way they were</b> <b>looking at things,</b> <b>she describes with an image</b> <b>that they accepted as accurate.</b> <b>The image of people</b> <b>standing in a line.</b> <b>So we're all standing in line.</b> <b>Behind us are our parents</b> <b>and our grandparents.</b> <b>And they worked hard.</b> <b>They did everything</b> <b>the right way.</b> <b>They're religious.</b> <b>They're conservative.</b> <b>They had families.</b> <b>Each generation gets</b> <b>a little better.</b> <b>That's the American way.</b> <b>But all of a sudden,</b> <b>the line stalled.</b> <b>We've stopped.</b> <b>Now, there are</b> <b>people ahead of us</b> <b>who are shooting into</b> <b>the stratosphere.</b> <b>Multimillionaires.</b> <b>But that's OK.</b> <b>That's the American way.</b> <b>The problem is the</b> <b>people behind us.</b> <b>The federal</b> <b>government, its role is</b> <b>to take those people</b> <b>who are behind us,</b> <b>who don't want to work, who</b> <b>are worthless, and so on.</b> <b>And the federal government</b> <b>takes them and puts them</b> <b>in front of us with affirmative</b> <b>action programs, and soup</b> <b>kitchens, and things like that.</b> <b>I mean, it's all a</b> <b>complete fantasy,</b> <b>but this is the way</b> <b>people see the world.</b> <b>And you can understand why</b> <b>they see the world that way.</b> <b>These are people</b> <b>who heard Ronald</b> <b>Reagan, an extreme</b> <b>racist incidentally,</b> <b>give his disquisitions</b> <b>about welfare queens driving</b> <b>in limousines to</b> <b>the welfare office--</b> <b>black, of course-- to pick</b> <b>up your hard-earned money.</b> <b>They're just saturated with that</b> <b>kind of stuff from talk radio,</b> <b>from Fox, everything they hear.</b> <b>So these people</b> <b>behind us are being</b> <b>pushed by the federal</b> <b>government in front of us.</b> <b>And we're stalled.</b> <b>Well, they are stalled,</b> <b>but it's not because</b> <b>of those people behind them.</b> <b>And you take a look</b> <b>at that amalgam</b> <b>and you can see why these</b> <b>people hate the government.</b> <b>We don't want the government,</b> <b>even though the government</b> <b>is keeping us alive.</b> <b>In fact, if you</b> <b>look at it, they're</b> <b>getting more subsidies from</b> <b>the government than anyone.</b> <b>Like Mississippi is</b> <b>subsidized by New York.</b> <b>It's the way</b> <b>transfers take place.</b> <b>But these are all</b> <b>kind of hidden.</b> <b>You don't see the subsidies.</b> <b>What you see is these</b> <b>people behind us</b> <b>who are maybe getting</b> <b>some Medicaid,</b> <b>let's say, something like that.</b> <b>That's what you see.</b> <b>A well-designed doctrinal</b> <b>system, propaganda system,</b> <b>can focus your</b> <b>attention on that.</b> <b>I think we're going</b> <b>to see more of that.</b> <b>Trump, his promises about</b> <b>bringing jobs back, obviously</b> <b>not going to work.</b> <b>What happens when his</b> <b>constituency recognizes</b> <b>that they're just getting</b> <b>kicked in the face?</b> <b>Well, what's going to</b> <b>happen is scapegoating.</b> <b>It's going to be</b> <b>necessary to turn</b> <b>their attention to somebody</b> <b>who's doing that to them.</b> <b>The Jews, the Muslims,</b> <b>the immigrants,</b> <b>the blacks, whoever it may be.</b> <b>And that could lead to</b> <b>a really ugly period</b> <b>unless there's some</b> <b>reaction, strong reaction.</b> <b>And the reaction ought to</b> <b>begin before it takes place,</b> <b>not after it takes place.</b> <b>But you can see what's</b> <b>very likely to happen.</b> <b>And there's plenty of</b> <b>examples in history,</b> <b>like the Nazis for example.</b> <b>Plenty of examples</b> <b>in our own history.</b> <b>Many.</b> <b>Race riots back in our own</b> <b>history are often like this.</b> <b>Take, say, the Irish.</b> <b>When the Irish came in</b> <b>the late 19th century,</b> <b>they were regarded as</b> <b>black, dark-skinned.</b> <b>They were treated like blacks.</b> <b>There were signs in Boston</b> <b>restaurants saying, no dogs</b> <b>or Irish, things like that.</b> <b>They finally whitened, became</b> <b>integrated into the society.</b> <b>But the racist treatment</b> <b>of Irish was miserable.</b> <b>In fact, there are some</b> <b>hidden stories there,</b> <b>which aren't so pretty.</b> <b>Like take gynecology.</b> <b>Gynecology was developed by</b> <b>professors at Harvard Medical</b> <b>School.</b> <b>Their pictures are up</b> <b>on the walls and so on.</b> <b>How did they do it?</b> <b>Well, you had to experiment.</b> <b>So who'd you experiment with?</b> <b>People who were good subjects.</b> <b>Black women, of course,</b> <b>and Irish women.</b> <b>They were the</b> <b>experimental subjects</b> <b>who were used to develop the</b> <b>modern scientific understanding</b> <b>of gynecology.</b> <b>This kind of thing just</b> <b>runs all through history.</b> <b>Everywhere you look, you find</b> <b>one or another aspect of it.</b> <b>And getting back to</b> <b>Michel's question,</b> <b>I don't think there's</b> <b>a chicken/egg issue.</b> <b>They all interact.</b> <b>
One other</b> <b>place where they interact</b> <b>is with language, right?</b> <b>So you mentioned</b> <b>the one drop rule.</b> <b>In Haiti now, when you look</b> <b>at, say, the politics there.</b> <b>Let's say it's put on a zenith.</b> <b>And you can see how based on</b> <b>the way you speak, what you say</b> <b>is being discarded.</b> <b>And we could call it</b> <b>the one accent rule.</b> <b>So if you shoot any</b> <b>creole phonemic pattern</b> <b>in your French, then</b> <b>you're discarded.</b> <b>And I think that's</b> <b>also true in the US.</b> <b>So in the US, there</b> <b>was this famous case</b> <b>of Rachel Jeantel,</b> <b>when she was testifying</b> <b>in the trial of Zimmerman,</b> <b>who had killed Trayvon Martin.</b> <b>That's how the Black Lives</b> <b>Matter movement got launched.</b> <b>And she was discarded</b> <b>by the jury.</b> <b>Why?</b> <b>Because she spoke with</b> <b>black English accent.</b> <b>And she was judged</b> <b>to be untrustworthy.</b> <b>And the jury never</b> <b>mentioned the testimony.</b> <b>And she was a star</b> <b>witness for the trial.</b> <b>So there is a</b> <b>linguist at Stanford,</b> <b>John Rickford, and</b> <b>one of his students</b> <b>who wrote a beautiful</b> <b>piece, actually showing</b> <b>how because of</b> <b>the way she spoke,</b> <b>what she said was given no</b> <b>credibility by the jury.</b> <b>And this is why I think</b> <b>that perhaps one could think</b> <b>of race alongside language</b> <b>as one of these tools</b> <b>that you can use to create</b> <b>this illusion of hierarchies.</b> <b>And maybe you're right.</b> <b>The two might reinforce</b> <b>each other, but perhaps</b> <b>this push for economic</b> <b>control and imperialism</b> <b>in the case of the Caribbean</b> <b>colonization and slavery,</b> <b>they become enlisted</b> <b>by these larger forces</b> <b>to impose these hierarchies.</b> <b>If you go say in</b> <b>French, the [french]</b> <b>that the French</b> <b>came up with, which</b> <b>we saw coming back up with</b> <b>Marine Le Pen, for example.</b> <b>
We've all seen</b> <b>that in our own experience.</b> <b>When I was in college,</b> <b>one of my fellow students</b> <b>was one of the rare</b> <b>black students, who</b> <b>had managed to develop not just</b> <b>a straight American accent,</b> <b>but an elite to American accent.</b> <b>So when he talked</b> <b>over the phone,</b> <b>you thought you were</b> <b>talking to a Harvard</b> <b>professor, or an Oxford</b> <b>professor, or something.</b> <b>We brought him to MIT.</b> <b>We appointed him, in fact.</b> <b>But when he came, he was</b> <b>staying in our apartment</b> <b>while he was looking</b> <b>for a place to live.</b> <b>And he would call places that</b> <b>were for rent around Cambridge.</b> <b>And they'd say, sure.</b> <b>Come over.</b> <b>But as soon as he</b> <b>went over, it suddenly</b> <b>turned out that</b> <b>the place had just</b> <b>been rented five minutes ago.</b> <b>But over the phone, he was fine.</b> <b>But not in person.</b> <b>The picking up of</b> <b>a fake acc-- it</b> <b>shows up in many strange ways.</b> <b>So for example, at Harvard</b> <b>in the early 1950s,</b> <b>there was a wave of Anglophilia.</b> <b>And people-- men, of course.</b> <b>No women-- dressed</b> <b>in English clothes</b> <b>and affected British accents.</b> <b>And if you kind of just</b> <b>listened and walked around,</b> <b>you might have thought</b> <b>you're maybe in Oxford,</b> <b>not in Cambridge.</b> <b>But these things</b> <b>happen all the time.</b> <b>They can be very</b> <b>pernicious when it's</b> <b>a part of the population</b> <b>that's vulnerable and deprived</b> <b>and under external</b> <b>pressures for other reasons.</b> <b>Of course, African Americans,</b> <b>as I said, it's 400 years.</b> <b>It's not something sudden.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
I think with</b> <b>the past election--</b> <b>so Donald Trump is, obviously,</b> <b>one of the 1%, right?</b> <b>So why do you think there's</b> <b>such different views on</b> <b>whether or not he's</b> <b>representative of America,</b> <b>whether being the American</b> <b>dream or being or being one</b> <b>of the "crooks" in the top 1%?</b> <b>
For one</b> <b>thing, there's no America.</b> <b>There's many splits.</b> <b>And some of the splits</b> <b>are quite sharp.</b> <b>In fact, this country has become</b> <b>polarized in the last 50 years</b> <b>to an extent that's</b> <b>never happened before.</b> <b>In fact, just take a look at</b> <b>the two political parties.</b> <b>Up until the 1960s, the parties</b> <b>weren't all that different.</b> <b>You couldn't distinguish</b> <b>a moderate Republican</b> <b>from a liberal democrat.</b> <b>They're more or less alike.</b> <b>By now, the parties are so</b> <b>differentiated that literally</b> <b>people don't intermarry</b> <b>across parties.</b> <b>And it's become--</b> <b>literally, there</b> <b>are some good studies of it.</b> <b>And what actually happened</b> <b>is it's connected with race.</b> <b>Up until the 1960s,</b> <b>the Democrats</b> <b>were kind of a coalition</b> <b>of Southern Democrats,</b> <b>all extreme racists,</b> <b>and people who</b> <b>were very powerful because</b> <b>they stayed in office forever</b> <b>since the South was</b> <b>a one-party region.</b> <b>Take a look at "The New</b> <b>York Times" this morning.</b> <b>There's a headline</b> <b>about one-party Alabama.</b> <b>Everybody is a Republican,</b> <b>as if that's something new.</b> <b>That goes back to the</b> <b>Civil War, except now they</b> <b>call them Republicans.</b> <b>They used to call</b> <b>them Democrats.</b> <b>But same system, basically.</b> <b>But the Southern Democrats</b> <b>were extremely powerful</b> <b>because the senators</b> <b>had senior positions.</b> <b>They'd been there forever.</b> <b>There was a big block.</b> <b>They had a big effect</b> <b>on legislation.</b> <b>So if you look at the</b> <b>New Deal legislation,</b> <b>there was a lot of progress.</b> <b>But it was straight racist.</b> <b>So Social Security, for example.</b> <b>The Southern Democrats</b> <b>were willing to permit</b> <b>it to go through as long</b> <b>as it excluded blacks.</b> <b>And it did.</b> <b>So Social Security was</b> <b>designed to exclude</b> <b>the majority of the workforce,</b> <b>agricultural and domestic</b> <b>workers.</b> <b>Namely, African Americans.</b> <b>The GI Bill gave--</b> <b>it was a segregated army.</b> <b>So it didn't go to blacks.</b> <b>All the way through,</b> <b>the Southern Democrats</b> <b>were willing to tolerate</b> <b>reformist measures</b> <b>as long as it maintained the</b> <b>Southern system of deep racism.</b> <b>And what amounted to</b> <b>new kinds of slavery.</b> <b>Well, what happened in the</b> <b>1960s was the civil rights</b> <b>movement broke through this.</b> <b>The Democrats began to pick</b> <b>up civil rights legislation.</b> <b>And the Southern Democrats</b> <b>simply pulled out.</b> <b>That's Nixon's famous</b> <b>Southern strategy,</b> <b>let's appeal to the South and</b> <b>turn them into Republicans</b> <b>on straight racist grounds.</b> <b>So the parties realigned.</b> <b>And a large part</b> <b>of rural America</b> <b>went along with it, too.</b> <b>That's quite separate</b> <b>from urban America.</b> <b>The lines became sharper during</b> <b>the neoliberal years, when</b> <b>rural America was just smashed.</b> <b>Rural America meant working</b> <b>class, industrial America.</b> <b>Small towns with those</b> <b>industries and so on.</b> <b>That was all smashed.</b> <b>Urban America, that's</b> <b>where the 1% is.</b> <b>That's where we come from--</b> <b>the people who prosper, the</b> <b>more educated, and so on.</b> <b>So these splits became</b> <b>extremely sharp.</b> <b>It's increased with new media.</b> <b>So the cable television,</b> <b>social media, and so on.</b> <b>One of their effects is</b> <b>to intensify the split.</b> <b>So one part of America lives</b> <b>in talk radio and Fox News.</b> <b>I don't know if you</b> <b>listen to talk radio here,</b> <b>but you should.</b> <b>I often listen to</b> <b>it when I'm driving.</b> <b>It's very interesting.</b> <b>And what's interesting is not</b> <b>Rush Limbaugh and those guys.</b> <b>They're predictable.</b> <b>But the people who call</b> <b>in are what's interesting.</b> <b>And that's a slice of Boston.</b> <b>We're not talking</b> <b>about Mississippi.</b> <b>We're talking about Boston.</b> <b>It's a slice of Boston which</b> <b>is in some other universe.</b> <b>The things they</b> <b>believe, the things they</b> <b>hear, the things they read.</b> <b>That's why, I forget</b> <b>the exact number.</b> <b>But maybe half-- some large</b> <b>percentage of Republicans</b> <b>think that Obama is a</b> <b>Muslim and that he wasn't</b> <b>born in the United States.</b> <b>You hear that drummed into</b> <b>your head all the time.</b> <b>OK.</b> <b>And of course, the hatred of</b> <b>Obama was substantially racist.</b> <b>I mean, why is the Affordable</b> <b>Care Act called Obamacare</b> <b>by Democrats, by liberals?</b> <b>Was Medicare called Johnsoncare?</b> <b>Of course not.</b> <b>We just accept the racism.</b> <b>It's part of us.</b> <b>So sure, it's Obamacare.</b> <b>This black guy</b> <b>pushed it through.</b> <b>And that's liberal</b> <b>Democrats call it that.</b> <b>We don't even think about it.</b> <b>But it's because it's</b> <b>all deeply ingrained.</b> <b>And when you get to</b> <b>parts of the culture</b> <b>where that's all they hear--</b> <b>talk radio, Rush Limbaugh,</b> <b>Loren McLaughlin,</b> <b>or whatever her name is.</b> <b>You really have to hear</b> <b>this stuff to believe it.</b> <b>And for large parts of</b> <b>rural, lower middle-class,</b> <b>small businessmen, the kind</b> <b>of people who vote for Trump,</b> <b>that's what they live with.</b> <b>And by now, with</b> <b>the reconstruction</b> <b>of the political parties--</b> <b>the Republicans are</b> <b>an interesting party.</b> <b>They are slavishly dedicated to</b> <b>the welfare of the super-rich</b> <b>and the corporations.</b> <b>Take a look at their</b> <b>legislative programs.</b> <b>Paul Ryan is the</b> <b>extreme example.</b> <b>The most savage, brutal</b> <b>programs you can think of.</b> <b>So that's the actual</b> <b>Republican Party.</b> <b>But their constituency</b> <b>is people who are</b> <b>being harmed by these programs.</b> <b>It's the kind of paradox</b> <b>that Arlie Hochschild</b> <b>wanted to understand.</b> <b>And I think when you read</b> <b>through what she found,</b> <b>you can understand it.</b> <b>But even true of things</b> <b>like environmental issues.</b> <b>So in the Bayou area where</b> <b>she was working and living,</b> <b>there's an extremely</b> <b>high rate of cancer.</b> <b>In fact, it's</b> <b>called Cancer Alley.</b> <b>And the reasons for it are</b> <b>the petrochemical corporations</b> <b>who just pour poisonous</b> <b>chemicals into the Bayou.</b> <b>And these are people who</b> <b>live in the environment.</b> <b>They hunt.</b> <b>They fish.</b> <b>Quite apart from the</b> <b>fact that their children</b> <b>are dying of cancer.</b> <b>These people are</b> <b>environmentalists.</b> <b>And the people they</b> <b>vote for are the ones</b> <b>who want to destroy the EPA.</b> <b>And when you look into it, it</b> <b>turns out there's a reason.</b> <b>For them, what the</b> <b>EPA means is some guy</b> <b>in a suit who comes from</b> <b>Washington and tells us,</b> <b>you can't fish here.</b> <b>That's the EPA.</b> <b>But that guy does nothing</b> <b>about the petrochemical plants.</b> <b>So why shouldn't they want</b> <b>to get rid of the EPA?</b> <b>That's their lives.</b> <b>That's the kind of lives</b> <b>that these people are living.</b> <b>And when you see it firsthand--</b> <b>and even you can see it right</b> <b>here if you want to--</b> <b>you can understand</b> <b>the attitudes.</b> <b>They're not as</b> <b>paradoxical as they sound.</b> <b>It's leading to a very strange</b> <b>situation in the country.</b> <b>Very dangerous one.</b> <b>Incidentally, most</b> <b>of the Trump voters</b> <b>were not working class and poor.</b> <b>They're mostly kind of lower</b> <b>middle-class or very rich.</b> <b>Those two groups.</b> <b>But there were working</b> <b>people who voted for Trump,</b> <b>or who voted for</b> <b>Scott Brown here.</b> <b>And many of those</b> <b>people voted for Obama.</b> <b>They believed his rhetoric</b> <b>about hope and change.</b> <b>And when they saw very quickly</b> <b>that hope and change means</b> <b>money for bankers</b> <b>and nothing for us,</b> <b>they became deeply</b> <b>disillusioned.</b> <b>And so they're now trusting</b> <b>the word of a con man who</b> <b>also says hope and change.</b> <b>But it's not that different</b> <b>from the 2008 election</b> <b>in some respects.</b> <b>Just a different con man.</b> <b>
How do</b> <b>you think we could</b> <b>go about trying to change</b> <b>the attitudes of like, what</b> <b>people think that the EPA does?</b> <b>Or like, that idea</b> <b>of being in a line?</b> <b>How do you think we</b> <b>could change that?</b> <b>
By</b> <b>interacting with people.</b> <b>It's the only way.</b> <b>That's what organizing</b> <b>and activism are about.</b> <b>You have to be in touch with the</b> <b>people who have these beliefs.</b> <b>Sympathetically, figure out--</b> <b>come to understand why they</b> <b>have the beliefs and interact--</b> <b>show them.</b> <b>Try to lead them to</b> <b>understand that there's</b> <b>another way of looking at it.</b> <b>That's what education is about.</b> <b>That's what organizing is about.</b> <b>You can do it right in Boston.</b> <b>That's why activists</b> <b>live in communities,</b> <b>work with communities.</b> <b>
Any</b> <b>questions on that side?</b> <b>
Speaking to</b> <b>what we've been discussing</b> <b>these last few minutes,</b> <b>I think there's</b> <b>a tendency in a lot of--</b> <b>not only in politics,</b> <b>but just in general</b> <b>to not be as willing to try and</b> <b>understand the point of view</b> <b>of somebody who opposes you.</b> <b>I don't know-- speaking</b> <b>now to politics</b> <b>specifically-- if that's</b> <b>something that's kind of arisen</b> <b>the last couple decades,</b> <b>or that has some sort</b> <b>of historical precedence.</b> <b>And given the</b> <b>polarization I've seen</b> <b>in media that's now trickling</b> <b>into various political</b> <b>constituencies, have you</b> <b>seen any similar effects</b> <b>in any, maybe historical</b> <b>elections or any sort</b> <b>of prior settings?</b> <b>
This inability</b> <b>to hear the other person?</b> <b>
Yeah.</b> <b>
Sometimes, it</b> <b>takes really extreme forms.</b> <b>Take Clinton.</b> <b>I think maybe the low</b> <b>point of the last election</b> <b>was her mocking of</b> <b>the deplorables, which</b> <b>had a big effect.</b> <b>Here is this rich,</b> <b>elite woman who goes--</b> <b>talks to Wall Street.</b> <b>Gets huge honoraria talking</b> <b>to Wall Street bankers.</b> <b>She's calling us deplorable.</b> <b>Why are we deplorable?</b> <b>We're hardworking and honest,</b> <b>religious, conservative,</b> <b>traditional people.</b> <b>What makes us deplorable?</b> <b>But that's an extreme</b> <b>example of what</b> <b>we see around us all the time.</b> <b>This contempt for the,</b> <b>say, Trump voters.</b> <b>How can they be so stupid?</b> <b>How can they be so ridiculous?</b> <b>They're not humans.</b> <b>And they feel the same way</b> <b>towards the people walking</b> <b>around Harvard Square.</b> <b>There's plenty of</b> <b>that all the back.</b> <b>Like Benjamin Franklin</b> <b>saying we shouldn't</b> <b>admit Swedes and Germans.</b> <b>Not very different.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
Do you think</b> <b>there's been any developments</b> <b>policy-wise that are</b> <b>helping the situation, that</b> <b>are making the situation better</b> <b>in any way in the recent years?</b> <b>Or is it all coming</b> <b>down, I guess?</b> <b>
Well, I think</b> <b>it goes in many directions.</b> <b>So actually, I think</b> <b>the most dramatic fact</b> <b>about the last election,</b> <b>the 2016 election,</b> <b>was not the election of Trump.</b> <b>The most striking fact</b> <b>was the Sanders campaign.</b> <b>I mean, the fact that a</b> <b>billionaire was elected.</b> <b>OK, it's not a big break</b> <b>from American history.</b> <b>That goes on all the time.</b> <b>But the Sanders</b> <b>campaign broke sharply</b> <b>with over a hundred years of</b> <b>American political history.</b> <b>I mean, if we had media that</b> <b>were talking about reality,</b> <b>that would be the headline.</b> <b>American elections are bought.</b> <b>There's plenty of substantial</b> <b>mainstream political science</b> <b>research on this.</b> <b>If you look just at</b> <b>simple variables,</b> <b>like, say, campaign funding--</b> <b>something as simple as that--</b> <b>the predictability</b> <b>of electability</b> <b>by campaign funding is literally</b> <b>almost a straight line.</b> <b>There's a recent</b> <b>study by Tom Ferguson</b> <b>over at UMass, one</b> <b>of the main scholars</b> <b>who works on this, who studied</b> <b>congressional election--</b> <b>for presidential elections have</b> <b>been shown for a long time.</b> <b>But that was the</b> <b>first main study</b> <b>of congressional elections.</b> <b>They went from</b> <b>1980 to the present</b> <b>all over the country</b> <b>simply asking,</b> <b>what's the relation</b> <b>between campaign spending</b> <b>and electability?</b> <b>And it is literally</b> <b>a straight line.</b> <b>You just don't get results like</b> <b>that in the social sciences.</b> <b>The more campaign spending,</b> <b>the more you get elected.</b> <b>And that goes back well</b> <b>into the late 19th century.</b> <b>There's a famous</b> <b>campaign manager,</b> <b>Mark Hanna, back 1890s.</b> <b>He was asked once, what's</b> <b>the secret for running</b> <b>a successful election campaign?</b> <b>And he said, you have</b> <b>to have two things.</b> <b>The first one is money.</b> <b>And I can't remember</b> <b>the second one.</b> <b>That's American</b> <b>politics, literally.</b> <b>Just with overwhelming</b> <b>near-certainty.</b> <b>Well, all of a sudden, you</b> <b>get a guy who's unknown,</b> <b>has no support from any</b> <b>of the sources of wealth,</b> <b>no support from the wealthy,</b> <b>no corporate support, dismissed</b> <b>by the medial as ridiculous,</b> <b>very negative media campaign.</b> <b>He was basically unknown.</b> <b>And he even used a</b> <b>scare word "socialist.
"</b> <b>He would have won the</b> <b>Democratic Party nomination</b> <b>if it hadn't been for the</b> <b>Clinton-Obama shenanigans</b> <b>to keep the party</b> <b>from reflecting</b> <b>its popular constituency.</b> <b>That's an incredible</b> <b>break from history.</b> <b>And right now,</b> <b>incidentally, he is</b> <b>by far the most popular</b> <b>political figure</b> <b>in the country.</b> <b>Way above anyone else.</b> <b>Well, those are signs</b> <b>that other things are--</b> <b>especially among young</b> <b>people, he's way up.</b> <b>That's the future electorate.</b> <b>So these are things</b> <b>that are going off</b> <b>in the other direction.</b> <b>There's plenty of opportunities,</b> <b>if they're grasped.</b> <b>And it's kind of easy to sink</b> <b>into sort of hopelessness.</b> <b>What's the point?</b> <b>But it's just really</b> <b>a bad mistake.</b> <b>First of all, the issues</b> <b>are extremely important.</b> <b>There's an awful</b> <b>lot can be done.</b> <b>Even in the kind of thing that</b> <b>Arlie Hochschild was doing,</b> <b>reaching people who</b> <b>were deeply embedded</b> <b>in the ultra-red</b> <b>constituency, the deeply</b> <b>religious, conservative,</b> <b>traditional, anti-government</b> <b>for the reasons that they</b> <b>see, which you can understand,</b> <b>can be changed.</b> <b>Incidentally, if you</b> <b>look back in the past,</b> <b>the same sectors</b> <b>of the population</b> <b>were the most carried</b> <b>forward, the most extreme</b> <b>radical democratic programs</b> <b>in American history.</b> <b>The term "populism"</b> <b>is thrown around now,</b> <b>but it used to mean something.</b> <b>And the populist movement</b> <b>in the United States,</b> <b>which began with</b> <b>farmers in Texas</b> <b>and spread through the Midwest,</b> <b>to Kansas and place like that,</b> <b>was a really powerful,</b> <b>radical democratic movement.</b> <b>Linked up with the</b> <b>Knights of Labor,</b> <b>the main labor movement, whose</b> <b>slogan was the working people</b> <b>ought to own and</b> <b>run the factories.</b> <b>And it was smashed</b> <b>by force finally,</b> <b>but this is by far the most</b> <b>real democratic movement</b> <b>in American history.</b> <b>And maybe the world,</b> <b>that comes out</b> <b>of Texas and Kansas farmers.</b> <b>It's the same people.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
So my question</b> <b>is sort of two-fold.</b> <b>It sort of goes back on</b> <b>some of the other things</b> <b>we've discussed with</b> <b>the recent elections.</b> <b>So I've recently been</b> <b>following some of your stuff.</b> <b>And there's evidence</b> <b>that suggests</b> <b>that you sort of predicted</b> <b>the rise of the Trump campaign</b> <b>about seven years ago.</b> <b>And I'm sort of curious,</b> <b>what was so different</b> <b>about this election?</b> <b>I presume you've seen the rise</b> <b>of many presidential candidates</b> <b>over the years.</b> <b>And I'm sort of</b> <b>curious to understand,</b> <b>what sort of prompted</b> <b>that speculation?</b> <b>And the second</b> <b>part of my question</b> <b>sort of is looking more</b> <b>so into the future.</b> <b>So as we've discussed, America</b> <b>is probably more polarized.</b> <b>And it's sort of been--</b> <b>in the past.</b> <b>And there's s of this</b> <b>idea that the clustering</b> <b>of like-minded Americans</b> <b>is tearing us apart.</b> <b>So where do you see the</b> <b>future of this nation going?</b> <b>
</b> <b>Well, I mean, there</b> <b>are many special factors</b> <b>in the United States.</b> <b>Societies are different.</b> <b>But notice that what happened</b> <b>in this election, what actually</b> <b>happened in the 2016 election</b> <b>is that the mainstream</b> <b>establishment institutions</b> <b>were basically destroyed.</b> <b>The two major phenomena</b> <b>in this election</b> <b>were the Sanders campaign</b> <b>among the Democrats</b> <b>and the Trump campaign</b> <b>among the Republicans.</b> <b>And they were both</b> <b>anti-establishment.</b> <b>And the voters went that way.</b> <b>If you take a look at polling</b> <b>results, you can see it.</b> <b>I mean, anger and</b> <b>contempt for institutions</b> <b>is just phenomenal.</b> <b>If you ask people in</b> <b>polls, do you have any--</b> <b>what do you think</b> <b>about Congress?</b> <b>You know, 90% say they</b> <b>should be all thrown out.</b> <b>In fact, about the</b> <b>only institution</b> <b>that has any respect</b> <b>is the military</b> <b>because you're taught</b> <b>somehow, the military</b> <b>is defending us</b> <b>from who knows what.</b> <b>And the same is true in Europe.</b> <b>Take a look at the French</b> <b>election a couple of days ago.</b> <b>The two candidates</b> <b>were outsiders.</b> <b>The political parties</b> <b>barely participated.</b> <b>Macron and Le Pen are from</b> <b>outside the mainstream that</b> <b>has run Europe for 50 years.</b> <b>The center right and center</b> <b>left parties are gone.</b> <b>Same thing happened in</b> <b>England with Brexit.</b> <b>Same thing's happening in</b> <b>other European countries.</b> <b>We're seeing the phenomenon</b> <b>all over the Western world</b> <b>in which the centrist--</b> <b>center left, center right--</b> <b>institutions are under attack.</b> <b>People are angry about them.</b> <b>They hate them.</b> <b>They think they're</b> <b>not working for us.</b> <b>And they're right.</b> <b>That's the effect of</b> <b>the neoliberal programs.</b> <b>We're all subjected to the</b> <b>same socioeconomic programs.</b> <b>They've had a similar</b> <b>effect everywhere.</b> <b>They've led to</b> <b>stagnation, decline,</b> <b>undermining of public</b> <b>services, inability--</b> <b>Reagan's slogan, the state is</b> <b>the problem, not the solution.</b> <b>Let's turn it over to private</b> <b>tyrannies, which make it worse.</b> <b>That's happening.</b> <b>Bits and pieces of that</b> <b>are happening everywhere.</b> <b>So it happened in</b> <b>the United States</b> <b>in its own particular way.</b> <b>But notice that</b> <b>it's two directions.</b> <b>It's Sanders and Trump.</b> <b>And I think in</b> <b>many ways, they're</b> <b>appealing to the</b> <b>same sentiments.</b> <b>And they could even converge.</b> <b>A lot of the pop--</b> <b>if you look at what</b> <b>people believe,</b> <b>the policies that they favor.</b> <b>A lot of it's shared.</b> <b>So for example,</b> <b>across the population,</b> <b>all the way over</b> <b>to the Tea Party,</b> <b>people think there ought to</b> <b>be higher taxes on the rich.</b> <b>Probably a majority</b> <b>of the population</b> <b>thinks there ought to</b> <b>be national health care.</b> <b>In fact, the support for</b> <b>nationally guaranteed health</b> <b>care has been so</b> <b>high over the years,</b> <b>that sometimes it's</b> <b>almost hard to believe.</b> <b>So in the late Reagan years,</b> <b>for example, about 70%</b> <b>of the population thought that</b> <b>guaranteed national health care</b> <b>ought to be in the</b> <b>Constitution because it's</b> <b>such an obvious demand.</b> <b>About 40% of the population</b> <b>thought it already</b> <b>was in the Constitution.</b> <b>Nobody knows what's</b> <b>in the Constitution.</b> <b>Just everything good must</b> <b>be in the Constitution.</b> <b>Since this is so obviously good,</b> <b>it must be in the Constitution.</b> <b>And right at the present,</b> <b>the most recent polls</b> <b>show somewhat over 50%</b> <b>supporting national health</b> <b>care.</b> <b>When Obama's Affordable Care</b> <b>Act was being pushed through,</b> <b>at first one of the proposals,</b> <b>one of the parts of it,</b> <b>was what was called a</b> <b>public option, which</b> <b>means you could</b> <b>choose if you wanted</b> <b>to be part of a government</b> <b>health care program.</b> <b>About two thirds of the</b> <b>population favored that,</b> <b>but it was just thrown</b> <b>out without discussion.</b> <b>And in fact, if you</b> <b>go case by case,</b> <b>there's a lot of</b> <b>similarity in the attitudes</b> <b>about social/political issues.</b> <b>Not necessarily cultural issues,</b> <b>but social/political issues</b> <b>among parts of the</b> <b>population that appear</b> <b>to be extremely polarized.</b> <b>It's very interesting.</b> <b>In England right now, if you</b> <b>followed it, Jeremy Corbyn,</b> <b>who's kind of a--</b> <b>a little bit like</b> <b>Sanders, is bitterly</b> <b>attacked by the media,</b> <b>including the liberal media,</b> <b>like The Guardian and by the</b> <b>parliamentary labor party.</b> <b>Meaning the guys who sort of--</b> <b>the elites who run the party.</b> <b>They just came out with a--</b> <b>I happened to be in England</b> <b>a couple of days ago,</b> <b>so I was reading</b> <b>the British press.</b> <b>They just came out with--</b> <b>the labor party--</b> <b>with their program,</b> <b>which was just</b> <b>lambasted by the media--</b> <b>crazy, lunatic, insane.</b> <b>It will destroy the labor party.</b> <b>At the same time, polls</b> <b>came out asking people</b> <b>about the components</b> <b>of the program.</b> <b>They're all very popular.</b> <b>Very strong support among the</b> <b>population for the elements</b> <b>of the program.</b> <b>But attack on the party</b> <b>that's proposing them.</b> <b>And in particular, on Corbyn.</b> <b>Serious activism and</b> <b>organizing could overcome this.</b> <b>I mean, you're</b> <b>fighting elite elements</b> <b>who have their own image of the</b> <b>way they want society to go.</b> <b>And in fact, there's quite a lot</b> <b>of interesting things going on.</b> <b>A lot of it is stuff that you</b> <b>guys might be interested in.</b> <b>It has to do with</b> <b>data harvesting.</b> <b>There's a company called</b> <b>Cambridge Analytics, I think,</b> <b>which is run by some</b> <b>ultra-right billionaires.</b> <b>Peter Thiel, the</b> <b>guy who owns PayPal</b> <b>and a bunch of other things.</b> <b>Robert Mercer, who's a</b> <b>big hedge fund manager,</b> <b>multi-billionaire,</b> <b>a couple of others.</b> <b>What they've been doing--</b> <b>and this essentially gave</b> <b>the election to Trump--</b> <b>is extremely careful</b> <b>data analysis.</b> <b>There's a ton of data</b> <b>now from Facebook</b> <b>and other things about</b> <b>people's personal preferences</b> <b>and attitudes and feelings.</b> <b>Are you depressed,</b> <b>all kinds of things.</b> <b>And what they're doing is</b> <b>looking carefully at this data</b> <b>and analyzing it</b> <b>to see if you can</b> <b>find small numbers of people</b> <b>in particular areas who</b> <b>are vulnerable to one or</b> <b>another form of manipulation.</b> <b>And what they're doing</b> <b>is trying to say, look.</b> <b>If they kind of</b> <b>tend to be liberal,</b> <b>try to get them to</b> <b>get so disillusioned</b> <b>that they won't vote.</b> <b>And if they tend to be, say,</b> <b>traditionally conservative,</b> <b>maybe religious or whatever,</b> <b>try to get them to--</b> <b>reach them individually</b> <b>in ways which</b> <b>will increase the</b> <b>probability they'll</b> <b>vote for the right wing.</b> <b>This has small effects,</b> <b>but the elections</b> <b>are won on very small effects.</b> <b>As you know, Trump</b> <b>lost the popular vote</b> <b>by a couple of million.</b> <b>But in particular counties</b> <b>that made the difference,</b> <b>a couple of 100,000 votes</b> <b>shifted the election.</b> <b>In England, Brexit was the same.</b> <b>They estimate about</b> <b>maybe 600,000 votes.</b> <b>The whole country switched.</b> <b>Now, these companies using a lot</b> <b>of complicated data processing</b> <b>and statistics, and so</b> <b>on, are working on ways</b> <b>to try to control the</b> <b>electoral process.</b> <b>I mean, this is way</b> <b>more important than any</b> <b>of the nonsense about</b> <b>the Russian hacking.</b> <b>That's all garbage.</b> <b>But this stuff is real.</b> <b>And it's right in</b> <b>front of our eyes.</b> <b>It's the people we know.</b> <b>MIT students going off</b> <b>to work on these things.</b> <b>And it has a big</b> <b>effect on controlling</b> <b>formal democratic systems.</b> <b>This is Steve Bannon.</b> <b>This is his image of how you</b> <b>control the world, you know?</b> <b>These are people</b> <b>who are influential.</b> <b>They have tons of money.</b> <b>Deeply reactionary.</b> <b>And they want to use</b> <b>modern data processing.</b> <b>You know, big data, statistical</b> <b>analysis, and so on,</b> <b>to see if you can just find</b> <b>ways of swinging elections</b> <b>by going after individual</b> <b>vulnerabilities using</b> <b>the massive data that people</b> <b>provide through Google</b> <b>and Facebook, and</b> <b>all of this stuff.</b> <b>Google and the rest of them</b> <b>use it for advertisers,</b> <b>so that advertisers can go</b> <b>after you individually and say--</b> <b>you try to get a book on Amazon</b> <b>and they suggest 10 other books</b> <b>you ought to buy because they've</b> <b>got so much information on you</b> <b>that they think, maybe I</b> <b>can sort of sell this one.</b> <b>That's going on all the time.</b> <b>But here, it's right in the</b> <b>middle of a political system.</b> <b>And it's really significant.</b> <b>Quite unlike the Russian</b> <b>hacking story, this is real.</b> <b>And it's not very far from us.</b> <b>In fact, it's</b> <b>happening right here.</b> <b>There are things that</b> <b>can be done about that.</b> <b>Like exposing it for one thing.</b> <b>These should be live issues,</b> <b>I think, amongst students.</b> <b>Do we want to devote our</b> <b>lives to destroying democracy?</b> <b>Maybe.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
You mentioned</b> <b>before, like we</b> <b>were talking about Trump not</b> <b>winning the popular vote.</b> <b>So how do you feel about</b> <b>our system of voting</b> <b>in the United States versus</b> <b>a preferential voting system?</b> <b>
Well,</b> <b>it's a kind of a--</b> <b>it's a fact known among</b> <b>political scientists,</b> <b>but not the population that the</b> <b>American political system is</b> <b>so regressive that</b> <b>it would not be</b> <b>accepted by the European</b> <b>Court of Justice.</b> <b>Literally.</b> <b>Some of the East</b> <b>European states that</b> <b>have come into</b> <b>the European Union</b> <b>have proposed voting systems</b> <b>like the American one</b> <b>and they're rejected because</b> <b>they're so undemocratic.</b> <b>This goes back to the--</b> <b>If you take a look at a--</b> <b>we don't have any political</b> <b>parties in the United States.</b> <b>Remember, they do in Europe.</b> <b>You can't be a member</b> <b>of the Democratic Party.</b> <b>What you can do is go</b> <b>into the voting booth</b> <b>and put an x somewhere.</b> <b>That's your role as a citizen.</b> <b>Elite party managers</b> <b>set up the programs,</b> <b>pretty much pick the can--</b> <b>they vet the candidates.</b> <b>Only certain ones get through.</b> <b>You look at a ballot, it says,</b> <b>Republican, Democrat, nothing</b> <b>else.</b> <b>Those are your choices.</b> <b>So these are</b> <b>candidate-producing machines.</b> <b>The role of the citizen is to</b> <b>show up every couple of years</b> <b>and put in an x,</b> <b>and then go home.</b> <b>That's not democratic politics.</b> <b>But that's a deeply</b> <b>ingrained system.</b> <b>It comes out of the way the</b> <b>party system developed here</b> <b>in the 19th century</b> <b>post-Civil war, mainly.</b> <b>I mean, these are</b> <b>really serious issues.</b> <b>It makes it almost impossible</b> <b>for an independent party</b> <b>to participate.</b> <b>And those things are</b> <b>not graven in stone.</b> <b>They can all be changed.</b> <b>
Talking</b> <b>about black matters,</b> <b>there's been claims that</b> <b>this system of voting</b> <b>reflects earlier concerns</b> <b>about blacks in America.</b> <b>
It does.</b> <b>
So can you</b> <b>say something about that?</b> <b>
Yeah.</b> <b>Well, you go back to the--</b> <b>slavery lasted</b> <b>until the Civil War.</b> <b>And incidentally, it was</b> <b>the most vicious system</b> <b>of slavery in world history.</b> <b>There was never</b> <b>anything like it.</b> <b>Slavery is bad enough, but the</b> <b>American system was unique.</b> <b>Also, it's worth remembering</b> <b>that the slave system is</b> <b>the basis of our prosperity.</b> <b>The main commodity in the</b> <b>early 19th century was cotton.</b> <b>Cotton was the fuel of the 19th</b> <b>century Industrial Revolution.</b> <b>Cheap cotton led to the</b> <b>manufacturing industries</b> <b>in New England.</b> <b>The early manufacturing,</b> <b>big manufacturing industries</b> <b>in Lowell and Lawrence</b> <b>were textiles.</b> <b>It led to the development</b> <b>of the financial system.</b> <b>Credit, loans, and</b> <b>so on, was mostly</b> <b>for the cotton-based system.</b> <b>Same in England.</b> <b>That's why Liverpool</b> <b>was one of the richest</b> <b>countries in the world.</b> <b>It's where the slave</b> <b>cotton came in.</b> <b>So cotton, if you look at</b> <b>it, was the real basis right</b> <b>through the 19th century for</b> <b>commercial and industrial</b> <b>development.</b> <b>And it was based on extremely</b> <b>cheap and highly-productive</b> <b>labor.</b> <b>The productivity of labor</b> <b>was improved by technology,</b> <b>but the technology</b> <b>was very simple.</b> <b>It was the whip and the pistol.</b> <b>That's all you needed</b> <b>to improve technology.</b> <b>The whole story is unbelievably</b> <b>hideous when you look at it.</b> <b>OK, theoretically it ended with</b> <b>the emancipation declaration.</b> <b>But that was only theoretical.</b> <b>There was a 10-year</b> <b>period, the Reconstruction,</b> <b>in which federal troops were</b> <b>in the South, union troops.</b> <b>And that's one of the decades</b> <b>in which African Americans had</b> <b>a small chance to</b> <b>get into the system.</b> <b>And they did.</b> <b>They were elected.</b> <b>There were people running</b> <b>towns and legislatures.</b> <b>There was a bit of economic</b> <b>development and so on.</b> <b>It ended in 1877 with</b> <b>a North-South compact</b> <b>that essentially left the South</b> <b>free to do whatever it wanted.</b> <b>So what they did was</b> <b>re-institute slavery.</b> <b>Literally.</b> <b>In fact, the main book</b> <b>on this is called Slavery</b> <b>by Another Name by a Wall</b> <b>Street Journal bureau chief</b> <b>who studied it in some depth.</b> <b>Scholarship [inaudible] wrote</b> <b>a good, popular book about it.</b> <b>What happened was black</b> <b>life was criminalized.</b> <b>So if a black man is</b> <b>standing on a street corner,</b> <b>he can be charged with</b> <b>vagrancy and given</b> <b>a fine which he can't pay.</b> <b>So he's in jail forever.</b> <b>If a black men look,</b> <b>somebody says somebody</b> <b>looked at a white woman.</b> <b>OK, he's charged</b> <b>with attempted rape</b> <b>and he's in jail for</b> <b>the rest of his life.</b> <b>And pretty soon, most of</b> <b>the black male population</b> <b>was in jail.</b> <b>That's a wonderful</b> <b>slave labor force.</b> <b>From the point of view</b> <b>of the capitalist,</b> <b>it's much better than</b> <b>slavery because you don't</b> <b>have to sustain your workforce.</b> <b>If you have slaves, you</b> <b>have to keep them alive.</b> <b>If it's a prison population,</b> <b>the taxpayer keeps them alive.</b> <b>You just get the profit.</b> <b>And a large part</b> <b>of the second wave</b> <b>of industrialization</b> <b>in the United States</b> <b>was based on that.</b> <b>Mining, steel, so on, used</b> <b>essentially incarcerated slave</b> <b>labor.</b> <b>I mean, we were familiar</b> <b>with the chain gangs</b> <b>because you see them.</b> <b>We don't see them,</b> <b>but you did see them.</b> <b>The agricultural workers.</b> <b>But it was happening</b> <b>all over the economy.</b> <b>Well, that lasted almost</b> <b>until the Second World War.</b> <b>And now we have</b> <b>another version of it.</b> <b>The massive incarceration</b> <b>since the Reagan years</b> <b>is kind of reconstituting</b> <b>that in many ways.</b> <b>Well, all of this has</b> <b>to do with voting.</b> <b>These people don't vote.</b> <b>What's more, we have</b> <b>very harsh voting laws.</b> <b>People convicted of what's</b> <b>called a felony, which</b> <b>could be being caught with</b> <b>pot three times or something,</b> <b>they don't vote anymore.</b> <b>They're out of the system.</b> <b>This is a big part of it.</b> <b>Also, their families are</b> <b>destroyed, so they don't vote.</b> <b>They live in--</b> <b>I once took a walk through--</b> <b>with a sociologist friend</b> <b>who studies these things</b> <b>through an area of</b> <b>the Bronx, which</b> <b>I happen to know from childhood</b> <b>because a lot of my family</b> <b>lives there.</b> <b>There used to be immigrant</b> <b>communities, poor communities.</b> <b>But now, they've</b> <b>been gentrified.</b> <b>High rises, and so on.</b> <b>And I asked this woman,</b> <b>sociologist from Columbia,</b> <b>what happened to the people?</b> <b>She said, well, the men are</b> <b>in jail in upstate New York.</b> <b>The women and children</b> <b>sort of followed them,</b> <b>so they can visit</b> <b>them on Sunday.</b> <b>So they're in upstate New York.</b> <b>And now, the rich</b> <b>people live here.</b> <b>Things like that.</b> <b>That is happening</b> <b>in South Boston.</b> <b>We've seen it in my lifetime.</b> <b>It's happening all</b> <b>over the country.</b> <b>These are things that</b> <b>are going on constantly</b> <b>right in front of our eyes.</b> <b>And voting?</b> <b>Sure, it's knocking a large</b> <b>part of the population out</b> <b>of the voting list.</b> <b>By now, I'm sure you know</b> <b>the Republicans particularly</b> <b>are dedicated to trying</b> <b>to reduce the vote.</b> <b>Because the more people vote,</b> <b>the more trouble they're in.</b> <b>So you have all</b> <b>these ludicrous--</b> <b>Jeff Sessions, the</b> <b>Department of Justice,</b> <b>head of Department</b> <b>of Justice, is now</b> <b>in charge of trying to figure</b> <b>out ways to, what they call,</b> <b>prevent voter fraud.</b> <b>There's virtually</b> <b>no voter fraud.</b> <b>It's so minuscule, it's</b> <b>physically undetectable.</b> <b>But on that claim to try to</b> <b>readjust the voting so that you</b> <b>can keep the poor</b> <b>and black people out.</b> <b>The French election, if</b> <b>you noticed, was on Sunday.</b> <b>There's a good reason for that.</b> <b>Sunday.</b> <b>people are free, so</b> <b>they can go and vote.</b> <b>Here, it's on a workday,</b> <b>making it hard to vote.</b> <b>Except for the rich.</b> <b>They can do it easily.</b> <b>If you're an executive,</b> <b>leave your office.</b> <b>Or a college professor,</b> <b>you can take off.</b> <b>But for a lot of people, you</b> <b>just can't vote on a workday.</b> <b>And there are all sorts</b> <b>of other rules now.</b> <b>You have to have a</b> <b>driver's license, photo ID.</b> <b>And a lot of people</b> <b>don't have that.</b> <b>OK.</b> <b>Poor people.</b> <b>Along with the gerrymandering,</b> <b>which is by now a bad joke.</b> <b>Many devices are</b> <b>being used to try</b> <b>to prevent the limited</b> <b>democratic system</b> <b>from functioning.</b> <b>But again, all of</b> <b>this can be changed.</b> <b>Bigger gains have</b> <b>been won in the past.</b> <b>Forget blacks.</b> <b>Think about women.</b> <b>I mean, if you go back to the</b> <b>early days of the republic,</b> <b>women were not</b> <b>considered people.</b> <b>Literally, they were property.</b> <b>In our constitutional</b> <b>system, the early days,</b> <b>a woman is the property of her</b> <b>father, which is transferred</b> <b>over to her husband.</b> <b>Then she's his property.</b> <b>In fact, if you take a</b> <b>look at the early debates</b> <b>about whether women</b> <b>should be allowed to vote,</b> <b>one of the arguments</b> <b>against it was</b> <b>that it would be</b> <b>unfair to unmarried men</b> <b>because a married man would</b> <b>have two votes, his own boat</b> <b>and the property's vote.</b> <b>And this lasted for a long</b> <b>time until pretty recently.</b> <b>It wasn't until 1975--</b> <b>not that far back--</b> <b>that women had a right to</b> <b>serve on federal juries meaning</b> <b>being treated as peers.</b> <b>That's half the population.</b> <b>It's not African Americans.</b> <b>It's been a long</b> <b>way to go to achieve</b> <b>some kind of real right.</b> <b>And a lot has been achieved.</b> <b>Lot of victories.</b> <b>It's a sign of where you can go.</b> <b>A lot of things we</b> <b>take for granted</b> <b>just didn't exist years back.</b> <b>Take this class.</b> <b>When I came to MIT in 1955,</b> <b>it was white males, period.</b> <b>Well-dressed, ties and jackets,</b> <b>deferential, do your homework.</b> <b>That was MIT.</b> <b>It's not what it is now.</b> <b>And it didn't happen that way--</b> <b>change by magic.</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high-quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
Talking</b> <b>about students at MIT.</b> <b>So some of you are</b> <b>graduating soon,</b> <b>and this might be</b> <b>your last chance</b> <b>to ask Noam Chomsky a question.</b> <b>So anyone who hasn't</b> <b>spoken yet would</b> <b>like to ask a question</b> <b>before you graduate?</b> <b>
We've talked in this</b> <b>class a bit about silencing</b> <b>and how that kind of leads</b> <b>into the whole system.</b> <b>Sorry, I can speak up.</b> <b>So we've talked about silencing.</b> <b>
About?</b> <b>
Silencing.</b> <b>
Silencing.</b> <b>
Yeah.</b> <b>And how stories are hidden,</b> <b>like you talked about--</b> <b>what can we do about that?</b> <b>
About silencing?</b> <b>
What</b> <b>measurable steps can we</b> <b>take towards</b> <b>uncovering the stories</b> <b>and actually making</b> <b>change happen?</b> <b>
Well, what you</b> <b>do about silencing is speak up.</b> <b>We have a lot of</b> <b>freedom if we use it.</b> <b>In fact, a lot more</b> <b>than in the past.</b> <b>Take the last thing I mentioned.</b> <b>For women, the opportunity is</b> <b>to speak up and become active</b> <b>and have an impact is way beyond</b> <b>what it was, say, 50 years ago.</b> <b>In fact, I was talking to</b> <b>Michel about this before.</b> <b>I got a ton of email.</b> <b>And a lot of it is</b> <b>from young people.</b> <b>And one question that keeps</b> <b>coming up over and over</b> <b>again is, everything's awful.</b> <b>What can we do?</b> <b>But if you look closely</b> <b>over many years,</b> <b>I've noticed that the</b> <b>kinds of questions you get</b> <b>depend on where the</b> <b>people are coming from.</b> <b>So if people are coming from--</b> <b>say when I go down to give talks</b> <b>in immigrant slums in South</b> <b>Boston, nobody ever</b> <b>asks what they can do.</b> <b>They tell me what they're doing.</b> <b>When I go to a remote</b> <b>village in Southern Colombia</b> <b>where people are being</b> <b>murdered by paramilitaries</b> <b>and there's a chemical</b> <b>company trying to--</b> <b>a gold mining company trying</b> <b>to destroy the water supply.</b> <b>They don't ask me,</b> <b>what should they do?</b> <b>They tell me what they're doing.</b> <b>When people are</b> <b>privileged and have</b> <b>every opportunity in front of</b> <b>them, they ask, what can we do?</b> <b>And the fact is almost anything.</b> <b>There's all sorts of</b> <b>opportunities open.</b> <b>Way more than there</b> <b>were in the past</b> <b>because we do enjoy the legacy</b> <b>of people who have struggled</b> <b>in the past under much</b> <b>harsher conditions</b> <b>and have provided us with the</b> <b>opportunities we now have.</b> <b>Like, say, the opportunities</b> <b>to be a student at MIT.</b> <b>That's an opportunity.</b> <b>It gives you all</b> <b>sorts of things.</b> <b>It was closed to half the</b> <b>population 50 years ago.</b> <b>It was closed, of course, to</b> <b>minorities almost entirely.</b> <b>Well, it just illustrates what's</b> <b>gone on all over the society.</b> <b>I mean, take, say,</b> <b>the Sanders campaign,</b> <b>which I think was the most</b> <b>important aspect of the 2016</b> <b>election, by far.</b> <b>Out of that, there are groups</b> <b>developing that are really</b> <b>trying to do things.</b> <b>You can be part of that.</b> <b>Make new ones.</b> <b>All sorts of opportunities.</b> <b>And I think the</b> <b>prospects for the future</b> <b>are not too bad when you think</b> <b>of people's actual attitudes.</b> <b>And the way those</b> <b>attitudes could</b> <b>be melded into activist and</b> <b>political and other programs</b> <b>to bring about</b> <b>changes in society.</b> <b>So I don't think there's a</b> <b>real shortage of opportunity.</b> <b>I mean, there are all sorts</b> <b>of efforts to atomize people,</b> <b>keep you alone.</b> <b>That's one of the functions</b> <b>of social media incidentally.</b> <b>You sit alone with--</b> <b>whatever it is--</b> <b>your device and</b> <b>you think you have</b> <b>friends in Indonesia and so on.</b> <b>But you're really separated</b> <b>from the people around you.</b> <b>Atomization is a very important</b> <b>way of controlling people,</b> <b>because you can only do</b> <b>things if you work together.</b> <b>So when you're all</b> <b>separated, that's great.</b> <b>Then, those who really run</b> <b>things, they do get together.</b> <b>They don't sit around</b> <b>looking at iPads, and so on.</b> <b>So you can keep the population--</b> <b>In fact, that's one of the main</b> <b>things consumerism is about.</b> <b>You take a look at the history</b> <b>of the advertising industry.</b> <b>It's quite interesting.</b> <b>The huge advertising and</b> <b>public relations industry</b> <b>developed in the freest</b> <b>countries in the world,</b> <b>in Britain and the United</b> <b>States about a century ago.</b> <b>And the reason was pretty clear.</b> <b>It was often stated.</b> <b>People had won enough</b> <b>freedom, so that you</b> <b>couldn't control them by force.</b> <b>So you had to control</b> <b>them in other ways.</b> <b>And the best way</b> <b>of controlling them</b> <b>is what was called</b> <b>convincing them</b> <b>to be concerned with the</b> <b>superficial things of life,</b> <b>like consumption.</b> <b>So if you can get</b> <b>to a stage where--</b> <b>thinking of my granddaughter.</b> <b>Teenage girls on a</b> <b>Saturday afternoon.</b> <b>The best thing they can think of</b> <b>doing is walking through a mall</b> <b>and looking at things</b> <b>they can't buy.</b> <b>That's great.</b> <b>When you get to a</b> <b>society like that,</b> <b>you've got people under control.</b> <b>And there's some</b> <b>pretty obvious things</b> <b>that are never discussed,</b> <b>like you've all</b> <b>heard 10 million times about</b> <b>the wonders of free market</b> <b>societies.</b> <b>But almost no one tells</b> <b>you that the business world</b> <b>hates free market society.</b> <b>And in fact, they spend hundreds</b> <b>of billion dollars a year</b> <b>to undermine markets.</b> <b>It's called advertising.</b> <b>Anyone who has studied</b> <b>economics knows that markets--</b> <b>the marvels of markets are based</b> <b>on informed consumers making</b> <b>rational choices.</b> <b>Then, you study the mathematics</b> <b>and you make up the models,</b> <b>and so on.</b> <b>Take a look at the real world.</b> <b>Hundreds of billions of</b> <b>dollars are spent every year</b> <b>to ensure that uninformed</b> <b>people make irrational choices.</b> <b>It's called advertising.</b> <b>If you had a market society.</b> <b>If, say, Ford Motor</b> <b>Company has cars to sell,</b> <b>what they would do is say,</b> <b>here are the characteristics</b> <b>of our cars.</b> <b>Small ad on television.</b> <b>Here's the characteristics.</b> <b>Here's what Consumer</b> <b>Reports says about it.</b> <b>That's not what they do.</b> <b>What they do is try</b> <b>to create illusions,</b> <b>so you will be uninformed and</b> <b>make an irrational choice.</b> <b>In fact, while the</b> <b>economics department</b> <b>is talking about</b> <b>rational choice models</b> <b>and so on, with</b> <b>informed consumers,</b> <b>the business world is trying</b> <b>to make sure it doesn't happen.</b> <b>Of course, that's the</b> <b>thing that has the effect.</b> <b>Speaking of silencing,</b> <b>how much do you</b> <b>hear about this in</b> <b>economics courses?</b> <b>Or in the general discussion?</b> <b>It's not deep.</b> <b>It's not profound.</b> <b>It's not quantum physics.</b> <b>We all know it.</b> <b>It's right in front of our</b> <b>eyes, but it's kept silent.</b> <b>And those are the kind of things</b> <b>you have to break through with.</b> <b>
Thank</b> <b>you so much, Noam.</b> <b>That was wonderful.</b> <b>[applause]</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>
I</b> <b>think it's a nice topic</b> <b>to end this semester, because</b> <b>in a way, it brings together,</b> <b>I think, some of the</b> <b>main threads that we've</b> <b>through this semester.</b> <b>But also, in fact, the</b> <b>last point of the text,</b> <b>alliteration I can sense my</b> <b>point to wait into the future.</b> <b>So without further ado, we have</b> <b>Lorraine, Will, and Cynthia.</b> <b>
So this is</b> <b>our presentation on language</b> <b>resistance and liberation.</b> <b>Welcome to the</b> <b>last day of class.</b> <b>Congratulations (CHUCKLING)</b> <b>for getting this far.</b> <b>So as the road map, we're</b> <b>going to go through first</b> <b>"Rethinking Literacy,</b> <b>A Dialogue,"</b> <b>the piece by Freire and Macedo.</b> <b>Macedo?</b> <b>
</b> <b>Freire and Macedo.</b> <b>
Macedo?</b> <b>
Macedo.</b> <b>
Macedo.</b> <b>And then we're going to look</b> <b>at the "Literacy and Critical</b> <b>Pedagogy" piece, and then end</b> <b>with the MIT Haiti initiatives.</b> <b>
OK.</b> <b>So this is just a little bit</b> <b>of a background on the writers</b> <b>of the first two pieces.</b> <b>Both of them are educators</b> <b>in critical pedagogy.</b> <b>And one is from Brazil, and</b> <b>one is Cape Verdean-American.</b> <b>
So</b> <b>in the first piece,</b> <b>Freire mentions the premise--</b> <b>the main premises of culture,</b> <b>and I think that</b> <b>these are really</b> <b>important to keep in mind,</b> <b>because this is a really</b> <b>interesting and, I think,</b> <b>powerful way to frame culture.</b> <b>The first one is that cultural</b> <b>processes are intimately</b> <b>connected with social relations,</b> <b>especially with class relations</b> <b>and class formations,</b> <b>with sexual divisions,</b> <b>with racial structuring</b> <b>of social relations,</b> <b>and with age oppressions</b> <b>as a form of dependency.</b> <b>Got it?</b> <b>[chuckling]</b> <b>OK.</b> <b>The second one,</b> <b>culture involves power</b> <b>and helps to produce asymmetries</b> <b>in the abilities of individuals</b> <b>and social groups to define</b> <b>and realize their needs.</b> <b>And third one, culture</b> <b>is neither autonomous</b> <b>nor an externally</b> <b>determined field,</b> <b>but a site of social</b> <b>difficulties and struggles.</b> <b>Those ideas are really</b> <b>important to keep in mind,</b> <b>because this paper</b> <b>also raises the idea--</b> <b>I know it is a</b> <b>social field, which</b> <b>is kind of the combination</b> <b>of all of the beliefs, myths,</b> <b>aspirations, hopes of a part</b> <b>of a society, of a culture,</b> <b>and so it includes things</b> <b>such as why racial hierarchies</b> <b>exist, to why sexual</b> <b>divisions exist,</b> <b>and like what roles gender</b> <b>people play in a society.</b> <b>And it's really important</b> <b>to keep that in mind,</b> <b>because if you're able</b> <b>to keep that in mind,</b> <b>then you're able to think</b> <b>critically about that.</b> <b>And through the critical</b> <b>process of engaging with it,</b> <b>you're able to change</b> <b>the social field.</b> <b>And so I think that</b> <b>that's important,</b> <b>because even though</b> <b>this paper raises</b> <b>a lot of really powerful</b> <b>ideas, it's still not perfect.</b> <b>So one of the</b> <b>quotes of this paper</b> <b>is, "Literacy's oral</b> <b>dimension is important,</b> <b>even if it takes place</b> <b>in a culture, like that</b> <b>of the United States, whose</b> <b>memory is predominantly</b> <b>written, not oral,</b> <b>like that of Africa.
"</b> <b>Africa is not</b> <b>(LAUGHING) country.</b> <b>And so something like that is</b> <b>also part of the social field,</b> <b>and would make it</b> <b>into this paper.</b> <b>Because even though Freire and</b> <b>Macedo are thinking critically</b> <b>about this, they</b> <b>are not engaging</b> <b>with the full reality of</b> <b>cultural objectivity and being</b> <b>subjects within that.</b> <b>
Yeah.</b> <b>So in the next</b> <b>article, Paulo Freire</b> <b>talks about the use of language</b> <b>as a form of resistance.</b> <b>And he first starts</b> <b>talking about how</b> <b>the current traditional methods</b> <b>in literacy how they tend</b> <b>to ignore the</b> <b>importance of language</b> <b>in a person's education.</b> <b>And he bases a lot of his</b> <b>theories on his experiences.</b> <b>And from the second quote he</b> <b>says that, "The sad reality</b> <b>is that while</b> <b>education in Portuguese</b> <b>provides access to positions</b> <b>of political and economic power</b> <b>for the higher echelon</b> <b>of African society,</b> <b>it screens out the</b> <b>majority of the masses, who</b> <b>fail to learn</b> <b>Portuguese well enough</b> <b>to acquire the necessary</b> <b>literacy skills for social,</b> <b>economic, and</b> <b>political advancement.
"</b> <b>And you definitely-- from the</b> <b>way that he mentions this,</b> <b>you could definitely see</b> <b>a lot of like correlations</b> <b>between what's going on</b> <b>around the world and currently</b> <b>in the United States.</b> <b>And you could definitely</b> <b>see that the 1%</b> <b>usually have the most</b> <b>control over things,</b> <b>and that they usually tends to</b> <b>drive how policies are made.</b> <b>But the fact is that</b> <b>when a person is taught</b> <b>in their native language, they</b> <b>tend to learn things better</b> <b>and comprehend things faster.</b> <b>And can we go to the next slide?</b> <b>There's usually backlash when</b> <b>it comes to different languages</b> <b>taught in schools.</b> <b>For example, in</b> <b>the US, people say</b> <b>that students should learn</b> <b>how to speak standard American</b> <b>English.</b> <b>But when it comes to providing</b> <b>these sort of opportunities</b> <b>to minority groups of,</b> <b>I guess for instance,</b> <b>like students of</b> <b>color, there is usually</b> <b>some racial tensions</b> <b>that develop</b> <b>that make minority</b> <b>students feel inferior.</b> <b>For example, there was</b> <b>an article mentioned</b> <b>of like recently, like</b> <b>March of 2017, that</b> <b>talked about a robotics</b> <b>team that recently had won</b> <b>the challenge, and the fact</b> <b>that after they had won</b> <b>the students-- or I</b> <b>guess people in the crowd</b> <b>started screaming</b> <b>at them and telling</b> <b>them to go back to Mexico.</b> <b>And so this form of like</b> <b>demeaning commentary</b> <b>tends to drive the dropout</b> <b>rates of minority groups,</b> <b>and all of this tends to</b> <b>contribute to like the prison--</b> <b>sorry, the school</b> <b>to prison pipeline.</b> <b>As you can see in</b> <b>the infographic,</b> <b>it gives a lot of</b> <b>percentages talking</b> <b>about this sort of issue.</b> <b>You can see in the</b> <b>bottom, it says</b> <b>that 68% of all males in</b> <b>state and federal prison they</b> <b>do not have a high</b> <b>school diploma.</b> <b>And this tends to be--</b> <b>gives statistics on students.</b> <b>And all of this tends</b> <b>to stem from the fact</b> <b>that minority</b> <b>students do not have</b> <b>the same access to education</b> <b>as the elite part of society.</b> <b>And from personal experience, I</b> <b>can definitely relate to this.</b> <b>I went to a school about like an</b> <b>hour and a half from my house,</b> <b>and I definitely did have a lot</b> <b>of good role models growing up.</b> <b>But were definitely students</b> <b>around my neighborhood</b> <b>that didn't have the same</b> <b>access to the same opportunities</b> <b>that I did.</b> <b>Some of them did drop</b> <b>out to start working.</b> <b>Others were influenced by</b> <b>gangs around my neighborhood.</b> <b>And you could definitely--</b> <b>things are definitely</b> <b>getting better,</b> <b>but things could</b> <b>definitely be improved</b> <b>from an emancipatory</b> <b>literacy approach, which</b> <b>is what Paulo Freire mentions.</b> <b>Yeah.</b> <b>So in emancipatory</b> <b>literacy, this</b> <b>involves using the</b> <b>language of the people,</b> <b>rather than the elites.</b> <b>It teaches students to</b> <b>accept their experiences</b> <b>and their culture.</b> <b>And this moves from the</b> <b>world of objectivity</b> <b>to the world of subjectivity,</b> <b>where individuals learn</b> <b>to accept themselves</b> <b>for who they are,</b> <b>rather than feel like</b> <b>they're a victim of society.</b> <b>And Paulo Freire does</b> <b>mention that achieving</b> <b>emancipatory literacy is</b> <b>hard, but it's not impossible.</b> <b>And now I'll-- and so Lorraine</b> <b>wants to talk about a few</b> <b>examples of how this</b> <b>is being used today.</b> <b>So considering specifically</b> <b>the case of Haiti--</b> <b>which Michel works a lot on,</b> <b>the MIT Haiti initiative--</b> <b>virtually all of the schooling</b> <b>in both K through 12 education</b> <b>and then in higher education</b> <b>is done in French, and not</b> <b>in Kreyol.</b> <b>But the problem is that most</b> <b>people only speak Kreyol</b> <b>fluently, so like 95% or</b> <b>so, and very few people</b> <b>speak French fluently, at all.</b> <b>So when you go through school,</b> <b>and everything is in a language</b> <b>that you don't</b> <b>understand, how are you</b> <b>supposed to be able to actually</b> <b>get any of the education,</b> <b>get any of this</b> <b>social engagement,</b> <b>especially if</b> <b>children are punished</b> <b>for speaking Kreyol in school.</b> <b>And they can't interact</b> <b>with each other,</b> <b>can't ask questions,</b> <b>can't gather</b> <b>any of the information</b> <b>that's coming at them.</b> <b>So it's based on this false</b> <b>belief and really outdated</b> <b>belief about the</b> <b>hierarchy of languages,</b> <b>about which languages are</b> <b>created or are younger,</b> <b>or anything like that, and that</b> <b>Kreyol isn't developed enough</b> <b>to be able to be used for</b> <b>technical language for STEM</b> <b>education to have anything</b> <b>translated into Kreyol.</b> <b>And part of that is</b> <b>saying that like Kreyol</b> <b>isn't widespread enough,</b> <b>even though Kreyol</b> <b>has almost 10 million speakers.</b> <b>Finnish has only about</b> <b>5.5 million speakers,</b> <b>and yet in Finland all of that</b> <b>education is done in Finnish.</b> <b>And people have-- studies have</b> <b>found that, in Finland, kids</b> <b>are much better equipped</b> <b>to be able to learn</b> <b>a second language, and to</b> <b>go into higher education</b> <b>and gather all of these</b> <b>like technical studies,</b> <b>because all of their</b> <b>education is done in Finnish.</b> <b>And over at IAP, I was doing</b> <b>MISTI GTL in the Catalonia</b> <b>region, in Spain, and there</b> <b>Catalan also has a little</b> <b>under 10 million speakers.</b> <b>But in Catalonia</b> <b>all of the education</b> <b>is in Catalan, which</b> <b>made it difficult for me</b> <b>as like a "Spanish</b> <b>speaker"-- like not even</b> <b>fluent, but trying to</b> <b>speak Spanish speaker--</b> <b>to be able to</b> <b>teach the students.</b> <b>But they had such</b> <b>a better experience</b> <b>with all of their</b> <b>textbooks in Catalan</b> <b>all of their computer software.</b> <b>So I was trying to teach</b> <b>them Inkscape and Gimp,</b> <b>but everything was is</b> <b>(CHUCKLING) Catalan,</b> <b>and I just couldn't understand</b> <b>like even how to open a window.</b> <b>So they taught me</b> <b>a lot about this.</b> <b>And they-- all of</b> <b>them speak Catalan,</b> <b>and Spanish, and</b> <b>English, and then they</b> <b>can like learn German or</b> <b>French on the side, as well.</b> <b>So all of them, because</b> <b>their initial education</b> <b>K through 12 and then</b> <b>higher education,</b> <b>if they want it to be,</b> <b>is in Catalan, then</b> <b>they can learn all</b> <b>of these other things</b> <b>just as well as we do like</b> <b>in English, which is mostly</b> <b>our one and kind of only</b> <b>language in the United States.</b> <b>Yeah.</b> <b>So that was my experience.</b> <b>And what-- a school that is like</b> <b>a really good model in Haiti</b> <b>is the LKM School.</b> <b>I didn't know how to pronounce</b> <b>it in Kreyol, so it's LKM.</b> <b>It's a K through 10</b> <b>school in a rural part</b> <b>on an island of Haiti,</b> <b>and their education</b> <b>is starting to be in this</b> <b>model of all Kreyol education,</b> <b>trying to teach all</b> <b>of these subjects, all</b> <b>of the technical</b> <b>terms in Kreyol.</b> <b>And the students are</b> <b>learning much better</b> <b>than the rest of the students</b> <b>in Haiti, who's schooling</b> <b>is French and who only 10%</b> <b>graduate from high school.</b> <b>So the linguistic</b> <b>apartheid part of this</b> <b>is that, when you teach somebody</b> <b>in a language they don't</b> <b>understand, you</b> <b>prevent them from being</b> <b>able to move into different</b> <b>socioeconomic classes</b> <b>to attain the education</b> <b>and the success</b> <b>that they should be able to.</b> <b>And this all leads to</b> <b>the MIT Haiti initiative</b> <b>that Michel started</b> <b>here between MIT and all</b> <b>the Haitian education systems.</b> <b>So a couple of their main points</b> <b>are to teach educators in Haiti</b> <b>that teaching in Kreyol</b> <b>is perfectly fine,</b> <b>that it is actually</b> <b>better for the students,</b> <b>that it gives them</b> <b>more opportunities,</b> <b>and that STEM and</b> <b>technical subjects</b> <b>can be taught in Kreyol.</b> <b>So a lot of the</b> <b>worries or concerns</b> <b>are that like Kreyol doesn't</b> <b>have words for these languages.</b> <b>We didn't have</b> <b>words for a computer</b> <b>either up until when</b> <b>they were created.</b> <b>A lot of other languages borrow</b> <b>words from other languages.</b> <b>So I learned Japanese,</b> <b>and in Japanese there's</b> <b>this lovely thing called</b> <b>[speaking japanese]</b> <b>where you just grab words</b> <b>from other languages.</b> <b>So if you don't know</b> <b>how to pronounce it,</b> <b>you kind of like make</b> <b>it sound like it,</b> <b>and oftentimes, it is</b> <b>what has been borrowed.</b> <b>So for example, bread</b> <b>is pan, from German.</b> <b>And all of these</b> <b>other languages have</b> <b>words borrowed from</b> <b>other languages,</b> <b>and that's what</b> <b>globalization does.</b> <b>So the same thing should be able</b> <b>to happen with Kreyol and STEM</b> <b>learning.</b> <b>And the really good</b> <b>results of this</b> <b>are that they affirm students'</b> <b>cultures and backgrounds,</b> <b>and their own upbringings, and</b> <b>how the society actually works,</b> <b>as opposed to a colonial</b> <b>oppressive society.</b> <b>And it equalizes students</b> <b>being able to have</b> <b>more social mobility, and</b> <b>to attain what they want</b> <b>and what they should be able to.</b> <b>
So</b> <b>to kind of tie this</b> <b>back to what we can do to stay</b> <b>woke, with regards to literacy,</b> <b>I think it's important</b> <b>to remember that,</b> <b>"language is a mediating</b> <b>force of knowledge,</b> <b>but it is also</b> <b>knowledge itself.
"</b> <b>And so one approach</b> <b>is for groups</b> <b>to kind of come together</b> <b>and create a language that</b> <b>describes their experience</b> <b>amongst themselves,</b> <b>and then to share that language</b> <b>and to share that experience</b> <b>in their own language.</b> <b>And I think that that is one way</b> <b>that we can start breaking away</b> <b>from the kind of putting</b> <b>standard English on a pedestal,</b> <b>and can also kind of</b> <b>validate other discourses,</b> <b>and other ways of being,</b> <b>and of other just languages</b> <b>that kind of already exist in</b> <b>the US and across the world.</b> <b>Questions.</b> <b>How have our</b> <b>understandings of literacy</b> <b>evolved in response</b> <b>to the internet,</b> <b>unprecedented</b> <b>global integration,</b> <b>and increased pressure</b> <b>to preserve human rights?</b> <b>For example, what does</b> <b>it mean that schools</b> <b>repress the development</b> <b>of subjectivity,</b> <b>but do largely ensure literacy?</b> <b>How can modern</b> <b>notions of literacy</b> <b>evolve to address the phenomena,</b> <b>such as the proliferation</b> <b>of fake news?</b> <b>So if we're moving past literacy</b> <b>as just the idea of being</b> <b>able to read, and</b> <b>we are entering</b> <b>this new kind of this new</b> <b>wave being defiantly pushed</b> <b>by the internet and the</b> <b>influence of technology</b> <b>on the way we live,</b> <b>literacy is coming</b> <b>to be a more critical</b> <b>approach to life in the sense</b> <b>that fake news is</b> <b>a thing, you know,</b> <b>and it's actually quite hard to</b> <b>distinguish what news is really</b> <b>what news is fake.</b> <b>And the ability to do that,</b> <b>having that critical lens,</b> <b>is maybe the next</b> <b>evolution of literacy.</b> <b>So any ideas on that, I</b> <b>guess, is the question.</b> <b>
So I'm just gonna</b> <b>answer the first part of this,</b> <b>because I'm not sure if I have</b> <b>an answer for the next two.</b> <b>But something that</b> <b>I think is kind of</b> <b>hilarious about like the</b> <b>integration of the internet</b> <b>is that I think it's</b> <b>simultaneously making us more</b> <b>literate and less literate.</b> <b>So like-- or like pressuring</b> <b>you to be more literate.</b> <b>So "pressuring" you</b> <b>to be more literate</b> <b>is like, now there's a bunch</b> <b>of sources that you can read,</b> <b>and like now you can</b> <b>be more educated,</b> <b>it's all open to the public</b> <b>if you have a computer.</b> <b>And then there's also</b> <b>like the pressure</b> <b>to speak English properly,</b> <b>based off either, you know,</b> <b>like there's one viral video,</b> <b>this old grandma screaming</b> <b>at this Hispanic woman</b> <b>in IHOP because she</b> <b>was paying in Spanish,</b> <b>or something like that.</b> <b>So like it puts pressure on</b> <b>you to speak English correctly,</b> <b>and then meanwhile, because</b> <b>of like things like Facebook</b> <b>and like any social</b> <b>media, people</b> <b>like don't speak English</b> <b>correctly, like typing,</b> <b>you know,</b> <b>abbreviations and stuff</b> <b>that's acceptable in text,</b> <b>but like meanwhile we're</b> <b>saying you should</b> <b>speak English properly.</b> <b>So it's kind of a</b> <b>weird dichotomy.</b> <b>
I'm</b> <b>interested in the age</b> <b>implications of what you</b> <b>just said, kind of aligning--</b> <b>kind of a critique towards</b> <b>the use of different varieties</b> <b>of English with older</b> <b>generations and the use</b> <b>of nonstandard</b> <b>varieties of English</b> <b>with younger generations.</b> <b>I'm interested in that.</b> <b>
Yeah, I think--</b> <b>I mean, I guess like--</b> <b>so like there's a grandma that</b> <b>I know that's on Facebook,</b> <b>and like she makes sure</b> <b>to type everything out,</b> <b>including punctuation.</b> <b>And like her Facebook</b> <b>posts are longer</b> <b>and they're more like here's</b> <b>a card, that type of thing.</b> <b>Whereas like my Facebook,</b> <b>A, I'm just sharing things,</b> <b>but B, if I write</b> <b>anything, it's probably</b> <b>not good, proper English.</b> <b>So I think it does</b> <b>depend on your age group,</b> <b>but also like going back to the</b> <b>original point of, you know,</b> <b>that grandma screaming</b> <b>at the Hispanic woman</b> <b>in IHOP, like it's not just</b> <b>like an older woman screaming</b> <b>because, you know,</b> <b>she's more literate.</b> <b>Like there's also like</b> <b>kids who think like, oh,</b> <b>you have to speak</b> <b>English in here.</b> <b>So like there is still that</b> <b>weirdness between like how</b> <b>older people act on Facebook,</b> <b>and stuff like that.</b> <b>But I still think like the point</b> <b>of, you know, pressuring people</b> <b>is regardless of age.</b> <b>
I think</b> <b>because we're gonna</b> <b>be talking about language,</b> <b>which is dialect in this class,</b> <b>I would kind of risk</b> <b>saying that we're</b> <b>kind of creating a new language</b> <b>because of the internet</b> <b>with like all the</b> <b>abbreviations and things.</b> <b>And there's the way--</b> <b>the push for speed and</b> <b>everything like that.</b> <b>So I think the reason</b> <b>that older generations</b> <b>don't use it as much is</b> <b>because they're not used to it,</b> <b>and they're in a different</b> <b>frame of reference</b> <b>and they just haven't learned</b> <b>this new dialect that we're</b> <b>starting to use.</b> <b>So I think that it's a</b> <b>good thing, in a sense,</b> <b>but a lot of people who</b> <b>want that standard American</b> <b>English are going to be</b> <b>pushing us to go back to that,</b> <b>as opposed to using</b> <b>the internet language.</b> <b>But going as far as our</b> <b>changes in literacy because</b> <b>of the internet, like I didn't</b> <b>know if the ending was satire,</b> <b>at first.</b> <b>Well, I just didn't know</b> <b>that the ending existed.</b> <b>And then when I saw that, I</b> <b>realized that that was satire.</b> <b>I think that nuance</b> <b>like that is something</b> <b>that you miss especially</b> <b>with the internet,</b> <b>because you can't really express</b> <b>sarcasm very clearly over just</b> <b>plain text.</b> <b>And I think that's something</b> <b>that we're like struggling</b> <b>with, because of the internet.</b> <b>
But I</b> <b>think that you said--</b> <b>you talked about The Onion.</b> <b>I think that shows like Stephen</b> <b>Colbert or the Colbert Report,</b> <b>and shows like</b> <b>that, show that we</b> <b>have been dealing with that</b> <b>sort of satire language issue.</b> <b>Because for a long</b> <b>time, people rushed</b> <b>[inaudible] didn't realize</b> <b>that it was satire.</b> <b>So I think that we've</b> <b>done this before,</b> <b>and it's sort of</b> <b>a different cycle.</b> <b>Now it's on the net</b> <b>instead of being TV.</b> <b>
I wanted to bounce</b> <b>off what you originally said.</b> <b>Like with like it's</b> <b>not a bad thing</b> <b>that we're checking</b> <b>the language,</b> <b>I agree with thi It's</b> <b>kind of like [inaudible],,</b> <b>as you mentioned.</b> <b>Do you think like there</b> <b>are certain rules,</b> <b>like the way we type,</b> <b>and like is there</b> <b>like a way to be literate like</b> <b>with our new abbreviations?</b> <b>
I think</b> <b>there definitely is.</b> <b>Like for instance, punctuating</b> <b>sentences with like LOL or ha</b> <b>ha.</b> <b>Like the beginning of the</b> <b>end, where you put that.</b> <b>Using emojis to get</b> <b>more emotion in it.</b> <b>Like there's definitely</b> <b>like a science to it</b> <b>that we kind of fell into, but</b> <b>didn't realize was [inaudible]..</b> <b>
Yeah.</b> <b>OK, cool.</b> <b>I understand what you're saying.</b> <b>
My question</b> <b>is, so knowing now</b> <b>that people grow up in</b> <b>the type of literacy,</b> <b>whether it's traditional</b> <b>or emancipatory,</b> <b>what are the pros and cons</b> <b>of your type of literacy?</b> <b>And explain why we should</b> <b>move towards or away</b> <b>from an emancipatory literacy.</b> <b>
I think you all</b> <b>gave some really great points</b> <b>towards moving to</b> <b>emancipatory literacy.</b> <b>I mean, if it's helping people</b> <b>learn, I don't see a down fall,</b> <b>especially because it's kind</b> <b>of broadening their world view.</b> <b>A couple of presentations</b> <b>ago, there were other students</b> <b>who were playing</b> <b>jeopardy with AAE,</b> <b>and that was kind of like</b> <b>broadening their worldview.</b> <b>Like even if you</b> <b>went to the school</b> <b>and didn't speak AAE you were</b> <b>learning about a new language</b> <b>that you weren't familiar with.</b> <b>And so they were able to do this</b> <b>through the emancipatory sort</b> <b>literacy.</b> <b>
What's AAE?</b> <b>
Oh,</b> <b>African-American English.</b> <b>Sorry.</b> <b>Not AAPE.</b> <b>
So kind of just</b> <b>playing devil's advocate</b> <b>a little bit, I think--</b> <b>I guess one of the dangers</b> <b>of emancipatory literacy</b> <b>might be kind of this</b> <b>non-standardization</b> <b>like with issues like Common</b> <b>Core and things like that.</b> <b>We kind of have an</b> <b>educational system</b> <b>that's set up so that</b> <b>everybody's kind of put</b> <b>on the same level,</b> <b>to some extent.</b> <b>Like everybody has that same</b> <b>kind of baseline education.</b> <b>But like with</b> <b>emancipatory literacy,</b> <b>I think you're going to find</b> <b>educational programs are more</b> <b>geared towards a specific</b> <b>like cultural norms</b> <b>and things of that nature</b> <b>related to whatever</b> <b>population it targets.</b> <b>So I wonder if that</b> <b>could potentially</b> <b>produce a system in</b> <b>which certain students,</b> <b>even though they're</b> <b>receiving an education that</b> <b>is geared towards them,</b> <b>it's objectively not</b> <b>that same quality</b> <b>of student who are</b> <b>receiving more of a traditional</b> <b>Thwere's the approach.</b> <b>[inaudible]</b> <b>
I</b> <b>want to push back</b> <b>on that just a little</b> <b>bit, because I think</b> <b>that the standardized</b> <b>approach to education</b> <b>is the approach that's</b> <b>been taken with, you know,</b> <b>teach for a test</b> <b>approach, and it's failed.</b> <b>It's been failing</b> <b>for a while now,</b> <b>and I think that a lot of people</b> <b>in education policy and even</b> <b>just teachers are</b> <b>recognizing that now.</b> <b>And I think that education--</b> <b>emancipatory literacy gives</b> <b>us an interesting lens</b> <b>to think about how to educate</b> <b>people so that they can learn.</b> <b>And I think maybe</b> <b>another place you</b> <b>can start kind of</b> <b>investing energy</b> <b>is thinking of ways in</b> <b>society that we can value</b> <b>those different ways of</b> <b>knowing and being and learning,</b> <b>so that it's not just,</b> <b>you went to college,</b> <b>so you deserve to be paid a</b> <b>lot of money kind of thing,</b> <b>and it's a, oh you can</b> <b>build a table from scratch.</b> <b>How do you get paid to do that?</b> <b>
Where do</b> <b>you think MIT falls</b> <b>on the spectrum of like</b> <b>emancipatory literacy?</b> <b>[laughing]</b> <b>
I think MIT</b> <b>is way far on the tradition.</b> <b>MIT is the Alt right of</b> <b>traditional literacy, I think.</b> <b>[laughing]</b> <b>I just think that there are--</b> <b>I think that the</b> <b>immense pressure that</b> <b>falls on students, just on</b> <b>students physical bodies,</b> <b>is definitely a symptom</b> <b>of a larger problem.</b> <b>And for me, the fact</b> <b>that that symptom is not</b> <b>enough of an impetus to</b> <b>change, is heartbreaking.</b> <b>Because I think that--</b> <b>I didn't come to</b> <b>college expecting</b> <b>it to be the best</b> <b>four years of my life,</b> <b>but I didn't come here also</b> <b>expecting to just suffer.</b> <b>[laughing]</b> <b>And so I think that</b> <b>there's definitely</b> <b>got to be a way</b> <b>where I can bloom,</b> <b>can reach my best potential.</b> <b>And I love learning.</b> <b>I love-- like I love</b> <b>school, you know,</b> <b>and the fact that I have</b> <b>just kept running up</b> <b>against the institution, I</b> <b>think, as a creative person,</b> <b>has made me kind of step</b> <b>back and think again</b> <b>about emancipatory literacy</b> <b>and how I can do that after.</b> <b>
So I have a</b> <b>question for you, Cynthia.</b> <b>So given what you</b> <b>just said, how would</b> <b>you-- so if you were</b> <b>to be MIT's president,</b> <b>what would you change?</b> <b>What-- how would you make</b> <b>it better more emancipatory?</b> <b>Keeping in mind that</b> <b>there are certain goals</b> <b>that you have to reach</b> <b>in terms of, say,</b> <b>chemical engineering,</b> <b>if you're a chemist,</b> <b>how would you do that?</b> <b>How would you advise</b> <b>administration be better</b> <b>on that score?</b> <b>
I think very</b> <b>first thing I would do</b> <b>is have people who are</b> <b>more interested in research</b> <b>not be professors.</b> <b>So the ones who are</b> <b>interested in research</b> <b>should be doing</b> <b>research full time,</b> <b>and the people who are</b> <b>interested in teaching</b> <b>should teach.</b> <b>Because I think that</b> <b>that gives like--</b> <b>that gives people</b> <b>more impetus just</b> <b>to connect with their students</b> <b>and actually get to know them,</b> <b>and I think in that</b> <b>kind of environment,</b> <b>you would foster like a</b> <b>safer, healthier environment</b> <b>for learning.</b> <b>Whereas if you have</b> <b>a professor who</b> <b>is more interested in</b> <b>winning the Nobel Prize,</b> <b>like what are you-- what do</b> <b>you do coming up against that?</b> <b>I also would have MIT think</b> <b>very, very hard about where</b> <b>the funding is coming from?</b> <b>What it's going to?</b> <b>And how that kind of fits</b> <b>in the global system, where</b> <b>MIT as an institution fits</b> <b>into the global system,</b> <b>as kind of not only</b> <b>an institution that</b> <b>does propel you to a</b> <b>higher class, but kind of</b> <b>does that at the expense</b> <b>of a lot of other people</b> <b>and systems across the globe</b> <b>even if it's not directly</b> <b>in the area.</b> <b>So I think that being more</b> <b>critical and intentional</b> <b>about our values, and</b> <b>also maybe adding soul</b> <b>to the end of that--</b> <b>end of the that motto.</b> <b>But--</b> <b>
OK, OK.</b> <b>
Yeah.</b> <b>
Attention</b> <b>to the third question.</b> <b>Thank you.</b> <b>
The last question is,</b> <b>from your personal experiences,</b> <b>or what you've</b> <b>seen or read about,</b> <b>how do you think we can</b> <b>improve students' experiences</b> <b>and confidence through</b> <b>intentional language</b> <b>use in schools?</b> <b>Like what mottoes do you think</b> <b>are working, or would work?</b> <b>
Dana?</b> <b>
Well there's a model,</b> <b>the only one of it's kind,</b> <b>it's in Canada, it's a Mohawk</b> <b>language immersion school</b> <b>that's K through six, but it</b> <b>struggles to secure funding</b> <b>because it also doesn't</b> <b>follow a standardized Canadian</b> <b>curriculum.</b> <b>In fact the community has sort</b> <b>of decolonized the curriculum,</b> <b>and they teach history, for</b> <b>instance not Canadian history,</b> <b>they teach Mohawk</b> <b>history, and it's</b> <b>been really affirming</b> <b>for children</b> <b>in the community,</b> <b>Especially a community that</b> <b>has its own struggles</b> <b>with language preservation</b> <b>and revitalization, and</b> <b>passing on the language</b> <b>to continued generations.</b> <b>Especially after a history of</b> <b>residential schooling that--</b> <b>main aim was a cultural</b> <b>and linguistic genocide</b> <b>of these people.</b> <b>So this is a great model,</b> <b>but it's funded entirely</b> <b>by community fundraisers</b> <b>and every year</b> <b>struggles with staying open.</b> <b>And it's never certain</b> <b>that it will be there</b> <b>in the next school year.</b> <b>But it seems to have</b> <b>had a really, really</b> <b>a positive impact on the</b> <b>Mohawk community in Canada.</b> <b>And so programs like that I</b> <b>think need to now be funded,</b> <b>and that involves challenging</b> <b>our federal and local</b> <b>requirements for funding, and to</b> <b>do that we need to interrogate</b> <b>what value there is in</b> <b>the curriculum we have</b> <b>standardized, and why that</b> <b>is considered objectively</b> <b>the standard that should</b> <b>impart the best learning</b> <b>experience upon students?</b> <b>
Go ahead Colin.</b> <b>
So bouncing</b> <b>of that it seems</b> <b>we get to [inaudible]</b> <b>thing where it [inaudible]</b> <b>is inculcated in this</b> <b>standard curriculum.</b> <b>It goes into society</b> <b>and then perpetuates</b> <b>the standard curriculum</b> <b>back on the Indian students.</b> <b>So I'm curious about</b> <b>if anyone has thoughts</b> <b>how you could make knowledge</b> <b>about-- because one</b> <b>of the main hurdles is people</b> <b>instituting, or interrogating</b> <b>the standard curriculum</b> <b>is just strictly about how</b> <b>the story of America's history</b> <b>has been with regarding</b> <b>the Native American</b> <b>genocide, slavery,</b> <b>and, for lack of a</b> <b>better word, mean</b> <b>that it is manifest destiny.</b> <b>How do you make people</b> <b>aware of that on a level</b> <b>that they can see those</b> <b>things playing out today,</b> <b>then work to interrogate</b> <b>that curriculum?</b> <b>Especially if they are</b> <b>coming from a background</b> <b>where race isn't talked about</b> <b>in this family because it's</b> <b>an uncomfortable</b> <b>topic, or if they just</b> <b>haven't heard about these things</b> <b>necessarily because of school.</b> <b>
For me, I feel</b> <b>like it's more about where</b> <b>do you learn to interrogate</b> <b>your own beliefs,</b> <b>and I feel like</b> <b>that's hard to find.</b> <b>And I came to-- this class was</b> <b>a huge part of that for me.</b> <b>And I don't know if that</b> <b>because I started out,</b> <b>or if I just happened</b> <b>to stumble upon a place</b> <b>where I learned to</b> <b>interrogate my own beliefs.</b> <b>But I think this isn't a</b> <b>point to make for themselves,</b> <b>but I don't really have an</b> <b>answer for how we get triple</b> <b>the value in evaluating in</b> <b>interrogating your own beliefs,</b> <b>or how much more in [inaudible]</b> <b>learn to interrogate your own</b> <b>beliefs before attacking</b> <b>someone else's.</b> <b>
Yeah, bouncing off</b> <b>of I think a common theme</b> <b>that we come up in</b> <b>this class is how</b> <b>do we have conversations</b> <b>with people who don't</b> <b>want to these conversations?</b> <b>And I think the only way</b> <b>for them to figure it out is</b> <b>for them to</b> <b>interrogate themselves,</b> <b>but most of those people won't.</b> <b>So finding ways to make</b> <b>them interrogate themselves,</b> <b>I think is gonna be a common</b> <b>problem throughout history.</b> <b>And unfortunately I</b> <b>don't know I, personally,</b> <b>have a great solution for that.</b> <b>But I think more</b> <b>opportunities like this--</b> <b>I mean obviously you mentioned</b> <b>this before, the people who</b> <b>are taking this class,</b> <b>were not really diversified</b> <b>at all right?</b> <b>We all sought this</b> <b>class out mostly,</b> <b>so I don't know how</b> <b>you can get someone</b> <b>who doesn't want</b> <b>interrogate themselves</b> <b>in to a situation like this.</b> <b>So maybe mandating</b> <b>something like this</b> <b>where you have to</b> <b>go a black history</b> <b>class, or a Native</b> <b>American history class,</b> <b>or whatever might force people</b> <b>to start talking about it.</b> <b>
Yeah, I almost</b> <b>think it might be better</b> <b>to not have a</b> <b>separate class where</b> <b>this is your</b> <b>well-roundedness class,</b> <b>this is where we're going</b> <b>to learn about the others.</b> <b>But to integrate</b> <b>what you would learn</b> <b>in a specific</b> <b>black history class</b> <b>into the general curriculum.</b> <b>Which of course is</b> <b>harder, but I do</b> <b>think would require the</b> <b>same amount of conversations</b> <b>and connections,</b> <b>and negotiations</b> <b>as making a requirement</b> <b>of a separate class.</b> <b>But I think that a</b> <b>better approach that</b> <b>might be more palatable and</b> <b>have more positive effects</b> <b>would be just straight up</b> <b>integration, decolonizing</b> <b>the current curriculum.</b> <b>
I totally-</b> <b>
Yeah, go ahead.</b> <b>
I was going to say</b> <b>I totally agree with Dana,</b> <b>I think that--</b> <b>I'm losing my train of thought--</b> <b>I totally agree</b> <b>with Dana because I</b> <b>think that the history</b> <b>will speak for itself.</b> <b>I think that history is--</b> <b>if you know the full history of</b> <b>the United States for example--</b> <b>it's pretty terrible</b> <b>and I think that you</b> <b>do need to know</b> <b>the whole history,</b> <b>and also need to get</b> <b>the alternative sides</b> <b>and have those be integrated</b> <b>into the curriculum.</b> <b>I also was interested in</b> <b>thinking about entry points</b> <b>into these</b> <b>conversations, and ways</b> <b>that our humor can</b> <b>function as entry</b> <b>points in these conversations.</b> <b>I've been having</b> <b>this conversation</b> <b>with a friend of mine of</b> <b>how humor, for example,</b> <b>gives us a way to</b> <b>kind of express</b> <b>some of these things</b> <b>that are more taboo,</b> <b>and kind of laugh them off,</b> <b>and kind of integrate them,</b> <b>as opposed to</b> <b>everyone shunning them</b> <b>and people feeling bottled.</b> <b>So I am interested</b> <b>about humor and art</b> <b>specifically as entry point</b> <b>into these conversations.</b> <b>
[inaudible].</b> <b>So Dana, I think</b> <b>what you say in a way</b> <b>answers the concern that</b> <b>Shope expressed right?</b> <b>Because Shope put</b> <b>it as dual choice.</b> <b>That you either do this</b> <b>or you do that, right?</b> <b>So Dana is right, right Shope,?</b> <b>One can imagine</b> <b>having in the class--</b> <b>the principle math</b> <b>class, chemistry class--</b> <b>paying attention</b> <b>to those issues.</b> <b>In fact there room</b> <b>for that, even</b> <b>if you take a simple</b> <b>photography class</b> <b>one can even think about</b> <b>what kind of choices</b> <b>go into making your camera?</b> <b>And actually, I have friends</b> <b>who are photographers</b> <b>and they explain to me that</b> <b>actually most cameras were not</b> <b>geared to take pictures</b> <b>of black people.</b> <b>So even as a white</b> <b>photographer he</b> <b>finds that when he takes a</b> <b>picture with his regular Cannon</b> <b>of a white person it comes</b> <b>much better than when</b> <b>takes it of a black person.</b> <b>Why?</b> <b>Because the lenses are</b> <b>designed with a normal gaze</b> <b>of the white subject right?</b> <b>So he had to tweak his</b> <b>camera to make it take</b> <b>good pictures of black people.</b> <b>So even there, even</b> <b>in physics there's</b> <b>a way to interrogate</b> <b>the curriculum.</b> <b>Why selections are made</b> <b>with the lens for example?</b> <b>So in a way it could</b> <b>both, you could</b> <b>both have the necessary interest</b> <b>in human beings the quote</b> <b>unquote more</b> <b>"traditional curriculum".</b> <b>And going back to</b> <b>the other point,</b> <b>can you imagine also</b> <b>in history, in biology,</b> <b>biology could include</b> <b>text like I mentioned</b> <b>earlier, The Mismeasure</b> <b>of Man if you wish,</b> <b>goes into lots of integral</b> <b>issues about biology.</b> <b>About how we make--</b> <b>heads were measured</b> <b>to show that humans</b> <b>are as intelligent.</b> <b>At Harvard for</b> <b>example, for many years</b> <b>when you were coming</b> <b>in you had to undress</b> <b>and they took your</b> <b>picture naked,</b> <b>and they would at your picture</b> <b>to decide what kind of students</b> <b>you would become based</b> <b>on physical traits,</b> <b>like because your head.</b> <b>This was Harvard.</b> <b>I guess this was one</b> <b>proponent of what</b> <b>was called then phrenology.</b> <b>Which is the measure of--</b> <b>So there is plenty,</b> <b>plenty of room</b> <b>to integrate those</b> <b>kind intricacies</b> <b>within traditional curriculum.</b> <b>
Yeah, and I want</b> <b>to express the fact that</b> <b>to do such a thing is</b> <b>not just to include</b> <b>all of the hushed</b> <b>historical abuses</b> <b>that we gloss over in</b> <b>our current curricula.</b> <b>But it's also to acknowledge the</b> <b>good that we have overlooked.</b> <b>So discoveries and innovations</b> <b>that other people who are not</b> <b>quote unquote "western", and</b> <b>who are not Anglo-American</b> <b>have made and contributed</b> <b>to the fields of science,</b> <b>and history, and literature.</b> <b>So not only to recognize</b> <b>these historical evils,</b> <b>but also the grand positive</b> <b>impacts that others have had.</b> <b>
</b> <b>Absolutely, absolutely.</b> <b>So are you guys happy</b> <b>with the answers?</b> <b>[laughter] OK, so</b> <b>thank you very much.</b> <b>[applause]</b>
<b>The following content is</b> <b>provided under a Creative</b> <b>Commons license.</b> <b>Your support will help</b> <b>MIT OpenCourseWare</b> <b>continue to offer high quality</b> <b>educational resources for free.</b> <b>To make a donation, or to</b> <b>view additional materials</b> <b>from hundreds of MIT courses,</b> <b>visit MIT OpenCourseWare</b> <b>at ocw.mit.edu.</b> <b>OK, so this is the</b> <b>second part of the class,</b> <b>and this the last</b> <b>for the semester.</b> <b>So I think we've traveled a lot.</b> <b>We see many difficult</b> <b>issues I think.</b> <b>And I want to give</b> <b>you credit, I think,</b> <b>for-- during the semester for</b> <b>being so open, and honest,</b> <b>and being able to engage</b> <b>into difficult issues.</b> <b>And each time that you</b> <b>guys were presenting you</b> <b>might remember me</b> <b>liking you to you</b> <b>to get deeper into</b> <b>personal issues</b> <b>and not just the abstract.</b> <b>And the reason is because</b> <b>I think you make a better</b> <b>case where everyone is</b> <b>of use you can bring</b> <b>your own narrative into it.</b> <b>And I think it's-- overall I</b> <b>think it has worked and myself</b> <b>learned so much from the</b> <b>semester from listening to you,</b> <b>some of your own personal</b> <b>discoveries about yourself,</b> <b>about issues of race,</b> <b>language, gender, sexuality.</b> <b>And I give you</b> <b>lots of gratitude.</b> <b>It's risky, and because</b> <b>when you came here,</b> <b>you didn't know each</b> <b>other, most of you right?</b> <b>And yet to be able open yourself</b> <b>to the difficult questions</b> <b>and to give answers</b> <b>that were so thoughtful.</b> <b>And so-- many of them I</b> <b>didn't expect some of you</b> <b>to literally come out</b> <b>in this class, in a way.</b> <b>Right?</b> <b>So I thought that very</b> <b>well in this class.</b> <b>So I give you my</b> <b>respect for that.</b> <b>So I'm going to</b> <b>end passing to you.</b> <b>
Yeah.</b> <b>
You get</b> <b>a little while more,</b> <b>maybe you've got</b> <b>things that you feel</b> <b>you take with you</b> <b>to the next steps</b> <b>so that you feel that</b> <b>all this will come</b> <b>after you this class might</b> <b>benefit from anything that you</b> <b>want share, in terms</b> <b>of your learning</b> <b>experience in this</b> <b>class, and how you think</b> <b>can bridge those gaps</b> <b>that we've seen because</b> <b>of our societies, our families?</b> <b>So I wasn't here last</b> <b>Thursday but I watched</b> <b>the video of the whole class.</b> <b>[inaudible] too,</b> <b>actually really impressed</b> <b>by how you guys brought</b> <b>up your personal issues.</b> <b>And so but these are</b> <b>going to have worked on.</b> <b>I assume you have to make some.</b> <b>For this class-- for this</b> <b>class 40 minutes or so</b> <b>you have to be</b> <b>something to bring out.</b> <b>We'll go to Nicky.</b> <b>You're going to take a pass?</b> <b>OK we'll go to Edna.</b> <b>
She's</b> <b>quicker with words.</b> <b>
Yeah.</b> <b>
So I'm not sure</b> <b>if I fully understand what</b> <b>I'm supposed to say, but I'm--</b> <b>
Anything.</b> <b>
--going to</b> <b>say what I was thinking.</b> <b>So for me personally throughout</b> <b>this class I think I realized--</b> <b>I explored myself and</b> <b>my identity way more,</b> <b>and I think I was forced to ask</b> <b>questions about myself that I</b> <b>hadn't.</b> <b>And for me the main</b> <b>thing I realized</b> <b>is like how uncomfortable I</b> <b>am with being multiracial.</b> <b>I enjoy being</b> <b>multiracial but I didn't</b> <b>realize that there's parts</b> <b>of me that don't necessarily</b> <b>fit together, and how I don't</b> <b>really see myself as one</b> <b>certain race, or whatever.</b> <b>And I think the things that with</b> <b>that were our personal essays,</b> <b>and definitely free</b> <b>writing assignments</b> <b>that one of the guest</b> <b>professors had us do,</b> <b>that got pretty deep.</b> <b>And I was kind of</b> <b>like, oh crap I</b> <b>have to write at 9:30 in</b> <b>the morning, when she first</b> <b>said that.</b> <b>But then I really enjoyed it.</b> <b>And I think like maybe bringing</b> <b>out more multi-racial topics</b> <b>next year would be</b> <b>beneficial to people</b> <b>to know that that was a</b> <b>problem with themselves.</b> <b>And I think also a</b> <b>really interesting topic</b> <b>that we went over was--</b> <b>I think-- What did we call it?</b> <b>Faking?</b> <b>Passing, passing-- and</b> <b>how it's a privilege.</b> <b>And talking about like all the</b> <b>controversial issues on that</b> <b>and don't think I really</b> <b>also thought about that</b> <b>before this class.</b> <b>So that was something</b> <b>I benefited from.</b> <b>
So you say</b> <b>faking instead of passing?</b> <b>
Yeah.</b> <b>And it reminded of a comment</b> <b>Cynthia made back then, right?</b> <b>That often, when</b> <b>we think of passing</b> <b>we think of it as something</b> <b>which is negative--</b> <b>
Yeah.</b> <b>
But contained</b> <b>in that point, which</b> <b>I remember very well, I</b> <b>thought very important</b> <b>that passing should not be</b> <b>viewed from that perspective.</b> <b>I say to that point</b> <b>that we can think</b> <b>of passing as a normal</b> <b>survival strategy in a way.</b> <b>So maybe--</b> <b>
Yeah.</b> <b>
That's a nice</b> <b>segue to think of [inaudible]..</b> <b>
I actually</b> <b>have another one I think.</b> <b>
Say it again?</b> <b>
Nothing, can</b> <b>I have another one?</b> <b>
Sure of course.</b> <b>[inaudible]</b> <b>[laughter]</b> <b>
So I guess</b> <b>it's sort of hard,</b> <b>I think it sort of takes</b> <b>a lot of reflecting--</b> <b>
Yeah.</b> <b>
Sort of think about</b> <b>all that that we've discussed.</b> <b>Sort of a quick note, one of</b> <b>things that this class has</b> <b>sort of solidified for me is</b> <b>the importance of conversations.</b> <b>And so I think most of learning</b> <b>that I've done in this class</b> <b>and other sort of seminar</b> <b>based classes sort of pales</b> <b>in comparison-- or what I've</b> <b>learned in my technical courses</b> <b>pales in comparison</b> <b>to the things</b> <b>that I learn in</b> <b>classes like this</b> <b>because at the end of the day,</b> <b>I think most of the learning</b> <b>comes from those around you.</b> <b>Class needs this.</b> <b>So, experiences that</b> <b>they have more so</b> <b>than what you learn in</b> <b>these technical classes.</b> <b>There's a saying that says</b> <b>five years after you graduate</b> <b>you won't remember 90%</b> <b>or whatever of what</b> <b>you learned in college.</b> <b>But I think what</b> <b>you do remember is</b> <b>these sort conversations</b> <b>that boil down</b> <b>to personal experiences.</b> <b>And so I think, again,</b> <b>there is a challenge</b> <b>in this world of</b> <b>sort of approaching</b> <b>people who aren't really willing</b> <b>to have these conversations.</b> <b>But sort of changing</b> <b>yourself is the first step.</b> <b>And I think if everyone</b> <b>sort of makes this, or has</b> <b>this realization, takes a step</b> <b>toward making this change,</b> <b>It's very idealistic,</b> <b>but I think</b> <b>the world be a better place.</b> <b>And so I think that's</b> <b>probably one thing that I've</b> <b>gotten from this class.</b> <b>
Go man.</b> <b>
Bouncing</b> <b>off of what Burk said,</b> <b>I think one way in which I would</b> <b>like to see his class grow is</b> <b>the integration of having some</b> <b>sort of conversation like open</b> <b>discussion instead of debate</b> <b>with someone who does not come</b> <b>from a similar theological--</b> <b>not theological--</b> <b>
Ideological.</b> <b>
Thank you.</b> <b>Ideological background</b> <b>as opposed to people</b> <b>in this class, to try to see</b> <b>and sort of gain practice</b> <b>in having a session</b> <b>with someone who</b> <b>is a lot different from you,</b> <b>and developing a common trust,</b> <b>and come to see each</b> <b>other's viewpoints</b> <b>in a way that would be</b> <b>more informative than what</b> <b>one could do among a set</b> <b>like-minded individuals.</b> <b>
Let's see.</b> <b>So one thing that I</b> <b>enjoyed about this class</b> <b>I can distinctly</b> <b>remember one class</b> <b>that literally blew my mind</b> <b>I think I turned to Sonny</b> <b>and was like, oh my gosh.</b> <b>And that was when</b> <b>we interrogated</b> <b>what MIT was as an</b> <b>institution and where we sit</b> <b>in this hierarchy of social--</b> <b>in the social</b> <b>hierarchy in the US.</b> <b>And I think when</b> <b>coming to MIT I was</b> <b>willing to give the</b> <b>institution a free pass a lot</b> <b>because I felt</b> <b>that I was getting</b> <b>a lot from the experience.</b> <b>But I think now that I'm</b> <b>senior and all the seniors here</b> <b>graduate-- last day of</b> <b>class with you guys--</b> <b>now that I'm at this point</b> <b>where I can think back</b> <b>on what MIT has</b> <b>done for me and what</b> <b>and how it functions</b> <b>I think I'm better</b> <b>poised to think more critically</b> <b>about the institutions I choose</b> <b>to join future for postgraduate</b> <b>education and after.</b> <b>
So I took this</b> <b>class as a challenge</b> <b>to myself because</b> <b>at the beginning,</b> <b>I knew about the</b> <b>prejudices and stereotypes</b> <b>that I had been taught, and</b> <b>I wanted to get rid of them.</b> <b>And I think this has</b> <b>helped a lot with that</b> <b>just by hearing everyone else's</b> <b>perspectives from the readings,</b> <b>from looking at the world,</b> <b>by taking a step back</b> <b>and looking at</b> <b>things as they are</b> <b>and not how we were specifically</b> <b>taught that they are.</b> <b>And I really appreciate that.</b> <b>I want to- from here on</b> <b>I want to move forward</b> <b>and try to figure</b> <b>out how to be an ally</b> <b>from an asian-american</b> <b>perspective</b> <b>because a lot of the times</b> <b>asian-american families</b> <b>try to be the status quo.</b> <b>Like my parents have</b> <b>always taught to try to be,</b> <b>not explicitly, but implicitly</b> <b>try to be as white as you can.</b> <b>And I think that that was</b> <b>a really difficult lesson</b> <b>to go through, and then come</b> <b>to a place like MIT which</b> <b>is so diverse and which has</b> <b>so many different backgrounds.</b> <b>And so come in thinking</b> <b>there's a hierarchy where</b> <b>there isn't and</b> <b>now going forward,</b> <b>trying to break that down.</b> <b>
Related to</b> <b>what you said actually</b> <b>is this concept of being</b> <b>multi-racial, and interrogating</b> <b>that for yourself.</b> <b>But also within a context</b> <b>of being in a society</b> <b>of multiple races, and</b> <b>understanding the spaces that</b> <b>exist for meaningful</b> <b>conversations</b> <b>across cultural, and</b> <b>ethnic, and other sometimes</b> <b>just socially</b> <b>constructed boundaries</b> <b>that don't necessarily</b> <b>need to be boundaries.</b> <b>And I remember in one</b> <b>conversation Cynthia actually</b> <b>really challenged this idea of</b> <b>forming alliances especially</b> <b>between historically,</b> <b>systematically</b> <b>oppressed groups.</b> <b>And in a class</b> <b>like Black Matters</b> <b>I think that we actually think</b> <b>about people who don't identify</b> <b>as black, and who may not</b> <b>be read as black in society,</b> <b>and we've engaged</b> <b>critically with the issues</b> <b>that they face as well.</b> <b>When we talked about immigration</b> <b>from Mexico, and in other times</b> <b>where I insist upon</b> <b>interjecting indigenous</b> <b>North American issues</b> <b>into the conversation.</b> <b>But there seems to be</b> <b>space for all of that,</b> <b>and so what I want to take away</b> <b>from all of these discussions</b> <b>is how can we create meaningful</b> <b>alliances, and be allies,</b> <b>and how can we</b> <b>collaborate between groups</b> <b>who are distinct and different,</b> <b>and face similar challenges</b> <b>in this society, but also have</b> <b>really specific perspectives</b> <b>and experiences,</b> <b>and future goals.</b> <b>So the idea of</b> <b>collaborations, especially</b> <b>among historically</b> <b>oppressed groups</b> <b>is one that I want</b> <b>to continue thinking</b> <b>about outside of this class.</b> <b>
A lot of what</b> <b>I was thinking about</b> <b>during this class</b> <b>for me personally</b> <b>was comparing and contrasting</b> <b>because I come from a very--</b> <b>I'm from Zimbabwe.</b> <b>So there's not a</b> <b>lot of diversity,</b> <b>and this is an entirely</b> <b>new experience to me,</b> <b>so a lot of what</b> <b>I were doing was</b> <b>comparing my past, my</b> <b>history, my experiences</b> <b>with you guys experiences</b> <b>and everything around me,</b> <b>and thinking about for</b> <b>instance, in the readings</b> <b>how they analyzed</b> <b>different situations,</b> <b>the situation in America,</b> <b>the situation in Haiti.</b> <b>And thinking about where</b> <b>these things are connected</b> <b>how they are different.</b> <b>What could have been</b> <b>done what should be done?</b> <b>And I think I rarely thought</b> <b>about issues like this</b> <b>before this class,</b> <b>and now I'm starting</b> <b>to think about it more,</b> <b>especially in education.</b> <b>I'm really interested in</b> <b>education, and how we can help</b> <b>teach people in the future how</b> <b>to use their skills that they</b> <b>learn, how to think about, how</b> <b>to approach certain problems</b> <b>in a way that challenges them,</b> <b>and also challenges the status</b> <b>quo, and challenges the</b> <b>hierarchies that are set up.</b> <b>So personally, I think</b> <b>that this class definitely</b> <b>helped me think</b> <b>a lot more, and I</b> <b>hope that in future it will</b> <b>help a lot more people think</b> <b>about their actions, think</b> <b>about what they're saying,</b> <b>think about what</b> <b>they're doing, and think</b> <b>about how they can</b> <b>help and how they</b> <b>can engage their communities.</b> <b>For instance, I</b> <b>remember when we're</b> <b>talking about how we came</b> <b>to MIT we might graduate,</b> <b>and then what are we</b> <b>going to do would we</b> <b>go back to the communities</b> <b>where we come from?</b> <b>How we are going--</b> <b>how do you plan</b> <b>to affect those communities?</b> <b>Do you go back as experts, or</b> <b>do we go back as one of them,</b> <b>do we to fit in?</b> <b>Questions like</b> <b>that are definitely</b> <b>things that we should</b> <b>think about, and I think</b> <b>are very important.</b> <b>
Yeah I think</b> <b>the biggest thing</b> <b>for me was learning about</b> <b>what we were learning,</b> <b>but then thinking back to how I</b> <b>grew up in a really white area,</b> <b>and I was always taught there</b> <b>is one correct way to speak,</b> <b>there is like one correct</b> <b>way to teach kids.</b> <b>And thinking back now</b> <b>with all this information</b> <b>how damaging that really</b> <b>is, and how it it's</b> <b>not really about the language,</b> <b>and it's not really about</b> <b>how we speak it's more</b> <b>about marginalizing groups</b> <b>that might speak that way.</b> <b>And it's just thinking about</b> <b>how my entire school system was</b> <b>based on what they thought was</b> <b>correct, and was telling us</b> <b>what we should think</b> <b>is correct, and I</b> <b>think coming out</b> <b>of this hopefully</b> <b>we can try to change some of</b> <b>that in the school systems</b> <b>because it can really</b> <b>affect how kids</b> <b>view the world as they grow up.</b> <b>
I think one thing</b> <b>I gained from this class</b> <b>is, I guess, verification of</b> <b>the idea that listening is very</b> <b>important, and there's</b> <b>kind of various ways</b> <b>you can think about it.</b> <b>Just in the context</b> <b>of this class,</b> <b>whether it's listening to other</b> <b>people share their experiences,</b> <b>and really taking</b> <b>in and understanding</b> <b>how they fit in within</b> <b>whatever social hierarchy,</b> <b>or whatever kind of</b> <b>issue we're discussing.</b> <b>And seeing how it's kind of</b> <b>played a role in their lives,</b> <b>and kind of seeing how you</b> <b>could apply it to either better</b> <b>someone else's life,</b> <b>improve your own interaction</b> <b>with the world in some way.</b> <b>Or whether it's listening</b> <b>in terms of language.</b> <b>We talked a lot about</b> <b>language and how</b> <b>certain modes of speaking</b> <b>are somewhat marginalized.</b> <b>So ensuring that as you</b> <b>go forward you really</b> <b>listen to people regardless of</b> <b>how they present their ideas,</b> <b>or present their</b> <b>thoughts, or whether it's</b> <b>listening as far as</b> <b>engaging people who</b> <b>have opposing opinions as you.</b> <b>Really understanding</b> <b>that in order</b> <b>to strengthen your</b> <b>own positions,</b> <b>and really understand why it is</b> <b>that you think the way it is.</b> <b>You have to be able to</b> <b>listen to people who think</b> <b>in a different way than you.</b> <b>So yeah, I've just seen</b> <b>a lot of different things</b> <b>in this class that speaks to</b> <b>the importance of listening.</b> <b>
I think</b> <b>what this class</b> <b>has made me call into question</b> <b>a lot is my lack of analysis</b> <b>of my racial background</b> <b>because I'm Japanese-American.</b> <b>It's not really</b> <b>something that I'm</b> <b>forced to address all the time</b> <b>because no one's really calling</b> <b>that much attention to</b> <b>it because neither one</b> <b>of those groups is really that</b> <b>attacked in modern society</b> <b>for being a racial group.</b> <b>And something I didn't</b> <b>really even think,</b> <b>my mom is an immigrant.</b> <b>I didn't really</b> <b>address that ever</b> <b>because no one was calling in to</b> <b>question her immigration here.</b> <b>And that also leads</b> <b>to a lot relating</b> <b>to how the opinions of</b> <b>other people, and especially</b> <b>my very conservative</b> <b>extended family,</b> <b>and how their opinions</b> <b>about certain things</b> <b>such as immigration are</b> <b>relying on people being</b> <b>of certain races, and being</b> <b>from certain places because it</b> <b>isn't something that</b> <b>they've ever seen negatively</b> <b>about my family.</b> <b>But it is something that's</b> <b>a cornerstone of why</b> <b>my family exists.</b> <b>So I think things like</b> <b>that, having people address</b> <b>their own identities,</b> <b>and their own backgrounds</b> <b>and how that's</b> <b>affected how they see</b> <b>everything else in their</b> <b>interactions with other people</b> <b>I think is a really great</b> <b>part of this course.</b> <b>
I'm having trouble</b> <b>just to lay everything down,</b> <b>but I guess one big</b> <b>takeaway from this class</b> <b>was accepting that</b> <b>there is no one right</b> <b>way to exist as a human across</b> <b>individuality, race, and class,</b> <b>and things like that.</b> <b>And that was a</b> <b>realization that I</b> <b>think I came to in this class.</b> <b>And it all goes back</b> <b>to the importance</b> <b>of interrogating</b> <b>your beliefs, and why</b> <b>you think the ways that you do.</b> <b>And it's really</b> <b>eye-opening and I think</b> <b>that it's really important.</b> <b>You have to open</b> <b>yourself up to that</b> <b>and it takes a certain</b> <b>level of vulnerability.</b> <b>I wasn't expecting to come</b> <b>out in this class at all,</b> <b>but I did because this was</b> <b>a place where I felt safe,</b> <b>and it was really</b> <b>surprising that I felt</b> <b>so safe in this environment.</b> <b>And I think that's something</b> <b>we need to recreate elsewhere.</b> <b>A lot of people I would like</b> <b>to take for this in the future</b> <b>is therapy who are</b> <b>seniors taking this class,</b> <b>what do you do now?</b> <b>What class do you take</b> <b>next, or what actions</b> <b>can you take because of what</b> <b>you've learned in this class?</b> <b>I think something</b> <b>that maybe they</b> <b>could have it in this is</b> <b>thinking about concrete ways</b> <b>to actually do stuff.</b> <b>Because we had Professor Chomsky</b> <b>in here on Tuesday and he said,</b> <b>almost anything, is the</b> <b>answer to what do we do.</b> <b>So maybe having some</b> <b>sort of an initiative</b> <b>coming out of this class</b> <b>is to make tape just</b> <b>at MIT for just</b> <b>impulsive [inaudible]..</b> <b>
So kind</b> <b>of piggybacking off</b> <b>of what Chris said,</b> <b>I think the most</b> <b>those meaningful</b> <b>conversations and talks</b> <b>that we've had for me, where</b> <b>when we talked to professors,</b> <b>people all over</b> <b>Cambridge who were using</b> <b>their own backgrounds,</b> <b>their own field of study,</b> <b>and they're making</b> <b>tangible changes.</b> <b>It's Christmas day.</b> <b>So for example, when we</b> <b>watch the virtual reality</b> <b>kind of simulation in terms</b> <b>of the Israeli-Palestinian</b> <b>conflict.</b> <b>And I feel like those</b> <b>conversations and the fact</b> <b>that I have had the</b> <b>privilege of taking</b> <b>this class in my semester</b> <b>has really kind of driven me</b> <b>to really to take</b> <b>action outside of MIT,</b> <b>and hopefully in the</b> <b>future back at MIT.</b> <b>And to really take that</b> <b>with me in my specific field</b> <b>of medicine, and how these</b> <b>various identities can really</b> <b>affect people's health care,</b> <b>and that's life or death.</b> <b>And so I feel like that is</b> <b>really been eye-opening for me</b> <b>and that's driven me to action.</b> <b>
Yeah, I think</b> <b>for me, from this class</b> <b>has made me more aware of how</b> <b>many things they go through.</b> <b>Compared to what I didn't know</b> <b>about the Haitian Revolution</b> <b>as it occurred.</b> <b>And I didn't really know how</b> <b>to question during the time.</b> <b>But it's all things</b> <b>that we are taught just</b> <b>because a lot the</b> <b>history that we</b> <b>learn is taught from a very</b> <b>westernized perspective.</b> <b>So I feel like</b> <b>it's very important</b> <b>that this class teaches that</b> <b>there's always many ways</b> <b>to view certain things, and</b> <b>there's many things that aren't</b> <b>necessarily taught just because</b> <b>society just doesn't want us</b> <b>to know about these</b> <b>things, so that we</b> <b>could learn from this, yeah.</b> <b>
I think similar</b> <b>to what Hermann was saying,</b> <b>I think this class is</b> <b>really about the idea</b> <b>of taking action.</b> <b>In a lot our readings passivity</b> <b>was just there all the time.</b> <b>People I remember it,</b> <b>the need to settle down,</b> <b>or afternoon it's just like,</b> <b>oh that's really strange</b> <b>but I guess the way things are.</b> <b>He noticed that things were</b> <b>wrong but he didn't anything,</b> <b>so I think just</b> <b>trying to take action,</b> <b>and Megan you mentioned</b> <b>interrogating yourself</b> <b>is an important way to do it.</b> <b>And I really liked</b> <b>first assignment,</b> <b>which was the linguistic</b> <b>autobiography,</b> <b>and I felt that just thinking</b> <b>of my whole life in terms</b> <b>of the languages I grew</b> <b>up with, reading, writing,</b> <b>and how they were</b> <b>presented to me</b> <b>was a really good way just</b> <b>to look into my own self.</b> <b>What I think would have</b> <b>been, would be better,</b> <b>in future versions of class</b> <b>would be maybe talking together</b> <b>in smaller groups.</b> <b>Would be harder.</b> <b>I guess I think it's</b> <b>just easier just</b> <b>to talk about yourself</b> <b>that way for some people</b> <b>are more comfortable, I think.</b> <b>The only time we got to</b> <b>that was last Tuesday we</b> <b>had Professor Olive ub here, and</b> <b>we worked in groups of three,</b> <b>and that was really cool.</b> <b>
OK, thank</b> <b>you, thank you.</b> <b>I think so what's adjusted also,</b> <b>you mean like a [inaudible]..</b> <b>So what Chris said</b> <b>we can try together</b> <b>because this class I</b> <b>hope will be often many,</b> <b>many, many, many times again</b> <b>here, [inaudible] I hope.</b> <b>Actually thinking of next year,</b> <b>I might do it with [inaudible]..</b> <b>So from that perspective</b> <b>maybe you guys</b> <b>could share with</b> <b>me what you think</b> <b>will be done to implement</b> <b>what you guys think</b> <b>in terms of taking this</b> <b>knowledge into action.</b> <b>Because throughout</b> <b>the semester I</b> <b>was struggling with what</b> <b>kind of project do I assign?</b> <b>Because I often</b> <b>felt that it would</b> <b>be so nice to do a project that</b> <b>puts you guys in the world,</b> <b>not just writing an essay.</b> <b>And I try to see what--</b> <b>Something could come</b> <b>up spontaneously</b> <b>from you that could</b> <b>bond in to out of space.</b> <b>So how would you advise me</b> <b>in next cycle of this course</b> <b>to make that happen?</b> <b>To do projects, maybe</b> <b>throughout the semester only</b> <b>or optional projects</b> <b>that would really</b> <b>turn knowledge into action?</b> <b>OK so it's getting</b> <b>that, and then that.</b> <b>
So,</b> <b>maybe some things--</b> <b>I still think you should</b> <b>have the essays because</b> <b>at least for me they were pretty</b> <b>beneficial in exploring myself,</b> <b>but maybe instead of</b> <b>having the essay revision,</b> <b>although I think revising</b> <b>essays is important,</b> <b>I think that can be done</b> <b>throughout your essay process</b> <b>instead.</b> <b>And maybe like putting</b> <b>in a project where</b> <b>you have to create a policy,</b> <b>or like a lesson plan,</b> <b>or something like that,</b> <b>and then present that.</b> <b>And maybe even have a</b> <b>plan of action of who you</b> <b>would go to talk to about this?</b> <b>I think if you just</b> <b>come up with a plan it</b> <b>can be pretty philosophical,</b> <b>or just an idea</b> <b>and will never be</b> <b>done with it, what</b> <b>would be your next step</b> <b>now you have an idea,</b> <b>how do you implement it?</b> <b>And then also, something</b> <b>I forgot mention before,</b> <b>but I thought I was really</b> <b>useful about this class</b> <b>was when we went</b> <b>to go to the movies</b> <b>all together on a Saturday.</b> <b>At first I was like, oh crap,</b> <b>now I have to like allot time,</b> <b>but I actually</b> <b>really enjoyed that</b> <b>and was hoping the</b> <b>conversation would keep going.</b> <b>So maybe having more</b> <b>things outside the class,</b> <b>because I think it's really</b> <b>hard to explore all the things</b> <b>that we want to do in</b> <b>an hour and a half.</b> <b>So maybe having somewhere--</b> <b>I don't know what</b> <b>types of places</b> <b>you can go to-- but maybe</b> <b>going to an inner city</b> <b>school in Boston together, and</b> <b>seeing how it's taught there,</b> <b>and what you would change?</b> <b>
OK, that's</b> <b>good to hear actually</b> <b>because one thing I'm toying</b> <b>with the Fox, Mr. Harald,</b> <b>is to see whether we could</b> <b>change the format, instead</b> <b>having a hour and a half to add</b> <b>once a week from three hours.</b> <b>In the past I tried that</b> <b>because of the schedule for MIT</b> <b>it's hard for students to</b> <b>enroll with a three hour class.</b> <b>when they graduate.</b> <b>So we stepped away</b> <b>from that, but now</b> <b>I don't whether</b> <b>we should do that.</b> <b>In admitting with three hours</b> <b>we could go someplace else.</b> <b>
Yeah, I that</b> <b>might be beneficial.</b> <b>I know negotiation at Sloan</b> <b>does it at Thursday at 5:00</b> <b>or something like that.</b> <b>So it's hard to schedule a</b> <b>three hour class during the day,</b> <b>but if you do it</b> <b>as a night session.</b> <b>
OK, OK, OK.</b> <b>Thank you for saying that Alisa?</b> <b>I will report back to</b> <b>Colton and see what--</b> <b>So it goes Dana, then</b> <b>Sarah then Jackie.</b> <b>
Well its</b> <b>related directly</b> <b>she just said about time--</b> <b>
OK, OK.</b> <b>
It's just</b> <b>like a short thing,</b> <b>but having a class at</b> <b>5:00 would be difficult</b> <b>because there is</b> <b>a lot of athletes</b> <b>who participate because they</b> <b>practice from 5:00 to 7:00.</b> <b>
</b> <b>I see, I see, OK.</b> <b>
Yeah there's</b> <b>another section at 7:00</b> <b>I'm pretty sure.</b> <b>So.</b> <b>
Closing class</b> <b>are from 7:00 to 10:00.</b> <b>
</b> <b>7:00 to 10:00, yeah.</b> <b>
Yeah.</b> <b>
OK.</b> <b>
Yeah we were</b> <b>talking about projects.</b> <b>And I think that could</b> <b>definitely work in this course,</b> <b>and I think one thing</b> <b>that I've really wanted</b> <b>is to be able to</b> <b>affect change in this--</b> <b>within the context of this</b> <b>course and sort of have</b> <b>a reason to do so</b> <b>based on everyday's--</b> <b>I would like to go</b> <b>into school and say,</b> <b>let me try my at</b> <b>teaching kids in this way</b> <b>and see if it</b> <b>actually works there.</b> <b>So if you wanted to</b> <b>organize that sort of thing</b> <b>it could be cool to also</b> <b>work in small groups</b> <b>and we diversify the--</b> <b>I mean we have a ton</b> <b>of courses in here,</b> <b>but I think there's a</b> <b>lot of working, going out</b> <b>into the real</b> <b>world and I think--</b> <b>I mean this is just me agreeing.</b> <b>I think a three hour</b> <b>course could be good.</b> <b>
OK.</b> <b>Sarah?</b> <b>
So I just</b> <b>think add stems</b> <b>from what Colin</b> <b>said about talking</b> <b>to someone with a different</b> <b>ideological perspective</b> <b>than yourself, so</b> <b>maybe having students</b> <b>go in groups of two [inaudible]</b> <b>out into world and maybe</b> <b>interview someone, or share</b> <b>some of the information</b> <b>about the Haitian</b> <b>Revolution should have,</b> <b>it was like one of visionary</b> <b>revolutions to have.</b> <b>[interposing voices]</b> <b>
Yeah, yeah.</b> <b>
Something</b> <b>along those lines.</b> <b>
OK, OK, OK.</b> <b>So going back to the last point.</b> <b>So in terms of going</b> <b>into the schools</b> <b>I have some good</b> <b>news, which is that, I</b> <b>think I should have [inaudible]</b> <b>with you about this new program</b> <b>that has been launched in</b> <b>the Boston public school,</b> <b>this new program.</b> <b>And Vindlam Khan actually</b> <b>she's I think Chinese,</b> <b>and she actually, she</b> <b>launched-- she had launched</b> <b>the program in</b> <b>Chinese and English</b> <b>can be introduced as</b> <b>going reasonable well,</b> <b>from what I hear.</b> <b>And then she's also responsible</b> <b>launching this Creole</b> <b>to English learning program.</b> <b>And she has offered</b> <b>that if there</b> <b>are students in this</b> <b>class and other classes</b> <b>at MIT if they want to come and</b> <b>volunteer, and take advantage</b> <b>of that new program in terms</b> <b>of doing research that's</b> <b>socially based, or</b> <b>empirically based,</b> <b>or even like [inaudible].</b> <b>Measuring students</b> <b>learning games</b> <b>she would be happy</b> <b>to work with you.</b> <b>There's one student</b> <b>in the other class</b> <b>posted it's [inaudible] with</b> <b>the [inaudible] twice, and was</b> <b>to be going on a weekly basis to</b> <b>the meetings who try understand</b> <b>how they are creating this new</b> <b>program where there would be</b> <b>I think half kids who speak</b> <b>creole and half who don't speak</b> <b>creole.</b> <b>And this would be</b> <b>truly bilingual,</b> <b>so you get American or other</b> <b>of these kids learning creole</b> <b>and get the Haitian creole kids.</b> <b>We've got to include both the</b> <b>English and bigger English,</b> <b>so I think it's really I</b> <b>think very radical move</b> <b>for the government school.</b> <b>They've had other</b> <b>programs of that sort</b> <b>but never one involving</b> <b>the non-imperial language.</b> <b>Typically it's Spanish-English,</b> <b>French-English,</b> <b>Chinese-English, but</b> <b>it's the first time</b> <b>that they are doing</b> <b>one that includes</b> <b>a language like Haitian Creole</b> <b>that's been over the years</b> <b>as you know [inaudible] history.</b> <b>So we're excited, we've</b> <b>hit some push back</b> <b>including push back back from</b> <b>Haitian parents, some of them</b> <b>would prefer to have it in</b> <b>French, as you can imagine.</b> <b>They don't speak French,</b> <b>it's a difficult neo-colonial</b> <b>[inaudible] into an imperial</b> <b>language, so to speak.</b> <b>But I think there has been some</b> <b>good progress on that front.</b> <b>Noam Chomsky was</b> <b>here, Tuesday he</b> <b>had an interview which I</b> <b>put on my Facebook page--</b> <b>I can show it to you--</b> <b>where it goes me,</b> <b>him, and I think that's</b> <b>[inaudible] together</b> <b>from Mexico, but he</b> <b>grew up in California.</b> <b>And this was a very interesting</b> <b>exchange between the two of us.</b> <b>Where Chomsky might be</b> <b>important of promoting these so</b> <b>called [inaudible]</b> <b>support for really</b> <b>of duration of the [inaudible].</b> <b>So, I think those</b> <b>are good suggestions,</b> <b>those are very good suggestions.</b> <b>And since Harald is from a</b> <b>background in computation,</b> <b>and as we saw doing the project</b> <b>on involving virtual reality</b> <b>that might be another part where</b> <b>we can do action in the course.</b> <b>
With the-- just</b> <b>based off like the Fox Harald</b> <b>computation part, I thought it</b> <b>was really interesting learning</b> <b>about what he was doing.</b> <b>I think if there was--</b> <b>but I don't know if this is</b> <b>where you're going, but it.</b> <b>I think there was a</b> <b>more of a requirement</b> <b>to do more computation based</b> <b>things I probably would</b> <b>straight away from this class.</b> <b>
OK.</b> <b>
Because, so I'm not</b> <b>like the stereotypical major</b> <b>here, I'm a business</b> <b>major, I think</b> <b>I would been slightly</b> <b>intimidated if there</b> <b>was a CMS type project in here.</b> <b>So maybe making that one of</b> <b>the facets of what you can do,</b> <b>but not a requirement.</b> <b>
OK, that's OK.</b> <b>
Yeah.</b> <b>
Yeah,</b> <b>good for you, thank you.</b> <b>
Yeah.</b> <b>
I think also taking</b> <b>advantage of the learning</b> <b>styles of the students in the</b> <b>class could be interesting.</b> <b>So maybe doing more</b> <b>creative projects,</b> <b>or doing projects</b> <b>that people feel</b> <b>like allows them to</b> <b>use their best skills</b> <b>and actually have</b> <b>fun I feel like--</b> <b>
So tell</b> <b>me Cynthia if you could</b> <b>a project that was creative</b> <b>what would you have done?</b> <b>Just to the extent of what you</b> <b>could think of in the future?</b> <b>
Just off</b> <b>the fly I maybe would</b> <b>have made a fucking 10 foot</b> <b>tall papier-mache person</b> <b>and written stuff all over it</b> <b>and then put it in lobby 10.</b> <b>
OK, OK.</b> <b>
Like negative</b> <b>comments or like--</b> <b>
Anything, just like</b> <b>stuff that's interesting.</b> <b>
</b> <b>OK, and then they</b> <b>would have to the part</b> <b>that includes communication</b> <b>of some sort.</b> <b>So how would do</b> <b>that, so you do that</b> <b>and have a paper</b> <b>describing why you did it,</b> <b>the mission of what you've</b> <b>done and the objective</b> <b>it's trying to accomplish?</b> <b>
I mean if it</b> <b>just like a black body</b> <b>or like a brown</b> <b>body of some sort</b> <b>I would even just to</b> <b>leave markers and stuff</b> <b>for other people to</b> <b>put messaging on it</b> <b>I feel like would be giving--</b> <b>I think art is a great</b> <b>way to give people</b> <b>an entry into having</b> <b>this conversation.</b> <b>It would be an artifact.</b> <b>
</b> <b>Excellent, excellent I'm</b> <b>so glad this is being recorded</b> <b>so that way we have this.</b> <b>No really, so that</b> <b>way we have this--</b> <b>I can watch it with</b> <b>Fox, Carol and we</b> <b>then we can play around with</b> <b>the idea, it's great too.</b> <b>Actually, anyone has concrete</b> <b>suggestions like that?</b> <b>So they told you--</b> <b>
So my class one</b> <b>of those I don't-- maybe</b> <b>you've taken it, Psychology</b> <b>of Gender and Race?</b> <b>No?</b> <b>OK.</b> <b>This [inaudible] psychologist</b> <b>who works in local area,</b> <b>I think her first name is</b> <b>Shiby I forget her last name.</b> <b>But part of the class is</b> <b>training or doing mock training</b> <b>on how do you present cultural</b> <b>awareness presentations.</b> <b>Where you go with--</b> <b>you basically form</b> <b>a group of students</b> <b>and kind of teach the other</b> <b>students her role play</b> <b>and how to college frat,</b> <b>members of a college frat</b> <b>or sixth graders out of a</b> <b>local Boston public school</b> <b>about how some things</b> <b>that they could be doing</b> <b>may be disparaging</b> <b>to minority groups,</b> <b>and it could be interesting</b> <b>having a conversation with her</b> <b>to see what things</b> <b>she incorporates</b> <b>into those presentations</b> <b>and if that would be</b> <b>useful inside of Black Matters.</b> <b>
OK, that would</b> <b>be useful, be very useful.</b> <b>Dana?</b> <b>No I was-- I'm done.</b> <b>
OK.</b> <b>[laughter]</b> <b>
With the</b> <b>wiki that I think</b> <b>was a new thing this</b> <b>semester, I think,</b> <b>it's another way for us to</b> <b>communicate as students,</b> <b>it would be great because</b> <b>just maybe someone</b> <b>said something to you that</b> <b>was really interesting</b> <b>and you only have their email.</b> <b>
Right.</b> <b>
It's hard to get in</b> <b>contact so maybe an opt-in way</b> <b>to have--</b> <b>
So the wiki</b> <b>didn't really work, huh?</b> <b>Why does the wiki not work?</b> <b>
It's hard.</b> <b>
It's not</b> <b>a good platform.</b> <b>
The wiki</b> <b>is not a good platform,</b> <b>all because we have you here.</b> <b>[laughter] The wiki</b> <b>not a good platform,</b> <b>I heard that as well in my</b> <b>other class, the students say,</b> <b>we don't like the wiki.</b> <b>
Yeah.</b> <b>
Yeah.</b> <b>[interposing voices]</b> <b>
What about like a</b> <b>Slack Channel or something</b> <b>like where you be more--</b> <b>[interposing voices]</b> <b>
Or like a Facebook,</b> <b>something that we already</b> <b>all use and wouldn't have</b> <b>to go out our way to use.</b> <b>
Yeah.</b> <b>
Yeah.</b> <b>
Who</b> <b>had their hand raised?</b> <b>Was there a hand</b> <b>raised over here?</b> <b>Was it Shope, was</b> <b>your hand raised?</b> <b>Oh, OK.</b> <b>
Yeah sure.</b> <b>So first, you were asking</b> <b>what kind of projects right?</b> <b>So from the teaching</b> <b>perspective,</b> <b>I think after the</b> <b>classes last week I'm</b> <b>learning about open</b> <b>and fixed mindset.</b> <b>Something that would</b> <b>have been interesting</b> <b>as the project would be to</b> <b>create a curriculum, just</b> <b>a lecture or a workshop for</b> <b>an age group, whatever age,</b> <b>it could be college,</b> <b>it could be lower</b> <b>and try to implement</b> <b>the like teaching</b> <b>techniques that we learned to</b> <b>foster a more open mindset.</b> <b>That would have been a</b> <b>school project to explore.</b> <b>
OK, great.</b> <b>OK so, Fenny, Dana and Edna.</b> <b>
But mine</b> <b>is off of that.</b> <b>
OK.</b> <b>
OK, so I think</b> <b>that would be a good idea.</b> <b>Also we can still enlist the</b> <b>help of the professors who made</b> <b>these lectures because I</b> <b>think that will give us</b> <b>a little more--</b> <b>when we actually go</b> <b>into these schools</b> <b>and say we'd like to bring</b> <b>this plan to you because have</b> <b>professor backing, we</b> <b>have this whole slew</b> <b>of research behind us.</b> <b>
Another cool thing</b> <b>that you could do about</b> <b>community outreach, is I know</b> <b>one time a couple weeks ago</b> <b>in Amphibious Achievement we</b> <b>had a bunch of people from</b> <b>Chocolate City come</b> <b>in, and also it's--</b> <b>there's a group-- it's like</b> <b>something something Latinas.</b> <b>And I don't know--</b> <b>[interposing voices]</b> <b>Yes!</b> <b>And so they came in and</b> <b>Chocolate City came in and--</b> <b>
But you have</b> <b>to give some background</b> <b>about Chocolate City and</b> <b>Amphibious Achievement,</b> <b>make it self-contained.</b> <b>[interposing voices]</b> <b>So Amphibious Achievement is</b> <b>we-- it's a program at MIT</b> <b>and we work with Boston</b> <b>public high schoolers</b> <b>and it's an academic</b> <b>mentorship program,</b> <b>but also we just</b> <b>work with them to try</b> <b>to build their</b> <b>confidence, and we also</b> <b>help them apply to college,</b> <b>and things like that.</b> <b>But, and then Chocolate City is</b> <b>just like it's a living group</b> <b>here on campus, and it's</b> <b>African-American men and then--</b> <b>say it again?</b> <b>
Mujeres Latinas.</b> <b>
That's a--</b> <b>[laughter]</b> <b>
--group</b> <b>of Latina women,</b> <b>and so they came in to</b> <b>talk to the high schoolers</b> <b>in Amphibious Achievement and we</b> <b>just had a discussion about how</b> <b>they feel about what they've</b> <b>been told they can and cannot</b> <b>do because of who they are.</b> <b>And it was really, really</b> <b>interesting for everybody.</b> <b>And I think that would</b> <b>be something cool to do--</b> <b>
Do you</b> <b>have a Facebook page?</b> <b>
We do, it's called</b> <b>Amphibious Achievement.</b> <b>
Right.</b> <b>That's great.</b> <b>Thank you, thank you.</b> <b>Actually, so which Facebook</b> <b>would be a better way</b> <b>to communicate instead the wiki.</b> <b>
Yeah, yeah way better.</b> <b>
Way better.</b> <b>So maybe next year we create a</b> <b>Facebook page for the course.</b> <b>
Yeah, a</b> <b>page for the course</b> <b>and then a messaging</b> <b>group for the students.</b> <b>OK, OK.</b> <b>
OK, OK that</b> <b>was good idea, good idea.</b> <b>All right.</b> <b>I should maybe</b> <b>[inaudible] a page,</b> <b>a Facebook page for this</b> <b>group, and that way you guys</b> <b>can keep in touch.</b> <b>And if there are ways that--</b> <b>even what [inaudible]</b> <b>has explained</b> <b>and then we should do</b> <b>other projects that</b> <b>might bring your strength</b> <b>and your knowledge together</b> <b>to bring actual</b> <b>change, that might be</b> <b>something we could figure out.</b> <b>But you would have be</b> <b>responsible for it.</b> <b>Colin?</b> <b>
Bouncing</b> <b>off of your idea here,</b> <b>I think it would be really</b> <b>interesting to actually</b> <b>have one part of that would</b> <b>be a [inaudible] support</b> <b>literacy class, or presentation</b> <b>or something like that.</b> <b>Where you going to take a</b> <b>certain section of literature,</b> <b>maybe just one book</b> <b>like Madam Bovary,</b> <b>and dissect the ways in</b> <b>which the society in which it</b> <b>is embedded create</b> <b>the themes within it.</b> <b>A major thing inside</b> <b>of Madam Bovary</b> <b>is centered on this</b> <b>one woman who's</b> <b>married in countryside of</b> <b>France back in 1900s or 1800s,</b> <b>and she gets really depressed</b> <b>because she has nothing to do.</b> <b>And just explaining that she</b> <b>has no options for employment,</b> <b>and is stuck at home because</b> <b>that's what was expected</b> <b>of women back in that day.</b> <b>And dissecting both</b> <b>that cultural phenomenon</b> <b>and then examining how that</b> <b>may play out in today's terms</b> <b>with different</b> <b>patterns in society.</b> <b>
OK, good, goo.</b> <b>SO a statistic fact, you</b> <b>may have the last word.</b> <b>
Wow, OK.</b> <b>So something that I</b> <b>was thinking about</b> <b>was what you had about</b> <b>having an art project,</b> <b>and I realized I'm not artistic</b> <b>at all, so I was trying</b> <b>to figure out what I would do</b> <b>if there was an art assignment.</b> <b>And I was thinking about</b> <b>something that's in a show,</b> <b>where like you put yourself</b> <b>up, you stand up there</b> <b>and then you have</b> <b>a projector screen,</b> <b>and you write</b> <b>words on it of what</b> <b>you don't like about yourself.</b> <b>And that could be</b> <b>instead of what</b> <b>we don't like of</b> <b>ourselves, things</b> <b>we've been told about ourselves</b> <b>based of our race, or gender,</b> <b>or sexuality, and also having</b> <b>things you would rather</b> <b>have been said, because what I</b> <b>really liked about this class</b> <b>was the self-exploration, so.</b> <b>
Good, good.</b> <b>All right guys so</b> <b>thank you so much.</b>
You've had a chance to look over the syllabus for this course.
What value do you think it might have for learners in the outside world, and would you recommend that they take a look at these readings and try to apply them to their own lives?
Yeah, quite definitely.
I mean, if we want to have any constructive impact on the world, any interchange with the world that will improve it will deal with the innumerable problems that exist.
It has to be based on understanding of social, political, economic reality.
Otherwise, you can't act in any serious way, just as you couldn't go into a chemistry lab and conduct an experiment unless you knew something good chemistry.
Otherwise, you could mix chemicals and make funny smells and so on.
But if you want to do something serious, you have to understand first.
And the syllabus of the course and conduct of the course aimed at and I think succeeds in providing the kind of fundamental understanding and challenge open questions that people have to think about if they want to be able to participate in a meaningful way in the general society.
What language issues are particularly poignant in today's political climate that they might talk about today?
Well, one issue that's critical is the right of people to have their lives, community affairs conducted in their own language.
So for example, if some, say, dictator were to take over the United States and produce an edict saying the official language of the country is Swahili and that we all have to talk Swahili from now on and all official documents are in Swahili.
And if you want to get a driver's license, you have to ask for it in Swahili and so on.
I think we'd probably object, but that's the actual situation.
For example, in Haiti, it's not Swahili.
It's French.
The language that the people speak is a different language, creole.
So that's kind of a problem that arises there.
Similar problems arise here.
We have Spanish-speaking communities.
Do they have a right to conduct their lives, to have an elementary school teaching in the language that the children know and speak?
Of course, they have to be assimilated into the general society.
But these are not incompatible goals.
In fact, we know very well that a multilingual environment is actually stimulating.
It doesn't impede.
It improves performance and understanding, and being able to live in your own mind certainly has that effect, too.
So when I grew up-- not me personality, but the communities that I was part of, the native language was Yiddish.
And they spoke Yiddish, and that was fine.
And when the kids went to school, they picked up English.
From the very beginning of this course is designed for this semester, we wanted to connect the humanities, the classroom, with social justice.
And luckily, we had guest speakers who themselves have been engaged in social change.
So for example, I mentioned before John Turman.
There was also Professor Fox Harrell.
In fact, most of our guest speakers have been engaged in real life trying to make change for social justice.
And because some of our students were themselves children of immigrants, many of them got very interested, for example, in the issue of migration.
Or to protect the rights of migrants who had been under attack in the recent couple of years.
Thanks to John Turman, some of the students connected to the MIRA group, which is the Massachusetts Immigrant and Refugee Advocacy coalition.
And they were actually very happy to do so, to be able to connect not only the personal biographies, but also the classroom work with actual action for social change.
And we also got to understand the ways in which what we are studying in this course-- for example, the issue of language and assimilation-- have impacts, say, in the courtroom.
So the issue of language and assimilation was particularly evident in the court case of George Zimmerman.
The trial of-- Zimmerman was accused of killing Trayvon Martin.
This young black boy who was seen in his neighborhood after seeing his dad and he was-- after he bought some Skittles and some drinks and he was killed by this vigilante.
And the reason why he was acquitted is because the star witness for the prosecution, Rachel Jeantel, who happened to be this young woman of Haitian and Dominican decent, and because of the way she spoke, she spoke with a black accent.
And that accent, undermined her credibility.
In fact, some of the jurors were clear that they would not believe her because she sounded-- because of the way she spoke.
And there's a beautiful analysis of this episode-- a very sad episode-- by John Rickford and Sharese King in the journal, the journal Language, where they showed very clearly that that was the main factor for Rachel Jeantel to not be believed by the jury.
So here this is a clear case where attitudes towards language have an impact on social justice.
So why the personal in this course?
Well, at every occasion when the students present their discussions-- because they often have a friendly discussion-- or when they write their essays, I do ask them to include some personal information in what they write.
In fact, one essay was strictly about that, about their linguistic biographies.
Because I feel that often students can be overly ambitious.
Of course, there's this earnestness that we appreciate, but often they want to go after global changes like changing the environment or changing the outcome of elections.
But those are the kind of changes that you often cannot make, especially not in a one semester course.
But if they are able to understand the way those global patterns affect their own personal lives in their own small social groupings, and if they can actually try to integrate what they learn in the course to actions in these localized spheres, then there's a better chance that they can actually make those changes.
So I like to tell them that the change that they can make in themselves resolving those issues are also affecting the wider world.
Because if each person can affect themselves in their small groups and, in turn, if everyone in that small group can also affect changes in their own small groups, then the effect is exponential.
You see?
And I think by the end of the course they do appreciate being able to share their personal experiences through essays, through their presentations.
And also they can see the effect that this has on their own classmates, because some of their classmates come from very different backgrounds.
So they get to understand how someone, say, from Guatemala who came to the US with parents that emigrated without papers and how that life experience can be inspiring, because if you can make it to MIT with that kind of background and then you can actually share it with others while you are actually succeeding as an MIT student, that can be life changing for some of us.
So critical computational empowerment.
That's one of the key concepts that Professor Fox Harrell, who was one of our guest speakers, discussed during his visit to class.
And it was a big hit in the class.
Because, as you know, we have many engineers, scientists in the course.
And many of them are so used to the notion of coding, of computation, being supposedly neutral when it comes to ideology.
And to the societal tendencies.
But what Fox should really clearly is that those hierarchies of power, that this course is about, really, are themselves embedded in these codes.
In these codes that create video games, virtual reality platforms.
So if we're not aware of these hierarchies, and how they are coded-- really coded-- in those platforms, then we become prey to them without even knowing.
We accept their assumptions, and we reinforce our own attitudes that might put down women, or immigrants, or Muslims, or gay people.
So we will become part of these hierarchies and maintaining them.
But also what Fox showed in his visit is that, once you understand how coding can encode those hierarchies, then we can actually create codes that will, in turn, give us more flexibility for identities that are more inclusive.
And more humane towards, especially, women, and blacks, and Latinos, and Muslims, et cetera.
So the one thing that Fox did in his visit was to show us one of his platforms where he brings together people on different sides of a conflict.
So one case that he showed was this conflict between the Israelis and the Palestinians.
So there's this perform where you can bring those soldiers together, who fight on opposite sides, and through that, they get to better empathize with each other's identities, and biographies, and life circumstances.
So that was really a powerful concept for the students to get to understand.
How coding can indeed help us code a better world for all.
So at the end of the course, what I've been wanting-- and I think that was achieved for the greater part-- is to have students become very analytical when it comes to the use of language, race, religion, gender, as technologies, technologies to create hierarchies of power.
But if you think of something that Steven Biko once said, Biko once said that the most efficient tool of the opressor is the mind of the oppressed.
So after the students to understand how when you grow up in a society where there are various hierarchies that are imposed on that society based on tools, like how we perceive race, how we perceive language varieties, how we perceive gender, how we perceive religion, and we ourselves-- both myself as a faculty of MIT, but also the students, are students coming from different social groupings-- we bring those hierarchies with us.
And often, it's implicit-- not aware of that.
So at the end of the course, one thing which became clear to me is that the students became more aware of how these cultural phenomena and the use of them and the position of them are used as tools to maintain those hierarchies.
So what became clear to me is that towards the end of the course, the students were indeed very self-conscious of the uses of culture, of race, of religion, of issues about migration in creating those walls.
And I think now they have tools that they can use to break those walls within themselves, but also within the social groups.
OK.
So the course is, at its core, very intensive learning, so every single text that we read includes at least two disciplines.
So when we talk about, say, linguistics, we look at linguistics from a very historical perspective, but also from a very social perspective.
So basically, we look at language in society.
We don't just focus on English, per se.
And we do the same with writing, with literature, with anthropology.
There's always a larger lens that brings in at least two or three disciplines.
So-- and plus we also bring in guest speakers from various disciplines having to do with the international studies, anthropology, history, of course, linguistics, my own field.
And also, the students themselves, they come from interdisciplinary backgrounds.
So some of them are scientists, but others do women's studies, anthropology, biology.
So we all, I think, in the course, we're primed to look at the issues of blacks in the US from a very interdisciplinary perspective.
From the student perspective, if we think of this notion of interdisciplinary, I think many of them come from the science, for example, so they're engineers-- I think we have mostly engineers in the course, and we have also some scientists biology, brain and cognitive science.
And in these fields, often, they look at particular problems from very-- relatively narrow perspective.
So I think that, for many of them, indeed, it may have been the first course they took that exposed them to a broader range of fields within one course.
The humanities, to me, particularly at MIT, are essential, indispensable, to help break those infamous walls and build bridges.
Why?
Because the humanities are the only disciplines of philosophy, psychology, linguistics, history, writing, women and gender studies, that can give us the tools to understand how phenomena connected to race, to language, to religion, to gender, to sexual orientation, to have the tools to be able to understand how they can be used or misused for domination or for liberation.
This is the only group of studies, of disciplines, that give us those tools.
As Paulo Freire and [INAUDIBLE] said, we have to learn to become literate.
Not literate just being able to read, but being able to, as we, as Freire says, to read the world, and read the word.
Because as we read the word, we have to understand the impact of what we read on the way the world works.
And the way the world works often is based on these hierarchies of power that are often embedded in the text we read.
But we're not able to critically understand those texts and see how they create a myth, or maintain those hierarchies of power, then we're going to become part of these hierarchies We help maintain them.
So the humanities give us the tools to be critical about these hierarchies and be able to distance ourselves from them, and perchance to even undermine those hierarchies, and create more inclusive, and more-- kinder, more humane societies.
Yes, I do think that the Black Matters course has taken new urgency in this current climate.
If we think, for example, of the most recent elections, the campaigns that we all heard on TV, on the radio, in the newspapers, it's clear that identity politics played a key role.
So if you think of the various claims about immigrants, about Muslims, about race, about blacks, identity politics, I think, was a major focus of the campaign.
And I think the students themselves, when I look at-- when we started, I gave out the questionnaires asking them why they took the course, what drove them to Black Matters.
And many of them actually assume that the course is called Black Matters because of the Black Lives Matter movement.
But in fact, that course started 10 years ago, before there was a Black Lives Matter movement.
But within themselves, they came to the course wanting to address current political issues, including issues of race, of identity, of migration, of religious hatred.
So definitely, there was a sense of urgency coming especially from the students' perspective themselves.
They wanted to discuss these issues and get a better understanding of how to process, but also how to counter some of these claims about identity, race, religion, migration, etc.
So as I was thinking about the syllabus and the course description, I couldn't help but thinking about the slogan of building walls to protect, say, the US.
And as I was picking texts and choosing guest speakers, I realized that it was important to create a new narrative around this issue of walls, because I think that our humanity is in reach when we can build bridges instead of brick walls.
So for me, the course-- and also for some of my guest speakers-- it was important to create a new narrative around the concept of building bridges that indeed will make this world great, not just the US, but the world great.
You know, so why this hashtag for the course?
The hashtags, Stay Woke?
Well I love it, because in a way, it's one of these phrases that take a very complex concept, and that makes that accessible to the larger public.
Because Stay Woke is a way to talk about critical consciousness.
And what we've seen in the past elections is this increase of quote-unquote "fake news."
And one thing which I've been trying to share with the students is that this concept of fake news isn't at all a new phenomenon.
So in fact, if you look at the way history is taught in the US-- if you look at, say, Howard Zinn's book about history-- it tells us how our history books are filled with fake news.
So the point is that, it's not just fake news that's fake.
And what the students get to see through, for example, the Haitian revolution is that, in the textbooks, most of them-- actually, almost all of them, maybe, except for one or two-- they got to learn lots of details about the French Revolution.
And then the American Revolution.
So they know all the things about that.
But almost none of them knew about Haiti.
But if you think of what revolution means, this revolution has had much more impact when it comes to human rights, civil liberties, than both the US revolution and the French one.
And the question that I ask them is, why is it that they went through the elementary school, the high school, some of them most of the college years, without ever hearing about the Haitian revolution?
Right?
Which, in fact, it has been much more potent than both the French one and the US one.
And to me, that was a great way to introduce to them the idea that, you have to stay woke.
You have to understand why history is being taught in such a biased way, that excludes the one revolution that actually allowed this concept of equality among all human beings to become true.
Because both in France, and in the US, when they were claiming liberty, equality, fraternity, there were still people who were in chains.
You know, in fact, blacks were still in slavery.
Women couldn't vote.
It's only in Haiti that revolution took its really meaning in really changing the world order.
And in creating this possibility for all human beings to be equal.
But yet, it's not taught in the classroom in the US.
And to me, that's one of the most blatant cases of fake news that's become so much a part of the way children are educated in the US.
So Stay Woke means to be alert to this kind of propaganda that passes for history.
That's Stay Woke.
And of course, we look at other examples where they, themselves, participate in these hierarchies of knowledge that actually subserve hierarchies of power.
That's Stay Woke.
So if I were to teach this course to a group that would be much more diverse than what we have here at MIT-- because I think-- so when we taught this course this past Spring, most students came with a very open mind trying to understand the way, indeed, race and language create these hierarchies that are not good for the country, not good for the world, and how to create more inclusive societies.
But if I were to teach this course with a more diverse ideological set of students, what I think I would do is to include more texts from, say, a more conservative perspective.
In fact, some of the students often they would bring in clips from, say, Breitbart or from Rush Limbaugh, voices that are very conservative and some of them even on the outright extreme.
And we use that for teaching moments.
But if I were to teach this class maybe to a more diverse group, what I would do is to not take views so extreme, but take some scholarly work that could advocate, say, for building walls and then try to examine them together in a respectful way.
So that way even students on the right and on the left could have a respectful dialogue and try to look at the data that might support a particular view versus the other.
And that would be a challenge.
But I think eventually that would be a good challenge, because one common motif throughout the semester was how do we take what we learn here in this course and make it available to the wider public including those who might want to build walls and not bridges?
How do we translate all these very important knowledge of these important findings about migration, about equality?
How do we translate that in a way that it can be discussed in the wider sphere?
And this remains a challenge to the very end of the class.
There was no clear answer on how to do that.
But I would hope that if we were to teach to a broader audience, we would come up with the right way to do it.
So why was I so happy to engage in having this course be ported on OpenCourseWare?
Well, first of all, I have a very deep affection for OpenCourseWare because I feel that OpenCourseWare has this potential to teach to the billions, to share knowledge, to unlock knowledge for people in very far-flung places, very remote places, like in Haiti where there's little access to quality materials.
So for me, I feel that OpenCourseWare's mission is very germane to myself, as a Haitian, [INAUDIBLE] MIT, wanting to share this wealth that I have intellectually with the rest of the world, including my compatriots in Haiti who don't have that access, that much access, to quality materials.
But also, when it comes to this issue of breaking walls and building bridges, as we see, say, in the US and parts of Europe, the world moving to more exclusion, to more walls, I feel that the kind of knowledge that we're building here at MIT and how we as a people, we as a global village, as we enrich ourselves by understanding other people's experiences, be it Muslims, and immigrants, and blacks, and Latinos, it is wealth there.
So I feel that it's important to take this knowledge, these discussions, and have it shared with the whole world so people can see that there's a different way of thinking that doesn't involve building walls for security, but instead creating dialogues, building bridges, so that we all can be the richer for it wherever we are, be it in Boston, Massachusetts, or in Port-au-Prince Haiti, or in Syria, or even Iran.
So that's why I feel it's important that such courses be on OpenCourseWare for the world to participate in.
When I enrolled in this course, I expected it to be a traditional one professor, large population of students sort of dynamic in a seminar style.
But what I found was that we had half a dozen different professors or more over the course of the semester.
And I've never been in a class in that style before.
So it was a new environment for learning.
And I wasn't sure how what we learned in class day to day would be able to carry over in a cohesive way as we switched lecturers, but what I found was that a common thread was woven throughout the course of the semester, and that thread drew from diverse individuals who have expertise in diverse fields of study, and who have used that expertise in diverse ways to work and collaborate with, again, diverse groups of people throughout the world.
And I think that ended up being the central theme of the class, that the black experience, you know, this sort of overused phrase that we might hear just as colloquially is monolithic compared to the diversity of experiences that this class was able to demonstrate to each of us
Bouncing off of that, I think something that was also interesting was not only the multitude of voices in the professors, but the multitude of different voices within the student population.
When we were introducing ourselves, I think most of us are different majors, and we also come from different backgrounds and different ethnicities and different sexualities and genders.
And I think having such a group discussion with different professors allows you to get a lot of different perspectives, not only from the professorial side, but from the student experience and what you're experiencing here as different individuals
Yeah, I really appreciated the different courses of everyone in the class, but also in the professors, so some of our guest lecturers were from women's and gender studies, or CMS, which is comparative media studies, writing, literature, so we had all these different perspectives, like Fox Harrell's Phantasms and the virtual reality project, to Professor Helen Elaine Lee's writing workshops, and when we had to delve deep into what our parents taught us and how we were raised, and all these things.
And I think each of them brought such important facets to the conversation that we could explore within ourselves and within the outer world, and how black matters isn't just anthropology or just history or something, but it's all of these different things, which is why it's so important to the real engagement
Yeah, it was like each speaker brought a different view on emancipatory literacy to the class, and just showed us the different ways we could dissect everything we see
Also, I think hearing from the different professors also, as a sophomore, opens me up to taking more classes in different areas.
I didn't realize how many actual parts go into this class, because I remember when I was looking through the class list it's like, oh yeah, this counts for CI-H, HASS-A, like everything.
And I was like, why?
And then you realize, with all the different professors.
In thinking about what was the most impactful takeaway from this course, I really realized that the most important part for me was the self-reflection, and kind of delving into what I thought my racial identity was, as I am multiracial.
So I just kind of identify with different things at different times, and seeing how that actually fits my personality and what that means for myself.
What about you guys?
Yeah, going off of that, as another multiracial person, I started to realize as we engaged with the social scientific theory behind the way people code-switch, that there is a science behind it.
And it's not arbitrary.
It's not chaotic.
It's part of a larger social system with institutionalized expectations for the way people engage with one another and other institutions in place.
And understanding that we're sort of following by an unspoken set, and occasionally very vehemently spoken set of rules, when we do that-- when we identify differently in different circumstances-- was something I hadn't actually explicitly realized
Mhm
I agree that the self-exploration and the self-reflection was the most important takeaway, I think, that I could get personally from the course, in addition to all the academic readings that we did-- all of the engagement with the outside world that we did.
I think, in order to be able to change the world, first we have to understand ourselves.
And it helped me do all of that, and navigate the tensions between how I am read and how I am, and trying to figure out those differences
I think what I most enjoyed about the class was, outside of the self-awareness, increasingly reading into the patterns that we see in society, and how that influences how we see others.
A really impactful part was when we delved into why creole and other developing languages are considered to be lesser languages.
And how there wasn't really much science behind that.
It was more just cultural leading of whites being superior to blacks and the preservation of slavery through another form-- through both the disparagement of Creole and African-American English
Bouncing off that as well, I think another important part was, I mentioned before, the multiracial aspect.
But in class, Dana.
You were talking about how one of the more ant parts was also talking about how historically under-represented groups or-- I forgot the exact terminology you used-- can form alliances.
And how there are similarities, but also dissimilarities between how the groups work.
And I think classes like these allow you to discuss people in general who are in those situations, but also allows you delve into the nitty-gritty like you're talking about with Creole
Yeah, I was surprised how much a class entitled Black Matters dealt with things that you would never identify as a specifically black issue.
And we ended up talking about populations that are not read, or may not identify, as black.
And that was relevant.
These were not tangentially related political issues or social issues or economic injustices.
And I started to see how black issues and black matters and black experiences are really not something that can be studied in isolation, and play into larger global networks
Mhm
Yeah, because a lot of themes in the course were centered around the mechanisms of oppression instead of what the oppression of a particular group.
So it was really nice having those case studies inside of immigrant populations and American populations as well.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This class is called game design.
It is not an intro class.
How many of you have taken a CMS class before?
OK, about half of you.
All right, a number you haven't taken to CMS class before.
It's actually a prerequisite, but I can really take as many students as can fit in here.
So let me tell you why a prerequisite exists.
The prerequisite exists because in a CMS class we typically expect people to do a lot of writing, we expect people to expect their grades to depend on the quality of their writing.
We expect people to do all of the reading in class, and to be able to do it before class starts so that you can actually be prepared to have a conversation about that in class.
Different departments even, different humanities departments do things differently.
In CMS class, if you've taken any CMS class, none of this should sound surprising.
You know, you've probably written way more in other CMS classes than you would [INAUDIBLE] write in this class.
You're also expected to present.
You're expected to come up in front of the entire class and say, and talk about your project.
You should expect that your grades will depend on the quality of your presentation as well.
So, one example-- You might design a perfectly good game.
In fact an extremely elegant, beautiful game for the ages like Go or something like that, and you designed in this class, and that's great.
But if your grammar is terrible in your rules, even though I can't so read the rules and I can play the game, and it says, wow this is a really beautiful game, but your writing sucks you are going to lose grade [INAUDIBLE] based on that.
That's one of the things that you should expect.
There is a writing center over in building 12 I think, they are going to be moving in the summer, but right now they're still in building 12.
Definitely take advantage of them.
We are looking for concise writing, we are looking for precise writing, but we're not looking for, how should I put it, any more words than is necessary to get a point across.
We want things to be fast to read.
We want things to be easy for laypeople to read, not necessarily MIT students to read.
You are going to see examples, many, many examples of very poorly written rules in this -- of games that are actually very well made.
I actually kind of like this company here, HABA.
How many of you have heard of this company HABA?
They make little wooden trains-- toys, if you have any kid siblings, they may have gotten toys from this company.
They make games for kids, and not for children under three years.
That's the barrier.
And the rules are written for, like six different languages, but for kids.
They still expect parents to be able to explain how these games go, but you can imagine a six year-old picking up a game like this and being able to play it just fine.
And this is actually a really elegant game and will be taking a look a little bit closer at this later today.
That's kind of like where you should be aiming for with the quality of the writing.
Try to get your stuff as clearly as possible, as quickly as possible over to as low a reading level as possible.
There are a couple of tips that we have written, and that file is up on Stellar, and I'm handing that sheet out so you can go home and download the PDF yourself, and that's called the Rules Style Guide.
Do I have a link for that in here?
I don't have a link for that in here.
I'll just put it on top of the Stellar announcement page.
The Rules Style Guide is basically a two page document, that just gives you a bunch of tips on when to use bullet points versus numbered bullets for instance.
Things that you absolutely should include, like a list components in your game-- so that when I get it as an instructor at the end of the assignment I can actually make sure that everything in your game was provided to me, so that I can actually play the whole thing.
Give players a way to decide who goes first, don't just leave it up to the players, pick a player to go first.
Pick something silly like, person who grew up closest to the sea.
That's a real rule from [INAUDIBLE], by the way.
So, we do encourage inclusive language.
It can be a little bit more verbose, but instead of writing-- instead of describing all players as male, we do appreciate it when people say he or she.
It's fine to be able to flip between he and she, but, of course, not when you're describing the same player doing certain rule.
They're not changing their gender in the middle of the turn.
But the great way to be able to use language to describe player is player B.
You can also just say the player, the player, the player.
It is a little bit more verbose, but in that particular case there is a sort of higher goal in mind, that's to get across the point that games are for everyone.
Especially the game that you're going to be making in this class.
There's no particular reason why in this class, you should be making a game just for men to play, just for women to play.
And right now it is too easy to fall into the trap that you just generalize that all players are male, and that's just bad.
So don't do that.
In fact, if you take the effort to do that, you will see that reflected in your grade.
If you take the effort to use inclusive language.
Let's see, what else.
Attendance you have to attend every single class.
If you are sick, and especially sick with something contagious, or sick to the point where you can't move, say you broke something, and you really should be resting that's fine to the email that before class starts.
Now you know, if there was an accident or something and you had trouble getting to email that's fine, you can email us after.
It would be nice to be able get some evidence from your doctor, but if push comes to shove I'm not going to be too much of a stickler on that.
Things that I don't give automatic absences include being hauled away for a job interview, being stuck in an airport because you planed your spring break flight back too late or something like that.
You have three free absences basically.
You have three absences where you don't have to provide any excuse-- you don't have to tell me this is why you were gone.
And that's what I expect.
Now you can miss about three classes and I'm still quite satisfied that you've gotten everything that you need from this class.
Illnesses are one of the few things that are really out of control, and can hit people for an extended period of time, and that wasn't necessarily your fault.
Whereas something like being stuck in an airport usually only you for class, so it's not that bad.
How many of are seniors on the job interview track?
Save those three for things like job interviews, because you're going to need that.
And you don't have to ask for permission or anything-- Just go for your job interviews and realize that you have three of those.
If you do miss more than three classes, and it was not due to illnesses, then you may well get a reduction in your grades.
I have students drop by a full letter grade.
So they would have gotten a B, they got a C based on absences alone.
All of the assignments, told them that they were going to be B average, but their attendance was poor.
and as a result of that.
Usually missing four classes doesn't drop you a full letter grade, but it really depends on how close you were to the tip off, like a B-minus and a C-plus before your attendance into account.
There's a little bit more detail in the syllabus about how grades are computed.
Plagiarism is usually not so much of a problem when it comes to the group projects, because you're going to be writing a lot of your rules, and everyone is keeping an eye on what's actually being handed in for your assignments.
Keep in mind that I certainly don't want you to be copying wholesale from somebody else's rules.
That's more than plagiarism.
That might be copyright infringement.
But when it comes to your own writing, there is actually a one page personal report for every time you hand in a team assignment.
We are expecting you to keep a journal of every team meeting.
A personal journal about how the team meeting went, what decisions you made, you can write down things like, wow, only half the team turned up and that meeting wasn't really useful.
These are all useful things to keep in mind.
You get to choose what you are what you want to write down your one page report.
Again I'm looking for something that's well-written, reflective would be very nice.
Based on the experience of doing this team assignment, what will I do differently next time.
Or what went well that I want to make sure that I do again?
Those things are very good to put down in your one page report.
Write those on your own, don't go into it like a group writing session.
You can compare notes that's fine.
But when it comes down to writing it, I want you to write it on your own.
How many grad students in here?
One?
OK, so grad students, if you're signing up for 608, that's no additional work.
If you're signing up for 864, which gives you grad credit, then there is additional work.
That largely involves checking out the games one week in advance, making sure that they're all punched, making sure all the pieces are in there, and making sure you know the rules.
You don't necessarily have to learn the rules by reading the rules, you can't do it by watching play videos on YouTube.
However it takes to learn the game, you it in class and you teach your classmates how to play this game, so everyone can get started a little bit faster.
And make sure that all of the pieces are inside.
That being said, everyone should be looking at the rules of these games when they're in class.
You will see in the class schedule, I actually have all the games that we're going to be playing listed so that if the name of the game escapes you, you'll be able to Google it and look it up later.
And you should be look at the rules because you want to see how the rules are written-- how good rules are written, how bad rules are written.
And you're going to again see a lot of really bad rules.
And you want to understand how certain things are conveyed.
Maybe there's a really elegant game mechanic that you want to try to use a version of that in your own game, but you are having difficulty explaining it.
Well, look at the rules of how other board games do it, and see whether they do a better job of it.
Look at videos-- I'm one of the few people who definitely learn the games better by watching a video of somebody who's actually played it, or talking to somebody who's actually played it and having them shown to me.
I have trouble parsing rules myself, but that's why I can really appreciate well written rules when I see one.
So make sure that you are looking at the rules.
As I briefly mentioned earlier, all the assignments in this class are team projects.
There is again, that one page individual report that you hand in, usually the week after the assignment, and that one you write yourself, but it's all based on that team project.
Which means you're going to be working on teams typically of three to four people.
I do not recommend teams of five people in this class, simply because once you hit five you get into to certain communication overheads that makes it kind of counterproductive, so you can probably get about the same amount of work done with four people.
Sometimes of all five people are living in the same living group you might be able to make it work.
It's still really hard to schedule a meeting with five people all at once.
Class does end at 4 o'clock.
Usually that last hour is game playing time, which means you might be able to take a set amount of time to meet up with your team.
It's even better if the hour between 4 and 5 o'clock is all free for every single member of your team, but that's not going to be the case for every single team-- for every single project.
And you can switch teams between projects.
Assignment one can be with a completely different group of people from assignment and two, that's fine.
We're going to team formation, I think the schedule has it on Monday, yeah, because I'm not sure if this is the entire class yet.
So on Monday we'll actually go a little bit more into brainstorming, talking a little bit about what projects you want to do, assembling teams around that, and then you're off to the races.
You've only got three weeks to finish this first game.
Is it three weeks?
Yep, exactly three weeks, actually less than that because you start on the Monday and then hand in on that third Wednesday.
So a very short amount of time, but that's OK because this is supposed to be very quick and dirty first assignment, get it out of the way and get into the groove of things.
We're going to spend most of this month just talking about the discipline of making games.
What is iterative design, what is rapid prototyping, what is game testing, why is it important and why you're going to be doing this over, and over, and over, and over, and over again all semester long.
That's the one thing that I want you to walk away from class with, is being better at itterative design.
Who here is familiar with-- has practiced some sort of iterative design from any discipline?
How many of you are from an engineering background where you practiced that.
OK, just to get a sense which sort of project kind of work that you came from--
[INAUDIBLE]
[INAUDIBLE]?
[INAUDIBLE]
Yeah, creative video games
Building a robot that is [INAUDIBLE] competition
OK, yeah.
Others?
Software development
Software development
Creating the [INAUDIBLE]
Right, Yeah.
I'll just leave the computer game design classes, very relevant experience here.
This class is, of course, a good prerequisite for getting into CMS.611 which is our creating video games class.
That class is about teamwork, is a lot more about obviously software because it's a video games class.
That's going to be in the fall, so if you're interested after this class is over, you might want to check that one out, and you can talk to the students around here who've done it before.
By the way, for those people who've actually been in classes with me I haven't gotten contacts, I'm just blind right now.
My glasses snapped in half, and I have a three-year-old, things are happening.
Actually I was playing StarCraft and then [INAUDIBLE] The final thing that I want to talk about all the way at the end of the syllabus, is this thing called a change log.
Some of you have done CMS.611 recently will find this quite familiar.
This is again one per group, not one for person, but you hand this at the same time as you hand in each assignment.
And what I want you to do, is that you end up every time you have a design meeting, whether you meet in class, or you might outside of class on a weekend, even online or something, every time you make changes to your game write down why those changes written down in this format.
We give you the sample format, we expect you to stick to it.
You can tweak it, but we expect to see at least this much detail.
What are the actions do in your games, what are the goals of the player, or players of course.
What are the problems that your team is noticing after having played through this game?
Turn takes too long, no one knows what to do, game takes too long, games over to quickly, game seems too random and out of our control.
All those things, write down in problems.
These should be useful for you as it is for us.
If somebody misses a design meeting, you should be able to keep this on a Google Doc or some publicly shared document, and you can look at this, oh that's what happened last week.
Oh, you change the wind condition to half what it was.
OK I guess that's what we're working with now, why did you do that?
And it says, Oh, because the game was taking too long or something.
I expect to see that coming with your final assignment-- with each assignment as well.
Any questions so far?
There's going to be a bunch of guest lectures but we haven't labeled them in here because we haven't quite scheduled them just yet.
The readings are the same, the games are the same.
Some changes might happen to the schedule on Stellar.
I will obviously make announcements on Stellar, I'll make announcements in class, I'll try to update the document on Stellar with schedule changes.
But, for the most part, assignments are not going to change, just some readings may get rearranged.
I have to point out something about reading.
So, here are books that we're referencing in class.
There's a few individual essays, but you're not going to have to read every single one of these back to back.
For tabletop, this is actually available as a free PDF from the publisher, and the link of that is on Stellar.
So, you don't have to buy the book you just download the PDF, scroll down to the reading and just read that.
We have one reading from his book, but I actually recommend this book.
How many of you have heard of the New Games Movement?
New Games?
New Games.
It's from the '60s, so new is relative.
The New Games Movement was something that-- OK, how many of you played with a parachute in gym class when you were in elementary school?
OK, you were part of the New Games Movement.
This was basically an activist group of game designers in the '60s, heavily influenced by the Whole Earth Catalog if you've heard of that-- Stewart Brand, yeah.
Basically typical counterculture, but in the games side of things was very specifically about making games that feature different kinds of cooperative play.
If they were competitive they were sort of light hearted competitive, and we're still trying to develop a sense of community.
The parachute type games are very much about taking military hardware and turning them into peacetime fun activities.
The parachute is something that used to drop soldiers right?
It's a really neat book.
You can see the title of a Playful [INAUDIBLE] Home.
It could be some sort of New Age writing, but it's actually a pretty nice treatise on a particular kind of game design, which is how do you make games to bring people together, which is a perfectly fine goal when if you want to be a game designer.
So, check that out.
The Design of Everyday Things, you should graduate from MIT without having read this book.
If you design anything mechanical engineer, software, anything that's meant to be used by a human being.
Maybe some of you will work but robots might be able to skip this, but I would suggest not, I think you can read this too, and get a lot about usability, accessibility, how to make good user interfaces.
We have two readings from here, these are PDFs that are up on our Stellar site.
Let me leave these two follow-ups for last.
The Oxford History of Board Games-- we actually have a lot of readings of it from here because it's relevant to our final assignment.
And what it is, is a surprisingly readable list of traditional board games that maybe you've heard about 10% of these.
It's a really interesting analysis, and the reason why I prescribe this book is two reasons.
One, you need this background knowledge to be able to execute assignment three.
Second reason, is that it's a good example of what game historical analysis looks like.
So, if you're ever interested in going to academia and doing this sort of writing, you at least now have a sample of what that looks like.
It's not as dry and stodgy as you think, it is about game play after all.
But we do have a lot of readings, this book is available on reserve in the library.
We're going to try to get more scans of this up on Stellar.
I'm not entirely sure what I'm legally allowed to put that many scans up on Stellar.
The hard copy itself is there.
I don't think you can even buy this book anymore.
This book has been out of print for a long time, so I'm going to try and increase access to this, but the best I can do right now is that it's on reserve in Hayden Library I believe.
Finally, these three books, you might be familiar if you've ever taken any of the game lab classes-- Game Design Workshop, Challenges for Game Designers, and Rules of Play.
The reading for today is actually chapter one from Challenges for Game Designers.
All of these books are available online, through MIT Libraries web portal.
The trick is you need to be on a MIT subnet.
How many of you live on campus?
OK, you're not going to have a problem.
Just got our Stellar site, there is this thing called the books 24/7 link on the Stellar, you click on that first before you read any book, that basically does some authentication to make sure that you're actually an MIT student, and have a browser with cookies it should work just fine.
And then you click the book as you want to read, you'll get a table of contents, you jump to the chapter, [INAUDIBLE] reading of this book [INAUDIBLE].
However, if you're not on the MIT campus, I think you have a browser with cookies you're still fine.
But, you might have better luck by actually being on MIT subnet.
You do exactly the same steps that I said.
You go to Stellar, you click on the books 24/7 link, it authenticates you, and then you can read the book-- you can even print it out.
But, if you do have trouble reading it for any reason, please let me know because we're going to have a lot of reading from these three books over the entire semester, so let's try to fix that for you as quickly as possible, and then you won't have any trouble with that.
It's free, but it's a bit of a hassle.
That's basically it for the formalities.
Again, any questions about how the class is organized, how grades are assigned?
It is a project based class, which means your grades are going to be heavily dependent on the performance of your teammates, so get along with your teammates.
Try to do good work, but it's-- your grade is not going to be determined by how fun your game is.
This may be coming as a surprise to some of you.
Fun is one of those things that is really elusive, even if you know what you're trying to design for, you might not get there.
Especially in the constraints of doing three projects in one semester.
As someone who has designed a lot of games, both in MIT and outside, I fully acknowledge that you might be perfectly disciplined and do everything that I tell you and still end up with a very unfun game.
So I'm not going to grade you on the quality that.
I am going to grade you on your ability to listen to feedback, from feedback that provided, feedback that you're getting from the testers, your ability to do things like regular testing sessions, the ability to stick to the discipline of iterative design.
If your game is mostly not playable until the last week, and then suddenly you pull some sort of marathon effort to try to get everything working together in that last week, you should expect a good grade.
That's not what I'm trying to teach here.
You should be constantly iterating on these games every single week.
And if you do that you should count on a fairly reasonable, fairly good grade.
This is a B-average class though.
So, typically when I grade that's how it ends up.
I'm not actually grading on a curve, It is possible to have an A-average class if everybody does incredibly well, but typically, in the past couple of years it'll always be B-average.
Let me start with the easy question.
What are people playing nowadays?
What are people in this class playing, maybe not right now with this instant, but--
You mean like computer games?
[INAUDIBLE]
Any games, board games, card games, computer games,
League of Legends
League of Legends
I got a lot of people on [INAUDIBLE]
[INAUDIBLE]
On Ascension?
I was going to say Dominion

You should learn [INAUDIBLE].
Settlers of Catan.
Yeah
Settlers
[INAUDIBLE] So when you--
Adventure Quest?
[INAUDIBLE]
Wait, wait hold on.
What's Adventure Quest?
I haven't heard this one
It's this online game.
Yeah.
Always-- [INTERPOSING VOICES]
It's like--
I guess like, it's a lot of-0 [INTERPOSING VOICES]
So it's an adventure game.
[INAUDIBLE] Cool.
OK.
So we've got [INAUDIBLE] Adventure Quest.
That one I just [INAUDIBLE] to be trail
Smash [INAUDIBLE]
Wait, hold on which version of Smash?
I like Melee
[INAUDIBLE] to play Brawl
So Melee is GameCube right?
Yeah.
That was the first one I ever played.
Liars Poker
What?
Liars Poker?
All right
Normal Poker
Wait hold on.
What's normal poker?
I guess Texas hold'em
Texas hold'em poker.
OK, all right
Innovation?
The card game Innovation?
Actually made locally
[INAUDIBLE]
Yeah
Alien [INAUDIBLE]
Alien [INAUDIBLE].
I'm just not familiar with this one.
This is--
It's an out of print board game, Euro style.
[INAUDIBLE] Empire Builder, but you already have that on one of the [INAUDIBLE] that you own
I'm not saying, what games do I have.
[INTERPOSING VOICES]
I know, but I want that game, so I'm [INAUDIBLE]
Why won't you try it out?
Zombie Apocalypse
Zombie Apocalypse
Oh yeah.
Munchkin?
[INAUDIBLE] One day I'll be able to add Nymph to this list
What's Nymph?
The board game that came out on Kickstarter [INAUDIBLE].
It's going to be delivered sometime this month.
That's one of the most exciting
I missed something after Zombie Apocalypse
Munchkin.
RoboRally
RoboRally, wow.
How did people get exposed to RoboRally, just out of curiosity?
I saw it at an internship.
They just had a bunch of board games on the wall

My brother played it somewhere, and then I bought it for my [INAUDIBLE]
Very MIT-ish game, it's a game about programing [LAUGHTER]
Programming and robots beating each other up
Cards Against Humanity
Hex Hex
I know Hex one, but what type of Hex?
Hex Hex is a game where someone casts a hex, and then everyone has a certain number of cards that just tell you what to do with the hex, like pass it left, and pass it right.
And then who ever has the hex in front of them has to deal with hex as one of their cards.
If they can't then they're hexed, and then they chance to play cards that say, play one hex
I hope that's a hexagon involved in there
Yes
OK [INAUDIBLE] [LAUGHTER]
We didn't play [INAUDIBLE]
[INAUDIBLE] Same people that did Innovation.
[INAUDIBLE] There's a game called We Didn't Playtest This At All, and We Didn't Playtest This Either, right?
I want to try those.
Air Baron
What?
Air Baron
Care Bear?
[LAUGHTER]
A-I-R, air, as in the sky, baron, as in not a noble or the lords of a baron
Oh, Air Baron
Yes
We should make a game called Air Bear.
[LAUGHTER]
This is digital?
No, Avalon Hill, 1997
Oh,
Yeah.
Yeah
This is like a Mafia like game right?
Yeah, love that game.
Cout
Clue?
Cout.
C-O-U-
C-
U-
Oh yeah
[INAUDIBLE] It's kind of like Mafia, except instead of having night and day papers, what they have is people take turns.
They have two different roles, and your role lets you take actions.
So what you do is you lie about what role you have in order to take actions that you wouldn't have access to.
Overall that mechanics are, you make money, and then you pay money to kill other people.
When you kill someone they lose one of their roles.
The way you lose rolls is killing someone, or by lying and being caught, or by challenge someone and being wrong.
Like Mascarade
[INAUDIBLE] Say what?
It's like Mascarade
OK, Yeah.
Actually, we could add Mascarade to this here as well.
So far I asked a couple of people to describe the games that have come out.
And you can see it's not really all that difficult to say this is what's interesting about the game, it's kind of like this game then it's got these changes.
That's a normal way of communicating.
You're going to have to do that multiple times this semester for your own games.
But, often what happens is that when students come up and talk about their game, they just start from step one-- Well, this game is a card game that has 52 cards, and is going to be played with four players, and it's like, can you like, cut to the chase and a little bit faster?
The way that you just.
The way you just described, you know it's kind of like Mafia but you make these changes.
We will want those details especially when it's written, or when we're actually sitting down to play.
But when you're coming up here and talking about your own game, give us those elevated pitches that you just did.
If you think of this game as a little bit like chess, but with different pieces or something like that.
And then you cut in, and then you can go into greater detail about what your game is.
When you give a presentation though, usually what I'm listening for isn't so much, what is your game right now, because I can actually read that.
What was your game before it became the thing that is now?
Because I don't necessarily always see that.
I'll see various play tests, but I may not necessarily get to every single thing.
I want to know how you made the decisions that you did in order to end up with what you finally submitted.
And I'll get all the details that I need from the final submission when I actually read it.
So when you give your presentations, talk about the history of your game, three week history of your game, or the four week history of the game.
So, these are the games that people have been playing.
A lot of board games, couple of digital games although I think I'm seeing-- how many of you are playing these games digitally?
Any of these board games playing in digitally?
[INAUDIBLE]
Yeah.
On the phone, yeah?
I think we [INAUDIBLE] Dominion web application
Web app?
[INTERPOSING VOICES]
Your hand was up, yes?
Ascension, I think it's on the iPhone now
Yeah, a lot of these games actually do have iPhone and iOS [INAUDIBLE], Android mode
[INAUDIBLE] not fun at all
Oh, because you don't-- yeah.
Why?
More fun to yell at people
Because it's more fun to yell at people?
It's more fun to guilt people into giving you trades that are beneficial to you.
We often refer to that as over the table talk.
Or, the over the table interaction.
That's something that when you're designing a board game or a card game, or even a live action game, I want to say that you're allowed to design live action games in this class.
Games where you're not actually sitting down, but you're actually moving around.
Mafia kind of sits in a gray area
I think Risk is the perfect example of that.
Because video Risk is an entirely different than board Risk because you can't cheat.
Cheating is an integral part of Risk
Describe the tactics of cheating in--
You hide cards in your shoe.
You move people's armies to places that they weren't on before
That one I've seen
[INAUDIBLE] people to your places
But not like re-rolls, right?
Because those are hard to get away with
Monopoly is the worst game ever
Because of the cheating, or because of the way how the game is made?
Because cheating.
And because it takes like 12 hours to play.
[INTERPOSING VOICES]
--you get the best game, right?
Play Riskopoly
[INAUDIBLE].
[INTERPOSING VOICES]
I think that's it.
Well, OK.
So let's briefly talk about Monopoly then.
How many people here like playing Monopoly?
[INAUDIBLE]
How many play Monopoly regularly, like with families?
[INAUDIBLE]
What are the good things about Monopoly?
What do people like about Monopoly?
Making bank
OK. All right.
There's this real sense of progression there, right?
I like putting four houses on one property
That's kind of like the real estate version of making bank.
Sure
When you have a long row of properties all in a row and somebody's like heading into them.
And you're like, OK, you're not going to get through here without [INAUDIBLE]
Those little plastic bits mean everything when--
It's family friendly.
So people like it.
The reason people like is that a little kid can just win.
Can do really well and win [INAUDIBLE]
That's kind of like a minimum age where they get the basic strategy, and then past that point, it's kind of like anyone's game
Anecdote.
I played with my family once.
The first and last time I'll ever play with them.
My sister, she's a year younger than me, and for some reason always has it out for me.
And so she was about to go bankrupt and my dad made a deal with her, basically.
So any time you make a profit, you give me two thirds of it and I'll make sure you never go out.
I was like, now I'm playing against two people.
I don't know what I'm supposed to do anymore
That situation has been described as king-making, where a player who isn't going to win decides who's going to win.
And it can be sucky if they're not the person who--
Basically
I actually really like-- kind of what he said, but in a different way, that it seems like anyone can win.
But the more you know about the game, the more you actually start to understand not anyone can win.
And in any game where you're rolling a thousand times, there's very little luck involved
It all averages out
So yeah, it does allow you to implement a lot of strategy, even though just about anyone can play it and understand the rules [INAUDIBLE]
How many of you play Monopoly with sort of over-the-table bargaining?
All the time
All the time?
Yep
Who doesn't?
Like only the rules?
Only the things that--
I do only the rules
You do only the rules.
But do the other people around the table also do only the rules, or will they play--
[INAUDIBLE]
Things like, I was going to give you money, things like that
They always make me offers that I don't want.
I'm just like, skip it
There is something that a lot of people do in Monopoly and that's put money in Free Parking
Yep.
Have to
Never do that.
Never do that.
Game gets four times longer.
Never do that because the game is designed to-- the rules of the game at least was designed to suck money out of the system.
There are inflows-- things like the collect $200.
Things like the certain Community Chest and Chance cards.
But the game is also designed to take money out so that people go bankrupt.
By saying, I'm now losing this money, but I'm now putting it on this square.
And if anybody happens to land on free parking, they get all this money.
That money hasn't actually been sucked out of the system.
It's just waiting to be redistributed to somebody else.
And that means people take a very long time to go bankrupt.
And if everyone's complaining about how long the game takes, you might want to check if you're actually playing with that rule.
And if you are and you read the rules, it's not there.
It's not there at all.
It's the sort of thing though that does make it a lot easier for somebody who is about to go bankrupt to get back into the game.
So now we're back to this whole family-friendly thing, which is-- if you are not that great at playing this game, or maybe even taking it that seriously, you still stand a chance.
There's a couple of other things about Monopoly.
It's a decent game if you're not at the table all the time.
Say, at Thanksgiving or something, and somebody is watching the game or making sure that the turkey doesn't catch on fire, or something.
That needs to stand up and check, and then come back to the game.
Everything that you need to know about the game is right there in front of you at the moment of time when you need to make your decision.
There's very little-- I don't think there's any hidden information, except for the amount of money that people have.
And that's usually fairly easy to just see.
Some people get very secretive, but most people don't.
Monopoly is an incredibly successful game.
I don't think this is a contentious issue.
It makes a lot of money.
A lot of copies get sold and bought.
And there was some ambivalence here about whether it was a good game or not.
Why does that game do well, do so commercially well?
Because it's easy and well-known
Is it easy?
Sorry, easy to pick up.
Not necessarily easy to win.
You can take 10, 15 minutes to explain it, and then you play a game
10, 15 minutes to explain if you already know the rules
If one person knows.
Just explaining to another person
But chances are somebody does know the rules.
Somebody in that room probably has played Monopoly
Because it's so well-known
Because it's so well-known.
[INTERPOSING VOICES]
It's sort of like the classic that-- it's an old game [INAUDIBLE] other games.
Also, it's easy to play because you don't make decisions, really
Yeah
It's easy to play without making a decision.
Like, roll the dice and move.
Roll the dice and move
If you can afford to buy it, buy it.
That's the decision-making.
It's not a decision.
It's strategy
I hate it when people make long decisions, like do I buy it, for like 15 minutes
And it's like, no.
If you can afford it, buy it.
All right.
So there's a lot to do in the game, though.
You roll the dice.
You pull cards out.
You move your little dog.
You can make bargains with people over the table.
You pick your little houses and you put the down and you turn into hotels.
So there's a lot of busy work that you get to do without having to make a single decision.
So you feel very active while you're doing it.
Any other ideas about why it might be popular?
We already talked about how well-known it is
They sell different versions of the same game.
If you want one that's Star Wars-themed, [INAUDIBLE]
Trivially reskinnable.
I have Monopoly Singapore somewhere.
It's where I'm from.
They just renamed all the properties.
All the amounts of money are exactly the same as your standard Monopoly.
I think there are some more clever versions of Monopoly out there where you add in a few new rules.
But you're right, you can easily sell somebody who has a monopoly a different copy of Monopoly.
But is that person buying a new copy of Monopoly?
If you already have a copy of Monopoly, are you buying yourself a new copy of Monopoly?
I guess you're buying yourself a new skin of Monopoly
Are you?
Does anyone actually pay money for that?
No
I think you get it
They're good gifts
They're a good gift.
That's the dirty, little secret about all these Parker Brothers, Hasbro.
Maybe it's not dirty.
It's the reality.
The majority of board games, these super, mega, ultra perennial favorites are not bought by the people who are playing them.
They are bought by somebody else or given as a gift.
Which means you can expect that they usually sell most of the copies at Christmas.
And people might get them for their birthdays.
And that's the difference between a game that shows up in Target and Wal-Mart and a game that shows up in a speciality board game store.
You walk into a specialty board game store, which is probably under threat by companies like Amazon, and you're buying a game for yourself.
You're buying special dice because your role playing game, game player or miniatures.
And right next to that are the board games.
And you say, hey, this will be fun for my next gathering, either at some friend's house or at my own place, or something like that.
I want to play this game.
You buy a Monopoly and it's usually not for your own collection.
It's usually for somebody else.
So you give Monopoly to somebody else who already has it.
And it's fine because you just gave them the Star Wars version and they didn't have that.
That's how they're successful.
I guess what I'm trying to say is not necessarily-- it doesn't necessarily have anything to do with how well the game is designed.
There are a couple of nice things that Monopoly does.
We talked about how it's relatively easy to learn.
The fact that it's been around makes it very easy to teach.
It's very friendly to people who are playing on some sort of [INAUDIBLE] schedule.
But that's not the reason that [INAUDIBLE] game
So I think a lot of it comes from like the snowball effect.
Everyone knows about [INAUDIBLE].
If we assume that's true, how do you then make a commercially successful board game?
How much marketing money do you have?
I mean, we've got Blokus here.
That's probably one of the more successful recent entries.
And there's a couple of ways to be able to get a large number of families to buy a board game.
If you're living in Germany, there is a big conference called the Essen Spiel, I believe.
I might be mangling the language here in the town of Essen.
There are smaller conferences in places like Leipzig.
But what they are-- in addition to the vendors bringing all of their old and the new board games saying like, this is what we have on sale.
We have tons and tons and tons of families going there trying to decide what they're buying for Christmas.
And in Germany, that is kind of like an annual tradition for families to buy a Christmas game.
Built on top of that, there is this thing-- I'm going to mangle all the German in here
BoardGameGeek
The [NON-ENGLISH]
[INAUDIBLE]
Oh.
Yeah, OK. Basically, game of the year award that is awarded, I believe, at Essen, they definitely do a big thing.
I'm not exactly sure how the timing lines up.
But definitely, the game that happens to be awarded game of the year is going to be the biggest mover at a convention like-- convention is not the right word I am looking for.
Its more like a trade fair.
That's probably a good example.
And just lots and lots of families will go there.
They will buy a lot of the game of the year because a bunch of board game critics said, this is the best new game that came out this past year.
And then, all that money gets circulated back into marketing.
You see more ads.
You see the awards sticker on the side of the boxes.
More families buy it.
And that just snowballs on top of each other, right?
So winning sort of similar things in the US I believe there are--
BoardGameGeek
ABC News
BoardGameGeek
Yeah, they have their own con [INAUDIBLE]
Yeah.
But not that many families.
I mean, I'm thinking more like ABC News doing a top 10 hot toys for this Christmas kind of thing.
That's the sort of thing that bumps you up from a game that you buy for yourself to a game you start buying for other people, which multiplies how many copies actually get sold.
So that's one way.
That's usually one way that it happens.
That's not to say that a game that-- what's called an enthusiast board game.
A game that's basically sold to people who are buying it for themselves can't be commercially successful.
It just means that you need to make enough money that you made a profit on it.
And a lot of these games-- I think of all the games that I've got today, let's see.
This is probably-- let me see if this won any awards [INAUDIBLE].
This is probably the sort of game that you would buy for families.
This is Set.
And of course, missing it's original box.
But you have played Set, right?
OK.
So this probably fits into the category of a game that somebody in the room has probably played once and can teach everybody else how to play.
Who's played Blokus before today?
Yep.
OK. How many people played with your family?
Before you came to-- how many of you played before you came to MIT?
OK. High school?
Middle school
Middle school?
Yeah.
So I think the reason for it-- this is Mattel.
So they certainly know their marketing.
But I think the reason why this got popular was again, one of those top 10 toys to buy that Christmas when it was released.
And that's why Blokus really took off.
It's a well-designed game.
You don't get on those lists without being a well-designed game.
But that's not the reason why these games are successful.
That's kind of the bare minimum to even be considered [INAUDIBLE].
So if you're in this class thinking that you're going to make a lot of money as a board game designer, I know people in Hasbro I can get you in touch with.
And I think they're probably going to say don't expect that.
You might be able to have a decent living working in a large company like Hasbro on Mattel, which I think is the same company, yeah.
But it's probably not because of the quality of the games that you're designing, but more about your ability to design a game based on s spec that was handed to you.
Based on a requirement.
It's like, we need a Star Wars Risk game by Christmas.
Design it in two months, or something like that.
And you execute that well and you will get a good career in a place like Hasbro.
If you're designing for yourself, you should feel very happy if you manage to make your initial investment back.
A lot of people who are sort of hobbyist game designers who make incredibly good games don't make back the money that they initially invested into it.
And that's fine if you know-- if you went into that business knowing that.
Because for some people, it's just good enough to be able to have a game with your name on it being played by a whole bunch of people who appreciate it.
That's fine.
More likely, a lot of people are going into careers, I'm assuming, that are not game design.
Some of you are.
But some of you will go into a career that might benefit from some of the design practices that you're going to learn in this class.
Not just iterative design.
Although, we've already talked about how that's relevant to a number of engineering disciplines.
But also, things like you want to teach people a new system that you've designed, or a new system that's been put into practice.
You want to create some sort of team building exercise.
You want to communicate to someone how complex a certain system is that's relevant to you through your work and your discipline.
There's a bunch of things that games can do to communicate ideas that might be relevant in whatever career you end up in.
Even here in MIT, I'm the Creative Director of the MIT Game Lab.
There's a bunch of different labs here in MIT who are using games for a variety of different purposes.
Eyewire.
Anyone heard of that one?
Yep
I was in [INAUDIBLE] class
Yeah.
Over in the [INAUDIBLE] Institute right across the street where the train tracks run under the building, they've got a game that's basically massively multiplayer, massively single player really, and you solve the puzzle which is kind of like [INAUDIBLE].
It's kind of like paint by numbers.
You're clicking and you're coloring a field.
What you're really doing is you're identifying neurons from scanning tunneling microscopes.
These are neurons from a mouse retina.
And you're basically doing all of the pattern recognition that computers have a tremendously difficult time to do, but humans actually are pretty good at.
And then, you're accumulating all of that data.
It's a fairly nice, very clicky Minesweeper-ish kind of feel to this game.
It's like you're just clicking things so that you find-- they meet a nice contiguous space.
It's a 3D game, So there's some sort of like mental hoops that your brain gets through.
And kind of nice build, puzzle feeling.
But what you're really doing is you're providing Sebastian Seung's lab with a ton of data about how a mouse's eye is wired, which is amazing.
And I think they've got a couple of papers.
So try it.
It's real fun.
Anyone here from systems, systems dynamics?
No?
Engineering.
I their parent group is the AeroAstro department.
[BACKGROUND NOISE]
OK. Do you think that's a kid out there?
I wonder if they're walking away or not
Just like a three-year-old or something, that's fine.
But I hope it's not someone who's hurt.
But a lot of system dynamics folks on campus, engineering systems design for instance, will use games to try to teach clients about certain engineering practices.
So there's a group here that has a game called Space Tug--
Space Tug Skirmish?
Yeah, skirmish
I've played that.
My roommate worked there last summer
Yeah.
At UROP?
Yeah
And I think they're still looking for more UROPs because they've turned their board game into a digital game and they need to keep working on it
[INAUDIBLE]
Yeah.
It's kind of got a-- I was going to say a Magic: The Gathering feel, but that's not really true anymore, the way how the game has evolved.
But it's primarily a card game about building satellites, launching them into space.
And then realizing that by the time you launch something into space, it's kind of obsolete.
And the question is, how do you design a satellite so that it can adapt to changing circumstances?
That's what they are teaching people from NASA, from the military about.
And they decided that games are the way to do it.
It started with a board game.
They're going to digital game.
It's a very large group here on campus called the Scheller Teacher Education Program.
And they design games for K-12 to freshmen undergraduate-level science, math, language.
How to better address some problems in curriculum, in sub-standard classroom curriculum.
So they've done MMOs.
They've done puzzle games.
They've done a lot of stuff on Flash because it's very easy to get Flash running in a school.
It's very much harder to get a whole school [INAUDIBLE] to download a piece of software and install it across all the computers.
All these places have UROPs.
My lab has UROPs.
So if you're interested in any of these things, you should definitely check them out.
You can ask me for more leads.
There are a lot of different.
And finally, back to AeroAstro.
There is a group that works with zero gravity satellites.
[INAUDIBLE] talking about.
They're satellites.
They work in zero gravity
SPHERES?
SPHERES, yeah.
These little compressed, I think, carbon dioxide propelled SPHERES.
They look like gigantic dice, actually.
And they spin around in space.
And they'll run C code.
And they have a high school programming competition, kind of like the FIRST robotics tournaments.
Think of that in zero gravity.
So they have you accomplish things.
It's a pretty cool high school education thing to connect high school kids to space, basically.
I know they are looking for UROPs.
So if you're interested in that, you should definitely talk to me about that.
I'll give you the contact information.
So I've been yammering for about an hour.
Let's take a break.
Stretch.
I'll start distributing some of these games.
So we have one more discussion topic before we go into game planning.
Some people should probably distribute [INAUDIBLE] as well
Where's the nearest [INAUDIBLE]?
Bathrooms are out to the right.
[INTERPOSING VOICES]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's do a bit of a discussion, we've got about 20 minutes.
So these are the games that we just played.
And I just want to have a discussion, again.
What are the mechanics in these games?
We've been talking-- I've gone to a lot of teams to ask about core mechanics, because that's really the thing that you're going to have to worry about for assignment one.
Well let's talk a little bit about mechanics in general.
Again, the definition that I want to work with-- at least for assignment one-- is a set of rules, could be more than one-- usually more than one-- that is going to allow a player to change the game state.
Now I never defined game state.
But anyone want to throw out how you would interpret game state?
Like all the public and private information on the board right now
OK. All right, that's-- all the information, whether you know it or not, that could be the [INAUDIBLE]
Yeah, for me, it's affecting other players
Affecting other players?
Yeah [INAUDIBLE]
It's-- all the informa-- what, what affecting other players?
We played [?
unavailable ?]
with them, or--
The decisions are available to--
Sure
--to them could be one way to interpret it.
OK, all right.
I thought I saw another hand
I was thinking, I don't know how to explain it, but, yes, you're having a board or whatever type of meeting that you have issue to change
Right
We go from, I don't know, something should move or if something doesn't move, then there should be a reason why it didn't move
OK.
So, one way that I interpret that is that there has to be a variable.
It has to be something that isn't necessarily the same every single turn, right?
It's like the board of the Monopoly-- the layout of a Monopoly board-- It's not what I would describe as being something, part of the game state, because that never changes from turn to turn
So if you were to take someone elsewhere, say, across the country.
Some groups of people have the same [?
baggage ?]
as the group that's playing their game.
And to send-- has each one of you [?
sent ?]
one telegraph-- telegram-- to one of them?
If the minimum amount you need, the minimum stuff you need to tell them in order for them to be able to continue your game from where you were
OK, so you've got to evaluation criteria of whether you just fully described the game play.
But I'll, yeah, all these are useful ways of thinking about what is the game state.
What's the stuff that can change?
Which is-- and what's all the information?
Whether you know it, whether an individual player knows it or not.
To be able to reproduce a state-- to be able to reproduce a game in progress and then be able to carry on, right.
Sometimes things are unreproducible, like sports, for instance.
If you try to halt the game halfway and then reproduce exactly the same weather conditions as the game it was-- [INTERPOSING VOICES]
It was altered.
It is kind of difficult.
But you know that if you really, really wanted to be able to continue that you have to make a decision on whether the weather is part of the game state.
In sports, often it's the reason why the game was interrupted in the first place
Do you think [INAUDIBLE] sports fatigue is part of the game today?
Yeah, I think so, which makes it kind of difficult, right?
It's, like, we're going to start the game at halftime, and then we'll pick this game up tomorrow, playing the second half.
And it's-- that's a very different game from the typical one.
[INTERPOSING VOICES]
Instead, make it so that they have to start [?
A ?]
different game, right, for the first half
First, exhaust yourself, or we'll throw those results out.
So-- so [INAUDIBLE] [INTERPOSING VOICES]
do something else with them.
[INTERPOSING VOICES]
Well, I mean, that's a problem.
If you actually had-- and this does happen in, not marathons, but multi-day races.
You have the situation where people get a good night's sleep before they resume the second leg of the race.
And at which point they decided that fatigue is not the thing that they're going for.
Why, they've tried to quantify your advantage or your disadvantage based on your start time the next day or something like that, right?
I try to think up a competition that that would advance to that.
I think that-- the Tour de France is not one day, right?
It's multiple days.
Yeah.
So you have to stop.
You have to sleep.
You have to wake up, and then, depending on when you've reached the checkpoint, determines when you get to leave the check-- the-- yeah
[INAUDIBLE] I know, in cricket, they have multiple day competitions.
And near the end of the one day, it could be dark [?
for the gingham and like ?]
[?
clothlike ?]
The game doesn't get finished for the day, and they'll put in-- they'll specifically put in $15 [INAUDIBLE], basically trying to delay until the next game starts there
Oh
They put in--
So that their rules will sort of counteract these exploitative strategies
They'll [INAUDIBLE] put in a pitcher [?
who's younger.
?]
They'll put in a batter who's job is to not really swing it very-- isn't really to swing it very much, and all they're to do is trying to delay until the game ends for the day, basically
Yeah.
I mean, it's weird, because you have a bunch of these strategies, and that there are probably rules that will come up to prevent some of the worst strategies from being put into play.
You've got the same thing in baseball.
Sometimes you put in these-- that there are late ending pitchers, right?
That you will, and they're your starters and-- that try to achieve completely different things.
So we've talked a little bit about game state, and we've played all of these games.
All of you played a good portion of them and I have gone to team to team to try to talk a little bit about the mechanics.
So let's just pick one that not that many people played so we can talk a little bit about it.
A lot of people, I think [?
Marcus ?]
will [?
state ?]
once today, so by this group, right?
So game state is characterized by--
[INAUDIBLE] location of pieces
Yeah, and the location of pieces onto the great board.
But there is-- I think every player has the same set of pieces, just in a different color.
And which pieces have already been played.
So it's not just the location, but also which pieces have been played in which teams, which people are still available to them.
Whose turn it is, because it's part of the game state.
Because you take turns facing people, right?
So-- so what's the core mechanic of [INAUDIBLE]
[INAUDIBLE]
You select a piece from all of the pieces you haven't placed yet.
And then you figure out where it goes.
And the rule that you have to fit to meet, before you put it down?
[INAUDIBLE] the corner of the piece [INAUDIBLE]
A corner of one of your own pieces.
Right
But not in any [INAUDIBLE]
Right.
So you can't two pieces butting up like that, but you can have two corners touching each other.
And you have to have corner pieces
And it is better where [INAUDIBLE] the board, where the [INAUDIBLE] except you can't put it [INAUDIBLE]
You tried to find a little space to place, with a open space, that your piece will actually fit.
Out of all of the pieces that you've got, which is part of the game state.
And, of course, that changes game state by not only changing what pieces are on the board, as we described earlier, but also takes away a piece that you've already got.
That's the core mechanic.
In, I think, the next class, let me just check-- take a look at this again.
Oh, wow, it's going to be February 19.
No, it's going to be, holy cow, it's going to be a month before we get to chapter 2 of this book.
Seriously?
OK. For your sanity, you might want to read chapter 2 of Challenges for Game Designers, because it's five pages.
It's not much.
It's pages with illustrations on them.
They go fast.
Chapter 2 goes into a lot of more specific definitions.
But they also talk about something called core dynamic, which is something I actually don't necessarily want you to worry too much about right now.
They are right in identifying that, often, the core dynamics are actually more important than the core mechanics of the game.
So the core dynamic probably has a bigger influence on what a player is going to experience.
Whether it's some sort of crazy frenetic game where you're trying to screw over your opponent, or when it is animal versus-- animal upon animal, where it almost feels like a cooperative game at times.
Even though there's a winner, you're all kind of playing together to not take the whole thing over.
Those things often come out of the dynamics.
We'll go into the theory.
Some of you have already encountered this in other classes, the mechanics, dynamics, aesthetics theory.
We'll get into that later into this semester.
And that's just a very convoluted way of explaining why sometimes game design is hard.
But the mechanic is the thing that you get to control as a designer.
You get to write the rules.
You get to decide what collection of rules that the players have to go through in order to be able to change the state of the game.
And I want you to go through deep, deep permutations of what you can possibly do with one core mechanic.
So in [?
Escape ?]
it's very clearly rated [?
easier.
?]
He's not a game designer.
In fact, there's a little bio of him in Chapter 1, I think.
Yeah.
In the chapter there's a little photo of him, the guy who designed that game.
And it's very clearly him trying to do everything that he can do with dice in a short design period of time.
And when you see his other games later this semester, you will see that he's trying to do everything he can to with options.
He's trying to do everything that you can do with [?
design, ?]
and he's thinking about these game mechanics.
And that's the thought process [?
I need ?]
to go through.
All right.
So leaves us with just a little bit more time.
I'm going to end this with the stupidest question in the world, which is, what is a game.
What is a game?
We've got 3 hours into a class, and we haven't actually talked about this yet.
Why?
Why?
OK, maybe [INAUDIBLE] why is it a stupid question?
It's not a stupid question
Is it?
Why did I describe it as a stupid question, maybe.
That might be--
It feels kind of intuitive.
Oh, I can look at something and say this is a game, this is not a game.
But it's actually-- describe-- maybe there's some [?
educations ?]
where it gets really difficult and then to actually define what is and what isn't gets really murky
That is definitely-- that is definitely true.
But you have a counterpoint?
[?
Game ?]
but usually something with a, I guess, set toll in mind given, [?
I don't know, ?]
to build objective and rule
Objectives, rules, objectives or sub-objectives, goals, as you describe them.
Some sort of constraints, some--
Like a mechanical restraint.
Like an objective and mechanical [INAUDIBLE] I consider it a stupid question because it is.
Everybody knows that a game [INAUDIBLE] that's a ridiculous thing to say.
Of course I want a game.
But when we actually look at it?
Then it does.
It gets hard.
It gets convoluted as you're playing.
That's why it's such a dumb question
Again, somebody [?
educate-- ?]
so many [?
educations, ?]
but there are these things that a lot of games do share.
These goals that you're working towards, these constraints that you are trying to work in, these decisions they're trying to make
They're definitely [INAUDIBLE] game, which have had objectives in them.
[INAUDIBLE] objective, and then charge [INAUDIBLE] 10 [INAUDIBLE] The way that the [INAUDIBLE] is different, I don't think a game necessarily has to have [INAUDIBLE]
It's crazy to think of a game that people might recognize as a game that doesn't meet all of these criteria
So I think of it as something with a goal, an entertaining goal.
So basically a goal that would be entertaining to obtain.
But I also think that's very dependent on what you think of as a game.
And someone mentioned that, oh, it's obvious.
If you look at a game [?
you feel like ?]
this is a game.
It's not.
But I think that it's not as obvious.
I think with animal upon animal, everyone of us can look at that and say it's the game, but there are a lot of things that kind of fall on the edge.
Some people will define this game where other people won't.
[?
Beat the ?]
[?
game.
?]
[LAUGHTER] Or my dad's really angry with me-- [INTERPOSING VOICES]
--cards
I personally like the vision of unnecessary obstacles, because it-- There's a play space that you don't have to be in it, but you should clean it and you should be giving yourself these obstacles to get over them or whatever
I think that's Bernard Suits who posited that and he likes to use golf as an example.
Right, it's because clearly golf is an inefficient way of delivering a ball into a cup
[LAUGHTER]
Yes, let's just use this very long stick that's weighted weirdly and then put the ball really far away.
And give you a rule that you can't just pick it up and drop it in there.
So unnecessary obstacles, it's kind of a nice way to describe a criteria that a game could meet
So it's interesting because the-- often times a lot of people, the creative people like artists and other such people, will say constraints actually let you be more creative.
So that's, I think what's interesting about games is, specifically with strategies and stuff, the fact that you can't just pick up the golf ball and walk to the hole and put it in.
It means you actually have to learn.
Well, this golf club, it goes for long distances and this and that and that.
So all these constraints yield different strategies, which is a really interesting sight
Right and even in the example of the cricket thing, right.
It's-- you have this weird collection, a set of rules.
We need to this emergent strategy, which is like-- And now we put in this particular kind of batter or pitcher that is going to help us maximize our-- what's going to extend this game by day.
So [INAUDIBLE] why is-- it's not entirely-- it's not an entirely futile experiment to try to define games.
In fact, I believe we do have a definition of games coming up in one of the rules of play readings, and it is one that is functional.
And it has goals.
It has the constraints that you're working in.
I believe they also have it describing an activity that's actually carved out of-- of regular life.
And the consequences that happen inside it don't include that you're outside it.
But as many people have already pointed out in class, if I gave you a little bit of time you will be able to come up with a game that people will recognize as a game, and for that [?
sort ?]
[?
of ?]
definition.
The reason why I personally-- this is not-- this is not going to be useful to you outside of this class.
So I'm going to tell you personally the reason why I don't like that question, what is a game, is because it invites you to carve things out of the game space.
To say that thing is not a game because it doesn't meet this definition, and I find that activity to be pointless.
To-- just telling people who've made something and have decided to call it a game that it is not-- that thing that you created is not a game.
I actually feel that is very antisocial, very noninclusive way of thinking about what could actually be useful in thinking about it.
But for your own benefit of working in a team, it is reasonable to set goals that you want to hit with you game.
Many of the descriptions that have already come up in the past 10 minutes are actually things I would use to describe good games, but not necessarily games as a whole.
Entertaining, for instance, it's like how many of you can think of a game that's not entertaining, but it's a game?
Monopoly?
[LAUGHTER]
Sure
The game
What else?
War and [?
golf are.
?]
War.
Golf.
It depends on you, right?
It depends on the player.
Actually, a lot depends on the player.
You can come up with the worst game in the world.
Conversely, you can come up with the worst game and, well, and find people who would enjoy it.
And you can make it a game that's trying to deliver a particular message.
And people are going to walk away with completely the wrong interpretation of it.
And that's fine, because they are players.
They-- I'll say it's not even a completely wrong interpretation.
They come up with a valid interpretation that they walked away with it, and you had very little control over that.
But it's OK, because the game's more about players usually than about the designers.
Even though you see the designers names on many of these boxes, a lot of them would have that, I think.
In the end, there are relatively few famous game designers out there, but there are many, many famous game players out there.
I think there's a good reason for that.
All the creative-- all the creativity that comes out from constraints, all the, as we're talking about sports, the athleticism.
The ability to work within these constraints to do something that people didn't necessarily think that was possible is usually the Hallmark card of a great player.
The designers just provide it, the sandbox for them to be able to express that way.
So I think when you come back on Monday, to be able to come in with a concept after taking a look at some of the readings.
This is what I want to [?
hit, ?]
right, and I want to have a game that-- that has a goal.
Or I want to have a game where the players have a goal.
I want to have a game where the players are going to express a certain amount of creativity.
Or whether it's-- they're really just sort of mechanically going through the possibility space.
I think we've got a couple of games that actually were very mechanical.
And you're are just stepping through the-- stepping through the [?
paces.
?]
[?
Some ?]
[?
are ?]
[?
closer, ?]
and some of them is almost a little bit like that, right, because you don't get to decide what question you ask, I believe
[?
You just draw ?]
the question
Yeah, you just draw it.
You ask the question.
You get information, and, at some point of time, you get all the information you need.
So-- so that's a game that has very relatively little decision making on the player's part, if you are playing optimally.
If you're not playing optimally, then actually you're making a whole bunch of incorrect decisions that actually makes it a little bit more variable in who's going to win.
And that's why the game's actually interesting.
Come in on Monday.
Be willing to discuss that.
Come in on Monday.
Be willing to change your mind when you actually meet up with your team.
We're going to do a little bit of brainstorming on Monday on what sort of games that you want to work on, and, more importantly, what mechanics you want to work on.
One more clarification about the first assignment.
It is not about the story of your game or the fiction of your game or where your game is set, in ancient history or in the science, sci-fi or anything of that.
We'll get to that later on in this lecture.
I want you to think about what are the rules that a player needs to deal with.
Right?
You are your own target audience, as far as I know.
That's it for class today.
Thank you very much.
Please make sure that you did sign in, if you haven't gotten the attendance sheet.
[?
Be officially ?]
on it.
[INTERPOSING VOICES]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Today I've got quite a lot of stuff to go through.
So hopefully we'll get to the actual game playing, but if not, don't worry, you'll actually get to play the games-- the games that we'll end up playing today are games from last year.
A lot of them are actually the final semester-- the final project, which means they are not answering the same question that you're trying to answer with your very first project.
But it should give you a sense of what the scope of this class actually is.
Last week, a couple of you played the LEGO game, Block Party.
That was a two week project, and that was just without staff.
These were more like four week projects.
And we'll design my students with larger teams and probably closer to the kind of game that you're going to end up making.
I want you to keep in mind that you're not constrained to building games that look exactly like those.
In fact, after you've got a chance to play some of the games from last year, I hope you actually have a chance to think a little bit about what did they not do.
All these games are like this sort of board game.
And you will see a lot of them are this sort of board game.
You find out what this is.
And you'll be able to do-- you can think a little bit about what would it be like to design a game that's not that.
What if you just wanted to make your car game, for instance?
You can.
That's no reason why you couldn't.
What if you want to do a live action game where you actually move around with your body, that sort of thing?
That's totally doable.
I really, really hesitate not-- try not to put any live digital components in your game.
I know some people-- unless it's something simple like a timer that you can run off your iPhone.
OK, maybe that's fine.
By actually writing code, then the scope of this project just went through what I expected you to do.
And the possibility of failure is-- go back please
Very much
But not failing the class.
But your project just failing during runtime.
So that's probably something you don't necessarily want to do.
We had two readings today, one for last Wednesday and then one for today.
So I want to-- I try to cover quite a bit of it from the first reading during class itself.
Was anyone here not here on Wednesday?
OK, all right.
So we need to make sure that your name's on the attendance sheet.
And you should come and talk to me after class.
And we'll make sure you get a copy of the syllabus and everything like that.
Make sure that you get expectations for this class.
Everybody who was here, hopefully you found your name on the attendance sheet because I had to manually add a few of you.
So I hope you got-- I got it all right.
Now one of the points that I try to get across-- is this recording?
Yeah.
One of the points that I tried to get across on Wednesday was-- all right, who's the most important person when it comes to actually playing a game?
(COLLECTIVELY) The players
The players, right, not the designers, not you.
But you when you actually play someone else's game.
Then you're the most important person.
If that person's experience is problematic or exciting or engaged or outright hostile to other players, or something like that, that's making it the way that they want to take it.
And then you as the designer are are going to run through a number of different challenges trying to be able to give them an experience that you're trying to create for them.
But you also have to acknowledge that if somebody wants to take your hardcore strategy game and turn it into a lighthearted party game, or vice versa, that's their prerogative.
And if they say, hey, I really, really wanted this to be a lighthearted game, and you gave me this hardcore game.
And it made me hate the people around me, the table.
And I don't like it, that is fine.
It's OK for them to not like your game based on my criteria like that.
However, one of the things that comes up in the Brathwaite reading is this concept of meaningful decisions, right?
Well, what's-- Sid Meire how many of you heard of Sid Meier?
Yeah, what games can people remember from--
(COLLECTIVELY) Civilization
OK, Civilization.
Railroads?
I think he also did the first [INAUDIBLE].
Pirates?
Yeah, yeah.
Mostly a computer game designer, although if you actually play his games, if you like digital board games a lot, especially Civilization.
And he has a code, which just says games are a series of interesting decisions.
Not necessarily meaningful but interesting-- the decisions.
And what are some of the ways that a decision can be meaningful?
What are some of the things that might come up in the reading or occur to you right now?
That a decision in a game can be meaningful-- what does it mean?
What does it mean to be meaningful?
Couple hands?
When you make that decision, the game's status changes or code that meanders the change?
OK, so when you make a decision, and then the outcome actually has changed.
So the corollary is that if you make a decision and the outcome hasn't changed, it wouldn't terribly be meaningful.
OK?
That's what I was going to say.
[INAUDIBLE] a lot like doing something and, I don't know, getting money [INAUDIBLE]
So getting money?
So getting some sort of quantitative reward based on the decision that you've made
Also you can get a [INAUDIBLE] actually good to wear or not.
It could be something really awesome in the game, and you make a decision--
So kudos of the [INAUDIBLE].
It's like, wow, that was an amazing-- like in football, for instance.
That was an amazing play.
It didn't necessarily complete, but it would have been awesome.
But maybe it was an awesome defense or something like that.
That could be a meaningful decision.
How about if you decide to roll a die?
Game state has changed, usually.
Is that a meaningful thing?
It could be if it's [INAUDIBLE].
You roll a dice in one direction of the game where you not [INAUDIBLE] different.
That's getting [INAUDIBLE].
It's like, do I roll the die now, or do I wait a minute before I take my turn.
That's not really a meaningful decision
OK, it's like, let me think about this.
Then I'll roll it eventually.
All right, someone had-- I'm going to go this way
So if you had meaningful alternatives as well?
OK, so again, if you had the choice not to roll the die, OK?
I would say most games that the die roll would be meaningful but not a decision.
So the die roll basically tells you what to do.
You're not actually thinking about it.
But still [INAUDIBLE].
You can still change the game state and that two wins or whatever happens to me
So the outcome of the die roll is usually meaningful, but you may not have had to decide to do that.
[INAUDIBLE]
To go with that.
In a game like Yahtzee where-- do you roll the dice you're trying to make room.
But it's about the die rolling where it's like something not [INAUDIBLE].
It's not really a decision.
You have to do it every turn
Mm-hmm.
Yeah, and that's an interesting thing.
I talked a little bit about mechanics on Wednesday and the idea that if you think of a mechanic as something a player does to change the game state, the roll die-- the die rolling thing is a weird thing because gameplay that's changing is something a player does but a player didn't decide how to do that, right?
And as [INAUDIBLE]
Well, they've covered [INAUDIBLE]
OK, yeah that's good
We can choose different behavioral [INAUDIBLE]
Oh yeah, like getting Yahtzee-- which die, right?
Yeah?
I learned this definition of [INAUDIBLE] series decisions, like the card game War, for instance, or the board game Candy Land
That's a version of Candy Land which lets you choose which pile of red cards to draw from.
But is that meaningful?
A stack of randomly shuffled cards?
Yeah, it has to do with [INAUDIBLE]
If their random pile [INAUDIBLE]
[INAUDIBLE]
I feel you something else that you wanted to add on to that
No, it's OK.
These are these things are in game.
And--
[INAUDIBLE]
The thing is that they might be games but that particular decision may not be the meaningful one.
And it's possible that in the entire game, maybe you don't get that many meaningful decisions.
That doesn't necessarily make it not a game.
It might make it not a very good game, which is a different thing.
That's a qualitative and often subjective experience, right?
So it's real and make you awesome game.
You get to play in a land of candy.
A few more hands [INAUDIBLE]?
Yeah, on that topic.
I think there's a reason why people who over a certain age never really wanted to play Candy Land anymore because they realized that they don't really do anything, so they get really bored with it.
For kids or some other targeted thing going on more than the meaningful decision, making it fun to actually just doing something and moving-- or winning cards.
[INAUDIBLE]
The thing that war and Candy Land let you do-- it's, for instance, it gives a five-year-old a winning chance against an adult, right?
That could be huge for a kid.
It's like, wow, I can actually play this with an adult?
Learning how to take turns is one thing that people have actually get good at the games like Candy Land.
I can't remember if I brought this up last time.
I believe Candy Land was invented to keep kids from getting polio from each other.
That might be urban legend
[INAUDIBLE]
Huh, it's true?
It keeps kids indoors
Keeps kids indoors and try to get them from their--
--they're working with other kids.
There's polio going around at the time it happened.
If you stay indoors with the people you already know you know aren't infected, you're going to be
So, OK, that was--
With polio, you can't do much of anything
There's a huge inversion from the get out and get some exercise.
It's a serious game.
It has [INAUDIBLE].
Let's play in the land of Candy.
I would like to go back a little bit to this idea about changing the game state, right?
You make a decision in a game, and you've changed the game state.
And it wasn't a decision-- it wasn't-- I get to try to roll from three identical dice.
Which die do I roll, OK?
All right, that's not real-- the real decision.
I [INAUDIBLE] loaded and then identical.
But anyway, so what do you need to be able to communicate to the player that their decision actually changed anything?
OK, actually let me flip that around.
If you don't let the player know what changed the game state, even though the game state might have changed, is it that meaningful a decision anymore?
You did something.
Some numbers changed inside the system.
It's good to affect how things go out later.
But you don't actually know what happened.
Has anyone played a game like this?
Does that sound familiar?
I feel it's very much potential to player and the system because a lot of times something like that will happen and whatever most people say, oh, nothing really changed.
My decision didn't matter at all.
Whereas someone that takes the game more seriously might actually realize something changed just from [INAUDIBLE] looking out for something ridiculous.
And also if you expect players to get very into your game, this is the stuff that you can leave with because they will understand it and figure it out on their own.
And that has to be part of the game.
Whereas if you expect players to pick it up and play it three times in their life and then move on, then it probably won't help them that much and you should you should probably give them more free time [INAUDIBLE]
Yep, some other hand?
I think, also, if you have an adventure game where there's a story or something.
And you tell some guys to just not help the guy.
And then he becomes an evil warlord later, there are some interesting things you can do with that
There's a long term consequences, so a lot of immediate consequences.
Immediate feedback, which is a term that you brought up
Looking at-- there are certainly games where [INAUDIBLE] example.
The minions, oftentimes you can draw cards that don't help you to [INAUDIBLE] causing [INAUDIBLE].
But normally if I were to do this without thinking and thinking doesn't matter.
Would it actually be slightly beneficial or slightly [INAUDIBLE] that actually do this?
But unless you really think about it, you won't even notice that it caused any sort of change there.
Because if you [INAUDIBLE] this winter game wherein-- there's a point where you must-- if there's a dog, and you need to give it a sandwich.
Otherwise you'd do it yourself 200 turns later, an hour earlier in this game
Right, and that hurts, right?
What's the game called?
The Hitchhiker's Guide to the Galaxy text adventure
Another really straightforward example is competitive games where you get only some of the [INAUDIBLE] information, like Starcraft or something.
[INAUDIBLE]
Yeah, it's interesting because usually a [INAUDIBLE] changing what your opponent does based on what you can't see.
And I say usually because it does happen.
Occasionally, your opponent sees something that you didn't see.
And they changed their strategy based on that.
I've played a number of adventure games that actually have this more as a puzzle solving thing.
So it's not like 200 turns later something changes.
It's like something changes right away, and you need to figure out what changed.
So it's not immediate feedback.
There's feedback somewhere in the world.
But I want to say I probably didn't enjoyed those games.
Maybe someone does.
Something [INAUDIBLE]
Well, you have to be careful that it doesn't have too much complexity or some sort of unanticipated change that there's two people the learning curve [INAUDIBLE].
Certain games you enjoy it, but you play it and it takes two hours.
And you realize something you did [INAUDIBLE] you might not want to play again
Yeah, it's like--
[INAUDIBLE]
--I got the wrong ending because I made this decision five hours ago or something like that.
OK, how about the alternative, when something-- my game state changes, and you're not really sure how your decision came up with that outcome.
So not random exactly, but overly complex maybe
Well, you don't end up with a good mental model of how the game works.
And so even though you make meaningful decisions, you don't know what the meaning is.
And so any time you're making another decision, you may be able to select, actually choose in a way that you want to
Yep, OK, mhm
It prevents a chess style of approaching the game.
Right, in chess you see the board and you-- a lot of the really good players will see X number of moves in advance.
But when things happen, they just randomly appear.
You have to react on the fly.
You can't just plan out all of your moves
This-- we [INAUDIBLE]
I think this one's a lot bigger issue than the other one because this could lead to a lot of frustration with players.
If they can see the game state changing and know that they're effective but don't know how they're affecting it, I just feel as a player, that would frustrate you to no end.
And that would make you think that everything you do doesn't really matter and that you're going to win or lose regardless of what you're doing.
And it turns it into a Candy Land game even if--
Mm-hmm,
[INAUDIBLE] the game-- occasionally I've played [INAUDIBLE] that are hard to understand.
And they're [INAUDIBLE] anything intuitive like [INAUDIBLE].
We sort of did stuff, but we had-- and then afterwards, we didn't really understand.
We didn't understand how we were changing it, but it took awhile before you really-- you don't understand what's happening.
And there are many games where you can do very poorly or very well.
And it takes a while to understand exactly why you're doing very poorly or very well
So sometimes even success can be bewildering, right.
Yeah?
I feel like this sort of [INAUDIBLE] from a digital game and a board game because in a digital game, usually people have to take their turns and have to play out more slowly, whereas in a good board game, you change something and-- if you do something and something changes, maybe you can just start over and do it again like it's a different thing
All right, hold on.
So in a digital game?
Yeah, [INAUDIBLE]
OK, so in a digital game, because some games let you save state and then reload state.
So you can say, well, what if I tried this decision?
And then there's an interesting phenomenon that goes with that in a lot of strategy games called save scrubbing.
And that is where you know that the outcome of something is based on a probability.
And so if you save and reload often enough, you will always be successful.
And that's an interesting strategy.
This is completely aside from what I wanted talked about today.
But it's an interesting strategy that game designers use.
And that is to-- in a digital game, save some random number seed at the time when the save is made so that outcome's always the same.
I expect tech support calls when people when you implement that, though, because people are like this-- your random number is broken.
I tried this thing 15 times.
And it's supposed to have a 90% success rate.
And I always fail when I do this one attack.
And it's like, yes, because you saved the random number seed.
So that is actually a problem because people have this concept about how the mechanics of what does 90% probability of success mean?
And [INAUDIBLE] specifically call right now.
A second one might say that.
90% probability of success to a lot of people means that most-- means that it's going to succeed.
Now for a lot of people, probably a lot of people in this room, this means you will probably succeed.
But for a game whose pushed the random number seed has been saved and you fail and then you reload from the start, it means you will always fail because that's a different concept of how the random number generator works.
It makes a lot of sense to people who are game designers or computer scientists but may not make a lot of sense to a lot of players.
I get to re-roll again when I save in those, right?
But now you get to re-roll exactly the same time, exactly the same way, which means you've got to come.
So this brings me to the second reading, which is Don Norman's first chapter in the design of everyday things.
The book used to be called The Psychology of Everyday Things, which had a nice little acronym him of POET.
But then people had trouble finding the book because he was looking for design books.
And it was shelved in the psychology section.
So he changed the name to The Design of Everyday Things, which I think is a really interesting application of the kinds of things that he's talking about.
Do you expect to find this book somewhere and is not in that section?
So you make a change and you iterate on it.
And if you look at the copyright, it actually still Psychology of Everyday Things.
He talks about visibility.
And what does visibility do in a syst-- in a design?
It helps people understand the qualities of what they're doing or what [INAUDIBLE]
Yeah, they have an intent, right?
They want to accomplish something.
And it gives them a clue on what they could do to accomplish that.
So already, that's a direct application to games, right?
You have a goal in mind.
It's like I want to accumulate more cash in this game.
I want to-- I'll produce my opponent or something like that.
All right, what are all the things that I can do?
What's telling me might-- what's in front of me right now?
Maybe in a board game, maybe in a card game, maybe in a visual game, that's telling me visibly, right now, that this might be the way I get to do that.
In a lot of strategy games, some of these things are very literal, right?
So and so technology gives you this bonus.
It's very-- I guess a little of the role playing games also have this, right?
I want to hit things harder.
Oh, look, this thing just needs plus one to attack.
All right, OK, that that's a very literal thing now.
How much effort you need to do, you need to go through to actually find that piece of information and whether plus one is actually a meaningful difference at all.
It's depending on the design of the game.
But visibility has something to do with the intent of the player of the user in Norman's case.
But he's not talking about games.
He's talking about the design of everything.
And what the system can actually do, the actual operations of the system.
So I'm playing a game like-- I'm playing a real time strategy game, and I have a tank.
And I want to make it do doughnuts.
And it's like, well, the system doesn't actually support that.
There's no physics simulations to this tank.
So I need to be able to convey to the player that this is not something you can actually do in the game.
You can tell which hex to move your tank, but that's it.
Maybe the tank doesn't even turn, right?
So the other thing is this concept of mapping, that there is this-- again, it has to do with the player's intent of what they want to do, what they want to accomplish.
But mapping, instead of the actual operations of the system, actually has to do it for what you can see of the system.
So there are affordances and there are constraints.
These are both words that are introduced in that reading.
I think affordances is introduced in this reading.
What's an example of an affordance?
If you can sit on a chair?
You can sit on a chair?
What about this thing tells you that you can sit on it?
It seems sturdy, and it's got a place for your butt
Yeah, it's got a nice little butt-shaped thing, here right?
It's not made of spikes.
It's got at least three legs, which may help, and evenly distributed, which means that it's not going to tip over
You can also measure it related to other objects to [INAUDIBLE] also.
And that you--
OK, there's a little bit of cultural familiarity with other chairs that you've seen
What about a handle on a door?
A handle on a door.
What does a handle on a door allow you to do?
It's like-- it's a place for your hand
It's hand-shaped.
It's--
If you hold your hand out in a natural way, you grab-- looking at the text.
Oh, I wasn't thinking of that handle.
I was thinking of the vertical [INAUDIBLE]
The one that looks like a U-shaped tube-- like that
Yeah, exactly
So yeah, it's kind the right shape.
It's one of those door handles or something like the size of a supporting column.
You wouldn't necessarily think that you had to grab it, right?
An outlet is for anything [INAUDIBLE]
Like fingers?
They're not finger-shaped
Because they're not finger-shaped.
Yeah, although it has a kind of weird happy face on it, which I always thought was a little bit strange about-- that might have been designed, too, actually.
That might have something because if you-- this won't kill you, really.
So you should put it in your home.
I want to find out more about the history of the [INAUDIBLE]
It's actually-- the design with the ground on the bottom is a bad idea because if something starts falling out that are too exposed to it, it will not be the ground one.
So it would be better to flip it
Where's the ground in the mid- up top?
Yeah, because then even if it starts coming out a little, then something falls down between the--
So it's bad design.
It's bad design because you want to put it on your wall in a way that it smiles at you all the time.
But when it's the other way around, it's actually a little more stable.
It's actually like a mo-- British plugs, actually, have the pin usually on top and--
The British have the [INAUDIBLE] pin.
There's [INAUDIBLE].
Right?
That would be the-- what would it be?
[INAUDIBLE]
For two round circles
Yeah, yeah, the two round circles, yeah
They're using--
Dan, I think you're thinking of your pins, which are round.
The original size is flat.
But they are twice the width of the thing.
You could stick your finger in a British pin, which means they have to design all kinds of protection mechanisms, plastic springs, and things like that, just to be able to prevent you from sticking your finger in it.
It does have-- yeah, this one's actually nice because the biggest hole in there is the lethal hole in there, is the one that doesn't have any current running through it.
So an affordance is something which suggests this is how you can use it, right?
Something that you can get hand around suggests you can grab it.
Something with a movable hinge suggests that's the direction that you move it in.
So they talk about materials like wood and glass, right?
Glass is for looking through.
Glass is for smashing.
Wood is for holding things together.
And wood is possibly for writing.
You've got this porous material.
It paints very easily, that sort of thing.
And in games-- let me just bring all the [INAUDIBLE] forward.
Let's try to identify the components of-- how many of you have played put before?
Really, really-- this is the box.
Let's talk about things in here that's a rule sheet.
I guess it affords reading, but I'm not going to talk about.
It has this thing.
Actually, what you do with this thing?
Ring it
Slap?
[BELL RING] Yeah, that's what it does.
OK, all right, so now that you've seen what this thing does.
What is one of the things that this-- not in the rule book [INAUDIBLE] completely.
If you have this in your game, what is this thing good for?
Getting attention
Getting attention, yeah.
It's loud
Annoying people?
Annoying people.
You could use it to annoy people.
It's like-- you want to lock out what they're trying to say or something and just keep hitting it
They knowing that you're the one that [INAUDIBLE]
[INAUDIBLE]
OK, yeah, so the completion of something, the end of-- because it's very progressive, right?
It's not just a loud-- like an air horn.
It goes eh.
This one actually has-- [BELL RING] --very, very sharp [INAUDIBLE] sound.
What else?
What else about this?
It's shiny
It is shiny.
Makes you want it, right?
You can hold it in your hand.
And so you might be able to completely pass it around
You could pass it around.
This could be controlled by different people.
It doesn't necessarily-- it's not a huge thing for anyone
There's only one of them
There's only one.
If you only had one, so then that becomes even more desirable because it's the only-- [LAUGHING]
[INAUDIBLE]
Nice
Something that may might be able to stand nicely on a flat surface.
But it's not rubberised or anything of that.
So if you put it on an incline surface or something like that, it not stop sliding.
So this implies that it's going to sit on a table somewhere.
It also comes with a bunch of cards.
Ooh, whoa.
What happened to these cards?
OK, now the paint could rubbed off or something that.
So don't worry too much about the text and the graphics.
But just look at the card.
What do cards allow you to do?
What are the affordances of cards?
Hm?
You can hold a couple of them in your hand
You can hold a couple of them in your hand at once,
[INAUDIBLE] because they're [INAUDIBLE]
Yeah, there's a site that you can put no useful information on, right?
Besides the brand of the game, sure.
But all identical, so you don't know what it is
[INAUDIBLE] each other
You can hold a lot of them in your hand at once.
What else?
What else about this?
Pattern recognition [INAUDIBLE]
The way how they've been designed makes it possible for you to do pattern recognition.
Something which they didn't do is use different colors, which might make it even easier, at least for people who are not colorblind.
But they've arranged similar elements in the same place.
And they didn't use colors to denote the numbers.
What else about these cards?
Just cards in general-- you can change them
Yeah, I guess that's up to you.
And in fact, Pit does that.
Pit's one of those games where exchange is a real time thing.
We're probably play it later in the semester because that's all light.
That's so easy.
Yeah?
It's easy to have a deck and then draw from it
Oh yeah, you can have this randomizing thing where you just have a whole stack sitting on the table.
And then you don't know what you're going to get.
And you just grab one.
Amazingly, it's actually pretty easy to just grab one as opposed to five at the time
It's easy to make them up too if there was a really good random aspect to the game
Because of this?
Because of shuffling?
Yeah.
Or 52-card pickup.
It fits in a hand.
It's a little bit smaller than it needs to be in order for you to hold it comfortably like that.
But these particular cards are a very, very good size for shuffling
They're black.
You can put them on tables
You can put them on things like tables, yeah.
You can deal them.
You can flip them upwards and downwards, sure
[INAUDIBLE] means that you have to put either face down or they get really [INAUDIBLE]
Yeah, they're really terrible for building things out of.
It's possible but really hard to make something, to make a card stand up.
So it makes it really obvious that it's either this or this.
Other orientations other than that face up or face down aren't really considered, aren't really part of the game
I assume that they're rectangular?
Yep
So going back to the point of orientation, maybe they're vertical and horizontal--
Mm-hmm, it's up top-- the path it took [INAUDIBLE]
It'd be hard to do with a square or circle card, but a regular shape, an elongated shape
Conversely, a square card could [INAUDIBLE] full rotation in any direction, rotation.
We'll get that, actually, in the next one where you can just rotate things around
Stiffness and shininess of them [INAUDIBLE] In some games you have papers that you write on them that are disposable.
Pass a paper
This is about left a little bit of time.
Multiple play sessions at least.
So yeah, a bunch of things that come through are already that you've identified which are all very accurate.
And that's a lot.
Cards do a lot.
And when you're designing a game, you need to think about whether cards are the right thing for your choice, for your game.
And you've gone through a pretty deep analysis about what this thing does that might make it appropriate.
Something that might be in those subtle-- the rounded corners actually make it much easier for you to do things like this.
If it wasn't around the corner, it's actually pretty uncomfortable to do a fan.
It's not like you couldn't.
You totally could.
The stationary stores actually do sell punches, corner punches around of your cards.
It is not something that I would actually recommend that you do during prototyping because it takes too damn much time.
But if you were to design a game for home, for your family, or something like that, and you want to make it a pleasant experience.
You might just want to spend $2 on a punch and just punch the corners out.
It's really, really hard to do it consistently when using your hands, by the way.
So it could take away the whole information hiding things.
But oh, the one that was badly punched, that's the joker.
Let's see.
That's, oh, that's next.
Let's see what else I talk about?
So mapping, so back to the idea of mapping.
You've got your intent.
Here is something that you want to do as a player, maybe hide information from other people.
And then there's the affordances of the system.
Now if I wanted to hide my cards from you, then I will hold my cards in a way that only you can only see the side that doesn't reveal any useful information.
So that's a very, very direct, clear what he calls natural mapping, although I am not quite sure that phrase is very easy to use in practice.
It gets a little bit more complicated when you actually look at the system that the game is trying to reproduce, right?
So far I've just been talking about cards.
I haven't been talking about what the rules of the game are.
How many people play cards or something?
OK, a couple of people.
We should be able to get a chance to play this later this semester.
I'm pretty sure it's already in the syllabus.
So there are a couple of things in this game that is this board.
There's a back of the board, which is not colored.
And it has a design on it.
We could just pass this one.
It's got no playing pieces that are referred to as meatballs by the hardcover board game fact base, I guess.
They look like little people.
Actually, there's probably enough in that for everyone to grab one or two.
And then you can just take a look.
I want them all back, but you can take a look at them.
Whoops, and a bunch of piles that I will also hand out.
I'll hand out to those people
Something you can look at.
[INAUDIBLE]
Just take a look at the pieces, and let's start with the titles.
Some of you have gotten just the meatballs.
Some of you have gotten the titles.
What do the tiles suggest, just by looking at them?
[INAUDIBLE].
Terrain?
Terrain, all right, something to do with land.
What else?
The various terrain features seem to match up
The various terrain features such as the--
[INAUDIBLE] and roads.
You can match up
Yeah, they line up nicely when you put the tiles in a grid with other tiles, right?
Yeah, there was a hand back in the back room?
No?
You can rotate them
Yeah you could ro--
Squares
Yeah, squares, because unlike the cards, which is taller and than it is wide or wider than it is tall.
This one is the same length-- more or less the same size from all directions.
So the idea is that maybe you could freely, just freely rotate these things
They're identical with a back
They are all identical with a back?
And that tells us about these tiles, something that we already know about cards
You want to hide the information
You want to hide the information?
Or
Or--
[INAUDIBLE]
Or you want to shuffle it, yeah.
The fact that you've got a hidden back gives you quite a lot of different possibilities.
This rule-- this game in particular uses mostly because of this shuffling and this randomization.
You don't know what tile you're going to draw.
So for people who haven't played this game, and you're looking at all these tiles, what do you think you do with these tiles?
You match them up [INAUDIBLE] on them?
Congratulations, that's [INAUDIBLE].
You figured out they game.
You match things.
You make big things.
You put people on them.
That track that's going around, the board that's going around, I've got one that is probably a little bit less insightful for this particular lecture.
Anyone want to guess what that is, someone who hasn't played the game?
[INAUDIBLE]
You haven't played the game, right?
Yeah, I've played some other games, but it's probably just a scoreboard or something
There's a scoreboard
You do something to move you along the path somehow.
And then the first person to reach it probably wins
Yep, OK, good.
Well, yeah?
Well, it looks like it connects back.
So it makes me think that maybe instead of winning by just getting around, maybe every loop, you get a new tile or something like that
OK, it keeps going on and on.
It's a combination of all three.
It is a scorecard.
If you loop around, I think it means that you've got 100 points or 50 points.
So you just add 50 to your score every time that you go around.
And what six are those things?
What would you place on that board?
The little people?
The meeple.
You place the meeple.
You place the meeples on the tiles.
You place the meeples on the board.
You wouldn't place a tile on that board because there's no hexes there.
So you've already got a mapping of probably what intent that you've got.
Let's make some big things and put our people on them.
And the affordances of what you can do with these titles that suggested that to you.
This is a game about making large patches of similar things and then putting people on them.
Oh, my notes are over here.
Why the--
It keeps coming on.
[INAUDIBLE]
I saw you keep being close to my family photographs.
So there's a whole bunch of ways that you can help people with these sorts of mappings.
We've be talking a lot of visual and physical.
I guess [INAUDIBLE] would describe a lot of these as spatial, metaphors to use.
Spatial mostly to describe things like driving in a car and you turn the wheel to the left especially toward the top of the wheel to the left.
And you car it's directed to turn left, that sort of thing.
That embodies metaphors as well.
Things that are high are either supposed to be good and happy, things that are low when you're feeling depressed.
And you can't pick yourself off the ground, making you sad or bad.
A lot of these metaphors are actually arbitrary.
A lot of us have learned them through culture and socialization in a world.
But that means that these might be things that you can play off.
And we're going to do a little bit more detail.
I'll give you a couple of more examples in about two weeks when we revisit the idea of user design.
He talks about things like single control, single function where if you've got something that does something, you might not want to make it do yet another thing on top of it because that starts to get really, really confusing.
The meeples, for instance, yeah you place them on the map, but you also place them on the scoreboard.
I personally think this game will be a little bit easier to learn if they just gave you a different piece for the scoreboard that was the meeple-- the same colors, just a slightly different piece-- because pieces never move from the scoreboard to the tiles or back.
The other thing that I want to talk about is what do these tiles not let you do?
Well, they did let you do that.
But was it easy?
No
No, OK, all right.
They're also not very good building but you realize, just like cards aren't
[INAUDIBLE]
Holding a whole bunch of them is hard.
They're really thick.
And you saw the difficulty I had just trying to pull half of them out of the box.
They were unwieldy
Shuffling
Shuffling's tough.
You can-- all right, maybe I'll do this once a game
In a pack maybe?
Hm?
If you had them in a pack, yes.
A pack is on affordance, right?
It affords its own like me picking Scrabble, for instance.
How then Scrabble was really, really well in a bag.
So if you wanted to randomize things, yeah, you could totally just put the whole back hiding thing and just put them all into a bag and then just pull one out of random.
So these are constraints.
You're not really supposed to have more than one tile at a time [INAUDIBLE].
You are supposed to draw one, figure out where it goes, put it down.
You're not supposed to ever hang on to two.
So it's OK that the tiles are designed in such a way that it makes it hard for you to hold on to do.
For the cards, for instance, you're not really supposed to stack them.
They are not very useful to you when you've got them face down because you can't really see the information.
To get a small number of cards, like in poker or something like that, maybe you could remember what you had face down.
But if it's a large number of cards, like in Pit, you really want then hooked face up, facing you.
So the idea of-- in Pit, you don't really ever put that face down.
I think there might be a rule about placing them face down right at the end of the game.
But you don't actually-- no, that's not even true.
You don't ever have to place them face down.
When they're face down, they're not interesting to you.
You should have had that information in a minute.
So these are constraints.
These are things-- this is another way that the visibility of the system can help you with those mappings.
You're looking at the system and the pieces that it's giving you, and you're thinking, what can't I do with these things?
And that's probably not what the game wants you to do with these things if the game is designed well.
Games can be designed poorly.
That's a caveat.
Everything that I say is a qualitative, subjective statement.
And that goes back to something that was brought up earlier about mental models, right?
As you play around with these pieces-- of course when you read the rules and you see the illustrations, maybe when they read it at the back of the box, I would suggest that usually you start trying to figure out what a game is when you pick it up off the shell and you start looking at it.
And it says, hey, this is a game that I want to play.
Delux Pit, over 100 years of-- 100 years of card game punch, these 100 years.
The front says 1904.
So this is probably not a science fiction game, all right?
Actually, for people who haven't played this game, what do you think this is about?
I don't think I've talked about it
Maybe about the stock market or something.
There's a bowl on it
There's a bowl on it
There's a trading pit maybe
Yeah, the title of the game suggests things.
It isn't like a commodity trading game.
On the side, it says [INAUDIBLE] market
[INAUDIBLE]
I wonder whether you should at the bell and over 100 years of card game fun intentionally.
This is like, this game back in the time when stock market are run with bells.
I guess they still are but mostly just run with computers nowadays.
I'd love to see Pit on some sort of updated 21st century thing
[INAUDIBLE]
So there are, of course, the text at a back that tells you not only what the theme of this game is.
Shout your deal and trade your cards to quarter the market, et cetera.
And then it also says, family age seven plus, 30 minutes, three to eight players, just to tell-- to give you a better idea.
And when you look at this thing, I'm immediately forming a mental model of how is this game plays.
You know that you're supposed to collect cards because they show you a whole bunch of cards.
You know you're supposed to slam a bell at some point in time.
And then you're just reading the rules.
All right, so what do I do this?
It's pretty easy.
Let's see.
When it comes to cards and so on, there is actually a deep, deep problem with this game despite how popular it is.
And that scoring is actually pretty difficult to do.
It's a math intensive problem.
It does largely map on to how many of the meeples that you have of your own color on large patches of things.
But how much those things are worth, those patches are worth, require a lot of capital, a lot of counting.
And that's why you need the scoring track.
So that is a big problem with the game on how the mapping works.
It doesn't really give you a good idea or a good conceptual model of how much something is worth.
But you've got a large patch of thing.
You've got your dude somewhere in there.
You can at least make the mapping.
That's worth something.
And then that's-- the feedback that you get back from games when it's Pit-- the feedback that you get when you hit the bell, right?
There's a huge ding sound.
That's the feedback that you get when you try to fit a piece that doesn't fit into something else.
And there's a couple of board games where you have to put pieces together, and that gives you an idea of maybe those two pieces don't go together.
What others things look for in the realm of feedback when it comes to board games?
Maybe a track?
Hm?
You mean a board track?
Yeah, you want to stay on the track
OK, all right, that's-- actually there are games where the only legitimate places that you can put your pieces can be is literally on the pieces that they provided you.
So if you fall out of it, you are no longer-- that's not a legal place for you to place your token
So it depends on the game.
But for a very easy example, in a game like Risk would have-- you can very quickly look at the board and say who has the most soldiers.
[INAUDIBLE] but it's a good container.
Just something like that would usually be nice to see you just quickly look at it and say, oh, this person's waiting for this reason
So the sheer quantity of similarly colored things
Yeah, it doesn't even have to be [INAUDIBLE].
In Monopoly, if you look at the board, whoever has the most houses.
[INAUDIBLE]
Similarly, in Catan, you can see if somebody has massive roads or a lot of tunnels and cities, you can usually tell that they're doing pretty well
Mm-hmm, that's, again, direct visual metaphor.
And, of course, in that game, you earn points by having the longest road to emphasize it's a good thing even though it's already a good thing to have in game.
What else?
Other players can give you feedback, right?
If people remember playing Code 7 7 last week, there's information that other players can provide you that you need.
But other players have to provide you.
So what other games can you think of where other players are your primary feedback mechanism on whether you've done that's OK or not?
Mound Builders?
Mound?
The only feedback you get
It is the only feedback you get,
Poker?
Poker?
Did--
If you make a bet-- they're bluffing and you make that a good, you know there's going to be a fold
OK, yeah, for a certain kind of bad, it's like, I won it.
Everyone folds, and it's like, wait what.
I had a royal flush.
Why'd you-- That was a bad idea, right?
OK, that gives you sort of feedback for this specific kind of thing
[INAUDIBLE]
Oh, yeah.
You stick a card with something written on it
Yeah, and particularly based on [INAUDIBLE]
Mm-hmm, again, it's like Code 7 7.
Everything that you know about the thing that you are trying to guess is something that only other people can see
The game Mafia.
All the feedback is filtered through the hosts
And both useful and possibly confusing feedback, right?
So yeah, all the information you're getting in the game is through other people in Mafia
Battleship?
Battleship?
Yeah, there's again the hidden-- information that's hidden from you, but it's completely available to your opponent and vice versa.
And [INAUDIBLE] I'd beat the mechanics to tell you that.
There is a computer battleship.
It's not as fun.
Charades and Pictionary-- usually your teammates are the ones who are guessing.
So I wouldn't necessarily call that feedback because it might be good feedback on whether your clues are getting across to your teammate.
But your opponents are also usually some sort of feedback mechanism that's keeping time, that's keeping an eye out for or listening for his [INAUDIBLE] things, like if you said something very Pictionary.
If they hear you say something, they go ah, ah, ah.
You can do that.
And are there a few more hands or something like that?
So just always remember that you can employ other players into your feedback mechanism.
It doesn't always have to be your game alone
There is [INAUDIBLE] wrong kind of tracking [INAUDIBLE] is all its ever known.
You get some feedback about what he's doing, sort of what he's doing.
You have that in terms of the information that [INAUDIBLE] feedback on whether or not you [INAUDIBLE] from the player [INAUDIBLE]
Yeah, I think it's like something like four or five detectives chasing one fugitive going through London's mass transit system basically-- buses, cabs, underground, yeah.
And the person who's running knows where they're going at any given time.
The detectives are working on partial information to try to corner and close this track.
So I think that's on the list.
Do you remember it?
That was on the syllabus.
It used to be-- Scotland Yard
I'll double check it if it's not.
If they changed the game--
Yeah, so we might get a chance to play that.
But that also falls in mastermind category of games where it's like, here's this person like all the information, only that person is changing the information as the game goes on.
That's a big difference in Scotland
Do you remember this year?
But we do have a copy
OK, maybe we'll bring it in.
It is a good game.
So one of the things that [INAUDIBLE] ends his very first chapter on is that why is it so hard to-- he asks this question.
OK, we've gone through this huge list of things in his book, including things like light switches-- well, actually, he hasn't talked about light switches yet.
He will talk about light switches, something like that.
He talks about cars.
He talks about doors.
He talks about clocks.
And he asked this question-- why is it so hard to actually make something right?
Anyone remember?
Or anyone thinking of--
Doesn't the designer never really [INAUDIBLE] one-to-one with the user?
They're taking through the object that repeats her design more continuous?
That is definitely true.
It is a second order problem.
You're designing something that then becomes this manifestation that somebody else uses.
And that's actually when all the problems occur.
You first, then
Well, he was talking about how like at first when something is invented.
At first it's very complicated, and then your iteration comes up with something very simple.
But then people want more functions.
And they start this whole process over again where you keep adding new things.
But it becomes less and less intuitive
Market forces push you to add things, to do additive design in order to distinguish yourself from the competition.
And that naturally leads to complexity in interface
And a lot of products, you said, don't get through that process because it'll take five or six times to get it right.
But if it's not good by the second time, people just won't buy it
Yep, so get back to iteration.
Five or six times means five or six times at the same problem, right?
So iteration is the reason why design starts off as being very clunky.
But it can eventually become something that works well, communicates well, or something that people can learn and maybe even enjoy in the case of the games.
One thing that's funny about the book is that it talks a little bit about the clock radio that could do-- make phone calls, and be used as a desk lamp, and keep track of your appointments, and used as a TV.
And I'm still thinking, this isn't that this, right?
Exactly the same thing they he's describing.
And it's interesting because he described a phone that can only be used with two buttons, right?
Here's a button.
And then here's a huge button.
You get to select things.
And it's exactly what he's talking about on the dodgy edge as he imagines that this was something to be possible at the time when he wrote it.
That was 1980.
It wasn't that long ago.
And cell phones obviously have gone through a lot of criticism.
IPhone, of course, has a lot of criticism because people who criticize it for being a lock-down system where you can't really do all that much.
It's certainly not as customizable as Android system.
It's expensive for what it does.
But then, arguably, by locking out a lot of things that you might want to do but maybe don't have to do, they are trying to make it easier for you not to do the wrong thing.
So depending on-- there are different ways to fall on this camp.
But it is it has been successful through a number of different reasons.
Don't discount marketing as being something that the does sell.
But what I want you to think about now is actually the process of prototyping.
Actually you know what, I'm not going to go right into prototyping because they will probably make more sense once I've actually got the prototyping materials out.
What I am going to do is to give everyone a five minute break.
We're actually going to come back and play some of last year's games because I think there's enough time for it now for about an hour.
And then the last hour of class, back I'm going to do is I'm going to go into brainstorming.
So then you can start forming your teams, thinking about what kind of game that you want.
Prototyping might be something we'll leave up to Wednesday.
So that there'll be a little more time for you to work in your teams.
So I'd like to get all the contest bits back.
So and then we'll pick this up in about five minutes.
[INAUDIBLE] [SIDE CONVERSATION]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
But this is going to be used to set some of the ground rules are based on the-- so that we can actually talk a little bit about what projects you want to be doing for assignment one.
Just a reminder, assignment one is all about picking one mechanic and going as deep as you can.
And that doesn't just mean using the same mechanic over and over again.
This is exploring all the different ways that you can use a single game mechanic.
And that game mechanic will be whatever you want to choose.
So a lot of brainstorming is going to be around that.
But you can do brainstorming around themes like-- I wanted to make a game about-- the game is about time travel and snowball fights and CPEs [?
chemicalese.
?]
So you can do that.
But, if you were to brainstorm with that, I would suggest try to develop that into, what is it about that theme that you're really interested in?
Is it about the stealth?
Is it about consequences?
And try to bring the conversation back to game mechanics.
And for those of you who can't remember the working definition I gave you on Wednesday, I'm describing game mechanic as something a player does to change game state, right?
So this guy Alex Osborn-- not the high tempered in the image there-- he was working in advertising, and he actually sort of coined the word-- I think he coined the word-- brainstorming.
And wrote it up in "Applied Imagination" as a practice to just get a lot of ideas on the table, sort of without any regard to quality but quantity.
And the reasoning behind that is that if you can just get a ton of ideas, then there's going to be some good ideas in there.
And you can go through a process later on-- usually for us it's going to be during the team formation process-- where you decide whether it's the actual idea that you want to work with.
But how to get enough ideas on the table so that you-- there's a couple of gems in there for you to find.
And that's what this exercise is going to be all about.
A couple of things I want you to remember while we do this.
One, you don't criticize any of the ideas that have come up.
And I do want to say anything built on that idea.
Suggest something that-- if you can't think of a better way to say that idea or a better spin to that just throw that out as a new suggestion.
You don't criticize the stuff that's been out there because you don't want to intimidate anybody from sharing their ideas.
The worst thing that you can do is, somebody says, I want to make a game about blah.
And then he says, that's stupid.
Wow, that was kind of dumb.
I can't believe you said that.
Things like that.
And then it's like that person may have even better ideas along the way but now would be hesitant to share that with the class.
And one of those ideas might be-- might end up having been the idea that you would have loved to be on the team on.
So no criticism.
If you want to judge the ideas on whether it is good or bad, you do it at the end of the brainstorming session, once you've already got all the ideas on the table.
Then you can just decide for yourself which ideas you want to work with.
We want to go kind of freewheeling.
It's OK for you to come up with ideas that are impractical, that are too ambitious, that maybe don't even really fit the criteria of this project.
That's fine.
The idea is to get the ideas out, and other people are going to come up with ideas that are appropriate.
This is not the time for us to self-censor and decide is this an idea that fits or not.
Just throw it out.
I will write things down.
I'll keep things on the computer screen so that you'll be able to see it [INAUDIBLE] Again, we're trying to go for quantity.
This is a direct quote from Osborn, we're just trying to get as many ideas as possible.
So try to keep it fast.
Give me a little time to write things down on screen-- to type out things.
But otherwise just keep throwing out ideas.
Bad ideas are great because it adds to the quantity and it gets-- it stops you obsessing about the bad idea and then you can move on to other ideas.
Build on ideas.
Try to combine stuff that you're already seeing, that other people have already suggested.
You can combine them to use something that's pretty awesome.
Lego and Star Wars is pretty awesome.
And then Lego plus Star Wars is pretty darned awesome.
Or Marvel and Capcom, right?
Marvel vs. Capcom, right?
And that's what I'm saying.
All right, so I'm going to talk a little bit about just the principles of doing this because this is the first time I'm going to be a facilitator.
But in the future, you're going to be doing this on your own.
And once you've formed your teams you may go through another second brainstorming phase.
So you want to keep it kind of relaxed.
And this is-- if it helps I'll probably turn off the recording, Start off by throwing out the worst ideas that you can think of.
Just start suggesting them, and I'll write them.
Don't interrupt anyone.
I'm going to be the facilitator and secretary, so I'm going to be the person who's going to be writing up.
Actually if you can help me with the facilitate thing.
Right.
No, I'll do the typing.
If you notice there are people who are hesitant to talk or trying to get a word in edgewise, help me identify them so that they can say what they need to say.
And I'll talk a little bit more about the facilitator role.
I've explained a little bit about the process.
Actually I [INAUDIBLE] process right now.
And then the principal question is what is the one game mechanic that a team could work on for assignment one.
what is the one game mechanic that you want to work.
What's the one game mechanic you would like somebody else to work on.
What is the one game mechanic nobody should be working on.
Throw it out here, OK. And I'm going to write down everything.
So you're going to have to do this on your own, in your own teams as well.
So it's important for us to define-- if you are doing a brain storming session by yourself, it's important to define what is the aim of the session.
Is the aim of the session to get so many mechanics that you'll be able to identify at least one that you will be able to start your project on?
So if you're creating a brainstorming session for yourself, make sure that you are defining that problem ahead of time.
If it's too complicated-- say you're trying to brainstorm a solution to a design problem, or something like that, and that problem has a lot of interlocking parts-- you may need to divide it up into separate brainstorming sessions.
So articulate them as simply as possible so that everybody knows what they're brainstorming towards.
So far I've been explaining the problem and aim of the brainstorm session.
The problem is that you don't have a project team yet.
And Rick is going to help the ideation and is going to discourage criticism if any of you automatically go, whew, that was dumb.
Just stop smacking or [INAUDIBLE] He's going to call you out.
And he's also going to identify if people are having trouble getting a word in edgewise.
Both of us will try to encourage some combination of the ideas.
So we may throw in ideas which are just combining things that would be on the board.
After this I'm going be playing the secretary, and my job is just record every idea.
And just keep an eye on the time.
Probably we'll run this properly for about 20 minutes.
And then after that, the rest of the class will be yours to be able to discuss among ourselves and identify what projects we want to work on.
I'll try-- [INAUDIBLE] brainstorming.
Again to try to combine, I guess.
But if you're keeping it fast and flowing, then I don't even need to do much more than just write
When you're brainstorming on your own, that's really, really hard to make sure you're rotating the role [INAUDIBLE]
Yeah, you might want to break your brainstorming session into like two 10 minute chunks or something and get a different person to be secretary.
And that just makes it easier for the person who is secretary to then contribute later.
OK any questions?
OK, all right, let me bring up a little text for this session again.
Sorry about the family photos.
Is this too dark?
Actually let me just [INAUDIBLE] What [INAUDIBLE] doing?
Probably too big but, uh-- Oh, jeez.
Just one sec.
Is this still chopped up?
Yeah.
It's still chopped up, OK. [INAUDIBLE] But this should work.
And I'm going to-- I can do columns in this thing, right?
Can I?
I can do tables?
OK, all right I don't think I can do tables.
Oh, wait, tables.
Perfect.
OK, all right, I am just going to type what you say.
What kind of game mechanics do people want to work on?
Go for it
[INAUDIBLE]
Building
Path building
What?
Path building
Path building
Auction
Auctions
Resource packaging
Guarding
Oops.
Oh, darn it, it doesn't just add new columns,
[INAUDIBLE] one row
And then [INAUDIBLE]
All right, sorry about that
Trading
Trading
Stealth
Attack
Sorry, one at a time
Attacking
I heard stealth
Attacking
Attacking
Destroying
Destroying.
Did I hear one from here that's not on the board?
I thought I heard--
Manipulating
Time
Pushing
Pushing
Regulating time
What?
Is that true though?
Yeah
Cheating
Cheating
Rules, changing rules
Changing rules
Lying
Inquisition
Sorry, one at a time
Guessing
Guessing
Prayer
Prayer?
Yeah, why not?
Bombs
Bombs
[INAUDIBLE]
Prayer bombs
Where's Armageddon, bro?
Manipulation of player's emotions
Manipulation of emotions
Gestures
Gestures
How about drawing?
Drawing
Writing
Writing
Hidden goals
Erasing
Hidden goals.
What was that?
Erasing
Erasing
[INAUDIBLE] effectively ending the game
Accidental victories
Asymmetry
Symmetry
Uncomfortable situations
Puzzling.
When a puzzling situation [INAUDIBLE]
Puzzling situation
Designated winner and loser
What was that about winner and loser again?
Designated winner and loser
Cooperating
[INAUDIBLE]
Everyone loses
Everyone wins
Defector
[INAUDIBLE]
What was that?
Defector
Defector.
And what's that one in the back I missed?
Hidden agenda
Well, that didn't [INAUDIBLE]
No one knows what's happening
[INAUDIBLE] organizations
What was that?
[INAUDIBLE] organization
You in the back
Oh, alliances
Story telling
Frisbee
More dice than you know what to do with
[INAUDIBLE]
Intimidation
Map building
Deck building
[INAUDIBLE]
Oh, army building.
I'm just keeping that up
[INAUDIBLE] building building
Meta building
Meta building
[INAUDIBLE] building
Recursively recursing
Damaged terrain
Damaged terrain
How about damaged terrain?
Wait, damaged?
What?
You said damaged, didn't you?
No, no, no, no.
[INTERPOSING VOICES]
Damaging terrain?
Yeah, terrain that [?
damages ?]
[INAUDIBLE]
How about terrain?
[?
Fights ?]
everywhere
Non-playable characters
Playable characters
Too many playable characters
How about multiple roles?
Climate
Shared player characters
What?
Shared player characters
Growth
[?
Solitude.
?]
Pain
Paint
Decline
I said, pain
Oh, pain,
[INAUDIBLE]
Oh, pain
No, I just [?
paid.
?]
[INTERPOSING VOICES]
Listening skills
Mispronunciation
Telephone mafia
Knowledge of random pop culture facts
Mash-up
Trivia
Mash-up, whatever that means
[INAUDIBLE]
Who gets the [INAUDIBLE]
Onomatopoeia
There was another one over here
Disease.
It was who gets the [?
shiny.
?]
Disease.
Who gets the shiny-- I can't believe I spelt that right?
[?
Time ?]
[?
to ?]
[?
leave.
?]
[INAUDIBLE]
Time travel
Collecting too much
Double meaning
Resource management
Poison
[?
Research, ?]
I mean, [INAUDIBLE]
Racing
So what?
Did I miss--
You wrote that earlier
What was that?
[?
Cannibals.
?]
Robots
Settlements
Settlements?
Yes.
And hunting
Survival
Pirates
Parallel worlds
Parallel worlds, and I think I missed something
Pirates
Exploring
Cool looks
Cool looks
Tabletop [INAUDIBLE]
There is another one over here
[?
Knobby.
?]
Zombies
Plants
[INAUDIBLE]
Copy right over it.
[INTERPOSING VOICES]
OK, I didn't catch-- what did you say?
Patent trolling
Patent trolling
Mercy killings.
[INTERPOSING VOICES]
[INAUDIBLE]
Pets
[INAUDIBLE]
[INAUDIBLE]
Music
Music
Fear
What was that?
Fear
Fear
Sound, why not?
Sound.
Other things in games that you like or don't like?
Wrestling
Forcing someone to change their moves
[INAUDIBLE]
Shooting stuff at people
Fake difficulty
What?
Fake difficulty?
Listen and repeat
Listen and repeat
It would be better for everyone if we all just worked together
Do we have cooperation?
Yeah
It would be better for everyone if we all just worked together
Competitive cooperation
King raising
Oh, yeah
King killing
That's a word for that
Stabbing
Stabbing
Designated psychopath
Social commentary
Social commentary
Prisoner's dilemma
Prisoners dilemma
Turkish delight
Morton's fork
What was it?
Morton's fork
I don't know that one
You have two choices, either one will doom you, so
Fork
Fork
Doom
Death
Glory and honor
[INAUDIBLE]
[INAUDIBLE]
I guess permanent changes in [?
journal.
?]
Growing changes that last between games
Rule making
Rule making
[?
Kettles ?]
and balls
Well, that's--
That's [INAUDIBLE] [?
huge ?]
[?
conference.
?]
I'll just write it down anyway
Brainstorming
Have to come up to [INAUDIBLE]
[?
Scatter ?]
[?
brainstorming ?]
Meta-gaming
Tree climbing
[?
Storm ?]
[?
champion.
?]
Yeah, [INAUDIBLE]
Possession
Tent building.
Illegal possession
I think I missed something
Tent building
Tent building?
Yes
Capture the flag
Fire building
Building.
Capture the flag
[INAUDIBLE]
You wrote it up and [?
captured it?
?]
What was it?
Confusion
Confusion.
And you said?
Packing
Packing
Confucius
Confucius?
Yeah
Philosophical
I can't believe-- Philosophical
Limited board information
Limited?
Information about the board
OK, board information
No talking
Progressive [?
walking ?]
mechanics
Screaming
[INAUDIBLE] moves
[INAUDIBLE]
Funny voices
Do we have voice in general?
You do now
Bankruptcy
Progressive bankruptcy in [INAUDIBLE]
Folding
Folding
Total domination
Last man standing
What was that word?
Last man standing
Last man standing
The American dream
[?
Completely dominated.
?]
Trying to find information
Manifest destiny
The rules are the game
Make the rules up
Charity
Game within a game
Cooperation where one person's secret leads them [INAUDIBLE] anyway
One person leads us secretly anyway
Coopetition
[?
Water ?]
[?
games.
?]
Basic [INAUDIBLE]
Searching
Don't worry about it
[INAUDIBLE]
Hiding
The game doesn't [?
die.
?]
Revenge
Torpedoes
Hiding [?
from the reds.
?]
Hiding from the--
Torpedoes
Torpedoes
Multi-level board
Not throwing dice at people
Which leads to throwing dice--
Anger management
Single player
Drinking
Single-player?
Single-player drinking game
Single-player drinking anger management
Players [INAUDIBLE] [?
learn ?]
If you can't beat them, join them
Unicycling
Therapy.
[INAUDIBLE]
What was that?
Therapy and unicycling
And unicycling
Unicycling therapy
What was that?
Cooperative unicycling.
[INTERPOSING VOICES]
I feel I'm missing this one
No, [INAUDIBLE]
You combined the two of them
Morse code
Build-your-own code
[INAUDIBLE]
Hidden messages
Language barrier
What was that?
Language barrier?
Yeah, language barrier
Mastermind
Horticulture
[?
Game ?]
after
Gaming [?
laughter.
?]
Mining
Mining we got
[INAUDIBLE]
[?
Data ?]
mining
[INAUDIBLE] you can say it again
Pick a card, any card
104-Card pick-up
Worst sequel ever
Worst sequel ever?
Which of the 104 cards Is missing?
Never playtested this game
What?
Never playtested this game.
[INTERPOSING VOICES]
Playtesting
Player modification of game
Knocking everything over
What?
Knocking everything over
Knocking everything over
No time to explain
Also a name of a great game
Follow the leader
[INAUDIBLE]
Take down the leader
The leader follows you
Opportunistic [INAUDIBLE]
Did I spell that right?
Oh, good.
Write
Figure out who's following who
Predator and prey
Prediction
What was that?
Prediction
Prediction
[?
Profitability.
?]
[INAUDIBLE]
[?
Panace-- ?]
Invisibility
Getting to roll [INAUDIBLE]
Through avatar?
Yeah
Moral choices
The world is ending, let's have fun
Moral ambiguity
Feedback loops
Mind reading
Immoral ambiguity
Mind control
Imitation
Immortality
Flattery
[INAUDIBLE]
What was that real fast?
[INAUDIBLE] we're going fast
When did we start doing this?
When did we start?
It's been almost 20 minutes, almost
Typing
Prettiest princess
[INAUDIBLE]
Highly adaptable
Discrimination
Who can eat the most food?
Fashionista
Coffee
Illness
Illness?
Did I miss something?
[?
Boiled ?]
[INAUDIBLE]
[INAUDIBLE] foil
Tin foil
Plane crash
Plane crash?
MacGyvering
Archery
Assassination
Pirates
I think we had pirates
Yeah
Beating the professor
Beating the professor
Not physically
[?
Feasibly ?]
broke
Bluffing
[?
Petty ?]
[?
cash.
?]
Time travel
Time travel's on there
Someone said that in the future
Space travel
Space time?
Space travel
Space travel
Multiple mediums
Slow and painful
Quick and painless
Quick and painful
Loyalty
Quick and pain-- what that royalty?
Loyalty
Loyalty
Swapping [?
piece ?]
Fully drunk
[INAUDIBLE] out from under
Running [INAUDIBLE]
Dogs
Dogs
Cats
Sore losers
Flipping the board
Flipping part of the board.
But not being able [INAUDIBLE]
Knocking over pieces
Air ducts
Organized mayhem
[?
Ducks ?]
in the air
Mischief
Finding your lost friends
Finding your drunk friends
Finding your [INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE]
What was that?
[INAUDIBLE] your lost friends
Vampirism
Vampirism
What was that?
Vampirism
Changing [?
goals.
?]
I think we had that, but I can't remember
Giving up on your dreams
Mobile board
Storytelling--
Mutually assured destruction
Storytelling that riffs off of Crayola colors
--on Crayola colors
Multiple boards, was it?
Yeah
And what was it between?
Mutually assured destruction
There you go
Knowing when to give up
Going outside
Well, that was a waste of time
[?
Black ?]
tile
300
[?
Wet ?]
[?
tile.
?]
OK, we have more than enough ideas.
So I feel that one thing that we could do is sort of weigh in on which ones might fit the criteria of class better.
So that's not necessarily saying that any of them-- that we're going to identify-- are good or bad ideas.
But it seems like if you were to do something on that you would fit the criteria pretty well
So you want people to call out?
What are the criteria?
Well actually I thought that we'd just identify it as instructors because-- some feedback for everyone.
What we suggest today doesn't necessarily mean these are the only ones that you can work with.
After this class is just open to you.
You form a team.
Come back on Wednesday with a team.
Talk with each other.
At the end of class you can stand up and say, I want to do something on-- OK I'm going to identify trading as one which I think is a mechanic, right?
So if I want to do something on trading, [?
I would ?]
go and talk and figure out whether you want to be on the team
[INAUDIBLE]
No more than four.
I'm going to suggest at least as many people as you have players in your game.
So if you are making a four player game, you need a four player team.
That's why you need a four person team because it just makes it easier for you to test, slightly-- at least in the early stages.
If you want to make a two player game, you need at least two people.
But you can have four people in a two player game.
More people than that it's hard to schedule and you go kind of slowly.
But four people making a two player game [INAUDIBLE]
Yeah, or try different-- simultaneous prototyping and riff on each other's ideas.
Things that I see that fit the requirements-- discarding, path building.
Building, I think we go into more specific stuff later, so I'm going to look at that
Hidden goals
Hidden goals
Manipulating time?
Manipulating time.
Manipulating time I think is probably more like combination of different mechanics
Yeah, more of a theme than it is a mechanic
Yeah.
Theft I guess could be a mechanic-- picking something that somebody else doesn't want you to take
Decline?
Like growth and decline, those two
Growth and decline
Is there [INAUDIBLE]
So these two, right?
I think these two combined-- yeah, definitely something that-- actually, I think even separately is fine.
Some sort of mechanic about slowly just rolling downhill, right?
You could make a game just about that.
Let's see.
There's a lot of stuff here now that I realize how many ideas got thrown out.
There's a lot of stuff here that does fit.
So everything that I'm identifying here doesn't mean that these are the only things that fit.
But just trying to give you some information about-- like [?
voting ?]
I think can be taken in a million different ways.
And that's exactly-- it's all [?
voting.
?]
But how you do different kinds of [?
voting ?]
can be explored in so many different ways in a game
Auctions?
Auctions?
Yes.
Frisbee, I actually think it's a great-- it's a [INAUDIBLE] like you can only do things with this one thing, right?
And I think [?
Tron ?]
only scratched the surface about what that could be-- although probably too dangerous but-- Terrain in general, all of these I think fit--
Especially if you were thinking of tiles
Or little armies shooting each other
Multiple roles is pretty close to it.
It does need a little bit more what the roles are actually doing
Whether you choose from a collection of roles
Or having multiple roles
Having multiple roles simultaneously.
I think you want to probably identify something a little more specific off the multiple roles basic idea.
But you can do that in your team.
Just scrolling down a little bit more.
Racing is more like a whole genre.
So maybe identifying something inside racing.
A mechanic will be things like boosting, for instance, is a mechanic.
I'm going to say, not Nazis, not pirates, not robots.
These are things that show up in games and often the more often they show up in games the less interesting the game often gets
Resource management?
[INAUDIBLE]
Yeah, resource management, again, is kind of a category of mechanics.
I think you want to identify a specific kind of resource management, whether it's like supply and demand-- I think there's already tons of things that you can do with just plain supply and demand.
Trading also fits in resource management, right?
But trading is more specific than resource management, I feel.
So inside resource management there's a pile of different ideas, and you want to focus on probably just one.
I want to be a little bit more specific on what I mean by that.
I don't mean that your game should just have one idea.
It means I want you to identify a game mechanic that can be played by different people in different ways through the course of the game.
So there's a lot of different ways you can execute a trade.
Then that's why I think that's a good fit.
But the end result is still exchanging something with somebody else, right?
So you can see that you're comparing apples to apples here
Guessing?
Guessing?
Yeah.
Yeah, there we go.
All the different ways you can set up-- like intuit ideas.
That's one that needs someone to do a really deep dive in because if you just do a shallow guessing game, it's a shallow guessing game and it's not interesting.
But if you really think hard about how do I-- all the different ways I can make guessing interesting-- Deck building is a good mechanic but really, really hard to do for any kind of assignment in this class, even in your last assignment.
The reason for that is deck building implies lots and lots of cards.
Lots and lots of cards makes it hard to iterate because you're generating a ton of things just to make a single prototype.
You want to make a second prototype you're going to generate a new batch of cards-- time consuming.
You're going to have blisters on your hands cutting things out.
That's my warning about deck building
[INAUDIBLE]
That game had deck building
That game was a kind of very, very narrow-- fairly elegant implementation of a deck building game.
Yeah?
A variation could be handled, but you only have seven cards that you can have at any one point [INAUDIBLE]
Yeah-- things like deck building, card drafting, and everything-- anything that comes from the collectible card gaming world, you have to be very cautious of because of the difficulty it is to prototype something like that because it generates so much stuff each time you want to do a revision.
Which is not to say it doesn't fit the criteria.
It's just difficult to execute.
Resource management definitely came up twice.
Instead of MOBA you might want to think of something like [?
lean ?]
pushing, or tower-- not tower in the sense of genre but defending a tower
What are MOBAs?
MOBAs are games like League of Legends, Dota 2.
It's a genre title for multi--
Multi-player online battle arena
--Player online.
Yeah.
It's a team based game, usually five a side.
But these are all genre conventions.
There's nothing that requires that.
Bankruptcy is something that shows up in a lot of games but usually as a lose condition.
It would be really interesting to think of all the ways that you can go bankrupt in an interesting way because reality is that actually there's a lot of them.
In reality you can go bankrupt to your advantage.
Permanent changes to the game.
Difficult prototype again.
But it's still possible.
You're going to generate a lot of games that you're going to hand out to people.
And the difficulty about it is that you don't really see the feedback until someone's played the game multiple times.
So good idea, difficult to prototype.
Again a lot of stuff here is great.
I'm just looking for things I can comment on.
Torpedoes is kind of really interesting actually.
I know it's a noun rather than a mechanic.
But there are things that torpedoes do that are not things that projectiles do in many games.
They're stealthy things.
They are launched from things that are already hidden, but they give away where they were shot from.
All the countermeasures that are involved in torpedoes.
Everything that has to do with something like code breaking, hidden messages, language barrier.
I feel that exchanging messages in a way that you're hiding the message at the same time might actually be something that one team could actually just take on for this assignment
[INAUDIBLE]
Yeah, try to identify something they can go deep in-- that you can do a lot of different takes within the same game on just that one mechanic.
Same thing that has to do with any kind of clothing, changing avatar customization, and everything like that.
That one's a bit tricky because usually that's in service of a larger game where the avatar customization has some varying level of utility.
If you're going to do something like changing your clothes, the point is to change your clothes.
The point isn't in service of some other game.
In that game I think the point is in fact to change your clothes.
And in fact [INAUDIBLE] changing clothes.
If you haven't seen the game that I'm talking about it's called Dress for the Show.
It was one of the games from last year.
These could be difficult to prototype because of the cleanup process.
You might want to-- yeah, it's difficult to play in class too so keep that in mind.
This one's kind of hilarious.
Anything that has to do with drinking, please don't make a drinking game in this class.
I can't officially test it in class.
And that would be-- I'd have to answer to the committee on curriculum.
And I don't want to do that
They are good expressions of the guessing aspect of it though
What drinking games?
No, the finding games
Oh, the finding games
And the drinking game, but--
OK, so hopefully that gives you some idea.
Of all the ideas that have come out, some of these ideas fit what we're looking for.
I'd say, if anybody wants to in class-- hey, I really want blah.
Just go for it.
I'm just going to copy and paste this into a file that's visible only to students in this class and upload it on our [INAUDIBLE] site.
So, they'll either be in the announcements or in the handouts section so that you can refer back to this.
The class email list is-- it should be game-design.
But I haven't updated the list yet, I think.
So let me just check this.
[?
Moira?
?]
Yeah.
It's just game design.
I suspect it has last year's students on it, which means it's not useful.
Yes.
It has last year's.
[?
S-P-14-C-M-S-6-0-8.
?]
That's our list.
OK, I'll have to fix that somehow.
I am going to add S-P-- I'm going to figure out exactly what the name of our email list.
And then I'll put that definitely on the announcements page.
So you can all discuss online as well.
Otherwise, that's class so if anyone has anything to yell out--
Show up Wednesday with your team or pitch on Wednesday?
On Wednesday you should come in knowing what mechanic or mechanics you feel like working on.
If you have your team, even better.
If you don't have your team by Wednesday, it's a very short amount of time, so I think it's OK.
But you should have your team by the end of Wednesday.
OK?
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so before we go into the stuff that I'm going to cover for the readings today, I want to get a sense of who's already found a team.
How many of you are already in a team?
Half the class, OK. And the rest of you, what we're going to do is try to solve that problem right now and get you in a team.
For the people who are in teams, can you say what mechanic they're working with, and how many people you have in your team?
What if we have two mechanics, so we're not sure of that yet?
That's fine.
Just talk about both of them, and then you'll figure it out, probably today

So what are they?
We were thinking between either like pathfinding or some kind of resource management
OK, path--
I'm sorry, path-building
Path-building and resource management
And we have three people.
OK, so three right now,
OK. What else?
Hidden information, and we're four people
Great work.
Hidden information, so that's four
Stealing or like team sharing
I'm sorry, stealing, or?
Or like teammate sharing
Team mate--?
Sharing.
[INAUDIBLE]
Oh, OK.
So stealing or team sharing.
I think they say team sharing.
How many people?
Is that it?
OK, and how many people are looking for teams?
OK, one, two, three-- is your hand up?
No
Ok, one,two, three, four, five, six people.
All right.
So we could make two complete teams out of the remaining six.
I think that's right, did I count that right?
So that's what I would suggest is that all the people who don't have teams-- we try to make two teams out of that, rather than try to join this team.
Because otherwise, we end up with a two-person team somewhere, and that's not good.
All right, so of the people who aren't on a team, we went through brainstorming on Monday as a whole class-- I can bring up the list again, but was there something that you remember from Monday that you felt like you would like to work on?
Either voting or bankruptcy.
[INAUDIBLE]
Bankruptcy
Your numbers-- there's 16 people.
You went ahead and said there's four 4-teams and six unassigned
There are six unassigned right?
But there comes a total of four, four, three.
Someone's being counted on two teams.
[INAUDIBLE] When they come.
They're not here right now
The remaining people who are not here, if we have a bunch of three-person teams, it's a lot easier for extra people to [INAUDIBLE].
And three-person teams are pretty easy to work with than four-person teams.
Also, we'll be working, but it'll be a lot easier to schedule a meeting with three of you.
So three-person teams are good.
Two is pushing it because if someone gets sick, you're in trouble.
Yeah?
Back to mechanics, I was interested in trading, or maybe building, or expansion
Trading, building, base expansion
To protect
OK, so by expansion, we mean territorial expansion.
OK. Yeah, we see these three things in one game a lot, right?
But for this assignment, let's see where we can deal with one
I'm interested in building or area control.
Would it be in deception or [INAUDIBLE]
What?
Unfairness
OK.
So All right, so here are the different kinds of game mechanics that the people who aren't assigned are currently-- at least one person is interested in them.
So what I'm going to do is I'm going to go through each one of them, and for the people who aren't assigned, put up your hand if you think that you might be willing to work on a team on that concept.
OK?
So, starting with voting, we have-- OK so three, bankruptcy-- one, trading-- three, building-- four, expansion-- it's territorial expansion again-- four, back building-- one, area control-- four, area control and expansion might actually end up beating each other then.
Deception-- one, two, three, four, and unfairness-- one, OK.
So we've got a bunch of things-- I thought there was a lot of interest in area control.
I think I'm going to leave that off because we will be revisiting the topic later in the semester.
So you'll get a chance to look at these.
So I'm going to take these two out for now.
I'm gonna take out all the good ones that will make it tricky.
That leaves us with voting, trading, building, and deception.
Of the people who aren't assigned, was there one that you're not interested with?
Any one that you're not interested in working with, tell me anyways
[INAUDIBLE]
One, two, three, or four.
If you're interested in one of these, I think we can make teams out of this.
All right, so later on in the class, when we start the prototyping, there's going to the time to actually talk with each other.
What I'm going to actually suggest is all the people who are not on teams switch with the front row, and all people who have teams, switch with back row.
Probably, one corner will also have to be a team that already exists.
So yeah, actually, let's do that now.
So if you're not on a team, switch with front row so we can all have discussion.
[SIDE CONVERSATION]
You might as well sit with your teams if you already have one.
Because we're going to talk about something new together today.
[SIDE CONVERSATION]
All right, for the people who aren't in teams, remember the goal is trying to make two teams out of this, of any combination.
If you end up changing the game mechanic, that's fine.
All I'm looking for is two teams, out of the six people who aren't assigned any.
[SIDE CONVERSATION]
Who hasn't signed in yet?
Who hasn't signed in yet?
Write your name in.
You're pre-registered?
Yeah, and I was there before.
My name is on the list the first time.
Now my name's not on the list
Very weird, something weird.
I'll take that straight outside, so write your name down.
You did awesome, and I'll take it out during the break and see where to fix that problem.
All right, so how many of you have seen this presentation from me before, in any of the 15 times I gave it in the past year?
OK, all right, so about four people.
This is a presentation that I give a lot.
It's also something that ends up getting covered in one of our new classes this semester CMS301.
This is probably gonna be the last time I actually give this presentation in CMS608 because it's kind of like a really basic skill, we're gonna be moving this into our intro classes in the future.
But it's also the one skill that, if you learn nothing else from the rest of semester, but you still didn't come to class, please learn this.
Because this is the core skill that we're going to be asking you to keep working and keep practicing and keep improving on, all semester long.
This is the thing that you're going to be doing all semester long.
So first of all, that's kind of jumping the gun.
Let me take a step back.
What's a prototype?
It's in the reading
Like a basic thing you just toss together to illustrate a concept
To illustrate a concept, yep
Isn't that supposed to do one mechanic and reiterate that one
For game, it could be something that just tests one mechanic or concept, and then, you have to reiterate on it, sure
Something you just put out there to get feedback
Yeah, to just sort of like gauge how other people are going to respond to it
In general, It's just an unfinished version of the game
It is unfinished.
It is not like your shipping product.
At no point you see it a shipping product
I'd say it's that first version of anything, that was specifically built just to test that thing, rather than to have a product
Again, just to test an idea, to test whether something could work, and, sir, you said first version could be an early generation of something
I was gonna say like a minimum usable--
Something that you can actually use, not like a sketch of a game.
An actual game that you can actually play.
Anything else?
I thought I saw a hand back there.
I think [INAUDIBLE] is getting kind of a good sense of what a prototype is.
You probably encountered prototyping in some other classes-- a lot of engineering classes involve prototyping.
It is this unfinished thing.
It is not meant to be an iteration of something that you're actually going to ship.
Now, in this class, a lot of things that you're going to end up building are building towards a finished class assignment-- the thing that you hand in that meets all of the criteria, that has all the rules, and if you put in front of someone who's never seen a game before, they should be able to figure it out.
However, the very first assignment, it's pretty build a prototype.
You're testing out one single mechanic.
And the question that you should be asking yourself is, what are all the different things I can do with this one mechanic?
And then, you can just deep dive into this one big open question.
And you're gonna end up choosing that question for yourself, but as an investigation.
And prototyping is a tool to help you investigate something, that is going to help you build your final game in the end.
So think of assignment 1 as an exercise that's actually going to help you build assignments 2 an assignment 3.
Even though we are asking for things like rules that we can read, but for the most part, this can be very sketchy, very unpolished.
If we can play it, if we can use it, I think someone said minimum usable product-- to be able to see the ideas that you're working with, that's good enough.
Do spell check your work, that would be nice.
But some of the reasons on why you want a prototype-- where are my slides notes?
Just one second-- I have no slide notes.
OK.
So some of the reasons for why you want a prototype-- we already talked about getting feedback-- being able to put it in front of people who may not necessarily have seen your game before, but may be [INAUDIBLE] of an audience-- to be able to get their opinion.
Put it in front of instructors who may never have seen other version of the game before or maybe have seen the game, to be able to get their feedback.
Or other designers or guests that we might be bringing in-- you want to be able to put something in front of everyone who knows something about games to be able to get their critique, right?
But the earlier and cheaper part is the important part.
You want to be trying to get feedback, as quickly as possible, in your game development process.
The earlier you manage to get new information into whether your ideas are working out or even appealing, then the cheaper it is to make changes.
So for instance, you've got a great idea for one of the game mechanic, and it turns out that everyone else on the team absolutely hates it.
But you'd rather find out about that on the first day of meeting up with your team, than, say, two weeks into the project.
That would be very useful information because-- OK, I came up with other ideas.
Are you on a team?
You are a team.
Awesome, OK.
So I thought you weren't able to get on a team.
The other thing you are trying to do with a prototype is to try a lot of different approaches to the same problem, to experiment a lot of alternative solutions.
One thing that often happens in design teams, professional design teams, amateur design teams in school or outside-- the people like to talk there, as you know.
They love to make sketches, they love to theorize about-- well, it worked in this game, so it should work in our game-- I really like that, or i really hate that in this game.
This offends me on some sort of primal way-- a game designer could explain it to you verbally in about an hour or so.
And that just wastes a lot of time, especially in a class like this when you have these very tight deadlines to be able to get something working.
You don't really want to be spending a whole lot of time talking things out.
You want to get actual answers as quickly as possible.
And one thing nice about prototypes, especially on the multi-person teams is that can make multiple prototypes.
It just has a whole bunch of different ideas out.
If solution a is better than solution b, well, instead of arguing about-- well, theoretically solution a is better than solution b, why not just prototype both of them and just play them.
You'll get an answer very, very quickly among you, and all of your teammates can see.
And it makes it special and actually more fruitful because you're working with evidence, as opposed to working with just theoretical notes.
Another thing about prototypes is that, again, prototypes are not your shipping product, which means you should always be willing to throw it away.
The less time you spend making a prototype, the easier it is to just discard it.
With looks, the uglier and sketchier it is, the easier it is to abandon it.
And you need to be able to abandon prototypes.
You need to be able to say this just isn't working, and I'm fine with that-- the whole team is fine with that.
You spent 30 minutes taking this thing, we can afford to lose those 30 minutes.
Which means you need to be making stuff either really, really fast and really, really shoddily.
You don't want to be spending a lot of time making a polished prototype.
So your goals that I am setting up for you when you're going into prototyping-- today we're actually going to start prototyping the games that you're going to eventually hand in for Assignment 1.
We need to find the [INAUDIBLE].
All of these game mechanics, the mechanics that the teams have already chosen, and the mechanics that the teams will end up choosing-- all of them have fun and un-fun implementations.
I can think of a whole bunch of ways to make building go at a really plodding pace, to make it so that you can never really get any progress, and things like that.
And you can come up with really, really nice implementations that are just going to engage everyone around the table-- they're gonna have a good time.
Fun doesn't necessarily mean everyone's happy.
It's like a game where, deception for instance-- it's like you're playing around with that mechanic, and you feel like you're just being an asshole to other people.
But that's what the game's actually about.
You are trying to be deceptive to other people-- maybe not necessarily without them realizing it.
But then, [INAUDIBLE] experience, that engages you and sort of puts you in the persona of what the designers were trying to achieve.
Then, that's engaging.
That's good.
If it's something that puts people off from ever playing the game again, then, you might want to re-evaluate that.
But there is some value in games that you're only gonna play once.
I'm not going to be very dogmatic about that.
What I want you to do throughout the prototyping process is figure out, of those game mechanics, what are the fun and engaging things that you can do, and what are some of the less fun implementations?
If you don't find anything that's not working, then, I don't think you're looking hard enough.
You need to be looking really hard to-- What you should be finding is whole bunch of things that don't work, and a few little gems that do.
The other thing that I want you to do with the prototype is use the prototypes to communicate to the rest of your team about where your ideas are going.
As you work in your team, if you want other people on your team to understand the ideas that you have in your head, try making a prototype to communicate that.
This is this thing that I did last night, I bring it into the team meeting-- together let's play this for like five minutes.
And then, you'll understand what I think is interesting about voting, for instance.
And your team may have a completely different idea about what they heard, about where they wanted the game to go, but this is a very effective communication tool.
When you're actually designing games, especially for assignment 2, and assignment 3, we are trying to hit some sort of desire, external aesthetic.
The final assignment is going to be for our client's needs-- then, you are working on some sort of external spec.
But then, you also need to communicate within your own team on how you're interpreting that spec.
That requirement, that request, from external party in this class will be on structures.
Now, if I say make a game that is going to get you to a certain aesthetic, which is assignment 2, then how do you think the team should start even proceeding in that direction?
Communicate that using your prototypes.
It can be very, very difficult to get those ideas out otherwise.
The third thing that I want you to do with your prototypes is to take it outside of your team.
Today is going to be very easy because we have a room full of people who are hopefully eager to help each other out.
And they're going to end up playing each other's prototypes by the end of class.
That's only gonna last for one class because, as of the end of today's class, all of you are going to know too much about each other's projects to actually be good testers in the future.
So I want you to take it to people outside of class-- your dormmates, your friends and family.
Email them stuff if they're at home-- take it to the library or the student center or something.
Offer people free tacos or something to play your game for five minutes, that sort of thing.
It's very, very important to make sure that you're getting feedback from people who are not on your team.
Of course, the instructors-- it's not just gonna be us.
We're gonna grab people from the game lab to come in here and play your games and give you feedback of them.
So occasionally, we'll just bring in new people who haven't seen your game before, but don't count on us doing that.
You should be doing that as part of your homework.
That is the process of prototyping.
OK, before I go down that shopping list, any questions so far about what we're trying to achieve with prototyping?
OK. Again, Assignment 1 is going to be a lot more about prototyping than making the full game.
Investigating one single game mechanic is something that you're probably actually gonna end up doing for both Assignment 2 and Assignment 3.
Because it's like, hey, this is the way how I think we can fulfill the requirements of the assignment, and then, you're gonna investigate several different mechanics to get to building your final game.
So think of Assignment 1 as the prototyping assignment.
Let's talk about what we've got for you today-- things that we recommend for prototyping include large sheets of paper.
Here are a couple of preprinted maps-- there's a hex grid on one side.
There's a regular horizontal and vertical grid-- is there a word for it?
Cartesian?-- square grid.
These are, I think, two centimeter squares, which is also the size of some of the wooden blocks that we've got.
I think the hexagons are also two centimeters if you to take them horizontally.
So you can print these.
These are all actually available in PDF on our class website-- the prototyping maps.
There's a second version of these maps that has marks.
It has just a bunch of lines in bold.
It's exactly the same map, but it has a few things boldfaced to help figure out what a square in here actually looks like, if you want to use a [INAUDIBLE] track, or something like that.
I like these better.
I know Rick likes the other one better, so we put both up for you to download.
Let's see, what else do you need?
Dice.
There's a big box of dice, 6-sided dice, 12-sided dice.
Has anyone ever used a 12-sided dice?
Really?
What are they good for?
[INAUDIBLE]
[INAUDIBLE] does d12s?
Yeah
OK.
I know they do that d20s a lot.
But, OK. 10-sided dice, 8-sided dice-- If you ever buy dice for yourself, a tip is to get them from kindergarten suppliers, rather than from gaming stores.
Gaming stores will probably cost about 10 times as much.
This box cost me about $12, plus the box.
You get a nice little [INAUDIBLE], Dice are good for randomizing, obviously.
I do not suggest using dice to keep track of numbers.
So say you've got a number that increments anywhere between 0 to 20-- sorry, 0 to 19, or 1 to 20.
Don't use a 20-sided die to keep track of the thing because it's very easy to lose that stat just by a flick off your hand.
Just grab a piece of loose newspaper or something, and just write down that number-- if you need to keep track of stats.
Don't use dice to keep track of stats.
But dice are good for randomizing, they can be used sometimes as tokens that move around in a pinch.
So you can use dice to do things, like one die is the tens, and one die is the ones.
So I rolled two 10-sided dice, and then, it's going to give me a number between 1 and 99-- no, 00 and 99.
Yeah.
Index cards-- we've got white ones in here, we've got colored ones in here.
They sell index cards that are a little bit closer to playing cards size, but remember the rule of keeping things big-- big sheets of paper.
I'll go in a little bit more detail of why you want to do that.
But if you're making like a card game, I would actually suggest using index cards.
They are a little bit harder to hold in your hand.
It's difficult to keep a whole bunch of index cards in your hand, like a fan
They're hard to shuffle too after a while
Yeah, they're hard to shuffle as well.
But for prototyping, not only are they cheap-- we talked a little bit about doing things cheaply and very disposable.
Those are larger because it's easier for you to actually see when you're prototyping.
And I'll get into some value of why you want to-- a few more reasons of why you want to be keeping things as big as possible.
Post-it notes are not in here.
They are in those boxes.
Post-it glue and notepads-- by notepads, I mean stuff like this.
Gets people to keep track of stats.
We've got pencils, and we've got markers and pens in there, as well.
Post-it glue- if you haven't see it, I'm pretty sure it's in one of the boxes-- it looks just like a regular glue stick, but it's blue.
I might only have some in my office.
Basically, it takes any piece of paper and turns into a Post-it.
It makes it into sort of a restickable piece.
And not only is that useful for the brainstorming phase, where you can take index cards and turn them into Post-its and stick them up in a wall, they're really handy for prototyping because you can lay things out, say on a desk or on a sheet of paper.
And it won't just go flying if somebody sneezes.
You can sort of keep things in place.
If you've never done prototyping for, say a user interface on a piece of computer software, it can also be really, really handy.
Because you can cut out pieces of paper that are exactly the size of your menu bar or your window, or whatever.
Just use the glue stick, make them replaceable and sticky, and you can place them anywhere on another sheet of paper.
Pencils, pens, markers, scissors, tape-- that should be obvious why you want all of those things.
Gamebits-- some of you saw some of these in last week's games.
These are sort of stackable counters.
We have the cubes.
You can also dice for gamebits.
You can also use pieces from other games.
You don't necessarily have to restrict your ideas to what we give you.
Those, I believe, are rubber animals-- often end up being used in tactical combat games, for some reason.
I don't know why people always want to tape the [INAUDIBLE] duct tape, [INAUDIBLE] right.
Just don't stand up, that's the problem.
That is the problem with our-- we have a whole bunch of little rubberized animals, and they don't stand up very well.
They do tend to tip over.
We also have a whole bunch of rubberized vehicles, and those tend to be a little bit more stable.
So keep that in mind before you decide, oh, I have to use this chicken piece.
Kindergarten counters, things I use to teach kids how to count, make great, great covers for your design
[INAUDIBLE]
Oh, you mean, like the ideas right there-- different colors on your side.
So you can use them for currency in your game, for keeping track of points-- another way to keep track of a status-- just give people pieces that help them keep count.
Your phone camera is extremely useful in keeping an archive of your work, of keeping track of your game and play, keeping track of who has what hand at any given time.
It's really, really easy, and a lot of phones now have a resolution, that you can sort of reliably use them to keep a record of your work-- way easier than trying to start everything on the photocopy machine.
So I used to recommend using a photocopier, but now, just take lots of shots with your phone camera while you are working.
So you want to be keeping your prototypes rough.
You want to be using hand-drawn materials, trying not to immediately go to opening up a Google doc and creating a spreadsheet, or anything like that.
Start writing things down.
Again, you want to make a bunch of cards-- just like writing stuff down on index cards.
So if you want to start making a map, just start using a marker and drawing it on the grid.
You want to keep it sketchy, and you want to keep it large.
And you don't want to be using too many inks.
Just use one dark ink, and run with that.
The reason for this is because you don't want people to be giving you feedback on how your game looks.
If they do say something, wow, this looks like crap, that's fine.
And just move on from there because that's not the feedback you were looking for in the first place.
If you start using lots of different colors, everyone will start talking about your color scheme-- maybe there should be red, maybe there should be green.
If you start making things, say printed out from a laser printer or something like that, people are gonna ask, wow, is this final artwork?
It looks not very good.
Or if you take the trouble of using colored pencils, for instance, to nicely render an image on your cards.
And people say, wow, this looks great.
You hand-drew that, but then, now, if you wanted to alter it, it means you're going to have to go through all that effort again to draw a new picture.
And that takes time.
That takes more time than you need for this thing.
You want to keep things sketchy to sort of convey to your testers that this is a work in progress.
If somebody sees something that actually looks very, very nice, they are going to think that you're close to final.
And they are going to be a lot more hesitant in giving you drastic feedback.
Things are going to, sort of, require drastic changes.
But if it looks like you've spent half an hour, maybe 15 minutes, just sketching stuff on bunch of cards, you'll get feedback from testers-- stuff like, I just don't like any of this, or like, maybe this is the one thing I like, but everything else is just crap.
But that's the kind of feedback that you want to get at the prototyping phase.
And keeping things sketchy can sort of encourage people to give you that kind of feedback.
I definitely have lecture notes, but they're not showing up on my screen.
And so, I'm a little bit off right now.
Here we go, OK.
So the other thing is-- this is actually a 608 class from way back.
The other thing that I want you to do is keep iterating over and over and over again.
Yeah?
So you said like, earlier on, after this class, we probably won't be able to [INAUDIBLE] much about the games.
Does that also mean we should be changing our prototyping outside this class?
Find a new group everytime we realize a product?
Absolutely.
The next time we do a prototype-- a playtest in class, I'm going to specifically say if you try to find a game where you don't know anything about that prototype before you start playing the game.
That's where the feedback is going to be the most useful.
You don't want people to come in with preconceived notions based on people's prototypes because your prototype may have changed completely from the last time that they saw it.
But then, they're going to come in thinking that your game is like some sort of natural progression from that previous idea.
And they may respect your ideas of what kind of strategies are used.
If they already understood the rules-- what you verbally explained to them on day one, then they can't give you any useful feedback on how well your rules were written because they already know the rules.
So when they read your poorly written rules, they can't tell you it's poorly written because they already understand it-- that sort of thing.
So yes, always try to find new testers.
So the purpose of iteration is to just repeat this over and over again.
You start with a question that you're trying to answer.
The broad question of Assignment 1 is what are all the different things that you can do with this mechanic?
But say, I know someone suggested auctions on Monday-- what can you do with a Dutch auction?
How does a Dutch auction actually work?
--that sort of thing.
You want a question that, not only is a clear thing that you can actually test, but also you can set criteria for what will be a successful test or an unsuccessful test.
The question might be very specific, like this game is too long.
Can we get this to run under 20 minutes?
Can we get this thing to run under 15 minutes?
Well, that's a falsifiable question, right?
If the game play actually took more than 15 minutes, then it was a failed experiment.
And if it took less than 15 minutes, then it worked.
And you want to be able to go into the process asking, what is the thing that we're trying to solve?
--before you start thinking of potential solutions.
How many of you have heard of axiomatic design?
It comes from McKee, I think.
The theory behind axiomatic design is that, for any potential solution to a problem, it needs to fit a certain set of criteria.
So you come up with a bunch of axioms, which you just take for true.
So certain axioms that you might come up with a game would be-- this game can't take more than five minutes, or this mechanic can't take a player more than 10 seconds.
So that axiom could be something like, we don't want the player to have more than five cards in their hand.
These are things that aren't necessarily always the right answer, but you're going to, sort of, set these criteria for yourself for a given test.
And then, you start thinking about all the possible solutions to get you there.
And if one of those solutions meets all of your criteria, or all of your axioms, then that's a successful test.
If not, then it's an unsuccessful test.
So a question in your game may be like, can a player execute this mechanic in more than one way?
Or will a player execute a certain given mechanic in more than one way?
We've given them three different ways to do it, but if they keep doing the same thing over, and over, and over again, then that will be an unsuccessful test.
OK?
Once you've got a question, you can start designing for that, right?
Maybe the designing involves something very small, like I'm just gonna tweak numbers off rules that we've already returned.
It might be, we've got to rewrite half of our rules, or we've got to throw out this rule.
Or maybe, we're gonna rearrange the order in which these rules are going to be executed.
That's all design.
The trick is to do it fast.
The word rapid is for a reason.
If you are spending a lot of time discussing about what the right answer is, just start designing two prototypes-- or more prototypes to sort of test out all the outcomes.
Anything that takes a long time to kind of get bumped down in discussion-- it's actually wasting time for your team, when you should be prototyping.
Because they're gonna learn a lot of things about your game on the side, besides the question that you're asking.
If you have a discussion, you'll probably only-- if you do stumble across the correct answer, you're only gonna get the answer to that one question that you asked.
Make more prototypes to answer your questions, rather than try to talk them out.
Then, you do a playtest.
And that's the second part of my presentation, which involves the playtesting phase.
But basically, you've got a bunch of people who don't know how this game is gonna play out.
First, you will probably end up playtesting within your own team, just to make sure that everything makes sense.
And then you take it out to somebody outside, maybe someone else in the classroom, to see how they respond to it.
And then you look at the results of that playtest-- did it address the problem?
Did it give us any information towards the question we were asking?
Maybe it was inconclusive-- you need to do another playtest.
Maybe it indicated that we were in the right direction, but the changes that we made weren't drastic enough, or maybe were too drastic and [INAUDIBLE] down.
That's when you make your revision, and then you repeat the whole process again.
You can improve the quality of your question, be more specific.
You might stick with the same question and just do a second version of design to it.
You just want to be repeating this over and over again.
The more times you get to do this, the more refined your prototype is, and the more refined your final games are gonna be.
This is the same process, whether you're making a prototype or whether you're making a full-blown board game or card game or computer game.
The more chances you get to iterate on something, the more refined it's going to be.
You don't want to keep changing.
Here are a couple of tips that-- actually, I will get back to this later after you've actually had a chance to prototype once.
Those are like tips for how to get out of rut.
So let me talk about this instead-- keep track of all the rules.
Write your rules.
You can write your rules on cards, which makes them very easy to rearrange, to discard, to say-- all right, we're not playing with this rule right now.
But then, maybe you can reintroduce it later.
So you can use the index cards for that.
You can rearrange them to rearrange how they end up getting played.
If you change a rule, update your card.
It is something like, I'm going to just change a number.
You can just do it right on the card-- if you're actually changing the way how a rule works, write it out on a new card.
Take photos with your cameras, and try to simplify your rules to the point where you end up with like a minimum set, in order to make a certain prototype playable.
If you have too many rules operating at once, it can be sometimes very, very-- really, really confusing to figure out where everything is going wrong.
It's a lot easier to add new rules than to take them out, which is why I place the emphasis on taking stuff out.
Because if I remind you to take it out, maybe you will do it once in a while.
So that's going to be the process of prototyping.
We're going to start having all of these materials.
People who haven't figured out your teams yet should be having a discussion on what mechanics you guys want to work on, and how are you going to split up your teams.
People who know what mechanics you're working on, or maybe are trying to decide between two mechanics, you can start splitting your team into two and working on two separate prototypes, for instance.
And the goal is to have something that somebody outside your team can actually play by the end of class, or more accurately, by 3 o'clock.
Because we are going to go into playtesting at 3 o'clock.
And around 2:30, I'll go back to the slide-- to give you some ideas on how else you can change your designs to be able to help you get closer to your design goals.
But right now, this is what I want you to do, so we're going to start handing out some of this material
[INAUDIBLE]
I'll be OK with the team moving up there.
[INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
[SIDE CONVERSATION]
Before we go into the test, there's a couple of tips that I want to give you.
Some stuff for you to take notes on.
But you don't really have to do this today.
At some point of time in your prototyping you might end up getting stuck.
And you're not quite sure what to change.
So [?
if I ?]
would have been asking you to do things like amend rules, right?
Replace rules with other kinds of rules, kill rules that aren't working.
Some other ideas that you might want to play with, if there is a resource that's currently limited you might want to make it unlimited.
And vice versa.
And I'll say the number of steps that you can move, the number of currency that you have, make it unlimited or you make it limited.
It could change the way how your game feels a lot.
[INAUDIBLE] with other players.
And maybe even sort of messing up the types of decisions they get to make, or maybe even the order that they get to make it.
Could be something as simple as reversing the order of who goes next.
But it could be more significant, like forcing another player to make a move that they don't want or taking away their options.
You can mess with the play order.
Like I said, you could reverse who goes next, but then also the order in which certain rules get played out.
Could be something that you should be always trying to do with your own rules.
Do your rules has to be executed in the same order?
In fact, the answer is no.
You can probably rearrange your rules and get a very different effect, even without writing a new rule.
If you're going to deal with numbers at the prototyping phase, generally you don't want to be making small changes.
You don't want to be making 10%, 20% changes to your numbers.
You want to be doing something like changing things by 50%, maybe doubling it, maybe halving it.
At least change of 50%, but multiplying and dividing by two.
This will give you a much better sense of whether that change to the numbers is going to be the thing that's going to solve your problem.
If you get the numbers wrong but the changes go in the direction that you want, then you can dial back the scale of the change.
Maybe you have a variable that was too drastic, but it started achieving the things that you wanted.
And so, OK, all right.
Maybe sort of halving that number would make it 25% of that number.
As an exercise, once your prototype is already starting to work one trick of trying to simplify your prototype is to try to identify the very few rules that are necessary for your prototype to work.
So you take all the rules out and introduce rules, then, one by one.
The rules that you've already got.
And you try figure out that bare minimum.
That's finding the core, and that's the core of your prototype, that's the thing that you might want to carry into a larger game project in the future.
Finally, throw away.
In fact what I'm going to encourage you, at the end of this class today, is to make a completely new prototype from scratch.
For each person on your team.
But I would suggest as an exercise is just go back to the mechanic idea and see what else you can do with that mechanic.
Whether it's stealing from other people or hidden information.
All of your prototypes right now probably test one kind of way to interpret that mechanic.
What else [INAUDIBLE] completely different ways to interpret exactly the same mechanic.
Make a prototype.
You know, spend about half an hour to an hour.
You only spent an hour on these prototype so far, and a lot of it was figuring out what mechanic you were working on.
Right?
So try to build something, test it with me or test it on your teammates.
It would be great if you had a weekend meeting, for instance, to go into that meeting with as many different prototypes as you have team members.
You just play each other's prototypes and have a conversation.
These are all different ways to approach the same problem.
So [INAUDIBLE] basically and make something new.
And then revisit what you've got at a later stage.
So that's some tips for you to do in the following week.
But in the next hour this is what I'm going to be asking you to do, and that is to do a play test.
Now, there are many different ways to do a play test.
This is probably what they're going to be doing today.
How many of you have made a game that's really only for one person right now?
All right, no one's made a multiplayer game.
So you're probably going to do a play versus player test.
That's fine, that's what we asked for in the assignment.
You can do multiplayer tests for cooperative and competitive games.
You can have [INAUDIBLE] player tests where everybody gets the same set of rules or different sets of rules.
So the [INAUDIBLE] symmetric or asymmetric.
The trick with multiplayer play test is that they tend to be very, very loose.
It's very hard to do, say, a really, really tightly controlled experiment.
One thing that often happens is that when you have people playing a game and later on in the semester you're going to just give them a set of rules and they're going to have to interpret it.
They might interpret it in a way differently from how you intended.
They may come up with-- they may negotiate the rules based on things that they don't quite understand from the rules and come up with a solution that wasn't necessarily what you expected.
Communicate openly.
They'll talk about, like, hey you know is that rule-- do you really want to move there?
You know?
And things like that.
One thing that you can do is to have the person who is actually running the playtest, someone from your own team where you're trying to get information, you will explain the rules to people at the beginning, in early playtest.
In later playtests we just hand them rules.
But if they come up with house rules, if they decide to interpret your rules in a certain way, I would actually suggest not stopping them right away.
Let them play out a little bet.
Because they're giving you a free iteration of your game that you may not necessarily have considered, and may end up working.
It might work out great.
But then you cannot-- But once they play a little bit of that and then you get a sense of how that works, take note of that, make sure that you know how that is supposed to work out.
And then explain the rules.
No, actually.
I'd like you to try playing it with the rules interpreted this way, which is the way how you actually designed.
And then you can get a second player test of the same group of people.
Let's see, what else.
This is absolutely necessary for multiplayer games.
You are going to have to do that.
[INAUDIBLE] If you go to other classes and make prototypes for digital games on paper, you are going to need to do this sort of playtest.
So you're going to get some experience doing that right now.
I do want you to know that there are a couple of other ways to do playtests.
The Wizard of Oz test where somebody-- the person that you're inviting to play test is basically playing with somebody who's on your design team, and that person is playing the computer.
This is very good for prototyping a digital game, especially a single player digital game.
You can be very constrained on what information you provide the player.
In fact, you don't necessarily even need to provide a player a full set of rules.
You can just say, this is the computer screen, your finger is the mouse, or something like that.
Or even giving them a simplified keyboard.
Up, down, left, right or something like that.
That's how you communicate with the computer, you just push, or you point, you click by touching on the screen.
The trick is to make sure that the player is-- that you're not giving clues to the player about what the computer is thinking.
The computer should be communicating primarily through the things that a computer will show you.
Either images, sounds, maybe numbers that change.
But the computer doesn't actively explain the rules.
If the game was supposed to explain the rules, you should have little prototype pieces of paper that you can hand out in front of the player.
Now say the player is completely confused and you think, this would be a good time to introduce a [INAUDIBLE] test.
Oh crap, I actually didn't have a [INAUDIBLE] test prepared ahead of time.
Well, grab a Post-it pad and write it down, slap it in front of the player.
Now that's [INAUDIBLE].
So you can be-- this can be a very, very deep prototype.
You can actually test a lot of different things in a computer game on paper this way, depending on how much time you want to go into producing something to be tested.
You don't to spend too much time because, remember, prototypes are supposed to be disposable.
They're supposed to be fast and cheap.
So it's a Wizard of Oz because it's like the Wizard of Oz.
It's somebody behind the scenes, manipulating what you're seeing.
But there's no actual computer there, it's just you.
You have to be very, very constrained yourself when you're running a test like this.
You don't want to deviate from the algorithm that you set up for your computer, if at all possible.
So if you've got some tough AI character, you shouldn't be making human-like decisions for that AI.
You should come up with a bunch of rules for that.
This is another kind of prototype that you can do.
Not for today's exercise, but later on in the semester.
If you decide to do a live action game or if your games really about just a person walking around in a space, even if it's going to be like a top down board game.
You could just do a live action game where you just have people actually walking around a real space.
You explain the rules to everybody who's involved.
Maybe your game's for five people and you explain the rules to all five people before the game, and then you let them go.
The problem, of course, is that they're all going to interpret those rules their own way.
And because they may be physically separated, they may not be negotiating with each other and as a result may be not even playing with the same set of rules.
That's also a problem.
It's difficult to monitor what's happening with a lot of people walking around a space and doing their own things, like individual agents.
But it's really, really fast to prototype because you may not even need to write anything.
You may not even need to produce any sheets of paper, you just tell people here's a bunch of rules.
It's like setting up a schoolyard game.
And so if your games about, here's a character moving around a space or if you're [INAUDIBLE] trying to actually make a live action then this is something that you can also do.
This is just as valid a prototype that anything that you're doing with paper and cardboard.
So here are the rules.
There's going to be people who are going to play.
I think we have five teams.
One, two, three, four, five.
Right?
So that's a little tricky.
How many of you have two player games?
Ours has two to four
Two to four.
How many of you have three player games?
One, two-- three to four.
[INAUDIBLE] four.
And how many players?
Four.
Yours is a four player game.
OK.
So there's three, three.
Yours is three to four
[INAUDIBLE]
Oh, actually.
That's a good idea.
OK, all right.
So we're going to grab one more MIT Game Lab staff member to come in and will help test whatever team is not getting a chance-- that doesn't have a team available to test.
We'll probably jump in on this game.
So can I just check the two teams on my right.
You're both three person games?
All right, you test each others games.
And then this group here, the two to four player games will test each other's games.
OK?
So you're going to test one game.
[SIDE CONVERSATION] So, to clarify.
You're going to test one game, and they are going to shift over and test the other teams games.
We're not going to do it simultaneously.
OK?
People, I'm not done yet.
I'm not done yet.
OK.
So we're not going to test games simultaneously.
You're going to actually-- we're going to test one team's game at a time.
So, say, the two groups on my left.
If you are testing your four player games, we're going to play one groups four player game first.
Then everyone's going to move over and play the other game.
And the reason for that is because the team that designed the game, even though they're not playing the game, has a job to do.
Someone's got to be the facilitator, sometimes.
In a Wizard of Oz test that person is often also the computer.
and that person's got the job of explaining the rules, right?
Everyone else has got to be an observer.
You should have a notepad, you should have something that allows you to take down notes really, really fast.
You should be like writing down everything you can about what you're observing.
In particular, keep an eye on the faces of the people who are playing the game.
It's too easy to just get mired and try to record everything about game state.
It's more interesting to-- it's more important to also pay attention to whether they're engaged or whether they get confused, whether they're being frustrated, or whether a particular game interaction-- what's interesting to them.
You can tell a lot just by looking at someone's face.
So, the observers and the facilitator, once you're done explaining the rules really shouldn't be talking all that much.
In fact, once you get onto the later stages of prototyping, when you have written rules, you shouldn't be talking at all.
You should be leaving it to the players to actually figure out the rules on their own.
And the reason for that is you don't want to bias your results.
You don't want to accidentally suggest good strategies, for instance.
Saying that no, you really want to be moving this first.
Well, maybe you should make that a rule instead of making it a suggested strategy.
So take that out-- One's taking notes, practice being as quiet as possible.
This is a small room, it's very resonant.
It gets pretty loud.
You can have a discussion after class is over or after the two play tests are over about what you recorded and we can discuss what you're going to do with that information.
OK?
So I think the three of us will be playing this game.
And you'll need to explain to us the rules and take down notes.
All right?
If you need note taking material, there's plenty of pads.
[SIDE CONVERSATION]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio lecture is not covered under our Creative Commons license.
Please listen to the end credits for more information
OK. All right.
So today's reading was the "MDA" paper by Robin Hunicke, Robert Zubek, and Marc LeBlanc.
I hope you found it pretty light reading because it was intended to be pretty light.
And the whole point of it was that we will be getting through this pretty quickly today so that you have time to sit with your teams, talk about your project, work on it a little bit.
All the prototyping materials that are in front I'm going to be putting the boxes on the tables.
Are you more or less seated with your teams right now?
More or less?
OK. Every team represented-- let's see.
You are the building team.
You guys are the-- which?
I'm also builder
The different building team
The LEGOs, yeah
Right.
The LEGO building team, OK.
Which team is that?
[INAUDIBLE]
The what?
[INAUDIBLE]
The head of the information team
Thievery
The thievery, stealing stuff?
The pap building.
Pap building
Oh, you're the pap building.
OK. All right.
So, all right, cool.
I'll type it here, great.
Hopefully we'll be able to make a little bit of progress on your teams on your projects.
So who managed to play a little bit more of the games, whether it was the prototype that you played on Wednesday or whether it was something else you developed along the way.
Who managed to get some playtime?
OK, a few people.
Who did you play it with?
So we went back to the drawing board and redid new games

Exploring.
So we just played them amongst ourselves
Among yourselves?
OK. Who did you play with?
Just different people in my dorm
OK. [INAUDIBLE] basically, right?
Anyone else?
Did anyone else do?
Yep
I mean, we went through a couple more iterations--
Just within your team?
Yeah.
PROFESSOR Yeah.
OK. Did you play an iteration that you didn't develop yourself?
Did anyone?
It's like, if you played within your team, obviously-- OK.
So if you played a version that you hadn't developed yourself, or if you played it with other people who were developing the game with you, how do you-- what were some of the things that you felt about the game?
Or what did people say about your game once the game was done?
What did people seem to like or not like about it?
What were some of the things that they were saying
It didn't end
It didn't end.
OK, all right.
What else?
It could have been negative feedback.
It could have been, wow, this is terrible because, blah.
What was the blah?
Or it could have been, wow, that was kind of cool because of blah.
Nothing?
Nobody gave you any feedback?
They just kind of sat there, passively, while you're making them play this?
[LAUGHTER] OK, well, I was hoping that that was a segue into a little talk about aesthetics, actually.
Maybe what I'll do then is, I'm gonna make you play a game.
A shot game.
The game that you will see-- who's played it before?
The game on the screen?
Yeah, Oasis.
Also note it's Defense of the Oasis, which is easily confused with DOTA, but it is not DOTA.
All right.
If somebody would like to volunteer to play through an early easy level while there's a lot-- you know what?
You all get a chance to play it through one round, because each round's pretty fast.
So we'll start with you and then we'll go on then, OK?
And then, everybody else pay attention because I guess he's pretty much playing the tutorial level.
So this was only designed for the PC.
It was actually one of the-- came out roughly-- you could just go ahead and play-- it came out roughly about the same time as casual games, downloadable, casual games, picking up on the market.
I don't think it was ever sold as a boxed product, or if it was, that really wasn't the point of its business model
There were people following me
So you click around
I can probably figure out.
Oh, this is really the point of figuring it out.
That one.
That happened.
[LAUGHTER]
I just keep walking.
151
Already searched.
[MUSIC PLAYING]
Oh, it happened again Already searched.
[MUSIC PLAYING]
You should do something clever.
[MUSIC PLAYING]
So notice the trends are going down
Yeah.
[MUSIC PLAYING]
Water.
[MUSIC PLAYING]
They like music, don't they?
[MUSIC PLAYING]

Big shiny thing
How do we increase population?
[MUSIC PLAYING]
Yay.
[MUSIC PLAYING]
Throw that towel
Look, notice this guy is down there
I am watching you.
Oh, hey, hello people.
[MUSIC PLAYING]
So this actually turns out not to be the tutorial level.
This just turns out to be the first level
The way most people play-- why don't you use the tutorial?
That's true.
People don't like playing tutorials, that's true
So we find these four technologies.
The barbarians are here?
Why are they-- no, they're not
You have 22 turns and about two barbarian hordes are arriving
So basically, movements is turn, and I'm just walking
Yeah
So people-- I should walk to an area I've already discovered?
Over here.
Oh, I did something clever!
I don't know what I did.
[MUSIC PLAYING]
[INAUDIBLE]
I don't know what to do.
I'll just walk here and watch things go bad
Barbarians are coming.
Capture the city.
Move troops there
Move here.
Yay, 94 people.
Can I move other people?
Try
Try
No more troops.
No more troops.
No more troops.
We're all going to die.
333.
[LAUGHTER]
Oh, I thought this was supposed to be me learning how to play
Yeah, learning.
Feel the learning.
[LAUGHTER]
You have to use your technology
Yeah, can you use any?
I'm tapping on technology.
I'm tapping Hannibal.
I'm tapping spear.
I'm giving it all my love, and it's not doing anything.
[CRASHING AND YELLING]
I think I won.
[LAUGHTER]
Come on, man, you got this.
Kill her
Whoa.
What did they do?
Oh, snap.
[MUSIC PLAYING]
What?
OK.
So you did, in fact, complete the level.
But you're probably [INAUDIBLE] about what happened here.
How do you feel at the end of that level right now?
Like I'm good at touching squares, but that doesn't mean much.
Like all I knew was if I found cities I'd tap them again and I could search

That's all I found
OK. That's what you figured out about the rules, but how do you personally feel about this game right now?
I mean, you only played one level, and clearly the game goes on
I feel like it's somewhat impossible to control the ending situation because I have no way of determining or upgrading my population
OK. All right.
So [INAUDIBLE] control right now
Yeah.
Or like I have no influence, so I'm just one dude exploring the map
OK.
So lack of influence maybe.
Lack of influence, or possibly lack of control, something like that.
All right, so next person.
We're going from right to left.
So you're next.
You're going to play level two, taking in everything that you've seen so far
Oh, crap
Like you saw, you can tap on purple areas that are--
Purple area to explore-- so I tap that
Anyone has-- any ideas about what he should be doing, go ahead and say so.
He doesn't have to play this on his own
So the question mark--
I'm doing this.
I want to discover every square
Just go to the--
Now that you know-- double-tap cities
Oh, double tap cities?
Yeah just tap them again, you'll search them.
Ta-da!
Oh, you got-- [INTERPOSING VOICES]
That's probably good.
I don't know if you asked him
Go to the question mark!
He'll get there.
He'll get there
Should I go save the question mark?
Yes
But then I'm gonna [INAUDIBLE] [INTERPOSING VOICES]
Every movement is one turn no matter how far apart it is
Oh.
[INTERPOSING VOICES]
--do it in one straight.
You have to go one purple square at a time
Yeah, but say I wanted to skip all the squares [INAUDIBLE]
You're gonna wait one turn where you discover it anyway
Yeah, it doesn't matter
Oh
You don't use a turn if you're not discovering?
Oh, look at that-- discover right beneath it.
That mountain does something.
I don't know what it does but it gives you ten people
What do people think that does, where he put 20 people?
When you click on the mountain, what do you think happened?
It's not about 20 people.
[INTERPOSING VOICES]
--technology
OK. Go right ahead.
Do whatever--
You can put in a lot of people
I don't know if that's good or bad.
I wait turns
[INAUDIBLE]
We're not seeing people form at this stage
Keep running around
Whoa
Yeah.
See, look.
Now I'm still one turn
Wait, do I lose followers when I do that?
[INTERPOSING VOICES]
We're investigating
Yeah, I do
Oh!
But I got something.
I got archers.
I like archers
Clearly because you have enough people.
Bows increase your chances of afflicting damage on the barbarians.
Good.
Keep doing that stuff.
I wonder-- it looks like there's--
Whoa, I'm huge.
[INTERPOSING VOICES]
Oh, look at that!
I don't know that means but you found another technology
Looks like-- I have no idea what that-- oh, helmet!
[INAUDIBLE]
Oh, another city.
Greetings.
[INAUDIBLE] I'll join your cause.
That means this is supposed to be a campaign game and last over a long period of time
What's that thing right there?
What thing?
The tent looking thing.
[INTERPOSING VOICES]
Yeah, you can't double click it so you just wait--
Over there?
--until the end of the turn
Damage.
Dude, good, another city
[INAUDIBLE]
How do you add people?
I should have searched more
Kevin, what else is falling apart?
Oh, look that's the animation that [INAUDIBLE]
Oh, you got more technology than I did.
Very nice.
[INTERPOSING VOICES]
Predict routes in the city.
[INTERPOSING VOICES]
--half that they're going to say?
Two [INAUDIBLE]
[INAUDIBLE]
Oh
They survived
Helmet for falling rocks
All right, here we go.
Let's do it!
Go, go, go!
Aw
Boo!
You should not put so many people on the line
Oh!
Barely survived
I won?
Yeah
So how do you feel about that?
I feel awesome, but I have no idea what happened

The greater the number of cities that are connected together by roads, the faster they all grow.
That dude built roads, and we don't know how
Maybe he--
Wait, did he actually?
Yeah, he had roads before
Did he?
Yeah, you saw him light up and connect.
All those things were basically roads.
You had the dudes moving along it.
It's population increased in size.
Our population has yet to increase in size
I think you [INAUDIBLE] before class about it
Yeah
OK.
So you feel awesome, but you're still not sure what--
I still don't know what's going on.
I do know that exploring is very good

I figured that one out
So there's some sort of exploring.
It's good for something
I got zero points.
[INTERPOSING VOICES]

And I got zero, too, but mine went from negative above
There's still a lot of confusion going on, OK. All right.
[SIDE CONVERSATION]
I think it's showing you how the points are added.
[INTERPOSING VOICES]
So how was that round?
It was fun.
I wish that I could see-- I think there was a way to see which city they were going to attack first.
I didn't know what it was.
I forgot.
Also, I feel like it's sort of pointless that you have to click on a city again to explore it.
I always want to explore it, as far as I can tell
OK. All right.
So you feel that there is some redundancy going on?
Oh yes
OK. All right.
All right, cool.
Thank you very much.
So these are kind of the things that, as players, you know, the sensations that you get.
Right?
You get kind of the joy of exploring, finding out new things, pushing back the fog.
And the beginning of learning again, this confusion, this befuddlement.
You don't actually know how your actions are changing things.
But if you play a lot of games, you are usually sort of primed to accept a little bit of that.
And sometimes it's balanced well enough that it sort of intrigues you rather than puts you off.
But sometimes it just puts you off.
Someone said that it felt good to save people.
You described it as feeling kind of fun.
In the paper there are some different ways how we can describe fun.
It's just eight different types, but even in the paper it states it isn't meant to be a comprehensive list.
It isn't meant to be a list that's exhausting every single option out there.
It's just to show that there are different kinds of fun when you play a game-- like exploring things, or inhabiting a role, or overcoming a challenge, or just passing time in a way to alleviate boredom.
Sorry, I have a bit of cold so my speech is a little bit stunted today.
There are a number of game mechanics in here that generate some of these sensations.
I want to put aside about confusion and lack of other influence right now, because some of that's addressed in the tutorial.
If you went through the tutorial, it explains to you what all of the actions were.
You probably wouldn't feel quite so confused about what you could do.
However, I think it's reasonable to address the lack of control that you might feel, even after knowing what you could do.
Right?
These are the things that people would describe as aesthetics.
The thing that the player feels.
The paper describes them as desirable experiences.
That's not necessarily always the case.
It's not always desirable.
I'm pretty sure that confusion was not a desired experience that the designer of this game wanted you to have.
By the way, one of the designers of that game is actually one of the writers of this paper, which is why we're playing this game in particular.
So these are aesthetics.
Let's talk about some of the mechanics of the game execution.
Hit the screen and bring it up.
So let's talk about some of the mechanics of this game.
Someone describe it
[?
Turn waits.
?]
OK, so [?
turn waits.
?]
The game waits for you until you make a decision.
It's not real time.
Is it not real time?
[INAUDIBLE]
Huh?
It waits
It waits?
Except for when you run into barbarians, right?
Don't you have 10 seconds or something to-- [INTERPOSING VOICES]
10 seconds is actually [INAUDIBLE] and collect more
They do that over the course of the game
Yeah, but like the countdown doesn't start until you send your troops
OK. All right, so the truth is until you make your final move, right?
And then the 10 seconds where it counts down and you see things continue to happen.
OK, all right
There's combat
There's combat as a mechanism.
There's some sort of AI combat.
Right?
You don't actually directly tell your troops to attack.
You just place them ahead of time, and then they just wail at each other, using some sort of algorithm, some sort of computation on how that number, of how that addition works.
What else is in the game?
Exploration
Exploration?
Go ahead
[INAUDIBLE] how you prepare for the attack
So how do you find cities?
How do you find cities?
You just have to walk on top of it
OK.
So you click on the purple squares.
And then that shows you what is underneath the purple squares.
It will defog.
It's not full of war, exactly, but they have a sort of unexplored fog, right?
Which you have to disappear.
Is that the only clue about how to find a city?
Question mark
Sometimes you get the question hint.
Right?
And there's something there
There's like lush terrain around cities usually
Farms actually.
It's hard to tell on the screen, but on the iPad.
Do the farms-- all make the cities.
So if you step on something green, you realize, wait a minute, there's a city somewhere nearby.
It's kind of Minesweeper-esque in that way.
What else?
The oasis?
What was that?
The oasis
OK.
So what about the oasis?
You're also looking for it.
As soon as you hit one of the squares of the oasis, you can more or less figure out where the other ones are
OK, because?
It seems to always be in the same 3 by 4 rectangle
So the oasis--
You see like a corner of the river, then you know it's flowing this way and then that way, right?
Yes.
Well, the oasis is always a rectangle.
So if you find a corner, yeah, you know that if you follow those things then-- [INTERPOSING VOICES]
You know it's grass on this side, water on this side.
So it's always inside
Right.
So if the grid was like that, the moment you find one corner, then you know which way to explore.
So the oasis gives you clues.
The title of your oasis alone gives you clues.
It is the same way that the farms give you clues about the cities
[INAUDIBLE] because we were trying to decide whether or not we should use technology, or to build roads, or outfit the cities with defense, or whatever
How do you all take resources with this game?
You just have it, I guess
You just have it, and what happens when you attack?
Certain actions [INAUDIBLE]
So you have a limited number of followers, and tapping assigns them to spend them, basically, to get them to follow some sort of task.
So limited followers, tapping assigns.
Assigns the followers.
OK. Other ones?
There's the scoring at the end?
Scoring at the end
Which is a summary in some sense.
It seems to be different from what actually-- like when you end the level, you get a feeling of, did you do well or did you do not well.
And then it didn't always correspond with the score
Right.
The score is unnecessary.
So the score is kind of different from what the game is immediately giving you feedback on.
The score is kind of like a completely different kind of feedback
I actually disagree that the score was like that.
So I was very confused when we won the first level, because everyone died.
But we got a score of zero, which kind of makes sense.
And when I went a bunch of people died, but I still passed the level.
And that made sense because I had two cities left.
And I still got a score of zero because I lost tons of people.
Then when we made it so we didn't lose a single city, we got a score of 1,000 or something We didn't lose a single city and we had a lot of people.
I thought the score made sense, maybe the level progression didn't
Nathan lost some cities and still had a score of about a thousand or so.
[INTERPOSING VOICES]
You lost-- you didn't get any points, and you didn't discover anything
There you go.
So the score makes sense
Also I was going to say that he said that the score was possibly accumulative over all the phases done.
And the score [INAUDIBLE]
[INAUDIBLE]
There was another mechanic.
We didn't really get to discover it.
But it was like glowing, O-A-I-S-I-
Right.
Clearly leading to something we haven't figured out yet
There was a special follower that we got that kind of like stalked us
OK.
I think they call them experts or advisors or something of that nature.
What was the advantage of doing that?
Having followers return?
[INTERPOSING VOICES]
So she started and you had more.
All right?
Here's a question.
Every level that we [INAUDIBLE] We had two of them along the side.
[INAUDIBLE]
It said that in one of the tutorials [INAUDIBLE]
OK.
So there's some sort of [INAUDIBLE] with [?
wind, ?]
you know?
[INAUDIBLE] Right?
There were also these little blue orbs that got transported into the artwork
Right
And I have no idea what that does
So where did those blue orbs appear?
Did anyone notice?
When you were uncovering the tiles I want to say
But when you searched, if you didn't get one of those technologies at the top, you usually got a blue orb that would go up.
Otherwise, I do not know what that blue orb does.
He got lot of blue orbs
When you pick up the oasis-- when you walk over the oasis, you get blue orbs.
So the oasis also gives you blue-- which makes sense.
It's like the bluest thing on the screen, and then when you click on it, blue things move.
So there's some sort of color coordination thing
The barbarians steal your blue
I know where this meme is going
Oh, I was thinking that the blue-- [INTERPOSING VOICES]
Oh, yes, yes, blue
The one time they got to the oasis, something happened where everyone-- all the blue went to the barbarians
It looks like it was a last line of defense.
[INTERPOSING VOICES]
So either the barbarians are taking away your blue, or you're using your blue to smite the barbarians.
Either way you end up with no barbarians and less blue.
Right
Do you think if that happened, we got down to zero blue, we would lose the game?
Yes
That was my guess.
There had to be some way to lose, but we couldn't figure out how.
[INTERPOSING VOICES]
You were at level four on like 15
13, I think
I can tell you one of the things that this particular game designer has told me is that he doesn't believe in game balance.
He just believes in making a game start so ridiculously easy and end at an almost impossible level.
So somewhere along the way you meet the level that you're good at, and that's a balanced game
[INAUDIBLE]
That's one way of looking at it.
So we've identified a bunch of games mechanisms.
We didn't go into things like roads and technologies and mines and stuff like that.
There's a ton of game mechanics in here.
These are the kind of things that a game designer can design, right?
These are rules.
Things as simple as the oasis is always in a rectangle.
There's no reason for that.
It could have been a river.
But you're right.
There is actually a rule in the game.
Every time you find an oasis, it's always going to be of a rectangular shape.
These are the things that a designer designs, and you get dynamics out of a bunch of these mechanics interacting with each other.
So I'm going to give one example of a dynamic.
One of the dynamics is that as you reveal information about the stuff that is underneath the fog, you start having more decisions to make.
What's your next step going to be.
Because you could click on the thing that you just discovered.
Like if you find a city, one of the things that people were telling everybody-- search it right now!
Right?
But then you can also find a mine, and you're going to find followers into it.
You could build a road, and some people I think might have noticed what road is this.
Did people find out about anything about what road is this?
They gave you more people, I think
Yes
I think one of the pop-ups said
Your cities start actually growing in number when you connect them on the roads.
And of course that helps with defense in the long run.
So one of the dynamics in this game is that the further that you play in a level, the more kinds of decisions you're going to make.
At the beginning of the game, you can make no decisions besides which one of these two squares am I going to start searching.
Up or left.
I think you always start in one of the corners, so there are only two directions that you possibly search.
What other dynamics are there?
These are things that arise out of a combination of these mechanics
Would one be huge kind of pre-perform a set of actions, like your exploration, you're searching and whatever.
And then, after that, you just kind of let it go, and the barbarians come and do their bidding?
OK.
So one of the things is you set up initial conditions for, hopefully, things to work out, for sort of the systems to emerge in the behavior, in the long-term behavior.
OK. What else?
That's actually a dynamic, in this case.
That's something you set up even before all your turns are over.
Like you said, people in a mine-- the miners just keep digging each turn.
You don't have to babysit them.
They'll keep digging.
And they'll find stuff without you telling them to do anything.
What else?
What else fits this kind of description of something that arises still kind of mechanically out of a combination of all of these dynamics?
Let's say combat, for instance.
When the red forces and your blue forces start, you know, like little ants start facing off against each other.
What happens?
Like really snowballing.
Like really snowballing
Right
It would start off kind of even or so.
And then all of a sudden one force would just take a nosedive
Uh-huh?
At one point, the two forces were even, and then one got up to like 90, and one just started dropping and dropping.
And this one just stayed at 90.
You can see that too with the really small cities that have like 20 people or something that get like squashed
Squashed, right?
So it has this interesting curve.
On like a successful defense, the barbarians are usually here, and the followers-- you usually have a slightly lower number, because a huge number of barbarians that show up all of a sudden--
But we have technology
Yeah.
But often what happens is that you've got a number like that, and it kind of goes like that.
Right?
It starts off kind of like the barbarians still maintain your numbers, then all of a sudden it just collapses.
This is what a successful defense in that game looks like
But you can also flip that curve.
Not necessarily flip it, but the followers can also take that kind of curve
Oh, yeah
Instead of just below the barbarians
Oh, yeah.
Totally.
That would be what an unsuccessful defense looks like.
That will look more like that, and then it's like bleh.
Right?
There's usually more barbarians than there are people.
So this is all arising out of the game mechanics.
Right?
This is not an aesthetic experience all by itself.
This is just how the numbers, if you plotted by a fraction of a second by a fraction of a second, you'll get a graph.
This is just the algorithm doing its thing.
All right.
What does this kind of-- I'm going to go back to a successful defense.
So this arises out of the mechanics of the game.
Everything that you do in the game to get salt, get followers into a city, get them technologies, mine stuff, search cities.
Make sure there are enough followers by connecting the cities up with roads.
The other thing that roads allow you to do is redistribute followers between cities that are reconnected.
That's the thing that you get to do in those last 10 seconds is that you don't just assign everybody that you've got.
All the people, go into this one city.
You can actually do that and connect them up with roads.
You have 10 seconds to do that.
So everything leads up to how many followers you've got in a city that the barbarians are going to attack, right?
You know there's this interaction that keeps going on until all the barbarians are destroyed.
Then you get this curve.
And what part of the aesthetic does this generate?
What does the player hopefully experience because of that?
You cheer for your forces--
Yeah
And then you feel disappointed when things start going bad, and you kind of feel for them
Yeah.
And then when they pull out of it, because they're not like, yeah!
At this point it's like we are both kind of in the same trajectory.
Sure they're dropping, but we're dropping too.
Things aren't going so well, and then all of a sudden your folks come through, right?
We were cheering.
We were shouting our heads off, as if our cheers had any influence on the outcome.
I'm not quite sure how to describe that aesthetic, but that's what the player takes away from this game, right?
There's this tiny little moment of the game.
It's actually very finely tuned to generate exactly that experience.
You're not quite sure what the outcome of this battle is going to be.
I like to play the game for hours and hours and hours, and then you can kind of predict what's going to happen the moment the battle starts, because you already know what each technology agent does.
But at this point in time when you're still learning the game, you don't know what each technology does.
You don't know how the battles are going to turn out.
It gets really, really exciting.
So maybe there's a kind of-- how Marc LeBlanc, the designer of this game, described it was dramatic tension.
I'm not quite sure if he's using that term exactly how it would be used in literary convention.
But in this view it does feel tense, because there's an uncertain outcome about who's going to come out ahead, and then when it resolves in your favor, you feel great.
That's basically one dissection using the MDA framework that we just went through.
There's a whole bunch of mechanics that create a dynamic that creates an aesthetic experience.
Which is the thing that you want your players to have.
In this particular case, this is explicitly what Marc LeBlanc wanted his players to experience.
He wanted them to cheer for the followers, even though that has no impact on the final result of the fight.
So when you look at your games, I want you to be asking your play testers what they're feeling about the game.
Usually after they're done.
Say how do you feel about that, what did you like about the game, what didn't you like about the game.
Both are aesthetic experiences that come out from the play of the game.
This is much easier to do with people who are actually not in this class, because you're all thinking about the games from the rules level.
If you go to somebody who plays games but doesn't really make games or isn't in the process of learning about how to make games, it's a lot easier for them to just think about well, I kind of felt crappy, because I didn't feel there was any way for me to come back in this game, you know.
Or, yeah, I felt like I was [?
roleplay ?]
stomping everybody.
[?
Roleplay stomp ?]
is an aesthetic.
Those are things that you can sort of take that to game designers, give you clues on whether your game is doing what you want it to do, for the mechanics that you're designing, whether the game is not doing what you want it to do, because maybe the mechanics are interacting in a way that creates a dynamic that you didn't want, or maybe the game is really not what you wanted to do and it's actually pretty cool.
There's always the accidental discovery of your mechanics doing something that you didn't quite expect.
Maybe you expected something to be kind of like cool and strategic and detached, but then your players end up feeling really tense and feeling that everything is on the line and everything could go wrong at any moment and they like that.
OK. All right.
That's something you can work with.
But you may not necessarily understand that's what your game is doing if you're not asking your players what's the sensation that they're getting out of the game purely from the rules.
When you're working with the prototypes in particular, it's very easy for you to make this connection back to your mechanics, because they can be experiencing terribly much from your artwork or from the sound effects, right?
Unless you make your sound effects like a mouse, or something, and then it probably sounds pretty goofy.
But they can only really get the aesthetic experience out of your rules.
Obviously this is not the only way that you can convey an aesthetic experience.
You can do things through art, sound, storytelling, characters, plot, and everything.
We'll go into that a little bit more before assignment two, because you're going to be exploring that with assignment two.
But there's this one particular kind of aesthetic, which I will refer to it in this class, as system aesthetics.
The aesthetics that come out from the system of all of your mechanics working together.
And, in this project, this first assignment, you don't have to worry about hitting the design aesthetic.
You don't really have to.
That's not the goal of this project.
I want you to figure out everything that your game mechanic can do.
And by the way, there was a question at the beginning of class, what if we wanted to change the mechanic because you discovered something along the way.
That's fine.
But whatever you change it to, make sure that you're going deep into that thing.
Make sure that you're not just running with what you stumbled with, and then not developing it any further.
Try to really, really explore what you eventually will declare that you're looking at.
The reason for that is I want you all to have the experience of going really deep into mechanics and figuring out all the different things that one mechanic can do.
So that later on when you all mix and match, and form them into assignment two and assignment three teams, you've got like a pool of knowledge and more importantly a pool of experience in sort of exploring what a mechanic can do.
So you can take on new mechanics, you can share information with each other and use that later to generate interesting aesthetic experiences.
Because in the end, this is what the players are here for.
They don't care about the amount of work that you put into here.
They only care about whether the game makes them feel the way that they want to feel right now while you're they're playing your game.
So any questions?
Nope.
OK. Well we'll probably go a little bit more into things about mechanics later this month, but right now the rest of the class is actually time for you to work as a team.
Some of you probably haven't seen each other since Wednesday and may not necessarily have talked to each other.
So that's fine.
Take time now.
If you've got something that you need tested, you have two options.
One, you can grab Rick and I.
Or two, you can grab other people in the classroom who may not necessarily have saved their game out.
Suggest asking people who haven't played a game for two tests.
But tomorrow is actually the formal play test.
So what you absolutely have to have Friday end of this class or what you have to have by the beginning of tomorrow towards that is something that can be played as a big class.
Today there's no required play test.
Tomorrow there is.
All right?
There's also reading for tomorrow, but it's not that bad.
I think the total number is-- let me just make sure of that.
Oh no, it's only part of it.
Yeah.
It's going to be just looking at one thing, which is the randomizing and add a functional dice in games.
So, OK. Work in your teams.
And all the materials right here we'll put it on the table.
Contents from the following sources is copyright of the respective holders.
All rights reserved.
Excluded from our Creative Commons license.
For more information, see ocw.mit.edu/fairuse.
Oasis.
PlayFirst.
2005.
Video game.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today's reading is kind of there for two reasons.
One, the David Parlett reading is to introduce to you this book which is called The History Of Board Games.
How many of you read the version from Stellar OK.
Anyone get the book from the library?
Because I think it's on reserve if you really want to read a hardback, paper book, you could just go to a library and you should be able to check it out, probably from the humanities library.
So I'm just letting you know that that option is out there.
So one of the reasons is; one, just to introduce to you a bit of what historical research looks like in game studies.
There are people who have careers, like David Parlett, who basically go deep into the history of how some kind of game evolved.
One of our old alums, Jason Begy?
He looks at railroad games.
I'm pretty sure he hasn't changed his thesis topic yet?
Again, he's been looking at that for like three years, four ?
Yeah.
Three years now
And so they'll go to museums.
They'll talk to archaeologists.
In the case of railroad games, obviously, there are no archaeologists involved.
But you talk to people who worked on classic game designs.
You talk to retailers.
You talk to people who actually owned or played some of these games.
And I'm wondering if a lot of the games that are written in today's reading were kind of ancient-- really, really ancient-- games.
But how many of you feel like you've actually played a game that sounded like one of those that were written up?
Well especially they had dreidel in there
Yeah, they had the dreidel in there.
And that's Purim and Hanukkah?
Those are the two--
Just Hanukkah
That was pretty much just Hanukkah.
OK.
It's funny because when I was reading about the dreidel I was actually; like the initials-- a great miracle happened there-- I didn't realize that there was a different version if you played dreidel in Jerusalem, where a great thing happened there.
Which makes sense for Hanukkah but I don't think-- that doesn't quite make sense for Purim because--
No, it's only Hanukkah
That's just to do with Jerusalem.
All right what else?
The other games, the State Board games or-- a lot of these games border on someone setting up shop on the street and then if the cops come they run away, one of those kinds of games, right?
Let's see, what were some of the other things?
They had a lot of lots.
And one of the traditional Roman implements was the knuckle bone.
I'm not quite sure if they mention what type of animal it was?
I think was--
Sheep
Sheep or goat, yeah.
But that's where the idea comes from.
And the idea comes up quite a bit in game studies, at least since Holsinger's time, since I think the 60s, where it's introduced as a generic term to describe games of chance.
I think the word earlier might also have been used like in traditional Latin scholarship to describe games of chance.
But that's where you get phrases like alea iacta est.
Does anyone know what that means?
The die is cast.
It is what Julius Caesar-- Julias Caesar says when he was about to cross the Rubicon, famously.
And originally there was some confusion about whether that meant that the die, like dice, has thrown?
Or whether that means that the die; as in the mold for something that you put liquid iron in, and then you cast it.
But that still has the same sort of sense of, all right, It's out of our hands, right?
It's already been done.
So I get the same point across.
But actually look at Latin, specifically talking about these knuckle bones.
These knuckle bones that you are throwing which are called alea.
So now it's up to date.
But Parlett makes an interesting point that we are trying to simulate something that seems realistic.
That even with your best efforts or with your worst efforts, sometimes certain things are out of your control.
And simply just describing them as games of chance might be doing them a disservice.
Because you see the same sort of randomization that comes up in computer games and things like combat.
Especially strategy games, where there's a chance that something may fail, a chance that something may succeed.
And when you think about games like war games simulations, they are obviously not trying to create this fantastic sort of fantasy element, unless you are talking about things like RPG fire games.
But we're talking about World War II games and things like that, that are trying very hard to simulate something that's real.
And one of the things that they're trying to simulate is sometimes you just don't have control over everything.
And that's where randomisation elements-- this particular chapter is about dice and other physical things that you use.
We included spinners.
We included things that were basically binary.
And that's actually a cute idea, if anybody wants to think about trying to introduce the Bell Curve to any of your results without having to be having people roll a whole bunch of dice.
Or if rolling two dice gives you numbers 2 to 12, but you wanted a different range, you could just have them roll a whole bunch of flat sticks and then it lands one, zero, one zero, and you still get a nice bell curve.
So that's an idea you can use in your game as well or you can use things like spinners, which pretty much just give you the equal chance no matter which side you land.
And you can make a spinner of arbitrary complexity.
You could make a 20-sided spinner.
It will look basically like a circle but you could.
So one thing that Parlett introduces is this idea that games of chance are also some of games of imperfect information.
Games of perfect and imperfect information, what's the difference between the two of them?
It's the first time we're encountering these terms in this class.
But I wouldn't want to hazard a guess.
What's an example of a game of perfect information?
Chess
Chess.
That Is a game of perfect information because?
Because all of the information you are going to be using in that game is right there in front of you
Right.
Players see all the games state, although Parlett says, but they don't know and one of them got out on the wrong side of bed that morning.
They don't necessarily know how experienced each other is, and that's kind of weird is that part of the game was the outside of the game, right?
And other examples of games of perfect or imperfect information?
Go is imperfect information
Imperfect, which would--
Poker would be imperfect
Poker would be imperfect because you--
You don't know the deck, you don't know what's in your opponent's hand, stuff like that
Does poker always have to be played with a single deck?
All 52 cards?
You could even not know how many cards, how many decks there are
OK, you may not know.
So that's a question too.
When typically we play it at home, it's just one deck of cards of 52.
You know that you're holding the King of spades.
There's no other King of spades out there.
But you play in some casinos, and they've got a five deck mix, there could be two kings in spades.
Yeah?
Poker is all about trying to figure out what other people have, and what's the chances.
Actually, you already know what other people have, but it's all trying to read what's the chances that they've got something, plus what's the chances of your hand beating that?
Because it's all the variance in poker, like Texas hold'em and everything, which have other kinds of cards that nobody gets to see, that everyone gets to see.
So perfect and imperfect information is kind of like a sledge hammer to deal with this, because you're not quite sure.
Like random numbers or random cards that you're drawing from the five card stack, or random numbers that you're rolling out of a die is a very, very different quality to that piece of information than a card that somebody is holding and not revealing to everybody else, because it that's a fixed quantity.
It's something that was dealt in the past, and Parlett describes it as sort of past imperfect information.
And then there's future imperfect information, you're going to roll it and then and only then is that information generated even though no one knew it.
No one knows that number.
So there's a scholar called Celia Pearce over in Georgia Tech, and she came up with a taxonomy just to help her discuss this with her design teams, and I thought it was really, really useful.
So instead of saying imperfect and perfect information, we start off with what the obvious one, which is public information.
Games of perfect information and games of-- all information is public, everything is public information.
The state of a board in chess or in checkers, or probably fun computer games where this is the case as well, although [INAUDIBLE] computer games-- but nothing comes to mind-- a of computer games take advantage of the fact that everyone has their own screen to do imperfect information.
So when you start jumping into the realm of imperfect information that's private, which is the stuff that one person, at least one person knows for sure, possibly, if you're playing a team game, there's more than one person who knows this information for sure.
If you're playing a game like bridge, where you're sort of communicating, but in codes-- if you've been playing with someone for a long time, you can get to the situation where even though I can't see your cards, I pretty much know what cards you have because you're my teammate and you're trying to communicate with me in a way that I understand.
There's random information which is stuff that you get from, say, drawing from the top of five stack deck of poker cards, or something that you get from rolling a die.
Stuff that you can't really intuit, what Parlett calls future imperfect.
Things that will be thought of, for all intents and purposes, that information introduced the moment you trigger that rendering-- that randomizing method.
The moment you throw the dreidel, spin the dreidel, roll the dreidel?
It spins, OK.
The moment you throw the knucklebones, whatever it is, that's the moment when the information gets generated.
There may be some probability in how you expect this to work out.
Like if you're rolling two dice, you expect number 7 to be the most likely outcome, and number 2 and number 12 could be the least likely outcomes.
But that information doesn't really exist until the moment it's generated.
So what's missing?
There's information that's computable out there, so, for instance, when you're playing cards or something like that.
I know technically that the rules are you can't see the round that's in play, but there's information which, if you were at a computer you would remember this information, but oftentimes it's a fact that you can't actually remember it
The stuff that's hard to remember, the stuff that like-- let me see if I'm hearing you right.
So for instance, the game off blackjack to a card counter is a very different game of blackjack for someone who sits down and plays for one round.
Right?
Because the game of blackjack for a card counter is aware that this five stack deck of cards has had a certain number of cards played out of it.
And knowing, being able to remember how much has he played out of it has changed the actual percentage of what could come out.
So what's been played before is technically public information.
So previous rounds that were played out in the open, if you were really, really paying attention you could have seen that information.
It's hard to remember, but you could have if you were a computer, if you're a video tape recorder, you want to be able to see that information.
But then there's all the other cards left in this stack that's already been shuffled, and it's kind of in a fixed order.
It was randomized before they shuffled it, so then not it's been shuffled and it's just sitting on a stack.
That's what [INAUDIBLE].
It doesn't even have to be like a five stack deck, it could be just the three cards that were dealt face down at the beginning-- is it three cards for Texas hold'em?
Two?
OK, the two cards that are dealt face down in the center, nobody gets to see that until right at the end.
But they're dealt at the beginning of the game, like in the second hand or--
The public?
The public cards?
Oh, that's three
Three, right.
And they're down, right?
After the first round, they're face up
After the first round they-- Yeah, that's right.
It's a long time since I played hold 'em.
So three cards, they're face down, but they've been dealt.
So the randomizing, they have been triggered, the information is there, but no one knows it yet
In other words, not really different from random information.
Like the difference between like, I haven't actually dealt the card yet, and I've dealt the card and no one's sees it.
It seems like hidden information that no one knows is equivalent to random, like it's hidden information that's equivalent to random information
Not quite.
There is a different quality with it, because that variable can no longer change in the rest of the game.
Maybe the Texas Hold em one starts to get a little blurry, and it's--
But random information, like-- it's like players can treat-- like you can't treat it any different.
Just because like if for some reason we've decided that we were, let's say I had a die-rolling-- let's say I wrote a die-rolling program and the random feed, and you are right-- no one knows what the next random feed was until what the next die-roll showed them.
And you could say this is hidden information and not random information because the random feed i-- it already knows the--
Yeah, if you knew the feed and you knew the randomisation algorithm, then I would argue that information is no longer random.
That's why they call it pseudo-random.
It's not actually random anymore
But it's random in the sense that no-one-- it's random with regards to players so that players can treat it as random information, because they don't know how--
You can treat it as random information
So if you deal two cards out, face down in the deck, then at the time when you're dealing the first one, it's random?
Yeah
But then the card that's the first one is a card that's already in play, and so when you take the second one, that card has a smaller phase of possibility.
There is a specific card that that second one can't beat, the first one
But we don't have to see it that way, we can just--
What I'm saying is that if you want to-- if you deal one card and you want to reproduce the exact same situation elsewhere and then continue dealing from the deck, then maybe you're going to shuffle the deck before you deal a second card, and then that new card is random-- the second card-- except it's random out of a smaller possibility.
And it's not just random because there is hidden information that the--
That's one other thing that you're not taking into consideration, and that's the fact that before-- after these three cards in Texas hold'em are dealt out face down, there was no a chance to look at it, but everyone's gotten a chance to see some of the cards that already have been dealt.
And that has already changed the likelihood of what that card could be down, because--
Just because something isn't perfectly uniformly random, chosen from the distribution of everything, doesn't mean it's not still random.
So what I'm saying is just because like-- so if I deal a face down cards, and then I throw a face up card that's the five of diamonds, even though I know it's a face down card, it's not the five of diamond, it's still random because it's now drawn from an equivalent of random information, yeah
I mean, this is perfect Schrodinger's cat, isn't it?
No, it's not, because what happens is that you keep getting new cards throughout the entire game, right?
And these cards slowly get revealed
I see that there is a distinction made between random and hidden for the purposes of the classification of games.
There's probably a useful reason for it.
I mean, sure, you can think of it as like now there's pendence between different cards and things like that, was probably just there for a useful way of thinking about games
What I'm saying is I don't think it's useful, and I think it's a kind of predicament.
I think that there's different relationships
I'm not talking about statistical difference.
I'm talking about what the players think about these three cards.
They know the three cards are not being re-dealt every single hand.
They know that no one can say, for instance, like someone can take the deck and shuffle it in the middle of a poker game.
No one complains about that.
You're allowed to do that.
You can't re-deal another three cards.
You're not allowed to
It doesn't change the probability distribution
It doesn't the probability distribution but you're not allowed to.
The rules do not allow you to re-deal those three cards.
There is a different weight that the rules have placed on those three cards that have been dealt saying those things are now fixed, whereas everything in that deck can be re-ordered.
Your hand and your hand
For a different example, is the game Clue, where you have like this one item, the one person and the one room, and that is the way to getting it.
In some senses it's random, but it determines who wins the game, and so in that sense it's like you wouldn't think if it as random necessarily, and while your rolling to move around and discover these things would be random
And that is actually also statistically different example, but a much clearer example of hidden information.
Because that one, as you play the game, you are getting more and more and more information about what those cards could be.
In fact, that's the whole point of the game, right, is trying to figure out what's in there.
So in that particular case, that's hidden information that's random at the point when it was generated, but after that it is-- but as soon as people start playing, start making accusations, it's no longer random, it's just hidden information.
So yeah, Clue is a much better example of that.
Maybe the three cards and the Texas hold 'em are different.
I still maintain that the quality of these-- they may not be statistically different but they are qualitatively different, because you can never re-randomize those cards according to the rules
It's something players think about, not because they're-- I think the reason different players think about them differently is that--
Well, that's--
That's not true.
That's just not true.
The cards are down and they're down.
You cannot un-put them down.
So qualitatively, they are completely different
When they're exactly the same, did you re-deal-- [INTERPOSING VOICES]
OK, he had his hand up first, so let me get with him first and then we'll get back to you
It's actually also statistically different, because let me just simplify the poker example a lot.
Let's say that we take one card from the deck and put it face-down, and then we flip a second card so that everyone sees it.
And then, let's say someone gets it into their head to take that one card that was hidden, and shuffle it back in and deal a new one, that actually does-- The two situations where you do that and where you don't are different, because in the first situation, the card that was hidden had a equal probability over all 52 cards and followed-- and when you get one piece, the extra piece of information, after that your adjusted probability distribution is equal over the 51 cards.
However, if you reveal that one card and then take it back, then if someone-- you don't necessarily have that card visible to everyone, for example--
But it's true that the example--
Two or three cards, it would be the same
Yeah, you are describing a situation where someone's actually had a chance to actually look at that card, face-down, yeah.
And that does change things because that becomes private information to someone else.
So that takes it kind of outside of the realm of example, but yeah, I do agree that statistically there's not actually a huge amount of difference between the cards that are face-down and the cards that you're drawing off the top of the deck.
And then get your hand and then your hand
So the problem I have is first of all, statistically, there's no difference.
Not a little difference, there's no difference.
And so as a player, you sort of think about cards that are faced a little bit differently than cards that are in the deck.
But if I'm playing to win, ultimately I have to think of them as mostly random
Sure
Like I have to think of them as what would probably be distributions of the cards
Strategically, you could, it's probably advantageous for you to think that they're the same statistical, difference because they are.
And I'm not disagreeing that statistically they have the same likelihood there.
I'm saying that in terms of how the rules are worded, they are very different
OK, so does the use of this definition come in when you're like just writing the rules?
When you're designing a game.
Not necessarily when you're playing the game, but when you're designing the game
One thing to think about is is when we were talking about that clear example, when you have something that's hidden, you can possibly gain information about it that will be useful, right?
You gain information about what that Clue card is.
There's no way you're ever going to gain information about what the next roll of the die is going to be, unless you're doing some kind of major statistical analysis
But that's possible
For roll of a die, yeah, I mean--
But it's still random, there's still a probability distribution over what it is?
Not really
No, if you're rolling the die--
I'm not going through that.
It's just theoretically possible that you could compute what the die roll is going to be before I roll it
There's one very clear example when you can compute it, right?
Because you know those dice are loaded.
Yeah, you know those dice are loaded, and in that particular case you've got private information just in the same way that if you're using, say a pseudo random number generator
But if you flip a Clue card more than once, it doesn't change.
And if you get information about what that is, it stays the same, right?
Do you think that the Clue example is an example of private information?
No
Is it an example of private and random information, it's not really like--
No, it's important to remember, to know that isn't information because no player actually gets to see what that card is when it is inserted into the envelope.
For those people who haven't played Clue, what happens is that you've got a whole stack, you have three different stacks of people, places, and things that you can kill people with from the 1950s or something.
And then you get one-- three-deck card shuffle, you pick up one card for each, and those three cards go into an envelope that nobody gets to see until pretty late in the game.
And that those cards will never change.
That's a very clear example of hidden information because at the point when they were generated, you're right, that was random.
But then it gets turned to this piece of hidden information that everyone is trying to find out during the game.
The whole point of Clue is that you're trying to figure this out.
And there are rules governing who gets to see that hidden information, when, there's rules governing how you generate that information, and there are mechanics for you to be able to figure out what's inside there.
Let me see.
There was another point that I was going to make about that.
And it's true that if you, say we implemented that on a computer or any game that has some sort of randomizing thing in a computer, and you could figure out how the randomizing was being done because very few things in a computer are truly random.
Then you will be able to, say figure out OK, the next die roll should be a certain number.
The other day I talked about how when you play certain strategy games, and you had like a 50% chance of succeeding at something, then technically if you save the game right behind-- right before you did it, and you kept doing it, then you will eventually have 100% chance of succeeding, because you just keep playing it until you win.
Unless, of course, it's a pseudo random number generator that is saving the feed, and so the outcome always turns out to be the same each time.
And if you have that information, that's actually private information.
The question is, is that actually going to be something that you're designing into your game, or something that you just choose to live with?
And as a designer, you get to make that decision.
You get to decide things like, I'm going to save the random number feed when I save the same game.
You can decide to reveal what you're randomizing scheme is to the rest of the world, and to a hardcore computer hacker, it wouldn't it be that hard to figure out what kind of method they're using.
You can choose to say no, this is a very closely guarded secret.
I'm actually generating it by analyzing this Geiger counter, and seeing when certain ticks come, I mean that's as close to random as we can get.
You can choose to do all of that as a designer, and the question is where do you want to set those limits for your own game?
Things like, for instance, yeah theoretically, a pair of dice can generate any number between two and 12.
But anyone who has played games for a certain amount of time realizes that 2 and 12 are really rare compared to 7.
And there are games that take advantage of that, and there are games who don't take advantage of that.
There are some games that basically treat a roll of 12 and a roll of two exactly the same as a roll of 7, especially like the race games.
You move that number of steps but they're not like balanced so that 2 and 12 are expected to come up less often or more often.
That's why you-- I've seen versions of things like Candy Land, where they replace the spinner with dice.
And it's like, wait a minute, that completely throws off the likelihood that-- the way how the pace of the game works.
But for Candy Land it doesn't matter.
So you're kind of equally likely to win.
So for that game, that's something where you can do those types of substitutions.
I think it is actually very, very useful to be able to talk about this when it comes to the design of your own rules.
It's useful to be able to talk about it with other designers.
Say he has this piece of information that we're storing early on, maybe because we got cards that are flipped down, maybe you got cards we're putting it an envelope.
Maybe just a quantity of things in a bag or something, and then you have to reach into it and pull things out, like the bag in Scrabble.
Things that sort of randomize in a bag, the actual which one you're going to draw next from it is somewhat random, based on some sort of probability.
But the possibility of that could be public information.
It could be hidden information.
You might not know what the distribution of stuff inside that bag to begin with.
And that's why I want you to be able to use these words in describing how things are working in your own game, to your own teammates, and especially to us.
Because we want to understand what it is that you're trying to achieve with this bit of information.
Hidden information typically is something that you're going to try to allow people to figure out over time.
And that's a lot of that's a lot of stake over what that hidden piece of information is.
Whereas that's not a whole lot of-- and it's kind of like shared among everybody who's looking at a piece of information.
Whereas something that's random is usually, kind of at that moment it's really important, and after that it's not so important anymore.
It has some sort of consequence that might be recorded in the score, but it's not like from turn to turn that particular role or that particular spin was all that important.
Private information and public information tend to be pretty easy to talk about, so they're just useful terms to get people to discuss with each other.
And so I just want to introduce these terms, this is the taxonomy that was provided by Celia Pearce.
It sort of expands on the perfect and the imperfect information because imperfect information gets kind of fuzzy because it has all these different versions.
And now that I everybody who's planning on showing up today has shown up, we should probably start setting up for play time.
So is everyone sitting in teams?
I think everyone's more or less sitting where you were, right ?
Any questions so far on other questions based on what are we talking about?
Like to clarify why I think these should be used differently.
How many of you have changed your game mechanic by the way?
What would you describe it as now?
Yeah, maybe deception
Deception
It's not built
OK, something like laughing, something like faking out your opponent.
Everyone else still pretty much sticking with what you've got?
Is that still building?
You're building like a board
You're building a board.
I haven't see that part of the game yet
Well, not building-- you're building on the board
I think I need to just play our game to get it.
Because it looks like [INAUDIBLE].
All right, so again, can I see which group has two-player games?
1, 2, 3, OK. And the rest?
Who has four-player games?
And you all have the be three-player games?
OK, all right.
So, wait, hold on, both of you are two-player games and then you played each other's games last time?
Oh, you didn't?
OK, all right then.
Then that's easy, the two of you should be playing each other's games.
Again, there should be observers who have recording material to be able to--
Is there a team of two playing the game, though?
No, don't play simultaneously
OK, got it
So that team comes over here, plays this game, then everybody here goes over there, plays that game.
There's probably enough time for that
Would it make sense if it all had two-player games on one set of tables?
I think it would help if just the two of you just came up here and brought your game with you.
And then you can just like hang out here because-- we're only going to end up playing one of, either you're team's game or their team at one time.
They're not going to play more games at one table.
The rest of you, let's see.
How many of you have your game ready to go?
What was your question?
How many of you have had a game ready to go?
All right.
I've played your game recently, you haven't
I've not played this one.
I've played an old version of that one, and I've played the one in the corner yesterday.
I haven't played those two
OK. Have these two groups played each other's games yet?
Probably
You have?
Then what I'm going to suggest is that three people from this group come over here and play this game first, all right?
And then Rick and I will join these two to play the middle game, and then the four of you will play that game
I've played that game already, though
Oh, you have?
Yeah
I think we can-- all four of you have played the same game?
Yeah
Has it changed dramatically since the last time you played the game?
Not really
How many times of each game are we trying to get to?
Do we play three for each game?
Just one, actually.
I'm pretty sure that after one they'll be things to change.
And if we have time, maybe we can get one more
Let me do a quick hand to see what other staff are available, and we can get--
Let's see if we can get a few more people to play that game over there.
All right, so let me just change this up then.
Three of you come over here and bring this game right now.
OK?
For the presentation at the end of assignment one, I do want you to talk a little bit about the game but don't explain the rules of your game before doing your presentation.
That will use up all the time that you've got.
Talk about a process, talk about what you started with, what you would try, what didn't work and what kind of changes that make you make.
You know, what work and how you roll with it.
I want to hear more about the story of how you got to where you are, because right after you do the presentation, we'll actually sit down and play each other's games just like today, only [INAUDIBLE] so you don't have to do like recording or anything, everyone just plays games
How is the presentation?
I think only five minutes.
It should be in the syllabus.
I thought I wrote five minutes.
It's pretty fast.
It doesn't have to be one of those like every single person on the team has to say something kind of thing, but if you want to organize it that way you totally can.
I just want to hear about what is the story of the game.
The other thing is that I actually didn't look up the situation with Texas hold 'em, but I am going to reverse my position.
That is, I think, not hidden information.
And two things swayed me.
One is I started discussing it with a bunch of other game designers online, and I discovered that there are a couple of variants of Texas hold 'em, where you never deal what they call the flop face down.
You just play it face up one at a time.
At this point, it's just a random view from the top of the deck and that makes it-- it started to be the same either way, only just because they're listed, right, that it's basically a random deal.
So yes, that is in fact random.
That's a very bad example to use for hidden information, whereas the Clue thing is because the whole point of that game is that it gives you all these tools to be able to reveal this information.
At that point in time where you put your cards inside it is made up by then once it's inside that person starts there's no hidden information.
Thank you very much.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Are you really wanting an excel style spreadsheet?
A word table is fine.
I would like to see a table
It looks more like a journal, I'd say, than a table
Yeah, that's OK.
So it looks as if you converted your table, and then you converted the table to text, or something like that.
And it looks like there's this paragraph on that
Yes.
It's like the journal of our game
That's fine.
As long I get a shape of the history of the game, and the reasons why you made changes.
That's the thing I'm really looking for.
The table format does make it easier for me to read.
In the future, I would like [INAUDIBLE] things.
But because I didn't make that clear in this assignment at all, as long as I can trace the history of the game, I am good.
Other questions?
If anyone has a copy of the syllabus, do check it out.
I do think the one-page write up is due the following week, on Monday.
This coming Monday.
I'm actually not going to be here on Monday.
I'll be traveling to give a talk, so Rick will be heading next week's classes.
Basically Monday and Wednesday.
So, I'll catch you-- grading might be delayed by a little bit
[INAUDIBLE]
A little [INAUDIBLE] Yeah.
That one in particular, [INAUDIBLE].
For the assignment itself, for the group submissions, I would actually like to see hard copies, so I can keep it all together.
The game itself obviously needs to be a hard copy.
But then if I can get a print out of your change log and everything is all in one place.
Oh, by the way, on Wednesday, I will come in with a whole bunch of Amazon boxes.
Old Amazon empty boxes, so that you can just put your games into an appropriate size box.
If you--
[INAUDIBLE]
Yes, after this talk I will go in and grab the prototyping cases.
I'll bring them down here, and you will probably have at least an hour and a 1/2, if not two hours to work on them.
So, today's reading was from The Design of Everyday Things, also known as the psychology of everyday things.
You probably noticed we skipped from chapter 1 all the way to chapter 4.
Part of the reason for that is chapter 4 is kind of a summary of chapters 2 and 3.
Chapters 2 and 3 have tons of nice case examples.
It walks you through the reasoning, explaining things in detail.
Again, I will recommend this book to anybody who is doing any kind of design where he would be expected to touch your design.
And it's really, really easy to read.
Hopefully you enjoyed today's reading.
So what I'm going to go into today is actually a little bit more detail about the stuff that wasn't covered in chapter 4.
In chapter 3 in this book, Donald Norman actually lays out this-- his mental model of how people make decisions when they are confronted with a piece of technology or a designed object that they don't quite understand.
We first saw this briefly in chapter 4, but I thought I'd like to go into a little bit more detail about this before going to the stuff that's actually in chapter 4.
So he's got this basic idea that every time you interact with any kind of technology, whether that's a chair you've never seen before, or a car, or a computer program, you go through this loop.
You have got some goals as an individual human being.
Get a new microwave oven, I would like to eat something.
And then you go through this loop of trying to figure out how this new microwave oven works.
You have some sort of intention.
The goal might be, OK I want to eat something.
So my intention is to warm up this TV dinner.
I plan.
OK, the plan is that I've got to prep this TV dinner somehow.
So I'm going to go through a series of steps.
I guess I need to take the wrapping off.
Maybe I peeled back the plastic a little bit.
Sometimes I get confused, too.
And then you actually have to peel it.
And then, instead of peeling up nicely, it peels up into little strips.
Has this is ever happened to you?
And it comes up in your fingers.
That's the execution part, right?
So I know what I want it to do, but I actually have to go and do it.
And then I get feedback.
I'm not even talking about the microwave oven, I'm talking about just the TV dinner right now.
I see the feedback.
The thing peeled back, but it didn't peel back the way that I want it to.
I see it.
I have to interpret whether that's getting me closer to my goal.
Well, all I really want is a couple of holes for the steam to come out.
So OK I guess that's good enough.
And I evaluate that as being good enough.
So I can proceed to the next step on my plan of getting dinner.
So my intention now is to actually put this thing into the microwave oven as I've planned.
To heat it up for 30 seconds.
Which means a series of steps to actually push buttons to heat it up for 30 seconds.
I have to make sure I don't set it to 30 minutes, so I punched a button to make sure I don't hit too many zeroes.
I see the feedback on the screen.
I perceive whether that's what I want or not.
Sometimes it's not what I want.
I punched in 30 minutes by mistake.
I interpret that 30 zero zero as 30 minutes, I evaluate that's not what I want it to do.
And then I make another set of intentions to solve that problem.
So this is a model.
And this is actually seven steps, even though I've got eight boxes.
One, two, three, four, five, six, seven.
This thing here happens in the system.
It happens with the thing that you are trying to interact with.
You can expand it into what you think of as a computer program.
This will be expanded into a huge algorithm, or the thing that the computer has to solve.
And figure out what you were trying to do, and tried to do.
And try to communicate back to you whether that was successful or not.
That's not really what I want you to worry about right now.
But these seven steps are all happening in the human.
And all these steps are an opportunity for something to go wrong.
Let me give you an example, from games in particular.
These are some of the considerations that you might have in a game.
There are-- say it's a competitive game.
You.
want to win, you don't want to lose.
So how do I win?
How can I win, gathering a lot of money or resources or getting more points, or knocking out my opponent.
What are my options right now?
Maybe it's a card based game, and the options all laid out in the cards that you're holding.
Maybe you have the same three decisions every turn.
So, given all those options, what do I want to do?
And then you actually have to execute that.
If it's some sort of dexterity game, where you have to flick tokens on a board, that's kind of tough.
Maybe you just place the card face down.
You need to make sure you pick the right card before placing face down and not placing it face up by accident.
The game reacts.
That could involve other people.
That could involve a computer.
That could just involve how the board looks.
And then you as a human being now see the perceived state of the game, and figure out whether that got you any closer to where you wanted to be.
You say, oh, I made that move.
And man, that was a terrible move.
And I know that because that took points away from me.
So now I need to figure out what's my next step.
I make a plan and go on, and on, and so forth.
So, the idea that Donald Norman talks about, mental models, is basically this idea that the human beings who are playing your game have an idea of how your game works.
And they might be right.
They might be wrong.
Or maybe somewhat vaguely right, but not quite there.
Now as a designer of a game, then you are creating the system that players are interacting with.
Everything that I talked about earlier was happening between the player system.
But you're the designer who is trying to machine the system into something that a player can understand.
That should actually sound familiar because we talked a little bit about this.
Mechanic-dynamic aesthetic.
If you as a designer, are designing a system, are creating the mechanics of the game.
Players experiencing the aesthetics of a game have all kinds of resultant dynamics that are coming from both how your rules define what the player is going to do as well as how the player decides they're going to execute your rules.
So it's this weird, horrible, second-order problem, which results in you trying to communicate to the player, this is what's happening underneath the hood and you really need to understand this to be able to make good decisions.
So again, the player develops a mental model of how the system works by interacting with the system.
The player tries to play a game by poking at it.
And the user then uses that model, that happens to exist in the player's head, to anticipate what that system is going to do in the future if he did something differently.
And that mental model gives the player an explanation for why the system is behaving in a way that it does.
The designer also has a mental model.
You all have an idea of how your teams currently work.
Whether or not your games actually work that way, because if I read your rules I may interpret it differently.
But right now you have an idea of how your game works because you've been working with these things for two weeks.
And you are materializing that into a set of game mechanics, a bunch of rules.
The player has to interpret, and then create their own mental model out of that.
So you can imagine so many different possibilities of things just going wrong.
I'm going to talk a little bit about that.
So one of them are mistakes that a player makes.
There are two different kinds of mistakes that I want to talk about and try to introduce this vocabulary to you so that when you're talking with your teams you can use clear language.
If you have gone through CMS 611 before, you've seen a version of this presentation.
I have had to change up the examples, so that these are board game examples.
Flips are when the player knows what's supposed to happen and what they're supposed to do.
The player just happens to accidentally do the wrong thing.
Real world slips including-- has anybody tried to call somebody on the phone, dialed in the number, and accidentally dialed somebody else that really well?
It's not like you didn't know each person's phone numbers.
You just automatically started dialing the wrong number, which you knew you also knew very well.
Driving, if you drive, you and you want to drive your friend's house you accidentally end at work because you just got into a routine.
Halfway through a task you forget what you're doing.
It's not like you don't actually know what your task is, you just happened to forget it in the middle of doing it.
Pushing a similar button to the one that you actually wanted, that sort of thing.
Dealing cards.
Everybody gets five cards.
How many have I got?
That person has five cards, six cards, five cards, five cards, or something like that.
People come up with systems like this to be able to help them keep track of how many cards they gave out.
But it's not like you didn't know that you were supposed to do five cards, you just lost track.
So that's a slip.
That's not a problem of a player's mental model.
The player, he understands how his game was supposed to work.
Well, there are also mistakes, which are errors in the player's mental model.
The player thinks the game works in a certain way and a game doesn't actually-- isn't actually supposed to work in that way.
So this is a product placement shot of-- a beauty shot, of a beautifully hand-crafted, machined [INAUDIBLE] What's wrong with that?
[INAUDIBLE]
Yes you can't put cities on adjacent spots.
I think you can't even put it adjacent your opponents
That's right.
Two apart
Two cities cannot be right next to each other.
It makes for a nice photo, but the person who set this up clearly never actually played the game.
It's still beautiful work.
That player has-- if the person was actually playing the game.
This person have the wrong idea about how this game is played?
That's actually sometimes perfectly fine.
That's part of learning.
You make a mistake.
If the game corrects you, and then you learn how to compensate for that, or play differently, that's just the process of learning the game.
People make mistakes while they're learning the game all the time.
If you're playing a computer game, sometimes that's all the fun of playing the game.
Trying to figure out how this crazy computer game works.
Probing it, and then the game punishes you because that was the wrong decision.
And then you learn not to do that and then you try a different strategy.
In fact, the game where you never fail-- and it's actually fairly easy to make a computer game where you never fail-- isn't that interesting for a player because that means you can just use whatever strategy you went in with and it will always work.
There's no reason for you to think that this game needs smart strategy or has any depth to it.
Whereas, a game where you try out the strategy you came in with and maybe works a little bit and, then it stops working.
And then getting feedback on the why it might not be working, well that tells you, hey he doesn't know there's something underneath the hood here.
Maybe I can explore this game a little bit further.
That was an interesting study from Jester Juul, who is a game scholar, who wrote the book The Art of Failure.
One of the tricks that-- he made this really simple kind of snake type game where you pick things up and it just gets longer.
And he ask people to just rate the game at the end of the game.
The game was the same, no matter who played it.
But some people played it better, some people play it-- played it worse.
The people who never failed once in beating, I think, five levels of the game, rated that game as being lower than someone who failed once or twice.
And somebody who played it once or twice rated it a seven or eight.
Some people who never failed, that rated it a five.
People who could never get past the first level will also rate it a five or four.
So people who get frustrated if they can't find the right mental model to play this game.
It's like, none of my strategies work.
I don't understand this game.
I don't get this game.
This is frustrating.
I hate this game.
And the alternative is, I completely get this game.
I saw right through it the moment I-- you put this in front of me.
I beat it in less than five minutes.
My strategy just worked.
Eh, not much there.
And the people who feel a little bit halfway but then managed to complete the rest of the game because they changed the strategy.
Woohoo, that's a pretty good game.
They gave me a challenge and I overcame it.
So let's talk a little about feedback.
How do we give users this type of really, really valuable information to correct those mental models?
I guess I'm skipping to the right side of the slide here.
So we need to tell the player that an error has happened, and we need to give them tools to recover.
Often, in a board game or in a card game, that kind of telling the player an error has happened needs to be enforced by other people.
So you need to give the other players tools to detect when the game is on the verge of crashing, when someone has executed an illegal move, and then give them some sort of structure to be able to correct that.
I'll give you a bit more example about that in a second.
The alternative of the process, you just let them make the mistake.
Their location-- or, on occasion they are making a move that is an error, that is a mistake, because they don't understand how this game is supposed to work, but its a legal move.
And then, as a designer, so you have the option of giving them tools to fix that problem.
It's like, all right you made a suboptimal move.
It isn't going to get you any closer to your winning goal, but I'm going to give you kinds of rules to be able to then recover from that mistake.
Of course you could also just prevent some of those mistakes from happening, right?
In the Settlers of Cataan board, they actually had slots of the right size to be able to put the pieces.
You know that's a constraint, that's a typical constraint that prevents you from putting the wrong piece in the wrong place.
Ways to be checked and confirmation-- again, usually tend to happen with other players.
Or maybe a referee.
Especially in sports, you want to think about the role of the referee and what the referee should be checking for.
But if you're making a computer game then the computer is doing that for you.
Let's talk about the kinds of error recovery.
That's backward error recovery, which is undo, basically.
I'm taking back that move.
In chess, the rule is if you've got your hand still on the piece, you can still take it back
Not in tournament play
Not in tournament play?
In tournament play, it's touch move
It's touch move?
As long as your hand is on it If you touch an opponent's piece, you have to capture that piece
Oh, yes
If you touch your piece, then you have to make a legal move with that
Yes, you have to make a legal move with that.
But you could-- if it was a knight, I move this way, or it could move that way.
If I moved it that way I can still move it over there if I haven't let go
[INAUDIBLE]
Yes I do hit a clock.
Because in tournament play, especially when clocked, in timed tournament play, which is how it is usually done, there's still a penalty for that.
You're still losing time, even if you're correct.
But at least there's a very clear stated set in the rules of this is how you take that move back.
And this is where you're not allowed to take that move back, which is basically when you let go of the piece.
Forward error recovery is more about, again, giving people tools be able to compensate.
Mario-- this is not a board game example, obviously.
You jump, and you can change your direction in mid-air.
So you can do a jump that you are not going to complete and you will fall to your death.
But you can go backwards while you're flying in mid-air and then land back where you started.
Again you lose time, but otherwise you haven't lost the game.
So that's forward every recovery.
You've made the decision, and that's a decision I wanted.
And then you fix it.
So you take a new action to compensate.
Now affordances comes up in Donald Norman, but I believe he goes into more detail about it in chapter 2 and chapter 1.
So we are going to go into different kinds of affordances and constraints.
So affordances are things that are aspects of the thing that you're confronted with that invites you to do something.
So we talked a little bit about affordances of materials.
Glass invites breaking.
Porous wood invites writing on.
Door handles are one thing that comes up a lot Donald Norman.
We have a door handle that is not all that different from this on the door in this class.
And that handle, it's about hand-sized, it invites you to grab it.
You could turn it, you could pull it, you could push it.
So there are bigger constraints when you look at a door like that door, that tell you about what that function of this handle is supposed to be.
First of all it's like a semantic constraint.
You've been taught: that thing is a door.
You know, you've seen a million doors just like that.
And so you say, it's a door, it must open somehow.
Have any of you just found a random door on campus that just doesn't open?
It's not locked, it just doesn't open.
Or maybe it opens into a brick wall.
Yeah?
[INAUDIBLE] No, not here
Right.
Did it just not open, or does it open into something bizarre?
Into a wall
A brick wall, right?
Yes, that's betraying the semantic constraint.
The door is not supposed to open it to a wall, it's supposed to open into another space that you can go through.
It's a door, it must open somehow.
And so you look at a door and see all the different ways that you can open it.
Now in emergency doors, you have these things often called crash bars.
And what you see over there is pretty popular, but not the only way to implement this idea.
A crash bar is designed so that you make it very, very clear that this door only opens in one way, to try limit how much mental processing you need to think about because it's usually put on emergency exits.
But this particular one, you could grab this.
You can grab this with your hand.
You could push it, you could grab it, you can pull it up, you can push it down.
There is a logical constraint, the more you grab your hands on something like this, you really can't see the lock.
You just see the door, and the door is closed.
As soon as you grab this, you realize that this hinge is spring loaded and can only go down.
So there is this logic, there's a logical constraint there.
This is a hinge that can go up and down, but already in the up position.
So in order to activate this thing, you push it down, which encourages pushing rather than pulling.
So it encourages you to exert force in the correct direction, assuming that the door opens outwards.
If it opens inwards, then you really put the door handle on the wrong side of the door.
There are also physical constraints, back to the example I talked about, the Settlers of Cataan board.
This is a different kind of crash bar but this one I think does a slightly better job because it actually kind of difficult to grab.
It is actually pretty wide.
And it's much wider and a lot of people's hands.
It's really easy to push.
So, just physically it makes it difficult for you to do the wrong thing, to pull out the door that generally opens outwards.
But because this doesn't tell you where it hinges, it doesn't tell you whether the door opens to the left or the door opens to the right.
If you push too hard on the wrong side of the door, where the hinges are, the door doesn't really open.
I guess it would be hard to open, but if you just gently push it, it won't open.
Which is why you get these, which is offset to one side.
And there are cultural constraints that are associated with that.
Like the blue slightly rubberized, grippy surface encourages you to put your hands on that.
There's a physical constraint.
Because it's off-center, if you push anywhere along this part you won't generate enough torque to be able to open the door easily.
I'm not so sure about this sticker that's over here, but I do think they kind of informed you that this is actually something where you put your hand.
because this is rubber and there is ink on there.
This is rubberised, and it's coated, and it's clearly hand-size.
It's inviting you to push it.
So that is cultural because of the colors, of the material, of the rubber.
That says, this is something for you to put your hands on.
So there are four different kinds of constraints that I went through.
There are cultural constraints, which is what I was talking about-- the blue.
There's the physical-- the physical constraints are just making it hard for you to do the wrong thing in the same way that certain pieces in certain games are easy to pick up because you are going to be picking up a lot.
Certain things are easy to roll.
Those things that are hard to pick up because you're not supposed to be picking them up, maybe you're supposed to be sliding them around.
There are logical constraints, where if you just look at it for a second, you will say well, it can't be this, therefore it must be that.
And finally, there are semantic constraints of when I say something is a door, you expect to be able to go through it.
Same thing for a game.
If you see a door in a game, it is labelled door, you expect that door leads somewhere in the game.
I think there is a version of Clue that I used to play when I was a very, very little kid.
And if you remember the Clue board, it looked like a floor plan of a house.
It has windows and stuff like that, and that implies that you can go outside of the house.
But you can't really go outside the house, that would be breaking the rules.
That confused me when I was five years old, because I expected that you'd be able to go through any door.
A couple of other things you can do to help people understand your game, it's context.
Where how things are placed next to other things that suggest how they're supposed to be used.
This is Boggle, and when you open it, you get these two pieces.
Well you actually get [INAUDIBLE].
But you get these things together, and as soon as you see this if you know what an hourglass is, you immediately know that this is a timed game.
Whether or not you read it on the box that this was a game that was timed, whether you've ever played this game before.
When you see a timer that's packed in the same box with things, that suggests that this is a game that you play with Time.
So that's one way of thinking context.
I'm going to go to a few more examples later on and I want you to think about how things are placed next to each other suggests how they're supposed to be used.
Context also comes from how you name things and how you provide art.
This is not something to worry about in assignment 1, but you're going to have to worry about in assignment 2.
How many of you have played Diplomacy?
Do you still play Diplomacy with the same people that you played with?
No
You don't talk to them anymore?
No?
OK. That's the problem with Diplomacy.
How good are you at playing Risk?
OK, if you just look at a Diplomacy board or a Risk board, they don't look all that different.
It's a world map.
I think in Diplomacy it's a modern European map.
But now, this game is a game of international intrigue, trust, and treasury, and called diplomacy.
And this game called the game of global domination.
Risk.
Just look at the way they're presented.
Why don't you tell me, what did these boxes tell you about these games?
Risk has a bunch of cannons and [INAUDIBLE], so you're probably going to be fighting and attacking stuff
OK. Yeah, more conflict
Diplomacy, it looks like guy is hanging around.
And there's a sword and wine glass, so maybe there's a bit of fighting, or a bit of wheeling and dealing, but it seems like mostly you'll be talking to people
OK, there'll be lot of talking in this game
I get the sense that they might leaders discussing a deal
There's a world map and a globe to remind you.
Yes, they're making decisions on behalf of all the world
If anything, it seems like these are people behind the throne and making shady deals
Ah, shadow presidents, or kings.
I guess at the time that this game is set-- the world of czars and kaisers.
So you're making shady deals with authority to be able to determine the fate of the world.
And Risk?
Stomp over everything.
I'm going to take my army and I'm going to meet your army and we're going to figure out who's got a bigger army.
That's the game.
That is another hint, in the titles Diplomacy and Risk.
Diplomacy is a game about diplomacy.
It's about talking to people and making deals, and breaking deals at opportune times.
And this is why people who people don't often play diplomacy with the same people anymore.
It is a great friendship killer.
Risk is a game about-- I'm going all in!
I might win this if the dice roll in my favor, so it's about risk.
The game is about risking everything on big gambles.
And it's actually a pretty good example of what this game is expecting you to do.
So immediately, just by looking at the box, the games are already trying to condition you to think the way you need to think in order to well in this game.
Let's see.
Visibility.
This comes up a million times in Donald Norman.
And when you're designing your games, you need to think about how to be able to make what's happening in your game more visible to the players who need to make decisions about that.
This is a very close-up view of the game called Cosmic Encounter, which we will play later on.
And it has this Risk- like element that I am going to put my army of flying saucers against your army of flying saucers.
But we're going to be fighting on multiple fronts at once, and that's actually going to be-- in the game, you can join forces.
So I can keep up with you to go up against a planet that's been held by you.
But somebody else is going to join in and try to help defend your planet.
And so they have this system of stackable tokens to basically be able to then very, very easily see whose pile is higher.
Right now, these are all the defenders, so they are all on this little [INAUDIBLE].
There is actually this big flat spike where you stack all of the attacking ships and you put them in the direction on the planet you are attacking.
So to just makes things very, very clear, this is how big the army is that's attacking is, and this is how big the army that's defending is.
Even though there are many, many, many, many different forces involved.
So they're making visible what the current state is.
Again it's kind of a diplomatic game of like Diplomacy.
In the middle of every combat, you have both sides asking all the players not involved in the combat to please contribute something to this-- please contribute a ship or two to this attack.
And they're making deals all the time.
Even days are not really about visibility take advantage of visibility.
The whole idea of Battleship is that you don't know where your opponents are.
But they gave you all these useful tools, these red pegs and white pegs, and a whole extra grid to be able to keep track of where you bombed before, and where you were successful
Can you imagine without the second grid?
Yes, it's possible.
I mean, you would probably end up drawing your own second grid pretty quickly just to keep track of where the you hit before.
There's other things up here, as well.
[INAUDIBLE] actually that's what it's describing-- how big each ship is.
You've got your own fleet to keep track of, what the possible ships are, and how long they are.
This person uses white tokens to keep track of where the enemy has bombed, which is something you don't have to do in the game to do well.
It is probably useful information, but you have the tools.
The player has all of these tools to be able to keep track of that information.
So this game has gone a long way to try to make the little bit of information that you do get over the play of the game-- the little of information that you reveal over the game as visible as possible, so that you can make the next logical decision as easily as possible.
Previous user experience goes a long way in helping people understand games.
And these are dice with nontraditional numbers that you expect to find on a dice.
But you see a dice, you kind of know what to do with it.
You pick it up, you roll it, and it will give you a number.
And that number is a number that you have for that turn.
There's also the little dot that distinguishes the nines from the sixes.
That comes from other games.
I think a lot of people see that from games in particular because I can't think of any other situations in real life where something might be flipped upside down and you have to interpret that number.
In fact, there's a Kickstarter out, or at least that was.
The Kickstarter for this project, we took a whole bunch of cards, and a whole bunch of different randomized things depicted in a photoshopped photo.
So it was a whole deck of cards.
I think it was a standard poker deck.
But it also had all these other randomizing things.
It had a dreidel, it had a dice with [INAUDIBLE] on it.
It had different size of dices.
It had one of those counting sticks.
There's a lot of things here that I am not really sure if I could tell you the name of it, but you could probably figure out how it works.
What does that look like?
It looks like a compass.
Now, knowing that this is a randomizing element, how do you think that will be used if actually had it in real life?
It's a spinner
Yes, it's a spinner.
It's a [INAUDIBLE] spinner, in particular.
You can spin for one out of six sides on the inside, and one out of eight, sides in the top.
But there are a couple of key things, like the coins.
You could use this deck of cards for coin flips, if you wanted.
And there is also these weird fortune cookies.
Anyway, it's a cute idea, but you don't-- I don't need to explain to you how to use this.
It's just a whole deck with different pictures on it, and you know and you can use it to randomize different letters and different numbers
[INAUDIBLE]
I forget the name of it
No, what was the point?
The point was that you will get a deck of cards with every card is different.
And every club looks kind of like this but it has different randomizing results on it.
Cultural cues.
This was actually played by a pair of friends.
There is a game called War on Terror.
There's a very satirical game.
Basically, everybody has exactly the same strategy.
Everyone's in charge of a country, basically.
Everyone is after oil.
Everybody has the same strategies and the same tactics.
But if you cross a certain point, you get branded as evil and you have to wear the evil balaclava.
And that immediately is a trigger for everybody to gang up on that person.
So what the game is really trying-- the point of the game is that the game is trying to drive home the point that everybody is engaged.
All the powers are engaging in exactly the same tactics.
At the moment that you actually get declared as evil, you actually get a little bit more freedom than what you're about to do because you don't have to worry about other people branding you as evil.
So the whole idea of-- here is this person in this balaclava.
And it had-- it had the word evil on top to make it clear, but I don't think that was actually necessary.
[INAUDIBLE] a regular [INAUDIBLE] I guess it caught the point that this person is now marked as the bad guy.
It is actually a metaphor for something that-- I actually had some trouble coming up with board game examples.
it happens a lot more often in computers, where you get some sort of metaphor to imply how it's supposed to be used.
You're not going to exactly use it that way.
So for instance, what are these things up here
[INAUDIBLE]
[INAUDIBLE] So what do you think you do if I'm in the game and in this piece of software?
You're going to read the inbox or put it into the outbox
You can send messages, and read, and check what messages you receive.
That's a metaphor that we keep it mail software today.
Anybody recognize that?
It's a Rolodex
Oh, people still recognise Rolodexes.
That's cool.
What does a Rolodex do?
Contacts
Contacts.
So that just gives you a list of all the people.
this gives you an idea of how old this screenshot is.
This looks like a planner, some kind of scheduler.
Magic lamp.
Genies.
You click on it and can ask things.
Help, maybe?
Not quite sure
Wishes
Wishes, maybe.
Wishes maybe connect you to tech support.
I don't know.
You can see the metaphor break down here.
This is a phone?
A fax, maybe?
A fax maybe.
I'm looking at this, and I do a little bit about the machine that this was designed for, and it didn't have-- you couldn't make phone calls with it.
So I have no idea what this thing actually does.
Maybe send pictures.
This thing looks like a--
A purse or briefcase
A purse, or a briefcase or a bag.
It's a metaphor or something, but it might be cash transactions.
But I doubt it.
It could be your files that you save, who knows.
Things are starting to break down.
But you still have this desktop metaphor that we still maintain in a lot of user interfaces.
This is somewhat-- this is a real kicker.
This is extra [INAUDIBLE] the hallway.
I have no Idea what to expect the hallway.
User interface.
Maybe other applications.
We have that a little bit in both games, especially board games that are produced in many, many, many different languages.
Because they have-- they can't slap words on the tokens.
Because if you slap words on it and want to then release the same product in a different language for a different country, then you have to change all the words and that increases manufacturing costs.
So they try to use generic tokens and then just explain everything in the rules which they can print fairly cheaply, and they can include six European languages if the same box.
That won't be a problem.
This is a game called Agricola.
And it's called a worker placement game.
Because what it basically is, is that you've got a bunch of members of your family of your little farming homestead.
And then you send them out every turn to do tasks, like go to the fireplace and cook some meat, or go to the market and buy some more feed, or build some fences.
Something like that.
So what you do is you take these tokens and you put that on the cart.
So this is both context and an interaction metaphor.
By taking this thing and putting it on this cart, I'm achieving quite a large number of different things.
I'm saying, this is the action that I'm taking.
I am saying I have one, two, three, four, five different things that I could do.
Actually, I am not sure.
but there's at least four different things that I can do on my turn.
Every time I take one of these things and then put it on a cart, like the cart that you see you on the right, that means that that's going to be my action for this turn.
The other thing that this game does is that once what I've decided that I'm going to do that, I think in Agricola once you take it, nobody else can take it.
So I can take over the fireplace and no one else can take over the fireplace.
I can put two people on the same fireplace,certainly.
So that's a metaphor, right?
Now it doesn't really matter that you-- you don't really have to count everything in family members.
You could just say you have four actions that you can take per turn.
And you just choose which four actions that you get to do, do all the math that is necessary to improve your farm and that's it.
But they turned your family members into your action counter, into your action points.
The more family members that you have, the more different actions that you can take within your turn.
So your family members are your metaphor for action points.
We get the same thing in other games with things like workers.
Finally, just a very quick word about accessibility.
This is usually something I go into much more detail in in my computer games classes, but with board games classes one thing I do want you to keep in mind is colorblindness.
I want you to, If you have colorblindness, or if you have friends who are colorblind, you definitely want them to take a look at your design, and see whether it is possible for them to play your game.
Take a picture through Instagram, or something like that, and put in a black and white filter.
See whether your game is still legible and possible to play with all the colors removed.
You can accomplish a lot of things by just changing up your tokens, by putting additional marks on the tokens that are high contrast so that they don't-- they are not completely reliant on color.
If [INAUDIBLE] you can look at bright and dark as well.
That's all.
If you take that into consideration in your design, you will definitely get kudos from the instructors when they are grading.
Because that means you have taken the effort in trying to increase the range of people who can play your game.
So, that's pretty much what I have to say about usability for today.
Any questions?
This is not something you have to worry too much about for assignment one.
But it's something that you definitely have to think about whenever you're trying to create a game for lots and lots and lots of other people to play.
So you will want to definitely start taking this into consideration.
What pieces are you choosing?
What do the pieces tell you about how to play this game?
How does your board tell you how to play your game, even if the rules are not printed on the board?
What are the clues that you're giving the player to be able to form that mental model, so you can get them past that learning stage where they're just making mistakes and start making decisions about how they want to play the game in a way that's satisfying to them.
And giving them feedback on whether those decisions works out for that.
So that's the presentation.
And now I guess it's team time to work on your projects.
I am going-- Rick and I will go get boxes and we'll be right back.
Anyone needs to take a break now is a good time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Again, some of this might be familiar to folks that have taken CMS 611, but I revised this stuff so that I can give you more board game examples.
I just want to go a little bit on the kind of aesthetic thing that we're talking about in Assignment 2.
We'll go more into things about user experience over the next couple of weeks.
But so far, we've been talking about this kind of aesthetic.
It came up in the NDA paper, where we've been talking about mechanics, we've been talking about strategies that come up and the dynamics of the game, coming up.
The way your rules play out to be able to give you a sense that some games are about subterfuge, and some games are about pressing your luck.
There are some games about just dealing with random things that happen to you, and some games about dethroning the king.
These are what a lot of your games have been about.
Some games are about feeling smart.
it's like when you make a set, it's like, oh, I feel really smart about that.
And those come out from your rules.
That's what Assignment 1 was all about.
You've done a great job with that.
But of course, that's not the only thing that's involved in the aesthetic of a game.
We're talking about these things as well.
All right, we already talked little bit about game bits with the Donald Norman discussion.
But also things like the tone of the writing in your rules.
In Assignment 1, it was fairly straightforward.
But you could imagine writing everything in Ye Olde English or-- I think Space Alert is a game where the rules are written as if it was a cadet manual, only a cadet manual that is setting you up for a mission that is doomed to failure.
Space Alert is actually a really good example of sound as well, and I'll play some tracks from it.
It's a game that has an accompanying CD.
But what other games have really distinctive sounds, like board games, card games, casino games?
Like what are the sort of things that you--
Operation
Operation
Operation?
Yes Bzzt.
Yeah
Sorry
Sorry?
Because you say sorry?
No, I think the whole thing is--
Yahtzee?
[INTERPOSING VOICES]
Yahtzee?
Taboo?
Taboo has a buzzer.
But also a lot of-- It's a dialogue game, so that's also an accompanying sound that's coming out of everybody's mouths
Gestures
Gestures?
Gestures
Oh, I haven't played that one
It's like charades, only that you've got a timer.
And the timer has a really loud tick to it
Oh, so it ticks down so you can hear it running out of time
Jeopardy
Jeopardy.
[HUMS THEME]
Poker, where they shuffle the chips all the time
Oh, yeah, yeah.
That's the little tinkle of the chips and--
Slot machines
Slot machines, of course.
Yeah Or rather, a whole bank of slot machines.
You very rarely hear a single slot machine operating by itself.
Electronic slot machines, obviously, have a slightly different tone to it than mechanical slot machines.
Roulette, I find, has a very, very distinctive sound.
Just that gold ball bouncing all over the pumps.
So definitely think a little bit about that, about the sound of a game.
Especially if you're making a game that is about conversation or about over-the-table.
exchange.
Taboo is a game that is punctuated with a lot of ums and ahs, whereas something like charades, the sound of that is a lot of shouting, usually.
Everyone just saying random things, the first thing that comes to their mind.
It's a free association game.
And, of course, the look of your game.
That should be taken for granted, and we've got a couple of examples of various boxes that you take a look at.
And of course, that's what people normally associate with storytelling and games.
But, it could also be thought of as world building in games.
What's the theme of your game?
What's the plot and the characters in the game?
Are you making instead of the Trivial Pursuit game, the Twilight Trivial Pursuit game, or something like that.
And you've got the setting, you've got characters, even though the game mechanics are all the same.
But the games that integrate better with the game mechanics and rules and art direction, and games that do a poorer job, we've got some examples of that.
So, let me see.
Let me talk about a few of these games.
How many of you play Forbidden Island?
How many of you play Pandemic?
OK, so Forbidden Island is by the same designer, and it still has a lot of the same cooperative feel.
First of all, you have this metal tin that makes you think that this game is worth more, and you could be paying more for it.
But you've got this nice little art, and because you have to open up this metal tin, makes think a little bit about an opening up a treasure chest.
Inside, you've actually got these nice treasure pieces.
A goblet.
This kind of translucent crystal.
They're all plastic, kind of rubberized plastic so that they will last a little bit longer.
But you can see that this has got [INAUDIBLE] lion, griffin, or something.
You know, its got some of that old world, here's some sort of mystic civilization.
And when you look at the cards, you've got the choice of the fonts.
You've got these kinds of very serifed and a little bit broken-up text that looks like an old letter press.
And, of course, all the art that's on every single card.
The game mechanics of this game are about various parts of an island turning into water.
And they don't actually have different illustrations when they get flooded.
They just have the same picture, only bluish.
It's cheap to print this because its monochromatic.
I guess it's not really monochromatic because they have a color token at the bottom.
But it gives you the idea that, when you play this game, everything looks nice and colorful.
And then as the game goes on, things start turning bluer and bluer.
So it gives you the sense that there's more and more water surrounding this island as it keeps flooding.
And all these pieces, which represents-- each one is different players' token, doesn't necessarily need to be shaped human.
But it makes you think that there is a human head on the piece.
All this adds cost to the production of this.
Of course, the metal tin also adds a cost to the production of a game like this.
But it goes a long way to making you feel like, I'm not just moving pieces on the board.
I'm actually exploring an island and robbing it of its treasures.
But it's trying to get that.
This is actually a counter of the flood levels.
And the game mechanics stuff, says novice, normal, elite, and legendary.
So it sets you up with a difficulty level when the game starts.
But you have a water level that goes from bottom to top, as opposed to top to bottom.
You can imagine this counter working just as well going downwards or upwards.
But since it's about flooding, it goes from bottom, and then the water level rises and rises and rises.
And of course, on top, they have a skull.
It's not just that the game ends.
It's the game ends and everyone dies.
Let's just be clear about that.
So it does a nice job of getting that across.
It actually is a cooperative game and does a pretty good job of taking all of these ideas, the setting of the waterlogged island, the art direction of what happens to the pieces as you flip them over.
And then, of course, the rules that determine when the water level rises and how much space you've got to navigate and how you deal with those problems.
I'd like to describe all of the stuff that we've been talking so far as systemic aesthetics.
So these are the aesthetics that the system is creating.
All the things that have to do with art style, writing tone, the sound, the pieces that you're holding, more or less stylistic aesthetics.
And then there's the fiction.
The fiction will be the story, will be the characters, will be the setting of the game.
But of course, it's a blurry Venn diagram, because there's a lot of things that come in-between.
So, for instance, the way your board looks.
It's, obviously, a combination of the mechanics that specify the information that you need to be able to put on the board.
But also the style of how do you display that information and how that reinforces the setting that you want your players to be in.
The kind of feedback that you give people, like the Taboo buzz, for instance.
The sound of that buzz is in here.
But obviously, the rules are set up so that, the moment you hear that buzz, it's kind of like it's as jarring and nerve-wracking as possible, because you're trying to concentrate on not saying the wrong thing.
And the moment you slip up, another player will buzz it, and then you get the shock of your life.
That's all intentional.
For art direction, and for style and fiction, those tend to be things that people have less trouble thinking about.
How realistic do you want your setting to be?
There's a game called King of Tokyo, which is really, mechanically, it's everyone ganging up on the leader.
And this is a game where you're trying to be a monster taking over Tokyo.
And there's a lot of other monsters who want your spot.
And once you are on top of the mountain, everyone's trying to take you down.
That's what this game is about.
But you can see, it's got this goofy, not quite anime style.
But still evocative of old Japanese movies.
You've got a little picture of Tokyo being on fire here.
You can actually see the monsters in here, you see a lot of helicopters and everything, see all the collateral damage you're doing.
I believe it's mostly a dice-rolling game
Yahtzee, basically
Yeah, it's Yahtzee.
So the mechanics are all about, you roll the dice and you try to roll well.
So they put in a lot of effort to try to piece that mechanic inside this aesthetic and fictional elements to make it a little bit more engaging and, frankly, a lot more saleable.
This is Richard Garfield, who's also the designer of Magic: The Gathering the original designer of Magic: The Gathering.
So it's got a good pedigree.
In between the systemic and the fictional elements, you can do things like events.
Twilight Struggle, another excellent game, is about the Cold War, basically.
And you've got cards in here that remind people of real things that actually happened.
They don't necessarily happen in the order that they happened in real life.
There are things like the Space Race.
There's things like various precedents, and various Soviet and US leaders from the Cold War.
This one specifically says 1945 to 1989.
And so while you're playing this game, there are mechanical results for all those cards that you're flipping over, right?
Because they also have pictures and names of the actual events that, hopefully, this is largely targeted at people who have lived through this era, or at least remember things that happened from that era.
And it's trying to be evocative of that.
Whereas it's a fairly mechanical-- I guess it's not quite a war game.
It's more like a politics game.
Balance of power game.
But they use events to remind you of what the setting of this game is.
I call it fiction.
But of course, this is not fiction.
This is based on what actually happened.
It's still a setting.
Roles and abilities.
Back to Forbidden Island, every single character that you play in Forbidden Island has a different set of abilities that you've got.
You're taking on a different role, like an explorer or a diver.
I can't remember what all the roles are.
There's no indication on the box on what you can be.
But since everybody has different abilities, that hearkens back to the unique defining things of the character that you're playing.
And you try to think in that role.
If you are a pilot or something, and then you try to think about rescuing people.
If you're a diver, you are trying to forge ahead and going and you can stay close to the water mechanically.
But also puts you in a fictional position in your team to be able to be closer to risk, because other people will just drown if they're too close to the water.
I think that's how the game works, if I recall.
So for Assignment 2, we want you to be thinking about all of these things.
Obviously, it's difficult to do this on the amount of time and the budget that you're working with.
But we want you to consider how you game looks, how your game sounds, the setting that your game is in.
We've already had a couple of things like the spies game, for instance.
Could this have been a formation of a game about hidden variables?
But then you have this layer of being an agent of an unknown country.
And sometimes fiction can actually help people better understand the underlying rules that you've got.
I know that this game, the level of the game started off as a building construction game.
And to a certain extent, some version of those rules might actually help people understand your rules easier.
But sometimes it gets in the way because your fiction may not necessarily match up with the way that your rules are evolving.
I'm not saying that you should let aesthetics necessarily constrain what you can do with rules.
But if you're going to change your rules, then you're going to have to iterate on your rules to make your game playable, and do everything that you've done with Assignment 1.
You want to make sure that your aesthetic is coherent with that, so that you don't have things in your storyline or your character design or your art style that are outright contradicting what your rules are trying to tell you.
This is a game about friendly cooperation, and you're saying all of you are on this island together and trying to save yourselves.
But really, the way your rules work out is every man for himself.
That would be a contradiction.
This is an interesting game.
Who's played Ticket to Ride?
OK.
Box design and title can play a very, very large part in setting that expectation.
So it wouldn't solve something I stressed so much in the first assignment, what's your title of your game.
But in the second assignment, the title of your game can sometimes be the first clue of what your game's all about.
It's a rail building game.
You can see from the back a picture of the board, and you're building these tracks.
But if you actually read the box, it says, "October 2nd, 1900, 28 years to the day that north London eccentric Phileas Fogg accepted and won a $20,000 bet that he could travel around the world in 80 days.
Now at the dawn of the century, it is time for a new impossible journey.
Travel across the United States.
The impossible journey.
"So some old friends have gathered to celebrate Fogg's impetuous and lucrative gamble, and to propose a new wager of their own.
The stakes: $1 million, and the winner takes all competition.
Your objective: To see which of them can travel by rail to the most cities in North America in just seven days."
It's interesting, because, at first glance, you look at the game mechanics.
And the game mechanics actually feel more like you're building railway connections, rather than riding the rails.
But if you look a little bit closer at the strategies of this game, what this game really comes down to is blocking other people from completing their routes across America.
Which I think is closer to, I think, the theme that it's suggesting.
It's not about who can complete.
It's not about, is it possible for you to complete a route around America in seven days?
No.
It's who can do it fastest, which means how do we slow down everybody else?
And that's actually what high-level strategy in this game turns out to be.
So it's about all these tickets that you're picking up, and you're going from place to place.
You're trying to compete routes, and you're trying to prevent other people from completing their routes.
One of the interesting things about this game is that you've got all these people that are on the box.
And, of course, the assumption is that you are one of these people.
And that's deliberate.
The board game designers wanted you to select someone on the spot that you could personally identify with and says, I want to be this person who looks like [INAUDIBLE], or this person who looks like Phileas Fogg because I like those folks.
But then, when you actually play the game, there is no real embodiment of any of these characters.
There's a little piece that you place down that says, I am this person.
And the only sort of level of fiction that's represented in the rules are the tickets and the routes that you're taking, not the people.
So they've gone to this very, very large effort to make you think that this is a game about people, just because they thought that games about people tend to sell better than games about trains.
But the games are really about trains.
So I guess [INAUDIBLE].
Let's see.
You want to talk about these other games?
So Alien Frontiers.
Does anybody know the term worker placement?
Worker placement game where you have a limited number of resources that you're just basically applying to different sections on the board, which give you points or do things for you.
It's a great game about colonizing this world.
So a lot of really good, old-style '50s and '60s style of science fiction art.
Lots of rockets, spaceships, things like that.
And your actual bits, and the reason I'm using this one is because this is a game that starts off with a plasticky Euro style.
So cubes and things like that.
They don't really represent anything.
Your bits, your ships are these dice.
You buy a rocket, you roll the rocket, it gives you a number.
And then using those numbers, you can apply it to various different places on the board.
So [INAUDIBLE] are actually pretty good, because you can put the piece down, it stays there.
If it gets moved around, it doesn't really matter if it changes number.
Because if you use that number just when you're placing down the piece, the number is no longer relevant anymore.
One thing we see in a lot of student games, people using dice as counters.
A little bit easier with die 6 because its squared.
But once you start using, like, the d20s as a counter, they've got really small facets.
And they're going to roll and you're going to lose your place.
Also, the amount of time it takes to go from one number to the other number really increases the times it takes to play the game.
But one thing they've done, it started off with these little chips.
Now they've added more plastic bits.
Special powers you can get to move around that look zany.
Little alien pieces of technology.
And colonies, you can play, they kind of look like colonies now.
I don't have it here, it's not released yet.
But it's on order.
They actually made the rockets dice, so the dice in the shape rockets.
So imagine this as an oblong shape that's kind of curved with cut faces on it.
One thing I'm curious is, how is it going to roll They've made it so that it feels more representational.
Like it feels more like part of the theme, but it might actually take away from the actual feel of rolling dice.
It'll be interesting seeing how that plays out when I play that comes out
When you actually role the dice, it has its own distinctive sound that is shared among people have positive associations with that sound
And then the other one, a game about trucking
Heartland
The Great Heartland Hauling Company.
So on the box, you see a semi truck carrying little squares.
On the back, you see semi trucks and squares.
But the squares are in a map.
But the squares don't really fit on the trucks.
So, actually, I brought this out thinking, that's a great way of saying it's not-- Kind of like the other one, they're showing something on the box that doesn't actually exist in the game.
But I found out, and I actually took it out and opened it, they have cards where you put your bits on.
So you are kind of carrying things around.
But there's a disconnect between where things are on a map and where things are on your truck.
And then where your truck is actually placed on the map.
So your eyes are going to different places when you're actually playing out the game.
Largely done because of the commercial manufacturing constraints of it.
As you can see, it's basically just two decks in a small box.
You'll see a lot of these games, they want-- If it's a game I want you to play in a short period of time, it's likely going to have a smaller box than a game that's going to play longer.
It's a code for how complicated the game is, or how much of your time the game is going to take
So definitely pay attention to how your pieces feel.
Again, we're not asking you to go completely to town, like laser-cutting pieces or anything like that.
That's not necessary.
Just that every time you think of the pieces in our box, or we decide to glue two pieces together or to build, make it a different shape or something, think about what that's communicating over to the player.
Think about what the box may be, or what the art in the rules is conveying about how the game is supposed to be played, and try not to mislead your players into thinking that this game is really about stacking things on little plastic trucks when it really isn't.
It's about stacking things on cards.
That sort of thing.
So, yeah.
Any questions about Assignment 2?
And then on the assignment, it's not asking for polished art?
Yep
So think about the materials, think about the textures.
It's still a short assignment, so don't think about that kind of art
Yeah
More which kind of pieces you decide to use.
Really think about the title.
It's really, really important
Yep.
The title is something, writing, it's something that you can spend-- you can get a lot out of a little bit more time investment.
The tone of your writing and your rules can go a long way.
But still, looking for rules that are easy to read.
So don't use Shakespearean styling or something like that.
That might make things a little bit tricky, unless you're very, very consistent with it throughout, and you're only using it for nouns or something, which could work pretty well.
Sketchy art is fine.
Just like a hint of what you are trying to get across with your art.
We're not judging anybody on your ability to draw.
But we are interested in your thought process and how you consider, how you represent your game.
All right
So the first line of the assignment [INAUDIBLE].
Do we get to choose the experience that we want to--
Yep.
And that comes down back to the overall aesthetic.
All of these things are working together to generate a player experience.
And we'll be going through some examples of other games that generate the user experience of panic.
The experience of close shave cooperation.
Again, remember, mechanics and aesthetics.
The aesthetic thing is the player actually experiences.
And that's true for any definition of aesthetic.
The question is, what are all the things that you can do to hopefully generate that.
Of course, you only know whether they're hitting that [INAUDIBLE]
Is it supposed to be a quick game?
Do we have a time limit in the--
I think I just copied it over.
20 Minutes
20 Minutes So aim for that.
It's a little longer than the last one.
But yeah.
So, yeah.
And the reason for that is largely, we want it to be able to grade these things and get your grades back to you.
So if you made a two-hour long game, and everybody makes a two-hour long game, we're never going to finish grading this.
So short games help.
And short games are easier to test
And for the first question, on Monday, we'll be talking about how to choose an experience [INAUDIBLE].
We'll do that right before we get into brainstorming and team forming
Cool.
All right.
I guess we're ending a little early today.
All right.
Thanks, everyone.
Don't forget to sign in if you haven't already
Just about everybody did
Thank you.
Oh yeah.
Please hand in your
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So let's talk about the reading, which was about randomness and player choice.
Everyone has an opinion about randomness and luck, right?
It started off the reading about that kind of thing.
So just to start things off, what are your opinions about games with luck?
Do you like them?
Do you not like them?
And what do you like about games that have a luck element?
I personally love a mix of skill and luck.
So my favorite example would be poker.
So you could pick up the game really quick, and you could be a novice, and you could win if you get lucky, and it makes you feel good.
But in the long run, over a hundred thousand hands or more, it's the person with more skill that will end up winning in the end.
So I like that dynamic of being easily accessible, but also having the skill element
OK, so that accessibility curve, that little bit of difficulty curve going in there as well.
Yeah?
I would say that a lack of predictability is good, and luck isn't, but but luck is necessary to achieve unpredictability sometimes
OK, how do you mean?
So basically, you need luck that the game won't go the same way every time, not because you actually-- the randomness-- on portions, it has to be random, but that's the only way to make sure that different games take different paths
OK. Yeah?
I like the unknowability aspect of it.
I agree with [?
John ?]
that I like a balance between luck and skill.
And what I was thinking of was something like Catan, where you have this whole plan that you're trying to work out-- OK, I'm building my city, and all I need is an ore, and I just need a [INAUDIBLE], and then I can build my city and do all this stuff.
But you just have to wait for it and plan around, OK, what if I don't get what I need?
Things like that
Yeah.
Yeah
I also think this is incredibly subjective
Absolutely
So [?
John ?]
mentioned that he really likes poker.
He likes [INAUDIBLE].
Personally, I like games with a lot less luck than poker, but that still have a little bit.
But then again, I know tons of people who are like, oh, my favorite game is pokeno.
Sorry, which is all luck, essentially.
And you feel like you're doing stuff, you do have decisions about which piece to move.
But in the end, it's essentially 90% luck
Could I comment?
Please
Can I go on what you're saying, Ben?
Yeah
Even if you take a classic, 100% skill-based game like chess, for instance, in a professional tournament setting, there's still luck.
There's a luck element to who you're playing, or whether you get white and black, or whether they're having a good day or a bad day.
I think there's always luck in games, even if it's 100% skill-based
Does anybody know how white's determined when you start a chess tournament?
Usually based on your rating.
And they also, in a fair tournament, will play a match where each player plays three games of white and three games of black
Yeah.
So they're trying to work as much of that luck out of the system.
But you're right-- there is some, a little bit.
And there's reasons for doing that
I would say chess is-- I mean, chess [INAUDIBLE] is sort of more predictable than other games [INAUDIBLE]
[INAUDIBLE] always going to be [INAUDIBLE]
Yeah, chess is more predictable.
I mean, I don't think what I'm [INAUDIBLE]
So it's like, for example, think of round robin tournament play, where the person you go up against depends on, to a large extent, how you've done in a tournament.
But that person who you go up against is basically chosen from a pool of people who have scored pretty well.
So there could be a grandmaster in that pool that gets selected, or there could be a weaker expert player
A lot it depends on what you consider part of the game, also
Yeah, exactly right
You could say the selection process.
You could also say, oh, I got lucky that my opponent didn't sleep last night.
Is that really part of the game, or is it just that your opponent was a weaker player on that day?
You have some more, [?
John?
?]
[INAUDIBLE] what I wanted to say is, was that, back to the question, with [INAUDIBLE] in terms of games, I prefer games which have skill at certain game elements, but it's not so much that they don't have luck as you don't have to deal with the luck instantaneously.
Like, for example, the board setup at the beginning Catan.
Or when you have, say, a deck where the top three cards are the cards you're choosing out of, or something, where it's randomized.
And so every time, it's going to be different, which is the good part.
And yeah, it might favor one person over another, because of luck, which is that part.
But on the other hand, you still have time to deal with it with your own strategy
I'd say that probably colors into that chess, too.
The setup of the board is a lot like the framing of a tournament, or the context the tournament is played in, the context of the players
So I would say that whether or not a game based on luck on chance-- it's also dependent on what my purpose for playing that game is
Absolutely
So I really like sometimes playing chance-based games, and I've gone home for break, and I'm going to see my brother.
And I haven't talked to him, and I haven't been with him in months, so we can play games.
It's not really about the game.
The game is just to do something while we talk about what we've been up to.
Whereas once I'm home for a really long time, we'll play a skill-based game.
And it's really all about the two of us competing at the game [INAUDIBLE].
So it's also not necessarily subjective by person, but by situation
Yeah, absolutely.
Yeah?
So in the game Dominion, there are basic [INAUDIBLE] cards all the time.
And then there's a bunch of special sets of 10 cards, which [INAUDIBLE] game.
The strategy [INAUDIBLE] the special 10 cards, and just buy-- and to do the optimal strategy on the stuff that's in every game, often you can win 30% to 40% of the time against the best possible strategy with [INAUDIBLE], even if you're using another set of cards, which is why-- which shows that-- But they're [INAUDIBLE], but at the same time, the game has a lot of strategy involved, despite the fact that it comes down to luck.
And so when people are like, oh, this strategy is better than this strategy, because when we simulate 1,000 games, it wins 60%, 70% of the time.
As opposed to actually playing the game and showing one game won
Yeah.
Well, that's a huge thing, too, of how many times are you going to play this game, especially the games that you're making in your assignments?
How often are these games played?
Hopefully, you're testing as best you can, as much as you can.
But I don't think you're going to get 1,000 plays out of them, right?
Unless you're going to throw it through a computer.
You don't have to.
Please don't.
Getting real people to play it is a huge component of it.
Because there's something about luck-- I don't think we've talked about yet-- the ritual aspects and the performative aspects of luck and chance.
Can anybody think of a game where there's something very particular to how luck feels there?
Maybe you can take luck out of the system, but why you wouldn't want to?
So I actually think about, in that kind of context-- but where you're talking about craps, you're talking about gambling games, the rituals that go around gambling games-- blowing on dice, rolling dice.
What's craps if you remove the dice and replace it with some other kind of-- technically, getting to the exact same probability system, but different materials?
Is that going to be a different game or not?
Could you replace it with a spinner [INAUDIBLE]?
Yeah.
Is the probability curve the same?
No
No, but you probably could
Yeah, you could probably break it down into [INAUDIBLE] the same
Yeah
I'm thinking from a pro aspect, where not exactly like craps, where you're playing against the house, the house is always going to have the edge.
But the poker aspect, where you're playing against one other.
And yeah, [INAUDIBLE] if you're that much better than everyone else, then you can play professionally [INAUDIBLE].
In which case, you wouldn't want to take the luck element out of it, because your customers-- that is, the people you're playing against-- aren't going to want to play against you if they realize that they're so bad.
But if they win every once in a while, and they feel like they're winning often, but not a lot-- yeah, [INAUDIBLE] You wouldn't want that [INAUDIBLE]
Yeah
[INAUDIBLE] typically a large percentage of people that actually play poker now make money, even when include-- I believe it's somewhere between 1/5 and 1/2 of people will make money playing poker even without [INAUDIBLE]
I think it's a lot lower than that.
It depends on how good the table is
Are both these cases you're talking about in casino play?
I'm talking about [INAUDIBLE] poker table at [INAUDIBLE].
And that's including [INAUDIBLE]
Well, I'm not sure if you mean in that session, or over the long term.
But I think [INAUDIBLE]
Even on [INAUDIBLE]?
I think in long-term, probably 10% [INAUDIBLE]
98% of poker players are long-term losers, [INAUDIBLE].
[LAUGHING]
What website?
I don't know.
TexasHoldEm-Poker.com
Sounds incredibly authentic
No, yeah.
[LAUGHING]
Go ahead
So to your question about craps, my first, I guess, response-- my thought was that if you change the dice to a spinner, even if it has the same exact distribution, it seems like a different game.
So I started thinking about it, and if you have a game like online craps, that's kind of what you did, except you just show them a picture of dice.
And that feels like it's the same.
So I guess I'm not sure
Yeah
Maybe new rituals would come up with the spinner.
Like maybe if you're blowing on dice, you rub your hands or something.
[INAUDIBLE]
And the question of mechanic versus aesthetic.
It's the exact same mechanic, right?
You have the random number generator that somehow gets you money or loses you money, with very specific rules on how it works.
Whether it's the die, or the spinner, or a computer, that's what the user's experiencing.
But at the end of the day, it's just like Monopoly-- at the end of the day, you have a random number generator that's moving you around the board, and you collect stuff.
How they skin Monopoly doesn't really matter-- it's still Monopoly
Exactly.
And they showed it
I guess going off of the feel, and the fact that with dice, some people believe it takes some type of skill.
They feel like they can roll 6's more often if they, I don't know--
There's a lot more tactility to dice.
Like maybe if I'm just cupping it the right way, maybe if I put the die facing the right side up on the palm of my hand when I roll it, maybe there's going to be something there.
Maybe I've got the skill there.
Maybe it's just for the back of the house.
Maybe it's for the casino sites-- who knows.
But yeah, you're tricked into feeling like there's some choice that you're able to make to change the outcome
So to build off of [?
Damon's ?]
point, that helps clarify my idea on it.
Because I think part of craps, then, is the aesthetic experience.
And to me, it feels like the dice, or at least the idea of dice, is fundamental to craps.
If you don't have it, then it ends up being a different game, because it's a different aesthetic
So going back towards the beginning of the reading, one aspect of this aspect the reading is framing itself in is target audience.
Who is the target audience for the game you're making?
And it starts off with Candyland.
Candyland, the most random, the least predictable, the least amount of player choice that there is in a game.
But because of the audience that it's being designed for, that's OK. That's actually beneficial in the examples they gave.
So have you thought about the target audience for your own games right now?
Do you have a clue about what that might be?
Or the context that you were saying, the framing you might think-- I know with your game, at least, there's some kind of party atmosphere going on in it because of the physicality involved, right?
Any others?
OK, because one thing to think about, especially when you're coming to this from a designer [INAUDIBLE] and know the assumptions you have, and the personal likes and dislikes that you have.
And you're going to be on a team, you're on a team of multiple people.
You're all going to have different likes and dislikes.
One thing to try to really figure out early on is who is the person playing the game, and what is their dislikes, and what are their likes?
At this point, just assume.
Just come up with a random target audience person, and just make that assumption about what they may or may not like.
Because I know you're going to have discussions, arguments over which one is best or not.
How much randomness are we going to include in this game or not?
Are we going to include this mechanic over this mechanic or not?
Rather than making it about what you personally like, maybe try taking it to what another person is going to like.
How it's actually going to played.
And you might not find that out until you're actually in the middle of play testing and you're getting it in front of other people.
[SNEEZES]
Bless you
Cool.
So I want to talk about randomness some more later on.
But first, let's spend about-- I think some of these games can be played in about an hour and a half, an hour?
So we'll play some of these games until about 2:30 and talk a little bit after them.
What we want to talk about as you're playing the game-- pay attention to how randomness is used in the game and how player choice is used in the game.
They're all going to be pretty similar to each other, but with a different mix and a different balance.
But similar types of mechanics are in each of these games, if I remember this correctly.
So [?
John, ?]
you want to describe the games that you learned?
So I checked out Race for the Galaxy.
How many people have played this game?
How many have played San Juan?
So I listened to the rules online a few times, but I didn't actually get a chance to play this game.
But it's a game where you're trying to get victory points.
And the main mechanic-- actually, there's really cool artwork, so I'd like to pass it around
[INAUDIBLE] think of the pieces in the game [INAUDIBLE]
[INAUDIBLE]
But I think the unique part about this game is that there are phases, and each player selects what phase their setup phase is.
And they each do different things.
One phase lets you gain cards, another phase lets you bet cards.
You can consume items, which gets you victory points.
You can score planets, et cetera.
And each person selects the phase that they want to play, and I think they get bonus for doing so.
Everyone simultaneously plays the phase that they want, and everyone can take part in that phase.
So seems like a really neat game.
Maybe [INAUDIBLE] more granular aspects, and you can play around.
Agricola-- I had the opportunity to play a few times, actually.
And it's really fun.
So check out the [INAUDIBLE] here.
There's a lot going on, actually.
There's a little-- what would you call this-- player board?
Yeah, a player board
Player board.
Everyone gets one of these.
And then there's a common area as well.
Multiple things, actually
Are you going to [INAUDIBLE]?
Yeah.
So there are these major improvements, which you can invest in, get victory points, and get power-ups eventually.
There are different rounds to the game, and it's a worker placement game.
So you start with a couple of workers, you're a family, and you're farming stuff, roughly.
And you want to expand your farm.
You can either build up your house, you could get livestock, or you can plow your fields.
And there's different rounds, a finite amount of rounds.
And resources respawn into this board every round, so it's actually quite cumbersome to re-up the resources every time.
And the whole point about this game that I thought was really unique-- oh, you also get a limited amount of profession cards, and also minor improvements.
And those are different every game.
There's a pretty big stack of these profession and minor improvement cards, but you only get seven.
So that's the hand that you have to deal with, and you don't ever get any more.
Yeah, so there's different rounds.
And you only have a couple of workers.
And the only way you can expand your family over time-- you have to pick and choose what your strategy is going to be.
You don't have time to do everything, because the rounds come really quick.
There's also a harvest phase, which is essentially-- so during the other rounds, you're amassing resources.
And a harvest phase, where you cull the resources out from your farm.
And you definitely don't have time to do everything.
But at the same time, the scoring, the points at the end, rewards you for that respawning.
Another thing that I noticed with that-- the cards that you get dealt, some of them have early game benefits.
For instance, if you play this game, wood is super awesome at the beginning of the game, so get the wood thing every time.
But as you go on in the game, wood [INAUDIBLE].
So it was a fun dynamic that I was able to experience
Did it come with beginner's rules, or first-time rules?
I think it's recommended for younger players
Yeah, I'd recommend playing those.
You'll get the same feel, and you'll probably get a full play--
[INAUDIBLE]
You'll still get a full play experience, and you can play multiple games a day
There's lots of bits.
If I had one criticism, there are so many bits, and it's really tedious to re-spawn all the resources
And the colors aren't all that different from each other, either
I saw some online where actually, the pigs are-- they just had little pink piggies, and that box would have been cool.
There's also a bunch of different decks, so I think there's tons of replayability.
And I actually kind of want to buy this game, because at first, there's tons of cards.
Really, the whole deck, it's basically all over.
And reading every line is kind of overwhelming.
But after playing it a few times, I really got the hang of it, and it was a fun experience
Every time I play this game, I also play [INAUDIBLE] with my friends.
And every time, someone pulls out some interesting combination combo that we haven't seen before, and it's like, oh, well
So let me ask how you do drafting, because online, I heard there's rituals around the drafting process.
Did you have house rules?
Usually, you do observation first and then [INAUDIBLE].
You deal everyone seven, they take one, pass the rest to the left
OK, and it's a public sort of process?
No, you only see the card that you have, until you can pass
Oh, so you get passed the same, you pick one, and then--
You start with seven.
Everyone has seven in hand.
Everyone picks one from their hand and passes the remaining six [INAUDIBLE].
And that person looks at that six.
[INAUDIBLE] know all seven cards or not
Maybe that was just something that came up through the community in the ritual of playing this game.
Or maybe it's written in the rules, but I--
Both are played professionally and for money
Oh,
And that's where a lot of the decks came out of.
You'll see people-- just like Magic the Gathering-- the new decks come out, you'll get new tournaments going on.
Really popular in Europe and Germany
So the drafting process-- is that something that spawned--?
It probably came from a pro play
Because we just distribute the cards
Which I think is one of the things you can do to sometimes balance skill.
Doesn't always work.
You're still going to have people who are really skilled at drafting and really skilled at reading the table, based on the cards and having a good memory of what cards came before them
OK, I can see that.
So it's a fun game.
Definitely check it out if you haven't
Cool.
Got another more simplified worker placement game called [INAUDIBLE] by-- I don't know the name.
And what this has is a lot of the randomness in this one coming from not knowing what's coming up in the tiles.
So you're kind of uncovering this forest area, discovering new things, placing your workers in these tiles to basically get victory points.
The victory points are traps around the board.
So you'll see, actually, this conceit used a lot in these kind of games, where the victory points are like a race, the [INAUDIBLE] at the right does this on a number of things [INAUDIBLE].
Very basic shapes for the pieces.
Simple color, natural wood colors.
And I think you can tell the difference-- I'm not sure if they do a colorblind test on this, but I assume that colorblind folks can see the different [INAUDIBLE].
Puerto Rico is kind of the board game version of, a simpler version of Race for the Galaxy.
A resource-generation game.
Again, played around rounds.
Each person has their own plantation that you're basically building up.
And the big thing with this series of games, and Race for the Galaxy does this as well, I believe-- so if you're the person who chooses what happens in a round or a phase-- I forget exactly what it's called-- then the person who chose it gets a privilege that they can do something a little bit extra.
So part of the play is choosing the role based on both whether it's going to be good for you, or whether it's going to hurt the other people around you.
Basically denying privilege and making something in the game happen before somebody else might have wanted it to happen.
Dominion, classic Magic-based card drafting game.
Lots of different types of cards of multiple number.
And again, you're playing for victory points.
If you have played Dominion before, I recommend playing one of the other games.
If you haven't played Dominion before, it's really easy and really fast.
So it's 30 minutes.
Your first playthrough will probably be about 45.
I have not played this one.
And I'm going to find out what it's about when I open the box.
It's in multiple languages.
Again, board with racing numbers around it, much smaller board this time.
And what are we to be doing?
You're archaeologists.
You're trying to acquire knowledge for an excavation exhibition.
You're planning excavations and exhibitions and getting victory points from digging in [INAUDIBLE] and finding valuable artifacts.
So yeah, unfortunately, I can't exactly tell you exactly what this is about.
But it's likely resource gathering and a little bit of worker placement.
So those are the 1, 2, 3, 4, 5, 6 games.
Grab a game, set up a group.
I'll call time at about 2:30 and see where we're at.
So don't put your game away yet.
If you guys could come back over to Race for the Galaxy-- what I'd like each group, each game being played, tell us a little bit about-- well first, tell us what the game was.
And tell us about-- this is assuming you were able to get this far-- how randomness and player choice entered into the game.
How was it used in the game, how did it play through?
Don't just focus on the statistics of probability, but also talk about performance, and feel, and the tactileness of the pieces in what you're doing.
So you guys go first
All right, we had Puerto Rico.
The randomness had to do with, I guess, right here in the tiles that the plantation tiles that flip up.
The player's choice comes in with the role that you can use for that round, and also, I guess, whatever strategy you choose, be it building up enough money to buy stuff, or trying to get enough colonists so that every single one of your tiles is occupied
So you had a lot of choices.
Basically, at the beginning of the round, you pick a role, and each role has a special privilege that you get.
And then everybody, including you, perform an action based on that role.
And so on your turn, you pick a role, keeping in mind there's a lot of public information.
So you can see what's good for you and also perhaps not as good for everyone else.
And then each one of these roles, the actions-- everybody gets a choice on how they perform it.
So for example, he was saying, these little tiles that flip up-- one of the roles lets you basically take one and put it on your board.
So there's choice there between how you compose your board.
And those lead into getting different resources and victory points and other such things
How did it feel?
It was really annoying to set up.
Tactileness-- nothing special, I guess.
It's a lot of tiles, little bits you use.
These are goods-- I think this one is tobacoo, maybe?
Corn.
Those kinds of things, you sell, you get gold or victory points for.
Here is a gold
You mentioned earlier about the affordances of the pieces to the play mat?
Yeah.
Is there [INAUDIBLE] play map?
Oh, here
Yeah.
So this is what the play board looks like.
So you can notice, basically, there are two main types of grid.
One's little rectangles, 2 x 1.
And one's a 1 x 1 square.
And so we have a couple of different-- so here is one thing [INAUDIBLE] clearly 2 x 1 and fit in that 2 by 1 square, and same with this one.
And there are also little 1 x 1 squares.
And so one of the interesting things is that with this board, you can tell there's clearly a limited number of things you can put on this grid, right?
I don't think there's a mechanic for removing something once you've placed it.
So it'll fill up eventually.
So you have to keep [INAUDIBLE].
While you do want to build things quickly and try to get more resources at the beginning of the game, you also don't want to just fill your board with crap and then be stuck at the end of the game.
So it's kind of similar to Dominion, in that sense
And while they do building [INAUDIBLE] really well, because [INAUDIBLE], maybe they could have done the other bits, where they're roughly the same size and shape.
And it was a little confusing [INAUDIBLE] square at the very beginning
How did the rules support those piece sizes?
You've already got some affordacnes going on with the pieces.
Do the rules piggyback off that?
Do they take advantage of it?
Not really
There were some diagrams that were pretty
Some of them, but they also didn't explain very well.
They didn't diagram the player board at all, so basically, there's nothing here.
This is just buildings and plants, and these are plantations.
I think you just kind of figured it out, like I guess we did
[INAUDIBLE] down, though
Yeah, it did
[INAUDIBLE]
We're only on our second round right now, like our second full, going around.
Because the setup-- first of all, just sorting all this stuff out, it's kind of like Dominion, where it's a pain in the butt to sort out all the tiles, and the cards, and whatever.
But also just trying to figure out what we're doing, and what's going on for the first round was really slow
I mean, I guess there's a little picture of buildings, and a little picture of palm trees, so--
It gives a lot of space on the map to describing these things that are already described on the card elsewhere
Kind of.
So these roles--
[INAUDIBLE]
That's true
This is a very skimmed down what's going on, basically.
[INTERPOSING VOICES]
--actually a little more information, and the rulebook is the full information
[INAUDIBLE]
Yeah, but I don't know how much I agree with putting the one-line summary on these little role cards.
Because you could look at this, it says "trader"-- what does trader do here?
[INAUDIBLE] information
Yeah, basically.
I can understand putting it on the board instead of the rulebook, because no one want to have to look through the rulebook every time.
But on here, it's an immediate I can see the roles, OK, I can look at the board
Do you think redundancies are bad in general?
No, redundancies are great.
Redundancies awesome.
We use them as best we can.
But you have only so much spaces, in the pieces and boards that you make you've got to really decide what's the most important thing for the person to know.
And the most important thing-- probably need to tell them more than once.
Through rules, through the play mats and pieces, through the just natural, this fits here and this doesn't fit there kind of thing.
But yeah, definitely.
And then text-- some people hate it, some people like it.
I'm in-between, use it as it's needed.
Great.
Race for the Galaxy is kind of similar to this, isn't it?
Basically
There you go.
So tell us about Race for the Galaxy.
Randomness, player choice, and then affordances for the pieces
[INAUDIBLE]
Yeah
Yeah.
[LAUGHING]
[INAUDIBLE]
Huh?
Yeah
[INAUDIBLE] game [INAUDIBLE] Yeah, [INAUDIBLE] are really just based on what cards you got dealt
Can you show us how-- somebody do an example layout while another person's talking?
You had a starting world, which was like a starting hand, which contributes a lot of your initial luck.
And then the card that you drew earlier particularly for-- you could say [INAUDIBLE], if you don't get any useful cards, then you're sort of stuck, and you just have to [INAUDIBLE].
Whereas if you get good cards, then you can sort of-- they'll allow you to [INAUDIBLE] So a wrench in the game is that the game is designed to-- it's like there is sort of a way to have an engine by every turn by producing points every turn.
But by the time you actually get the engine up and running, the game ends in one or two turns, anyway.
I don't know
Does that feel like what you guys were doing in this one?
Struggling to build something that would produce stuff in the end?
Yeah, yeah
And on the other games, too?
It's producing immediately, actually.
After the first full round, I think every one of us got something to sell to get stuff back, basically
It's interesting to me that those are the same games, sort of.
But this has so many bits, and bobs, and gadgets [INAUDIBLE].
And [INAUDIBLE]
From a deck of cards.
What is this doing that that one is doing, too?
The role selection.
You choose what you're going to do, but everyone gets to follow suit with that.
Then you get a bonus [INAUDIBLE]
The little bit there-- they exist here, but they're in the form of cards face down [INAUDIBLE].
And so it's not apparent that you have [INAUDIBLE]
Did you have to sort out those goods beforehand?
Or is it just--
Nope.
You literally-- it's like if you suddenly get a mining good on a mining planet, you take the top card in the deck, put it face down under there
So you're taking cards out of play when you're doing that.
By generating a resource, you're reducing the amount of choices.
Granted, there's a ton of choices up there.
The deck is really large.
So it's not making a huge effort.
Is there a card that you can kind of bury cards underneath and just completely remove from play for this one?
I mean, [INAUDIBLE] could be put [INAUDIBLE] until you discard it and then [INAUDIBLE]
So in San Juan, there's a chapel where every time you put a card underneath it, it creates a victory point, and that card can never be used for the rest of that play session.
So a good strategy in that game is to take things like another high points-giving card that maybe you can't to build, and somebody else could, and burying it-- removing it from play
Yeah, this game, you just hold it in your hand, never play it
Yeah.
Can you talk about the player mats?
The kind of player aids it gives you?
A very expansive cheat sheet
Yeah, the game uses a lot of iconography on the card.
One team used a lot of the iconography there
To somebody who hasn't played before-- was the iconography helpful or hurtful?
It was certainly very helpful.
I think just thinking along the lines of [INAUDIBLE], there are very few in this game.
So even though the iconography has a hand next to a card with a "2" on it, I still had to work to make sure what that means
On the other hand, one thing that they did was hexagons are victory points.
They have the hexagon victory points that are actually little tiles that you can see.
But then on the cards, you can see that on every card, there's the victory point value, and that's in a hexagon.
And whenever there's a card that says gain victory points, you see it has a hexagon
Yeah.
It could have very well just be shitty fireworks just flowing out, like this is the card to use.
It's calling it out, really clearly out.
That's great
Or the shapes-- I don't know, the icons make use of the card shape.
They have a little rectangle--
There's also diamonds, and rectangles, and circles, yeah?
Yeah.
Diamonds and circles are for the two types of cards
Are they in different places than [INAUDIBLE]?
Yeah, each card has a hexagon thing that [INAUDIBLE] as well as the [INAUDIBLE]
So yeah, thinking about placement of where you're putting your numbers, and where you're putting things on the card, where your eye goes when you look at the card.
I mean, those are really complex-looking cards there
They even have a little-- in the upper right, they have sort of reminder symbols on the cards there.
Because there's [INAUDIBLE] just to remind you, oh, this card is-- you might forget about this power, you should remember it
Yeah.
That game-- I think all of these are made by the same company.
Yeah.
Rio Grande?
Oh, no, [INAUDIBLE].
So you're going to see different kinds of-- what do you call it-- style guides that they're using.
The rules for these three games are very similar.
I think Race has a little bit more advanced layout, but there's the basic kind of layout, where the rules on one side, some columns and sidebars on the other side, the occasional diagram.
But that designer believed in, everything goes on the cards in some form.
Everything's there.
You can play the game without the rules, unless you have a really good memory
I mean, the thing is, you need to use expansions.
And you can introduce expansions, and you don't need to read the rules in the expansions.
And they'll introduce a whole bunch of iconography, and you'll look at it, I'm like, oh, I bet this is a [INAUDIBLE].
And I'll bet that this is what card means, and it's usually right
Yep.
Cool, so we're still doing good on time.
[INAUDIBLE] Randomness, player choice, and then affordances
Well, the randomness came when we first dealt out the application cards and the minor flipping cards.
There's the [INAUDIBLE].
[INAUDIBLE] we deal out seven to eac player, for each.
So that's sort of where the randomness comes in
And it gets amplified, because these occupations and improvements seem like, you could just go creating things with them.
He got something that let him [INAUDIBLE] throughout the board that he would continue to play.
And then he had another thing that he could use that allowed him to double that amount.
So it really feels set up for-- yeah, it seems like you can just make these crazy combinations that basically give you farming superpowers
I forgot about that part [INAUDIBLE]
What was it like choosing?
How did you decide what to choose, when to choose?
Did you get far enough?
Yeah.
In the beginning, I had no idea what I was doing and why I won.
But as we got through, I started to get a sense of it.
And now, I'm finding it easier to think about, OK, I'm going to need wood to build a room.
I'm going to need food, because the harvest is coming up.
But at the same time, I feel like I don't have a good enough sense of what's the deck.
Because he was just pulling out things that I had no idea it could even happen.
And then like, oh, wow.
OK, that changes a lot
So what's the choice for him is random situation for you.
You just have no clue that might even pop up, because you don't have that information, right?
Yeah.
So I think it would take a while to get a good sense and fully understand what's happening.
But I could already plan what I'm trying to do on my end
Can you talk a little bit about the affordances of it?
Tactile feel, how did it feel when you were playing the game?
I like the board setup [INAUDIBLE] know that either [INAUDIBLE].
And the [INAUDIBLE], the [INAUDIBLE] shapes, [INAUDIBLE].
In terms of the resources, it felt weird, because they were all either little circles or cubes.
And you just have to remember what the colors mean.
But then again, they're also [INAUDIBLE].
The little pictures on the common area that tells you what they are whenever you [INAUDIBLE]
Yeah, and there's this interesting thing where every round, if a resource hasn't been taken, then it increases.
And now let's say there's three woods sitting here.
The next round, there'll be six, the round after, nine.
And so on, and it keeps building.
And so to make it easy to remember that, what they have you do is they have you put it on these little squares to start out with, so you can know what you just added.
And then there's an arrow pointing into this little bucket, so you know, OK, place it here to set up the round, put it into the bucket so that everything's there.
And I feel like it'd be pretty easy to lose track, like, oh, have we added wood here yet?
We kept forgetting to add sheep [INAUDIBLE]
So you're farming.
There's animals, and there's other things-- vegetation, stone, raw materials.
It breaks them up, right?
The squares are things that are animated-- they're living?
Mm-hm
Circles are things that don't animate.
Is there some kind of affordance going on there?
Does that seem useful to you, why those are being distinguished that way?
Yeah, because animals and resources behave very differently.
Because these resources, you're usually expending to either get improvement, or build houses, things like that.
Whereas animals, you have to manage a little bit more in terms of OK, I could kill this animal and eat it, or it'll give me points later.
Or I can breed them, and I'll have to keep track of spacing them here.
So there's a nice separation of what you're really doing with them
Cool
[INAUDIBLE]
No, he's not in today.
He'll be in tomorrow
OK, thank you
Yep.
Great.
Anything else you have to say about this one?
It's really fun.
I'll probably go buy it, because I want to play it again
Why is it fun?
Probably the choices.
There's just so much going on, and it feels like I could really make my own strategy.
And I don't know-- I just had all these crazy plans of, OK, if he [INAUDIBLE] a room, that means that I can, which means I can build my family next turn, which means that I'll have more actions.
And now I can do even more stuff, and get more resources
So there's this complicated system going on-- if I can try to remember it, understand it-- a complex system going on that you're going in the right direction, do you feel like you can kind of switch?
Yeah
For people who played it multiple times, did you feel the same way?
Yeah, because if you and your opponent aren't choosing a choice, it gets juicier and juicier every round.
And so it's this game of chicken-- like, are you going to go for the wood?
Yeah, there was one round where I kept nine or 10 wood
I don't know.
That was fun for me.
I like that
Just thinking back on what he said, I actually like this game a lot, too, having a lot to do with all the different choices.
And there's a really complicated system that it took a while to kind of understand.
And I don't understand it completely yet, but I'm getting a much better feel for it, you know.
And I think that's really appealing, not having something so absurdly complex that nobody can actually do it.
But not having something really simple, either.
Just the sheer number of choices here makes it really interesting to me.
Because I guess one of the things about having not that many choices is it almost feels like I could just make a computer play it, because there are only n possible outcomes of the eventual game.
But something like this, that's just unbeatable, really, unless you actually make it a really smart player
Great.
[INAUDIBLE] in the back.
Randomness, player choice, affordances based on what you got to work with
So there's definitely randomness in it, because you draw your initial hand and every subsequent hand.
And a lot of times, you can put together some ridiculous combination, [INAUDIBLE] would just get market, market, market, market, smithy
What about [INAUDIBLE]?
What?
What about [INAUDIBLE]?
Why'd you do that?
Well, so what he said-- right away, you just draw.
And so what you can do is buy these action cards.
And with these action cards, you can get free moves.
And the market-- I think it's the best one, because you get an action, a buy, a coin, and something else, whatever.
And so what I would do is I'd play the market one, then he'd draw.
So I'd play the market one, I'd draw.
And then I'd draw market, I'd play market, and then I'd draw, and then draw market.
And then I'd play market.
[LAUGHING] And then when I'd draw, I would draw a smithy.
And then you could draw two more cards, and I would just draw two more cards.
And I'd be sitting here with 15 coins, or 20 coins, and I can do literally whatever I want.
I can buy it however many times I want with however much money I want, [INAUDIBLE]
So yeah, there are definitely some really powerful combinations that plays with the luck factor.
But at the same time, it feels there's a lot of skill in the game.
It does also feel like if I played it 10, 15 times, I would have a very good sense of what to do, and the skill would start to go away.
Like if I played it 10 or 15 times, and everyone else did, I feel like we would be always much better at it.
And there would be a developed strategy, or maybe two strategies, and counter-strategies, something along those lines.
So they were saying it feels like infinite possibilities, only it didn't feel like infinite possibility
So there are affordances in the game that can avoid that.
What do you think they are?
[INAUDIBLE] There are a million cards
Yeah, so that's the basic set.
How many cards were in the basic set?
500, [INAUDIBLE].
Although I don't know how many [INAUDIBLE] that actually change [INAUDIBLE] 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17-- lots
Lots, yeah.
[LAUGHING]
OK, so that should change it up, I guess
Possibly.
I mean, still, your inexperienced players will say that exact same.
And maybe those expansions-- you can bring an expansion in and change some things up.
I don't know.
Anybody else who's played Dominion a lot-- have there been strategies that you've seen come and go, like there is in Magic?
I mean, it entirely depends what's out on the board
Yeah, exactly
Is there [INAUDIBLE] engine and still have the money to buy things?
Or is there just really interesting action, or the other cards [INAUDIBLE] tiny deck.
Or are there no specific [INAUDIBLE]?
And that changes each time you play depending on how you set it up
So there's a related game called Ascension, wish I pretty much prefer more, but probably just because I've played it more, and that was my first one.
It's really interesting, because instead of at the beginning, of Dominion, you randomly select a couple different tiles, basically.
In Ascension, you actually just have a really, really big deck, and you shuffle that.
And you deal out just hold 'em style, just stick cards across the board.
And then you take turns just being able to grab something from the middle and then replace that [INAUDIBLE].
So I feel like it makes it a lot more interesting, because it's much less predictable.
At any point in the game, something amazing or something crappy could flip.
Rather than you know that you will always be able to buy, until the pile runs out of a different kind of card or something like that.
So that's something interesting
Anything else you guys want to say about the affordances?
Like the rules or how the cards are laid out?
Anything that you got out of that?
I guess it's kind of a bigger game.
So the rules were incredibly complicated at parts.
But that being said, we did figure out, I had to ask him a couple of questions.
But after that got settled, I think we all understood it pretty well
That one's a great example of look at all these rules, oh wait, the game's actually [INAUDIBLE] simple to play
Yeah, if we had someone who knew how to play sitting here, we could have probably started 10 minutes earlier
Could they have come up with a different way of writing those things, or laying them out, or presenting them in a way that might have helped you out?
I don't know.
I bet.
Wait, [INAUDIBLE] question there?
Oh, one of the questions for you guys, why do you think that was included in the list of these other games?
What about that game is similar to some of these other games that you heard?
[INAUDIBLE]
No, but that's an interesting-- I just noticed that coming out.
That's why I wanted to talk about rules.
That's all
Definitely the "luck versus skill" dynamic, I think, is pretty key in terms of, like you said, there's strings of luck.
But I chose [INAUDIBLE] markets, right?
So I was putting [INAUDIBLE] together, and that's definitely skill.
So I think the mix is exactly [INAUDIBLE]
There's a depletion mechanic in that one, right?
Did you read how you end the game?
Yeah.
So we did run out of market-- we never-- we didn't have time to finish.
We did run out of market
Yeah.
And I think it's once you run out of a couple of different kinds of cards, then it ends, right?
Re-tile before the [INAUDIBLE] card.
[INAUDIBLE]
So you have to always have some kind of--
[INAUDIBLE]
No, [INAUDIBLE].
That's not true.
That's [INAUDIBLE]
Oh, really?
Yeah
So all these games have some kind of Solitaire-ness to them, where you can be playing on your own.
But even if there's like a Solitaire-like work replacement style game, there's something that you're doing is interacting with the other player and preventing them from moving ahead.
OK, so it is almost 3 o'clock.
You can either continue playing your game or break into your teams and work on your projects.
It's your choice.
I'm going to go grab all the kits.
We've got one person-- are you one a team?
I'm not on a team
Not on a team.
So we've got one person joined a team.
We've got a three-person team?
[INAUDIBLE]
For a two-player game or a four-player game?
Two to four
Two to four?
[INAUDIBLE]
OK, so I would recommend hang out with them for the last hour.
And then maybe even scrounging around to find a team to go onto
[INAUDIBLE]
What's that?
The other teams are four-two people?
You can have a five-person team
[INAUDIBLE]
[INAUDIBLE] check out what [INAUDIBLE] Yeah.
Yeah, not requiring you to be [INAUDIBLE].
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at oc.mit.edu
We're going to do a little talking about the reading, and then we'll play a couple of games.
And then the last hour will be time for your teams to able to work on your projects unless, of course, you are playing Monopoly, and your property [INAUDIBLE].
So today is kind of like the historical part of class.
I mean, we've looked a little bit at some board games, some card games that [INAUDIBLE].
But now we're looking at a very specific time, largely 1900 to about 1950, prior to World War II.
There were a number of games that I could have brought out, things that you would also be familiar with, things like Boggle, for instance.
But those come a little bit later.
Some of these games, the Scrabble, for instance, have predecessors that come from the time period that we're talking about, but these versions that we're familiar with now, probably had some rule changes along the way.
Similarly, how many of you played Othello or Reversi?
Yep.
So Reversi dates from the time that I'm talking about.
It might actually be slightly before 1900.
But Othello comes much, much later, like the '70s.
And how many of you knew the game of Othello?
Looks like [?
we're quite ?]
[?
reversing.
?]
So that's kind of like a trademarked version of the rule set, which basically specifies what the initial start position of the game that I believe it was adapted with a game designer who wrote up the rules, set up [?
deployment ?]
rules and publicized the game of Othello.
But the version of the Reversi may be not black and white, may be different colors on each open face is way before that.
And Uno comes much later, but in doing this reading, you read a little bit about a game-- I think it was in today's reading-- about a game that Parker Brothers tried to publish that was kind of like Uno, only it had five colors instead of four.
So that's the reason why I brought in to this today.
It's not necessarily because all these games, they come from 1900, but a lot of them came from, started around about this time.
And [INAUDIBLE] local history, Salem is really not that far away, and I'm not sure that the building still exists, but has anyone been to Salem recently?
Yeah
Yes?
Then, do they have the Parker Brothers building there because it's huge?
Or it was
I don't know if it's still-- I'm not actually sure
It's probably a tourist attraction
No, definitely not a tourist attraction
OK.
This is all [INAUDIBLE]
I think it might be repurposed to [INAUDIBLE] apartments because they've repurposed a lot of--
A lot of Parker Brothers properties are now owned by Hasbro, which is also not that far away, over in Rhode Island.
In fact, if you look at all of these games, they say Milton Bradley, they say Parker Brothers.
But if you look at who currently owns the property, it says Hasbro.
That the [INAUDIBLE] And all sorts of games, like Tiddlywinks-- how many of you played those growing up?
I never played them.
Some of the [INAUDIBLE] showed me how to play them.
This is not part of [?
the talk.
?]
[?
Ted ?]
briefly talked about the [INAUDIBLE] And anyone remember why Parker Brothers went into card games?
Because they were only were making board games.
They were making things like ping pong before that
They were smaller and easy to manufacture
Uh-huh
Larger profit margin
You print a sheet of cards, you cut it into identical slices, pack it in a tiny, little box.
You can put a lot of these on store shelves or a truck or [INAUDIBLE] as opposed to big boxes of Monopoly.
So I think that the takeaway from everything that we're going to discuss today is really all about how marketing and sales concerns are going to effect you guys.
If you read the stories of Phil Orbanes, [?
one of ?]
the [?
most wanted ?]
[?
ones, ?]
vice president of R&D at Parker Brothers, also, apparently, the best Monopoly player, [INAUDIBLE] material.
He's also writing from a very biased point of view.
He's writing supposedly as a Parker Brothers [INAUDIBLE].
And you can take everything that he says with a grain of salt.
I'm sure not everybody who works at-- I'm sure [INAUDIBLE] as the book makes it out to be.
But when [INAUDIBLE] I think we can rely on them and because I think [?
it's ?]
a pretty good designer, a sort of design diary of how various games became the way that they were.
One thing that he didn't go into a lot of detail on was Monopoly because that kind of comes after his [INAUDIBLE].
But there's a little bit of that that I wanted to talk about.
And we mentioned that in [INAUDIBLE] it used to be called [?
Landlord's Game.
?]
This was a patented product.
And the name of the original designer was Elizabeth Magie, somewhere right in the beginning of the 1900s, although it took about three years between the patents and type of publishing to actually figure out all the manufacture [INAUDIBLE].
Parker Brothers only came into ownership of this product around about 1974.
So it took about 30 years.
And the original feedback from George Parker himself was that "the game is rejected because it's too complicated and too technical and takes too long to play."
So they didn't actually buy Monopoly from the [INAUDIBLE] They bought it the following year, after Christmas, when they saw how well [INAUDIBLE].
And in fact, George Parker invented the quick rules and, I believe, actually imposed a time limit rule, which does not exists any more.
But everything that we've talked about, the Parkers and Monopoly, was something that George Parker and the Parker Brothers were very, very much aware of.
They knew that there was a problem with the game.
But they also realized how it tapped into the zeitgeist, that there was an opportunity there that could be capitalized.
It was one that fit very well with their strengths as a board game manufacturer and publisher.
You saw how quickly they could turn around the company to make it [INAUDIBLE] when the demand was there for it and what they had to do to hire people who basically operated coin machines to become jigsaw operators.
[INAUDIBLE] So that was a story of why Monopoly made its way into the Parker Brothers collection.
It was just [INAUDIBLE].
And it certainly wasn't the design of the game that attracted them to acquiring the thing.
They just wanted to dominate the American board game industry and in some cases, the British and Western Europe game industry, as well.
Clue, on the other hand, it's one of those things that has a very British origin, [INAUDIBLE] before it came over to the US.
But I think in writing, we had a couple of examples of how the Parker Brothers [INAUDIBLE] initially and then eventually hired people based in London to try to acquire work that had been already patented in London and why the American publishing [INAUDIBLE].
And that kind of gave you an idea of what the board game industry looks like around about the turn of the century.
Why is it that it's a little bit hard to think of many examples of prototype games?
There's a lot [INAUDIBLE] in chess boards and playing cards.
All these things are good.
You might have a design on the back of a playing card or such a thing [INAUDIBLE] the king or queen is illustrated that could be copyrighted.
But the games that you can actually play on then were pretty much [INAUDIBLE] games, both games are games that you can just really share with [INAUDIBLE].
Demand [INAUDIBLE] but around 1900s is where mass production and mass distribution all come into the fold.
It's not the driving that Parker Brothers was getting in Salem because of the seaport.
Even though it wasn't the best seaport because, by then, the war was moving the steam ship, and you needed a deeper harbor, like what Boston did, it still had the means of moving large amounts of merchandise.
At least two [?
barges ?]
were [INAUDIBLE] of the United States.
So prices-- I just want to give an idea of how recent this whole idea of buying a game off the shelf is.
And I'm only talking about the 1900s.
And prices, actually-- you might buy a toy.
You might buy something that you played a game with golf clubs or something like that, a cricket bat.
But you wouldn't necessarily just buy a whole game with a branded title on it, just off the shelf.
The jigsaws are kind of already a pretty extreme case where this is its only game, and it's not interchangeable with another game with the same title because [INAUDIBLE].
And until the 1900s, we didn't necessarily have a situation where people assumed that games could be copyrighted or trademarked.
And games were just things that people played, and if you liked it, you got the products that needed to play it with, but then you wouldn't necessarily own it.
Anyone else could go ahead and get the same number of products or similar products and, basically, the same game.
One example of this is something that's not in the reading, but I guess this will be over in [INAUDIBLE].
So how many of you read H.G.
Wells?
The War of the Worlds?
[INAUDIBLE] Do you know that he made a board game?
He made a board game with little, miniaturized tin soldiers.
It's called Little Wars.
You can find it on Project Gutenberg.
It's about 24 pages.
And the rules are only about six pages long, so it's a quick read.
But the previous 18 pages is his development diary of how the rules are made.
And basically, [INAUDIBLE] about how he was actually specifically ended up developing a set of war game rules on how you set up troops, how far do they move, how you launch cannons, shot at each other.
And it was all based on this one little toy that you can buy from any toy store.
So it wasn't like, here's a box that you will buy with H.G.
Wells' Little Wars on the title.
He just printed it as a [INAUDIBLE].
And if you got that you would read a little story by H.G.
Wells on how to [INAUDIBLE] and then right in the back of it [INAUDIBLE] that you can buy these parts.
You can play this game for yourself.
So again, he kind of owns the book, but he doesn't really own the game.
You are expected to find the parts yourself, in particular, those little breech loader cannons.
I think actually there were little explosives in it, and then you could light the little cap.
And you could use that to fire little wooden projectiles [INAUDIBLE].
It was pretty darn cool.
So you can do things like terrain.
And there were instructions in there on how you make model terrain.
If you made a building, it had to be completely solid, you could fill it up with toy blocks because you don't want anyone to put their troops inside the building, but you can put them on top of the building.
Pretty cool.
But that's some time in 1914, 1915.
That's already past the date when these products were starting to appear on the shelf.
If it gives you any indication of what it was like before that-- is that people printed out rulebooks.
And you can still find books for card games, for instance, on your shelf, like 101 Games You Can Play With a Deck of Cards, or something like that.
But that's how games were shared.
They weren't put in a box and shrink wrapped, tied together with twine or something like that and sold as a product.
So today we've got these games.
Just a quick look through some more details, by the way-- I want you take a look at these boxes that I'm going to be handing around.
And look at the sizes of them.
I want you to think about how you would put this on a store shelf like Target or Walmart.
I guess at that time it would be a [INAUDIBLE] store.
And these are modern boxes, obviously.
These don't date back to the [?
past ?]
century.
Because I kind of want to talk about, also, the reality of what it's like to be able to sell a product like this in stores today.
Actually, I should have brought in a [INAUDIBLE], but the jigsaw box is about the size of the size of box of the time.
The first thing is orientation.
This is meant to be sort of seen this way or this way, possibly back this way.
How many of you have gone shopping in a Target or in a Walmart or a Sears or something for gifts in Christmas?
OK. Yeah.
You're buying it for family members, [?
among them?
?]
Friends?
For yourself?
Anyone buying games for yourself at Christmas?
No, actually, this is recorded.
I don't think it's any great surprise if I told you that the majority or a very huge chunk of the profit that a company like Parker Brothers or Hasbro will make will be during the Christmas season.
But they are not usually bought by the people who are going to play them.
They're usually going to be bought as gifts.
So all of this stuff is positioned not to attract you to think that, oh, this is something I want to play, but rather, this is something that I want someone else to play.
Or I think someone else is going to like this.
How many of you bought Monopoly for a friend or for a family member?
Yeah.
Did you think it was a good game at the time when you bought it?
Eh
Eh.
So why did you buy it?
Because other people liked it--
Because other people--
It was a gift
--because someone you were buying it for might--
They [INAUDIBLE]
Oh, they were your--
Yeah.
And [INAUDIBLE]
And maybe you remembered liking it when you were their age
Yeah
Anyone else?
We got it for my grandpa
You got one for your grandfather?
And had you played with your grandfather before, Monopoly?
No
No?
A [?
lot?
?]
OK.
So maybe this is a opportunity to play this, OK.
But had your grandfather played it before by the time--
I'm sure.
He's old
Yeah.
The game's old
That's right
Who else?
I thought I saw some other hands
Yep.
I think we played with my cousins at Christmas.
I think it's the only thing any of us play
Is that like the thing you can do with your cousins?
Well, they're younger.
We have a kind of wide age range.
We have second or third grade up to high school at the time.
Yeah
This is a hardcore, ruthless, capitalist game for second graders, right?
But you can play it.
But people have fond memories, not necessarily of the game itself, but of sessions playing Monopoly with family and friends.
And even whether or not the game's any good, you kind of hope that other people will have those fond memories of playing it with friends and family, maybe even because it's you giving the game.
So you might end up actually playing the box that you bought.
But that's what all these games are designed to do.
These game sets are packaged to be gifts.
And so you look at something like Battleship and Risk and Monopoly and Clue, they're huge things that look good if you wrap it up with paper.
And Tiddlywinks?
Tiddlywinks, it's hard to make an argument for a much bigger box because we really don't need a lot space for a Tiddlywink box.
Go ahead and open it, actually, so we can see what's inside.
I bet it won't be there.
[LAUGHTER]
In fact--
[INAUDIBLE]
Yeah.
And probably the box-- let's see how much of that box is actually occupied by stuff.
There is a bell.
And that requires a certain depth of box
And [INAUDIBLE]
[INAUDIBLE].
[LAUGHTER] So that's a little deck of cards and a bell.
And the bell, I'm not really sure the bell was clearly in the original version of it that was started in [?
the beginning.
?]
The deck of cards certainly was
[INAUDIBLE]
Good.
Is there a copy deck?
Yes
OK.
There actually is a copy deck.
Good.
All right.
It's not just a cardboard pole.
So again, these products are all designed to catch your eye-- it's really up on the store shelf [?
like that.
?]
I believe the standard shelf is 18 inches.
That's [INAUDIBLE].
That's a standard industry width.
If anyone's worked in retail and knows that number by these different [INAUDIBLE].
And if you sell boxes, basically, this wide, you can sort of stack them either on top of each other.
And then put one vertically in front to show it.
This is Risk, and if you take that box off, then you can see a whole bunch of other Risk boxes that are back there.
You can take that.
I believe that there is, with the Candyland and the Chutes and Ladders-- that's kind of an interesting situation where they're a little bit off, they're a little shorter, both horizontally and vertically so that you can actually fit three in a row.
So the Candyland, the Chutes and Ladders-- there's probably some third very simple game-- but I can't think of one-- that all fit in a row.
And then you just see this giant wall of Hasbro.
That's what they want you to do when you come in.
Where is it the boxes are placed in a store, whether it's Target or Walmart or some specialty game store-- it's all paid for and sold ahead of [?
price.
?]
How much Hasbro pays the retailer for placement determines whether it's the first thing you'll see when you come into the store or whether they're stocks that they are keeping in the back room that you have to ask for.
That's [INAUDIBLE].
So maybe they'll look at one display case somewhere where they're [INAUDIBLE], but for the most part, you're not going to be able to pick a game off the shelf unless the board game manufacturer themselves have actually paid for that shelf.
And they're going to make it back, for both of them, they're going to make it back by the end of the season because that game [?
is repeatedly going to be ?]
sold.
And it's OK that they're selling the boxes to people for [?
any ?]
[?
or Monopoly.
?]
[INAUDIBLE] to somebody else.
[?
You all might ?]
already have a copy of Monopoly, but they don't want people to take that in consideration when they [?
want to buy this.
?]
How many people have multiple copies of Monopoly?
I have two.
OK. How many?
Four, maybe
Four?
Different kinds of Monopoly?
Yeah, they all have the different skin, like there's [INAUDIBLE], Star Wars, Millennial, on and on
There's just different kinds
Yeah.
I have Star Wars, Nintendo--
Nintendo?
--Pokemon, and then the original
Yeah
I can imagine Pokemon.
I'm interested in Nintendo
So we have Junior.
I learned to play Monopoly when I was, like, three years old.
But that [?
has big board.
?]
What's the difference?
Is it--
Everything is just simpler.
And I also do like that the dollar amounts are one, two, three, four, five instead of all these hundreds and stuff.
The ones don't really matter
Right.
You don't have to have to [INAUDIBLE] under a hundred
Yeah, [INAUDIBLE], but it's basically the same game with no Community Chest
Is it [INAUDIBLE]?
Yeah
I mean, [INAUDIBLE] at least?
Yeah
Yeah.
OK. And you have the Junior, too?
Yeah
And then, I have Pokemon, and I also have those electronic ones
Oh, are you talking about electronic devices that you carry or something you load into a computer?
Both
Oh
I have a disk and I have a Monopoly thing that's it's kind of Game Boy, but it's Monopoly
Oh, OK.
I can see LEDs that light up
Those ones with the disk, you're playing against the computer or against a person, and the little Game Boy thing, you're playing against the computer
Cool.
Yeah?
I have the new one
Different versions?
Yeah.
Regular, I think [INAUDIBLE], I have a rip off of Monopoly that's my town.
And so different business in town-- back in 1985, now they don't exist anymore-- paid for their business to be bought on the board
Oh, so it's [INAUDIBLE] game, basically
Yeah.
And I have Junior Monopoly, and I also have Junior Monopoly, dinosaur theme.
It's by far the best
[INAUDIBLE]
Yeah.
I do think the board was a little bit smaller.
I don't think there were quite as many spaces
OK.
So was it the best because of dinosaurs or because it's actually best to play?
Well, Junior Monopoly is best to play, [INAUDIBLE]
Oh, it's the dinosaurs.
Right.
I think the [?
version ?]
is very similar to the local town version.
And it's properties of the [INAUDIBLE].
Does anybody know where the original road names cam from?
Atlantic City
Atlantic City, New Jersey.
And that's the version that we've got.
The version that we've got now is being published by Winning Moves Games, which is kind of like a boutique Hasbro subsidiary that sells classic versions of all of these games.
And so they sell it at fairly higher price points.
I don't know if the price tags are still on these games but you can take a look.
But they're more expensive than the versions that you buy in Target.
They have little pieces.
Therefore, people who like physically buying these games want [INAUDIBLE] rather than the cheapest [INAUDIBLE].
I'm trying to remember other things that Rob mentioned.
Every little thing that you'll find inside any one of these boxes costs money.
That's just [?
the only ?]
thing about that.
Scrabble, you all the have the tiles.
You have the board itself.
If you're lucky, there is a nice little piece of cloth tape holding the board together, which [INAUDIBLE] opening and closing and can fold up flat.
If you're unlucky, just [?
one that ?]
doesn't keep [INAUDIBLE] from folding in half and falling out later on.
That crease, it's just not going to last.
Risk, of course, has had many different versions where some of them with soldiers and horses, some of them with just numbers.
I had the version at home that was big [INAUDIBLE] class of numbers.
There was a designer who came to our lab who spoke about Risk.
He used to work for Hasbro.
And he had a version where, instead of soldiers and horses, it was arrows.
And so you move all of these arrows.
And it was like a war map, and all [?
these forces ?]
[?
were marked.
?]
Unfortunately, the arrows were flat.
And you couldn't even pick them up.
And the side things were kind of sharp and pointy.
And people kind of could get hurt.
And that version [INAUDIBLE].
There's a good story about why that happened.
So today, when you actually play these games, I'd like you to actually take a look at all of these pieces and from [?
there, ?]
you might want to try to [?
determine ?]
how each piece actually costs and just add up what the manufacturing costs of this box might have been.
I don't know if anybody here is from chemical engineering or any kind of manufacturing.
Actually, Todd, you know [INAUDIBLE]
Yeah.
Not injection molding, but--
So it might just be a fun exercise versus, like, a deck of cards
And I'd also recommend reading the rules and actually try to play it the way the rules say, not maybe the home rules you might have remembered when you played it as a kid just to see how they are presenting this game.
How are they presenting the rules to somebody who just got this as a gift, and they're going to try to figure out how to play this thing
Yeah.
See if they've improved the rules that [INAUDIBLE].
I actually hear about a classic version of Monopoly where the original set of rules are [INAUDIBLE] updated print out.
You can see how they described the Scrabble game.
Does this one have a bag?
Does it have a bag?
Oh, wooden pieces.
And a [INAUDIBLE] bag and think about how that makes things easier.
[INAUDIBLE] And if you've played these game before-- and I think many of you have-- definitely go into it by reading the rules first, so not just playing the same way that you remember.
And also try out some of the other games.
OK?
Cool.
Let's talk a little bit about the games.
So I think everyone got a chance to.
play.
It's all right that the jigsaw puzzle game is still going on.
So let's talk about the pieces.
Let's talk about things that you're moving around.
I mean, how many of you feel that the pieces that you are playing with are significantly different from pieces that you remember playing with?
No?
Yeah?
For Candyland, the board is [?
totally better.
?]
Oh, yeah.
So the board design is kind of insane
It's really scary
And I remember it being a little candy thing like a square.
[LAUGHTER]
The board is potentially like this.
It's like--
It's like [INAUDIBLE].
It's like a [INAUDIBLE]
[INAUDIBLE]
But the actual cards weren't all that different, right?
No.
I remember the game being each space [?
like a candy.
?]
Right
Now, each space is a color
Oh,
I thought I remembered that
Me, too
Yeah
Wait, colors or candy?
OK, colors?
Hands up.
All right.
Different pieces of candy?
Well, he--
Well, I think it was--
It wasn't every piece, every space.
But there were a number of every other space had some kind of image on it
I think it still has that.
I think it still has the occasional
[INAUDIBLE]
The thing is that if every piece was a different kind of candy, it would probably be more colorblind-friendly because you'd be able to actually just see the kind of candy even if you couldn't tell the colors
It's not very colorblind-friendly.
[LAUGHTER]
This is not friendly with anyone with eyes, I guess.
[LAUGHTER] [INTERPOSING VOICES]
But that's just the visual design of the board.
The actual spacing on the board is not all that different.
So you've had the chance to pick up the pieces, slam on the bell and pit and stuff like that.
Some of these pieces, like in Clue, have a lot of money put into some parts that aren't actually all that useful, like the knife and the lead pipe and things like that.
Was that solid metal?
Yeah
Were those pieces?
They are sort of like metal pieces
It's a sharp knife
It's a sharp knife?
Well, it was when I was 10
So because, in assignment two, you are thinking about aesthetics, what do you think about the choices of the materials that they used for the various parts of the games that you played today?
Or that you [?
four ?]
played today?
Well, one thing that bothered me with Clue was if you committed a murder, or if somebody committed a murder, and you find a body, you're going to know whether they were murdered with a knife or a pipe.
They don't look the same-- or a gun.
[INTERPOSING VOICES]
A very astute murderer.
I don't know.
So yeah.
There is a problem.
I played a version of Clue where the body is down a dark staircase.
So you can't actually see what's at the bottom.
It's kind of-- you only see the outline.
So I think that was kind of the visual explanation for that.
But you're right.
It's not a very good justification.
Although, if people remember the movie, he's kind of stabbed by everything, right?
He's hit by a lead pipe and stabbed by a dagger
[INAUDIBLE]
Yeah.
Which was what was the first blow?
It doesn't matter what hit him after he died.
[LAUGHTER] How about for Risk?
You've got horses, people, and cannons?
Is that right?
Yeah
Yeah.
I always found it difficult to remember which was which, like what was what value
I seem to recall that the pieces were relatively bigger.
So I remember the cannon being at least as big as the cavalry, maybe larger.
And that way, oh, that's worth more than the cavalry, which is worth more than that little guy
The pieces could just have been shrunk due to cost.
It's possible.
I remember big pieces, too, but we had that [?
compilation.
?]
And maybe we were smaller, so the pieces looked bigger.
[LAUGHTER]
Our pieces were also Roman numerals
Yeah.
I remember seeing people who had that type of pieces and envying them because all I had were freaking Roman numerals.
But they were easy to count, at least
Yeah
All right.
So for the rest of today, you've pretty much got time to work in your teams.
A few of you can continue playing, finishing up your games.
The prototyping materials are all up here.
Feel free to come up and grab what you need.
Rick or I will be in the room at any given time.
So you can come up and ask us questions.
But use this time to be able to meet up with the team
Yeah.
If you want us to play your game, we can't, but we're in class play tests on Wednesday
This Wednesday?
I have the schedule here.
One second.
Let's see.
March 12th.
Yes we do, in fact, have a play test this Wednesday.
So make sure your games are ready.
Make sure you have a draft of your rules so that you can test whether people actually understand your rules.
OK?
On Wednesday.
[INTERPOSING VOICES]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
I'm just going to get started.
First, a little administrative stuff, definitely fewer people signed in on Monday than actually showed up in class.
So there were a bunch of people who just forgot to sign in.
I tried to check off people that I remembered being in class.
But if you were in class and you think that you may not have signed in, I have Monday's sign up sheet here.
So this is up front.
I'm not going to pass it around.
You can just come up here and sign it if you need to.
The actual attendance sheet is, I believe, on the table right there now.
Yeah.
So make sure that gets handed off to anybody who comes in later.
And I'm going to start off talking about the reading.
We're going to play some games that are related to that reading.
And then we're going to take a break, do your set up, and then you'll be play-testing with each other.
All right, who is on a team who doesn't have a game yet?
You're waiting for team members to bring it in.
You don't think you're going to have a game ready today
We don't have a game
We don't have our board, but we can do it on tiles again
You can do it on tiles?
So far we've only play-tested in the hallway

We actually bought a board, which is on its way
OK. On its way is fine, because we're going to do play-testing in the second half of class.
But on its way, you mean like US Postal--
On its way as in on a truck
Oh
We've only played it on tiles.
So far it's worked.
So we can just do it on tiles
On tiles?
You mean--
Like out in the hallway
Oh, because it's a live action game
Yeah
Oh, all right.
That makes sense.
Actually, that will work fine.
All right, so today's reading was partnered.
And some of you may be surprised that we jumped all the way back to chapter 1.
But it's kind of revisiting an old topic, right?
First of all, he talks about the basic concepts of games and play and sports.
And I kind of want to go back to that.
But I've already voiced my own opinion on "what is game" question.
And I think it's not very productive to talk about what is a game, because then you now have to decide what isn't a game.
But what I do like to talk about are all the different things that play and game can mean-- especially in the English language.
Even in Parlett he always talks about how it's a little weird that we have two different words with two different origins to mean very related things.
But as a result of that, these two words have kind of been used and reused to describe a whole bunch of related-- but not quite the same-- concepts.
And I just want to be able to run through that with you.
So we have play.
We have game.
We have sport.
So that's let's talk about what would be like the noun definitions.
Like, what are all the things that play can mean, game can mean, or sport can mean when you are using it as a noun?
Game as in-- I don't know.
Like a fighter maybe, like to spit game
Uh-huh.
To spit-- to spit back game.
So that is-- OK.
I'm going to use the word swagger, because I think it gets the concept across well.
Yeah
Like gaming the system.
You know how you're like--
Well, that's kind of verb-y, but yeah
Oh yeah, that's right
To game-- which means to--
Sorry?
How would you describe that concept-- to game the system?
Gaming-- gaming the system
I don't know.
Ways to cheat, or--
Look for loopholes, exploits, you know-- exploit, maybe
Play-- it can be like a move, or like an action you take
Like a baseball player.
Right, you know-- that was a great play
Yeah
So like tactics almost, right?
Yeah
No wonder you actually have to execute.
I'm glad that you caught that.
It's something that actually had to have happen
Like a theater performance
A theater performance-- so also stage, or opera, or something like that-- a stage play.
What else is?
Play as in not that serious

Playing around
So child's play.
You know, this idea of the thing that kids are doing most of the time.
That's playing in some sort of general noun sense.
OK.
So I'm going to go an put child's--
Like pressing play-- for a video or starting like a game
OK.
So play as in to initiate a sequence.
Sometimes overlaps a little bit with this.
But yeah, you know, it will do.
Because you can use play to describe something that happened in the past, or to make plays, and to start a play.
All right.
Or press Play to-- you know, actually, that's-- instead of initiative, maybe it's like this icon, right?
Yeah
This icon on the VCR-- no one uses VCRs.
How about on a QuickTime streaming window.
All right, so what else?
No one's touching sport here
Sports you may like to show off.
Like if you're sporting a [INAUDIBLE] thing
OK.
So like sporting your colors
Yeah
OK, so like a display of some sort.
I'm going to put that more in the verb side of things, OK?
It could be like a nickname for a young--
OK.
Sport.
Yeah
It can also be like a good sport, so like a nice game
So some of those virtuous kind of things.
Yeah.
I'm not sure if I got that right, but--
It can mean like fun or hobby-- like you do something for sport
Fishing for sport
Fishing for sport?
Yeah
Like hunting for sport
Fishing for sport as opposed to sport fishing, right?
Those are, you know-- [LAUGHTER] That's another pun, but never mind.
So for fun.
So I mean it's funny, because often it is used to make something sound like it's for low stakes when it actually is very high stakes.
Like, you know, oh, we're going to have people for sport.
[LAUGHTER] I just think of like 18th century literature when it comes to like, he plays at relationships for sport, or something like that.
You know, it's like something that's supposed to be high stakes, but it's not.
OK, so I think maybe instead of for fun, I'll say for low stakes or no stakes-- which is bizarre.
Because sport is usually a very high stakes thing.
What else?
People think sport is something athletic
Mm-hmm.
OK, so there's a whole section in Parlett where he talks about the possibility that the word game might have come from the bending of knees, because gam is a sort of Welsh root for a leg-- gam, cam.
But that's nowadays more associated with some sort of-- physical activities and sport tend to go together.
What else?
Like something that you hunt-- like that sort of game, like--
Like poultry?
Yeah, game as in animals that you hunt.
Yeah
[INAUDIBLE]
I guess this is more of an idiomatic expression, but people tend to use game when something is also low stakes.
Like, you've heard this as a phrase-- "do you think this is a game?"
Yeah.
So again, something that is low stakes or is inconsequential, almost.
Like there is no penalty for having done this thing.
All right, what else?
[INAUDIBLE].
Well, along the same lines, what if you're like specifically making a joke out of something-- like you've made is a game to do that?
Um, hmm
Like that's sort of like an intention of mocking or--
Oh, hmm.
I feel that's related to this one
Yeah
Because they're kind of-- when you make a game of something, what you're really doing is kind of belittling it, right?
Yes
So I think that's just a different application of the same thing
But when you're making a game of something, aren't you also making a structure of it?
Or is that different than what you're saying?
For me?
Yeah
Well, I was saying from like the sort of a mockery sense of it.
Like he made a game of our process, or something like that

But I think making a structure of it is a different application, right?
It's like you are applying rules to something that might have not needed it, necessarily.
But, you know, now you're come up with rules.
I'm going to make a game of tipping, right?
Because we have five people, [INAUDIBLE].
All right
It can also refer to [?
hooks.
?]
Like before the Superbowl, everyone talks about the big game
Big game-- right, OK.
I'm just going to say the big game, or the game
And I'm also thinking of the phrase like it's all part of the game, where someone might do something--
Are they a sports lover?
No, isn't that skin in the game?
No, it's not that-- even in sports, it's all in the game, right?
Even in sports it's in the game
But skin in the game is almost certainly a morsel of some sporting good that has--
It's all part of the game
It's all part of the game, yeah
Like when someone does something unexpected, or maybe you might think it's unethical, or slightly immoral, or going out of the boundaries of the--
Are you thinking of Game of Thrones?
I'm thinking of like a lawyer doing something backhanded.
And you know, like the prosecutor is surprised.
And the defense attorney is like, it's all part of the game
Right.
Yeah, so there's just sort of like this bounded space where things are permitted-- like specific kinds of things that might-- this introduces a concept that-- it introduces the magic circle.
I'm not sure if we have any reading that touched on this-- but this idea that a game is this bounded space where you can do things that you wouldn't necessarily be allowed to do in real life.
And, similarly, consequences that happen inside don't necessarily apply outside-- like it's incredibly important to hold it inside.
Like when the position of a ball is generally meaningless outside of the game.
But inside the game, it's everything, right?
But you know, you can body check somebody in a hockey rink.
You know, you're not really allowed to do that on the sidewalk.
So I think that gets close to what you just told about your lawyer application.
You're allowed to do this within these parameters, because this is how the game is played, all right?
Whereas, I'm wondering about the game-- there's this concept of the game, as in the big game, like a Superbowl And I'm wondering is that only used in broadcasting?
Or is that--
So there's certain rights.
So you can't say like Superbowl without getting permission of whoever has the rights to that name
But even something like "are you going to come over to watch the game", which doesn't necessarily mean that it's the Superbowl
Right
Like there's this weekly thing.
But it seems to be very TV related.
"The Game" as this thing that you see on TV is this event that's happening at a specific time, rather than football in general
There's some crossover, then.
Because like the stage aspect-- because you're watching it-- but also the athletic aspect--
Yeah it goes back to this
--because it's a sport, and you watch the game
Yeah.
So I'm just going to put TV next to that.
OK. You had your hand up for a moment
Yeah, I was going to say games as like skill, as in like he's got game-- sort of like skill or ability
I think that goes back to the swagger a little bit
Yeah, kind of
But swagger's like attitude, whereas this might be skill
I guess if you go by--
The assumption, I think, is that those two are related
Well, one is the appearance of skill, and the other is having skill
Oh, spit mad game versus got mad game.
OK. All right.
OK, OK.
So actual skill versus the portrayal of that
The game, like as a pick-up artist thing
Oh, that's sort of the dating game, basically.
It's kind of like-- I know that in those books that's very specifically called a game.
But I think that the use of the words "the game" to describe dating in general predates that book by far
Play could also be referring to sports.
Like in football, there's a certain book that you [INAUDIBLE] plays-- a play book
Yeah.
That was kind of like defense, right?
But you're right.
You can write these things down.
You can talk about them as a library of things that you can do
There's the Great Game
The Great Game?
Was it the Afghan War or the Crimean War?
Oh.
So now we're thinking part two
Yeah.
OK.
So like the England versus Russia-- or Napoleon versus Russia-- I forget exactly the years and dates.
But it was a social political conflict going on
So there are specific wars that are referred to as great games.
Game of Thrones probably is related to that, too.
Although, that includes political things.
I mean, even just on the noun side, we've gone through a lot.
If we include adjectives-- and we've brought up a couple of adjectives so far-- part of those to talk about how some people can say that you have a [INAUDIBLE], which means you actually have [INAUDIBLE], which is kind of weird.
But it's his extrapolation of where the word game might actually have come from Celtic origins rather than from Latin origins.
How about verbs?
Sporting-- to sport-- well, I guess we already did to sport.
Although-- to game, to play, [INAUDIBLE]
Like playing [INAUDIBLE]
Yeah, exactly
Like to execute.
Like if you make a play, that's like you're executing it
Building on that, it could be like a performance-- like to play an instrument

When you play [?
a part ?]
you kind of execute it
You're executing a series of things.
You execute a stage play [INAUDIBLE]
But to play something in a game seems like a much more time-limited thing than to play a musical instrument.
It seems kind of like a lifetime thing, almost.
Like I play something.
I play the clarinet has a very different connotation from I play this queen, or I play this card.
So those are two-- probably on the same spectrum, they're like almost on opposite ends-- closely related, but on opposite ends.
Sporting?
Sporting I generally associate-- again, sort of like "to be a good sport".
But this is more like fairness-- sporting-- give someone a sporting chance kind of thing.
If it's just to finish
It actually might be
It's funny how this seems to be like the only connotation of game as a verb, right?
But gaming is also kind of associated with the gambling industry.
It you go to a gaming conference, what you actually mean is that they're talking about stakes-- stake games, roulette--
What about like to game-- like I play games.
I game
Yeah, so that's a fairly recent connotation-- usually in association with digital games, but not always, right?
So to play.
So we can say to someone, I am a gamer.
It's not implying that I look for loopholes in everything that I can do.
So I'm pretty sure we could keep on going for a while.
I just think it's kind of neat that we have all of these different terminologies that all use the same words.
It often can get confusing in sort of casual speech.
Specifically, it doesn't get too confusing when you're talking to fellow designers or other people in the game industry.
Because the context of which they're using any one of these words, to me, is pretty clear.
And you're always thinking about the context when you say, you know, I game the system, or they've got game.
You know, it makes it very clear.
But I do think that Parlett kind of slightly opens the box.
He tries to cut it into a sort of a cross section of all the things that these words can possibly mean.
Yet, it's even richer than that.
And the reason why he does that is so that he can actually get into the topic of his book, which is board games specifically.
And he starts breaking it down.
He's like, all right, now that we know that games and play and sports and everything are all kind of fuzzy, and muddled, and not really very clear, how do sort of at least clarify what we're talking about when it comes to board games?
Anyone remember the five things that he ended up with-- the five categories of board games that he kind of ended up with?
Four of the rhyme, one doesn't
Race games
Race games
Space games
Space games
Chase games
Chase games
Displacement
And displace, which he admits he only gave that name because it rhymes.
And then number five?
I think he just give up at this point.
It was interesting.
Because he was kind of saying like here are these four that, traditionally, other scholars like to study, because they are bereft of theme.
And they look at themed games, and they say, like, that's not worthy of studying.
There's this whole chunk.
And Parlett says, wait a minute.
First of all, most of games that you'll buy off the shelf right now are probably themed games.
And the reason why there is this interest in race games, chase games, displace games-- space games?
I think I'm confused.
He also describes those more as positional games.
And positional games of the sort that scholars like to study tend to be folk games that have been handed down from generations.
Of course, he's narrowing it down to board games.
He's not including things like card games, which are also a very rich game tradition.
He has a whole separate book-- that I think we also get to in this class-- on card games.
But this is just specifically his board game work.
But he wants to start looking at theme games.
And this was him introducing scholarship on theme games later on on his book.
I just want to make you realize that the whole idea of studying games as a product-- we already talked a little bit about how the idea of a game as a product to begin with is fairly new.
We're talking about 1900s when that starts to become a thing.
But then the idea of studying something like that-- I think we're talking about like '50s and '60s-- very, very recent-- the idea that games as a product is something that we can study.
Whereas things like games as a sport can probably go back a little bit further than that.
Statistics, for instance, has been obsessed with baseball for a long time, because it's very rich and we have good products.
So I don't want to get too much into how he arrives at those definitions.
You know, he takes a couple of definitions that other people have come up with and tries to build on them.
Like all scholars, he citing previous work.
But what I'd like have us play today are what I feel are a couple of modern takes on those categories that he came up with.
So in the race game-- what's a race game, if you have to describe it in one sentence?
You get to the end point first
Get to the end point first, right?
So Cartagena-- has anyone played this?
OK.
So if you look at the game, it feels a little bit like a tile-laying game.
Because what you're really doing is you are trying to create this path from the jail that is keeping you-- you're a bunch of pirates-- from the jail that is keeping you all the way to the boat.
And then you can run off scot-free.
But one of the core mechanics is that every player controls a different set of pirates.
You have a bunch of different colors.
You're all trying to get to the boat first.
So the first person to get your entire gang of pirates to the boat wins.
So it's a race game.
You're racing against other people.
It just happens to be the track that you're racing on happens to be something that you build over time rather than something that's pre-determined.
So there's something they'll take I think.
Let me skip over to read the other things.
For displaced games, has anyone played Twilight Struggle in this room?
OK.
I may need you to help explain this game to other people
OK.
I've played it once, like at the end of my first semester
OK. All right.
I'll just take a look at the rules again.
This is unfortunately a little bit of a tall order to ask someone to pick up Twilight Struggle in class.
But this was looking at ease of the design system.
This is a war game.
You are given a map of the world
Oh, gosh
OK?
It's the Cold War-- 1945 to 1989-- and there's a whole bunch of things that come from modern game design, like you have event cards that recall important things that happened-- the Cuban Missile Crisis, Korean War, Pershing II rockets deployed, things like that.
But again, it's about influence across a space, which just happens to be laid out on top of the map of the world to regions like the United States of America, to South and Central America, to Western Europe, Eastern Europe, and so on and so forth.
So you're deploying influence points, which is not all that different from deploying what he calls mans-- men-- mans-- across a space, and trying to be able to push influence towards your faction, rather than to the opposite, depending on whether you're playing the US or the soviets.
So displace games-- hold on, are those displace games?
Well, let me just set up this one too.
Scotland Yard-- I think this might have been-- this was what I intended to bring out as displace.
But I'm not quite sure that you capture any pieces in this game.
Do you recall?
Do you capture things?
You capture influence.
You trade influence back and forth over--
Right.
You capture it, but then you can reintroduce it later on
Yeah
OK.
So this has been displaced.
Scotland Yard, which I believe we have mentioned in class before, is kind of like the evolution of the chase game.
Chase games are super old.
Some of the more famous ancient ones are Scandinavian in origin-- a lot of Norse-- a lot of Viking games-- where you have a King.
You have a couple of bodyguards for the king.
And you've got a whole bunch of low level pawns all trying to flank the king.
That's the theme of a whole genre of games called Tafl, of which chase games will form a big part of.
In this game, you are a Mr. X on the run from a bunch of Scotland Yard detectives.
The map is London.
More importantly, it's the London public transport system.
And what you're trying to do is you're trying to go underground.
You're trying to grab cabs, take buses, and conceal where you are while the rest of the players were all controlling the detectives.
Actually-- yeah, three to six players.
The people who are controlling detectives are basically just trying to flank you.
And so-- a nice little modern evolution-- they have a neat little doodad, which is the way to be able to track the moves that you made without revealing it to your opponents
Is the bad guy's position private or public
It is private.
But little bits of information pop up.
So you reveal that in the public.
Empire Builder is a train game which one of our grad student alums now almost exclusively studies-- but a very interesting kind of training game that you may not have heard of, called crayon rails, where you are given a big sheet of plastic laminated board-- I guess it's kind of like a jigsaw-- and a grease crayon-- a bunch of grease crayons, actually-- Crayola washable crayons.
Here we go.
And you are just drawing your rails across the map, trying to establish your railroad empire.
So this was the one that I was thinking of as space, because it's all about occupying space.
But you're occupying space with a network, rather than occupying space with just pieces that you're placing down.
You're creating lines across the United States while you are do that.
So I think this particular genre of train games are an interesting take on space games.
There's a whole other genre of train games that's really more about the economics of what it's like to be a robber baron train lord.
Those are really fascinating as well.
And those fall kind of more on the economic sense, which Parlett doesn't really address.
Or maybe those fall under his definition theme games
Yeah, I think they do
Yeah.
And Power Grid is just this huge mishmash of everything.
But I am using it as an example of a theme game, where if you play any of these games and you play Power Grid, you'll see their connections.
It's a game where you play on a network of Germany.
And you're basically trying to create efficient links between different power stations that you are building.
And it's not all that different from creating a railroad empire.
You just happen to be trying to create an energy generating empire that's going to use different commodities-- like oil and-- I think you can burn refuse.
Yeah, you can burn refuse.
You can burn coal.
And you're all playing on the same space.
You can occupy space.
So it's kind of like a space game, because when you take a space, no one else can take that away from you.
But most of this game is about the economy rather than about the specific positions where you place things-- although, that's going to influence your economy.
So it's a complicated game.
It's very well designed if you have the right set of rules.
The rules that come in the box have misprints in them.
Specifically, they break things into turns, phases, and steps.
And then they kind of don't use those words consistently.
If you are interested in seeing how fairly straightforward rules can be really, really confusing, take a look at the rules of Power Grid, then take a look at the printed rules that I printed out from the printer.
Because these are the revised rules from Board Game Geek that are basically translations from the German rules.
And they're a little bit more consistent.
It is a complicated game.
We might not be able to get through a full game and learn the game at the same time.
Has anyone here played Power Grid?
I encourage you to set up the game to help more people do it.
All right.
So what we're going to do-- actually, any questions before we break out into game groups?
No?
OK.
So we'll play this until about-- actually, I think all the way until 3:00 probably, or at least 2:45 or so
We'll call time [INAUDIBLE]
Yeah.
We'll have about 15 minutes for you to get all of your play-tests ready.
And then the last hour is when you're play-testing.
Cool.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So today's reading comes from a game designer, Lewis Pulsipher, who traditionally worked on war games.
I believe he's done some of the board games, as well.
But for the most part, I think, he's recognize for his war gaming work.
And there's actually quite a lot of war gaming journals, mostly written by designers for fans or by fans for fans.
Things like the [?
Compleat ?]
Strategist used to be like the PC Gamer of war gaming, right?
It's like, this is what's happening and these are our thoughts, these are our critiques.
And that particular reading I feel is very typical of that era of, maybe, late '80s, early to mid '90s magazine publications.
And it's interesting because already at that time, sure, there are people who, say, wrote the core rulesets for various games, like Axis and Allies, or things like Dungeons and Dragons.
But there was going to be this assumption that if you are interested in something like war gaming, you're also interested in creating your own scenarios.
You are already halfway a game designer.
And so you have to keep all of these things in mind.
Particularly if you're a game master, has anyone been like a dungeon master or a game master for a role playing group?
OK. You're basically a game designer
Yeah
Yeah.
You know, you're taking rule sets that somebody else [?
came up with ?]
and cherry picking the stuff that you really want to work with and discarding the stuff that doesn't and then coming up with your own rules.
So just as and example of reading, I believe this is the first time I've asked you to read anything from the table top book?
[?
Tic a Tac.
?]
[?
Tic a Tac?
?]
Yeah
OK, all right
You just ran this first
That's true.
Yeah.
But this particular piece of reading is also a nice little time capsule of how people used to write about game design.
It's a fairly modern piece of writing it still has that same sort of style.
So there's a lot of examples of, like, this is how this theoretical game is going to be played and I'm going to write it out completely in prose.
And even when they refer to games that already exist, like, I believe they do cite [?
Vinci, ?]
at one point
[INAUDIBLE]
Yeah, again the game mechanics that they're discussing are all in prose.
It's not like in point form or anything like that, it's just free prose.
I don't think there's ever any assumption that if you contributed an essay for something like [?
the Compleat ?]
Strategist that diagrams that you included would be included in the magazine, right?
Because a lot of people who actually created those essays weren't necessarily artists.
And that's not to say that you didn't have diagrams in magazines like that, but it was expensive.
This was before desktop publishing.
So people just got really, really used to describing things in prose, describing core game mechanics in prose, not just in point form or bullet points.
So there were basically two suggestions that he has on how to deal with the whole three player problem.
Actually, before I get into it, can anyone quickly sum up what's the main problem with designing a game for three players?
Your answer
Well, one of them was [?
turning-- ?]
Mhm
--where two of the players sort of go at it for most of the game and then they weaken themselves, and then the third player [INAUDIBLE]
OK, so, for a certain set of rules, that's one kind of strategy which encourages you to basically not interact with anybody else and then win the game, right?
What I going to say is, this is where it's just one player chooses which of the other two players wins
Mhm, so I believe [?
kingmating ?]
was the name that was given to that.
You may not be in a position to win the game but in a game with only three players, you might have enough influence to basically say, I don't like you, therefore you are not going to win.
Therefore, the other person is going to win, even though I can't win.
There were a few more subtle problems that he described.
Anyone remember any of them?
[INAUDIBLE] Yes?
Politics, so two people decide to team up on the other player and the other player really can't win
Uh-- [INTERPOSING VOICES]
--yeah, he brought that up.
It does come up in reading.
It's not necessarily described as a problem, but something that does happen and it happens often.
So you either take advantage of it as a designer, or your game suffers because you hadn't thought about it, right?
This whole idea of, somebody's in the lead and two people probably want to do a temporary alliance at the very least to take down that player.
There was one called sandbagging and I actually am forgetting what that means here
If we could set that up, saying that you're [?
doing ?]
less well off than you actually are
Oh, yeah.
Pretending that you are-- like bordering on lying, right?
It's like, no I'm not going to win the game in the next turn, don't worry about me, deal with him, he's scarier.
And that sort subtle deceit where you have to actively conceal the fact that you are not in the lead.
There are some games that say, all right, here are some ways that you can do that.
I'm not sure, are [?
we enough ?]
for Crunch?
I think we are.
[INTERPOSING VOICES]
Yeah.
So Crunch is a game about being a banker.
You're trying to fill up your golden parachute before the market crashes, and you are encouraged to play the game in a three-piece suit so that you can actually actively hide cards.
And the idea is that at the end of the game, the amount of money that you've managed to keep yourself for your own personal use, including any cards that you happen to [?
have ?]
on yourself, will win you the game.
It's not about keeping your bank afloat, at all, it's all about leaving with the largest personal fortune.
So it's a rye, satirical game, but one that actively encourages you to conceal how well you're performing.
Now, it's a two-player game, it's not like a three-player game, so that you are also well aware that your opponent is doing the same sort of thing.
You're just trying to do the best that you can in accumulating as much cash as possible because that's what's going to win you the game.
But in the situation with three players, you can imagine a situation where someone could figure out that you're in the lead and that person can catch up.
And what can you do?
Like, pass cards to the player that you want to win, for instance?
And that might be tough because someone might not be able to interfere with that process
In the game [?
Junta ?]
you're a politician in some Latin American banana republic, and your goal is to embezzle as much foreign aid money as possible.
And one of the things is that, the number [INAUDIBLE] drawn each turn is variable, and only the president really knows, and so [INAUDIBLE] is that sometimes the president won't draw very much, which might cause people to launch a coup against him, because they think he's lying and taking a lot for himself and there scared he'll win for that reason.
But, basically there's so much uncertainty about how much people have.
It's not even computable how much people have in their--
It's Junta?
With a J, right?
Yeah
So, [?
this ?]
[?
is ?]
[?
be, ?]
right?
It sort of plays around with this idea that you have this dictator that seems to be amassing a large personal fortune, whether or not the dictator can be successful in a process of amassing a large personal fortune, that game is designed to paint a giant target.
And we have another game here called King of Tokyo, which is basically giant monsters trying to take over Tokyo bay.
Every sing generic giant robot, giant dinosaur, octopus thing, mecha dinosaur.
And whoever happens to be a king of the hill at that point has a giant target on them, and the game mechanics all support that, that's what the game's all about.
So those are some of the problems and he proposed two solution, not necessarily the only two solutions, but these are the ones that occur to him, particularly in war gaming, that might work.
And they're actually kind of opposite ends of the spectrum.
Does anyone remember one of the two solutions?
Equilibrium?
Mhm.
[INAUDIBLE]
If one person gets ahead then they cannot be significantly defeated before they win the game
I think equilibrium was the opposite of that
Opposite?
Yeah, so you've actually just completed two answers into one, which is good.
[LAUGHTER] So, the equilibrium, if I recall correctly from the reading, the equilibrium design goal method that you can manage to design is taking advantage of things like the balance of power theory, which is, if there is a person who is clearly ahead, everybody else who is weaker is probably going to band up and take you down, and if you just balance the game in a certain way you give them the tools to do that, and that takes anyone who seems to be ahead back to a sort of equal power level.
That's a balance of power theory is something from political science.
I've only heard about it in passing.
If there's anybody here who has taken political science classes, you can tell me if I'm getting this right or wrong.
The whole thing is not a far fetched idea, right?
If you've got a powerful nation other nations who are bordering that powerful nation are likely to either team up with the powerful nation, or more likely to band their [?
nations ?]
together, even though they may not be ideologically linked or anything.
They're ideologically linked against the powerful nation, so same thing goes for players.
So basically it comes down to giving everybody the tool to take down the leader a notch, and does restore everything to a sort of equilibrium state.
And then your success criteria for the game probably isn't who is going to gain such a massive leap, because the game isn't deliberately really designed to wait for everything to equilibrium, you need to have some other kinds of success criteria.
Maybe the game is time limited, for instance, maybe the game has different goals for each player, or something.
So even though your power levels are about the same you might be able to get victory points on some things faster than others.
The alternative, which you hinted at, which was, basically, making it difficult to take down the person who's in the lead once they're already in the lead.
He had one very specific implementation of this, or one particular rule of thumb, on how effective must a person in the lead be in order to prevent the king making, and sandbagging, and [?
turtling, ?]
and everything.
Remember this?
Remember?
The basic idea is that you must be able to do it in one turn, in a single turn.
If you are in the lead, you should be able to complete the game and win the game in a single turn.
I'm not so sure whether I agree with that.
And I don't think he means win it on your turn.
I think it means in a round of people.
If you're in the lead in the game, you should be able to do something to cement that lead so that within that turn no one's going to be able to do anything about [?
changing ?]
the [?
turnout of ?]
the game.
This is for games where there is a clear win condition that is based on overpowering your opponents.
That you don't want to have the situation where, yeah, I'm in the lead but everybody else still has enough time to take me down to the point that somebody else can then catch up and be in the lead, because that basically means that the first person to get in the lead will probably lose, and that's sad.
But we see that in certain computer games.
We've seen games where whoever who is in the lead has a giant target painted on their back and you're expected to be taken down.
If you're the first one in the lead you are probably going to get hit, but then that now [?
there's ?]
enough time to be able to pull ahead.
So that's closer to the equilibrium style, right?
So I think I heard someone give--
Oh, Mario Kart
Mario Kart being the obvious one.
You get increasingly powerful weapons the further behind you get, or useful things.
There's an example in this book here, the Rules of Play-- this is the hardback version-- of Super Monkey Ball.
And, basically, all their weapons face forward.
So again, number one, you can only shoot forward, there's nothing to shoot, there's nobody ahead of you.
It's not actually a technical disadvantage, it's not like a weapon that slows you down, but it's useless thing.
You get these powerups that are useless.
So a lot of these feed into a different way of looking at games that is a very formal, very [?
technique, ?]
and one day you've probably heard of, mentioned multiple times in this class and in other classes, as well, and that's just feedback systems.
I don't care about UI feedback, I'm talking about feedback from the cybernetic [?
sense.
?]
Anyone here has heard about cybernetics from another class?
Yeah
Oh, I was going with the feedback systems that you used in your other class, like we did [?
in the engineering one.
?]
Yeah, yeah.
I definitely brought it up in there
Yeah, where you had a forward feedback system and a backward feedback system, and, basically--
Mhm, positive and negative.
Yeah
--If you were winning, you either start winning by more, or your losing, you start to catch up.
For example, [INAUDIBLE]
Right.
So, the whole basic idea of cybernetics is based on, sort of, automated control.
You create systems, mechanical or electronic, or with a set of rules that basically are going to either accelerate or snowball the performance of the system, or bring it back to equilibrium.
If this sounds very familiar to something [?
Mech-E ?]
or [?
EE ?]
or something, that's completely expected because this all originally came from that.
So, the basic model is that you have some sort of sensor, something that detects an environmental change.
It could be something that detects the state of your game, some way of telling you this is the current state that you're looking at.
Then goes to a comparator-- which needs more space because it's a longer word, comparator-- which performs some sort of logical assessment on the state of the system that you're looking at.
And then you're going to an actuator, which performs some sort of action.
And it's called feedback because the thing just keeps feeding back, all right?
Real life systems that do this, what are some of the simplest real-life systems that do this, electronically or mechanically, without human intervention?
Washing machine
Washing machine?
Thermostat
Thermostat.
Washing machine?
That would watch water levels I think, right?
Well, the [INAUDIBLE]
Oh, yeah, yeah.
That's true.
So, thermostat, things get too hot, turn on the cooling system or shut off the heating system
Wonder if that emergency [INAUDIBLE]
Emergency?
[INAUDIBLE] There's a hole at the top of sinks where if the water--
Yeah
[INAUDIBLE]
Yeah.
The sensors, whether the water has actually hit that hole.
[INAUDIBLE] I guess the sensor is the level of the water, the comparator is the height of the hole, and then the actuator is if it's at the height of the whole, go out the hole, right?
So, it's actually just like physics is the entire system for you, but it's engineered in a way to take advantage of that
Systems like autopilot
Mhm, keeping you on the path that you're supposed to be on, always just doing minute little corrections.
The majority of these things are real-world applications of these negative feedback systems.
And the idea is that if there is a change in one direction, pull it back.
If goes in the other direction pull it forward so it always hovers around the same point.
These are deliberately designed systems trying to do this all the time.
But there are a lot of real-world systems that do the opposite, right?
Stuff that snowballs.
For instance, a snowball rolling down the hill.
As it goes down it gets larger and it makes it slightly easier to continue rolling.
I think some of the area in contact to snow actually decreases by a proper ratio [?
required ?]
to make it easier to keep going.
It picks up more snow and gets larger and larger.
Another positive feedback system in the real world?
I don't know the name of the game, but there's this [INAUDIBLE] game where you start off as an asteroid
[?
Astrobrah.
?]
Is that what it is?
Astro-what?
[INAUDIBLE]
You start off as a small asteroid, and what happens is you meet other small asteroids and you grow up to be a planet, and then, at that point, asteroids become fairly easy to gobble up, but you want to go after other smaller planets than you, and then [INAUDIBLE] and then keep on getting bigger until you become a black hole
Hm
And then, eventually you get so large that you can just eat everything up afterwards
There are two games that reminds me of.
One is Orbital, which was a Nintendo game.
Do you know?
Wait, which one?
Which game were you saying?
That sounds like Katamari Damacy
It sounds like Katamari Damacy
I think that sounds like a genre
Yeah
So, get bigger so you can get bigger
Yeah.
Let's see.
What's the game done by Andy Nealen?
[?
Orthos?
?]
No.
Osmos
Osmos.
Yeah, I think that's a PC and portable device game.
So, Osmos is an interesting situation because yes, the bigger you get, the bigger you can get, because the idea of Osmos is like you're just a little amoeba-like blob, so asteroid, amoeba-like blob.
You can eat anything smaller than you, and everything that you eat makes you bigger, so you can eat even bigger things.
But actually it makes it more difficult for you to navigate because you're more massive now.
So you actually have both negative and positive loops going on at once
I have another possible one
Mhm?
Bombs
Bombs?
Yeah, because then [INAUDIBLE] exploding and--
It makes other things explode?
Well there's that, but there's also some bombs where--
Nuclear bombs
Yeah, nuclear bombs
Oh, so like you're using an atomic bomb to set off a hydrogen bomb, that sort of thing?
Something like that
OK, yeah.
Positive feedback often happens in military conflict, right?
If you've got overwhelming forces every engagement just makes the ratio, if everything goes according to military doctrine, every engagement is going to give you yet a greater advantage.
[INAUDIBLE] So, the way this has been applied in games is-- just to make it literal-- you have some sort of state of the game.
Am I getting this right?
Yeah.
You have the state of the game.
Wait, hold on, let me just make sure I've got this perfectly right.
Here we go.
You have some sort of scoring function.
It doesn't have to be the literal score, but it has to be some way of assessing how close you are to some sort of goal.
So either the short term goal or the overall goal of the game, some sort of scoring function.
And that goes into some set of [?
prior ?]
input.
There are some versions of this diagram that separate this into two things, that's kind of like the player decides what they're going to do and then actually executes the input.
Actually, there is one more stage, sorry.
There is one more stage, and that's the kind of the mechanical bias.
So the state of the game can be revealed in some sort of scoring function.
A player assesses what they want to do and then tries to execute it.
That is filtered within the computer's or the board game's or the card game's interpretation of what that input is going to do, and then that feeds back into game state, right?
So for games like Mario Kart, you perceive that you are doing badly because you are the last one at the back of the races.
You drive your vehicle into one of those rotating boxes because you're deliberately steering to them because they're probably your only chance to be able to catch up again.
You get usually something really, really sweet because you're all the way behind and the way the game is designed, if you're all the way behind you get all the best toys.
Every single tool that you need to be able to catch up is given to you when you are all the way at the back.
That's the mechanic bias, all right?
It gives you the right tool and that affects game state because now you have this wonderful tool and you use it, hopefully it works, and then you gain a couple of places up in the race.
So, there are a couple of other examples that are a little bit clearer.
So, say, basketball.
If you've got a situation where you've got five people on each side, you could do a game where if you start to have a lead of five points, you lose a player.
Someone's benched.
[LAUGHTER] So is that negative or is positive?
Negative
That's negative, right?
Because you're saying, all right, you are far enough ahead, we're going to take away one of the advantages.
Is there another way to do negative feedback on that example?
Instead of taking away a player?
Add another player to the others
You could add another player to the other side.
That's also negative feedback, even though you're adding something, what you're really doing is you're going to give an advantage to someone who's not doing so well, and that's trying to bring things to equilibrium.
The positive feedback version of this would be-- [LAUGHTER] [INTERPOSING VOICES] What?
Turn the three-point line into a four-point line [INAUDIBLE]
Oh, OK. All right
Every time you score a point the other team loses a player
Yeah, OK. [LAUGHTER]
Do you ever play basketball?
[LAUGHTER]
I was thinking something more literal like just giving the team that's ahead extra players.
Right, that's right.
You could do it, every five point that you get, you get one more player.
Then pretty soon you've got two more players, then pretty soon you've got three more players, and then--
Pretty soon you have too many players
--and then you just have more people than anyone can move on a basketball court, and as a result of that you kind of win by default
Some game have a similar mechanism, like, somebody's on fire, they make three shot in a row, and all of a sudden they're just way better at playing
They run faster, they jump a little higher.
Yeah, and the computer games can totally do that
[INAUDIBLE] [LAUGHTER]
Well-- [INTERPOSING VOICES]
Being on fire is the same as momentum
Yeah.
Yeah
Statistically, if you're throwing a good game and you're doing more and more pitches, better and better pitches, you are playing better
But I'm wondering whether that's correlation or causation.
Is the reason why you're doing better because--
Right now they're thinking it's causation
It's causation?
They thinking it.
There's a new study that just came out
But it definitely comes up in sports psychology, right?
You're sort of building on the mental momentum of doing well.
And similarly a team that's not doing well can lose hope.
Again, that's positive feedback, I just want to be extremely clear about the use of positive, if there's anything that amplifies differences that already exist.
So, there are a couple of things to think about when it comes to feedback, especially as a problem solving technique for design.
We've already talked about how negative feedback tries to bring things down to equilibrium, so it's a stabilizing force.
If your game is out of control and unpredictable and chaotic, negative feedback can rein things in a little bit.
Sure, it may be unpredictable, but it's not going to hurt me so bad.
Or if somebody gets and advantage it's not going to be a runaway advantage.
Similarly if your games just actually always hovering at equilibrium and nobody seems to be getting an advantage, then positive feedback will destabilize that.
It will make it more likely that somebody who gets an advantage will actually get a large advantage, and that could be a good thing if your game is kind of dull and it's always just kind of hovering at the same level.
But there's a trade-off with that, because any amount of feedback that you're adding into your game, especially this kind of automated feedback, this is something that your rules are automatically generating, can actually be taking control away from players, and as a result of that might actually make the game less engaging.
So even a game with like, all right, this game is too stable, we're going to add some positive feedback mechanisms so that people who have an advantage can capitalize on that advantage.
But if that's not done right, then you've just got a situation where once someone starts winning everybody else loses interest because there's no way that they can catch up.
So it destabilizes it in a point that, yeah, someone's going to win this game whereas previous it was always stuck at equilibrium.
But it's very easy to lose people in that process because you're just creating and automated system to take control away from them.
So that's one danger of putting in a feedback system.
Also we talked about bringing things down to equilibrium to make things a little bit easier for people to catch up or to make advantages a little bit less drastic.
That can also make our game much more, right?
Because nobody gets an advantage, and depending on your win conditions, depending on your end of game conditions, could actually just make it interminably long.
You could take a half an hour game and make it a one hour game.
So on the flip side, the positive feedback game can actually help rein in the length of the game.
So if your game is currently taking an hour long [INAUDIBLE] and your victory conditions happen to be on gaining an advantage over your opponents, then a positive feedback system can help speed that along.
Right?
OK, maybe we can cut it down by half by just giving some advantages the the person who's ahead or penalizing the people who are behind.
That's an interesting little side effect, which is because positive feedback builds on itself, if you have an advantage early on, like you have early success, then positive feedback is going to be the mechanism that's going to help build on that.
On the other hand, if you have a negative feedback system in your game, which means everything is kind of being brought down to an equilibrium, but everyone's getting closer to actually finishing the game, but no one really has a huge lead, then late successes mean a lot more.
That last boost when you are near the final-- What's it called?
The finish line
The finish line!
That last boost before you hit the finish line on the very last round of Mario Kart means a lot because the whole game is designed with a huge amount of negative feedback, and could make all the difference.
People have played Mario Kart where that was pretty much the game, there was this last fight [INAUDIBLE] Someone just got hit with a blue shell in the last five seconds and [INAUDIBLE] So negative feedback prioritizes what happens late in the game, whereas positive feedback prioritizes what happens earlier on in the game, the strategies that you pick, the cards that you have which you draw in play right at the beginning of the game [INAUDIBLE] Positive feedback stresses that.
And finally, , always keep in mind that any interactional game systems probably create emergent feedback loops even if you had [INAUDIBLE] So what you want to do is you want to try to find that.
There is a tool out there that I've seen some student teams use, and I don't know whether it's going to be useful for you, I'm just going to give you it.
Has anyone heard of causal loop diagrams?
It's from economics.
You have?
[INAUDIBLE]
It's a very simple concept.
You just basically write in your variables in your game, and you can add in a few in between variables if you know how players are thinking about your game.
I've been using Mario Kart too much for an example.
Let's pick another game with a obvious [?
defect ?]
in it that everyone in this room probably knows
Chess
Chess?
Chess, does our piece-- OK, so, [?
our ?]
[?
pieces ?]
in chess sort of influence the number of attack [?
patterns.
?]
Which means the number of pieces will influence the number of pieces being threatened.
Of your opponents pieces being threatened.
And so it's just the number of opponent pieces taken.
Now, if I flip this around a little bit, I think I could make an argument that if I switched it to the number of my own pieces, rather than my opponents pieces, then it connects.
But not everything is a positive connection.
Number of pieces that I have means if I have more pieces I have attack power, sort of.
It's not exactly a linear connection, but [INAUDIBLE] chess set, right?
I have more pieces than you, I have more ways to attack than you.
If I have more ways to attack than you, then actually I have fewer pieces threatened because I get also more ways to defend.
So I'm going to say, defend attack [?
powers.
?]
Anything that I can attack, I can prevent from-- I say, well, if you take this piece, then I'm just going to take the piece that you just moved, and with a one for one trade, I also have more pieces than you.
And if you fewer of my pieces are threatened, that means I can-- let's see, if number of pieces that are threatened goes down, then the number of pieces that I can take goes up
Wait.
[INTERPOSING VOICES]
I think, yeah
[INAUDIBLE] be ignored
Hm?
If the number of your pieces threatened goes down then the number of your pieces taken goes down, as well
Yes
Those two are positively correlated
Yes, yes, that's right.
You are right.
And that should just decrease.
And if the number of pieces taken goes down-- If the number of pieces taken goes up, then I have fewer pieces.
If I have fewer pieces taken, then I have more pieces.
So we have two classes and we have two minor [?
suspicions.
?]
Over all, this entire loop is called a reinforcing loop.
Right?
This is part of the feedback loop.
You can connect this to other things.
Maybe in chess I'm not terribly good at causal loop diagrams to begin with, but in chess you might be able to connect it to other things like positioning of pieces, or the value of pieces.
So those might be reinforcing, and reinforcing is basically another way of saying it's a positive feedback loop.
Another way that you can say is that it's a balancing loop, which means actually a negative feedback loop, and that's usually you put that with [?
another ?]
[?
B.
?]
[?
Just remember ?]
that thing used as well as students from economics and management who have actually done that because they're trying to study things like supply chains and how a system [INAUDIBLE] and industry.
That's why they've already used the tool and then they can put it in.
But, it might be worth looking up, it might be a useful diagnostic tool for your game if you're just trying to figure out what are all the different loops and just write down all of your variables and figure out whether things even up.
Two negatives and two positives give you a reinforcing loop; positive, positive, positive, positive gives you a reinforcing loop; positive, positive, negative, and that's it, gives you a balancing loop.
Basically, [INAUDIBLE] add up to positive, positive, [INAUDIBLE] negative.
So, the one thing to keep in mind is that occasionally what happens is that you have something like a time delay, and that's notation for a time delay.
And that means things sometimes won't end up oscillating because of your [?
timeline.
?]
So you get a better state in some things.
We had a game, for instance, which was about climate change and you can invest money into research.
Yeah, at some point of time in the far, far future that pays off, right?
But that doesn't necessarily mean that it's going to pay off right away.
So that may end up in situations where you can make money, big fluctuations around the [?
store, ?]
so balancing in terms of equilibrium.
But just like a thermostat that's very slow to respond.
You might have something like a heater that warms up a room and it makes the room too warm before the sensor actually realizes that it's warm enough and then shuts it off and then the room becomes really, really cold before the sensor realizes that it's too cold and then turns it back on again.
And that can lead to these oscillations.
So keep in mind also things like timelines when you're drawing out these diagrams.
How long does it take this advantage to turn into that advantage, or this increase to turn into that decrease?
Yeah, any questions?
Yes
So isn't [?
a change in ?]
negative and positive feedback usually-- it seems really weird because usually positive means a multiplier greater than one and negative means a multiplier between zero and one, in the sense that, how much does your point [?
mean ?]
really near?
If I had a one point lead right now, positive feedback means that it's really like a two point lead--
Hm
--because the points provide more for me later and, even though, [INAUDIBLE] negative loop that wins but it will often mean that it's actually more like a half point lead because it means less than it really does, whereas--
Yeah
--it means power differences definitely have a case where sometimes if it's negative then that can literally be negative in the sense that taking this lead right now could actually hurt me because in the game it could actually hurt me in the long run
Yeah, I think positive and negative signals originally inherited from math, cybernetics.
It's actually referring to a differential, rather than the-- it's referring to the rate of change, rather than the actual multiplier.
That's some baggage that we've taken on.
It's funny because some people think negative feedback means bad, right?
And it's like, no, negative feedback could mean good, and could mean that negative feedback for someone who's behind could be an advantage.
So the terminology doesn't quite make it easy for everyone to use, but it's something that is currently in use, so it's good to know that game designers do use these terms.
Hm?
Talking about flight control systems and positive feedback.
Bad
A control system where positive feedback would be bad.
Yeah, I think that's kind of the opposite of the word control.
[LAUGHTER] Chain reaction might be a great way to describe that, for instance.
Any other questions about these ideas?
A lot of the stuff that I've been talking about is known as first order cybernetics, which is basically the system is doing its own thing happily, and if I'm taking a look at the system from the outside I don't really have any influence on how it's going to perform.
But, there is a second order, a whole school of second order cybernetics that I'm totally unfamiliar with, which actually takes into account having a person in the loop, which you would think would be a lot more applicable for games, especially games being played by people.
But I don't know much about second order cybernetics.
So, if you're interested in a research project or something like that, a thesis or something like that, come and talk to me.
I would love to [INAUDIBLE] because I don't know about that just yet.
All right.
One final word, this is a very formal way of looking at game systems.
Just as I have [INAUDIBLE] I just admitted that it doesn't take people's involvement in the process very well, it is not necessarily the case that the formal ways of looking at a system, just looking at how the rules interact, is necessarily always the best way to look at games.
In many ways, in many occasions, it is actually not a useful technique.
I can have [?
all these systems ?]
in my rules, but if people are going to read my rules and interpret my rules, as you've already seen happen in this class, differently, then they may be motivated to do things that operate against my assumptions because that's something else in my narrative, my aesthetic, in their own individual motivations.
If somebody wants to take down somebody else in a game just because they happen to hate that person and it's outside of the game rules then nothing in the rules are going to tell me that.
But it's still going to affect how that game gets played, and I might want to take that into account.
The Game of Diplomacy, for instance, has to take into account the fact that you're probably playing it with people that you know, and you have some sort of existing relationship with them.
I don't think it does a very, very good job of insulating you from the fallout of the game.
[LAUGHTER] Everyone knows what I'm talking about, right?
OK. Just unfriend the people that you play diplomacy with anymore.
That's why I don't like playing that game, but I like talking about it.
So, we'll be going into things like games for social play, and the social function of games, how you interact with people, right?
Some other topics are going to be things like games of simulation, games that have a slightly flawed mirror to the real world, but an interesting one.
It could be a fun house mirror which could be fun.
So, we'll be looking at that in weeks ahead.
This is probably about as formal as we get.
We've been talking about things like information systems, that's also very formal.
Just keep in mind that's just one school of game design and writing about game design.
OK.
So we have games that all have an interesting way of dealing with this.
Some of them, you're going to see these problems come up, like especially in [?
Imperiums.
?]
How many people do we have in class today?
One, two, three, four, five, six, seven, eight, nine, 10, 11.
OK.
So, 11, that's like one four-person game-- [INTERPOSING VOICES]

[INAUDIBLE]
So we'll get about three games going on at once.
Some games like King of Tokyo is very much a king of the hill game, so it's deliberately trying to ask you to take down the person who's in the lead, and the game mechanics make wonder whether being in the lead is really all that great.
So, it's playing around with [INAUDIBLE] Small World is occupying territory with your army.
Yeah?
[INAUDIBLE] when he talked about Vinci, Small World is a update of Vinci
Yeah, though faster to play, which fits in this class.
We have Vinci too if anybody wants to take a look at that.
We have that at home but we don't play that often because it takes like two hours.
Unless, of course, you already know it.
Lifeboats is just about a bunch of sailors trying to get to safety.
Basically, their boats are sinking and they're all trying to get to safety and everyone's jumping on and pushing people out of lifeboats, and swimming to lifeboats, and stuff like that.
So it's all about knocking people out.
Hoity Toity and--
El Grande and [INAUDIBLE]
Yeah, I actually know relatively little about these ones.
I forget--
Bluffing
Yeah.
Hoity Toity's a game about bluffing?
OK.
These were recommended by somebody's [?
father ?]
when I shared with them the [?
syllabus.
?]
And El Grande is a Spanish game about being territorial?
Yeah
No, I totally [?
made this up.
?]
Do you have English rules in here?
Yes, the English rules are in there.
The box is just [INAUDIBLE] but I'm pretty sure I looked in there earlier and I saw British or English rules.
Intrigue is the game that I like the thing the most.
It's basically, you're running a university and you're trying to put your foot down with another university.
That's not what the game says in the box, but I would like you to try playing the game with that theme in mind.
[LAUGHTER] So, yeah, it's interesting game because you are doing exactly the same thing that everybody else is trying to do, but in order to succeed in the game you have to put your troops, your colors, into other people's territory, and they're trying to put into yours.
And there's some mutual benefit [INAUDIBLE] So a lot of opportunity for you-- It's a pretty good negative feedback game because it gives you a whole bunch of tools to basically take [INAUDIBLE] take the lead.
Take a look at which games hide the progress of players from you.
Some of these games, you can easily see who's ahead, and some of these games it's really hard to tell, but that's deliberate so that you don't have these situations where someone all gang up on the people who are in the lead.
Look at how they're avoiding things like [?
turtling ?]
and how they discourage [?
stuff like ?]
sandbagging, or maybe sometimes encourage them and use them in favor of them.
Like Intrigue definitely encourages sandbagging.
All right.
And then we'll have these boxes out at three o'clock, working on teams, [INAUDIBLE] I think we're going to see your teams before spring break so you might want to give each other tasks to work on during spring break.
If you can't think of anything, test over spring break.
A good number of you are going to see people who haven't seen your game yet.
And--
So that you can have written rules, a first draft of your written rules.
It's a great opportunity to take the test then
Yeah.
And if you don't have a first draft of the written rules then this last hour of class is probably a good time to bring out the laptop and just start editing that text so that you can go into spring break with something that you can test with.
All right?
And for the game King of Tokyo the people will probably be able to play twice, maybe you can play another game.
But yet again they're all about 40 minutes
Yeah
Is El Grande longer?
Yeah, El Grande is probably 90 minutes or so
Small World is 40 to 80.
[INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR 1: So, yeah.
So the publisher can demand, you know, I want to see this feature.
EA demands that this game works with Origin.
Ubisoft demands that U-- is it Uplay?
Is that what-- that their universal sign-up system gets added on.
Things like that, right?
That happens all the time.
Other stuff that's maybe money-related or market-related?
Sometimes expectations just--
This feature in the past didn't sell very well, so you should add it to every-- PROFESSOR 1: Yeah.
That usually is similar to the publisher thing.
Someone's trying to push that.
But somebody could come internally from the development team, right?
They look at competing products and say, hey, this thing seems to be doing well.
You know, let's add hacks.
Because hacks seem to be doing well for some games.
So we can add hacks to our game.
It's not going to necessarily change the way how the game functions, but it's still a new feature.
It's still a new-- you've still got to come up with mechanics to unlock new hacks, and equip new hacks, and make sure they animate correctly, and things like that.
Even if necessarily the gameplay decision making you make in the game doesn't change.
Of course, hacks actually do change the way some games work.
The changes in marketplace in general.
And sometimes they're making a sequel to a game, or sometimes you're adding number two, or number three, at the end of the title.
And your players are going to expect something new.
And so there's some pressure for you to sort of add new mechanics instead of simply, say, iterating on the old ones.
Why would you cut mechanics?
Not necessarily cutting features, but cutting mechanics-- game mechanics.
There's one good reason I've been drilling in to you all semester long.
Yeah
Simplify gameplay and decrease the learning curve?
Or decrease whatever-- PROFESSOR 1: Decrease learning curve is a little bit separate from simplify gameplay.
Both good points.
Decreasing the learning curve is like-- testers just don't get how this game functions.
We've got too much stuff going on simultaneously.
You could try to tune those mechanics to make it a little bit more legible, or you could just try taking the mechanics completely out so that people can actually learn it.
Simplifying the game-- could be that, but it also could be just trying to make what's core about the game more obvious.
You know, this game is really about generations and generations of heroes, you know?
That is a game, right?
Hero Generations?
Yeah
Yeah.
PROFESSOR 1: Yeah.
And you've got, say, some sort of very elaborate weapons crafting mechanic or something in there.
Well, the weapons crafting mechanic may not be really what your game's about.
So you might want to take that out because people sort of forget about what the core experience of the game is supposed to be.
So focusing back on the core of the game.
What else?
Say your game-- oh, yeah
New technologies come out?
PROFESSOR 1: Oh.
New technologies come out-- like what, physics?
You know-- maybe?
Yeah.
Or has something to do with things like-- well, they mentioned-- well, I don't know.
3D, for example.
3D comes out it changed-- PROFESSOR 1: Well, that's true, right?
I'm trying to think of a game where that's a really obvious example.
Can of think of a game mechanic that substantially changed when it went from 2D to 3D?
I'm thinking Mario, but--
Mario--
Movement, because now, instead of moving left, right, up, down-- PROFESSOR 1: Just straight-up movement.
Well, the way how the camera responds to movement as well-- that's a big one, right?
In a 2D game, say a side-scrolling game, like Metroid or something like that, you don't have this situation where the place where you're trying to go to is obscured, unless it's actually physically blocked by something that you have to destroy or open up or something.
But in a 3D space, that happens all the time.
You know, you've just got bits of background scenery getting in the way of the camera.
Mario just changed the way jumps work entirely, when it went from the 2D Marios to Super Mario 64.
And every iteration on top of that has added new ways to jump-- double jumps, triple jumps, jumping off the wall, and stuff like that-- now, the latest Mario games expect that you know all of that.
Which is why I can't play Mario anymore, because I didn't go through that process.
Although the games are not that hard, at least the first couple of levels are not that hard.
Let's see.
What else?
There's one that I've been always mentioning, this entire semester.
It-- hmm?
Maybe just to try something.
PROFESSOR 1: Just to try something?
Yeah.
PROFESSOR 1: Oh, yeah.
Yeah, just to see how it's going to change how the gameplay works, right?
You know, let's take something out and see whether that was actually essential to the experience or maybe getting in the way of the experience.
You don't know.
You know, and the emergent results of taking out a mechanic might actually end up as a more interesting experience.
One thing that the book does mention is to take stuff out that might actually change the ESRB rating of the game, or the PEGI rating if you're selling something in the UK or in Europe.
So for instance, if you're building on top of a game engine like Unreal or something like that, which has body location damage-- and then you shoot a head, the game knows that you shot the head.
But maybe you don't want your game to be that specific.
Maybe you're designing a game for kids or something like that, and it's like, well, rewarding kids for head-shotting each other is maybe not the feel that we want to go for.
It might actually push us out of the E for everyone and into the-- what's the next rating, T?
And E-- PROFESSOR 1: Yeah.
There's like an E-10 rating.
At least there used to be.
OK. And sometimes, you've just got to cut mechanics because you don't have time to do them.
They're great ideas.
They would actually make your game better, if you had time to be able to polish them and make them understandable and smooth and everything.
You just ran out of time, and you just want to be able to ship.
So that's a perfectly legitimate consideration.
So that's this process of, like, generating ideas.
Say, this is like an x or something, the number of ideas that you've got in your game.
And mechanics in terms of either rules, and in a computer game, there'll be code.
In these games, that's just the rules, it's pieces, it's boards, it's everything that you actually have to make your game work.
And you kind of go through this brainstorming phase where you get a ton of ideas, right?
You've got some good ones up here.
You've got some crappy ones down there.
Actually, it's probably more like that.
A whole bunch of crappy ideas.
And you've got a couple of good ideas.
But yeah, there's a big explosion of ideas, where ideas turn into experiments.
Some of them are thought experiments, that you're thinking-- just discussing them among your team-- what if we did this, what if we did that, how would you solve that problem.
They may be real, testable experiments where it's something that you can play, something that you whip up within an hour or half an hour, and you actually put it in front-- we've been doing that all semester.
Some of the ideas may just come up off different ideas interacting-- and just, what if we had dogs in our game?
What if we had solar power in the game?
What if we had solar-powered dogs in our game?
And things like that.
You know, and then ideas just happen.
However, it's important-- so far I've sort of been describing the brainstorming process that you've all been through already.
But one thing that we haven't necessarily asked you to do nearly as much is to come up with actual discrete experiments.
You know, it's like, here's an idea.
Actually come up with a criteria of success that you're looking for.
How do we know whether this particular idea, this particular experiment version of the idea is succeeding or not.
And you're going to need that.
Because you're going to need to cull.
You're going to need to like reduce this back down, try to keep this, or maybe even build this up a little bit.
Actually, it's probably more like, you're going to cull a little bit.
Then you want to bring this up again.
And then each time this goes, you have kind of an increase in that you'll have an increase in bad ideas, but a decrease of bad ideas.
But your total number of good ideas just keeps going up.
And by good, I'm using a very vague term.
It could mean polished.
It could mean actually good for game play.
It could mean something that's been proven in a marketplace, something desirable that you want to keep in your game.
PROFESSOR 2: Things that could be-- like, it should be good for the core experience, but one of the problems could be, they are just-- they're good working mechanics, but actually don't help the core experience, or detract from the core experience.
PROFESSOR 1: Mm-hmm.
Yep.
Let's see.
There are a couple of dangers, even in this early brainstorming process.
So I'm just looking at this whole part here.
One is that you pretty much already think you already know what game you're going to make.
You've got a single solution.
Say you're taking the computer game version of this class and you decide that you're going to make an RPG.
And your model is Final Fantasy 6.
OK, so everything is going to basically follow the Final Fantasy 6 template.
So you're basically pursuing one single solution.
And the problem with that is that your implementation ability of actually making all of those things fit that template is already-- you are not Square.
Right?
You don't have all of the resources that Square had, even back when Final Fantasy 6 came out.
How old is Final Fantasy 6?
1994?
2?
I don't know.
PROFESSOR 1: Something like that.
I mean, even then, they were a large team with many years to work on-- at least a year to work on that game.
I don't know how long they actually worked on that game.
There's a very, very good chance that your ability to actually even implement something that you know how it already works may not be able to meet that.
So you will have a failed experiment and then you're kind of stuck.
You don't know what to do.
So you're going to have to do lots and lots of experiments.
But experiments are expensive.
Everything takes time.
Everything is going to need culling at some point in time.
But you need a certain amount of time that isn't going to end up going into the game-- this chunk.
Actually, it's agood chunk.
Which is why we've been trying to teach you how to prototype really, really quickly.
It's a little bit of [INAUDIBLE].
Because we want you to try to be able to get on to experiments, come to a conclusion about whether that idea was good or not as fast as possible, so that you kind of reduce the cost of doing a single experiment.
And yes.
It's true that you're not going to be able to conduct an experiment on every single idea that you've got.
So you have to kind of pick and choose a select number of ideas to be able to take the experiment to the experimental stage.
Now, there are a couple of things that you can look out for to say, maybe we don't want to conduct an experiment on something.
One is a brittle idea.
And a brittle idea is that you kind of have to kind of twist it really, really, really hard to be able to make it fit with any idea that you've already got.
I'm assuming that you've already got a concept that your team was kind of building around this concept.
Somebody comes up with an idea and a huge amount of twisting and turning to be able to even make it fit close, even though you think that on its own, this idea is a really good idea.
You might actually want to see how many different ways could this particular idea be executed before you actually decide on the way-- on an experiment to conduct on that idea.
Two ways that you can do that.
One is that you just specify, what is the goal of this possibly brittle idea?
It takes a lot of twisting.
Why do we need this idea in the first place?
Is it because the challenge in the game is too low?
Is it because players have too much information or not enough information about what they're supposed to do?
Is it because you want to introduce a little element of chance or maybe give some sort of strategic decision making?
Try to identify, what is this problem that you're trying to solve in the first place with this kind of unwieldy idea?
All right?
And once you've been able to identify that, then you come up with, all right, what are all the different ways that we could address that problem.
As opposed to, here's this one solution that's kind of unwieldy, that might very well solve that problem, but maybe there are alternatives.
Has anyone encountered this in your own prototyping?
Maybe in this assignment, maybe in a previous assignment.
Like, somebody had an idea that was just kind of hard to use, but then what the idea was trying to solve was actually a good problem to solve?
No?
Not yet?
I thought I saw some nods.
If you've encountered it, I'd love to get an example from one of you.
I can throw out examples.
But I'd love to be able to hear examples from one of you.
No?
I guess in our first game, we realized a couple problems with the game.
So we came up with various ways of trying to resolve it, that didn't really work for the game itself, but solved the smaller problem.
PROFESSOR 1: Can you be more specific about what the problem was?
Yeah, I'm trying to think back to specific ones.
One of them was that the way our game was with the set of the tic tac toe, it was really hard to see what sets were available on the board.
PROFESSOR 1: Right.
This was the set assembly game.
Sure
Yeah.
And so one of the things we tried to do was removing the center tile altogether, so that you could only play around the edges.
Which did solve the problem of, now you could really see what everything was, so it was a lot easier in that sense.
Except it was just a lot less interesting of a game, because there was a lot less strategy to it.
PROFESSOR 1: So you had a version of the game where all the squares were in play, including the center tile, and it just made it confusing for your players.
And then you have a version of the game where there's just, like-- this one possible solution to this problem is just take the center tile out.
But that made it trivial, or-- yeah
So we tried to find a middle ground.
And that's how we ended up with the idea where we had one single card in the middle.
Because you could still use it, but you didn't have to worry about the lines of play going through the middle, and how incredibly complex it is to try and reason about that.
PROFESSOR 1: OK.
So it sounds like, instead of a brittle idea, you actually had a good, robust idea.
Which is, that center square is really problematic.
It has hope-- it has possibilities.
It has really interesting gameplay possibilities.
But for learning the game and starting up the game, it's problematic.
So you found a way to say, all right.
How do we deal with the center square, instead of the solution of just delete the center square.
You know, let's find a different way to work it in the game.
And your final game actually does that.
Something else that you can do is, if you've got an idea for your game that's kind of unwieldy, and you're not quite sure what to do about it-- discuss it among your team, but don't experiment on it.
Don't work it into your rules.
Don't try to implement it.
Just get into everybody's heads.
Because that idea may come bubbling up later.
And little bits of it might be applicable and slightly easier to implement as you make progress in your game.
So in other words, put it on the back burner.
But make sure you have actually had a chance to discuss it, so that everyone's kind of thinking about it, even if it's not something that you've decided that you actually want to do right away.
It might be a lot faster to just have a quick discussion in that case than to conduct the experiment.
Yep.
So those are a couple of strategies on how do you sort of focus your experimenting time.
And this is probably not going to be very useful for assignment two, but it is going to be useful for assignment three.
Because in assignment two, hopefully you're well past the creating and adding new features part and now you're just trying to get this out of the door for Wednesday.
So let's talk about this stage now.
Well, yeah.
In fact, I'm going to focus on this one instead.
The sort of, like, cutting phase.
Where you're both cutting good stuff and bad stuff.
But you're cutting stuff in general, because you have to.
Maybe because you're running out of time.
Maybe because you are trying to get rid of the bad stuff.
So, things that you need to do-- and this is something that I'd like you to do today, when you meet up in your teams-- is to determine what is the criteria for which you're going to cut a feature.
For instance, come up with a quick description about what your game is supposed to be all about.
You may have already written this in your rules.
It might be in the very first paragraph of your rules, saying that our game is about getting entangled with other people, you know, or pushing them over, for instance.
Which is it?
And being able to identify it.
And then, when you look at all of your rules in your game, you can actually determine, OK, which chunks of our mechanics actually serve that goal, and which ones don't.
You know, there are things that are there just to keep your game going.
Those may be fine to keep, but those aren't directly contributing to how good your game is.
Those are just basically there to keep the thing working.
And maybe they can take a little bit of improvement, but they're kind of suspect.
And there's things that'll directly contribute to your core gameplay that you're trying to achieve with your game.
And those are probably things that you'll want to spend a lot more time making sure that the rules are absolutely clear, and your players understand what they mean, that if there's any other mechanics in your game, they're supporting that-- you know, that those meet your criteria for what should stay in your game.
Because once you know what your criteria is, it becomes a lot easier to be able to just look at everything that you've got, deciding what to keep and invest more time in, and what to cut.
Now, there's a couple of problems that people typically run into when it comes to cutting stuff out.
A couple of game developers refer to this as culling.
If you come from 3D graphics, culling is a very common way just to be able to just cut all the stuff out that you don't need.
So, for instance, if you don't have an explicit culling criteria, then you don't actually know why a certain mechanic exists, and you don't actually know whether you should keep it or not.
So that discussion should be something that happens today.
In the end, what is your game about, and does your game actually meet that?
You might have other culling criteria that aren't that.
Does this game run in five minutes.
What was the limit for this one?
10 minutes, I believe?
PROFESSOR 2: 20.
PROFESSOR 1: 20 minutes?
OK.
So if the game needs to run in 20 minutes, because we as instructors gave you that limitation, then that's probably a perfectly reasonable culling criteria for you to use.
What in our game is sort of contributing towards this, and what's actually getting in our way of hitting that target.
Let's see.
Something else that comes into play, and that affects culling, that affects this phase, is when you conduct experiments here, but your experiments are not very tangible.
They're kind of floaty and wavy.
This usually happens in teams that like to talk out all of the ideas, but don't actually like to do a lot of hardcore prototyping and playtesting.
I hope that that's not so much of a problem for anyone in this class by now.
You've got in a lot of practice in that.
But you may have a couple of rules and mechanics that have ended up in your game, that you never actually prototyped, that you've never actually tested out with real people.
Make sure that whatever actually ends up in the version that you hand in on Wednesday is something that you've actually tested-- is something that you've actually gotten some real data about whether it's working or not.
This one happens with every student project.
Students-- game developers in general always assume that adding more stuff is better.
It takes a very, very, very seasoned game designer to realize that sometimes simplicity is good.
And again, I hope this is not a terribly new revelation to the people in this class.
But I see it in student projects all the time.
I see in student projects where they assume that more features and more complexity is going to be better.
And in fact, I saw some of them in assignment one.
And we're going to be talking a little bit about that.
More complexity doesn't necessarily always make for a better game.
It may make it a more interesting intellectual problem.
It doesn't necessarily always make it into a more entertaining game.
Let's see.
One thing about culling is that cutting features and cutting stuff out kind of takes practice.
You've got to do it fairly regularly.
For assignment three, which is going to be starting on Wednesday, right-- we actually cut that on Wednesday-- what I'm going to ask you to do is try cutting a feature every week.
You know, just say, what's on the chopping block this week?
Every Monday, or every Wednesday, or if you have a regular meeting time on Saturday or something like that-- just, like, what is the least useful thing that we've got in our game right now.
And just try that.
Because you can always add stuff in later.
Remember, this is a cyclical thing.
You're going to be adding, then cutting, adding and cutting, adding and cutting.
I think you've got in a lot of practice on the adding part.
But you've got to practice a little bit more on the cutting part now.
And you've got time to be able to do this for assignment three.
You've got a little bit more time on assignment three than you had on assignment two.
So, start setting yourself a schedule, of all right, we're definitely going to cut something.
I don't know what it is, but we're going to meet up as a team and we're going to decide that.
And if it turns out that it was the wrong thing to cut, you can add it back later.
But that allows you to focus your efforts on the things that you've deemed as more important for a short amount of time.
It can be hard to be objective about this process.
Something that we've done-- in mostly our video game development, but you can do this in paper prototypes-- is to hand people very, very simple survey forms that basically say, how fun did you find this?
How fun did you find that?
Say your game has different roles.
You have a tinker, tailor, soldier, spy.
These are your four roles in your game.
And you just hand out the survey form-- how fun did you find being a tinker?
How fun do you find being a tailor?
Just one to five.
And that isn't terribly much information, but it gives you just enough information to help you concentrate your efforts on what's the most problematic thing in our game right now.
You can go into greater details, like do you think that the game was too long, too short, too challenging, too easy, too luck-based, required too much thought.
You can put these things on these five-point scales to be able to give you a little bit of feedback.
They don't necessarily tell you what the problem is.
But it tells you where the problem might lie.
And that will help focus your discussion and your efforts a little bit.
Yeah.
And finally, the most important thing is that you've got to keep doing this over and over and over again.
If you just do this once, you kind of end up in this situation where you've got a bunch of good ideas but the bad ideas are still there.
I'm going to expand this for you.
What you eventually want to get to is a situation where the only stuff that's left in your game is the stuff that is worth keeping and that works well.
And all the stuff that was bad, you've either completely cut it out or you've improved it to the point where it is actually a desirable feature.
And you're only going to be able to get through that through multiple iterations.
You're not going to get through that just by introducing something at the last minute and hoping that it works.
Because you haven't had time to actually improve it and polish it yet.
Now something that's less of a problem for this class, but may be a problem in your other game design experiments outside of class, is that you may not know when to call it a day.
You can repeat this process forever.
And you've heard about games that have been going on for four or five years.
How long was Duke Nukem Forever in production for-- 15?
At least 10.
And that was actually one example of where new technology just ended up changing the foundation of which the game was built on, over and over and over again.
It just kept resetting the game engine development and just took forever.
One thing that that game could probably have used is just some sort of standard criteria of saying, once we hit this criteria, we're shipping this game.
It could be, once we've spent x amount of money.
That probably should've been the case for that particular game.
But it could have been, once our informal internal surveys hit a rating of four out of five.
You know, that might have been good enough.
Some companies don't settle for that.
There are companies that say, we're only going to release the absolute best games that we absolutely can.
But, don't kid yourself.
Internally, they do have criteria on when these games have to ship.
Even Nintendo does.
And Nintendo's notorious for shipping late games.
Because they have this mantra that a game is late for a short while and a bad game is bad forever.
So that's what they say to the public.
Internally, they ship by Christmas.
They get those games out to the market.
Because they have a time box.
That's the other way to do it.
And so it's like, well, we have to be able to get into the manufacturing process to print those discs-- less of an issue nowadays.
But Nintendo still operates in disc fashion because they grew up as cartridge manufacturers.
So they actually had to print it.
Now when it comes to publishing a paper game, that timeline can be even longer, right?
Because you have to put ink on paper, you have to have it folded, you have to get all the plastic and stuff put in plastic bags, and put it into the box, and then put into pallets, and then shipped from China, usually, over to stores all over the United States so that they're in stores in time for Christmas.
And that's, like, a huge timeline.
So that became the time limit.
Sometimes you just set the time limit because you have to be able to meet ship dates.
It's not very complicated.
Now, when you're working on your own internal Kickstarter project or something, where you already know who's going to buy your game, because they paid for it upfront, before you even started manufacturing-- I think we'll hear more about that maybe in about two weeks, or about one week.
April 9?
PROFESSOR 2: 9th.
PROFESSOR 1: Yeah.
We'll have a couple of folks who actually work in the board game publishing industry talk a little bit about how they set goalposts and deadlines for themselves, what are the considerations for that.
But in the end, it's just-- they had to set a goal, somehow.
Either it's a quality goal or it's a time goal.
And once you hit that, you have to ship.
Because you can always do-- if you have set a quality goal, you can always do better than that.
The more time that you spend, theoretically, if you are sticking to this, the higher quality your game is going to get.
But then you don't ship.
You just have diminishing returns.
You'll never be able to move on to your next project.
You're never going to get revenue from the project that you started.
And your game may not be improved by that much more, just by spending extra time on it.
OK.
So again, we're going to talk a little bit about things that are relevant to this from what we've observed from assignment one.
And then for your team time today, start talking about, what is your culling criteria.
What are you trying to achieve-- what's the simplest thing that you're trying to achieve with this game, and the simplest way to describe it.
And then look at all the things in your game based on that criteria.
And whatever doesn't meet that might be something that you want to cut out, especially if it's already causing you problems.
And maybe you'll just save some time-- free up a little bit more time to work on the stuff that's already working, and you can polish it up a little bit more.
So any questions about adding and cutting features?
OK.
PROFESSOR 2: So, taking all this and relating it to assignment one-- and then going into assignment two-- also starting with when you cut in assignment one.
So, when it comes to advanced rules and basics rules, if you're going to do an advance and a basic rule set, make sure your basic rule set matches the core experience you're trying to make with the game.
It's really, really important for assignment two, for this third assignment.
Because that's what we said when set out, right, is we said, come up with something that's going to have a very particular experience that you want the players have at the end.
So if you're going to do advanced rules and basic rules, that criteria should be part of what your basic rules are.
The advanced rules, as Philip mentioned yesterday, for some of them, it could add more interest.
It could be more intellectually stimulating.
But really, the basic rules should support everything the game has.
The advanced rules are just add-ons, actually.
If you don't want to send us advanced rules, don't just cut it completely.
Say, we're not going to use the advanced rules because we just could not get them to work in time.
And that's absolutely, perfectly fine, so long as the basic rules still match the core experience you're trying to get across.
PROFESSOR 1: I think one thing I do want to add to that is that for the most part, your basic rules are pretty darn good.
Much better than a lot of the advanced rules.
Maybe because the wording in the basic rules had a little bit more care.
Maybe you tested it.
We found the basic games very playable and pretty enjoyable, was probably just fine for what we were expecting from assignment one.
So a lot of the stuff that you had in advanced rules, I do realize, some of those were things that came up in the testing process, and you thought, you know, this is a game that our game has evolved into.
But once you realize that there is a core, basic game that works, and then that this sounds like a slightly more complicated version of the game, that slightly more complicated stuff is all candidates for cutting.
Because you've already identified what the important part of this [INAUDIBLE].
PROFESSOR 2: When it comes to time and your deadlines, of course, adopt a common lingo.
Adopt a common language.
Stick with it.
Proofread your rules.
I think I have it written down in the very bottom.
Proofread your rules.
So I'll say that a third time later on.
And that means, like, naming conventions for the pieces, for what you've chosen to call things in your game.
That could be naming conventions based on the rules, but also proofreading your cards, too.
Making sure all your pieces are using the same language.
When it comes to talking about what the player does, we did notice some should versus may versus must, kind of getting mixed up.
So when we're playing the game, we're not quite sure, well, if I should do this, if I may do this, why-- if it's may here, is it also may over here, or is it should over here?
Just wasn't quite sure.
Wasn't quite clear.
So I definitely recommend spending today working on your rules.
Spend today having us play your game using your current rules, and then taking our feedback and then working on it for turning it in on Wednesday.
And again, when it comes to cutting things-- when you are cutting, making sure that you are filling in the gaps.
So if you cut something out, there might be a system, there might be a rule-- in this case, it generally was polish issues.
So if something was removed from one of the rule sets, we saw a little bit of-- PROFESSOR 1: Remnants.
PROFESSOR 2: --remnants of it, spread in the rules.
PROFESSOR 1: So for instance, say you are one of those groups that had basic rules and advanced rules.
And you took a chunk of the game that you moved into advanced rules because you realized that the core-based game was more playable without that.
But your basic rules still refer to bits of the advanced rules.
And that gets really confusing.
Because advanced rules are a completely different section of your rules that we haven't even gotten into yet.
But we will.
So do that cleanup work.
PROFESSOR 2: For setup-- so if you're going to have a setup section before your rules-- it's actually highly recommended-- have a setup section that is split out that is different from your rules section, so we know exactly how to set up the game.
Include diagrams and photos.
Where do players sit?
What is the orientation of cards?
If you have a card that has all this information on it, what is that common information across all cards?
If you think about a Magic the Gathering card, and all the different icons-- hopefully your cards aren't as complex-- but you'll have the same thing in the same place on all the cards, so you can then tell us what that thing is, and why it's useful, and how to use it as you're playing the game.
And in particular, let your game teach us how to play, if possible.
If you've got time.
If not, it should still be in the rules.
What I mean by let your game teach us is, if you're going to have a board, and the board has a lot of empty space on it, you can put some your rules into the board.
You can put some of the setup features into the board, if you've got time right now.
Proofread your rules.
And turn in as many copies of your rules as you have players in your game.
So, the assignment did say, it's a set number of two, three, or four players for your game.
If it's a four-player game, give us four copies of your rules.
That's all the basic stuff.
Yeah.
And then we'll be getting your feedback, specific feedback for each individual game, tomorrow.
Yeah.
Yeah.
Tomorrow.
I'm not sure what time tomorrow.
But we'll get you that before you're actually turning in.
That's it for me.
PROFESSOR 1: I just want to clarify one thing about should, may, must.
I'm not sure, because I'm trying to think of a more concrete example.
So it's like, if in your rules, you say, the player should move his or her piece from one square to another, we're not quite sure whether you're describing this as a good strategy-- as something that the player should generally be doing because it's going to put the player ahead-- or something that the player must be doing because the rules require the player to do this thing.
Should is a very, very difficult word to figure out for us.
And if you're to begin--
Would you suggest not using the word should?
PROFESSOR 1: It's possible.
Sometimes context explains it for you.
But it does muddy things more often than it helps.
I find should makes perfect sense if you're going to give an example.
If you're going to give an example, make it [INAUDIBLE].
Make it obvious.
Or even, like-- PROFESSOR 2: Put it in a sidebar, or-- PROFESSOR 1: Right.
Do indents, or a sidebar, or something, to make it clear that this is not actually part of the rules.
This is an example of how this set of the rules plays out.
And then you can use words like should.
Right?
You know, player one should do this.
Because you're describing a strategy.
You're not saying that the rules require this to happen.
Must-- say must when it's must, if a player must do something.
And when you say a player may, be very, very clear of what else the player may do.
Say, for instance, it's one out of five different options.
You say, the player may do one of these five options-- bullet point, bullet point, bullet point, bullet point.
And the last bullet point is probably, like, do nothing.
If a game allows something.
PROFESSOR 2: Bullet points are really, really useful if there's any kind of option.
Even if it's just, like, a player may do this or this, break up the or in its own line.
Give it bold.
Put a couple lines around it.
Put the next thing that they can do-- the other thing they can do-- after it.
Make it really, really clear it's one or the other, not both.
PROFESSOR 1: Yeah.
Yeah.
Yeah, that goes the same for not just your printed rule sheet, but also cards, stuff on the board.
Because sometimes there are games where you have options on the card, right?
And you may do this or you may do that.
It's not entirely obvious if, say, the card does both of those things, or you get to choose from one of those things, and who gets to choose one of those two things.
So that's something to keep in mind.
You only use numbered bullet points-- 1, 2, 3, 4-- if something's going to be sequential.
It's good for things like, here are the steps of a round of game play-- first you do this, then you do this, then do-- do, do, do.
Don't say the player may do one of the following-- 1, dot, 2, dot, 3, dot-- it's like, uh, now I'm not quite sure what's going on.
OK?
PROFESSOR 2: Yeah.
Yeah, and actually, this brought up rounds, stages, phases, turns.
PROFESSOR 1: That's the vocabulary thing again.
Yeah.
PROFESSOR 2: Choose one.
Stick with it.
Proofread it.
Make sure you chose it.
Make sure it works.
Especially if you're going to make any changes today, and you're turning it in in two days, have somebody who is not you proofread it and make sure you're using the same language throughout.
PROFESSOR 1: Rounds usually involves everybody around the table doing one thing in sequence, the same sort of thing, in sequence.
Steps are really big.
Phases-- it's usually describing large chunks of game.
you know, things like opening [INAUDIBLE], mid-game, end-game phases, and usually not used in rules.
But I know that some games-- PROFESSOR 2: Connect those games.
PROFESSOR 1: Yeah.
Use those
Sometimes they use other terms.
PROFESSOR 1: I know This is why it's maddening.
Because it gets used in both contexts
So what would you use to describe the individual pieces of a turn?
PROFESSOR 1: Steps.
Especially if it's a sequential sequence of things.
Like, this is my turn, in a round, and here are the three steps I get to do.
Step 1-- draw a card.
Step 2-- choose a card from my hand.
Step 3-- play it.
Right?
You know, things like that.
That sort of works.
Phases-- I've also heard it described in things like there is round one is one kind of action, and round two is like a completely different kind of action, and round three is [INAUDIBLE] like phases.
And it was like, oh my god.
That's-- [INAUDIBLE] that, actually.
Yeah
In [INAUDIBLE], there are turns in phases within rounds in phases.
PROFESSOR 1: Yes.
This is why the game is actually so much harder to learn than it is actually play.
To play is actually not that hard.
But when they desc--
The phases don't matter, but the turns and everything else was actually really, really good.
PROFESSOR 1: Yeah.
So that's why phases complicate things.
Sometimes it is, in fact, the right word, but it's just hard to be able to expect that every single player is going to interpret that word the same way.
OK.
PROFESSOR 2: So you can break up into your teams, continue working on your games.
This is playtest time.
So, in particular, playtest your rules.
If you want us to play your rules and give you feedback before you turn it in, today's the day to do it.
PROFESSOR 1: And attendance sheet-- PROFESSOR 2: Attendance sheet's right there.
PROFESSOR 1: Attendance sheet's over there.
[SIDE CONVERSATIONS]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, I'm going to start handing out assignment three.
And on the back of assignment three are the changes that we've made to the syllabus, which is mostly in the order of reading, but also we've lifted some of the games that-- we've [INAUDIBLE] some of the games that that we're going to [INAUDIBLE].
Because basically we change the assignment three in the middle of the semester.
Who actually read ahead to figure out what assignments are you on?
[INAUDIBLE]
How long ago did you read ahead?
Like two weeks ago
Two weeks ago.
OK.
So whatever was online is different from what it is now

But if you just read like the one line description--
Yeah
It's that.
But all of the text that we wrote up completely changed.
So this is [INAUDIBLE]
[INAUDIBLE]
What?
Are there any [INAUDIBLE]?
It's mostly rearranged.
Mostly just scheduling, rather than the actual readings.
I don't think we introduced any new reading, except for maybe [?
MO ?]
7, which is [INAUDIBLE].
That's been on our website for a while now though
I did all the reading
Yep.
So I don't know if you've seen that one.
[INAUDIBLE] Simulation 101.
It's like a series of webpages.
It's all online on the--
Yeah
OK. All right.
So you better read it.
So I'm just reading this because people are still cuttting things up.
So I figure I might just talk.
So assignment three is going to be about applying everything that you've learned so far.
Remember, assignment one was kind of just getting used to this, the discipline iterating.
Of prototyping, of iterating.
I feel that you've all shown like a lot of good experience and skill in doing this.
One thing while I saw those presentations it reminded me that we actually have notes about all of your individual write ups, that I should copy and paste into the grading section of the [?
fellow ?]
webpage
I'll do that next
Oh great.
[INTERPOSING VOICES]
That'll be some help.
Yeah, I just forgot to do that.
So you'll also be getting a little bit more detailed feedback on your individual writeups.
Like, in terms of the game, designing things are coming along pretty well.
And if you want more feedback on the individual elements of each game that you worked on in the past we could totally get that to you.
I'm just guessing a lot of you are probably more eager to just move on to the next project.
So I can talk about that.
Remember, assignment one was about the skill of prototyping and iterrating.
Assignment two was about trying to come up with, on some part holistic aesthetic experience.
And assignment three we're asking you to apply that to the end of trying to depict the perspective of somebody who lived all this in the real world.
That person could be a very specific person, like I'm going to bring up some examples here.
Campaign Manager 2008 specifically states that you are the campaign managers of either the McCain or the Obama campaigns, that you are-- that's one person.
Now, the powers of what you have in the game is really a whole network of campaigners, but you're the person who is calling the shots.
You are trying to win the electoral college though this.
We probably don't have time to play this, but anyone who wants to take a look at what's in here, all the bits, the way how each state is represented, how the United States is represented, the fact that you're basically just trying to win points among demographics by raising hot button issues.
That's all this game is about, [?
and it's ?]
putting you in the shoes of a very specific person who actually exists in the real life.
An alternative will be something like Tulipmania 1637, which is a game about-- anyone want to hear about the Great Tulip Crash of 1637 in the Netherlands?
[LAUGHING]
[INAUDIBLE]?
One of the first bubble markets-- who [INAUDIBLE] bubble markets?
You've got to have heard about it in the past four years-- eight years.
2008 was a real long time ago.
Oh, geeze
[LAUGHING]
It's OK. Next one's coming up next year, so it's cool
Oh, yeah, I read those articles, too.
So this is about prices of tulips, flowers, being driven up by speculation to horrendous amounts and trying to be smart and exiting at the right time and often not being quite pulling it off at the right time and losing your shirt in the process.
And this game puts you in the shoes of a [INAUDIBLE] speculator, someone who is trying to play this market at that time, when the market is behaving in this very, very volatile, incredible increases and yet even faster crashes.
[?
It's ?]
actually designed by a professor in Syracuse University, Scott Nicholson, who runs [INAUDIBLE] with Scott, and it's pretty awesome.
If you like playing it, you can see the little token, little flower heads, and you get to do things like try to convince buyers to buy at a higher price than what you had originally bought it at.
Things like that.
[?
Care to talk ?]
a little bit about E3?
Yeah.
So I chose three games.
So what we're asking you to do is choose a real world historical, political, cultural, economic situation, so you might be choosing some problematic situations like the conquest of Africa by Europeans.
So it's a game about African exploration during the 19th Century.
It's made in the 70s.
They tried to be less problematic.
They were not entirely successful because the point of view of this game is entirely of the European explorer entering this forbidden Africa and finding things there, and basically finding Zulu tribes, which were not all throughout Africa, of course, and fighting other explorers who had died there previously, biblical things like King Solomon's mine, things like that.
But mechanically, it is an interesting way to talk about exploration, not so much about the bigger issues around what imperialism was, what that kind of conquest was.
If you're thinking about doing a live action game, we've got a couple of different versions of how to do live action games.
It can be a live action playing game.
It can be a party game, a conversation game.
This is a book called Heads of State: Nine Short Games About Tyrants.
We've got-- I haven't played many of these yet, but we've got, basically, the first game here it is called Coup D'etat
Basically, why are dictators what they are?
How did they become a dictator?
In this case, Gaddafi's idolization of Egyptian president Gamal Nasser, reading Nasser's philosophy of the revolution, thinking about plots against the Egyptian monarchy, but also entering into the Military Academy in Benghazi in 1963, and then having other officers in training with him organizing the group that's going to overthrow the pro-western Libyan monarchy.
Basically your dictators in school, learning how to be future leaders in one of these games.
So what this book is trying to do is try to give you multiple different viewpoints of tyranny and of those kind of problematic things.
There's communism in here.
There's maoism in here.
A game about the disappearances of people, so if you're thinking about making a game about Argentina in the '70s, good luck, but they tried it, so you can see what they did.
And then another version is Dog Eat Dog.
It's a game about imperialism, in particular the assimilation of the Pacific Islands.
The great thing about this game is you've got people who are acting as the new colonizers, but you've also got people who are acting as the native populace, and the different interactions that happen there.
Basically, if you decide you want to do something that's coming to-- trying to do something more serious, if you try to do something with more of a problematic tone, you might want to consider having multiple different perspectives, or saying a similar perspective, but at least understanding what the other different perspectives are of the people who are inside of that system.
It's just good to take account of--
I think if every player is taking on the perspective of a single person, that's OK, even if different players in the same game are taking different perspectives.
I think that might complicate things a little bit because, again, you end up in the situation where you're designing basically two very different kinds of games that are interacting, but some of you have had experiences with it and made it work.
So it's certainly not out of the question.
What we don't want is a situation where you are simultaneously two people, right?
It's like I am controlling the survivors and the zombies simultaneously.
That's a little bit weird.
Yep!
Can we use a circle persona who has multiple personalities?
[INAUDIBLE]
That is-- We can talk about that, but good luck
So if it's really about the [INAUDIBLE] interesting way of pulling it out?
Maybe there are multiple players playing those individual personalities?
I think the biggest focus is the system, right?
It should be less about the individual person, because I get the perspective of that person, but more about the conditions in which that person lived in
Yes
[INAUDIBLE] game before where you could be the president of multiple corporations.
And like, in one corporation you'd just like embezzle all their money [INAUDIBLE]
That sounds like [INAUDIBLE]
Yeah, and that's where ostensibly on a lot of the [INAUDIBLE] games, your actual role is one of a robber baron, of a capitalist who lived in the 1800s, who owns shares in multiple companies.
You're not really the president of a single company, but you may take on that role somewhere in the middle of the game, but your role is not defined by being president.
Your role is being defined as being a person with a lot of money in the 1800s, and trying to increase that wealth through railroad systems
Can it be satirical?
Yes
Absolutely.
Just because you may take your game to make a game about a serious topic doesn't necessarily mean your game needs to be dull or somber and I've got some examples of that.
So how many of you play Crunch?
[INAUDIBLE] Nope?
Two-person card game.
You are a CEO of a bank.
You are trying to get out of the bank with as large a golden parachute as you possibly can.
To hell with the health of the bank itself.
This game encourages you to take advantage of the government's generosity in bailing you out.
It encourages you to hide cards on your body to secret fund your own personal wealth, to be able to take money out of your own bank and put it into your own personal accounts, so that it becomes your fortune rather than the banks fortune.
To make really risky loans that may never pay out.
It came out in 2009, so you can understand why this game was made.
This company, Terrible Games, basically makes satirical games that comment on real world situations, but this one is very, very clearly, you are a banker.
Well, the satirical version vision of a banker.
They have another game called the War on Terror, which I did not bring up because it's a little bit unclear who you are in that game.
You're supposedly a world power, but are you the President?
Are you the government?
Sort of like all of the government at once.
Or are you some sort of media construction of what that country is because you can get branded as evil, for instance.
[INAUDIBLE] evil in it.
But then that doesn't actually change your role in the game, which is I think part of the point of game.
The point is that whether you are a terrorist state or not you kind of all engage in the same sort of activities anyway.
Just being branded as evil just let's you do it with a little bit more impunity.
Anyway that game's not a great example of what we're looking for because who you are in the game is not quite well defined
Are these games usually a set number of players or do they have [INAUDIBLE] range like 2 to 4, or is it [INAUDIBLE]?
This is a two-player game.
Specifically, Campaign Manager 2008, again a two-player game.
Tulipmania has a range.
It really depends on your ability as a game designer to design for different people, but for this assignment in particular we're asking for two to four people, but a specific number.
So you can design a two-player game, a three-player game, or a four-player game, but raid you don't need to make the game playable by a range, and that means you can tune it very specifically for a fixed number of players.
The gentlemen of the South Sandwich Islands is an absurd game of logic discovered in 1821, which is not true at all.
It was really kick-started as a project about three years ago.
Four years ago, maybe?
Yeah.
This is interesting because this is probably right at the edge of what I would consider acceptable for classical.
This is a game about sort of fantastical courtship where you are trying-- where you are two gentlemen trying to win the affection of ladies who are walking around this island, but and also there are-- there-- what's the word?
the handlers
What do you call it?
[INAUDIBLE] chaperone
Chaperone.
Yes.
But the chaperones are also following them around here trying to get them alone, so that they can have a quiet talk with them.
Kind of the game of Jane Austen.
Now Jane Austen, of course, was writing about some of the social situations, but every single character that she wrote about was fictional.
But, the world that she was writing about is at least plausible.
And it's based on [INAUDIBLE].
You want to look at something and make the game set in Dickens' Oliver Twist or something like that, you can.
You could be individual people in that setting.
Do pay some mind onto how grounded in reality those things were.
The more grounded they are in reality, the more different sources you can pull out for inspiration.
If it's just like, well this was just the fantastical invention of one person in this game or something like that, then you'll only have Scott Card's writing as a reference.
Whereas with Jane Austen, you can actually look at real historical stuff to be able to get more ideas about how that social system works and actually Jane Austen's books are actually these social systems.
And they're incredibly complicated.
But this game is really just about walking across bridges and being on a very small island, so this is a sort of a holiday island or just trying to get quiet talks with ladies.
Yes?
[INAUDIBLE]
Yes, I don't want a game that takes more than four people to play definitely
For us to grade, it's hard to get more than four people
Yeah, you will be getting your grade in August if you give me a game that requires six people.
I can't get that many people in a room
You want your team to be no bigger than four or five still
It's easier to test [INAUDIBLE] at least [INAUDIBLE] the game.
Something to think about is agency.
Not just how does the world look like to that person.
Like for instance in Source of Denial you are colonial European power for the most part right?
And you're seeing the continent of Africa as this resource rich land to grab stuff out of.
That's the perspective.
But then what are the verbs?
What are the things you can do in that context?
In [INAUDIBLE], you are in communist Poland in the 1980s.
Poland?
Poland, right?
Yeah.
And the only thing that you can do is stand in line for rationed goods, but what you can do is you can affect your place in line by let's see carrying around a cute baby is one of them that are on here.
I believe you can knock somebody's hat off or you can change which line you're standing on so maybe instead of waiting for dry goods you are waiting for furniture.
That's our thing.
It's kind of neat because all the goods that we actually have in here are actually pictures of the actual rationed goods that were available in Poland in 1980s.
In fact this was a project that was funded by some Polish historical institute.
But the only agency you have in this game is, which line are you standing in and where in line are you standing because that's the perspective that they want to get across to you.
And so your whole game is all about jockeying for position.
And it's actually a pretty well-designed game.
We played it a few times.
That has both the perspective of this is what the world looks like.
The world looks like a bunch of lines for rationed goods to me as someone who lives in that world.
It's a kind of satirical lighthearted tone, but it's grounded in reality as it's the only thing I can do.
What can I do about this world that's going to give me some interesting decisions to make, right?
And it's going to be which line do I want to wait for?
[INAUDIBLE] again, historical-- I can't remember which decade this is in, though
1890s is it?
I think that sounds right.
Somewhere in the rules they mentioned it and it's about Manhattan.
It's about ruling New York where it says it's a game of backstabbing, corruption, temporary alliances and you're basically just running for mayor.
And if you can't get mayor, you're running for other city offices.
Turns out that actually as soon as you become mayor, you have a giant target painted on your back and everybody else who is in other positions, but you can become chief of police, precinct chairman.
You then abuse those powers to sort of place new immigrants in different worlds and change the demographic makeup of different worlds of different boroughs in New York to be able to change the way how they vote
[INAUDIBLE]
The object of the game is victory points for the most part.
But the victory points you can get it by becoming mayor, but then you're not staying there for long
[INAUDIBLE] machine and trying to control it?
And the agency that you have is really just you were able to coerce people to come to various locations and you're trying to get the vote out.
So [INAUDIBLE] were 14 people [INAUDIBLE] or doing things like having free food [INAUDIBLE] things like that
You could possibly evict people from boroughs and move them into other boroughs to create not ghettos, what am I thinking?
[INAUDIBLE]
[INAUDIBLE] the districts, right?
[INAUDIBLE]
Yeah, instead of changing the minds, they're actually moving people.
Yeah because you know if you're chief of police, you can just grab people and put them somewhere else.
You can throw them off New York.
So that's their view.
Yeah that's a perspective they are trying to place, this abuse-- they're asking-- they're saying that in this era if you're one of these people you are actively engaging in the abuse of power to sort of play with the lives of immigrants.
So, yeah, it turns out that it's just more of a game of territory.
When you actually play the game you are looking at Manhattan as a bunch of resources that get moving around, but it's a very specific perspective and I think the game is played over a period of 16 years, which is [INAUDIBLE] or something that.
So check it out when you get a chance.
Any questions about assignment three?
Now one little wrinkle is on April 16th, which is just two weeks from now we want you to come in here and give us a pitch presentation on what you've decided to do with your project and we want to see a concise description of how you expect it to be played at least two weeks into the project at what you decided so far.
Because [INAUDIBLE] you want to create so you want to tell us what kind of feel you want to get out, whose perspective this is.
What type of agency that you're likely to have in the game.
Very important to tell us where you are drawing your historical references from if it's a history game, where you're drawing information from.
If it's a contemporary game what are you using to inform this thing, your experience.
Now you could theoretically all make a game about being an MIT student at MIT in 2014.
I'm going to encourage you to think a little bit broader than that because you don't need any additional information.
And that's why it's kind of boring to get that degree.
You can draw inspiration from other games but also books, websites news articles, not just Wikipedia but Wikipedia can be a useful resource to find other stuff but we want you to give that presentation as if you were seeking green light as if you were going to a publisher saying, please give me money to be able to continue work on this game.
We're not going to fund you down we're not going to cancel your project but we are going to give you feedback both on how you can improve the project on what we've heard, also on your presentation skills.
That's going to be one of the few time in this class where we're talking specifically about how you deliver the presentation because you might have to do that one day and there are other classes especially [?
CMS 610, ?]
where you will get a lot of practice doing pitching so this is a little taste of that if you want to be able to get more practice I encourage you to look at that class.
everything else at the end of the semester you're going to do the same sort of a presentation you gave today.
We expect change logs.
We expect a one page write up.
The one big difference is that everything needs to be handed in at the same time.
Previously you could hand in your write up the following week, I believe that's the case for this assignment.
We can't do that for the final assignment in a MIT [INAUDIBLE] because of MIT course rules.
You can't hand in anything late and only the class can ask you to hand in anything later than that.
So everything is due on the 14th.
Finally, we have a bunch of guest lecturers that will be coming in.
This is starting to segue into what you do with the information that you're getting in this class after this class is over.
April 9th, exactly a week from now, will be the Game Makers Skill people who actually are a collection of people here in Cambridge who make board games and card games for retail, but have been funded through a range of different sources all the way from traditional publishing model to Kickstarter, and they all talk about their experiences and what it took to be able to take these ideas into something that people could buy off the shelf.
We also have another guest lecturer from The Geneva University of Art and Design.
These are mostly masters students and people who recently graduated from masters programs and are going to a design school that are thinking about game playing.
They're mostly tech designs, so they're digital games for the most part but because it's an art school they're actually pretty broad minded in what happens in the computer and what happens outside of the computer.
So that should be fairly interesting.
They're gonna talk a little bit about their work.
[INAUDIBLE] will be coming in May 7th.
He is a theorist.
He actually currently teaches at a number of places, including New York University.
He used to teach here at MIT.
He is probably best known among game scholars for writing his books on the Art of Failure, A Casual Revolution, [INAUDIBLE].
These are the three books that are about games and how-- not so much how designers think about games, but how to analyze games from sort of abstract perspective.
[INAUDIBLE] If you're interested in those books, please feel free to check out our library and game lab.
I'll be happy to let you have a look.
They're really, really easy reads because he believes in padding out his books with images.
He has told us this very specifically.
He was like, "Well, I could write about half as much and then just have larger images and then the book becomes bigger. "
He also makes games and he'll be here to not only talk about his current work and what he's working on right now, but also play test games on me, so I can give you some feedback, because he's taught here before, he's been part of this class a couple of years ago.
So please don't miss any of the guest lecturers.
They're great folks and give you an idea of what you could do if you want to go to design school, if you want to go to academia, if you wanted to make your own games and publish them, and make commercial product.
Just give you some perspective on what you can do.
All right who's games are actually ready at this point in time?
Everyone's ready?
Awesome.
Let's play those games.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so the plan today is that we're going to talk a little bit about the reading, which should feed directly into your thinking for the final assignment in this class.
We're going to play a couple of the games.
There should be just enough games for everyone to get through one round.
And the rest of the class today will be used to form teams, and then talk in your teams and try to figure out what project you're doing.
Actually, the other way around.
We're going to talk about projects, and then you're going to form teams around them.
So today's reading was-- in the title Simulation 101.
Right, literally 101.
It was very, very introductory.
Here's a bunch of terminology and a bunch of concepts about simulation.
Has anyone read that before, any other class?
Because I know it comes up in some other classes.
OK, so we have some sort of source reality.
Which I am going to put question marks because we live in a postmodern world.
And some sort of source reality where we are trying to create some sort of model.
And then we've got something that we thing of as a simplified abstracted version, which ought [INAUDIBLE] simulation.
Anyone remember what are some of the things that simulations do that-- well, actually, what are the alternatives to a simulation that the article was talking about?
How do we typically talk to people about [INAUDIBLE]?
How do we usually convey an idea of this is how reality seems your work?
What are some other ways That you do that?
You extract it
You can extract it.
Extractions are useful words.
I'm going to write it here.
That's-- you can do that to a bunch of different moves.
When Jane Austen tells you how life was like in-- when was that?
18th century?
19th century
OK. How was she doing it?
Metaphor
She tells a story.
She uses the process of narrative.
She does use some metaphors, but for the most part it's just very plain.
This is what happened to these people, and everyone is upset.
That was what the reading today was.
All right, you have this way of telling people about how something is.
One way that you do it is you could tell a story about it.
You could do a painting, a visual version of it.
You could create a little physical sculpture of it, if the concept is-- if that is a good way to convey a concept.
You can create a simulation of it.
He lumps together things that represent how the world works as a representation.
It's like a picture of a pipe is a representation of a pipe.
It's not the actual pipe.
But it is a visual representation.
And the narrative will be a sequence of events, a linear time representation of what happened.
But it doesn't tell you anything about how that version of reality works.
It doesn't-- it may give you a clue that you can intuit.
But It's not actually trying to show you how this thing behaves.
And what we trying to do in simulation, in the broad sense you can think of simulation as a kind of a representation.
But very specifically, a simulation is trying to tell you about how the system actually behaves.
So, it is looking at some sort of reality.
What are the kinds of things that you could be looking at?
Before you actually try to make some working simulation about how reality works, what are some of the things that you want to look at?
Let's be a bit more concrete about this.
Your reality is student debt.
OK, let's just say that is the system that we are looking at.
We are looking at the world of student debt.
And you are trying to create a model.
If you wanted to explain to somebody else, who doesn't really understand student debt, that these are all the things you've got to worry about.
What are the things that you would try to-- that you might consider explaining to them?
A high school student is thinking-- let's give this [INAUDIBLE].
The high school student who doesn't understand student debt but is thinking of [INAUDIBLE] college.
What would you tell them?
What are some things you want to tell them?
The situation [INAUDIBLE] under which student debt accrues.
What things [INAUDIBLE]
Break it up a little bit more.
What are some of the components of that?
Situations where you would take out loans
OK, situations.
Maybe cases, you know?
These are examples of things that they [INAUDIBLE].
OK, what else?
Eventually they're going to make games about this sort of thing.
So what you want to know [INAUDIBLE]
You want to tell them how [INAUDIBLE], and the how to pay it off eventually
OK, those are mechanics right?
Those are verbs.
How do I do that?
How to avoid having to take out loans.
Getting scholarships and things instead.
Trying to work around it
I would call those strategies.
These-- you have a range of different strategies, all of which to achieve the same goal.
But some of them are better than others, and some of them completely avoid.
What else?
I'd say that any sort of [INAUDIBLE] that usually happens.
[INAUDIBLE]
So probability of how often things happen
Also trends.
Student debt is increasing.
[INAUDIBLE] A big picture of what it's looking like
OK, so that's coming from a more historical point of view, right?
There were also two things that [INAUDIBLE].
Things to compare yourself against
The effects that having debt will have on the person or situation
Consequences?
Yeah
One of the things I would include would be things like variables.
What are the numbers that you want to keep in your head?
Easy monitoring that will tell you how the system works.
The situation, cases, verbs, strategies, that's probably something-- I don't have a good word for it-- to explain how one thing affects something else.
But that is like consequences.
Maybe flows.
Where does the money go?
What's the rate of money coming in from getting student a job?
Was it getting a [INAUDIBLE]?
Was it some other source of revenue?
[INAUDIBLE] and loads, that sort of thing.
And how does that affect how much money you have versus how much money you owe.
So you have got all these things that come into the model.
This other reality, you create some sort of model that you understand.
This is how you understand the world as it is.
In the team project, this is how your team understands the world as it operates.
And then you decide that you are going to make a situation out of it.
And you can choose what are the things that you want to bring over.
I'm not to go through that part of the exercise because they everyone will have a different answer.
But for some people certain variables are going to be more important than other variables, depending on the kind of simulation you make.
if want to play this game-- actually, I've got to add a rule system.
That's really what our assignment is about.
I you want to play this game as someone who is giving [INAUDIBLE], then the worlds, and the variables, and the cases that you are thinking up are going to be different.
They are going to be thinking about [INAUDIBLE] Whereas, if you are playing this same as a student, then you-- the simulation [INAUDIBLE] what it will look from the student.
Then what these sort [INAUDIBLE] won't matter that much to you.
You'll just care about yourself.
So the variables you care about will be different.
How those things affect your life are going to be different.
Now, there is a [INAUDIBLE] to abstraction.
And correct me if I'm wrong, but that's where [INAUDIBLE] put simulation the gap?
Yes
Yes .
That's this thing where not only do you decide that there's a bunch of things in here that you are going to choose not to bring over to you simulation, or choose to bring over to your simulation.
Then you can think of a way to simplify that.
Maybe we're not going to-- maybe certain kinds of consequences from interest from loans or whatever normally increase on some sort of non-linear scale.
But for the sake of making a simpler simulation we're going to assume that it is linear.
We are going to say that every time-- every turn the amount of debt that you have increases by a fixed amount, rather than a percentage or compounding interest.
Because simplifies things, but maybe you still get certain points across.
This thing down here, some people call a simulation gap.
Other people sometimes place it over here.
But I'm going to make the case that has to happen.
No matter what you do, you have to do this because you're never going to be able to fully simulate everything that you know, and the fact you don't know everything about reality anyway.
So this has to happen.
But there is a design choice that you make over here of what you want to simulate and to what degree you want to simulate it.
So, for instance, one of the games that we have today is Kolejka, which is, again, standing in queues, waiting for rationed goods to come in while living in 1980's communist Poland.
You have no idea when the next truck of goods is going to show up.
The reality Is that there is-- there are schedules.
Even in communist Poland, people actually-- there was a schedule in which objects were supposed to show up in certain stores.
I believe that there's actually an action that hints at that, but all it does is that it allows you to place an object in multiple stores at will.
And it's a special thing that you [INAUDIBLE].
But for the sake of the abstracted version of the game, which they made, they are going to say, when you-- as someone who was living at that time, yes, there were schedules.
It was possible to actually figure out when different goods were going to show to some reasonable level of accuracy.
But the majority of people who were involved in this system had no idea.
And so for the most part, it was random.
That the decision that they're making, and the point they're trying to get across.
Now whether that's true or not, that's something that the designers want to convey through design of the game.
That's something that you can also decide.
If there is that you want make across, a lot of the rhetorical points are going to be right here.
So if you want to say-- we had a game last year which was about people cross-dressing to get into the French theater.
Not all of them were cross-dressing, some of them were dressing up, some of them were dressing down for various reasons of social class and trying to get a cheaper ticket and being able to get into parts of the theater that you otherwise couldn't.
It wasn't explicitly a simulation, but they were trying to get across a point.
That this is a sort of thing that that you can do.
Everybody had a-- if you wanted to get into the theater, there were multiple ways you could get into the theater and there were different reasons for every single person.
So a noblewoman might cross-dress as a man in order to be able to get a cheaper ticket, and go slumming with the people buying the cheap tickets because it is a lot more exciting down there.
Somebody who is in the clergy will have to cross-dress to be able to buy a ticket as a woman because the clergy are not supposed to be going to the theater.
So they are hiding themselves for other reasons, even though they are in a more visible position.
And there is a lot of bartering, there is a lot of theory of [INAUDIBLE] and clothes and such.
It is a pretty fun game.
I'm going to grab that and bring it over.
So when we play the games today, I want you to think a little bit about the situation they are trying to respond.
The Gentlemen of the South Sandwiche Islands is a game about crossing bridges.
If you have ever done any classes here at MIT about bridge-crossing theories, that game is all about that.
But it is ostensibly a game about match-making, or finding a match.
And you are sort of chasing, where the men are chasing the women who are also being chased by their--
Shepherds
Shepherds, yes.
Really good.
So that's a very, very, true-- that giving the whole, actual, dialogue, where the gentleman and the ladies there.
It is just making it this whole traveling game.
And you could ask yourself, what are they trying to get across?
Whether intentional or not, it doesn't really matter.
But the game sends a message about how the system is supposed to work that you can take away as somebody who has played the game.
[INAUDIBLE], which is a game about a bubble economy.
And Crunch, which is also about a bubble economy, only a lot more recent.
And it is Crunch, in particular, where this [INAUDIBLE] is.
It is trying to be a satirical commentary about how big banks need to be operating, whether or not that's actually how they operate is kind of beside the point.
But they do have certain elements of simulation in there and then they just abstract them to make you focus on the things that they want you to focus on.
So, let's see.
Two of those games are two-player games, so that is only going to account for four people in this classroom.
The other two go up to five players, so we can probably split everybody else into groups of about four.
12 people, so eight.
That should make the games go at a pretty good clip.
Crunch and The Gentlemen of the South Sandwiche Islands are pretty fast compared to the other two games, so the people who are playing those games might want to switch games and play [INAUDIBLE] after that.
It will probably be a lot faster if the people who just competed a round then turns around and explains it to the next group because then they won't have to read the rules.
So let's do that for about an hour.
Especially because Kolejka takes about an hour.
And after that, we'll talk about class.
We'll talk about the assignment.
So, any questions about this?
No?
Make sure you talk with your teams, especially when you're looking at how the rules work, but also while you're playing the game.
I talked to both the team that played Tulipmania-- not the team, but the group that played Tulipmania and the group that played Kolejka.
I haven't had a chance to talk to the folks who played Crunch and The Gentlemen of the South Sandwiche Islands yet.
So let's talk a little bit about--
There are cards on my body that I would like to get rid of if we're done
I don't want to call it.
We're still in
I'm going to say the game is on pause.
But I do want to talk a little bit about, not so much how this one particular game is being played out, but rather what does this game render in very high detail?
And what does it sort of gloss over a very low-level granularity?
So what is very, very well rendered in this game?
In the opinion of the people who played this game?
I would say corruption.
I think there's a lot of-- for example, when you are sneaking a card, you kind of wonder if someone is watching you.
I think that's actually pretty similar in the real world, dealing with a corrupt CEO.
You wonder if someone is going back and checking your records, if you can actually get away with taking this money and storing it somewhere.
Right
I wouldn't quite agree, because in this game they can see the number of cards in your hand going down.
And as long as they don't see you actually putting them away, they are like, you are clearly embezzling, I can't do anything about it
So there is on one hand the not wanting to get caught in the act of embezzling.
On the other hand, the very clear assumption that everybody is.
You can see it, it is happening.
You know that it is going on, that everybody in the game is engaging in this, but you don't.
The only thing that you're worried about is just getting caught in the act.
Of having your hand in the kitty.
But it is OK that the kitty-- that the cookie jar seems to be emptier than normal.
The game does that well.
What's really sort of vague, deliberately vaguely represented in the game?
All the actual bank mechanics
Can you give me an example?
Like the whole investing on [INAUDIBLE] thing.
It's probably a pretty good simplified version of it for the game.
Which is fine, because that's not the main focus, it's not to make smart investment choices.
It's to embezzle things
The odd thing about it is that the actual operation of the banks is remarkably trivial
Right
Or it has been trivialized to the point of-- that's the point of this game, is not to run a good bank
I also don't know how this works in the real world, but the government bailouts in this game seem like candy, basically.
You just, oh I want a government bailout.
Oh, OK. Government bailout?
Sure, we need a government bailout like [INAUDIBLE] in the financial crisis, and all the banks were failing.
The auto manufacturers started asking the government for money.
[INAUDIBLE] so Ford decided they didn't need it, but GM did.
[INAUDIBLE]
Again, it's a deliberate choice, right?
I think what you're saying is that yeah, it is ridiculously trivial to get a government bailout.
It's clearly designed, to me, to make you think that yeah, the government is going to come in and take care of this.
Even though I'm in a hole of several-- $50 million dollars or something.
It is like, yeah, the government will take care of it.
So, I had a similar discussion with all of the other teams.
In Tulipmania and Kolejka, what are the things that are really in high resolution?
Everything that you can possibly do in standing in line is-- it is what that kind of game is about.
Whereas things like money, for instance, or, oddly enough the way how the black market played out in Kolejka was less in the mind of the group that was playing because everybody had stuff to do.
We didn't really have to stand in line but that might change with a smaller number of people.
And Tulipmania, what did we talk about, the stuff that was really high resolution?
Trying to encourage the buyers, or use the buyers to [INAUDIBLE]
Right.
So, getting other people, who have money.
Why even play the game?
These are not [INAUDIBLE] to put money down on the investment that you are telling them to make.
It is really what they whole game is about.
There is a lot of detail about how that works.
It is all about.
speculative activity.
Whereas things like the actual prices, and where the bubble decides to burst.
Those things are simplified to the point where the players can make a decision about it in a way that maybe real people can't make a decision.
OK, so hopefully that gives you ideas about the way-- how you can play around with the level of fidelity in your game.
And now we're already down to the last half hour in class, so let's talk a little about your assignment 3.
First of all, has anyone been thinking about this?
I know you guys want to get back to your game.
Has anyone got an idea about assignment 3 that you want to pitch to your teams-- to the class now, to be able to form a team now?
It's probably a good time to do it.
It doesn't need to be fully fleshed out
I have an idea for a board game based around the first triumvirate, where the players are on either [INAUDIBLE] And--
The Roman triumvirate
And basically, all the players just want to want to march on Rome and become dictator there.
But there's other players in the way.
And also the Roman establishment, the senate doesn't want a dictator.
And so you are trying to campaign to acquire more soldiers, and more influence around so that you can overcome them.
You are also pushing your luck and taking riskier plays, so you can do this back to the other players.
And, eventually, someone gets far ahead of them.
Or someone or something doesn't pay off and they get killed.
And then the whole thing [INAUDIBLE] and the players start fighting each other
So it's kind of tenuous cooperation
Yes
So it's an interesting-- certainly a lot, a wealth of material to look at, when it comes to the fall of the Roman republic.
Basically, if anyone points to people who-- the individual people caused the fall of the republic [INAUDIBLE].
Obviously, [INAUDIBLE].
That might be, if you are interested.
Or if you don't actually know that much about ancient Romans and would like to, that is probably a good project to get into.
I can't think of any better way to learn about a topic than to try to make a game about it.
Anybody else?
So this isn't a complete idea at all, and I'm probably coming about it from the wrong direction, but I would be very interested in finding some historical event which-- or some role where the person consistently felt that things were out of their control, and kept getting pushed in the same direction.
And then make it so that the game always does the same-- it always ends in the same way.
So it is a deterministic game, apart from how you get there.
So, make it so that, for example, there's two players.
One of them is guaranteed to be the winner.
So make it actually follow history, and then don't tell the people this.
Just have a game where eventually people realise that this is a game where, hey, this turned out remarkably like actual history.
And then they play it again, and it is still remarkably like actual history.
And then they eventually realise that you are playing through the motions
You could certainly make a game where the governing forces are very powerful, so it's going to be playing out in pretty much the same way each time.
But then, the question is, what is the agency that you have in there?
What does [INAUDIBLE] take to be able to influence things.
So even a game like [INAUDIBLE] is not a good example of this, but it's a game that's based on the events that are actually in a deck of cards.
And so what you do to play out, these are things that actually happened in real history and they may not be in the right order, but they will always happen.
So it ends up being what kinds of decisions you are to make in all of those circumstances.
[INAUDIBLE] slightly different parameters
Most operational-level war games do exactly that.
The game is not about getting the Germans to win or winning Vietnam, it's more about what are the various strategies you have to use at the time.
But more than likely, though, the game is set to end the way history does-- played out.
So they're really complicated games.
There's some less complicated ones, so if you end up going down that route, I might have some card-based versions of the game to show
You might also want to zoom in into the very-- instead of a wide swath of history, go really small
Most board games about battles will allow you to have a battle a different way.
But anything operational-level tend to be very much about you just exploring what is going to be in that
That's true.
So you've had your hand up forever, and then we'll go back this way
I have a separate idea, but just for that idea, maybe the Russian Revolution.
Everyone was just caught up in all this craziness.
It's like-- the game is basically forcing you that your revolution is going to lose and another one is going to come in.
But along the way you are trying to set up your perfect table
You could imagine a game where there's already this tidal wave of change happening.
You don't stop the wave, but how well can you ride that wave
But an idea I was thinking of was the dot-com bubble, back in the late '90s, where you would be some startup with a ridiculous business plan who is trying to acquire as many users as you can in a short amount of time.
Get an IPO, got a lot of money from VCs, and then get out before everything crashes.
So, I guess, about a bubble like Tulipmania is, but a little different in terms of the mechanics.
How do you deal with the bubble?
For that one, think about what the people who are going through that.
What do they actually know about the bubble.
Did they know that the bubble was going to happen or not.
At least for this assignment.
It's about being in the perspective of the person at that time, in that real time.
What about it is hindsight versus what they thought at the time
Yeah it's easy to make a game-- we have had games in this class that were also about the dot-com bubble A lot of it was making fun of how ridiculous it looks like now, in hindsight.
But it's good to try to think, what information did they have at the time?
It looks like-- did this look like it was going to go on forever?
I have three different ideas.
Most of what I think is I want to make a two-player game, where one person is in a position of power and the other person is working for them and otherwise subversive to them.
And the person in the position of power has the ability to [INAUDIBLE] the other person.
And what the other person can do is-- should they try to overthrow them?
Does this-- How does this describe their life?
Another idea I had was a love affair thing, a three-player game.
I'm not sure that a three-player with 12 pairs
Maybe you can identify a specific three people
Yes, maybe like Yoko and all the Beatles or something
Yeah, that would go
And then another one, [INAUDIBLE] A game about feminist actions, on how they're all not quite the same thing but they are arguably minor points.
[INAUDIBLE]
[INAUDIBLE] by in time, because will be [INAUDIBLE]
Cool, so I have two vague ideas.
The first one, I am really interested in technology that changes the way people live their daily lives.
So, when you look at 20th century, we had personal computers, so it could be like you're trying to hack things together in garages in Silicon Valley in the 1970's.
Another idea is being Henry Ford, and building cars or bringing cars to the masses.
Maybe you are a plant overseer and you are a line or something.
Another one could be you are the Wright Brothers, inventing airplanes.
It could be an experimental game.
Another idea I had was that you are in the 1500's or the 1600's, and you are a merchant.
You've got your ship, and you need to go buy stuff and bring it home and sell it.
But the complication is there is-- you know how those old-time maps had sea monsters all over them, because the ships would disappear?
But to a sea monster that ship is really good, am I right?
That was the only rational explanation.
I'm thinking that, [INAUDIBLE] What's that?
When you go off the Earth
This is after 1492
What's that?
This is after the circumnavigation, and the world is actually round.
But they could be [INAUDIBLE]
[INAUDIBLE] Here's your home, here's the place where all the resources are, and you have to get the resources back.
But there is a sea monster in the middle.
It's a path-building game, where there is a random aspect of the sea monster and if you [INAUDIBLE], you're dead.
But if you can survive, you will profit
So you're a ship captain
Yeah, you are a merchant ship captain
Everyone had a chance to say, who had their hand up?
[INAUDIBLE] you're getting data on which sensor is appearing, and you're trying to figure out where the monsters are on the map.
So you're getting this information that-- [INAUDIBLE] really make it obvious that that's not actually what's going on by rolling dice or something.
Or doing something very clearly random for whether-- which ships out of a particular set come back and which don't, based on where they were going.
And then somehow you are supposed to make your case that, this is exactly where the monsters are.
My map matches the data, and these other players maps don't
You want to be looking at cartography technology that was out there, or what sea-going timepieces look like at that time.
Navigation, but also how does society of information ran at that time.
Was it all just the royal society of blah, or was it more entrepreneurial?
So for that one, possible primary sources if you wanted to keep the fantastical element, but still make a little bit more of realistic of the time Umberto Eco wrote a book, Baudolino, about the mystery of Prester John, the Prester John who went from Europe to bring Christianity to the Orient.
But it is a book about the Orient It is a book about map making, cartography, that kind of knowledge sharing.
And the idea that if you just say something existed, it existed for real.
Enough people believed it, the people in power believed it, so that it might as well have been real.
And that's the kind of-- that's the weird kind of world they were working in the 13th and 14th century in Europe.
It was that you had these itinerants come around and say, yes, I met Prester John, here's this map, and by the way, here's a relic of Jesus, or here is a relic of some other saint.
And just by saying that made it-- gave those objects power, gave the person power.
At least for a little while within that little burg that he was traveling through
I was also going to suggest a cartography kind of game, or an explorer trying to map out something
So map making before modern technology is scary seeing that one of two things could proceed.
Anybody else have ideas?
It doesn't necessarily need to be game mechanic ideas, it could be just time period and person ideas
So I don't know if I want to do this or not, because it is really controversial.
I'm from North Dakota and I have seen a lot of the Native Americans struggle and I've read a lot of their history.
So I was thinking about making a satire, Like one person is the Native Americans and one person is the United States.
And just to illustrate the crazy injustices.
But that is kind of heavy, and I don't know if I want to explore it.
But there are ridiculous things that happen
Well it's not that far off from of the ideas you had about two very different levels of power, right?
And that also goes to very different levels of access to information.
So, maybe you should your group and see if those ideas work out
Not really fleshed out, but I was thinking about looking at a mass murder in history

Just any one of them.
Trying to make a game out of it
OK, so mass murders on the scale of armed conflict-- armed public conflict, and mass murders of the serial killers sort, where it is a mystery.
Do you have a preference?
I'm thinking serial killers
All right, there are a couple of famous ones.
What sort of roles?
Like the investigation part of it, or the try to get away with it kind of thing?
I think the trying to get away with it thing.
All right?
I think you might get visited by the
Have people watched the Meet the Team videos for Team Fortress 2?
So one of these videos is about the pyro, he's one of the characters.
And in that video it shows the pyro's worldview as Candy Land, basically.
And this is a guy who goes around and burns everything.
And I'm thinking something-- just in response to what you are saying about a mass murderer.
If you can find a mass murderer who is historically known to be crazy [INAUDIBLE].
And then you make-- you could have two different boards the way that there were for the [?
feed ?]
game, with the last project.
And you could have one of the boards be very clearly fantasy land.
This guy is going after something, he's trying to find something.
I don't know, trying to kill women or something.
So he's going around slaughtering them, and on the other map it is the real view of what has happened there.
And then you can have whomever is trying to stop see the completely different view on the situation
So similarly related to that idea, a good example [INAUDIBLE] but it [INAUDIBLE] There is a book called Devil in the White City.
At the Chicago World's Fair, they were building this beautiful city, they wanted to show off to the world what America could do.
There was all this modernization, and showing off the electric lights, and how exciting that was.
They built this beautiful, huge white building.
They put up the whole thing.
Then, in the shadow of this, there was a serial killer who was exploiting the fact that there were migrant workers there all the time trying to build this.
Which helped obfuscate his very criminal activities
That's a great [INAUDIBLE].
It also keeps in the spirit of the assignment, too.
Just thinking about how you've got this new situation, you've got these new systems involved, and how are you exploiting the system various ways.
For anything [INAUDIBLE] I would be really, really cautious about.
And even with the Native American aspect, or any kind of system of oppression, be really cautious about representing the person being oppressed.
If you're putting it in a point of view of the oppressor, just be really, really careful about that.
Actually, I personally think it's more interesting to be in the point of view of the person being oppressed, and how they're responding to that situation
That's a group [INAUDIBLE] who has done a couple of games on systemic oppression.
That every time there-- if you look at any historical case of human-- a period of human oppression, you can always find a system underneath it.
That system actually enables people to more easily inhumane to each other.
Both as game designers systems are very practical tool and look at that.
But just be aware that exploiting the history of people, who are oppressed, by the way, by taking their culture away from them.
Try to do it with some respect.
Try to bring some empathy into it.
This is what it was like.
And the limits that that they had on the kinds of decisions that you could make, for instance
A lot of historical games are very asymmetrical.
Are the ones that are [INAUDIBLE] I don't know of any simple historical games [INAUDIBLE] All those historic games that are good, and simple, don't have these imbalances in power between the players because that requires very complicated rules make sure that it works
It is a huge design challenge, for sure.
It is not going to be easy.
It's
Is The Apprentice, Irish game, asymmetric?
Yeah that is an example.
That's an interesting game because-- so there is a game [INAUDIBLE] Romero.
Has the designer of many many famous computer games, including the Wizardry series all the way back in the '80s.
But what she's more often known for nowadays is a series of games that she designed for museums on various systems for human oppression.
Trail of Tears.
She did Train.
She did a game about he English occupation of Ireland.
The way that game deals with it is that all the players are symmetric.
All players are different Irish armies.
But the system is playing against you.
The system is basically this mechanical influx of the English taking more and more Irish land and cramming all of the Irish into smaller lumps of land.
And people get pushed off the island-- forced immigration, basically.
But that's another way to approach it, right?
It's asymmetric but in a way it's easier to design, because it's the game system versus the players.
And the games that those actually get the games is that is forcing the players to conflict with each other, because there are less and less resources for more and more people in the small space
I'm trying to think.
Yeah, you're right.
All the card-based two-player games that GMT puts out, like Twilight Struggle, they a lot of complex-- there's a lot of complex things going on underneath, and a lot of it is buried in cards
Well, I can think of games that are highly asymmetric but not incredibly complicated, and a lot of it [INAUDIBLE].
For instance, which is not historically-based, but whatever.
It is about alien races, and each alien race basically has a rule they can work in a very interesting way.
But everybody starts with the same core rules, plus this one thing that you get to break.
Tammany Hall, which we didn't get a chance to play today, is another one where it actually comes out to every round that you play in a game, there is a particular rule that you can break because you are occupying a position in city government.
They can think of it as that break on special ability and execute.
That is another way to be able to think about it.
So, we've already had The [INAUDIBLE] which was really more like two different games that are being played simultaneously.
So if any of you were on that team, you already have a perspective on what it's like to design a game with magic information, and magic abilities for-- in different goals.
It has been done in this class before.
It is a design challenge, but you can do it
The original Netrunner is still a little complicated in that, there are a ton of different cards that are carrying a lot of the stuff.
But [INAUDIBLE] it is closer to what you do in this class
It's not like you've got a game that has different goals you already enjoy using [INAUDIBLE]
[INAUDIBLE] Cosmic Encounters, but it's not balanced at all
No it's not.
It doesn't need to be.
That's not a requirement of this class.
The class is something playable and engaging.
I never asked for balancing.
First of all, the time frames that we're giving you to completely finish these games is a little bit unrealistic for balance
Iteration
Visibility--
Iteration gets more balanced as time passes
Yes
But at the end, we still don't-- we're not really grading on balance as much as we're grading on iteration and--
I think these like usability is something that we are going to look at.
Do understand your rules is more important than how well your rules look.
If your rules completely break, that's a problem.
But if your numbers are off, sorry.
A few percentage points or a couple of [INAUDIBLE] or a couple of [INAUDIBLE] even.
But I don't understand how your rules work, that means a lot more to me
Just throwing an idea out there.
The role of the-- one of the people who really founded the board game industry
Do you mean the Parker brothers?
So from the perspective of a game designer or someone who is really trying to make this business a business
Or you could be something like a Hasbro, that's just trying beat other people to the business.
Or you can be something like the independent Richie Branson, trying to [INAUDIBLE]
I figure that will be closer to people's hearts, too
You have to do a little research, because we obviously haven't covered a lot of them.
But it would be interesting research.
That would fit in nicely with the time period that John was talking about, the 20th century, the mass productization of America.
But also you probably do need to limit that to one country's industry.
America is probably easy to do.
It's a bit difficult to look at the whole world.
[INAUDIBLE] The rest of class today, I know this group wants to get back to the game, you can do it.
But time for you to talk around with your ideas
Feel free to email your ideas the game design mailing list.
If you need to find other people, we'll do another.
We will do this again at the end of the day Wednesday, right?
Wednesday
And then, teams should be formed by Monday, because on Wednesday the 16th, that's the first presentation
Yeah, the pitch
The pitch.
So for a reminder about that, we want to know about what primary and secondary sources you're using as your inspiration, what kind of game you think you might be making.
Nothing set in stone, it's much more about, give us a pitch of why this thing-- this game idea is interesting, and what is the source of information you using to make sure that the game is realistic
But let me make sure that we get this April.
14th, no, April 16th is when you do the pitch part.
On April 14th, you should already be able to talk a little bit about, this is the idea that we want to work with.
These are the people in my group, these are some of the sources of information that we're looking at, because we'll have guests, and they will be able to give you some feedback on that.
Both in the pitch and in the guests we're not actually so much grading you on the quality of your pitch.
But we are going to provide you feedback on how you're pitching, so that if you have to do pitching in real life, in your career after graduation you can get some feedback on how you present yourself.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality, educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, So let's get started.
Today we have got the pleasure of two local game builders.
So they're here to talk to us about some of their work.
We've got Glenn, and I don't have your last name in front of me right now.
Given.
Glenn Given
Glenn Given from Games by Play Date.
And you're located in New Hampshire?
Yes.
Well, 2/3 of us are.
It's like a three-person studio.
Two of us in Andrew and one of us who's in [INAUDIBLE]
Great.
And then you do board games and card games?
Yes, a lot of them
And then Mack.
How do you spell your last name?
Mackenzie Cameron
And he is one of the co-organizers from the Game Makers Guild, a local meet up, meets up at various places around the Cambridge area for people who are interested in developing board games and card games and meet with each other and learn more about how to make them.
So each here, we're going to go just Glenn then Mack.
Talk a little bit about what you do, about the kind of work you do, and then afterwards, we're going to open up for questions
Sure.
So make games.
I make board games.
I make card games.
And I try and never stop doing them.
I don't have a particular-- anything that I can play that I can make happen, I'll do it.
We do Targi games.
So I just got 2,500 copies of my first party game delivered to my house three hours ago, which is interesting, moving those myself and then getting here.
But I also am developing a gigantic, stupid board game that's just called a Big Dumb Wargame, which is like these guys play Axis & Allies or any of those old.
So I used to play that when I was in high school because I liked being lonely and then I decided I wanted to make a version of that that's actually fun.
We're also working on a game called Pack the Pack, which is-- you guys know Cards Against Humanity?
So they're running this contest called the Cards Against Humanity Tabletop Deathmatch, and they got indie developers from all over the country to submit stuff.
And so one of our games is in that.
It's like if you've ever played Old Diablo, remember when you would collect loot and you would have to fit it in your bag and it was all weird shaped?
So it's been inventory Tetris games where everyone is pulling tiles, which are effectively dominoes with different images on them, and then you're aligning them to make gems.
So it's like an analog version of Super Puzzle Fighter.
But the moral to the point, everything I've just described to you doesn't tell you anything about the games except in reference to other games that I talked about.
But that's how I do design.
I look at games that I like and then I dissect them and take the parts that I want and then put them back together with other parts and see if there's a theme that goes with it.
But yeah.
Also, I did printing and publishing for 10 years, and then that drove me madder than normal.
So I left that to do this.
So me and my team very rapidly design games.
We bring a design all the way to a publishing level in about a month, from conception to execution.
They're not big, huge endeavors, but it's good to stay really busy with it and exercise with it and realize that sometimes a project is going to fail and you can back away from it and go, ah, well, that month is kind of a bust.
And so we're actually also exploring different kind of business models for board games because this isn't really a huge leap.
There's very few people who are going to be very wealthy on it or even do good on it
Scare them off, right?
Yeah, seriously.
So maybe you make the next Settlers of Catan and then you're selling a steady 5,000 copies a year, which is really not that much when you think of it.
Depending on the way your business is set up, you may see a fraction of that money.
So that's why a lot of very, very good designers, like I'm a big fan of Bruno Faidutti who just came out with Mascarade.
Mascarade is a really cool game
[INAUDIBLE]?
Yeah.
So he'll put out a couple of games, and then he just gets a cut from each of them.
But using short-run printing, we directly manufacture our monthly games, and then people subscribe to us through stuff like Patreon.
And then we send them physical copies of the games that we make.
And I brought nine different games that I've made in the past six months
I didn't bring any
Yeah.
Well, you can borrow some of mine.
And then you can borrow the bad ones.
And then also we go to Kickstarter for larger games, ones where we want to make multi-thousand copies of it.
But yeah, I will never stop talking.
OK, cool.
Yeah.
Board games are really awesome.
I think that you guys know that.
And they're more awesome now than they used to be, not just because they're more complicated because complexity isn't what makes a game really good.
In fact, usually a really complex game is really bad because it's a sign that there was a huge problem and that the designer said, oh, I know how to fix this.
I'll add another page to the rule book.
That's poor planning.
But games are better now because people want them more.
I mean, what I think the reason people want them more is because we crave physical, analog time with other people now more so than when I was a kid because we're super interconnected with smartphones and all that stuff that we can't live our life without.
I'm twittering all the way down here doing 80 on the highway because I'm a bad person.
But the ability to say, OK, I've been staring at these screens all day long, and my entire life is scheduled.
I need to schedule my relaxing time and it needs to not be me staring at the screen.
So I think there's something in the zeitgeist of that personal feeling that people had that has caused a resurgence in face-to-face gaming
Yeah.
Because Even like apps and stuff, I will straight up just hands down beat anyone that plays the Race for the Galaxy app than playing them face to face because the skills that you need are completely different.
Just like this board games do really well that you don't find in other mediums
So I play a ton of Netrunner, which is a really awesome game
The rule book on that is awful
The rulebook is the worst.
But some people have hacked together.
There's a program called Octagon, which is like a virtual tabletop where you can basically simulate any game that's been made.
I mean, there's a whole interesting other discussion of how do you do internet piracy of board games.
That could be a class.
It's a good idea for a class.
Anyway, so Octagon is really neat except one of the things about Netrunner is that-- if you played it, raise your hands so I know--
Does the original version count?
Yes.
Sure.
Although the new version is better.
You should totally get back into the new version.
It's amazing.
So it's by the same guy who designed Magic, the Gathering.
It has some sort of similarities to that.
But what makes it really neat is that it's an asymmetrical game.
Like I'll be an evil corporation, for instance, and you might be a scrappy computer hacker.
And you're trying to infiltrate my servers, which are represented by my deck of cards, my hand of cards, my discard pile of cards, or like other things that I have installed and find what my nefarious plans are.
And you get all these crazy programs that allow you to circumnavigate my defenses, and I get all these programs that allow me to put down defenses, or maybe I get a card that says, I orbitally bombard your apartment building, which is always fun.
But a huge part of that game, although you can play it on Octagon, a tremendous part of that game is it's bluffing because what makes the game really rich is not that it's asymmetrical, but there's a huge component of hidden information.
As a corporation in that game, everything I do I do by placing my cards face down.
So I know what they are, but you don't.
And you have to start to count cards a little bit or start to make educated guesses about what you-- can I attack his hand even though I have no offensive capabilities here?
Is that going to mean that he explodes like a thing in my brain and my computer hacker dies?
There's all that.
So a lot of that game is about managing what's called pilt, which is something you get in poker.
If you play poker, when you start losing poker, if you forget that poker is kind of a long game and that you should really be playing it with an eye on what the rest of your weak looks like as opposed to this hand, you'll start to play worse because you're doing worse and you'll just pilt.
So it's kind of like a tell, but it's more than just saying I've got a bad card in my hand.
It's going, I'm losing so I need to play harder, which would be great if it was handball or something, but in a card, that doesn't work because really what you want to do is establish regularity in what you're doing, especially in a game that's that mathematically perfect
I think I'm going to go ahead and introduce myself
Please do.
I don't even know where I am
So my name is Mackenzie Cameron.
I'm an event coordinator for the Game Makers Guild.
So we host a lot of events, which you guys have had any games at any level of prototype, whether it's nearly finished or you just scrapped down something on a napkin and you're like it might play.
Let me go grab some dice or whatever, we accept that.
Check out our site.
meetup/gamemakersguild.
Actually, we just recently got gamemakersguild.com.
So you can go to that website
That's a coup
--which I'm a little [INAUDIBLE].
It's sweet.
I'm slowly trying to make sure that we build copies for that
[INAUDIBLE]
gamemakersguild.com, which is awesome.
I don't know how we managed to get that without somebody else already having it.
But I host some of other events that we hose.
We have your standard play testing, but we also do indie board games showcases, actually.
I've hosted that in Brookline three times now, where we just get anyone that has a cool design.
So make your designs, and then we try and bring it out to the public.
And then what's the other stuff that I do?
I try to get my hands in everything board game related in Boston.
I help out at Knight Moves, the board game cafe out in Brookline, which is a lot of fun.
If you guys don't know about it, you pay $10, you get in.
They have a library of something like 700 games.
You can play for as long as you like with anybody that's around.
I also do a board game seen to web comic, overboard-comic.com.
And then there's one more thing.
I'm going to be doing a panel of techs this year.
Me and five other folks are going to be talking about board game language, which should be pretty interesting.
It's talk like a board game geek, if anyone is going to PAX.
Who's going to PAX?
Not many of you.
You should totally go.
It's the best thing.
It's The reason I moved to Boston.
And then in the sort of a distant future, I've got plans for my maiden voyage into Kickstarter called Killer Croquet, which is the croquet-based murder simulator.
That will be a board game.
And it's amazing.
Not to say that Glenn is wrong about everything, but I'm doing the one idea, build it up, do the kickstarter sort of thing.
The rapid iteration is really important, but there's definitely lots of different paths towards making a board game and having it be successful
I think the main difference is that he has another job
Oh, yeah
And I don't
That's true.
I do this for no money at all
Yeah.
I did this for no money too, except that's a problem.
So rapid iteration.
But do you want to talk about how you went from game idea to the prototype?
Sure
I mean, do you want to let them ask questions?
[INAUDIBLE].
Process
Yeah
And then the next question
So process.
Mine is definitely rapid iteration, but I've finally settled on one idea that I actually watch you push forward rather than trying to bundle them up in small ideas, get the one big one.
I think the main difference of that is that I'm hoping that with that one idea, I'll be able to use momentum to generally generate more funding for the general brand name of my design studio.
But coming up of that idea is definitely a rapid iteration process where you come up with the very sketchy idea.
Some designers start with a mechanic, so they're like I want to do something that's deck building where you build decks over the course of the game, which makes an engine that you then draw to then do other cool things.
And then other designers, myself, I'm usually like this.
Start with some sort of system or some sort of theme, like a croquet, and see if you can turn that into a board game by adding different mechanics and seeing what works and what doesn't, which the Game Makers Guild is pretty fantastic for that.
Because when you get a game and you design it and on your first [INAUDIBLE] you look at your game and you go this is amazing.
This is going to work.
And you try and tell someone about it, and you try and have them play it.
And that doesn't always work so well
It never works
It never works, actually
The first time you put your thing down on the table, it's going to maybe burst into flames
Yeah.
A best case scenario is that it bursts into flames and you've made some mistakes.
And it's basically unplayable because you need dice.
And you realize that when you roll the dice, the numbers always add up to some combination of factors that when you apply that to system, it's like oh, well, you're supposed to move one space, but every time you do this, you actually just die immediately
Yes.
So people can break it in mechanically.
You find a tautology element in it and just repeatedly--
But one of the worst thing that you can do, I had a game that I was developing for a while.
I pitched it to Game Salute.
And it was a semi-cooperative game.
So the idea was that all players have to work together so that everybody doesn't lose, but then once you're going to reach a certain threshold, only one player can win.
And kind of a difficult concept to really balance well, but I'm eventually going to really finagle it and get it working.
And a bunch of my play testers played it, and it worked out exactly the way I wanted to.
They'd play it to a certain point and then they turned on each other and then there's this big, climactic battle, and it was fantastic.
I was Watching just the mechanics of how it works.
And then I asked them, it seemed to work out really well.
How did you guys enjoy it?
They said, it was awful.
I said but it works.
You guys functioned in the way that was really functional.
Yeah, but it wasn't fun, which is something that you realize as a designer, you're putting stuff together that the things that you necessarily think are really interesting or even if they're working don't necessarily come across as fun
But that's really interesting.
They finished the game.
They didn't like playing the game, yet they both played the game
Yes
Is that common when you do play tests?
[INAUDIBLE]
They're involved in the game
Usually like, especially if it's in a dedicated play testing group and especially in the Game Makers Guild where it's primarily designers, that has its own downsides to it.
But yeah, they'll see it through.
I mean, they'll go [INAUDIBLE] on it, push that boulder all the way up just to see-- I mean, even when we found stuff that is completely broken in a game.
I was playing a game where it was robots rising up against humans or something and there's this whole propaganda element.
And then in the first few turns, we had identified that if you just went to this one space on the board and continued to buy propaganda posters, you could shut every other player out of the game.
And so it's just I'm going to go there.
I'm not going to move.
I'm just going to keep doing this for the entire game until I win.
And at that point, you could be like, OK, I don't want to play this game anymore.
This is dumb, but we just did it for a half hour and different people tried different things using the rule set to unseat that bad decision.
So sometimes even when you identity something that's broken, it just gives you a new wound to start poking at to see does this hurt?
It's like a doctor.
That's how doctors work.
Stab you
Well, that's that doctor's game.
Solve them.
That's great.
Encouraging play testers to finish a game can be an art in and of itself.
Oftentimes, when you get designers that think that they can fix your game for you, it will be hard pressed to get them to finish the game because they're like no, no.
This is where you should do it like this.
But in trying to urge your play testers, we actually had an event where we brought somebody in who was an expert on generating play tests feedback, actually, for video games, which need a lot of paper prototyping, so it was a carry over.
And often times it's just trying to encourage players to go all the way through because sometimes even as players, even with published games, you'll play a game like in the first playthrough you'll be like, oh, this one strategy is completely broken.
I don't know if anyone is familiar with the card game Race for the Galaxy.
Maybe a couple.
But there's one particular strategy where the first time you play through, there's a military strategy where if you have a certain level of military, you can play a lot of cards roughly for free, which is based on the mechanics.
And almost everybody that plays the game for the first time realizes, oh, man, military is such great strategy.
How did they make this game?
And then the second time you play through, two players try and use that strategy, and the third player does a different strategy.
And all of a sudden that's the strategy that's overpowered.
And oftentimes, because the game doesn't completely lay itself out in front of all the players in the first play test, the perception that it's broken is not always necessarily true
And you can't find that kind of stuff until you play test it.
[INAUDIBLE] Questions?
Essentially, I think [INAUDIBLE] example of that is [INAUDIBLE], where how many people [INAUDIBLE], like oh, man.
This strategy is so overpowering.
This person scored 70 points with it, and none of us were above 40 or so.
And then you respond like 70.
[INAUDIBLE]
Well, there is always stuff like that.
I think that illuminates another interesting thing about play testing.
The group that you're play testing with really can make a huge difference, not just is this person a designer or is this person not a designer, but how deeply they want to get into mastering that system or how much they care about mastering that system
Yeah, because we offer multiple tiers of play testing at the Game Makers Guild.
We have like just the standard where you come and you play with a bunch of designers, and they tell you it's crap.
And you go home and you cry, and then you make it a little bit better and you feel a little better.
But then also we have intensive play testing, which is phase two after you get a certain number.
We have a nomination and serration system.
We're trying to set up a game maker's seal of approval so we can hopefully better pitch our games to designers and the general public.
But the phase two intensive was the chance for you to get the same group of people together and play your game a bunch of times.
And I mean, again, part of the fun of that for Play Chesters, for a designer, it helps to iterate that even if the game is broken so you can find those little bits and pieces where the games completely falls apart.
But at Play Chester, it can kind of fun to just find-- I mean, it's like when you play a video game and find some element of the game where the graphics are screwed up or you fall through the world or almost just the exploration of that.
And I have a lot of play testers.
They enjoy finding the chinks in the armor, so to speak
Well, I think the other thing about do repeat a play test especially if it's the same group, has anybody played the Vlambeer game, Luftrausers, that just came out?
It's great.
You should play it.
It's Totally worth the money.
It's kind of like a Lunar Lander meets Asteroids game.
So you're kind of drifting, but you can customize your airplane with different engines and different bodies and different weapons.
And they all make the game play-- they all screw with the physics of the game in different ways.
Well, unless you put three or four hours into playing that game, which is a lot of time for a very arcady-style game, you won't really know how do I get past this certain threshold of points.
And what is my sweet spot in controlling this airplane that I can maximize how powerful I am with it.
The same thing in boardgames.
Often, it doesn't happen in that time frame.
But for instance, with Netrunner or Magical or whatever, you play and play and play and play.
And you start to realize that some of the things in this game are just red herrings.
They're just included to dilute the power of other things, or they're included to counterbalance very specific scenarios.
And things that seem like total waste are actually really important later on in the game and you just can't get that up front
So just real quick.
Going back to Sulkeen really quick is that there is a certain difficulty with the length of games.
So if you have a game and you're trying to play test it and you hit 30 minutes and it's not fun, and then you tell your players, don't worry.
There's two and a half hours more to it and we'll get it figured out, you've got a much harder row to tow on that, which actually, you'll find I think, in part you'll see a lot more games that are designed for the 45-minute mark because that's usually-- I mean, that's the easiest way to iterate a game
It's also about the time.
I mean, think about the amount of free time you really have in your life
Well, they certainly have a lot.
They're students
But even that, I mean, there's kegs to stand on and all that stuff.
I don't know what they do at
It's some sort of giant-like robot
Yeah.
You build a robot.
I'm sorry.
I have a dime in my time because I'm a professional.
It's just driving me mad.
Otherwise, another important thing about being a designer, develop a mental disorder.
That will really, really help you get the tiny problems like dimes in your tie out.
So yeah.
Half hour, 45-minute games are becoming the thing.
As board games become more and more popular in America, because they were always-- well, for the past 25, 30 years they were very popular in Europe.
And when we were talking about board games, I think we're not talking about traditional games like Monopoly, Scrabble, that kind of early American family board game, I think
Because they're still a great game.
There's so much.
When we talk about board games, we talk about they're roughly this big because the designer's name on the box, they take between 45 to 3 hours.
Produce some things like War Hammer and different war games.
Because there's whole other cultures out there, which is really cool.
But when we say board games, we're talking about that again
And that's different from say, I like to use the term tabletop games, but I'll expand that really far.
I did a tabletop producing for the Boston Festival Indie Games, which is coming up in September
We should talk about that
The game is like a hockey rink filled up with indie games.
I don't know much more than that.
We just opened the submissions yesterday
[INAUDIBLE]
Well, one of the differences from last year that when I came onboard because it was my first year doing it, I really wanted the stress was that a tabletop game or an analog game is a different term than a board game.
A board game is literally something with a board.
But a tabletop game, I'll stretch that and say card games are tabletop games.
Roll-playing games are tabletop games.
You can be playing the clunkiest most grognard DMD-- "grognard" is a French term for curmudgeonly old guys.
That's basically it.
I think it actually is for generals or something.
They're the worst.
So Apples to Apples is a tabletop game.
What's the game where you're pulling out the sticks and the marbles fall down?
Ker-Plunk.
God, I love that game.
Ker-Plunk is a tabletop game.
Dominoes is a tabletop game.
Bocce Ball is not a tabletop game, but it's great.
But also if roll-playing games or tabletop games, does that mean a LARP is a tabletop game?
Because a LARP is just the roll-playing game without dice.
And if LARPs are tabletop games, then how about--
I think they're technically dexterity games
Well, if you're doing proper LARPs.
But if you're doing something like a Nordic LARP, which is the brand of live-action roll playing from Norway, Sweden, and Finland--
God, those folks leave me nuts
I was invited to a convention with those people last week, and they are hilarious
The only one that I know that they do there's a Battlestar Galactica, where they actually they rent out an old--
Battleship
--battleship
For a weekend
[INAUDIBLE] on it for a weekend, and yeah, they go and they play this game on right on a frigid waters and military bunkers playing Battlestar Galactica-themed live action role playing game
So it is a roll-playing game, but it's also like an interactive theater performance.
That game was crazy, apparently.
So is everybody familiar with Battlestar Galactica?
OK.
So there's robots, and they look like people.
So it was a Battlestar Galactica, and then they had people who were cylons who were agents.
And then anyone who was wearing red was a hallucination.
It could only be seen by specific characters
It gets better
From the second day of the LARP, and this is like a three-day thing.
You frigging sleep on this battleship and everybody is in character the whole time.
On the second day of the LARP, one of the guys who has red who was part of the plot as a hallucination, they brought his twin brother in who nobody else knew that he had a twin brother.
So all of a sudden there's two identical hallucinations wandering around the ship, and everybody is flipping their fucking wig.
So I guess what I'm saying is, if we can rent a battleship for [INAUDIBLE].
And then like games like Johann Sebastian Joust, which is a digital game, but is it really a digital game?
It's really kind of like tag
No.
I wouldn't say-- what is it?
Spaceteam.
I don't know if you guys know Spaceteam at all.
iOS game.
You play it by basically you have a user interface.
And you press buttons before a timer runs out.
Unfortunately, what button you have to press is not on your screen, but on someone else's screen.
So they'll tell you to flash the paper screw, and you look on your thing and do I have the paper screw flash?
But at the same that is that a video game because there's not really much to it in terms of the digital aspect so much as that it's a timer and you have to press buttons at the right time.
You could make an analog version of that
You can make an analog version of that
So that brings us into the funding aspects of things.
All these different categories, all these different markets, you can think of.
How are you exploring?
How are you marketing your games?
What are you marketing your games as?
What's your [INAUDIBLE]?
Well, certainly--
I have no idea
Certainly, when we say board games, we mean like against the box about yeah big, which is, in this instance, I call it the hobby industry, though there's a lot of very nebulous terms of what that falls under.
But the hobby industry is separate from the toy industry, which I was actually at Toy Fair in New York.
Mostly, I actually just wrote an article on my blog about there is a difference between a game inventor, a game designer, and a game writer, and how it separates out the different industries that are out there.
But for myself personally, our kind of industry and where we're hoping that we're seeing the most growth is the feeling that people that are mostly on social media, a lot of people that are really into Kickstarter, and very into thematic games that are both themselves semantically interesting as well as the systems that are within them are fair and balanced and are worthwhile to do.
So my strategy is actually to initiate.
My Kickstarter is in about six months.
And in that time, I'm going to be creating a trans-media storytelling ad campaign that will take the theme of the game, turn it into sort of a story, and then promote that as well as talking about just the general design as a result hoping to engage social media networks that will then generate the needed buzz for when the Kickstarter campaign actually launches
A lot of fundraising has less to do with what you're making and more to do with what you can conceivably accomplish in marketing.
So if you have no budget, you have to rely on certain marketing techniques as opposed to others.
A lot of people who have no budget like to think that commercials don't work.
That's stupid.
Why would anybody do that as a way of kind of convincing themselves that they've made a noble choice?
No.
They just didn't have the money.
Sometimes commercials really work.
For instance, as I was touring on my way down here, I relaunched a Google AdWords campaign for our game Slash because now I can fulfill orders for it.
Using multiple marketing vectors in order to get people interested in your project is really, really important, especially because the means-- it's a very competitive space.
It is an industry that is like a hobby and toy industry was like $6.1 billion last year, which is not insignificant, but it isn't video games.
But unlike video games, it is an industry that has grown 11% on average, for the year over year for the past five years
Before that it was even growing more so than that
No.
It's actually gone up, but it's predicted in 2018, I just got the University of Texas research.
I take it real serious.
So it's a very fast growing industry because there's space in America for it.
It's not as fast growing in Europe because that's already a supersaturated market.
But stuff like Kickstarter, Patreon, and Indiegogo, even just going to shows and selling your wares, which is actually really the best way to do it.
The way that you get people to come to them is to find the audiences that are already keyed into that.
They like the idea of I'm helping a person make a thing.
I'm not investing, but I'm kind of like pre-ordering this thing.
So you've got to identify what those markets are and find the best ways to get them and convince them to throw money down a hole called Kickstarter.
Because not everything on Kickstarter gets funded, and not everything that gets funded on Kickstarter gets done.
And then--
There's some horror stories
--many of the things that get funded and get done are crap
[INAUDIBLE] a little bit.
Like you said, it's a growing industry.
So the hobby end of the growing industry as opposed to the toy industry.
So if you're trying to get your games to Hasbro or I guess Parker Brothers, but anything else that ultimately leads back to Hasbro
Yeah, because Hasbro, [INAUDIBLE]
But anything you want to get you game into Toys R Us, at that point, that industry is pretty static.
So as a result, it's very hard to get known or mentioned at all in that.
But because the hobby games industry is growing, there's more and more people that are looking for the ground floor.
And as a result, if you can get your foot in, make a little brand for yourself, get your name out there, then when you actually do you start making games, people will recognize you, and you'll be able to kind of market yourself as a brand.
Again, that's probably one of the most important things.
And the only reason this is possible is because game designers start putting their names on the box.
When we talk about designers, let's talk about Reiner Knizia, one of the most prolific game designers.
And all of his brands are great, but his name is known in the hobby game industry because the dude designed like 300 games and actually has them all published.
And as a result, when you see Reiner Knizia's name on a box, you're going to start to recognize that a little bit.
When you're able to generate--
I mean, they're not that all good
They're not all good.
But Reiner Knizia.
I've heard that name before
I agree.
I mean, he created a brand for himself just like video game companies d, or labels in music.
The music industry, you guys weren't around in the '90s, so you don't know it exists.
For you, still exists.
Yeah.
So I think the interesting thing about the market and being able to have that personal brand as a designer is really important because the people who are playing board games right now and the growing market are, I guess, they're iconoclastic people.
They're hey, we're all individual types, but they're all we're really big on geek culture.
And then geek culture is bringing 75,000 people here this weekend, so maybe it's not as marginal as we'd like to think that it is.
But it is a market that values individuality and individual productivity.
So finding the ways to market to those people directly can often be really rewarding, especially if you're putting your face on what you're doing
Another example is Daniel Solis, who's down in--
North Carolina
North Carolina.
For the longest time, I mean, he was a graphic designer, but he started a board game blog.
He just talked about his expertise in designing a lot of games.
He's a very prolific guy, and he does a lot of stuff that's just completely open source, print-and-play games
Actually, he works out of his house like I do, and we actually do a conference call every morning with a number of other designers
I was wondering how you knew him
He's just released-- you guys ever watch Firefly?
That show?
So he just did the layout for-- like his normal job is doing graphic design layout
Where's your other stuff?
So he just laid out the Firefly BG, which is available for download now as PDF and then they're provisioning it later.
So he does that, and then he has this whole I'm going to make games a lot and put them up on DriveThruCards, which is an online service that allows you to do print on demand, which is another one of the things that in the past year has really shaken up the board game industry.
You used to not be able to get anything done unless you're are ready to do 3,000 copies of it, but now I can make a game, put it up through DriveThruCards, and you can go send $10 and download it.
Or they would actually send you individual print in seconds
But again, some of [INAUDIBLE].
So Daniel Solis has been doing all this stuff, just presenting himself as an expert creating a brand of himself.
Actually, I brought him up to Hacks last year as an expert.
He, about six months ago, ran a Kickstarter for Bell of the Ball.
And he leveraged some of his connections.
And the game itself is very good, but the Kickstarter was wildly successful.
I think he got something like three times what he asked for just in terms of kick starting the game.
And as a result, I believe, a lot of his connections and leveraging his brand as an expert.
I think at some point, he's got 5,000 Twitter followers and decent blog presence.
And as a result of having that before he launches a Kickstarter of starts to generate pre-sales for a board game, that he's going to do much better.
And the effort that he puts into it is really just talking about board games, putting information for free online and establishing himself as someone who's an expert
So all of these crowd funding models, there is a little bit of precedence in a publisher [INAUDIBLE] groups like GMP.
But for the most part, they're fairly recent developments.
because we'll be willing to give money online.
I'm assuming that your interactions [INAUDIBLE].
But what is this role of stores, especially hobby stores?
I do you also work at a games store Eureka!
Puzzles down Brookline.
And that's an interesting function as well.
I do a little a little bit of research in general on retail stores, not just board game stores, but anywhere that sells things.
It's taking a nosedive as an industry, which is hard to imagine just stores selling things as an industry.
But idea is with the internet and [INAUDIBLE] like with Amazon Prime, oftentimes, you can find the same product online.
And even with shipping, they can beat the price of any retail store, which is board games, war game retail outlets are pretty hard.
Because you can go and hell, I work at Eureka!
Puzzles.
I get in and play discount, and it's still cheaper for me to go on Funagain Games and buy a game on that end.
With shipping, it's still cheaper than what I pay at Eureka!
Puzzles for a board game.
As a result, a lot of retail outlets are starting to realize that if they're going to survive, they can't do it by trying to beat prices with online stores
I turned it down
As a result, they're realizing that they're strength is that they have a physical presence.
So their idea is that if they're going to charge more for board games that you can get online, they need to add value to that
They'll probably do stuff like the Knight Moves Cafe, where they're taking that.
So the interesting thing about this-- well, one of the interesting things about this is comic books, right?
So comic books, in the '90s were huge, and then they went through this bust, and the people who owned comic book stores started to see the margins on their products get shorter and shorter and shorter and less and less and less people buying comics.
So it's a really dire situation.
And in the past 10 years, it's become really easy to pirate comics online, but more so than that, there are apps for your phone or your iPad like comiXology or the Marvel app that actually do a really, really good job of presenting comics to you.
My wife wouldn't read comics at all before just because sometimes people who don't really know how to do layout end up doing layout for comics.
And so literally, it can be confusing to read just because they don't have artistic mastery of how to make someone's eye flow across the page.
One of the things that comiXology app does is that you can just zoom in panel to panel to panel to panel to panel, and it's a surprising shift in the way you can read it.
Anyway, the important thing is that comic stores were getting a lot less money, and so what they did is they took all their floor space and started to dive heavier into the parts of their business that were generating the money, which was board games and hobby games.
And the ones that survived-- because a lot of them have gone out of business.
The ones that survived tend to be really, really good.
And the reason that they're really, really good is because they realized that they're not in the business of selling you comics.
They're in the business of creating a community.
I drive an hour to my comic store because I'm an idiot.
I could just be buying them online.
I drive an hour to my comic store because they have their good operators creating a good community for people to come together and talk about different projects.
They're always running events.
It's barely a store.
It's like a little mini convention that's always slightly happening.
And so a large part of that is realizing we can get people in this store for a longer period of time if we're all playing games together, if we're running card game tournaments and board game tournaments
The interesting part is how do we as designers take that?
What's their role to us?
And I mean, before it was definitely, you could your game picked up by distributors or to a publisher and a publisher gets picked up by his distributor.
And all of a sudden, you're not selling to a direct audience online.
You're selling to hundreds of thousands of little mom and pop stores across the nation, the numbers are huge.
But now--
You can still wind up making a lot less
True.
Because your margins are going to be--
Because your margins are worse
But the part of the thing you can do now is that board games can really help with the promotion of your game.
I know Eureka!
Puzzles, Knight Moves are huge online.
If you've got a game that you want to show people that you come and teach it to a bunch of people at their store, they're happy to do that because on the one hand, you're adding value to their store which they love and need.
So people will, like oh, your new puzzle.
They had one guy come in and that was really cool.
I learned how he made this game.
But then you're getting value out that, and now people know about your game.
And then when a store buys games from you, you still get maybe not necessarily the margins you need, but--
They're still a lot better.
So normally in traditional publishing, if I was going to put out a board game 20 years ago, 15 years ago, I would have to find a distributor like Alliance or Diamond or something, and then I would sell my game to them for 20% of the MSRP, which is the manufacturer's suggested retail price.
So if I had a game that was $25, I would sell it to the distributor for $5, and then he would turn around and sell it to a store for somewhere in between that $5 and half the MSRP, and that's his cut.
And in between, there's all these other people who are taking their cut and taking their cut and taking their cut.
And the reason I've made jokes about the music industry model from the '90s is because it's the same fucking model.
It's The same thing.
There are people who are creating stuff, but because there is no really awesome way for them to get it directly to the people who care about it, they have to go through all these weird side projects, side industries or middleman industries.
Only a few people who are tapped by very, very powerful companies got to be famous in the '90s.
And then the internet came and destroyed all music, even though music is making more than it ever has and all these other things.
But what was rediscovered in that shift, and this is something that's happening in board games right now, is that people want the direct audience to create a connection.
If you could spend $20 on a CD because they used to be $20 or $20 on a ticket to go see a show, of a band that you enjoy, even though you don't get to keep that show forever, come on, it's so much more enjoyable.
Because you could get that CD in a million different ways from a million different locations.
It's really easy, but you're getting an experience with the other thing.
And you're getting to know that you're directly supporting the creator, which is something that a lot of audience members are really invested in.
And as someone who is a creator, I'm very invested in that.
But also it's financially beneficial.
So for me, for instance, I can make a short-run print product for $3 and sell it for $20 and then cook the mailing price into that and get some-- I've actually made a reasonably good profit on it.
But if I make it for $3 and want to sell for $20, I end up having to sell it to a distributor still for like $4 or $5.
And then in order for it to be worth my time, I really need to make a whole lot of them.
And the distributors probably only want it if I can make a whole lot of them.
So now I've got find a way to make 1,000 of this thing, which, in board games, is a ridiculous number of game to have.
Unbelievably large number.
But unfortunately, you can't get it affordable unless it's above that number
Right.
And that's the front end of financing too.
But if you think about Bell Publishing and stuff, OK, to make it, I'll need 1,000 copies.
And you'll be able to get a decent budget and figure out how to use it.
All right.
That's fine.
But then when you do things like, OK, I'm just going to keep these in my house until I sell them all, you don't realize how much space 1,000 board games takes up until you see it
Oh, actually, I have a video because I have 2,500 in my garage right now.
And it takes up exactly a car's worth
You have $2,500?
2,500 copies-- 25,000.
[LAUGHTER]
It's a card game?
It's a card game.
It is this card game.
And I've got 2,500 of them in my garage and then some more of them down at the port [INAUDIBLE]
We should bring up-- I don't think you ever said it.
There is some benefit to going through traditional publishing
It's right for some people.
It's not right for--
And the reason I would suggest that is if you get a really good game idea and you pitch it to a publisher and they really like it, there's so much more to that than just being like take this game.
You love your game.
Well, take it
And here's money.
Forever
If you create an idea and you sell it to a publisher, you're done
And it's not your idea anymore
And it's not your idea anymore.
But all of a sudden, with self-publishing, I mean, the amount of effort to make a game fly is huge.
It's massive, especially if you're doing it on your own.
But if you sell it to a publisher, you're not going to make as much money, but there comes a point where you're not doing anything and you're still getting paid it
Yeah.
100% of a small number or 5% of a big number
Yeah
And it's a question of what are you in it for?
And what does your time look like?
For me, I make games and I chose to leave a life that was not making me happy but was making me lots of money for a life that is making me significantly less money but is making me really happy, because I get to create things and they're fun
You seem like a really happy person
It's the medication.
Also, I don't work in an office so well.
So it fulfills that artistic or productive drive.
And that's not something to be ignored.
Total up.
[INAUDIBLE]
The major point is that it is you want to make your career making games, do yourself and you're really big.
If you want to put a few games out there and do a lot of other things with your life, like once a month sending out all of your games to every publisher you've got email for, you don't have to make it your life
Yeah.
It's like if you were going to start a band, is the band your life, or is it what you do for fun with some of your friends?
Neither is an illegitimate way of doing it.
They're just right for different people
We have more than five minutes.
So one question
Really?
We've have time for two hours?
I can do that
[INAUDIBLE].
Self publishing is undertaking that [INAUDIBLE].
Learning curve, so to speak.
And you guys have your fingers on the pulse of the community, and you know what uP and coming designers have and the skill sets.
Have you considered publishing other people's games?
I have considered it, and I've been approached to do it, but I've said no to it.
Because I worked in publishing for a long time, and I am ideologically opposed to that model.
That being said, one of the things that I do I do this on a consulting basis and I also do this just through our website and the fact that all games are Creative Commons licensed and the whole process-- Frank, can you turn that down?
--is teach people how to do it.
Because it is an undertaking but it's nowhere near as hard as you would think.
The kind of effort you would put into studying for a class is the kind of effort you need to put into figure out how to deal with Amazon fulfillment services, or what does it take to, oh, god, figuring out shipping.
If you ever start at Kickstarter for anything, figure out the damn shipping.
That will kill you.
I had a friend of mine who made this really cool game called Mobile Frame Zero, Rapid Attack, which is a robot fighting game where you make LEGO robots
Oh, yeah.
It's on [INAUDIBLE]
Yeah.
So they're doing-- and some of it's new on this weekend, which is like a rockets in space or something.
Anyway, he raised $85,000 on Kickstarter to publish what was just a book and mail it to people.
But because he had not-- and he's a smart person-- because he had not figured out the shipping costs, he ended up $25,000 in the hole.
Yeah, because you didn't go to the United States Postal Service and go, what are your rates?
So $85,000 and it cost him roughly $100,000 just to make it
So that is what a publisher does for you.
They figure don't even frigging worry about it.
We're take 90% of whatever this game makes, 95% of whatever this game makes, but you don't have to worry about any of that crap.
And so for some people that's really--
What you're actually finding a lot is that there's tons of small publishers and these are actually starting-- I mean, Game Salute is one, Mayday Game, I don't like that one as much.
Anyway, Mayday, they do--
Like my neighbor
[INAUDIBLE], which you might have seen on TV Cops
Yeah, that's Dave Chalker
Yeah
Yeah, he's a really good guy
But lots of small companies that-- it was a designer.
He had one games.
He's like I'm going to do it myself and try and make this work.
[INAUDIBLE] was another one.
There's a lot of these little--
[INAUDIBLE] Games just started like that.
Now they have whole conventions [INAUDIBLE]
They start with an idea, they do it themselves, and hey, it works out.
And then from there, they realize, hey, we should do a lot more of these because that was successful, and if we're successfuyl at it, we can make it work.
And after the one designer takes and build the company.
It becomes three or four people, and it becomes mildly successful, and they push out all the games that they've felt they can.
I made all the games that I want to setting out
Or they've used all their hours in the day and they just can't do it anymore
Well then once that happens, a lot of publishers will then start to look for other designers and publish their games.
Because you're designing your games, you've got your A idea, your B idea, and your C idea.
these are all great.
And once you publish all of those, you start to realize maybe those other ideas that you have are good, but maybe you'd be more successful if you could pick out some other ideas that other people submitted to you.
Really, that's actually better than what I would do myself
Or maybe a lot of times what will happen is as a designer you do have a lot of ideas and if you're a smart [INAUDIBLE], you realize that most of them are probably not good.
But that's fine.
That's absolutely fine.
Nothing springs forth from someone's head like Athena perfectly formed.
That is not how things are made.
All of your favorite musicians practiced forever to get really good.
It's just the way it is.
In game design, it's the same thing.
You have to make a lot of really crap.
There's a really good phrase about writing which is that every writer has 2,000 bad pages in them, and the good writers are the ones who got the 2,000 out somewhere else.
So you've got to keep that in mind.
But I think that the ability to find publishers is not diminishing, but the ability to self-publish is rising, especially, although we have great self-publishing tools now and the internet is really good for sharing things, the biggest thing that's happened in the past year and going into this year is 3D printing.
So while we've hit the head into what self-publishing is, we're just beginning to see what desktop manufacturing is, like people who are just running their own little factories in their houses, which is super cool
Actually doing that for my game
Yeah.
That's so cool.
I really want to do that so much
I think we have time for a question
[INAUDIBLE]?
It depends on the scale
[INAUDIBLE]?
Yes.
This was printed in China.
Like I said, just got off the boat.
So I worked with a printer on this because I've been doing printing and publishing for so long, I know the jargon and know the lingo.
Again, another thing that a publisher can be really good for is that you don't need to study to figure out all of the things.
They have made all those mistakes in the past for you
You don't need to screw up your bleed lines, your print cards and all of the-- they're all offset
If you've got the expertise or you can organize the expertise amongst the people you know, you can get past a lot of it.
But think about it, everyone who's printing stuff or making things, they want you to use their services.
Because they are a service industry, they're going to make it as easy as possible.
And if they don't make it easy, there's someone like four blocks down that is probably going to do it because yay, capitalism.
There is, I think, god, one [INAUDIBLE]
[INAUDIBLE] like buy a printer?
No, it is.
It's actually a huge expense to buy a printer.
The problem with printers is that they're super expensive to have and not use
It's better to have a team run it
Oh, yeah.
You want that machine running losing money because you'll lose less money than if it wasn't running at all
There are a decent number of American manufacturers, certainly for prototype-level materials
And in Massachusetts, like [INAUDIBLE] of Massachusetts
Specifically, Massachusetts, there's a lot of stuff.
I would say from what effort, nothing will beat China in terms of cost
It depends on the exact scope of your project and the number of units that you're doing.
So if you're doing a few units of a very complicated thing, you could actually probably get good prices in America.
If you're doing a lot of units of something relatively simple, you're right.
You can't beat it
Right.
And then oftentimes, some people will do piece mail so they'll get some printers for some things like the boxes, the rule books.
And you can pull that together, and that can be a nightmare
It can be.
But there are specifically services that you can hire.
They're called pick and pack warehouses where let's say I had a board game and it had components coming from the four corners of the earth.
I could get them all shipped to one place and then pay a person a quarter per unit to pack it all together and whatever.
And if I filled out my spreadsheet correctly, I will realize whether that is a financially viable thing or not.
So one of the things that I like about self-manufacturing is that I can figure that stuff out as I'm doing it.
So for instance, on early prototypes of Slash, I was printing all the cards and cutting all the cards and killing myself because it's a lot of cards to cut.
And so it digs back into the design.
Well, if I change the size of the card, I can use this machine to cut it, and it's going to save me a half hour on every single unit
And that's a whole other thing is design around like--
Your typical capabilities
Which you'll actually see the rise of super small games
Micro games.
Because they take a lot less to make
Yeah.
The biggest game you can create using 10 wooden chips and miniature dice and fit into a package about this big
So there's that threshold.
So one of the things I have with this is that when you do printing, you usually have one big, let's say for cards, one big piece of paper with a bunch of different cards on it then you cut it all out.
Now, if I'm making 24 copies of the game, it just so happens that I can fit 24 cards on one sheet of paper, which means instead of printing card A, B, C, D, E F, G, H, I, J, K, L, M, N, O, P, Q, R, S, whatever on that one page and then cutting it out and then sorting it, I can print A 24 times and then do that 400 times in a row, stack them all together, chop it, chop it, chop it, chop it, chop it, chop it, chop it, chop it, chop it, and I don't have to sort me thing anymore.
I have cut hours off of that pick and pack time.
And now multiply by the number that I just got out of it.
Instead of making one game every half hour, I'm making 12 games every 15 minutes or something
I just want to go back again to the China versus the States as well as I'm finding a lot of kickstarters.
Realize that the margins they need for small amounts, they have to go to China if they're going to break even.
But if they exceed goals or go much and they can do larger quantities produced in the States, then they use the same margins.
So a lot of people will, if they can get a game big enough, will try and have a stretch goal, bring games to the United States if they can sell enough of them
This is a challenge that is unique to analog games, obviously.
You don't need to worry about where you're getting your cards cut, I guess unless you're making some kind of Pokemon AR game.
That would be super cool.
How do they have that?
They've got to have that, right?
Do they actually interact with each other?
They should.
Because last year they did a kickstarter for Goal Marcana, which is like an app and a miniatures war games where your dude has a QR code on it, and then it uses the app to do all the combat
There's a [INAUDIBLE] tradition of Japanese arcade games that reads cards probability
Oh, god.
I remember that
Mario Kart of our games, I've got one.
That's great
[INAUDIBLE]
[INAUDIBLE]?
Yeah.
[INAUDIBLE] We're going to play until.
Thanks for coming.
One second.
Thanks again for everybody.
So we're going to spend the last two hours of class.
Well, first we're going to have to take a break, check in with y'all.
Did you do that already with games?
We'll check in with y'all to see how you've formed into teams yet for that project.
And then we're going to be playing-- do you have a couple of your games?
I brought I think eight different games that I've made
Well, great.
So we're going to play some of those games, and we're all going to play, we've got three games that are related to the assignment that we've talked about
Also if anyone is interested, I'm going to go ahead and leave some of business cards up here.
As I go through the process of making my game through Kickstarter, I'm basically going to be posting my experiences every step of the way from design to manufacturer.
So if you want to watch someone struggle immensely through the process and watch the mistakes that I've made and laugh immensely.
It's going to be hilarious.
Just grab a business card.
If you're interested in playing Croquet and also murdering each other, this is definitely the game for that
And so just to show you guys, these are the three phases of prototypes.
Well, these are the two phases of prototypes that I went through
They each have VHS cassettes
Yeah.
And so here's the trick.
The reason that it's in a VHS cassette is because this will fit into the smallest flat rate priority mailer
Oh, really?
I can get any game that fits in this size anywhere in America for $5 in two days.
And you can't beat that.
And so if I wanted to get a lot of these games out and not get a second mortgage on my house because I'm not an idiot, I designed that.
So I actually do that with all of my monthly games
And the cases are probably--
The case are dirt cheap.
They're like $0.08 each
Yeah, because nobody uses them anymore
Yeah.
No one needs it.
So I found a place that is still selling them.
So supply chain stuff
So [INAUDIBLE]
No, I did not.
I don't know Maybe I did.
My artist is supposed to be getting it to me today
Thanks for coming, guys.
What are you guys doing?
Oh, thanks.
That was great
[INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
--have your attention.
I'm just going to go into the lecture part.
I was hoping to start with the pitches, but since so many people are missing, we'll wait and do the pitches after the lecture part of class.
And then we'll end class with a chance for you to play games based on the lecture.
So it's gonna be kind of a weird lecture, talk about your game, play games with it, or lecture, and then time for you to talk in your teams about your game again and to continue working on your projects.
We'll bring in the prototype kits for that last bit, so you can start building stuff.
So today's reading was [INAUDIBLE]-- where's my copy of [INAUDIBLE]?
Hey folks.
So today's reading was I think chapter 11 and 12, was that right?
Space control and [INAUDIBLE] And the reason why I wanted you to take a look at these, is because it's got two sort of historical investigations-- well, it's got a whole bunch of historical investigations, a bunch of different games.
But it kind of goes in depth into two particular games, one in each chapter.
So people, remember which game is covered in the reading?
No
No.
Really close to that, but you're one letter off
Go?
Yes, Go.
And the other one was--
A chase game--
Chase game called--
[INAUDIBLE]
Tafl.
The second chapter that I had you read today starts with like fox and geese and a bunch of things, but it really goes in depth into tafl.
And this is probably the most in-depth sort of historical research that you find in this entire book.
Most of this book is like the stuff in between Go and tafl.
Where it's like, four paragraphs at most on any particular game, and it's just talking about variants, right?
He talks about reversi, which is, what, 200 years old?
Something like that.
I can't remember the exact number, but he only gives like maybe two or three paragraphs on this history.
Everything else, is just this is how the game is played.
These are the mechanics.
And so that if you wanted to figure out how this game-- all the things that could come up from the game, you could recreate it, just based on the illustrations and the mechanics.
But when it comes to Go, and when it comes to tafl, it's a little bit more about the culture surrounding these particular games.
And that's a very different take, and an interesting take.
So if you are interested in doing sort of academic research in games, that's one way.
Kind of like game archeology, game historian work.
One of our grad students, who's now doing his PhD, Jason [?
Bakey, ?]
he's kind of doing that for train games, and evolutional train games.
Even though, obviously those couldn't have been invented before trains existed.
So they're not that old, but there are tons of them.
And he's again, tracking is the evolution, but also of the culture, the people who play these games.
So let's see, how many of you play Go?
OK, about five people.
How many of you have played a version of tafl, enough to hitch [INAUDIBLE] tafl.
I have no idea how to [INAUDIBLE].
You've played it?
Yeah
On paper, on iPad?
[INTERPOSING VOICES]
[INAUDIBLE] I have a board, [INAUDIBLE]
Oh, cool.
OK. You can actually find craftsmen making these things, tafl boards.
Even though the rules, as if you read the book, the rules aren't really well-understood.
There's no canonical explanation of here are the pieces, and here are the rules, here's what you do with them.
Because if you find a little bit about one, you find a little bit about the others.
So there are some archaeological evidence for here are the different pieces for a full set of tafl, and they don't tell you what to do with it.
And then there's another set that's just like, just what you do with all the pieces, but it seems to be describing a different set of pieces.
Or a different number of pieces, and you're not quite sure.
But you get a lot of stuff like poetic records of the significance of the pieces.
That gives you a clue about what people thought of these pieces, and gives a clue of what these pieces can do.
And again, there are some descriptions of particular plays of this game, which gives you an idea of how these pieces can be made.
But viewed within sort of larger context of his asymmetric board game, that is largely involving one group of pieces trying to capture either all or a particular group of the other players.
That's basically the whole genre of the chase game, of the fox and geese type game.
And viewed from that context of here are all these other games that have survived a little bit better Like the English fox and geese, this is what we can intuit.
It is kind of a siege game, where both sides have soldiers, but one of them is explicitly trying to kill the king.
But one side doesn't have a king, and is trying to kill the king.
And the other side has a king, and a bunch of bodyguards, and they're trying to defend the king.
The whole idea is sometimes it's to get the king to an escape route, sometimes it's just survive a certain number of moves, or to get to a particular point on the board.
And one thing that I would like you to think about, if you haven't read it, do read through these two chapters just to get a slightly better idea of what you can do with space.
It's one thing that we've been noticing, that in some of the games that have been prototyped in this class, space isn't always used very creatively.
We see games that are being played out on grids, we've been seeing games where you can move freely throughout the whole grid.
But then, what does each individual square actually mean in relation to other pieces on the same board?
It's something that these games actually do really well, both the games of territorial capture, which the goal was all about.
And the chase games, which are these asymmetric games where you try to surround your opponent.
Go and tafl are not the only two games that describe in here.
There's a huge amount of variation and thought about what you can particularly do when it comes to manipulating space, being surrounded.
What does it mean to have two of your opposing pieces on either side of you, or at the l-shape of you.
What does it mean to be on a square grid?
Especially in a game like in Go, it's a game that's played on squares.
On the corners of all squares, in particular.
And it's very, very important that what's to the top, and what's to the bottom.
And what's towards the left, and to the right of every piece is really, really important.
But what's to the diagonals isn't really so important, until you consider meta strategy.
What's the effect of a clump of pieces, rather than a single piece.
The other thing about the section on Go, in particular, is that you get this huge amount of vocabulary.
Words like liberties, the way how scores are being added up.
Words from different languages.
Korean, and Chinese, and Japanese.
But most of the research in this particular book refer specifically to the Japanese tradition.
So most of the terms are used in Japanese.
How many of you heard of Atari?
Come on
You said Atari?
OK, all right.
That's actually supposed to be a derivation of atari, which is a Go term.
Don't ask me what it means, I don't actually play Go.
But these words have kind of worked its way into even cultures that don't play Go that much, like in much of the Western world.
In the same way [INAUDIBLE] has made its way into talking about games of chance, because that means dice in Latin.
Die, specifically, in Latin.
It's in the same way that sometimes we talk about being checkmated, even when you're not playing a chess game.
I'm trying to think of other chess terminology that tends to make its way into common English parlance
[INAUDIBLE]
[INAUDIBLE]
Yeah, [INAUDIBLE]
The least worst move.
That's a new tone.
I don't know if they're specifically from chess.
I'm thinking more about like in-- It's a concept that applies to chess in particular, a lot.
But not--
[INAUDIBLE] it's probably a term that derives elsewhere that's [INAUDIBLE] in chess.
But do you think about tempo in games?
Well, I think about tempo in games.
I don't really think about it in chess so much, because I don't play chess.
But I've heard chess commentators talk about that a lot
This isn't really specifically chess necessarily, but a lot of [INAUDIBLE] can say your move, after you're doing something, anything.
Usually moving.
[INAUDIBLE]
There's still a general cheating the world as a game.
Very Sherlock Holmes-y kind of, I have such a high intellect that the world is a game to me, and this crime is a game to me.
Something like that
[INAUDIBLE] I'm just thinking [INAUDIBLE]
I'm just trying to think of all the chess--
[INAUDIBLE] material.
[INAUDIBLE] the word material, as a concept of [INAUDIBLE]
[INAUDIBLE] position where you're kind of engaging, but you probably [INAUDIBLE]
Yeah, I think that was a terminology that came from outside chess and that applied into chess.
But I've definitely heard checkmate being used outside of chess specifically
Oh, what about stalemate?
Oh yeah, stalemate.
Stalemate is a situation I generally do associate with chess more than any other game.
But a stalemate that happens in a business situation, or in a negotiation, both of us have no way to be able to gain an advantage upon each other.
So that's a good way that terminology from something that's been as old as chess, has been as old as Go, we haven't preserved much from tafl unfortunately.
But it can work its way into sort of common parlance, and that's something that he does in the analysis of Go that I don't see in any other analysis inside this one particular book.
But you do find it in other people's writing.
There's also a lot of talk about who play Go.
Right now, what is the difference between the Japanese culture of masters.
Are like few, almost more scholarly than scholars, kind of rarefied elite, who are assumed to be the absolute best players of all, of Go.
At least in Japan.
Although, that [INAUDIBLE] the best player of Go in Japan, you're kind of the best player of Go in general.
But then, in Korea, you have exactly a 100 person professional association.
Professional, as in you are going to make money playing this thing.
There's like a broadcast culture around it, kind of like e-sports before the e. [INAUDIBLE] game on TV, right?
And there's a lot of hype, there's money involved in it.
And becomes this rarefied, grand master class that you're trying to get into.
Because once you're into that class, then you get to make the big bucks, a TV personality.
In Chinese culture, it's a little bit more of a very historic thing.
It's this thing that you could play, but you're playing it for historical reasons.
It's not a concept at quite as contemporary a game as mahjong, for instance.
Which is very much a social game, this is a game that you play with friends.
It's like a game of poker when you're playing a game of mahjong.
[INAUDIBLE] doesn't have that kind of cultural resonance.
Even though, it's the same physical-- more or less, the same physical game as Go.
I think there's some minor scoring changes, but otherwise the play of the game is the same.
So the evolution of how this game started off as almost a spiritual activity.
It was something that was promoted by monks, as something that helped meditation.
And then moved its way from that into royal courts, and then eventually, the rest of the population.
For people who aspire to those kinds of high, lofty, social positions.
It was seen to be a sort of activity that really smart people will do.
And then, if you can demonstrate that you're really smart, then you can get jobs for really smart people.
Which apparently pay pretty well in Asia.
Then if you've got the actual description of how the game is played.
But even in this particular write up, the way how they describe the play of the game goes beyond the mechanics very, very quickly.
Because the nice thing about Go, is that the actual rules of Go are pretty darn simple.
The idea of Go is pretty darn simple.
There are black and white stones, each player plays all black or all white stones.
You play on the grid.
I'm not going to draw the exact number of lines, but-- it's that sort of thing.
You play on the intersections.
And if you've got a situation like this, where you've got one colored completely surrounded by the other color on all four sides, this piece is eliminated.
It's captured, it's taken off the board.
And you can't repeat the same board [INAUDIBLE] immediately.
Like if I have a weird situation where I've got a black piece here, I've got a black piece here, I've got a black piece here.
I think that's right.
And then I immediately place a black piece here, so I captured this piece.
Your opponent can't then put a white piece here, and capture that piece.
Because that returns the board back to the state that it was just a couple of seconds ago.
That's basically the rules.
There's a whole chunk of rules on how you figure out who won, by basically scoring things.
And I've seen students show me many different ways of scoring that, some of them pretty elegant, some of them pretty clumsy, but they all add up to the same numbers.
So it's basically a lot of different algorithms to get the same result.
But in the end, the basic idea is just surround your opponent.
But once you get from that into all of the higher level strategies of the-- OK, out of this game mechanic, what are all of the things that you have to do to be able to play out this game to your own advantage.
You start going into things like, how important is it to be able to capture the corners of the board, for instance.
What are the typical opening moves that you see in this game?
Where in his same analysis, in the same chapter of reversi in Othello, it just largely comes down to, are you playing your pieces like this.
Or are you playing your pieces like this.
Or do you not have the choice at all, because you're playing Othello, and that's what you started with.
That's basically the entire analysis of your opening moves of Othello that he gives.
But then when you're go into Go, there's a whole section on why the opening moves have such a long term impact.
And like chess, there is an opening game, there's a mid-game, there's an endgame.
The end game is kind of like, no real surprises at that point.
You're just kind of rounding up what was developed in the mid game.
The mid game is where all the really sort of strike from behind, caught you by surprise stuff happens.
And the beginning of the game is just setting up those situations so that that kind of thing can happen.
I think it is nice to be able to try to imagine what a game that has been around that long, that can develop all that richness.
This cultural richness, its game mechanic and strategy richness.
That sort of linguistic richness-- how nice it will be to be like the person who designs the game like that.
Except no one will ever remember you, which is kind of sad.
But also, what would that kind of game have to have?
And I like also indulge in a little brainstorming.
It's like, if you wanted to-- if it's explicitly your goal to work with a team, to try to develop a game that people will be playing 1,000 years from now.
What would a game have to have?
What do they need to be?
It really has to be just like chess or go, in the sense that it has to be simple to learn, and infinite strategy.
It has to be entirely unbreakable
Unbreakable?
As in--
As in no computer could ever solve Go.
In the history of humanity, it'll just probably never happen
Well, no computer's solved it, yet.
I'm not sure if anyone's proven that it can't be solved
No, [INAUDIBLE]
Yeah
Probably can.
The numbers are astronomical.
The chess numbers are astronomical.
[INTERPOSING VOICES]
The test numbers are huge
[INAUDIBLE] no way to simplify the problem.
If you're saying [INAUDIBLE] can't be solved, [INAUDIBLE].
Rubik's cube can be solved [INAUDIBLE]-- any Rubik's cube can be solved in 20 or fewer intervals.
[INAUDIBLE]
They actually sort of just figured out [INAUDIBLE] and then solved all those pieces.
[INAUDIBLE]
So again, that's hard to reduce into a simple form, which then can be solved.
It gives you an idea that there's always going to be at least a minimum a level of complexity
[INAUDIBLE]
Yeah, I can imagine that.
But there are sports that have been around for a real long time, and you know, I try to think of one that's been around for 1,000 years.
I'm thinking javelin probably has been around at least that long?
Those are dexterous games.
But [INAUDIBLE] make a board game, you're gonna make a board game to last the length of time, it can't be something like a flicking game.
I've played flicking games that have been around, but maybe not 1,000 years
[INAUDIBLE]
How long has chess been around?
Someone with Wikipedia should answer that question
[INAUDIBLE]
It has gone through several different versions.
Like there's a time where the queen was a vizier.
We're talking about versions of the game that existed in India, and in the Middle East
Sixth century
Sixth century, yeah
[INAUDIBLE]
I think the most recent version-- the most recent change I think, would be the power of the queen.
And that was Queen Isabella?
Kind of, it's like mid 1400s, early 1500s in Spain.
[INAUDIBLE] but that's about when she gets the--
The zoom right across the board thing
[INAUDIBLE]
Yeah.
And I'm not so sure.
There are a lot of iterations over how a tournament needs to be played out, the long details.
But then things like-- basic things, like how does a piece move.
I think the biggest change was probably the queen's.
So I was talking about like 1400s or 1600s, but then that's a 200 period.
When you have a bunch of really powerful European queens, right?
That inspires this
The game has to be very, very, very balanced
Very balanced?
As in, both sides has a good chance of succeeding?
Yeah
[INAUDIBLE] but it's not like--
I don't think [INAUDIBLE] is saying that it has to be exactly balanced.
Just pretty darn close
[INAUDIBLE]
The 55-45 is not close at all.
I [INAUDIBLE]
[INAUDIBLE] [INTERPOSING VOICES]
Play a [INAUDIBLE]
So consider the concept of grand master play [INAUDIBLE] in tournaments where they're playing [INAUDIBLE] they're not going to play to win every single game
[INAUDIBLE] something along the lines of like, right now, a vast majority of games are [INAUDIBLE]
So think of the people behind chess games, why would that be [INAUDIBLE] against [INAUDIBLE]
Go being black is supposed to be a six point advantage
I wondered, does this game have to be competitive in nature?
Does it has to be competitive?
Can you make a cooperative game that--
Think are competitive.
We still don't know the rules for [INAUDIBLE] Ur is before senet
All we have is the board
Senet, I think, was solitaire, in one version of it
No rules exist
[INAUDIBLE] I was wondering if the game happened to be two players [INAUDIBLE]
Again, let's think, not necessarily Go and chess, but also things like athletics, sports, right?
All these things that we associate with games, rather than just a pure test of strength
Sorry, but I thought each course it tends to be [INAUDIBLE] so that's sort of an abstraction.
But not necessarily [INAUDIBLE]
I'm thinking, what's the oldest team game that we can think of.
Hurling?
The one [INAUDIBLE] balls, [INAUDIBLE]
Is that what people associate that with basketball?
I don't know, [INAUDIBLE] basketball, I just [INAUDIBLE]
[INAUDIBLE]
Yeah, I think rolling on the ground-- balls rolling on the ground are a little bit more common.
So I'm gonna get [INAUDIBLE]
So I was gonna say, sort of based off [INAUDIBLE], it seems like it's helpful if it's the sort of game where you can build up a culture of competition around it.
It seems that seems to help keep things rolling
So it implies that there's some type of skill that you can develop, and then you can compete against other people
[INAUDIBLE]
Let me get Laura first
[INAUDIBLE] look at any evaluation six and a half points
So for--
[INAUDIBLE]
According to what [INAUDIBLE] percent for white.
But it said that, and this kind of makes sense, if you're like novice players, it's pretty much [INAUDIBLE].
It's much closer.
Which makes a lot of sense.
And it also means it does affect people in some way to like try to learn strategy, because [INAUDIBLE]
The more you know, the better it is for you when you start off as white, as opposed to it being just like a completely [INAUDIBLE] thing
For a lot of these, I think a cultural incentive can help a lot.
So for both these games, you were basically considered intelligent if you could play it well.
And then for a lot of the sports, you were-- if you won the Olympics in ancient Greece, or something.
That was a huge deal.
[INAUDIBLE] cultural significance could really help you develop more of a-- an entire population might try as opposed to a game that doesn't really matter
So what I'm hearing is either you'll end a war by playing a game, that's one way to get somewhere, like real cultural significance.
And a lot of athletic events, the Olympics are kind of about that sort of thing right.
Or you get a king to play that game.
[INAUDIBLE] Which could work, definitely chess has had intellectual leaders.
You know, artists for instance.
I try to remember which authors were really, really into chess
[INAUDIBLE]
Yeah.
I wouldn't be surprised if military leadership were into that.
I'm actually thinking that maybe instead of balance, what we're going to say is that each side has a fighting chance.
It's not so off balance that games often end in foregone conclusions.
Even when black has a disadvantage in chess, black can do something.
Like force a stalemate.
And that gets that both player's agency over the outcome of the game, even if what they're trying to accomplish is different.
White is trying to accomplish victory, black is just trying to prevent it
In chess, actually, the most recent rules changed [INAUDIBLE] 2001
2001?
That's a while ago
[INAUDIBLE]
Yeah, it's pretty stable.
I will say that once you've been around that long, you've had enough tweaks to your rules that things can work out.
People are still changing the rules of things like American football.
That's--
[INAUDIBLE]
Sometimes those rules changes are for things like, well, we need to make it easier for us to insert ads into this game.
And in football and such, it was just like that.
So whether that makes it have longevity, I don't know.
But it certainly [INAUDIBLE] the current health of it, it works with the current economics of times.
I don't know if Go and chess ever did that, but but maybe that's why the intellectual-- if you can go into the actual cultural value, you can go for commercial value.
That works too, right?
And you know the idea is, if you want to make a game that is really, really going to stand the test of time.
Either as a full game, or a sport, or a board game, card game
I guess now, it doesn't have to be simple learned, but it should also help make it simple to teach.
Pass on the game really easily
You've got to get generations to pick it up over time
Especially where it's a board, then two different colored rocks.
So even if you don't have manufactured board, you could teach someone
Like a [INAUDIBLE] or on the whiteboard
Or like, pennies and quarters
Yeah, exactly
And I was going to say basically it also helps a lot where it's simple.
Because obviously that means that you [INAUDIBLE], and that it adds extreme amounts of value, the ability to pass it on
In sports, you are sort of like it's-- you are sort of constrained to essentially having a large enough space, flat space, that you can play a game.
But assuming that's something that's fairly easy to find around the world, maybe increasingly hard as time goes on.
Something like soccer is very easy, right?
You know, it's like you don't really need a goal, you just need things to mark goalposts, and corners.
And then you can play.
It's probably a little bit harder to do something like baseball, where not only do you need to be able to mark out the playing space, but you've also got to have all the right-- a stick and thing that you can throw without hurting anybody.
And theoretically, you could play baseball with a rock and a stick, but you know, that game's probably not going to catch on.
Maybe it does
Building off of [INAUDIBLE] old objects and simple the game [INAUDIBLE]
It is amazing how that's worked with chess, because chess has so many different pieces.
But you know, you've seen boards being made by people who were basically hermits, handcrafted stuff.
And pieces that just like made out of like-- people who were like survivors of a plane crash [INAUDIBLE] waiting to be rescued, and things like that.
You can actually reproduce the border pretty easily.
So it can just use pieces to be able to reproduce that.
I wonder at what point complex becomes too complex.
Because if chess is just on the right side of just easy enough to reproduce, I can't imagine it gets much harder, much more complex than chess, to get to the point where we just can't reproduce anymore
Mahjong or shogi
Oh, mahjong pieces.
Oh, good Lord
There's something about them that's going on.
Chess piece doesn't quite-- [INAUDIBLE]
Chess is kind of nice [INAUDIBLE]
This is sort of like--
There's like an accepted aesthetic for it, but you could play it with cut out pieces of paper into different shapes
Right.
Or you just draw the icons on it.
Or you can do the Lord of the Rings chess set for instance, where instead of a king, you've got Gandalf, or something like that
Another game as an example is mancala.
[INAUDIBLE] century.
Another one that you can play with rocks
Mancala's a great one.
I think I [INAUDIBLE] lot about mechanics, but I wonder if it's a cultural or symbolic ritualistic that is also required?
So like mancala, you've got cultural [INAUDIBLE], or so it seems.
I learned how to play it in Bible study school as a kid.
I forget exactly what the message was, but it was [INAUDIBLE] [LAUGHTER]
Something seems to [INAUDIBLE]
It's that [INAUDIBLE] cultural stuff that is built into religion there.
I mean, chess kind of has war.
We really saw that in the '70s.
But what is go got-- what does this say?
[INAUDIBLE] Taoism?
Yeah it's this sort of Taoist monk's kind of thing.
So there's a certain amount of spirituality that's associated with Go.
There's a lot more spiritual writing that I've seen on Go than on say, military strategy writing.
Whereas chess, is very much a military strategy thing
The Egyptian game of senet.
You put the board into the Pharaoh's tomb for the Pharaoh to play after going into the afterlife.
It's not played anymore
How do you know?
Maybe they are still playing it
But there's no current cultural significance.
At one point, it stopped having cultural significance, so it stopped being played
Yeah.
I mean, that's a game that people have theorized is actually about a journey into the afterlife.
So it has cultural resonance for a society that thinks very heavily about what happens after I die, right.
And chess makes sense for a society that's either constantly in battle, or offered in battle.
And there was a resurgence in the '60's and '70's here in the
Probably had to do with Bobby Fischer
Also that.
Also that.
Celebrities, again-- it's nice to have a king play your game game.
All right.
So a couple of things just to think about, we've been talking in class a lot about [INAUDIBLE] games.
And you know, especially the discussion about the Parker Brothers.
And what it will take to be able to make a game that's going to sell.
The last time, we had a guest in here to talk about Kickstarter, or Patreon, and Crowdfunding.
How to put a game in a box and ship it to people.
But then also, the other direction of what if you just try to make a bigger set of rules that anybody can-- with a ball and a field, or a chunk of wood that had a good curve.
A bunch of different colored stones, or holes in the ground could reproduce.
And what's the design process around designing those kind of games?
It might be that's a perfectly legitimate application of game design.
And so you might want to think about that.
And now that everybody's here, let's hear about your games.
Who needs to use the projector today for the pictures?
You don't have to, we can turn it off.
No?
OK. All right.
[INAUDIBLE] shut the projector off.
And can the ship racing team come up here?
And we're going to give you feedback on your pitch.
We're also going to give you feedback on ideas about tips on the game that you're pitching.
So imagine that Rick and I are the decision makers of the publisher that you're trying to get money from.
You're trying to get seed money to be able to finish designing this game that you're currently designing.
Maybe even employees in our company.
We're the heads of Hasbro.
You're all designers in Hasbro.
And you're going to say, this is the game we all want to work on next.
All right?
Everybody else is kind of like your competition in the company.
Yes, but we're not really doing that.
They all want something too.
So give us that pitch
Let's do it.
So as you know-- well first of all, thanks for having us.
[INAUDIBLE] run through the team real quick.
[?
Fro, ?]
Jory, Matt and Joan.
And we are team ship racing.
And so we're here to tell you why ship racing is going to be the number one game for Phil Rick Studios.
And as you know, games like Agricola and Tsuro are some of the top grossing games in recent history.
So since we're passionate about Euro style games, and we love path building games, why not do a mash-up of the two kinds of games?
So that's for-- enter ship racing.
A mash-up of Tsuro and Agricola.
So let me walk you through our vision for this game.
There's gonna be a common board that looks something like this.
We're experimenting with [INAUDIBLE] size, but right now, five by five is pretty fun in early prototypes.
And this is the common.
And then you have tiles that go on the spaces.
And each tile-- here, I'll blow it up for you-- has two nodes on each side.
And there will be multiple permutations of paths like this one.
You've got you guys that played Tsuro?
Tsuro [INAUDIBLE]?
A So these tiles will go together and will help navigate your ship.
Your ship starts here, and you play this tile, and it ends up here.
So the idea of the game is to start here.
It's a race to the finish.
First person that can go from the left side of the board to the right side of the board is the winner.
The cool part is that each player is gonna have-- this is gonna be a zoomed in view of the ship.
So this is where the worker placement Euro-style turn based game comes into play.
So on your ship, it will be a simulation.
You're going to get a really good understanding of what it's like to be a ship captain.
You're going to be able to steer the ship, man the sails, act as a navigation using the sun sky horizon as tools.
You can populate the crow's nest, you can be the cook, swab the deck, you could retreat to the captain's quarters to sit with [INAUDIBLE], fish off the side of the boat to get food for your crew.
And we're not quite sure how, but these are the two elements of game-play that we want to mash together to really crush it for you guys next year.
Is there anything I'm missing?
Focus groups loved in tests
We have 100 pre-orders already.
We'd love to hear your feedback.
Let's see
I'm just going to give you feedback not playing the character that I [INAUDIBLE] I think that being ship captain is actually the center point of this.
That's the whole [INAUDIBLE] trying to get people excited about.
The actual game mechanic of everything, like putting the tiles together, intellectually and designerly very interesting.
But it's not the same [INAUDIBLE] to get [INAUDIBLE]
So there's like a hole in the market, you can't be a ship captain in any other game, but we're going to give you a chance to
I think you don't even have to do that.
I think to sell the fantasy, to sell the fantasy of this is the age of sails, or something like that.
You are a ship captain for the East India Company.
Sort of create the fantasy that [INAUDIBLE].
Like Ticket to Ride, you are the globe, continent trotting, aristocrat, basically.
Even though really the game is nothing about that.
But that is the fantasy that they're trying to sell would be try to sell you the box.
And of course, that is a focus you will need to better flesh out-- what the decision that you're doing on the road, how that influences your progress on the map.
Because it was unclear.
It looked like the tile assembling thing, it's already game all by itself, like Tsuro.
You didn't make it very clear what drives what.
Is it the map-building that drives the ship [INAUDIBLE] directions, or do the ship [INAUDIBLE] give you the ability to use on the map of [INAUDIBLE]
I hear you
I'm curious, what are the primary and secondary sources you're using for how ship captaining works.
Why is it that this is-- how do you know that you're actually showing the experience that this is from the perspecive of this person?
What's the date?
What's the range of dates?
What's material you're using, the background material you're using to [INAUDIBLE]
These are questions that we're asking ourselves, too.
[INAUDIBLE] making it explicit, because we really need to dive into the research.
We had a lot of game ideas and now we're just choosing this and we're gonna run with it.
So I think that's a good next step, thanks for helping us with that
[INAUDIBLE] I think so.
[INAUDIBLE] I think [INAUDIBLE] would be [INAUDIBLE] You could do a lot worse than just saying, that sounds awesome.
[INAUDIBLE] British navy, you are one of the boats that has carried cargo at the same time [INAUDIBLE] It helps that every one of those boats has a map, has a cross section of the ship, [INAUDIBLE] explains [INAUDIBLE] rest of the book is just use of terms.
[INAUDIBLE] which symbol [INAUDIBLE]
We're going to have to beef up our research.
Never been on a sailboat, personally.
The fantasy is something that people can relate-- [INAUDIBLE] guys
Great competitions.
[SIDE CONVERSATION]
Thank you, thanks for your feedback.
[APPLAUSE] [SIDE CONVERSATION]
I want you to close your eyes for a second.
Rick.
May 28, 1953.
Your name is Sir Edmund Hillary and you are 300 feet on top of the world.
Philip.
Same date.
Your name is [INAUDIBLE].
You have walked-- looked at this mountain your entire life, and wondered what was at the top.
For the first time in the history of your people, you're going to actually [INAUDIBLE] by your face.
You spent the last week climbing this thing.
You're finally at the top.
You can open your eyes now.
This is the experience that we're going to create with Tenzing Norgay.
We want to show people what it was like for these two men that had no idea what was going on.
There was a second attempt to climb, the first two died in the attempt.
And after 12 days of climbing, they set up base camp in each spot, and reached the top.
We're going to make a three-dimensional board with 12 different base camps.
I'm not gonna draw 12 base camps right now, that'll take a little bit.
But each level essentially is a base camp.
If you've ever played the game Deception-- Descent, there is one kind of dungeon master type player.
Who, it's not like Dungeons and Dragons dungeon master, who creates an entire game.
But he controls the monsters, controls tracks, how they track, stuff like that.
We're gonna have a similar element with the environment.
The environment challenge that you face various equipment failures, and just the unplanned for things that happen during the climb, will be laid out by this environment master.
It will be him against the two climbers on their way to the top.
There will be different boundaries placed on you, based on the stuff that they had to fight for.
The last two base camps, you have to ascend in 24 hours.
If you don't, you will die of oxygen asphyxiation.
By the same logic, when these players pass the [INAUDIBLE] base camp, they will have called three turns.
We'll have to work on the details to reach the top.
Things like this can make the experience realistic, make them feel like they're actually on the climb, on the mountain, trying to reach the top.
Become the first person in the world to make it.
Simple enough, if you make it to the top and get down alive, you win.
You go down this street, you get knighted.
Sounds good?
That, my friends, that is [INAUDIBLE]
Source material that you're looking at right now?
I found a couple websites on just the climb.
I also read a book two years ago, not necessarily about this climb, but on the 1996 Everest disaster.
Which goes into a lot of detail on what it takes for a pair to climb Everest.
The challenges that people face, how they feel while they're climbing.
And then, obviously, the worst case scenario of what can happen, when you get caught on base camp four in the middle of a blizzard and 15 people die
[INAUDIBLE]
I planned on doing that the entire time, and then you said [INAUDIBLE] [INTERPOSING VOICES]
If you have a strategy going in, always volunteer to go first.
[INAUDIBLE], about your style of pitch, and more about the things that you said in the pitch.
3-D board-- Really, really think about how you're going to be handing that in on [INAUDIBLE].
Come in with a box that can actually fit this, give us instruction about how we're supposed to assemble this thing.
Make sure that it will hold up to repeated assembly and disassembly.
Possibly something that may not be essential for your game, something that [INAUDIBLE] which is a really interesting--
We met yesterday to talk about it, and I said I was thinking about a 3-D board, if that would look cool.
And both Matt and Eddy both said they had the exact same thought.
So I was like, OK. Then we have to do it
You might actually want to incorporate the board in your box somehow, so that the box becomes something that can actually be used to support the board.
[INAUDIBLE] it's up to you, really.
But do consider things like cost and time, [INAUDIBLE] There is actually a genre of mountain climbing games that looks like two [INAUDIBLE] with ropes [INAUDIBLE] straight from [INAUDIBLE] attached to the-- and they both attach to one weight.
The board itself, [INAUDIBLE] quarter of the board are slightly inclined and are [INAUDIBLE] there [INAUDIBLE].
So the whole idea is get from point A to the top, to the other side, without falling into the hole.
And each player holds onto to a different rope.
[INAUDIBLE] A couple of ideas [INAUDIBLE]
I think for me, if it is Tenzing Norgay, I would've wanted-- hinted a little bit towards this character [INAUDIBLE]
Mainly, just put that out there because it was the first name that came to mind.
[INTERPOSING VOICES]
So be very clear if that's what you're going to go for.
There's actually some really interesting stuff going on there, [INAUDIBLE] He's Tibetan, [INAUDIBLE]
He's a local Sherpa, I have I have his Wikipedia page
He's a local Sherpa, he's supporting a European--
Nepalese
To support European climber.
Europeans coming in with money, he's coming in with knowledge of the area, how to survive, and things like that.
A really rich amount of material to dive into, that may or may not be useful to the game you want to make.
So just choose which-- if you're going to do that, choose why you're doing that.
So an alternative would be [INAUDIBLE] for the North Pole, South Pole, [INAUDIBLE] Basically, two European guys are going to find the North Pole, or South Pole [INAUDIBLE].
They are competitors trying to get there first.
Is your game to competitors trying to get there first?
[INTERPOSING VOICES]
Two situations and the one we decided we liked was two people working together against one environment person
Great, OK.
Cool.
I just wasn't quite sure about that, so that's really great.
That's really interesting
I like the idea of playing up exactly who these two people were.
Rather than two [INAUDIBLE] this is offset.
[INAUDIBLE] can buy things.
Maybe not in the middle the mountain.
[LAUGHTER]
Actually, that's a really good thing.
How much of this [INAUDIBLE]
You can start the game off at the base camp, or something.
And you'd start out with a certain amount of money.
And have to buy the various materials or something like that
There's a lot of stuff you can go in there.
The 24 hour limit for oxygen asphyxiation, really interesting little detail.
So really think about the levels of abstraction you're using, and how much you're abstracting away, how much [INAUDIBLE] Some of this, if you're going to do this 24 hour limit, then that might mean that you're focusing on the climb itself.
You're not worried about how you got there.
You're just setting the player up with all the materials they need to do it.
Just be consistent throughout the game.
What level of abstraction are you looking at?
It's interesting that you brought up [INAUDIBLE] because that implies not only do you have to get to the top, you have to [INAUDIBLE] down.
And possibly getting up to the top isn't the end of the game.
And that again, comes back to [INAUDIBLE].
One of the things that he does, is make sure that this thing gets published.
[INAUDIBLE] The environment does not necessarily need to be [INAUDIBLE] it could be another player.
You could make it a two player versus one player game.
[INTERPOSING VOICES]
So when you said when you're positioning as a GM, the way we understand it, is that the GM's really disinterested.
[INTERPOSING VOICES]
I compared it to the set where you're playing-- you're actually playing against the other person
There's more than rules enforcement then.
Then you're actually working against--
There's a set of rules that everyone has to learn.
Specified not like Dungeons and Dragons dungeon master, but where you're actually competitive against the other player.
And there's a set of rules that you both have to work with
You might as well call that Dungeons and Mountains.
Thank you, thank you.
[APPLAUSE]
[INAUDIBLE] of which [INAUDIBLE] [SIDE CONVERSATION]
I'm Liz
I'm Laura
Michael
And we're pitching [INAUDIBLE] game [INAUDIBLE].
So a very relevant problem in modern day, is the lack of women in the tech industries.
And how they're being generally harassed, and gaming culture, and things that are heavy tech-wise in general.
And we're trying to address that by exploring that boss employee dynamic that women in the tech industry currently have to deal with.
And very much the reason why women are averting the tech industry every day, 'cause it's just easier to do other things.
So we're planning to do a live action role play to explore this relationship, where one player plays as the boss.
And he has the ability to create a new rule every minute, or something like that.
And one player plays as the employee, and she has to follow all the rules that the boss says.
And maybe her ability to be productive is affected by certain phrases that the boss says.
Which he's required to say, but he isn't sure-- he doesn't know that they might affect the person in this way.
They are just on the character sheet.
It would probably be a minimally re-playable game, unless we introduce new characters and new scenarios.
And they'd be modular conditions on rules, introductions, and the way they respond.
And both players will have goals that require the other player to cooperate in order to succeed.
So they're going to have to learn to work together in order to accomplish their goals.
So they have to learn to communicate, despite the rules making it hard for them to do so
In terms of search source material, there's currently a wealth of information on the internet about people bringing up this issue in the tech community.
It's also something that's particularly relevant on MIT'S campus, just because so many people here will go into tech.
And so for source material, we're looking at a lot of-- there have been a lot of recent articles written on the subject
Also, I have a survey that I sent out for another project that I did, that's basically asking for women and the problems they encounter every day at work.
And a lot of these are in the tech industry.
And even if they're not in the tech industry, there's still boss-employee dynamics that we can use to further explore that topic
So if we have the time, there's an alternative idea that we haven't had as much of a chance to discuss.
It's a little less on the side of something that's an educational tool that's minimally re-playable, and you just play it once to try-- end up cooperating.
And more for like, maybe toning down the message a little bit for the sake of making it more of a game.
And the idea is similar, where the boss is in charge of the rules, as far as the employee goes.
So that all of the rules that are in play for a particular section, are going to be given to the boss at the beginning, or over the course of the game.
And the employee would be told only whatever the boss tells them, and penalized for whatever the boss decides to penalize them for.
And there would be some sort of inadequate system for the employee to disagree.
And overall, goals might be less cooperative, and more directly opposing.
Where, for example, the employee's goal might be to get the boss fired without getting fired themselves first.
And that's something that they might do that would let them do that.
Like finding out things about the rules, might take their focus off of being productive, which would generally lead them to get fired.
Whereas the boss's job, boss's goal, could be to end up with a golden parachute.
End up being profitable, end up exploiting the employee, enough to live happily ever after on their own, and quit themselves.
At a time where they have enough to just like, coast by for the rest-- If they get enough out of the employee before having to either fire them, or ending up in trouble themselves, that would be the other side that would be the success for the boss
OK.
So is there anything else you want to get in [INAUDIBLE] Couple of things.
It's [INAUDIBLE] nice to have the clarity of this is-- if you can specify your audience [INAUDIBLE].
You mention minimally re-playable.
I'm wondering whether there's some value in playing it twice with inverted roles
That was considered.
We meant that it's just-- if you keep playing as the employee, and you already know that you're being lied to.
And even worse, if the rules stay the same, and you already know the rules, then that's completely defeating the purpose
I mean, you could still play it to explore the relationship further, and maybe see what else you could have done.
But fundamentally, [INTERPOSING VOICES]
[INAUDIBLE] because it's live action, because you're two people working together, talking about this stuff.
Even if the rules are the same every time you play it, that other person is going to be different.
I don't worry minimal replayability.
It's very replayable
And definitely reusable, among-- in a training setting, absolutely.
[INAUDIBLE] reusable.
If you don't play it again, at least no matter what you do, [INAUDIBLE] how do you talk with each other after the game is over?
And [INAUDIBLE] because that sort of helps people reflect on what they should have picked up while they're playing the game.
[INAUDIBLE] thinking about how [INAUDIBLE]
Great thing for that, if you can do that without requiring a facilitator, you're already doing much better than a lot of the games in this sector.
It gets your game a little bit more useful, a little more used, if your players can debrief each other
You describe specifically the relationship between a boss and an employee.
And it seems like that-- I guess you should be clear whether all of the tension specifically between these two people, or whether also there's any abstraction of the college of [INAUDIBLE] Say in a stereotypical, male boss, female employee situation, there's a lot of culture supporting the male boss.
That's not there [INAUDIBLE] so how do you put them together?
The sort of touchy phrases thing reminds me a bit of Taboo.
Which may not necessarily be a [INAUDIBLE] For research, micro-aggressions, I think is one thing that comes up a lot.
[INAUDIBLE] you just [INAUDIBLE] get a lot of stuff.
Not just gender, but also ethnic discrimination, disability, [INAUDIBLE] You can find a lot of material [INAUDIBLE] actually.
Because--
Just to make clear, those were the kind of very two, very separate--
For future-- for pitch-- don't pitch two ideas.
I had a hard time following the second one.
If you're going to pitch the second idea, a very simple two line-- and if we did this other way, this is how it would be.
Enough to get it [INAUDIBLE]
I mean, it obviously came up here because we're not at the stage where we have one idea
So we heard a great, clear idea that's still not specific, but pretty general.
And then with this extra route, this alternate route, it's really just a matter of knowing a two line version of how is this different from the other, without having to go into detail on it
Describing it, I think, as a variant, this is a very [INAUDIBLE] right now, rather than here's a second idea.
This is just a general pitch.
[INTERPOSING VOICES]
Talked about it, if you haven't talked about it enough, we don't need hear it.
If you were trying to get money, [INAUDIBLE]
But since you did bring it up, I do want to give my perspective from a design point of view.
I think that actual competitive game might-- different goals is fine.
But it's sort of like, opposed goals [INAUDIBLE] I think takes the game out of a situation that a lot of people can see [INAUDIBLE] in.
But a lot of people don't actively want to get [INAUDIBLE] or at least don't see themselves that way.
So just to be able to make a game where people can easily make connections to their real life, the goal should be a little bit closer to non-competitive even though it may not be a completely aligned goal.
Any questions for us?
[INAUDIBLE]
I had just another quick [INAUDIBLE] for research.
We did an equal pay game [INAUDIBLE] two years ago, I think.
And there were some interesting things that came up with like, talking about specific scenarios.
So how does the fact that women don't necessarily-- at the time, it was thought people didn't have the data to get a pay raise.
So maybe it was just a matter of, well, they don't have the data.
So if we just give them the data, maybe it would help them.
It's not much more complex than that.
[INAUDIBLE] see what kind of scenarios are you going to do.
Are they going to be very specific, like going to your employee to try and get something that you need?
Or is it going to be what I think I heard, and correct me if I'm wrong, an employee going to a boss to try and just do the job they need to do.
And the boss [INAUDIBLE] can get difficult for them to do it
There's a lot of stuff about how the same adjective, feelings, you use to describe a female professional or a male professional, it can be positive in one sense and negative in the other.
It's the same word, literally.
[INAUDIBLE] definitely [INAUDIBLE]
And really good seeing you already did a survey of getting actual phrases that you can use, and actual perspective [INAUDIBLE]
So I guess I have a question.
Which is, how would you recommend dealing with the fact that the players might not be the genders that they're playing?
That could be a very valuable learning experience
No, I understand.
I mean, the fact that if it even-- if you just said about the phrases.
If you're having a conversation, and that word comes up, it just won't mean anything necessarily to someone.
[INTERPOSING VOICES]
Game mechanic for that.
If it's known that that word is going to have a positive or negative affect to that particular gender, then you make a mechanic around that.
Your mechanics should support these things.
There's times when you don't need a mechanic because players will all understand.
But any game where you're talking about these kinds of cultural things, the mechanics kind of just help buttress the message.
The things you're trying to get across
Yeah, all the research implies [INAUDIBLE] game mechanics are just making the system obvious.
[INAUDIBLE]
That stuff should just come out in debrief.
Especially if you have two people with different genders and different experiences [INAUDIBLE] [APPLAUSE]
[INAUDIBLE] [SIDE CONVERSATION]
We have a third group member, Damon, who's not here today.
So our game mostly explores the dynamics of the sort of the balance of power that can occur when there's a bunch of-- when there's three people who are sort of all fearful of the others there.
And how there's an unstable equilibrium there, where they're trying to keep up with the other two to make sure that no one gets far ahead of them.
But the game's dynamics will be such that this is guaranteed to fail in the long run.
In such, one of the things that players will have to do is try to realize when they have a large enough advantage to strike, and it's really about finding the moment of opportunity when they can make an attack that will work out well.
They mostly explore some of the-- it's intended to be sort of a strategy game.
But sort of lighter than some of the versions that cover similar topics that can take several hours.
It's going to cover a lot of the historical trends around the time.
So the conflict between populist reformers and the conservatives, and the rise of the equestrian class, and their importance as a power base to some of the leaders there.
And also how the legions, the loyalties of legions was shifting from the Senate to individuals.
Who are actually paying them, and giving them the lands that they would retire to.
That's what this game covers
Source material?
Source material is I took a class in Roman history, and it's mostly just stuff that I remember from that.
I've looked at [INAUDIBLE] sources of lines for detailed stuff online, like the number of details of numbers of legions, and where they were stationed, and that's about it
So we remember the games about the Tribunal.
I don't remember what personal perspective it's from, so the pitch should have that in there
I'm sorry
So you want to reiterate that [INAUDIBLE]
Yeah, I just completely forgot to like describe exactly what the game was.
It's the [INAUDIBLE] members the First Triumvirate and they're all sort of-- Caesar, Pompey, and Crassus, and they're sort of jockeying for influence.
And eventual plan to sort of of usurp the other team and to become the sole leader of Rome
For a pitch, you have to assume that the audience doesn't know that.
The audience may have heard of Caesar, and not realize that at the time that your game is [INAUDIBLE], he's not in charge.
Because everyone assumes-- Caesar means king, right?
[INAUDIBLE] who is Pompey, who is Crassus-- especially Crassus.
A lot of people haven't heard of him.
[INAUDIBLE] You talk a little about light strategy, and I wonder if that means it's still working?
Or is it a little bit more of a political points type game?
Do you expect it is going to be like a victory point base system?
Or is it going to be more about military [INAUDIBLE]
[INAUDIBLE].
It's not going to have a victory points system.
It's going to be sort of-- the way you win, is by crushing the other two players there.
There are multiple paths that this can happen.
You can get enough legions to become personally loyal to you, that you can just beat the other two people in the [INAUDIBLE].
Or you can gain enough support from various groups in the realm and that you really can just start to snowball.
And you can have a tremendously large force that the other players can't deal with there.
And it's going to be a war game, but with sort of more-- mostly what it is is action selection.
And that the players will have a choice [INAUDIBLE] on their [INAUDIBLE] choice of actions that they can take.
The game will probably be such that the players will go in order over about a 10 year period, or maybe a 15 year period.
They'll have about four actions per year [INAUDIBLE] That's the sense that, for instance, if the players build new buildings, or conquer new problems, it makes more options available there.
Also the struggle, if you-- depending on whether the optimize the conservative to the popular are the populous are sort of in power.
Have more power, or different actions are available also
So I just had maybe three different envisions, just from that last [INAUDIBLE] of what this game could be.
On one hand, you describe it as a Roman war game.
And I'm thinking units and pieces on a hex grid
It's very much you go and-- it's in what you take, sort of.
It's very much [INAUDIBLE] where the players are just choosing if you want to go [INAUDIBLE] you just say, OK.
I'm going to go launch an invasion of Gaul here.
I'm going to take this [INAUDIBLE].
You have to prepare for it.
You have to make sure that you have the position that you would go to do this, that you have the resources to raise the legions necessary there.
You also might want to [INAUDIBLE] to call your veterans if you're successful.
And then, once you launch [INAUDIBLE], there'd be a factor of luck to determine how successful you are there.
And you would probably continue repeating the same few actions to continue your invasion for several years
And that gives me a completely different image of what your game is.
Which is more of a collecting resources [INAUDIBLE] tokens.
That's the second image [INAUDIBLE] And then you said action selection.
And I'm thinking that [INAUDIBLE] like a card game.
Almost like a game where these that I could be doing right now.
Maybe [INAUDIBLE] if you get say, if you [INAUDIBLE]
The actions will be normal in [INAUDIBLE] global there.
So in the sense that if the populous are in power there, there is a set of actions that you can take is sort of different.
Because the political possibilities are different.
Because the political [INAUDIBLE] So like, if the populous are in power, it's much easier to get land for your veterans, for instance
So you still have the same actions, but how hard it is [INAUDIBLE]
Some of them might get grayed out or something.
[INTERPOSING VOICES]
Specifically, what happens is as the [INAUDIBLE] has mentioned, that the actions will sort of flip over there.
Like instead of the populace being able to do this, instead, you're able to get conservatives to do this [INAUDIBLE]
So I think [INAUDIBLE] of what your game's gonna look like.
But that's something you want to get across in the pitch.
[INAUDIBLE] board, you could use hand gestures, like you just did.
Referencing other games.
Just so that when you're talking about the game, you're trying to help me see it in my head.
[INAUDIBLE]
One of the last things.
So we're going to have three players.
Each be one member of the Triumvirate.
I imagine each of those historical people had different perspectives of how they saw things.
Were they equal?
They had different backgrounds?
The game is going to be portraying them as having equal views, the same goals, the same views, the same perspectives, and everything.
But it treats them different.
They will have different starting resources, and different advantages and disadvantages as the game goes through.
So like Crassus will have more money, and [INAUDIBLE] money, for instance.
I imagined that Pompey will start with a lot more influence than the others
Is there a historical or realistic basis for any of that?
Yeah.
Crassus is often called the richest man in history there.
He is known for just acquiring such just massive-- he had more money than basically the Roman government
So that's going to at least help him with the players starting position.
Did that wealth-- is that wealth then carried over to the kind of actions that he's able to do historically?
Or was it really just setting up?
It's going to offer-- I haven't said whether or not that will offer him bonuses towards acquiring wealth, or whether or not he will just have more wealth at the start of the game there
From our point of view, for the assignment, we're really interested in seeing the personal perspectives.
So you could do the generic, they're each an equal person.
It'd be more interesting for us to see be a reflection of the actual personage
The differences, I think, are really going to shape their strategy, and how they start the play.
So is for instance, in Caesar will have sort of have a bonus towards campaigning, or not necessarily bonus towards military invasions, but making his legions loyal to him.
His legions become loyal to him quicker.
I'm not sure which of these is sort of-- and Pompey will sort of start out with more powerful and prestigious than the others.
But during the first several years of the alliance there, he was obsessed with his wife, Julia, who was Caesar's daughter that he had married there.
And he was neglecting lots of his responsibilities because of his wife there.
So he will probably have penalties, he would miss every forth action, or something.
He starts off in a much more powerful position, and he sort of has to use limited actions to try to not fall behind in the earlier part of the game.
So [INAUDIBLE] these players sort of have the same goals, but these small differences are going to cause them to take very different strategies oftentimes
So I'm actually going to suggest that the differences are what the game should be built on, not the bedrock of similar [INAUDIBLE].
It seems to me that you are describing that there is an objective reality which all three of them agree on and are just playing on this field with their individual advantages.
But I'm going to suggest that you start from the differences.
Start from the fact that one of them's richer than everybody else, one of them's more loved than everybody else, and one of them just [INAUDIBLE] And then you can just-- what if the game is all about those differences, rather than everything else that was involved [INAUDIBLE].
Because then we get a little bit closer for how these three particular people see the world, rather than [INAUDIBLE]
The thing is, I really like a lot of the differences, the sort of [INAUDIBLE] and to have these differences, [INAUDIBLE] I intend to have these bonuses that are really not going to be-- they're just going to be one [INAUDIBLE], something fairly simple.
But will cause the players to be playing almost completely different games.
Where Crassus is basically trying to take actions that are just [INAUDIBLE] is basically going to try to acquire lots of money, and then use it to brute force his way through money.
Try to brute force his way to get the most efficient ways to use this, or try to--
Set it up to be [INAUDIBLE] it's not just [INAUDIBLE] project.
It really should be all three players end up playing very different games, [INAUDIBLE]
That's definitely [INAUDIBLE].
Like Caesar's going to be-- he has the potential, but he needs to be able to get the resources so that he can start-- that he can really [INAUDIBLE].
Whereas like Pompey has this power and is basically going to be trying to consolidate it.
And try to threaten the other team's growth, and threatening the growth of the other two.
Thank you.
[APPLAUSE]
So that was probably as good a time as any to take a short break.
We'll have about-- these games generally don't take more than 45 minutes.
So these are related to the reading that of space control.
Not so many chase games, mostly control games.
And although, Ticket to Ride I guess has some sort of set building in there as well.
Play through the game.
It seems like a lot of people didn't do the reading, so I'm not so sure how much context they give you.
But I do still want you to.
End up having played a wide range of games before the end of the semester.
And then at about 3:30 I guess, a little before 3:30, we'll bring in the prototyping boxes, and you can spend the rest of the class working in your teams
Go over this again really quick?
So Panic Station is a little bit of a-- it's got a lot of trader mechanic.
You are trying to survive in a space station, but there's aliens.
And they're infecting you, and you're going to infect the others too.
[INAUDIBLE] is another one of those path building games that has area control, and other things like that.
French, medieval towns, [INAUDIBLE]
Through the Desert, it's kind of like mapping paths
Path building, mapping, you're creating basically a supply chain line of camels through the desert.
It's a [INAUDIBLE] game.
I've not played [?
Purple.
?]
[INTERPOSING VOICES]
And every one of these games that we put up so far, these three games are all about putting tiles down.
This one is more about putting-- I was going to say putting dues down, but really you're putting down camels.
They're really cute camels
This is putting down trains, but you're putting them down creating sets.
To create matching sets of train
Cool.
All right.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so this weekend's reading was met because be giving a YouTube presentation.
Actually it was a Google presentation that was recorded onto YouTube.
And he's the designer of Pandemic, he also made the game Forbidden Island, and I think he also did-- Desert.
I'm assuming that he did Forbidden Desert.
That's my guess because they're not that different of games.
I mean, these are cooperative games, for the most part.
How many of you have played one of these games?
Forbidden Island, Pandemic-- Pandemic the board game, specifically.
Yeah, OK. Pandemic the computer game is a completely different game.
Huh?
Pandemic Two, the online game?
Yeah.
The one where you shut off Madagascar
Yeah, yeah, yeah
OK, that one is a different game.
It's same title, same concept, right.
I mean, it sort of made sense that they would have [INAUDIBLE].
I mean, there's probably more of a similarity of civilization of board game than civilization of computer games.
Although the civilization that you play as a board game nowadays was not the same civilization the board game that existed when the computer game [INAUDIBLE]
[INAUDIBLE]
The computer game aside, you can play the-- how many of you basically played a game where it just turned into one person telling you to do everything?
That's my [INAUDIBLE].
How many of you had the same thing happen with either Forbidden Island or Forbidden Desert?
It happened with you two?
Or were you the one telling people [INAUDIBLE]?
[LAUGHER]
So it seems-- actually, how many people play Forbidden Island or Desert?
OK, only one
So 100%
So maybe it happens a lot more often.
I don't know.
If find it happens a little bit less often with Island, but it happens to me every time I play Pandemic.
On one hand I'm usually the one receiving orders, and I don't mind because I'm not that invested in it.
But on the other hand, I'm not that invested in it.
It kind of like takes me out.
When someone starts telling me, this is what you should do, clearly this is the optimal move.
This is the thing that you should do.
It's like, that's fine.
I'll do it.
I don't care about this game anymore because I don't have any [INAUDIBLE].
But I don't know.
Yeah?
So in my experience it's always been each of the players is doing that
Everybody else is telling you to do something?
Every single person is telling every single other person exactly what to do, and every person ends up making a decision for themselves.
Just at the end being like, OK guys, here's what I'm actually doing because I think that [INAUDIBLE]
That sounds reasonably interactive and you still have a decision to make anyway
It's fun because you actually get to like-- people actually listen--
Yeah
--on one hand because when you bring up a good point they'll be like, oh yeah, that's a good point.
But on the other hand, no one-- it's not one person playing the game
It's like a board of directors giving you advice, and then you've got to figure out what's the right thing to do
Except as you're done you start giving advice
And everyone switches roles
I don't know if you're saying it's a negative, necessarily
I'm not necessarily saying that
But I feel like that might have very well been what he was going for.
Like in the video he was all talking about his wife, and like, he wanted to be able to play with his wife, right?
So this is kind of a serious gamer can play it with someone that's not as invested is kind of what you're saying a little bit
Yeah, I mean, as a result I've played many games of Pandemic, which is a good thing.
Yeah?
Sometimes I think, like, [INAUDIBLE]
Right.
Like deliberately sabotaging
Yeah.
Also, there's a game that [INAUDIBLE] being play tested basically.
Called [INAUDIBLE], it's like, really pretty tactical.
There's like a [INAUDIBLE] strategic components and you have teams of players [INAUDIBLE], but there's huge-- there are large player [INAUDIBLE], but at the same time it's really hard for one player to go help someone else on their team and do everything else components of two minutes for your turn.
And so it happens [INAUDIBLE] someone is, where like, someone steps in and tries to help someone else, and they end up screwing it up even worse because they didn't realize that they [INAUDIBLE] everything that was going on and didn't have the time to analyze it properly
I think that's like an example of what you're describing as well as like here.
In the realm of [INAUDIBLE] co-op games or games with a very strong [INAUDIBLE] component, you got-- there is an assumption, at least in games like Pandemic, that it's not so much that something is out there strategizing to beat you and you're trying to over come it-- like a different player, for instance, who is trying to achieve victory at your cost.
It's more about, can you get your affairs and your communication and your decision making lined up [INAUDIBLE] so that you can actually overcome the odds that the designer has set up for you?
So I think in the case of the [INAUDIBLE] games, there's actually somebody out there who's actively playing the game who's actively trying to win at the expense of your victory.
So one way that I certainly see is the [?
timing, ?]
right is it's OK to cooperate as much as you want in a minute.
And that just leads to, like, confusion and bad judgment calls and sometimes hilarity and [INAUDIBLE] Space Alert, which is maybe a game which you should [INAUDIBLE] because it's-- for those people who don't remember me describing it in previous classes, you're all members of a starship.
You all pretty much aren't sure what you're supposed to be doing.
You're playing the game in real time because events happen as you play an audio track that just runs nonstop.
It never pauses until the game is over.
And you all have to be very, very coordinated to get anything done.
Just like firing a single gun requires you to charge up that gun, turning on a shield so that you can block enemy fire needs to be on at the moment the enemy fire is about to hit you, I think-- as I recall [INAUDIBLE].
But [INAUDIBLE] it just [INAUDIBLE]
[INAUDIBLE] and also because there's so many threats [INAUDIBLE] it's very easy to, like, accidentally run out of energy or to do things where they [INAUDIBLE] or miss if you don't [INAUDIBLE] one of the [INAUDIBLE] things is obviously, like, [INAUDIBLE] fires their gun at like some threat, but they fired it a turn before it appears [INAUDIBLE]
Yeah
Which is like [INAUDIBLE]
Yeah, because [INAUDIBLE] has a range, if I recall, right?
An effective range.
So, you're all trying to achieve simultaneous victory.
You're all trying to not die in that game.
But because you have so little time to be able to figure out what the right thing is to do, you just [INAUDIBLE].
That's not the case, I believe, in Desert Island and Pandemic.
You have about as much time as you like to make a group decision.
But it's fairly [INAUDIBLE] these sort of competing cooperative games, where your only opponent is the game system the designers really created for you.
You can of course set difficulty levels in a lot of these games.
If you're learning it for the first time, you play it on kind of the easy set up.
And if you're [INAUDIBLE] with all the mechanics of the game, you can sort of crank up the difficulty by starting the game at a point of [INAUDIBLE].
But I think-- anything else that anyone noticed from his thoughts about the design of cooperative games?
You mentioned-- I think it might have been in the video or something else [INAUDIBLE] the rule that said that you couldn't show anybody else--
Your hand?
Your hand
Yeah
Because that was a very important component of [INAUDIBLE]
Yeah, yeah
I didn't understand the [INAUDIBLE]
It's kind of giving you still something to do.
Because you have information that no one else has until you share it, there's still the thing that you can do, which is share the information that you got, even though it's not necessarily information that you probably-- in a game like "Pandemic," it's not information you want to hide.
You want everyone else to know this information So there's still a decision to be made even though there's activity to be done.
But in a game where time is limited, then yes, there becomes a decision of, do I want to share this piece of information?
Because it's taking away time from doing something else.
I worked on a game with the Education Arcade, which is a research group here based in [INAUDIBLE].
And a long time ago, we made a game where it was a location based game.
So everyone's walking around with GPS [INAUDIBLE] smart devices.
And everyone is trying to investigate the ground water contamination, basically, but everyone also was one of three different classes and they would get different information based on which class they were.
Some of them were, like, [INAUDIBLE] some of them were more-- I'm trying to remember what-- some of them were biologists.
Some of them were geologists.
I think one of one was, like, construction engineer or something like that.
So you knew things about how things were built but you didn't necessarily know much about what type of animals will be affected by [INAUDIBLE] And so in the game, information that you got were private, but it was the kind of thing that you want to share.
Now, in a location based game, sharing information becomes tough because everyone is physically spread out over an area.
This was a game that was played over the entire MIT campus.
Everyone had walkie talkies.
But the nice thing about walkie talkies is that only one person can talk at a time.
Otherwise, no one really can make sense of everything.
And it's still kind of the case as you try to play the game with cell phones because you can only talk to one person at a time unless you, like, set up some crazy network.
Even if you try to do text messaging, that's [INAUDIBLE], right?
So that's one way that you can get people involved and feel like they're contributing, which is just, give everybody little fragments of information that they need and then get them to-- and then the game becomes more about how do you go about sharing this information rather than what decision you make with the information.
[INAUDIBLE] once you got all the information, there is a decision to be made.
And in some cases, it feels like a board room where everyone's got different ideas and trying to encourage you to do it in certain ways.
And that's great, because [INAUDIBLE] is weighing the pros and cons of every possible option.
But in some cases, it's just, like, one person just decides to be the decider and is going to tell everybody else the decisions and then everyone else [INAUDIBLE].
Has anyone played, like, sort of cooperative live action-type games?
[INAUDIBLE]?
Oh, cooperative live action type games.
I played [INAUDIBLE] where there was, like-- I mean, a lot of [INAUDIBLE] to do when you're [INAUDIBLE] make sure that-- you need to get everyone's [INAUDIBLE] because that will make people communicate and work [INAUDIBLE].
And oftentimes-- so, for [INAUDIBLE] like, I remember [INAUDIBLE] in particular, there was [INAUDIBLE] You would need, like, [INAUDIBLE] components, which you can [INAUDIBLE] on your own.
You can gather the resources and stuff.
And then you needed to successfully [INAUDIBLE] in the blank, Hangman or Pictionary or something that's describing [INAUDIBLE] And it was nice to have this cooperation [INAUDIBLE] cooperation is like, did someone want to, like, [INAUDIBLE] trying to describe [INAUDIBLE] face to someone.
And it was also [INAUDIBLE] like, share a little bit about them.
It also was interesting is that oftentimes, someone says, does anyone want to, like, when you're doing Pictionary, trying to describe the word destruction.
It's like [INAUDIBLE] people talk about your research after that
Right, right.
Of course, yeah.
Yeah.
So I think what you're describing is, like, the live action role playing [INAUDIBLE]
Yeah, live action [INAUDIBLE]
Yeah.
But a lot of live action role playing games, even though you may be cooperative on that one activity, you don't know what the motive of every single person in that room is.
Whereas in non-LARP type games but that are played out in live action-- say, training simulations, for instance-- often, everyone's actually on the same team.
Everyone is very, very clear [INAUDIBLE] working toward a common goal, just given different roles.
So, [INAUDIBLE] I trained a couple of years in the Army and we had a lot of live action training simulations where, yeah, everyone's on the same team and there are instructors out there trying to make your life difficult, sure.
But everybody that you're actually working with is communicating and trying to do the best job that they can towards the common goal.
Has anyone gone through anything like a sort of simulated exercise or anything-- either an evaluation or-- you know, like an emergency evacuation?
Yeah, [INAUDIBLE] fire drills
We have fire drills.
All right, that's a great insight.
Everybody wants to get out of the building alive without falling all over, sure.
And that's a great example, yes
[INAUDIBLE] in a lot of the, like, scenarios that [INAUDIBLE] you and your team [INAUDIBLE] even a company trying to design this product and you had all these challenges.
And it was primarily intended to be cooperative [INAUDIBLE] an element of, like, [INAUDIBLE] other team [INAUDIBLE]
Do you-- so you compete with the other teams for the-- do you feel like you're in competition with the other teams while you are basing your decisions?
Or is it really just like right at the end when you present everything?
Mostly right at the end, but there were certaion ones where, you know, you had like a shop and there were pieces you could get while you were waiting in line to get different pieces.
Like, you were kind of looking at what all the other groups are doing

And part of it was because it was the same groups all semester.
Even if there wasn't any [INAUDIBLE] competition between groups at first, but because over the entire semester, you were working with the same team.
You kind of develop a [INAUDIBLE]
Yes, yes, I've been in similar exercises, mostly in school as well
Could this go as simple as just playing Lava when you're, like, three years old and you can't thouch the ground, and you get from one end of the room to the other without ever touching the ground, jump pillar to pillar or something?
Well, let me see.
I guess [INAUDIBLE] cooperative, I guess.
There's player versus gravity, pretty much.
And yeah, I mean, it depends on whether your goal is to be the last person standing, in which it does become a competitive game
Sure, but it was one where you're just trying to get from one side of the room to the other, something like that
Yeah
And you're working together
Yeah, if you're all working together, I would say that is a fairly cooperative game and the rules are fairly simple.
Just don't fall off pillows which are unstable.
And that's very different from having different roles.
Everybody has the same role.
Everybody is just put in a situation where they can't accomplish much on their own but you can do quite a lot if they pull--
I always played with my sister, and we kind of had different roles, because I was three years older than her.
When we were five years old, I could jump [INAUDIBLE] further than her, right?
[INAUDIBLE] put up different roles there a little bit
I mean, she also has a lower center of gravity, I'm suggesting then?
Sure, she could also barely walk
So you're playing with your sister.
So, it's kind of interesting.
I don't actually like a lot of creativity competitions where they put teams against each other.
I understand that it does get people enthusiastic and engaged and [INAUDIBLE] they want to be able to do the best.
But it discourages a lot of information sharing, which could actually be better for learning in the process.
So when we were [INAUDIBLE], for instance, a lot of [INAUDIBLE] run by colleges will give out awards.
They'll quite simply have prizes for people who win maximum applause or something like that.
I think we do give verbal [INAUDIBLE].
Has anybody ever done, like, [INAUDIBLE] awards?
No, I've never done any awards.
When We're doing [INAUDIBLE] the joy of engagement is being able to share [INAUDIBLE] and making sure people share throughout [INAUDIBLE]
And those are usually the shout outs that give [INAUDIBLE].
These are the people who really helped as a team, the people who were clearly a benefit to more than just one project.
We will do those kind of shout outs because in learning situations [INAUDIBLE] For competitive situations where you don't expect to have a lot of information sharing-- say, a school versus a school competition.
It could be math.
It could be sports.
It could be-- we're working on one right now, which is a [INAUDIBLE] competition, a satellite programming thing.
Some of you might have seen the email about that, where a lot of the schools that are working together are geographically separated by different countries, probably speaking different languages.
So there may not be terribly much information sharing to begin with.
So it makes sense that they won't be sharing information with people that are geographically close to them.
So it's OK to sort of put them in competition with each other while expecting they're going to cooperate entirely within their own team.
But there's a deep wrinkle to that one, which is when it gets to the final rounds, three schools are put together and one school is always geographically separated from the other one.
And that gives students an experience of what it's like to actually work on the state, right?
Because you are working with different companies which are writing code to all run on the same machine.
But-- and all trying to accomplish different [INAUDIBLE] problems that they're given [INAUDIBLE] so difficult that one school-- one school could probably do a mediocre job on it, but you need the resources of three schools and all the kids in three schools to be able to solve this particular space problem.
So you need to communicate and you need to share.
And that's an interesting [INAUDIBLE] of the game where, again, it's all about what are the limits of communication?
We talk about time.
We talk about physical distance being a limit of communication.
What about things like language?
About things like time zones?
All right, so any of the games that you are making, cooperative games, I think are also competitive in a way?
[INAUDIBLE] the end goal is [INAUDIBLE] I mean--
Yeah, it's--
In the end, it's like a competitive thing, but to achieve the most you can, it would be be cooperative
OK, is this the base [INAUDIBLE] one?
No.
[INAUDIBLE]
No.
Well, yes.
I was going to say, we're definitely cooperative.
[INAUDIBLE] well, two teams, I guess
Two versus one [INAUDIBLE]
Yeah
Oh, it's two competitive teams
Yes
So you have cooperation within the team and competition between teams.
And which game was the one that [INAUDIBLE]
The ship captain one
The ship captain one, right, right, where [INAUDIBLE] trying to [INAUDIBLE] put all of this information together, right.
So, is there anything from Matt's presentation that you might want to try out in a game?
I really like the idea small leveling, kind of.
So you said he had three-- he gave the flow chart, which I really like
Right, right, right
Right?
So you want to make it balanced between players' skill and [INAUDIBLE].
And if there's too much [INAUDIBLE] players' skills, you have anxiety.
And if there's too much player skill and too little [INAUDIBLE], you get boredom.
So we have like a basic version that you can just start with.
And then if you're a really good gamer, you might move onto the normal version or the advanced version really quickly.
And first, I was like, oh, this is really complicated, right?
Three versions of the game.
Then, you play [INAUDIBLE] and you just add one [INAUDIBLE] to that.
And that's [INAUDIBLE] changes [INAUDIBLE]
And that's another-- in Forbidden Island, I believe the only change in difficulty is that you start-- the way [INAUDIBLE] works is that you are on an island that's slowly sinking into the water.
And [?
there's a ?]
water level marker.
So by increasing difficulty, I believe you just start at a higher level of water.
And more stuff is submerged at the beginning of the game.
But the rules don't change.
And if you're going to try to do something like that in your game, I would certainly encourage you to think of difficulty levels in that way.
Don't give us, like, here are the simple set of rules for simple players and the advanced set of rules for players.
Those don't change things like just the starting conditions so that I-- I wouldn't encourage you to change, like, individual variables in your game because that might be too complicated to get across.
But things like setup-- that's, OK. Once you've got the whole game set up at the correct difficult level.
Don't have to worry about the differences between setup for easy or setup for hard.
Giving people extra options, for instance, by giving them extra cards or something like that could also be a good way to be able to [INAUDIBLE] change the level of difficulty throughout the game.
That's a good idea.
[INAUDIBLE] there's something I was talking about cooperative and competitive.
He just talks a lot about what it was like to design his first board game.
I believe he said that that was his first board game.
[INAUDIBLE] his first published board game [INAUDIBLE]
First one was [INAUDIBLE] Fish and Chips
Oh, yeah.
Oh right, Fish and Chips.
That's right
Oh, wait.
I just got that joke.
Wow
So-- but anyone observe or hear anything interesting from that one?
From the [INAUDIBLE] talking outside of the cooperative and competitive [INAUDIBLE] cooperative game design?
[INAUDIBLE] do you just mean something interesting about the talk?
Yeah, anything else that was interesting.
I've been asking you questions specifically on his thoughts on cooperative games like [INAUDIBLE].
But that's only part of the talk.
That's not his whole talk.
So I was just wondering if anyone noticed anything interesting
I think [INAUDIBLE] he mentioned Ameritrash [INAUDIBLE] games
Ah, yeah
Something I haven't herad of.
And then when I looked online, it seems to be either something that has made it [INAUDIBLE], sort of?
It's one of those almost invented internet forum arguments.
But it's one that-- I mean, it's like the [INAUDIBLE] of the [INAUDIBLE] in game design.
That's another one where-- that's the sort of thing that you see now just to make people roll [INAUDIBLE] But back in the '90s, I guess, it was a bigger thing.
It was like, are games about the systems or are games about the stories?
The whole thing was just that and it was people sort of arguing past each other-- not actually waiting to talk to each other, just wanted to shout at each other about it.
And Ameritrash and Euro games are one of those things purely, from the description of Ameritrash, you can sort of expect that the people who like Euro games gave it that term
[INAUDIBLE] argument, because the Ameritrash games were-- the big problem with the Ameritrash games were the Milton-Bradley Gamemakers games.
I think they were called Gamemakers or Game Masters series in the mid '80s.
Like [INAUDIBLE] There's another [INAUDIBLE] one.
But games with a board and lots of little plastic pieces where there's a lot of, like, details given to the [INAUDIBLE] molding on the pieces rather than the system.
Like, that's where it was thought the money was spent.
Where a Euro game is, well, it's little wooden cubes, and not much theme going on.
And so the system has more the amount of attention and detail [INAUDIBLE]
There is a lower cost version that is along those same lines, because Ameritrash is a lot about presentation, right?
This is just-- so, a lot of-- I tried to think.
Munchkin?
Anyone play Munchkin?
Yeah, OK.
So, you know, this card game about sort of dungeon crawling, being really bad role playing gamers.
And it's all about the art.
It's all about funny text that they've written.
It's not necessarily about cool-looking pieces, but it's about cool looking cards that you can laugh with, you know, while you're playing the game.
But a game [INAUDIBLE] itself, [INAUDIBLE] actually [INAUDIBLE]
I might call Magic Ameritrash
And Magic-- the Gathering sort of trades [INAUDIBLE] same sort of thing as like [INAUDIBLE] layered complexity on top of complexity [INAUDIBLE] very little elegance to the [?
system, ?]
right?
But every once in a while, they sort it out and then it becomes a pretty elegant system and they then keep adding things on top of it.
That's because that's how their business model works
So I guess a big question about [INAUDIBLE].
How is Ameritrash different from the theme games like Monopoly [INAUDIBLE]?
Not that different.
It's a more derogatory term, but I think when it comes to the stuff that's really old like "Monopoly," there's a limited level of presentation they could possibly achieve with traditional manufacturing.
And by the time the [INAUDIBLE] term Ameritrash had shown up, games like Monopoly and Clue and all that had already been shrunk down into budget versions.
You know, it's like, you're no longer getting metal pieces.
You're getting little plastic pieces that barely look like the dog or the hat they're supposed to be.
Games like "Battleship" where it's like, the whole idea is that it's not really, like, the "Battleship" kit looks nice.
It's just like this small part of [INAUDIBLE] manufacturing and people grew up playing Battleship.
And those games are being sold on a completely different premise.
Those games are primarily nostalgia games.
Those games are games that you bought because you played them at one point in time.
But I think Ameritrash was mostly used to brand a certain kind of game that you'll be buying because you want to play for the first time-- like "Munchkin" that you buy to play with a bunch of friends, often published by companies that either had a war gaming background or a role playing book kind of background like Steve Jackson, for instance.
Because they're used-- actually I believe [INAUDIBLE].
Actually, I'm not so sure if this came up in class.
This is something that I remember hearing recently.
The idea of the game writer, you know-- what is [INAUDIBLE]?
Yeah, Mack was talking about-- writer verses inventor versus designer
Right, so the idea is that games were things that you would write comes out from the role playing game industry, where their primary product was books.
And sure, books included descriptions of systems, but [INAUDIBLE] prose is your tool to be able to get things across.
[INAUDIBLE] to try to use prose on the things that prose is good at-- on things like creating atmosphere, giving you exclusive detail about the setting a whole world or characters, and things like that.
Whereas the Euro games, the Euro games were originally designed for quite a different market from the role playing.
Traditionally, role playing games are designed for enthusiasts.
Euro games are designed for Christmas presents.
Very specifically, they're designed to sell lots and lots and lots of copies at Christmas to families.
So, not only had the systems have to be simple enough for kids to be able to learn, but also advanced enough for adults to be able to want to play them and give them away as gifts to other people.
And so that's a lot of attention paid to, like, simple rules, very, very complex interactions.
And I don't know what the push towards the simplicity of [INAUDIBLE], sort of the abstraction of, like, cubes and circles and maybe [INAUDIBLE] like the are, like, [INAUDIBLE]
A lot of it is [INAUDIBLE] factory [INAUDIBLE] the one factory that makes it all
[INAUDIBLE] one?
There's one factory in Germany that made most of them.
They don't make them in China, those games
OK.
So that's economy of scale there, I guess.
[INAUDIBLE] easier to make than anything else?
So, when it comes to-- like I said, Euro games are for families and Christmas presents and stuff?
That feels kind of weird to me because I'd expect for children to care a lot about the presentation of the game.
So, how would you reconcile that, I guess?
I don't know much about that social dynamics of the Euro game family.
Who makes the call of what game they're going to play, right?
Is it the kids or is it the parents?
I certainly grew up in a family where the parents would say, this is what we're playing now.
We are playing "Scrabble" because mom likes "Scrabble" and mom will destroy all of us at "Scrabble".
So I don't know if that's the same thing in European families
And so we had a professor come in who talked about this a couple years ago where one thing that happened, at least in Germany, is there is a conference called [INAUDIBLE] Fair?
Trade Fair
Trade Fair.
[INAUDIBLE] fair happened in Germany.
Thousands upon thousands of people in Germany go to that fair to play the new games before they're published.
And [INAUDIBLE] fairs, they give this spiel [INAUDIBLE] awards, the game of the year awards, which then if you get that award, they'll sell a [INAUDIBLE]
Because every newspaper will cover it
All the newspapers are going to cover it.
But those families were going to go there, play those games.
The kids are going to be interested in those games.
Then, when Christmas time comes around, the game they get is going to be the game that won the [INAUDIBLE] award, not necessarily the game they played at the convention.
But that's the game that got the award.
Everybody bought it that year.
And the next year at flea markets, that game is all over the place [INAUDIBLE] So there's a little-- it's a very different kind of consumption economy going on over there.
But here, it's a hobby market.
Here, it's the [INAUDIBLE] stores.
It's not necessarily family oriented at all or family [INAUDIBLE] included in a particular kind of family organization
So they're like toy [INAUDIBLE] in the US, then, like Tickle Me Elmo and Furby
Yeah, the toy fairs are-- yeah, in the US, they're inventors, products from China and again, one thing gets bought and one thing is sold and everybody wants that one thing every single year
But the majority of people who go to [INAUDIBLE] are, in fact, families-- tourists, you know.
But a lot of them are probably tourists from other parts of Germany [INAUDIBLE], but also from Europe as a whole.
But of course, there's a small number of people who are going to [INAUDIBLE] who are the [INAUDIBLE] the people who are going to decide what's [INAUDIBLE].
And they are looking at the families They're trying to figure out what is going to be that hot product this year so they can get new [?
orders ?]
in.
[INAUDIBLE] [INTERPOSING VOICES]
What time of year is it?
I think it's August
Yeah, it's summer.
[INAUDIBLE] [INTERPOSING VOICES]
Yeah, so-- but I don't know the extent to which the manufacturers need to have lined up their pipelines before [INAUDIBLE] in order to be able to capitalize on the [INAUDIBLE] Because if you fail to miss your window-- if you get the award and you can't actually put [INAUDIBLE] product down there, then you can't make as much money as you [INAUDIBLE]
And the toy fairs in the US, are they mainly consumers going to it or the industry, the professionals, and they're sort of, like, rubbing shoulders?
Press.
Press go to toy fairs as well
Yes, press, industry, Walmart, basically.
[INAUDIBLE] Walmart and a couple others
So it's like, you're an inventor.
You're trying to get on Walmart's shelf.
So that's why you go?
Yeah
You're trying to find, the next hot toy for Christmas.
And that's why they--
Well, I'm not so sure that the toy fairs in the US are that inventor friendly
Yeah, the inventors have already gotten things [?
off ?]
by somebody.
So it's the publisher.
It's the maker who has already had the stable of inventors they purchased the product form.
And they're making it to this place to test it out, see [INAUDIBLE]
But who are they testing it on?
The press and the people who are going to sell it.
But-- [INAUDIBLE] Exactly
Isn't [INAUDIBLE] backwards, though?
Because it's the consumer that's going to be buying it, not the--
The consumer doesn't have a need for it until they find out about it, though.
It's a very different process.
The consumer is like, when you look at toys, just look at the US toy industry.
The customer just isn't involved
It's just whatever the hype machine builds up?
Yep, whichever hype machine has the most money behind it gets the product on the shelves.
And these days, it's [INAUDIBLE] properties [INAUDIBLE] trans media, but like, most media properties.
There's-- the game, the toy, the [INAUDIBLE], the comic, everything around it
Like a multi platform
Yeah, yeah
[INAUDIBLE] because you have multiple advertising campaigns pushing each one of these threads.
But they're also sort of pushing the entire [INAUDIBLE]
If this is something of interest to you, go to a Walgreens or a CVS.
Go to the toy area, and look at the toys that have really poorly designed games on the back of them just to kind of see, like, these are the things [INAUDIBLE].
Somebody made it.
It didn't get much press.
It didn't get that [INAUDIBLE] behind it.
And just, these are [INAUDIBLE] what is the effort they put into that thing?
For the most part, it's just packaging.
The actual thing [?
that's inside ?]
of the package
I'm trying to think.
Where is the big toy fair in the US?
In New York
In New York f Do you know when it is, ish?
The summer, I suppose?
I want to say summer because [INAUDIBLE] Toys "R" Us.
Well, I helped [INAUDIBLE] Toys "R" Us [INAUDIBLE] do this thing for a toy fair.
I want to say but I could be wrong
Did you [INAUDIBLE]?
I didn't actually go.
I just shipped the product
OK, so, my [INAUDIBLE] You may see a [INAUDIBLE] collectors, in particular
My customer was, like, they made point of purchase displays.
They made point of purchase displays for retail outlets and it was like a huge deal for them
OK, so this is where they're selling things to stores to sell things, not the actual product that [INAUDIBLE].
Be caring sure.
[INAUDIBLE]
February
February?
Gee
Toy fair in New York-- 2015, February 14 [INAUDIBLE]
Oh good lord.
February-- OK.
I have a lot more to learn about the toy industry
So I guess it's Valentine's Day, after cards, then toys
The other thing that I thought was actually kind of interesting in the video that was how he was talking with the folks at [INAUDIBLE].
And he was describing about embodying the player, giving the player something that he can identify with either on the board or if all else fails, somewhere in the packaging, something that he can project himself into.
I'm not entirely sure if that is sort of universally accepted across all publishers or even conventional wisdom across the board game industry, but it's certainly interesting to see it from the point of view of one publisher saying, this is something important about the games that we want to publish.
And we got a little bit of this from the [INAUDIBLE] gaming [INAUDIBLE] the other day.
But it's [INAUDIBLE] I tend to go to the publisher and try to say, this is a game that [?
could be ?]
interesting, definitely try to do some research on not just the kind of games that they make, but also how they present those games to people, right?
You know, you can probably make a good game.
Like, the people who are at those companies have seen enough good games to be able to see past anything you may have missed.
But it might improve your pitch a little bit and say, we've taken into account that you like-- I've taken into account that you like to position your games, present your games a certain way.
And this is how you could do it with the game that I'm pitching to you, even though what I'm showing right now, the prototype.
They didn't have that.
Just being able to be able to speak to that means [INAUDIBLE] you're aware of the company's strategy and how [INAUDIBLE].
So if you are talking to a publisher, which generally does [INAUDIBLE] you know, I don't know who publishes [INAUDIBLE] but someone does, because I [INAUDIBLE].
But to say that this particular set of game mechanics would be a great tie in for this particular property or something that you already have.
Even though that may not be something that you want to do, it's probably good for a publisher to be able to realize that you understand [INAUDIBLE] particular section of the business that you're in
It's also some so Fantasy Flight, don't want you to have IP attached to your games.
They're going to bring their IP to the games.
So they want mechanics.
They want rules.
They want how it's going to play, but they don't care about--- the space travel game.
You might sell for them.
And they say, all right, it's a [INAUDIBLE] fantasy game, the IP [INAUDIBLE] with us here
Also, probably gives you an idea of the kind of deal and contract you can get out of them.
Because if you say you want to retain absolute creative control on it and determine the art that's going to go on the box or something like that, you might even be able to get that contract from someone like Fantasy Flight, but it's something that might be negotiable for--
[INAUDIBLE] there you would talk to them about what Kickstarter campaign you're going to run.
That's what they're going to do-- a Kickstarter campaign with the game that they're publishing with you
So in other words, before you go and do a pitch at the publisher, just don't go in with a [?
generic ?]
pitch.
[INAUDIBLE] go into something like a convention.
You know, there are games that just have different [INAUDIBLE], for instance.
[INAUDIBLE] of EB Games.
There's one that's going to be right here on campus.
If you've got a prototype that you want to let people know about and make-- I guess we don't get many publishers here, but you might get--
That's probably going to change soon
Yeah, Gathering of Friends is one such group where you have inventors and publishers all sitting at a big table.
You don't need to [INAUDIBLE] prepare something special for publishers [INAUDIBLE] because they show you to multiple publishers at once.
[INAUDIBLE] actually going to submit it to Hasbro or something that.
You might just want to say this might make a great Transformers game because parts connect to each other or object [INAUDIBLE] mode, or something with that.
Yeah, that's what I want to say about that.
So, today is [INAUDIBLE] play test day.
It's almost 2:00.
We had all our parts up here.
So, how about we start play test at 3 o'clock?
That should probably give you enough time to be able to get something playable by then, I'm hoping.
Figure out what you want to test, how you're going to present it to people, and what questions you need answered to [INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, let's talk a little bit about today's reading.
[INAUDIBLE], this giant book that you see in front of you and that you have the link to-- how many of you, by the way, read the actual book and how many of you just went online?
How many got like a library copy?
[INAUDIBLE] It is basically divided into three sections, there's rules, play and culture.
Basically it's a collection of just what other people have said, but tried to organize in a way that makes for a pretty good textbook.
The first chunk is all about analyzing games, of systems, of rules, the bits that actually go into the construction of games.
The play section, which is what today's reading was from, is really about what people do with these games.
And then culture comes after that, which is the world around the games, the world that might have been created by the games as people actually interact with them.
I don't think we have that many readings, this might be the last reading from this book.
Yeah, let's see, culture.
We've got-- oh, there is?
OK.
There is one more reading, but I don't know if it comes from this book
[INAUDIBLE]
Yeah.
Bernie De Koven is the last reading, and that's going to be Wednesday, I think?
And that's going to be-- that's one of actually the writers that's cited quite a lot in today's reading.
And it's a much smaller book.
It's this thick and it's easy reading.
But it's a good jumping-off point for this class because this is literally where the book ends on thinking of games as designed objects.
And things that could stand alone as a product, there's a set of rules that can be passed on from generation, this solid thing, into what people make out of all of this.
And if you're interested in that, then there's another class-- I think it's CMS 616 Games and Culture.
And it should be offered-- I can't remember if it's a fall class or a spring class.
Spring class?
OK so it's offered this semester.
If you're not a senior and you're going to be around for another year you might want to consider that if you're interested.
To sort of see where it is going.
And obviously there are a lot of researchers here at MIT that are starting to get into the realm of sociology.
There are folks here who study history of games, folks who study what people currently do with games.
For instance, how many of you have met or taken a class from T.L.
Taylor?
A few people?
Yeah, OK.
So she comes from such a sociological background.
A lot of her work is ethnographic, which just means it's like a researcher going into a native culture and seeing how people behave there.
And what they value, and what are the common practices.
And she does it with massively multi-player online games.
Now she did e-sports, that was the most recent book, now she's looking at live-streamers.
And she has a lot of research, a lot of information.
So that's a good person to learn from.
Some of you also have come from Todd Harper's class.
He's a postdoc here in our lab.
He's taught quite a number of classes, both CMS 300 and CMS 100, as well as his own game design for expression class, where he looks a lot at competitive styles of play-- I'm sorry, competitive communities of play.
So fighting games community was his dissertation.
Looking at League of Legends right now, I think, pretty much what he is looking at now.
As well as some other aspect like queer games and the people who play and make them.
So that's another good angle to look at.
So fact to today's reading.
There's a couple of really, really useful ideas in it that can be used for a variety of ways to look and analyze your game.
Especially when you're trying to observe people playing your game and trying to figure out what's going on here.
And the games that we've got today, including some of the games that John has are all the sorts of games where what's interesting about the game isn't so much what's written in the box, what's written in the rules, but what happens between people.
And I would argue for many, many games, even the ones that are traditionally recognized for the magic of what's inside the box, what's actually interesting is what happens around the game.
Last week we talked about Go, for instance.
And it's like, OK, what's actually written about the rules of Go is not that elaborate, right?
Really, really simple, and mathematically is really elegant.
Entrancing in the sort of way that mathematicians like an elegant formula.
But what's also interesting is centuries and centuries of strategies and culture that have built up around this particular way of playing.
So there's a couple of useful tips that help us think about what happens in a game socially.
That's internal-- internal interaction and external interaction.
So let's talk about internal interaction.
What do you think Zimmerman's talking about when he says an internal social interaction in a game?
So I think he's talking about how you feel about your role.
So different people have different jobs in the game.
So I'm supposed to be doing something different than you're supposed to be doing, even though we both might have the same goal, might not even have the same goal, but the same ideas to win or whatever
And the reason why you have those differences is set up by basically how the game's rules are set up, right?
And so the internal social [INAUDIBLE] might be my idea about that.
My idea of what my role in the game is
OK so I think I'm mafia, right, in the game of Mafia.
I'm the mafia, or one of the members of the mafia, so I think I'm going to play this way.
The game rules tell me about when I can communicate and when I can't.
Basically everyone can communicate at the same time then everyone shows [INAUDIBLE] And then you come up with the strategy of how you're going to play it.
It evolves from the roles that was given to you by the rules of the game.
How about external interactions when it comes to social play?
What's the flip side of that?
Friendship outside of the game [INAUDIBLE].
Mafia's [INAUDIBLE] Say your friends not one of the Mafia.
You'd be like, oh, he's my friend.
I'll save him for last
Yeah, I'll torment that guy for last because he's my friend.
I'm not going to eliminate him right away
Another interesting one was, I think, also the past experience with Mafia.
I know that a person's really, really good at Mafia, so I'm going to kill them on first round just because I don't want to deal with it
Right, I just want to get them out of this game.
Sure, yeah
I think that happens with a lot of games.
Just like you team up on [INAUDIBLE]
Couples playing a game and that's supposed to be all where everyone's supposed to be playing an individual game.
Yeah, knowing that someone is a pathological liar.
So that actually also comes in design, by the way.
I used to write games for the MIT Assassins' Guild.
And there were four people who were playing games at the same time when I was writing games, and we always used these four people as ways that games are often broken mostly through weird social interactions.
Someone who's going to be the rules lawyer, for instance.
And someone who's going to stop the game in order to make sure that his interpretation of the rules is what everybody else is going to play with.
Not necessarily to make sure that the rules are adhered to, just make sure that his interpretation is going to work.
Somebody who's going to stop a game because they're always going to keep asking for reminders on how the rules work.
Just like, uh, I can't handle this right now, all this is supposed to be happening simultaneously, too much math, stop.
And I'm going to continue that, so this is the second kind of person that I had to design for.
The third kind of person, the person who is so capable of just summoning the collective brainpower of everybody to be able to churn through any kind of puzzle that I can create, because this person was such a convincing speaker that basically he can use your head like a node on a parallel computer.
You would just say, can you do this thing for me, it won't take a minute, and somehow he will do it.
This person had that social skill.
I'm trying to remember what the fourth one was.
Oh yeah, there was one person who could do the same thing entirely on his own.
If you put a puzzle in the game, the puzzle will be solved by this one person sooner or later, if it wasn't already solved by someone else.
So here's this giant macroeconomic system that I've set up, or something like that.
And that one person will figure out the optimal way to solve everybody's problem simultaneously.
I could design for that.
So I could design to make sure that each of these people don't break the game, but still can do the thing that they want.
Some of the folks that I'm describing actually enjoyed that, some of them didn't enjoy that.
That's just how the brain works, but if the game falls to that then they I'm not having any more fun than anyone else.
So I knew something about who was going to be playing my games by knowing that these four people.
I could cover a very wide range of different ways that my games could just fall flat on their faces.
These are real people, but anybody who plays my games could exhibit that same sort of behavior at an opportune time, I have to make sure that my rules could hold up to that.
So when you design your games, how many of you have somebody in mind?
Someone particular about who's going to end up playing the game.
Friend, family member, person in class
For me, I think when I worked on the game [INAUDIBLE] Hurricane, I think that it definitely reminded me of specific friends that I played Twister with in the past, a game that I felt was a game to get inspiration from.
But sometimes I don't have [INAUDIBLE]
The easiest one for me is just developing for myself.
And people who enjoy the same types of games that I do.
If I find a game fun, then people who enjoy the same types of games that I do will likely find it fun.
If I don't enjoy the game, I shouldn't be making it
Yeah, you are a sample point, you are a valid sample point.
It just happens to be one that you know very well-- yourself.
There is a mantra out there that often mentioned by people in the indie game development community, which is you make a game that you want to play.
And I'm not entirely sure that I agree with that.
But I can see where it is coming from.
Because if you can't satisfy yourself, then the likelihood that you're going to satisfy any other audience member out there who would actually play your game is lower.
But I think there is something about having to design for people who are not you which is also valid but obviously it's harder
I've heard it described as a Venn diagram.
One is games I want to play.
The other one is games other people want to play.
A third is games I can make.
Ideally, you want to be right in the center
Yeah.
Vast set of ideas that you have in your head, only a small group of which are things that you actually want to do.
And then a smaller group of that are things that other people want to do.
A smaller group of that is something you can [INAUDIBLE].
That makes sense.
The feasibility thing is what we've been addressing this entire class.
How do you actually make a game that you can make, as opposed to make a game you wish you could make.
So if we've got lots of different kinds of people who you're making your games for, people who you played similar games with in the past, for instance.
People who you know are the kind of people who tend to-- who won't break your game.
When you're making the Twister game, you have to make sure that you can play the game without breaking it, because you are physically larger than a lot of folks and able to reach much further.
And make sure there are rules, that you can design your rules for that sort of thing.
But then there's also broad categories of people.
And that's why we get into it the reference to Richard Bartle in today's meeting.
I did not include the whole article, but the whole article is online in plain HTML.
If you just do a search for Bartle-- I forget the name of the article, but if you just do a search for diamonds, hearts, clubs, spades, Bartle, you'll find it.
I think it's "Players Who Suit MUDs" or something like that.
A lot of that work was created when he was basically the originator of the very first multi-user dungeon.
Anyone heard of a multi-user dungeon?
How many have played one of those.
I've played way too many.
You see people playing, when you're reading the text that is typed on the screen
[INAUDIBLE]
OK, so how many of you have seen or played a text adventure?
So text adventure, basically everything is text.
You've got verbal descriptions instead of graphics of the rooms that you are in, and the objects you can pick up, and what you can do.
You got an inventory that's explained to you text.
You've got text commands you can enter.
Usually when you're entering commands it's a verb and a noun.
Get key, open mailbox, get [INAUDIBLE], that sort of thing.
Some are just single words.
Inventory, north.
Actually the full north command is actually go north, but there are shortcuts where you can just type N. North, south, east, west.
A multi-user dungeon, traditionally, was a multi-player text adventure.
Everyone would just occupy the same space on a server, they would use something like Telnet, or maybe a dial-up, on [INAUDIBLE].
And call into a server that will be serving them ASCII text.
And then simultaneously going to two other people.
It's very, very important that in multi-user dungeons, multiple people are connected simultaneously.
It's not like an asynchronous thing where one person connects at a time, even though they're all logging in to the same world.
So not all that different from what you expect from MMO RPG nowadays, the massively multi-player online role-playing games.
Only obviously no graphics, and much slower pace of game, because everything has to be typed.
You don't just [?
walk ?]
[?
off, ?]
you have this inventory.
All my examples are really fast examples.
Give envelope to Bob.
It's like you could actually give that kind of thing and then you just transfer an object to somebody else named Bob in your space.
And he described four different categories of people that he often saw play his games that I think still holds up pretty darn well.
So who remembers any of the four categories?
The kind of people who really get deep into underlying [INAUDIBLE], who want to learn about how things work
The diggers, right?
The miners.
The people who-- Minecraft is a great example for a game that is designed just for those people.
People want explore how the system works, how the environment works, who want to discover things and to see what's behind that next mountain
I'll go with [INAUDIBLE]
Killers or plugs, and what was the justification for what they do?
What do they do?
[INAUDIBLE]
The people who are there for the other people, primarily.
In a way, killers and the socializers are there because other people are there.
There are different reasons.
The diamonds want to win.
That is by any definition of winning.
Now MUDs traditionally did not have a definition of winning.
You couldn't win that game.
There wasn't any final bosses you could kill, so you had to find your own metrics for winning.
So it could be there are PDP type diamonds.
You want to rack up the highest tournament score or something, but you're not there explicitly to cause people grief.
You're not trying to hit them when they're not looking.
Sort of like we're going to organize a duel kind of thing.
We're going to see who can play this game the best, as opposed to making people upset.
There's a way to remember this that I think has really been useful.
I find the playing card suits not so easy to remember, but I could actually remember the categories pretty easily by whether you are acting on something or you're acting with something.
And the question is whether you are acting or interacting.
I just put two different [INAUDIBLE] that meant [INAUDIBLE] It's also supposed to be this.
Interacting with [INAUDIBLE].
And then at the world, all the systems, and then other players.
So if you're interacting with other players, we've got the socializers.
And people who are acting on other players are the killers.
Acting on the world.
[INAUDIBLE]
[INAUDIBLE]
[?
Assignments, ?]
which leads this to be the [INTERPOSING VOICES]
[INAUDIBLE]
These are the glitch finders.
These are the people who figure out how to get outside the boundaries, your invisible walls.
Who want to see whether there's a way they can get below the very lowest layer on Minecraft.
And what happens when you cram so much explosives in one area that you can't perform this experiment without killing yourself.
They're the ones who are going to do it.
Because they want to see what happens.
Whereas these folks are really a little bit more about how do I use this world to be able to obtain some sort of other goal.
Which is usually points, some sort of achievement, some sort of recognition from the other players.
They don't usually get the recognition, but they have a numerical number to show that they're deserving of it.
And I find this to be a really useful way to think about what games can do to be [INAUDIBLE] to different people.
Because the truth is that even though when you play a single game, you may be primarily one of these people, you may not play every game this way.
Say I'm a [INAUDIBLE] someone who's trying to chase all the points.
Someone who is trying to get the top of the leaderboard in [INAUDIBLE] or something.
But that may not be the way that I'm going to play World of Warcraft.
That may not be the way that I play contract bridge.
I might be playing more socializer when it comes to contract bridge.
And different people are often different things in different games.
And sometimes different things in the same game.
Right now, I am trying-- everyone in this room probably knows by now, I play too much StarCraft, right?
Usually, I am trying to win.
I am trying to raise my rank.
They have a ladder system.
Sometimes, I just want to zone out and talk to people.
I just want to type-- I just want to go into free-text chat and be friendly for awhile.
I'll do something very dumb that takes no actions whatsoever.
And I'm just like, at least my fingers are going to be able to type my [INAUDIBLE].
And that's what [INAUDIBLE].
Different times of the day, different moods that I'm in, I'll do something different.
And if you can make a game that would be able to cater to people in different moods, then that's great.
You've got a game that can be played in different situations, or more importantly, among a group of people who are all feeling kind of different-- [INAUDIBLE] type.
This is not the only breakdown that I've seen.
There is a system that's been pitched by, I believe, a developer at Ubisoft Montreal-- although don't take my word for that-- Jason Vandenberghe.
And he suggests, actually, a system on the OCEAN model of psychology.
Has anyone heard of this?
It's five big drivers of motivation that psychologists have, basically, created as a framework.
And one of them is neuroticism.
There are people who are motivated because they are neurotic.
And I can totally identify with that.
It's like they want to be-- it gives them comfort to be performing neurotic things-- things like hundred-percenting a level, or a game, or something like that.
It's a motivator.
It's a driver.
It's no particular achievement to hundred-percent a level in Metroid, because a lot of people do it.
But the fact that that number is there motivates a certain group of people to be able to bring that number from 99% to 100%.
So I want to make clear that this is just a theory.
This is just-- Richard Bartle-- creator of the very first MUD in Oxford, I believe-- United Kingdom-- and he just wrote an article on a website-- he would have put it up on a blog and blogged it at the time-- about, these are the people that I happen to see play my games.
And if I-- I can generalize these people in these four categories.
Let's see.
Other stuff that's covered in today's reading includes the idea of people bringing things from outside of the game into the game.
And we talked a little bit about metagames.
And this article goes into a little bit more detail about that.
So there's-- that's, again, four things that the book goes into when it comes to metagaming that-- and these were posited by Richard Garfield, designer of Magic: The Gathering.
But he was-- the way how he was describing it is that these are all kinds of metagaming experiences.
And I'm trying to make a game that capitalizes on people playing the game when they're not even really playing the game.
When they're not sitting in front with their decks of magic, they're still playing the game, because they can do something else.
And so, anyone remember what those four things are?
Things that you bring to the game--
Yep
Things that you take from the game--
Right
Things that you do between--
Mm-hmm-- between games-- yep
--people-- between games.
And the last one is--
Last one is, actually, probably the more traditional understanding of metagaming.
So if-- even if you haven't read, you might have an-- or, you might be able to hazard a guess.
You got through--
[LAUGHS]
--spot on
[INAUDIBLE]
That's right-- things that are-- things that you do during the game that are not, actually, part of the game rules.
So what do-- what are some examples of things that players bring to the game-- from outside of the game they bring into the game?
So there's physical objects, but also knowledge
Right
So to the baseball game, it would be like your bat
Right-- your ball-- things that you might require to-- not have even start the game in the first place
[INAUDIBLE] it might be cards, I guess
Mm-hmm
But you can also bring knowledge.
So if chess is a game that you're not familiar with that you study up-- they call it booking up-- on opening, different tactics, things like that that you can study, bring into the game-- basically, like preparation
Yeah, yeah-- so if you are going to call into a competitive game, you might come in with a strategy that you intend to execute.
That happens in sports a lot.
We train these particular maneuvers-- these particular plays
Even, like, metagaming [INAUDIBLE] that.
I guess that might be [INAUDIBLE].
Magnus Carlsen-- the world's best chess player-- he played openings that are not that good, just to screw with people
When you're playing the players?
Preparation, yeah
Yeah, and if you're playing the player-- I think that still fits within the category of fix your brain to the game, because that means that you are bringing to the game an understanding, or a guess, about what your opponent is like, and how your opponent may have played before.
Someone's reputation is, actually, [INAUDIBLE] in today's reading as something that you bring to your game.
It's not something that you voluntarily bring to the game.
It's just something that precedes you.
So then, what stuff do you take from the game?
I mean, so I guess the typical items might be enjoyment from the game, or whatever.
And Avery talked about [INAUDIBLE] Magic, where you win a card if you win the game
Yeah, yeah, there can be prizes.
There can be cash prizes.
There can be stakes
If you're gambling-- [INTERPOSING VOICES]
--incentive [INAUDIBLE]
Yep, again, reputation
You can also take knowledge from the game
Sure, especially a game that you lost, right?
Yeah, exactly
Yeah
I think that's what [INAUDIBLE]
Yeah, you-- hopefully, especially in a game that-- in the kind of game that I wish I could design one day, you take away something that deepens your understanding of the system that you just played.
And that's wonderfully intrinsic motivation of something that arises from the rules.
But it's not going to affect the game that you just played.
It's going to, maybe, affect your next game
This is super cheesy, but you could take away new friendships
Oh yeah, well, social games-- there are people who end up in this area during--
[INAUDIBLE] enemies.
[LAUGHTER]
Or new enemies, yeah.
You can--
[INAUDIBLE] diplomacy games
Yeah, well-- yeah, diplomacy is great at creating bad relationships.
So yeah, OK.
So the stuff that you take away from the game makes a lot of sense.
The interesting thing about the ante situation is that that almost moves into a third category, which is the stuff that you do between games.
Specifically, you're taking out of the game something--
[INAUDIBLE] you said that Richard Garfield made that game.
Was he the first person to do that?
I'm not sure about that.
But he made Magic: The Gathering.
He's the designer of it
OK. You said that he was [INAUDIBLE]
No, no, no, no, no, no
[INAUDIBLE] as soon as the game was created
Yeah, yeah, yeah.
No, no
[LAUGHS]
Well, I mean, he wrote it into the original rules.
And nobody liked that part.
Oh, I think very few people seemed to like that part of the rules.
So--
Yeah, every [INAUDIBLE] been banned for as long as the game's been around
Well, OK.
But it is original rules.
But that's a mechanic that-- it's a mechanic imposed on top of the game.
I guess it was part of the game, originally.
But now, if it's played for ante, it's being played outside of the game, because it's not part of the rules anymore.
Where, if you put your game-- if both of you are putting a card up-- say, a really, really skilled player is going to play a really, really not skilled player.
And the not skilled player says, well, chances are, I'm going to lose.
But the skilled player is anteing up an incredibly valuable card that I might not, otherwise, be able to get.
So I'm going to ante up this crap card that I will always be able to replace if I lose it.
But it makes it worth playing that round, and then the game [INAUDIBLE]
It's an interesting [INAUDIBLE].
So playing [INAUDIBLE] a game that would have some luck in it, where it actually has a very high [?
appearance ?]
of luck involved in there.
And so, it's really, really easy to play this game and talk about-- and say that you got really, really, really unlucky-- or to say that you got really unlucky.
And I have definite [INAUDIBLE] after playing this game.
It was [INAUDIBLE], but how unlucky they were during that game
Well, OK.
I mean, that's something-- yeah, sorry, sorry.
Go on
Oh yeah, that's just-- oftentimes [INAUDIBLE] people have a tendency to be lucky or unlucky in this game.
And they're, like, well, I've lost-- and someone might say, I've lost-- [INAUDIBLE] Oh, I've lost a lot of [INAUDIBLE] games in a row.
But I'm just unlucky in all of them.
Something like that there
Well, something that happens between games-- it's like-- you brought up a couple of things.
One is the discussion about how this game seems to be operating.
And on Reddit, that tends to be a very angry conversation among friends who have played games of Twilight Struggle, or something.
It's a combination of storytelling, which is, man, we're just being able to recollect how that game went, and how unlucky I was.
[INAUDIBLE] all the time.
Man, that dice just does not want to roll sixes.
And I think that's more something that you take away from the game.
But if you're going to go back into the game and then understand, OK, I thought this was a game all about logic.
Now it's really just luck-based.
And then you're going to the next game of the same-- you're going to play the same game again.
You're bringing different expectations.
So you're adjusting your expectations.
So I will put that in a category of something you take between games, in adjusting your expectations.
Then what happens during a game, other than the game itself-- stuff that's not in the rules?
Yeah?
There's taunting, or--
Trash talking?
Yeah
Yeah, trying to mess with your opponents psyche while the game is going on
Also just more metagame stuff-- thinking strategies over in your head, and playing out different scenarios of what might happen
Right.
Does this person even know about this strategy?
I am seeing something that looks like the player might.
But really-- are you really going to do that?
It happens in card games all the time.
What else happens during the game other than the game itself?
There are times when gameplay just stops.
And I, actually, gave a couple of examples of that.
What's the out-of-game considerations that forces the game to stop, or things you have to do to keep it going?
Remember any of these things?
Shuffling a deck of cards?
Those are usually mentioned in the rules.
You reach the end of the deck, and then you have to reshuffle your deck.
I meant physical, safety considerations, for instance.
Somebody fell down in the middle of a game of tag, or something.
And then it's, like, nothing in the rules say that you have to stop the game.
But people do, because you want to make sure the person's not hurt.
It happens during the game.
And metagaming is sort of concerned.
But it's not, necessarily-- and the reason why it's important to the game is because you want people to say, hey, I want to play tag again.
I want to play tag.
But I want to know that I'm safe while I play tag.
So if people understand, on the metagaming level, that if somebody looks like they might be hurt, the game will not continue until we verify this person is not hurt or got this person adequate medical attention, for instance.
But that doesn't necessarily always have to be like a physical hurt.
That can be-- there's a great account in the reading about playground groups fighting over what nice rules meant, and the words that they were using to be able to describe what nice rules meant.
And they had become increasingly-- at first, they were just starting with, well, we're going to be playing nice.
They call that [INAUDIBLE] rules.
And then, people who didn't want to play nice started joining the group to play Four Square.
This is not the mobile game, Foursquare.
This is the bouncing ball version of Four Square.
And then every time they did something outside of the rules, the people who were trying to play nice then had to add in-- get more, and more, and more specific about what nice meant.
And that wasn't actually the intent.
They didn't actually want to get all that specific.
They just wanted people to play nice.
Because once you get more and more specific-- because there's so many kinds of rules-- lawyering mode, then the context should be nice.
So that's something that's happening during the game.
The game's actually still being played.
Part of the game of four square is just bouncing this ball around.
But if all these rules-- negotiation-- that's happening to be able to try to steer the game into the direction of different-- in this particular case-- different groups of kids prefer to play the game.
One of them-- one group wants to be very competitive and winner takes all.
The other side was like, the whole point of playing this game is so that we're not annoying other people-- the traditional-- this divide, I guess.
So Magic: The Gathering was, ostensibly-- this could be something that he came up with after he had designed the card game, in order to justify why he decided [INAUDIBLE] the card game.
Often when you ask a designer or a writer, why didn't you do this, it's very easy to come up with a justification that sounds good.
But really it isn't necessarily what they were thinking at the time when they came up with the game.
So we have no way of knowing.
Even Richard Garfield will not be able to tell you honestly whether that was the case or not, because no one really remembers very clearly what they were thinking when they were designing the game.
But it's a good justification, I think, which is, what does a player bring to a game of Magic?
Well, they bring their deck of cards and the knowledge about how that deck is supposed to work, because they built that deck themselves.
And it's supposed to be this little engine to be able to play our very specific strategy, which is something that they want to try for the next game.
They may take the ante away-- but suddenly all of that knowledge about, OK, now I know that my deck is actually a pile of crap, for instance.
What happens between games?
You assemble new decks.
You chat on forums.
Maybe new cards get released.
And then you have to try to figure out how your deck is going to work against them, and then what happens during the game itself.
I don't know, actually, how much taunting happens on Magic tournaments.
Anyone play Magic competitively?
Is there taunting?
There's never anyone actually taunting them, like, you suck.
I'm going to destroy you
All right-- [INTERPOSING VOICES]
OK, the Magic people are going, you sure you want to play that?
You know, that sort of thing
I got goaded into that once-- yeah, actually-- being competitive [INAUDIBLE]
[LAUGHS]
There's also a huge backlash right now of people just being animals during games, and just--
Well, yeah
The tournament organizers are, specifically, supposed to look out for that and try to not do that, because [INAUDIBLE] bring them back to play
Right, because they're trying to build up a social community
I've already-- [INAUDIBLE] admit that you have something that's-- that you have a [INAUDIBLE] you can finally use.
Or if you've gotten to a position where you're about to use an ability that will almost win you the game.
And sometimes your opponent will realize that he's unable to stop it and [INAUDIBLE].
And so, I read a comic which was about someone like, oh, my God.
I'm going to get to use my play blockers all for the first time.
[INAUDIBLE] And he's, like, I'll pay you money if you just want to keep playing [INAUDIBLE].
[LAUGHTER]
Wow!
This didn't actually happen
But that would be hilarious.
OK. All right
[INAUDIBLE] to an entirely different level of gameplay.
There's a famous story about one of the best players in the game-- forgot to bring the one card in the deck he could win.
So he completely controlled the game and then couldn't do anything [INAUDIBLE]

He won the whole tournament
He won the whole--
He got every, single person to concede before he had to play [INAUDIBLE]
Yeah, sure, sure
[INAUDIBLE].
Why wouldn't he?
Yeah.
[LAUGHTER]
Well--
And it brings another level to the metagame, where everyone knew the deck.
So everyone knew exactly what he was trying to do.
And at the time, where he was just going through the motions, they were, like, OK, sure.
I know what happens
You know this person has this card.
You just don't know that person had the card at the moment.
Right?
[LAUGHTER]

Just building off of that, [INAUDIBLE] very limited StarCraft higher-up games that I've [INAUDIBLE].
There would be often times where neither base were destroyed.
But one person would, automatically, concede defeat
That's the majority of StarCraft games, actually
I never understood it.
But they're not still going.
[INAUDIBLE] Their economy was completely destroyed
Yeah
Well, in that game, you can lose [INAUDIBLE], but not, actually lose [INAUDIBLE]
You can stall the game for a very long time by just running around building pylons across the map.
[LAUGHTER]
It's a game full of positive feedback loops.
So it's one of those things where, once you feel that things are snowballing, if you are right that things are actually snowballing out of your control, then yes.
There's no way to actually win.
But yes, you can draw it out.
But theoretically, there really shouldn't be a way to win, because it's a game of positive loops-- of positive feedback loops
Some people quit prematurely
Yes
Adrian was famous for rage quitting and commentators [INAUDIBLE]
Did he just rage quit?
It was pretty even
Yeah
He could still come back
And that's the thing.
If you are wrong, and you are not, actually, in a bad situation-- if you are actually pretty even-- then, because it's a game of limited information-- imperfect information-- you don't know exactly how bad off you are
On the [INAUDIBLE] the person not having their [INAUDIBLE].
So that's-- sometimes you-- [INAUDIBLE] you make them play it out.
I want to make sure that-- [INAUDIBLE] if the person didn't know how to win.
They had it.
But they just didn't know how to use it properly to win.
And so [INAUDIBLE], except for one person who's like, I'll make them play that.
But then they couldn't do it.
But one of my friends was playing [INAUDIBLE].
And he had-- he was in a situation where he could make arbitrary many copies of his card.
And every time he made-- his copies would last for one turn.
He'd [INAUDIBLE] And so, he ended up-- [INAUDIBLE] made him go through [INAUDIBLE] and said, [INAUDIBLE] and click, and actually make 50 copies of his card.
He just spent almost 10 minutes going through and making copies of this card.
And if he just clicked-- if he made a specific [INAUDIBLE] that was easy to make, he would just lose the game.
So--
So, yeah-- so execution error-- I mean, that happens even in real-time games like StarCraft.
Yes, you should not be able to win this game unless your opponent makes a huge mistake.
And if you're-- that happens, by the way
There's another interesting thing that happens from hidden information in League of Legends, where, as a spectator, you can see how much gold both teams have total.
But you can't see, even, how much gold your own team has total, when you're actually playing.
You can only see number of kills, and maybe number of towers, and relative farm counts, basically.
But you can't actually tell, oh, we're actually up in gold.
So some of the interesting things will happen where a team can be down eight kills or 10 kills or something, but still be even or better in gold just from having gathered different objectives and more [INAUDIBLE] and stuff
And I think bringing this to a point of design, whether you allow players to concede the game in progress has, actually, been something that's been actively enacted in DotA, and Moebius, games like League of Legends and in DotA 2.
DotA 2, in particular, for a long time-- and I'm not sure if that's still the case-- actually did not allow you to concede the game in progress.
Can anyone-- does anyone play DotA 2?
Anyone confirm this?
Once.
I think it still doesn't allow you
Still doesn't allow you?
I haven't played in awhile
OK, it's definitely one of those things that struck me as, what?
And it's because of these reasons.
It's because-- just because you think that you've lost the game, doesn't necessarily mean that you lost the game.
And the designers decided that it's more important that you fight it out to the bitter end than to concede.
But of course, just because you can't quit the game, doesn't mean that you can't quit the application-- [LAUGHTER]
--or [INAUDIBLE].
Huh?
Yeah?
What?
[INAUDIBLE]
Yeah, [INAUDIBLE].
Or you could just choose not to do anything-- deliberately walk into your opponent [INAUDIBLE].
Let's see, your hand?
The reason-- one of the reasons that I don't play DotA is because they don't allow you to quit.
It's irritating to me when the game is clearly over for me
Yeah, I mean, it's a good 40-plus minutes per game
Yeah, it can take a long time to finish out a game
Yeah
And I want go do something else, or I just want to take a mental break.
And they won't let me
I watched the presentation on [INAUDIBLE] last week in another-- in a class I'm taking.
And I believe a lot of the justification for that was also that it irritated people who were allegedly more serious gamers, because there are all these people just casually playing the game who will just give up.
They ruin it for the rest of us who want to keep playing
Yes.
They are the ones who feel if you are slightly behind you can still fight back.
Whereas, people who just play this game to chill out.
They're just like, oh, I don't feel like fighting back right now.
Let's start a new game.
And if they had the ability to concede they will do that
I was just thinking that don't the most serious of gamers join with the team and they're on the--
Yeah, absolutely.
And even when you're playing by yourself in these games, high level players don't concede, because they know can come back
But it's a spectrum, right?
You don't start off by being serious.
You start off by being sort of, hey, what's this game all about, and end up this middle tier-- well you're still playing with strangers, but you are getting really interested in doing well.
And right now Todd Harper, which I was describing, is actually thinking pretty hard about what is the value of having taunt as a game mechanic.
Fighting games have that action which deliberately wastes time and opens you up to attack, right?
But the whole point is that you will do that in order to-- It's the digital equivalent of trash talking.
I'm going to make this move, deliberately making myself vulnerable.
Because I know I cannot lose this game, because you are so bad.
Or I think you are so bad.
I think there's things like that in League of Legends although I don't know about the others.
And so the question is, what is the real value of that?
You can already taunt.
Especially if you're playing something like an online game on the computer, typing up a taunting comment takes actions.
That means that it takes your fingers away from actually playing the game, because you have to type these words out.
And as a result of that, it's already a drawback to be able to do that.
Why do you need a game mechanic?
Which means that game designers are explicitly supporting you doing the taunting
I think that actually, at least in League, when you have a character you have four.
I think it's joke, laugh, taunt, and dance.
But actually people just do it for fun.
For example, with high ELO streams sometimes all 10 people will group up and have dance parties, in close agreement to not fight and just goof around.
Other times it's really funny.
I saw this stream where Sion was getting jumped on by two people and decided to dodge a skill shot by doing a taunt, which is taking his axe and going like this.
And then dodging the skill shot, and then just walking out.
So that's just funny little [INAUDIBLE]
Right.
In Starcraft you can make your characters dance just by clicking back and forth real fast.
Then they just walk left, walk right, walk left, walk right.
And I've seen that and other game mechanics that aren't explicitly taunt mechanics.
To be able to tell the other player, no.
You've basically lost.
You should just concede now.
Or even if you don't concede, I'm going to win this game anyway.
So you might as well concede now.
But then they also have a slash dance command to actually make your characters dance.
And that's a game mechanic, right?
When you fight slash dance you do no damage, and you will take whatever damage is being inflicted on you deliberately.
And I've seen a couple of games lost because someone decided to type slash dance.
And that's the risk.
But again that's the game explicitly supporting this this sort of behavior.
It's supposed to be meta.
It's supposed to be the sort of thing that happens above the game.
We're going to be playing a bunch of games.
Obviously these are board games.
The talking, the communication, what's going on in this game?
And what are you going to take out from the game?
It's all happening above the board as opposed to inside the game.
Except for Space Alert.
Maybe Space Alert it is all just part of the game.
I want you to be thinking about this, about what's happening above the table.
about what it's about.
Information they are exchanging or interactions they are having that's not part of the game itself.
So what does the game explicitly do to be able to support social interaction?
To be able to either support the kind of social interaction that's happening inside the game system, or to be able to draw on the social relationships that you have with other people on the table to be able to enhance the game experience.
Did you have your hand up?
So in League, theoretically it can be real-- there are people who often-- The enemy should not just try to use their abilities on me because I taunt them.
There has been a case.
I just sit there taunting and then their enemy uses all their abilities on me.
And then my team ends up winning the fight, and the record off of that.
Sometimes taunting someone will have an actual impact
But you could arguably have achieved the same goal without and explicit taunt, right?
You know, just by standing right in front of your opponent
In your taunting you're telling the opponent that you're not really paying attention.
That you could also be attacked.
You're less likely to dodge that move
True.
Something to keep in mind when you're designing your games is that there are all these things about how it would be fun to be able to design mechanics to be able to support these kinds of social interaction.
But you got to be very, very careful.
Right?
And every time you design something you're both giving your players a tool to do something else and it's now officially mandated by your game.
Because you have a rule that actually says that this is something they can do.
And they may use that rule creatively.
Or they may use that rule destructively.
Again it all comes down to testing.
You don't always know, but you should know what you're trying to achieve with the game socially when you're making a game.
You should have that conversation today when you finally break up into your teams.
But for now what we're going to do is just play a bunch of board games But primarily about social play.
And then on Wednesday I think we're going to bring you [INAUDIBLE] games with very changing rules.
John, you want to talk a little bit about these games that you brought?
You want to do it right now?
Yes.
And then we'll get all the games out and have a break simultaneously.
All right, I'll start with this one.
Has anyone played this?
OK, maybe you could help me explain it
I watched two hours of videos and I don't really get it.
There is really a lot to it.
There is a lot going on.
But the gist of it is that everyone gets to pick a role.
And each role pretty much stems straight from the TV show.
Have you guys seen Battlestar Galactica?
It's a really good show.
It's on Netflix.
You should go check it out.
And actually it seems like the consensus on the board game community online actually really like this game.
So it seems there's a lot to it.
But essentially you choose a character from the game, and they have different powers.
Some of you are human and some of you are Cylon.
The Cylons are robots that look just like humans.
So there's spaceships coming to get you, and some of the Cylons are on the ship.
And their goal is to basically wipe out humanity.
And the human goal is to survive and you don't know who is who.
There's a critical moment in time where the Cylons will reveal themselves and get even better powers.
So there's lots of cards.
So there's a military leader and a political leader.
And they each get different decks of cards.
If you're the political leader you basically can put people in the brig.
There's so many components
It is, I'll say, the most complicated game that I've ever played.
And this is true of many replays
I think you have to find it fun, because after you invest all that time to learn how to play it, you're not going to not play it
A lot of the rules are sort of unnecessary.
You can literally clear 90% of it.
And the end of the game has all these weird rules.
It has too many rules, but it's still fun
Yes.
So depending on who you are, you can either be a military leader, a political leader, or you could be a pilot.
The political leader gets to put people in jail.
The military leader gets nukes, and they get to strategically use them against the enemy.
And whenever there's a decision that has to do with fighting, they get to take the lead on that.
So there's different roles.
And it's interesting that each role has different decks of cards that support that role.
The turn is really complicated.
It's a bunch of sets.
You draw a card, and then you collectively face an objective, a crisis.
And you all have to vote for commit resource cards, of which some can be good and some can be bad.
So the Cylons try commit bad resource cards to make sure that this objective is not completed.
And the humans tend to put in the good ones so the objective is completed.
But there's a lot of meta-game where the Cylons don't really want you to know that they're Cylons.
And then there's special abilities where you can flip a card up and see what everyone's resource card is.
I think they're called skill cards.
It's a cool simulator.
And you can jump.
There is so many components.
We probably don't want to play this game, because I think that the play-through is two or three hours.
But all the [INAUDIBLE] you might want to change
It's a great game to know about.
It's a great game to play, if you've got a full weekend dedicated to board gaming or something like that
And there's expansions, too.
This is a pretty popular game
I think we can play this in two hours
I think you'd have to leave it, though, to explain it to everyone
I want to make sure there's enough time for teams going to help you to verify [INAUDIBLE] So there's probably not enough time.
But I ask you to do that so that people will get a sense of what is involved.
Also look at it a bit.
One of the nice things I like are these little counters.
It's just a counter.
Five, four, three, two, one, 0, which means you have normal pod condition.
But it's a nice way to be able to keep track of stats
Is that OK for that game?
Yeah.
That's great
This game, Space Alert, has anyone played this one?
It's really cool.
I think this game is really neat.
Maybe I can talk about the board?
It's this cooperative game.
And you're playing as a team.
You're this crew on a spaceship there.
And it goes like this.
You have 10 minutes to prepare a hyper jump.
And during those 10 minutes new information is coming in from the audio CD that you play.
There's 16 different tracks, so there's a lot of replayability.
And at the end of the game, there's this frantic playing phase.
And there are going to be periods where you can't talk to your teammates.
There's periods where you can trade cards.
And basically there will be these trajectories, depending upon which enemies.
So there's the red part of the ship, the white part of the ship, and the blue part of the ship.
They each have an upper deck and a lower deck.
And there will be enemies coming at you from space.
And you man your shields, but also use energy to shoot the objects down and basically survive.
And if they do six damage to any one part of your spaceship, then it's game over.
When you take damage you get certain debuffs.
Your shields would become weaker.
So you have little avatars that move around on the board.
And you need to be in each room to do your action.
So there's an A action, a B action, and a C action.
So just taking a look at the white one, you all start here.
The C action is the computer, so the first two moves of the game.
I think there's 12 moves, and there is three sections.
And the first two moves of each section you have to have someone be the computer.
This is the gun and the shields.
The gun and shields are powered by green cubes that go down here.
And you have to constantly be replenishing them.
There are also internal threats.
Creatures will crawl into your spaceship, and you have to get these bots up and running to take care of them.
There is some really good tutorials and play-throughs on YouTube.
So if you don't get a chance to play, you should definitely check it out
The play-throughs on YouTube are probably pretty hilarious, I'm going to guess
They're amazing.
I saw this solitaire one where this guy was playing solitaire.
It's a really good way to learn the game
Playing the solitaire is probably a lot like playing FTL, if anyone's played that on iPad or a computer
It reminded me of FTL quite a bit
You imagine FTL where every single member is actually controlled by different players, then you start to get several more people there.
So you put these action cards down, but you don't actually resolve the game until after 10 minutes are over.
There is different difficulties, so you can put in harder cards.
There is a lot of details that I am missing
The primary one is that if you've played RoboRally, or any other kind of game that's got a little bit of a program event to it, you're basically programming your character to do these things.
You're kind of figuring out and visualizing where things are going to go.
And you can talk about what you are going to do, and what to do at specific turns.
But you won't actually know how it actually runs out until after the 10 minutes is spent.
And a big part of the fun is just flipping over the cards, seeing what people actually did.
Realizing that oh no, wait, we both clicked the same button at the same time
Or you're in the wrong room.
You think you're in one room, but you're in another.
And you're clicking the wrong thing.
Or two people take an elevator at the same turn and it will actually delay one person's actions a bunch.
So all your cards are laid out and that's your plan, but then a delay happens and you have to shift all your cards to the right
It's a game about coordination where every single mechanic in the game is to prevent that coordination from actually happening
If you listen to the tracks, there are periods where you're not allowed to communicate, which can be really frustrating.
And it's constantly throwing new monsters at you from all different directions.
So managing where the monsters are coming from, where the threats are coming from.
These are the cards that either move rooms, either to the red side or to the blue side, where you can hit buttons.
And you only have a certain amount of these in your hand.
And you have to coordinate with your teammates at a certain point to get the action that you want of your team.
Is there anything interesting?
Pandemic
This is a co-op game where everyone on your team is playing against the board.
You are playing against the computer.
So you're all a team and basically there's four deadly virus outbreaks all over the world and they're spreading.
And you need to research the cure.
There's different roles, which is cool, because in co-op games we have roles.
So there are five roles.
So at each turn you pick a couple cards.
And basically there are cards.
There's cities all over the map.
And your little guys go all over the world for treatment with these, set up research centers, research cures.
And you need a certain amount of cards, like five of that color.
And you have the role of your researcher, you only do four.
So the medic can cure people faster.
There is a logistics guy that can fly people around the world with ease.
And you only have so many actions.
And constantly there will be breakouts where if you get too many diseases in one area, it will cause the neighborhoods around it to be infected.
So you have these chain reactions.
Again, coordinating well with your team in a cooperative manner for a cooperative gain, which I wasn't able to do.
We lost a bunch
It is a tough game.
But the more you play it, definitely the more you start to get used to how the systems work, especially the explosions, the virus explosions.
Forbidden Island is same designer.
And I feel the rules are a little easier to understand.
It's not actually all that different a game.
We'll also bring it out for you to check out.
And these three games over here?
Shadows over Camelot.
The Wrath of Ashardalon.
We're here primarily so that you can take a look at that them, but we don't have enough time to play games in class today.
Shadows over Camelot is basically, is sort of Arthurian traitor mechanics.
Ashardalon is actually just Dungeons and Dragons, only packaged into a nice little-- You don't have to think too hard about creating your own character ahead of time.
So you can just jump right in and get into the playing part.
It's kind of interesting.
They take a chunk of the meta out.
The whole, how do you create a character for instance.
Which a lot of people do enjoy Dungeons and Dragons, but it does take an awful long time.
And how do create a campaign?
Well, the box comes with the campaign and all the maps that you need.
So this is basically one box, a Dungeons and Dragons campaign.
That's going into dungeons and killing monsters and getting loot.
I think we just played this one
[INAUDIBLE] I just played the expansions-- [INAUDIBLE] fade in and out of reality.
[INAUDIBLE] jump between different zones
I could see that
If there is-- there is [INAUDIBLE]
A lot of expansions are designed for people who've mastered the original game.
Because theoretically, if you play the game often enough and you're coordinated enough as a group, you should be able to beat everything that the game throws at you.
Same goes for Pandemic.
The question is that coordination part and some luck of course.
All the audio tracks will also be available on MP3.
If somebody wants to play Space Alert, I would suggest actually taking a corner of the room, maybe that one.
Because you need to be able to hear what's going on.
Actually, could you just plug it in there?
It's pretty loud.
If your computer has a CD drive you could do something over there
I feel like the tracks are designed to disorient you
But you need to be able to hear what's going on.
So it needs to be loud enough that everyone around the table can hear on top of all the talking that you're going to be making in the game.
So give it a try just on the computer speakers.
If we can't hear what's going on, then we'll try it again with other speakers.
We should definitely be able to hear.
Definitely try to pick a corner where the noise of the other games isn't going to bother you.
Also, just take a look at the parts
Two to four players for Forbidden Island, it takes about half an hour.
Two to four, for Pandemic?
For Pandemic two to four
And then this one, one to five.
[INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, thanks for showing up on time.
I really appreciate it.
It's kind of weird at this.
And I never see this front row completely empty, but this will be [INAUDIBLE].
Thank you.
That really makes me feel better.
So, today's structure of class is going to be very similar to what we did last time, which was we're going to talk a little bit about a reading, play a bunch of games.
[INAUDIBLE] the reading [INAUDIBLE] end of class probably with more time for you to work through your prototype.
Just as a reminder, Rick and I are here to do any play testing that you might want.
And we'll give you feedback.
Along with the feedback, we also put all the grades up.
Thanks to Rick for getting them all down for assignment two.
And all the-- if you look at the comments that we've sent, there are two sections.
One is a section that is just about your individual write-up which only you see.
And then there is a chunk that is the comments will be geared to every single one of the team members, which was basically based on your product itself.
Do take those in mind.
Do keep in mind because there's a lot of stuff there that-- a lot of feedback that we're giving you in the hopes that you will be able to put that feedback to good use for assignment three.
And Did you have a general comment?
Yeah, in particular, rules, rules, rules.
Proofread your rules.
Test your rules.
The presentation and expression of your rules is just as important, if not more important, than the design of your game.
That's why we have you play games you haven't played before and read the rules you haven't seen before to see how bad the professionals get it done.
Try to be better than that.
The main things we're looking for-- coherence, clarity, organization, figures, and illustrations.
So, for assignment three, please try to put in illustrated-- at least, bare minimum, illustrate what it looks like when your game is set up.
Illustrate what it looks like when a major state change happens in the game.
Give us a basic section that just says here's kind of goal, how you play the game.
Here's how you actually play the game.
Here's some examples for how various things within the game happen.
There is general guidelines on the [INAUDIBLE] materials [INAUDIBLE] readings we've given you.
Really pay attention to that for assignment three.
Basically, in each assignment we get a little bit more tougher when we're grading the rules section
Because you've have two rounds of feedback already on rules.
And hopefully, the third time around, you know what-- you get into the idea of what we're looking for.
Also, use illustrations in your rules to get your point across.
You know, that's something that we typically find under-used where you spend a lot of words trying to explain some things which [INAUDIBLE] is far easier to understand where [INAUDIBLE] a simple, [?
minor ?]
drawing.
Even just like a cell phone camera, like a sketch, would probably have been more informative than paragraphs and paragraphs
Actually, I have a [INAUDIBLE] working for me on a board game.
And she did something amazing what I've actually never thought about doing for rules.
She made it basically in PowerPoint.
It's really, really long, a lot of pages.
So I'm really not sure how well it's going to work that way when we're going to be testing with it.
But it's very illustrative and very diagrammy-- lots of big words, very little written out.
Very little written out, like , long paragraphs of language.
I'm going to get the latest version of that and share it with the class so you can kind of see what that looks like, if that's useful for your game or not
I personally have had a lot of luck with Google Docs built-in drawing tools.
So you, for the word processing doc, just bring up the drawing [?
in it.
?]
And just, like, drawing things like cards that overlap each other with text in them-- like, Google Docs has all the tools you need for that.
Drawing arrows that arc in curves with one arrowhead and two arrowheads and everything that you need there is already [INAUDIBLE].
So if you don't have a drawing program, that's a lot built into Microsoft Word and in Google Docs that you can take advantage of.
They don't have to look great, but they could often get across the point a lot easier than text.
So, do try to use those
The next playtest is March 7.
Is that right?
I believe that's right.
I don't--
May 7.
[INAUDIBLE]
May
[INAUDIBLE]
So, I highly, highly recommend bringing a draft of your rules in and having that as part of the playtest.
You'll get feedback from [?
Phil and myself.
?]
You'll also get feedback from our guest lecturers
Every time somebody asks you for rule verification in class or when you're playtesting with your dorm mate or whatever, that's a good opportunity for you to think of how you could reword your rules or reformat your rules.
That is clearly a situation where this could be a question that's going to come up again when we play the game.
Or it could be a situation like they just didn't understand the rules as written right now.
So you've got to rephrase it somehow.
Use the bullet points, examples, and make sure they're visually distinct when someone is [INAUDIBLE]
Game designer I follow on Twitter is a very, very well spoken [INAUDIBLE] just posted.
There's a difference between a broken rule and a broken rules presentation.
The problem could be in either [INAUDIBLE] both those places.
Without testing, you're just not going to find it
If the only way that people are playing your game is that you are explaining it to them, then you're not getting the feedback that you need on your rules.
So, make sure that for assignment three, your rules have been tested too.
The rulesheet has been tested as well as [INAUDIBLE].
Also, one tip for people who are doing personal write ups-- people are getting better at sort of identifying things that went well, things that didn't go so well in the project.
And that's good, in general if you did a decent write up, you probably got a extra half grade on top of your grade.
So if you got like a B minus-- if you got a B plus, your write up alone could push you to an A minus, for instance.
But what some people don't [?
necessarily ?]
do is just say next time, I'm going to do this, or next time, I'm going to be sure not to do this.
Just like, little simple little things like, what are you going to take all of this for future projects?
So right now, [?
we are seeing ?]
a good analysis of your previous project, but usually write ups will also [INAUDIBLE] what the take-away is.
This is what I've wanted to make sure I'm going to try to do.
Or maybe you're not sure.
But this is something that you're going to try to do.
The next time you make a game, this is what we're going to try and do.
That can help a lot
First time is free.
If it's the last game you'll ever make, then what are you going to [INAUDIBLE] for a project?
What are you going to do on another team project, another design project?
Think about the future
It doesn't have to be a design thing.
It can be, maybe next time, if I work on a team on a creative project, I am going to make sure that I communicate with them this way or make sure that you understand blah.
Just something like that will be-- some sort of lesson that you're taking out from the previous assignment [INAUDIBLE].
Any questions about the assignment, assignment two?
Grades?
By the way, did you get an email from something from [?
Stella ?]
that tells you grades have been posted?
No
Because I think with other classes, [?
they post ?]
a more grades like p-sets and other stuff.
So it would be kind of annoying.
Like, four classes posting a p-set every week, or once a week
That makes sense.
OK, well, OK, then.
It's a good thing that we haven't [INAUDIBLE].
Today's reading is mostly about [INAUDIBLE].
It's actually a bit of a post-mortem.
It's written in the sort of same style as the flavor text of [INAUDIBLE].
And I thought it might help to actually open up the box, give you an idea of what it looks inside.
It might [INAUDIBLE] write up for-- actually, [INAUDIBLE]
I've seen this.
I've watched people play this.
This is the latest edition.
You might have seen the [INAUDIBLE] edition
And [INAUDIBLE] before that.
[INAUDIBLE] bigger than [INAUDIBLE]
There's the special power that [INAUDIBLE]
Ah.
[INAUDIBLE] So you get a bunch of planets, different colors, and I believe the colors correspond to different players.
[INAUDIBLE] Got your tokens, some of which are used, some of which are not used depending on the assortment of character that you've got in your game.
[INAUDIBLE] particular [INAUDIBLE].
Or you may not use that either.
This is the counter.
You can see you can put [INAUDIBLE] a race.
[INAUDIBLE]
[INAUDIBLE]
Yeah, so you can track them.
And [INAUDIBLE] how each player keeps track of [INAUDIBLE].
So, here's the yellow player.
Here's the green player.
And say both of them are at three.
So [INAUDIBLE] that.
[INAUDIBLE] not able to [INAUDIBLE] but it's just a way to keep track of [INAUDIBLE] for any one [INAUDIBLE].
And a whole tight pile of cards of two different sizes.
It's a really bad [INAUDIBLE] Some of these cards, I'm not entirely sure what they do.
But the cards that I do know what they do are the cards [INAUDIBLE], [?
Oh my god, ?]
[?
it's a mess.
?]
[INAUDIBLE] separate [INAUDIBLE] cards.
[INAUDIBLE] cards which have these numbers.
And what are you basically doing is you're using these cards [INAUDIBLE] identify who is going to initiate [INAUDIBLE].
Here we go.
Yeah, so you add-- so these numbers, depending on how many cards that are played, basically give your ships some sort of attack powers.
But then you also have individual ships that [INAUDIBLE] move on to attack [INAUDIBLE] attacking a red planet.
And what I'll do is I'll point this arrow toward the red planet.
Anybody defending the red planet puts chips on the red planet.
Anybody who's attacking it puts it on this [INAUDIBLE] very nice point [INAUDIBLE] are attacking.
That whole game basically largely comes down to you trying to encourage other people to contribute either to your defense [?
or the ?]
attack.
Please give me some chips.
Please give [INAUDIBLE] and [INAUDIBLE] negotiate [INAUDIBLE] mutual benefit [INAUDIBLE].
But this is the really interesting part of the game, which is when you start a game, you get one of these cards.
And this tells you what race you are.
And each race basically has a power that lets them break the rules somewhere in the rules.
Only you have this ability to break this power.
So for instance, this is the [?
hate ?]
race.
And they have the power of rage.
So at the start of a turn, you use this power to force every player to either discard a card [?
on the ?]
ship [INAUDIBLE] discard [INAUDIBLE].
Everybody else must then discard at the same time.
So if I put [?
down a ?]
[?
card, ?]
[INAUDIBLE] an attack card, everybody has to discard an attack card.
All these [INAUDIBLE] ships [INAUDIBLE] they make people angry with this power.
The filch is where you steal cards from other people, the power to hack.
You can compensation from other players, basically.
And so you've got this kind of like weird almost like CCG, Collectible Card Game, kind of a [INAUDIBLE] where depending on what you start with [?
is one way to ?]
determine the shape of the rest of the game board.
And every time you play the game, you can have a different set of cards because each player gets a different card between-- it's very, very likely that you have to play two games with even the same [?
set of basis ?]
at the table.
[INAUDIBLE] And the whole idea of the games we picked out today is that all of them are kind of like a sort of an interesting case of what it means to have changing rules in games.
Risk Legacy-- how many of you played this particular?
OK, how long have you had a game going [INAUDIBLE] how many rounds?
I played with people in my home.
I played in like three games and I think we played a total of 12 or so-- 12 games
Every time you play this game, the game changes
[INAUDIBLE]
What?
I've heard of this
Yeah, [INAUDIBLE] changes made [INAUDIBLE].
When you first buy the game, four packets, four envelopes are actually [INAUDIBLE] the past [?
year, ?]
You can see we played this a bit [INAUDIBLE].
And that gives you new cards, stickers that you can stick on the old cards.
You can see this card had a whole bunch of stickers added onto it that basically introduce new rules into games.
You can change the map.
And usually, this comes down to the people who won, the people who lost get certain abilities to make changes [INAUDIBLE] areas [INAUDIBLE] these [INAUDIBLE].
Certain cities have been established.
So for instance, this is the kingdom of [INAUDIBLE].
This is the kingdom of [INAUDIBLE]
That's mine, by the way
That yours?
Yeah

That's mine
[INAUDIBLE] cover here.
[INAUDIBLE] Here [INAUDIBLE]
Yeah
[INAUDIBLE] in Japan.
So this was interesting because one way to think about it is it's not the [INAUDIBLE] played the game of Risk?
Yeah, it starts off like Risk but closer to like Risk 20-something or other
Yeah
One of the later editions of risk where there's like turn order cards and some other-- it does resources differently than classic Risk did.
But basically, it is Risk
Yes, and you have stickers that you can attach on the [INAUDIBLE].
And so basically, every time you play this game, you're playing a different game [INAUDIBLE]
The great thing is, in this one, there's a story that goes along.
As you do things, the story changes.
So you're in this post-apocalyptic hellscape.
With these factions that are fighting each other.
And then all of a sudden, when a certain condition is reached, one of those classes opens up and new elements to the story are discussed.
And then a new race-- a new faction comes up or a new race comes up.
So, spoilers-- there is aliens right there from alien island
[INAUDIBLE]
[INAUDIBLE] becomes the alien collaborators.
And that faction from now on is considered the collaborator.
It gets special powers because of the aliens but they don't get-- I think they get special powers and I think they get a special something that goes against them
This game is actually probably not best played cold.
But so what I will suggest is that folks take a look at this box.
Because just trying to internalize all of the changes that have happened to this game since [INAUDIBLE] probably [INAUDIBLE].
But it's very, very [?
unique way to ?]
take a look and see what changes have been made.
One way that a designer is going to [INAUDIBLE].
A lot of the rules that you get, a lot of changes you can make later in the game are actually tools for you to fix [?
the events.
?]
So there are really, really powerful changes that you can make in the rules.
But the whole idea is that after you played something like 15 rounds or 10 rounds of the game, you kind of know where the game is going, where the board is [INAUDIBLE] or maybe [INAUDIBLE] how [INAUDIBLE].
And the players now have the tools to officially fix those imbalances to play.
So in a way, it's [INAUDIBLE] game design process.
And [INAUDIBLE] you have [INAUDIBLE] is kind of like a deck building game only your dice and [INAUDIBLE] part of [INAUDIBLE].
And I don't have a whole lot of information about this game
Plays like Dominion
Yes
[INAUDIBLE] the cards are out.
[?
We ?]
build your deck just like it's Dominion.
And when you do attacks and certain things, you have cards that represent a thing that you can use and then dice that are reflecting the [INAUDIBLE] card
So, like a lot of other deck building games, you know, [INAUDIBLE] conquer.
Often, when you're actually going into a game, [?
what you're seeing ?]
is not quite what [INAUDIBLE] before in previous games.
So every time, you're trying to, like, figure out what is [INAUDIBLE] momentum.
What is the strategy for this particular round that I'm playing?
[INAUDIBLE] just happens to be an interesting twist on that [INAUDIBLE] cards.
And Settlers of Catan, it seems like an odd thing to bring up.
But back when it first came out, it was kind of neat that it was one of the games that definitely popularized the whole idea that every time you played this game, you had [INAUDIBLE] because you have the hexes that are arranged [?
by the water ?]
and systems to be able to make a random map, [?
into a ?]
table no matter how it's arranged.
And you're playing a different scenario in the game.
So we also had the Seafarers expansion.
That's kind of like the traditional way that publishers [INAUDIBLE], right?
They give you all the parts and maybe a new set of rules to be able to add to an existing set of pieces on the board.
What happens is that the map is a lot larger, as you can see, because you have additional cards.
Actually, I don't know anything [INAUDIBLE].
But [INAUDIBLE] larger.
Users [INAUDIBLE] then the old ones.
Gives you a few new pieces, new rules, and you're playing [INAUDIBLE].
So this one might also be-- and I [?
wouldn't ?]
suggest just playing a straight out game of Settlers of Catan because I know a lot of you already have.
But in case you need it, [INAUDIBLE] especially if you haven't played the Seafarers expansion [INAUDIBLE] look through how they present the rules so that knowing how to play this game, how do they introduce new sets of rules to change the way [INAUDIBLE] played
Big difference for that one with islands, rather than having rows of settlements, you have shipping lanes that you build that connect ports
Yep, and I believe you don't know what's on the island before you get there
Yep
So you don't actually know [INAUDIBLE] island at the beginning of the game.
From the reading that-- anything strike anyone from the reading?
Just wondering if--
It said that I guess the unbalance of the game was the actual point
Yeah
And that they were going for the feel of that if you find that something is not fair, that you have your own unfair power to sort of counter that unfairness
Right, everyone has a win cheat basically, but it's officially mandated by the game and the different ways you [INAUDIBLE]
It's kind of similar to Diplomacy, which is inherently unbalanced.
Like, [INAUDIBLE] played several games of Diplomacy, [INAUDIBLE] like some of the-- I don't remember which countries, but some of them are, like, in much stronger positions and two of them together can take out the entire board if people let them.
And so in general, more advanced and experienced players will not let that happen or will all gang up and crush the first one or something like that, which kind of makes an interesting meta game
So you can spot the patterns.
But then you also have a lot of tools to try to push back against that once you know that the game [INAUDIBLE]
I consider Cosmic Encounter to be sort of like the version of Munchkin in that they're both sort of stop the leader games where it's literally everyone-- it's very easy for almost everyone to get [INAUDIBLE] to victory and it's all about, like, how you [INAUDIBLE] people are [INAUDIBLE] everyone's trying to [?
stop her.
?]
And when [INAUDIBLE] people run out of stuff to stop the winner
Yep, every game comes to an end.
I remember games of Cosmic Encounter dragging on for a really long amount of time, but I'm not entirely sure that's the case of this particular edition of the game
The idea of finding balance for [INAUDIBLE] where sort of the motto is if it's broken, break it until it's fixed
I haven't heard that one
[INAUDIBLE] basically, they took Super Smash Bros Brawl and just way overpowered a bunch of different people's moves so that you can do ridiculous things the entire time.
It actually ended up working pretty well [INAUDIBLE]
OK, yeah.
League of Legends, the ultra-rapid fire is kind of that too.
But I think that's-- one of the things that I remember from the readings was if you have a game where it's fun to lose, you've got something really, really good there.
They were a lot more flippant about some of the other ways that you can tell you have a good game.
Like, you lose the game when you have a divorce because your spouse wanted the game.
But I'm not sure how that metric is useful.
But some other metrics that tell whether your game is-- you've got something really interesting as a game is that one is the game system is actually continually surprising the people who actually designed the game.
Then just every once in a while, you just see something you've never seen before and you're the ones who made it.
You know you've got a really interesting system there.
And if people have stories to tell about their game after they're done, I think Munchkin has that.
Cosmic Encounter definitely has that.
They do try to give you the tone through the writing.
Both games, Munchkin and Cosmic Encounter, these are very [INAUDIBLE] games where that's a lot of verbal humor in the text.
But I feel in the dynamic of Cosmic Encounter, much like [INAUDIBLE] earlier, that's also [?
create an origin ?]
story.
I was like, this person betrayed that person.
I guess Diplomacy does that too.
But you don't want to tell stories about Diplomacy because there's so many bad things associated with that whereas what happens in Cosmic Encounter is kind of hilarious usually.
So--
[INAUDIBLE]
Yeah, and Munchkin does that too.
So you can tell stories about that and say, oh, I remember the time when, you know, I was this close to winning and then all of you [?
stopped me.
?]
And then I get this power and then I managed to pull myself out.
And then this person just came from nowhere and won the whole game
I used to be competitive about Munchkin and I played like two or three games.
And I was like, this game's a joke.
Like--
It's one long running joke, yeah
So there's a role or a class, something like [?
the Beef, ?]
where you can roll a die-- actually, even better is the one where you just take people's shit all day.
Like, I had a friend who like literally, every time, he was just like rolled-- like, keep going downhill until he was level one and just keep taking people's shit and it's just like, OK, man.
You clearly don't want to win.
All you're doing is trolling me right now
Right, right.
It's supposed to be a situation of what a role playing group actually does when they're getting together, not how a role playing game actually works.
OK, so I think [INAUDIBLE] with playing these games now, I'm going to guess about an hour.
Again, I would not suggest this with Risk Legacy for playing, but you are definitely welcome to take a look at it.
The rest of the games, go right ahead.
And probably, [INAUDIBLE] hopefully, the [INAUDIBLE] at least one hour, possibly 1 and 1/2 hours for you to work in your teams for the second half of class.
Cool?
Yeah
Are we [INAUDIBLE] then?
Yes, [INAUDIBLE] on Monday.
Sorry, did I say Wednesday?
I don't remember
OK, all right.
Yeah, Monday-- Monday is the day where we go outside and get ready to play Joust, which means wear comfortable shoes and--
It's [INAUDIBLE]
And hopefully it'll get warmer.
[INAUDIBLE] because outdoors is usually too noisy [INAUDIBLE].
All right, so [INAUDIBLE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We go into a reading today.
So this is an interesting book that came out from something called the New Games Movement, which I have mentioned a few times in class but just as a reminder.
I think it was mostly in the 60's.
How many of you kind of got that vibe while you were reading this?
But it's interesting because it's written in this almost Hemminway-ish kind of short little sentences-- short little declarative sentences about wellness, and happiness, and getting along with the community and stuff like that.
But all of the principles, at least in the chapter in today's reading, aren't actually all that different from what I've been teaching all semester long regarding game design and play testing, if it's your job to actually make a game.
What Bernie DeKoven and some of his other colleagues-- has anyone heard of the Whole Earth Catalog?
Brant, was it?
Brant, yeah.
What was it?
The Whole Earth Catalog
That was the thing with the parachute?
Well, actually, that's a New Game Movement in particular.
Yeah.
It's kind of the same community of people who are just trying to find ways to-- not only to find ways for people to get along with each other, but also things like [INAUDIBLE] where you're taking military equipment and turning them into things that you play with and stuff like that.
So today's chapter has a lot of stuff about how people who don't necessarily see themselves as game makers can use some of the practices that we've been talking about in this class to basically change the way they play games that they already play.
And in the process of that create new games.
The New Games Movement-- there's actually a New Games Book that is more about the kinds of games that you can play.
It includes things like parachute.
If you've ever done that game where there's two people back to back-- you're sitting back to back and then your job is to stand up, I think without using your hands.
Is that right?
Oh, you lock your arms.
Yeah.
Also that game where everyone gets into a tight circle, reaches in, grabs somebody else's hand and the whole job is to untangle it.
You see that in schools a lot nowadays, as well as games like parachute.
And all that came from this New Games Movement.
Again, it's a reaction to wars in Vietnam and, I guess, Korea.
Am I getting my chronology right?
Yeah, Korea then 'Nam
Yeah, and just trying to say, how do we learn how to play with other people instead of fighting other people?
But there are a couple of interesting ideas in here that I felt were kind of unique to today's reading that we haven't necessarily addressed in class.
Did anything leap out in Changing the Game?
Yeah?
He talked a lot about borrowing rules.
So taking something from another game and kind of putting in your own.
And making it your own and getting you over the hump of having to make up something completely new
Yeah, and it's kind of funny because the example that you talked about-- it's kind of like borrowing the rule on the fly, like from a different variant of the same game.
But I think the same principle can actually be applied to a much wider range of things like safety rules.
Like say you're playing a game like tag or something like that, and you haven't necessarily mapped out the boundaries of your game, but the game that you've played with your friends before decided that one of the boundaries was going to be the edge of that road, right?
Nobody can run past that road.
That's a fairly sensible safety rule.
It'd be easy enough to just borrow a rule from that and just say, just like that last game that we played, no one crosses that road for safety reasons.
Yeah?
Another one on this combat that's really, really confused as far as where it comes from, especially when your game isn't necessarily about the whole combat.
Like nothing like just not having to worry about how the combat actually works.
We borrowed a few different ones.
Like we tried Risk, we tried [INAUDIBLE].
It's like, we don't actually want to spend time on developing a fair combat system, just trying stuff out
Sure.
I mean, as game designers you can certainly sort of be inspired by what other people have done.
And of course, again, you're designing a game meant for somebody else to play, where it's what Bernie DeKoven is writing about.
It's about changing a rule that you're about to play or maybe even a possible play.
[INAUDIBLE]
I just found it interesting how [INAUDIBLE] rules that sort of help players play the game as it is right now.
And rules that actually change the strategy that you have to [INAUDIBLE]
Right.
So there are rules that clarify what it is that you can or you cannot do, right?
And then there are rules which he explicitly states as the definition of a change game.
It's one that requires you to come up with a new strategy.
So things like that safety rule that I described earlier doesn't necessarily fit that.
But it fulfills other purposes like being able to play well, giving you the feeling of wellness.
I can't believe that line, but it's great.
But then there's also rules like, we're going to change the game because it's not challenging given the way it's currently played.
Maybe the current set of rules has a simple exploit that just leads to one person winning all the time.
Maybe someone is just way more skilled at this game than somebody else or you're playing with a five-year-old, something that like.
And you want to have some challenge, you want to keep some challenge for the kid.
But if you just stick with the rules right now, the kid will be endlessly challenged and will not ever be able to win, and you're not being challenged at all.
That's where you give yourself-- and he brings this up, what's that the thing you introduce into the game to be able to rebalance that?
You should know it even if you didn't read it.
If you're playing with somebody who is way less skilled than you, you give yourself a handicap.
Yeah.
Yeah, so handicapping is a really interesting concept regarding game balance.
Because normally when we think about game balance, it's all about how do we think of the rules so that everybody is kind of equally skilled, everyone kind of has a fair chance of winning, a decent chance-- maybe not exactly an even chance of winning, but at least a fair chance of winning.
But he's introducing the concept of handicapping as a way to be able to reintroduce challenge in the situation where players are not equally skilled because of physical reasons, because of expertise or familiarity with the game, that sort of thing.
So let me see, is that it?
Anything else that people remember from the reading?
Again a lot of the stuff-- there is an interesting little bit that he talks about cheating as a thing that players should be trying to do if you are already in the mood to be able to then suggest rule changes to the rest of the community.
And you should be finding the loopholes, right.
You should be trying to figure out, given the current set of rules that you're playing right now, what are the problems with that set of rules and how do you improve on them?
And the way how you figure out whether these rules are important, or well written, or problematic, is you try to break them.
And then, if you break them, then you realize that, oh OK, that rule was really necessary in order for the game to play, but your attitude is, I want to continue playing with all of you.
Then you say, all right, we're not going to do that anymore.
I think it takes a tremendous amount of trust among players to be able to accommodate that.
I don't know how many people actually play regularly in environments like that
Are you saying that you should be trying to cheat to show the rules [INAUDIBLE]?
That you should be trying to find the exploits.
You should be trying to figure out exactly how ironclad this rule actually is, right.
Or whether this rule is a good idea in the first place
By breaking the rules?
By breaking them.
It's very, very strange.
It's a very, very interesting suggestion.
I can't recall of a situation when I've been in a play community, as he describes it, where there's enough trust to be able to let people get away with that
Honestly, I think it's just less interesting.
I think it's more interesting-- I forget who said this-- this is like a quote that's like, creatively actually comes from constraints.
I think it's a lot more interesting to actually try to find the broken strategy in a game as it is
OK, to find the degenerate strategies within the rules as supposed to figure out where the rules are broken
So some of them play the-- [INAUDIBLE] change the rules [INAUDIBLE].
Sometimes when playing games you're like, how do we-- but I feel like every time I've done those with other people, it's always been important to everyone that everyone knows what the rules are [INAUDIBLE]
And that's one thing that he very, very clearly states.
It's like, you are cheating publicly.
You are broadcasting the fact that you are cheating, which brings up the question, are you really cheating or are you just proposing a rule change, right?
But the idea of treating the rules as something that could be maleable, I think, is what he's really getting at.
He doesn't fight to use the word cheating, which I'm not entirely sure whether that's the right word to describe it
To me it seems sort of similar to the phase idea of [INAUDIBLE] what's underneath the game.
So if you start breaking a rule and you're doing it publicly with the knowledge of other people, just to see what happens if we start to break this rule, I think you can get into some interesting issues.
Like well, what makes the game tick?
I don't know, I feel like a lot of games I used play as a kid, we would explicitly break rules.
We would know of the rule of the game, but we would choose to either not play with it or to just limit it.
This resource is limited.
Like well, you know, this time when we play it it's not going to be.
Let's just see what happens
Right.
Yeah, and that's the attitude that he is trying to encourage
I feel like if you're like, for whatever purpose you're playing a game-- like if you're playing to win or if you're playing just for fun or something, players are too keen to change the rules when they don't do [INAUDIBLE]
Players aren't or players are?
Players are.
Like if you're just not very good at a game or you run into some sort of obstacle or strategy that [INAUDIBLE] done that.
I think players, in general, jump to changing the rules too frequently
Too frequently?
Yeah
OK, let's dig into that.
Why too frequently?
Why do you feel that happens too often?
I'm thinking along the lines of competitive games here.
You just get tired of how they have blame other people besides themselves

Just in general
But what if you were trying to change the rules for other people?
Yeah, no, I think that sort thing requires a really in depth understanding of all the strategy [INAUDIBLE]
Well, say I'm in a fighting game tournament or something like that where usually the rules are pretty rigidly enforced, right?
Yeah
But I'm not like a top-level player.
I just happen to be on a real winning streak among people that I know and I don't seem to be able to lose right now.
So I think what Bernie is talking about is describing a situation where, especially if you are that person who seems to be on a winning streak, you might want to suggest a rule change to be able to bring the challenge back to you, right.
I've seen players do this specifically in our kids where it's like, I'm going to pick the character that I'm crappiest at because I can't seem to be able to lose by playing the characters that I'm actually good at.
That's [INAUDIBLE] very interesting for me
I mean, [INAUDIBLE] rule changes to [INAUDIBLE] I'm calling it cheating
Yeah
Because to me cheating implies that you are not.
Me changing the rules and everyone agrees on the rules and how the rules change, and it's not.
Then you have changed the rules.
You are now playing by a different set of rules.
If you play by those, you're not cheating because you're not playing by the original.
I feel like cheating is anything where you are breaking the set of rules that was agreed upon
Well, cheating also, I think, has a certain assumption that everybody else is not breaking the rules, right.
No one is playing with a different set of rules but you.
All but the cheat.
In fact, a lot of cheats only work if everyone else sticks to the rules as written and then you're the only one cheating.
So again, I think he's trying to get across the attitude but used the wrong word for it by describing it as cheating
You brought up the example of fighting games and picking a character that you're not very good at.
That's not even a rule change, in my opinion
That's true.
Well, yeah, I guess.
I'm trying to think of the implicit-- maybe not a rule change, but it's more like a values change that you should be playing to win, right
You can make a rule that, OK, normally everyone can choose whatever character they want, but yours is always better than us so that when you play in our play group, I choose your character
Oh, yeah.
You get to choose my character or something like that.
You tell me who you want to play against and I'll try my best to play that game.
Let's start in with Miguel and then over to Laura.
Yeah?
So, one thing I wanted to bring up was so you think if one person isn't being challenged enough then they can start experimenting.
But in my experience it more often happens when the entire group is not being challenged anymore.
In the sense of, you two just played Catan three times over the last few days.
And she was kind of tired of Catan.
You're kind of ready to do something else, but you don't have a different board game [INAUDIBLE] or something
So then make a variant, sure
--different variants up.
And it's like everyone is trying to increase the challenge, not because it's not challenging.
There's still [INAUDIBLE] who is going win and you all sit down and start playing.
But it's not challenging in the sense that it's not engaging and not interesting
I think the reason why it's easier to do it when there's a lot of people finding a game uninteresting is because it's easier to get a consensus to be able to enforce a change of the rules, right.
It's a lot harder to say-- say I'm 12 and I'm playing with a whole bunch of other 12-year-olds, and there happens to be a 9-year-old in the group and we're playing soccer or something like that.
It's harder for anyone to convince the rest of the 12-year-olds to take it easy on a 9-year-old.
Like, all right, what if we give an extra player on the 9-year-old team or something like that.
Not to say it can't be done and I think that's what Bernie DeKoven is saying, no you should be able to do that sort of thing.
And it kind of makes the game more interesting for everybody, not just for the person who is having a hard time.
It makes the game more interesting even for the people who have an advantage because now it makes it a big-- more of a fight, I guess, for them to be able to win.
Laura?
Well, I think the other issue I have with the word cheating in describing changing the rules-- and less so against for individual players-- I know, for example, there are games that I've played since I was really little where it wasn't really intentional that we were not playing by the original rules.
It's just that when I was seven and I learned how to play Monopoly or Clue or some really generic board game that I learned when I was really little.
I did not sit through all the rules or I thought I knew the rules and didn't really remember them.
And so you had a change in the rules unintentionally.
And I think that's a very different category
Yeah, house rules
Your house rules become so much different.
Even if you play with other people and you're kind of all playing by your own house rules, it doesn't really feel like-- I wouldn't always call that cheating.
It's not because you're not trying to play by the correct-- what you think are the correct rules
And more importantly, everybody in the room's agreed to that set of-- this new set of rules, right.
So cheating is again not a great word to be able to describe that
I feel like when you're modifying rules there's a few different ways to do it.
Sometimes you just add a little to a game.
You're just like, it's not interesting enough, make up some initial variants to make it sort more [INAUDIBLE].
And other times if you're doing it it's [INAUDIBLE] Catan with friends there, we [INAUDIBLE] because it's really overpowered if you just-- it's really overpowered strategy [INAUDIBLE].
Some groups a lot more [INAUDIBLE] overpowered strategy [INAUDIBLE] after playing twice we've [INAUDIBLE] one beater in there [INAUDIBLE] because he's ridiculously over powered.
And so [INAUDIBLE] different levels of [INAUDIBLE] They see things and there's experimenting [INAUDIBLE]
I think in the end, Bernie DeKoven's kind of coming from, you should be free to challenge authority.
That's kind of like the implicit thing that you get from the whole catalog as well
Going back to what Nathan said, people are sometimes too keen.
For example, something might seem initially overpowered but then you play it, you might be after the first game, everybody's just like, oh yeah, it's broken, might as well just change it, right.
Whereas, if you'd played another game, you might have realized, oh wait, you can actually counter this by just doing
So actually a really interesting counterpoint, which is not part of our reading is Dave Sirlin who is a MIT alum, actually, who wrote a book called Playing to Win.
This is not part of our class reading but he will definitely back that up, right.
His is like, no it's everyone's responsibility to play as hard as they can because you have an-- especially for certain kinds of games-- you haven't discovered everything that your game systems capable of.
And if somebody can beat you playing that same game with the same set of rules, then clearly it's possible to beat that player or at least get up to the point where they are equally skilled.
So now there's a whole bunch of game design philosophy to come out of that as well.
And I believe we played-- did we play Yomi in this class before?
We used to
We used to?
Yeah, OK.
There's actually an iPad version of it now.
It's definitely not an iPhone version because it's way too much text to fit on an iPhone screen.
But Yomi is a game that-- it's a card game version of basically a Street Fighter type fighting game.
And it's all about anticipating what the other player is going to do and if you manage to get it right, you can build up your combos and everything.
But the whole idea of everything that Davie Sirlin's worked on, I believe he did a remix version of a Street Fighter, which was [INAUDIBLE].
Yeah, the HD remix versions.
The whole idea of everything that he wrote about it and everything that he makes is that these kinds of competitive games really should be more about, how do you challenge yourself to improve better?
And it's a lot less about the community around you.
It's a lot more about you and the game system.
So you are submitting yourself willingly to the authority of the game system, which of course, game designers create.
And then if there's anything that you should be challenging, you should be challenging your ability to be able to rise to the occasion, right
Everyone should be doing [INAUDIBLE], right.
If that person's arguing that, they're not giving you a good experience.
They're letting you down
Right.
They're not fighting as hard as they can.
And even Bernie describes this, right.
The whole idea is what happens if you give somebody a chance to-- it's like, I could have played this game optimally because I know this game so well and have beaten you into the ground.
But I'm just going to-- instead of handicapping myself, which is a sort of public declaration that this is how I'm going to be doing it, I'm just not going to play optimally.
Then I think both Dave Sirlin and Bernie DeKoven are actually in agreement.
This is not a good way to play.
Because the person who is getting the benefit of that feels cheated of experience as well.
Did I see?
I think I need to clarify that oftentimes you can use [INAUDIBLE].
So in a game with really defined strategies, you could sort of be going for some strategy that you believe to be sort of optimal because you wanted [INAUDIBLE] other players that you were like-- and it could be, be and you sort of are [INAUDIBLE] along the lines of, oh you [INAUDIBLE] possibilities of the game in there.
So you can just try hard and you can do that.
I think you're actually working toward that purpose if you do something like-- you're not going to try a strategy that's totally bad, but who knows
You know, I do that a lot and I always feel compelled to publicly declare that this is what I'm doing.
I think because I don't want anyone to think that I'm stupid.
I really do something bad [INAUDIBLE]
A lot of times when I play a board game, I play with people were really, really good at understanding and making board games, too.
If I think that something might be an interesting way to play that game, I'll say it out and then it become more of a cooperative, how could this strategy be--
Let's play this out.
Let's see what happens, right.
A learning experience for everyone
Well, what would you have done from my shoes?
It's like at this point we're not even playing to win, we're playing to develop a strategy
Yeah
Yeah, I was thinking [INAUDIBLE] you're not necessarily playing to maximize your fun right now.
You're playing to explore [INAUDIBLE] the game has.
And maybe find a bigger strategical future later on, I guess
And I think that's in keeping with what Bernie has tried to write about.
He ends up the chapter by talking about how you are scoring, right.
How you're deciding what a win is.
And in this particular case, it's a situation that we're describing, a win is learning how a new, untested strategy is going to play out.
It becomes less about who is the individual who's actually winning and more about, are we getting something new from our understanding of this game?
Which can be really entertaining.
Whereas I think Dave Sirlin, not quite so willing to let that go.
He's a little bit more adamant that, no you should be winning based on the definition of the rule set, not based on something like extra community arrived position.
And Bernie DeKoven is trying to fight for, no community determines when the conditions are fine.
In fact, there are many good reasons for you to go for that.
Different points of view.
I'm not saying that anyone is [INAUDIBLE] about that
I mean, I think if you want to maximize-- like in the long term-- your chances of winning has to be [INAUDIBLE]
Oh yes, absolutely.
Yeah, I will agree.
If you're a professional player who is in a team, you are probably going to be working with your teammates a lot of times-- not to determine who is better at the game, but to try to make everybody else play better, right?
And there's just two different perspectives on how you arrive at that.
One is to always play as hard as you can so that everyone has to sort of rise to the occasion.
And the other way is just-- no, it's OK to do things like experiment.
It's OK to do things like handicaps.
It's OK to do things like-- I'm just going to play-- I was just playing a game recently where like I'm going to do a sub-optimal strategy on purpose, just to see whether my opponent's capable of fighting it off.
Because it's just ridiculous.
So yeah-- I had fun even though I didn't win.
So today I was thinking of going through three games.
But what time is it now?
3:00
It's almost 3:00?
OK, so I think we're going to do two games.
I'm going to very quickly have a discussion about what are all the different versions of Mafia that people have played.
How many of you have played a version of Mafia or Werewolf or something like that?
No?
Everybody?
OK. All right, let me quickly give a description.
The general framework of these games everybody gets a card, and the card gives you a role.
These are usually just regular playing cards where-- one version that I have played is that if you have a king or queen or a jack of clubs, you are mafia.
And it's your job to kill everybody else who didn't receive a king, queen, or jack of clubs.
And then everybody else is a villager whose job is to figure out who are the mafia players.
So everyone closes their eyes and open their eyes based on the cues of a person who is like a moderator.
If I ran the game here in class, I would be the moderator.
And I'd say like, nighttime, everyone closes their eyes.
Mafia awakes.
And then the mafia do not say anything, but they can communicate with things like gestures and moving their head.
And they're basically sort of using body language to vote on who dies that night-- usually by nodding, or by pointing, or something like that.
And then everyone closes their eyes.
The moderator declares who dies.
Everyone opens their eyes again, and that person reveals their card.
And that shows whether they were mafia or not.
And the whole idea is that in between rounds everyone speaks freely.
And so there are accusations that are flying around, and suspicions, and exchanging of very, very tangential evidence-- like, I thought I heard rustling over there.
You know, and it's like-- you know, that could be the mafia person just trying to throw you off the track.
That could be somebody who is actually the villager.
Only the mafia know for sure who is who.
And then there are other hidden roles.
So what are some other variants that you've played?
Yeah?
[INAUDIBLE]
So I've played where the card doesn't get revealed when you die
Oh, yeah?
And one particular other thing that-- like all sorts of different roles, as well.
And the voting mechanic for who gets killed-- so I've seen it played where people decide when the discussion is over.
And then they little vote.
And then the person who has the majority gets killed
They vote by printing?
Yeah
Yeah
But the way that I always play is that all it takes for someone on the block is to have someone say, I'm putting that person on the block, and another persons says "seconded".
And at that point, until a vote happens, you can't put anyone else on the block
Oh, interesting
So either everyone will vote and will say, OK, this person-- and a majority will result in a kill, at which point it's night instantly-- and that's that day.
Or they fail to kill that person, and then people can feel free to like put up more people-- like to just point out another person which they like-- no, I'm like [INAUDIBLE]
You had your hand up
The two most basic roles, in addition to like mafia and villager-- that I've seen in pretty much every game-- is the medic and the detective.
The medic, every night, gets to point at someone, and if the mafia were trying to hit that person, that person stays alive.
The detective, every night, gets to basically point at someone while everyone else is sleeping.
And the moderator will either nod his head saying, yes, he's mafia or no he's not
So I have sort of a variant of Mafia that's at MIT, mostly [INAUDIBLE] called Live Action Mafia.
And the way this works is one day is one day-- so like often there's [INAUDIBLE] where like mafia lasts one [INAUDIBLE].
But in this version, the mafia can kill someone once per day by tapping them on the shoulder and saying bang.
And essentially, because you obviously-- like in this game, you couldn't do this.
If you're mafia, you couldn't kill someone in front of other Mafia players.
Because then they will see you kill that person, and will tell everyone that you killed that person.
And then you'll be lynched for it.
It's an [INAUDIBLE] game of-- often times you try to figure it as like-- people like [INAUDIBLE] so they get friends to lie for them about where they were in that location there.
In one of the games recently there, someone said, oh, I was working on my lab until like 4:30 or something.
And then someone found out who the partner was in that lab, emailed the partner, and the partner said they got out at 4:00.
And this kill was at 4:15.
That person was lynched.
And so there are a number of special rules and powers, but like [INAUDIBLE] the game, where the focus is on actually like figuring where people were like [INAUDIBLE]
One thing that I think-- one reason why that works well on the MIT campus is because it's a fairly small campus compared to other university campuses, even in Massachusetts.
And even when you're off campus, you are often in close proximity to a lot of other MIT people.
Do people know who else is playing in this game-- in a round of Live Action Mafia?
Do you know who [INAUDIBLE]?
Occasionally there are some-- there are like [INAUDIBLE].
Like most of it's random.
There's people in campus.
There's always people playing at-- there's a fraternity across the river in Brookline.
They're [INAUDIBLE].
And so it's when a kill happens there, and so immediately everyone's trying to figure out who was on the MIT campus.
[INAUDIBLE] people are trying figure out who couldn't have gotten back there in time
Right.
That's nice because it helps you do the detective work
I used to play, but I'm not friends with like a lot of people who are in the game.
I just had a couple of friends going who always invited me to play.
And so I usually get lynched pretty quickly, because because they'd just be like, we don't know this kid.
So long
My biggest problem with Mafia is like even just the like the table [INAUDIBLE] like there's like no--
[INAUDIBLE] Mafia, if you don't know--
Yeah
And so like what will end up happening based on the first round it like, John, you look suspicious.
I think John's mafia.
Everyone vote yes.
And then everyone votes yes, and he dies.
And he's sitting there like, I'm sorry, what?
Yeah.
That's--
That's kind of how the game [INAUDIBLE]
Sorry?
I had more
Oh, sorry
I was going to say that a couple of interesting variants I've played with are like where you have things like the fool, where his job is to try and get killed as Mafia, or an innocent, where if the town accidentally kills them, then they just like automatically lose, and things like that
I think the most confusing variant is the [INAUDIBLE] where they have the mafia, the medic, the detective-- then they also have, I think, the serial killer, who is supposed to be the last person standing, and then the arsonist-- I can't remember his goal.
Then there was a transporter who could switch targets.
You know, like if the mafia picks one person, and the transporter hasn't been picked, that other person can switch the mafia's turn.
And then there was a seer who can talk to dead people
Who specifies the roles?
Is it just like a stable set of rules on an online server?
Or is it like every round has this set of check boxes?
I think that how it works is that you choose what type of game that you want
Oh, I see.
So you're setting up a little lobby for this-- OK.
I see
One of the funny ones that I played-- like there's one that was called Epic Mafia.
It's actually like imbalance, but it's still fun.
It's a crazy cop, who always-- so there's basically three cops and a mafia.
And the cops don't know which cop they are.
There's a crazy cop who always gets everything inverted.
There's a regular cop.
And then there's a cop who like always sees everything as truthful, or always sees everything innocent or everything as not-- I don't remember which.
It's actually really unfair for the mafia [INAUDIBLE] where you just like say, this is what I got.
And then someone dies, and then you say this is what I got.
And then, basically, you can just piece together who is lying.
But it's interesting
[INAUDIBLE]
Yeah, I was going to say.
[INAUDIBLE] the game before that.
There's a game where someone will swap God with one of the players.
And decided to allow this to proceed
Wait, hold on
Did God [INAUDIBLE]?
[INAUDIBLE]

So this person role-swaps God with one of the players
Oh,
And so the God's role became a villager.
The moderator became-- it was like the moderator's role was now a villager, and like one of the random villagers-- they were all just now God

[INAUDIBLE]
They're like-- Yes.
And they're like this person who has now the new God made a bet with the player who was the head of the mafia that they could live until the end of game.
And after it's money, like 10 different people won this thing
So the different aggregates of roles happen more or less-- not by accident, but it's not like someone-- it's not like there are standards.
It's like you get together with the group that's playing, and then you decide what goals there are.
Or someone is like, oh, I thought about this beforehand, and I have some set up I think would be really cool.
So my brother once put together a set up like that, which had I believe 17 people-- three of which vanilla villagers with-- I'm not sure whether it was four or five different sides
Factions
So there's like four or five different factions.
So they were just playing like-- there's like mafia, and then there's like werewolves, which are identical to the mafia, and also get a kill at night.
And then there's the serial killer, who's bulletproof, so he can't get killed at night.
And he was also a separate faction trying to kill everyone else.
And then there's the vigilante, who's like a good guy who also gets a kill at night.
And so this game, it was actually, surprisingly, very good.
Because everyone basically had a role, so no one was unhappy about that.
And everyone more or less tried-- and the game is quick, because even though it's 17 people, you've got like four or five people dying every round for the first couple of rounds
I'm trying to figure out why there are so many variants on Mafia.
There's something about Mafia or Werewolf-- or any of the names that people use-- that seems to lend itself to people coming up with new variants and then adopting it.
I mean, clearly it makes sense for-- you need to be able to have a large group of people agree on the roles before it starts.
So I'm not so confused about that.
I'm wondering why Mafia--
I think, one, it's really fast to play through, which makes it very easy to playtest.
And then, two, it's also just really, really easy to extend.
You could always make your own role, but you honestly don't need to.
There's at least a hundred different roles.
You could just like eenie-meenie-miney-mo and like aggregate into a game
All right
I was going to say that it's sort of boring for the villagers that they're just getting kicked out one by one
So there's a flaw in the game that every one of these variants is trying to address, which is making things a little bit more interesting for the poor villagers.
And that's, I think-- the fact that it's very clear that all the rules are just imposed by the people in the room-- definitely that's a Mafia tournament as far as I know.
I guess we could have Mafia tournaments spring up in the middle of conventions and stuff like that.
But they don't seem to last very long.
It's hard to imagine a version of Mafia where you are in direct contact with the person who established the rules for the round that you're playing right now.
So it's very easy to just think that there's no canonical set of rules
I was just reminded of some things.
There was one time where there was a game of Mafia where God failed to give out any Mafia cards
Oh
Like, he cheated, as far as that goes.
And so god chose someone to kill every night, and pretended that there was a mafia doing.
And so the group started as probably around 12 people.
And at around five people, they were like, OK, this is impossible-- as in, the previous [INAUDIBLE] they were like, OK, if this happens then blah.
And if this happens, then blah.
And then they killed someone, and that person was innocent, even though they had to be guilty in order for it to make sense.
But they were like, what's going on?
I don't know.
I think that's the sort of cheating that we'll need to go to actually describing.
Maybe we should try that once in a while
You know, they were like pretty mad at the person
I can imagine.
I can imagine.
Yeah, yeah.
Maybe that's the counter argument
You are never god again
So I don't actually know what the game is called, but we call it spies.
It's basically there's a certain number of spies, and a certain number of resistance players.
It's very similar to Mafia
Resistance
Oh, Resistance?
So this is a Mafia type game, yeah
It's called Spies.
It's like the opposite.
It's very similar to Mafia, except that you have to have a certain number of missions by putting people on missions.
And the spies try to fail them, and the resistance is trying to pass them.
And we did something very similar to that, where we had-- one of our friends left the room for a minute.
And we go, OK guys, next round we're all spies.
And we came up with like pecking order for who does what at what time.
And we all tried to like blame one kid for being the spy the entire time.
Then, at the end, we finally managed to get to the point where everyone had voted for the only person that was not the spy to be the spy.
We basically just messed the game up to try and like execute him and have fun with it
Right
But it kind of relates to that point of taking the rules and--
And Resistance is a little bit easier to identify that there is actually a set of rules.
Because Resistance is a boxed product with a rulebook in it-- unlike a lot of other games of Mafia, where you kind of think things are pretty malleable.
I do want to move on to something else.
But before I get to Lockup, a couple of questions.
If you have a smartphone, try downloading this.
I think it's free on both the Android and the app store, because we're going to play a couple of rounds of this-- if the Wi-Fi works in this room.
Which it may not
I really like Resistance, because there's actual public information which you need.
It's not just, John's guilty.
He's dead now.
Right?
You don't just start the game by making stuff up
Yeah
Now, anybody who doesn't have a smartphone, you're welcome to use mine.
Who would like to?
[INAUDIBLE]
Yeah, go ahead
But yes, I have a chart of Resistance games
Wait, am I the only one who has [INAUDIBLE]?
Yeah
You could play [INAUDIBLE]
You can [INAUDIBLE]
Although, actually, I'm going to get my iPad.
I'll [INAUDIBLE].
[SIDE CONVERSATION]
OK. Yeah, SpaceTeam is a game that I want to give a try.
And we'll see if it works.
If you are surrounded with people who have like the same kind of phone as yours-- like Android and iOS are two separate things-- you can attempt to do it over Google
There's some way you can play them--
I'm not on the Wi-Fi right now
You can play Android and iPhone together
Over Wi-Fi.
If Wi-Fi works in this room
I have Wi-Fi.
[SIDE CONVERSATION]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I'd like to introduce Jesper.
Jesper Juul is a old friend of the lab and currently at the Royal Danish Academy-- So it is very long
It's actually longer than this.
Royal Danish Academy of Fine Arts Schools of Architecture Conservation and Design, School of Design
Right.
And you've also taught at NYU.
You're a visiting professor here.
And you've been a games scholar for a decade now?
Yeah, a bit more than that
Maybe it's more than a decade.
If you take a bunch of the classes here, we sometimes have readings from Jesper-- articles that he's written.
But this is another example of the talks that we've been having on this is what people do once they understand game design, right?
Some people study games.
Some people do research and write books about it.
And ask interesting questions that try to find out more about what makes games fascinating.
Or what makes the environment in which we create games or we play games interesting.
So with that, I'd like hand it over to Jesper
Hi
And this is game design 608
Hello, game design 608.
So are you doing digital games, or analog games, or?
We're doing analog
Analog, all right.
So I think I'll mostly be talking about digital games right now, but I think it probably applies to some extent.
Phil, introduced me so nicely.
So this is really a part of a conference presentation I did earlier in April.
And so if you're interested in reading the full paper, it's on my website.
And it's kind of URL or independent style.
And if you just go to /text you'll see some of the other stuff I've written.
And I've written a few books about video games.
So I wrote one about video games and storytelling.
And one about casual games.
And my most recent one, called The Art of Failure is, in a way, about being a sore loser.
So it's an essay on the pain of playing video games.
So I focus on the question of why we claim we enjoy video games even though if you look at people who claim that, they often look quite unhappy actually.
So it's a discussion on that.
But this one is a social study of different things across the culture of independent games.
And so, there seems to be a kind of consensus that independent games would become a important aspect of game culture.
And you can see this in several ways-- with a very cheap kind of Google engram way of showing that something is popular.
I don't know if anybody knows why it's never flat, this curve.
Does somebody know?
So in economic game theory, there's actually a concept called independent game.
So it means a game that's not attached to other games.
So that's why it's never quite fled the curve.
But it shows you that, from 2000 on, people started talking about independent games.
And you can also see that somebody at Microsoft, they could spend a good deal of energy to claim how independent game friendly they are with the new console.
[INAUDIBLE] And we can tell by-- probably a few different reasons why we talk about independent games.
And one of them may be the budgets have become too big and too late development.
So there is this kind of opening for games that are made on a smaller budget.
And also that it's become easier to distribute games.
So it used to be, in order to distribute a game, you had to find a disc and a box.
And that's no longer true, fortunately.
And this means it's become possible to make games in different styles.
And they also found that there's this thing that-- which can sound kind of weird-- the independent games, in a way, pre-supposed the idea of independent games.
So let me explain what that means.
If you're making a game on a small budget, one of the things you need to be able to be sure of is that people who see your game, don't just think of it as a game with too small a budget-- that they actually understand that this is a game that has some positive qualities to it.
Or that there's a particular reason why it has a small budget.
Or the fact that a bigger budget wouldn't have made this game any better.
And so I think that the idea of the independent game is really that idea that you can actually say I'm making an independent game.
And people will understand that this is a game that has a particular set of qualities where it's a feature-- the fact that it's a game with a small budget.
And so, it doesn't mean-- you can see the question, what makes people assume that something is an independent game when they see it?
I don't think it's just that it says independent game somewhere.
But I do think there's actually a particular style that's come into independent games.
[INAUDIBLE]
Essentially you see that when people write about independent games, the first thing everybody will say is that you can't define anything.
It's the first thing you have to say.
So you always have people say that you can't define it because this creates discord in the game community.
Or you want to talk about independent in different ways, economic, technological, or cultural status.
And then the third people say, well, there's no point in trying to define independent games in the first place.
So I think that's actually [INAUDIBLE] On the other hand, if you look at what people who make independent games say there's actually a lot of things in common.
So Dan Cook talked about that independent games largely favor someone who's authentic and deserving.
Edmund McMillen liked to talk about honesty and speaking from your heart.
He made Super Meat Boy, and Dan Cook made Triple Town.
And [?
Robert ?]
[?
Aumann ?]
talks about your personal relationship with the work-- to independent games.
[INAUDIBLE] [?
Chavez ?]
a whole discussion comparing independent games to funk music-- small, kind of personal, anyone can do it kind of thing.
And various [INAUDIBLE] has talked about the smaller budgets allowing games to be more personal, more relevant.
And so I think these are quite similar in some ways.
So they talk about the honest and the traditional and, what I call, minimal complexity.
You can understand who made the game.
Like we talked about just before with Kickstarter-- if you buy Triple-A game it's not necessarily clear who actually made the game.
And with independent games, there's much more of an idea that you will have a feeling of the author's-- of the creator's-- personality.
All right, so you see that these are what I call moral and aesthetic claims at the same time.
So it's not just saying that these are better games.
It's a bit more deep than that.
It's also saying that this is a better way of making a game.
So not just that it will be a better product, but that, even say, the quality of life of someone who makes an indie game will be better.
But also that we will all be better off if more people make independent games.
We'll have more communication, and more ideas being spread, and more diversity, and so on.
Now, the thing is this is not exactly new.
So this is called the trellis wallpaper by William Morris and Philip Webb from 1862.
So one of the things that strikes you as indie is that it actually is quite similar to somethings that have happened several times.
But I think particularly it happened with the arts and crafts movement of the 19th century.
So you can see when we talk about independent games I do think it ties into several things such as the idea of DIY or the maker movement or the idea of local food production, like the locavore movement-- this idea that if you go to a restaurant you should know where the chicken you ate actually came from.
Only yesterday I was at this restaurant called Emu-- I didn't know [INAUDIBLE] and they did list, on the left side of the menu, all of the farms where they get their ingredients.
And it has this kind of like, wow, isn't it amazing.
And of course I've never heard of any of those farms.
I have no idea where they are.
But it kind of still works for me because you can understand that there's a kind of honesty or something authentic about it.
But the arts and crafts movement, specifically discussed as being this late-19th century movement, where people reacted against the industrialization and machine production.
And they felt that this included a loss of quality or personality.
So that you didn't even know who made the product.
And the product itself would be, kind of, worse.
And then the proponents of John Ruskin-- he talked about this idea that in medieval times there was a much better-- you had the [INAUDIBLE] by the medieval guild, kind of a small group of people making something like a Gothic building.
And William Morris talks about this idea that handicrafts-- the craftsperson making something that's much better than what would be made by machine production.
And again, not just that.
It would be better products, but that society at large, the world, would be a better place if we made things that way.
And I do think you can say that-- I think it's very clear, with the idea of independent games.
It's kind of similar in the sense that when they talked about the revival of the craftsmanship and handicrafts, that we actually get a big machine, or big corporate productions-- a mass production.
And I think, very similar, you can say that independent games are a reaction against very large Triple-A production teams.
And so we have this idea that we should return to the smaller productions, and this will make everybody-- everything better on a number of levels.
Does that make sense?
I don't know, do you have strong feelings about independent games?
An independent game that you get [INAUDIBLE]
I do completely [INAUDIBLE] I think independent games now have more freedom to explore different mechanics and step away from what sells because they don't have the million dollar investments that they have to make sure it sells.
And they don't have to be like, oh, it's been proven to do well with the market.
And they have more freedom in that way.
[INAUDIBLE]
Yeah
So I found that, just for me personally, I don't really play games unless I'm planning on playing them a lot.
So I play competitively League of Legends or [INAUDIBLE] Brothers.
And some of the problems I have with independent games is that I don't feel like they're as developed.
They're generally on the more creative side.
Which is definitely something appreciable but not something that I enjoy.
I prefer getting very good at, physically, mechanics and such
Yeah, I guess, [INAUDIBLE] multi-player actually does not [INAUDIBLE] adaptable.
There's this new collection out called [INAUDIBLE] Does anybody play that?
[?
Bobby ?]
[?
Pinchot ?]
[INAUDIBLE]
[INAUDIBLE] [INTERPOSING VOICES]
Yeah, that's true it's not, in a way, a competitive sport.
It's not that at all.
There's a game I like, [INAUDIBLE].
Yeah.
So that's kind of part of that complex
So kind of on the opposite note, I think, not all, but a lot of independent games have the ability to just be quicker in single-player mode because-- While not all competitive aspects might be not be there, because massive multi-players [INAUDIBLE] They can be hard in single-player mode because they don't, for example, they don't need to remake the investments up.
So a lot of people are frustrated when they lose.
But they explore that realm more readily
I do think it's also a kind of nostalgia for an earlier time.
And I do think that it's also part of the question of difficulties.
So it's a part of it in the sense that all of the edges are being removed, or the challenge is being removed from big game productions to please everybody.
If you make an indie game, you can make something that's more harder, more focused, and have things like extreme difficulty.
I think that's certainly an argument that people make
Actually I have a question.
Does anybody remember the phrase, Nintendo hard?
Yeah
OK, because that's not what Nintendo is anymore, right?
Not really.
But on the flip-side to that coin, aren't you sort of breaking the number one design rule, flexibility?
When you make games incredibly difficult such that you're carving out a very small niche audience and saying, screw you to everybody else
Yeah, and so these are subjects we'll get to in a bit-- that the arts and crafts movement also has this political theme that it would be for everybody-- art by the people, for the people.
But then the criticism is that it ended up making conspicuous consumption for rich people.
And you see the same thing with the locavore food movement.
It's usually pretty expensive actually.
And so it's something that happens, that in a way you cannot have these ideals of broadening things, this kind of production.
But often it also [INAUDIBLE] flip-side of actually narrowing it and making it an elite object for [INAUDIBLE] One thing I thought was interesting was anything common in the way-- just talking about the visual style in independent games.
And he had three games that we often talk of as being independent, and [INAUDIBLE], which has a pixelated or large pixel style and yet it moves this torn paper and crayons [INAUDIBLE].
Obviously, children's drawings with crayons.
And so on one hand these are different graphical styles.
But actually, you can see what they do have in common is what you can call a double layer.
It's a representation of a representation.
So the [INAUDIBLE] represents 1980s size pixels which then represent a game world.
And yet it moves with the sense paper which then represents a game world.
[INAUDIBLE] the sense experience which then represents a game world.
And compare this to a modern Triple-A game.
You don't really have that.
You have 3D graphics within the game world.
So when can say that-- I think that's pretty common.
And I think without a representation of independent game-- I think when we usually see a game and recognize it as being an independent game, it's often because you have this type of style-- a representation of a representation.
And often we'll have to use something from contemporary technology, obviously.
But you said, to emulate something that's low-tech and cheap, right?
So you can see that-- you can compare this to some casual games like matching games which might signal jewels or diamonds or something.
I think it's very clear that most independent games tend to emulate very cheap materials like torn paper and things like that.
I think also the reason why people do this is that, in a way, what you've done is that you're signaling that we have made a deliberate choice to have this style.
And we have deliberately chosen to make a game on a small budget.
So I think this is [INAUDIBLE] but I think that's what this kind of style signals.
And it's also signalling this thing of [INAUDIBLE] authenticity, or knowing who made the game, or transparency in the production process.
And so you can think of it like this, that indie is using this [INAUDIBLE] to mean two different things.
Indie, on paper, means a financially independent team.
Then I think that's also [INAUDIBLE] that people talked about indie from the game-development community-- talked about it in a way that it was morally and politically and aesthetically better.
Not just better games but also better for everybody when games are made this way because you can communicate values and ideals, and so on.
And that indie has a particular kind of style.
And I think that this kind of style of having these kind of representation of a representation, is one that people use to signal, now we are making a game with a small team.
And it has this positive value of being authentic and something that we can figure out what's going on on who made it.
I'll just show quickly how that kind of style appeared.
So this is looking at the-- do you know Independent Game Festival?
Did you follow this?
So that's all right.
So anyway, this is a game dealers conference every year since 2000.
There's been Independent Games Festival.
And this is the longest running festival of independent games.
And this is a jury-based competition.
And so one of the things that's interesting is looking at the winners of the grand prize in this Independent Games Festival.
And one of the things that's kind of odd is that if you look at the first five years, none of the games actually signal independent very well in a contemporary way.
So you can see that the three of them are somewhat regular games of armed conflict.
And then Wild Earth is a Pokemon snap-style game.
The Bad Milk is this weird, never released associational CD-ROM.
You click on things, and then other things appear, and so on.
It never came out.
But you can see that this feels like it's from a different time.
If you see the picture of the two top games, there's nothing that signals independent game in a modern sense.
And I think part of this is because at this time online distribution just wasn't that big a deal.
And so when people submitted to the festival what they did was sometimes probably just they hoped that they would get noticed by a publisher.
Who would then fund them, so they could make a very big version of the game which could them be shipped on a disc.
And then we see that from 2005 on, the winners of this festival gradually became more of this style I'm talking about.
You see the paradigmatic 2-D platform with some kind of twist.
[INAUDIBLE] Then with the winner we see a low-poly style that I think refers to a movie like Tron.
And then we see various takes on water colors and the hand drawing graphics.
And you see, this actually coincides with the gradual rise of digital distribution.
So first there's things like downloadable casual games, that it becomes possible-- or those Flash games sites.
It was more possible to distribute a game without having to put it in a box.
More recently then we see, we still have this pixel style.
But then that gets merged with different things, like in a game like Monaco it has these various lighting effects.
And then in Minecraft and [INAUDIBLE] it gets moved into a third dimension-- so not that-- and this is what I call counterfactual nostalgia-- in a way it's pixelated as if there was a time in the 1980s when people would make [INAUDIBLE] games with big pixels.
This never happened obviously.
But we have this Steampunk anachronism about it.
But still, you can see, it signals that this is a particular game-- or a particular style and then they used modern effects on it.
In the last two years you have this flat mostly gray-scale pixels [INAUDIBLE] now.
But then there are certain things that seem to be happening in the game play where half [INAUDIBLE] to me the life of a poor [INAUDIBLE] and pay the police simulates being an immigration officer.
And so I think it goes to [INAUDIBLE] people talk about the moral current.
Like there is a lot of discussion about participation, in various ways, in games and certainly a lot of emphasis on trying to make games with more serious themes so they'll cover a broader range of themes.
And so you see-- so this shows you where this kind of style comes from-- the representation of the representation.
You'll also see that basically every single winner since 2005 of this festival has had this representation of representation.
All right.
What does it mean?
Well, I think there are a few contradictions in independent games.
So one of them has to do with what we call the DIY movement.
It's that anybody can make a video game now with independent games-- versus the idea of independent games as being a way for people to do games that are particularly expertly crafted.
So some of those like Terry Cavanagh-- he talked about how it's easy for him to make a game with this pixel style because he doesn't consider himself that great a graphic artist.
And so this pixel style is easy for him to do-- to make in a convincing way.
On the other hand, if you want to make a game like that in a tool like Unity 3d.
But people at Unity 3d really doesn't want you to do that.
So the people at Unity 3d will do a serious filtering thing on your texture so if you just draw something with big pixels it will be very blurry, like the image on the left.
So you have to do various things of changing the settings in the rendering of Unity 3d to actually do pixel style.
That's not really what the tool is meant for doing anyway.
So this develops the possibility of demonstrating technical expertise, by working against the intentions of the tool.
And that is a kind of feature.
And so I think certainly that's the conflict within independent games-- between making independent games being these very small games in which you develop and have the opportunity to show off how great they are at various technical skills in a delicate and very small system.
Versus independent games as being something where it's open to everybody.
So you can take a designer like Anna Anthropy, who wrote a book called Rise of the Videogame Zinesters-- How Freaks, Normals, Amateurs, Artists, Dreamers, Drop-Outs, Queers, Housewives and People Like You Are Taking Back an Art Form.
So it's about making game development more democratic.
And at the same time, when you see Anna Anthropy's games, she's actually a very, very good designer.
So she makes these kind of games that are very-- she's good at combining elements from game history and to use them in a new way.
The second element-- second contradiction I think is more on the use end.
That, on one hand the idea of independent games tends to have this idea of democratization.
It becomes games by the people for people.
On the other hand, I think also that now that it's so common to play video games-- and more than 50% of the population actually plays video games on a regular basis-- then I think to some extent indie games can be this way of showing that you have a more sophisticated taste than the great masses, right?
That if everybody played Candy Crush, then you can show that you are playing some obscure game from the Humble Bundle that regular people don't really under-- have learned to appreciate.
Then it becomes more this kind of fine wine tasting issue.
So I think that certainly you have break-out indie games like Minecraft, obviously.
Which of course is a very, very broad hit across a lot of countries.
It sold, what, 50 million copies?
Yeah, mind-blowingly large
Mind-blowingly large.
And then of course still the developer, [INAUDIBLE] He still has this scruffy look to him.
So [INAUDIBLE] they said he had stylists to make him scruffy so he keeps his indie credibility
He's [INAUDIBLE]
Yeah, but you can see this a ongoing conflict within independent games-- whether this was something that's supposed to be very broad, or whether it is a connoisseur thing.
And I should say, I do think that you can actually see this as a result of the fact that so many people are playing video games.
So it used to be that you could say, the fact that you play video games made you belong to a particular category of people.
But now that video game playing is so common you need to-- people need to select a certain subset of video games to have an identity as video game players.
And I think that indie games can be seen as a kind of response to that.
It's something-- it allows you to feel that you have a particular place, right?
So according to authenticity and the [INAUDIBLE] Richard Peterson talked about the idea of the authentic-- [INAUDIBLE] So one of the things he studied is country music.
And he looks at how different people argue for various types of country music as being authentic in different ways.
So it can be authentic in terms of who recorded it, or in terms of style, or various things like that.
And so he says that authenticity works when people try to put in effort in order to make something appear authentic.
So it's just like the restaurant that they spend energy listing all of the places from where they got their ingredients to make the whole menu appear more authentic and local.
A kind of critical-- this particular kind of critical argument against the thing that you like is, to be experienced as authentic, something must be marked as authentic.
And this makes it authentic-- inauthentic rather because something has been done deliberately.
And you can certainly see all kinds of products, obviously, where you can see there's some kind of advertising agent.
He has spent a large amount of time trying to figure out how to make that particular thing look authentic-- by choosing the right [INAUDIBLE] or colors, or making it appear like an old country store even though it's a big multi-billion dollar corporation.
And so you could see this as-- you can think of-- this would be a critical idea of-- this would be a critical think you can say about independent games.
But of course you don't necessarily have to make games with this particular style, right?
And then test it in ways that something inauthentic about choosing a kind of style to seem authentic.
And so I think it's also a bit more complicated like that, right, because, in a way, just the fact that it's possible to make a game and choose a particular style.
That style is so interesting because it's actually cheap-- it is fairly cheap to make games with large pixel or scanned paper, or things like that.
So even though you could say it's not-- it's something that people deliberately choose.
And so it's not authentic in that sense, but it's still something that enables game development.
It does solve that particular problem of how do you make a game on a small budget and make it appear as a deliberate choice, rather than just a game with too small a budget.
So you can see that this is what that particular style does.
I should say there are a few games that we tend to talk of as independent which doesn't necessarily match this style.
So [INAUDIBLE] is a particularly interesting case because it does have this representation of a representation style.
It's made to look at the painting.
So in a particular way, it's not meant to look as somebody tried to off-hand improvise a painting or drawing.
It's actually meant to look a bit like fine art.
And so, you see, it-- in a way it has this thing of being a representation of a representation.
But here it's actually supposed to signal this having fine and nice or sophisticated.
And there is-- and I think this [INAUDIBLE].
You guys play [INAUDIBLE]?
So [INAUDIBLE] is meant to be a kind of work of art with a capital A.
And I do think this is why that particular style was chosen.
I think it's a bit simplistic
My name's Andrew Grant.
I'm the technical director for the MIT Game Lab and sometime lecturer.
I've been playing games pretty much all my life.
That's what my family does when they get together.
And then when computers started doing it too, that's what I do with computers.
I went to the games industry in 1994 working for Looking Glass Technologies and then I moved out to California to work for Dreamworks Interactive.
And actually, all told, that didn't last terribly long.
I burned out after about five or six years in the games industry.
And then I went to be a solo programmer, doing random contracts and work-for-hire, kind of like a gunslinger from the Wild West for awhile.
But no matter what I do, I keep getting dragged back into the games industry.
I like making games.
It's what I want to do.
Right now I'm playing a lot of things I'm playing a lot of the cooperative LEGO games with my son.
He's four and we're playing through LEGO Batman, that sort of thing.
And we recently started LEGO Lord of the Rings, which has been a bit of a challenge for him.
It's a much darker LEGO game, oddly enough.
And so he'll say things like is Boromir a bad guy?
And I'm like, well, that's hard.
Let me explain to you, you're four.
He's confused.
Is Gollum a bad guy?
No, he's definitely a bad guy, yeah.
But why is he going with the hobbits?
Right, he's kind of a good guy, too.
I don't know.
Anyway, it's a complicated story for a four-year old.
And it's kind of fun.
I'm working on a number things right now.
I'm working on a side project with the astronomy department on some software for them.
I'm working on an IOS port of a game called Yomi.
And I'm also working on a fractal tangram independent game.
And with Rick, also, at the lab, I'm working on a massively cooperative strategy game.
This semester we're trying a couple new game engines.
We always try a couple of new game engines.
But I always like to let the students sink their teeth into a couple of them and then come back and tell us what they found out.
There's a degree to which I can do that myself, but it really is more useful, I think, to get some people with fresh eyes to come in and try them out.
Usually they come back and tell us the old standbys are the ones they want to go with.
We usually end up going with Unity and Flixel as our reliable ones.
But this year we're trying out a new language hacks, which also has a Flixel implementation.
And we're also trying a game called Phaser, which is used by the Education Arcade.
So we really want to get our teeth into that and see how we like it.
I've always been interested in artificial intelligence.
When I have a chance to sit down and play with stuff, I want to my computer figure out how do things that it couldn't do before.
That's a lot of fun.
We are in a really interesting place, I think, in games right now.
It used to be the case that when I first started playing, that there would be two or three interesting games a year.
And you could play them, and then you waited very impatiently for the next ones to come out.
And now we're in a place where you cannot play all the interesting games.
In fact, I can't even play all the really interesting games.
There's just too many of them.
But in spite of that, I still kind of want more.
One of the neat things about games is that there's a whole bunch of different games you can make.
I feel like, a lot of times, we get stuck on a couple types.
We don't necessarily need another first-person shooter that's super realistic modern combat.
There are some good games there.
Don't get me wrong, but we don't need five of them a year.
But what I'm really looking forward is seeing all the weird stuff, the stuff that no one expects.
The new, crazy things, and the more games get made, the more crazy stuff gets made.
And I love that stuff.
So when we're looking at assessing the students, we're looking an awful lot at their work on their project.
More so than the actual project itself.
And we're also thinking about what did they learn and what can they demonstrate that they've learned from the project?
So a fair chunk of their assessment is making sure they're in class and participating.
And that's actually separate from projects.
And then, within the project we're looking at how well they managed and ran their project, and worked together on a group to get their project done.
And that, again, has less to do with how the project turned out and more with how they worked on it.
We've got about an equal chunk on does the project meet the specified requirements for the assignment?
As we say, we're not actually looking for good games, here.
We don't expect you guys to come up with good games, given all the constraints we piled and the time you have.
What we want to know is, can you make a reasonably complete game that fits all the requirements we given you, and runs well?
And then, the final chunk is their own personal reflection.
We ask after every project, we ask each of them to write an essay, basically.
It's a short essay.
But it's talking about what they did on their project, what worked well, what didn't work well, what changes they would make to the process and to the project on their next project, because we really want them to be learning from these experiences.
And so what we're really looking for is proof that they have learned something, and it's going to affect the way they work going forward.
One really great thing that we're finding this year is we've got a large number of women who are taking the course.
We really try to focus on getting a good balance of diversity of voices in the course.
Of, I think, 65, we're going to have 25 women first day.
And we're doing basically this first day is to see what we can do to keep them engaged and keep them as part of the course
We have had, in the last couple of years, we've had almost twice as many or three times as many women on our first day.
And by our second week, we will be down to only four or five women in the class out of 30 odd people.
So we're really hoping we can do a better job at keeping them this year
And we're trying a couple of new techniques this year.
We put a harassment policy into our syllubus-- as well as an initial email that went out to all of the students-- just making it clear that no, it's not OK to harass or to make your classmates feeling uncomfortable.
And if you are in a position where you feel uncomfortable, you can always speak to the instructors.
A lot of this is just standard MIT policy, but it's not normal practice for classes to actually state it upfront.
Still, we want the students to understand that they have these options.
What we also try to do is make sure that we have a lot of women speaking in our classes, so lecturers, for instance, or guests.
We try to make sure that it's very easy to be able to see yourself as a professional because all of our speakers work in games in some manner or other
I mean, we're not entirely sure why we're losing so many women.
Because it's all in the beginning.
It's not that we lose them later on.
The women who go through our class, we get a very high percentage of them coming back for 617, The Advanced Game Studio Course.
Once we've got them in the class, once they decide it's going to be an interesting class to take, we seem to keep them.
The question is how can we keep from scattering them off in that first week?
Which is when MIT students can say, oh, you know what?
This class looks like, I think I'll go take some other class.
So I think we've put more of our work into focusing our upfront making it clear what the course is about and trying to make sure that we are, in fact, being welcoming
A lot of it has to do with being welcoming but also providing challenges that more people might be interested in.
So in this case, it's not necessarily targeting women, but really just targeting a diverse group of voices in the class.
So for this semester, the design challenge, we're really focusing on what the design challenge is in the class from day one.
Where, in previous semesters, we talked about a little bit in the first couple weeks, and then we didn't get into it deeper until midway through.
So in this case, this semester it's about decision-making in games.
We're talking about the types of games many different kinds of students would want to play and have played.
So it's not just that you're coming to a game course because you play all sorts of games, and then you find out, well this one's just going to be about war games.
So talking about narrative, talking about fiction, also talking about strategy, talking about chance, and talking about all of the different elements they're going to explore the next semester in the beginning, hopefully, we'll keep them interested.
And then also having a client in the class where they're actually making a game where it's going to do some good to somebody else.
They're not just making a game to make a game.
In this case, the games they're making are going to be used by the Red Cross Red Crescent Climate Centre, particularly to help policymakers to better understand the need to devote resources, time, and money to disaster preparedness.
So it's a big topic.
It's a timely topic.
Climate change is being talked about a lot right now.
MIT students like those kinds of MIT-styled engineering problems.
So hopefully that also keeps a good number of the students keeping in the course after the first week.
We never have a problem keeping our number high.
Our numbers are always high.
It's keeping the diverse number of voices high from the get-go.
That says choices for your game project.
These are the six choices that I want to make available to you.
The next one is perhaps the most challenging one intellectually.
It's about new funding mechanisms for disaster preparedness.
I will explain this.
We call it forecast-based financing, or FBF, and it's the one that has the most potential to transform the humanitarian world.
All right, moving to the next one.
The next one is the most complex.
Do you see the drawing of the time bomb with the fuse?
OK.
So allow me to give a little bit of context.
We did a little bit of this conversation when I showed up a few weeks ago, but I imagine you all forgot by now.
The humanitarian sector has two main funding mechanisms.
Imagine a pot of money.
Even if there's almost no money there, if the pot exists, money can go in to then be taken out to spend.
One of the pots of money that exists, one of the funding mechanisms that exists, is for disaster response, meaning hurricane or flood or volcanic eruption.
Something has already happened, is happening, and is killing people.
People are dying, but if you did something, you could save lives, for example, get on a boat and rescue people who are stranded on a roof.
Or people are sick, go give them medicine.
This is disaster response, and if you spend money, you can do very good things when the funding mechanism exists.
There's another pot of money which is for a normal day.
The Red Cross has vehicles.
The vehicles need maintenance.
They need to change tires.
You need to buy a new computer.
You need to pay rent and electricity bills.
You need to train staff.
So on a normal day, you have a budget for, you know, either maintenance or development of new projects or writing proposals and so on.
And that pot of money exists.
We call it the annual appeal.
Every year we appeal for help, and people or donors or governments give us some money.
What does not exist is what the drawing, the cartoon depicts, which is when there's a signal from science that a disaster-- that something bad is likely to happen soon.
We don't have money to fund action before the disaster, after the forecast.
So you can click to see the text.
The context is that there's that missing money pot.
We have my for after a disaster, we have money for the normal day, but we don't have money for before the disaster after the forecast.
The issue is that if you spend money in that time window before the disaster and after the science says a disaster is likely, you can do very good things for very cheap and relatively simply and much more reliably.
It's not only more expensive, but more difficult, to save a life when the storm waters are flashing through a city and people are about to drown.
The boat is unsafe.
Whereas if you do it the day before, when the water has come down from the sky, but it's still in the river and not in the city, people can take action by just walking away on their own means, with their valuables, and not just depending on someone to show up with a boat.
So we came up with a financial mechanism.
We call it forecast-based financing for disaster preparedness, which is [INAUDIBLE] in our particular method.
There's a pot of money, so like a bank account.
You need to have a threshold of disaster risk that needs to be exceeded.
So it can't be two drops of rain.
It has to be a lot of rain.
It cannot be some breeze.
It has to be a hurricane of Category 1 or more.
But once the threshold is established and reached by the forecast, then there's some standard operating procedures, some actions that people have to take, spending money to save lives.
And that is something that if you could come up with a game that captures that, this is the one that if we help communicate forecast-based financing-- I said to the public, because it can work for a broad audience digitally.
Ideally it would reach those who can contribute either money or decisions, muscle power.
The chances of embrace for this one are not as high as all the previous ones, but if it were to happen, it would have the highest impact.
Because right now there is billions of dollars being spent in the two extremes, and if a small fraction were spent-- invested in that precious window of opportunity when you see the fuse getting close, that time of effort can really have a very, very high impact.
There is a link to a journal article that should be very clear for MIT students.
We also have some simpler readings on general.
And I have been engaged with it.
And I'm ready to help.
This is, I insist, the one that is intellectually most challenging.
But I think if you nail it, it will also be the most rewarding.
Questions?
So one thing.
When we first started designing this class, this was actually the problem that we were most interested in, mainly because the other problems we just hadn't heard about since.
The cholera was a recent proposal that was asked for, and Ebola, of course, very recent as well.
So if you really enjoyed some of the challenges from projects two and three, and you really took on the planning aspects, the trade-offs aspects of those two games, if you're looking at kind of trying to communicate to another person how these kind of probabilities works, this is probably a really good project you might be interested in.
And I should say also that working with a few MIT teams, including the Humanitarian Response Lab, including the Environmental Engineering, we have two generations of three students plus on professor go to Uganda to work on making this happen for real.
We got German money to do it in Togo and in Uganda.
So it's beginning to happen, but it's hard to explain because it's dry.
It's like explaining insurance.
People get bored before they get it.
So maybe a game can help motivate.
STUDENT 1: Our team is working on forecast-based financing.
So because we're about planning for disasters, our game is pretty heavy on chance.
But we want to teach the players to do is to use planning and forecasting in order to reduce the effects of that chance and sort of gain valuable skills in mitigating that
So basically if you have-- there's one, two, three, four, five testers, plus let's say each team send out two people to test other games.
Remember to rotate.
We're going to do this for about 20 minutes, as long as it takes, and then see where we are and do it again to get some just quick testing and make sure that the five of us get to play a good number of your games and give you some feedback on that.
So remember if you're using digital-- if you're not-- If a computer's not being used for testing, close it, or make it look like it's not being used for testing by typing on it.
All right.
Let's get started.
[INTERPOSING VOICES] STUDENT 1: Hi.
We're forecast-based funding.
The general idea behind our concept is that planning ahead for disasters is much better than trying to react to them.
So if you can have operating procedures or ways of planning for them and money allocated for that, you can reduce loss of lives in the event of disasters.
We are actually between two prototypes right now that we're testing to try and get at the ideas.
The first one is a sort of higher-level city-based simulation of cities at risk of disaster, which you then have to fortify by training volunteers or preparing for upcoming disasters in order to prevent too much damage from happening to them.
One of the problems that we're seeing with the game is that it's kind of abstract and not as interactive for players to connect with, but they are getting a good understanding of the idea of planning ahead.
STUDENT 2: And so to try and address those issues, we have a second prototype right now, which is about trying to actually, like, rescue volunteers in a flooding-- rescue people in a flooding city.
And so that hits the other end of the scale, where the player is told ahead of time this disaster was planned for, versus this disaster was not planned for, and the appropriate effects for each.
And then they have to rescue people under those two different conditions, and then they get to directly compare what the experience is for, like, acting in one circumstance versus the other.
STUDENT 1: Over the next couple days, we'll be looking at what we learned from both of them and trying to either combine them or pull out the parts that we thought were really useful to make one game
I can tell you've worked hard on this.
This one's a hard one.
STUDENT 1: Yeah
Did we assign you a target audience for this one?
STUDENT 1: I think it was stated a couple of times that we should be looking at people like policymakers or donors in the sense of people who would be allocating funds from governments or nonprofits, things like that, basically to make it clear that this type of planning is a good idea
Oh.
And what-- have you decided on a technology yet, or are you still pondering it?
STUDENT 1: We're probably-- STUDENT 2: We're using Phaser
OK. [LAUGHTER] All right.
Thank you.
[APPLAUSE] STUDENT 1: Our game is called Hello Waves.
We don't have the title screen in yet, but the idea is still forecast-based financing.
So it's a little washed out on there, but if you can see, there are these five different sand castles, with workers associated with each, and up here in the corner, we have a forecast of what the water level will be like in a couple days.
This is the water level right here, and throughout the game it'll rise and fall, and the idea is not to let your workers get drowned by the rising water.
The other aspect of the game, though, is that you have a certain amount of supplies, and whenever you move someone from their castle in order to prevent them from getting drowned, they will consume supplies from that stock, and if you don't have enough supplies then they'll take damage.
So as you can see right now, everybody's in their own castle.
If we click Next Turn, the water level will change, and we'll see that the forecast will update.
We have a range of values on this forecast, a high prediction and a low prediction, and if you mouse over any of the castles, you'll see a red line appear on the forecast, just to help you orient what heights are on the forecast, or if you're looking at the forecast, what it is in the real world.
So you can see the water move over these couple days.
Right now the forecast is pretty low.
If we see that it's kind of getting close and we're worried about someone, we can click and drag them to another castle, and on the next turn they'll move over.
You can also click people to toggle between building and gathering.
It's very hard to see the status text right now because it's small.
We threw that in quickly for right now.
But he is set to building and these three are set to gathering.
So we'll see that the each turn these three will gather one supply each.
This teddy bear will consume one.
So we'll see supplies go up by two and victory progress will go up by one.
What we have planned for the rest of the game is we've made a lot of progress in terms of the intuitiveness of our game once you understand it.
But right now, there's a lot in the game that isn't explained from the get go.
And so if I give that spiel to anyone, they can play the game fine, and they do pretty well with it, but you just open up the game you're lost.
So a lot of our work is going to be on the UI, of making things more self-evident and making sure that people can understand what's going on and what their actions will do without me having to stand there and tell them
Any questions?
All right.
So I'm going to ask you again, Tom, what is your biggest risk going forward?
STUDENT 1: So-- STUDENT 2: I mean, I think in terms of our technical implementation, we've already got good playtests of our work done, and we have all of our major graphics in, and things like that.
So I think our major risk going ahead is not being able to explain the game properly, and so even if we have a finished product, it may still not be playable if we don't write good instructions
All right.
So on that front, think of incremental additional features.
So the game begins with a super-simple, even maybe too boring choice, but then a new choice arrives.
So the learning by playing can be staggered in a way that is intuitive.
Good.
Good luck to you, and we'll talk more about choices later, but-- I should say that this high versus low prediction is something that I wish scientists did like you're doing, because most people say one value and then it's not that value, it's up here, and then everyone doesn't believe forecasts.
Good.
Thank you.
STUDENT 1: Thank you.
[APPLAUSE]
OK. One down.
STUDENT 1: OK.
So it's a little light on there, but our game is Hello Waves.
It's a game about forecast-based financing that we're developing for the Red Cross.
The idea of forecast-based financing is using the idea of forecasts about the future in order to make decisions that are more effective than just reacting to disasters when they happen.
So I'd like to show you a playthrough of our game.
So, as I said, Hello Waves.
Here are the instructions.
We would have done a tutorial if we had enough time, but this gets the idea across.
And designing a good tutorial that actually teaches the player well is kind of hard to do in a logical flow, so we have these instructions that explain how to play, what the point of the game is, and a couple of tips about how the game works.
So if we go into the Play screen, you can see something similar to what we showed you a couple weeks ago, where you have all these different toys at their castles, and as you go through the days you'll see a forecast of what the water level is going to be at over time.
And so you can see that right now they're gathering candy.
The water level will change.
This character is underwater, and so he takes damage.
And so what you actually want to do is you want to move toys out of the water so they don't get damaged.
However, they can only move one castle over at a time.
So because we didn't think ahead well enough, we'll see that this truck will actually take damage on the next turn, because he's underwater.
Actually, he ended up not underwater.
We got lucky there.
But theoretically he would have.
And so as you go through it, the end goal of the game is to build a castle.
And so every toy when they're home can either choose between gathering candy to feed evacuated toys or building the castle to reach your end goal of the game.
And then the idea of the forecast is that you want to use the forecast in order to know how much candy you're going to need over the next couple days, and to use it to know when you're going to need to evacuate various toys from their variously-heighted homes.
So back to the presentation.
Full screen.
So we had a couple of challenges in the design process.
The first-- and the major one-- is really that we were trying to teach about forecast-based financing, which was a bit of an abstract topic.
It's a little different than just thinking long term, because you have to use the idea of there's some information we know about the future that we want to use to make optimal decisions, or at least decisions based on some idea of the risk that's out there.
But we also wanted to avoid things like just pushing a button to win, where you have all the information you ever need, and there's just one option that you know you're going to pick every time, and the game has no thought whatsoever.
And we also needed to think about how we were going to communicate the forecast to the player so that they could then use that to make decisions.
One of the problems that we also ran into related to this was that we focused a lot on the idea of forecast-based financing as the topic and then tried to build a game built on top of that topic instead of building a game that used forecast-based financing.
So that held us back a lot in the beginning when we were trying to come up with ideas.
We also had a really difficult initial target audience of policymakers, 50, 60-year-old government officials or people at NGOs who were going to be using forecasts to make decisions of some sort, and it was supposed to teach them about how they can use forecasts to make these decisions.
Except this was a really difficult audience, because they don't generally play games and they don't have a lot of time to learn about this kind of thing, and so we couldn't really expect to get them to sit down for a long period of time and play around with our game.
In order to deal with these problems, we came up with a couple solutions.
The first thing was that we had no idea what kind of game would work or would make sense or anything like that.
So what we did is we just broke our team up into multiple groups and came up with a bunch of different prototypes.
On paper we had two different prototypes, one that focused on the idea of managing a city and its resources and its response to disasters, versus focusing on individual people and how you're going to move them around to keep them safe from the disasters.
And then we went into a whole bunch of different digital prototypes, one that was a text-based game about managing a city.
We had one that looked like this, where you had two different cities and you were specifying what workers were going to do or when they would leave the city in order to stay safe.
And then we would take the different ideas that we were learning from both of these prototypes, combine them together, take things out, and our final prototype actually uses ideas from all of these.
Another solution that we kind of got lucky with is when Pablo came to play our game, he told us that we actually shouldn't focus on the policymakers, because he wasn't confident that he could actually get them to play the game.
And so we switched our target to being grade schoolers, which is why you saw the sort of cutesy art with the beach and the toys.
This actually made it a lot easier for us, because we could target people who probably had some experience with games, or at least wanted to do something fun and would be curious to learn about our topic.
STUDENT 2: So another for our development process-- Or I can just speak here, right?
Yeah.
Just speak right there.
STUDENT 2: So another big issue-- another set of challenges that we ran into was through our development process.
And so we had initially issues with communication and facilitation.
Our team had a wide variety of experiences and backgrounds.
Some were hardcore gamers, some mostly mobile gamers.
And so there were initially a lot of disagreements on what level of game we wanted to create and what sort of game, casual versus hardcore, that we wanted to create, and so we needed to overcome challenges of facilitation and communication within our team.
Another major challenge in our development process that we had to face came from our design issues, where for a very long time we had a vague vision of what to do.
We didn't know what kind of game we wanted to make, and so we purposely tried to keep our game ideas vague as we built prototypes.
But then we ran into issues where we would have solutions but, like, no consensus on which solution was best, and where we went for long periods of time without having a clear direction of where we wanted our final game to be.
So our solutions for the challenges posed by the development were a team structure.
And so we structured our team loosely into three subteams, a production subteam which would take care of production, like the deliverables and making sure that all the game ideas are being communicated properly, a technical team which worked primarily with the code and making sure the game got done, and then a user experience team which handled art, UI, and sound.
And so we kept the responsibilities flexible.
So as team members got busy over the semester or as changing conditions led to different people contributing, we kept-- responsibilities were able to easily flow between teams and team members.
Additionally, another idea we started with in the beginning was the idea of subteam leaders, which were the two people marked with the Ls.
But that was an idea we later abandoned in favor of just having a more flexible team structure or flat team structure.
Another solution for helping our development process was the use of good code practices, and I cannot emphasize enough that this really helped speed up our development, because we didn't run into trouble with code.
It was mostly with design.
So we used Yeoman, which is a JavaScript module system, and basically it allowed our code to be interoperable.
We could write one module separately from another module.
So that solved a lot of issues with dependencies or people working in parallel, because it allowed people to work in parallel without overwriting each other's code.
We also used good code practice with state machines and MVC, which is model-view-control, and so we had a very object-oriented code, very modular.
And when we did need to change our code, rip it all out and put it back in, it actually didn't turn out to be too painful, because we just had to switch a couple objects around.
And then one final solution that we used for our development processes were Slack and Scrum.
So Slack is like a modernized IRC chat room, and it's very feature rich.
It has a lot of integrations with GitHub and Google Drive and things like that.
And so that sort of real time communication actually made it so that we didn't really have to meet outside of class too often.
If someone was working, we would just email out saying I'm on the Slack, and then people could meet on the Slack, and it was full-featured enough-- like we could send attachments and things like that-- that most of our in person communication could be done in class.
And in class we adopted a sort of, like, daily Scrum format, where we simply said what we had done since the previous class and what our goals were until next class.
So in the end, though, we did have to cut some features.
These features mainly were the idea of multiple-- again, a tutorial or multiple levels, simply because there would just be too much content that we would need to playtest in order to make sure that it was of consistent quality and got our message across.
And also trying to add more individuality to the toys that you saw.
They have different graphics, but that's as much as we could do given the time constraints.
So kind of just bring it back.
Our three worst decisions were first, we did end up spending a lot of time on code and assets that never got used.
We maybe, like, used 10% to 20% of our final work in our final project.
Actually, maybe that is a bit overkill.
But OK. Maybe 30% of our final code and assets in our final project.
This really was due to this second bad decision, that we kept the game and its direction too vague for too long.
We always were holding out for, oh, maybe we'll be able to come up with a better game idea, or maybe we'll find some magic solution to how we can make forecast-based financing into a game.
And because of this mindset, we spent probably the first half of the project, like, just staying too vague.
And that hurt us in the end, because we spent so much time going in all these different directions.
And really the decision that kind of captures those first two, though, is the fact that we tried to make a game on top of forecast-based financing.
So we had forecast-based financing, and we were like, how can we skin this as a game?
Whereas once we switched that mindset and thought, let's have a game, and how can we put forecast-based financing into it?
I think that was the moment that we then came together as a team and really started making the final game that we wanted.
So our best decisions-- STUDENT 1: So one of our best decisions was that we chose good tools at the beginning, which meant that as we went through all these different digital prototypes, we actually didn't have to completely rewrite our game.
We could pull out the way that workers worked in one game.
We could pull out the view that we were using in another game, and then we could just combine them together, and that allowed us to move quickly whenever we were changing our prototype.
We also weren't afraid to trust each other, both in terms of what everyone was working on, but also in terms of the decisions that we were making.
And so when we said that we had to throw something out, we all understood that it was for the good of the project.
And we didn't have a lot of complaining or hurt feelings when something didn't get put in or when we decided to throw out certain assets or code.
And when we got to the end, we had been through enough of this vagueness that we were all kind of frustrated with it, and we realized that we had an idea that we all liked and we really got on board with it and made it happen.
Once we started working on our final idea, and we saw that it worked, basically every decision from that point on was how do we make this game better, and we were all on board with that vision.
Thank you.
Any questions?
[APPLAUSE]
Anyone in the audience have feedback for them first, before--
You can't see those sand castles very well.
STUDENT 1: Yeah, we'll have to make all of our slides darker
Yellow and white
Yeah.
Like, the yellow and white, on your early screens.
I was doing this on your instruction screen
It's also--
I realize that's the game, not the presentation, but wow
It's also a little faded in the game itself.
It could be the display resolution you're using.
So we're all done with everything, try a different resolution.
See if it makes a difference
The music in the game level was too high.
It was overpowering your voice.
So I think you were almost hollering just to be able to be heard.
it doesn't need to be that loud.
So you can just crank down the volume, either on the computer or on the controls over there.
Something that I had a question for, you don't have to answer it today, but you might want to put it in your presentation was, how long was the design of what you eventually established on the table before you decided that this was the thing that you were going for?
Because I'm assuming it was one of your vague-- it came up during your vague idea phase, and you're implied that you were in that mode for too long.
But I don't know how long it was on the table, or was it only something you figured out at the end of the vague idea phase?
Because otherwise you wouldn't have been able to switch to this idea if it was not on the table in the first place.
And how did you decide this was it, that the sand castle game was going to be it?
I understand-- I think you very clearly explained the benefits of deciding this was it.
But how did you come to that conclusion?
So that's stuff I wanted to hear more about.
But you don't have to answer it now
Yeah.
Just a little bit of specificity.
You mentioned you spent too long on that design phase.
I wonder how long that was
You said Pablo switched your target audience for you.
Telling us whether that was because of the game that he saw or because of something else-- if you know that information, throw it in there.
If you don't know that information, that's fine.
Knowing why you dropped the team lead.
You mentioned you dropped it, but you didn't exactly say why.
Again, really quick.
This is what-- we weren't getting blah out of it, or the flexibility was more, whatever.
And then, yeah, to-- Oh.
Defining your terms.
You were saying things like casual versus hardcore.
Give a little bit a definition of what you mean by that.
It means different things for everybody.
So what is your use of that
Yeah.
I think he specifically said hardcore and mobile, but there are hardcore mobile players out there, so this is like--
And casual can be considered a version of hardcore, just in a different way.
So just be really-- just be a little more clear about-- because I think you were talking about the target audience and the kind of players and the kind of games they might play.
So just be a little bit more focused on what exactly you mean by that.
That's it from me
Yeah.
So in terms of technical observation, Yeoman is not really a module framework.
It's just a system to generate the project and it puts different frameworks.
Again, just check that
So that's a terminology issue, then.
Clearly it worked out for your team.
So we're not saying don't mention Yeoman.
It's clearly a good thing.
But just check your definition of what it is.
Because it'll be more helpful for other people to understand what it is so that they can think about whether they want to use it in the future
Oh, and actually-- So your demo, you spent about three minutes doing it.
We are going to have you have a player from the audience play your game live, without getting a lot of help.
You can talk over it.
You can, after a while, start helping them.
We want to see them at least just to start playing on their own.
Decide when you're going to put that demo in.
You could actually combine them both together if you do it at the beginning or end.
But if you do that, give the player a little bit of time before you start talking
That's all I got
Great.
Thanks.
[APPLAUSE] Snap, come on down.
STUDENT 1: Hello.
We are Hello Waves.
We're a game about forecasting, specifically this idea called forecast-based financing, of using forecasts to make decisions about possible disasters in the future and how to prepare for them.
We'd actually like to start with a playthrough of our game.
So if anybody would like to be a volunteer to try it out
The guest from not our class.
Would you please come down, Andrew?
[LAUGHTER]
Thank you for volunteering.
[LAUGHTER]
Hi, I'm Andrew
Oh, do you need a chair?
There's one right there.
All right.
[?
STUDENT 1: I'm not sure how to make it full screen.
But anyways, this is our game.
I'd recommend looking at the instructions first and reading through them.
And we'll keep the description minimal as you read through, just to show the player's initial reaction to it.
[MUSIC PLAYING]
Music.
STUDENT 1: So yeah.
Start the game.
So as you can see, our game, as I said before, you control some toys on a day at the beach.
And so by dragging and dropping them between the castles, you'll see that their status is changed, and say that they want to move on their next turn or that they want to be collecting-- or that they're going to be collecting candy or anything like that.
And the game is turn-based, so all of the actions will resolve on the next turn
Am I reading this right?
STUDENT 1: Yeah.
STUDENT 2: You can also access the Help by clicking in the bottom right corner.
STUDENT 1: And so when you go to the next turn, you'll see all the toys move as specified by the status bubble above their heads
So go ahead to the next turn?
[INAUDIBLE] and Next Turn.
STUDENT 1: But unfortunately you'll find that when toys have been evacuated, they're unhappy and need candy to survive, so they'll all take damage
OK.
I don't want to be in that [INAUDIBLE].
STUDENT 2: So you can try returning them to their homes
Ah.
STUDENT 1: And so you'll see that on this turn instead now they have their statuses set to gathering candy, except for the dump truck, who's still evacuated
Gotcha.
OK.
I think he needs to be evacuated.
STUDENT 1: Exactly.
And so the idea is that as the player plays through it, they get better at understanding how to use the forecast to make decisions about the future, both in terms of how much candy that they need to have stocked up in order to weather out the rising tides, and also in terms of when they're going to need to move their workers out-- their toys out of the areas that are in danger
They're gonna be really affected.
STUDENT 1: I think you have those two guys
Oh, right.
So is there a reminder of where they started?
STUDENT 1: Yeah.
It's on the castle.
It's actually blocked there right now by the toy in front of it.
But there's a little shadow on there, an imprint.
And because he ran out of candy and then-- Actually, sorry.
Because he went to a place that was underwater, he took too much damage and then was swept away by the waves.
[LAUGHTER] STUDENT 1: And so on the forecast, you can see that high water is coming for quite a while, which is going to be a danger for the toys, both in terms of possibly getting swept away and not having enough candy for all the toys that you're going to have to move out of the way
[INAUDIBLE]
Ooh.
He may be out of luck
Yeah
And when you've lost two toys, you lose the game
Pretty good.
STUDENT 1: Thank you for playing.
[APPLAUSE] STUDENT 1: So that's Hello Waves.
So in our game we had a few challenges to overcome.
The first was that our game was based on forecast-based financing, which is a very abstract topic.
It's a pretty understandable idea of using information about the future and ideas of risk in order to decide where to allocate resources.
But it's still a bit abstract and building a game around it took a little bit of work.
It's useful to note that it's different than long term planning.
It's not just thinking about what will happen in the future, you know, building a dam to prevent water or things like that.
It's actually about using the information you have to make the best decision for events that may be upcoming in the semi-near future.
We also wanted to avoid making a game that was overly preachy or simplified, where it was clear exactly how you're going to win and you could just basically push the forecast-based financing button to win the game.
We wanted players to actually think and understand the concept there, instead of just coming up with the buzzword of forecast-based financing.
And finally, we had the challenge of actually communicating that forecast to players in a way that they would be able to understand and then make use of.
You could see in that game that we had water levels, and it would show on the map, and we found that players were pretty good at using that in order to make decisions about what was going to happen in the future and how to allocate their resources.
The other big challenge that we had in the beginning was that our initial target audience was policymakers.
Like for Snap, Pablo had come in and pitched us this game idea, and originally he had wanted us to build a game for policymakers that would help them understand the benefits of forecast-based financing and therefore convince them that they should develop policies that would give resources to plans based on forecast-based financing.
So I would like to take you through a couple of our prototypes, just to show you the evolution and comment on our process.
Actually, to preface that, we had a lot of prototypes, because our idea was so abstract and because we weren't sure how to address our audience.
So we built a lot of prototypes to start with.
We had ideas that ranged across levels of scope of what you controlled, where you controlled entire cities or where you controlled individual workers and moved them around, and then we would pull from all these different kinds of ideas, what worked, what didn't, what did people understand, what confused them.
And from that we got a really good idea of what concepts helped people understand the idea and brought them into this final game that we ended up with.
So the project started on October 15th, and this was our first prototype.
It was a terminal-based game where you had some kind of information about a future rainfall, and then you had to type in your commands of how you controlled different cities.
This-- Actually, people found it fun, but as you might expect the feedback wasn't very good and people didn't quite understand how to move forward with it.
It put a lot of cognitive load on people.
So when we moved forward, we tried to give people more easily understandable actions to use.
But the problem with this game was that it was time-based and it updated every second.
And so there were so many numbers flying at people that even MIT students who playtested it couldn't understand.
So we figured that people like policymakers who didn't have much experience with games really wouldn't be able to understand the game at all.
So instead we went to turn-based.
And that helped, but at the same time-- it's tough to see on this projector, but we have a forecast underneath that says how much rainfall is expected.
And again, that wasn't understandable to people, because they couldn't understand what three inches of rainfall meant for their city and they couldn't understand how that contributed to a possible disaster.
At this point, Pablo actually came by the class and played the game, and then he told us that he wasn't even sure that he could get policymakers to play the game, because they may not have enough time.
Instead he wanted us to switch to grade schoolers, because we could teach them something about forecast-based financing and help them understand as they grew up.
And this was great for us, because making a game that was serious, easy for somebody who didn't have experience with games to play, and also fun and engaging was too difficult for us, actually.
So moving to grade schoolers was awesome.
He also suggested that we try to make the idea of rainfall or water levels more visceral, and that's when we came upon this idea of the rising waters.
Whenever players looked at this, they instantly understood the concept of the game.
The feedback might not be there.
The beautiful UI might not be there.
But the idea of having cities with workers in them and a rising water level coming towards them, everybody understood that, and it made it a lot easier for players to reason about the game.
From there we added things like nicer art, better feedback, which you can't quite see in a static picture, more improvements to how the forecast worked, and eventually we ended up with our final version today.
STUDENT 2: So to talk a little bit about our actual development process.
So our team was structured into three main subteams-- production, which was in charge of managerial roles, deliverables, and playtesting, and so there was a shared responsibility there, a technical team, which was in charge of the bulk of the coding work, and a user experience team, which would be in charge of assets and UI design.
We also initially envisioned subteam leaders, where we'd be kind of communicating through them.
But we found the concept kind of redundant, and so we worked pretty much with, like, a flat structure between the three teams.
So from the beginning, we encouraged good coding practice, and so we used good tools available to us.
One of these is Yeoman, which is a JavaScript scaffolding framework.
And this helped us out a lot by basically automating a lot of our JavaScript tasks and making our code modular.
We also used Phaser's state machine, which is this kind of badly documented new feature in the Phaser game engine, which was the JavaScript game engine that we used.
It's a bit badly documented, but it did save us a lot of headaches, and once we figured that out, that proved immensely helpful.
And we also used MVC, which is a software engineering term standing for Model-View-Controller.
And again, by encouraging these good coding practices, we reduced dependencies and made sure that our team was productive.
In terms of communication, we-- also similar to Snap-- used Slack, which is kind of like a modernized chat room.
It's very feature-rich, and so you can share files in channel and things like that.
And we also used the idea of the daily Scrum.
We implemented it in class, and we would say what we had done that class, what we would be doing, and what we wanted to do until next class.
And so the major challenges that we faced during the development process, though, were that our team members came from very different backgrounds and had very different preferences about games.
You know, some of our team members were very hardcore StarCraft players and very good at RTS games, while other members of our team preferred like a more laid-back mobile game, Fruit Ninja kind of approach.
And so trying to mediate those two viewpoints and trying to create a game that would engage both types of gamers was a challenge that we had to overcome.
And so another big challenge that we had in our development process, as you saw through our progression through our different prototypes, was that our direction was not very clear until about halfway through the project.
And so partially because we had different ideas on what the game should look like, and also partially because we had such an abstract idea of forecast-based financing that even we didn't have that good of a grasp on initially, it took us a while to really get settled on what we wanted to build.
And so this really challenged our development process and made us have to build a lot before we got something that we liked.
And so eventually we did end up having to cut some features, like multiple levels, or a guided tutorial for the player.
We thought that this would introduce too much new content that would need to be playtested, balanced, and tested to ensure consistency with the rest of our game, which we viewed as taking up too much time.
And we also cut the idea of adding more individuality to the workers or to the toys that you saw, other than different graphics for each.
STUDENT 1: So some of the worst things that we did on our team is that we spent a lot of time on work that got thrown out entirely.
All the prototypes we did, they were actually pretty useful because of the things we learned about the concept and about how people would play the game.
But we did spend a lot of time on things like art or nitty gritty details that didn't really need to be figured out and that we could have put off until later in the project.
We also kept the game direction to vague for too long.
As Norman said, we spent a lot of time with that, and it probably ate up too much of our time.
Although it helped us learn, we could've moved faster in the beginning to get to a solid idea.
Because once we got to a solid idea of the rising waters, our team started to centralize around a lot better because we could actually deal with something concrete.
While we were dealing with the abstract ideas, everybody was all over the place and arguing about things that didn't quite line up.
And the worst decision of all that we started with, and that sort of made the problem of going too vague and all these other things happen, is that we were originally thinking how do we skin forecast-based financing as a game?
How do we take this idea of using forecasting decisions and then just gamify it?
Which we eventually realized wasn't fun, and didn't help us actually come up with any ideas.
Instead, when we flipped it and started talking about what game could we create and use forecast-based financing to improve it and teach players how to play the game, and therefore allow them to come out of the game having learned about forecast-based financing, the world sort of opened up.
Everything became a lot more interesting and we found that we started to move faster.
Some of the best decisions that we made, on the other hand.
As Norman said, we had good tools, which meant that even when we threw out prototypes, although we wasted things like art resources and things like that, we actually didn't end up wasting very much code, because things like the idea of workers or cities could literally be pulled out of the old games we had, put into our new game, and then reworked to build our new structure.
We also weren't afraid to trust each other and throw out the things that didn't work.
Once we started moving fast, we had a lot of ideas that would come out, and we would say, OK, this doesn't work.
We're actually going to scrap it.
Or we think that this isn't the direction we need to go in.
And everybody was willing to go along with it.
It's not a good feeling to see your things thrown out, but everybody understood that was for the best of the game, and I really appreciate their understanding with everything too.
And part of that all comes down to the fact that we were on board with our final idea.
We were all excited about the concept that we had.
Part of that might have been the relief of coming to a concrete idea after spending so much time being vague.
But once we had that concrete idea, we really moved fast and worked well around it.
So thank you, and any questions?
[APPLAUSE]
Who did your sound?
It's awesome.
STUDENT 1: That was from our UI team
Oh.
OK.
It's really cool
Where did you find it?
STUDENT 2: So it's on our credits page.
Most of it was online.
The credits page in our game.
[LAUGHTER] STUDENT 1: It's a little small, I guess, up there
Oh, OK. STUDENT 1: But it looks--
[INAUDIBLE].
STUDENT 2: Yeah.
"Hold My Hand," AJP, by--
Would it be possible maybe to create a short URL for this?
For the game?
STUDENT 2: Yeah, a bit.ly link?
STUDENT 1: Yeah, sure.
We can make a bit.ly link, and we'll send it out to everyone
Yeah, just so we've go it.
Yeah.
STUDENT 1: That's a good call.
Yes
Having watched the early crash and burn of the playthrough, how often have people-- I don't know if you've actually had a lot of people to play your current version of your game-- do people usually take a playthrough or two before they start getting the concept?
STUDENT 1: Yes.
That's why something like a tutorial would be really nice.
Unfortunately, we didn't have the time to put it in--
Yeah, no, no.
I was just wondering how that-- STUDENT 1: Yeah.
Usually what happens is even if after maybe a couple turns of playing through, they start to get the idea.
The problem is that as the water starts to rise, they haven't prepared enough, and so all their toys will starve or get carried away by the waves, which is a bit unfortunate and probably makes the players feel bad on the first time.
But then they-- It actually teaches them to think ahead about it.
So the next playthrough, they're much more careful and understanding
So I was wondering if you were playing any other games or thinking about other games as inspiration or thinking about how to deal with some of-- hitting the right level of strategic thinking in your game.
STUDENT 2: Well, so I guess in terms of early on, because we had a very different idea of what we wanted to do it early on in the process.
We were thinking about games like Civilization and how did they communicate all these complex worker movements and managing multiple cities.
But once we actually came up with this game concept, I think we had much, much smaller goals, and so did we have any specific game models, do you think, or?
STUDENT 1: There's none that I specifically think of.
There were some things that we were sort of inspired by, standard tricks like when you hover over one of the characters, they got bigger.
Some idea of showing off that this is clickable, things like that.
But specific games themselves, not really
OK.
I was just thinking it ended up being kind of board-game-y.
And I feel like there's a lot of board games where they've been thinking a lot about getting that right level of tactics rather than strategy.
But yeah.
I guess it worked out too
So you mentioned that you were able to change how your workers looked and just keep your models.
So that's the holy grail of object-oriented programming, that you have an object that's reusable and you don't have to throw out the code.
Do you think there's a reason why in particular you were able to achieve that?
Because I think that's not common, necessarily, in object-oriented programming.
STUDENT 2: Partially a little bit of OCD-ness, like, very early on, very strictly saying we're going to write this object-oriented code, and we're not just going to hack things together.
I think that helped a lot, because we actually spent the time in the very beginning to think about, like, which objects were responsible for what and what their purpose should be.
So basically, I think because we moved more slowly in the start and thought more carefully about how that code should be structured, we ended up with having an easier time later on.
STUDENT 1: There's also the fact that because we had learned these things from the prototype that we were putting into this game, that also meant that the objects that we created-- because we wanted similar functionality to things that we had already seen and we knew worked, it meant that we were comfortable pulling out the functionality into that.
So I don't want to say it was designed to fit, but there was the fact that we moved it on purpose, really
I would like to something on that.
So we had a very good MVC model.
So models were in this tree-like structure, and it was very easy to change models.
So models knew about-- Sorry.
User knew about models, but models had no idea about [INAUDIBLE].
So basically, it was quite easy
Thank you.
STUDENT 1: Thank you.
[APPLAUSE]
So one of the big mantras in any kind of environment is testing.
You have to keep testing your game, because no plan meets first contact with the user.
You have to be able to see how real people are going to respond to the stuff that you're making as soon as possible, because many of your assumptions going into a project are going to be wrong.
Now, we encourage students to go out and test their games with as many people as possible outside of class.
This is crucial for the development process.
And there are a couple of assignments where they actually have to show us the results of that testing.
And it's pretty much as simple as sitting somebody in front of a computer with the latest version of the game, or maybe with the paper version of the game, giving them a couple of instructions just so that they can get started, and then stepping back, watching, and taking notes.
Trying to provide as little guidance as possible, so that they can actually see how a player who knows nothing about the game responds to it for the first time.
However, sometimes it can be difficult to get a lot of testing done outside of class.
And students may be a little bit shy approaching strangers, for instance, even other MIT students.
So what we do also in class is we get different teams to play each other's games and give the feedback.
The quality of that feedback's a little bit different, because everybody the a room at that point is in the same boat.
They are all making games, and all trying to give feedback in the way that a game developer will give another game developer, as opposed to, say, a naive player who just wants to have a fun time playing this game.
We will also be giving feedback as instructors.
We give them our experience, both as developers and this instructors.
And we can tell them, that's a great idea, but there's no way you're going to get it done within the amount of time left in this semester, because we've seen it.
We've seen how these projects tend to go during the class.
No one else in the class has gone through this class multiple times.
So this is basically a way to get feedback about the game.
And we teach students various methods to get the information they want, and to record it, such as questionnaires, for instance, different observational techniques, different ways to script the process of getting a new user to actually get started.
And most importantly, the importance of just doing this a lot
So there are a lot of different ways you can do project management.
The one that we teach and we favor at in our course is we look at agile project management techniques.
Most specifically, we look at the one called Scrum.
And by the nature of agile, agile project management really requires a team to work with itself a lot to understand how the team works together and to understand each of their team members' strengths and weaknesses.
This is exactly not what we give our students a chance to do in the course of the course, unfortunately, because they are working on short projects.
Each project tends to have slightly different goals.
And they're changing teams frequently.
So what we do do is we give them a set of agile tools.
We introduce them to the concept of backlogs, project backlogs, creating very short vision statements to describe what the goal of their project is, but not go into a whole lot of detail.
We have them make project backlogs that list all the features they would like to have.
We have them take a look at how they break that amount of work down into how much can we get done in a week?
And realistically trying to figure out how much they can get done in a week, at which point they realize their project backlog is about nine weeks long.
And they can trim down from there before they get started on some of the less important things.
But we show them the tools.
We walk them through using the tools.
And then we encourage them to use the tools that work best for the group that they're with and the project they're working on.
So instead of saying, this is how you do it.
Shunk.
We say, here are a set of tools.
Work with all of them.
Use the ones that work best.
And try to iterate not just your project, but also the way you are managing your project and the way you're working together with each other.
Because that's the only way to get to a better working team, is to find out what works.
Accept that some of your techniques don't work.
And try something new, try something better, until you do find what works with that particular group of people
So we're not only talking about the tools and processes, the hard skills when it comes to project management, but we're also looking at iterating on their soft skills, which often comes down to just how do you talk to another person respectfully?
We talk about team dynamics.
So teams tend to grow in a particular way.
And often, there are shared qualities that all teams have, when it comes from forming from when they're getting their first conflict, and trying to get over their first conflict, to when they're actually performing pretty efficiently, and then when they feel like they're just really kicking butt and really moving forward.
We try to help them get through to that final stage.
At best, they usually get to right to the point where they really start performing along, really start being efficient in what they're doing.
But what we want to do is have them see the whole process, see how the stages differ, and then, come up with ways and approaches to then get themselves through the process, and being able to talk to their other teammates about, maybe, a problem happened.
Maybe somebody didn't check in some code.
And it caused two days of work to go away.
There are better things to do than just to rant and rail at this other person.
So what are the other things they can do to get through that problem?
So we want them to talk about that.
We want them to think about that.
We talk about it a little bit in class.
And we ask them to give us what would they do again?
If they had do this again in the future, what would they change?
What would they do differently?
So we're asking them to do that through their individual write-ups.
We ask for a lot of reflection on the class, on not just the game design process, the game development process, but also this project management process, and the communication between team members, as well.
So my name's Lauren Merriman.
I'm a senior right now.
I'm actually graduating in February.
I took this class because I decided I really want to make video games, and this is a good class to take.
My team is seven people, and we are mostly seniors.
We made a game about [?
cholera.
?]
The other team focused on the younger 8 to 13.
And we focused on a little bit older-- about 13 and above-- so we could have a little bit more complicated mechanics and text where we needed it.
So we didn't really have to rely on figuring out how to make everything really easy to read.
So it uses English that assumes someone who speaks English could read and understand very easily.
And our game is a strategy game.
It's real time, where the days are-- you have to survive a year and the days are automatic.
So you have you go and take care of four different localities, and give them either soap or water containers or electrolytes to help heal them.
And later on you get boiling water.
And you just have to take care of these four localities for a year.
So we learned early on, the class, you start by your first project is just paper prototype.
And then after, you have to make digital prototypes, but they usually expect you to have a paper prototype.
And for this one, they expected you to do it however you wanted to.
And so we started off with a paper prototype, and we sort of then quickly implemented it digitally.
Because there were certain things that doing it with paper would take a long time to make a calculation.
But we originally had part of it was you could see on the screen what the localities, where they were.
And then you would try to implement things.
But we didn't have a screen to help you select that.
And so you'd tell us what you pick, and then we quickly put it in the game and reset it.
So that's what we originally did, and it was really hard to do it that way because people going from the screen to a paper prototype, they were confused.
And they didn't really see how things interacted.
So then our next big thing was make it all digital.
And in the presentation that you saw, we had that original shot of what the game looked like when we had it all digital.
And it was just a really poorly designed GUI.
And it was hard to read and hard to understand and people were really struggling with it.
So I actually was the one who redesigned the GUI, and then it went through a couple of iterations with the new GUI.
So we decided how to make it the most clear, and I sat down and I looked at other strategy games, mostly Civilization.
I'm actually not a big strategy game player, so I actually had to go look around and see what other places do.
And I stole a little bit from Civilization, and a few other games that just had good elements of design that looked clear and understandable-- even to someone who's never touched the game before.
And so that's what I wanted in our game.
So that's why we implemented it the way we did.
And then the game kind of turned into the way it is now.
So one of the biggest things is I'm not a programmer.
So I was doing either design or product management or the scrum master-- is what they call it, which is someone who has to take care of the team and make everything fit together.
Whether it's including art and music and coding-- making sure all those elements fit together into one cohesive whole.
And so that's sort of the role that I started to do and then really liked doing.
At first I thought, on some of these projects, I should take on a coding role to try to force myself to learn.
But I realized I didn't really want to.
I really liked doing this role instead.
But you learn it's really hard.
Especially in a class setting where people aren't really tied monetarily to this class.
You can't lose your job if you don't do your job.
You can do poorly in the class, but because it's a group thing, you might not do that poorly because the group presented like, this is our game.
And so unless you specifically call out that person, they're still going to get credit for it.
And so it was really difficult as the product manager to get people to do what they said they were going to do in a timely manner, and get people to respond to my emails.
And so it was really something that you have to get people to understand-- especially towards the end where we didn't have a lot done, and we needed to change a lot-- that there's not a lot of time, and there's still a lot to do.
And then when I finally got everyone in a meeting and they all sat down, we talked about this, like, everyone, you just heard it-- oh, that's when that's due?
So I finally got people in gear, and so I wish I could have done this earlier in the semester.
It was just this project.
With the other projects, people didn't usually fall into that trap as much because it was only two weeks.
So two weeks doesn't give you a lot of time to fail.
You've got to do it quickly.
And but when it was eight weeks, it was a lot of time and so people are like oh we can push this off a little further.
It was really hard.
I actually ended up having-- I talked to the professors to try to figure out a way to get my team to listen to me.
And it was still really hard, but once you finally get everyone together and really lay it out very clearly, like the clarity is what I was missing.
I thought I was being very clear, and this is how things would do.
And even with the way I was assigning tasks, I thought I was being very clear and I wasn't.
I was really missing concrete details.
I said, guys, implement this idea-- implement that you can take the actions.
But I didn't tell them this is how it should look.
This is how it should feel.
And so when it got implemented, it wasn't what I thought it was.
And I was like, this is really disappointing.
This is really ugly.
So when I sat down and redid it and then told them exactly what I expected, things got done faster.
Because, like we said in our presentation, people weren't really motivated for this game.
And so having the motivation and really vague instructions made it that people wouldn't do it until-- you put that off.
That's a really hard task.
I need to figure out what she expects, and then I need to do it, and I don't really want to.
So that's something you put off to the end.
And so when you give people clear instructions, it makes it easier just to do it.
Because they're like, oh, that's simple.
I can do that really quickly.
And so I think that was the easiest way is when you really set clear instructions, clear deadlines, and you can't really punish people.
There's no way of slapping them on the wrist when they don't do it.
You can just not assign them work.
And you move it to somebody else.
And eventually people realize that they weren't getting assigned any work.
And they realized just, hey, I'm not actually doing anything for this.
Can I have work?
And I said, you have to do it by this time, and if you don't, then we're pretty screwed.
So really doing that, really making sure you have clear deadlines and shifting the work around so that people who will do the work and can do the work and have the time to do work are getting the work.
And the people who either just are doing the work or maybe don't have the time that week because that also happens, where one week will be really tough.
So you shift the work away from them.
So you have to be better about-- I think if I had started being better about how I gave assigned work rather than just evenly dividing it, that really would help with the project.
So working with a client was hard because he was unavailable a lot.
And the times he did come to class he said, if you have to email me a question, it better be really clear and concise, or we're not going to answer it.
And so luckily, in the beginning, we kind of knew what we wanted to do with our game.
And so we just went in that direction.
And then whenever Pablo came into class-- Pablo was our main client.
We also had Ellie, but she was only in class a couple times, and so we don't get as much feedback from her.
So whenever Pablo came into class, he told us what we needed to fix, and that was the biggest interaction with him that we had.
At first, we didn't really know who to design it for because they said, we need two cholera games for everybody.
And so we were really unsure and so finally, three or four weeks into the project, we talked to the other cholera team and said, what are you doing?
Because we're going to do the opposite.
Because they wanted two games that would be for different groups, and so then we did that.
We talked to Pablo the next time he came in, and he said, yes, this is what I want.
So luckily we managed to guess what he wanted.
It could have gone worse if we did something that he didn't want.
So if that had happened, and he said, you guys are really going in the wrong direction, I think we would have made sure to email him more.
But most of the time, he's like, I like the direction you're going.
You just need to fix these sorts of things, so that's what we did.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So welcome to Creating Video Games, CMS 611 and 6073.
Is everyone in the right class?
Yeah, all right.
Cool.
So this class is taught by-- it's a Comparative Media Studies course.
So we'll be talking a lot about the kinds of things that they talk about in Comparative Media Studies.
We'll study research and design practices, but as it relates to software engineering, and in particular to developing video games-- so digital video games for, basically, screens.
The CMS department has a lot of different, really cool research labs that are a part of it.
And we are one of them.
So we are the MIT Game Lab.
Basically the whole MIT Game Lab staff teaches this class together.
Because we really like each other, and it's really fun.
But also because there's a lot of you.
And it's a really big, complicated class to teach.
So we need all the extra support we can get.
We teach classes in game design and game research.
Those are all the course numbers that we teach other than this one.
Maybe you've taken one of these before.
Maybe you're thinking about some other game courses you want to take in the future.
So take a look at that.
We also have UROP opportunities.
So we'll advertise those classes as they come up.
We do mentor Course 6 students for the UAPs.
What we usually ask is for you to bring to us an interesting research topic that you want to research.
And if we are also interested, then cool.
A match is made.
Same thing when it comes to masters.
So if you're going to get your master's of engineering here in Course 6.
And you want to do it in game design or game development, and you need a mentor, we can be your mentor.
We also host a lot of events and outreach events and guest lectures.
A lot of the guest lectures will be part of this course-- that we'll be opening to the public, as well, for a few of them.
We've got Riot coming up in September.
We've got Japanese developer Swery65.
Does anybody play Deadly Premonition?
Yes!
It's awesome.
And he's coming to talk about it.
And some other cool stuff-- we also do a lot of events.
So like game jams-- we'll have an alumni panel.
And the next big thing is the Boston Festival of Indie Games is going to be on campus in 11 days.
So 200 developers are going to be showing off their games over in the Johnson Activity Center.
And we'll have a conference track going on here.
So take a look at that.
You'll get all that information as the course goes on.
So next up, we just wanted to introduce ourselves and our individual-- basically what we do, and why we're teaching the course
Hi.
I'm Phillip Tan.
I've been teaching various game courses here at MIT for about 10 years now.
I'm a research scientist over at the MIT Game Lab.
And I've worked with all of these fine folk for many years.
Up here are a couple of projects that I've worked on.
On the right is a game called A Slower Speed of Light.
If any of you are interested in relativistic speeds and simulating them on a computer, you are welcome to check it out.
We also have an open source version of our code that lives on GitHub.
So you can download that version using the same tools that you might end up using in this class.
But basically it's, you travel near the speed of light, and you see what this might look like.
I play a lot of StarCraft.
Anybody here play StarCraft?
A few people.
All right.
I've been working with Blizzard this past year to work on their spectator interface.
The idea is that there are a lot of people who watch e-sports right now, especially on Twitch.tv.
Who watches Twitch.tv?
OK. All right.
Lots of people.
E-sports specifically on Twitch.tv, or just like Let's Play?
OK. A few.
Those are spectators, just as if you're watching a football or hockey game on TV.
And they need infographics to come up in real time when something interesting is happening in the game.
So I write code together with UROPs to be able to bring up cool infographics bring the hype, basically.
So that's me
Hi.
I'm Sara Verrilli.
I'm the development director at the MIT Game Lab.
I came to the Game Lab out of industry, where I've worked as a producer and a game designer and originally, how I entered it was as a QA tester.
So I've held a lot of the jobs there.
I've worked with a lot of students on games when we were still running the summer program, the one we did with Singapore-MIT GAMBIT Game Lab.
And now I have focused to mostly seeming to work on classes and teaching game design.
So that's a lot of what I've been doing and what I've done
So I'm Rick.
I'm the studio manager for the Game Lab.
I teach this course.
I co-teach with Sara 617, our advanced game studio course where you spend the entire semester making one game.
These are the games I've worked on at the Game Lab.
I worked on all sorts of different games from anything to games about depression to games about identity, games that are demonstrating advanced 3D rendering techniques, and conference games-- PAX POX, a game that was played live action at PAX East in 2009 at our booth there.
So I do digital and non-digital game design.
But for the most part I'm studying and working on project management
Hi.
My name's Andrew Grant.
I'm the technical director at the MIT Game Lab and help out with the classes too.
I'll probably be helping you out a little bit with technical issues or what have you.
I was in the games industry.
I started with Sara at Looking Glass and moved to Dreamworks Interactive.
And after that, I kind of got burned out, to be honest.
It's a pretty rough industry sometimes.
I spent about 10 years consulting.
And I kept coming back to games and coming back to games.
And so when I got the opportunity to come to the Game Lab, I said, well, yes.
So anyway, here I am.
I've been playing and making games all my life.
And I think we'll have fun doing it this semester
And if you can stand up-- our TA is Paulina.
Wave and say hi.
She'll be helping with all the administrative stuff.
She'll be helping us with grading.
And she'll also be helping with any questions you might have about some of the technology that we're using, too.
She took the course this past last year.
Great.
So get down your pencils.
This is the cheat sheet, the one-line version of the class.
If you learn anything, you learn this.
When you're working in this class, the best way to succeed is working face to face, testing your games often, prioritize, integrate, and cutting features early.
Sleeping is awesome.
Avoid doing 3D.
Avoid doing network code.
And please use version control.
We're actually going to require all of this.
But every year, a team or two doesn't do one of these things.
And it's OK.
But they struggle.
So if you do all these things, through the semester, you'll do really, really well.
Yeah, please.
What was the question?
Slides.
Oh, the
Slides?
Yeah.
All the slides will be available on Stellar.
We're posting all of our slides to Stellar.
I don't actually expect you to take notes on all these things.
But it's awesome if you do.
I remember it that way
This is 140 characters
This was actually Phillip's tweet from last night, in case anybody was following him already
Then they were probably not sleeping
Actually, yeah.
So the next bit of administrivia, we're going to go over the syllabus a little bit.
It's in front of you right there.
If you have any questions, feel free to interrupt.
Fellow instructors, if I say something wrong, or if you want to say, hey, but also this, please do.
So first thing-- the course is structured into four game projects.
So you're going to be making four projects over the course of the semester.
Our first three projects are about two weeks each.
The third project is a little bit longer, if you decide to work over the student holiday.
And then project four is our final project, our main project.
And that project's going to be done for a client who will be coming in to talk about what he works on later today.
So your first project is a non-digital project.
It's a prototype.
Your second and third are small, short, digital prototypes-- so making a digital game very quickly.
Your fourth is a longer time period project.
But it should be about the same amount of features, the same amount of work, the same amount of game play as the first three.
It's just you had a lot more time to actually do it right, and to do it well, and to polish on it, and iterate on it.
At the end of today, we'll talk about what we mean by these common themes.
This common theme-- what did I say-- meaningful decision making, or meaningful decisions in games.
And each time we start a project-- our first project starts on Monday-- we'll introduce the theme in more detail and talk a little bit more about it.
But you can see all of the details about the projects that we have available for you at Stellar right now.
Our grading rubric for each project is largely team-focused.
So all of your projects are worked on in teams.
They start in small teams and get a bit larger.
You'll see each of these lines is 20% of the grade.
So the actual working of the game is only 20% of your grade.
Following the iterative design process that we'll be talking about, using the project management practices that we'll be talking about, doing a good presentation-- a group post-mortem presentation about your game-- is another 20%.
And then every student will write an individual post-mortem, their own one to two page essay, at the end of-- turned in at the same time as every project.
And what we're asking for all these things is we want to know about your process.
We want to know how you made the game, what challenges you had while you were making the game, what difficulties you faced.
How did you surpass them?
And even better, what are you going to do in the future?
When you either go into the industry or go into another software engineering project or another class, what would you do differently, after the experience you just had?
So 80% of your grade is team-delivered.
20% percent of your grade is your own work.
So speaking of in-class expectations, the class is a three-hour class.
Holy crap.
And it runs twice a week.
Class time is actually allotted for your work in teams.
We don't expect you to get your games actually built entirely within class.
But important team meetings, test sessions, code review-- you're going to generally have a lot of time in class to do a lot of that stuff.
For projects one and two, you'll probably have about 60 minutes at the most per session to work on your team work outside of an activity.
But we're also going to have a lot of activities where you're doing some of the work that's required in each project as part of a guided activity that we're going to run.
So it's actually a little bit more than 60 minutes.
Products three and four, we're probably going to have about 90 minutes per session for working in your teams.
And that's going to generally be at the end of class.
At the beginning of class, we're going to have, usually, a guest lecture for those last two projects.
And then we'll have a playtest session.
And then it'll just be open work.
Our original classroom had flat tables.
This classroom-- that moved.
This classroom does not.
So it'll be interesting.
I don't know if it's got power on any of the stations.
But the short version of it is, bring your development station to class, if you can.
If it's a laptop, bring it to class.
You'll be using it a lot starting Monday-- well actually, starting two Mondays from now.
And the other thing-- teamwork is the heart of this course.
Actually, participation is the heart of this course.
So even though we're in this tiered thing, and I hate giving lectures to people this way, we're going to ask you to talk back to us a little bit.
So fill in the gaps when you come in.
Participation is actually very important.
Class participation-- so showing up to class on time, showing up to class at all, and participating in class is 25% of the grade.
So speaking of that, attendance-- you will be penalized if you miss more than three classes without justification.
So what is that justification?
It's like if you're sick.
If you're sick, we don't want you here.
Stay home.
Don't make us sick.
Don't make your team sick.
He's a real stickler about that.
I'm invincible.
I won't get sick.
But everybody else will.
Do not be late.
Again, we're recording.
We have very important guest lecturers who are in our network that we'd really like them to enjoy being here.
So I really hate it when people show up in the middle of a guest lecture.
It makes me feel like a jerk for asking them to talk to MIT.
So don't show up late.
But do show up.
So if for whatever reason, the doors are locked, it really just means, hey, there's a guest lecturer here.
And coming in is going to disrupt things.
The doors will get unlocked.
I don't expect to have to use that.
But if I will, I will.
Ha ha.
And usually, guest lectures end about 2:10.
So you can come back.
Come into class later to work, and actually get counted for the rest of the day.
So I'd rather you come in later than at 1:15 while somebody's talking up here.
Class is generally going to start-- lectures are generally going to start at 1:10.
We'll do announcements at 1:05, if that makes a difference.
Because I know a lot of people are coming in from recitations and whatnot.
The other thing-- MIT has an anti-harassment policy.
We've also got an anti-harassment policy.
If anybody's on Twitter these days, or anybody's engaged in game culture these days, it's toxic.
And it sucks.
And it makes me mad I don't want to see it in this class, please.
So this is on the syllabus.
It's the last page of the syllabus.
Basically, don't make offensive verbal comments or any other kind of comments about all these different things, when it comes to gender identity, sexual orientation, disability, physical appearance, body size, race, religion.
Just don't be a jerk.
Don't be an ass.
More importantly, though, if you see it happening, let us know.
If you are a target of this, let us know.
There's ways to get around this.
There's ways to report this.
In the syllabus, it actually lists the website for the official MIT way to report it, if you'd like to go that way, if you'd rather not come to either the instructors or your TA.
And our email address-- videogames-bosses@mit.edu.
Email that.
Don't email us individually.
That makes sure that one of us reads it
Can I add just one more thing?
Yes, please
Rick put up a giant list of things that you should [?
worry about ?
], and that you should not be harassed about.
But I will go ahead and say that you should not be harassed about anything.
Right?
We have a list.
It's not meant to be exclusive.
So if there is for some reason some other things coming up and bothering you, come and talk to us
We are-- team development and game development is stressful.
Yeah, question
One phrase that I've heard recently that I love is, "Rights work up.
Rules work down."
You have a right to be respected.
The rules that are listed will be paper mached on to make that work.
But in general, be respectful
Yeah.
That's really what we're asking for.
Just be respectful for each other.
You're going to be working on large teams.
There are going to be deadlines.
It's going to hurt.
It's going to be painful.
Things are going to slip.
Don't let that be the way that you express it.
We'll actually give you some better tools to express your pain and suffering.
So the class is called Creating Video Games.
How do we create video games?
We have a whole bunch of people involved in creating video games.
Was this the slide you were coming in on?
Or is it the next one?
This is the slide I was going to chime in on.
But you were chiming so nicely
Come in whenever you like.
So basically, all sorts of people are required to make games.
We have all those people in the room
Actually, a whole lot of skills are required to make games.
You should not come in and think, I'm going to the programmer on this team.
You should not come in and think, I'm going to be the game designer on this team.
Or I'm going to be the artist on this team, or the sound designer on this team.
The teams are small enough, even when you hit the eight-person team for the large project.
And the projects are always challenging enough.
Even if you think you have a short list of things to do, you have a challenging project.
That's one thing we've noticed every single year.
Everybody manages to come up with a challenging project.
The team's going to need you doing everything you can do.
And that means that if you can program, they probably want you to program.
If they need programmers, chime up.
If you're primarily a programmer, or you think yourself of, they need you to chime up on the game design and comment on places where it will work better.
And you understand how it will work better, because of your understanding of the code.
If you're the game designer, you've got to be ready to pitch in and help make all those things you see in your head happen, whether that's creating some sketched doodled art, so your team can go with the little pixel figures, or whatever.
We don't-- usually these teams aren't big enough to support someone who does nothing but draw art.
We're not actually grading you on the beauty of your games.
I mean, yes, we want good, well-functioning, rounded games.
We want games that are relatively easy to use.
But you don't need super beautiful art for that.
So if what you'd like to do is primarily art for the game, think about what else you can be contributing to the game as well, so that your game gets more functionality in it.
And let's see.
For community managers, business analysts, and marketers-- the people who are pitching your game and talking about it-- don't mean on the person who ends up stepping up to organize your team and steps into the producer role.
They're already doing an awful lot of job just producing.
They're going to need your help figuring out presentations and talking about your game, and doing the reflection, and keeping the team running.
So don't think of these as a whole bunch of individuals.
Think of it as a whole bunch of skills, one of which may be your primary.
But choose at least two or three as your backup.
That's my shtick
Cool.
And that's basically how indie game development is run.
When you see a game made by three people, they're doing all of that.
And so that's basically what we're expecting from y'all.
So if we're not teaching a lot of the skills, the main skills that we are teaching in this class is giving you hands-on experience working on complex projects with large teams.
So what does that mean?
It's basically project management, team management, and communication skills.
You're coming to us through your prerequisites with some design skills.
You're coming to us with some programming skills.
You have support for some of the game engines you haven't seen before.
But like any other engineering course at MIT, you pick up a tool.
You learn how to use the tool in the course, if you need to use it.
So the tools that we've chosen should be short enough that you can be able to pick them up pretty quickly.
And actually, we're going to go into that today as well.
But the remainder of the course-- particularly project two and project three-- are going to be focused on communication.
If you're on a large team, how do you talk to each other?
How do you communicate with each other?
Is it going to be through email, instant message, chat, post-it notes, mash notes, hate notes, karaoke?
Bottlenecks are going to form in your work process.
You're going to find out that oh, all the assets are going to one person.
And they're sick today.
The quality of your code and assets are going to quickly nosedive if you don't have a plan
Yeah.
This is another one of those, don't assume that one person on your team is going to do all the art or all the sound.
And that's all they're going to do.
Because if they get hit by the flu truck-- as frequently happens around here-- you'll discover you've got nothing to put in the beautiful game that you've made
So to get around that, we're going to introduce to you tools and methods through the course that are going to help you get through these problems.
How to do code workflow, how to do asset workflow-- we'll talk about that a little bit.
That's actually the first assignment.
We'll talk about code review.
We're not necessarily teaching a lot of this.
We'll be talking about a lot of this.
I know some of you have done some of this in other experiences.
We actually want you to help other people out in the course.
Talk about your previous experiences working on a project.
If you've done an internship before, if you've made an app for a company before-- because everybody has apps these days-- you've probably run into some of this.
So we want to hear about your experiences, too.
We're going to talk a lot about product backlogs and task lists-- paper!-- using paper and spreadsheets to tell each other what you've been working on and how you're going to work on it.
I'll be talking a lot about retrospectives-- about how, at the end of a project, you can talk about what you've just done, and how you can do it quickly.
But ultimately, we can going to give you all the tools, all the methods.
The best way to learn all this, to know all this, is really just through prior experience.
So this class is going to be that prior experience for you when you go to your next challenge, either working in the industry, working doing any kind of software engineering, or going to another class.
So again, the cheat sheet-- work face to face.
Test often.
Prioritize, integrate, and cut features early.
Sleep.
Avoid 3D.
Don't do networking.
And use version control, please
And this is where the cut I have from the last three years of this course, and the final presentations from pretty much every group that's given a final presentation for their final project-- in which they look at us sheepishly and say, you said we should test early.
And you know what?
We really should have tested earlier.
I can't actually be 15 different MIT students who have taken this course saying this sheepishly up on the screen.
But I've heard it from all of them, pretty much every single group.
So I'm going to give you guys the really early heads-up warning.
Test earlier.
We say it.
We mean it
I'd like to introduce Pablo Suarez.
He and his group will be our client for project four.
So we're asking him to give an introduction of the kind of work that he does, and give a feeling for what the kind of work we're going to do in class.
The first three projects are going to build slowly towards that final project in the theme and strength of those games.
And in that fourth project, he and his group will come back in again and tell us more detail about what they're really looking for, and what kind of games they're really looking at from us.
So take it away
Thank you.
Thanks Rick.
Thanks Game Lab team.
I'm very grateful for this.
And you-- I am grateful to you, even though you don't have much of a choice that I show up and you have to suffer through me.
My name is Pablo, Pablo Suarez.
My strange accent comes from Argentina.
I live in the Boston area, where a fraction of my time I'm a researcher.
I used to be a researcher affiliated with MIT.
I just came from the Lincoln Labs, where I'm collaborating with MIT folks there as well.
But most of my time, I work with a humanitarian team-- the Red Cross/Red Crescent Climate Centre.
It's like a think tank within the humanitarian sector.
It's based in the Netherlands.
We cover the planet-- 187 countries.
I am in charge of research, in charge of innovation, mostly in charge of Africa.
And unfortunately for the world, we have way, way too much work.
The workload is so ridiculously huge that in my opinion, especially coming from science and technology, we need to change the way we think about humanitarian work.
And if it is about changing how we think, traditional ways are not working too well.
And that's why I'm here.
I love this slide.
We have a comfort zone, which is doing things the way we've been doing them.
Someone's hungry?
Go get food.
Someone's homeless?
Go give shelter.
Someone's sick?
Help them heal.
But we no longer have the money, the brain power, the equipment to deal with all that's happening.
Too much is happening.
The climate is changing.
Environmental degradation, urbanization, political instability, weird diseases-- you name it.
How can we learn more about what we can do?
We need as humanitarian workers to get into a new place where we can do more with what we have.
And that place is outside our comfort zone.
Now magic can happen.
But very often, we step out of our comfort zone and we fall into the abyss.
It's not easy to find the right place.
This is a risky endeavor for me.
I'm here hoping that you guys will accomplish one of two things.
One is maybe you come up with a game concept that has a life after the end of your course.
You get your credit.
You graduate.
And the Red Cross or others can play a game that helps someone understand something, helps someone do something, helps someone collect data about something.
And we'll talk more about what are those things that we want when we get to project four.
The other one is maybe we can lure you-- even if one of you, even if one of the Game Lab team-- to consider humanitarian work as part of what you may be doing in your future life, either as your job, or as your hobby.
It's great fun.
It's a great intellectual challenge.
The head of my team, Martin [?
Van Ahls-- ?]
he's an astrophysicist by training.
OK?
So you don't have much awareness of the extent to which humanitarian workers can come from unusual places.
And we hope that MIT stops being an unusual place for humanitarian stuff.
So let me illustrate the kinds of challenges we get.
This is an actual fax that actually arrived in May, 2008 to the office of my colleague Yusef [?
ayd-Shaloush, ?]
who at the time was based in Dakar.
Let me see if this works.
OK. For those of you who know Dakar, it's near those green pixels.
That's where, at the time, the regional office was for West and Central Africa.
28 countries going from Mauritania-- where there have been a military coup, and there was bloodshed in the streets and Red Cross volunteers needed to learn first aid to prevent someone bleeding to death-- all the way to Congo Brazzaville in the southernmost part of this area, where there was flooding-- urban flooding.
And as Red Cross, we don't fully know how to operate in rapidly changing urban contexts.
We're used to a planet where poor, suffering people were either in conflict or in rural areas.
And Darfur-- people going to Chad-- needing shelter, aid, all sorts of things.
Yusef has to deal with those three and everything in between-- chronic food insecurity, Burkina Faso, you name it-- insufficient training for his staff.
And he gets this fax.
Now I want you to think you're a decision maker.
You're someone who has to make decisions and gets this fax.
It was issued in May, referring to the three coming months-- June, July, August.
And it says if we define an extreme precipitation event-- extreme rainfall-- as being in the top 15% of the historical record, then this year-- there was a lot of text.
I won't bother you with that.
This year, because of sea surface temperature anomalies and other things that maybe you know if you're an MIT student, but it's not what we're used to-- this year, because of that stuff, the probability of extreme rains is enhanced from 15% to between 40% and 50%.
So this fax is in the office.
And Yusef looks at it.
And what does he do?
What do you think he does?
What do you think the humanitarian sector does when we get information?
First of all, to understand it fully you need to stop doing everything else you're doing.
You have to not answer an email.
You have to not sign the check for the fuel for the driver for delivering the something or other.
And once you start thinking about it, what do I do with the place with the green pixel?
Do I send-- I don't know, one tent?
A thousand tents?
Where do I send them, and to do what?
What are the chances?
When will it happen?
So my job is to help the Red Cross family and the humanitarian sector at large to understand science that can help us make smarter decisions.
Now if you want to represent, to model, with an experience of forecast, this is how it normally works.
There's information that allegedly can support action.
That is communicated with an audience in a passive mode, like a fax or an email or a TV or radio announcement.
And then you hope that people will get it and do the right thing.
Of course, I leave a question mark, because-- not always?
Now when I talked to people about climate science-- about changing climate conditions; sea levels rise, so we need new locations of shelters; about El Nino and changing risks of drought and food insecurity; about rainfall upstream, therefore flooding downstream-- I would be talking science.
And the people I was talking to-- they had something else to do.
They had to go save lives.
And they were forced into a room just like you now.
And I was talking science.
And I could feel the extent to which I was successfully shrinking their brains.
This is a metaphor.
There are very talented and smart people in the humanitarian sector.
But my putting them into a passive mode, talking jargon to them, showing maps and graphs and so on, was really not accomplishing much.
So I had to try something you.
Oh, by the way, I remember hearing the snoring while I was talking science.
So it was shocking.
Because they were wasting their time listening to me unsuccessfully try to communicate information that can help save lives.
So I started doing other things.
And we're going to get a five minute flavor of the kind of things we're doing.
So let's play.
You're a team.
Split in half.
Each one of you choose, are you on this half or this half?
You're a team.
You're a team.
You're a team-- four teams.
There will be one winning team.
The winning team is the team that has the most people standing by the end of the game.
We'll have a few practice rounds, and then we'll play for real.
OK?
You are not who you think you are.
You're not some MIT student or something.
You are magically, in the magic circle of this game, a Red Cross disaster manager.
OK?
Good.
Someone is celebrating.
You're hired.
We have no money, but you're hired.
So there are three kinds of things you can do.
And everyone will be standing when the game begins.
One is to go like this.
This means I hope there's good rains.
I'm going to take advantage of this opportunity to do good things when there's good rains.
Train people.
Change the tires of the vehicle.
Build a shelter.
Write a proposal for donors.
Write a report to the donor that gave me money last year.
All those things, you can do when it's a nice day.
Now maybe there's too much rain coming.
Then you can invest in preparedness against too much rain.
You make this gesture like an umbrella.
And it means you're preparing against the risk of too much rain.
This could be prepositioning relief items, like tents, for those who have to be displaced with the flood.
But it's better to do it before the flood.
Because if you wait for the flood, then the bridge has been washed away.
And you have no way to take tents to the other side.
The other option you have is to do a bucket-- something to hold water from the sky.
If there's a risk of a drought, you want to collect water before the drought so people can have water to drink.
So each one of you can do three things.
Good rains, you go like this.
Too much rain, you go like that.
Too little rain, you go like this.
What determines rains?
The probability distribution function of precipitation based on the past record.
So you guys are affected by this PDF, this die.
You guys are affected by this one.
A six is what?
Too much rain-- it's a good idea to be like this.
If you're not like this, you sit down.
You got it wrong.
A one is what?
Too little rain-- it's a good idea to be like that.
Otherwise, you got it wrong.
You sit down.
Everything in between is what?
Good rains.
You better go like this, or you got it wrong.
Practice round-- everyone stand up.
Because the winner is going to be-- well, let me show this.
Losing players [?
all ?]
get it wrong.
You sit down.
You're basically-- not fired, but people suffer because of your bad decisions.
And the winning team is the team that, at the end of the few rounds-- I will arbitrarily declare the end, because in the real world, there is no end.
We'll have three practice rounds, and then for real.
Most people standing-- so if you guys have two people standing and you have 10 people standing and they have less than 10, you are the winners.
And the winning team will win a publication.
I will explain how you'll have to fight over it.
So let's go back to-- actually, here.
So that you remember the three things you can do.
Each one of you has to make a decision.
Two, three, four, or five, it's a good idea to do this.
Six-- good idea to do that.
One-- good idea to do that.
You can have a collective conversation.
But each one of you has to be making one of those three.
If you're making none of those three, you sit down.
You're fired.
You have 30 seconds to consult with your teams.
[SIDE CONVERSATIONS]
The rains are coming.
Make your decisions.
10, nine, eight, seven, six, five, four, three.
When I say stop, you have to be in the position.
Three, two, one, and-- be in the position or you're fired-- stop!
OK.
Here we see a lot of thumbs up, two buckets, two umbrellas.
Roughly the same kind of configuration-- all right.
Let's see what happens to this half.
Camera, I wish you good luck.
This is for you guys.
It is a five.
So if you're like that, stay standing.
Otherwise, sit down.
You didn't invest in the good things you could have done.
For you guys, let's see what happens.
And it is a four.
If you're like that, stay standing.
Otherwise, sit down.
Moving on to your two-- practice round.
What are you going to do?
Those who are sitting, stay sitting, but can give advice.
OK?
Make your decisions.
Five, four, three, two, one, and, stop.
Lots of thumbs up.
Are you with these guys?
All right.
So you're the only umbrella on this side.
Let's see how it happens.
There we go.
There's A four.
Thumbs up, stay standing.
Otherwise, sit down.
This team not surprisingly is weak.
They started with few.
Over here, you guys-- there we go.
A two.
So if you're like that or like that, sit down.
Now, the last practice round.
You receive this fax.
That's all you know.
You got these fax
Are we using the same dice?
I am answering your question by saying, all you got is this.
There's a deliberate scarcity of information.
You have 30 seconds to make your decisions
Wait, where are we?
You are in the place with the green pixel.
[SIDE CONVERSATIONS]
10 seconds left.
Five, four, three, two, one, and, stop!
OK.
So in this team, there's only three left.
One third are investing in too much rain.
Here we have about eight or so.
Half or more are investing in this.
No one-- oh, only one person is going for drought.
Over here-- all right.
I see over here we have three.
So about half, also, investing in flood protection.
Over here we have more than before, two out of six.
Well, of course, what this says is that on a normal year the chances of this is 15%, or about one in six.
One in six is 16%.
But this year, because of unusual conditions, it's a flip of the coin.
So for you guys, I'm going to flip this one.
Of course, you cannot now change your decision, right?
It's too late now.
You either sent the tents or bought the cement for the shelter.
So I'm going to throw this up in the air.
If it falls like this, no problem.
It's like a three.
If it falls like this, too much rain-- 50% chance or so.
It's a good idea to have the umbrella.
Otherwise you sit down.
If it falls like this, meaning it stops moving and it stays standing, it's a good idea to have a bucket.
You guys will get the yellow one.
Ready, set, and go
Yes!
Flood.
Someone's very happy.
We love it.
So if you're not like this, you have to sit down.
Now notice this interesting feature of reality.
If you take the right action, an extreme event is not a disaster.
He was prepared.
He's happy that a disaster happened.
Because he was prepared.
Which is a good thing.
A disaster is only a disaster if it leads to negative outcomes.
If someone tells you it's going to be 110 degrees today, and you have some health condition, you turn on the air conditioning.
There's no disaster.
Unless there's no energy for your air conditioner.
OK. Let's see about you guys.
You also got a flood.
So if you're like this, stay standing.
Otherwise, sit down.
If this were the end of the real game, we'd have three standing here.
No other team has three.
So you would be declared the winning team.
Oh, you got four!
So my apologies.
You would be declared the winning team.
Congratulations.
Now.
[APPLAUSE]
Thank you.
Everyone sit down.
And now we're going to see what you would do for real in this scenario where we have climate change.
Have you heard of climate change?
Things are changing.
This is based on the past record of precipitation.
But the past no longer explains the present.
Things are changing.
So I hereby introduce you to climate change.
This one is going to be for you guys.
I'm going to throw this up in the air with some spin, just like the dice.
Look at you.
I love this face.
This is a very productive silence.
I'm going to throw it up in the air.
if it falls like this, it means normal rains.
No problem.
It's like a three-- a good idea to go like that.
If it falls like this, too much rain.
It's a good idea to invest in your umbrella
Oh I get it
If it falls like that, what is it representing?
It's too little rain-- bucket-- good idea to go like this.
This is for you guys.
We're going to imagine we play three consecutive years.
Can you hold this for me?
No spinning, please.
No throwing.
For the other half, we have this other one, which is also climate change.
But the climate doesn't change in the same way in every place.
So I'm going to throw this up in the air.
If it falls like this, it means no problem.
Good idea to go for thumbs up.
If it falls like this, too much rain.
If it falls like this, too little rain.
Go for that.
As a team, you have two minutes to come up with a decision-- or more likely, one minute.
Go.
[SIDE CONVERSATIONS]
Less than a minute.
30 seconds.
10 seconds.
Everybody stand up.
Make your decisions.
Five, four, three, two, one, and, stop!
All right.
Compared to the first round with the die, there's much more proportion of people investing either in too much rain or in too little rain.
Can you notice any difference between this half and that half?
Not really.
Very substantial, phenomenally different climate change projections coming from science-- this could be like the IPCC report, right?
And yet, you don't know how to interpret those differences.
Thank you for incarnating what happens in the real world.
OK?
We just are very confused to understand that the risks are higher.
So I will not roll it.
Because I would rather keep sharing with you some of the things we were doing.
But I can share with you that you already have-- no, I want to know!
Let's play.
I want to see what the climate will bring.
And that's exactly the point.
We are creating your appetite for learning.
And that's what games can help me and my colleagues do.
We have a bunch of [?
reckless ?]
people, and suddenly they want to know the real climate projection for their project site.
All right.
Everyone can sit down.
So we went through this-- oops, sorry.
This is where I wanted to go.
So that's the fax that Yusef received.
You played, and you had many more people going like that-- which is the right thing to do.
Now it's 50-50.
It could have been that it doesn't flood.
But if you understand the risks, you can say to the face of your staff, to the face of your donor, or to the face of the communities you're helping, look.
I had reasons to act that way.
Because it went from one in six to one in two.
And games can help process that.
What games can do is to help embed information into a system where decisions have consequences, and shape of the information that becomes available for the next turn.
So you can have more or less resources and so on.
Yes
Yeah.
I was wondering.
If you have a 50-50 chances, right, and let's say it turns out you made the right choice this year, why not invest more into better measurement [?
increments?
?]
Exactly!
Why not invest more in better everything?
We can.
But we don't have the money.
Right?
So we're talking to Lincoln Lab and donors and so on to help us improve our ability to sense what is going on in terms of hazards-- including too much rain, too little rain, and other things-- including vulnerabilities-- it's not the same for me to be exposed to 100 degrees of heat wave versus to an elderly person with asthma.
Right?
And it's the same with capacities.
What do we have, in terms of skills and knowledge, in terms of equipment, in terms of experience?
The disease chikungunya-- has anyone heard of it?
So what are you going to do if an outbreak of chikungunya comes out to MIT tomorrow?
Right?
Knowing can help.
Sensing can help you make better decisions.
In short, in my experience, only games can do this.
Only games can get a bunch of people together, and in a very short time understand the role of information in the context of decisions that will have consequences-- including lifesaving or life-losing consequences.
This is an example-- I'm going to skip it, so I can get to the end.
But basically, games can also help you collect data and compare one scenario versus another, and see which are the things, the aspects of the real world, where people are failing to get something.
And so you can target your efforts in capacity-building training, et cetera.
This is one of the things I will talk to you about.
I will want you want to help us design games or playful activities that can help people make their decisions based on forecast, as opposed to always doing as if it were the normal die.
And then if there's a flood, as the forecast said was more likely, you go, eh, I don't know.
I wasn't prepared.
Which is what happens quite frequently.
There are many advantages.
One is that we get better performance in terms of outcomes-- fewer sick people.
Better efficiency-- so you save more per unit money or equipment or et cetera.
Flexibility-- people can do more if they understand the forecast.
If you know that Katrina is about to hit your village or your city, there's more you can do in anticipation, as opposed to waiting.
And once it slams, there's very few things you can do at that point.
Because if you step out of your door, you're in trouble.
Games can create the appetite, through confusion.
With the cone, you were like, huh?
We deliberately create systems that push people to the tolerable edge of confusion.
So that they want to figure out, to make sense of what's going on.
And then participants themselves-- individually or in conversation with peers, they go like, huh.
So if we do this and this happens, I'll be better off than if I do this other thing.
The aha moment-- the epiphany, the revelation-- is constructed by the players.
And that has much more lasting effect than if I give you the answer.
No one remembers the answer if no one was asking the question.
Very few examples-- these cones are the result of a collaboration with the World Bank Chief Economist for Sustainable Development.
He was trying to explain a confusing concept called deep uncertainty.
[?
Jana ?]
over there-- can you wave, [?
Jana?
?]
She's my partner in game design, and facilitation, and life.
She had the brilliant idea of taking-- our cone had surgery.
Sorry.
Our cat had surgery.
And she thought, how about using the cone of shame-- you know, for the creature not to lick its surgery-- she said, how about using that for depicting this uncertainty?
Now we had students tossing each one of those cones 100 times to see the probability solution, with and without spin, and augmenting the gap between what you think will happen and what actually happens.
A few other cases-- this is from Hanoi University of Science and Technology.
These was with 350 students playing a game on dengue.
It's like a rock paper scissors, where three players are mosquitoes.
And they can either bite, or with your blood, lay eggs.
And the other players are humans.
And they can either protect from biting or attack the breeding grounds.
This was designed by students at game design students at [?
Parsons ?]
and professors collaborating with the other students.
This one is a publication from NASA.
It doesn't get more NASA than a rocket launch.
But if you look here, it says, Erin Coughlan-- former intern and now staff in my team.
And the Red Cross/Red Crescent Climate Centre project.
You go to page 47, and you see [?
Munu.
?]
[?
Munu ?]
is a volunteer of the Zambia Red Cross, from the village of Kazungula in Western Zambia on the Zambezi River.
I mean, there's not much electricity or NASA-ness in Kazungula.
But he was trained as a facilitator for a game where, with cards and dice, there's rain that happens over there.
So her cup gets some water.
And then from her cup, she passes the water to the person next to her, representing the upstream going downstream.
She can get more water or not.
And it keeps going until maybe he gets water to overflow his cup.
He's in trouble, unless he takes some action ahead of time.
And the game we're designing is not only with cups and to mimic the upstream/downstream.
But we're also making a digital game where people actually monitor river levels.
And they make predictions.
They make bets as to what the river level will be in two days or so.
And then the winner wins points.
But they can use points to buy information from upstream.
And then with that data we're developing a hydrological model that can help us better know what is going down the stream, and are we going to have more flood trouble?
As you can imagine, it's awesome to have NASA report on some little project in Western Zambia, in part because we know that we can partner these on-the-field game-based data collection with satellite image for ground truthing and so on.
This one-- because we are being recorded, I will not disclose the full story.
But in short, we play the game that had the same story.
You play with a 6-sided die.
You know your probabilities.
It was being played with the top-level climate scientists in this planet.
And as you can see on the left, there's all sorts of calculations with probability distributions and the cost benefits and so on.
Then we change the probabilities on them.
And then we gave them that.
Just like you were saying, what is that?
What are you going to use?
We just gave them that graph and the caption.
And some of them were saying, but wait a second.
Why are you giving me this information?
It's completely useless in the context of my decision situation.
And then we told them, this is the information that you yourselves produced in the summary for policymakers of climate science stuff.
And they became fully aware of the need for them to improve the way they communicate climate science.
This-- you may recognize that dude.
I think it's the last time I wore a necktie-- at the White House.
I was given five minutes.
I begged for seven.
Seven minutes given, preceded by extensive PowerPoint presentations or other forms of uni-directional, so I had an audience that had brain wave activity declining.
And I said hello.
My name is Pablo.
And this is not a Frisbee.
It's a hurricane.
Everybody stand up.
And people were somewhat confused.
Some were standing.
Others were distracted, and they saw people standing, so they stood.
And I said, OK.
I'm going to throw this frisbee in some direction.
It's a hurricane.
If it hits you in the chest, it's trouble.
You're suffering.
Now if you sit down, it will fly above your head, no problem.
It's like evacuation.
But if you sit down and evacuate, and the hurricane goes in some other direction, someone may still your refrigerator.
Right?
You're doing the wrong thing.
So where it will go will be determined by these two dice.
So you either say standing or sit down to avoid it.
And I throw the things.
And I look.
And I throw in this direction and in this direction.
People were being hit.
There was huge confusion.
No one understood what was going on.
And then someone got it.
Double one, I throw over here.
Double six, I throw over here.
Interpolate.
So someone figured it out.
They said, oh, a nine!
It will go in that direction.
Everybody sat down.
And I threw the frisbee.
And it flew elegantly until it hit the wall of the White House.
And then I said, well, you know, climate science can give us this.
Think Katrina.
Think all that we could do.
We're here to help.
Thank you.
And that game was memorable.
It opened doors.
And it created awareness of the value of linking climate science with decisions of the humanitarian sector, of the government sector, and beyond.
So game play beats PowerPoint in a million ways.
For me, as someone who has to help others take actions that save lives, I'd say first of all, there's active learning.
When you were playing, and you were confronted with that fax, you were saying, what is that?
What does it mean?
And what should I do about it?
It's about action.
And there's plenty of peer to peer-- no, but look!
If it's 50%-- and people learn from each other.
There are plenty of "aha" moments.
It's fun, but it's serious.
And the fact that it's fun is important.
Neuroscientists, including some in this building-- I think Satra works here-- have evidence that if you are emotionally engaged, your learning goes deeper and lasts longer.
There is also the ability to collect data.
I didn't go into it.
But that's one of the things I'd be keen on using with you, if you are able to make a digital game that can collect data and help us understand, what do people know?
What do people think?
What do people want?
And then we can use it as an optimization platform.
How much should we do?
How many people should stand up next time we have that forecast?
It's like a Monte Carlo simulation, right?
You do things many times.
And you figure out, what are the things that you call optimal, based on objective or subjective metrics?
That's the website for my team-- climatecentre.org.
Note the British spelling.
This is at the core-- why we love games.
There's an experiential learning.
That's from our conversation with your colleagues at the Lincoln Lab.
You want to do things, reflect what would happen, form abstract concepts-- maybe I should do this or that-- test it, and do it again in a better way.
And it's much safer to do this in game play than to do it in the real world, where if you fail, you can fail soundly.
These are some of the partners we have worked with, including UN system, academic organizations, Rockefeller Foundation, and many other partners like Oxfam America and so on.
It's been delightful.
We have severe shortage of human capacity to make this happen.
We have Jana, who will likely come as a resource person to interact with you when I'm traveling, which will be a lot over the coming months.
There may be other colleagues who live in New York, who may come.
We also have Willow.
Willow, would you like to wave your hands?
Willow is very likely to become a consultant for my team going to Nairobi to help people use digital devices to map risks at the community level.
Maybe there's a way to infuse it with games.
She already developed games for Tanzanian communities.
So in short, I wish you all the best-- learning a lot from these guys, doing a lot based on what you learn, and your own motivations.
And whether you like it or not, project four will involve doing something for us.
Thank you in advance for accepting those rules.
And there shall be more.
Thank you very much.
Oh and by the way, I will leave this with you guys.
It's our publication-- a shared endeavor of my Red Cross team and Boston University Pardee Center.
It's called, "Games for a New Climate-- Experiencing the Complexity of Future Risks."
I promise you it's fun to read.
And it's available online for free as a pdf
I'm going to talk about game engines.
And before I do, if you want to think a little bit about one of the things that Pablo was saying.
It's about, one of the things that a game does is it forces you to look at what happened, and sort of say, huh?
You figure out why did that happen.
And then you try to fix it.
And that's actually one of the things I think is very much like programming.
If you're trying to write code, and something goes wrong, often that's what the process of debugging is like.
You go, why did the screen flip upside down?
That just makes no sense.
And then you have the aha moment.
And you can go and fix it.
Turns out that making software is a lot like playing a game.
So anyway, we're going to talk about game engine selection.
Because one of your early tasks will be to sort of try out a game engine.
We're not going to be using a game engine for the first game project.
But we want you to be ready for it when it comes to project two.
Now the first thing to realize is that digital games are basically a software project.
They share a lot of things in common with software in general.
You're going to have a UI that your user has to interact with.
There's going to be something going on in the background.
For a commercial or business offer, it might be a database or something.
There's probably going to be either a design you're working from.
Or you're making it up.
Or maybe you've got a customer who cares about what's going on.
You've got features.
You've got bugs.
We've got task lists.
All of the same sort of managementy things that happen, happen on software projects and game projects the same way.
There's a little bit of a difference, though.
Games are a little bit harder in some ways, and easier in others.
But we're going to talk about the ways that they're harder, because that's more interesting.
The thing that happens with most business software-- or even something like Photoshop.
Have any of you used Photoshop?
Excellent.
Photoshop is one of those software packages that-- a lot of people like to take classes in Photoshop before they get started.
If you sat down to Photoshop, you probably didn't think to yourself, wow, this is easy.
And if you did, wow.
But basically, Photoshop has hundreds, or perhaps even thousands, of features in there that you don't know about until you've taken the classes.
And if you want to figure out something about Photoshop, your best bet, really, is to go to a search prompt.
And go to your favorite search engine to find out what to do.
Or ask someone who knows.
Because actually trying to figure out Photoshop from first principles is really, really, really hard.
The thing is that with a game, you can't do that.
You can't get away with that.
If your game is complex, no one's going to take a training class to learn how to play your game.
Now it might be the case that someone will look at a walkthrough or a cheat sheet or something to play your game-- after they've played for three hours and decided they liked it.
But they're not going to do that first.
That's a general, broad, sweeping generalization.
There are some people that like nothing more than to read the manual for a game.
They mostly died out about 10 years ago.
But there are still some hanging on.
Generally speaking, nowadays, you're going to play the game, and then you're going to decide if you're going to read the instructions or not.
And this is going to be true for your games as well.
So your UI task-- your user interaction piece for your game-- is going to be hard.
And the only way to make sure that works properly is to test it over and over again, as early as possible.
So that's one way in which games are going to be a little bit harder than normal software.
Also, games have to be fun.
And we use the word fun kind of loosely.
And we're always scared of using the word fun.
Because there are serious games.
There are games that are supposed to make you think instead of get a big smile on your face.
But what they have to be is engaging, or interesting.
Photoshop's not a great example.
If you're playing with Photoshop as an undergrad, you're probably thinking to yourself, this is kind of fun.
But if you're using a fancy piece of software for work, you're not doing that because it's fun.
You're probably doing it because someone's paying you to do it.
The games-- not so much.
People are going to play it because they want to play it, and for really no other reason.
So your game has to be fun.
And what you think is fun they may not think is fun.
So the only way to make sure that your game is fun is to find some users really quick, and try out your game on them as soon as you possibly can.
The same thing happens with your game play difficulty.
By the time you're done with your game, you're going to think the game was too easy.
I guarantee it.
And everyone who plays it for the first time will think it's too hard.
I guarantee that, too.
So you have to test it with other users, not yourselves.
Because you will think your game is too easy.
And you'll be wrong.
And then, of course, finally, your game may have an impact you want to have.
And I'm going to keep saying this over again.
You're going to have to test your game.
Remember, there's a theme that's going to keep going on here.
And we're going to use this theme with absolutely everything we do.
When you're playing a game, you try something.
It doesn't work.
You think about it.
You try something new.
When you're developing a game, you're going to try something.
It's not going to work.
You try something new.
When you're trying to make your team work smoothly together, you're going to try something.
And that's not going to work.
And you'll try something new.
We're going to talk about that circle over and over and over again in the class.
Because it works.
So moving on, we're going to talk about game engines.
And if I talk to quickly, as I have a habit of doing-- even more so when I'm talking to people-- let me know.
Tell me to slow down.
So this is the game you want to make.
It's beautiful.
This is the tools you have to make it.
[LAUGHTER]
So obviously, what you're really going to do is going to be somewhere in between these two extremes.
And you have to think really hard about how you're going to accomplish it.
The vast majority of game developers, especially new ones, are going to come into it thinking, this is the awesome, perfect thing I want to create.
And then they're going to come to their tools.
And they're going to discover that it's really hard to do.
And so I encourage you-- rather than thinking of the big vision of what you want to make, and then figuring out how to get there, I recommend thinking about what you've got.
And then figure out, what can you make?
Now these are a couple things you can do here.
One, reduce in your mind the scope of the thing you're going to make.
You're not going to make your favorite RPG.
You're not going to make an MMO.
If you try, we will laugh at you.
You will occasionally think about this top row.
How do I get the awesome thing?
But I really want to encourage you instead to think, as much as you possibly can, this is what I know how to do.
These are the tools I have.
What awesome thing can I do with these tools?
Rather than, how do I get over there when I'm way over here right now?
This is a quote from a discussion I had with a game team once where we were talking about a particular technical problem.
And I said, how hard is that?
And the developer says, well, it's really hard because of this, that, and the other thing.
I said, wait.
Stop.
Actually I didn't mean that.
I don't care how hard it is.
What I want to know is, how long is it going to take?
And we're going to say that a lot in this class, too.
We're going to say, how hard is something?
And often what we really mean is, how long is it going to take?
Because we don't care if it takes an hour of really hard thinking, to some extent.
What we care is that it takes an hour.
If it's going to take you an hour to do an easy thing-- like sort a bunch of yellow stickers and a bunch of orange stickers because that's part of your game prototype.
If it takes an hour to do that, that's an hour.
It doesn't matter that that's an easy task.
That's time.
And you have to be careful how you spend your time in this class-- actually, through all your classes at MIT, I suspect.
But in this class, we know that you don't have enough time to get done every thing that you want to get done.
And so you have to think very carefully about how you spend your time.
It turns out that game development is already hard.
You don't need to add hard stuff to it.
So a lot of what we're going to focus on with choosing a game engine and all the decisions you make is you want to write as little code as possible.
And that's not because we're afraid of code, but because every line of code you write is time.
Every 10 lines of code you write is a bug.
[LAUGHTER]
Right?
So what we want to do is minimize the number of lines of code you write.
We want to minimize the number of bugs you write.
Because when it gets to be the bug stage, that takes even more time-- to find it, to fix it, to write new code to fix it.
So we want to reduce as much as possible all of that time.
So we want to use all the tools we can to reduce the amount of code we're going to write.
One of our big ones is we use what we call paper prototyping.
We don't actually mean paper.
We mean whatever materials you can use-- chalk, pieces of plastic, dice, frisbees.
Anything you can use to make your game without writing a line of code, you should try it.
Test it that way.
Because it's so much faster to test it that way.
When else do we have up here?
Iterative design-- I've talked about this before.
We're going to use the phrase iterative design to talk about that whole cycle where you try something, see how it worked or didn't work, change it, try it again.
That's iteration.
And we're going to talk about that a huge, huge amount.
And one of the big parts of this, of course, is game engines.
Which is not to say that you're going to build your game engine.
But rather, you're going to try stuff.
And the game engines will let you iterate faster, do more with less code, and get more done.
All right.
So again, you're going to use a game engine to save yourself some time.
You don't want to reinvent the wheel.
I'm confident that you could write a game engine if you wanted to.
But I don't think it's worth your time.
Someone else has already done it.
They may or may not have done a better job than you can do.
But you can bet that the thing that you're going to get from your game engine will be faster than if you stopped and wrote it from scratch.
And then finally, the game engine that you write from scratch-- for every 10 lines of code, you've got a bug.
I'm making that number up.
But it sounds good.
That means that a game engine you write has bugs.
Now I'm not going to claim that the game engines that we're going to suggest you use don't have bugs.
They totally have bugs.
But what we have is 10,000 monkeys on the internet, using that game engine, reporting the bugs.
And there have been a lot of cycles of people fixing bugs.
And so hopefully, the worst ones are out of your way.
The other thing you can do with a game engine is you can use it as you tinker toys, as we talked about before.
Your game engine has a list of things it does well.
Great.
Use those things as tools.
Take advantage of that.
Do everything you can with those tools.
And if it can't do something, don't fight it.
Don't say, game engine, darn you, if only did this thing.
Think, how can we change our design to not need that?
That's another advantage you can use, another trick you can play.
If you limit your design, sometimes it increases your creativity.
And it can actually help you get your game done faster.
So throughout this class, I use this phrase.
I'm not sure anyone else does.
But I consider certain things to be nooses.
There are some things we will tell you you cannot do, or that we tell you not to do, flat out.
And those things-- don't do them.
OK?
But there are some things that we're going to say, we recommend that you don't do that.
Or, you know, no team has tried that and succeeded yet.
Those I consider nooses.
You can try them.
And you can put your head in there and see how comfortable it is.
But probably by the end of the class, you'll say, wow, you know what?
When you said not to do multiplayer networked play, you were right.
We have heard that every year.
We will hear it this year, I suspect.
But again, we're not telling you to never try crazy things.
But merely be aware of what the crazy things are.
If we're suggesting that you not try something, but not telling you you can't do it, and we're looking kind of pained when we say it, be aware that this is a huge risk for your project.
Be aware that this is a place where you should be putting a lot of early attention to make sure that you have a back-up plan if it fails.
All right.
On we go.
Now we're actually going to talk a little bit about the thing we're supposed to talk about.
Picking a game engine is super important.
Everything involved in your game relies on it.
But again, contradictorily, don't worry about it.
Because you have four projects.
And the first one doesn't count for game engines, because you're not using a game engine.
But for two and three, I'd say experiment.
Try some stuff out.
By the time you hit project four, you should have some opinions about what game engine works well for you.
And try to get there.
Try to use the one you're comfortable with by then.
But experiment on the first two.
No game engine's going to be perfect.
They all have flaws.
They all have advantages.
And your job is to find the game engine whose advantages you like and whose disadvantages you can live with.
And that will be different for different people.
All right.
Here we go.
So the most important things we're talking about for game engines-- this is not an issue for this class.
We're only recommending free game engines.
But in general, if you're trying to pick a game engine or a software of any kind to help you, these are the three categories of cost.
I call things free if you don't care about the cost.
If it's $20, it's practically free.
But if it's $1,000 and you're a college student, that's not free.
That's painful.
And if you're corporation, however, $1,000 is free.
So these categories change.
For us, we're talking about free.
Impossible is CryENGINE or something brutal where you get the source license and do everything.
But we're not going to do that.
So this is the first one that actually matters.
It's does your game have requirements that the game engine has to meet?
And for most part, in this class, this also won't be true.
Because you can choose your requirements based on the game engine you're choosing.
And our games are pretty small scope.
But basically, if you need a special input, if you want to publish to iOS, if you need to support network play-- which you don't, remember-- then that might affect which game engine you choose.
This one is actually my biggest important thing.
Whatever you choose for your game engine-- and we're actually going to give you a short list.
So it's not quite the open free-for-all you might be worried about.
It should be easy to learn.
And there's a couple reasons for that.
But the main reason is simply the amount of time you've got.
We don't have a lot of time.
You don't want to spend time mastering a game engine.
You want to spend time mastering the art of creating games.
So you want to pick something that's well-documented, that's got a good online community that's going to help you out.
Even if you don't ask the questions, in a big enough online community, someone's already asked the question you have.
I suspect you're all familiar with this point.
We can do a search for the question you have on something.
And some forum post pops up where someone else has already asked the question you have.
And you just read the answers and think, yay!
There was an answer.
Or probably, there wasn't an answer.
I'll look at a couple more of those.
And so you can get an idea for your game engine as to how robust the community is, how good the support is, by sort of looking and seeing how much support there is online for it.
One thing on in-house experts-- it's very common for a team to come together and say, well, we all know this game engine a little bit.
But there's this one person on our team who's just an expert in this other one.
So we're going to rely on that person to really teach us how to do it.
That only, only works if that person is really willing to be a teacher.
And it's not a bad thing if they're not.
But don't choose their engine based on their expertise if they're not willing to really be a teacher.
And what I mean by that is that that person will have to be willing to not write code a lot of the time.
Because their time is better spent teaching everyone else how to write code.
And that really means that if you find yourself in that situation, where you think that you are the expert in a particular engine, and you're trying to get your team to use it, really think about yourself and how you're feeling about that.
Make sure that you are willing to not dive in and write the code.
Make sure that you're willing to help everyone else do it instead.
Because that is going to leverage more of your team.
On a small enough team, that's not true.
So on your two- or three-person team, you can probably get away with that expert.
On an eight-person team where five of you are coding something, you really can't lean on that one person that heavily.
You're going to need to spread out the knowledge as best you can.
And so some game engines are easier to learn than others.
It's not just a matter of the online support.
But we'll get to that in a little bit.
On a team programming project-- and this is a personal bias of mine.
And I admit that some people disagree.
But I personally think that a strongly typed programming language is a must.
If you're talking about five to eight programmers, there's an additional problem.
Whenever I write a function, and you're going to call my function, you have to know what the arguments are to know what's going to happen.
And a strongly typed language will enforce that contract.
It will make sure that we're on the same page, at least a little bit, as to what that function is doing.
A lot of you are probably used to-- MIT teaches, in a lot of the programming classes, it uses Python, which is awesome for quick work.
And it's awesome for a single person doing work.
But it does kind of start to fall apart on a larger team, when you have to read someone else's code to know what their function is doing, what the arguments are supposed to be.
If instead, you have a strongly typed from the beginning, the compiler will enforce this communication.
And we're going to talk a lot about teams and communication.
It turns out your code is one of your methods of communication.
The actual written text in your code is a way you talk to your team members.
And a strongly typed language sort of forces you to keep that communication line clearer than a weakly typed language will.
And again, this is one of those things where programmers might have battles over whether or not it's true.
But I'm pretty confident of this one.
I would say it's a noose to use a weakly typed language.
A lot of our game engines are going to be talking about C#, or ActionScript or whatever.
And I'm going to tell you that if you know one of those languages-- if you already know Java, for example-- it's really easy to transition to C#.
It's pretty easy to transition to ActionScript.
ActionScript and Haxe are very similar.
So I would say any of those top four, you can transfer back and fourth pretty easily.
JavaScript, kind of.
But don't try to transition to C++ from those quickly.
I mean obviously, you should someday learn C++ if you are considering yourself a Course 6 major or whatever.
But don't assume you could pick up C++ in a weekend because you already know Java.
They're just gotchas in C++ that don't exist in those languages.
And other important note, if you know Java-- [LAUGHTER]
It really looks like you know JavaScript.
And you kinda do.
But you don't.
And that's not to say that you shouldn't make that transition.
But be aware that it's going to take a little bit longer than you think.
There's a lot of synthetic sharing.
But JavaScript is a weakly typed language that has a lot of weird, very web-specific features.
And just be aware that that's not as easy a transition.
I talked about ease of understanding and learning.
Actually using the engine is also very important.
How are you going to get stuff done in it?
If the engine has an interactive debugger, that is wonderful.
If it doesn't, then you're relying mostly on print statements.
And that's hard.
If you've never really had the joy of using a debugger before, I recommend you try to learn.
Because it's really an extremely powerful tool.
We're not going to get into that in detail right now.
And source-control-friendly is another important one.
If you're on a team with eight people, you're all going to be making changes.
We're going to talk more about source control in the future.
We're not going to go into details on that now.
But basically, it's a way for your team the team to communicate and make sure you're always in the same, up-to-date source code for your project.
And if your engine doesn't support that, then you can be in trouble.
In the past, for example, people have talked about some of the cool, sort of hobbyist game engines, like GameMaker, or back in the day, just bare Flash.
And those are really cool tools.
But they would save the entire project as one monolithic binary file.
And so if one person makes a change, no one else can change the code while they're doing that.
Because there's no way to merge our changes.
Whereas with code or text files, or a good game engine that separates things out a little bit, one person can change an art asset while someone changes a code file, while someone else changes yet another code file.
And it all merges together pretty seamlessly.
And that's what you're looking for, if possible, on a team project.
I talked about this before, but you really want a product that's been around a little bit.
It's cool as a hobbyist thing or a solo product to take on something really brand new.
But I think for a class where you're trying to get stuff done really, really fast, and you're working with a team, you really don't want to go too far in the direction of the new, latest, and greatest thing.
Because you want to have a game engine that people have put some miles on.
They've tested it and shaken out some of the bugs before you get to it.
All right.
Those are things that I consider-- you've got to have them for a game engine.
Here's some stuff that's kind of nice.
Yeah, it's nice to be able to easily import images and sounds and stuff like that.
It's nice to be able to get the source code to the engine.
But it's not essential if the engine's well documented.
Let's see.
It's nice if you have an integrated development environment that can help you with auto-completion of various functions and really help you along with your development.
Definitely nice, but not necessary.
And then finally, an editor of some kind-- if I want to lay out my levels in some way, is there a graphical editor for me to do that?
That's certainly nice.
And a profiler that would tell me where my game is slow-- that's also nice.
But without these things, you could probably get by.
Bells and whistles-- this is actually relatively unimportant, especially for a small project, prototype-style stuff that we're going to be talking about.
For the most part, you don't need all the fancy stuff like particle systems and shaders and physics and all that stuff.
You might use physics in this class.
But be careful.
So that's one of the things.
You don't need that stuff in your game engine for most small games, at least in the scope of this class.
These are things that I really don't care about-- scripting languages that your designer might use.
Why is your designer not writing code in your strongly typed language?
I never understood that one.
Some designers are afraid of writing code.
But hopefully in this class, you're at least willing to try writing some code.
And then finally, just because someone has made a beautiful game in a game engine, that has no bearing on whether or not your game will be beautiful in that game engine.
The vast majority of the time, a game built in a game engine is beautiful because they have a really good art team.
And we don't have a really good art team here.
We don't have the time to do that.
And we don't have the resources to do it.
So you shouldn't measure your game up against the beautiful AAA games.
You should be measuring your games up against some indie, quick, prototype type of games.
All right, so Flixel is the thing we've been recommending for many, many years.
This is the game engine that is easiest, has the quickest learning curve I've seen.
It gets everything done.
It's very quick to learn, to use.
And it runs on the web.
Great.
That's pretty good easy stuff.
It was developed by Adam Atomic for a game called Canabalt, that you can see up there.
The game that, as far as I know, sort of popularized the infinite runner genre, which we've seen clone after clone after clone of.
But I believe this is at least where it entered my consciousness.
And therefore, it must be where it came into the world, right?
But anyway.
The cool thing about it is it's got a very, very, very simple object-oriented state-based system.
It's really, really good for 2D sprite stuff.
Last year, we had a bunch of game engines for people to try.
By the end, pretty much everyone just used Flixel.
Because it was easier to use.
And this kind of says it all to me.
Even if you haven't used ActionScript, the learning curve on Flixel is awesome.
You'll spend a day or two staring at it.
And then you'll know most of everything there is to know.
And you'll just be able to do stuff.
You won't be able to do all the cool, awesome stuff other game engines will let you do.
But you'll be able to do pretty much everything you'll need to do on the scale and scope of games you're looking at in this class.
Let's see.
It doesn't so good at GUI.
If you wanted a fancy GUI-- lists and lots of text and buttons and sliders and stuff-- it's kind of a pain to code that up.
Right.
So Flixel is also unfortunately, in its current form, kind of in an end of life cycle.
And I've been saying that for two or three years now.
And it's still holding on.
Because Flash is still holding on.
Adobe is trying to sort of get out of the business of maintaining Flash as a web development place.
And I'm not really sure why.
Apparently, Adobe decided a while back that HTML5 was the future.
But it's been four or five years now, and HTML5 is still not the future.
In the meantime, we're sort of hanging on with Flash and the occasional JavaScript HTML5 thing.
I suspect it was actually more of a financial thing than any actual HTML5 preference thing.
But anyway, we're trying to get out of Flixel a little bit in the long term, because Flash is going to be a dying platform pretty soon
Can I just add something to that?
Please.
Stand near me.
Because I have a microphone
All right.
I'll stand right next to him.
So the mic can pick up.
One thing to keep in mind-- that are free versions of the Adobe compiler for Flash, called Flex, that you can download for Linux, Mac, and PC.
However, there is also this licensed version, which you can download for a 30-day trial, called Flash Builder.
Unfortunately for Mac users in particular, if you want to get those, I use it.
I like writing code in it.
But as soon as that 30-day free trial is over, you are on an annual subscription paying Adobe money every year.
So "free" with an asterisk.
If you want to do free development on a Mac and use Flixel, be very, very careful.
And use this upcoming exercise to actually ensure that the things that you're downloading and the things that you're signing up for are actually free, and not free for the first 30 days.
For PC users, you have the advantage of using FlashDevelop, I believe.
That's what it's called.
That is actually a tool that you get without having to pay for it.
For Linux, you are pretty much stuck to using command-line compilers.
And if you really want to do free development on a Mac, you're also using command-line But there's all kinds of clever things that you can do with text editors to sort of make that a little bit more comfortable.
But just keep that in mind.
That's another reason why Flixel isn't our number one choice, nowadays, is because Adobe's really trying to squeeze money out of everybody, including students.
Adobe's insane
One can't blame them for trying to make a little bit of money.
However, it's no longer as attractive as it used to be.
That's certainly true.
Unity is the next one on our list that we've had good success with.
It's a free, full-featured game engine.
It's got all the bells and whistles you could want.
It's a little bit dangerous to use it, because it has all the bells and whistles you could want.
And you should probably only use some bells, and not the whistles, and whatever.
You should be very careful and judicious in your use of the features.
Because the tinker toy model of, only do what the engine lets you do is a little bit dangerous.
Because it lets you do pretty much everything.
So you have to be very careful about which features you choose to use.
However, it's great.
It's free.
It does a whole bunch of neat stuff.
There are paid versions.
But you won't need them for the class, certainly.
I call these disads.
3D is harder than 2D.
And 3D is at least three times as hard as 2D, really.
It's rough.
If you've never done 3D before, it takes a while to get your head into it.
Even if you're doing a 2D game in Unity, sometimes you have to think in 3D a little bit, which makes it a little bit harder to use.
On the other hand, Unity does have this nice editor.
And it's possible for a non-programmer to get some stuff done in Unity in a way that's not possible in others.
You spend more time learning the GUI.
But you can actually get some more things done.
So it's an interesting little trade-off there.
It is also, again, not good at GUI work.
They're starting to address that a little bit.
But it's hard.
You have to kind of hand code your buttons a little bit sometimes.
That is improving, as I say.
But it's not quite there yet.
It suffers a little bit from the problem of merging.
Unity uses particular objects that you really wish you could merge-- just so they're text files or code-- but you have a whole scene of objects, for example.
And only one person can really work with it at a time, unless you're really very careful.
So when you're using Unity, you have to be extra careful with your team to make sure that people aren't messing with the same files at the same time.
The code, however, works the way you want it to.
And source control works fine.
So you've got to be careful.
But you can work around it.
Moving on to the ones that-- you know, we want some of you to practice with these.
But-- oops, that's the wrong one.
I left a slide in I shouldn't have.
I will ignore that.
Let me see if I can find it here.
All right.
Apparently I did not include the slides I thought I included about the two other game engines.
So I'll talk about those instead.
So there's another engine call Haxe Flixel, which is sort of a port of Flixel to a language called Haxe.
This is an engine that we have not used yet.
We've looked over it a little bit.
And it certainly looks very promising.
It's using the very same Flixel style.
It is, however, using an interesting language named Haxe, which is mostly based on ActionScript.
But it has some Java and C#-like features to it.
It does some crazy stuff, though.
So it might be a little bit interesting to use.
Haxe compiles down to a language of your choice, which it then compiles.
So for example, if you want to export your Haxe to Flash, you'll compile your Haxe to ActionScript, which will then get compiled to a Flash app.
Or if you want to make an iOS app, you'll probably compile it to the [?
XCode ?]
Objective-C, and then compile that.
But basically, it tries to be all things to all people.
Which is really kind of dangerous.
But hopefully, for the kind of smallish project we're talking about, that'll be find.
I call that one a noose, because we haven't used it much.
But it has all the simplicity of Flixel for the Haxe Flixel game engine.
So that is a good thing.
The other one we're looking at using this year is called Phaser, which we know not too much about in this lab.
The Education Arcade is using it.
And they seemed to like it.
And it is an ActionScript, HTML5 style of engine.
That is not the most mature platform to build on, because of all the browser differences and stuff like that.
But it is certainly possible.
I recommend if you end up using Phaser that rather than coding in pure JavaScript, you code in TrueScript, which is basically a strongly typed version of JavaScript.
We'll get on to the rest of it
TrueScript?
TypeScript?
Sorry TypeScript.
Yes, TypeScript.
So when you're learning an engine, you start with tutorials.
That's the easiest thing to do.
Once you've learned three or four engines, you might not use tutorials.
But until then, I would say you should actually go through something that someone's written and thought about how to progress.
Do something extremely small.
Don't start with the be-all and end-all game of awesomeness.
Start with something very, very easy.
And then after you've done that, you're going to discover that you've done some things.
Then read the documentation a little bit more.
Because you're going to be better at picking up stuff after you have questions in your head.
Even if you don't realize what questions you've got.
And we're warming up to an assignment where we're going to hand you a game engine and you're going to take a look at it over the weekend, or over the week.
But the thing you need to remember is that all software really does suck.
And game engines are no exception.
At this point, invariably, somebody will claim that Linux doesn't suck.
To which I'll have to claim that it does.
But it's also awesome.
And that's the thing about software and game engines.
You have to really understand and appreciate the ins and outs of Linux to understand the ways in which it really does suck, but there are ways in which it's really also really awesome.
Operating systems all, really, are pretty bad, I suspect.
Windows and Mac-- you can make huge arguments about all of them.
Photoshop we talked about earlier.
It's an awesomely powerful tool.
But to sit down and actually start using it as a beginner, as a newbie, is a miserable experience.
I think you've probably all had very similar experiences with software.
It's buggy.
It's painful.
This is true for game engines, too.
You're going to try out a game engine.
And you're going to probably come back to class thinking, I tried Unity.
It was terrible.
I didn't understand what was going on.
It was really hard.
And then you'll hear someone else talk about Flixel.
And you'll think, wow, I didn't have any of those problems.
And so this is the trick.
All software sucks.
And your job is to find the software that sucks in ways that you can handle.
Right?
You don't mind so much that you have to think in 3D for Unity, because you're willing to put up with that because you get these cool, nice, rendered look.
Or maybe you have to deal with the fact that, yeah, Flixel's written in ActionScript 3.
And ugh, I hate that language.
But boy, it sure is easy to put stuff up on the screen, and have spaceships moving around shooting each other in, like, a day, with no problem.
And these are the kinds of trade-offs you're going to have to look at.
So when you're trying to evaluate a game engine, or any piece of software, really, but game engines for this class, you're going to think about the ways in which it's terrible.
But also think about the ways in which it's great.
And when you get together and talk with a team for projects two through four about what game engine to use, you're going to want to be able to say intelligently, I don't like Flixel because of these things.
But actually I can live with those things.
Or, I don't like Unity because of this thing, and I really just can't get past it.
And if your team's insisting on using that engine, and you really know you really don't like it, consider another team.
And that's not bad.
Different people are going to have different preferences for different styles.
And that's part of what we're going to be doing.
Any time you're going to be forming a team or trying to work with a team, you're going to have to figure out what works for you, what's hard for you, what's easy for you.
And match yourself up with tasks or software that matches your set of skills.
Or, if you want to challenge yourself, OK. Be ready to challenge yourself and try something that you thought was too hard or too painful.
But be aware that you're doing that.
Because you need to let your team know, hey, I'm going to be on a learning curve here.
You're going to have to help me out.
So as you look at a game engine, be aware that it is going to disappoint you.
That's just the way it is.
But try to figure out where the disappointments are.
To that end, one of our assignments-- yes
Which 2D game engine would you suggest for those who are comfortable with Java and C#?
With Java and C#?
It kind of depends on the situation.
I would actually still suggest, for most simple-- which game engine should I use for 2D?
And the answer is, if you have no 3D experience whatsoever, I would still recommend Flixel today.
If you are comfortable with 3D and some of the barriers there, I would go with Unity.
I personally use Unity for my own development.
That said, what we're going to do is everybody's going to take a look at a game engine.
So you'll have at least one of the four that we're talking about for the class that you're familiar with.
And hopefully in the next couple of weeks, you'll get your own preference.
Because I can easily say, sure.
Flixel's great for 2D games.
And it is, for the right kind of 2D games.
It's not good for all the 2D games.
An isometric thing with lots of farms and stuff-- you might be able to do it.
But you might be more comfortable using a 3D approach or whatever.
So in other words, it really depends on the game, too.
I wish I could give you an easy answer.
But I can't, which is why the whole lecture.
So we have an assignment, which we'll hand out to you now.
And we're going to talk about it one week from now.
And do stay tuned.
Because we're not done talking at you, or giving assignments
Do we hand these out, too?
Oh, the game engine tutorial?
Yes.
What I want you to do is pick a game engine out of this hat.
And pass it on.
If you do not like the game engine that you get, trade.
Um, sure.
Feel free.
Take a handful.
If I run out, I have more tokens.
But this is what I start with, just to make sure there's a good distribution.
Just go ahead and hand this back.
I don't need to be with it.
So one of the things we're going to do next Wednesday is try to do a source control exercise, which will introduce you to source control, and hopefully refresh some of you.
So ideally, if you can, we want you to bring in a development machine, a laptop if that's what you're using, on that Wednesday.
If you only have a desktop, don't.
If you've got no computer, don't.
But you know, try to bring in a development machine of some kind that you can actually use on Wednesday.
Ideally, it's the machine you used to do the tutorials, so you have your code on it.
Let's see.
What else do we want to say there?
We don't want you to spend more than four hours on this exercise.
There is no "turn in" for this exercise.
What we want is, on Wednesday, we're going to be able to talk about the game engine that you looked at.
We want to be able to talk about its pluses and its minuses, and what was frustrating.
If it took you all four hours just to install the stupid thing in the first place, that's information.
We want to know that.
Because that will help people choose to not use that game engine, for example.
But basically, we want to be able to talk intelligently in the groups, amongst yourselves, about the game engine pros and cons.
That's the main goal.
OK. Any questions or comments so far?
[INAUDIBLE] can't just pick just any one of those [INAUDIBLE]
Pardon?
So we can only trade.
But once you have the [?
others ?]
you can't [INAUDIBLE]
Right.
Correct.
Do not pick a game engine at random.
You may trade the one you've got in your hand with someone else who has a different one in their hand.
This is not just-- it's partially arbitrary.
But mostly it's to make sure that all the engines are covered about equally.
Also, there's two entries for Unity.
One is sort of thinking about it in 3D, and one is thinking about it 2D.
And because Unity has recently added sort of a 2D layer on top of it that you can sort of use.
It's a sprite-based engine.
So we want you to put that through its paces, too, to compare to the other 2D engines, to Java.
[INTERPOSING VOICES]
So I've never had much of a problem transitioning between the two.
[SIDE CONVERSATIONS]
Let's see.
Yeah, I would trade if you could
So, like we said, this is a video game development or video game production class.
It's about team management, project management.
We're spending a lot of time on those kind of topics.
We aren't spending a ton of time talking about design in class.
But we were jerks.
And we're giving you a really, really hard design constraint to work with for product four.
So we're going to have some optional assignments, optional reading-- basically things that we're going to list on Stellar, and a couple lectures we're going to give-- basically just kind of opening your minds towards design.
So you can think through some of the big, hairy problems when it comes to designing some of these things.
The first thing is talking about the theme for the course for the semester.
So we're talking about meaningful decisions in games.
So we're asking you, when you're creating these four projects, to think about what decisions your players are making during the game at all times, and to make sure that all those decisions they make are meaningful.
And this is going to be the foundation for the four projects that you're working on.
All the constraints are going to be based on this kind of thing.
So first off, what do we mean by meaningful decisions?
This is what I mean when I say it.
There's other definitions out there.
But this is the bare minimum.
It should do this.
It should have this.
So three things-- any decision a player makes in a game has an effect on the game's overall state.
So I make a decision.
Something changed.
Not only did something change, but that change ripples through the systems within the game.
It affects the entire game state, the entire game world-- not just what the player might have, not just the player's performance, not just the player's abilities.
So a jump in these games-- it can be meaningful in some games.
But for this particular game and this particular use, if the jump affects everything else in the game in some form, that was really meaningful.
And I'd be surprised to see a jump do that.
But it probably isn't.
It's probably just a way to get over an obstacle.
So there's something else.
There's some other kind of decision a player has to make, rather than just that one small decision.
Third-- there should be sufficient feedback-- and we'll talk about feedback later on and what exactly we mean by that-- that lets the player be aware that they made that decision.
Not only did I make a decision, but I knew that I made a decision.
There was something that happened in the game that prompted me to make that decision.
And I actively made the decision.
I made the choice to make the decision.
I know I made the decision.
And I actually see all those effects happen in some form.
At the bare minimum, I make a decision.
I see some short-term effects.
There might be long-term effects that I don't see right now, but I will have the opportunity to see later.
That's a little bit more advanced.
So basically this lecture is really just going to be talking about some example games that, when we're talking about the games we would like you to make in class, these are kind of the exemplar games for this.
These are the games that we think of when we think of this kind of thing.
And the problem that I think is going to come up is we're going to sometimes call them strategy games, which is a really loaded term.
What is a strategy game?
Does anybody have a definition for what a strategy game is?
Somebody who took-- yeah
[INAUDIBLE]
Yeah, a game that's going to require a lot of thinking.
A game that's going to require-- and by thinking, there's a lot of actions that happen when we talk about player thinking.
Like all right, so I'm thinking.
So if I'm thinking, I might not be taking action.
So I might be thinking to take an action.
I might be planning.
That's going to be part of it.
I might be thinking.
But there's something that's distracting me for thinking.
Maybe I have to make actions really, really fast.
Or I can make actions and I can think a long period of time.
All that is the big wide spectrum of what we're talking about with strategy games.
Can anybody list some strategy games they've played or have heard about?
Age of Empires
Age of Empires-- which one?
The second one
Yes it is.
[LAUGHTER] Civilization.
Yes.
Which one?
OK.
I like that one, too.
[LAUGHTER] More.
What's that?
No?
StarCraft
StarCraft.
Which one?
I've only played
I played I.
But I haven't played II.
But he plays II.
A lot.
More
Fire Emblem
Fire Emblem-- ooh, that's a really good one.
Which one?
All of them, yeah.
Yeah
Go
Go.
Ooh.
You took the board game class
I play MOBAs
MOBAs-- so which ones?
What's a MOBA, really?
Like, [?
Noldor ?]
or League of Legends
So what does MOBA stand for, for people who don't play it?
Massive Online Battle Arena
Massive Online Battle Arena.
And you were talking about League of Legends.
How many people here-- raise your hand if you played League of Legends.
OK. Riot's coming September 24 to talk about to us about that.
How many people have played DotA, Defense of the Ancients?
The same people, for the most part.
Cool.
So that's a really good spread.
And actually, that's exactly what I expected-- except for the Go person-- to hear, are games about war, and games about conflict, and games about fighting against another player, or against a computer-controlled opponent.
That's not exactly what we mean when we talk about strategy games for the broader aspect of the class.
But it's a great place to start.
And that's where we're going to start.
So here's Civilization V, and here's StarCraft II.
I'm going to use these for a couple examples.
And then we're going to split off.
And each of us are going to talk about some of our favorite games that we think of when we say, this was a really cool mechanic.
This is a really cool thing I've seen in a strategy game, or an aspect of strategy or an aspect of tactics that I've seen, and I'd love to see happen in some of the game we make over the next semester.
So there's two types of decision-making that I see.
And again, this is all just me in my head a couple days ago putting a lecture together.
It's not an academic taxonomy.
It's to help us out when we're thinking about the kinds of decisions we're making in these games.
There's two kinds of decisions that happen in these games.
There's a micro-level decision-- short-term decisions I make that help me out right now.
There's macro decisions or long-term decisions.
And I'm using micro and macro because I'm obsessed with economics, even though I don't understand it.
But also because you see the games represented that way sometimes.
There's the micro, where I'm looking really tight at this one aspect of the game, and the macro, where I can see a bigger, expanded version of the world.
I can see the systems and the gears that are happening behind the world.
And they're represented in many ways.
And I'll talk about a couple examples there
People often use "tactics" to describe micro, and "strategy" to describe macro.
So there's actually a difference between strategy and tactics.
That's one way of thinking about it
Yeah.
So for the micro-- exactly, tactical.
The way this is often represented in these kind of war games or strategy games-- there's usually some kind of spatial component going on.
There's some kind of placement going on.
There might be some direct conflict happening.
So I've got an army.
And it's facing off another army.
And we're throwing spears or shooting each other with lasers-- basically the same thing.
But there's also firefighting.
There's another kind of conflict that happens.
And by firefighting, I don't mean fires.
But I've got a problem that just happened.
And I need to solve it.
I need to react to it.
So it's reactive.
I might have this really long long-term strategy.
But unfortunately I've got to make some decisions right now that may or may not hurt my long-term strategy.
There are some tactical decisions I need to make.
It's not always reactive, though.
Sometimes what we're talking about is you're executing a plan.
I've set up a long-term strategy.
And now here's the final dramatic moment where I get to actually implement the thing and see if it works-- throw the dudes on the map, and they're going to die or not.
And when we're talking about tactical, there's always a short-term resolution.
So I do a thing-- feedback.
I know it happened.
There could be long-term ramifications.
But there's definitely a short-term resolution to the decision I just made at that point.
So my example, because I am a huge Civilization nerd-- Civilization V brought some very tactical game play to the series that wasn't really there in previous installments.
So in the past, you could stack units on a tile.
So Civilization is basically a game made of tiles where you're placing items on tiles.
And the tiles give you bonuses or whatnot.
That's the very, very basic version of it.
In this case, what we have-- and it's kind of blurry-- but you have a city in the center, where that arrow is pointing to.
You have these blue units surrounding the city, and a couple small yellow units going on.
Can anybody describe what's going on in this scene?
A raid
What's that?
A raid
A raid.
So we're fighting.
What kind of decisions is a player being asked to do right now?
And the player is the blue player that has the red arrow there
[INAUDIBLE]
What's that?
[INAUDIBLE]
So yeah.
You're trying to implement the strategy.
Can you kind of see-- apologies if you can't make out the quality of the slide.
Can you see what is exactly going on?
Yeah
They're about to make a ranged attack with the cannon against the city
Yeah.
So first thing is, all right, so I've got a range attack that is really diagonally away from the city.
It's two hexes away.
I can do a range attack on it.
The cool thing about this is that the city can't attack me right now.
Well, actually it can.
But I'm making a mistake there.
But it's not the knight's turn.
So the knight can't attack the ranged attack right now.
The other thing that's happening is the player had a long-term strategy, which meant that they brought some foot soldiers in to surround the city, protecting the ranged attacker.
They've also got this thing called Zone of Control going on in this.
All right, here's an enemy person.
Here's an enemy person.
Here's an enemy person.
If any of these people try to attack any of these blue people, there's some kind of negative modifier going on.
Because the blue player has set it up such that they're being flanked.
So there's some spatial placement going on that's been decided.
Not only did they have to get all the units over here, but once they were here, they had to put those units in the right places for this tactical maneuver to happen.
So really, in Civilization V, we're talking about tactical in one aspect of it.
In the combat aspect of it, we're talking about flanking, army positioning, Zone of Control, preventing other players from entering into tiles, controlling areas, and making this area basically more effective for the blue player to attack the city than for the yellow player to defend it.
So how did the player get there?
How did blue get into that situation?
They had an operational or strategic plan that they set forth.
They were able to develop those weapons that they had.
They were able to make them in the right location.
And then they were able to transport them over to the right location.
So if we're talking about military strategy, tactics, that might be closer to logistics, supply chain, getting supply into the area.
The strategy that might actually be happening is even bigger than that.
So this is where it gets into the realm that I really nerd out in is spreadsheets of numbers and modifiers.
There's resources you need to manage.
I need to anticipate what's going to be there and have the right things built in time for them to be useful in that situation.
There's an economic system I need to manipulate and do well with in order to afford the units that I want over there.
If I'm playing well, I need to be proactive in my strategy.
All strategy really should be proactive.
And someone can argue with me if there is such a thing as reactive strategy.
But really, it could be proactive and wrong.
But you did it beforehand.
You set up the gears in motion for it to play out the way it did.
And you're always planning for some long-term future.
So in Civilization V, one way that this plays out non-combat is city placement.
Where I put my city is incredibly important.
Using the real world map of Africa, what they've done is the designers put these resources-- those circular items-- spices, bananas, silk, diamonds, oil.
Some of the resources are luxury resources.
So they help my cities be happy.
They help my citizens work well and be productive.
There's agricultural resources that help my cities grow and be healthy.
And then there's strategic resources-- and I could be mixing these terms up.
They're things that are going to help me build things.
So oil or steel or iron will help me build better units.
So my strategy here-- there's a little bit of tactics going on, where when I actually get my settler in the right place, I need to then find a very specific place to go.
But long-term, if I'm thinking about the long term, I want to place my cities in a way where they're near resources.
They're near my friendly cities.
They have room to grow.
Because if I group my cities too close to each other, they're not going to be as efficient.
This is really apparent if anybody's played Civilization II, where there's one real strategy to the game.
And that's just build a city every three hexes.
That's just how it works.
And in the later games, they tried to introduce some nuance to that.
And again, I'm mixing strategy and tactics as we're talking about this.
They all go hand in hand.
A lot of the games that we've played, and that are these strategy games that you talked about, are actually more tactical games.
There's much more emphasis placed on the short-term thing I do now than the long term.
That's not to say there isn't long term in these games.
If anybody's played League of Legends, the metagame about the pick process, so what champ to pick or not.
That's a strategic maneuver.
You're planning for a long-term strategy that you're going to play out within the game.
And not only are you planning for it.
You picked it, and then you're stuck with it.
So those decisions last.
With StarCraft II it's build order, which can be a little bit more flexible.
There are build orders that you can-- that's build order in that you are building units and buildings and upgrades in the right order, such that you're prepared for multiple different strategies coming into them.
And I could be butchering that.
I'm not a
That's definitely a good [INAUDIBLE]
Cool.
But that's more flexible.
Because I actually have time within the game to make some tactical changes to that long-term strategy that I decided on early.
Unless you're playing professionally.
And then I've just seem that they just GG after five minutes.
So anybody recognize this quote?
Sid Meier, the developer for Civilization, famously said, "a good game is a series of interesting choices."
It's very much the case for the games that we're making in this class.
The thing I want to note is strive for interesting choices in your games.
Be prepared for your choices not to be interesting.
Try to iterate through them and make them more interesting.
But we don't have a ton of time to really help you make the greatest games in the world.
Right?
So we want your choices to be meaningful.
And if one or two of them are interesting, awesome.
You did a really, really good job with the game development process of it.
I'm going to do a really, really quick talk about how you can make things interesting.
And we're going to come back to it in later projects.
But I just want to kind of get these two terms out there.
We're going to talk about conflict and tension.
So what is conflict?
If I say the words conflict and tension, what's the definitions you're more familiar with?
Anybody heard it in drama or writing?
Like the narrative arc-- there's a conflict.
There are two people who want the same thing.
Or there's an internal conflict.
And that's really what I'm thinking of when I'm saying conflict in this case.
It's, what is the conflict in the player as they're making these decisions?
What are the trade-offs they're making?
Do I send an army this way?
Or do I send an army that way?
Or do I not fight at all?
If that is a really hard decision for the player to make-- if they can't calculate it beforehand, if they have enough information to know that one may be better than the other, but they're not quite sure-- that's what we're talking about with conflict.
With what Pablo was playing with us, the conflict in the individual teams of, we know the probabilities.
We can calculate that.
But are we sure?
Are we sure it's going to play out that way?
I really wish he had used to cones.
Because I really want to know how often the cone's going to fall on its end.
And then tension are ways we can modify that conflict.
Maybe there's no conflict existing.
And we need to use tension to create some conflict.
Maybe there's a little bit of conflict going on, and tension's going to amplify that and make it more dramatic.
And there's just some basic things you can change.
And when we talk about the example games, and if you play these example games over the weekend, think about these terms.
If I were to change the timing in a game, or the speed with which you can make decisions, or the precision that's required to make those decisions-- would that increase the conflict?
And by increasing the conflict there, are the choices more interesting?
Is the game more engaging, or the game more fun?
Limitations are awesome in games.
Limitations are really how you can design and control where and what a player can do.
So how many options do you give the player?
Do you give them two, or three, or five, or infinite?
I can tell you right now.
Having infinite choices is really, really difficult, but not that interesting.
Space, costs, economics, information, fog of war-- we've seen in some of these strategy games where you can only see things that are around where your units.
And you can't see things outside of that.
These are all methods we use as designers to create conflict, to make those choices that we're asking our players to make interesting.
So example games-- what I'd love for you to do-- and this is totally optional.
But if you're interested in design, I totally recommend it.
Over the weekend, play some games.
We're going to talk about these games.
And think about these things.
And the slides are on Stellar.
Think about the types of decisions you can make, the timing of the decisions, whether the game is single-player or multiplayer, if there's competitive or co-op play, if there's teams or alliances, and what kind of conflict and tension exists in the game.
All of these are things that we're going to be asking you as designers over the semester to decide on, to design, to craft, to create a very particular experience that you're trying to create.
So Drew's going to kick us off with this game
Who's played backgammon?
Wow, really?
No one plays backgammon anymore.
All right.
Backgammon is a pretty classic game.
It was invented a good, oh, 5,000 years ago or so, about the time that we learned how to write.
It's got a lot of planning, oddly enough.
There's a lot of probability and randomness to it.
But you can plan for that randomness by setting your pieces in the right spots.
You can choose different strategies.
Are you going to be aggressive to take the other person's pieces?
Or are you going to be defensive, and just try to protect yourself?
Are you going to leave something in the opponent's backfield, so if you're losing at the end, you can maybe mess them up somehow?
Or when do you change from trying to mess them up to shooting for the end goal?
There's a couple different ways to play backgammon, oddly enough.
And it actually is a fairly deep game.
There's a lot of probability involved, but some tactics and some strategy going on in there, too.
If you haven't played it and you like board games, it's well worth playing.
Because I think a lot of the things that you're going to see in modern board games really owe a lot to this game right here, especially when you add randomness as a mechanic.
Go and chess are obviously really, really awesome games, too.
But they're a very different style of game.
Because they don't have the randomness in there.
It's a very, very different way of thinking about it.
This is another one where I think there's more planning than people give it credit for.
Who's played Twister?
That's more like it.
OK.
So in Twister you actually can plan ahead.
You probably don't.
But if you played Twister a lot, you might start.
You might realize, if I put my right foot there I'm really screwed if they call red next time.
So I guess I won't.
I'll put it over there.
It kind of depends on how many people are playing.
If there's enough people playing, you can't do that.
You just have to be fast.
So Twister is an interesting game where sometimes you have to make your decision extremely quickly.
Is there strategy in Twister?
Anyone?
I see some nods.
What decisions would you make?
Maybe to go under somebody, if you have to reach and lift them off the board
All right.
Yes.
So there's a strategy to Twister if you want to play Twister to win, darn it.
And maybe you're thinking that I've got better balance.
Or maybe I'm a little bit bigger than they are.
I bet I can nudge them a little bit.
That would work.
Right?
That does work.
Or maybe you want to play off in your own little corner?
That works, too, unless no one else lets you do it.
Or if you get yourself too tangled up, you can't move even when you're by yourself.
Have you done that or seen that?
Someone's off by themselves doing Twister and they fall over with no one near them?
It totally happens.
There's other strategic decisions for Twister.
Are you trying to win?
Are you trying to stay near someone, or away from someone?
There are totally other game goals to Twister.
And they're up to the people who are playing it.
What is next?
Sara
OK. That would be me.
So this is not what everybody thinks of when they think of strategic games.
But I want to point out that decisions come in a lot of different styles.
You're not going to ever ruin your game by any decisions you make in FarmVille, particularly.
But it is a game that makes people make a lot of choices about what they want to do, relatively freely.
The strategy-- where strategy and resource management comes into FarmVille is you have a limited number of actions you can take.
Which means you can only play the game for so long.
Assuming you do not want to become one of those people who spends a big pile of money playing FarmVille, which I'm going to assume you are.
So you need to manage your actions, and think about how many actions you're taking, and think about when you want to come back to the game.
And the way they do that, and the way that you can game that system, is by looking at how long it takes things to grow, what you want to grow, what you want to make.
FarmVille is actually at this point kind of outdated, because they've moved to FarmVille 2.
But I did not actually want to go there and play it to have the expertise to tell you about it.
I did spend enough time playing FarmVille to know that there's actually a whole lot of interesting decisions you get to make there.
And it's not a game people think of when they think of meaningful decisions.
Because they've heard a whole lot of, oh my god, it's that click game.
Just because it's a game that doesn't have a lot of deep consequences doesn't mean it can't have a lot of deep play.
Am I hitting the right button?
Let's find out.
So Plants vs. Zombies is essentially a really popular tower defense game.
And tower defense was not very popular until Plants vs. Zombies came out.
Because most people hadn't done it very well or presented it very well.
But in Plants vs. Zombies, you've got tons of choices.
Who here has played Plants vs. Zombies?
Let me ask the first question.
OK.
So yeah, you know what I'm talking about.
They got the UI right, absolutely.
So it's easy to play.
And every time you go in to play a level, you get to choose.
Which strategy are you going to use?
You've got sort of the long-term strategy choice for each level, where you get to choose which creatures you're taking or which plants you're taking.
And you can go back and play each level with a completely different set of plants, and still succeed, or fail at it.
In my case, it's often more failing.
They give you just enough information so you can think about what you're going to bring.
You can see what's coming in.
You don't know how many of those are going to come.
You don't know when they're going to come.
So you have to both plan for, I know there's going to be a zombie need.
But I don't know how much of them it is.
How much resources do I need to devote to worrying about the zom-- zom-- zombies.
I'm not getting that right at all, am I?
How much resources do I have to worry about devoting to that versus all the other threats I've got coming in?
Now I have to admit something terrible.
My daughters play this game.
And I have watched them play this game.
But I haven't actually played this game.
And I was going to play it last night so I could talk about it.
And then my daughters hid my iPad on me.
It's very hard to download a game when you don't have the device to download it onto.
So has anybody else played this game?
OK. A few people have.
Your goal in this game is you're a virus.
You would like to evolve and hopefully take out all of humanity.
And as you're playing the game as the player guiding the virus's development, you get to make a lot of choices.
How lethal is it?
What kind of symptoms do you have?
How easy is it to spread?
My daughters are working on the infect the whole planet with a completely symptomless disease tactic, and then evolve quickly up to lethality and take everybody out.
They haven't succeeded at this.
It sounds like the perfect strategy.
But they haven't succeeded at this.
Because you kind of need symptoms to cause damage.
And they can't evolve the symptoms fast enough.
Once they've gotten everyone infected and they're going from completely harmless to deadly, they can't manage to evolve it fast enough.
So it's got a lot of interesting play there, where you've got a lot of different tactics.
I think they can get it someday, but they haven't yet.
They'll need to try a different tactic.
And finally, a game I have played an awful lot of is Pandemic.
Who here has played Pandemic?
OK.
So what is the main tactic and resource you have in this game, if you've played it?
Turns
Turns-- well, time.
Time.
Yeah.
In a lot of ways, in this game, what you're playing against is time.
Because when you run out of the deck, if you haven't managed to clear things, the game is over.
And the resource you've got, and the choices you've got, is where people are going and which area they're fighting.
The overall plan of the game, of course, is there are four epidemics raging.
You're the CDC.
You need to find cures for all the plagues before they take out the rest of humanity.
And so you have to balance both your long-term strategy of finding a cure for each of diseases with the short-term strategy of keeping the diseases under control, so that you don't end up with everything being destroyed
All right, my turn.
[INAUDIBLE] Who's played Dominion?
OK, pretty popular.
I certainly recommend it as a good introduction to the basic idea of deck-building games.
Actually, how many people have played Magic, Yu-Gi-Oh!, Pokemon card game?
OK. All right, a lot of that.
In many of those games, and in Dominion, a lot of the fun is just building your own machine, your own little engine, that is going to execute out your strategy.
Now your engine happens to be a deck of cards.
You get shuffled up, so it's in some sort of random order.
But you get to choose what cards go in there.
In a collectible card game like Magic: the Gathering, you are buying those cards, and then assembling the cards, or there's a draft [?
mechanic ?].
In a game like Dominion, there is a way where you buy more cards while you're playing the game.
But in the end, what it all comes down to is as you play this game, you get cards that go into your deck.
They are going to end up being shuffled in your deck.
And at some random time in the future, it's going to get played out.
And so the cards that you choose to put into your deck turns out to be your main means of motivating you towards a [INAUDIBLE] strategy.
There are a range of different ways to play this game, and different possible success criteria.
And the really interesting thing about the Dominion deck is I think it gets something like over 20 different kinds of cards in a box of Dominion.
But you only play with 10 of them-- 10 plus victory points.
So every time you play the game, the strategy starts by just seeing, what are the 10 cards I've got in front of me for this session of play?
And then you choose, out of these 10 cards, what are the cards that I'm going to select and in what proportion, in order to build this engine that is going to motivate me towards victory?
So it's an interesting game.
I would strongly recommend it, if you know someone who's got a copy of this game or any of the expansions.
They're all interchangeable.
Drop7-- mostly played on mobile, although I think there is also a browser version of this game.
How many of you have played this?
Yeah.
Perfect for a T ride, I find.
You can finish a session in a single trip on the T. The idea is that you're just dropping one number from the top of the screen.
And that number ranges anywhere between one to seven.
And if you manage to make a row of numbers that add up to-- let's say I have a row of sixes.
And I have six sixes in a row.
Then all of the sixes are going to disappear.
And it's going to also clear out some of those gray circles where you don't know the numbers yet.
It's going to review the numbers behind these gray tiles.
You can think of them as tiles that were turned upside down.
So what you really, really want to make is chains.
Chains are where, if I have a six, five, four, three, two, one, then what happens is that six is going to burst.
Then that five is going to burst.
Then that four is going to burst.
Then a three, and so on and so forth.
You set up these massive chains.
And basically, you're just assigning how you're going to place this.
And if you played a game like Puyo Puyo-- I'm trying to think-- certain kinds of Match 3 games like Bejeweled, you'll get this sort of chain reaction feeling.
And some games more than others encourage you to set up these chains before you drop the triggering piece.
Drop7 is very much about that.
So I believe, if you can find it on a browser, it should be free.
I'm not so sure whether the iOS versions are free.
They are?
Great.
I keep forgetting the computer is on this side.
The StarCraft II visual novel
Wait, is it done?
No, no.
It's still in development.
Full disclosure, I am a Kickstarter backer of this game.
As has been mentioned many, many times, I am a StarCraft II player.
But I am not encouraging you to go and learn that over the weekend.
Because StarCraft II is more of a lifestyle kind of thing.
[LAUGHTER] But if you're just kind of wanting to see what the strategy is about, this is an interesting game where-- if you've ever played a visual novel, they're kind of like Choose Your Own Adventure in digital form.
And they are usually very strong narrative- and character-driven games.
You play a teenager who has moved to Korea in the hopes to start his or her pro gaming career.
And you go into a LAN center.
And you play random games against people.
Only these are the kinds of decisions you get to make.
Which strategy are you going to go for?
I usually find this to be a really, really good way of conveying the macro-level decisions, the really, really high-level long-term strategic decisions that you're going to make.
Because you're not actually playing StarCraft.
You are choosing between two or three choices.
And then it plays out.
But it gives you a good idea of the kind of high-level decision making.
Whereas most people see someone play StarCraft as like, the [KEY-CLACKING NOISE].
And you're not seeing that long term strategy.
So we do have a Let's Play up.
And the URL is there.
But if you just Google MIT Game Lab, StarCraft II visual novel, you'll see the YouTube video and get an idea of that.
Because I'm not so sure if you can still download the demo.
But the demo does have some of these features
I am last.. Mini Metro.
All right.
So my games that I chose-- what's up?
Oh, thank you.
Cool.
So this one I really like as an example.
Eww, it looks awful on the screen, though.
So I really like this one as an example because it has not yet been published.
On their website they have their alpha build.
And if you were to buy their game, what you would see is the beta build.
And then later on, when it actually does publish-- you guys can even see this on YouTube.
You can see what it's going to look like when you actually buy the final game.
It plays exactly the same.
It looks polished.
It's a really good example of here's a game that works and is functionally perfect.
And they could have just shipped it.
And I'm really glad that did, because I play it a lot.
And when they finally ship it, it's still going to play the exact same way.
It's just going to be a little bit prettier.
And it really doesn't matter.
I don't really care about that.
That's something that we're going to stress in this game.
We're more concerned about that, is it playable?
Does it work?
Can I understand how to play it-- more so than the pretty, more so than the aesthetic and visual polish.
So in this game, it is a game about managing a subway.
All of the sudden a subway station appears in the middle of space.
And you have to build a line to it.
So you build your line.
And then passengers randomly appear on the subway trains.
But you don't have to actually get them from place to place.
You don't control the trains.
You just build the lines.
The trains move on their own.
And they're moving at a constant rate.
So you can kind of guess when the train is going to get to the station.
There are strategies to-- you can move your lines around a little bit.
But once your lines are in place, for the most part, they're static.
You can't really change them all that much.
So there's some limitations going on between how much you can move a line or not.
At the end, if you last a week, it's going to give you one of three options to do a power-up, basically.
Either you can build a new line, add a new train, increase the capacity of a train.
So you're making a permanent choice there.
And that's going to affect the later game.
So you're making all sorts of these decisions at both a macro and micro level.
And there's a really good write-up about the game right there, if you just want to read that.
FTL-- I lost a ton of productivity a few months ago because of this game.
It is a game about fighting in spaceships without actually controlling all the fighting so much.
You're actually upgrading your ship, increasing your capacity, buying crew members-- and hoping your crew members survive is a lot of it.
You could-- and I do this-- manually control all your weapons.
And actually to succeed, you really do have to.
Because the interesting thing with this game is it's a perma-death game.
As soon as you die, it's dead.
You have to start over.
You have to restart the game.
Luckily the game only takes about three hours to play.
But you're lucky if you get through those three hours.
And when you finally get to the last boss, and you're there-- the thing about this game is in order to get to that boss, there's all sorts of different challenges you're facing.
And you're making these decisions about, do I buy this weapon or that weapon to help me with the thing I know that I'm going to face in the next sector?
Because I'm losing health.
I'm really dying here.
But is this weapon really going to help me when I get to the boss?
You're constantly going back and forth.
So I'll often find, after three or four, that I'm at the boss and I'm completely ill-equipped to fight this thing.
And I just die.
But I do play out to the end, because it's fun.
Last game I have-- this one's free online.
It's called Gridland.
If anybody's played the game, A Dark Room, a text-based supply chain game, this is the Match 3 version of that game, a little bit.
There's two stages in the game-- day and night.
It's Match 3, so you just collect three gems.
And you get whatever the gems represent-- resources like clay, wood, and stone, food, paper, gems.
You can upgrade your little factories to make better gems.
Every time you upgrade one of these, though-- so if you upgrade your wood, you'll get a better sword.
If you upgrade your clay, which you need to make the better wood, you're going to get in the nighttime a stronger monster.
So in this case, this is immediately daytime and then shifted over to night.
The exact same layout is there.
It's just the logs have become shields.
The little blue things that are supposed to be iron become swords.
And all the other things become enemies.
So a lot of the strategy for this game is, what's going to help me now?
What's going to help me in the future?
What's going to hurt me in the future?
And what's going to hurt me immediately as I switch over to night?
Once I get into the night mode, I'll have known exactly what I'm going to be facing.
Because I set that stage up.
So unlike Bejeweled where you're just constantly just clicking things, or Candy Crush where you're just constantly just clicking things, there's actually no time limit on this.
And it took me-- it's embarrassing how long it took me-- a day or two to realize that I didn't have to play that way.
I could just wait, pause, and play.
There wasn't going to be a time limit pushing me to continue moving forward.
And that's an interesting presentation thing.
When you're presenting your games, especially if you're using a previous genre, there's going to be genre conventions that the players are going to understand.
That is going to shape their user experience.
And we'll be talking about that about project three.
So that's it.
If you want to play these games or watch these games on YouTube, please do.
And if you want to know more about choice and conflict, there's a nine-minute video at Extra Credits.
We use Extra Credits a lot, because they're short.
They're fun videos on game design.
I highly recommend it for this class to give you a leg up on some of the design challenges that we're facing here.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So I get to do the introduction spiel for today's class.
I'm going to start by just saying what we're doing in class today, so you've got sort of a framework of what we're doing, because we're doing a whole lot of things.
And we're going to be jumping between speakers, and workshops, and lectures, and in class work.
So we actually have a lot of stuff to try and get through today to get you guys started on Project 1.
So we're going to introduce Project 1.
And that's actually what my brief lecture's going to be after I get done with the class spiel.
We're going to hear lectures on how to do brainstorming.
And then we'll ask you to brainstorm.
We're going to do lectures on prototyping.
And then I'm going to give a more pointed lecture on what we mean by low-fidelity prototype, which is what Project 1 calls for.
And we are going to do all those bits of in-class work, hopefully, scattered along the way.
OK.
So moving on to introducing Project 1, so that you've got-- and the reason we're introducing Project 1 before we give the supporting lectures is so you've got something to frame.
You're going to be hearing lectures about prototyping and brainstorming, which we want you to do in Project 1.
We want you to be thinking about the project as we give you the lecture, so you can try and figure out how you're going to be applying that information to the thing we're going to be asking you to be doing, OK?
So the overall goal is-- and I have to actually stand back so I can either see my notes, or I can see the slides.
But I can't see both-- to demonstrate a working mechanic via a low-fidelity paper prototype, to service the prototype for a digital game while tracking and understanding how your game design changes over time.
Any words there you didn't understand?
Because we've got a lot of jargon, actually, up there, a lot of game design jargon up there
[INAUDIBLE]

What is actually a mechanic?
A mechanic?
That's actually a really good question.
A mechanic is usually a key chunk of a game by which the player takes actions in the game or the game takes actions.
So if you're playing like a basic platforming digital game, one of your basic mechanics is your character can jump.
And that's what your character often uses to solve a lot of problems.
A very common mechanic in games is shooting.
So a mechanic is basically an in-game action.
It's a game design term for that.
Any other questions?
OK then.
So we're hoping that this project will teach you to do prototyping, which is a skill we will hope you will use again in Projects 3 and 4, and we expect you to.
And we're also going to introduce our very first project management tool, which is the design change log, which is, hopefully, a reasonably quick and concise way to record the changes your team makes to your design as your design evolves and helps you keep a group memory of what you've done, so you don't end up doing the same things again.
But that's not actually all we're hoping to do.
That's just the official goals.
We're also using it to give you a small group experience working in teams on a creative project.
This is the first and last time you'll be in nice, small, tight teams of three people.
Project 2, we boost you all the way up to six people.
And by the end, we're going to be in eight-person teams.
And you're going to get to see just how more complex it gets working with bigger and bigger groups.
But we're going to start you small for your first project, so you get a chance to learn how to communicate.
We're going to teach brainstorming.
We're going to expect you to brainstorm again for Projects 3 and 4, although we're not going to formally tell you where to do it or when to do it.
We're just going to expect you, when you form your groups, to start going over ideas to use the techniques we've taught you in this project to do that on your own.
We're also going to teach you-- and I think the most important thing here is-- working fast.
You've got one week to do this project.
The final version and your project reports on this are due next Monday.
You're going to get some time in class.
But unfortunately, we have a lot to say, to teach you everything you want to do in this class.
So you're not going to get all of class.
You're going to get a small portion of class Monday and Wednesday.
And we're going to expect you to be able to play something on Wednesday in class.
Or hopefully, you'll be running play tests.
We expect you to have something working.
So you're going to have to learn how to work fast.
How do you get those ideas out of your head, onto paper, and usable quickly?
And finally, the prototypes you're using here are going to move on to Project 2, probably about half of them.
Your team sizes are going to be shrinking, so we'll be doing fewer of them.
We, the instructors, are going to take a look at the prototypes as they come in, both during the presentations and during the playable playtesting sessions.
And we're going to decide which of those projects we think really do have the potential to go on to be Project 2.
If your prototype isn't chosen to go on, that doesn't mean it's not a good prototype.
We may choose to exclude things because we think the scope is too big for a two-week project, we think it's a little too complicated to try and communicate the ideas.
We may also think that there are other projects that sort of nailed the requirements better.
So your prototype hasn't failed if it doesn't move on, it's just that there were other ones that we think fit all the constraints better.
And all the prototypes are important to do.
All right.
So that's that.
OK.
So the breakdown of the project is, today, we'll be forming teams, doing brainstorming, and getting started working on things.
We will do the first formal playtesting session on Wednesday, Monday, everything gets turned in, and including your change log and a presentation.
The vision document, we will be making in class.
So don't worry about that, because we'll be explaining what a vision document is and how you make one in class.
So don't worry too much about that particular requirement.
The presentation, the one-minute pitch, will be one of the things that helps sell your classmates on joining your team and will be one of the things that we use to decide whether or not we think you guys have a good enough core idea and enough core excitement to move that prototype forward.
So it's actually pretty important.
And remember, it needs to be one minute.
We're going to have like 10 groups up pitching, so it's going to have to be fast.
Finally, design change logs, which some of you may never have heard about.
Whoa.
Oh, no.
My slide, it isn't there
Oh.
Because that screen's [INAUDIBLE]
Because this one isn't-- aha.
Oh, boy
Let me restart this presentation
So who here has used a design change log before?
Because if you've taken 608, I know you have.
All right.
So those are your experts.
If you have one of those people on your team, you're in good shape.
Talk to them.
But essentially, what a design change log is, it's actually really simple.
It's a table, three columns.
You've got a date.
You've got some action you made on your design to change it in some way, and then you've got the reason why you made the change.
And every time you make a significant change to your project, and/or every day you work on your project, you make another entry so that your team and the people working on it and the people looking at your project to see where it is now can see why you made the decisions you made and what decisions you made, what features you dropped, what features you added, what problems you were trying to solve when you made those changes.
Yeah.
I was probably-- anyway.
So that is, in fact, a design change log.
There's an example of one at the bottom of the Problem 1 handout of the-- there's an example of one at the Project 1 handout, so you can see an example there.
Are there any questions?
Hearing no takers, I hand you over to Phillip for brainstorming
OK. All right, let's talk about brainstorming.
How many of you have done brainstorming in any class?
High school?
Before high school?
[INAUDIBLE]
Wow.
OK. All right.
OK.
So it's a term that gets bandied around a lot.
Let me see.
Awesome.
However, there is a problem with brainstorming.
And a lot of people have practiced it without actually knowing why certain things are done in a certain way and, as a result of that, may actually be doing it incorrectly.
We have an excerpt of an article-- I'm sorry-- a chapter from a book called, Applied Imagination, which is written by Alex Osborn.
This is a guy who actually came out with the term brainstorming and the practice of it.
He worked in advertising.
And he has lamented since he wrote that book that a lot of people kind of liked the output of brainstorming, but didn't really understand the practice of it.
So I'm going to try to start from first principles so that you understand why brainstorming is and how it's supposed to be working.
And hopefully, you can sort of get the benefits of it, without maybe, say, overstating its power.
Now, the main reason why we're doing brainstorming in groups-- you will hear people out there in the game industry saying that brainstorming's not necessarily always the best way to be able to come up with an idea for a game.
However, in this class in particular and in a lot of video game design outside of this class, game design is coordinated, creative collaboration involving different skilled workers.
And if this is going to be the first time you're going to work with a team, you need to understand who are the people who you might be working with.
Brainstorming, if nothing else, can be a really, really good way to just be able to get a sense of who it is that you're going to be working with.
And when we start forming up teams, you're not going to necessarily be able to choose who you're brainstorming with.
So you want to be able to get a quick read of their ideas and what their interests are and maybe decide, hey, this is a person I'm going to do Project 1 with later on.
However, there are very strong social pressures of being imaginative in a collaborative environment, especially here in MIT.
There's a lot of pressure to hold back your ideas or maybe not say what's coming to your mind because you think that's kind of dumb, but other people are going to think that I'm stupid, or anything like that.
And brainstorming has certain practices to be able to get you past that.
So there are four principles of brainstorming.
The [INAUDIBLE] just do this thing.
This was the four principles laid out by Alex Osbourne.
And he was Advertising Manager at BBDO, which is a very, very large advertising agency.
And there are certain similarities at marketing and video game design, right?
You can produce ads, you can produce marketing, you can produce games, without necessarily saying, yes, we solved the problem.
There's a lot of different solutions for exactly the same problem.
And advertising is trying to get people to pay attention to the ad, and maybe you'll buy stuff.
In video games, it's trying to create an entertaining experience for somebody else.
Millions of ways to be able to accomplish that.
So we're not looking for like, here's a engineering solution to a engineering problem where it does, you know, if the thing doesn't explode, you know you've done a good job.
That's sort of like a very easy metric to be able to measure success.
For games, for advertising, not so easy.
So here's the first principle, no criticism.
How many of you have been in a brainstorming session where you said your idea, and then somebody said, oh.
Yeah, OK. All right.
I see some hands go up.
All right.
OK.
So this is the most important rule of a brainstorming session.
If you do not stick to this rule, if everybody around the group doesn't stick to this rule, you can ruin the entire session.
The whole session can just become worthless.
You can give positive feedback that says, thanks for the idea.
Just simply writing it down is positive feedback of we've received your idea.
But you want to hold back on adverse judgment or any kind of negative reinforcement.
You can share that after the brainstorm session.
You just don't want to do that during a brainstorm session, because it inhibits the process of creating new ideas.
Again, that's sort of like professional shame, especially in a place like MIT, but also if you go out into the corporate world, this idea of you don't want to look bad.
And then you can't have that in your head to have a proper brainstorming session.
If you're worried about how your ideas are going to be taken, then you're not actually generating ideas at the speed that you need to be.
So you need to be able to let your entire group know that we now have total freedom to explore all territories.
Whatever idea you generate, we're going to write down.
And then we'll figure out which are the good ideas after we've generated all of these ideas.
Secondly, you kind of want to be freewheeling.
Typically, the crazier the idea, the better.
It's easier to be able to tame down a crazy idea than to take a fairly unambitious idea and scale it up.
So typically, you will get your best ideas from some sort of pipe dream or blue moon idea, something that's out of scope, something that's unfeasible.
And then you sort of scale it down into the thing that is actually feasible.
So go ahead and shoot for the moon.
Shoot for the game that you can't possibly develop in the next week or in the next couple of weeks.
And then, after the brainstorming session, figure out how you're going to pare down that idea into something that you can do in a week or in two weeks.
Now you can expand on simple ideas.
And I'll talk a little bit about some strategies on doing that.
But usually, reworking a wild idea to become feasible, there's usually a nugget there that's a lot easier to identify when you start big and then shrink down.
What are you going for?
You're trying to come up with as many ideas as possible.
Quantity.
The greater number of ideas, the more likelihood that some of those ideas are actually going to be useful.
You're not actually trying to solve the problem in brainstorming.
Say, if the problem that we're asking you to solve is create something playable in a week, we're not asking you to solve that problem during the brainstorm session.
That comes after the brainstorm session.
That comes with things like prototyping, which we'll get to later today.
Now, that's a simple metric that you can use to gauge the success of a brainstorm session, how many ideas that you generate, at what pace are you generating them at.
Again, that's why you don't want to be judging ideas while they're being generated.
All ideas should be included.
Nothing is excluded.
And you're going to need to reinforce that within the group of people that you're brainstorming with.
There's somebody who's actually got a job to specifically do that, but you've got to help with that.
And you can build on ideas.
You can refer back to ideas that have already been mentioned.
Take the notes off, and then combine them with your own idea, even if ideas that other people have come up with, to be able to come up with something synthetically more complex.
Great ideas are often generated when disparate ideas are combined, LEGO plus Star Wars equals LEGO Star Wars.
[LAUGHTER] Which is awesome, you know?
Marvel plus Capcom, Marvel vs. Capcom.
OK, great ideas there.
And often, that's actually where a lot of your really good ideas are going to come from, because you're going to come up with a whole bunch of really mundane things that basically refer back to games that you've already played or ideas that a lot of people have encountered, but maybe not encountered together in the same context.
So this is how you're going to do it.
You're going to start off with a really, really-- try to create a little casual spirit.
This is not like a really straight laced, formal affair.
Start with giving your worst idea, the worst idea that you can possibly think of, the most useless, terrible.
If you have to do offensive, sure, but usually there's just an idea that you know is dumb.
Just tell people that.
Everyone put it down.
Write it down.
While someone is speaking, don't interrupt them.
But do designate a facilitator at the beginning of the session.
And it's going to be that person's job to keep things moving.
So you don't really need to explain so much, because again, you're not trying to solve the problem here.
You're just trying to be able to get enough ideas out so that whoever's performing the role of the secretary can write it down.
And again, before you start the entire session, make sure everybody understands this process.
If there's somebody who walked in late in the class or something and is just joining your team, make sure they understand all of these rules, these four rules that I went through.
And write down everything.
Now, don't just write down the title of the idea-- turtle tower defense, I don't know-- those are three easy words to write down, but it might be difficult for you to then remember exactly what that idea-- were they like anthropomorphic turtles?
Are the turtles the towers?
Are the turtles marching into towers?
You know, it's like, what was that again?
Let whoever is taking down notes take a little bit more information, maybe like a line of information, just so that, when you come back to that idea, you don't look at a title as, that looks like a-- what was that idea again?
Does anybody remember what his idea was?
Just take the time to be able to write things down.
Now, this is more general.
Today, we're asking you very specifically to brainstorm about the game that you're going to be prototyping for Project 1.
But in general, you can apply brainstorming to a bunch of different ideas, even in the middle of a project, Project 1 or later or to other domains.
But it's very, very important to be able to know what kinds of problems are good for brainstorming and what kinds of problems are terrible for brainstorming.
You need to be able to find a problem that is very clearly defined.
The problem should be simple.
If you've got a problem that's really, really complex with lots of very different opposing factors that you have to be taking into account, optimize, something, you can either try to divide those complex problems into subdivisions where each one of those subdivisions could have a separate answer, or you could try to use a different problem solving strategy entirely.
You don't necessarily need to apply brainstorming to everything.
You should be able to articulate the problem simply and clearly.
But just because I can articulate the problem doesn't necessarily mean that it's appropriate for brainstorming.
Osborn, for instance, gives you an example of get married.
That's your problem.
All right.
OK. And then you're trying to solve that.
Brainstorming's a really terrible technique for that.
Here are all the ways that you can get married.
It's like that doesn't help.
Maybe that doesn't identify the problem in the first place.
It's is a complex problem that's a little bit too large.
What's your game going to be about?
Or what is your player going to do 80% of the time?
What's the one action that your player is going to do 80% of the time.
Where is our game set?
Who is our main character?
These are sort of well-articulated problems that brainstorming does work for.
Now, responsibilities.
Again, I said designate someone to be a facilitator.
And this person's job is to basically keep this whole session moving.
We don't like to use the word "leader," because their job is not to be the head of the brainstorming session.
Your job is to facilitate it, is to be the clerical equivalent of moving the process along.
So you need to understand the problem and then make sure that everybody around the brainstorm group also understands this problem.
And then, when the session actually goes, every time, you sort of sense that there's some criticism coming out, and you say, all right, we'll put that aside and move on to the next idea.
Try to defuse any criticism when you see it coming.
And encourage people who aren't speaking up to speak up.
Just simply, it's like what's on your mind?
Or you'll call out someone's name and just say, any of these things give you any ideas?
Try to sort of call out those people who aren't already very much involved in the process to try to get them into the process.
Because the more people riff off each other, the more ideas you're going to be able to generate.
Encourage people to join ideas together.
Sometimes, you'll just do that yourself, just like look at two random things that you wrote down and just try to slam them together, just to get the process started.
And maybe other people will do more logical compositions.
Make sure that no voices are lost in this process.
It's not the facilitator's job, by the way, to identify what's the best solution on the board or on the computer when you're taking down these ideas.
That may not actually be the job of brainstorming at all.
That's actually a problem that you come to later when you see the whole pile of ideas that you generated.
And now, you figure out what it is that you're going to choose for, say, Project 1.
Now, there's a second person who's involved.
This is the person who either has a giant Post-it pad, or a chalkboard, or a computer usually connected to a projector.
And that person is the secretary whose basic job is to record every single idea in a way that everybody around the group can actually see those ideas.
You don't want to just work on your own personal laptop, on your own screen where no one else can see it, because then other people can't, say, look at ideas that have been mentioned before and combine them.
So make sure that you're writing things up on somewhere where everyone can see.
We've got some giant Post-it pads right here.
And you can grab them.
And we've got some markers.
Stick them up on the wall and write on them.
So you want to try to just report on what people are saying.
Don't try to be editorial about it.
Don't say, good idea, bad idea.
That's your job either.
You do want to keep an eye on the time, especially because we're doing this in class and time will run out.
So make sure that you actually have enough time to get down the to those last couple of ideas before we go into our next part of our class, which I think is prototyping.
The secretary can participate in a brainstorming.
It's sometimes a little bit tough.
It's tough to write down and listen to what other people are saying while trying to generate your own ideas.
So what you might want to do is split up your brainstorming session into, basically, two halves.
How much time are we giving them for brainstorming?
30
30 minutes?
So split it into like two 15-minute chunks.
And either the facilitator and the secretary can switch, or just get two other people to be secretary and facilitator.
So the people who were facilitator and secretary can sit down and have the space to generate their own ideas while someone else is taking notes and making sure the process is running.
So that's what we're asking you to do.
Any questions about brainstorming?
Or anything else I've said?
Yep?
Is there a limit or a minimum number of ideas we have to come up with?
Is there a minimum of ideas?
No.
In fact, that's probably counterproductive.
Don't say, oh, we have to hit a least 30, no-- it's 30 ideas and we're done-- because that implies that once you're over 30, you can stop.
You don't want to stop.
You want to keep generating ideas until you run out of time.
And if it takes you 30 minutes to come up with two ideas, that's fine.
That's the group.
That's what your group was able to come up with.
Generally, you want to be generating like a lot, several dozen ideas in a 30-minute session
I'm going to ask you to form groups of six people.
Grab the people who are nearest you.
These groups of six are going to like-- do us a favor and everybody stand up.
As you form groups, sit down.
You've got three minutes to form groups.
[SIDE CONVERSATIONS]
So we just finished the brainstorming session, and I'd like to get some people's impression about what worked.
Do you need some time to get started?
Were people kind of feeling it went a little slowly at the beginning and then picked up or died off halfway?
Tell me some results.
Yeah?
I think it's like the sessions were like, instead of two-minute, three-minute sessions, they were like even shorter.
And like broken up into kind of a mini theme [INAUDIBLE]

And it felt like it ran out of steam
OK. All right.
So sort of like breaking up even further into smaller chunks and that.
OK. Sure.
That's a good idea.
How about the note-taker as a facilitator [INAUDIBLE]?
I think starting with words and themes helped us get just basic concepts out there and then sort of applying them, rather than jumping into game ideas
Right.
And one of the reasons why we did that will also give you an idea of what's a theme versus whats a mechanic, right?
Yeah
It gets hard to avoid the scenario of someone writing an idea, and then everyone's like, oh, that's cool.
Let's talk more about that thing, instead of just throwing out completely different ideas
Right.
Oh, so switching to a different idea, right?
Oh, you know, that's not necessarily a failure of the process either.
But you're right.
You're trying to generate quantity.
And instead, people trying to improve the quality of the idea.
So yeah, but still that's a happy problem to have, right?
It is a good point.
Yeah
I think, for our team, jokes helped us a lot
Sure
Our first situation was more tense.
And then, after one or two specific ideas, everyone became more casual in saying more ideas
And that's why the throwing out the bad ideas part helps as well.
It helps you get into that mood of we can be silly now.
Something that might help with the switching gears problem is the act of writing it down, to begin with, of course, sort of says the idea is on record.
And you can always say, let's get back to that later, because you will get back to it later anyway.
And there will be time for that discussion.
It doesn't always work, but it's worth a shot.
Cool.
OK. Let's talk a little bit again about Project 1, just as a reminder.
There is a theme for Project 1, and that's planning for randomness.
You generated a lot of ideas.
We didn't really over-emphasize that constraint right at the beginning of the brainstorm session, because we wanted you to generate tons and tons of ideas.
But now, you should be looking at that theme and the ideas that you've generated and figuring out which ones of those kind of match that theme.
Now, that's sort of your overall constraint, over-constraint of what we want the players to be doing is planning for randomness.
But is it in space?
Is it some sort of like dice rolling mechanic?
Is it some sort of race mechanic, are just a branching decision tree or something like that?
That comes into themes and mechanics.
So you want to be identifying that, as well, as part of Project 1.
So what I'm going to be asking you right now is that, from your brainstorming group, we want you to split into two groups.
That's your prototyping group for Project 1.
You only have to work with this group for one week, you know?
This is not a long-term commitment.
And there were eight brainstorming groups, so basically, we're looking for 16 teams.
And so we're encouraging everyone to be in groups of three.
Not only does that mean that we will get the right number of groups, it also means that if somebody falls sick in the next week, your team still has at least two people to work on it.
So for each group of three, first of all, you figure out who you're going to work with.
And then you choose one item from your list, one of these ideas from your list, keeping in mind those constraints that I just mentioned.
And we will be using that idea for the upcoming prototyping workshop, which I will be leading.
So once you know what your idea is, you know who you're working with, I will go into the prototyping workshop to tell you what you do next.
And the goal will be to then test that idea through the process of prototyping.
And if it doesn't work, you are free to choose a different theme.
But you still have to stick with your team.
OK?
You stick with your team, you can choose a different theme, just to be clear.
All right
10 minutes?
We have 10 minutes to figure out who you want to work with and what your initial idea that you want to work with, so go for it.
[SIDE CONVERSATION] And your team was six, right?
Cool.
You used the computer control, so it was really voice-activated.
It just didn't work
[INAUDIBLE]
How many of you have seen me give this presentation before?
OK. How many of you are thinking does Phillip give his presentation every class?
Correctarino.
I do.
[LAUGHTER] So I apologize to everyone who's seen this presentation.
Maybe there's one or two little points in here that you might not have seen before.
But for the most part, it is the same, because it is the fundamental skill that we try to teach in every single game design class, because it is basically the first skill that you're going to need to be able to figure out whether an idea works.
How many of you have made prototypes in any field, say engineering, software, whatever?
All right, why do you make a prototype?
Because it's cheaper
Because it's cheaper than making the real thing.
Sure.
I thought I saw some hands up there.
Yeah?
It's faster than making the whole thing
It's faster than making the whole thing.
Yeah.
But that kind of goes hand in hand too, right?
Time is money
Making drastic changes is a lot cheaper on paper than it is the built system
Ah, OK. All right.
You can sort figure out what's broken and then make changes
It shows the proof of concept
It shows the proof of concept.
You can show it to other people, and not just prove it to yourself.
Right
It's a lot easier to experiment and make changes that are necessary when you're not committed to something like [INAUDIBLE]
All right.
You can try a lot of different ideas, right?
It's like, I'm not quite sure which way I want to go, so I'm going to try a lot of different alternatives
It helps you realize, maybe, your flaws [INAUDIBLE]
Catch the big problems that are going to break your design.
Yeah
To move from 2D to 3D?
To move from 2D to 3D?
You'll make a 2D prototype, before taking it into a more complex design.
All of those are absolutely true.
You know, just for the general sense of why you want to do a prototype at all is to get feedback from other people.
And you get it early, you get it cheap.
And of course, we talked about making changes early on in the process.
Usually, it costs less money because you haven't committed anything.
You haven't invested a whole lot of resources in something just to have to change it again.
If you're not quite sure which direction you want your design to go, you want to be able to have lots of different ideas on the table and actually see how all of them work in one way or another.
We talked about how easy it is to be able to change our ideas when you're prototyping.
But also, it's easy to discard a prototype.
It's easy to just say, well, that was a bad idea, because you haven't really spent all that much time on it.
And in many ways, that's something you're going to need to do in game design a lot.
You're going to pursue some of these ideas, some of these ideas that you've already been discussing.
And you're going to start prototyping it today.
And then you're going to say, well, that was a bad idea and shove the whole thing off the table into a trash can somewhere and start afresh, because you've spent what?
An hour designing this thing?
It's not going to be a whole lot of time.
And then you can learn from that process and build something new.
So that is very good mindset to be able to have in early iterations, that this thing that you're making is meant to be informative.
It's meant to teach you something about this project that you're working on, but it's also meant to be disposable.
It's something that you do not have to make look good, or you don't have to make it last.
You don't have to make it perfect.
You are just trying to learn something through the process of prototyping.
Now, when you're making games, this is what you're trying to figure out.
You're trying to find the fun.
And we've talked about games being a series of interesting choices.
And when you're prototyping, what you are trying to do is you are trying to identify what are those few game mechanics that are going to be able to sustain the interest of your players over a significant amount of time.
So the nice thing about prototyping when you're, say, trying to design a computer game using paper materials or something is that you have to define those mechanics down to the very purest form.
So for instance, I could say, oh, this is going to be a game about space aliens.
And you're going to run around shooting different kinds of weapons and occasionally throwing grenades and taking cover.
And you get to fly ships and drive things around.
But what I can do when I'm prototyping is that, all right, I'm going to make a game where you come out and take a few potshots, go under cover, wait for your [?
shoots ?]
to recharge, throw a grenade, wait for [?
shoots ?]
to recharge, come out and take a few potshots.
That's basically what game?
Halo
That's basically Halo.
That's the thing that you do every 20 seconds in Halo all the time.
And that's really the thing that you have to nail and get absolutely right.
So when you are making a prototype, you can identify, oh, this core little loop is the thing that's interesting.
And if we can actually get it right and tuned perfectly in [?
additional ?]
game, then we know players are going to do this over, and over, and over again, and it's gong to feel good.
So you're trying to find that fun.
So you're going to focus on the small handful of choices that a player must make in order to play your prototype.
And you model the system with a few basic rules that creates interesting choices for the player.
Always ask a question about your game.
Make sure it's falsifiable.
Anyone heard that term, "falsifiable hypothesis"?
No?
Yep.
What's that mean?
Basically, it means some kind of opinion that can be proven wrong
Yeah.
You could say, well, say the question is that we think the players are going to enjoy making these kinds of decisions.
And the answer could be, no, they don't.
You want that information.
It could be something like this particular design is going to save time, right?
It's going to be less of a cognitive load.
Or it's going to be easier for players to understand.
And it should be that kind of question that you ask when you're making a prototype.
Your prototype should be able to be designed in a way where you can sit someone in front of it, play through it, and then tell you yes or no, was that hypothesis true or not.
Now, physical prototyping, paper prototyping, doesn't give you insight into everything that you can achieve in a digital game.
It doesn't say, for instance, tell you terribly much about the sensory experience of playing that game or how feasible a certain feature is going to be.
So you don't discover everything through making a prototype, but you do get a lot.
Now, the other thing that a few people alluded to is the ability for your prototype to communicate to other people, especially other people on your team.
So imagine getting into a room full of developers.
All of you are game developers.
And you don't know what's in each other's heads right now.
You maybe have come down to a certain theme.
And all of you have different versions of that game floating in your head somewhere.
Now, if you want to be able to explain that to somebody else, it can be near to impossible to communicate that.
You can try doing it verbally.
You can try writing it down.
But all of those methods don't really compare very well to something you can actually set somebody in front and play.
So what you are trying to do is you're trying to give you own team, to start with, something solid to work from.
It may not necessarily be-- it will almost certainly not be the idea that you're going to end up implementing in the long run.
But you're going to start creating some sort of framework for you to then say, well, I didn't like that.
I should change that.
Or I really like that, and we should try to polish that a little bit.
So it's a communication tool.
And any kind of game design definitely benefits tremendously from early stages of physical prototype, because it forces you to think through how your game is going to work.
Every little detail about how your rules worked, at least.
Maybe not how your game looks, maybe not what your input mechanism is, but certainly what are the decisions you're asking for your player and what are the consequences of those decisions.
You have to define them.
You have to write them down in rules that your players, as well as your design team, actually understand.
And then they're going to try it out.
And you're going to see all kinds of emergent little dynamics pop up when all of these rules are put together that you didn't anticipate, and could be really cool or could completely break your game.
And you want to find that out.
I talked about bringing in testers.
And as we keep harping on this class, and again, every single class that I teach, testing's incredibly important.
But the nice thing about paper prototyping is that you don't really need special skills.
You need third grade arts and crafts skills, really.
Maybe MIT math.
Sure.
But actual, we're talking about using markers and scissors and kindergarten counters and dice.
And the nice thing about it is that, as your team gets larger and larger, everyone can still contribute to the paper prototype.
It could be an artist.
It could be a designer, a producer, a [?
quoter, ?]
a QA tester, even someone that you just brought in for a day to play your game, can all contribute to the design of the game, because it can give you feedback of, well, what if, instead of rolling this dice, I roll that dice.
Or what if the grid was a little bit bigger?
What if my pieces were a little bit larger and took up more space, for instance?
A paper prototype itself is not just interactive in its play, but is interactive in the design.
You can just open up the hood of your paper prototype and change a rule.
Do not do that when you're working with code.
In the middle of a playtest, do not change your code, for god sake.
But you can do that with the paper prototype.
And many times, you might want to do that in the middle of a test.
It's like, you're playing a game.
10 minutes into the game, it's like, oh, my god, this game is hurtling downhill into a pit.
And you can see it.
And the player sees it.
And you know there's a problem.
You can just change a rule and just say, what if we did this instead, mid-game even, and then try it out.
And says, oh, that seemed to maybe address the problem.
Then start the game from the beginning and see if it actually did.
But you can do that with a paper prototype.
Paper prototypes, in particular, are very good at revealing usability problems.
The simple issue of a player has a decision, knows what he or she wants to do, and has no idea how to actually execute that.
It's like, does that mean I roll first and then move, or do I move first and roll, which piece do I move, things like that.
Paper prototypes are very, very good in sort of just laying that bare.
Your team will see where those problems are, as soon as somebody who's not part of your team sits down and tries to play it.
Let's see.
Now, players do tend to more readily give criticism when they see something that's very clearly incomplete, very clearly something that you're still working on.
And if you've got a hand-drawn cardboard paper prototype in front of them, they know you're right at the beginning of this project.
They know that they can tell you something about, I really don't like how this character looks.
And it's like, that's fine.
We spent two minutes drawing that character.
We can completely redo that.
And they'll tell you that, if it feels like you haven't spent a whole lot of time in it.
And that's a useful trick, by the way, if you are doing, say, the digital version of your games later on.
It can actually be easier to get feedback if you deliberately insert some things that look incomplete, a giant box that says Place Order right there while you're doing your testing, just to get a little bit more feedback from your testers, so they feel, OK, yes, they're working on this.
I can tell you that character looks terrible.
And it's like, we spent three days on it, but you don't say that.
You don't say that in front of them, because you want that feedback.
But when they give you feedback, you take everything that they're telling you with a grain of salt, because, in the paper prototype, it's very, very easy for a tester that says, I don't like pieces that are this color.
I want pieces that are that color.
And it's, OK. You could just make the substitution.
But what you should do is you should think back a little bit and try to figure out why they're giving you this piece of feedback.
They may have a personal preference, and they think this is the way how it should be executed.
But probably what they're say is that they have some sort of actual problem with your game, and that's the only solution that they can think of.
So for instance, why does this person want to change the color of the pieces that they're moving around?
Is it because they're having trouble telling the pieces apart?
That may be the real problem.
And the solution may not be change the pieces to a different color, it may be like change the piece into a completely different shape, stick a little sticker on them.
On a computer, it's going to be different kinds of solutions.
Vet every little bit of information that comes into you.
People can make suggestions on how you're going to change your design, but it's up to the team to actually provide some sort of unified direction on where you want the game to go.
So the feedback is valuable.
The solutions they're providing may not be the solutions you want to use.
Any questions so far about why we're doing prototyping?
No?
OK. We'll have more questions later.
So here are the kinds of tools that you're going to be using, very large sheets of paper.
This is probably the smallest sheet of paper-- well, we're going to be using index cards as well.
A lot of you have seen this before in my other classes as well.
This is a square grid on one side, hex squared on another.
For spatial games, you can use this.
But sometimes, it's much better to start with a blank sheet of paper.
So don't immediately start designing grid games just because we happen to have sheets of gridded paper.
But we do want them to be large, so that everyone around the team can actually see what you're working on at the same time, as well as the people playing the game.
If you're going to be designing something like an iPad game or something like that, don't use something that's literally the size of the iPad.
Use something a little bit larger, so that everyone around the table can see what the tester is doing while they're testing your prototype We use index cards for a lot of different things, writing down rules.
You can shuffle them and use them as playing cards.
You can record data on them.
They're very disposable, more accurately, very recyclable.
Please, please recycle any paper material.
Dice, of course.
They can be tokens.
They can be used to help keep track of stats.
But something that I've seen a lot, that students like to use a lot, is they use two 10-sided dice to keep track of a number between zero to 99.
That's a really bad idea.
One brush of the hand, and that variable is gone, right?
You can swap the tens and the ones really easily.
It's like, was that 91 or 19?
I can't remember.
So if you want to record variables, just gram an index card and just write down the variable.
And then, whenever the variable changes, cross it out right in your variable.
It's much better for you to use a dice either as actual randomizers, or you can use them sometimes as tokens that move around on the board.
Let's see.
What else?
Game bits, pieces from other games are wonderful, but sometimes a little irritating because, if you took them out of the game box, then you have to put them back into the game box, otherwise, you've just ruined your copy of the game.
So I like using disposable kindergarten counters a lot, because I don't really care about returning exactly the same number I took out.
And there's usually less assumption about what a kindergarten counter is supposed to be.
Is it a number?
Is it a person?
Whereas, if you use game bits, that's kind of person-shaped, [?
meatball-shaped.
?]
Post-it glue and notepads-- well, I don't think we actually have any Post-it glue right now, but they're basically glue sticks that don't stick permanently.
So you put that on an index card or on a piece of paper, and suddenly, it becomes a Post-it.
But what we do have are a lot of Post-it note pads, both big ones and small ones.
And these are useful for you to simulate displays, for instance, or variables and information that a player is supposed to have.
And you don't want that to go flying off if somebody sneezes.
Of course-- oh, right.
I have a picture of some Kindergarten counters up there-- pencils, pens, markers, scissors, tape, just to be able to hold things together, write down information.
Something else about Kindergarten counters.
We have a particular kind of Kindergarten counter, which is an interlocking cube.
You take it out?
Every year, we've--
Well, it is possible that we took them out because they're very problematic.
Because once I give you interlocking cubes, you make games about interlocking cubes.
It's like handing LEGOs.
And then, the first thing everyone thinks of is let's make it a game where you put things together and assemble them
Like stacking counters
Stacking counters
So you make games about--
Stacking.
Yeah.
The pieces that you end up prototyping with sometimes ends up shaping the direction of your game.
And then, sometimes, if you are dealing with a designer's block, that can be helpful.
That gives you a path to follow.
But in many ways, that's a distraction.
You start becoming constrained by your tools, instead of the idea that you're trying to work with.
So if we managed to get rid of the interlocking cubes, that's awesome.
But if they're still in there, by all means, use them.
But don't automatically assume you have to make a game about interlocking cubes.
Now, don't forget to keep a record of everything, including all of the notes that you've had.
Use your phone camera doing the session.
At the end of your project, make sure that you try to get them scanned, photocopied in some way.
There are a lot of photocopy machines around campus now.
There are actually scanners.
And you just put the raw materials on top.
And instead of generating a copy, you can have them email a color scan to your email account.
Those things are really useful for your final assignment.
OK.
So those are the tools.
Who are the people sitting around the table?
When you've actually made a prototype, what happens is-- I guess I got some of my slides mixed up-- this usually happens at the end of having designed a workable prototype is that there is going to be a bunch of people sitting around the table.
And each person has a different role.
That's, of course, the person who's actually going to be playing the game, preferably somebody from a different team or somebody who's even outside of this class.
It could be one of us instructors, for instance.
It could be one of the OCW people.
And somebody in your team is going to be a facilitator.
This is the person who's going to present the rules to the person who's going to play, you know, tell them that we're looking for your feedback.
Make them comfortable.
Make sure that they understand what they're supposed to be doing and basically making sure that the session runs according to script.
That person, or somebody else, can also be the computer.
This is the person who is actually making the prototype work, updating variables, moving the positions of pieces on the board, revealing new dialogue boxes, or basically making the board respond to what the player decides to do.
And the computer is trying to do all the computation that your actual computer will do, if you're making a digital game.
The computer should not do anything that a real computer wouldn't.
So for instance, if somebody doesn't understand a rule, the computer person shouldn't be correcting the player.
The facilitator could.
But sometimes, what I find is really interesting is that somebody thinks a rule works differently.
I let them play through it.
And that becomes like a different iteration of the game.
And sometimes, it turns out to be better.
Then you can run through the iteration with the correct rule.
Yeah?
So the person who's the computer does have to make a choice at that point of whether they would let them play according to what they think the rule is?
Yes
Or what the computer-- like if it were a real computer, it would just think [INAUDIBLE] something that doesn't work [INAUDIBLE]
And it puts it in play.
And the computer says, nothing happens, you know?
It's like you can do that, right?
It's like if the person really doesn't [INAUDIBLE].
But if you notice, say, this person is clearly playing a really sub-optimal strategy, just respond like a computer will, lead them to [?
that loose ?]
condition or whatever it is that happens.
Because in many ways, that's how an actual player is going to learn how to play your game in the long run.
They're going to make a lot of bad decisions.
The computer is going to respond, and then they're going to learn from the mistakes that they made and then try it differently.
Let's see.
Something else that the facilitator can do while the game is going on-- say, the player is actually playing-- is to encourage the player to think aloud.
So the person that you brought in to play your game is trying to understand your game and is just like sitting there in silence looking at the board.
And you don't know if they're bored, confused, or really into it.
So you ask them, what are you thinking now?
And if they're saying, what are all these pieces?
All right, then.
You know that they're confused.
The facilitator can help, can say, all right, let's start again, and I'll explain what all these pieces are.
They could be, ah, I've got like five different strategies, and I'm trying to figure out which-- you know, I'm thinking, maybe, if I move this here, [?
I move this out, ?]
OK, that person's into it, right?
That person's really thinking hard about this problem.
Everybody else in your team watching that board.
And this is the reason why we have a big board.
As an observer, you want to be able to see what's going on in your game.
You want to be able to write down the information in a note book or on your laptop and try to record whatever information.
Most importantly, look at the face of the person who's playing the game.
Again, that helps identify whether they're confused or whether they're really into it and trying to make a tough decision, or they're just bored.
Don't offer help to the user, especially if you're on the development team and you know all about this game.
It can be a real temptation to basically just tell the user, this is what you want to do.
Don't do that.
Sit on your hands.
Bite your tongue.
It can be hard to take notes when you're sitting on your hands, but, you know, take notes.
And the person who's being the computer and being the facilitator really doesn't have the bandwidth.
Those people do not have the bandwidth to take notes.
So someone has to be officially designated with the job of actually taking notes and observing.
What we're going to be doing is a Wizard of Oz test for your games.
And that is someone's playing the computer.
It's very constrained in terms of communication between the computer and the person who's testing your game.
The rules are usually pretty rigid.
And the nice thing about it is that you can do a pretty deep analysis of a game.
You can make a game that sort of goes through multiple decisions into sort of long-term consequences.
That's not necessarily what we're asking you to do for Project 1.
I'm just telling you this because you might want to do this for Project 2, 3, or 4.
A Wizard of Oz test allows you to really do a deep dive into how a particular feature works, for instance.
And since it's a human simulating the back end, simulating the computing, you can be very high-fidelity with very little cost.
That's not, again, we're asking you to do for Project 1.
We're asking you to do a low-fidelity prototype.
And we'll go into a little bit more detail about what that is.
Very important.
Don't let the player know what the computer is thinking.
The computer only displays information.
You do not reveal how the computer is coming to those conclusions.
There are a couple of other kinds of prototypes that you could design.
I'm going to run through them very quickly, because that's not what happens in this class.
Player versus player, what you'd expect in a board game or a card game situation.
We have a class just for that.
And you can make cooperative games, competitive games.
You can make games where everybody has the same role, a symmetric game, or asymmetric game where everyone has different roles.
The problem about that is that there's a lot of loose miscommunication happening between the people who are actually playing your game.
And they may decide on a house rule that was totally not what you intended or designed as developers.
And that turns out to be the rule that you end up playing with.
Again, what I like to do is I like to see them play through that rule.
Then, at the end of the game, I correct them, and I have them play through the rule that I originally designed, because sometimes, the rule that they came up with is better.
The facilitator, again, helps to explain the rules at the beginning of the game.
And then you kind of just watch after that.
It's hard to get consistent results across tests when you're doing a multiplayer game, because it's very hard to duplicate exactly the same situation.
Whereas, you can do that on a Wizard of Oz test pretty easily.
But if you're making a multiplayer game, you have to do a multiplayer test.
So this might actually happen-- actually, are we saying that you can only do single player games this semester?
I'm not quite sure what the constraint is.
You can also do live action, not for today.
But I actually enjoy these where you actually have predetermined rules that are explained between the game.
And then you just let real human beings walk around and do stuff.
The problem with this is there's a very limited communication between people who are playing a game or even the people who are running the game, because everyone's kind of physically spread out.
But it can be really useful for a game, if your game is really about a human being walking through a space, such like a stealth game or something like that.
Yes, that's actually a person in that U-Haul box.
All right.
So when you're actually making your prototype, here are the things that you have to make sure you do for every single prototype.
Keep it hand-drawn, because hand-drawing is fast.
You want to keep it really, really sketchy and look really sketchy and rough, again, because that actually gets you feedback from the people who are testing.
You make it look too pretty, people are going to hesitate on commenting on how your game looks, for instance.
Or they may actually hesitate on commenting on how your game works, even though that may not be the prettiest part of the game, because they can tell you spent a lot of time on this game.
Again, you want it to be big.
Everyone needs to be able to see what's going on during testing.
And just use one dark color, if you can for your inks, for your markers.
It needs to be dark, high-contrast, so that everyone can read it easily.
You can use different color cards, for instance, to be able to tell this card is a different kind of card from that card, but don't use multiple colors on the same card.
That's just not worth the effort.
And again, it gives people the impression that you spent a lot of time on this prototype.
And even if you have, you don't want to give them that impression.
You write your rules on cards.
So keep track of your rules.
Eventually, you're going to end up typing it down on your laptop, because that'll make it easier for you to turn in your work for Project 1.
But while you're working on a prototype, try writing your rules on the cards instead, because that allows you to do things like rearrange the steps in which certain rules operate.
It allows you to change out a rule really easily, while keeping the old rule around.
And you can just swap things in and you can try things differently.
Every time you change a rule, change the card.
If you are changing the number of rules, instead of taking, say, take three steps every turn, you say take two steps every turn, just cross out the three and write two in there.
Now, periodically take photos with your phone cameras.
And always try to simplify.
Always try to get down to the smallest set of rules that is necessary for you to test your idea.
Part of that is just to speed up testing.
You're going to test within your team first, just to be able to see whether this basic mechanism works.
And eventually, you're going to end up testing with each other.
And the simpler your rules, the less time it is to be able to start a new playtest, because there's less for you to explain to somebody else who's never seen your game before.
So try to get your rules to be as simple as possible.
Also, if you have too many rules and something breaks in your game, it's sometimes harder to figure out which rule was the culprit.
If you have fewer rules, it's usually more obvious for you to spot.
Now, we talked about iterative design a lot.
And this is basically the loop that we're using while we are paper prototyping.
You start with a question, a falsifiable question, a yes/no question.
And what is this prototype going to achieve?
Maybe in this particular case it's going to be, does this theme that we picked actually have randomness in it that you can plan for?
Or is it the randomness so random to the point where you can't plan for it?
Or is it really just very deterministic, and when you play the game, you can really tell, yeah, here's the optimal solution?
You can look at axiomatic design-- anyone heard of this term?
[?
Mackee, ?]
in particular, uses this a lot-- which is, basically, you establish axioms, which are things that you are taking to be true, a baseline that need to be fulfilled by any workable design.
So it could be we assume that this game needs to have some sort of random condition that's generated that changes during the game.
And we need the player to identify that this number is changing and anticipate where that number might end up going.
And if your prototype fulfills both of them, then you know that prototype worked.
And you can build a lot of different prototypes that fulfill those same two conditions.
That's another way to start with a question.
And then you actually start designing, grabbing those pieces of paper, writing out your rules on cards, brainstorming on what your game's all about.
But you want to do this as quickly as possible.
The less time you can spend talking about this, the more different kinds of ideas you're going to be able to run through.
And that's what we're trying to do during a prototyping process.
If you're ever in a situation where you've got two different options, this might be better, or this might be better, or you're not quite sure, it's usually faster in paper prototyping to just start making both prototypes, you know?
All right, you do this version, you do this version.
Generate the cards, play through both of them.
And you'll get actual evidence, actual playtesting evidence on which idea seems to work better.
After you design, you play through it.
First within your team, and then you grab somebody from some other team, or one of the instructors.
And we'll sit down and we'll play a game and see where the strength in this design, where are the weaknesses.
Whatever you design within an hour is going to be clunky and broken and really, really not quite where you want to be.
That's find.
Do a playtest with that expectation that there are going to be problems.
And you'll get a lot more information out of that playtest than you will working on your own.
And then you use that information to revise your design.
Make changes, and then you repeat this entire process again.
Either you amplify it on the strengths, you figure out what your weaknesses are and you make big changes.
So you repeat that.
And here are a couple of tricks that can work, sometimes, if you have trouble figuring out what sort of changes to make in your design.
Killing a rule, just like taking a rule out sometimes fixes problems more than trying to fix the rule itself.
A certain thing is bogging down the game, a certain thing is too confusing.
Just does the game work if you just take that rule out?
Sometimes it does.
You can make a resource that's limited, unlimited, or make an unlimited resource limited, limited health, unlimited health, limited ammunition, money.
Player interference is more for multiplayer games where, sometimes, if everybody is playing their own little solo game and they're not interacting with each other, you'd come up with ways to mess up each other's plans.
That doesn't always apply with single player games.
But if you've ever played a game like a role playing game where you can delay an enemy.
Has anyone seen one of those where you can like cast slow or something on an enemy, and then they can't attack you for another turn or something.
That's player interference, right?
Only the player that you're interfering with happens to be the computer.
What can you do to basically get into somebody else's plan and change it?
And the computer can have those rules too.
Mess with the play order.
Mess with the order of your rules.
Just try rearranging the order of rules.
And sometimes by grouping things into logical chunks, it can make the game easier to understand.
Instead of saying, first, you decide whether you're going to do this.
Then you decide whether you're going to do this.
Then you're going to decide whether to do this.
Maybe you just say, make two of these three decisions, right?
And every turn, you get two out of these three.
Make them any order you want.
That's a way that you mess up play order.
If you're going to change variables, either multiply or divide them by two.
There is not enough granularity in the prototypes that you're making today to do like a 10% change in a certain variable and then be able to detect whether that change was the right change or not.
That's not going be obvious enough.
Do like big multiples of two, of three, to be able to see whether that's the change that's necessary.
You can tune things later when you are preparing the project for the playtest on Wednesday, for instance.
But when you're in this class, do things by big multiples.
If your game's really, really, really kind of getting bogged down and taking too long to play, try to get down to the core set of rules, the minimum set of rules that's necessary even just for your game to work.
Just for the game to run at all, try to figure out what that core is, the minimum set of rules.
And then you add in each rule, one by one.
Reconstruct your game.
That's one way to identify where the main problems are in your game because, as you add the problematic rule, things start to break.
And finally, be willing to just throw away everything that you generate in the next hour, OK?
Anything that you make in the next hour might, in fact, just be terrible.
And you're going to learn that.
And that's fine.
That's why we're making prototypes, so that they can be disposable.
That's how I started this talk.
So any questions before we actually start making stuff?
Coming up with rules?
You're going to grab pieces and create some sort of a play surface or cards that people are going to play with.
You don't need to put anything on a board.
And before everybody runs down and grabs things, I guess we're going to go back and remind people what a low-fidelity prototype is.
[SIDE CONVERSATIONS]
--terrible as long as you're aware of it
I thought I replaced it with a stacking counter, so I guess it didn't work.
[SIDE CONVERSATIONS] [?
All right.
?]
So we just have a couple more minutes left of class.
A couple of things to remember before you leave.
Take pictures of all of your brainstorming notes.
And keep your notes with you, so take them away with you.
Make sure everyone on your team can access those notes.
Make sure you know the email addresses of the people you're on your team with, so you can talk them again.
And before you leave the classroom today, start up a document and enter your change log.
So put in what you designed today and what problems you found by the end of the day.
So I'm going to repeat that one last time.
Before you leave class today, start up a document, open it up, and start up your new design change log.
Put the date.
Put what you did today, so a brief, two or three-sentence description of what you designed today and what are the problems you're facing now.
That'll be important to you both for when you meet as a team again to remember what you did, but also later on to have a full history of everything that you've done.
OK?
One more thing.
Yes?
One more thing.
Tomorrow-- [?
Wednesday.
?]
Wednesday, rather, we are going to have playtests.
We're going to be playtesting all the prototypes.
So you need a playable prototype for Wednesday, 3 o'clock.
[?
And ?]
then we're starting Wednesday's class off with discussions about the gaming tutorial, so bring your problems, bring your questions about the engines that you were assigned.
And we'll talk about them.
All right.
See you Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so let's go ahead and get started.
A couple quick announcements before we jump into the day.
We had a couple questions about what happens if one of my team members dropped out of the class for this first project?
Has anybody had that?
OK, so if somebody drops out of your team at any point in next first two projects, that's OK. You're not going to be penalized for having fewer people.
You're not going to be propped for having fewer people.
It's basically we're not necessarily grading the quality of the games for this first project right now.
Double check the project one assignment to see what the grading rubric is for that.
What we are checking is are you doing iteration?
Have you tested the game multiple times?
Do you have multiple versions of the game?
So that's basically reflected in your design change log that you're doing for this class.
There's also an individual write up.
So we're asking you to write a one page paper about your experience with the requirements for that are in the assignment.
We want to know what happened when your teammate dropped out.
Did it feel different?
If you had two people versus three people, do you feel like you had fewer ideas?
That's actually a really interesting thing for you to understand about.
When you're coming together and working on group projects and you're doing collaborative projects, what does team size do to the kind of ideas you have, the number of ideas you have, and the quality of ideas you have?
In all the later projects, your team is going to be big enough that dropping one member isn't going to be huge difference.
Dropping two might.
So if that happens, just let us know about it.
We'll talk you through it.
It'll be OK. We'll get through it
If I may, this is actually part of the experience of working on a team.
Sometimes you lose people, and sometimes that's a problem.
I like to call it the hit by a bus test.
If your team can't stand losing one person-- any one person-- then you have to plan around that and make sure you can handle it.
Other people like to call it the winning the lottery test because that's nicer, but basically one person gets taken out of equation.
You do have to adapt to that and be ready for it to happen.
Again, on this project, it's a relatively-- the teamwork is easier, because it's a simpler deliverable and a simpler project
But things are going to come up throughout the semester-- somebody falling sick, somebody going away for interviews, especially seniors going for job interviews.
Those are things you're going to plan for to react to and have alternate plans, just in case something like that happens during the project.
So just be aware of that throughout the class.
So what are we doing today?
Today's a very, very busy day that's going to involve lots of movement on your part.
We're going to do a workshop on version control that Drew is going to walk you through.
We're going to go into discussion groups and talk about the game engine tutorials that you all completed before today, and they'll be a mechanism for talking about that.
Basically people talking about forming groups in the same project and then later on forming groups where we can have people compare-- somebody who worked on Unity talking to somebody who worked on Flixel.
Things like that.
After that, we're going to have project one play test.
So this is going to be an informal play test.
Basically it's a check.
Can you play your game today?
If you can't, then play what you've got, because you're going to be forced to play it.
Maybe you have some rules written down.
Great.
Just act them out in that 15 minute block we give you to act out those rules.
And maybe within that 15 minute block, come up with a few different rules or a few different pieces.
So that's going to happen, and we'll go through how that's going to work later.
And then at the end of the day, we're going to do another workshop.
Sara is going to lead us on that-- talking about vision statements.
So this is a tool that we use to help large teams understand what it is we're actually creating.
So with that I'm going to switch slides and bring it over to drew
So the first thing, I see a lot of laptops, which is pretty awesome.
How many of you brought a computer that actually has the code you used for you tutorials on it?
That is more than half.
Awesome.
So you get the short version of presentation, not the long one, as soon as we get it up there and going.
Let's see.
Any other questions?
Who here has used version control or source control before.
Most of you.
Fascinating.
Who here feels confident that you know why you used version control, and that it was awesome, and you're cool, and you're good at this?
OK, the hands went up and started to come down slowly, which would make sense.
That's what they should have done.
[LAUGHTER] All right, so I'm not actually going to talk about the nitty gritty details of exactly which commands you type, because there's a bunch different version control systems.
They all work very similarly, but you're going to basically need to read the documentation on the particular one that you're using.
I recommend today, if you're familiar with one, that you stick with that if possible, but don't worry about it if you can't.
So let's see.
I'm going to give a high level description of what is revision control.
I'm going to use different words for it.
Source control, revision control are the big ones.
The basic notion is that-- and this is a very simple notion-- that you've got one copy of the source on your local machine, and you've got one copy of the source somewhere else, and that copy out there is the perfect copy.
That's the true copy of what you're working on.
That's the platonic ideal of your project, and your job is to copy files down from it, change them, and copy them back up to that one.
Not much going on there, right?
All right.
I'll get into more details on that.
So we started with a whole bunch of words that go along with this topic.
Revision control system, RCS, is often used as a generic term.
It's also a specific piece of software that you probably won't use because it's really old.
Source control, revision control, and you'll also see SCM as a common TLA for source control management.
CVS, the concurrent version system, is also older, but one you might actually see used.
It's mostly command line only.
SVN is a more recent open source version of CVS.
I think CVS may have been, too.
I'm not sure on that one.
SVN is more usable, and I highly recommend it.
Perforce is one that's used quite often in the games industry, and then you're going to see Git and Mercurial, which are often mentioned in the same breath.
They're both what's called distributed source control.
We'll get to that, but they're interface is very similar to all of the above actually.
Almost all have a command line interface where you can do it everything from command line, and most of them have a GUI that you can use as well, but you're going to see a bunch of different words used to talk about the stuff.
So again, in its most basic form, you've got a server.
You've got your local.
You copy stuff back and forth.
Pretty basic.
And it's a little bit easier to see why you might want that when you've got more than one person on the team.
It's a really good way to make sure your files are synchronized.
It's even more important the bigger your team.
So what's really, really important is that that server copy of your code is useful.
So you want to make sure that you are only contributing to that central repository of code the useful pieces.
Now, what we really want to do is that central version of the code is sort of what's considered, as I mentioned, ideal version of your code.
It's authoritative.
It's the one you can trust.
So that actually becomes a little bit more than a backup system, right?
So a backup system is quite useful.
If you lose something or you make a mistake, you want to be able to revert to a previous version of your code, and that is certainly one of the things that source control does for you.
And we'll also talk about how it's a way to share your code, but it's actually more important than that.
So the history of your project is one of the big things you get out of it.
When you submit your files, which can be source code file-- it could be images.
It could be anything.
When you submit them, you should include a comment about what the heck you've done.
And this is one of the things I find new programmers have the hardest time with.
In fact, I know plenty of old programmers that can't do it, but really it's useful, and the reason it's useful is all because of this.
We're going to talk about different methods of communication throughout this course, and I've talked about-- or I will talk about the fact that your source code is a way to communicate with other people.
But it turns out your comment log in your version control system is also great way to communicate with your team.
So for example, if you submit a whole bunch of files and it's not obvious what that did to your game, that comment can tell everyone else on your team what just happened.
Maybe you rewrote the entire AI so that the bad guys now run away from the player.
And it may be that someone else on your team is trying to figure out why are the bad guys' AI broken now?
And the answer is it's not broken.
They're just doing something different.
And when they do get that code from the server, they're going to see that comment and say, oh, I get it.
The AI changed.
So if you fail to report that in a Scrum-- or maybe you don't have a Scrum because you did it in the middle of the night-- this is a great way to communicate with your team.
Even more than that, that history is also really useful.
Let's say you check in this code that causes bad guys to run away, and then a couple days later you find out that there's some problem in the AI where things actually crash horribly if the game sits there for an hour.
Well, where do you begin looking for that bug?
You may not know that it's in the AI.
You may just have some clues, but you know that it changed in the last two days.
This behavior started.
So you can look at the log and say, what happened in the last two days?
And someone further discovers that, well, it's crashing somewhere in AI.
Then you can have a hint that it's in the AI system, but more than that, you go to the exact last change in the AI system and look at those files, and that's probably where your bug is.
That's pretty handy.
And let's say that you do something really, really horrible.
You also get an undo button out of this.
You can say, you know what?
That version two days ago-- let's go back to that.
That's better than what we have right now.
You don't often do that, but it is really nice to have that kind of security, especially when you're working in your local version.
Let's say you're trying to find a bug.
You need to change 15 files, and putting into debug statements everywhere to figure out what the heck's going on.
And then you find the bug finally, and you realize it's a one line change.
Well, it's easier just to revert back to the server version and make that one line change than to make sure you undo every little piece.
Let the computer be your memory.
Let the computer be your undo button.
You don't need to rely on knowing which files you changed.
It will tell you.
And this is the cool thing that is kind of scary, but pretty awesome-- is you can actually simultaneously edit the same files if they are text files.
This involves a merge process afterwards which is a little bit frightening the first several times you do it, but actually the tools are pretty good.
As long as you're not modifying the same line of the file, it's pretty good about figuring what should happen, but you do need to manually check it with your eyes to verify it didn't do anything stupid, but it's actually pretty common to work to be able to work with same file with two people and actually have it work out OK.
The exception to that is extremely complicated text files or binary files-- you can't merge changes to a PNG of GIF.
That doesn't make sense, at least on the file level.
And a lot of the data files in Unity, especially the scene files and prefab files, are going to be too complicated probably for you to actually manage to emerge.
So for those, you need to be a little bit more careful about it, but it's actually a really useful tool to lean on that.
So we're going to talk about the various operations very, very quickly.
When you copy things from the server, different source control systems will call it different things.
You're going to see the term get to mean pull the files down from the server.
Update, pull, and checkout in different systems will mean that same thing-- copy files from the server.
I also have in there the word revert, which is kind of a fancy special case.
When I said you can undo your changes-- you've got a whole bunch of files changed, and you don't like those changes-- you can revert to the server version, which is-- under the covers, what's really going on there is you're copying down from the server, but you'll call that revert because it's going to ask you, wait a minute, you changed that file.
Are you sure you really want to wipe that out?
And at that point, you should say yes or no depending on how sure you are that you really want to do that.
Similarly, when you have decided that you've finished with your changes locally and you want to copy them out to the server for everyone else to see, that's going to be called submit, commit, put, push or check in.
Generally speaking, people talk about that copying up to the server directionally.
The reason they use some fancy words like commit is that they want you to realize that this is not-- I'm just copying a file.
You're copying a file for everyone to see.
So you want to ensure that your changes are good.
This one's easier.
If you want to find out the status of what's going on, almost all the source controls use the word status for that.
But basically what you can do is you can say things like, so I've changed a bunch of files, and I forget what I've changed.
I'm sure there's a lot of them.
You can actually ask your source control system, what did I change?
And it'll say, well, these five files have changed, and in fact, you can even get more information from that if you want to compare what those actual changes are.
And diff is the term for that.
Locking files so no one else can change them is not often used in source code, but it's very useful for-- as I was talking about, there are some files that are so complicated that it can't automatically merge them for you, and the binary files as well.
You may find that some source control systems don't automatically lock these for you, and so in that case, you might want to make as your team policy that you want to lock the file so that no one else messes with it.
So again, I'll go back to Unity as an example.
If you're changing the main scene file for your game and it's this complicated gigantic XML thing, you might lock that file so no one else changes it while you're working on it.
And then finally, merge is the last thing that you'll do.
And I talk about, if you're changing one code file and someone else changes a code file while you're working on it and checks it in before you do, you're going to include their changes into your version, and that's called a merge.
Most of them have tools that do it for you automatically, but if there's a conflict, it will then force you to do what's called a resolve, which is to say, hey, this file changed more than I could figure out automatically, so a human needs to interfere with it, and that point-- that's called resolve.
And that's what the process is called, too.
You'll have to go through that manually a couple times.
So we talked about how this is an authoritative server, and that's true for Perforce And Subversion.
And basically there's the one server that's going to be taking care of you, but it's not always true, especially if you're looking at Git and Mercurial, which use what's called distributed source control.
There actually is no difference between your version of the code, and the servers version of the code, or your teammates version of the code.
It's just a bunch of lists of changes, and you can send them left, and right, and upwards, and downwards.
It doesn't matter.
But if you actually do this, you're in trouble, unless you're really careful about it.
So whenever you do a sideways shift-- you send some code to your teammate, but you don't send it to the, server you want to make sure you know what you're doing and be very careful with it.
So I recommend, if you're using a distributed source control system, until you're really comfortable with what's going on there, that you mostly treat it like a standard server based system until you're really-- and make sure if you do any other kinds of cross submits that you know what you're doing and you think about it a little bit.
The cool thing about these systems, though, is that, if I make a change and I send it to my teammate, and both of us submit it to the central server, it actually keeps track of that.
It says, oh, it's the same change.
I don't need to apply it twice.
It's actually a really very clever technology.
It's very, very cool and very powerful, but like many powerful technologies it can be very confusing.
So as I say, start out here, and if you're feeling really cocky, go there.
A couple quick rules about this.
They're not going to apply to you yet, because you're not yet writing code in teams, but the policy is for most teams never ever, ever break the build.
And when I say break the build, I mean you want it to be the case that that one canonical copy, the perfect copy of your project, anybody can go to an empty machine, run a get, get all the files, hit the Build button for whatever environment you're using, and have it build and work perfectly.
That is the goal.
That's what you always want.
And if you ever submit something to the server that makes that impossible, it's highly likely that you have just made everyone else in your team have to wait.
And that's a big pain because they're going to be sitting there wishing they could get work done, but they did a get to make sure they could do the merge, and then all of a sudden they can't get any work done at all.
So you want to make 100% sure that you can always, always, always do that build.
Another thing-- and this is me sort of stepping aside to my coding hat-- you should always have zero errors and zero warnings if at all humanly possible in you project.
A lot of times I see programmers who are just starting out who think that warnings are optional, and it's true.
Unfortunately, most of them are kind of optional, but there's two problems with leaving them in your build.
Problem number one is some of them are optional, but they do in fact highlight an issue that you should think about.
And if you leave it there on the screen and ignore it, you might be ignoring an important tool that'll help you find a bug.
But in the second one, I think it's even more important actually-- is that as soon as you get used to seeing warnings in your build every time, you start ignoring all the warnings.
So let's say there are five warnings that pop up in your build every time you build it, and this has been true for a week and a half now.
And you're writing some code and you add two warnings, and you don't notice because there are warnings.
Five to seven, you didn't notice the difference, but one of two warnings that you added might be really important.
So if you can get down to zero warnings, as soon as you had one morning, it will instantly hit your brain, and your brain will say, oh, wait a minute.
I did something that the computer says is a little bit risky.
And in computer science, you want to avoid a little bit risky, because it's going to come back and bite you, and I promise it's easier to fix a bug 10 minutes after you wrote it than a day after you wrote it.
And a week after you wrote it, it's even harder.
We in this class won't get to a year after you wrote it, but sometime in your career, you probably will, and you won't enjoy it.
Avoids bugs the first time if you can.
So therefore I call use a check build.
On a team, you might say to someone sitting next to you, hey, can you do a get and build and make sure I didn't mess it up?
That's one easy way to do it, but actually you don't need to do that.
On all these source control systems you can have more than one-- what we call-- workspace.
It might be your local version of the code.
You can have two workspaces-- one where you're doing your work.
You check it all into the central server.
Second one, you go to the second workspace.
You pull the code down.
You do a build.
You verify that everything still works, and that's a safety way that you can in a couple minutes verify that you aren't about to make the entire team wait and lose time.
Couple little tips that, again, you probably won't need a little bit, but I'm going to talk about it very quickly now so you don't forget.
All these things I'm talking about are things you're probably going to have to look up the details on how to do them, but I want you to at least have heard the words, so that when you see them on the screen, they'll look familiar to you and you'll know what you can find.
Ignore unneeded files is a really powerful thing.
Almost all these systems can ignore types of files.
For example, if you're working in C#, there might be some object files or whatever, and you don't want to check those in.
It doesn't help you to check them in.
In particular, if you're using Perforce, checking them in is actually problematic, because one of the things that Perforce does that the other systems don't do is Perforce uses the read only flag as a way on your local machine to tell you whether or not you have a file checked out for changes.
So if a file is read only, it's saying you haven't told me that you're going to change this file, so I'm not going to let you.
If you try to do a build on a bunch of read only files, your build environment has two choices.
It can ignore that, which is kind of scary, or it can just fail.
So that's kind of annoying, but more to the point, there's no reason to check in these gigantic product files that are all part of your build when you could actually build everything from the bare source.
The only things you want to actually check in are the original source files for your project, whether that be art, audio code, whatever, but you want to be able to rebuild the entire thing.
And all the intermediate files or the final result files-- generally speaking, you don't want to check those into source control, unless you've got a special build folder or something.
You want to have one folder, which is purely pull down the project and build it, and it contains all the information necessary to regenerate everything that you can.
So if you can ignore all the files that you don't want to check in, that's good.
This is particularly a problem with Unity.
You don't want to check in anything but the asset folder and the project settings folder.
There's a bunch of gigantic folders in Unity that look important, and they are important, but Unity will regenerate them for you, and you don't want to check them in.
And then finally, I talked about locking before.
Lock you binary files and your big text files that don't merge well just to avoid headaches.
If you don't do that and more than one person touches that file, eventually you'll get in a situation where you lose your changes because someone else checked in something over you.
Similarly with a merge be careful.
If someone has checked in a-- even to text file, and you do get it down to the server, and it has to merge with your changes, and you don't do that, and you just say, nah, forget their changes.
Use mine.
That's going to be a problem, because someone just loss some work there.
Luckily with source control you can recover from that, but it won't be smooth teamwork.
No one will be happy that you wiped out their changes.
That is the quick version, and now we're going to go on to a workshop.
So you should be sitting near people who used the game engine that you used.
Hopefully most of you actually did this thing.
So what we want to do is team up in pairs with people ideally who used your engine.
If there's only three of you who used your engine, I guess you're a triplet.
And what we want to do is-- ideally two ways, but one way will do.
One of you should say, yes, I've got working code.
It's pretty cool.
Run it.
Show everyone.
See it works.
And then pick a source control engine of your choice.
I recommend GitHub, because it's easy to get things set up.
Check in your code, have the other people check it out, and see if they can build it.
And then if you succeed at that, swap and do the other way around.
Actually, you can do this simultaneously.
Everyone can start checking in to new projects and then pull down.
Well I'll take that as a cue that I can start talking a little bit, but not for long.
I promise.
So the next step we want to do is discuss the game engines that you did the tutorials for.
You probably did a little bit of that already because you've been hanging out with people near you who use them, but I want you to try to get into groups of five to six people.
If you can't, that's fine, but try to balance it out.
There probably should be about two groups per game engine.
And talk about what your experiences were with the tutorials.
So things like, what was easy?
What was hard?
What was more difficult than you expected?
Compare your experiences on different operating systems if you happen to have that experience.
Compare your experiences if you used different tutorials or the same tutorial, but basically I want you to get an idea of what other people's experiences we're like compared to your own.
And the main reason for this is that you want to figure out something a little bit more about your game engine, because after this, we're going to shuffle and get into groups where we've mixed up the game engine.
So for example, if you used HaxeFlixel, you'll be in a group with someone who used Flixel and someone who used Unity, and you want to compare and contrast those engines.
And so think of this first discussion as a way to figure out how to talk about your game engine and maybe learn more about other people's experiences with it.
So I believe that we're already seated mostly near each other in the game engines.
I think that's all I need to say really.
So split up into groups of five or six in your game engine, and if you need help with that, we'll help
[INAUDIBLE]
15.
15 minutes of that, and then we'll swap in to do multiple-- we'll help you with that part, too
Your attention, please.
May I have your attention, please.
So I think a lot of you-- I see quite a few groups are winding down on their discussion within the same game engine.
Now what we're going to do is ask you to gather back in your original brainstorming groups, which is probably going to be pretty close to the combination of the teams that you are currently in.
So remember when we did brainstorming on Monday?
Those people?
You are going to gather with them, and hopefully you're going to get a good spread of different game engines within that discussion group.
And what we want you to do is share your experience having used that game engine.
First, take a quick survey around of which game engines are being represented, have a talk so that you know which game engines you're discussing.
Then talk about what you found good, what you found terrible about those game engines, and what you recommend it for.
OK?
As a primary motivator for this, project two starts next Monday, and you're going to choose an engine and make a game in it.
And you're going to be committed to that engine for about 2 and 1/2 weeks.
So that's the whole reason why we're talking about this stuff
This might not need to be said, but if half of you used one game engine, do some swapping.
I would like to see we can put a final couple of details as to what we discovered here.
If you were to try to describe the experience of Haxe in a sentence like thumbs up, thumbs down, what was the hardest thing?
Anybody?
Please?
There weren't a lot of resources out there [?
for it.
?]
Not a lot of resources.
So you couldn't find help me when you got stuck?
Yeah.
It's pretty [INAUDIBLE]
OK, I've heard that the install process is painful.
Let's see.
What else have I heard?
Anything else?
Anything to recommend it?
Once you got it up and running, it was easy
Once you got it up and running it was easy.
All right, cool
Personally I was on a Mac, and I found the installation really simple.
It was just a couple of [INAUDIBLE] calls, and it seemed like it was really useful for a simple game.
I'm not sure how [INAUDIBLE] will get when we were trying [INAUDIBLE], but for getting a very simple game running fast, it's nice
So it sounds like there's some mixed experience on the install.
Mac may be the difference.
I don't know
I was also on a Mac, and once I figured out how to install on a Mac, it was extremely simple, but the thing was the tutorial was just for Windows.
Install Flash [INAUDIBLE].
Do this.
Do that.
I was like, OK. [INAUDIBLE]
So the main thing I've heard then is the support is limited.
The install may or may not be hard, but if you get stuck, the support isn't going to help you through it, at least in that experience.
The main reason why I think HaxeFlixel might be interesting is because I actually know that Flixel is interesting, but you guys may have a different opinion on that.
So Flixel.
Any other big picture takeaway for Flixel?
You're all going to disagree with me now, which is fine, but say something
I think it's support is also limited
OK. And one of problems I've seen in Flixel is that it, in fact, has evolved over about five or six years now.
And so if you look for a bug in Flixel and try to figure out what was going wrong, it'll say, well, in Flixel 4, you want to do this.
It does end up being a little bit more complicated there, too.
So in addition to there not being as much support as there is for, say, Unity, it's fragmented support.
And then of course, Flash is one of those technologies that people are considering to be aging.
I will say, however, that in my experience for 2D stuff, Flixel is really, really easy to pick up once you get going, and it can really do the 2D stuff pretty well.
On the other hand, we have a new contender there, Phaser, which I heard the [?
trial ?]
was really, really good for Phaser.
Does anyone have any takeaway on Phaser?
The trial was good.
It handled 2D stuff pretty well.
We haven't gone in depth on collisions
[INAUDIBLE] it was definitely good for 2D.
It's very intuitive, and it's all written in JavaScript, so not too complicated.
[INAUDIBLE]
Does anybody know if it supports TypeScript?
[INAUDIBLE]
It does support TypeScript.
Did anyone use TypeScript for Phaser?
No?
Oh, you did?
Yeah
Right, I just talked to you about that.
In fact, you were able to use a debugger with Phaser through Visual Studio, right?
That actually I find pretty encouraging.
Someone's put some thought into that.
How about support?
Where there are a lot of people using Phaser?
Enough that you can get some help through searches?
Yeah.
OK, cool.
So it sounds like Phaser is a-- I haven't heard about many of the HTML5 engines that I would actually want to recommend, but it sounds like so for your experiences with Phaser have been pretty positive, which is pretty cool
I have a question about Phaser.
What about the install process?
Any thoughts on that?
[INAUDIBLE] files could just [?
be ?]
[INAUDIBLE] HTML [INAUDIBLE] scripts
So you just copied it in, and it works?
Yeah, so Phaser has version control pretty much [INAUDIBLE] because you have to pull everything from Git.
The second tutorial of making the game was great, but as a couple of us noticed, the first tutorial had installation-- like setting up a web server and then choosing your ID was a little bit more obfuscated.
It's like, why are they telling us to do these things [INAUDIBLE]?
Oh, right.
Right.
Yeah, because I do think that-- there was somebody said if you set up a web server, this is how I would do it.
And I think the answer is there are easier ways to do it than they have suggested, especially if you're an MIT student.
You can copy it to your web locker, and whoa, you've installed it-- is a fine way to do it
[INAUDIBLE].
With the TypeScript installation it enables to [INAUDIBLE] copy paste in [INAUDIBLE]
Cool.
All right, and then I split Unity into concentrate on 2D, concentrate on 3D, but fundamentally there is a difference between those two-- that Unity pretty much-- and I heard someone even saying that Unity is designed for 3D, and they've kind of pasted 2D on top of it, which I think is an accurate criticism about usability on that.
Has anyone experienced the-- trying to use the 2D, was that a problem?
Are you going to still need to think in 3D to make it work?
Yeah, the other thing was the initial learning curve was pretty-- took a while to get something simple started.
[?
But I feel ?]
like there's a lot of potential making more complex 2D stuff
Sure
It definitely doesn't feel lacking.
It's like Photoshop in the sense that there's all these features, and you maybe not even 10% of them, but they're there if you need them, which may or may not be a good thing, because that can be very intimidating and sort of hard to work
Sure.
I agree with that 100%.
There's too much there, perhaps.
If you're trying to do a quick project, you might be tempted to use more than you should.
Also [INAUDIBLE] some things get lost in the features
Yeah, in some of the videos they demonstrate how you can actually have 3D objects in the 2D environment when you're developing 2D games, which I guess they were excited about.
And they were just like, hey, you can do this cool thing, but I think it has the opposite effect in that it intimidates people.
There's a lot of options that were built originally for 3D, and looking through menus and looking through all drop down things, most of them are things you can' touch or you can't use, because they're meant for 3D, and there's no way you can restrict the UI to be only 2D.
So I feel like they should instead make it into a standalone Unity 2D software, which would be easier
I understand why they don't turn it into standalone Unity 2D software, but I think your point is well taken.
I think that it would help with their uptake if they were to try to limit their feature set, oddly enough.
This is not the first time this has happened actually.
Way back when there was Torque, which is a 3D game engine.
It was really cheap indie-- one of the first indie game engines for like $100 for use it all you want.
And they started with their big fancy Torque 3D package, and everyone said, wow, this is awesome.
And a bunch of people bought it, and they never made any games, because 3D is, as we've mentioned, more than three times as hard as 2D, and so they came out with Torque 2D, and that did a lot better.
Again, because developers-- 3D is an extra hurdle to jump over, and when all the developers have to deal with a short time frame, and that 3D can get in your way.
So I think the takeaway for me on Unity is basically the same, which is it's a lot.
You can do a lot with it, but it's got a learning curve.
It's going to be a little bit harder for you, so you might want to think twice about doing that, especially if you're concentrating on a 2D game, which you probably should be doing, if you can, this semester.
Unity also has a couple things that are a little bit different.
I think mostly Flixel certainly-- and sounds like Phaser has a little Flixel blood in it.
They're both designed for 2D.
Very simple, straightforward implementations in an object oriented kind of way.
Unity has this crazy component system where you've got one object, but you staple anything you want to to it.
And it's really, really powerful, but also it takes a while to your brain around it.
So long story short, I would say that, if you can do what you want to do without taking on a big engine like Unity, I kind of recommend that, but obviously if you need to do it and will kind of-- the reason [INAUDIBLE] if you tell us that you do need to do it, but if you do need to do, it's worth considering, because I think it is a very powerful and useful friendly engine
Which one has [?
faster ?]
development cycles?
Good question, but I'm only going to guess at this point in our experience we have not used Phaser in a class before.
I know Flixel is faster than Unity.
It sounds like from the feedback we're getting here that Phaser might be a good contender, so I would recommend in project two you might want to take a look at those two engines as your first go to things and check it out, because there's not good options.
I would go there first, though, rather than going straight to Unity, unless you've got a team that really wants to take it on
Are we limited to these options [?
that ?]
[INAUDIBLE]
Yes, you're limited to these options.
So let's see.
I think our next step then is something else
Yes, it is.
So congratulations.
You've gotten through the first 2/3 of class today.
[APPLAUSE] Yay.
And that was our gaming tutorial, so keep all of these thoughts in mind as we start up project two next week.
The remainder of class is going to be devoted to finishing up project-- or doing more work on project one.
We're going to do two activities that we mentioned at the beginning of class.
We're going to start off with play test.
So if you have something testable, great.
You're ahead of the game.
If you don't have something testable, we're going to give you a little bit of time to get you to something testable.
What do we mean by testable?
It means somebody else that is not on your team with a little bit of guidance by somebody on the team can make things happen in the game.
That gives you feedback that says that was cool.
That's what we were going for or that's really broken.
That's awful.
Let's not go that direction.
That's all we're asking for you to do for class today.
So at 3 o'clock, we are going to start the play test.
It is 2:47 PM right now.
Ignore that first part.
Basically what we're going to do is set up your in your teams.
Every team should have their game set up and ready to play at 3 o'clock.
The team that is closest to them-- you're going to first one team is going to play the other team's game.
You can have 15 minutes to do that and then swap.
If you followed project one's examples correctly and you have a five minute game play game, so your game can be played in five minutes or less, you can get three tests done today.
Awesome.
But if you only get one or two, that's OK. You've got the whole weekend to finish up on that, too.
So come back into this room 3 o'clock with your game set up, and we'll start the play test.
Does everybody have a playable game?
Yes?
Thumbs up if you're ready to go.
OK. All right, so we are not just playing our games.
We are play testing our games, and we are following the Wizard of Oz method that Philip mentioned on Monday.
By that what we mean is all of your games probably have some kind of computer that's doing something in the game that the player is interacting with the computer.
What that means is there is some constrained communication between the computer and the player.
This is a time where you want to think about what are the things that the player knows versus what are the things that the computer is doing.
There's going to be a little bit of a difference going on there.
Don't let the player know what the computer is thinking, unless it's actually part of the game.
If you could imagine the computer saying here's a screen with an icon on it that says some stuff, that's something the player knows.
If it's just some calculation going on in the background, that is not something the player knows.
So just think about it that way when you're doing this.
So how many teams do we have over here?
One, four?
OK, so pair.
How many teams do we have in here.
One, two, three, four, five, six.
Is that right?
So pair to your closest, and then pair.
And how many teams over here?
One, two-- is that three total?
All right, instructors will test the back one.
So you have 15 minutes to play one game.
I'm going to come back in at 3:15 PM and tell you to switch
OK, so we're going to do the shift from the very fun play testing and working on games to talking a little bit about project management.
And as he said, we're trying to get the project management in it bit by bit.
Hopefully the point at which it becomes helpful in each of your projects.
So the first thing we're going to talk about here is vision statements.
Why are we talking about vision statements?
I'd rather be play testing someone else's game.
The very short answer is because it's good for you and because it's good for your project.
The longer answer is because a vision statement is really one of the shortest and simplest ways to make sure that you, the people on your team, and the person you are making the game for-- assuming you are making the game for a client, or an instructor or someone else who might be interested in it-- are all on the same page and understand that understand what they expect to get at the end of the project.
It can't guarantee that you all believe you're getting the same thing, but you can get started by being on the same page.
Another very common early design document for games is the design doc.
Has anybody here heard of design documents, I assume?
Yeah.
Do you know what they are?
Does anyone not know what I mean when I say a design doc?
OK, traditionally-- and this is actually going further and further back into the history of game development, for which I am thankful.
Traditionally, a game design doc was a document, usually a big word page that might occasionally get printed out, that pretty must described the game from start to finish-- everything that was going to happen in it, all the mechanics, all the characters, all the stories, all the interactions, everything that's going to happen.
It was traditionally a very big, long file that, A, either someone spent a lot of time maintaining and nobody be spent any time reading, or, B, nobody spent any time maintaining and nobody spent any time reading.
So it was not what I would call a very useful investment of time, but a lot of time tended to get invested in it.
So you may ask, why?
At least partially because people feel better when things are written down.
They'd like to have a place they can go to look at to say, am I doing what I think we're supposed to be doing?
I'd like to go check the documents and see if that's true.
Unfortunately, a really long document that tries to detail everything is not the right way to try and do this on a game.
Games change too fast.
And the thing you're trying to get to, the end product of the game, is not written.
It's not something that's going to be something you look at.
It's something that happens in action.
So our proposed substitute-- and if someone comes up with a better solution, I would love to hear it, because vision statements are also not the perfect thing.
They're just better than anything else I've seen.
Our solution to this is a vision statement.
We tend to think that, in between a prototype, and a vision statement, and really good project communication, that's about-- and really good communication on your project, that's most of the design documentation you need.
All right, so we're going to talk about vision statements, and then we're going to make one.
So there you go.
I forgot that I included the animation on the slide.
I did not time it right.
Oh well.
That's what I get for trying to be fancy.
So if a vision statement is going to define the project-- so vision statements defines the project-- or the game if it's for a game-- for the team, for the client, and its investors.
What do you think it ought to include to do that?
I see my chalk boards have already been used, and the chalk has been hidden.
There it is.
All right, any ideas?
Any suggestions?
What would you like to have in such a document?
You're the ones using it
Goals.
[?
Scope.
?]
Goals.
Scope.
I heard ideas.
What kind of ideas?
[INAUDIBLE]
The game
Goals, ideas
Goals, ideas, the game.
So I've got goals.
I've got ideas.
I've got scope.
I heard game
Theme
Theme.
Theme, OK. Theme, that sounds like a really good idea
Core mechanics
Core mechanics, yeah.
Anything else?
The story if there is one
Story
The audience
Audience.
Anything else you think you really need?
[INAUDIBLE] characters
Characters.
I think this is actually pretty darn good list.
The tricky part with a vision statement is trying to get most of that in on one page or less.
The other thing that you vision statement may want to include that is less relevant to the vision part of the statement and more relevant to the team is, who's working on the game?
What's the name of the game?
What is each person on the team going to contribute to the game, so that the vision statement becomes both a contract about what the game is going to be and what the team is going to do to get it done.
So surprise, surprise.
We actually have a format for this.
Actually, we have two different formats to help you get all of these things crammed in on one page and hopefully express it well enough that you will understand and be able to use it.
We have what we call the back of the box vision statement and a high level design doc, and you guys are welcome to choose to use either one of those two frameworks.
They're up on Stellar.
They can be downloaded, and once I finish talking at you, I will pull them up so you can see them up on the screen and actually see what the files look like.
The idea behind the back of the box vision statement is it takes a marketing pitch approach.
Instead of actually stating here's the goals, here's the theme, here's the core mechanic, here's the scope, the team comes together and tries to find three, or four, or five bullet points that would go on the back of the box that capture those ideas within the bullet points.
You've all seen the box-- OK, maybe you haven't seen the back of a game box these days because half the games are being so downloaded from Steam, but they still have the little marketing pitches where it says fire 38 different guns, form teams with your friends and back stab them.
That sort of thing.
[LAUGHTER] But if that is the way your team thinks and your team envisions the game back, that can be a very good way to formulate and encapsulate all of those ideas, especially if you can really drill down to what are the four things you want people to say about your game, and then every time you make a decision about your game, every time you add a mechanic, every time you add a story, a character, a setting, you make a change of the way the game works, you can go back.
Read those four or five bullet points and say, is this change actually enhancing and working with that pitch?
And if it is, it's probably a good change.
And if it isn't, it probably is a bad change.
It gives you something to really have a core of your game around.
Some people don't think like that.
Some people don't think in terms of marketing pitches.
They'd much rather have a better spelled out this is my goal.
This is the setting.
This is the mechanics.
A high level design doc is the solution for those people, and in it, it asks for a goal statement for your game as a short paragraph written out, rather than as pitch.
It's got a little less bang, but it is often more precise, and both of them are equally effective.
What really depends on is which one resonates with the team, which one the team thinks works well for them.
That's why we have both of them.
They're really covering the same information.
You don't save anything by choosing one format over the other.
It's just how are you presenting it to yourself and to your team.
Both documents also ask for a little bit of game play.
A risk assessment-- what are the big risks on your game?
They may be related to your team.
Hey, we're trying an engine we never tried before.
We may not be able to get to run.
It may be related to the game play.
We are going to try and do a mechanic no one has ever managed to do before.
Whatever your big risk is.
Just one or two sentences of it.
And then some space for your team members to put in your names, what they're going to do, stuff like that.
It's really pretty simple.
And the next step is, in fact, to do exactly that.
So as a three person team, go ahead and decide which format you're going to use.
Create your vision document.
Decide who's going to upload it for Monday.
Make sure you've got people's names on it so we know who did what, because if we're only uploading one document, that means we're only getting one document for every three people in class, and we would like to give credit to everyone for having worked on them.
And you're going to have until the end of class, so about 20 minutes.
Let me see if I can find where Rick has hidden the forms.
Are they--
Yeah, sorry.
Don't look at my calendar
I'm not looking at your calendar.
Oh no.
I am looking at your calendar.
I'm sorry.
I'm not looking at the calendar.
You're looking at the calendar.
I'm sorry
I don't know how well they're going to fit
Well, we'll do them one at a time.
We'll just do them one at a time
[INAUDIBLE] kind of look
To clarify, the upload comes before Monday.
Not before the end of class, right?
Yeah.
Yeah, the upload-- the turn in date is for Monday, before the start of class on Monday.
And the main thing is you're not actually writing division doc for the prototype you just made.
The prototype you made is the core-- is the core mechanics for the game you envision being made with it for project two, OK?
So this is your vision document, your sort of pitchy document, for project two-- with the game that will be made for project two, OK?
All right, anyway so here's the high level design document.
I think that there's both an RTF format and a Word format up there.
When you upload them, we much prefer it to be in PDF.
That way easy for us to read.
I love trying to do this while not being able to see what I'm doing on the screen
Look on Stellar
I like mice.
I much prefer mice.
I hate touch pads
I also make mine kind of wonky
Oh, thanks
You're trying to scroll
Thanks.
OK, yeah, I'm trying to scroll.
So this is basically the high level design doc, and what it's got-- vision, breakdown, major game play concepts, 30 seconds of game play, and then your risks, and the team responsibility and breakdown.
Let's go ahead and jump over to the vision document.
And here's the vision statement one.
Basically all of that-- goal, and stuff needs be contained in your back of the box advertisement.
And then the rest of it is pretty much the same.
Any questions?
OK, if there's no questions, go
OK, before you leave, just want to make sure we all understand what is due tomorrow, because all of our projects are pretty complex
Tomorrow?
Monday.
Monday.
Thank you.
You're awake.
So for project one on Monday before class starts-- so before 1:00 PM-- turn into Stellar the following three things.
An individual post mortem-- so this is a one page write up of your experiences working on this project.
The specifications are in that project one assignment.
That's so we can know what you got out of this project.
The design change log.
You made some changes today.
So on your change log, you should at least have one entry on it at the end today.
Now you've got two entries on it hopefully.
Test your game this weekend.
Make some changes to the prototype this weekend.
Add them to the change log.
Only one per team.
So appoint one person to submit all the team documents, submit it, make sure all of your names are on it.
The vision document, which you've just been working on-- one per team.
And again, it's a little weird for project one because you're talking about what it would be like for project two.
So under team roles and responsibilities, project two is a six person project.
If you have an idea of what are the complex things that are going to be needed for project two that those mystery three people would be working on, put them down-- or in your case, four people.
So imagine that you are working on this project for project two.
Question?
Yeah, so for the vision statement things, are we-- for the vision documents [INAUDIBLE] write
Can you hold on one second?
I can't hear
For the vision document that we have to hand in on Monday, are we assuming that-- for the team responsibility break down, are we assuming that we'll be working on this--
Assume you are working on this as-- yes
Splitting up responsibility [INAUDIBLE]
Exactly right.
Now, what we're doing in class-- one minute to pitch your game as a project two contender.
You're going to come up here and just give us a quick elevator pitch.
What is the game?
Why is the game exciting?
How does the game do what it's going to do?
Based on this vision document.
And then you're going to give us a demonstration of the prototype, and that's actually going to happen in this big-- basically how we did play testing today.
You'll be demonstrating the game, and more people will play your game so that we can figure out which games we want to work on.
Question there and then question there
So does it have to be [INAUDIBLE]
Has to be paper.
This project one is paper.
Project one is non-digital
So when do we know which projects everybody is going to be working on
By the end of class on Monday, all of the teams will be formed for project two.
So project two-- again, it's a single player game, project two.
Some of you have made multiplayer games.
That's OK.
Imagine what the game is going to be like for a single player.
Are those other players AI?
Make them AI, but project two [?
as a ?]
digital game is only a single player game.
Do not turn in a multiplayer game for project two.
For project four, we're going to relax that, but for two and three, just to make sure we're getting some games in here let's make them single player.
Again, five minutes of game play.
Again, planning for randomness.
Is your game right now-- if project one doesn't quite fit that planning for randomness theme, that's OK.
In your vision document, what would you do for project two to beef that up to make that really pop out.
And again, project two-- those four engines that we worked on today.
Not photon, Phaser.
I miswrote that.
Those are the game engines we're working for project two.
So I kept you a little too late.
Sorry.
If you want to keep working in the room, I don't think anybody has it afterwards, so feel free to work.
Feel free to leave.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR 1: So today is all about you.
We are introducing Project Two.
We are starting off with the Project One pitch presentations, then we're going to have prototype demonstrations.
The idea is that everyone in class should have heard about all the games that were made and should have at least seen all the games be played.
it's going to be kind of difficult, but we're going to try to do that.
After that, we're going to form teams.
So we're going to choose a few of these games and form teams around them.
And then after that, assuming we have time left, we're going to work in our new teams.
You're going to rewrite that Project One vision statement and you're going to modify the paper prototype on volume here has a group just to kind of see what the mechanics were and what you're going to have to do for Project Two.
So starting off, Project Two-- it's another low fidelity prototyping, but this is more digital.
So you're going to actually make a digital game.
We're asking to create this digital prototype in a short time frame.
You had about a week to do Project One.
You're going to have about two weeks to do Project Two.
There will be some of the work done in class.
Unfortunately, it's not going to be the programming.
That's going to happen outside of class.
In class, we're doing a lot of the meetings and a lot of the project management stuff in class.
And we'll be walking through a lot of that.
You're going to be basing that game off of a Project One game.
And then yeah, we're going to integrate project management concepts into your process, and we'll be talking about some of that stuff starting Wednesday.
Extra goals-- you're going to be working as a team for the first time using a common game engine.
So hopefully some of the lessons from that game engine tutorial we did last week work out-- I think that was last week, yeah-- work out for you.
You're going to be doing version control, so it looked like we had everybody able to use some kind of version control with all those engines.
That's a good sign.
And then you're also going to be managing your time commitments.
This is the first time you're going to have six people on one project for this class, and then you're going to continue like that for the rest of the semester.
So coming up with strategies on how to work with each other, how to maximize the work you do in person, and what you can do when you're not next to each other, when you're working from your dorm room, from a study area, things like that.
So the deliverables for this one-- today, we are going to do this workshop, this team information workshop.
On Wednesday, we're going to be work shopping the product backlog, and you're going to be turning in to Stellar-- actually, I meant the vision statement.
But on Wednesday, by the end of class, we'd like you turn in the Stellar an updated vision statement.
So take the statement you were working on for Project One, modify it based on the new team you have.
You'll do a little bit of that today in class, and then you'll turn it in by the end of class on Wednesday to Stellar.
The following week, you're going to turn in to Stellar before class starts a product backlog based on the workshop you did the week prior.
And in class, we're going to talk about sprint back task lists.
So you don't know what a sprint task list is right now.
You don't know what estimates are right now.
You'll be working on that on Monday, and then each team will give a short presentation to us about the work that you did.
On Wednesday you'll turn in that task list you created.
And in class on Wednesday, we'll be playing all the digital games.
So you have to have something running by Wednesday at 1:00 PM.
It could be running and broken.
You might not get much good feedback out of it, but it must be running by Wednesday at 1:00 PM.
And then the following Monday, the project is due.
Yeah, really, really short.
On Stellar you're going to turn a whole bunch of stuff.
All of your games should compile out to HTML, either be playable on a website or use a website plug-in like Unity or Flash or whatnot.
So there is more details in the handout in Stellar.
I updated that this morning, so please take a look at that once your teams are formed.
So it's probably one of the first things you should do is take a look at those requirements.
Again, you'll need a written a written postmortem from each of you.
We'll ask for a design change log, which your first entry will be today.
You'll need an updated vision statement if anything changed from the one you turned in on whatever I said it was, Wednesday.
If anything's changed, you'll turn it in again.
And you'll be doing focus test reports.
So we're doing some focus testing, some testing in the next week.
And then, you'll also do a postmortem presentation.
And so, this presentation will actually be five minutes per team.
The requirements are on the handout, and we'll talk about it a little more later in the next week or so.
But basically, it's tell us what went right, tell us what went wrong, tell us what you learned.
We don't necessarily want to see the game being played.
We want to see what you've learned from the game, so highlight that with screenshots, with video, stuff like that.
All right, so this is what we're doing for the next-- we've got 15 slots-- so for the next 15 or so minutes-- elevator pitches.
Why are we asking you to do an elevator pitch?
If you can succinctly describe your game in one minute, it's probably pretty well scoped for Project Two.
Also, you probably have a handle on what you're actually making.
You know what the game's about.
You can describe the game to another person.
To us that means, great, that project is probably going to be a Project Two project.
If you have a little bit of difficulty describing it, if you go a little bit over time, we're going to allow you to go over time, but don't.
Why do I even tell you that?
I just messed everything up, didn't I?
But if there's some issues going on with that, that'll let us know a little bit about the scope of the project.
Don't show us the game, don't use visuals, don't use laptops.
You're in an elevator with an executive.
They said, fine, you got a minute of my time.
Tell me what you're trying to do.
Bam, convince them that your game is cool, depends on what your game does.
In this particular case, we want to know why it's about planning for randomness.
Remember, that's our design constraint for the Project One and Project Two.
What about your game is planning for randomness?
What is the core piece of game play in it.
Remember in your vision statement, in one of the versions, we asked you for 20 or 30 seconds of game play.
That's a great thing to put in a pitch.
So you've got all the material, just take your minute, go.
Afterwards, we're going to do demonstrations, and I'll bring that slide up later.
But basically we're not going to just rely on you talking about your game.
We're going to be able to see all the games being played.
And we're going to, again, try to get everybody able to see all the games being played if they haven't been played already.
And after that, we're going to try to form teams, which is really hard.
So that comes later.
All right, any questions before we move on?
Any additional comments from y'all?
Oh yeah, remember to check handout 2 on Stellar.
So yes.
by the end of class today.
All right, so first team is Lazy Beaver.
Again, you only need one person from the team to come up if you don't all want to come up.
Come up here, stand about right here, there's a microphone right here.
It'll capture you-- talk.
Lazy Beaver, come on down.
STUDENT 1: All right, hi everyone.
Our name is called Lazy Beaver.
Lazy Beaver is a survival game in which players must take on a chaotic world full of surprises.
These surprises come in the form of random environmental events such as hurricanes, predators, and even Ebola.
The player takes one action every day which either replenishes or depletes their finite resources with the objective of surviving the elements in order to build a set number of dams.
Since the environment is so unpredictable, the player must carefully plan their actions or risk losing by running out any of their resources.
This lets the player either choose to be risky or play it safe.
The digital prototype will maintain the same core gameplay of choosing actions, reacting to the environment, and preparing for disasters.
We will take the opportunity to carefully balance events and resources to tune the difficulty to appeal to all types of players.
Lazy Beaver should move on to Project Two because during testing players found our game both delightful and difficult.
Also, the technical scope of the game is fitting for Project Two's timeline, and the mechanics of the game are well suited for the preparing for randomness theme.
PROFESSOR 1: Dragon's Lair.
Come on down.
STUDENT 2: Hello, everybody.
Our game is called Dragon's Lair.
The game's point is you are an adventurer and you come into a dragon's lair.
You have 20 actions you take, but after every time you take action, the dragon destroys one of them.
The point is you want to try to collect as much gold as possible.
The game ends when all the parts are either discarded by you or destroyed by the dragon.
And then you count up how much gold you got at the end.
The game is lost, thought, if you run out the dragon's patience, because the dragon can only put up with you being in its lair for so long.
The game would be good for Project Two because it has a small scope.
The only thing we want to do for the digital prototype is get some artwork in there, a little bit of ambient music, add a timer so that people aren't taking too long at each turn.
And add a high score table because the game has high replay ability value.
So we want people to be able to see their achievements.
People found it fun and it was difficult, and so yeah, we think it's a good game.
PROFESSOR 1: Great, thank you.
Fight or Flight, come on down.
STUDENT 3: Why is it that in every Indiana Jones movie ever there's a temple that gets destroyed.
A temple gets destroyed because it turns out that running out of a temple is really, really fun.
Our game-- Fight or Flight-- forces the players to run from a randomly but not.
Environment that gets randomly destroyed behind or in front of them and balance their actions between fighting their pursuers and just running as quickly as possible.
It works for Project Two because it's fun, dynamic, and actually pretty simple at its core.
Thank you.
PROFESSOR 1: Plunder Winds.
STUDENT 4: Hi, everyone.
Our game is Plunder Winds, a pirate themed treasure hunting game.
Our game is based around three simple core mechanics that interact with each other.
The first one is a random encounter mechanic, which controls the probability of finding treasure.
And you can affect this as the game progresses.
An exploration mechanic, which tells you what the risk level of nearby encounters will be.
And then wind mechanic, which limits your moving and forces the player to make strategic decisions about where they move.
Our game's digital prototype would have the same core mechanics as our paper prototype, but it would be faster and more graphics so it'd be easier for players to pick up.
We could also add on unique encounters and abilities to improve replay ability.
So our project should on to Project Two since it's easy to implement.
And also, throughout play testing, we found that players found it easy to pick up, but also found it hard to master.
So you should come join our crew-- Plunder Winds.
PROFESSOR 1: Thank you.
[APPLAUSE] PROFESSOR 1: Blind Aliens.
WT.
STUDENT 5: It's a working title.
PROFESSOR 1: Uh-huh.
STUDENT 5: Hi, everyone.
We're Blind Aliens.
Our game is Blind Aliens.
The premise of the game is aliens, blind aliens, have invaded the Earth and you are one of very few surviving humans.
And the game is very simple, it's in a very small room.
There's aliens that come in through random gates around the edges.
And they randomly roam around searching for you, but they can't see you because they're blind.
So you have three options.
You can sneak away very slowly.
Or you can run very fast, which is loud.
Or you have a gun, you can shoot at them, which is also very loud and you'll be risking getting discovered and eaten alive.
So the game should move on to Project Two because it's very simple.
During testing, we found that people found it very engaging in that it was not boring at all.
So, yeah.
Thanks.
PROFESSOR 1: Next up is Live On.
STUDENT 6: Hi, everyone.
Live On is a survival themed game that takes place in a hexgrid based world.
The player needs to stay alive and move from the start to the goal.
There are four type possible terrain to raise-- river, mountain, plain, and forest-- each with simple costs and benefits.
The key mechanic for the game is that at every turn, the player needs to choose which tile to move on to next based on what information he has available.
So the player only has limited vision on one hand is important to choose grids that will lead to a good location for the next move.
On the other hand, the player need to plan for the [INAUDIBLE] tiles that he can't see at the moment.
The game will be a good project for Projust Two because, in my opinion, not only does making plans based on incomplete information and random events lies at the heart of the games mechanic, we implement a good set of tradeoffs and balances between the different terrains that serve to offer the player meaningful, consequential choices of which path to go.
In the digital version, in addition to the core game play, we seek to add randomized map generation and multiple characters with different abilities to further enhance the gameplay.
We'll stay with a 2D game, the grid representation of the world, and turn based actions, as opposed to continuous work over the time game plan.
Thank you.
PROFESSOR 1: Beaver Evolutions.
STUDENT 7: All right, so Beaver Evolution is exactly what it sounds like.
You have a colony of beavers, and you have to fight against nature and evolve your beavers over generations to survive against the trials and tribulations that nature will throw at you.
The core mechanic of the game is based around making a choice between three different options for every generation.
You can either build your exam, popular beavers, or choose an evolution traits help develop you colony to fight against the natural disasters that will try to hinder your progress.
Our game is going to be a great game for Project Two because we have a lot of ideas for implementing it in a real time world, currently as a turned based game were you make a choice and then you respond to a natural disaster.
We think that it'll be great to implement a real time game because you will be able to respond to natural disasters and allocate your beavers to either work on you dam or work on evolving.
And you have to have quick reflexes in order to survive through the game and win the game.
Thanks.
PROFESSOR 1: Comcastic.
STUDENT 8: Hello.
Comcastic is a cable monopoly simulator where you are trying to place your oddly-shaped service centers into an existing city landscape.
Your goal is to make as much money as possible while caring about your customers' satisfaction.
STUDENT 9: So in the digital prototype we're basically going to have a grid where all the homes and businesses are laid out, and the user will see the inconvenient-shaped pieces for your service centers.
And then, when they select a center, they will see how many people it can serve and the price that it'll cost them each month.
And then, they will need to place these on the board.
STUDENT 10: We think this is a good project for our Project Two because we've found that playing the game and creating the game is both very simple.
And we found that it was really engaging.
We had a few random testers kinda describe it as a strategic version of Tetris, basically.
PROFESSOR 1: Sparkly Redemption.
STUDENT 11: Boosh!
You're a lady with only one arm, and your other arm is a gun that shoots things.
You're a disgrace in your nation and your only chance at redemption is to get all of the sparklies.
However, the place where all the sparklies are are filled with monsters.
And depending on which sparklies you pick up, sometimes you get better arm gun powers, but sometimes the monsters change their behavior and you have to adapt to that, or just be strategic about picking up your sparklies.
Our game is super cool because, even with our simple paper prototype version, people had a lot of fun with it.
And the sort of game that's pretty easy to implement at a most basic, just fun to play stage.
But then it's good to add stuff on top of it, like extra monster behaviors and super shooty arm powers.
Plus you're a lady with one arm and a gun that shoots thing, which I think is cool.
PROFESSOR 1: Modgi Dice.
STUDENT 12: Hey, everyone.
So Modgi Dice is a math puzzle game like Sudoku or 2048 where the user navigates a dice around a square grid, rolling the dice one side at a time, and adding that side of dice to the square.
The cool mechanic of this game is that each square is modular 7 and the goal of the game is to reset all of the squares to zero.
This game will work well for Project Two because it is a fast paced mover game kind of like 2048.
And you can move those around very quickly while also having to plan out your moves.
Thank you.
PROFESSOR 1: City Evacuation.
STUDENT 13: In City Evacuation, your goal is to escape the city because there's an earthquake in your city.
And you have to do that while saving three friends along the way.
And what will happen during the game is you might get hit by an explosion, or abridgement fall down, or you might have some [INAUDIBLE].
STUDENT 14: As far as the digital scope, we're going build a 2D game that's dependant on your dice roll and will generate random disaster cards that will come up and the hardest part will be the graphics.
STUDENT 15: We think this is a good project for number two because there's a lot of randomness, but we also give the player tools to plan and strategize around that.
Additionally, people thought it was really fun to play.
And though it's sort of simple to implement at the beginning, we think there's amble challenge in the graphics, the sounds, and the animations that can be built on top of that framework.
PROFESSOR 1: Thank you.
Shoutkey.com/doorway?
STUDENT 16: All right, the next 12 hours, our game is called Shoutkey.com/doorway.
Here's the idea-- you're playing World of Warcraft-- sorry, Warcraft 3-- and you see that guy with the second and a half ping, and you get really jealous.
Man, I wish my ping were that high.
So then you start encouraging your pet to chew on your ethernet cables, get those dropped packets.
You really want that slow, unresponsive feel.
So we take that desire and mix it with a platformer to make a really frustrating game where as you play it, your controls become increasingly unresponsive.
And I think it's hilarious and fun.
Yeah, it's fun.
Yeah.
Shoutkey.com/doorway.
PROFESSOR 1: Thank you.
Lost Underground.
STUDENT 17: So, Lost Underground.
You are a traveler and you fall down a hole, and you get lost in an underground mine.
So the goal of the game is, equipped with some bombs-- infinite amount of bombs, because that's just how games work-- you're supposed to escape from the underground by planting bombs and trying to move through barriers that you can clear.
The randomness comes in with the addition of some of ghost bombs that will be floating around.
And they will not chase you, but they would go through you and would explode at random moments.
So you have to plan around getting to the end of the level and also not dying before you get there, which is common in a game.
This is a good project because it's simple and it's sort of reminicent of Bomberman, which is a great game.
So, yeah.
PROFESSOR 1: Thank you.
Dice Traders.
STUDENT 18: Dice Traders is a multiplayer card game where you play a card combo in order to score points.
Importantly, you can trade your cards with other players in order to improve your combos.
But if you do that, you might be giving your opponents what they need to win.
And so this introduces a lot of strategic considerations about things like what your opponents score is, how large there hand is, and so on and so force.
And you have to keep all of that in mind when you're playing.
And our playtesters really enjoyed the complex strategy that arose from these simple elements.
It would be good for Project Two because the mechanics are very simple, which would make the game easy to implement.
However, it is designed as a multiplayer game, so it would require some sort of AI.
PROFESSOR 1: Thank you.
And, last but not least, Gravity Shift.
That is the last one, right?
Nobody else?
All right.
STUDENT 19: So our idea was for a puzzle platform with the following characteristics-- it's a basic platform, you have a guy on one side trying to get to the other with a bunch of platforms.
Unfortunately for you, the platforms are all spaced so that it's completely impossible to reach them fortunately for you, you can place a number of blocks anywhere you want on the stage.
Unfortunately, again, they all cost points and you'll lose points if you place them.
And every time you land on a space, it'll disappear.
The twist is, once you get to the end of a stage, everything will rotate 90 degrees.
All your platforms will become walls and vice versa and you'll have to make it again.
If you weren't ahead, then the second part of the stage will not possible, which is why it requires the actual thinking ahead to solve the challenges.
The reason that this would be a good idea for Project Two is because it's a pretty basic idea, but it's actually a deceptively difficult game, and it's really easy to make a stage that's really hard to solve.
PROFESSOR 1: OK, that was really good, you all kept to under a minute.
Next up, we're going to take five minutes, set up your games, try to put a lot of space around where you're set up.
I know it's going to be hard.
But put enough space around where you're set up so people can watch what you're doing.
Take one of these Post-its and-- how the hell do these work?
There we go.
Oh, yeah.
Take one of these Post-its, put your name on it wicked big, planted down on your table so we know what we're looking at.
So set up your games, all the stuff that you might need is up here in these two boxes.
[INAUDIBLE] The goal is to make the [INAUDIBLE] geological center.
So you roll the dice-- PROFESSOR 2: All right.
It's about time, so thanks everyone.
And first of all, all of you have playable prototypes.
So for Project One go you.
You were totally there.
A couple of things that I wanted to say about all the projects-- actually we expected that a lot more projects were going to be vastly out of scope than we actually got.
A lot of stuff that we saw here could actually just be fine for Project Three.
A lot of games that we saw here, once you're a little bit more comfortable with the technology and the build process-- and in some cases, it's going to be just the efficiencies of knowing the people on your team a little bit better-- you'll be able to execute any of these games for something like Project Three.
But we would like to actually guide this next step where we move from Project One to Project Two a bit of a better.
So we are going to cut a couple of projects, just to make the team building process go a little bit faster.
Otherwise, having to select which one of the 15 projects that we're going to carry on is just going to take way too long than we have class session.
But we didn't do it completely arbitrarily.
We're going to talk about our concerns.
If you're one of those games that we're going to cut, and you really, really want to do that for Project Three, I want you to use the time over Project Two to think over your design and see how you will address all of these concerns while you're working on somebody else's project too, because we saw a lot of really, really good stuff here today.
Let's see, so a couple of games that we're cutting.
Gravity shift we are going to cut that because the randomness isn't really integrated into the design at this point in time.
Again, think about it, figure out how you would do that for Project Three.
I actually had a concern about the UI of the game, because a lot of these games-- sure, some of you are working in Unity 3D-- that seems to be a game that UI wise seems to suggest that it should be a 3D game, because you need to know what the sides of the dice are before you roll things.
But how do you convey that to a player in a way that's not going to have them to deal with a camera?
And you could also do it in 2D, but then how to convey that information to the player?
[INAUDIBLE].
PROFESSOR 2: Yes
[INAUDIBLE].
PROFESSOR 2: Oh, sorry.
That was the feedback I had for Modgi Dice.
My mistake.
Sorry, I did, in fact, get that mixed up.
For Gravity Shift, the UI problem was how do you convey to your player which way things are going to shift next?
Right?
Is it relative to the player, or the relative to the level?
When the gravity shifts, are you rotating the entire level or are you rotating the players.
Now the player's falling sideways on the screen.
These are all like big UI challenges.
Assignments we actually happens to be about UI, so that's a good time to be able to tackle that.
So think about that.
Sorry-- sorry to Modgi Dice.
PROFESSOR 1: For project three, though, for that one, still bringing in the randomness and actually bringing in the design constraint for project three will be important for that one.
PROFESSOR 2: Yep.
OK, then for Final Flight and Dice Traders, we have very similar feedback.
Right now, your design is specifically-- the fun of your games are really in the multiplayer right now.
That is absolutely fine.
In fact, they are really, really solid game designs.
The problem is that project two is really short and AI is time-consuming.
It's not necessarily hard.
It's just going to take a lot of time to work on.
So we're going to say let's not do that on project two.
You might want to save that for our project three.
Finally, for two more games, for City Evacuation and Doorway, actually, technically, both games are feasible.
I am actually fairly confident that the games that you showed today could be done in the time span of project two.
However, I feel that a lot of the fun in those games comes in multiple levels-- being able to get different environments and different scenarios.
And that's going to be very time-consuming, which you don't have time for in project two.
So it's an evaluation.
It's kind of like the size of your level and all of your elements that you've got in it.
It's just going to take a lot of man hours just to be able to generate one map, let alone multiple maps.
Doorway, it seems to me that playing through one round of the game on a computer can go through pretty quickly.
Actually, where's Doorway?
I like to make eye contact.
Yeah, there.
OK.
It seems to me like playing through a level, at least the first couple of levels, could go through pretty quickly, which is fine.
Players like that, especially when they're wrapping up right at the beginning of the game.
But that means you have to generate a lot of levels.
And that's just a lot of time that you don't have.
So for the rest of the projects, we're going to put them up around.
And we are going to ask you to put your names using the little Post-its to show which team you're signing on.
But I still have more feedback, because the other games have challenges that you're going to have to overcome too.
Dragon's Lair, there, so much information, so much information for players to have to understand what's going on.
How are you going to convey that over to a player?
The good news, your information's relatively static.
It's not like you have numbers changing in the middle.
[INAUDIBLE], you do have numbers changing in the middle of the game-- so many numbers, so many little variables and factors.
So how are you going to convey all of that information to a player?
So you have a UI challenge there.
Beaver Evolution.
There's a-- Beaver Evolution?
Beaver Evolution.
You have a range of possible bad things that could happen to a player.
How do you explain that to the player before it actually happens to them so that you can sort of like anticipate and plan, right?
You could say, oh, well.
They'll play it through once, and then they'll die, and then they'll restart the game.
That's not a great solution.
Think about something better than that.
Comcastic-- Comcastic, very specifically is a UI problem, because how are you going to show the players what they're about to put down on the screen, and then show them the consequences of what they've just put down?
Because your game is all about how do I rotate this thing and then figure out where it goes and what's going to be the outcome, your problem's mostly user input-- how to get user input in game.
And Modgi Dice, I already mentioned it.
How do you do this in 2D?
It's-- yep?
Is that a hand?
Oh, sorry.
I thought you were putting up a hand.
How do you explain to the player if you rotate it, the dice, this way, this is a number that's going to be coming up, versus that way?
If you're going to do it in 3D, fine.
But 3D's at least twice as long in terms of development time.
So you have to keep that in mind.
For Blind Aliens and Sparkly Redemption, you're going to need some art just to be able to convey to players what the heck is going on in your game.
The good news is that as long as you have some art, I think you can make it work.
But that's going to be your bottleneck.
That's going to consume up all of your time.
Things that we feel that can probably work that we're fairly confident is within the scope of this project include Lazy Beaver, Live On.
Lost Underground.
Some of these games resemble other games that exist, so you already have kind of like a template to work on.
For some of the games, the paper prototype really kind of already gives you a pretty good idea of how the computer game is going to work.
So I'm not too worried about-- if your game just ran on text, it could still be comprehensible.
So I'm not so worried about that.
But so these are the games that we have short-listed.
Not all these games necessarily become project two games, because we've got-- one, two, three, four, five, six, seven, eight, nine, 10-- 10.
And we're hoping for teams of six.
PROFESSOR 1: Teams of six.
PROFESSOR 2: We do not have 60 people in a classroom, OK?
So-- PROFESSOR 1: We have about 47 last time I checked.
So about seven teams?
PROFESSOR 2: About seven teams, so that means three of these projects will have to get cut.
And we'll probably just cut them based on whether they have enough people on the team, which means then the people who have signed up on those teams will have to find some other team.
Now, that's going to be really obvious.
Once you have your Post-its up there, you're going to see how many people have signed onto each team.
And we want you to try to get to a team of at least five.
Six is good, OK?
PROFESSOR 1: So team formation.
Come on down.
Put your name on a Post-it.
You do not need to choose the game you were on previously.
Once you've put your Post-it down, sit back down.
Sit down.
Come on back.
Come on back.
We'll read off the names and we'll see how this is going.
Come on back.
Come on back.
Come on back.
Did you put your name down?
What's up?
PROFESSOR 1: Did put your name down?
I did, yeah.
PROFESSOR 1: Then sit back down.
Sit back down.
Sit back down, back down
Can I add my name if there's already six on something?
PROFESSOR 1: Yes, you can.
It just means you're going to get moved or somebody's going to get moved.
OK.
If you're not decided, you can sit down with your name, too.
All right.
Everybody's down?
All right.
Lazy Beaver, sorry to say.
Maybe we'll see you in project three.
We've got three, three, three-- no, wait.
That's more than three.
Three, three, three and two.
And we've got one, two, three, four five; one, two, three, four, five six, seven.
The Future-- maybe you're moving.
One, two, three, four, five, six, seven, eight, nine-- oh, my god-- ten.
We're cutting that with two.
One, two, three, four, five, six.
You're OK. One, two, three, four, five.
All right.
So Blind Aliens is set with Roy, [?
Mikael, ?]
Miriam, [?
Shalam, ?]
[INAUDIBLE], and [INAUDIBLE].
Apologies if butchered the name.
Blind Aliens, you're done.
Where there any other sixes that I'm missing?
No.
OK, so the big ones, Comcastic-- one, two, three, four, five, six, seven.
We need at least one to leave.
Is that you, Sam?
Maybe it's Matt.
Julia?
Sabrina?
Anderson, Sean, or [?
Tage?
?]
Modgi Dice-- one, two, three, four, five, six, seven, eight, nine ten.
Devon, Peter, Caleb, Megan, Jordan, Kevin, Jeremy, Bennett, Derek, Harry.
Derek, I said big letters.
[LAUGHTER] You're lucky I'm wearing these.
Anybody I just named, come on down.
Move.
You have a couple minutes to discuss down here.
So Modgi Dice, come over here.
OK, great, Comcastic, taken care of
[INAUDIBLE] has a problem that it's just so cool.
PROFESSOR 1: You'll get to play it
[INAUDIBLE] us.
Look
[INAUDIBLE] on the ground.
PROFESSOR 1: If anybody on these other teams are like, woo, come over to our side
[INAUDIBLE].
PROFESSOR 1: Why?
Why do you want people?
We're all very nice people.
[INTERPOSING VOICES] PROFESSOR 1: All right.
One, two, three, four, five, six, seven, eight-- I need two more people to leave.
Get out
We love you.
PROFESSOR 1: We need some chants.
Come on.
All right.
Modgi Dice, you are now one, two, three, four, five, six.
You're golden.
Sparkly Redemption, you've got five.
We're going to let you stand for awhile.
Lost Underground, you've got four.
Beaver Revolution, you've got one, two, three, four, five-- ooh, you've got six.
Yay.
Making my life easy.
Plunder Winds, one, two, three, four, five.
You're cool for now.
Dragon's Lair, you've got three.
[SIGH] Live On, you've got three.
Lost Underground, you've got four.
We need one of you to self-destruct.
Come on down if you want to self-destruct, or I'll blow it up for you
Roll a die?
Roll a die?
PROFESSOR 1: You want me to roll a die?
Yes.
PROFESSOR 1: Oh my god, I love this.
This is the best ever.
Ooh.
We've got one, two, three that we're rolling.
We've got a 6-sided die.
How should I do it?
One, two, three, four, five, six
But Lost Underground has four people.
PROFESSOR 1: What's up?
Oh, you're right.
One, two, three, four, five, six.
Four.
One, two, three, four, five, six-- Dragon's Lair.
Sorry
It's chill.
PROFESSOR 1: It's cool.
You could do it for project two.
Come on down.
Move yourself
Were you moving [INAUDIBLE]?
PROFESSOR 1: What's up?
Aha.
All right, that's six.
It's done.
Sparkly Redemption, you've been redeemed.
One, two, three, four, five.
One, two, three, four, five, six.
Plunder Winds, you are all set to go, to sail away.
How many more of these do I have in me?
We do have uneven numbers.
Life is unfair.
Randomness, people, randomness.
One, two, three, four, five.
One, two three.
Choose another team, any team, please.
Live On, choose another team, any team please.
[INTERPOSING VOICES] That's up to them to decide.
But they can do it if they want.
I don't recommend it, by the way.
But you can choose any one of these that's on the board, too
Wait, anything on the board?
PROFESSOR 1: You can choose anything on the board
Wait, not [INAUDIBLE].
PROFESSOR 1: One, two, three, four, five, six, seven-- you've got seven choices, so many.
No, too many, too many.
No
I got there first.
PROFESSOR 1: He got there first
I want to join this one.
PROFESSOR 1: Were you this one?
Uh-huh.
PROFESSOR 1: Sorry.
You were second.
You were so close.
So don't choose this one.
And don't choose this one.
One, two three, four, five, six, seven.
You've got Sparkly Redemption.
You've got Blind Aliens, Beaver Evolution, Plunder Winds, or Lost Underground.
All right.
We are done.
[APPLAUSE] What are we doing the rest of the day?
We're doing-- PROFESSOR 1: You had two for project one.
You've got five for project two I think you're cheating somehow, myself.
Yes?
PROFESSOR 3: I would encourage, actually, we ask people to move around because [INAUDIBLE], and we just give people spots to regroup in [INAUDIBLE].
PROFESSOR 1: We meet as teams?
PROFESSOR 3: Meet at their team tags.
And then find a place in the room to sit down, re-discuss vision statements, and start planning out [INAUDIBLE].
PROFESSOR 1: So no.
Huh, you're right.
I don't have a slide for this.
All right, so we're going to do exactly as Sara said, which you did not hear but I did because I was listening.
Sparkly Redemption, hang out over here.
Actually, before you move, what we're going to do, come down here.
Meet each other.
Shake some hands.
Get to know each other.
Sit back down.
You've got one hour left today.
Talk about your schedule.
Talk about your vision statement.
Look at your prototype.
Work on your prototype some more.
Re-write your vision statement, and talk about what it actually means to make a digital version of this game.
So Sparkly Redemption.
PROFESSOR 3: And whatever you do, do not forget to exchange contact information.
PROFESSOR 1: Oh, my lord.
PROFESSOR 3: Email-- PROFESSOR 1: Were you not going to give each other your email addresses?
PROFESSOR 3: Phone numbers, whatever.
PROFESSOR 1: Modgi Dice.
Blind Aliens.
Beaver Evolution.
Plunder Winds.
And Comcastic.
This is the safe, right?
PROFESSOR 2: That was fast.
PROFESSOR 1: Yeah.
No, it's one of the things where you might not be happy, but it gets done.
PROFESSOR 3: Hey, I think you ran that really well, by the way.
PROFESSOR 1: Thank you.
PROFESSOR 3: That was a hard team formation, especially when people had to jump ship.
PROFESSOR 1: I wanted to just put one of them on that one, but I didn't want to be the one placing people in teams.
PROFESSOR 3: Yeah, and I think that's the right way to do it.
PROFESSOR 1: Yeah.
What's up?
What should we do with the stuff--?
PROFESSOR 1: Oh yes, someone just asked.
What do you do with the old prototype that didn't get chosen?
Take pictures of it first.
Set it aside.
Maybe put it in an envelope.
And then maybe it might come back for project three, so don't forget about it.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Welcome to class.
Some very quick announcements.
We had the date wrong on the vision statement.
The vision statement you guys worked on on Monday is due today.
We had the date wrong in Stellar.
So it will not be late as long as it's turned in by tomorrow morning.
We have adjusted the date on Stellar so now it shows the real date.
But since we made that mistake, it's not late until tomorrow.
But we will expect them all in by the end of the day today.
Secondly, the things that are due next Monday, you will be expected to be turning in a product backlog.
We're going to work on creating product backlog in class today and then you will have some time to refine and think about it as you're working on the project over the weekend.
One thing you should be working on along the way, remember we require a design change log for this project as well.
So you should have already started it because you probably made some design decisions in your meeting on Monday.
So don't forget that and don't try to do it at the end.
I think that's all the announcements.
Yeah?
OK.
So today's class is going to be, we're going to start talking about project management.
What we are trying to do with project management is give you the just in time version of it.
Which is to say, we're trying to give you the pieces of project management you need for the project you're on right now in the state it's on.
And we're going to be just in time project management for project two.
And then on project three, we're going to expect you to use all those pieces from the beginning and all the way through on your own so that you'll have had a walkthrough on project two, a practice run on project three, and then on project four hopefully you will need it and it will all work.
That's sort of the philosophy.
So we are giving it out to you in bits and pieces, and I acknowledge that.
I suspect that between today's lecture and Monday's lecture, we will have covered sort of all the things we expect you to be doing as far as project management.
But today's lecture's sort of the first half and then Monday's will be the second half and tidying it all up.
So we're going to do a brief history of how project management was done in the past in the software industry.
We're going to talk about agile, which is where many software companies are using now or moving to with various levels of success.
Not all of them, unfortunately.
Then we're going to start getting into the nitty gritty of the agile management system that our lab is most familiar with, which is Scrum.
And we'll start walking through all the aspects of Scrum.
We will concentrate today on product backlogs.
We'll take a break from our lecture and you guys will have the opportunity in about a half an hour to make your own project backlogs and then do very quick presentations of sort of, OK, here are the top features that we found in our product backlog when we sat down and actually thought about our game.
Then unfortunately I will get up and start talking again.
We'll go over the meetings in Scrum, how they run, and what they are, and why you do them, and their level of importance.
And when we hit the end of that, we'll have you run a small one yourself and then we will actually get to use the rest of time in class for working in your groups and working on your projects.
OK?
Any questions?
And let's go.
Now that we have the class interest guides, I can mostly skip that slide.
OK.
So you all took this class to learn how to create video games.
We, however, created this class to teach project management in video games.
Fortunately, we're all going to get what we want, because you really do need some good project management skills to successfully make video games.
The better your project management, the better your chances of coming out with a good game.
It won't guarantee it.
You need a lot of other good things as well.
But I can pretty much guarantee that without any project management, your attempt to make a game is going to fail.
So do try to listen.
I know this is some of the less interesting stuff we cover, especially coming after the high of prototyping and making games.
But this is just as important as prototyping.
So what is project management?
Let's start by explaining what the heck I'm talking about.
Project management is the work on the project that's devoted to planning, organizing, securing, motivating, and controlling the resources to successfully complete a project.
You've all heard of fast, good, cheap, pick two.
Most projects are actually lucky if they manage to choose one.
And it's project management allows them to figure out whether they're going to be fast or good or good or cheap or whatever.
Project management is all about predicting.
I knew the definition up there says controlling, but you really can't control a whole lot of things.
What you can do is you can make some estimates, make some predictions, plan for the risks you're going to run into and then deal with them.
So it's really about predicting what you're going to need to do when you're going to need to do it and how you're going to need to do it.
If you follow game and other large software project releases, you know how rarely they actually manage to come out on time with all of their features.
Games are especially famous for coming out late, for being over budget, and for often under delivering the shiny things that they had mentioned.
Can anybody think of a couple of games recently that have done that?
No?
[INAUDIBLE]
Yep.
OK definitions may vary.
But yeah, it is really, really common for games to come out late and over budget.
Here's another one.
Anybody name a tripe A game that shipped on time?
I'm not actually sure I can do this one.
With all the features that they predicted.
Can anyone name one?
[INAUDIBLE]
Call Of Duty.
OK. That's impressive.
OK, so how do you manage to plan what you're going to need to do, get all of your resources in order, have them all in the right place at the right time, and get them working together?
The old model of doing this, which is actually the model that I worked at in several studios and I've seen several other studios that use it.
And it doesn't work very well.
But sort of the model that things started with in the industry and grew from is waterfall.
And I want to talk about that because it's where some studios still are and it's where a lot of software projects still are.
And so it's good to understand what the history of project management is so you understand some of the problems that were found in that model that the agile model is trying to fix.
Agile has its own problems and we'll talk about them.
But let's start with waterfall.
So the very traditional tool is waterfall, where you prepare for every stage.
You sort of finish every stage 100% and then you move on to the next stage to see what happens.
First you need your spec.
You need your requirements.
Which in game design is sort of your concept and your design phase.
You figure out what you want to make, you figure out what all the features are.
You also figure out other constraints like what is your budget, how long you're going to take, what can the people on your team actually do.
If you don't have a physics programmer, you probably don't want to make a physics game.
All that pulling all those constraints together to make a design that is reasonable that works with the things that you think you're going to have to work with.
That concepting and design is going to flow into pre-production where you sit there and you made a whole lot of assumptions in design and concept.
And in pre-production, you test them in very small scale usually.
You figure out if people can actually do the things they said they were going to do.
You figure out if the software you were going to buy is actually going to work or not and all those other details.
Using the data you gathered in pre-production, you create a whole lot of big fancy schedules.
And using those schedules, you hire up your team and you go into production, and you implement everything and you get it all built.
Well, you get most of it built.
Then in game processes, you go into alpha when you've got most of your features in.
And that's when you usually bring on a few small testers to do a little bit of testing and make sure your code is working the way you expect it to.
Then when you've got all your features in, it's mostly working, you go into beta.
You bring on more testers.
In big studios these days it's often common to have open betas.
If you're looking at some sort of massive game where you're not going to hire 100,000 testers.
So what you do is you have an open beta and you let people play the game for free in exchange for testing it.
That hopefully nails out the rest of your big problems and then you ship it.
Does anyone see the big problem with games in this model?
It's a little subtle, so it's easy to miss.
It's OK if you don't.
Yeah?
[INAUDIBLE]
That is it.
Almost all of the testing is here at the end at beta.
It's a little late to change your design if you're doing all of your testing at the end.
if you're doing all your testing at the end, the team can't respond to it.
They can't fix things.
And that is where a lot of schedules get stretched out and a lot of budgets get stretched out.
Because all of the planning you had goes out the door somewhere around early alpha and you have to start reworking your whole system.
The problem here is, of course, you can't predict fun.
You can't design an engaging experience without interacting with the people who are going to be using it, which means not just the people on your team, not just the small group of pre-prod.
I mean, in waterfall they do do testing.
In pre-production you test, in design, you test.
But you're almost always testing with a very small group of people.
The other designers on your team, the other people in your company.
You're not going for any big testing.
You're not going for any user testing.
You're not finding out how the people who are actually going to use your product are going to react to it.
So in many ways it's doomed to fail.
There's not a lot you can do about it.
None of these problems are unique to games.
It's why big software projects often run over budget and run into a lot of feature creep.
But games have the really unique problem that their primary feature, their primary goal, is to be interesting and engaging to use.
If you are looking at business software, and it's hard to use and it's annoying, but it does all the calculations you need and it's the one that the tech department bought and deployed, you're going to use it.
There's just no getting around it.
But nobody has to play Call of Duty.
Nobody has to turn on a particular game on their phone and play it.
So if a game fails in that sort of user connection, it has completely failed.
There's nothing else you can do.
It really kind of comes down to, if your users aren't happy, it's not a good game.
Your design changes, your schedule changes, and your project management has now-- all of the time you put in project management has mostly a waste of time.
So what do you do instead?
First of all, you think about building design iteration into your project management model.
You assume that you're going to be iterating and changing your design all the way around.
That is what the agile project management movement I guess is the right way to put it was invented from.
There are actually several different types of project management that call themselves agile and there are several different techniques and tricks that call themselves agile.
The one thing that they have in common is they try to follow the agile manifesto, which is individuals interactions over processes and tools.
Does anyone want to-- let's just go ahead.
Working software over comprehensive documentation.
Customer collaboration over contract negotiation.
And responding to change over following a plan.
So for the first one, individuals and interactions, means it's better to get up and talk to someone than to send a whole lot of change requests over.
Working software.
That's why we ask for paper prototypes and vision statements instead of a big design document.
It's why we want to see what it's doing rather than having you tell us what it will do.
You don't really have so much customer collaboration.
So you don't even have to worry about signing contracts and getting people to observe milestones.
That's less of a problem in a class specifically.
And responding to change over following plan means that if you discover what you're doing isn't working, you sit down and you figure out how to fix it rather than continuing to go down the rabbit hole
I just want to state that everything on this are good things to do.
Even the things that are sort of low in priority.
It's good to have processes and tools.
But it's more important that you have good interactions between the individuals in your group.
Say if you are saying, no, I can't actually get up and talk to this person because we have a process that we have to follow.
Then you're now prioritizing incorrectly [INAUDIBLE].
I guess the [INAUDIBLE] a lot of your classmates are going to be people who are going to play your games.
And you [INAUDIBLE] collaboration is important in that sense because you just replace that with work with your players.
Work with the people who are going to play your game.
And yes, we did specify this is what we want you to see.
This is what we wanted to see in the game that they are going to create.
That's my contract.
Well, that's less important than actually coming and talking to us about problems that you might be having and [INAUDIBLE].
Don't just use our syllabus as a laundry list of things that you have to do [INAUDIBLE]
Yeah.
Yeah.
Thank you.
So the model of making a software project.
I keep wanting to say game, but this is for any software, actually, agile model.
Is that the project start you've got your vision and your idea what you're making.
And then on every iteration, you go through every step, essentially, of the waterfall process.
You do a little bit of design, a little bit of detailed design.
You do a little bit of implementation of that particular chunk of the design you've just created.
Then you spend some time testing it to make sure that, A, it works the way you expect it to, and B, it's achieving the result you want to get.
I.e.
the feature does what you expect it to do as far as improving the quality of your project.
Then you spend some time reviewing what your overall project is doing.
Is it still going the same direction you thought it would be going at the start of project?
Is that the direction you want to be going in?
And then you can modify your plan accordingly and you start over with another iteration.
The big savings you have here and the way in which agile management tends to save time over waterfall management in this project is you spend less time designing features that people don't want and more time working on the features that people do want.
And that's actually where the big time savings is.
Agile can't make you write code faster.
What it can do is make sure that the code you're writing is the code you want as opposed to the code that you end up cutting at the end of the project.
Some caveats before we start actually talking about how this applies to you and your project.
Agile is good, not perfect.
Which means once you've learned the tools and processes for agile development, you will want to keep adjusting and improving them.
You may discover that we're going to give you a set of outlines and a baker's list of things to follow and a recipe to do project management.
And you may discover that following some portions of this recipe does not work well for the group you're in.
Well the right thing to do is to set those pieces aside and figure out a way to work around it.
A good agile team is always adjusting their processes and trying to improve them.
Agile assumes that all of your developers are interchangeable, which they're not.
Different programmers have different skills.
And on a game project especially, you have people who can do art, people who are better design, you have people who are better at x or y or z.
They're not all interchangeable.
Which means that it takes more work to set up what your task set that you can get done in any particular period of time on a project than pure agile was.
Agile says, OK, you've got this list of tasks.
You've got enough time to do them.
They'll get done.
When you're making a game, you have to look more closely at the dependencies and who can get each of those tasks done to make sure that you have not in fact arranged for two of your team members to do nothing this sprint and two of your team members to do twice as much as they actually have time for.
Sprint is the iteration cycle.
I will explain it again later.
I'm trying not to jump ahead and use jargon I haven't introduced, but it's sometimes a challenge.
Let me know if I use a phrase that you don't understand, because I will explain it or redefine it for you.
Finally, and this is a problem you're likely to run into a little bit less in class, but you are more likely to run into when you go into the outside world and you work on teams with different people.
People trained in different disciplines communicate differently.
They have different terminology.
On game teams, this is really hard.
Because you will often find, and I'm not actually trying to stereotype all programmers talk like this or all artists talk like that.
But it is the case that the people who do programming and the people who are creating your assets, be they art or design or audio, often will talk past each other on specific problems.
The artist will not be able to explain why they can't meet the programmer's technical requirements and the programmers will be unwilling to figure out why it is the artist can't meet those technical requirements.
They've given the technical requirements.
Why don't you just make something that works in it?
Since agile depends on really good communication and the willingness to solve problems like this, this can create a huge tie up, especially in an agile team.
It can pretty much stop anything from getting done.
So that's another big red flag to be aware of when you're using these methods.
OK.
So there's more than one agile system.
The one that we use, that we have the people work in our labs [INAUDIBLE] use, is called Scrum.
Is anybody familiar with Scrum?
Yay, there's a few people.
Anybody used it at internships and things like that?
OK. All right.
So you guys can help your teams out a little bit hopefully.
So Scrum has three buzz words, and I may as well go ahead and call them buzz words, because that's really what the are, that are supposed to define it and let you understand what it is about.
They are transparency, inspection, and adaptability.
You can think of them as Scrum's vision statement.
Much shorter than a document.
Essentially, if you are doing Scrum right, all of your decisions and processes are transparent.
Everyone on the team who needs to be involved in a decision is and anyone on the team who wants to understand why something was done can.
No one on the team should be able to say at any point, well, no one told me about that.
Because it's both the team's responsibility to share information and each team member's responsibility to stay informed.
Everything done by the team is inspected at some point.
Decisions, processes, team actions, deliverables, code.
The team thinks proactively about what they're doing, how they're doing it.
And if the team discovers a problem, for example, the build keeps being broken every night, the team doesn't just say, oh, we'll work around it.
Instead it takes responsibility figuring out how, why, and fixing not just the broken build, but the processes that led to the broken build.
And finally, which feeds back into the other two, the team is willing to adapt any processes and decisions to the reality they're actually dealing with.
When they find a structural problem, for example, how does art assets getting in the game?
Well right now we're having the programmers hand load them in and we're losing six hours of programmer time every week.
Instead they sit down and say, well, how do we fix this?
Do we take 12 hours next sprint to code up a tool that allows the artist to put it in, and then we'll save all that time going forward?
Do we teach the artists how do put their art in directly?
I've had this discussion on a team in which we were losing an awful lot of time to programmers putting art in game.
And the programmers were convinced it would be faster to just keep putting it in because it would take too long to teach the artists how to do it, despite the fact that it was eating eight or 12 hours of programmer time every week.
I still think they were wrong.
Anyway.
Note that I keep saying team and team members.
In Scrum, the responsibility for project management doesn't sit with a producer or design lead or the programming lead.
It's the team's responsibility, guided by the team members, to manage their own project.
That doesn't mean that a Scrum team gets to do whatever they want.
They still have constraints, requirements, expectations, project handouts handed down to them from managers, clients, instructors, and other people.
But in exchange for taking responsibility for the project management, Scrum teams do get to explain to the producers, managers, executives, and outside forces just how much they think they can achieve in a given time.
So you can go ahead and ask a Scrum team for the sky, the earth, the stars, and the moon, and an experienced and empowered Scrum team will explain that, OK, in that time and on that budget, we can get you a terrarium with a fancy night light.
So how does this work?
With vocabulary.
There may or may not be a quiz later.
So the project cycle.
This is the cycle of a whole project with the internal cycle right there in the middle where you can see it.
Looking at it from sort of the whole project life, you've got pregame sprint, which is-- let me start with this.
Let me start with what a sprint is.
So a sprint is the way that a Scrum project divides up its design iteration cycle.
Sprint length depends on the project and what the team wants.
One month was an early standard.
But a lot of people are now using two weeks instead.
In this class, we encourage one week sprints, given the length of the projects and the sort of reality of how much you can or can't get done in a week.
So a six month project can be divided into six one [INAUDIBLE] sprints, 12 two week sprints depending on what the team wants to do.
So much like a waterfall project, you've got your design and vision time.
You've got your development time and then you've got your wrapper time.
Your vision time is going to be when you're coming up with the idea and the overall outline.
In many ways, that's what you did on Monday.
That was kind is you formed teams, you got a vision statement.
Some people were even sitting around and making feature lists.
That's really your pregame staging sprint right there.
In the development sprints, that's when you're actually making the game.
And you're taking a couple of features every sprint.
You're getting them working, you're testing them, and then you're moving on and grabbing a few more features, getting them working, and testing them until it all comes together.
Product backlog.
Does anybody here know what a product backlog is?
Go ahead and tell me
[INAUDIBLE]
Exactly.
Actually not in a sprint, in the--
[INAUDIBLE]
In the overall project.
Yeah, the product backlog is in fact the feature list for the whole thing.
Once you've got it, you take bits of it off to make your sprint backlog.
The sprint backlog is what gets done in every sprint.
Eventually hopefully your product backlog is at 0.
And some teams, I'm afraid of your teams are probably not going to have luxury of this, take what's called a wrapper sprint or two to kind of hardening.
You can think of it as polish and hardening.
Taking the time where you're not putting in any new features but you're making sure that all the systems really do work.
They're kind of shakedown crews.
So that is the overall project plan.
What does it look like on the inside of the spirit?
Let me see if I can see what that looks like.
There we go.
So here we have our big pile of definitions.
So since you're doing a little bit of project planning, design implementation and testing in every sprint, you have to figure out when and where you're doing those things.
Scrum has a fairly I don't want to say rigid, but they do have a very well defined set of processes and tools to do that with.
So to handle project planning and communication, every sprint involves a set of standardized meetings.
Sprint planning, daily Scrum, sprint review, and the retrospective.
To handle the features and task management, Scrum teams create and use a set of tools.
Artifacts is what it's called.
The product backlog, the sprint backlog, the task list, and a Scrum board.
At the start of its development sprint, the team turns to its product backlog, which is, like we said, the feature list.
Holds what's called a sprint planning meeting in which they decide just how many of those items that can get done.
That list of items that they're moving from their product backlog becomes the sprint backlog.
From there they break those features.
Because features coming off of a backlog, usually fairly large.
So they'll break them down into much smaller items called tasks.
A task is just a chunk of work that you can estimate fairly well.
Having gotten that big list of tasks, they start working their way through it.
Every day while they're working on the project, you'll hold a daily Scrum meeting where you talk about what you're doing, how you're doing it, what you hope to get done.
At the end of the sprint, you end up having a sprint view where you take a look at what you've got done.
You compared what you wanted to get done with what you actually did.
You go ahead and you adjust your product backlog if you need to.
And then you hold a retrospective meeting.
And that's where instead of talking about the product and specifically you're talking about the team and the processes.
That's where agile tries to improve the way the team is working together is the retrospective meeting.
And then you're ready for a new sprint planning meeting and the whole thing starts over again.
OK, I'm almost done throwing vocabulary at you.
And then I'll go back and make sure that I actually did define all the vocabulary I've used.
So on the team, I claimed that the team was responsible for all of its project management.
That's not quite true.
Teams typically have three kinds of members.
There's you're sort of average everyday team member, who's someone on the team working to keep things running.
The team has a product owner, may have a product owner.
In classic Scrum, you always have a product owner.
Your teams probably won't.
It's much harder to have a product owner on a student team.
And the reason for that is because the product owner is the person who holds the sole vision of the product.
For classic Scrum teams, the product owner is the person who creates the product backlog, helps them prioritize it, and lets them know which of their features are the most important and whether or not the features as implemented are working the way they need to.
Your team may be lucky enough to have someone who has a good enough grasp of the project.
And if you do, it's really, really valuable.
It's really helpful to have one person who you can turn to and say, is this going to work or is this working or is this what we had in mind?
And have one person answer that instead of having a big debate over whether it does or not.
But on student teams, it can be really hard to have someone in that position.
So that's your product owner.
The other unique person on your team is a Scrum master, and you will need one of those.
The Scrum master is the lucky person who runs the meetings, serves as the facilitator and the mediator, and is in charge of making sure that Scrum is followed and the team is working reasonably smoothly.
They frequently get stuck with updating product backlog, sprint backlogs, task lists, Scrum boards.
Whatever record keeping paraphernalia the team feels it needs to have.
Setting up meetings and making sure they happen.
Try to think of the Scrum master as a bit of the team coach and as the keeper of documents rather than as the producer or the person in charge of the project.
Just trying to keep the backlogs straight, keeping meetings flowing, and keeping communication following is a lot of work for one person.
You don't also want to burden that person with being in charge of leading the team and making all the final decisions.
They often end up drifting together.
And we see a lot.
And we also make a slip of calling the person who's serving is the team Scrum master as the producer.
See if you can try to keep those two roles separate.
If you can let the Scrum master just be the person keeping the team running and you can let someone else step into that kind of vision holder, guide the team role.
OK?
All right, I think that that is, in fact, what I have to say on that.
Everybody think they have at least a definition or an idea for all of these words?
Here's the quiz later you see
[INAUDIBLE]
OK. Should I be mean and choose the one that I haven't defined that nobody admitted to?
OK, who can tell me what a Scrum board is?
[INAUDIBLE]
That's exactly what it is.
So what a Scrum board is is it's frequently, if you've got a team that all works in the same place, it's usually a big physical board that you have room to put cards with all the tasks on it on.
And you just have it divided up into sections and show the status of each task.
By using a Scrum board, teams can figure out exactly how far along they are, what the state of their project is by just looking at the board.
Teams that are going to end up working in a much more distributed fashion and who don't have a designated place to sort of leave a Scrum board set up.
For example, you guys are not going to be able to Scrum boards up on the back of the classroom and come in and check up on them every couple of days.
There are a couple of online options.
And I will talk about them when I actually talk about Scrum boards a little bit more, asking you to get one set up
Just out of curiosity, who has already got something like that?
Like an online site with all your tasks on it?
We're using Trello
Trello uses exactly a Scrum board style visual representation
There's also a Scrum plug in for Trello for Chrome that we can get points if you're [INAUDIBLE]
I did not know that.
That's new
Oh, that's new
It's called Scrum for Trello or something

Scrum for Trello.
I'll google that.
What else?
I thought I saw another hand go up.
What are you using?
Asana
Asana?
OK. [INAUDIBLE] So is that just [INAUDIBLE] card like representation, or is it more like spreadsheet?
It's more like a [INAUDIBLE]
And multiplayers can do this?
Yeah, I haven't used that site
I don't know any alternatives
I will actually talk about Scrum boards a little bit more when we actually have gotten around to making our sprint backlogs and task lists and talking about the techniques you used to make them.
OK, so we're finally actually getting to the action part of this story.
And I'm lying there.
I'm not actually talk about time boxing right now.
I went back and updated my lecture without updating my slides.
But we are going to talk about user stories.
So let's talk about how you make a product backlog.
Because it is not just a feature list.
There's actually some very important differences between a feature list and a product backlog that makes a product backlog a project management tool as well as a design tool.
So it's an organized, prioritized feature list.
So the team always knows what the most important next feature is.
So if I actually sat down and looked at one of the product backlog you've turned in, every feature in your game should be described or mentioned in the product backlog.
And each item should have a priority status indicating how important it is, when are you going to do it.
Are you going to do it first or you're going do it last?
Or is this the one that's going to get cut when you realize you don't have time to finish the project?
So how do you decide what goes into it and how do you estimate and how do you prioritize?
There's the big question.
So here is a sample project backlog.
It's not for a game but it's for some sort of log in page or something.
But it serves pretty much the same purpose.
This is sort of the minimum you should have in your product backlog.
Some people get fancier and include lots of extra information, but that's not necessary.
Keeping it simple, especially to start, will make your life a little bit easier.
If you discover there's more information you want to track here, go ahead and do that.
But start with the basics.
The basics are an ID number.
ID number's not its priority.
It's actually in the order of which we thought up the features.
This is the fifth feature we thought of.
IDs come in useful when you start moving tasks off of your backlog and into your sprint backlog and breaking them down into smaller tasks.
So you can use your ID number to know, oh yes, this feature is the same as it is on the product backlog and all of these eight tasks that came out of it are all related to each other, because they all still have the ID number that goes back to the original product backlog.
Then you need a description.
This backlog is using user stories, which I'll explain in a moment.
But they all have sort of the same pattern.
As an authorized user, I want to create new account.
As a authorized user I want to.
As an unauthorized user, I want to.
So that.
So there's your description of what the feature is.
Then you need an estimation of how big a job this is.
For the product backlog, we don't actually recommend trying to do really fine estimation.
They're using numbers in this backlog.
I should make a new one that uses my preferred system, which is actually just using the word small, medium, large, extra large.
And how big is small?
I'm not really sure.
When you first create your backlog, you won't be really sure either.
But you could probably tell the difference between a small feature and a medium feature and an extra large feature.
So when you go through and you write down all your features, you can say what are all the smallest features here and call this small.
What's bigger than that?
That's probably a medium.
What's the biggest feature I can see here?
Make that a large.
When you're making the product backlog, you don't want to spend a lot of time figuring out estimation and trying to figure out exactly how many hours it's going to take and how many subtasks are involved.
What you want to do is get a feel for the overall size of the features and a feel for how big they are in relation to each other.
Because if you're fairly good at saying this is about the same size feature is this and this is about the same size feature as that.
Once you realize that feature A took you three hours and you have four other small features, it's not that hard to go, oh, so that's a total of about 12 hours between those four features.
So that is the format of a product backlog.
Are there any questions?
No?
Just want to point out occasionally you come across a feature that's actually not one feature.
It's actually a whole bunch all rolled up in one.
Like multiplayer or something like that
Networking
It's not a single feature.
It's a whole bunch of different things.
If you come across features that clearly warp everything else that you've got on your list, you should try to bring those down and do small and large chunks.
So you can be like multiplayer [INAUDIBLE], multiplayer matchmaking.
All of these things are separate features
But I will say that when you're making your first pass at a product backlog, it's OK to stick multiplayer on there as a feature and then right next to it, extra, extra large.
And realize you're going to come back and break it down later.
When you're making your first product backlog, you don't want to be worrying about, well, what are all the little sub features of multiplayer?
But you do want to go ahead and acknowledge that, oh my gosh, we've just put something ginormous on the list.
And using a very rough estimation tool lets you do that and move onto getting all of your features in.
And then you can come back and honestly estimate them a lot more finely when you're actually thinking that you might put them in or when you're actually determining what their overall priority is and how badly you want them.
Because if you end up deciding you want to drop multiplayer, there's no point having spent a lot of time dividing it up into 12 or 14 different features.
All right, so I mentioned user stories and I did not actually define them very well, and that's because I was going to come along and define them here.
So a user story is just a very specific way to describe a feature.
It is, I think, unique to Scrum.
Although there may be other agile methodologies that have stolen it as a way of describing a feature.
And we're going to ask you use user stories for project two at least.
Just to get you familiar with it and understand why we think they're a neat idea.
You may end up deciding that you don't like them and that's OK.
But we want you to at least experiment with them.
So the goal of the user story is to help you phrase your feature in a clearly testable and understandable manner to someone who did not help you write the document.
OK?
So a user story follows the formula of as the person who is going to use this feature, which could be the user.
It could be the designer if you're creating a level editor.
Could be the artist if you're creating something for them.
It could be another set of coders if you're creating tools.
But the important thing is to recognize who are you making this feature for and who is going to use it.
Then you want to describe what it's going to do.
And that description should be something you can test.
So that when someone goes ahead and gets this completed feature on the other end, they can go in pretending to be whoever that person is, use that feature, and confirm that it works.
Finally, you want the end third, which is so that-- explain the reason why you want the feature there.
What is the purpose of this feature in your game?
What is it achieving?
Can anybody think about why this is an interesting or useful format for stating a feature in?
Can you see any benefits in it?
I can go ahead and tell you what we think the benefits are, but I'm wondering if you guys can think of any
It's testable and focused on the player, or other relative entity
Yep.
That's one.
OK.
So it's testable, which means that you can confirm it's working.
It's focused on the person who's going to use it.
We're going back to that sort of user oriented creation of product.
And finally, having the reason there means that if the person who originally asked for the feature did not understand the game well enough and asked for a lousy feature or one that doesn't work well with the game, the person who is going to implement it can come back and say, hey, you asked for a physics engine.
But you want it so the player can drop rocks into a pool and watch the rocks sink.
That's the only use of your physics engine.
We can make this a lot faster and easier if I write an animation hack that shows this happening.
And we can give you some sliders to control how fast it falls depending on the weight of the rock.
And people will often say they want something that is bigger or smaller or doesn't actually do what they want it to do.
It's really common when you're writing features.
It happens all the time.
And so having that reason there means that the team can go back and do a double check when they actually go to implement it that the thing that's being asked for is something that is really needed.
And so it puts a double check right back in the feature.
And it's not as good as going back and talking to the person who wrote the feature, but it's at least a warning signal telling you that you might want to go talk to someone and make sure this is really what they need and really what they want.
So I think I already just went over this.
So because user stories have that check in them, you can make sure you're actually doing the work you want to do and not some other work.
One of the things that agile is really trying to do is focus on making sure that you're not doing extra work.
You're not doing work that doesn't need to be done, that's going to get cut later on.
And so a lot of the tools and a lot of the processes and a lot of the language and a lot of meetings are all about making sure that before you go out and put a lot of time into something, it's really the thing that you want to be doing.
And user stories are just one more way to add that check into your process.
I'm going to skip on time boxing right now because I've got a better discussion.
It's more relevant to meetings.
And I'd rather talk about it in meetings.
So we'll skip talking about it there.
OK, that is actually the first two thirds of the lecture for today.
And I was going to propose that we take a 10 minute break.
And when you come back, it will be time to make a product backlog for your team.
I'm going to give you half an hour to do it in.
And that half an hour will also include the time to prepare a two minute presentation on what the features for your product are and what an updated vision statement for your game, since things may have shifted a bit when you added new people.
No visual aids.
Just come up, talk for up to two minutes.
It's OK if you talk for less.
The faster we get through presentations, the faster we get to other things.
And then we'll move on to talking about meetings
Is there a sample on Stellar, a sample backlog on Stellar?
There is not the sample backlog on Stellar, I think.
We think there is?
If there is, we'll put it up by the time we get back from break
OK. Yeah, in many ways I actually recommend you go into Google and create a Google spreadsheet.
Give it an ID and priority and status and going from there.
But yeah
[INAUDIBLE]
Is it there?
Yes, sample product backlog.
OK.
It is there.
Thank you, Rick.
I forgot to check on Stellar.
All right, that said, it is 1:52.
Come back in 10 minutes and start working on your product backlogs.
But go stretch your legs a little bit.
OK so it's 2:02.
If you are not already sitting in your project groups, you should probably go ahead and get together in your project teams.
At 2:32.
2:35, we'll give you some to move around.
2:35 we will go ahead and we'll make a list for signing up and we'll run through the presentations.
OK?
If there any questions or you want to know what's going on, if you'd like some advice, we'll be all hanging on down here.
So please feel free to come and ask us for any advice or any help or anything you need.
OK?
OK. One moment.
We don't need that plugged in right now.
OK.
So let's go ahead.
I'm guessing everybody's ready for presentations.
OK. Then let's ask Comcastic to come on down.
STUDENT 1: OK.
So the goal of Comcastic is to be able to place internet service stations into towns so that you can service neighborhood and get money.
And then eventually you want to build your monopoly cable empire to become very wealthy.
And so the core features that are necessary in order to enable us are being able to look at the town map.
You need to be able to place pieces.
You need to be able to rotate the pieces because the orientation does matter.
You also need to be able to have some kind of [INAUDIBLE], so you want like an end turn button or something that happens when you place a piece.
And that will also give you the money results for that turn.
You also need to be able to know how much money you have and be able to place pieces only on legal spots and to be able to start the game, quit the game, things like that.
So those are the core features of Comcastic
Thank you.
[APPLAUSE] All right, Blind Alien.
STUDENT 2: Blind Alien's team goal is to create a creepy and enthralling game.
In order to do that, we first need to have really basic structure, which is what we're going to focus on this week.
We're going to focus on having first a start menu, where we can just say the instructions really fast and input the number of aliens, which is going to be important for testing.
Because you need to be able to change things quickly.
And then we're going to have the main screen, a character, character movement, aliens, alien movement, and then this weekend we're going to get up together and work on the interactions of those aliens and the character and then work on shooting.
So that's what we're focusing on.
Thank you.
[APPLAUSE]
All right.
Plunder Winds.
STUDENT 3: Hi.
So for Plunder Winds we're implementing a similar game to our paper prototype.
So we're a grid based, or tile based, turn based game.
And built around three core mechanics.
Our mechanics are a random encounter mechanic where every square possibly has either treasure or pirates.
A exploration mechanic where you revealed the risk level of nearby tiles.
And so that can change what pirates will do to you if you actually land on that square, something like that.
And then a wind mechanic, which controls which directions you can move.
And so our goals for this week are to try and implement as much as our core mechanics as possible so that we can possibly look into stretch goals for our games, like possibly implementing one or two unique abilities to improve playability over next week
Thank you.
[APPLAUSE] Lost Underground.
STUDENT 4: Lost Underground.
Scared for a second.
The overall goal is to create a game that people will like.
It'll be random and will allow exploration.
Our goals for the weekend are to create the basic bare bones you could walk around a map, you can drop bombs.
Make the most basic part out and then build up on everything else.
The features that we're thinking of planning.
The most important features of the game, which are the ones we'll be working on this weekend, will just be being able to move around the board, a board, a general board, dropping bombs, and possibly the most basic, nice looking art that we can.
[APPLAUSE]
Beaver Evolution
Just a quick point, the most basic art that you can make is probably not the nicest looking art you can make.
So just [INAUDIBLE].
STUDENT 5: All right, so Beaver Evolution is a 2D turn based game where you're trying to progress your beaver colony.
And one of our main core mechanics of this game is that you have three different choices you can make every turn in order to progress to one of the many wind conditions we have in the game.
So this is going to be one of our main focus for this weekend.
We want to be able to implement a board, want to be able sort of select how to build our dams, and we want to make it really easy to incorporate the three different choices we have every turn.
So our main focus will be also on creating a UI, which makes it really to sort of figure out what you can do, what sort of resources you have, like how far is your beaver colony progressing.
And I think our second main mechanic, which is including the randomness factor, such as the disasters and that kind of stuff.
That sort of stuff can be added later in the projects.
We don't need to focus on that.
Right now we just want to focus on being sure we can play through the game, make sure that's not confusing to the player, and that you can do it without sort of adding any sort of the features that require more balance.
[APPLAUSE]
Magi Dice.
STUDENT 6: Hi, we're Magi Dice.
We want to make a fun, engaging puzzle games where the player moves around dice on a board to try and using modular arithmetic create 0's and set of combo chains.
So our goal for the first week is, first of all, just set up the board and the dice so that the player can see what they're doing.
And then our immediate following goal is to incentivize players to create combos that multiple moves earn them, like, create 0s in a row.
And we want to do this by giving them more points and having a pretty UI so they're incentivized to set up and do fun stuff.
[APPLAUSE]
Sparkly Redemption.
STUDENT 7: OK, Sparkly Redemption.
2D real time game where you shoot monsters and pick up sparklies.
And the important thing is that when you pick up sparklies, you either get a different weapon or the monsters change their behavior.
That means the core thing we really need to implement is to have a [INAUDIBLE] you can move around, have monsters with at these two different behaviors, because the swapping is important, and have sparklies that you can pick up.
[INAUDIBLE] needs at least two different attacks.
So that's probably what we're going to focus on at this time.
I guess it'll also be sort of important to have it be easily visible at a glance the difference what the current monster behavior is, what your current attack is, and I guess how many sparklies you have.
And so I guess it's not the most, most critical thing to have this week, but we're probably also going to try and get it so that you can see what's going on and not just have to sort of guess at it.
[APPLAUSE]
All right.
OK.
So you did product backlogs.
On Monday you did a fair chunk of your planning sprint.
You finished up sort of the second half of that today by getting your product backlog pulled together.
To close out today, I'm going to talk about the meetings that are involved in Scrum.
We're not actually going to have you hold them all here because A, they wouldn't be very useful.
It would take a lot of time.
And because we're introducing Scrum alongside the project, the timing doesn't fall well as we introduce things, would be the right way to put it.
We are hoping you will use these meetings in project three as you start planning out your project and figuring things out.
So I do want to talk about how you run the meetings we talked about that occur within a Scrum cycle.
But first I'm just going to talk about meetings.
And meetings as a force for good as opposed to how most of us probably think of them, which is, let's be honest, as a force for evil.
As a force for evil, they suck away a lot of time.
If you have a six person team and you have a one hour meeting, that's six hours of work gone.
So you want to make sure that those six hours of work are actually good and useful hours of work.
There are three primary tools to do this that we're going to talk about here.
We're going to talk about time boxing, having a clear agenda, and having involved participants.
Has anybody here heard of time boxing and do you know what that concept is?
OK. Timeboxing is when you decide before you have the meeting or before you start any particular activity how much time you're going to let it take.
You give the task a time budget and you don't exceed that time budget.
For example, sprints are just one big time box.
You're defining that.
Here's the set of things we're going to get done, we have a week to do them in.
When we're done with that week, we're going to stop and see how much we got done, and we're going to move on from there and reevaluate.
Meetings can do this too.
And all meetings in Scrum should be time boxed.
It's one of the classic Scrum rules is that meetings are time boxed.
They have a limited amount of time you spend on them.
Time boxing acknowledges that there's sort of a maximum amount of time you can spend on any particular problem before any time after that is wasted.
It does mean that you won't always reach the perfect solution.
Because if you've given yourself an hour to discuss a problem and you've talked about it for an hour and you haven't quite come to a great solution, you stop talking anyway.
You decide what the most reasonable solution you've talked about is and you go ahead and do that.
It's sort of an implementation of a theory that the perfect is the enemy of the good.
It may be that if you went ahead and talked for another hour, you would find the perfect and ideal solution to the problem.
But it's also very possible that you're just going to spend another hour talking about it.
It may be better and it usually is better to go ahead with the best solution you came up with and start implementing that and see how it works.
So that's the idea behind time boxing.
Clear agendas.
If you don't know why you're having a meeting, you probably shouldn't be having it.
The meetings that Scrum recommends come with a built in agenda for what you want to do.
You'll probably have to have other meetings.
Because at point you'll have to talk about to figure out how your version control is working or how this feature is working or how the art assets go into the game or what kind of sound assets you're going to have.
Before you go ahead and have that meeting, make sure you know what the goal of it is, what you want to come out of it with, and honestly, leading into the next point, who actually need to be there at the meeting.
Not all meetings require the whole team.
A lot of meetings are really good as just one on one conversations or just a small group of people, a subgroup of people who are working on a particular task and get together and talk about it without including everyone else.
The other half of involved participants is if you are in a meeting, be in the meeting.
Don't be coding in the background.
Don't be checking your cell phone and texting something.
Don't work on a problem set.
Go ahead and be there.
Accept that you have dedicated the next 15 or 20 minutes to working with your coworkers to solve this particular problem.
Get the problem solved, get the meeting over, and then get back to doing useful things.
So those really are the ways to make meetings useful and to make them work for you rather than working against you.
All right, that said I'm now going to go ahead and talk a little bit about each of the meetings.
So starting at the sprint planning meeting, which you've actually already mostly done.
We will do it in class on-- you've already done a good chunk of that, which is making the product backlog.
We're going to do this on Monday.
We're actually going to go ahead and take your product backlog and break it down into a sprint task list and start making tasks.
So we will have a in class sprint planning meeting for your next sprint next week.
But a sprint planning meaning sets the team's goals for the sprint and has the deliverable, its goal, of a sprint backlog and an estimated task list for the sprint.
So you know exactly how many hours of work you think you are going to get done over the next week.
And it is time boxed to one to two hours.
For a team your size, one hour's probably good.
For an inexperienced team your size, 1 and a half hours might actually be better.
So the more you do this, the faster you get at it because the easier it gets.
At the end of the sprint, on the other side of it, is the sprint review meeting.
It has the deliverables of demonstrating your working product, in this case your finished project two.
It is a chance for your team to stop and review and evaluate your product and decide if that's actually what you want.
If you have clients to go ahead and say yes, that's what we want, keep working in that direction.
Or to say, oh my gosh, this is exactly not what we were thinking of.
Quick, change everything.
So the other thing you do in the sprint review meeting after you've reviewed the product is change and update your product backlog.
Because it may be some of the features that you thought were really important before you went into this sprint have turned out to be less important.
Or it might be that you thought of five new features that you really need to put into the game and that it's worth cutting some of the other features you already had on the list.
So the sprint review meeting sets up your sprint planning meeting by getting your product backlog in shape to go and prepare for the next sprint.
It also is time boxed to about one hour, again, for a team your size.
The other end of sprint meeting is the retrospective, which is not the same as the sprint review.
A lot of teams try to smash a sprint review meeting and a retrospective meaning into the same one.
And they have very different purposes.
Sprint review meeting is all about reviewing your product.
The retrospective is about reviewing your process.
It's about talking about the team.
Ideally, it kind of recognizes the things that went right, the things that went wrong, and generates a short list, at least one or two items, of things to do differently next time because they will improve the things that happened.
So it's a forward looking meeting despite its name.
The name is retrospective.
It's looking back at what happened in the past to see what you can do better in the future.
If you do your team retrospective well, it actually will be a lot of what we're asking for in our postmortem write ups.
Because really that's what we're looking for in the postmortems is understanding what you did, why you did it, how it could be improved.
We want to know what sinkholes you fell into and how you climbed out of them and how you would build a bridge over the sinkhole next time so you don't fall into it.
Finally, there's the daily Scrum.
And this is the meaning that happens it regularly over the sprint.
Every day, or if your team is not meeting every day, every time your team meets.
The goal of the daily Scrum is to keep the team accountable to each other and to encourage communication across the team.
If everyone's being honest at the daily scrum, there's really no excuse for anyone to not know the full state of the project, especially when it's combined with a Scrum board with all the tasks up.
Daily Scrum are also actually called daily standups, because a lot of teams do them standing up in order to keep them short.
Classically it's time boxed at 15 minutes.
For teams your size, five to six minutes is probably actually enough.
Because all each person has to do is stand up and answer these three questions.
What did you yesterday or since our last meeting?
So what have you gotten done?
What are you going to do today?
What's on your task list?
And finally, what is keeping you from getting things done?
What's blocking you?
How come you can't get things done?
And sometimes the answer is nothing and sometimes the answer is something, in which case it needs to get dealt with.
The place where daily Scrum often go wrong and end up taking longer than their five minutes or 15 minutes is when people start mentioning what they're being blocked by and everyone jumps in to help them solve the problem.
That's not actually what you do at a daily Scrum.
You don't solve the problem at the daily Scrum.
You mention it, someone makes a note of it so everyone's aware of it, you get the meeting over, and then the people who actually need to be involved to solve the problem talk to solve the problem.
Once again, it's invoking that principle of, if you don't need to be at a meeting, don't be at a meeting.
So that is my comment on daily Scrums.
And I was going to have you guys try and do one today, but I actually don't think that would be very useful.
I think you've already really probably done a lot of that in sitting down and talking about your product backlogs.
So I think I would prefer to go ahead and close out my lecture here and give you the rest of the hour to continue working in your teams on your projects
One thing we can recommend is that since most of your teams seem to be planning on meeting up sometime between now and Monday-- and I certainly encourage that-- plan to do this during that next meeting, if it's on the weekend, if it's on Friday, if it's tomorrow.
Whenever that is, do this and ask every single person these three questions.
And be prepared, personally prepared, to answer all of the three questions.
The thing that could be blocking you is you've got a horrible head cold.
Your computer just fried.
[INAUDIBLE] I have a problem with my web game engine.
And there's probably somebody else on the team who can help you figure that out.
[INAUDIBLE] It could be another class that's giving you problems.
That's also a perfectly fine answer.
You want your team to know what's preventing you from getting stuff done
The other thing you should do over this weekend and have chosen by the time you come in on Monday is a Scrum master for your team.
And from the instructor's point of view, that's the person who is primarily responsible for uploading documents, vision documents, product backlogs, design change logs, making sure that we have a link to your project when the project is done.
Because we don't need everyone on the team to do the joint team turns.
We really only need one person to do it.
From the team's point of view, should be the person helping you do Scrum.
And that's all I have for today.
So do you have anything else, Rick?
Deadlines.
So turn in your vision documents, the ones you just worked on, that you updated with you new team.
Turn them into Stellar by the end of class today please
I was telling people since we had the date wrong, in case something went horribly awry, they had until the end of the day today.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Just very quick-- very quick what we're doing today and what we're doing next on Wednesday.
Phil's going to start with an estimation lecture.
We're going to follow that up with a lecture from me on Sprint Backlogs and Task Lists.
And then we're probably going to take a break.
You'll come back and you'll have essentially a Sprint planning meeting here in class where you will create a Sprint backlog and a task list, hopefully with estimations down to how many hours you think all those tasks are going to take you.
Then we'll ask you to come up and give a very brief two minute presentation on hey, how did the process work, and what was it like?
How did it work for your team?
We're not actually interested in hearing about your game or what the top task you have to do is.
We trust that you have a good game in progress and that you've got a reasonable set of tasks.
But we actually want to hear about how that meaning worked.
So we'll also give you about 10 minutes to have a little sort of post-mort review to talk about the process, after you've made your task lists.
Andrew will then come in.
He'll be leading a discussion, sort of a review discussion on good software to see what you've been doing and see if you've remembered anything he said from earlier.
And you should have turned in your product backlogs.
And you should have code running on your machines.
We're not going to walk around and inspect all your laptops to find out if you have running code.
But to be on target, you really should have something running, and if not playable, at least pokable.
For Wednesday, we'll be talking about testing.
And you'll be running a focus test in house.
And Genevieve Conley, yes, from Riot will be also giving a guest lecture after the focus test.
And I believe she'll be here to wander around and help out in the focus test a little bit.
So I think that's everything.
Now!
Those were my two slides.
Now you can have the controls
Do try to come in on time for class.
If you are unsure about whether you can get your game set up in time, come in early.
And make sure that your game is ready.
When we have guests coming to our class, we really will prefer that people don't walk in terribly late.
It's just kind of embarrassing.
All right.
So over the weekend, I found myself in an unenviable position of having to teach-- prepare for a video game class, and not having a working computer.
So I had to prepare for the possibility that I wouldn't have a computer as of today, and redo my entire presentation in analog format.
But it turns out that IS&T actually does miracles and got my computer back and working on Monday.
If you happen to be working at IS&T, give yourself pat on the back.
You are doing God's work.
But I'm going to try to do the analog thing anyway because might is worth a shot.
I'm going to do to be very clear, this is the first time that I'm doing this.
I'm going to need three volunteers to come down.
Come on down.
Come on.
Come down.
All right.
OK.
So on Wednesday, we asked you to split things up into small, medium, and large features, right?
Small, medium, large.
And I'm going to give you an idea of what your product backlog currently looks like.
Your product backlog currently looks like a bunch of features.
All right.
Now I'm going to assume that all of you have sort of already assigned small, medium, large things to your features.
Sort of small, medium, large categories to your features.
But I want to try doing it slightly differently.
I'm going to give each of you three cards.
These are the large currents.
Medium.
Can people see this from the back, by the way?
The letters on the card?
Small.
So this is what's going to happen.
I'm going to pull something out from here, and you all are going to all look at it.
And you are going to figure out whether that is going to be small, medium, large without actually discussing anything.
All right.
So you keep the cards hidden, and only you can see it.
And then when I ask you to reveal it, show it all at once.
Show me what you've picked.
All right.
I'm not going to tell you what's small, what's medium, or what's large.
I want you to tell me what's small, medium, and large.
All right.
So everyone clear on how this works?
OK. Don't show it to anyone yet.
Make sure everyone's picked.
Yes?
[INAUDIBLE]
Good question.
Hang on to that.
Is this a small, medium, or large feature with no frame of reference?
So everyone's got one?
Show it to me.
Medium, medium, medium.
All right.
I'm going to put this here.
Next one.
Can everyone see this?
All right.
Everyone's picked one?
Show it.
Medium, small, small.
OK. Why is this a medium?
Looks similar
It looks similar.
All right
[INAUDIBLE]
OK.
The other one does have a hole in it.
If you haven't seen it.
OK.
I'm going to put this one aside for now.
This one.
It is red.
All right.
Still being picked.
Small, small, small.
OK.
So this is unequivocally small.
And this piece.
This one also has a color in it.
All right.
Show it to me.
Large.
OK.
Going back to this one, what do you think?
Everyone show me what you're thinking.
Everyone thinks that now is small.
OK. All right.
So, don't go back yet.
But I want to say thank you very much to all of our volunteers.
Why did I just make you all go through all of that?
That seems like a very, very inefficient way of sorting out features.
Like if I just gave you this entire box, this is what I want you to do now.
Sort this whole box into small, medium, large.
All three of you together.
You don't need the cards.
Go for it
[INAUDIBLE]
OK.
Thank you very much.
If you can just hang out here, just a little bit longer, that will be great.
Now that is probably a fairly good analog of how most of you figured out small, medium, and large on Wednesday.
Which was you just compared features next to each other.
Figured out which one's seen on the larger end.
Which ones was on the small end.
And just kind of put things in place.
But there was some discussion going on.
Is this small?
Is this large?
It's just like do two of these put together make one medium, for instance?
Now think about the first method of first choosing whether this was small, medium, large and revealing it, was that I forced you to actually voice out why you thought that was different.
I believe you picked a medium.
And the others picked a small.
For a lot of these other pieces, you didn't voice anything out.
And one of the reasons why we can do this so quickly, is because human beings are actually pretty good at reasoning about spatial volume.
You're pretty good at-- and being at MIT, you also understand geometry, 3D geometry, in a sense pretty well.
So you can make these spatial reasonings pretty well.
But when it comes to things like time and complexity, human beings tend to have a tougher problem.
So I'm going to say, for instance, I'm going to give you three blocks.
This represents for you, seven hours of work.
OK?
This represents for you, six hours of work.
This represents eight hours of work for you.
All right?
So you know you've got 16 hours of work to do.
Pick out features that you're going to implement in your next print
So this is how much time I have in my life
That is how much time that you spent on the last feature.
If all three of you worked on identical features--
OK, so this is already done.
This is already done
Yeah.
This is done.
This is a known value.
You spent seven hours on that.
You have 16 hours
So I spent seven hours on this.
And then there are the two small triangles
[INAUDIBLE], right?
The first one is just a reference.
And the little feature.
OK. All right.
All right.
What did the cube represent to you again?
Seven hours of work.
And you picked?
And you think that that's a reasonable amount of work to get done in 16 hours?
Not including that one.
So this is about 16 hours of-- assuming volume equals time.
Sorry, volume equals effort, actually.
All right.
And over here?
How long did it take me to do the other block?
Six hours
Oh.
I guess I could have done a little bit more.
Well, that's fine.
I'll just do [INAUDIBLE] like them.
And then if I have leftover time, I'll do more
If you went for more, what would you try to pick on?
I guess maybe this one.
It's a little bit more than perhaps I could do in exactly 16, but it wouldn't hurt to touch it
It's the smallest thing that you can find.
OK. All right.
And for you?
This exactly fills my estimate.
But realistically, I'd probably go with this
So you're using kind of programmer patterning.
You got a question?
Sometimes you do not [INAUDIBLE] the small things, for example, adding balls to your game set is small.
But bringing out the framework itself is a large test that must be done before you have the choice of choosing smaller tasks
That's true.
That's true.
I am not indicating priority in here.
I'm assuming that you're picking up high priority tasks.
The next thing that needs to be done.
Thank you very much.
And you may have a seat.
Big hand to our volunteers.
[APPLAUSE] I'll be going back to this example shortly, as soon as I get my presentation going on.
I want you to hang onto these examples because I'm going to come back to them during the talk.
On Wednesday-- and what I first asked our volunteers down here to do was to do a small, medium, large, extra large feature sizing, basically.
The reason why we use these broad categories is because actually it can be pretty fast.
And on Wednesday a lot of you did it pretty quickly.
However, what I noticed on Wednesday was a lot of people weren't actually discussing how big or how small these features were.
Like you all using your own little metrics.
So that was a good question.
Small, medium, or large to what frame of reference?
And everybody had their own frames of reference.
When I forced people to make the assessment and then declare it in front of the entire team, and then discuss if they disagreed with the entire team, then assumptions started coming out.
You had to vocalize why you thought a certain feature was a bigger or smaller.
You could adjust your assessment after the discussion.
But it very, very important to get that discussion out to everybody.
So as long as you're discussing with your entire team, I don't really care what method you use to split things up into small, medium, and large.
But you need to be discussing as an entire team.
In fact, every single member of the team should be in on the discussion, because somebody in charge of sound design, or somebody in charge of art, or in charge of art of the code architecture would have very different perspective on any given future from other people on the team.
And you want all of those perspectives in the same time.
There's oh!
Playing a sound effect.
That's going to take no time at all, because we already have sound effects in our game, and we know how that code works.
That's what a programmer thinks.
The sound designer says wait a minute.
I need to create that sound effect.
I don't even know what this is going to sound like yet.
I have no idea.
And that's going to be larger feature from my point of view.
And you want that discussion to come out.
So each row should try to lend its perspective.
And is very important to listen.
The reason why I forced everyone to do this-- the volunteers to do this-- right at the beginning was because it was a very turn-based method of getting that discussion out.
I had to force everybody to listen to everybody.
And something that I didn't ask our volunteers to do was to offer alternate solutions.
However, they kind of did that automatically by themselves.
When I just left them to their own devices, I said, sort these out into features.
They started discussing well, this is how I look at a problem, this is how you look at a problem.
And they quickly came to some sort of consensus.
Of course, something I mentioned on Wednesday was if you've got a really, really huge feature that's kind of hard to figure out.
It doesn't really fit neatly into any of these boxes.
Break it up into smaller features.
You want to converge.
You want to converge down in to a consensus.
But there are right ways and there are wrong ways to go about that.
Every time you have that discussion, you're getting new information into the team about how complex a certain problem is.
Maybe sometimes through a discussion, you discover a certain problem is easier-- a certain feature is easier to implement than you had originally expected.
And then you redraw a C. Other people's estimates is actually new information.
And that's actually what happened here.
There was small, small medium, and the people on the volunteer team figured out how other people were seeing the problem.
Here's something that's very important.
Don't negotiate.
Don't say I think it's small, you think it's small, most of the team thinks it's small, somebody thinks it's medium.
Just change your estimate down to a small.
I suppose it's kind of adamant.
No, I think this is a bigger problem than you think it is.
Don't try to negotiate with that person because that person has a valid point of view, and this starts to focus a discussion on changing the person's mind, not focusing on the feature.
You actually want to figure out how complex a certain feature is.
For the same reason, don't take an average.
Someone thinks it's a small feature, someone think that's a large feature.
That doesn't make it a medium feature.
That means that your team has no idea what this feature is because you aren't in agreement with how complex something is.
So what I did-- does anyone remember what I did when-- let me see.
Which feature was the weird one that people were kind of hung up on?
I think it was this one.
When we got to this one, and there was some disagreement, can anyone remember what I did?
Yeah.
I just put it aside, right?
I actually gave it away.
It is a very, very easy to get into a long drawn out discussion about how large or how small a feature is.
It's usually much more clearer if you sort of tackle the easy ones first, and then come back to the ones that you disagreed on.
But you remember that you disagreed, and you got new information in the process of talking it out.
So that's a role for the Scrum Master.
The Scrum Master says we don't want this feature estimation process to take too long, so we're just going to put that feature aside first, look at the other features, do the estimation on that, and then review it later.
But a Scrum Master does not decide what size that feature is.
The whole team has to come to a consensus.
You want to converge on a single estimate.
So if you put together your product backlog, and you never really discussed with the rest of your team what those size estimates were, then that's something that you're going to do later today.
Before the end of today I want you to have a discussion about whether these things are actually small, medium, or large.
And the reason is because estimation-- it is not just a process of just putting down numbers or letters on a spreadsheet.
There is a goal for the process of estimation.
And it's this.
It's to get a clear enough view of the reality of your project so that you can make informed decisions to control your project in order to hit the targets.
Now what's a target?
A target is a goal that you wish would happen.
It's your fondest desires is to get this project done.
Entire product backlog of all features into your game before the deadline.
That's a target, right?
Maybe your target's a little bit closer term.
What can you get done by this week?
What can you get done by this spread?
It's still a target, and something that you want to hit.
But you can't base all of your decisions on your fondest desires.
You have to base it on what your team's actually capable of doing.
An estimation is a process of actually trying to understand what your team is capable of doing.
Not whether what you want to happen, but what can happen.
So what are you estimating?
This is where you are at right now.
You have a product backlog, and you have size.
You have feature size that is going to get broken down into a bunch of tasks.
Each feature is going to be broken down into individual things that individual people can do.
Maybe a certain feature like jump involves art, involves code, involves sound, involves debugging, involves integration.
All of these are individual tasks that might be carried out by different people.
So what you're really doing in the second step is to figure out how much time something's going to take.
Remember what Drew said on the very first day of class.
It's not so much how hard something is, how hard a feature is, but how long it's going to take.
And what you want to do is figure out how long it's going to take.
Now that's a process of arriving at a number of how many hours this is really going to take, and is it going to fit within the amount of time that we've actually given the team?
And you need to translate it first from size into effort, which is like, total amount of effort that's going to be used to tackle the problem.
Split it up among the people who are actually going to work on it, and then each person has an individual amount of efficiency.
So for instance, I gave exactly the same cube to all three volunteers.
But I told them this meant different things to each one of you.
And when they picked off tasks off the product backlog, they picked it based on their own personal metric.
That's what you've got to do too.
When you decide that you're going to take on a task, you have to assess it based on your own ability to actually execute on a task.
Say it's an AI problem, and you're not very well versed in AI.
It's going to take you longer than somebody that has done artificial intelligence before.
But that's OK because maybe the person who's doing artificial intelligence is busy trying to solve or harrier problem establishing the base framework of code.
And that needs to be done first.
And that needs to be done sooner.
So that person is busy.
You can start working on certain features, even though you may not be the best person to do it.
That's fine as long as I make an honest assessment of how long it's going to take you to do it.
Size estimates are pretty easy.
You're just sorting things into buckets.
Very large buckets.
Time estimates are very much like hitting a moving target.
As you start to make estimates, you are going to discover that many of those estimates just flat out wrong.
You're going to say oh, that's going to take me four hours to do.
And it takes you four days to do.
And you're going to have to communicate that to your team.
Giving an estimate is not the same thing as making a plan.
It is part of the same process, but I want to be very clear about what's the difference between estimation and planning.
Say somebody on your team asks you for an estimate.
[INAUDIBLE] alarm bells going off when he says, how long is this going to take you to do?
Everyone starts to get worried because you're not quite sure whether they're actually trying to ask you for how difficult is something, and how much time do you think it's really going to take.
Or are they asking you, can you get it done by Monday?
That's kind of like the hidden question that they're not asking, but they think that they're asking.
So you need to be very clear when you respond, whether you're giving an estimate or a plan.
You plan to reach your targets.
Obviously, none of you-- I hope none of you are planning to not hit your targets.
I plan to get this done one week after the deadline.
(WHISPERS) No.
But once you've made your plan-- here's our entire product backlog, and we think it looks reasonable, and we think that it can actually be done by the deadline-- you need to start making estimates to see whether that plan is realistic.
You can say, yeah, I think that's a reasonable estimate when you think that assessment is accurate.
I've done something similar before, so I figured that this is something that we can get done before the end of the week.
You can commit to that estimate.
But you haven't committed to the plan.
You haven't said, I'm going to take responsibility for getting it done by the end of the week.
That's a very different thing.
When you accept the task and say, all right, put my name on that spreadsheet, and I have decided that I think I'm going to be able to get this done in four hours, so I'm now committing to this plan.
Of course then everything goes wrong, and you re-estimate.
You're allowed to make re-estimations while you work on tasks.
You have to because no plan survives first contact with actual development.
So here's a quick quiz.
Here are four statements that you might hear in the middle of a team meeting.
OK.
I want you to tell me what each one of these things are.
I've already used the words describing each one of these things in my talk so far.
It has to be done in two days.
What is that?
It's a deadline.
I used another word to describe that.
It is a deadline.
Is it a plan?
It's a target.
Someone.
All right.
So the next one.
It'll take about two days to do it.
You're talking in a team, you say, this feature looks like it'll take about two days to do it.
That's an estimate.
That's right.
I'll need two days to do it.
That is a plan.
That is a plan that you're committing to, more accurately.
It's like, I will take two days to do it.
You might take three days to do it.
You might be able to do it faster than [INAUDIBLE].
But I'll need two days to do it.
And of course the last one is, actual reality.
It actually took two days to do it.
I mean, in reality it'd be like, it took four days to do it.
So I want you to be very, very clear that when you're communicating with your team, what you are trying to say.
Are you giving estimations?
Given all the information that you know, this is how tough something is going to take.
And this is how long it is going to take to get this feature working.
Or are you making a personal commitment, saying this is my responsibility now?
I'm going to get it to you by Monday.
Just a quick side note, you've probably seen these two words come up in numerous engineering classes.
The difference between accuracy and precision.
Which one is more accurate?
Raise your hand in the direction that you think is more accurate.
OK. That's actually-- OK.
I think that's a majority on this side.
Which means that that one is more precise.
OK?
So that's right.
This is more accurate.
It's closer to the target that you're trying to hit.
Even though it's kind of like broadly spaced out.
None of them actually hit the target, but if you took the average of all, then you get pretty close to the center.
That one's a really nice tight grouping.
Anyone here do pistol, or archery, or rifle?
So we always talk about how tight the grouping is when you're shooting.
That's very precise.
That can be corrected by say, fixing the sites on a pistol, or on a rifle.
But for time estimates, this is consistent underestimating.
Right?
It's like, yeah, it's going to take me two days to do it.
I'll get it done in two hours.
Really, it took four days and four hours.
And you're always doing this.
When it comes to estimation, accuracy is more important then precision because you're going to work on multiple tasks.
And if your estimates are a little bit off all the time, but sometimes you're under, sometimes you're over, that kind of averages out.
Which means that you can still make some effective plans.
But if you are consistently underestimating or consistently overestimating, you need to figure that out quickly.
You need to learn that about yourself during this class so that you can start to correct that.
And if you can't figure it out, hopefully somebody else on your team can realize that your estimates are always off in a certain direction and then adjust the plan accordingly.
It's also important to know what the uncertainty of your estimates are.
It's very valuable to know how imprecise an estimate is.
It's like, this could be done in a week if I find someone else who's implemented a feature just like the one that we want.
If not, I'm going to have to implement it from scratch, and it's going to take four weeks to implement.
I mean, that's a fairly common scenario.
So it's very valuable to know how imprecise an estimate is.
But while I welcome that kind of discussion in a team, don't just give a range.
Don't just say, well that's going to take anywhere between one week and four weeks.
That's too wide a range.
You can't establish a plan on that.
How many of you have been in this situation?
Two minutes remaining.
And it just kind of sits there.
And it's really four minutes later, and it's just what the heck's going on?
It would be much better if you had actually, that progress bar.
And that progress bar is actually giving you a single point estimate.
This progress bar is kind of telling you this job is about 15% complete.
And it's telling me there's two minutes remaining.
And that bar still has a long way to fill up.
So you realize this estimate may not be terribly accurate, because it not working on a lot of information.
It may be a pessimistic estimate.
But you still need to commit to a single point estimate, if you want to make a plan around it.
Don't just use the average of a range.
The difference between a one week estimate and a four week estimate for the same feature, is not somewhere in between.
It's not like it's going to take two and a half weeks.
With the one week estimate and a four week estimate, are two very different circumstances.
One week, if you found somebody else's code that basically does the same thing.
Four weeks if you had to write it from scratch.
One week in, you know exactly what situation you're in.
Either the code is done, or the code is not done, and you know it's going to take four weeks.
Your estimate is not like, two and a half weeks.
You're not halfway in between.
The good news is that the time estimates are entirely internal to your team.
No one outside your team needs to know this.
I don't need to know this, Sarah doesn't need to know this.
None of the graders need to know what your time estimates were.
We're not holding you to-- your grade is not dependent on this.
This means that you can revise your estimates over time.
And I'm going to encourage you to do that.
You are not Neo.
You aren't going to be faster, just because you want to be faster.
So it's important to actually understand what's your actual pace of work.
Now this is something that I'm going to recommend that everybody do starting from today.
And that's to track your own estimates.
This is one way that you can do it.
It's not the only way, but it's good enough.
Where you write down the feature that you're working on.
Say it's the jump feature.
And you have the task that you took on.
I'm a coder, so I'm going to say I'm going to put some upward velocity when the player hits the space bar.
I'm also going to write some code to play a sound effect when that happens.
These are two separate tasks.
And with the original estimations, that was going to take eight hours.
It's going to take five hours.
And then you actually write down how many hours you spent on this task so far.
For the first task, it was done.
It took me 10 hours to do.
Or longer than I expected.
So my eventual estimate was 10 because I actually knew it took ten hours to do.
If I ever have to do that same feature again, let's say a feature of the same size, again, a task of the same size.
Again to be accurate, then I'm going to estimate, next time if I think it's going to be eight, I'm going to write down 10 instead, because that's what previous history has taught me.
Let's say I'm in the middle of writing my jump sound code.
I thought it will take five hours.
I've spent two hours on it.
And I look at it, and I think actually this is a little easier than I thought it was going to be.
So I'm going to say probably only another two more hours of work on this.
And my current estimate is actually being revised downwards.
So this is going to get faster.
And if I put this in a public location, where my Scrum Master to read this, it can actually see my progress on my work.
You might be able to also do this on certain task tracking tools.
Don't estimate in ideal work hours.
Don't say, if I had four uninterrupted hours to just work on this game, I'm going to get this jump thing working.
How many of you ever have four uninterrupted hours?
Really?
Is this like between midnight and 4 AM?
OK. OK. Uninterrupted?
You're not like, checking Facebook, or having to get a snack in between?
No?
Really?
If there are distractions, if there are team meetings, if there are other problems that [INAUDIBLE] that you expect to have to do, you should count it.
You should count it in your task time.
You should be able to account for this.
Because say so you've blocked off 12 hours of-- No.
Let's say six hours.
You are going to spend six hours on this project over the week.
And you know you've got that, because you're going to have to spend the rest of your time going to other classes, and doing other work.
And you know that you've got a one hour meeting somewhere in there.
Part of your task is actually going to that meeting, and figuring out how a certain feature should be implemented, before you actually start writing the code.
Or actually start doing the art for it.
You want to factor that into your task estimation.
You don't want to say that's a separate thing.
At least if you do count it as a separate thing, make sure that you've actually got that as a separate task with an actual time estimate on it.
Count all of your distractions.
Do try to schedule and time box all of your meetings, because it makes estimation easier.
If you know that a meeting's time boxed to one hour, you can't spend more than one hour in a meeting.
And you can actually account for that in your estimations.
Don't just always have ad hoc meetings, for instance, like on Skype.
Because then you're just stealing a couple of minutes from everybody's schedule all the time without any one being able to plan for it.
So you've got something like this, where you're tracking all of your progress, then you just add up all of your time estimates when a feature is done.
And then you can go back to your original small, medium, and large and say this thing took me seven hours to complete.
And if I get another feature that looks like these two features, for instance, and say it's about two half features compared to that seven hours.
I said, OK, three and a half, three in a half.
And you can use that.
It's not going to be perfect.
It's not going to be accurate.
But you're basing it on real evidence of how fast you personally work.
So all of this information helps you become a better estimator of your own performance.
And in the future, you can use those numbers for future sprint planning.
So when you finish one sprint-- say it's project 4, and you've got multiple sprints.
You finish one sprint, and you know that this takes you eight hours to complete.
So the next time you get this size feature, well it took me eight hours last time.
So I'm going to say that that feature looks like a eight hour task.
You should re-estimate every day.
So you animate your jump and say that's too big of a feature, so I'm going to split into two tasks, the animate, the jump take off, and the animate, the jump landing.
Since they are two very similar tasks, I'm going to put the same time estimate on both of them.
And then I'm going to start working on jump take off since you have to go in the air before you can come down.
And I'm going to start working on this.
The task is a little bit harder than I expected, so actually it was a five hour estimate.
So I'm going to increase it up to a six hour estimate.
And then eventually, I got it done.
Down to zero.
I have no more hours on this feature.
It works.
It's tested.
It's integrated.
Everything's fine.
Now I have to do the second feature.
Animate the jump landing.
What should I write down for that?
Let's say by the time I actually came to the situation where I wrote down I had six hours remaining, I had already spent eight hours working on the feature.
I spent eight hours working on jump take off, and it wasn't done.
Then I wrote down I had six hours left.
So what's my new feature estimate?
14.
A lot bigger.
Because that's real evidence.
It may be that during the process of doing this earlier feature, you've learned certain techniques that's going to speed up your next thing.
However, that's wishful thinking.
That's hope.
That's what you desire will happen in reality.
The actual information that you have is that actually if you add up all the hours I actually spend on this, it took me 14 hours.
So I'm going to revise the next feature and let the rest of my team know that I think this next feature is going to 14 hours.
Divide and conquer.
If you've got a very big feature and very, very big task, break into little bits, and do your estimations.
Don't forget to account for debugging.
This is one of the very first actual bugs found by Admiral Grace Hopper, stuck in a vial somewhere in a computer in 1940, that actually caused a program to fail.
That's one theory of where the term debugging came from.
Don't forget integration time.
Taking your code and your assets and your writing and your dialogue and sound effects, and making them work with other people's code and contributions.
Don't forget that some of you are going to get very exhausted and overloaded while you're working on this project.
And don't forget to take breaks, and to account for those breaks in your time estimation.
If you know that you don't tend to work well on a four-hour stretch, and you tend to work much better with one hours, with a five minute or 10 minute break, and then another hour, then use that in your time estimates.
Try to estimate so that you can work efficiently.
Not just how you wish you could be working.
Any questions?
I thought that was kind of fire hosey.
We'll probably get back to this-- actually we will actually have you practice this later on in class.
Five minute break?
All right, so right now we'll do a five minute break while we switch our computers around.
Be back here by 1:53.
OK. We are back on.
So we're back on.
Hopefully this will actually not be terribly long, because at least some of the stuff I'm talking about, Philip has already really touched on, and discussed in more detail than I need to hit.
Since he just talked about it.
I'm just taking a lot of the terms he has introduced and explained and talked about all the estimations and tasks and so forth, and stuffing them back into our Scrum framework.
So you can think about-- given that data, about estimating, what do you do with it?
How do you show it?
How do you share with it?
How do you record it?
But first, let's have a really quick review.
Remember I said there would be a quiz later?
Here's our quiz.
I want to see how many of you actually still remember we talked about on men day.
So can I get some volunteers to give you some definitions up here?
One per customer
All right.
[INAUDIBLE] In charge of the vision of the products, and also in charge of maintaining the backlogs of the team's [INAUDIBLE]
Yep.
Specifically prioritizing the backlog, but yeah.
OK.
Someone else?
This is going to be a really long lecture
[INAUDIBLE]
Great.
Next?
Grab them while they're easy
The product backlog is a list of features to be implemented to have the whole product done
OK. And I saw someone
[INAUDIBLE]
OK. You're already a repeat customer
The sprint is the period of working on one installation of the product.
It should be kept short, so we can [INAUDIBLE].
And each sprint should have [INAUDIBLE] that can be published, or that's running, so that you can only do incremental work
OK. Somebody else.
Somebody else.
We got two meanings left.
Grab the meetings.
Team members?
Don't make me do the whole lecture over again, because I don't want to give it, and I don't think you want to hear it
The retrospective is the meeting at the end where you review your processes
Team member is really easy
Nobody's grabbed that
Team member is everyone on that's on your team
Well done!
Do I have anyone that will take Scrum Master?
You've already answered twice.
Thank you for noble participation
A Scrum Master is in charge of just making sure the Scrums go quickly, and everyone knows what they need to do, but he doesn't actually have to do anything.
He's not actually in charge of planning anything.
He's just in charge of making sure everything goes smoothly during [INAUDIBLE]
Yep.
Yep.
Coach is a great description for that.
OK. Now I think we have not hit the sprint review meeting, but other than that, I think were pretty good.
So the sprint review meeting is the meeting that you're going to get to do in class today.
No it's not.
Sprint planning meeting.
Ah!
Sprint planning meeting is the one you're going to get to do in class today, where you go ahead, look at your product backlog, decide what you can get done, and make a sprint backlog.
And then break that down into sprint task list.
And don't think anybody gave sprint review.
OK. And sprint review is just at the end of your sprint, when you were evaluating your project for how close it came to what you had intended to make.
And it is a chance to change and update the product backlog by the team, usually led by the product owner, since they're the person with the vision and a lot of strong opinions, to modify any changes to the plan that you guys have realized now that you see what you've actually made.
I'll stop torturing you now.
So backlogs, task lists, and tasks.
First step is creating a sprint backlog.
It's pretty basic.
It's pretty simple.
Usually a product backlog ends up having lots of really big stories.
Big fancy features.
The things that you want your game to do when everything is in and it's all working at the end.
By nature, these are usually big stories.
We want to have physics so that the game feels realistic.
I want the player to be able to develop lots of different powers through the game.
I want to be a long complicated story.
I want there to be lots and lots of characters to interact with.
I want a fully immersive world.
Never ever put that in a backlog.
But I want a fully immersive world.
These are all somewhere passed extra large, often.
But in a product backlog, it's OK to go that big.
But when you're actually thinking about what you're going to do this particular sprint, and this particular iteration, you got to get a lot smaller than that.
For example, I want a fully immersive world is not getting done in one sprint.
Even if it's the only thing you're working on.
So clearly you're going to have to break that down into smaller stories.
And the place you do that is usually your sprint planning meetings.
So as you break those really big stories into smaller stories, you can leave some of them on the product backlog, because oh my gosh.
We're not touching those yet.
And then you can decide which of those pieces of the story are complete enough and small enough that you can get them done in the sprint.
Be it one week, or two week, or a month.
And again, the size of stories you're grabbing is really going to vary, based on how long your sprint is.
By definition, the sprint you guys have coming up is one week.
Because your project is due next Monday.
So once you've got your stories broken down that you've pulled out, and you've got your sprint backlog, then you can start taking a look at the tasks.
Except that's not quite how it works.
Because it's kind of easy to grab too many stories onto your sprint backlog at your first pass.
And when you actually are going through and estimating things, and you realize just how much work each of those stories take, you may find you have to put some of those stories back.
You may even realize that some of the stories that were at the bottom of your priority list are actually more important than you thought because you've got other stories higher up that depend on them.
So while you're doing your planning, and while you're making your sprint backlog, and making your task list, all a little bit-- you're going to go forward two steps, back a step, forward three steps as you both reorder your backlogs, put stories back, take stories back off as you're trying to figure out how you get all those dependencies and squished into what is actually a really short amount of time.
You guys don't even have what a sprint team thinks of as a full week.
You do not have a 40 hour work week ahead of you.
You have, per person, probably a 10 to 12 hour work week.
Because that's about the amount of time the class expects you to put into it.
So small.
Small stories.
Small tasks.
OK.
I just went through all of that, didn't I?
How do you break things down?
When you're breaking down your stories, think about what's the most important part of what your breaking down into?
Because your stories have a because, or so I can, you can start thinking about what is most important part of that story.
And which portions of those stories you can just dump.
You don't need a full physics engine to create realistic driving.
Because, what the heck does realistic driving mean in this case any way?
Does it mean that it accelerates, and it really feels like it's accelerating properly?
Doesn't mean that when you handle curves, you can feel it spinning out on you?
What does feel mean anyway?
Does it mean that the computer is playing really realistic sound effects for me?
That I'm getting really cool sparks off of the wheels.
I'm getting this view of the car spinning out.
When you understand what you actually want, then you can create smaller stories that let you break it down and sign up just for that.
You can also take a look at what the abilities of your team are.
And what you can actually do.
If you have somebody who's really get at sound effects, maybe you really want to try to create a realistic aural environment.
You're not going to worry so much about the physics.
You're not going to worry so much about the sound effects.
But boy, is it going to sound sharp.
That's OK. That's working within your capabilities and your limitations to do your best to fulfill your overall vision.
Philip really already covered this pretty well.
I will zip right through it.
Tasks.
Think of them in terms of going down to a half hour is probably a waste of your time if you're looking at a whole bunch of tasks that are a half hour long, that's a really small amount of time.
On the other hand, eight hours is probably the biggest you want to look at, given that that's most of a week.
Ideally, you don't have one task that takes up your entire week's worth of work.
If so, you're rolling some really big dice there.
Because you're assuming that your estimate is pretty good.
And especially the first time you start estimating, your estimate probably isn't that good.
It is more common for people to overestimate by up to twice as much.
So, I can get this task done in eight hours often means I can get this task done in 16 hours.
Think about that as you are tasking things out, and as you are creating your sprint task lists.
So I think I've mostly covered that.
Finally, once you have this big pile of tasks on an Excel spreadsheet that no one actually ever wants to look at again, because they're kind of hard to pick out there, what do you do with them?
I mentioned Scrum boards.
I mentioned them being preferably there a physical Scrum board.
That's really not realistic for us.
So you're going to want-- and I really strongly do recommend that you use-- a visual Scrum board solution.
Two that I have used on projects that work well for different situations are Trello and Scrummy.com.
Don't feel like you have to use either of these.
If you find something better, use it and tell me about it.
Because I want to know.
Especially if it's free.
So let me show you very quickly-- let's see if I can pull them up.
So here it is-- is that showing?
Here's scrummy.com which I actually-- I think it's kind of the easiest and quickest one to use.
This is always going to be more work for your Scrum Master.
So when you're thinking about tasks, set some time aside for your Scrum Master, who's probably going to create this board.
It's going to be your responsibility to maintain it, and move things over, but somebody's going to have to type tasks into it.
And it would work best if everyone could be relied upon to type in their tasks, but it rarely works that way, especially since you often have a whole bundle of tasks that haven't been assigned, at least at the beginning of the sprint, because people start picking them up as they discover that they've gotten stuff done.
So pretty much it works much like a Scrum board ought to.
So you can create all your tasks.
It divides things up naturally by stories.
Although if you'd like to put it by user, you can.
They're two standard ways to organize a Scrum board.
Either organizing all your tasks by the story it belongs to, or by the user who's working on.
Feel free to do it how ever works well for you.
Scrummy, because it's free, and you don't want to pay for the upgraded version, won't let you change the names of your columns.
But it's pretty hard from the end of the world.
So if my stories demonstrate so many of the [INAUDIBLE], I have recorded the URL, so I can get back here.
That's done.
I've made a new board.
That's all done.
And I confirmed the board was there.
I've assigned a task to Philip, so that there can be a task there.
And it's a really pretty quick and easy-- ah!
No, don't delete it.
It's really pretty quick and easy to go ahead and edit things.
It changes color, so you can have different colors.
Easy to tell apart.
And it's really just sort of a click and grab.
Click and drag, except that I don't have a mouse.
Sorry.
I don't have a track counter, and so it takes these fancy two-fingered maneuver to actually move it.
So I'm going to fail to move it over, but they're pretty easy just click and drag to the various things.
To the various sides.
So that's Scrummy, which has less functionality than Trello, but it's actually, in exchange for that, it's quick and easy.
You type in a task, you type in a name, you're done.
Trello-- and I apologize about the resolution here.
Let's see if I can close this.
Trello let's you go ahead and create as many columns as you'd like.
You can name them whatever you want.
You can put in all the information you want.
If you want to track your estimations here, if you want to have a checklist of subtasks that are related to it, if you want to put on pretty little labels so that every user has a different color, or every story has a different color, as long as you only have five stories-- but on the other hand, you can also waste a lot of time organizing your project on Trello.
And it will look really nice, and it will have all the data there, and you will have sucked all those hours that you could have been coding, or drawing, or actually making a game.
So Trello can be really nice to use, but don't let it steal your project from you, if you choose to use it.
I was told that there's a cool Scrum-free download extension.
Runs in the Chrome browser that makes it work better for Scrum because it adjusts lists, and it does some previous set up for you.
I haven't tested that, so I don't know how it works.
If you'd like to try it out, do.
But don't do it this project, because you don't have time to waste mixing it around and fixing that.
Do that on outside time, if you really want to is my advice to you.
I don't want you to spend a whole lot of time getting the perfect system together.
I want you to get a system that works well enough and use it.
Yay.
Back to the slideshow.
So that is Scrum boards.
Ah.
Don't use PowerPoint either.
Apparently it has to be difficult.
So the classic line up on a Scrum board is you've got all the tasks in your to-do pile.
You've got the in-progress.
And anything in-progress should be assigned to someone, should have a name attached to it.
It can't be in-progress if someone isn't working on it.
And it should have a time estimate attached to it.
Ideally, the time remaining.
Although, it's good to keep track of how much time has actually gone by on it.
The task cards are really good places to keep track of your estimates as they change.
Then it's all in together in one place.
Once you get finished, it should not go over to done.
It's going to testing.
Someone who is not the person who did it should test it, and then move it over to done.
That way you've got a second set of eyes on every piece of code and every feature, just confirming it's working.
Many Scrum boards also have another column called Blocked or On Fire.
Which is to say this is a task that someone can't get done until someone else gets something out of my way.
It's nice to have.
It's not required.
And if you're working on a board that you're having a hard time fitting everything into one screen, Trello has always a problem for that with me, it's OK to leave it off.
But it can be really helpful to have.
That sort of is a Scrum board.
And I do really recommend them.
They are useful.
I know it feels like a lot of extra time often, when you're setting them up.
But if your team is actually looking in there and checking it every day and updating, it's a really good way for the team to know where it's at without having to spend a lot of time talking about it.
Because all the really basic stuff is there to be seen, which means you could freeze you up to talk about any real problems that are coming up.
OK.
So that said, it's your turn.
You've got a week left on your project, so let's try creating a sprint backlog and a task list from that backlog for what you're going to get done for Monday when you have your turn in.
I'm going to give you-- like all meetings, they should be time boxed.
So I was going to give you half an hour to do it.
I will check in at half an hour, and if people are screaming oh no, we're nowhere near done, I will give you another 10 minutes or 15 minutes.
Not actually a whole lot more time, but I'll give you a little more time.
This is probably doable in half an hour if you guys have been working on your project, and you've got your product backlogs in good shape.
But it's the first time you're doing it, so it's always-- it's hard to know what the right amount of time is.
After that, I will give you five or 10 minutes-- probably call it five minutes to talk about the process, and have someone come up and give a very short presentation of what the process was like.
What worked, what didn't, whether you liked it or not.
It's OK to come up here and say bleh.
I don't feel like this helped us at all.
Just come up and tell us what happened, and how it works for your team.
[CHATTER]
OK.
So [INAUDIBLE] meaning we used Scrummy to try to organize our product backlog into our sprint task list.
We found Scrummy to be a little annoying, because it doesn't automatically update instantly, you have to refresh the page.
And whatever happens last, we find, is what it shows as what's edited.
So it shouldn't break, but it's not the most useful thing.
I think more useful would be to Google use the Google Doc of the product backlog.
Section out the stuff that you do in that sprint, and then just update it there.
We had it all color coded, and so it looks really nice.
And that's much easier to look at and do than importing that into some other program, and then doing everything there and then going back.
It just seems better to use Google
OK.
Thank you.
[APPLAUSE] [INAUDIBLE]
OK, so we used Trello for our Scrum planning.
And we liked Trello because you can just create cards that have lists of tasks and easily move them around between different columns.
So when someone takes on a task they can move it to in-progress and assign their name to it.
And it' really easy to look at and see what's going on at a glance.
Trello does have a lot of features like checklists that you might not want to use a lot of, especially if you have really complex tasks.
You don't want to put all of those inside the checklist because that gets confusing.
The thing that we had the most problems with was doing time estimates for the tasks, because no one in our group has used Phaser before.
And so we have a very kind of shaky grasp on how long things are going to take, so our guesses were mostly random at this point in time.
But we will update that as we get more experienced.
One thing that we did find was really helpful was when we were drawing up goals, we broke down all the features that the game needed into as fine detail as we could because we know what results we need to have to get the game to work.
But we don't know Phaser yet, so we don't know exactly what the processes are going entail yet.
But having like the list of things that need to go into the game gives us some guidance on what we should be developing towards
Thank you.
[APPLAUSE] Sparkling Redemption
Sparkling Redemption.
We basically just opened up Google Docs and made a whole bunch of Excel spreadsheets.
I think we're kind of trying to use Scrum Do and see how that works, but we figured it'd be best to just kind of finish the thing first, and then take a look at that.
So no thoughts there yet.
I guess we basically just took a look at what we wanted to get done by Wednesday, and what we wanted to get done by the end of the week, and then broke that down into the tasks, and then worked with that.
I guess estimation was-- it wasn't too bad.
Because some of us have worked with Phaser before, but there was also a lot of things where we realized oh, well, assuming x is finished first, then y would be really straightforward.
But do we count x's time inside of y?
Well maybe, but maybe not really.
So that was as complicated as it got, I think
Thank you.
[APPLAUSE] Blundermans
So for us, we started out with a fairly detailed plan.
And we used Asana, which is multi-player to-do list.
So, similar to Scrum.
But ultimately, our entire team more or less got sick over the weekend, and we didn't actually get much work done.
So we can't speak to exactly how good are estimates would have been, or how useful our plans would be.
So all the planning in the world doesn't help if your entire team is in bed
Yep.
Thank you.
[APPLAUSE] Blind Aliens
So our team used primarily Google Docs.
What we did was we had our product backlog.
And we were a little confused about the product backlog.
So we made it way too detailed.
But that really helped with this.
So we ended up copying and pasting our product backlog into our sprint task list, deleting everything we'd already finished.
And then adding based on what we had done.
And we found that because we were further along in our project, it was really easy to tell what needed more work, and to get the tasks more detailed and better estimates, just because we had done a lot of already.
[APPLAUSE]
So we started out planning on using Trello, but did the product backlog in Google Docs.
And since it's sort of this information repeated twice, we just stopped using Trello, and ended up using Google Docs anyway.
And that sort of helped us out now, because all we did was sort the things in Google Docs by priority.
So we looked at the things that were highest priority so we're doing this week, and then we just sort of thought about each one.
How can we break this down?
What are the specific tasks?
You started working on this.
What's left to do here?
And then we put that into the sprint task list
Thank you.
[APPLAUSE]
So we've been using Google Docs from the beginning.
We decided to not use any other site, because we had already started working on Google Docs.
And it's worked out pretty well for us.
We were able to take our product backlog and take the highest priority that hadn't been completed yet, and just split them up into a finer detail task that we can use for the sprint task list.
And I think that that worked fine for us.
It helped us really scope out how much time we had-- helped us reevaluate how much time we needed to complete the task that we had to complete.
That is going to help us for planning for the rest of the week
Thank you.
[APPLAUSE] All right, so I have in fact, as we say, iterate on everything.
I've iterated on this presentation.
And been changing the order of it.
So it may be I get confused a couple of times.
I hope not.
We'll see.
We're going to talk a little bit about high level and low level here.
But as we said, in this class we want to iterate on as to the every process you might use.
This is how you control your team.
This is how your game works.
Everything.
And this is actually not a bad philosophy to do everything in your life, that you might want to consider trying stuff, fixing what doesn't work, embracing what does work, and moving on.
But this also applies to how you write your code.
And I can talk about-- on a high level and a low level.
The where you put your parentheses, and exactly what kinds of variables you use is a process.
And you make decisions on that basis every moment that you're coding.
And it's a really good idea to think about what you did and how well it worked for you, or someone else.
And this is the kind of thing that maybe in your early programming career you don't want to think about too much, because just getting the stupid thing to work in the first place is kind of hard.
But I'm going to try to convince you that it's worth your time to get these habits going, so it's easier for you in the future to debug your code, or someone else's code.
So I want to talk a little bit, very briefly, about something I mentioned in the review on the lecture on game engines, which is that all software sucks.
And it sucks in different ways and all that, right?
Well, the problem with this is is that yours does too.
Your software is going to have bugs in it.
And your job is to try to make that happen as little as possible.
You want your software to suck as little as possible.
Now we also talked about one reason we use a game engine is just so that you write less code overall.
You spend less time doing that.
That's because everything that you do when you're running code is slow.
And in particular, debugging is the slow part.
It's not too hard to write new code, but it's actually pretty hard to go into old code and fix what happens.
At it varies, depending on the complexity of your task, but generally speaking, when you've written a bug, later, you have to spend the time to figure out that there was a bug, figure out where it was, and then you actually have to fix the stupid thing.
And that is actually where most of your time is going to go, when you're trying to develop code.
So again, figure out the problem, and then change the code to fix your bugs.
But I acclaim that most of the time, figuring out the problem is more of your time.
Now this isn't true for the small bugs, when you're typing something at your computer, and it doesn't work when you compile it or whatever, and it runs.
You go back and fix it right away.
Yeah, that part of debugging is not so much figuring out the problem.
Because you just wrote it.
It's fresh in your mind.
You know what's going on there.
But if it's been a couple of days or a couple of weeks, then actually the hard part and the slow part, is figuring out what went wrong.
And so one of the things that you need to think about when you're writing your code, is you want to make it as simple as possible so that you're less likely to write a bug in the first place.
Or once you've written a bug, it sticks out.
It's easier to see.
And the important thing to remember here is that all this code you have written is not as easy to read as you think it is.
And the right way to do this is to make it as easy as possible.
And the way I like to think about it, at least today, is to make it require as little knowledge as possible to figure out what's going on in your code.
If you use some really fancy language features, that's really cool, and it feel like you're really smart when you're doing it.
But a week later you back to look at that code to try to figure out, I don't even know what I did there.
And heaven help you if someone else tries to read your code.
So ideally, you don't want to go too obscure into the computer language fancy features.
You don't want to go too much into your subject matter.
But there are exceptions to this.
If for example, you work on a very specialized piece of software where everyone has to have an arrow engineering astro degree to actually work on the project, you can assume some level of competency there.
But generally speaking, in game development, and most application development, you want to assume was little knowledge as possible, so that when you want to debug your code, you can look at the little part of the code you're at right now, and see if you can fit what's going on there.
If possible, you want to assume no knowledge on the part of who was reading your code.
And I'm going to talk a little bit about why that is.
This is a useful thing to think about for user interfaces or really any problem solving.
Humans can't hold that much in their brains at once.
And if you're trying to write code that requires you to think about 10 different things at once, you're in trouble, because humans really don't do that.
Now generally we get around this problem with what's called chunking.
So when you started with math, you thought about the number five wasn't even a numeral to you.
It was five objects.
But you couldn't really wrap your brain around 12, because you can't look at 12 objects and think oh, I understand what that means.
But as soon as you practice with your numbers more, you begin to think of number 12 as a unit.
It's not 12 things, it's one thing.
It's the concept of 12, and that's really useful to you.
And then you started working through algebra and you got variables.
And now you're using calculus, and all the other cool things.
You talk about vectors in math.
A vector represents three numbers, or a direction or position in space, but you don't think about all three numbers.
And you certainly don't think that 35 is one of those numbers, and that means that there are 35 things.
Your brain isn't wasting time on that.
Instead your brain is thinking, it's a vector, it's a point space.
That's all I need to know.
So your brain is ignoring a lot of information by clumping a bunch of information into one spot.
I often use phone numbers as an example for this.
And it's an increasingly poor thing to use, because I bet most of you don't memorize phone numbers, because why would you?
But when they started [INAUDIBLE] phone numbers, there are three parts to a phone number.
First part is an area code, the second part is a sub area code.
The third part is the four digits that are more unique, right.
It's really hard to remember those 10 numbers.
But actually, once you learn the area code, the area code is three numbers.
You think of it as one number.
Boston is mostly 617.
The second set of numbers is also three numbers.
You clump that as well.
And the last four numbers, yeah, you have to memorize those.
But basically instead of remembering 10 things, you have to remember six things.
The four numbers and then the two prefixes.
And another problem with coding is that as you keep doing it, you're making decisions constantly in your coding.
What's the right way to do this?
What's the right variable you're going to use?
Et cetera, et cetera.
And as you do this, you get tired.
Humans only have so much decision making in them before they need a break.
So a couple of things we can do is we want to reduce the amount of things you have to remember while you're looking at your code, and we want to reduce the number of decisions you're going to have to make.
And to some extent, although I'm not really clear on a psychological basis for this, I'm going to make a crazy claim that figuring something out and deciding what this thing must mean is a decision.
So even if you're just trying to figure out something, that also is a burden on your brain.
And what you want to do is keep your brain alert and hopping as much as you can, for as long as possible.
And to do that again, we want to simplify your code if you can.
So to go back here, simplicity-- when I say simplicity, I mean you want fewer things to think about.
Should be easy to read the code, so you don't have to memorize what the code is doing.
You can just quickly glance and get a reminder of what it's doing.
You have fewer bugs because there's just fewer moving parts.
Fewer things to go wrong.
I'm going to say this hopefully a couple of times, but a short variable name is not simpler, it's just shorter.
If you start thinking about what you spend time doing when you're writing code, and trying to get something done, what takes the most time?
I'm willing to bet it's probably going to be debugging part, but maybe you think it's going to be getting the stupid thing to compile in the first place or whatever, but I'm willing to bet almost none of you think that your typing speed is the thing slowing down your coding.
If only I could type 80 characters a minute, I could double my coding speed.
It doesn't work that way.
You do a lot of thinking, and just a little bit of typing.
So if your variable name is 12 characters long instead of four characters long, that is not cost you time.
It just costing you eight keystrokes.
And in a modern development environment, you've got auto complete.
You type the first four characters, you hit control space, and you get all 12, hey.
You can't even claim that you've saved eight characters, you've only saved six.
So anyway, I'm going to talk about that a lot, because I believe that the longer variable names are really useful cues to remind you what you're doing.
And one of my favorite things to talk about is write once, read never code.
Who here has used Perl?
Anyone?
Have you tried to read other people's Perl code?
Yeah.
Why would you try?
Have you tried reading your own Perl code?
He goes, no.
That's right.
You just write it again.
Absolutely.
So, Perl if you're not familiar with it is this cool scripting language that's not used as much now, but it was once the open source darling.
PHIP is kind of modern equivalent of, my god, don't try to read that code.
Perl had the nice feature where pretty much every symbol on your keyboard has a meaning.
And if you want to do something, there's probably three different ways to do it, with three different obscure characters on your keyboard.
Worse, each of those characters has a different meaning, depending on what you're using it on.
But this is an example of APL code, which a stands for A Programming Language developed in the late '60s, early '70s.
And it's really painful because in APL not only does every character on the keyboard have meaning, they've minted special keyboards with an extra three characters per key, so that you could have more characters.
And this is awesome if you think of writing code as if piece of paper with your pencil and you're doing math, and you're writing fancy integral symbols, and this and that, sure.
It's handy in math to chunk those concepts this way.
But for coding, generally speaking, we're stuck on our alphanumeric keyboard with a couple of symbols.
And I advise you to use as few of those symbols as possible.
Try to stick with your language if you can.
All right.
Who here thinks that you write enough comments in your code?
Wow!
Some of you do.
But almost none of you-- maybe a quarter of you are like, yeah, I'm proud of my comments.
It's cool to be proud of your comments if you're writing them.
And if you're not writing them, don't be so proud of them.
But sometimes I see comments that are just duplicating what the code is saying.
If your if statement says, if x is less than 5-- don't put a comment up there that says if x is less than 5 do something.
That's a waste of time.
I can see that in your code.
But I want to know why you care if x is less than 5.
That's the kind of comment I would like to see in your code.
And we'll get to this a little bit later, but I also talk about longer variable names, longer function names.
Those are useful ways to dodge writing comments if they are correct.
But really quite frankly, if you're trying to figure out gosh, I don't know if I should comment this or not, write one.
It's not going to hurt you.
So here's an example.
These comments don't mimic the code, but what they do do is they tell the story of what the code is actually doing.
Random aside, font is actually very important with your editor.
Make sure you use a fixed width font because pretty much all programmers do.
And sometimes you use spacing to have everything line up nicely, and you want to have that be crystal clear to anyone else using your code.
But there's another way to this.
So this is code.
You can figure it out.
Go ahead and figure it out.
There's no rocket science going on here.
We're using the Pythagorean Theorem to calculate a distance, and make sure that if it's small, we return 50.
There's nothing hard here.
A programmer can figure this out, but the programmer shouldn't have to figure this out.
And that's my argument.
If you takes you-- I don't know, maybe you're great and it takes you two seconds to figure that out.
Maybe it takes you 30 seconds to figure that out.
It doesn't really matter.
It should take you less than that if the code has been written correctly.
So for example, I'm just looking at the 900 to 30 times 30.
Because why?
The compiler doesn't care.
It's a little bit easier to read.
You can say if the distance is less than 30, as opposed to, if the distance squared is less than 900
And it's also the bug.
This also has a bug
This does have a bug.
Good catch.
There's a bug here.
This code doesn't work.
Because we're saying y0 minus 1 instead of y0 minus y1.
So there's a better way of doing that.
This is even easier to read.
We are using a vector instead.
The vector class has some internal knowledge of what it needs to attract vectors, and what the length of a vector is.
Not only is this less bug prone, but it's just easier to read.
And that bug that you caught there can't happen.
Now this code is a little bit cheating, because I used v for the name.
So you can guess that they're vectors, but that's fine.
Here's comments.
Basically the same code.
I'm saying, hey, here's some context.
This is a lot more code.
In some ways it's harder read.
In some ways it's easier to read.
That's one of the things that you're going to have to figure out for yourselves.
What coding styles make sense for you?
Obviously, simpler is a subjective question.
But I kind of like the style of code because what it does is it means that even if you don't comment your code, your code still is commented.
Because you can read this and know what's going on.
More importantly-- and this is the one I've change since the last year I gave this presentation.
I thought that the comments saying why, and then the code that did it was probably the better way to go.
But a friend of mine pretty quickly convinced me that actually as codes get maintained for a year or so, you might change the code and forget to change the comment.
And this actually happens a lot in real software projects.
With the kind of commenting or the kind of self documenting code that I'm using here, that can't happen.
If you change the code, you change the comment.
It's innate.
They're intrinsincally linked because they are the same thing.
If you can do this, it actually helps a lot.
So I'm going to go through a variety of tips and tricks and things that might make things a little bit easier to read.
But before I do that I want to talk to you a little bit about to why and who it is you're helping.
So a while back, I was complaining to my grandmother that wow, you know, last year I made a bunch of stupid decisions.
I didn't do this right, I didn't do that right.
I'm so much smarter now this year than I was last year.
It's amazing.
My grandma said, yeah, I feel that way too.
All the time.
She was 84.
And I thought oh no, when I'm 83 I'm still going to be a moron.
So one of the people that you're writing code for-- one of the people who will be reading your code is not other people on your team, although they're very important, but it's you.
You will read your code later.
Have any of you looked back at code you wrote six months ago yet?
You're early in your career, so not many of you have.
But I'm willing to bet when you look back at the code you wrote six months ago you think yourself a couple things.
One, wow, I was an idiot.
And two, you're going to think to yourself, I could write this better now.
And three, you're probably also thinking even if the code works, the algorithm is solid, this is decent, it could be more readable.
Maybe you're not thinking that yet.
I want you to start thinking that because you're going to forget the code that you write.
And again, when you're just starting out, that seems impossible.
That you can't forget code that you just wrote.
But I guarantee you after you've been writing code for five years or more, a lot, you're going to discover that you don't remember what you wrote yesterday, or last hour, as well as now when you're just learning and getting deep into it, you remember a lot more than you're going to.
But keep in mind that experience.
And you go back to your code and you don't remember what you wrote, that's the experience your teammates have every time they look at your code.
They didn't write it.
They cannot remember it.
Your code is a communication opportunity to talk your team.
To let them know what it is that your code does.
So one way to make code more readable, shorter functions.
You hear this sometimes, all of us are eventually write that 500-line procedure that does this, then that, then the next thing to the next thing.
We all do it, because it's the easy way to do it.
You think, I'm going to do this thing, so I do step 1, step 2, and you march on through.
It's reasonable.
It's a way to get things done, especially when your work is in progress.
But if you ever want to go back and fix that code, and now you have to figure out what pretty much all 500 lines are doing.
And that's kind of dangerous.
The easiest thing I can recommend there is really if it is a step 1, step 2, step 3 kind of thing, why not divide steps and sub functions?
The subfunction may only get called once, but that's OK.
The reason this helps you is twofold.
One, it gives you a nice name.
For this section of the code is where I am going to tell all the planets to do build spaceships this turn.
I make that a function call.
I don't have to worry about what's inside that function if I'm trying to debug a war mechanic.
Because probably, the part where all planets are building spaceships isn't relevant to the gigantic space battle somewhere else.
But 2, when you subfunction this, inside that subfunction you are guaranteed that the local variables there are unimportant for everything in the big function.
The scope is such that they can't affect the big functions.
They are local variables.
You don't to worry about that.
So this is a tiny couple little class of bug that you can kind of dodge by something as simple as subdividing a function.
Two-- and this is the other important thing-- you've got this function, you strip it down so hopefully you can see it on a page in your editor, and you can read in English kind of, what it's doing.
You see here the five steps.
There they are.
It's a little bit easier.
And that's kind of a simple, straightforward example, but it's that way of thinking again, that I want you to think about.
If you ask five programmers, what's the right thing to do?
They're all going to give different answers.
And that's fine.
But the thing I want you start doing is thinking about what is it that you think is the right thing to do?
Why would you consider doing it this way versus that way?
And if you just write the code and don't think about it, you're not going to get better as quickly as you will if you think about why you wrote the code that way, and how it's working for you.
Here's another fine example.
Making wrong code look wrong.
This is perfectly valid code unless I make my variable names a little more descriptive.
And then you discover that that's wrong code.
That code is terrible.
Degrees times radians is a classic mistake.
Centimeters and inches has destroyed Mars rovers.
Milliseconds, kilometers, and miles per hour.
That's probably not going to work so well.
And these are actually-- you say, I never do that.
But again, I kid you not, we have lost billions of dollars to unit conversions because people didn't do this.
It's going to be annoying to you, when you're typing it in, perhaps.
But I guarantee you, you'll be safer if you do it.
Again, random stuff.
You've probably heard all this.
And we you're doing games, you get a chance to really experience it firsthand.
Try to avoid sticking numbers deep in your code.
No magic numbers.
You want to stick it in a variable somewhere.
This has a couple good features.
One of the biggest is, you've got someone else on your team can easily look up that number and change it.
They don't have to find it in your code.
They can just look at your list of variables somewhere, and figure it out.
Immunity, it's even more awesome, because you can just go to the editor and change it around.
Also, you can change it globally everywhere.
If the number 50 is magical, great.
But call it something else so that you can change it to 51 when you're tweaking your game plan.
I've talked with this before, and I will talk about it again if I get half the chance.
You want longer variable names that say exactly what they are.
You don't want to have to guess.
They should be pronouncable.
And this isn't true for everyone, but most people, will tend to in their mind, to say a variable name out loud.
And so if it's a bunch of random consonants that don't go anywhere, it's a little bit harder to read.
If they look similar, you don't want to have a typo kill your project.
Spell them correctly.
And this is actually really important, especially in interpreted language that just doesn't do anything except running at run time.
In Python, for example, I believe, if you declare a variable that's never been seen before, it probably has a valid value.
No.
OK, good.
I'm glad that's true.
JavaScript.
That's what I'm thinking.
But JavaScript-- be very careful, especially if you're using Phaser JavaScript.
If you mistype the name, it's 0.
That's no good.
More and more modern languages are rejecting that.
But the other thing is just it's easier for your teammates to know how to spell it.
If you all agree on a spelling.
And if you're all going to agree on a spelling, you might as well agree on the correct one.
And then reusing a variable name is also a little bit dangerous sometimes.
If you're exiting a loop, and you've maybe failed to initialize the thing properly.
You reuse a variable name elsewhere.
You got this weird value coming in, and that's a bug waiting to happen.
And it'll be kind of hard to find.
Function Names.
All the same rules basically apply.
You want to have it be distinctive, spelled correctly, you want them to say what they do.
It's tempting sometimes, to make your function name really, really short.
Again, that doesn't speed you up.
If a function is longer, you get a chance to read what it actually does.
But the other thing about function names-- and this is even more true than it is for other parts of coding-- is that your function name tells your other programmers what the heck you just did.
I wrote this function.
Well, what does it do?
Well, read the function name.
It's right there.
That's ideal.
You can't always pull that off.
And when your function does something more complicated than you can describe in four, maybe five words, then you need a lot of comments to describe what that function is doing.
Ideally, your function should be simpler than that, if you can manage it.
By the way, I took to mention variable names and links, and a lot of you are probably thinking right now, yeah, a longer variable name sounds like a good idea in principle, but I don't want to type that.
It would slow me down to read it.
There's actually been research done on this topic.
8 to 20 letters on average, is often a marker for good code that works with fewer bugs.
So I'm not talking a little bit longer.
If you can push it, push it harder to be as long as you can stand.
And then you'll be a little bit happier with that.
If you are coding in VI or Punch Card still, you could complain about a 20-letter variable name, but you aren't.
You're using a modern development environment, with auto complete, I hope.
Variable scope and names.
Ideally, and this is a thing we strive to do, and don't always succeed at, you should name your variables so that you can tell by looking at them where they come from.
By which I mean is it a global variable?
Is it a local variable?
Is it an argument to the function?
That kind of thing.
You can sometimes do this by prefixing, or post fixing, or camel casing, or not camel casing.
That kind of thing.
It doesn't matter which system you use, as long as you're mostly consistent.
Trying to be consistent among different programmers is often difficult, because everyone will horribly disagree as to what the correct answer is.
But if you can try to sort of remove that from your brains and say, actually it doesn't matter what it is this time, I just need to know what you're doing.
That's the important bit.
Parallel rays.
These are free arrays.
And number 3 in each array is related.
This is something that you're going to find yourself very tempted to do a lot, especially in a game.
And I recommend against it.
Even if you only have two parallel arrays, I recommend making a container classic that contains both pieces of data, and have them stick in one array.
Just because you're going to get bit sometime by having these arrays get out of synch.
You just want to avoid that if you possibly can.
Order of Operations.
There's an expression up there, which is probably legal code in most languages.
And we are taught in algebra that we should memorize the order of operation so we know what gets multiplied when, and all this.
But in computer language we've got a lot more, we've got incrementation, we've got raise to the power of, we've got modulo, we've got all this crazy stuff.
And my argument is don't.
Why waste brain space on memorizing the order of operations for the eight different kinds operations you have, when you can just use parentheses and remove all doubt.
Plus, you could also even divide that-- I'd divide that up into a bunch of statements, actually.
There's no reason to keep it like that.
There's no way that someone looks at that expression and thinks, this is the most intuitive way to understand this.
If you're writing on paper in math class, you might find that because you can use two dimensions, you actually can make your equation more intuitive.
But in code, you mostly only have the one dimension.
And so it's kind of hard to really make your math intuitive.
So I would put extra effort into just split it up.
I also mentioned this last time, I'm going to mention it again.
It's very important.
Warnings should always be treated as errors.
I can could repeat it again, but basically every new warning is a hint something might be wrong.
And if you let your build be cluttered warnings, you will not notice when the important warning pops up.
And you can lose hours to that, just because the build wasn't at zero warnings.
Backwards Conditionals.
Which of these looks wrong?
The second one looks wrong.
I agree.
The second one totally looks wrong, and yet it's useful.
Because what happens if you mistype the equal to equal sign?
If you mistype the equal to equal sign, that second one doesn't compile.
Depends on your language of course.
But generally speaking, it's a handy little trick that I find useful, and everyone else I know thinks is terrible.
So don't necessarily use it, but try to find tricks like this that cause to be the case, that when you mistyped your code, you get a compile error instead of a bug in your code
Do you think what the signal equals might be a client warning?
It will be a warning, absolutely.
Which is an even better reason to not to go down to zero warnings, right?
Doesn't help you if it's a warning that you just let go by.
And it should be a warning.
I'm glad they added that.
When I start programing, it wasn't a warning.
Wasn't that wonderful?
I mentioned splitting up the math.
I think that that first thing is, you can read it.
You can figure it out.
I think that the second thing might be a little bit more comprehensible.
Again, it depends on who you are.
This is one that I'm emphatic about.
I mentioned order of operations before.
Bar minus minus.
That's fancy.
In some languages, and I assume most of the modern ones these days, bar minus minus means do the math, and then decrement bar by 1.
If I'd said minus minus bar, I'd have meant decrement bar by 1, then do the math.
And this is pretty cool.
It's this nice little language trick.
But if you don't happen to know what that means, you just confuse your fellow programmer with your language trick.
That second thing is perfectly clear.
You know exactly what happens.
Do The math, then decrement the variable.
And if you really want to get pedantic, I guess you could say bar equals bar minus 1.
But you have to draw the line somewhere.
I'm a fan of Booleans being asking a question.
Like the Boolean purple probably means is purple.
The Boolean hungry probably would mean are you hungry.
But the Boolean desk might not mean anything.
But in any event, try to make your Boolean into a question that has an unambiguously yes, no answer.
Which way is true?
Which way is false?
Status is a particularly bad one, because I don't know what that means.
Is status true good?
Is status true mean that something bad happened?
I don't know without some help.
You've probably heard this before, mysterious constant, don't do it.
Ideally, if you can define it in your language, you get an error.
When you mistype it, and that's much better.
Mysterious string constants is particularly the worst.
And you're going to be very tempted to do it in JavaScript.
But you should probably have a defined of some kind at the top, so you don't get that problem.
This is a language construct which is very handy, but I recommend using it sparingly.
If you are asking a question-- if you want to set a variable to a value, it's often very convenient to do this syntax.
But really, you don't want to ever nest these.
If you nest them it becomes unreadable gobbledygook.
And if your expressions become complicated, it also becomes unreadable gobbledygook.
Again, I'm not saying you can't figure it out, because I'm confident that you can.
But why do you want your future you to have to spend 30 seconds to a minute, figuring out what the heck you were saying instead of two seconds figuring out what you were saying?
Goto.
We are told that goto is evil all the time, always.
I claim it's not always evil, but you should be careful when you use it.
It's particularly useful when you're exiting a horrible set of nested if statements.
But the main reason goto is evil, is because of a thing called proximity.
Which is that you should use things near each other in the code.
If I declare a variable, and it's two screen pages up from when I use it, that's kind of a difficult thing to figure out.
That's one reason to keep your functions short.
But instead, if I can clump it together, I'm less likely to make simple little errors.
Right here, I'm doing the same thing to x and y, and I'm interleaving it.
Because I'm thinking of it as, I want to prepare them, I want to calculate it, I want to print it.
But it's a little bit easier, I would say, to debug and think about it, if you treat each one separately.
And if you get a lot of these, write a loop or something.
There's no reason to actually type it all out.
All I'm saying is keep it simple.
As simple as you possibly can, so that later, when you go back to it, you can figure it out quickly.
You're going to very tempted to make complicated systems in your prototype games, even though you're game spec is changing, probably daily.
So you might want to be a little bit cautious about that.
Rather than making a complicated system, solve the problem you have today, and then maybe two days from now, when you know for sure you're actually doing that, you might want to put more effort into it.
This is advice that's not good for all projects, but it is good for very short projects.
It's tempting to create a complicated system that will do everything you need.
And it's tempting to create one solution that will solve all of your woes, but you might very well be borrowing trouble and solving a problem that you're never going to have.
So given that you're probably talking about fast iteration.
Your game is changing all the time.
Don't-- and this is counter to my architectural design thoughts-- but don't design too much, because you're going to change it all anyway.
Other things that cause bugs are recursion.
Recursion is awesome.
Recursion is fun.
Recursion is really hard to debug.
Try debugging someone else's recursion, and you'll be very sad.
To try debugging two things that call each, other sadder.
And it looks really cool, but I recommend don't do it.
Similarly, optimizing.
Wait.
You think you know what slow and what's fast.
Chances are, you don't.
If your development system has a nice profile, it'll tell you exactly where the bottlenecks are, exactly why your code is slow.
Wait for that, and fix the actual problems.
Don't waste time chasing ghosts.
Particularly this happens a lot when you're trying to do some code, and you're thinking oh, this is really important.
It ends up being called once per second.
Well, once per second in computer time is never.
So don't bother optimizing that.
But the nice thing here is that a profile can tell you what you need to actually fix.
I've mentioned fixed treat warnings as errors before.
I'll mention it again, probably.
But similarly, you want to fix a bug as soon as you find it, if it's in your head.
Don't think oh, that bug is easy to fix, I'll get to before the end of the sprint.
Because you might not remember it four days from now as clearly as you do right now.
Also, if you've just written the code, it's much, much easier for you to actually fix it, because it's all much more fresh in your mind.
Strongly consider not fixing bugs that don't matter.
And this is a tough one because I just contradicted myself by saying fix them right away.
And I'm telling you sometimes don't fix them.
But basically, if fixing the bug is going to cost a lot time, and it's not going to improve your game enough, then consider leaving it.
That's going to be a thing you'll have to talk about with other people your team.
But hey.
So for example, if you've got every j in your text renders green for some reason.
And to fix that, you need to go into the font library and do some crazy code.
Well don't.
Claim it's a feature.
J's are green.
Obviously, if you're doing Microsoft Office, you can't do that.
But we aren't.
We're doing games.
So we can cheat like that.
Use a debugger.
If you do not yet know how to use a debugger, take advantage of this class to learn how to use a debugger.
I can't really say more about that.
Do it.
When you're also try to figure out a bug, talk to a teammate.
Even if your teammate has no clue about your part of the code, the act of putting into thoughts and words, this is what my system is doing, will, a good third of the time, cause you to realize where the bug is.
It's kind of silly, but it totally works.
So if someone comes to you on your team and says hey, I need help.
You're like cool.
I'll listen to your problem.
And they talk to for 15 minutes, and then say, I got it, and walk away, don't feel useless.
You just saved them an hour.
And it cost you 15 minutes.
And that's a bargain.
And don't feel embarrassed about it, if you're the person talking about your bug.
It's great.
They may ask you questions that are irrelevant, and you're like, no, no.
That's OK too.
You have to explain it.
That orders it in your head, and then you can solve the problem.
And then of course, take a walk.
I have solved more bugs going to get my lunch, or in the shower in the morning, or whatever, than I have at my desk, I would say.
At least more tough ones.
Binary search is a thing that's kind of hard to describe.
But rather than necessarily saying, this could be the problem, I'll test that.
This could be the problem, I'll test it.
If you can do a test that says, well, roughly half of the potential problems that could cause this bug are over here, and half of them are over here, and I can do one test that won't tell me where the bug is, but will eliminate this entire tree, that's a good test to do.
You can quickly narrow in on where your bug actually is.
And this one happens a lot too, with beginner programmers.
If your bug goes away, it probably hasn't gone away.
It's just hiding now.
So be sure you know why the bug went away.
If it's the green j's, you might not need to worry about it.
But even then, you might want to think about it.
Because you thought you knew what the green j's were, and they went away, and you didn't fix it, but who knows what's going on here.
It's worth putting a little bit of time in to make sure that it isn't some serious bug hiding under the covers.
I'm not going into source control again.
We've already done a little bit of that.
You need to do source control.
If it all possible, everyone on the team should be able to just get the code and do a build.
That should be as hard is it is.
And on that realm, daily builds are awesome.
If you can afford to, have some on your team whose job it is, once a day, midnight, pick a time, do a get, a fresh get, do a build, make sure everything works.
This sort of forces all the programmers on the team to sort of be more careful about making sure their code works.
And also if something gets checked and it breaks everything horribly, you know about it right away, not three days later.
There are automated tools that will do this for you.
At least they'll check the compilations step.
That might not check that makes sure everything is still working, but, you know.
It gets there a lot.
Coding Standards.
Why we use them.
We like tidy code.
And obviously, the braces go in one place or another.
Those are terrible reasons to have coding standards.
Instead, these are the reasons that I like a little bit more.
Which is my code-- don't touch my code.
Well, you're on the same team.
I'm going to touch your code if I need to, and it's midnight, and you're asleep and I need to fix the big.
That's the way it is.
You shouldn't be mad about that.
We're trying to fix the game.
The fact that that you wrote the first pass in the code doesn't really matter.
And then, of course decision fatigue is a thing I mentioned earlier.
We talk about what variable names to use, and what algorithm to use, or this or that.
We got plenty of decisions to make.
Where you put your curly braces should not be something you waste time stressing about.
If you have a coding standard, use that.
If you don't have a coding standard, be consistent.
If you're modifying someone else's code, stick with their coding standard.
Because if nothing else, when you do the check in, it will look like the entire file has changed just because you changed the indentation strategy.
And that's not very useful if I want to know what changes you made.
I'd much rather have a minimal set of changes that I can look through and see what the actual meaningful changes you made were.
And of course, easy to read.
Easy to debug.
A lot of things I've been talking about are things you can read a lot more about.
There are people that have written serious books on the topic of making your code as easy to read as possible.
Steve McConnell in particular, is at this point-- his books are older, but they're still perfectly valid.
Because the science of writing code has not really changed much.
Joel Spolsky also has a much easier to read set of blog posts that talk about this kind of thing.
It's much more accessible and it's free.
And he's the person who-- his company is behind Trello.
Which kind of indicates that his method of thinking kind of matches with our method of thinking.
There's a lot more things to think about in these realms, in terms of ways to make your code easier.
But again, I want to go back to the core concept we talked about before.
You're going to have a couple projects during the semester where you're going to see a bunch of different coding styles.
Yours and other people's.
Yours might even change over the course of the semester, and that's good.
But you should include, perhaps as part of your post mortem process, what kinds of things did you like in the code you saw.
What can you pull into your coding style that is going to make your code easier to read, or easier to debug?
And maybe you can try to influence other people to write prettier code.
You're not always going to agree what pretty code is.
Try to let that go when it happens.
The important thing is you should think about what you're seeing, so that after your project is over, you are better programmer for it.
And I think at this point, unless you have any questions, we'll go on to have this project time with the remaining 15 minutes.
[CHATTER]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
Welcome to class.
The theme of today's lecture is all about testing.
So I'm going to give the intro to testing lecture where I talk about testing and how, what it is, and why we do it.
Give you some basic testing tips, and let you know what we expect you to be doing in terms of testing for your projects.
And I'll spend a little more time on focus testing, because that's the part people are frequently least familiar with.
We have Genevieve Conley from Riot, and she's going to give a lecture related to testing.
But more about what Riot's doing and stuff like that.
So hopefully that will be fun.
We'll take a break.
And then since we just told you all about testing and how important it is, we will go ahead and run a focus test in class.
We'll give you some time to get things set up, probably about 10 minutes to plan your test.
Then we'll set things up and we will run testing.
We're going to ask that everyone on your team test at least two games and that everyone on your team observe at least one person playing a game.
How exactly you're going to make all of that happen in flow is your problem.
And then we will come down.
We'll give you some time to let your team compare observations and talk about results, and then we'll ask you to come down for a standard in class, two minute no visuals presentation.
Monday, project two is due along with your design change log and focus test reports and your individual write-ups, and the project two presentation.
This is a five minute presentation.
There are presentation guidelines on Stellar.
You can download and look at them.
Once again, we're primarily looking for postmortem and reflection.
We want to hear about your process.
We want to know what you learned.
We want to know what you did right, what you did wrong, and what you're going to do in the future.
And we want it in five minutes.
Grades for project one are up, along with comments about your individual postmortems and some comments about the overall project stuff.
Overall, we were really impressed with the quality of project one.
So keep it up.
But do you stop and read the comments so that you guys can make sure to do good jobs on your individual postmortems going forward.
Anything to add?
All right.
So people say testing.
People say quality assurance.
What are they?
Quality assurance is actually just a whole process.
It involves a whole lot of different types of testing.
But in general, it's the systematic process checking to see whether a product of service being developed is meeting its specified requirements.
The most important part of that definition really is systemic, which means you're doing regular, repeatable testing.
Looking at making sure that all the portions of your game or project are working.
And requirements, making sure you're meeting that it doesn't just "work" as far as you think it does.
It works according the requirements of the game, of the project, of whoever you're shipping to and et cetera and so forth.
So you need to know what your standards are to be doing good quality assurance.
Without testing, your project will fail, and will take more time, and will take a lot more effort.
You can make games without programmers.
There's a lot of graphical user interface engines out there.
Some of them are getting older and older, but they're out there and you can use them.
GameMaker, for example.
You can make games without artists.
I bet several of you are figuring out how to do that right now.
But you can also make games completely without assets using text engines like Inform 7.
You don't really need designated producer or distinct game designer on a small team.
One person can really do it alone.
But you're never going to make a good game without testing, and the more systemic and planned your testing is, the more results you'll get from less effort.
So who gets to do all this testing?
That's really everyone involved in the project.
Do not assume that because you are the key programmer or the key designer or the key whatever that someone else will check your code and make sure it's working after you've integrated it.
You really want everyone on the team to consider it to be their responsibility to keep the game working as well as possible all the way through, and also to take the time to go in and check other people's stuff to make sure that it went in.
Remember that this is one of the cornerstones of using Scrum as well, where before a task is actually done, someone else has gone in and verified it.
Well, someone else is going to be someone on the team, which means you're all going to have allot some time to doing it, and you're going to have to think about that.
Speaking of which, anybody have testing as one of your tasks on your sprint task list?
Anybody?
Woo, you win.
It's going to take some time.
And it's going to take some time that probably otherwise could be used for making code.
So go ahead and remember that now and think about that as you're going into your last week of work, that you need to allot some time for testing.
So types of testing.
What is the terminology I'm going to use so that we all understand what I'm saying when I say it?
I divide testing up into three major areas, and there isn't a lot of formal methodology on this, especially not in the game industry.
So I tend to talk about technical testing, play testing, and user testing.
When I say technical testing, I mean getting your game to run.
Specifically, I often mean getting your game to run exactly as how you intended it to run.
So if the designer claims that the peach mobile should go up to 30 miles per hour and fire eight peaches per second, if you go into the game and you discover that it goes 70 miles per hour and fires two peaches per second, that's not working even if it makes game awfully fun with it like that.
When you start thinking about whether it's fun or not as a developer, then you're moving into the area of play testing which is where you, the team, are playing your game trying to figure out, hey, all those requirements and constraints and plans we made, are they working?
And are they working well, and is a player going to enjoy it?
You're play testing your game as a developer.
You're checking to make sure it works.
In that case, yes.
The fact that the peach mobile is currently going 30 miles per hour and it's so slow that no player is ever going to want to use it.
That's a bug.
That's a really big bug.
But it's a play testing bug if your specs say it should be going 30 miles per hour, not a technical testing bug.
And it is good to know the differences so that you can know what you're talking about and why you're talking about it.
Finally, there's user testing, which is getting someone who is not on your team to play your game.
You, the developer, while you do play testing, can pretend to be an actual player.
But you are not an actual player.
You will never be an actual player for your game.
You know it too well.
You are too close to it.
You have spent too many hours playing it.
So you really need to get someone who has never seen your game before, ideally, to come in and sit down in front of your game and play it.
We're going to make pretend when we run our focus testing workshop that your classmates here are actually good testers.
Tell you a secret, they're not.
They've seen your game because they've play tested it before.
They've heard you talk about it.
They heard you pitch it.
They might even have worked on it if they switched teams.
So they're not actually your ideal testers, but they are very convenient testers.
So we're going to use them today.
And for your next projects, we're going to encourage you-- and, in fact, require you-- to go out into the cold, cold world and find actual testers who haven't seen your game.
People in your dorms.
People in your other classes.
Friends, people at the coffee shop.
Whoever you think will do the best job of giving you the information you need.
User testing is another one of those terminologies that means a whole lot.
So for my convenience, I tend to divide it up into two kinds of things, focus testing and "user testing."
Where "focus testing" is specifically when you're sitting down and you're playing the game, looking at how it plays.
Is it fun?
Are people enjoying it?
Is it challenging enough?
Are the mechanics interesting?
Is your game a good game, and getting that information back from your players.
When user testing, I tend to use more for looking at the usability of your game.
You may have a really, really fun game, but if it takes your players 12 minutes to get off your starting page because they can't figure out what they're doing or how to do it, that's a pretty serious usability issue.
And you need to figure that out as well.
So just so you know that those are those two user types.
It's all user testing, but having some granularity to find the difference is a little useful.
So the basics for technical testing, which hopefully we can go through pretty quickly.
I suspect that if you guys have done programming, you've done this before.
If you've had classes that wanted code that worked, hopefully they've helped teach you what the good things to do are.
A lot of this sounds an awful lot like good software practices lecture that Andrew gave, because a lot of good testing is, in fact, a good software practice.
So have some technical standards.
Have a bug database.
Or, given the size of these projects, at least a bug Excel spreadsheet with a list of everything that you've noticed.
Build and test before checking it in.
Have daily builds.
Have a daily play-through.
Planned feature cuts.
Scope.
When you realize that you're not getting everything done on Friday night as you're working on your game, start cutting then.
Don't wait until Sunday night to decide what you're going to cut.
Cut as soon as you realize that you're blowing your estimates.
That will allow you to choose what you cut and to adjust changes in your game to reflect the things you cut.
It will also mean that testing your game to make sure-- it will introduce a lot fewer bugs when you do it in a controlled manner than when you do it in desperation, which is why I put it under testing.
Finally, code freezes and asset freezes.
These are great, and no one likes to use them because it means you're going to waste an entire day not working on your game.
Who wants to do that?
On the other hand, has anybody here turned in a project that they've made a last minute change to Sunday night, it was due Monday morning, and then it didn't run at all?
I got some honest people out there.
Good.
The advantage of taking that whole day to test your game is you turn something in that works on Monday.
Often, you turn in something that works a lot better on Monday because you've taken the time to test it.
You've found the bugs.
You've had a friend help you review your code changes, and you really got it working.
If you want to make a good, polished, solid game, you will think about it and give yourself a cushion for project two.
That's probably maybe a day.
It's a two week project.
If you give yourself much more cushion than that, you're not actually working on it.
But actively give yourself that deadline and give yourself some time to play your game.
So that's the basics.
Some advanced tips.
It's all pretty simple.
I think you've all heard this all.
How we maintain your feature list.
Agree on how features are going to work, so talk with each other.
Test those features as soon as they go in.
This may not be worth it on these games, because they're small.
When you hit your project four game, you may want to have one of these.
Think about having a standardized test plan that runs through all the buttons and the branches and the pages and the options of your game.
I realize that when you've got randomness as one of your core design goals, you can't predict how your game is going to play.
But you should be able to take a look at your game and know most of the states that it's going to get in.
And you can create a checklist that says, hey, have we exhausted all the possibilities that a player will encounter?
Have we tested them all in the last two builds, yes or no?
And the answer should probably there be yes because one of the things that's very common with projects that have a lot of highly integrated systems working together and depend on emergent behavior-- which is most games-- is you can introduce a feature over here that changes the way that a feature that you implemented two weeks ago works.
And you'll never know that it did that until you go through and you test and you find that something that was working is now broken.
So it is worth it to take the time to go back and check those features you know nobody touched.
Bug reporting.
Because so many people are so bad at this, I'm going to take the time with a captured audience to tell you how to do it.
Be kind to your fellow engineers.
Be kind to yourself, if you end up getting your own bugs.
Write good defect reports and not bad ones.
Good bugs include what happened, what should have happened.
Because it doesn't do me a lot of good if you tell me that the peach mobile is going 30 miles per hour if I don't know why that's a problem.
How do I reproduce it?
If it's obvious to reproduce, you don't need to include it.
But if it took 38 steps, include a text file.
How common is the bug?
How serious is the bug?
Is this something that the team needs to jump on right now, or is this something that Sunday afternoon you're going to look at and say, eh, ship it.
Because some bugs you look at and you say, eh, ship it.
But you want to know that it's there, and you want to know how it's going to affect your game before you just decide to ship it.
Include some supporting data if it's going to be helpful, and it often is.
If you're trying to describe something visually, give them a screenshot.
If you can run it in the debugger and catch some debugging code, great.
Save the files.
Know the version of the game you were using.
Be ready to help your team member who has to fix your horrible awful crash that happens one out of every four times, and you're not sure how you made it.
Be willing to help them recreate it, because it can be really hard to recreate bugs.
Finally, know the difference between a criticism and a critique, and make sure that your reports are critiques of the game and the bugs and not criticisms of your fellow developers and their ideas.
And when I say criticism, what I mean here is an unsupported opinion usually delivered in an unprofessional tone.
These purple elephants are junk and it's totally stupid that they fly.
A critique is a statement of opinion about the game backed up by examples and hopefully facts.
The purple elephants make the game too easy because they can fly.
I can skip almost a third of the game play in the bottom area of the screen.
There's a problem.
There's a reason why it's a problem.
And there's a fact that you can talk about, and you can have opinions.
And it doesn't matter that the purple elephants go in the game or not.
You can just talk about what they're doing in the game.
You don't have to talk about the person who put the purple elephants in the game at all.
So user testing.
I think we've actually covered this one to death.
But if I did a lecture about testing and didn't stop and talk about why you're doing it, it would be a little odd.
Once again, every time we say iterative design, that means you make a design, you test it, you get the data, you move on without actually testing it with users.
You don't get real data, et cetera and so forth.
So I think that drum has been well and truly beaten.
But here's a warning we haven't really talked about yet, which is test-driven design.
Does not mean do what your testers tell you to do.
Test-driven design means gather data from your testers, analyze it, and act on it.
90% of all MIT students will tell you that your game is too easy.
They're wrong.
90% of all people who like to play first person shooters will tell you that your game needs a gun.
If you didn't plan on having a gun in your game, it probably really doesn't need a gun.
Testers are often full of solutions and suggestions to make your game just like the game they wish they were playing.
But you're not trying to make that game.
You're trying to make the game you're trying to make.
So when you gather your data, put it through the filter of what your vision of the game is and what you intend your game to do, and don't let your testers drive your game.
Let your vision and your data drive it.
So how.
How do you get ideas from testers?
And how do you actually turn them into something useful?
So you all remember the experimental method.
It's actually really pretty similar.
Going to form a hypothesis, or ask a question.
Figure out how to get the data to that question, collect it, and analyze it.
So forming a question.
What do you need to know about your game that your team can't answer?
The more specific you can be, the easier it is.
Sometimes it is hard to get specific because you've got a game and you've got a whole lot of interlinking systems, and it's hard to figure out which one is making things not quite right.
But do your best.
In there a particular mechanic?
Are people having a hard time or an easy time getting started on your game?
How engaged are people in your game play?
Are they playing the game and grinning, or are they just sitting there and hitting the keys?
How many of your players complete your game in about the time you expected them to?
How many of them complete too quickly or take too long?
So when you're running scientific experiments, you try to standardize your conditions as much as possible to get data that can be compared with each other.
When you're collecting qualitative data, which is what most focus testing data is, this is actually pretty hard to do but you want to try to do it anyway.
So some of the ways you can do that is by make sure you're giving each player the same set of starting information by having a script to introduce your game, to give them any information about the game they're going to need because your game is in too early to have a UI.
Games often don't have UIs that are particularly useful while they want to be tested early.
That's OK, just tell your players what your UI "should" tell them.
You can double this up with user testing by giving them pictures of the intended UI, and you can see if they can figure it out from there or not.
But that's advanced testing tricks.
Have a dedicated observer.
Don't depend on their memory for what they're looking for or what they see.
Give them a list of specific things to look for that you think will answer the questions you need to answer.
Give them space to write it down with.
And give them the autonomy to stand back and just take notes while someone else offers your player assistance or introduces them to the game.
And finally, you really can't observe everything.
Think about what your players might be able to tell you that you can't see.
Write down a few questions for them to answer afterwards if you need to.
Keep it short.
You don't really want to make somebody write an essay while they do your testing.
If you can encourage your players to talk to the game or to talk out loud while they play, that's the greatest thing ever.
The best people I've ever heard play games are middle schoolers because they connect their brain straight to their mouth without any filter, and they're just there.
They're telling you what they're doing.
They're telling you why they're doing it.
They're telling you why this particular aspect of your game is the worst thing ever, or why this particular aspect of your game is the greatest thing ever.
And you can just sit there and listen to them and you know why they're doing things.
Adults are a lot quieter.
We often don't want to explain why we're doing something.
Often it's because we don't know we're supposed to be doing.
And we don't want to admit that I don't know what I'm doing.
I'm just going to poke stuff until I figure out what's going on.
So that's where you want to watch people.
And if you really need to, you can lead and say, why did you do that thing?
Or even better, what are you thinking?
People will often answer those questions.
But you really want to keep those prodding questions neutral.
Don't say, why did you do that crazy thing?
Or, what are you looking for back there?
But just, what are you thinking?
What are you doing?
What's your plan?
That sort of thing.
Once you've got your data the team can review it together to notice trends, spot common problems, and so on.
And that's where the things that four or five or a majority of your testers did or said or ran into, that's what you're going to be pulling out and using to think about how you're going to fix those problems.
Or if you noticed a trend that you hadn't noticed before that people were really liking an aspect of your game you haven't thought about, it's something you can work on magnifying so that you can share it with everyone.
So for those of you who fell asleep while I was talking, the too long didn't read version really is who are you testing with, because your audience will affect your data.
What are you testing with?
The state of your game will affect your data.
That shouldn't stop you from testing with early builds, but it does mean you need to adjust your expectations and make sure your players are ready to play with it.
And finally, why are you testing?
Only you can answer that question.
We can give you a lot of reasons or suggestions, but if you're running a focus test it really needs to be because you're looking for data that your team intends to use.
And you can always use data to support your game.
You really can.
Even if you don't realize that you need it, you do.
And I keep saying "and gather data."
And I talked about observations.
I talked about surveys.
I talked about interviews, which are our three major tools unless you are going to go to all the work of videotaping and audio recording and getting releases from your testers, which I really don't recommend.
That's more work than you should be doing.
But the three major tools really are observing people and taking notes, asking them a standard set of questions afterwards-- either on paper or in person-- and thinking about what kind of data you can gather best with each of those methods.
If you have 20 different levels you want someone to rate at a one to five as to whether it's good or not, you should just give them piece of paper and when they finish each level, they can check a number between one and five.
If you want to see how long it takes someone to get started playing the game, you probably just want to watch them with a timer.
How long does it take them to get from that first screen to getting into the game?
And finally, if you want to get a general impression, sometimes talking to people helps bring out things that you didn't think of.
You can get some insights you wouldn't have thought to ask for by talking with people.
But be a little careful talking with people, because people don't want to tell you bad things to your face.
They'd rather write the bad things down on a piece of paper where they don't have to admit that they said bad things about this game.
That covers less for middle school students, as I mentioned earlier.
But most people don't really want to say mean things to the person who made the game, because we don't want to make people feel bad.
So you need to balance that out when you're asking people questions about well, how did you like the game?
The answer is usually, oh, it's great.
Yeah.
So having run through all of that, let me fight with my computer and bring up a couple of examples.
That's not the right form yet.
So this is a script created by students by a team that had a dedicated QA lead person.
I do not expect your sheets to look anything like this, but I want you to see an example of what someone did when it was their job and they were taking time to get things ready.
So you can see how serious we are.
And so you've got your observer instructions.
And what's day's goal?
Finding out if the feedback in the game works, along with a couple of other things.
And there's a testing script written out pretty clearly along with some basic instructions.
Since these teams had a dedicated focus test lead, they had someone who'd spent a lot of time thinking about testing and a lot of people who'd spend a lot of time working on the game.
So the lead tester was in the position of trying to make sure that the team understood how to run tests, because they weren't experts at it either.
So I just embedded all of the good advice right into the script.
And there you have it.
And then there's a sheet to write notes down.
They've got the big questions that they're worried about, so they can write down.
They've got space for other notes.
And it means that whatever they've got, they've got.
And then finally, it follows up with a fairly short questionnaire trying to get some answers to questions they're wondering about.
The specifics are always going to vary from game to game.
Looking at the focus test sheet from one game isn't going to help you wildly with yours.
But you can think about the way the questions are worded and the number of questions so that you're not overloading people.
I believe these are up on Stellar as a sample if you want to download it and look at it as an example.
Finally, the paper we expect you to do.
So this is the focus test report which is also download-able on Stellar, I think in either Word or Rich Text Format so you can use it.
When we say we expect you to turn in a focus test report, this is what we're expecting you to turn in.
If you take a look at it, it walks you through the whole process and there's a space for most of this stuff.
So what's the goal of the focus test?
Record what happened at the focus test and what are you going to do afterwards.
When you are planning your focus test for this afternoon later in class, I recommend you download this and use it as your sheet to record notes on, because it'll help make sure that you get the data that we are asking you to get.
And I think that's actually most of what I have to say about testing.
Are there any questions?
I have a question.
What's currently using some bug database, and if so, what are folks currently using?
Or let me rephrase that.
Who has used a bug database before?
What are some of the ones that you're familiar with?
Chehra
Chehra?
Bugzilla
Which one?
Bugzilla
Oh, Bugzilla.
Yeah
Buganizer
Which one is it?
Buganizer
Buganizer?
Yeah
Oh
I use [INAUDIBLE]
Yeah
[INAUDIBLE] That's the [INAUDIBLE] database, yeah
And as Sara mentioned, you can also just use a Google spreadsheet or something
[INAUDIBLE] is a great bug database too
There is always this intermediate step between someone entering a bug into a bug database and then deciding that they're going to fix it, right?
So priority, somebody actually says I think the bug that the characters are too short or something, and then the art director says no, not a bug.
Will not fix that.
So just because you're entering something in a bug database doesn't necessarily mean that your team's committed to actually spending time on that.
But when a team does decide that, OK, this is a bug.
Maybe because it crashes the game.
Maybe because it's actually not how we want the game to perform.
It needs to be entered into your task list.
So that means your task list is changing and you need to be able to put that in a priority.
Possibly, this bug is more serious than some of the planned tasks that you already have on there.
So you need to rearrange that so that [INAUDIBLE].
Something else that Sara mentioned very briefly, but I want to stress, MIT students are actually horrible testers.
I've heard this many times from many different companies here and [INAUDIBLE].
MIT students are not a representative population of the general public.
You look at systems differently.
You look at screens differently.
So even though most of your testing that we're going to be doing this semester, I imagine that a lot of testing you're going to do on your own is going to be with other people in your dorms, for instance, and that's fine for this class.
But don't think that it's normal to use even university students as your only test audience.
That's really not representative of the general public.
MIT students in particular
MIT students do [INAUDIBLE]
The final thing, as Sara mentioned, user testing in two contexts.
One for overall category of you are testing with users and then it's broken down to more specifically, focus testing and user testing.
And you may be wondering why user testing is used and that [INAUDIBLE] two different things, as Sara said, it's good to be more granular.
What we normally refer to as usability testing, as Sara mentioned, has only fairly recently become the standard name for it
Yes.
Yeah
Back in the '90s and even the early 2000s, user testing and usability testing were basically the same thing, because we were talking about websites and enterprise software.
We were not talking about games.
So the only kind of user testing that you'd be doing, putting a piece of software in front of someone, getting feedback, was usability testing.
That's why they had the same name for so long.
But now there's focus testing, which is very, very applicable not just for games but also for things like user experience.
I had not heard that term user experience [INAUDIBLE].
So where it's not just can the player figure out what to do, but the person actually having the desired experience for playing the game.
That's why now there's multiple names for that
Thank you.
Then I think I'm out of here.
I'm going to sneak off with my computer
Hi, guys.
My name is Genevieve Conley.
I'm a researcher at Riot.
And some other terms that you might have heard for that, user experience researcher, user researcher, or the way we were just talking about it, like user tester basically.
And a little bit about me.
I graduated from MIT in 2010.
I got my degree in Course 9.
And I actually worked at the MIT Gambit game lab on a couple games there.
So that's actually where I got my start doing user testing, was in the summer projects we would run play tests with local community members, have them come in, and that's where I actually did my first user tests.
So I work at Riot Games.
How many of you guys have heard of League of Legends before?
OK. How many of you actively play League of Legends?
OK, good.
So I'll give a little bit of context on the game just so I'm not losing anybody.
And I'll try to make sure I explain all of our jargon, because sometimes when you're in a game company you just start using the language like everybody speaks that language.
But a little bit about Riot is we make League of Legends.
And our core mission is that we aspire to be the most player-focused game company in the world.
And this is going to be relevant to what I'm talking about today, because you can't really be player-focused if you don't talk to your players.
So what this means at Riot is we think about all of our decisions in terms of how much player value it adds, or how much player pain something negative causes.
So this is like-- I don't know if any of you familiar-- we have an eSports scene, and right now we're in the middle of our world championships.
And we created a video and a song that would basically, we hoped to be the anthem of the world championship.
And this is something that a couple people at the studio thought was really important, so they just went out and made it.
And the point was just to make the players pumped up about worlds and to give them a song that represented how they feel.
And this is just a small example of something that happens at Riot where we really just want to do something that we think players will think is cool.
And one of the ways if we do that is through play testing and user testing.
And I'm going to use a little bit different terminology, so I'll make sure that I tie it back into the way that Sara's been talking about it as we go.
So a little about League of Legends.
It's generally a 5v5 game.
It's a team oriented game.
So you play as a member of a team.
And you usually play a specific role or position on that team, just like basketball.
And you play as one of many champions.
And it's online and it's competitive.
So like I said, we have an eSports scene where you play against another team.
And you go basically across this map, try to take objectives, and ultimately blow up this thing called the Nexus which is a giant crystal on the other side of the map.
And you play as a champion.
And these are player characters that each have an individual set of skills, different play styles, and there's over 120 of them in our game.
So even though each game is session based-- you don't level up your character necessarily over the lifetime of playing the game, each session is a new session-- there's lots of different ways to play because we have lots of different characters, which I'll talk to in a little bit.
So one of things that I talked about is it's really hard to be player-focused if you don't talk to your players.
And one of things at Riot is that we all play League of Legends.
In fact, some of us play it a lot.
Some of us play a lot at work.
But just because you play your own game doesn't mean that you actually know what it's like for all of your players.
And using MIT students is a great example.
If you guys just play with MIT students, you might not know what it's like for people in other countries, for example.
So one of things we try to do is make sure that we understand our players broadly and know that we are our players, but we are a small sub-segment of our players.
The way we use research is it equips us to understand player beliefs, their needs, their desires, and their relationship with the game and with other people in the game.
And what this allows us to do is design experiences.
So there's specific experiences, interactions that we're hoping to design.
So in general when we're creating a feature, we have a set of goals that we're trying to achieve.
And what we want to do is we want to help designers.
developers, QA, artists, we want to help them achieve these goals.
So we want to know exactly what they're trying to do and then get them feedback on how well those goals are being communicated to players, how well they're actually able to interact with those, and then what their reception is as a result.
So the way I'm going to break this down is play tests and user labs, which is essentially the way we were talking about it before, is play tests and user tests.
So play test is the backbone of creating your game.
It's really, really difficult to create a game without play testing it.
So I'm glad that's really part of how you guys are designing your games.
And play tests basically allow us to quickly gather information about a game or a feature to see how it's working.
And typically, this is done with resources that are readily available.
So people in your dorms, people in your class, or at Riot, we often do it with other Rioters, and even on the team.
And some of the benefits we can get out of this is we can identify technical issues.
Now, Sara talked about a specific type of this, which is quality assurance testing, which is less what I'm talking about.
I'm more talking about testing with your team to just quickly identify technical issues that might prevent you from playing that game.
You can also look at things like balance.
So in a competitive game like League of Legends, you play against other players.
And if one type of character is really overpowered or has a significant advantage over another type of character, we can quickly identify that through play testing.
So for instance, our designers have two play tests a day where they test their in-progress champions to see if there's anything basically game breaking like that that they can quickly identify and iterate on.
We can also look at things for fun.
I mean, this is the most obvious one.
But it's really important when you're making a game, if the game is supposed to be fun, that it is actually fun.
And in our case, we do want to have a fun game.
So we have a whole group of designers and developers that have a team which is known as the play team and they make temporary game modes.
So I don't know if anybody played recently, but we had a big in-game event called Shurima, and we had this game mode called Ascension.
And when they were developing that, they were often several times a day-- many, many times a day, in some cases-- testing every iteration that they made to that game mode to make sure that it was actually fun.
And a lot of times what you'll find is that your game isn't necessarily fun sometimes.
And that's OK, because each time you make a change, you're getting closer to that fun experience.
One day, you'll be playing your game and like, oh my gosh, it's actually fun now.
This is great.
Let's do this.
So it's a really good way to get that feedback.
And another specific area that we're looking for a lot when we play test is clarity.
And clarity is a word that we use to mean basically how clear is the information-- either the visual information or the mechanical information-- how well is that being communicated to players?
So this could be something like with UI, if you just have a picture and you show it to players or to your dorm-mates, your classmates, how easily are they able to identify what a button means?
Or which things are clickable and not clickable.
Those things are all really, really important in a game like ours where we have five people on a team, 10 people on the map, each with their own set of abilities and visual effects.
And there could be a lot of visual clutter if we didn't think about this type of thing.
And then the way we typically get these, at least at Riot, is like I said, team tests.
That's the example I used with the designers and the play team.
They're just playing within their team frequently to make sure that they can quickly and easily give each other feedback.
Quality assurance test is another type that's a very specific implementation of this, which Sara spoke to.
And in house "guerrilla tests"-- and I always have to be careful how I say "guerrilla," not "gorilla."
"Guerrilla."
This is when we bring other Rioters who aren't part of a team who maybe haven't had exposure to that particular feature.
This is particularly useful for things like clarity or usability tests, because we don't necessarily need to have that perception or that sentiment that is really unique to players.
What we need here instead is just see, hey, how easily is this usable for a reasonably tech savvy person?
But even then, of course, there are some cultural differences.
There's some hardware differences.
So it's a first phase when you do them as guerrilla testing.
And you always want to make sure that you follow up with real players after the fact.
So user labs, this is what I really focus in.
So I use a lot of different methods at Riot, but one of the key ways that we're able to get player feedback is by bringing players to Riot to have them play the game.
So this is a way to help us understand how real players will actually approach a feature.
What we get out of this is really, really important.
We get reception and interest.
Like, how stoked are players going to be about something, or what we call the table flip reaction.
Which is like, we're working on an update to Summoner's Rift right now.
And it was early on.
It was really important for us to know, were players going to be really upset that we were changing something that's so key to the game.
So we made sure to loop them on very early in on the process to see how they were responding to changes we were going to make.
We also look for things like usage patterns and errors.
And this is important that we get a broad swath of different types of players to look at this type of thing, especially with things like we just updated the client.
Making sure that the changes we made to the interface weren't going to confuse players-- experienced players who were used to our old version and new players who would be coming into our game for the first time.
We also look at needs and desires.
And this is not like Maslov's-- Maslow's hierarchy of needs.
It's more like, what are things that players really want from our game?
And understanding that can be difficult sometimes because, like I said, we all play our game.
We're very entrenched in this experience.
We have our own experience in the game that we feel very strongly about.
But understanding that other players might have different needs and desires, and understanding what proportion that represents within our broader player base, these are really important things to understand.
How we get this is a lot of different ways that I'm actually going to go into in a little bit more detail.
And each one has a specific application, but there are a few that we use pretty much in every user lab, which I'm going to go through right now.
So the first is something we just talked about, which is just observation.
And I bring this up because it's something that anybody can do.
You don't need a dedicated researcher like myself.
You guys should and will be doing this.
And so in this case, I have Chris, who is a developer at Riot, sitting in for a player.
And I have [?
Jisan, ?]
who is another researcher at Riot, sitting over his shoulder watching him play.
In our case, we typically just have one researcher in the room both taking the notes and doing the observation, just because of resource limitations.
And because we've done this a lot and it's our full time job.
So in this case, [?
Jisan ?]
is watching Chris interact with the client.
And you can see in his hand, he's actually got the script-- or what we call protocol-- in his hands so he can remember what things he should be looking for, what things he wants to say.
In this case, we don't generally try to use it as a script.
We try to use it as a memory refresher.
That way, players can feel really comfortable with us and feel like they have a relationship and can have a comfortable conversation.
One of the ways we achieve this is through what we call the Think Aloud protocol, which is what Sara was referencing, which is basically we try to make sure that they have this direct link from their mind to their mouth.
And there's a couple ways to achieve this that I'll speak to, but the goal of this is we want to know exactly what the game is communicating to players, and then how players are processing that and creating an internal model.
And the best way to do that is just to step back and let them tell you exactly what they're seeing.
And if you aren't quite sure what they're seeing, then you can follow up, which I'll get to in a second.
So here's an example of a player who's seeing our character Lucian, who is a champion who was released fairly recently in the game.
She's seeing him before he had been released in a work in progress state and giving us feedback on what she sees.
[VIDEO PLAYBACK] -I can see that he walks faster than the normal-- [END PLAYBACK]
Sorry, we forgot to plug in the audio.
I think you guys could hear it all right.
All right.
[VIDEO PLAYBACK] -[INAUDIBLE]
Oops.
-I can see that he walks faster than the normal character would, but his animation doesn't make it seem so.
It seems like he takes a really big stride, but I was imagining him to walk really fast.
But it seems like it looks like that way.
[END PLAYBACK]
So this is an example where the player has just brought this up to us that she expected him to move one way, and he's actually operating in a different way.
You can think of this is as like, basically a violation of expectations, which is often what we're looking for especially with our characters.
Does what you see and what you hear about a champion match with how they actually play?
So one way you can do this is to just let them talk.
And it is really important not to be giving feedback on whether or not you think they're correct, especially if they do have a question.
It's also really important not to be judgmental, right?
So the way you phrase your questions, even if you aren't actually being judgmental, sometimes players might think that you're judging them.
So one of the contexts we give players upfront is always for us, since we actually didn't work on the game or the feature that they're seeing, we can say, we didn't work on this.
It's really important that you give us honest feedback.
I'm not going to go home and cry at night, so I want you to give me totally honest feedback.
And we also let them know that sometimes we're going to ask them questions, and we're asking it so that we can understand what they're thinking because that's what we're interested in understanding.
We're not asking it because there is a right or wrong answer.
So one of the ways that makes it easier to make sure that we understand what their mental model of the game is is a follow-up with some probing questions.
So the player in the previous example was pretty clear about what she was seeing and why it violated her expectations, but sometimes players don't necessarily know why it doesn't actually fit what they expected.
Or they might know that something's frustrating, but they're not necessarily giving you the answer why.
So one of the ways you can follow it up is either with probing questions, which you ask in the moment, or you can follow up with interviews after the fact.
So here's an example of [?
Jisan ?]
following up with a player who just saw Lucian for the first time.
[VIDEO PLAYBACK] -How was that play [INAUDIBLE]?
-I thought it was fun.
I thought it was [?
different.
?]
-What makes it fun and something different about it?
-So he, I guess, normal [INAUDIBLE] skills, just increased the intensity.
A lot of his skills based off of move speed and [INAUDIBLE] speed, and I guess he feels [?
confident.
?]
[END PLAYBACK]
So this is an example.
A lot of times you'll find this, is that when you ask that initial question, you get a short response.
And what we're always trying to do as researchers is invite a conversation where they're just really comfortable telling us exactly what they feel and giving us a lot of detail.
So in the first question, he asked a really broad opening question, which was "how was that?"
He didn't even ask, "how did that feel," which might lead in a certain direction-- although that would be an appropriate question as well.
He left it wide open for the player to take us where they wanted to talk about it.
So the player mentioned that it was fun and something different.
So [?
Jisan ?]
wanted to get more depth into what does that actually mean, because that alone doesn't give us enough direction to be able to provide to our designers to say what was good, what was working.
So he mirrored exactly what the player said back at him.
He said, oh, what was fun and what was the thing that made it feel different?
And then the player, that way we're not actually putting our own spin on interpreting what we think they're saying is fun or maybe what's different.
We're just mirroring exactly back what they said so that they can give us more detail.
You have to be careful with that, because you also don't want to sound cold and clinical like you're sitting in an office somewhere writing down everything they say, which you will be.
You want to make it feel like it's a natural conversation.
So we're really lucky because we get to work with League of Legends players so we already have a connection built up with them where we can talk about it, but it's really important that you make that as hey, I want to know what you think.
You're really valuable.
Let's just have a conversation about it.
And it can be really challenging sometimes.
So this is one that you guys are already pretty familiar with, too, is part of interviews that I've clumped in here is basically the questionnaire.
Sara gave you guys some great context about when this might be useful.
Here's an example of when we use it in League of Legends research.
So this is an example with Azir, who is a character who just came out in League of Legends.
This is actually from quite a long time ago when he was in the concept phase.
And this is something we do actually with most of our concepts, is we put them in front of players before they go into development to see, hey, is this something that feels like it fits in the world of League of Legends?
Is it clear where maybe the source of power is?
And does this look like a champion you would actually want to play?
Now, we don't use data to drive decisions at Riot.
I want to make that really clear.
We use it to inform.
So just because something wasn't resonating with players in our labs wouldn't mean that we would cancel a character.
It's just a way for us to get feedback for our designers who already have their experience and the experience testing with other Rioters to help them make informed decisions.
So what's useful about a questionnaire in this case is that we can compare across champions over time and across different roles or positions and see what is resonating and what does good look like.
We also use this as a jumping off point.
So we start with a questionnaire, and then we go into a more detailed interview about it to make sure that we're getting enough detail to be able to provide useful information to the designers.
So you guys are also probably pretty familiar with the idea of experiments in general.
But part of research testing can be devising experiments.
So we often hire people from psychology backgrounds or research backgrounds in academia because they have experience setting up experiments.
So this is like the classic one.
I don't know if you guys have already covered this, the Engelbart experiment.
OK.
So this is foundational human computer interaction.
It's one of the most cited studies.
And it's English, Engerlbart, and Berman basically were looking for a way to improve a way to interact with a screen or text manipulation.
And what they proposed was something that was radical and was called the "mouse."
Maybe you've heard of it.
So that thing in the right, I know it's not a really clear photo, but that is one of the first early prototypes of a mouse.
And what they wanted to do is they want to see, is this better than some of these other interactions out there.
Qualitative experience alone might not be a very strong way to prove it in this case, especially when you can collect quantitative data to back it up.
So what they did is they used the time that it took to complete a task, a very specific task that was given across all these different ways of manipulating text.
And they saw how many errors the user made in this case.
So you can see that the mouse did pretty well in terms of time overall.
It didn't take the participants very long to be able to complete the task.
But it had very few errors, which made it overall the best out of these different ways to select text.
And you can see some of the other ones on here were things like knee control, joystick, a graphicon, which is like an arc drawer, a light pen, and a joystick.
And all of these were ways that we could have gone instead of the mouse.
But based on some of this research, they were able to improve their design and now you can see I'm using one right now.
And as part of the experimental approach, I broke it out, but there's a specific way you can look at it, which is using physiological techniques.
And there are many, and I'm just going to cover two.
And typically, these are used for very specific types of research.
They're very exciting.
So oftentimes, you'll see people want to use them on everything.
But they're very particularly useful for certain cases.
So this is an example of eye tracking, which is basically fundamentally, we put a camera up that's able to specially track where eye movements are.
And there are more sophisticated ways you can look at this-- and I'm sure a bunch of researchers at MIT could tell you a lot more about it.
But in this case, it was pretty simple.
We were just looking at where people were looking on the screen and how often and how long they were looking there.
So you can see the red parts of the map here are the parts where there are more intense gazes or more gazes overall.
So unsurprisingly to probably most of the League of Legends players in here, or you guys as gamers in general, the mini map has a lot of gaze.
The bottom part, which is where the abilities are displayed, also has a lot of gaze.
That has a lot of information that's really relevant to players like the cool down timer, basically when the ability is going to be ready again.
And the center of the screen is where most of the action happens.
So this type of information is helpful for us.
If we want to, for instance, introduce new information on the screen, we'd want to know, hey, are we going to introduce this in a place that's going to get a lot of traffic naturally.
But also is it going to interrupt things that that gaze is there for a very important reason, and maybe we shouldn't interrupt that.
So we can help us to inform those types of decisions.
There's also many other types of physiological, and I'll just hint at one right now which is things like galvanic skin response, anything that has to do with physiological arousal, either for stress or excitement.
These can help with situations where maybe players wouldn't be able to vocalize why something was frustrating or exciting.
Or it can also help combat some biases like recency effect, where if you ask a player at the end of the lab or at the end of a play test what was interesting or what was frustrating, they might preference things that were at the end.
Or they might preference things that were really emotionally significant for them, but might forget smaller things that are still maybe small frustrating things that build up over time.
So none of this is really that interesting if I don't give you an example of how we actually do at Riot.
So here's a case study.
And I actually presented this last year as part of a talk that we did at MIT.
So I apologize if a little bit is re-run for you guys.
I'll try to spice it up a little bit as I go.
So I'm going to be talking about Lucian, who I introduced to you guys earlier.
He is one of the player characters or champions in League of Legends.
So where we always start as researchers, and part of the job that I haven't really alluded to yet is we spend a lot of time talking to our designers, to our developers, to our artists, to our producers, to make sure we understand the goals they're trying to achieve.
And I don't know how many of you guys got to catch August Browning's talk last night, but for the champion designers, they start with a very specific idea of what goals they're trying to achieve.
What is this player character going to feel like?
What are you going to feel like when you play it?
What is the mechanics supposed to feel like?
What is the theme supposed to feel like?
How does this resonate with players?
How does it fit in League of Legends?
These are all really important questions for them that they're constantly asking themselves.
And in the case of Lucian, he was supposed to feel fast, sleek, speedy.
He was like a gun ninja.
Like pew, pew.
No?
All right.
Trying to see if you guys were awake there.
And he's a range damage character with finesse game play.
And he basically has the high skill ceiling.
And what that means is basically he's harder to pick up, but when you master him it's a really rewarding feeling.
So that's a trade off between accessibility, maybe how quickly a player can pick him up, or what level a player can pick him up, for a reward of how rewarding that feels when you do master that champion.
And some of the challenges in this were, this guy, his theme is pretty different.
His equilibrium style, if you've ever seen that movie.
And that's not something we had necessarily done that much in League of Legends before, so we wanted to make sure, does he even fit.
And we also wanted to know how do you let a player feel super fast and speedy in what is essentially an RTS style game.
It's a challenge.
So this is what the test plan eventually looked like.
It's not necessarily where we started from.
But we started with concept testing, and then we went through actually three lab studies with iteration in between based on the feedback we got from players.
And after he launched, we did some post-launch evaluation.
So step one, just like I showed you earlier, was to do this concept testing.
And we got some pretty good feedback based on that.
His fantasy was good with players.
They liked the mood.
[THUD] They felt like he fit.
Are you all right?
Yeah
And he made sense for the role of an AD carry, which again is like a range damage dealer in this case.
So he had high appeal scores and the theme felt really strong, but one thing we very quickly identified-- it's a little hard to see on this one-- but his guns were different colors.
So players expected the guns to do different things based on their colors, which was not what we were planning.
So this was very good early feedback before we got very far into development where we could pivot and make sure that the idea of the two guns doing the same thing was conveyed.
So based on the feedback we got, we were able to give that to the designers and they continued their development.
And then after a little bit of time, we were able to take it into the lab.
So we started with a Lucian that was designed to be a speedy range assassin.
He had a double shot and a utility kit.
And when I refer to kit, just to be clear, that means his set of abilities.
And characters in League of Legends generally have three abilities and then an ultimate ability, where the ultimate is like the super big bad power that you get after some leveling of the character.
So I blew the reveal here, but in the first lab we found out that he didn't really feel that sleek and fast.
He felt stiff.
He planted and then fired his double shot, which made him feel like a turret.
And people did like his visual theme.
They thought it was compelling, but it just didn't match his play style.
So we took this back to the designers we said, hey guys.
Here's the videos.
Here's the notes.
We usually make a highlight reel based on what we see to support our evidence so that the designers can get enough feedback from that, as they are our subject matter experts and really know how to interpret what they see.
And we said, OK. Let's go back.
You guys take some time, do some changes.
We'll take it in the lab again.
So we did lab study two.
And this time, we had a different Lucian to bring in.
This time, he was more mobile.
He was basically a range [INAUDIBLE], running gun, double shot guy.
Well, we had a little bit of a mixed bag of results here.
He did feel a little bit faster.
But some poses and animations felt awkward.
And he was high speed in the numbers-- which I'll talk about in a second-- but he still felt slow for some reason.
But worst of all, we got quotes like this one about the ultimate, which was "It worked in the way I did not want it to work," which is a pretty good sign that something's wrong.
So we took all this back to designers again.
We said, hey guys-- and the animators and the artists in this case, of course-- and we said, he feels slow.
His ultimate doesn't really have the usability that people expect, but the double shot theme is working, so we're definitely on the right track and players are still very excited about this champion.
So now we're in a bind, and this is often the case when you're working with research from players because they're not always able to articulate exactly what's wrong because they don't have the context that you do.
So this is why it's so important for us as researchers to pair with our designers, our artists, our producers, and our players to be able to get the right kind of data to make informed decisions.
So here was our problem.
The frame delay between his basic attacks-- which is actually very, very low in this last lab.
Usually that means things feel fast.
It's a pretty standard way when we're developing characters to be like, OK, this will feel faster.
He also had a very high base movement speed, which means that the time it took him to run from point A to point B was pretty high relative to the rest of our champions.
And if that weren't enough, he actually had a dash ability which means that he could just jump basically from one point to another.
All of these things should have made him feel fast.
But players were saying he didn't feel fast, so there was something wrong.
So our designers and our artists and our producers took all of this information back, looked at the videos, play tested the heck out of it, and came to this conclusion.
They increased the ability responsiveness and they changed how animation flowed from one state to another.
So I'm not an artist, but what this means to me is that they are basically able to make the transitions from animation a, animation b, or ability a to ability b, and make those feel more natural.
Not like me.
And they also gave more feedback to the double shot.
And again, this is where subject matter expertise is so important because our animators, our visual artists, our designers knew this bag of tricks to be able to apply to it.
And finally, they were able to make the ultimate ability clearer.
So they were able to the telegraph better what it was actually supposed to do so that players would have more accurate expectations.
So we took it to lab study number three.
We put it in and finally, the game play anesthetics fit the concept, and players were quite happy.
But not everything is always happy ending, everything is perfect.
There was still some confusion about how the ultimate was supposed to work.
We found that players wanted more control of it.
Basically how the ultimate currently works is he locks his arms forward and shoots in that direction.
So if you stray left to right, he continues shooting in that direction which, for some players, was pretty confusing.
Now what they expected based on some of our other champions, and it wasn't able for them to learn it in one session.
So one of the things that we have to take away from that is labs have some external validity problems, which basically means that we're sitting in a lab.
There's not other players around.
They're often not playing on live.
There's a lot of things that are different in our test situations than real life.
Usually, you wouldn't try to master a champion in one session.
So this is one of those cases where we put it out to our designers and they decided, and we supported, that he was going to be a high skill ceiling champion, which meant that players were going to need some time to learn how to play him.
And that just because they didn't learn him in one session didn't mean that they wouldn't learn him over time.
So this is what Lucian looked like when he did go live.
And it helps if I play the video.
[VIDEO PLAYBACK] [WARFARE SOUNDS]
So he's in the top there.
Pew, pew.
[WARFARE SOUNDS]
So you can see his dash there.
[END PLAYBACK]
So yeah, sometimes a little confusing if you don't play the game to have to watch 10 champions all in one little team fight like that.
But ultimately, the important part's there.
He's using his ultimate and he dashed at the end.
Ultimately, ended up actually delivering on player expectations for this champion.
[VIDEO PLAYBACK] [END PLAYBACK] We don't need to watch that again.
So what did we learn?
Well, we knew going into it that Lucian was going to be a difficult champion to get right.
And we also learned that, you know what, it's OK to take a champion back to the lab multiple times if we really feel that it's worth the effort.
We knew that his theme was compelling, but there was something that wasn't just matching up between his mechanics, his visuals, and his theme.
And this is one of those things is that everything on paper can be totally 100% right, but if players don't agree with it or players aren't feeling those goals that you're trying to hit, there's going to be a problem.
So we're able to collaborate with our subject matter experts, with our players, in order to get the experience right.
And I mean, I'll say it again.
Just because the mechanics stacked up correctly does not mean that it necessarily will in the feeling, which is why you play test it in house, why you play test it with other people at your studio or at your school, and ideally, why you test it with the people who are actually going to play your game.
And then we also learned that you can sometimes have these trade-offs.
It goes back to what you hear from players.
Sometimes they're going to make suggestions and you have to look for the note behind the note.
It's like, what is actually the problem?
You guys are the subject matter experts.
You're going to be able to look at what they're saying and never take it as, from players, a criticism.
Always think about it as if they said something wrong, there's something that has to be causing the problem.
What is it underneath their note that's actually causing that problem?
Let's look for that thing.
And we're still iterating after launch.
That's one thing that's typical with our champions is that yes, we do play tests with players.
Yes, we do play tests in house.
But once we launch the champion, there are thousands and thousands of play tests with real players on live.
And that's when we learn about things that happen maybe at scale or under very specific situations, or for things that we didn't account for, for these external validity concerns.
So we keep iterating.
And in the case of Lucian, we actually continue to collect metrics and user sentiment through things like surveys and through game data to understand how he was doing.
And based on this, we're able to make more iterations, fine tune, and hopefully make him a little bit more accessible.
And today, Lucian is well regarded, widely played.
And if you watch the professional matches in the world championship, he's use quite frequently.
All right, you guys, I hope I didn't bore you.
My name is Genevieve, again.
If you guys ever have any questions, you can feel free to reach out to me at gconley@riotgames.com.
My Summoner name in game, if you play, is RiotCapernoited.
And you can also find me on Twitter there.
I'm always happy to answer questions.
Thank you guys.
[APPLAUSE]
We get a 10 minute break
So 10 minute break.
Come back at 2:20 and be ready to start working on your focus test stuff.
OK, welcome back to class.
Now that you've had a chance to be talked at a whole lot, we're going to go ahead and ask you to actually do.
The plan is to give you about 10 minutes to meet in your groups, plan your test.
I actually strongly recommend you just grab a copy of the focus test, either in the Word version or the RTF version of off Stellar so you've got that as a guide.
Sit down, plan your test and how you're going to run it.
We'll give you 10 minutes for that.
Then we'll have about 45 minutes for people to circulate and test.
And one of the things you should also be planning is when your team members will be out testing other people's games and when your team members will be observing.
Just given the number of people in class, the number of games we have, and the number of people on the team, chances are good that you aren't going to be either observing or testing all the time.
You're probably going to have to wait.
But go ahead and circulate, take the time to watch other teams doing focus testing.
Try not to watch a game in progress that you're going to focus test.
If you realize that you're just sitting around waiting for a chance to focus test, let the team know that you're in line next.
I'd rather you pulled out a book and read it than watched someone else play a game you're about to focus test.
It will be more useful for the poor people who you're testing for if you haven't already seen somebody do all those things.
That doesn't mean hide in the corner and read a book because there's too many people wandering around testing games.
We do expect everybody to participate both sides.
Observe and test.
So try to balance that out a bit.
We'll circulate and keep an eye on things.
And if it looks like at some point nobody's actually testing, you're all sitting around twiddling your thumbs, we'll end it early.
I don't know how long this testing session will take because we haven't had such large teams on this project before.
So it's a little harder for me to know what the amount of time is.
After that, we'll give you some more time to talk in your own teams and come up with the what are the results of your test to analyze your data.
And then we'll ask somebody to come down and for each team give us the summary of your focus test report.
How did your focus test turn out?
What did you learn?
Do you have a plan for what you're going to do with it, or are you still figuring out how you'll deal with it?
Those are all reasonable things given you only had 10 minutes to look at your data.
Thank you.
It is 2:23.
At 2:33, we'll go ahead and start focus testing.
Actually, I'm going to give you 15 minutes to get set up, because I just realized you probably need 10 minutes to talk and then probably five minutes to make sure you have a machine up and running.
So I apologize, it's going to be 15 minutes to plan and prepare for your test.
So that's 2:38 at this point.
[CHATTER]
Yeah.
I think he just wants to--
OK. You should be ready to go now, I hope.
And actively go ahead.
You should have a computer ideally set up to run your game.
Who is actually ready to run a focus test?
Everyone who is manning a demo computer, a computer ready to run the game, put your hand up.
All stand up so that we can see you
Yeah, go ahead and stay up so we can see where you are
I'm not seeing any hands from these teams over here.
OK. All right.
All right.
All right.
As I said that, someone is standing up back there.
How about the Sparkly Redemption team?
OK, all right.
There's someone there to run a game.
All right
So close all the laptops that are not actually running the game would be my advice, and then let's go ahead and get started
All right.
Start playing each other's games.
Go
Yep.
[CHATTER]
Next time, we'll ask them to have two or three
Yeah
As part of the Stellar thing.
He was talking about the--
I got the FOA for free
OK.
It's OK if you don't have a plan of action as you're looking at your results as long as you've gotten ideas to what you're seeing out of your data.
It's hard to look at your data, get it evaluated, talk about it, and figure out what you're going to do about it all in 10 minutes.
We are giving you a short amount of time because we're running out of time.
That said, let's get started with Sparkly Redemption.
[APPLAUSE]
So we had a small hiccup when we started where we didn't have-- part of our core game is collecting sparklies, and we didn't have a version with that when we started play testing.
So we actually got to test out some different movement controls that we were trying out, which actually turned out to be good because the controls that we liked, which was right clicking with the mouse and then left clicking to shoot, a lot of people had a lot of trouble with that.
So it was a good thing that we messed up and then got some information from that.
Then we got a version with sparklies working, and there was a big bug where if you killed all the monsters, they never respond.
So that bug has now been fixed in the last 10 minutes, so that was good.
Other than we, we got just random things like people staying in the corner and just shooting, as opposed to just moving around the map like we want them to.
So we have a few problems that we have to figure out that.
Right now, we don't know the answers.
[APPLAUSE]
Hi.
I'm with Lost Underground.
As a refresher on our game, the idea is that you are lost underground in the darkness trying to find an exit.
And so as part of our game, we have this sphere of visibility around you.
And as we play tested, we got a lot of good feedback on what that visibility means to people and how we can actually use it to make our mechanics and our [INAUDIBLE] design more compelling.
Another big issue we ran into with people play testing was our collision boxes are basically pixel perfect.
And so people would get caught trying to go through the map, which was really helpful for us, because although we already knew that this problem existed, we actually learned a lot about how people were actually trying to make these decisions.
And we learned that instead of using the idea of the entire [INAUDIBLE], a lot of players were actually looking at where the feet were planted and trying to move based on that.
So when we go and fix it, we'll make sure to put the collision box around where the feet are.
We also got a lot of good feedback on the items that we have in our game and where people are actually focusing their attention.
[APPLAUSE]
[INAUDIBLE]
Oh, Plunder Winds.
Yeah
So our game involves moving around a grid and running into random encounters.
The favorability of the random encounters are indicated to the player by numbers and colors on the board.
So one of the biggest things that we got from testing was that people didn't understand the role of the numbers and colors played or how they were related or if they were related.
And they began by moving around randomly and just waiting for things to happen to them.
There's also a mechanic in our game that involves moving with the wind and against the wind.
And so moving with the wind doesn't count stamina, but moving against the wind does.
Or moving sideways of the wind does.
Moving against the wind is not possible.
People had trouble figuring that out as well.
So one of the biggest things we're going to try and do is more visual feedback to the player, help them figure things out as it goes.
Also, people felt like there should be more long-term consequences instead of just planning for the next move or two.
And so we're going to see how we can integrate that into our game.
[APPLAUSE]
Hi.
Blind Aliens.
So we got a lot of great feedback from the use play testing.
So one of the main things we realized is we needed more clear goals and more long-term goals.
So for example, our gun automatically reloads, and that was a little confusing for people because they didn't really see that it was reloading.
But instead of doing something like that where the gun automatically reloads, we're thinking of adding bullets around that you have to go and collect so it's more short-term goals and also long-term strategy.
How do you save your bullets up, when do you use them all.
Another thing we noticed was that our game was way too hard at first, and way too easy after five minutes.
So we decided we need to focus on ramping up the difficulty.
Instead of starting with three aliens attacking you, maybe only one.
And then maybe after five minutes, instead of only having three aliens, maybe 10.
[INAUDIBLE] difficult.
So another thing we focused on was our AI.
Not AI.
User interface.
So I talked about how there was a reloading thing, it was flashing in the corner, but no one saw it so no one knew that their gun was reloading.
They thought it just wasn't working and they were frustrated that they were clicking and nothing was happening.
So we need to make that more visual.
Along with our aliens, when they're half-dead or they're still attacking you.
People thought they were dead.
They'd run into them and they'd die and be frustrated.
So instead of maybe turning them darker, maybe turn them red or something like that to make that clearer.
So thank you guys for all the feedback.
[APPLAUSE]
Modudice
Hi, we're Modudice.
So a lot of the feedback we got was really based on our UI.
And it's because we're modeling a 3-D die using 2-D graphics.
So a lot of people have different opinions about how that should look, and no one thought that the way it does look is OK. [INAUDIBLE] And so we really got a lot of opinions.
Maybe you could do it this way.
Maybe you could do it this way.
And that really helps, because now we can really build up a discussion of how it should look.
How we can make this mental model in people's heads match what they're seeing on the screen, which is the real hard challenge that we're facing.
So thanks for the feedback.
[APPLAUSE]
Beaver Evolution
I told them it was going to be the problem, someone needs to go up
So through play testing, we really realized that we weren't conveying to the user exactly why all the instructions worked clearly and what was happening at each turn.
But otherwise, the game mechanics worked pretty well.
So in Beaver Evolution, you can choose whether to build, populate, or evolve, but it wasn't clear to the player exactly how many beavers you can populate or how many dams you could build, or what natural disasters could happen.
And then between each turn, when a natural disaster happened, there wasn't that much engagement between the player and the game being like, oh, this drought happened and then this is a consequence of that.
So I think going forward, we're going to really try to make it easier for players to understand what happened during that turn, and then what their choices are in terms of picking it.
So a lot of it is clarifying what the instructions are, what the game mechanics are.
[APPLAUSE]
And finally, Comcastic
Well, the two main pieces of feedback that we got from players were about the UI of the game.
And we had some issues with that because our game has a lot of complex UI requirements and we didn't have time to implement all of those prior to this test session.
But we did get a lot of feedback about UI improvements that would make it easier for players to see what was going on with the game.
For example, highlighting houses that are currently being served by a [?
source ?]
center.
The other main complaint that we got was that the game is nebulous right now.
There's not a whole lot of challenge or clearly defined goal.
And so one thing that we're really going to look at going forward is tightening the focus of the game, addling more tension, and making the goal of the game more concrete.
And since it doesn't fit very well into an established genre, we had some more complex design decisions to figure out how to make this a fun and engaging game.
And so that's pretty much what we're going to be focusing on going forward, would be the UI and adding more challenge and tension to the game.
[APPLAUSE]
I have a few words.
Do you have anything to say?
You go for it
OK.
So I just want to remind everybody that you started on this project less than 10 days ago.
OK. And you just finished your first playable prototype, and you did a good test.
So give yourself a hand.
[APPLAUSE] Let's see.
Things to keep in mind.
For those of you who are making mouse-driven games, you want to make sure that you have enough mice to have for your next play test which will be probably project three.
And you've only got another five days, if I'm not mistaken, when this is done.
This is exactly a two week project.
So if you haven't started cutting features, this is probably the time to do it.
Let's see.
Something that, of course, we did notice and you all felt it was that there weren't enough stations for playing the games.
For future play tests, for project three onwards, we are going to expect at least half as many game computers as you have at the average team size.
So I'm not sure whether we're doing with teams of six or teams of eight.
We're doing teams of six for project three, which means each team should be prepared to have at least three computers running with the games.
Something else that we're also going to make it a lot clearer where the game stations are is that the computers that are not being used for game play should actually just be closed.
Take notes using paper, because it gets confusing to figure out which are the game stations and which ones are just like people taking notes and which ones are the ones where people actually coding on the sly trying to fix last minute problems.
You really should have all that sorted out before the play test, not during the play test.
So next time when we're doing play test, just make sure that half your team, basically, is ready to go with the game.
And I would suggest more than half your team should be able to run the code, just in case someone falls sick, right?
And doesn't bring their computer or their mouse.
Rest of the class is for you to work on your own team.
If anyone hasn't signed into the attendance sheet, we still have it up here.
And I want to have everyone give the folks from Riot a big hand for coming and giving their feedback.
[APPLAUSE] Thank you
[INAUDIBLE]
Oh, yes.
They have lots of information about internships.
Oops.
Oh, whoa.
A very, very large stack of documents and positions that are available.
And a deadline, October 15th, which is not that long from now.
So definitely check those out as soon as you can.
Let's see.
Actually, while we're on internships, there's one other thing that I wanted to mention.
I sent out a message for Europe over in AeroAstro.
They are looking for game developers.
Even better if you also know MATLAB, but they just want game developers for a high school science competition that they're running.
It's like first robotics.
It's a robotics competition, only in zero gravity.
So if that sounds cool, you should definitely respond to them quickly because Europe deadlines are coming up.
And that's it.
See you [INAUDIBLE].
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so it looks like we've got everybody here.
It's about 1:11.
Today in class, we are going to end our project twos.
Yay.
We feeling good about this?
Kinda?
I'm feeling good about this.
And we're going to start project three.
So we're going to start off with our project two presentations.
Again, we're going to go through here.
Each team has five minutes to give their presentation.
Slides.
You can set up your slides up here.
We've carved out enough time for today to be able to swap out between computers and for us to ask a couple of questions after each presentation.
After all the presentations are done, we're going to take a break.
Instructors will kind of talk to each other about what we saw in the presentations, and we'll address some of the feedback that we heard, some of the things that we saw in the presentations.
As a reminder, presentations, we want to hear about what went right, what went wrong, and also what would you do differently?
Then we're going to start project three.
And we're going to do brainstorming in class.
Basically, the first week of class we're going to compress into about 30 minutes of this class.
We're going to brainstorm, we're going to do our elevator pitches, and then we're going to form teams.
So we're going to do that really, really quick.
And it might require rolling the die again.
It might require moving you from project to project.
And that's going to be OK. We'll talk about why we're doing that when we go.
So I think that's everything from me.
Any questions?
And if not, Sparkly Redemption, come on down.
PRESENTER 1: So what went right?
The game got made.
It isn't broken or sad or dead.
Even though we had to cut some things, I feel like the general idea of you are a player and you pick up sparklies and your sparklies change monster behavior slash your power-up slash whatever got implemented pretty well in a way that carried across our original idea.
What went wrong?
I guess we had to drop some things.
We had a person on our team not get hit by a bus, but they did basically not have as much time to work on the project as much as we thought.
So we had to adapt to that.
On the other hand, I feel like we adapted to that pretty well.
So not good, but still good, I think, personally.
We were told to test early and test often, which I think we actually kind of did-- maybe not as early or as often as we could have.
But we still tested even when we had something that we felt wasn't really testable but was still touchable.
And I feel like that was kind of interesting to do because people kept telling us, you should totally be able to die.
And we were like, yes, we do think you should be able to die, but we haven't implemented that.
And so on and so forth.
But on the other hand, that meant we still got lots and lots of good feedback and things to think about, like stuff that we probably wouldn't have thought to have implemented but still implemented because we caught that early on.
So yes, test early and test often, even if it's annoying, is what I learned, and I feel we all learned.
And that might be all I can think of
What would you do differently next time?
PRESENTER 1: What would I do differently next time?
Meet up with people.
Well, not necessarily physically, because we scheduled meetings but then we decided maybe we don't need to physically meet up because it's a pain getting people onto campus if they don't live on campus, which would have been fine.
But then there were still times we were like, well, we meant to meet up but then we got iffy about scheduling a time, and everyone got confused.
And I feel like we managed to work our way through it.
But it was still more of a hassle than it probably needed to be if we just sat down and scheduled out a chunk of time and listed things that we were going to get done.
So differently would be better about meeting with each other, either physically or through the interwebs
Any questions from anybody?
OK.
Which game engine was that again?
PRESENTER 1: Sparkly Redemption.
It's the one-armed lady
What game engine did you use?
PRESENTER 1: Oh.
Phaser
Anything to say about the experience of working with Phaser?
PRESENTER 1: So you do the thing where you have a bug and you think it's Phaser related, so you Google vague terms.
And I've worked with Flixel and Unity, so I know personally I'm used to having long form posts pop up that you then dig through.
But I feel like with Phaser it was a bit of a mixed bag.
And sometimes you got relevant posts.
Or sometimes you just went to the Phaser examples and kept digging through it until you found something vaguely relevant.
So it wasn't terrible, but it was still not quite what I was used to in terms of figuring things out.
PRESENTER 2: So we're Lost Underground.
One of the biggest things I felt our team had a problem with was communication.
Because at the very beginning, we were really good.
We all got really pumped for this project, and we got together.
We had the paper prototype.
We edited it a bunch.
We quickly got that over to a digital prototype.
And then it kind of stagnated.
We had the play testing, and we learned a lot from that.
One of the most interesting things is we added the darkness mechanic, where you can only see a certain zone around you.
And some people felt like you shouldn't be able to see as much.
People felt you should be able to see more or less.
Everyone had different opinions.
But we liked the way it worked.
But it didn't work the way we wanted it to.
People weren't concerned about walking around at all.
The whole point of the dark is to be afraid of it, like oh, is there a monster over there, but there could be something to collect, and people weren't feeling that.
And so we found out from the play testing we needed to change it.
And the best way to do this was to shrink it so you could see even less, to zoom out the camera so you feel really claustrophobic and far away from everything.
And I think that really helped us.
We also learned-- what was really funny is we had just a placeholder, a little coffee cup, instead of a coin, which is what we originally went for.
But every play tester was like, oh, coffee.
That's the coolest thing.
I love the idea of collecting coffee.
And so one of the cool things about play testing is if you don't say anything, they will give you really useful information, even though that's not what you intended at all, which was really cool to experience firsthand.
And so we decided to stick with the coffee mechanic, and it ended up being a little more silly than what we originally planned for.
But like I said, communication was really hard.
Planning out who was going to do what with a digital prototype was much more difficult than with a paper prototype, I personally feel.
Because code intermingles in a way that paper doesn't.
And so it becomes really difficult when people are pushing things and pulling things and then things break.
And so that was really hard.
A way to change this, I feel like, would be to just make sure everyone replies to their emails, and make sure people are talking and everyone knows what's going on.
And I don't know the best way to do this.
Maybe an IRC chat, where they constantly have that chat channel there and people can look at it when they want to.
Because emails kind of get lost.
And text messages people forget to reply to.
So something like that, where it's constantly there and you can go look at it and hopefully reply to it in a timely manner.
Or maybe just have-- one of the things I did in my other classes is the very beginning, everyone had to write up a contract of when you'd be working on things, when you reply to things.
And so we had to sign and say, I will reply to any emails about this at 9:00 PM every day.
And so you can expect a reply by my by 10:00 PM.
And so because we had this contract, we knew-- if I send an email to this person before 9:00 PM, I will get a reply an hour later, which was really useful when people were a little hectic and they had other things to do.
At least I knew I would get that email back.
So any questions?
So same question as before.
What game engine did you use?
PRESENTER 2: So we used Phaser.
And any coding questions, I'm going to direct to another team member who was more heavily invested in it than I was
Anybody want to speak up about coding?
[INAUDIBLE] curious about Phaser.
PRESENTER 3: I thought Phaser was an interesting engine to use.
I think in terms of the organization of the code, you have to do a lot of thinking beforehand if you want to make it work well.
I guess it could be a little finicky when you just start building, because we thought, OK, well, we'll add code, and then we'll sort of architect it based on what we see happening.
But then we came to realize that our code wasn't going to be as separable as we thought it might be.
I also ran into a couple of bugs that were in Phaser itself that I had to sort of dance around.
So I would say watch out for the fact that there are still some bugs in it.
PRESENTER 4: Yeah, just callback nastiness and having to think about something will happen now-ish.
Time isn't really now-now.
It's like, now kind of now, then kind of then, floating point numbers that are somehow always floats and nothing is ever actually equal to anything else
Is any of that documented?
PRESENTER 4: This is just found out.
That's JavaScript.
But I thought we could get away from JavaScript by wrapping that all in an engine, but no
Thank you.
PRESENTER 5: Hello.
We were Plunder Winds.
So starting off with the good.
I think our team was actually pretty well organized, and our roles were well-defined.
So from the very beginning, we set up task lists and lines of communication.
We set up a mailing list and then assigned everybody specific roles.
And then from that, we were able to-- none of us duplicated work.
And we set up a good architecture early on.
And so we used Phaser.
And unfortunately, there's this one part of Phaser that's not really well documented in any of the examples and tutorials which is the state machine that Phaser has.
Surprise, there's a state machine in Phaser.
And then once we discovered that-- we discovered that early on and used it.
And so that allowed us to really architect our code well and separate out modules.
And we also used a JavaScript module thing, which basically made it so that all of our coding efforts were really-- were really efficient, I think.
And as for the bad, five out of six, or basically our entire team, came down with the flu the first week, or over the first weekend.
And that really impacted our schedule.
So originally we had wanted to get a playable prototype by the first weekend so we could get people touching it and start getting feedback.
But the plans best laid, it doesn't help if everyone gets sick.
So we had a large schedule impact.
And the way we had to deal with that was, like, there's one guy who was able to code throughout the entire first week.
And then the rest of us were playing catch up.
And so we didn't get anything playable until the in-class workshop on Wednesday.
And I think that really hurt us because we weren't able to get a lot of important feedback until very late in the process.
And so there were some features that we kind of added.
And we were like, this may address player concerns.
But we just didn't get enough feedback or data to figure out whether or not those concerns were actually really addressed
And anything you'd do differently?
PRESENTER 5: I think we would not get sick.
But also, we set up Asana and everything on Monday.
But then we learned, like, oh, we have to do a sprint task list and a product backlog on Wednesday.
So I think there was a bit of a weird issue there, where we already had one task management system, and then we introduced another one.
And then we introduced scrum boards and we were like, well, which one do we use?
And then we just didn't end up using any of them
Any questions?
So my question, I think, is you had the flu.
A bunch of people were out.
You didn't cut features.
You tried to just catch up.
Did you think about cutting features?
Do you feel like you ended up cutting anything at all?
PRESENTER 5: So I think our core game was actually, like-- it was pretty simple to implement.
And I just want to give a shout-out to our team.
You guys pulled off an awesome job and implemented everything by Wednesday.
And so we did actually manage to get a feature complete game by Wednesday.
And then we started adding tweaks, like interface tweaks and maybe a visual cue here and there throughout the rest of the week.
But in terms of cutting features, we didn't really have any complex time-intensive features that we would have benefited from cutting, I don't think
Thank you.
PRESENTER 6: So our project was Beaver Evolution.
And so I guess I'll first go through three of the bigger design challenges we came up with or that we encountered during these two weeks.
And our [?
system ?]
was working with Phaser, which is kind of design, kind of the project challenge.
And so one thing we ran into is we didn't realize there was this thing called stages, and I think you guys just mentioned that.
But since everything is turn-based, and in each turn you can make a choice between building, populating, and evolving, and then disaster might strike.
So there was different things that happen.
So we thought it could be very modular and we could use stages.
But then we learned that stages really start and stop every time you switch stages.
So it doesn't take the info about what happened in a previous stage, which was really important to our game.
So then we ended up having to switch everything over to Phaser groups, so we could hide groups and show groups.
So that was one of our first challenges, just kind of figuring out how Phaser worked and what would be best for our code.
It turned out our code was very modular because there were so many different elements to each turn.
So it was very easy to split up the work.
And that was helpful.
Overall, we really liked Phaser.
Our second challenge is balancing the game, so making it not too easy to win and not too hard to win.
And we kind of changed that by changing some of the winning conditions and then also adding a few mechanics here and there to make the game harder.
Because often it was just too easy.
So for example, you could just keep evolving.
Or you could keep populating your beavers.
And then once you reached the number of beavers you needed, you'd win.
But we kind of changed that by implementing a rule where you can't populate every turn because you have to [INAUDIBLE].
So I guess the slide is not really working.
PRESENTER 7: Yeah.
So I'm not sure why the screen won't show our slides, but we can keep talking about the game.
So Rachel talked a lot about the design challenges we ran into for the game.
And so I'll talk about some of the project challenges that we ran into.
I was assigned the role of the product manager.
So you remember the product manager in the scrum roles is the person who's supposed to sort of lead where the game is going and make sure that all the design implementations are the way that they're supposed to be and handle any changes that we need to make to the code.
So unfortunately, we didn't explicitly assign a scrum master.
So by taking on the role of being the product manager, I also assumed some of the roles of the scrum master.
And I know that this was stated to us in the lecture about scrum, how we should definitely not do that.
And we now see why.
I unfortunately had to run some of the meeting-- I unfortunately had to focus on running the meeting as well as on making sure that the design was going in the direction it was supposed to go.
So I definitely lost some efficiency there.
And we started off very efficient.
We were able to assign tasks, and our product backlog, our sprint task lists, were all very laid out.
We had an early meeting to make sure that everything was going well.
But then by the end, I just sort of started running out of time.
We were unable to schedule a meeting after the play testing session due to everybody's schedules being very tight.
And so some of the changes that we wanted to implement after the feedback from the testing session we were unable to implement.
But we were still able to get a good game together.
Our organization just sort of fell apart towards the end.
So those were probably some of the biggest design problems we ran into.
I would definitely next time, for our next project, make sure that there's a clear product manager and a clear scrum master so that those two roles can be done very efficiently and not overlap.
PRESENTER 6: Yeah.
And I'll just add on, so our testing we really realized that user experience could be improved between having all the players know all the game mechanics and rules coming into the game.
So yeah, in addition to not being able to meet up, not being able to play all the features, we really just focused on improving game experience by adding audio and by adding smaller things.
But I think earlier in the game, we should have thought more about user experience and making sure that was a good flow
Are there any questions?
I think you covered everything.
Thank you.
PRESENTER 8: All right.
We're Comcastic.
So as you'll recall, our game involves placing service centers on a map and trying to cover all of the houses.
So what went wrong-- setting up Phaser took a long time.
It was really hard for us to actually start writing code.
We were using TypeScript, which was important, I think, for us, especially as a documentation.
But it took some time to get a good set-up where you had a posted Phaser where everybody could develop and you had TypeScript integrated.
And it would be nice if-- future projects, I think, will be set up for that.
But for the first project, that was hard.
Meetings-- it was really hard to schedule meetings.
We had seven people when we started, and we went down to six.
And even at six, it was still hard to schedule meetings, mostly because of PSETs.
I had one PSET, but everybody else had a lot of PSETs, and it was a lot of conflicting times.
What went right-- so Phaser itself, I think, really helped us develop fast.
Once we got the hang of Phaser, it was a lot easier to develop.
It was pretty consistent.
The code is pretty good.
It was not hard to debug by just looking at Phaser code for us.
Also, I think we got the synchronous communication down, because we really didn't meet that often, especially as a whole group.
So we were using Trello, and it was really nice to just sort of bounce an idea off Trello, just put in [INAUDIBLE] feature, and then we would sort of discuss it via Trello.
I think that that's something that we want to keep doing.
Also balance-- I think our balance is actually pretty good.
It was sort of a coincidence that our pieces-- so the components of our balance are basically how the map looks, where the houses are, and then also the service centers you can place are.
And you can imagine it'd be kind of difficult to make these two match up, but I think our first random attempts ended up working pretty well.
What we would do differently-- we would definitely stick to one communication tool and stick to something simple.
I think Trello was fine, [?
but even ?]
GitHub issues would be good, and they would integrate with the code.
Definitely we want to keep scheduling partial meetings with some of the group, just the programmers, just a handful of people, whoever can make it to one meeting.
I think with students that ends up being the best thing to do.
And getting a [INAUDIBLE] viable product out early on-- I think we underestimated how important that was, even just getting a Phaser set up where you have an empty game running quickly.
Even that, if you get that done in two hours versus two days, it makes a huge difference
Any questions from anybody?
[INAUDIBLE].
PRESENTER 8: Yes, face-to-face meetings.
Even Google Hangouts actually weren't easy to get everybody at one time
All right, thank you.
PRESENTER 9: Hi.
We're Modudice.
That is our game.
For those of you that played it, it's sort of like a number-puzzle type game.
You move around a dice.
You add numbers, a module of seven.
And you want to make as many sevens on the board as you can, and you get points.
And so what went well for us is our design meetings went quite well.
We had two or three of these that were basically dedicated to figuring out, how do we do this better?
And how do we answer some of the questions that came up during testing?
And I feel like those went really well.
We were able to get good discussion.
We actually came up with, like, solid answers.
And we were convinced, like, OK.
This may not be the best way to go, but it's definitely comparably good to anything else.
So we're going to just stick with this.
And we were able to make task lists as a result of that.
So we decide on something and then say, OK, what do we need to do to actually do this?
And I think that went across fairly well.
And we did a lot of good testing, especially the testing we did in class.
And on top of that, like, all of us just tested the game with our friends, since it's, like, fairly easy to play and sort of fun.
So yeah, the testing was really useful.
And we actually did change a lot as a cause of it.
PRESENTER 10: I'd like to add to the task list that I think we also did a very good job in terms of prioritizing what tasks were important.
I'd like to say that we cut two features-- the dice rolling animation and sound.
And I think those were really good feature cuts because they weren't central to our game-play as a very fast-paced browser game.
And cutting them actually saved us a lot of work and let us have the time to spend on other features and stuff.
PRESENTER 9: And Phaser was good in the sense that it was something that went well, because it was easy to start with and it was fairly easy to use.
Like, once you knew what you needed to do with Phaser, it was pretty simple to get it going.
But it was also a bad thing because it's Java.
At least we did it in JavaScript.
So JavaScript is hard to work with, hard to fix bugs and stuff.
And I could see-- like, Phaser was fine because it was a fairly small project.
I could see Phaser being absolutely horrible for large projects.
Like, I would never want to use it ever.
PRESENTER 11: I actually thought that it actually did not have a lot of good documentation when we were trying things.
I thought the community was not the best for Phaser.
PRESENTER 9: And so Git, I think, is awesome.
But a lot of people weren't familiar with Git.
And so the branching was confusing.
And that became an issue.
So if you're going to use a [?
version control ?]
like Git, make sure everyone understands it.
That's something we learned.
And getting tasks done separately-- we thought that would be most efficient, since we're everywhere across campus, it's hard to meet up and stuff.
But I think it's actually better to just at least start working together.
When you have a set of tasks, begin the work together, do as much as you can, and make sure everyone's clear on how to do what they need to do and what exactly they need to do so that everyone feels like they are actually contributing and working together on this.
PRESENTER 12: I want to add one more point on Phaser, in the sense that-- I feel like Phaser-- we're trash-talking it right now, but I think Phaser is good for a one-person project because it's very easy to manage the stuff on your own.
But with larger groups, I just don't like reading through JavaScript spaghetti code.
So maybe we needed to find a better way to organize our code with more people.
But I just didn't like-- like, I edited a piece of code, someone else edited it, and I was like, wow, it's totally different from what I expected it to be.
PRESENTER 9: And so yeah, next time it would definitely help to have scrum meeting and work physically together, so basically just spend more time together as opposed to messaging on email and stuff.
Make sure everyone understands the source control and the engine.
And like, have more people really think about-- like, have everyone on the team really think about design decisions.
And then communicate, communicate.
More communication definitely would hurt.
PRESENTER 10: I would like to add to that in terms of the communication.
Because as a new member of a team who joined the Modudice, not the original Modudice team, we didn't actually host an information session about what were the critical things about the game, that we made it the way we made it.
So the new members, such as me, had difficulty to put in variable input, at least for the starting duration of the project, and then we picked up later on.
But if we had an information session, something like that, it would be very helpful
Any questions from anybody?
Well, [INAUDIBLE].
When you said that your design meetings-- this was all the way on your first slide, were those online?
Or were those in person?
PRESENTER 9: It was in person.
But those were mostly focused, like-- so we had a lot of questions of how to represent the dice in 2D and stuff like that.
So the design meetings were mostly focused on big questions like that.
PRESENTER 11: In addition, I think we had [INAUDIBLE] meetings.
And I thought that we should just have multiple smaller ones, [INAUDIBLE].
PRESENTER 9: It ended up going, like, two hours
Thank you.
PRESENTER 13: So what was bad?
We're going to start with the bad things.
The first decision we made as a team was using Unity 2D, because that was just the majority vote.
Most people had worked with Unity for the project before.
But it ended up having too many complications.
And it was actually a good thing that we switched to Phaser very early on.
Like, two days into the project we switched to Phaser.
So Unity [INAUDIBLE].
Second thing, we weren't paying too much attention to the assignments.
Everyone was focusing on the game development and making the game look good.
And we weren't updating the product backlog, and a change log [INAUDIBLE].
So we had to go back and remember what we worked on in the past week.
And it was very asynchronized with that.
Another thing is we put off a lot of the work until the day before they were due.
Like, the last project we had a lot of things that could have been done during the week, but the night before, we always have hundreds of emails to sending the emails around, trying to get everything to actually work.
And that was a thing that cut off on our productivity.
Another thing is that we didn't really have a lot of meetings.
We only had two meetings throughout the entire project, which had a downside that we weren't very synchronized.
Even in some of those meetings, some of us couldn't make it.
So we had to update whoever wasn't there with what happened in the meetings and things like that.
And we definitely could have used more meetings to keep our work synchronized.
The last part is CoffeeScript.
CoffeeScript was a good and a bad thing.
I think it was a good decision overall, but it definitely had bad parts to it, which is that we had to sort of-- there's a hack-ish with Phaser that had to compile CoffeeScript to JavaScript, which caused some problems with the [?
gulping, ?]
where it would sometimes not work.
And we had to run it twice.
It generated JS files which we had to not commit into the source control because they were changed when the CoffeeScript files would change.
And so there was some problems with it.
But once we had it up and running, it was so easy to write code and just iterate on the code.
It was also easy to read other people's code, because it simplified 100 lines of JavaScript into two lines of CoffeeScript.
I'm exaggerating.
And then finally, also, it optimized the JS when compiled, so it would run faster.
PRESENTER 14: So talking about now some things that went well in addition to finding CoffeeScript, we had really great group direction from the get-go.
Even though we didn't have a lot of meetings, I think one way we still managed to make that work was we divided bigger tasks into the sub-categories really quickly.
And we also set up a Google folder in which we kept everything.
We didn't end up using Trello or anything like that because we really found that a Google folder with everything in it was the simplest for us-- that along with just one chain email where we sent everything really kept everything focused and everybody in communication, in the loop, as best as possible.
So source control, we used Git and GitHub.
And we actually had a great experience with that.
I'm sorry some people didn't.
Nothing ever broke, and we barely had to merge because I don't think we did very much branching, which is the way I think we avoided any problems there.
So iteration-- we iterated our game a lot.
We changed a lot of things, because we'd play it and be like, well, people keep dying, so how can we make that better?
Another example is with our sprites, as you can see up there-- so the initial designs and then what we changed them to.
We actually had a problem where somebody looked at our first alien design and said, that looks like a piece of the female anatomy.
That is distracting.
You need to change it.
And we said, OK, you might have a valid point there.
Even if we don't think so, maybe it does.
Let's change it.
So we iterated a lot.
And then focus testing-- we really actually implemented-- talking about iteration, in our focus testing, the two things we found were that new players were dying really fast.
It took them, like, five times before they realized how to play the game.
And then, in addition to that, after five minutes, they were getting bored.
So we actually took that to heart, and we changed one of the main premises of our game, which was this reloading thing, that it would take a couple of seconds before you could shoot again.
And instead, we added a short term goal of actually picking up ammo.
So you can see-- it's kind of hard to see.
But the ammo-- the big thing is just the animation.
But if you can see on the screen, there's a player, there's the ammo, and there's a bug.
And basically we changed that whole way the player actually shot to make the game a little more difficult over time but also starting easier, by having everybody smaller and then the bugs get bigger.
So a lot of changes and iteration even based on our focus testing.
What we would change-- we'd start earlier.
I know we talked about a lot of changes we made were the night before things were due.
Pretty much all of our documents changed the night before they were due.
And it's kind of frustrating, you know, having to stay up really late the night before it's due when you've had the whole week to do it.
So just start earlier.
I mean, we started pretty early, but finish earlier, definitely.
Hands off for the last 12 hours, kind of thing.
And then we also talked about this-- being focused on what's actually due.
So one thing that was good about having a lot of focus on our game was that we had a viable product really early.
But a lot of our other documents kind of fell by the wayside, and they kind of came up to one or two people to just fix them all the night before they were due.
So definitely focus on those things and more accountability with that.
And in conclusion, even though we had some frustrations, it went pretty well.
Our game ended up pretty clean and creep and fun.
So we really like it.
Thanks, guys.
[APPLAUSE]
Questions?
How did you stumble upon CoffeeScript?
What was the decision when you were trying to [INAUDIBLE]?
PRESENTER 13: Jen, you want to take this one?
If you could come down [INAUDIBLE].
PRESENTER 15: I used CoffeeScript a lot before I was really familiar with the build systems and stuff.
So I figured it'd be best to kind of set up a build pipeline that optimizes the [?
alpha ?]
JavaScript to make the game run faster because [INAUDIBLE].
PRESENTER 13: And CoffeeScript also has a lot of shortcuts.
It's like Python.
It's like using Python to write Java.
It's so good.
Like that.
PRESENTER 14: Any other questions?
Thanks.
OK, feedback.
And instructors, please feel free to jump in if I'm missing anything.
Oh, and I forgot to copy all my notes.
All right.
So yeah, I was wrong.
Ha, you caught me.
Slides were intended to be required for this project, but they were not.
So you're not going to get penalized for that.
The issue for that is slides and visuals are recommended to help your presentations stay on topic.
Basically, you had some issues sometimes with keeping track of changes.
We had some issues with keeping track of our changes to our documents.
So QA is important in everything that we do in life.
But for project three, slides and visuals are recommended to help your presentations stay on topic.
And what we saw presentation-wise, slide-wise, was perfect for these five minute presentations.
Quick, dirty, gets the point across, gets the bullets across.
Visuals, like images or something, if you need it.
For project four, that's a 20-minute presentation.
Visuals are more important there.
And we'll talk about those presentation requirements later when we start project four.
So discussion on meetings, things you should do in project three-- address how you're going to meet in your initial design meeting.
The most important thing you have, before you even start designing your game, is to talk about who can meet when, what is your schedule.
You all have syllabuses from your other classes.
When are your PSETs due, when are your recitations, all that stuff.
Put it in your high-level design document.
It's actually one thing that we use often in that one section where we talk about team roles, is team meeting times.
And I think I heard this team mention that, that you've used in previous classes.
That's a very good tip.
Use that.
If others can't make it to your meetings, is there a way you can take video?
Is there a way you can archive what happened in the meeting for those people who couldn't come?
Here's something that I've seen some local companies do-- drop in IRC.
Basically, even something as simple as keeping Skype open.
Even better-- actually, my notes weren't put in there.
Even better, an IRC solution that has some kind of archival process in, so all the previous chat can be saved.
I know there are some solutions out there that actually hook this into your GitHub or Bitbucket or what you're using.
Those solutions are out there.
Try one of them.
But again, even something as simple as having Skype open and online, you can always tell if one of your team members is online.
You can throw a text and say, hey, can I get your feedback on this?
Can you take a look at this code?
For your daily scrum, 15 minute Hangouts probably is-- all you need is 15 minutes for doing that kind of daily scrum meeting, so trying that out.
Problems we saw with backfilling your product management.
Integrate your project management from the beginning.
We know that this was a problem for this project because we introduced it to you midway through.
That's OK. You actually did-- we mentioned this, you did some pretty good use of product management.
The goal for any project management tool that we're teaching in this class is that it should be helpful.
It shouldn't be a hindrance.
That said, we'd like you to get some practice with it first before you start throwing it out.
So for project three, please use the project management tools that we described as we've described them to you.
In particular, product backlogs, sprint task lists, and the design change log, and the vision statements, those high level design documents.
Scrum boards, mostly optional, but we found them useful.
They might not be useful for how your team is distributed.
It's up to you to figure that out.
For project four, we'll give you the ability to create the system that works for your team, so long as you give us what information we're asking for.
So you know you need this information.
You know you need a way to manage your tasks.
You know you need a way to manage your features and to prioritize your features to estimate.
But how you actually do that and how you actually save it, that can be up to you for project four.
We will want to see it actually cause some improvements.
And if it doesn't cause improvements, let us know why and what kind of problems you found with that.
And again, the design change log.
This is a design diary.
It's even better.
It's your meeting minutes.
You should be able to fill it out in five minutes.
Along with your task list and backlog, it kind of shows the history of the project.
Take the time just to spend those five minutes at the end of each meeting to just throw something quick down so you know what has changed in the past.
Phillip, you want to chime in on this one?
Modular code doesn't mean your tasks are actually separate
Yeah.
This is just something that I got a sense of from watching everyone's presentations today.
A lot of folks, even the folks who mentioned that the code was well separated and people could code independently, it doesn't mean that there are no benefits to actually still working side by side, even if you're working on different parts of the code.
And some teams mentioned that.
So actually, again, being on Google Hangout or Skype or IRC or whatever, if you're working in your own rooms, or even better, a lot of people mentioned scheduling those face-to-face meetings-- it's OK for you to have a face-to-face meeting, even if you're not planning on working on the same batch of code.
And you can actually still save a lot of communication time-- if nothing else, just to tell people what you're doing.
So maybe it's not speeding you up, but it might speed everybody else up, and suddenly it helps the communication
Form strike teams.
You've got six people, maybe seven people, and eight people on project four.
Not everyone needs to be touching code.
There are other tasks that need to get done.
You can form up into smaller teams.
One team might just be working on some kind of-- again, if it's modular, it's going to have some dependencies.
But if you can modularize it out, great.
But even better, especially with project four, when you have longer time, one team doing paper modification, doing UI elements, running focus tests, while another team is writing code.
Think about how you can better utilize all the personal resources and all the man hours you have without requiring all these clashes with so many people touching the code.
It's that too many cooks in the kitchen kind of thing-- not everybody needs to be doing that, need to be touching that.
Here's another thing we noticed.
A lot of what you asked for was basically more time-- just various different ways of saying you wished you had more time, you wished you had started earlier, you wish you had iterated earlier, the flu hit.
A lot of what you're describing there are things you're not going to have any control over ever.
You never have control over that stuff.
What you need to do is focus on those things that you can control.
So if you have illnesses, if you have major technical problems, the game is whatever you decide the game is.
You've actually got some freedom in this class in that you've told us what the game is, and we're actually even telling you in the last week, again, tell us, what was this game actually supposed to be, so that when things change midway through the project, be flexible.
Change what you're delivering to us.
Make it so that whatever you're delivering to us runs, works, is playable.
This isn't a design class.
We're trying to teach you some design skills.
But it's not focused on design.
So if the final game isn't fun, it's sad, but it's OK, so long as it runs, so long as it's playable, so long as it fits all the requirements that we're asking of you from the beginning.
I know it's weird.
This is the only class we do this in.
Trust me.
All right.
So that's it.
Any other questions about project two?
I think you all did really, really well.
Really happy to see how the presentations went out.
Yes?
Is there any way we can see what everyone did?
What's that?
Is there any way we can see what everyone did?
Oh, yeah.
So one thing I highly suggest, somebody on your team email the video game mailing list.
Video games, yeah.
I did the plurals wrong.
I never do that right with mailing lists.
videogames@mit.edu-- email out a link to your game to your classmates if you want people to play it and give you feedback on it.
At the end of every project do that.
Actually, use that mailing list as a resource for you to talk to each other across teams.
Hey, here's our game.
We need a little bit of feedback on this.
My team is all sick.
Can somebody please play this and tell me what's wrong?
Can you play this on a different browser and tell me if it's broken?
Take advantage of that mailing list.
Any other questions?
Yeah
So [INAUDIBLE] postmortem, will we get more feedback on project one and project two, what the group grade was?
So there are no group grades.
So what you're getting is-- your grades, if you look at the grading rubric, 20% of your grade is individual.
The other 80% percent is your group.
So the grade that you get is basically adjusted based on the quality of the write-up
OK. And on project two, will we get more feedback on everything we've done, so we can better plan for project four?
Yeah.
So for project two, you're going to get feedback on your design change log again.
You're going to get feedback on your task list and product backlog.
We just gave you feedback on the presentations and on those skills.
You're going to do that again for project three.
You get your feedback on the individual write-up.
Is there anything I'm missing there?
And then game functionality-- we're going to give you more feedback on game functionality, because last time we did that in class.
This time we're actually going to play your digitally released games.
We'll give you some design feedback there for product three
Yeah.
Because we could just show the actual numerical grading proclamations that we do, but that's meaningless without actually explaining what it is.
So what we'll try to do is focus more on actual text feedback on what we thought were the strengths or the weaknesses of whatever you've given us
That answer?
Cool.
Any other questions on grading?
All right, give yourselves a round of applause for finalizing project two.
[APPLAUSE] Let me double check to make sure I didn't miss a break.
No, we just took a break.
Ha, ha.
All right.
So project three, you're going to practice project management, meaning you're going to use those tools that we talked to you about in the middle of project two from the get-go, with a slightly larger team.
Some of your teams were five people this time, minimum is going to be six.
We're going to make sure we leave class today with at least six people on each team.
And again, that's just to get used to it.
Project eight, your minimum is going to be eight people.
So with a larger team, you're going to focus on design iteration to maximize usability using user feedback through independent user testing.
So the keywords there-- users, feedback, usability.
Those are things we're going to be looking at when we're grading the functionality of your games.
So for project management, we mean create and use a product backlog and a sprint task list.
There's few turn-ins for that.
For design iteration, you're going to conduct focus testing and user testing on your own.
And I'll touch on this again, but basically you're going to turn in two focus test reports, one of which can happen in class on a scheduled day.
The other one must happen outside of class with a group of people who are not in class.
So get people to test your game and report back to us on that.
And again, maximize usability.
All the iteration that we see you do on your games, you should be iterating on that user interface or that user experience.
So we are going to ask you to make a little bit more complex game than before.
But really what we're trying to get you to do is make a complex game that has a possibly problematic user interface.
Give us an interface that's good.
Give us a user experience that matches the user experience that you want to strive for when you're designing the game
Just a quick thing.
We do have a couple of lectures coming up on designing for that.
But as Greg mentioned earlier, this is not actually the design class.
There is actually a different design class that some of you have taken and are going to be distributed among the teams.
Or you might have taken, say, Rob Miller's user interface class, which also has a lot of those same sort of concepts.
If you've taken any of those classes, you should be identifying yourself to your teams, because you probably have actually practiced designing for usability a lot more than a lot of the other people in your group.
And that's what we're going to be looking for.
But most importantly, the testing that we've been talking about, we are expecting you to be able to do that.
And that anyone of you have already got the basic foundation of
So our design constraint-- we're leveling up from planning for randomness.
We want to see trade-offs in decision making.
So every decision made by the player must have a positive and negative outcome in their play state.
So first off, what does that mean to you?
What do you think that means?
Jenny?
Good things and bad things happen when you take steps.
So you have to decide whether the good things outweigh the bad things at your current point in time?
Yep.
Basically it-- opportunity costs.
Risk management, future risks, unknown risks.
There should be some unknowns and knowns there.
So while the players should know that there's going to be some positive and some negative, they might not know the values of those positive and negative outcomes.
So there's some kind of uncertainty going on in the game there.
And then some of the side effects-- sometimes when you make a decision, there's going to be an immediate change in player state.
Sometimes it's going to be a long term.
It's going to chain or combo up from previous decisions, causing problems in later things.
So easy way to think about this, again, is of course strategy games.
But you're also welcome to apply this to any genre that you might like to experiment with in this project.
So here's some suggested goals.
Think of project-- yeah, go ahead
Do we still have to use planning for randomness in this game?
Yeah.
So there should be some amount of randomness in this game.
So it's building onto it.
So there should be some kind of uncertainty.
It's not the focus of it anymore.
So project two and project one were focusing on just that randomness.
Project three should have some of the randomness in there.
But really, what we want you to focus on is trade-offs.
So that's a tool in your toolbox.
Try it out.
If it doesn't work, if there's a reason it doesn't work, then don't use it.
But there should be some uncertainty there.
And again, all this is building up to project four.
So things you should try out in project three-- work with new people.
The method we're going to have to put you in teams is going to try to do that.
So you'll be working in different teams with different people.
These are basically auditions for project four.
Use a new game engine.
Did everybody use Phaser?
Yeah.
You're not using Phaser for project three.
So we're going to basically want you to go back to that game engine tutorial, think about the other game engines you have available.
We had one team to use Unity, and they had some issues why they wouldn't use Unity.
And actually, those issues will probably apply for project three as well.
Maybe you don't want to use Unity.
Maybe it's too scary.
Maybe it's a challenge.
Figure it out.
And the last one-- I posted this to Stellar.
It's a light read.
Games For a New Climate is basically our handbook for project four.
Our client presented us with this information.
This is basically how games are used for game based learning, for climate change, for talking about planning for future risks, the complexity of future risks.
Take a flip through the book.
Read it now.
Start it now.
Have it read by project four.
Again, we don't have required reading here, but you'll have a better game, you'll have a better product if you take a look at this and maybe even do a little bit of research on your own, which is actually going to be part of project four.
We'll talk about that later.
But if you want to make a game that does this, you can.
If you want to make a game for project three that is a game to help a policymaker understand the need to spend money, time, or resources on disaster preparedness as a result of climate change, feel free.
Part of that trade-off design challenge is actually inside of that statement.
The uncertainty is inside of that statement.
Project three actually would turn into a possible springboard and a way to recruit people for project four.
But again, you're not necessarily going to have the same group.
So it's up to you to figure out if that's going to be useful for you on this team or not.
All right.
And then the hard requirements-- please read this handout on Stellar.
I'll actually read the handout on Stellar too this time.
Sorry about that.
Maximum play length, again, is five minutes.
Again, single player game.
User interface tested for legibility and usability.
It must use and play audio for the player.
And then, like previously, players can pick up and start playing the game with no external instructions.
So if there are instructions, which there can be, it must be inside of the game or at least inside of the frame of the web page that you are using as a link for us to play within.
And then, of course, must be delivered as a browser game, running on Chrome.
And also, only in those engines that w e used in the tutorial.
Deliverables-- so actually, this is Monday the 29th.
Wednesday is the 1st.
On Wednesday the 1st, we have a guest lecture coming in.
Swery65 is coming in to talk to us about his new game.
His lecture is probably going to last about an hour.
That means you're going to have two hours in class on Wednesday to work in teams and create a low fidelity prototype.
It is not something you're required to turn in.
We are asking you to have a playable game on Monday, the 6th.
Whether that's digital or paper is up to you, but we highly, highly, highly recommend making a quick, low fidelity prototype on Wednesday in the two hours you have, and then based on that prototype create the product backlog.
It's the best way that I've found to make product backlogs.
You might find a different way.
That's OK, as long as you make one.
But on Monday, on the 6th, turn in the Stellar, your high level design doc, and what your product backlog is.
So what are all the features that have been estimated, that have been put in priority order?
In class, you're going to give a really quick--
Next slide
What's up?
You're talking about stuff on the next one
Oh, wow.
I'm talking about stuff on the next one.
Thank you.
That's weird.
In class, two minute presentations.
Basically, just the core of your game design idea.
Let us know what you're making.
And then we'll be able to test your prototypes.
I think-- let me look ahead.
Yeah, so testing prototype is not required on the 6th.
But if you do it, please we'll test them then.
We will also test on-- and this is where we actually do want you to have a playable, digital version.
Basically, let us know that you have a playable game by October 8.
And turn in via Stellar your sprint task list-- basically all the tasks with time estimates that are going to happen between Wednesday, October 8 and Monday, October 15.
And the project due is the same kind of stuff that we did for this project-- your game prototype builds, your post mortems, your design change log, updated design document if you have an updated design document.
And again, two focus test reports this time.
One can be created in class on either the 6th or the 8th.
The other one should be created outside of class with external testers
Question
Yes?
Is that Monday the 13th or Wednesday the 15th?
Oh, my lord.
Did I really make that mistake?
Can somebody check on Stellar for me?
So it should be Monday the 13th, right?
No, we think it's Wednesday
Wednesday the 15th is when we're having people turn stuff in?
Yep
All right, holiday.
Right?
Is there a holiday that week?
Yeah
OK. Stellar is the right place, and I will update these slides before I post them to Stellar
Let me check on the big schedule.
It's Columbus Day.
So Monday, there is no class
So Monday, no class.
So Wednesday, the 15th.
So you got a little bit of extra time.
But it's a holiday, so maybe you don't have extra time
But if you get it done by Monday, then you can let your code soak a little bit [INAUDIBLE]
All right.
So we are going to do brainstorming, and we're going to brainstorm in groups that you are not currently working with, so you can meet new people.
You're going to get a card.
It's going to have a number on it-- one, two, three, five, eight, or 13.
I'll hand out the cards really quickly.
Go to one of these stations, erase whatever's on the board if you need to erase on the board.
If you're in five, eight, or 13, you're going to have these easel pads.
We are going to give you two timed quick brainstorming sessions.
Five minutes, small break, another five minutes.
When we break, change whoever's writing things down so everybody has a chance to speak.
After this, each brainstorm group is going to be allowed to make two pitches to the larger class.
What we're going to ask you to do is write a title for the game on a large Post-It, and we'll hand these out.
One of these, yeah.
We'll hand these out when we get there.
In your elevator pitch, make sure to address the core mechanic of the game and how it's applied to the design constraints.
Games from project one that were not selected for project two can be pitched if your brainstorming group says that was actually a pretty cool idea.
Let's pitch that too.
But it must be altered for design constraints.
And I'll give you a break after that second brainstorming session to come up with these pitches-- about 10 minutes, I think.
Any questions about this?
Psyched?
Pumped?
All right, I'm going to hand these out.
[SIDE CONVERSATION]
Team one, pitch one, come on down.
And just like before, you've got about a minute.
Describe-- oops, wrong ones.
Give us the name of the pitch, the core mechanic, and how this pitch approaches the design constraint.
Pitch, go.
PRESENTER 16: So our first pitch is called Dragon's Lair.
It bears no relation to the previous Dragon's Lair idea.
The idea we came up with is that as a dragon, when you're acquiring gold, the more gold you have, the more heroes want to come in and mess you up to try and steal your stuff.
So we thought that there's a fun trade-off there between acquiring more gold and becoming a more powerful dragon, but at the same time having more and more heroes wanting to come in and attack you.
We also thought that just in that, in itself, you don't quite get the idea of meaningful decisions, because everybody's just going to want to be a more powerful dragon and prove themselves as, oh, I can handle more and more knights.
So we thought it'd be interesting to have this idea of not just being an enemy to everyone.
You can actually ally with some of the villages in the game so you'll be making less money, but then you'd have more people on your side to help you defend.
So the trade-off here is between being nice to people in order to get more defense or slaughtering them or just steal their money
Nice pitch.
Number two.
PRESENTER 17: All right, so our second game is called caffeine worker.
And it's based on the idea that caffeine will pretty much simulate your [INAUDIBLE] where you can actually get more work done under less amount of sleep.
But it also tends to have side effects.
So our game pretty much revolves around you are a worker, and you want to get these tasks done.
And you want to get these done as efficiently as possible.
So we're going to have something like a 24 hour clock.
We're going to have night and day.
We're going to have a lot of different sort of actions that you can sort of, like, decide on executing, such as you can figure out how many hours you want to sleep, how much caffeine you want to take, and also you can also figure out your eating schedule as well.
So we don't really have the nitty-gritty details, but we imagine that this game will involve a lot of scheduling, a lot of sort of-- I don't know, the trade-offs between sleeping and eating and drinking coffee.
But in the end, I think what we want to do is that we want to make this game sort of like a platformer, where you are actual in control of the character and for [INAUDIBLE] your tasks, something you need to go from point A to point B.
But these different sort of attributes, how much caffeine you have over time, how much sleep you have, whether you're eating enough, whether you're happy or not-- those sort of effect your motor skills as well as some other implementations that we have yet to figure out
All right.
Thank you.
Thank you group one.
Group two.
Pitch one, category A.
Remember, it's one-minute pitches.
PRESENTER 18: All right, so here's the idea.
You are an administrator at MIT.
And your goal is the management of MIT and promote MIT's ideals and maximize their endowment and all that.
And your goal is to carefully trade-off the short-term versus long-term policy.
For example, you've got this plot of land.
You could lease it to Pfizer for 60 years and get 2 billion straight to the endowment.
But then you can't expand it till later.
And do you want to appease the students to get more in tuition?
But maybe you need that money, so you raise tuition, but you also get rid of your unpopular flag policy at the same time.
So you can play as many different administrators.
You can play as, like, Dean Colombo.
You can play as Kevin Kraft from the Office of Student Citizenship.
You can play as the shady MIT corporation with this weird board of self-voting members that no one really understands.
That's our pitch one.
[APPLAUSE]
What's the name of that one?
PRESENTER 18: MIT Simulator 2015.
PRESENTER 19: Our second idea is build a car.
And we thought that you could start out with a limited amount of money, and we will send the car you build after a turn through a simulator.
And so you don't exactly know which maze it's going to get.
So depending on the result of that, you have trade-offs of how to spend your money, what parts to buy, and build a car
Great.
Group three.
PRESENTER 20: Hi, everyone.
I'm going to tell you about Fight or Flight, which is from our first project.
And if anyone played the paper prototype, it's going to be completely different.
So the idea is that you're playing this platformer, which is a map that has a bunch of separate little rooms.
So any given little room might have four or five platforms.
It's not really big.
And so this whole big map has a bunch of these little rooms, all connected by different passageways.
And you can get from different places to different places.
And all of these rooms are potentially exploding.
So this is some sort of spaceship, and there's a meltdown.
Or, I don't know, this is, like, the tavern.
And near a volcano, things are filling with lava, or something.
However you want to skin it.
So each of the rooms will give you some sort of warning about whether or not it's about to explode.
And then some time after the warning starts, it explodes.
And then if you're in there, you're in trouble.
So the trade-offs in this game are that-- so the overall goal is to get somewhere, is to get out, is to get from one side of the map to the other.
And in between, there's a whole bunch of other people who want to fight you, just for the heck of it.
So they want to fight you, you want to fight them.
But the more you fight, the more tired you get.
And the more tired you get, the slower you move.
And the more slowly you move, the harder it is to escape a room once it starts exploding.
So you can choose to fight, and you're probably going to kill them, you could kill the enemies because they're AI written by people in this class.
Versus, like, the player who can actually just implement strategies and things.
But if you do go and have fun and try to kick everyone's ass and you succeed at that, well, then you're panting in the middle of a room that's about to explode.
So fight or flight everyone.
PRESENTER 21: The second one is Jigsaw World, another platformer where you are inside a jigsaw puzzle, where every time you get to the end of a block section square thing-- they're oddly shaped because it's a jigsaw puzzle-- you get a choice of a new set of tiles to put on the end of the one you've already gotten.
Sometimes you have to go back and pick one up off the beginning of your puzzle and put it at the end.
And various power-ups make certain ones incompatible, like being able to fly.
That means you can't swim.
Being able to swim means you can't jump-- all those kinds of things.
You're kind of creating your own world that you then have to navigate differently.
Oh, yeah.
And deconstruct it.
You never get to see the whole thing at once because it's falling apart as you're taking it and moving it somewhere
Thank you.
Group five.
PRESENTER 22: So our first idea is a game called The Gun Wars, in which you're managing a defense company.
And you're trading off who you're going to sell your product to amidst a bunch of different clients.
So you could have somebody go up to you, you get some information on who they are, say, hey, this is James, seems like an OK guy, and he wants to buy, like, 10,000 missiles, air-to-surface missiles or something.
You have to surmise him, figure out if that's somebody you want to sell to, figure out at what price you want to sell.
And then there may be some consequences for that.
Maybe you sell to Jimmy, and then he launches an attack on some country you like or don't like.
But more importantly, maybe he does something your other clients like or don't like.
And so you lose other business.
Or you get new business from people that are Jimmy's friends and realize they can buy weapons from you.
So it's sort of a social responsibility game.
You're a defense contractor.
You're managing some pool of money, trying to make money off of contracts.
And the trade-offs come with who you're deciding to buy and sell from.
PRESENTER 23: Our second idea for the game is Bullet Craft.
Bullet Craft is a fast-paced, top-down shooter game that follows the simple formula, the classic formula, that the player controls the ship, moves around, [INAUDIBLE] that's coming at him, and shoots the enemies that spawns from the top of the screen.
Well, you might ask, where does trade-offs come into play for Bullet Craft?
So there are actually various ways we can augment the game play so that we will incorporate trade-offs.
For example, on the micro level, we can have ammo limits or overheating guns.
And we can have a fuel limit which if the player moves, he consumes fuel which pollutes the environment, and the enemies become stronger because of that pollution.
At the macro level, we could also have the player choosing different types of weapons, spending their money on different types of ships and even a crew.
What I like about Bullet Craft is it is very easy to implement, and it's guaranteed to be fun, so that if we make Bullet Craft, we are sure that we will be able to make a successful game
Group eight.
PRESENTER 24: So the first idea we had is Daydream Inc.
So in this game, you're like a regular office worker.
And you have to try and not get fired.
But the goal of the game is to succeed in your daydreams.
So there's basically two mini-games within the game.
There's surviving the office life, and then there's also your daydreams where you're fighting your boss or something.
And it's interesting because you use tools from the office to build weapons for your daydreams.
So you might get a stapler and a rubber band and make a staple gun or something for your daydreams.
So you have to spend both time and research from the office life in the daydreams without getting fired, so you can eventually win the game.
Also, if you become CEO in the office life, so if you do really well there too, you lose.
PRESENTER 25: So our second idea was Score High.
So basically, you're an MIT student.
And you have a list of maybe three classes.
And you have some stats-- like, your food, sleep, happiness, health.
And the thing is, each of your classes has PSETs and tests.
And you have to get to the tests on time, and you also have to turn in your PSETs on time.
And so how do you do that?
So you run around the maze, and you have to get to the right building at the right time.
But you also, when you're there, actually have to take the tests.
And depending on-- you pick up different items.
And say you drink 10 cups of coffee, you're really jittery.
Instead of seeing all of the options for the multiple choice question, suddenly you can't look at it, and you only see one question.
And somehow you miss the back of the page.
So you fail the test, right?
So the whole game is about choosing-- you know, when am I going to drink coffee?
Or maybe I'll buy my friend coffee, and he'll give me the PSET answers so I don't have to spend five minutes doing the PSET.
And managing those trade-offs while also running around this maze and getting everything done on time.
And if you fail any of your classes, you lose
Group 13, come on down.
PRESENTER 26: Our first idea is Ghost Face.
The title's a working title.
The idea is you're in a maze, actually a maze.
And you have very limited visibility.
You can only see a certain radius that is much smaller than the actual size of the maze.
And there's ghosts coming at you.
And basically the idea is to get to the other side of the maze, solving the maze, but you only have limited disability.
So you have to explore the maze and remember which path was a good decision and which path wasn't.
So you have to trace back a lot.
You can also add in more things.
Like, you may have a certain number of beacons, light beacons, that you can place in different areas, but a limited number of them.
So you want to be-- you want to make a decision as to what would be a strategically good place to place a beacon and reveal an important and critical part of the maze so that you can see more of the maze.
So that's our first idea.
PRESENTER 27: So our second idea revolves around DNA.
We haven't found a good title for it.
But the idea is, let's say there's an organism.
And you have a certain set of diseases that you want to try to get rid of.
And your goal is to somehow go around finding other organisms that have their set of traits.
What's the diseases that you don't have, some other diseases that you have.
And play a game of chance-- so whether it's worth it to spend your time and energy eating or mating with the other organism and seeing if those genes come across to your [INAUDIBLE].
And the trade-off is-- the trade-off is, do you want to mate with this organism or not?
We're still working on the details of this, but the idea is this gene competition [INAUDIBLE].
PRESENTER 28: And random mutation.
PRESENTER 27: Yeah, random mutations [INAUDIBLE]
OK.
Thank you.
All right.
We're actually really good on time today, so you'll actually have some time to work in your teams after we get you all sorted out.
So we heard about Dragon's Lair, Caffeine Worker, MIT Simulator, Build a Car, Fight or Flight, Jigsaw World, The Gun Wars, Bullet Craft, Daydream Inc., Score High, Ghost Maze, and DNA.
12 ideas-- actually, a few of them could probably combine pretty well together.
So we're not really throwing the ideas away.
We're really just trying to get together into groups of six around a base idea we can start with.
So if you've got a game that you're really, really excited about working on, come right down and put your name on it.
And make sure your Post-It has your name and the game engine you used in the tutorial assignments.
Let's see what we got.
All right, raise your hand if you still have your name tag, your Post-It.
One, two, three, four.
All right.
We have Caffeine Worker only has one person on.
[?
Zigamantias ?
], you have your name tag back.
Nope, sit down.
Just sit down.
Jigsaw World.
Sorry, Kathleen.
All right.
We've got one, two, three, four, five, six.
DNA is done.
So DNA is Derek, Eduardo, Ava, Harry, Trisha, and Lauren.
One, two, three for Ghost Maze.
Three for Score High.
Two for Daydream.
None for Gun Wars.
Two for Bullet Craft.
Three for Fight or Flight.
One, two, three, four for Build a Car.
Two, three, four, five, six.
Oh, cool.
MIT Simulator.
Don't get me in trouble.
Norman, Jordan, James, Peter, [?
Szeun, ?]
and Sam.
Bullet Craft and Daydream-- first off, does anybody who has their name tag on them, are they all excited about either of these two games?
I am
You are?
Is your name-- do you have your name tag in your hand?
OK, raise your hand if you have your name tag in your hand.
All right, come down here and put your name on a game.
I'm going to get them all.
I think we have enough to get every game to six.
10, nine, eight, seven, six-- hey, five.
Cool.
Thank you.
All right, Fight or Flight, Bullet Craft, Daydream I think are all either going to go away or combine somehow.
Score High, one, two, three, four.
One, two, three, four, five on Ghost Maze.
Just needs one more person.
Five on Build a Car.
Just needs one more person.
Four on Dragon's Lair.
One, two, three, four, five, six.
OK. Daydream is gone, Bullet Craft is gone, Fight or Flight is gone.
Move yourself to another team.
All right, Ghost Maze has six and is done.
Score High has five, six.
Done
Everything is full, isn't it?
Yes.
So put yourself on one, two, three, four, five, six.
Put yourself on any of them.
Five, four, three, two-- thank you.
So here are our final teams.
We're going to do one last shuffle.
If anyone is like, I don't want to really be on that one, they can move to another.
For Dragon's Lair, we have Roy, [?
Mikael, ?]
Liz, Zigamantis, Devon, Caleb, and Kathleen.
You have a good mix of Unity and Flixel, so I imagine you might use Unity or Flixel.
Build a Car, we have Rodrigo, Anderson, Matt, [?
Taj ?
], Jeremy, and [INAUDIBLE].
We have Unity, Unity, Unity-- three Unity, three Phaser.
I wonder what you're going to use.
Score High, six.
Miriam, [?
Sen ?
], Megan, Kevin, Sabrina, and [?
Zinue ?].
And we have Haxe, Haxe, Flixel.
Oh, wow, lots of Flixel.
And two Unity.
So maybe one of those.
Ghost Maze, [?
Jutoshka ?
], [?
Salaam ?
], Daniel, Kevin, Justin, and Rachel.
And Unity, Unity, HaxeFlixel.
Ooh, it's going to be kind of tough.
We've got three Phaser, two Unity, and one HaxeFlixel, so that's going to be a hard decision to make.
DNA, we've got seven-- Jenny, Lauren, Ava, Tricia, Harry, Eduardo, Derek.
HaxeFlixel, HaxeFlixel, HaxeFlixel, Flixel, and one Unity.
So probably one of the Flixels.
[GROANING] You decide.
MIT Simulator has Unity, Unity, Unity, Unity, Unity, Unity, Unity.
Four Unitys and two Phasers, so probably Unity.
And that is Norman, Jordan, James, Peter, Sam, and [?
Szeun.
?]
We have how many students are not in class today?
Five or six students not in class today.
They are going to have to-- what's up?
Make them a team
Game to learn how not to be late
That's a good question
[INAUDIBLE] simultaneously on Wednesday, that might be possible.
But if a few of them don't come back on Wednesday, that may not work
So teams will be finalized on Wednesday.
These are the teams as they stand right now.
You have 40 minutes to meet as a team to figure out if you all match schedules, to talk about the idea you just pitched, and make a really quick lo-fi prototype, or write down that page that you just said out loud on the spur of the moment.
Figure out what is that game that you're making.
Work on that vision statement.
And feel free to grab your things and put your names on it if you like.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So I am really happy to say that we've got SWERY.
Do you go with SWERY or Swery65?
SWERY please
SWERY.
Swery65's Twitter.
Your Twitter account
Yes
So a developer.
He's visiting us from Japan, visiting for the Video Game Orchestra, which Shota-- is he in the room right now?
I'm here
There he is-- is organizing that orchestra.
They're playing tomorrow.
That's right?
Yeah.
So we sent you the code for that.
So please, if you're interested in attending, please do attend that.
It'll be really fun.
They'll be playing some of the music from SWERY's most recent game.
SWERY's from Access Games?
Yes
And previous games he's made include Spy Fiction, Deadly Premonition, and the Deadly Premonition director's cut.
And a few others
To Japan.
Ace Combat.
Or [?
Ganda.
?]
Something
So from Namco Bandai.
Those were Namco games.
Yeah.
So he's made a bunch of different types of games.
We really like, at the lab, we really enjoy the Deadly Premoniton game.
It's a great open-world simulation game that it plays a lot differently than if you're used to the normal Grand Theft Auto.
So if you haven't played that, Deadly Premonition, he'll be talking about some of those games today.
But definitely check out YouTube, and check out playing that.
We have it at the lab.
So if you'd like to use the lab and play that game at all, you're more than welcome to.
They're going to also be demonstrating their most recent game on the Kinect once we get our tech issues fixed.
So we'll start with lecture.
We'll have some Q&A at the end of that, and hopefully we'll have a chance to play their latest game.
So I'm going to give you this
You can just clip it anywhere?
Yeah, just clip it like-- near the top of your-- yeah

And feel free to hold it if you don't have anywhere else to put it

Hi.
Nice to meet you guys.
I can't speak English well.
Sorry.
I need a translator
Hi
[SPEAKING JAPANESE]
I'll be speaking in Japanese.
So I will translate
[SPEAKING JAPANESE]
As the teacher has said, that he makes games.
So in order to accommodate everyone who might not know the games, we've presented this picture over here.
Can you hear me OK?
Is this good?
[MUSIC PLAYING]
[SPEAKING JAPANESE]
These are all the games that he's worked on previously
[SPEAKING JAPANESE]
You're familiar with these games, here?
You know, some were only released in Japanese market, also
[SPEAKING JAPANESE] [APPLAUSE]
We've been making games for about 20 years, just like these games up here.
And today, we'd like to talk about the very newest game using the Xbox Kinect in the front there
[SPEAKING JAPANESE]
So this is basically the structure of today.
We want to start with a small lecture.
If we can get the video going--
They're here to set that up right now
Yay!
We can play it, and then have a short Q&A at the end
Yep.
[SPEAKING JAPANESE]
So for some of you who might not know, D4 is a game that was designed for Xbox One Kinect, so that you can play using your motions from the very beginning all the way to the very end in an action-adventure kind of story
[SPEAKING JAPANESE]
So this is a type of game that you actually don't have to hold the control in your hand.
Everything is hand-motion based.
So it has this sensor with the Xbox One Kinect bar
[SPEAKING JAPANESE]
So the Kinect, it actually takes a skeletal image of you.
And then, using that, it allows for it to detect the motion of the person who's playing the game
[SPEAKING JAPANESE]
It's kind of like an all-in-one kind of sensor device.
It doesn't only have just the body movement sensor, but also has a microphone, a camera, all the different kinds of things.
So it could accurately capture what you're doing
[SPEAKING JAPANESE]
So in one rough sketch, this is pretty much all the sensors that are incorporated into the Xbox Kinect sensor.
As you can see, there's a hand-tracking one with the three different options there.
Face-tracking, mic sensors-- all of this is inside that little
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
We don't actually know what it's used for, but there's also a glasses tracking device sensor
Doesn't make sense.
[SPEAKING JAPANESE]
So in order to come to this point, we've had a lot of different input methods that we used
[SPEAKING JAPANESE]
So this is, basically, all the different devices, from the very beginning all the way to 2014.
So all these devices have been previously made up to now
[SPEAKING JAPANESE]
So basically, at the very beginning, if you look over there, there's games that only have a couple buttons.
And then we've progressed all the way over here.
Do you know Femicon?
It's on the corner there, too
[SPEAKING JAPANESE]
Yeah, in the very beginning of the 90's, there's also this rifle controller thing down here.
That's all very interesting
[SPEAKING JAPANESE]
He understands how much we all love shooting games
[SPEAKING JAPANESE]
And then, if you look at the next decade over, or the next generation, you notice that there's actually controllers device for each specific game.
So we have the new gun controllers over here, and then the actual handheld controller over here, specific to whatever game that you are designing for
[SPEAKING JAPANESE]
And then in this era, you can see that they also utilize a piano keyboard up top.
So you use all different kinds of medians in order to play these games
[SPEAKING JAPANESE]
And then, in the 2000 onwards, you noticed that we actually have these kind of controllers.
Basically, everything has some kind of screen panel attached to it.
And then you have guitars and drum sets, and things that we see nowadays in gaming
[SPEAKING JAPANESE]
That's Kinect
[SPEAKING JAPANESE]
So the Kinect is over here underneath the Wii.
So perhaps it's like whoever made the slides preferences
[SPEAKING JAPANESE]
So probably whenever we game nowadays, we have something similar to this, where you're holding a handheld controller
[SPEAKING JAPANESE]
Also, the other is like when you use actual computer-- PC games.
And you have a mouse and a keyboard to play with
[SPEAKING JAPANESE]
And then also nowadays, a particular style is to use these kind of things, like a pad over there, or a guitar to play the games nowadays
[SPEAKING JAPANESE]
So now we have a question about Kinect.
Is it one of those games that you have something like a guitar in your hand and you're playing, or is it something controller-based in your hand?
What do you think?
[SPEAKING JAPANESE]
Anyone that thinks that it's like a gamepad device?
[SPEAKING JAPANESE]
Something that's very specific and different.
Anyone?
So there's two sides here.
We have a couple hands
[SPEAKING JAPANESE]
Some brave souls
[SPEAKING JAPANESE]
In his opinion, he thinks that Xbox One is traditional.
It's more traditional, and not special.
It doesn't have those different elements to it
[SPEAKING JAPANESE]
So in his opinion, there's two distinct differences between a traditional game and a not-so-traditional game.
The not-so-traditional game would be something that does not-- you're not designing the game based on the medium that you're using.
And then the other way around, if it's a nontraditional device, it is based-- sorry.
I got that wrong.
Sorry.
A traditional device would be a game that you play based off of the medium that you're using to play the game.
And a nontraditional device is not dependent upon the medium that you're using, but something else
[SPEAKING JAPANESE]
He feels that a special device-- a special game would be something like, let's say then Guitar Hero, when you pick up a guitar, and you have a guitar to play with, you automatically know that this is a music game.
It's not going to be a fighting game
[SPEAKING JAPANESE]
We're going to explain a little bit about why the Kinect is considered a traditional game
[SPEAKING JAPANESE]
So a game is made by having the players input a code through a different medium into the game itself
[SPEAKING JAPANESE]
So the symbols-- like pushing this button will fire this missile.
Kind of shorthand symbol
[SPEAKING JAPANESE]
So then in D4, instead of actually having a motion mean something, it is more like you follow the arrows.
So if you wanted to turn the page, you follow the arrow, and it will go around.
So it's like a symbol based
[SPEAKING JAPANESE]
So there's also, like Minority Report and Iron Man that came out, it's not necessarily a game that you use because it's based off of this Kinect, but kind of like a halfway semi-using the Apple products and the OS, iOSes in order to make the games
[SPEAKING JAPANESE]
So if you kind of really focus into whether you're going to make a special game or a traditional game, and you narrow your vision, your game is probably going to become kind of small.
In his opinion, he thinks that in order to make it a bigger deal, and make it a bigger thing, you should break that boundary between traditional and special, and find something that works really well with the medium that you're using
[SPEAKING JAPANESE]
So with D4, what he wanted to challenge was not only the actual game play itself, but how the players interact with the device in order to make actions happen
[SPEAKING JAPANESE]
So instead of having the user have a handheld controller to click on buttons, he wanted to find a way that you can actually have the people interact with the game by having these hand gestures, or motioning a bat, and having a story go along
[SPEAKING JAPANESE]
Are you interested?
[SPEAKING JAPANESE]
Do you have a preference?
Does anyone like Kinect, particularly?
Hate Kinect?
Have an opinion about Kinect?
It's good?
I saw the-- over there
[SPEAKING JAPANESE]
So now that we've piqued your interest a little bit, I think that some people are probably thinking, yes, let's incorporate motions into our game.
But actually there's a bunch of little problems that come up when you want to use motion
[SPEAKING JAPANESE]
The Kinect is, as you see, stupidly honest.
It only registers the exact motion that you are doing.
So you have to be precise when you play this
[SPEAKING JAPANESE] [LAUGHTER]
There's a function-- especially like in D4, where if you take from an outside hand motion, and you swipe to the left, the camera will also follow you.
Your vision will change as well
[SPEAKING JAPANESE]
So let's say if I was going to swipe from my right to my left, the camera will move this way.
And if you move from your left to the right, the camera will move that way.
It didn't really work out.
Have you tried to do it?
[SPEAKING JAPANESE]
So I know some people probably have this, if you're a gamer-- I know I've experienced this.
But if you try to use one motion.
Like, OK, I'm going to slide to the left.
That didn't work.
Let me try it again.
When you try it again, this second sensor also catches.
So then your computer's just moving around like crazy.
Instead of having a one reset, it registers this backwards motion as well
[SPEAKING JAPANESE]
So in order to try to compensate, or to change this method, he tried to make it so that if you swipe really fast, the camera will react.
But if you swipe really slow, it will not react
[SPEAKING JAPANESE]
But at first, he thought this was a brilliant plan.
But when you're really hyped up and energized about playing a game, adrenaline pumps in, and then everything becomes really fast.
Or if you have maybe an older person who maybe cannot move their hand quite so fast.
It doesn't react
[SPEAKING JAPANESE]
So then he thought, OK, let's try having a little bit of time.
So once you move the camera angle, if you don't do anything you'll be OK.
It won't move
[SPEAKING JAPANESE]
So it didn't work, because this speed limitation was also one thing.
But once it-- [SPEAKING JAPANESE]
[SPEAKING JAPANESE]
And then, even if you do something, and then you're like, OK, I'm going to wait two seconds before I do to my next action, it actually just goes back to what it was originally.
It reverts back
[SPEAKING JAPANESE]
So just from this one swiping action, he's changed it this many times
[SPEAKING JAPANESE]
So in the game D4, in order to select something, like back or next, you actually have to grab the back or next.
There's a hand that hovers, and then he makes this grabbing function, and it pushes "Enter" pretty much.
But then, in order to differentiate between the swipe and the grab-- or some people do a combination of a swipe and a grab at the same time.
So it didn't work per se.
And so even if you're doing one small action, like swiping, it caused this much different trial and errors that came up
[SPEAKING JAPANESE]
So what he wants to say is that even though there might be some negative views about Kinect, it's just basically maybe certain developers, directors can't design well for the game.
However, these things could possibly easily be overcome with a software change
[SPEAKING JAPANESE]
And so if you try to find a game designer who doesn't think about changing the software, and actually only concentrates on the hardware version of it, then they're going to have to make a game specifically designed towards the hardware, instead of having the simple solution of let's just change something else to make it better
[SPEAKING JAPANESE]
So instead of how to make a good game, instead of thinking, "How do I use the hardware that they've presented me?"
well, using that hardware incorporating a different software trying to find a different approach to make the hardware itself more stand out is what he wants to push
[SPEAKING JAPANESE]
Is it?
We're going to try
Oh.
OK.
If this works, we can play D4
[SPEAKING JAPANESE]
Does anyone want to play D4?
Oh.
Oh, oh yeah.
You all have to raise your hand at the Kinect, and it'll just register everyone
[SPEAKING JAPANESE]
Anyone whose name starts with D?
First name.
Like David, or Diana, or Danielle?
Nobody?
No one?
Really?
[SPEAKING JAPANESE]
Last name?
Yes
Yay!
You can play.
Please.
[SPEAKING JAPANESE]
If it works.
Don't move too quickly.
Walk very slow.
Has anyone played D4?
No?
Not yet?
I was supposed to go to a meeting yesterday, and instead I sat at my house for six hours on my D4.
It's really addicting
[SPEAKING JAPANESE]
So maybe, as you said, no one here has really played this game.
But once you play it, you'll know that this is not your typical game.
It's a little bit different
[SPEAKING JAPANESE]
But if this doesn't work, it's affordable.
It's only $15.
Yay!
[SPEAKING JAPANESE] Thank you for PR!
[SPEAKING JAPANESE] Too dark.
[SPEAKING JAPANESE]
Would it be possible if this doesn't work to just do a Q&A now, and then maybe set this up in a different room and have people play it?
There's no other room that has HDMI, especially a room that's big enough for the class
I see
What I could do is-- it wouldn't work for today, of course, but we can do games in our game room
[SPEAKING JAPANESE]
Ah, it's supposed to look like that?
Yeah.
It has to look like that
So it's the white.
There's a whole layer of color that's not being shown
Right.
So if it looks fine on that screen, and then it doesn't look fine on another screen, then I can't say that it's the connection that's the problem
Yeah.
So it could just be the--
It's possible it could be the HDMI cable as well.
But there is two things that I could suggest.
We could actually [INAUDIBLE]
Just walk really slowly.
[INTERPOSING VOICES]
And then after-- [INAUDIBLE]
We'll be back in 5, 10 minutes
5, 10 minutes?
Great.
Cool
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
OK.
It was a really fun game.
Yeah!
So we're actually going to try to do this one more time, but we have about 5 or 10 minutes while they figure out the technical aspects.
But if anyone has any questions, we can entertain them right now
So you mentioned adjustments to the software that you made to the Kinect firmware.
Is that something that-- is it very accessible?
Is it easy to change the software that it comes with?
I would think Microsoft wouldn't want people messing with-- and they muck it up or something
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
So it's actually pretty easy to change, apparently, the software part.
But in order to think about how it could be better-- like why it's buggy, or what could be better-- what could make it smoother, in order to even make D4, it took three years.
So coming up with all the different things that could possibly go wrong is more of an issue
Yeah.
You mentioned the problem with the camera.
How did you finally fix it?
Which?
Sorry?
He mentioned the problem with the camera.
How did he finally fix it?
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
So basically, it's just a bazillion trials and errors, asking people-- like any users, hey, let's play Testament a bazillion times over, and just trying to somehow come up with the solution
[SPEAKING JAPANESE]
So actually, with the Kinect, it registers everyone's height and weight kind of things.
So if you go to Japan, where everyone tends to be a little bit vertically smaller, as opposed to if you go to America, Europe, and the people tend to be a little bit bigger.
So he has to go to all these different countries and have play tests done at those specific countries in order to adjust for physical problems
You mentioned the input device affecting how you design your game.
What would you do if you had a brain-computer interface?
You would laugh, apparently
[SPEAKING JAPANESE]
So he actually wants to challenge it sometime.
But even with the motion sensors, you have so many problems that are going on.
So as a game designer, if you were to use the brain sensor ones, you'd have to even think of one step even ahead of the player in order to get it to go.
So it'd probably be very difficult
[SPEAKING JAPANESE]
But even in D4, when we have even like an arrow, people, humans, we tend to automatically follow the arrow anyways.
So if there's just an arrow and no directions, you just kind of follow the arrow.
So perhaps there is a way to actually have it so that we can do the brain sensors, because of existing motor recognition
You mentioned that you have this more generalized input devices and specific input devices.
What's your opinion on that?
Even for the generalized input devices, some of them are better for some games and not others.
For example, keyboard and mouse works well for strategy, but you can't really do StarCraft II with a standard controller
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
So his opinion is it doesn't really change based on having one set-- the design for what you want to make.
But as a designer, you definitely should think about which one is going to work better.
As you said, StarCraft would not really work so well with a handheld controller.
And so that's kind of up to the designer to decide whichever is better
[SPEAKING JAPANESE]
So we're not trying to be like, "Go play Kinect, everyone!"
But it's more of instead of having this is one option, by using the Kinect, how can we make it so that the user actually feels more like they are playing in the game, actually, controlling the character, and having different options that way?
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
You?
So I found it really interesting how you talked about the differences between the different kinds of controllers, and what you just said about having to think carefully about which controls do you use.
Is there something about the Kinect's motion detection that you think really contributed to the feel or the atmosphere of the game?
And then you talked a little bit about immersion, but something maybe a little more specific than that-- something that you feel like, because I used the Kinect, I really got to do something specifically cool?
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
So with the D4 game, let's say if you wanted to use a mouse, and you can actually follow the mouse with the arrows, or use the control of the D-pad and follow around with the arrows.
But with Kinect, what is really cool is that you can actually move your hand and click on something, or do an action based off of Kinect functions
[SPEAKING JAPANESE]
So let's say by using the Kinect, there is-- it's kind of easier if you have played this before-- as with everything.
But there are certain things that happen-- like real-time, different events that are triggered because of another thing that you have done.
And with Kinect, it's actually smoother.
Like your emotions, if you feel really on edge about something, it kind of reflects in your body movements, and then it catches that.
So with the Kinect, there's a little bit more in your personal-- like you're in the game kind of thing, as opposed to a controller
[SPEAKING JAPANESE]
And then, so if you had a controller in your hand, or a mouse or a keyboard, the only way to make something harder-- to raise difficulty-- is to increase the speed of your tapping, or something has to-- that kind of thing.
There might be other ways.
But for the most part, we increase the speed.
However, with the Kinect you actually physically have to emotionally gut yourself into it in order for it to increase
[SPEAKING JAPANESE]
Oh
You?
Yeah.
In my experience with these kind of things where you have no direct control over what several users may have-- like you talked about the grandma and speed up.
So, in my experience, the way you do it is you make sure that that is a variable.
And I'm asking, why is it not possible here?
So, for example, before the game starts, you ask the grandma to register how fast she can do this with the camera
Oh.
Like a Windows double-click speed testing?
Yeah, exactly.
So you set up all those things as variables.
And when the game starts the grandma can go at her pace, the young guy can go at his pace
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE] You can in D4.
In D4 you can
[SPEAKING JAPANESE]
But they still took an average, pretty much.
So most people would be able to still play it without having to finagle though all the settings
[SPEAKING JAPANESE] Any other question?
What's the cost for an average game that, say, like cost, like D4?
What's the cost in terms of capital, number of people you need-- developers, artists, whatever?
You want a very large number, basically
How long would it take to develop a game?
[SPEAKING JAPANESE]
Oh
Many zeroes
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
So it's kind of difficult to put a number to how much it costs.
This is kind of hard for me explain, so bear with me a moment.
But if you were to say how many people were to work on this game, and then kind of use that as a monetary equation, in one month if you were going to make this game, you'd probably have about 600 peoples' worth.
Does that make sense?
So if you only had 20 people, and 20 people worked on one game for one month, it would maybe take 30 months to finish this game kind of thing.
Does that make sense?
[SPEAKING JAPANESE]
So D4 is still a smaller-- it's not a huge, giant, giant game.
So if you go to maybe PlayStation, like a big company game, it might be 300 people, one month, for 30 months kind of thing.
So it kind of depends.
And it's a roundabout, not really a good answer.
But--
[SPEAKING JAPANESE]
Yes!
It's working!
Finally!
[SPEAKING JAPANESE]
Yay!
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
Color.
Can everyone see?
If you can't, you could consolidate, probably
[SPEAKING JAPANESE]
Binoculars
[SPEAKING JAPANESE]
So you have to move your hand, and pick and grab.
Yeah
[SPEAKING JAPANESE]
Yes.
The action is good
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
There might be some graphic morals, or graphic language.
I hope we're all
[SPEAKING JAPANESE]
Get out here now!
[PUNCH] [PUNCHES AND GRUNTS]
[INAUDIBLE].
Set your ass on your head!
No more messing with me, Papi!
[GRUNTS]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
Bad.
Real bad!
[INAUDIBLE] I said.
You're never, ever, ever gonna [INAUDIBLE]
That's the main character
There you are.
[INAUDIBLE] Hey, Papi!
Ha-ha-ha-ha!
So how about--
Action
Oh.
And this is the action.
You have to move your right hand down.
There.
[SOUNDS OF FIGHT] [LAUGHTER] Yeah.
Things just got serious!
[LAUGHTER] [INAUDIBLE], I'm walking this one down to the park.
[LAUGHTER] [APPLAUSE]
[SPEAKING JAPANESE]
So it wasn't used here, but there's actually moments where you scream into the mic-- the sensor in the microphone-- to catch it.
Yeah.
You'll have a little command on the side that says, "Scream."
And you're like "Ah!"
And it goes whoop.
Like, OK!
[SPEAKING JAPANESE]
So we're kind of running low on time.
So he's going to do the adventure one
[SPEAKING JAPANESE]
Thank you for laughing
[SPEAKING JAPANESE]
The loading time, however, is something we cannot overcome
[SPEAKING JAPANESE]
So this adventure scene, we're just going to show you how to investigate the rooms themselves
[SPEAKING JAPANESE]
[SPEAKING JAPANESE]
[SPEAKING JAPANESE] [MUSIC PLAYING]
[SPEAKING JAPANESE]
So he has a special ability to go back in time.
And just now, when he fell, he just came back from back in time
[SPEAKING JAPANESE]
So by using Kinect, you can interact with the characters by using these hand signals
[SPEAKING JAPANESE]
And then there's an exclamation point.
If you hover your hand over it, you'll be able to see important information
[SPEAKING JAPANESE]
He's a detective.
Basically, his wife was killed two years ago, and he's looking for the true murderer of the case
My name is David Young.
Former narc with the Boston
It takes place in Boston, too!
My likes include 100% agave tequila.
My dislikes are mainly drugs and chewing gum.
Two years ago someone killed my wife.
Since then I've been using every second of my life to solve the case
My name!
That's him!
[SPEAKING JAPANESE]
Just by moving your hand around, you can pick which one you actually want to interact with
[SPEAKING JAPANESE]
And then, if you clasp your hand together, it's enter
[SPEAKING JAPANESE]
And if you follow the movement of the arrow, it moves the character's hand
It's been broken since that day
[SPEAKING JAPANESE]
You can use both hands
[SPEAKING JAPANESE]
If you wanted to change the camera angle, you put your hand all the way to the side and swipe across
[SPEAKING JAPANESE]
And then if you tried to swipe it twice, it'd be back-forth-back-forth-back-forth instead of having this basically holding it and sliding over
[SPEAKING JAPANESE]
Of course, you're not actually shutting something physical.
But by using the arrows, you kind of have the feeling that you are shutting the toilet seat down
In a slightly different form.
Like all these forgotten treasures, mementos
[SPEAKING JAPANESE]
OK.
Put your hands up, it closes the screen
[SPEAKING JAPANESE]
We're going to skip this serious part and just go into action
No results this time either.
This case is in the clear
[SPEAKING JAPANESE]
So there's a lot of things in the rooms that you can interact with.
And if you're not sure which one you should do, if you put your hand like this, it activates a thing called Vision, which is actually voice-commanded also.
You can say "Vision on," and it will turn on.
And it shows you the different options of what you could pick up or touch
[SPEAKING JAPANESE]
So usually with the controller, you use like Y button or something to enter something.
But with the Kinect, by using an actual hand gesture you can connect with that even better, the characters
There's only one thing I need at the moment.
Tequila.
Straight up
Me too
Him, too
[SPEAKING JAPANESE]
If you had to do a gesture every few seconds, it might get kind of annoying.
But as you see, it's like spaced out.
It's only occasionally you do these gestures
[SPEAKING JAPANESE]
This is his dead wife
Maybe I'll just try a little.
Oh!
[LAUGHING]
[SPEAKING JAPANESE]
Yeah.
Everything he does is very violent
I have no memories of that day.
When I came to, I was already lying in the ICU.
The only thing I do remember are the words that little Peggy whispered as she died.
"Look for 'D.'"
Who is "D"?
Which we tried to do.
But no one's name started with "D." So sad!
At the time, there's no compelling evidence.
[INAUDIBLE]
[SPEAKING JAPANESE]
Such a depressing story
Under the right circumstances, I now have the capability to solve even a dead-end case
He even has an accent, though
I'll do everything in my power to find this "D." I swear I will.
And when I do, oh, Peggy.
Amanda?
[SPEAKING JAPANESE]
So we're going to investigate this door.
Huh!
[MEOW] [MEOWING]
[SPEAKING JAPANESE]
And then he's going to [INAUDIBLE]
You can't escape now.
[MEOW] [MEOWING] This is Amanda.
She just suddenly started--
And you can move your body and change the camera angle
Sometimes goes out and gets food for us.
And that's something of a lifeline for me, because I don't really go outside.
Just who she is, though-- well, my memory holds no answers
[SPEAKING JAPANESE]
That's the end
Thank you.
[APPLAUSE]
Again, the game does take place in Boston, so it's kind of adorable.
BPD comes out
[SPEAKING JAPANESE]
So if you can get a chance to maybe play around with it sometime, it would be wonderful.
If you can experiment around with what we talked about today, and see how the interaction will change your game-play.
Thank you very much.
[APPLAUSE] [CROWD NOISE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So before class today, you should have turned in your vision statement and your product backlogs.
Wednesday, you should be turning in a copy of the sprint task list, and you should have a prototype ready for testing in class.
Remember that while we're offering you one in-class testing session for this project, you need to do at least one more testing session out of class for the two focus tests that are required for project three.
And when we look over your focus test reports, we're not going to be impressed with focus tests that were done so late you clearly couldn't respond to them.
So the project is due Wednesday.
Don't do a focus test on Tuesday, because we know you couldn't do anything with that information.
So sort of plan out when you need a focus test accordingly.
So today is class, we're going to open with just a very quick two minute presentation from each of the groups.
Since you've turned in your product backlogs, and hopefully you have finalized your vision statement, you can come and tell us, what is the goal of the game you're now going to make and what its major features are.
That's all you need to do for that.
Then we'll have a fairly brief project management check-in.
We'll sit down, I'll stand up here, and we'll just do a little bit of talking about what is your team doing for project management.
Is it working?
Should you change something?
Then Philip will be giving the usability lecture.
And after that, we'll take a break.
If anybody has something that they would like play-tested, after the break, we'll come back and have an opportunity for some play-testing and working on your projects in class.
If you don't have something to play test, that's OK.
This is sort of a hey, if you actually had your act together, get a little extra play-testing in.
Wednesday's the day you have to have something that can be play-tested.
Any questions?
All right, then.
Let's move on to presentations.
Actually up there?
Yay.
It is.
First up we have Hardcore Dragon.
PRESENTER 1: OK, so I'm on the team for Hardcore Dragon.
We've changed from Dragon's Lair to Hardcore Dragon.
And the point of the change in the name is to reflect the slight change in the direction of the game.
One of the new elements we've added as a win condition is collecting an entire set of hardcore armor to be the hardcore-est dragon in the land.
And so we're intending on adding a little bit of a cheeky, funny, dialogue flair to the game to make it a little less serious.
Outside of that, the main concept of our game is that you're a dragon, and you have a horde of gold that you are protecting and also trying to grow.
So the player can make decisions between attacking villagers to gain more gold and also staying on their lair to be protected.
And we're also adding elements of melee combat so that the player is not only making these decisions cognitively but also has to act on them and actually go and perform those actions, rather than something that's, like, just click, oh, I want to do this action, something like that.
So we're introducing elements of strategy and melee combat and also collectible funness of RPGs.
And then elements of our backlog-- we have the player's desire, such as the playing wanting to be able to attack, defend, and also stay alive to fight, despite the attacking armies.
And we also have a lot of focus on making our game usable and easy to understand for the player
Thank you.
DNA.
PRESENTER 2: Hi.
So probably should have written on the board the project previously known as DNA, because we've completely reskinned our game.
So it's no longer about a biology type thing.
It's about exploring space.
So DNA is now the name of your mothership, And we're going to put you as a small spaceship-- RINA, so R-I-N-A, basically.
RNA.
And your goal is to travel across different planets and eventually make it back to your mothership before running out of fuel or exploding on a bunch of asteroids.
So our game has two primary mechanics that we're working on.
First is a menu-type screen where you can see different planets that you can head towards on your way back to your mothership.
And each planet would tell you-- if you hover over each planet, it will tell you, at this planet you can pick up X fuel, or you can get this jet boost type thing.
And so the strategy comes from the planning for which planet to go to relative to the other planets.
And the trade-offs also come with which planet do you go to, or what do you pick up at each planet.
And the other mechanic is actually getting to the planet.
So using a stat boost and/or fuel that you picked up from each planet, you have to navigate through a platform, or not really a platform.
More like Flappy Bird or a Copter type level, where your stats increase, like how fast you go through the level or how maneuverable you are and how you can avoid obstacles.
So our game has two mechanics, and that's what we're kind of working on right now.
We have the moving from one level to the next motion part down, so now we're working mainly on balancing the game and seeing what should the stat caps be, or what should the starting caps be, so it's not too boring at the beginning, but it's also not too hard at the very end.
And our backlog has a lot of art
Thank you.
Ghost Maze.
PRESENTER 3: So for Ghost Maze, our design for the game hasn't changed much.
We solidified our original idea, is that you're going to start off in the middle of a dark maze and you have very limited visibility.
You're going to explore the graph, or the maze, and basically find the exit that leads you out of this dark scary place.
You have a sanity bar that goes down the more you are in the dark.
And you have the ability to increase the brightness of your lantern, which would give you a higher radius of visibility or decrease it, which would make it lower.
But the higher it is, the less slowly your sanity goes out, but the faster that your battery dies.
So you want to optimize for that-- how fast can I explore this maze versus how slowly do I need my sanity to drop.
And you can completely have it off if you want, because they're battery.
But that would be chipping away from your sanity much faster.
There's ghosts in the maze, so you can't just do the bug around strategy, which would always guarantee you to get out of a maze.
But there's going to be ghosts at strategic, random positions.
These ghosts were originally intended to have AI that would have them find you or attack you or whatever.
But we changed it so that they're just patrolling in a predetermined pattern so that you can focus on the one mechanic of the game, which is moving around and finding a good path, as opposed to escaping from things chasing you.
The one change we made, the biggest change we made so far is we switched from Flixel to Unity 2D because we couldn't get Flixel running on most of the computers we had, and it just brought so many complications.
So we ended up switching to Unity 2
Thank you.
Score High.
PRESENTER 4: So Score High focused on simplifying our game.
We realized we were probably overextending ourselves with the number of features we wanted.
So even though our goal stayed the same, we kind of simplified it, and I'll talk about that in a minute.
So Score High's goal is to survive the week of school.
If you can do well, even better.
And that becomes difficult because as a player, you have your stats to maintain.
You need sleep, and you need food.
But you also need to not fail your classes and get kicked out, so you need to do your work.
So the main features of the game are you play your stats, so how are you doing as a person, your schedule, which is going to list your classes and what PSETs are due and when they're due, and then also the map.
You, as a character, go around the map, and you have to go to certain buildings to do your work.
And as I said, the trade-offs are managing you internally, so you have to go to your dorm to sleep, but your PSET might be due in two minutes, and if you don't run across campus, will you be able to get it done and pass your class?
In terms of updates, I think that's about it.
Thank you
Thank you.
Build a Space Car.
PRESENTER 5: So originally we were Build a Car, but we decided that the game would be better in space, so now it's Build a Space Car.
The idea is that you have some kind of spaceship that you need to add components to, and then you send it out on a mission.
Out on this mission, it'll run into random events, and you'll get a report of what happened.
And then it'll come back and you need to iterate, add more components to either make it faster or stronger or have more attack.
And then that way, you'll be able to succeed on missions.
The trade-offs come into what components do you add, and what effect does that have on the stats of your spaceship, and the fact that the more components you add to your spaceship, the more it costs and the less money you're going to make from going out on that mission.
The main product backlog features that we have are the things like adding components to ships, coming up with stats for all of the components that we have in the game, and designing a system where the ship will go out on missions, get some results from all of that, and then loop back into this initial building phase
Thank you.
And MIT Sim.
PRESENTER 6: All right.
So the idea of our game is we play as an unnamed dean for student life at MIT.
And you make and manage the decisions at MIT, and you have to trade-off all of these various resources that you have, like student happiness and the endowment and whatnot.
And you have to please the MIT president above you and the MIT corporation above them.
And then you also have to please the parents of students who might hear of some crazy things happening and you have to respond to them.
So for example, you might make shoes mandatory on campus.
And suddenly East Campus revolts.
But maybe it improves your standing with the parents or something.
So that's the general gist for our game, yeah
OK.
So it's good to hear that people are both refining and changing and scoping down and kind of figuring out what they're doing with their games.
That's a good start.
The next thing I wanted to talk about, ever so briefly, is-- I've been reading through the project write-ups from the last project.
Haven't gotten all the way through them.
None of us have.
We're still working through.
But a lot of people are like, ugh, project management got inserted into the middle of the project, and it was a really hard.
And it made things clunky and more painful.
And yeah, I bet it did, and I apologize.
What I'm wondering, however, is what are you doing for project management on this project, and is it working better So far?
I realize we're only partway through, but I'd kind of like to hear about how has your team decided to handle communication.
I know that meetings were a big problem last time.
Is the product backlog being a useful thing to sort of help define your design?
Or are you doing it because we're making you turn one in?
How are assigning tasks and sharing work going?
So I can either do this as a discussion, or I can call teams one by one.
Do you guys have an opinion as to how you'd like to run it, or should I go ahead and make an executive decision, since I'm up here?
All right, I'll call on people team by team, and we can just sort of go through.
I'll start with MIT Sim, because you went last last time, so I guess you can go first this time.
You don't have to come down here.
You can stay in the seats.
But OK. OK, a microphone might help.
I wasn't trying to make you do an extra report.
I was hoping this would be sort of a class discussion where you're-- PRESENTER 7: Well, I'm already down here
All right, go ahead.
PRESENTER 7: So I think one interesting thing about project management for our team as 611 teams is that the driving difficulties of working on a team like here are very different from what might be in the industry.
So we have to adapt all these industry methods, which might work really well for a game company, where you have your eight hours a day of working.
But here it's much more sparse and not necessarily-- it's not always one to one how you would manage an industry project versus this project.
So I think just figuring out how to adapt these methods for students who are really crunched for time is a challenge
Have you made any adaptations?
Has your team made any adaptations or changes, or are you just trying to do as little of it as possible to help keep things moving?
PRESENTER 7: Probably the latter, yeah
OK.
If you can think about the adaptations you're making that are working for your team and share them, that would actually be really helpful for us to hear, and for actually everyone to hear, because I think every team is running into the same problems you are.
But the questions is, how do you take these, and how do you make it work with a team you actually have and a reality you're actually working with, as opposed to, this is the way your should do it.
That's actually the really important thing here to try and figure out.
Thank you.
OK, Build a Space Car.
Do you have a sacrificial victim?
Scrums masters are a good person.
Yeah, it could be a scrum master.
Could be either way.
PRESENTER 5: Our producer slash scrum master is actually out of town, so I guess I'll handle it.
But basically coming off of the last project, we saw a lot of things that we weren't happy about and that we saw a lot of problems with the way that we communicated.
So this time we're trying to make sure to stay in communication as much as possible, let people know what we've been working on and the problems that we see
What tools are you using to stay in communication?
PRESENTER 5: So we're using Trello for all of our tasks.
We have them there.
We have them color-coded for different groups of coding or UI or testing, things like that.
And we have a Google Doc that's sort of all of the ideas that we had and sort of the design decisions and things like that.
We're also all programmers, which I think has been helpful so far, in that we sort of all understand what goes into the task and what's reasonable and things like that.
Not to say that there isn't a place for non-coding people, just that it has definitely helped on such a time-constrained team
Yeah.
When you're all talking the same language, and you all come to the same estimate assumptions quickly, that's beautiful.
Thank you.
Score High.
PRESENTER 4: So I think one thing we realized right away was that meeting outside of class was pretty much impossible, especially when you have seven people with completely different schedules.
So having the product backlog and that list of things was really helpful.
But at the same time, a lot of the other project management stuff is more just on the side.
So for my team, I've been taking care of it and trying to let everybody else program.
But the product backlog, I think, is really helpful.
And the vision statement, having that breakdown of who's doing what, has been pretty useful.
We use an email thread, just to keep everyone in contact
All right, Ghost Maze.
Two people can come down.
That's fine.
PRESENTER 8: Yeah.
So I guess as other people were saying, it's been hard to coordinate meetings in person.
We tried to have one on Saturday, but only half of us could get together.
So definitely we have to rely a little more on online communication and such.
And people are working on different schedules.
So I think one thing we realized is you have to budget how much time people have at certain points in the project.
So if someone has a lot more time, maybe they're more free on the weekend, then they could get started on the earlier stuff, whereas if someone has a big project due today, then we would give them more stuff for later in the day.
Because when you develop a game, you can't do everything right away.
Some other things are dependent on other things.
I've been taking the role of scrum master, project manager.
So figuring out delegations and responsibilities.
So I think it helps to have one or two people plan that
Thank you very much.
OK, and DNA.
In space.
PRESENTER 9: That actually was the other working title.
It might become Finding
Pretty much, whenever you tack "In Space" to something, it becomes infinitely better.
PRESENTER 9: I'll do that now then.
So meetings are hard because people have lives.
But we do little meetings where we eat lunch with each other, and only two or three of us is there.
And the lunch is tasty, and we talk about cool stuff like game design and weird bugs and what exactly are we doing with our lives?
As you may be able to tell from the fact that things have changed drastically from the pitch to now, we kind of basically scrapped everything at one point and said, re-brainstorm.
Shoot out all of the ideas.
But of course, since we already said the word DNA, everything kind of ended up DNA deep down inside at heart, kind of sort of.
So I feel like we did a lot of email communication then, and we still do a lot of email communication now.
It's just like, spew all of the things at each other.
And I feel like, that's kind of nice.
There are times when it's like, well, I'm really, really, really busy until Day X, so until Day X, I will have nothing to say if we have a meeting.
And we're like, OK, then I guess we won't have a meeting until Day X.
And it's like, well, Day X is the day right before Monday or something.
So was it really the best idea to put that off, or should we have done it anyways and have you just sit there and feel lame about yourself?
I don't know, but we did what we did
OK. And finally, Hardcore Dragon.
PRESENTER 10: So looking up at that list, the first thing on the list is team communication.
And we actually have quite a few different things in place.
We're using Trello, so we do have that.
We're using a Google Folder, which is just shared between all of us and all of the documents.
And we're also using a chat, which I assume every single one of us just has open on our computer and is just sitting there.
So everyone's looking in on it once in a while, people are commenting on it once in a while.
That's how team communication is going.
Design, I think, is probably the most solid point of all the things.
PRESENTER 1: Yeah.
I think we came up with pretty solid ideas the last time we met in person.
And since then, we haven't really deviated from that.
It's been pretty straightforward that this is our goal.
We might have to cut-- we'll probably have to cut features eventually, but for now, it's pretty solid.
PRESENTER 10: I think we're fine without cutting features.
I think it will actually fit right in with the time we have.
I don't think we can add features, but we-- and I think I know exactly what the game is at this point.
And I think that everyone else is in the same place, but I don't know.
So I really think that figuring out what our design is is completely solid at this point.
As for assigning tasks, sharing work, that stuff is sort of a work in progress.
PRESENTER 1: So we have a sprint task list already that we've been using to assign estimates and people to certain things.
So far it's kind of been a free-for-all.
But from looking at the task list the last time I saw it, it looks like everyone's completing things in a pretty good pace.
So I think that so far it's worked out nicely
OK, I guess you're done.
OK, thank you.
I think that's everybody.
And now, Phillip
OK.
So just a reminder that this is project three, digital prototype two, user interface.
So in addition to all of the project management stuff that we've just covered in the past half hour, there's also the intent to maximize usability.
And we've already talked a little bit of usability when Sarah gave the last testing lecture.
Now we're going to do a bit of a deep dive.
So this is going to take a bit of time, but I hope you will follow me through this.
You're supposed to use user feedback through user testing on your own, which means not just relying on the testing that's happening in class, but also doing it outside.
So we're going to talk a little bit about that.
Before that, I'm going to walk you through an exercise about what's the intent of usability.
The whole point of testing for usability and designing for usability is basically just to figure out whether someone can figure out how to play your game.
Not so much whether they're having any fun doing it, although obviously you don't want them to be frustrated or confused.
But you actually are just trying to figure out whether people can understand how to interact with your game.
The game could be improved through play testing to sort of say, oh, the challenge level might not be right for instance.
You can use focus testing to be able to figure out specific design issues in your game, like for instance, do these armor upgrades make sense to somebody who's not on our design team?
Usability testing is we have this thing that we designed with the intent of having it learnable.
And can people actually learn how to play this game?
Now, just a straw poll-- how many of use speak or read German?
OK. How many of you have seen this game, or played this game or have heard of this game?
All right.
So if you do speak or read German, just hold your responses for a second.
And I'm going to start by just looking at this game.
And this is a game that you'll find on a shelf, maybe in a speciality game store or an Amazon shelf.
And just by looking at this, what can you intuit from what this game is or who it's for or how it might be played?
Any ideas?
Throw it out?
It's for young-ish kids, where they stack randomly-shaped blocks on top of each other
Wow.
How do you figure that?
Because it's bright and colorful and it has cutesy cartoon stuff on it
OK, bright and colorful.
Cutesy color stuff
The animals are making funny motions, implying that they are precariously balanced.
Also, the alligator isn't eating anyone
The alligator isn't eating anyone.
It is kind of nonviolent, bright color.
What else on this stuff here, assuming that you don't speak German, gives you any other clues.
I thought I saw a hand up.
Yeah?
The type
The font.
The cartoonish-- yeah.
I think, Matt, you had your hand up
I was going to say something, that there's an exclamation point
Oh, yeah, yeah.
It's like, you can imagine there's something zany going on.
Anyone recognize the brand?
Has anyone heard of this brand before?
No one has four-year-old toddler relatives or anything?
Because it's a big kids brand.
OK, folks who do speak German, what's on this box?
What's this box telling you?
The title is Animals on Animals
Animals on Animals.
So yeah, again, you're putting animals on top of each other.
If you want to open the box, and I've actually gotten a box, just one second.
You see these kinds of pieces, brightly colored wooden pieces.
There is a crocodile.
But there's also this bright red die with these symbols on them.
Any clues on how this game is supposed to be played?
I mean, we already talked about putting animals on top of each other.
But here are the pieces.
If anybody actually wants a closer look, you can come in and have a look.
I'll go back to the previous slide for you to see.
Any idea how this game is supposed to be played?
Anything more specific than put animals on top of each other?
I feel like you're probably going to take turns, and every turn, you're going to roll the die and do something based on what it says
OK. And that's based on?
Making that assumption based on?
The fact that there is a die and you've probably played games with dice before.
And actually, in the previous picture, I think it actually says players-- is that a player number?
I guess it's off in a corner.
It is a multiplayer game.
And if you could see that corner, you would've assumed, yeah, taking turns seems like a normal thing.
That's based on your own experience of having played board games.
What else?
Any idea what these symbols might be?
Yeah?
Based on the hand symbol, it seems like it might be restricting which hand you can use to place the animal
OK.
So you are already assuming that this is a very dexterous game.
It's going to require item manipulation, a hand connected to that.
There is a picture of the crocodile.
There's a question mark.
I'm sort of walking you through the process of somebody actually looking at this box for the first time.
And sometimes when you see this kind of game in the store, you don't even get a chance to look inside the box.
But if you had a display version, you'd be trying to figure out the game based on these things that you're seeing.
Now, there is a newer version of this box cover that makes the stacking animals on top of each other really, really obvious.
That's the same cover that I've got over here.
And this is the English version, Animal Upon Animal.
But if you wanted to figure out how to play a game and you couldn't look inside a box, what's the next thing you do after seeing it on the shelf?
You look at the back of the box.
And there, everything that you've intuited from just looking at the components and looking at the front of the box is made even more explicit at the back.
Now, it's in a tiny little font, but you've got a picture of two charming kids who are stacking things.
Notice, they're stacking everything in one plane.
There are some pieces scattered around, but the only things that are being stacked are stacked within this single plane of the crocodile.
And of course, there are indications of how long the game is supposed to be, how many players it takes.
And of course, if you were to actually buy the game, you would look at a manual, which happens to have six languages in it.
And it would explain things, like what these icons on the die are supposed to be.
But ideally you wouldn't even get to this point.
And the kid is not going to read this, because I believe this was targeted at kids age four to 99.
So if you're age four, you're not reading this manual.
A parent is going to be reading this manual, or some sort of older sibling is having to explain this to the kids.
So this isn't actually targeted at the target audience of this game.
Maybe it's targeted at the buyer of the game, but certainly not at the people who are going to have the most difficulty playing this game even though the game is intended for them.
By the way, this is a great game.
This is a whole lot of fun to play with family.
So a lot of what I'm going to discuss going forward is just kind of a theoretical analysis of that process that we just went through.
We just used a whole bunch of different cues, just looking at the box and the pieces, and the text on the box and the colors and cultural knowledge to be able to figure out how it is that you play this game.
Now, this is a game for kids.
It's not a very complex game.
If I gave you some sort of really, really abstract German board game, where everything are cubes-- and there are many, many games just fitting that description-- you'd probably have to read the instructions just to be able to figure out how to play the game.
But ideally, you'd want the experience to be about as easy as the process that we just walked through with this board game.
You look at this game, you're presented with information upon launching it or even before you bought it, and by the time you've actually started playing the game, you already have a sense of what you're going to expect, which sets people up to learn the more complex things that you need to teach them along the way.
Now, there's a book-- which I should have brought upfront.
This book I strongly recommend, The Design of Everyday Things, which was previously titled The Psychology of Everyday Things, because that's really what this is all about.
Usability, designing for usability and you are testing usability, you are asking questions about the player's psychology.
And you are trying to use that to your advantage to be able to help them understand how to manipulate your game and how to get the experience that you want out of it without being confused.
Now, there's this model that Donald Norman, the author off this book, goes into great detail.
And it says, there are goals, there's an intention that the person using an HCI, a Human Computer Interaction system.
And that person has some sort of goal.
They are using Excel because they want to balance their budget or something like that.
They have an intention.
They're going to put numbers into the spreadsheet that reflect their actual spending.
They have a plan, which is they're going to collect all of their receipts together and look at those bottom lines and start typing them in.
And then they actually execute it.
They're actually-- all right, I think this is the way how it works.
And then the computer gives them some sort of feedback-- visual, audio, vibration, if you're designing a game.
The person using the system has to perceive it and then figure out what that means.
OK, a dialog box popped up on my screen.
I did something peculiar and didn't expect that, and then they have to evaluate, did that get them any closer to the goals that they had.
And maybe then the goal changes to maybe I should learn Excel before I try to balance my spreadsheet.
And then now they have a slightly different goal.
So you can translate that into games.
You start off with the goal.
How do I win this game?
How do I lose this game?
How do I avoid losing, and how do I win this game, or at least complete it?
And then the player decides, what do I want to do and comes up with a plan, which is how am I going to do that thing that I want to do, and then tries it.
The game reacts, gives some sort of feedback-- audio, sound, vibration, whatever-- but the player still has to perceive it.
And this is a big point right there.
If you give the player feedback and the player doesn't perceive that feedback because they're too distracted by something else, because it's the wrong color or it's off in the corner of the screen where the player isn't looking, the player doesn't get that feedback.
The player has to perceive the feedback and then figure out what that means.
It's like, oh, a big giant exclamation mark appeared on the screen.
What does that mean?
I have no idea.
And even if a player understands what that means, they have to figure out whether that's getting them any closer or further away from the goals.
So here's an example, back to my StarCraft fandom.
You're playing StarCraft.
You wanted a giant army to roll over your opponent.
And that's the goal.
Actually, the goal is just to roll over your opponent.
And what do I want to do?
I want to build a giant army to accomplish that goal.
How do I do that?
I'm going to build lots of buildings that is going to create all of these army units.
And those army units are going to serve that purpose that I just described.
All right, let's try it.
Game reacts, and you get this.
What do I see or hear?
The computer is yelling at me, saying not enough minerals.
What does that mean?
I guess I need minerals.
And is that what I want?
I guess.
If I have minerals, then I can build those things that will get me the army that I want to roll over my opponent.
All right, my goal hasn't changed at these long term, but my short term goal has.
Now I have to figure out how to get more minerals.
So in The Design of Everyday Things, Donald Norman introduces this concept called affordances.
And the idea is that there are certain kinds of interfaces, things around in the world, among the machines that you use every day but also within games, that encourage you to interact with them in a certain way.
If you see, for instance, a glass wall, and your goal is to vandalize it, what's the most natural thing to do to that glass wall?
You smash it.
All right.
What if it was made out of plywood, and your goal is to vandalize it?
You want to write on it.
What else can do with a plywood wall?
Concrete wall?
You want to break it?
Or spraypaint-- yeah, yeah because concrete holds paint pretty well.
So there are these materials that already suggest things that you can do with them.
And you can take that concept further in a more explicitly designed way.
So let's look at door handles.
We've got some door handles around here.
I guess there is this kind of crash bar.
You've got this sort of door handle, and it's just the right size for your hand.
It's kind of curled in a pleasant way that affords gripping, that affords turning because there is a hinge on one side, and there's a sort of a lever action so you can use less force to be able to turn this thing.
So you are immediately encouraged to gripping this thing.
But what does this door handle not tell you about the door?
It's very important if you want to open the door.
Yeah, do you push or do you pull this door?
So let's think of different kinds of door handles.
This is something that you could grab and pull.
But the designers of these intended to use this for you to push, because it's a lot easier for you to exert force going outwards than to exert force going inwards, using this kind of bar.
That's why you see these on doors that open outwards.
However, that's still a problem because there's a little rod all the way through, and your hand can go all the way around it.
And it's still tempting to just grab the thing and pull it, which is a really bad situation if it's an outward turning fire escape, for instance.
You don't want people trying to pull a door open when they should be pushing it when they're in an emergency situation.
You don't want to have them do that kind of processing.
So that's why they came up with these crash bards.
And eventually this one, which is very similar to what we have on our door.
They have these very wide bars that you can push, and you can put your hand around it but not comfortably.
It's much easier for you to use the flat of your hand to be able to push it outwards.
And the version that we have on this store over there, the part that you push is off to one side, which tells you where you should be putting your hands so that you can get better leverage to open the door-- which way the door opens, left or right.
And that's the kind of thing that automates this process of how do I open this door in an emergency situation.
These are affordances of the design itself.
The material is mostly metal, although I guess the version that I've got up here has a rubberized grip.
And the idea of a rubberized grip is that, well, maybe you want to put your hand on the rubber part, not on the metal part.
So you can design these things in your games, these affordances to sort of encourage people to, say, click on something.
You can design something that's meant to be clicked on to look like a button, for instance, that's slightly raised.
And Apple does this a lot, especially if you look at the older iOS design.
Every single little icon kind of had this little glossy bump.
It tried to give you this impression that this was something that was raised off the screen, even though of course it was completely flat.
And it tried to encourage you that this was a thing that you can tap with your finger.
I'll go into more examples of this later.
Anyone can think of something that you've seen in a computer interface that encourages you to use it in that way?
It could be mechanical.
It could be on screen.
Look at your computers right now
It's not like this on Mac anymore, but the scroll baller has a little-- I don't know what you call it, the lines that look like you can grab them and scroll
Yeah, it had a texture, sort of three little lines that sort of gave you the idea that this was a raised thing for gripping, in the same way that a rubberized grip over a door handle encourages you to put your hand on it.
It probably would make more sense nowadays, actually, given that so much of it has gone to touch screens.
The scroll wheel would be another one.
I just broke my mouse scroll wheel this morning, so it's on my mind.
Now, scroll wheels on mice actually do two different things.
You can click them, and you can rotate them.
But they're kind of really designed to be rotated.
And that's why I broke my scroll wheel, because I kept clicking it.
You are encouraged to roll that thing up and down.
They're usually made out of some sort of rubberized material and a place right where your finger has easy reach.
You can flip that around and make things deliberately hard when you don't want someone to be doing something.
Here's a screenshot of-- what game is this?
Plants vs. Zombies.
There are no zombies on screen, but it's very popular.
And early in the game, early in the development of this game, they had different values for these early items that you'll get right in the beginning.
This is I think the very first level in the entire game, and they're just trying to get you to play this game.
Now, this is a tower defense game.
You plant plants that do various kinds of offensive or defensive attacks against zombies who are invading your lawn and trying to get into your house.
And there are things like sunflowers that actually create the energy that you need to be able to power the rest of your plants and to give you the resources to be able to put the rest of your plants down.
But if the game starts like this, what is first a first time player who's encountering this game likely to do?
Put a plant in the dirt
Put a plant in the dirt.
Which plant?
A sunflower
You think to put a sunflower first?
Or the pea shooter
Yeah.
The one on the left is--
The pea shooter
Left to right
Right.
If they read it from left to right, then they'll probably start with the one on the left.
Maybe there's a 50/50 chance at least that they will put the pea shooter first, which is the little green guy.
And that's actually going to get the player into a lot of trouble real fast, because once they put that plant down, they can't do anything for a really, really long time, by which point you may have lost the game.
They need to put the sunflower down first.
So designers recognized, oh OK, so we could just flip things around.
But again, there's a 50% chance that they will just get it wrong.
At it's the first level of the game.
You don't want to turn somebody off the game that quickly.
So what they did was they reduced the cost of the sunflower and started you off with just enough money to actually plant that sunflower.
They didn't actually flip the cards around because they figured, that doesn't really solve the problem.
What they did was that they made it so that the only thing that you can do at the beginning of the game is plant that sunflower, because that's pretty much always the right thing to do in this game.
Now, the cost reduction might have been also out of other game balance considerations.
Maybe they decided, wow, we really overpriced those sunflowers.
We should make them a little bit easier for you to get.
But this also serves a usability system.
They want the people to understand, the first thing that you do on any level, plant that sunflower.
Now, they could have told you that, but this is a constraint.
They've made it impossible for a player to do the wrong thing right at the beginning of the game.
However, as the game goes on, this, and due to game balance issues, they started to come up with other plants.
This thing is called a walnut, this brown thing over there.
It makes a wall.
It looks like a nut.
And they figured through game balance, well, it's pretty much got to be about 50 solar energy, so we're going to have to price it the same as the sunflower.
But we don't want players putting down walls at the beginning of the game.
They don't actually need to.
They should still be putting sunflowers.
So the solution that they had was, they have these recharge meters that fill up from the bottom.
You can sort of see that this is right at the beginning of the game when you've got a lot of different kind of plants that you can plant.
But only the sunflower is fully charged up, and it's the only card that you can use.
The rest of them are slowly filling up over time.
Right in the first, maybe, five seconds of the game, there is no other card that you can plant, even though the costs may imply that you can afford to plant the walnut right at the beginning of the game.
You can't.
You have to plant the sunflower.
So again, they've made it impossible for you to do the wrong thing.
There's also that shovel, but you can't use the shovel right at the beginning of the game.
The shovel right at the beginning of the game has no effect because you haven't planted anything.
So that's a constraint.
That's preventing someone from doing the wrong thing.
Now, if the decision of whether you do the right thing or wrong thing is a major game play issue and it's something that you want people to sort of agonize and to decide over, then you don't want to constrain it.
You want to give them the option of making that mistake.
But if it's an issue of learnability, if making the wrong decision makes your game hard to learn and hard to figure out how to even get ahead and how to make progress in the game, or even understand how your game works, then that's something that you should consider doing things like constraints and use affordances to sort of suggest, this is what you should do, constraints to demonstrate what you shouldn't do.
Now, that's affordances and constraints.
Any questions so far?
All right.
There are other things that you can do in addition to affordances and constraints that sort of suggest how something should be used.
So this is from Minecraft.
And how many of you have played Minecraft?
OK, about a quarter of the class.
This is something that you see really early on in the game.
The way how Minecraft is played, you are dropped off in a sort of wilderness environment.
Everything's made up of blocks.
So it's kind of like a wilderness environment made out of LEGOs.
And the only thing that you can do is walk around and punch things.
So you start punching the ground and you get some dirt blocks out of it.
You start punching trees, and you start getting some wood blocks out of them, and they go into your inventory.
What you very quickly discover, I would say, in the first minute of game play, is that if you went into your inventory and looked at those blocks, you can move them into this little four-by-four square, just using your mouse and clicking and dragging them.
And the square is named crafting.
So OK, I've beat up 15 trees.
So now I've got 15 blocks of wood.
What do I do with them?
Well, if I drag them into the-- I can turn them into planks.
Is it planks?
Yeah, planks-- just by dragging them up there.
OK. By dragging things into the crafting box, I can make other kinds of things.
That's the craft part of Minecraft.
The mining part is the punching part of the world.
So you punch the world, and you make planks.
You put planks into the crafting square, and you get sticks.
And you can do all kinds of things, like make torches and make arrows and so on and so forth.
Later in the game, you encounter something-- you discover that you can make a crafting table.
And you see a nine-by-nine square instead of a four-by-four square.
Because you've already seen the first one, you've already seen this four-by-four square, the word crafting, and the arrow which points to the right, and now you're seeing this very, very similar thing over here, and inventory is still visible there, even though it's not just you anymore, it's you and this crafting table, the implication is that you should interact with this table pretty much in the same way that you interact with objects back when you were examining your inventory.
So this is context that has been established to the previous things that you're seeing.
And the things that you see around it, like the arrow and your arrangement of the boxes.
And the word crafting-- you're sort of telling the player, treat this very much in the same way that you solved that previous problem.
And then eventually you get to make things like pickaxes, for instance-- so giving some context to what the players are doing.
Now, there's one particularly powerful kind of context that you can set for any action that a player does in the game.
And some of your games already have this-- and that's some sort of storyline or setting or narrative, some sort of fictional setting that your game is taking place in.
And often this is done very, very poorly in games.
You are being told your games are in space, and you're being encouraged to walk outside your spaceship without a spacesuit or something like that.
And in your game, that's fine.
You have no asphyxiation mechanic or something like that.
But players don't necessarily assume that they should be able to walk outside in space and not die.
So when you choose your settings, when you choose your-- when you choose your storylines, you have to make sure that they are supporting those kinds of mechanics that you come in with.
A lot of your games started off by what mechanic that you wanted to do.
Make sure that your storylines are actually supporting that.
Can you think of any games, commercial games for instance that you've played, where the storyline was really at odds with what the players were actually being asked to do.
I see some nods.
Some hands?
Let's call out a few bad ones
Well, one this is that most-- playing games, especially Japanese ones, tend do have battle systems that have basically nothing to do with the story and have a lot of strange mechanics that aren't actually related.
For example, the Final Fantasy games-- your character can summon these really powerful monsters, but for some reason you never actually use those to solve your problems in the storyline, for some reason
Right.
And I would say that's actually the difference between the better and the worse Final Fantasy games.
In the better Final Fantasy games, they actually explain why that doesn't work.
In the worse Final Fantasy games, you have all this power.
Why aren't you just solving the problem instantly?
That's a good one.
Any others?
Well, I feel like the classic example is probably Skyrim, where you're a warrior fated to this grand destiny and probably just run around punching chickens for a while
Well, I'm trying to remember.
Skyrim, you start off like a prisoner or something, right?
But I do remember some of the earlier Elder Scrolls and a lot of Western RPGs.
It's like, let's go into the tunnels and kill rats, because you're the chosen one.
It comes up.
That is actually, I think, partly fiction against fiction.
It's like, you have the quest which tells you to kill rats.
And you have the overall storyline that's telling you that you're the savior of the universe.
And that's at odds.
But there are mechanical issues as well.
Why is your savior of the universe picking up all the cheese wheels in the world, for instance, which does happen in Skyrim.
Something else that you can do is expose more of your system.
How many of you are playing Destiny right now, by the way?
Just out of curiosity.
A few hands, OK. One thing that Destiny does that Halo-- how many of you have played Halo?
OK, more.
One thing that Destiny does that Halo doesn't do is that it actually shows you how much damage you're putting up with every single shot.
A little floating number, in a very RPG-like way, showing you how effective your weapon is being.
And that seems to be a necessary consideration by Bungie, who made both games, that, well, even though our games kind of play in the same way, we need to reveal this extra little bit of information, because the way how weapons work in Destiny is a little bit less predictable than how they work in Halo.
So we're actually just going to show you how much damage each weapon does every time you shoot it by showing you the numerical amount.
This is a screenshot from SimCity.
And one of the neat things about SimCity is that you can kind of open the hood on all of the variables that the game is tracking at any given times, and they show you these lovely little bar graphs.
I think this is, like, citizen satisfaction or maybe it's property value.
I'm not sure.
But you can go into all of these different modes where you can look at your city through different lenses to see all of the different variables that the game is tracking and how they're interacting.
So it takes a lot of work to be able to reveal this sort of information.
And if you're not careful, you can reveal so much information that a player is just confused because there's so much data on the screen at once.
But being clear on this is the action that you're taking and this is the result, or here are all the things that the game currently cares about, is one way to help people better understand how your game is working, so opening up the hood a little bit.
I don't mean giving them the source code.
No one's going to read that.
I mean putting things on screen or through audio to better reveal what's happening numerically in your game.
Previous user experience, we focus-- yes?
What about the danger of you revealing that information, the player seeing how something works, and then them saying, I don't like how this works, I don't want to do this anymore?
Then I think you have a play-testing issue to consider.
It's not like the player is confused about the situation.
The question is-- that there are two possible answers.
One, maybe our game system is badly tuned or badly designed in the first place.
And yeah, once people realize how this game works, they're not going to want to play it anymore, which is a game design issue.
And then the other situation which is, well, if we never explained these numbers to them, they will actually better like our game.
Even if they figure out how this game worked, by not showing them these numbers, they better like this game.
I have that issue with Destiny, actually.
I don't want to see those numbers.
And you can make that decision to hide it.
It's like, this is complexity that we don't want players to care about.
You guys have to be very, very careful, but that's why design is an interesting, tough problem.
But those are the kinds of decisions you get to make as a designer.
Other questions?
We used this earlier when we looked at Animal Upon Animal, Tier auf Tier, And people immediately assume that this was a game for kids based on user experience, other kinds of games that you've seen-- the color of the font and the shape and how it's written.
This is an illustration of how pull-to-refresh works on iOS.
If you want to refresh all of the email in your mail client, you pull it down, and this little circle starts to stretch, and there's a little icon there.
And then it starts refreshing.
That is not an interaction with a computer that anybody's born with.
This is something that someone had to learn from somewhere.
Anyone can think of when was the first time you saw this on a cellphone?
On Instagram?
OK. Any other earlier things?
I think Twitter actually has the patent on this, but they don't seem to have sued anyone for using this, and Apple uses it now-- pull-to-refresh for your Twitter feed.
And now it's in many mobile apps.
And mobile apps just assume that we don't have to teach anyone this.
They just assume that if you are a regular user of mobile apps, pull-to-refresh is something that people will understand how to do.
Here's another example.
This is early in Half-Life 2.
And early in the game, you're just given a crowbar.
And the dialogue even says, you know, assume you know what to do with this thing.
Why can the game just give you a crowbar and not tell you how to use it?
Why can Half-Life 2 do that?
Because people know how to use a crowbar in real life
OK.
There is some real-life personal experience about how to use a crowbar.
Although I would argue that how to use a crowbar in Half-Life 2 is not actually how you use it in your life.
In real life, you sort of lever it, right?
In Half-Life 2, you kind of do that.
Of course, having to use it in a game is using the mouse and keyboard.
Why can the game get away with not explaining to you what this object is used for?
Well, it's a sequel, so you could assume that the person played the first game
The assumption that they've played Half-Life 1.
It's a game for fans.
It's not that crucial a tool, although eventually they start to have puzzles where you need to solve it using the crowbar, but you can usually solve it using some other weapon as well.
The crowbar happens to be an iconic weapon of the Half-Life series.
And so they just assume, you're getting into Half-Life 2, you probably realize that the crowbar is a thing in this series.
So again, previous user experience.
There's cultural cues that come from outside of games.
Like, this is a shot from Grand Theft Auto.
And even though you may never have driven a car that looks exactly like that, you can make certain assumptions about this kind of car.
What kind of car is that?
It's kind of blurry, but maybe you can make it out.
Does it go fast, or is it kind of a clunker?
Fast
It's fast, because it's kind of a streamlined, sports car look.
So you know that you're in a fast car.
Grand Theft Auto is a game.
There's a lot of violence.
If somebody shot at you while you're in this car, do you expect to be taking damage?
Yeah.
There's no windows.
There's no steel frame.
If you wanted to get onto that bridge back there, how would you do it?
Just go up the on ramp, right?
I mean, you can kind of see it, but it actually curves off and then goes onto the bridge.
But you kind of assume that players have seen real-life on ramps and sort of understand how urban geography works.
In this case, it's particularly an American city.
And they see a bridge in the background.
And this thing that slopes upward, even though it curves away from the bridge, people can assume that it actually leads to the bridge.
So they're using all of these cultural cues.
But you can take that idea too far.
So this is a screenshot from the Magic Cap PDA user interface.
So this was pre Apple Newton, I believe.
And this was a user interface for a personal data assistant.
And there are some things here that sort of make sense.
What do you think the little postcard thing in the middle is supposed to be for?
Writing postcards?
Hm?
Writing postcards?
Writing postcards?
This is a digital device.
Do you think you're actually using it to write postcards?
Internet postcards
Internet postcards, OK. What do you think this thing in the lower right corner does?
Recycle bin
It's a recycle bin.
It's a trash bin.
You've seen it in a lot of different desktop-- this time it's a trash truck, I'm assuming because all of the other recycle bin and trash bin things have been copyrighted or trademarked by Apple and Microsoft, respectively.
What does the magic lamp do?
Grants wishes
Grants your wishes.
Programs?
[INAUDIBLE]
Like, apps.
Yeah, OK.
Some other hands?
Some other ideas?
Help
You think help?
Yeah, you rub the genie for help.
So we have three fairly plausible entirely different interpretations of what that magic lamp is.
This is an indirection metaphor.
Everything on this desktop-- and we still use that terminology today.
This is from the early '90s.
Every single one of these things is a metaphor for something else that you can use on the PDA.
You're not going to be writing postcards using a digital device because that's a medium that exists outside of digital technology.
But you're going to be sending emails, so you're going to use that postcard icon in the middle to send messages.
Are you going to be sending faxes exactly?
That's a fax machine.
First of all, in 2014, I'm not entirely sure whether anyone recognizes a fax machine anymore.
Are you going to be using that thing to make phone calls, or are you going to be sending pictures with it?
The clock is not such a difficult thing to understand.
So some of these interaction metaphors work, and some of them don't.
Again, if you put story in your game or even some sort of a fantasy setting or fictional setting, that's already establishing a metaphor for your game.
If your game is about child grows up and learns to battle the evils of the world, for instance, that maps on pretty closely to the mechanics of a lot of games.
That's why the Zelda franchise works kind of hand in hand, the storyline and the mechanic.
You start off as this kind of helpless kid, and then you get all of these tools and take on bigger and bigger challenges.
It's a big metaphor for growing up.
But you can also confuse, or at the very least, date your game by over-relying on metaphors.
So if you, say, have a big stop sign or something that, well, that's probably pretty timeless.
But it's going to assume that everyone has seen a stop sign, and your users will be unclear whether that means stop the current action or stop forward velocity.
Those are two different things.
So I'm just saying, be careful of using too many metaphors to do the heavy lifting on getting your concepts across.
Often you're going to have to rely on multiple means of feedback to help people understand how your game works.
Now, one thing that you're going to be keeping your eye out for when you're designing for usability errors-- mistakes that people are making, but I want to clarify there are two different kinds of errors, and mistakes are only one of them.
Slips are skilled behavior errors.
Someone who knows what they want to do and accidentally did the wrong thing.
How many of you have gone to class when you had intended to go to a friend's dorm room, and you just ended up in your class?
OK, yeah.
Or driven to work or something while intending to drive to your friend's house?
So it's not like you didn't know where your friends were.
You knew where your friends were, and you knew that you were going there.
But somewhere along, your skill behavior of getting to class just automatically took over.
And you did the wrong thing.
I have calling your boyfriend by your last boyfriend's name on my examples here.
That's a tricky one.
But things like clicking a button that looks very similar to the button that you actually wanted to click.
This is, by the way, something that you find in Dark Souls and many, many different fantasy games, is you have a monster that happens to look like a treasure chest.
And this was intentional.
So the game designer is using this kind of player error deliberately to sort of trick you into a powerless position.
Ooh, a treasure chest.
I'm going to open it.
And it bites you.
So it's going to mimic.
And those are things that might be OK to leave in your game.
If it happens too often, it can be frustrating, especially if you're sort of designing your user interface in a way that leads people to confusion, and you want to be able to catch those errors and fix those.
But a little bit harder to fix are mental model errors, where somebody actually thinks your game works differently from how your game actually works, and they're going to make a mistake because of that.
So how many of you play Portal?
All right.
Up until you meet this thing, is there anything in the game that can actually hurt you?
Oh, yeah.
You can fall into water, that really, really icky-looking water.
So that's the first time you die.
But that's something that you fall into.
That's something you kind of like-- yeah, you can kill yourself, but you have to kill yourself.
Then you meet these things, and you haven't met a single thing that is out to get you.
And then you walk around the corner, and what does this thing do?
It gets you
It gets you.
It shoots you.
It's a turret.
It says some really, really cute things, and then it's like, are you still there?
Bum-bum-bum-bum-bum.
And then you suddenly realize in this game that there are things out there who are actively going to be trying to hurt you even if you're not moving, for instance.
So now, up until that point in the game, the player will be perfectly legitimately thinking, the only way that I'm going to lose or die in this game is to make a mistake myself or to deliberately cause myself to be in a position that's going to kill me.
But then they find these things, and they realize, no, actually, there are hostile elements in this game that are out to get me.
And now they have to revise that understanding of what this game is all about.
Now, this is the challenge.
You as a designer, you are the people who are making the games.
You are making these systems that players are going to interact with.
And when players are playing the game, they're sort of having a little conversation with the system.
But your conversation with the system is kind of one way.
You are making this system, and then the system isn't there to tell you, by the way, the players are doing this thing that clearly indicates that you don't understand what's going on.
But the system isn't going to be able to give you that information.
Actually, if you take our design class in the spring, it gets even worse, because what you're really in control of are the game mechanics.
And those are going to result in emergent dynamics-- strategies, for instance, things that are optimal, things that are going to lead people in a downward spiral.
And that's going to create an experience for the player, the aesthetic which we will get a little bit into later in the class.
They're just what the player actually experiences.
So the player may think your game is very threatening and dangerous and it's out to get them, when really to you, it's just a perfectly harmless logic puzzle, because this is what you have control over.
But through all of that interaction, what you've got here is kind of a second or even third order problem, where the thing that you get, where you can actually create, is not the thing that the players are actually experiencing.
So that's why you've got to test.
That's the only way that you are actually going to get information about what the players are actually experiencing.
You can't intuit it just by looking at your design.
Maybe with a lot of experience, you will eventually start to get some best practices, but you still have to do usability testing, even if you're extremely experienced.
So one way to do testing is to actually set goals for the player.
Sometimes when you're doing this in the middle of play-testing, the players can come up with your own goals.
But what I like to do is actually ask players, all right, I want you to create a new game with a new player and give it a name of whatever you want.
Give them a specific goal, and see how they go about that process of actually accomplishing that.
And of course, while they're doing that, you observe them.
You encourage them to talk aloud.
You observe where they're moving their mouth, or if they're doing a tablet game, observing where the fingers are looking.
Try to figure out what they're looking at.
In big companies they can do things like eye tracking-- Riot Games talks a little bit about eye tracking-- to actually figure out what they're looking at.
But sometimes if they're talking aloud, they will say, well, I see all these buttons.
And they all have icons on them.
And this one looks like it might mean delete game, so I'm not going to click on that.
This one looks like a disk drive, so I think it's a save game, so I'm not going to click on that.
And then while they're saying all of these things, you start to understand how your players actually understand the things that you put into your game.
And you can figure out why they're making these errors.
So you're looking for them to make these errors, and you want to record them down so that later on, when you go back to design and go back to implementation, you can fix them.
Or you can give players the necessary feedback to avoid those kinds of errors.
But it's important-- when they make an error, you have to try to identify what the player's reaction to that is.
Say something happens that they didn't quite expect.
Are they surprised, or are they confused?
I thought I was creating a new game, but I ended up opening an old game.
That's probably a confusion situation.
I thought this was going to take out one enemy, but it took out five enemies.
Pretty nice surprise-- players might like that.
So even though it's an error, you might be able to just live with that, or even amplify that.
Let's give these players these weapons that occasionally do critical damage and surprise the player by how bad-ass the players are.
Are they frustrated, or are they engaged?
I chose Flappy Birds in particular as an example here because when people are playing Flappy Birds, they get very angry, right?
But then they go back and play it again.
Are they frustrated?
Well, yes.
But are they engaged?
Yeah, they are.
For some people, Flappy Birds is an extremely engaging game, and it's kind of a pity you can't get it anymore.
I also have two different versions of Flappy Birds.
One is kind of an Arduino powered version, but it's really made out of paper and cardboard.
And one is the actual digital version, so to remind you, you can do this without having to actually implement a digital version.
You can do this with your paper prototype.
You just have to create a paper prototype of a user interface instead of just game play.
So you can do testing even before you've written a single line of code.
So I've talked a lot about constraints.
Now, so far everything that I've been talking about has been about informing the player and then trying to catch those errors and then feed that back into your design of how you provide more information to the player.
But you can also prevent certain problems.
Constraints is one way, where you just make it impossible for them to do the wrong thing, or at least early on, when you had multiple choices, you sort of take some of those choices away and maybe later reintroduce those choices when you are fairly sure that you've understood what the consequences of those are.
You can do conformation, which is you take an action, and then the game asks you, are you really sure you want to do that?
However, this is really sub-optimal.
Any idea why?
Why is it bad to get a little dialogue box asking you, are you sure you want to do this?
If you wanted to get to it, it might be really annoying
OK.
It gets annoying, yeah
And it might make you doubt what you were doing in the first place
OK. Are you really sure you want to close this web page?
You really sure that you want to save this game.
Well, maybe not, but maybe that was the right thing to do.
You can also get really automated on that.
Players can sort of learn to just click yes every time that thing pops up, and then kit kind of defeats its purpose.
If all you're doing is throwing up a dialogue box and they're going to say every minute or something like that, and the answer 90% percent of the time is yes, then players start to learn to just click yes every time they see the dialog box, whether or not they actually intended to click yes or not.
And that's bad.
You're training them to sort of defeat the confirmation process.
And then you get dialog boxes like this.
I don't know what the buttons do.
So this is Skype, obviously.
This itself can be so confusing for players.
The confirmation dialog itself is a piece of user interface that can go right and can go wrong.
So you've just introduced a new element that can be misinterpreted in the design of the game.
So confirmation is something that exists, is often used, and can backfire, and we need to be aware that it can backfire.
You can also do checks.
If you've ever entered a code into Xbox Live or PlayStation network or the Apple App Store, the numbers that you're typing into the system have to fit some sort of checksum or error correction protocol.
And you can prevent certain errors by doing checks on the data that you're entering.
Now, this is probably very familiar for anyone who's done any kind of user interface, software engineering class.
But this is one from Microsoft Xbox Love.
And if you type in a code that you might have gotten from a giveaway or a free coupon from a game developer, or maybe a card that you bought at the store, they're always 25 characters long.
They're always separated by these dashes in chunks of five.
I don't have to type those dashes in.
I just start typing, and those dashes fill themselves in automatically.
If I do type the dash, the input box will just ignore the dash, and it will only allow me to enter in those characters.
So that makes sure that the dashes are always exactly five characters apart from each other.
And that's a visual way to provide a check while I'm entering this long string of characters.
I can do this sort of visual check of, are my characters appearing in the right position in the string as they are printed on my card, and as it is being displayed on my screen.
You can do other kinds of checks-- like, if you try to do the same thing on the Apple App Store, I believe it could be numbers, it could be letters, or whatever, but you can hit OK as soon as you enter it in, and the system actually does a quick check before it even sends the code off to Apple on whether that's actually a valid string, that fits a certain kind of checksum algorithm.
Before it sends it off to Apple.
And if it's a wrong one, it just tells you right away, no, you actually made an error.
You can fix that now.
So you don't have to wait for it to hit the server and come back.
But say the error does happen.
The player has made a mistake, and it happens to be a mistake that you know about as a designer.
And you know this mistake is going to come up often.
This kind of error is going to happen.
And maybe it's a slip, maybe it's a mistake.
And you want to provide some feedback.
So first of all, make sure that you are informing the player that an error has happened.
And you need to test whether the information that you're providing, whether it's audio or visual, some combination of the two, is actually something that people notice and see, and then they realize that, oh, I've made a mistake somewhere.
And then you've got to get them some tools to be able to fix that problem.
There's backwards, which is basically undo.
This is Words With Friends, for instance.
If you enter in some sort of word-- this is an online game, by the way.
It's a multi-player game that you play on your phone.
And because it's Words With Friends, you're basically playing Scrabble against somebody who's geographically separated from you.
And if you've taken a turn, you've entered in a word.
And then you say, wait a minute, I've come up with a better word.
You can hit Undo.
I think it's called Recall.
The button is actually called Recall.
Take your move back and then just replay your turn before the other person has taken their turn.
But that's backwards error recovery, very good for things like turn-based games, games that you play by email for instance or games that you are sort of-- like Final Fantasy Tactics, or strategy games.
A lot of strategy games which are turn-based allow you to sort of undo your decisions.
Even if you've said, yes, this is what I want to do, wait a minute, this is not what I want to do.
I want to undo that.
There's also forward error recovery.
This is a screenshot of Mario where Mario's mid-jump, and based on player experience up to this point, you realize halfway through the jump that you're not going to be able to make that jump.
What can Mario do?
You can go backwards.
You can change direction in midair and say, oop, not what I'm going to do.
It doesn't rewind time.
You're still moving time in the game.
And time is important in a Mario game.
But the player has the tool to say, I'm going to try to jump.
No, I'm not going to complete this jump.
I'm going to go backwards before it's too late.
There is a consequence to that, and the player is losing time.
And in many of your games, you're going to want to allow those errors to happen but then give players tools to be able to take a new action to compensate for the mistake of the previous action.
And that is called forward error recovery.
Now, if it's a mistake that happens near the end of a game session-- say you're making a multi-player game and somebody's about to win.
Forward error recovery may not be something that you want to make too easy, because somebody's spent a long time actually getting themselves in a position where they can win the game.
And then the person who's losing uses whatever forward error recovery tools you've given them to say, ah, let me just take back all of those mistakes I did and then come back from behind.
Well, if you were explicitly designing a game where someone is supposed to be able to come back from behind, like Mario Kart does something like that, then that's great.
But if you're trying to make some sort of game where long-term strategy is supposed to pay off, then yeah, you don't want to make those forward recovery tools too powerful.
Let people live with their mistakes.
One last point, and I save this for last because I want you to be thinking about this in your team, so something for you to go into discussion about after this talk, is accessibility.
One challenge is that when people think, my game is for everyone, my game is for anybody who plays games, is that that's not actually what the game ends up being.
Often it becomes this, which is you're going to assume that your player is focused and has full use of their hands and full visual acuity, that they speak English so they'll understand any text that you give them, that they know how to install stuff in a computer and they've played games before, that they are around about your age, that they have desktop computers that have lots of processing power, especially if you're doing a 3D game, that their mice have more than one button, that they're actually using mice and not a trackpad, that they've got headphones so they're going to listen to every audio cut that you're going to give them without distraction.
So these particular red line things are the things that I want you to be willing to question about your own game.
There are some things like-- they're speaking, computer-literate young adults.
It's great to be able to design games for people who are not like you.
It is a little bit beyond the scope of this particular class, although you do have to keep in mind that we are also designing this games, your final fourth project, for people who may be working for the Red Cross Red Crescent Climate Centre.
So you want to keep them in mind and consider-- many of them actually speak English.
They may be foreigners, people who work in other countries, who will appreciate a game that is a little bit more language agnostic or even localized for them.
Shall we assume that the computers are capable of handling your game?
OK, I think for this class that's all right.
But shall we assume that the computers are only running your game and nothing else, that they're not running any IM clients-- or a lot of the games are going to be browser games so that they don't have any other browser windows open, for instance.
I think that might be asking too much of your player.
You might want to assume that they have other things running around in the background-- their iTunes playlist, for instance-- and which is going to interfere with your sound.
You should be able to design your game so that you don't assume everyone has a left mouse button and a right mouse button, or necessarily a mouse.
For one thing, it'll make testing easier.
A couple of you discovered last time you did a visual test, that if you don't have a mouse, a lot of your games actually became difficult to play.
You might want to have that discussion-- at least have that discussion with your team, and if you decide, no, a mouse is essential, make it very, very clear that's what you're saying.
And put it in your game.
Say, this game is best played with a mouse and headphones.
There's a lot of iOS games out there that require headphones, and they will state right upfront, this game is best played with headphones.
And I would like you to at least consider colorblindness.
Actually, yeah, there we go.
When you are designing art for your game, buttons that need to be distinguished from each under or visual cues, consider that there are people who can't tell the difference between certain color pairings.
What are common ones?
Red, green.
Red, blue?
Blue, yellow
Oh, blue, yellow.
And red, blue.
Red, blue.
Blue, yellow.
And red, green.
Those are the most common ones, but there's a lot of colorblindness out there.
And it affects, I think, 20% of the population.
10?
10 to 20%?
I think it's as much as 10% of the population
10% of the population.
So there's a lot of people out there who might well want to be playing your game but can't because they can't tell apart those visual cues that you're giving them.
So make sure that they're also taking advantage of things like shape and brightness.
These are things where if somebody is colorblind, they can at least still distinguish shape from each other.
So instead of having red diamonds and blue diamonds, you have red diamonds and blue stars or something, and that makes it easier to tell that there's a difference.
I mentioned other application windows, touch pads, or the possibility that somebody might be playing your game muted, which means make sure that not all of the feedback that you're giving them is audio only.
Make sure that you're also giving them some sort of visual feedback so that they understand that if they're playing in a cafe or something like that, they will still be able to understand that they're making mistakes, or errors or slips.
All right.
That went way longer than expected, but I did warn you guys it was going to be long.
Any questions before we wrap this one up?
OK. All right.
Well, now is a break time, I believe?
Yeah.
Take a 10-minute break.
At 2:47, come back here.
If you want to play-test a paper prototype or have any kind of testing in your game, let us know.
Otherwise, working in teams
Yeah.
A lot of you have mentioned how hard it is to schedule a meeting.
You've got the rest of this class to be able to meet with your team.
And either test something, or you can just work on your game.
If anyone wants to look at these books or this game, come up to the front, have a look.
Thank you.
[SIDE CONVERSATION]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today we have from EA, Timothy Cowan's going to give a lecture on development best practices from Electronic Arts.
This is about the time of the class where we start doing hour long lectures and then the remaining two hours is open for your work time.
It's really going to start being like that probably not next Wednesday, but the following Monday is when that's going to be a pretty stable time.
So when you're working on project four, you're going to have a lot more in class time to work in teams.
Hopefully you've been taking advantage of the class time to work in your teams for project three.
Today we are doing play testing.
Has everybody already prepared for what they're going to do for their play test today?
Do you already have questionnaire, survey, observer notes, things like that?
I'm seeing some blank faces.
That's OK. We're going to give you 15 minutes to set that up.
So we're going to do lecture, take a short break, then you'll get 15 minutes to prepare for the focus test.
We're asking you to set up three workstations for the test at all times.
I'll go through the details when we start that up.
But you'll get a little bit of time to set up for play test.
Remember, we're asking for two focus test reports for project three.
This is going to count for one of them.
The other one you need to do on your own time or have already done.
Grading is practically finished for project two.
And now it's just entering it all into the Stellar form, and you'll have all that by tomorrow night.
What I can say just in general for grading, project two looked pretty good.
Most of the games completed all of the requirements.
The main requirements that we saw that were lacking in some of the games were really the legal requirements.
Putting your credits, putting your names on the game.
We asked for that.
So please make sure you're doing for project three.
Even if it's not in the game itself, it can be on the full sheet, the full HTML code for the game.
So make sure you're putting your names on your games when you're turning them in.
And also if you're using any outside code, any outside assets, that should all be listed as where it's coming from and what license you have for that.
Phaser is MIT licensed, so there wasn't anything special for there.
But if you're using Unity, take a look at Unity's license requirements.
Make sure you're fulfilling them.
If you're using hacks or Flixel, make sure your filling those requirements.
I believe we already gave you some comments on the post forms.
But for the most part, those also looked good.
For project three, here's the cheat sheet for product three's postmortems.
You've got a page to write a postmortem.
Make half of it about your team practices and processes.
The other half about the game and the design.
That's really what we're looking for is understanding how your team organized itself, what communication problems you had, how you solved them, and even more importantly, what are you going to do in project four.
Because in product four, you're going to have eight people on your team and problems are going to show up more often.
So hopefully you're coming up with ideas and ways to be prepared for those kind of things.
Any other comments?
Otherwise for the other turn ins for product two, task lists were good.
Backlogs were acceptable.
The focus test reports for the most acceptable.
We give you some comments about your focus test reports.
We gave you some comments about your design change logs.
We also are giving you a big chunk of comments about the game design.
It's going to be labeled not part of the grade.
So like I said, it's not a design class, but we do want to give you some feedback on your designs.
So we can help you out with future assignments.
That's it.
Any questions about anything?
Great.
So I'm just going to hand it off to Tim
Great.
So can everybody hear me?
Sound good?
You can just raise your hand in the back if you can't hear me.
Good?
All right.
So let me start off a little bit with just introductions.
Who I am, my background, where I came from, why I'm here, maybe.
I'm actually originally from Oklahoma.
I was born in Tulsa, Oklahoma.
And went to Oklahoma State University.
And going to university, I actually had no idea what I wanted to do.
Kind of just sort of landed at one of the in state tuitions for university.
But I knew my dad was an engineer, a programmer specifically, and I kind of thought that I wanted to go do what he did.
So I went into university, was going to go be trained to be a programmer.
He actually gave me advice to go work on more pure engineering.
And so for my undergraduate degree, I was actually focused on mechanical engineering.
And I stayed at Oklahoma State University and went on to aerospace engineering.
Did graduate school.
Was in pursuit of a Ph.D. with ultimately the goal I wanted to be an astronaut.
I didn't make it there.
I made video games today.
But I did get a chance to through graduate school work on research for Dryden Flight Research Center in California and actually spent about 10 months there as a visiting researcher while I was working on my Ph.D. And what I did for them was what I'd say were flight simulators.
But not that kind of flight simulator.
It was more this kind of flight simulator.
Computer simulation of aircraft.
I did a lot of both the simulation of the aircraft and generating visual representations of the simulations.
And ultimately out of all that work I did finish my Ph.D. and sort of officially became a rocket scientist.
And so then I had to decide what I wanted to go do.
And my plan really was to go work at Lockheed, design aircraft, work on flight simulators for them.
That was basically my life's mission coming out of college.
But that didn't quite happen either and I ended up at EA Sports.
So instead of working on flight simulators, what I've done for the last 14 years is work on NFL football simulators.
A recruiter from EA just basically blind contacted me from my resume posting on Monster.com and they brought me down to Austin, Texas to work on my first video game, which was Madden NFL 2002 for the PC.
There was a small team in Austin of about six people that built this product off of a shared code base from the PlayStation to Xbox and Game Cube.
And I spent about 18 months in Austin.
Built two versions of Madden PC in that time, until they eventually moved me out to Florida, which was where Tiburon Entertainment was located, one of the studios that's part of EA and part of EA Sports.
And I've been there, really been with Tiburon for I guess it's counting 13 years now.
And in that span one of the things I'm most proud of is I built 14 versions of Madden in that time.
And I can say, with the exception of Madden 15, because my roles changed, I've contributed at least one line of code to 13 versions of Madden over the last 13 years.
Things are pretty different at Tiburon from when I started in 2001.
There were about 200 people at the studio when I started.
There are 700 at our location today.
We work on three different products.
And I'm the group technical director for the studio.
So I help make sure that all three of the products we build succeeds and sort of monitoring engineering and technology development for the studio.
Our three products are obviously Madden.
We build NBA Live and we're working on the next generation of PGA Tour Golf, which will be out this March.
So that's my background.
A little bit about my role.
It is very much it is an engineering leadership role as a group technical director.
And I very much don't write much code.
In fact, I kind of joke with the other engineers that the coding that I do is PowerPoint these days.
Let's talk a little bit about Madden.
Set a little bit of context about some of the philosophy and best practices that I hope to a kind of pass along today.
Madden in North America has been a we'll call it a top five product for at least the last 15 years, typically is a top three product in North America.
Generally the first person shooters are the big sellers, especially in North America.
Sometimes you've got some of the action games like Grand Theft Auto.
But depending on what's being released at any given year, Madden's always a number two, number three in North American sales.
And you can see FIFA top 10 North American sales.
They're a different sort of a sports product.
That is the big sports product but they have global appeal, because the sport has global appeal and it sells in a lot of markets that Madden does not.
But as a North American company, Electronic Arts, the number two title in North America.
In fact, on that last chart, the number one title in North America that EA sold is Madden.
And it's very important to EA.
It's very important to EA stock price.
If you look at any sort of financial reports or just even news briefs on financial listings for EA, you'll often see that the little blurb at the bottom where they kind of describe who Electronic Arts is, Madden's almost always listed as Electronic Arts, developer of products such as Madden, Battlefield, FIFA.
Madden is very important to EA.
And the success or failure of that product is very important to the public company's stock price.
If you look at Madden development, given its importance to the company, given the size of its sales, it's a big team.
There's over 200 developers on staff that do direct development on Madden.
That includes engineers, artists, designers, producers, product managers, quality assurance testers.
And there are additional developers that contribute through central.
Technology We're a very large organization that contributes in a lot of different ways.
But sort of at our studio we call it 200 people doing direct dev on Madden.
10 million lines of code, give or take a few.
That's physical lines of codes.
So it includes white space and comments and those kinds of things.
But a lot, a lot of code to look at across 150,000 plus files.
And we do development distributed across four different locations.
So we've got primary development in Orlando, but we also have remote development in Austin and a contract company that we work with Nova Scotia.
And Vancouver is the sibling studio for Sports, EAC, who makes FIFA, NHL, USC.
They contribute a lot of central technology that helps ship Madden each year.
So there's really sort of four locations that we're distributed across.
Really the message here is Madden's big.
It's a big seller.
It has big importance.
And it's big large scale development.
Large teams, large code bases, large complexity.
And the other characteristic about is kind of important to sort of understand sort of the nature of this project, this big project that also iterates at an iterative cycle every single year.
So our sports products come out at the start of the sports season every single year.
And it's critical that Madden comes out on its date in August when the NFL season kicks off.
Our gamers expect it.
Our shareholders expect it to sell.
And if you miss that start of the NFL season and were to, say, slip your ship date, that means a lot of lost opportunity in terms of delighting our gamers.
That's very different from other types of game development.
Like certain games, they're done when they're done.
And even within EA we do this a little bit.
This was one of the announcements we made about Mirrors Edge.
That's not Madden.
Madden can never miss its date.
And it has come out in August for the last 25 years consistently and reliably.
So given that context, what I want to talk about a little bit first is just some philosophy.
And this is some philosophy I've come to over the past couple years about what I think makes a good engineer at EA and specifically on Madden and our sports products.
What some of the mindset then things that you may need to break out of in order to be successful doing development on the big title that's got to come out on a particular date.
And when you are big title and a big team and a complicated project that has to hit a particular date, the first thing that happens is we go add a lot of really complicated process.
Because there's a lot of moving parts.
Everything's got to get tracked.
And if anything slips, then we're not going to hit our date.
You add a lot of checklists and you check things off.
Hopefully get everything checked off.
But the reality is there's always a lot of work to do.
We've got 200 plus people doing a lot of different types of work and you only have about one year to get work done.
And we want to do and put as much as we possibly can into that product every single year.
Because if nothing else, I know I'm very sensitive to our sports products.
Because they come out every year, we tend to get beat up a little bit over just a roster update.
They haven't done enough.
There's not enough improvements over last year's product.
Well we do a lot of work and we want to do a lot of work to make sure that every single year we're putting out the best possible product that we can.
And it's not just us.
Also our gamers really want-- they're giving us a lot of feedback.
We want this, let's get this feature in, Madden would be better if you just did this.
So there's a lot of work and our kind of human nature is let's do it all.
We're going to get it all done, we're going to ship an amazing product, and we're going to get it all done on time in August.
Well if you take on too much and you try to do everything, disaster happens inevitably.
And one thing we have to remember in sports is development's a multi-year journey.
It's a path.
Most game products, especially if you're looking at a open world action adventure game or first person shooter, they tend to have two if not three or four years of development.
We kind of start to think about Madden development in that same way where it's a three year process with incremental releases along the way, where the game just keeps getting better and better and better on a three year roadmap.
And it's important in this sense because I think all game makers want to do everything, want to make the game as amazing as they can.
But in sports, you really have to embrace iteration.
And that means kind of finding the right balance between doing everything, doing everything right, and especially if you're an engineer remembering that sometimes you have to balance between getting everything done right and maybe using some duct tape and kind of hacking in a quick solution.
Because you've got three years to make sure that you're getting things done on that road that you're on.
So another idea I'd like to talk a little bit about, again, just more philosophy about engineering.
I talk about the difference between science and engineering.
And I'm going to sort of give the extreme examples and maybe overgeneralize things.
But one of things I talk about to my engineers is, if you look at science, science at its core is about the pursuit of knowledge.
The idea is king.
You're going to try to figure out how things work.
You don't know what you're going to do once you find it out.
But just the fact that I know how that works, I know this idea, that's what's most important.
Engineering, on the other hand, if sort of like the best engineer, if you're just the extreme engineer, the thing you get the most excitement out of his taking that knowledge from the scientists and figuring out how to apply it to solve practical problems.
And cost effective solutions is definitely one of the keys in terms of that well rounded engineer that goes in and is figuring out how to solve a very practical problem based on a body of scientific knowledge.
It's really important for engineers remember that there are solutions to even the hardest problems.
If cost is a factor, you often find that engineers will tend to say this type of problem's not possible.
But what they really mean is it's very, very expensive.
But engineers need to really be focused on, how do you break down a problem and come up with a solution to that problem?
And sometimes it's duct tape.
And for the Star Trek fans out there, remember there are no unwinnable scenarios in engineering.
There is no Mission Impossible.
I've had engineers on Madden in particular where Madden in the early Xbox 360 generation actually ran at 30 frames per second.
And there were a few years there where we were told it was impossible for Madden to run at 60 frames per second.
And so in a leadership position, a lot of your job is to kind of break through that this is impossible and start to ask questions on how can we break this problem down and come up with some cost effective solutions that will actually allow us to solve this problem.
And at the end of the day, you're not going to solve problems if you don't attack them with enthusiasm.
And so my encouragement is really to challenge the impossible.
Everything is possible.
It maybe might not be possible today.
It might not be possible with the money you've got in your wallet.
But given enough time and money, we can do anything.
And every problem can be solved.
At least at the core, a good engineer really needs to believe that.
So let's go on next idea.
And I want to dispel the myth of perfect code.
Because this is one of those things that engineers coming into EA, really coming into any job, have this myth that there is perfect code.
Because engineers love perfect code.
And they really love to argue about perfect code.
Even the best engineers in the world, this is ingrained into every engineer that not only is there perfect code, I know what that perfect code looks like and I'm the only one that writes it.
But the reality is, mistakes are always made.
It tends to be the first time you write code, especially if it's the first time you've written code for a problem you've ever solved before.
You're going to make mistakes.
And I do very much fundamentally believe that kind of at the core inside that you learn more from mistakes.
Hopefully you guys have heard that before.
And if you never make any mistakes, you're not learning.
And if you're never taking on really big potentially impossible challenges, you're not learning.
You're not stretching yourself, you're not growing.
So it's important to remember that mistakes will be made.
You're going to learn from those mistakes, and you know what?
Your code will get better as you make those mistakes and refine it.
So remember, there is a software development life cycle.
And in fact, a lot of times when you're arguing about perfect code, it's very early on in the cycle and you don't even know what you don't know because that code hasn't been through that software development life cycle.
There's almost no guarantee that there's no bugs in this code that you're writing.
And it really isn't until you make it through that software life cycle that you're going to understand what those bugs are.
Towards the end of the project where you have this perfect code and all of a sudden now it's been all the way through testing and certification and now you see, oops, I made a mistake.
And when you tie that into our sports iterative cycle where we're embracing iteration, taking on the impossible, one of the big things that try to impress on people is try to value less the perfect code and start to think about proven code.
Proven code that's been through that software development life cycle, has had the bugs shaken out of it.
Even if you didn't write it, that code is proven and will have less bugs typically than code that you go right from scratch.
And then the next idea, and this is the last one before we can get onto the other section and get out of the philosophy stuff, is on a big team, on that iterative cycle there's a lot of tasks.
There's a lot of work to get done.
Everybody's given their tasks and they have to finish their tasks.
They've all kind of got this little piece of a puzzle and they own their piece of the puzzle.
They've been given this task and they're going to get that piece done.
And it's going to be perfect given the context that they've been given on that task.
The trick is there's a lot of dependencies.
And it's often very hard to go see, how do all those pieces together?
And when you've got this kind of situation, you start to end up with you've got these features, which are actually a collection of tasks.
And those features don't really have an owner.
All the tasks have an owner.
Someone's doing the tasks.
But what about the feature?
Does the feature have an owner?
Does anybody know how all those pieces are going to fit together?
And if you don't have someone owning that feature, you end up with unfinished projects.
Or potentially you end up with not quite what you were looking for because you didn't really understand from this task how it all fit together and was going to ultimately deliver whatever it was that we were looking for.
And so it's super important when you get immersed into a really big team and you're hand and a bunch of tasks to pick your head up every once in a while and make sure that you're focused on that feature.
And potentially you're asking, how does this task that I'm working on fit into this feature?
Because if you've got that context, you may even realize, hey, my task actually isn't going to help me deliver on that feature.
I'm not working on the right thing.
As an engineer with a technical background, I'm building good software, functional requirements, et cetera.
You really need to be focusing on that feature.
Because your task is just a means to delivering on that feature.
So in summary, just a little bit of philosophy type stuff.
Embrace iteration.
Challenge the impossible.
Value proven code and focus on features, not tasks.
And that's kind of my top four.
It's easily a top 10 list.
The thing is to kind of think about in terms of coming into a big company, working on a big team.
Some of the philosophy that you kind of need to take with you in your approach on integrating with that team and being successful.
Now I want to talk a little bit about more on the process side, engineering quality.
How do we as a engineering organization at the studio help make sure that we're delivering a quality product.
And game development's notoriously not very formal.
We are game developers, after all.
The challenge, though, when you're not terribly formal in your development methodologies and your development practices is success starts to rely on individual heroics or individual heroism.
And that's a bit of a danger zone when you start talking about shipping a quality product that's big and on time.
Because quality's never an accident.
I love this quote.
It's never an accident.
It's the result of intelligent effort.
Now, before we talk about processes for this, let's decompose quality a little bit.
Because this kind of plays into some of the thinking on our approach to quality.
And really, it's going to focus on the different aspects of quality.
There's perceptual quality.
Is the game fun?
Is it beautiful?
It's how the gamer experiences the game.
There's functional quality.
That's kind of, does it work?
An engineer is given a spec and writes code and it either does or does not do what it was intended to do.
And then there's structural quality.
That's the underlying, behind the scenes quality.
It tends to be the type of quality that engineers care about the most.
That's kind of the code quality space.
Is my code perfect?
The top two perceptual quality, functional quality, those are things that the gamer is going to see in your product.
Structural quality is not something that the gamer will actually ever see or notice.
But it is something that will hold up your project.
If you don't have the right level of structural quality in your product, it will prevent you from delivering that product.
Now ultimately, at the end of the day we're probably most interested in that perceptual quality.
Because that's the one that's most going to resonate with the gamers.
Is it fun?
Is it beautiful?
And we kind of like it meta critic as one possible measure of whether our game's being perceived as a high quality.
This is basically a reviewer sits down, plays the game, or even if you go look at user reviews.
A gamer sits down and plays the game.
And then that game experience gets processed through their filters and they perceive it to be at a certain level of quality.
Now, functional quality is inevitably going to play into this.
But we generally see, I think, with meta critic is a lot of the functional issues may not be seen because it takes hours and hours and hours of playing the game before those maybe surfaced.
And it tends to be, especially with meta critic, you spend four to eight hours playing the game, you process that experience, give it a score.
So I tend to kind of argue, at least when we're talking internally about quality, only the absolute worst functional issues are going to show up and play into your meta critic score.
But the gamer who plays your game for 100 plus hours is really going to notice a lot of those functional quality issues.
We also have been talking recently about kind of a spectrum of quality.
And this is kind of tied to meta critic.
And I steal this from one of our executive producers where from 0 to about 69, those games are sort of just foundationally solid.
If you're anywhere below 69, sort of the upper end of that spectrum.
You get to about 69 or 70 by being foundationally solid.
Anything below that, you're buggy, you're broken, you've got a lot of issues.
And from there, quality kind of builds.
If you're buggy, broken, and have just glaring problems, you're never going to get to that next level where you've got good function.
It functions and if you have a competitor in the space, it functions at least as well as the competitor.
And that's kind of that 70 to 79 range.
But what we really want to do is we want to get up above that.
This is interactive entertainment.
We're trying to really delight the gamers that play our game.
And from 80 to 89, we kind of start to talk about, all right, that's the point where your game is getting immersive.
Especially for Madden, it's starting to be very realistic and they feel like they're there on Sunday watching a game and they're sort of immersed in the experience.
And then even above that, which is where we really want to get to is 90 to 99 or even 100.
I don't know if it actually goes to 100.
But that's where now not only are you immersed in the experience, but the game is evoking emotion.
So when you get up into those upper bands, that's where perceptual quality really starts to come in.
I'm going to talk a little bit more about that foundational and that functional quality, because that's what engineers and development directors are ultimately paid to solve.
And so the question is, on a big project with a lot of complexity that's got to hit a particular date, inevitably need to put a lot of process in place to try to do make sure that you're putting out that quality product or at least at a minimum were able to hit that foundational and functional quality so that the designers and everyone else can help push us up into those perceptual realms.
The first process is code review.
So let's talk a little bit about code review.
Because engineers love code review.
Especially because their code is perfect and no one needs to look at this code to figure out whether or not it should be allowed to get checked in.
At EA and on Madden in particular, we do have a blanket policy, all code gets reviewed.
If you're going to check code in, it's got to get reviewed.
We even in this day and age where data is a huge part of development.
Actually there's a lot of data that gets reviewed before it gets checked in.
But code review is sort of the primary function where all right, you're going to check in code.
It needs to get reviewed.
And the goal there, let's make sure there's no bugs.
Get a second set of eyes, have someone look at it.
Shake out all the bugs.
Because the second set of eyes will find the bugs before it makes it to QA.
There was an interesting study recently, Rebel Labs.
They did a survey and asked development teams, how many of you do code review and how often do you code review?
Code review is very prevalent coming out from this industry survey.
I think it was 76% of teams do a lot of code review.
And their ultimate goal with this survey was to figure out the impact of code review on quality.
And quality for them was defined by lack of bugs.
So the functional realm of quality.
And their study produced kind of an interesting result.
They were actually looking at two factors, quality and predictability.
Both super important for sports.
And they found that code review really helped with predictability.
It indexed well.
You could see if you did no code review, there was an 11% or 10% swing between no code review and always doing code review.
Much less influence though on quality.
They found that there was not actually not much of a difference between teams that did code review and didn't do code review in the span of quality.
Again, quality is measured as bugs
What is predictability?
Predictability is able to hit your schedule on time.
Now, that's not to say that we don't find bugs in code review.
But I think that's interesting because it kind of aligns with some of my experience within EA that, now, don't get me wrong, we put code review in place because we wanted to find bugs early.
And when I started at EA we did not do any code review.
In 2001, again, this is kind of individual heroics, pretty informal.
On a 200 person team, you've got to do code review.
And we do want to find bugs.
But it's actually a little surprising that we don't find many bugs in code review.
And I see that at our studio as well.
How many people here know C++?
All right.
Awesome.
Let's play a little game quick here.
It's called find the bug.
You guys are code reviewers.
Raise your hand if you can spot the bug.
There is a bug here
[INAUDIBLE]
You can't see it?
OK. Is it the contrast or the lights?
Anybody that can see it spot it?
[INAUDIBLE]
Good.
Good.
That is the mistake.
It's a cut and paste error.
So someone moving fast.
And it tends to be pretty easy not to notice this.
It's just kind of hidden in a block of code.
It actually was intended in this case.
It's just kind of following a pretty simple pattern.
They should have been copying PURL into the STR web URL.
This was code that was written, tested by the engineer that wrote it, I presume.
Code reviewed, checked in, and made it all the way out to QA.
I don't know if an actual bug manifested, but this code did make it through code review.
In fact, all this code we're going to look at made it through code review.
How about this one?
[INAUDIBLE]
All right.
Yeah.
That is kind of the trick with code is it's kind of tiny text.
[LAUGHTER] It's probably the sunlight.
People in the front row can see it?
No?
I guess this game's a little less fun than maybe I was hoping.
In this case, you've got two different files and this is a function call.
And there's one line of code where they're using the function.
In the second file, if you're doing a code review you've actually got to-- in this case, someone added a line of code.
And when you get that code review, you get just that file with a little diff and you see their line of code that they added.
You won't actually spot this blog unless you open up the other file that they didn't edit and go look at that function and how it's used.
Because they've actually got the two parameters reversed.
They're both bools.
Track actives versus track passives.
And the function they're calling has those two parameters reversed.
Again, written, tested, code reviewed, checked in.
I don't know if this one's readable at all.
I kind of have to skip through this.
In the interest of time, I'll go ahead and just skip through these.
Because I think they're all somewhat fuzzy and small text.
In this case, if you don't know C++ or don't know the syntax, with that indentation there's an if statement there.
With the indentation it was probably expected that that code would only get it executed if the if condition was true.
They missed their curly braces.
And that that indentation tends to, if you're reviewing the code, kind of draw your eye.
It's like, yeah, that code looks OK. You're kind of inspecting it.
And I'm sure that they knew what they were doing.
And you kind of missed that the braces aren't there.
And this one's maybe-- no.
So in this case, their if statement, it's actually a constant number.
This will always be true.
And they probably actually meant to check if that value was equal to some particular value.
But they actually just put the mask right into the if statement.
And that will always be a number greater than 0, which will be true.
And last one.
This last statement here.
This last statement doesn't do anything.
They're just assigning one variable to itself.
Again, really easy cut and paste errors.
And I think the surprising thing in a lot of cases, again, we've got this process.
We do find bugs.
But you don't find all of them.
And a lot a really simple ones, things that after the fact, when you get a bug report from QA and you go look at that and you're like, how come no one caught this?
How come the person that wrote it didn't catch it?
How come the code reviewer didn't catch it?
I'll tell you, in some cases it's not even just one person code reviewing, but you may have five people that code review it.
Again, we're big, multi-disciplinary.
Sometimes one check in may take five different subject matter experts where I need a rendering engineer, I need a physics engineer, and I need a UI engineer to review this code.
And five different people review the code and no one catches the problem.
But there are other reasons for code review.
And this is one of the things I'm pretty adamant on.
Because when that report comes out that says, code review doesn't really find bugs.
And in fact, if you don't do code review, you're not all that bad off in terms of finding bugs.
I say, wait a minute.
We're not going to stop our code review process.
Because there's other reasons for doing code review.
And finding bugs is actually, I would call, my fourth reason.
Really, code review's about monitoring structural quality.
And that's why that report says if you do code review, you get better predictability.
Because your structural quality is this underlying quality that's going to help you be successful.
It's going to help with your predictability.
It's going to help with knock-on effects, where if you write code that's consistent, reliable, robust, and file certain programming patterns, you're going to end up with a better result in the long term and you'll be more predictable.
It's also about knowledge sharing.
So on a big team of 150 people where everyone's got their own little task and they own their little task, we need to make sure that the whole team has some insight into what all the other developments are doing.
And if you're out sick or you win the lottery and stop working, someone else at the company's has maybe done some code reviews and has seen what you're doing and has learned from that and knows about how that feature, how that task works.
And then it's also about education.
So if you are the best engineer on the planet, you've been writing code for 14 years, and you've learned from all your mistakes, and you write the most perfect code, having a new engineer who just came to work at the studios, is writing their first line of code ever review your code teaches that person something.
And then the last point is obviously catch bugs.
We do want to catch bugs through code review.
It's just it's not the only thing that you get out of code review.
And really, static analysis and rigorous testing is the best way to catch bugs.
And in fact, all those examples I just went through are caught by static analysis that we apply to our code base every day.
Static analysis today is actually very good at pattern matching and identifying those types of issues.
And on a code base our size, you see a lot of them.
And static analysis is actually a very important additional process, I'm not going to talk a lot about that, to layer into a robust development process.
All right, so the next piece.
All right, if we want to find bugs we want to drive functional quality, we need quality testing.
And your game development's messy.
You're never going to find all the bugs.
And really testing is only good to find bugs.
No matter how much testing you do, you're never going to know what bugs you haven't found, obviously.
So we've got to make sure we do testing.
And there's a lot of different types of testing.
There's the engineer doing their own software testing.
There's our QA department who does black box testing.
There's automated testing.
There's a whole spectrum of testing that we do on our product to help make sure that the product we put out is amazing.
But inevitably, game development's messy.
No matter what we do, things do slip out.
In fact, in the first 48 hours of Madden launching, there is more testing done on our product by the gamers than we do an entire 10 month development cycle by ourselves.
So one of the big processes we have in place today is test automation.
You can't hire enough QA people to test your game for the scale and the size of games and complexity of games today.
So we do lots and lots of test automation.
And basically the goal is to get hours and hours and hours of testing through automation on hundreds of dev kits that sit idle at the studio at night when people are at home.
There's one big piece that I want to talk about with test automation, which is when we talk to engineering, verification, and validation, testing is a means to compare the execution, the results of the test against the expected results for that test.
And one thing that we're starting to do a lot more at EA is we're kind of looking at testing where automated testing in particular where kind of the only expected result is that the test completes.
That test may take five minutes.
It may take 50 minutes.
All we're doing is looking, hey, did that test complete.
And within that span of say 50 minutes, there's all kinds of other things that went on.
And we're not really testing whether all those things between here and here met what our expected results are.
So really, this is about measurement and functional quality measurement, which starts with technical KPIs.
Over a 50 minute test, did the game run at frame rate?
What are the load times between your different modes?
Did they meet what our expected requirements were?
Obviously there's crashes, desyncs, and disconnects, which are very abrupt failures for that test where it won't actually complete.
But there's all these other technical KPIs where the test will continue, but it may not have been doing what you'd expect it to do
[INAUDIBLE]
So what we're starting to do more now is measure and make sure that every build so that every test where an automated test, no one's sitting there watching it.
I may not know.
If a QA person doesn't sit there and watch that automated test, we don't know that it's not running at target frame rate unless we measure it and report it.
And with online services today, you can do a lot of measuring and automated reporting, capture all that data, and create dashboards that can show, all right, what's the performance of my Game this is one the first ones we put in over the last couple years.
We've always had performance analysis.
But what we now have is automated solutions with very detailed level of reports, so that every time someone plays our game, we're getting frame rate reports out of that game.
And you can drill down a little bit to look at for Madden by stadium.
Every game that's been played in each stadium, what does the frame rate look like?
And look and see are there certain stadiums that have problems.
And you can even look at an individual session and say, all right, was there an individual game session that had a strange frame rate problem?
Now beyond our technical KPIs, those key performance indicators, there's other types of functional issues that we need to look to measure.
Bugs do slip out into the wild.
This was one that got a lot of publicity this year, the tiny titan.
UFC had some well known glitches in its game where a lot of funny animation and physics problems.
Inevitably when these types of things happen, if you're familiar with the notion of asserts, sort of non fatal errors or warnings that engineers put into their code to catch non fatal problems and warn them that, hey, you might want to look at this.
I guarantee when both of those problems showed up in a debug build, an assert probably would've yelled at a developer and m, hey, something's wrong.
So there's this class of non fatal functional problems that are pretty severe, because if they get out into the wild, this breaks the immersion and ability to experience emotion when you play the game.
Or at least in some cases, this one's probably a negative emotion.
We need to start to shift our focus on as you build engineering systems.
This is, again, as an engineer, what you bring to the table is this methodology and process of verifying and validating against functional requirements.
Make sure as you're building your features, you're thinking about testing and you're thinking about measuring.
Is this doing what it's supposed to do?
Because I've got a code base of 10 million lines of code, 200 people contributing to it.
There's a lot of complexity.
And you never know throughout that software development life cycle what types of problems might show up.
So as you're going into your design and requirements, start thinking about what types of failure modes are critical that I should be measuring and reporting through online systems.
We're starting to do a lot more of this at our studio.
So in summary, we really talked about two things, code review and testing.
And specifically measurement when you're doing that testing.
But it's really about, for me, an engineer's responsible for driving the functional quality of a product.
We write the code, we write the specifications.
We have to drive the functional quality of game software.
And a well executed project has both the technology and processes in place to do that and make sure that you're able to take that big project and get it done by August every single year.
So that's it.
I think I maybe talked a little bit longer than I originally expected.
But we've got a little bit of time for Q&A.
I think I was told you guys take a break at 2:00.
But you can go ahead and ask any questions and I'll answer anything I can.
When we take the break, if one on one if you want to ask more questions, I'll probably hang around while you're doing play testing and can talk more.
Any questions?
[INAUDIBLE]
Today it is all C++.
100%.
I did a talk yesterday sort of to a general audience about how the industry's evolved.
There used to be a lot of assembly language intermixed with C++.
We don't do that anymore today on today's consoles.
And there is in data we do do Lua scripting.
There is ActionScript.
There is shader code and shaders for graphics programming.
I bucket those more as data.
And they tend to be-- we have a Lua interpreter that's written in C++ the can load Lua data scripts and process it.
We have an ActionScript interpreter that loads ActionScript and processes it.
So generally when I'm talking the 10 million lines of code, it's the C++ game engine client.
But it does interact with other types of code that we consider data today
So I've been in code review situation all the way from [INAUDIBLE] mostly automated through some sort of code tracking system or commit system where someone's about to commit something, [INAUDIBLE], or all the way to the other end where someone just grabs me and makes me look at their screen.
How does that work?
Can you walk us through how review works?
Sure.
The policy is all code gets reviewed.
With that kind of policy, there's a lot of flexibility in how the teams actually implement the process.
But the tool set we give them, we do have a server.
We actually use a software from SmartBear called Collaborator.
And when you're going to submit code to Perforce, we use Perforce for source control, you upload the diffs to Collaborator and assign that to a reviewer to go look at those diffs.
And they can mark it up, put little bugs and comments, send it back to you, and you can kind of go back and forth.
So it's one option.
That's how 90% of the code review gets done.
But the thing I tell the teams is you don't have to do it that way.
In fact, having someone come to your desk and talk to you and look at the code over your shoulder can, in a lot of cases, be much higher value.
Because that can help set some context on why did you write it this way, and you can have a conversation.
So that's an option as well.
And you also have the flexibility, depending on the situation, sometimes code gets reviewed after check in.
It doesn't have to get reviewed prior to check in.
That's kind of up to the teams, up to the urgency of the check in, the seniority of the review.
But the request is that all code gets reviewed.
Again, it kind of hit the four bullet points on why.
So I guess basically it depends.
It is game development, so we try to be a little informal about it and give you the flexibility to do the review the way you see fit.
But provide a lot of tools and process and formality kind of through a prescribed automated system as well.
I think there was a question in the back
Yeah.
How much additional work is for you to do things like hoarding the same game to Xbox, PS1, [INAUDIBLE]
For today's consoles, Xbox One, PlayStation 4.
You're really starting to get to hardware that's converging.
It's becoming more PC like.
Probably the biggest difference is actually going to be in the operating system.
Each of those consoles has an operating system.
Xbox One ends up being sort of flavor of Windows.
And PlayStation 4 ends up being some variety of Linux.
So you've got this operating system and driver layer that is different.
But with the way EA's structured, obviously we're cross platform.
Madden, when I showed the charts earlier, actually shipped on seven different platforms that one year for Madden 13.
We've always built cross platform support.
I characterize it as about a 70-30 to 80-20 rule, where the majority of your code has no idea what hardware it's running on.
And there's just kind of this base layer that interacts with the operating system and the drivers and does what it has to do specifically for the console.
And so it's call it 80-20 in terms of code and 80-20 in terms of effort.
So that 20% layer generally gets built by our central technology groups, whether it's the base level rendering layer, the base level file system layer, the base level of memory management layer.
At EA we've got central technology where that gets built centrally and all teams use it.
And then the teams take that technology and build their platform agnostic game on top of it where it kind of doesn't care whether it's Xbox One or PlayStation 4.
But then the other space that is different is just really, I'd say, performance.
Each of the systems has slightly different bottlenecks in terms of what's fast and what's not, whether it's memory bandwidth, shader pipelines.
They have slightly different strengths and weaknesses.
They're tuned differently.
So you tend to have to look at your content and your data and structure it in a way where it's optimal for the two systems.
In the space of just kind of art contact, how many polys, how your textures are structured and bundled, how you write your shaders especially.
There's quite a bit of space, quite a bit of, I'd say, platform specific optimization work to make sure we're really taking advantage of the different bottlenecks in the system
Why did you choose to join [INAUDIBLE] in the first place?
To be honest, I had no idea that you could make a career in video games.
Like I said in the intro, I wasn't looking to go work on video games, mostly because I didn't know that you could.
One of the games I played the most growing up actually was Madden.
In fact, really football games.
Anywhere from Tecmo Bowl on the original Nintendo Entertainment System to Madden on the Sega Genesis.
I played a lot of NFL football games growing up.
It was actually the NFL football games that I played that got me interested in the sport of football versus the other way around where I was a fan of the sport so I played the games.
It was actually the games as a junior high, high school student that I really enjoyed playing those football games the most and enjoyed beating my friends at them if I could.
And so when I got contacted by the recruiter where I'm going to go work at Lockheed, hey, come work on the next generation of interactive entertainment, make a football simulator, I was like, well, that would be great.
I used to love playing those games and was really excited about coming down and getting to work on some of the games that I loved playing when I was younger.
Anything else?
Do know of SB Nation's "Breaking Madden" series?
And if so, do you have any opinions on that?
They deliberately set the game up with extreme characters, like 90% skill everything and 0% skill everything
I did actually.
I think it was probably about two weeks ago I watched that.
So I do know what you're talking about.
My opinion, I think that's awesome.
Madden and EA Sports, at its core, is all about authenticity and realism.
EA Sports, it's in the game.
And we're pretty serious about that authenticity, realism, almost to the point that, it's just me talking as a gamer, where we tend to be a little too serious.
And I kind of like that kind of stuff where it's more just having fun with the game, which is what it's all about.
And giving, I think, the content creation tools in the hands of the gamers, where you're starting to see a lot more gamers love to create just as much as they love to play.
And giving them the flexibility that they can go in do, it's an open sandbox, do whatever you want.
We've maybe put a few walls and barriers around.
But I would love to be able to put into Madden, hey, let's tweak the gravity.
Let's really change up the physics.
Now, the core of madness is about authenticity in realism.
So there's certain things that we're not going to deviate from that core.
But what little flexibility we can put in where you can break Madden and have a little fun.
Because ultimately it's games.
You want to have fun and that's what you want to do.
Personally I love to see that we've got that support in there.
And that's where the tiny titan that slipped through.
That wasn't intentional.
I think we later just sort of having fun and laughing at ourselves we went ahead released a ultimate team card pack that allowed you to unlock that as an intentional bug.
But the original case, it was actually another one of those we make a change and in our excitement about getting that change out to our gamers, we move fast, maybe don't have all the processes in place that we should've.
Someone does some rough testing, looks good, push it out, and something slips through.
But it's a big game.
When you start talking about that program testing and verification and validation, there are so many different permutations that have to be covered.
So many different modes.
It's not infinite, but it's a very large finite number.
And it's almost impossible to test everything, which is why we're trying to start to get to a lot more rigor in our testing and how we measure and verify that everything is on track.
Let's go ahead and take a break.
And I think, like I said, if you have a question that you just want to ask individually, I'll be here to talk a little bit longer.
So thank you.
[APPLAUSE]
So the clock says 2:08.
Back in this room at 2:18.
We're going to give you time to start planning.
You're going to run two tests.
And then you're going to have a little bit of time to reflect on your test and just a really quick tell us what you learned today.
OK, it's 2:18 or thereabouts.
Schedule for today is you've got about 15 minutes to do planning.
That means you should know what observations you're going to make when you're doing these tests.
If you're going to do questions, either creating a quick survey or having questions you're going to ask your testers.
Just like we asked for last time.
You're going to spend 30 minutes doing a play test.
That means you're going to set up at least three computers with three observers to run those tests.
At 3:05 I'm going to say, hey, switch.
That means anybody who's doing an observation is switching with the people who are doing testing.
Everyone should have a chance to both observe and play at least three or four games.
That lasts for another 30 minutes.
At 3:35, you're going to time in teams to complete your focus test reports.
And then we'll just ask you a few questions about what you observed, but no formal presentation today.
We just want to kind of know what you saw today.
Questions?
All right.
So I'm going to yell at 2:35 to have your play tests ready to start by 2:35.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR 1: It's 1:06.
We're going to start presentations at 1:10.
So Ghost Maze just get started.
You don't have to come down here yet, but just make sure you're ready.
So just a brief going over what we're doing today.
Presentations, I hear a lot of really good energy in the room.
So I hope to hear that when you come down and tell us about your games and all the foibles, all the things went wrong, all the things that went right.
Hopefully this is a really good prep for you for our next assignment, which is going to be much, much longer.
So after we get through with presentations, we'll take a brief break.
And then we're going to just jump right in to project 4.
I'll introduce project 4 again, going through all the boring slides with all the deadlines and stuff.
When I get to that, if you want to fall on Stellar, I just updated the project for assignment today to make sure that the deadlines were correct.
And we'll double check that again as well.
Unfortunately, Pablo who came to us in the beginning of the semester, is not here with us in person.
But he will be here with us on Skype.
He's in Geneva right now, but he'll be talking to us about what they're looking for project 4.
In particular, we're going to be introduced to six different game topics, game design topics.
And you're going to choose one of them.
We're not necessarily making your choice today.
We're going to brainstorming today.
We're going to have people form up in a brainstorming groups around these topics to talk about them.
And then you will have over the weekend to talk about it on the mailing list, to read some of the resources and some of the materials we posted to Stellar to see which topic you're most interested in.
We're going to start the day off Monday with forming teams.
So a little bit longer time to form teams, bit longer time to really think about the topic and think about what the project's going to be-- again, you have more time to actually do concepting on this project.
It's an eight week project.
Nine weeks, if you count spring break, which you shouldn't.
We designed it as an eight week project.
What's that?
Not spring break?
PROFESSOR 1: I said no.
Yeah don't work on spring break.
Like you were going to.
Any quick questions before we start?
We've got two minutes left.
Any quick questions you might have about project 3?
About turn ins?
What did you mean [INAUDIBLE] spring break?
PROFESSOR 1: I said don't work during spring break.
We designed this as a eight week project.
Project 4 is an eight week project.
We've designed it so that you are not working on a project over spring break.
What's that?
You could give me some details about what I'm screwing up over here.
So do we have a fall break?
Don't we have a week long break?
Thanksgiving break?
It's 70 degrees outside.
I'm in a different place right now.
All right, I said Ghost Maze, come on down and make sure your projector, if you're going to use a projector at all, works
[INAUDIBLE].
PROFESSOR 1: What's that?
[INAUDIBLE].
PROFESSOR 1: I still didn't understand
[INAUDIBLE].
PROFESSOR 1: No, just like 4, you only need as many people as you need to present to come down.
STUDENT 1: So if you remember, our game was Ghost Maze.
And we'll run you through a quick refresher of what the game actually was first so you remember what it is.
STUDENT 2: So here we have our character in a maze.
You've just woken up and you're in a dark maze.
All you have is your lantern.
And so your objective is to explore the maze, avoid ghosts, and drop beacons to help you explore the maze.
And find a key, which will then unlock and exit for you get out the maze.
And so there are different effects that happened during the games, such as your sanity starts decreasing as the games goes on.
And then randomly, as you're sanity starts getting lower and lower, your movement will get impaired randomly, and the light will turn red as you can see in this picture here.
And then another interaction that we had was when you get close enough to a ghost, your sanity start decreasing faster.
So the light turns blue to let you know that you're too close to a ghost and you need to get away from that ghost.
STUDENT 1: So a few things that went right was that it was hard to balance everyone's schedules, but everyone was pretty flexible in terms of what features they were working on.
So people would adapt to the features that needed to be done at that time.
So flexibility was, I think, something that went more or less well, in terms of working on what needed to be worked on.
The second thing that went right was user experience.
So we spent a lot of time focusing on user experience, and playing with the camera, playing with shadows, playing with the player movement.
We spend a lot of time putting in graphics and audio so that it seemed a little more realistic to put you in the context that you were in a dark maze and lost.
A few things that went wrong is we started out using Flixel.
But it took us a few days and we still couldn't get up a dev environment reading.
So we decided to switch over to Unity.
That was a pretty wise choice, but we made it pretty late.
We found that collaborative coding on Unity was really hard.
Certain changes to the main scene would wouldn't merge well.
So also when we were working on different branches, and then tried to merge it back into master scene, people's changes were lost or overwritten.
So we found that Unity wasn't great in terms of like having a big team project where a lot of people are coding at once.
And sometimes we just re-applied changes on to the master branch.
Something else was we all had really busy schedules.
People were out of town.
In town, they could only work on the project at certain points.
So I think something that helped with that was at the beginning, we did a pretty good job of delegating tasks to people who work on the project earlier.
But then later on it, it was pretty haphazard in terms of who is available at one of time.
So maybe re-evaluating priorities as the project went on would've, I think, really helpful.
STUDENT 2: So lessons learned, we spent a lot of time working on user experience.
Because we had a 3-D perspective game, there was a lot of thought that went into how the light was hitting the player, how to get the shadows to cast properly, how the player knows that the players in the maze.
And being able to move around and understand how your sanity is decreasing.
We spent a lot of time testing that.
Working early and quickly-- this was partly hindered because we lost at least four days of work because we were trying to get Flixel set up.
And then it just didn't work.
And we had to switch to Unity, so basically starting over from scratch.
Working early and quickly is super important because we ended up having to drop quite a few features because we didn't get enough done on time.
And we also learned that working in 3-D is very, very hard.
We were originally going to do a 2-D game, and then thought, oh, how hard would it be?
We have a 2-D game, but let's just give it a 3-D perspective.
That was not a good idea.
Doing the textures took a long time.
It ended up looking really, really nice, as you can see in the pictures.
But the time investment was a huge investment that we could have used to put in some more complex mechanic into our game.
And future improvements, team communication and availability is a huge one.
We, as Rachel said, we had a task assignment at the beginning.
But then due to people's schedule, we didn't really take that into account when we made the initial task list.
So people ended up having to pick up other people's tasks.
And it became very hard to figure out who was working on what.
And scheduling meetings was near impossible.
Getting familiar with the game engine-- using Unity at first was very hard because we weren't sure how scenes work.
And eventually we figured out that if we make most of the stuff dynamically generated then we can just worry about the code, and not have to worry about all the user interface stuff that Unity does.
And that would have made our game develop a lot quicker.
And setting deadlines was very important.
We had a lot of tasks laid out.
But we didn't set strict deadlines for when we wanted each task to be assigned, or when we wanted each task to be finished.
And that would have helped push along our game and get some of them more mechanics that we wanted in it.
STUDENT 1: Yeah, I think something that a little bit arose was when we were going to plan on doing our testing session because we should have set the internal deadlines to match the testing sessions so that we get as much feedback as possible.
But that said, we still got something out of the testing sessions.
So that was good.
I think that's all we have.
STUDENT 2: Yeah, that's it.
STUDENT 1: So I don't know if you guys have questions.
STUDENT 3: I'm from team Hardcore Dragon.
And as a refresher, I mean our game is about being a hardcore dragon.
So to us, that meant razing villages, being chased down by knights and accumulating a horde of gold that you have to protect from looters.
And we all were pretty excited about this idea.
And that was probably the best thing about the project, as a whole, is it was a fun idea.
Even when we got bogged down with work from other classes, or something emerged, or didn't go through correctly, or there's was a bug, and we couldn't understand what was happening, it was still really fun to be like, OK, I really need to make this dragon set this village on fire.
And I want to do this because that's awesome.
And another thing that worked is that we use Gitter, which integrates with GitHub.
Whenever something is pushed, it'll show up on to the right and you can link to it.
And everybody who's on your project is on the Gitter chat.
So it was easy to keep track of who was working at what time, who was online at that time, and who you could ask for help to let you debug whatever was going wrong.
Another thing that helped was Trello is actually really good tool.
The last project, I personally used Google Docs.
But Trello is much easier to look at something, drag it while I'm doing this right now, adding yourself to things, and breaking it down to simpler task.
It's all very easy on Trello.
I mean, it's built for this.
So it made long-term goal planning a lot easier.
And you could see at this moment who was supposed to be doing what.
Unfortunately, the idea was also a downside to the project because it was Hardcore Dragon.
I want to do so many things with this idea.
When I blow fire on this thing, I want it to turn into a crumbled building and become a pile of flames, and it'll be awesome.
That's not necessarily at the top of the feature list, obviously, because we have other mechanics that are more important than that.
But we kept coming up with these little additions that would make the game a lot more cool or awesome, but are hard to prioritize and just way too much to stack on top of the game.
So that was the downside.
And it was just hard to control-- it was hard to scope the game down to the five minutes game we needed to create for the project.
Gitter was good with the [INAUDIBLE].
If everybody was on and everybody used it all the time, it would be ideal.
But since a lot of us are on the go and working on other things, in classes while people are working, using something else might have been better, like g-chat or Facebook, something you can access from a smartphone, or something more instantaneous.
We would just get-- if you weren't on Gitter at the time, it would send you a notification being like, you missed these messages.
Which is almost as delayed a message as email.
So it was kind of pointless other than the fact that it was integrated with GitHub and you could see who was online at the same time as you.
Another thing that didn't work is that meetings are really, really hard to coordinate with seven people.
And what ended up happening was that we would all be like, OK, so half of us are free this day, six of us are free this other time.
Let's meet then.
And then we wouldn't decide a specific time within the time were free to meet.
And then by the time we noticed this, ti would like that day.
And it was just hard to coordinate everything after it got to that point.
So for the future, I believe we would have to have a fully dedicated scrum master.
And I really mean fully dedicated, not scrum master AKA programmer on the side, or scrum master who also gets music done and play tests, well it does play testing.
Everybody play tests, but a fully dedicated scrum master to make sure that everybody's on tasks, that the meetings will get scheduled, and that the tasks are prioritized.
And if something falls through, they're the person to talk to, they're the person who's going to make you stop freaking out that this task took like three times as long as you've estimated it for.
We didn't have this one person dedicated to that.
Everybody was like a jack of all trades in our group.
So not having that one person to talk to, go to, made task prioritization really fuzzy, and made everything kind of come together really late.
And it was hard to hold people accountable because there wasn't that one person who was on top of everybody.
Another thing we had to do earlier would be feature cutting.
Because if you don't cut features definitively earlier, you'll still have it lingering on some people's minds.
Oh, I have to do this so that in the future we're going to maybe use this feature.
But if you cut these features early, you limit the propensity to have bugs, or stuff like that.
While it's unfortunate that you have to cut features, it's necessary.
So doing it earlier is the best time to do it.
Another thing is, like I said before, a native messaging system.
Gitter was cute but didn't actually work in the way that we wanted it to work.
It wasn't instantaneous between all members.
It's hard to find a messaging system that everybody likes.
But I think something like G-chat or Facebook messaging, something instantaneous like that that most people use already should be used as a group so that everybody can be on the same page and have access to the instant messaging.
Yeah, our game is there if you want to play it.
Anybody have questions?
PROFESSOR 1: So if you were to do the feature cutting early, did you feel like you had the information in order to do that?
STUDENT 3: So there was some stuff that like after even half the halfway point, that we could've cut features.
But instead of the halfway point, we did like two days ago, which is too late already.
So even earlier than that.
It doesn't have to be like day one feature cutting.
Just at maybe points throughout, be like, now that I'm here, what features look like they're not going to get through.
And with a dedicated scrum master, who's like looking through, looking at people and how they're struggling with certain tasks, maybe people don't know exactly what their pace is internally.
But from an outside perspective, a scrum master might be able to more accurately help with estimates and stuff like that.
So a dedicated scrum master would help with early feature cutting as well, probably.
PROFESSOR 1: This is a Unity project, right?
STUDENT 3: Yeah, a Unity project.
Yeah, sorry.
We actually didn't have that many problems with merging because we had, I think, at least one of us was really experienced with Unity, so that's was lucky, I guess.
PROFESSOR 1: OK.
Thank you.
STUDENT 3: You're welcome.
STUDENT 4: So our team is MIT Simulator 2014.
So our basic concept was you play as a certain dean of MIT you may all know.
And you're making decisions that trade off between the endowment, wealthy donors, and student approval.
And the basic goal of the game is to try and survive as long as possible, while shitty things happen to the institute.
So what went right in our project?
I think what went right in our project is we connected with our audience.
A lot of MIT students know what it's like to be at MIT, and they know the setting, they knew the administrator We're not really pointing at, but we kind of are.
And so a lot of people connected very well with our game.
And so they enjoyed it and were able to understand it easily in their own terms.
So the second thing that went well is that humor works.
A lot of our things in our game were like kind of unpolished, and kind of like slapped together.
But because we had taken the time to make them funny and make them entertaining, people we're OK with losing all their money in one turn [INAUDIBLE].
Because, hey, the description of it was funny.
And so the third thing that well is that simple is OK. We ended up having like very few core mechanics.
But then because of the setting that we had and because people connected well with our game, I think people are still able to get-- we think people were still able to get enjoyment out of the game because of those.
So now what went wrong?
So first, Hex Flixel, AKA what do you mean it's not on Stack Overflow?
So Hex Flixel, while it did play well with version control and I heard some horror stories about Unity version control, Hex Flixel played pretty well with version control.
But if you ever had an issue, you were pretty much out of luck.
You could search on Google, and you'd get like zero results.
Sometimes the official documentation wouldn't even turn up on a Google search result.
So documentation was a little bit lacking.
I think the second thing that we could have improve on is that paperwork is not management.
I think in the class in particular, this isn't emphasized enough, is that you can do all these agile development methods, like you can have a sprint task list and everything.
But unless if you actually have people who are actively taking charge and managing a project, I think stuff still won't get done.
I think that's something that could be improved in the future in this class is like spend less time worrying about paperwork, spend more time actually like teaching about how to handle personal situations.
Like what happens if someone doesn't have time.
Or what happens if somebody like does this?
And so learning those managerial skills isn't a trivial task.
And I think in the beginning, we underestimated how much those issues would be a roadblock to progress.
And then the third thing that went wrong I guess is that responsibility is shared.
So even if somebody has a role assigned to them, we should all still like take the time to make sure that like everybody is completing their role.
So just making sure that people are checking other people, to make sure tasks don't fall between the cracks.
And so overall people do like the idea of playing as a certain dean of MIT.
Any questions?
PROFESSOR 1: So what would you do different?
Not necessarily things about the Hex Flixel, [INAUDIBLE].
But when it comes to your management, managing stuff we talked about, what would you want to do different, or what [INAUDIBLE] particular problems that you had.
And you know you're going to have a feature that you need to have scrappings for?
STUDENT 4: I think one thing that we would do differently is like take project management more as like a people problem then like a paperwork problem.
Because I think like we kind of just assume that by filling in the sprint tasks list and by writing git commit logs, we magically work together as a team.
But I think in the future, just taking a more active role in insuring that people are interacting with each other and communicating.
PROFESSOR 1: How about if I'm [INAUDIBLE] the responsibility of sharing things, like how would you address that [INAUDIBLE]?
STUDENT 4: I think for the responsibility being shared thing, I guess this was just in reference to a few particular instances where like we had someone who would claim that they would do something.
And then because we assumed oh, because they claim that they would do it, that it would get done.
But I think sometimes we got too complacent and then thought that it would actually get done anyways.
But I feel in the future maybe just sending out a couple more emails, just having people confirm that they've finished things, require that people send out emails confirming when they finish things, so we know that this thing didn't get submitted yet.
Oh, shoot class is in an hour, or something like that.
So just like encouraging people to either like poke each other more, and also to be more responsible about like when you submit something.
Or when you do something important, you email out to everyone just to show that you've done the work.
PROFESSOR 3: Would email be primary means of communicating between the team or was there face to face meetings outside of class, or [INAUDIBLE]?
STUDENT 4: So email was our primary means of communicating.
We had one face to face meeting outside of class.
And then there was another one that was supposed to be scheduled over the long weekend, but things happened.
One or two people were gone for the weekend, and then things fell apart
[INAUDIBLE]?
Thanks.
STUDENT 5: This Is our game Build a Spaceship.
It looks like this.
You have your ship with a set of components of a bunch of components that you can add to the ship.
You click and drag them onto the ship.
And then you send it out on a mission.
After you go out on that mission, you'll get a log that looks like this.
All this text is really small, so you can't see it.
But it says things like, you met some bandits, you had a battle, you won and earned this much money.
Or you got hit by asteroids and lost this much health.
And the point is that after you design your space ship and sent it out on this mission, you'll get all this feedback telling you how you ship did so that you can then update your ship and send it out again, and hopefully make even more money.
The goal of the game is to complete a series of missions that we have in the game currently, and make as much money as you can while doing it.
Of course given more time, we would do things like add more components, upgrading your ship, and other tweaks like that.
But this is the core of the game right here.
So what went right on our game?
We made a prototype really early, which was great.
That helped us all get on the same page and understand what we're talking about when we're talking about the game.
There's was point before that where we had all these cool ideas from brainstorming, and a vague idea, a personal idea, really, of what the game would be.
And that made it really difficult to communicate.
So once we had that down, everything just became way easier.
We did a lot of UI testing.
Even before we were sure about the game would specifically be, we knew that was going to involve adding components to a ship.
And so we knew we could UI test that and see how people manage with different ways of adding those kind of components, or customizing their ship.
Hex Flixel, we found it really simple to work with actually.
And whenever we needed to add more screens or make changes the game, we all had a pretty intuitive understanding of how that would go.
That might also be an artifact of coming over from Phaser, which is very similarly structured.
And on our team, we got lucky because everybody had coding experience.
And it made it a lot easier because we were all speaking the same language.
And so on a time constraint project like this, it meant that when we talked, we were more efficient about it.
That's not to say that there is a place for people who don't have coding experience on these teams.
It's just that when we only have two weeks to get things done, the speed was a huge help.
What went wrong was communicating our progress on the game.
We had a lot of problems where we wouldn't talk about what was that done.
We didn't know what was even in the game because somethings weren't clear yet.
Like there was one part where we didn't know that we could actually add components to ship because somebody had added that in but not actually told us.
And the UI wasn't there yet to actually make it clear.
We also run into problems with heroic efforts where the night before something was due, somebody would like to come in and put in all this effort.
And we didn't know what they were doing, and we didn't know what was getting done because we just weren't communicating about that.
A problem that's related to this is that we didn't assigned tasks well or take ownership of what we were doing, or assign responsibility between people.
And so we ran into problems where things just wouldn't happen for a long spurts of time.
And then we'd run into the heroic effort problem at the end where we're all jumping in to make things happen.
We also ran into problems where sometimes people didn't push commit for an entire day and we had no idea what the state of the game actually wa So what we learned from this, and what we would do differently next time is first we'd keep track of our progress and make sure that it's contiguous, whether it's using Trello, or emails, or any other sort of solution for it, it would be really important to make sure we all know what the state game is, and what's being worked on.
We should come up with standards for how we're going to email and meet.
We talked about this, and said it would be a good idea, but then we didn't actually do it.
So it would be good to have a schedule of when your team is going to meet, how they're going to email and why they're going to email, if it's a daily update of what you're doing, or if it's every time you push a commit.
And also coming up with standards for how you're going to Git.
Are you going to use feature branches, and all these kind of things, or are you going to do it all on master?
And are you going to push every time you make a new commit?
Or are we going to push at a set time every day so that we can track it all at once?
If would be good to know who's doing what.
That comes in with the idea of communication and tracking progress.
What we did at first was we had specific domains that people were working on.
Jeremy was working on the UI, Rodrigo was working on the ship model.
And this made it really easy to assign tasks and to make sure that things were getting done in the beginning.
But then in the end, when we had some tasks that were kind of in the middle and we didn't specifically say what was going to happen, we ended up with this last minute scramble of, OK, I'll take this thing.
Actually I'll take this thing.
Somebody else is coming in.
And it made a lot more difficult to track what was actually getting done.
You can also give tasks to people.
We were kind of reluctant to do that at first.
But we got better and it, and it was actually super helpful to just be like, hey, you take this.
And then it's completely clear what happening.
And the very last thing to do with this is to just ask what's happening.
It's good to get some reassurance that something actually is happening if you're not hearing anything about it.
So you can just ask someone.
Hey, what are you working on?
Or hey, is anybody working on making this component up here?
Then you just get an answer.
And you get a lot of peace of mind from that.
It's good to set up specific times to work together.
We didn't do this enough, but it would've been great if we had.
And like I said, we made a prototype pretty early on.
But I think we could have made one even earlier.
We found that it didn't limit our creativity or our ability to talk about the game.
Instead it just gave us a platform to work off of.
So we could build a basic prototype with some idea of a core mechanic, and then continue to brainstorm and expand from there.
And that's basically what we learned from this game.
And moving forward, we're looking forward to implementing this on project 4.
Any questions?
The site's solving problems with HaxeFlixel all on your own, did you find any online resources that turned out to be helpful?
Or did you just [INAUDIBLE]?
STUDENT 5: The main problem that we ran into with HaxeFlixel was we were trying to build an HTML5, and the text just wasn't working.
And we found nothing about it.
We found one page by HaxeFlixel that said, sometimes we have problems with text.
Try doing this.
That didn't work.
And nobody else has an answer for it.
I don't know if anybody else did
Basically you have to read the Flixel documentation and ignore the HaxeFlixel documentation.
[INAUDIBLE] identical and the static typing does save you a little bit.
And you have to jump into the source occasionally.
There's nothing wrong with that.
STUDENT 5: Anything else?
Thanks.
STUDENT 6: So we did Looking for Deena.
And we have some screenshots of the games in case any of you didn't get to play it or see it.
So it's just a search screen.
The first thing-- so the game is kind of in two-- you have to do two different things.
The first thing you do is you choose which planet you're going to go to.
And you can't really read it, but they all different distance stats as well as stat games that each planet will give you.
Then you have to navigate through an asteroid fields, which depending on how far along you are in the game, makes it harder as you go along.
But hopefully you have those stat games that should make it easier to actually go through it.
You're looking for Deena.
And you can lose and win.
So what was right and what was wrong is we did communicate a lot.
And by a lot, I mean from midnight to like this class, there's 108 emails.
And that was just today.
So we communicated a lot.
But we didn't communicate really effectively.
There were a lot of people saying like, this what I'm doing, this is what I'm doing, this is what I'm doing, this is done.
I submitted this.
But then going through those hundreds of emails trying to figure out, oh wait, who said they were doing this, who said this was submitted, was really hard.
And so we didn't have those daily scrum meetings that we talked about.
And I'm realizing those would be really useful to have somebody say like, OK, this is what everybody's done.
This is what everybody's going to do.
This is what we've submitted.
Having that one thing somewhere where everyone can view it easily would make it a lot easier than trying to go through email and figuring out who did what and what went wrong.
One other thing we did right was we went in smaller group.
So people were working on the design, just those people met.
And then people working on the code, just those people met, which was really useful for trying to meet.
Instead of having seven people meet, you only had like three or four.
And so much easier to arrange those meetings.
We really divided the tasks effectively.
Everybody was doing something.
And that's what the last point is.
Everybody tried to make sure they were always doing something and contributing to the project.
No one was just sitting there wondering, what do I do.
Everyone tried to constantly be doing something.
Another thing that went wrong is that we worked in bursts.
People were really busy.
And then the night before we assigned an internal deadline, everyone's was like, oh OK, let's work on this right now.
And so then it was just people staying up all night to try to finish something.
Which things got done, but sometimes it's like, did we do this the best way we could.
Was everything optimized?
And we weren't really sure because everything was done in short bursts, instead of like a little bit of every day.
And do you guys want to talk about Unity?
STUDENT 7: Yeah.
As people have said, Unity is annoying because it has all these binary files that you cannot merge.
So if two people were-- we had to basically always keep track of, I'm working on this scene.
No one else touch it.
Because if two people work on it, someone is going to lose everything in it, which is really annoying.
Emails worked for that.
And so that way you could say, I finished working on this scene.
Someone else can touch it now.
So that was sort of helpful.
STUDENT 8: [INAUDIBLE] spits up like five of them.
And you're like, I think it's fine.
But sometimes a conflict happens.
And you're like, I don't know what to do with this.
I think I can check all of them.
But then things break and you're [INAUDIBLE].
I feel like that scheme is weird in the scene because at least we tell each other, stop touching the thing, or touching the thing.
But something you touch the thing and something else happens entirely.
And you're like, OK. STUDENT 7: I also want to point out how impossible it is to schedule anything for seven people.
We wanted to meet on the weekend.
And there was literally not a single period of time, not one hour that wasn't at 3:00 of 4:00 in the morning where everyone was asleep.
It was impossible.
It was insane.
STUDENT 6: Yeah, schedules are hard.
STUDENT 8: [INAUDIBLE].
STUDENT 6: Any questions?
You have two points about [INAUDIBLE] items up that appear to [INAUDIBLE].
STUDENT 6: So we did keep to deadlines.
So what I was trying to say with that point is that we did kind of keep the deadlines where we worked in those bursts to get to the deadline.
But then because it was such a long burst, by the end everyone was drained.
And if it was like, I'll do this, and this other little small thing, a lot of little small things didn't get done.
And so things were kind of push back a little bit.
And so this led to more bursts.
And it was a really bad cycle that I think ended up happening
Do you think if you planned for the bursts, it would be better?
If you actually scheduled meetings at 3:00 before?
STUDENT 7: So I think we need more granularity in the deadlines.
So we'd say, let's get this bucket of things done by this day.
And that would-- people procrastinate.
We have other work to do.
Well, I have this other deadline for this other class is due first, and leave everything for this till later, until you're finished with every other class.
So if you leave smaller tasks, more spread out throughout the week, I think that would enforce a more, I'll work a little bit today and a little bit tomorrow, as opposed to I'll work a whole time two days from today, which is what ended up happening.
STUDENT 9: We also didn't have that [INAUDIBLE] scrum master or project manager who would reinforce those deadlines [INAUDIBLE].
STUDENT 6: Yeah, you would just get an email saying I have something due at midnight.
I'll start working on this after midnight.
So a lot of times that would happen, and things like got pushed back because that person didn't get that done.
And so you kind of have to wait.
STUDENT 7: But I think we got it done.
STUDENT 6: But everything got done.
STUDENT 8: It's very cute.
I feel like that's what [INAUDIBLE].
PROFESSOR 2: [INAUDIBLE] you can [INAUDIBLE] a lot if it wasn't well organized.
Do you have any thoughts on how you might better organize your communications on the project?
How do still have that good level of communication, but have it be something [INAUDIBLE]?
STUDENT 7: So I think a big problem we had is we had just one enormous email thread, which became huge and monolithic and impossible to look through after a certain point.
I think it would have more helpful to break up into smaller email threads.
So like people that were working on a specific feature in code, not everyone needs to receive all those emails.
So it would have been, I think, really helpful for the future to maybe organize it in the send that if me and Jenny are working on something, we email each other a lot.
And at the end of the day, we send one email.
Here are the details that everyone needs to know.
You don't need to receive the 10,000 emails we sent before that.
So I have to say, keep everyone in the loop.
Don't leave people out of the loop.
But keep little pockets of communication so everyone's not flooded with all this stuff.
STUDENT 6: And even having the designated scrum master, I think, would help with that a lot.
Because then it's not even they're sending it out to the group email thread.
They're sending it to the scrum master and the scrum master makes one thing.
It would be really tiresome to do that every day though.
So maybe every other day or something.
I'm not really sure how to break that up.
Because it seems like you'll just end up having a lot of things, like a lot of reports.
Report this was done this day, this was done on this day.
PROFESSOR 2: Did you [INAUDIBLE] place to track where what tasks were done [INAUDIBLE]?
STUDENT 6: We had the Google Doc with the change log.
PROFESSOR 2: You did the change log.
But were people updating, changing, task lists or anything like that?
STUDENT 7: I think that was mostly done in like-- so we'd say, these are the sets of tasks that we want to finish today, or something.
And then I, for example, I'm going to work on this.
[INAUDIBLE].
I'm going to work on this scene.
No one touch it.
OK, I'm done working on the scene.
And then once everyone finished working on all the scenes, then we'd go back [INAUDIBLE].
STUDENT 8: It does feel like we probably could have done some weird joint thing where the sprint tasks list keep track of bigger things.
And then we have some kind of chat email system going on where we should stay in contact, especially with [INAUDIBLE].
At the same time, it's a little bit like , hm, what level of tasks, size, goes into the sprint task list?
Because [INAUDIBLE] something really small, it's like yeah, this is what we spent the last two hours doing.
[INAUDIBLE]
[INAUDIBLE].
STUDENT 8: [INAUDIBLE] task [INAUDIBLE] is hard
But what really [INAUDIBLE].
That's what, if you can figure out a way to have something [INAUDIBLE], whether it's just a list [INAUDIBLE] where you can say, hey, I finished this task, can sum it up [INAUDIBLE] task was based on that.
Or if you can go to [INAUDIBLE] troubleshooting or something
Thank you.
STUDENT 10: Hi.
So I'm going to be talking about Score High.
This is actually our start screen.
So just in case you don't remember what Score High is, it was a game where you played as an MIT student.
You went around to different buildings, did your homework assignments, ate, slept, and basically just tried to survive school.
So I'm going to start off by talking about what went poorly.
And the first thing we had an issue with was direction.
So this time, we didn't have a paper prototype.
We had an idea that we wanted to do.
But it ended up changing a lot right from the start.
And it became really difficult to stay on topic because we didn't really know what we were doing.
And very much related to that, we then had issues with scope.
We started off with a lot of unrelated features and a lot of components.
And even now in our final game, there are a lot of components and the way they interact is complicated sometimes.
And then finally, I think every single team has mentioned this, just planning ahead, and getting work done earlier, and planning time to work together-- the issues we had especially that came up later were a lot of issues with midterms, things going wrong, everything that could go wrong kind of went wrong.
And if we had started earlier, we wouldn't have had-- we would have been able to deal with those situations better.
So based on that, what was well?
Well, the great thing is that we did start right away just setting up everything we needed to work.
We set up our email thread, our Google Drive folder.
We decided on Unity first meeting, and we set up the GitHub project right away so that we could all collaborate.
And the nice thing was in our Google Drive folder we put our task list.
We really utilized that.
And our email thread, like other teams said, it did get a little long and complex.
But it was just so nice to have.
And later on, we actually also got a group text messaging service.
So that works really well dealing with that last minute hustle kind of stuff.
The next thing I went really well is I told you that our scope was way out there.
We realized that very quickly, and we just started cutting, cutting, cutting.
So we cut items.
We cut interactive AI characters.
We cut playable quizzes.
We just cut everything that wasn't key.
Because our game, even without all the things, already had a lot of components.
And then once we have a playable game, the next thing we did was we looked into feedback.
So one of the main feedbacks we got was our game was not visually cohesive.
It did not make sense.
The nice thing about Score High was that because you're playing as an MIT student, you're just getting work done, people understood how to play and what they were supposed to do.
But they didn't understand that.
So if you look at the thing on the left, you're probably all just staring at it being like, what's that.
The rectangles are buildings, and the bottom's the stats bar, on the right is the schedule.
It's confusing.
People didn't notice the stat bars, or they thought they were buildings, and things like that.
So we responded to that right away.
And we said, we need to clarify this.
We need to separate the status, separate the schedule, and make the map very clear, and pretty, and easy to get around.
So that was one thing we responded to, and I think it went well.
The next thing that we responded to feedback was we really listened to players, not only big picture, but on smaller things.
Players wanted WASD, so we put WASD.
They wanted a clear way to leave buildings.
They wanted to know what was going on when they were in a building.
So we actually inserted a progress bar that says, what's going on this building, how it's affecting you.
And then to leave, instead of pressing space bar, which people found unintuitive, they press a button that says leave-- much more intuitive.
Now given all that, we did make a really good game.
But there are definitely some things we would change if we were to do it again.
As I mentioned before, start with something simpler.
We did not design the game around the scope of the project.
And if we were to do this again, we definitely would.
We tried to bite off more than we could chew.
And related to that, minimize the number of components and understand how they will interact.
Because especially if you're trying to break down tasks, people need to know what has to be conveyed between different components.
Plan for the worst.
Midterms suck.
People get sick, or go out of town.
And laptops break.
Literally all of these things happened to us.
We're still waiting to hear, is the laptop working.
We don't know.
Somebody's laptop broke last night and they were working on something and finishing it up.
And suddenly all that work's gone.
So as I said, start earlier, plan for the worst, and just get things done.
Finally, it worked out really well.
And thanks for listening.
Any questions?
Any questions?
This was Unity?
STUDENT 10: Yeah, Unity.
I'm going to take that as no questions.
PROFESSOR 1: Great.
Thank you.
STUDENT 10: Beautiful.
PROFESSOR 1: Thank you so much.
And actually round of applause for everyone here.
You actually did a really great job.
I think we're going to give you more detailed feedback in a few days.
But I think one thing we heard definitely is by working in these large teams, you're coming across these issues that you don't initially-- you still need tools to get over some of the management issues you're finding out, the communication issues you having, the scheduling issues you're having.
Part of this is just experience.
That's kind of why we just throw you in the deep end, is just to figure out what are all the problems.
So that we could then talk about and have some common things to talk, how we're going to solve those things.
Looking at the schedule, we'll probably end up moving a couple things earlier.
Did you have a comment?
PROFESSOR 2: I just wanted to make one comment.
It's possible your groups might be slightly larger for the next project.
I know.
But you've got a lot more time because you've got seven to eight weeks.
And what I really want to encourage you guys to do is if you realize you're having communication and organization problems with your group, if you're not sure how to organize things, or if it's not working, stop by the office and talk to us.
I don't think we said this clearly enough in the previous projects.
But yes, our goal is, in fact, to throw you out into the deep end.
But we're not trying to throw you out in the deep end without at least a floaty to hold onto, or a safety rope or something.
We are happy to sit down and talk with you, a small member of your group, or your whole group about how to come up with some individualized solutions to your team's communication and organizational problems.
OK?
PROFESSOR 3: That's just a reminder, if you don't know where our office is, it's in the syllabus.
It's in building 26 on the ground floor.
You can see our rooms all have the MIT Game, that logo on it.
And what Rick was saying about tools, not all our tools are like mechanical tools.
Not all our tools are like are things that you put in a spreadsheet.
The things like the meetings that we've been talking about, where you are actually supposed to talk to each other.
It's important to actually talk to each other during those meetings, for instance.
Of course, if you don't have a way to be able to schedule those meetings because it's hard to get people together, you need to find a replacement to be able to have those opportunity to talk with each other.
You can't just say, well, we don't have time to do this thing, therefore we're not going to do it.
However, this being the final project, you're going to have a lot of flexibility to try out different ways to communicate with your team.
So what worked in your previous projects should give you a good basis on what you can do in the future, but may not necessarily work with your future team because it might be different groups of people.
And learn from the previous three projects about what worked.
Try it out.
And that doesn't work, try something different.
You have time this time around.
Although still, eight weeks goes by really, really, really fast.
So don't over scope.
PROFESSOR 1: So we actually are running really short on time today.
We've got a really long-- a lot of content that Pablo was going to go over with us.
So let's take a five minute break.
I'm actually going to skip the project 4 introduction.
And I'll put the slides up on Stellar.
And I'll give that on Monday morning.
It's really just me saying out loud what all the-- Monday class.
Don't take everything I say literally.
It's the spirit.
Monday at 1 o'clock.
So the slides will be up.
I'm really just going over what it actually says in the schedule by saying it out loud.
So you can hear it and see it.
But we'll do that on Monday.
So take a break right now.
And I'm going to get Pablo set up.
You can see some folks, Pablo?
PROFESSOR 4: I can in the distance, yes.
Hello, everyone.
PROFESSOR 1: So let me crank up the volume.
All right.
the room is yours.
PROFESSOR 4: All right.
PROFESSOR 1: Whoa, loud.
PROFESSOR 4: Thank you again to all of you for still being there.
Thank to the team in charge of the [INAUDIBLE] for persevering with the invitation on the guidance of how to use this time.
I'm sure you, or at least some of you, should remember the slide that I hope is on the screen.
There is a comfort zone where people feel safe.
And most often, where people are safe, not enough happens.
And one of the things-- one of the reason why I hope we can work together is to see if we can create true games.
The magic circles that are slightly outside of the comfort zone of people will be a Red Cross volunteer, a Red Cross staff, a person who is a scholar, a donor, a government official to try to make something new happen.
And that something new can be just understanding.
It can be about dialogue.
It can be about perceptions, or it can be about some of the ingredients that eventually lead to action.
So now let's see if this thing changes.
Did you see a change in slide?
Yes.
PROFESSOR 4: Good.
So let me move this thing.
We're going to start by playing [INAUDIBLE].
I'm going to invite everyone to stand up.
And Reid, can I ask you and someone to partner up with you to be in a place where I can see you.
PROFESSOR 1: OK.
PROFESSOR 4: Maybe Sarah, or whoever you choose.
PROFESSOR 3: I'll do it.
PROFESSOR 4: [INAUDIBLE].
PROFESSOR 1: Where are we in the camera?
Hey, there we are.
PROFESSOR 4: [INAUDIBLE] PROFESSOR 1: Stand.
PROFESSOR 4: So everybody find-- yeah, right.
When I say stand up, I mean it.
So everybody find a partner, shake hands with your partner.
PROFESSOR 1: Oh, no.
Cutting out.
PROFESSOR 4: What you're going to do now is each one with two in the duo is going to [INAUDIBLE].
PROFESSOR 3: Oh, you've dropped out.
PROFESSOR 1: Oh no.
PROFESSOR 3: How's the network?
PROFESSOR 4: Uh oh.
PROFESSOR 1: Yeah, we're getting a little bit of network drops here.
PROFESSOR 4: OK, may I suggest the following then.
If it is a matter of bandwidth, you can stop your video and that should work.
PROFESSOR 1: We'll do that.
PROFESSOR 4: You'll be in charge of through audio making sure I know-- PROFESSOR 1: Yeah, it dropped again.
PROFESSOR 4: But you're still there and kind of following.
OK, can you hear me now?
PROFESSOR 1: We can hear you.
PROFESSOR 4: So each one of you is going to hold an imaginary deck of cards.
Cards are number 1 to 10.
And I want you to shuffle these imaginary cards.
I want you to take the top half of the deck and swap it with your partner.
And then you shuffle again, those imaginary cards.
So you should have a lot of imaginary cards numbered 1 to 10 in random order.
If you're thinking, what does this have to do with Red Cross, we'll get there.
So now, this is crucial.
Each one of you is going to take the top imaginary card, make some body movement to reveal the moment when you're flipping it.
And [INAUDIBLE] state that the number that is on the top card.
For example, [INAUDIBLE] as well so say your number at the same time.
PROFESSOR 3: 4.
PROFESSOR 1: 6.
Oh hey, you don't have 10 fingers.
PROFESSOR 4: Let's try it again using body language to make sure it's simultaneous.
PROFESSOR 3: 1, 2, 3-- PROFESSOR 1: 10.
PROFESSOR 3: 5.
5.
PROFESSOR 4: All right.
So I couldn't hear the number, but first of all, you don't need to say 1, 2, 3.
You can use body language to make it happen simultaneously.
It's to your advantage to make it as fast as possible.
Two things can happen.
If the two players say different numbers, there's no consequence and you try again a different number-- PROFESSOR 1: Oh, no.
The video is slowing down.
PROFESSOR 4: or the same number, whichever you want, any-- [INAUDIBLE].
OK, it's the audio [INAUDIBLE].
PROFESSOR 1: Yeah, the audio is cutting out.
And the video is pausing.
Do you want to just switch back to me controlling the slides and you on audio only?
PROFESSOR 4: Sure.
PROFESSOR 1: OK, let's do that.
PROFESSOR 3: I think he can just turn off his-- PROFESSOR 1: So yeah, if you just go ahead and turn off your video, and I'll make sure I have slides.
PROFESSOR 4: All right, yes.
Let me get back to this.
How do I do it?
There.
PROFESSOR 1: Cool.
Let me go back to this.
PROFESSOR 4: So is the audio now better?
PROFESSOR 1: So far, yeah.
PROFESSOR 4: OK, good.
So what I was going to say is that the two players are going to say numbers simultaneously.
If the two numbers stated simultaneously are different, you keep going.
You keep saying numbers at the same time.
But if the two players say the same number at the same time, the first player to say "snap" takes all the imaginary cards and wins one imaginary point.
PROFESSOR 1: OK.
PROFESSOR 4: You keep going, OK?
And you start again.
The purpose is in one minute or less to snap as much as you can.
Those are the rules.
Shuffle your cards.
Ready, set and go.
You have less than a minute.
PROFESSOR 1: 5.
PROFESSOR 3: 4.
PROFESSOR 1: 6.
PROFESSOR 3: 2.
PROFESSOR 1: 10.
PROFESSOR 3: 7.
PROFESSOR 1: 8.
PROFESSOR 3: 3.
PROFESSOR 1: 3.
PROFESSOR 3: 9.
PROFESSOR 1: 2.
PROFESSOR 3: 12.
PROFESSOR 1: 1 PROFESSOR 3: 2.
PROFESSOR 1: 7.
PROFESSOR 3: 6.
PROFESSOR 1: 5.
PROFESSOR 3: 8.
PROFESSOR 1: 4.
PROFESSOR 3: 3.
PROFESSOR 1: 10.
PROFESSOR 3: 4.
PROFESSOR 1: 4.
PROFESSOR 3: 5.
PROFESSOR 1: 6.
PROFESSOR 3: 3.
PROFESSOR 1: 3.
PROFESSOR 3: 3.
PROFESSOR 1: Snap!
PROFESSOR 3: 3.
OK, you got the point.
1.
PROFESSOR 1: 5.
PROFESSOR 3: 6.
PROFESSOR 1: 6 PROFESSOR 3: 2 PROFESSOR 1: 7.
PROFESSOR 3: 1.
PROFESSOR 1: 7 PROFESSOR 3: 2.
PROFESSOR 1: 8 PROFESSOR 3: 3.
PROFESSOR 1: 8.
PROFESSOR 3: 8, snap!
PROFESSOR 1: Ah, OK, you get a point.
PROFESSOR 3: All right.
PROFESSOR 1: 9.
PROFESSOR 3: 4.
PROFESSOR 1: 10.
PROFESSOR 3: 3 PROFESSOR 1: 4.
PROFESSOR 3: 8.
PROFESSOR 1: 7.
PROFESSOR 3: 6.
PROFESSOR 1: 6.
PROFESSOR 3: 2.
PROFESSOR 1: 3.
PROFESSOR 3: 1.
PROFESSOR 1: 4.
PROFESSOR 3: 1.
PROFESSOR 1: 5.
PROFESSOR 3: 5.
PROFESSOR 1: Snap!
PROFESSOR 3: Snap!
All right, I think you got it.
PROFESSOR 1: OK.
PROFESSOR 3: All right.
PROFESSOR 1: 3.
PROFESSOR 3: 2.
PROFESSOR 1: 4.
PROFESSOR 3: 3.
PROFESSOR 1: 7.
PROFESSOR 3: 2.
PROFESSOR 1: 8.
PROFESSOR 3: 8.
PROFESSOR 1: Snap!
PROFESSOR 3: Ah, OK. You got two points.
PROFESSOR 1: OK.
PROFESSOR 3: All right, is that good?
Or go again?
OK.
PROFESSOR 1: [INAUDIBLE].
5.
PROFESSOR 3: 5 PROFESSOR 1: 7.
PROFESSOR 3: 2.
PROFESSOR 1: 8.
PROFESSOR 3: 9.
PROFESSOR 1: 10.
PROFESSOR 3: 3.
PROFESSOR 1: 3.
PROFESSOR 3: 6.
PROFESSOR 1: 2.
PROFESSOR 3: 12.
That's [INAUDIBLE].
PROFESSOR 1: 1.
PROFESSOR 3: 8.
PROFESSOR 1: 5.
PROFESSOR 3: 9.
PROFESSOR 1: 4.
PROFESSOR 3: 10 PROFESSOR 1: 3.
PROFESSOR 3: 4.
PROFESSOR 1: 2.
PROFESSOR 3: 5.
PROFESSOR 1: 5.
PROFESSOR 3: 6 PROFESSOR 1: 7.
PROFESSOR 3: 8.
PROFESSOR 1: 9.
PROFESSOR 3: 2.
PROFESSOR 1: 3.
PROFESSOR 3: 3.
PROFESSOR 1: Snap!
PROFESSOR 3: Ah.
OK, [INAUDIBLE].
PROFESSOR 1: So I have four?
PROFESSOR 3: I think that you win.
I think you just win this game.
[INTERPOSING VOICES] PROFESSOR 1: Are you talking?
[INTERPOSING VOICES] PROFESSOR 4: [INAUDIBLE].
PROFESSOR 1: OK, I can't hear you.
Should I shut them up?
PROFESSOR 4: Yes, please.
PROFESSOR 1: OK. All right, can we hear you?
PROFESSOR 4: Well, I was actually trying to be heard for the past 30 seconds.
OK, so what we're going to do now-- can you hear me?
PROFESSOR 3: Yeah.
PROFESSOR 1: Yes.
PROFESSOR 3: Yeah.
PROFESSOR 4: Good.
What you are going to do now is to find-- we are going to play a second round with two minor differences.
The first minor difference-- don't do it yet.
But you have to shake hands with a different person.
You will find a new partner.
PROFESSOR 1: OK.
PROFESSOR 4: The second difference is that instead of having numbers 1 through 10, you are going to have the names of animals
What?
Holy crap.
PROFESSOR 4: Each one of you should know more than 10 animals.
So it shouldn't be a problem.
Find your partner.
Shake hands.
PROFESSOR 3: Someone else?
PROFESSOR 4: Shuffle your deck.
[INAUDIBLE] PROFESSOR 3: Oh.
PROFESSOR 4: And I-- PROFESSOR 3: OK, you guys play.
PROFESSOR 4: --request you stop it in 30 seconds or less.
PROFESSOR 3: We're suppose to do somebody else, right?
PROFESSOR 1: Yeah.
PROFESSOR 4: So let's do anybody else.
How about I play with you.
You play Rick.
PROFESSOR 1: Go.
[INTERPOSING VOICES] PROFESSOR 1: All it's 30 seconds.
OK, all right, back to the screen.
OK.
PROFESSOR 3: OK.
PROFESSOR 1: All right, Pablo, you're back.
PROFESSOR 3: This was harder
Yeah.
PROFESSOR 4: So clearly I'm not in the room.
Can you hear me guys?
Yeah.
PROFESSOR 4: Good.
Even though I'm not in the room, I imagine that what usually happens may have happened again here.
First of all, it seems much harder to snap.
Second, more people find themselves going, ah, and not saying an animal.
There was like clogging of the brain cells.
And then very interestingly, very often people don't listen to each other.
So for example, the first time I played, I was saying, dog, cat, mouse, and my partner was saying, zebra, giraffe, lion.
And it took us a long time to notice that we were in different universes, that we were never going to snap.
And then interestingly, when you finally say the same words, a lot of people don't snap.
Their brain cells do not get in the right place.
So when you play the next round, I want you to go back to your original partner.
And in your deck of cards, you are not going to have numbers 1 through 10.
You are not going to have animals.
You are going to have words or concepts that you associate with the Red Cross.
[INTERPOSING VOICES] PROFESSOR 4: Whatever you want-- PROFESSOR 1: OK.
PROFESSOR 4: --it has to be short, three words or less.
PROFESSOR 3: OK.
PROFESSOR 4: And of course, you're trying to snap.
So you have 10 seconds [INAUDIBLE].
PROFESSOR 3: OK, all right.
PROFESSOR 1: Yeah.
PROFESSOR 4: [INAUDIBLE].
PROFESSOR 1: OK, sure.
PROFESSOR 4: On your marks, ready, set, go.
[INTERPOSING VOICES] PROFESSOR 3: You want to try this?
STUDENT 11: [INAUDIBLE].
PROFESSOR 3: OK. One-- STUDENT 11: I don't know anything about Red Cross.
PROFESSOR 3: Yeah.
Health.
STUDENT 11: [INAUDIBLE].
PROFESSOR 3: [INAUDIBLE].
STUDENT 11: [INAUDIBLE].
Red.
PROFESSOR 3: Beds.
STUDENT 11: People.
PROFESSOR 3: People Oh, snap.
STUDENT 11: Snap.
You got it.
PROFESSOR 1: All right, back to the screen.
PROFESSOR 3: OK, [INAUDIBLE].
PROFESSOR 1: All right, back to you, Pablo.
PROFESSOR 4: OK, thank you all.
If what usually happens in this game happened in the room, first of all, it's even more difficult for people to come up with words.
It's even more difficult to snap And much more than before, people are realizing in the process of trying to play this game, that they don't really know what they think about the subject matter we proposed in the third round of the game.
So what I'm going to invite you to do now is very quickly form groups of three.
Don't start yet.
Let me finish explain.
Form groups of three.
PROFESSOR 1: Oh, no.
Your audio just cut out and the video just cut out.
Are you back?
PROFESSOR 4: I cannot hear.
PROFESSOR 1: Can you hear us now?
PROFESSOR 4: OK, now I can hear.
PROFESSOR 1: Yeah, yeah, the audio just cut out briefly.
But you're back again.
PROFESSOR 4: OK, good.
Now I'm back with audio and video?
PROFESSOR 1: You're back with audio.
I don't see the video.
But I've got the slide up.
PROFESSOR 4: So let me take video out of there.
So what we are going to do now, if you can hear me-- PROFESSOR 1: Yes.
PROFESSOR 4: --is to form groups of about three people, [INAUDIBLE].
And you're going to, as a trio, write four terms, four things that you would have wanted to write in that last deck of cards.
Is that clear?
So it's not about numbers.
It's not about animals.
It's about words that are constantly associated with the Red Cross.
Form a trio and write four things that you hope would be in the deck of cards so you can try to snap Go.
You have three minutes.
PROFESSOR 3: Oh, OK. Everyone, gets four cards.
PROFESSOR 1: Actually, you can just use scrap paper too.
This is just in case you don't have any.
[INTERPOSING VOICES] PROFESSOR 3: Do you have enough cards?
I've got a lot.
I've got another deck here.
Do you need some more cards?
There's a pile here.
STUDENT 12: Thank you.
PROFESSOR 3: Yeah, there you go.
Yeah.
[INTERPOSING VOICES] PROFESSOR 3: Extras?
Everyone has four, right?
Oh, groups of three.
[INTERPOSING VOICES] PROFESSOR 3: I gave you a few extra just in case you make a mistake.
PROFESSOR 4: Rick, can you hear me?
PROFESSOR 1: Yeah, I can hear you.
PROFESSOR 4: OK, so I want you to give them no more than 30 seconds left.
PROFESSOR 1: OK.
PROFESSOR 4: And let them know that time is running out.
PROFESSOR 1: All right, 30 seconds left.
PROFESSOR 4: And then, Rick, what [INAUDIBLE] going?
PROFESSOR 1: The slide is showing, yeah.
PROFESSOR 4: Sorry?
PROFESSOR 1: The slide?
PROFESSOR 4: No.
Well, yeah, actually, [INAUDIBLE], no slide yet.
PROFESSOR 1: OK.
PROFESSOR 4: It's easier if you just do it.
PROFESSOR 1: OK.
PROFESSOR 4: What I want you to do is to pick some trio, ask the trio to choose one of the words [INAUDIBLE] they have on their cards and invite all the other groups to say "snap" if they wrote that exact term.
PROFESSOR 1: OK, did we get that?
So one trio come up and give one of your terms.
Just stand up and start over here.
So what's one of your terms?
Disaster relief.
PROFESSOR 1: Disaster relief?
Snap, snap.
How many?
Snap.
PROFESSOR 1: Snap.
Three.
So we had four or five.
PROFESSOR 4: OK, now choose another trio.
PROFESSOR 1: OK, another one.
PROFESSOR 4: Another trio to say one of your words.
PROFESSOR 1: All right, term?
Red
Snap
Snap
[INAUDIBLE]
Red PROFESSOR 1: What was the word?
Red
Red.
PROFESSOR 1: Red?
All right.
PROFESSOR 4: OK, so do the same with a few more trios, maybe three or four more trios.
PROFESSOR 1:
Blood donations.
PROFESSOR 1: What's that?
Blood donations.
PROFESSOR 1: Blood donations
Snap
Snap.
PROFESSOR 1: Snap, snap
Snap.
PROFESSOR 1: Snap, snap, snap.
OK, another trio.
Yeah
Aid.
PROFESSOR 1: Aid?
Snap
Snap.
PROFESSOR 1: Snap, snap, snap.
All right, another one
CPR.
PROFESSOR 1: CPR?
Ooh.
PROFESSOR 1: No CPR?
PROFESSOR 4: Interesting.
No snapping with that one, huh?
PROFESSOR 1: OK.
PROFESSOR 4: OK.
So there are two reason why I wanted to do this.
Thank you all very much.
If we had the time, we would play this game in a version that has winners and losers, where we would eliminate the most and the least snaps.
And the second most and second least snaps are the winners, for example.
So you have to choose which of your cards to say, and so on.
There are two reasons why I wanted to share this with you.
One is to re-energize the room and bring it back into the game zone.
And the second one-- it's the flavor of one of the games that I hope maybe some you will consider doing.
When we play this game in the context of [INAUDIBLE], for example, we play [INAUDIBLE], first with numbers, then with animals, and then we with the subject matter of the course, which can be climate change.
It can be humanitarian, law.
It can be [INAUDIBLE].
It can be shelter.
It can be urban risk.
And people come up with words.
And they write them down.
And we keep those papers.
And then after [INAUDIBLE], which can be an hour, or a day, or a week, we [INAUDIBLE].
And we ask them to form different trios and to write the words that they [INAUDIBLE].
And that collection allows us to compare the before and the after on how the thinking changed.
[INAUDIBLE], did you show the slide that has the two printouts?
PROFESSOR 1: Yeah, I got that.
PROFESSOR 4: So [INAUDIBLE] and change this here.
This is where we played about insurance.
The one on the left was the before.
And you see [INAUDIBLE] and [INAUDIBLE], premium, accident, disaster.
When we played the game afterwards, there were more nuanced terms.
There was [INAUDIBLE] collected.
There was buffered.
There was future.
There was return.
There was long term.
And this was very difficult for us to see how the thinking changed as a result of the session.
So moving on to the next slide that says choices for your game project, these are the six choices that I want to make available to you.
The zero one is to create a digital version of the game that could be played along the lines of what you just did, but for us an accelerated collection of data, so that everyone can see at the same time what just happened.
I'll explain it more later.
Then in the list, you see UNICEF Ghana.
And this is about cholera, a waterborne disease that is very tough.
And invite two groups if possible to take this one, because it's an important one to us.
And it's about helping schoolchildren in West Africa to manage health risks.
The next one is about linking early warning forecasts with early action, decisions that can help save lives.
Since it is 75 in Boston or Cambridge today, we thought it would be relevant to say that we wanted to focus on heat waves.
We could make it about hurricanes.
We could make it about volcanic eruptions.
We could make it about many things.
But we invite heat waves, because there are a few reasons that make it useful to us.
The next one is perhaps the most challenging one intellectually.
It's about new funding mechanisms for disaster preparedness.
I will explain this.
We call it forecast-based financing, or FBF.
And it's the one that has the most potential to transform the humanitarian world.
The fourth one is Ebola.
As you can imagine, this is a very challenging time for my colleagues.
And maybe some of you can give a hand.
And the last one involves adaptation to climate change.
I think you may have met [INAUDIBLE], my partner.
He designed [INAUDIBLE] project for an organization called Plan International, focused on work with children [INAUDIBLE].
So moving on to the next slide, before we go into the details of each one of those potential topics, I wanted to show you some thoughts about why digital games for these things.
As you remember, hopefully, from the previous visit to MIT, the vast majority of our games have been non-digital.
We've played them face to face.
They have been very good for what thew were intended to do.
But they were reaching a very small group of people, only those who were here.
Recently, we designed-- or recently collaborated in the design of our first digital game that was to be used in a MOOC with [INAUDIBLE].
And it reached 35,000 people, which I'm pretty sure has doubled the number of people that, since the beginning of my work with games, have been playing games that are non-digital.
There's clearly [INAUDIBLE] to many, many people.
And we're thinking of Red Cross staff, colleagues of mine who have a job, because this is their managers, [INAUDIBLE], whatever it may be.
There's also Red Cross volunteers, people who are not paid [INAUDIBLE] some work.
It could be digital games that people at risk, people who would make some decisions to save their lives or save the lives of their neighbors.
It could be about youth and the schoolchildren, who maybe don't know something could learn through a game, or had a [INAUDIBLE] that could inspire through digital games.
And there's also the bigger interpretations, especially now in a time where there's crowd funding, there is people who have the ability to act in [INAUDIBLE] number of little things and make them bigger.
[INAUDIBLE] is that games, including digital games and [INAUDIBLE] of the many ingredients needed to support a bigger change.
We are not widely optimistic.
It's not that we believe that someone lives their life in a certain way, and then they play a game, and boom, they're a different person, and they change their behavior.
Maybe that can happen.
But we are aware that it takes more than a simple game to play to change minds.
But we believe that games can help invite people to change behavior [INAUDIBLE].
It can be about creating awareness.
It can be about engaging their desire, their want to learn more.
It can be about a way give them interest, the appetite.
Or it can be about inspiring people to take something that [INAUDIBLE] they already do and put it into motion, [INAUDIBLE].
Importantly, because of [INAUDIBLE]-- sorry-- last bullet.
There's so much hope in digital technology, in people who can program, in people who have cell phones and [INAUDIBLE] Africa.
And we see a lot of room for growth within our team designing and deploying digital games, particularly with UNICEF, but also with many examples of organizations [INAUDIBLE].
So moving on, I should probably come to the next one.
Rick, would you like to either inject anything or invite questions at this stage before moving to the details of each game?
PROFESSOR 1: Sure.
So yeah, are there questions about the target audience or the reasons for digital games before we move on?
Yeah, one question?
Will we be able to play test with our target audience?
PROFESSOR 1: So play testing with the target audience-- are any of these folks in the Boston area that we could invite in for a play test session?
PROFESSOR 4: Almost certainly not.
So what we can do is to play the games with people who are as close as we can get.
So you know, schoolchildren [INAUDIBLE], most of them are different from children in West Africa.
But I imagine that [INAUDIBLE].
There are a lot [INAUDIBLE] prototype for Ghana if you play it with children in Cambridge or [INAUDIBLE].
PROFESSOR 1: OK.
PROFESSOR 4: Having said that, I may be able to get colleagues in developing countries to download your code and play in [INAUDIBLE].
So [INAUDIBLE] on the possibility for play test with the target audience.
But it is in the nature of this work to work for the benefit of developing countries.
So we'll have to accept that there is going to be a limited ability to play test.
And that's something that can happen in the early phases of the project design.
Basically, we could do it if you had the budget for a planet ticket.
And it might be hard.
PROFESSOR 1: Was there another question I saw?
Yeah
Are you envisioning this being used as a one on one thing, where one person is playing the game?
Or are you envisioning this being used as sort of an alternative to a presentation, similar to the games that we've seen here.
PROFESSOR 4: Yeah, it depends on the game.
Generally speaking, mostly because of the [INAUDIBLE] in your ability to spend student time on both the games, we are sticking to ideas that can be implemented for one player with one computer or one device, [INAUDIBLE]-- PROFESSOR 1: Yeah.
PROFESSOR 4: --which can happen in the house, happen in the street when they're waiting for the bus.
Or it can happen in a group, in the context of a [INAUDIBLE].
But we decided to [INAUDIBLE] multiplayer variable for more sophisticated endeavors.
PROFESSOR 1: Yeah.
So yeah, when I talk further about the constraints on Monday, I'll be mentioning that you could do a single player or multiplayer game.
But it will be a single device, no networking.
And we are asking you to think about the spectator experience.
So what does it look like to be looking over the shoulder?
What does it look like to be looking down on the table, if it's an iPad game or a tablet game?
What does it like if somebody's playing the game on a screen like this?
Consider one of those.
And design it towards one of those.
Yeah?
So by schoolchildren, is this elementary, middle, or high school?
And also, can we assume they speak English?
PROFESSOR 1: So for schoolchildren, is there a particular age range, elementary, middle, or high?
And is there an English language assumption going on?
PROFESSOR 4: Yes.
So yes to English.
And it will be [INAUDIBLE] official language is [INAUDIBLE].
So children go to school and learn at some age.
And different products have different age ranges.
But generally speaking, it's roughly the ones that are schoolchildren are not [INAUDIBLE].
So we'll get to that in the details for each game.
PROFESSOR 1: OK.
PROFESSOR 4: So moving on, if you want to press the slide for the [INAUDIBLE].
PROFESSOR 1: Yeah.
PROFESSOR 4: Thumbs up.
PROFESSOR 1: Yeah.
PROFESSOR 4: Those of you who were at the game's initial [INAUDIBLE] may remember the game where you all standing, and there was a die, [INAUDIBLE].
And if there was a 6, you had to do the umbrella.
Otherwise, you had to do thumbs up.
Or if you push again, [INAUDIBLE] sitting [INAUDIBLE] on the cement, and [INAUDIBLE].
PROFESSOR 1: Yes.
PROFESSOR 4: Those are the two basic [INAUDIBLE] that we generally remember things.
You either do the usual, or do something new.
And what we want is to [INAUDIBLE] a player the idea that you can't make wrong decisions without negative consequences.
But it's really hard to know whether the decision is the right one or the wrong one before the disaster [INAUDIBLE].
[INAUDIBLE] most of you in the process of designing games will digitally incarnate some form of mechanic that [INAUDIBLE] that.
The player is going to say, do I do the usual thumbs up or do I start [INAUDIBLE] the umbrella.
It will depend, of course.
If it's a game about health, an umbrella won't be it, right?
It will be some other image.
But you get the point.
It's about the repetitive act.
The next slide, which I also showed the previous group, this is more what we want.
We want players to experience an aha moment, a moment of discovery that is the result of the process where they go though some not knowing what the next choice is.
Otherwise, it would be too obvious.
It would be [INAUDIBLE].
And [INAUDIBLE].
So good luck to all of you in coming up with an [INAUDIBLE] balance because huh and aha.
So moving on to the next one, this is about the first [INAUDIBLE].
It is in the context of this game that you just played.
As I mentioned, when we played with 200 people, we can very quickly match all the cards [INAUDIBLE] by groups of three.
And we can quickly enter them into a computer connected to internet, [INAUDIBLE] before and after, so people can see at the end of the workshop how their own thinking has changed.
Now, when we went through the activity of Rick pointing to a geometric, choosing a word like [INAUDIBLE], and other people saying [INAUDIBLE].
If you have 50 people in the room, it becomes much more difficult.
Because if you're doing [INAUDIBLE], it becomes boring.
So what I ask you to consider is to create a digital tool that can be embedded or utilized or employed to [INAUDIBLE], so that we can collect and process information of what people are thinking.
So you'll be face to face, just like you were then.
But I imagine that your interface, this could be made with 3,000 people [INAUDIBLE].
And then each [INAUDIBLE] works.
And there's some game mechanic that creates an incentive and pitches [INAUDIBLE] at least [INAUDIBLE] of the time that helps people visualize their result.
And that helps make them analyze the data, to assess the input to monitor or to evaluate, and eventually to [INAUDIBLE] this animation of this [INAUDIBLE] concepting.
But folks, you have no idea how much [INAUDIBLE] was shouting at the first round.
[INAUDIBLE] And you were so energized.
That's great.
We love that.
But we need to channel that so it becomes more useful to the subject.
So the chances [INAUDIBLE], if you deliver something, and gain a product life that works, it's near guaranteed that we will use it.
If your tool helps us collect information in a somewhat playful way, and share it, and think about it, we're going to use it.
And we're going to use it in five continents, starting the day after you [INAUDIBLE].
And there are researchers for this.
There are documents that cover [INAUDIBLE] called data [INAUDIBLE] initiative.
And you can also consult with me and [INAUDIBLE].
I should be here some of the time if you have any questions about it.
[INAUDIBLE] like what we did for a tool similar to this one.
PROFESSOR 1: Was that a question?
PROFESSOR 4: I asked if you prefer that we move on to the next game idea, or if we pause and invite for questions.
PROFESSOR 1: Yeah, let's do questions after each idea, just really briefly.
So any questions about this game?
Yeah?
I thought we were not supposed to make anything that has to do with a network.
PROFESSOR 1: Yes.
So in this game, there is a little bit a networking going on.
If you were to do this game, it would mean it's not a multiplayer networking experience.
But you are transmitting data to some kind of server.
So that's where this one gets hairy.
It looks like a simple design, but it's actually a bigger-- there's a bigger back end to this one.
PROFESSOR 3: This is possibly, just by looking at it fully, it's possibly, technically, the most difficult one.
Because it looks like it's still a networking game, even though we are encouraging you to do single players, single device.
PROFESSOR 1: And designing for touch is very difficult too.
It just kind of it gets big.
Any other questions?
PROFESSOR 3: There probably is a middle ground where you can think of, all right, what can be done with a single computer that would still make it easier for someone like Pablo to be able to run this in a workshop, for instance.
Even just making his job slightly easier.
Remember, we just played this game with imaginary things.
What can we do that doesn't required the players to have a device, but speeds up the process from a central computer to be able to say collective data?
I don't know what that would look like.
That would be more of a design challenge, rather than a technical challenge, but possibly more feasible for the size of this class.
PROFESSOR 4: [INAUDIBLE] the results are shared in real time.
But if people have a cell phone, they can just send an email to the words to some address.
And then some [INAUDIBLE] someone punched buttons these skills then.
[INAUDIBLE].
Write down [INAUDIBLE] we don't understand the [INAUDIBLE].
So as it was said before, anything [INAUDIBLE] that you may have to an [INAUDIBLE] experience, our to improve a [INAUDIBLE], would be appreciated.
If no group chooses this one, no problem.
PROFESSOR 1: So let's move one.
Actually Pablo, can you turn off the screen sharing.
Maybe that will speed up the-- PROFESSOR 4: Oh, I wasn't aware that I wasn't sharing.
PROFESSOR 1: Yeah, I just noticed it too.
PROFESSOR 4: Here it says I am not sharing.
Weird.
Let me go to this.
Stop sharing screen, good.
PROFESSOR 1: There we go.
So let's go to the next one.
PROFESSOR 4: The next one is the important one.
It's the UNICEF Ghana.
UNICEF is the most loved UN agency.
Well, I don't have [INAUDIBLE].
Generally speaking, it's fun for a very well [INAUDIBLE] agency.
It works with children and education and it does very good [INAUDIBLE] work.
And the team in Ghana has decided to reach out to [INAUDIBLE].
Originally it was in the context of cholera.
But they also have some money for Ebola and cholera.
And they want us to design games for school children that would help them becoming familiar with an improvement instrument of risks [INAUDIBLE].
I have no contract yet.
But it's very likely that this relationship will grow, and will grow in time, and will grow in size, and will be able to get into the digital very soon.
So the context is that cholera is a waterborne disease and is on the rise, in part because of [INAUDIBLE], in part because of the organization, in part because of the changing character of flooding, access to water, and so on.
It's almost [INAUDIBLE] risk of getting cholera in countries that the disease is epidemic.
There are [INAUDIBLE] million cases every year.
And if you've never experienced it, I can assure you, it leads to a very miserable experience.
It's a [INAUDIBLE] disease.
And of course it changes from one year to the next.
But there's 100,000 people dying out of cholera every year.
And that's not good.
And importantly, many of those deaths are avoidable if certain measures are taken.
Some of those measures are difficult.
For example, providing clean water [INAUDIBLE]-- that is not something you can do in five minutes.
But other things are important and manageable.
For example, how [INAUDIBLE] and protection of water sources, or basic hygiene and using soap-- many of those are simple.
So we have a money project-- work with some children in Ghana.
Rick will have all details of the project and description.
The purpose is to help reduce the risk of waterborne disease, in particular, cholera.
The chances of risk are very likely [INAUDIBLE] something to be further developed.
I would be very surprised between and December if [INAUDIBLE] something [INAUDIBLE] thousands and thousands of children.
But if any of you have the [INAUDIBLE] clean, clear, useful game [INAUDIBLE], then maybe we can become direct [INAUDIBLE] or someone could be hired with money [INAUDIBLE].
There's the cholera toolkit, which if you pick-- show the next slide.
You see a [INAUDIBLE].
There's a chapter 1 program shown.
There are many stages.
There's the situation of preventing how to coordinate, how to prepare community aspects, all sorts of things.
The toolkit [INAUDIBLE] is extremely rich and detailed.
And we are now conversing with-- actually, we need two days.
We should be getting from them, which are the thing that the [INAUDIBLE] practice.
So who ever ends up taking this challenge, and we hope it will be two groups, it's going to be something that very directly capable of [INAUDIBLE] class.
And that must feel good for whoever successfully takes on that.
I do not remember the age group.
But I'm pretty sure it was less than 18 and more than 10.
But it may be more narrow than that.
I can't remember.
Ghana is an English speaking country.
It in [INAUDIBLE] East Africa.
In terms of [INAUDIBLE], the UNICEF team works with [INAUDIBLE] games.
I'm going to pause here and invite for questions.
PROFESSOR 1: Any questions about this one?
OK, no questions on this one.
PROFESSOR 4: Very good.
Was I very boring or very clear?
Moving on to the next one.
If you're having [INAUDIBLE] slide show, you can see through the text?
PROFESSOR 1: Yeah.
PROFESSOR 4: Good.
So this one is linking early warning with early action.
Science [INAUDIBLE] progress of something [INAUDIBLE] likely to accompany [INAUDIBLE] future.
And there are timescales that are going to be at the seasonal level, like El Nino, and weeks to months.
We're more likely to focus on it hours to days.
And in the meantime, in the case of [INAUDIBLE] that's very useful, but there is a lot being done.
In the case of heatwaves, that's not very much lower.
An issue with heatwaves is on one hand because of global warming and the changing climate, the recent heatwaves is very steep [INAUDIBLE].
You need to remember from almost a decade ago, Netherlands, France, Belgium, they have tens of thousands of people die because of the heat.
And most of those people die for very, very avoidable reasons.
So if you're elderly, or have respiratory problems, you have certain legal health condition, very simple things like being in a colder room with your condition or drinking more than you want, that alone can be enough.
Construction workers [INAUDIBLE] sports collapse.
All you have to do is to suspend the competition, or to [INAUDIBLE].
So it's very [INAUDIBLE] and consistently take people by surprise.
So the purpose of these games will be to support Red Cross, especially as a [INAUDIBLE].
So they can learn what are the simple actions taken to save lives, not only at home but more importantly, the lives of [INAUDIBLE] members of the community.
The chance for the [INAUDIBLE] also is pretty big.
But it's a little bit more of a timeline.
So it will begin-- we start looking in January.
We will need some [INAUDIBLE].
But there are some institutional needs to embed.
Once they exist, it has to go through Red Cross channels, which can be more complicated than going in through school children, where if it is picked, it goes viral quickly.
But it's still a very good one.
Again, it's one that can lead to saving real lives.
And there plenty of resources within the Red Cross and beyond.
Importantly, there is a master's student at Harvard who is doing her entire thesis on this.
And we're going to have at least one, maybe two, hack-a-thons dealing with heatwave risk.
The Harvard student, her name is Elle.
She's really great fun.
And she would be a collaborator.
And this will have a lot of chances of growth.
If you did see a photo from the Netherlands heatwave, and Red Cross volunteer helping an elderly lady just hydrate, and do that lady did not die, whereas tens of thousands did.
Can you come up with a game that helps people make such simple decisions?
And I'm sure there's a way to make it fun.
So overview-- one or two questions?
PROFESSOR 1: Any questions on this one?
So the target audience on this one's different.
It's Red Cross volunteers and staffers.
It's probably as part of their training?
PROFESSOR 4: Correct.
So the way we envision this-- actually I dind't mention-- the way you envisions this one is to have the game as ready, deployable but on the shelf.
And then when there's a forecast of a heatwave likely to happen over the coming six days, then we blast the game to the Red Cross volunteers.
So they become very quickly familiar with the very core aspect of it, of course complemented with others like maybe face to face training, or mainly sending documents to read But this would be potentially very easy to adapt or translate to many languages.
And again, Chicago, Boston-- there's plenty of places even in the US.
This is the one that would be easiest to play test locally because we can-- And the profile of a Red Cross volunteer is not necessarily too different from the profile of university students who are engaged in do-good activities.
PROFESSOR 1: OK, on to the next one.
PROFESSOR 4: All right, moving to the next one.
The next one is the most complex.
Do you see the drawing of the time bomb with the fuse?
PROFESSOR 1: Yes.
PROFESSOR 4: OK.
So allow me to give a little bit of context.
We did a little bit of this conversation when I showed up a few weeks ago, but I imagine you all forgot by now.
The humanitarian sector has two main funding mechanisms.
Imagine a pot of money.
Even if there's almost no money there, if the pot exists, money can go in, to then be taken out to spend.
One of the pots of money that exists, one of the funding mechanisms that exists is for disaster response, meaning a hurricane, flood, volcanic eruption.
Something has already happened, is happening, and is killing people.
People are dying, but if you did something, you could save lives.
For example, get on a boat and rescue people who are stranded on a roof.
Or people are sick.
Go give them medicine.
This is disaster response.
And if you spend money, you can do very good things.
And the funding mechanism exists.
There is another pot of money which is for a normal days.
The Red Cross has vehicles.
The vehicles needs maintenance.
They need to change tires.
You need to buy a new computer.
You need to pay rent and the electricity bills.
You need to train staff.
So on a normal day, you have a budget for either maintenance, or development of new projects, or writing proposals, and so on.
And when that pot of money exists, we called it the annual appeal.
Every year, we appeal for help.
And people or donors, or governments give us some money.
What does not exists is what the drawing of the cartoon depicts, which is when there is a signal from science that a disasters, that something bad is likely to happen soon.
We don't have money to fund action before the disaster, after the forecast.
So if you click to see the text, the context is that there that missing money part.
We have money for after a disaster, we have money for everyone all day.
But we don't have money for before the disaster, after the forecast.
The issue is that if you spend money in that time window, before the disaster and after the scientist says a disaster is likely, you can do very good things for very cheap, and relatively simply, and much more reliably.
It's not only more expensive, but more difficult to save a life when the storm waters are flashing through a city and people are above the ground.
The boat is unstable.
Whereas if you do it the day before, when the water has come down from the sky but it's still in the river and not in the city, people can take action by just walking away, on your own means, with your valuables and not just depending on someone to show up with a boat.
So we came up with a financial mechanism, we call it forecast-based financing for disaster preparedness which is nuanced in how it's implemented.
There is a pot of money.
It's like a bank account.
You need to have a threshold of disaster risk that needs to be exceeded.
So it can not be two drops of rain.
It has to be a lot of rain.
It can not be some wind.
It has to be a hurricane category 1 or more.
But once the threshold is established and reached by the forecast, then there's some standard operating procedures, some actions that people have to take, spending money to save lives.
And that is something that if you could come up with a game that captures that, this is the one that if we help communicate forecast-based financing, I sent to a colleague because it can work for a broad audience digitally.
Ideally we would reach those who can contribute either money or decisions, muscle power.
The chances of embrace for this one are not as high as on the previous one.
But if it were to happen, it'll have the highest impact.
Because right now there's billions of dollars being spent in the two extremes.
And in the small fraction where spent investigating that precious window of opportunity-- when you see that fuse getting close, that time of effort can really have a very, very high impact.
There's a link to a journal article that should be very clear for MIT students.
We also have some simpler readings and journal [INAUDIBLE] I have been engaged within and are ready to help.
This is, I insist, the one that is intellectually most challenging.
But I think if you nail it, it will also be the most rewarding.
Questions?
PROFESSOR 1: Any questions on this one?
So one thing, when we first started designing this class, this is actually the problem that we're most interested in, mainly because the other problems, we just hadn't heard about since.
Cholera was a recent proposal that was asked for, and Ebola, of course, very, very recent as well.
So if you really enjoyed some of the challenges from projects 2 and 3, and you really took on the planning aspects, the trade offs aspects of those two games, if you're looking at trying to communicate to another person how these probabilities works, this is probably a really good project you might be interested in.
PROFESSOR 4: And I should say also I'm working with a few MIT teams, including the Humanitarian Response Lab, including the Environmental Engineering.
We have two generations of three students plus one professor going to Uganda to work on making this happen for real.
We got German money to do it in Togo and in Uganda.
So it's been meaning to happen.
But it's hard to explain because it's dry.
It's like explaining insurance.
People get bored before they get it.
So maybe a game can help launch it a bit.
Good.
Then we'll move on.
Number 4, Ebola.
And I have to be very candid.
I come from climate research, climate and disasters.
And as far as we know, Ebola does not have much of a climate signal.
But, as I'm sure you know, it's a very importantly thing.
And the idea for bringing this up is that one of my [INAUDIBLE] colleagues, Steven Ryan one day sent a message saying, tonight I'm arriving in Liberia to work for a month with a Red Cross national team that is doing it's best to respond to the Ebola outbreak.
And guess what?
What they're doing, as its best, is not enough.
Steven is a communications specialist.
As you can imagine, I feel phenomenal admiration and respect for his dangerous choice to go to the place where people are in very serious trouble to try to help.
And that made me think, well, maybe we can help.
So I sent him an email, and he kindly responded saying, yes, they're keen on the possibility of a game.
Bear in mind that because of the cycle of the disease, by December chances are that either it became a complete pandemic, or they [INAUDIBLE] is completely gone.
So we are not going to make a game to be played in the equivalent situation as the situation now.
It would be more of a game about an awareness of how it works and how it can be prevent in a hopefully less than complete crisis mode.
But nonetheless, Steven and one of his colleagues were very graceful to offer themselves to be the subject matter experts and to help, should a team choose Ebola-- please no more than one team.
There's someone on the other side of Atlantic Ocean that is going to try to engage.
In this one, John and I do not have expertise.
I imagine it is possible to find expertise in the Boston area with so many phenomenal hospitals and so on.
But it's outside of my control to offer reliable support when it comes to subject matter.
I'm sure that there's plenty of documents from the CDC and from health practitioners that can help you design a game.
But whoever chooses this, it'll be heroic.
It can be really useful.
But you have to be most prepared to work autonomously.
Any questions?
PROFESSOR 1: Any?
[INAUDIBLE] PROFESSOR 4: Are you guys there?
PROFESSOR 1: Sorry, I had the microphone muted.
No questions on this one.
PROFESSOR 4: So moving on to the last one.
If [INAUDIBLE] hasn't joined yet, then I'll do my best to explain.
PROFESSOR 1: Yup, I haven't seen her yet.
PROFESSOR 4: So this is a project that is being led by two of my colleagues, including [INAUDIBLE].
So forgive my ignorance.
And it's very similar in its nature to the UNICEF one.
The difference is that the UNICEF project is for small children about a very specific health condition called cholera.
Whereas this one is for a partner called Plan International.
It's an NGO that works with children.
They're very good.
They work a lot on the rights of children.
They are about empowerment.
And they work intelligently.
And the Asian team has reached out to the planning center.
And I don't if you can read the words, but in summary we want a game to help children in Asia understand climate change and what can be done to adapt to climate change.
Plan International seeks to partner with the Red Cross [INAUDIBLE] Planning Center to develop a participatory learning game for children in Asia, focus on Southeast Asia.
So all the way from [INAUDIBLE] Cambodia, et cetera, that would strengthen their understanding of local climate change impacts, and help them develop a [INAUDIBLE] what they and their community can do to adapt and increase their resilience.
The project will include the [INAUDIBLE] game package, testing, and a training officer [INAUDIBLE].
The current project does not include a digital component because there wasn't enough money.
But like the UNICEF one, this is an area that inevitably has to grow.
So if any of you comes up with a prototype that is sufficiently playable to capture something about climate change, there's generally two [INAUDIBLE] ideas that children can benefit from learning.
One is that the trends of change are dangerous.
[INAUDIBLE] sea level rise, like drying of some sources of drinking water, as well as the changing pattern of extreme events, which can be about rainfall, can be about temperature, can be about the winds and cyclones.
And so children who have not lived long, and therefore don't have much of a memory, are likely to have to confront information about things, extreme events, disasters that they've never experienced, but they have to do something about.
So the game is not about one specific threat, but is more about the concept of adaptation to planet change, which of course can be embedded into a specific manifestation of planet change, like sea level rise, or like changing rainfall patterns.
There was a more clearly defined age group for this one, which unfortunately I don't have off the top of my head.
But [INAUDIBLE] will be sending the materials to the team.
So within 24 hours, I pray that you will have all the ingredients you need.
I know I didn't share too much.
But are there any questions about what I could share?
PROFESSOR 1: Any questions about this one?
No questions on this one.
PROFESSOR 4: OK. Then move onto the next slide.
We have about 50 minutes left [INAUDIBLE].
And based on some questions and comments, I'd like to get a [INAUDIBLE] front room.
You have all choices here.
The first one was snap.
I need a digital tool to enhance the snap experience of collecting and processing information.
Then we have the UNICEF Ghana, Cholera, where we hope we can get two groups because this is a big one.
Then we have linking early warning and early action maybe focusing on heatwaves.
Then we have the more sophisticated, conceptually, [INAUDIBLE] forecast-based financing, funding mechanisms for disaster preparedness before the disaster, after the forecast.
And then we have Ebola, where there's not much I can offer other than if you make a game, we can try to hook you up with people.
And if you come up with a good game, it can be really useful.
And the final one is adaptation to climate change to be played by children in Southeast Asia so that they understand climate change.
Here's what I propose.
We're going to give you guys 30 seconds.
Each one of you choose one or two secretly based on what [INAUDIBLE] which is completely incomplete information.
But that's the way it is.
Given what you've heard, choose one or two of those in the come 10, 15 seconds.
I'll give you that time.
Thanks for your patience.
PROFESSOR 1: Not a problem.
PROFESSOR 4: And so what we're going to do now is-- you're all sitting I imagine.
Actually Rick, can you show the room?
PROFESSOR 1: Let me go back to the Skype and turn on our video.
PROFESSOR 4: So what I want to do is [INAUDIBLE], and if that number of your chosen one, I want you to do two things.
One is to stand up, the other to shout snap.
But I cannot see everyone.
I cannot hear everyone.
But it will give me a flavor of proportion.
So item 0, if you're ready to [INAUDIBLE] stand up, [INAUDIBLE]
Snap.
PROFESSOR 4: All right, so we've got maybe one in four [INAUDIBLE].
Excellent.
You may sit down.
Number 1, if it's of your liking, do it?
Snap.
PROFESSOR 4: All right.
Good.
This one, [INAUDIBLE] all of you guys on a group because I don't see enough for two groups.
[INAUDIBLE] Then next one now, for two.
If you're going to do, do it
Snap.
PROFESSOR 4: All right.
This one seems popular.
Number 3, go
Snap.
PROFESSOR 4: Thank you.
Number 4, show it.
PROFESSOR 1: 4 was the funding one?
PROFESSOR 4: Yes
Three was the funding one.
[INTERPOSING VOICES] PROFESSOR 1: Yup, you're right.
I'm wrong.
Yeah, we're counting from 0.
4 is Ebola.
So nobody stood up for this one.
PROFESSOR 4: OK. And if [INAUDIBLE] do the last one, number 5.
Go for it
Snap.
PROFESSOR 4: OK, so we're focused then.
I don't know, Rick and team, what you had in mind for this time.
But given that we went relatively fast, what we could do is you could point to different places of the room and talk [INAUDIBLE] 0 middle, left is 1, and so on.
And invite people to just answer with your feet.
Which is the location they most want to go to?
So we get a sense of how many people want to engage in which game.
PROFESSOR 1: OK.
So we've got it on the board.
Cholera here, Ebola here, early warning here.
These are actually behind you, so I'm going to have to rotate the computer.
Snap over there, adaptation here, and disaster prep funding, the forecast-based funding over here.
PROFESSOR 3: Let's draw out the numbers as well.
PROFESSOR 1: Oh yeah, I'll put the numbers up.
PROFESSOR 3: So snap was 0, cholera was 1, early warning was 2, funding was 3, Ebola was 4, and adaptation to climate change, 5.
PROFESSOR 4: So answer with your feet.
You can not stay sitting for [INAUDIBLE].
And now Rick and team, I'm going to invite you, because you know best what is required for each group, be it finance grouping, in terms of quantity of people, in terms of profile [INAUDIBLE], there won't be code geeks in one, and then narrative geeks on the other.
If you understand this, at least what happens now is in now way definitive.
But it's to get a sense [INAUDIBLE].
PROFESSOR 1: Yeah.
So actually this leads really well into what we're about to do.
We're going to do some brainstorming on these topics.
So we're going to first have each of the groups just do a little bit of a quick brainstorming about the topic that they're standing with right now.
And then midway through, we're going to invite you to go over to a new group to talk about a different project if you're interested in talking about that other project.
On Monday, we're going to actually form the team and make sure all the teams meet our requirements.
But it's basically seven to eight people per team.
We're hoping for eight people per team, hopefully not more than that.
But we'll figure that out as we go.
And over the weekend, we'll be using the mailing list to talk about the projects.
So I'm actually going to let-- we're going to start off with this brainstorming.
We're going to take a break, we're going to do some more brainstorming.
And then I will more clearly outline how the mechanism is going to work between now and Monday.
PROFESSOR 4: Do you need me to stay online.
PROFESSOR 1: No.
What we would like is-- are you going to be on email between now and Monday?
PROFESSOR 4: Yes, but only with very limited ability to read and respond.
So make sure to make the subject line very clear and the questions as snappy and answerable as possible.
PROFESSOR 1: And you don't mind if give all the teams your email address?
PROFESSOR 4: No mind-- but teams, please, please, please, I beg you, consolidate your questions.
Be very efficient with your writing.
Because if I get too many emails or too wordy, excessive verbosity, I will not be able to get it.
PROFESSOR 1: So I'll figure out a mechanism-- PROFESSOR 4: [INAUDIBLE] fine, a channel within each group to communicate and consolidate.
PROFESSOR 1: So I'll figure out a mechanism to make sure you're not overloaded with email, and to make sure everyone gets the benefits of those emails to you.
PROFESSOR 4: Good.
And I suggest that you always copy [INAUDIBLE].
All right, so good luck to you all.
Thanks again for your time and inspiration.
And I wish you good luck [INAUDIBLE].
PROFESSOR 1: OK.
Thank you Pablo.
OK so you're in your groups.
I have paper for people who are not on chalkboards.
Do we have nobody for Ebola?
PROFESSOR 3: No one.
PROFESSOR 1: Then let's give snap the Ebola-- actually you're pretty big.
Let's not.
These two groups are pretty big.
Let's give them space to spread out.
So what we're going to do is just a really quick brainstorm.
Five minutes-- just choose one person to be a secretary.
Have them come up.
You've heard a little bit about the topics.
You haven't done any of the reading yet.
We're going to have all that reading available for you on Stellar right now for you to read over the weekend.
It's a lot of it.
We don't expect you to read all of it.
But we do expect you to look at it.
Based on what you understand, what kind of mechanics could come out of these kind of topics?
Just based on the word cholera, based on the word early warning, early action, what kind of game mechanics pop out of that?
And just through them on the board.
So disaster prep-- let me hand these off to those two.
PROFESSOR 3: To disaster prep, or to more than one group?
PROFESSOR 1: One to one, and one to the other.
PROFESSOR 3: We had a question like that, by the way.
Rick, we have questions in the back.
PROFESSOR 1: Yeah, one second.
What's the question?
From who?
PROFESSOR 3: Let's stick it up on the wall?
PROFESSOR 1: What's up?
PROFESSOR 3: You guys probably need the stand.
STUDENT 13: What game engines would-- is it the same as other previous ones?
PROFESSOR 1: Same as previous ones.
We'll talk about the constraints on Monday.
Don't worry about the constraints right now.
Right now just focus on game mechanics.
STUDENT 13: [INAUDIBLE] mobile [INAUDIBLE].
PROFESSOR 1: Yeah, so keep it blue sky.
You can be mobile.
You can be on tablets, you could be a computer.
It could be single player, it could be multiplayer.
Just don't do multiplayer networking.
That's it.
STUDENT 13: Our game has-- PROFESSOR 1: What's that?
Our game's definition is [INAUDIBLE].
PROFESSOR 1: Then what are the ways you're going to get around that like we talked about?
STUDENT 13: Without doing multiplayer?
PROFESSOR 1: Without doing multiple devices connecting to a single server.
There's other ways.
Don't even think about it for this brainstorming.
Just don't worry about it for now.
Here we go.
Pass these out.
All right, so five minutes, go.
[INTERPOSING VOICES] PROFESSOR 1: This is in case my wife sends pictures.
So I have another group that we work with is doing an EdX course.
[INTERPOSING VOICES] PROFESSOR 1: We've done about 15 minutes of really quick and dirty brainstorming.
Somebody from your group shout out one or two really, really interesting idea that you're really excited about.
Just one person, one group, interesting ideas, this group go.
STUDENT 14: Spatial placement of wells affecting where cholera spreads.
PROFESSOR 1: So spatial placement of wells affecting where cholera spreads?
STUDENT 14: Yeah.
PROFESSOR 1: OK.
So a spatial puzzle kind of game.
STUDENT 14: Exactly.
PROFESSOR 1: One more idea?
STUDENT 15: Animal crossing with cholera and puzzles.
PROFESSOR 1: Yes, animal crossing.
I'd think about the scope on that one.
STUDENT 16: Playing as a heatwave.
PROFESSOR 1: Playing as the heatwave, you are the heatwave, cool.
One other idea?
STUDENT 17: Playing as a Red Cross volunteer, managing resources.
PROFESSOR 1: OK, not as cool.
Heatwave [INAUDIBLE] volunteer.
Over here.
STUDENT 18: You're on a tiny island and you live on the island and you do stuff on the island.
But your island is shrinking.
So you have to collect resources on your island to move to the bigger island.
The bigger island is shrinking too because the water keeps going up.
So you hop, and hop, and hop, and you arrive at the mainland.
Yay, everyone's saved!
Except the mainland is a giant island too.
Whaa, whaa, whaa.
PROFESSOR 1: Whaa, whaa, climate change.
Cool.
Thank you.
Another idea, or is that the big one?
STUDENT 19: The decision between chopping forests and to build more buildings for faster development, or to keep the forest air help make climate change less severe.
PROFESSOR 1: Cool.
So one's kind of, you don't have much action.
The world is going around you.
This one is kind of, you have action.
The decisions you make are going to affect things.
Cool.
This group here?
This was snap right, this group?
One or two ideas?
STUDENT 20: Well, so I was thinking about not just choosing words, but given a scenario, making a particular decision can vary in two decisions other people made given that exact same set of [INAUDIBLE].
PROFESSOR 1: OK, so really thinking about how you're going to make this simple decision, or this simple interaction that the game has, make it a little bit more complex and really kind of bring the decision making out in the forefront-- is that kind of what it sounds like?
Right?
Yeah, what's up?
STUDENT 21: The was I understood the game, or the description for snap is that it's more of a data collection method, like collecting what people-- PROFESSOR 1: That's at least the purpose they want, is they want that data.
Well they want to collect that data.
But they also want to reflect that data back.
So it's a little bit of a visualization.
It's actually a of a visualization, as much as it is collection.
Are those the only groups, or the only concepts?
Oh yeah, disaster preparedness.
Everybody's disappeared.
Anybody from that group want to run over to the sheet and say an idea or two?
Or you just kind of-- STUDENT 22: I think we just got stuck on the brainstorming because the program is not as easy.
PROFESSOR 1: So it's hard.
It's going to be a hard one.
STUDENT 23: One idea we came up with was building an infrastructure and you would decide whether to keep expanding or to reinforce the structure.
And that way, you would oh, are you preparing for before the disaster or after the disaster.
Did you have something to add?
STUDENT 24: I was just going to say we felt like we needed to do more reading to understand exactly what [INAUDIBLE].
PROFESSOR 1: But something about there is something that can be built to spend that money on.
So it's not just about collecting the money, it's that money's going to go somewhere.
Which I remember him saying at least-- I don't know if it was a meeting with me, or if it was one of the first times he came to class to talk.
But I remember that came up.
Take a quick couple minutes, one person from each group make a copy of all the ideas you have on the board somehow, like a photograph, or-- actually just take a photograph of it, and then come back to your seats.
We've got two more slides, then we're going to release you.
[INTERPOSING VOICES] PROFESSOR 1: So after you've taken your captures, sit down.
What next?
I'm going to stand in front because I can't see what my slides say.
So today we brainstormed in groups on these topics.
Between now and Monday, what we'd like you to do is use the mailing list, the videogames@mit.edu at mailing list to form teams.
In class on Monday, we will actually also set aside time in the beginning of class to make sure that all these teams get formed.
And we're going to do some more brainstorming and some more pitching on Monday.
And I'll go into more detail about the requirements for the project 4.
But make sure you read the project 4 assignment.
That's basically all I'm going to be doing is just saying out loud what it says in there.
So before next class, do these things.
Read the material in Stellar.
All the topics have some material in Stellar.
The Ebola topic did not, the snap topic did not, but cholera has the tool kit that he'd that he mentioned up there.
Early warning, early action has a like a task force guideline, like a idea of world disasters from 2009, I think.
And then another journal article.
Disaster prep funding has that forecast-based funding journal article attached to it.
Adaptation for climate change, we don't have materials yet for that.
But that should be coming in today.
And that'll be put up into Stellar.
That's all of that, right?
So when you're in your groups, think about these questions.
And think about these questions until Monday.
Did you actually have eight people interested in your topic when you were in your group together, like think back about the brainstorming.
Did you see seven other individuals interested in the topic?
If you did, you can probably make a game based on that topic.
If you didn't, it's going to be difficult to find people for your team.
So really use the mailing list and try to figure out ways to use the mailing to try to attract people to your topic.
If there's something about like the game design that you came up, a personal reason why you'd like to see this game get made, throw it to the mailing list.
If you really don't care about the topic so much or any of these topics, but you really want to do programming, you really want to do art, you really want to do code, whatever you want to do, if that's the thing that you're most interested in, that game engine you want to use, throw an email to the class to say, hey, use me.
Here's what I'm interested in doing.
Put me on your team, basically.
If you're more interested in the mechanic design over the topic, that's also OK.
So you might not be interested in the topic, but you're interested in some of these mechanics that we've been talking about in class, or some of the mechanics that you heard mentioned in the group brainstorms, so mention that in mailing list.
Again, team information will be finalized on Monday.
And if you are undecided, I'm just going to stick you in a team.
That's the worst thing that could happen.
Because if I just put you in a team, I'm not thinking about who you're going to work best with.
I'm not thinking about what game engine you're working on.
I'm not thinking about is whether or not you're interested in the topic or not.
I'm just grabbing you, and sticking you on a team.
So do all the work for me, so I don't have to do that.
And if you do just want me to do that, that's OK.
I like rolling dice.
Do we have any questions about this, any concerns about this?
Yes.
STUDENT 25: Are you targeting five teams?
PROFESSOR 1: No, I'm targeting teams of eight.
So that's five teams, but it could be four teams if we need more people on the teams.
I would recommend 10 being your biggest cap.
In the past, we have seen 10 person teams work-ish.
They failed everywhere you think they failed, communication largely, scheduling largely.
The one thing they didn't have that I think you're more prepared to do, if you were in a big team like that, is that they didn't understand how to break out the strike groups, into strike teams, how to separate tasks and form into smaller groups.
I've actually seen you all do this on these previous two projects.
So if you do find yourself in a large team, think about how you're going to break the work up into the small tasks, into these bigger, modular kind of things.
Especially think about why you're in this class.
What did you expect to get out of this class?
What do you expect to get out of this final project?
If you wanted to do some really big tech thing, there are some ideas out here that require some heavy tech lifting.
If you're in designer, there's some really, really challenging design issues, especially with that forecast-based funding.
I'm really leaving that up to you.
But I'm going to make sure that all the teams have eight people.
Any other questions?
PROFESSOR 3: Just one statement [INAUDIBLE].
Especially with the snap team, I do realize that as presented to you today, it really does seem like a networking game.
That really does seem like a game that will involve a lot of networking.
And of course, we've spent the entire semester so far telling you, oh good lord, don't do that.
So it could be a huge tech challenge, or it could be a huge design challenge.
The huge design challenge is how do you accomplish everything that we think we need networking to accomplish without networking.
Or it could be a tech challenge that's, screw it, we're going to do networking.
Then it becomes what's the least brittle way to accomplish that.
The problem with networking and that particular game is that to require networking means that you need to play this game in a room with networking.
Which means if you take it to a room without Wi-Fi, for instance, that means it's got to be something that's going to work over a phone system, or it's going to require short messages, or something ridiculous.
And if you don't have networking, it's going to be like, what are you going to be doing.
Are you going to point a camera in a room and do image capture, or something?
It's like some crazy things-- are you going to do manual data entry?
That's actually a possibility.
Anything that you can come up with that makes that game that we played in class slightly easier might be beneficial for Pablo.
And he's guaranteed that he will use it as soon as you provide it.
So that's an incentive right there.
But it is possibly, despite how simple that game is, possibly one of the toughest design challenges that I can see on the board today.
PROFESSOR 1: Up until now, we've been really strict in allowing you to use specific tools, use specific methods.
We're going to start easing up.
But if you break any of the rules, you need to tell us why.
That's all we ask.
If you are going to use multiplayer, tell us why.
If you're not going to use one of these game engines that we've been practicing on for this semester, really, you really need to convince us.
So use the office hours like [?
Sarah ?]
suggested.
Use the mailing list.
Talk to us during class.
I've already had quite a few students talk to me individually, and talk to them individually about these kinds of thing.
The rules are there to be broken.
But don't break all of them at once.
Break one.
PROFESSOR 3: If you're going to get rid of like the task list or something like that, what are you replacing it with?
Tell us if you're going to be not doing daily scrums because it doesn't work for your team.
What are you replacing that communication strategy with?
Because it's clearly there for a particular reason.
Hopefully you've seen the reason, even if you haven't necessarily managed to make it work on a previous team.
So what are you going to do in its place?
Sarah, did you want to-- PROFESSOR 2: I think you guys covered it all.
About all I can do is restate the [INAUDIBLE] about breaking rules.
And it's break rules with care.
We didn't give them to you just to be arbitrary.
We gave them to you in our experience of watching a lot of students make a lot of games, and make a lot of mistakes.
These are the big pits that people dig themselves in and then climb down and wonder what the heck they're doing in.
So if you're going to break a rule, have a plan for it .
Come and talk to us about whether or not you think your plan will work.
We are here to try and support, and give you advice, and help out.
PROFESSOR 3: I think it's fair to say that we've seen every single rule that we provided actually successfully broken in every single year.
Not all the time-- PROFESSOR 1: Except for that bad year.
PROFESSOR 3: The what?
PROFESSOR 1: Except for that bad year.
PROFESSOR 3: Except for that one bad year where every project was crap.
PROFESSOR 2: Don't be that year.
PROFESSOR 1: No, you're not that year.
PROFESSOR 2: You're not that year.
PROFESSOR 1: And Paulina was in the class last year.
So if you're looking for student level, what was it like during this project, talk to Paulina.
She can help you out.
She can give you some tips.
And talk to each other.
That's why we do all these presentations.
Yes.
STUDENT 26: What level of polishedness [INAUDIBLE]?
PROFESSOR 1: We're going to talk about that through for the rest of the semester.
What does polish mean?
Polished, to our eyes-- I'll talk about this on Monday.
But basically what we're asking for is the games you've made these past two projects, that's about the scope you should have think for project 4.
So if you thinking about that kind of scope, but then you have four times the time, what are you applying that time towards?
Polish and making sure the user experience is easy to use-- it doesn't crash, it doesn't break.
A player can start playing and figure out through the systems of the game how it works, what the keys are.
You've got good credit screens.
We're not necessarily asking for an art, sound, or music polish.
We're not asking for indie level art.
We're asking for just a level below that.
A unified level of polish-- so everything in the game has the same amount of time, care, effort being put into it.
PROFESSOR 3: It doesn't feel uneven and bumpy.
It doesn't feel like, here's this awesome soundtrack with this really, really terrible art.
Or here's this really bleepy, 8-bit thing with gorgeous, hand-painted scenery.
That's not cohesive.
We want everything to look like it's on an even level of quality.
PROFESSOR 1: This might be a preview for future lectures, but play Ponycorns.
Look up the term Ponycorn.
It's a game made by a five-year-old and her father.
That is actually a high level of polish that I've seen happen in this class.
If it happens with your project, it would be really great.
Yes, Sissy's Magical Ponycorn Adventure.
PROFESSOR 2: The key thing to note here is that all the assets were created by the five year old.
And I believe the story was also written by the five year old.
PROFESSOR 1: And the sounds.
PROFESSOR 2: And the sounds were done by her.
And her father just did all the coding and put it all together.
PROFESSOR 1: What's that?
STUDENT 27: $3,199 in donations to date.
PROFESSOR 2: If you can draw as well as a five year old, you can make good assets for you game.
PROFESSOR 1: So we've got nine minutes left for the rest of class.
Capture all the ideas that you wrote down and mail them to videogames@mit.edu.
So whoever took a capture of each of these things, just turn it into a text file, and mail it.
Yeah?
STUDENT 28: Somebody created a Google spreadsheet.
Can we all just put our pictures in there?
PROFESSOR 1: That's fine.
STUDENT 28: So we don't get million emails.
PROFESSOR 1: Absolutely.
PROFESSOR 3: Also there are a couple of students who aren't in class today.
So they may need some-- you might need to sort of explain to them what happened today and bring them up to speed, especially if you are like dormmates, or hallmates or something like that.
Please help them along and help them get ready so that by Monday-- get the conversation going on online so that they can at least have a clue of which team to join on Monday.
PROFESSOR 1: The rest of the time is yours.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I also had a couple of students asking me about, what do you do after this class?
If you want to take more classes in games, if you want to take more classes in game design, if you're thinking about the spring semester, I just wanted to throw some course numbers up here.
So there's CMS.301.
It's Introduction to Game Design Methods.
If you're interested in design research, if you're interested in design methods, and you happen to also be interested in games, but you're likely going to be doing things in design elsewhere-- so that might be application design, toy design, anything physical, anything game-like, whatever.
The methods they're going to teach in there are going to be useful for all those things.
But the class is focused on using game design as a way to talk about design methods-- so CMS.301.
CMS.608 is a class Phillip teaches, and I co-teach with him, on game design-- so if you want to know more about how games are designed.
Some topics we talk about are things like game balance, tuning.
The projects are all board games and card games.
So the entire class is done with analog game design.
So we can actually get more detail into how rules work, how rules mix together.
We play a bunch of board games.
We analyze them.
And then we make board games.
It's very similarly structured to this class.
It's team-based.
You'll do four game design projects in that class.
There is CMS.610.
That is Media Industries and Systems, the Art, Science, and Business of Games.
That's taught by Chris Weaver from Bethesda Softworks.
He worked on the Elder Scrolls series for a while.
His class is focused on the business and publisher side of the equation of game development.
So if you're thinking about doing this as a professional career, at some point you're likely going to have to deal with publishers.
Either you're going to be working for one, you're at a studio that will be working for one, or you will be pitching to one yourself.
So the big focus of that class is the pitch.
How do you pitch your ideas to another group?
And it's also focused on the demo.
So if anybody's heard of the term E3 demo-- basically how to make your game look good even though it doesn't work.
Hopefully the game works at the end.
But it's really focused on, what do you need in your game in order to communicate that game to another person in order to provide funding for your game?
So that's CMS.610.
It's a really great course.
He has a lot of really good war stories from what the game development process was like from the '80s on up.
And then lastly, CMS.617-- Sara and I teach that course.
It's called Advanced Game Studio.
It's basically an independent study, but with teams, to make games.
You spend the entire semester making a single game.
Sara and I act kind of as your publisher, kind of as your mom and dad, kind of as your mentor.
Basically we are there to help you make your game, and to help your game from start to finish.
But you have something that you can either submit to a festival, you could show to another person, or put in a portfolio.
Before you start this class, you should be in a team already.
So we'll be sending emails out about this.
We try to do a few events in January for networking and for people to meet each other to form teams.
But you could form teams with people you've met in this class and other classes.
I have a feeling that people who took this class last year are going to be looking for teams for this year.
So there'll be some emails going out around December and January, and hopefully one or two events going on at the Lab in January about that.
Yeah.
And then if anybody has any questions about if you want to minor in games at MIT, if you want to do a concentration in games at MIT, you can do that through Comparative Media Studies and Writing.
You can talk to me about that after class.
You can also go to the CMSW website and talk to Rebecca Shepardson about that, and what the requirements are for that.
It's comparative media studies, so it's multiple media.
But games is one of those media that you can focus on and concentrate in.
OK.
So that is the advertisement part of this class.
So today we're going to finish up team formation.
I'm going to introduce the Project 4 deadlines, which I briefly mentioned last week.
And then we're going to start Project 4 today.
You've got a lot of time today to do brainstorming and to do prototyping in your teams.
We're recommending, because you're going to have eight-person teams, breaking up into smaller groups to do some initial prototyping.
You want to basically think of this as prototyping as a means of brainstorming to come up with lots of different kinds of mechanics and actually try those mechanics out, and play with them a little bit, before you decide on one.
We're giving you a lot of time to do this this week because we want you to have something playable by next Monday.
So that's that.
For teams, it sounds to me like we had five topics that were interesting to folks.
Is anybody still interested in the malaria topic?
OK. Not malaria.
Why do you let me come up here and talk to you?
What was it?
Ebola.
Ebola, yeah.
So nobody was interested in that.
All right.
So I've got early warning leading to early action for heat waves, cholera prevention and hygiene, forecast-based funding, SNAP, and adapting to climate change.
Even though we've got five topics, that does not mean we need to have one team per topic.
We can have multiple teams doing the same topic.
And that's OK.
So raise your hand if you don't think you're on a team right now.
OK, that's a good portion of you.
So what we're going to do is, just like last time, we're going to break up into groups based on the room.
If you're interested in the topic, go to that group.
Talk with those folks.
If you are in that group and you signed off, you said that you are definitely on this team through the mailing list, identify yourself to the group as yes, I signed up for this on the mailing list.
This is what the team is so far.
Here are the people on this team right now.
And actually, to help that through, what I'm going to do is put these on the board.
And then I'll have people who signed up for that team come down first.
And then everyone else come down.
So over here, let's do heat waves.
Over here, let's do cholera, SNAP, climate change, and forecast-based funding.
So if you are already on one of these teams, and you've said this is the team I want to be on, go to that area right now.
Ah, whatever.
That's OK. All right, so heat waves-- one, two, three, four, five.
Cholera-- one, two, three, four, five.
Cholera?
Six.
Cholera, seven.
Are you-- standing up-- anybody here on SNAP?
One, two, three, four, five, six, seven.
OK, cool.
If you're on SNAP, come on down with the rest of the group.
Adapting to climate change-- if you're on the team, or if you think you're on that team-- OK. And then forecast-based funding, if you think you're on that team-- one, two so far?
Everyone who thinks they're on a team, put their hands up
OK. All right, so if we're going to-- one, two, three, four, five, six, there's only six.
OK.
So move to whatever team you're interested in right now.
Talk with some folks.
If you're on the team already, talk about some of the things that you talked about last week, brainstorming-wise.
If you want to be on that team, talk with the members and ask if you can be on that team, basically.
I'm going to check in on you in ten minutes to see what it looks like
Right now it seems small enough that we could be one team.
Also Liz wants [INAUDIBLE].
[SIDE CONVERSATIONS]
All right.
After this amount of discussion, how many people are still undecided about what team they want to be on?
Kind of undecided?
Anybody else kind of undecided?
All right, so if you are 100% decided, put your name on your post-it.
Put it up on one of these things.
Do it really, really quick.
Come up.
Put it up.
Sit back down, please.
All right, so the dangers of having more than 10 people
Yeah.
So in fact, there is already dangers of having eight people or more.
But this class was actually intended for you to actually feel that pressure, right?
When you've got an eight-person team, you're going to have communication problems up the wazoo.
That table might tip.
Be careful
It'll be fine
Yeah.
Your laptop's on it.
Once you get to a 10-person team, what we've seen from previous years is that 10 people and larger teams don't actually outperform six-person teams.
They actually get about the same amount done
Or less
Or sometimes less.
Because the communication overhead actually breaks down further.
You might get a lot of content done.
And it turns out to be very uneven in quality, because you have multiple people doing content.
Now you could succeed in breaking your team up into little subgroups and be able to organize yourselves that way.
But we've never seen it done successfully in this class with students with your level of experience.
So what we will recommend is that if you've got something like 12 people-- or 13 people, now?
13
Yeah.
You will probably get more done by splitting into two groups.
If you are deliberately brainstorming in order to take advantage of 13 people, then of course you're going to come up with a game that can only be executed by a very large number of people.
What we're just saying is that those games have traditionally not done very well in our class.
They may be functional.
They may actually deliver on the thing that you intended to make.
But we've seen six-person teams do much more.
Partly because they iterate a whole lot faster.
And they may have smaller ideas.
But the ideas end up being a whole lot more polished.
So that's our advice to you.
If they really want to do a 13-person team, are we going to let them?
Um, wow.
Yeah
Against our better judgement
Against our better judgement, we might.
But what we're going to ask, I think, today, is to break up into two smaller teams for brainstorming.
We actually want all the big teams to even break up into smaller teams anyway to do some brainstorming and to do some prototyping.
Once you have all that figured out, in class today-- by the end of class today-- you can decide how big you want the team to really be.
The one issue that we have, though, is we've got some people who are not in class today who will be coming into class.
We've got the three that we talked about.
There might be one or two more.
If there are any other people like that, they will be automatically put on the smallest of these teams.
So I'd rather a six-person in cholera turn to seven, rather than an eight turn into nine.
Because these eight-person teams are actually just good enough
Yeah.
The eight-person teams are just about the size that we were hoping for.
Six-person teams, actually-- to be perfectly honest, I think the six-person teams have an advantage.
But that's just how the numbers break down
Yeah.
When we have seen a 10-person team actually work, what really happened was it was 10 people.
Eight of them worked on the game.
Two of them worked on the networking component.
The networking component did not work.
Yet the game was still as feature complete as it could be without networking.
It could work without the networking.
That's not what we're looking for.
We want the game that you are submitting-- the game that you're creating, the design document that you're submitting to us on Wednesday, and the game that you're playtesting on Monday-- to be the actual game.
All those features that you're saying are going to be in that game.
We want that to be the actual game that you submit at the end of the semester.
So don't think of it as, well, if two people flake, then it's back up.
It's not.
If two people flake-- if you're too big, and you've got two people who don't know what's going on, and they cause some issues, it's just going to be a disaster for the entire team.
So this year, all of the Project 2 and Project 3 teams were actually bigger than we've done them in previous years.
Previous years, they were about four people, maybe five tops.
And we had you do six and seven, I think, right?
Did anybody feel like with seven people they got a communication method that worked really well?
Like already--
I see-- what was the method?
[INAUDIBLE].
It's not perfect.
[INAUDIBLE]
Yeah.
It's the sort of thing that-- whatever method you use will definitely improve with practice.
So if you've got something that sort of worked well, that's something you try again.
But the fact that most people didn't should already give you some warning of what a large team is going to feel like.
It's going to be tough
For the structure of the class, on Wednesday we're going to be talking about more of the requirements-- the aesthetic requirements-- for Project 4.
We'll have basically another design discussion next Wednesday.
On Monday we're having a short design discussion about educational games.
And then we'll have one of our first discussions trying to talk about some of the person-to-person issues we see in making games.
In this case, it's team dynamics, and how to identify team dynamics.
We'll have a couple more talks like that to kind of help you get used to running in a bigger team.
But between now and next Monday, we're hoping in class to give you plenty of time to actually work on some of it.
One of the biggest issues with working in a big team is just figuring out, what is that core thing that you're actually making?
What's the thing that you're trying to communicate to each other?
We've talked about how you can use the division document for that, and how you can use the actual prototype you make for that.
But often, it is just having everybody in the same room at the same time talking and just working together.
So we're trying to give you as much time as we can--
In class.
because it's hard enough to get six people outside of class to meet up, let alone an eight-person team.
Right?
So we're going to give you more time in class to meet up with each other.
That does not mean that you should be spending all your time in class planning on getting your face-to-face coding done.
That's actually a waste of time.
Because you could be talking to each other.
You can use it for things like code reviews
Yeah, code reviews, stand-up meetings-- things like that.
All right.
So we're going to put this on hold for a second.
I'm going to go over, very quickly, the deadlines.
And then we're going to take a break and come back.
So if this will move forward-- all right.
So why did I react so strongly about the Animal Crossing, all these sorts of mini-games?
It's because of what we're asking you to do-- whoa.
Don't do that.
So what we're asking you to do is create a small, fully functional, polished browser game.
One of those many games would count as a small, fully functional, well-polished browser game.
And we're asking you to use the design iteration techniques you've used in the past.
What does that mean?
We're actually talking with the same design scope as the previous two projects.
It's just you've spent more time on it to actually get it working exactly the way you'd want it to work.
It may be slightly larger.
So if your previous two products had maybe one core mechanic and one supplementary, there might be two supplementary mechanics to that core.
But that's about it.
If you've got a game that has an overworld, then the overworld should be the bulk of your game.
If the mini-games that you decide to make are the same size as that overworld in scope, that's already way too over-scoped, if that makes sense.
It's an eight week project.
You've got about two weeks to concept and prototype multiple game experiences.
So what we're asking for today and tomorrow, or on Wednesday, is you're not just prototyping one game.
You're prototyping lots and lots of different little games.
You're looking at all of those games.
You're actually going to have time to spend on multiple different prototypes, rather than just a single prototype that you iterate on.
So what we're asking is if you've got an eight-person team, break yourself into two groups of four, four groups of two-- create some really small, board-gamey-looking things, some very small interactions.
Maybe they're playable.
Maybe they're something you just describe.
But really go big blue sky.
And try to think of all the different things that can happen.
Because these topics that you're dealing with are very large.
Make sure that your design is grounded-- very, very well-grounded-- in the sources that we've been given by the client.
There's going to be multiple times that they'll be here to see your games and talk about your games.
They'll be on your email, so that you can ask them questions about that as well.
So design constraints-- use all the mechanics that we've been practicing in class with your previous projects
Not all of the mechanics simultaneously.
Use any of those mechanics
Any of those mechanics.
So it could have trade-offs.
It could have randomness.
It is not required to have both.
It's required to have whatever it takes for the topic that you're working on, whatever makes sense for the topic that you're working on.
Those top two tend to be in the realm of what those topics kind of lean themselves towards.
But if you find something else and you want to use that something else, great.
Spend some time really working on it.
Spend a couple weeks working on it before you make a full commitment.
So your target audience-- each of those topics has their own target audiences, which they talked about.
The top three are the ones to really focus on, depending on your topic.
So Red Cross staff and volunteers-- people who are knowledgeable and motivated about the topic already, who just need a little bit of extra information-- community organizers, and then youth-- people who are motivated, but maybe not knowledgeable about the topic.
Again, the other requirements-- we are holding you to these.
Maximum play length of 10 minutes.
Single or multiplayer game.
Test your user interface well.
It must use and play audio for the player, unless audio is absolutely not required for your design.
If you design really does not need audio, then don't use audio.
But a, if it's a web browser experience, there probably should be some way to use audio as a method of getting feedback to the player.
So it could be as simple as a ding when something happens.
And that's OK for this project.
What matters to us is that you have a unified aesthetic, which we'll talk about on Wednesday, what exactly we mean by that.
And then also we're asking you to give design thought to spectating users.
In the final presentation, you're going to be presenting the games on the projector.
People are going to be watching the game as it's being played.
They should be able to understand how the game is worked, just by watching it.
Maybe they might need to watch it one or two times to figure that out.
But they should be able to figure out, what are the things that are going on in the game?
And how are all those interactions happening?
So your deliverables-- that was from up till now.
So on Wednesday, turn in to Stellar a high-level design doc or "back of box" copy-- the same form we've been using previously.
It's some kind of vision statement that says at this point, these are what we think the features are going to be.
If at any point you make a significant design change that says, our feature list is going to change, our game is going to look different, update that document.
You have up until the end of the semester to do that.
And whatever you turn in-- whatever that document is on the last day of class should be what game you're turning in.
So that document and the game should have the same features in there.
But we would like to see any changes that happen between now and then.
We'd like to see how those changes happen and why.
So reflect them in that.
On Monday, turn in a product backlog.
You're going to do a two-minute presentation-- the core of your game design idea.
And then we're going to do a playtest.
On the 27th, we are going to have at least one, possibly two, of our clients in on that day to play those games.
Jano and Jared-- both of them came in for the first day of class.
They were not in class when Pablo was on.
We are asking for some kind of Sprint tasklist to be submitted to Stellar weekly.
What exactly that is is completely up to you.
We've given you a format that we like.
We've given you a method that we know works.
If you decide to change that method, let us know what you decided to change it to and why.
But whatever you have-- every week, I should be able to go to Stellar and see what tasks you have to do for the next week's worth of work.
We are asking for product backlogs.
That's actually a concrete.
Please give us a product backlog.
The Format you use for the product backlog doesn't matter to us-- the feature list, if it's a list of user stories.
What matters is that all the features that you think are going to be in the game are listed.
All the features that are done are listed as done.
And you've done some kind of estimation on what size those features are.
And then we're going to have you do two-minute presentations in class.
And all these dates are in Stellar, in the Stellar calendar.
So the next two-minute presentations are also on November 12 and then November 26.
Any questions so far about any of this stuff?
All right.
So the project is due on Wednesday, December 10.
On Stellar, turn in all of your previous builds, your written postmortems, your design change logs, your updated document.
And this time we're asking you for focus test reports.
There are in-class playtests on the 27th, the 5th, and the 24th.
Two of those can be used for focus test reports.
The other two must be external.
And then, yeah.
We're going to have postmortem presentations.
So on postmortem presentations, instead of five minutes, you're going to have 20 minutes.
We're asking for a polished presentation of the work that went into developing the project-- all of the iterations that you made.
So we want to know about your design iterations in this presentation.
We also want to know those five-- what went right, what went wrong, what was interesting in the project.
So that's about 10 to 15 minutes given on process, and team management, too.
For all of your games, someone who hasn't played the game before will play the game live on stage.
So that's basically one way for us to see somebody who hasn't played the game before try it out and see how it works
Who's in charge of acquiring the person who hasn't played the game before?
It's a little bit of you and a little bit of us.
We'll do the best we can.
So we're asking guests to come in.
We're hoping our clients are going to be in.
Some of them might have already seen your games before.
Some of them haven't.
If you have friends, we actually suggest you bring in friends to come in and to watch the presentations-- maybe not play your game, but play somebody else's game.
But we open the final presentation to the public.
We have a rehearsal on Monday the 8th.
And we have the final on Wednesday the 10th.
So on Monday the 8th, you don't need to do the play the game part of the rehearsal.
But you do need to do a full dress rehearsal of the entire presentation.
And the reason why we're doing that is we're going to give you feedback on that, on what you might have missed, and what we'd like you to see in the presentation.
And you've got a couple days to apply that feedback to your presentation and to make it better, basically.
And I think that's it.
Yeah.
That's all from the other day.
So any questions about the postmortem presentations?
Previous years-- I'll send out the link to previous years, so you can see what these tend to be like.
But we've recorded them in the past.
So you can see what other people have done.
And that'll go to the Announcements page on Stellar.
All right.
It's 1:50.
Take a 10-minute break.
Come back at 2:00.
And work in your teams
So the rest of class today is pretty much for you to figure out what is this project, and get working on who you're going to be working with and things like that.
So what we're going to first do is the typical brainstorming process.
We're not going to guide it so much this time.
But just a quick reminder-- you want somebody who is in charge of facilitating it, which means reminding everybody what the problem is, and also calling out people who may have trouble getting their words in front of the group, like people who are trying to get in a word edgewise-- so basically moderating a discussion.
And someone, of course, should be taking notes.
We have our giant pads of paper in front.
You can use chalkboards.
For the cholera team, we suggest splitting in half.
But that doesn't necessarily mean that your brainstorming team is necessarily your game team.
What we are thinking is that after brainstorming you will better know whether your ideas can be split into two projects or not.
Given how many cards disappeared on there, we really, really, really recommend that you split in two groups.
But there may be a more organic to split that.
Because your ideas may actually fall down on a more clearly defined lines.
I noticed that you already did some brainstorming up there.
But think, what would you do with half the size of the team?
Then after that, have the discussion about who is on which team.
By the end of class today, you should know exactly who is on your team, how to get in touch with each other, what your email addresses are, what your phone numbers are, or instant messaging handles or whatever.
And make sure that you've figured that out.
Let's see.
One thing that I wanted to tack on to Rik's presentation earlier about the expectations for the postmortem-- we brought up the person who has not played your game before coming up and actually playing your game.
And I know some of you might be worried about that.
Do be aware that however well or badly that person performs, no matter how much they like the game, or completely misunderstand what your game's about-- that is going to be, at best, 1.5% of your total semester grade.
That's not really significant.
It's not going to really swing anything.
So don't worry about it.
But we are actually using those sessions to gauge usability.
To what extent have you actually designed a game to make your game easy for people who've never seen your game to get into it?
These are the things that we want you to be iterating on.
These are the things that we want you to be catching for.
And without having a real-life, high stakes-- there is going to be a person who hasn't played your game in front playing your game-- you may not take it so seriously.
So we want you to be taking usability really, really seriously.
That's why Rik said, the scope of the game doesn't necessarily need to be all that different from the past.
But the amount of polish, the amount of time that you've thought through your UI-- it's what you should be spending all your time on.
Finally, we're going to go a little bit more into games for learning in the next week.
But one thing that we want you to be thinking about while you're brainstorming is, of course, what is the player doing?
Now if you're trying to teach something to someone, if you're trying to get them in a situation where they're learning about a certain concept, it is ideal if the thing that you are trying to teach them is actually the thing that they are doing.
Now there's a couple of other ways-- actual, can we bring up Constantine's slide?
Sorry.
The dongle's there, too.
There are a couple of other things that you probably are already considering putting into your game.
Things like, what is the theme?
I noticed the brainstorming for the cholera already started talking about things like story line.
So does the theme of the game actually match not just the topic that you're trying to teach, but also the mechanics that you're designing for it?
You want to be thinking-- and a couple of you have already been asking-- about things like, who's going to be playing your game?
What's the demographic?
Also be thinking, what situation are they likely to be playing in?
Are they likely to be playing it on their own at home?
Is what you want to design for?
And you can specify this.
You can say, this is a game that's meant to be played by an individual person with an individual computer on their own time.
Or this is meant to be played in a group like the SNAP game was played in a group, in front of everybody else.
And that, by the way, falls into the framing idea.
What is the context in which someone's actually going to be playing the game?
To what extent have they played games like what you're describing?
If you are making some sort of hard core strategy game, to what extent do people need to be familiar with strategy games in order to be able to play the game?
Or do they just need to be familiar with things like Facebook games, for instance?
Or do you really want to make a game that doesn't assume much play literacy at all?
You want to be looking, of course, at the data, and that means all of the material that we provided you, as well as any information that you're looking at in Google searches or academic research, or information that you're getting from your domain experts, from either the Humanitarian Response Lab or the Climate Centre.
Is that right?
Yeah.
So make sure that we are representing that as fairly as we can.
Because this is a game, you do have liberties to at which point you can sort of abstract out those numbers.
Maybe they're not mathematically very, very precise, because you're trying to get a sudden feel of the game.
And that, of course, goes into the aesthetics part-- not just the graphical aesthetics, but the way how your game feels.
If you're trying to represent a really, really tense situation on the ground, and the way how your data model makes it resolve makes it feel like your game is really clinical, and it's very, very easy to deal with the situation as long as you know the tried and true formula, then you're not accurately representing the situation.
Even though you may be accurately using data.
So make sure that you're balancing that aesthetic feel together with the content and whatever real world facts you're putting into your game.
Because these games are all really about real world things, you might want to make sure that your fiction and narrative is something that's not too fantastical.
Or at least, if there is a fantasy element of it, it is the part that has least to do with the believability of the things that you're trying to communicate.
You can say that there is a village or a city that's dealing with a cholera epidemic or something, and that's not a real city anywhere on a map of the world.
But you can look at that city and imagine this could be a real city somewhere in the world.
You could go as far as saying, OK, this is like the dark ages or something like that.
Because the disease has certainly been around for a long time.
But you want to be able to keep reminding people that this is a real thing.
If you say that this is a science fiction thing where you're on a different planet and it's a completely different kind of bug that behaves exactly the same as cholera-- I'm starting to feel that maybe the fiction that you're creating for your game is undercutting what you're trying to get this game to do, which is to actually teach something real
To jump in on the fiction narrative-- so things to think about.
Your games are being played by a specific culture-- people with a specific cultural background.
Especially the cholera game will likely be played by-- I believe it was Ghana.
Yeah, it's Ghana.
So students who are in Ghana-- what kind of culture do they have?
You're going to have to research that and figure that out for those games, to figure out what kinds of common backgrounds might they have that you can talk about.
For the heat waves game, that actually has more of a focus on Red Cross volunteers and university students.
You can specify, we understand that the people who are playing this game are likely going to be from Europe.
And they're likely going to have this kind of background.
When you're creating your high level game design document, come up with what you imagine your target audience to be, so you have some kind of cultural background to use.
Again, you're choosing fiction that is going to be so supportive and helpful, and not fiction that might be not known or might undermine the idea.
One thing to keep in mind with the fantastical-- studies have found that some games-- when it comes to pandemics and outbreaks, using the zombie fiction actually can work sometimes.
There is something to be said about using a common fiction that people know.
People can say, all right, I understand how zombie pandemics work.
Because I've seen all these films about it.
Whereas if you just told me it was a disease, I might not have seen enough films that had anything about how disease actually spread.
So really think about what is your problem?
What fictions might be useful?
What narratives might be useful for the problem that you're fighting?
Use fiction to keep the topics relatable.
And if that means drawing on things in popular culture, that's great.
That's a tool in your hand.
Don't use fiction as a way to paper over design problems, which I've often seen.
You know, if we set this game in space, then we can sort of skip over all reality, or something like that.
When you're brainstorming, make sure you know what you're brainstorming about.
Are you brainstorming mechanics?
Is it about what people are going to be doing?
Are you brainstorming what the fiction is going to be?
I would say do a separate brainstorming session for each one of these things.
Maybe just spend 10 minutes.
You don't have to brainstorm every single thing that we're showing you here.
You could brainstorm what is they aim and impact that you want to achieve in this game?
How are you going to do that?
Are all of you even on the same page about what you want your game-- the kind of change you want to see in the world?
And that's going to be your purpose.
That is going to be-- do you all understand what this thing is?
A lot of it has been defined for you.
But I'm not guaranteeing that everybody on your team necessarily shares the same idea.
So the brainstorming process might be able to get that out.
And at some point in time within the next couple of weeks, you should have a pretty good idea.
You should have had a chance to at least talk with your team about every one of these elements.
This is a research framework that's been put together by one up our colleagues, Constantine [INAUDIBLE].
I think we can put a link to this
Yeah.
I'll post the slides, and this particular slide, to Stellar
And we might get back to elements of this later in the week.
But right now, this is just basically something to help guide your brainstorming
Yeah.
Aesthetics we're recovering on Wednesday.
Fiction we're covering in a couple of weeks
Finally, one reminder about brainstorming-- if your brainstorming session runs more than 10 minutes, you'll want to flip your secretaries around.
So that somebody else is taking notes.
Let the person who's been frantically writing things down actually get their ideas in, too.
OK?
We're not going to time that for you.
That's going to be something for you.
And once they've got their teams, email, video game bosses?
Put your vision statement-- it should have all your team and all your member names in it.
And that's due on Stellar on the date that it says, on Wednesday
So put all your names in the vision statement.
The rest of class is yours.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so a couple really quick announcements-- on Monday, I mentioned a number of other classes that are offered in the spring if you're interested in game development here at MIT.
But I forgot this one.
This is 11.127.
It's also cross-listed as CMS.590.
It's taught by Eric Klopfer and Scott Osterweil.
Scott should be in our class next Monday, I believe, to talk to us about learning games.
But if you end up being interested in games for learning, games for education, this is a great class to take.
And it's offered in the spring.
We also have a class, CMS.615, Games for Social Change, that's being taught this fall.
So if the topics of the games that we're working on for Project 4 are something that you just find yourself really interested in in the next couple years, those are two classes you can take to explore that further
All right, so I guess for me, it's talking for an hour, and then after that, teamwork, right?
Absolutely
OK, so today, what we're going to do is talk a little bit about aesthetics, about what we mean when we say we want your game to have a cohesive aesthetic.
We do realize that the games that we're asking you to make are extremely time constrained.
We've been asking you not to go terribly much beyond the scope of what you've done before.
But what we want to see in your game is a really, really cohesive aesthetic.
We used to tell students that we wanted them to put out polished games.
But there was a really unclear definition when we used to do this about what a polished game would be.
And we also realized that for some students, for a lot of you, you've seen games that are really, really, really polished.
When I say a polished game, can you come up with some examples of things that immediately come to mind?
Limbo
Limbo?
Yeah, that's the black and white side scrolling platform, really, really spooky, gorgeous visuals and sound design.
What else?
Yeah
All of the Final Fantasy games have really polished UIs
Pretty much, yeah.
Every single Final Fantasy game for its era is probably like the high watermark for production values-- not just the UI.
But you're right in saying the UI is alive.
Whenever you even switch to a different menu option, it does this little animation
It's one of the only [INAUDIBLE] that has a really good UI design.
And I've looked at a bunch of them.
And it's like, Final Fantasy is just way better than all that stuff.
And of course, the graphics, too
Which is amazing, because they change it so much every game.
Unlike a lot of other franchises, they change the formula a lot, but also of course other things, like the animation quality and the music and the voice act-- maybe not the voice acting.
Yeah
Meat Boy
Super Meat Boy?
Yeah.
[INAUDIBLE] physics
Yeah, for folks who haven't played this game, it's a really punishing game.
But it makes it super easy for you to try again.
In fact, there's like a single frame between you dying and you starting the game again.
And it goes right off.
A game built by two people, by the way-- it's an indie game.
You can see it in documentaries like Indie Game-- The Movie.
And you can see even though they're working on their own-- and they do spend a considerable amount of time on an extremely visually polished and musically amazing game as well
Monument Valley
Monument Valley for the iOS?
Yeah, I can't remember, has it come out for Android yet?
But it's definitely iOS right now.
But what strikes you as polished?
The graphics and the gameplay is just very smooth
Mhm, and the game's kind of like a flat shaded isometric game, if you're familiar with that 3D concept.
It means that things that are far away are the same size as things that are close to you-- isometric orthogonal projection.
The camera is kind of looking at the world at a 45 degree angle up and sideways.
Everything's flat shaded, but it's got incredible art design.
It's got art direction, I would say, just how all of the characters, even though they basically all have one color, maybe black and white, move with a lot of expressiveness.
They have iconic characters, and you immediately sort of recognize what each character is supposed to be, even though the game is pretty much made out of geometric shapes.
I call out Hearthstone.
Anyone playing that?
Anyone else?
Yeah, it's like just every little bit of movement that the screen does as soon as your finger touches that screen is amazing.
Even little silly things, like when you are looking for a match, and you see all the other people who you could have been matched against.
And it of course always selects the perfect match for you.
It's a ridiculous little UI thing.
But it sort of gets you in the mood.
And of course, that's like music and stuff for what's really a card game, a collectible card game, right?
So what I'd like to do right now is play through a game that I feel has a coherent aesthetic.
But maybe you wouldn't necessarily call it polished.
But it gets you a little bit closer to what we're looking at.
Because it might be a little bit more feasible for this class.
So who hasn't played Sissy's Magical Ponycorn Adventure yet?
Who would like to, actually?
All the hands went up.
Come on down, yeah, you, yeah
Sissy's Magical Ponycorn Adventure
Is that too loud?
Is that OK?
Hi, my name is Sissy.
And I simply love ponycorns.
It is the best thing in the world, because it's a pony and unicorn.
It's a ponycorn.
I don't have any ponycorns right now, so I'm going to go get some.
Come with me on my magical ponycorn adventure.
I can go through rainbows to visit new places.
That's not a ponycorn.
it's a goat on a pole.
[INAUDIBLE].
Hi, Orange Boy, my name is Sissy
[GIBBERISH]
Do you like ponycorns?
[GIBBERISH]
Me too.
Got any jars?
[GIBBERISH]
Thanks.
If I find any ponycorns, I'll put them in these jars.
This is an empty jar for holding ponycorns.
I see a new rainbow
[CACKLING] I'm an evil lemon
[INAUDIBLE]
I will squirt lemon juice in your eyes, and that stings
Oh no, what will I do?
Stay away
[SCREAMS]
Stay away
[SCREAMS] I found a key.
Hmm, this [INAUDIBLE]
Hi, I am a tiger, and I'm going to eat you
No you're not
Yes I am
No you're not
Yes I am
[SCREAMS]
I'm going to eat that ponycorn
I got a coconut.
It's a coconut.
I'm going to throw this coconut at you, lemon
Oh no.
Ow, my face
That's what you get for being evil and a lemon
Stay away
[SCREAMS]
Oh, I think you might have broken the game, but that's cool
What about a jar?
Oh, jar, yeah, try
This is an empty jar.
I got the green ponycorn.
I put it in the jar
OK, all right, so I think everyone gets the idea of the game.
Thank you so much.
Yay, thank you.
[APPLAUSE] So let me actually switch to my presentation.
There we go.
I'm going to talk a little bit about aesthetics.
Now, what about that game seemed off?
Let's start with, what in the game seems out of place?
The artwork, I guess?
The artwork seems out of place?
Out of place compared to the-- it is a bit of a trick question.
I have trouble figuring out what's out of place.
I think maybe the cursor hand actually looks like this default kind of browser cursor hand, is the one thing that I feel is not really cohesive of everything else.
The artwork kind of sets the tone, is pretty much the first thing that you see.
Even before you start playing the game, you see the artwork and the website.
And then of course, when the music comes on, you actually hear pretty much the girl's voice for most of the game.
You get to about halfway through the game before you hear a dad's voice playing the lemon and Orange Boy, I guess-- wah, wah, wah, wah.
Did anyone notice the user interface?
It's not obvious at first what's clickable and what isn't.
An outline could maybe help or something when you hover over something that you can actually click
What are any visual cues that you were working off to figure out what to click on?
Things would be drawn on different pieces of paper.
That wasn't clear.
But in general, clicking on everything is just what I do normally in point and click games
It is a click everything in the game kind of game.
So that makes sense.
There is actually a slight little bit of drop shadow behind everything that you can click.
And some things that you can click but don't do anything, like the goat on the stick, for instance.
I thought I saw another hand over here about the user interface.
Anyone?
I was going to say the drop shadows
Yeah, OK, the drop shadows, yeah.
Yeah
I don't think there's always a drop shadow between everything you can click on.
Because I'm pretty sure you can click on the cactus, and the cactus has no drop shadow
Oh,
She's like, ow, cactus.
And I was like, oh, that's weird.
That didn't give me any indication that collecting it would give some sort of feedback
I forget, does the cactus do anything?
Uh-uh
Ah, so I think there is a big difference, then
[INAUDIBLE] cactus
Yeah, everything that's like crucial to advance in the game I notice has a drop shadow.
But yeah, if you click everywhere in the world-- if it looks like something that might stand out from the blank sheet of paper, it should respond, right?
So even though the cactus is not sort of a crucial piece of the plot, such as it is, it is something that a player might reasonably expect to be able to click on and do something.
And yeah, you get pricked by it.
She says, ow.
There's some voice.
And there's a little bit of payoff on that.
Every time he clicks, there's a little blue crayon circle that sort of emanates from where you're clicking.
So there's another little typical UI feedback of, all right, you clicked on this thing, and we're acknowledging your mouse click, but we're going to do it in a way that sort of works with the rest of the art style.
I think something else that also stands out a little bit is-- let me see.
If I go back to the game-- oh, I closed it, didn't I?
OK, all right, does anyone remember if the border between the game space and the inventory, was that like a straight line, or was that kind of a raggedy thing?
I thing it's pure straight line, right?
Yeah, so that's another weird thing.
It's like, this is a game that's made out of like ripped sheets of paper.
Then all of a sudden, there's this hard white line right across.
What works in this game?
Why would someone want to play this to the end-- which just takes about five minutes.
It's not terribly long
It's sort of all whimsical.
And you kind of want to find out what this girl came up with, and what they worked together to create, basically.
The story behind the creation of the game is also interesting
Yeah, it's father/daughter, but the daughter led the project, basically determined how everything was going to work.
The dad just did the coding and some voices, of course scanning, and things like that.
But very quickly, you're introduced things like the goat on the stick.
Very, very quickly, it just establishes that you're just playing in five-year-old logic right now, and this whole world is going to work that way.
The fact that it's narrated largely by this character-- who has an opening cut scene, actually.
It's just like, I'm Sissy, and I fricking love ponycorns, and then explains what a ponycorn is.
And then you don't actually get to see one until pretty far into the game.
But as soon as you're told, yes, that lump of green is a ponycorn, is a pony with a unicorn horn, you then say, OK, all right, that's a ponycorn.
That's something I want.
And at that point in time, you already know you're supposed to put them in jars.
So that starts to talk about something that we've already been exploring a lot in this class.
It's kind of like the systemic aesthetic, how a person feels when they're playing through a set of rules.
And Sissy's Magical Ponycorn Adventure, it's the set of rules as devised by a five-year-old.
But they're actually all internally consistent.
When you play through the whole game, you start to figure out, yeah, clicking everywhere is actually a perfectly reasonable strategy.
Because nothing's really going to hurt you in this game.
You may get scared.
You may get pricked by a cactus.
But nothing's really going to hurt you.
So it's OK to click on everything.
And the kinds of decisions that you get to make are more like, is now the right time to use to jar on a ponycorn?
Do I need to do something first to get past the lemon or the other bad folks who get in your way?
And the kinds of rules and strategies that you have to devise to be able to navigate through your space gives you a certain kind of systemic aesthetic.
And you've been working with that since the beginning of the semester.
Prototyping-- some of you have made games that make you feel very tense.
Some of the games are more zen-like.
Some games are more strategic.
And you have to really think carefully about your next decision.
And some of them are kind of frantic.
And those have all come out from the rules that you devised first on your prototypes.
And then you moved them over to your digital versions.
And that's one form of aesthetic.
But then there's also the other kind of aesthetic that we've been talking a little bit when we talked about polish-- the art direction, the music, how a 3D object is rendered, or even how a 2D graphic is represented.
Is it drawn in crayon, or is it high quality line art, or is it something running in the Crytek engine, and you've got mobile mapping and specular lighting and things look like fire and glass and concrete?
Do the sound effects sound like they're coming from a place in the world?
Do they sound like the thing they're supposed to sound like?
Do they sound other worldly or little MIDI notes, the sort of thing that you would expect from a Nintendo game, from an old school Nintendo game?
And then there's the things that we would normally refer to as sort of the story of the game.
There's the plot of what happens to characters as they navigate through the world, and who are these characters, who are they really, and what sort of emotional development they're going to go through.
What's the theme of the game?
Is it a game about being isolated from everybody else?
Is it a game about US military strength?
I think the theme of Call of Duty is actually, America starts wars, England finishes them, which is kind of weird if you look at history.
And where is this game set?
When is this game set?
Hopefully that's clear enough.
And I think one of the things that can help is just a little bit of vocabulary so that when you're talking in your team about what's working in your aesthetic, it helps to be able to put words to it so that people can understand what part of the aesthetic that you're talking about.
Are you talking about the things that have to do with a game system, your rules and the logic of your game?
And how is that going to make someone feel while they're playing your game?
Are you talking about the style, the musical and the visual style of your game and how things move, how things animate, and how things [INAUDIBLE] the things you hear?
Or are you talking more about the fictional elements-- what's going on in the sort of storyline?
All these three things are going to work together to create the aesthetic of your game.
And all these things need to work together for your final project.
So I want you to be thinking very, very hard about what's the fiction of the games that you're creating.
Is it being set in a real world actual city, an actual named city?
Is it kind of like an abstraction of a city in a certain part of the world?
Is it not a city at all?
Maybe you're playing as the whole world?
Maybe you're playing as a completely different planet, and you're trying to get through your learning that way.
Is it about a family?
Is it about an individual?
Is it about an entire town council or something like that?
Style is something that you have to leverage in order to take advantage of whatever strengths you happen to have in your team.
If you have really good artists or a really good musician, you're going to take advantage of that.
But you don't want to do anything that's going to sort of show the assets that you don't have a lot of control over as being poor.
You want everything to be kind of a matching even standard.
So for instance, if you did a game that has mostly pixelated 8-bit style graphics, and all of a sudden, this vector art shape comes in, you better have a really good reason for that thing to be there, or a photograph shows up in your game or something like that.
You better have a really good reason why you're throwing in two very, very different visual styles in there.
You can make it work.
But you need to be able to think very, very hard about how you're going to make that work-- even things like what your cursor is going to be.
And of course, how's that going to work with your system, right?
If your game is about a dire situation-- floods are coming in or something, there's an epidemic-- and you put everything in kind of nice, sunny, tropical, everyone's at a beach kind of a setting, well, you can sort of do this like weird subversive thing.
But then you have to be very, very conscious.
It might be a little bit easier for you to just use a visual that actually supports the kind of game play that your game actually has.
If the game is about feeling tense, use visuals that also communicate this sense of feeling tense.
There are games like-- how many of you have played Tropical?
A couple of hands out there.
How about SimCity?
Who's played SimCity?
OK, a few more hands.
So what's the difference between Tropical and SimCity from someone who's played Tropical?
Yeah
The flavor is completely different
The what?
The flavor.
Tropical is overall kind of various Caribbean [INAUDIBLE]
There's a tropical sort of equatorial setting, as opposed to SimCity, which is this very, very urban probably temperate climate.
In one game, you're the mayor.
In another game, you're set up as a dictator.
And the kinds of things that you do in Tropical are supposedly things that are very similar to SimCity, like building and zoning and constructing up your little island.
But at the same time, you also have to do sort of dictatorial quashing your resistance kind of things, which you don't usually do in SimCity.
But it's set in this kind of very idyllic tropical sunny environment.
And developers of that game are usually trying to play that for laughs.
It's like, yeah, you're a dictator.
But you're not really that evil compared to some of the real atrocities that you've seen in tropical dictatorships in the real world.
And so they're trying to use the visuals to get across that you're kind of this comic dictator.
I can certainly imagine a game that's a lot more about real dictators of small island nations.
And that will probably be a lot more-- the visuals will probably not be quite so cheery.
Even though it might be sunny, it will probably show a little bit more of the grit of what it's like to live in a very humid and very muggy and rainy place, and try to play that up so that that aesthetic works with how the game play actually is going to work.
So I should expand on that.
So we talked a little bit about how these things work on their own-- fiction, system, and style.
But in between, there are sort of these domains of game development, which are going to draw from both of them.
How a player interacts with the system and the visual feedback they're going to get from that in the user interface is going to also affect how they feel about your game.
And it's going to affect that aesthetic.
The way a certain character has a theme song, for instance-- this one character who's been lurking in the background, and you've always heard the theme kind of in the background.
Then they show up in front.
And then suddenly it has a full orchestration of their song.
Things like that are how style and fiction work together, this particular character's theme, whether it's Gollum, or the theme of one of the big cities in Lord of the Rings, or something like that.
And of course, the level design of your game itself both have gameplay implications and sort of fictional implications.
A post-apocalyptic racing game looks very, very different from, say, a Formula One modern-day racing game, or a horse racing game.
And that's going to not only change the way how the game looks.
It's also going to change the way how the game plays.
Is there debris on the track?
Is everything very dusty?
Is everything highly polished and very clinical, for instance?
And of course what your characters can do in an environment-- if you say someone's a ninja, then I expect this character has ninja-like abilities.
Is this, oh, you're a ninja, and you kind of punch people in the face in sort of one on one fisticuffs?
And it's like, that's not a ninja.
You're thinking of something else there.
Maybe you shouldn't call that person a ninja.
So we've looked a little bit at Sissy's.
I want to show a couple of other games that I feel do really, really nail a cohesive aesthetic on a very low budget.
And just for the sake of time, I think I'm just going to bring them up without asking people to come down and play.
But how many people have played Thomas Was Alone?
Just about five people, OK. Oh, there we go-- OK, it was just loading up, cool.
So this is a game that's mostly about rectangles.
There we go
[INAUDIBLE] I'm glad I'm not making you play this.
Because this is really giving me a crick in my neck
[INAUDIBLE]
OK, I'd like to show you a little bit more.
But I find the contrast on that screen is a little bit too low to actually see what the level is like.
But it's pretty much all geometric objects.
They're not all at right angles.
Not everything in the game is a rectangle.
And you saw some of the water kind of had some texture to it.
But it's pretty much all flat shaded polygons.
The majority of the game is all flat shaded polygons.
What in that very brief clip that you saw sort of gives you a sense of what the developer of the game wanted you to feel about the game?
What were some of the elements?
What were some things that were working in that game?
There's a lot of empty space.
And it sort of adds to the environment of being alone
Yeah, you're very, very small in the screen compared to the rest of the screen
The music also gives that same impression.
But it's not too depressing.
Because you've got this sort of story quality, like [INAUDIBLE] 0.8.
So presumably you're going to get to the part where he's not alone, or whatever
Mhm, yeah, it's Thomas Was Alone, not Thomas Is Alone
Also, I haven't played it
No, you are making some very logical assumptions about this game, which are actually entirely true about this game.
Eventually, you find other cubes, rectangles actually.
All right, and so that's the way how the music works to sort of set the tone of, yeah, it's kind of melancholy, but not completely depressing.
You've got a bit of a beat.
What else?
I can't remember if you mentioned the voice acting.
But what's that doing to the game?
First of all, who's speaking?
Is it the cube?
Some external narrator
Some external narrator, yeah.
So that's this.
And what does that imply?
What does that tell you about what this game is trying to get you to feel?
What comes to mind?
Hmm?
[INAUDIBLE]
Say what?
It's a story
That it is a story.
This is actually more of a narrative based game
It makes you fell sort of isolated, like you're being spoken to by something far away, almost nonexistent, and you're sort of by yourself
Mhm, so I think there's a little bit of the game that you're kind of being observed by this third party that's not actually part of the story, but is narrating the story of this rectangle.
So you're alone, but not exactly, right?
There is this third party there, which is there to tell your story.
And somehow, this story is going to go places, as suggested by the music, but also as suggested by some of the things that he says.
He makes mental notes for the future, even though the mental notes are kind of a silly one.
That implies there is a future, right?
Yep?
It makes you feel empathy for the character, even though he's just a rectangle
Mhm, yeah.
Actually, let's talk about that rectangle.
What does the rectangle do when it jumps?
[INAUDIBLE]
Hm?
It gets thinner
Yeah, it gets a little squash and stretch animation there.
For something that's really just four right angles, it's actually remarkably expressive when it actually does a single job.
It gives you the sense that this thing is not an inanimate geometric object.
This thing is alive.
This is an organic square, or rectangle.
OK, so we've gone through a couple of ways how this game achieves its sort of aesthetic quality.
Let me see what my next slide was going to be.
All right-- oops, darn it.
There we go.
I'm hitting the space bar.
Whenever you see a big circle radiate, that's me hitting the space bar.
What does that even say?
Oh jeez, OK. OK. OK, so I'm going to quit that game.
So let's talk a little bit about that.
First impressions?
[INAUDIBLE] panel.
You have the whole fish-eye thing through the tube thing, kind of like weird pixels.
And you also have this guy going down the screen
Mhm, so there's sort of a vertical hold jittering going on.
And actually, it kind of doesn't look like it's a flat screen, right?
It looks kind of--
Yeah, I kind of said the old school kind of screen with the fish-eye kind of thing
Yeah, a little tube shape to it.
Yep, that's a good one.
What else?
That's developed by one guy, by the way
You hear him breathing the whole time.
So it kind of makes you feel claustrophobic, like you're in a small space looking at this monitor
Mhm, yep, so the fact that you can hear your own breath sort of gives you an impression of the space around you, which is simultaneously really, really small and really, really large at the same time.
You're in this tiny little thing, but there's this huge world outside of there.
What else?
Is there any music?
I didn't get what the objective was.
But it kind of sounded [INAUDIBLE] kind of space ship or something.
You're an astronaut in some kind of spaceship or something, and you're trying to search for air
Yep.
So let's say, OK, air is important.
And there's a lot of things that remind you of that, right?
You have the breathing sounds.
You have the things in the world that are sort of [INAUDIBLE] that you can pick up.
There's a little oxygen meter.
I'm not so sure if all of you saw that.
There's a little oxygen meter that's constantly running down.
And the sound of your breathing changes depending on how much oxygen you've got, or if you pick up an air pocket.
Then you can breathe.
What else?
OK, so you said space ship.
And a few people said space ship.
What else gives you the impression this might be a space ship?
Because the game never tells you that you're in a space ship
[INAUDIBLE]
Hm?
I actually thought it was underwater instead of a space ship
It could be underwater.
So what's the evidence for each?
Yeah?
So for underwater, whenever you press space bar, it kind of looked like a radar sort of pulse
Mhm, It's kind of like a sonar ping, which is something you'd expect to do underwater.
What are things that remind you of space in this game?
I would argue that-- oh, yeah
Some of the objects on the screen look like they could be stars [INAUDIBLE]
Mhm, they look like constellations, right?
There's that issue when you're trying to turn around you had to kind of stop.
And it almost seemed like [INAUDIBLE] thruster or some kind of [INAUDIBLE]
Mhm, yeah, there's a lot of inertia in changing your direction, yeah.
Apparently there is an answer on whether it's space or underwater.
I don't actually know the answer to that one.
But they're both plausible conclusions from actually nothing that the game explicitly tells you.
All of that you derive from hints, from the aesthetic.
But because the game kind of works together really, really well on the game mechanics of running out of air and the visual representation of having this old school monitor, as well as the sound of you breathing and everything that you kind of know about what it's like to be in a situation where air is important, which implies certainly not walking around on the ground somewhere.
Or if you are, you might be on a different planet, or something.
And what you know about inertia gives you this impression, this very, very deep kind of world that you can sort of make theories about and imagine where you might be without the game actually having to tell you any of that.
So that's a game where I feel it's really, really successful in getting that across.
This next one I'm just going to play a demo.
This is Defcon.
It's kind of like a real-time strategy game, only on a sort of global thermonuclear war kind of scale.
Oops.
Screen-- here we go.
Is this 1,280 by 800?
I think my screen resolution changed on me.
Yeah, when I tried to fix that problem earlier, the screen resolution changed on me.
That doesn't matter.
I'll switch it to something else.
Mostly, this is fine.
So I'm just going to play a demo.
This is just a game that you see.
This is as if the game is just playing against itself-- AI versus AI.
If anyone has seen-- how many of you have seen War Games?
OK, wow, OK.
It's Matthew Broderick, is that right?
And this is very much inspired by that.
And the idea, of course-- actually, I'm not going to tell you beyond what I just said.
I want you to start calling out things that you sort of think work aesthetically about this game.
[CLASSICAL MUSIC PLAYING] I'm going to move the camera around, too.
This is an actual game that you're watching right now being played with the AI playing against itself.
Just call out stuff that strikes you about this game
The type face is very nicely apocalyptic, like all angles and-- I don't really know how to explain it, but it feels good
It's stark, if nothing else.
There's a certain amount of brutishness about it, but also very technological.
It's not just like hewn out of rock.
It's not that kind of brutishness
It's technological, but it looks very old school.
It's very much like a radar screen
Yeah, not necessarily even year 2000, right?
This seems like a much older aesthetic, something from the '80s maybe, Cold War.
Scroll over to something a little bit closer.
No one's in America.
This is continental war.
What else?
What song is this?
Does anyone recognize the music?
Does anybody hear anything other than music?
[INAUDIBLE]
Thunder?
Is it thunder?
It's this explosion
Where do you think--
[INAUDIBLE]
It's a little [INAUDIBLE], really soft.
Where are you?
Where are you supposed to be while you're playing this game?
War room
In a war room underground, several miles underground, maybe.
It's a disturbing game.
The music-- I believe it's all classical music.
But it's heavily filtered through a lot of audio filters, giving it a sort of a reverb effect in the sort of sense that you might be hearing this inside a small enclosed space.
There's a little cassette player inside your bunker somewhere.
So besides the font, what about the UI?
What about the UI seems to be-- in the way how the units are represented?
Like those ships-- are those ships life-size?
They're not, right?
They're not the size of several cities put together.
These are some abstract representations.
These are sort of army pieces that you're moving on a board, only it's a digital board, that represent armies that are out there in the water or flying through the air, squadrons that are flying through the air somehow.
So I'm going to quit now.
So Defcon is, again, a very small team, I think less than five people.
At least they were less than five people when they made Defcon.
Now they're making other games.
It was one of the first successful games by Introversion, which is the name of the indie developer that did this.
And yeah, they explicitly stated that this is very much inspired by War Games the movie.
It's about global thermonuclear war.
It's about specifically the movie that was done in the '80s.
So it's very much about what global thermonuclear war would have made sense in the '80s.
Obviously, the graphic representation of what you see in that game is a little bit higher end than what you would normally have had in the '80s.
But they're trying to sort of not necessarily represent things realistically, but sort of how things as Hollywood might have seen it.
And even the marketing of the game says-- I believe it's, "Everybody dies," or something like that.
And you can't really win that game, right?
This is what the packaging of this game looks like.
I think if you go to [INAUDIBLE], this is what it looks like now.
Uplink is another game by them.
It's worth checking out.
It's a game about hacking.
And if you look at that, you'll see all the same sort of '80s influences.
But it's not about launching missiles at each other.
It's about breaking into bank accounts and changing your school grades and things like that.
Finally, this is a game I actually do want people to play.
So who hasn't played Aaaaa!?
All right, I want a volunteer to come down and play this game.
Come on down
Dejobaan Games and Owlchemy Labs, two lovers entwined in an awkward embrace
This is from Boston, by the way.
[ROBOTIC VOICE]
Jump?
Go for it.
What can you--
Stage-- I don't know, [INAUDIBLE]
Hit Unlock.
Oh
Oh, bye
Just try-- OK. That one?
Try it.
Yeah, OK, then hit Play.
OK, so the basic idea of this game is it's a base jumping simulator.
So use WASD to jump off the side of a building, and then just fall
Checkpoint
Try not to hit anything
Shit
But try to get as close to things as possible, because that's how you get points.
Oh, something [INAUDIBLE].
Ouch.
Is this in Endless mode?
Ah, I don't know
This seems like it is
No
Checkpoint
Sorry, controls are really light
OK, try hitting space bar.
Oh, no, no, no, not yet
Oh wait, oh no
It's cool.
It's fine.
All right, now try to land in that little circle, little red circle behind that building.
Yep
Come on
Yeah, try landing there.
You did it.
OK, thank you.
[APPLAUSE]
OK, I'm going to turn the volume down.
What's working for this game's aesthetic?
Did I see a hand back there?
Yeah, right there
We're all really used to games with cylindrical coordinates through a camera.
And this game appeared to have it be the case that as you look down, one can then rotate your camera.
It really felt like you're looking straight down.
And so we're very used to having cylindrical coordinates
Mhm, yeah.
This is a game that uses sort of WASD first person controls.
Only first of all, you're not carrying a gun and looking around like that.
You're pretty much doing this for the entire game.
So you're always kind of rotating around your center.
So that's a systemic-- essentially how the controls work
Checkpoint restart happened really, really fast when you screwed up
Yeah, yeah, so it's like, survive to this point, get to the next checkpoint.
And things are actually flying past you at a pretty high rate as well.
What else?
Music?
Art?
Storyline?
[INAUDIBLE]
I think the music-- does it start the moment your foot leaves the first building?
I think there are some levels where it does that, where the music only starts the moment you jump off the building.
It's like, you can stay on top of the building as long as you want.
But nothing happens.
And then as soon as you jump off, the music starts.
So it's like, all right, now you're doing something that you should feel excited about.
So we're going to play this exciting music at you
You can hear the air while you're falling
Yeah, there's a very strong sort of rushing sound.
And you're getting other UI sounds as well.
Those are the things that are actually giving you points, getting close to other buildings.
And so there's some visual feedback
The game menu before it starts is really confusing.
There was just way too much.
I don't think that was working at all
Yeah, the thing is I've played a bit of this game.
So it normally directs you to the first thing that you can click really clearly.
And then at some point of time, it stops leading you to a linear path.
So you've got this whole grid of different things that you can do to choose one.
So admittedly, the game UI may not achieve what it's trying to do.
But what do you think it's trying to do, even if it fails at it?
It's sort of like, here's this ridiculous grid of all of these different things that you can do.
But maybe some of them are unlocked, or some of them are still locked and some things are still locked out.
What's the point of showing you all these things that you can't do?
Try and motivate you to unlock them so that you can do them
Right, this whole idea is that-- it's the same idea in Zelda.
Sometimes you see this heart in a location which you know that you can't actually get to yet.
But the whole point of that is to make you want to get there.
And that's what the unlock system in this game, I believe, is trying to do.
It wants to give you an incentive to keep playing.
Because there's all this other content that you haven't seen yet.
But you kind of have a clue that it's out there.
At the beginning of the game, there's a little narration actually that you may not have caught.
It says, the city of Boston has outlawed base jumping, so this whole idea that what you're doing isn't actually sanctioned by the authorities.
And eventually, you get these commands later where you can flip off people while you jump off, unless they happen to be your fans.
And then you give them a thumbs up.
So there's a sense that what you're doing isn't advisable.
This is not supposed to be just how the future looks like.
Even in this weird future where you have these mega skyscrapers that go all the way into space, this is still not a good idea.
But you're going to do it anyway.
That's what this game is about.
So it's a 3D game.
This one was done in Unity, actually, although it was on port from a game that wasn't originally developed in Unity.
So you think of the very first version, which is just called AaaaaAAaaaAAAaaAAAAaAAAAA!!
!-- A Reckless Disregard for Gravity.
Is that right?
Yeah.
That game only runs on Windows.
And all the versions that have come since then have all run on multiple platforms.
Because they were developed on Unity
Including Oculus Rift
Including Oculus Rift, an Oculus Rift version of that game.
You can imagine how that would work.
Anyway, so something that I want to end on today-- there's a second presentation very quickly.
Actually, you know what?
I've been talking for an hour.
Let's take a quick break.
And we'll come back in 2:15, 10 minutes.
All right, let me see, am I on mute?
OK, so what I'm going to do now is I'm going to have your groups actually have a discussion about what you want your game mechanics to do.
And the reason for this is because I feel that a lot of you probably are decently equipped to talk about what you want your story lines and your characters or your art style or your sound design to achieve in your game.
And the reason for that is we've seen a lot of movies.
We know how we react to music.
We realize what visuals have on sort of human emotion.
But I'm not sure if you've had this discussion yet in your team about what you want your mechanics to do aesthetically.
What kind of experience do you want to generate for the player?
So really, it's what you want everything in your game to do.
But right now, I'm asking you to do a discussion about what you specifically want your games to do.
I'm going to throw some words at you to help you be able to frame these ideas inside your team.
First is that, is this a game?
How many of you feel like you're making a game that's kind of about strategically outwitting the game system, being smarter than the computer, basically, or being smarter than the game designer, that's how you win the game?
How many of you feel like your game is really about some random thing happens, and now you have to deal with it?
OK I see a couple, a few, hands out there, maybe.
And some of you don't know where your game falls in that spectrum.
Of course, it could be a bit of both, right?
A French writer named Roger Caillois came up with a categorization of what these different kinds of game play could be termed.
And he came up with Agon and Alea.
Alea, some of you, if you've ever studied Latin or Roman history or anything, might recognize it from alea lacta est, which is, the die is cast.
So that word comes from dice.
And it sort of represents games of chance.
And Agon is a little bit more about conflict.
It's about usually person versus person conflict.
But in case of games, it's just kind of like competitive contests of skill.
Basically how well can you deal with this challenge that's put in front of you?
Caillois goes on and describes this as, this is sort of ludus.
This is the ludic kind of play where you're sort of playing-- it's very specific to what games are all about.
There are other forms of play out there in the world that don't necessarily have anything to do with games.
A game isn't required to have one of these things.
But for the most part, what Caillois claims is that these kinds of actions are the things that really, really sort of set games apart from other kinds of play, this Agon and Alea.
But then there's-- what else is there in play?
There's paidia and ilinx, which are, again, words that I don't necessarily expect you to remember.
But here are words that might be a little bit easier to recollect, the sense of being someone that you are not, mimicking something that you might fantasize in being, or that you're being asked to perform.
Are you a person on the run from zombies?
Are you a military soldier?
Are you a detective?
Are you an assassin, a scoundrel, that sort of thing?
You're being asked to play that part.
Are you a town hall mayor?
Whereas vertigo, it's about the sort of physical sensation of playing the game.
And when it comes to digital games, usually that's one of the things that digital games have the biggest problem.
Unless you're doing something like Dance Dance Revolution, where you're actually having to play the game as a full body or a Kinect thing.
But even games like Aaaaa!, we were describing a game that's very much trying to give you this sense of physical motion.
So they're trying to sort of conjure up this sort of bodily freedom, even if that freedom necessarily means a loss of control, to sort of give you a pleasurable experience.
And you get that in other kinds of play.
How many of you played that game where you go around a baseball bat with your head down, and then you have to walk?
No one does this?
That's my favorite game
[INAUDIBLE]
Yeah, you get yourself as dizzy as you can.
Sometimes you look up and you spin.
Sometimes you look down and you spin.
And then you try to make your way across to the other side of the finish line.
And that's like literal vertigo, right?
So we've got these different kinds of sensations that this one series has already described, and probably not everything that games can possibly do.
But they're kind of, in one way of describing it, kinds of fun.
There is the sort of competitive test of skill.
There is the, does fortune favor you today?
Are you feeling lucky right now?
Do you want to press your luck?
Do you get to be somebody else and live through that experience?
Or do you just get to be in a sort of physical situation that you haven't been?
These are all different things that your game mechanics can achieve.
What I'd like to describe is like the stuff on the right is the stuff that sort of puts you into the moment.
This is where you're being asked to spontaneously react.
Can you think like a mayor of a city, or president, or monarch?
Can you react in a way as if buildings were flying towards you at 200 miles per hour?
Whereas everything on the right is what we typically described as the structure of the games.
You've been experimenting with this in your prototypes since day one.
But there's also kind of the differences between the decisions that you get to make and the things that kind of just happen to you, and you get to react.
So I like to describe the stuff at the bottom as kind of fate.
This is the fate that you have.
Buildings are going to fly into your face.
And you kind of need to be able to then make decisions to be able to just deal with that reality.
Or fate being, well, the dice is going to show what it is.
But the test of skill is the part where you get to decide what you're going to do given that you know the odds and that you know what kind of actions that you can take.
So right now, in your teams, what I'd like you to do is actually have a discussion about given your prototypes that you've developed so far-- I don't think we're doing a test today.
We're doing tests on Monday, right?
So what do you want your prototype on Monday to try to achieve?
It could be any combination of this.
It could be all of it.
But I would like you to have that discussion in your team, just maybe 15 minutes?
Yeah, that's plenty of time.
And then what we're going to do is just do a very quick report back about where you think your team-- just one person from each team will come down and tell us, what are the things that you think you want your game mechanics to achieve in your prototype on Monday, OK?
15 minutes.
That will get us to basically about 2:40.
All right, so go right ahead.
I'll just leave that up.
[BACKGROUND CHATTER]
So our game, as it is right now, you're basically playing like a child or an animal.
We haven't decided.
And you're basically going to go around town figuring out this mystery.
So you're kind of playing like a detective type story.
So based on the slide, ours is more like structure and decision.
So it's going to be sort of a competition against the computer where you're trying to figure out this mystery of what's causing cholera, or what's causing people to be sick, actually.
You won't know that it's cholera.
So yeah, that's basically where our game stands
OK, all right, cool.
Thank you.
A very analytical detective game that you're working towards-- that's for the prototype and tests on Monday.
Who's next?
Come on up
Hi, we're Snap.
So we're planning on making a very competition heavy game.
Because you're going to be playing with a bunch of other people.
So for Monday, we want to have that competition structure in place so that we can see how it works.
We're also adding a little bit more fiction to Snap, which is very abstract.
And so we're going to later on test how that works out.
But for now, it's just really going to be the basic competition of the game that you've played
OK, all right, thank you
So for Heatwave, we're mainly going to focus on originally decision, deciding who you're going to help and how long you're going to try and convince them to drink some water or go inside.
And there's also going to be a small aspect of fate.
Because you can spend 10 turns trying to convince a person to go inside.
And at the end of the 10 turns, they say, no, I don't want to talk to you anyone.
I'm not going inside.
So decision and fate
OK, cool, thank you
Our team is working on forecast based financing.
So because we're about planning for disasters, our game is pretty heavy on chance.
But what we want to teach the players to do is to use planning and forecasting in order to reduce the effects of that chance and sort of gain valuable skills in mitigating that
All right.
So it's very much on the right side of this screen
Hi, so we're the other cholera group.
And we decided to create a simulation based game where there are many villages, and they have their own water source.
And we have cholera as some sort of entity that can sort of spread from village to water to other villages.
And pretty much every village has their own population.
They have their own infection rate.
And what you are, you are pretty much some sort of person who really has a lot of power and wants to fix this cholera outbreak.
So this game focuses a lot on decision making, because you have many different ways to try to fix this problem-- either short term using simple purification systems to launch for waste management.
But this game also has a lot of chance.
Because obviously, these things aren't 100% efficient.
There's going to be some failures, going to be some breakdowns.
What we want to focus on right now is to make sure that we can get our villages to have their own sort of parameters, make sure that the cholera-- we have some way of representing cholera and how they spread to all the other villages
Cool, thank you.
Is that all the teams, or was there one more?
That's pretty much it.
OK, just a quick thing, so a couple of thoughts-- one thing that you should be trying to figure out as you work on your prototypes is to what extent the decision making, which came up with a lot of the different groups, going to be reacting to things that you hadn't anticipated, versus crafting a long term plan.
The stuff that you deal with on the short term is going to go very heavy on this side.
And it's going to play a very big part in making someone feel like the kind of person that you want to feel, whether you're a big decision maker or an individual on the ground.
The long term plan part of it is really more game-y.
You're really thinking more as someone who's playing the game, rather than someone who's actually living the game.
And you can also decide that's more what you want people to be going for, especially games that have to deal a lot with planning for chance.
Now, just because you have random numbers in your game, and probability rates and everything, doesn't necessarily mean that your game is all about chance, even though you have these things in your system.
Because if what chance in your game really means is that there is a certain rate at which certain things are going to happen, then really that's just a rate.
That's something that you can plan for.
You can just expect every 10 turns, you're going to get three outbreaks.
That's how the numbers are going to break down, even though you don't know exactly when those rates are going to be.
And if your game is where every single one of those times where chance hits is really catastrophically or really, really beneficial, then you've got to get more of a game of chance.
If it's the sort of thing like a certain number of these things are going to happen at a given rate, and you just have to plan for that rate, that is, to me, not really chance anymore.
It's more about that skill based test of whether you can plan well.
So again, always keep in mind, what do you want your game to feel like, while you're trying to figure out whether your simulation is right, whether your facts are right, whether this is actually what happens with the diseases or the catastrophes that you're dealing with.
Also keep in mind this is what you want your game to feel like.
Make sure that's also informing how you're designing your game mechanics and your art style and your story line and your characters and so forth.
That's what I've got to say
So administratively for Monday, we have a lecture in the morning about games for learning.
Then we're going to do play tests.
And then at the end of play test, we're going to do the two minute presentations, so that when you're play testing, even though we heard a little bit about what your games are, we're not completely-- we'll still be a little naive when you're playing your games.
We'll know exactly what's going on within the games.
In that two minute presentation, we want to know a lot about process.
So let us know how you got to the point where you are now and where you think you're going.
I believe product backlogs are due at that point, too.
So use your product backlog as part of your telling us what you're planning on doing on your projects.
I know a few of you have come to me and asked me about certain aspects of your projects that might not be answered by the resource materials that we've given you.
So hopefully you've read through the resources, and you're finding some of your answers there.
If there are any questions you have that are not being answered by your research, email that to videogame-bosses, and we'll forward that along to the clients.
I'm working to get them to allow me to just give you their email address so you can contact them directly.
I'm still figuring out who exactly for each of those projects that's going to be.
But that should be solidified by Monday.
But yeah, things to look out for that might not be answered in your resources are cultural sensitivity, particularly because of who your target audience might be.
So if you're doing anything graphical, if you're doing anything representational, really think hard about who it is is playing the game, what kind of cultural background they have, and how are you representing it to them.
Are you doing it both authentically and also not patronizing them?
It's going to be a really hard challenge for you.
We don't expect you to actually solve it this semester.
We expect you to come up with some ways to figure out how you might solve it in the future.
And that's kind of what we want to hear about in some of these presentations.
Or how are you solving those kind of problems?
Any questions before we let you go and let you work in your teams?
Can you give us an idea of how much time we'll have in class to work with our teams so we can better plan?
Hour and a half.
We went a little bit long today, about 20 minutes long
That's all my fault.
I'm sorry about that
Blame him, but hour and a half basically.
If there's a guest lecture, those tend to be an hour long.
And then that would be two hours long.
If it's not a guest lecture, we tend to ramble, so hour and a half.
Also part of that is because we start class at 1:10.
That kills some of the time.
So if you want to come into class at 1:00 and be in your teams and work in your teams, and then we yell at you to shut up, that's fine.
That's actually good.
You're using your time well
Yeah, you come in at 1:00, and then do a daily scrub, and then you are ready to start class
Any other questions?
All right, so I'm putting these materials up here.
And also a reminder, the play test, if we weren't really specific about it, we are expecting low fidelity prototypes.
But it could be digital or paper.
It's really whatever is useful for you, that's what we want to see tested.
[BACKGROUND CHATTER]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So today in class, we've got a guest lecturer.
Scott Osterweil from the MIT Education Arcade at MIT Game Lab's going to be talking to us about learning and play in games to put us in the mindset for the games that you're making that feel like they might be educational or more about awareness, to take some ideas from him on that.
Then after that, we're going to do a play test.
So this is your first opportunity to give staff and clients, if our clients show up, the experience of playing your games and giving you some feedback on the low-fidelity prototypes that you have right now.
And again, they could be digital or non digital.
That's going to be pretty quick today.
We don't need everybody to play everybody else's games, if you don't like.
So what we're going have you do is set up your games.
I expect if they're paper, you probably only have one copy of your game running at any one time.
So just make sure that everyone who's on the team who is not playing is observing and taking notes.
If you do have digital and would like to set up multiple stations, please do.
It's always helpful.
Our next play test is November 5.
We're having the class from 21W032, the Introduction to Digital Media class taught by Ed Barrett.
They'll be coming in at the end of the day at about 3:00 PM to test your digital games.
So on November 5, it's a good opportunity for a first playable version of your digital game to get tested by people who are not in the class.
And then after play test today, we'll take a quick break.
And then we'll do presentations where each team will come up here, and you've basically got the floor for two minutes.
We want to hear what is the state your game right now, basically referring back to your product backlog.
What are the features that are planned?
What did you test today?
What does your build look like today, your low-fidelity prototype look like today?
And just let us know how it's going.
We're going to ask for a number of these short two-minute presentations throughout the rest of the semester.
And those dates are in the handout for project four.
And I'll be making mention of them as we go along.
And then the remainder of class, you've got about hour to an hour and a half at the end of class to work in class.
I'm going to let you know how much time you're going to have time for working in class for future days.
This Wednesday, you'll probably have about two full hours in class to work, looking at what our lecture schedule looks like.
So that's that.
Any questions about what we're doing today?
Any questions about project four?
OK.
I'm going to hand it off to Scott
[INAUDIBLE], this is the [INAUDIBLE] switch you were talking about?
Yep, I'm switching it over right now
Well, it's nice to see you all.
I see at least two faces from-- three from classes that I taught, so forgive me for those of you who may have heard some of this before.
So this was sort of teed up as if I'm going to talk to you about education, but I'm actually not going to talk to you about education.
I'm going to talk to you about something else entirely.
Maybe we'll get around to education by the time we're done.
But what I want you to do right now-- hang on one second-- is turn to the person next to you and play a quick round of tic-tac-toe.
So I do like doing this, even though it's hard to get people to stop.
It's actually kind of interesting to me.
When I've done this with an audience of about 300 people, it takes a long time to corral everybody back, which is really remarkable.
If you weren't playing a game, I hope you were at least looking around and noticing that the room got kind of loud and there was a fair amount of laughter, and noise, and animation, which is really remarkable when you consider the fact that tic-tac-toe is a stupid game.
That by age eight, you had figured out that there was no point in playing tic-tac-toe because you almost always play it to a draw.
I'm always amazed when I ask a bunch of adults, which you are, to play tic-tac-toe, how into it they get.
And I think in the end, it's the thing I really want to talk about, which is play.
I know you guys have been in a class studying games.
Have you talked much about the word play?
Good.
We are all in the business of making games, and yet we don't stop much to think about play.
But play is not something that was invented with the Atari.
People have been playing games for a long time.
The oldest known game implements are older than the oldest known writing.
I don't know how many of you know art history, but this is the late 16th century in what's now Belgium, the low countries-- Flanders in those days.
The great genre painter Pieter Bruegel, who did a painting called Children's Games, in which he documented over 100 games that scholars have been able to identify as real games that were played at that time.
And if you think about it, at that point there wasn't really much of a thing called childhood.
That by the age of 11 or 12, you were probably working in the family business, whatever it was.
You were expected to take on adult responsibilities.
And yet clearly, play was still a huge part of their lives.
Now, this is a fanciful painting.
He was not trying to be realistic here.
But the point was play was a huge part of their lives then.
So we know that they were playing back in the 16th century.
We know they were playing 6,000 years ago.
We actually know that our ancestors played too, that other vertebrates play.
In fact, Edwin Wilson has sort of argued that ants play too, but let's just stick to vertebrates for a minute.
When mountain goats play-- there's a alpine mountain goat-- they play by-- and by the way, let me just say quickly from my definition, play is the stuff you do when you don't have to do something else.
You don't have to get food.
You don't have to evade capture, or protect your young, or procreate, or find shelter.
When you don't have to do all that stuff-- some of which you do playfully, by the way-- but when you don't have to do that stuff and you're on your own, you play.
And so mountain goats play by actually chasing each other around the mountains and jumping from cliff to cliff, ledge to ledge.
And they do it in spite of the fact that mountain goats will occasionally fall to their deaths.
And if we know anything about evolution, we know the behaviors that lead to the deaths of individuals are behaviors that are more likely to die out, unless there is some advantage to the behavior that outweighs the risk.
And it would easy to assume from this that mountain goats are learning how to do the things they need to do, like jump and land on precarious ledges.
That that's what they're learning through their play.
We might extend that thinking when we look at an image like this.
By the way, you can find images like this on the internet.
Anyway, so we've all seen images of cats or puppies fighting, playing at games that look like fighting or hunting.
And if we take that example and the example of the mountain goats, it's easy to conclude that play is about rote memory.
It's about learning to do things through repetition.
But they've done a study where they prevented kittens from the opportunity to play while they grow up, play with other cats.
And it turns out they learn how to hunt just fine.
What they don't know how to do is how to interact with other cats or make other more complex decisions.
What I'm going to argue is play is about something far more involved that just sort of rote memory and repetition.
Last animal example.
They did a study where they gave an otter a fish every time it swam through a hoop.
Now, if you think about what an otter's needs are, that otter was rich beyond its wildest dreams.
Everything it needed was right there just by swimming through the hoop.
So did the otter retire as we might do if all our needs were taken care of?
No, it started swimming through the hoop upside down, backwards.
Playing with a hoop with its way of exploring how the world worked.
The otters-- I assume there's more than one-- the otters knew that swimming through a hoop got fish.
They wanted to find out what else it could do.
And again, this is when survival is no longer an issue.
So play is really the way in which we explore the world.
Just looking at these images of children, there are four different continents here, four very different kinds of games.
But I'd argue the affect is the same in all of them, and that what's going on in all of them is the same thing, that the kids in these pictures are really constructing their understanding of the world through play.
What I want to argue is that through play, we begin to build the kinds of conceptual structures that we are going to then engage with more formally in other spheres of life.
And I could argue that it's only for children, but I'm going to argue further that it doesn't stop in childhood.
But sticking with childhood for just a moment and using my own personal example.
I loved playing with blocks when I was a kid.
And this is all pre-kindergarten.
I can remember the pleasure of discovering that two square blocks were the same size as one rectangular block, and two rectangular blocks were the same size as one big block.
And the kicker was that that was equal to four of the little square blocks, and that those relationships could then be replicated elsewhere.
That that was a pattern that I could find elsewhere in the world.
And so pre-kindergarten now, what I am really doing is developing a primitive, but I would say robust sense that math is the way we actually model the world.
That we can actually model the world mathematically.
Now obviously, as four-year-old, me couldn't have said that.
Four-year-old me couldn't even necessarily answer 2 plus 2 equals.
But four-year-old me knew something far more robust.
I was starting to really understand that the world could be described mathematically, and that that was why I was primed to be a relatively good math student.
The challenge, though, is that I just played with blocks the way I chose to play with them.
No one made me.
In fact, if you'd sat me down and say play with blocks and discover what you can about numbers, I might have refused and age four.
Players' motivations are entirely intrinsic and personal in all play.
You cannot make somebody play.
I think it's helpful to think about it as if there were four freedoms present in play.
Freedom to experiment.
I thought I had revised the slides.
I really sort of revised that.
I think it's better say freedom to explore is the better word.
But what I was doing with those blocks was seeing things about the blocks that were not-- the package showed pictures of things you could build with the blocks, but that's not what I was doing.
I was sort of exploring properties of the blocks that no one was telling me to look for.
Freedom to fail.
If you played with blocks, you probably built a tower that eventually fell down, and you probably learned a lot from the tower falling down.
You probably, in fact, persisted at trying to get the tower to stand up.
And along the way, you were learning all sorts of stuff about mechanics and physics.
Freedom of identity.
If you think about doll play for a minute, a kid in the floor of their room acting out conflict between two dolls, or stuffed animals, or action figures is really exploring all the roles in their own family, in their world.
A kid who plays at Luke Skywalker versus Darth Vader is really figuring out what part of him or herself is Luke Skywalker and what part is Darth Vader, because we think we have both of them in us.
And that's what we explore through play.
In less than a week, on Friday, a fair number of you are going to engage in identity play at a fairly large scale, so it's not just a childhood thing.
I mean, I'm talking about Halloween, obviously.
And anyone who's ever played World of Warcraft or any number of games knows that in fact, through games, we play with our identity over time.
Finally, freedom of effort is the freedom to really play hard or play relaxed.
You cannot make somebody play hard.
And if you watched the pattern of play, people will play intensely.
They will suddenly ease up.
Again, stick to World of Warcraft for a minute, sometimes you want to grind.
You just want to do the mindless stuff for a while.
And sometimes you want to enter into an intense battle.
They both happen.
So here's the challenge for us.
The player's motivations are entirely intrinsic and personal, as I've already said.
And I'm also arguing that obviously, learning is happening through play.
But how do we channel play into learning while still allowing for its fundamentally open-ended nature?
So if you, as Rick said, are going to be making games in which you want to-- well, I hope you're thinking about real learning rather than just conveying information.
And I'll get to that in a minute.
But whatever it is you're doing, theoretically you want to make a good game, they still have to have the freedom to play.
You don't have a game without play.
You can have something that has the structure of a game without play, but it won't really be a game if they're not playing.
But anyway, so that's where games come in.
That's how we can channel learning sometimes while still being open ended.
And just to use the most absurd example, think about golf.
It was Bernard Suits who originally used this example.
But in golf, you're hitting a very small ball with a very long stick.
Anyone golf here?
Yeah, that's what the normal percentage is in a class.
But it's hard, right ?
The first time you swung the golf club, you might have missed the ball entirely, right?
Yeah.
Sometimes even when you start connecting with the ball, it goes the wrong direction.
It goes 10 feet, 20 feet.
When you can finally hit it some distance, it goes into the water, it goes into the woods.
It costs $2.50 every time you lose a ball.
If your goal, after all, is to get the ball in the hole, why don't you just pick it up, and walk to the other end of course, and drop it in?
The golf game would go much quicker.
You'd never lose a ball.
You have a lot more success.
But no one chooses to play golf that way.
If you think about it, people choose to play golf by moving the ball to the hole in the single stupidest way possible.
And as Bernard Suits said in this context, games are really about overcoming unnecessary obstacles.
And unnecessary is critical here because every game is, by definition, unnecessary.
If you're really playing, it's voluntary.
It's not required.
Your survival does not depend on it.
So by definition, every game is unnecessary, and therefore the obstacles in every game are unnecessary.
And the game is the voluntary overcoming of unnecessary obstacles.
So why do we do it?
Why would we do all that?
Well, in games we willingly submit to arbitrary rules and structures in pursuit of mastery because that's what's going on in golf.
Even in golf.
You think it's hard, and yet even though you missed the golf ball the first time, you think, I think I can hit it.
And then eventually you do hit it and you say, I think I can hit it further and I think I can hit it straighter.
And in fact, the game gives you feedback.
The game is continually giving you feedback as to how you're doing.
No matter how ridiculously hard golf is, you set for yourself proximal goals.
I'm going to hit a little straighter, a little further.
And the game lets you do that.
No one runs out into the golf course and says, stop, you didn't hit the ball far enough, or yells at you and says, hit it further, further, right?
They let you playfully explore what you can do with that golf ball.
And you keep saying I'm getting better, and so you keep playing golf.
And that's true with every game.
And I'll talk about a couple other examples going forward about that.
So games give you proximal goals which seem worth achieving, but only if you can continue to be playful.
And that's I think the thing we sometimes lose sight of when we're making games is the playfulness.
We remember the goal.
We remember that there's an outcome that we want.
And we remember that we want the player to get to that outcome.
But we forget about playfulness, which means we either make a game that's too easy.
We lead them right to the outcome.
That's like picking up the ball , and walking to the other end, and dropping it in the hole.
And a lot of games do that.
Or we just figure I'm going to make it really hard.
I don't care whether they enjoy themselves.
They're going to get there.
And of course they don't.
They quit.
It's one thing to define a challenge.
That's important.
The real art in it is defining a proximal challenge, one that people can reach.
And I would argue that if you're talking about games in which you want to convey information or you want people to learn something, all of that has to hold true.
And in fact, the other thing I'm arguing, obviously, is that at in every game, people are learning.
That the reason you like playing golf is because you're learning.
And Raph Koster sets this out really well in the Theory of Fun, the book A Theory of Fun.
But that basically, the fundamental pleasure of gameplay is learning, is learning to master the game, which means in a sense, if you're doing a game and you've got some goal for some learning to happen, all you've got to do is make that learning interesting and worth achieving by giving people the right set of goals to work toward it.
So I keep talking kind of interchangeably between play and learning.
And yet the four freedoms of play, which I'm arguing are the four freedoms of learning, are not the four freedoms of school.
If you think about school, and I'm not talking about MIT right now, if you think about your own high school experience-- high school is particularly bad-- what kind of freedom is there?
Freedom to fail?
Not so much.
Freedom to explore?
Well, I mean, even a high school science lab, everyone is expecting to get the same results by following the exact same procedure, right?
And that's the most experimental you ever get in school.
Certainly no freedom of effort or freedom of identity.
You sit in your same seat every day and you're expected to behave the exact same way.
And you're expected to work equally hard.
You can't come in and say I don't feel like working today.
So there's very little play in school, at least as it's currently embodied.
And this is why I mention school here, is because I think one of the challenges if you're an MIT student is that you were probably pretty good at the game of school.
You probably did what was required of you.
You probably didn't necessarily recognize that doing the things that school required of you is not when you were actually learning, that that was just playing the game.
That the learning was more self-motivated and more self-directed.
And so frequently people, when they turn around and try to make anything to do with learning, whether it's a game, or write a book, or create a curriculum, they replicate everything that's bad about school.
And the reason I don't like to say I make educational games is because when you think of educational games, you think of all the bad games that have replicated what's bad about school.
Games that have largely taken the dead carcass of a game and stuffed it full of academic content.
So you're going to shoot at aliens and you're going to memorize your times table.
Now, what aliens have to do with times tables, I don't know.
Any I'd even argue that memorizing your times table is of questionable value.
There may be a place for it, but it's certainly not what being good at math is about, fundamentally.
And too often, games for learning end up being about simply I'm going to feed you content that would be boring in a lecture or boring in a textbook.
And guess what?
It's going to be just as boring in a game.
The only difference is we're going to surround it with things that we think you think are fun, like shooting at aliens.
So it sort of translates into people thinking that what the world needs is something like Grand Theft Calculus.
But in fact, without playfulness, a game is just going through the motions.
It's just gym class.
Volleyball in gym is not the same as volleyball at the beach, and there's a reason for that.
And even smart MIT kids making games when they think there's learning involved end up reverting to gym class, to just I'm going to make you play this game to learn this stuff you don't want to learn.
Just to talk about the difference between a good learning game and a bad learning game, let's talk about difference between spelling bee and Scrabble.
In a spelling bee, most of us, when we do a spelling bee, are nervous.
Our palms are sweating.
We think we're going to fail.
Eventually the moment comes where they say no, you're wrong.
Sit down.
You're probably relieved.
When they say, you're wrong, sit down, nobody says to you, well, that was interesting that you spelled it that way, or I can understand why you might have chosen to spell it that way, because it rhymes with-- no.
They just say you're wrong, sit down.
And that's the end of it.
The end of a spelling bee, one person feels good, the winner.
Maybe the kid who comes in second feels OK. Everyone else is relieved because they don't have to do spelling anymore.
And I first coined this analogy thinking about bad games.
But I later came to realize that it's bigger than that, because if you think about-- so I don't know how many of you know this, but 7% of the population in the US graduates high school saying they're good at math, which means we take an hour a day for 12 years teaching 93% of the population that they're not good at math.
We would be doing them all a big favor by in kindergarten, saying you're not going to do math and just leave it at that.
Or better yet, we could figure out ways that teaching method that actually were meaningful and relevant to people, rather than making them feel like they're not good at math.
So that's a bad game.
And so what I'm really arguing is that school is a bad game.
School is a game in which we reward people for learning how to play at school.
Sometimes they're smart at some things.
I'm not saying that there aren't people.
But largely our goal is to weed people out.
And we filter some people into some fields because they seem good at it.
For everyone else, we're sort of convincing them that they-- whew, I don't have to study anymore.
I'm done with school.
I never have to learn anything again.
That's the way most people end up leaving school.
Scrabble, you sit down.
You got your board.
You got your tiles.
You're moving around all the time.
You're being creative even in the downtime just thinking about all the words you know.
If you never win a game of Scrabble, you have all sorts of other proximal goals, like getting a 50-point word, or getting a triple word score, or getting the highest score you ever got.
Just like golf, it gives you lots of feedback.
By the way, most people who play golf, they're not in a tournament.
They're not playing to win a game.
They made their own game.
Maybe I'm going to get a lower score than I got last time, or maybe I'm going to get at a lower score than the person I'm playing with.
Well, at Scrabble, it's the same thing.
So you're continually setting your own goals within the game.
The game has goals.
It has something called victory.
And we all may aspire to that victory, but we have lots of other ways of measuring ourselves that are not about victory.
And every player makes up their own game within Scrabble, and golf, and any good game.
We actually all play a different game when we play a game, and that's not a fault to the game.
So one last thing I sort of want to bring into the conversation as we think about this is an expression.
It dates back to around the same time as the Bruegel painting.
But in English, we first see the expression all work and no play makes Jack a dull boy.
And at first, your first response is that's a good thing, right?
Yes, play is important.
So it seems to be a statement in support of what I've been saying.
But the only trouble with it is that it also sort of suggests that there is this dichotomy.
There's work and there's play.
And I may have sort of suggested it by saying it's the thing you do when you don't have to do anything else.
But then I did modify that by saying that you do some of these other things playfully as well.
And in fact, when we go into school, we think that there's learning and there's play and they're different.
But in fact, I'm going to go quickly through those slides and just go up to here and just say I think we really need to think of it more like this.
And we need to think about situating things here.
And fun, by the way.
Let me just be clear about fun.
Fun is not giggles.
Fun doesn't even necessarily require a smile on the face.
If you think about yourself playing a video game at age 11 or 12, there were probably moments where your tongue was sticking out, and you're cross eyed, and you're yelling at the screen, that's not fair, and you threw your controller.
And then you beat the game and said, that was fun.
And Seymour Papert a retired professor from the Media Lab, is the one who coined the term hard fun.
And hard fun is not a special category.
I would argue that lots of fun is hard fun, maybe most fun.
Certainly most games are hard fun, and that most good learning is hard fun.
The things you learn that aren't fun, yes, there may be some value to memorizing your times tables, but it doesn't make you a mathematician.
That's not where the real learning happened.
There's things you have to memorize-- although I heard somewhere that a 10th grade biology student has to memorize more words than a 10th grade French student.
And if you think about it, how many of us, other than those of us who go to biology, ever use all those words that we memorized about?
None of it.
So, I mean, I think too much of our vision of learning is still based on memorizing stuff that you then get tested on, rather than building up cognitive structures that you can then work with through the rest of your life, which is what we really want people to do.
That's my point, because I know you're doing this work with the Red Cross.
You have information you want to convey.
I want to argue that if the information you want to convey fits on a 3 by 5 card and people can carry it around with them, then there's no point in making a game of it.
So just to do a parallel example, I've done games for the working poor, games where the goal is to explain to them that they shouldn't take out payday loans or take on credit card debt.
Well, in fact, you could write that on a 3 by 5 card.
If I handed it to most people and said don't take out a payday loan, don't take out credit card debt, they would nod, and smile, and agree.
And then they would go and do it when they were in extremis, anyway.
They don't do it blithely, but they do it.
But my point is that the understanding that you really need is more subtle and more complicated.
You really want to give people some functional understanding of something.
So when we did that game, what we really tried to do is put people in the situations in which they might otherwise get loans and help them see what the alternatives were.
That that was the important learning, and helping them recognize those moments.
The learning was sort of helping them, and helping them feel empowered to be able to make decisions.
So we were doing lots of stuff beyond conveying the information don't take out of credit card debt.
And so similarly, I think you're doing games where you probably think you want to convey information, but that's all information that could fit on a pamphlet, or on a 3 by 5 card, and you probably really don't want to make a game out of that.
You probably really want to make a game that's going to be about helping people through some experience master something.
And through their sense of mastery, change them.
So I think that's it in a nutshell, what I want to say, and just use the rest of the time for questions.
And then you're to see your paper projects with everybody else.
Any questions?
Yes
So a lot of your initial description, it actually sounded like grad school
Yeah?
So I feel like grad school is quite the opposite of what you're saying school is
Yeah?
Say a little more, because not everyone knows.
So [INAUDIBLE] and fail, because [INAUDIBLE] why did you go to grad school?
Because you can explore things.
You can fail.
You can do a project that doesn't work, that nobody's really going to get mad at you.
You just do the next paper.
Afterwards, you can try as hard as you want.
And there are days when you don't feel like [INAUDIBLE] too much.
I'm not sure about identity
Right.
Well, a lot of people enter grad school thinking they're going to do one thing and end up leaving doing something else.
They have that freedom to.
I mean actually, that's true of undergraduate too.
I would say undergraduate, and certainly at MIT, it seems to me slightly more playful.
Well, I don't know.
I'm not familiar.
I don't know what it's like the freshman year, when you're doing all those psets.
I don't know what that's like.
I was a theater major at a different school, so I don't know what that's like.
But I do see, and particularly in upperclassmen, a fair amount of play in your work.
But I think it is true.
What I'm really arguing at core is that what real education is about is learning how to learn, is learning all the kinds of things you need to do to know how to learn.
And some of it means just having your natural curiosity positively reinforced instead of negatively reinforced.
Kids are naturally curious.
They don't have any trouble asking questions.
We slowly start doing things that make people stop asking questions.
We killed the curiosity in most people.
And so I'm really arguing that education should be about maximizing your curiosity so that you continue to ask interesting questions for the rest of your life.
So my example of math, I don't see why there's no reason why everyone who graduates high school couldn't, when they then hear a politician quote a statistic, say how does he know that?
Where did that statistic come from?
And when you read a survey, say that question doesn't seem like a good survey, or that's correlation, not causation.
Those are all things we can learn in high school math, for example.
So it's much more about learning how to think than it is about-- and for statistics, it's far more important to know how to ask those questions than it is for everybody to know what the r value is of something.
I think that's a term in the statistics, isn't it?
Yeah.
I haven't taken statistics.
And the reason I'm going back to sort of trying to talk to you about what education should be about is because if you do games for learning, particularly because they're games, get you've got to shake the false model of learning is and go for the truly playful model-- because otherwise, it'll be the same kind of boring educational game that probably made you look askance at having to listen to me at all in the first place, because that's what you thought I was going to be talking about.
Yes
What's your opinion on games that by structure makes it easy or hard for the players who play with a lot of effort, like hard work, or play without a lot of effort?
You can do a game that's all-- requires nothing but 100% effort.
And in a sense, the play is who decides to play that game.
The people who are willing to put themselves through 100% effort all the time, they voluntarily do that.
And so you've found your audience.
I'm not saying you couldn't do a game like that.
I'm saying that it's likely that in the course of most people's gameplay, they're going to have this need to-- a game where you get on the rails and the clock is going, and you guys [INAUDIBLE].
That's why there's usually plateaus of some kind or another.
There's an acknowledgement that people need to stop and take a breath.
Even if it's just a plateau, a savepoint, there's some acknowledgement of that.
I think games where there's actually some combination of really intense play and more relaxed play can sometimes be more satisfying.
So I'm not saying that a game can't be all hard-- or the other side of it, Farmville clearly was very popular with lots of people who really wanted to do kind of mindless stuff.
I guess we all have the need sometimes for non-taxing-- so there's nothing wrong with a game which requires almost all in or very little in.
It's just that the reality is, the player is going to move fluidly from one state to the other.
And if your game can accommodate that, so much the better
[INAUDIBLE], there's something that I want to comment on [?
that.
?]
It is totally possible to play Farmville extremely in hard fun, isn't it?
Yeah
I had a interesting [INAUDIBLE] just playing [?
for them ?]
and that required very, very precise timing [INAUDIBLE].
So it is possible [INAUDIBLE]
Yeah.
And the point is that Phillip chose to make it that kind of game.
And everybody will choose to make it that kind of-- too many games, I think we freely make the mistake of imagining a certain path for a player on the win state, and we design the game around that player, following the path to the win state.
And we forget to think about all of the time that players are likely to spend either trying to break the game, or play in different modes, or in fail state.
I mean, one of the things I encourage students to do is make sure that the failure is the most interesting part of your game, because if your game is at all challenging, people are going to be spending a lot of time in failure mode.
A, you want to reward them and say that's OK. We're happy to see you here.
And b, you want to make sure that it remains entertaining while they're in failure mode so they don't quit.
Like I said, I think it's an easy mistake to fixate on what's the path to success look like, and not think about what the whole gameplay experience is like.
And that gets worse when people have an agenda, like a game for the Red Cross.
We really fall into that trap.
And that's why so many serious games seem so serious, and earnest, and humorless, because all the designer thought about was the player earnestly achieving the goals that the designer set out for the player, rather than thinking about the player playing
[INAUDIBLE]
I guess this goes back to your example about golf [INAUDIBLE] and not having somebody yell at you for not hitting the ball straight.
How do you think that kind of goes with the existence of golf teachers, who are paid to essentially tell you you're doing it wrong?
[INAUDIBLE]
I guess I'm thinking of my own dad and [INAUDIBLE] my sister, who have very different opinions about a golf coach telling them to do it right
So the relevant thing there is that they elected to have a golf coach tell them to do that at a certain point in their-- if you started somebody with a golf coach yelling at them-- now, it could be very gentle, loving, helpful person, in which case it might go fine.
But chances are if you started somebody with someone standing over their shoulder telling them what to do every moment, they probably would never develop a real interest in it.
So the point which you elect to have somebody there, you have your reasons now.
You have your motivation.
And so that's a different experience at that point.
I think lots of kids who get turned off to musical instruments or sports because too early in the experience, they're forced into sort of just reproduce the results that some adult wants you to reproduce, rather than explore this and figure out where your motivation is.
Well, thanks.
But I'm sticking around, so if you have any other questions, I'm happy to take them
Another reason I wanted to ask you to come and talk to class.
So we mentioned a couple of the other game classes we have at MIT that we're teaching.
And you teach 615, the Games for Social Change?
Games for Social Change, yeah
Next fall, right?
Yeah, that'll be next fall
Can you say a little bit about what that class entails?
Yeah.
And Sabrina took it.
It's sort of taking the same principles that I was talking about and using them to think about if you're interested in social change and how you could use for that, understanding that you can't make people change.
And so the question is, how can you use play to actually encourage change?
And so probably we're looking critically at how society works.
And the task we're trying this year for the first time-- well, you're doing two big projects this year.
We're just now finishing a project on the theme of walls to go in conjunction with the 25th anniversary of the Berlin Wall coming down.
And that's going to be on display at the Goethe Institute, which is a place in Boston, and it may also be on display in Munich at the same time, the games.
And then we're going do a project where we try to actually look at some system in society, like school, which is a bad game, and try to redesign it as a good game.
Not make a game about school, but rather redesign school itself as if it were a game.
Every year we try different stuff.
The one thing I like to try to do is come up with projects that actually have-- for which you're doing it with somebody besides just me.
I don't just want students doing stuff for me.
I want them doing it for some bigger audience, and we try to do that in every game, every semester [INAUDIBLE].
And that's [INAUDIBLE]
Cool.
Yeah.
So if you really like what you're doing right now with the Red Cross/Red Crescent project, and you want do more of this kind of stuff, take a look at that in the fall.
And if you're looking for some help or insight about the design of your project this semester, the Games for Social Change materials are on the OpenCourseWare website.
And we'll post those readings and materials there so you can take a look at that
It's a bit of a seminar, so I think there's no lectures.
And so I think there's a lot--
The readings
Yeah.
And they're not always explicitly about what we end up talking about in class.
So I don't promise that they'll always be helpful.
But anyway, they're more like thought starters than [INAUDIBLE]
All right.
So it's 1:48.
Take a really quick break.
And then set yourself up into your five groups, and set up your play test.
We'd like to start doing play test at 2:00.
So 12 minutes from now, if that's possible.
Do you think it's enough time?
So we're about to get started.
How many stations do you have for testers?
[INAUDIBLE] many [INAUDIBLE]
How many are you going to have?
Three?
All right, three.
Cool.
If you're not using your computer to run a game, then close it or set it aside so people know.
How many stations are you going to have over here?
Two
Two?
And group behind you, how many workstations?
Four?
How many workstations are you going to have?
Two?
And in the back, how many?
One?
1, 2, 3, 4, 5, 9, 12.
So basically if you have-- there's one, two, three, four, five testers, plus let's say each team send out two people to test other games.
Remember to rotate.
We're going to do this for about 20 minutes, as long as it takes.
And then see where we are and do it again to get some just quick testing and make sure that the five of us get to play a good number of your games and give you some feedback on that.
So remember, if you're using digital-- if you're not, if a computer's not being used for testing, close it or make it look like it's not being used for testing by typing on it.
Let's get started.
Can I get everyone's attention really quick?
Can I just get a really brief report?
How many teams-- how many tests did you guys get?
Two rounds and three [INAUDIBLE]
So you feel like you did a good amount of testing?
For the team close up here, how many rounds did you get?
How much testing did you get?
We got about four [INAUDIBLE]
Four?
[INAUDIBLE]?
How many tests did you get?
How many people played the game?
A good number?
We had two who played [INAUDIBLE]
Cool.
I'm just trying to get a sense.
You guys feel like you've got a good sense of number of people?
Did you guys get four or five?
And in the back, how many players did you get?
We only had one [INAUDIBLE]
Has it been useful?
Yes
OK.
So let's close off testing for now.
If you'd like to test later today or later during the class period, you can.
Go ahead and finish off whatever testing you're doing right now.
And at 2:50, we're going to come down and each team's going to give a brief, two-minute description about the state of their game and your product backlog.
And those are actually going to get recorded so that we can send them to Pablo and the rest of his team, so they can see what you've been working on and give us some feedback about it.
OK?
So 2:50.
Everyone set?
[INAUDIBLE]
So Snap, come on up.
And again, two minutes.
Talk about your features.
Talk about what's in your game now, what will be in your game.
Also, give a really brief description of the topic your game's about.
So this is for our clients to know what we decided to do for all of our topics
So we're Snap.
So we've decided to [?
return ?]
Snap into a multiplayer game.
So it'll be all everybody playing at the same time.
Right now we have a game that looks pretty similar to what we played in class, where you can enter words and you can snap with anybody else who has played the game.
Right now we don't have any fiction and we don't have any indication of score other than just the number of times you've snapped.
We're hoping to change that.
We got a lot of really good feedback today, and we've thought new ideas for how to convey feedback to the player and some improvements to the UI, because right now we don't really express much to the player.
That's about it
Any challenges that [INAUDIBLE] leading up to where you are now?
Is there anything [INAUDIBLE] questions [INAUDIBLE] risks that you have right now?
So the networking is still a risk, as we just saw.
It's a lot more confusing when something doesn't work, why it's not working.
We also do want to confirm that the game that we build is similar enough to Snap in terms of satisfying the client's goals.
So we don't want to change the game, even if it's more fun.
We want to make sure that the game still satisfies the client's requirements and that they'll still be able to use it and gather the information they want to.
And that will require some careful testing to see what people end up doing with Snap
And what did you decide on?
Have you decided on tech yet, or are you still trying out multiple--
Yeah.
So I think we have a tech.
So we have a server running on Node.
And we're going to use Phaser for the front-end so that we can make a more expressive game.
For now, we're not using Phaser, though.
The current prototype doesn't use Phaser at all
One thing you asked earlier was [INAUDIBLE].
Does that basically mean that you have to run on two different flight, or--
Yeah, yeah.
So the idea would be to experiment with what sorts of entire group visualizations would be interesting to put up for the entire room.
And the back-end team might work on what sort of visualizations we want to show separate from what we want to export at the end of the game
[INAUDIBLE] anybody else?
[INAUDIBLE]
Thank you
[INAUDIBLE] Cholera is Awesome
Yeah.
So the current state of our game, we spent a lot of time thinking about design and how exactly we want to do things.
So we're currently making sort of a game where you control-- or don't control.
You take care of a bunch of villages.
So you see different villages, how infected they are, and how many people are dying sort of thing, and then implement measures to prevent cholera spreading.
So you give people soap to wash their hands or set restroom facilities, or use vaccination, and that sort of thing.
And so a lot of the feedback that we got on a lot of questions that we're facing basically revolves around the issue of making this game realistic versus making this game more fun sort of thing.
So you can make a really unrealistic game that can be tons of fun, but we also want to stick to reality because at the end of the day, we do want to teach the people that are playing how to prevent cholera from spreading.
So there's sort of this trade-off that we have to make.
And the second trade off is sort of like you can make a blantanty educational game that just throws back at you that can sort of be fun for the first time you play it, but doesn't have a lot of replay value.
And it's sort of hard to make a great educational game that also has a lot of replay value.
So we're thinking, maybe consider maybe we don't want that much replay value.
Maybe this is a game that these people will play once or twice, learn what they need to learn, and then never play it again.
So it's sort of trade-offs that we need to think about
So you're still working on the paper prototype that you need [INAUDIBLE]?
Yes
OK.
Cool.
Thanks, [INAUDIBLE]
Yeah.
But we're starting to build up the back-end.
And we're going to be using Phaser to [INAUDIBLE]
So you've already [INAUDIBLE]?
OK, cool.
Forecast-Based Funding, right?
Yep.
Hi, we're Forecast-Based Funding.
The general idea behind our concept is that planning ahead for disasters is much better than trying to react to them.
So if you can have operating procedures or ways of planning for them [INAUDIBLE] afford that, you can reduce loss of lives in the event of disasters.
We are actually between two prototypes right now that we're testing to try and get at the ideas.
The first one is a sort of higher-level city-based simulation of a city that's at risk of disaster, which you then have to fortify, [INAUDIBLE] train volunteers, or preparing for upcoming disasters in order to prevent too much damage from happening to them.
One of the problems that we're seeing with the game is that's kind of abstract and not as interactive for players to connect with.
But they are getting a good understanding of the idea of planning ahead
Yeah.
And so to try and address those issues, we have a second prototype right now, which is about trying to actually rescue people in a flooding city.
And so that hits the other end of the scale, where the player is told ahead of time this disaster was planned for versus this disaster was not planned for, and the appropriate effects for each.
And then they have to rescue people under those two different conditions, and then they get to directly compare what the experience of is for acting in one circumstance versus the other
Over the next couple of days, we'll be looking at what we learned from both of them and trying to either combine them or pull out the parts that we thought were really useful to [INAUDIBLE]
[INAUDIBLE] on this?
This was the hard one
Yeah
[INAUDIBLE]?
I think it was stated a couple of times that we should be looking at people that are policymakers or donors in the sense of sort of people who would be allocating funds from governments or non-profits, things like that.
Basically we make it appear that this type of planning ahead is a good idea
And have you decided on technology yet, or are you still pondering it?
We're probably using Phaser
All right.
Thank you.
Heat Wave
Hi.
So as you guys have all heard, Heat Wave is going to be a game that will hopefully be used to help Red Cross volunteers help people in areas that are either suffering from or about to suffer from a heat wave.
So what we did was we decided we wanted to test out a very simple digital prototype, not with the same type action as our final prototype, but not in Unity because having a working game in Unity at this point we didn't think is a viable option.
So we made a Python test base game.
And people were able to choose what they were going to do based on the scenario and the people they were interacting with.
And what we really want to test it was how does this work.
Is playing a good way to learn?
And if so, how can we make more learning come out of the fact that they're playing a game and then making these choices.
So what we did was we showed people, and a lot of times, people noticed right away, well, I don't want to sit there and do nothing, which is what we wanted them to notice.
And oh, this person passed out even though they were only outside for three hours.
Why was that?
So we did see a lot of that.
But something we didn't see was people sometimes got stuck in a pattern.
It's just like, well, I'm always going to do the same thing.
And then they don't get different results and they don't really learn anything.
So we want to give people more interesting options and more options in our actual game so that they try more things and they learn more.
So that's what we're going to do
So you already [INAUDIBLE]?
Yeah.
Any other questions?
[INAUDIBLE]
[?
Great.
?]
Thank you.
[INAUDIBLE] Animal Village
Hi, everybody.
We're saving the Animal Village.
Our goal is to empower around 8 to 13-year-olds to reach into their community and learn more about what cholera is, what its symptoms are, and how they would convince people that cholera is actually happen.
Our play tests were focused on bringing as many minigames as possible, and to testing.
So in this case, we had four minigames.
The main feedback we got on those games was that they were far too easy and simplistic.
And we agree, it turns out.
Our goals moving forward are to essentially improve our game beyond just reading an informational pamphlet, and to do things where it requires actual abstract thought and in general, more effort from the player to think of the behaviors and actions they need to potentially change or encourage in their community.
We found a lot of success with this in our game already present in the mayor dialogue.
And we hope to branch that up as moving towards, we begin to implement our game in Phaser.
Any questions?
[INAUDIBLE] what's the upside of having minigames versus one [?
full ?]
game?
So our goal with the united minigames is to be able to drill down and be very specific about which behavior we want people to show.
There are approximately three to four core behaviors that are very necessary to prevent cholera.
And having one minigame devoted to each of them in theory will allows us to focus more and teach more, as a result
That's a good statement.
Just riffing off something else that you said.
I haven't looked that closely at the cholera documenation.
I'm not so sure if convincing people that cholera is a problem is necessarily the thing you need to do, but convincing people to change their behavior and whatnot
So not convincing people cholera is a problem because obviously.
It's convincing people to report cholera as it happens instead of waiting to see if that one isolated case of diarrhea is actually a symptom of an outbreak or just someone eating something bad
Right.
So one of the things is like this early alert thing [INAUDIBLE].
OK.
Cool.
All right
Thank you
All right.
Thanks.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, most everyone's here.
We're still going to take a few minutes to wait for the last few people show up.
I don't have a ton of announcements, but if you looked at the syllabus, you might notice is there are a couple days where we have, I think we've got TBD as the topic, like to be determined.
What I'd like to do is just take a quick poll from y'all.
Is there anything that you are interested in learning about in the creation of video games that you either we haven't talked about in class yet, maybe we've talked about it but not fully, you've looked at the syllabus and it's not there on the syllabus.
Or it's something that you think is going to be directly useful for the games and you're working on right now.
So just putting that out there.
If there's any topics you might want to talk about, learn about, let me know.
Yes?
You really, like, in other classes you didn't say, really, in the last class or the last two classes, say really, like, what people actually do in the industry
OK, so yes.
Cool, so what people do in the industry.
Cool.
You are in luck.
On December 1st and 3rd.
On the first, we're having a business in games panel.
So two independent developers in the area are going to come by and talk about how they started their companies, what do they do, what's their daily life, things like that.
And then another one following that is the community manager for Fire Hose Games, Sean Baptiste, is going to talk to us about how do they get their games out in the world, and how they get people to play their games.
And how do they engage their fan base.
And how do they even just create a fanbase in general.
So that's not on the syllabus, But we will be doing that.
Any other topics?
OK.
I'm going to ask this again in a couple weeks in case something pops up, because you probably haven't done a lot of technical things with your game engines yet.
So if there's anything gaming specific that you might want to know about, either let us know through the through the video game bosses email address, or let us know in class.
Yeah?
Who will be giving these lectures?
Like, is this open to-- can you get--
I can get people.
So I can get people who are local, we've got connections to local industry.
We've got connections to the people who make the tools, in some cases, depending on the tool.
That's why I'm asking now, because it gives me time to fill new gaps.
Also, it might be something that we can talk to.
So if there's anything design-related that you think is useful for your games that we haven't talked about yet, and you don't see it on the schedule, let us know.
If there's anything technical related, Drew can come up and say something.
The other thing we can do is if there's anything that you want to do, or if you want to set up office hours in class to talk about a certain topic for your games, specifically technical issues, technical questions, that's what Drew's here for.
To come and talk with your group as a whole, or just individual members in your group to talk about the issues that you have.
So we can do that during class, during our schedule, one and a half to two hour work-in-class periods, or you can set up work office hours with him.
Just give people couple more minutes.
The attendance sheet is moving around.
So is everybody sitting with their teams right now?
It looks like-- I'm going to go across and just try to figure out which of these teams are.
So it's the snap team, is everyone here?
Two, OK.
Which team, this is forecast-based futures over here, right?
How far do you go?
John, are you in this team, or are you in the next team?
Cool.
Next team over, which team are you?
Cholera?
No.
You're in cholera team?
Were you the island group?
Animal Crossing one.
Cool.
And then in the front here we've got heat wave, and the back it's the other cholera group.
OK.
I did not get a chance to play your game.
But I will eventually.
So there's going to be a little bit of interactivity in the talk.
I've given this talk before.
I rewrote the lecture, and I haven't given it since I rewrote lecture, so we'll see how it goes.
They're my backup to let me know if I forgot anything.
There will be some reading on this I'm going to put into Stellar.
So if you are interested in the topic, if you have more interest in the topic and you want to read the reading, I would highly recommend it, especially if you think you're going to be in a leadership position in the future, or any kind of managerial position.
Or you're interested in just psychology, and how people talk to each other and work together.
So our three main topics, this will actually be pretty quick on this lecture.
We're going talk about, we're going to do a really quick review of Agile.
We're going to talk about team dynamics, and we're going to talk about how do distributed Agile teams perform?
So what are some of the best practices they have for performing?
And then you're going to work in class.
You'll have at least two hours to work in class today, a little more than that.
So agile processes.
If you forget everything in this class, you got into the real world and somebody asked you, what is agile?
I hope you remember this part.
Iterative development.
Does anybody consider what they're doing iterative development on their projects right now?
Are you working in short periods of time?
So you broken up your time in the small sprints.
You can't really read it that well on the slide.
But are you within that short period of time designing what you're going to work on, planning how it's going to work, doing it, and then testing it evaluating, the test and changing it?
Are you doing all that within one week, or two week cycle?
Raise your hands yea-- so yes, raise your hands.
No, raise your hands.
It's OK if you say no.
OK.
If you're doing something that looks like this, then great.
And remember, agile, and in particular, Scrum processes, they're based on these three pillars that Sarah talked about when we first brought this up.
So the pillar of transparency.
And what we mean by transparency there, we really just need a common nomenclature.
To be honest, I don't care what you call the things that you use as part of your process in your team.
So long as everyone within the team knows what it is, so long as whoever you're submitting those processes to also know who it is.
So we're not acting as your product owners in the class.
But if you had a product owner, they should also know what you're calling things.
What you're calling your product backlog, what you're calling a task, how you're estimating and how you're doing your processes.
So everyone on the team should know it, not just in a more traditional manner, a project manager knowing what all the processes are.
There's a number of inspection that's going on.
So through the process, you are taking a look at all the artifacts that you've created.
So those backlog items, tasks lists, things like that that you're creating during the process.
You're inspecting it, and in particular, you're inspecting it during meetings.
So you're taking a look at what you've done in a meeting, and talking about it.
So you're not just doing a process, you're doing a process and you're talking about it, and you're actively thinking about the process.
And then you're adapting.
So based on evidence, based on changes in needs, you're changing that process.
Again, if you're doing any one of these three things, you're doing processes well.
You're doing some form of agile-like process.
That's another thing I hope you take from the class.
The agile manifesto.
It is a manifesto, It is an idea.
It is something that has some political weight behind it.
It's saying, this is the way to do things, this is the right way to do things.
But it's actually more vague than that.
It's really just saying, here are some four key things that we feel are more important than other four key things.
So the first item is individuals and interactions over processes and tools.
So we've been talking a lot processes and tools for the first part of the semester, now we're going to talk about individuals and interactions between individuals.
So your team members.
All right.
What's a team?
Anybody give me a working definition for what you'd call a team.
What'd you say?
A bunch of [INAUDIBLE]
A bunch of puppies?
Is it a group of people?
Is it just a group of people?
A group of people working toward a common goal
Yes.
Anything else?
So it's a collection of individuals, so it's a group of people.
They're working toward a common purpose.
They have shared responsibility for common outcomes.
If one of those puppies trips, that stick goes in another puppy's mouth, and it hurts a lot.
So keep that in mind.
Shared responsibility, common purpose, common outcome.
So teams evolve over time.
And the little bit of research I'm going to talk about actually comes from studying group behaviors.
And this is all from about 1960s and on up, and around the 2000s, and I'll post this reading up.
There were some studies showing that this actually works for teams.
This does work for people who have a common goal.
But as you get a bunch of people together, and get a small group of people together.
And in our terms, we're calling a small group of people eight to 10 people.
Maybe a little bit bigger, but about 10 people total.
They're going to evolve based on those interpersonal relationships.
So how they know each other, what they talk about, how they talk about.
And their task behaviors.
So we're going to be talking about not the tasks that you're doing in your teams, but the behavior surrounding the tasks that you're doing it.
So if you're working on a task, if you are doing some coding, how are you coding, or who are you coding with?
How are you talking to other people about what you're coding?
And you can apply to any task that you're doing in your teams.
There's a lot of different theories.
Actually, Wikipedia is a pretty good nutshell of what are all the different kinds of theories, and different kinds of models we can use to describe how groups talk to each other, and how groups evolve.
But they all come down to these common things.
There's a period of getting to know each other, there's a period of experiencing conflict, and there's both positive and negative conflict when you're talking with each other.
Your roles shift based on changing knowledge and changing experience.
And hopefully at some point, there's some kind of consensus, there's some kind of moving forward.
Decisions happen here.
The point is, though, what kind of decisions get made?
Are they effective decisions, or are they ineffective, unproven, they get made because you have to make them, and you have to move on.
And we're hoping that by analyzing how your teams work, you can then move towards having some effective decision making.
So the model we're going to talk about is Tuckman's model from '65.
It's got four phases.
Forming, norming, storming, performing.
It's very cutesy.
It gets used in training sessions a lot.
It can just be something that you can just spit out and say, and someone says, oh, you're smart, you know that model.
But maybe you don't know everything behind it.
I might be more in the you're smart, but maybe you don't know everything behind it.
I'm still learning about how this works.
But each phase has items you can identify.
More useful for us is this is a model you can use to figure out where your team is right now.
What phase are you in, so you can then make decisions about where you need to go, and how you need to work together better.
So the forming phase of it is when you first are getting together.
Everyone wants to be friendly, everyone wants to get along and to be accepted with each other.
There might be some serious problems from the get-go, but they're usually avoided.
And the team learns about challenges and goals.
But not much gets done.
There's no real task completion going on in this stage.
When you think about your previous teams, you've been in this stage, right?
When you first got together?
Say project two, project three, I'm going to ask you some questions about projects two and three as we go forward.
All right.
That's a happy stage.
Then storming happens.
People are able to express discontent.
There actually might have been discontent in phase one, but it actually wasn't being expressed in some form.
In phase two, it's being expressed.
But how it's being expressed can be different.
So it could be someone actively saying, no, you're wrong, that's stupid.
It could be someone not talking.
But they're feeling the same thing, and they're thinking about the same thing.
It's the same kind of disconnect can happen.
Opinions tend to get challenged.
It can be contentious, it can be unpleasant.
In order to advance past this stage, what your team really need is tolerance and patience just to get through it.
But even more important, you need to establish lines of trust and open communication to get through this stage.
This is what we've been doing for the past first of the semester, is creating processes.
We've been giving you some tools that you can use, so then you can then create those processes in your teams.
So just a quick hand raise from teams.
Does anybody think that they're in this stage right now?
Do you think, in your team, you are currently, you're expressing discontent, you are creating processes, you don't have processors yet, but you're figuring out what your process might be?
Maybe?
What do you say?
You're not sure?
Yeah
OK. Next u p, norming.
So this is where you have those processes.
Individuals start giving up their own ideas and goals.
You're actually moving toward a team goal right now, and people are starting to take responsibility, and rules are established.
OK.
Comparing storming to norming, raise your hands if you think your team is in norming right now.
OK, yeah.
Now raise your hands if you think your team is in storming right now.
OK. All right, so.
Is anybody performing?
Are you functioning as a unit efficiently?
Are your team members already autonomous?
Is descent expected and addressed right now?
This is where you want to get to.
You probably aren't this far yet.
My bet's you're probably somewhere in the storming area things, maybe getting close to the norming area things.
Basically, you're still talking to each other, you still trying to figure out what are the different ways of communicating with each other.
What are the different processes and rules you might have.
And you might not get this far in class, by the end of the semester.
You might not be fully efficient by the end of the class, and that's OK.
But if you are, it's kind of cool.
So the model in practice tends to be nonlinear.
It gets talked about, and it was originally established as a you go from here, to here, to here, to here.
You go through the phases, at the end of the thing, you're done.
Your team's awesome, go you.
What we Actually see in process is that teams tend get in the forming, they go in the circle where they go from having some issues, figuring out how to deal with those issues, establish some rules.
All right, we're going really well again.
New challenge comes up, we're back in the storming.
We've put ourselves in a position where we don't know how to talk to each other, or we don't know how to communicate with each other.
So there's some problems with the model.
This is really a descriptive model.
It's a tool for us to use to see how things are going.
It's basically a way to evaluate symptoms.
But there's no triggers in the model.
We don't know when those things happen.
When does your team switch from storming to norming?
So this is what I'd like you to do in your teams, just take a couple minutes.
Talk about your previous teams you've been on, talk about the team you're on now.
When do you think your team would move from stage to stage, and how could a team move intentionally?
And in particular, think about that storming to norming part of it.
So what is it going to take for your team right now, or what happened in your team right now to go from, we are having some difficulty talking with each other, we have a bunch of opinions, to we're all on the same page.
Take three minutes
Just one thing I want to chime in.
There's a very, very good chance that your team actually is still in forming stage.
And one of the reasons why very few of you might have put up your hands that you don't actually want to express dissent, which is exactly the situation that you get in the forming stage.
It is uncomfortable, but hopefully you can logically see the benefits of being in a team where you can talk about the things that you disagree with, right?
So it's possible that you are still in forming, and it's good to identify your teams as, OK, this normal evolution of a team, and that's where our team is at right now.
So talk about it among your team
So take three minutes, just talk to your teams.
I'm going to ask you some questions when we're done.
[SIDE CONVERSATIONS] So each group, give me something you talked about about what are the triggers, what are the things you saw as you moved?
Just let me know which phases you're talking about.
So go ahead
So we have the answer to [INAUDIBLE].
We're somewhere between storming and norming
How do you know?
What caused you to get there, you think?
We got a lot of the core game squared out.
And that was when we had the most storming.
The reason that I think we're closer to norming now is because it's like past standard.
Pick them up, [INAUDIBLE] and marking this done.
Which is faster.
On the front end, we're between forming and storming.
Because we're still working on a lot of design decisions
All right, so it's big decisions.
How big are those two teams?
Four and four
Four and four?
OK.
So somebody else next.
A trigger that you saw that made you go from one stage to the other
I think as a team, we're moved from storming to norming.
And part of that trigger was working on solidifying our game idea.
Because at this point, while we all recognize that we're building as a team, we all have different ideas of what this game should be.
And as we're solidifying the game, we're also getting ourselves in line with what we're actually doing, and how we're going to do it
Do you think that the task of solidifying the game idea is helping you, because there's this big decision you have to make.
Is that helping you work as a team?
It's not getting in your way?
I think it's the fact that as we're doing this, we've definitely had this conflict in our team, where people were shouting out different ideas, and trying to work through it.
And at first, we weren't doing the best job of getting those ideas and working through them.
But I see that now, we're doing a much better job of understanding each other, and seeing how it fits into the bigger picture
OK, cool.
Next team.
A triggering moment that you saw in this team or previous teams
So we had a bigger power team at first, and I think that having the sides split up was the way to help us jump from forming to storming.
Because we had to use scripts in order to get our team to get to ideas.
Also, there were conflicting ideas in terms of where ideas go in the brainstorming session
So it's basically just reducing the number of opinions, really, right?
When that is purely through people
So with our game, I think getting us to norming was really-- you talked about assigning tasks.
But also just the accountability aspect, because instead of spending time arguing, we had a designated person, or a couple people who are capable for different things.
Which allows us to just not argue [INAUDIBLE]
So like certain people have different they own some subtasks, or their own subroles?
Yes
OK.
Cool.
In the back?
What do you think?
I guess what got us over to storming was meeting together as a team to make design decisions, which was like what has been said there.
And I think that the next step for bringing us over to norming would be actually getting everyone to participate in design decisions.
Because people have both not shown up, and not particularly shown interest in getting things going.
So having everyone on board is [INAUDIBLE]
So that's a big one.
That's a huge difficulty, especially with student projects.
You've got all these other things going on around you.
Do you think you need everyone to contribute to the design, or do you need everyone to be on board with the design?
So I understand some people are just less interested in having deep discussions about the design of the game, but getting everyone to both be aware and willing to participate when they need to
OK. Because yeah, there's important conversations being done that, if they're not there when it happens, they're not going to know where the status of the game is
Yeah.
They just need to be willing to participate.
If they're not there, take the step of, oh, what'd you guys talk about?
I'm interested in knowing
Thank you for that lead, because that gets us to our next stage.
So I think we find ourselves talking a lot about the product that we're making.
It's just easier.
It's that hard thing you can see.
It's being made, it's being done.
We can play it.
It means we're probably doing something right.
But it's still very difficult to talk about the people.
But we saw been talking about the people a little bit here, especially with accountability in this one.
It was good.
Cool.
So that brings me to the next topic.
Teams are composed of individuals, and each person on your team is going to have a lot of different factors affecting what they do, how they do it, when they do it.
I can't give you the magic formula to get them motivated.
What I can do is show you some of the different ways that you can try to either understand the people you're with, understand the people on your team, or find things to talk to them about.
You might notice a number of these things.
So to get someone who's motivated, someone who's going to be able to participate, think about all these things.
Think about personal development.
So you're in a class, you're here hopefully because you want to learn.
You want to learn a new skill, you want to try a new skill.
Everyone wants to learn a new skill.
There might be a skill that they want to work on on your team, and I think we've talked about that in the past when we were talking about the vision statement, and people putting down why they're involved on the team, and why they're involved in this project.
So being aware of that, and constantly coming back to that for each person, especially if you have someone on your team who is just not participating, and you need to figure out how to engage them.
Think about the skills that they want to bring to the table, the skills they're trying to practice.
When it comes to motivation, it largely comes down to intrinsic or extrinsic.
The extrinsic might be the grade that they're going to get, but it's really difficult to really hone in on that, because the grade's shared between your group.
So that might not be the place to look at.
Try to take a look at the intrinsic things.
If they had enthusiasms about the project at some point, try to remember what that was.
Ask somebody else on your team what that might have been.
See if you can bring that enthusiasm back into the project.
Morale or self-worth.
There's been some research that shows that the reason why people work well on teams, and the reason why we want to work on teams is because it brings us self-worth.
And in particular, there's some self awareness that can happen by being on a team, and getting-- and actually, it links a little bit to extrinsic rewards.
By getting rewarded, getting feedback on my performance, it's going to help me do better.
Empowerment, giving ownership.
So raise your hands if your team uses assigning tasks or taking tasks.
So assigning tasks, raise your hand.
If your team assigns tasks to individuals.
That team.
Kind of?
Yeah.
You assign, so there's one person on your team that says you are going to do this.
Has anybody tried ownership, where someone takes a task from a set of tasks that are available?
Little, little.
When does that work?
When does ownership work?
[INAUDIBLE].
What's that?
When people are eager to work, and they're interested in what we're doing
So there's some of those other things.
There's some enthusiasm over it.
Yeah?
It's usually a while before the deadline, too
OK, cool.
Yeah?
You can run into problems if you say who's going to do this, and nobody volunteers.
So my experience has been that it's good if people can volunteer for tasks that they're enthusiastic about, but then, if nobody will volunteer to do something that needs to be done, someone has to step up and say, OK, you're going to do this.
So it's like kind of a hybrid [INAUDIBLE]
So where does-- yeah, go ahead
Yeah, and I think also, it depends on if everyone on a team is willing to take ownership versus, like, only some people.
Because that'll create a different dynamic.
There's some people who are really taking initiative, but the others aren't, so you're still going to need to assign them tasks.
So I think something that's where at the beginning ask for people to take ownership, and then the tasks that are left, there's one person who's going to delegate this out
So at the beginning, everybody takes on tasks.
And at the end, when you have the leftovers, they get assigned.
Yeah, Matt?
One of the biggest problems I've seen with this is when you're not thinking about the game you are working on, especially this is very easy since we have a lot of other work, then things can stall, because you're not thinking about what kind of game, so you're not thinking about being passive
So you might not actually have ownership.
It might look like you had ownership.
There is the evidence of ownership in that someone said, OK, I'll take that task.
But there's no follow through, there's no commitment.
And that gets to the next one, commitment.
The best way to have commitment on your team is for one person to be committed to express that commitment, to express that loyalty.
For everyone else to see that loyalty and see that commitment, and then hopefully they pick up the ball, and they're like, OK, I'm going to be committed, too.
If that second person doesn't do that, pick it up, then it doesn't travel through the chain, it just doesn't happen.
Even though that one person went out of their way to show that kind of sacrifice, it just didn't get picked up.
Do see that chain happening on your teams, do you see it breaking?
Can you identify where that breaking point is, and can you think about ways to fix that?
Trust is a big one.
So raise your hands if you have worked with the majority of people on your team before, in some fashion.
This class or other classes.
You feel like you know your teammates.
Raise your hands if you feel like you really know your teammates, you've worked with them before.
High if you really think so.
So one, two, maybe.
One, two, three.
So think about how well you know each other, think about ways you can figure out how you can know each other better.
Social interactions.
Going out for coffee, going out for lunch, things like that.
Is there something that you can do that will help that kind of thing happen?
And then here's the big one, this is the MIT one, stress.
Stress tends to be the negative.
It can be positive, in that it's the end of the project, and it's due, and you just need to get things done, and that could lead to crunch.
But then there's also the huge negative aspect of stress, and that's very much an external factor, for the most part.
It could be in an internal factor.
If there's some stress being caused by internal things going on with your team, it's likely related to one of the ones above it.
So one way to think about these 7 things, you can think of them in terms of the desirability to learn new skills, or the desirability to take on task.
So if you're trying to come up with strategies with your team on how to solve some of these problems, think of it in those two terms.
So how do distributed teams perform?
So what do I mean by distributed?
What is a distributed team?
Yeah?
Chopped up into groups, and each group is working on its own secret?
Yeah.
Or just everyone's all over the place.
It could mean that you are in Singapore, and someone else is in Ohio.
You could be separated by time zones, by oceans.
And really, I think that might be east campus and MacGregor, maybe.
It could just be through distance, it could be through your schools, could be through your departments, your majors, whatnot.
There's some kind of boundaries going on between people.
In this case, it's largely time.
Some people can work during some hours, some people can't work during those hours.
So what have you done so far to fix that?
And think back on the postmortems you've given us.
What are some of the strategies you've already used and seen used to figure out how to work across different time periods?
Well, right now, we're using Slacked, which is pretty helpful.
[INAUDIBLE]
What's it called?
Slacked.
It's like Flowdock
It's like what?
Flowdock
What's that?
It's group instant messaging
Yay
And it keeps track of all our assigned tasks.
And also, [INAUDIBLE]
Cool.
anybody else using group IM, or an IRC chat?
Yeah?
We're using something called [INAUDIBLE], which is like a chat tied to your GitHub
Cool
I think Skype also just allows communication [INAUDIBLE]
Cool.
OK.
It's great when it can hook up to the things, and it works with your workflow.
It's also just great to have just period.
If you don't use any of those, maybe think about Skype.
Any other things people use?
Yeah
In my experience, let's say we're working on something, and [INAUDIBLE] at that point.
It would be good to keep the whole team informed of all the programs, what's happening on your grant.
So we really like to just use a Google Doc explaining exactly what you have done so far
So reporting.
I remember some people had issues with email on previous teams.
So anybody still using email as a primary means of communication on your team?
How's it working for y'all so far?
It hasn't gotten to the hectic point yet.
We think we started talking about emailing subgroups more, except [INAUDIBLE]
Yep.
Are those emails going to be archived and viewable by everyone at a later date?
That would be a good idea
Yeah.
And I think that's why we tend to think about group IMs.
And particularly, Philip likes IRC for that very reason.
Certain IRC servers will keep an archive for you.
Other kinds of groupware will do that too, but they tend to be pricey.
Has anybody started using Trello on your teams yet?
Yeah?
Are you using it as a communication method, or as a recording of things?
Recording of things
Anybody using that as a communication method at all?
Trying that out.
Because I think it can do email notifications and things like that when things change.
Assigning members to tasks, stuff like that.
All right.
So again, we got talking into tools.
What other things can we do?
So another paper that it's going to be on Stellar if you want to look at it was basically a survey of distributed teams using agile processes, and giving up some best practices of what worked best for them.
And I was nice, and I'm just give you the summary of that.
So one thing they noticed is having a one team mindset was the thing that got them through working together as a group.
What do we mean by one team?
That the group first identifies as a team, like what we talked about before.
They all have a goal that they're going towards, they all identify as going towards that goal.
They don't identify as just an individual group.
But that's actually a major part of their identity.
For them, it's because that's their 40 hour, 80 hour, 120 hour work weeks.
Something I think you you're going to have an issue with, because you're working smaller periods of time.
But the way they're able to do that is they've got these frequent team interactions.
And actually, when they're working for a long period of time, there are people who are working on their own in their home or their home office with another team that's very far away.
And yet, these teams still do a daily stand up.
Does anybody here do a daily stand up on their teams right now?
So what they're doing is they're going on video, they're talking to each other for a very short period of time about what they've done, what they're going to do, what's in their way.
The cool thing with daily stand ups, and why they think this works for them, it goes back to those seven items I listed about how you get people engaged on a team?
It shows commitment.
The fact that there are people who are showing up to meetings.
There's some kind of personal sacrifice.
Some people are meeting at a very strange hour of the day that they're not normally going to meet, and they're doing that because of the team.
They're sacrificing something personal for the team.
Everyone else on the team sees that, they see that kind of commitment.
They feel like they need to reflect that back themselves.
Also trust.
And in this particular case, it's the trust of not letting meetings take too long.
These daily stand ups really should only take 10 minutes.
An effective team can do it in five.
An effective 10 person team.
30 seconds each, go, everyone says what they've done.
Everyone sees each other's faces.
The great thing about seeing each other's faces is you can see what stage of fear and anxiety they're in.
And you can then talk to them later, and have some other personal interactions with them.
The other thing they do, they do co-located work.
They've got money sometimes, so they're flying people back and forth.
But again, they're also doing co-located work by doing things, having Skype always on in the background while they're working together.
But they're not doing it the entire time, they can't afford that.
So one thing you can do is to schedule your co-located time at the beginning of the project, which is basically what we've been doing in class for the past few weeks.
Identifying important milestones, and making sure you've scheduled a co-located period during that milestone, right before that milestone is going to help you greatly.
Doing insert teams, and doing video chat.
And here's a big one we haven't talked about in class so much, but I get the impression that some of you might be interested in this kind of thing.
Having coaches.
So we've already seen some of the teams have a dedicated scrum master or producer.
Think about having a dedicated coach.
So someone who's on your team, who's there to cultivate team spirit.
They're going to emphasize the importance of one team.
They could be your scrum master or producer, but they're not actually the same role.
The scrum master or the producer is measuring productivity.
They're measuring tasks on how well things get done.
The coach is very different.
They're looking at how people interact with each other, how people talk to each other.
How people work with each other.
Oftentimes, people who are naturally inclined to do one can be inclined to do the other, but that's not always the case.
So if you have someone on your team who is one of those extroverted, outgoing, or even introverted but really interested in what individuals do one-on-one, and how individuals work.
See if you can incorporate some coaching strategies with your teammates.
And again, you can do one-on-one meetings, and we can talk about this in the future if you're interested in this kind of stuff.
We can unpack on this topic and talk about it further.
But I highly recommend it.
So I did mention meetings.
You do have agile meetings, these people have agile meetings.
These are what they are.
So who has a formal sprint process, where you plan your sprint before you start?
So you basically create your sprint task list at a meeting?
Is everyone there?
Not everyone's there
We have the tasks, and we've focused [INAUDIBLE].
So we do have a formal week-by-week, here's what we want to get done this week
OK, cool.
And you had a meeting to create that?
Then you have your daily stand ups.
At the end of the sprint, you want to have a sprint review.
Just basically the inverse of the sprint planning meeting.
You go back and see what you made.
Hopefully, you have a build and you just basically look at the build, compare the build to the completed tasks, and make sure it matches.
The other meeting that you may or may not be doing is the agile retrospective.
And that's how did you work this week?
Is anybody doing that, is anybody at the end of their sprint, talking about the processes of what you did and how you did it?
OK.
It's what I expect.
I think you're doing yourself a disservice if you're not doing that.
And particularly, if you are doing that and you're making that a face to face meeting, that's the time when you can talk about some of these interpersonal issues.
That's the time when you can talk about your feelings, that's the time when you can open up, and try to figure out what is about, why are some of these tasks not getting taken on, who's not doing them, and what's in their way from doing them.
But really, it's not really about what's in the way, it's difficult to talk about.
It's not about what's in their way, but it might be about what outside of the project and outside of the team could be in their way?
And can you work through some of those issues?
So I'm done.
What I'd like you do before you go and work in your teams is schedule a sprint retrospective.
So take a look at your sprint process.
How long is your sprint?
Is it one week or two weeks?
We've been assuming it's one-week sprints.
Schedule a time period that everyone can attend, and an amount of time that they can attend.
That they can see each other face to face, but not necessarily are in the same room.
If they can be in the same room, awesome.
If you can't, try to figure out a time that you can chat using Google Hangout, or some other kind of face to face video chat.
Make sure the meeting is focused on talking about your processes.
So don't talk about your game, don't talk about the work that you've done.
Talk about how you organized yourself.
And in particular, talk about your interpersonal matters.
Basically, communicate about communicating.
So talk about how you're talking to each other.
And yeah, I want to talk to you about this again in a couple weeks.
I think most of you right now are probably in the concepting phase, pre-production phase right now.
You're still figuring out your game, you haven't quite-- I guess you guys are actually working on your back end, because someone did the initial design for you.
The rest are probably still working out the design phase.
When I talk about this again, you'll likely be in full-on production mode, and so some of the activities I'm going to show you might hopefully help when you're doing these meetings.
Any questions?
OK.
So it's 1:50.
Go ahead and take a few minutes to break, and then the rest of the class is yours to work in your teams.
But still do this.
[SIDE CONVERSATIONS]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
SPEAKER 1: So the schedule for today-- once again, another one of those one hour lecture, two hour group work things.
We've got a guest lecturer today.
We're got Luigi, here, to talk to us about art and artists, especially when it comes to games and, particularly, visual artists.
Yeah?
Yes.
SPEAKER 1: Yeah, so visual artists when it comes to game development.
After his lecture, and after Q&A, we don't have any scheduled play tests today, but we do have a couple of guests here who have some really good experience making games.
So if you'd like to set aside 10 to 15 minutes per team, have a person come and play your game, I'd highly recommend taking advantage of it.
I mean, it all depends, I think, on what state your games are in.
I do want to know that.
How many teams are now working in what is going to be their final development environment.
So if you're working in Phaser, you're building in Phaser now, if you're working in Unity, you're working in Unity now.
Looks like one team, two teams, three teams-- is that two different teams over there?
[INAUDIBLE] SPEAKER 1: Here.
OK, great.
So everybody's pretty much working in a development environment.
Great.
Cool.
So like I said, as it says on Stellar, Wednesday we're going to have a play test at 3 o'clock.
So at 3 o'clock we're going to have a guest class come in to play your games.
We'll do a play test, just like we've always been doing play tests.
And then before that, I'm going to be giving a talk on fiction and story for your games.
And in between there will be time for you to meet your teams and set up for your play tests
Cool, artists and games.
And So this talk's going to be about what artists do in games, how to find artists, how to work with artists, how artists are involved in game development, the different roles that are assigned to them.
I'm going to be talking from the independent games perspective, so smaller teams, because that's sort of my background and my experience with games.
But I will talk a little bit about the diversification of artists' roles in bigger studios.
First, who am I?
I'm me.
I'm this dude in the corner.
My name's Luigi Guatieri.
I'm a local Boston indie artist, when it comes to games at least-- no so much comics anymore.
Yeah, so I work with a lot of teams in Boston developing games.
Since so few of these developers know a lot of artists, I get to work on a lot of different projects, because they don't have a lot of people to turn to.
And I'll go into that in a little bit.
Sort of games I've developed in the past-- I worked on Girls Like Robots with Ziba Scott-- up there-- which was published by Adult Swim, iOS.
And he likes to tout it as the 13th best rated iOS game of 2012.
I don't really care about that statistic, but I'll mention it anyways.
I worked on The Counting Kingdom which Jenna Hoffstein, which is actually coming out this week on iOS.
So if you have kids, please buy it.
If you're not a kid, buy it anyways.
I've been working on Elegy for a Dead World, also with Ziba, and, as a partnership, with Dejobaan Studios, which had a successful Kickstarter so if anybody backed it in here, that was great.
But if none of you did, then sucks to be you.
All right, that's sort of my summary.
I'm not going to show my portfolio, or anything.
But let's get into it.
So you need an artist.
You're a programmer, or a designer, or a sound guy, and you can't draw for crap.
You think you can be like, oh no.
I'll just wing the art.
It'll be fine.
I'll release my game.
No, don't, do that.
Get an artist.
Artists make games look pretty.
Artists make games more immersive.
Artists take a player who-- maybe you have the most amazing game mechanics in the game, sort of like a dwarf fortress or something, which turns away so many people because of how ugly it is.
No, you need artists.
Artists make your game better, more interesting, even objectively better in a lot of ways-- if you get the right artist, of course.
It also makes something you're proud of looking at, and proud of showing off at shows.
When you have your booth at PAX and you've got all the AAA studios around, all the high-end indies, and then there's you with blocks that hit each other, you're going to be kind of disappointed in yourself.
But cool.
What kind of artist do you need?
But of course, that depends on what kind of game you're making.
Obviously, here I have just doodled out a few different styles.
Style's a big part of art for games.
But of course, there's different roles and different kinds of artists in the world.
I'm sure a lot of this will be redundant to any of those who really like games, but I'm going to go over it anyways.
Obviously, in big studios you have a lot of very specific roles for artists.
This is a character artist, environment artist, texture artist, this is the animator.
But when it comes to indie studios, I'm the guy who does everything.
I do the concept art, the 2D assets.
I do the promotional material.
I do the graphic design.
I do the logo design.
I make trailers.
Because usually indie studios don't have the funds.
They don't have the ability, or the time, and/or the resources to get more than one artist, so it sort of comes down to the one guy.
But sort of going to go through the different kinds of art that are typically done in games.
I'm going to break it down to their specific roles.
And I'll talk about-- I'll be showing mostly my work here.
I'll sort of break it down to the different kinds of art, and the different kinds of artists that work on these.
So first is a swath of indie games that are 2D now.
Used to be that retro games were 2D.
And then older games before polygons and all such things were all 2-dimensionals-- 2-dimension.
So 2D makes up [?
like-- ?]
illustrators, graphic designers, concept artists, sprite artists, texture artists, 2D animators, UI designers-- it is a large field.
Again, with indie developers, it's easy to go 2D, because you don't have to worry about 3D, which takes a lot more time.
So this is sort of some 2D assets from a prototype that I worked on a while back, but this is sort of [?
my old ?]
[?
end ?]
game stuff.
Of course, the first part of 2D art, which also pertains to 3D art, and at bigger studios they have specific roles for this.
This is concept art.
Concept art is far my favorite part of pre-production, at least, in terms of you get to take what the programmer wants, and actually make it look like what they think in their head it's supposed to look like.
They'll come to you with a crappy pencil sketch on some paper, and be like, here I've got a knight, and he's on a mountain, and he's fighting a dragon.
And I'll be like, this doesn't look anything like that, but I know exactly what you're talking about.
And he's like, I'm feeling red!
I want red!
And they just took like a Sharpie or something, and just scribbled all over the piece of paper.
So I'm like, all right, I got that.
Give me like a day, and I'll come back, and then you'll have this painting of a knight fighting a dragon.
And the [?
programmer's ?]
like, yes!
That's exactly what I drew on my piece of paper.
It isn't, but you let them have that.
It makes them feel better.
So concept art is definitely for both 2D and 3D.
You need to know what your game is going to look like before you dive straight into it-- into the visuals and even some of the design, before you have a clear idea of what the game's going to look like.
If you have no idea what the game is going to look like, and you already have an artist, and you've made this amazing prototype.
And you're like look, just swap out my art and put yours in.
That's not always going to work.
There's a lot of design decisions you have to make around the art-- how the characters contrast from the environment, how the environments will animate, how the environments will disappear into the mist, or how this sort of UI will work.
These are all things that it's really good to have an artist's eye on as early as possible.
It also gives a sort of visual Bible for the game.
Concept art helps-- the artist draws it, and then hands it out to the rest of the team, and like, this is what we're aiming for.
In a big studio, you would have the concept artist would hand that off to the 3D artists or the other 2D artists, the animator, the modelers, texture artists, and say, look, this is what we're aiming for.
This is what I want the final product to look as close as possible to.
But of course, if you're making 2D games, you go into 2D assets.
This is a screenshot, or an early screenshot, from Elegy for a Dead World.
So with this, I sort of made the concept art, cut it up, and stuck it in the game.
That's a really basic way of making 2D art.
I have one with this sort of painterly style that I decided to go with the concept art.
I made a painting.
I said this is the style that we're aiming for.
So I cut it up and put it in the game.
And now you can move around when things happen, and you can explore this piece of art.
2D assets are part of-- Illustration, it's more of a old school term, especially when it comes to games now.
People don't think of it illustration so much, but illustration is used for promotional art, for advertising the game, for inspiring your team, for websites, for trying to solidify your game into like a poster, or an image, or a banner.
So many games are sold on Steam now with just a tiny little banner that you click on, then you get a little slightly bigger banner, and then maybe a game-play video.
So much of that first click, like making that person go, oh that tiny little banner looks kind of interesting.
Let me click on it-- is such a crucial point, especially if you haven't done your job marketing, that you need to have illustrations and promotional art, and posters, and graphic design.
And that goes under the Illustrator sort of dome.
In a big studio, they'd usually have the concept art also be the illustrations a lot of the time, but usually the concept artists are very proficient, because they have to be fast.
And they also make the illustrations for a game.
Obviously as an indie, you're making the illustrations too.
UI design-- this is like in between three different facets.
You got design, illustration, and graphic design.
Graphic design, I mean, by like sort of logos, and making things more readable to users, but that sort of goes into game design in terms of how the UI is going to work.
And then you also need a graphic designer to actually make it look good.
This is a big part of 2D design-- obviously, it's in 2D games, but usually UI's flat, because you're dealing with buttons and menus and HUD overlays.
So that's sort of a weird intersection between all three of these facets, where it'd sort of have to be usable, has to make sense in the game design world, and it has to look good.
Let's jump into 3D.
3D is the more complicated side of the art.
Painting a picture doesn't take me a very long time, but making a 3D model takes much longer time, especially if it's highly detailed or needs a lot of texturing, or I need to make it work in a very specific type of game.
There's a reason a lot of indies avoid 3D.
It's because-- I mean, especially-- it's getting easier all the time, especially with tools like Unity and Blender but a lot in Indies do stick to 2D, because 3D just gives you a lot of hassles that you don't have to deal with in 2D.
And usually you need more artists to work with 3D, because, to make a game in 3D, it takes a long time.
You have to model environments.
You have to model characters.
You have to model objects.
Whereas in a 2D game, I'd be like all right, a box is just four lines and just color it in.
Box in 3D?
All right, I made the cube.
That's easy, whatever.
Now I got to texture it, and I got to inclusion map it, normal map it-- It takes a lot more time.
And it usually ends up looking really nice, but usually you'd need more artists.
The roles in 3D is modeling-- this is one of my earlier attempts at modeling.
So the modelers, they sculpt 3D characters in environments.
They sculpt what's going to be put into the world.
Modeling and texturing makes up the bulk of 3D work.
It takes a long time to get right, and then sort of having to make it work within your game is another big facet of that.
ZBrush is a really fancy tool that a lot of 3D artists use.
And they will sculpt the crap out of it, and be like, oh man, you can see every-- I sculpted every skin pore on this guy's face, and every vein is actually underneath his skin, and the fat, and the muscle, and the bone-- yeah, good luck getting that to run in any game right now.
Obviously, things getting more powerful.
Again, as an indie guy, if you tried to sculpt every pore into a guy's skin, you're going to end up spending a month working on that where you should have been spending a month just pumping out art.
So those get broken down further into character art, which is-- at bigger studios, you do have a specific character artist.
They only work on the characters, because the characters are a very important part of the game, obviously, and usually have to be more detailed than a lot of the other parts of the game, especially if it's the player character.
You're always staring at them if they're a third person game, or if they're NPCs you deal with regularly in the world, they're always there.
They're always looking at them.
They have to be visually nice to look at.
And the modelers have to spend a lot of time making that work right and look right.
And they also have to deal with a lot of uncanny valley stuff, which-- what the uncanny valley is, if you try to make it realistic, and it doesn't quite look realistic-- like this guy-- people look weird, and people are weirded out by that.
And it's a big problem character artists have to deal with.
Environment artists-- they're the ones who make the world.
They make the buildings.
They make the trees.
They make the mountains.
They're the ones who fill up the world.
Sometimes concept artists are actually broken up between character artists and environment artists too, depending on how big the studio is.
Again, as an indie, you're always making the environment art, the character art, the concept art, UI, and everything.
But usually when you hand in a portfolio to a big studio, as an artist, they always ask first, are you a character artist or are you an environment artist?
Unless you're going for like UI design.
But as an artist, if you go up and say, I don't-- I'm kind of both-- which is what I do, kind of both-- they just turn away.
They don't have time for you unless you're just unbelievably amazing at both.
So the big facet I skipped out on both of those things is animation, which pertains to 2D and 3D.
Animation is very, very difficult to get right in 2D, and in 3D, but 3D is way easier.
There's a reason Disney closed down all their 2D animation studios, because it's a waste of time now.
It looks good, but it's very expensive, very time consuming.
A good reference point is-- has anyone played Skullgirls?
Which was like a 2D indie fighting game.
They wanted to add a single character to their game.
And they started a Kickstarter and they asked for $250 grand just to add one character.
Because of how exact the animations had to be and the quality of the 2D animations.
Hand drawing animations is very time consuming for sprites-- and sort of hand animating 2D guys.
So that was sort of solved by tweening.
Tweening is-- as you probably all know-- is I have my arm, and I draw it, and I can do this with it.
And that makes animation more fluid, and you can sort of blend in animations with other animations much easier, especially with like, I've been using Spine-- that's it.
Spine in Unity to help blend a lot of my tweening animations.
3D usually only works with tweening animations, in sort of, you have the animation rigged.
And you move the body parts around, and sort of algorithmically draws in the in between frames.
Whereas in hand drawn 2D animation-- old school 2D animation-- characters-- if you want to move the character like this, you have to draw all of those frames usually.
So that's why a lot of indies, they turn to pixel art.
Because an arm doing this is like two blocks moving, which it takes way less time.
It's way easier.
And it's a style that a lot of people like so it's sort of easy to sell it.
If there was no such thing as pixel art, and you tried to make a pixel art game now, people would be like, this is kind of lazy.
The whole world was Disney quality animation.
All right, great, so you're thinking about what kind of artists you need, what roles you need to fill for your game.
But wait!
The most important thing is, how much money do you have in your wallet?
Because artists cost money, obviously.
They have to eat, as much as the starving artist stereotype is perpetuated.
Obviously, if you're a big studio, you can just throw down the fat stacks of cash, and say, I want four concept artists, an army of VFX artists, 3D art modelers.
I need 2D texture guys and UI designers, you name it.
And they can bring in every one and get the best of the best, because they have millions of dollars to work with.
As an indie, you don't have anything to work with, usually, unless you Kickstarted magically now, and made a ton of money, or you have some sort of outside funding, you sort of have to coax an artist into your grasp.
And either promise them with some money you have or revenue share, but usually you're going to only end up with maybe one, two artists, depending on the size of your team.
It's tough.
Amazing artists cost a lot of money.
If you're trying to go for the best of the best, the famous guy you saw on the internet one time, they're going to charge for being the famous guy on the internet you saw one time.
Middle-of-the-road artists are harder to find, because-- how do I put this-- either they're too old to have kept going with art and they've given up, because they couldn't be an amazing artist, and they're not making enough money as an artist making a living, and they're sort of given up, or you can nab them right out of school at some point or in school.
But there are a lot of crappy artists out there.
If you want to find any artist, there's a ton of them.
I will go into how to find those guys too.
First, back to the next-- one of the early slides here-- artists are important.
I know as a designer programmer, you think you can do it all, but don't skimp on artists.
If you have an amazing game idea, you want to make that product the best you can.
And having a really good artist is a big part of that.
You might think, I'm just going to sell my game on mechanics.
It's going to be great.
The best mechanics you've ever seen.
But why stop there?
Why not have amazing art to go with your mechanics?
If you can hire multiple artists, please do.
Because it'll take a load off of them, because-- especially at a big studio, even indie studios-- making art is a huge chunk of time.
As graphics get more and more intense and more and more realistic, art is the thing that takes up the most amount of time during development and it's the money sink now.
You know, OK, let's say we're working on the new Call of Duty-- just came out.
All right, this is a first-person shooter.
We can do some level design.
There's some added features.
But like, all right, we need robots, and destructible buildings, and soldiers, and everything has to look as top quality as possible, or else everyone will decry the game as looking like crap, because of one other game that did go that extra mile.
And as an indie, maybe you do want to try and compete with that, in that-- let's say, all right, you have your illustrator.
He can do the concept art, he can do the 2D assets, in a lot of ways.
But maybe he's not the best at animation-- I'm not the best at animation-- so we'll get an animator, as well, who doesn't have to come up with the cool designs-- maybe he can help or she can help-- but you have this artist.
You can be like, look, these are the monsters I need animated.
Can you do the animation while I move on to the next thing?
And especially if you're working in 3D, having multiple artists is almost a must, because otherwise you're never really going to get anything done, unless you can sort of design around having one artist, which becomes a huge limiting factor.
But again, with everything, especially in game development, you get sort of a law of diminishing returns.
The more artists you have-- or the bigger your team is-- you know, going back to Destiny with it's ridiculously high budget.
It was like $300 million or something, $500 million?
You know, how much more game did they get?
How much more entertainment value did they get with their like studios of artists?
They didn't just have one studio of artists working on that game.
They had multiple.
So if you're an indie studio and you're like, right, we have two programmers and, like, eight artists.
Eight artists, that work is diluted between them.
You have to try and make sure that each one of them is working in the exact same style.
You have to sort of understand that it is difficult to switch your art style.
If you've been drawing one way your whole life, and then you're hired to a team, maybe wrongly, to work on this game that looks nothing like anything you've ever done.
And you can't work within those limits, within that style, you're going to make work that looks different from the other artists, and it will stand out and it'll look weird.
Yes, really.
Artists are important.
I got to really stress this point because I like work.
Artists not only make the game look good, but they inspire the rest of the team.
If you're just sitting, staring at blocks hitting-- you know, pongs-- hitting a block back and forth all day, you're going to get tired of it if you're working on it for a year.
If you have constant art coming in, like concept art, animations, illustrations, and you see it when a website writes about your game, or a You-Tuber does a stream about your game, it's exciting to see like, look how good my game is, and I need to make this better to match to the awesome art that I have in the game.
It helps motivate everyone to be excited about the project they're working on.
To know that, look, when this comes out, it's going to look great.
And it's going to be compared to the best of them.
Because that's awesome!
And everyone should be inspired.
And art is such a quick way of inspiring people, that it's almost worth the money to pay the artist just to keep your wheels turning.
Because if you just, again, if you're just watching blocks bounce back and forth-- yeah, great, OK.
If you get excited about that, that's great.
But if you've got five other people on your team, and they're all just working on blocks bouncing back and forth, they're going to go and work on something else that's more interesting.
So where do you find these so-called artists?
Usually in sewers, like the Ninja Turtles.
But artists, in general, are usually introverted people.
Sort of like a upbringing of-- I'm just going to sit in the corner and doodle all the time.
A lot of programmers have the same thing-- I'm going to sit on my computer and code.
A lot of them don't go out of their way to make attention to themselves.
I first learned this when I was sort of going to meet ups around Boston for game development.
And how few artists I was meeting.
It was kind of disheartening in a lot of ways.
And the few artists I was meeting, a lot of them were like, eh, OK, get out of here, you're done.
Because artists, sort of, even some of the best artists I know, they sit in their rooms, they make their art, maybe they get a blog post on the internet, and it does really well.
But a lot of them are just sort of like, oh I'll just work on this drawing.
It's OK. And it's crap.
Work on the next one.
So you do have to look for the right artist, which is sometimes more difficult than you think it would be.
You would think all artists are always looking for work.
They're all starving.
They need work.
They need work to live.
Well, yeah sure, a lot of them.
They're not going to be coming looking for you specifically.
So here's some ideas of how to track down this elusive species.
Networking.
Meet-ups, like I mentioned, a lot of game dev meet-ups.
They do exist.
They are there.
There's some comic book meet-ups in Boston with a lot of artists.
It's all about maybe you'll meet someone who knows someone.
That's how most of my work comes in, is someone who knows me from someone or from a project I worked on through networking, they get in contact with me, and I get work.
I actually used this in a slide for one of my classes.
This is PAX.
PAX is actually another great place, surprisingly to meet other artists.
Because a lot of people are walking around booths, looking for work, surprisingly, at PAX.
Which is kind of weird.
They'll hand you business cards and you're like, oh, I'll check out your portfolio later.
Maybe they're pretty good.
And then maybe there's something to that.
Obviously, bigger conferences like GDC, IndieCade, like Unite, or something.
There's a lot of artists there that hang out, and they're a great place to meet, socialize, see if you find someone you sort of can relate to and hang out with, and maybe they have some really cool work they can make for you.
I just thought it was funny, because I randomly found this picture of PAX East on the internet.
Ziba and I at our first booth-- we're actually in it-- and it's circled.
Our birth, I was very excited about that.
Internet, that scary place.
There are a lot of artists on the internet, obviously.
There are some big sites that are great to find artists, and sort of remotely discuss with them.
CG hub used to be a big one, but for some reason the guy just shut it down.
I don't know why.
But deviant art, you have to shift through a pile of shit to find good art in deviant art, but it exists.
Tumblr is actually really good.
A lot of artists I know, especially good artists, post their art on Tumblr.
And usually with tags like game art or concept art or indie art.
And again, you have the shift through a lot of crap to find the good stuff.
But I know a lot of people who find work through Tumblr.
Conceptart.org has some of the bigger, well-known artists, but there's also a lot of smaller artists.
There's like a jobs board you can post on there.
This was conceptart.org.
A lot of the big studios post there, looking for jobs.
This is more for 2D painters, and concept artists, and 2D guys more-- Polycount, for 3D artists, which is another great website and resource.
But if you want to scrape the bottom of the barrel, schools.
Schools are a great way to get free artists.
I like the claw grabber, because that's basically what you're getting.
I mean, I'm not saying there aren't amazing artists at school.
Of course there are.
But most of them are, obviously, inexperienced.
Most of them will work for cheap to nothing, because they're just excited about the fact to work on a game.
So if you just desperately need an artist.
And you're just like, I got this prototype.
I want to pitch it somewhere else, but I just need some art in it.
Schools are a great places.
There's lots of schools in the Boston area.
A lot of art schools around here with lots of people looking for work, obviously.
But again, scraping the bottom of the barrel.
There's a lot of shows that aren't about games where are great to pick up artists.
In Boston, we have the Massachusetts Independents Comics Expo, where a bunch of artists sit around tables and they sell their comics.
This is a great place to meet artists who, maybe, aren't into games, but maybe you find someone who's like wow, your style's really unique.
Your style's really cool.
Come work for me.
Come be a part of my team and we can make something really interesting together.
There's a lot of artists' tables.
There's a lot of-- I used to do these a lot.
I don't really do them so much anymore, but-- just selling your prints on tables.
There's lots of festivals all the time.
I know this isn't really a stereotypical where to find a game artist, but they are great places to find artists who maybe never thought about making games.
And maybe they have a really cool style that'll fit games perfectly, but they've just never even thought about it.
Because, again, in this very competitive environment that we are in, game development, you want to make your games stand out and be unique.
You don't want it to look like, again, Call of Duty number five.
Why are you bothering at that point?
Why don't you find someone who has like-- no one can draw like this person.
It might not be the most realistic, and it might not be the theme of the time-- like low poly is a big thing now-- but like, this guy will stand out.
This guy will make my game look so unique and so interesting to people who may not have even been interested in my mechanics in the first place.
And it jives with my theme.
I want this person to work for me to make my game the best it can be, because that's really what it's about.
It's all about finding that special someone.
You're going to have a working relation together.
You're going to have some rough times, you're going to have some good times together.
So you really want to find someone, especially as an artist, who you guys can work together very well.
You understand a common goal.
You're going to be both-- or if you have multiple artists-- you're going to be all your harshest critics, including themselves.
You're going to-- because if no one in your group is telling you what you're doing is bad, then everyone else will and.
They will do it much harsher and much meaner than anybody in your group.
Because if you're like, hey man, you know, this is a good drawing, but why don't you try doing this to make it better, is much better than a comment on, like, [?
Kotaki ?]
was saying, this fucking sucks.
Don't buy this fucking piece of shit.
Because that's-- yeah.
You want to, again, you want to find someone who's good as an artist.
You want to find someone who's competent, who turns things in on time-- which is a big thing.
I've recommended too many artists to developer friends who did not turn things in on time, and that made me look bad in a lot of ways, because I recomm-- I thought these-- I'm like these guys are great.
They work hard.
But they weren't hitting their deadlines, they weren't getting stuff in, so they obviously lost those contracts.
So you guys are really going to be like partners in crime, especially if you're in a small, two-man group.
So these are some my working relationships I'll talk about.
Girls Like Robots, I developed with Ziba.
He's up there.
You can all look at him if you want.
He took a chance on me.
I was out of college.
And I'd done my three month period of mourning looking for a job.
And he found me and said, OK, you're OK enough.
I'll let you work on my game.
And so we really hit it off.
And we worked really well together.
This game went places we didn't think it was going to go, which was great.
It's still doing things that I didn't think it could do as a game.
And so we've continued to work together from this.
This was The Counting Kingdom with Little Worlds Interactive, Jenna Hoffstein.
This was a more standard contractual artist hiring, game development cycle.
Jenna came to me and was like, I need these five things.
Done.
I need these 10 things.
Done.
I need these five things.
Done.
All done with [?
sculpture ?]
works and work for hire contracts.
She is the most efficient game designer I've ever met in my life.
It was just boom, boom, boom, boom.
We're aiming for this release.
Done.
Released.
It was kind of amazing.
I don't think any other game I've ever seen in my life developed that smoothly.
And she didn't do any coding, which was also the crazy thing, as the lead designer and coder.
She only used PlayMaker, which was kind of fascinating.
Elegy for a Dead World was a weird in that I worked with Ziba again from Popcannibal, but this was a game jam game that turned into a full game when it started getting some attention.
It started out as a rev share agreement between a few different studios in the office.
And it's been, less of, these are my boss's working on the game, but I've sort of been like a partner in crime.
I'm on the same level as all the designers, which is kind of interesting for once.
All right, enough of that.
Great, you found an artist.
Now what do you do with them?
You have them, all right?
You just going to poke 'em a little bit?
Hey, make some art.
Well, first you have to hire them.
And there's one way I like to think that artists should be hired is with an art test.
Before you sign them up for a huge massive contract, you should say look, you know what?
Your portfolio looks great.
I really like you as a person.
I think we're going to work really well together.
But how about you like take this design that I did, and I want to see the final product.
And I want to see how long it takes you.
I want to understand your workflow and your process.
Because they might have the most amazing portfolio in the world, but if it takes them three months to draw a single picture, then it's not worth your time.
And maybe they just lied.
That's always a thing.
They might have just been like, I just found this picture.
I found a photo and I drew over it, which a lot concept artists do to get things done quickly.
They maybe can't do what you think they can do, and this is a good way to figure out exactly what they can do.
So you do pay them for this.
You pay them for their time, whether you hire them or not.
How many art tests I've gotten, which were like, oh no, no, I'm not going to pay you for the art tests, because it's just an art test.
I want to see if I want to hire you.
No, no, no.
They took time out of their schedule, doesn't matter if they're busy or not, to make this art test to show they are capable of doing the work.
You pay them for the time they spent on it, whether they continue to work for you or not.
If they don't, you say, thank you for your time.
I don't think you're exactly what we're looking for.
Get lost.
But you do pay them for their time.
Contracting, most indies are going to be on work-for-hire basises for independent artists.
What I mean by work-for-hire, those of you who don't know, any work they make, they don't own anymore.
When I draw a picture on my own, it is my copyright.
I can do whatever I want with it.
I can hire a 70 foot billboard and put it up, and there's nothing anybody can do about it.
But if I'm under a work-for-hire contract, and I'm working for a game designer or game company, and I make art for that game, I longer can put it up on a 70 foot billboard without any permissions.
The company, or the designer, or the person I'm working for now owns that art.
And that is what most artists and other people work as for game companies.
In art school, I was always told work-for-hire contracts aren't fair, because then I can't sell my art again.
And that is a-- I mean, that's good as an artist, meaning like, good, I can sell my same painting again and again and again and again for lots of money, and make more money off of this one painting.
But that's not the norm in game industry.
It's the norm in other illus-- in other illustration fields you can do that-- some magazine publishing.
Since they're all dying, they're much more lenient on that thing, on that.
But game companies is, you no longer own the art.
So you're going to have to make a work-for-hire contract, usually with a scope of work attached to it of what exactly is needed from the artist.
Another interesting point for independent contractors when hiring for work-for-hire work is software licensing.
It's usually up to the artist to have their own software.
If they're hired full-time-- if you somehow magically are able to do that, usually you have to supply the software, especially for working at an office.
Great, so you've hired them.
You've got them down.
They're sitting there.
They've got their tablet out.
They got their pen.
Now what do you do?
You got to art direct them.
You have to play art director, even if you can't draw.
As the person who hired this artist, you have to critique them, and understand how they work, and to give them feedback constantly, even if it's just encouragement.
You have to make them make the art that you want them to make for you.
They're not making art for themselves, they're making art for you.
They're not fine artists.
Fine artists just sort of waft in the air and discover what they want.
But as an illustrator, or a 3D artist, you're like a plumber.
I need you to fix this pipe.
I need you to draw this picture specifically.
It's a great variation on do what I say, not what I do, especially with a programmer or designer who can't draw.
But here are some helpful things to yell at artists.
Aesthetic, just instead of saying do it better this time, just not as crap.
Aesthetic, the general feel of the art.
This one is a little hard to critique, aesthetic, but if you have a general sense of what the theme of the game you're going for, definitely be like, all right, this is a zombie shooter.
OK, you've put Hello Kitty in here.
This isn't really the aesthetic I'm looking for in the game.
And that ties into style.
They're very close together, but style is very easier to point to and be like, we're looking for this style of game.
And that helps guide the artist.
OK, that's what they're thinking.
It's a bad idea, but I'll sort of work with it, and see what I can come up with.
Contrast is probably the most important term in art, in general, and specifically for game development art.
Contrast is how different things look, like in contrast in real life.
Like when two things contrast against each other, that you're telling the difference between them or the similarities.
In games, you want the character to stand out from the background.
You need items to be obvious.
You need certain things to stand out more than others, maybe things to disappear more than others, maybe things to be more similar, to be more hidden.
Like if you're making a Metal Gear Solid game is like, oh, I need the character to blend into the environment.
Contrast is a very helpful word when you're having trouble defining what's wrong with this image, why isn't it working in the game?
OK, it's because these animations blend in too much with the character animations.
I can't tell what's going on.
Or, this image, there's not enough depth in it.
This image, I need more depth.
Contrast is a big word.
It's a really great word that helps with that.
Saturation is another handy one, but that just means how colourful something is.
The more saturated something is, the more vibrant and colorful it is.
The less saturated it is, the more gray into whites and blacks it is.
That really helps with saying, all right, this is sort of the objects and the theme I'm going for, but why don't you make it more colorful?
Maybe make it more saturated so the world feels more fantastical and interesting.
Or, it's like I want this to be dark and grim.
Let's lower the saturation quite a bit to really make it feel like oh, it's a gray day and it's dark and dreary.
Composition, just the layout of art, and the layout of levels, or the layout of what the player is seeing, what the player is dealing with.
It's just sort of the arrangement of items.
It's another handy word to know.
But of course, encouragement, encouragement is always needed in any part of game development.
It's always great to hear you're doing a good job.
Give yourself a high-five, or me a high-five, whatever.
It's really helpful to encourage your artists, so it makes them make the best work they can.
If you're just yelling at them all the time, and they're not doing a good job-- or you don't think they're doing a good job.
Maybe they aren't, but they're not going to get better if you're just always dissing what they've got, and it's not fun.
Let's go into-- so you've got the artist, you're going to art direct them, now what are the phases?
Pre-production!
Pre-production is the happiest part of game development.
You're coming up with all the ideas.
Your scope is increasing through the roof.
You're like, oh, now we're going to have open world this.
The world's dynamic, the characters die, their families grow up, it's going to be great.
The art is going to be realistic!
It's going to go into space!
We got to have all these textures!
And the concept art is amazing!
But as an artist, you're working on, usually, the concept art for the game.
You're helping with ideas sometimes.
Maybe your designer, he's so happy to let you help out with ideas for the game.
Pitches, another good thing if you're making concept art.
If you're pitching a game to get funding, like Kickstarter or maybe it's one of the bigger studios like Sony, Microsoft, Nintendo, or others.
If you're trying to get money upfront, artwork really helps a pitch, because you have nothing to show.
On Kickstarter, you're selling hopes and dreams.
You're not selling a game.
Because people are imagining in their heads what the actual game is going to be, and it's way better than what you're actually going to make, because they have no idea what they're thinking about.
Pitches with art work better than pitches with nothing.
If you just have text, and just be like, here's some scribbles of how my game's going to work.
They're going to be like, I'm not excited.
But if you have like, here's a painting!
And then they're like, wow!
That's amazing!
How's the game going to work?
Don't worry about that.
Just look at the painting.
Web stuff, so making a website, obviously, it's early pre-production.
Maybe you want to make a blog or something about your game.
It's a great place to start.
And posting about art as it comes in from concept art, and pictures-- Logos are another big thing, especially if you're doing a pitch.
Logos really help with identifying your game, not only to your selves, but if you ever want to talk about it really early on, it's a really good way of getting your icon out there.
And early prototype work, so if you've made a prototype, and you're like, look, let's just slap some art in there to see what this game's actually going to visually feel like.
This is all part of pre-production.
Production, the bulk of the work.
So this is all the asset creation.
This is, maybe, you're doing concept art here too.
Maybe you've said, all right, that style isn't working.
Let's go back a little bit, but we'll keep moving forward with making models and texturing.
It's all about iterations too in production.
It's sort of oh, OK, this level doesn't quite look right.
Let's change it a little bit.
Let's make it look like this.
This is the longest part of game design usually, unless your pre-production was, I've thought of this game from five years ago.
This is just making the stuff that will actually go in the game and making it fit together.
Near end of production, and this is when you start panicking and realize your open world space game with dynamic NPCs and families that live and die and have children isn't exactly panning out.
You start cutting things just to get things done.
The artist is usually hopefully done by now towards the end, but they have to start doing the promotional push.
They have to start making the illustrations, and the art, and the graphic designs, and the Steam banners, and making sure everything looks as best as it can so when this game fires out to the world, a lot of people get it.
As indies, I have to make trailers too.
Usually you have a trailer dude for that at a big studio, but as indies, usually the artist has to make the trailer.
So that takes up more time you would think.
On The Counting Kingdom, Jenna came to me and said, I need a trailer in two weeks.
And that was harrowing because I fully animated it for some reason.
But a lot of the art is also like putting together store assets like trailers and banners and promotional stuff.
So sort of in conclusion, indie artists, they fill many roles.
They do lots of things.
At a big studio, you have these very specific jobs that everyone is assigned.
They go in the morning and they know, I'm going to be the guy who animates shattering glass all day.
And they do it for three years and the Avengers comes out.
But at a smaller studio, you know, all right, I'm doing the concept art, at the same time, learning how to model it, at the same time, I'm figuring out how the items in the game are going to work, at the same time, making the logo and maybe some early illustration promotional arts.
And you have to like balance a lot of these things.
So is this is me in my zen drawing state, I feel like.
I don't usually get to that.
I usually get really angry and I don't get this calm when it comes to that.
But yeah, indie artists, they have so much stuff to do, they have so much stuff to offer.
Hire artists.
Don't make your game without an artist.
That's boring.
If you think you can do it yourself, that's great, but artists, they add a lot.
They inspire you.
They usually work hard if you find the right one.
And you can form long-term meaningful partnerships with them and it's great.
Thank you.
There's my email address and my website, if anybody cares.
[APPLAUSE] Cool.
44 minutes.
It was close.
SPEAKER 1: Excellent.
Do you got time for questions?
Sure.
Nothing?
Really?
[INAUDIBLE]
Actually, quick question
There you go
Do you ever work on multiple projects [INAUDIBLE]
I usually work on multiple projects at the same time.
I guess that was a good point.
I need to bring up-- that was a good question.
As a indie artist, I'm not getting paid a lot of money.
End of story.
It doesn't happen.
So usually I have to work on more than one project at the same time just to let ends meet.
Usually the work isn't full-time because either the company can't afford to have it full-time, or the amount of work they need isn't full-time.
You have to juggle a lot as an artist.
I'm working on a couple projects, doing contract work, sort of freelance illustration on the side.
I'm also teaching a class.
And you have to juggle all these things at once to survive, basically.
Because as sad as it sounds, indie games don't make a lot of money usually, unless [?
you're at ?]
Goat Simulator.
Yes, go!
I didn't quite catch what you said, 2D animation is harder than 3D animation
Have you ever seen an old Disney movie?
They usually had armies of what they called in-betweeners, in that there would be a key frame artist.
He would draw Snow White dancing like this.
And then there would be the poor bastards who had to draw every single frame of her going like this, all 30 or 40 frames.
So in a game, if you want hand-drawn animations that look really interesting and nice, you have to get someone who will bother.
Like, Street Fighter III is a great exam-- or King of Fighters-- those games take forever to animate, because they are done in that Disney style, in terms of just frame by frame by frame by frame by frame.
But the other thing they have to worry about is to make sure that the hit points are in the right place, that the collision boxes are in the right place.
It takes a lot of time to do that.
That's why when I mentioned tweening as a quick way of trying to mimic that, it doesn't look as good usually.
And in a lot of games that use a lot of tweening, like Limbo, for example, they hide it with silhouettes.
Because you get these weird seams and it doesn't look as good.
There are obviously tools that help with this like Flash animation.
You can sort of do an in-between of tweening and hand animation.
But sort of frame by frame, sprite animation, takes a long time.
And 3D-- I mean, it's easier, because it's all tweening usually.
Unless you have some character that transforms all the time.
It's the same way in 2D, in that a 3D arm that is rigged like this is always going to look like this because you can just sort of manipulate it in 3D space.
Doing this in 3D is really easy.
Doing this in 2D takes forever, if you know what I mean.
That answer your question?
Kind of
Is there something you need me to follow up on?
I kind of see what you mean, like you have to draw every frame
Yeah
Unity has a very cool way to go around this.
Like, [INAUDIBLE] that's actually 3D that is snap-shot into 2D, so they don't have to worry--
Yeah, that's one way to do it.
That's one way to solve it, but then you have to make all your models in 3D first, and then animate in 3D, which the modeling process takes time as well.
And then you have to think about-- all right, I have 2D artists.
Can I do this?
No, I need a 3D artist to do this.
It's just something to think about.
But yeah, that's a way around it that a lot of people use.
Yes, go
Does it make sense for a software engineer to learn the basics of drawing, so that they better understand maybe the artists and give more reasonable tasks?
Sorry, could you repeat the question?
So does it make sense for a software engineer to learn the basics of drawing, so that you better understand artists and maybe give more reasonable tasks?
Yes.
Programmers, designers, software engineers should totally try to understand the basics of art and color theory and the process of making art, so when it comes time to either advising the artist on what they need to change or what might not fit, you have some understanding of all right, these colors go together.
These colors don't go together.
Could you composition this a little better, so you like put these down here and that over there.
But again, a lot of art direction is about contrasting, and composition, and color is a big one.
I didn't really talk about color, color theory, too much.
Like for instance, Jenna in Little Worlds Interactive, when I worked with her, she already had a pretty good graphic design sense, which really helped in, when I gave her art, and the sort of feedback she could give me.
And also when she utilized my art.
So I didn't put anything into that game.
I wasn't directly working in Unity on The Counting Kingdom.
She knew how things look good together, just because she'd already had like a background in web design.
So yes.
Yes?
You were saying earlier that usually it's good to have-- or it's basically a must to have-- different people specialized in different roles.
Like, someone is the game designer, someone's a programmer, and then you have to find artists to do all the game art.
Do you ever find outliers to that?
Like, exceptions?
Like, someone who does everything and still comes up [INAUDIBLE]?
Of course, there's always exceptions.
There's always some magic man who can do it all, or magic woman
[INAUDIBLE]?
An example of an artist who can do it all
Like, a game
A game designer who can do it all-- I don't know.
Can you name anyone?
Other than [?
Dwarf Fortress?
?]
[?
Dwarf Fortress, ?]
yes.
I mean, a lot of game designers-- especially if they do it as a hobby-- they make their own stuff.
They make their own art.
It's just a good idea to spread the work out, because as an indie-- and obviously, a lot of my developer friends-- they have to focus so much of their time on marketing, and programming, and designing, that, to also add art to that, you've just extended your game development cycle by a lot.
And maybe you're not as competent in that.
And people are more apt at telling bad art than they are from telling bad game design.
Because all you need to be able to see is just a picture of the game or video from the game and go, that doesn't look very good.
Even if the viewer doesn't understand why it doesn't look good, they can tell that immediately.
I don't like that.
So if the designer, programmer is confident enough, yeah, more power to them.
But you're going to be putting a lot to work on yourself.
And you're not going to get your game done in a timely fashion at all.
And it probably won't be the best you can do
Can I ask one more question?
Sure
Do you ever as an artist think about designing a video game?
Do you ever think about making [INAUDIBLE] video game?
Like, [INAUDIBLE]?
So I work with a few developers now.
And I'm always giving my design ideas.
I always have ideas for games.
I love coming up with ideas for games.
I don't think I would call myself apt enough to design the whole game by myself yet, so I'm definitely still focused on the art side of things.
But yeah, I love designing games.
It's fun.
Yes?
What have you had to do for an art test?
So usually for an art test, it depends on the game.
So I recently did an art test for some sort of mobile MOBA game, and what they asked-- they said here are your character descriptions, design these characters.
And I said, all right, that'll take me this amount of time.
They said, great, that's fine.
This is my rate.
It was a quick contract.
Out the door for that.
And I designed the characters.
I didn't end up getting the work, but I don't think my style was really what they were looking for.
But yeah, it was that.
I designed the characters.
I did a quick turnaround.
So it was like, characters standing this way, this way, this way, this way, from the back, of each character.
I fully painted one of them, and said, this is what I got.
I think that's a pretty good art test, especially if you're building a game focused on characters.
If you're building a game focused environments be like, draw this mountain for me that I'm describing in this text.
So usually an art test is here's a description, draw this, especially for an illustrator or a concept artist.
It's not so much here's a quick doodle I did, make it better.
It's usually here is a paragraph of text, make this thing exist.
I feel like that's a pretty solid art test.
Yes?
You talked about finding someone who has the style that you're looking for?
Yes
Is that generally the case with artists, or are there artists who are more flexible in style?
So I like to think I'm more flexible.
I didn't used to work on a lot of cute games, but now I do.
That's just what happened.
There are definitely artists who more-- so I'm going to go back to art school here a little bit.
This is time travel.
And I'm going to remind myself of what my Illustrator professors told me, is to never do anything else other than your style.
Get really good at that one thing.
And don't bother with anything else, because you need to be the best at what you do.
And I was like, eh, that's boring.
Most artists have their style they work in.
A lot of video game artists, they go for the hyper-realism concept art-- the art movie, books, that you see from VFX houses.
That's why a lot of them can work together in big groups, because they all have a very similar kind of like, we're going for realism, robots, tanks.
OK, we can all work together.
The more unique your style is, I think, the more time you have to spend working on that style.
And the more reason, if this person's style is very unique, you're hiring them for their style, not their art talents.
Again, that's why a lot of big studios, they hire people who can do realism, because that's a little bit more malleable in terms of what they can turn that into.
But if you're hiring artists for their very specific, unique style, you're getting them for that one task.
But again, it depends on the contract for me.
If I'm-- all right, this is for children's magazine.
I'm not going to put the bad guy from Doom in there, obviously.
That doesn't make sense.
As an artist, I say yes to almost every job that comes my way, because I don't know when the next one's going to come.
That's just how it works.
But that's how I work.
A lot of other people are different.
All right, is that it?
Thank you very much.
It was awesome.
[APPLAUSE] SPEAKER 1: All right, so take five minutes to get with your teams.
If you'd like to have them take a look at your work, just have your computer set aside, close to the edge so they can easily see it.
And at 2:10, we'll walk around and see who wants some feedback on their games from Luigi
Cool.
[CHATTER]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Otherwise class announcements.
Sprint task lists due today, hopefully you all turned them in.
Paulina's going to check that out and make sure that each team has a new sprint task list turned in.
We're asking for one of those once a week because we want to know the state of your game as it is right now.
A lot of it is for our use to track where your game is going and how your game is going.
The actual format you're using isn't as important to us for this last project.
Again, if you're using a non-standard format, once we get to the next product backlog presentation-- which I believe is next week, I could be wrong about that, it's in Stellar-- we'll be asking about your project management process, what you've been using, how it's been working, what changes you've made, stuff like that to know how you're tracking what you're doing and when you're doing it.
Yeah, so today.
I've got another minute.
Today we're going to talk about story in video games, then you have time to work on your class, and then at 3:00 PM Ed Barrett's class is coming in.
We've got 18 folks coming in to play test your games.
Remember, we did require you to have a playable digital build today.
Please test that build.
If that bill does not get you the test data you need, also test another one.
So if you have a paper prototype and you want to test that, you can test that concurrently.
Again, just like before, having three workstations open at each team would be awesome.
It means everybody gets to play the games quickly and get out quickly so that you can then finish up working.
You don't need to test each other's games.
So actually allow our guests to test games first before you start testing each other's games today.
Any questions about procedures and whatnot?
All right, so we're going to talk about stories in games.
This is a lecture that I've adapted from a previous colleague of ours, Claire Fernandez Vera.
She studies adventure games.
She studies games through the lens of theater, so she's got a really interesting view on play as performance.
This is much more of a talk about what is usable for you right now in the games you're making in this class.
I'll start out with talking a little bit about bigger games, but then I'm going to move quickly into talking about things are more useful for you all right now.
Just a question for you.
Do games tell stories?
Yes?
I see you nodded.
Why?
How?
Obviously there are games that are more narrative-heavy like with cutscenes and words and happening.
But I feel like you can also easily convey stories just with the images and the way the player interacts with the game itself.
You kind of create a narrative then
Great, you did my lecture.
I'm done
Yeah
Thank you.
Any other ways games tell stories?
We heard cutscenes, we heard narrative, we heard words, we heard pictures.
Yeah?
One more hand?
Where?
Game books also tell stories
Yes.
Game books, the physical things around games
It's also really rare and hard to do, but it's really great when games can tell the story through the mechanics
Yes, absolutely, telling the story through the mechanics.
Which game did you mention?
Papers, Please?
I think so
Excellent.
So when we talk about games and stories, sometimes we think about these.
Actually, you're good, you mentioned some other types of games.
Final Fantasy 13, Metal Gear Solid 4, both games kind of derided for, in certain circles, the linearness of their games.
You start the game, you watch the movie, you hit some buttons, you watch the next movie, you hit some buttons.
Some people like that, that's one way to play games.
That's one kind of game that is out there.
Not exactly the kind of games that we're making in this class.
But also not exactly the kind of games-- not the only way to tell stories in games, as you just mentioned.
To make those kind of games, we're calling them AAA games if anybody's heard that phrase before.
AAA just meaning expensive.
Blockbuster.
The Marvel blockbuster movies are basically what's happened to film, Call of Duty is the game version of that.
So they require many expensive resources, artists, sound designers.
We heard an artist talk earlier this week about how art's important and why art is important, and how difficult it is for one person to do all those different kinds of tasks that are required for a complicated game.
Just to note that those two games that I mentioned previously usually borrow techniques from cinema.
So things like camera placement, lighting, visual effects and sound effects, a lot of those things are their borrowing concepts from cinema that have been proven to work in cinema, and then just throwing them in the game because they know they work.
One way a AAA studio might think about making a game with story is, all right, so it's this balance between interactivity and narrative, right?
And there's this cost, so think of that triangle as the cost.
It costs money to make these games, and that money is generally either spent on people or it's spent on time, so people times time, basically.
And the idea is that, all right, so we've got this interactive stuff going on right here, the player presses buttons, great.
We've got this narrative stuff here and I'm borrowing comedy and tragedy.
When I say narrative, think back to your Aristotle.
So plot, comedy, tragedy, melodrama.
A lot of the AAA games tend to just basically be big melodrama pieces.
But it could be folk tales.
That narrative could be-- sometimes it has conflict, sometimes there's not conflict.
Sometimes there's a dramatic arc or not.
So they say, all right, so we've got to balance the two.
And how do we do that?
State of the art AAA 2011, this is Arkham City, that's Batman, his parents are dead.
Press X to pay your respects.
You press the X button, he places down the rose, you get a nice little cutscene reward with it so you can see a little bit of a movie going on with him being grim and putting down the rose.
But you also get a little achievement, so it's a reward, it's an Easter egg.
But it's a crucial back story to the character, they put it in the game.
They spend a lot of money making that animation.
Not all players are going to see it, and when they do see it, it only lasts for a very short period of time.
That was 2011.
It's 2014, we've got really cool computers, we've got Xbox One, PS4.
Your state of the art, AAA, 2014 Call of Duty: Advanced Warfare.
We've got an upgrade.
Hold X to pay respects.
Basically, it's a quick time event.
We've seen quick time events with other games before, if anybody's played Heavy Rain, if anybody's played the Shenmue games, Resident Evil games, things like that.
Basically, you've got a video footage going on, and you can press a button and it'll give you a new set of video footage.
There's a lot of animation going on, there's a lot of cost going into this.
I think the hope for this kind of thing, why it's hold X instead of press X, is by holding X, you're embodying that action.
Maybe you're grieving, maybe it's praying, maybe it's holding the button.
Maybe the longer you hold it, the more you grieve.
So maybe there should have been a score that's counting down how much grief you got.
But if you've read the internet about the game, you notice that, yeah, it hasn't quite worked out the way that they might have thought it worked.
You don't have AAA resources, so why did I talk about all that?
And you should probably just give up on stories, right?
I'm going to say no, of course, that's why I'm talking about this for about another 45 minutes.
But some games don't need stories to be fun or have a meaning.
Anybody recognize the pieces on the screen?
Tetris.
The tetrominos of Tetris.
Is there a story to Tetris?
Some people say there is.
And I'm talking about this Tetris too, there's not even color in that.
You can't even tell what the tetrominos are once they hit the pile, they're just there.
And actually, that might be that-- I don't know, I think the bottom of that bottom row is the bottom of the bucket that's holding it.
So our own MIT's Media Lab Janet Murray would say that Tetris is the monotony of the work day, of putting things in their places.
Another professor, Jim G. over at University of Wisconsin Madison, he says Tetris is an escape into the very desire for order, control, and workable solutions that we have all the time.
Some other less notable but equally valid people on the internet say it's the story of life, it's never won, it always has an ending, and that ending is your death or everything just piling up to the top.
Because it was made in Russia, some people might say that it's a critique of communism, maybe, or of sameness, or of bureaucracy.
Although the creator, the Russian creator Alexey Pajitnov would say-- he does say-- that for him, it's about playing on the edge of your abilities and about playing to a life well lived.
So the point of all this stuff is people read patterns in the most abstract things.
So maybe you don't need story, maybe you just give them some abstractness and then move on.
Tetris was made in 1981 or thereabouts, on that one, 1983?
I should have put that in the slide.
So that was 30 something odd years ago.
What if they made Tetris today?
Here's Tetris today, or one version of Tetris today.
Why would somebody make a Tetris game with story?
This is the Japanese only Puyo Puyo Tetris.
It's taking two different game mechanics and splicing them together.
Why would they put characters in the Tetris game?
What's up?
[INAUDIBLE]
It might, yeah.
And people are used to seeing characters
[INAUDIBLE]
Possibly, yeah.
Any others?
Any other ideas?
Yeah?
Tetris already exists, so they need to make it at least slightly different
Absolutely, yeah.
Actually, we were just talking about this before class.
Would you be able to sell this today?
Do you think this could sell today?
You've never seen Tetris before, it's a brand new thing, you'd look it up.
Maybe it would, maybe it wouldn't.
Has anybody played the game Hexagon or Super Hexagon?
Yes
Does Hexagon or Super Hexagon have a story?
No
OK. Then why is there a narrator and a voice?
Why is she telling you--
It sounds like-- I feel like it's partially to let the player know where they are, maybe?
How does she talk?
Kind of robotic?
It's robotic, but also like a woman's voice
There's a little bit of story.
Why is the character a triangle?
Why is the character a triangle?
The thing that you're moving on a screen.
Why a triangle and not a square?
Or circle?
[INAUDIBLE]
Maybe
It shows you--
I'm going to argue that there's a little bit of--
Triangles are non-deformable
What's that?
They're more sturdy than squares.
If you squish a square it turns into like a weird slanty square
There you go
Rhombus
There's little hints of story there
The character is the triangle.
I think
Yeah?
One other reason for characters is merch
Merchandise.
Actually, yeah, especially with this one.
What do they call the characters when they make-- like, Internet Explorer as a character.
Is it [INAUDIBLE], or is it a tomo?
It's like something tan, so like Internet Explorer tan
Something tan, yeah.
So L-tan and other L-tan, maybe.
So that's story.
That's reason is the story that historians of a game play purpose I would argue that Pac-Man has a story.
A noticeable story, right?
There's cutscenes in this Pac-Man.
But what's the story of Pac-Man?
Where are the hints of story there?
I guess it could be in how the enemies have names?
Absolutely.
And why did they give the characters names?
Why is Blinky Blinky, and his character, his personality is shadow?
I think as far as I remember, that's really-- the English translation is a really poor translation of it.
In the original, it's very clear as to their names gives a hint as to how they will act
Absolutely.
They have deterministic actions.
So I've given them a little bit of character, but that's going to help you to identify one before the other.
If anybody's played the Atari 2600 version of Pac-Man, my first experience with it, all the ghosts had the same image.
You couldn't tell which one had which type of action.
They probably actually didn't have different AI.
It was a really difficult game for them to make.
They had to make it in two or three weeks, I believe.
What do we gain from story?
This is where I come in and say, this is what I think story is going to give to your games.
Even if you're making a strategy game, it doesn't matter what kind of game you make, you're going to get something out of a story.
We understand our lived experiences as stories, so providing a story is going to help a player understand what's going on in the game.
If you think about what you've ever done in the past, you're always telling it to another person as a story.
The way you are able to memorize things, often, is you're constructing stories in your head of things that you've done.
It's events with causes.
You're placing together, you're finding patterns, and you're putting the story in there.
It helps you understand what's going on around you.
A story can help you explain the world to the player, it can also encourage the player to explore it.
So if you've got something in the game that's hidden later on, you want them to find it, you can use story to give them hints towards, there's this cool thing coming up that you might want to check out.
On a parallel level to explanation, stories provide consistency to the world.
So if you've got various things going on within your game, the way you can kind of massage the differences between things going on, between the mechanics, is by covering that up with story, or rather, embellishing and supporting it with story rather than covering it up.
So a couple quick videos.
When I'm talking about story games, especially in this lecture, I'm mainly talking about games with story, not story-driven games.
The games we talked about earlier, those were story-driven games.
This is a game with a story.
[VIDEO PLAYBACK] -The heroes finally meet under the star of destiny
That's Sophitia, in the white we've got Cassandra.
-I'm sorry, but I can't back down.
-What do you want?
-Battle one, fight
Have you heard of this game?
Soul Calibur, yeah.
I think it's 3.
2008 or 2010, I think it's from.
Did anyone catch what she said?
Don't try to confuse me
Don't try to confuse me.
So KO, you basically have-- -You win.
-God, please forgive me
God, please forgive me, what is she talking about?
[END PLAYBACK]
Does anybody know what what she was saying there and why she was saying it?
You might only know this if you've played the game before and you're really, really into the game.
So Matt?
Yeah.
So Cassandra is Sophitia's sister, and they're fighting for the Soul Edge, for this demonic sword that's going to destroy the world.
And she has to fight her sister and kill her sister to get to the sword.
I think that for Sophitia's version it's to save the world.
Why didn't she work with her sister?
It's a fighting game, who knows?
I meant to update the slides to something a more modern audience might understand.
In particular, League of Legends does this kind of thing a lot, right.
There's story moments going on in games.
You had mentioned a moment in the game
Certain characters on the same team, I believe, they have different voice prompts or different vocal lines that they would say to acknowledge that they have a history
Does that help you play the game Better
No, no gameplay mechanics
No.
It's a valid use of story, but it's not an efficient use of story, especially for your kind of games, right?
It doesn't directly connect.
The story doesn't directly connect to the player's experience of the game, but it is something players like.
Like we talked about with characters, like we talked with merchandise.
And in particular, in particular for this game, for Soul Calibur, why is Soul Calibur doing this but Street Fighter isn't doing this?
Soul Calibur stands apart from the other fighting games by having these complex story lines that are actually visible to the player within the games.
Street Fighter does have complex storylines, but they're largely fan service or outside of the game, merchandise, books, things like that.
You maybe see a little cutscene or two at the end of the game, but not while you're playing it.
And also, to really enhance the single player experience of the game.
That's actually how I know of the Soul Calibur series, I played it single player.
Because they added an RPG to it that was really basic and stupid, but it made the story a little bit more concrete.
They took the RPG out, that's when I stopped playing the game.
Because that story didn't have any-- there's no reason for that story in the gameplay anymore.
So I was asking you about games telling stories and how do games tell stories.
What I'm going to say now is that technically, games don't exactly tell stories, if you think about it.
Like, who is the storyteller in a game?
You.
So it's not storytelling, it's story building.
The game, the designer, the developer is giving you the tools to build a story of your own.
Why we say they're different, so just to compare and contrast, storytelling is continuous.
There's a one way communication, it is from auteur and author to the player.
The order and, more in particular, the order of events and disclosure of information is determined by the author of the game.
I'm going to show you this, and I'm going to show you this, and I'm going to show you that.
Story building is more fragmented.
It could be as simple as I'm going to show you this, now you can go see all these other things, but I'm going to pull you back and I'm going to show you this next thing, and you have to see those two things in order.
In a more open oriented game, you can see everything at any point of time depending on when you like it.
Rather than a one way communication, there's collaboration between narrative design and the players.
Anybody heard the term narrative design before?
Or the job title narrative designer?
What do you think that is in a company?
And let's say Rational.
Rational has narrative designers, she was actually a CMS student for Infinite.
What would a narrative designer do in Bioshock Infinite?
Probably write all the [INAUDIBLE] and stuff, and all the little pops that you hear?
And what's going on?
A little bit.
I think Ken wrote a lot of that, Ken the director wrote a lot of that.
They touch that, though.
Yeah?
Think of how the story motivates the player and ties into the actual gameplay.
So like, Bioshock, you're told to do this and the reason is the story.
I guess figuring out how that fits in and how the mechanics will be fit into
Yep, they touch that, too.
The other thing they do is, you've got a level designer, and you've got a writer.
And right in between, you've got the narrative designer.
In some companies, this isn't for all companies.
This is just for some.
In particular, this is how I understand how Rational works, right, is the level designer creates a level based on how a person's going to play it, how they're going to get through it, how they're going to get from point A to point B.
Writer says, here's the story I need everybody to see, here's all the elements I need them to see.
Narrative designer figures out where within the level those things are going to exist.
And we'll talk about that when we're talking about environmental design.
I'll point out a couple places where a narrative designer had an active role in that game.
All right, so I'm going to say that we don't need expensive cut scenes to tell stories.
We've come up with a few, we've kind of touched on this a little bit.
And instead, what I want to say is, let's reduce the cost of the game, but let's keep interactivity and narrative.
So things a player does and the story that the player experiences are about similar.
So what tools are we going to give the player to build their story?
And where we give them the tools depends on the type of story we're giving them.
So there's two stories to games that the player builds.
There's the story of the world, which largely is seen and heard, and it's interpreted by the player.
And there's the story of the player, the actions that the player does, and why I say told, I kind of mean built as well.
Told is in that the player is telling it, not the-- so the world might be something that the game is telling the player, but the story of the player is something that the player is telling back to the game, or back to their friends, or in their own head.
So story of the world, it's the world's history, right.
It's the events that happen in the fictional world before the game starts.
You've got a lot of control over all of this.
As a developer, you're writing all of this.
For the player, they're discovering it.
They're discovering it piecemeal, they're discovering it through the game play.
So that story, with a great world, that story becomes a puzzle for the player to want to explore.
It can be as literal as in Myst, where it literally is a puzzle that you were going around and solving a puzzle here and solving a puzzle there, getting little snippets of background and putting them together in your head to come up with a whole view of what the world is.
Or it could be things like I'm going to show you with environmental.
Yeah?
Ideally, too, if your story is becoming a puzzle, it would be really, really cool if it was useful to have solved it, right?
So maybe as a player, if you can figure out something cool about the world that can be useful to you back in your game again, that would be really great
So the tools are giving the player to do that, to do exactly that.
And these are three tools that you have at your disposal in your games right now.
They're really simple and easy to use.
At least the top two are.
Title, character design, and your environmental design.
So we're going to go through about four tiles.
I'm going to give you the name of the game.
I'm cheating a little bit because I'm actually giving you the cover art for the game.
I'd like to know what the story is for the game.
So, Zombies Ate My Neighbors.
What is the story of Zombies Ate My Neighbors?
It's about zombies.
What are the zombies doing?
Eating neighbors
They're eating neighbors.
Where does it take place?
In a neighborhood
In a neighborhood, all right.
Anything else?
They're coming for you
They're coming for you.
Damn right.
Yeah?
In the back?
It appears that it's in a kind of campy style
Absolutely
Maybe like '60s-ish?
Mhm.
You get the kind of distressed woman in the front.
She's being chased by very slow, they're slow zombies, right?
These days we have to kind of specify.
Back in the day they were always slow.
All right, next one.
The Earth Dies Screaming.
What's it about?
Earth dying?
Space
What's that?
Space.
Anything else?
What's that?
Aliens
War.
Did I hear aliens?
Something?
Yeah.
Anyone recognize the title?
Was it a movie?
Yeah.
Do you think the game is based on the movie?
Yeah
It actually isn't.
So the movie's from '55, '57?
I don't even remember if it's an alien invasion movie.
It's just one of those things where in the '50s, you made a movie and then somebody came and put a title on it based on what titles were working that day.
But they licensed it from 20th Century Fox.
But there's these spaceships, these very '80s looking spaceships.
This is actually a cartridge for the 2600.
Looks awful.
But yeah, The Earth Dies Screaming.
What happens at the end of the game?
The earth dies screaming
Yeah.
Arcade games, back in the day.
How did they end?
They always ended with you dying.
It was always about throwing another quarter, playing it some more, getting more time.
That actually came back over to the consoles, especially in the 2600 days.
We've moved beyond that, for the most part.
In some cases, it's actually brought back in iOS.
But yeah, Earth Dies Screaming.
All right, next one, this is harder.
A Mind Forever Voyaging.
Ooh.
What's going on there?
Discovery
Discovery, yeah
So we're allowed to look at the images and make something up
Absolutely
So you're a guy, you imagine all these things, and your brain interfaces with some kind [INAUDIBLE], so you can see that sort of imaging.
So maybe you're exploring all of these virtual worlds in your head.
And maybe--
Have you played this?
I just looked at the image.
The title image, and that really helps me--
It does help a lot.
But yeah, that's exactly right.
Except you're wrong, you're not a guy.
Right?
You're not a guy.
You're an AI.
Right, yeah.
I probably spoiled it for you, it's OK.
It's an Infocom game, it's a text adventure from 1988.
So you're exploring, text adventures were open door, go right, open mailbox, die from grue.
And in this case, you're going to all these different worlds, and it's the mind.
So it had to do with the AI attaining sentience and figuring out what was going on, and what was around it, and what the world was like at the time.
Really good game, cool, little bit of tidbit of info for these kind of games.
They came with these things called feelies, so elements, like physical elements, that were shipped with the game box that kind of added a little bit to the game.
Because they didn't have graphics to show you everything.
So in this case it was a file folder, a pen, I think it was related to a company that you thought you worked for.
Here's a more recent game.
868-HACK.
What's it about?
What is 868-HACK?
A phone number
Yes, it's a phone number.
There's no area code, right?
Normally we'd have an area code there.
So it's kind of referencing back to maybe the '80s?
Are you a hacker?
Maybe.
Probably.
Anything else we can get from the title?
We're going to come back, I'm going to use this example again, because I love this game.
So titles provide clues towards setting, character, action.
Even better, while you're thinking of your title, think about what it's like on Google.
Think about marketing right now if you're making a game that's going to be marketed to somewhere.
Can you Google the title and come up with your game?
If you can, you win.
It's very hard for some of the games, unless you add a ton of money into it.
Next up is character design.
Who's that?
Sonic
Sonic.
What does Sonic do?
Runs
He runs fast.
What's he doing here?
Stopping
Waiting
Yeah, why?
Because you're not running
Exactly right.
Wow, I could have written this out.
Yeah, he's pissed.
He's like, what the hell?
Make me move.
You're not moving him on the controller, this is his idle animation.
So a ton of expressiveness in this idle animation.
It's maybe three or four frames of animation of just him tapping his feet, looking at you, breaking the fourth wall.
Some really great stuff you can do with character design in that way.
Yeah, like why do we know he runs?
If this were like the first time we saw this and this was Christmas Day, back in the time when we didn't find out about games three years before they came out.
You get this spiky-haired dude with big feet, right?
He's fast, he runs.
So yeah.
Back to 868-HACK.
As I said, I love this game.
Where is the player on the screen?
What element of the screen is the player?
The tile
Yes, which one?
Left
Yeah, the top one, right?
What are you?
What is it doing there?
How do you think it interacts within the system?
Yeah?
Walk down and [INAUDIBLE]
Basically, yeah.
Do you see, you see, I kind of said.
There's other smiley faces there, right?
What do they do?
Chase the [INAUDIBLE]
OK, so--
[INAUDIBLE]
Yeah, so there's the purple one, the magenta one, the purple one, the red one.
Those are bad guys coming to get you.
There's these little ones in the bottom left and right.
There's a couple on the top row that say data siphon.
That's something that you leave behind.
So you're leaving behind a piece of yourself in order to claim space.
It's a territory control game where you drop a piece of yourself to consume the things around it.
You can consume the programs there and you can use the programs there.
But I really like it for this.
Why do you think I like it for this?
When I'm talking about hackers, when I'm talking with a smiley face, what does that connote to you?
What did I hear?
It's fun
It's fun, yeah.
Yeah, Matt?
You're identity-less
Your identity-less, right?
You're anonymous.
I just kind of like that.
All right, so moving on.
Character design provides clues towards your actions and your verbs, although in that last example, it was actually not very good at that part.
But it does give clues towards your background, who you are, what you are, why you are, and then the feel.
What's the feel of that game supposed to be.
What's the feel you're supposed to have when you're playing the game?
Question?
OK, cool.
Environmental design is that third thing I talked about.
This is Portal, this is when you first start Portal.
It's very clean, it's very spacious, except for that little box that you're stuck in.
You eventually get out.
But something happens and you see something later in the game.
Does anybody remember where this is?
Oh, not played it?
I'm like in the middle of playing it
Which level?
Like three or four, or five or six?
Have you seen a turret yet?
Have I seen a turret?
Oh, no
OK, all right.
Close your eyes, shut your ears.
It's 2014, come on.
Yeah, so this is the cake is a lie.
It's later in the game.
It's actually in a part of the game where everything still looks really clean, that's as far as I'm going to go.
Somebody left stuff back there.
This is actually something that not a lot of players saw when it first came out and then a lot of other players told each other about it later on.
So it's also a game event, which I'm going to talk about later when we're talking about the story of the player.
But this is where level design and narrative design meet.
So level is created, there's a story moment that had to be placed somewhere, the narrative designer placed it in a spot that was hidden, that was hard to get to, but still reachable.
And when we talk about environmental storytelling, we often talk about-- there's been some articles about bad graffiti in games, and just how trite this has been used.
We're trying to find new ways of doing the same thing.
But it's basically, there was someone in there before you, is what happened there.
That's as far as I'm going to go.
Environment provides clues towards what happened in the past, but also what's going to happen in the future.
And it provides clues towards the feel, so again, what's this game supposed to feel like?
All right, so moving on.
We've got the story of the player.
So there's the embedded story of the player.
And what I mean by that is that's the story of the player that's pre-established by the game.
Has everybody played Ocarina of Time, or heard of it, or seen it?
OK.
So you're Link, you start out not knowing a lot about the world, and slowly the game lets you know more about what happened in the world behind you.
You don't get to choose to have this friend, I don't think, right?
What's her name, Saria?
The game gave you this friend.
And actually, later on, the other friends you get, the game's going to give you those, too.
You don't get to choose to have them or not.
She gives you this ocarina.
The ocarina is a very important aspect of the game.
So in this case, it's this embedded story of here's this object that you're going to use.
It's going to be incredibly useful for you later in the game, to the point where you're going to get really annoyed about having to bring it up all the time and play the same song all the time.
But it's also part of the character design.
It's giving you what your capabilities, like what are you capable of doing?
In this case, you're capable of playing songs on the ocarina that do magical effects within the game.
There's also emergent, and this is the cooler thing that I want to spend more time on.
This is the result of the player interacting with the system.
In this case, it's Sims 3.
And much of the systems of, at least, Sims 1, and I think still in Sims 3, the entire game is basically based on Maslow's hierarchy of needs.
So I'll bring up the slide, I think I bring that up in a later version.
Basically, it's a sandbox game.
There are things you can do in the game, and the story that's being told is the events and things that happen to the characters based on changes you make in the environment.
You're not to get the same story over and over again.
You're going to get new stories.
Some big.
Like in this case, the entire game is that kind of thing.
In other games it's more of a smaller aspect of the game, it's an added bonus to the game.
The story of the player in either of those cases is usually told through three things, and these are three things that you can use in your games right now.
Game premise, or the goal of the game, is your story, right.
Why are you going, where are you going.
Game events are things that happen within the game, either predetermined or emergent.
And then micronarratives, which I'll explain what that is on at the end.
Here is a premise.
Again, we're at MIT so we talk about hacking, right.
[VIDEO PLAYBACK] -A stroke of genius.
A true original.
From Introversion Software, the award-winning makers of Darwinia, a futuristic, high-tech computer crime game for the hacker elite.
Uplink.
You're an Uplink agent, a freelance hacker.
Your clients, global multinationals.
You hack into rival computer systems, steal research data, sabotage other companies, launder money, erase the evidence, even frame innocent people.
And all from the comfort of your own home.
In Uplink, you are the hacker elite.
Find more dangerous and profitable missions, change peoples' academic, and even criminal, records, and then siphon money from their bank account into yours.
And why not construct the most deadly computer virus ever designed.
But watch out.
When you're online, you're being traced.
Uplink.
The game for the corrupted heart, where trust is a weakness.
PC Gamer-- [END PLAYBACK]
Yeah, so.
Threw a lot of words at you, but basically every single thing inside of that game is something you can do in the game.
You can hack another person, you can create a virus, you can download their personal records, you can change their personal records.
Every single one of those things mentioned in the premise, in the trailer, is a mechanic that is usable within the game in some form.
A lot of it is a user interface.
It kind of looks like a futuristic computer terminal or an interface terminal.
Made by the same people, if you remember, we showed DEFCON earlier in the semester, same folks.
They're really good at that kind of atmosphere, building atmosphere
Futuristic for the '90s
Yeah, it's still futuristic.
So game premise, that's an easy one to get across.
Basically, it's just, think about all the things that you do in the game.
All those things, when you put them together one by one, that is the story that the player is going to be telling when they play your game.
Think of it, in that case, what they're doing is they're using a genre convention and they're using some tropes that you might already know.
Tropes can be useful, tropes can be baggage.
Tropes are usually baggage.
But genre conventions are great, especially when it's just, I want to get this theme across to you, I want to get this hard punch of you know exactly what you need to do.
I don't need to explain the game to you because you know what a hacker is supposed to do.
That's where those things are incredibly useful.
Give me my slides.
Game events.
Has anybody played Final Fantasy 4?
Or 2, if you played the original, or the US version, I mean?
So there's a character called Rydia the Summoner, that is not what she looked like in the game.
That was the illustration that she looked like.
She looked like a very small green haired person.
She is a summoner, she has magic powers.
What kind of magic powers are common in the Final Fantasy series?
What's the three basics?
Fire, ice and thunder
Yeah.
Or blue or, yeah, fire, ice, and thunder.
Exactly right.
So she's a summoner but she has other black mage powers.
Pretty common, most characters would have those three powers.
Except for her.
She hates fire.
You don't have fire for a good part of the game.
There's a moment in the game, you're at Mount Hobs, there's an ice mountain, you need to get through.
You can kind of see ice in the background there.
You need to get through the cave, and there's a story moment.
One of the characters, Cecil, on the left there says she's afraid of fire, because her village was destroyed in a fire.
And you see a snippet of this early in the game.
You get moments that she's got this backstory where traumatic past, it's preventing her from doing a thing.
Again, this is a trope that's pretty common.
After this point, they're all like yeah, you can do it, you're awesome, you're great.
And then she gets fire, now all of a sudden you've got the fire skill for this character for the rest of the game.
Any time you play the game after that and you're using fire, there's some part of you that might be remembering this story moment and how important the story moment was.
It's being linked back into the game mechanics really, really well.
But it always happens.
Every time you play the game, that's embedded.
That's always going to happen.
One more video.
Here is an emergent event.
Has anybody seen this video?
Everyone recognizes the game though, yeah?
[VIDEO PLAYBACK] -All right, guys.
And just for video here, I'm getting some mushrooms on hand to make indefinite fire
So indefinite fire, that's cool.
--[INAUDIBLE] or a castle.
[INAUDIBLE]
Here comes the room.
-The bedroom.
[INAUDIBLE] So let's go ahead and just make a fireplace.
They're already made it, as you see, is intact here.
That's the swimming pool.
So yeah.
Now all you need is some flint and a log, and just to chop down a tree to get a log.
Oh yeah, [INAUDIBLE] and to the fire.
And I always, or mostly-- uh oh
I knew that was going to happen.
-Crap
He gets some water to try it.
-Uh oh.
Sorry.
Sorry guys.
This was not supposed to happen.
And-- -Yeah.
What the hell just happened
Fire.
-You have got to be kidding me.
[END PLAYBACK]
There is a system within the game that fire burns pretty much anything it touches, that's wood, and it just travels, right.
You set this little system up, you use it the way you use it, and in this case, he tried to tell a story of building a fire, and it turned out he burned his house down.
You also see this Minecraft, the Creepers.
Everybody hears that hiss of a Creeper, especially after you've built something.
You've got this tension, this moment of panic.
That's all a memorable game event that you're going to tell another person after you've played the game.
And especially these days, you're going to record it onto YouTube and put it online.
This was, I think, from 2010, because it's an alpha build of Minecraft.
Continuing on the theme, here's the Sims.
Has anybody had this happen to them in The Sims?
They're just trying to have a party, they're trying to meet their Maslow's hierarchy of needs, right?
They're trying to love and belong to each other, and build some esteem, and self-actualize themselves, but they forgot about the safety part of it.
So the oven blew up and their fire happened, and they all burned and died.
Common trope with The Sims.
Another example I love.
Anybody played Far Cry 2?
Yeah?
So fire exists in Far Cry 2.
I love fire.
Far Cry 2 takes place in Africa.
You're a mercenary, you're going to find a warlord and kill him at the end of the game.
That's basically what you're trying to do.
You have different ways of doing that, different means of doing that.
You are in the savannah, it's very dry, you can burn the savannah, it's great.
And you can use that to get past machine gun emplacement, or set a trap for somebody.
In game character, not another PC, but in game.
In an early version of this, an early test, playtesters set fire to the savannah, and then 10, 15, 20 minutes later it said OK, you won the game.
What happened?
Fire spread.
The boss was set in the fire, in dry grass.
The fire spread all the way through the game, the many, many miles, square miles of gameplay, and it killed the boss and ended the game.
I really wish they had kept that in, or a version of that.
They ended up reducing the spread of fire because of that, they moved the character from it.
But that's a game event they could have capitalized on.
That's a piece of emergent storytelling that's going on that people tell the story to each other.
In the developer circles I heard about this in GDC the year after this came out.
That's a great one.
So that's game events.
Next up micronarratives.
These are similar, but different.
A micronarrative is basically a little bit of storytelling inside of the game that, again, really, really small.
So maybe it's an animation.
In this case, this is Jet Set Willy.
In Jet Set Willy, you are a rich guy.
You made all your money in the previous game by mining, and you came home, and you've invited a bunch of friends to your house and you had this awesome raucous party.
And you trashed the place.
Unfortunately, you rent.
You rent a mansion instead of buying a mansion.
And you're trying to go to bed, and your landlady is just like no, you don't get to go to bed, you don't get to sleep.
You've got to clean all this stuff up.
She doesn't say any of that, though.
This is all she does.
Just, she's standing in front of the bed, she's going that.
This is for the ZX Spectrum.
I think it had 64k of RAM, maybe.
You can look that up.
Maybe it's 8k, it's really small.
Really bad keys, really bad keyboard.
Not a lot of graphics.
That version is actually a later version, I couldn't find-- this is the closest thing we found for what it originally looked like.
You can really see, there's character design there, but it's also just saying no, you can't come back here
16k at launch, it did go up to 128
Now that I've spewed all this out, what can you do with all this?
So I'm actually going to show an example of a game created last semester by a team in this class for the final project.
They made a game called Sugar Rush.
What's the game about?
First, what's the title?
Well, I told the title.
Sugar Rush.
What's the game about?
[INAUDIBLE]
Yeah, candy, candy, candy, candy, candy.
Look at him and all that candy.
And it's a kid.
So title, character design, all working really well right there.
Here's the game premise.
One day, strange leafy beasts spawn from four mysterious portals, and they plundered the villages, terrorizing Candyland.
What do you think you're going to have to do?
Kill the vegetables.
And they're mean, because look at them.
So mean.
Now, when we say, we said don't use cutscenes, that doesn't mean we say don't use exposition.
A simple screen with a little bit instructions.
Basically, we told them, have an instruction screen.
This was part of their instruction screen.
They had the screen that says these are the keys, but also, this is what you do.
Really, really useful.
We want something that fulfills that for a game the way it's necessary for your game.
The difference between a game that has something like this and doesn't have something like this, is basically it's a level of polish.
A game that has a little bit of exposition like this feels a little bit more concrete.
It just feels like it's more complete, there's more thought put into it.
It just feels a little bit richer.
It's still playable without it, it doesn't need this screen, but this screen's really, really helpful.
And the game play itself.
It has mechanics that are consistent with the design.
You are a kid, you are fighting a tomato.
What do you fight with?
You have weapons, what are your weapons made out of?
Candy.
How do you heal yourself?
You eat candy.
How do you make new weapons?
You combine and craft candy.
Everything's revolving around that premise of candy is the world.
You are the lone human being in this world that we see.
And so it's not like Adventure Time, you're not harvesting candy from candy people.
The weird thing about it, I think you when you kill vegetables, you take the candy from them.
Maybe they're stealing your candy?
Yeah, you are stealing the candy, it says plundering.
A ha, they thought ahead.
And yeah, the health points.
What are the health points called, if you can read it?
Blood sugar
Yeah.
Really simple stuff.
Really, really simple.
Just theming it, thinking about what you call things, how you call things, where you place things, what verbs you're using.
All that ties into the game.
Here's another student made game, it was actually made by a larger team over the course of the summer.
So they actually had nine weeks to make the game.
And they had dedicated artists on this game.
But this is a game called Gumbeat which we have our website there.
You are a little girl going around chewing bubblegum in this idyllic land that, I don't think, there's an information sign up there that says keep the place clean.
Basically, gum's not allowed.
It's not clean.
But you go around, you recruit people.
You blow bubble gum in front of them, they say pop the bubble, you pop it, they're like sweet, I'm joining you.
And then they trail behind you.
And you see on the right there, a little guard.
Metal Gear Solid style, he's got his own line of sight, if he sees you, he's going to chase you down and hit you.
Early in the development, someone-- so if you pop the bubble on somebody, like on your friend, they'll get stuck with gum.
You pop the bubble on the guard, they'll get stuck with gum and they'll not be able to move.
But we found if you put the bubble gum on the guard, and you have a guard, basically a second guard looking for people, all the AI was was if gum, attack.
Guard with gum found the guard, attacked.
They didn't even think about this, this wasn't a feature they were planning, it just happened.
And it happened in the space of gameplay, play testing, just like that fire in Far Cry 2.
Except in this case, they're like, that's really freaking awesome.
We are going to polish the crap out of that and make that a really interesting feature in the game.
We're not going to tell the player how to do that, but we are going to create a level design that encourages the player to experiment with that.
So that's where you get that kind of game play.
So play testing is a way to discover emergent game events.
So that's why we're asking to play test your game, especially people who are not you, is to see what-- you've created this amazing possibility space, you can't explore the entire thing on your own.
They might see something and not knowing what the game is possible to do and what you should be doing in the game, they might come up with some cool things to do in the game.
So play testing is key.
Lastly, not a student game, but another favorite of mine, FTL.
There is a story to FTL that's really complicated, and I'm not asking you to try to emulate that sort of thing.
It's really hard.
It's a lot of detail and it's a lot of writing.
A lot of thought being put into this randomized event thing.
But there's a feature in the game that I'm showing here right now that is only used on this screen.
It's probably kind of-- it was probably a high price for them to put the feature in.
Can you-- maybe you can't read it.
Actually, you can't read it that well.
Basically, you can change the name of the characters.
This is something that strategy games have had for a long period of time.
I think we remember through XCOM back in '93, '94.
Earlier?
No, '94.
So you name your characters.
If you name your characters, you might be more inclined to make stories of their game play.
So I've got my ship as the good ship GameLab with my buddies Andrew, Philip, and Sarah, and we're going to be awesome and take down-- are you taking down the empire?
I can never remember.
Empire or rebel?
Are you a rebel or are you the empire?
You are a rebel
We're the rebel,
No, actually, I think you're--
Yeah, you're the empire.
You're trying to take down the rebels
You have to run from rebels
Yeah.
OK, cool.
So one thing that sometimes happens with this game, though, some players are going to optimize it.
What did they name their characters?
Pilot, Shields, Engines.
Who's dying?
Looks like the pilot's dying.
How do you know?
It's named Pilot.
I forget if you can-- I don't think you can rename characters when you bring them back on your ship.
I should looked that up.
If anybody plays, take a look at that.
But you can get new characters later on.
Whether or not you can name them or not, you might name them, you might not name them.
But either way, they've spent a lot of time and money putting this feature in, some players aren't going to use it.
So what do you do?
You blow shit up.
The game is basically about traveling the ship, things catch on fire, and hopefully, maybe they're not concerned about Pilot, but they are concerned about their ship.
So give them these cool little game events to keep them going
So one of the interesting things about this kind of story is that it's actually-- for all that you're talking about, it does take time and money to create it, but it's actually pretty cheap to be able to let players name their characters.
But it's even cheaper to do it, FTL did at the beginning, where they automatically name all their characters.
When you get a random human, the character isn't named human number 38, the character is named Fred or something.
And that was actually originally from the Kickstarter.
Backers got to have their names in, which is kind of cool.
But it's actually worth doing, because there are certain kinds of players will really get into those names in a way that you don't expect.
I've got two friends who really get into their games.
One, Marc LeBlanc, was playing XCOM.
Some of you have seen the more recent XCOM, I assume?
But maybe not the old one?
So you can name your characters or whatnot.
And it gives you some names that you can use.
He had went through the entire game, and this is back when games were hard, so 40 to 60 hours of gameplay, and he had a squad of super uber soldiers.
They all had names.
He didn't quite have personalities for them, but they were kind of like personalities.
And the second to final battle, one of his top A squad people got a broken leg or something.
And he was like, oh, that's really annoying, because I'm going to Mars tomorrow.
Well, all my guys that are high level are gone, but I've got these guy's really cool equipment.
Like he'd got the laser rifle of doom and the super uber armor, and so he went to his second string squad, the guys who were second level.
He was like you, suit up, son, you're going to Mars.
Which for him was a story, and it was all inside his head.
The game didn't make that.
The game did not make that story.
The designers could not have anticipated that story.
But in his head, because he's the kind of player who really tried to personify his characters a little bit, it was actually a really big thing for him.
Similarly, another one of my friends, who will probably come in for a lecture later, Laura Baldwin, was playing XCOM as well.
Basically the same game.
She, in her head, kept all the default names.
The default name is a randomized first name and a randomized last name.
And in her head, when she saw the same last name, she assumed that meant the characters were related.
Now of course this meant nothing in the game, the game didn't care.
But here are the two brothers, here are the two sisters, here's the married couple.
And so apparently in the final, final battle, you're going in to destroy the big bad thing, and the big bad thing does horrible psychic damage to all of your soldiers on the map every turn.
And she was losing horribly.
And the husband and wife team made it into the final room with the big brain alien, and the wife runs in and discovers the final boss.
And in XCOM, you had movement points.
So she went in and revealed the fog of war, here's the bad guy.
She has not enough movement points to run away.
And the husband, meanwhile, is at the doorway with a rocket launcher.
And so apparently, Laura, the way she tells it, she spent about 15 minutes having a conversation between these two characters.
As the husband's like, I've got a rocket launcher, get out of the way, honey.
And she's like, I don't have enough movement points, fire the rockets.
I can't.
I'll kill you.
I'm going to die anway, fire the rocket.
And this goes back and forth until finally she decides all right, boom, fires the rocket, kills the person next to the big bad guy, and wins the game.
But again, this is because you had a player who's very cooperative, something as simple as the names actually helped.
Now, you can't guarantee you're going to get players like that.
But those players do exist.
And if it's easy to give them the opportunity to have that awesome experience that she tells 20 years later, do it.
It's relatively cheap.
All right.
That's it for lecture.
Any questions about any of this stuff and how it applies to your games?
Anyone have an idea what they can do in their game right now just thinking about any of this?
Has anybody thought about their title, at least?
Yes, now.
OK. All right.
So take a couple minutes.
It's 2:03 right now, break.
Work in your teams, whatever that might be.
We've got a test coming in at 3:00, so be ready for the test at 3:00.
I'll remind you maybe five minutes beforehand
[INAUDIBLE]
So everyone, raise your hand if you are ready to test at a workstation, if you are ready to test.
So I can count off.
Two, three, four.
One, two, one, two, kind of.
All right.
You get two more minutes.
Philip, you want to talk to?
May I speak to Ed's students, I don't know if you're scattered around, but it would help if you could meet up here.
You one of them?
Yeah, you can come here.
So the first thing I want to say is thank you so much for coming and you may have overheard what Rick was saying earlier, this is going to be our first digital playtest for the games.
There are five games, some of the teams have multiple stations, and there's probably not going to be enough computers for every single one of you who are here.
So some of you may be waiting in line, but please be patient with us, and we'll hopefully get your feedback on at least one or two games, if not more.
It's good for you to give us feedback while you are playing the game.
So if you are-- and the students will get more specific about the kind of feedback that they need.
But if something's frustrating, if something's confusing, if something's intriguing, or you really think something looks cool or is an interesting decision, try to verbalize it so that we can actually realize that something's working or something's not working.
Don't worry about our egos at this point.
We absolutely need the harshest criticism that you can possibly give us at this point, because it's easy for us to change things at this point of our project.
If we learned about the same things that bother you two weeks from now, reversing all of those changes is going to be harder.
So please be harsh now.
Other than that, if anything is confusing, do of course vocalize it.
But the people who are making the game may not tell you what to do next, because they want to see if you can figure it out.
If you can't figure it out, that's not a problem with you, that's a problem with the game.
But they need to be able to see you sort of muddle through what's already there in order to figure out how to fix it later.
So if you do start getting frustrated, of course, you can always say, I'm done with this game and move on to another game
That's a very valid datum.
That's a very valid datum
Yeah, that's very valid data.
It's like, I'm done, no more.
That's absolutely fine.
It would be great to be able to then tell the students why you're done.
Maybe it's because you feel like you've seen everything there is to see into a game, and that's very different from, I have no idea what's going on.
That's two different reasons to leave a game, and they need to know what that reason is.
Otherwise this, by the way, it is Creating Video Games.
It's 6-- what's the course number?
6073
6073 or CMS 611.
As a course it is joined humanities and computer science.
And it runs the full semester, students actually work on four projects for the entire semester.
But the first three are really, really short two week projects, and this is kind of like the mega big project
If you have questions about the class, feel free to grab us.
We're the instructors, we can answer those questions for you
This is Rick, I'm Phillip, you all know Ed, and Sarah and Paulina are over there.
Just look for any one of us and we'll be happy to help
So playtest workstations, stand up if you are a computer that someone can come to to play your game, and testers in the back, stand up if you are a test station.
And testers please go find a standing person and play their game
See what's currently active on the sign
OK. And it's going to come up
So for improvements, I figured out what to do.
I think the instructions are clear this time.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR 1: OK. Just a few announcements before we get into the lecture today.
So the structure for today-- our lecture, two hours' work in your groups.
We have, unfortunately, Elle, the grad student who's working with the heat wave, who would be using the heat wave game, is sick today.
So she won't be able to come in.
But I'm working on rescheduling a time for her to come in.
What I'd love to do is if we can get her a web link of the heat wave game that she could then give you feedback on, that might be good for you all.
For the rest of the class, we've got Pablo in today.
He talked to the snap team already.
Is he going to talk to you again today?
Or are you guys all on the right page with him?
It'd be nice to show him the game.
PROFESSOR 1: OK. Oh, you didn't show him the game then?
OK, cool, cool.
What I'd like him to do is take a look at the forecast-based future game today if he can at 2 o'clock.
I'm going to just show him a build, get some feedback from him.
And then he'll take a look at the snap game.
And then if he's got time, the two cholera games.
He can take a look at those.
And again, if you want to give him web links to playable versions of the games, just send them to the Video Game Bosses list.
And I'll forward along.
Other announcements-- I'm going to email this to the announcements list later today.
This Friday there is a free conference workshop thing going on at a hotel, I think, in Boston.
But it's being run by Epic Games, about Unreal.
So they're going to do a bunch of different workshops from about 2:00 to 5:00 and end the day with a networking party.
So if you're interested in using the Unreal engine and all the different new things about that engine, they'll be running that.
They're also going to be on campus Tuesday at 5:00 PM, so Tuesday, November 18.
That can be found at the GameLab website.
And they'll be talking about Blueprint, a way to do scripting in Unreal that's kind of similar to some things I've seen for Unity, but it's baked into Unreal.
And if you used previous versions of Unreal, I think it's related to Akismet, if you used that tool before.
Otherwise, upcoming guests we have at the class that are different from the syllabus-- on Monday we're going to have two writers and talking with Drew, doing a little panel and then a Q&A with y'all talking about different ways of looking at writing for games that I think would be very directly related to at least two of the games that I've seen.
Because yours was a very dialogue-driven game, yeah?
And then the Animal Crossing kind of game, I imagine, would be really useful for you all.
Probably useful for the other two, as well, but those are the ones that first came to my mind when I was scheduling that one.
And then on the 19th, we have Blizzard coming in, right?
PROFESSOR 2: I think that's right.
PROFESSOR 1: Yeah.
So Wednesday night, Blizzard's going to come in.
And it's teams from Heroes of the Storm?
PROFESSOR 2: I think so.
But-- PROFESSOR 1: But yeah, we're not quite sure exactly who's going to be there for that.
But we talked to them.
And we requested either Heroes of the Storm or Hearthstone.
So one of those groups, we'll begin to talk about their production process.
And you can ask them things about what it's like to work at Blizzard.
That's everything I've got.
Yeah, so I'm going to bring it over to-- you're wired?
Cool.
Great.
PROFESSOR 2: So when I went over to GDC Next just last week, I met Richard.
Actually, he's been to all that before.
And he gave this talk on game audio.
And then I remembered, wait a minute, there's a talk on game audio that's coming up in class.
And I was just going to fake it.
And then I heard his talk and realized maybe he should be the one talking to you, instead.
So I invited him and Andy Forsberg, who's his partner at Hexany Audio.
Let's see.
You'll see the name of the company come up in the next slide.
And they do professional sound design and audio design and music.
That includes things like sound effects, as well as scoring and also advising companies on things like the technology that they're using.
And he's going to talk a little bit about how to do good audio on a budget, as well as just things to think about in your audio design of your own game.
So I'm going to hand it over to you.
And thank you very much for speaking today
Yeah.
Well, thanks so much.
Happy to be here.
Just curious-- so the class, I'm wondering, what level is it?
Is it more introductory, advanced?
What's the focus?
PROFESSOR 2: Intermediate.
So they're making video games.
But it's a class that actually has prerequisites.
So they needed to take another classes, either in game design or in programming, before they came here
OK.
Cool, cool.
So I'm just curious-- show of hands, how many people are planning to go out into the world after you graduate and become game designers, game developers?
Anyone?
Anyone?
OK, cool, cool.
Nice.
That's cool
Sweet.
No, very nice
Great.
So yeah, we're going to talk to you about music and sound in games and then give a little bit of very practical advice about working with composers and working with sound designers.
And so that's the situation most of you will probably be in.
So we'll give you a little overview of ourselves in a second.
But if you want to go on to the next slide, we're going to talk about three key areas.
Yeah, we're just going to tag team this.
So the three things we're going to talk about are, number one, Interactivity.
So we're going to talk to you about what makes game audio different than film audio and why you should care about it.
And then in Agreements, we're going to talk about contracts and what that looks like for you hiring a composer or a sound designer, which is really most often the case with an indy game.
And then lastly, we'll talk about money and budgets and how much hiring one of us audio guys takes.
So a little bit about us-- I'm Richard Ludlow, again.
And I'm the audio director of our company.
And Andy is the music director
Yeah we both went to school in Boston, actually
Yeah, just across the river over at Berkeley and did their game audio thing.
And now we started the company Hexany Audio.
And basically, music, sound, and dialogue, as well as implementation services are what we do with our clients
And we're based in Los Angeles, obviously
Yeah, based in LA.
And we have experience working on larger AAA console games, lots of mobile indie things, so a really wide range.
And we do film and commercial media, as well.
But our focus really is on games and interactivity and the technology that drives that.
So yeah, without further ado, if you want to jump ahead to The Basics
Yeah, absolutely.
So music in games is all about the dynamic experience and player input in the game and how the music and sound effects adapt around that.
Obviously, it's much different than linear media, such as film, TV, et cetera.
When you watch a film, that same explosion is going to happen in the same point every time.
So obviously, with games we have to change that.
Because games are different.
We have to make it adaptive to what's going on.
So there's four major parts to audio in a game usually.
There's the music.
There's the sound and the dialogue.
But the major part of this, again, is the implementation into the game.
If we write amazing music or sound effects or we have the coolest voiceover artists in the world and we get them to do that stuff for our game-- if we implement it incorrectly or the developer doesn't do such a great job because there wasn't a dialogue between us, then all the stuff on the left side, those three things, almost don't even matter.
So it's all about how it gets put in the game, as well as how it's crafted
And we're going to talk about that in the Interactivity section if you're like, what is this implementation witchcraft we speak of?
Exactly
Anyway
Next slide?
Yeah.
So The Importance of Audio-- really talking about why you guys should care about audio.
Because so often you think, OK, I've got this mobile game.
I'm going to shut off the audio.
Who cares about it?
But the depth that it adds is really important.
And we'd just like to touch on three key areas of why music and sound is so important in games, if you want to--
Yeah.
The first thing, obviously, the emotional factor.
And you can't get around that.
With everything else, basically, audio goes straight to the center of your brain that controls that stuff.
It just skips everything else and stimulates that.
So obviously, when we were kids we turned the music off of a game or turned the music off of a film, and we'd realize that we weren't having those emotional responses that we did when it was on.
So obviously, emotion is a huge part of what we do
Yeah.
And that's a chemical response.
Like Andy said, it's actually eliciting-- just emphasize again-- a chemical response.
Visuals do not trigger these emotional processing centers.
It's sound and music.
That's stimulating parts of your brain that are different and actually creating the emotion.
It's pretty amazing.
At least we like to think so.
[LAUGHS] Anyway, cool.
So the next one is Function.
And this isn't a great term.
But audio, it supports the ability for the player to really absorb a lot of the game information.
So when I say "function," it has the function of giving you information and also the function of telling you what's going on with regards to your action.
So for example, you're walking down a hallway.
And there's a guy up ahead that you hear maybe walking.
And you know, OK, there's a guard up ahead.
And then maybe your oxygen tank is running low.
And you've got a little "beep."
And your health is maybe at 50%.
Maybe there's an alarm going off in the building.
You hear a guy in the next room.
You hear all these different things.
And all of that is information that's being processed and sent to you.
And you're able to absorb it all instantly without needing to think and see text on the screen that says, oh, OK, I'm at 50% oxygen.
There's three enemies nearby.
It's getting dark out because I hear the crickets.
This is just stuff that your brain subconsciously absorbs.
So it's functional in that it provides that information for players in an entirely new dimension that is subconscious to them.
So you can convey lots of subconscious information
Exactly
And then also, it's that feel-good factor for games.
So you punch a guy.
Oh, well, hold up on that maybe.
But yeah, you punch a guy.
And it feels good in the game.
And that is because of that trigger, the sound meshing with the psychological.
Anyway, so the sounds we were talking about that are memorable-- I know you just played, what was that, the Zelda sound or something?
Yeah
I couldn't quite hear
Yeah, it obviously has to be memorable with game audio, as well.
And yeah, you played the Zelda one.
Think of the countdown for Mario Kart where it makes you want to hit A really fast and get everyone, obviously-- but there's countless examples.
We might have-- [VIDEO GAME SOUND]
You could just play them all.
Just play them all
Let's see
I'm not sure if you've played them.
But I'll just talk about-- [COUNTDOWN SOUNDS] So these are really iconic sounds.
And I think the Mario coin sound is next.
Let's see.
[COIN SOUND] So that sound has been in an entire generation of games.
And the Zelda sound, when you hear that, that's extremely rewarding for the player.
It's extremely memorable.
Sound is burned into a player's conscience in a way that doesn't necessarily happen with visuals.
You hear that.
[SNAPS] And then you hear the achievement sound that we played.
And it's like, oh, I've unlocked that.
There's this stimulating reward response.
And so good sound and bad sound is going to stick with your players.
And it can really help set that game apart so that later on the players are remembering your game in a very unique way.
Anyway--
Next slide
We'll move on to the real stuff now.
So Interactivity.
That was kind of the preface.
So the first section that we talk about is it that implementation and how that works in games.
And that interactivity is a huge part of that.
So if you want to jump ahead to the next slide--
Yeah.
So on the music side of things, obviously, if you want music that changes dynamically based on what the player is doing, whether he's walking into a field and staying there for five hours because he left it on to go make a sandwich or because he's walking in to buy something from a shopkeeper.
So how do we do that?
There are several different things we can do, whether a player's going into battle, et cetera.
We can use one system called layering, which Richard's going to talk about in another slide.
We're going to talk about crossfading.
And then we're going to give you an example of branching, as well.
And then there's also generative.
And these are just a few.
There are so many options that we can use to implement music in a cool, interactive way
Yeah.
So let's jump over to one of the examples.
And go ahead and throw that up.
Don't hit the Play button quite yet.
So you can hit the next slide.
And it'll pop up the graph.
All right.
So what you're seeing here is the idea of a layered score.
So in this one, this is a piece of music I wrote for a game where there's three different intensity layers.
And you're going to hear a first one that's very low intensity.
And then you're going to hear these other layers come in on top as maybe the player starts to go into an area with enemies.
And then they go into an area where they're in battle.
And you'll hear the music stack in a very dynamic way.
Anyway, go ahead and hit play.
Let's see.
[MUSIC PLAYING] So that low intensity.
Players done some [INAUDIBLE].
The second layer's going to come in on top.
[MUSIC INTENSIFIES] You hear all three layers, though, I think.
Back to just the first two.
And then just the first one.
So you can see this as well.
You could hear the layers fading out in and between one another.
But it was a very dynamic experience.
So this is where we'd take the music, and we'd specifically craft it and write it so that these things could stack and be triggered dynamically based on whatever the player is doing.
So let's jump ahead to the next slide.
So this is one more example-- and this is the last one we'll show you for music-- that deals with the way that music can move dynamically between states again.
So yeah, so we've got the low, medium, high.
But this time, we've broken it out into smaller, little chunks.
So the player will do something.
And the musical will actually wait until the musical point, like the next bar or beat.
And then it will begin a transitional segment into the next level intensities.
And this is going to be a much more dynamic and fluid experience than the layering.
So go ahead and hit Play.
[AMBIENT MUSIC] Low intensity, again.
So a player's going to do something.
And it's going to come in with the transition.
[MUSIC INTENSIFIES] Transition.
[MUSIC PLAYING] Now we're going to go into medium.
We're in medium.
The player's going to do something.
And the music's going to wait to do the transition.
Transition.
[MUSIC CHANGES] Now it's running the high.
[MUSIC PLAYING] The music's going to wait and then go into the transition.
And back to low.
Cool.
So you saw the music move through-- one, two, three, four-- six different pieces there.
But it sounded like the experience was being scored.
So again, the idea here is we're trying to-- again, with the film, you watch it.
It goes by.
You know what's going to happen.
We don't know what's going to happen with the game.
So we write music that can change and adapt like that, in a very film-like way to score the player experience.
So why do we do this?
So obviously, it creates a more immersive and dynamic player experience.
Any time we are taking you guys out of that escapist experience, we're not doing our job.
It extends track longevity, which we'll explain later with budgets why that's nice, as well.
But you get better replay value out of it.
And it's the greatest value from each individual minute of music
Yeah, instead of having to hire someone to write three different pieces of music, if you're on a budget, you can say, we'll do one piece of music.
And we'll make it layered.
And it'll be one better piece of music
Exactly.
Because if a movie is 90 minutes long and they want 60 minutes of music, you write 60 minutes of music.
One small mobile game could have 15 minutes of music.
Or it could have one minute of music or three minutes.
So it all depends
Yeah.
And that goes into-- we'll talk about with budgets how that affects that.
But so yeah, jumping onto the next slide, how do we do these interactive systems?
We build them with what we call audio middleware.
And two of the commercial ones are FMOD, that you see here.
And if you want to jump ahead to the next one-- Wwise.
And these are just screenshots.
And essentially, these allow us to interface with programmers and set up these musical triggers and these points of musical change based on different parameters, like number of enemies around or game state or health, and let us set up these musical transitions and things like that.
So yeah, we can jump onto the next section.
Agreements.
So you've learned all about interactivity.
And you're like, yes, this is awesome.
I'm going to hire a game composer so that they can do this stuff.
And then with music, though, it's obviously a creative piece of content.
So the agreements look a little different sometimes than other agreements.
And so we're going to show you three different kinds
Yeah.
So the purposes of these contracts are, obviously, to see who owns it, obviously, first and foremost.
What we're going to be delivering to you.
What the delivery milestones are in a game just as there's development milestones.
How much you guys are paying, which is super important for you and us.
And what's your rights as a developer?
Where can you use it?
Can you go around and license it to use it in all the things involved with your project or no?
And what are our usage rights?
Can we use it in our portfolio, on our Facebook page?
Can we do that sort of thing?
So we're just going to go through a few of the contract types here
Yeah.
And it's really important to spell out all those things.
So yeah, let's look at the first one.
Work for Hire.
This is the most common type of agreement that you run into with composers.
And it's essentially where I'm the composer, I've written music, and I am selling you my music or sound.
You're buying it from me.
And you're taking all the rights for it as a developer.
You become the owners of the music and sound.
And so you've got the rights associated with it.
And because you have the rights, the developer then can usually use the audio however they want in the project.
Because you own it.
You've also got the exclusive right to use the audio, meaning you're the only ones that can use it.
I can't go around and turn around and sell it to Bob or Joe over there.
You've bought it.
It's yours.
And then lastly-- this is the disadvantage for developers-- is that this is a more expensive agreement.
Because you are buying out all of our rights.
And I'm going to show in detail in a few slides some important clauses to be aware of in these contracts.
But yeah, so let's talk about on the other end of the spectrum.
We've got work for hire, where you and me-- the composer selling the developer my rights.
And then on the other end of the spectrum, we've got licenses
Right.
So we have two sorts of licenses.
The first we'll talk about is the non-exclusive.
So this means that we are retaining the rights of the music, sound effects, anything we do for your project.
We could go around and turn it to another game, et cetera.
However, don't be afraid of these.
We're writing the same custom music for your game, which is super important to remember.
You're getting the same exact product.
It's obviously a lot cheaper.
And it's a great way to save yourself some money when working with people like us.
And again, you can still have unrestricted use of the audio in your project and things involved with that usually.
But again, we could turn around and use this for whatever we would like to use it for.
So yeah, those are the main reasons--
And we use this negotiating with developers all the time.
So if they don't want to buy out all of our rights and pay that full price, we'll say, OK, well, we'll retain the right to go sell it to Bob and Joe and still give you the same music
And then we have, obviously, the exclusive license, which is you're going to retain all of the rights.
But it's usually for a certain amount of time.
And then those rights transfer back to us.
And sometimes it's industry-specific.
So we might be able to use that audio in a trailer or a film or something.
However, pertaining to the game or anything in the game world, we're not going to turn around and license that to someone else.
Usually this lasts about one to five years.
We like to call this the happy medium between the work for hire and the non-exclusive license.
And it really is a healthy balance of price and control.
Because it's right in the middle as far as price is concerned.
And everyone's happy.
You guys could use it for what you need.
But then we also get those rights back.
eventually
Yeah.
So work for hire-- I make the music.
I'm the composer.
I sell it to you.
You get all the rights.
Non-exclusive-- I'm the composer.
I make the music.
I retain all the rights.
And I can sell it to other people.
With exclusive, I'm the composer.
I make the music.
I retain the rights.
But I give you the exclusive right to it.
So I can't sell it to anyone else.
And then it usually reverts back to us after a brief time, as discussed.
So most common, though, is the work for hire.
And you're seeing that contract, and you're like, oh my god.
This is awful.
I don't want to see this.
But anyway, let's jump through that really quick and just a few key points that really go to developers helping protect themselves.
And I've got to hit Play over here on my key notes so that I can see all the different slide changes.
One second.
OK, cool.
So the first thing is services.
It's like, what are you paying for music, sound, dialogue?
What are you buying?
This will be really quick.
Jumping onto the next one, so payment for services-- what are you getting paid?
You want to spell out exactly how much you're going to charge and how much you're going to pay for the music and sound.
Next point, and this is the independent contractor point.
That's important for you as a developer.
Because this essentially says that the composer or sound designer you're hiring is not on salary and does not need benefits.
They have to take care of their own taxes.
It just protects the developers.
So they make everything clear there.
Next, warranties.
These are things, again, that protect you, as the developer.
So this says, OK, the music we're writing, its original.
We're not stealing John Williams' music, as I always say
[HUMMING] score game
Yes, right.
[CHUCKLES] But then if we do steal it-- and we're not going to steal it.
And there are some other warranties, as well.
But we'll jump onto the indemnification.
So then it usually says, if we do steal John Williams' music, you guys are protected.
And there's different levels of indemnification.
But the indemnification just says it falls on us if we screw things up.
Next, there's usually a little confidential information clause, a little NDA clause about not disclosing things and then some specific legal language about the ownership and assignment of rights specific to work-for-hire, legal mumbo-jumbo.
We'll skip the subcontract.
And then there are some rights that are specific to us, as the content creators.
So if you remember, we talked about composer-specific rights, things like the ability for us to post our work on our website, something that we like in our contract so that we can advertise music to other people [INAUDIBLE] them to hire us.
Similarly, we often spell out how we're going to be credited.
And the credit is a great way, as well, to negotiate with your composer.
If you're on a budget, maybe you say, OK, I'll put your name at the opening cinematic instead of just in the back to help compensate for our lack of budget, get you a little more exposure, things like that.
So you spell out how you want to be credited.
That's really it for the contract.
I don't want to go too detailed there.
But some key points to be thinking about if you guys ever hire a composer or a sound designer.
Let's talk about money.
This is the question most people have.
And whenever I meet a developer, without fail, within like 10 minutes of the conversation, they're like, ah, what do you charge?
What's generally what you charge for any game?
It's funny.
And I gave this talk at GDC Next.
And you'll see by the end of it why I'm like, that's not a question I can just answer.
And then I got like three emails from people who attended the talk after and who were like, can you just give me a range for an indie budget?
Anyway, but you'll see why that's funny in a minute.
Because my response is always to them when they ask, how much do you charge?
My response is always, what's your budget?
And the reason being because are so many factors, like what the deadline is, musical style, whether or not we're talking about a solo piano piece versus epic orchestral piece
Are we going to get live players to play those orchestral pieces or a piano player to play that song, maybe
You can throw out the bullet points if you want, just FYI.
We're just going through them.
Anyway, and what's the release platform?
Is this the AAA console?
Or is it a Facebook game for the American Cancer Society?
Again, are we going to use a work-for-hire or what one of the license deals?
Right.
Like we just talked about, buying out the rights or us keeping the rights?
Really big effect on the price there.
How much are you buying?
We're like Costco.
[INAUDIBLE]
But if you're going to buy 100 sound effects, that's obviously going to be different than if you need five sound effects or 50 sound effects
Yeah, we give bulk discounts all the time
Exactly.
And that goes into, obviously, the depth of the audio experience, as well, which is how crazy we're going to get with interactivity, if we're going to be using middleware
Like Andy said with the mobile game, you could put one minute, three minutes, 10 minutes, 20 minutes, all in the same game.
It's going to be a better game if you've got-- maybe you don't need 20.
But if you've got more than one looping track, it's going to be a better game.
But the game still exists with a single piece of music.
So saying, how much do you charge, is a very, very difficult question to answer without knowing--
Any stipulations
--more about your game.
Yeah
One of our clients that we work with, on their page where you go to contact them, it says, when do you need your project done by?
And one of the options is, Yesterday
Yeah.
And deadlines are a huge part of that.
Sometimes we get calls where it's like, we need this in two days or in a day.
And we'll do it.
But it's going to cost more than if you need it in a year.
So contacting, guys, early is important.
Anyway, let's jump on to payment models
And you can hit all these bullet points again
Yeah, I mean, we were just basically talking about how composers and sound designers charge.
Typically with music, it's a per-minute music, so not per minute that we're working
And then sound effects-- yeah, that we're working-- and then sound effects are usually per-asset
Or sometimes a flat fee.
That's also common [INAUDIBLE] quote a project and say, OK, I'll do your whole project regardless of how many sound effects for, you know, $20,000.
I don't know, whatever
Sometimes hourly rate.
We usually don't do this.
This is a really hard one to engage
It's a valid thing.
People do it.
We don't
We don't
But yeah.
[KNOCKING SOUNDS] There's some weird thing happening below us
Sorry.
And then usually, we really like to get back-end profit sharing
If the budget isn't there
Yeah, that's a big thing with audio to think about.
At the end of the day, is it really going to cost you that much to give 0.5% of back-end when you can't give anything up front?
Because we always have passion projects.
We make money on some of our projects.
But we really like to believe in games that are really cool.
And sometimes the best to do that is just to share the game
The indie scene, they often can't afford because they don't have that upfront money.
And so we'll talk about a back-end profit sharing percentage.
And if you talk to people who have been doing this 30 years, that didn't exist 10, 20, 30 years ago.
Audio guys did not get back-end profit sharing usually.
Because they were just paid upfront.
But that's kind of gone away.
I guess that is probably true of all industries.
But so thinking about that is a very valid thing.
And it's usually accompanied by some upfront fee.
It's usually a reduced amount.
But the back-end thing is definitely valid.
We want new developers to be less afraid of that and more open to it
Yeah, absolutely
And then any combination of the above, too.
Cool.
Anyway, so how does this break down?
And you can throw all these up here.
Just showing you music, sound, and dialogue.
There's a lot of components here.
So when we're talking about what we're charging, we charge different prices for themes in the music.
So if I write a musical theme that has its setting, and we charge one price for that.
But then we might put an arrangement of that in an underwater level, a desert level, a jungle level.
The same theme, but different arrangements for different levels.
And we charge less for those arrangements.
Those little transitional segments you've heard in that demo I played, those are transitions.
We charge less for those.
And any different price for things like stingers, like a coin pick-up or an end game, like, [HUMMING] sound like that when you die.
With sound effects, we break things out into primary sound effects, like if you've got a creature having an attack sound, "ch."
And then it's like a die sound, idle sound, those are all primary sounds.
Iterations might be like, attack01, attack02, attack03, die01, die02.
And we charge less for iterations.
UI and ambiences, different prices there.
And then dialogue, there's just a whole slew of components, like casting.
Usually you have to hire someone to cast all the actors
Getting Kevin Spacey for the next Call of Duty game.
Or where--
Kevin Spacey for the next--
--we're doing it ourselves
Who doesn't want to use Kevin Spacey?
You're right.
And then so they charge a few hundred, a few thousand an hour, depending on the actor.
Usually there's someone directing them while they're being recorded.
Someone's recording them, editing them, processing.
Just throwing this up here so you know all the different components of music-sound that are out there.
So again, asking, what do you charge, what does your game need, is a very-- we need to know what your game needs before we can say
Cool, next slide
Yeah.
So we're getting close to the end here.
Then we can do some questions.
But just talking about a theoretical project-- say you've got a low budget, $160,000 Kickstarter project.
So talking about music-sound, we're typically looking at like 5% to 15% of budget being allocated to audio.
5% to 15%, usually hovering around 10, 15, 20, 25-- very good
100!
[LAUGHS]
No, but seriously.
And so a common problem with this, though, is that your game, to be awesome, needs this much music.
But your budget is only going to allow for this much music.
So let's say we've got a hypothetical game here where a project might need 18 minutes of music and 300 sound effects.
So say we were going to charge like $750 a minute for themes and $500 a minute for arrangements, which are kind of indie prices.
We charge up to-- we have a different scale because of all the different factors involved-- but up to like $1,000 a minute is pretty typical.
The big guys on AAA's will be charging $2,500 a minute.
But up to $1,000 a minute of music is pretty typical, between $500 and $1,000.
And then maybe sound effects.
Maybe you use a combination of all those, total $12,500.
Let's say this game, to be awesome, again, needs $23,500 for the audio.
And you're like, oh my god, that's a lot of money.
And let's say the developer has only allocated $11,000 to audio
Right.
So some things we can do to reduce the costs are-- we can reduce the original music themes and increase the arrangements.
So we can just do a lot of iterations with things, which is actually a very feasible option, because those composers were trained to do that.
We can reduce the number of sound effects iterations, just have more of those primaries
Only attack01 instead of 02, 03, 04
Exactly.
We can do back-end profit sharing, as we mentioned before.
So maybe at 3% on that.
And then we can use a license agreement instead of saying you're using the work-for-hire
So just some common things to think about in that negotiating process about, how can we work with the audio guy to reduce the cost to something we're both comfortable with?
Yeah, so let's just wrap up really quick.
What did we talk about?
Interactivity.
It's music.
It's sound.
It's responding dynamically to player input.
The player does something.
The music, the sound, it changes, it reacts.
It's going to create a deeper player experience.
It's going to reduce the cost if you do it right.
And we're doing this with FMOD, Wwise, Fabric, different audio middleware
And then for budgets, we're trying to look at 5% to 15% of the total budget of your game allocated to audio.
Remember to consider back-end payments.
We really think that's going to be part of the future of how audio works within, especially, the indie market.
And just be willing to negotiate and figure things out.
Your costs fluctuate.
Our costs fluctuate.
So it just needs to be a negotiation.
We can usually figure something out.
Any audio person can
Yeah.
Oh, and then we jumped over Agreements
Oh, sorry.
I went right and down
No, it's cool It's all good.
And then, agreements, again, we're talking about work-for-hire.
I'm the composer.
I'm writing the music.
You, as the developer, are buying it.
Licenses-- I'm the composer.
I write the music.
I obtain the rights.
And then I give you some type of license to use the music.
But that wraps up, I think, the four points we were going to talk about.
If you guys have specific questions, we're happy to answer them.
Or Phil, if you have any additional points you want to touch on.
[APPLAUSE]
Thanks
Thanks.
PROFESSOR 2: I think for questions, if anyone has any questions, I'll repeat it since I'm a little bit closer to the mic.
Although, maybe the mic might pick you up if you're in the back of the room.
We'll give it a shot
It doesn't need to be a question about what we talked about, too.
Any audio related things
Favorite color.
PROFESSOR 2: Yeah, as loud as you can
[INAUDIBLE].
PROFESSOR 2: Did you [INAUDIBLE]?
I think so.
So was the gist of it basically being folks using music and sound and then getting takedown notices?
Is that what she said?
PROFESSOR 2: Yeah, takedown notices on some of those by YouTube or Twitch
Yeah, absolutely.
So the whole takedown notice thing has really come about because people are just using the music or just using the sound without asking for permission for it.
Now, in an actual gameplay video, if you're streaming Twitch, that is really a very developer-specific question.
And that's not something that we, as the composers and sound designers, really even have control over.
If it's for a game and if it's in a game, if we're working on a Nintendo game and it goes out to them, then that is owned by Nintendo.
And if it's not owned by Nintendo, they've got the license for it.
So it's all on them with regards to that.
So some developers are totally cool letting it slide.
Some developers are not.
But it's a shifting-- it's all being figured out right now
And usually that happens, obviously, in the bigger game world.
The biggest game that would probably happen with in the indie world would be something like Journey, or something like that
And most often, at least now, developers are pretty cool with it.
But I know it's all being figured out right now.
That's a very shifting landscape
And it has to do with royalties in music and--
That's another thing
Collections have been set up for film and TV for eons now.
And there's a system in place for that.
But there's really no landscape for games yet
It's a little different with YouTube, and the Twitch, as well.
We get paid 0.0001 cents every time someone will watch something on YouTube
Yeah, it's insulting
In five years, hopefully, that will all be figured out.
But sorry I don't have a better answer.
our fault.
It's the developer.
PROFESSOR 2: Just a follow up question on that-- it seems like in order for all these services to actually detect that they're using the music, they need to cross check it against some sort of audio database.
So is that something that you, as composers, usually end up submitting?
Or is that something that the game companies usually end up doing?
Or does it really depend on the license agreement that you have with the company?
It's usually the developer
Yeah.
So there's a couple different ways.
We have friends who work for PlayStation.
And they were just talking about how they posted things on their PlayStation North America account.
And they got taken down by PlayStation Japan or something.
So their own companies aren't even really able to quite mesh this together yet.
But the databases are sometimes internal to publishers.
But also sometimes they're scanning surfaces that these royalties agencies like BMI and ASCAP actually have.
And they've got these automatic scannings that are set up.
And that sometimes comes about because we submit our music to these royalty agencies like ASCAP and BMI in order to be able to get paid.
But then they use them to find out how many times things are getting played.
They don't use them to figure out what should be taken down
Exactly.
Yeah, I think, especially on the indie level, the only way we would be able to detect something is if we went and watched a YouTube video, our music was in it, and we're like, oh, crap!
That's our music
We don't have these engines going.
The publishers have teams of people doing takedowns on their content
Yay, team
Yeah, and I don't know all the details behind that
Sorry?
The ASCAP stands for--
American Society of Composers and Publishers?
I want to say Publishers.
I think they probably should.
I don't know, man.
We just get the checks from them
BMI is Broadcast Music, Incorporated.
PROFESSOR 2: So I can take a look at that
They're called PRO's
Composers, Authors, and Publishers.
PROFESSOR 2: Next one?
Yes.
PROFESSOR 2: Other questions?
Yeah, Liz?
[INAUDIBLE] art talk of last class, they have done an artist test to see if the particular artists are right for the game.
Is there an equivalent for music?
Yeah, absolutely
Yeah, we do those all the time.
And that's a totally legitimate thing if you aren't sure.
Asking them to do a 30-second demo.
I'd say a 30-second demo is totally legitimate.
Yeah, and usually the best way to do that, guys, just so you know, take some gameplay, be it from your game, a different game, concept art for your game, take some visuals as a video, send it to them.
Or just concept art, send it to them.
And let them work with that
Because we could even do an interactive demo for it at that point.
The big thing, again, is there's the difference between being able to write good music and having a core concept for a game from the beginning and thinking about interactivity when you're composing and doing sound effects.
Because it's a whole different animal
Yeah.
But the demos are totally legitimate.
We do them all the time.
But sending some visuals along is very important to making sure you get what you want.
And then we're able to see what the game is going to look like and then make that decision.
Otherwise, we might write something that you might not have the vision for because we haven't seen the art.
And maybe we could easily do what you wanted, but we just misinterpreted it and went a different direction.
But yeah, tests are legitimate, absolutely.
PROFESSOR 2: Questions?
How much are they paid for the music test?
PROFESSOR 2: Will you be paid for an audio test?
What did the art guy say?
I'm curious
They said [INAUDIBLE]
We have not found that to be extremely common with what we do.
We'd like it to be more common
It used to be a thing
Yeah, it used to be a huge thing
For commercials, it was a big thing.
You'd get paid for doing a demo
I think it's definitely more common in the game world.
In the film and TV world, it's so saturated with composers and sound designers right now that they don't need to.
Because so many people want the work.
But it is a respectful thing to do, I think if it's a longer demo.
If it's a shorter demo, then it's like, we're happy to get the gig
We're going to keep the rights to that, anyway.
So you could say a couple hundred bucks maybe.
But it's not a required thing.
If we want the gig, we're happy to do a 30-second demo to try and get it often.
It's nice to throw a couple hundred bucks.
But at that point, it's a couple hundred bucks.
It's not like-- it's pennies in the grand scheme of things if the project is larger.
So yeah, anyway, hopefully that solves it.
PROFESSOR 2: Yes
Any more?
Do you find that part of your job is convincing people that audio is important?
Oh yeah
That's a big part of it.
We use the three things-- emotion, function-- no
Yeah, I think the biggest argument in that is probably a mobile scene.
Like in class, someone was probably playing a mobile game during our talk today and-- (LAUGHING) just kidding.
But it's a big thing, is like well, do they just mute that?
And then it's like, yeah, you could say that.
But some people do listen to it.
When they do listen to it, they listen on headphones.
And we were at a talk at a conference this year called GameSoundCon with a composer named Guy Whitmore, who did the music for Peggle and Peggle 2.
And he was talking about, there's not a more intimate experience for audio than in headphones or in that arena.
But I think in general, it is undeniably important
You can put out a game.
A mobile game can do OK with bad audio.
We're not going to lie.
That's a thing, like Flappy Bird.
Believe me, I love Flappy Bird.
I have triple-digits score on Flappy Bird.
But if you want to set your game apart, people look at that polished experience.
If we're talking about indie games, they look at something like Journey or mobile games, something like Peggel 2, they look at that, and they're like, wow, this entire experience is super polished
It's a great game
And it's broken down into the three parts, really.
It's the functional, it's the mechanic of the game, it's the visual, and it's the audio.
And those are all a third of the game.
And if you take out one of those, you can sell the game.
If you don't have great art, that's fine.
There's a lot of great games that don't have great art.
If you don't have great mechanics, OK, maybe it's pretty and sounds nice.
That'll still sell
If that was a pie and you had people over for a party, would you hand them only 2/3 of the pie?
They'd be like--
Crazy
--this dude is selfish.
[LAUGHING]
He says that because I eat like a pie-- I have a thing where I eat a pie a week
That's a whole other--
But no, seriously, so your game can function without one of these things being great.
But it's not going to be as great of an experience.
[INAUDIBLE] PROFESSOR 2: No?
Then I want to thank you again very much for speaking to our students
Thank you for having us.
[APPLAUSE] Yeah, and feel free to put up our email addresses.
Mine is rludlow his is aforsberg if you want to-- Oh,
--@hexany-- oh, yeah, perfect.
OK, cool
And yeah, feel free to shoot us an email.
If you have any questions, we're happy to support audio with the new generation of developers.
There's a lot of great people over at Berkeley, too.
If you're looking for composers and sound designers to collaborate with, other students, there's a lot of great people over there.
There's the Video Game Music Club
Don't be afraid to cross the bridge
Yes.
The Video Game Music Club, if you shoot them an email, you'll get like 10 responses from people who are like, yes, let me help you work on this game.
You're not getting paid.
I'm not getting paid.
They might be OK with that
Yeah.
You guys have the facilities for what you do.
And Berkeley has the facilities for what they do
Anyway, but then you can call us when you've got paid gigs.
[TSKS]
(WHISPERING) Just saying
All right.
Anyway, well, thanks so much.
I really appreciate you guys caring about audio, really, honestly.
We really appreciate people caring about music and sound a little bit.
So thanks so much
And good luck with the rest of your semester
Absolutely.
Thanks.
See you guys around
Cheers, guys
Adios.
[APPLAUSE] PROFESSOR 1: All right.
So yeah, on that note, if you are taking CMS617 next semester, the advanced game studio, CMS610, the media arts industries law class, both of those are making a good polished experience for the end of the semester, with the idea of either showing it to publishers or submitting it to a festival.
So audio is really, really important in those two classes.
And going to Berkeley for that kind of support is highly recommended.
Also recommend them this class if you've got the time, as well.
And let us know if you'd like more information about connecting with Berkeley folks.
PROFESSOR 2: Yeah, we actually know the instructors and even some alums.
But the alums know the people who are still in college.
PROFESSOR 1: Yep, just email Video Game Bosses if you'd like that contact info.
Otherwise, you've got five minutes to take a break, meet back up with your teams.
And Pablo will be in at 2 o'clock to play three games.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right.
So I'm going to hand it over to Andrew.
And this is basically-- we're continuing our track on narrative and writing for games.
On Wednesday we're switching back over to production and we'll be talking with-- we'll have folks from Blizzard to talk with us.
But for today, more narrative
All right.
We're going to talk about writing for computer games, or games in general.
I'm here with Heather Albano and Laura Baldwin, who have very different experiences with writing.
And so I'm just going to go ahead and dive right in, and we'll just sort of see what happens in this conversation.
And in a bit we're going to start opening up to questions from you folks, then we'll sort of play it by ear, and go on.
So I want to start with-- I want you to go ahead and answer first, Heather.
What was your path towards getting into writing for computer-- for games?
Not for computer, just games?
I usually tell people I got into it sideways.
So my freshman year in college-- my first week of my freshman year in college I looked out the dorm window and there were people with boffer swords out in the quad.
And my new best friend and I looked at each other and said, I'll go check it out, if you come with me.
And so I got involved playing with a LARP, and-- Live Action Role Playing a game.
And then I got involved writing for it.
And I did that as a hobby for a number of years, through college and into the real world.
And in 2009 I lost my job in marketing.
It was the year that everybody lost their job.
And Adam Morse, whom I knew from the LARP, called me and said he was founding a games company.
This is Choice of Games, which possibly you've heard of.
They've been pretty well in the four years since.
At the time it was a lark.
And he said, do you want to write for us?
They never turn into anything.
And I said, yeah.
Why not?
So it started me on a really amazing adventure of-- started writing for them, and now I do freelance for other people as well.
And here I am
All right.
Cool
Mine, actually, is not so dissimilar.
I went to college and got involved in the MIT Assassins Guild, which hopefully some of you have heard of, which is another LARP, and started writing for them.
I did sit-down games.
Then I went to grad school, got a degree in immunology, and decided I didn't want to be a biologist.
Which is much like you, losing your job
Yep, [INAUDIBLE]
And I got a half-time job working with MIT, because they only had half of a head count in the thing that I was doing, and my other half-time job was a friend of mine who I had written some Assassin games with and had done some sit-downing with.
Had, at a point in the past, founded a computer game company, or been part of creating a computer game company-- Looking Glass, I think it was Studios firsts-- And said, well, I know that you know how to write things, and know things about games and can work with well with others.
So do you want to come and work for us?
And since I wanted to keep my job with the health care I worked half-time for them and half-time for MIT, and did that for a number of years until Looking Glass ended up going bankrupt, and now I just work for MIT.
And still do sit-downs and stuff.
But yeah, it does seem to be about-- you get into games first and then you have connections with other people and they get you into game companies
Networking works, is the moral of the story
It really does
Meet each other
You never know later.
Everyone you meet today might be helpful.
Yeah.
So you've both done a fair bit of writing for games.
Have you been writing outside of games at all?
Do you want to take it first this time?
So I work in customer support now, and that involves a lot of writing email back to people.
That has trained me to be concise and clear and friendly in writing.
It's a little bit related, but that's about it
OK.
In contrast, I do a lot of non-game fiction writing.
My dream job for years was to be able to write novels, and it's actually less so as I've had success in writing for games, which is fulfilling in its own way and pays a lot better.
But I've sold some short fiction and some poetry to magazines.
I've self-published two novels.
And that's sort of on the hobby side of my life these days
Got it.
We'll come back to that in a little bit, I think.
So I want to talk about what kinds of games you've worked on, and so what kind of writing that you've done has appeared in those games?
And feel free to name names and whatnot of what you worked on.
Why don't head, Heather?
So my professional game writing divides into three categories pretty neatly.
I've done most of it for Choice of Games, which does multiple choice text-based games of the Choose Your Path variety.
So, kind of like the old Choose Your Own Adventure books, if you're familiar with those, although better-written, if I say it myself.
So that's all text, right?
All text all the time.
I was lead writer on Codename Cygnus two years ago now, which is an interactive radio drama, which is sort of the same idea in that it is all word-based, but obviously the interface is very different.
And I have done some work for paper-and-pencil RPGs.
TimeWatch by Kevin Kulp is coming out next year and I've got some work in that
There's a small world again.
OK.
I know Kevin Kulp
I figured you did.
Looking Glass, right?
Wasn't he-- no?
[INAUDIBLE]
Yeah.
His wife worked for Looking Glass, and et cetera.
So talk about interactive radio show.
How does that work?
So you play it off your iPod, and I think now your Android.
It talks to you and at the-- it narrates what's going on.
You play a spy in a James Bond kind of world.
And so it tells you the circumstance you're in and then asks you if you want to do this or that in response, and you say the word.
The voice recognition is pretty good, but it's not super-sophisticated.
It wasn't designed to be.
So you don't get to talk back to it in a natural way, you have to save the-- it's a little bit better than, pick a or pick b, but you have to say the words that it gives you
Got it.
So you can say, shoot gun, not for a villain or whatever
Exactly.
Not, I draw the gun and duck behind the-- no.
I shoot the gun.
But yeah.
So other than that, it's mostly multiple choice
All right.
So the behind-the-scene mechanics are very similar to the multiple choice text adventure, but it's text that's going to be heard instead of read
Exactly
Oh, all right.
Neat.
And you, Laura?
What was the original question?
The original was what sorts of stuff has-- what kinds of games have you worked on and how did your stuff appear in there
So I was kind of a-- since I was part-time they didn't want to put me in charge of any specific levels or anything, so I was sort of-- I did a lot of odd jobs.
I ended up writing the manual for System Shock 2, which was really exciting because a lot of the time you have to write the manual before the game mechanics are sorted out.
So you would say things like, this psi power provides defense.
You really have no idea, sorry.
And then in Thief and Thief II I did a lot of sort of like random books that are scattered around, or dialogue that characters would say to each other.
There wasn't much that characters said to each other.
And then the guards and so on you're trying to sneak around, and the guards are basically walking around being emitters of what they think is going on right now which, you know, if you're really sneaky they're like, I have no idea anything is going on.
I am bored.
I see nothing.
I hear nothing.
But you know, if you let them-- if they see you briefly or hear you briefly, then they sort of have to admit their state which is, I am mildly suspicious because I heard something but I don't know what it was.
Or, I'm really suspicious because I just saw Thief going that way.
But you know, they're not very bright AIs, so they don't have a lot of different states that they can say.
They have a level of suspicion and then a level of what caused the suspicion.
And there's a big matrix of that.
And then you don't want your guards to say the same thing every single time, so you have to have, like, 20 different things for each of those categories.
And it gets kind of tedious and difficult to come up with 20 different ways to say, I am mildly suspicious because of a noise, without sounding like you've been coached to say that.
You know, how many natural ways can you say that?
And then yeah-- I think that's most of it
All right.
Yeah, that's good.
And so let's jump right into a little bit here, the difference between writing for stuff that people are going to read versus stuff that people are going to hear.
It sounds like you've got a lot of experience with that, with long-form read words, and you're talking about guards that are saying things out loud, or also written books.
And those are very different things, I assume, in terms of how the player's going to appreciate them
Yeah.
So the thing that I learned pretty quickly about stuff that is said out loud is that the players want it to be really, really short, which makes it in particular extra-hard because, you know, "huh," or "what was that?"
You know, they want to have about that much of interaction with the guard that they're hiding from, and you have to convey all this little settings because you want to have people know the difference between a "huh" that's like, "hm," or a "huh" that's like, "huh!
I'm going to get you."
So there's also a lot of little stage notes, you know?
"Huh.
Angry."
But I ended up writing a lot of little, short dialogues for guards to be talking to each other, or other-- it was mostly guards-- that you could sneak by and hear, and sometimes it was to convey information about where something was.
And sadly kind of the first thing that we got told after the playtesting was that those are just too long.
Nobody wants to sit there for three minutes and, you know, hide in the shadows listening to guards talk to each other about something.
Especially not more than once.
And so really only one of the conversations that was more than four lines survived into the shipping game, which is at the very beginning of the first level.
And you can skip it.
It's the guards talking to each other in the courtyard about going to bear bits, and you can go and you can listen to them.
And it tells you a little bit about the world.
Or you can just, you know, sneak around and go back to the secret entrance.
And so if you're playing the game for the second time you do not have to listen to all of it
So is there a lot of conversation left on the cutting room floor, as it were then?
Yeah.
Yeah
Because it wasn't until playtest that you realized that it was too long
I think-- I guess it must have been recorded because most of our guards were being voiced by programmers, at least early on.
I think we definitely kept Dan Thron as the stupid guard.
He became very popular
He's got a very expressive voice.
So I imagine your form of fiction, Heather, is a little bit different because it's a longer form to begin with, so you expect a little bit more patience on the part of your players
Yep.
And also all they've got is the words.
The words are-- the dialogue in the radio drama, for instance-- the dialogue isn't a distraction from the mechanic.
The dialogue is the mechanic.
It's all they got to react to
Yeah.
You don't want to have-- they're not going to be saying, oh, I want to fast forward through all this, because why would they be playing?
Right.
Exactly.
All they're doing is listening to other voice actors, you know, putting together a drama and then they're responding to it, interacting with it.
The difference between-- there is a difference between writing dialogue that will be read and writing dialogue that will be spoken.
In the Choice of Games we have a lot more freedom.
We have a lot more-- we have a lot of things that aren't the dialogue.
You can put in a tag that says, "hey, he said angrily."
Right?
Or you can say that he said it while folding his arms or whatever, that gives the reader a clue as to what they're dealing with.
When you're writing a radio drama all you've got is the dialogue and the stage queues for the actors.
I got a lot better at writing dialogue very, very, very fast out of necessity, working for that project.
People who write traditional fiction will tell you that there shouldn't be a difference.
You should always be writing dialogue that doesn't need the prose around it.
You should always be able to get the character's personality and how they're reacting to the particular situation just from their word choice, which is a great gold standard.
There are some writers who do that very well.
Dorothy Sayers, who wrote a series of mysteries back in the earlier part of the century-- in the early part of the 1900s-- does this very, very, very well.
You can read-- she's got-- there's pages and pages where it's just open quotes, close quotes dialogue between two characters and you never lose track of who's saying what because the personalities are so very different
So very clear
Very, very clear.
Most of the rest of us aren't that good.
That's really the gold standard.
But when you've only got dialogue to work with, you have to get that good.
It forces you to think very hard about what the different characters are like
Sure, sure.
I imagine the stage direction helps a little bit.
So you want to get across he said angrily, but we can tell the actor to do that
Exactly.
Yeah
And then, of course, you probably want to pick actors with different sounding voices to help it along a little bit, and that sort of thing
Yep

That's more on the production end, but yeah
Sure.
Sure.
So one of things that a friend of mine was talking about was, he was adapting a Shakespearean play for a comic book form, which is very cool.
But when he did that he said he had to take out about half the text of the play because in the play most of the information appeared twice, because the listener was hearing the play and you don't, in real time, understand it.
For a radio play do you give people redundant information or do you just rely on them paying really intense attention to it?
That's an interesting question.
I think I would probably have to go back over the script to answer that for real.
My recollection is that we did a little bit of repetition to make sure that any time the player was making a choice they knew the context they were making the choice in.
So if there had been some dialogue they couldn't respond to they might get like a summary from character number three.
"It looks like John wants to do this, but Mary wants to do this.
What do you think, agent?
What should we do?"
Do you have a Go Back button so you can hear stuff again?
Not in the current implementation, as far as I know
But it sounds like you could almost play this game without touching anything, is the idea
That's the idea, yeah.
You can-- it does come with a button that you can choose visually if you like, but the idea was that you should be able to play it in your car
Got it
So yeah.
But of course we weren't dealing with Shakespeare, either.
So it wasn't that level of complexity
Well, and the poetry must complement it
Like, in role playing games where they have a lot of voice-overs and they have subtitles, I usually turn the subtitles on, just in case.
Like, sometimes I miss a word or two and I'm like, I can't go back.
It's a conversation tree, you know?
Right
Right.
Yes
That's kind of interesting too, because a lot of the sit-down role playing games that I've been in, a lot of times the gamemaster's very patient with saying, wait, what did that person say?
And if you missed it, will help you figure it out.
But in this meeting you're talking about, no, there's not a lot of help.
If you missed it, you missed it.
And part of the game is, in fact, comprehension the first time through.
That's kind of neat.
So, all right.
So you've talked a little bit about this already, but all these things are placing, pretty much, a lot of constraints on your writing.
You know, you're talking about-- you have to get across ideas for dialogue alone.
And I think it's kind of-- there's a difference between-- there's conversation that gives you an idea of who the characters are, or just sort of some filler, and there's conversation which has got vital information in it.
You might know before you write that conversation what that information is.
Does that change how you approach that writing, knowing that you have to stick some important information in there?
Or do you now think about it that way at all, and I'm just off in the field?
No, I've got to think about that.
So I think every scene has a particular point you're driving towards.
If there are multiple choices in a scene-- I'm thinking now Choice Of as well as Cygnus-- then every choice has a particular piece of information that you're needing to get across so the player can make the choice.
It's not quite a constraint in bad way, though.
It's more of-- I don't know.
It's a more positive word than that.
I feel like I've actually had the good fortune to be remarkably unconstrained in the writing I've done for games, because so much of it has been text-based and dialogue-driven.
I think if you want to talk about constraints, writing for actual video games is the--
So, I mean, Yeah.
The little grid of "I'm suspicious" is very constraining.
But I mean, to some extent the constraints are what tell you what you have to write about.
I mean, I pretty much never wake up in the morning and say, "I am going to write a thing."
It's always, you know, what are the things you have to write?
I think I read somewhere that poets said that the very strict poem forms-- it's the pushing against the boundaries that are what make them interesting, as opposed to prose where you sort of write whatever.
But it's more that sometimes the constraints are kind of funny.
Like, for the cutscenes the rule was you did them in screenplay format.
And, you know, it's particular margins, particular fonts and capitalization and everything.
And the rule of thumb was one page per minute, or one page per 30 seconds, or-- I don't remember what it was, but it was-- and so it was, OK, well, we've got two minutes and 30 seconds budgeted for the cutscene so it can't be more than five pages.
And you can't do the trick of making the font smaller.
So there was a lot of very strange text editing where, you know, someone would say things and it was, like, a line plus a word, and you can't have that.
That's taking up 10 seconds or something with the one word at the end-- which it isn't, really-- but by the measurement, you know, no.
It has to be-- so, you know, I would-- sometimes you put in a semicolon instead of a dash because it's shorter.
And then, you know, you would put in the descriptions of the scenes or the action or whatever, which is like, you know, "view of a study, zoom in on the desk which has an ink well," or whatever.
And you had to make those also very short but descriptive because you can't just say, "study" and leave it to the artist to imagine what you mean.
And the reason that the rules are like that is because the more description you have in the set scene, in the scene description, the more time it's going to take to show all of the things.
You know, if you have 15 different things on the desk and you want to be able to see them all, you'll want to focus on it for longer.
So it's kind of true that the more lines you have describing the scene, the longer it takes.
But it was a very strange constraint to write to, just because it seemed so surreal
So you're kind of gaming-- you had a constraint and you're kind of gamed the constraint to fit on one line.
And that's actually not at all surprising.
I mean, that happens with all rule sets everywhere, right?
People game the system
And I'm sure screenwriters have to do that, too.
And they're probably-- like, if there are screeners out there they're probably like, oh, yeah, use the short words
And, interestingly, there is an almost identical constraint in Choice of Games, where we only want so many words on any given screen.
People are playing these on a phone a lot of time, right?
So there's a rule of thumb that I can no longer remember, about how many words you can have before you should have a choice, and how many words per screen.
Perhaps you might have two screens before a choice, but probably you shouldn't do a lot of that.
You get the idea.
But so that leads to some gaming, like with finding a shorter word to say a thing, but it also forces you to think about what you actually need to say right now as opposed to later, or never, or, you know.
And what you said about, if you need more words to paint the scene it's going to take longer, that's definitely true when it's prose, right?
If the player needs to know about-- he's sitting at a desk where there's an ink well and a book of poetry and a something else that's going to be relevant later, you have to describe them all without knocking the player over the head, necessarily
By the way, there's three things
By the way, there's an ink well.
Yeah.
Yeah, exactly.
You just slip it in
So you talk about these constraints coming in.
Are these constraints that were handed to you by the rest of team?
And if so, is there cases where-- how much creative controlled do you have to sort of fight back against that sometimes?
Or is it pretty much written in stone?
Or how do you interact with the team that way?
Well, when I was at Looking Glass I was kind of the odd jobs person, as I said.
So I would had really no creative control, but I wasn't looking for it.
I was happily doing tasks that I was given.
Sometimes, I guess, I would say-- you know, sometimes you would get feedback saying, so you need to make these things more this way, and I would be like, well, it isn't possible to make them more-suspicious-sounding and still have them in the not-suspicious category, or, you know.
But it would be a conversation.
It was like, OK, the thing that you're telling me to do doesn't make sense, please clarify, as opposed to, I think the thing you're telling me to do is wrong and I want it different.
And--
So there would be back and forth, communication being the key thing there
Yeah.
And, you know, often there would be a level designer who would say things like, OK, I need books for this, this, and this.
Or can you translate the books that I've-- I have these pieces of information.
Can you translate them into Hammerspeak for me?
And so it was--
What's Hammerspeak?
Ah.
So the factions in Thief-- there were the Hammerites who kind of spoke like King James bibles, and there were the trickster minions, the Woodsies, who spoke weirdly.
And that's kind of the-- you know, they-- I think Terri Brosius came up with their dialect originally, and it was, you know, "the Woodsies sing in the treesies."
And, you know, it was just a little weird
A little crazy, kind of
Yeah.
And it tended to be not very grammatical and there were a lot of diminutives on it.
But it made them all sound like they were from the same subculture, and I think that that was useful.
Where did I start with this?
You started with working with a team, creative control.
If there's any-- how do you manage that, or was it frustrating or any of that?
I can't do artists stuff, and I wasn't very good at the level des-- So I don't want the creative control.
I want to do the stuff I'm good at and let the other people who are going at stuff do their stuff.
That's--
I think that's certainly the attitude I would take if I were working on a video game, which I haven't done yet.
Maybe someday.
Because I have great respect for people who can do visual storytelling.
I have no artistic ability myself.
So I would definitely be pushing that off.
The Choice of Games-- I'm doing a solo project the first time, galloping towards the end of a solo project.
The other ones I have done have been co-authored, and so there was-- the first one certainly I was kind of a junior partner, learning.
And so the Choice of Broadsides was the first game I worked on, and when I joined the team it was all storyboarded.
And so it was a matter of, right, well, I'm going to learn how to do this, so I'll take that scene to start with, you know?
The second game, Choice of Romance, Adam Strong-Morse and I were definitely colleagues.
And so there's a give and take in a-- well, any kind of partnership.
Any kind of creative partnership, for sure, where you have two people with different opinions on how to do x and y and z.
And, you know, we have a pretty solid working relationship.
The one I'm doing solo has been both freeing and terrifying in that I have creative control-- yay.
It's wonderful.
It's totally my story, and that's phenomenal, and it does mean, however, that when I don't know how to do something there are times when I do kind of wish I had a co-author to say, I actually don't have any good ideas for this.
Can you--?
What I do have is a very, very good editor.
And so we work in pretty close partnership where I can-- for instance, last week-- write her an email after a playtesting session saying, I think we might have to eliminate this section completely.
If we do it we'll cause these other problems, but if we don't, I don't know how to solve these problems.
Which do you think?
Leading to a two hour conference call where we came up with a third option, which I would never have come up with by myself, which meant we didn't have to cut the scene and we solved the problems a different way
That's really cool
So hooray for editors
I really like the fact that you mention that, as you're playtesting your work and it's not done.
You're playtesting your writing, which I think is awesome.
And so that's a great example of what we're talking about, where playtesting totally just told you that something wasn't working.
Does that happen a lot?
Unfortunately.
Well, I mean, fortunately, right?
Because you'd much rather have it when it's still under your control than when it's out in the wild, right?
So the Codename Cygnus thing I was the lead writer, so I have a lot of influence, a lot of control of the script, right?
It was me and the Creative Director, John Myers, working together on it.
But John was in charge of production, so I actually wasn't involved in the playtesting for that.
I would get notes that were along the lines of, this dialogue sequence is running way long, we need to cut it.
You get the idea.
The Choice of Game stuff I have been much more in the trenches, like with this game I'm working on now, my solo project.
I'm buying dinner for friends and sitting behind them and watching them play it because it's the only way I can tell when they're confused.
You can ask people to write you notes, and they will, and they mean well, but they never remember everything.
And so when you're sitting there and somebody's taking 10 minutes to make a choice you can ask, so what are you thinking right now?
And you can get an idea of whether they have no idea what's going on, which was actually where the problem in this most recent-- that the player was utterly confused and that wasn't the first time I'd gotten that feedback.
Although it was the first time it was utter confusion, rather than mild confusion.
Or whether the player is so engrossed that they want to make sure they do exactly the right thing, which is a fine response.
And they can take 10 minutes if that's what I need to do.
So yes, I've sort of wandered away from-- what was the original question?
Playtest.
You found it.
You do it all the time.
In fact, you're going out of your way to do it on the solo project which is mostly writing, and you're still playtesting a lot, which I think is really fascinating.
You mentioned an editor, you have to imagine-- so you've written books as well
Yes
Do you playtest books?
Um, yeah.
I do, actually.
You don't call it that in traditional fiction, but absolutely.
You have drafts, and I belong to the Cambridge fiction workshop
My husband gets really grumpy when I call him a playtester instead of
Who this?
Say that again?
So Jerry works in QA and I often refer to him as a playtester and he's like, it's not actually playtesting
Is he QA-- not for games?
Not for games, yeah
OK. All right.
That's hilarious then
So Laura, have you had experiences with playtesting being essential or showing you stuff you didn't expect?
Yeah.
I mean, in general for computer games you have to have playtesting.
And I feel like there's actually a new, modern version where you don't playtest and you ship broken games and you just patch as you go, but I don't approve of that
That is the new, modern version.
Yeah
Looking Glass had in-house playtesting, so you not only had sort of the naive people saying, oh, this is confusing, you had people who were like, OK, I know how is this, this, and this, and they would try to do all of the things and tell you which ones were broken.
And that's a little bit more for the programming side and the level design side.
But I guess there was one thing that came up pretty early, was the head designer thought that it would be really cool if there was sort of a thieves can't.
And so all of the internal stuff that the Garret character would be saying to himself-- and sometimes the guards might be saying-- they would use dialects.
They would say things like, "eagled" instead of "saw" or-- I forget what we came up with for "heard."
And he had this idea of it being very Clockwork Orange where it has its own, you know, internal language that you can mostly figure out if you're paying attention.
And so my first set of dialogue stuff was full of this stuff, and the playtesters were like, we have no idea what they're saying.
This makes no sense.
They're saying random words and it tells us nothing.
And so that all got cut out, and pretty much the only thing that got kept was the word "taffer," which was sort of the generic swear word.
You know, you would say, "you taffer, I saw you back there," because, you know, we were trying for a no-swearing game.
But sometimes you want to have character sounds like they're swearing so we had that word and it stayed
That's awesome.
I find there's a difference between testing and playtesting though, too
What's the difference?
Well, so one is technical.
Does the game work?
Are their crashing bugs?
And we have certainly at Choice Of, it's two different things.
There's the, can you play to the end of it without it crashing on you?
Good.
All right.
First level.
That's great.
Can you play through it with it doing the variable hands off between the scenes correctly, or do you suddenly find that you're married to this person instead of that person?
I mean, problems like this happen all the time.
So although my undergrad degree is in English, I did work in consumer electronics, such that I was working with programmers.
I was in QA, in fact, a long time ago before I was in marketing.
And so thinking about it in terms of the black box testing of, are the variables being passed, are the flags being set, are the typos in the English on the screen, everything capitalize?
And then there's the playtesting which is, are you delivering the experience you're trying to deliver?
Which you can only tell by watching a human being play it
Got it.
I like that you split those two apart because when we talk about it in class we often don't make that distinction, but I think it's an important distinction that there's the, does it do what we think it does technically, and does it do what we want it to do emotionally or enjoyably or whatever?
That's has a good distinction
In consumer electronics I worked for Bose Corporation.
They separated it into verification and validation, were the sub-parts of the test department.
It was, did we build the thing right, and did we build the right thing?
We thought that was pretty clever
Awesome.
I like that
I feel like, from my experience, it would be kind of more programmer bugs versus design bugs, do the variables do the right thing?
Programmer bugs.
Does it crash?
Programmer bugs.
Is it fun?
Design bugs
Yeah.
That's a great way to think of it, yeah
Sure.
So I've just got one more question for you and then we'll turn it over if anyone out there has questions.
But before I do that I want to point out that, while I did sort of warn you that I'd be asking a little bit about, have playtests been useful for you, I did not ask them to go on as long and as enthusiastically as they did about how awesome playtesting has been
Is this something that you're trying to--
We say this a lot.
Playtest, playtest, playtest, playtest.
No matter how much you're playtesting, you probably aren't doing it enough.
Anyway, so the last question I have kind of comes back to something you were talking about how, when you did your first project you had the storyboards and you're like, oh, I'll take on this part and another writer might take on a different part.
Did you have any problems getting the voices to match?
Presumably you've got different authors, different writers for different sections, but you want it to feel like it's a coherent piece.
You obviously want to write in a similar style to the other writers
Did we have a problem?
The answer is yes, but not like in a funny-horrible way, in a, this is one of the challenges with-- Broadsides was three people.
It was me, and Dan Fabulich, and Adam Strong-Morse.
And so it was sort of part of the cost of doing business, of dividing the work among three people, was the, hey, this guy has a different-- has speech patterns that imply a different educational level when one person a is writing than when person B is writing.
Either is fine for a sailor in the Napoleonic era, but we should pick one.
Like, these were bugs in the spreadsheet when we were-- We were like, we gotta fix that because so-and-so sounds like he has a split personality.
And there were sometimes issues between-- like, that's a design thing, as you were saying.
That's a flavor thing, right?
There were sometimes issues with handoffs on the technical sense, but that's also-- somebody wasn't setting a flag that someone else needed, type of thing.
And that was in the spreadsheet.
I think that's just a collaborative thing, you know?
Sure
What it means, interestingly, is that having three writers doesn't save you as much time as you might think.
It is probably quicker than having one, but not three times quicker because you have the integration phase that you don't otherwise have
Sure.
I like how you put that, because you all have encountered the same thing through code right now
Yeah.
I found that having, for any given sort of tone thing, having one person who is the expert, or at least that was their thing, like Terri would tend to do the pass on all the Woodsies and, you know, make sure they seemed Woodsy enough.
And I would do the pass on the Hammers and make sure that people hadn't confused thee and thou and translate things, you know, into King James.
I mean, it wasn't even real King James.
It was sort of, like, what people think King James sounds like
Right
For Broadsides we actually had-- Adam was serving as Lead Writer.
It was-- so this is a secondary world Horacio Hornblower, so not the real Napoleonic war, but the Napoleonic war, you know?
But so it was his idea and he was the Hornblower fan, right?
So if there was a disagreement about the right way to name something or present something or whatever, Adam had final say because-- I read three of the books really fast as I was getting up to speed for the project, and so did Dan, but that's not the same thing as being immersed in it, right?
Right
It was actually harder when Adam and I were working as complete partners and we were building our own secondary world medieval fantasy from the ground up, because there was no-- sometimes it's nice to have a final arbiter as opposed to having to come to a resolution-- not necessarily fight it out, but come to an agreement with everything under discussion
It's just faster when you start to get to an impasse if someone can say, nope.
We're going to go with

All right.
Any questions from out there, while we're here at the moment?
Please
So, something I think is true about the Choice of Games is that, if you make choices at the beginning of the story that, like, carry over and possibly have grand repercussions on the three-hour mark, or something like that, how would you actually playtest them?
So it seems like playtests having to deal with something that happened three hours ago without having the playtest go on for three hours--
Ah, so--
Let me paraphrase that, just for the microphones.
You're asking about, you run a playtest that would take a long, long time to see the results, but your playtester isn't that patient.
So how do you work with that?
Yeah.
Well, I actually don't know how you work with it in that constraint, because the-- if you are the author and you know the game you can self-playtest it such that you skip ahead to see how scene five feels when you're married to, Grace instead of to Diana, right?
But if you're playtesting-- playtesting for real-- Choice by Gaslight, the game that I hope will ship early in the winter, has about four hours of gameplay.
And the only way I have found to test what it feels like for the players to make the choices and be in it is to sit behind them for four hours, which is why I buy them dinner, because it's the very least I can do.
And these are all informal.
These are all friends of mine
Of course
So I think there are things that you can't shortcut.
You've just got to do it if you want to know what it's like for the player.
On the other hand, there may be a better solution to that that I just haven't found yet, I don't know
I mean, so for a larger company, the answer is you pay them a wage to do it for three hours.
I mean, it's their job.
And they do become faster at it, like, if they were playtesting Choice of Games they would probably be able to read pretty quickly the things that they've read before.
But--
Absolutely.
When I say four hours of gameplay, that's the first time.
The first three or four scenes are not substantively different, no matter what you pick.
I mean, there are flavor text differences but they're not meaningfully so.
So once you realize that, then the second play through is just, yeah, right, right, right, right, right, OK.
Here's where it starts getting interesting.
Now I'm going to slow down and read again
Sure.
And I think that's, in large part, actually why we encourage shorter games in the class, actually.
Partially because of scope for your own development, but also partially because of that playtest problem
Speaking as the author of some of the longer and more convoluted Choice of Games, speaking also because the company encourages people to download the code and make their own, they have a Host of Games program.
You shouldn't start with long, convoluted anything.
You should start with-- seriously, seriously, seriously, seriously start with something short and simple and prove that you've got the language and you've got, you know-- and then branch out into things that will take you forever and lots of dinners to properly test
So what might be a good sort of first project?
Like a conversation or a one-page short story that ends up being branching, a lot more writing than that?
So a typical Choice of Game is divided into 10-ish chapters which, at least within the senior staff we refer to as vignettes.
I don't know if that terminology got to the website or not, but we call them vignettes.
So writing a vignette, which has-- I forgot how many words they're supposed to have.
And, you know, let's say four or five choices is a good way to get a handle on how it all works on the rhythm, on the technicalities.
And would go back to what I said earlier, which is when I joined the company I picked a vignette from the 10 that the partners had already laid out.
They asked me which one I wanted, and I picked the courtship scene in no small part because there was a similar one in the game that had shipped, and I had something I could work from.
So what are working towards?
It should be about this long
Got it
It should have about this many choices.
It should have about this pacing.
OK. All right.
So I can copy that.
Got it, got It.
All right.
Now I'll do something totally different.
Now I can do a whole game.
You know, you get the--
So you started kind of-- that's actually really interesting because a lot of people doing computer game development start as mod-ers, you know modify an existing game or an existing level, and you almost start with, you know, you mod the existing writing, but of course replace all the words
Hey, you know, a lot of traditional fiction writers start writing fan-fic, you know?
I mean, it's a thing, right?
It means that you have a-- I'm sorry, we're off in the weeds now-- but fan-fic in particularly means that the world-building is done, the character voices are set.
What you need to--
You aim for a voice, as opposed to make one up
Exactly.
And similarly, the constraint of the world are known, rather than things you have to impose, right?
So what you have to figure out is a plot that works with this world and these characters, and I'd say it's just a plot.
And then once you've got that down you can grow into sort of the other things
That's a good little way to do it, I think
So, something that Laura mentioned earlier was the whole standing around and listening to MPCs talk for, you know, minutes.
And it seems to me that in the Shock Games in particular, the answer to that would be audio logs
Yes
A lot more pervasive than just a shock series.
And I'm wondering has that started to become too much of a crutch to sort of get around this whole, we want to do all this exposition but we don't want to tie people down to a location, or do you find that there is a very specific use for that kind of [INAUDIBLE]?
So I think that Shock, in particular, it's trying to be sort of a role playing game which at its hallmark has dialogue trees and stuff, except there's no people there.
And so it's replacing dialogue trees with little monologue trees.
And so it's both exposition and also atmosphere setting.
So I think that actually-- I thought that worked very well.
Shock 2 had a lot of sort of cutscene-- in engine cutscenes where you would be going down a corner and there would be a window and then there would be something that happens on the other side of the window.
And that felt a little bit more like the Thief listening to people talk, because it would be this scripted thing to happen that you would have to watch, or not watch, and it would take place whether or not you were paying attention to it.
And then it always had to be built so that you couldn't interact with it because you didn't want it to be the case that you saved the person who was being killed, or whatever.
But yeah, the audio logs felt like, I thought, were a nicely atmospheric way to do text stumps
Yeah, the question was, is it overused now?
And I don't know-- I imagine, just like any trick you use in a game, if everyone does it, yeah, it'll be overused.
But in some games it's the right tool.
In some games it's not
Just following up on that, it seems to be a very similar function to, say, written books that you find around in a game like Skyrim or something.
It's such a global thing [INAUDIBLE] building it.
But because you hear it as opposed to read it, it's something that you can do while doing other things, you know?
You can go ahead and move the garbage cans while you're receiving all this information.
And I'm wondering, if that diminishes its value, because you don't really have to pay attention to it and you can just kind of like be the pack rat and collect all the audio logs without really caring about any of that.
Or is it more prevalent because there's no reason to avoid them?
It doesn't stop you from getting around
Yeah.
I mean, I think that I would listen to all the audio logs, but-- what is the-- there are some games that have had too many books.
I could not possibly read all the books, and so I didn't.
So I think it's almost more setting the right number for any particular player's enjoyment.
And I have no idea how to pick what the right number is.
I mean, probably somebody read all the books in Morrowind.
I did not
It's player-specifc too, I imagine
Yeah
Some players will read none of them, some players will read all of them
Yeah.
But the question of-- I don't know if you even need, want to try to say, well, we don't want to let the player not care about the things.
I mean, if the players don't want to care that's their choice, I guess.
I mean, that's a totally different thing, you know?
What if people are playing your game wrong?
You know?
Nothing you can do about that once it's out there, right?
Yeah.
You know, so there's the question of, should you be able to use the mods to let you just kill anything?
Is that playing it wrong?
Or does everyone get to pick how they want to play it?
Is using cheat codes evil or, you know, letting--
Helping you enjoy it more?
Yeah
Is there an analog of concept art in writing?
Like, for example you have a paragraph that explains what a character is that's not in the game?
Yes
Yeah
How often are you responsible for that as well?
So I guess that's why we design documents.
And those are traditionally not as well kept up as they should be.
So they often describe the state of the game as it was three months ago
[INAUDIBLE]
Those tend to be written more by the higher-level people than I ever was.
So there would be the overall design document.
Here is what we want the game to be like.
Here's what we want this level to be like.
But that is a good way of being sort of a tone reference or a gameplay-- here are the things we were aiming for, and you can always go back and look at that
Specifically character.
You said concept art, so were you thinking characters or were you thinking overall?
[INAUDIBLE] talked about concept art being partly to explain to the artists what they're going to do, partly just to inspire the team, which I thought was an interesting idea
Right.
OK.
I see that.
So the games I've worked on-- all the games I've worked on actually-- except for the RPG where I was in the world-building part of it-- have had character bibles separate from other design documents.
So there's the design documents that say Choice of Broadsides will have 10 vignettes and this is the one where you capture a prize ship, and here are the five main choices, and here's the one where you have the evil lieutenant over you and here are the five main choices.
And so there was that, and those do fall out of date very quickly because once you're in the code, you realize you want to do different things
And nobody likes documentation
Absolutely.
Absolutely
I'm in IT now.
It's still true
The character bibles though I have found to be more inspiring, right?
And so you sit down and you say that the-- you can do it in a few lines, right?
That the backstory of this character is this and the things they care about are this.
And I recently learned a technique-- Richard Dansky from UberSoft is a friend of a friend, and boy, is he a very, very, very skilled game writer.
And a technique he referred to was-- oh dear, what was it?
It was interview questions, right?
But the point is, you answer them in the voice of the character, not just what the character would answer-- their favorite-- their best friend is Susan.
But there's a difference between a character who says, "oh, you know, Bob, he's always got my back," and, "I first met Susan when we were in kindergarten and we were friends all through elementary school, except for that time when we had a fight in sixth grade over the"-- and, like, you know something about both of the speakers just from the way they answer
Most design documents are not necessarily inspiring.
But something that is possibly more so is the One Minute of Gameplay.
And it's sort of like the little narration of what you're doing as some random moment in the game, and why is it fun?
Because why is it fun is kind of the thing that you need to keep focusing everyone on.
You know, the point of the game is not to show you the cool art, or to make you listen to my really cool dialogue.
It's to be fun.
And I think that if you can write a One Minute of Gameplay which is somewhat inspiring for your designers and programmers, then that's really good
Questions?
Then I think I will follow up with one last thing and we may have already used up an answer here, but we'll all try it.
What can you think of that you didn't know when you started trying to write for games that kind of surprised you, and now you would think to yourself, yeah, I didn't know that, I did not expect that to be that way?
I think the screenplay thing was the weirdest.
Just-- a screenplay has these fonts and these margins and you can't go over the--
[INAUDIBLE]
Yeah.
I gather that you can submit academic abstracts and it has the same sort of thing, you know, no more than 800 characters or whatever.
But I think that was the strangest.
Like, I just had-- that was a very big surprise.
Most writing, I think, feels more natural
Sure
Screenplays are pretty specific, yeah.
Personally speaking, my biggest surprises have all been the positive ones
All right
I thought I wanted to write novels-- and I still think I probably want to write novels-- but I didn't expect this to be as artistically satisfying as it has turned out to be.
Now, granted, I'm in a pretty narrow writing field.
What I do is not typical.
What Laura's done is more typical of writing for the games industry.
Something that I think surprises a lot of people who have a background in traditional fiction and then transition into writing games-- because I've seen this a lot-- back in the LARP days we used to call it the oops I forgot to add the interactive part where you have people who are very, very, very skilled at plots and at character arcs and at story arcs and are not accustomed to having to leave room for the player to interact with the world.
And this is, I think, a problem you get specifically when you have somebody transitioning from non-interactive fiction to interactive fiction.
I think I was fortunate that I made those mistakes very early, when I was running LARPs when I was in college, and learned those lessons fast
Right
It's sort of-- at least at that time doing Quest at Wesleyan and it was one of the typical first-time Jam mistakes, was putting too much on their staff members, their non-player characters and not giving enough room for the players to, you know, play the game and affect the world and impact what happens.
You can't impact what happens because we have this scene for the end.
No.
It's not a play
Right, right
I have another answer
OK, excellent.
It is awesome how cool your writing sounds when it is spoken by an actual professional voice actor, and I did not expect that
As opposed to programmers?
Or by me, like when I say it in my head.
And then, you know, there was a little ritual at the end of Thief where the Dark God is doing this ritual and I was like, well, he's doing this thing, he has to say something.
You always have to have a poem for rituals.
So I wrote him a poem to be saying.
And then the actor for Constantine came in and spoke it in his Dark God voice, and it was awesome.
And then even better was when it was translated into French and spoken by the French Dark God, and it still rhymed and I was like, they translated my doggerel into the doggerel in French!
Oh my god!
That was neat
All right.
Well, thank you both very much for coming to share your experiences with us
You're very welcome.
Thank you for having me
Welcome.
[APPLAUSE]
I'll stand here for your microphone
Yep, thank you.
So, it's 2:08.
We're going to give you a few more minutes to take a break.
They're going to be here.
How much time do y'all have?
Do you have any time after?
Sure
Yeah
So, if you'd like them to playtest your game, just come on down, grab them, bring them to your game.
If you'd like to ask me questions personally, please do just come on down.
The rest of the time is for you.
[STUDENTS CHATTERING]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Schedule for the remainder of the semester.
Today is a short week.
Normal day today, slightly different day on Wednesday.
I'll talk about that in the lecture today.
Basically we hope to have the official class ended by 2:00pm.
So 1:00 to 2:00 will be class on Wednesday.
After 2:00, working in your groups or taking off early if you need to take off early for a flight or whatnot.
So it's up to your teams, whether or not you're working in here or not on Wednesday.
This is also going to be the last lecture by us, I believe, on the schedule.
Next week we've got two guest lectures.
One, on December 1 we've got a panel from the Indie Game Collective.
They're going to be talking to us about the business of making Indie games.
Two people who have their own businesses, their own studios, and how their studios work together.
So that should be interesting for you.
And then on December 3, on that Wednesday, we have Sean Baptiste from Fire Hose Games talking to us on marketing, communication, and community management for games.
Because we're seeing you can't just put out a PR release and buy some advertising, you need to build a community if you're making games professionally.
You need to find your players, because there's lots of games they could be playing besides your own.
So he'll be talking to us about that.
The week after that is entirely based around your presentations.
So December 8 is a rehearsal in class, live reversal with your game.
If your game works on that date-- hopefully it will-- but definitely your slides and your spoken presentation.
We will give you feedback, you will then take that feedback, improve your presentation, improving your grade, and give it as the final presentation on Wednesday, December 10.
At the end of the class on December 10 it'll be open play.
So after all the presentations are over we're going to have some guests here.
You'll be able to open your laptops and play each other's games and have guests play your games.
So that should be fun and exciting, and maybe a little scary, which is good.
So plans for today then.
I got caught on that-- not 5 minute, but 2 minute presentations.
Basically short, informal presentation just like we did a few weeks ago, I believe.
We'd like to know about the status of your game, what features you're working on, what features are not being worked on right now, what features might be dropped and cut, and I'd also like to see your game.
So if you have got one person talking, maybe another person showing the game, maybe playing the game.
This could be what the game looks like today.
This could be the version of the game you tested last week, but have something show-able, something visual for us to see so that we can-- when you're talking about your game we understand what you're talking about.
With no slides, of course, for this presentation.
For presentation on 8th-- on the 10th that is more formal, slides, all the good stuff.
There's a description about that in Stellar and we'll talk more about that next Monday.
After that I'm going to give a short, about 30 minute, lecture on cutting features, some strategies on how to do it, why you should do it, things like that.
Might be obvious, may not be obvious.
After that is work in class.
Pablo is here to give you feedback on your games, so we'll be giving you feedback on your presentations.
If we have questions about what you just talked about, but also he'll be going around and available for each of your teams to talk about the status of your game and what's going on with your game.
That is class today.
Any questions?
All right.
So it is 1:10.
Are all the teams feel ready to give presentations right now or do you need a couple minutes?
Couple minutes
Couple minutes?
All right.
So we're going to give you 5 minutes.
1:15 I'm going to come back up here and Snap is going to start
All right.
Hi everybody.
I'm Snap-- I'm from Snap.
I am not Snap.
I'm Tej.
And so we've made a lot of progress on our game and we've been able to run it a few times, but it has not changed a lot since probably anybody who's played the game has seen it.
So I'll just go ahead and play one round of the game just to show off what we currently have, and then I'll talk a little bit about what we're planning to do in the future.
So everybody from the team, you want to play?
And can somebody start the game?
I got it
All right so the way this game works is we're going to throw out a topic.
In this case it's animals.
We're all going to try to enter as many animals as we can think of, and we'll get points whenever two of us enter the same animal.
All right.
And they can obviously all cheat, but let's--
Wait, we already started?
Yes, we started.
Obviously I would not be showing this normally.
This is a little bit easy to cheat on
[INAUDIBLE]
Although we've had some other forms of cheating.
When Andrew was here, they just entered A, B, C, in a row next to each other.
Anybody want to help?
Cow
Cow.
I guess this is also cheating
Lemur.
Snail
Scorpion
Amoeba
Jaguar
Chicken.
[INAUDIBLE]
Pig
Paramecium
Veal.
[LAUGHTER]
Veal is not an animal.
I guess baby deer is
Geese.
Human
All right.
So I think that gives you an idea of how this game plays out.
Can somebody stop the game?
I got it
So right now we have--
[?
Yes.
?]
[LAUGHTER]
So right now we have the game completely manually controlled.
And this is because current use case-- in our current use cases for the client we've been playing the game with a limited number of words input and without a time limit.
So we let the client just implement whatever time they want by starting and stopping the game appropriately.
But we plan on also supporting an actual strict word limit as well as a natural time limit that can be displayed on the front end.
We initially went with Phaser because we had a much different idea of how we would represent the game.
But we ended up focusing more on words and, like, a more bare-bones representation.
And so we're actually going to remove Phaser.
Right now this is a Phaser window, but it's not really helping us at all.
We're just entering words into a list.
So we're dropping all of that and moving to just normal CSS, HTML.
We are also planning on-- we've experimented in our thought processes about what sorts of ways we want to show scores and activity.
We want to be able to show not just what words you're scoring and submitting, but also when are other people submitting words, when are other people scoring, how am I doing relative to everyone else.
So we have a couple ideas around that.
We also would like to show you a word cloud in real time.
We currently have a word cloud for the back end.
So this is the game we just played.
And this is a-- and this is something that you can watch actually as a moderator, while you're playing, in real time.
And then here's another game which is an actual game that the client ran in Berlin a week or so ago.
So that was kind of cool that we actually were able to do a real playtest with the actual client in the real world.
But in the future we definitely want this to look more interesting and to feel more competitive.
We want people to be able to see what else is going on in the game, whereas right now you really can only see your scores.
Any questions?
[INAUDIBLE] questions or comments, [INAUDIBLE]
What's your biggest risk right now?
Like, what's the thing that's most catastrophic?
I think since we're switching the front end over to sort of new technology we need to make sure that a, that transitions over well, and b, that we have enough time to finish something new and useful rather than just reverting back to what we have now
How does Snap happen?
So we have a server that is receiving all the words.
And whenever it gets a word it adds it to a list of words for a player.
And then every time it gets a word it also checks against all other lists
So it's basically any other player types the same word?
Yes.
In the same game.
So for example, the game is now over.
If you started a new game, that's an independent word list.
Yeah?
Just for curiosity, so let's say you put in "dog" and [?
Priam ?]
put in "dog" and then Sam put in "dog"--
Everybody gets one point right now.
We're thinking of changing that and letting people get more than one point, or maybe even making it configurable.
Most likely-- intuitively it seems that you should get more if more people have entered the word
Thank you
All right.
[APPLAUSE]
Next up is Awesome Cholera.
Yes, that is not the real title
While Awesome Cholera or other real title shows up, we will have a long conversation about details, but I'm very excited about the fact that it's already being used and working.
And now the question will be how many features you want available to the [INAUDIBLE] player.
And that selection process will be based on both client need, playtesting, and also your ambition on how much can be accomplished in this time.
[INAUDIBLE] STUDENT 1: Do you want to do the talking?
STUDENT 2: Yeah.
So this was the last build to the game we playtested.
We hadn't changed a lot.
We tried to add instructions so that people kind of knew what was going on.
And this is running, right?
Can you click Start?
So right now this is what people saw.
they saw their villages, and some health bars.
It was still kind of unclear.
And over here you can see, like, the text is still really small.
And when people clicked something to do education or something they either thought that, like, when they clicked that button it instantly did it.
They didn't know that was a button-- like Educate About Soap was a button they could click.
Also we had this really terrible pop-up bug that we're still trying to fix, but hopefully that'll be fixed.
And so because we were having so many problems with the GUI we finally sat down and tried to redesign it.
And so I haven't actually-- we haven't done this yet, but here are some really great drawings-- A plus, I said-- of what the new GUI will have.
So above each locality we're going to have this little box.
I kind of stole it from [?
Civ ?]
and a few other places, but-- So it's going to have the name of the locality.
And it's even worse because it's going to look different from what this is because we've already talked about how we want to change it.
And so behind the work locality is going to be a bar, and the bar is going to have a percentage green and a percentage red.
And the percentage green will be the percentage of people in that locality that are healthy, and the red will be the percentage of people that are unhealthy.
Above that-- sorry.
Above the word Locality, there's a little arrow that represents the infection rate, which is something-- a lot of people used that word and thought that was the information they were getting, but they weren't.
They were getting the percentage of people infected.
And so a lot of people wanted to know how's the infection rate being changed.
And so we're going to have an arrow.
And right now it's just going to be, like, up if it's positive, down if it's negative.
But we want to also change it so that they can tell the magnitude of that as well.
But we're also trying to keep it so that people aren't getting too many numbers thrown at them.
So we're still trying to iron that one out.
Below the Locality are a bunch of symbols representing the different things you can do.
What we thought we were going to do is if you've implemented them they'll be there in color, and they'll be counting down so you know how much longer that will be in effect.
And they'll also have a gold ring around them or something to say, like, if you've educated [?
them ?]
about it which means it's more effective.
And then the other screen is the screen you get when you click-- there will be a little thing that says Aid Locality above the Locality thing for each of the different localities.
And over [?
on this ?]
screen-- [?
which ?]
[?
got kind of ?]
messy-- but it'll show all the things you can do instead of just the ones that are unlocked, because a lot of people didn't know that things unlocked over time.
So we'll have it showing everyone, hey, these are all the things and they'll eventually unlock, either over time or if you do a certain thing.
And there will be two columns, one for educate and for prevent.
And as you mouse over everything on the right-hand side of the screen, the description will come up telling you about the effectiveness and the duration, and giving you a description of what it's doing.
We're hoping that this'll make it more clear as to what everything's doing, and it'll be easy to read chunks.
So if they want to know more about it they can mouse over it and just kind of read it.
Or if not they can just kind of quickly say, hey, this is pretty effective but the duration's short so I'll just use this for a bit because it's cheaper and then maybe I'll use the thing that's even more effective and has a longer duration when I have more money.
The other thing on the map that I didn't scan yet is, on the map, on the very top of it, we're going to have a bar that tells you about what day it is and when you're getting money, because a lot of people were like, why am I getting money?
And they didn't know time was passing.
So we want to make all these things clear.
And so I think the GUI redesign will help with that.
Hopefully we should have it done after Thanksgiving.
But yeah.
That's where we're at right now.
Any questions?
I have one, but you guys first.
Anybody?
STUDENT 2: Yeah
Is your goal for people to win this game and then have done the right things?
Or is it for them to lose the game and then, based on that know what the right thing they should have done is?
STUDENT 2: Right now the game's really hard, and it ends up you kind of lose if you just leave it alone, which is not what we wanted.
STUDENT 1: As you saw, earlier, it just went to game over.
STUDENT 2: We want them to realize the different things they can do.
I don't think it's going to be superbly challenging right now, is what I'm imagining.
Because you don't want people to do these things and then be like, oh, these things obviously don't work because I keep losing the game.
We want them to know that, like, hey, if I do these things it will help.
And so that's what we're trying to get across.
STUDENT 1: Another idea I think that we're not communicating effectively but [?
is ?]
[?
an ?]
idea we also want to communicate is that the position on the map is important.
So like if the village is-- upstream villages-- stuff you do there compounds to the downstream villages just because, like, the water, you know-- if you're not treating the water correctly in an upstream village that water is going to be contaminated down to the downstream village.
And at the end we also want to communicate through this game is that, like, if you focus on the upstream villages you'll have better results in the long-term because--
So I guess how would I discover that?
Is that just playing the game many, many times?
STUDENT 1: Exactly.
So that's something we have yet to communicate.
And we should figure out a way to communicate that to the audience
So thank you.
Good luck reconfiguring the things that need to be reconfigured.
My intuition is that you would be better off [?
amputating ?]
choice, and leaving somewhere very visible [?
other ?]
that says, these are other things that could be done but are not represented in this game.
Because otherwise it will be just too [INAUDIBLE].
And unless you make a really awesome game that people want to play and play until they've tried everything, it can be too intimidating.
STUDENT 1:
One of the options for your consideration-- because right now you have the options of soap and the latrines and vaccination and so on-- you could reduce it to the things that the player, the macro-decider could do in terms of public awareness campaigns.
So it could be about [INAUDIBLE] the water, about washing your hands campaign and maybe one more thing.
So limited to three things that are of the same kind-- you know, radio advertisement or sending people to [INAUDIBLE].
And maybe less villages.
Again, just reduce choice.
Reduce features so the gameplay experience is more engaging.
I just want you to know this is also applicable to the other Cholera team.
The Ghana team is now really interested in promoting hand washing because it's beneficial both to cholera and to Ebola, which is a serious threat in the area.
So you are [?
already know ?]
doing what you're doing and if hand washing does not fit in the game, that's fine, OK?
You are doing what we asked you months ago, not what we say now.
But if you find a way to make hand washing fit as one of the choices, it's much better than having three vaccination choices and two moving waste choices and stuff like that.
STUDENT 2:
Good.
And good luck.
STUDENT 1: Thank you.
STUDENT 2: Yeah, thank you.
[APPLAUSE]
Hello Waves.
STUDENT 3: Our game is called Hello Waves.
We don't have the title screen in yet, but the idea is still forecast-based financing.
So it's a little washed out on there.
But if you can see there are these five different sand castles with workers associated with each.
And up here in corner we have a forecast of what the water level will be like in a couple days.
This is the water level right here, and throughout the game it'll rise and fall.
And the idea is not to let your workers get drowned by the rising water.
The other aspect of the game though it that you have a certain amount of supplies and whenever you move someone from their castle in order to prevent them from getting drowned, they will consume supplies from that stock.
And if you don't have enough supplies then they all take damage.
So as you can see, right now everybody's in their own castle.
If we click Next Turn the water level will change and we'll see that the forecast will update.
We have a range of values on this forecast, a high prediction and a low prediction, and if you mouse over any of the castles you'll see a red line appear on the forecast just to help you orient what heights are on the forecast.
Or if you're looking at the forecast, what it is in the real world.
So you can see the water move.
For these couple days right now the forecast is pretty low.
If we see that's kind of getting close and we're worried about someone we can click and drag them to another castle, and on the next turn they'll move over.
You can also click people to toggle between building and gathering.
It's very hard to see the status text right now because it's small.
We threw that in quickly for right now.
But he is set to building and these three are set to gathering.
So we'll see that each turn these three will gather one supply each.
This teddy bear will consume one, so we'll see supplies go up by two, and victory progress will go up by one.
What we have planned for the rest of the game is we've made a lot of progress in terms of the intuitiveness of the game, once you understand it, but right now, there's a lot in the game that isn't explained from the get-go.
And so if I give that spiel to anyone they can play the game fine and they do pretty with it.
But if you just open up the game, you're lost.
So a lot of our work is going to be on the UI, of making things more self-evident and making sure that people can understand what's going on and what their actions will do without me having to stand there and tell them
Any questions?
All right.
So I'm going to ask you again, what is the biggest risk going forward?
STUDENT 4: I mean, I think in terms of our technical implementation we've already got good playtests of our work done and we have, like, all of our major graphics in and things like that.
So I think our major risk going ahead is not being able to explain the game properly.
And so even if we have finished product it may still not be playable if we don't write good introductions
All right.
So on that front think of incremental additional features.
So the games with the super simple, even maybe too boring choices, but then a new choice arrive.
So the learning be playing can be staggered in a way that is [?
enjoyable ?].
Good.
Good luck to you, and we'll talk more about choices later.
I should say that this high versus low prediction is something that I wish scientists did, like you're doing.
Because most people say one value, then it's not that value [INAUDIBLE] earlier that [INAUDIBLE] forecast.
Good.
Thank you.
[APPLAUSE]
Heat Wave, come on down.
STUDENT 5: So the main things that Heat Wave is focusing on right now are playability and education.
As you can see this is our start screen.
It doesn't have any instructions.
It's very lame.
We're completely redoing that.
We're also redoing our end screen and our forecast screen because we want visual consistency and just overall improved playability by making the gameplay states more clear and more connected to each other.
The other thing we're working on is we don't currently have an explicit end condition.
You kind of just play for however you want to play, and we're going to add in like a something happens that eventually too many people faint, get heat stroke, the game's over, so you have more of an incentive to play well.
And then finally something we're working on is user investment.
So we are actually-- where's my mouse?
Sorry.
We are currently working on redoing pretty much all of our character graphics because we think that having better character graphics will make the game more intuitive.
So one of the ways you play this game, if you haven't played it before, is you have a bunch of characters-- I'll show in a second-- and you click on them based on different characteristics to try and offer them water or tell them, maybe you should go inside, it's really how out today.
And different people will react differently to you.
And right now you have a bunch of characters-- where's my mouse?
And I'm going to play this.
They all look pretty much the same.
And we have labels-- [LAUGHTER] Uh, you're already laughing.
Did something--
There was a flare.
STUDENT 5: Oh, yeah.
So I'll talk about that in a second as well.
[LAUGHTER] So we wanted to increase emotional investment in this game.
So one thing I talked about was an explicit end condition.
The other thing is-- we know that setting on fire is extreme and, at this point, kind of funny.
We all just laughed.
It's not meant to scare you, but it is meant to make you care.
Before, we just had people disappearing.
And, number one, people didn't know that people had disappeared on the screen.
Players were like, what happened?
Suddenly there are less people.
And then they'd get to the newspaper scene and they'd say seven people have fainted.
Nothing.
So we added the fire just as a somewhat comedic element, somewhat just emotionally investment element.
I don't want my people to get set on fire, you know?
So that has to do with playability.
And also the same thing with the graphics and the fire, is education.
We wanted to make this stick with people long-term and we thought that maybe having, like, a more surprising reaction to them not doing anything would make it stick with them.
Something else we're working on [?
with get ?]
education is we need better and more dialogue.
So we redid the backend of our system for dialogue, and it doesn't-- uh-oh.
Sorry.
I'm really bad at this.
We redid the backend so that it would support better choice trees.
We haven't actually added the content yet, which is something we're working on right now, but we have the whole backend working so that you can make decisions that actually really do effect what happens next.
Something people complained about throughout is they felt like their decision weren't having an impact.
That's going to be a lot more clear now.
And then finally we're implementing on setting up longer-term choices.
So right now you can click on people and you might talk to them and they might not accept the water, they might accept the water.
You might have a discussion with them and make them drink.
But we wanted to have longer-term choices.
So what if you wanted to set up a public water fountain or something like that?
And another thing that we're talking about is setting up umbrellas and it might be worth more of your time to just, you know-- the forecast for the next day it might say it's going to be 108, but today it's only 80, so let's set up a water fountain today so that tomorrow all the people can just get water and be a little healthier.
So that's the two things we're focusing on, playability and education.
And we still have a lot of work to do, which is obvious, but we've done a lot of the backend work already and we think it's going pretty well.
Any questions?
Questions anyone?
STUDENT 5: Yes?
[INAUDIBLE] STUDENT 5: Oh, Julia did this.
Isn't it pretty?
That's dope
Thanks.
STUDENT 5: It is dope.
I agree
[INAUDIBLE] [LAUGHTER] STUDENT 5: Any other questions?
Thank you.
Quick comments.
I agree with the idea of making something like the flame that is visually compelling.
You may want to think-- if that's what you end up, fine.
But beware that the idea of fire is very much elicits a hazard, something external, whereas the nature of the heat wave risk is a combination of the hazard like the fire and the vulnerability of the player.
How susceptible are you, which is different from an elderly asthmatic person versus a healthy sports person.
So if you can brainstorm about how to depict that, which can be more evocative of the fact that it depends on the person and not just some external thing.
STUDENT 5:
I have no idea how.
Maybe it's not doable, but if you can, that would be [?
an invitation.
?]
STUDENT 5: And even if we can't do that we're going to try to implement that same concept through the dialogue by making the dialogue very specific, based on the person
Excellent.
Great.
STUDENT 5: Yeah, you had a question?
Yeah.
We found the fire funny.
Do you think everybody would find it funny?
STUDENT 5: So you have to think about the audience for this game.
The audience for this game is the Red Cross worker.
Now I don't necessarily think that all of them will think it's funny, but they'll definitely elicit some reaction, and I think that's what's important.
Because before in our game we weren't eliciting any sort of reaction, so it wasn't memorable.
The fire might be funny to some, it might be scary to others, thinking about people going on fire, I don't know.
Me, personally, I think it's a little terrifying.
But either way it elicits a reaction.
That's really what we're going for.
Any other questions?
One minor comment.
STUDENT 5: Yes?
Most of the rest of the world uses degrees Celsius as opposed to Farenheit-- STUDENT 5: Oh
So consider having a choice for [?
that.
?]
STUDENT 5: That's a great suggestion.
Thank you.
We'll take that into consideration.
You had a question?
And your biggest risk?
STUDENT 5: Our biggest risk is-- so we're not adding any features, but from our product backlog it's just we're redoing a lot.
So just the sheer amount and the breakdown of work-- it's just we're redoing a lot
And that's content work, or is it front and backend work still?
STUDENT 5: So we redid a lot of the backend already, but in terms of the adding of objects, that's more backend.
But in terms of the dialogue and stuff, that's more content.
So we have work on both sides
OK. STUDENT 5: Thank you
Thank you.
[APPLAUSE] Next up is Saving the Animal Village.
STUDENT 6: OK.
So I'm just going to start talking while Justin sets up.
We are a subset of Saving the Animal Village.
And our game is trying to teach cholera prevention through a series of minigames, three.
And then the story is that the mayor needs help deciding whether or not cholera is what is plaguing the city or if it's this monster that you encounter in the beginning.
And so the player will go through, explore the town, and try to convince the mayor that it is, indeed, cholera.
The most prevalent things that we need to fix, I guess-- the prevalent problems that we need to fix are-- when we playtested with Pablo some time back he noted that our minigames-- for example, the filtration minigame-- wasn't really something that would connect to our audience, which were children who were in affected areas, like Ghana, from the ages of 8 to 13.
And we had this, like, factory setting before and it was not something a child would encounter ever.
So what we're changing it to is, it's going to be inside a home and a mother's going to be the one that's bringing the water.
And it's going to be basically things that children that will be playing this game can relate to.
And we're still in the progress of changing that.
And we're also in the progress of adding conversation UI that's functional and that we can use while not alienating the lower range of our age range.
Because Pablo mentioned that the children in Ghana [?
who are ?]
8 years old might not be able to read as well as we want them to.
So we're going to have more visual instructions as well, which is important.
And that will probably be the deciding factor of whether or not this game will be successful.
I would say the thing that is keeping us back the most right now is the art, which we have to synthesize to match Ghana culture and communicate our message as well.
We're also attempting to implement a way to integrate the minigames more through our main scene, but we'll see how much progress we get through [?
and ?]
[?
we'll have ?]
[?
to correct ?]
that feature.
Any questions?
Just one clarification.
When I referred to matching, I like what you just showed way better than what was there before.
And I think if this is the end aesthetic of that minigame, that's fine.
My inclination is to think not so much matching to Ghana culture, but making it more adequate or suitable.
So if it looks like science fiction to a Ghanaian child, that's fine.
There must be someone from Ghana around the Boston area.
Send an email, talk to taxi drivers, I don't know.
Just like you're doing playtesting with kids, try to find someone from West Africa, if not Ghana so that they can tell you, no, cats are actually spiritually poisonous or whatever they may say.
I have no idea.
But you want to get some kind of feedback.
I love cats.
I hope they're not spiritually poisonous.
Good.
And then there were many, many games.
We will converse later, but this core idea I think is a very useful one.
I'll be keen to learn more about how the game can be adjusted in terms of speed of learning and the speed of-- as you master it and it becomes more complicated because of things moving faster or some kind of additional complexity.
But good.
Well done
So biggest risk?
Are you going to say it's art?
You mentioned the art.
STUDENT 6: Yeah
What about the art?
Is it the UI art or the representational art that we talked about?
STUDENT 6: Yeah.
So it's mostly making sure the representational art aligns with-- what we think of it aligns with what our intended players think of it.
And also just, like, we have so much art to produce.
Like, getting it done
The gameplay is there, all the buttons work, it's just getting stuff onto that?
STUDENT 6: Also the conversation UI, but our biggest risk, I think, is the art.
STUDENT 7: It's mostly functional.
There's a few bugs to iron out, but the structure's all there
OK. Last questions?
Thank you.
[APPLAUSE] All right.
Any generalized feedback you want to give the whole class?
Yeah.
You're now in a place where you have on one hand more to do than probably you think you can, but also more temptation to do more things, new things, explore new ways of representing, et cetera.
It will be a balancing act.
In order to get a passing grade you have to deliver something that works well enough.
I'll let them define that.
I invite you to remain inquisitive as to those simple-to-implement new ideas that can help something be really better.
So don't try to do something really better that require starting from scratch.
That'll be suicidal at this stage.
But that balance is a balance that you will be experiencing throughout the rest of your lives, whether it's designing a video game for the Red Cross or designing, you know, rockets.
Good luck with it all, and we'll have more personalized conversations after the half hour presentation.
Thank you
Thank you
And I'll be outside.
You let me know when
We're done?
Do you guys need to take a-- should we take a break?
Should we just keep going?
[INAUDIBLE] 10 minute [INAUDIBLE]
Yeah.
So we're going to take a quick 10-minute break.
It's 1:50 right now.
So 2 o'clock.
If you want to talk about and synthesize the feedback we just gave you, do that right now.
On the presentations you just gave.
You did really well.
You knew the problems that you had, which is good, because I was worried on playtest that you might not know some of the issues that you were seeing in the playtest.
So that's good.
We're really glad to see that.
The big challenge I see for your future is planning for fixing those-- responding to those problems that you know.
There's just a lot of unforeseen stuff that can happen between now and the end of class, and that is what my talk is going to be about, is how to plan for disaster.
And part of that is, just don't have the disaster.
Cut it from the get go.
So let me just make sure this is up.
Next.
All right.
You have deadlines.
This is the creative process.
Deadlines happen at the end of the process.
Hopefully work happens before the process.
In this class your deadline is the last day of class, right?
Double check your Stellar, you might be surprised.
In the real world some of your deadlines are going to be demo days.
E3.
Anybody heard the term E3 demo?
And soul-crushing-E3 demo, and studio closed after failure of E3 demo?
Basically, show your game off to a bunch of press and hope it works.
Usually that means don't allow any press to play it.
You just play it for them.
And by playing it for them, that might be just showing them the video while you manipulate the controller.
You don't get to do that in this class.
Other terms you might hear from the industry.
Going gold.
It's not just for disco.
It is-- the Gold Master there on the left side, that's a game that went gold and actually was canceled.
It got all the way to Gold Master, and then they said, no, we're not going to ship it.
We're not going to produce it.
It's just not good enough.
That can happen.
Ship dates.
Back in the day games came in these things called boxes made out of cardboard with printed graphics that were in interesting-- Eidos would make them in these trapezoidal things so they wouldn't quite fit on the shelf.
So you would see-- that's an Eidos product.
Thief-- were you on this one, Thief, The Dark Project?
Yep
Yep.
Tomb Raider as well.
Again, you don't exactly have a boxed ship date, but you have a give us a playable version of the game ship date.
So how much time do you have left?
December 10.
You have 17 days left between now and then to finish your game.
Well, let's assume you don't work on Thanksgiving.
Anybody going home to see family or anything?
A good portion of you.
Are you going to work while you're with your family?
Maybe you're going to try.
Maybe you're in a turkey coma, or veggie turkey coma for me.
But what about the presentation week?
So December 8 and 10, you're actually in class working on your presentations.
Hopefully your game works by the start of class on December 8.
Hopefully.
We're actually going to recommend that you do not change your game on December 8 and December 10, and you do a thing we call "soak," which I'll talk about later.
It's totally up to you to do it.
I offer it every year.
It's almost never taken.
But I'm offering it.
So actually, was that 13 days or was it really 30 hours?
You've got 12-- this is a 12-hour-a-week course.
I took out the time that you're actually spent here in lecture, that I'm stealing from you.
Sorry.
I like talking sometimes.
So you really have less than 1 work week to get all this stuff done.
So think about that when you're thinking about your projects.
Can you do all this in one regular professional, less than 40 hour work week?
So that's scary.
How do you meet your deadline?
Do you crunch or do you cut?
Who's heard the word crunch before?
Just raise your hand it you've heard it.
All right.
Raise your hand if you do crunch right now, if you are actually in the middle of crunch, not even on this class but in other classes.
Yeah.
As students, you already know how to crunch.
You do this.
This is your daily life.
And it's expected in some cases, it's some problems with the system, guess what?
The same problems in the game industry.
There is a way to do crunch-- to do a viable crunch.
And that's actually what next week is probably going to be for you, is your crunch week.
So ways you can do crunch and ways crunch is viable in the actual, real world industry.
You spend a limited time period in crunch.
So if you hear about death marches, that's not a limited time.
That can be months, years spent working 80-hour days on some studios, unnamed, in Australia.
We're recommending spending one sprint.
If you do a planned sprint we say, in advance, that we are going to crunch on this week.
That means we're going to spend that week picking up the slack and getting through a bunch of stuff and we're going to work a long period of time.
And it's going to be awesome.
Because guess what?
It's voluntary.
It is team-driven.
We decided, as a team, to do this.
The management did not say, you are going to crunch right now.
And in fact, a lot of game industry would say, that's not even crunch.
You're having too much fun.
That can't be crunch.
For them crunch is compulsory, unpaid over-time.
Three words that I despise.
So if you are going to crunch and it is voluntary and you plan for it in advance, you would follow it by deep soak.
Deep soak, in this case, means test, play, itemize bugs.
It means you are playing your game, you are understanding every subtle nuance of the user experience of your game.
You probably have a very good understanding of the programmer experience of your game, of the artist, of the producer, of the person on the team.
You know how your game ticks from your point of view.
But you don't yet know it from the players point of view, even after focus testing, even after seeing other players play it.
You probably haven't played your game enough to really understand what's going on.
What it actually does from the get go.
So you play it a lot, you itemize bugs, you prioritize bugs, you write bugs down, but you don't actually fix them.
You just see what they are, see how bad they are, decide whether or not afterwards you want to fix them or not.
All right.
So that's how you do crunch.
I give that to you.
You will probably do it.
That is OK. Good luck.
But, advanced course.
If you want to cut features, this is how you cut features.
So from now until December 8, ask yourself, can you ship your game right now?
That's part of the aspect of sprint, right?
At the end of a sprint you should be able to ship your game.
It's a viable product.
I can send it over the wall, someone can play it, and it is good and ready to go.
In order to do that-- and in this talk I'm assuming that, in order to do this, you're going to need to reduce the scope of your game because you need to ship it soon.
So what can you cut?
Some developers say these are the only things you should cut.
Features that are not fun and features that do not advance the game's goals.
So play the game a lot.
Decide whether or not those features are fun, collect evidence for it, have a theory about what is fun or what is not fun, identify those features that go towards that theory and remove them.
Just don't even continue working on them.
In this case you're thinking about the game from the idea of your player and your stakeholders.
These are basically features that if you spent more time, it would not be time well-spent.
But there's some other ones.
There are features that are too big to be finished in time.
Really take a realistic look at your product backlog right now and decide if you can finish it in time or not.
Could you cut those features?
Just give it a moment's thought.
Think about it.
Features that just don't work.
Why bother fixing something if it's just going to stay broken in two weeks?
Can you cut it?
Can you just remove it?
Does it change the game or not?
Can you even fix these in time anyway?
You just have to look at it and you have to analyze it and decide.
So how would you actually go ahead and do that?
Hopefully you can say yes to these things.
Is your product backlog currently prioritized right now?
Did you estimate your features well?
So did you estimate the size and the time it takes to create the features?
And did you create good blocks of work in your sprints?
And my guess is, probably not.
And that's OK. We talked to you about Scrum, we talked to you about Agile.
The thing is, Scrum is not a process.
Scrum is people doing a process.
You could be the best person, the best programmer in the world and be put onto a team, and still have all the problems that you're having right now because the team doesn't know each other that well.
That's one of the things we try to do on the first couple projects, is get you used to working with the other people in the class so you can find people that you've compatible work styles with.
So that when you're in project four you have a chance at forming a good, solid team and having some team dynamics.
And remember when I talked about team dynamics earlier in the semester?
Would anybody say that they are in the performing mode?
Like, do you feel like you are performing well?
Do you have an engine going?
Do you see tasks get done?
Do they just fall off the list because, oh, that was done.
That was verified.
That was awesome.
Has that happened yet?
Probably not.
That's OK. Yeah.
A lot of head-shaking.
That's OK.
So we need something else we can try out, because that's not going to work.
And even in the ideal world, you're still going to have to cut.
Like, developers cut features every day.
Those are the tools they have to use them, but when those fail we have other tools.
So based on what I read from your project three, some people think cutting features means dropping features.
That's slightly different.
It's not exactly the same.
The difference between a cut feature and a dropped feature?
A cut feature has intention.
You've planned for it.
You're going to tie up loose ends around the feature after you've removed it from the feature list.
You'll actually even delete the feature from your product backlog, and feel really good about it, and not have to ever worry about it again because it's gone.
It's no longer going to be in there.
Dropped features are those things at the bottom of the backlog that just don't get into the game at the end and could cause major issues if they're connected to other systems.
So how do you have that intention?
You think through the dependencies.
Scrum does not have a good tool for identifying dependencies.
It's actually one of its weakest points.
It assumes that, if you do all the other stuff your dependencies will just fall into place because you've created really good backlog items and you've organized them well.
So this is one of those things where, if you had a couple more weeks maybe you would want to switch to waterfall or go into a Gantt chart or figure out a way to graph every little thing that's going on in the game.
And I'm going to say, don't do that.
But think about that and get in that mindset.
So take a look at your code and backlog for dependencies.
And here's one really easy thought experiment to take.
look at your biggest unfinished feature and just think what would happen if you just didn't do it.
So in one of the games I saw-- this team here.
One of these teams.
You two teams over here.
What's your biggest unfinished feature right now?
Either team.
Somebody just say it out loud.
Maybe Matt, while he's chewing.
Or somebody else
Our time to gather all the clues in our endgame
Time to gather all the clues in the endgame.
That's your biggest unfinished one?
What would happen if you just didn't do that?
There would be no purpose to the game
OK. [LAUGHTER] So is it still the same game?
Not really.
Would the other features feel its absence?
Hells yes.
All right.
So trace your dependencies, take a look at that thing, go down the line and see, what all the other systems that are tied into it?
This is where graphing could be helpful.
A little mind map, node map.
If I wasn't crunching and actually made this presentation this morning, I would have had that graph.
But can you identify the loose ends?
And then if you've cut those loose ends off of that feature, is your primary feature usable, understandable, and polished?
You don't have to do this for every single feature on your feature list, but identify one or two really big ones and just do a little thought experiment and see what would happen if they went away.
Because they're going to go away at some point in the next week and a half.
So you've identified a feature and you're going to say, we're going to cut this feature.
We're not going to drop it, we're going to cut it.
What exactly does that mean?
So think cut plus.
So cut the feature, replace it with a different, smaller feature.
Key word there is smaller.
Based on new knowledge you know your game better now than you knew it when you first made that product backlog.
That fits a similar purpose.
This is an OK solution.
It takes a little bit of time and you actually might be too late to do that right now.
An example could be, you've got a first person shooting game and you have rocket launchers, but you don't have animated missiles.
Cut the rocket launcher.
You have-- what else do you have?
You need another weapon.
You need another big weapon.
Maybe you have a ray gun.
Maybe you can already animate things on top of another character.
So you have a ray gun that if you press the button, anybody in front of the button has a little animation that vaporizes and dies.
You've got a very similar feature going on.
It's a cool weapon.
It's damage heavy.
You don't need to do that animated bullet, that animated missile, and you save yourself some time.
And you haven't hurt the game.
The player doesn't even know that that wasn't even there.
You didn't give the player a rocket launcher that didn't fire missiles, and just made people randomly blow up.
You've given them something different and new and exciting.
Next one, cut and redesign.
So take the same feature and then shrink it.
Redesign it for a smaller scope.
Again, based on new knowledge.
I'm playing the new Pokemon right now.
Pokemon, that really outrageous, complex, matrix of strength and weaknesses.
When you break it down it's rock, paper, scissors.
I'm trying to make a game and I want to out-Pokemon Pokemon.
I'm going to have modifiers for everything, I'm going to have even longer lists of different enemy types.
Maybe I just need to go back to rock, paper, scissors, do a thought experiment, create a short little prototype.
That's one thing you can do right now is you can create paper prototypes on your own.
You've got big enough teams that one person can say, let me do a really quick paper prototype of that feature to see what it would look like.
Let me look at all the previous features we have, make sure it matches, and come up with a little paper version of it and see, is that going to fit the same thing or not?
And if it does, great.
You've saved yourself a ton of time.
[?
Today in ?]
content, as we saw in the presentations today.
Content is ginormous.
Churning through that is really hard.
Lastly, cut and amplify.
Take an already developed feature and make it better.
So did you promise 32 guns for your shooter?
Could 20 just be enough?
And then spend that time you have with those 20 guns to make them better and more polished.
So I already went back to guns.
That's just where my head was today.
If you think, if you had one good, really, awesome weapon-- double sawed-off shotgun in Quake, maybe.
That was pretty cool.
I used that one a lot.
And then a bunch of poorly developed ones-- BFG.
I don't even understand why they even put that in there.
Was that in Quake or was it just in Doom?
Doom
But either way, though.
You rarely saw it.
It was cool for one second.
Maybe if that was a hugely important, expensive thing, maybe cut it and the player doesn't even feel it.
So in all three of these things when we're cutting the feature what we're thinking about is, what does the player see?
Does the player have-- what did I say?
Expected feedback.
If you drop a feature might the player still expect some feedback that they're not getting because that feature's no longer there?
If you cut it the player should never know that it wasn't there.
Here's a game where they dropped features.
This is one of our favorite examples from 2003, Big Rigs.
Where's my mouse?
Anybody played it?
[VIDEO PLAYBACK] -Hey, kids.
Strap yourself in for some action-packed racing.
It's Big Rigs.
18 wheels of thunder, and we got trucks.
Yeah, trucks.
Big Rigs.
Off-road traction.
More power for non-stop driving action.
Big Rigs
It's Big Rigs off-road driving.
You can drive off the road.
-Who knows?
Big Rigs.
Never lose a race again.
You're always winner with Big Rigs
You always win, every time you get through it.
There's no timer.
-Phasing molecular mechanics to pass through solids--
Whoa!
- --so as not to interrupt--
Is this the future?
-Nothing stands in your way when you're Big Rigs.
Rear-spinning tires with warp-drive velocity for inter-dimensional exploring.
Leave the game behind and exceed the boundaries of existence!
This game shipped.
This game was put on a disk, was put in a box, was delivered to your local game store and Walmart.
People bought it, I think.
-Big mother-fuckin' Rigs!
-Big Rigs.
[END PLAYBACK]
Yeah, that was advertised as the trailer.
I quickly found out it wasn't, but I kept it anyway.
Humor.
Oh, where's my mouse?
No!
[INAUDIBLE] speaker notes.
[INAUDIBLE] back.
Apologies.
There.
[INAUDIBLE] here, down there.
All right.
So that is the ultimate expression of dropped features.
And-- oh, and I forgot to copy this.
All right.
So another example of cut features.
This is The Sims 4.
This is a quote from the executive producer, Rachel Franklin.
Basically what happened was, when the game shipped-- well, right before the game shipped, players identified that through whatever they did-- I think through watching community forums, listening to what the developers were talking about the game-- they identified 89 features missing from The Sims 4.
89 features that were not in Sims 4 when it shipped.
How did they know about those features?
Well, they were in Sims 1, 2, and 3 but they didn't put them in 4, so they should be in 4, right?
Basically, she says "I want to put everything we've ever created in the last 14 years into a base game, but we simply can't.
What we have done instead is to create an incredibly rich, robust experience.
If we didn't build the core foundation but I gave you pools, that wouldn't be the right thing to do."
They didn't quite meet the player experience.
There could've been some things that-- maybe there were some problems going on the production, but they decided that they still needed a core game that could work because they could always add some of those features in later.
There are some people on the forums who say, no, that's actually not going to be working in the game engine because I'm the nerd on the internet knows everything and I know you can't do that, developer who actually made the game.
So that is a problem that they knew they were going to have.
They know that they have it when they ship the game.
They have to come up with a community response to figure that out.
And actually, that's something we'll talk about next week when it comes to community management, is when you're creating this game and people find out about this game, how do you then talk them down?
But actually, in the most part you'll be talking about bringing them up.
All right.
So we're going to cut some features.
We're not going to drop them.
So who decides which features are going to cut?
Who decides what and when, and then who's going to implement it?
How are we going to do this?
So the people who decide which features get cut usually are the stakeholders.
In the professional world that's going to be that executive producer.
For you that could be your client.
That could be your product owner, if you have a product owner on your team, if you have someone you consider a vision owner on your team, it could be that person.
It could default down to producer or Scrum Master, if that's the person we're saying that we're following their lead, they're going to make that decision.
It could be your game designer.
But it's going to be somebody on your teams.
One, maybe two people on your teams who are coming up at least with the what and the when.
Not everybody on the team knows the schedule as well as maybe the producer does.
But the who implements and how it's implemented-- that's the entire team's job.
This is why we have those long, boring Scrum meetings at the beginning and the end of each sprint.
It's because that's the time period when we have all the team together to make these kinds of decisions.
It's definitely not the producer and Scrum Masters' role to decide how you're going to cut that feature.
They're there to facilitate it and make sure that everybody who's attached to that feature can talk about it, can explain why, can give some argument, can give some defense.
Rather than giving the entire team the need to decide everything, one or two people decide the initial part and then the whole team focuses on just how we're going to cut those things.
And that's because how you implement it requires knowledge of both code and content.
It's more than a feature list.
A feature list can't do this for you.
It can do the top, it can show you the what, but never the how.
All right.
Last little section.
So you were working on a feature and your feature was cut.
You spent a lot of time on that feature.
Your roll was to implement networking.
And this is actually an example from two years ago.
But was cut.
In that case, it wasn't cut because of these things.
I don't want to cut it.
I want to see this finished.
I put some commitment into this thing, dammit.
I want to see this all the way through.
I feel guilty about cutting it.
I really let my team down.
I was just working on networking the entire semester, and it's not going to make it into the game.
What have I been doing?
I feel sad about cutting it.
It's not fair.
How dare you cut my feature?
You're going to see a lot of that, if you haven't already.
But that's OK. One thing to keep in mind when you're cutting a feature-- and this is one thing that we told that person-- was, your work needed to happen because otherwise you wouldn't know that we needed to know about your work.
You described this better, by the way
I'm trying to remember exactly what I said, but it's something to the effect of, sometimes the work that doesn't make it into the game had to happen anyway because you had to do the work to explore networking to find out if it wouldn't work or not.
Just like you had to do the work to explore several other features that did go into the game.
But you didn't know that the-- I don't know-- you didn't know which of those features were going to work out well.
But you were hoping that some of them would.
Some of them did, some of them didn't.
The fact that one particular feature-- the one that you worked on-- is not a reflection on you.
Not at all.
It's a reflection on the game, and the project needs to go forward
Absolutely.
So yeah.
Work that gets cut is never wasted work.
And that's actually a problem I have when game developers do their postmortems and post in the Gamasutra is they always talk about it as if it was wasted work.
But really, again, they wouldn't have had that information had they tried it.
Work well-cut is exploratory work.
It means that you cut it early enough that it was exploratory.
You got your feelers around it.
You figured out that was probably a direction you didn't want to go into, and you pulled back.
And actually you did that in the beginning with your paper prototypes.
I don't think your games now actually are one-to-one relationships to those paper prototypes you created at the beginning of this project.
And you threw those away, and it was OK.
So why is this important?
Because the team is more important than the product, which is, unfortunately, more important than you.
But that's OK, because the team equals you plus others in the same situation.
You are actually part of the team.
So just keep that in mind.
This is where I go to my Goodfellas-- was it Goodfellas?
There's no I in team, baseball bat?
You are on a team.
Everyone's contributing to it.
Everyone's doing good work for it.
All right.
so that's my spiel on cutting features.
Here is my-- what I'd like to do on Wednesday.
So Wednesday is going to be a short class.
We're just meeting from 1:00 to 2:00.
You'll have the remainder of the time to work in your group, in your teams you as you like, but if you want to take off, you can take off.
What we are going to do-- and it's going to be me, Phillip, and Drew, because unfortunately Sara won't be here-- is we're going to meet with each of your teams individually for about five, seven minutes tops.
What we'd like to do is look at an updated vision statement.
So between now and then take a look at the vision statement you turned into Stellar, compare it to the game you have right now, and modify it.
The vision statement is your publisher contract with us.
That vision statement that you actually finally turn in on December 10 with your game, is the thing that we are judging your game by.
If your game has all the features that are described in that vision statement, then aces.
If you created a vision statement that doesn't have the features are in the game, or if you don't describe the features well in the vision statement then we're going to use that as an understanding of what could have gone wrong with that game.
This should be a really quick thing you can do.
You can probably do this in five minutes today with your team.
You have everybody here right now.
You just had your presentations about your products.
So if you can take five minutes today, reassess your vision statement, take a look at your product backlog, change it.
We're going to talk about that.
You can re-turn that in to Stellar or you can bring in a paper version on Wednesday, so long as we can see it when we're sitting with you on Wednesday.
That's all we ask.
And again, like I said earlier, plan as if nothing is being done over Thanksgiving, just in case.
If work happens over Thanksgiving, cool.
You did good.
That's bonus work.
So just keep that in mind.
Any questions?
All right.
Pablo is back in the room.
And you're working on your teams for the rest of the day.
Thank you.
[INTERPOSING VOICES]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK.
So the way to this day is going to be set up is we've got one hour to go Philip, me, and Drew.
We'll go to each team, and we're just going to talk with each team about your updated vision statement-- you should have one by now-- your product backlog you just turned in, and your sprint tasklist you just turned in.
The main thing we want to talk about is just some more detailed discussion based on the presentations you gave, as well.
So what are you doing over the next week and a half?
What do we think the biggest issues are?
What are your plans for Thanksgiving, and not working during Thanksgiving, hopefully?
And we'll go in that order-- so Heat Wave, Hello Wave, Snap, Cholera, and Awesome Cholera.
It should only take about five to 10 minutes per team, unless you have more questions.
And then after that, you are free to depart or work in class.
We've got the room until 4:00, as well.
Before we do that, Phillip had some generalized feedback he wanted to give
So after looking through all of the state of the projects in terms of the deliverables that you handed us, when we looked through the product backlogs, we noticed something fairly consistent among all the teams.
And that was you know how you have these columns, right?
You have your serial number, your item number of your task, the description of the task, usually who it's assigned to, and then a priority list.
Well, first of all, everybody sent us this list sorted by serial number rather than my priority list.
That's not so bad, because we can always just resort it ourselves.
The problem was that the priorities was something like, task one was priority one.
Task two was priority one.
Task three was priority one.
Task four was priority one.
Task five was priority two.
And then it goes, two, two, two, two, two, three, three, three, three, three.
And that-- if you get granular enough, that might be fine.
But I'm a little bit worried that's not quite granular enough when it comes to your priority levels.
Everybody seems to have about three to four levels of priority.
And you get these huge swaths of tasks that all have the same priority.
So you don't actually know which one's more important.
Now, you might be thinking, well, they're all assigned to different people.
So for everybody who has a set of tasks, they have a priority one task and they know what they're supposed to be doing right away.
And that's true for the individual.
The problem is that if you've got-- I'm just going to bring this up.
If you've got-- OK.
So say this-- what the?
Oh, here we go.
Say this is your product backlog, and you've got a chunk of priority one tasks, priority two tasks, priority three tasks.
And each one of these tasks have been assigned to some individual person.
Maybe you've got something like all your priority one stuff done.
All that is cleared.
All the most important things got done.
We are fine.
So now you're somewhere dealing with all of your priority two tasks.
And some of them are done.
Some of them are not done.
Hopefully you're finishing off a priority two task before you get to your priority three tasks.
And so this is person A, this is person B, this is person C. And person C's done their job.
Person B's done their job.
And person A's still working on it.
How complete is your game, actually?
Your game's only as complete as person A is capable of finishing their task.
So if you've got like a giant list of priorities-- if I were to take this list of priority one, priority two, priority three and forced sat you down and went through the arduous process of actually giving each individual task a separate priority number so that you could actually solve it and rank it-- I'm going to do that.
Because I know it takes a heck of a long time.
But if I were to do that, what you'll find most likely is that you've got a bunch of tasks that are done.
You have an undone task, a bunch of tasks that are done, an undone task, and then some stuff done, and a whole bunch of undone tasks.
And the game's only this done.
You are not here.
You are not here.
You are here.
And you need to be able to see that as a team.
You need to be able to actually see, what is the highest, most important thing that's still not completed?
Right now, at the level of granularity that we're looking at all of your product backlogs, all of your product backlogs don't quite have that level of granularity.
But, it is possible that in your head, you actually have that level of granularity-- that you actually know in your team what that highest level undone task is, even though they all have the same priority.
They're all priority two tasks, or something like that.
So something that will be useful before you depart class today is to actually figure out-- off all of the things that are the highest priority and undone right now, what is really the most important thing?
And who's responsible for that?
In fact, all of your highest priority tasks should all be assigned to somebody.
And you should have a good sense of what order of priority they are.
Because not everybody who's been assigned a task is actually going to be able to complete a task before the next time you meet.
And the next time when you meet, when you actually check in and you discover some tasks are done and some tasks haven't been done, you need to know how much progress your team actually made towards completion of the game.
Otherwise, you may say, oh, we got five out of our six tasks-- only that one task that didn't get done turns out to be the most important thing, which means you actually haven't made enough progress on your game at all.
And you need to know that so that you can actually start to cut features, scope down, and do all of the things that we talked about regarding crisis control for your team.
So we'll talk a little bit more with each individual team about that.
But that's a conversation that you should be having within your own team.
Of all your highest priority tasks, which one's actually the highest priority?
OK?
All right.
So take a minute just to meet up with your team, and we're going to go to Heat Wave in one minute.
Also, make sure you've got a playable version of your game, because Drew would like to see it.
He was not available for the presentations.
Amelia wanted Drew to play, since he's played it during the play test.
And there might have been some stuff that changed
Oh, excellent.
That person's on fire.
I like it already.
People are dying, and you're asking me questions why?
Seriously?
You're dying, and I'm talking to you.
Why are you talking back when you're dying?
All right.
Cool
So, I just want to be clear.
Catching on fire is fainting?
Equivalent, yeah
Yeah?
OK. All right
It's a metaphor
It's meant to invoke an emotional response
Yes
It does that.
Oh my god.
They're all dying.
[LAUGHTER] Hello.
Go inside.
No.
OK. No
Ah.
Something you should know is the dialog is just-- the back end's working, but we actually haven't put the content in yet
Sure
The dialog will be much more educational and--
And have choices, which it currently doesn't
OK. Sure
You guys are going to want to--
So right now, we don't have an and condition, because the dialog is just--
Well, you eventually will
Yeah yeah
It's currently the Space Invadors style.
You will eventually die and condition?
Yes
OK, cool
So I've got the old vision statement.
Did you make a new version of that, and is that on Stellar right now?
No, it is not.
I didn't realize we needed it for today, because it wasn't on the homework assignments
Oh, yeah, yeah.
That's OK. Then just--
I'll redo it today
OK, cool.
Yeah, that's-- I should have put that on the thing
In therms of the task breakdown, its' actually pretty much the same.
And the overall vision I haven't looked at in a while, but I think most of our goals are the same
Yeah.
It actually looks pretty-- so, like, vaguely, when you're reading through the old 30 of game play, it feels like this game.
A lot of these elements are in there
We haven't had any major design changes in terms of that
OK.
So between Monday and today, has anything new gone on the chopping block?
Are any other features getting cut?
Monday is when the product backlog was due?
Yeah.
Yeah
No.
We caught secondary scenes, and everything else seemed doable, because a lot of it-- the only other thing that could possibly be cut is items breaking over time
Yep
Everything else is actually in progress, and just being improved upon

So they might not break.
It might be that once you've installed an umbrella, it will last forever, kind of thing-- or a water fountain, or whatever we choose.
But everything else is in progress, and I really think we're on track to finishing
OK.
So you said a lot of the content you have to get in is the dialogue.
Then you're also going to be doing some new artwork for the characters
Yes
Is Julia the only person doing artwork on that?
She is the only person doing artwork
Is she also doing dialog?
No
Uh, no
I'm doing the dialog content
All the backup stuff was for my UAP, so I was kind of--
OK.
But, I mean, are you still coding the dialog system for this game?
Not anymore
Unless it's debugging
So you're saying it's bug
We need more testing.
We know that
If there are issues, I will fix them
That was like the big red flag when Phillip and I were talking, is that one person doing two important things
We know that there's-- no, no.
The dialog is split between these two lovely people
Oh, yeah.
No, not the creation of the dialog
Yeah
That's not the thing we're concerned about.
It's the creation of the art with one person, and that person also being the person in charge of the back end for your dialog
Oh, yeah.
The back end--
Well, I mean, it was a lot of work, but again, it's like I was counting the dialog for my UAP instead, so I was definitely doing this fast.
It's not a big thing.
So the artwork is my, like, contribution for this project for this class
OK. All right.
So if you have enough time during the day that if that breaks--
I do.
I can get the art done.
I know that
and you're doing both.
OK. All right.
We'll see how that goes
Worst case, we'll have stick figures.
It'll be
Yeah.
Oh, yeah.
And from our point of view, it's fine
These people are fine
These people are fine
Unless they don't--
The issue with that is one of the main learning things about this game is choosing who you're going to try and help, and these people do not convey at all their characteristics, which is why we're redoing the art
That makes sense
And art is one way of doing it, floating dialog, floating box is another way of doing it, and floating boxes
We have labels
Are they-- they're broken, or?
We deleted them, because they-- they were just hard to read, and we didn't have them moving with the characters, so they would stay in place.
We could have made them move, but we just decided it was better to redo the art than try and make the little captions better
No.
Oh, yeah.
That's--
If you guys don't think I need to do the art, let's not do the art
Well, I mean, the art is awesome, but it is also a huge bottleneck.
And if you fall sick and then a lot of the reliance on your art to be able to convey that important information falls through, and the project fails.
If you don't fall sick and you get all your stuff done, your game just looks better, and it's awesome.
Right?
So
Yeah.
So, I mean, we have placeholders.
So if for some reason I don't complete it, we can just use what we've got now
But I'm just thinking about tinting the [INAUDIBLE], or something that like-- like just through software or something.
What else can you do that's a little bit like as a backup?
So with heat waves, one thing that happens when you get sick is you turn red.
So we could tint them red over time, and like have people starting at different levels of red.
But I think-- and that's actually something that we might be looking into separately.
Because something Pablo talked to us about was wanting to convey that people are getting sicker over time, and that maybe once you see, OK. Somebody's turning red or has an exclamation point, or some way of conveying that.
Let's show the person that they're sick.
The users need to know.
But that's something we're doing as, like, a secondary thing, because we want to focus primarily on making the game better as a whole right now
OK. That's cool.
All right.
Yeah.
And all the things we're saying are not at all, do this.
It's more--
Yeah.
No.
I--
Think about this
It's more we're flagging our concerns rather than telling you how to solve it
And prove us wrong, please
All right.
I'll prove you guys wrong.
Don't worry
All right.
Cool
The other thing we had was about tutorial and ramping up, right?
Well, actually more like ramping up.
So, right now, the game's frantic.
Right?
It feels like you have to be everywhere at all places.
And that's great for somebody who knows how to play the game, and also great for someone who's just learning how to play the game

So you already have this concept of this.
I don't know if difficulty is ramping up over this--
Actually, currently, right now, it's not
It's not.
We were planning on just changing the number of people that come up every day
Yeah, so maybe, I mean, just--
Starting with less people
Yeah, just starting with less people, or reducing the rate in which people, like, fall over or something like that.
Just like, how do you ease people into this situation?
That is kind of set, because that depends on temperature right now
Right.
So that-- it's--
So we could just have lower temperatures in the first few days
The first couple days.
We could definitely do that
Yeah.
Just think about how you're going to run people in.
That's definitely better than going to the expository tutorial route, you know, of the I'm just going to tell you everything that you need to know and hope you remember it when you play the game.
Ease people into the experience
Yeah.
In fact, if it's just the case that you have your high risk person.
And if you've got your-- really cool-- I'd like you to tell how that might map to reality.
And you don't need to tell me that they're at risk if they're the ones that keep dying like flies.
I'll be like, oh my gosh.
That's kind of a visceral way learn that lesson.
Is the game-- when you guys play the game, do you feel that you are winning it?
No.
[LAUGH]
No
All right.
So if you don't have it really high in your backlog, you probably should put it there.
You need to do the gameplay balancing first, first thing.
Because you need to play test that.
Even if you guys are-- you can use yourselves as play testers to some extent for difficulty, but keep in mind that if you think the game is hard, the game is impossible.
If you think the game is easy, the game is hard
Well, I think the game's impossible right now
All right
Which has to do-- I think we have, like, a bug where too many people show up, which I think is something you noticed also.
It's just like, they're all on fire all at once.
And I think we definitely-- we only intend to start with like, four or five people, and the fact that there's like 20 there is an accident.
We're fixing that
Sure.
And you could easily have it-- they could arrive over the course level, or whatever, if that's what professors--
Yeah.
Definitely
But, yeah.
I would say make sure that that game play piece is really high
Definitely-- game balancing
Have you all played Diner Dash at all?
Is it free online?
Yes
Yeah?
Yeah, I think I have.
Yeah
Yeah.
You should be able to find at least a video of it
Yeah
Or Tapper
Yeah
Oh, Root Beer Tapper
Diner Dash or Tapper-- they're basically time management games, very similar to this one.
Take a look at those just to see how they ramp up difficulty-- how easy it is when you first start compared to after about five minutes of play, and try to think of that.
It might be as a baseline
I had a question.
So, somewhere it says that we should have explicit end conditions for our game
That's an explicit end condition.
You died
Loss is an end condition
Yeah
You can--
Well, you don't technically ever lose.
You just lose people
Oh.
So there should be some point where I know that, OK.
I can stop trying right now.
I can try playing the game again.
We try to keep it vague, because every game is going to do it differently.
If it's A, the heat wave lasted four days, and you ended those four days of heat wave or whatever, and that heat wave's over, then you're done
OK. We were just going to do it by, like, 20 people dead or something like that
Yeah
That will work fine
That's fine, too.
Whatever is best for the project
You can actually do both, right?
If you keep this town going 20 days without losing x number of people, you win, otherwise you lose
That's an aesthetic choice, I think.
The modern game shtick is that all games have a win condition.
But classic games didn't.
They had a lose condition only
Yeah.
You could be like a Tetris kind of thing, where you just-- how long can you last?
But summer doesn't last forever.
If somebody goes for, like, you know, 200 days, it's probably not a heat wave any more
That's funny.
That's funny.
After 90 days, you're like, summer is over.
No one is dying of heat wave.
Wait for the sequel-- Winter.
[LAUGHTER]
Cold Snap
Maybe it's only winter all year long
Oh
That's true.
Some parts of the world actually is heat wave all year around
Mhm.
Singapore.
[LAUGH]
Cool.
Any other stuff?
No, that's it
Last thing for me, then, is product backlog item 20-- menu user interface.
Outside menu
Oh,
What is that?
Yeah.
So that's not actually well explained.
That has to do with the insoluble items, which-- Roy isn't here.
But Roy is working on it
It's kind of like the pods are like add water stations, add umbrella
All right.
Cool
Yeah
We just weren't sure what you were talking about there.
But that's enough
I meant to fix that.
I forgot
No.
These documents are for you.
As long as you understand what's going on-- that's why Phillip was saying, yes.
We didn't understand.
There's probably some things in your head that you understand.
But also, in case you didn't see that, then we wanted to go over that
Yeah
Yep
All right
And another thing-- so add water-- is it red on there because it's high priority?
No.
Red means it was cut
Ah.
Awesome.
OK, cool
And we like that-- cutting the features and leaving them there is fine.
Sometimes, as a morale, removing them is also good.
Have a party when you do that
Well, trying to--
But as memory, they're good for institutional memory, to know, hey.
We thought about this.
We cut it.
Later on, oh-- assuming you had another couple weeks.
We can go back to it
Actually, almost every project I work on, I now have a do not do list.
Like, not a to do list.
Like this is the thing I thought about, and I really want to do it.
And I'm going to write down here so I know I'm going to ever do it.
So later, when I think about, what about that?
Oh, right
You're not going to do it
Never do that
Never do that.
Because I don't have time.
And so I think even if you don't keep those on your main to-do list, it's cool to have them there for a little bit, just so you remember-- oh, right.
We cut that
Mhm

Well, thank you
Thanks.
And actually, so the vision statement isn't due until December 10

Is just we wanted to give you the chance to make an update now if you could, but this discussion is fulfilling that.
So, that's good
Cool.
Thank you, sir
All right?
Thanks.
Have a great holiday.
All right, Drew.
When you're ready, go ahead and play another game
Yes.
Oh, excellent
There's still some instructions which are missing from within the game

So I'll just explain those now.
So the goal of the game is to build up your victory progress to 100%.
And the way you do that is by setting your toys to build or--
Or gather.
Mhm
Yeah.
And you do that by clicking.
So the story is like you brought your toys to the beach, and they have these sand castles.
But then the waves-- like, if they cover the sand castles, the toys will then take damage.
And they can only get hit by the waves twice.
They take damage if they start underwater or end underwater.
And so in order to prevent your toys from getting swept under the waves, you can move them by clicking and dragging them to castles

And then they will move one castle at a time.
So this one will move here, here
OK.
It's pretty much the same as [INAUDIBLE] play test, then, the last I saw
Oh,

And then the last thing-- so you can hover over this to see what the predictions are, and you can hover over the castles
Oh, I see.
And they're no longer perfect.
Excellent.
I like that.
I like that change.
All right.
Cool.
I'm going to say, again, this toy is immune to water damage
But he does not need to be put away
Ah,
So while he's playing, did you update your-- did you have a discussion about cutting features after Monday, as a team?
Yeah
Did you cut anything?
We cut, like, kind of any sort of big changes where we would give people special abilities, or give any sort of long tutorial kind of like that
OK.
So you cut things you hadn't started yet, basically
Yeah
OK.
Anything that if you had to cut today, like could you cut anything right now, or does everything have to go in the way you've thought about the game?
So what we mainly have left is UI changes-- like make this a cloud it fits into the theme of our game, or add arrows to show where--
Where they're going to go, and how they're going
Yeah

And so a lot of that is necessary to make sure that our game is standalone.
But I guess the worst case scenario, we make that a blob of text, which would be, like, absolute worst-case scenario
Do you already have the feature to do a blob of text?
Yeah.
That would be our--
It's already in there, or?
Yeah.
That would be our menu screen, which we didn't show you, but
OK.
Cool.
OK. For your vision statement-- so I said if you have a new one today, we'd like to see it.
If you don't have a new one today, make sure you update your vision statement before you turn in the game on December 10
OK. Yeah.
We do have an updated vision statement now
Oh, you do?
Yeah
Cool.
Cool.
Then I'll review it.
So we review--
Should I submit it to Stellar though?
Yeah.
Submit it to Stellar.
Submit it under the December 10 one.
And then on December 10, if anything else changes, resubmit it on the December 10 one.
That's the one we're going to use and judge the game by.
The main thing I saw on the one you have now is that just the 30 seconds of gameplay is really, really vague.
So if you changed that for this new one, then you did good
Yeah.
OK. Yeah.
We made it a lot more concrete, now that we have a more concrete--
Great.
Cool.
That's exactly right.
So the biggest thing that Phillip and I saw was about your-- basically about the representational art, the quality spent on the representational art compared to other feedback
Yeah.
So right now, the graphics that you have in the game look really cute, and they're attractive.
They're endearing.
They make people want to people want to play the game.
And then your UI isn't up to that, right?
You have this issue-- and this happens in a lot of games-- where the quality of one asset or one kind of asset is showing how far back the rest of your assets are.
And this obviously means that you might be inclined to make sure that the rest of the stuff looks as good as what you've got-- things like the background, for instance-- probably the most important part of the entire game is the background.
But it also looks like the least visually thing that you've got on screen
That's true
And also things like when you want to instruct your characters to move around, for instance.
Like you said, right now you have to explain it.
But I expect to see some sort of visual feedback, UI feedback, when I have, say, instructed someone to move somewhere else
Yeah.
That's what we're planning to do.
We'll probably have the go to destination and the character goes--
And the danger right now is that your characters and your sandcastles and your clouds have set the standard of visual quality that the rest of your stuff now needs to meet, which is a pretty tall-- I mean, you clearly have the skills to do it.
The question is whether you have the time to do it.
And that's what I'm worried about
Sure
Because you don't just want people-- you want people to look at your characterize when they're trying to think about whether they want to play a game.
When they're playing your game, you want them looking at that background, and where they're going, and the quality--
So when it comes to a feature like-- OK, here's an arrow going from one place to another.
Do you already have code in place to do that, but just you don't have the art asset?
Do you have the art asset, but not the code?
Or do you not have any of it?
I think that was like our, actually, highest priority to implement highlight to implement by, like, the end of Thanksgiving weekend.
So that would be, like, our number one--
But you're starting from 0 and going--
Yeah
OK.
Cool
Don't they have arrows showing where--
They have arrows showing that they are moving, but not where they're moving to
Oh
And do you have to click twice on a character?
Is that correct?
No
No?
OK.
It's just a click and hold?
Yeah
Yeah.
But-- yeah
Yeah, it's a little difficult
The cancel is-- I mean, the cancel is intuitive to a computer programmer, but no one else.
Is there sound?
Uh, no sound
That's another one to be implemented
OK. And how far are you on sound?
Is it a--
We have potential background music, but we're trying to figure out which one we actually want to put in
Well, what about sound for I clicked a thing, and sound for--
I have worked on the sound, but we haven't put it in yet
OK. That's--
My recommendation on that stuff is don't worry about picking the right background music
Yeah
Put in stupid background music, and then you will hear it, and then you'll be motivated to change it.
But most importantly, you'll have proven that your code can support background music and click simultaneously, et cetera, et cetera-- which it can, because it's not hard.
But just be safe.
Put in placeholder sounds, put in placeholder art for whatever.
If it is, in fact, the case that you end up with your cool, cute toys being more visually exciting than your UI, there's another cheat you can do, which is lower the quality of your characters if you don't have time to improve the quality of your
Just pixelize them
[LAUGH] Obviously, improving the quality of your UI is better.
But again, you can cope with that.
But, yeah.
You do want people to focus on the right things
From a usability standpoint, we're going to be looking stronger at if I click a thing, it gives me an audio feedback that I clicked a thing, more than there's background music going on
Sure
Background music's an aesthetic element that we're also going to look at, but when we're ranking our priorities, usability is higher than aesthetic
Yeah.
Another thing is that, right now, for usability, there is no hint.
So we're talking about hints about where is this going?
The guy is moving, not moving, whatever.
But currently, there's no clue that he's clickable, except for the instructions
Yeah
So you want him to react in some way.
The easy, easy way is to make them bigger, and just scale them.
So have them scale up, or something, when your mouse is over them, or something.
Just give us some really simple clue.
It doesn't have to be complicated code-wise, but you definitely need that clue.
Also, the part where this guy, if he's-- this castle's perfectly safe.
It's never in danger.
I saw in your backlog we've got to figure out how to solve that problem.
I would make that pretty high priority-- actually, higher priority than any polishing you're doing.
Because your game, currently, has a pretty easy strategy of leave a guy there and click for 100 turns, right?
Well, the only thing is you have to be at [INAUDIBLE]
Sure.
You may never win, but it's easy to never lose, certainly.
That's not interesting.
Actually, no-- this guy, by himself, doing nothing but building, can win the game in 100 turns.
Right?
So--
Our proposed solution to that is after you lose two guys or one guy, you lose the game
OK.
Cool
That's not [INAUDIBLE]
Sure.
All right.
Cool.
But I would try to make that sooner, because your game is not-- make sure anything that your game has not done-- like, I feel like that is not done with that flaw.
And, as you actually mentioned in your backlog, this may add new tasks and bugs, because it will.
So make sure that's relatively high priority

I would say--
Because then you have to test all of that on people
Yeah
Because you are going to have a different kind of bar graph.
That's what I'm getting from what you're describing.
You're going to maybe rearrange the way how your fonts and your progress bars and stuff, and all of that is going to be understood differently by people who've never seen your game.
And you need to get that information pretty soon
Oddly, in some sense, the poor bar graph tells me is that, right now, I know this is a bar graph.
It's probably giving me information.
So I should figure out what that information is.
As soon as you turn this into something that looks like a cloud or something, I might stop knowing that it's information
Oh.
That's interesting
Right.
So right now, I see it and I think to myself, what does it do?
That's something you can really quickly paper prototype and get in front of people who have never seen this before
Yeah
Yeah.
In fact, you could take a screenshot of this.
And just stick it into Photoshop.
Put your thing on there
Mock it up
What does this mean, random person on the street I've never met before?
[LAUGH]
Family member that you might see
Over Thanksgiving, for example.
But, yeah.
You don't need to put it in the game to test it, so

Yeah.
There's a lot of different ways you can get that bar graph to look like it's part of the scene while making it look like a bar graph.
It doesn't necessarily need to be made out of clouds.
It could be in a cloud
Yeah
It could be projected on a cloud.
It could be--
Yeah
Yeah
All right
Moving on.
First things first.
On Monday, I mentioned try to do an updated vision statement
Yes
It's due December 10.
If you have one now and it's in Stellar now, awesome
Yeah.
We did not submit it

We haven't submitted it, but we did just rewrite it
OK.
Cool.
Then submit that to Stellar under the December 10 final one
Final--
And then resubmit it again with changes
Resubmit it again for changes?
Sure
OK?
The big thing we see--
What do you want risks to have?
Because it seems like by the end, their risks are kind of meaningless
By December 10, risks are too late
Yeah.
So just keep the risks you originally had

If you want to have
So we updated the risks right now, because pretty much ever risk we had before didn't make sense
That's perfect.
The thing I'm really looking at is 30 seconds of game play
That completely changed
That's cool.
Because that's the thing we're judging your game one.
If your game does not look like the 30-second game play, it's a huge red flag for us
Yes
Actually, there probably are still risks on December 10.
There won't be implementation risks.
It won't be like, can we implement something?
It will be more of the what happens when real people try to use this thing?
Yeah
Right?
So you can write those up
Well, I think that risk is December 8
Possibly
So--
Well, I mean--
Because we're testing with 100 players
Yeah.
But what happens when the next 100 players come along?
Sure, sure
What happens long-term?
Yeah.
There's real-- there's real deployment
You can change it then
Charlie
For your product backlog and task list, are these prioritized when you look at them?
Like, you're using Asana, right?
Yeah
Do you use priority?
We did not do a good job of--
We sort of do soft priority-- small, medium, large-- on some tasks
Yeah.
We have different--
No.
That size
No--
That size.
But you don't necessarily have--
So I think theoretically, they should be ordered by priority

Yeah.
That would be--
OK. Because it doesn't look like they're ordered by name and something else.
I don't know.
We couldn't tell how you were authorizing it, so hopefully you are thinking about that as you are moving towards the deadline.
You're going to have to choose-- I'm going to work on this right now.
Make sure it's organized
Yeah.
I think now that will be more important
How are you-- are you doing work over the weekend, over the break?
I will
I probably will do a little bit, but mostly other work
Oh.
No.
We tell you don't.
Like, don't plan on having, like-- that's bonus work, right?
But if you are working individually elsewhere and you're not able to talk to each other, how do you know what you're working on?
That's the big thing for that
One of the big things that we're noticing on your list of things-- a lot of them are both the server and game-system based.
Probably also because you've been getting a lot of feedback from Pablo and people how have been running this game in front of the audience, we're not getting a whole lot of feedback from the people who are actually playing it on the playing site
Yes.
That's true
So the thing that I'm really, really worried about is that the client-side user interface doesn't look like it's actually changed all that much.
I'm sure the technology has, but the actual user experience hasn't
Yeah
So I am worried that you're not actually getting in useful usability information and that iterative playing experience
Yeah
And so you mentioned in the presentation you were dropping Phaser.
are you in the process of dropping Phaser, or started?
We are done with with that
You're done with that.
So you're no longer in Phaser.
You've got something else?
Yeah
It's buggy, but you've got something else
It's a little bit buggy, but it's almost as good as we had before
OK.
Cool.
Because we were going to ask do you really need to drop Phaser?
And so you're already done, so why bother asking?
Yeah
What else?
Oh.
And give some time looking at your scoring mechanism.
While you kind of hand-waved through it during the presentation-- as we've got, here's a couple of different ways we can do scoring.
Are you thinking about scoring as a team?
Or are you relying on the client to come up with that?
So I'm actually not sure why our scoring works the way it does, because it's harder to implement--
Yeah.
It's also not very intuitive
It doesn't make sense
Yeah
Like, there's a specific function that makes sure that you only get one point when you might as well get more than one point.
I think that's just something that has been low-priority, and hasn't been fixed-- especially because nobody actually understands how their getting points.
And they only ask at the end of the game, wait a minute.
How did I get points?
Yeah
Two ways that you can resolve that-- one is you play test it, and then you ask people how they think that they should be getting points
Yeah
And the other way is you just ask Paolo
Yeah
And let him make a decision for you
The third way-- if you're with family members, you can play the regular, the analog game.
Come up with a scoring mechanism-- like, just do it.
You don't need to be programming to do it.
You can probably do it really quickly
My guess is that a accumulated scoring system will give you more differentiation between players, whereas the one point for everybody kind of situation means a lot of people get the same score
Yeah
Yeah
Are these-- it looks like you have, actually, three backlogs.
Is that true?
This is backlog, and these are tasks lists
Oh, I see.
I see.
I see
That's on the--
I see
Same system, yeah
Oh, OK.
So does that mean it's repeated?
This is the backlog and these are task lists, are the same thing?
I think they're repeated in both places.
Do you have one list--
No
Yeah
So you have somewhere one list that lists everything you need to do, prioritized.
So that way, one of you--
OK. We don't have something exactly like that
And the reason why you want that is so that if I'm working on the back end and I have gotten all of my priority two stuff done.
I'm working the priority three stuff.
But on the front end, there's a whole bunch of priority one bugs.
I need to switch to the front end
I see
Right?
So far I don't think anybody has-- initial on the switch.
It was permanently switched
Yeah
Now, part of that, there's a learning curve problem for doing that.
But if you get that far out of whack, you really need to be ready, or to be aware of the fact that that might happen
I see.
Yeah
So it also does you no good job to implement a feature on the back end that's not implemented on the front end, and vice versa, really-- for the features that pair, at least, right?
So there's got to be really close communication.
And one way to handle that is to have a unified list-- bugs and features intermingled sometimes
Right
What's the most important thing to do next?
But I would be a little bit worried if this is your entire organization system, about stuff falling through the cracks pretty easily, especially since you're dividing your team up into subgroups to sort of handle front end, back end problems
Yeah
That means that there's a-- it's not a communication barrier, but there's no highway of communication.
Since you are separated out, you will tend to forget to inform the other side of what's going on
Yeah
So, and the one list also helps with that.
And also, let's see.
Did I see here that you're already adding word lists or did I make that up?
No, I just saw cloud, words, and export
They have a word cloud on the back end now
Yeah.
Not for players--
I see
--for the back end.
That's been around
Got it.
So the back end-- I see.
I see.
I see.
OK.
So are you, for example, not bothering to have a feature for collusion or cheating?
No

Yeah.
Cool
I think we have no anti-cheating tasks
All right.
Cool.
You probably should mention that explicitly somewhere.
I'm not sure where that goes, though
Maybe interpretation document
Yeah.
It's an anti-task, certainly
That's actually--
And to the client, for example, you'd want to make sure the client knows this is the thing we're not doing.
And you probably don't care, because of your use case.
But we should mention in case you go, no.
Wait a minute
Your final presentation's a great place for that, as a here's something we thought about and didn't do because of that
Yeah
Yeah
Although if you do have any conversation with the client, mention that.
Because if they're like, that's crazy, you want to know that before the final presentation-- you might not be able to do anything about it with the time you've got, but it'd be good to be aware if maybe there is an easy solution the client come up with or whatever that you haven't-- because maybe full cheating stopping is not necessary, but some minor things are
I mean, I think the way they run it-- if you cheated it would be obvious, first of all.
Like, they're going to show you did it
Oh, because the cloud
So, like--
OK, yeah
What you did--
Who put cat and dog in here?
That's ridiculous
Yeah
OK. Sure.
All right.
Cool.
That may well be enough right there
And also, they wants us to focus on pretty serious players, right?
So who aren't going to be cheating?
They're not going to cheat
All right.
Cool
[INAUDIBLE]
That's a nice advantage.
And that's a thing to include in your-- if you mention your target audience is, in fact, people who've already drunk the Kool-Aid.
They're already on your team.
They're not trying to-- it's not whack people on the internet who are going to do whatever they want.
That's an important--
Yeah.
It's almost like not players
Yeah
It's not a typical player
That's an important thing to mention in your vision statement, actually, if it's not in there already-- I'm behind on that, but
Yeah
One other thing is given that your game is designed to work both on phone and on laptop, on PC, you do want to set up multiple test sessions so that you can test your user interface on both clients.
Maybe you're testing them simultaneously, but then you have to make sure that when you're getting your usability feedback, you are getting some usability feedback on the phone users, and some-- because it's very different on the different device
So I think that's sort of a weird feature where, as far as the class is concerned, I don't think we care about the mobile interface
OK. All right
Like, that's not something that we're promising in terms of the course
OK. Oh, good.
Good
That wasn't clear on the priority list, too
On the other hand, it seems easy enough, and the client really wants it

You might want to do we'll guarantee that the PC site will work, and if you wanted us to do the mobile site, maybe pay us
Here's 0.9B.
You get 1.0 with the dot
I mean, seriously, this is real work that you're doing that real people get paid to do.
And for the sake of this class, we want you to turn in a polished product, and we never ask you to target two platforms.
The client does, obviously.
But then it's also reasonable to tell the client, first priority is get one--
Beyond the scope of the class
Well, I think he's-- he's been very good about mentioning that every time, that he wants us to do well in the class, not make him happy
Sure.
Sure.
Well, and that's another place where your prioritization can help a lot, right?
You want to make sure that the web interface on a thing with a keyboard is the one that's the highest priority.
And if you have time, then you make the mobile one work.
And you're already going to have problems for you poor mobile people on their phones, who are typing really slowly compared to the people on their computers
Yeah.
This is something we talked about, in fairness, that sort of got dropped because it's very hard to do
Oh, yeah.
You can't compensate for the fairness, but it's a thing to be aware of, obviously.
All right
Yeah.
So I have one question about deployment.
How long after the course will the game have to be up?
At least through January
Or through the whole of January
Yeah.
Through the whole of January, if possible.
And if you can't do that, let us know.
And we'll come up with a different solution.
Grading is really soon.
We're going to do grading.
Yeah
Grading will be before then-- way before then, obviously
If you have a way to deploy it to a different platform, a different server, if that is what is necessary, then we'll think about that.
But that's the other thing.
We're going to have a conversation with Pablo at the end of the class to figure out how it's going to be used
And OCW would like to figure out, also
Yeah.
OCW wants to actually-- that's right.
Forever
[LAUGH]
So ours is a little bit more tricky
Yours is more tricky
I'll find a solution for that
Something we could talk about after class, because I don't think we can talk about it during class.
That after January 1, it's not going to be part of the grade.
Because the grade's already done by December 15
Yeah
But for OCW capture, we'd like to have something there
What does it run on now?
It's a node server
It's a node server
[INAUDIBLE]
Actually, it's two node servers, unfortunately.
One for the back end and then one that runs the front
The front end is just static files
Actually, the front end is HTML now
Yeah
So maybe we can talk to [INAUDIBLE] about finding money to pay for the servers, basically
Yeah
And maybe even money for one of your [INAUDIBLE]
Exactly
So that's something to put in your vision statement too-- or as a release note or something, somewhere
Is there any place the caveats go for the risks--
The risks section might be
Yeah, actually risks is a good place.
That's as close as we've got for that.
Risks, cheating, risks, server goes down.
[LAUGH]
All right.
We've taken too much of your time.
Thank you.
Have a good holiday
Thank you very much
So looking at your project, the biggest thing we found was can you cut one of your main characters?
Is that even possible?
So, yeah.
We already cut one of our games.
And we're at three
So how-- you're at three?
So you cut one down to three.
Do you need three?
Is three the magic number that just says, everything's wrapped up together?
Or if one of those fell away and we polished two, is that good enough?
I think two would significantly change how the game fundamentals.
It's kind of a puzzle.
So it's like if you only had two clues, which is three clues, so more of a puzzle element
Also the fact that each game covers-- we cut down one game and combined it to minimize-- to still try to cover everything we want to teach
Yeah.
We'd have to find to teach the-- what was in the game we cut in other game dialog or something
Absolutely.
Cool
So cutting them in the game would, in fact, mean a lot of work
Yeah
Great.
That's the perfect, excellent answer
Otherwise, we're looking through your product backlog.
Your prioritization looked really good to me and Phillip
Great
So I think you're-- do you feel like you're in control of the information that you have right now, of where you are on your team, where you are on your project?
Yeah.
I think our team, since the way we structured it, it's like everything has to come together.
But putting it together will not take as long as it would originally.
Because we kind of all divided up tasks.
Everyone's working on the same kind of tasks, like for the same thing, but we just have to put it all together, if that makes sense
Yeah.
So it's very modular, since we have three mini games.
Then we have somebody to work on the main scene.
And then we have end scenes.
So you just have to put some pieces together
OK.
Cool.
Can you pull up the vision statement really quick?
Yeah
So on Monday I mentioned talk about vision statement.
Change it if you need to.
On yours, it looked pretty good.
I really like the narrative aspect of the 30 seconds of game play.
Does your game still feel like there's a narrative to it, though?
Like, is it as strong in that game?
So I think it's probably not as strong as we want it to be now but, that's only because the dialog hasn't been--
--really written.
I mean, been drafted
Yeah.
Exactly
Yeah.
We're going to that [INAUDIBLE]
OK.
Cool.
It's not due until December 10, for reals.
So that was an exercise just to make sure that you do it now rather than waiting until the last minute
Yeah
So once you have a finished version in, turn it in on Stellar under the December 10 deadline.
And, yeah.
Really make sure that the 30 seconds of game play are more representative of what the game really is

Are you having problems with the title?
Hmm?
Are you having problems with the title?
I would say so
Yeah
Yeah
Is the problem too many ideas, or too few?
Too few
Too few
OK. That's actually the easier one to deal with, I think.
Because when you resolve it, no one's mad
Yeah.
[LAUGHTER]
What are your strategies for that?
We haven't-- we've been ignoring it
You've been ignoring it
OK. That's-- that's actually a fine place to be.
Because it's relatively to brainstorm it.
But if you tried that and it didn't work, eh
Because, yeah.
A title does-- even if it happens last minute-- it's another art asset to create.
Another thing to integrate
Yeah
Yeah.
We've been thinking like, as we do it, we'll just get a name for the village, and we'll be like, Saving Village Name
Village name?
Beautiful
Sure.
That sounds easy.
That's fine.
Have you been brainstorming water themes, like water names?
Water names?
Maybe like the Ghanaian word for water?
We looked through like Ghana stuff, like Ghana city names.
And then we haven't actually, like, looked through specifically-- we have to work on that

All right.
Is that on your backlog?
It is
All right.
Score
It's at the--
The very, very bottom.
Yeah, yeah
All right.
Cool
Find a name that isn't offensive
[LAUGH] Yeah.
That's important
How has that been going along since we last talked to you, about the--
Research?
Research and stuff?
Are you done with that, or are you still on-progress with that?
Well, we got together and did a mega research, like-- find all of the things and get inspiration from it, mainly for the artists, so that we can get a cohesive theme together.
And I think that also helps inform the decision, the design behind water filtration, for example.
So, yeah.
I mean, I think still if the artists ever need to make something new, we still look up and see if we can find references
Yeah.
OK. That's good
But--
All right
So I don't think that I have played this game, because mini game sounds like a new thing to me.
Saving All the Village.
Give me a quick five-second play-through.
I hit that, I'm sure.
Mini Game 1.
Yes.
Purified water conversation goes here.
I will purify.
Yay!
This is the mini game?
Hmm.
Hmm
The down button should be a Next button
Oh, excellent.
Oh, excellent.
And now I play the game where I'm doing essential sorting, and this is an actual game.
All right, cool.
I am going to assume that I do something like this.
All right, cool.
The main thing I-- so when I am putting on my feasibility hat and I hear the word mini game, I always think [GASP].
But it seems like you actually have a mini game.
Good job-- a mini game
[LAUGH]
Mini games tend to just be--
Game games?
--many games
Many games.
[LAUGH]
M-A-N-
Yes.
Many games is not what you want.
What you actually want is mini game
[LAUGH]
Oh my gosh
This is [INAUDIBLE].
[LAUGH]
All right.
You've got a lot of-- you've got a lot of features, here
Yeah
And that is cool
Is this a mini game, or is this--
This is the mini game
All right
All right.
All right.
I see here.
OK.
Cool.
And so how much more do you think is going into these mini games?
So we just-- for two of them, I haven't actually had an update on the symptoms one
On the second one
But the symptoms one should probably just be dialog and the pictures.
That's it
Got it
The water filtration should be instruction page, or interactive tutorial beginning, so you just don't like dive into it and it's like, what's happening?
And I think just clearer instructions with the water placement
I see
The main thing that we have to do right now is just the end game-- making sure that once you finish all the mini games, you have the clues you need to take it from there.
Then you can have that conversation


And also art assets
Cool
Cool.
I win.
Yay.
All right.
Cool.
All right.
So you have three mini games, and they're, like, doing stuff.
That's good.
All right
Those are the symptoms
Do you have any-- because it seems like, for the most part, these mini games are relatively simple.
The code is actually not--
You're raising--
Click the right thing.
You're using assets because of that
Yeah
Do you have any unknown sort of tech-wise at this point?
You sort of know everything, or is there-- we're not sure how that works.
that left?
So I don't know how-- I think something that has been buggy consistently-- maybe it's changed now.
I'm not updated on it.
Is the conversation UI?
Yeah
Yeah.
We need to iron out a few bugs with the conversation UI and make the visual aspect of it more appropriate with the theme that we're going with the assets.
But other than that, code-wise, there's just a couple bugs that won't be hard to fix
OK. All right.
Cool
Yep
So do a glance-- I'm sure you already have.
But make sure that anything that has any question marks at all with it gets higher priority
Yep
Many not even finishing it-- right?
Like if it's the case that you go over the tasks and you're like, OK. Now the question marks are gone, and I'm comfortable moving it down in priority, feel free to do that.
But make sure that those question marks are done sooner than later.
Because anything that has a question mark, it might become five question marks.
And then you're in trouble.
So just make sure you look at it seriously as a high-priority item
When is sound going in?
So we have sound files in a folder, but, yeah.
They should probably go in

We ended up-- they were taking a long time to push.
According to Rachel, they were taking a long time to push to our test server

So we just removed the assets and just pushed it without the sound

We do have some sound
All right.
Cool
And generally speaking, sound, much like art, if the file is the problem, use a stupid file.
At this stage of development, it's more important that it makes sounds than it make the right sound.
And so-- again, you might discover that, oh.
It turns out playing that sound is harder than we thought, because we can't use an Ogg file.
That's the kind of thing you want to know
Yeah
But, all right.
Cool
Great
Thanks
Thank you.
Have a good holiday
Thank you guys.
You too
I think I've only played the paper version
No, you played the computer version
No, I did-- way back when.
Way back when, yes I did
You were our first tester
You are so right
Because we have a new version that we're building up, and it's still very broken-- the new one

So nothing more playable than that
Correct

Unfortunately
And the minute we kind of rehauled the system, the game play's focus is a lot different.
We cut out a lot of stuff, and we're focusing more on, washing your hands is great.
And cholera travels through water
Yeah
Good
Is that based on feedback from Monday, or this previous lecture?
Yeah.
That's correct.
We got some very good feedback
Good.
Good, good
It was interesting.
So one of the crucial things we were wondering about is we were sort of designing our game to have a huge variety of options.
But we recently switched to this new thing that we really like a lot, which is that basically we roll out the mechanics one by one, so we start with one build, using like two or three [INAUDIBLE]-- maybe only two.
And it actually just is the case that one of them is better than the other, right?
Because if the point of the game is to teach you something, it's OK if just washing your hands is [INAUDIBLE] and something else is not so great.
Right?
Uh huh.
Mhm
And then it actually slowly becomes more and more of a game, where we roll out another village.
And then you have to trade off balancing your resources.
But then another village appears upstream of both of them.
And suddenly, the trade-offs become not so clear.
Where before it was just one thing was more price-effective than another.
Suddenly you actually have to care about how infected they are.
Because boiling your water will reduce the extent to which you're infected by upstream water.
And he really liked that as a--
So that's sounds awesome.
How does that get affected by that change based on the UI that we saw?
Is it using the same--
It's going to end up up being the same sort of system.
It's only that the pop-up screen now eliminates a bunch of options, because they aren't necessary anymore
OK.
So these are you removed a ton of options, you weren't creating new options
Correct
That's what I'm hearing, right?
We're basically--
We basically gutted a whole bunch of the options that seemed redundant and apparently weren't helpful to people on a local scale.
And then we changed those options so that they're more distinct.
So while before it was just, like, the higher magnitude to price ratio of affect, now it's like some things only really touch infection rate.
Some things just cure people [INAUDIBLE] infection rate
Cool
And now you have things that have that upstream, downstream play affecting it
Exactly

Yeah
So the other crucial thing we were struggling with before is how to make legitimate, actually meaningful gameplay choices.
When, if you just have a bunch of options, there's going to be some static build-- just the correct one, right?
So we need to make it so that it was more situation-dependent.
And [INAUDIBLE] had a great idea, which was to be highly effective.
Basically we have a thing that's the best for reducing the infection rate both ways, and then a thing that's the best for reducing the symptom which could spread infection downstream.
And I think that's the best at protecting you from infection
Oh, that's great
And then suddenly-- that can be made incredibly clear to the player, right?
We're not hiding what the best build is
No.
You're playing Rock, Paper, Scissors, is what it sounds like
Yeah.
Exactly
OK.
Awesome
And--
But-- so it sounds like UI-wise, obviously this is a huge, huge improvement, just removing the options, et cetera, et cetera.
But when you start having a little bit of-- do you already have cross-talk in your game code between villages going on?
Yes.
We had that previously
All right.
So again, it really is you're actually mostly cutting rather than adding, it sounds like
Pretty much
All right.
Cool
But you are still adding the new UI--
Correct.
Which
--which I imagine we would see.
If we're looking at your backlog now or soon, a lot of that would be in the backlog
Yeah
Because I think right now you've got some very-- actually, go to the task list.
I think this might have been the vague one
OH, yeah.
The sprint tasklist?
Yeah.
Yeah.
That's, like, wicked vague
[INAUDIBLE] tomorrow, but this needs to be submitted today.
So you get this really vague description
Yeah.
We figured that
Sorry
No, that's--
It'll probably be in sprint tasklist five.
I'll outline the stuff that we're assigned
I don't care what's in this.
I use what's in this to know what you're doing.
So I knew exactly what was going on
So that's fine
That's
All right, good
And if you didn't me in anything, that's like, OK. You're really on fire
[LAUGH]
All right.
Cool.
So Phillip and I were talking about the UI that you were doing.
You had mentioned, in the presentation, civilization.
But Phillip and I-- Phillip actually remembered this.
If you go to Molleindustria-- Paolo Pedercini is a serious game designer.
He might hate the term "serious."
But he made a game called-- I think it was called the McDonald's Game?
I don't know
I think it's called the McDonald's Game-- again, this is Phillip's idea.
Basically, there's a lot of different screens going on in that game.
But if you were to make your game--
Oh, is it the cutesy one?
It might be, yeah
OK.
I know exactly which one that is
If your game had as much UI as one of those working screens, then you'd be golden.
So that's kind of like a bar to set yourself towards, that I think you could probably match, especially now that you've cut a bunch of this stuff
Yeah
Yeah
But you can look at that as yet another way of seeing how he's putting together all of these UI elements, or doing all of these systemic changes in this very simple interface.
So take a look at that.
Otherwise, can you go to the vision statement?
Yep
So vision statement-- on Monday, I asked for you to revise it
Yep.
I have it revised.
I haven't submitted it yet
OK.
Cool
It's right here on my computer
Great.
Submit it under the December 10 deadline

And then if you have a new version afterwards, submit it there.
And again, the big thing that I was really looking for is that 30 seconds of game play actually reflecting what the game really is
Yeah

Did you do the--
If you want to read it real quick, this is what I have now for the 30 seconds of game play, if that's better
Yeah
Yeah
That sounds like-- well, kind of.
That looks to me like an overview of a lot of game play
More than 30 seconds
The thing is it was hard to cut it into 30 seconds, because if you don't know the beginning of the game, then doing 30 seconds in the middle doesn't make sense, and doing 30 seconds in the beginning doesn't make sense either.
So that's why I kind of like--
Well, I might question that.
What is it that you are doing in 30 seconds in the game-- is you're probably looking at one village and saying, what's the right intervention for me to do right now?
Can you describe that experience in a way at all?
Like I decide that because there's nobody downstream, I don't need to worry about this thing.
So I'm going to just direct everyone to boil their water.
Like--
OK.
I--
Describe the thought process
Yeah.
So that might be more like the major game play concept, or the vision itself
Yeah
[INAUDIBLE]
That's what you're doing for the whole game, absolutely.
You're doing all of that stuff
Yeah
But the 30 seconds of game play is really about what does the game feel like in the moment?
And it's a planning game, so it's hard to do in 30 seconds of game play that's meaningful.
But--
So if Rock, Paper, Scissors is a way to synthesize that, or I'm making choice A versus Choice B now
Right
Because your game is about weighing choices and making a good one
Yeah
So 30 seconds of game play should be about making a choice

So if what you have that runs now is very similar to what I saw a while ago at this point, is this backlog up to date, out of curiosity?
So not really.
It still needs to be fixed
All right
A lot of this stuff is like-- [INAUDIBLE].
It's difficult
OK.
So--
Tasks have been split up, and, like, I'm somewhat removed from the coding.
So I don't actually know what's being done.
So, for me, it's hard to edit it.
And so I exactly sure what--
Do you have a--
So I would claim that not over the holiday.
But, you know, the priority on Monday would be to make sure it is up to date so you guys know what's going on.
Because for me looking at it, hearing that I can't play a newer version of the one I played a while back and seeing this backlog, I think, uh oh.
So it may be that you're much better off than that.
But I think your highest priority task on, say, Monday is find out if you're in trouble or not
Yeah
And if the answer's we know we're in trouble, then, OK. You still need to do it so you know what trouble you're in and how to adapt.
Because I don't have much information, but I'm a little bit nervous right now.
But I suspect you're better off than it looks on that page
Yeah
So you have a one, two, three, four, five-person coding team?
Mhm
Would you say one person on that team fits lead?
Like, do you have a lead coder?
Uh, Harry, probably
Yeah
Does he know he is lead--
Harry is mostly the back end.
I think he does.
Because he mostly doles-- him and Derek dole out the tasks, so I assume they know that they're the leads.
But--

Yeah.
Because that could just be you meeting with them to make it easier rather than a full eight-person--
You might not need the entire eight-person team for that.
But again, also there's a difference been the person who knows what the coders are doing and the person who's doing the most code.
Those are two different things.
And so you want to figure out who's really wearing what hat there.
Yeah.
Cool
Have we kept you longer?
Thank you
Yes.
Thank you for your patience
And, yeah
Do you have any specific sort of questions, worries that we can try to talk about, or just get me out of here.
It's Thanksgiving.
[LAUGH]
I don't know.
I think the big unknown for us right now is the new UI.
We have the mechanics that we want.
We've really been playing with the numbers for a while for what infection rates and what efficacies and treatments and what prices into a--
Got it
--beatable game that's also not trivial
All right.
Cool
So we have that.
The big question is the UI.
The old version of the UI has all sorts of problems, where it'll constantly orphan UI elements, and they'll sort of accumulate on the screen
Got it Got it.
So it's buggy, among--
It's buggy.
So I'm much more worried about having a functioning UI.
So one thing I'm thinking about is if we suddenly find we don't have anything working at some point, what sort of bare-bones UI would work?
Like, for example, you could concieve-- and this is sort of unclear.
You can see that the UI can work entirely-- it would be very, very simple.
It would take half an hour to write.
For absolutely every single action in the game, it's like a keyboard thing.
The four villages are mapped to the columns Q through Z, W through
Right
And that grid of 12 buttons does every single action
Right
So it could be reduced to that.
It would take a half hour to write
Sure
The question is--
That would mean having to explain that, though
Oh, yeah
Yeah
So you can't ship with that.
But it might be worth doing it, [INAUDIBLE] half an hour, so you can test your game as it exists, internally
There's also concern whether we-- we're still clear if we have to-- maybe other people aren't.
But I remember at one point hearing from you tablets were what we--
We don't need to worry about that
No, no.
No.
You've got a single platform.
It's what you've been developing on
Yeah
OK.
It's this
It's that.
If that's what you've been developing on
At this point I would say, yeah.
Whatever's-- we actually want to grade it on a computer rather than a tablet
Yeah
OK, good
That's exactly right.
Yeah
[INAUDIBLE]
The tablet was if it was a challenge that you wanted to take on
OK, good
The client was like so, what's the--
I totally misunderstood that
So the after class life of this game and the other Cholera game is they are being presented to UNICEF as a possible funder for games of that nature

So it's like, you want to make-- like UNICEF wants to make games about Cholera for Ghana, and you're saying, here are what those games might look like.
Here are prototypes

These are not the games we're going to ship to you.
And if they were going to be the games-- so when we're setting one of these games, we're going to be saying, imagine this is a tablet-style game

And the other thing is if you are graduating and wanted to work on this project, and [INAUDIBLE] had funding for that project, they might hire the students who worked on these games to make that project
Sure
That's very much a talk-in-January kind of thing
Right
Yeah
And so generally speaking, if that takes a big worry of the UI off your shoulders, awesome.
But regardless, if you have an unknown like that-- like, as the new UI.
Is that a big question mark?
I'd make sure you schedule it, prioritize it higher.
It sounds like you have your game play under control, which is awesome.
But you want to make sure that whatever unknowns you have are scheduled sooner so that if they go horribly awry, you have time to adapt to that problem
And my last one is going to be, you've already cut a lot.
Can you cut more?
When you meet on Monday and you say, are you in trouble?
Like, all right.
You've proved you can cut a bunch of stuff.
You could possibly cut a little bit more out of it too-- assuming it's the kind of cutting that doesn't create new work
Right.
But I would start with that-- where the heck are we, if you don't know
Yeah

And probably all eight of you don't know where you are, so it's worth getting all eight of you to know where you are
I can't even-- I see such-- I see resulted-- I didn't even realize [INAUDIBLE] until I forgot it
[LAUGH] And that's why the last project is so tough, actually
It doesn't fit the school year very well
Yeah.
Unfortunately it's a shorter term than we'd like to have, but, yeah.
Eight people's a lot to keep organized, even with full time
We only have seven.
Don't worry.
There's no eighth person
[LAUGH]
OK, cool.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
All right, so today is the last of our guest lectures before we go into, what else?
Sorry.
Not the last day.
This is Monday, right?
It's Monday
I'm already feeling the crunch.
So we've got-- this is the last week of guest lectures.
Today, we've got Michael Carrier and Jenna Hostein talking to us about the business of Indie development.
They are representatives from PD Game Collective.
That happens-- they are a group of people who make games together over in Intrepid Labs?
And that's over which [INAUDIBLE] building?
The American Fine Building
The American Fine Building.
They'll talk more about that.
They often have open lunch days and things like that for students to come.
So I'm sure they'll talk a little bit about that as well.
They will also be available after the guest lecture.
If you want them to play your game and get some feedback on your game if your game is ready to get more feedback, they can do that.
It's again entirely optional.
If you have a build-- hopefully you do-- I'll allow you to play briefly and get some feedback from them.
That's it.
So I'm going to hand the stage over to them
Is my audio working?
We're good?
All right.
Hello.
My name is Jenna Hostein.
I'm the founder of Little World Interactive, and I'm a part of the Indie Game Collective, as Rick said.
We're about six or seven blocks north, so we'd love to see you guys either in the evenings, or we do Friday lunch get-togethers.
Michael can fill you in on that.
He is more familiar with the details than I am.
But I am here to talk to you today about the business of Indie development.
If you're an Indie developer, you can't get away from business.
When I first started, I-- I am a designer by trade and a developer by necessity, and a business person dragged kicking and screaming into it.
It was really something I was interested in.
It remains something that I'm not super passionate about, but it's absolutely crucial to have any level of success.
So I'm going to talk to you a little bit about what the past couple years have been like.
So first, let's talk about the realities of time as an Indie developer.
When I worked at a larger studio, I used to daydream about what it would be like to work as an Indie developer.
In those dreams, it was me by myself sitting at my desk just working, and that, to me, was like this beautiful, glorious manna from heaven.
I just wanted to sit and work on my game.
So this was, naively, my expectation going into Indie game development-- that this was going to my life.
I actually found out, that was not the truth.
So there are events all of the time in the game world.
There's PAX east, PAX south, PAX prime, PAX Australia.
GEC, Indie K, Games for Change.
I could probably list five or ten more events off the top of my head.
It's absolutely crucial that you attend these both as an attendee just to network and to exhibit your game.
For example, PAX prime this year, it took probably two, three, maybe even four weeks full time prepping for that, attending that, and then coming down from that and tying up all the loose ends.
So do not underestimate the amount of time that it takes to actually attend a lot of these events.
Marketing.
Another thing that is really not my favorite, but I think a lot of Indie developers have a tendency to make their game their beautiful precious baby, and then they throw it over the fence and they say, am I a millionaire yet?
And that's just not how it works.
It's a constant ongoing effort to make people aware of your game, to make people care at all about your game.
Because guess what?
You're one of thousands and thousands of developers who are doing exactly the same thing.
So this has to be a constant ongoing effort to rise above the noise.
Communication internal.
So I discovered very quickly that I am terrible at art, I am terrible at sound, I am terrible at marketing.
So I brought in other people to do these things.
And when you work with other people, you have to talk to those other people, so that takes time.
Communication external.
There is a huge Indie game dev community, and if you're not taking advantage of those folks and really learning as much as you can from them, then you are doomed to fail.
I saw somebody recently who has been at Indie for about a year and a half, and he was doing this-- where he was sitting in his house very, very insulated and he had created a game that didn't look wildly interesting and clearly had not gotten input from almost anybody.
He was working on his own engine and it looked like something we could have built in Unity in probably a couple of weeks.
So that, to me, was clearly a failure of just talking to other Indies and learning from them what they're doing and their successes and their failures.
QA and deployment.
Learning how to do QA and learning how to deploy was another huge hurdle for me, and this just takes time.
I have released very, very broken builds before, and you don't want to do that.
So QA is a necessity.
So this chart, to me, is really interesting, because if you look at how squashed that bottom bar has gotten.
I spend less than half of my time actually making the game.
And that's why I got into this in the first place-- I just wanted to make my own games.
So this is the reality of the situation.
This is not a bad thing.
This is just how it has to be to have really any level of success.
So when I first-- I developed The Counting Kingdom-- sorry, I totally forgot to introduce myself.
I made an educational game for kids called the Counting Kingdom.
Just released for iOS recently, also on Steam.
When I first budgeted out how long I thought that would take, it was about six months.
It ended up taking about a year.
And if you take into consideration this chart, I was right on, because given I was only spending about half my time working on the game, it took twice as long as I thought it was going to make.
So now, whenever I try to estimate how long something will take, I take my gut check and then I double it, and that's usually pretty accurate.
OK.
So let's talk about working with other folks.
So I discovered very, very quickly, I'm not particularly good at art.
I'm not particularly good at sound.
So I needed to bring in other people to compensate in these areas that I was failing.
So I work with six people very, very part time.
Luigi does art, Will does PR, Sam does animation, Jonas does sound effects.
Brendan does music and Emma does community management.
And all of these people were not just random people.
At no point did I put out something into the world saying, I'm looking for a community manager-- get in touch.
I would never do that.
That sounds like a lot of work and just a terrible way to actually find somebody.
So both Luigi and Will are folks that I met through the local games community.
Luigi did the art for Girls Like Robots, and I basically stalked him and tracked him down.
I was like, whoever did this, I need them to make my game.
Fortunately, he was a local guys, and so I got to meet him, talk to him and discover that he was a great for the team.
Both Sam and Jonas were folks that were recommended by people I trust.
Sam was an intern that I brought into a local university and Jonas was somebody that had sent in a great portfolio piece to another company.
They were not able to hire him, but they basically put the word out saying, this audio guy looks amazing.
If anybody is looking for an audio person, you should snap him up.
So that's exactly what I did.
Then Brendan and Emma are people that I previously worked with.
So the lesson to take away here is, you have to be networking.
All of these folks were either people that I knew directly or people that knew somebody that I knew.
As an Indie developer, I have no time.
We've discussed this at length.
I literally have no time.
I am working more than full time, and so I cannot micromanage somebody, I cannot be on somebody's ass to make sure they're getting their work done.
And so, when I bring somebody into the team, it has to be somebody that I trust to not only do work, but to do amazing work and to be able to do that independently.
So, for me, that means working with folks that I already know, that I already trust, and with folks that are highly recommended to me by people that I trust.
And nobody else.
So if you're interested in working with an Indie company, don't sit around waiting for them to put out job requests.
You need to be actively trying to meet these folks.
Network.
So I want to talk a little bit in detail about what it means for me to hire somebody.
This is how I do it-- it's not necessarily how everybody does it.
When I bring somebody in, I have them sign a work for hire contract.
That basically just means that they do work, I pay them, and my company owns the work that they do.
I then create separate schedules, and that's a tiny one page document that says, reference all that other stuff in the contract.
Here is what needs to be done by the contractor.
It needs to be done by this date, and here's what they're getting paid.
So it's a very, very simple document.
You can do the payment in a couple of different ways.
So when I first worked with Luigi, who's our artist, we worked by bundles.
So I would say, I need this list of art assets.
When do you think you can get them to me, and what would be a reasonable price?
And we would negotiate both of those last two things.
It was usually three or four weeks, and we would put that up in the schedule and then he would go off and do that work.
Right now we've switched to hourly, just because it made sense towards the end of the project, because I would be like, oh, I need this one particular tiny little thing that I completely forgot about, and that didn't make sense to do entire bundle for like $20 worth of work.
So we switched to hourly.
It's also common to do revenue share.
That could be the entirety of the payment, or that can be on top of hourly payment.
Really depends on who you're working with and what make sense for you.
So this is just an example-- oh my gosh, sorry.
This set up automatically.
Oops, there we go.
OK.
So this is an example of a bundle that I did with Luigi.
You can see it just lists in detail all of the tiny little things that I needed at that point.
So we ended up doing four or five big bundles and then swapped to hourly.
This was about halfway through the project, so you can see some things are on the first strap, like the world map.
Some things are getting a lot more detailed, like the level backgrounds.
But this is it.
A schedule is not super complicated.
It's just a list of what needs to be done, when it needs to be done by, and how a person's getting paid for it.
The last thing I want to talk about is the long tail.
We have this idea, I think, in Indie development that you're launch day needs to be your big day.
Especially in mobile, you see these graphs where it's like, launch day the rest of the time.
Those graphs are super scary, because it means if you're not making a huge amount of money on your launch day, you have absolutely no income.
I think this is 100% the wrong way to think about it.
When I launched The Counting Kingdom on Steam, it sold 35 copies on launch day.
35 copies.
I didn't think that was possible for a Steam game to sell that few copies on launch day.
So that blew my mind, and I was kind of like, OK, I need to really rethink what I'm doing here.
And since then, it's sold well.
It's sold steadily.
I've sold a lot more than 35 copies since then.
But I've had to really flip my thinking about where the money is being made.
So the point here is, this is not a business plan.
Bullet point two has to be a lot of different things and it has to be over a large span of time.
So for The Counting Kingdom, we started on Steam and the Humble Store.
We got into a Humble Weekly bundle, which was absolutely phenomenal, and so far, has been where the biggest chunk of our money has come from.
Just launched on iOS, and I'm looking towards the future.
There's a lot of different things I could do.
I'm considering porting to Android, I'm considering trying to get into schools.
I'm also talking with other companies about potentially licensing The Counting Kingdom for various opportunities.
This has to be ongoing.
There's the work of making the game, and then there's the work of making money from the game, and those are two separate things.
That kind of sucks, but it takes that much work to actually make money from something that you've already poured your heart and soul into.
But that's just the reality of the business of making games.
So this is frustrating, but that's just something that, again, is a reality of being any developer.
It's something you have to be aware of.
So that is all I have for you.
And Michael, come on down
So my name is Michael Carrier.
I'm the founder of Zap Dot, Inc.
I also am one of the co-leaders of Boston Indies and the founder of the Indie Game Collective.
It's a, as Rick mentioned, a small group of studios in the Cambridge area that work together, and we do a lot of things to try to help ensure that everybody is doing some awesome work and that our products are coming out really pretty.
Some of the projects that I've worked on-- so I started off at Harmonics and I worked on a bunch of titles there.
From there, I did an apprenticeship with an MIT alum, [INAUDIBLE] on the game Fallen Frontier after he left Bungee.
I worked with the BBC and a company out in west coast called Pipeworks for Dancing With The Stars free to play online web game.
I worked with some local Indies, Alchemy Labs on Jack Lumber, and Dejobaan Studios in a game called Drop That Beat Like An Ugly Baby.
Another contract gig for me was, the bottom left, it's called Culture Shock, which was a game that taught people who are being deployed to the Middle East cultural awareness.
This is really a cool game for a group of neuroscientists to try to prove that games can help people recover from concussions effectively and efficiently.
And my two current projects, [INAUDIBLE], which is some contract work and Pleasant Dreams, which is a digital reboot of a physical table top game.
Out of these-- one second-- out of these, the top four were employment, and then Dancing With The Stars, Culture Shock, [INAUDIBLE] and this neurologist game were contract work.
So what I'd like to do is give everybody a little bit of context for the information that we'll be talking about today.
There is a differentiation between AAA and Indie.
I would prefer not to use that terminology.
I want to talk about big company versus small company.
Main reason being, with the tools that are out there today, AAA and Indie used to be a description of quality or subject matter, and I really think those lines are becoming blurred.
So talking about largest to small company really can encompass the type of work that you will be doing at those small companies or big companies.
Either way, the direction that you take is going to be really challenging for one reason or another.
So if you work at a larger company, it's really important for you to have a vast amount of domain knowledge.
It's going to be challenging, because you're not only competing with local talent, but global talent.
There are people from all over the world that want to work at these companies, and these are some of the larger companies that do hire and have hired in the Boston area.
But if you know the story behind some of these companies, there is a lot of overturn as well.
You end up having people that come and go and are laid off from project to project just because of the state of the industry being more of a project-based industry that a long-term employment type of industry.
So not only are you competing with people all over the world, but you're also competing with really bright people who have had years and years of experience at companies like these.
Smaller companies are slightly different.
You have smaller teams, you have smaller budgets, and you have to do more with less.
Typically, what you do is you find people doing multiple jobs to serve a lot of roles and share the load as much as they can.
With both of these, both small and big companies, the hours are long.
People don't always respect you for your work, and unfortunately, the pay isn't always great.
Typically, you will make a fraction of the salary that you'd make with the same skill set outside the industry.
But you know what?
That doesn't faze me.
I love the challenge.
Games are a really crazy field.
Video games haven't been around in the public for 50 years, and it's an incredible medium already.
You want to think about this-- the printing press was released or invented or became into mass use in 1450.
It's 564 years later, and now we have global information networks and we're complaining about these e ink displays, at how bad they are or good they are at displaying information to us.
The medium constantly changes on how that is distributed.
Just imagine where games can go when we are so young in this medium.
So I'm really here to talk to you today about contracting.
It has served as my company's primary method of revenue over the past four years.
So there are some really great benefits of it.
You get to work on a ton of projects.
You meet a lot of people, and you learn a lot about different things.
If you're going to go back and think about the projects that I've worked on, I got to work on a free to play property with a really big company who had a huge marketing budget, and it was really well-known name-- Dancing With The Stars.
A lot of people know that TV show, or that property.
In terms of brand recognition, it's great.
I got to work on a game with some neurologists and try to help show that games have positive impact in the world, and that there are hopefully ways, if the study does come through positively, that we can help people recover from concussions more effectively.
I got to work on a game with the army.
Despite what you may think about our involvement, if we can help people learn how to work with each other better, I think that's a really awesome thing to be a part of.
Now I'm getting to a point where people are approaching me to consult them for the experience and knowledge that I've gained over the course of the past four to six years that I've been in the industry so far.
So one thing to keep in mind, is that the great thing about consulting and contracting is that there is a ton of work out there.
There are plenty of people with ideas and money, but they don't necessarily have the skill or experience to be able to bring that to fruition.
Or their ideas were a bit too large, and they need some more people to be able to make those ideas come to life.
Sometimes, they just decided they want to bring someone in to help them out with the project that they're working on.
I would also argue that contracting, once you get into the rhythm of it, is a very stable way to make a long-term living in this industry.
From what I've been doing by contracting, I'm expecting and anticipating the project to project work that the industry typically creates.
With the network that I've been able to create for myself, I actually feel very stable that I will be able to find something that someone will be able to recommend me for whenever a project ends if I decide to keep contracting.
Ultimately, though, my goal is not to do contracting forever.
The reason why I created Zap Dot was because I wanted to make and release my own games.
When I started off, it was important for me to start generating some revenue for the company so that I could hire people who were worth the-- to bring people because they were worth it.
I wanted to be able to pay them.
So after the past four or five years, I've been able to gain enough capital for the company where Zap Dot will actually start working on its own titles.
So I figured I would talk a little bit about the contract life cycle.
Very much like Jenna said, events are super, super important.
So you'll go to industry events or you'll go to local meetups, and there are a ton of options, especially just in the Boston area alone.
And you network and you get to meet people.
The best advice that I can offer you when you're networking is, to not network for yourself, but network for others.
Find ways to make introductions between different people.
Find ways to get jobs for other people.
The way that I've been able to get such an awesome network that I have right now is by getting other people jobs.
In doing so, everybody comes-- a lot of people will come to me now for advice or when they have a project that needs to be done.
So I become a holder of information in terms of the projects that are available, which, when my company wants to pick one of those, I have, essentially, the pick of the litter for it.
Also, it's really good to continually help people also find stability in the industry.
To illustrate that, the first job I got was because I had a conversation in a floral shop when I was buying some flowers for my girlfriend.
The second job was based of a recommendation, and the third job was also based on a recommendation.
Another job was checking the Boston Indies email list.
And the most recent job, someone approached me about doing the work because they had heard that I deliver on time and my work is pretty high quality.
So let's say you do find someone.
You find someone and you want to do work with them, or they want work being done.
So what you do is, you do this really weird dance with the client and you talk to them for a bit about what kind of work they want to do and what kind of project they want to work on, why they want to involve you in it, and why you should be involved in their project.
You either become really scared about the expectations that they have or the money that they have to be able to spend, or you'll fall in love with them, or there'll be some in-between where this is a good calculated decision for you or your company, and it makes sense to do it.
From there, you take those expectations and you put them down to paper.
Contracts are really, really important.
Never do any work or have anybody do any work for you unless you have them sign some sort of contract.
Essentially, what these contracts you do is, they should manage expectations for both you and the person doing work.
What happens when something goes wrong on either end?
Be very careful with things like non-disclosure agreements or non-competes.
I prefer to have all non-competes written out of contracts that I have signed.
I've heard of other people asking for multiples of their time because the customer wanted the non-compete to stick.
One other really important thing to remember, especially when you're doing contracting is, you have the opportunity to fire a client as much as they have the opportunity to fire you.
If people start going off contract and they start requesting things that they didn't have and they're not following the rules that you've established between your relationship, then understand that you do have the right to get out of it if it goes sour.
So you've signed the contract.
Now you just need to make the game.
Then after you make the game, you get paid.
Jenna mentioned a bunch of different types of payment.
You can do hourly rates-- I've done that.
You can do milestone payments where you agree, typically, on two to four weeks of work.
You deliver that work, and as long as it is of acceptable quality, the client will accept it and then you the amount of money that you've worked on.
Or you can do a full fee for a project.
I've done all three.
I would definitely caution against doing full fee for projects, just because-- especially if you're a perfectionist or the client starts trying to add more things-- scope can creep up over time and you-- well, typically for a full project fee, unless you're very, very good about your time management, you'll end up doing more work than what you charged for.
Sometimes you'll find out that you need help.
Well, the cycle kind of starts again.
You've gone to these events that you people.
You do a little dance with them to make sure that they kind of fit the situation for them.
If you find someone that you want to be able to work with, you write up a contract.
You set up expectations for the work that they will be coming on to help you with.
And now, as an employer of some sort, you have to worry about managing two people.
Not only are you managing yourself and the work that you are delivering to a client, you're making sure that your contractor or employee is doing the same thing, that the client's happy.
Depending on how many people you add, it can take a lot of your time.
Over the past couple of projects, I have hired help or outsourced some work for the projects that I've done.
I've been able to hire about five to six people-- not all at once, and not all for the same thing.
But some of the work that I've had people come in for has branched from a couple hours to a monthly milestones that I've paid people to work on.
One really important thing is that, with contracting, especially if you're running your own company, there are a lot of other responsibilities that you have.
There are taxes that you need to worry about paying.
Stuff that normally would be done if you were an employee is not your responsibility.
There are definitely a lot of different laws, especially in Massachusetts, about hiring people.
You need to be very careful that you are treating your contractors like contractors and not employees.
Otherwise, you should be providing those employees benefits.
And then that's also the same thing, to make sure that the people that are hiring you are treating you like an employee or expecting you to act like an employee.
So make sure that you pay attention to the work that you're doing and really round yourself out on the rules and regulations around work.
That's it.
So I wanted to open the floor for Jenna and I to let you jump in and talk about any questions.
[APPLAUSE]
Yes?
Yeah.
So do you have any advice for marketing new games?
Because I think that's a thing that a lot of [INAUDIBLE] how do you promote your product and where do you do it?
So I've been doing a lot of ongoing social media.
We have a Little World Interactive Twitter account and Facebook page.
A lot of it depends very much on the type of game you're making.
The best approach to it is a slow drip type of thing.
You don't get a week away from launch and go, OK, time to market this.
One thing that I've seen work very successfully is, we have somebody else working in the IGC who's creating a first person dodge ball game, and it's just wacky and fun.
So he's had huge success on Reddit and with Live Streamer.
So he's an early access, but he's already been very, very successful because he's been able to tap into those communities.
It's a lot of, what companies can you tap into?
How do you reach those folks?
Then going to as many events as you can afford slash have time to.
In the past year, I've exhibited at probably a dozen different events-- some really big, some really small.
But that has been crucial to meeting players in person and also to meeting the journalists that attend those events as well.
So definitely approach it as a very ongoing got to put in a little bit of effort every week type thing
Yeah.
Marketing should be a role on itself on your project.
So very many people will leave it towards the end, like Jenna mentioned in her talk, or not do it all and just kind of expect people to flock to your game after you've released it.
The other thing is, also as Jenna mentioned in her talk, is that marketing is something that needs to start before your project has launched and continue after it has launched.
The more you can kind of interweave it into your game and the entire development process, the better it will be, the more people will know about.
The objective is just to get the most people know about your title as possible
And to add one last thing to that-- the way that I try to approach social media is to give people something that they're excited to share.
So if you're just like, the game's still on the store-- nobody's going to care about that tweet or that Facebook post.
But we do a lot of cute little cartoons of the characters from the game, share a lot of reviews, a lot of testimonials from parents and teachers.
The NASA online presence is phenomenal.
If you want to see how to do social media well, follow the NASA Twitter feed.
It's seriously astonishingly good.
They just did something the other day where they have a new upcoming mission called Orion, I think.
And they did the ABCs of Orion.
So they did 26 tweets-- one for each letter-- and each tweet had this beautiful graphic image of like, A for-- I don't remember, I didn't look it at all of them.
But it was very, very clever.
I ended up re-tweeting the letters, S-P-A-C-E, just because that made me feel clever.
But they got me to re-tweet five of their tweets.
That's a win for them.
So little things like that, you have to think about, what content can you make that other people are going to be really excited to share with their networks?
Yes?
What's the street off of [INAUDIBLE] and Massachusetts right now?
I know that there was some stuff going to state legislature about whether that's actually enforceable
Sure.
I've heard about that.
I don't think it's completely unenforceable yet.
I don't think that that's passed to make non-competes unenforceable
So there's debate?
Yeah
Yeah, that's a really good point, though.
You have to look very carefully at any contracts that you sign.
A lot of companies will put wording in there-- well, studios will put wording in there that they own anything you make regardless of whether you make it at work or at home.
So especially if you're someone who likes working on side projects, that can be very dangerous.
So just something to be aware of
Especially ideas that you've come up with during working on their project.
There can be a large-- the more vague they are, the more dangerous it is to you
We have another question
Have either of you worked with either state grants or small business grants-- organizations that basically give money for help for new companies?
What's the experience like working some projects if you have?
I have not yet.
That's something I'm actively looking into for the next game.
The thing that's holding me back is, it looks like a lot of work in a subject area I don't know.
I don't know how to write a grant, so that's something I would have to learn for a shot at money.
So that's what's had we not pursue that so far.
I'm going to be looking into education-specific grants, which I think I would just have a better shot at, because it's a more narrow field.
But-- not yet, but maybe
The neuroscientist game that I worked on was funded in part, I think, by a grant of the National Institute of Health--
I got you
Yeah, and I've heard of a few Indies who are working with institutions who have gotten grant money.
But I don't know of too many Indies who have gotten grant money on their own in the United States.
So other countries like Canada and Australia-- though, I think, Australia-- maybe not anymore-- how government-funded grants for the arts, so they include games in that.
But we do not, unfortunately.
I would love some government money
How common is it for studios to be acquired, and is that something they target?
So I would say, it's uncommon at a smaller level.
Definitely, the way that you're going to make money as a studio is not from an exit strategy of being purchased by a larger company.
It's more along the lines of the revenue that you'll be making from either your contracts or the profits from your game.
That being said, some companies have been purchased.
I know one of the other subcontractors I worked with Dancing With The Stars was purchased by a larger company.
I know that Amazon has purchased a few game companies when they were launching the Kindle to try to bring games in for their platform.
So it's not impossible, but it's definitely not an exit strategy that most people would start a studio with the expectation of having
Yeah, and there's definitely a group of Indie studios who are generally between 6 to 12 people and they have an actual CEO who can go out and raise money.
Those are Indie in the sense that they're small and self-owned and independently publishing, but it's kind of a different class of Indie, because they can be raising potentially millions of dollars to fund their games.
So those are the types of companies that would be most likely to be purchased.
Little Worlds Interactive is just me and my games, and so if somebody were to purchase my company, they would just be hiring me, basically, and like, purchasing my games.
So that's probably not something that's going to happen, though it's not unheard of entirely
And I guess a tangential approach that you could have-- that is, actually taking your games and licensing them out for other properties.
Yes?
How long were you guys in the industry before starting to do your own thing?
So I was in the industry for-- it depends on what you count.
So, grad school for two years and working for four years, and then I've been an Indie for two
I worked at Harmonics for just under two years before I started doing consulting.
The Culture Shock game was my first project out of it
A lot of people-- just to add onto that and then look at you up there-- a lot of people go straight from undergrad to Indie, and that's a totally valid approach.
But there is also so much to be learned in a studio.
Even if that is not your ideal environment-- it certainly wasn't for me-- but I regularly use things that I learned in my time working at a studio.
So I wouldn't necessarily recommend just going directly Indie straight from undergrad
And this is part of a different talk, but-- I didn't go into that.
But it's a bad idea, just because there are a lot of mistakes that you'll make on your own that you could have learned from a larger company who's already made those mistakes and can teach you better ways.
Not that they'll have all the right answers, but-- and it's not to say that you won't be successful if you were to do it right out of school.
It's just much harder to hit
I was just wondering, Jenna-- what did you work on before you became an Indie?
What were you doing, or what was grad school and then after that experience?
So after undergrad, I worked at a local startup called Hangout Industries and we were making one of the original 3D virtual worlds.
So it was meant to be a social chill place for teen girls.
It crashed and burned hard.
Most of the lessons I learned at that company were how not to do the things we were trying to do, which is valuable.
Which is a very valuable lesson.
So then I went to Indiana University and got my master's degree in game design, and then came back here and worked at Stomp Games, which was formerly Tencent Boston.
We did Robot Rising-- and that was a diablo-esque game with robots on Facebook.
So I've kind of bounced all over the place.
I've made mobile titles, I've made Facebook titles.
I had an internship at Turbines where I've worked on MMOs.
And like I was saying before, that experience really has just given me a breadth of knowledge that I've been able to apply to Indies
So a number of the other people we brought into class were talking a little bit about the internships that they had [INAUDIBLE] larger companies.
Can you talk a little bit about-- so you mentioned you used an intern for your process, Jenna.
I forget if you mentioned [INAUDIBLE]
I have not used interns yet, but it's something that I am very excited about
So can you talk a little bit about how it works for your small studio and what the experiences might be for being an intern at one of you studios compared to a larger studio?
Sure.
So the intern that I worked with was Sam.
He was an animator, and he went to school at a local animation school, I don't remember what it was called-- and I knew one of the teachers there.
As part of their graduate requirements-- I think they called it a co-op-- they just needed to put in a certain number of hours with a local studio.
The expectation was that it was unpaid, because they were basically getting course credit for it.
So for me, it was a risk-free situation.
Like, basically, I could get somebody to do art for me for free, but they were getting something out of it too, so I didn't have to feel too horribly guilty about not paying them.
And it worked out phenomenally.
So one of the big differences with working with an Indie versus a large studio is that, I basically said to him, I need animation in my game.
If you could do that, please, that would be great.
So he did all of the animation in The Counting Kingdom, first as an internship, and then as a paid contractor down the road.
So he got a massive amount of responsibility.
The only door art direction I gave him was, like, make it cute, but also scary a little bit, but not too scary.
So he had just a huge amount of creative control.
The downside to that was, we all worked remotely, and so I saw him maybe once every couple of weeks.
But we stayed in touch to make sure that he knew what was going on, I was going on.
So at a larger studio, he would have more face-to-face experience, but be working on a much smaller slice of the pie.
Given that Indie budgets are zero, basically, I would not be able to bring in a paid internship intern.
As much as I would like to, it's just not a reality.
If I'm paying somebody money, I have to know that they are going to be able to produce incredibly high quality work.
So unless you're very, very cheap and you already have an astonishingly good portfolio, it's not something I can afford
I would say one of the major reasons why we haven't brought in-- why Zap Dot hasn't brought in any interns right now is for the same reason.
I would want to try to pay something, and I've not been able to be in a position where I could bring someone in to give them a specific set of tasks to be able to work, or to have a project that is ready to do research and development.
I think one thing that is great for interns that I'm very interested in is actually having them do prototypes to test ideas that the company has for future plans and to get a good feel for that.
Because I think it could encompass lot of things that are great for interns-- being able to have a project, being able to deliver on things, and then also being able to have something that may have some progress going forward in a project that is released
That being said, a lot of times, we think about internships as kind of an all or nothing type thing.
But a lot of us could use somebody a couple of hours a week to help out with things here or there.
So if you genuinely are looking for the experience-- and again, I hate not paying people-- but if you do just want some experience working with an Indie studio and you're willing to give just a couple hours a week, come to Boston Indies.
Comes to Boston Unity Group where we're all there.
Come talk about the projects you're interested in and where you might be interested in helping
And those things can turn into paid work as well, as Jenna's intern did
Any last questions?
Three or four
So both of you are located at the IGC, but the impression I'm getting is that not all the people you work with are physically in the IGC [INAUDIBLE].
Can we talk about how you balance that?
And I guess [INAUDIBLE] the whole communications part of it, but what does that work flow look like with some of your people right next to you and some of those people on Skype?
Go ahead
Want me to go for that one?
Go ahead
Many of the projects that I've done have been remote work.
When I was doing Dancing With The Stars, that was a bunch of people.
So New Hampshire, Oregon and Florida.
Jack Lumber was Manitoba, Canada.
Then my apartment, and then Alex's apartment as well, before we had the opportunity to work together.
Typically what you need when you're working remotely is, you need a couple of things that are set in stone, essentially.
You need a schedule, you need deadlines that people actually make, and you need to make sure that you have the people that are working on those projects be very, very good at time management in the sense that they know they have work to do.
They don't necessarily need to have co-workers working around them.
So what you find is, you find a lot of tools being reused.
So Google Docs for shared collaboration, Trello for task management, and then Skype for group chats, I think.
Or Google-- we've done Google Hangouts as well for different projects.
But essentially, having something that allows the team to communicate with each other and knowing what the expectations are throughout the project.
Especially when the contracting work is remote.
Like, I'm going to do the work because I want to get paid.
I guess it's just a matter of making sure you have really good work ethic and a schedule that you stick to or update as things change
Yeah, it definitely varies from person to person.
But having the right tools to make your life easier is crucial.
So I'm on Skype all the time, and I also use BitTorrent Sync, I think is what it's called-- where basically, I have a shared folder with my contractors.
Any time they update a file on their computer, it'll be updated on my computer.
So it's just a very, very easy way make sure we have the most recently updated files, and don't have to keep renaming them, like, final-- final two-- final, really final.
It just keeps all nice and clear.
So having the right tools-- just as a few examples, because that'll probably be easier talk about with Luigi, who is the artist-- we would sit down and talk either in person or on Skype before every new big major task.
That made sure that we were both on the right page.
We both had a similar thing in our heads, and any design requirements, I was able to specify.
He was able to ask questions about some of the things I would forget about.
So making sure you have those regular face-to-face check-ins every time you're starting something new is crucial.
Emma, who is my social media guru, works four hours a week and she basically keeps our Twitter and Facebook pages updated.
We check in regularly, if there are any sort of campaigns that I want to kick off.
But part of the reason I brought her on board is because every time I would sit down with a Twitter account, I'd be like, I have no idea what I'm supposed to say.
Like, still working on the game.
And so her job is to come up with creative ways to be like, just a reminder, the game is still out there and you should purchase it!
In a way, it sounds like something people should be excited about.
So her job really is to work a little bit more independently and come up with that stuff on her own, but then there are some times, like, I totally want to steal this NASA ABCs thing-- and so I'm going to talk to her about it and plan that out and put that together.
So she does a lot of independent work, but then any major projects, we check in for that
Not steal-- appropriate
Borrow.
Borrow
Thank you so much for your time
All right.
Thank you.
[APPLAUSE]
You can go ahead and keep the microphone on, just in case.
So we've got a five minute break.
Come back down here and sign in if you haven't signed in for class yet.
They're available to play your games if you are ready for that sort of thing.
So if you are, find them, bring them to your computer, and have them play
We'll leave a bunch of our business cards up here.
If you want to send a portfolio, a resume, just get some feedback, please feel free.
I'll answer any email that comes my way
And I'm going to email this out to them, I haven't done it yet, but there's a party coming up, right?
Yes
Can you mention that?
Yeah.
The Indie Game Collective-- there's actually two parties coming up
Mention both
The Indie Game Collective is having a party at the Aeronaut Brewery in Somerville.
It's in between Porter and Union.
It is on December 11, starts at 5:30, goes until 11:30.
It's Thursday, so you'll probably have to leave a little early so you can get to class early on Friday.
But the-- ah, no one laughed, even politely
I think that's after the last day of class
Oh, OK.
Cool.
So the collective is doing a bunch of art and interactive installations.
It's games that are not our own games, but they're just fun things that we wanted to make for the sake of art, not necessarily making money.
Bit Fest Boston will also have 20 old school arcade cabinets there.
We will have a funk band and a food truck that has gluten-free, vegan, and non-alcoholic options as well.
On the Monday following, which I think is December 15th, Boston Indies is having their holiday festivus where people will be just playing games.
It'll be some local Indies who'll be demoing some games that they've made, but we'll also just be playing really fun games like JS Joust or Hokra.
Jenna did an Artemis game last year.
So who knows what type of table top or digital games will be there?
But it'll be a lot of fun too.
And that's going to be-- that is in the new Bocoup office.
Go to bostonindies.com and join the community in terms of finding out more information about that.
I'll be sending it out shortly
Thanks.
So take five minutes.
2:05, 2:10 we're going to take that thing away.
All right?
[CHATTER]
The following content is provided under a creative commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
And [INAUDIBLE] them for next week.
So for Monday just to really settle expectations for what we're doing on Mondays, we are doing rehearsals of presentations.
You do not need to do the demo of your game part of the rehearsal.
You do need to do the actual talk.
That means we're asking for the visual slides, any visuals you're going to use.
Go through the full talk as if you're going to do it.
Don't try to speed through it.
If it's in outline mode, then maybe you're testing the actual material that you're going to be going through.
Please pay attention to all the talks that you're listing to.
Give them feedback.
Give the other teams feedback about what they're saying.
We will be here basically kind of like The Voice but without the cool chairs, letting you know what we think about your stuff.
Yes.
A great way to get a good grade for the presentation is to listen to all the feedback and make changes, so keep that in mind
Just as a suggestion, even though you don't have to demo your stuff, it might be a good idea just to bring the computer [INAUDIBLE]
Absolutely
And test the hardware just to make sure that the screen resolution works out, you can go full screen, you don't have that Google Chrome has taking over your entire screen.
Hit Escape
Yeah.
So make sure the game works, but you don't need to actually play it.
Works in a period of time.
Time length for these presentations-- each team is given 20 minutes.
You will be timed.
We will flag you once you hit 20 minutes.
That's total for the entire presentation including the demo.
Limit the game-playing part of your presentation to no more than 10 minutes.
Anything more than 10 minutes, we're going to make a note of that, but please keep it under 10 minutes.
It can be as short as you like.
You could have someone play the entire game.
You can have someone play just one part of the game.
You will have somebody who has not played your game before playing the game live on stage, so that's also a test of ours.
Time for the actual slides part of your talk has to be at least 10 minutes, so 10 is a good suggestion.
Five minutes of demo, 15 minutes of talk is also fine, as well.
After you're finished with that talk, we're going to open it to Q&A from people in the group in the room, and that will include us.
So rehearsal is give your presentation.
We won't do the Q&A.
We'll just do feedback.
Final presentations are the real deal.
It is being recorded, of course.
We are still figuring out if we're going to live stream it.
Please invite your friends on Wednesday the 10th to come.
We're sending out invites to a bunch of people, to get some of our local developer friends to come in and talk, and also some folks from our department and from some of the other game design classes to come and watch.
Attendance for those two days-- so attendance for Monday is going to be normal like we've just been doing it.
Attendance for Wednesday-- come in at 1:00, please.
Let us know if you're going to be late, like email the video game bosses list if you are going to be late on Wednesday for the final presentations.
We'll keep the sheet in the back of the room so you can sign in as you come into the room.
Anything I'm missing from any of that?
And then at the end of all the presentations, we'll probably going to end up with about 30 minutes left of class, so you can open up your computers and have people play your games if you like.
Totally not required, but it tends to be fun when we do that.
Cool.
So if there's no questions about any of that, we are going to do our last lecture.
We've invited Sean Baptiste from Fire Hose Games to talk to us about community management, marketing, and all sorts of stuff, and basically how do you get people to play your games.
And after he's done, attendance will be back open
Cool.
Thank you very much.
Thanks for having me here.
This is great.
My name is Sean Baptiste.
I'm a 10-year veteran of the video game industry.
My first job was at Harmonix, which was a huge AAA sort of deal.
And now I'm at a small indie game studio right here in Cambridge that was started by some former game lab alumni, Ethan Fenn and Sharat Chat and Eitan Glinert.
I think I should probably start how I got into the industry in the first place.
I don't have a tech background whatsoever.
I came up through theater.
I was a theater major, and one thing I've realized in my years is that there are a lot of theater majors in video games.
I've been thinking about it a lot.
I think a lot of it is that video games and theater both have an interactive element where there's interplay between the audience and the creator that film doesn't have necessarily.
And I think a lot of theater people drift towards there when they realize there's no money to be made whatsoever in theater.
That's how I made it.
So I was working in film and I was working in theater and stuff and I got sick, and just the only game I could play was Amplitude, which was an early Harmonix title.
And as I was playing it I was like, oh, I think people make these.
Like, I don't think games just happen.
I think somebody makes them.
And I realized that was-- I lived up on the North Shore, so it was close, and I just started pestering Harmonix to hire me since I was close.
And they kept telling me no until eventually I just got a friend there who sort of said nice things about me, which was lovely of him, and that sort of got me in in QA.
QA, I'm hesitant to say that that's a good way to start in the industry anymore because I don't know if it's quite as valid.
QA teams have gotten smaller and the turnover is a little bit difficult, so it's not necessarily the entryway into the industry that it used to be, which I do think is a bit unfortunate.
I worked as a QA person on the Karaoke Revolution series early on and then on the Guitar Hero series before I end up becoming a technical artist, which meant that I wasn't a programmer or an artist but I helped make them have less fights with each other, which is probably something you guys are used to at this point.
So I did a lot of that and learned how to animate and hook things up.
But then they realized-- one of the things I was doing was-- so Harmonix, I don't know how familiar you guys are with Harmonix's [INAUDIBLE], but this was before Harmonix got bought by MTV.
And Guitar Hero had come out, but that handled by RedOctane and Activision-- RedOctane, then Activision-- and we had an outside PR place handle a lot of the PR.
And mostly we just made games and put them out there and there wasn't as much of a consideration for us doing PR from inside.
But we realized as we wanted to make Rock Band and we're moving away from Guitar Hero and kind of owning our own destiny that we were the ones who were going to have to talk up our own games and kind of create something about it.
So I wasn't the best technical artist, I'll be honest.
So they moved me over to running the community.
From Frequency and Amplitude, there was a very small community of I'm going to say about 50 or 60 people on the Harmonixfreak.com forums and I was only one willing to talk to them pretty much in the whole company.
So as part of my job is technical artist, for an hour each day I would sit and talk with people on the forums and try to manage that community, which was very, very small and very angry that there wasn't a sequel to Frequency or Amplitude, that that was kind of a dead series.
So a lot of my time was spent apologizing, I guess is how I would-- that my big start to community management was apologies.
So I did that for a while.
And when it came up that we were going to have to start doing our own stuff, I got promoted up to create the community development team at Harmonix.
And I put that together, and that's when there's a big change because then we're owned by MTV and they have a lot of money-- like, a lot of money.
And they put a lot of money into advertising for the game and sending us around to, like, every event and stuff.
I went to the Grammys and was giving out Rock Band to Morris Day and the Time and weird stuff like that.
And it started moving a little bit away from just general apologizing, daily dealing with problems, and more with kind of creating, or at least sustaining, a cult of personality around the persona of Harmonix, which was sort of punk rock and kind of outside of-- it wasn't as much like traditional game development.
Everybody there was a musician and there was actually kind of this identity that we could point to and kind of play that up and make that sort of the character of the company.
And we definitely did that.
We played that up and because one of the things-- and we'll get into that more when I talk a little bit more about the difference between marketing for AAA verses marketing for indie-- is a lot of games in AAA, you're selling the games.
As far as marketing goes, you're selling the games.
For indie, what you're doing is you're selling the people who make the games, and Harmonix was in this interesting place where even though it was owned by MTV, it worked a lot more like an indie company.
So once we were able to promote ourselves, we promoted the company as a group of people.
Like, don't just buy Rock Band by Harmonix.
Buy the whole company.
Buy what we do.
Stick with us because we're going to keep making cool stuff.
And I did that for a number of years and it was crazy-- flew constantly everywhere, stayed in hotels that didn't have lice and stuff.
It was nice.
A lot of getting to do really cool things.
I ended up getting sick after that and spent a few years away from the industry.
And when I came back, I'd been playing so many independent games from small indie teams-- things like stuff that John Blow had made or Fez or just a lot of the games-- Castle Crashers, Alien Hominid, those sorts of games.
And I realized there was an analog to what I was already doing with Harmonix, which was kind of the-- in indie games, you really do have to sell the artist.
You're selling the artist.
You're trying to get people to come because if you don't have an interesting story in a way or that you can sell to the press or to journals, you're not going to get anybody writing articles about you for the most part.
That's just the case.
The big AAA companies have lots and lots of money so they can plant lots and lots of articles.
And as far as indie stuff goes, the only way you can sell yourself is having a good story that a journalist wants to tell because if you don't have that, there's really no upside for them to write an article about you or your game or anything like that.
I know that seems a little cynical, but there it is.
That is the case.
So let's see-- so when I went to Fire Hose, the differences in the marketing, like I said, was when I was at Harmonix, a lot of what I did in community management was making sure everybody was really invested in the product, in ourselves as a team.
But a lot of it was also keeping our secrets-- making sure that people weren't leaking information from inside of the company, making sure play testers weren't leaking inside of the company, making sure an unannounced project that a coder on there didn't go in and update their LinkedIn profile and say, hey, we're working on Rock Band 2 at the time before we'd announced Rock Band 2, and then there's a huge thing.
So a lot of my stuff was managing what information got out.
In independent games, my job is to make sure all of the information gets out and that we are constantly screaming for people to notice us because there's so much competition.
I mean, just look at the explosion of independent games in the past let's say five years, six years.
2008 I think is really when it really started becoming this definitive thing that you can point at.
There's a lot of people, and a lot of people who also equally want people to pay attention to them, and there's only so many outlets and there's only so many places where people are going to be able to read about it.
So you're really fighting.
There is a sort of-- it could very easily turn into a sort of crabs in the bucket situation, just everybody pulling each other down.
Luckily that's not really how the indie game scene works.
A lot of people are trying to push each other up, which is nice, but you're still competing with so many other people for a limited amount of slots of articles.
The one thing I've learned since I've been at Fire Hose is that actually matters very little if in the end-- and by that, I should say I mean that in terms of sales of the game.
A big article on Kotaku isn't going to sell your game, not anymore.
When I first started in the industry, that absolutely was the case.
A big article in GameSpot or IGN, one of the big-- 1UP at the time-- that would put you over.
That could definitely help put you over.
But at this point, that's just simply not the case.
And even a year ago, I was saying that, and that everybody should probably put more of their marketing time towards getting YouTube people to cover their games.
And now that's sort of the predominant theory, but I think that's going away, too, because everybody who likes games has a YouTube channel and does Let's Plays.
So in a way, it's sort of a reflection of what indie is doing at the same time.
Now that there's just so many people, the market sort of flooded.
The novelty has worn off.
So it certainly can help, but it's something you have to be careful, too.
You can't put all of your eggs in one basket.
But with indie, too-- so when I was at Harmonix, I ran a team of I think at our height 12 people.
So that was 12 people doing just community management and just doing public relations.
At Fire Hose, I do that, all of it, and I'm also a designer.
So it's like a part of my overall job and I'm the only one doing it.
So I have to choose what we do far more carefully and I have to do it without any money whatsoever.
Certainly you can put money into buying ads or doing Facebook ads or that sort of thing, but I don't think that that's necessarily the best use of your time all the time.
And a lot of companies don't have the budget beyond their salaries to actually buy these ads, so what do you do?
And that goes back to what I was saying.
It's community management, and I guess I should get into a little bit of what community management is.
When I started doing community management in 2007, it wasn't a thing really.
I think there was one video game company that wasn't an MMO, that didn't make MMOs, that actually had anything approaching a community team, and that was BioWare.
Right around Neverwinter Nights, they put together a community team of two people, I think, and that was just to manage their forums.
And let's see, that's probably around 2002/2003.
And in that time, you really only put together a community team if you had a large multiplayer game, generally an MMO.
So at the time, Asheron's Call had something or Ultima Online had it and early World of Warcraft all had these community teams.
There was an element of promotion to it.
There was an element of marketing to it, but it wasn't so much that.
It was more customer service.
They were really there to deal with problems that came up as they went.
BioWare where they sort of broke the mold a little bit was it wasn't about that.
It really was about the company having direct access to communicate with their fans, and vice versa.
I once was half-joking that my job as a community manager at Harmonix was ombudsman because I felt like I had to represent the company to the fans, and then I had to represent the fans to the company, which involved being as honest as possible at all times with both, which is a real difficult tightrope to walk.
So with community management, what you try to do is you try to build out the people who love your games.
You're trying to make them into the people who talk about your games online.
You're basically creating a kind of an ad hoc sales force, a word of mouth way of promoting your game because that's like if you read an article on Kotaku, do you immediately go out and buy a game?
Probably not.
You might do a little bit more research on it.
You might read up a little bit more on it.
You might go to Metacritic and check the reviews, that sort of thing.
But if a friend tells you, like just a person you know is like, "Hey, I'm checking out this game.
There might be multiplayer.
We should check this out," you're far more likely to buy it.
Community management is the art of managing word of mouth.
You're getting people to tell their friends, the people that they know.
You're getting people to tell the people who trust them that this game is good so that those people then go on and buy it.
And join your community, hopefully, and then they go on and tell.
So what started for Harmonix, a lot of what we did is because-- and there's a scalability aspect to it.
When I created the Rock Band community site, that was 100% forums-based.
We did Twitter.
We did Facebook.
Like I said, we did tons of live events, but everything was very much based around the forums.
Part of that was because Twitter hasn't happened yet, or was just happening and not as many people were talking about it.
This was early 2007.
So everything we did, we build our this huge site.
We had forums.
I think I was managing at that time a registered community of about 350,000 people on the forums, and that's why we needed a team.
It was big.
And there was an aspect of customer service.
You might remember-- I don't know how many of you guys were really playing that at the time, but when it first came out, there were issues and articles about the issues about the instruments.
Any time you debut new hardware, there's generally going to be problems or there's going to be issues, and that happens.
So leading up to it, it was very much like, Rock Band's going to be awesome.
We're doing rock.
And then as soon as it came out and those issues started coming up, the job just clamped down into 100% customer service all the time, just trying to manage all of that.
Which was something that having come from technical, I wasn't necessarily prepared to do.
But I think with any game, no matter who your audience is, no matter what, it's very valuable to have some sort of protocol for customer service leading up to it.
That's something I think that you should definitely do.
At Fire Hose, we don't have forums.
We have a Facebook page.
We have Twitter.
But forums, they're not what they used to be.
The communities aren't forums, aren't the same as they used to be, and sometimes they're just more trouble than they're worth to manage.
It could be it's very time-consuming to stop all the spam, to get in the middle of fights and stop people from hurting each other.
And we were very hands-on.
We weren't a website where it was like, say whatever you want.
Post whatever you want.
We were very hands-on about abuse and that sort of thing.
At Fire Hose since we don't have as big of an audience-- we don't have 350,000 people who are following what we're doing who are registered users who are going to follow everything that we say and everything that we do and are really hugely invested in the company or our games-- so we have to find other ways.
One of the things that's probably the biggest thing-- I should bring this up here-- is we started up a Twitch channel, and this was just as the live video and live streaming was starting to really become a thing.
So it was a couple years ago.
We had Go Home Dinosaurs, which you might have seen on the scroller.
That was a game that we put out last year.
That was coming out and we wanted to get attention for it.
And we realized we wouldn't be able to get every the YouTube video star to feature our dumb game that was awesome-- it was awesome-- about dinosaurs stealing barbecue from early mammals.
We knew we weren't going to get every YouTube star to cover it.
TotalBiscuit did actually, somehow, but we realized we kind of had to build our own community and really kind of make our own press for it rather than spend all of my time.
Our marketing budget is my salary.
There's not extra stuff where I'm buying up ads and everything.
So we started meeting with Twitch.
Actually, they came out here to talk about how they wanted to get more indie games on there and we sort of struck up a relationship with people at Twitch, and that got us front page promotions.
So for instance every day, Monday through Friday from 12:00 to 12:30 on Twitch, you will see my big stupid face staring out at you from the screen because we're on the front page right in the thing, and anybody who goes to Twitch is going to see our game first.
That cost us absolutely zero dollars.
As far as budget goes, all that takes is my time because we talk to them and we try to create good content that also helps their brand, as well.
But that brings us-- you can see so far since we started up our channel, four million people have watched it.
Four million impressions is way outside of our price range.
It's just outside of our-- that would be tens of thousands of dollars to get that many people to get an impression of our product, to get an impression of Fire Hose, to get an impression of our games, to get an impression of the people who worked there.
It would take that long, and part of the way we do is like I said, every day, Monday through Friday 12:00 to 12:30, I put on a show.
It's about Let's Quip!
That's actually my game.
I should mention Fire Hose, we have three games going right now.
We also do outside projects.
We take on contract work.
Right now we're working on a project with DARPA that's pretty cool that actually teaches people not to shoot people.
It's pretty fun.
But we have three eventually commercially available products that we're working on right now.
One is Let's Quip!
That's a game I'm designing.
That's a comedic debating game.
That's what I do on the show Monday to Friday 12:00 to 12:30.
Just I invite local comedians in and we just talk about different aspects of comedy while playing the game, and that has actually been wildly successful.
I had Jackie Kashian, who's a very big touring comedian who I'm a huge fan of.
She came in on the show.
And on that show, I think, over the course of that one half-hour, we had something-- I want to say it was 24,000 people, which is great.
Like I said, I would not be able to pay for that.
I would not be able to afford that.
But all it takes is my time.
We also have three other shows that we do.
So there's the Monday show.
So that's 2 and 1/2 hours out of the week.
And then we have a Tuesday show in the afternoon from 4:00 to 5:00.
That's one of the other games that we're working on.
Chris Chung, who's a local developer, he's working on a game called Catlateral Damage that you may have heard of.
Catlateral Damage is really, really rad.
You're basically a cat.
It has first-person shooter controls.
Your family's left the house for a little while, so you have a limited amount of time to just destroy all of their stuff.
And it's great.
You're just pouncing around knocking things on the floor.
You get points for destroying people's monitors.
It's great.
It's exactly what cats do.
It's very, very fun, and Chris is this lovely guy.
He loves cats in a way that's sort of very honest.
There's nothing ironic about, and he's kind of the perfect person to host a show about making a game about cats.
So we gave him this spot, 4:00 to 5:00, where all he does is he sits there with Unity open and he shows what he's been working on that week.
And he tells people, like, we do this, we do this.
I just added in these things.
This is how I did that in Unity.
So it's one part demonstration of his game and one part tutorial, and that brings in people, too-- people who want to learn more about making video games as well as people who are just interested in a game about cats knocking your stuff over.
And that really works for him, and it really fits the theme of this game.
The other game we have is a game called 20XX, which we just announced two weeks ago.
If you are a fan of games like Mega Man X or those sorts of platformers, it's a game very much similar to that.
We're working with a team out of I believe Maryland.
It's where they currently reside.
So part of that is we-- I should clarify that Fire Hose Games late last year as opposed to just a studio, we opened up an incubator for independent games.
So for the most part, local indie game developers who have good games that they really want to work on can come to Fire hose, get a monthly stipend, get office space, and we'll help them with the development of the game.
We'll help them with the promotion of the game.
We help them with kind of everything.
And it's more like a partnership.
Both Catlateral Damage and 20XX are part of that program.
And it's great working because that's like Catlateral Damage and 20XX and Let's Quip!
are three incredibly different games.
Like I said, 20XX has a very sort of Mega Man X feel.
It's very hardcore.
It's very platformery.
Catlateral Damage is about cats, so that has a completely different audience.
I'm sure on a Venn diagram, there's some crossover.
And then Let's Quip!
is just a game where you write words down.
Like you write tweets and then you try to be funnier than other people.
The three games have almost no shared audience across them, so we have to approach everything very, very differently.
I think showing Let's Quip!
but actually showing comedians playing it, that helps because that actually plays into the narrative of the game, the fiction of the game, which is allowing you to be funny.
And then we have comedians come in and kind of vote on whether you're funny or not, and that's creating sort of a cult of personality.
And at the same time, they're also meeting the creators.
So like I said earlier when I was talking about how people pushing the artist as opposed to the art, we sort of play into that a little bit.
Having Chris Chung come in and just show development on [INAUDIBLE], it's so earnest.
If you get a chance, you should really watch it because he's just the most likable, smart dude who's just going through.
And for a live stream where he just basically has Unity open and he's just moving things around, he gets several thousand people over the hour watching him, which is nuts.
But it's great.
People love it.
With 20XX, we've just started.
I'm just working on the marketing plan for that, the marketing map for the next six months or so.
And I should say that's when I'm just planning out this month, these are the things we need to talk to.
We need this many screenshots.
We need to have a trailer for February.
We need to have this.
And just coming up with those milestones in the same way that I'm sure you come up with your milestones for your project.
We're doing the same thing, but on the marketing end.
These are the goals we need to hit in December.
These are the people we need to talk to in January.
If we can get an exclusive for February, that means we're going to need to be talking about it in December and have a pitch for that, have some sort of PowerPoint presentation or just a really good email with good resource screenshots and videos and stuff that we can send out to different outlets to try to get a little bit of coverage.
Even though I said that articles don't have a direct effect on sales, they do have a direct effect on zeitgeist, I guess I should say, so there is a sort of cultural cachet that comes from having a big article on your games there.
It doesn't necessarily translate to sales.
But if you know how to use it and take advantage of that cultural cachet, you can bring more people in, which I find useful.
But every single one of these games, I have to market differently because they're marketed to different people.
Like I said, 20XX, also that game is going to be on Steam.
It's I think it was $14.99, and that game we put out in early access.
That's one of the early ways we're trying to market it is getting it out on early access and getting people who are excited about Mega Man-style platformers really talking about that game and very excited about that game.
Whereas with Let's Quip!, that's just going to comedy fans.
All I'm doing is talking to comedians for the most part because comedians if they like the game, they're going to tweet about the game.
And it turns out comedians have lots of fans who like comedy.
Who would've thought?
And then with Catlateral Damage, as far as I can tell, the audience for that game is every person on Earth.
It involves cats.
It involves mindless destruction.
It's everything the world is asking for right now.
It's just beautiful and I love Chris and I love the game.
It's amazing.
But since in a way that's a little bit more of a mainstream game, that also takes a completely different approach because we're not selling that game to hardcore gamers.
Hardcore gamers are certainly invited, but this is a game that I can give to my mom or I could give to a friend who already likes video games, and both could get something out of that.
And that takes a different approach because you don't want to come up with a-- like, I wouldn't want to come up with an ad or a promo for the game that was like, this is the baddest game ever.
It's going to destroy your life.
I'm not going to be advertising that way because that'll turn off the people who are not hardcore gamers.
They're not going to want to play something that acts like it's going to destroy them.
Whereas also if you go too soft on the other side, that's going to turn off other people.
So you're going to find sort of a midway line between that, and that's another thing I'm working on right now with Chris.
But as far as video goes on Twitch, I think that's very, very helpful.
I should probably start getting ready to take questions, I think.
But I guess one of the things you asked me, too, is about IGF in the IndieCade.
For me, it's a hard thing because I do certainly find value in marketing for as far as award shows and stuff, like the IGF getting awards.
I personally have a whole lot of problems with the whole how awards for any art.
I guess I think I just have personal problems.
I think it's very, very valuable.
And I think especially for young developers who are trying to their name out there, getting your game into as many festivals as possible is great.
Don't break the bank doing so.
A lot of them require submission fees.
So if you can find festivals that fit the genre of the game, just do your research.
Know what you're submitting to.
Like IGF has something like 7,000 people submitting every year, and that costs money for you to submit.
So if you think you can compete, definitely do it.
Otherwise, put your efforts into a different festival which still you could win something and then you can put that on everything that you market the game with.
Like, the winner of the 1989-- I don't know why I went with 1989.
I'm old.
So, yeah.
You just put that award on there.
I'm sure you've been on Steam and you've seen they have a section where you can actually just put links to every award you've got, and then it shows up and there's a little graphic.
That's really good and that can get your name out there.
And even though I have a problem with it, I can still see the value in it.
IndieCade is one of the funnest things I've ever been to.
And every time I get really down on this industry, which happens, going to IndieCade makes me remember why I love working with the people I love.
I love this industry, and I love the weird crap that people come up with because it's a great place for games that were completely outside the mainstream to exist and live.
That's how I found out about this game last year called Dominique Pamplemousse.
I don't know if any of you have played it.
It's a musical.
It's an opera game, which is adventure singing.
It's nuts and it's great, but people never would have found out about it if it weren't for just winning the IGF Award, for noticing weirder games that are outside the mainstream like that.
But I think I should open things up for questions if anybody has anything specific.
Yeah?
So you mentioned about how articles don't directly translate [INAUDIBLE].
Do you think that's also starting to be true for AAA, or is it more of a direct [INAUDIBLE] expect to get [INAUDIBLE]?
I think there's probably still some value to for AAA, and part of that is because the one thing that's great about being a AAA developer is you have resources, and lots and lots of resources.
So one of the things we would do any time you talk about the launch of a game, it'd be like, OK, so let's say Assassin's Creed.
The next one supposed to come out on September 9, 2015.
You want to own that Newsday, and that means you are planting articles on it for September 8 and September 9 at every outlet in the world so that that day when every person who checks video game blogs, reads a video game blog, the only thing they're going to be reading about is your game.
It's basically like playing a commercial on a loop for an entire day until you've subliminally made that person buy your game.
That's sort of the whole idea.
Independence-- we cannot own a Newsday.
It's very difficult because it's expensive and time-consuming-- really, really time-consuming to own a Newsday as an independent because just planting one major article in a big thing, like say a cover article for GameInformer-- that's so many man hours between the person who's negotiating that deal and making it happen, coming up with a story like the narrative for the game you're trying to sell.
Plus also art resources, like getting your artists to deliver a whole bunch of screenshots, but also Photoshops of all of the characters.
That's so expensive in time and money, and indies can't do that.
So I think for us, we can get articles.
But really getting an article in everything simultaneously on the same day-- the funniest thing that ever happened is when Go Home Dinosaurs came out, I was sending review codes out to things.
And I was like, so we have an embargo for this, and one reporter literally wrote back, Ha, ha, ha, ha, ha, ha.
We don't have that.
We don't have the muscle to be like, we can back up this embargo.
AAA places have a hard time backing up an embargo, but they have the benefit of being like, well, if you don't adhere to this embargo, we're not going to give you coverage of the next game.
Whereas if you're an indie, you would be cutting your nose off right there.
Like, fine, you don't get to do the next game, and they'll tell you won't have a next game.
So like I said, there's certainly cultural cachet I think for indies, but it's more valuable for
So community management as you described it is a word of mouth deal for a lot of it.
Is part of your purview then doing bundle sales and other types of sales where if you get somebody to play the game, maybe they'll talk to somebody else about the game?
Yeah.
Doing bundle sales-- see, that's another one I'm torn on because I see the value in bundles sales, but I see that that value is diminishing day by day.
There are so many bundles out there, Humble Bundle being one of the biggest.
But then there's, like, 70 other ones that you can go to.
And Humble Bundle's great to work with because they're very protective of the Steam keys that you put out, whereas a lot of the other bundle places, they don't have as much security.
So what happens is you put your thing in a bundle.
That bundle sells 10,000.
You get $0.13 on the dollar for whatever people paid, or less.
Probably less-- probably much less.
But you're not getting that much money.
You're selling at a discount to try to get a huge amount.
The thing is what a lot of the people buying those are, they're overseas.
And what they're doing is they're not buying the game because they're fan of you or your game or they want to play it or your game was just one other thing in the bundle.
They're buying up Steam keys and then they are selling them on their own sites overseas.
So basically they're buying your game and 15 other games for $5.00 total in the bundle, and then they're selling all of those at half off of retail price.
And that cuts into your sales so bad because when people actually do want to play your game, they're looking for a deal.
So they end up going to this off-brand site that's selling Steam codes from it, and it's rough.
There's a lot of indies who've shot themselves in the foot by going to Bundle Crazy and then really killing the long tail on their sales in my opinion.
Yep?
How big an impact did TotalBiscuit have on Go Home Dinosaur sales?
I think a few more people found out about it.
I don't think it had on sales, not much.
Not much.
And even at the time, that was when that was sort of happening.
And it wasn't the most glowing play-through of all time that he's ever played it, but he liked it and he said it was really fun and dumb, which is what we were going for.
But in the end as far sales goes, it wasn't as big of a driver as one would think.
I do know of games that TotalBiscuit has covered that has had a pronounced effect on sales, but they're multiplayer games.
Like our game, it was single-player.
Whereas if you have a multiplayer shooter or if you have something like Hearthstone, a game where people are playing together online, that's his audience.
His audience does care about e-sports and does care about big multiplayer games, and ours wasn't that.
So I don't know if that was necessarily the best fit, but still nice that he liked it, nice that he played it.
Oh, yeah?
Go ahead.
No problem
Do you also manage the Steam forums for any of the games, or do you solely do just--
I do you manage the Steam forums a little bit, mostly for customer service because that would be the first place people would go rather than emailing us.
We had a link inside the game where people could email us if they had problems, if there were bugs or something like that.
[INAUDIBLE].
[INAUDIBLE].
But a lot of people went straight to the Steam forums and would talk about their problems on there and do direct sort of stuff.
But as far as building up the community there, some games work for that and some games don't.
I think games that are multiplayer just do better as far as Steam forum communities.
It's just easier to grow a community like that, build a community there.
But where for single-player games, people are less interested in, I think, talking to other people about them.
Maybe Skyrim, that sort of a single-player game where there's a huge expansive world and there's more to share and it's less linear, I think there might be more room for community on Steam.
Yeah?
Can you talk about IndieCade as an event [INAUDIBLE] rejuvenation.
On Monday, we had Jenna come in.
[INTERPOSING VOICES] She was talking about how she spends so much on events.
And I'm wondering how do events factor in and how you think about events in terms of sales, in terms of [INAUDIBLE]?
So events are difficult for indies because events by their very nature cost money to travel to, to stay at.
Luckily the independent community is great because we'll just call up 19 other companies and be like, hey, you guys are going to this?
Cool.
Want to get an Airbnb?
And then we just share it and try to keep costs down, but it's still expensive.
You can't do that for a flight or to get places.
Well, you might be able to.
You shouldn't.
But I will say this-- if you can go to an event, if there's any conceivable way you can go to an event, and I mean just about any event where there's other game developers there talking about games, any sort of game-based conference or expo, go to it.
Absolutely go to it.
I've been talking about word of mouth.
One of the best ways of word of mouth is going out and meeting other developers who have other communities supporting their games-- talking to them about their game, you talking about your game, and just sort of sharing because then they are going and they're going to be probably be, hey, check out this new friend I made.
He's making this game and it's pretty cool.
And you're getting more people noticing it.
And it's better to be part of the overall game development community than it is to be outside of it.
But by the same token, sometimes you just don't want to talk to people, and that's fair, too
Actually, it made me think, in the past, did you play other people's games on your FireHost?
Yeah, I should mention that.
So when we first started out, like I mentioned, we did it because we wanted to sell Go Home Dinosaurs and another game that we had made a port of it called Slam Bolt Scrappers.
And so we started off with that, and it became very apparent after two weeks that we only had two games, and that's not a good show.
That's not a great show.
So that was our Wednesday show.
That was our inaugural show on our live stream channel.
So after those two weeks, we were like, what else can we do?
We shouldn't give up on this because we don't have two games.
How do we build our audience more?
And then we were like, what if we played other people's games?
And then we realized, well, what if we bring in other developers and then they bring in their games with them?
And then we play their games while they're standing next to us, and then we just talk about it?
And I have a standing desk at work and it's just two of us standing there and playing games.
And we just interview and talk to them about the game that they made while we're playing it.
And I've called that sharing.
You create it, you bring your own spotlight, and then you share it.
And that helps.
We help get other, smaller-- because we don't do a lot of big games.
It's very rare.
We did Super Smash Brothers, I think is the biggest game, the biggest marketable game, that we've had on our live stream, and that's just because everybody at the office is addicted to it.
But largely, we have other people in.
Like in January, we had Zoe Quinn in.
We had people before that, like just a bunch of other developers throughout the country because sometimes they couldn't come in, so we'd have them Skype in.
And it's helped them, but it also helps us because it gives us content for our show.
It helps give them exposure and it helps build our audience over time because they're tuning into an exciting show.
And watching game developers talk about game development and seeing that sort of inside baseball is for some reason exciting to people.
I don't know why, but there's something that people enjoy tuning into about it
Anyone else have comments, question?
All right.
Thank you so much for coming
All right.
Thank you very much for having me.
Thanks, guys.
[APPLAUSE] Good luck on your projects this week.
I look forward to seeing them
Are you going to be able to just stick around for a little bit, or you have to run off?
Yeah, I'll be around
Cool.
So if anybody wants to talk to him now, feel free to come on down and talk to him.
We're going to break for about 10 minutes.
Come down and sign up.
Other thing I should note-- [INAUDIBLE] is coming today at around 2:30 to talk to some of the teams if you want to talk to her.
If you have burning questions about your game that you need to get answered by one of our clients, she's here and available to talk you.
Otherwise, the rest the class period is for yourself.
[CHATTER]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR 1: So we want to maximize time today, both to make sure that we're doing-- all of the presentations get good feedback, but also so you have time afterwards to make some changes.
So, if you haven't signed in yet, come on down sign up while I talk.
Just get that out of the way really quick.
Schedule for today is, we're going to probably be done with rehearsals by 2:30.
After 2:30 we are going to have Luke from OpenCourseWare interviewing teams and producers from teams about the class.
So, we're thinking about how many minutes per team?
Not too long.
10 to 15, casual chat.
PROFESSOR 1: Yep, casual chat is optional, but I would appreciate it if at least people who did production and Scrum Master roles would talk to him.
But it could be the entire team talking with him, and maybe one or two people dominating the conversation.
That's OK.
The goal for this is for people who are looking at the OpenCourseWare site, particularly teachers who might want to use these materials in their classes, to find out from the student perspective what students got out of the class.
Because it's kind of like student evaluations.
We're going to be out of the room for that period of time.
So that means after 2:30 we're out of the room, Luke's doing interviews, and your teams are doing whatever teamwork you need to do to get ready for the demonstrations and presentations on Wednesday.
PROFESSOR 2: Whatever you say to the camera, we will not even get a chance to see it before we give you grades.
So you can whale on if you'd like.
PROFESSOR 1: Please do.
Or at least, please be honest-- let's try for that.
I should also note, student evaluations are available online.
Please fill those out.
We're going to-- actually it's a great period of time from 3:00 to 4:00 in class today to just open up your browser and fill out your student evaluation, right here and now so that you get that out of the way.
And again, that feedback's going to be really helpful for us.
It's going to change how we teach the course in the next year.
We change it every year based on feedback we get from students.
So topics you thought you wanted to hear more about, topics you thought we covered too much, whatever feedback you have for us, do let us know.
Last announcement we have a guest here, [?
Leigh, ?]
from Odyssey magazine
It's a kid's science magazine for kids aged 9 to 14.
And we're doing an issue devoted to gaming.
And [INAUDIBLE] graciously allowed me to come and observe you guys as you rehearse your presentations.
And I'm going to be taking some pictures, and I may ask some questions.
I have some photo releases I'm going to need you to sign if I use your photos.
So I'll pass those around.
The good news is, if you don't want me to use a photo of you, don't sign it.
Because I can't use your picture without it.
So that's just an easy way to handle that.
So, I'm just going to pass a pile of these around.
And if you're willing to let me use your photograph, fill one out.
Thank you.
PROFESSOR 1: OK. All right, so for our order of presentations just for today, not necessarily for Wednesday-- we'll figure out the actual order by the end the class today.
We're going to start with Heat Wave, then Hello Waves, then SNAP!, then saving Gora Gora, then Cholera Control.
What we're asking you to do is come up, plug-in the computer that's going to be used to demonstrate your game and make sure your game works on the projector.
The display resolution the native display resolution is 1280 by 1024.
If you're doing a 4 3, or 1280 by 768 if you're doing widescreen.
It shouldn't make a huge difference for your slides, but it might be a problem for your game.
So, make sure you're coming up, doing a quick two- or three-minute check to make sure the game works.
Afterwards, plug-in your slide presentation, and we'll just go step by step.
One thing I'd like to know from each team is how many speakers do you think you're going to have-- how many main speakers you have in your presentations.
So, starting with Heat Wave, how many speakers do you have?
One.
Hello Waves?
Two.
SNAP!?
Two.
PROFESSOR 1: Saving Gora Gora?
Five.
PROFESSOR 1: Five.
And Cholera Control?
One for now, maybe two.
Not sure.
PROFESSOR 1: So, for Heat Waves, Hello Waves, SNAP!
And Cholera Control-- actually for all of you, whoever is doing the most speaking, strap this little thing on.
For everybody else, when you speak, make sure you are standing on the normal spot in front of this, so that the microphone can capture you.
If you have-- for volume control, it is over here.
Hopefully you've used this before.
It's set at about midway.
Take a note of all the different settings you are using when you're plugging in your stuff to the projector, so we can make things go fast on Wednesday.
Any questions?
All right take four more minutes in your teams to get yourself ready, and Heat Wave, please come up at 1:15 to plug-in your game
[INAUDIBLE] PROFESSOR 1: No, so time yourself with the clock.
Take a note of your own time.
We're not enforcing time limit right now.
We're concentrating on content
Well, hello, my name is Mary [?
from Providence ?]
and today I'm representing team Heat Wave.
We made a game about heat waves, not surprisingly.
And these are the people on our team.
I'll talk a little bit more about that in a minute.
So, first of all, our game was about heat waves, and the main goal our game was to educate Red Cross workers about what to do when a heat wave was upcoming or was currently in progress.
How do you prepare for it, and also what do you do to deal with it.
And while we focused on these goals, our design goals were more specifically usability, play--
[INAUDIBLE]
OK. How did I get that far without someone pointing that out to me?
[INAUDIBLE]
OK, we're going to start over if that's OK. Do you mind?
I really didn't get that far in, so that's OK. Heat waves-- oh, but now I can't see the slides.
OK. Heat Waves, this is our team.
Heat Waves is a game to talk and explain to Red Cross workers what to do in order to both prepare for heat waves, and also what to do once you're in a heat wave.
And that was the purpose of our game.
And while we had those goals for our game educationally, our goals overall for the design process were playability, usability, and education.
Usability was first, because if the game isn't usable, no one's going to use it.
Same with playability, if you can't play through the game, then it's kind of useless.
And finally, our main goal was education, because we really wanted this game to serve a purpose, which was to educate people.
For our design process, we did the same process that most of the people in this class did.
We started off with brainstorming.
Then we went into team formation, responsibilities breakdown, and then just constantly updating, using Trello and different Scrum features.
I'm going to talk about all of these in a little bit more depth.
So first of all, brainstorming, we started off right on the get-go, drawing on the board.
What do we want this game to look like?
What are going to be the main parts of it?
And we actually settled on a somewhat complicated design, with the main screen a dialogue system, and then kind of an options menu.
And once we had this, we actually formed our team.
This is our team minus-- I don't think Joe's in this picture because he was sick that day.
And one thing you notice is we had a team of nine people.
So it was a lot of people doing a lot of stuff.
So we had to really deal with that.
And one way we dealt with that was using Scrum.
We had a lot of emails.
But the way we managed that is, we learned from previous times that if you have one giant email thread it gets overwhelming, you don't see everything.
So we separated it, had a couple different email threads based on different topics and subgroups.
And then also we updated our Trello.
One thing we realized really quickly is that Trello is not so good for listing every single Scrum.
So you kind of have to have a to-do, a done, and a working on right now, just for future reference.
And that's kind of how we organized ourselves later on.
And then just because it's Scrum, it's an iterative process, which is the picture.
Next, we had a lot of group meetings.
And something we spent not as much time in class because we spent a lot of time outside of class.
We had a bunch of meetings where we had snacks, and we would all just sit around for like three, four hours, and work.
And those were really productive times for us, because it was a time where everyone who was there was really dedicated to fixing bug after bug, issue after issue, getting things done.
And they were really what made our game come together as much as it did.
So, focus test, we ran four focus tests.
And each of our focus tests had a different theme.
So once we had our team together and we were building the game, there were different things working at different times.
So for the first focus test, we had a low-fidelity prototype that with digital, and it was a Python game.
And we've tested concept-- I'll talk a little bit more about that in a second.
The second one we tested our initial unity game, and we looked for over-arching issues.
The third test, we worked on visuals, just how can we emotionally resonate with our audience.
And the fourth test was game balancing, because at the end of the day, once you have everything together, it's really easy to ignore if the game is balanced or not.
So the first test we ran was a concept of general mechanics.
So as I said earlier, our general focus was education.
It was our big goal, and we wanted to make sure the type of game we were building could really convey information and teach the audience about something.
So if any of you played our Python test game, you would remember that you basically had a bunch of people you could choose to talk to.
And if you talked to them, you could offer them water, tell them to go inside.
And they'd either listen to you or ignore you.
And you just went that way in loops until either everyone fainted or everyone went inside.
And what we learned from that was, first of all, it worked.
People were noticing, OK, I'm going to focus on the drunk guy, I'm going to focus on the older person, try to get them water first.
Because they faint more quickly.
And the other thing we noticed right away is something we did try to try to make this game more lively, is we had fun dialogue.
So for example, the first quote up there was like a more factual, drinking water even when you're not too thirsty is important in this type of heat.
That's a very true fact about heat waves.
You're just supposed to drink more water than you need to.
And then the second one is just more fun trying to make it more lively, hide your wife, hide your [?
kill, ?]
it's heat wave.
So that's something we experimented with early-on, having fun or dialogue to kind of increase the excitingness of the game.
So, the second focus task we really were looking for overarching issues, and it was our first unity-based game, so we are working with issues with our dialogue system, issues with how is the playability.
Does this concept of interacting and thinking about education still work in this unity-based game, where it's less of like a, oh, there's just these old person or young person, which one do I choose?
Instead now you have these figures.
And we tested this out a little bit.
We found the educational aspect was still coming through, but we had a lot of playability and usability issues.
So, we needed to majorly revamp that.
So then we got our focus test three, where we focused on the visuals.
So once we went through play test two, we realized, OK, we can fix a lot of this usability stuff.
And now we want to see, can we make this game emotionally resonate with the players.
And make it feel like less of just kind of like, I'm sitting there looking at this red screen and it's like, argh.
So we had this, and what we were looking to talk to people about was the different characters, were they still prioritizing in this scene.
And it turns out they weren't, because those labels didn't move with the people.
And even if the labels had moved, people said the labels were too small to read, they were confusing-- really should just get rid of them, and instead either add little icons or caricatures of different types of people in order to choose who to help.
So those were the kinds of questions we asked.
And we actually showed them a sample artwork of a redone homeless person drawing.
And people said that, that was a lot more informative and that way we didn't have to have like little text that no one could read.
So focus test 3 was really about the visuals, and we really re-did a lot of our visuals based on that.
Sorry, this mouse is a little non-responsive.
Finally, focus test four-- I think you all probably had one of these-- was just game balancing.
The end condition of our game is basically when people faint, how many people have fainted.
And if a certain number of people have fainted, then we stop the game.
And the problems with that was that it was, at first, there was too many people, and everybody was dying right away.
And then we instead of ramped up the number of people, and then there was an issue where, you were fine, you were fine, you were fine, and then suddenly everyone died on the fourth day.
Because there were suddenly 100 people, and you could not help them all.
So focus test four really focused on that, and trying to balance the game out.
Along the way, we made some bad choices.
Every team does.
And the main ones we made was, we decided to depend on external code.
And while we thought it would allow us to make a more complicated game, we didn't really foresee how much we would have to adjust ourselves around the external code.
It's a two-way street.
And so it kind of impeded our progress sometimes.
It added a lot more complexity, things that we weren't expecting.
And then the other thing we made a mistake on, which I'd like to emphasize, is we focused on education, which was a great thing to do, but we forgot to focus on fun.
And one thing we did to compensate for it, as I mentioned, is add that dialogue, that banter, that makes something that's kind of dull a little more interesting.
But even so, I think one thing that if we went back and we did things differently, we'd focus on fun as a number-one priority, along with playability, usability, and education.
So ours became a pretty standard learning game, as we talked about them.
Good choices, we had a lot of group meetings with snacks, great idea.
Turns out people come if you have snacks.
Just if you were wondering.
And then we prioritize.
So everybody prioritized.
But one thing we had to deal with, because we had such a large team, was not everyone was available at the same time.
So it was really prioritizing not only based on what needed to be done but even if something kind of had already been assigned who can actually work on this right now and get it done, and like when are they going to get it done.
So we really worked with that and kept iterating and figuring out, based on what needed to get done right now, who was going to do it.
So the game itself is composed of really three screens but a couple of them have sub-components.
So, the Start screen with an instruction screen; the main game, which has the daytime screen and a newspaper scene in it, which is like the end of each day you see that; and then the end screen, just you can take a look at all of these.
So originally, this was our start screen.
Main feedback with that was, it was not pretty, and it was not informative.
So when we redid it, we added an instruction screen, which is actually still having a little bit of bugs with the readability.
But what we did was we added a way for players who didn't play the game before to find out what they actually needed to do.
And then the actual game, the original, we had-- I showed you on our focus test two and then our focus test three with the people moving, but the labels not moving.
We really iterated the main game to make that as usable as possible.
And our final result-- well this is actually in progress.
So one thing we updated in progress that I almost forgot to mention, was that we saw in focus test two and focus test three that visuals really important.
And one thing we weren't displaying at all that was very educationally important, that Pablo actually mentioned, was that as people are about to faint, they usually get sick.
If you see somebody who's about to faint, they'll like start sweating, and turn red, and all this stuff.
And in our game, they just kind of keeled over and died.
So, we needed a way to implement that.
But we couldn't necessarily like draw a million different character art.
So, we decided on the simple solution was just to add a little-- that little circle, like the thing above their heads, you can all see it with the exclamation point.
And that was something we did in progress.
And finally we integrated that with the new character art, and this is the final game-- I don't know what those squares are, they're not in our game.
But you can see the people are a lot more descriptive.
I think that's pretty obviously that's an old person, they have a walker.
And we had to rely on these caricatures of people, which really worked out in terms of like getting visual feedback when we talked to people.
They were much more responsive to this than like labels or even like little symbols.
They really liked the art.
And finally-- not finally, but the other part of our main game was the newspaper scene.
And initially we just had a lot of information displayed, and no one actually read it, because they didn't think they could do anything about it.
And they also just didn't like looking at it.
It was confusing.
So we re-did that, and so made it a little more visually appealing.
And something we did that I didn't mention on the previous side about the main scene, is that we actually added more longer-term options.
So this was more relevant as you were playing.
Because if it was cooler on a day, you could start installing umbrellas.
And then you would actually be like, OK, well, it's actually 115 today, so I'm not going to install any umbrellas, I'm just going to try to give people water.
And then finally-- so originally at the end of our game, as you can see, there was no end scene.
And that's kind of important.
So, people really wanted feedback as to-- they didn't just want the game to restart, they wanted to know how they did.
Even if it's educational, part of that fun that was something we obviously didn't focus on, was just knowing, oh, I actually did pretty well.
This many people-- like I lasted this many days, or this many people like went inside because of me.
And that's because we weren't focusing on fun, we didn't even think about the importance of an end screen.
Right?
So, when we re-did it, we added this end screen, and it tells you how long you survived.
So, two hours, five days, you can get a sense of how long can I keep it up and make sure everybody was working out well.
So that's our whole game.
And looking back at this whole process, something we've discussed as a team that we would do differently, is passion.
So one thing about heat waves we all thought initially, oh, it's such cool topic, heat waves, it gets hot.
And then we read about it, and it was actually a surprisingly simple phenomenon.
It gets hot, people drink water, go inside, and that's about it.
There's not really a lot of complexity to the issue.
And so a lot of us lost our passion very early on.
And I think if we were going to do this again, we'd somehow incorporate something in addition to the heat waves that would make us more passionate about the game.
The other thing we'd do differently is accountability.
Just making sure that-- with such a large team you have to really keep track of everyone doing everything.
We really focused on prioritizing who was like actually had time to give and making sure they got tasked on-- which worked out really well.
But at the same time, there were people who sometimes were kind of lost and didn't know where the game was going.
Because we weren't always on the same page with such a large team, nine people's a lot.
But it still worked out really well.
And I'd say that given everything, we did pretty well with it.
And if we were going to do more features, something we talked about was more long-term options, people like those.
They made it more fun, because you had more of an optimization problem if you had more long-term choices.
Those were kind of hard to think of, because in heat waves there aren't that many.
Like you can install an AC unit, and then maybe a water fountain.
But we were thinking maybe we can make this a bigger scope game, add multiple environments so you have more long-term options.
And really build up the complexity over time, so this game can be more realistic of a community as a whole instead of a single environment.
And that's really what we wanted to do.
Thank you so much.
Are there any questions?
[APPLAUSE] PROFESSOR 1: Oh, so comments about presentation.
Anybody up there?
Yes, so can you load up your game make sure your game works?
Can somebody come down here from the team and do that?
And you can answer questions while we-- so feedback from us then about the presentation.
The main question I had-- like you covered a whole bunch of stuff, which is really good-- the things we're asking for, what went right, what went wrong, you did in a different format, but you got to it, and that was fine, that was good.
The one thing I would have loved more information on, if you can fit it, is the external code issue.
Specifically, what was the external code, why was it in there, what did you think it was going to help you with?
And then what did end up like-- just a couple lines of those details about
More specific.
PROFESSOR 1: More specific
Yes, definitely.
[INAUDIBLE] PROFESSOR 1: And really what is it giving you that basic unity wasn't giving you
OK, absolutely.
[INAUDIBLE] PROFESSOR 2: Let's see, I like the structure, as [?
Rick ?]
said, you got to everything.
But actually the way how you structured it also works really well for all the points that you intended to hit.
If we are following this particular order of presentation-- PROFESSOR 1: We're not necessarily, we're going to figure out what the order is after we do the all.
PROFESSOR 2: Whoever goes first will probably need to explain who Pablo is to the audience
Oh, sorry-- PROFESSOR 2: No, no, no, it's not something that you would necessarily have expected.
But whoever goes first on Wednesday, if you talk about any of the clients, you have to assume that your audience doesn't know who the clients are.
And different teams had different clients that they were talking to, too.
So do just explain [INAUDIBLE] PROFESSOR 1: We'll introduce-- basically we're making games about the topic, but your specific topic--
If I have to mention Pablo, I should say who it is, or I shouldn't mention him directly.
PROFESSOR 1: Yes.
PROFESSOR 2: Yes, or just give his name and who he is, that will do wonders.
Something for anybody who's using Google Presentations, Google Slides or whatever that thing is called, I think the Ask button brings up your notes.
So if you have notes-- I wasn't sure if you were relying on--
No, I just wanted to be able to see the slides.
That's why [INAUDIBLE] PROFESSOR 2: You also get a little preview slide.
So if you hit the Ask button, you bring that up.
Also it automatically brings up the, "Google Drive is using full screen, click this to make it disappear."
And that's really distracting.
So whoever is using Google drives, make sure you do that.
And you may want to use the keyboard, because the mouse was not--
It was flipping out.
PROFESSOR 2: Yes, it was there was not cooperating
I brought it for the actual game play, but we're not playing the game today.
PROFESSOR 2: OK, so recharge, and if it starts to give you that same problem, for the slides at least, you may want to just avoid using the mouse.
PROFESSOR 1: Actually, putting it in mirror mode, too, instead of two screens, would at least give you a full-screen version of the slides.
The speaker notes give you a very tiny version of the slides.
PROFESSOR 2: Yes, it was very hard to see, actually.
So mirroring your screen back and forth will also make it so that you don't every have to do that
Can you write that down?
PROFESSOR 2: I can email this actually, at least these notes.
At the beginning of your presentation.
If you can just-- when you introduce that your game's about heat waves, before you go into the things that you did, if you can just very briefly talk about what Red Cross workers have to do in heat waves
OK, yeah.
PROFESSOR 2: Yeah, of course, if that's after you demonstrate your game, then that may not be so necessary.
But just to give people an-- just to make sure that before you start talking about this is what your team did, just give people an idea of this is what you're going to do in the game.
PROFESSOR 1: And you all have a choice of demoing the game before the presentation or after.
It's whatever works best for your presentation.
PROFESSOR 2: So if you're showing the game first you may not need to describe this, but it if you're showing the game after your talk then you definitely need to explain that.
When you say, "over-arching issues," as a bullet point, that tells me nothing.
It's like, you had a big issue.
But then later on of course you went into detail and talked-- oh, it was about dialogue, it was about learning, and stuff like that.
I think the first time you introduce a bullet point, say over-arching issues, and you say, such as, the dialogue, bugs, and the learning, very quickly, just so that bullet point means something.
[?
Rick ?]
had already mentioned the thing about external code.
When you talked about how you swapped your art assets, and you said, we talked to people about the art.
Did you the mean play testers, or did you mean your client, or did you mean your team members?
Right?
They're all people, so which people?
Yeah.
I was assuming play testers based on the context.
And finally, this has nothing to do the presentation.
Does the game still use Fahrenheit?
Can we change that?
What was the question?
Fahrenheit?
I don't think we have changed--
The game is still on Fahrenheit
It's still on Fahrenheit

We can change that today
It's a bit risky to change it at this point.
You might want to list that on your things to change in the future maybe, just add [INAUDIBLE]
That's enough
Yeah, that's enough.
I wouldn't touch the code
Don't touch the code.
Please don't touch the code
Yeah.
Just acknowledge that, because that was one of the big pieces of feedback, is that only the US uses Fahrenheit
Yeah.
[INAUDIBLE]?
I'm not following, so he got [INAUDIBLE]
Sorry
No
So make sure it actually plays.
So you're going to demonstrate it in unity rather than as a web hosted version?
We might do a [INAUDIBLE] version, but we haven't had luck yet
OK, cool
We're meeting after class
Great.
But it looks like whatever you're going to run it in, it's going to work at that resolution.
And is there audio for it?
Yeah.
There is.
But it's not playing right now
OK. Oh, there's some
One of the things got lost, apparently
So at least it's not muted
I'm sorry
It looks like heartburn.
It looks like it ate something bad
Thank you so much
Thank you.
[APPLAUSE]
And thank you for being first
Yeah
I'll email you the notes
"Hello Waves," come on down
So it's a little light on there, but our game is "Hello Waves."
It's a game about forecast based financing that we're developing for the Red Cross.
The idea of forecast based financing is using the idea of a forecast about the future in order to make decisions that are more effective than just reacting to disasters when they happen.
So I'd like to show you a play through of our game.
It should open up.
Great.
And let me turn on the sound.
[CLASSICAL MUSIC PLAYING] So this is "Hello Waves."
Here are the instructions.
We would have done a tutorial if we had enough time, but this gets the idea across, and designing a good tutorial that actually teaches the player well is hard to do in a logical flow.
So we have these instructions that explain how to play, what the point of the game is, and a couple of tips about how the game works.
So if we go into the play screen, you see something similar to what we showed you a couple weeks ago, where you have all these different toys out there, castles.
And as you go through the days, you'll see a forecast of what the water level's going to be at over time.
And so you can see that right now.
They're gathering candy.
The water level changed.
This character is underwater, and so he takes damage.
And so what you actually want to do is you want to move toys out of the water so that they don't get damaged.
However, they can only move one castle over at a time.
So because we didn't think ahead well enough, we'll see that this truck will actually take damage on the next term, because he's underwater.
Actually, he ended up not underwater.
We got lucky there.
But theoretically, he would have.
And so as you go through it, the end goal of the game is to build a castle.
And so every toy when they're home can either choose between gathering candy to feed evacuated toys or building the castle to reach your end goal of the game.
And then the idea of the forecast is that you want to use the forecast in order to know how much candy you're going to need over the next couple of days and to use it to know when you're going to need to evaporate various toys from their variously hidden homes.
So back to the presentation.
Full screen.
So we had a couple of challenges in the design process.
The first and the major one is really that we were trying to teach about forecast based financing, which was a bit of an abstract topic.
It's a little different than just thinking long term, because you have to use the idea of there's some information we know about the future that we want to use to make optimal decisions, or at least decisions based on some idea of the risk that's out there.
But we also wanted to avoid things like just pushing a button to win, where you have all the information you ever need, and there's just one option that you know you're going to pick every time.
And the game has no thought whatsoever.
And we also needed to think about how we were going to communicate the forecast to the player so that they could then use that to make decisions.
One of the problems that we also ran into related to this was that we focused a lot on the idea forecast based financing as the topic, and then tried to build a game built on top of that topic instead of building a game that used forecast based financing.
So that held us back a lot in the beginning when we were trying to come up with ideas.
We also had a really difficult initial target audience of policymakers-- 50, 60-year-old government officials, or people at NGOs who were going to be using forecast to make decisions of some sort.
And it was supposed to teach them about how they can use forecast to make these decisions, except this was a really difficult audience, because they don't generally play games.
And they don't have a lot of time to learn about this kind of thing.
And so we couldn't really expect to get them to sit down for a long period of time and play around with our game.
In order to deal with these problems, we came up with a couple solutions.
The first thing was that we had no idea what kind of game would work, or would make sense, or anything like that.
So what we did is we just broke our team up into multiple groups and came up with a bunch of different prototypes.
On paper, we had two different prototypes.
One that focused on the idea of managing a city, and its resources, and its response to disasters, versus focusing on individual people, and how you're going to move them around to keep them safe from the disasters.
And then we went into a whole bunch of different digital prototypes-- one that was a text based game about managing a city.
We had one that looked like this, where you had two different cities, and you were specifying what workers were going to do, or when they would leave the city in order to stay safe.
And then we would take the different ideas that we were learning from both of these prototypes, combine them together, take things out.
And our final prototype actually uses ideas from all of these.
Another solution that we got lucky with is when Pablo came to play our game, he told us that we actually shouldn't focus on the policymakers, because he wasn't confident that he could actually get them to play the game.
And so we switched our target to being grade schoolers, which is why you saw the cutesy art with the beach and the toys.
This actually made it a lot easier for us, because we could target people who probably had some experience with games, or at least wanted to do something fun, and would be curious to learn about our topic
So for our development process-- or I can just speak here, right?
Yeah.
You can speak right from there.
That's fine
So another big issue, another set of challenges that we ran into was through our development process.
And so we had initially issues with communication and facilitation.
Our team had a wide variety of experiences and backgrounds.
Some were hardcore gamers, some mostly mobile gamers.
And so there were initially a lot of disagreements on what level of game we wanted to create, and what sort of game, casual versus hardcore that we wanted to create.
And so we needed to overcome challenges of facilitation and communication within our team.
Another major challenge in our development process that we had to face came from our design issues, where for a very long time, we had a vague vision of what to do.
We didn't know what kind of game we wanted to make, and so we purposely tried to keep our game ideas vague as we built prototypes.
But then we ran into issues where we would have solutions.
But no consensus on which solution was best, and where we went for long periods of time without having a clear direction of where we wanted our final game to be.
So our solutions for the challenges proposed by development were a team structure.
And so we structured our team loosely into three subteams-- a production subteam, which would take care of production, like the deliverables, and making sure that all the game ideas are being communicated properly.
A technical team, which worked primarily with the code and making sure the game got done, and then a user experience team, which handled art, UI, and sound.
And so we kept our responsibilities flexible.
So as team members got busy over the semester, or as changing conditions, lots of different people contributing, responsibilities were able to easily flow between teams and team members.
And additionally, another idea we started with in the beginning with the idea of sub-team leaders.
[INAUDIBLE] the two people marked with the l's.
But that was an idea we later abandoned in favor of just having a more flexible team structure, or a flat team structure.
Another solution for helping our development process was the use of good code practices.
And I cannot emphasize enough that this really helped speed up our development, because we didn't run into trouble with code.
It was mostly with design.
So we used Yeoman, which is a JavaScript module system.
And basically, it allowed our code to be interoperable.
We can write one module separately from another module.
So that solved a lot of issues with dependencies, or people working in parallel, because it allowed people to work in parallel without overriding each other's code.
We also used good code practice with state machines and NBC, which is motor vehicle control.
And so we had a very object oriented code, very modular.
And when we did need to change our code, rip it all out and put it back in, it actually didn't turn out to be too painful, because we just had to switch a couple objects around.
And then one final solution that we used for our development processes were Slack and Scrum.
So Slack is like a modernized IRC chat room, and it's very feature rich.
It has a lot of integrations with GitHub, and Google Drive, and things like that.
And so that real time communication actually made it so that we didn't really have to meet outside of class too often.
If someone was working, we would just email out saying, I'm on the Slack.
And then people could meet on the slack.
And it was full featured enough.
We could send attachments and things like that.
Most of our communication in person communication could be done in class.
And in class, we adopted a daily Scrum format, where we simply said what we had done since the previous class and what our goals were until next class.
So in the end though, we did have to cut some features.
These features mainly were the idea of again, a tutorial, or multiple levels, simply because they'd just be too much content that we need to play test in order to make sure that it was a consistent quality got our message across, and also, trying to add more individuality to the toys that you saw.
They have different graphics, but that's as much as we could do, given the time constraints.
So kindly just bring it back.
And our three worst decisions were first, we did end up spending a lot of time on code and assets that never got used.
We maybe used 10% to 20% of our final work in our final project.
Actually, maybe that is a bit overkill.
But maybe 30% of our final code and assets in our final project.
This really was due to this second bad decision, that we kept the game and its direction too vague for too long.
We always were holding out that oh, maybe we'll be able to come up with a better game idea.
Or maybe we'll find some magic solution to how we can make forecast based financing into a game.
And because of this mindset, we'd spent probably the first half of the project just staying too vague, and that hurt us in the end, because we spent so much time going in all these different directions.
And really, the decision that captured those first two though is the fact that we tried to make a game on top of forecast based financing.
So we had forecast based financing.
And we were like, how can we scan this as a game?
Whereas once we switched that mindset and thought, let's have a game, and how can we put forecast based financing into it?
I think that was the moment that we then came together as a team and really started making the final game that we wanted.
So our best decision
So one of our best decisions was that we chose good tools at the beginning, which meant that as we went through all these different prototypes, we actually didn't have to completely rewrite our game.
We could pull out the way that workers worked.
In one game, we could pull out the view that we were using in another game.
And then we could just combine them together, and that allowed us to move quickly whenever we were changing our prototype.
We also weren't afraid to trust each other, both, in terms of what everyone was working on, but also, in terms of the decisions that we were making.
And so when we said that we had to throw something out, we all understood that it was for the good of the project.
And we didn't have a lot of complaining or hurt feelings when something didn't get put in, or when we decided to throw out our assets or code.
And when we got to the end, we had been through enough of this vagueness that we were all frustrated with it.
And we realized that we had an idea that we all liked, and we really got on board with it and made it happen.
Once we started working on our final idea, we saw that it worked.
Basically, every decision from that point on was how do we make this game better?
And we were all on board with that vision.
Thank you.
Any questions?
[APPLAUSE] [?
Anyone in the ?]
audience have [INAUDIBLE] first before--
We can't see [INAUDIBLE] across the [?
room.
?]
Yeah.
We'll have to make all of our slides darker
Yeah.
Yellow and white
Yeah.
Like the yellow and white on your early screen, I was doing this on [INAUDIBLE].
I realize that's the game, not the presentation.
But wow
It's also a little faded in the game itself.
It could be the display resolution you're using, so after class, after we're done with everything, try different resolutions.
See if it makes a difference
Yeah.
That's be great
The music in the game development was too high.
It was overpowering your voice.
So I think you were almost hollering just to be able to be heard.
It doesn't need to be that loud.
So you can just crank down the volume right down on the computer or on the controls over there.
Something that I had a question for.
You don't have to answer it today.
But you might want to put it in your presentation.
How long was the design of what you eventually established on the table before you decided that this was the thing you were going for?
Because I'm assuming it was one of your vague-- it came up during your vague idea phase.
And you're implying that you were in that mode for too long.
But I don't know how long it was on the table, or was it only something you figured out at the end of the vague idea phase?
Because otherwise, you wouldn't have been able to switch to this idea if it was not on the table in the first place.
And how did you decide this was it, that the same household game was going to be-- I understand-- I think you very clearly explained the benefits of deciding this was it.
But how did you come to the conclusion that it was?
That's stuff I wanted to hear more about.
But you don't have to answer it now
Yeah.
And just a little bit of specificity.
You mentioned you spent too long on that design phase.
I wonder how long that was

You said Pablo switched your target audience for you, telling us whether that was because of the game that he saw, or because of something else.
If you know that information, throw it in there.
If you don't know that information, that's fine.
Knowing why you dropped the team lead-- you mentioned you dropped it, but you didn't exactly say why again.
Really quick, we weren't getting [?
blah ?]
out of it, or the flexibility was more, whatever.
And defining your terms.
You were saying things like casual versus hardcore.
Give a little bit a definition of what you mean by that.
It means different things for everybody.
So what is your use of that?
I think he specifically said hardcore and mobile.
But there are hardcore mobile players out there
And casual can be considered a version of hardcore, just in a different way.
So just be a little more clear, because I think you were talking about the target audience, and the kind of players, and the kind of games they might play.
So just be a little bit more focused on what exactly you mean by that

That's it for me
So we can start testing for observations in the young man who's not relaying [INAUDIBLE] module frame [INAUDIBLE].
It's just a system to generate the project, and it posts in the front [INAUDIBLE].
So again, just show that

So that's a terminology issue then
Yeah
Clearly, it worked out for your team, so we're not saying, don't mention Yeoman.
It was clearly a good thing.
But just check your definition of what it is, because it'll be more helpful for other people to understand what it is so that they can think about whether the want to use it in the future
So your demo-- you spent about three minutes doing it.
We are going to have a player from [INAUDIBLE] play your game live, without getting a lot of help.
You can talk over it.
You can, after a while, start helping them.
We want to see them at least just start playing it on their own.
Decide when you're going to put that demo in.
You could actually combine them both together if you do it at the beginning or end.
But if you do that, give the player a little bit of time before you start talking
OK. Sure
That's all I got
Great.
Thanks.
[APPLAUSE]
Snap, come on down
Hi everyone.
I'm part of the Snap team, and this Sabrina, who will talk about the front end aspects.
I'll talk about the back end aspects.
So I think we actually wanted to start with a demo of the game, and we'll do a real demo right now.
So could everybody go to snapgame.org?
You should turn on your volume, as well.
It'll be interesting to see [INAUDIBLE]
Oh, nice.
My screen's hidden.
And I will also play the game.
Everybody ready who wants to play?
So the way this game works is I'll explain it again, in case anybody hasn't heard this before.
We're going to give you a topic, and then you're going to enter words related to that topic.
Whenever you enter the same word that somebody else entered, you'll get a point, which we call a snap.
And the topic is algorithms.
And you can cheat by looking at my screen, but please don't.
I have no idea how I'm doing
Oh, no.
What happened?
What happened?
Oh.
Oops
We found a UI bug
I think it's this resolution.
Probably.
I did test this.
I'm sorry.
I never entered this many words.
I can't scroll.
I think we're going to stop this one.
Wow.
That was impressive.
Justin won, followed by Rodrigo, and Rachel.
[APPLAUSE] This is what you guys submitted.
[LAUGHTER] This is a very MIT heavy.
Domain
Someone put [?
socks.
?]
So that's our game.
So the way we did this project was we split up into back end and front end teams from the very beginning.
This was a bit of a challenge for new features, because initially, we both were really highly interdependent.
But eventually, it became really nice, because it was simpler to manage a front end and a back end team individually, and they generally didn't need to talk to each other once the back end was off the ground.
I think that structuring our team with more individual roles, besides just having generally back end and front end people would have been helpful.
UI design was something that we didn't think about early on enough, and that's why the UI is not as polished as it could have been.
We could have had somebody just dedicated to taking information between the back end and front end, say, documenting the back end and making sure everybody knew how it worked.
We also definitely had problems with paperwork, which would have been nice to have somebody who was really on top of that.
And then the game design, as well, was something.
In addition to UI design, we also could have had somebody just looking at the game and thinking about the core game concepts, things like scoring and what we're going to display to players.
We did a pretty good job with planning.
We used Asana and Asana worked really well for us.
It was quite flexible, let us manage independent task lists for front end and back end, which we think worked well all the way from the beginning to the end.
So separating out the task lists we liked, Asana we liked.
Our project had a lot of dependencies, and this was something that Scrum inherently had a problem with, and our tools also had a problem with.
It was hard to know, for example, when a task on the front end and back end were dependent.
There wasn't a good way to encode that and see that the back end needed to get something done more quickly, because the front end could use it.
We also had trouble just creating tasks.
I think that there was a diffusion of responsibility as to who would be creating tasks and who would be assigning them.
Once the task got assigned, then it went pretty well.
But we had trouble just getting all our tasks listed out.
I think the core of all these problems was really getting people invested in the project and making sure that they treated it as a priority and really spent some effort on it.
Want to talk about this?
Sure.
So we also used Slack for communicating as a team.
And I think it was really helpful.
I liked the fact that we could have multiple channels so that people working mainly on the front end didn't have to get all the communication going on with the back end.
And then we also had channels for our Asana and GitHub so we could see what was going on with that.
Some problems though where I feel like people weren't as active as they could have been-- I feel like maybe everyone should have downloaded the mobile app, or at least signed up for notifications so that everyone's aware constantly of what was going on.
And if someone mentioned them in a post.
They were quickly able to respond and get back to them.
And that's the main thing that I think we could improve upon if we use this in the future
So going a little bit specifically into the back end, what went right for us was definitely our technology.
We used node.js as the server and socket.io to do the two way communication so that you could receive snaps from other players.
And we deployed on heroku.
All those three things worked really well.
We had no problems with them.
And the design of the back end also went pretty well.
It's a pretty simple server to implement, and we did a good job of figuring out what it needed to support.
We didn't document as well as we could have, in terms of what is the published API that the front end should be using.
So in socket.io, you send all these events, and the names and the formats of the events were not well documented for a long time.
There wasn't a lot that I would change on how the back end was structured and how it went.
But I think maybe we had too many people working on it for the small amount of work that it was, and it would have helped if people more people had worked on the front end.
That's also part of finalizing and freezing.
If we had just finalized and frozen the back end, then we could've stopped worrying about it and just moved on.
Networking was explicitly forbidden in the rules of the class, which we did break, and we intentionally broke it quite early on.
Networking really didn't give us very many problems.
We had a couple heroku issues, where we had to quickly do a deploy, or there was a bug, such as the one you just saw.
But really, heroku went really well for us, and you could easily roll back.
And heroku gives you good documentation on what code exactly ran every single time.
We actually have been deploying to the client.
The client has been using this for a while now.
These are some of the tests they've run.
And you can see that they're quite different, in terms of how many teams there were, how long they went.
I'm not really sure what the really long games were like.
We still need to talk to Pablo about his feedback for those.
But indeed, there were words being submitted for those entire three hours.
And just to give you a sense, I'd actually like to show one of these word clouds.
So these are much more serious topics than algorithms, where actually, looking at the words will give you a sense of what people are thinking about a topic that you care about.
So for example, I think I'll just include screenshots of those, because that doesn't work very well.
So here, the topic of this conference is zero poverty, zero emissions within a generation.
And you can see, education seems to be a big theme in this conference.
I can tell that just by looking at the word clouds, of several of them, actually.
And a lot of our effort ended up being focused on the client, as opposed to the class.
I think that's part of what made our paperwork fall behind, is that I've been working a lot on supporting these runs that happened in the past four days, as opposed to the deliverables of the class, because I thought that this was important.
It's part of why we have snapgame.org
Developing the back end, we were really successful.
But I felt when it came to the front end, we struggled a little bit.
I think a main challenge was the game is game that was intentionally made to play person to person, and have that communication.
And representing that in a digital game was a big challenge that we had to do.
So in our first design, we decided we were going to use Phaser, and it looked something like this.
And obviously, Phaser wasn't even really being used, so what was the point?
So we dropped Phaser, and then we went to just using Bootstrap.
Here, we're just using Bootstrap in JavaScript to handle all the input from the users.
And then in our final design with as you saw, the dots and the drawing, we're using a JavaScript library called 2JS.
So what went right was I think we had really good people to do UI design.
But we didn't really begin thinking about that until later.
And it took us a really long time to be able to represent the game in a really good way to the user.
There were a lot of challenges we faced in doing that.
So I will come back to this.
Some of the first ideas that we had for how we're going to represent the information to the player was by using a word cloud that we used on the end, but blurring out the words until they actually submitted a word.
Then we would display it.
But this is actually really, really challenging, and so we didn't want to spend our time doing something like that.
Then we considered having some sort of line where you're racing against the other players to get to the top.
But this isn't entirely interesting, so we considered including both of them.
But then again, the word cloud, we ended up deciding that was too difficult.
So we came up with the idea of everyone's a dot, and your score would be represented by how big your dot is.
But if you're playing with 100 people, this is just unreasonable.
And so then we finally came up with this last idea, which is nearly complete, where the height of your dot represents where you are in relation to the max score.
And the lines are representing snap connections that are going along with the other players.
So basically, I think for future projects for the front end, we definitely need to think about design sooner.
This process happened really slowly and really close to the end.
We were very focused on the functionality of our game and the mechanics, and not a lot about the player's interaction with the game.
It would have been awesome if we did some prototyping early on, because we had some other ideas.
And we just never put them into fruition, and we never tested them out.
So it would have been interesting to make those and see what users thought of them
So I think working with a large group was definitely difficult.
And splitting it up was a good idea.
But I think that having more responsibility and roles within those people would have been very helpful.
We also should have thought about design sooner.
I think also, from Miriam's presentation, the idea that we should have thought about the game and making it fun was something we should have focused on, as opposed to just thinking about porting a game that we had seen.
We really should have focused on what makes this a fun game, and what could we do to improve it?
Because the core game can be done very quickly, and within the first couple weeks, we had the core concept done.
So I think more quickly, we could have figured out, what feedback should we give to players?
How can we make it more competitive, which I think it makes it more interesting as a game, as opposed to as a tool to collect data?
That's all we have.
Are there any questions?
[APPLAUSE]
Anybody else from the class?
Any suggestions?
No?
Well, we've got some.
You want to go first?
So one thing that I noticed was there was a screen where you had the [INAUDIBLE] didn't future.
That made no sense to me at all.
I stared at that slide for a while.
And then I heard what you were talking about, and then ah--what worked, what didn't, what are we going to do in the future.
I can see how you might do it, because it fits with the snap theme of having just a single word.
But wow, that was really confusing
Oh, that's why all [INAUDIBLE] are in small letters
Yeah.
It took me a while to realize what was going on.
I'm guessing that.
I don't know for sure
Is that what you were going for?
Not really.
It was just for consistency

Slightly more clarity would be helpful.
You talked about the task diffusion, having a hard time getting tasks assigned.
But when you talked about dividing your team into groups, you didn't talk about what the structure was running those groups or anything.
You didn't have any of that was what was going on.
But you might want to be maybe more explicit about that
Yeah, like how did somebody take a task from Asana, by the way
Yeah
Was there a process for that?
Did that actually happen that way?
Or was it an assignment?
Yeah.
Was there one person generating tasks?
Or did the groups generate tasks collectively?
That sort of thing.
I think that's most of what I have
So first of all, the demo for the game probably is what we're going to have to do on the real day itself, because it's such a large scale game.
So it's not going to be like two people playing the game.
So we can probably run the demo pretty much the way that you did
[INAUDIBLE] that's what we were [INAUDIBLE]
Is that the way that they do it when they run it?
It depends.
So Pablo has been actually doing a limited number of words.
And only at the World Bank are they doing it like the way we are doing it, where everybody's submitting.
And that, they haven't done yet.
That's actually this afternoon
You might want to talk a little bit about the history of this game.
Everyone in this room right now has played the original game
Absolutely
Yeah.
Similar to some of the things.
I have it-- why are all your subheadings in small letters?
Because it really, really threw me.
Now, maybe it was unintended to make it look like your game [?
snap, ?]
but it actually does make it hard to read, your slides.
All caps would be even easier than all small letters.
There's one slide where you talk about adding roles for specific game design, UI, and everything.
But the heading just says, future.
The big takeaway is more specific roles.
You'll see it.
So that should be on your slide.
Your slides move really fast on average.
And some of your slides are more word-heavy than others

So for the ones that are word heavy, them up a little longer, or make them less wordy.
One or the other.
Otherwise, I'm trying to read everything, and I can't even listen to what you're saying before the next slide changes.
When you described the mobile app, I'm assuming you meant an Asana mobile app?
The slack mobile app, yeah
The slack mobile app, OK. Just be specific, because it wasn't entirely clear what you meant by that.
So yes, put screenshots into your presentation instead of switching windows.
That will just make things easier.
There are maybe people in the audience who actually haven't used Phaser or know what Phaser is.
So when you say, Phaser clearly isn't doing anything useful, well, they don't know what Phaser does.
So just explain.
Phaser's a sprite-based game engine.
We have no sprites, something like that.
Two points about the game itself.
First of all, how do you tell which dart is yours?
It's the green one
OK.
I think because of the peripheral vision, I really did not even notice there was a green dot
Yeah
So maybe it's just my eyes or something
I didn't see it either
OK. How many people actually noticed a green dot?
[INAUDIBLE] mine was so small
There was a random green dot in the screen, and I couldn't do anything to it, so I kept [INAUDIBLE]
There is a green dot on screen, and it's really hard to see a green dot.
There's something about the way how the human vision works on that
I think it's also because there are so many people
Yeah
We've never tested with more than five people, and there are a lot of dots on screen when I don't know how many people were in that game--
I think it's cool that there is a dot that is yours, and that it has a meaning to it.
But just being aware that that dot even exists and is different from the dots is more challenging than you might have realized
Yeah
And then finally, one thing about the networking thing.
it's actually really, really neat that [?
Heroku ?]
and [INAUDIBLE] seems to have solved problems that we traditionally, in this class, have just derailed entire teams.
So it's a possibility that you just have an awesome team
Yeah.
I think one thing I forgot to mention is that we have a lot of experience in all of these technologies.
And so that helped
What happens if somebody who doesn't know anything about node.js or hiroku decides, they're going to make a networking game?
I have no idea
So you might want to talk about that
Mine, to reiterate-- client support versus class deliverables was a good point.
Speak more about the client, and what you did for them, and what their role was on the team.
It sounds like they had a more active role.
And dependencies-- when you talk about dependencies and scrum, having difficulty with that, absolutely true.
What did you do to get past it?
Explain that.
Or if you didn't do anything to get past it, say that.
But say whatever it is, rather than-- you do a lot of [INAUDIBLE] does this.
But you don't go into the how.
So go a little bit into the how for at least that point.
Thank you.
[APPLAUSE]
Would you need a break?
Or are we good on time?
No, let's just hammer through them
All right.
"Saving Gora Gora?"
"Saving Gora Gora," come on down
So we're the "Saving Gora Gora" team, us plus some devs in the audience.
And I'm Liz, and I'll be starting you guys off.
So quick overview of our game's goals coming into the project.
We wanted to inform players about a cholera, and specifically, the causes, and the systems, and the prevention of the disease.
And our audience, we wanted to narrow that down really early.
Specifically, we were more interested in targeting the younger age range of 8 to 13.
And we had specific issues of that, which we'll talk about later on.
And mainly, we wanted to have all these education goals and audience goals; be a part of a fun game, basically, like we wanted our game to be fun, and not one of those educational games they play, and then you fall asleep, and it's basically a poster.
A quick overview of what the game ended up looking like.
This is "Saving Gora Gora."
It's a children's game, and you are visiting a village of animals which is falling ill. And as a player, your goal is to help figure out what is causing the illness and trying to defend the character in the game, which we'll talk about now.
A big part of our game is narrative, which is why we're going to do the demo later, because we wanted to ease you into the narrative without having you read all the text.
Everyone in Gora Gora is sick.
We have this monster that is being blamed for it.
He is totally a cool guy though, and he wants you to help you prove his innocence.
You're Kojo.
You're a little bunny kid, and you need to work with the town to prove to the town that it's not Sal that's making everybody sick.
It's actually cholera.
Basic gameplay is you're the main character.
This is Kojo.
And you'll unlock minigames by exploring the map.
Each minigame will give you a clue to prove that Sal is innocent, and your ultimate goal is to convince the mayor that Sal is a cool guy, and he shouldn't be in that cage that you see in the background
So our first minigame was about water filtration.
And the basis of the minigame is that friends and neighbors in the village are bringing water to Kojo's mom, because they don't know whether it's safe to drink or not.
So the goal of the game is to have the player decide whether each water source is safe to drink from.
And along the process of developing that minigame, we had to actually iterate over the physical design of the minigame quite a bit, because our first idea was since it's water filtration, we wanted to make it perhaps a facility.
But we realized that wasn't relatable.
It wasn't culturally relevant to all the kids who would be playing this game, so then our final design ended up being outside a home, as you can see in the background.
So a couple other challenges we had.
First, we had it working in real time, where the player had to rush to get things done in the minigame.
But we thought that that took away from the overall purpose of teaching the kids, so we made it a lot more relaxed, a lot less time constrained, and allowed player to play at their own pace.
And lastly, we wanted to make sure that the game was clear.
The game plan was clear.
But we didn't want to make it too text heavy.
And we considered making a tutorial, but we didn't have enough time to actually implement that, so we added some instruction text boxes on an overlay to just try to get the player a brief overview.
And then during the game, they're also able to go back into these instructions and reread it so it becomes more clear
So the next minigame is a water collection game, where Kojo, the main player, needs to help his friend find a safe water source, since you have to go look for water and figure out where to drink, or where to get it.
He basically helps his friend choose where to get the water from.
So they basically show you a village map, and you would choose which places on the village map you would want to get the water from.
In terms of challenges and how I started developing the game, we wanted to make sure it was "fail-safe" so that even if the player chose the wrong choice, then they would get feedback.
They would learn why that wasn't the right choice, and then be able to continue playing the game until they made the right choice.
Originally, we did have an excreter.
You had to find where you would also excrete.
But we cut that feature pretty early on, since we thought it was less crucial, and we wanted to focus on water.
And then other challenges were, as Kevin said, we had some cultural issues, in terms of design, so we revamped our images.
And also, we wanted to provide more feedback to the player as they were playing.
So throughout our development process, we added things like helper texts and more explanations about why you did or didn't do things correctly
Our third minigame is focused around understanding the symptoms of cholera and understanding how important it is to act early on these symptoms.
So in the game, the doctor of the village, Dr. Quasi, is very busy, and he can't go out into the village to help patients, because he has so many patients that are coming to him already.
So he asks Kojo to go help him find sick patients to go talk to his friends and see if they have any of these symptoms, tell them to come to the clinic so that they can get treated.
During the development of this game, we initially started off with the idea that you'd be helping the doctor diagnose patients at the hospital.
But then we realized that this would quickly turn into a quiz and wouldn't be as engaging or as fun.
And it would essentially be the same as reading off of a pamphlet.
So we wanted to add more of a game aspect to it, so we decided to make it such that you are going out and trying to find people, and suggesting to them that they go to the doctor's office.
And then another one of the cultural things that we ran into is initially, you're playing as a kid.
But we initially had it such that you're going around and talking to some of the adults.
And as Pablo pointed out to us, there's a much stricter social hierarchy in Ghana, and it might seem appropriate that we are suggesting to kids that they demand that their parents go to the doctor's office, because that's not something that would necessarily happen in the Ghana social structure.
So we decided to change that around to you're talking to your friends, who are the same age as you.
And you're talking to your friends, and you're having them go to the doctor's office.
Some of the challenges we ran into, since this is a very conversation based game, making sure that the dialogue UI was understandable, and that you knew when you had choices available to you, to choose, and when you were talking, versus the other person was talking.
This was something that we had to integrate over multiple times.
And finally, making sure that the dialogue was age appropriate.
Since these kids don't speak English as their primary language and they're children, we wanted to make sure that the dialogue was simple enough for the children to understand-- but also, still had the meaning behind it so that they realized what actions they were taking through their dialogue.
And now, we're going to start our demonstration
So it starts out with the monster hiding.
And the monster is hiding from Kojo.
And Kojo's like, why are you hiding?
And so you see, people are blaming him for everyone's sickness.
And so you say you'll help and you know who to talk to.
But then when you bring him to the mayor, he's locked up by the mayor.
And you're trying to convince the mayor that he's not the actual monster.
So you can see the beginning dialogue really sets up the scene for the story.
And now, as Kojo, I'll say, I don't know, but I'll figure it out.
And so now that you're in the town, you can explore the town.
And you might be able to talk to some of the people.
And so the people around town will give you things that will help you.
So this dog is giving you matches.
And you might figure out later, oh.
And now, you see of matches in your inventory, and you might be able to use that later.
So for example, we can also click on the different buildings.
And now, we realize that your friend is missing a bucket.
So you tell them you'll go look for it.
And we see that oh, the monkey has a bucket.
So then you can get the bucket from the monkey.
And then now, you can go back and tell your friend, oh.
You found the bucket.
And so this brings you to one of the minigames, the collection minigame.
And you can see, you can drag the bowl to different locations.
And you can also click on the locations to see what they are.
And you might say, I'm done.
But it looks like this water spot is rusty.
So that's where the fail safe part comes in.
And if you eventually drag it to a good source of water, it'll say, thanks.
And you've acquired a clue.
So then the next minigame, as Kevin was talking about, is you have to purify water.
And so as you see, there's a lot of help or text to help you get started.
And you can just drag the water.
It says it's discolored, so I think we'll have to boil it.
And eventually, the game, you have to figure out what to do with all the water and collect enough water.
And so I think we'll just cut that short, since we're a little short on time, and then we'll show the rest of the game for the final presentation
So in terms of like too technical challenges we ran into first is like the states and stays there, obviously, there are six of these are and we have to find a way to work around keeping a lot of paired data, in terms of clues and which games you had completed, what dialogues you were on, which dialogue should show up.
So that was something-- that was a little bit of a challenge that we figured out.
And then additionally, balancing difficulty with accessibility.
So we mitigated that by using things like helper texts, and using a hand cursor when you're allowed to click on things-- stuff like that
So one of the other biggest problems that we had, or I would say, the biggest problem we had is that our target audience and [INAUDIBLE] are completely two separate groups.
We're targetting 8 to 13-year-olds in Ghana.
We're college students in the United States.
There's this complete different culture.
And so we had to do a lot of research to make sure that our game was connecting to this audience.
I think we did a good job.
I think we could do a better job.
We don't know.
We didn't get to play-test with our target audience.
So unfortunately, we aren't 100% sure that this game's going to connect to these people.
And another challenge is playing meaningful-- again, just because [INAUDIBLE].
So things that went wrong.
Our communication was really good, but we never talked about deadlines for things.
We would say, oh, we will have this done by the end of the week.
But then somebody would need [INAUDIBLE] assets, and it wouldn't be done.
Sorry.
Or we would want to play test the game at a certain date, and there wasn't a running game at the time.
So I think one of the things that we had a problem with, just making deadlines, making sure people were doing things on a timely manner.
And then we were having good communication, just not constant communication.
We were using several tools, but unfortunately, I think that's the next slide.
We were using Trello, meeting weekly.
We did use Slack.
Unfortunately, not everyone was on Slack.
Not everyone was using Trello, and not everyone was able to go to the meetings, and so sometimes, we would communicate with three fourths of the team, but there would be people that were completely out of the loop.
So things that went right.
We did separate our team into basically two different sub-teams.
There were four people coding, and then me and Liz were doing the art.
She was doing character art.
I was doing background art and all of the other assets.
Then we'd have someone doing the [INAUDIBLE].
And so just by separating the team, it made things a little easier.
You didn't have eight people touching code and breaking things or five different people making the art for something and having completely different styles.
We had a meeting every week, pretty much.
Not everyone was always there, but it was very useful to just get up to date on what everyone was doing.
And then something that we did early was just cut a few features.
We cut a fourth mini game that was picture matchmaking, which didn't really make sense, so we just completely cut it.
Just cutting things like choosing where to dispose of your waste, something that we did early on that helped us.
So then moving forward, things that we would want to do is just test the game more.
We would have loved to find someone familiar with our target audience or to play test with our target audience just to make the game better.
Then changing things like difficulty and making sure the game is culturally relevant.
I know I would want to play test more and then change some of the elements that I don't think are too great right now.
And then even going further, just expanding the game and making it more interesting, having more mini games and having more, an interesting mystery, rather than little paragraphs.
That's all.
[APPLAUSE]
Thanks.
Any questions?
Comments?
I didn't think so.
OK.
The demo that you did mid-presentation, you might want to rethink how you do that.
Again, you're going to have somebody playing the game anyway.
Maybe you could put some of that description while they're playing the game.
Maybe you just demonstrate the map and say, here's a mini game, and then you'll see these mini games in play.
Or if what you're trying to do is help us understand what the games are, just give us a screenshot.
That's probably enough.
I don't think we needed it when we were watching the presentation to understand what you were talking about.
I think you did a pretty good job of that when you were talking about individual things.
And then my big question, what I'd like to know more, is you mentioned the big thing as being the research you did for cultural appropriateness, what was it?
Tell us what you did in the presentation.
Maybe even give it its own slide.
I think it's pretty important
The social hierarchy example was a very good, specific example.
We need more of those.
So what changes you actually had to make once you realized that was a problem.
You've got it in this umbrella category of cultural issues, but the specific stuff is really cool
That's it for me
Sara?
No.
I think Rick actually got most of what I was--
You might want to move the Tools slide before What Went Wrong because all the way things worked out, you just covered that first.
The water filtration picture, where you say you can see it in the background, no.
We can't see it the background.
The background for that particular slide was totally invisible on this projector, so increase the contrast on that image or actually put it in as an inset.
You two need to warm up before you present.
Your voices get louder as you speak, so practice outside of something.
Warm it up and you're more audible.
This is more of a game design thing.
There is a bucket right next to the well while you are trying to find a bucket, and every player is going to say, why isn't that bucket just good enough?
Stick a hole in the bucket or something, whatever is the easiest way to solve that problem.
That's pretty much all I had
OK.
Thank you.
[APPLAUSE] All right.
Last up, Cholera Control.
STUDENT 2: Hi, everybody.
We're Cholera Control.
We're going to make this really simple.
Our game was about cholera and it is a strategy game.
You have to sort of manage many villages and try to tell them what they should implement in order to help prevent cholera within their village.
And so we're going to just go through what went right, what went wrong, what we learned, and any future changes we would make.
So firstly, what went right.
What happened over the course of the class is we simplified the game, and that was one of the best decisions we made that was really hard to do.
Adjusting scope is really challenging because there are a lot of things you want to have in your game, you want to happen, but when you realize you just can't do it, it's like a load off your shoulders and you can really focus on the things that are important.
One of the things we did simplify is what the player can do.
They have less actions, and we'll show that when we have some screenshots, but they're more meaningful.
What the player is doing has a bigger effect on the game, and so they don't feel like they're just clicking buttons.
They're actually thinking.
They're strategizing rather than just, oh, I need to click these as fast as I can.
And it's also more fun for the player.
There's not too many.
There's just enough.
The iteration was really one of the other things that went really right.
It was the best experience in the class as a whole for changing the game is when you do that iteration, it feels really good when you look back and you see, wow, look how far our game has come.
Look at the changes we've made.
And comparing those really helps you realize the choices we made are good.
Some of them are bad, but most of them are good.
And we also had design meetings that were occasionally a struggle for everyone to attend, but when they did, they were really productive and really helped the group know what was going on, know what had to be done.
And lastly, the huge design change we did halfway through the eight weeks, this is what it originally looked like when we had a full digital prototype.
It was really confusing.
There are four villages.
They had little bars over them.
People didn't know what that meant.
There's text in the top corner of the screen that you can't read and you couldn't even read when it was screen that told you your money and the population of each village, and then those two buttons that people weren't really sure were buttons, so that was really difficult.
This is what it looks like now.
It's much clearer.
There's a nice bar at the top telling you your money, how many days, and a nice little pause menu, and then there are only four villages.
They take up the whole screen.
It's very obvious that you want to click those.
And also, there's a lot of indicators as to what's going on in each one.
Before, you had to click on each village and then up in the corner, that text would change to tell you what's going on in that village, and now, you can see it.
It was a quick overview when you're watching.
And so you have the population that's sick and that's not sick, and the green bar is healthy and red is infected, and then an arrow telling you the direction in which the infection rate is going, whether it's going towards more people getting healthy or more people getting sick.
And then you also have those little circles beneath each locality.
We call them localities.
It's a better term than "village."
We wanted it to be very neutral, and that's what our clients told us we should use.
And so below each locality are those four little bubbles that show what you've implemented there, and they count down.
You might not be able to see it.
It's a little small on this screen, but they count down and become less colored in or a little bit darker so you know how much longer it'll last.
And then for the menu of when you clicked a village, that's what it originally looked like.
It was really hard to read.
There were a lot of choices that you could do.
We had eight, I think, is there, which at first, we were like, yeah, this is a strategy game.
It's going to be really hard.
There's going to be a lot of choices, a lot of things you can do, and this ended up being a very poor choice.
And so this is what it looks like now, where it's much clearer, first of all, how to close it, how to get back to the main screen, also what you can do.
When you hover over something, in the Description box, its description pops up.
If something's implemented, which nothing is in this locality, but if something was implemented, that circle would also be colored in and counting down to how long until you can implement it again.
So then what went wrong.
First thing was coding.
Working together is really hard.
We learned this over the project.
So we thought, OK, we'll divide everything up.
We'll make it so everyone can work independently and make sure there aren't any real dependencies that are going to be roadblocks.
But even doing that, it's still hard when you have to bring everything back together and there were just inconsistencies.
People had different coding styles and merging just took a lot longer than it should have taken.
Motivation was also a really big problem.
At the beginning of this project, people weren't really passionate about cholera or making a game about cholera, and even as we came up with designs and interesting choice and things we thought would be fun, that passion just didn't grow.
It was always just like, I don't want to do this, which is really hard.
I don't know how we could change that.
Also, the topic was really restrictive.
It's something that probably that I'm guessing must happen a lot in the real world that sometimes you work on things that you don't want to work on, but you have to find a way to struggle through that.
We also had crunch problems.
Our team was just a lot of crunches.
There wasn't constant work.
There was a lot of crunches.
And so we had a lot of crunching right before a play testing class.
We had a lot of crunching after Thanksgiving before now, and the work on the game was really low priority because everyone has other classes that they need to do things for.
And so it is hard to tell people, no, you need to do this because they're like, I have a thesis project due.
That's way more important than a project for this class.
And another thing that happened over time is that people weren't adhering to the communication we agreed on.
We said, OK, we'll do emails and we'll use Gitter, which is sort of tied in with GitHub, and then people weren't on Gitter or they didn't reply to emails.
And so whenever you sent out a message, it was like throwing it out into the ether and then hoping people would respond, which is really challenging.
Sometimes you forget to log onto Gitter or you forget to look at an email until later, so it's difficult.
It's something that happens, but it's a hard thing to deal with.
So then what we learned.
Within game design, there's actually a lot we learned from this project.
The biggest thing is that more complicated and more choices does not equal more fun.
It can, and if you're making a really big strategy game like Civilization, I could see where you'd want to have the ability to do a lot of things, but in something simpler, you don't need to be able to do as many things.
But if those things have meaning, which our choices now do, it's a lot more fun.
Play testing revealed that a lot of people didn't like having so many choices.
They didn't know what to do.
It was real time, so people were dying and they were like, I lost the game before I even felt like I started.
Design is very restrictive.
You need to pay attention not only to what the client wants, but also who you're making this for.
How accessible does it need to be?
Who's the target audience?
And also, it's really hard as a designer to realize you need to put both the player and the client's needs before your own.
You as a designer know, I have this grand vision of what our game should be, but then play testers say, we don't like your vision.
You need to change it.
It needs to do this.
And your clients say, we don't like your vision either.
It needs to change.
You need to do this.
And so doing that is very hard as a designer, but we learned to do it and the game was better as a result of it.
Some other things is none of us knew anything about cholera when we started, so we learned a lot about cholera.
And it's surprising how easy it is if you just wash your hands.
That's it.
So if you're in Ghana, just wash your hands, guys.
And probably don't drink infected water sources, but the big one is wash your hands.
And that also applies to a lot of other things.
Pablo actually asked us to emphasize the washing your hands because Ebola also, it benefits if people wash your hands.
It helps you so you don't catch other diseases.
And then about coding is tweening and other animation APIs make a game look more polished for little effort, which is something we didn't really implement until the end, and then when we did, we were like, wow, that looks a lot better.
And then future changes.
So for just within the game, things that we never got to implement but we really would like to are subtle effects that would force people to change their strategy because sometimes it can get really easy, and so we'd want to make it so people who quickly figured out a strategy and then think, oh, we can win is we wanted to make some things that their strategy doesn't work anymore.
So we originally we had the idea of a change in season, which is something we would like to do, but time constraints, or random events where there's maybe not a change in season, but there happened to be really heavy rain one day so everything flooded, so now all the water sources are more infected.
Also, multiple levels of difficulty.
We did like simplifying it, but there are still some things we'd like to add, at least one more action and maybe a few more villages.
So that's something that we would like to add if we had more time.
And also, more advanced infection models.
You can talk about the infection models if you want.
STUDENT 3: I'll chime in there.
It turns out to be really, really hard to balance the spread of infection of a disease.
It's almost impossible to make the game relatively simple at the beginning but then the right level of difficulty at the end because the nature of infection is that it's exponential.
You change a parameter by nothing and it just swings the whole thing over and makes the game go from very easy to completely impossible.
So that made it really, really hard to balance.
If I had more time, if I had a few more weeks to work on it, probably the things I'd want to work on would be work on something more advanced, like some more advanced mathematical models, as opposed to just a simple exponential model that we currently have, because that could make game balancing much easier.
I had a really hard time balancing infection.
STUDENT 2: And lastly, some things we'd change for the class is we wished that we had something that enforced the sprint tasklist.
We submitted it every week but we were never held to actually doing what we said we were going to do.
So it felt just like, oh, it's something we have to do.
We have to say we'll do things this week in order to be working on the game.
So we wish there was something like the instructors asking, OK, can we see your completed sprint tasklist?
The restriction to the Red Cross topics was a huge constraint for the creative freedom, especially for the last project.
So because all of us didn't really like the topic to begin with, it was a really long eight weeks.
So we wish that we didn't have to do that, but I understand for this semester of this class, that was what happened.
And then another thing is that we wouldn't work on an educational game regarding an unknown culture for the final project because it was just really hard to understand, OK, what do people in Ghana find normal?
What would they want in a game?
Do they understand how this works?
It's something that's even hard because there are a lot of assumptions people who have played games make about games that's really hard to get over.
And so doing that for the final project that also has to be an educational game that also has to be about cholera, it's just like there's a lot of things piling up that made it really hard to do.
So any questions?
And we'll also show a demo of the game, but if you guys want to ask any questions first.
[APPLAUSE] Should we do demo first and then questions?
Yeah, go ahead and demo
Both on this screen and with some of the earlier teams, many of you are running off Mac and you are using a screen resolution that's better for your screen than for the projector.
So it's actually rescaling it to fit the projector, which means actually, every pixel that you're drawing on screen is actually too small for the projector and things get blurred out.
In the Screen Resolution Settings in System Preferences, you can actually select whether you are scaling it optimized for either display.
I guess in this case, you are optimizing it for display
Which resolution are you using?
It looks like you're using 1600.
STUDENT 3: What's the best one?
It's on the board right behind you.
There's a wide screen and a four three version
So check that because you're using a lot of visual detail when you're scaling it like that.
It makes your games look worse.
That looks way better
And the color scheme is going to be better, too
Yeah.
Your colors do look better as a result.
STUDENT 3: Cool.
Awesome.
So we still have a little bit of things to iron out, but the idea of the game is that you manage these villages.
Cholera is spreading and you have to use various prevention measures to stop it from spreading further.
So when the game starts out, there's just one village.
People were having difficulty figuring out that you could actually click it, so we made it do like that so you know you can click it.
So you click it and then there's various options that you can use to stop the spread of cholera.
So it's just the first village, so I'll just go with soap, which is the cheapest option, and that usually stops.
I'll make it go faster.
The first village is sort of a tutorial.
Once you've figured out how everything works and you're able to stop the infection there and you cure it, then a second village pops up.
As the game goes on, more villages pop up, up to a total of four villages.
Basically, at the beginning, it's quite easy.
You can't really lose on the first village unless you sit and stare at it and do nothing for three minutes.
But the second village comes up and now you have to think about things, then you get an idea of how the game is actually played.
By the time the third village comes up, you really need a strategy or you're going to lose.
And the fourth village, it just gets really, really hard.
Generally, most people actually fail in the third village, don't make it to the fourth one.
I guess if you guys want to see more game play
Is there sound?
STUDENT 3: So we don't have sound right now.
We're going to add that in today.
So the three different options, basically, soap is the cheap, all around option because we want the takeaway of the game is soap is overpowering.
These two options distinguish themselves in that water containers are good for stopping an infection from spreading but they don't cure anymore, and electrolytes don't stop anything from spreading but they immediately cure just a percentage of the infected population.
What we found is that I thought electrolytes weren't actually going to be fun, but then they are, because at the end of the game when the infection is all over the place and everyone's dying, you just keep spanning electrolytes.
It has a cool down so you can't keep spanning, but you span it as much as you can to desperately stay alive.
I don't know how far to actually take this
That's fine for us.
A live player is going to play it on their own, so they'll be figuring stuff out.
Again, give them a couple minutes, and then you can start talking if you'd like to talk over.
For the presentation itself, when you started out, we could use a little bit more context about what you're talking about.
You kind of just throw us into what went right and one of the things that went right, so maybe telling us what you're going to talk about might be a way to give context for the full presentation.
STUDENT 2: Would starting with the game give a better idea?
That would be good, especially if you're talking over the game and talking about some of the design.
Then we can kind of call back
Another context related issue is you refer to your client and you refer to the game and you refer to making the game about cholera but you don't actually explicitly state who your client is, what your game is for, what the goal you were trying to create your game for, what specifically it was.
You could gather it from context, but it's a nice thing to have upfront
So the early slides, they're pretty wordy.
Both of you are actually really good speakers, so I feel like you can get a lot of those points in your actual verbal-- you did, in fact, get all those points across in your verbal things.
The slides don't need to also restate every single thing.
So stick to high level headers, but the way how you're actually presenting these points are coming across very clearly verbally and the words are starting to distract a little bit
Yeah.
Each one of those could have been just one slide plus an image
That becomes a backdrop to the things that you're actually saying, which are actually coming across quite clearly and you don't really need all of that additional visual noise, especially when you have sub-bullets, like you have a heading and then you have tiny little sub-bullets underneath.
Those sub-bullets, just say them.
Don't represent them.
The image of selecting which updates you have for each locality, right now you have the blank rectangle with mouseover text in it.
Nothing is lit up.
If you just replace that with a screenshot of something's lit up, something's not lit up with the mouseover text, that will be way more informative, that one slide alone.
When you talk about balancing the infection rate, that's a really good detail case example.
I just want to call that out.
That level of detail is neat because you're talking about really something very specific.
Other things that you can talk about-- I'm trying to remember the way how you had set up a communication system of using email and Gitter, and this is what Gitter is, but people weren't responding.
That was good.
That was very specific.
That was like, this is the way how something specifically broke down.
You can talk about how things eventually got resolved would be nice, but maybe it didn't get resolved, and you can say that, too.
That level of specificity was nice
One more.
Specifically on the slides, you talked about how you did iteration.
You talked about you made changes and you stayed very generic there.
I think some actual specific examples we really would like to hear.
What are some things that you dropped?
What are some things that you had in the beginning but you changed a whole lot?
They're in your end game but we don't recognize it from the beginning game because it got changed [?
in right now.
?]
You don't need a whole lot of examples, just one specific ones would tie it back
Yeah.
So if, in that particular section, you just added, here's a specific feature that got changed in this particular way, that's great because you clearly have that information
All right.
Thank you.
[APPLAUSE] So thank you all.
Everyone, sorry we ran a little bit late, but timing wise, everything looks like it went out.
Tech wise, it looks like we've got most of our stuff figured out.
A couple points of order.
This did go long.
It's probably going to go about full two hours on Wednesday.
Do you want to take a break midway through?
Would you like an intermission midway through?
I'm seeing some nods, so we're going to put an intermission midway through.
Lastly, based on the number of speakers that we've got and the topic matter that we're talking about, I think what we're going to do is start with Snap because everybody will have their computers out from the get go and then we won't need that again afterwards.
Just take care of that right there.
You're going to have people playing live in the audience.
Then we're going to go to Hello Wave.
Basically, we're going to swap Heat Wave and Snap and Saving Gora Gora and Cholera, swap those two.
And for that, because we're going to have lots of speakers at the so that will just make it so that you're doing all the moving around.
Cholera Control, you're going to have two people for that, so we're going to make sure we have two microphones for all of that.
We'll have the intermission right after Heat Wave.
So that's about an hour and a half, and then we'll have another hour, and then that's the end of the class on Wednesday
So that's the order of presentation right there
Yeah.
One, two, three, four, five.
There we go.
Cool.
I'm trying to think if there's anything else that I should get to before we release you.
No.
Student evaluations, web.mit.edu/studentevaluations.
I'll write up the real URL right there.
Please do that in class today if you could.
Thank you so much.
You actually are doing a really good job.
I really like these presentations.
I'm really excited to see what they're going to be like on Wednesday, and I was really happy to see working games today, so aces on you for that.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So to all of our guests, thank you for coming.
This is the final presentations for CMS.611 crossed listed as 6.073 the Creating Video Games class.
In this class, we have 40 students, learning how to make games and learning about the production process of making games.
So a lot of the things we're talking about in class, a little bit about design, but often also about how do you communicate as a team.
How do you get work done on time and on schedule, as I overheard a lot of things happening this last week.
Every year, we choose a client to make games for.
This year, we were very blessed to have the Red Cross/Red Crescent Climate Centre, in Pablo Suarez.
As our common client, he presented a number of different games, ideas, and topics ideas to the class.
The class formed teams about those for the final projects.
And that's what we're going to be seeing today.
Our order of presentations are going to be over here.
So we're going to have three presentations.
Each presentation is about 20 minutes plus another 10 for Q&A and then just tech switches.
We'll have the intermission after Heatwaves is finished.
And then we've got two more.
Cholera, I did not write down the full name for your game.
Your team, Lauren.
What is the full name for your game?
Cholera Control
Cholera Control.
Make a note of that.
So they and [INAUDIBLE] will be last.
I also wanted to mention to all the students here, if you're interested in continuing your education on game development, as you can see, spring is loaded with game courses.
CMS.301, which is taught by Sara and Michael in the back-- hi, Michael
Hi
It is an introduction to game design methods.
In particular, it's unpacking the idea of prototyping [INAUDIBLE] for digital games.
And the lessons learned there can be applied to pretty much any kind of designer, but they're focusing on game design methods.
CMS.608, Phillip and I teach, is board game and card game design.
It's structured, actually, very similar to this class with a few small projects and then a final project at the end that you spend about half the semester on.
And those, of course, are non-digital games, so board games, card games.
We've had people do live action RPGs and tabletop RPGs, role-playing games for that class.
The next two courses, CMS.610, Art, Science, and Business Video Games, and CMS.617, Advanced Game Studio, also built on the team building aspect but not as harshly as we do.
You won't be in eight-person teams, unless you really want to.
Teams tend to be about four to five people for those classes, and actually as low as three in some cases for 617.
Right now in the catalog, they both look like they're being taught on Thursday.
We are trying to move 617 to Tuesday.
So I'll send out an announcement about that.
And then lastly, if you liked creating the specific games you made in class this semester, these kind of educational games, or do you want to know more about how games are used in education, how to make games for education and how to simulations in particular for education, CMS.590 cross-listed I think-- I didn't look up the number-- but my memory said 11.129 is Computer Games and Simulation for Investigation and Education, tion.
That is taught-- last year, it was taught by Scot Osterweil and Jason Haas with Eric Klopfer is the faculty on that course.
A really great course, I highly recommend it.
You can look up the syllabus in OCW if you want to know more about it.
And with that, we are going to start with Snap
All right, hi, everybody.
I'm Tej, and I'm from the Snap team.
We're going to start off with a demo of our game.
So the way our game works is it's called Snap and everybody participates from their own computer.
We'll give you a topic, and then you'll enter words related to that topic.
And whenever you get the same word as somebody else, you'll get a point.
So it's pretty simple.
So we'll invite you to join us at snapgame.org.
We'll be showing the moderator interface.
So normally, you wouldn't be seeing this.
The moderator would do it on their own.
But we'd like to show this, because it is a part of our game.
All right, and the topic we'll be doing today, to mix it up from last time, is video games.
And we'll run the game for about two minutes this time.
All right, so I have no idea how long it's been.
But I'm going to give a countdown, and I'm going to stop the game.
10, 9, 8, 7, 6, 5, 4, 3, 2, 1.
And that's a game.
Congratulations, Julia, and Kevin in second place.
This is what you guys invented.
Thank you, guys, for playing Snap.
That was [INAUDIBLE]
Luigi [?
book.
?]
What was that?
Luigi
All right, so as you can see, this gives you an idea of what you were thinking about, what you think about when you think video games, which not surprisingly right now is Snap.
And with that, we've given a feel for the moderator interface.
And hopefully, you got a chance to play the game and see what the player sees.
And we'll start the presentation.
So this game was made for the Red Cross.
Pablo actually helped co-develop this game.
And he presented it to us as a face-to-face game that they play at conferences.
He provided feedback and guidance.
And in our project, in particular, the client was extremely helpful in giving us an idea of what they wanted and also helping us run the game in their actual settings.
So we were able to see what this looks like with the players that we expect to play with.
For our organization of our project, we split up into backend and frontend teams.
The backend worked on both the server and also the moderator interface you saw here and to word cloud, for example.
This split was challenging for new features, especially, because features would be interdependent or frontend couldn't do anything until there was a backend.
And there was a communication gap throughout.
When we needed to add a new feature, the API between the backend and frontend wasn't necessarily well-documented.
But overall, we think that it led to simpler management within the teams.
We didn't have a lot of organization within each team.
There wasn't a lot of explicit organization.
But it did help to know that a smaller set of people would be working on the frontend or the backend at any given time.
In the future, I think we would have gone even more specific in our roles.
Like a few roles that we thought would have been important include a UI designer.
This would have made sure that UI design saw some first class love from the beginning and would have made sure that we actually focused on UI from the beginning.
It also would have helped to have somebody communicating between the teams.
This person may have done some documentation or may have simply just made sure that the frontend knew what the backend was working on and the backend knew what the frontend needed, so we could prioritize well between the teams.
It also would have helped to have somebody dedicated to paperwork.
This would have helped both in submitting class paperwork and also just doing the documentation that we needed, both for ourselves and the class.
And finally, somebody working on game design would have helped us develop more iterations early on and think more about the game aspects that go beyond what we already knew the game would look like based on the game of Snap we played from the clients.
As far as technology for our organization, we used Asana to organize our tasks.
Asana worked really well for us.
In particular, it let us split the task lists for the frontend and backend.
This was useful, because generally, the frontend didn't need to touch the backend tasks and vice versa.
But on the other hand, we had a hard time encoding task dependencies.
This was particularly a problem for a feature that crossed both the backend and frontend, where some API was needed on the backend and then was also needed on the frontend.
And you couldn't do one without the other.
We also had trouble with task creation.
And this basically meant that people didn't create tasks.
And part of this was that we didn't have a specific role for creating the tasks.
And so, because of the diffusion of responsibility, nobody would go through and list out everything that needed to be done.
To improve that in the future, we'd like to get people more invested.
Part of this was that people had a lot of other classes.
And it would have helped if people both explicitly said how much they were going to commit to the class and also we made it easier for people to contribute with the small amount of time that they had available
So for communication, our group used an instant messaging application called Slack.
And this was really great for us, because it allowed us to separate into different channels.
As you see here, we have frontend and backend as two different channels and then a general channel where everyone can communicate.
And this let the frontend people work on their thing separately from the backend, not needing to know exactly what was going on.
But they could go look, if they needed to reference some conversation there.
The bad part about this was that people weren't always online and responsive right away.
And I think we could of made this not a problem, if we required everyone in our project to download the mobile application.
That way, they would get a notification immediately when they were mentioned.
And they would be more likely to respond quicker
On the backend, generally things went really well.
Our technology, in particular, worked really well for us.
We used node.js for the server, and socket.io for two-way communication with the browser, and Heroku to do deployment.
We were explicitly forbidden from using networking in this class, which we violated from the beginning.
But I think that our experience with networking and all three of these technologies really helped make that possible.
We already knew how to use them.
And that helped us both debug issues and understand what we were working with.
On the other hand, the backend it seemed had too much manpower.
This manifested itself as a problem on the frontend, where sometimes we felt like we didn't have enough people.
We could have had fewer people working on the backend, which would have made sure that it was a small unified code base, and we could quickly finalize the backend and just move on.
We actually deployed the application, and the client has been using what we showed.
This is actually a quite early deployment on November 18 in Berlin with some students there.
And you can see the words that they generated.
Here are some other deployments.
One of the things that made this class-- what was challenging for us is that we had to manage both client expectations and the course expectations.
So because they were asking us to do these runs, it seemed important to us to support these runs.
And it was hard to balance doing this and also during the tasks that the course required of us and needed for us to do
So I'm going to talk a little bit about our frontend design and that process.
So initially, when we started the project, we had this idea that people would be represented as a car and they would be racing other players.
So we thought it would be a great idea to use Phaser, which is a sprite-based game framework.
But as you see here, we didn't go through with the car idea.
And we had no sprites.
But we still used Phaser to make this, our first design iteration.
And we realized we didn't really want that type of look for our game.
We needed something more professional and more abstract that could be used in any sort of setting that Pablo, our client, may use it.
So we completely scrapped Phaser.
And we redid our frontend design, deciding to just use Bootstrap and CoffeeScript.
And this was our second design.
As you can see here, it's just very typical Bootstrap styling, if you've ever used Bootstrap before.
So we tried to add a little bit of flavor.
And we also needed a better way to kind of represent how the game is progressing to the player.
We currently had no idea for that in this design.
So then we came up with the final design that you guys played, our theme of having blue.
And then we created this visualization to the right, where each player is represented by a dot.
And snaps are represented by lines being drawn to dots.
And your height is representative of your score in relation to all the other players.
So how we kind of got to this final visualization idea was an interesting process.
First, we thought it'd be really cool to have a word cloud on the frontend, as well as the back end.
But a player would only be able to see the words that they had submitted.
All the other words would be blurred out.
But this didn't really convey to the player where they were in relation to all the other players.
So there wasn't any sort of competitiveness.
So we came up with this idea of having a race line, where all the players would be on the line and the height would represent their score and their position.
And then we thought, well, we can put these both together.
But we investigated the word cloud.
And we realized it'd be extremely difficult to blur out these words.
And it wasn't really going to be the best use of our time.
So then we came up with this idea of using dots to represent players.
And we thought it'd be cool if the size of your dot represented your score.
But the use case for our application could involve 100 players or even more.
So a lot of growing large dots could really clutter our UI, so we needed to rethink this.
And this is when we finally ended up deciding on height representing score.
So I think we had really good UI designers.
And they were really quick.
Once we thought of ideas, we were quick being able to implement it.
But the bad part was that we were really late in coming up with design decisions.
We didn't start thinking about how we wanted to convey the game to the player and how we wanted to make the player feel about the game until really late in the project.
We focused more on what the client needed as far as mechanics and not really what the player was getting out of the game.
So in the future, definitely we need to think about design right from the get-go.
And we need to start prototyping all the ideas that we have and testing them out more thoroughly
So in summary, the high points of our project are that we did make the client happy.
And they've been using our application and the game.
And they really like the game.
We're glad that we focused on fun.
The game has been fun throughout.
And players seem to really enjoy it.
And I think maybe this is just because of the core concept.
And we're glad that, even though this is an educational game, it is intended for purposes honestly of data collection, people really enjoy playing it.
Unfortunately, we had slow design iteration.
This prevented us from really fully realizing all the ideas that we had and figuring out which one would be best for players.
In the future, we'd like to get people more invested to try to solve these issues and make sure that we best use our time and create the best product possible.
And with that, we'll take questions
[INAUDIBLE] Is there a maximum number of players?
We did some stress tests to see how much the backend on Heroku could support.
And that seemed to work OK up to 150 players or teams.
We are not really sure what the frontend might look like at 150 real players.
That might be actually the limiting factor
Is that 150 people playing the same session or in the multiple sessions 150 people?
We currently don't support multiple sessions.
This is actually a request that the client repeatedly gave us.
But we thought that it was just much simpler to avoid that for now, because it wasn't strictly necessary.
Most likely, the load on the backend would be the same, whether they were split into multiple sessions.
But this is a solvable problem by getting a beefier backend
I see.
But right now, the backend supports 150 people logged in simultaneously
About 150.
We can obviously pay money to get better servers and then run with many more

Any questions from the audience?
So you got some feedback from Pablo specifically from the moderator point of view.
And I'm wondering whether you got any indication about what the players felt outside of this class plus your [INAUDIBLE]
Sure, I think we can both talk about our focus tests, where we played with players outside this class
Yeah, so one of the main things, I played with maybe a group of at most 10 people.
And they knew each other, so they wanted to know who they were snapping with.
And they wanted to see the names of all the dots, so they could know who they were beating or who they needed to catch up to.
But this would be unreasonable in a large environment, especially when all the people at the conference probably don't know each other.
But I think that would be a really cool thing to do, if we continued working on this in the future
And I also ran a similar test with people that knew each other.
And they actually used it as part of their strategy.
They knew what other players would be submitting for that topic.
And they could use that to their advantage.
So, for example, we did a topic of systems papers.
And we had a professor in the group, so his paper showed up a bunch.
And I would echo what Sabrina said, that players really wanted more feedback around the actual players in the game, which we specifically didn't do, because the client didn't care.
And in conferences, people don't generally know each other.
But I think that if we were to use this as a more popular game, then definitely focusing on players and getting more feedback, especially after the game is over.
People wanted to know who they were snapping with.
They wanted to see that graph, all those lines overlaid, so they could see who they were snapping with most often
Does the team have any plans for this game after this class?
I was wondering if you've ever had discussions yet
I think we haven't discussed that yet.
But I think definitely we're interested in it
Thank you
Thank you
Hello.
We are Hello waves.
We're game about forecasting, specifically this idea called forecast-based financing of using forecast to make decisions about possible disasters in the future and how to prepare for them.
We would actually like to start with a playthrough of our game.
So if anybody would like to be a volunteer to try it out
A guest from not our class, if one could please come down, Andrew
Thank you for volunteering
Hi, I'm Andrew
Well, do you need a chair?
Here's one
I'm not quite sure how to make that full screen.
But anyways, this is our game.
I'd recommend looking at the instructions first and reading through them.
Thanks.
And we'll keep the description minimal as you read through it just to sort of show the player's initial reaction to it
Now, this is real, correct?
You're signed on?
Oh, is it?
Oh
Yeah, right.
OK, sorry.
Music
Yeah, start up again.
So as you can see, our game, as I said before, you control some toys on a day at the beach.
And so by dragging and dropping them between the castles, you'll see that their statuses change and say that they want to move on their next turn, or that they want to be collecting-- or that they're going to be collecting candy, or anything like that.
And the game is turned based.
So all the actions will resolve on the next turn
Am I rating this race?
Yeah
You can also access the help through the lower right corner as well
And so when you go to the next turn, you'll see all the toys move as specified by the status bubble above their heads
So I go to the next turn, or do I get a next turn?
But unfortunately, you'll find that when toys have been evacuated, they're unhappy and need candy to survive, so they'll all take damage
They don't want you to evacuate
You can try returning them to their homes
And so you'll see that on this turn instead, now they have their statuses set to gathering candy, except for the dump truck, who is still evacuated
Got you, OK.
But he needs to be evacuated
Exactly.
And so the idea is that as a player plays through it, they get better at understanding how to use the forecast to make decisions about the future, both in terms of how much candy that they need to have stocked up in order to weather out the rising tides and also in terms of when they're going to need to move their workers out of-- their toys out of the areas that are in danger
They're going to be really impacted
I think you have those two guys
Oh, they're in there?
Is there a reminder of where they started?
Yeah, it's on the castle.
It's actually blocked out right now, but toward the front of it.
But there's a little shadow there, an imprint.
Because he ran out of candy and then-- or actually, sorry, because he went to a place that was underwater, he took too much damage and then was swept away by the waves.
And so on the forecast, you can see that high water is coming for quite a while, which is going to be a danger for the toys, both in terms of possibly getting swept away and not having enough candy for all the toys that you're going to have to move out of the way
[INAUDIBLE] this time
Oh, he may be out of luck
Yeah.
And when you've lost two toys, you lose the game

Thank you for playing.
So that's Hello waves.
So in our game, we had a few challenges to overcome.
The first was that our game was based on forecast-based financing, which is a very abstract topic.
It's a pretty understandable idea of using information about the future and ideas of risk in order to decide where to allocate resources.
But it's still a bit abstract.
And building a game around it took a little bit of work.
It's useful to note that it's different than long-term planning.
It's not just thinking about what will happen in the future, building a dam to prevent water or things like that.
And it's actually about using the information you have to make the best decision for events that may be upcoming in the semi-near future.
We also wanted to avoid making a game that was overly preachy or simplified, where it was clear exactly how you're going to win.
And you could just basically push the forecast-based financing button to win the game.
We wanted players to actually think and understand the concepts there, instead of just coming up with the buzzword of forecast-based financing.
And finally, we had the challenge of actually communicating that forecast to players in a way that they would be able to understand and then make use of.
You could see in that game that we had water levels and it would show on the map.
And we found that players were pretty good at using that in order to make decisions about what was going to happen in the future and how to allocate their resources.
The other big challenge that we had in the beginning was that our initial target audience was policymakers.
Like for Snap, Pablo had come in and pitched us this game idea.
And originally, he had wanted us to build a game for policymakers that would help them understand the benefits of forecast-based financing and, therefore, convince them that they should develop policies that would give resources to plans based on forecast-based financing.
So I would like to take you through a couple of our prototypes just to show you the evolution and comment sort of on our process.
Actually, to preface that, we had a lot of prototypes, because of our idea was so abstract and because we weren't sure how to address our audience.
So we built a lot of prototypes to start with.
We had ideas that ranged across sort of levels of scope of what you controlled, where you controlled entire cities or where you controlled individual workers and moved them around.
And then we would pull from all these different kinds of ideas what worked, what didn't, what did people understand, what confused them.
And from that, we got a really good idea of what concepts helped people understand the idea and brought them into this final game that we ended up with.
So the project started on October 15.
And this was our first prototype.
It was a terminal-based game, where you had some kind of information about a future rainfall.
And then you had to type in your commands of how you controlled different cities.
This, actually, people found it fun.
But as you might expect, the feedback wasn't very good.
And people didn't quite understand how to move forward with it.
It put a lot of cognitive load on people.
So when we moved forward, we tried to give people more easily understandable actions to use.
But the problem with this game was that it was time based and it updated every second.
And so there are so numbers flying at people, like even MIT students who play tested it couldn't understand.
So we figured that people like policymakers, who didn't have much experience with games, really wouldn't be able to understand the game at all.
So instead, we went to turn based.
And that helped.
But at the same time, it's tough to see on this projector, but we have a forecast underneath that says how much rainfall is expected.
And again, that wasn't understandable to people, because they couldn't understand what three inches of rainfall meant for their city.
And they couldn't understand how that contributed to a possible disaster.
At this point, Pablo actually came by the class and played the game.
And then he told us that he wasn't even sure that he could get policymakers to play the game, because they may not have enough time.
Instead, he wanted us to switch to grade schoolers, because we could teach them something about forecast-based financing and help them understand as they grew up.
And this was great for us, because making a game that was serious, easy for somebody who didn't have experience with games to play, and also fun and engaging was too difficult for us actually.
So moving to grade schoolers was awesome.
He also suggested that we try to make the idea of rainfall or water levels more visceral.
And that's when we came upon this idea of the rising waters.
Whenever players looked at this, they instantly understood the concept of the game.
The feedback might not be there.
The beautiful UI might not be there.
But the idea of having cities with workers in them and a rising water level coming towards them, everybody understood that.
And it made it a lot easier for players to reason about the game.
From there, we added things like nicer art, better feedback, which you can't quite see in the static picture, more improvements to how the forecast worked.
And eventually, we ended up with our final version today
So to talk a little bit about our actual development process, so our team was structured into three main subteams, production, which was in charge of managerial roles, deliverables, and play testing, and so there was a shared responsibility there; technical team, which was in charge of the bulk of the coding work; and a user experience team, which would be in charge of assets, UI design, and UI design.
We also initially envisioned subteam leaders, where we'd be kind of communicating through them.
But we found the concept kind of redundant.
And so we've worked pretty much with like a flat structure between the three teams.
So from the beginning, we encouraged good coding practice.
And so we used good tools available to us.
One of these is Yeoman, which is a JavaScript scaffolding framework.
And this helped us out a lot by basically automating a lot of our JavaScript tasks and making our code modular.
We also used Phaser state machine, which is this kind of badly documented new feature in the Phaser game engine, which was the JavaScript game engine that we used.
It's a bit badly documented.
But it did save us a lot of headaches.
And once we figured that out, that proved immensely helpful.
And we also used MVC, which is a computer software engineering term standing for Model View Controller.
And again, by encouraging these good coding practices, we reduced dependencies and made sure that our team was productive.
In terms of communication, we also, similar to Snap, used Slack, which is kind of like a modernized chatrooms.
It's very feature rich.
And so you can share files, channels, and things like that.
And we also use the idea of the Daily Scrum.
We implemented it in class.
And we would say what we had done that class, what we would be doing, and what we wanted to do until next class.
And so now, the major challenges that we faced during the development process though where that our team members came from very different backgrounds and had very different preferences about games.
Some of our team members are very hardcore StarCraft players and very good at RTS games, while other members of our team preferred like a more laid back mobile game Fruit Ninja kind of approach.
And so trying to mediate those two viewpoints and trying to create a game that would engage both types of gamers was a challenge that we had to overcome.
And so another big challenge that we had in our development process, as you saw through our progression through our different prototypes, was that our direction was not very clear, until about halfway through the project.
And so partially because we had different ideas on what the game should look like and also partially because we had such an abstract idea of forecast-based financing that even we didn't have that good of a grasp on initially, it took us a while to really get settled on what we wanted to build.
And so this really challenged our development process and made us have to build a lot before we got something that we liked.
And so eventually, we did end up having to cut some features of like multiple levels or like a guided tutorial for the player.
We thought that this would introduce too much new content that would need to be play tested, balanced, and tested to ensure consistency with the rest of our game, which we viewed as taking up too much time.
And we also cut the idea of adding more individuality to the workers or to the toys that you saw, other than different graphics for each
Some of the worst things that we did on our team is that we spent a lot of time on work that got thrown out entirely.
All the prototypes we did, they were actually pretty useful, because of the things we learned about the concept and about how people would play the game.
But we did spend a lot of time on things like art or nitty gritty details that didn't really need to be figured out and that we could have put off until later in the project.
We also kept the game direction too vague for too long.
As Norman said, we spent a lot of time with that.
And it probably ate up too much of our time.
Although it helped us learn, we could have moved faster in the beginning to get to a solid idea.
Because once we got to a solid idea of the rising waters, our team started to centralize around it a lot better, because we could actually deal with something concrete.
While we were dealing with the abstract ideas, everybody was all over the place and arguing about things that didn't quite line up.
And the worst decision of all that we started with and that sort of made the problem of going too vague and all these other things happen is that we were originally thinking, how do we skin forecast based financing as a game?
How do we take this idea of using forecast to make decisions and then just gamify it, which we eventually realized wasn't fun and didn't help us actually come up with any ideas.
Instead when we flipped it and started talking about what game could we create and use forecast-based financing to improve it and teach players how to play the game and, therefore, allow them to come out of the game, having learned about forecast-based financing, the world sort of opened up.
Everything became a lot more interesting.
And we found that we started to move faster.
Some of the best decisions that we made, on the other hand, as Norman said, we had good tools, which meant that even when we threw out prototypes, although we wasted things like art resources and things like that, we actually didn't end up wasting very much code, because things like the idea of workers or cities could literally be pulled out of the old games we had, put into our new game, and then reworked to build our new structure.
We also weren't afraid to trust each other and throw out the things that didn't work.
Once we started moving fast, we had a lot of ideas that would come out, and we would say, OK, this doesn't work.
We're actually going to scrap it.
Or we think that this isn't the direction we need to go in.
And everybody was willing to go along with it.
It's not a good feeling to see your things thrown out.
But everybody understood that was for the best of the game.
And I really appreciate their understanding with everything too.
And part of that all comes down to the fact that we were on board with our final idea.
We were all excited about the concept that we had.
Part of that might have been the relief of coming to a concrete idea after spending so much time being vague.
But once we had the concrete idea, we really moved fast and worked well around it.
So thank you and any questions
Who did your sound?
It's awesome
That was from our UI team
Oh, [INAUDIBLE].
OK, I really liked it
Thank you
Where did you find it?
So it's on our credits page.
Most of it was online.
The credits page in our game
And so you don't want to type it in
It's a little small, I guess, up here
Oh, OK, great
But it looks--
[?
Switch games.
?]
Yeah, "Hold My Hand," AGP by--
Would it be possible to do a [?
shortened ?]
URL of these?
For the game?
Oh, yeah, sure.
We can make a Bitly link, and we'll send it out to everyone
Yeah, just to entire-- yeah
That's a good call.
Yes
Having watched the early crash and burn playthrough, how often do people-- I don't know if you've actually had a lot of people to play your current version of your game.
Do people usually take a playthrough or two before they start getting the concept?
Yes.
That's why something like a tutorial would be really nice.
Unfortunately, we didn't have the time to put it in
Yeah, no.
I was wondering how that--
Yeah, usually what happens is, even if after maybe a couple of turns of playing through they start to get the idea, the problem is that, as the water starts to rise, they haven't prepared enough.
And so all their workers or the toys will starve or get carried away by the waves, which is a bit unfortunate and probably makes the players feel bad on the first time.
But then they-- it actually teaches them to think ahead about it.
So the next playthrough, they're much more careful and understanding
Yes, I was wondering if you were playing any other games or thinking about other games as the inspiration or thinking about how to deal with sort of getting the right level of strategic thinking in your game
Well, so I guess in terms of early on.
Because we had a very different idea of what we wanted to do early on in the process, we were thinking about games like Civilization and how did they communicate all these complex worker movements and managing multiple cities.
But once we actually came up with this game concept, I think we had much, much smaller goals.
And so did we have any specific game models, do you think?
There is none that I specifically think of.
There were some things that we were sort of inspired by, standard tricks, like when you hover over one of the characters, they got bigger, some idea of showing off that this is clickable, things like that.
But specific games themselves, not really
I was just thinking it ended up being kind of war gamey.
And I feel like there's a lot of war games that where I'd thinking a lot about getting that right level of I guess tactics rather than strategy.
But, yeah, I guess it worked out kind of
So you mentioned that you were able to completely change how your workers looked and just keep your models.
So that's sort of the Holy Grail of object-oriented programming, that kind of object that's reusable and you don't have to throw out the code.
Do you think there's a reason why, in particular, you were able to achieve that, because I think that's not-- that's not common necessarily in object-oriented programming?
Partially, a little bit of OCD-ness like very early on, very strictly saying we're going to write this object-oriented code, and we're not just going to hack things together.
I think that helped a lot, because we actually spent the time in the very beginning to think about like which objects were responsible for what and what their purpose should be.
Yeah, basically, I think because we moved more slowly in the start and thought more carefully about how that code should be structured, we ended up with having an easier time later on
There's also the fact that because we had learned these things from the prototype that we are putting into this game, that also meant that the objects that we created because we wanted similar functionality to things that we had already seen and we knew worked, it meant that we were comfortable pulling out the functionality into that.
I don't want to say it was designed to fit.
But there is the fact that we moved it on purpose, really
I would like to say something on it.
So we had a very good MVC model.
So models were at and has a tree-like structure, I think it was very easy to change the model somewhere else [?
and knew it ?]
involved-- sorry, used our new evolved models once [?
models and no idea ?]
of what happens.
So basically, it was [INAUDIBLE] change
Thank you.
All right, thank you
Hello, my name is Miriam.
And today, I'm going to be presenting for Team Heat Wave.
And we made a game called Heat Wave.
So the goal of Heat Wave was and is to educate Red Cross workers about how to prepare for and handle heat waves.
And if you've never been through a heat wave, it's just defined as elevated temperatures for an extended period of time.
And the way you prepare for that is you set up umbrellas for shade.
You get water coolers.
And during the actual event, you try to go inside and drink an excess amount of water.
So while we had that game goal, we also had design goals of usability, playability, and education-- usability because if the game doesn't open, it doesn't turn on, that's a problem, it doesn't work, playability because we wanted people to be able to play through the whole game, it's just very fundamental, and education because this game is meant to educate Red Cross workers about how to deal with heat waves.
So we really wanted that to be one of our primary objectives for the game.
So to meet these goals, we had our design processes.
And I'll talk a little bit about each of these in detail in a minute.
But we started off just as the other teams did with brainstorming.
Then we formed a team based off of our ideas.
And then we broke down responsibility.
So one person was in charge of one aspect of the game.
Another was in charge of like the graphics, all separate tasks.
And then we continuously updated.
And what we used for that was Trello, primarily.
And I'll talk more about that in a minute.
So brainstorming, we were actually right there on the board drawing things for a while.
And we had two major ideas going as a group of people just considering this topic.
The two ideas we had were the game we actually came up with, where you play as a Red Cross worker trying to help people.
And then our second game was playing as a heat wave, trying to make people faint.
And while that actually sounds like a lot of fun, we stuck with the first idea.
And the reason we did that is we thought we could get more education into the game.
Because education was such a primary focus for us, we really wanted it to be the core basis of the game.
So we went with the game that would directly correlate to what these Red Cross workers would be doing out in the streets before and during a heat wave.
And based off of that idea, we formed a team.
We actually had a pretty large team.
This is eight out of nine of our people.
Joe wasn't in this photo, because he wasn't there that day.
But we had a really large team.
So we had to be really focused as to like who was doing what and what was getting done when.
Because when you have such a large team, things can slip through the cracks.
So we used Scrum.
And as you can see up there, it says so many emails, because there were so many emails.
But the way we dealt with that based off of previous projects was to say, well, we're going to have a lot of emails.
There's nothing we can do about that.
But we can make the emails relevant, so we can break them into different threads.
So the people who were handling the character objects, they're just going to have their own thread.
And maybe like one person who's communicating between teams will also be on that thread.
But it's really their thread.
So we had a lot of emails.
But they were divided up as to who got to see them and who it affected.
And the other thing we did is we had our Trello.
And we constantly updated that.
And we kept track of what was being done in this sprint and previous sprints.
And it was really easy to see what was taking longer than we expected and what was done, what was done but could use improvements.
Like you could make a note.
We had like a not-done-yet section.
So we really managed to stay organized that way.
And the other thing we did to make sure things got done was group meetings.
And these were quintessential for our groups.
We had a bunch of like three-hour long meetings.
And we would get task after task done, bug after bug fixed, and really just worked through for hours on end.
So once we had this game and we were all working on it, we had four focus tests that we ran.
And they all had a theme.
So our first focus test was about the concept, can we educate people in the basic mechanic of that we think we're going to use.
And I'll talk about that in a minute.
The second one was overarching issues.
So I said our three design goals were usability, playability, and education.
So that was really our second focus assessed.
The third focus test was visuals.
Are we connecting with the user?
Are these the visuals we want to have?
And then for the fourth one was game balancing.
Does this game feel fun to play?
Is it working out?
So the first focus test, we focused on the general mechanics.
We had a low fidelity prototype, which is actually a Python-based game.
And there was a list of people.
And you could say, I want to talk to this person based on some attributes and offer them water, tell them to go inside.
And each turn, you could either continue talking to them, say forget it, you're being annoying, or oh, you accepted water, I'm going to move on to the next person.
Or you could do nothing.
And what we learned very quickly was that it was working.
People were choosing to try and help people based on their characteristics and based on oh, well, this person fainted really quickly, and they were homeless, so I think homeless people might faint faster.
Though actually, I don't think homeless people were in this game.
And we also experimented with different types of dialogues.
So we experimented with having like very factual dialogue, drinking water is really important, even if you're not too thirsty versus something like, hide your wife, hide your kids heat wave.
So once we had that general concept in place, we built a basic unit game.
And we wanted to see overarching issues.
So we had these four-- well five, but one person has already fainted in this picture-- five different characters with different attributes.
And you could choose to talk to them.
It was the same exact concept we had in the Python text-based game.
This time, it was a little more visual.
There were people and there was a dialogue that would show up.
But it wasn't fully there yet.
And what we saw was the usability and the playability were definitely having issues here.
You can see that.
But the education was definitely coming through it.
In fact, people wanted it to be more visual.
And we saw that, and we said, well, we have to test more visuals.
So you can see, this was the version we tested later.
And we said, how is this working, who are you choosing to click on?
And people said, well, first of all, it's confusing.
All these people look the same.
I can kind of tell who's a woman and who might be a man, but that's about it.
The labels aren't moving with the people.
Even if they were moving with the people, they're really too small.
We can't tell who's who.
And we said, well, we have this art that we think we're going to redo all the art in the style of.
We're thinking about getting rid of the labels.
Do you think that's a good idea?
And they said, yes, get rid of the labels.
They're too small anyway.
And so we showed them this caricature of one of the types of the characters, which is a homeless person.
They're a very vulnerable group to heat waves.
We said, this is kind of the art style we were looking at.
We think it displays more information.
Do you think that's good?
And we got enormously positive feedback on that.
They say, definitely, I would click on this guy, because he looks like he needs help.
And you'll see that later.
Something else we were having an issue with visually was that the way people would go inside is they would just kind of wander out.
But the way they fainted, they would actually just disappear, proof.
So people didn't notice it actually.
And then they'd suddenly be like, well, there was 10 people, and now there's five people, and what happened?
So we said, OK, we know this is an issue.
What do you think would be the best way to visually show that people are fainting and that would emotionally resonate with you?
And they said, well, we want them to fall over.
And maybe it'd be nice if there were some other feedback, like sound or something.
And we said, OK, we'll take that a step farther.
And we did.
And we'll talk about that later.
And the third thing that we really got back from this visual feedback focus test was that there was no indication when people were getting sick.
So in the real world, if you're getting sick, you might like turned red, get really sweaty.
If you're about to faint, it'll be very obvious is the point.
And there was no indication here.
So we said, well, we're going to do something about it.
We're going to put that on the back burner, keep thinking about it.
And we are going to fix that somehow.
And so we put that all into practice.
And you'll see the screens in a bit.
But our final focus test, we focused on game balancing.
Is this game balanced?
Are people fainting at a reasonable rate?
The problem was that we would have like six people all fainting at the same time.
They said no, they have to faint at different increments.
And also on top of that, we had an issue where the temperature was somewhat random, going up and down.
And people were not adjusting to that.
One day it was 110.
The next day was 80.
And that's in Fahrenheit, because I assume most of you use Fahrenheit typically.
But our game is actually in Celsius now.
But the idea was that people were not adjusting to this change.
And it wasn't realistic.
So what we do now is we actually ramp up the temperature, and so that the first day, very few if any people faint.
And then the next day, by the end of day, people start fainting, because it's getting hot.
And then the third day, they're fainting faster.
And you'll see that in the playthrough after this.
So during this whole process, we made some good and bad choices.
And I'd like to talk about bad choices first.
So we made two major-- these were the two, I think, biggest bad choices we made.
They're not completely horrible or anything.
The first was that we depended on external code.
And the second is we focused on education, not fun.
So when I mentioned the external code, one of our teammates had a dialogue system for unity, which is really cool, it looked really nice, had lot of features.
And we said, let's go for it.
We can use this as a foundation to build a really complex game.
And something we really didn't stop to think about was how much work is it going to be to develop our game around this system?
How much work is it going to be to change the system to fit our game?
And also are we going to have time to do all of that and also focus on making the game complete as a whole, instead of just kind of showcasing this one aspect.
And it turns out it was a bit of a pitfall for us, because we didn't expect it to take as much time as it did.
And if we were to go back, the teams kind of said, well, maybe we still do this and we just deal with the issues early on.
But also, maybe we should have just designed a simpler game and not use this external code.
Another bad choice we made was we really did focus on education, which was good, because our game does teach people.
But it is an educational game in the sense that it's not the most fun in the world.
And if we were to redo this, we probably should have focused on fun earlier.
We played with having funny dialogue and making more emotionally resonating effects.
But at the end of the day, it's an educational game.
And it's really not that exciting to play.
So that's something.
But in the process of doing that, we made some good choices too.
And one of those good choices we made was group meetings.
You get so much done when you sit down with like most of your team for three hours and you just hammer through.
And we did that multiple times.
And it really worked out for us.
Something else we did well was prioritize.
Because we had a team of nine people, we really had-- availability was kind of all over the place.
And even when we had group meetings, you could only really expect like six people to show up.
You were guaranteed that at least three people weren't going to be there.
Even if you did a doodle, which we did.
So what we prioritized was based on group-- we had groups of people with different responsibilities.
But if there was a bug, we said, OK, who is available right now?
What are you're working on?
Is this more of a priority?
Can you do this now?
And that process worked for us.
We got bug after bug fixed that way when we weren't in a group meeting.
And when we got to the group meeting, people generally had a sense of what was going on, because we were doing this process of they kind of had maybe worked on something for a little bit based on it being a priority.
So now, let's get to the actual game.
So the game has a start screen within an instruction screen, which is kind of the sublayer.
The main game, which has the day scene and then a newspaper scene at the night, and I'll show you all these.
And then the end screen.
So originally, this was what our start screen looked at that we play tested.
And it was beautiful, as you can see up there.
No need for changes, right?
So we actually did change it.
And so now, it's the same background as a game for Cohesion.
You have your play, but you can also use the instructions, which are pretty useful if you've never played before.
And then that takes you to our main game.
Our main game originally, you saw that on the first focus test.
I think this is pretty similar to our second focus test.
You see we had a progression with the main game.
It took a lot of iteration.
And something originally we didn't have, I said, was there was no indication of when somebody was getting sick or their characteristics.
It was a lot of issues.
So we made some progress.
We added the indicator that say, I'm about to faint.
You should probably try to help me now, which is that little explanation part on part of their head.
And then finally, we redid pretty much all of the graphics.
And that took a while.
So we were in the process of redoing them at this point.
And now, the game looks like this.
So it's much more cohesive.
All the art fits together.
The people are really visually caricatures of the different types of people you need to help.
Elderly people, homeless people, drunk people are all really vulnerable to heat waves.
And then there's some characters that actually aren't on this screen right here but that aren't as vulnerable, like normal, like adults aren't as vulnerable.
And athletes aren't as vulnerable.
So at the end of each day, when you play this through, and you've helped your people, and you've given them water or set up coolers or whatever you wanted to do for the day, you get to a newspaper scene.
And the original newspaper scene looked like this, a lot of text.
It doesn't really help you with much.
No one really cared much about it in the first iterations of the game, because the long-term objects hadn't been adding yet.
So it's just the umbrellas and the water coolers.
And so we changed it to look like this.
It's the same information, just a little better organized.
And the other thing that we added in the main game, we added those buttons that says, you can set up a water cooler or an umbrella.
And that takes your time up.
So if it's cool outside, maybe you do that.
But if it's like 110 degrees in Fahrenheit, then maybe you should focus on helping people not faint.
So we had that.
And then say you finish, 20 people faint or whatever-- I think it's 20 people faint-- and you get to the end screen.
And our original game, we had no end screen.
And the reason for that was, I told you we really focused on education not fun.
And we just didn't think, part of the fun of a game is seeing how long you survive.
But we just weren't thinking about that.
So we really reevaluated that.
And we said, OK, we definitely need an end screen.
And we have it looking like this.
So it tells you how long you lasted.
And then you can see the credits for what was involved in the game.
And you can play it again.
So there's more feedback for the user of how they did.
And if we were going to do this project again, there's two things I think we'd do majorly differently.
And the first is talking about heat waves.
Our group thought heat waves were going to be really cool.
We were like, oh, people are fainting everywhere, and it's getting hot, and we have to do something.
And then you start reading about heat waves.
And basically, it gets hot.
You tell people to go inside or drink water.
And that's about it.
That's all a heat wave is.
And so what we'd do differently is really the passion aspect.
If we're going to do a game about heat waves, we depended kind of on our passion coming from the thinking heat waves were cool.
I think we should have focused more on having passion for the game and then incorporating the-- well, also having the game build around heat waves but really having a more exciting game that we could somehow also get that educational aspect in.
What wed also do differently is accountability.
I said we had nine people.
Sometimes, people disappear when you have that many people.
And keeping dibs on everything, especially when you have separate email threads, can be a little confusing.
So maybe look into some of the tools that the other teams mentioned, where you have multiple group chats and you can see them all.
That looks really interesting.
Future features, more long-term options.
I think that's one of the things in our game that really helps with the planning aspect is OK, do I want to be doing this, like helping somebody now?
Or do I wanting to be installing an object now?
And really, our game kind of depends on that balance of choosing who to help and then also what to do.
Am I helping or not?
So more long term options.
And then something we originally conceived was to have multiple environments.
And we didn't have time to implement this.
But in different environments, you do slightly different things.
Like if you're in an office, you might want to install an AC unit, which is different than what you'd do outside.
So really look into multiple environments.
Different countries have different environments, might have slightly different strategies for what to do.
And we really would like to look into that.
So before we play the game, do you guys have any questions?
In that case, I'd like a user tester
So someone who hasn't played the game before, again one of our guests.
Alex?
Sure
Thank you
It's not showing up there.
Give me a second.
Display.
You're going to have to give me a minute.
I didn't realize it wouldn't show up
Make sure that's not plugged in
Yes.
I [INAUDIBLE] have it plugged in.
How do I mirror the screens?
Sorry, it just-- otherwise, he's going to have to stare up there.
Oh, thank you.
Don't look at my emails.
There's so many of them.
OK, yeah
Are we good?
Yeah, go for it.
My sound is on silent.
There
All right?
Let's see, solo, go [?
to tech ?]
the instructions so thanks for [INAUDIBLE].
All right.
Let me read through the instructions.
It's going to offer homeless people water, tell old people to go inside, install umbrellas, or many other options.
OK, so pause, click on Pause button.
OK. [INAUDIBLE] whoop
Sorry, what just happened?
I may have clicked that actually
Oh, did you start the game?
Here, we'll just quit.
No worries.
Great
OK, thanks.
OK, I think we're ready now.
Let's play.
I'm going for the umbrellas.
OK. Yeah.
All right.
Wow, that's nice.
Does it get darker as it goes through?
Cool.
Whoop, uh-oh.
Wake up.
Hey, OK. Should I do it around here?
Yeah, sorry.
Go through again.
Try talking to someone
All right.
Is it--
No, it's something long
Oh, OK. No worries
All right, that's good.
Thank you
Maybe, just start the game over
Yeah, should I just-- yeah.
It was working.
OK, let me refresh the page.
There
Oh, there you go
I don't know what happened.
Sorry about that
No worries.
Oh, I see, you can give them options to do.
Oh, OK, that changes it just a little, yeah
Kind of important, just a little
All right.
Oh, OK. Let's see, I'll talk to her.
Uh-oh.
Oh no
I actually caught the rest of the game
Thank you.
Any last questions for the group?
I liked the bird sounds and also the fire sounds.
Are there other sound effects?
So we had music.
But it was really hard to find good music for a heat wave, because you think it sounds hot and you don't really think of anything.
So we had this very eerie music, which didn't really fit either.
So we went for the minimalistic bird sounds, because we thought it was the only thing that kind of fit.
Is there any other sounds?
No.
Yeah, we had other sounds.
Just they sounded wrong.
We had fire sounds, like in the background and that made the whole thing sound really dangerous and terrifying, which is not what we want to portray.
We want to portray that you can do something and prevent anything bad from happening.
And any other questions?
Yes
So a lot of things have changed in your game since Monday.
A lot has changed since Monday.
How did you manage the process?
So we actually, we were going to have a meeting before Monday.
But really, our schedules didn't work out.
So we actually had, I told you, group meetings.
That's where we got everything done.
We had like a three, four-hour group meeting on Monday night, which we were planning to have.
And while there was like a couple core things that we wanted to fix, just like small like bugs that we were working on, and also while some people were doing that, we're just like, well, we have other people who aren't doing anything and can't really help with what's the major thing, so why don't we just like add other features, like it getting light and dark.
Because that's something we really had considered and we hadn't had time for.
But in that group meeting, where somebody had time, we were able to add that.
And it looks good
How did you test it?
Like, did you--
Oh, so we've been running it like nonstop all of yesterday and today.
But then it broke
Yep, definitely, it happens
Yeah
You also went from a screen that had pretty small characters on it to a screen where the characters were about 3/4 the size of the screen.
How did that decision come about?
So we had our drawings.
And when I told you we focused tested, then we had like a big picture of them on the screen.
People loved it.
But when we actually put them onto the old screen, they were really small and you couldn't see their characteristics.
And we said it's better to have people overlapping more often but actually being able to see their characteristics.
Because the caricatures are really where the learning comes in.
Like, if you didn't notice, the people who fainted were, I think, the drunk people and an old person.
So you start to notice as you play the game, OK, homeless people faint really fast if you don't give them water.
And that's just like a fact.
People who are homeless typically don't have enough fluids in them.
And they're also outside a lot.
So it's really hot.
So that's relearning comes in.
So we wanted to make those bigger.
And to make them bigger, we thought, OK, we'll just put in like a simpler one-lane background.
So that's where that came from.
We were planning on doing that before yesterday, I mean before Monday.
But we just didn't have the assets in yet.
So like a lot of things changed that were already in the works since you last saw it.
Cool.
Any other questions?
Well, thank you so much
All right, so we're Cholera Control.
And we're going to talk a little bit about our game and the making of it.
So firstly, Cholera Control is a game we made for the Red Cross.
Our client was Pablo Sanchez, as a few others have mentioned.
And the point that we wanted to get across with our cholera game is that it is an easily preventable disease, if you just like wash your hands.
It's really not that hard.
But people just don't seem to do it, because there's a lot of factors going on in their life and they sort of just forget.
But it's really easy to do, if you just wash your hands.
So we want to just do a quick demo of the game.
Where is-- OK.
So if someone would like to come down and play our game

Cool
Cholera is fun
Not supposed to show up on our home video, so that's
Are you hot?
No, I'm just kidding.
I don't know
We'll find out.
Don't have any policy for that
OK.
Play a game
Can you guys read that?
Yep
Can anyone in the back not read it?
Because, it--
It says cholera is spreading.
You got to control it and watch out for outbreaks
OK, I'll just-- Oh, OK.
I have $1,400?
So as you can see, the game is kind of a strategy game.
Right now, there's only one locality to deal with.
We sort of didn't want to throw too much at the player at the very beginning.
So you can kind of-- also, the game is paused, so it's going to open in that corner there.
And so you can have time to sort of just read, make sure you know what's going on, what all those different things do.
Then you go back to the map.
And time keeps elapsing.
And so over time, more localities appear, up to four.
So right now, you just get the second one.
They are on different branches of the river.
So they're not going to really hurt each other right now, still relatively easy
We go cheap
Hopefully.
I don't want to offend anyone, if they're like, this is really hard
So the average [INAUDIBLE] that the infection is still growing, more so
So one of the things you could see, if he goes back to the map, is all the bars above the villages, green is the healthy and red is infected.
And there's an arrow, which I think is a-- you can see it pretty well on there-- the arrow, if it's pointing that way, that means that the amount of green is growing.
So there's more people getting healed.
If it's pointing the other way, then more people are getting infected, which is bad.
So if you see your arrow switch directions, you usually want to get on that really quickly and make sure that nobody dies-- well, no, nobody gets infected.
Nobody dies in this game.
We didn't want to be too harsh
Are the products accumulated?
So my like--
Yes, yeah, so if you buy more than one, they work together
OK. Ew, boiling water
So boiling water is special, because it helps downstream villages
I clicked that
That doesn't help
I know.
Is my budget growing?
Yes
OK, oh great.
Oh, crap, I didn't mean to do that
You can undo
Oh, I can?
Yeah, well, now you can't
Now, I can't
That one's going to-- you're [INAUDIBLE]
OK, I need to buy them boiling water, and that's it
You're done

Well, thank you
Yep, thanks.
Yeah, so if you manage to survive longer-- no offense-- you get to-- up to four villages pop up, or four localities pops up.
And then it gets really hard.
Cool.
Cool, so first, we're going to talk about what went right, both from the creation of the game and working as a team sort of perspective
So firstly, I'll go over the iteration was a lot better on this project.
In this class, we had a total of four projects.
The first three were each over-- like, the first one was two weeks.
Second one was two weeks.
Third one was two weeks.
And then this one was eight weeks.
So this time, we had a lot more time to sort of decide what we wanted to do, what we should try out, and get to sort of test each and every single one of the ideas that we had, rather than just focusing on one idea of having to do it, because you only had two weeks.
The other thing is that after three projects, we finally figured out how to do design meetings really well.
We got them down to that they were only like a half hour each.
And when people showed up, they were really productive.
And so everyone figured out what they wanted to do.
They talked about the game.
It got done, and people got out of there.
It was very nice.
Another thing we did right was simplifying.
It was really hard to adjust scope.
It was easier in the other games, because you realize you only have two weeks.
When you have eight weeks, you think you can do a lot, and you really can't.
You think you have a lot of time, but classes and applying to jobs, and everything takes all of your time.
And so you have to really learn to, OK, like, I only have so much time for this class.
I can only do so much on this game, so I'm going to adjust the scope.
And it was really hard even when the client, when Pablo told us, you should change what you're doing, we didn't want to.
We thought, no, this is a great idea.
This is how it should work.
But then when the client tells you do it and all your play testers say your idea is bad, you need to just give in and change.
And we ended up having really great results.
With the simpler gameplay, it's a lot more fun.
And the options, there are less options, but they're much more meaningful.
As you could see, the player was sort of reading through each and every option.
And it's because they each do something different.
Before, we had a lot more options.
But they were all sort of the same.
You just kind of click them, and they do something.
But they're all just about the same effect.
With this, they all have different effects.
So you sort of have to strategize.
Do I want to spend more money for this effect?
Or can I just use this cheaper effect over and over again for the same result?
So I'm going to talk a little bit about what changed from beginning to end.
So the UI in the beginning looks much uglier.
I mean, that just came with sort of refining that.
I want to talk a lot about sort of the design changes that really changed.
So in our first prototype of the game, you sort of click these buttons here on the right.
And that would give you the actions.
Whereas, in the final version of the game, you click on the villages, and that gives you the actions.
And this came from something that Pablo told us early on and really made a big difference, which is like people that are used to playing video games get this concept of click action, apply to object.
But we realized, both through play testing and again Pablo telling us, that when you don't have that sort of experience, working in a lot of videogames and just having that context in your head, that doesn't really make sense.
It makes much more sense to the people that we're aiming this game for to click on the object and then choose an action.
So like that inverting of the order of things actually makes a big difference in helping to understand the game and just making the whole thing smoother.
On top of that, we sort of like really refined the information that we're showing when you look at the map.
So right now, like in the original version, you look at the map and you sort of don't see straight away everything you need to see.
You need to go to each of the villages individually to know what's going on.
You sort of can only see where each village is, or where each locality is and how much health it has.
Whereas in the final version, we sort of give you every you need to see.
And it took a long time to figure out how we wanted to do this UI.
Like, how do we show what prevention measures are in place?
How do we show whether the infection is growing or decreasing?
And how do we show which way the river is flowing?
All of these questions were really important and really led into the final design.
Secondly, about like options, at first we had lots of options, too many.
People did not even want to figure out what each of them did.
It was too much information thrown at once.
And although we thought that having lots of options would lead to meaningful-- like, oh, well, you can make these advanced strategies.
It really didn't, because people didn't even want to bother learning all that.
Whereas, in the final version, having fewer options made it much easier or like much less scary when you opened up the list of options, the actions that you have available to you as a player.
And it was much more accessible, while still making the strategies diverse.
So now, we're going to talk a little bit about what went wrong in the process of making the game.
So working together is really hard, because people are stepping over their toes.
Oh, what do I do now.
Oh, I can't touch that, because you're touching that.
Like, all these sort of problem arise.
And so at first, we really tried to avoid that.
But then we realized working apart is even harder than that.
Initially, you think, well, if we're each working on really sort of independent parts and we bring it all together in the end, that'll be happy.
Everyone will be happy.
We're not stepping on our toes.
We can each work on our own time and that sort of thing.
But that turns out not to work well, because people do things differently.
People follow-- like, even simple things, like coding style and how you name your variables and all that sort of stuff makes a difference.
And if you're not working together closely, and if you sort of working apart, bringing that together was a huge hassle at the end.
Like we spent a lot of time just making sure everything, all the components that we had made as individuals worked together, just to get the game running
So another thing that went wrong was the motivation for this project.
At the very beginning, the team wasn't very passionate about the project to begin with, which is really hard.
It's really hard to work on something that you don't really love and want to do.
And I thought, you know, maybe this will just get better.
People will start to have ideas.
They'll feel ownership of the project.
And then they'll be more motivated to work on it.
But that's not true.
People, if they start off in a bad place, they're not going to get any better.
You really have to focus on somehow helping people to either learn to love the project or learn to just be able to work on it.
And this can be done through different ways.
The easiest way is just to get people to realize like we have to do this, so you have to work on it.
And I found it as-- towards the end of the semester, it took me to figure this one out.
But if you give people vague instructions, they will wait until the very last second.
But if you make it crystal clear, exactly what you want them to do and give them is as detailed instructions you can, it's much more likely that it'll get done in a timely manner.
And also, we felt really restricted by the topic and the requirements, which was another just problem in motivation.
And so crunch is something I was sort of alluding to, is that when you have the instructions and you don't want to do something, you will put it off.
I do it.
I am sure everyone else does it that I don't want to do this.
I don't understand what they want me to do, so I'm just going to wait until the last possible second when I have to do it or I'm going to fail.
And that's when it gets done, which is what I think was a leading thing to our crunch.
And so we had constant crunching.
Every time we knew we had to play test, the night before, everyone was up working.
And there was a huge crunch after Thanksgiving.
We took until Thanksgiving, and then everyone realized we don't have that much for longer in this semester.
We only have like a week and three days before this final presentation.
So there was a lot of crunch there.
And the work was low priority because of other classes, which is a really unique problem to doing a school project, as opposed to doing a project in the real world, where people still might have other things to do, but this is their job.
Whereas, when this is a class, you can't really fire someone from your class.
Let's see, you want to talk about what we learned?
Yeah.
So what we learned in terms of game design, at least for me and probably for everyone on the team, what was most surprising is that giving more choices and making a game more complicated does not make it more fun.
Initially I thought, more choices means more strategic diversity or whatever, and that'll make everything more exciting.
And every time you play the game, you can try something different.
But that's not really true.
Having less choices can be even more fun than having more choices, because then you don't have-- like when the player doesn't have to think about as much at once, they can really go deeper into what the current options are.
So that was really surprising.
Also in terms of game design, requirements can be restrictive and often you can't do what you want to do.
You as a designer can't do what you want to do, can't fulfill your own fantasy for what the game is opposed to look like, because of the requirements, because of what clients want, and because of what players want.
And you have to learn to sort of deal with that.
The designer is not always right.
Just because I have an idea for what the game is supposed to look like, if I test on 10 people and all 10 people say I don't like it, I'd prefer it to be this way, then I'm wrong.
I have to change that.
And that's something that's hard to deal with, but it's something you have to learn to do.
Yeah, we learned a lot about cholera.
Basically, as we said in one of first slides, washing hands is overpowered in real life.
If you just wash-- very simple things can prevent cholera.
And that's something we wanted to get across in this game.
In the game, washing hands is overpowered, just so that they get that idea across.
And about coding, there's a lot of simple things you can do to really polish up the game, like tweening, which is a animation API available, and Phaser that we used.
But really any animation API that does like little things, like having the window pop up instead of just appear, having things go like "paching!"
and move in and out, that makes a huge difference for very little effort.
And it makes the game look much more polished.
In terms of future changes for the game itself, we initially planned to have seasons and lots of random events that would sort of make the game more exciting, especially near the end, once all four villages are up and you sort of get into this pattern.
But we basically didn't have time to do that.
And then the second thing that at least I personally wanted to add to the game was more advanced infection models.
It turns out that modeling infection is-- well, it's not surprising really.
But modeling infection is really hard.
Balancing your model of infection is really, really hard.
And it's sort of almost impossible to make it have this sort of smooth curve, where it starts out easy and then gets sort of hard at the end but not too hard.
Infection is exponential in nature.
And that's really hard to deal with.
If I had more time to work on it, I'd probably spend a lot more time thinking about different models to use for infection and make it easier to balance.
Like right now, you change like one little number by hardly anything, and the game goes from really, really easy to completely impossible, just because of the nature of infection
And lastly, future changes for the class.
One of things that we all felt would be really useful is if the professors had enforced the sprint task list, because you could submit your sprint task list at the beginning of your week.
But you never had to tell them if you actually did everything.
You just had another sprint task list, which maybe if that it was looked at carefully, there might be the exact same things two weeks in a row.
But a lot of times, you never heard anything about your progress.
You just kind of had your progress.
And it wasn't until the end that we really realized like, oh, we're kind of far behind.
We need to fix that.
So it would have been nice earlier in the project to have that sort of feedback, like, hey, you should be-- are these things done?
You should be doing more.
And more freedom in choosing the topic, like we mentioned before, we felt really restricted.
There were things we wanted to do that we couldn't, because of the confines of this project.
And so a little more freedom would've been nice.
But you know, it's a class.
So finally, does anyone have any questions or comments, feedback?
OK. What would you do if-- what would your team have done if they were able to choose a topic or have a little bit more freedom?
Was there anything in particular you talked about as a team?
Or when you're talking about things you wanted to do but the client said no
Humor
Yeah, in terms of like if we were doing about cholera, things we wanted to do about cholera but we couldn't, like death is not in the game.
We can't make it like overly morbid or anything, not that we-- But there are certain things, like we wanted to have more options.
But since the game is sort of oriented towards-- or rather, the options have to be oriented towards what an individual can do.
So the game is supposed to teach kid to young adult as an individual how can I prevent cholera, which is like you wash your hands.
You boil your water, that sort of thing.
But things like put basically bathrooms, the sorts of things that a government or the group of people as a whole could do, but an individual can't really do, were options that we wanted in the game, and could have sort of interesting mechanical effects within the game, we couldn't do, because it doesn't make sense.
I can't teach you to make bathrooms.
You can't, not really, as an individual
Yeah, and sort of as Jen shouted out, tongue-in-cheek humor.
There was a lot of time during the game where we're like, oh, we should have this happen, but it was offensive.
It was funny.
It would be funny to an American audience.
But because it was for an audience in Ghana, we changed the-- we didn't know whether to use village or town or city.
And so we asked Pablo.
And he said use the word locality, which to everyone playing it, play testers, we're like this is a really weird word.
Why does it say locality?
And even we forget to use the word locality.
And we call it a village accidentally.
And so we'd want a little bit more freedom there and sort of make it maybe a more humorous game.
But we didn't really-- we were really afraid of being offensive.
That was like one of the first things we talked about when we started making the game is how do we make sure we don't offend anybody accidentally
Any other questions?
We're done?
All right
Thank you
Thank you
Thanks
We're the Saving Gora Gora team.
I'm Liz, Rachel, Kevin, Justin.
There are others in the audience.
Go to the beginning.
Yeah.
So to start us off, I'm going to talk about, let's see, the goals that we had from the beginning.
Our product is about cholera and cholera prevention.
And our audience was Ghanaian children.
So we had to find a way to connect Ghanaian children to this information about cholera prevention, positive behaviors, proactive things they can do in the community, while also having a fun game that they can relate to.
What this ultimately meant is that we had to do a lot of research, because we didn't have a direct connection with our audience.
And we couldn't directly play test with children from Ghana, ages 8 to 13.
We had to make a lot of assumptions, which ended up being wrong after we talked to Pablo, specifically about them.
So we did some more research.
We looked at Ghana, cartoons targeted towards children in Ghana.
And from those cartoons, we looked at the color palettes, some behaviors within the cartoon specifically, and names and sort of character designs from the cartoons.
And we got Ghana research from documentaries and stuff, also informed decisions that we used to change minigames, which we'll talk about later in the presentation.
A quick overview of the narrative-- you start the game.
There is this monster hiding in a bush.
You go to check him out.
It turns out that he's being persecuted, because everybody in Gora Gora is sick.
And they think he's the cause.
And Sal is actually a cool guy.
You're a bunny, and your name is Kojo, and you're a cool guy, too.
So you're asked to prove that he's innocent.
And you take that up, because you're an awesome person.
The gameplay ends up being that you explore the town, looking for ways to prove that Sal is innocent.
And you do that by unlocking these minigames and getting clues from the minigames after you defeat the minigames.
And after each minigame, you can go talk to the mayor, or you have the choice to, because he has an office.
And when you have all the clues, you finally convince the mayor that Sal is harmless, which is great
So our first minigame dealt with water filtration.
And the concept behind it is that friends and neighbors are bringing containers of water to Kojo's mom.
And they want to see whether the water is safe to drink, or whether they have to purify it beforehand by boiling.
So that, and when we initially designed our game, it turned out to be water filtration kind of factoring, in which bottles of water were going down a conveyor belt.
And you were asked to remove the bottles that were unpurified and that needed to be cleaned.
So as you see on the left, that was our very first iteration.
And I guess the issue with that, as Pablo brought up to us, was that it wasn't relatable to the kids that we were trying to target, because these kids likely won't ever see a facility that has these conveyor belts, that does this kind of purification.
So we had to go through several iterations to figure out a way to make the game a lot more relatable.
And our final product is on the left-- or on the right, sorry.
And as you can see, it's outside one of the homes in the village.
And your mom is just standing behind a pot of boiling water.
And now, you just have plain containers, bowls, buckets, tanks.
And your goal is to kind of see where the water is coming from, how warm it is, and try to figure out whether you want to boil it or not.
So some challenges we faced were, in our first iteration, it was more of a real time game play.
You were kind of rushing against time to pull off all the unfiltered water, as all the bottles continued to scroll to the right, regardless of how fast you were going.
But then we realized that took away kind of from the educational aspect, because it forced the player to just kind of hurriedly just pull everything that they can.
So instead, we changed it to a styled gameplay that was kind of at the player's pace.
They can do everything as fast or as slow as they want.
And that, I think, gives the player a better chance to kind of learn everything there is to learn from the minigame
Cool, so the second minigame is a water collection game, where you're asked to help your friend [?
Korku ?]
find where to collect water from.
And so with this development process, it was kind of similar to what Kevin was saying about making it, like have as high of a learning value as possible.
And then another aspect of this game was making it fail-safe.
So obviously, we thought there was one right answer that the player should take away from it.
But we didn't want them to fail and then not win the game at the end, if they were to not choose the right location.
So one way we dealt with that was, every time the player chose a wrong location, there'd be immediate feedback, saying, oh, this isn't such a good place to collect water from because.
And so as you can see, this example is, if you were to collect water from the river, you don't know if there's excreta from upstream or if someone's peeing in the river right now.
So then it'll ask you to try another place.
We also did a feature cut pretty early on, which was also finding someone to dispose of your excreta.
And so we focused on the water, which would hopefully focus the player's attention in figuring out where to place water and really think about the water aspect and not so much as another aspect of the game.
So the other challenge was, as Liz and Kevin already pointed out, was there's a huge cultural difference.
So again, I have like a before and after shot of what our landscape looked like.
And so originally, there was snow-capped mountains, a lot of grass.
But from our research, we saw that this didn't really look like Ghana landscape.
We had a water treatment facility, which originally was the correct place to collect water from.
But we eventually changed that to be a well.
And so that's more realistic in terms of where people collect water in Ghana.
Additionally, at the end there is a task bar, so it provides you more feedback.
If you clicked on any of the buildings it'll provide you feedback about what that place is, and whether or not you should or shouldn't be placing water there
Our final minigame is focused around teaching the symptoms of cholera and the importance of early prevention and recognizing the early signs of cholera to prevent the disease from spreading.
So the premise of this game is that the doctor in the town is very busy.
He can't go out into the town to actually check on patients to see if they're seeing any of the early signs of cholera.
So he asks you to help him by going out and talking to some of your friends in town, and then asking them if they're having any of the symptoms, and then ask them to come in to the doctor.
We initially started off by having this game be a game where you help the doctor in the doctor's office.
And he's diagnosing patients, and you're sort of diagnosing the patients with him.
And we realized that this would quickly turn into sort of a quiz-type of game, and it would really detract from the learning aspect.
It would essentially the same as just reading the systems out of a pamphlet.
So we decided that by adding more interactivity by forcing the player to go out into the village and actually find people and talk to them and then ask them to come to the clinic would be a lot more engaging, while still maintaining the same educational aspect about trying to search for the right symptoms.
Additionally, we made a change where we had the player go talk to some of the villagers that were adults.
And then we realized that-- after speaking with Pablo, that the social hierarchy in Ghana is not necessarily that of the United States.
And this sort of interaction, where a child asks something or sort of demands something of an adult, isn't really something that would happen, just because of the stricter social hierarchy.
So we decided that it'd be more appropriate if we had the child go talk to some of his friends that were of the same age.
And then tell them, since they're on the same sort of level, to go to the doctor's office, rather than talking to elders.
Some of the challenges that we saw were that this game is heavily dialogue-based.
It's almost entirely interacting through our dialogue UI.
And so we needed to do a lot of iteration on that in order to make sure that it was clear who was speaking and give the appropriate choices so that the player can make meaningful decisions while having his interactions with the villagers.
And finally, since it is so dialogue-heavy, there's a problem that comes when making a game for children when you have a lot of text, because they might not be the strongest reader.
So we focused w lot on making sure that the vocabulary was simple, and we tried to keep the sentence structure as simple as possible, and straightforward, to get the message across to children.
So some of the overall challenges that we faced while doing the game development was mastering the states with Phaser.
This is sort of an intricacy of the Phaser engine itself, where when you spawn off the different stages of the map, making sure that we can maintain the data across all these different states, so that there can be a sort of progress through the game.
And finally, we want to make sure that the game wasn't too difficult.
For example, with the facility game at the beginning when we had the very fast-paced clicking, and then you would fail almost all the time.
We wanted to make sure that it wasn't too difficult, and that it was accessible.
The cultural disconnect we touched a bit on in some of our previous games, making sure that the target audience will be able to play this game and relate to it in order to pull the message from it, and making the play meaningful.
We wanted to make sure that the game play was still getting the message across, and finding a good balance between education and play
So now I'll talk about how our teamwork worked out, and what our process like as a team.
So we used four main tools.
The first one being Trello, to keep track of our tasks and progresses.
So, we'd put all our sprint task lists, our product backlog, on to the Trello.
People would claim the different tasks, and then be able to say if they were doing them, if they were done, or if they haven't started.
And then that means we could track how people's progress was.
The second one was weekly meetings.
So, every week we had a meeting to either do some research about Ghana, talk about how we could improve the game play and make it more meaningful, if we should cut some features, and whatnot.
So, these meetings were really great for quick and efficient decisions, and also really important decisions.
Our third tool was Slack, which some other teams talked about.
But it was really great because, first, they had Google Drive integration, so you could see when people were pushing code if you should be pulling code.
And then, in addition, we could have different channels.
So we had a separate coding channel, where all the coders could talk to each other, and that might not be relevant to other people.
And then having a whole team channel was also efficient, because it would be faster response than email.
And then our final one was Google Drive, which is where we have all our assets, had all our dialogues, our characters.
And this worked out to be a really great tool for collaborative storage.
Some downfalls of this was-- I'll talk about the second part, was basically with all these tools you need to make sure that everyone's actually using the tools.
It's not actually effective to be using Trello if not everyone's using Trello.
And we found this to be the case, not everyone would be updating their progress on Trello, or not everyone would be on Slack .
But some mitigations we had to that is, in like Slack, you can mention people's names, and then it'll send them an email if they're not actually on Slack at that time.
We did have to remind people to be updating the Trello board or updating their progress in Slack, or just ping people individually in email.
But I think we worked it out.
Something we could have improved on was enforcing deadlines, and having deadlines.
As Kevin will talk about, we had different roles between coding and art designing.
So, obviously the art assets had to be done before you could really code things and put it into the game.
So in that aspect, we should have probably had clearer deadlines so that we could stick to a stricter schedule.
And then, in addition, we should have probably had a bit more frequent communication with the entire team.
So we did have a lot of communication, but sometimes not all the team members were present, and we should have probably improved a bit about sending out meeting notes and catching up people who weren't at the meetings when we made certain decisions.
And to that regard, something that we could just do for next time is, send out meeting notes, and keep everyone up to date, and possibly even have smaller meetings where not everyone is present.
So I guess in terms of what went right, Kevin will talk about that
So, as Rachel was saying, we separated a lot of the tasks of the project between the team members.
And since we had seven people, we designated four to coding, two to design and art, and then a few others to the other aspects such as narrative, project management, and sound.
So, that actually helped us a lot, in that having only four coders rather than seven really cut down on the number of merge conflicts and issues that we had in previous projects.
Because, obviously, the fewer number of people working on each file, the less likely it's all going to break when people merge their stuff together in GitHub.
And with the art designers, it actually helped to have just two.
Because, although they had to do more work, it was easier for them to collaborate and make sure that all of their art followed the same theme, rather than coming from three or four different sources that look slightly different.
Eduardo and Liz were very capable in making sure that all the art had the same kind of color palate, and the same kind of cultural theme to it, which helped, I think, unify our game in the end.
And, as Rachel also said, we had semi-frequent meetings, mostly at the end of the weeks, just to catch up and see how everybody was doing on their parts, and what we were planning on doing for the next week.
And it also served as a good time for us to really sit down and get some work done, whether it's drawing some new assets, coding a new minigame.
And having most of the people there together at the same time allowed us to really bounce ideas off each other when we were implementing this, which made things a lot more efficient.
And lastly, we realize that at the beginning, we had probably more than we could take on over this eight-week period.
So we did a good job of cutting features early.
And the main one we did was cutting the fourth minigame out of our game entirely, so that we could really focus on making the three minigames that we have currently really robust and really relevant to the ideas we were trying to convey
So moving forward from this where hopefully we would be able to find more of our target audience, our focus tests helped a lot whether it was speaking to Pablo, other MIT students, people in the class.
They gave us a lot of good feedback about the game.
But obviously we didn't get a chance to speak with Ghanaian kids, which is our target audience.
And that could definitely help if we had a longer period and were able to find even kids ages 8 to 13.
In terms of iteration, I think we could still focus a bit more on balancing the difficulty.
And then we mitigated that by doing things like what Justin said about changing the text dialogue, and also providing a lot of help tutorials or instructions throughout the game.
But that's something we could always think about, so that even kids younger than our target audience could access the game.
And then, obviously, just making sure the game is culturally relevant, thinking about word choices, thinking about assets, something we could always improve on.
And then, finally, to actually keep moving forward we could implement another minigame, find out other things about cholera we really want to get across, and add other adventures to the game.
But the game right now is pretty contained, and gives, we think, three valuable lessons about cholera, and three very concrete preventative actions that you can take, and bring back to your village and your family.
So I guess that's what we would think about if we were going to go for it.
And now, we'll move into our demo.
So, I guess, if there's any questions first we can take those, or else we'll be looking for a demo-er
I have a question regarding how you used like the structure of like having the many minigames inside of a big game.
I wonder like does that help splitting the tasks up among different members in the team and make the process better?
Or did that add more difficulty to finish all the contents and correctly tie them together?
Yes, really, I think the modularity really helps, so the three of us took the lead on one minigame each.
But then obviously there were components that cross all the different minigames.
So, for example, our other coder worked a lot on the conversation and dialogue UI that shows up in all the minigames.
And additionally, at the beginning we did run into an issue where we thought, oh, really the game is just three minigames.
So we worked to come up with an idea to tie everything together.
And that ended up being an inventory clue kind of thing.
So you could go around town looking for clues that you needed to unlock the minigames.
So, for example, to play the water collection game you need a bowl to collect the water.
So you would go around town and find someone-- maybe someone was like wearing a bucket on their head.
And you'd be able to collect that bucket from them and then play the game.
So, in that sense, we were able to come up with this idea that could tie together all the different minigames.
Does that answer your question?
Is its mother a cat?
Yes, [?
Bunnykins' ?]
mother is a cat
All right.
[INAUDIBLE] [LAUGHTER]
So, you cut the minigame number four?
Yes
About when in the project did you do that?
And about how much work had you actually put in to that minigame before you decided not to complete it?
That game did not make it past the paper prototype.
After we did the paper prototype, we realized that, that game and the symptoms game were actually very similar.
And so we [?
introduced ?]
some or the storytelling aspects into the doctor minigame that we pulled from the fourth minigame.
PROFESSOR 1: Thank you.
[APPLAUSE] [INAUDIBLE] will volunteer.
Thank you.
[LAUGHTER]
I sure helped!
[LAUGHTER] From our previous discussion, I wonder if I can take his bucket
[INAUDIBLE] like, go closer
Oh, gosh.
I don't know if [?
Korku's ?]
is going to get that bucket.
Maybe I'll keep it.
Wow, I already have two quests.
[LAUGHTER] [LAUGHTER] Never play with fire by yourself.
[LAUGHTER] Whoa, look it's turned into a cat.
[LAUGHTER] Ah, cool.
I mean, if I say no, I should come back later, don't I?
[LAUGHTER] Thank the bowl, and click me when you're done.
All right.
Well, I know where to go, but we'll do this anyway.
There's no water there!
I guess that makes sense.
If there was water in the house, I wouldn't need the bucket.
I do all the wrong things first.
It looks like there are a lot of bacteria in the lake.
I have really good eyes.
[LAUGHTER] You-- I don't want rusty water.
Oh, the bathroom, that's a great place to get water.
I won't go there.
Yay!
I have water.
Can I go swimming?
Yes, yes, I can.
I forget what you wanted.
What did you want again?
Paper.
I know-- no, I don't actually.
Oh, I guess I do know something.
Ah ha!
[LAUGHTER] I guess I need to go in the houses now.
That's my guess.
Oh, cool.
You-- I better boil that
You might want to start the game first
Oh, start the game.
I'll do that first.
I like this water.
This is great.
I just know that the well is good, because I was just there.
Pump and clear, well, rust was the problem before, so where am I getting this hot water from the pump?
Discolored.
Discolored.
Well, I am going to be safe, because I don't have to gather firewood.
I'm running out of firewood.
Now I'm going to be in trouble here.
We'll go for it.
It's from the pump, how bad could it be?
[LAUGHTER]
Is this the rusty pump?
[LAUGHTER]
Oh, right.
That's the bad stuff.
The well's the good stuff.
Oh, that's [INAUDIBLE] pump.
Ah, whatever
It is the cholera that you've got to test in this game, right?
Hm, I only have three firewood.
Let's just do this.
One of the next one will be clear, I'm sure.
Well and clear.
There we go.
We'll boil the rest to be safe.
All right.
Throwing up and have diarrhea and very thirsty.
Oh.
So in other words, you've seen the monster.
You should go see the doctor, because you have all those symptoms.
All right.
Where are my check marks?
Oh, look I got three clues.
Is that good enough?
Let's go out to mayor.
I don't have any more spaces for clues, it must be good enough.
Nope
[INAUDIBLE] [LAUGHTER]
Yay!
I just jumped over that office
I guess the repetition's not a bad thing, is it, for your demographic?
That's good.
[LAUGHTER] PROFESSOR 1: That's it.
Thank you
I think that's it, that's all we have.
Are there any last questions?
PROFESSOR 2: I actually do have one question about the music.
Because you have it on the credits.
Is this like a piece that the person wrote for you?
Or is this something that you downloaded from the website?
We downloaded it.
PROFESSOR 2: You might just want to check their website to see exactly how they want to be credited.
[INAUDIBLE] title of the song they want used, or something like that.
But if you've already done that, that's good.
PROFESSOR 1: Great.
Thank you
Thank you.
[APPLAUSE] PROFESSOR 1: All right.
We're done!
Thank you so much.
[APPLAUSE] We really enjoy teaching this class every year.
We really hope you got a good experience for it.
I liked seeing a lot of the presentations, and I liked the changes that we saw from rehearsals to today.
So there was a lot of really good information in there, both in the games and in the presentations.
Again, if you have any questions about future classes-- we don't actually offer a game design degree at MIT, but we do have a concentration of comparative media studies.
If you take any four CMS classes, you get a concentration.
It can be from this list here.
There's information on your desks about other things you can do with games at MIT.
Course six students, if you're interested in doing UAPs, if you haven't thought about that, start thinking about that now.
We could help with mentoring and advising on UAP projects and other kinds of research.
And I'd like to give a thanks to the instructors, Phillip, Sara, Andrew, for all your help with running the class this year.
So, thanks again
Thanks, Rick.
[APPLAUSE]
I'm Lennie Martinez.
I'm a sophomore in AeroAstro, actually.
I'm part of the Saving Gora Gora group.
Our game is called Saving Gora Gora.
It's a game about-- well it has to do with tropical cholera.
And our goal was to teach children how to prevent cholera, because our target audience were Gonian children ages 8 to 13, approximately.
And pretty much our main goal was just to educate them in a fun way that wouldn't seem like we're educating them.
Oh, OK.
So our design brief was pretty general.
It said, look at the cholera tool kit that's online, that you find online, and try to create a game to teach kids about cholera.
So they gave us our target audience, which was Gonian children ages 8 to 13.
And we we're thinking, originally-- well, it was an idea that we had based off Animal Crossing, where it's a bunch of different mini games.
So each mini game would include a small lesson.
And then, they would all come together to become one large MEDI PUZZLE.
And in solving the MEDI PUZZLE, the players would learn how to, I guess, prevent cholera or counter cholera, and that was what led most of our design process.
The visual style came about- well, we had a very-- well, we had an original visual silo that was very different from what you guys saw in the presentation.
We had to research-- we had done research on children, but we had to take a look back and redo everything because it wasn't as-- once we compared it, once we had a finished build, we compared it to-- we had to do research on what cartoons would better look like.
And we found that it was very different and very clashing.
So we did some research some more on Gonian culture, and we tried to find, make it fit better.
And so we went for more artsy, woodsy, cartoonish little people, very little kid, but it's still not too childish for other people.
I originally, really, was interested in the class because one of the textbooks was Design of Everyday Things.
It's a book I find really interesting.
I've been trying to read it, but it's hard with the curriculum, the overall MIT curriculum.
So, I really came into the class thinking that I really wanted to work on the design aspect of things.
But as we started working on projects, the match win side became a bit more interesting, a bit more, something I started working a bit more because it was more accessible for me with my not so robust knowledge of coding.
But then, later on for this project particular, I took the role of the writer.
So I was in charge of writing most of the dialogue, and that was also very interesting.
So I feel like I gained a lot more skills than I thought I would have.
I think a really big part of the process is to start writing really early.
The writing, we left it off a bit until we had the general mini games and used the writing to tie everything back together.
But I think it would have been a much more smoother process or a much more interesting process if we had started the writing, had we used the writing to lead the design process.
But it wasn't, I mean, it would have been better had we started early, but it wasn't too bad or too much of a difference between the work we have now and the work we had before.
But aside from that, I think it's really important to start and meet early and sort of come to a consensus on what you want to do.
We had a lot of mini-- the reason we-- we had a lot more mini games to begin with than when we ended, and it was good that we met early and decided what to cut and what to include.
And that helped us work towards one common path.
Working with the client was very interesting.
We met him two, three, four, or five maybe, almost five times, I think it was.
We've met him a lot and interacting with him was interesting.
The first time we met him was where he did a lot of demonstrations was, sort of, I guess, showed us more what we were supposed to be aiming for, because it was a very social interaction, very deep interaction.
And that I think the goal of our games is to be a bit more, is to be really deep while also appearing to be very just playful.
And he also helped us when we were going on our art reworks, that he was suggesting, like, maybe you should look at these.
You should look more at towards this type of the culture, this part of the culture or this part.
And that was very nice.
That's the sort of interacting.
The class has been actually really fun.
It's a lot of work.
But it's a lot of interesting work, because once you see the game-- I actually hadn't seen the final builds we had until today, and it was like, this actually is really-- I had written the dialogue separately, so I was just looking at the art assets.
And it was very nice to see everything come together in one nice build of, here's the game you helped make.
I'm Matt Susskind.
And for this game, we are a team of eight people working on forecast-based financing, which is a technique using forecasts in order to make decisions about what to do in response to incoming disasters.
Currently, a lot of techniques are we're going to build long-term protection for cities in terms of dams or disaster preparedness.
Or, after disaster happens, we will react to the disaster.
But there's not a lot of work done in terms of we see that there is a 50% chance of disaster within the next week.
Let's start mobilizing to prepare for that.
And part of the reason for that is because if the disaster doesn't happen you end up wasting money.
But if you actually do an analysis, you'll realize that the money that you waste is actually way less expensive than trying to make up for your lack of preparedness after the fact.
So Paolo had come in from the Red Cross Red Crescent Climate Center.
And he pitched a couple of different concepts that they wanted games around in order to educate either members of the Red Cross or people out in the world who they wanted to show these different ideas to.
And so what he showed us was actually just a PowerPoint with a couple of different slides, where on each slide was a different concept.
Ours was forecast-based financing.
But then, there were also ones like cholera, or Ebola, or preparing for heat waves, things like that.
And so he gave us a quick pitch of this is what the concept is, this is who we want to educate with this, and these are some of the characteristics that we think are really important.
And we actually weren't assigned to any topics.
He put those topics up on the board and let teams form around them, depending on what you're interested in.
And so beyond just the idea of this is forecast-based financing, and this is how it works and who we want to teach it to, we got no further instructions.
Paolo did come in throughout the class to teach us a little bit more about who we would be talking to, or to work over the ideas that we had been exploring.
But on the whole, we just got the general topic, and came up with the game ideas on our own.
One of our biggest challenges was with forecast-based financing, it's an understandable topic, but it's sort of abstract.
And it doesn't quite lend itself to a game.
It's very easy to come up with we're going to take forecast-based financing and hide it as a game.
But that's not a fun game to play.
So a lot of our challenge is figuring out how are we going to work?
How are we going to create a game around forecast-based financing that explains the ideas that we want, that teaches people something new, and is a game that we're actually proud of?
Another compounding factor was that originally our target audience was policymakers, which was a very difficult audience, because they don't generally play games, which meant that we were starting with trying to teach them everything from scratch.
Whereas when you're creating for somebody who has experience with video games, you can take a lot of things for granted-- that they're going to know how to drag and drop things in order to change things, or what the various design ideas you're using actually mean to them.
So that was a big challenge for us, was designing for this new consumer.
And we also had a lot of design challenges with getting the forecast-based financing game to work.
And the way that we approached it at first was by creating a whole bunch of different prototypes of lot of different types of games, and trying them out with people, seeing what they understood, what they liked about them, what didn't work, what they learned from them, which was really good in terms of exploring different ideas and learning more about the subject area.
But it created a new difficulty in that we were all over the place for the first couple weeks of the project.
We got to the point about halfway through-- we had about eight weeks to do this project, I think, and halfway through, we still didn't end up with our final idea yet.
Instead, we were exploring all these different ideas, and pulling in various parts of them in order to make our game.
And then, we realized that our game lacked a coherent vision.
We had been working on a couple prototypes.
We didn't like any of them.
It didn't feel like players had a connection to them, both in terms of what was happening in the game, but also a mental connection of I understand exactly what's happening here.
So we decided to take everything we had learned and some of the systems we had built, and package them into a new UI, which was this side view of the rising water, which everybody just understood intuitively.
You see a village.
You see someone standing there.
You see the water rising toward it.
And it doesn't matter who you are.
You understand that this is something that's in danger.
And you're going to need to do something about the rising waters.
And once we hit on that idea, everything became much easier.
Of course, there were still ideas we had to explain, concepts we had to design into the game in an understandable way.
But for the most part, the game was actually there.
The mechanics themselves didn't change very much between the prototype we had before and this idea of the rising waters.
But just packaging it in a new design changed the game, the feel of it, completely.
So I had game design experience beforehand.
So I like game design a lot.
I was hoping to work more on that.
But also to get some interesting games made.
I think I did get some of that.
But probably the thing I learned the most that I didn't expect coming in was just how to work with a team.
We're all busy MIT students.
And so we would have weeks where all of a sudden, everybody would have a test.
And we had no time to do anything else.
And so it was a lot of learning how to work with people who are extremely busy and who have a lot on their plate.
And so it's really easy for things to slip, for things to get forgotten, fall through the cracks.
So it taught me a lot about project management, basically, and solving problems as a team, and learning how to get everybody-- I don't want to say herding cats, because it's not like people didn't care.
But it's just that we were so busy.
It taught me how to work with these busy people to get to where we need to be.
My name is Miriam Prosnitz, and I was on Team Heatwave.
You play and there are characters who move around.
And different people who are at risk for being vulnerable to heatwaves move around the screen.
So one person was in charge of researching who are those people?
And another person was in charge of OK, I'm going to make a character object.
And we split up all of those, actually, right at the beginning so that we knew who to contact about what.
And then, based off of that, we would come together and say, what needs to be done, and whose job is it, and is that still balanced in terms of who's working on what?
And it worked out really well actually.
Actually, funny enough, I took this course solely because I had no computer science courses this semester.
Because I was abroad last year, I did a full year of engineering coursework.
And I was like, don't need any more requirements, might as well just take one for fun.
So here I am in Creating Video Games.
And so my goal was to write code, and the funniest thing-- and I'm not representative of my team at all-- I'm like the one person in this class who has not written a single line of code this entire semester, because I ended up being project manager a lot.
And so what I learned is a lot of the process-- a lot of making things work, is really just emailing people, calling them on the phone, being like, are you supposed to be at this meeting right now, going and buying snacks for a meeting.
You think it's like silly, you know, nobody really needs to get snacks.
But I've been told by my teammates that if I hadn't gotten snacks, they wouldn't have come.
So-- I learned a lot about the project management and the work that goes on behind the scenes, which I've seen before, but I just didn't realize the scale of it in games.
Something I would give feedback about this class is I really like the fast pace of the first three projects.
And this project, because it was so much longer time, the team was so much larger, and we had this kind of set topic we had to deal with, I didn't enjoy it as much.
As I mentioned, one of the main issues for our team was passion.
At the end of the day, I can summarize what you should do in a heatwave in like two sentences.
That doesn't make for a very interesting game.
So that's something I would say, if I was to do it again, I would've played the game where you play with the heatwave, and you try to attack people, because that sounds like more fun.
And I think that's the other thing I learned about games-- that you really have to care about what you're making.
Somebody mentioned in one of the presentations that for the last project we would submit these sprint task lists, and they wouldn't necessarily be what people actually would do, because people were so busy at the end of the semester.
For my team, if it was the case that we hadn't done something, I would put it on the sprint task list again.
But I didn't have to, and if I hadn't done that, then there would've been a big divergence between what we said we were doing and what we were actually doing.
So as an educator, I would really say check-in on your team.
See actually what they've done.
Make sure they're setting realistic goals and following through on them.
At the Red Cross/Red Crescent Climate Center, our job is to help others understand and address what's going on with climate change and with extreme events-- hurricanes, typhoons, too much rainfall, peak temperatures.
And these things are complicated.
This science is complex.
And how to turn science into action is not easy if you're too busy to learn.
Most of our colleagues in the humanitarian sector are insanely busy.
When science is communicated, it is very often either incomprehensible or difficult to link to ongoing humanitarian processes.
So about six, seven years ago, we started with creating playful activities where participants in a workshop have to make decisions that have consequences.
And that's at the core of games.
Games are systems where you have to make a decision with limited information.
You're trying to accomplish something, and you don't really know what may happen because there's something in the system that you don't control-- which can be the weather.
It can be what other people do.
It can be just you're not fully understanding the system.
In our experience as humanitarian workers, trying to infuse awareness of complexity into our work, games are the most wonderful tool to wet the appetite, to make people want to learn more and want to do better.
It has been beautiful to work with the MIT Game Lab.
We have, as Red Cross family, access to a lot of talent, a lot of enthusiastic brain power that combines analytical competence with creative juices.
That is prevalent in the field of game design and among students of games.
What we find at MIT is that it's much easier to link to other departments that are directly connected to what we do.
It can be about environmental engineering.
It can be about climate science.
It can be about technology development.
So this partnership is the beginning of something that is inevitably going to grow, to grow beautiful, to grow big.
It was also really nice to see how the MIT Game Lab is well-connected to other partners that we have in the game design sector, such as the Engagement Lab at Emerson or the PETLab team at Parsons, and the awareness of reach in the world of serious games.
Games that are not just as a playful activity, but with the intention to improve the world somehow.
We could always do better as partners of the academic team if we could allocate more time.
Unfortunately, it is in the very nature of our Red Cross work that we have to travel insanely.
I was out of the country for at least half of the time when the course happened.
So we did bring in partners and friends who gave a hand.
But I think one of the key challenges for this kind of partnership between game related academia and humanitarian teams is to find ways to interact more frequently, more deeply, to learn about each other.
We just don't know how we can do more with the limited time we have.
When students engage with partners, clients from the real world, it is not easy for them to get into the shoes of the institution of the organization that has to try to accomplish things.
So one of the most important things that we can offer as partners in this course was to give a flavor of the context in which the game would be played.
So we have the team working on cholera prevention in West Africa.
And they developed a video game that had the flavor of modern wealthy homes.
And in West Africa, most of our target audience-- if you're trying to prevent cholera-- chances are they are poor, and the aesthetics need to resonate.
It can be about a super stellar context, or it can be about a thatched roof, but it should be something that can evoke engagement on the part of the player.
And that's not easy for students to find out.
They actually have to do research on what does a home in West Africa look like?
The other thing we can do is to help narrow down and focus on what matters to us in terms of the gain in forming better decisions, and to weed away and filter out all the things that students want to include because they can.
But that's not a good enough reason.
Just the fact that it's technically feasible or that you like it, doesn't mean that it should make it into the final prototype.
There were many other things that were a lot of fun in terms of engagement with students that I think were useful to them.
In particular, to see how they could get inspired by being aware of how their technical competence can be useful for development work, for humanitarian work, for actually physically saving lives in the long term, or to help people engage with each other in other continents, in other contexts, in fields that they don't even know, but they can help.
When an academic team is offering student brain power and air time during a course for an organization-- specifically in games but also more broadly-- it is our experience that it takes more time than we think to develop the relationship and to understand how we can really get juice out of it.
And it is always worth starting.
If the client organization, if the partner-- like in our case, the Red Cross family-- can find someone who spends a little bit of time at the beginning, a little bit of time reviewing prototypes, and some feedback along the way, it's definitely worth it.
But don't expect magic solutions.
Don't expect people to be able to read your mind.
We come from different universes.
And we have an opportunity to bring the real world to academia.
By just showing up and spending time, but also to bring academic brainpower, talent, and creativity into our work.
One of the things that I have to say with enormous pride is that one of the prototypes that emerged from one of the groups in this course was actually used in three global events and helped hundreds of people in real time learn what was being thought about.
People entered three words as individuals.
There were 100 or more people in the room, and everyone saw a word cloud that emerged through very fun game play.
And this can really help, but we need to be aware of the time it takes to mature.
We have been working with game designers for at least six years.
And that really helped us channel our energy in a more narrowly defined way.
Otherwise, it's very easy to go broad and to be disappointed.
Be prepared to acknowledge your own limitations, but be prepared to explore the space of possibility.
So the game Snap is already ready to deploy-- has been deployed.
I hope we can refine the user interface to give more choices to the facilitator and so on.
But that is one game that I predict will be able to change the way real-world events happen.
I have that admission.
I think we can do something with it.
Of the many other games that were good because they were more about a subject matter that is outside of Cambridge, Massachusetts, they will take more time to refine.
I'm very excited about the Cholera game, which we now have a partnership with UNICEF.
And we anticipate the demand from UNICEF Guinea, in particular, to try to add digital flavors to their relationship to schoolchildren in prevention of anything-- in promotion of the things that help-- like hand washing with soap, which we take for granted here.
But in other parts of the world, they just have to choose between soap and food.
It's hard to make that case.
And some of the other game prototypes have very insightful game mechanics-- how to display territory in a small screen.
How to display quantities for decisions in terms of disaster preparedness.
The representation on a game on heat wave of how people can just collapse if they think they can just go out and walk on an excessively sunny, hot day.
These are things that are there, are latent, and we have opportunities to work deeper either with these students or with future students of MIT and beyond.
They will need more time to refine.
And we have, as Red Cross, more work to do in terms of identifying the potential player and identifying what are the decisions that can be improved, or what are the discussions that can be enriched by engaging in this game experience.
From the Red Cross/Red Crescent Climate Center and from the humanitarian family, I hereby thank profusely the MIT Game Lab, and by extension, game designers that have helped us become partners with you now, for setting up a space where we can learn how magically powerful but analytically clear game play can be to help accomplish humanitarian goals.
Thank you so much.
My name is Philip Tan.
I'm a research scientist here at the MIT Game Lab, and I've been teaching various game courses here at MIT for about 10 years.
Every night I play StarCraft for a couple of games, just to help me calm down and get to sleep.
So that's kind of my regular ritual.
Otherwise, lately, I've picked up this game called Desert Golfing, which is just kind of endless golfing on the iPhone, which is strangely soothing, yet maddening at the same time.
Right now I'm actually working with Blizzard Entertainment to try to create different ways for players-- for people who are interested in StarCraft-- to spectate a game being played by professionals.
So we're about fixing.
I'm really looking forward to what the students end up making.
They usually impress me with some sort of far-out-of-left-field approach to the problems that we put forward to them.
And this year we are ramping up the design challenge, so I'm interested to see where they go with that.
Outside of this class, I'm pretty much focused mostly on my four-year-old kid and relearning what fun is, actually, through her eyes.
I'd like to just see a lot more different kinds of games about a whole bunch of different things.
Right now I feel we're right at the cusp off like a Cambrian explosion of different ideas.
Tools are out there.
The knowledge is getting out there-- with this idea that anyone can make a game.
But I feel that there are still a lot of social and systemic barriers to prevent people from getting every idea into distribution.
So we're still constrained to a small set of genres-- familiar genres.
We see a lot of those games.
I'd like to get a greater variety out there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from 100 of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Hi, Pablo
Good to be back
Nice having you here.
So we wanted to talk about a number of high-level postmortemy things for how the class went.
So we've got some high-level things we wanted to talk about.
We want to talk about the collaboration between MIT and the Red Cross/Red Crescent group and your folks-- how student preparedness went, in particular, comparing those teams that got a lot of support, like client support, like Snap, of course.
But also, some of the other teams got a little bit more, and then the unsupported teams, which aren't exactly unsupported, but they got less, so that's largely the Heatwave Team, I think was the main one that we want to talk about there.
So collaboration what do you all think about how we talked to each other?
How we met with each other-- things like that?
I like the way how we started in the beginning of the semester.
We had to talk-- and we had some Skype issues.
But I think the idea of kicking off the entire class with the initial presentation of this is what your final project is going to be.
It's going to take us a while to get there, but you come into class understanding this is what it's going to be.
And Pablo being the person to describe why this is going to be important, I think, set lot of expectations well.
So I think later on, when we actually got to that project, the degree to which each team actually could have been supported actually became more of an issue.
The teams that you could directly talk with-- and you were going to be the primary user of the game-- clearly benefited.
But you were right there from the beginning.
These are the teams that, maybe, they did receive support, or some of them didn't receive support, but weren't talking with people who had been there right from the start, I think maybe had felt a little bit more disconnected from the main thrust of the class.
At least when they got to that final assignment.
So something I like to think about for future classes is the extent to which-- if we're going to repeat this format-- where the primary client comes in right at the beginning to set the tone for the entire class.
How do we represent more of the variety of different things that you can do right at the beginning?
So that when students think about what they're going to be doing throughout the entire semester, they are actually keeping full scope of the project in mind, not just the one example that might be presented to them on the first day of class
And we also may want to have something like an engagement meter.
Because I wasn't aware of the fact that some teams were feeling disengaged.
I would show up, some people have questions, some other have less questions.
And from my perspective, those who have questions I spent time and it was about the same chronometric time per group.
I could see that some were more lost, but I wasn't aware that what was needed was more fire in their belly.
So if we could--
I couldn't see that either
Right
The students never reported it to us
Yeah, I was going to say, when we asked the students to do their postmortems, they were really clear about how this had been a big motivation, and interest had been a big problem for them.
But they never mentioned it to us during check-in meetings when we were doing play testing, when we were working with them.
So that's the sort of thing that we need to-- probably one of the things you do is probably to warn them about that.
These are hard topics, and they can become boring topics very quickly.
And they need to step up and let us know when they think they're facing a brick wall
I think the clues were there.
If we had means and a way to remind ourselves, like a checklist or something simple like that, just to say, are you engaged?
What's your motivation?
And, again, that's assuming that we even want that to be our job.
I think for anything client based, it has be our, probably, our job
I think that when we're asking them to work on projects where they have relatively little control over the topic, I think we actually do need to.
Because I think part of it was I don't think the teams knew how much freedom they had to deviate from the topic they had chosen and the specific.
They thought they had chosen this thing, and now they were stuck with it.
Even though one team was like, we chose a topic that just wasn't deep enough.
There wasn't enough there.
And if they had come and talked to us, we could have given them the freedom to well, go change to a new topic, or how can we go--
Go deeper
--how can we go deeper in it?
Right.
But we were not sufficiently warned
But that's where rapport with someone who knows a lot about the domain can help.
And if they don't feel like they have that rapport or someone who they can readily contact to ask even seemingly silly questions, then we're not going to be able to get down to actually ask the important questions.
Because sometimes they can't tell.
They can't necessarily tell what looks like flagging motivation on a part of the team or what just seems like end-of-semester pressures on the students, right?
They haven't been working on enough projects to be able to tell the difference.
And so we do need to help them through that, I think
By that you mean the students, then?
The students.
I mean the students
And one of the things you may want to consider is we know that both in the real life-- in the future, in their future, as well as in future courses-- there will be limited ability of those who are experts to interact with the game design team.
The game design team, you took the effort-- and I thought it was very well-balanced-- to have some who know programming, some who know narrative, some who know user interface, and so on.
They will, by default, not have expertise on heatwaves or cholera or etc or climate science.
They cannot assume that all they need to know-- including the appetite for engagement-- will be only in the client.
MIT is a bubbling powerhouse of engaging stuff.
And if they could, as part of what they are told-- they will have to do in the future, and what they have to do in this course, to reach out to engaging people, fellow students, and junior faculty or whoever may give them some time-- to just go and say, what is exciting about this?
What are the cutting-edge areas?
What are the institutional challenges.
So that they also have to fuel their own appetite
Having an assignment or having part of that final project being like here's the research component that we expect you to do, but we didn't put an underline over it
We didn't make it explicit that they needed to--
We did when it was in trouble.
Like for Saving Gora Gora, they were in big trouble.
We told them a number-- you told them, we told them.
I think you'd be surprised what they came up with.
Because they actually, I think, they took the baton and went with it
They did that [INAUDIBLE]
Nice
The other cholera team did not, and they were given the same-- almost the exact same-- feedback.
That is the thing
I think there is a huge social barrier between the undergraduates and the rest of the institute.
The noble laureates, and even the post-docs here, that's a lot of social pressure keeping students away from that kind of interaction.
You're all right.
All of these things do exist here at MIT, but some of this is reinforced by professors themselves-- professors who keep their doors closed when they in their office, things like that.
And so I do think we have to do give them the license, and sometimes that's pointing them to specific people
Yeah, giving them resources like here's a list.
One of these might pan out for you.
The others might not.
And that's OK, just do what you can, kind of prime them for it
It's hard to be a teenager in this space
Actually, since we're talking about the student preparedness in that research like what we saw between what we consider the supported teams versus the unsupported teams, which may or may not be fair-- could be the motivated teams versus the unmotivated teams.
Snap, in particular, really--
[INAUDIBLE] correlation
--really well-supported.
They actually reached out to you all for help with the game.
You actually even did testing for them that was actually coming from you and Jana, yeah, because you were running workshops
Right
How did those workshops run-- how did you interact with the team?
So, first of all, I think it's important to distinguish not only what you are calling supportive versus unsupportive, but also Snap versus everything else.
Because Snap was a game concept that preexisted the course that needed refinement and digital interface.
So from that perspective, it's not surprising that they have something clearly defined that they could run faster and deeper.
And the group was a group that engaged and went deep and went fast.
The interaction we had with them, also, very importantly, we have events that we could use the product.
So it was much easier to give feedback saying when we do this, we expect people that would get bored, why don't we try this other thing
It's milestones, which we give them milestones.
But there is no weight behind them.
These ones have an actual, if it doesn't work by then, someone's going to have egg on their face, right?
And from that perspective, we know this, but we can try to do more of giving students more proportion of work that we know-- that we, the clients know-- that we have milestones.
And that is limited by how much of the real world gives us milestones to abide by.
But we can try that.
I do not feel comfortable with the label supportive versus unsupportive because it sounds like some got time and support and some others didn't.
I think it's more about the form of the relationship and how the time was used or the opportunity to use the time was used.
Because I have no doubt that you supported the team when you knew support was needed.
And as far as I know, the same was true on our part.
I think it's more about having sensors about when support is needed that is not being delivered
I think when we're talking about them in this way, I'm using it based on how they describe their own relationship and their own experience.
So that the teams who we are calling unsupported are the ones who, in their postmortems, called out and said, we didn't feel like we got client support.
They got the same amount.
It was a different way, different quality, different quantity
The way to put it is the same amount of support was there for them.
One other way to put it is they didn't know how to reach out and get it.
They didn't use the resources that were there because, at some level, we weren't pushing them at them.
And a lot of MIT courses tend to encourage students to solve all their problems themselves.
They don't want them reaching out.
They want them-- figure this out.
And so it may come as more of a second-- it may not come to them as first nature to go, hey, we need to go and reach out for more help and grab that--
And so that's actually the same thing.
So what we saw with the Snap team is we wanted-- the feedback they were getting from us was look at your front end, look at your UI.
We wanted to see a lot of changes on the UI.
Because they were working on these milestones, they were really worried about the back end so they were doing a lot of work in the back end.
They were getting two different messages, and they chose client message rather than instructor message.
Totally OK and totally viable.
We took that into account.
But I think the main issues we had with them is when they got our message, it was, they heard it early on.
They didn't react to it until it was too late-- had they done a little bit extra in the beginning.
So one thing I think we need to think about with the class is thinking about how are we helping them understand how to prioritize feedback?
Right?
What's more important?
Could be because here's something that's actually going to get used.
It could also be here's a lesson we want to make sure they get.
I think they got the lesson.
They just got it in a harder way
Sometimes they get a lesson at a point of time that they can't actually execute on it.
But hopefully they can do it for their next project after class.
Something I want to suggest, actually, because we've been talking about how it's been difficult for us to figure out what's going on inside the teams.
We can see the games and the testing, but the team dynamics, we don't find out about that until near the end of the class.
And something that we used to do outside of this class, but in our research work, was to get all off the scrum masters together on a regular basis to just do an exchange of information with each other.
They don't even have to be necessarily reporting to us-- although we can moderate those sessions.
It's more about them sharing their experiences with each other, and then it may make it more obvious that there are resources out there that some teams are taking advantage of and some teams aren't.
Of course, we can use those situations for teaching, but also for teaching each other
Creating some group, group support
We talked about that before this semester started, but we couldn't figure out a way to do that fairly.
I already feel producers bear the brunt of the work in the class.
They're the ones we see all the time.
We don't see the students working on programming or design as much.
Design a little bit more, but programming, we just almost never see them
And we let the teams use the producers as their face-- so giving all the presentations, giving all the reports, answering questions when we ask questions in class.
Think about how we convince the teams to share out that work a little bit more evenly
Well, perhaps, maybe, we can reduce the extent that we require these producers to do presentations in class, and replace that with these more closed-off sessions where they don't feel like they have to be on stage and put on the best face for their team.
They can actually just talk about this is the problem I'm having with my team
So one thing I also wanted to make sure that we talked about what was it like for you?
How much of your time did we demand?
Was it a lot compared to other projects?
Was it a little bit?
Was it about moderate?
I want to start by saying one of the things that I really appreciated was the time we spent-- both face-to-face and emailing and so on-- before the beginning of the course, to have understanding of what you and your students can offer, what we need and can offer, and to manage expectations, as you said.
I think that was very well done.
And it's not always the case.
So with the students and professors and Red Cross, in general, we [INAUDIBLE], so thank you.
That was really appreciated.
I think that amount of time that I personally had to give was about as much as I anticipated, about as much as I think is reasonable to expect for the future.
We do have Jana, in particular, and a few other members of the team-- of the Red Cross team-- who were available to give a hand.
And I think we needed to improve because I did make it very clear that I couldn't get five emails per day asking questions.
But to have some kind of ability for the right contact when either engagement levels are too low or mega confusion levels are too high or sense of purpose is diluted and so on.
So to refine that.
I think it would be better to think of times when there can be slots of decided interaction between each team, and someone from the client organization that doesn't have to be during class.
Because I would always interfere with your plan.
You gave us one opportunity that I thought was very useful, but a little bit late in the development process and also very crammed.
By the time we said, OK, these are the kind of things you can do, there was no time for them to think and reflect on, OK, we can do A or B or C. Show that to us, and then us providing feedback.
So even if it were one more chance via Skype but scheduled, planned, and if-- it is in the nature of our work to have to travel, and it is in the nature of our work that travel timing changes outside of our control.
So where there's a default date, and if things have to be changed, it is changed.
So to know that there's going to be an instance of feedback that may be too little, too late, still but to anticipate that
Yeah, part of that is the students, we don't always know the student's schedules.
So it could possibly be planned in the beginning of the semester.
But planning it before the semester would be really difficult.
Unless we just say it-- this is the time that you're going to have.
If you don't take advantage of it, you don't get it
Right.
It has to be we propose it's this week.
This is the default time slot.
If not, that time slot, agree on a different time slot.
And then it's up to the students
So you're actually saying you would have talked to them a little bit more
If the problem was lack of engagement or perception of lack of support
So it's really that planning and scheduling.
So as long as that's in place, then the time is there, because it's been planned
And also I think it would have been nice for them to have access to, say, someone from the Ghana Team to look at the color, someone from the Heatwave Team in Buenos Aires-- to at least to get a flavor, there's another human being there that eventually can use their game.
And I didn't do enough to help them envision the user-- either the Red Cross user or the community level or policy or government user
I think we need to state that strongly that it's not just the information that you're providing.
It's the fact that there is a real person who can really benefit from the work you're doing is much more powerful than some sort of abstracted audience
So we provided plenty of documentation, but that's not the same as a human being
That's not the emotional drive that you're going to get-- oh, someone is counting on me doing good work
So maybe it might just be something for us to talk to the Terrascope folks to see how they do that
Well, they specifically get people who can commit on regular meetings.
And the idea is-- my understanding is that it's for the domain knowledge primarily.
But you get that side benefit of here is a real human being who's excited about this problem, or maybe really concerned about this problem, and that can influence the motivation of a team
And also, I don't know to what extent we can keep cramming what students have to do-- I know they already have to do way too much-- but to look at how that kind of field is currently being communicated in the game universe and outside of that.
So PowerPoints or documents, they're, across the board, generally boring and unsuccessful-- even if they have the right content.
So for them to get a sense that if someone really needs to create awareness on that topic, and this is the best they have in what exists in the real world, how about trying to give something that makes people want to do more through their game?
I think saying, this is how low the bar is, can we get it slightly better, is less motivating than saying, what's the best you can do?
But it's also choosing something that's going to get used.
Why go in a direction that's-- if this is what those people are thinking about, if you give them something completely not like that at all, they're probably not going to know what to do with it
But I mean, I can imagine things like evaluating current methods of teaching about a certain concept, to be part of that research process that we've been talking about-- it's understanding the problem
Cool.
All right, so I think it's about time to let you go.
But thanks so much for coming in
We thank you.
Huge gratitude.
[MUSIC PLAYING]
Creating Video Games is an MIT course, in which we introduced to students all the different roles of game development.
Not just the creative aspects of design, but also project management in working in the team
So what we're publishing on the MIT OpenCourseWare site is the entire class that was taught in fall of 2014.
And that Includes the print materials, the syllabus, the readings for the course, our course calendar, and our product description
Well, so besides all the printed materials, there's a lot of video.
Not just the lectures from the faculty, but also guest lectures from the industry
People from companies like Electronic Arts, Riot Games, Fire Hose Games.
And some local indie developers in the Boston area
They talk about issues of project management, music and sound design, visual aesthetics, and all of the roles that are involved in putting together a creative piece of software
So it's not just these course material on the site, there's an entire section devoted to educators labeled This Course at MIT that describes how we taught the class
It includes infographics about how we spend our time as instructors, and how the students spent their time on their projects.
There's a interactive timeline that shows how they iterated it on the projects throughout the semester.
And there were video interviews where we talk about how and why we did the things that we did
One exciting thing we did this time around, is we brought in an outside client.
That was Pablo Suarez of the Red Cross/Red Crescent Climate Centre.
Our students created games for his group, designed to meet humanitarian needs on a global scale
Most of these games are going to be available on the MIT OpenCourseWare site.
And you should be able to them using a web browser.
So I'm really, really excited about all this stuff being published on OpenCourseWare
It's free.
There's no registration.
There's no sign up.
It's available to anyone
It should be interesting for students and educators.
But really, anybody who's interested in finding out how creative software is put together.
[MUSIC PLAYING]
So the course evolution has actually been more a refinement than any really big evolution.
We've always had the four project structure.
We have done some juggling of the amount of time allotted to each project, and what we're asking for the goals for each project.
The first year we ran the course we were kind of asking for a full game prototype from each of the iterations.
And we were asking them to do everything for every project, which was a little much to ask them, although they did it.
I mean, they did it.
We've been refining each of the projects as it goes to try to emphasize a particular thing we're trying to teach, especially in the first three projects.
So the first project is prototyping, and the second project is project management, and the third project is now user interface and interactions, feedback especially.
And then the fourth project remains tie it all together into one overall synthesis.
Also, I think that our use of guest lecturers has gotten better over the years.
We've been able to reach out to more people and we've realized more and more the value that they bring into the course, having people to come in and talk, often agreeing with what we've said, but giving a different perspective and rooting it back into the game industry.
The students really like to hear that this is actually what's going on in the game industry.
They're not just being taught this stuff in a course
When we first started teaching this class we were primarily looking at the skills that MIT students had and didn't have when they came and joined our lab as undergraduate research assistants.
And we found that there was a certain skill gap of things that we had to teach every single new student who came.
Even though they may have been very skilled programmers, they didn't understand what it was like to work in a team.
And so a lot of the genesis of this particular course comes from just us trying to increase that pool of students that we could hire as undergraduate researchers.
And of course, in the process of teaching a class about making video games, a lot of the students taking the class were also interested in working professionally in the game industry.
And so we expanded the content on the skills that you're going to need outside of MIT.
But we've always been very, very careful to make sure that a student who comes out of that class is in good shape to be able to get a research internship here in our lab
So this semester, product three is different than last year.
Last year the focus was on the aesthetics, so think about how art and sound and visual combine to make it an aesthetic experience.
That's not as important this year for our final project.
So we changed it to something that's actually going to help them with their final project a little better.
So this case it's user interface.
We're asking them to make these decision making strategy-like games.
There's going to be a lot of elements on the screen that a player is going to have to understand immediately what they do and how they interact with each other.
So focusing on that user interface and that user experience is important for us, and in particular, feedback systems.
So because our theme is meaningful decision making, if I'm making a decision, part of it is I should know what the decision was that I just made and how it affected the rest of the world and how it affected my own play.
So that kind of reinforcement, that kind of feedback loop is going to be very important for that third project
We've also brought back something that we temporarily departed from, and that's the idea of having a client, a real client who has a need for the games that need to be made.
In reality, if you go out and work in the game industry, you're not making games for yourself most of the time, you're making games for someone else.
So in this case, we're going to be working with the Climate Center and the Humanitarian Response Lab here at MIT to talk about issues of climate change, and more importantly about preparing for disasters.
We feel this kind of topical relevance will make this class additionally more interesting for certain students who aren't just enamored with the technical challenges of making a game.
But I think this might also be the first time that we specifically had the client give us a design theme as opposed to a technology to work with
Yeah
So it's a departure, but in many ways it's taking what we've learned from the previous two iterations of this class when we didn't have a client or when we had students working on a design theme rather than a technology and putting it together this year.
My name's Rick Eberhardt.
I'm the studio manager for the MIT Game Lab.
I've been with the lab since 2007, 2006, so since we formed a long, long time ago.
Before I worked at the lab, most of my background has been doing information technology for academics, institutes, so colleges and universities.
I always had an interest in games.
I actually had a cultural studies degree in college for my undergrad.
So coming to the lab, working at the lab for that first year as a system administrator really opened my eyes to all the different things games can do and the different potentials games have, but also made me realize I can actually start making these kinds of things on my own too.
So since I've started working at the lab, I've taken on more duties related to project management, things I'd already done a lot in IT-- managing people managing projects.
I teach this Creating Video Games course, and for the most part in the course, I'm interested in teaching the product management techniques, but also the soft skills and the social skills.
I'm an introvert.
I have a really hard time having conversations with people.
And I've had to, in my years in my profession, figure out ways to not let that be a restriction, and actually use that to my advantage when I'm talking with teams.
And I find it really successful, actually, in game development.
I don't know if it could be successful elsewhere, but there's a lot of other like-minded people like me in game development.
So it's good to have, for me, to be able to use those skills in that setting.
The other advantage I have at the Game Lab is I get to do research.
I get to help with research on various different types of projects.
So my personal research interests tend to be towards closer to what this class is talking about-- this meaningful decision making.
I've done some research in using board games to teach decision making, in particular, like economic concepts, like opportunity cost.
I'm interested in social experiences in games, and cooperative gameplay.
Past games I've worked on have been about all sorts of different things.
So anything from tools to help game developers make 3D characters animate faster-- basically to speed up the workflow process-- to games about depression and games about other serious topics.
I really like working with researchers.
I like working with clients, and I like making games.
What am I playing now?
So I am playing this game called Gridland, made by the same developer who made a previous game called A Dark Room.
It's kind of difficult to describe.
It's basically a match-three game where you're like Bejeweled, you're matching gems.
But there is a small supply chain mechanic going on in the background.
Where as I have the gems combined, I can upgrade my weapons, and I can upgrade my armor.
And then when night comes-- so I'm matching gems in the daytime to upgrade things-- and when night comes, I'm matching gems to summon enemies and to kill enemies.
So this Gridland game has a daytime and a nighttime view of the game.
And what I find really interesting about it is that all my decisions I make in the day are going to have some kind of influence in my nighttime decisions.
So as I make my daytime experience easier, I'm actually making my nighttime experience harder.
So it's just a really interesting little challenge that the designers put for me to try to solve.
So outside of the class, the kinds of games I'm making are non-digital prototypes, so card games, in particular, that are experimenting with things like a tech tree and a 4X-style game.
So if you've played a game like Civilization, or StarCraft even, they'll have these tech expansion trees.
So I'm playing around with designing a game that's largely just the tech tree.
I'm doing that with a coworker here, Drew.
I've also got a game I've been working on, off and on for the past couple years, about managing cities-- about three players trying to manage a city.
They're counsel people.
I'm trying to convince the mayor about what decisions the mayor should make for the city, but, unfortunately, the mayor is just the role of a six-sided die.
So you can try to convince him or her what you want, but it really doesn't matter because the mayor's got its own agenda.
So then you're trying to solve the problems with the city, while also trying to solve your own personal-- trying to basically get political gain while solving the city's problems.
There's a lot of different co-op experiences out there-- from anything from over the table like cooperative board games to couch style co-op in front of the TV.
I'm trying to think my favorite co-op experience is definitely not Pandemic, which I think we're talking about in class today.
I have a hard time deciding what my favorite co-op experience is because so often, one of the challenges of co-op games is somebody takes control, and somebody says what the whole everybody on the team is going to do.
And unfortunately, I'm one of those people.
If I know the game, I'll start being bossy and I'll take over.
And what I'd love to see more co-op games do is put in mechanics to prevent me from being a jerk.
Or enable other people to override my jerkiness somehow.
So the thing I look forward to most in this class every year is the playtest sessions, especially when it's the first time the team has brought the game into the class and they're putting it in front of new users, which we only have a couple opportunities for each project for that to happen in class.
After that everyone in the classroom has seen the game before, so they are not giving the same kind of feedback.
And what I really like about it is when the games come in and they just don't work.
They're broken horribly.
Not when it's the technical problems-- not when it's the code crashing or it's the broken build.
That's actually really sad, and I hate to see that.
But when it's a design issue, when they thought it was going to do this one thing and see the game doing this other thing, or they see the players doing this other thing, it's at that point that the students understand that OK, people do people things.
Humans are human, right?
You can't always predict everything that they're going to do.
But also that the systems that they created are actually incredibly complex.
So the things that they need to do to fix the systems are going to be reducing mechanics, removing things from the game, making it a little bit more simple, so that they can predict a little bit better about what's going on.
But then on the flip side of that, though, there's sometimes when you see something happen in the game, and you just get this emergent gameplay that is better than what they were going for.
They didn't realize they wanted it.
When they see it in action, they're like, oh, this is actually really great.
What can we do when we're finishing this game to really capitalize on this mechanic or this system that we just didn't even realize was there?
I really like systems.
Outside of the class, I think this might be the thing that I actually, I always had in some capacity.
But really coming to MIT, working at MIT, it's much more part of my life is seeing the systems that surround us and the systems of history.
I've always had an interest in history, and a little bit of interest in anthropology.
But reading Charles Mann's books about 1491 and 1493, what happened during the Colombian Exchange, when all of these different ideas and products and agricultural things swapped places from East to West, from China to the Americas to Europe.
There's just some really interesting systems.
And a lot of the problems we're facing today are the direct result of that big expansion of ideas and trading of ideas.
So I've always been interested in those kind of things.
And I'm watching The Wire.
I really like The Wire.
And just seeing the systems that are involved in city planning, in police work, in education, in all these civic institutions that we just take for granted every day.
Yes, The Wire is a very critical and a very pessimistic view of it.
But it is a great expression of just how all of these problems are connected.
And some of the problems that we are facing right now, it's just we can't see the connections.
And I hope that, at least with games, with playing games, with making games, people can better understand how those systems work.
For the future of games, games as expressions of systems, games as ways of exploring the world around us.
We've always-- 1,000 of years ago, we used games as ways to communicate with each other, to have rituals with each other to better connect with each other.
And I think we can do that with just games in general-- not just video games, but all sorts of different kinds of games.
So seeing that come back to both what people are talking about when they talk about games in mainstream.
Other than just this thing that's done for fun, it's actually this thing that's very important to human daily life.
And we've been doing it for years.
It's just it looks different now, but it's still based on a lot of the same things.
I'm Sara Verilli.
I'm the development director at the MIT Game Lab.
I came to the Game Lab from the game industry, where I've worked as a producer and a game designer and a QA person.
I've done an awful lot of jobs and seats.
I've worked on Thief, Thief 1 and 2, Flight Ultimate.
I started at Looking Glass.
And that's actually where most of my work was done.
I then worked at Irrational for a while.
And that's when I moved to the Game Lab.
I came on to be a specialist in production, production management, and QA testing.
And those remain the areas that I teach in the course, specifically.
We all do a little bit of everything.
But I lead the lectures on agile and organizing your team and keeping it running.
And I also do a lot of the testing lectures and focus testing lectures.
What am I playing now?
Right now I am not playing, but I'm hoping to play soon, is the newest release of Minecraft, which has bunnies in it, which both I and my daughters are very excited about.
So I am hoping that tonight I will be able to actually go home and arrange for a play session of Minecraft with them.
I'm also playing a fair bit of Sentinels of the Multiverse.
And most recently, we've been tackling the Pathfinders card game, which is almost a way to play an RPG in a short amount of time, except that takes almost as long as a full session of playing a sit-down RPG so far.
We'll see if we can get faster.
Right now, I'm not actually working on any games.
I was working on a climate change game.
I'm moving away from that, and trying to think about what is a more interesting problem to work on, in the area of not just teaching game design, but using game design, using people learning game design to learn other subjects, as well.
So you can tie two things together.
And I'm playing around with how do you teach that?
How do you structure a course that would do something like that?
I'm actually really excited to having a specific design question that we're asking the kids, the students, to look more deeply into, into making decisions for the player, and making meaningful decisions.
Because that's a part of games that the simpler your game is, the harder it is to invite your player to make meaningful decisions, and to set up situations where they are making those.
But that tends to be the part of games that I, at least, find the most interesting-- the places where I'm making choices, and I can see how my choices have completely sunk my ship or enabled me to win the game.
I like knowing that I, I have control of my destiny.
And so getting the students to get to explore that aspect of games better and more deeper, I think, is really nice.
I'm currently also working on creating a game with the Step Lab Introduction to Game Design on MIT EdX.
I think it's 11.126x, is the number, I believe.
I've spend so much time making the course, I have not been looking into all the administrivia tack to do it.
But working on that, trying to condense a game course into six weeks, has been really hard, and has made me really stop and think about what is the actual important things in game design?
I would like games to go-- well, they've already gone some of the places I want them to go.
Games that are looking at more social-- integrating more social experiences.
Games that aren't just-- board games where you're not just me playing against everyone else, but you're playing as a team, and you're playing cooperatively.
And video games are also doing that.
They've been doing that for years, in the massive multiplayer games, and things like that.
But seeing video games go more for how do we work together?
How do we make this a more social experience?
And what I really love to see is games getting to the point where players get to affect the games back, especially in the RPG space, where if I'm a big enough player, and I can now defeat everything in the game trivially, can I go in and change the game to make it more interesting for other players?
Can I take some of that?
And then, that actually goes all the way back-- that was already being done back in MUDs, right?
This not actually a new concept.
It's just that production quality has gotten so high that you can't do that anymore.
Well, with all the tools coming online, can we bring that really neat players giving back into the game and creating experiences for other players?
Maybe we can start bringing it back into the high-end games, too.
The students spend about six hours in class each week.
And right at the beginning of the semester, most of those six hours are either lectures or activities or workshops that we have planned to have inside the class.
It's very, very tightly programmed.
This may involve, say, brainstorming for the team projects, or forming up with teams, going through the project constraints, or teaching them the techniques that we want them to learn.
Near the end of the semester, we heavily scale back the program content.
We give students a lot of class time just to meet up with the teams.
Because for many, many teams, it's difficult enough to gather six people in the same room, in the same place, at the same time.
To get eight of them in the same room, well, they're doing it during class time.
At least you know everybody's going to be there.
So we try to give as much time as possible later in the semester for teams to actually work on their project.
But sometimes that is planned out, like playtest sessions, for instance, where students play each other's games
In the first half of the semesters, all the lectures and workshops are given by the staff.
But by the second half of the semester, we're getting them to see other aspects of game development.
So within MIT students-- and actually, this year, a few Wellesley students-- most of our students are going to be coming to us with a programming background.
A few might have some design classes under their belt.
Maybe they've taken one of our game design classes or another design class offered at MIT.
Some of the students are going to have some art aptitude, visual art aptitude.
Some, a little bit more a sound design or music composition aptitude.
But for the most part, they don't know what it's like to be an artist.
They don't know what it's like to be a sound designer on a game development project.
And they might actually not know anything about the business of games.
So the last half of the semester is bringing in guest lecturers from the industry.
We've got a great network here in Boston, an independent game developer network.
Also, quite a few of the AAA larger studios are here.
So we mine our network in each year.
We get new speakers to come talk to them about how to work with an artist, how to work with a sound designer, what it actually means to be an art director in a game company, or what actually does it mean to even be a level designer on a large project?
But then also, we're bringing people in to talk about business models.
Kickstarter is often an interesting topic for the students.
We try to balance it between here's what the industry what looks like, and here's the things that are going to help you, right now, with the game that you're working on now.
And if you wanted to take the game a little bit further after the course, here's the things that you could do.
Things like fundraising, submitting it to festivals, marketing it, and just getting other people to play it.
So when students come into class on their first day, I think they usually expect to walk in and start working on their dream game right away, and that's just not going to happen.
They have a lot to learn.
Simply to learn about working with their teammates-- figure out who else is in the class and what skills they bring to bear.
And chances are they haven't really thought all that much about the game design that they want to make.
And usually the very first idea, the idea that you've been nursing ever since high school or something, isn't the best idea for a single semester project
Students often want to make the big expansive epic RPG.
We get a lot of students who want to make the next Legend of Zelda-- maybe trying to remake Sky Run, but in 2D, and they think they're scoping it down, but they're not.
So it's one of the first things we're doing-- especially the first couple weeks of the class-- are just getting them to understand what is the scope of the game development project, but also just getting them to understand how difficult it is to work in a team.
How much effort an individual needs to put forward in order to work efficiently as part of a team, especially when it comes to a creative thing like game development and game design.
A lot of students want to make their personal project.
And the first thing they need to do is realize that there are other opinions on the team that should be respected.
And there are some opinions that are going to be stronger because there's more design experience there.
So figuring out how to balance those teams is a big, big step for them to take when they start
The biggest thing we want them to learn when they come in and take the course is how to work as a group on a creative project.
Almost everybody's worked on creative project before.
People have worked in groups before, but working on a project where the group together is setting the goal and the goal is fluid.
Because, with a creative project, with a game design project, what you're making is always evolving as you keep working on it.
And so being able to, as a group, discuss those changes, acknowledge those changes, work together to get to something that everybody wants to work on and everybody is capable of working on, and you are all capable of getting done in the very short amount of time you have to work on these complex projects.
That's actually one of our biggest goals
One thing about creative projects is there aren't right answers.
There are a lot of different ways that a project can go when you are basically just trying to create an aesthetic experience for the players.
And so multiple people with different viewpoints can all be right at the same time.
And it's not like you're designing a machine to solve a specific problem, and if the problem is solved, you know that you've done your job.
And this can lead to a lot of ego and clashes and arguments over basically something that is already difficult to implement.
And every time you get into those arguments, you are actually taking time away from the actual development of the game
We try to give the students the tools and methods they can use if they have an argument to provide evidence for the argument.
And often it comes down to testing out ideas, rather than having those discussions as a team.
Using prototyping methods, using either paper-- if they just got the paper right there before they've even written code.
Just throw something on paper, get another person look at it, and then to see, did it work or not?
Getting away from having the personalities involved, getting away from having ego involved in all this.
For first-time designers, for the students that we have, if they don't have the evidence, the first thing they're going to go back to it is personal ego, which is not the way to do these things.
It will just cause further problems down the line
So the course is overall organized with four projects.
All of our work is group work because we are emphasizing how do you work together on a team to solve a creative problem.
All of our projects are group projects.
And we grade primarily as a group.
Our projects start from very small and scale up to big.
So we have our opening project, which is just doing a paper prototype-- even a fraction of a paper prototype-- not a full game.
It's just what are the main actions in the game?
And that's about a two-week project.
And then we build up to, OK, you've got a prototype.
Form a new group, a bigger group this time, about double our group in size from three to six, and turn that into a digital game.
For the third project, you, once again, have that two-week time period, but during the second project, we taught you our basic project management schemes and some project management skills-- using that second project as an example of how to do that.
And on the third project we're asking you to come up with your own game design idea, prototype it out, and finish it as a group using all those project management techniques that we taught.
It's sort of our test run project for the fourth project, which is the final project.
It lasts, I think, seven and a half weeks, and we double the group size again up to eight.
We are actually intentionally giving the students large groups that are harder to work with.
And their first thought is we've got all these people, we can make a huge project.
And our response is you've got all these people.
You're going to have to figure out how you're all going to communicate well together, so you can actually get something done.
And that's sort of the final capstone project where we give them a fairly tight set of constraints.
But other than that, they are designing the design.
They're planning out the project management.
They're in charge of planning their testing sessions and their polishing sessions.
And we use that time to check in with them and give them feedback along the way.
But it really is like, OK, we've given you the tools, time to really practice using them.
I'm Tej, I'm a grad student in computer science, and I worked with a team of eight on a game called Snap.
All of us were course six majors in computer science.
We created a game called Snap, which our client, who is from the Red Cross, he presented us with this game in class.
The game works where you both come up with words, two players come up with words, and whenever you see the same word you snap and you get a point.
So our task was to create a digital version of this game, both to make the game more engaging and also to help them collect data on what words people were submitting, because it's sort of a free association game and helps them get a sense of what people think about a complex topic.
So for Snap we actually had a concrete game that the client gave us and we knew exactly what project we'd be working on.
Snap is also a little bit different in that it's a much more technical game, because we're not coming up with sprites, we're not coming up with a story.
It's quite bare bones in terms of interface.
And so we had a much more standard coding project for our game.
But at the same time, there were game design aspects to what we had to do.
We still had to think about how to go beyond what we played in class and port over the game to a digital version that will still be engaging and didn't rely too much on the human interaction, the face to face interaction.
So at first we actually thought that our game would be much more restrictive, because we thought that the clients had a lot of existing ideas about how the game should be played and what works well for getting good feedback out of people.
As we went on, we were pleasantly surprised to find out that we actually had a lot more creative freedom than we thought.
And really, they just wanted something that resembled the original game and accomplish what we needed, which was to get people to give words and say what they were thinking.
So that was really nice for us to be able to really think of what we wanted to.
At the same time though, the client was interested in using the game much earlier than we expected.
The goal for all the projects is that they remain open and available for people to see over the next month.
We've actually been running the game in the past couple weeks.
And that has also required some effort on our part to do exactly what the client needs for their specific runs in the past week.
So traditionally, this course does not allow people to use networking, because it tends to be very buggy.
It tends to be something that if it doesn't work, simply makes the whole game fall over flat.
And we decided to violate this rule very early on.
The rules are all suggestions, after all.
And we were confident in our ability to do this, partly because we had a lot of experience with networking, with distributed systems.
I actually work in distributed systems.
And we also were experienced with the technologies that we were using, which definitely helped.
I think that the learning curve for those technologies was a little bit more than we could have done in these eight weeks.
So this course is nominally about creating video games, but in reality, as the instructors admitted, it's about project management.
And I think that the course does a really good job of teaching you project management by example, by forcing you to actually do it.
And I think that's why it's structured as four games as opposed to one big game, which is what you might expect.
And I think that that was really useful for us to learn project management.
Game development was a nice side benefit, but I don't think we learned a lot of game design.
I think we just tried it and mostly we were learning, how do you manage a project, how do you manage a software project?
Games are a little bit different in that the goal is fun and not so much coming from a client.
The last project notwithstanding, mostly games are about you want to create something fun.
I think the biggest challenge for us in Snap was getting people engaged, getting our team engaged in the project.
There's a lot of other classes that people are taking and this is not necessarily the highest priority on everybody's list.
And that was something that you really have to work around.
You really have to find ways to get people engaged in the topic and get people to contribute where they can in the number of hours that they have available.
So I don't know how completely successfully we were at getting people to contribute as many hours as we needed, but definitely part of it was making sure the tasks were very clear, so that not everybody was thinking about the whole game, but instead could think about their specific tasks and what they need to do on their part.
And that was part of what the project management was, was making sure that people have the information they need to do the tasks that they need to get done without thinking about the whole project.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
It's a best selling techno thriller turned into a movie, which is about a demo, a semi autonomous program gaining full autonomy and using levelling systems from an MMORPG to basically turn humans into a second speaker society controlled by computers instead of the other way around.
And connected to that theory is definitely led by the rise of the term or the concept gamification, somewhere around 2010 or 2011, if you follow Google search interests.
The idea that you could use elements from games or game design as a kind of monosodium glutamate or crunchy flakes to edit some other stuff to make some other stuff more palatable.
And if there is one thing true to this term is that it has the heated debate from the get go.
On the one hand, you have people like the ones who read, Gabe Zicherman, he says's it's the best tools humanity has ever invented to create and sustain engagement in people game.
On the other hand, you have people like Ian Bogus for those choice words [LAUGHTER] [INAUDIBLE], which of course is a technical term, it's a technical term in philosophy by Harry G. Frankfurt, describing people who just don't care whether what they say is true or untrue.
So when me and my colleague Stephen [INAUDIBLE] we're looking at the debate around 2011-2012, we couldn't help noticing a couple of things.
First thing we noticed is-- as I said, this is a really heated debate.
This is so heated that this is not just the difference of opinion, there must be something deeper underlying here than just difference of opinion.
And second, while evangelizers were evangelizing, and critiques were chastising, very few people asked, well, what does the gamification even mean?
What are the larger consequences, the larger ramifications of this for us as individuals living in a society where these kind of systems become more prevelant, or for us as a society?
That's kind of the origin story of how this book came about.
In this book, most of what I'm talking about in the first chaper, the history chapter, is basically what I'm talking about here so.
While we were digging a bit deeper in this thing, saying why this heated discourse?
Why are people so angry?
We found that gamification-- the gamification discourse-- is very much in the same state that the discourse around playing was in the late 1990s, when the late Bryan Smith [INAUDIBLE] game [INAUDIBLE] play.
Show of hands, anybody know the ambiguitive play?
OK. Show of hands who's involved in some form of game studies or game research program?
Yeah, that mapping sort of makes sense, if you're into game research, you should at a certain point come across this book.
What Sutton Smith and this book basically makes the argument and then shows is that he says the reason why there hasn't been a lot of agreement about what players and what we should do with the play is that the different disciplines of different rhetorics, different ways of framing the play that are grounded in different deeper cultural rhetorics about what plays should be and how plays should function.
As it says, rhetorics of plays [INAUDIBLE] placed within broader value systems.
The popular rhetorics in the listed settings, the ambiguity is that there would be seven different rhetorics, and nobody can ever agree on one.
There are these different, large scale cultural ways of thought in which we, more or less, all participate.
And that's the reason why that debate never really comes really close.
And in the same way, when we look at the gamification [INAUDIBLE] and say yeah, that really captures pretty well the happening here.
There is a disagreement here about what the proper plays of plain society should be to begin with about should playing games be used to begin before anything else or not?
And if so, how should they be used?
By whom should they be used, and for what should they be used?
Because the thing that we observe, the series games of gaminification with playful designs-- all these different terms, it's basically a kind of dual cultural moment.
On the one hand, games in play leave the traditional spaces that they've been confined in within our modern Western nation states and cultures.
This was the leisure time of things you did together with your friends in the evening, and there were certain kinds of games.
And suddenly, there were also games for learning.
Suddenly, there was games for politics.
And suddenly, everybody plays.
And then part of that, what you get as a response to that is that just games become more and more pervasive in everyday life, which we call dilutification of culture.
You see that other social actors become interested in using games and in impressing norms, and regulations, and values onto games.
So suddenly, you have a big debate where people say well, games should be gender aware, and games should be more diverse, because look at what turbo misogynistic space this is.
And then you have the existing male game cultures revolting against that, as this result that game play becomes more pervasive.
Suddenly, you have a cultural discourse.
Leave out for all the terrible, terrible online harassment of gamer [INAUDIBLE].
But underneath that is this underlying cultural discourse, who should be allowed to play games?
Who should games be for?
Who should be able to make these decisions?
So if Ian Bogost says something-- gamification is marketing bullshit as a means to capture the wild, coveted beast that is video games and to domesticate it.
Games are a mysterious, magical, powerful medium.
He basically says there's this nature of games.
This is how games are.
They're beasts.
They're this thing.
And then suddenly, they try to tame and domesticate.
What he's basically saying is games belong to us, the game designers, and the gamers, and the game academics.
But you marketers, please stay out of our playing fields.
This is not you should lose games, which obviously forgets that games have been used for all kinds of purposes for the longest period in history.
The art game has [INAUDIBLE] and fluxes where Ian Bogost, with this game [INAUDIBLE] definitely also belongs to, or be it war gaming, which basically goes back to Sun Tzu and The Art of War, because Sun Tzu, The Art of War, partially was based on the culture where samurai were taught strategy and cunning with good things of [INAUDIBLE] as a board game.
So the idea that you could use games for other things than entertainment is culturally very old.
But still, we have suddenly this new cultural debate about how should we use games in society?
What should their place be?
And even deeper underlying that, what kind of society do we want to live in to begin with, and how do we get there?
And as we were noticing these kind of clashes of rhetorics, we found two theoreticians very useful in mapping the total debate.
The first one is this very unlikely, with Victor Turner, of the [INAUDIBLE] to theater.
And he's an anthropologist, very famous for his early work about ritual as a social phenomenon for all of those pictures.
And he noticed that in pre-modern cultures that was played obviously that was also played, the play was usually very well coached and functionalized in these cultures, namely in passage rights.
Passage rights, he says, were these liminal spaces, boundary spaces where people would temporarily leave their existing space in society, and play around, and act excessively to then be reintegrated into society into their new role, usually in their passage between being a child, and then being the responsible adult.
You still have that kind of thing in the Amish [INAUDIBLE] in certain Amish communities where once you're considered to be at the age where you can become an adult, basically, you get to leave the community, do all the things you were never allowed to do in an Amish community-- jump around is the literal translation, [GERMAN]; it's from the Germans-- and then make the decision whether you want to return.
So you get to play around very literally, as [INAUDIBLE] said, in these places.
But this is completely functionalized as a ritual to reproduce society.
And what we noticed is in modern societies, singular societies, these liminal spaces disappeared.
We don't have a lot of them, because we're not a strongly religious society anymore.
But there were societal spaces that retain forms and shapes of that, only they're used and functionalized in different ways.
And to notify this, to indicate this distinction, he makes a distinction between liminal and liminoid.
He says where it's liminal spaces, like passage rights, like the pre-modern, religious, and collective things that are obligatory.
We have to do them, their play is instrumentalized.
It's a means towards the end of getting you into society.
It reproduces the existing social order and improves the means.
It's a means towards producing society, whereas limioid spaces play becomes something that's not instrumental, but also telling, done for its sake.
Play becomes something where you can change the order, even if temporarily.
You can upset the social order, and play becomes a space where you can question the end society pursuits.
Should we get through this, instead of just reproducing it?
Yeah?
Well, what did you say about the [INAUDIBLE]?
What is that?
Autotelic is from the Greek.
Auto, self, and telos is goal.
And autotelic means something is its own goal.
I do it for its own sake

I eat because I enjoy eating, not because something else, as an example.
So he notices that we have not these ritual spaces like passage rights so much anymore, but we have these liminoid spaces.
These are spaces like art.
He uses bases like science, where you get to toy around with ideas.
These are spaces like theater.
These are spaces like play.
These are time play.
All these spaces retain these ritual functions, but they're somehow used differently.
Give you a very practical example.
Art-- religious art in pre-modern times was a liminal space that just presented and reproduced the religious order as it should be.
I'm just showing you how it should be, [INAUDIBLE], and so on, whereas modern liminoid art is allowed to toy around with this and use the imagery literally, play around with the imagery of previous pre-modern art, remix it with Coca-Cola presents the crucification of Christ to make critical, soulful commentary, to speak about is this the kind of commercialized society in which we live and should be living in that kind of society.
And we found this distinction between lininal and liminoid helped us very well to tease out the different political positions people had towards communification.
The second poll that we found was very useful is from another book that most of the game studies people know, that is [INAUDIBLE], a French philosopher in the 1950s, Men Playing Games, [FRENCH], or [FRENCH]?
[FRENCH]?
[FRENCH]
Yeah, where he said that basically, all kinds of play and game forms, that we have a culture, a loosely folding on a spectrum of two poles.
One pole he declares with the Latin ludus for basically a rule based game that's very orderly, where you strive in order to reach goals; every typical board game, or social game, or video game that you can think of on the one hand.
And on the other hand, you have this free form improvisational, explorational, tumultuous play.
Think about children playing pretend play, or Lego.
So all games and play forms have basically these two ideal poles in them, and things fall somewhere on these poles.
And we notice that if we take these two dimensions and look at all the different rhetorics-- and we could talk about the rhetorics later.
But if we take all the different rhetorics there are, then they pretty neatly form this clear clustering.
They fall into two clusters of opinions which are these underlying rhetorical disagreements about what games and plays should be in society.
On the one hand, you have people who very much like games-- structures.
There's more structures, more rules, more feedback systems that are very liminal, but not liminal in the traditional, pre-modern way reproducing religious order, but liminal in a modern way, reproducing the existing social order of society.
Basically, their position's the current social order is good.
It's perfect.
Society should be as it is, but it has some flaws in execution, and we should use gains as technologies that governments and organizations can use to perfect the execution.
It's not that the way organizations or businesses organized as broken.
It's just that people aren't motivated enough, so games are a great tool to motivate them more.
I thought maybe we should fix the organizations themselves, whereas the opposite poll, most people who say gamification's bad are the ones who more or less see the current social order as bad.
They are typically progressive and critical in general in their political views.
But say play-- the practice of play, like the [INAUDIBLE] thing is a great practice for people to resist the existing order and discover together what kind of world we want to begin with and to give you just some ideas of these different rhetorics that we find.
So some of the rhetorics are, for instance, things like well being.
That's people like Jane McGonigal, people like Miguel [INAUDIBLE], people like myself that speak about self determination theory, [INAUDIBLE], the good life.
And we all say well, what's the good life that we all aspire to?
And obviously, in light of that, our current society's pretty broken.
But here's the thing about play.
Play is a realized form of the good life, so we all strive to get there, or things like performance.
So these are people who do pervasive games or who do LARPs, like action role playing games; all these much, much more free form play form to say, well, these are great activities for people to get together and experience that they can form a community, even if a temporal community together.
And on the other hand, you have rhetorics like reinforcement, so people like [INAUDIBLE] and others who raw on behaviorist modes of thinking or behaviorist modes of thinking about games and say games are basically these incentives doling machines.
And so therefore, they're a great way of reproducing or incentivizing certain behaviors.
Or a bit more complex, nudging.
A lot of people say games are basically nudges.
Just a show of hands, who knows the term nudging as a technical term?
OK.
So who knows behavioral economics as a term?
So neoclassical economics basically says people are rational actors.
Give me the right information.
Give me the right incentives, and I will do what you want me to do.
If I know this is helpful for me, and I care about my health, I'm definitely going to act healthy.
Only then, psychology showed us that is not the case.
You can give people as much information, and they might care as much as they want about their health, but they still don't behave healthily necessarily which led a lot of economists saying to us well, why then do people behave unhealthily?
And they started to run empirical experiments where they say, this is how economic theory predicts how people will behave, and here's how they actually behave.
So there is a gap there between the two, but this gap is predictable.
It's a repeating.
It's a reoccurring pattern.
One of these reoccurring patterns, for instance, is temporal discounting, which is just a fancy technical term for I care for the thing right in front of me much more than for a thing way in the future.
If they did actual economic disbursement to people saying, do you want $1 now or $100 a month from now?
A pure rational actor would always take the $100, let's say.
Because again, in no way, shape, or form will inflation, by that point, have invalidated the $100 that much.
But they found the majority of people took the $1 now.
And so they enlisted all these kind of cognitive biases and heuristics, these weird ways in which we deviate from actual rational behavior.
And then they said look at games, at least that rhetoric.
They said look at games.
They're basically machines for exploiting these kind of reliable flaws in our thinking.
And so therefore, games are great for us to learn how to exploit them to get people to do other stuff, as well.
So so much for that.
I'd like to finish, as I said, with teasing out scenarios, so what are the visions underlying here then?
Then we'll get into debating.
Namely, what are the positive or negative visions that come out of these different rhetorics?
Starting with the positive one, on both sides that we face, you can slice and dice them as you want.
I slice and dice them in three positive scenarios, and lots of issues with them that we can no longer debate.
The positive vision, the best possible world version of engagement-- the first one is engagement, which is a terrible, over-used term.
But it's basically the idea that we can harness all the three cycles, all the free energy, all the cognitive surplus', [INAUDIBLE].
Thanks to the fact that people are now online, and no necessarily always watching television, we can use all the hours they poured into Wikipedia for really productive purposes.
And then people like Jane McGonigal said well, that's great.
If we could use games, you can learn from games.
And she put in this paper, because it's basically the white paper that underlies, "Reality is Broken," which she wrote back at the institute for the future.
We can use games as engines for a massively [INAUDIBLE] collaboration and participation to organize and incite people towards productive things like solving complex scientific issues, and science games, or stuff like that, or use games for civic participation, or the aspect of games, like DIY democracy, where you would get points for being an active citizen.
The one issue-- lots of issues, but one issue I have with that is even if that would insight engagement of people-- for instance, use this to report pot holes in your street, it would always structure participation of people in a certain manner.
So if we see this as a vision of society, we have to be aware that every structure for participation creates structures of who gets to participate, who's included in the executing.
You can also say who has technological access to these kinds of tools?
Who runs around the whole time with a smartphone?
Who even cares about this?
Who has the technological literacy to use these tools?
That is with all the citizen participation tools, who actually uses this to report pot holes?
And again, you find that this is very much stratified by socio-economic status.
The second positive vision is community.
That's what I talked about as performance is the idea that we can use games and play to bring people together and to experience hey, we like each other.
We're pretty much alike.
We want the same things, and we think that this thing in our society is built, and so let's together get together and fix it.
So it's a way of bringing together people into new forms of community.
This goes back to early movements, like the new games movement back in the Bay Area in the 1970s which underlies a lot of the still Utopian thinking in Silicon Valley where they basically said with the Vietnam War going, can't we create non-competitive games that get people together in order to understand that that is also a form of co-existing instead of constantly combating each other?
And as Stewart Brand back then said, you can't change a game by winning it, or losing it, or refereeing it, or spectating it, because then you're still playing by the rules.
You still have to create a new game together to leave the old society and to move on.
And contemporary examples of that are things like you're guerrilla gardening, and there are games around guerrilla gardening as a community.
Or my favorite example is Massively Multiplayer Soba by tilt factor.
The treasure hunt through New York, through one of New York's most diverse neighborhoods.
And the idea is the difference teams get sheets with ingredients that they have to bring to the final cookout, where people cook together, only the ingredients are written in different languages; Hebrew, Hindi, whatever.
So you have to go through this neighborhood, find people to translate the ingredients to you, and then tell you where you actually find all these local ingredients.
That way, naturally getting into conversations with people in that neighborhood you would otherwise never have talked to.
And then the game gives you extra points for bringing stories about food from that culture or bringing actual people from the neighborhood to the final cookout.
One issue again-- lots of issues, but one issue with that is that I'm always wondering to what extent-- these are basically again, things hipsters can do to make themselves feel well versus to what extent are these things that actually cite large scale social action or commodify your [INAUDIBLE], you'll find.
The third positive vision is hacking; hacking systems.
The underlying idea there, and a lot of the references will say well, if there's one thing about games that's great it's their systems.
Or one thing game's good at is systems, and one thing that you can do with games is you learn systems thinking.
Anybody know September 12?
One of the early news games, persuasive games.
The idea came out right after September 11th, shortly thereafter by Gonzalo Frasca.
You basically see this little landscape in some long, disparate Arab country.
You see some people running around in quote, unquote, "terrorist" uniforms, and then you just have this targeting circle.
And then you can choose whether you want to fire bombs with them or not.
When you fire bombs, inevitably you will create collateral damage in terms of people and other things.
And inevitably, people standing around that will mourn for the dead, and then some of them will likewise transform into terrorists.
So it's the systemic model of violence beget violence.
By attacking, you will only create more resistance, basically.
You may disagree with the argument, but it's a basic way of showing you how games can model a system and show a system and can [INAUDIBLE] systems.
So the argument of hacking, as well; games are great for systems thinking or learnings to this kind of systems thinking.
And by making games about the broken systems we live in, like in those airport and security, about constantly changing new rules; what is allowed, what is not allowed.
You notice how broken that kind of TSA system is, and by learning to build systems ourselves through game design and all kinds of tools that open up game design for other people, we might actually educate a generation of people who can then use that game design and knowledge in order to redesign the broken social systems in which we live.
The issue with that is obviously, anybody has yet to show me that there is data for that, that that actually happens.
OK.
Final three, dystopia.
The first one is obviously the one that you most likely will come across the most [INAUDIBLE] when people were talking about gamification.
That is this is a perfect control society.
And my favorite one, has anybody seen the talk by Jesse Schell?
The Gamepocalypse.
That's one of the other initial pieces of media that the whole gamification thing started.
It's a talk he gave in 2010 at a game design conference, and he described the future world in which game designers would rule.
And on, and on, and on it goes.
It's a five minute riff within the talk where he just runs with the idea and says, what if all kinds of organizations and governments would basically start to use point of incentives systems to get people to do everything they want to do.
And people say, this is silly.
This is ridiculous.
This is also scary.
The only thing is it's also becoming a reality.
Shortly after the talk, there was this sort of green goose which actually did sensor kids.
So you can use ubiquitous computing everywhere, especially to brush your teeth.
Some people are obsessed with brushing their teeth and getting points for it-- to measure everything, and then set up your own reward system for that, and you have these default senses that you can use for everything else.
And if you say well, that's nice if it's for people.
That's not what organizations do-- well, this year's one part of the health program that I was introduced to at Northeastern, which I find very cute, which is Virgin Pulse.
Basically, it's one of these health trackers, step trackers.
And if I use the step tracker for my university, hey.
I actually get points, and the points are translated into reduced insurance premiums.
Now you might say well-- a lot of my colleagues said well, this is great.
You're getting money, so why shouldn't I do it?
I said well, I don't know if I actually want either my employer or my health insurance to know how healthy or unhealthy my behavior [INAUDIBLE] myself.
And then the next time I switch an insurance, the insurance looks at the data and says-- hm.
We should raise your premium.
The larger issue behind that is basically that most of these systems are at face value are sold as more fun or rewards or empowerment to the individual.
Like Rescue Time is this application that you can let running.
And then it tells you, just by keeping in the back end, what kind of applications were running all through the day and what kind of browser windows you had open in order to discover what you're time syncs are in order to be more productive as an individual.
Yay.
Only then there's this nice button, Looking for Rescue Time for Entire Team or Organization, click here.
And then you see the management feature which allows the manager, who installs this all across the team to see exactly what everybody's actually doing on their computer the whole day.
And obviously, this issue is true everywhere we use gamified applications are installed, which according to the vision, will be everywhere.
So you can even imagine something like Amazon saying hey, because we want people to read more books, we're going to deal out virtual rewards and incentives for people to read more, because then we make more money only while I do so.
And then I read all the volumes of Das Capital, somebody else might also be interested in that kind of data, that I'm actually currently interested in Marxism.
The third issue there is that face value of fun and self realization at companies.
And all Silicon Valley companies have to have sleds.
This here is at YouTube.
Obviously, a YouTube red.
I recently did a bit of research around playful offices.
They really have all sleds somehow, for some reason.
We can talk about that.
But that, in the back end, is really covert exploitation.
We make it so fun and so comfy here in this place that you never want to leave the workplace, at which point then, you get suicides and exhaustion in places like the Las Vegas downtown project run by Zappos there, or as [INAUDIBLE] nicely put it, gamification is a mechanism of decoupling alienation.
I feel like I'm not realizing myself.
I feel I'm not doing what I'm passionate about from exploitation.
With gamification, I may not be alienated by my working, or this is fun.
This is great, but I'm still exploited.
The majority of the revenue still stays with your employer.
And the big issue with these systems is they're all implemented in code.
This is all codified, algorithmic, automated regulation.
And the big issue with that is you can't argue with it.
If you have an overdraft on your bank, you could still go to your bank teller, depending on which country you are.
Here, they are typically very neo autonomized, but in European countries, not.
And then you can say listen, you've known me for 10 years, and I only need an extension for another day, and I'd pay the money back, and he does so.
We know there will always be exceptions to the rule in reality.
The only issues with these algorithmic system.
I can't argue with them.
I don't even know according to which rules they tell me yes or no or give me certain rewards.
And I can't change that, contrary to other political systems.
So basically, the control fear is that it's on the one hand, big brother watching you, because it's constant surveillance in the back, and all your data is captured.
It comes with the playfuls of veneers, so it's the Aldous Huxley soma.
All this is fun and happy, and why should we care about it?
And it's a little bit overload, because all of this is run by computers.
It's not even run by people anymore.
As I said, this is going to be the worst case scenario here.
The big issue obviously, which I pointed out in text here is that these kind of control systems destroy the very thing that makes games fun to begin with.
They give you the freedom of playing, voluntary choosing to the activity.
Second, and third, and then we're done with negative vision.
The second negative vision is gamification doesn't even lead to more control in society.
It just reproduces existing society as it is.
It reproduces the systems and the rhetorics that we have in our system.
The overarching narrative, if you look at persuasive technology with the quantified self, [INAUDIBLE].
The overarching rhetoric of all of these systems is when we have social issues or when we have individual issues like obesity, like climate change, or stuff like that, it's the individual's fault.
It's the individual's fault, because all of these systems address individual behavior.
If only I motivated you to run a bit better, you'd not [INAUDIBLE].
If only I motivated you to drive a bit more fuel efficiently each day, we wouldn't have global warming, basically.
That's the argument by addressing these things through specified applications.
So basically, the underlying rhetoric there is this very American one.
Anything can be accomplished if one has the desire and determination.
If there is a social issue, if there is an individual issue, it's because the individual didn't have the willpower.
And basically, these technical tools just give you an augmentation to your willpower by motivating you if you can't motivate yourself.
Is there anything wrong with this rhetoric?
Well, let's see.
There are other places where the same rhetoric was around, like Maoist China, where we had slogans like when discipline's reinforced, revolution cannot fail, saying if the great leap forward trying to mass industrialize a country of several hundred million people within the term of five years, if that kills millions of people, it's not because the planet was bonkers to begin with.
It's because the individuals didn't put in enough effort.
In the same way, I think gamification and lots of persuasive technology makes the same issue of the background systemic root causes.
The deeper underlying societal issues that are not caused by individual behavior necessarily or that are not just [INAUDIBLE] on the individual weather.
You'll interpret the [INAUDIBLE] of environmental psychology that recently made the same argument where it said, we're basically fooling ourselves if we believe that getting people to recycle more will solve any of our environmental issues, because municipal loves production, just as an example.
So all the waste that everybody of us produces in a city that's collected by the municipalities, all the waste that we produce-- how much is municipal waste production from total US waste production?
What share?
Any numbers?
10%
10%, more or less
5%
5%, more or less
2%?
2%-- too low-- 3%.
So if you all recycle more, we're reducing 10% of 3%, whereas most of the waste productions happens in industrial production.
Most of the waste production happens in supply chains.
It doesn't happen at the level of individual waste.
Same goes for energy savings.
Same goes for all the issues.
We're addressing the wrong things with this kind of thing if we're always focusing on the individual behavior.
Third and final one is it's not control.
It's not even reproducing existing social order.
It's just reference.
My favorite example for that is the PlayPump, a system invented by a civil engineer in South Africa sometime in the 90s.
And in 1994, a retired advertising executive discovered it and said, this is a great idea.
Let me buy the patent, and let me make this big.
The idea of the PlayPump is basically lots of cities in Africa and other developing nations don't have enough fresh water pumps.
So what they said is hey, instead of the traditional fresh water pumps, let's install a fresh water pump that's actually a roundabout and put a reservoir on top of it that's seen here, behind.
And so water is pumped for the local village's water consumption by the children playing.
Nobody has to work for the water.
The children enjoy using the roundabout.
They will pump the water up.
And you know how we pay for this whole thing?
We use the reservoir as an advertising space.
So the advertisers will pay for the installment of this.
It's a great story.
It's a great new story to say hey, water is easy as child's play, dah, dah, dah, dah, dah.
It got a great, favorable PBS documentary, lots of other favorable press attention.
He got then $60 million US dollars in aid promised from US aid.
Only then, development aid workers started raising their hands and saying, sorry.
Nobody wants to advertise it to the developing nations, it turns out, especially not in rural villages.
And they found out that the pump is more difficult to install, less efficient, more difficult to service than existing water pumps in other cultures.
And they found that the average water consumption of an average African village is so big that children would have to operate the roundabout with their body power for 27 continuous hours per day.
So it's impossible.
It's impossible that children can produce enough water [INAUDIBLE].
So therefore, it ended up that the adults were operating the roundabout, only the roundabout was built for children.
So the adults always had to hunker down and use this one, so they were breaking their backs as they were doing so, very literally.
So this thing is a great new story that gives you great pictures with all kinds of things.
And the argument, or the fear, or the negative vision here is that gamification, like the PlayPump, is very much this.
If companies, or organizations, or governments talking about we're using games to power, engage, and dah, dah, dah, dah, dah, or when the current administration says we're going to put constant [INAUDIBLE] into to the White House Office of Science and Technology Office in order to run games for brand challenges, this is not so much because it actually changes stuff, but because it is, in Elizabeth [INAUDIBLE] word, the [INAUDIBLE].
That is it's nice sounding words with no meaning, whose only purpose is to appear cool and modern as organizations.
Enough talk for me.
The rest of the talk should be about exactly that, or you think the proper place in plain society has .
Thank you.
[APPLAUSE] I left my water up there.
GABRIELLE TREPANIER-
Well, thank you.
It's very interesting how you really frame different kind of rhetoric about gamification, because that's really the way I feel when I read the three texts that were at the program for today that was Zichermann's text that was really what you said, like the type of rhetoric that is very Utopian, conception of gamification, something that allows engagement, and so on, and so forth.
And there was this very distopian text that we read about that was a Schrape text, and we were thinking gamification that was linking gamification with control, governance, surveyance at some extent.
And then that there was your text, so I don't know where you rate yourself on the scale of utopia and dystopia.
I felt you were in between trying to compromise all this, but maybe I'm wrong
Multiple things.
So A, I am aware that I am, myself, in place there, because I have my own politics.
I have my own values.
I just tried to be explicit to compensate.
Listen, this is the background from which I'm coming.
And viewed through this lens, this is what I see.
If you look through a different lens, you might see something else.
So therefore, I say, I'm personally a bit more interested in play than games, although [INAUDIBLE] games are structures that [INAUDIBLE] place, that they're not unimportant as that.
But it's the same way as you say ultimately.
It's men hammering and hammer, hammering and nails, but it's still nice to have a hammer to do something to do with [INAUDIBLE].
So even if a play's important, and in the end you want the hammer, but the nail in the wall, it's nice if you have a game to afford play, because that usually makes things easier.
So in that sense, in that spectrum, I'm sort of in the negative.
And then definitely, in terms of my values, I come from virtue ethical errors to the [INAUDIBLE] perspective.
So the question, what is the [INAUDIBLE]?
I just think this is my framing.
And viewed through that framing, I think, play presents as a vision of how living could be and what we should aspire to.
And other people might have other's points.
So I wouldn't view myself as Utopian or dystopian.
It's just I have a view for this.
I think there is potential.
The big issue for me, as I said, I also design [INAUDIBLE].
And parallel to studying this stuff, I've been running around, doing workshops, or actually designing work with companies, like BMW, the motor company who said, can you gamify for assembly training for us?
OK. And that drove my interest.
Is there a way of doing this right?
Is there something we can actually learn from games?
And I think that is the case.
It's just not what the current popular conception of game indication presents as what we can learn in the case.
Those are just two different things.
When I now come into an executive board room, basically three years ago, the challenge was games?
What are you even doing?
So making people understand the concept versus the potential.
Now the issue is banks to immediately [INAUDIBLE] of the concept.
That therefore, they all have already an existing preconception of what this means.
Oh, fee [INAUDIBLE].
Yeah, sure.
More of that.
And then you have to break that conception.
Say yeah, I know, but that's also something, but that's not what you actually can learn from games.
So that's my perception on this issue.
So questions?
I had a couple of initial questions from the text [INAUDIBLE].
I've clustered them into three clusters for me that we could go through.
But other questions, as I was talking-- I'm just starting collecting, and then we give them to a [INAUDIBLE] other questions that popped up for you.
It's late, I know
Perhaps it's better if we group them in categories?
Yeah.
I would do that.
Or if you need energy, we can do a little game to get the energy going.
Happy to do so.
I was serious.
GABRIELLE TREPANIER-
I have many questions
OK.
So here's a thing.
I actually would suggest doing that, if you're all OK.
If you don't want to, it's fine.
You can just sit here, but in order to get the energy up, because it's actually late, I suggest all of us go out on the floor for two minutes at most.
That's all I ask.
And again, if you don't want to participate, it's perfectly fine.
GABRIELLE TREPANIER-
It's their spontaneous reactions to what has been said, because it was slightly different than what we read
I know
And perhaps that's why
I know.
GABRIELLE TREPANIER-
We can also ask you questions about the readings
You can ask me all kinds.
So the three questions I took out of the readings, and I'll just list them as far as I understood them, and then you can add ad libitum, and we'll start a discussion about this.
The first one is just when is a game slash gamification?
It's really this definitional debate.
Is it still the game [INAUDIBLE], or is this already gamification?
Or when starts gamification flipping through a game?
That was part of the questions.
The other one was this yeah, yeah.
There's all this criticism about this thing.
But what is good, as in useful, actually applicable learning from games slash gamification?
So is there-- this is all criticism.
Is there actually a way of [INAUDIBLE] the situation in which it can be used?
Maybe just a specific audience and get the audience's change based on more peole get used to games, and stuff like that.
And the third question is basically, I don't know who that was.
Somebody said, can I use that in my startup?
I know it's very risky, but you pretend like there's a question.
It follows from the other.
Can I use it, or when can I use it?
When can I use it?
So those are the three.
If you feel misrepresented or left out, let me know.
Or if you feel no, based on this one here, or now that I wrote them down, other stuff comes to your head.
So we'll just collect a good one.
Yeah?
I was just curious with the [INAUDIBLE] space you had made earlier between the styles.
Is there anything that exists in the other two quadrants really?
[INAUDIBLE] something's on the line, but so there isn't anything between?
That's a very quick one.
Yes, there are two, but they're outliers.
So if this was liminal-- so basically reproducing society, and this was liminoid, questioning being outside of society.
And this one here is [INAUDIBLE].
So this is games, whether it be [INAUDIBLE], and rules, and feedback, and stuff.
And this is [INAUDIBLE], so this is free play.
Everything should be Lego.
And as you said, there's the big cluster here, and there's the big cluster here.
There is one cluster here that's basically Ian Bogost and a couple of other people.
And then there's this cluster here, which would be called industrial playfulness.
This classic procedural rhetoric is basically we can use games, with a capital G as systems to model other broken systems in society.
This is how airport security is programmed.
This is why immigration's broken.
However, you make a system in order to make an argument and invite people to see how the way this game models a piece of society.
Fits, or it doesn't fit my mental model of that, how that piece of society works.
And me negotiating that clash between well, I thought airport security works this way.
This thing drove me that way.
How do I make sense of that?
Based on that clashing, you start thinking about society, and then maybe come to a different idea or come to criticizing yourself.
So this is basically a version that's deeply critic or liminoid in its politics, but still says let's use games for that.
The other one in that corner, industrial playfulness, these are things like innovation games or game storming.
Here's a recent [INAUDIBLE].
These are people in design agencies like IDEO or Frog saying, we can use the playful activities, ways of structuring processes in order to reliantly produce creativity.
The great thing about design agencies, and I've worked with a couple of them is that they can promise enterprises to produce innovation accountively.
You give us x amount of money.
And in y amount of weeks, we have produced that amount of [INAUDIBLE].
And we use playfulness.
We use yellow rooms, and we use brainstorming, and we do these silly design exercises in order to do so.
That's very much used in play, but very much for existing structures.
Other questions?
GABRIELLE TREPANIER-
Well, I have one.
I wanted to talk a bit about gamification in workplaces, because I think that a lot of students raise these questions and on the forum.
In your text, in rethinking unification, you say that most of the gamification practices and workplaces disregard the context in which people come to play and ignore that play is supposed to be a voluntary act.
So you mentioned that forcing workers to play decreases that positive feeling of performance, and that having to play in an appropriate context can lead to embarrassment, right?
So I think I was not the only one to ask myself, does it mean that gamification techniques should never be used in the workplaces?
Is the work context not suitable for gamification?
Or can gamification be optional in a workplace?
I think that Ryan-- not Ryan-- Shawn, he had this similar question.
I don't know if you want to elaborate on it
Oh, yeah.
I was just disheartened reading even a somewhat feeble attempt to incorporate gamification--
Yeah
--so as far as the worker's reception went.
And I was wondering, is there a way to include playful elements, like an opt in strategy?
Because I would enjoy a playful work environment, or that sort of thing that they see as an electronic web
OK.
I would tease out two things there.
That is I always like to use the technical term, autonomy, because it has an established meaning.
There is an established theory behind it, and psychology was voluntaryist, freedom; very overdetermined [INAUDIBLE].
This one, I use it, and then I point to a certain theory, and that's why I prefer that term and to talk about this.
And autonomy is still a muddled mess, in terms of work life and games.
Can we go to that?
And then the second one, can we learn from games in the work place in general?
So disregarding the voluntaryist issue, are there ways in which we could make life and work anonymous?
Let's speak about the autonomy one first.
There is this interesting early book, Julian Dibbell, games journalist who now became a lawyer.
Before time, he was a great journalist in the late '90s, looking at all the online games craze, and he wrote a book.
Rape in Cyberspace, and all these kind of books.
So it was in plain money, how he basically made a living with designing virtual [INAUDIBLE] space.
And he noticed he was one of the first Western observers who sold gold farmers.
For those who are not familiar with the term, gold farmers are basically people, supposedly always Chinese gold farmers.
And then there are other articles saying that's stereotyping, because we also find in other places-- anyhow, Chinese gold farmers are people who level up characters or win virtual items in games for a living.
And so I gain a powerful sword in World of Warcraft, and then I sell it to other people.
And usually, these kind of gold farmers themselves don't do it, but it's other people hiring poor people off the street in China, young people to basically play World of Warcraft.
It's day by day, and day in, day out in 20 hour shifts or more, and just sleep.
They go back and play as a waged war.
So the more gold you play, the [INAUDIBLE] played for the virtual gold in real money, which is a classic form of peace work which other people, if you've ever done peace work in an actual assembly line, so you know that's pretty excruciating as an activity.
And he found two weird things.
A, he said, they were playing.
They were working in this game.
And according to standard theory, the should have hated every minute of it, working this game.
But it seemed pretty playful at times.
At times, it also seems pretty stressed.
But at times, they were still playful in that context.
And then he said the weirdest thing is when they handed the day off, they would hop into the internet cafe next door and play games-- the same games.
And so how can that-- I'm puzzled.
I can't even make a story to begin to make sense.
Why would they do the thing voluntarily that they were forced to do there?
Why would the forced thing sometimes still be fun to them?
And the part of the argument, or part of the way that you psychologically can tease that out is say that autonomy has a lot to do with your internal perception of the situation.
That is if in the moment you feel like this is what I want to do, I currently feel like doing this.
And so therefore, I enjoy this.
This is what I want to do.
You don't have to force yourself to do it.
It feels like this is a true expression of yourself.
And so therefore, it feels autonomous.
if you go to a cloister for a six week retreat, and somebody tells you at 6:00 AM in the morning, get up.
Sweet the stairs, because that's part of your retreat.
If you then think about I actually put myself into this oyster, and I want to do this, and I know it's a good thing-- you might still go-- mm.
But you feel this is an expression of yourself, and you still feel that certainly this is a voluntary act.
This is still an autonomous act.
And you might still find a way of getting absorbed in the activity of [INAUDIBLE], and then you forget the fact that you're actually tired.
So the degree to which I perceive myself that this is currently an outflow of my goals, wants, and my identity is chiefly responsible to whether this is autonomous or not for me, as an experience, which is something that [INAUDIBLE] and [INAUDIBLE], author of [INAUDIBLE].
One of the core books in game studies [INAUDIBLE] has found where he said even in concentration camps, people could be absorbed in the activity.
People could feel a degree of control, voluntarism in autonomy, because they chose how to deal with the task that were prescribed to them.
Even if they work for the [INAUDIBLE] that lent in their ultimate demise.
So it's this what internal stance do I take to what's my situation?
So it's not that because workplace, therefore, obligatory context, that are therefore necessary in autonomous.
It's how do I perceive this to be an outflow of my personal self?
And there's been research, especially in education how people can facilitate an autonomous orientation towards an activity, even in the mandatory context, like school.
There's stuff teachers can do to tell the students, listen.
I know the next exercise is shitty.
Sorry for that.
Here is why doing this is still useful and helpful for you personally.
And I would ask you to do it, because it's helpful for you; asking permission, explaining why, acknowledging that it's tedious-- all these giving me free options how to do it, with whom to do it, how to approach the problem; lots of ways in which you can structure an obligatory situation to still be autonomous [INAUDIBLE].
So it's a complicated psychological thing that has a lot to do with what is my internal stance towards it?
But there's a lot of design things around the social situations we still can do, which leads us to the second thing.
Is there things we can learn from games in order to make a work environment playful?
As I said, with the sleds, I looked into their research, because I was invited to give a talk at a conference about office furniture.
They wanted to gamify office furniture.
Sure.
And so I looked into the whole thing and said, how do people gamify offices and office furniture?
And notice that it's actually a big trend, the playful office.
And if I say playful office, just what are you thinking?
What images jumps to your mind?
Playful office
Google
Google
Nerf guns in the office
Nerf guns in the office.
What else?
Ping pong?
Ping pong?
Ping pong in the office
Ping pong in the office.
Yes, what else?
Insert fun thing in the office.
Bean bag chairs and other horrible-for-your-back furniture
Yeah.
What else?
Playful office, just off the top of your head
Free food everywhere
Weird looking, ergonomic furniture
Not working around my furniture, free food everywhere, plants.
Yeah, plants are totally important, even fake plants
Scooters
Scooters, yes
Friendly bosses
Friendly bosses?
Then we're getting into different topics.
Yeah, exactly
Very scary company open work spaces without cubicles
Yes, open work spaces.
The weird thing about cubicles is in terms of their design history, their point was to be free.
Back in the 1960s, people started working on designing the cubicle, and their point was to actually create playful architecture.
Their point was to make this is a way in which individual workers, without big effort, could continue to redesign the office to their own needs.
They would just take the balls and rearrange them to huddle their space as they see fit for a project or for individual close time.
Only then the corporations discovered hey, we can also use cubicles to give people a bit of sound, and visual privacy, and cram the most possible people into the smallest possible space.
Cubicles where initially invented in order to be a more free and open office work, and then they would reuse the floor very differently, which points to an important part this is what you put in there as a design element is often not so important as how people use it in practice.
The practice is the way in which people live something is just as important as the things you put into the office, like bean bags.
So no.
All of these things are true.
Bean bags usually will find everything that screams playground; bright colors
Slides
Slides, natural textures.
So you have fake grass, real grass.
You have sand.
You have round shapes.
There will always be round shapes and all these kind of things-- things that look playful.
But structurally, if you ask yourself what makes a toy?
What makes a good toy, or the opposite of a good toy.
What makes a good toy?
Any suggestions?
What are toys that you actually enjoy toying around for a long, long, long time ago?
Mhm?
Call of Duty?
That's a game.
A toy.
Call of Duty can be used as a toy.
We'll get to that
Simple, multi-purpose objects?
Kickballs, and shopping lists, and buckets, and stuff like that?
Yeah.
Wide shock.
GABRIELLE TREPANIER-
It stimulates creativity?
Because you can use it for more than one thing
You can use it for more than one thing
Because when a teacher's not looking, I could hit people with it.
But if it's still the other stuff with it, [INAUDIBLE] like oh
Other suggestions?
Nothing.
Nerf gun?
Why?
It's fun
What makes Nerf guns fun?
Give it to a couple of people
Yeah
And then it becomes fun, all of a sudden.
[LAUGHTER]
Again, Nerf gun, as such, is not fun.
Fun is when the people actually decide to go around and--
Oh, I see.
I see
And I would say the Nerf gun invites people to actually temporarily go out of bounds and go out of their regular rules.
So a lot of playing--
I was going to say a Barbie doll
Barbie doll.
Why Barbie doll?
Because you could basically do whatever you want.
You could play house with it.
You can make up a story with it.
You could act out going to work.
You can pretend that your Barbie doll's having a wedding.
[INAUDIBLE] sort of thing
I think it totally solves the blank canvas problem that involved-- if I say I want to play something, but I have nothing in front of me, I'm not going to be inspired to do anything necessarily.
But if I have a toy, [INAUDIBLE] it's a Barbie doll, oh.
It's a person.
I can pretend it's a person.
I can play like it's a person.
Or a Nerf gun gives me, oh, it's a gun.
I can pretend it's a gun.
It gives me something to work on though
Mhm
It's like modular things.
It's like those K'nex
Yep
Little sheets, and marbles, and things like that
Yeah.
You teased out very successfully, the two [INAUDIBLE].
The one thing is it involved a purpose.
It's underspecified.
It doesn't prescribe one specific way in which [INAUDIBLE].
Think about Fisher Price will often [INAUDIBLE] toys.
They're the worst possible toys imaginable.
They just have a number of buttons, and then you hit the button.
It makes a certain sound, or it flashes a certain light.
And I discovered yeah, that button always produces that light.
That button always produces that sound until it becomes a social experiment where you only try to find out which is the button that annoys mom and dad [INAUDIBLE]?
That's fun.
The thing itself is not necessarily fun, so it's underspecified.
There are loads of different things you can do with them, and you can explore this possibility space, which is the possibly of space in Legos, or in Barbies, or in any other thing.
And in addition, it gives you a constraint or a starting point.
It's not just a blank canvas.
It gives you a seed for the exploration of that [INAUDIBLE], whereas most of these other spaces, or like Fisher Price toys [INAUDIBLE].
They look playful.
Colors, [INAUDIBLE] sleds.
But they are not an underspecified possibility space.
They are not things that you can use in very many ways that invite you to reuse, and abuse, and jump over existing [INAUDIBLE] how to use this thing as you see fit.
And if you look into what acclimates [INAUDIBLE] office architecture, you find stuff like actually, what's the building-- building 20?
The historically famous building where most of the MIT innovations that you were involved with--
Yeah.
Quite a different [INAUDIBLE]
Yeah, exactly.
That building.
And when [INAUDIBLE], some people here at MIT, looked at what made that thing such an incubator, apart from the fact that it was a collision space for lots of people in different disciplines, they had no way but to bump into each other the whole time.
The second thing is it was a barrack.
And it was such a run down barrack that nobody cared if you wanted to have this big machine in there that took three floors that you just [INAUDIBLE] your balls through, forced through, which was that everybody was doing it.
It was a modulous case that everybody deems as they see fit for weird things.
So the question, can we learn from games for office work to make it playful?
Well, if one of the things that invites play is an open possibility space for exploration where I get to choose where it's autonomy supported, because I have a lot of choices what to do.
Give me an office space that works like that.
Give me working rules that are like that, like results only work environments.
I don't prescribe to you how you do it.
I give you a goal, and you choose who you want to do it, how you want to do it, et cetera .
If you feel that is too little structure for you, then the good game designer would give you a bit of scaffolding and say, then come back to me as a supervisor, and I give you a tip.
But other than that, go out.
Just come back when you have it done.
That would be one of the many ways you can learn from games that are not about having a feedback system
I play a lot of games, and we move towards implementing trying to make an exterior to the game economist and things like that, and trademark [INAUDIBLE] exteriorly, and start to model the real world in that sense.
What do you think about a lot of that?
I think you gave it as the other half of the equation, where you take the real world into the game, so to speak, where people now dish on commodities, and so a form of betting is they can't bet real world items
It's interesting.
It's the point where I am then the sociologist, and you start to notice.
We have certain cultural notions about what games all should be, a traditionally modern notion, which is this leisure time activity.
It's voluntary.
You can't gain or lose anything by it.
The moment you do that, it's gambling.
It's not really gaming.
That's a different thing, gambling and gaming, although the gambling industry, because of that always wants to say, no, no.
Gambling is gaming.
But culturally, we make that distinction to say the moment there is something at stake, it's not really gaming anymore.
And the interesting thing with all of these things is that it basically upturns existing institutionalizations of how games traditionally happen.
And then people look at this thing and say, how the heck are we going to make sense of this?
And how the heck are we going to regulate it, or legislate it, and what's right, and what's wrong?
So first of all, I just find it interesting, because like every other media convergence phenomenon, it just throws existing orders into relief, and then people have to scramble through the mess.
What else I make of the-- it's the same as freemium games, where you pay for in game advancements, where a lot of them, the traditional triple [INAUDIBLE] says no, no, no, no, no, no.
This is dirty.
This is evil.
You bring economic things into our shiny, pristine little temples where economics has no place.
And then as terrible as the concept of the magic circle is, what they're basically describing is in the old AAA industry economic model, they play games, the Call of Duties, money was admittance and allowed into that game space.
But when I'm in the game space, money has no role.
In the freemium world, you get into the game for free.
And then to get ahead, within the game, sometimes you have to pay money.
It's just a different way in which money factors into game play.
It's just that most people are used to this model and say game play should be free from economic considerations.
Only it hasn't been for AAA games either, because AAA games traditionally, because of the kind of business models they have, were built such that they gave great tradeoff for it, because that's the way you get a lot of people to do this.
AAA games are notoriously risk averse.
You just get Call of Duty 24/7, and the next version, the next version, or the next version.
Why?
Because traditional AAA games are so investment intensive; dozens of millions of dollars, that the people who give the money say, the only way we know we're going to recoup our investments is due to the very same recipe-- again, that we did before where we reduce the risk.
So in that sense, money influences what happens within the game as well, just in a different way, and in a way people are not used to, and people scream murder.
There are then different things with [INAUDIBLE] I would ask is that do I like that aesthetically or not?
That's a different [INAUDIBLE].
Based on my personal, ethical evaluation, do I think certain ways of free to play games are organized in a nice way or not?
And I would also say no, but it's the same phenomenon.
Mhm?
So I know you talk about using people or [INAUDIBLE] to [INAUDIBLE] you hide within games, and then you turn around and sell that.
What about games or situations where you end up making a lot of money, but not really intending to?
Like you're still out there playing the game even online or something and you find items, or you acquire assets that are then worth a lot of money
Mhm
And games like Dodo 2 and TF2 that now, [INAUDIBLE].
So people sell characters after a point, and then [INAUDIBLE]
Yeah
You play through a game, you get past a certain point of aggression.
You cite your character, and statutes like that.
Or like he was saying, recent games like Dodo and Counter-Strike, a lot of the battle games, where they try the policy that you can get drops that are worth large amounts of money by simply playing a game
So they're not necessarily purposely farming for items, but being aware that that's a possibility, and that also being part of the [INAUDIBLE] proclaim
Right.
From the outset I would say, whatever floats your boat.
If you, as an individual say that's something that I enjoy doing, or that's something that I want to do for all kinds of reasons, then feel free to do so.
So to what extent is that then still a game?
Or is it a game that I play is an interesting question, which basically brings us to when is the game a game, or when is the game a gamification?
The first thing I would do is as long as I, as an individual, play this predominantly in order to have fun, it fits the regular mold of how we would say that.
Incidentally, in the end, you find that the exhaust of what you can do, you can also set off.
It's just oh.
At that point, it's nice, interesting, and weird.
So let's try and explicate this one here, actually.
Just shout out.
When is something a game?
What are the conditions that have to-- or how to determine whether something is a game or [INAUDIBLE]?
Yeah?
I think that the biggest thing is the voluntary playing.
So it's a game if it's voluntary
Voluntary,
A reward system?
OK.
If it has a reward system, it's a game.
If it doesn't, it's not a game?
Not necessarily, right?
I'm asking you
Oh
You suggested a reward system.
I'm just asking
I think it is not unique to games, but games should definitely have a reward system
They should have one because--
Well, what's the incentive?
There should be one behind a game.
So it may be a feeling that you're getting smarter, or maybe you're getting points.
Or solving a puzzle.
There should be something that is captivating you.
But the reward system-- when you walk in doing your job, and you're getting paid for it, it's also a reward system.
So I feel like it's not unique to games, but games should definitely offer a reward system
OK. Is voluntarynism unique to games?
GABRIELLE TREPANIER-
No
No
I can cook for the enjoyment of cooking.
So we were playing a game outside.
Did you get a reward for that in your understanding of a reward?
It may not be something as trivial as hey, just get plus one point or plus two points.
But definitely, you could bond with people
Yeah
So yes, it was a reward
OK.
So I'm just making sure that you're--
Besides the energy of waking up--
Yeah.
I'm just saying that your way of understanding the reward here is something that is not necessarily tied to a reward system.
As you do x number of things, you get the x number of points translatable into the y number with something else.
So the suggestion is you award [INAUDIBLE] broccoli as there is a desired result, however you want to construe the desired result.
That's like economists say preferences, which basically means everything [INAUDIBLE].
Mhm?
I have a question.
Are we just saying when is something a game?
Or are we trying to differentiate between when it's a game and when it's gamification?
We'll get the gamification.
Now it's just when is something a game?
OK. Because then voluntary is not [INAUDIBLE], because I was thinking of it in terms of--
Ah
--is it a game or a gamification?
So a game is voluntary.
Gamification is [INAUDIBLE], or the other way around.
Why?
Because gamification is more of a system that I guess you could choose to be part of it.
So if you go to a weird place that does have a game like culture, or you're choosing to do that.
But then you could also inevitably end up in a gamification culture where you don't-- if everything becomes like that, or you don't want to be in it.
But a game, like say video games are playing.
I could have refrained from going outside and playing tag.
So it wasn't a voluntary action
Yeah
I don't know if I agree that it's necessarily not voluntary

So I think that games are that may be non-voluntary, but then gamification, something like it's a game if I just play sports because I enjoy playing sports.
But if I'm playing soccer because I want to get the statistics on my Fitbit or my pace counter for my health insurance, then it's gamification
What about professional sports?
Like I play soccer, because I earn $5 million a year for playing soccer.
And if I know if I lose that next game, I won't win that marketing contract [INAUDIBLE].
I really have to play well at this game.
Is that then, still a game?
Or is it gamification?
Or is it something else?
I can't play
I could, and that goes back to what you're saying about the economy, [INAUDIBLE].
If you say, it's really cool that I get paid to do what I love, yeah.
I think it's cool.
I have tons of money.
But also it [INAUDIBLE] like if you pay me half as much, I would still do it
Yeah
I would still enjoy this as much, then it's more of a game
So it's voluntarynist less in terms of the empirical conditions, or you pay the amount.
It's voluntarynism more then in terms of how do I perceive myself as I do this thing?
Which isn't important at some point
I feel that my time would be better used doing something else when it's a game.
It's pretty useless
Ah.
That's interesting.
So a game has to be a waste, is it, in some way
[INAUDIBLE] professional player, you have to make more money [INAUDIBLE]
Really?
Really?
What are you going to do?
Like look at it honestly.
Aside from some of the most recent tournaments and things like that where they've actually stepped up [INAUDIBLE], but a lot of that--
Like football players, right?
So yeah, you're stepping [INAUDIBLE].
Some might say conventional sports, and things like that, But what's your chances of making it there?
Well, I'm saying, if you're Tom Brady, you're not making any more money doing anything else.
It's a horrible example.
Brady keeps taking pay cuts.
Man, [INAUDIBLE].
You're not going to get any more money doing something else, but it's still something they find [INAUDIBLE].
Does that mean they don't think of it like a game?
I don't know.
That steps into the same space that we were just at before where we're asking a question of if you're playing a sport, because that's what makes you all the money, and it's almost no longer in your own choice to do it, because this is the best possible option for your utility.
You make more money doing this.
You're Tiger Woods.
You make more money by playing golf than mowing your own lawn.
So naturally, you pay somebody else to mow your own lawn.
Well, at a certain point in time, you make the most money by being filmed as if you were playing golf
I really disagree with calling the game a waste
Sure
And if I were to tweet that out right now to Anna Anthropy, she would kill me
Fair enough
People don't generally consider reading a book or reading a newspaper article a waste.
But there are a lot of games that are like the equivalent of doing that.
Like Anna Anthropy's games, pretty much just all of her games.
Or like Squinkies games are all very-- they're meant to teach you something a lot more than the bottom right hand corner
So I would say the idea of it being a waste can be more voluntary.
I'm not playing this game because I need to do it to finish a graduation requirement.
I'm playing this game, because it's a recreational thing.
So in some sense, it's like games can be a waste so that's a really negative turn, but that gamification is usually framed into this is a.k.a.
You should be doing, because its ulterior motive is [INAUDIBLE].
The market is something that has to be valuable in some other context than just the enjoyment of the experience.
like you're being more fit or learning more about something
You're just describing the reverse case of what they were actually-- a couple of nice [INAUDIBLE] experiments about in the younger learning.
In fact, when people say if you give somebody a reward for doing that, they will later on reconstrue why they did something.
Oh, it was a good reward.
Therefore, it cannot be something that I should do for its own sake, just because I enjoy doing it.
Or therefore, maybe I didn't do it for its own sake.
So next time, I should expect [INAUDIBLE] for that so I'm less motivated to do it without a motivation the next time.
So if you add an extrinsic reward [INAUDIBLE], it becomes less motivated to begin with, because you underline the perception of autonomy of the description [INAUDIBLE] to begin with.
Whereas if you say well, if it's really wasteful, if I say there's nothing that comes out of it reliably, I can look at the situation and say, well if it's really wasteful, the only reason why I may do this is because I enjoy the [INAUDIBLE]; not because I'm getting a reward for that.
Ergo, this is an activity that I'm really [INAUDIBLE].
Therefore, if it's very autonomous, [INAUDIBLE], this is something where I fully just-- this comes to us out of my own personal beliefs, wishes, and goals, nothing else.
Or you could use the Calvinist version of that saying, this is a waste.
I shouldn't do this.
And by the fact that I put myself against the moral tub of a Calvinist culture around me that always wants to be productive, and so I intentionally [INAUDIBLE] completely wasteful to show you that I'm not always controlled by your damn culture.
In the same way, you are reinserting your autonomy by doing something intentionally [INAUDIBLE], potentially.
So yes, they're definitely in these psychological ways, intertwined
I'm curious if that means the example we said where they were being paid to play the game, but then they also played it in their days off
Yeah
Then would it be a game when they played it on during their free time, but it's gamification when they do it as a job?
The same exact game, with the same exact, enjoyment for the person playing it?
It's a good question, but I'll get to that in one second please
I think it's worth unpacking the word game, and that we're using it in two different senses.
I think you might have just said there is the people who are planning to play the game for work, and then play the game for play.
But when they're playing a game outside of the context of work, I think they might see it as I am playing a game right now, whereas that other thing they are playing, this game-- how do I describe it?
There's a difference between the game that everyone calls a game, this program system that everyone plays, and the activity of the person actually playing the game
[INAUDIBLE] as a subset of [INAUDIBLE]
Right.
Gold farming is a thing you can do in this system that we call a game, because people tend to play it.
In that case, there are ways of playing the game that don't meet that normal expectation and might look like work.
And then there's the normal way of playing the game that they return to on their own time that perhaps is more fun and less for this external reward
Yep.
That is a very important one, and one that is especially in game study is usually completed that is the distinction between object and activity [INAUDIBLE] of engagement.
Because [INAUDIBLE] like to say well, if I'm the toddler, I have no concept of game theory.
I haven't been socialized into games, and I accidentally happen to hack under the chess computer, entering the exact commands that are legal commands based on the rules of chess.
Is the toddler playing chess?
Or is the toddler toying around with the chess set, [INAUDIBLE] the concept of what playing the game of chess means?
No, but those are free.
Give the toddler his [INAUDIBLE].
I'm trying to play chess here.
In the same way, if I'm the usability engineer, and I am clicking my way through Halo in order to [INAUDIBLE] usability products.
Am I playing Halo?
Or am I testing Halo?
The activity that I'm engaged in-- and this is actually the court subject, what was the subject matter of my PhD research [INAUDIBLE], for exactly that kind of experience.
The activity that they're engaged in is testing, even though the object they're operating with is a game.
So keeping these two separate you could say well, it was the game.
But maybe some points, when they felt oh, now I'm engaged in World of Warcraft in order to make money, they felt this feels like-- it's still game play, what I do, but it's game play as work.
It's game play reorganized, reframed as working to me.
So it's not really playing the game anymore.
It's work playing.
It's a weird mixture.
It's a new type of activity, but it's not playing as we know leisure time [INAUDIBLE].
So that's definitely an important distinction.
The second thing, or the final thing that I want to turn on, in terms of when it is something a game or a gamification?
First thing to notice-- all of this was tested, and you could throw in new points, and you would get it to the same [INAUDIBLE]; yes, no, yes, no, yes, no.
So how do we know if a term is correctly used for something or not?
What determines whether a word is currently used within the device [INAUDIBLE] or not?
When is something correctly called oxygen?
When is correctly or incorrectly identified as oxygen?
Shouldn't all these depend on the context?
How so?
Because if you just give it time by itself, without much context, you can actually figure out if it is correctly used or not.
So for example, oxygen, just given the word like that, I can say something like we all need oxygen to break
Yeah
But then you can also say something else like too much oxygen can asphyxiate us
Mhm
Actually, is that true?
[LAUGHTER]
I see what you mean.
So definitely, whether a statement using the word is correct or not depends on the total statement.
Totally.
Second point is so when is the term correctly used or incorrectly used?
How do we determine if the term's correctly or incorrectly used to identifying something-- a phenomenon, an object in the world, [INAUDIBLE]
I would say it's really hard to do that.
And there's a reason.
It's like oxygen, everyone's thinking here.
It's also the name of a TV channel
Yeah
And that's fully correct.
It just felt like that.
So I think the context of what you were saying is important.
So when you wrote that word down and asked that question, the answer was what you were thinking of, because it was in the context that you have.
So we would have to figure out what context you were talking about and then answer that question correctly.
But someone else could come up and ask the same question with a different context, and the answer would change.
And you could make this argument for almost every word.
That's part of the reason why it's really hard to get computers to understand what humans are talking about, because you have to give them this conflict and the context.
So I don't know if you can for sure define something like that.
I may run around with a different [INAUDIBLE]
OK.
If you ask a scientist, a chemist, how do I identify whether something's correctly or incorrectly called oxygen?
How do you go about that?
And at that point, you claim that there's some fundamental principle of the object as such that you can identify about it that identifies it as oxygen.
And so you can concoct some series of tests that allow you to verify the things that make it as such, because you have some set of rules.
You verified against all the rules, and you claim that therefore, this thing's oxygen
Scientists that basically were chemists in that case, have operationalized that concept.
Or the other way around the concept is defined by the very set of operationalization.
So if it passes these tests, it counts as oxygen.
That's how we define that term.
So that's the definition [INAUDIBLE], which works great for scientific constructs, if that's how scientific concepts work basically.
Now, what about something like delicious?
When do I know whether the word delicious is correctly or incorrectly used to apply it to a given object?
Isn't that extremely subjective?
That's correct, and identify against everybody in the room and prove to them that that's delicious
[JOKINGLY] It is delicious when I tell you.
Right.
It's a subjective term.
Delicious isn't true or untrue based on whether it passes a number of tests.
Delicious is an expression of a subjective value experience.
If I find it delicious, I can use the term delicious.
It would be incorrectly used if I used the word delicious, although I think this is their choice [INAUDIBLE], but it's delicious for me or not.
What about as a kind of middle term?
It's not a middle term.
This mixes all kinds of things.
What about something like faux pas?
A social faux pas or inappropriateness-- something like that.
When is the words inappropriate and so faux pas appropriately used?
It both depends on the context and if it's subjective
How is it subjective?
In France, something might be illegal, or in the US, it is not
Yeah
And depending on the context of the thing, of what happened, then even in the same country or same location with the same set of rules, it might be good or bad, depending on who is the person actually looking at it
Mhm.
So I flush that out.
It's something that we have, like a formal definition to the way that it works.
There's a set of [INAUDIBLE] rules you're supposed to adhere to, a way that the context has been misinterpreted, or something of that nature.
It's like using the word irony, or something like that.
There's a specific formula for how you know something is [INAUDIBLE].
But past that, there's also still the fact that you have identify it as such, and that's dependent on the context or the situation you read that inside of, because what it is in one situation may not be in another.
And it's definitely my situation.
It's highly situation dependent on [INAUDIBLE] culturally.
How to properly or improperly [INAUDIBLE] taken that into the business partisan [INAUDIBLE] different depending on the different cultures.
I do this usually to how people think about-- if you'd look at a certain concept, is this more like a scientific construct?
Is this more like a subjective value statement, or is this more a social convention?
And I would say games and gamification, because they weren't around before people were around.
But it's still something that we use as a sure term to [INAUDIBLE].
And don't think that somebody can say for me, this is a game.
For me, this is not a game.
Like you would say for me, this is delicious.
For me, this is not delicious, that it's basically the social convention [INAUDIBLE].
This is this sure cultural sense of this is what a game is or shouldn't be.
So to an extent, teasing out when is the game, or when is not a game.
What you're doing here is basically you're doing social linguistics.
You're teasing out what are the conventions of a shared community of people what they consider to be a proper game or not?
And then you run into all the [INAUDIBLE] cases where the language community in front of you.
And you hear people say, is that a game I don't know?
That's weird.
That's odd.
It doesn't fit my prototype case of how games should be.
And gamification presents us with all of these cases for real money trading and stuff like that that, [INAUDIBLE] all these cases that don't fit our existing social conventions of the category of gamers.
And so therefore, we have all [INAUDIBLE], because their challenge-- they're an empirically new phenomenon that don't fit our existing conceptual categories.
I was [INAUDIBLE] existing categories.
So to a certain extent when is it games, or when is it gamification?
Let's see how society solidifies opinion around these terms.
Right now, it's up in the air, to a certain extent.
Or that's what I suggest, so that's what I sometimes do.
If I introduce an article I say, I'm not doing social [INAUDIBLE] here.
I'm not trying to tease up what people think collectively, the way they police language together, whether this is a game or not, or whether this is an unsettled matter [INAUDIBLE].
I use it as a scientific construct, and here is my operationalization of that scientific construct.
I use the term [INAUDIBLE].
It is supposed to mean that it's to pass the following tests.
And if it fits these tests, it is gamification.
If it doesn't fit these tests, it's game.
You can disagree with that.
You can make up another operationalization.
But in that sense, the meaning of the word is I use it in [INAUDIBLE] of this text.
Another interesting thing in a second.
Another interesting thing is that is this still a game or not, or is this this or that is that you often conflict normative and descriptive [INAUDIBLE].
This isn't a game.
If you look at [INAUDIBLE], it very often just means this is not a game I'm liking.
It doesn't mean that it doesn't fit the descriptive categories that you can say, according to this game theoretician, it's set up list the five defining criteria of whether it's a game or not.
It fits the bill, but it's not a kind of game I'm liking.
So if we say, is this still a game or gamification it often says is this a certain kind of game that I like versus the certain kind that I dislike, Dido?
So if we're talking, in regards to the more subjective view of this distinction, I was wondering if you think whether or not a person looking at you for a game or a [INAUDIBLE] context, whether or not if they identify as the gamer, or if they had prior experience with even more traditional games will effect their typecasting of whether or not it's a game or a gamification.
Will people who are more familiar with games recognize things that are more subtle and say oh, that's gamification faster?
Or is it they're more accustomed to it?
They won't see it as much, and just wondering the different-- that would be like me or my mother basically
There are lots of interesting [INAUDIBLE].
A, there is beginning data, especially when people do gamification [INAUDIBLE].
We find old people play games.
They just don't play video games.
They play board games, card games, social games, all kinds of games.
It's not as if they were not introduced to the concept.
But they don't know the video game lexicon [INAUDIBLE] necessarily.
So they don't know progress bars.
They don't know achievements.
And so it's all of that [INAUDIBLE].
What's that then?
What happens now?
And I remember a former girlfriend of mine, the first time she played.
[INAUDIBLE] obviously, there were achievements.
She was in the achievements screen and say, the game is broken.
How do I get back?
She wasn't absolutely not introduced to the concept.
And as gamification for the elderly do is they show data that indicates there is another nice little identification in the work place by [INAUDIBLE] molecule a bit that [INAUDIBLE], that people who are socialized into games tend to pick up gamification more easily or usually a bit more lenient towards it, whereas people who don't have a game of socialization are more [INAUDIBLE].
The second [INAUDIBLE].
Yeah?
This is going to sound really stupid coming out of my mouth, but are games these days gamified in the sense that so it takes me to the achievement system.
That wasn't in games that should go back in time?
Yep
And now you see that it's now almost a requisite part of any game that comes out.
A game doesn't come out without achieving things.
A shooting game doesn't come without her filling health bars, and fields, and things like that.
These tropes that define the topic itself necessarily comes reflexible with [INAUDIBLE] on it, and it's pointless to describe things as gamified and everything.
Both games and non-games can have that slim set of [INAUDIBLE]
I said one of the things in terms of distinctions, you typically-- and I apologize for my terrible handwriting.
This isn't working [INAUDIBLE] involving AIDS for me.
It is between descriptive and normative.
Are you using these terms of indescriptive or a normative way?
I've seen a lot of people using gamification in a normative way.
[INAUDIBLE] communication, evaluating that.
Another one is if you do a descriptive way, how do you operationalize that definition if you say, it is gamification if it has certain features, like an achievement system.
Or is it gamification if it describes the outcome of a certain design process?
Designers about to make something more [INAUDIBLE] about thinking about [INAUDIBLE] games to make this more fun.
Then the out come of that, by definition of unification.
Or not even by the design process, just by the intention.
Does the user see this as something that is gamified or not?
Or did the designer, in doing this, think about this as now I'm doing gamification or not, which answers the question that some people here ask saying well, how come this [INAUDIBLE] of ordinance called gamification, whereas we have the same kind of incentive schemes in the big corporations for decades before that?
And key performance indicators, this intelligence dashboard.
Nobody called it gamification, even though it was the exact same thing.
It comes down to how do you operationalize and formalize that [INAUDIBLE]?
If you say it's dependent on tension and you say well, that wasn't the attention.
This wasn't intended to be [INAUDIBLE], no matter what it was [INAUDIBLE].
If I define gamification by a certain feature list.
So as long as it fits the feature list.
Yes?
So again, how do you define the term?
And there, it follows the [INAUDIBLE].
GABRIELLE TREPANIER-
Maybe you can talk to kids about the debates around gamification, ideologies, governance, surveyance.
Maybe it's not like you're an expertise, or I don't know
If I talk about reproduction, like gamification of reproduction, that to me is an immediate outflow of [INAUDIBLE] gamification as government [INAUDIBLE].
GABRIELLE TREPANIER-
Mhm
Definitely.
So I'm happy to talk about governmentality and ideology.
That's a good question.
Other questions in the room right now?
We also have two more questions on the board.
Sorry
I was going back to what you talked about earlier on the note of autonomy being the marker for whether something feels like a game or feels like gamification.
I feel like autonomy is almost a loaded term, in this sense and that yes, you could have autonomy, and that when someone gives you a test, you can decide how to do it.
And as long as you return the test in a finished state, then you're fine or whatever.
But there's also not the autonomy where you can go against the logic of the system, which in this case, would be the logic of the business.
So I feel like in this example, it felt more like gamification or a bad attempt at gamification to these people, because the logic was pretty one directional.
It was only ever increasing the end product for the business, not letting OK.
If I want to, I can have agency against the business.
And that might have certain consequences, but it would still be like I already know the consequences beforehand as part of having some sort of agency
This leads to an age old question in humanism.
It goes back to Kant and the Enlightenment.
It goes back to Aristotle.
That is the question, how do you not instrumentalize people?
If you take the idea that people are self-determined actors, or that they can become-- we can help people race towards being self-driven actors.
How can we not make them other determined actors in [INAUDIBLE]?
And that's actually an interesting question for me, because I started out working in civic education.
And civic education very much, at least in Germany thanks to the Second World War, and then thanks to the Allied Forces occupying Germany and saying, we should learn a thing or two about democracy.
So civic education in Germany, which originally was a democracy age that happens very interesting [INAUDIBLE].
How can we create a democratic state?
Or can we educate people about democracy being a good thing without ourselves becoming a totalitarian rule?
Without this itself becoming again, [INAUDIBLE]?
And the argument, or the point that civic education in Germany camp as the moral stem point which reflects the point that Aristotle had for education, in general-- how did we get to educate people to become really selfish [INAUDIBLE] is to say as long as for all intensive purposes, according to your own best knowledge and intentions you try to set up this system of education such that you enable the person going through that education at the end of it to make the independent judgment whether that education is actually the best possible thing happening to me, so retrospectively embracing what was done to [INAUDIBLE] saying afterwards, without you training me some discipline, I would have never developed the faculties in order to be able to be an independent flourishing [INAUDIBLE].
So thanks for that.
That was actually good afterwards, even though during the process, I felt other [INAUDIBLE], and I hated it.
But as long as you're doing it in my best interest, to bring me to the situation where you can reflexively look at what you did to me and then say, that was wrong.
Let's change this.
So as long as everybody going through the system is the critical corrector that comes out to look afterwards at the system that says this was actually good, or this was actually [INAUDIBLE] way to change the system.
Then things [INAUDIBLE].
So that's the procedural order.
The system is built with the intention of making people critical-- changes of the system they themselves run through, then it's fine.
Then it supports their independence and their autonomy [INAUDIBLE]
I see
There are very few businesses that do that that I'm aware of.
That's in civic education.
Oh there it is.
That's the answer to it.
We'll get to govermentality, and we'll get to the two others on the board.
Any other thing that you would like to be addressed?
[INAUDIBLE] to worry about.
Sweet.
Then let's talk about governmentality very quick, and then about what is learning from games or gamification?
So can we do this interestingly or responsibly, and can I have some of it?
So governmentality-- just a show of hands.
Who knows the term governmentality?
Yeah.
It's this ugly [INAUDIBLE] created by Michel Foucault, which was the bald, gay Frenchman.
It was correct, right?
Anything incorrect here-- GABRIELLE TREPANIER-
It's correct
I can only say that, because I wrote my master's thesis chugging through old books of Foucault, so I feel I've earned saying that.
Governmentality is basically-- there are lots of dimensions of how it's basically been played.
But one part of that is to say modern liberal nation states, in specific, do not rule by punishment.
They do not rule by if you do x wrong, I'm going to whack you.
Or if you do x well, I'm going to give you something.
They rule by putting people into the mindset that they individually regulate themselves and that they believe.
It's in their own best interest to regulate their own behavior that they internalize it.
It's a Freudian argument.
It's not my parents telling me stuff to do.
I have my inner parents telling me that I'm horrible.
So in the same way, the state operates by getting you to internalize regulations so that the state doesn't have to inefficiently whack you the whole time.
How does governmentality connect to games?
Well, you can make the argument gamification is specific.
Well, we can make the argument, or I would make the argument that a big part of how current Western nation states manage to keep this governmentality is to tell us in order to realize ourselves.
In order to be our best selves, in order to be all we can be, we have to self-manage.
We have to self-optimize.
We have to be productive, productive productive, and manage ourselves, and become ever more productive throughout.
And then gamification will quantify itself.
All of these are basically tools society then gives us or people give us to say, these are ways in which you can manage yourself better, and these are ways in which you can monitor yourself better to make yourself evermore productive.
And that was supporting and reinforcing the very societal system at which we're all kept to be as productive as we can be-- that way, actually regulating ourselves, instead of having other people have to regulate some of that form.
And then you can ask yourself, is that actually a society in which I want to live in?
Do I subscribe to that?
So that's my one minute stick on governmentality.
What is good learning from games, or what is good gamification?
I tried in the Eudaimonic design article to tease that outwards, [INAUDIBLE] on how to do this well would look like, and there's another article out the lens of intrinsic skill [INAUDIBLE] right now on SSRM, and the proofs should fly around.
Somewhere we've reached the [INAUDIBLE] journal.
But there's another article you could find online, the lens of intrinsic skill [INAUDIBLE] where I describe an actual design method.
The basic things, I would say, is do not put the cart before the voice.
That is if you're a good designer you do not say, this is a solution space, gamification.
Now, where can I use it, but work the other way around.
Look at the problem and say, what problem do I try to solve in the context?
So good, very practical example.
A friend of mine uses for Fitbit, came in Rochester, Upstate New York.
She walked about 3,000 steps a day.
She's now on a Fulbright in Croatia in Dubrovnik, and [INAUDIBLE].
Is that a sudden boost of motivation?
Is that a sudden boost of willpower?
Is that because the Fitbit has a new feature or some new achievements to unlock?
It's because if you try to get around in Dubrovnik with a car, it's the most complicated and inefficient way of getting around.
And the city is basically just one huge flight of stairs.
So there is no way for you to do anything but walk through your whole day.
So if I look, Step Trackers has this solution to people's lacking mobility.
I say well, it's not as if Americans are slackers and Europeans are monsters of willpower.
It's just when I was back in Hamburg, getting around in [INAUDIBLE] was the easiest, cheapest, most efficient way to get form A to B, so I was biking all the time.
And it was completely self-sufficient, because whenever I couldn't reach something with a bike, I could reach it with a well built out public transportation system.
And drivers, therefore, were expecting that there were bikes all the time.
Here, especially in Rochester, which is a car city-- other way around.
I couldn't bike, even if I wanted to.
The city architecture works against me.
Drivers not being used to that works against me.
Wrong roads works against me.
So suddenly, I need all willpower to put myself against the friction of the environment to bike against all odds.
So don't put the cart before the horse.
Look at the problem and say oh, we actually have to change the zoning.
If we introduce mixed zoning, people will walk, because there's an easier way between supermarket and home, and you will just walk the small ways [INAUDIBLE].
So that's the first part.
And then the second part is part of your analysis.
So say to yourself, is motivation the issue?
What games do well is indeed, motivate in certain ways.
That are things that are fun and enjoyable to a lot of kids.
And look at the issue and say, is motivation really what is lacking here, motivation due to a lack of structure, due to a lack of frequent feedback, due to a lack of clearly explicated goals, due to a lack of interestingly varied challenge?
And again, all of these are the components by which games actually create fun, engaging experience of competence being absorbed in the activity.
If that's the case, great.
Take structures from games.
Take those structures.
Have better articulated goals.
Have more frequent feedback.
Vary the challenge in interesting ways so I don't do the same old thing over, and over, and over again.
If that's not the issue, why should you even try games?
Again, using the wrong tool.
Like 80% of problems where I get flown in, I look at whatever software thinking they built and say, you have a usability issue.
You don't have an engagement issue.
First people have to figure out where even to click before you have to motivate them to click, and that's hard, and [INAUDIBLE] interface.
Let's fix that first.
So that's the short answer.
And the long answer then, look into that article, and I go into great lengths to [INAUDIBLE] one, what is it we can assure in games?
Any final questions?
GABRIELLE TREPANIER-
Thank you so much
Thank you.
[APPLAUSE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
That demonstration I had been to was for peace in Lebanon during the war in 2006.
And so it wasn't-- there was no reason for so much police to be there.
We were there to ask for peace in the region.
So it wasn't about revolution or toppling the president or whatever.
So I was really shocked that there was this really ridiculous kind of fear from the Amn ad-Dawla, the State Security.
And of course, what happens is that when somebody makes a little gesture, and there's like a panic movement.
And of course, the police takes advantage of it and panics, because they're 17 years old or so, the policemen.
They're terrified.
And everybody's terrified.
Except they're armed.
And they start beating people up and arresting people.
So from there, I thought on the 25th of June, I thought, I'm not going down.
I know what's going to end up happening.
I'm going to get beaten up.
And then I realized I'm like everybody else.
I'm afraid.
And the whole point of the revolution was to break that fear.
It was really that suddenly nobody was afraid anymore, and that we actually really went.
And the fact that we were all together in the street was, like, OK, we're not afraid, because also we're all together.
And it was really extraordinary.
But I also saw that I wasn't a photographer anymore.
I had, like, 25 years ago began my career as a photographer and kind of got a little bit out of it from watching digital cameras and everybody becoming a photographer was also part of my-- sort of how I evolved as an artist.
And so I moved into using images, but not so much being so focused on making images.
So for me, my role in the revolution was to really be there as an Egyptian, and to be there as a person who loves that country, and who wants to really see something through in that movement.
So I really put myself, as an artist, actually outside of everything, and for quite a long time, just watched.
I participated.
I was there.
I met people.
I helped people.
I was in the Square all the time.
But I did not take lead in a way.
I did not produce art.
I didn't want to do anything.
I just thought this is too much.
I need to process what's going on.
So it's much later in the process that I started to kind of take action in a way, or a different action than being a citizen demonstrating.
And this is when these other Tahrir Cinema and sort of very time-based experiences sort of happened.
But that we'll talk about a bit later.
But my role, really-- I saw my role as someone who could, from my experience and the community that I know, someone who could really connect people to each other, and also watch a lot of younger people than me, and see that I could really-- I had the experience that they didn't have.
But they had, maybe, the tools and the ways to operate that I also-- my generation-- didn't have.
My generation was much more individualistic, and also a little bit like a satellite.
We worked as, like-- we were apolitical, in a way.
We were sort of-- I think we were-- a lot of us were a little bit outside of the political sphere, because that wasn't the way to operate at the time.
So To watch suddenly this new generation come in and be very, very strong about their political views and using social media and so on was, like, OK, what can I give that they don't have?
And so it was really great.
Because I met a lot of people and I worked in that way with a lot of people.
So it was a very organic time, in that sense.
And it led to a lot of things which we'll talk about [INAUDIBLE]
We have a lot to talk about.
In the classroom last week, we were asking ourselves, what is an activist?
How much someone should be involved in a cause to qualify as an activist?
And we were comparing anonymous members with Aaron Swartz.
And we were concluding that Aaron Swartz was much more dedicated to the cause than people who just do sit-in, online sit-in from time to time.
I guess my question would be in this case, are all protesters activists in the context of a revolution?
And do you consider yourself as an activist?
And when does the artist become an activist?
So no, not every protester is an activist-- for sure not.
I think every protester is a citizen.
Every protester that comes to a demonstration doesn't necessarily know why they're coming to a demonstration.
But this is a huge discussion.
But in the case of Tahrir, a lot of people-- there was something kind of almost funny, when you look at it in retrospect, is that-- I don't know if you remember, if you know-- but on the 25th of Jan, the first group of people go and demonstrate.
On the 26th and 27th, they-- the main activists of that first demonstration-- start to kind of confuse the police and exhaust the police, which was a technique.
On the 28th of Jan, everybody is called to come to the square.
It's a Friday.
In Egypt, Friday is the equivalent of a Sunday.
On the 27th at night, before this Friday where everybody's on holiday-- nobody's going to go to work, so it's much more scary for the government, because they know more people are free to go demonstrating-- the tension was growing throughout the country, not just in Cairo.
And at 7:00 PM, there was rumors that the government would stop our telephones and internet.
At 9:00 PM, we couldn't talk to each other on the phone.
We can only text.
At 11:00 PM, we couldn't text.
At 6:00 AM, we couldn't email anybody or open internet.
So it's kind of stupid.
Because what do you do when you don't have internet, telephone, and you don't have a landline anymore?
What's your natural instinct?
[INAUDIBLE]
You go out, try to go to [INAUDIBLE].
So it's in a funny way, almost, I'd say it's almost the government helped initiate this by shutting us down.
And so of course, we're all going to go look for each other.
So everybody goes down to the street in their area and talks to each other, says, what's going on?
Oh, let's go to Tahrir.
Because of course, we already know that there are demonstrations going on.
So most of the people are already informed that there are demonstrations and marches that will start after the Friday prayer at 1:00 from a lot of the main mosques around the cities.
But most of the people who are not part of these marches necessarily will sort of naturally join.
And so this is what happened.
So people found out about the reasons why this is happening, how this is shaping itself as it was happening.
There was no real-- it's not like everybody knew why they were going down.
So people found themselves activists, maybe after five days, 10 days, maybe overnight.
So a good example-- and he's here Harvard if you have a chance to go, maybe meet him or find out about him-- but there's somebody, like a good example is Bassem Youssef who's a comic now that's compared to Jon Stewart in the US.
But he was a heart surgeon.
He was a doctor.
He was studying.
I think he was still, like, finishing his studies or something, or a very early practitioner.
He filmed himself talking about what was happening, posted the video on YouTube, three months later had already five million people watching him.
And then a month later, a TV that came out of the revolution-- so a free channel-- asked him to do a show.
And he became a really famous figure over the period of two years that followed.
So this is an example of someone who's a citizen, but then becomes an activist, but then becomes-- finds a new voice for himself
So it's a fluid category, kind of
So I mean, everybody changed our life.
Like, I mean, everybody-- maybe a lot of people.
But I would say that we all lost our friends and made new friends.
Because suddenly when you can't talk about politics, you realize that you don't get along with anyone around you.
And it's like, oops, I don't think so.
This is such a huge, impactful moment in history.
You cannot make a compromise.
There's no time for it.
There's no energy for it.
And it's going so fast, and it's so intense.
You have to go to what you believe for.
And so you make choices.
And these choices-- you watch people sort of drop out of your life.
Suddenly, you're completely negotiating a completely differ-- so these aren't fixed-- what you are and how you [INAUDIBLE] that.
And to talk about how an artist becomes, what an artist does in a revolution, I mean, this example behind me is a piece that I did in 2007.
And the revolution started in 2011.
To give you a configuration of Cairo, you have those famous lions that you see the picture are the entrance of a famous bridge, called the [INAUDIBLE] Bridge, [INAUDIBLE] means bridge, so the Nile Bridge.
And in front of those lions, actually, is Tahrir.
So it's a landmark a little bit like the Pyramids or the Eiffel Tower.
It's a famous area in Cairo, and people would naturally go visit it.
And so in 2007, I made this piece, which is called Justice for the Mother.
Egypt in Arabic is called the Mother of the World, Um al-Dunya.
And so this work was really a work about-- it was actually a portrait of my father and a portrait of the law of the jungle-- the strongest is always the one that wins a fight.
But while I'm doing this work and sort of studying this kind of psychology of how human beings fight each other, and who wins and who loses, using these images that I've collected in my work, I attended this demonstration I told you about, where this huge cordon of police was circling us.
And I was really shocked by it, because nobody in that demonstration had the same purpose.
There was five or six people who were actually there for Lebanon.
But really, people were there because either they were passing by, or they were activists, or they were asking for the bread to be less expensive.
I mean, everybody had a kind of different cause and purpose.
And so this sort of like weird incoherence of that demonstration really stuck with me.
And so this was a parody.
So this is a detail of the piece where it's a parody of people demonstrating and asking for something.
But everybody seems to be, like, on their own.
So you can see John Travolta, which is not your generation, but a little bit mine.
Condoleezza Rice during the war in Lebanon, so with bombs and missiles that say, I'm sorry, I'm sorry, I'm sorry, and all kinds of character that seem to be revolting.
And so interestingly, when Tahrir started, suddenly I look at my work, and I think, what am I supposed to do now that there's this huge revolution or uprising happening?
Am I supposed to work?
And I'm watching all these kids and all these people produ-- even citizens, not just artists, but citizens producing videos, and uploading footage, and making jokes, and doing this crazy, amazing stuff.
And I don't work like this.
I work-- I take time to work.
And I worked under dictatorship.
So in a way, my work was shaped by the stagnant regime under which I was working, and which I was trying to also sort of mirror by making work that kind of demands attention and in a sense are heavy sometimes, but that mirrored that kind of rigidity of the regime I was working under.
And looking back at this work as one of the works was really funny, because I was, like, wow, I depicted a kind of problematic paradise behind those lines, which is-- it's actually a portrait of Tahrir.
And that's how it felt.
That's how the being 18 days in the square really felt like there was a possible world that we had to experience, and we were experiencing in very small moments.
But those moments where this is a possibility, this is a seed of something that could happen-- maybe in 2,000 years, maybe 200 years, maybe in 10 years.
We don't know, maybe tomorrow.
I have no idea.
But it's possible.
And it's that sort of sense that really was really interesting for me.
And it took me a long time to kind of think, OK, how do I work now?
So this is also another question.
Because then I start collecting footage and working with archive, and collecting pictures.
But I wanted to show this as a kind of relationship to how the world changed overnight.
And so there was a moment of reflection and kind of absorbing what was happening before doing things
This was made before the revolution
Yes, 2007, four years before
I think it kind of shows that your work was already political before the revolution, even though you didn't really notice.
There was something political in--
No, I did notice.
I did notice.
But it was just a different language I was using.
I mean, this work-- this is Cairo.
I want to show you.
These are informal areas in Cairo, which are basically illegal constructions and slums.
And this was a tower that I built in the Opera House in the center of Cairo, which is a military base, and which is where the Biennial of art takes place-- used to take place.
And so this work was directly a kind of statement to the government to say OK, the subject of your Biennial is the others.
So what are you doing with the others?
Who are the others in relationship to the government?
They're all the people they're not addressing.
They're not addressing the issues that these people have and face every day.
And so I built an informal tower, a slum tower, in the middle of the garden of this military base.
I made my own brick.
You can see the donkey here.
And it says "hope" on it.
And here you can see the tower at night with the Opera House just behind.
And of course, this was very controversial.
Because it was like, wow, like putting people-- you don't go there.
So I'm going to bring there to you.
And this was really like a year before the revolution, a year and a half before the revolution.
So this work was really, really loaded and very much discussed later as a kind of movement in the art world, as how artists were putting their-- saying, you know, hello, alert, alert, beep, beep.
There's something happening here that you need to look at.
So this was the kind of landscape we were in when the revolution started in terms of the art scene
Yes, Ali
[INAUDIBLE] were you inspired by Bosch?
That's what everybody always tells me
It looks so similar
Thank you very much.
It's always a very beautiful compliment.
I actually love painting.
I'm not a painter.
And I wish I could be.
But I'm sure I will be at some point.
I've made other people paint for me.
Because in Egypt, we have this tradition of making painted billboards for the cinema.
I mean, in the good old days, they still existed like this.
But yes, a lot of people compare my work to Bosch because of the kind of--
[INAUDIBLE]
The delight, the Paradise depiction.
But actually, when you get closer to it, it shows all the disruptions that are happening in this depiction of Paradise and how, in fact, it's closer to Hell
[INAUDIBLE] I saw the similarity to the free expanse, but also the two lions separated into a very three-paneled piece.
It also just seemed very similar, and I really like that [INAUDIBLE]
Thank you
OK, so now we'll enter in the discussion we're supposed to have today related to the assigned readings.
And what I did, actually, is that I used the assigned readings to build my questions.
And I'm going to use these questions as a pretext to kind of push in a little bit of theory, so it looks more like a course than just a casual discussion.
And then after, we'll try to connect this to your own experience as someone experimented the revolution.
So the first thing I would like to talk about is the rise of digital rebellions, citizen journalism and cyber left, which did not start with the Arab Spring and the Occupy Movement, but started 12 years before with the creation of Indymedia, which is a global, federated network of independent media centers, radio stations, newspapers, video collectives and websites.
And Indymedia was already only funded as an alternative and a direct challenge to corporate media like CBS and CNN.
Most [INAUDIBLE] report on the counter demonstration and the police brutality that took place during the 1999 World Trade Organization meetings.
And during its first week, the website received 1.5 million hits, outpacing CNN website for the same period.
It was quite something.
It was really big.
And as the platform developed, it became more than an alternative media outlet, but also a think tank and activist laboratory and mobilizing tool for the anti-globalization movement.
So for those who don't remember, the anti-globalization movement was against free trade agreements that would be at the disadvantage of the poorest.
And so this platform was allowing people different movement, different struggle, to kind of merge together on the same platform-- movements as diverse as the Zapatista-Mayan peasant army from the poorest region of Mexico, the South central farmer movement in LA, a movement against Wal-Mart in Philadelphia, and movement against war in Iraq.
So it was very diverse, but it was all converging on this platform here.
And at that time, the activists were using chatrooms, the listservs, wiki pages where people could upload their stories, their report, their pictures, their videos.
And so Indymedia was really one of the first websites where 100s of people were uploading content, generating content.
And in this book, Digital Rebellion, Todd Wolfson claimed that Indymedia, with its slogan, "Don't Hate the Media, Be the Media," was kind of a precursor or catalyst for social media, but also the grassroots journalism-- citizen journalism.
Lara, in your article, "When Seeing is Belonging" that is very good, and I recommend everybody to read it-- it's online-- you argue that September 11 also contributed to the rise of citizen journalism.
You want to tell us a little more about this idea?
Yes.
I mean, I wrote this article because of my relationship to photography, and how this is how I began my career.
And I found it was quite shocking to watch everybody-- can I go back to some of these images-- to watch everybody take photos in the square.
Like this is quite an interesting image.
So watching this kind of scene, and I was myself holding a camera that was actually broken.
So it was making scratch-- it had scratch like that.
I was, like, hmm, look at everybody's really documenting this.
And I look back at my life, and thinking, OK, what do I do now?
Because I felt a bit frozen.
As I described, the way I work is I need time.
And in a moment where you're in the middle of a-- when the dam breaks, the river goes really, really fast.
So you better be your head out of the water and go with the flow.
Which, in a stagnant regime, it's more like being in a lake that is very stagnant.
And you have to keep going down, and kind of trying move [INAUDIBLE] so that something happens.
So it's a very different situation.
So that sense made me look back, and maybe look back at photography, and the use of media, and the use of image making, the history of image making.
And I remember starting photography when I was coming out of school and the first Gulf War started.
And that was a moment in media where the war in Iraq was shown live on CNN.
And that was one of the only channels that was actually filming.
But you could never see anything.
It was always dark.
And little lights would come out.
And you didn't know what was going on.
It was just like you could see explosions, or-- that was a really important moment, in terms of discussing and debating the role of media and watching war live on television.
10 years later, there was 9/11.
And I don't know how old you guys were, but it was a time when social media didn't exist.
Most people didn't have mobile phones.
And if they did have mobile phones, they didn't have cameras on their mobile phones.
So what was much more the thing of the time was to have a digital camera in your bag.
And it was usually very low resolution.
But it was nevertheless digital.
And that was the early days of creating media, in digital form.
So suddenly, 10 years later, here I am in the middle of this revolution, and watching everybody upload material, I mean, as fast as I'm talking.
There's already 1,000 videos that would have been uploaded.
So this whole-- over 20 years, but every decade, this new platform that image making reaches was, for me, really, really fascinating.
And yes, one of the things that came out of all the discourses and the debates around photography and image making and media in general after 9/11 was that, for the first time in years, the power had changed hands-- where the citizen-- the ones who were underneath, or very close, or watching from buildings and over windows-- were able to document what was happening.
The media wasn't there.
[INAUDIBLE] citizens.
And that was a really, really big moment in history, in terms of suddenly the citizen had power that the media didn't have anymore.
So that hold on the power and in whose hand is always at the core, I think, at the heart of those debates, especially moments like that
So it was interesting, and this is where I speak about citizen journalism, and how and why that moment in Tahrir was really interesting for me.
Because I had seen this throughout my career as a photographer and artist, and watched that.
And I was really fascinated with it.
Let's talk about the role of media in mobilizing people.
So Facebook and Twitter are often depicted as the main instruments of the Arab Revolution, to the point that the revolution has been called a social media revolution, a Facebook revolution, Twitter revolution.
And apparently, one of the main elements that contributed to this [INAUDIBLE] uniting the young Egyptians that were not, as you said, necessarily into politics at that time, was this Facebook page that we spoke about earlier.
We are all Khaled Saeed.
I know I don't pronounce it well, but I try
[INAUDIBLE]
I'm sorry about that.
And apparently, this page was protesting against the death of this young blogger after his arrest by the police.
And it said, even showing pictures of his disfigured corpse, I think, at the beginning, I guess, maybe.
And according to the author of one of the texts that we had to read today, Gerbaudo, this page was successful because it individualized Facebook interaction.
I think he was talking as if he was Khaled Saeed, actually, at the beginning.
And at some point during the revolution, apparently it became kind of a place to collect evidences against the regime and against the police brutality, and a sort of megaphone that was relaying information about events that were coming.
And apparently, it also served to build up the confidence about the movement.
So my question was, when did you see this page for the first time?
And how did it affect your motivation to kind of get involved?
And do you believe that, when you saw that, did you believe that it was an important, a positional movement, that was kind of rising in Egypt?
Was it the proof that something was going on?
So that's a nice question, because I was not on Facebook when the revolution started.
So I had no idea even, like, why people even went on Facebook.
It was just I could hear my friend talk about it.
I was just horrified.
I was like, [INAUDIBLE] there.
So no, I was not on Facebook.
I did not have a Twitter account.
I didn't even know Twitter existed.
I knew about Facebook, but-- and I didn't have a television I still don't.
So my relationship with internet was a good relationship, but it wasn't like a very-- I wasn't like a freak surfer on the net.
It wasn't like I was spending hours looking for things, or using it in ways that I use it now.
So I wasn't aware of that page.
I remember when Khaled Saeed was killed, and I felt something is about to happen.
And this was maybe 18 months before the revolution started.
It was in June, 2010.
The revolution started in January, 2011.
What I can say-- I mean, of course I saw the page later.
And I think it had an extraordinary impact.
But I think the interesting story here, which-- to go back to some of the images-- the interesting story here is how this Facebook page began.
Because we're not talking about making a Facebook page and-- this is [INAUDIBLE].
So we're not talking about making a Facebook page now, or in a normal kind of situation, or in a kind of relatively normal situation.
I don't know what the normal situation would be.
We're talking about a Facebook page under Mubarak regime.
And at the end of the Mubarak regime, Amn ad-Dawla, the State Security, was incredibly powerful-- probably one of the most powerful securities for sure in the region, and probably across many other countries in the world.
So you're talking about a very corrupted time in history, and a very, very intense time, where there is no way anybody could say who they were or say anything about the regime, because they would be immediately arrested-- let alone on Facebook, which is a public space, as you know.
So the story goes that there was the administrator of Google Egypt, who was in Dubai for a conference, who's a young guy who's now in LA, Wael Ghonim.
And one of the activists of one of the very small but very new and probably, obviously, very important-- that became very important-- political opposition group that formed itself and called itself Sixth of April-- formed itself after the events that took place in 2010 of all the workers in the Delta in the north of Egypt that rebelled against the conditions of how they were working and how much they were paid and so on.
And a lot of these people were killed.
A lot of them were arrested.
It was pretty much-- it was one of the very important moments pre-revolution that marked this real disruption between the corrupted regime and the people.
And so after that day, that group formed itself.
So one of the people from that group was in that same conference, and sat next to Wael Ghonim, but didn't tell him who he was.
And Wael Ghonim didn't tell him who he was.
They were just on internet, as people do it in internet cafe, or when they have-- like you guys were at the beginning of the class.
They sort of sit together, google stuff, and sometimes talk.
And at one point, one of them said, how can we get people to actually participate?
How do we collect people?
How do we bring people's attention?
What would we do?
And so Wael told him, you know, let's make this Facebook page.
And you can do this, and you can do that.
And that's the way that you kind of attract people's attention.
So I don't know what was the conversation exactly.
I wish I was there.
I wasn't.
But from that conversation, and not knowing each other, and still not knowing each other afterwards, that's how they created this Facebook page.
And people started to join and like, and like, and like.
And before we knew it, there was thousands of people on the page.
And of course, with the events in Tunis followed by the events that started to escalate in Cairo and Egypt, the Facebook page became more and more and more kind of catalyzer or something that was about to happen.
So I think that's the real role of that page.
Now, that page was essentially addressed to people who are in the age range of age of Khaled-- so late '20s.
I don't know what's the percentage in America, but in Egypt there is 65% of the population is below 30 years old.
That's huge.
We're a very young country.
There's lots of babies everyday that are born.
So it's like when you're talking to somebody who's 27 being killed-- imagine the impact on the similar kind of people in the population.
So that kind of merging of moments, events, the fact that also it's a generation that, as you know better than I do, lives on the internet and knows internet since they were born-- it was a very natural thing.
So just sort of like all these elements kind of contributed for something to happen
So it reaches more the youth.
The authors that we had to read for today, they seem a bit skeptical about if the social media were really motivating people, mobilizing people.
There's no doubt that they did, but what was important of it compared to ground-level organization?
And so in the tweets on the street, Gerbaudo mentioned that it took a few months before actually the Facebook RSVP translated into massive physical attendance in public spaces.
And also, the DFK [INAUDIBLE] from Tunisia.
And he mentioned also that the Occupy Wall Street Movement was not magically born out of a Twitter hashtag, but gained importance after a slow process of ground-level organization.
So I know it's a tough question, but do you think that the Egyptian revolution would have happened without the digital technology, without social media?
And or maybe in a more easy way to answer, how important was the role of Facebook in recruiting part spent and mobilizing people, bringing them to Tahrir Square?
I mean, when it happened-- and this is why this image actually is very important, because suddenly there's this young woman.
And she's in front of the police.
To see this in an Arab country was in itself an image that shocked people.
Wow, all these women were at the front line of the revolution.
I mean, they weren't just standing at the back.
We were actually at the front.
And so to see that, and to see that generation kind of come out and not want a revolution.
They didn't come out asking for a revolution.
They actually came out asking for dignity.
So please stop the police from abusing us, because I don't want to get into details, but most people have been abused by the police, and sometimes sexually abused by the police.
So there's a huge amount of torture that came out-- a revelation about how many instruments were hidden into these caves of all these police stations.
The context was that you have to imagine the Khaled Saeed case leading to the Facebook page, leading to bringing all this generation-- essentially that generation-- to the street on the 25 of January.
That was a very important moment.
And yes, it came out of universities.
It came out of a new generation that was kind of like, why are these older people not doing anything?
What's wrong with them?
And very quickly, when the whole thing turned-- very quickly became violence.
So from the first night, the police attacked all these demonstrators, even though they were much less in number.
But they waited, as they do, because they're cowards.
They waited until everybody went home.
And then they attacked when there was less people in the street.
So that happens, usually, at midnight.
So then this two days that followed, there is more violence that takes place in the streets.
A few people get killed-- the first martyrs of the revolution.
And that sort of stated a kind of enough.
Now that's it.
Those kids went to the streets.
We adults are actually cowards for not having done what they're doing.
We've been waiting for 13 years to even have the guts to do that.
And they kill people.
And why?
And it's enough.
And so on that Friday, which was called the Day of Anger, the whole population went out.
Everybody went out.
Every single per-- and we're talking the day where there's no more internet.
There's no more telephones.
There's no more anything.
So everybody went out.
So yes, there was a trigger that was associated to Facebook.
And people called it the Facebook revolution because there was something really beautiful about it.
There was people who were young, people who were courageous, people who died.
There was a real desire for change.
There was something very innocent, and very, very really very hopeful.
And the shift happened very quickly when that whole population followed, but very quickly also turned into who's going to take advantage of what's happening?
So this kind of unfolds as the revolution unfolds.
But the role of the social media at the very beginning is really a kind of trigger and a sign that this need for change comes from this generation that wants to have a future.
But then, everybody else who is politicized and everybody else who is part of the society also has different interest in things changing or not.
And so then a lot of other things happened.
And then the media, as we know, also take on another role as everything changes every minute, and then every day, every week, and so on, and so forth.
So yes, it's a very important role.
But it doesn't define the society because it's the trigger of something at the beginning.
And also it doesn't really-- what this is not addressed also in that image-- kind of utopic image-- of watching all these young people working towards something they're dreaming of-- is that most of the population, especially people who are in the countryside, which has been very ignored during the revolution, especially by the foreign media-- everybody was focusing on Tahrir.
Tahrir is just a little area in Cairo.
It's actually not even that big.
And yet there's Egypt.
And Egypt is essentially countryside and people who are peasants, or people who are workers in the delta.
So all of these people are not necessarily connected to internet.
They don't have a Facebook page.
So two million people signed up on Facebook in the first 10, 20 days of when the internet came back.
So I'm one of them.
So then you have also this whole relationship of how people have focused on that first moment.
And it kind of colored what happened after.
But no, it's much more complex than this.
And the economics of the country is at the core of the movement.
The different religions are at core in the movement.
There's a lot of other levels that we can get into.
But it's not the subject today
Yeah, considering the low level of internet connection in Egypt-- there was 25% of the population connected in 2011, according to the statistic that I saw-- and the low alphabetization rate-- 65% compared to 99% in the States.
And the fact that only--
So you found that number?
Yeah, but--
That's huge.
That's amazing
Well, alphabetization is something subjective.
So I'm pretty sure it's not 99% that read very well
99 is not bad
Yeah, I mean, so it was the one beside the 65%.
And only 4% of adults were on Facebook before the revolution, so it's not much.
The question is, how did the techno-savvy youth manage to bridge the digital gap, the digital divide, during the Arab Spring?
And were they employing other techniques, other means, to kind of get these people who were not connected on the street?
I think that's part of the problem, is that-- I mean, I wouldn't associate that problem to necessarily-- people were using the tools they have.
But very quickly, it appeared that you have community.
It's a little bit like being in a university.
So you have your group of people.
You have your community.
You have those do sports with.
You have those you have the same classes with, so on, and so forth.
And in a way, they start talking to each other.
But they're not really talking to everybody else.
And so suddenly, when you start-- anyway, I got on Twitter very early on-- as soon as the internet came back.
Because I didn't have a television.
I had to know what the hell's going on.
So I tried to figure it out.
I didn't really like Facebook, but I was, like, wow, Twitter is pretty cool.
So I really got it very quickly.
And I got really agitated on Twitter very quickly.
And so I'm, like, OK, who's important?
Who's saying what?
So you start following people, and understanding, and kind of mapping who's the activists here, and who's interesting, and who's saying stupid things, and who's saying interesting things, and who's calling for violence, and who's not.
So it's actually quite funny.
Because it's like this whole thing is unfolding.
And you have all these characters without face that you discover.
And you become part of that whole film, in a way.
And the interesting thing was that some of those characters or activists or participants were really focusing on this notion of organization or this notion of criticizing or freely talking.
Because now we could talk.
But there was no sense of-- I never felt that they were talking to me.
They were always kind of talking to one another, like knowing who was on Twitter.
And they were kind of talking to each other.
And then the media, after the 18 days that toppled Mubarak, the international media was much more present and able to document the revolution.
And of course, the first thing they did is they transformed people into heroes and icons of the revolution.
And yes, some of them were.
Some of those people were amazing.
And they're still amazing.
But what it did is that it kind of intensified the sort of ego of some of these people.
And it intensified their importance in a movement, and kind of blurred the actual reasons why this was happening, and kind of almost disconnected them from the rest of the society.
And one of the big criticisms of that generation, whether it was in Alex, in Suez, or in Cairo-- doesn't matter-- was that nobody was addressing the countryside.
Nobody was addressing the [INAUDIBLE] around Cairo.
Nobody was addressing the actual role of all these people outside of Tahrir.
Go down the street, and go to another area.
And see what's going on.
So this became really problematic, because the way the focus became so about the square and so much about some of these important activists, it also created a kind of disconnection with the rest of the communities.
And as young people are-- they're very passionate and they're very radical, and sometimes a little bit blinded by their passion-- so they tend to kind of obscure themselves from the possibilities that-- to be strong, they need to have people on their side.
And I think one of the big mistakes was that they were so, so radical and believed so much and in a certain way for democracy to be implemented that anybody who didn't believe the same thing, they kind of dismissed them.
What they forgot to do is that they didn't realize that who they were dismissing were actually the majority of people in Egypt.
And so to lose the majority of people in Egypt is also what got us to where we are today.
And that majority said, enough of this bullshit or mess in the street that's happening.
And let's go back with at least having our military regime taking care of security.
So I think that was one of the big mistakes, is that you throw yourself into a movement, but then you forget to bring people in with you
Do you think-- I read that there were some tactics to kind of agitate people with face-to-face relation.
Like people were going in the street, and there were some regimen points at different places in working neighborhoods.
And people were meeting there, and then they were going down the street chanting slogans, positive slogans, to kind of drive more people with them walking towards Tahrir Square.
Do you think that these kind of tactics are more efficient than the Facebook or the digital media tactics?
Or it's kind of a combination of both that makes it more-- that allowed to reach more people, more diverse crowd?
I think it's all of-- it's everything together.
It's almost like it's something-- you have to imagine it like something climaxing.
So you have all these things happening, but you're not really aware, maybe, that they're happening.
But suddenly, everybody finds themselves in the same place and things collide.
And it's that explosion that's actually incredible.
And that creates that fascinating time that we lived.
And so it's a combination.
It's like a timing.
It's like a moment where everybody was ready.
There's maybe some [INAUDIBLE] I don't know.
But who knows.
But there was a real kind of an invisible march towards that time.
And the tactics that where you were-- of course, they started on Facebook.
As [INAUDIBLE] Facebook, which I find out later, prepares people for the 25th of January, saying, go down with an onion in your pocket, with a bottle of vinegar, and with a mask if you can.
And so I found myself in Tahrir on the Friday, on the Day of Anger at 10:00 in the morning.
I didn't even know there was marches going from these mosques and all this.
And I just knew everybody was going to meet in Tahrir.
But I didn't realize there was a kind of movement that was going to actually take place.
So I go before everybody, very early.
Which means that I find myself in the center of the city where then, few hours later, there's already demonstrations.
So people are marching up and down the streets.
And there's already these groups of people kind of occupying the main streets of the area around Tahrir.
And there's police kind of blocking the entrance to Tahrir from the outskirts of the city, of the center of the city.
And all these marches kind of come towards the center, and of course, confront the police.
But I'm also inside.
So I'm confronting the police from inside.
It's like a big mess.
And it's really funny.
Because suddenly, I'm, like, what the hell is going on?
And then boom-- tear gas.
It's horrible.
I don't know if you've experienced it before.
But you start really coughing and everything hurts.
And it's really itchy and so on.
And some guy next to you will give you vinegar and you wash your face with vinegar.
And so it's all these tactics that were shared on the Facebook page.
But then when you're on the street, you're dealing with the street.
You have to react quickly.
And you have to hide when you have to hide.
And you have to understand violence very quickly, and understand where you stand in all of this.
So it's a mix of always adapting and having people with you that everybody supports each other.
So there's a real solidarity that's very natural that takes place.
Because when you have a very clear enemy, it's very easy to be friends with your neighbor.
When you're not sure who the enemy is, then everybody's your enemy, in a way.
So it's a very interesting time, because it's very clear who the enemy is.
And so the energy and the kind of togetherness is incredibly strong.
And so methods of responding to the very fast-changing times that we're living in those 18 days, and then a bit slower as times goes in the aftermath of Tahrir, is all about people finding solutions to everything.
And those solutions are shared on Twitter.
They're in the street.
And maybe you can show-- I don't know where it is in the sildeshow, but the way Tahrir [INAUDIBLE]
I have photo of this
Yes, here, the camp.
Can you do it to slideshow?
So this is an image of [INAUDIBLE] and how it was organized.
And so at the center of it, under the tents, you have the blogger tent, the media tent.
Then you have an area where you have the medical part, where people who were hurt were being helped by whoever's a doctor, was actually sudden improvise themselves as making a hospital.
And the space in the square where people could be taken care of.
On Twitter you would have suddenly hashtags of medical supplies and Tahrir Square-- different types of names and people improvising accounts, but also organizing in little groups.
And saying if you're going to bring something, bring it-- let's focus.
Not everybody brings band-aids.
If you want to bring something, this is what we need.
This is what we need.
And this is what we need.
And this is where we are.
And so you can start helping people by following what's going on.
So it's a combination of street intelligence and communication online, and communication between people.
So it's a very fast-moving time and vary fascinating, because it really plays out on so many layers
There's kind of an interaction between organizing on the ground, organizing in the social media.
I want to switch subject and talk a bit about the role of social media for bringing the world attention to the cause.
There's no doubt that Facebook, YouTube, Flickr, Twitter facilitated the citizen journalism, helped desseminating the news, showing the police brutality to the world, eliciting international attention.
And similarly, the activists involved in the Occupy Movement creatively used social media and cutting-edge technology like live streaming to kind of bring the world attention to the encampment and to the cause itself.
And both movements really managed to secure an impressive degree of public support.
In your article, "When Seeing is Belonging"-- I'm referring to it a lot.
I really liked it.
You mentioned that the image of Tahrir Square circulating on the web and in the news, and I quote, "Turn all established cliches of Egypt on their heads."
And I really like this idea.
And I would like to ask you how exactly did these images change the world perception of Egypt?
It feels so old already, and sadly so, because of what's happening now with ISIS and all this horrible stuff.
But at the time, if you remember, 2011-- this is like 10 years after 9/11.
And there's been this whole War on Terror and this whole really-- I mean, it's not new from 9/11, but it's multiplied.
And it grew exponentially after 9/11, that whole visible of the Arab world, and putting everybody into an Arab is a Muslim and Muslim is a terrorist.
And it's like this kind of generalization of what the Arab world is.
So one of the cliches was-- I should have put this picture, but it doesn't matter-- but one of the first things that kind of comes up is suddenly, the dirty Arab-- the terrorists-- is not a terrorist anymore.
He's actually a revolutionary.
And he's working for peace in the world.
And he wants change and he wakes everybody up.
The whole world was stuck to their television looking at Tahrir.
It was one of the most [INAUDIBLE], and still is until today.
So that was one thing.
Then, if you remember, there was the Wisconsin actually happened more [INAUDIBLE] time of Tahrir.
And one of the funniest images that came out was this woman holding a banner saying, Egypt, help us.
So from America being very anti-Arabs and anti-terrorists and anti- anything that comes out of the Arab world, you have this kind of like calling for help from-- we actually are friends.
And we think the same way.
And we discovered that maybe we can get along.
So this was really interesting.
And of course, to go back to photography, and to my interest as an artist and the medium that I use in making images, what happens-- not so much in the 18 days as much as just after 18 days, the whole period, the aftermath after Mubarak is toppled is also to watch how I've experienced Egypt all my life as a kid, and all my life as an adult with this relationship it has with the West, and with tourism, and how Egypt has always been a place where-- I'm sure all of you have studied ancient Egypt when you were 11 years old, more or less.
So that's in every schoolbook in the world, you have ancient Egypt more or less at the age of nine or 11, something like this.
So it's the first fascination with the world.
And it's the first civilization.
And everybody wants to see the Pyramids.
But here we go.
We're like, we have Tahrir.
And this pyramids, which here is not a cliche.
It's actually another way of looking at the Pyramids-- which itself takes away the cliche of the idea that these Pyramids are unobtainable-- gets replaced by the birds-eye view of Tahrir.
And after the 18 days, you start watching all of these Western people that come and do tourism in Tahrir, and don't want to see the revolution, want to be part of it.
So you have people who are activists.
You have people who are just people who are fascinated by what's happening.
They want to be part of it.
They just come.
And so you see this new kind of tourism evolve, and Egypt becoming associated with the revolution, Tahrir, people, et cetera, et cetera.
And no one's talking about the Pyramids.
I'm not even sure anybody's even going to see them.
And I'm not sure how long that's going to take for them to come back.
But I think it's a very interesting sort of shift, and how overnight, that image that's been there for centuries gets kind of deleted, and replaced by this other Egypt that we're in the middle of at the moment
I will definitely visit Tahrir Square if I go
You'll be sadly disappointed, but I can still take you to the Pyramids
All right, I'd like to talk a little bit about the role of social media as an alternative to commercial media.
Earlier in the semester, we learned that a few media companies own the majority of the world's media outlets.
And with Herman and Chomsky, we saw that this was a problem, like a threat to democracy, because the mass media tend to construct a worldview that conforms to the need of corporate advertisement, valorising some political perspective and marginalizing other perspectives.
And in this context, a lot of people see internet as a space where counter-hegemonic point of view can be expressed.
And in countries like Egypt, the situation is even worse, since the media are controlled by the government and subjected to censorship.
Actually, that's more a question than an affirmation.
How biased was the Egyptian coverage of the revolution?
And to what extent all these events happening were censored by the regime in the mass media in Egypt?
I want to maybe answer this and look a little bit-- can we come out of the slideshow so we see a little bit?
[INAUDIBLE] So yeah, it's a joke.
It's propaganda, but old-style.
It's like grotesque.
Grotesque-- you know that word?
So it's really ridiculously-- it's one big lie.
And it's sadly almost funny.
It's dark humor at that point.
And when the revolution starts, nothing really changes.
I mean, actually people who did not want to continue to participate to that lie left their jobs at the state media.
And the state media continues to have a huge amount of power, because of what I was saying earlier, is that most people actually don't have internet.
A lot of people do, but a lot of people still don't.
And a lot of people believe in their army.
I mean, why we're back in the military regime is because Egypt has a relationship that's a little bit incestuous with their army.
We have an army that is not really that interested in keeping the country protected from potential external enemies, as much as we have an army that is there to control us.
But that army is made of us.
So every single family in Egypt has at least two or three people that are in the army.
So nobody wants their army to be at war, either with them or with anybody else, because that would be their son, their father, their brother could be endangered.
However, the army and the regime control the media.
And so at one point in the evolution of the events, and the continuation of the role of the army post-Mubarak being toppled-- so we go from Mubarak to having the rule of the SCAF-- the Supreme Council of Armed Forces, which is the army.
We have a whole series of events that take place.
And one of them is a very terrible massacre that happens in October, 2011 in front of the state media-- Maspero-- this huge building that is on the Nile in Cairo, where the media is broadcasted.
And there's a demonstration because a lot of-- at the time, a lot of Christian churches are being attacked.
This is a period for months and months and months where we don't have police in the country, in the city, and nowhere.
So we were very happy, actually.
But there was still a lot of little crime, given that there is no police.
So of course there is going to be crime.
But these crimes happening in upper Egypt lead to these demonstrations to ask for the Supreme Council of Armed Forces to protect the Christians in Egypt who are a minority.
That demonstration leads to the army actually rolling over people.
And the whole thing is atrocious.
People get rolled over.
The photos-- I can't even tell you the horrible photos that surfaced on Facebook and whatever.
Lots of people died.
It was really, really ugly.
And of course, the army goes on TV and says, oh, but we don't use live ammunition.
We don't actually use live ammunition.
We didn't do anything.
And so this graffiti says [ARABIC] liars.
And it has the army cap.
So [ARABIC] becomes a movement where people do sort of video flash mobs in the streets projecting videos of martyrs and others at very impromptu moments in the street, saying the army is lying to you.
And this is in a popular area of Cairo
And what about the American media?
Because we saw in the course that the American media tend to be very biased, especially when they cover international politics.
And we know that the States were supporting the Mubarak regime to a certain extent before the revolution.
So were the American media also biased, very biased, or where they more fair during the revolution?
I don't know if [ARABIC] is-- what [ARABIC] is-- this one.
Do you know what this child is?
Anybody recognize where this is?
I'm not sure where is it from, but--
It's Russia
It is, right?
So this is quite funny, because with your text
Let's talk about the role of social media in organizing the protests.
So we often refer to-- the scholars often refer to social media as a means to coordinate gatherings, road blockage, protests, other forms of actions on the ground during the Arab Spring and the Occupy Movement.
Apparently Facebook was used to set the date of events.
Twitter was used to connect people across distance and to share logistics in real time.
As you said earlier, you have to react quickly on the ground.
So sometimes Twitter was saying this is the rallying point.
This is the location where the police is.
This is the things we need, the medicine we need for-- because there were people injured.
And Gerbaudo uses the expression "choreography of assembly" to describe the role of social media in facilitating the physical assembly of highly dispersed individuals, as well as the scene setting and scripting.
So how did you use, yourself, the social media as a means to organize your day, where you're going to go, what you're going to do during the revolution?
I mean, again, I wasn't a Facebook, Twitter person.
So I was using it more to kind of know what was going on, and to make sure I wouldn't miss something dangerous or something I should know about.
So it was very-- and actually, for a little while, I always felt a little bit late.
Like, I always felt like I'm really sure what's going on here.
I spent a lot of time trying to make sense of it.
And in retrospect, talking to people, everybody was in that same situation.
Because no one really was expecting this to happen.
So everyone was trying to figure out who was who in all of this.
Who's doing this, actually?
Is there somebody doing this, or where is this coming from?
So there was a sense of first understanding minute by minute where you stand.
Because this is also where I live.
This is my life.
So for four years, I lived in the middle of-- I mean, one night, one day-- I don't know what time it was.
It was maybe 11:00 PM or something.
And I live-- the Nile is not so far from me.
And on the other side of the Nile is a whole area that is called Imbabah.
And it's a popular area.
It very, very, populated.
And Cairo is 7 million people.
So it's a very, very dense population and dense city.
And I live in an area where there's a lot embassies and schools.
And it's between two branches of the Nile.
And so on one side is Tahrir and on the other side is other areas of Cairo.
So I could walk to Tahrir.
But at the same time, I was across the Nile.
So there was a sense of, also a feeling of security.
People actually moved back to where I lived, because it was almost the safest place to be in Cairo, in that period.
But all of a sudden at 11:00 PM, I get up.
And I'm, like, what the hell is going on here?
And I look down at every window from my house.
And I've never seen this.
All these tanks are just driving super-fast in every street, and heading towards this area.
And I mean, this was never seen.
You never saw this.
Or wake up in the morning and go to my kitchen.
And I'm about to go to Tahrir.
It's really early.
We're still in the 18 days.
And I look at my window, like, what's going on here?
And I see the building across the Nile that is burning.
So this whole area is burning.
And that was the Day of Anger.
Also, the following days after the Day of Anger, everything had been looted and so on.
So it was constantly like negotiating the danger zones, the moments where you should be in the street, the moments where you shouldn't, what you're ready to go through or not.
I wasn't necessarily ready to get killed.
It wasn't really-- I don't think it was a smart thing to do.
So for me, Twitter, much more than Facebook, was really a way of making sure I would know what the kind of tension-- where we were at in the tension scale.
And so it was very interesting, because I was always in the Square.
But I also always was aware that, oh, the army's coming in half an hour.
We're going to get into a battle.
Tonight, there's going to be a sucker.
And this would happen.
So before this would happen, I would just go home and say, I'm not going to get into this.
So it was really interesting to watch also that movement on that.
But it was also very confusing, because some people were also acting from the spur of the moment and a bit of hysteria.
And so not everything was real.
And not everything was true that was being spread on social media.
So it's this whole constant kind of figuring out what is real, what is not.
Is it really happening?
Who's there?
Someone you trust?
Try to get somebody on Twitter that you think will not be talking bullshit.
And then you have, also, groups that form themselves according to how the events unfold.
And so some of them are focusing on giving you the right news.
Some of them are focusing on giving you information about what medical supplies are needed.
Some of them are organized.
And this was really incredible.
Much later in the game, we had a huge amount of sexual harassment in Tahrir.
So you have this amazing crowd, especially of women, that got together-- and young women-- that got together to organize bodyguards, and people who would help people go through the square without being sexually harassed.
Of course, they got sexually harassed themselves following that.
So it was pretty crazy.
But you had this amazing amount of energy of people really putting themselves together, and coming in groups, and creating a different support system.
So this is really fascinating, in how people used social media.
And that way was really beautiful
The authors that we had to read for today, they seem to be convinced that the most successful movements are always driven by face-to-face relation, on-the-ground organization, not only by social media.
And moreover, they would go as far as saying that, when we put all our energy on building websites, innovative software, technological infrastructure, we're taking away the focus from building more movement and alliance on the ground, and connecting, like making real relationship with local communities.
It may be more accurate for the Occupy Movement.
And there's also this idea that the virtual proximity that we can experiment online doesn't compare to the community feeling, the body density, of the protest camps where people eat, sleep together, defend their territories together.
So there's this proximity of the bodies.
I don't know how you feel about this.
Do you think it's important to keep this face-to-face interaction and physically occupy a space during a revolution?
Or a revolution can only happen online?
I don't know.
I'm not a specialist in organizing revolutions.
But I mean, I don't know.
I mean, it's--
Maybe a more easy question for you, because it's kind of abstract, but did, at some point in the revolution, Tahrir became a kind of coordination platform more than social media?
Meaning like you could get information on the ground?
No, no, of course.
It was very different what happened on the ground and what you could-- depending on-- I mean, if you are somebody on Twitter since two or three years, I think you would probably have a much more understandable kind of map of actions, events.
And you would probably have been aware of the Facebook page of Khaled Saeed and so on, much more before all of this would start.
So if you're part of that generation, or a journalist, again-- were the main two categories of people that used Twitter, for example.
I'll leave Facebook out for a while.
You would be really aware of which kind of people have an interesting perception of the events, and how to negotiate that.
And then on the ground, you would be, therefore, definitely magnetized or attracted by different areas in the square, and where people were organized, or understand better the kind of actual mapping of the ground of the picture I showed you where you can see the bloggers and all of this.
I mean, as far as I'm concerned, it was kind of weird.
Because I lost my phone very-- not only there was no internet, but I also lost my phone, like two, three days after the whole thing started.
And of course, every shop was closed, so I couldn't get a new phone for a few days.
So I really wanted a phone just to call my mother in Lebanon and just make sure she wouldn't worry.
But I was completely disconnected from my normal, daily life.
And I ended up being much more connected to people on the streets, and knowing people on the street, and figuring out how the square is organized, and participate with that.
I think your question can be answered by yes and no, because we're talking 18 days where something very specific, where people really conglomerated to Tahrir-- and in Alex and every other city.
We're always talking about Tahrir, but there is the rest of Egypt.
But something, again, in that period, you have a very clear enemy and you have a very careful goal.
But then afterwards, it's a very different situation.
And then so the negotiations that happened on the street or online are a very different kind of-- they've become very different in terms of the dynamics and how they influenced what happens.
So there were periods where Facebook and Twitter were at the front of what you needed to look at to really know what was happening.
So I would say this was for sure the case during the 18 days.
At the later stage, there was much more focus between foreign media, local media and social media.
So it was really covered by everybody and everything.
And that was like a total concentration of media from all directions.
And less action on the streets, because the sit-in finished.
Some people stayed, but not the majority.
And then you have other waves, where there's much more analysis going on, where the politics on the ground are not as violent, or not as regularity violent, or not as regularly intense.
But what happens online is extraordinary.
And you have like 100 people talking, every day writing blogs, and so on.
Plus, television becomes occupied by talking heads.
And it's talking heads, talking heads.
There was two years of talking heads, of people finally talking.
And so it's analysis.
So what's going on?
What do you think is going on?
What do you think is going to happen?
What do you think of these guys?
What do you think of the Muslim Brotherhood?
What do you think of the army?
And we could say anything we wanted.
We just didn't realize we wouldn't have that much time to continue to do so.
But it was quite an extraordinary period.
And so again, it's this kind of power of what has more power at a different moment than another moment?
And so it's also, again, a kind of balance between how social media is influential, or how media is, or how being on the ground is.
And there was a very clear moment where people said, you know, it doesn't do anything to go to Tahrir anymore.
We have to go back home.
And we have to do the real work.
And the real work is change very fundamental aspects and values within society.
It's not going to the street and rebelling.
Rebelling is important, but then you have to follow up.
And follow up means you have to educate people.
You have to work with them.
You have to get involved in your communities.
You have to act directly with people that are your neighbors.
And that can take 10 years, 20 years.
And not everybody wants to spend that kind of time doing that.
So that's, again, another subject.
But that's the kind of landscape
I want to make sure we have time to talk about your current work.
So we can go more quickly on the next questions.
In the book we had to read, Digital Rebellion, there's the one problematic aspect of the contemporary activism that was underlined.
It is the difficulty to connect with local struggle, local communities, people from the working class.
So we said this before.
But also, the fact that these movements are using the new technologies and internet so much that it tends to favor the leadership of white, male, college-educated activists at the expense of homeless people, unemployed, women, people of color, indigenous.
was it also the case in Egypt?
How diversified was the crowd on Tahrir Square?
Were there, like, people from very different background, women and men?
Or they were more favoring men?
So in the 18 days, it was very different.
Everybody remembers the 18 days as something unique, and then there's the rest.
But during the 18 days, of course, it was mainly occupied by all these young people that were on Facebook and did this Khaled Saeed page, and so on, and who were coming out of universities, and so on.
But there was also a majority of population from every kind of-- every city, every area of Cairo, from Alex.
People were very diverse, people who were in the square.
But there was a kind of attitude that was very particular during those 18 days.
And again, I think it has to do with the clarity of who you're fighting.
And so nobody complained of sexual harassment in those 18 days.
Although I think I'm sure there was.
But I mean, it wasn't like-- there was something that held people together, because it was so dramatic.
And it was so-- we didn't know what's going to happen.
And we knew that if this fails, we'll all end up in jail.
So it couldn't fail.
It's too late.
We've gone already too far.
And actually, everybody is in jail now.
Most of the people who were really, really at the front of-- like obvious activists are, for a lot of them, either in jail, or not even.
There's thousands of people that have been arrested.
And the others have left the country because they've been threatened to be arrested.
So you have a real escape of intellect as it happens in those situations.
But the general sense is that, in the 18 days, you have very clear objective.
And everybody's together towards that objective.
So nobody hurts anybody.
And time hasn't come yet to divide people.
So it's much later, after Mubarak is toppled, that suddenly everybody is like, OK, so who's going to be in power now?
So I want the power.
No, I want the power.
No, I want the power.
So there's this kind of-- now society gets divided.
And people remove their masks.
30 years, maybe actually 60 years, where the guy who sells the tomatoes downstairs, or the man who you buy your bread from-- you don't know what he thinks.
You don't know who he is.
You know he's Muslim, or you know he's maybe a little bit conservative.
But you're not sure-- you don't care, even, what he is.
I mean, personally, I don't care.
But suddenly, after the 18 days, I know this guy is a Salafi.
And then this guy is a Muslim Brotherhood.
This guy is against the Sadat.
I mean, everybody becomes who they really are.
And they're not afraid of showing who they really are.
So the beards grew.
Also, the young people like you started to have beards.
And I was, like, why is everybody growing a beard?
And I started asking everybody, like, OK, I get it that Muslim Brotherhood will have their beards, because they've been arrested for the last 30 years and they've been in jail for the last 30 years.
So, but why you, my friend, why-- I know you are not the Brotherhood, so why are you growing your beard?
What's up with this bloody fashion?
And they're like, well, when you're young, in your early 20s, 30s, the police arrest you systematically if you're a man and you have a little bit of a beard.
So actually every young guy in the street let their beards grow, because finally, they were not harassed by the police for being maybe a Brotherhood or maybe not.
So it's really interesting to watch this happen.
And of course, when the military came back after the Brotherhood took power and so on, it was this joke on Facebook, which was the Bic-- the company Bic that sells razors-- is going to make a fortune.
Because everybody's going to have to shave again, which is what happened.
Nobody has a beard now.
You don't want to have a beard now.
It's not a good place for that
I want to talk a bit about the role of social media as a emotional conduit.
I think you had a video about that somewhere.
So it's clear that social media cannot be reduced to a purely instrumental tool for organization and tactical coordination.
It also participates, as Gerbaudo says in his text, to assemble a construction of a collective identity, a sense of togetherness, as you said earlier, among activists, physical occupiers, and internet users.
And this is kind of clear with the page of "We are all Khaled Saeed", but also with-- if I can find it-- the Tumblr page blood page of Occupy Movement that was kind of collecting different testimonies of people who consider themselves as the 90%, 99%, kind of as a rallying point for the construction of inclusive popular identity.
And this moment was really appealing to everybody.
So do you think that social networks serve as an emotional conduit that condense sentiment of indignation, anger, pride, victimhood?
The other example that the Facebook page, "We are all Khaled Saeed," [INAUDIBLE] maybe show about that?
This one-- oh, we show it already
Yeah, that was with the army
But did you feel that in the end, the Facebook page was really canalizing feelings and generating people?
I mean, we are in a world that is not just a generation of 25 years old.
But we're in a world where we all use internet, and we all use our computers.
We all use social media-- I mean, the majority of people does.
And we are also at a time when after-- again, I repeat-- after 30 or maybe 60 years, even, of a military regime, you have a little bracket of time where everybody's allowed to talk.
And so imagine the avalanche of information that we get.
So from not being able to do anything-- talk, take photos in the streets without being harassed by some kind of policeman that is in civil clothes-- or by a sign that says "Do not take photos here because it's a military base," because Egypt is owned by the military.
So every street is military base, really.
You have from that situation, you have a complete shift overnight to everybody can say who they are.
Everybody can say what they think.
And so everybody does.
And in fact, it was pretty amazing and magical and very creative.
It was a very, very creative period where people were allowed to be the best of themselves.
And they were.
And so the random citizen became the biggest artist.
Suddenly, overnight you had this amazing video, very funny or maybe very powerful or very emotional or very poignant that would be posted on the net or on Twitter or whatever, and would communicate something.
And people, again, created not just with a hashtag or not just with a Facebook page, but they created communities.
And so you have thousands of websites that were born from the merging of people to collect.
So for example, you have-- this can kind of go into the work I'm doing now.
But you had suddenly a group that organized to collect all the papers, transcripts, messages that were circulating every day in Tahrir, according to what was happening.
And this is, in itself, an extraordinary amount of documents.
And so they would take these documents, and scan them, and put them on the net.
And so this is a website called TahrirDocuments where you can go and you can see, according to the date, what was being circulated, what was available in the square, what people were giving each other, handing out.
So you have a great community-- a number of communities that have archived different aspects of the revolution.
You have people who have created, of course a lot of a music came out of the revolution.
This was one of the biggest outcome of the first couple of months.
You really had a huge amount of creativity.
And in a way, this is what I was telling you-- it's kind of the difference between activists, artists, activism, also people doing art, but in an activist kind of way, and not necessarily in a kind of contemporary art kind of way.
So it's more something that is using creativity as a tool, but as a tool to fight back, to answer something that is happening.
So there was a whole lot of expression and ways of expressing oneself that was born from the first few months of the revolution.
I'm not sure I'm answering you, but I think--
Yeah, just I don't want to go too far on what we're going to talk later about your work.
So let me just ask you this question about the logic of Orientalism of the Cyber Left movement.
Because all these new activist movements, like Occupy, Anonymous, Arab Spring and even Indymedia before, they claim to be a movement.
They present himself as a direct [INAUDIBLE] democracy with a preference for horizontal structure instead of vertical.
So basically, they say they don't have any leaders.
They don't do elections.
Everybody's a leader, kind of.
And then the authors of the text that we had to read for today-- they claim that these movements were not as leaderless-- as flat-- as they claim.
And I'm curious to know if it's the same feeling that you have.
Gerbaudo used the concept of choreographer to describe the kind of soft form of leadership that took place during these movements-- people who kind of are Facebook admins, Tweets or bloggers, who set this stage but without being in the forefront, people who orchestrate the, for example, the Occupy Wall Street advertising campaign that was really carefully organized.
So you see that there are some brains emerging in these movements.
Do you have any example of soft leaders that were organizing the stage?
I know that there were some bloggers that were maybe having a bigger influence
I mean, there's a whole culture of bloggers that was born in the Middle East, I think very much after 2006 war in Lebanon and the influence of how people used internet at the time.
And so they created a kind of community across the region, not just in Egypt.
And so of course, there's some very influential people online.
Hamalawi is one of them, because he is the one who-- he's a [INAUDIBLE] activist who is also a photographer.
And he documented 6 of April, 2008 Mahala demonstrations that I was describing earlier, which then led to the group that became opposing the power-- the 6 of April movement.
So Hamalawi, for example, was documenting already in 2008 those movements.
Because he works a lot with workers in Egypt.
And he blogged already three years before the revolution that so revolution will be Flickr-ized.
And his statement was that if we take photos, and if we manage to document what is actually happening, and posting it, and making people see what is happening, then there will be a movement that will come out of it.
And again, this was countering the media propaganda, which was not showing any of this stuff.
We all knew it, but not really, but kind of.
So of course, there was very important people who were doing the work already.
It's not like it happened overnight.
But they were not-- they were activists.
They were not trying to take power.
And a lot of them, they didn't know how to take power.
And it's not what they do
Some of them became micro-celebrities against their own will, right?
Yeah, I mean, they were happy to be
They tried to be modest
No one refuses to be a celebrity, I think.
It's kind of a human thing.
But these people are not-- because they're activists, they are not politicians.
It's not because you in the street that you're a revolutionary.
It's not because you are an activist that you understand politics.
It's not because you are an artist that you necessarily do good art.
So these things are not simple.
And it's great to have a movement.
But where it fails is that there is no handing over that moment that we have a little bit of power, but it's very fragile.
And we let go of that very fragile shift that can happen.
And instead of giving the lead to someone who actually understands politics, and is able to take over the role of a part in a political party that could hold together a country like Egypt that has 90 million people, I mean, insane.
Instead of that, you watch people disagreeing and ego fighting.
And something very important is that the nature of a revolution, and the nature of dictatorship-- so Egypt or anywhere in the world-- a dictatorship by nature is to dictate to eliminate anybody opposing them.
So there is no actual opposing power in place during a dictatorship.
And that was true during Mubarak regime.
So there was a political party called the Kefaya.
Kefaya means enough.
And they were really fighting almost alone and always being put in jail and out, and put in jail and out.
And they were pretty upset, because no one really talked about them when the revolution started, but they were the only ones really fighting the regime for the seven years that preceded the revolution.
And then you had the 6 of April movement, which was relatively new-- three years-- but still not so public.
And they were preparing a revolution for September 2011, because they had gone to, I think, the Republic of Czech and been trained in different parts of the world where there had been peaceful revolutions, that had actually succeeded.
But that was a plan just before the next election where Mubarak was going to put his son in power.
So when this happened, there was no plan.
We didn't go to the streets saying, we want the revolution.
There was no plan to hand the power over, to have somebody ready to take power if that would work.
So--
Would you say that--
That's where the gap sort of--
Yeah, because the authors that we read, they kind of argue that these leaderless movements, they don't allow to make proactive decisions, build long-term powerful organization.
And not investing in leadership will eventually lead to the dismantle of these movements
Yes, that happened
You think the revolution would have benefited from having stronger leaders to take responsibility for some failure or some--
Yes, of course it would.
But the problem was that there was nobody.
And everybody who tried was rejected by everybody else for many reasons.
So even Baradei, who came back to Egypt three years before Mubarak left, was not liked by the Egyptian population, because he's lived 20 years abroad.
And also for a very good reason, which is he was one of the actors that had influenced the Gulf War, the Iraq War in 2003 by being the one who was appointed to give the answer on whether or not there was nuclear weapons in Iraq.
And he said yes.
We never found any.
But you know, what the consequences of that is terrible.
And so of course people don't forget that.
So when he came and said, hello, I'm going to be the next president, of course everybody said excuse me?
Who are you?
So that was the case for everybody who presented themselves as a leader.
So even the Brotherhood, even everybody who-- but who can be ready when there's a dictatorship that does not-- that puts people in jail if they say, I'm going to present myself at the next election.
Next thing you know is they're arrested or they have to leave the country.
So that's the nature of dictatorship.
So you're surrounded by a wall.
You need to break the wall.
And you don't know what's behind the wall.
And you're not ready to go behind the wall.
But you have no other choice.
That's the first step.
You need to break that wall.
So we broke the wall.
We're back into another movement.
But now we know what's going on.
And now we know what's going on.
And things will not be the way they were.
It might take more time.
And it's a very different battle that will prepare itself.
But it's a necessity to go through these steps
So we don't have much time left-- ideally, 10 minutes
That's OK. We talked.
Everybody had enough, I think
No, but we want to talk about your current work, because it's very interesting.
And the obsession of scholars for the role played by Twitter and Facebook during the revolution kind of eclipsed a very interesting conversation that we can have about the role of traditional media in the revolution.
And since you're the co-founder of a radio and a cinema, during the revolution, I think you're the best person to talk about this.
So can you talk a little--
Just traditional media?
Yes, cinema and radio are traditional media compared to digital media.
So can you talk briefly about these two initiatives before we talk about your project [INAUDIBLE]?
Sure.
So again, I'm an artist.
And when all this started, I watch this amazing stuff come out of nowhere by people producing music, films and so on.
And I watched one video that really fascinated me, which triggered this kind of interest for what was happening every day.
What I was experiencing was resonating with events that I remembered from my childhood-- which I didn't live, but I remember watching on TV and being very impressed by.
So that first video of a truck and somebody standing in front of the police truck, the water cannon, but not going anywhere.
He would just stay there.
And so the police goes backwards, which is incredible.
And it's one of the first videos of courage that appears and viral on YouTube.
And it was posted on YouTube, and it was called [ARABIC] Courage Video or something.
And so this connection with the history, with another event from '89 in China of-- if you remember this video, if you know this video of this man standing in front of an Army tank and not moving, and the whole world watching 20 tanks kind of going towards this man-- triggered the need for me to see and understand what's happening in relationship to history in the world, not just to Egypt and Egyptian history.
And so I started collecting things, and archiving videos and just learning, and collecting, and organizing, and so on.
And one of the first things I did was-- I didn't want to do any art work.
That was not the time for me to do any art work.
But in the square, I met a lot of people.
And with a friend of mine during the 18 days, we wanted to do a free radio so that we could connect ourselves.
And the 18 days ended.
Mubarak was toppled and so on, but we still wanted to do the radio.
So suddenly, it was not just my friend and I.
It was actually this guy next to me wants to do the radio.
And then he knows this guy who also wants to work with him.
And then this person is doing this.
And it was very organic.
And before I knew it, we were a whole group of people.
And we created this radio.
So it was very spontaneous.
And it was a lot of-- there's one very funny show of two characters making fun of politics.
And then there was a lot of music from the revolution.
So we were kind of a receptacle for all this music production during that period.
Following that-- that's another image.
Following that, a few months later, I realized that most people probably hadn't seen all these animation videos like the one I showed you, that most people weren't on internet, that actually, not everybody had seen what really happened in the 18 days.
And we were, again, doing a sit-in in Tahrir.
So I decided I have to use the archive that I started building.
And I have to show this to people.
And there's a sit-in that's starting.
And I know it's going to take a long time.
And I'm not going to sit there in the square and not do anything.
And actually, there's no images in the square.
So we need to focus on something.
People need to see something.
So I decided to make a cinema, but I wasn't sure what and how-- all of this.
And then I'm on Twitter, and I see somebody saying, oh, we should project films.
I was, like, oh, he wants to do that, too.
So I connected with him.
And then somebody else said, oh, you want to do cinema.
I know your work.
I've seen your work.
I don't know what.
Are you going to show your work?
I said, of course I'm not going to show my work.
This is like the middle of a revolution.
Who cares about seeing my work?
But let's get together and talk about it.
And so we created the cinema and the sit-in.
And every night, we projected material that was circulating on the internet, but that most probably, most people didn't see.
And it created a space for a platform for debating the revolution, a platform for convening and sharing the archive.
So people would come with their phone and say, can you give me a copy of what you show?
Because I can show this in my area.
I can show this to the coffee next door, project it in the street next to my house.
I can show it to my family.
I can bring to my village.
And I can give you what I filmed.
And so it was a way of exchanging material and kind of disseminating, also, on the ground, footage from the revolution that was countering the media propaganda that was happening.
So these are some images of the crowds in the square watching the footage
For the last five minutes, let's talk about your project on the archiving the revolution, because it's also a concern that we should have with digital media
We even have popcorn.
Yeah, so these are the two projects that I've done as the first sort of actions or outcome of collecting material, and feeling like, to make sense of all of this, because of the collage that you saw originally, you can imagine how my head works.
And I have to have lots of stuff, and make connections and all this.
So it doesn't happen immediately.
It takes a little bit of time to find these links.
And same thing here.
I just felt there's no conclusion to be made now.
But all these questions are popping every minute in my head.
So my collection of archive was really about keeping this for when I'll be able to think about it, and when there will be a moment where things start to kind of come down and I can make sense of it and find a link that holds it all together-- if there is one.
I don't know.
And so now my project here is about looking at that period of two, three years, and trying to build a timeline of the revolution that is interactive.
So this is an example of a very fantastic, beautiful, visual timeline of the history of the world.
Which this part is about ancient Egypt, as you see the Sphinx and the Pyramids.
And I find it really fascinating, because it has color coding.
It has images.
It has text.
It has a sense of continuity.
It has layers.
So things happen in one country, but you can start seeing also what was happening in parallel to Egypt-- so the tsunami in Japan, bin Laden arrested and killed, Occupy Wall Street, and all these resonating events between Tahrir and other countries in the world.
And at the same time, being quite a personal view of it, because it's a collection of elements that I'm interested in because of the nature of the work I do and the type of material I normally work with.
And so I actually focused my archive on, again, that production-- that creative production from the random citizen and what I call the popular language of Tahrir.
So these are early sketches of my timeline made from a photo that I took, videos that I collected, et cetera, et cetera.
This is a detail.
And then this is, one of the graffitis from Mohamed Mahmoud Street that's is one of the areas that come out of Tahrir and where the street graffiti artists of the revolution have, over and over and over again, and overlaid all these fantastic paintings as things happened-- portraits of martyrs and so on.
And that huge, walled, continuous street is so extraordinary, and constitutes a really ongoing tableau vivant.
And that mix between that kind of art in the street and that influence of that beautiful painting and the social media production that we experience is, for me, sort of missing something that it needs to come together.
And so what I'm trying to do is a kind of painting of the 21st century, but that is interactive so that you would be immersed in as an audience, but also be able to kind of talk to and look at in a way that you would say, OK, I'm interested.
And your question is, what is the role of the Army or the social media in the revolution?
And so you would only see-- suddenly, the rest of the timeline would sort of disappear.
And what comes forward would be all the aspects and all the moments in which social media is actually mentioned in an article, or is in a hashtag of one or another type of media that is being displayed
Well I cannot wait to see it.
Seriously.
Is there questions specifically on the presentation?
Because we're going to have a discussion later on things that you posted on the forum.
But is there any questions related to what we spoke about?
Yeah, there is kind of a place which you said you built next to real military base or something like that?
How did they let you do that?
So the military base is a huge area that was a little bit elitist.
So poor people were not very welcomed.
And usually, they tended to be kind of kicked out.
In the middle of it is the Opera House, the Library of Music and the Pavilion of Arts.
And all of this is managed by the government, the Ministry of Culture.
But the area and that whole space belongs to the military.
And there's a military base just next to the opera.
So it's surrounded by security.
It's very well guarded and blah, blah, blah.
And one of the events is, funny enough, Egypt has one of the oldest Bienniales of contemporary art.
So it's almost as old as the Venice Bienniale that's very famous.
And so for 25 years, there's been every two years this event, this art event, which is a disaster.
And it's like a horror show.
But that year, December 2008, was one of the iterations of the Bienniale.
And for the first time in 25 years, they changed the curators from the Minister of Culture to two young men that they appointed to be taking care of that event.
And they asked me to be part of the event, which was in itself crazy, because I'm a woman.
I'm of Lebanese origin, so I'm not a pure Egyptian.
I'm Christian.
I'm from the art world that is more private, not the government type artist, so all of the things that normally they hate.
And so it was, like, wow, they're inviting me.
Of course I have to say yes.
But I also have to make a point of my participation.
I don't want to just show a collage or a piece that is an interesting piece, but not in this context.
So I want to actually respond to where I'm going to show.
So I wanted to do a work in situ.
And so I talked to the curator.
And I told him that's my idea.
I want to do this.
And I was very lucky, because he said, oh, my god, I'm really interested in these areas.
And he, because he was one of the first ones-- I mean, the first in 25 years to have something really radically happening and changing in that structure of the Bienniale, of course he was happy to do something provocative.
And he wanted to do something provocative.
And he wanted his Bienniale to be successful.
And already to choose me was a problem.
So it was really funny.
So of course, he was totally on my side.
And so I negotiated to have a piece of land, and to occupy this little piece of land.
And then I had the gardener come and tell me, what are you doing to my flowers?
And I don't know what.
I was, like, OK, I'll do them again.
Just leave me work.
So it was very, very interesting for all of these reasons, because everybody was waiting to see what's going to happen, what are they actually going to show?
Is it really going to make a difference?
And it was a fascinating, very exciting time.
And the work was really very engaging, because the people who built the tower live in these areas.
And when they understood what I was doing, they became completely in love with it.
And then the people who guard that military base are people who live in this area.
So they actually came and did their naps.
They took naps inside my tower.
The people who owned the cafes around, the library and all of this inside the garden surrounding became familiar with me building this for two months, and then showing it for another month.
And so every time there was a visitor, they would come and offer coffee.
So it became a cafe.
I mean, it was incredible.
And I would come in the morning, and find these huge ashtrays full of cigarettes.
I was, like, who smoked all this?
I would ask [INAUDIBLE] who smoked all this?
He was, like, well, the Director of the Opera House came with his meeting, with somebody he was meeting.
And they stayed three hours, and smoked all night.
And so they were having their meetings in that space.
So that space became like a space.
It was super alive.
And it was really interesting, because it helped, sort of.
People came to me and said, you know, normally, we see these areas are as crime-ridden, ugly, horrible people.
They have very negative connotation.
And now I'm going to go back to these areas and look at it from a different angle.
So it was this kind of energy, and trying to bring people to look at themselves in a different way and say, OK, wake up
Well, I wish we had more time to discuss.
It's so interesting, all of this.
But we need to do another presentation.
So I want to thank you.
Let's applaud Lara.
[APPLAUSE]
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
This is our first lab session for CMS.608 and, for folks, who wasn't here this past Wednesday?
One, two--
Who wasn't?
Who was not here on Wednesday?
[INTERPOSING VOICES]
And so, you weren't here.
Let's see, we have some blank index cards underneath the--
The plastic Tupperware, I think.
[INAUDIBLE]
So you can just fill in your name, your email address plus favorite board game or card game.
The email address, basically, just gets added to the email list.
Did anyone not get an email from Jason yesterday?
Yeah, last night
You didn't get an email?
I don't think so
OK.
So the email basically said that there was a swap [INTERPOSING VOICES]
--of Friday the 17th, and the 20th.
So, basically, we're swapping your readings around-- and that's really the only thing, right?
Oh, and I fixed the link for Zimmerman's article
And, unfortunately, Eric Zimmerman didn't migrate his website terribly well, so all of his pictures-- [INTERPOSING VOICES]
I actually have one of his games, SISSYFIGHT 3000.
In fact, this is before SISSYFIGHT 3000, so this is probably 2999, which is going to be the non-digital version, which is the whole point.
It is actually a computer game.
It is a pretty decent flash multiplayer game, back in the day when flash multiplayer games were not that common.
And if you haven't played it before online, after today, especially if you had a chance to play with the card deck, try out the online version.
Do a Google search for SISSYFIGHT 3000, and try it online.
Hopefully, there are still people playing it, because you need kind of like a minimum of five people to get a good game going
[INAUDIBLE] the one that was taken down
Aw, suck.
It was such a great game
Never mind.
Basically, what you had was, you got avatar customization in your game.
You could create your character, and you could-- you had this really cool retro pixelated look back in the day when retro pixelated stuff was not that popular yet.
But the game-play was pretty much exactly the same as what a card game has.
The readings today basically covered a lot of jargon that we're going to be using a lot of that for the rest of the class.
For the most part, I'm not going to go through every single one of them because you folks can read.
And if I mention something and you don't get it, you can always check back there.
So just remember that the first chapter in Braithwaite is a pretty good glossary of terminology that we're going to keep using.
But one of the big things that they ask is, what is a game?
And it's one of those-- Does anybody actually want to offer their definition of what a game is?
A game is something that involves players and they set up, define rules beforehand, rules that you have to abide by, whether they be, you can't go outside this limit, you have to abide by certain questions and orders.
Something that has structure that is meant for one person to go from Point A to Point
OK.
Structure, rules, limits, players.
Anything--
It's fun
Fun
It accomplishes some goal
Goal
So you always have some goal
You're working towards something
Anything else?
[INTERPOSING VOICES]
Or it's an abstract system.
Often an abstraction of some real system
An abstract system.
So these are actually pretty good benchmarks.
Now there are, again, as Brandon describes, there isn't one that's just the canonical answer for everything.
It really depends on which theorist you are reading at any given time.
But usually it will involve a significant subset of all of these in the definition.
And I like to keep things broad and vague because, for the most part, if you define yourselves too strictly, it might be useful.
You're an academic and you need to write papers about it, but if you're a designer and you're just trying to create something fun-- the word I would use is "engaging," because fun means a lot of different things to a lot of different people.
Engaging basically means that this is going to be something that you're going to want to do for a while, that is motivating.
It might be terrifying you out of your wits.
Some people enjoy that, or would find that experience compelling.
I mean they may not call it fun.
So I call that engagement.
The problem with using fun and engaging as a metric for defining what a game is, is that that means that a bad game is not a game.
A game that fails to achieve fun, that fails to be engaging, is not a game.
But I wouldn't go that far, largely because that means a lot of stuff that I created is not a game.
But it should give you the-- there is no magic point where a game becomes --the thing that you're building becomes fun and then suddenly becomes a game.
I can't even imagine someone making that argument, that it's not a game until it's fun.
But I much rather sort of say, fun and engagement is kind of more of a metric of how well designed your game is, not necessarily whether your game is a game.
But that is a goal of a game designer, to try to create an engaging experience, something that people are going to want to do.
I know some game designers who think that is not, in fact, their job.
Their job is to maximize revenue, for instance.
Especially if you work in social games, there's a lot of discussion now in how business and game design is basically just one thing now.
The whole point of game design is, how do we get people to pay a little bit more?
Zynga's going to be in town on September 20th, they're giving a talk.
That has those two points in it.
[INTERPOSING VOICES]
So I always thought of design as something that you're working towards.
But all these other things-- a game should have players.
Again, I don't want to be too strict on that because I've certainly seen games that play themselves.
Anyone here involved in Battle Code?
That game pretty much plays itself, right?
I mean, you set it going and then one side wins, but there's no human person at least during the time when the games actually being played.
Or you could argue that the whole course of Battle Code-- for those people who don't know it, it's an IP course, I believe, where Course 6 students basically write bits of AI to play a real-time strategy game against some of other team's AI.
And the basic idea is that, you win a real-time strategy game, your AI wins, but there's no human actually playing it at a given time.
But, of course, there are humans writing the AI, so if you think of the entire class as a game, then there are players, there are people.
I suddenly see people make a case that there are games that are zero player, and I've heard, for example, there's a game out there called Novick, which is largely a game about coming up with rules.
And there is a system, which are actually in the rules themselves, about how you change rules.
And, of course, those rules themselves can be changed.
And that changes the way how you change rules and, by the time you actually play the game, someone's already won.
Because somebody in the game has no legal moves.
That's the whole game, is basically making it impossible for someone else, or someone else except for you, to win.
So does that game actually have players?
Well, yes, it does because you guys are arguing over all the rule changes through the rules of the game.
But, in a sense, it had no players because the game never actually got played.
The rules are basically stating, this is an unplayable game for everybody except for one person who's playing the game.
Goals are one of those things that I used to be very, very adamant about.
Like a game had to have a goal.
You have to know what they're going towards, and if you don't have that, then it's a toy or something, it's a dollhouse, it's an experiment.
The Sims was the one that I was like railing about when I was an undergraduate.
You know-- that's not a game.
And, of course, I was working with many colleagues and eventually came across, came over to decide that it doesn't matter whether I think it is a game or not.
If it's in the game shelf of Best Buy, it's a game.
Because people think it's a game, and if enough people think it's a game, then it might as well be a game.
Otherwise-- because that's what people think it's going to be.
So when you come up with your game, I would say that goals are one of those things that helps steer player behavior.
So it's one of those tools that you have as a designer to basically say, this is important, this is not.
Reaching the end of 100 meters is important for a sprinter.
That end is really not interesting for somebody who is running 400 meters or a 50 meter dash.
But that gives someone a direction for them to go towards.
But there are plenty of games where you have to invent your own goals.
Other than Sims, can anyone think of any?
Where inventing your own goals is kind of the fun
A lot of NLRPGs
A lot of NLRPGs.
They have goals.
They have quests.
But that's not where a lot of fun is, necessarily
Second Life
Second Life.
A lot of virtual worlds
A game like EVE?
EVE?
Yeah.
EVE, you kind of, want to, like you have to set your own goals, because whatever is in the game is only good enough for a single player-- to tell whether a single player whether they're doing well or not.
But EVE is not about a single player.
EVE is about thousands and thousands of people playing simultaneously.
Actually, have people heard of EVE?
All right.
Yeah.
So, cool.
Cool
[INAUDIBLE]
Oh, cool.
In Iceland?
Yeah
Was it like really cheap when you went over there?
Yes
Could you have bought the entire studio?
No.
Unfortunately.
But it was really cheap
So they must have been making tons of money over there
Yeah, they're making money in US Dollars and the Euro.
[INTERPOSING VOICES]
They probably hire most, yeah, support most of the economy.
[INTERPOSING VOICES]
Yeah.
[INTERPOSING VOICES]
The capitol is 20,000 people so
So I'll say goals are very important tools and as a result of that you see them in a lot of the games.
And for most of the games that you guys are going to be building, you're going to give a very clear goal of this what you want to do-- earn a certain amount of money, prevent your opponent from doing anything else, take over all the land.
That sort of thing.
Let's see.
Oh, I'll get to the rules last.
But, system.
Systems are-- I might already have mentioned this in the last class, but games are systems in a way that most MIT engineers would think of systems.
They're a bunch of interconnected little modules, all of them with sort of predictable behavior, and then you put them into a big interconnected system, and it is no longer really all that predictable anymore.
At least that's the kind of games that we're interested in in this class.
There's also the game-theory games, where the whole point of it is trying to predict what could possibly happen.
And those are great thought experiments and there's some really, really interesting math behind all them and some interesting rules of thumb that can come out.
But for the most part, we aren't going to go into too much detail about that, largely because I'm not a mathematician or economist, so I don't really have a good sense of game theory.
Just be aware that there is this whole field of game theory in economics.
If you're doing a game like an auction system, if you're doing a game where people are doing either a lot of simultaneous moves or the whole point of the game is trying to predict what strategy your opponent's going to do, you might actually want to read up a little bit on game theory, if nothing else, just to give you some vocabulary to talk about it with your teammates, and some tools to be able to think through some of the design problems that you've got.
And we do have one session where I'll introduce some of that.
Finally, we are getting to rules.
The things that give your game structure and the constraints of the things you can't do.
You can't move from here to there without having your foot tied to another player, that sort of constraint.
Or a rule being you can't see anything during a certain part of the game.
There are two bits of terminology introduced by Brenda and Ian Schreiber in their book, which is in Core Mechanic and Core Dynamic.
So before I'm going to go into the core mechanic and core dynamic, I'm going to talk about mechanic and dynamic.
Anyone want to throw out the definition of mechanic?
I reckon it's sort of the actions like a player takes during a game like the physical actions he takes
OK.
The actions that a player decides to take?
Decides to take, yeah
Sorry if this is a little low, I saw Patrick and-- or did those hands just go down?
I thought I saw a few more hands
Actions that are sort of designed by the game designer.
So it's intended actions
Design is definitely a good point.
A game mechanic is definitely something that you as a game designer control.
There are-- if a player decides to do something in the game that you did not explicitly think about, and you did not explicitly say, all right this is what players are going to be doing in a game, then it might not actually be a mechanic.
It might be something that you're discovering in your play testing that you will turn into a mechanic.
But because it's not designed, it's kind of-- it becomes more like a strategy, actually, like an emergent strategy.
Sorry.
You were saying, Jeremy?
Is it more like a dynamic ?
Possibly.
There are times where it is.
Yeah.
Actually let's throw out definitions of dynamic, what do people-- well, Jeremy?
It's behavior that emerges from mechanics?
It's emergent behavior?
What else?
Use of mechanics that isn't directly stated in the rules of the game
OK.
So unstated.
I'm running out of space here.
I thought I saw a hand on this side.
No?
OK. All right.
So, yeah, dynamics are basically interactions of mechanics.
It's like you don't have anything that may necessarily explicitly say, because of this one rule and because of this other rule, and because of the things that the players can do in that, a good strategy is to do something else.
So let me see whether I can think of an example.
Right now, everything that's coming to my head is StarCraft
Has that been the case for the last few weeks?
Unfortunately, yes.
By the way, if anybody is playing StarCraft, I need testers
StarCraft II or StarCraft I?
Two
Ah, yes
I need testers for a math game I'm designing
I play Zerg.
[INTERPOSING VOICES]

So there's this role selection mechanic where everybody picks the thing which they want to happen during their turn.
[INTERPOSING VOICES]
and mostly everybody chooses to build a particular cross.
And the dynamic that emerges from the sequence of play, which goes clockwise is that it's best to optimize your strategy so that you were doing, so you were benefiting from the role of the person to your right, is likely to choose.
But you're making the [UNINTELLIGIBLE] of the person you elect is most likely win a ship.
So mechanically it's role selection and turn order and then the dynamic is the strategy written by the program
Which gives you some idea what to select as opposed-- the mechanic is you can select.
The dynamic is, here are a couple of strategies that you can use in sort of assessing what would best to select.
And if all the players start to understand that, then it almost becomes like a second layer of rules almost.
If anyone has played a game like Bridge, for instance, there's actually a ton of things that are assumed in the play which are not actually in the rules of Bridge when it comes to bidding.
Like when you call out a bid, it's highly dependent on what you actually have in your card hand.
And that's not actually written in the rules anywhere.
But the assumption is that there is optimal way to do it.
And since there's optimal way to do it, everyone's expected to know those optimal ways.
The example in the book was chess moves, like every single piece of chess has a set of moves that it can perform, right?
Pawns go one or two squares, depending where they're starting.
The bishops move diagonally.
But there are also opening gambits.
Opening gambits that are also well known, at least among professional chess players, and the idea being, oh, this person is using this opening, therefore, if I counter it with this response, I actually have a reasonable chance of actually winning this game, and professional chess players know this.
The other thing that comes up in chess, which is not in the book, is there is the concept of threatened squares.
If you move, if there is a square where there is no piece on a chess board-- everyone here knows the rules of chess, more or less?
So say you've got a bishop-- and this is quite a simple chess board-- and you've got the bishop, and it can move like that, right?
So every single square on these lines is threatened.
It's something which the bishop could move to.
Unless, of course, it's lost by a pawn or something, then these are not going to be threatened by the pawns.
If your knight obviously is no longer straight lines, it becomes a specific or sort of like stochastic pattern of squares that become threatened.
But the idea of here's this square now that nobody dares move into until something's done about that bishop.
Either something gets moved in its way, or something takes out the bishop or forces the bishop to move by threatening it instead in a way that it cannot respond.
So threatened squares is not a rule in chess, but it greatly shapes your decision making and that's the difference between dynamic and mechanic.
There's a lot of strong correspondence between mechanics and rules.
But-- and I had a long, long little rant about exactly what the difference is between mechanics and rules.
And what I basically-- I know I'm going to get the rant right now, but it's like overlapping venn diagram.
There's a very, very large area where it's exactly the same thing.
My general rubric for trying to figure out whether something is a mechanic or not is whether this is the thing that changes the state of the game.
It is an action.
It is something that someone is going to be doing to change the state of what the game is.
In chess, that's largely the movement of the piece.
If it's ultimate.
It could be just physical movement.
You are limited on when you can physically move.
If you have that frisbee, you're not moving.
Well, your feet are not moving.
I believe that's a rule of Ultimate.
So the reason why I use that definition is because there are occasionally times when I need to talk about mechanics that players aren't actually doing.
The game requires certain pieces to move in a certain way, certain things to happen in the game in such and such a time.
For instance, you have these die rolls and-- actually, that's not a good example.
Let me think.
Right at the beginning of Settlers of Catan there is the way how you set up the board, and that establishes what a game state is going to be.
It changes game state by giving you a game state in the first place.
I consider that a mechanic.
That is something that's going-- that's not one rule.
That's actually a whole collection of rules.
And if you're interested in finding out what those rules are, the box is actually in that closet over there.
There are something like five or six different rules on how you set up your board.
It's going to be, set up your game state, and I consider that a mechanic.
How people do bidding and buying things in Settlers of Catan?
That's dynamic.
There are no real rules on how much something costs.
That's up to the players to work out on your own.
So that's how I sort of consider the difference between dynamic and mechanic.
They're really, really useful concepts because really, for the most part, as a designer you have an idea of what kind of dynamics are interesting.
And the more games that you play, the more inspiration you get from various sources, your range of different ideas of what kind of dynamics might be interesting is going to expand.
But all you have control on are mechanics, the specific rules that you're giving, note that game mechanics don't necessarily need to be written or described out.
This is one thing that I think you and your teammates should figure out when you're talking about mechanics for a given project.
Are we just talking about the things we are writing down?
Because there are some assumptions that are just given.
We are playing a game, and we're not playing some sort of schoolyard game or something, hitting people with solid objects that's disallowed.
Not necessarily something you need to write down.
Unless you're playing with certain people.
You can assume that the game again -- things like how fast people normally do things.
There's a game over here called Falling, which is actually a real-time card game.
It goes as fast you can possibly play it, and there is no there is no limit on how fast you can play these cards.
But there is a real physical limit on how fast you could physically grab the card out of your hand and put it down.
So, actually, it's not even on your hand, your just moving cards on table.
So make sure you give that a try today.
Finally, we'll be getting to aesthetics, I think-- next class?
[INTERPOSING VOICES]
So, which is another thing which a game designer-- in fact, what I would like game designers to actually spend more time thinking about, which is, what is the feeling you want your player to have?
As opposed to, this might be fun, or this might be engaging.
And specifically, how is it fun?
And how is it engaging?
Is it fun as in like, yes, victory!
I have crushed all my opponents.
Or is it fun as if, well, we had a really good time together and we got to know each other really well.
There are games that are designed for both of those.
And what is the aesthetic you're going for?
Are you going for something poetic?
Are you going for something fiercely competitive?
Tournament competitive?
Are you going for just a nice little social experiment?
Core mechanics and core dynamics are a tool, again, for designers.
The idea of-- there really isn't a huge difference between what's a core mechanic and what's just a mechanic, except inside a designer or design team's discussion.
The idea of having a core mechanic is you identify this is what our game's about.
Or a core dynamic means this is what our game's about.
And a lot of core mechanics in first person shooters is pointing and shooting.
I'm going to point at something and I'm going to launch a projectile that way, and that projectile is going to move in some physical properties.
Maybe it arcs, maybe it goes straight.
I would say that the core mechanic of Halo isn't actually that.
Who has not played Halo or seen someone play Halo?
OK.
So the idea, core mechanic-- or I think my Bungie calls it the core game play loop-- of Halo is, you shoot at something, you take cover, you throw a grenade at it, you wait for the grenade to go off, and then you shoot at it again until it dies.
That's pretty much all you are doing for the entirety of Halo 1.
I'm sure Halo 2, Halo 3 gets complicated, but de-identify that as you actually read the post-mortems and if you read the talks that are given, they say those-- I can't remember if it was 20 seconds or five seconds of game play which is used to describe that.
I think that five seconds of game play is what they focused on.
And if they could get every little thing in that sequence correct, they can repeat that same thing at infinitum, and you'll still will be having fun.
Halo 1, everyone.
They're kinda right.
But the idea being, if you've got a core mechanic, and you've got one thing in your game that someone's going to be doing over and over and over again, all your players are going to be doing over and over again-- maybe it's bidding because it may be some kind of auction game.
In Falling and Pit, it's transferring cards to other people.
I'm trying to think of the other games that we got here.
In SISSYFIGHT, I would imagine the main core mechanic would be putting down the card and everyone revealing the cards simultaneously, because of you all take simultaneous turns.
Make sure that's really, really fun.
And Yahtzee is rolling dice.
That action, that thing that you're going to be doing over and over again, has to be at least fairly engaging.
Because if you can't get that right, then what happens is that that same movement, that happens 30 to 50 times in your game, just-- the badness just gets multiplied by 30 or 50.
And you don't want that.
So for core mechanics, I would say the thing that happens in your game the most-- and you want to make sure that that part is as engaging as possible because it gives you a rubric to say, what's not a core mechanic might be, in fact, expendable.
You might not actually need it in your game.
So if you've got a problem-- this game takes too long to play, this game's too hard to learn, this game's over too fast, it's too random or something-- take a look at all the things that are-- but first make sure that your core mechanics not the cause of the problem.
But if the core mechanics doesn't seem to be causing the problem, that seems to be working, something else in the game is not working.
Everything else expendable.
And so you can look at everything else in the game, all these other rules that you've got, and say, how does that work with a core mechanic?
How does that help?
How does that hurt the core mechanic?
It just helps your discussion as a team.
Or as an individual game designer, it helps you to think about what could we throw out?
What could we keep?
Similarly for core dynamic.
It's like here's this particular interaction that we really, really want.
And SISSYFIGHT is guessing what other people are going to do.
I'm trying to think of other examples.
And like territorial space acquisition.
I guess Go is about taking over territory in space.
If that's your core dynamic, you say that's kind of fun, just making sure that I'm in control of space and nobody else can do anything they want to do in a certain part of the board.
That's fun.
That's what I'm going for; that's my core dynamic.
That's what this game is all about.
Then you can look at every single mechanic, even your core mechanics, and say, is it all serving that goal?
Now there's a whole bunch of tips right at the end of chapter one of Challenges for Game Designers about how to break designer's block.
But I would say, actually, those things are just things you should be doing all the time, not just when you have a problem.
But you should be experimenting.
Things like, OK, you've got a variable.
Draw two cards every turn or something.
Multiply it by two.
See what it does to your game.
Halve it.
Draw one card.
See what it does to your game.
Through a rule-out, and that sort of gets you to the whole iterative design thing, is that you don't actually, because games are systems, these complex little collections of seemingly simple things, you don't actually know how these changes are really going to affect the game.
You may have a hunch, but until you actually play it out, you don't actually know.
So the whole point of iterative design in engineering as well as in game design is that you make the change.
It has to be a substantial change.
And then you actually put it to the test and see what happens.
This means, of course, testing, and testing takes a long time and if you're going to be making a game that takes a long time, then your testing gets even longer.
So for the first project that's going to be coming up, assignment one, you want to be aiming for games that take really not very long to play.
These games that we got today, I'm not sure about [INAUDIBLE], but I know everything else that we got on that table, is pretty darn fast.
Which means if you play an hour with four people who've never played your game before, you get through two or three rounds, four rounds, of them.
Each time, you'll be teaching a rule once, sharing and seeing how that changed the game.
If you've got a game that takes an hour to play, and believe me, a lot of the projects that we've gotten from previous semesters take at least an hour to play.
You've got four people for an hour and another definition of a single-player game.
Because by the time they finish learning your game , half your time's up.
So that's a process that we will like you all to be using in your teams and your assignments, is iterative design process.
Right from the beginning-- I wish I had more space-- but we're going to start with brainstorming.
That's actually going to be Friday, the 17th, where we'll actually sort of-- for folks who haven't necessarily formally learned brainstorming we're just going to do a quick refresher so that everyone sort of is on the same page.
You learn how to do brainstorming.
You come up with a basic idea of a game, and you start to identify things that could be your core dynamics.
You think of what sort of core mechanics could address that.
Knock something up really, really quickly with the crappiest materials you can get your hands on and start playing.
Since nobody on your team knows, really, what game you're making at that point of time, you can probably test it with your team.
As soon as you do it a few times, your team becomes useless as testers because you already know too much about the game and what the game could have been.
And you start playing it in ways that a fresh person, who's never seen your game before, wouldn't even consider.
So you actually want to keep looking for people who've never played your game before.
So the next step would be to broaden it to other teams in this class.
Because everyone's going to be broken up in more or less teams of three or four.
That's pretty much, I'm going to take this game that we designed for a team of three or four, and give it to that team of three or four.
And then you play their game, and give each other feedback.
And that lasts for about one more session, and after that, everyone in the class is useless now.
So you have to look for dormmates, friends, people on the street, just go right ahead and try to get as much feedback as you can.
There is some value in replaying, especially later on once your rules are pretty set in that people start developing strategies, still some of them emergent behaviors, things to understanding sort of higher level play of the game, and you do want to test that.
But don't let that prevent you from also testing with fresh players.
Because in the end, you have to get past that initial hurdle.
Anytime someone sees your game, they are going to have to learn your game from scratch at least once.
And if they don't even get that far, they're never playing your game a second or third time.
So don't optimize for the people who play your game for four weeks, because you really want the people who've only played it once to have a good time as well.
That's not necessarily the case if you're making a huge, well-funded game based on a franchise that's been around forever.
If you're doing Halo 3 or Modern Warfare 3 or something like that, then sure you can sort of cater to the fans who've already developed these strategies and skills.
But in our case, we're going after a new audience.
We're going after someone who's never played anything like what you've played with before.
Cool.
Any questions?
Feel like we've been soapboxing here for a while.
Is there anything?
I guess the thing I would add from my experience, especially with iterative design is, a big trick of it is learning to kind of distance yourself emotionally from what you're working on, because half the time you end up chucking things that you really liked.
You think, oh, this would be so great, but it's best to just remove it.
And it helps to think of it as kind of putting it on the shelf for later, and saying, OK, I'll deal with this later, or I can try this in another game.
And on a related note, I don't know about you, but I found that, when games I'm designing have a problem, it's almost always fixed by removing something as opposed to adding more stuff on it.
Unless you're going for something, I think, really complex off to start, I always prefer to try to take something out and simplify it.
Especially on short projects like these it almost always works better.
And then, in terms of testing, too, one thing just kind of a tip that I find works, is if you're designing a game that has a lot of different, like, passive victory and this is a little bit higher level, which you'll see when we get there, where if you want a game that has different viable strategies, a really good testing method is to have different people kind of play like a perfect version of that strategy.
And you say, I'm going to do this strategy, or I'm always going to do X, and it doesn't matter if I think it's dumb, I'm just going to do it.
Because that will really help you find rules that are appropriate, or find something that's overpowered
A good example would be Settlers right.
Like if you say, obviously this wouldn't work because, the game's a good game, but like, I'm always going to build roads.
You know what I mean?
In practice, that would be stupid,but when you're designing a game, you do that kind of strategy, and you just limit yourself to one action, you can really find things that are broken really quickly
Thank you.
Much appreciated.
Cookies.
Good, we have cookies.
Let's see.
So I think after this, we're just going to play games.
We do have pre-testing as a topic coming up later in the semester, where I will actually sort of talk about the discipline on how do you do testing.
Honestly, you shouldn't wait for that class to necessarily start practicing.
It could be as simple as just grabbing a unit that you have never seen before and just trying to get a whole bunch of people to play it.
And critiquing it at the same time.
It's like, why did I do that?
Why did you do that?
Why did you do that move?
And there are sort of practices to make sure that when it's your game, you are not sort of telling the players what they should be doing while they are trying to play the game, because that affects their play of the game in a way that biases it.
Well, we'll get to that later in the semester.
For now just be aware, the more that you're testing, the more you're sort of tweaking your rules in between tests and then getting another chance to test and test and test, the closer you are to getting a game that is actually fun and engaging to play.
And that's what we're looking for.
That's the process that we expect you folks to suceed in.
When do we ever get to fun and engaging?
Who knows?
We don't know whether you're going to get there, so we're not grading you based on that.
Just remember that.
We're not grading you on how engaging your games are.
But how well do you stick to the process?
Yes.
Absolutely.
There was a question in the forums about when do you stop?
I didn't get a chance to look too closely at it.
What--can you repeat it?
It's essentially, it's the question of, say, there is this iterative design process.
When do you know when to just sort of get out of that process?
But--
Due date
Due date.
OK. That's always a good reason.
I guess, if you didn't have a due date, like how would you know when to shift?
Well, let's say you have control over the due date.
I guess part of that is, you should always be updating, blah blah, so I guess the question is when do you know when you have something you can put out into the world?
So the cynical answer is that, yeah, is that you wait for -- you have to ship, right, or you're not getting any cash.
I personally think it would be sooner than you think.
As, like, my game's almost ready but I know there's something wrong with it, especially when you're a student.
Right now, I'm in this mode with my StarCraft games, which is, this path is cracked.
I know there's something wrong with it.
I have no idea what it is.
Therefore, I'm going to release it.
Because until I do that, I have no idea what's wrong with it.
I need to get that information.
The nice thing about stuff like computer games is that you can ship updates.
It's annoying but you can.
But even board games and card games now have rule revisions, rule clarifications that you can release over the net.
For the most part, if you're going to go professional, your publisher will tell you.
And then the answer really is due date.
It's going to the presses on this date.
And that's one maxim, which is, game projects are never finished.
They're just abandoned.
It's just horrible.
I can't do anything else about this because I'm out of time.
Ship it.
But one thing that you can do to prepare for that-- Let me see.
Is there a laser around here?
There is a guy who runs a blog called Lost Garden.
His name is Dan Cook.
He works over in Microsoft.
And he talks a lot about game design.
Specifically about video game design, but actually his stuff is actually really, really impeccable.
And he has this concept that game design-- or iterative game design kind of goes in this kind of cycle, where this list down here is the number of features you've got in the game.
The trick, though, is that what you're really doing is, you are taking-- let me see if this is right.
So this is all the good stuff.
This is all the stuff that's not so good.
And the whole idea of sort of like using iterative design is to allow yourself to the space to come up with a whole bunch of ideas, not knowing whether they're good or bad.
But it also gives you a gate to cut out all the bad stuff.
So at frequent periodic intervals, you have a collection of rules, of cards, of designs, of whatever consists your game, that, is pretty much as good as it possibly could have been given the amount of time that you got.
And that could ship.
So if ship date was here, you had as good a game as you possibly could do it.
If ship date turned out to be there, all of a sudden you've still had the opportunity to put out all of the stuff that you knew was bad.
So iterative design is not about, let's just keep accreting more and more and more and more features.
It's more about, we're going to add some features and then we're going to remove the ones that don't work.
And that's really, really important.
So at any given time, you are prepared to sort of release it.
As for when you actually decide that this is now ready to go out, again I will repeat, sooner day than you think because you need that feedback.
Sometimes you don't know how a market is going to respond until you actually do it, but you can do a soft launch.
If you are doing a commercial product, you can do things where, we're going to release it to a few select stores and get that feedback before we go into major print run.
Or for 10,000 different copies of the same board game.
We're going to 200 copies and see how that does for us.
Gives you the opportunity to make a revision, but it's still an official release, because those 200 people think that this is the final packaged product.
That's pretty much how a lot of manufacturing industries actually work, anyway, so you think about any kind of product design, a lot of them have a sort of a trial period
Soft drinks you get a lot
Soft drinks you get a lot.
Japanese electronics, and other things.
You release it in Tokyo, and then you release it in the rest of Japan.
And sometimes it never makes it out of Japan because that initial test-- yeah, it was a good product but not enough people bought it because of these problems.
So instead of releasing it into the world, we're going to make version two, and maybe that will be one we release to the world.
That's one example.
All right, so we have games.
Who has not played Uno?
OK, we have more games that-- [INTERPOSING VOICES] [LAUGHTER] OK.
So Jason, you're going to do a Euchre table?
So--
It sounds like 4 player trick taking game, kind of like bane or current speeds or which?
Looks like we have exactly four people-- one, two, three, four, one, two, three-- it's an average of three people per table.
I think most of these games are three to five player, although I'm going to be absolutely sure.
[INTERPOSING VOICES]
--so it might [INTERPOSING VOICES]
You guys might actually want to try the round table in the corner?
Yeah, it's a [UNINTELLIGIBLE]
I really want to play bowling.
[INTERPOSING VOICES]
So basically, take a game from the table.
I'll bring some out and I'll see --play them.
Once you've had a chance to play a few times, feel free to switch.
And try to get as many games played as possible during the next couple of hours.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Welcome back.
Thanks for not dropping the class yet
I'm going to do that right now
Funny you mention that.
Let's see, just a reminder.
Actually one big reminder for folks who weren't here on Friday is that we swapped around Friday and Monday's--
Yeah, I think 17th and the 20th have been swapped
Yeah
In terms of reading that we're going to do in the class
So, we're still going to be playing the games on Friday.
But the brainstorming activity that we had scheduled for Monday is going to be on this Friday instead.
And then later on in the course.
Anyone manage to get through the reading?
Let's see, where should we start.
So there was a Doug Church article that was on Gamasutra.
And if you're familiar with Gamasutra, or if you're not actually, it's a fairly well-read news and industry news blog for the video game industry.
That particular article, I think, was originally written not for the website but for the magazine that the associated company publishes, the Game Developer magazine.
And Doug Church is actually an MIT alum.
The other article that was read, Mark [INAUDIBLE], they're both MIT alums.
So that's one thing.
So Doug Church proposes a set of tools.
And remember what they're called?
[INTERPOSING VOICES]
So, and Mark's framework which is--
M-A-D?
And we covered mechanics and dynamics on Friday.
And today we have a little bit of time to talk about aesthetics.
But the reason why these two essays were put right next to each other was MDA is an example of what Doug Church was describing as [INAUDIBLE].
In fact, at the time when MDA came out it was pretty much the only one that caught on.
Everybody, all of the problems that Doug Church lays out in his article about the problems that we don't have a vocabulary, a consistent vocabulary, to talk about our craft and thus we can't actually deal with it.
And thus we need some formal tools that everyone uses.
Unfortunately not much has caught on.
MDA being one of the few that has.
For a number of different reasons.
First of all, it actually is a useful framework.
You can think of many, many, ways to properly look at games but for a designer you need to think of something where you can actually apply it to help improve your games.
And MDA actually gives you a reason why design is hard.
So if nothing else, designers like it a lot because it says this is why their job is hard.
The reason being, you get to control the mechanics.
But that's not what you're going for.
You're not going for really interesting mechanics.
They're going for really interesting aesthetics.
They're going for a fun, engaging, play experience.
There are mock-- in the MDA paper, this lists eight kinds of fun.
And I've asked him about this when we give this presentation here in MIT.
Those eight were not meant to be comprehensive.
They were meant to be, these are the eight that I could think of at the time.
After this I'm going for [INAUDIBLE] that's a bouquet of different kinds of fun.
Different kinds of experiences that people are trying to get out of playing games.
That's what a game designer is trying to get.
The only problem is that a game designer doesn't actually control that experience.
The game designer only gets to upload mechanics.
Depending on what company you are, maybe you have the title of game designer but you're also in charge of the art.
You're actually also the lead programmer.
It's possible that you also have more control over things than just the mechanics.
But most designers, mechanics is what the job still is.
So they write the rules, the rules interact in some sort of like ridiculously hard to predict way, and then you get the experience of playing the game, the aesthetic.
And you hope that's what you were going for.
It usually isn't.
And then you make a change to the mechanics and try again.
The nice thing about that, is that works really well with the rate of design.
In fact, without a rate of design it becomes really hard to control this kind of process.
Because basically, we are two orders removed from the experience that you're trying to create.
It's like trying to draw on a piece of paper where not only can you not see the paper that you're drawing on, you're also controlling the brush or the pencil tied to a stick, or something like that, blindfolded.
That's kind of what making game designs are like.
So that's the experience of the game design side.
You can control the mechanics, and they're going towards some sort of aesthetic.
And from the other side, the player is experiencing some kind of aesthetic and they can sort of work their way back.
It's like, I'm getting this kind of experience.
Why am I getting this kind of experience?
If you've got a particularly insightful game player, maybe they're taking CMS.300 Introduction to Video Games, and you're analyzing games.
And you're trying to say, how did we get here?
If anything in this class, you're going to be doing that a lot.
This game, Heroes Might and Magic 3 for instance, has-- give me a particularly-- a sensation.
Describe a sensation that you got when you were playing that game, that you recall
Strategic.
Out-smarting.
To saying to friends.
Outsmart them.
Stump me
Seem smart, make you have to think really hard and everything.
So then you think about what dynamics might have resulted in that
Fog of War
For you to figure out is behind the areas you can't see.
But it has some information that you can see that will give you clues.
And then, Fog of War might already be a mechanic.
The idea of, there is information that you can deduce even though you can't actually see it.
And there's some probability of getting it right based on their experience with games.
Fog of War being one game that gets you there.
So you can analyze it backwards.
And figure out, OK that's one thing Fog of War gets you.
Sometimes Fog of War just confuses you.
There could be anything behind this corner.
I mean technically, Final Fantasy random combat system is like Fog of War.
I don't know if there is a random encounter here, I will walk into it
[INAUDIBLE]
That's where Unix merger is complete-- Plan to the classic Final Fantasy one with slides and the classic [INAUDIBLE]
But there is actually a problem-- actually, I don't need to write this down.
There is a problem with-- well, maybe it's not a problem.
There is an issue that I have with [INAUDIBLE].
Doug Church uses it specifically to say, well it should be sort of a serious business right?
It's formal with a capital F. We're not talking about how cool this game is, we want to be very precise in our vocabulary.
We want to treat it like a serious business with a capital S, capital B.
However, the other assumption with the word formal comes from [INAUDIBLE].
You are thinking about the form of the game.
In a way that if you were trying to apply the same technique to talk about art you'd be talking about the form of visual art.
The form of cinema.
And that is actually a very narrow way to look at something, at a topic, that is as broad as a game.
It is as if we were trying to analyze something like [INAUDIBLE] and completely ignoring the fact that you were playing it with friends from French class.
That would be an example of, well that had nothing to do with the game.
Well actually, it probably had a lot to do with how you experienced the game.
Play, that's the rules of chess.
And you could just look at the game based on the rules of chess.
You could also look at the history of chess.
You could also look at who you're playing it against.
Are you playing against a random person or against your dad?
Big difference, or in my case my mom.
Are you playing it in a club?
Are you playing it against the chess master in Harvard Square or something like that?
What does this game mean?
Are you using the game metaphorically in speech?
My opponent wasn't checking his position.
Doesn't necessarily mean that you were playing chess with someone else in order to get your point across.
There's a whole range of different ways that you can look at games, even as a game designer, that isn't specifically formalist.
That isn't specifically just looking at the form of the game.
Just understanding what context people are going to be playing the game in.
While a lot of the games that you guys are going to be designing this semester, specifically in this class, you have to think about the fact that a lot of these games are actually going to be played by your instructors.
Are going to be played in a setting very similar to this by your colleagues.
Are probably going to end up being tested by dorm mates, fraternity friends, folks around campus.
And all that is going to vividly influence the kind of game that you make.
For better or worse, you are pretty much making a game for an MIT audience.
I am trying to change that a little bit for the last assignment.
But keep in mind that's not necessarily every single game that you're going to make in the future.
And that shouldn't be.
But, knowing that, you can try to cater to the audience that you've got.
That has very little, in some ways, this is going to effect the design decisions that you make in your game.
What will be perfectly acceptable to a competitive MIT audience might not be perfectly acceptable to your average working buyer in Target or something like that.
And so what may be a perfectly engrossing game, mechanic, dynamic, aesthetic experience, that you will get.
But just like looking at the formal aspects of one game, this will be really cool if you understood the mechanic, played it out in the way the creator designed the dynamic to give you the aesthetic experience.
But if you don't have the standard context when you're playing the game, as an example, then your player may not even get that far.
Your player may look at your game and say this game is not for me and just walk away.
It might mean that you might-- I'm thinking of things like we just had this good discussion over lunch about the sort of games that you find on Facebook for instance.
And a lot of social games-- I might have already made this point, but it has been argued that social games aren't all that social.
All you do is really beg and scam your friends.
That's the whole point-- In many ways, it's also the same to your friends who are doing the same thing.
And we may be separated by time, and separated by distance, we're all doing the same thing.
And we could be playing this game, we could be talking about a show that we are watching in our apartments and get to discuss it, but in many ways this is just me telling you we're doing the same thing, that we're part of the same group.
And that serves a function that has almost, that really doesn't have anything to do with the formal analysis of how a game mechanic works, but it's still incredibly crucial to the popularity of those games.
So keep that in mind, that even though Church makes his point-- I think a lot of designers actually ignore that when you read his essay-- that it's only one set of tools.
The formal tools are only one set of tools, even.
That our tools at looking at games culturally, personally, and other things that video games cast into light-- that will effect how people experience a game and how much pleasure they're going to get out of a game.
To understand video games doing what it does right?
That aren't, that don't have anything to do, necessarily, with just by looking at the self-contained system of the game.
Just as a side note, how many of you folks have heard of the narrative versus ludology debate?
How many of you are sick of this?
I'm sick of it too.
But I bring it up for one reason, there is an interesting case that if you actually look at narratology as a study and ludology as a study, they're both formalist studies.
In fact, they're more accurate more precisely, they're both structuralist.
If you look at philosophies as a sub-sect of formalism, or at least a more recent version of it, they're both basically focused on these are the elements of this art form.
And this is how they interact to produce some sort of effect.
And all that matters is what's contained in this art form.
What's outside this art form doesn't matter.
That's what narratology does for stories.
That's what ludology does for games.
That's, in fact, probably why most people who study games have walked away from ludology because it's not that interesting.
You just study what's inside a system.
People want to see what's outside.
And so when you compare structuralistic and structuralism wins that's one case.
And you only have, you may not realize that you're actually looking at your game through a very, very, narrow lens.
So I'm trying to broaden the range of tools that you've got.
This class is going to give you a small set of them.
And it's not by any means the be all and end all of every single way and that you can look at a game.
And again, you're not just restricted to the tools that I specifically teach you in this class.
So keep looking out for all the new ones.
Let's see.
Any questions so far?
That was my rant.
OK, no more ranting.
We have an activity.
Do you want to run through?
OK I'm going to pass around-- this is my crappy race game.
So you're going to be getting one copy per pair.
There's also some nifty set of dice.
You'll need one copy per, one die per.
And then, each person's going to need three of these poker chips.
It doesn't matter the color so just grab three.
You're each going to need a marker to indicate yourself.
And I recommend just grabbing any given value-- There should be more than enough copies.
So you only need one board per group
So one die for every two people?
Yeah, one of those six-sided die for every two.
And then you need markers for yourselves and then bonus markers.
And that's what the poker chips are for.
So keep the poker chips circulating
Three per person?
Three per person
One playing piece per person
You can use coins, you can use die-- [INTERPOSING VOICES] paper, whatever you want
So we have three of the important questions [INAUDIBLE]
How many [INAUDIBLE] take back?
If you need markers for yourself, you can try one of the multi-sided polyhedral dice over here.
I guess-- [INTERPOSING VOICES]
OK, OK, what now?
I'm more well-versed in the--
The die is to roll
OK, so can everyone hold on for a second?
So what we're going to ask you to do is just play through this game a couple times?
Then we're going to come back and apply the MDA framework to the game itself.
And then if we have time, I think we're going to spend some time tweaking it?
OK, cool.
So we'll interrupt you guys-- actually, ten minutes probably.
It's a pretty quick game.
This has sort of gone flat.
So what we'd kind of like to do is hear about-- I guess we'll just go around the room in pairs and ask you to talk.
Or do you want a minute to talk, to apply MDA to this and focus on some things?
So I'm not like calling anyone on the spot
Anybody want to volunteer what, how they felt?
What was the aesthetic of this game?
There is none
There's no emotional affect
It felt competitive
The thing is, it's a very simple aesthetic.
So in the paper they say you're going to the pleasure of using the aesthetic, but here you go straight to the mechanic and you're wondering about it
You, it definitely is easy especially because especially for board games the mechanics are written down right in front of you, it's really easy when you look at that.
But how did you feel while you were playing this game?
There was a sense of tension.
There was maybe a sense of [INAUDIBLE] perhaps?
There was a sense of-- I mean there was the intellectual challenge of working out is there really a meaningful decision to be made here?
And if there is, what is the correct decision?
So thinking can I optimize this?
Especially with this crowd, I think there was an immediate try to figure out what the right thing is.
And I know you were trying to figure it out
I think he correctly figured it out
So is there an optimal way to play this game?
Yeah, OK, so if you're losing-- If you keep putting tokens down you're going to lose anyway.
So you want to be rolling in order to increase the variance.
And if you're winning, you want to be putting tokens down in order to move the variance.
You can imagine if it's like a distribution.
If you're below the winning line you want to widen it out as much as possible so as much of that distribution is above the winning line.
And if you're above the winning line you want it to be as narrow as possible so you stay above the winning line
So your strategy is basically based on what's the currency of the game
Yeah, until the last turn where obviously you don't want to put a token down if you're two from the end.
As long as no one can win on this turn, if I'm behind him I'm rolling the die.
And if I'm ahead of him I'm putting down a token
Did anyone prefer a strategy before you started thinking about what was the best one?
I'm too lazy to use the tokens so I just keep rolling
And that's also part of the aesthetic of it right?
You just happen to prefer the die roll
On the first three spaces you should be getting the average if you roll [INAUDIBLE] if you put a token down.
I personally like being conservative
You win no matter what
I'm sorry what?
That's more exciting because it does different things
The token gives you the illusion that you're actually playing
Did anyone like get in the situation where they are really far behind in a different color?
No
The actual difference-- one of the things that you do get in this game is the perceived distance between players.
It's actually not that far.
If the one behind actually has to still get in.
I think people refer to this as the American Gladiators effect.
You know how, American Gladiators-- actually, is this your example that I'm stealing?
I don't remember, honestly
Someone I can't help from stealing
I think it's in the book at some point, but I think we had applied it to this
Plagiarism!
Yes, yes, well I'm crediting someone for this question.
Citation needed.
The basic idea being-- who's seen American Gladiators?
They might be too old
No, no, we had American Gladiators
I've never-- I don't know what it is
OK, there are also a number of Japanese game shows which are similar to this.
The basic idea of American Gladiators is that it's basically an obstacle course where the obstacles are people.
Happen to be these athletic dudes and women who are either pummeling you with waffle sticks or shooting tennis balls at you out of a gun and stuff like that.
And you basically have to get from the beginning to the end faster than the other team.
Actually, faster than the other player.
So it's two regular dudes and a whole bunch of athletes who are throwing things at you.
Every time you get hit you have to go back a little bit.
But the thing about American Gladiators is that you have this long obstacle course.
In fact, this was one of the events.
This was always the final event of the entire TV show.
Is that, you will have two people doing the same course.
They will have to climb up a wall and then they will have to go down a zip line
But it was a cargo net specifically.
So it was really difficult to climb up it
Yeah, the big webbing kind of thing.
It was like climbing the rigging off a ship.
It was really, really, difficult.
So what happens is that if you have two people who are really, really, far apart-- one person is way ahead-- and that person starts climbing the cargo net?
It takes that person forever to get to the end of the cargo net, because climbing up cargo net is really, really, hard no matter who you are.
That gives the person behind plenty of time to catch up and start climbing the cargo net.
But then, the person who was the leader in the first place hits the zip line.
And all of a sudden there's a huge increase in distance.
So this whole idea that they never really were all that far-- they never really changed their position.
For most Gladiators games the winner has already been decided by their points.
Because the first person who reaches the cargo net tends to win the match.
But it looks like the lead just shrank because suddenly you've got both of them in one camera shot.
And then you've got that cute-- mom likes to call that dramatic tension.
Which I think is a better description than it's other term for the same thing which is drama.
Because it's not drama.
It' s just tension really.
But that's an aesthetic.
You're going for the aesthetic of tension and coming up with a game that makes you feel like the fight is closer than it actually is.
Or allowing someone to close the gap quickly because, many of you have pointed about, these things probabilistically mean exactly the same thing as a die roll.
Let's see.
But the ability to increase a awkward gap to open up and then to close, that's actually a dynamic.
That's actually a result of a number of different rules that are in the particular of the game.
It's not written down that it's going to happen.
But it's pretty much going to happen if you have two people playing those particular strategies, and one thing that happens pretty often.
What are the constraints that you're seeing in the design of this game here?
What did this game have to do in order for it to be a viable in-class exercise?
Fit on a sheet of paper
Fit on a sheet of paper.
That's a huge one actually.
Because first of all, it limits how many spaces you've got.
Assuming everyone's using something like a die or a coin or something
I mean, you can laugh at that but it's literally how it went.
I wanted it to fit on a sheet of paper and the spaces had to accommodate a die.
So the number of spaces is a result of that
It was almost like an accident
Well you probably could have fit a few more squares in there
Probably, but you place some margins
Yeah
The rules have to fit on one sheet of paper as well.
So the rules can't be that complicated
Most of these rules are single sentences for the most part, because we are more than [INAUDIBLE].
It had to be quick.
It had to work within 10 minutes.
In fact, it had to work so you get through a few games within 10 minutes.
Anyone only manage to get through one?
That'd be really hard
So different aesthetics.
So, this game tries to get really, really, tense.
But obviously, the decisions that you're actually making in this game is minimal.
You get to say, well if you are already losing, that has [INAUDIBLE].
One rational thing to do.
But if you were playing this game, as Jason recommended, if I were playing this game against myself there really is no optimal strategy.
I could pick two different strategies for each side
It's whoever plays first basically.
Whoever plays first is by default going to win
On average
I won every game but one.
I did bad rolling dice
Because if you roll them now--
So I would like actually, folks to try mixing this up a little bit.
Pick a rule in here, any rule, and alter it.
You can do things like adding more squares.
You can change the movement chart.
One idea that I had, and someone was thinking probably more recent than me, which is what happens if you use the higher of two dice rolls instead of just one?
[INAUDIBLE] to the probability.
Play around with that.
See what that feels like.
And if that feels just boring try doing something else.
So we've got 15 minutes?
Yeah, that's reasonable
And then, you'll tell the class about what you changed and how that changed the feeling of the game.
Cool?
So, every two people, work with a team.
Explain how your game, how your version works, and have the other group play it and vice versa.
And what we're going to do once everyone's done, you will tell the class what you felt the other group's game felt like.
So let someone else figure out what your game aesthetic is.
So, does any team want to just go first in describing the other team's game?
OK, starting with you
Do you want us to describe the rules they changed?
Uh, sure
Or just say-- the main change in this version was that it's modular.
So you have to land exactly on the end or otherwise you wrap around.
It changed the aesthetic a lot because it stopped being a race against your opponent and became more of trying to figure out how you can use your tokens to precision just hit the end
Oh,
More methodical and thoughtful.
You decide whether or not you're going to stockpile tokens to give you more winning squares and then you just waited until you landed on a winning square
So you get a dis-computational motive to the game?
It was not as competitive
So one of the discussions we had was, what if you had three tokens you could just get to the end.
Because we made our tokens move four spaces

But I just realized, you have to do the die roll after that
Yeah
Oh, yeah
They had a really creative role where they had a 10 here.
So it goes from 0 to 9.
And you roll under a cup and that's the number of spaces away from the end.
So from 0 to 9 that the winning point is.
And at your turn you can either play normally or you can check to see where the winning point is.
If you've won, that triggers you winning.
But you cannot win unless you physically check and confirm that you've won.
So it eats up a turn, but it could be that after two rolls of two and this was a nine you've won.
And it was a lot of fun basically because we had to figure out the optimal strategy but it was just fun
So what's it feel like when you're trying to figure out what you're going to do?
He beat me by checking when it was far away and being lucky
So if you don't check you have to reach the end to win.
So you have to check, usually, anyway
One thing Alecia had in mind when we were thinking about the rule was there ought to be information that you can get from how your opponent reacts after they look under the cup.
So if they then go and start rolling again or what have you, or they then start options.
But there must be some way that you can get a leak of information from their behavior which makes it interesting in a different way
If you've have already seen the cup, and you get past the point, do you have to check to win?
Yeah, we played two different variants.
One where you did.
One where you didn't
Oh, I see
Well, you don't have to check the second time.
If you check the first time and then you pass it you win
So that's why there's one way of playing it
So the person who checks could win even they're already passed the point?
Am I understanding this right?
Well, if he's the first past the point.
If you don't have check to win, so if you just need the knowledge of what the endpoint is, then if he passes it first he wins.
If when he checks both players are at that spot then it's a tie
Oh, ok
But since it's a sequential game it's impossible.
Other than, at the moment of the check if both players are at the same spot is it possible for them to win at the same time.
So that's not really an issue
What if I check, and you're past me, and you have passed the check spot?
I don't win

Because I never checked.
So as long as I don't know I don't win
Hey, he makes me think of a different variant.
But
So what they did was they introduced a couple new rules.
The first rule being that, the person who rolls first on their first turn they cannot put down a token.
So they actually have to roll.
The second rule is, whenever you roll if you have a bonus token on the stockpile you have two options after you roll.
One is, you can take it off and move two spaces like normal.
The second is, you can choose to make that block impassible.
So what this does is your opponent cannot land on that block.
If they are already on that block they get pushed one block back.
If they are behind it, and they roll and they move to that block, they get pushed one block behind it
So they're changing the block while you're playing
Yes
And how did that feel while you were actually playing?
We didn't actually use it that much
I felt like the penalty for the impassable block should have been more.
Because if you land on the block that they're on, and make them move back one, that token would otherwise have made you move forward two.
So it doesn't really reward you for that.
I felt it could have been really awesome if the penalty was significantly more severe.
Because it's hard to get in front of them enough to actually make a block impassable and have them land on it later
Probably to all the way back or something
Yeah exactly
So try multiplying it by two and see what happens
Yeah, and then you can always go land on it
If you're like, hmm, that should have changed the game play but that didn't.
One of the tricks in chapter one was just make it bigger and see what happens.
If nothing else it will just give you more information on whether you should be spending any more time with this direction.
Or whether it really isn't going to affect the way how people play because people don't end up caring about it.
So the rule of twos is just multiply something by two, or half it by two, and see what happens.
Obviously halving by two would not in this work.
Who else?
So actually they had these tokens as power-ups.
And you would never actually take them off to move more spaces.
You would just keep adding tokens and you could roll.
So you can spend one turn to put a token on and in consequent rolls you could essentially say, I have N tokens therefore I roll N plus 1 dice and subtract
I think that's one dice.
[INTERPOSING VOICES]
--and then you roll a die for each token you have, and then you move back one space for each token you have
Interesting
So it moves your average and your maximum up each time
So you could act it as a power yourself up
And you could also remove tokens to take away tokens from the other guy, right?
Yeah
Which we never actually did
I guess we forgot to go into that, you don't take out all the tokens when you do your special move.
They stay in there.
Because, mathematically speaking if you take them off it makes it worse
So, I mean, that's an important object lesson in how to explain the rules to the people.
How'd it feel while you were playing their version?
It made my mind tired, being N plus 1 minus N. But, it was interesting.
I actually used it.
Before I said I would only use the die, but this time I actually used tokens
And what happened when you actually used the token?
Was it like yes!
Like I said, the math made it too hard
Well something happened.
It made you use the tokens as opposed to ignoring them
It was interesting
So, the rule that they gave was that if you are behind the player-- by behind, they mean if your piece is behind the other one
Losing, in second place
Right, but not in the imaginary sense.
Well, whatever.
If you're physically behind them then when you use a token you get to use one extra space ahead
OK, if you use two tokens do you end up getting?
It's still one space
So how did that feel?
It actually changed the strategy a little bit because in the beginning you basically don't want to be the person that moves first.
Because then your opponent can just use all their tokens and get a lot better than you
Oh, I see
You may want to talk about tokens and what it feels playing it ahead of you so that you can use them, you can-- [INTERPOSING VOICES]
Rather then you just sit at the start point every time
So pretty much the person who goes second?
Is at an advantage
So that should be a negative feedback loop, which we'll we get to I guess in the next class, but the whole idea being there is a gap between players.
And you have game mechanics specifically designed to narrow the gap.
Those are negative feedback loops
Mario Kart!
Mario Kart's blue shell, it's the other direction right?
To knock back the person who's leading rather than helping you catch up.
Actually, a lot of Mario Kart's mechanics are basically designed to help reduce the lead.
All right, cool
They add one rule in keeping everything else equal.
They add a second die and then the way it worked was you arbitrarily assigned people odd or even numbers.
And then you roll and would go on that particular turn.
And then that person, whoever it was assigned to, would play.
And then you would roll again to see who would went the next turn and so on and so forth?
How'd that feel?
I got a tad memory with Mario Kart
It's like what, again?
Many mechanics describe as very Mario Kart
Well, Mario Kart is actually a collection of a lot of fairly fundamental mechanics
I felt like just rolling to see who'd win.
The sense that you were just playing for luck
So it amplified the randomness basically?
Yeah, which I actually detest personally
That rule is like initiative rule in Star Wars Miniatures and Dungeons and Dragons.
And like in that case where there's actual strategic decisions to be made it's actually cool.
Because it means that you never know whether you're going to be able to capitalize on the position that you're in at the end of the turn
But since we both know optimal strategy for the original game it was just pure luck.
So there was no strategy
So it almost becomes like a gambling game or a game of chance.
Cool
They added different dice you could roll for different effects.
Such as, if you roll the D20 you can go 10 spaces.
If you need to go 17, 18--
Or 18, 19, 20
-- Or 18, 19, 20.
So they added the D10 which also had similar effects and deviate.
So Patrick and I pretty much experimented with-- well, he just rolled D20 the entire time.
And I experimented with rolling the D8 and the D10
Why would you not roll the D10?
I started feeling impatient.
I would forget everything about Alec and what he wanted to do.
And every turn I would roll the D20 hoping to get 18, 19, or 20
By the way, with the D20 if you don't roll any of those you go zero spaces.
So that's actually the risk
So you could either win the game or go nowhere
So I roll the D20 wanting to get 18,19, or 20.
And not until I roll the 18, 19, or 20 would I care where he was on the board
It's you and the dice.
It's just you and the randomness
I rolled twice too
Like what you described earlier is that whether you're behind or in front of someone changed the way that you would play in the original version.
Now, no longer really matters anymore, right?
It's just you and the dice.
And when the universe loves you, you just automatically win
The intention was to have the D20 for when you were way behind and the other one if you were only slightly behind.
Apparently that's not how it was used.
[INAUDIBLE] design, right?
Well, a good example of the aesthetic you were going for not being the aesthetic that was actually experienced.
So, it's still a powerful aesthetic it's just a different thing than what you planned.
I think I might have skipped you guys?
Sorry about that
So their new rule was, if you are physically ahead and you roll a six instead of only going three you would go four
If you're physically ahead and you roll a six you go?
Oh,
[INAUDIBLE]
At first, we tried playing it in a similar way of how we did before.
And it really felt like the person who was ahead was ahead even more.
And now everyone's rolling, because as soon as you roll you get a higher expectation value
So you prefer the low-ball luck rather than the?
So the problem is if you put a token down and they roll at least a one so you're guaranteed you're going to be the high.
And then they have a higher expectation
How does it feel to play?
Mathematically that's what it means, but-- [INTERPOSING VOICES]
Actually, in playing the first game I don't think it actually came up at all
And then in the second game, I believe it did?
And to me it didn't really feel like it changed
For me, the entire time I felt like the person who was ahead was more ahead
It would come up more if your board was longer.
It really encourages you to stockpile for the first few turns.
But if you're encouraged more than you might want to search.
That's contrary to--
When you guys were going over mistakes did you feel like it changed your strategy?
Yes
It made rolling feel like a [INAUDIBLE].
Which was the point right?
That's what you guys were aiming for?
The alternate rule that we had toyed with was, if you're ahead and you put down a token you have to roll.
And if you get a six you have to move one step back.
So there's a one in six chance that there'll be a penalty for [INAUDIBLE].
Because precisely what you mentioned, which is that when you're ahead the dominant strategy is to reduce the variance.
So we were just playing with stuff to make you want to roll more basically
OK, so dynamically that tells you to roll more.
I would say aesthetically you have the whole Monopoly situation where the person who's winning just keeps winning.
And there are times when you actually want that feeling.
Especially when it came to the game of Monopoly, the whole point of the game was huge financial embarrassments.
And it's like, wow I have no money and I'm just going to keep losing money.
That was the point of the game actually.
That's why it's called Monopoly
The thing about it is, the person who's physically ahead may not actually be ahead in terms of absolute game play because the other person has stockpiled tokens.
And we weren't sure whether determining who was ahead by counting the tokens in the equation or not was a better game rule if we didn't have that type of test
Give it a shot [INAUDIBLE].
The one thing we came up with was-- at first we thought it wouldn't change the game at all I guess.
Which was basically, instead of spending tokens to go forward two steps you spend tokens to knock your opponent back two steps.
Completely changed the game play
We tried that.
It sucks
Stock piled first and then you knock people back so you didn't go anywhere for the first 10 turns.
And then you realize that nothing is going to happen
And then we just kept rolling
Rolled the rest of the time
Which is a completely different way to play the game.
That whole process of stockpiling right at the beginning is-- sure, we could have both started with a stack of three each because that's really what happened three turns in.
But the whole act of-- that felt really like, OK clearly you're setting out to kill me
Can you move?
No, we had it so it wouldn't be a point spending it when you're right at the beginning.
But there was that whole arms race feel that really didn't happen the first game.
All it did really was make the game longer, but it felt mean.
OK, that's pretty much it for today I guess
And we need the poker chips and all the dice back but you can keep the game boards if you're really happy with them
How long did this game take you to make?
It took longer to make the board than to make the rules.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
One example will be what happens when the battery runs up on the computer and what bubbles up when it's running.
And there's certain computers that might [UNINTELLIGIBLE] hard drives.
Some computers just keep it in RAM when you restart it.
There's no big difference.
What happens when an IM window pops up in the middle of the game and leaves the control screen?
And all the keys start [UNINTELLIGIBLE] IM window.
Ideally, they give you pop ups that doesn't work.
But catching those is one of our favorite things that we do when we're making digital games here in the lab is like slamming all of our fingers on the keyboard at once and see what happens.
And we're kind of looking for those things.
We're also looking for subtler things like is the amount of RAM that those games are taking up in the computer slowly going up all the time?
And of course, comparing it against a spec grade.
So a character is supposed to be able to create a certain file hold style or something when you enter this key sequence, and 99 times out of 100 that works and that 1 out of 100 doesn't and we need to figure out why.
That's our feat.
So in a lot of those situations, what happens is that output comes back in the form of other parts and-- how many of you actually do programming?
Oh, OK. How many of you have found a book report?
OK, not that many.
There are a couple of best practices where you follow a book report and this one translates a little bit into when you're really working in a team in a game.
How do you communicate that something is wrong to someone else who is also responsible for the game, but may not necessarily have seen the problem?
First of all, you describe what you were expecting to happen and what you were doing.
I was expecting to go right when I push the right button.
And then you write down what happened.
You try to get as much detail as possible for the person to re-release this problem.
Now, when it comes to war games, in many situations, the kind of detail that you need to provide largely comes up against it.
So 'a' had this much money, 'b' had that much money, they try to do this game mechanic and no one knew what to do because the way the rules are written there's only one way to interpret it.
And that gives your team enough information to be able to start acting on it.
Because you are bargaining and your rules are kind of right out there in the open, there's usually less of a mystery to see where the problem occurs.
But you know there's problem and it could end and you largely see where it lies.
You may not know what to do to correct it, and that's actually where beta testing comes in.
So but I'll get to the testing shortly.
But technical testing, a lot of it again is comparing a given spec.
Sometimes that spec is created from your team.
This is what we want our game to do.
This is what we expected to happen when we made this game.
We expect that mid in the game, a certain game mechanic will start to take priority over other mechanics.
Early on, everyone's buying property is a monopoly.
Later on, everyone's going to be building houses and then even at the end, people start to mortgage stuff.
That's what we kind of expected out of it.
And then you play the game and you start realizing you mortgaging something way too early here, something's out of whack here.
So that's the kind of technical testing you do in games.
This is what we expect, why isn't it happening?
The other side over here is kind of like being the defender of the player, being the player's advocate.
How, can you be the person on the team that is actively looking for things in your game that are going to hurt your players and then mentioning it and saying, maybe we should do something about that.
It's as simple as, here's the information that we want our players to have at a certain time.
Here's the game state.
Your characters have a certain state.
They are this close to winning, but are this far away from winning to have these results here.
Do the players understand that?
Have you provided them with enough information to be able to make a good decision?
If they know the game state, do they know what options they have in front of them?
So as we play more and more board games this semester, you can actually look at the way how rules are presented redundantly.
Sometimes they're written up in the rule sheet, but and of course you've got your index card-sized thing which goes through step one, two, three, four, these are the four things you've got to remember every single turn.
You've got to get it, and everybody has a copy right in front of you because somebody on the team decided that that was a good idea.
How did they come to that conclusion?
Occasionally, they are doing it because they see there's a problem right away.
There's too much information, and you just don't expect any player is ever going to be able to grasp all this information.
So that's come out with a couple of ways to simplify it.
Sometimes that involves taking up the mechanics, making something that was written up a text graphic report and some icons, reducing all the numbers by two so the math becomes easier.
But sometimes, you do it by usability testing, and that's one kind of testing that often comes together with beta testing.
I've invited Sara Verrilli, our development director, to come in here and talk about the kind of pre-testing that you can do, which is illustrated by this picture.
And the book goes through several different kinds of environments where you do pre-testing that's sitting one-on-one with people and doing site testing for the group.
Because this style is all about board games and card games, for the most part, we're designing games for three or four people to play at a time.
The chances are, almost every single test that you're going to do [UNINTELLIGIBLE] at a single time.
That's just the environment you're in that's meant to be played.
That's why you end up testing it.
Obviously, a single-play computer game, that's not necessarily the environment you're testing in.
Actually, before I transition over, there is one thing I want to mention about technical testing.
Sometimes the specs you're testing against are not the ones that came up that would necessarily come up with phi over t. If you're doing a computer game-- say for an Xbox or a PS3 or a Wii.
There are these things called technical [?
evaluation ?]
requirements, which are basically huge lists of things that your game must do before Microsoft will sign your title or allow it to be released under that title.
PC obviously doesn't have this, but if you release it on speed or something, then chances are, you'll have requirements.
It could be something as simple as generally, it doesn't crash-- and games do crash, even on PS3 and Xbox-- you get things like what happens when you're playing a game on the PS3 or Xbox-- I guess more the PS2 and the first Xbox-- and someone yanks out the cable?
All you're thinking is that you're-- and then the keyboard comes out, what happens?
Does it continue?
It buzzes
It goes into pause, right?
And usually it tells you, your controller is disconnected.
And that's a requirement.
Someone had to write that code specifically to detect that situation and then throughout the suite across the game.
And the reason everybody had to do that was because Microsoft and Sony and Nintendo-- well, not so much Nintendo nowadays because Nintendo is wireless-- but these are the kinds of requirements that they expect.
If you lose your connection to Xbox live halfway through the game then something's going to happen.
And so again, for that kind of technical testing, you're sort of comparing it against spec, but spec may not necessarily be something that you're keeping up with.
It may be something that the publisher or your platform developer, platform manufacturer mandated.
How does it affect the board games is that sometimes, board games are actually going to require half the requirements to that.
If you're responsible for the design of the box, for instance, there are things they're going to require on the outside of the box, like what ages are the game applicable for, how long is it meant to be played, how many people are going to be playing your game.
I think Days of Wonder, which is a publisher, actually requires a character to be on the box.
Even if your game has no characters, there must be a human-like presence on the box or your players are on the box because they think that it increases sales, and they probably have the numbers to back it up.
That's why if you look at things like Ticket for Ride, for instance, has people.
If you play Ticket to Ride, it has no people in it, but it has people on the box because they want people to think that's what users are.
So that's just an example of things that a board game publisher may require you to do.
Very common that you'll have to do translations in a range of different translations for the different countries it's going to be released in.
It may not be your job to do it, the publishers may end up being the ones helping you with the translation, getting all the publishing data formatted decently, but someone along the stage is making sure that happens before the game hits the shelf.
OK, that's technical testing.
That's a little bit about becoming a player and so let's talk about data center a bit
Let me bump you a little here, if you don't care
Oh, right
OK. Let's see.
Let's actually go ahead.
So focus testing.
This is a very similar track for the summer program folks.
I have about 10 minutes to go back and relook at my notes and relook at the lecture, so if it's a little stumbly in parts, it's because I haven't given it for a while and haven't thought about it.
It's much more geared, the original talk is much more geared towards computer testing, and testing specifically single-player computer games than it's testing board games, which are almost a multi-user.
So first of all, focus testing is actually dissimilar.
Focus testing is only a specific case of user testing, which is to say, you're testing your game using actual users, not your development team, not people who play board games 12 hours a day because that's what they love to do all the time, but more the sort of average person who might want to pick up your game off the shelf, go buy it and then go play it with their family and then go out to dinner at Chuck E. Cheese or something.
Another type of user tester is usability testing, which is fairly similar to focus testing in as much as you're still using users for it.
Focus testing is usually much more geared towards are people enjoying your game?
Do people want to play your game?
Are people going through the sort of play experiences you want them to have as they play their game?
Because usually when you create a game, you're thinking, oh, like for Settlers of Catan, you expect people to spend a lot of time talking with each other trading things back and forth.
When you're making Diplomacy, you expect people to spend a lot of time arguing with each other over well, if you invade that country, I'll invade that country and we'll gang up on him and we'll all come out of the game and then we'll just divide four between the two of us and win while you're making secret underhanded arrangements with the other guys, but OK, so he's going to attack you on this ground, we both know that, that sort of thing, as opposed to, say, Transamerica which is just more putting pieces down and it's pretty much independent play in many ways.
Usability testing is much more about how you easy is it for people to play your game.
Do they understand your rules?
Are the mechanics clear?
When they sit down and start sorting out the tokens, can they tell which tokens are the player pieces and which cards are which and how you use those?
Do the icons on your game make sense to people?
Do they understand them?
Things like that.
It's not really about whether or not people like your game or are enjoying your game.
It's about can people understand your game and start playing it.
So here we go.
One big problem we run into when people sit down and we have people playing your game and giving lots of comments, this was really fun or this really sucked, or I hate this part where I have to spend time talking to the other players, I want to be able to stare at just my cards and board completely without interacting with anyone else.
Focus testers will give you lots and lots of feedback, some of it is useful, some of it is not.
And you need to remember that focus testers are not actually people building the game.
Focus testers are the people who you are trying to get information about your game from and one problem that we often see in experienced game developers and people making games doing is, oh, the focus tester said we have to make the whole game red, quick, let's make everything red.
So and that's usually, that's not a very good solution, right?
Who knows what the problem with this actually was, but what you need to do is find out why did the players want to make everything red?
Is it because they don't like the artistic design of the game?
Is it because someone in the game is colorblind?
Someone playing the game currently is colorblind, is having a hard time telling things apart so they'd like everything to be the same color so everyone's having the same problem they are.
I mean, who knows?
It's hard to tell what people are thinking.
When they give you advice, and when they give you these comments, you have to have some way to filter those comments, some way to think about them and some way to decide what actually is the right reaction to take in view of what you want your game to do and how you want people to play the game.
Since you're the game designers and developers, you have to make those decisions.
Focus testers can only give you data.
You shouldn't ask them to design your game for you.
Maybe you actually have good ideas that you want to use, but you want to make sure that they're good ideas that you do want to use.
So you may notice, these are both my lectures, and these are pictures that Mike [?
Dropho ?]
drew for a poster, and we love him so much.
So in the ideal focus testing situation, you're really trying to get as much data as you can about your player playing the game without spoiling the information.
What you'd really like to recreate in a lot of ways is, so you just bought the game and have gotten home, what happens?
And that's what this picture is all about, there's somebody luring somebody to play the game without any information.
So things to think about when you're planning your focus testing is, what are you testing?
What questions do you want to ask?
What information do you want to get out of it?
How are you going to test it?
How many players do you need?
What kind of interactions do you want to see?
On the backhand, who are you testing with?
What kind of people are going to make your game?
Are you making a game for children age 9 to 15?
Are you making it for college-age students?
Are you making a game that you think that older players will want to play?
Because if you're making a game that you think is going to be popular with the senior citizens market, testing it with a bunch of elementary school kids probably isn't going to go very will.
Trivial Pursuit Classic Edition, a game that came out about 20-odd years ago, I suspect that if you sat down a group of high school students with it and had them play it, about three quarters of the questions they'd just stare at you and be like, I've never heard of that.
So, that's certainly where the audience comes in.
So, the what, planning out the questions that you think you're going to get information for.
When you know what information for focus testing, it's a lot easier to know what to keep track of, how to take notes, what reactions you're most interested in as people are playing the game.
So, what questions are you going to ask?
What data it takes to answer your question?
And how are you going to get that data?
Are you going to go all through observation taking notes?
Are you going to just step in and out of questions?
So why did you move that character from there, just out of curiosity I'm wondering why you made that move.
Will you tell me about it?
Or, you can ask them to fill out a questionnaire at the end of it.
Depending on what data you want, different methods will work best.
In general, usually the best way to do it is observation.
As soon as you ask someone questions, you sort of perturb the system and when you ask them to fill out a survey, it's after they've done it.
They've kind of forgotten what they were doing.
They may not exactly remember, and to be perfectly honest, people would really rather play your game than fill out a form.
So they'll be more engaged more interested and more interactive when they're playing a game and that's when you're going to get your best data.
But there are times when you want to ask someone something sort of analytical and you don't want someone to be thinking analytically while they're playing the game in a lot of ways , and so there are times when that reflection comes out so much better and it's better to have that written down and get it in your own words
You would hope it plays better than a survey
If your game is better than a survey, you probably already observed a problem right there and you're probably glad you found it.
So how?
In order to get all that data, you're going to have to actually interact with the user, someone who's never seen your game before, someone who doesn't actually care whether or not you get good information out of playing this game.
They came here to sit down and play the game, and while they would like to help you by giving you information, that's not their primary goal in a lot of ways.
Usually, their primary goal is to sit down, and play the game, and have a good time.
So you need to be able to structure the interaction with them so that getting the data to you is as easy for them as possible.
So we need to think about what this player needs before he or she sits down to play.
Usually, in a board game or a card game, that's the set of rules.
In a computer, it's always trickier.
Usually when we're bringing in people to test the first prototypes, we have a game that works, but we don't have any instructions in game.
If we're going to have a tutorial or anything like that, that's usually one of the last things that goes in.
So when you're testing computer games early in the process, usually you end up sitting down and writing out, well, this is what the tutorial would look like if we had it, so read this and then sit down and play it.
With board games and card games, it's actually a little bit easier, because you can say, a ha, this is the rules that are going to package with the games.
Give you the rules, step back and see what happens.
So, it's actually an easier aspect for testing board games.
And finally, treat the players politely.
When they find a misspelling in your rules or when they discover that you've left out six of the key cards to play the game, oh, yeah, you're right, thank you for pointing that out, I wouldn't have noticed that if you hadn't shown it to me.
Whether you would have or not doesn't really matter.
They're doing you a favor by playing your game that isn't quite working well, so you want them to know that their contribution is invaluable and you appreciate the time and effort they've put into it.
A few things, the more they talk about your game and the mechanics they find that break or that do or don't work, that's all data right there, That's all great.
Another great advantage of testing board games is you usually get groups of players, like two or three or four players, and when you have multiple players, they'll talk to each other about it.
Oh, I noticed this.
Did you notice this strategy?
Yeah, I did, but I noticed that if you add that strategy that you can do this thing and this other thing and that's all really neat.
When you're testing single-player computer games like we usually do in the summer program, you've got a single user that is sitting there between them and the computer.
You have to watch and figure out what they're thinking.
When you have a group, it's not usually that hard to get them talking about what they're doing and what they're thinking, and so that's a really good way to try and get more data.
So yeah, here's sort of the end summary.
Have a specific question and questions in mind.
Know what you're doing in focus testing, know why you're doing usability testing.
If you don't know what you're trying to test for, then your data is going to be a lot harder to figure out what to do with.
So you always want to we have that question in mind.
That's actually a really big one.
What data you need to answer that question so you can collect it and know how to collect it, and make sure that you can give the information to the players, a standardized set of information to the players and step away so you're not sort of coloring the information and giving them the things you're talking about by, oh I forgot to mention, oh and did I say, so you've actually got sort of a standardized standard size play group scenario set down.
OK, I think that's actually what [?
I came for.
?]
So a couple of words about when to be testing, which is really as soon as possible.
Usually, before you think it's possible, you should be already starting to plan your fist test because at that point, you're probably going for the lowest-hanging fruit, which will be probably somebody else in this class because they're going to give you feedback anyways, you can use that favor in exchange.
The whole point of making testing as simple as possible is you want to be able to get information while you still have time to fix it, to do something with that data.
What has happened a lot of times in this class is we have a project team where they come up with a game requires say, generating so many cards or that they're going to spend most of the project deadline actually creating those cards.
They'll test it, but they'll test it on themselves because the team already gets it and they're not looking for other people.
They have a plan to ask their friends to come in for a specific time, they've set aside an hour or something like that.
So what happens is that the last week comes around and they start playing it with people who haven't seen the game before and then discovered problems.
Unfortunately, the game has 100 cards that all now need to be fixed.
This is a problem because you just discovered that problem too late.
A couple of ways to handle that, though, one, don't decide on a game where making changes across the entire game is going to take you more time than to test it because that's going to make it impossible to fix anything.
But the other one is just testing early.
As soon as you've got a couple of key mechanics down, sure you are planning to play this game with 100 cards, start with 20.
Start with a small of group of cards, hence your core can see people and understand that they're going to discover things in the production of those 20 cards and in the testing of those 20 cards that maybe you don't want to be doing for the next test and you can redo those 20 cards pretty quickly the way that you can't redo 100 cards.
Just an example.
I'm really not encouraging people to do a 100-card game, by the way.
A regular poker deck, 52 cards, that's all you need for a really interesting game.
Let's see.
Some other things.
Say you're doing a test, but it's not with your entire team present.
Say it's just you and your teammates are all living in different dorms and you decided to play with those in your dorm.
That's great, because it's a cool game and you've got your feedback, how do you communicate that back to your team?
I talked a little bit about writing up a little report, but here's a couple of other tips because one thing you have to remember is that chances are, every single idea in the game, somebody in your team is probably really passionate about it and what you're basically trying to do is try to convince someone in your team that maybe this idea should be changed.
And your game will probably be better for it, but there are some tactful was of telling that.
So let's see.
Keep it short, this also applies to book reports.
I talked about you giving people enough information to reproduce what the problem was so that they can understand the state of the game and why the problem is occurring.
And the flip side of that is not too much information, not more information than is necessary to keep it short.
Little [?
farm ?]
codes, give them something that's easy for people to scan.
Don't make them read and essay for them to figure out what comments will be solved.
Now actually, one thing to keep in mind and if you keep in mind and think about what Sara said earlier about the data that you decide to collect, one problem with board games is with all the interesting conversations happening around the table and multiple players suggesting strategies to each other, you can come up with reams of notes in a single play test session.
How do you decide to digest that is really important.
And you're the only person there and you're the only person on your team, you need to try to let the rest of your team know.
You need to put in the work of digesting all of that information before necessarily giving it over to your team
One possible solution also is you can get permission from play testers for a tape recorder.
Just get a digital recorder and record all of the conversation as it goes on
But you still want to pick out the keys, you don't want to revisit the entire conversation.
Because if the game took an hour, and every single one of your team members visits, by now you've lost three hours.
So you want to find that, do that little bit of work to save yourself the detail.
But that's the way because you can focus, you can take down notes on the main points and just let the tape recorder record all the details so that you don't have to worry about recording and really, the main points are the things that you want to make sure you get across to your team.
The tape recorder is just evidence.
You could use video depending on who you're recording with.
Sometimes people get kind of antsy when they know there's a camera.
That being said--
And also, just put the video tilted at the board so you're not recording the players, you're recording the game
Actually webcams are really good for that sort of thing
Yeah
You don't run out of tape, just record straight to hard drive
Easy to fast forward through, also
Yeah, with your editing tools.
Let's see.
So when you're reporting to other people, think like bullet points for instance.
Huge long paragraphs of prose, not so much.
Let's see.
Try to avoid using the following words, broken, problem, unusable, confusing, kind of odd, you can get more specific than that, but these are words that kind of like imply not only is there a problem, someone made a mistake in his design of the game.
And truth be told, most of the time it wasn't a mistake, you just didn't have the complete play testing, you probably just had the designer.
But you don't want to make someone feel like they've made a mistake because that makes them defensive and then they're going to try to defend how what they wrote in the first place is actually correct, which is actually not correct either.
You can talk about features that need attention.
With attention, this feature could be confused.
I guess that's actually on the edge there.
So but just generally, pick your words carefully and one thing that Sara mentioned earlier was that testers are not designers and key designers are actually not a good testing team.
So for the most part, you are immediately going to exclude yourself from your target audience because you know too much of the game.
This is also covered in your book.
As soon as you've played through your game once or twice, that's it.
Your kind of work is finished.
You can still be a good technical tester, I guess.
You can still check whether some mechanics are working the way that you want to work, but you can't tell whether the dynamics in your settings are working anymore.
You have to get your focus testing done
You also really can't tell, you can never tell-- you can never share your usability tester.
It will always make sense to you.
It's amazing that what makes sense to you is completely incomprehensible to the very intelligent person you give the game to.
Really, truly, I've been thinking about this one myself
Usability testing, by the way, is one of those things that even if you only have the chance to do a pop-up partial test like, all I have is this one card, the game's not complete, but grab someone walking down the corridor on the way to the washroom and say, come in here and take a look at this thing I'm working on.
Even that is better information
Hey kid, come in here.
Come in here, I have something to show you?
I have a surprise for you.
[LAUGHTER]
Actually, you can get going testing for [?
serious.
?]
Free candy
[?
Mona ?]
actually calls it employee kidnapping.
[LAUGHTER] Of course, he's the creative director, so he can do that
It works best if you're accosting people whom you have power over
The audio thing, by the way, is one of those things that could be really useful if people are difficult [?
pundits ?].
And so it might be one of those things that you want to keep in reserve, it's like, OK, there is a problem here and the team goes, I don't think there's a problem here.
I don't understand why anyone's having problems here.
Then you quit, because it's kind of a bludgeon actually on the team.
On one hand, if you have a good agent team, they're going hear it and go, oh my god, we have a problem and immediately try to fix it.
But it's awesome.
On the other hand, it's like, well, do you need to do a recording for every single offer that we have?
That's something to be known about, so keep that in reserve.
I think that's pretty much it.
I have an exercise, but I think [UNINTELLIGIBLE] so I can turn that off.
Do you have any questions while I graph the data segments?
No?
OK.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Anyway, today we have a guest speaker.
His name is Abe Stein.
He's the audio director for the Singapore MIT Initiative.
And he's going to run through brainstorming exercises.
We're not actually planning out the game that you're going to be doing for your first assignment in this class.
This is a-- you're doing other--
Yeah, these will just be practice
So basically, there will be a group activity involved in this, but the things that you're working on, those have to be the same groups that you will eventually be using for your first assignment.
The idea is just to get used to the process of brainstorming and some of the general rules of thumb.
And before I spoil the entire talk--
That's OK.
So my name is Abe Stein.
A little bit of background on myself.
I am the audio director here.
I've been working in sound design officially for a number of years now, five some odd years.
Unofficially, way longer than that doing TV and video game sound design.
And then before that, I was a high school teacher.
And so I think to the extent that talking about creative collaboration, I have a fair amount of experience ranging even back to when I was teaching high school, for trying to encourage what we might call organized ideation.
I guess first off, I'll thank you guys for reading the text.
It is funny.
It does have these I guess now a little antiquated, 1950s references.
And some of his scientific evidence for why brainstorming is good is not particularly well-supported.
He's like, we got 95 ideas.
And it's like, well how many ideas did you get doing it the other way?
He doesn't tell you.
But hey, 95, that's a big number.
So a little bit of background about Alex Osborn.
He was an advertising executive.
And so you can see where brainstorming and advertising is a creative industry, as a design industry.
And so getting multiple people in interdisciplinary different skills working together on a creative project, it could be a challenge.
And so you can see how brainstorming came out of the work he was doing in advertising So just to get started, I would say you've read-- and so you have a sense-- but forget what you think you know brainstorming is.
Some of what you think it is will apply.
Some of the most fundamental principles to brainstorming maybe you don't realize, or you didn't realize until you did the reading.
And that's a little bit of what I want to talk about.
I think the foundation for his brainstorming ideas is actually really solid.
It's something that we should explore today.
Why brainstorming for video game design is an interesting question.
But I touched on that a little bit with its relation to advertizing, right?
I mean we're talking about a creative industry where you have people with various skill sets, whether they're artists or sound designers or game designers explicitly or producers.
All of these people existing in a collaborative environment together.
They're bringing different things to the table.
How do you get them to work together and be collaboratively creative?
And so this is a first step in that process.
And so this is why brainstorming is particularly useful for game design.
And then the other thing that I think is important to understand-- and we'll talk about it a bit more-- is the social pressures that are created in professional creative environments, even non-professional actually.
I mean it would exist in an educational environment too.
You guys are all at MIT.
It's one of the most prestigious universities in the world, right?
And so you're probably no stranger to the idea of social pressure in the process of ideation, right?
You sit there and you say, if I say this idea, am I going to look like an idiot?
And usually the outcome from that is, if I don't say anything, maybe I don't get the gain of presenting a good idea, but I don't get the hit of making a bad idea.
So it's self-preservation.
Why take the risk if it's not worth it.
And so I think MIT is an interesting place to talk about brainstorming and this kind of organized ideation because there is social and academic pressure here too, pressure to be right, pressure to not say something that would seem wrong.
And that sort of pressure is completely antithetical to the idea of a brainstorming session.
And so that's a basis for why not only it's good for video games, but why it's also good to talk about here at MIT.
So there are four basic principles of brainstorming.
These were outlined by Alex Osborn.
He was an ad agent, and he says it was 1938 when he first started using it.
People started talking about it since then.
And I like when he said that it spiraled out of control, that people ran away with it.
They were like, oh, this is a great idea.
But then they were practicing it wrong.
And so in applied imaginations, which is the text you guys read, it's him reflecting on this brainstorming idea and saying this is really how it should be exercised.
The first principle is single-handedly the most important principle, and that's no criticism.
That's a blanket way of saying it, but really in a more broad sense, it's saying judgement needs to be excluded.
In some sense, it's both positive and negative, but especially you don't want to criticize each other's work.
You also don't want to spend time during a brainstorming session talking about how good an idea is, right?
If someone throws out an idea, and someone's like, oh, this is the best idea.
We don't even need to keep going anymore.
Then it's like, whoa, no.
Wait, now we're not brainstorming anymore.
And so really judgement is absolutely supposed to be excluded from the process.
I think that this is the primary principle that goes against that social pressure, right?
Because the social pressure of judgment, the people who are saying what you have, an executive who's in charge of you, a boss who's saying whether or not the ideas are good or bad, or whether you have colleagues that are saying whether or not ideas are good or bad, that sort of social pressure that forces people to stay quiet and maybe hold onto good ideas or throw away good ideas.
If everybody's on board with a lack of criticism, then you have more freedom to express yourself.
Just pulling back to the text a little bit, there was this great quote from this vice president I'll read to you guys again.
He said, "It was hard to get through my head what you were trying to do with us.
My 15 years of conference after conference in my company have conditioned me against shooting wild.
Almost all of us officers rate each other on the basis of judgment.
We are far more apt to look up to the other fellow if he makes no mistakes than if he suggests lots of ideas."
That is something that may sound familiar to you.
"So I've always kept myself from spouting any suggestions which could be sneered at.
I wish our people would feel free to shoot ideas the way we have been doing in these brainstorm sessions.
And so really, this lack of criticism, this creating a safe environment for ideation, is the most fundamental principle in this brainstorming.
And it really goes right at those social pressures which can stop creativity and really hamper creativity.
Any questions about that?
That makes sense right, this principle of no criticism.
Cool.
Principle #2, freewheeling, as Mr. Osborn likes to call it.
The wilder idea the better.
This is also hard to get used to.
Again, it was like the guy said in that quote.
You resist the urge to throw out something that's absolutely crazy and absolutely wild for fear of ridicule.
Well, if we eliminate the fear of ridicule, and we understand that we're just trying to get every idea that we can, then freewheeling is absolutely encouraged.
And you need to get passed that oh, this will never work.
Or nobody would ever like this.
And I put up a katamari up here because that's the sort of idea that someone throws out at a meeting, and if there's criticism, someone will be like, that's the dumbest thing I ever heard.
You can't just roll around picking up stuff for the king of the cosmos.
That's insane.
And it probably was a synthesis of a lot of different ideas that created that game, which we'll talk about later.
But really, having these wild, crazy ideas, you may think it's over the edge.
And it might be over the edge.
But it is way easier to make something feasible than it is to take something that's pretty normal and make it innovative and interesting.
It's much easier to go in that backward direction.
So really the wilder the idea you can come up with, the stronger.
Any questions about principle #2?
Making sense?
On board?
Cool.
Quantity.
This is really the goal of a brainstorming session.
Come up with as many ideas as you can, right?
It stands to reason that if you have lots of ideas, there's a greater likelihood that you'll have a useful one.
That if, say, you have two, your choices are limited.
You're going to pick the better of the two, I suppose, but it might not be the best idea you have.
And so really, this adds onto that wild idea, throw everything out you can.
You need to just get everything out and everything on the page in order to even be able to start judging it.
And that's the important thing too that needs to be clarified is that brainstorming is a first step in a creative process.
It's not the whole process.
It's not like, oh, we have this creative problem that we need to solve.
We'll have a brainstorming session, then we'll have it.
And actually, that's something that hampers brainstorming a lot.
You have a brainstorming session.
You come up with these great ideas.
And then everybody leaves, and you never return to it or you never reflect on which these ideas is going to be best, or you just end up picking one.
And so the judgment needs to happen as well, but not now.
In the brainstorm session, it's just get everything out that you can.
And really it just stands to reason.
And I made fun of Alex Osborn when he was saying, well, in this instance in the government, they had 95 ideas.
And we came up with eight new [UNINTELLIGIBLE].
How many did you come up with before?
But it's true.
The more you have, the better you're going to do.
Cool.
And then this one is related to that, which is the idea of synthesis, or building on ideas.
Which is that oftentimes the best idea, the best creative idea, is not going to be something that's just a unique idea that stands alone in this corpus of ideas that you've collected over the brainstorming It's going to be an idea that is a synthesis of one idea an another.
Or someone suggests an idea, and then that makes you think of something else that's related to it.
And you build on that.
And you can see, in really positive, good brainstorming sessions, there's a momentum.
And one of the things that can really hurt that momentum is when people are starting to riff on an idea, and then someone is like, oh, but in other news, we could do this.
And so it's really encouraged that you synthesize ideas.
If you have that other left field idea, don't throw it away.
Jot it down so that you can bring it up later, but don't kill the momentum of the synthesis if that's happening.
It's really important that you start-- it's the collaborative part of this creative ideation.
It's this combining of ideas that's going to create good ideas.
And I think we can take that as a given, that if you collect a bunch of skilled, thoughtful, creative people together, combining their ideas is going to create something that's better than just any one idea that one creative professional is going to have.
Or at least just a different solution that might be more ideal given the problem.
And so you can think of things like, Lego is pretty awesome.
Star Wars is also pretty awesome.
But Lego Star Wars [INAUDIBLE] is also awesome.
Similarly, Marvel is dope.
Capcom is also pretty dope.
Marvel Versus Capcom is tremendously dope.
So building on ideas can create obviously really interesting things.
And it's a good idea to keep working on that in your brainstorming sessions, what can you put together?
So how is it organized?
We talked about the basic principles.
Any questions about those four principles?
Did they all make sense?
Did they all seem to be grounded in-- you're nodding yes.
It's the greatest thing I've ever heard.
OK, cool.
So the question then becomes, how do you organize your brainstorm.
One of the points that Alex Osborn makes is that it needs to have a casual spirit or a casual atmosphere.
He refers to it as a picnic, which is another one of those-- I haven't been on a picnic in a really long time.
But it's like, we brought out lunches in, and then we talked afterwards.
We call those lunch meetings now.
But he has a good point that if a large part of the brainstorming session is intended to combat social pressures that stop you from being creative and inhibit your creativity, then it would stand to reason that creating a casual spirit, a casual environment for this process is important.
And so most video game processes and most video game companies have a pretty casual spirit and environment because of the nature of the work in general.
So sometimes this might not be that hard.
But things to consider are, what is the makeup of your team?
If your president of the company does a really bad job of putting himself or herself on the same level as all the people, you might want to exclude that person from the brainstorming session.
Because then everybody is going to feel the pressure of the president.
And oh, do we need to make an idea that makes that person happiest.
You have to think to think these things through in terms of what the environment is going to be for your brainstorming session and who's involved.
So it is an important point and an important step.
Starting with a bad or a worthless idea encourages that freewheeling sense.
A lot of times, people will contribute ideas that feel safe.
Again, you want to come up with the most ideas that you can.
So having someone with the courage, or even bringing to the beginning of it a crazy, worthless, bad, freewheeling idea can help encourage people to keep doing that.
So either starting with some who has a crazy idea or even bringing it to the brainstorm at the very beginning can be a really good thing as well.
This is a key one.
Don't interrupt anyone.
It's really important in an brainstorming session to encourage that every voice is heard.
Let me see if I can find that quote from that one guy.
Yeah, here we go.
It's great
Think up or shut up
Yeah, there's that one too, which is a good one.
But it's talking about "the spirit of a brainstorm session is important.
Self-encouragement is needed almost as much as mutual encouragement.
A perfectionism complex will throttle effort and abort ideas."
He says, "One of the ablest members kept mum throughout one of our sessions.
I button-holed him"-- never heard that expression-- "afterward and begged him to spout whatever ideas might come to his mind in our next meeting."
And this is where it gets good.
This is the really able member of this ad team.
He says, all right, I'll try.
But here's what happened.
"After our first meeting, I jotted down about 15 ideas with the thought that I would bring them to our next session.
When I looked them over, I decided that they were worthless, so I just tore up the list."
And Alex goes on to to say it took awhile to encourage this guy to understand that there's no such thing as a worthless idea, that it can either be expanded upon or revised.
And that's that synthesis thing, that having a wild idea.
And getting back to this idea of the interruption, nothing will stop a person from wanting to contribute more than not being to even have their voice be heard.
They're trying to say something, and then they just get cut out by somebody else who is like, oh, this is better, especially for a shy person or a person who is otherwise feeling like my ideas are worthless.
They won't be able to get it out because they've been cut short.
So some of this responsibility falls on what we might call the leader, which I'll talk about later, of the group.
But it's a collective responsibility that everybody makes sure that as someone is expressing an idea, that it's [INAUDIBLE].
And to that, keep the description of your idea short.
If people are sitting here, and you've now hijacked the 30-minute brainstorming session with your one idea that took five minutes to explain, that can also hamper it, right?
Part of it is not only don't interrupt people, but also make sure that when you are contributing, you are allowing for other people to contribute as well, something that maybe is not considered as much.
For the organization, it does need a leader.
I hate that term, but that's what he used in the book.
I tend to use the term clerk, which is a Quaker idea, which I grab onto.
A clerk being more of a facilitator than an actual leader.
Leader suggests boss, which is not great for everyone's even brainstorming session.
And then of course the secretary, or as he calls it, an associate leader, this is someone who writes stuff down.
There's some of that good, old-fashioned hierarchy going on in this text.
But obviously, we'll talk a little bit more about what the roles of these people are, but having those roles defined early is important.
And then this is something to talk about a bit, explaining the process and the principle question at the start of the session.
How many of you had heard of these four principles of brainstorming explicitly prior to reading this text?
Anybody?
Well.
I know.
You heard from me.
But the process is something that needs to be clear.
Otherwise, people won't remember to do it.
You'll see when we do the exercise, it's even hard sometimes to curb your criticism, especially when you're comfortable with people.
And you may not mean it in a malicious way, or your may not mean it in a way to stop the brainstorming session.
But you'd just be like, Oh, that's nuts.
You're laughing, and everybody is having a good time.
And then when you think back, you're like, wait.
That was me passing judgment on an idea and possibly inhibiting the kind of ideation that we're trying to get across.
So explaining the process at the beginning regardless of the group is really important.
And that is usually a responsibility that falls on the leader or the secretary, whoever is organizing the session.
They even suggest in the text, have it posted.
Have the four principles up so that everybody can remind themselves through the process what these four principles are.
But explaining the process is really important.
And then this is so important.
Having the principal question and the goal of the brainstorming session clear at the start.
Obviously, it's encouraging freewheeling.
It's encouraging free thought.
It's encouraging all kinds of wide open creativity.
And at the same time, if you do that in an unbounded sort of way, you might end up with things that really in the end, you don't have too many useful ideas.
You have useful ideas for something else, not for the actually creative problem that you have.
So if you take it from an ad concept, you have a vodka company that's like, oh, we need an advertisement for our awesome vodka or whatever.
And so you're like, all right.
We're going to have a brainstorm session.
And a bunch of people come up with ideas.
Now you're selling dryers because it was wide open, right?
It's like, oh, we've got these great ad ideas for dryers, but they're just really not going to work for vodka.
Bringing it back to the video game contract, you've got a contract with a producer to make a first person shooter, and in your brainstorming session, you've designed a cool new platform.
Neat, but that's not where the money is coming from, right?
So bounding the brainstorm session is important too.
And you'll see that in the exercises that we do today, how it's important to bound that.
Oh, and write down everything.
Sorry, I forgot about that.
This is an important point.
This happens to all of us, but you have this great idea, and then you don't write it down.
And then you go to bed, and then the next day, you're like, oh, I had this great idea and you can't remember it, right?
So writing down everything is absolutely essential.
You'd be amazed the number of times that a week later, you're like, oh, we could put these two ideas together as long as you can remember what those ideas were.
And so what you think, it's hard to not mentally pass judgment on things during the brainstorm session, and what you think might be great now, may be a week later not be as great.
And what you didn't think will be.
And so making sure to have everything written down is really important.
And they even suggest having two secretaries if you need it if you have a large group with lot of ideas and they're each writing every other one.
That can be helpful to make sure that you get everything.
Next is the question of what sort of problems are suitable to a brainstorming session.
The great example that Alex Osborn gives is setting up a brainstorm session on whether or not to get married is not a particularly suitable question.
And that was an interesting concept.
He's like, well, to decide to get married, you apparently list the pros and cons.
That's not exactly the process I went through before I got married.
I was like, well honey, I'm going to have to write this down.
But, fair or not, that was the suggestion that he came up.
And he had a good point, right?
The question of whether or not to get married is a judgment call.
It is not a creative ideation process.
And so that's not a particularly useful problem that's suitable to the brainstorming process.
You want a problem that is going to be benefited by having a large, selection, a large corpus of ideas.
And so those are the kind of problems that are more useful.
And you'll see when we do our example exercise what some of those might be.
I also put here that having a clear definition of the aim is really essential.
It's what we talked about before.
You need to make sure that everybody understands and everybody's on the same page.
Otherwise, you'll end up with stuff that might be useful for something, but not useful for what you're particularly trying to do.
And so making that clear for everybody is really important.
Keeping the problem as simple as possible is also really important.
Trying to keep simple language for the problem.
Keep it constrained in a way that is simple enough so that people can work on it as opposed to just feeling limited by it, if that makes any sense.
If something is overly complicated and overly constrained, then it becomes, well, how do I fit everything into this?
How do I match every single one of these things?
If its' simpler, then it's a little bit easier to work on and try to be creative with.
So keeping problems simple.
Which isn't to say that complex problems couldn't be dealt with, but just divide them into sub-problems.
Divide them into little sections.
So there might be something like, how could we deal with the UI?
How could it look in this game?
So rather than say, well how are we going to design our first person shooter, you've got a brainstorming session about what the UI could look like for the first person shooting game.
You have maybe a brainstorming session for what the fiction for this first person shooter might be.
You're dividing it into sub-problems rather than having it be this gigantic monolith of a thing.
And you can see how that would be a problem, right?
Because if you just brainstorm on what's our first person shooter going to be, you're going to get ideas about the UI.
You're going to get ideas about the fiction.
You're going to get ideas about what sort of guns you should use or what they should sound like.
And it'll just be lots of interesting ideas but all over the place.
And yeah, the problem should be clearly and simply articulated.
I think that's what I said by the shooting example.
Make sure that they're articulated in a simple way.
Sometimes you can have a simple problem that's not articulated particularly clearly.
So it's just something to consider.
Responsibilities of the leader, or whatever, the first being to understand and explain the problem and the aim of the brainstorm session.
I mean a lot of times, people maybe will write a memo beforehand.
As the facilitator of the thing, there's a person who perhaps wants to see the brainstorm be a success.
But mostly, having a clear understanding of the problem.
And explaining that to the group is an important job for the leader.
I use this term facilitating the session, encouraging ideation, discouraging criticism.
You are the watch guard for the process, not necessarily-- you can sometimes be a contributor and a watchdog for the process.
But most important is that you are a watchdog for the process.
You'll see as we do it, it's just so easy to slip into these conversational habits that are contrary to the brainstorming process.
And so having someone who's making sure people don't criticize and encouraging people like oh, I haven't heard from this person.
Do you have ideas?
Even pulling people out is important.
And that comes down to ensuring that no voices are lost during the discussion, right?
I mean we had that ablest member who had 15 ideas that he wrote down, and he's like, they're worthless.
Well, in some ways it's the part of the leader, either during the session or after the session to try to encourage those able members to contribute.
Yeah, some of the most crack pot ideas become the best ones, so you really want to make sure ever voice is heard.
And then encouraging the synthesis, encouraging people to riff on other ideas, trying to find ways for people to build on each other.
Even offering synthesis towards the directives.
So maybe you have a couple ideas.
And you're like, oh, 1 and 10 are similar.
Is there any way we could combine this?
And somebody is like, yeah, we could give him a helmet.
Awesome.
So it's simple.
I don't know what two things you can combine to get a helmet.
Can't get shot in the head.
How about a helmet?
All right, so responsibilities for the secretary.
Most obviously, record every and all ideas.
Write it all down.
You have to write it all down.
This is so important.
This one is often overlooked, monitoring the schedule and the duration of the session.
Something I learned a long, long, long, time ago is one of the worst things that can happen is meetings that run over or meetings that start late or meetings that don't adhere to a schedule.
No better way to lose your employees or lose your staff than to bite into their time.
And so making sure that it's constrained and that it meets the duration is really important.
So, they seem simple, but timing's a thing.
And then of course, they look like participating in the brainstorming.
Like I said, all voices should be heard, and that oftentimes will mean that the secretary is a creative professional as well.
And so making sure that you participate [INAUDIBLE].
So we've got a few exercises now to get you acclimated to the idea of brainstorming.
The first one is an actual individual brainstorm.
So if you have paper, grab some paper at the top of the pile
If anyone is lacking either writing equipment or paper?
You could use a note card too.
Really just something to write on other than the table.
Note cards are totally fine.
As you guys are getting ready, I'll continue to talk.
This is actually something that I didn't mention in the process but is actually very useful to the brainstorming process.
I think doing warm ups before you actually brainstorm is useful.
You know how people show up just from lunch, especially if it's one of those picnic meetings.
People sometimes are tired.
People just might have other things on the brain.
So getting into the habit, getting warmed up to being creative is important.
I tend to find it's a rare kind of person who you just give a blank piece of paper to, and you're like, oh, make something.
Go, be creative.
And they're like, vroom.
They just whip something out.
You have to get warmed up.
Visual artists sketch constantly, and that makes sense.
And then they build on sketches.
I just make dumb pieces of music that I'll maybe never use for anything, but maybe I will.
Who knows?
I make stupid sound effects in the shower.
I consider that warming up for work actually.
So you have to get warmed up.
And so this is just a really quick, five-minute warm up.
And the idea is, write anything that comes to your mind for this specific problem, and I'll show you a picture.
Brainstorm possible uses-- it's not really a brainstorm.
It's more just that you individually come up with creative ideas, creative uses, for the following items.
And I'll give you five minutes starting now.
All right.
Hopefully it was good for a warm up.
Quick question by a show of hand.
Be honest.
This is a safe environment.
How many of you had an idea that you did not write down?
How many of you self-censored yourself with an idea?
Yeah, Tommy, you were like, oh, that's dumb.
See, it's hard to get out of the habit of doing that.
It's hard to get out of this habit of self-judgement.
And it's really important.
That dumb idea that you think was worthless might be a good one.
Does anyone who raised their hand want to share what their censored idea was just to exorcise the demons, so to speak?
No, no one?
All right, who wants to share some ideas that they did write down?
See, I have a lot
Yeah, well throw us a couple that are your favorites
C, lock pick, MacGyver, connecting circuits
There you go
Caring hurts, a knife, clothes hanger, chopsticks, dowser, stirrer, connect batteries, let air out of tires, emo tool, nose picker
Sure
Staple.
And a paper weight.
I have a lot of things
Yeah, good, that's awesome.
Try it out?
All right, I got, let's see, fingernail cleaner, an eyelash straightener, basically rail gun ammo, a balloon annihilator, and an official Tamagotchi TM resetter
It resets a lot of things actually.
Over to this side.
Owen, go ahead
I had [INAUDIBLE].
I had scratch messaging tool.
I had used as a key ring, conducted electricity with it.
Made it into jewelry.
Fixed umbrellas and what have you with it, which I've done before.
A money clip.
Use it as a screw for cooking.
Put it on top of the door, and so when people open it and it falls on the ground, you know that somebody has been in your room.
Lance boils, poke eyes out.
Hair accessory.
[INAUDIBLE]
That was a good question.
Did you have a whole paper?
[INTERPOSING VOICES]
I had hairstyling accessory, projectile, something you could melt down for metal and other stuff.
Something you could bend and then put on chairs, and then when people sit down, it pokes them.
Modern art.
I thought you could get a bunch and fill a dorm room with it as a prank or something.
Toothpick.
Hold stuff open for emergency surgery
Cool.
Some other ideas?
I thought you could bend it into an interesting flat shape, and then put it in ink and use it as a stamp
Very nice, very good.
Cool, yeah, go ahead
Game bits and tokens.
Holder for dipping something into liquids.
Tie clips and keeping small boards together
Very good.
Sure, yes?
A mini rock garden rake.
Also a [INAUDIBLE]
That is very good.
Go ahead
Heat generator.
If you work hard on it by bending it and unbending it, then it gets hots
Very nice
As long as it doesn't break
Yes?
I think about a souvenir as a bulletin for a lost robot
That's good.
Say that again
I was going to say that it's a robot that is made by humans, but it's lost in space.
Then we have this thing that reminded us of [INAUDIBLE] a secret thing
My favorite use for a paper clip is cleaning the lint out of my iPhone ear jack, which drives me absolutely crazy.
So I'm running on reserve battery power, and my slide show has been suspended.
In my office, there's a air powered connector thing.
So yeah, how did we feel about that exercise?
It got you to loosen up a little bit?
It got you thinking about some ideas?
It got you feeling the creative juices flowing?
We have three more exercises that we're going to do.
So much brainstorming.
This is the hardest work I've ever done at MIT.
No, so we have a few more exercises to get you guys comfortable, to get you guys relaxed with this brainstorming process.
The next one is going to be a group exercise.
I'm going to wait to describe it until I can get my slide show up.
Anybody else have any particularly choice paper clip ideas?
Oh, I just thought of one
What?
A sine wave generator
Oh, very good
Wire frame doll
[INAUDIBLE] shaker
Oh yeah, totally.
[INTERPOSING VOICES]
Hold up a second guys
If we stop looking at it as an [UNINTELLIGIBLE].
Take it apart like an Army man and use it as a spear
Yeah, like Man Vs. Wild catching fish with it.
Bear Grylls.
Something
To chew on
You could use it as a catapult if you bend the top part
Rubber band launcher
Letting air out of tires.
[INTERPOSING VOICES]
There we go.
[INTERPOSING VOICES]
Do you remember that guy who had an oversized red paper clip and ended up trading it for a house?
Currency towards a house
Cool, so hopefully that got you loosened up.
I do want to just emphasize again, for those of you who raised your hand, and even though of you who did not raise your hand because you were censoring the truth because you were afraid, try not to self-censor.
Really, it's important to brainstorming
It sounds like the military
Yeah, stop self-censoring.
I bet those crappy ideas that you didn't write down are actually really awesome.
So activity #2, we're going to work as one large group.
This is our conch shell.
It doesn't look like one, I know.
But the principle still applies.
This is just to get us in the habit, in the shape of speaking one at a time, not interrupting each other.
And in some senses building on each other's ideas too, right?
There's something in the physical passing of the ball after you've heard someone's idea that might encourage that sort of synthesis of ideas as you're going.
I will be your fearless leader.
All that means is that I just make sure you're not interrupting each other.
And we're going to brainstorm solutions to a design problem.
This one I think is actually a lot of fun.
So, I will show you the design problem.
I will maybe throw out an idea, and then I will toss the conch shell to the first person who wants it.
And then it doesn't have to go back to me.
Just anyone who wants it, throw up your hand.
And we'll keep moving around.
Please close the lids on your drinks.
Put away your crystal stuff that you don't want shattered.
So we'll take approximately 15 minutes on this.
We'll see how our momentum goes.
So you've just finished updating Tetris for Xbox Live Arcade.
Awesome.
I don't know how you got the rights, but that's great.
Microsoft is demanding a set of achievement for integrating the game into their online service, because unless you have achievements, it's not a real game.
So, we are going to brainstorm possible player achievements for your new version of Tetris.
You know what achievements are, right?
Those little things that pop up on your Xbox every time you do anything?
So that is our idea for Tetris achievements.
And I'll start.
How about for actually getting four lines, you get a Tetris achievement.
Who wants to be next?
You get an achievement for losing as quickly as possibly, which they think will be like, really obvious
Getting three lines with an S-piece
Drawing some sort of unique little picture with the Tetris blocks, like something new perhaps
Putting one of each block down
Lasting for a certain amount of time without ever getting above a certain height
Very cool
Lasting a certain amount of time without ever locating a piece
Modern Tetris games have a system where you can rotate indefinitely when your piece is down.
So you rotate indefinitely for a minute or two or three
Lasting a certain amount of time without ever clearing a line with a straight piece
Putting your piece down underneath another piece into a little hole
Sliding it in?
The T-spin
Spelling USSR with your pieces.
That was a bad throw
Making a conglomeration of all pieces into a shape of a regular piece.
So the whole board is one L-shape
Very cool
Basically only moving the pieces in beat to the music that's playing
Spelling USA with your pieces, but doing it
Capitalist centered
High score and high score without pairing lines
Managing to clear the entire board in a game that has already reached the top line, but not quite overflowed
Putting a piece down without actually looking at the piece at all
Most pieces per time, so [INAUDIBLE]
Sure
One day of continuous Tetris play
Marathon challenge
Breaking the previous high score
Only ever shifting the pieces either to the right or to the left the entire game
Ah
Doing the marathon challenge but only listening to one piece of music the entire time
Clearing a level by dropping pieces before they fall five spaces
Oh, all right, cool
Rotating every single piece that you play
Oh, very good
Never going above a certain line for a certain amount of time
Ooh, a sustaining effort
All of the above, but on different difficulty levels
Clearing three lines with a T, I guess
There you go.
Sure
Starting up the game
Clearing an entire screen
Using the L-block.
It doesn't matter how
[?
Turning ?]
a box a different color.
So clearing lines always with the same color
Rotating a piece 720 degrees
Only matching pieces such that the U-blocks that fall down are touching blocks of the same color
Cool
Clear two, four block segments with two consecutive lines
Clearing a level using only tetrices
Sorry, you can't talk anymore
Challenging somebody on your friends list to beat the score
[INAUDIBLE] someone on your friends list to lose faster
Having no holes in the golf course at all times
Oh cool
Inviting one of your Xbox Live friends to play with you
Five consecutive line clears
Beating one of your friends on Xbox Live when you play with them
Losing to a friend on Xbox Live
Playing with a friend for one hour continuously.
Plus one day
Beating a score set by the original creator of Tetris
Clearing a line somewhere above the first line when the first line only has one block filled in
I think it's possible.
It definitely is.
You get the flat ones
Someone has to have ideas.
There's an idea
Lose more than 10 levels to a friend.
[INAUDIBLE]
Play the original creator of Tetris on Xbox Live
Losing 100 times
Yeah, there you go
Playing on increasingly ridiculous difficulties with achievements for each.
Or speeds, increasingly difficult speeds
Completing the Tetris challenge, which is presumably some kind of game where you have to throw a box in the right sequence in order to clear all the lines
Clearing 1,000 lines
Playing invisible mode
Playing for a certain amount of time while holding down the Down key continuously
Only rotating in one direction, always.
I don't know if someone said that already
Getting 10 achievements in Tetris
Getting every achievement in Tetris
Oh no, how can you get it?
It's a paradox.
It's impossible
Always have lost on the far left and on the far right column.
Oh yeah, sure
Cool, I think that had some decent momentum, but I think our mission was accomplished.
We were really only giving one idea at a time because it was only the person who had the ball that was talking.
That, probably with a little bit more speed and momentum, but it was nice.
We had a little building on ideas.
We're starting to get this brainstorm.
There was still some shyness, which is-- people have different personalities, but stop being shy.
You're allowed to have different personalities, but stop it.
No, so that's activity #2.
Any questions about that process?
How did we feel about it?
Any thoughts on it before we go on to the next activity?
All right.
Next activity.
Actually, this is our last activity.
Sorry, I said there were four, but there are only three.
[INAUDIBLE] I screwed up.
OK, so this one is a more formal brainstorm.
And actually, I screwed up that last one.
We didn't have a secretary, so all of those great Tetris ideas are now lost because I can't remember any of them
[INAUDIBLE]
Oh yes.
Did we not have that?
Let me just get these [UNINTELLIGIBLE].
So what we're going to do is we're going to separate into two design teams.
So it will just totally be by location.
We'll just get half of you over here and half of you over there.
I want the two to designate one leader.
That's going to be the person who's responsible-- clerk-- for facilitating it.
Did I say clerk there?
A leader and a secretary.
It's supposed to a clerk leader and a secretary.
So one person to write everything down, and one person to obviously facilitate.
And we're going to do 20 or so minute brainstorm for a game pitch.
What do I mean by game pitch?
Think of the, you're in an elevator with Miyamoto or you're in an elevator with [?
Philip Haas ?
], and you have this game idea that you need to pitch.
But you're only going to the 10th floor or so.
And so really, you have 30 seconds, a minute, two minutes, to get the core information about this game out.
So that can be something like a title, maybe a tagline for it, any main features you might have for it.
Of course, it would be way too easy, or way actually crazy for me to be like, any game at all, because there are lots of different types of games obviously.
So it's going to be a game based on the following constraint.
So what I'd like to do first before I give you the constraint is let's divide ourselves in half.
We can draw this imaginary line diagonally dividing the two of you back there, and we'll just get one half here, one half there.
[INTERPOSING VOICES]
And once I give you the constraints, we'll actually find a separate room for the other team
You're stuck on that wire.
That's an achievement there.
[INTERPOSING VOICES]
If someone has a piece of paper, they can jot down their ideas.
Just write on a piece of paper.
So these are your teams.
You guys will go in that room.
You guys stay out here.
As soon as I give you the constraint, designate a clerk/leader, a fearless leader, and a secretary, someone to write everything down.
Because our groups are relatively small, right, we can have the leader and the secretary involved in the brainstorming process.
Please don't be afraid of contributing.
But if you are designated as the leader, keep in mind your primary responsibility is facilitating the brainstorm.
You want to make sure it runs smoothly.
You want to make sure people are not criticizing ideas.
You want to make sure that it's going OK. Any questions on this before I give you your constraints?
Everything makes sense?
All good?
OK.
Here are the constraints for the game.
It must have zombies.
It must have a love story.
It must be massively multiplayer.
So those are the constraints for the game.
That's what the publisher is going to give you money for.
So you need to develop a pitch for this game idea.
Title, tagline, what have you.
All right, so you guys, we'll take you into this room over here.
You guys stay here.
Designate a leader and a secretary, and then start brainstorming.
Oh, I love this.
It's like, I don't want to be the leader.
I don't want to be the secretary
I said I'll do it
But you wanted to be the secretary
One of you is secretary.
One of you is leader.
Be a leader then.
[UNINTELLIGIBLE] and you lost
It brings up the age-old question.
If they're both vampires or they're both zombies, does it count as necrophilia?
Fun, let's see
Pride and Prejudice and Zombies manages to have--
It encompasses all of those except the massive and multiplayer part.
Licensed title right there
Make it Nazi zombies.
Two Nazi zombies.
No, a forbidden love.
A Nazi zombie is in love with an Allied zombie
Jewish
Yes
Yes, and then so there are two factions.
Well, they are all people online.
One group-- there are two factions.
There's the Allied group and the Axis group
What if there are two factions.
There are the zombies and the humans, and they're drawn into war because there's a love story between a zombie and a human, and and the leaders get pissed over this.
And so we're at war because of Romeo and Juliet-esque zombie human--
So what you're saying is we're having intelligent zombies
Yeah, intelligent zombies
So clearly they're Nazi zombies
Semi-intelligent zombies
The zombies should decide to become Nazis
We need a tagline
Title
We have to somehow bring Hitler into this though.
Because if Nazis, there must be a zombie Hitler
What is your name?
Jason
I'm Brian.
On Jason's idea of is it necrophilia, I think we could do a love story.
The title could be Necrophilia?
Is it necrophlia?
[INTERPOSING VOICES]
I like it as a title
They say it is a love story, but is it a sex story
How about this?
Or maybe even zombies strippers that fall in love with them
Have you seen Zombie Strippers?
No, I haven't
It's a terrible, wonderful, terrible movie
Well, I'll watch it then
You should, not, should.
Actually, how about we have the zombies as the good guys.
That way, the Allies are trying to make a weapon that will destroy half the world, and the zombies are like, no, that would be bad.
Let's make everyone else zombies so they can be with us
What if you're just a zombie, and everyone is trying to kill you.
So you want to make everyone zombies so that the world will be peaceful and happy
And you want everyone to love you
I like that a lot
And then it'd be a love story
Even better, love zombies.
Who says they have to be in bed?
They can be infected by emotions.
That's the tagline right there, "Infected by Emotions."
Nudist zombies
What if 95% of the world is zombie, and so then the weird creatures that are scary are the humans.
And they're sentient and they do weird things, but the zombies all are happy and love each other
Maybe the humans have the love disease
They're trying to hump zombies?
Like they're infecting zombies with love
So then each person would either start as a zombie or a human.
They'd have to complete a bunch of little taks so they can come [INAUDIBLE].
They challenge each other, so it'd be like, zombie against zombie, and one of the is infected and one isn't
So humans can infect zombies with love that makes them not zombies?
No, they are just love zombies
What is the dynamic between-- so there are the two factions, obviously humans.
And every player is given a match.
And in this massive multiplayer world, one of their main objectives is to find their match who's in the opposing faction very deep among hostile hoards of either humans or zombies.
Find them and live happily ever after together
And become either zombies or humans together
Correct
But love zombies
Find a cure or just get bitten
And become love zombies
Love zombies
And so that's the question.
Is there truly a difference between in love and being undead?
That's a great tagline
[ZOMBIE VOICE] Love
Maybe some [UNINTELLIGIBLE]
What can make you more blind?
Having your eyes go dead by a zombie or being in love?
So then we can draw it into the actual Nazi zombie game, maybe the Nazis zombie is from the other point of view.
So they're just trying to have love, and the humans are trying to kill them
What?
Wait, are we having Nazis?
We can
Most of the ideas can include Nazis, and some can't, and we can decide later
Phil, should we get into the point of judgement towards the end of thing so we can finalize a pitch?
No, just be brainstorming
Just brainstorming?
That's awesome.
Yeah, this is fun
How about the player is a human fighting against the zombies, and you have some kind of companion-- may or not be a cube-- but accompanies you on your journey to defeat the zombies
So a companion.
I like companion cube better.
That has a heart.
It's like a love story between you and your cube
But can the cube become a zombie?
Yes.
I mean it could be a person left for dead
Four people against all the zombies [INAUDIBLE]
Then we need spitters and stuff
Or witches.
Maybe you're trying to get-- the love story is you and the witch, considering the scene
What if all players play zombies, and most missions are stuff like you have to infect a city, and you run around biting people through the entire city that you conquer and try to zombify.
But your ultimate goal is that your girlfriend from back when you were human, or your boyfriend from back when you were human is at someplace.
And you want to infect all cities until you can get there, and then infect her so that you guys can live happily ever after
Ahh.
How sweet
Also, building on the idea of being zombies--
We're not done yet
You are zombies with Cupid's love arrows trying to infect the entire city with love, at which productivity is all gone in the city
A zombie with love arrows infecting people.
It could be set in the 1950s where everyone is trying to be productive and not love each other
What if you build along the lines of having to infect the city, but one zombie can't infect the entire city, right?
So you have to ally with other zombies.
But there are different factions of zombies, like two zombie mob families that are trying to infect.
They have two different varieties of this love-- [INAUDIBLE]
Can it be a drug trade?
So they have to infect people with a certain drug, so it's a drug war as well?
And there should be swords because swords are cool
Zombie Mafia?
What if when you have a zombie virus, instead of going in to eat people, you are overcome with an urge to passionately make out with them.
And then if you make out with somebody, then they turn into a love zombie too.
And so there's no decomposition going on so much as there are hordes of people running through streets trying to passionately make out with each other
Decaying and decrepit love zombies suddenly become very attractive
And they start taking care of their appearance and stuff
After the zombie make out, what's next?
Well, we make it [UNINTELLIGIBLE]
What if we just make it like a World of Warcraft type MMO, but every player plays a human.
You have a zombie companion
Pet zombie?
You have to raise your zombie from a little zombie kit.
[INTERPOSING VOICES]
Shaun of the Dead?
No, but they had a pet as a pet zombie
I think it's called Fido.
It's a movie called Fido
No, it's not that one
I've not seen it, but I've seen a trailer
So what if you have to fight with Pokemon zombies.
Get their power-up [INAUDIBLE]
I like that
Different kinds of zombies like Left for Dead.
There are the big fat ones that vomit
Spitters, I hate the spitters
The smokers.
The smokers are the worst
Instead of power-ups, you have love potions
When we said smokers are the worst, that sounds like a dating site
Wait, I like the Pokemon idea.
So you're human, but you just have to capture all the different types of zombies.
And then you fight other people that are zombies.
And it's a match, zombie versus zombie
What if the love story is not for the player but is in the story of the game in that the reason that you're waging war at the moment is because you're trying to steal a cure to zombyism because your fearless leader, 20 ranks up, your president's girlfriend has been zombified or his wife has been zombified.
And so the whole game, you're the only means in a quest to try to find this cure.
But you don't realize that you're not trying to find a cure for your use or your girlfriend's use or for people you love.
It's for your leader's
And what if she is actually the leader of the zombies.
Oh crap
What if the zombies are aliens?
Zombie Aliens?
The flood
What if the zombies are ninjas?
Combine the zombie Pokemon idea with the Sims, where you have a zombie that's under your control, and you're trying to teach your zombie to love.
And maybe you're a different avatar.
You're a human avatar.
And you go through quests where you take your zombie places and teach your zombie things
Like not to bite people
Yeah exactly.
I don't know.
Train your zombie
So the whole thing with the ninja zombies, why not have pirates as well, pirate zombies
Pirate versus ninja zombies
Zombie edition.
Special edition
In space wrapped in bacon
Chtulu, but time traveling zombies
It would be a great game to make, just pirate zombies versus ninja zombies in space wrapped in bacon.
And just make that game
So, is it necrophilia?
Probably.
Pirate edition
So maybe we should divide it into little sub-brainstorms for a name, tagline, all that.
As a name, do we have any more brainstorming?
I think we should figure out a mechanic that works.
So far we have a lot of ideas about what the game world is.
We have no idea how you play with game
So maybe more brainstorming on that then?
It's massive multiplayer
Ones working such as typical WoW-style RPGs, typical massive multiplayer IPS.
Pokemon style
Pokemon
Or, as we were saying before, we could get The Sims online
The Sims online was so good back in the day
Did you play the Pokemon?
Yeah, we could play the Pokemon, but it's a first person shooter.
But you still have an RPG, first person shooter as Pokemon
It could entirely text-based
You have to fight-- you have battles with words like Monkey Island
I like that, text-based zombie love
Text-based zombie love
And then he sticks his really, really long tongue down your throat
Zombie make out disease
I thought of a title
If it's massively multiplayer, what if you randomly get paired with five other players who are playing this game, and you all have some task you have to complete.
And you are given these particular zombie Pokemon that you can use this particular task.
So you'd be working with those five, six people
Wait, will you have zombie Pokeballs?
What?
We have a title
So Necrofeelia with F-E-E-
The first zombie sex game
Exactly
And this can be [UNINTELLIGIBLE], you have to be tentacles somewhere
What if you are a zombie.
And all the players are zombies.
And the NPCs are the humans, and you try to go on quests to make yourself more attractive and learn human customs.
And you get points for seducing humans
You have to make sure your brains meter doesn't get too low
So you have to eat them
Use them and then eat them
And then if you eat them with certain qualities--
That actually sounds like a fun vampire game.
You're a vampire.
You have to seduce a human female to bring back to drink
It's like Prototype
Or that you have to seduce a human because you want to seduce a human.
But at the same time, you have to eat humans.
So you have to hide the fact that you're eating from the human you're trying to seduce
Oh, I see.
So you have a human girlfriend, and then you eat other humans
While she's not looking
Have you been eating other girlfriends without me?
Where's my stepdad?
Uh, about that
Oh, that's who that guy is.
I thought he was the delivery guy
Let's see.
Zombies meets American Psycho.
You've never seen American Psycho?
He goes around killing random people, but he has a girlfriend on the side.
But he's crazy.
It's fun
So it's like Psycho
But better
So Necrofeelia
Get infected with emotion
Necrofeelia, infected with love
Brains, love, it's the perfect game
All right guys, I'm going to have you stop.
We're going to bring the other group in
Love after Life?
I like it
That's really good
All right guys
Our publisher is going to give us so much money
It may seem odd to have you do this brainstorm, and then what I actually want to talk about is rather than give [UNINTELLIGIBLE] because I'm sure they're all really good
Ahh
I know, so sad.
But what really matters is the process, because the point of brainstorming is to withhold judgment.
Judging which of your ideas is best shouldn't have even been happening at this stage.
And so one kind of question-- we'll come back to what went well-- but what sort of stumbling blocks did you guys run into in this process as you were doing it?
Did it all go perfectly?
I highly doubt it
I would say we were trying to design this whole game, so sometimes we would say features.
And then we [INAUDIBLE]
Yeah, in some senses, it may have been constrained in one area.
Sure, zombies and a love story.
But then it was asking for a whole lot, and so it could have been, can we just brainstorm a title.
Or could we just brainstorm this or brainstorm that.
What about you?
We very briefly got sidetracked on Pokemon.
For a second, it was Pokemon only, Pokemon first person shooter
Any other thoughts?
I mean you can talk about the whole group.
You can talk about your own difficulty you had personally, whether you had any challenges with--
So I feel like if somebody says a really good idea, then everybody is stuck on that idea.
and you have to get through that whole idea before any other ideas can come out
Yeah, that's true
We had a stumbling block.
What other MMOs are there out there?
So [INAUDIBLE] a few other ones.
That was a dead spot in the brainstorming
Sure
I was the scribe.
It was hard to jump in and talk about your own ideas because I was so busy writing everyone else's
It was really hard with brainstorming.
You guys actually-- both groups-- but I stopped in with you guys.
You guys had this interesting momentum going, and I was watching you try to write it all down.
It was way hard because the ideas were coming in as you were finishing the last idea.
So if a group has really good momentum, I actually really like the idea of having two people where each person is writing every other one.
Of course the difficulty is then you pull two people out of contributing.
So the organization of the actual group is important
Do you ever recommend using a tape recorder?
Yeah.
Yeah, absolutely.
I mean it certainly makes sense.
One of the things that Osborn says which I think is--
You can't put a tape recorder down.
I remember that
Yeah, but he also says that one of the most important things for the secretary scribe is to number things so that you know how many you have.
Is that just to support your work?
Look, we came up with 110 ideas.
Amazing.
How many doesn't seem to matter as much as the qualities of the ideas.
So I thought it was weird that he said it.
But yeah, tape recording, video recording.
Any way of documenting the process is I think the most important part.
And actually, he does make a good point of saying that one of the problems with brainstorming as it evolved from when he first introduced it at this one place and then people started running with it without thinking about the principles is they would brainstorm, and then that was sort of it.
And you really have to look back at what you had come up with.
It's really important to reflect on what you've done as a second step.
And so that's important too.
How about the judgement thing?
I didn't notice a whole lot of criticism.
It was really good.
Nobody was like, that's a terrible idea.
But did you find it hard to not pass judgment on ideas as they were coming out?
Even your idea?
Yeah, I mean I definitely let more than one, "I like that" slips
That's OK. What you want to avoid is like the, oh my god, we've got to do this one.
That will derail everything because then it's just a loud voice that proclaims that we're done because we got the idea.
But nothing encourages a person who is shy more than feeling like their idea actually was good or meritorious.
And so that sort of stuff is fine.
You want to avoid the stop the process
I think I did say at some point, ah, we've got our title, which is probably not good
Yeah, it's just a hard thing to train yourself to do
Is it Necrophilia?
A great title
Are there self-censors still out there?
Were there a lot of people who thought of ideas but then just didn't contribute?
Good, you didn't.
It's great if everybody felt free.
Cool, that's really good.
So I would say that's one thing that went well.
What other things went well for you guys?
The tagline
Specifically, just coming up with taglines?
Infected by the-- Is it love after life
That's cool.
Did you guys feel like you had momentum?
It seemed to me watching that there was momentum building
I think we did a really good job of taking a lot of different ideas that we had and synthesizing into different things.
Saying, oh, that idea would work well if we did this.
Oh, and then we could use that name, and that would fit there
Yeah.
Who were the two leaders for the two groups, the clerks?
Did you guys feel like you had to do much facilitating
Not really, the hard part was really at the beginning, where it was, how do we want to approach this thing?
And so I guess we decided to start with the title.
It was either the titles or the main features.
And we were just like, let's come up with a whole bunch of titles and go from there
Sure
I forgot I was the leader.
I didn't lead anything
If you don't have a whole lot of facilitating to do, it generally means that the group is running OK.
I do think you have a good point.
A lot of the facilitator's work comes at the beginning of the process, or even before the process began, making sure the constraints are clear.
And then you guys know what you are going to be working on.
OK, so bringing this back-- this is a game design class, right?-- I touched on it earlier.
How do you guys see this as being a useful mechanic or a useful tool You know what?
It's funny I called it a mechanic.
Did any of you catch that paragraph in Alex Osborn where the guy was saying, "When my team feels like they're playing, it's a way more productive brainstorm than when not."
And I thought that was really interesting because there are elements of gaminess to this, which I think is interesting.
So how do you guys this could improve the general game design process?
I feel like after you have a game and you're testing it, and there's a major flaw that is taking too long or something like that.
And everyone sits down in a brainstorm and says, this is exactly what our problem is.
How can we fix it?
Yeah, overcoming road blocks
We never would have come up with half the cool ideas that we came up during the process.
Just me thinking alone never would have come up with Pokemon zombies
Sure, anyone else?
Well, this is good just in general just to think up a game.
Because maybe it's like, oh, every single game has been decided.
People make games of all types.
How am I going to make a unique game.
This brainstorm is like, hmm, those two things.
I've never seen that before.
That could be interesting
Yeah, and that's a really important good point actually.
The synthesis thing is actually I think something that is really important because we tend to see a lot of recycled mechanics.
And what becomes the interesting thing in a game oftentimes is not that it's a completely new rule, right?
It's the way that rule interacts with the other rules in the system.
And in some instances, it's also how that rule is read and interpreted by the player, or perhaps tends to be better in terms of other players.
So yeah, I think the synthesis is actually an interesting part of this where people can bring a lot of different ideas from a lot of different experiences with games to fuse ideas
When it comes to coming up with ideas for games, the nice thing about brainstorming is that it gets past the good idea that everyone happens to make, which is the problem that we sometimes have in the classes.
The people will find one idea that they like, and they will never think past it because they were happy to find one that everybody agreed with
Yeah, and so you guys are divided into design teams, and one of the things I would encourage-- I don't know how it's set up schedule-wise, but try to divide the brainstorming process from the judgment, like I said.
It's true.
Even if you do the brainstorm, it can help you get over.
But a lot of times, whatever had momentum during that brainstorm or whatever got the most yuks during that brainstorm is going to be the popular, strong idea.
And then when you come back to it with a little space, you might be like, oh, that it is a really good idea.
But there was this other thing that got mentioned half an hour earlier that's really interesting.
Maybe we can incorporate that somehow.
And so making the judgment happen in a separate phase can be really important.
Because it's true, creativity is a little bit more organic.
It's not necessarily a strictly logical process, right?
So applying logic later can be a good thing, I think, to the creative process.
Cool, any questions about brainstorming at all?
You are incredibly enlightened by Alex Osborn?
[INAUDIBLE] using things like Google Wave or Google Docs where things are automatically recorded because everyone is basically typing in their ideas as it's happening
Yeah, and I think it's interesting.
I don't have a whole lot of experience with-- well, I should rephrase.
I don't have a lot of experience with remote brainstorming, but I one of the games I worked on this summer was this game called [UNINTELLIGIBLE] where you were asked a lot of personal questions.
And so we had to think of personal questions.
So we were like, well, if you have ideas, write them down in Google Docs.
And at one point, myself and two of the other designers were all on the Google Doc at the same time.
And you could actually see that that sort of brainstorming would happen.
Someone would type something in.
And then someone would type next to it, ha ha, that's so funny, and then write an idea that's related to that.
And you're seeing it happen in this digital space.
So I think we've gotten so accustomed to having these digital personalities or these digital personas in IMing and text messaging and all that other stuff that I think we've gotten pretty adept at typing fast enough to get ideas out that I think it could totally work.
There's something to be said for the momentum that gets generated in person.
But yeah--
You could try both ways
Yeah, I mean it's so funny.
I was going to say, no that's a terrible idea.
You have to be around each other.
And I was like, actually, it works.
I think it all depends a little bit on the personalities of the people that were involved.
Certainly for a prolonged brainstorm-- one of the thing Alex Osborn says is time box it.
If you don't time box it, it's going to be a problem.
I think for in person, that's true.
It's so easy to get onto a Google document now whenever you have an idea that actually having a long form brainstorm where anytime someone has an idea over a week or something like that, write it down.
That's also useful too because then you separated the process into collecting and-- But you still should do that, building on ideas
It's probably easier just to do this [INAUDIBLE] when you're simultaneously online or in the same room
Yeah, and to see the ideas come out as they come out.
Because it's a little uncommon to just read something and be like, oh yeah, what about this when it first shows up.
Or when it first comes into existence as you're trying to think of things that can spur ideas
The other danger about doing it like, OK, this document is open for the next 24 hours, everyone throw your ideas in, is that then people start spending too long on any one idea.
They start thinking about what they're going to write down before they write it down.
No, that's not the point
Yeah, think of the paper clip experience, and those of you that raised your hand for saying that you were self-censoring.
On your own-- I wasn't going to make you all hand in those paper clip ideas you were self-censoring.
So the same sort of thing applies.
You might censor yourself or spend a long time on ideas.
Any other thoughts, questions?
Cool, well I hope this was helpful.
I hope it'll be helpful for your game design stuff.
Email me if you do have any questions about this stuff.
I can try to help out.
Other than than, good luck on your game design on this class and for the rest of the semester.
Cool.
[APPLAUSE]
I do like that of the keynote things it says, "End of show."
Let's see now.
We're going to play some card games.
But before that, just a little bit of admin.
On Monday, you'll notice that we have two readings, one of which you've already done.
As for the other reading, [UNINTELLIGIBLE], it's a little bit dry.
Tough it out caffeinate yourselves before reading it.
Tough it out.
It is worth it.
It's a translation.
Let's see now.
There's at least one person who hasn't filled a form that you need to fill out.
So just come up here and fill that out.
Is Courtney in class today?
[INAUDIBLE]
Yeah, I'm not sure if we have enough time to get through all of them.
But most of them are pretty short.
A lot of them today are going to be set up, which is why [UNINTELLIGIBLE] is first because that takes a while to set up.
That being said, if you do get a chance to look at the rules and study the cards, even [INAUDIBLE] a bunch of smart things [INAUDIBLE] easier to set up.
And you really want to pay attention to that.
But the name of the game today is pretty much variety.
We have all fairly different games that use cards in a very different way.
And if you don't get a chance necessarily to play all of them, at least get a chance to look at other people playing them.
And if you wanted to check them out over the middle of the week, we'll keep them around the office.
But there'll be time to come back here next week and [INAUDIBLE].
But that's basically it.
It we can clear the tables because that's for the cards.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
So a lot of folks, for instance, the rise in online poker or something.
And they will say that, oh, we don't fall into the tax bracket for gambling because we're a skill-based game.
But literally [INAUDIBLE] is about competition.
In some ways, you can class it as sport.
But the reason why I was asking entirely is because the way how Caillois writes it, or at least the way how Roger Caillois has been translated, kind of makes it sound like he's actually drawing hard divisions between all these categories.
But really they're all adjectives
What was the original language?
French, I believe
I believe that's the original version
Correct me if I'm wrong.
[INTERPOSING  VOICES]
If anyone can confirm that, that would be awesome.
Write a blog post about it
Jeremy, you're on it
Yeah, I'll try and confirm that it's actually French.
I'll confirm it, and then write an obnoxious post about translation errors.
[INTERPOSING  VOICES]
I'm sure it could have been translated in far more interesting language, because it's kind of dull
[INAUDIBLE] French [INAUDIBLE] if it actually it is French
And then we have more meetings on Caillois on Wednesday.
Yay!
But it's an important text.
But [?
Ellings ?]
will be, I use the word vertigo a lot, because it sometimes gets sort of substituted.
But vertigo, I think people have a very specific understanding of vertigo as getting dizzy because of heights.
And obviously not all games are about heights.
But people, sure, they use it to describe the sensation of taking a roller coaster, which literally is about heights.
But it's also about speed.
And you can use the same thing.
Have everyone played the game called Descent on the PC back in the day?
So imagine a first-person shooter.
The game is filling up the entire screen.
But there is no up.
There's no gravity.
Whenever you want, you can turn yourself this way.
And you just stay that way until you want to turn yourself again.
By the way, the controls work just pretty much the same way as a first-person shooter.
That was Descent.
And the whole pleasure of that game, you'll be doing this all the friggin' time.
I'm going down this straight corridor.
And I never just go down the straight corridor.
I go down the straight corridor, and I turn [INAUDIBLE]
Inception.
[LAUGHTER]
Yeah!
That sort of thing.
If you ever get a chance to play the game Prey, which is probably like five dogs on like a pre-owned nowadays.
You have that same sort of thing.
And Portal has a lot of that kind of loss of control, physical sensation.
They're trying to get that.
But obviously real physical experiences that are half an inch.
They're talking about three-legged races or that horrible game where you spin around on a bat with your head.
[LAUGHTER]
And [?
Ellings ?]
can be used to describe any one of those things.
I like to use it as describing the sort of fun of the loss of control.
But that's not necessarily the case.
Just the sheer physicality of the game that's part of the experience in a way that digital chess doesn't really have any of that.
Even real chess [INAUDIBLE].
So actually I'm just going to write out those words for those folks who didn't get a chance to read through that page.
So [INAUDIBLE] is kind of like intercompetition skill.
I talked about [?
Ellings ?]
which is kind of jumping down through the last one of these such as-- am I spelling this right-- where just kind of physicality just kind of sucks.
[LAUGHTER]
Actually Jason, can you turn the light in the room on.
Its extremely dark.
Loss of control.
Other ones were--
[INAUDIBLE]
Javier, what was that again?
Chance
Chance, luck, not skilled.
You guys know the phrase [INAUDIBLE]?
The die had been cast
Yes.
Yes, the die has been cast right before Caesar [INAUDIBLE] and says I now am able to attack someone.
I couldn't remember who.
But Javier needs dice basically
Actually--
[UNINTELLIGIBLE]
Say what?
He matched on the [UNINTELLIGIBLE]
Matched?
Matched on [UNINTELLIGIBLE]
Yeah.
Although I've heard two translations of The Die Has Been Cast as a phrase actually.
Because on one hand, you can think of it as the die has been cast.
So we'll see where fortune takes us, and there's nothing we can do wrong.
But I've also heard it described as the die as in the thing that you put molten metal into, as in cast.
And so whatever we were making is now set.
And they had that same idea, but because we had because one way that you can think of is this is not auto control.
And the other way you can think of this is and now this is set in stone.
Which are two very different connotations, but both saying we don't really have control of the situation
Yassa means thrown
Thrown away, yeah.
Yeah.
But the English phrase, [INAUDIBLE]
Are there words in English that [UNINTELLIGIBLE] root to mean random?
In French the word for random is aleatoire.
Come straight from [UNINTELLIGIBLE]
Sure.
Yeah
Which is undoubtably why he chose that word I just checked, he is French
Obviously that's why Caillois used it.
But yeah, I can't think of any English words that use that as a root, although there might be some.
I know there's a game committee.
And I think they actually make strategic games.
And there's one more which is--
Mimicry
Which is kind of odd because that's in English, not in French.
Mimicry's mimicry.
It's actually in English, so role playing, dress up, let's pretend.
Simulations sometimes?
You know, it might actually fall into that.
So you can imagine in tactical simulation, for instance, being very much about competition and skills.
But then if it's a tactical simulation about Russian [UNINTELLIGIBLE], something like that, then [UNINTELLIGIBLE] territory.
Do you-- It's part of the pleasure of believing that you are something else, as opposed to I'm just guy playing [INAUDIBLE] as opposed to I am this 17th century scholar [INAUDIBLE].
And making a game about that.
So these are the adjectives that he talks about to describe games, but in the bit where we actually ask you to read, it's more about why.
These words give you a clue to think about why are people playing games.
These are types of fun, much in a way that gets touched on by the MDA paper that we had you read last week.
What are these aesthetics that people are going for?
Are they going for the thrill of competition or the thrill of luck?
Are they going for physical motion and how that feels, or are they going for the feeling of being someone else?
And it's not-- Again, these are not mutually exclusive terms.
These are things that you could have one game that has all of this.
You could have one game that has two in some strong competition and the rest are all minimized.
That's fine.
But understanding that this is why people are enjoying the game more, and more importantly this is what people are enjoying playing with each other for.
One nice thing about this is that fodder for [UNINTELLIGIBLE] and for mimicry, it's really, really difficult to think about those two in particular without thinking about the players.
Competition sucks if there's no one else you're competing with.
Sure playing against computers nowadays, but traditionally we couldn't.
At least you were playing against the designer, and especially when it came puzzles.
It's really, really annoying to pretend to be somebody else when there's no audience, even if that other person is someone pretending along with you.
[INAUDIBLE].
Now, luck-- you can continue to play a game flipping a coin.
But even when it comes to luck, and it comes to experiences of physical motion, these things tend to be measurable [INAUDIBLE].
And that's why these four particular divisions that [INAUDIBLE] picked out are really, really good for [INAUDIBLE] social [INAUDIBLE].
A game where is there a name for this game where you put a bat on the ground and you go around?
Bat?
Bat irritating
What is he talking about?
Wait, wait, and someone gets fixed and then?
So there's a type of relay race where you pick up a baseball bat, and you put your head on it.
You go around it three times, and then you just run back to your team, carry the bat, and try to hand the bat off to somebody else on your team.
Usually ending up hitting someone
I've heard it called Dizzy Bats
Yeah
Dizzy Bats
Dizzy Bats
Dizzy Bats.
Yeah.
So that game is a lot more fun when there's somebody else around.
[LAUGHTER]
I do that by myself
You start handing it off to yourself
[INAUDIBLE]
Bats flying around
And even with games of chance, I mean sure, if you cook the games of chance by itself, but for the most part most of my enjoyment of the game of chance is usually with a group of four people, actually, either playing poker or playing something like mahjong.
So these are all very support [INAUDIBLE].
If you think about when someone's going to come down and sit in front of your game, and what sort of frame of mind they're going to be in to play your game?
What sort of frame of mind do you want to put them in in order to enjoy the games?
Think about why they might want to hang out with other people.
Because [INAUDIBLE].
So if there's this other person they play with, that you may or may not know, but try to think about how you're going to construct enough clues to sort of get them to enjoy it.
So the game is about pretending to be someone else.
I'm going to make a game where I am playing a negotiator in a hostage situation, for instance.
[INAUDIBLE], awesome.
I'm going to be the negotiator.
You all are going to be the guy taking hostages.
And then you want to play that out, and harm them.
Friday we had the game Crunch, where people were playing CEOs, I think.
I'm hoping [INAUDIBLE] that you will already hate [INAUDIBLE]
That's awesome
[INAUDIBLE] less corrupt.
And you think about how does the game, even from the rules, even from the design of the box, get you into the mood of thinking, OK, whoever I am in real life, I'm going to take on this persona.
And I'm going to enjoy taking on this persona for a while
Yeah, but then you're really not [INAUDIBLE] because they have this game's kind of built in where you actually hide cards on yourself
Did you guys catch anyone doing that in the act?
I got caught
No, I caught him
I feel like arguing about that's part of the rules.
He already got it.
He had already hidden them away, and then the cards were sought way afterwards
And even arguing over whether something was illegal or not, even when there's evidence that it is, no one caught you doing it.
It's appropriate for this theme of being a CEO of a failing bank.
It's like, well, so we have all of these CDOs.
Yeah, we purchased them at some point, but nobody caught us while we were doing it.
So now you guys have to help us fix it.
That's what the game's about.
Was there a game that was particularly competitive that people played on Friday?
That game was pretty good
Which one?
[INAUDIBLE]
There was a competitive moment in Saboteur when Alec, revealed himself to be a saboteur and ruined our chance to find the gold
Saboteur is what I would describe as cutthroat.
It's a game that's deliberately about winning by other people losing
It made me upset with Alec
It made me cry
And John Nash, actually, [INAUDIBLE].
Oh.
Because he did spend some time here at MIT, possibly when visiting, but he designed one game called Hex, which is ostensibly designed based on the washroom tiles of one of the Building 39 or Building 37 bathrooms.
But anyway, that game and another game called Screw Your Buddies are all cutthroat games.
The only way to get them to win this game is by screwing over the other person playing.
Screw Your Buddies in particular is just like boiled down diplomacy.
The only way you're going to win this game is to betray someone else
Yes
Let's see.
Games of chance.
What was kind of chancy on Friday?
And you-- [INTERPOSING VOICES]
Think or go?
Yeah
That's 17.
[INAUDIBLE] 4:00 AM you've got--
Every threat was on the board.
Two people go in.
17 drops.
Oh my God
It's a press-your-luck game, right?
Do you dare?
Do you think you can?
I enjoy that game a lot.
[INAUDIBLE].
I don't think that we had any influence specifically.
It's hard to get really physical in card games
Maybe in that that game, Falling.
Yeah.
That was a pit
Pit actually, you're right, two weeks ago, some of the games that were played were really kind of kinetic.
So even with card games, you already have very good examples of that.
I want you to keep in mind because today we're actually going to start the process of getting new folks to think about what you're going to do as social assignment and break you up into teams.
Before we continue , Jason, do you have anything to raise regarding [INAUDIBLE]?
No.
Anybody else?
I'm still confused about the whole Eskimo part
Agreed.
[INTERPOSING VOICES]
Eskimos
Take a look at the French.
[INAUDIBLE]
[INAUDIBLE] and what took universities, some are a good friend of ours did.
I mean we might know someone who knows him.
I'll find out tonight.
[INAUDIBLE] is not that old.
So maybe--
We to just have him come over over, chill at MIT for a little bit.
We could talk to him about the Eskimos.
[INTERPOSING VOICES]
He is dead, but I might know some people who knew him.
Just be a necromancer, that's
All right.
So this is what we're going to do.
We're basically going to do brainstorming today.
The basic idea is that we're going to have a bunch of index cards and one job to make it easier on the job of the secretary is that you'd actually be reading off something written.
So especially while other people are throwing out their ideas, and you get something, and that person's still talking.
You might as well just write down your idea.
So we'll have a whole bunch of cards.
I'd also like you to start off by writing your name right on top.
And the reason for this is because I don't think every single person in this room knows each other's names yet.
As we do this process, it's more that you remember oh yeah, it's that guy who came up with that idea.
Oh, and that person's name is so-and-so
That's what you want?
To have all the ideas from each person on one card, or have one idea per card?
All on one
What if the professors took notes and we could just bounce out ideas?
Well that's what the tape recorder is for.
But the thing about writing your ideas down is that someone can look at the card and say, yeah, that guy.
I want to follow up with this guy.
Because we're not actually going to form the teams for you by the end of this class.
The idea is so that after this brainstorm, you've got a pool of ideas of which you guys can make games.
Actually, frankly, we don't care if two teams ended up making pretty much the same game, out of the same concept.
Because I know that two different teams of two different sets of people are going end up making different games anyway, even if you start from the same.
That's not a requirement.
You folks can make-- you can take any set of ideas from the book today.
Obviously, if you're the one who pitch an idea and somebody else thinks, hey, that was actually pretty cool.
Take that opportunity to talk to the person and [UNINTELLIGIBLE PHRASE].
That pretty much is going to happen between classes.
So by Wednesday, I want you all to come in.
This is basically one of the ideas of the homework, is to tell me which team you're on, who's on your team, and what's the general one line idea that you're working with.
Now, the idea that they're giving us is what you started with.
It may not be what you end with.
That's fine.
I'm very comfortable that ideas change while you guys work.
I think that's fine.
The teams shouldn't change.
So you're working with these three other people, or a team of a four.
Then that should be intact for the rest of this-- for this assignment.
OK, so teams of four.
Now, one more clarification about the project.
The card game, all the cards need to be identically sized.
We do like cards that are going to be legible, but you don't need to spend a whole lot of time making them pretty.
I will just appreciate it that by the time you actually hand in your cards, you have them written in ink.
Pencil is great when you're prototyping because you can erase [UNINTELLIGIBLE PHRASE].
Ink is great for creating the [UNINTELLIGIBLE PHRASE].
Your game can be playable by two, three, or four players.
It does not need to cater to any grouping other than what you decide.
So if you're saying, this is a two-player game, we will test it with two people.
If you're saying this is two to four, we'll test it with some number with however many people we manage to rustle up to test.
If you say it could be two or four, but not three, that's fine.
You think I kid, but there are some games that work really, really well with even numbers.
[INTERPOSING VOICES]
Keep it four and under.
Or at least make it possible for us to grade it with a maximum of four people.
Because if you give us a game that requires five people, we have to find five people and get them together in order to create your [UNINTELLIGIBLE].
I think there was one more requirement for how long the game could take
I think it's 5 to 10 minutes
It was short
That's really short
That's super short
Falling is really short
Yeah
Crunch is not.
[INTERPOSING VOICES]
So when you're brainstorming for game ideas, how much do we [UNINTELLIGIBLE PHRASE] come to up with in our brainstorm?
It's a cool idea.
It's an amusing space
I will say that if the character's the thing that's encouraging you to [UNINTELLIGIBLE PHRASE].
I have this cool game that comes to mind.
Then that's the thing that you're throwing out.
If instead, I want to make a game about fish.
I have no game mechanics in mind, but I think fish are awesome.
Then, OK. That's the game you're doing now.
Maybe that's the first bad idea [UNINTELLIGIBLE PHRASE]
If we have like several game ideas, should we take several index cards and write one per?
Actually, no I would rather see them all on one if you can fit them.
And if you run out of space, flip them over.
Again, the idea of the index cards is not just to record the ideas down, because what I am going to do is I'm going to try to type them all out, so that you can [UNINTELLIGIBLE PHRASE].
But also to find you folks again.
So I said, yeah that was that guy who had this idea about fish.
Who was that again?
So I guess I will leave you.
[INTERPOSING VOICES]
Want me to write them down?
I think that [UNINTELLIGIBLE PHRASE]
I mean, you're not a facilitator?
Yeah, I'm facilitating
I like clerk better
[UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
But I am actually wonder whether we should define this into two groups
We get to cross-pollinate.
[INTERPOSING VOICES]
Let's work as a whole class first.
I know there were some folks who actually started off with ideas already on the forum.
So why don't those folks throw your ideas first because you already mentioned in the forum
Yeah, I wasn't part of that group in the forum
Well, to start off, Jeremy and I have been sort of discussing the idea, you want to go ahead?
Sure
Two ideas, I guess
But yeah, [UNINTELLIGIBLE] titles.
So the basic concept of the game in which players are forced by some game mechanic to refer to each other by name constantly throughout the game, and at a very rapid pace.
Their name is generated by the game and accumulate increasingly longer [UNINTELLIGIBLE] this game as they succeed.
So you would start.
You would be the lord of something, and then you could accumulate extra title, son of this.
Like bringer of flame.
And you could just like have all these modifiers and just kind of accumulate and accumulate until your title becomes ridiculous.
And then, have people start screwing up as they say your name.
And it'd just be hilarious
OK, I'm just going to write down like, memory jogs
So like procedurally generated names, titles
Name game
Name game
Yes
You said you had a second one?
Yeah, can I go with the second one?
OK, so I had this idea of sort of like a survival horror game, kind of inspired by falling.
Like the premise is, you have a group of like player characters that are charging through a building, and you can have like a dealer that is putting down hallways as you're going through.
And occasionally threats come up and you have to react really quick and be like, it's a zombie.
We have to go upstairs.
And thrown down a card to do that.
Then you have to keep doing this until you get to the end of it and people can die, and get eaten, and stuff
That's good
I like that a lot
Kind of like cooperative falling off?
Yeah
Should we be writing his ideas down?
No, I'm writing mine down and then
Cool
Write down your own ideas
So I have an idea.
I made this up a year ago when I took Game Design.
Or not Game Design, Intro to Video Game Studies.
Actually, it's like majority rule.
So it's kind of like crunch where you can just hide cars and stuff.
But if you get caught, then you get called out.
And in order to be penalized for it, you have to be-- a majority rule has to be by every one playing.
And so you can actually bribe people to vote for you.
It's more of like a-- yeah, and then the way you get points is, it depends on how we'd [UNINTELLIGIBLE] it
I'm just going to go around this way
OK.
So I have an idea of a card game based on like having success in life.
So the beginning of the game, all the players are randomly assigned different goals in order, which are like wealth, and adventure, and love, and things like this.
And then all the cards represent different activities that take up a different amount of time.
So you all these activities you can keep in front of you.
And then, however much free time you have, you draw that many cards looking for new opportunities.
So it's like you take opportunities and leave opportunities and things
I could write out time and opportunities.
Let's see.
Who's next?
I have a very mechanically oriented idea.
Essentially I'm stealing and trying to improve the drop mechanic from Magic the Gathering booster draft.
For those who don't know, the idea is that you're trying to collect a set.
And so let's do it in terms of regular card games.
You're trying to collect a straight flush as long as possible, as high valued as possible.
But other people may be trying to collect that same suit.
So you want to collect a different suit from other people.
And you do this by looking at the cards in your hand, picking one of them, passing the rest to the person on your right.
and then, continuing to do this, and then going back in the other direction with a new hand.
And then in the same direction again with a third hand.
And the object is collect a big, straight flush.
Also, guess what the person to your right was collecting, and guess what the person to your left was collecting.
So you're trying to signal the other people what suits are open and collect the suit which is open
Let's see.
Who's next
So it's going to be there are different types of cards, like action cards, like pick up cards or lose cards.
But then [UNINTELLIGIBLE] Stock pile from your friend's hand to the discard pile.
You can play a different card in a different order so the rule always changes.
Kind of mess other people up or help yourself out
I will just write that the rules are constantly changing
Yeah, but you get-- so there's actually an order to it.
So you can choose.
So if it's going to be your turn and it says pick a free card, you can put down, like from a friend's hand or lose three cards [UNINTELLIGIBLE].
Makes a lot of sense in my mind
So rules changing with cards
So one idea I have is a game about character growth through narrative arc.
So the start of the game you're given a character, and the idea is you want to follow a certain narrative arc so that they grow in terms of the hands of the game you're playing.
So if you start out with a cocky character, then you may want to actually lose a couple of the hands of the game, such that your character experiences failure and his character growth increases.
And so, basically the winner of the game is determined by who manages to get their character to grow the most emotionally
It's very deep
So I had this idea for a poker-like game, except it's played with more than one deck, but the normal rules still apply.
So if you get dealt five aces, you can't actually play that hand because that would be cheating.
So you have an incentive to try and like slip cards into your sleeve, slip back and forth.
So you're almost forced to cheat at the game to be able to actually play it
[UNINTELLIGIBLE PHRASE] if you get caught then, just pretty much you have to figure out how to get out
So the idea that I had was kind of like Bananagrams, but with a card game.
So you have cards with lines or preset lines or shapes, just something on the card.
And you have to try to connect them all in some way on your personal board, so that it forms a closed shape.
And once you're done, you have to take another from the pile and try to fix your shape so that it's still closed while trying to beat everyone else
So collecting cards, make shapes
So my idea was just using normal poker cards and doing a trading game.
Just like trading four kinds of pattern as four kind of stocks.
And you can randomly draw the card to be your own stock.
And everyday, you randomly draw a card to determine the day price.
And you can trade with bank or other players.
I think you can use 8 and 10's as the stocks and the J, Q, and the K as the special cards [UNINTELLIGIBLE]
[UNINTELLIGIBLE PHRASE]

So this was another procedurally generated idea.
The game generates for you encounters.
And it's supposed to be ridiculous and somewhat humorous.
You'd have a base card, like a bear.
And then there are various modifiers that you would draw to create that encounter.
Or it could be like a ninja zombie pair in space wrapped in bacon.
And then those would all give different attributes to that encounter.
And then you'd have to find a way to beat it.
And your characters might also maybe be procedurally generated in similarly ridiculous ways and you'd have to counter it with-- basically, I like procedurally generated ideas.
Cards games that are never played the same way twice
I need something [UNINTELLIGIBLE]
With laser beams
Of course
Of course you need laser beams
So my idea was a musical RPG in which players' cards and musical notes.
And you build chords.
For example, to raise your character's strength, you might make an augmented chord, et cetera.
And then you can chain chord progression to attack players.
Diminished your life
I just dominated your [UNINTELLIGIBLE]
I got obsessed with drafting.
I have another drafting game.
Uses the same drafting mechanic except that you're drafting tiles.
You play the tiles in front of you and they make a machine.
Drafting the same color of machine bit or same type of machine bit, and then laying it in consecutive blocks.
Multiple [UNINTELLIGIBLE] makes them better, and then your machines like attack other people.
Or they can do crazy shit.
But the challenge being that you have to drop the piece that fits into your machine
That can do crazy shit
That can do crazy shit.
It has to be pronounced like crazy.
[INTERPOSING VOICES]
So my idea to have like a monarchy type thing.
So it starts out everyone is like heir to the throne.
Everyone wants to become the king, or queen, I guess.
And the point is you have to get the king to really like you, or to assassinate him.
But if you want to assassinate him, you have to kill off all the other players first.
So it's kind of a like a betrayal and deciet game
So a betrayal
Yep.
So another game, it's about reducing your opponents mobility by placing cards that represent walls, and placing other cards to rotate various walls.
And your opponent is doing the same thing.
So you could sort of envision the Tron Light Cycle game without any hint of reflex removed completely
That would be really cool
So it's all just a strategic placement of walls to reduce your opponent's mobility
Yeah, I think you can just make a Tetris box and then every time you select one give to the other player [UNINTELLIGIBLE PHRASE]
So Tetris blocks.
Got it.
OK?
This is kind of like Pokemon, except now you raise zombies or you catch zombies.
So you have different types of zombies.
Like maybe you take some from Left 4 Dead universe or something, or just Resident Evil or something, and you can do like versus other people or something
I thought of having cards-- I don't know whether they would be like words or a few words together.
But you would go around and each round, the goal would be to build a story.
So each person would play a card face down, or something.
So it'd be a mechanic like Apples to Apples, except you're progressively building a story and if people like your addition to the story
So Apples to Apples [UNINTELLIGIBLE]
Yeah
[UNINTELLIGIBLE]
Sure.
Moving away from drafting.
OK, so I have another game idea called, Are you a Cylon?
Essentially--
[UNINTELLIGIBLE]?
Are you a Cyclon?
Are you a Cylon.
And essentially broad brush strokes steals the skill resolution that came from Battlestar Galactica the board game, and then strips it down to secret Cylons sabotaging people.
And then people accusing them.
And then everybody crying
So I really like Harvest Moon, so I was thinking that maybe there could be Harvest Moon competitive card game.
And I guess you have a certain amount of stamina each turn and cards are actions.
And whoever has the biggest harvest at the end of the game would win.
I don't know how many turns there would be
So stamina in farming
So sort of building on the monarchy of the trail idea, this is a game about the heir to the throne.
The king is dying in x turns.
You and the other players are going to compete to prove who is the correct heir to the throne.
And you can do so through combat directly with other players, assassinations, or political machinations, such as like accusing other players of being base born
Assassinations
Actually, I just have the paper when you throw out your idea, also throw out-- also mention your name because I'm still learning everyone's name
OK, so I'm Ian.
So I had this idea for a game called Admiral Snack Bar.
It's a competitive game where each player is building up their own sort of like snack shop, and they make a certain product.
And the other players need that product in some way to enhance their own things.
But you can see secretly poison parts of your product, so that when they purchase them, they'll like get messed up in some way.
So it's sort of like a trade-off and you can make alliances and stuff like that
So it's sort of like a trap?
It's a trap
There's the tagline right there
It's a snack
So I'm thinking of a game, I guess to have like the total wars mixed with Nazi Germany.
So the point of the game is you want to snatch up territories.
And you can do this by maybe attacking other players.
Well, in particular it's just like cards.
And you can do this by attacking other players who may be making alliances, which you can easily betray.
Or maybe have like all the cards in a pile and every turn you can take one and try it.
And you get points by getting certain combinations of [UNINTELLIGIBLE].
More like Risk right there as well
[UNINTELLIGIBLE PHRASE]
I'm Brian.
So I was thinking of like a trick taking game.
But the idea is there's four players, and you're trying to get the second most tricks each round.
And then there are cards that allow you to swap out other cards at the end of a round and other things that are tricky
So hit the second-best trick?
Or second most tricks or something like that.
But yeah
So I was thinking of sort of a-- put succinctly-- Scrabble meets chicken
Hm?
What?
You have a bunch of people.
And they play cards in the middle and you have to make words out.
And if you make longer words, you get more points and so forth.
So maybe your letter is more rare, it gets more points.
But the catch is that if the word is longer, you can't gain as much point.
The point gain is--
Diminishing
It's diminished more.
It's proportional to how long the word is you make
I'm Bato.
And I wanted to combine Mario Karts and Chrononauts.
So the idea of you're racing through time.
Somebody is the main guy that's trying to get to someplace.
Everyone else is trying to stop him.
And your abilities or weapons only work in certain time periods and stuff like that.
And so you have the time travel and the items from Chrononauts and the racing and attacking from Mario Kart
Go, Michelle
My name is Michelle.
So I was envisioning something like Ricochet Robots but where the two people-- basically, the point is that everyone has a set of cards that have pathways or grids or walls that you can bounce off of.
And you build a grid with your opponent.
And there's one robot piece that you have to try to bounce off the walls to reach some specific goal.
You can screw up the other player
Build walls and bounce robots around.
Good
I'm Patrick
Go ahead.
Oh, Patrick, then Jeremy
So this is a game where the players are owners of businesses.
And depending on the nature of the business, they might have mutual dependencies that would be beneficial for people to get more money.
But then as the market goes bad, they'll have to play cards to cut off the ties between companies.
And so I call it Objectivism, The Game
That was [UNINTELLIGIBLE]
OK. You?
So just a slight variation on the procedure [UNINTELLIGIBLE] idea but now I'm visioning it possibly as a sort of Falling real-time type deal.
So every turn, you're getting dealt cards.
And you have cards that can modify the dealing pattern so they can discard other players' cards or can get them to be skipped, so on and so forth.
And you also have cards that can add various modifiers to your beast.
And the goal of the game is that by time the deck has been entirely dealt, you have the most bad-ass creature that you've created and that-- you'd have to figure out-- and there might be an optimum way to rearrange things, so as you're getting cards, you have to try to figure out how to rearrange them.
And at the same time, other players could be discarding stuff.
And you have to deal with all these things at the same time.
And then at the end of the game, you have to freeze and see who has the best one.
And it could be a really quick game and very hilarious
OK, so [UNINTELLIGIBLE PHRASE]
So kind of like building a creature, plus Uno
OK, this game is Monkey or Shakespeare?
So there are several different piles of cards.
Each of them has words on them in Shakespearean English.
Some of them are going to be played by players.
Some of them are going to be like random number generators, essentially.
The players that are playing as covert Shakespeares select cards and then place them face down in a common pool.
Then the top cards from all the other decks get placed in the pool.
And then everything gets shuffled up.
And then each of these different piles is going to be a different color.
And so you will spell out sentences in the color progressively, based on the card draws.
And the object of the game for the other players who aren't hidden Shakespeares is to try to work out who's the Shakespeare, who's just trying to spell the prearranged sentence beforehand is, and who are just monkey piles.
All right.
So it's detect the random number generator
Bato, again.
I want to take that tanks versus robot game that we played before on the paper and make it into a card game between players.
And the tanks can level up.
And you could change your specific strategy and that kind of stuff
[UNINTELLIGIBLE]
Would the robots still be deterministic from something?
Up in the air
Let's see.
Who from this side hasn't gone yet?
I don't think you--
I did.
I had my convoluted idea
I think everyone's gone.
[INTERPOSING VOICES]
So one mechanic is any game that has a high amount of trading of cards.
You can imagine anything from a Go Fish type thing to something more complicated.
However, you don't mechanically enforce the traded cards.
So there is basically lying in that [UNINTELLIGIBLE].
Also, playing off of the robot tank idea, a game where, basically, your strategy is mitigated by the cards you play, rather than the actual characteristics.
So the cards played dictate how another deck or another mechanic is run, rather than you directly interfacing with that
Robots
[UNINTELLIGIBLE] robots with mechanical modifications.
[UNINTELLIGIBLE]
I am Justin.
So I just have an idea about your asylum game that seems very interesting.
[UNINTELLIGIBLE PHRASE].
Maybe it will help if you can play two characters instead of one.
So if you have four people, you have eight characters determined [UNINTELLIGIBLE PHRASE].
So two player asylum.
Two characters
Sure.
Andrew.
So you could have something that's sort of an MIT case.
So you're an MIT student.
And you're going through pizzas, you're doing energy drinks to help you along through the night, social events.
You can hang out at Harvard for a night.
[UNINTELLIGIBLE PHRASE]
That would be a debug
Yeah, exactly.
Sorry, you'd force other people to hang out at Harvard for a night
Anything else?
[UNINTELLIGIBLE PHRASE]
Deion.
This is MIT or Harvard.
So let's say you're in a design class.
And you're trying to build a project.
So there's some students that are MIT.
Most of them are.
But then there's one or two Harvard students.
And so everyone's trying to build this project.
But the Harvard student is messing it up.
Or maybe it could be Harvard, MIT, or Caltech.
So the other student, that's not MIT, is trying to screw up your project.
And so by the end of the game, you want to figure out who that person is.
Or maybe you take turns making additions to it.
Something like that
So this came up only a few seconds ago in my brain.
The setting I imagine for this might be something like a dark elf realm underground.
And there's various houses or various families of dark elves.
And there's one that is not controlled by players.
So I guess it's deterministic [UNINTELLIGIBLE] set rules.
And players every turn can place written cards to influence the behavior of this non-player entity the next turn.
And basically, players are vying for influence over this non-player entity, which would then do benevolent or malevolent things towards players.
And it might be only semi-obvious, depending on how cards are played who exactly took certain actions to get this master house to give favor or remove favor from one player to another.
And so it would be a game where you're trying to beat other players.
At the same time, you're trying to figure out who's on your side, who's not on your side, and forging alliances and betraying each other
Second one is a guy at a party.
He's trying to get the girl at the end of the night
[UNINTELLIGIBLE]
Oh, frat boys
Is that a card game?
You could screw other people over by going up to them and bringing up embarrassing stories.
Or you could be really nice and try to meet the girl's friends.
It's all based off an AI kind of [UNINTELLIGIBLE PHRASE]
And hook up
I'm Ian.
You have a game where the objective is all the players are actually trying to build a physical house of cards out of the cards.
But each card has an in-game mechanical effect as well.
And then there's possibly the actual structure determines something as well.
You'd have to do it with very thick cards
So-- [INTERPOSING VOICES]
Jenga with the cards
Jenga cards
There would be a mechanic-- actually, my idea was building a house, but not with a house of cards.
But that's a better idea.
Adding a mechanic who'd be like you can play an action card that's like, a storm destroys the west wall of your opponent's house or something.
In your case, [UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
Jason.
So you're on a student project or corporate project or some team doomed to fail.
So your objective is to make you look as good as you can, whether by making the project better, or by making sure other people are responsible for it being worse.
[INTERPOSING VOICES]
A terrible microcosm for the first assignment.
[INTERPOSING VOICES]
Don't fail too badly
Who is next?
Brian
Brian
With all this talk of drafting, I started thinking about auction games.
So an auction game where there's some different points.
And maybe if you collect certain types of points, it's good.
And then also some of the cards, you can add a few mechanics for yourself.
So you can bid for a card that later you can play.
And have everyone else bid.
And then you have to go bid after them, or something like that.
But everyone has the same amount of points at the beginning [UNINTELLIGIBLE PHRASE]
I'm Jason, by the way.
Going back to my guys at a party one, you could work in a mechanic where people could bring in real stories and real traits that they have [UNINTELLIGIBLE].
[INTERPOSING VOICES]
But you could honestly lie and make it worse for someone else, but then you lose points for lying.
[UNINTELLIGIBLE PHRASE] dishonest guy.
[INTERPOSING VOICES]
So again, it's like an RPG card game, where you each have your set of things you can do and equipment you have.
And then there's an encounter deck.
And these keep getting flipped over.
And then people have to react to this.
But it's not quite cooperative because at the end, there's something that only one person can get.
So maybe they have to duke it out at the end if there's more than one person left
Sounds like [UNINTELLIGIBLE]
Yeah, kind of
So bringing up auction mechanics, there's a bidding mechanic that is interesting from a theoretical standpoint that I never see implemented, where if you have a thing that's being bidded for-- or you have everyone assign something that they get from some pile.
And then if someone feels that they are unhappy with what they have, they trade it with someone else's, and either put some amount of money on the other item or some decremental type object on their item.
The bidding system comes out of the pirates trying to split up booty and making everyone happy
So bidding by reducing other people's bids-- is that a good description?
It's a bidding system in which everyone has something.
And then you trade and add value until everyone feels it's equal
[INAUDIBLE] something.
Let's see.
Yeah?
So my idea was you have two teams, maybe four players.
It's like an RPG.
And so before you start, you have to draft the equipment and spells you can do.
And so then you have team battles against each other.
And the way you determine what happens is maybe a War-like system, where each one has a deck which is shuffled.
Take the top card of the deck.
The highest card wins.
And so that determines damage and such done
[UNINTELLIGIBLE]
I like it.
So searching off the RPG idea but adding on a randomly generated fort or castle that one player has to defend and everyone else is trying to attack into.
And he has to set up his defenses around.
But he's also controlling a character.
And so he can move his guy around the board to choose one of the guys to defend against.
So essentially an RPG with randomly generated board as well
Defend your castle?
Yeah, yeah
Castle defense
I like it
So it's sort of building off of the idea of pirate's booty.
I like the idea that you have a bunch of items within the game.
The value for them is chosen randomly so that it's unknown what the value of the items are.
So you just have the items and various players can make various moves like choose to appraise an item by paying some gold.
And the idea is you want to try to get the item that's worth the most amount of money during any round to get the most amount of gold
Get the maximum amount of [UNINTELLIGIBLE].
I'm running out of space, so I better be ending this soon
I'm out
I'm Jason.
[INTERPOSING VOICES]
I kind of like the bidding [UNINTELLIGIBLE].
So you build competing factories with different products.
So you want to bid on different items, but for some people, it might be necessary to have a certain [UNINTELLIGIBLE].
Things are going to have different values for different people.
And you could also get high-end stuff or low-end stuff.
And the randomness is going come in where if you have low-end stuff, a user could hate your product.
And then you lose money and stuff like that
Michelle?
In a bidding sort of game, you could be bidding for space, let's say, space to use the Hubble Telescope.
But before you can do this, you have to pass a couple of hurdles, like taking J-lab, or--
Bidding
Use the Hubble.
That's awesome
One of the first hurdles
I wrote it down as bidding [UNINTELLIGIBLE PHRASE]
Tower Defense, the card game.
Interpret as you will
Tower defense
That's a lot of math.
[INTERPOSING VOICES]
Slight variation-- Dungeon keeper, the card game.
So I'd imagine that there would be several lanes, down which enemies would come.
And you would collaborate with other players to create buildings on the other end of those lanes that would then generate either traps along the way or your own units that would knock down the lanes, taking out the enemies.
And every turn, you run the game yourself.
So you run your opponent and it automatically advances and sends stuff at you that's drawn from decks.
And you have to be able to time stuff right and all that, not to get your core destroyed, basically
[UNINTELLIGIBLE]
I'm just thinking of the idea of Tower Defense, the card game.
Combine that with Falling, the idea that turns are happening at the same time.
So you have one player who's playing the zombies invading.
And he's just basically going, one, two, three, four.
That's like a zombie advance, four steps, or something, by placing cards down.
And then the other player is trying to place defenses just as fast, but has to discard cards as well.
So it's really a game of how well you are able to handle cards
[UNINTELLIGIBLE]
I had another idea in the vein of Falling.
But it would be like Russian Roulette.
Everyone's trying to have everyone else die and not them.
And then you can play excuse cards, be like, oh, I have to go to the bathroom.
I'll be right back.
Or calling somebody else out and calling them the chicken or something.
You play with macho points
I think that's Russian Roulette with excuses.
[INTERPOSING VOICES]
So players take turns being a prospective high schooler who wants to go to college.
And the other players are, therefore, admissions officers.
And as a student, you're trying to get into the college that gives you the most--
Financial aid?
Most swag, anything.
[INTERPOSING VOICES]
And as admissions officers, you're trying to give them the least
So admissions [UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
Admissions officer is easy.
You're just just like, no, no, no.
You got into community college
I've come up with a game mechanic for his idea.
So you could have bluffing where you say, oh, I'm going to the bathroom.
And then someone else could call you out on it.
And you either have the bathroom card and you're actually going to the bathroom or its an excuse.
So that's a mechanic [UNINTELLIGIBLE]
That's [UNINTELLIGIBLE]
Yeah, the full bladder card
Isn't there a simple mechanic-- you basically place cards down to try to create a path from some predetermined point to another but in that way, you can block other players by placing cards
So create impasse and blocking them.
[INTERPOSING VOICES]
I think you're done
I think we're done
[UNINTELLIGIBLE PHRASE].
All right
Just use the bottom
I guess I could write down to the bottom, but it would be a pain for folks to look at.
So these are just memory jogs of what we've got up here.
None of these ideas need to be taken up wholesale for your original idea for the team.
[UNINTELLIGIBLE PHRASE] things that folks [UNINTELLIGIBLE].
I would actually like folks to grab one of the thumbtacks over there.
Once you've written down your ideas on the card and pinned them up on the wall.
They don't have to be closely packed.
In fact, spread them out a little bit, so that people can look at them one at a time.
And the rest of the class basically is just time for you guys to talk.
You [UNINTELLIGIBLE PHRASE] get to know each other's games better.
And start discussing about projects that you're working on.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information, see the course materials on the MIT OpenCourseWare website
Grabbing a bunch of index cards and scribbling things down
Yes
Pretty much how I get by.
Another group?
So our group is [UNINTELLIGIBLE], [UNINTELLIGIBLE], and Jason.
We don't have a rep [UNINTELLIGIBLE].
But [UNINTELLIGIBLE].
[LAUGHTER] We're doing like a real time, supernatural horror/cooperative game
OK. Oh, like real time [UNINTELLIGIBLE] theme?
Yeah
I'm excited.
I'm excited
One, two, three, four.
All right, who else?
it's me, it's Tanya, Patrick, and Justin who are here
[UNINTELLIGIBLE].
OK. You guys have kind of a [UNINTELLIGIBLE]
And how do you describe this game?
It's building a family tree game
Geneology path building
Yeah.
It's a path building game involving family members
Consider it geneology path building
It's kind of like a Family Feud game.
Except, it's a little bit stranger
So I think Jay and Mia
Yeah.
[UNINTELLIGIBLE PHRASE]
And Calvin and Jason, you've been [UNINTELLIGIBLE] in that
Oh.
[UNINTELLIGIBLE] and things
[UNINTELLIGIBLE]
So right now we have a-- we don't have a [UNINTELLIGIBLE].
The idea we have is pretty much you want to get a certain sets of cards.
And we get sets by drawing cards or trading with other people or-- but it's also you can take cards if you want, but if you get caught, then it's up to the other players whether you get punished or not
OK, if you get caught then-- OK.
So [UNINTELLIGIBLE].
That sounds good.
So it sounds like we have four teams.
[UNINTELLIGIBLE PHRASE] So today I'm planning on, for the most part, just going through the lecture stuff that I had planned for today and for last Wednesday.
And if there's any time left, we could finish up with [UNINTELLIGIBLE].
And that's what I'd like to do.
But if there's only like an hour left, then I think I'd probably rather you play one more game.
As this is a useful exercise, but playing these games are just as useful and frankly a better design.
All right.
Let's see.
So I think I'll start with Wednesday's stuff first, which is the last [COLERA] reading for the entire semester.
I did, in fact, actually look a little bit into the weird eskimo stuff from Monday, and one thing, I don't actually think he was speaking figuratively.
I think he was actually describing something that he had done some anthropoligical research on-- this is a ritual that a very, very [UNINTELLIGIBLE], just a fine group of eskimos do.
It reminds me a lot of the sort of recitations that you get that sometimes has a morbid quality, sometimes they don't.
But this reminds you of either historical events, or important things to do.
Like the easiest one that I keep in my brain is righty-tighty, lefty-loosey.
That teaches you something of how screws work.
But then they're a little bit like ring around the rosy, with a pocket full of poseys, ashes, ashes, we all fall down.
But it's actually a commemorative little poem about the black death.
That's what that song is about.
So you have these little things.
That particular example that he was describing was more about this recitation that you sort of remember the procedure in which things are supposed to be done.
But they're [UNINTELLIGIBLE], they're sort of [UNINTELLIGIBLE] reenactment about what you do in order to I guess [UNINTELLIGIBLE PHRASE] or something like that to basically perform taxodermy on an animal.
One of those ways where a playful experience was pretty much it's been used as a learning tool.
And that kind of gets back to something I was mentioning much earlier in this course.
In many ways, learning how to play a game is literally very exotic.
You can't figure out how to play the game unless you have figured out all the rules.
And if the rules happen to have some sort of correlation to something real world, you might end up learning something about that real world.
Whether you transfer to back to actually something real world is kind of actually the topic of what Wednesday's reading about.
[UNINTELLIGIBLE] talks about mimicry and things being kind of taken over by [UNINTELLIGIBLE PHRASE].
Now he uses these grand, sweeping statements which I think sound a lot more modest when they're in the original language, and what to translate [UNINTELLIGIBLE] sound like you're saying, and this huge wave of competitive games took over witch doctors.
But I think what he's referring to, and if we notice the readings for that chapter three and chapter eight.
So there's a whole bunch of stuff that's missing, and we kind of wanted to spare you from the bad translation.
But the idea is that there is a lot of stuff that basically mimicry, the old process of putting on masks, putting on costumes, and being someone else has great societal power.
This had great societal power.
And this is more of an anthropological thing.
But the idea being you have these people-- who are effectively leaders of groups-- shamans, witch doctors, fortune tellers, maybe just elders.
And they would have costumes, they'd have masks that not only gave them station, what when I wear this robe I am now in charge of this tribe, but also turned them into somebody else.
Now, if you look at ancient accounts of the Egyptian pharoahs, for instance, in order to actually give up their own name and then they take on a new name.
And a lot of them actually just become Pharaoh in the same way that a lot of Roman emperors just become Caesar.
You can think of Caesear as just being king.
So it becomes a title, but you also become a person.
A lot of the-- I'm trying to remember-- Persian, I think, emperors become Darius or Cyrus.
Kind of just one word.
This is one name that was given to the king.
That's who you become.
And that kind of combined with aesthetic solemn rituals.
A lot of these are not what you would recognize as games right now.
But in many ways they sort of tap into the same sort of pleasures of sort of critical and losing control.
Once in a while, you don't have to be sort of all inhibited and obeying laws.
You can just cut loose.
And in superstitious tribes, superstitition will be the excuse.
But you don't actually necessarily need that reason.
And I think one thing that [UNINTELLIGIBLE] sort of describes in chapter eight is how-- we kind of get that today too.
He was describing stuff like identification, for instance, when somebody who could be you wins something for no particular good reason, against incredible odds.
And is there some sort of mechanism for you to feel I am now happy partly for this person's success, therefore, I'm a winner.
Then you get to have that experience of ecstasy.
You get to that news.
A couple of videos.
Some of them it's tongue and cheek, but hopefully that you-- it will kind of illustrate my point as well.
So was anyone here in Boston in 2004?
You were in Boston in 2004?
What happened in Boston in 2004?
With the Red Sox?
Yes.
The Red Sox won after, what, 86 years of coming real close and never actually winning a World Series.
And I'm going to-- This is a neat little video that I have.
I just kind of then just use this as a back drop.
I think this is right around Fenwick Hall.
I'm not so sure it's [UNINTELLIGIBLE].
Sorry, the wireless in this particular corner of the office can be bad.
This is-- actually let me skip a little bit.
So for those of you who don't know the story, actually how many of you don't know what happened in 2004 around here?
Well I don't know exactly what [UNINTELLIGIBLE PHRASE]
So basically what happened around here was the Red Sox were down 3, 0 against the Yankees.
And managed to come back from them.
And then they beat the Cardinals and they won the World Series.
Now, this is, of course, after 86 years, again.
I mentioned really people just not winning at all.
But coming really close.
Each time they get really, really close and then they'd magnificently flub it.
[UNINTELLIGIBLE PHRASE].
So this earlier video here is what Boston looked like when it was down 3, 0.
[UNINTELLIGIBLE PHRASE] The funniest thing that every time the Red Sox actually won it was like an awakening.
So they weren't at Fenway.
When all of these celebrations that-- can you see once the buffering stops.
All the celebration that's going on is occurring miles and miles, probably 100 miles away from where the game's actually happening with the Cardinals.
And of course, everyone was watching it on TV, everyone's identifying with the Red Sox.
And right now you are sort of seeing like, as far as I can tell probably the Northeastern students who were around Fenway.
But in here, the vehicles driving by and [UNINTELLIGIBLE PHRASE].
Everybody was listening on the radio, you can see the cars driving by.
Everybody knows immediately.
Everyone in Boston immediately knew it [UNINTELLIGIBLE PHRASE].
If they weren't watching the game, they would have heard it anyway.
Because they have no affirmative claims.
The only way that you would cheer the Red Sox is to say Mackie sucked, which is kind of sad actually.
For those of you who weren't around, it was a really, really awesome night.
There was, of course, a little bit of violence.
Things got a little bit out-- a bunch of [UNINTELLIGIBLE PHRASE].
But it wasn't like a huge riot or anything.
For the most part it was pretty much this.
A lot of people with a lot of the hands in the air.
No cars being turned over or anything like that.
But it was being-- But these are the kind of aesthetic movements that [?
Colera ?]
talks about in his write-up.
Yup?
I have a video on my phone in Barcelona right after they won the semi-finals over the summer.
And I had a video of the streets at 11 PM.
I could show you [UNINTELLIGIBLE].
I have a video of the streets at 12:30, so half past midnight.
And it's the exact same scene.
And the exact same people are still in the street with their shirts off, still twirling their shirts around
It was probably more rowdy I would imagine
Oh, it was [UNINTELLIGIBLE] rowdy [UNINTELLIGIBLE].
This is America, they don't know how to be rowdy in [UNINTELLIGIBLE]
I think for every three people there was a cop.
You don't see that in the video.
But there were a lot of cops out there that night just to make sure that [UNINTELLIGIBLE PHRASE].
But I think Barcelona also would have been partying
Yeah
So the idea-- of course, the idea of a sports team is something which you identify with because associated with the city that you live in.
They win.
I didn't do anything, but because I because I'm a supporter of sports team, I get to identify myself with them.
And [?
Colera ?]
makes the assertion that this is important because if you look at the forces of human chance, of there, which I think gives the various athletes the opportunities, as well as the physical ability, to actually perform well.
The likelihood of any of the people actually celebrating, actually being able to perform that same feat is extremely low.
In fact, the likelihood of even someone who has the ability and the opportunities of actually winning is pretty much random and the odds are against you.
But because you identify so strongly with a group of people who can win, you are allowed to sort of enjoy that victory, even though you weren't the one who actually got that victory.
And it gets stronger actually for some games-- well for some kind of games where you actually did have something to do with the victory, for instance, talent shows.
Especially the modern talent shows.
Maybe you've already seen this.
[BEGIN VIDEO PLAYBACK]
So [UNINTELLIGIBLE] the Outward Bound are on about next week's show.
They're going to-- remember that when you look at things like Britain's Got Talent, like American Idol, all of these things are popularity contests.
You have judges, but the judges don't actually determine very much who's actually going to win.
The only people who actually determine who's going to win are popular vote.
And that vote is happening through short messages basically, which people are being charged for.
The basic idea is that because you might have had voted for someone, you now identify with that person who might win.
Whether that person wins or not is effectively random for other purposes.
Now sure, that some of them have talent, but later on-- but when you look at who actually tends to progress in these competitions, I think some of the results are really, really random.
Like you have some people who move onto the next stage because they are just strange.
And you just want to see them next week because they may be entertaining, not necessarily because they're talented.
So this is [UNINTELLIGIBLE] example.
[UNINTELLIGIBLE PHRASE] [WATCHING VIDEO]
Yes, but their [UNINTELLIGIBLE]
Dancing
That's an obvious one.
Singing for everyone
[UNINTELLIGIBLE PHRASE]
[INTERPOSING VOICES]
Getting married
[UNINTELLIGIBLE]
Cypress
Yeah, what do they do?
I should have looked up what their particular is, but they can all do different things.
So the first one is short message vote in
[UNINTELLIGIBLE PHRASE]
Slick-backed hair
I notice the camera moves at this point.
[LAUGHTER]
They just like the looks of their [UNINTELLIGIBLE]
[UNINTELLIGIBLE]
[UNINTELLIGIBLE PHRASE]
They're obviously [UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
It looked like a dance thing.
And if you look at the other celebration's like the Susan Boyle video, where I think it got pretty viral.
So I'm assuming a lot of you have seen it.
It's one of those things where even if you don't necessarily that you're rooting for a person.
It's so easy for you to be convinced that you're actually rooting for this person as soon as you have a reason to.
So--
It's like oh wow, that guy is from Texas
Hey, [UNINTELLIGIBLE].
It's good for you.
You don't really need that much of a reason to identify with somebody.
And that's where you get sort of still see some of that power of the vertigo, basically.
It's not the vertigo of actually playing the game, the vertigo of watching a game, but it's still part of the entire game experience.
Britain's Got Talent, American Idol, everything like that wouldn't have the power if it didn't have the hundreds and hundreds of screaming fans voting in and having their faces changing on camera, and applauding and screaming their heads off.
And of course, that's the identification part of it.
But even when you're talking about sort of just basic [UNINTELLIGIBLE PHRASE].
Here's a shrine festival in Japan.
And basically what they're doing is just getting started.
Everyone's gotten assembled.
All of the folks here who are wearing basically the same colored outfit are basically from the same town.
Including that logo.
And what you've got is basically like a little house for a deity.
They're both carried on the shoulders.
And those three up there look really precarious, but I think they're having a good time.
That's a completely different talent [UNINTELLIGIBLE].
But you know, this is not unlike what you'd see in a sporting event.
It's kind of getting [UNINTELLIGIBLE PHRASE]
Isn't the town versus town like competition?
It's not really so much a competition.
But I think there's a certain amount that you have to represent
That show existed in France, it's called [UNINTELLIGIBLE PHRASE], and they literally get towns in the countryside and have them compete in various competitions.
So like each town, for example, designates the strongest person to have an arm wrestling match
Sort of like champions.
You send your champions
You send your champions.
Then you have like a horseback thing.
And I think like other weird events.
And it's pretty popular.
1,000 people/person town versus [UNINTELLIGIBLE PHRASE]
I think you get that in the Scottish Highland games as well.
Basically it's sending people to throw giant pieces of lumber as far as they can.
But [UNINTELLIGIBLE PHRASE] competition.
This is not [UNINTELLIGIBLE].
This is a lot closer to kind of the earlier kind of situation.
Because they're not vying to win anything.
They are really just trying to all get into it together.
So if you think of it as what kind of thing.
The aim of the game is don't drop the damn-- [LAUGHTER] It's quite simple.
Get thrown-- in fact, it's actually a fair amount of effort because I think they walk a couple of miles.
I'm not quite sure what the traditions are on where it starts and where it ends.
But it's like a huge street parade.
And the whole idea is let's get everybody involved.
The people are packed really close with each other, so it's not like you can actually move one foot without colliding into somebody else unless you're all moving in unison.
And can you keep in sync.
And all the shouting, all the whistles, and all the chance actually just helped that entire process move along.
It continues today, not just in sports, not just in televised shows, but in street fairs and [UNINTELLIGIBLE PHRASE] that's great.
And those folks really look like they're having fun.
So that's kind of what we were [UNINTELLIGIBLE PHRASE].
And the version of this, the version of what [?
Colera ?]
describes as since your ability to succeed is so random, especially if you think of it outside of the game context, but more in the silo context-- your ability to be a great athlete, your ability to be a great politician, your ability to make a lot of money, just to succeed in life is so random.
But people kind of don't necessarily want to acknowledge that it's random because you have rules, because you have sort of rules set in to promote merit to sort of equalize systems.
Is this the equilaterian [UNINTELLIGIBLE], which I think actually he meant.
I think the original word was egalitarian.
While there are a lot of processes trying to stabilize, in the end you can't really ignore the forces of one's birth.
At least that's the way he sees it.
When you get competitions that are effectively just completely random, which means anybody can win, people really do understand and sort of ascribe to that kind of power.
They accept.
The likelihood of me winning the lottery is about as good as the likelihood that I'm going to succeed in life.
A lot of it is based on who was I born to, and which year, and which region are we born in?
What's happening in the world as I-- at least that's the way of how he puts it.
So he has a really hilarious video, by the way, [UNINTELLIGIBLE].
This is an ad.
Let me get--
[UNINTELLIGIBLE]
I guess so.
Sorry about the-- It looks damn pretty
[UNINTELLIGIBLE] [BEGIN VIDEO PLAYBACK] -Mos of us when we buy a lottery scratch-off ticket, we're thinking this is my day, right?
OK, if that's the case, then a 19-year-old Mills college student is super lucky tonight.
-Louis Jay became an instant millionaire with the scratch of a ticket, and he told Rob Robert all about it today.
-It was on his birthday when Towson University student, Louis Jay, decided to buy a scratch-off lottery ticket.
-I actually won $20 on the scratch-off the prior week.
-Louis Jay went to this giant [UNINTELLIGIBLE] Mills and used the $20 he won on a scratch-off ticket and bought another one.
As he began scratching off the numbers, Louis couldn't believe his eyes.
-I'm at the second to the last line and it's not going too well.
And I was starting to see six.
I was like oh, it matched up.
All right, I won $30 or something, why not.
I saw a million.
I started double-checking, triple-checking everything.
Make sure all the numbers matches and everything.
I was like oh, my God.
-And when it came time to scratch off the last number, his mother told him-- -He was scratching on every one.
And when it was the next to the last one I said, don't scratch it from right to left, do it from left to right.
[LAUGHTER] -And there it was.
Louis had won himself one million dollars.
-This little dollar sign bag right here that I won on, I'm actually got number six for my lucky number right here.
And down here it matches the six with the prize for a million dollars.
-He's a hardworking young man, and he has fun and he likes the lottery.
He buys a ticket on his 19th birthday, and lo and behold, he wins a million dollars.
I don't know if he told you the other part of the story.
He bought a ticket on his 18th birthday, his first one, and he won $100.
So I don't know what's going on his 20th birthday.
Everyone in the family about Louis's good fortune.
-I'm happy for him
Isn't it a story that they're trying to tell [UNINTELLIGIBLE PHRASE].
-And yes, he does have plans for the money.
-I'm putting some money away for my brother and sister to finish schooling.
They go to [UNINTELLIGIBLE] school.
I'm going to put some money away for myself, help my parents pay off some bills so we could all-- We always said what we'd do if we won the lottery, we'd go to England, visit my father's relatives and stuff.
We're going to go there next summer hopefully.
Just going to do a lot of things we always said what if we won the lottery, and now we did.
-Rob Robert, Linda.
BBAL TV, 11 News.
[END VIDEO PLAYBACK]
So this is just a story that they're trying to tell.
It's like this is hard-- he's a hard-working young man, as if that had anything to do with his [UNINTELLIGIBLE]
He deserved it.
[UNINTELLIGIBLE PHRASE].
[LAUGHTER]
Behind that is that well, and and if you're a hard-working person as well, and if you bought a lottery ticket, you could win.
That's the story that they're trying to tell.
[INTERPOSING VOICES]
[UNINTELLIGIBLE] is like perfect time for opportunity.
Like with the lotto idea, what if a really, really unlikeable, despicable character wins the lottery, what's the news reports going to be like?
She never worked a day in her life
But I think you can still turn that in the way of this is not a person whose odds in life were already in favor.
And everyone can sort of identify with that.
It's like yeah, things aren't quite working my way, but you could be a lottery winner tomorrow.
That's the story that they are telling.
And of course, the idea being what the lottery director was saying, yeah, remember, he didn't just buy one ticket.
He bought one ticket the year before, he might buy another ticket.
They're trying to encourage you to buy more tickets.
That's the story.
The story is buy more tickets, which oddly enough is actually the only way to increase your chances of winning the lottery.
Buy more tickets.
Pity that doesn't work for like past deals and things
[UNINTELLIGIBLE PHRASE]
And reapply for more [UNINTELLIGIBLE] classes [UNINTELLIGIBLE PHRASE].
But the basic idea that they're trying to because the forces of life are so random, these kinds of like totally random, high stakes games have incredible power.
And not just because of the possibility of winning.
But because it resonates very, very strongly with the idea that you could win.
And if you identify with somebody else, even better because then you don't even have to win.
That person could win.
And in fact, increases the range of different winners that you can identify with.
Help people sort of get that thrill of winning without necessarily even having competed.
So that's kind of the power that we have in chance.
I haven't actually talked about, well so how do we actually use chance in games yet, which is what [UNINTELLIGIBLE PHRASE].
But I think in their book, they describe-- the reason why I like challenges for you to design is that they're very concrete about well if you're stuck and you don't know what to do with a game, have you considered bam, bam, bam, bam, bam, bam.
They're very, very systematic about that.
You know, have you considered doing options?
Have you thought-- That's not chance.
That's the second chapter.
The first chapter was have you considered using dice, using cards, using spinners.
I don't remember what other randomizing systems they had.
Computer or random number generated [UNINTELLIGIBLE].
Are you using it to equalize different skill levels, or are you just trying to make it so that there's sometimes predictability, that sometimes solveability in the system.
There are all these different ways that you can use randomness.
But don't forget how powerful randomness is.
It's one of those things where you normally assume that you don't actually [UNINTELLIGIBLE] in the game.
And you don't want him to [UNINTELLIGIBLE].
First of all, as the designer, randomness makes it hard for you to predict what's going to happen, and as a result, it becomes difficult to design.
But the power of the this I could be a winner for reasons that are entirely out of my control is actually more powerful than a lot of folks what will naturally assume right off the bat.
So I'm not going to go too much into the various options that Brenda and Ian described as a challenge because you can look those up.
But do look those up, especially if I'm having difficulty in your game.
If you're having difficulty, oh this game is too predictable, and you can see that there is the optimal strategy.
Well then you can think about maybe writing those might be a good [UNINTELLIGIBLE].
Or maybe I want to be able to play this with people who have never played this game before I had met them, and still have a fighting chance.
That's another good opportunity to win [UNINTELLIGIBLE].
But know what problem you're trying to solve before you're throwing something away.
Because sometimes you can have a lot of fun with games that are effectively just like fun butteries.
You know, oh, I draw a card.
Munchkin is kind of like this actually.
You draw a card.
Oh, look at, funny.
And you read the text.
That was silly.
And now they've added a Twitter version of Munchkin where every hour you introduce a new rule.
And it's like the player whose name starts with the letter M gets double the hit point for something.
But totally random.
And that can be kind of fun, but that's really short-lived fun.
That's one of the things where once you look through all of the cards-- the game's pretty much done for you.
Even if you haven't even finished playing the game, the game might end for yourself.
So try not to go down that track.
There was a group last year that did a game where pretty much you just drew silly things to do out of a hat.
And, yeah, it was fun until everyone got through the hat, and then once everyone knew pretty much the range of the little silly things, that's it, game was over.
There was no replay.
So [UNINTELLIGIBLE PHRASE].
"The [?
Side Note ?]
About Replayability," that's actually a really excellent essay on I think gamasutra about how replayability isn't actually a well-defined word.
Nobody really knows what that word means, but everyone uses it.
So if you're interested, check it out.
I haven't actually had a chance to form a real opinion on it yet.
So by the way, a lot of the roots of the kinds of randomness things that we use, like dice and cards, have some sort of root in fortune telling, which connects it back to kind of the ritualistic aspects of games again.
The idea of you can tell whether you're going to be a success in life by your success in this game.
And the game might be draw a card from this deck and hope it's good.
Which parallels what a lot of people think like it.
If I roll the dice, and if I'm lucky, things are going to work out.
But there's a couple of interesting things about how strategy and skills start to work with things of randomness.
For instance, there is one argument that [?
Colera ?]
makes that the reason why you get things like chance and skill games taking over mimicry and vertigo experiences is because we started developing systems of enumeration.
Started being able to count, we started being able to rank things to tell whether something was actually better than other, and some sort of empirical numerical way.
So if you think about it from that sense, then actually when you look at skill and when you look at chance, they're actually kind of on both on the same side.
They're sort of dealing with number systems one way or another.
If you had a system that has a lot of chance, any number could come up.
Any card could come up.
Any dice roll having some set of probability.
What skill is dealing with what could happen, having a plan, having a strategy.
Maybe that element of chance isn't a random die roll.
Maybe a person that you've never met before that you're playing against, and you don't know how good they are, they don't know what the strategies are, they don't know how they play.
They're effectively a die roll, and your skill is kind of or the element of skill in the game are all about how do you deal with all the probabilities that have happened.
So one of the things that [?
Colera ?]
talked about is a lot of the competitive games kind things actually come up from tests-- tests of physical skill, tests of mental skill.
Tests the ability to be able to take an administrative position and in society.
That sort of thing.
But your ability to actually do well on those tests also kind of depends on your luck of having the ability in the first place.
It's like did you get the right of education?
Did you study the right questions?
Do you happen to have really good genes so that you can jump really high?
That sort of thing.
So the idea of whether it is actually just luck, if you would talk about a modern game, you can talk about skill of someone being able to play chess.
Or you can talk about the luck of somebody having had the chance to play chess many, many, many times before they're playing this one particular game.
Because not everybody in the world has that option.
And that's kind of the case that [?
Colera ?]
is trying to make.
When you're bringing a new person to a game, they haven't had the luck of playing your particular game.
I'm going to say the good fortune of playing your particular game, whereas you have had that.
So if you wanted to think of it as that, as well they're bad luck, now how do we use rules, either rules of chance or rules of skill to be able to give them a fighting chance, a fighting edge.
Probably more rules of chance.
So a there's a couple of different things.
One thing that I do want to bring up is you [UNINTELLIGIBLE PHRASE] information [UNINTELLIGIBLE PHRASE].
Sorry.
I could probably just draw it.
But I just want to make sure I get it right.
[UNINTELLIGIBLE PHRASE]
Can we also [UNINTELLIGIBLE] shift it over to the big screen?
It's really very [UNINTELLIGIBLE PHRASE].
So it's incredibly blurry-- OK, all right.
So this is the basic idea.
Claude Shannon, back in the '60s, I believe-- I think [UNINTELLIGIBLE PHRASE], basically came up with a model of how communication works.
How information gets from a sender to a receiver.
How does this have anything to do with games?
I will get to that quickly.
But let me just talk about models first.
So the sender had some sort of like message, right?
And they're going to send it end of this conversation I believe-- yeah.
This is transmission.
I'm going to shorten it and say X-mission.
So a sender has a message.
They want to set up-- they want to translate it, and then it needs to be received [UNINTELLIGIBLE] by the receiver.
In between the two, there is some sort of channel.
So if you think of this as I got a computer and my message is an email.
I'm going to transmit it through the internet.
And then it's going to be received by some sort of other mail server and then it's going to the other.
This is the important part of this model is that this is where noise gets in the channel.
The reason why this model is useful for thinking about games is you can actually think of the sender as the game.
This this game space.
And the message could be something, it could be valuable information about the current state of the game in order for you to make a good decision.
In poker, it will be, well everybody's hands.
That's the message.
That's valuable information.
If you knew what everybody had in your hand, and if you knew everything in the deck in the order in which the deck is shuffled, you will be able to make a great decision about whether to raise your bid on the floor.
And so you have rules that sort of encode what that message is going to be, and you have rules that-- you have strategies to help you figure out what to do with that, with the receiver kind of becomes and you think of it as [UNINTELLIGIBLE PHRASE] as a player's position, actually.
Or the player's strategy if you would think about it over a long-term strategy.
The trick is noise.
And noise is actually something that you get to control [UNINTELLIGIBLE PHRASE].
What is hidden?
What is obfiscated?
What can be deliberately introduced to confuse you about what the game state actually is?
Noise can be random, or noise can be just plain old you don't know.
Just deleting information out of the [UNINTELLIGIBLE].
So for instance, you guys know the rules of Texas Bulldog?
There is a set of cards in the middle called the river.
I could be wrong, so if you really know the rules [UNINTELLIGIBLE].
And these cards are dealt pretty early in the game.
They're not dealt right at the beginning, but the--
[UNINTELLIGIBLE] at the beginning?
Not all at once [UNINTELLIGIBLE]
The first three
The flop, yeah.
And then [UNINTELLIGIBLE]
Oh, the flop and the river, right.
Which is the one [UNINTELLIGIBLE PHRASE]?
[UNINTELLIGIBLE]
The flop is data?
[INTERPOSING VOICES]
Everyone gets dealt the two cards, and then you turn the flop over, which [INTERPOSING VOICES].
And then the river is the [UNINTELLIGIBLE]
The river's the big one
And betting zero, yeah
Well no, technically when you deal you put one card face down then you put the flop [UNINTELLIGIBLE]
That part doesn't have anything to do [UNINTELLIGIBLE]
Yeah, but I don't know what--
Yeah.
But the cards would get dealt out.
It's pretty much an unknown.
It is random.
But on and above this, if you just put five cards in whatever side and then just dealt them out.
It's a fixed quantity.
But you still don't know what the heck it is.
Someone else could have picked those cards and still don't know what it is.
So they're effectively a random thing.
They're noise.
They're a bit of information we don't know.
And in Texas Hold'em, a bit of information that everybody does know, so that's great.
The information is kind of equally shared among everybody except for the one card.
They'd be fine.
So-- --when you think about how you can even things out for the players, or how you can make the challenge a little bit higher for decent players, you can think about, well how much information am I currently giving them about what a game state is?
And maybe I can, I'll just [UNINTELLIGIBLE] them.
Not necessarily lie, just hide.
Just deny them information that is necessarily [UNINTELLIGIBLE].
It's also a good way to sort of think of well maybe if you're a little bit behind and you need a little help, then what are you going to do to give them a fighting [UNINTELLIGIBLE] channel.
Let them have the next card that's coming up.
Because we're talking about card games here.
Let them re-roll the next die roll.
[UNINTELLIGIBLE PHRASE].
This model, about like thinking what sort of information are you giving your player can also sort of be useful in thinking whether they're giving too much information strictly from a decision-making point of view.
It's like I'm throwing all this game state information, but really, all the player needs to know is half of that in order to make a good decision.
You might actually want to consider reducing that somehow.
Let me think of an example.
I think Settlers of Catan that should be a pretty good example of that.
You can see the entire board.
Everyone can see the entire board at any given time.
And everyone's got cards in their hands.
You don't know what other people have in their cards.
But for the most part, when you're trying to make a decision about what to do with your cards, you're really just concerned about your cards.
As time goes on, you pretty much know what other people have in their hand if you've been paying attention, because people are picking up resources fairly openly.
But you don't really care.
And that's one nice thing about Settlers is that they're hiding this information from you, even though it is information that need to be hidden.
Because in order to make a good decision in that game, you do not need to know that information.
So they're introducting noise, they're deliberately hiding information from you so that you can just focus on the two or three decisions that are actually worth making on your turn.
And that's one reason why Settlers games actually go pretty fast because people aren't paralyzed by too much information.
If your game's getting too long.
The game's supposed to be a half an hour and it's taking an hour long, think about the information that you're giving your player.
All of it might be relevant to the game state, but not all of it might be relevant to what the players need to know right now in order to make their next move.
And if you find a way to control that, sometimes just a clever color coding.
Making your card in such a way that it's really, really obvious that there's a certain part of the card that you need to look at at certain times, and you kind of ignore it outside on a [UNINTELLIGIBLE] call other times, and that might help.
But often what I find is in a lot of games like [UNINTELLIGIBLE] card games, for instance, board games, they would use a lot of icons to describe what the value of something is, or what kind of effect it has.
And the reason why they use icons instead of using words is because you don't have translate icons.
And they want their games to sell in Germany, in Spain, in England, and US, and Greece, and Italy, and so if you made cards, you couldn't use those same cards in every one of of those countries, so you use icons.
The only problem is that now you have to learn what every one of those icons mean.
If it's just a stack of coins, oh 5 coins means 5?
OK, all right, I can probably figure that out.
That's fine.
But this is a picture of a hammer, what does a hammer do?
I'll have to check in the reference manual, which happens to be in six different languages.
And that's one of the barriers that a lot of folks have to [UNINTELLIGIBLE] in card games and board games.
It's not necessarily that the games are all that complicated.
It's just that you have to actually have translation that has to be done because they're not really built for one language.
So I probably have more to say about information, but I'm going to save it till next class.
Any questions so far?
Everything that I'm sort of directing at you right now are pretty much just different ways to look at the game [UNINTELLIGIBLE].
It's a card game, but try to see it from different ways.
Not just how do I get this one game mechanic to work.
But what are people going to be getting out of this game.
When they're going to be drawing this big card, how are they going to be feeling?
Are they going to be getting excited?
Apprehensive?
Are they going to be counting on the next card to win them the game, or is this just going to be giving them options, and do they know what options they have?
So I'm just going to do a complete transition at this point.
So new topic.
One thing I want to talk about, which is not actually covered in any of the readings is the body motion think plate.
I think it's probably used as, sort of-- Jason and I use this story quite a bit.
We haven't really designed it all that well.
But if you think of the definition of what a good game could be, [UNINTELLIGIBLE] offers, a good game is a series of interesting choices.
Our answer is all interesting decisions.
So what's an interesting decision?
One argument is that an interesting decision is one where you're using mechanics to affect the game state.
And you're going to get some information, you're going to use that mechanic and effect the game state.
The game state, the game's going to respond in some way.
And how that decision you make corresponds to the result that you get back from the game state as could be interesting.
If there's no correspondence at all, if it's completely random or, even worse, completely predictable, than these are not interesting options that you've got.
And [UNINTELLIGIBLE] huh.
That result logically made sense, but I didn't expect that it was going to do it, or that turned out better than I expected.
Now it becomes really interesting.
And if it's just a-- it could be anything.
It could be awesome.
I could win this game with this next-- oh, look I won.
That's not interesting either.
So if that's an interesting decision, [UNINTELLIGIBLE] calls it a meaningful decision, then the converse will be a meaningless decision.
So like a game like what I just described.
A decision where making a choice doesn't actually affect your outcome.
Because [UNINTELLIGIBLE PHRASE] makes card flip, or you had to do the card flip anyway.
You can't choose.
Then that's not a meaningful decision.
Or you make a decision and have no change in the game state at all.
That's not interesting either.
So what you want in your game is something where you've got a system that's nonlinear, that's one that's still deterministic.
All of your rules will result in some sort of outcome.
But nonlinear enough that you can't really sort of anticipate the entire possibility space when you're making that decision.
If you're thinking of games, that's a huge possibility space, then you, as the designer, your job is to try to make them most frazzled areas of possibility space interesting and fun.
So actually do folks know the phrase possibility space.
Yeah, and if you think of a game as just a series of terms or series of decisions.
Then possibility space is basically every single result that would come out from every single legal decision in your game.
Some of them are going to make sense.
Like--
[UNINTELLIGIBLE]
In process, which now I'm getting back to the [UNINTELLIGIBLE PHRASE].
Someone's got zero and zero, and it's my turn.
Putting in actually what makes sense.
Putting it anywhere else doesn't make any sense.
So I would argue that putting an X anywhere really isn't-- well it is part of the possibility space, because I could put it here, here, here, here or here.
You as a game designer, your job is to make sure that the things that people won't naturally do are interesting.
You do not have necessarily any responsibility for somebody who's just trying to play your game in a way that's entirely not what you planned.
Say somebody plays one of these card games and says, well I'm just going to play all my card face up.
Those specific rules that says that I have to bite my hand, but [UNINTELLIGIBLE PHRASE].
You don't really need to keep it for that.
Nice to have a rule that explains, OK, this is what you should hide your cards is crucial to your game.
But for the most part, when somebody plays your game in a way that doesn't logically make sense, you're not really obliged to make sure that they have fun.
On the other hand when people are making rational, if you are making rational decisions-- somebody puts an x here, and then the next move goes on and you design your game, then it's your job to make sure that OK, within the realm of the things that are legally good, and strategically make sense, we're going to try to make that game as interesting as possible.
That's your job.
So in order to sort of like help you figure out how to get that, the reading from [UNINTELLIGIBLE] today, introduce a few terms, if you've already taken game theory and economic class or a math course, you probably already know these in far more detail than you necessarily need it for the [UNINTELLIGIBLE] is useful.
But utility it's a very useful concept.
It's difficult to quantify sometimes, but you're talking about just like regular game design as opposed to the pure mathematic thought experiment.
But a basic idea is if utility is like how winning a move is.
Is the likelihood that something is going to actually call you a winner.
And in some ways, it's a measure of the games that you're [UNINTELLIGIBLE PHRASE].
So [UNINTELLIGIBLE PHRASE] example has this like ridiculous game where you guess whether somebody has heads or tails.
And the person's like literally just choosing whether it's going to be a heads or a tails, and you just guess.
The idea being if [UNINTELLIGIBLE PHRASE].
So this is the guesser, this is the flipper.
So if the guesser guesses right, the guesser gets a point.
And that's a choice of [UNINTELLIGIBLE PHRASE].
If the flipper chooses tails, the guesser chooses tails, the guesser will get one.
And if not, then it reverses.
So the flipper again gets the penny if the guesser gets it wrong.
So you can sort of tell that, just by looking at this, this is a ridiculous game.
It's like at best, your average over multiple games is just going to be zero.
On average it's just going to [UNINTELLIGIBLE PHRASE].
And you use this as example to describe how we can come up with a strategy where you're not basically doing the same thing all the time.
In this particular case, he's advocating that the best way to win this game is pretty much to break even in this game, and that's to just play it randomly.
Every single time just really caught up in it.
As soon as you get consistence, and either you'll get this for your flips, then you are now in a different package.
But what you're trying to do in-- but in this particular case, the likelihood of whether it is a heads or a tail, and you're actually doing it randomly, but if you throw a kink and say, well if it most comes up heads, instead of winning one penny, you win one million dollars or something.
There's certainly the probability that the flipper is going to choose heads, drops it, drops like rubber, right?
Not a wise strategy.
If a flipper chooses heads 50% of the time, then the guesser is going to be winning several million dollars pretty soon.
They'll be losing a couple of pennies once in a while, sure.
fine, I'll lose a couple of pennies, I'll just guess heads all the time.
And if I know that they are flipping heads 50% of the time, I'm going to be getting [UNINTELLIGIBLE PHRASE].
So in this particular case, the way how the flipper balances it out is basically to flip tails about one million to one.
I think it's more accurately it's like one million to two.
So it's like for every one million tail flips, I'm just going to throw in a head flip every once in a while just don't force the guesser to occasionally guess heads.
Because if the guesser knows the flipper will be crazy to guess heads all the time, and does tails all the time, then the guesser just guesses tails all the time, and after 50 games the guesser is ahead.
So the idea is every once in a while you throw in a head.
And it's roughly the opposite probability of the ability.
Approximately.
The numbers don't quite work out that easy.
But it's I guess in the scale that I'm using right now, one million to one.
And when you're talking about balancing a game, if you can't actually break down a decision in your game, say a particular decision that a player has got to fake given a certain hand or having a certain card.
And now they can make one or two or three decisions.
And knowing the card distribution well enough saying well, 30% of the time decision A is going to be good.
20% of the times decision B is going to be good.
And the other 50%, it's going to be decision C. Then a balanced game, at least one definition of it, is that the utility of all those decisions, the benefit that you're going to get out of that maybe is raw, winning points.
Victory points that you need to be able to get in the game.
Maybe some sort of cash.
The benefit of actually making that decision should probably be in reverse proportion of those numbers.
So the thing that you will pick 50% of the time, that you [UNINTELLIGIBLE PHRASE].
So say you've got decision A, B, C. And-- actually, I'm not sure if I need [UNINTELLIGIBLE PHRASE].
I'm actually really bad at this.
[UNINTELLIGIBLE PHRASE] C is 50% of the time C is good.
30% of the time A is good.
[UNINTELLIGIBLE PHRASE] If anyone knows how to do this better than me, [UNINTELLIGIBLE PHRASE]
What are you trying to do?
So the idea is that A should give you disproportion will be higher than it is.
Not disproportionally, but should give you proportionately higher than it did in decision C. So if C gives you--
[UNINTELLIGIBLE PHRASE]
Say what?
[UNINTELLIGIBLE PHRASE]
I'm just picking pure probability right now.
Assuming that you're just playing a game with this.
They're not playing as we presume [UNINTELLIGIBLE PHRASE].
So say doing decision C gives you five points
You should do it with 30 the numbers would work out better
Say what?
If you use 30 the numbers will work out better
OK. 30 coins for 50?
Yeah.
[INTERPOSING VOICES]
So if you choose B, then what sort of benefit should you be getting?
About 13?
Yeah
20
20
But B only works less of the time.
This is [UNINTELLIGIBLE PHRASE]
Oh, great
Oh.
[INTERPOSING VOICES]
It would be 48, right?
Yup
I would say 50?
I'm going with 50
I'm going with 50.
You want the multiplication in the probability [UNINTELLIGIBLE].
So when you multiply them [UNINTELLIGIBLE]
And what would A do?
[UNINTELLIGIBLE PHRASE]
And this makes it a situation where choosing A, B or C becomes fairly even.
You could choose A, and you could get and 20% of the time you've got a pretty good payout.
I'm really terrible at math, so I'm going to see if this is right.
But this makes it so that you don't have an immediate dominant strategy just like looking at your cards.
It's like well, obviously, I'm just going to do this all the time.
This is very, very, very basic.
And I crap at doing this other thing.
But once in a while, it's a good idea to actually just write out the numbers.
OK, I know that my opponent is always going to have five cards in hand.
And out of the five cards they're going to have some combination of cards that's going to be result in the probability of A, B, C affects [UNINTELLIGIBLE PHRASE].
So I have part A, I have part B, I have part C. I could play any one of those, and making that decision, you want to give them rewards that are in inverse proportion to this.
That's one way that you can balance a game.
It is very much a first stab.
This is how you mathematically take that first stab, and then you actually need to test it out, out of that.
You can't actually just assumed that that's going to work out.
But it can get you pretty close, which is kind of nice.
If you're getting a situation where OK, they'll still seeing a dominant strategy, but it's only a small number [UNINTELLIGIBLE PHRASE], then you don't [UNINTELLIGIBLE PHRASE].
Actually those can sometimes be the hardest kinds of games of balance because you keep making really, really small changes, and you know when you get them.
But still, [UNINTELLIGIBLE PHRASE].
That's really the only contact that I found utility to be a useful part of this [UNINTELLIGIBLE PHRASE].
It's an interesting concept to be able to talk about strategies, we talk about decisions making.
When it actually comes to now I'm going to sit down and design a game, this is the only time when I'm actually using [UNINTELLIGIBLE].
If you found better uses for utility, please by all means share it because I have not found any.
That's pretty much all.
Any questions?
The utility.
When we played the x game--
Which one?
The game where you can either click on tokens or roll the dice
[UNINTELLIGIBLE]
The one that [UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
Oh, yes, yes, yes, yes, yes, yes.
Yeah
[UNINTELLIGIBLE] if you viewed the kind of strategy developed and the game we built based on different dice and things, that was kind of the utility model.
That we're trying to maximize [UNINTELLIGIBLE].
And if you win you expect your utility [UNINTELLIGIBLE PHRASE].
And then like you can do a bunch of path [UNINTELLIGIBLE] to maximize these things [UNINTELLIGIBLE], which is-- I don't know
Yeah.
I mean the thing about it though, of course, if the method you do basically says well the way to maximize our expected value is just to do the same thing all the time, and then that particular game, that's a very standard strategy that tends to work.
Then you've got a game that's not interesting.
Then you've got a game where those decisions that you're asking someone to make don't actually matter because there's [UNINTELLIGIBLE].
So yeah, that game is a fine example of a game that you could use for work.
I'm not going to say the broken game because that was specifically designed for the [UNINTELLIGIBLE] that class act.
Here's this game, now I'm going [UNINTELLIGIBLE].
But yeah, that's a good game where the utility's pretty obvious, right?
How far are we going to get?
Well I could pick this token or I could roll the die, wait a minute, on average you all come out to be about the same.
So it doesn't matter anymore.
Why this decision is no longer interesting
[UNINTELLIGIBLE PHRASE] because the whole point was that you, the meaningful weave is that you're pulling at the variance table, right?
Your expected values are the same.
Both of the cases when you're doing just a kind of computation, right?
Yeah
Now I can [UNINTELLIGIBLE PHRASE].
Sometimes it's incredibly painful to not get the reward.
Sometimes you just want consistency.
So imagine in Settlers of Catan where you can go for a strategy where you have a diverse set of numbers that you can possibly rob.
And before you can just get up and [UNINTELLIGIBLE PHRASE].
Usually it's better to go for consistency, because you'll get robbed
You can use a large range of numbers
Yeah.
The chances you'll get a huge payoff off turn, and then a seven will be rolled before you get [UNINTELLIGIBLE PHRASE].
They would have been better off generating [UNINTELLIGIBLE].
Even if the expectation you generate [UNINTELLIGIBLE]
But in the back of your head you always know that you could go to the big payoff round.
And usually in a game--
What I'm saying is there's a meaningful difference between these things.
Even if your expected value/computation shows you that there [UNINTELLIGIBLE PHRASE] strategies [UNINTELLIGIBLE], the variance can really be different.
[INTERPOSING VOICES]
The decisions are very different.
If you're playing a three player version of Settlers you want to be equally very diverse.
Whereas you imagine if there were a version of Settlers where there were 10 players, you would want to like cluster all around a couple different numbers.
Because then it's unlikely that if everyone's diverse, everyone's not equal.
So you'd rather have a little bit of a [UNINTELLIGIBLE], you'd be like way ahead of everybody.
Than having just the [UNINTELLIGIBLE PHRASE]
Well basically, again, counting on the huge payout
Yeah.
But that comes out to like variance.
And where the expected value of the resources you get, this is the same as the expected value of like winning the game where you're really trying to maximize [UNINTELLIGIBLE PHRASE]
There was a really, really, really funny video that I didn't show you guys about how to win at a lottery.
And they actually did trot out various facts, basically saying that oh, some percent of the time two numbers-- this is one of those lotteries where you pick six numbers, and the number of numbers that correspond will get you a certain payout.
And it's like some percentage of the time you will get two numbers adjacent to each other, like 39 and 40 or something like that.
and there's like that's all, you should pick two numbers.
That makes no sense at all.
Because [UNINTELLIGIBLE PHRASE].
In that particular case, [UNINTELLIGIBLE PHRASE].
I think it was a good point to say, that you get to choose how much variance [UNINTELLIGIBLE PHRASE].
How much risk are you willing [UNINTELLIGIBLE PHRASE].
Because in a game like Settlers, the nice thing about it is if they're going for the big payout is it increases the risk and increases the payout [UNINTELLIGIBLE], or potential payout.
So that's what's nice about it.
That's a nice balancing act
I don't know if anyone's played-- it's not really a game, it's more of a math problem
I haven't played that one
[UNINTELLIGIBLE PHRASE]
It's pretty cool.
Can you put that up on the board maybe?
Yeah.
I can tell you right now.
It's first cube has three 5s, the three 2s
Oh, [UNINTELLIGIBLE PHRASE].
Three 5s and three 3s?
Three 2s
[UNINTELLIGIBLE PHRASE] One of them is 5, 4 and a 1
5, 4--
They all have expected value of 3 and 1/2.
same as the regular value.
Last one is one 6 and five 3s
One 6 and five 3s
They all have the expected value of 3 and 1/2.
Now, the game is first player chooses a die, and the second player chooses a die.
And then [UNINTELLIGIBLE PHRASE]
Just [UNINTELLIGIBLE PHRASE]
Yeah
And then do you re-roll with the game dice?
No.
The first player chooses the die, and the second player sees what the first person chose and then the second player chooses a die.
So truthfully, the [UNINTELLIGIBLE] take it reverse, and the [UNINTELLIGIBLE] second, even though they [UNINTELLIGIBLE PHRASE]
I see
If you're first, you want to [UNINTELLIGIBLE] the board, is that right?
But five 2s beats the 4s, but [UNINTELLIGIBLE PHRASE]
They can probably just use the 4s for [UNINTELLIGIBLE PHRASE]
It's the first player tries [UNINTELLIGIBLE]
I don't know exactly, but I'm pretty sure if you're first, [UNINTELLIGIBLE PHRASE]
Yeah
Yeah.
It's three 6s so dominated by the 4s, but it's like [UNINTELLIGIBLE PHRASE]
Yeah.
[UNINTELLIGIBLE PHRASE] The second always wins.
[UNINTELLIGIBLE PHRASE]
They can probabilistically win
So what you're describing is a strategy for the second player.
Basically, if the first player can [UNINTELLIGIBLE], you pick something, and on average you will win [UNINTELLIGIBLE PHRASE]
It would be more interesting if you did it in two rounds [UNINTELLIGIBLE PHRASE] everyone takes a die sequentially.
And then they roll to see who goes first.
Because then you're aiming to try to like hit the middle
Will you do that one more time?
So you randomly decide who goes first.
So the first person chooses a die, and then the next person chooses a die, then next person chooses die.
And they roll to determine the order of the actual game.
So what you're trying to roll for is to get in second place rather than first because the optimal strategy is the second choice
Oh that's bizzare
Yeah, but the problem is if you take 6 and 3, you also have a pretty high chance of getting the last one
[UNINTELLIGIBLE PHRASE]
Well yeah, [UNINTELLIGIBLE PHRASE].
[INTERPOSING VOICES]
Evolution is something which happens [UNINTELLIGIBLE] is well.
OK, you show that it's clearly a dominant strategy, but how you get to deciding what [UNINTELLIGIBLE] strategy to use gave meaningful information because [UNINTELLIGIBLE]
That's possible as well
Once it gets to that point, then you want to think about well maybe there's something you can do to your original game to make it easier for people to get to that second level of the game.
And you don't necessarily always have to, because there are going to be people who are just going to have fun at this point, to sort of like playing this game as opposed to figuring out who goes first.
But it'll be interesting if in the process of designing this game, you've got rules to actually let people decide who goes first.
And the people who haven't figured out the dominant strategy, this is where the game is at.
Just like the first player has an equal [UNINTELLIGIBLE] as the second player.
And then later on when people start to realize that there is actually a strategy, they got playing the other game of who goes first.
But if you design your game right, then you can make it all the same game.
It's just which set of rules that they're most concerned about and you could [UNINTELLIGIBLE].
It would be nice if your games could do that.
Let's see how [UNINTELLIGIBLE] they get
[UNINTELLIGIBLE] move away from game period [UNINTELLIGIBLE]?
Are you even going to mention Prisoner's Dillema because I think that's a great tool to try to look at dramatic tension and interesting discussions for players
That does happen often in strategy games
Yeah.
Well--
Not in a game where you're [INTERPOSING VOICES]
There are two arguments about that one
[UNINTELLIGIBLE PHRASE].
I guess we actually have human beings playing
So the Prisoner's Dillema as covered in the reading.
I don't know the exact numbers.
But basically you've got two people--
Well they didn't say exact numbers [UNINTELLIGIBLE PHRASE]
I think we have a chart
[UNINTELLIGIBLE] punishment [UNINTELLIGIBLE]
In fact, I may be thinking of [UNINTELLIGIBLE]
[UNINTELLIGIBLE PHRASE]
The one that I know is that-- let's see, I know defect is one of the cases.
Anybody remember what the term we use for cooperating?
[INTERPOSING VOICES]
Yeah.
[UNINTELLIGIBLE PHRASE] So if both people cooperate, they get something like what, five years in prison?
So it's like the town.
OK, we get two people who are involved in the same [UNINTELLIGIBLE] and they both get caught but then there's not enough evidence to throw both of them away for a real long time.
So the basic idea is each of them is separated, and then you give us information on the other guy.
Then we will reduce your sentence, and that other guy is going to get hammered with a sentence but we'll give you a get out of jail free card.
But if you don't tell us anything, if you cooperate with your partner, and don't give us the evidence that we need, we still have enough evidence to put you away for five years.
So that's [UNINTELLIGIBLE].
Both of you will go away for five years.
Now, if you do decide to give us the information, then you really get away with just one year in prison.
But the the guy you just ratted on goes away for 30 years.
[UNINTELLIGIBLE PHRASE] And of course, if both of you decide to rat on each other, oh now we've got [UNINTELLIGIBLE] charges.
That's awesome.
Both of you go away for a very long time.
[INTERPOSING VOICES]
You need to make [UNINTELLIGIBLE] defecting [UNINTELLIGIBLE PHRASE] a person cooperating and another defecting
Oh, right.
Right.
[INTERPOSING VOICES]
I seen versions where it's not equal.
But yeah, it's--
Separate game
RXY72?
No.
Defect should be 66.
Defect should be-- [UNINTELLIGIBLE] were you cooperating, but better than getting out
Yeah.
Yeah.
You're right.
[UNINTELLIGIBLE PHRASE] So this [UNINTELLIGIBLE] is probably too high too, too.
But--
I mean it [UNINTELLIGIBLE PHRASE]
All that matters is that it doesn't matter what your opponent does that he will [UNINTELLIGIBLE]
Although--
[UNINTELLIGIBLE] looking at
--they told me that, which is true for a single instance of this game.
The thing that gets really complex is what happens is you had to play this multiple times because you're multiple, repeat offender bank robber who gets caught all the time
Just give up
And that's where you get into various strategies.
The strategy in that particular dilemma is you should always betray your partner because on average, you're going to come out ahead.
Is that right?
The strategy is you do whatever the partner did to you last time
No.
That's a single game [UNINTELLIGIBLE]
Oh yeah, single game.
[UNINTELLIGIBLE]
You always send out [UNINTELLIGIBLE PHRASE]
So let me see, if you defect, then what happens is that you could get away with one or six years.
Six, you don't know what your opponent is going to do.
That's something like three months time.
If you cooperate then it's going to be between three and seven.
So that's more like expected value of five
You're not trying to win this game, you're trying to [UNINTELLIGIBLE]
[UNINTELLIGIBLE PHRASE] is actually right.
Say your opponent couldn't cooperate.
Then what if you didn't cooperate?
If you were to cooperate you get three years.
If you defect you get one year.
So you should defect.
What if they were to defect?
Well if you were to cooperate, then you'd get one year.
And if you would cooperate then you'd seven years.
But if you were to defect, you'd only get six years.
So you defect then too.
So you should just go ahead with that
That's--
[UNINTELLIGIBLE PHRASE]
That's basically [UNINTELLIGIBLE] what's the minimum-- [UNINTELLIGIBLE PHRASE] higher numbers to represent that thing happening to you.
But the whole principle of [UNINTELLIGIBLE PHRASE] we are trying to maximize your minimum benefits, and the minimum benefit in this particular case, happens in defect.
Because the most you can possibly get is six years if you defect.
If you ever cooperate, there's a chance that you're going to get to the years.
So [UNINTELLIGIBLE PHRASE].
The only problem is, again, once you get into this repeated over many, many times, then you're in a situation where if you build up a reputation, that all you're going to do is defect, then you can almost assume that your-- you can pretty much assume that whoever you're playing against is also going to defect.
At which point you will always [UNINTELLIGIBLE PHRASE].
You will never be able to get here.
There is a concept that helps deal with situations like this that actually make cooperation, again, a viable strategy.
I'm hoping I'm getting this right.
Well [UNINTELLIGIBLE] in the reading suggested there are saddle points.
There are saddle points that basically mean this is where all the strategies pretty much end up because of the reasoning that we just [UNINTELLIGIBLE].
You are maximizing the minimum chaos.
The minimum benefit [UNINTELLIGIBLE PHRASE].
And they all [UNINTELLIGIBLE PHRASE].
If everyone did the same analysis, they all end up here.
But John Nash comes along and he suggests that there are other kinds of saddle points that could be found.
And these are points where, assuming that everybody sticks with whatever strategies that they've got, if I were to change my strategy I will end up in a horrid situation.
So--
[UNINTELLIGIBLE PHRASE]
Defective for that.
Defective [UNINTELLIGIBLE PHRASE]
Yeah.
[UNINTELLIGIBLE] remains one of the saddle points Nash [UNINTELLIGIBLE].
But the other thing about it is that what actually becomes an unstable equilibrium.
It's one of those things, especially in a multiple game, is like if you're going to assume and you're going to be playing this game over and over and over and over again, switching out of coop has a long-term detrimental effect.
Because especially-- if you know that other people have a strategy of basically I'm going to do what you did to me.
It's called tit for tat.
If you actually stick with coop, and if everybody else is using the same reasoning as you are, which is another interesting bit of a wrinkle that Nash throws in.
Whatever analysis you are capable of solving, everybody else because everybody's a natural player.
Therefore, what actually becomes a unstable, but possible because [UNINTELLIGIBLE] points.
Because switching out actually leads to the net loss for [UNINTELLIGIBLE PHRASE].
And that was kind of the little thing that he threw in that kind of made his name and then [UNINTELLIGIBLE PHRASE].
But the basic idea is what is useful from a one table like this becomes really, really interesting when you think about what you're going to do in the long-term.
I believe there's a couple of other strategies that involve things like instead of just doing what I did to you last, just sort of like well if you defect and I were to defect the next time, which I think that's a pretty good perspective.
But they didn't cooperate and I wouldn't cooperate next time.
There's also strategies like well as soon as you defect, I'm going to nothing but defect.
And that actually was an interesting group, because that I think changes the utility numbers in a way where a single defection has a huge negative [UNINTELLIGIBLE] in a way that you might want to [UNINTELLIGIBLE].
But for the most part, I think tit for tat turned out to be the best strategy over the long term.
Depending on hos the numbers are weighted, tit for tat basically says whatever you do, I'm going to do to you next.
And that actually turns out pretty good because that actually is not [UNINTELLIGIBLE] coop actually works [UNINTELLIGIBLE PHRASE].
Yeah.
[UNINTELLIGIBLE PHRASE]
This also only works if there's an unknown number of future games.
Because if you know you're going to play any finite number of times--
True
Yeah
Yeah.
There is basically it's just like induction.
It goes on forever.
But you're going to hope that your game's going to get played infinite number of times.
We are planning for it.
Not an infinite number of turns
The winner can only resolve after [UNINTELLIGIBLE PHRASE]
First one who gives up [UNINTELLIGIBLE].
I really need to get some water
Game break
Yeah.
I'm going to give a quick break.
But does everyone have your [UNINTELLIGIBLE PHRASE] sheets?
Yeah
OK.
I will lecture and pick up on that in about 10 minutes.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.MIT.edu
OK, let's see, some administrative stuff, first of all, Wednesday's our play test.
So it's an internal play test.
You're going to be playing with each other games which means chances are you are going to get barely enough time to play one person's game, and enough time for that group to play your game.
So I'm just going to just, basically, pair up teams.
Hopefully, I think we have four teams.
So I think that works out just right.
One thing I will certainly recommend before going on to Wednesday's class is making sure that you had a chance to play test with, at least, someone who's not in your group.
And the reason for this, what you want to be keeping an eye out for, are rules that you've already agreed on in your team, but might have to learn it to write down somewhere.
I want the rules written down.
They don't necessarily need to look great.
Typed is awesome because they're easy to read, and they're easy to revise later.
It's not actually a requirement.
But you've got to have them written down.
You should be able to just put it in front of a person.
I will suggest that you bring a copy to the office.
If you want to come in early on Wednesday, or I could, maybe, grab the Xerox if you just bring a couple of copies so that the entire group of four people is going to be testing your game, which means, all users at the same time while you're explaining it to them.
If you're in a situation where you already have rules that you think could be understood without explaining, I would suggest not explaining at all.
Just give them the rules and see what they make of it.
It's going to be very informative that way.
Not every single team may be in that situation.
They may not have that much confidence in their ability right now.
If you think you've gotten all the rules written down but you just kind of feel not quite so confident that everything is in there, do it anyway.
Just don't explain it.
Give it to them.
See what they're having problems with and then you can explain it.
It will still be informative.
You'll still be able to get that data point of what should we have written down as a clarification, as a complete omission, what needs to be reworded before the game ever starts.
But I do fully recognize that there may be certain teams that do not have a set of rules that comprehensive yet.
In the end, you'll want to get the operating test and time.
So it's going to take half and hour for somebody to understand your game just by reading your rules, then you do want to try and cut that down.
Eventually, before you send me your game you'll want to be in a situation where someone can understand your entire game quickly, comprehending the rules.
What's up?
When we're play testing and timing, we weren't sure or we didn't remember, or we didn't read carefully enough, what is the maximum time that the game has to take?
And does that include set up from a complete novice, or does that include, are you allowed to explain the rules and have totally experienced players go at it, and once experienced players go at it they have to take a certain amount of time?
How does your--
It was supposed to be for novice players but I'm just going to check the actual syllabus
Do we have to be able to explain the entire game, run through an entire play through and all that, in 10 minutes?
That one I wasn't sure.
I think it's actually on your play through but give me a second.
I'm going to look up
I'm pretty sure it says 10 minutes.
Yeah, 5 or 10 minutes including setup and play
Including set-up and play.
But that does not necessarily--
It shouldn't include reading the rules
It should not?
It should not.
10 minutes is way too short a time to read and understand and then play a game
Yeah, let me just verify that
Walks and words.
That's about it
Our game takes two minutes to play at the moment
What we'll be able to do is take a really long time for premium effect and have some more information written on these cards.
It's just taking the time to write it so that way if somebody's busting a move--
So it's true.
It doesn't say 10 minutes that you need to be able to play and understand the game in 10 minutes.
So I'm going to say, set-up and play has to be 10 minutes.
So it's like, every time you reset the game and you play to the end, that should be under 10 minutes.
If it takes longer than that to understand the game, that's fine.
But frankly, if it takes more than 10 minutes for someone to understand your game, you have a problem.
And the problem isn't going to be one that's going to affect your grade, it's going to be one that you can use the processes in this class to reiterate on to improve your game to the point where it doesn't take that long to understand.
Reading the rules should not take longer than playing the game.
There are games like that but try not to make those games and not when it's the first time.
So I would say in for the explaining of the rules should take within 10 minutes.
Right now, it might be OK if the actual explanation requires someone who's actually an expert in the game to explain it.
By the end of this assignment you should be definitely getting to the point where you don't even need that.
Just a new player with your rules should be able to understand and play your game in 10 minutes.
It could involve discussion of the entire group of your players.
You've got four people in-- let's say it's a card game-- you have four people and they're all looking at the rules simultaneously, and your friend says, where does it say so and so?
And somebody else who has never played your game before answers that.
That's fine.
And that's actually how a lot of people play games.
Sometimes people play in quick thought rules.
This is all you need to know to play your first round of cards, and that actually helps a lot of people to get started.
And then you're picking up the--
Lowers the activation energy
Yeah.
In the end, more people play your game.
That's good.
So that's on Wednesday.
Everyone please come on time because every minute of this time wins some recognition or pizza [UNINTELLIGIBLE] and if whole teams don't show up-- Who's aligning a game that has to be played by both players.
So there's some teams.
Two?
Three?
Two or four
Two or four.
OK. All right.
No, that's cool
So even if a guy didn't show up on time that's
OK, so that's the preparation for Wednesday now.
I do want to apologize for Friday's lecture.
I was on a lot of meds.
You may or may not have noticed that I was having a splitting headache and was not the best lecture that I have given in my entire teaching career.
In fact, it's probably the worst lecture I've given
No.
I didn't notice
It occurred to me that there might just be a better way to get some of those concepts across other than me yammering for three hours.
And what I'd like to do is revisit some of them but actually do a bit of an activity.
I have never done this before so this is not tested.
You guys are the first iteration
Oh.
I feel honored
But basically, what I'm going to do is I'm going to go through these terms again, explain them one more time.
You guys have accurate clarifications or anything.
And then each in a group of about, I think two people, we're going to pick the game that pretty much every one in the class should sort of know.
Maybe they don't know the exact rules but you know of the game, and analyze that game across these different axes.
And the idea of that is that, if you get used to this then you can do that same thing to your own game.
And that may shed light on how your game is balanced, how your game is balanced especially between chance and skills because we get a topic of this class.
But also question, is our game doing what we want our game to be doing?
So that's the general outline.
So there's one more thing I want to mention before I go into that.
And Jason actually brought up a point at the end of last Friday was that he noticed that our two end robots, Mr. Roboto and Badass Mofo 600
610
Yeah, 610
It's the upgraded version
It started at 608 and then there were a couple of iterations at least on Saturday
One of them was purely deterministic but horrendously complex in the sense that--
It wasn't that complex
Not in the sense that it was difficult to explain what it was doing but difficult to predict what it was going to do.
This was the one that will start it's turn order based on the number of hit points that it had
And take the number of moves based also on the number of hit points it had
So it's entirely algorithmic.
It's just following it's root every single time.
The other one was pretty much random, right?
Who of Mr. Roboto which should have in a die roll every round.
And it just hit something.
And we ended up with the closest fight of that day, I think.
And that does drive home the point that at some point and time you can get complexity through showing deterministic rules.
That frankly, it's as hard to predict as pure randomness
My card game is like that
Your card game is like that?
The nice thing about that is that if you have deterministic rules, you can see why it happened.
You can see why, wow I didn't expect that to happen but I can see why it happened
He was so sure he was going to win.
He was so sure he was going to win but then we--
It did not turn out as such
That's a desirable trait.
It's a lot better than just saying, well it's a die roll.
The nice thing about die rolls is that it does take it out of your control.
It's kind of designed to make it not skill based and as a result of that, you get people who are lagging, who are having a harder time playing and finding the unskilled base way of catching up.
Which is awesome if that's what you're going for.
And then the other one is if you just make it deterministic but complex enough so the behavior is just hard to predict, but logical ones that are actually happens then you kind of get the same result.
It all balances out.
Die rolls are really easy to enter.
It's difficult to actually come up with rules that deterministically give you the kind of complex that you're really looking for.
But it's a nice thing to go for.
Of course, to get the complexity sometimes you need so many rules that people just can get bored.
It's like, wow, this is really complicated.
I'm not sure whether I can figure out what the next step is going to be and then they just give up.
That's the down side of really complex deterministic behavior.
It's neat that you saw both of those playing off of each other and them behaving in pretty much unpredictable ways.
And yet, getting nicely balanced results.
So the book we leave [UNINTELLIGIBLE].
So now, I'm going to be actually deeply, deeply plagiarizing from Salen and Crumb.
Katie Salen in Eric Zimmerman from Rules of Play which is one of those big thick books I brought in on the first day.
I'm not making you read it because I find that book takes three pages to explain what could have been explained in one, which is why I have a whole bunch of summaries in here.
But what that book does do is give you very, very concrete terms that are pretty well accepted.
They're not necessarily among game designers, among big people [UNINTELLIGIBLE] experiments.
And I've got some of those for you to refer to.
I'll touched a little bit on information theory on Monday.
That model of how a signal goes from someone who wants to send it through a transmitter and goes through some sort of channel, is received through some sort of receiver to a person.
If the person who wants to send a signal is actually the game, the game state, and is transferring information to a player, who need to get information about what's going on in the game in order to make a decision then, you can see how this applies to game design.
And in between you have [?
moints ?].
You have the thing that prevents you from knowing exactly what the game state is.
And there are certain words that are associated with that level of uncertainty.
Uncertainty and certainty are two word study searches to begin with.
So the uncertainty includes both your inability to know what's going to happen next due to chance or due to really complex behavior, as you saw in both of the robots.
It's just the degree to which how much the game-- all the different states the game could take, either they need the next turn or throughout the entire play of the game.
And in information theory, they actually also use the word information almost interchangeably.
Unfortunately this gets kind of confusing because people don't actually use the word information in that sense in day to day.
In fact, I'm going to propose a different way of looking at it that also uses the word information in a completely different way.
So I'm actually going to stop using the word information for now.
I'm just going to call it uncertainty which is pretty much all the different things that could happen in the game, either in their next turn or throughout the entirety of the game.
Certainty is the degree to which you have knowledge about what could happen.
Chess board pieces could be configured in any position but one move after the opening, there's only a small set that could possibly happen.
It's probably on the order of where only one side is moving for the first time.
Any one of the pawns could move 20.
20?
Yeah, the knights could move, yeah.
So there will be 20 different possible things.
So that's kind of the degree of certainty that you have among all the different states, all the uncertainty of this chess board.
You are certain that it's going to be limited to this chunk.
And your knowledge of what that ratio is, of how much you know could be happening, based on what you know of the game rules, based on your experience of the game, based on your skill in the game, to the stakes that the game could possibly take is called risk.
So you can flip it the other way around and say that if the uncertainty of the first turn is 20 different moves but you are certain that the guy-- let's say I'm the opening player, I played white and I really only know how to make three of those openings effective, and the person I'm playing against knows that really, I only know three openings, even if I only know two, then that person's degree of risk is limited to the range of possibilities that branch out all of that 2 out of 20.
So that's one way that you can sort of describe risk.
Let me see whether there's a more technical term that I can describe that with.
I skipped a page, didn't I?
So one thing that I want you to think about later, actually for the analogy step we're going to do.
I want groups of two.
2, 4, 6, 8, 10, 12, 14, 15.
Mostly groups of two and one group of three, actually, let's just do it in groups of three.
It will be easier that way.
And just think about and pick a game first that you think that you can probably mention in this class without having to explain to everybody else exactly how this game is played.
It could be a computer game.
It could be a board game.
It could be a card game, something fairly [?
modern.
?]
And I want you to think about what are the things that, what are all the variables that you need to know in this game.
Doing the [?
variable terms ?].
If you're doing something like a game of Caustic Encounter or something.
Then you can say that every single race as a unique power without explaining to me every single power.
I think there are 100 races in that game
Which game?
Caustic Encounter.
Oh it's fun.
You will get a chance to play it later this semester.
Let's see, so think about what are the variables that you got to keep track of.
That's kind of the degree of uncertainty.
That's the amount of information that could be in the system with that [UNINTELLIGIBLE] User does all the different range of possibilities, all the different things they got to keep track.
What do you know for sure a single playing in this game?
What do you know in your first turn and what information do you tend to pick up along the way?
Let's see, so let's start with that.
You've got just five minutes.
It doesn't have to be an exhaustive list.
So let's just split into groups of three.
The first activity we're going to [UNINTELLIGIBLE] every single one of these games through all of these different answers.
So I'm just going to start, actually now, and if we run out of time to do this, we can always do it after class and we can talk about it.
But we're going to go through all of them.
So I'm going to breeze through this.
So starting with this team then, moving all the way over and do you want to just talk about--
So we decided to go for Scrabble.
And so, in terms of variables, you have to invent it by individual tiles.
Their tiles are uncertain and yours are fairly certain.
The one's on the board were certain because they're faced
It was [UNINTELLIGIBLE]
Yes.
Another variable, well I used a variable, it's the bonus squares on the board.
Even though there's stakes on the board, they're variable in terms of, as more tiles get played, some of them become open and some of them become closed and you don't know which one's are going to be working closed
True.
True.
Yes.
So that's something as you don't know how that's going to turn out during the middle of play.
So one thing that's interesting about word games, especially in Scrabble and crossword puzzles, in general, is that a lot of them rely on the fact that the English language, in particular, is actually pretty redundant.
We can take about 50% of the letters out from a word, this very long word, and can still figure out what it means.
That works really well in crossword puzzles because if you get something like 25% of the word, you can actually still figure why the associated that clue.
But Scrabble rules are made easier so that it is now possible for you to make connections between disparate characters and find a word that's going to fit that.
Of course, part of the problem actually-- has anyone played Words with Friends?
Yeah, against my brother
This one like Scrabble, basically, right?
The same folks who made Words with Friends made a game called Words with Pirates.
And I have a screen shot here of the game I was playing yesterday.
And basically, you have way fewer letters.
You have h, a, r, a lot of r's, I and an exclamation mark.
So the number of letters that you can get, the quantity of the letters that you get are the same as in Scrabble, but the range of the different letter you can is way lower.
With means there's a lot less uncertainty.
Because this language, this parrot language has so much-- I want to be very careful to use the right word here-- so if I'm using the right word here, this is actually a very non-redundant language because a, r, r, r is a word.
A, r, r, r, r is a different word which are balanced.
So there's actually very little redundancy.
I see a, r and I think, that could be any number of words.
And as a result you notice that this game of Words with Pirates it results in an incredibly dense word.
Of course, I can show [UNINTELLIGIBLE] But you can't do that with Scrabble.
Scrabble spaces go empty and get locked out because, unless you have an incredible knowledge of the Scrabble dictionary, for the most part, you don't know those exploits that I'm going to then slide in an extra character.
But most people's knowledge of English doesn't happen in that board game.
So anyway I thought that was specifically useful for an illustration.
OK, so Scrabble.
All right, cool.
Next game
We did Go Fish so some of the variables depending on the equation we had were like every players hands and the pile of cards in the middle that you can pick from.
We thought about some of the uncertainty being what cards the other player has.
And the challenge that comes out of it is them asking what cards they want and you asking them for cards.
And there's noise because just because they're asking for a certain card can imply what they have in their hand, but it doesn't necessarily tell you what they have
Right.
It only tells you what they don't have in their hands.
Sometimes it tells you what they don't have in their hands.
Sometimes they give you a card and-- I forget the rules.
If you have multiple threes and someone says, got any threes?
You give them all three?
Yeah
Yeah, OK.
So
Why do you have three 3's?
You put a pair down.
That would just be dumb
That means you're a bad player
For the most part in that game that's your one channel of finding out what the state of the game is.
The game states what everybody has and hasn't had.
Cool?
Cool?
Cool?
We did Mario Kart
There's a lot of uncertainty and a lot of other things to keep track of.
So the things you can't possibly keep track of but that are really important in the game are what's going to come up when you hit an item block and what items people around you have.
So you might think it's safe to bump another player but if they have three green shells they can activate that then and blast you out of the map
Another one is what position you're in because if you're in first place and a blue shell hits you that's also what position you're in, which is what other people are in, and stuff like that
Right.
So the probability of being hit by the blue shell is way higher when you're number one
So you gather information through your people range which is also a function of, basically, where you are on the map.
And so the information you're giving about other players depends on where you are, interestingly.
If they're immediately in front of you, you have a lot more information than if they're behind you, you have a lot more information than if they're way far away from you
Right.
Because if they're behind you or they're way far away from you, you don't know what weapons they've got ahead for you, for instance.
You generally know what position others people are in but that's all you got.
So [UNINTELLIGIBLE]
And then there's different paths and levels so you might get someone in the same position as you but they're not at the same point because they're at a different branch.
And then, good also the very given motives of different cars.
So in Mario Kart 64 if you take Bowser and you're playing as toad, you can ram into toad until you go flat
Yeah.
I've run into Bowser before
So one of the little risks that you get in Mario Kart is that the kinds of weapons that you get are pretty much weighted based on where you are in the game.
If you are at the back, you get all the awesome weapons.
Because the game--
You get the star.
The eraser on Hero station was way more weighted than--
Sure.
There are more obvious versions of this.
I think Super Monkeyball axis, too, actually.
When you're right in front, I think, all you get is bananas, is that the only--
In Mario Karts?
Bananas and one green shell.
That's pretty much it
You get a pile of junk later
And also, the fake mystery box
And half way through and half way games, the waiting is, I'm going to guess roughly linear.
So the basic idea is one of the things about risk is that every time you take up a new weapon you are, basically, taking a gamble about what you're going to get.
If you're already holding a weapon sometimes you can choose to discard that to pick up something new.
And I believe leader iteration you can [UNINTELLIGIBLE] think of that.
So now, that's one element of risk, right?
What do I know?
I know what I got.
What else do I know?
I know what position I'm in and I know that if I run into this box right now and I got rid of what I had, I could pick up a cooler weapon.
There's a probability associated with that and that's, yet, another [UNINTELLIGIBLE].
One way to think of it is that there's such an outcome, which I said a probability associated with it, you are going to get a weapon
It might be cracked.
You only really gain a sense of the hard way from experience of playing with it
Yeah, it's one of those things where your certainty of this game goes up as the number of play.
Cool
We did Monopoly?
Sure
So Monopoly basically, what's certain is who owns what property, essentially.
And it's technically certain how much money everyone has though keeping track of all that information on a turn-to-turn basis can be difficult.
So there's a little bit of relative uncertainty just due to the quantity of information there.
Yes, so you gain uncertainties.
It's actually that Monopoly has very different opening and end games, essentially.
Towards the start of the game there's a lot of uncertainty about how other players are going to value property that you give up.
Owen was telling me.
I wasn't even aware of this.
In Monopoly when you decide not to purchase a property it actually goes up to auction.
It's like I have never played with that rule
And it's an open auction and so you don't know what other people's valuation are so each time you'd be taking a risk if you didn't actually want it at that price
That actually makes the game more interesting
It turns out the game is not that much more interesting.
Even when you have the auction system [UNINTELLIGIBLE] it's still pretty boring
Can you see the like degenerate strategies.
We will get to them
Towards the end of the game, basically, it's also a lot of uncertainty about what players are going to be-- well, these aren't part of the rules-- but whether they're going to be willing to trade with you, whether you can do things like manage to get into jail so you can stop landing of people's properties.
Chance is, of course, another level of uncertainty there
Yeah, it's weird about the risk associated with an auction because you kind of know the seller.
Especially in the early game, you have a vague idea of how much people could possibly bid on this.
Not more than the amount of money that you have, although [UNINTELLIGIBLE] a mortgage something that he had, just to get the cash flow at auction.
But it's one of those things that you know is fairly built with certainty, and so it's true risk.
Ok?
So we chose chess.
Chess has no uncertainty in the game.
If I would say an answer to you would be, you don't know what you're opponent is going to do.
So you have to make a move assuming he's going to do something completely different.
And as well as risk in the game, the only risk is also not knowing what your opponent's going to do.
So he could do a series of moves which causes you to be in a situation where you have to lose a piece.
That's pretty much it
That's cool.
I would argue that not knowing what your player is capable of is deep in uncertainty.
And in fact, it's a usual amount of uncertainty in the game.
It's true that the way and how the pieces move and where the pieces are placed, everyone knows that.
And we're going to get a mixed one.
It's a game of perfect information.
The game stake is no dice.
You know exactly what's going in the game unless you're playing with a five year old who didn't put the piece right down in the square
So I guess something where it would be a lot more noise for the game would be if you're playing something like speed chess because then it's like oh, you just have to think fast and it's a lot harder to realize what's going on
Yeah, that is true because now you're limited by your own cognitive bandwidth.
So that's true.
It doesn't change the amount of uncertainty.
The uncertainty of the game really is your opponent and what's in that person's head.
The knowledge of openings and knowledge of different moves, whether that player has seen something that you haven't seen or knows of a counter that you [UNINTELLIGIBLE]
Or there's another way to give the impression that they know exactly how to counter if you're doing any kind of loading like
Sometimes other counting on them to do something, and then they don't it's like, why didn't you do that?
I would have done that
It's like golf where you have to pretend like you really look like you know what you're doing
Or poker.
OK, so that's one way to look at your game.
What you know, what you don't know.
What's the probability of the certainty and you know the answer.
And, of course, not as you.
As a game designer you're thinking, what do your players know?
What do the players not know?
What can your players do?
And a reasonable guess of what likelihood something is going to happen.
Also, the things going to be true.
It's like, OK, I don't know what card that person has but there's an 80% probability that that person has jacks of higher of the trump because that person bid on the trump.
That's a fairly good guess.
A low risk guess.
OK now, we'll go to the other schema which is the information systems.
Now, we're talking a little bit about perfect and imperfect information.
Perfect information games, you know game state.
Everyone knows games state.
No player does not know a game state.
Imperfect is, the information, OK, there's something you don't know.
Maybe it's card games in particular because cards have two sides.
Cards are typically used to hold back imperfect information.
Anyone play Stratego?
Yeah
Now that's incapable of complete, imperfect information [UNINTELLIGIBLE] the game.
You know every single piece.
There's nothing random in an entire game.
There are no die rolls.
There are no random shuffles
So when the other guy chooses to play where he just places pieces, it can be perceived as random to you
Can be perceived as random, it is true.
But let's get really specific about turn around.
What it is, is that it's actually uncertain.
It's not actually random, it's a decision made by that player who's got the right moves.
So, actually one, objectively, it is not random.
It is perceived as random.
It's perceived as, effectively random because really the person couldn't put his pieces anywhere within the first three rules, or something.
So however, that's kind of a blunt tool.
Is this game a game of perfect information or imperfect information.
That's a yes /no question.
Does it really tell you that much about your game.
Asking, OK, what is actually random and what is not actually random but may be perceived as that, is a slightly better question.
Because you can think, OK, if I isn't clear I know that's not random but it could be anything because of what the other player has done.
I will treat it as random.
I might as well do all.
And when you're designing for that player and you are doing all your calculations, just taking it as random.
There is a game theorist-- not game theorist-- the game studies scholar of games, Ceila Pierce, who works over at Georgia Tech.
And her proposal is that a more useful range of questions to ask would be what's known to all the players?
And in a game of perfect information everything is known to all players.
What's only known to one player?
What's known to the game?
Especially for a computer game, the computer knows a lot more about game state.
Sometimes that even comes down to rules.
So if you've ever done Force Battles in something like Metriod or Castlevania, for instance.
There are rules that every single process is going to react to anything that you do.
Maybe it's based on time.
Maybe it's just the sequence of action that are already done.
Like the EIHI robot that you guys programmed in class [UNINTELLIGIBLE] You don't know those rules going into the game.
Those don't prevent you from playing the game.
In fact, part of the fun is figuring out the rules and figuring what the pattern is and explaining it.
So that's an example of a thing that's known to the game.
Other things that are known to the game are actually now, maybe that's not a good one.
I'm going to stick with my topic because randomly generated stuff, like there is a shuffled deck.
Shuffled decks are one of those things where it's a little bit unclear about whether the game knows what's in that stack.
It's true that you shuffled it, so it's randomly generated, but once it shuffled it, it is now in a fixed order.
So the game knows what order that's in even though essentially randomly generated, it's not randomly generated every single turn.
But it's a good question to ask.
Even if it's not an easy question to answer, it's a good question to ask what's known to the game but not known to the rest of the players?
And what's this randomly generated?
Even if the game had a brain we would have no idea what these things turn [UNINTELLIGIBLE] because it's pretty terrible.
So use those games again.
Every single one of those games they are talking about, what's known to all players?
What's known to one player?
What's known to the game and what's just randomly generated?
Go.
Shall we just do the [UNINTELLIGIBLE]
So we changed our game to poker.
Texas Hold 'em now.
So know on your test.
So it's imperfect information because you know what's in your hand, you know what's on the board, but you have no idea what anyone else has.
They can try and trick you so it's maybe perceived.
It's perceived knowledge.
So let's say they play a large bet.
It's perceived that they must a good hand but it's not necessarily accurate.
So it's not really an objective base however, it is subjective whenever you can finally show your hand.
Then either you've won or you've lost.
It's not really up for debate
So I guess it could also be whether you know, like again with chess, what the other player knows, whether the other players know conventional poker conventions for folding or not
OK, let's see.
Other things that are a lot of [UNINTELLIGIBLE] especially poker, that is known to all players will be things like, what are the number of kings in the deck?
What are deposits and what are all the different hands that you could make, and what are the point values?
So that's just one example of things that need to be known by all players.
The assumption is that all players know that.
Sometimes all players don't know all these
Yeah, memorizing the probability is a huge endeavor
Really big player know the objective probabilities in any given hand
Actually, a lot of casinos will let you bring in a card with probabilities
Really?
I'm surprised that they'd let you do that.
OK. All right.
I guess they couldn't--
Even blackjack.
I'm pretty sure with blackjack, too, you can bring in a card with probability
So the things that are known to the game would be, of course, shuffle order.
I guess this happens more in blackjack where you've got five decks all shuffled together and you know there's a number of kings and they're in there somewhere.
So if all the kings get played out you know there's no more kings left in there and then you can do computations from there.
And one thing I want to clarify on objective and proceed.
Objective doesn't necessarily mean that any one player knows whether this is good or bad.
The idea is that if there was a third party observer who was not playing the game and knew all the game stake, looking to that, looking to the hands, knows what's coming up, or a computer running the game, that person, that computer will know the extent to which something is good or bad.
So if you've got a hand probabilistically you can tell well, this is a pretty darn good hand, or this is all ready probabilistically a crappy hand.
It might be good for that game, right?
It might be just like I have one pair, but it might be good if nobody has anything
But probability you're not gonna have a good game
But probabilistically you want to win.
That's why you have poker player's saying things like, right up until this point I was winning and then I maybe lost in a tie game.
But it's like, probabilistically they were doing everything right.
And actually, a lot of professional players have disparages to say well, they were doing everything right, sure they lost a hand, but to them they were winning every one up until the point when he had to fold.
OK, next game
The Monopoly game was run through the all players, one player game, etc.
All players know the board stakes so the people that are owners of any of the properties and [?
what's owned ?]
by the bank.
Every given player knows the amount of money that they have, and also what valuation they placed on all of their properties, which is not known to other people except incidentally through their actions.
The game knows the state of Community chance and, I'm sorry, Community Chest and Chance decks, and then there's randomly generated die rolls
Die rolls, yeah.
I'm forgetting whether Monopoly requires you to show your bankroll clearly
I think so.
I'm pretty sure
If you want to know how much money somebody else has they have to tell you
It would be an interesting thing to look up and see what the actual rule on that is
Why is the game so popular?
Good marketing
But then again it's very shallow and--
It's really familiar.
I asked friends of mine who happily play Monopoly but don't like to play, for instance, Settlers, and a lot of it is really, you don't have to think too much about it.
It's just an excuse to sit around and talk with people.
There's not that much that apparently distinguishes somebody who's like a incredibly good player or an incredibly bad player.
It's got a bit of that any given Sunday feel to it so that you always have an opportunity to win.
There can be dramatic swings of fate, and all that kind of thing
So it's kind of like adult Candyland.
You just go through the motions
But I also have friends who are like avid Monopoly players who really believe that there's a deep skill to playing Monopoly.
But I'm yet to be convinced
There is a strategy.
I'm not sure if it's a skill.
OK, all right, cool.
Thanks
We did Mario Kart, again.
Except for a couple of special stages, in general, players are aware of their welcome position to other players during the race
Everyone knows the maps.
[UNINTELLIGIBLE]
You're not sure if you know what other people, like what their special power ups are, because you can look at the screen and see what they have
Yeah.
Since you're playing to other humans there are situations where you definitely know what they had.
I don't think--
That's cheating
It's a strategy
In Mario Kart DS you know all your opponents items
That's true.
That's lame
Suddenly, on a simultaneously multi-player game where you're screen's divided into four or two, yeah, you know what your opponent has and decide if you're going to get into a Kart.
Same thing for Halo
Yeah
It actually [UNINTELLIGIBLE]
Yeah.
And let's see what else.
So there's you perceive it as completely random especially if you don't pay much attention to it but it's more like weighted random in what items you get.
So that's something you'll have to figure out as you go along, that actually, as you're further up, you get worse items.
I actually think it's a good strategy in that game to remain in second place for most of the race and then go to first
Now, the game knows those probabilities anyway
The game definitely knows that.
The game also knows who's going to get hit by a red shell, for example, when you launch it.
It's not entirely clear to me as a player, when there's multiple people in front of me, who it's going to pick.
When I shoot a green shell back behind me, the game knows what's going on.
I don't.
Is that my field of view?
I hear it.
Am I getting a result?
Am I not?
The game definitely understands the physics way better than any human player can possibly understand
Yeah.
For sure.
For sure
The game knows all the player's positions and dull leads but it doesn't know what they're going to do or how their going to interact with each other.
It also doesn't presumably, actually know what items are coming out
Oh, and then another thing as players, you can learn the patterns of some of the hazards and be able to predict which way to go depending on how you know a certain--
Just like knowing the map.
Just like knowing the map
Yeah.
One of them are random, actually
Some of them are random
But one thing to keep in mind about randomly generated computers obviously, well, maybe it's not obvious but computers actually have a real tough time generating really random numbers
It's starts to assume
That why they're called pseudo-random.
So to some extent the computer knows but it's trying really, really hard not to know.
What's really, really useful if you ever go into game design for digital games, to use that to your advantage.
Because you can use that to reproduce problematic situations.
It's like, yeah, write another generator that we have absolute control over and then use that to your advantage when you're trying to find bugs.
Cool
So just to remind you, we did Go Fish.
So clearly every player knows what's in his hand, knows what pairs have been played thus far, knows how many cards people have, and stuff like that
Generally, people should know what cards have been given from one player to another player.
Although, they may have forgotten that.
Sometimes there's a situation where most of the players don't know which pairs could be possibly made at a given point.
Sometimes one player may know that because he might have one card and then somebody else gave their card to somebody else.
Something like this.
But the game definitely knows this.
Also, the order of the deck is randomly generated
Yup.
And he's right at the beginning, first turn
So For Scrabble all the players know the pieces on the board.
They all know the letter frequencies
They don't know what tiles they're going to get next.
[UNINTELLIGIBLE]
That's randomly generated
What if someone knows the dictionary, though?
Like your opponent could have a dictionary, but not everyone knows it
That's the knowledge that is both available to one player and to the game.
Technically the game has an official Scrabble dictionary.
If you're playing a digital version, they're probably using it.
Your family, they probably don't accept certain two-letter words
Until Pictionary
They also don't know what the other players have.
So one person only knows what they have.
But then the other three aren't really-- But then again, we discussed how if you're playing with two people you can probabilistically figure out what they've got.
Or, figure out what you're going to get next when you draw
It's also more likely for you to figure out as the game goes on and tiles get played what letters will never come up again
At pro levels of Scrabble they do know exactly what's the final set.
There's just a certain point that you get to at which point Scrabble players just know what play and work out and all the possible--
At that point do you just keep letters that are more likely to work with what's in the bag.
How do you use that to your advantage?
Well after all the tiles are drawn you know what your opponents have
Yeah, you are trying to prevent them from using the letters that they've got.
But most Scrabble boards, although I don't know if all Scrabble boards have this, they actually show the frequency in which the number of tiles where I'd be given that are in there.
So it's actually expected that you're going to take this into account.
It's like oh, there are only 4 h's or something like that, and they're all out.
All right.
There's none coming out of here
Random other Scrabble facts.
I found another one about Scrabble when I was reading about computational Scrabble playing.
Apparently, in Scrabble tournaments, they have a chess court system, right, to making sure it doesn't go over time.
And apparently, people set aside half their time to playing the end game in Scrabble.
Which is the point at which there's basically full information.
And then, you're just sitting there nodding out.
Like, what the best sequence of play is to beat your opponent so then, it becomes like chess
How much time do you get?
I don't even remember.
So, high level Scrabble is completely different from the Scrabble that you play.
It's crazy
OK, all right.
We've still got some time.
Great.
Let's talk about cybernetics.
Cybernetics are, given that it was kind of invented at MIT, it's something that you probably encountered there.
And we've mentioned this a few times, the whole idea of feedback.
I want to break this down a little bit, though, because feedback works.
The official sense of feedback, and this theory of cybernetic systems as written by Norbert Wiener back in the 40's, I think, is that there is a sensor, something which measures what the system is doing.
There is a comparative, which was, in his era, was that machine, and in our era is probably a rule, a logic, a statement, that compares this against some sort of set value or some sort of rule and decides to take action-- an activator.
And you can think of this this way, all as your sensor is the variable that you're looking at.
Or, a set of collection of variables that you're looking at.
So what are you looking at?
Your comparable is a rule that you're comparing against.
I'm trying to pick a game that no one has mentioned here, so far, that has a good few facts system.
Let's say I played a version of Cops and Robbers as a kid where every time the cop caught a robber, the robber would have to go into a jail area and put him down until someone tagged him.
Someone from the robber team tagged him.
And the basic idea is that the fewer people there are, and a few of us humanly are playing in a big space, the more robbers there are in jail, the easier it is for a robber to run.
Actually, because they have more freedom.
Sure, there are more cops to any given robbers so that you can do flagging [UNINTELLIGIBLE] But one of the variables that we're looking at are how many people are trying to be in the jail.
And an activator is basically what's the rule or what's the game dynamic that comes with the play.
So that's a kind of interesting things about the Cops and Robbers situation.
One had more space to run when all my buddies are in jail but there are a lot more cops going after me, and that actually means that it's easier for him to flag me even though I had more space to run.
So it goes up.
Both are negative and a positive feedback system.
More space to run is actually a negative feedback system.
And how you tell the difference is is it stabilizing to my state or is it cumulative to my state.
And it's stabilizing, which means I'm doing badly, my team's doing badly but this is now giving me an edge.
Or, I'm doing really well and this is really crappy.
Mario Kart is a type of these.
Those are negative feedback.
Those are things that basically reduce the gap between the best and the people who are doing well and the people who aren't doing well.
Positive feedback is cumulative.
They're doing well, they're going to do a bit better.
So the cops have got everybody but one and now it's a 5 to 1 game.
That's a positive feedback system because now they can do tactics as opposed to everyone going after someone separate.
And you have these two feedback views working together.
There are tons of these in games.
Games aren't really the kind of closed systems that cybernetics were actually originally designed to handle.
In fact, only a few systems are digital systems in that way and games are actually better than most.
But let's talk about what's in all the games we're talking about.
What are some of the negative feedback you experience, and some of the positive feedback you experienced.
Mario Kart team, you got way too easy [UNINTELLIGIBLE] Think of some other game.
Think of some other game
Wait, that's what I didn't know.
The AI automatically goes faster.
There's a huge gap
That's more than just a Mario Kart, actually.
That's a lot of racing games
Also, when if you're playing like the Grand Prix there's always three computers that are always better than everyone else, then all the rest of them.
So they will always be the ones that are in the top four.
Like they're in the top, always
Mario Kart team, pick a different game.
Go through all of the games.
Think about chess.
Actually, chess might actually be an interesting one for you
No, but shot chess, where you retake the shot after--
Shot chess?
Yeah, shot chess
That gets too easy
Well, it tends to be negative.
It can be negative feedback but there's also the method of doing it where there's positive feedback.
It depends on how you play
Yeah.
Winner takes any drinking game, pretty much
It depends on what your goals are, right?
Such that you'd have it that whenever you take a piece you have to drink that piece.
But then there's also whenever you take a piece you opponent has to drink a piece.
So it can either be negative or positive feedback depending on how you play
Yeah, but let's pick a version of the game.
Ideally, pick the same game but if it's too boring pick something else.
OK, so let's start from somewhere in the middle which and I think is this group right here
So we took a pretty easy one but it's pretty like mathematical feedback.
We did War.
OK, statistically if you're winning, every time you win you're lowering the value of your cards
You had to average the value to your deck
Yeah.
So mathematically it's negative feedback
So, it's positive feedback, right?
Negative.
[UNINTELLIGIBLE] the negative impact is you're lowering the average value of what's in your deck
So which makes it harder for you to keep do winning
Exactly
So that's just negative feedback.
Any positive feedback?
The small positive feedback would be insofar as you can't win unless you get all the cards.
So that's winning hands gets you more cards, gets you closer to winning
The more cards you have the closer you are to winning
That's like saying, and you get with points, the more points you get, you're closer to winning.
Therefore, positive feedback
Yeah, I guess it does count
Logically it works, but it's kind of a guess
Well the only other one is if you have a war then you win those three extra cards and depending on what they are it could be positive or negative
OK. All right.
Which way should we go?
This way
We switched to Settlers.
We got a bunch of feedback loops.
Obviously, the more points you get, as soon as the points are cities and settlements, then the more resources you're going to get in the future and the more points you can build.
So potentially one player just has so much stuff that he can just do whatever he wants to
And that's positive?
Yeah.
There's also negative feedback [UNINTELLIGIBLE] like it censors the relative prices of resources.
The more one resource is worth at the time, each person, the other one's the more people try to build on that resource brick, then the less that resource will be worth in the future.
So, over the long term they tend to balance out instead of having one go rapidly out of control
Think about that one.
That's interesting
Also, the more, this isn't exactly feedback, but the more, the better you're doing, the less likely people are going to try to trick you somewhere
Yeah.
That, in my opinion, is the biggest negative feedback in the entire game.
If you're in the lead, everyone hates you.
And then you have eventually [UNINTELLIGIBLE]
There's also if he had a bunch of resources, like if you had a bunch of cities on a resource and if someone draws a six then you end up getting six resources from a tile then that's higher probability that if someone rolls a seven, [UNINTELLIGIBLE] So I guess that's negative feedback, too
Well your going to seven and losing your cards mechanic is very much a negative feedback.
I'm so rich!
Now, I'm not so rich.
What kind of poor is this, ha, ha.
I still could have seen them [UNINTELLIGIBLE] That's negative feedback.
A couple of things about negative feedback to keep in mind is that negative feedback does tend to make games longer.
It sort of staves off the ending of the game because obviously, the difference between the people in the lead and people not in the lead narrow it to the point that sometimes puts it back in connection again.
And positive feedback tends to make things end sooner.
OK, all right.
Now, next team
So in Scrabble, the main one we came up with was that if you play a long word, you get a lot of points and you also get a lot of tiles which gives you the opportunity the next turn to play more points and get more tiles played
Oh, yeah
There's also that the more tiles you play, the less tiles there are available.
So that's kind of negative
Yeah.
It's both positive and negative.
That was proposed
Near the end, yeah, it becomes negative feedback.
You possibly end up with just fewer tiles.
That's true.
Any other negative loops in there?
Well we tried to make it into a shot deal.
Yeah, every time you play a bigger word it's easier for someone else to play a word off of that.
That's negative feedback
That's an interesting thing because that's actually counter examples to a point that was made by Eric Zimmerman.
Because he argues that positive feedback actually magnifies early successes.
And negative feedback tends to magnify big ones.
Just like in Mario Kart, it's great to be the second place because if you do a late game success that propels you to first place just as the game ends.
Whereas, some of the examples that we've been hearing on all the feedback, if you manage to be the first one to build a city in Settlers, that tends to perpetuate your success for the rest of the game.
The game isn't quite as predictable as that but it's designed that way, at least.
But that's a very, very good example of you've got this negative feedback near the end of the game where you have a success by playing all of your tiles but the bag is almost empty.
So now, you are not getting, not whether you are or not getting an advantage, that late game success isn't translating into [UNINTELLIGIBLE] Still, it's as a general rule, if you aren't dealing with a limited resource, like tiles, then we do this situation where a positive feedback game, a positive feedback loop is going to have to find early successes.
And negative feedback make late successes more hard
So we're basically, [UNINTELLIGIBLE] StarCraft.
But I think with StarCraft was some of the obvious positive feedback is that the more a worker was built and then the more mineral you can get.
The more mineral's gathered you can get more troops
The more workers you build the more cash you get.
The more cash you get the more workers you can build.
That's [UNINTELLIGIBLE]
Another positive feedback loop is the more that you conquer your opponents, the harder it is for them to build up and finance you while the easier it is for you to beat them.
On it the other hand-- do you want do negative feedback?
I don't think [UNINTELLIGIBLE]
The only thing with positive feedback is just like emphasized a lot of the earlier game strategy
So a negative feedback loop in StarCraft would be the faster you, the more you expand your territories the harder it is for you to defend because your spraying half your resources, and the more friends you have, so the easier it is for your opponents to attack you
Yeah.
We were having that little discussion earlier and finding other negative feedback in this was kind of tricky, actually
Another one that I thought of was the more research that you do, the less resources you have for building units or workers and such.
And so you have to decide.
I mean, your power will be units but you may have less to actually build those units.
And probably not really-- it's a very small negative loop
Another thing, too, the more shoots you have, the less attention you can give to all the [?
children.
?]
So it's harder to manage
The less effectively use them
The more different screens you've got to keep your eye on makes it--
That's the great thing about the control wand.
It's like one, go here.
Two, go here.
Three, go here.
One thing a team gives, though, is there's a huge positive feedback.
If you're doing 3D, if you take out one of them and then it's so much easier
[UNINTELLIGIBLE]
Then you have a bunch of risk because what happens if you try taking out one person?
Then, you fail and now you're weak but then they have a lot more troops
So in keeping in comparison with Warcraft III which didn't have built-in negative feedback where if your army gets past a certain size and it costs more, at least you can get taxed income.
Yeah.
I don't know if you guys know about this.
But basically, think like StarCraft but then you have a food count that once you get past a certain food count, each of your peons brings in less gold from the farm.
I'm sorry, from the gold mine.
And it gets even worse the more troops you get.
So you're kind of punished for being too far ahead
That's a very key negative feedback
This isn't a strict thing to keep up with but it's interesting to look at the resource management.
Basically investing in units that bring in more resources or investing in other units that you're fighting are all different positive feedback loops.
And you can view not investing as negative feedback loops in those areas.
Although, it's not really completely true
Yeah.
are trying to think of that as negative feedback but you're right.
They are different positive feedback groups.
They work very differently.
Winning a battle makes it easier for you to win later battles.
But building more workers makes it easier for you to go more units
Kind of in a not apparent positive feedback issue with StarCraft upgrading a Zealot's attack power by just one greatly increases its efficiency because now two attacks will take down a Zealot.
Where before it would be three attacks--
I don't want to get into that much detail.
Let's go onto the next game.
[UNINTELLIGIBLE]
So, let's just say, Crunch we can have a bigger workforce and they're returning to pay for that.
So you get more money for your workforce but you still have to pay for that.
That's negative feedback
Yeah.
That's a killer
It turns out in Monopoly if you have a lot of money, and a lot of stuff, which people then land on, which gives you more money
With what?
That's top secret secret strategy.
I mean, Monopoly is to buy lots of stuff and then have people land on it so you can get more money
So this one little negative feedback into, actually two, for the income tax and--
--repairs on property and the like
So several thousand dollars to build hotels
Our actual government is like that, too
So it patterns out to real life but then again, so is the whole of the Monopoly game.
The whole of the Monopoly game is about real life land ownership monopoly in Atlantic City, I believe.
Yes
It's Atlantic City
So yeah.
And that gives us a good segue into degenerate strategies.
OK. All right.
So I'll go into that a little bit.
But maybe I'll get at what we did on Wednesday.
But Wednesday we're going to focus that mostly on play testing your game.
If we have no time for anything else, we are play testing your game.
And so bring in your games.
Bring in your cards.
Make sure your cards are ready before class.
Don't be writing stuff.
Try not to be changing stuff right up to another class.
Sometimes you might be feeling the pressure to do that.
Try to be ready before class starts.
And try to show up right at the beginning of class.
Because it will really, really suck if you finish play testing, if you finish play testing someone else's game and that team doesn't have enough time to play test yours.
OK, and when it may happen and that will suck.
So let's try to start that right on time next Wednesday.
Thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
We'll start with a little bit of recap from two Fridays ago.
But just a reminder of things to keep in mind while you guys are testing.
On one hand, most of you are pretty much the most expert players in your game at this point in time, [INAUDIBLE] You know the game better than anybody else.
And the kind of things that-- that means you suck as a tester.
That means you still have a role to play as testers.
You can be looking for the degenerate strategies, and you can be looking for exploits.
Sometimes it takes more imagination than a sleep-deprived MIT student has available to them, to be able to see these exploits.
But, because you know the systems better than anyone else, you're in a good position to spot that.
The kinds of things you're looking for are obviously value [UNINTELLIGIBLE] strategy.
I mentioned that the last time we were in this room.
But also, look for those strategies were, like, OK, I have two [UNINTELLIGIBLE] those decision points where I could do decision A or I could do the decision B, but if I change my strategy, and nobody else has changed their strategy, I'm screwed.
And that makes it really really hesitant for someone to say, well, Strategy B could be really interesting, I can go for this kind of play, but on the assumption that the only way how this is going to be beneficial is if everybody else also changes their strategy.
Now you have a situation where nobody wants to try that second option.
That's one way to think about cyberpunks.
It's like, if you change your position and the only way that you can change your position is at a cost to yourself, then the benefits of changing that position is going to be lower than the risk that you're taking on, then you've got a situation where it's very difficult to break out of one dominant strategy.
So if you've got a situation where there are three dominant strategies, then none of them are dominant anymore that's great, now I have an option of different things they can choose.
So that's just one thing to keep in mind if all you guys are looking at a game.
If everyone just ends up adopting the same strategy, and you're just hoping for a chance of drawing the right cards, in order to win your game, then [UNINTELLIGIBLE], then you've got a game where the individual player's decisions aren't quite that influential on the outcome.
And I still need to play your games to be able to see what you guys are going for.
But, for the most part, it looks like you guys are not going for a purely chance-based game.
You're looking for something where someone's decision actually matters.
So that's one reminder from the game theory point of view.
And the other one I never really got time to talk about games or system of conflict, and we don't have a huge amount of time because we have today's readings to go through.
But if you think about games as games where multiple people are striving for some sort of goal, and there are going to be winners and there are going to be losers, then the idea of thinking of games as some kind of conflict makes sense.
If you think of them as, are games competitive, or are games cooperative, for throughout some games that are known to be cooperative [UNINTELLIGIBLE] there are certain modes in certain different games that flip it from competitive to cooperative
[UNINTELLIGIBLE]
But it is designed for four people to play, all going for the same goal
Gears of War
Gears of War multiplayer, yeah.
[UNINTELLIGIBLE] interesting, yeah.
Because it's kind of the risk of cooperation.
Pandemic is you against the board, actually.
You against the game rules.
And actually, that leads into today's lecture a bit.
So hang onto that idea of Pandemic.
But there is one argument that all games are competitive.
Even games like Left for Dead, where you're playing against the computer, or Pandemic, when you're playing against the game rules.
Or--
Shelter [UNINTELLIGIBLE]
[UNINTELLIGIBLE] a bit-- [INTERPOSING VOICES]
You're mostly playing against the game rules.
Unless there's a trader, there's not always one in the game
But the basic idea is that even in a game that's generally assumed to be co-op, you're still playing against the world.
So as competitive against the world, just not here, computing against the other human people that are playing it.
If they're competing against.
Anyone is probably going to decide who set it up, to [UNINTELLIGIBLE].
So that's one way to think of all games as competitive.
Obviously, the games that people generally think of as competitive, real-time games like chess, for instance.
There's one winner, there's one loser, and if there's a draw then nobody wins.
Or, those games people generally think of as competitive.
But then, the other way to think about cooperation is that all games are also cooperative.
Because if you don't agree on the rules-- if you're all in fact not playing the same rules, then you don't actually have a game.
So you all have to cooperate to play.
We have the same set of rules and we're going to play according to them.
That's not necessarily the same thing as saying that everyone knows all the rules.
There are plenty of games, especially computer games, where you're going into the game not knowing all the rules.
You're going to try to figure out the rules as the one system lets you.
Role-playing games, where you start off like, I want to do this.
I want to-- you know, get drunk now, and how do I do that?
And then the GM tells you the rules.
Roll against your--
Constitution
Constitution, is it constitution?
Yeah, [UNINTELLIGIBLE] but.
Drunkenness
Always good.
Possibly.
And then you get told the rules.
And then you go and do it.
Live-action role-playing games.
I know there's a bunch of people here who play them.
You usually get a subset of all the rules that were written for the game.
You get the subset that pertains to you.
And you assume that everybody else has the subset that pertains to them.
And that they're playing according to it.
There's a strong honor system, and even in a game, especially the kinds that are played in MIT, which are fiercely competitive.
There's literally assassin games.
You literally want to lead someone into a dark alley and shoot them.
Even then, you have to assume that whatever rules that person, who you're leading down a dark alley in order to shoot them.
Whatever rules [UNINTELLIGIBLE] they're playing by those rules.
They're not going to say-- if you shoot them, they can say, that didn't work.
You have to assume that actually didn't work, because they have rules that tell them when it works and when it didn't work.
So it's still cooperative, even though it's competitive play.
And, some terminology now.
There's systemic cooperation, which is what I'm describing.
It's the, you are agreeing that the system works.
And you are agreeing how the system works.
These are the ways how I'm going to interact with the system.
If I'm going to be playing-- Goldeneye or [UNINTELLIGIBLE] on the split screen.
Or Halo on the split screen.
And you agree not to look at somebody else's screen.
Then, that's a systemic rule.
You might also decide, no, it's on the screen, I can't avoid seeing it, therefore we can all look at each others' screens.
That would be a systemic rule.
Even though the goals are the same, let's shoot each other.
And then there's clear cooperation, which is a little bit more like the games like Pandemic and Left for Dead, where you're explicitly trying to help each other in your team in a shared goal.
And I'm just trying to differentiate them.
Because it's useful sometimes to sort of try to figure out the differences.
There's a game that I don't think we're making you play, called The War on Terror.
That we have in the lab.
And one of the things about this game is that it actually doesn't end.
If you look at the rules, it seems that there are conditions for the game end, but they're really, really hard to reach.
And if you're interested in seeing how the game ends, I'll be happy to pass you to Cory, and he can try to figure it out, or play it.
But of course, that's the point that they're trying to make, right?
It's a War on Terror, it never really ends.
It'll keep going.
It perpetuates yourself, and it's designed to be a competitive game.
Everybody tries to take down all the other states.
The other nation states, by sponsoring terrorist activity, or trying to defend themselves from terrorist activity.
Which means everybody's responsible for terrorism in the end, in that game.
But in the end, what you're hoping is at some point in time, the players are going to cooperate, because this game's not going to end.
That's the whole point of the game.
And the entire group coming to that conclusion in a shared way, is what the game designers were trying to go for.
It's a funny game.
Anybody who played Crunch, it was done by the same folks.
But it's a little bit more of the, we have an opinion and we want you to share it, then this is a game that you actually want to try to win.
It's funny.
[UNINTELLIGIBLE PHRASE] So, that's some opinions about cooperation, and that is, systemic cooperation, where you're agreeing on the rules, or you agree that the system works.
And that's their cooperation, where players are actually agreeing towards some sort of shared goal.
And there are games where it flips between player cooperation and player competition.
It's like-- Castle Crashers, who's played Castle Crashers?
It's a great game, xBox Live arcade game.
You run around, it's a side scrolling game, where a little knight with a sword--
It's a sword?
No, it can be an axe.
It can be a fish
Some sort of bludgeoning device.
Fish is awesome.
And in general, if you're playing with three other people with controllers right to you on the couch, you're pretty much going for the same goal.
There are you four, and there are a whole bunch of baddies on screen, and you're trying to kill it all.
But then, as soon as you beat the last one, the game switches from defeat all the baddies to save the princess.
And, wait a minute--
No, you've already saved the princess.
It goes to get the princess
It becomes get the princess.
There's one princess, there's four knights, and it becomes a competitive game.
All of a sudden, everything that you were using against the baddies becomes something that you use on your teammates.
And there are games that flip-flop between that.
Diplomacy is a really fluid game in that sense.
You have to assume everybody's competitive in that game.
But the only way to obtain your goals is to cooperate occasionally.
And so you can adapt.
All that's clear cooperation.
No-one's disagreeing on rules.
So that's systemic cooperation all the way through, but then there's player competition and cooperation going back and forth.
So, that's cooperation.
Finally, think a little bit about level playing field and what it means in your games, whether they're [UNINTELLIGIBLE].
In many of your games, especially card games, they're going to start with some sort of deal.
They're going to start with some sort of hand, or maybe a drawing in some sort of systematic way from a shuffled deck.
That pretty much means that you're starting out with something that's effectively random.
But then it could be also influenced by your knowledge of the game, by your skill at the game.
And you want to see how-- at what point, how do you want to balance it between someone's skill, familiarity with the game, against pure chance.
And a level playing field means different things to a lot of different people.
For some people, it may mean, no matter what skill level I am, I have a chance at winning.
So you're sort of biasing towards chance.
For some people a level playing field is, no matter what hand I get given, there is a valid strategy.
Or the goals that get set in a way that allows me to obtain it.
A lot of trick-kicking games, especially games with bidding, actually have an automatic ranking system.
In various lives that you only bid a game that you think you can win.
And the whole idea is, the game is to push your opponent to bid on something which they couldn't actually get.
So, that's a level playing field.
I can have a crappy hand, but that's not chance so much as the decisions that you're making.
So, what I would advocate, in this class, if [UNINTELLIGIBLE PHRASE] is, when you're trying to balance it out towards whatever your interpretation of a level playing field is going to be, try to pick decisions that push you in the direction of meaningful decisions.
Things that players can make a decision on, and can see the consequences.
They can understand how those consequences play out in the game.
And the outcome of the overall game is going to be resolved then.
That is, again, not necessarily the only definition.
So I want to be very very clear.
What I'm asking of you here is not necessarily the only definition of a well-designed game.
Because designing a game to be played from ages 3 to 90, you probably want a game where making a meaningful decision isn't going to bias the game to the point where [UNINTELLIGIBLE] people, or really smart because that's just not [UNINTELLIGIBLE].
And this particular task.
Try and bias it so that, in that competitive sense, you want everyone to feel like they have a chance of winning.
But I want you to bias your game towards the-- and whether you win or lose is going to be based on the outcome of these decisions.
Clearly understanding the outcome of the decision that you make in the game.
Not necessarily just what you wish , or what you were dealt.
You can still use those things in your game.
Sometimes it's the whole idea of the, oh my God, I hope it's the right card, aargh.
But then there's things like probability.
There's, how much information are you giving to the player about what the probability that you have the right card going to be.
Are you letting them look at the deck ahead of time?
Are you giving them information about how many of those cards are in the deck, so that you can actually make guesses on-- all right, we played through half this deck and we haven't seen any of those cards yet, that means the probability of that card showing up has now gone up.
And now we come to a decision that you can make.
Do I draw, do I hold-- that sort of thing.
So that's my suggestion for this class.
I would love to be able to see those kinds of games come out of this class.
So that's all the things that [UNINTELLIGIBLE] So, actually, before we move on, does anyone have any questions regarding that?
One point of administration.
Who here needs to print on the Gambit printers?
Or who here has access to other printers that you guys are fairly confident about using for your assignment?
You need a printer?
No, did you say that you need a printer or that you have access.
Because you said both?
Oh, right.
So who here needs to use Gambit printers?
Who absolutely needs to use Gambit printers.
Because we don't actually have computers set up for everybody to log in.
So I would suggest not using any of the computers in this lab because [UNINTELLIGIBLE] lab work on it.
So you've all got access to printers, that's awesome.
I think you sent out an e-mail about card stock
That card stock's actually out of stock
[UNINTELLIGIBLE]
I went to order and I was like, oh, shit.
[UNINTELLIGIBLE]
Is there like a Staples somewhere near here?
Harvard has a Staples, downtown has a staples
CopyTech has--
I went to CopyTech and they said they can print and cut your cards for you
They have a pretty awesome card printer.
Their cutter is harsh, harsh
What does that mean?
Well, first of all, it's really pointy on the corners, which is kind of cool if that's what you're going for.
But [UNINTELLIGIBLE]
Can they round it or do we have to use--
I don't think they can
They can't.
I have a rounder.
I have a corner punch that, you have to do it four times for each card, but--
We'll do it.
We can do it.
[INTERPOSING VOICES]
Just ask me and you can borrow it.
You may want to do it on some test cards before you really get a sense of how it works.
[UNINTELLIGIBLE] And I wouldn't suggest punching more than one card at a time.
The other thing is, if you're printing it on a, not like CopyTech, where they obviously know what kind of paper works best with their printers.
If you're printing it on something like a Laserjet, if you go to the paper manufacturer's website, a lot of them actually specifically state Laserjet compatible.
And the problem is actually the width of the card.
There's not the folding of the paper as it goes through the printer.
It's whether the toner will bond to the paper.
Like, does the paper get hot enough, and open up [UNINTELLIGIBLE].
Does the paper get hot enough for the toner to actually bind.
And a lot of card stocks actually don't, because they're so thick that the heat doesn't transfer well.
And as a result of that, after print it looks great when it comes out and it starts leaking.
So, that's the problem with the card stock.
Some card stocks will jam your printer, but the majority of the modern Laserjets actually deal pretty well with that.
[UNINTELLIGIBLE PHRASE]
Did you want us to print on card stock for Wednesday, or was that for when it's due?
When it's due
OK.
So Wednesday--
And in fact, printing card stock for when it's due is not necessary.
It is much appreciated.
It was a great thing for you to do for yourself, if you want to make copies for your friends or your team, that you actually want to end up playing.
To be perfectly honest, if you say-- if our printer wasn't working and all we could do were these regular white papers, if I also cut them in the right shape, so then we could just play the game, that's fine.
We're not going to take the weight of your paper and do the [UNINTELLIGIBLE]
[UNINTELLIGIBLE] the deck first, erm.
[INTERPOSING VOICES]
Has anyone played, what was it, High Society [UNINTELLIGIBLE]
Yes
Those are awesome
They were like kids' books, you know?
[UNINTELLIGIBLE]
If anyone does have issues printing, before you jump straight into the [UNINTELLIGIBLE] talk to Jason and I. Jason usually has room by the TV lounge, my room is just down the corridor.
And we might be able to help you with that.
But in general, if you've got access to other printers, go right for it.
Because going through us is a bottleneck.
Again, not for Wednesday.
For Wednesday just have something testable so that we can [UNINTELLIGIBLE] before the date it's due, try to print something.
You don't want to wait until the last minute and you don't want to suddenly realize that your paper doesn't work very well with the printer at the last minute.
So, today's reading was Bartle and Chapter 11, I think, of the [UNINTELLIGIBLE].
And in general, both of them are asking the same question.
Which is, who are you designing your game for?
On the assumption that you're not designing the game for yourself.
Of course, realistically, if you're going to get a job making games, the majority of the games that you're going to design are not games for yourself, they're going to be games for some other demographic.
And the Bartle paper is a little bit more influential in how people think about games, because it shifted the question from, what's the age group, what's the demographic, what are people out there who are already buying games going to want?
And you fit that into, what sort of things are people looking for in a game?
He doesn't actually take it to the next step, which is where-- I think he wrote that in the 70s or late 80s, does anyone remember?
In the 90s
I think it was in, like, 1990
[UNINTELLIGIBLE]
But a lot of the ideas come from the 70s, actually.
Because that's when he made the first MUD.
RIchard Bartle [UNINTELLIGIBLE]
What exactly is a MUD?
They never actually described it.
They just assumed everyone knew what a MUD was
MUD stands for Multi-User Dungeon.
Anyone want to describe one?
It's like, an online text-adventure thing, where everyone's interacting with the same text-based dungeon thing
The best way to describe it is, imagine if WoW was presented entirely [UNINTELLIGIBLE]
I found that kind of annoying in the text.
He just assumed everyone knew what a MUD was.
But never actually said, this is what a MUD is
Yeah.
Because the whole sort of, reading the article, actually I believe he formatted that for a forum post for for community.
So there's an assumption.
But yeah, apologies for that.
Actually, if you're interested-- and here he does define it right at the bottom.
Which is right at the end of the article here.
He talks about specifically, this is what I mean by MUD.
Somewhere in the footnotes--these text adventures, text-style online games where multiple people are interacting.
And if you look at the rest of the website, there's actually a bunch of stuff about how MUDs work.
But, for the most part, that was sort of online multiplayer gaming, back when he started.
He's credited for having built the first MUD.
And, frankly, not all that different from MMOGs you get today.
The difference being that they were all text.
But you still went out hunting monsters.
They call them mobiles, or mobs.
And in fact there's still some of that terminology going around.
They drop loot.
You will pick up loot and show it off.
And sometimes, some games [UNINTELLIGIBLE] loot.
Most games let you exchange loot.
And then you would use your loot to go and kill other monsters and get better loot.
Sometimes you'd do it with other people.
Sometimes you attack other people.
Sometimes you just hang around and chat and talk about things that are completely not in game, like what did I have for lunch today.
[UNINTELLIGIBLE] And sometimes it will take multiple people to take a monster.
And it was just handled on a how fast can you type kind of thing.
Attack monster, attack monster.
I think you can just hit repeat and but, the behaviors are also not all that different.
And how he describes it is, a game will sight itself in a position where it is variably appealing to various kinds of players.
And that's how he described it.
Where game studies has kind of gone on to since then, is the realization that just because you're-- some people through all the four types that you--
Explorer.
Killer.
[UNINTELLIGIBLE]
So, there are these four types of players.
And just because you're an achiever in one game doesn't necessarily mean that you're going to be an achiever in another game.
Different people play different games, or possibly even different types of the same game, for different reasons.
And that's where we go back to earlier concepts that we brought up, is [UNINTELLIGIBLE] kinds of fun.
What are people trying to get out of their game playing?
So, but he was-- if you're looking at the content of any one of MUD, and a little slice of what you saw of somebody's personality, [UNINTELLIGIBLE] and all the things that they were doing.
And the kinds of things that you tended to keep doing every time you logged in, it was very easy to stereotype them into somewhere along those lines.
Even he acknowledges that nobody's really all the way on one of these layers.
Griefers, I think, are the words that I hear generally used for killers nowadays.
But even then, that's not quite the way that he's using it.
He's actually using it as somebody who's generally still playing by the rules of the game.
They are there to cause misery to other people, but they're not there to break the game.
There is a class of people called griefers, who are deliberately trying to find ways to exploit the system, to just make the game just kind of silly.
But all the other dynamics that he draws on are very much there.
You get a whole bunch of socializers together.
People who want to do nothing but chat and have a good time, they don't really care about the game, and there's going to be a whole bunch of people going right after them, saying these are great people to grief.
Because they are enjoying themselves so much.
We could end that.
Second Life gets a lot of this, because Second Life has that whole, you can do anything you want.
And socializing is one of the big things.
Personalizing is one of the big things.
So, do keep in mind that the way that he's talking about it, sounds like all of these different personalities are somewhat mutually exclusive when really the discussion has now come to, how do you cater to all of these different desires at any given time?
I may be an achiever today, I may be an explorer tomorrow.
And he even talks about explorers thinking that achievers are beginner explorers, because it's like, once you've already beaten the game, what else is there left to do but explore what else you can do?
So he himself acknowledges that there's some personality transfer that can happen.
But it's kind of interesting how he talks about the feedback loops of people.
You'll get a whole bunch of achievers, and you get a whole bunch of socializers.
[INTERPOSING VOICES]
[UNINTELLIGIBLE] killers, and achievers--
[UNINTELLIGIBLE] you've got the competition [UNINTELLIGIBLE] But he also makes this weird assumption that there is a finite pool of people who are going to be playing the game.
Which is actually true, even in a 10-20 million player game like in World of Warcraft.
All those things have to be-- we don't have 20 million people logging onto one server.
They say, I have [UNINTELLIGIBLE] or something.
[UNINTELLIGIBLE]
Exactly, exactly
And I'm like, once [UNINTELLIGIBLE] you can start [UNINTELLIGIBLE] that are just really big numbers.
Like, 25-- it's currently 25 is the highest.
They used to go to 40.
But yeah, and you just kill bigger monsters and get lots of loot.
And then there's PvP, which you go-- the main thing right now is arenas.
And you have the team of two or three or five, and you fight other players with your gear by doing battlegrounds and [UNINTELLIGIBLE] arena.
And it's basically, Wo
I can't remember.
Are there role-playing servers?
Yes.
There are role-playing servers.
I have actually never visited one, so I can't really talk about that
That's a lot of dancing
[UNINTELLIGIBLE] slash dance, maybe, or your character dances.
And it was those things that really is popular among socialized type players all over-- and Diablo.
But it's not just [UNINTELLIGIBLE].
I mean, slash dance means other things now.
It could mean I'm bored.
It's like, I have nothing to do right now, I have to wait for this stupid thing to respawn, slash dance.
It's more fun than just standing there.
But there is a group in WoW.
And I guess-- maybe not a group.
There are folks who play World of Warcraft and they engage you to think on theory crafts.
Which is, how we optimize the way how you build a character.
And the things that you equip a character for, to make that character really, really good at doing one or two different things.
So as you're building up a character that's designed to heal, what class should they be, what race should they be, what equipment should they be getting, depending on what [UNINTELLIGIBLE] you are in the game.
And they're going to try to derive as much of the math of the cause and effect in the game as they possibly can.
There is actually [UNINTELLIGIBLE] which is-- [UNINTELLIGIBLE PHRASE] It's actually kind of funny.
The Elitist Jerks' forums had one guy who just happened to write a forum post about dressing up in real life.
Like, this is how you get really nice [UNINTELLIGIBLE] because he worked at a men's fashion place.
And it it turned out that that community was really, really, really interested in learning more about that.
Which is actually not all that weird.
It's like, you care about what armor you get in the game.
You care about what you wear in real life.
Especially if it might have some impact on your real life.
What job that you're looking for.
OK, you know.
So, I would actually put those more in the explorers part of things.
Sure, they're looking to optimize.
They're looking to do their best.
But they're not looking to be necessarily rewarded by the game system.
They're using the game system to test their ideas against.
It's like, I'm looking for a better way to suck up more damage, because I'm a tank.
I'm supposed to be absorbing all the damage.
Or the other as they call it.
Or the attentions of the characters being controlled by the computer, while the rest of my folks put out a lot of damage, can do their job.
But I am not interested in what the system has deemed to be, you have absorbed tons of damage, good job.
I don't care what the system says.
I care how the system operates, so that I could optimize it through some sort of metric that I've figured out and my friends have figured out and the community has figured out.
So I want to understand the system.
Again, that's what it comes down to.
The metric that they're using is, how do I optimize it.
But for the most part, people who engage in that kind of theory craft, it's like about this armor combined with those pants, I've proven something, those are well in the area of explorers even though their behavior seems very much, that's the best solution.
Whereas achievers are very much focused on, how do I get to level-- it's like, how do I get the coolest mount.
In World of Warcraft you can ride around and every once in a while, Blizzard, the company that makes it, will release some new stuff.
And the achievers will say, I've got to get that.
Because the space [UNINTELLIGIBLE] it takes a lot of effort and a lot of money to be able to get one of those things.
But to be able to get it, it gives you some recognizable status.
You will see someone with the limited edition of some kind of mount.
And you'll know that [UNINTELLIGIBLE] is to spend a heck of a lot of time in this game and play the game really well, that's the kind of recognition they're looking for.
So that's the difference between achievers and explorers.
So I want to try one little activity.
Which is, I've got the chart that Bartle lays out.
The whole player versus-- interacting and acting, player and the world.
And he uses this to describe where your game could be at any given time.
If you're acting on players that your game's biased towards killers.
If you're designing your game to be more about interacting with other players, then you're somewhat biasing it towards socializers.
Interacting with the world is more for the achievers.
Acting on the world is more for the achievers.
Given these quests, I've beaten all the quests.
I have 100% of the quests complete.
That's achievers.
And interacting with the world is more of the explorers.
It's like, what else can you do in this world?
There are these goals that are set, that's great.
But how does this world work that I can do cool stuff
[UNINTELLIGIBLE] distinction between acting and interacting, incredibly [UNINTELLIGIBLE], I didn't really understand what he was trying to say
It was [UNINTELLIGIBLE].
I think he came across those terms because they sounded cute.
Acting, interacting, is like a--
I mean, can you interact without action?
Well, [UNINTELLIGIBLE]
What does that mean?
And, like, the really classic example he gives.
Like, the acting on a play is like a pure [UNINTELLIGIBLE] the other player fights back.
There's an interaction there
Sure.
But the thing about it--
But the difference there--
So I believe what he means by that is that the killer is actually less interested in what the player does back to him.
In some sense, they are interacting because they're trying to see what sort of response they get.
It's like, I'm going to do something to this player.
This player's either going to die, or fight back, or whine about the fact that I'm hitting him so much.
But they're not looking for the competitive bit.
They're not really interested in the, and now that person's going to turn around and attack me
I know, but the whole point is supposed to be that killers are griefers, so they really care about how people respond to them.
He says later on that one of the most annoying things you can do to a griefer is just ignore them and not rise to the bait.
So interacting seems to be the crucial thing.
Again, I feel like his-- [INTERPOSING VOICES]
I kind of see it as, does the person care about how the other works, in a sense?
And so it is the case that a griefer wants to get a certain reaction.
But beyond that it doesn't matter.
So in the same case within the game system, I want it to do this thing, I want it to give me points but why it did that or how the mechanic works is kind of irrelevant.
And that's how I understood it
Well, in particular, with the interacting, when somebody is like trying to player kill or something, the reaction is not necessarily-- the reaction they're looking for is to piss somebody off, right?
And that reaction is not necessarily part of the game.
So, it is acting on them because you are taking what the game allows you to do, and using this against a person.
But you don't care about what they use in the game against you.
You more care about this abstract idea.
So I think the interaction is not actually part of the game
[UNINTELLIGIBLE] specifically to do with game mechanics, whereas interacting is to do with something which is-- so maybe acting is something mechanical, whereas interacting is something dynamics-related?
Or?
Well, I see is as like, acting is sort of like a single direction with mechanics.
So, for example, the mechanic there is being able to hit somebody in the head with your [UNINTELLIGIBLE].
Whereas interacting would be the case where you are hitting each other on each others' heads with your swords.
Which is an entirely different thing.
And it's more of a competitive thing rather than the, I just want to make you angry
I think the unidirectional perspective makes sense.
You could think of it as going with the hitting on the head example.
If I'm acting, then I care that information is flowing one way.
I am dealing damage to you.
If I'm interacting, I'm curious as to how that is computed and what that means for me in return.
Does that make sense?
So like, if I'm really bad at terminology-- [INTERPOSING VOICES]
A bunch of other examples of things that you do to maximize interacting versus acting.
There might be interesting examples of things like make super [UNINTELLIGIBLE].
And then the acting ones are like, make [UNINTELLIGIBLE]
So, in the context of a MUD, a puzzle is often like a door.
So you've got to solve this puzzle if you want to get into a new area.
Because that's really the only thing-- sometimes you get your loot.
And the idea behind that is, you are either privileging it to getting people to talk with each other more and socialize with each other more.
Or you're privileging it to getting people to look at the world more, see more of the world and understand how it works.
Then in a sense, you don't want to put too many barriers between the people who've been in for a very long time and the people who just started playing, the people who are in one area of the world and people in another area of the world.
You kind of want quick flow.
So in MUDs, that's one thing where the definition of puzzles is very specific.
It's like, but I have to move this set of levers
Another point that I've found that's really strange, because I would expect an explorer, who is supposed to be a world interacting person to really enjoy deep puzzles, which reward them for knowing more about the world
But a problem with puzzles in MUDs is that they're usually simple solutions.
So there's less pleasure to be found in a very difficult puzzle, because all that told you was, you just solved this one thing.
There is still some pleasure there.
The interest that explorers are way more compelled by-- kind of the latent dynamics-- is understanding the interaction of how mechanics-- especially [UNINTELLIGIBLE] in a way that might not have been originally designed that way.
That's what they're looking for.
They're not looking for, what is the solution that was designed in the game.
They're looking for, how am I going to get this game to do something that it might not have been designed to do originally, but would be cool once I figure it out.
So that, unfortunately-- you are definitely right.
But that one suffers from the very specific terminology of what a puzzle is in a MUD, as opposed to the more general definition of what a puzzle is.
The reason why, at least for achievers, hard puzzles are very important, because if you make a puzzle that most people aren't going to be able to get, being able to get that puzzle gives you some [UNINTELLIGIBLE] status.
Usually some sort of visible reward, even.
You are now in a location not everybody else can get to, or you now have a piece of equipment that no-one else has.
Whereas explorers, they will probably have an advantage in solving difficult puzzles.
But they're not necessarily the sort of thing that makes them want to interact with the world.
Lots of the mechanics that interact with each other in and new and interesting ways, are more likely to be interesting to an explorer.
But you're right, acting and interacting are crappy, crappy terminology sort of thing
Are you talking about how griefers are more like actors, where they just try to get people to interact with them, or try to get people to get annoyed by them.
I've definitely seen, [UNINTELLIGIBLE] game design.
But [UNINTELLIGIBLE] 300.
Where we watched a video of griefers in Team Fortress 2, where there's a certain door where, if they stand in a certain spot, it won't open.
So this person started making everyone interact.
OK, we're going to play a game today.
If you want to leave, you have to answer correctly
Yeah, what is the capital of--
That I see as more of an explorer things.
Like, you bounce some weird combination of the interaction with the mechanics that allows you to do something new, which is turn Team Fortress 2 into a game show
It was pretty funny
But people who can't play
Griefers and explorers are a combination of bottle types, if you're going to be a successful griefer
And that's absolutely true.
If your personality's oriented one way, you may have to engage with some other behaviors in order to be able to maximize that core behavior they're looking for.
So there are people who are looking at exploits and trying to understand how the world works so they can cause maximum damage and maximum grief to other people.
That's definitely one good way of doing it.
But what I find is that as soon as they find it, they are not interested in finding more ways until that first way gets boring.
Or gets fixed.
So, people are going to say, there's a lot of teleporting you directly in front of a turret, stuff like that.
Teleporting your own teammates in front of a turret, that sort of thing.
[UNINTELLIGIBLE]
I don't understand
You don't understand?
[UNINTELLIGIBLE]
It's fun
Griefers
No, it's not
So, I'm going to mess with you.
Just for the hell of it
I remember in Sims Online, an effective way to grief would be to organize a band of people and arrive on that property.
Since there was a population limit of 12 people or something on any given lot.
In that lot, was a business that required a lot of people to come through.
And then you just had 12 of your people show up and mill around.
And say a lot of profanities.
You could tag the place and lock it up and no-one else could go there and visit it, sometimes including owners and stuff like that.
It was very frustrating
In fact, some of the behavior they did got towards organized crime level.
Actually, I mean it wasn't real crime, it was virtual.
But there are actually a good number of articles about the Sims Online mafia.
Because that game had so much built in for-- actually for socializers-- to be able to communicate, express themselves.
I mean if you played the original Sims, there's a whole range of different expressive things that you can do to communicate how you're feeling and to be able to chat with people.
And someone, I think it might have been Maxis itself, actually charted out the web of influence of all the players.
And there was one alpha player who was in charge of everything.
And if she decided that this person wasn't wanted, she didn't want, spending too much time [UNINTELLIGIBLE] because of how it pissed her off or piss one of her friends off or something, she'd organize something like that.
Effectively.
It was called the Sims mafia.
and you get that [UNINTELLIGIBLE PHRASE] And it's kind of interesting to see how different worlds respond to that.
Even though in [UNINTELLIGIBLE] which is-- actually, one thing I'd like to do is, I'd like to throw different games up on that chart.
So I am looking mostly at multiplayer games.
Preferably online multiplayer games.
But, and I don't know all the games that are out there.
So, I'd like people to call-out games.
And tell me roughly what I should put up.
If you disagree, then by all means put up your hand and suggest that it should be moved
Minecraft should go definitely toward world interaction andexplorer type
Like, [UNINTELLIGIBLE] far, far
So what is Minecraft?
I've heard of it, I just don't know what exactly it's about
You basically go around mining things and creating objects, and you can create your own worlds
So it's like-- I guess?
EVE is definitely world interacting as well
Yeah
So that's same as
I would say EVE is also player to player interacting.
[INTERPOSING VOICES]
I'd see it as in the middle, then
In EVE you have to form corporations with bunches of people in order to do anything
That's true.
You have to be really good in your social skills, actually.
It's one of the games that's most ruthlessly [UNINTELLIGIBLE].
[INTERPOSING VOICES]
If you interact with the people and the world a lot
I think it's more strongly player attracting and is world attracting than most games
You cannot [UNINTELLIGIBLE]
But there's also a huge amount of achievement and player versus player conflict in this kind of stuff.
[UNINTELLIGIBLE]
You can die far down the interacting scale.
I mean, consider things in Second Life instead.
Which would be way further down the interacting end of the spectrum than EVE Online, surely
[UNINTELLIGIBLE]
Assuming that [UNINTELLIGIBLE]
I'm not sure it's hacking.
Like, PvP is the opposite of [UNINTELLIGIBLE]
I mean it's not--
I think it's definitely player interaction more than player action
Yeah, I believe it's interaction
Something like having PvP tournaments, I think, would be definitely a lot of interaction
Yeah.
I think that's the distinction between griefing and like people fighting each other
Griefing can be an [UNINTELLIGIBLE]
What I got from this article is that acting is like, I go and kill someone and ha, I killed them and that's it.
Or I go through the world, it's like, I killed a monster, I killed a monster.
And you just have a goal.
If you want to get the goal instead of diverting from your goal.
So, I don't know.
I definitely think griefing can be hugely interactive.
But there could be a whole bunch of-- in EVE if you infiltrate someone's company and you--
Steal ten thousand dollars' worth of their money
Exactly.
That's hugely interactive griefing
I don't think that's griefing
Consider Team Fortress, right.
There's Team Fortress player interaction.
Because that means it's a socializer game
I personally do not think Team Fortress falls under player interacting with team.
[INTERPOSING VOICES]
But Left for Dead is
[UNINTELLIGIBLE]
There's definitely interaction happening-- [INTERPOSING VOICES]
I'd say probably more-- it depends on what you're doing [INTERPOSING VOICES]
So with Left for Dead you can play-- you're interacting, but then you're also online and you can go against other players who are actually the zombies, against you
Right.
So let's talk about the basic [UNINTELLIGIBLE]
So the basic, most common mode is going to be four players playing cooperatively against the game world
That's just interacting with the world
And there is a lot of interaction that goes on between players, because you cannot basically survive unless you get each others' backs
So I would say it's--
And there's built-in functions to help other people.
So someone might get back up on a ledge and he'll be holding on and you have to pull him back up
Or smokers
At the same time, there's also an intelligent engine that plans that level for you and places zombies specifically to maintain you in a state of nervousness.
And so it's different every single time you play.
So those balance out and probably place it somewhere in the middle.
Maybe a little bit more player biased than world biased, because you really have to stick together
So, it's weird because it's almost like, you want to have like a plane.
[INTERPOSING VOICES]
I think the big problem with this sort of graph is that you can't really necessarily separate games into one of these quadrants.
[INTERPOSING VOICES]
And that is actually also-- [INTERPOSING VOICES]
It might be a little more useful to actually draw areas within the graph to represent a game, I guess.
Draw the area that encompasses all the pieces the game represents.
[INTERPOSING VOICES]
Like Halo co-op and Halo multiplayer, and player versus player.
That will probably be a little refinement, if I create some sort of scatter chart or Venn diagram
Because I could definitely see there being a game where there's a lot of acting on the world and interacting with players.
And if you're just separating these into distinct points, that isn't possible with this graph
So that's another problem, is the assumption that acting and interacting are--
Mutually exclusive?
They're mutually opposite.
[UNINTELLIGIBLE] Do people generally accept this?
No, no.
They do less acting than interacting
If you just warp space, then you can have [UNINTELLIGIBLE].
[INTERPOSING VOICES]
I think if you had different values for player world interacting and acting would necessarily be connected.
I think EVE definitely in world is very high, and in player is very high
World very high, player very high.
But I mean, player is interacting and world is acting on the world
On the world still.
[UNINTELLIGIBLE]
I mean, because-- you're exploring the world spatially a lot.
Are you exploring the systems of the world a lot?
Or are you really exploring the systems set up by people?
That's true
The nice thing about things like EVE Online-- also [UNINTELLIGIBLE] EVE Online [UNINTELLIGIBLE] game basically, where you start off as a captain.
Correct me if I'm wrong with this
It's a super-sandbox game
But for the most part, you're traveling in space
Can you get out of your ship and walk on foot?
Have they added that, because I know they wanted to add an expansion where you could actually walk on planets.
Last time I heard, they hadn't
Last I heard they hadn't
Dust 66 or something like that
[UNINTELLIGIBLE PHRASE]
It's a separate game
No, you're traveling through space.
[UNINTELLIGIBLE] But the whole idea is that it's all built around this economy, where [UNINTELLIGIBLE] mining, [UNINTELLIGIBLE] cash.
But the real power in the game comes when players start to ally with each other.
Form armadas, form trade groups.
Form areas that you can patrol and control.
And areas where you want to agree that if you don't get along with the people who are in control of this space, you will die when you fly in there.
And the only thing that's making that happen are other players deciding that this social contract is [UNINTELLIGIBLE].
So it's pretty damn amazing.
What some folks were talking about earlier was a big story about how, basically, one person got into a very, very powerful position in one of these clans.
And decided to defect.
Take a majority of one clan's financial resources with him.
And-- remember, this is a fairly modern massive multiplayer online game.
In-game assets can now be traded for real money
It's illegal so that--
It's illegal
Like, in-game.
So you can get your account banned.
So what happened, if I'm talking about the one that you're talking about.
Someone got to a really powerful place in this banking clan.
And then took all of their assets and traded it for real money.
So he got $10,000 by selling it off
[UNINTELLIGIBLE]
One was a guy who infiltrated the clan [UNINTELLIGIBLE] that much money.
One was like a parent who was going into financial trouble and held a fifth of the assets of the major bank, and cashed it in for $6,000 to help pay off part of his house and his kids' medical bills.
And then posted an apology on the board and quit the game
I thought he got banned from the game
He was banned from the game, but he also quit the game
[UNINTELLIGIBLE]
He was like, I'm sorry, everybody, but this is kind of important
The amazing one was the guy who actually infiltrated the company and just took it down as a years-long operation--
[UNINTELLIGIBLE] thought he was a friend and all that
Yeah.
Trying to get in and then managed to take down this gigantic in-game corporation
How many active players are there on EVE Online?
A hundred thousand?
No.
I feel like the active players are--
[UNINTELLIGIBLE]
At any moment?
No, I mean active current players who have active accounts
Do they have new players arriving regularly, or--
Occasionally
They don't have a game that's been-- [INTERPOSING VOICES]
It's like a ridiculous learning curve, yeah
It's like four months or something.
It's insane
But the interesting thing about the particular case where someone infiltrated, took all the money and basically decimated this clan deliberately, was that that person [UNINTELLIGIBLE] [INTERPOSING VOICES]
The most recent one that I read about, theoretically had in the tens of thousands of US dollars of assets in the game and did not cash in any of it for real money because there is no authorized mechanism.
And the article that I read, basically, the end note was, no-one knows what he can do with it.
He's technically rich but there's no legal mechanism [UNINTELLIGIBLE] I don't think he got banned at all because--
Its part of the game.
At least for the definition of EVE.
Stories like that tend to have earned its reputation in a way.
This is lawless in space.
We're trying to encourage this kind of player to player.
And [UNINTELLIGIBLE] So I'm not entirely sure if [UNINTELLIGIBLE] griefing.
It was an incredibly smart tactical--
See, I would not call that griefing.
It may have been done to annoy the people who were running the company that he took down.
But see it as exploration.
I'm going to try and-- it's [UNINTELLIGIBLE] player interaction
[UNINTELLIGIBLE] [INTERPOSING VOICES]
I started a major corporation all by myself
Yeah, might actually be-- [INTERPOSING VOICES]
Do you put that on your resume?
Single-handedly [UNINTELLIGIBLE]
[UNINTELLIGIBLE] been hired by a lawyer
[UNINTELLIGIBLE]
So, what I would actually like to end this with-- we've got about 15 minutes.
Is to spend some time-- I would say, how many of you have we got?
One, two, three, four, five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 16.
So, groups of four.
And what I'd like folks to do-- [UNINTELLIGIBLE] -- I'd like folks to come up with other axes.
Not necessarily anything having to do with [UNINTELLIGIBLE].
Give me two things that you feel are diametrically opposed but in describing a game, or any series of games.
You can place games on other sub-axes.
If we have time, we can talk about an attitude difference.
But something to help describe two people, OK, this game is on the hard extreme of one, this game's on the hard extreme of the other.
The goal here is to come up with terms that will help you identify who this game might be target towards.
What kind of player might this game be targeting?
Not necessarily using Bartle's terminology at all.
Bartle's terminology, as we have said, is kind of weak, so let's throw out some other words to describe games already.
If that fails, because you haven't really thought about it, to come up with some new words.
Real game-ish words.
[INTERPOSING VOICES]
Random versus strategy
Oh, very good
Original versus copy
[UNINTELLIGIBLE PHRASE] I'd rather use more chance or random to get the point across pretty clearly
[UNINTELLIGIBLE]
Right.
So I think that axis should be random, deterministic and/or reactive and deliberative.
[INTERPOSING VOICES]
It's fancy
[UNINTELLIGIBLE]
Say that one again?
Reactive
So reactive and delibrative
Reactive and deliberative
Or random and deterministic
So you can definitely still have randomness and have strategy
Right
But random and deterministic
They would be somewhere on the axis, right?
So the problem is, you can have incredible amount of randomness but if you can just figure out the probabilities, you can still have an optimum strategy, right?
But then that's less random
In doing this, you can't have a system that is so complex yet deterministic that there's no way you can predict what's going to happen next
Yeah.
Killer robot
[UNINTELLIGIBLE]
It's theoretically possible, but this is-- [INTERPOSING VOICES]
Player versus system agency?
Player versus-- also system-- player versus system agency is kind of your--
It's distributed
It's the name of your tactics, right?
Yes.
[UNINTELLIGIBLE]
You also had a low learning curve versus high learning curve.
You're definitely not on both ends of that, and--
Fast versus slow
[UNINTELLIGIBLE]
I like that idea.
Fast versus slow
So, another axis you may have was you could call it freedom and predetermined paths in the game
[UNINTELLIGIBLE]
Huh?
[UNINTELLIGIBLE]
Freedom versus constraint
I can't remember the terms, but ludus and pitheus, something like that.
[INTERPOSING VOICES]
So one more that I use here often, [UNINTELLIGIBLE PHRASE] [INTERPOSING VOICES]
13 hours of Final Fantasy--
[UNINTELLIGIBLE] [INTERPOSING VOICES]
For 60 hours.
[INTERPOSING VOICES]
Sandbox, I would say, [UNINTELLIGIBLE] is a very broad one.
It's one that people generally understand as embodying a sort of freedom.
You get to dick around, basically.
You get to play Grand Theft Auto totally not following the storyline.
And-- actually, it kind of goes on, it becomes a little bit less of that, you can get to do , you get to play but the whole idea is that that world has ramps everywhere, so that you can drive cars and vehicles [UNINTELLIGIBLE] Anything else?
There's realism versus attraction
Oooh
What?
So the difference between EVE Online, for example, which is supposed to be very realistic, simulating something.
Whereas poker, for example, is just cards and numbers
Something on the freedom sandbox versus [UNINTELLIGIBLE] on rails, there's a very interesting thing in [UNINTELLIGIBLE PHRASE] there's a feature in one of the games where you were told you need X amount of money to be able to pay for the ship out of here.
And you're placed in a sort of sandbox environment.
And you're given-- you're not told, but-- whatever ways you can think of to make that amount of money.
The amount of money you need to leave that part of the game.
So it's a predetermined sandbox
[UNINTELLIGIBLE] a lot of adventure games, a lot of roleplaying games, are largely [UNINTELLIGIBLE] areas of freedom.
Even in a game like Final Fantasy, it is not something [UNINTELLIGIBLE] in other games in the series, the story is largely predetermined.
But what you do in combat is very, very fluid.
And that's kind of like the dichotomy that [UNINTELLIGIBLE] Square Enix, or Square in the past, have decided that that was OK.
This helps us cater to people who want some flexibility in how they're going to play, by making the combat system fluid [UNINTELLIGIBLE] they want.
But also lets us cater to people who really want that long, over-arching, epic narrative.
And we can only pick one.
We can't branch [UNINTELLIGIBLE PHRASE] Are there others?
I mean, [UNINTELLIGIBLE] a lot of [UNINTELLIGIBLE PHRASE] puzzle games there's only one way to do it, whereas [UNINTELLIGIBLE]
[UNINTELLIGIBLE]
I mean, you could have multiple solutions, I guess
Multiple and single
[UNINTELLIGIBLE]
I'm not so sure if that tells you [UNINTELLIGIBLE] if I tell you something about the constraints on [UNINTELLIGIBLE] budget of--
I prefer games that are much more [UNINTELLIGIBLE PHRASE] And I guess it goes back to [UNINTELLIGIBLE]
Because the interesting thing about multiple solutions and single solutions is that multiple solutions-- it could be all design.
I could design in the game-- in fact, now, if you play Deus Ex, or any of the sequels, they specifically stated that they made this game with multiple solutions in mind.
And with the data sets to, they designed it so that they thought, this puzzle needs to have three solutions.
And they actually will design an environment for it.
So, while the game kind of tells us, you get to play with the system, you get to figure out how all these things combine and come up with solutions on your own.
Really, they tried to design it so that you have three very clear options at any given time.
[UNINTELLIGIBLE] And so you can have multiple solutions.
Whereas this is like freedom sandboxing, kind of applies not necessarily to design.
You've got lots of little things that you can do and lots of different spaces that you can explore and design.
It's not like you've got one obstacle and three different ways to solve it.
You've got all kinds of different things that you could be doing.
You could be like [UNINTELLIGIBLE].
So these are different axes.
I just want to point out that these are different ways of doing different things
You can have, in terms of rule changing games, whether it's dynamic rules or static rules
Hmm.
So, hyper-dynamic, static, and then [UNINTELLIGIBLE]
Are the rule going to change if [UNINTELLIGIBLE]
There are some games where the whole point is changing rules
Like Flux
[UNINTELLIGIBLE]
[UNINTELLIGIBLE] [INTERPOSING VOICES]
And assuming basically [UNINTELLIGIBLE]
Timescale of decisions
So, short model?
Yeah, like twitch games versus thought games
[UNINTELLIGIBLE] decisions.
Sort of like that are contemplative and reactive.
[INTERPOSING VOICES]
Are you guys going for basically the same thing.
Like, reactive means things are coming at you and you have to respond really, really quickly.
Maybe [UNINTELLIGIBLE] about it.
And deliberative is like a trivia game
[UNINTELLIGIBLE]
Doesn't necessarily mean that.
The kind of turn-based game is very fast-paced.
Because the weight of any given decision is not going to be so deep.
So, [UNINTELLIGIBLE] I'm just going to put this up here because I'm running out of space [UNINTELLIGIBLE] that's one way of describing a game, reactive versus deliberative.
[UNINTELLIGIBLE] I bet you could take two of these and put them on an axis and then we could talk about them.
On a [UNINTELLIGIBLE] but every once in a while we find a game that [UNINTELLIGIBLE] Which is fine.
Which is part of [UNINTELLIGIBLE] to talk about these games.
These are really, really useful terms.
[UNINTELLIGIBLE]
[UNINTELLIGIBLE] [INTERPOSING VOICES]
[UNINTELLIGIBLE] it's a little bit more boxing, it's a little bit more restrictive than predetermined.
[UNINTELLIGIBLE] sandbox and rails are more like [UNINTELLIGIBLE PHRASE]
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Before we move on to the rest of the topics, maybe just some administrative stuff about the next assignment.
So the idea is to model a system, right?
So the idea is to pick some sort of dual system.
In the past people have picked stuff like applying for college because they knew all about that.
People have talked about any type of idea, you know, running a nuclear reactor or something.
Specifically there's a-- but hey, any of you nuclear science majors?-- oh, perfect.
One of my favorite Apple II games was actually nuclear reactor simulation, and it always melted down.
That's why it was fun.
But pick a real world system to start modeling.
Obviously, you're trying to find the fun.
Obviously, you're trying to make something that people are going to enjoy playing.
But the idea is to start from a slightly different place from where we had in the first assignment.
Now a couple of you guys already did that for the first assignment, though.
I mean things like picking up girls at a party.
You know, modeling real systems and things.
Still haven't played yet, a genealogy game yet.
So, we're going to play next March.
So [UNINTELLIGIBLE].
The survival horror game, I would say it's not a real world system.
It was a system.
Horror, wizards, trolls.
So, yeah, pick something within games negotiating around-- like negotiating greenhouse gas emissions, that was one which was really, really cutthroat.
That's what probably you imagine.
Actually I'll try to get that team to come in later in the semester.
They did this and like talk about their card game.
It was a freshman team on Terrascope.
Let's see, On Wednesday, we're going to do the usual brainstorming inside of the second half of the upper-class.
You know, everybody throw out your ideas.
Come in with a bunch of ideas, we'll try to build on that.
But this time around you're allowed to actually pre-build your team.
You're allowed to ask people, hey, you guys want to work together?
You don't actually have to figure it out on the spot if you don't want to.
I don't really care how you do it, as long as by Friday you know who your team is.
And who's on your team.
Team sizes are the same.
So, any questions?
Are we allowed to simulate things that aren't from this era?
Yeah, like normal history?
Yeah
OK, not from like the future.
[INTERPOSING VOICES]
Yeah, I'm serious.
This is how I think [INAUDIBLE].
OK, yes, absolutely there are no historical stuff-- but you do have some liberties.
I'm not saying that your game has to be a faithful simulation.
You can even build your model simulation [INAUDIBLE].
But it doesn't have to be a faithful simulation, that's certainly a poor design.
But, you know, in a sense you're trying to capture the spirit of it.
If the point is, I'm running a nuclear reactor.
And you can sort of capture the, oh my God, it's so dangerous , or the, oh my God, it's so boring aspect of it.
Have you actually put the nuclear reactor--
: Well, like, what's there.
And I know the mechanics of it, I think, were there
I have been told it's not necessarily the most exciting job in the world
No, you just sit there and do your homework.
And every once in a while you look up.
Very reasonable.
And you get paid for it.
Like $12 or $13 an hour
So if your whole idea of the game is let's try to get as much homework done without letting [INAUDIBLE].
That might be more entertaining than just looking [INAUDIBLE].
But I got my problems in then
[UNINTELLIGIBLE].
You have to do [UNINTELLIGIBLE].
[INTERPOSING VOICES]
And besides, I think we still have a time limit, like half an hour.
It's not [INAUDIBLE] homework, it's really easy homework.
Really not homework.
Let's see, just a reminder for everyone who has been falling off on the foreign posts, please make sure that you keep up on your foreign posts.
Those things actually do affect your grade.
[UNINTELLIGIBLE] one person, not one team.
If you have any-- since we're in between reading assignments right now, you can write about the assignment.
But if you wrote up something, let's say a reflection on how the class assignment went, that's fine.
We're not expecting more than one foreign post a week.
Of course, feel free to comment on other people's foreign posts as well.
If a discussion or a point has been raised that's kind of interesting, you want to kind of develop on that.
Put some thought into it.
Write something interesting you're going to keep on the compilation, that counts too.
And that's fine.
OK, so puzzles.
So Brenda's, before we start with Brenda's, let's start with Scott's.
What were some of the definitions, or some of the things that Scott's game was mentioning, that were good for puzzle design
The book or not in the book?
This was on the web page.
[INTERPOSING VOICES]
--and had the right answer.
[UNINTELLIGIBLE]
He didn't really differentiate between puzzles that could be generated like a pseudo-GUI or something, where you kind of have an algorithm that lets you create multiple puzzles.
But as long as that particular iteration had one answer.
So that was really there.
Anything else you remember?
Tactical difficulty.
Make it accessible but not super easy.
I think targeting a specific audience.
It was also the ability to change difficulty, it was dynamic difficulty.
So some areas may actually be easier settings, and then actually what they want to be is harder settings
I think that was Brenda's.
But, yeah, am I wrong?
There's also the distinction between types of difficulty.
Things where, you know, the letter puzzle you had at the beginning where it requires thinking about it, figuring it out.
It was patients versus something else.
I can't remember what the editor used.
But then you have things like mazes and locations where you have to be very careful, and separate your way through
And sometimes they bring in information from other domains.
From outside the puzzle, it's not just like all the information that you currently need in the puzzle.
But the interesting thing about the puzzle that he particularly shows, is that he shows a range of different ways to actually solve it.
None of which are actually satisfying except for the final answer.
Which irritated me, I'm telling you.
I'm not giving you another puzzle.
[UNINTELLIGIBLE] But it's kind of interesting to think about, actually, what audience it's for those kinds of specific puzzles.
I mean, there are magazines out of there like Games magazine in particular, but sometimes you see stuff nestled into magazines that are not just about games.
You see stuff like crossword puzzles in newspapers and things like that.
And now things like Sudoku and all that are way more popular that are meant to be really in mass audience.
Whereas a magazine that is specifically targeted at puzzle enthusiasts can be super hardcore.
With puzzles through the whole magazine then you can have a whole range of different difficulties in there.
The assumption that you can solve something not so hard
I'm kind of interested, does anybody know how people in newspapers generate crossword puzzles and Sudoku puzzles?
Does anybody know what the actual process behind that is?
I mean, I can imagine how you might do it.
You might start with the filled out thing and then remove bits to get your Sudoku.
Or you might brute force, dictionary search to come up with words which fit together in order to form a crossword
What they do with Sudoku now is, they have some computer algorithm-- and it has scales of difficulty-- and it generates a random Sudoku puzzle from your input
Sudoku is solved enough that you can actually just algorithmically generate [INAUDIBLE].
An actual numerical difficulty.
You scale it from 1 to 10, 10 being where where there's like literally only one step to take in each point
The one that took them three months to solve
But like crosswords, there's people who actually just get paid to sit down and generate those
I know that Premiere takes submissions from other people and they're edited by [INAUDIBLE].
And then people submit that and he kind of like steps it up
So he kind of crowdsources crossword puzzles.
[INTERPOSING VOICES]
But the difficulty in a crossword puzzles isn't just in the geometrically how the words go.
The specific clues that they give you can be more or less obfuscated in some ways.
And New York Times, in particular, is super irritating with this.
They'll give you two words with no grammatical context about how they're supposed to fit in.
So you can actually compare it to see how they're difficulty scaling just by looking at the clues.
Not even looking at the puzzle itself.
And I thought the other thing about crosswords is, just to bring up a point that we talked about in the earlier class, the crosswords, [UNINTELLIGIBLE] that these are dependent upon the redundancy of the English language.
You don't see every single letter of a word in order to figure out what the word is.
So that if it's a word that you couldn't otherwise have gotten from a clue, you can still get just by seeing the letters that are available.
Because the English language is kind of about [INAUDIBLE], redundant information.
If you dropped every single vowel in a sentence, you could probably still make out what that sentence is.
And you don't necessarily get that with Chinese.
How many folks here have, [INAUDIBLE].
There are probably other languages here where like every single character means a word or something like that
Hebrew has no vowels, right?
Well, they do up until third grade and we start dropping them after that.
So it's basically, because it is understandable without vowels, you eventually just stop using them
So that's interesting.
If you had a crossword in Hebrew, where you were expected to write in the words with vowels, you probably could actually do one very similar to an English one
It's difficult because the vowels in Hebrew are attached to a letter, not a separate character.
They're just down at the bottom of the lower letter.
[INTERPOSING VOICES]
Another thing, in a crossword the letter can make a different sound in both.
[INTERPOSING VOICES]
Because a "v" could be vo, or va, or ve, you don't know.
You only know by context.
[UNINTELLIGIBLE] like "c" could be sss or ca.
Like a hard or soft c. Or "g" could be ga or ja
That is true, but although in English in particular the phonetic meaning, the way how a certain word is pronounced isn't necessarily essential for solving a crossword puzzle.
Unless the clues happen to be phonetic clues.
You know, sounds like a fish.
So that may not come in so much to play when it comes to finishing a part of it.
Jason, you had your hand up earlier about Sudoku
It was a comment about the algorithmic [INAUDIBLE]
Also, one thing about algorithmic, about Sudoku difficulty, a lot of it comes from the selected removal of numbers.
There's not what is the final pattern going to look like that's actually the difficulty.
It's how do we remove the numbers in a way that gives multiple possible answers or multiple possible dead ends, because really there's only one solution.
I'm not actually familiar with the algorithm with which it does so.
But it seems to be very clear that there are at least two algorithms.
One of which is very easy, which is just provide a legitimate Sudoku solution.
And then the one that is slightly harder is now which number is to be removed so that's it's still solvable
: Have you all played Picross?
Someone want to describe it for the folks who aren't familiar with it?
We can use the white board if-- sure
I think the green one works better.
I remember green is better than black
I'm really only familiar with the [INAUDIBLE] 3D [UNINTELLIGIBLE]
: Sorry to put you on the spot Nick, as always
[UNINTELLIGIBLE] Here you'll have written poems.
And then they all have something [INAUDIBLE]
Oh, this.
This one where you spell in the blocks
And then given bits of information, we have to work out which of these blocks is built and which can be removed.
So the idea is the number tells you the number of blocks in that row or column which has a filled block in it.
So in this case, nothing is in it's column.
This column and that column have nothing in it.
And this is probably not actually a very good example because I just pulled the numbers out of my head
Then you have the sums are not the same
So it's not going to work, but you get the idea.
And things can get more complicated, traditionally there's just, I think, three that are names.
That's there a connected block of three.
So, for instance, this, that, and that.
And I thought it had to be broken.
But if there's a three which, I think, is in a square or something along these lines, that means there are three in that column.
But there's a gap in there somewhere.
And then if it's like a three inch circle or something else complicated like that.
Then that means that there are three but they all have gaps between them
But I've also seen versions where they have just a list of numbers.
And those are the-- there are [INAUDIBLE] blocks of that length
So if there's four in a column you might say two, three, five.
Which means there is a block of two, followed by a block of three, followed by a block of five
And they have to have a gap between them
And they will have to have a gap in between them.
So it's not like you can actually get a block of eight in there somewhere.
But the whole idea is that you've got this information that is actually redundant.
You've got more information in there than you necessarily need to be able to figure out which box and the situation to be filled in.
Whether this should be empty.
But it's presented to you in this obfuscated way.
The evidence, the clues that are given to you in a way that requires you to perform these algorithms in order to figure out which ones should be changed.
And usually that's an algorithm you can use to solve.
Although, I think Kim was the one that was talking about pattern matching.
But eventually as you get good at these things, you start to recognize patterns right of the bat.
Like, that's a zero.
So that entire row is empty.
That's the sort of thing that I do that becomes automatic for you
So, let's see.
I don't have much left more to talk about Scott [INAUDIBLE].
Now these are, these are more generated stuff.
[UNINTELLIGIBLE].
So this basically is along the lines of what Scott [?
Pim ?]
is talking about.
But obviously this is more physical not the pen and paper kind of thing
Will you print up a copy?
Oh, you want the copy?
No, because I just want you to look at this.
Look at this first, don't look at that.
Don't look at that.
So the basic idea is, let me just start with one, you get a card that is actually a puzzle.
I wish we were doing these outside now, because we're going to zoom this out.
But the puzzle basically just tells you how you are supposed to set up your board initially.
So, there's a-- oops wrong [INAUDIBLE].
There's a green guy there.
There's a green guy there.
This is supposed to be an easy puzzle, I hope it is.
Because I'm getting bad at this.
Red guy go to [INAUDIBLE] I think.
Yeah, and that's the puzzle.
And there's one little square on it, and that you have to check.
So the basic idea is you tip over the pieces so that the guy moves.
And you don't want to be tipping over the pieces in a way where the white guy can't move anymore
Yeah, so you're tipping over things to make a path that he can walk along, right
If I make a closed loop, he has to touch every piece?
I think he just has to get to this one
Yeah, you have to get to the red one
There's a plus mark.
[INTERPOSING VOICES]
I'm not exactly sure how to solve this one
Yellow that way, green, yellow that way
Wait, you're allowed to tip him over?
How do you die?
My understanding is that he needs to fall onto another piece, actually.
So let me just confirm I set this up right, actually.
Yeah, I did
No, so you push the yellow that way.
And then the green that way.
And the other green that way
So put the yellow ones first?
So do the green ones first
Yeah, I don't actually have the rules for the guy.
[INTERPOSING VOICES]
How does he fall?
[INTERPOSING VOICES]
He's actually allowed to move on anything that's contiguously connected.
So, as long as you can continuously connect up to this, you're fine.
[INTERPOSING VOICES]
10: So tip that first green.
No, other one.
And then form a path, right now, if you tip that.
Oh, there's a hole
11: Are you sure tip them over means more brown spot and not--
12: Well, if doesn't then you can't reach the red after it
Yeah, I am absolutely sure it means that you're supposed to tip it over in a way that-- [INTERPOSING VOICES]
13: Tip the yellow one first.
Then move the guy out to the green.
Then tip over the green.
[INTERPOSING VOICES]
That is why it's supposed to begin a [UNINTELLIGIBLE].
You can actually do it.
This is why this is puzzle one
14: You can even fall on top of anything?
That's bizarre.
[INTERPOSING VOICES]
But, you know, this is not entirely unlike the kind of puzzle that Scott does.
I mean, sure, this is one particular piece of plastic that the game is designed to work with.
But basically it's just like a book, right?
It's like a book of puzzles.
So there are also solutions in here, but I find the solutions harder to read
The [UNINTELLIGIBLE] was in the book
It was in Brenda's book?
Yeah
So, actually, that's probably a good excuse to switch over to Brenda's.
Well, actually, before that I want to take a look at the tangrams.
Everyone play the tangrams?
So, the basic idea is you've got these shapes.
Does this one get in, like this?
You've got these shapes, and if you assemble them right as is usually described on the side of the box somewhere.
Where the heck?
It's on the inside
I've got a big picture on the side.
Is that you can actually fit all of these shapes inside that square.
So these are basically just a square that's been divided in a specific way.
So once I've got this assembled everyone can just pass this around.
Anyone who has already seen it can pass it on.
That one doesn't fit very well, but it works.
Hey, that's, ta da.
OK, so this is the basic side.
Now this is how it's been sold, at least, is that this is actually a calendar.
So every day you get a different shape.
It's a cute little thing.
It's a pretty rudimentary geometrical shape.
Some of them have, I think this is supposed to be a sphynx or something.
It looks like a frog.
This looks like a Pac Man thing.
So the idea is that you rearrange those particular shapes into all of these different shapes.
And you could play the game without getting these puzzle solutions, right?
And it talks a lot about, if you've got a toy that's worth playing with, you've got the beginning of something that might make a good game.
The nice thing about tipover is that it's kind of fun to tip over things.
And the thing about tangrams is that I actually had more fun playing with tangrams than trying to solve specific problems just by just trying to rearrange these.
Who hasn't seen these before?
[INTERPOSING VOICES]
Kind of embarrassing
You should be ashamed of yourself.
And all the work that you did is gone
But the thing is, as a toy it doesn't really matter, right?
Because it's just a certain piece of anything.
And you kind of want to take it apart and make new shapes with it, like a little house.
So if you've got something that's kind of fun to fiddle with to begin with, you've got possibly the beginning of a game in general.
But the way how Scott was talking about it, there's this intermediate step where it's a puzzle.
It's a toy with a specific solution that you're trying to get to.
And a game is kind of one step beyond that where now there is multiple solutions and competitive.
At least, that's how he's describing it.
He's not really a game theorist.
He just works with puzzles
He talks about it in terms of a goal, right?
So then with a toy that you don't have a goal with them.
With a puzzle, you have one goal with no interaction.
And then a game is with interaction
And possibly multiple goals as well.
But again it becomes more and more directed as toys are very free form.
So there are people who describe games, like the Sims for instance, as actually toys.
In the same way that you describe a doll house as a toy, they're saying it's kind of like a digital doll house.
But later versions of the Sims added more and more specific goals.
Like now you have to throw a party with at least five people in it.
Or you have to attain career level five or something like that.
I can't remember exactly how they measured it.
But that kind of has a goal.
We are going to give more and more specific things for you to work towards.
But the Sims is very careful not to lose it's toy sense.
This adds up.
If you really want to just dick around with these guys, go right ahead.
Just play around with them and see what happens
Why is there a distinction for physical objects between toys and games, but for like video games-- like Mindcraft it is technically still a game but--
It kind of has different modes.
Yeah
I guess it has different modes, but it's more of a toy, I think
I would say that's entirely marketing.
The only reason is people haven't figured out how to sell digital toys.
And there are games aisles in like Best Buy and so, well, this is a fun thing we're going to put it into the game section.
Everyone starts calling it games.
And that's kind of the same thing with the Sims.
It's that really the Sims should be it's own genre, but you can't sell it's own genre that well.
[INAUDIBLE] Was there a hand over here?
OK, so let's start with Brenda's-- if it stops reading, so let's just move on to Brenda's.
And she's very specific about the kinds of puzzles that she's talking about.
Is that these things that get in the way of a very specific narrative experience that a game wants to guide you through.
She describes them more as roadblocks than anything else.
But, you know, things to make the challenge higher, but possibly so high that no one is going to be able to get past them.
Some of her examples are things that come up as, sort of, intermittent but optional mechanics like hacking in Bio Shop.
And sometimes there are things that you have to solve, especially in adventure games.
There's this, you have to combine this object with that object and bring it to a specific location.
And then now you can proceed with whatever narrative they've got
She had this specific axe to grind against puzzles having meaningful rewards or not being completely bought in.
So, for instance, she says you should never have a puzzle being required to open a door.
She said, solve a puzzle and then get some kind of tangible reward like loot or something.
This is after I just spent the bus ride to New York City playing Professor Layton's.
Which was nonstop, with on the door, it's got a lock on it, looks like a puzzle.
So I wasn't entirely to sure what to make of that.
I thought that my Professor Layton experience was fine, and I didn't need to get fat Layton loot from some [INAUDIBLE]
That's actually something, and there's a really good article actually written by Clara over in our lab.
Actually, it wasn't an article, it was the transcript of a talk that she gave at Austin University-- I'll send on the link to everyone-- about how you can integrate, and why you should integrate, puzzles really, really well into the narrative.
It's like really why is there a puzzle in your game at all unless it's been integrated into the narrative?
And Brenda seems to be, sort of, providing a cautionary tale of the-- But if you do it badly, no one can get on with the story.
No one can enjoy the rest of the game.
It becomes this one thing that prevents you from proceeding?
Is there anyone who had that moment?
Yeah
About five minutes into Mist.
I don't know.
It had some sort of cool live action video and stuff.
And it looked like it could be an interesting story.
That's kind using technological prowess of integrating video into 3D.
And then suddenly I looked in to the picture and I was on an island.
And then there was nothing happening and I had to turn things to face each other.
And I couldn't figure it out
When I originally played The Legend of Zelda.
The original one.
So they don't tell you anything pretty much.
And so the temples, you just walk around the sun, then it's like, OK, there's a tree and a forest.
You're supposed to burn [INAUDIBLE] and there's the entrance to this temple that you can't proceed until you do this thing.
And so, it's just like, how am I supposed to know that I'm supposed to burn this particular tree in the middle of this huge map?
Did you beat any of the other--
Yeah, and they didn't say anything
No, I mean the other Zeldas?
Yeah, actually yesterday we [INAUDIBLE]
I was just saying, so the interesting thing is most people nowadays they end up playing, like the Ocarina of Time, Majora's Mask, they are moving backwards.
So they end up playing the first Zelda after having played these old ones.
So you sort of know that you're expected to do these random things to get like across.
So I feel like it's easier for people, but then--
But with the original Zelda-- you know how in new ones they have, let's say, a wall that you can blow up.
It's discolored or something.
In this one every tree looks exactly the same.
They use the same stripes
If I remember correctly, though, it's placed such that it's in the center of your screen, right?
So it does look a little bit different
No, it was just a random tree.
It was just a whole bunch of trees.
And so you just really have to get, whatever a type of thing to burn it-- a specific one-- and burn it
I remember it was a little suspicious.
Because I think there was a one tile column, you kind of navigate around just this loop.
You just entered and--
That's suspicious
And I'll feel very frustrated by games that require walk-throughs in order to play them
Sure, and that was the other thing that she suggested some game designers do.
They put the solution in the Prima Guide.
So that you have to buy the Prima Guide in order to [INAUDIBLE]
Does Prima have a deal with the--
Oh, they must have
They say "official" on the top for a reason
It's also ironic, because modern Zelda games, it's very obvious that you have to use this or that item
You've got a key.
Use this key to open
Oh, you got this item in the dungeon.
You have to use this item to kill the boss.
That sort of thing.
It's blatant
Well, for people who are really, really hopeless, there's that little fairy.
[INTERPOSING VOICES]
They're usually less useful than you think.
So one big difference between the old Zelda games and the new Zelda games is just this continual refinement in their user interface.
And it goes all the way from little things like the notes that you hear when you find a new dungeon.
And exactly when, precisely, you hear the sound of, you know, too da loo-- at least I think that's how it sounds.
You know, as a clue that now you have opened a door.
That's one of those things that, if you look at post [INAUDIBLE] Ocarina of Time.
I think there's a couple on Gamasutra, exactly when that sounds plays is a decision that was very carefully designed.
And it seems to me that in digital puzzles those kinds of, especially in some sort of narrative structure like in Ocarina, that's the big problem that you have to solve.
You have to give enough clues.
You have to give enough UI feedback.
And you have to think about your UI so hard in a way that is going to make it possible so that we get all the information [INAUDIBLE].
So, say that burning fire tree situation, if there is nowhere else in the world where you can get that information, then it's back inside puzzle.
If there is a guy in-- I can't remember Zelda I-- but there is a guy who actually tells you that that's the tree that you need to burn.
But if it's not obvious, especially for someone who hasn't encountered that guy, that that's where you get that information it's still a bad designed puzzle.
Even though now there's a solution.
But thinking about, and some examples will be in a lot of tech space adventures as well as text heavy graphical adventures.
You have a, I can't solve this puzzle, but what my character says to the player about why you can't solve the puzzle is itself a clue.
It's like, hm, I must not have the right item.
Maybe there's something else I can use.
That's a huge clue
I feel like digital popular games have gotten really good at integrating hint systems in recent years.
But I can't think of any physical puzzles that I've come across that may just not have quite been up to speed with crossword puzzles.
That have that kind of not totally revealing the solution but giving you a gradual hint which can help you figure it out
Well, mystery puzzles kind of do that.
[INTERPOSING VOICES]
Describe the mystery puzzle for folks who don't have--
It's basically, it's a competition run by MIT over IAP where teams of people compete to solve a bunch of puzzles, like 70 puzzles, that kind of build on each other.
So you have, say, 20 base puzzles.
And if you solve those 20 puzzles you get 20 words as solutions.
And then those 20 words become a new puzzle, and that's the meta-puzzle.
And it kind of keeps them like that for a couple layers.
But the point is the winning team from the previous year writes the next year's mystery hunt.
So you get all kinds of varying puzzles of varying difficulties and styles.
A very well-designed puzzle actually does that, because you have this thing called, flavor-- Oh, I guess I should say that the puzzles aren't like Sudoku or crosswords.
You literally just get something on a screen or something on a piece of paper, and you have to figure out what to do with it.
You're not given any instructions for how to solve it whatsoever.
And the only clues you get are either by the title of the puzzle, by the group of puzzles that it's in, or by this thing called flavor text.
Which is like, they say some kind of random phrase that seems to not have anything to do with anything.
But that kind of gives you a clue about what to do in the thick of this puzzle.
So that's kind of like what you were describing
That's interesting, because it's really challenging when you don't have any mechanism for giving feedback to the person who is working a puzzle.
In Zelda, or what have you, you can see that they've spent a long time dicking around
It's kind of a double-edged sword, in that particular case, because you have so many mechanisms and now you're expected to use all of them in a game like that.
You use sound, you have to use contextual buttons, you have to use the inventory.
The position of the player matters or something.
I've seen games where the main clues is what is the character looking at, for instance.
And you walk around, why does my character keep looking that way every time something happens?
Oh, that's because-- you know.
So, yeah, as far as of a lot of physical stuff, I would say that one of the reasons is they're just starting to build these physical games from toys.
A lot of the importance is, a lot of things you could possibly do, are kind of built into the physical object.
And if you're building a puzzle based on what you could do with these things as a toy, then you're necessarily already using the constraints of what you can do with it as a toy in order to create the puzzle.
This might be a good exercise actually, if time-- we could take a shot at this.
And just by, you know, trying to come up with a puzzle and see.
But instead of imagining one in a digital space, try to imagine one in a physical space.
Because I suspect what you actually get is, you get a free language with physical puzzles.
Because things physically have interactions that we [INAUDIBLE] from a digital puzzle
I feel like Coral Reef or something, and that's a game where like the puzzle-- I mean obviously it's still a digital game-- but the puzzles are very much physical puzzles.
Like you kind of have an idea about what-- has Patrick played Portal, of course?
Or know the idea?
I mean, you have these two portals and you understand what the physics is going to be like.
So the puzzles aren't frustrating in the sense that it's like, oh, you realize there's this random statue here.
And if you pull the lever it raises the water level or something like that.
But it's very intuitive.
It feels like a physical puzzle in a sense that I know what will happen if I do this.
And I just figure out how to put the pieces together.
And also Portal what I think it does a really good job of, as well as some physical puzzles do, is having a learning curve.
Like Tangram will have some basic puzzles, or I'm sure the falling thing, a lot of them do it.
Physical puzzle things where you like, I don't know, how to jump over pieces or whatever, have basic levels-- basic boards-- where they show you various techniques.
And you only have to figure out one technique, and it's very obvious what you have to do.
And later on you have to use those techniques
Well in Tickle Her, this particular example does something that multiple portals does really well.
Is that the very first puzzle that we did earlier only has one solution.
And you do not get that solution unless you understand all the mechanics of the game.
But we tried to solve it a whole bunch of different times.
And it's like because we didn't quite understand the rules, we couldn't get it until we understood the mechanics of the game.
Now we can suddenly solve it.
So from puzzle number two onwards, we can't be confident unless you know it.
Portal, and even-- actually just about any well-designed game-- but I would say Portal is a fine example-- if you've beaten a game, you can play through the game again with developer commentary.
And when you show that, you actually realize that every single fairly intuitive physical thing that you pick up in the game had to be taught to you.
And they've designed the level very clearly.
So right in the beginning of the game, if folks can't remember, you're trapped in the middle cell.
And the GlaDOS talk to you, and a portal opens and you can walk through it.
What do you see out of the first portal that opens?
You see yourself going into the other one
You see yourself going through the other portal.
And it's like, they force you to have to look at what the basic game mechanic is going to be before you can do anything else that's productive in the game.
You know, you can pick up the radio, but-- So that was a design decision.
They could have decided not to make it.
They could have just made you go through that into a different room, and the learning opportunity would have dissolved right there.
So if you've got a copy of Portal, I would suggest go back and page through the commentaries because they are hilarious.
And you're also rereading good form, demonstrating in this room, this is what we had to do in order to make people get it.
I'm also thinking of games like Mario World I where you don't get very far on this kind of basic game mechanics now.
But they make it really gentle in that I don't think there are any hits that instantly kill you
That's exactly what I was thinking in Mario.
In the first Gamasutra article we read on late-game abstractions, we read about a Mario world.
He's like, we expect the same kind of actions, key the system, all the time-jumping.
And I guess you can kind of look at different things there, like double-jumping.
And things like that in a specific puzzle.
And the physical [INAUDIBLE] also
I mean a lot of Mario's worlds are laid out basically as spatial puzzles, right?
It's like you know you have to get there, but you don't know exactly how you're going to use all of the mechanics that you've got
In Mario 64, I think their way of solving the impasse problem of not locking up all the content in the game because of the one puzzle you can't get, they're giving you so much to do that even if they don't really tell you-- even if a good 30%, 40% are going to remain mysterious to you, you're not really going to figure them out-- there's so much [INAUDIBLE] between, you never walked off with a significant amount of content
So if you've got a hub-like structure you can do any one of five different things at any given time.
Plus you have some secondary solutions, like you can always go back to a world that you've been to before and try to collect all the stars, for instance.
I think that's kind of what Brenda was kind of indicating.
You don't necessarily want, if you're putting puzzles in your game, she was advocating for-- Brenda and Ian were advocating for-- something that wouldn't necessarily progress on to the next one.
But I think that's not a requirement.
I think that's a, if you want to make your life easier, this is what you do.
If you decide that you're up to the task, you can achieve some really amazing effects by having your puzzles narratively incorporate it.
Has anyone heard of the game called Eternal Darkness?
It's a sort of Lovecraftian horror game on the GameCube.
I'll sort of describe it, but basically the whole basic idea is that you have four gods all warring against each other.
Three of them hate one of them, but then those three also hate each other.
And most of the puzzles are just sort of understanding that there is this mystical object.
One god defeats another god defeats another god in a rock, paper, scissors circle.
And all of them hate this one other [INAUDIBLE].
So early on in the game there is actually a puzzle, there are actually a number of different puzzles, where you see the symbols of these gods.
And you have to kind of rearrange them in the correct order.
And the rest of the game is all going to be variations of understanding what these gods do to each other.
So those are all puzzles that inform you of how the world works.
And that kind of works in two ways.
One, information that you've already gathered about the world-- about how this world works-- gives you clues on how to beat the puzzles.
Beating the puzzles, finding the solutions for the clues, gives you information about how the world works.
And that's kind of like the holy grail for a puzzle designer in a digital game, is that you're going to design a puzzle that's going to reinforce how the world works.
There's a chap-- who gave a talk, and I believe his name is Jeff Howard.
He came to Camp [INAUDIBLE] and gave a talk last year.
And he makes this case that games, and the logic that you present in games, do basically the equivalent of what magic systems actually do in the real world.
What people like cultists or astrologers do is they basically give you logics that explain how the world is going to work.
And while that may not necessarily be obvious, but if it is viewed as a whole knowledge you'll be able to understand how it's [INAUDIBLE] in you.
But more importantly you'd be able to do things in the world that allow you to accomplish things that you wouldn't otherwise do.
Learning a game is very much like that, it's like I don't know how this world works but if I understood these particular rules I could be able to accomplish greater things.
And puzzles can be one of those things where it becomes a test.
And it becomes a test of, just exactly how well do you understand this?
Even if it is like in a combat game like Quake or God of War or something.
I can understand how the combat system basically works if I just keep mashing the x button or something.
Or just shoot everything with rockets.
But if I really, really understand the combat system, I might be able to hit something that I wouldn't necessarily be able to hit.
I might be able to take down a horde of enemies instead of hitting them down one at a time.
I might be able to beat a boss.
And actually a lot of boss battles in digital games are actually puzzles.
There's not, how well do you generally react, it's can you figure out the pattern.
But that pattern doesn't mean anything unless [INAUDIBLE] how all your other combat skills work.
So a boss battle in Zelda only makes sense if you figured out how that one item that you picked up in that dungeon works really, really well.
OK, it shoots fire.
That means that not only does it hurt them from distance, it also lights some of the things on fire.
And that's what I'll tell the boss, basically, is combining this leaders ability to hook up.
Like everything else that you've done before into taking it out.
That's why boss battles, that always kind of puzzled me
One thing I wanted to say, before I get going, you were asking about hints and a physical puzzle.
The one thing that jumped to my mind was a couple years ago the weekly Day-- I don't know if anyone reads it, it's like a free Boston newspaper.
And they always have crossword puzzles.
For the longest time they were thematic, and so you would have a guest crossword.
The first one I ever did was Hulk Hogan.
So all of the hints in the crossword puzzles are related to Hulk Hogan.
So it's a question of, not what does this mean to me but what would it mean to Hulk Hogan?
And they don't do it anymore, which is really depressing, because every week it was something different and it was like a built-in hand write as you went.
You didn't just pick them out literally, you have to kind of contextualize it
You can think about all the things that come with the puzzle, that potential hook, [INAUDIBLE].
Like a title, like a data text-- as you were describing-- the fact that you get in puzzle and mystery crimes, they may be [UNINTELLIGIBLE] and they may be disillusioned but it lies on a different puzzle.
Not the one I'm currently looking at right now, because that's how [UNINTELLIGIBLE].
Metal Gear Solid has a really, really bizarre one where--
Look in the back case
Where they tell you, oh, there's a specific radio frequency that you need.
It's written in the back of box.
And it's like, the back of the box?
[INAUDIBLE] Anti-piracy
Yes, but it's still a puzzle.
And I was like, I know how to solve that.
I mean, it was pretty obvious, because maybe it's just me, because it about half an hour to figure that one out.
It took me some effort
I mean Metal Gear Solid a lot of meta-puzzles like that.
Like when you fought Psycho Mantis for the first time.
Well, so the scene you had to do is you went to unplug your controller and plug it into the second port because he would read your movements.
And it was this cool, physical layer of it where you're playing the game.
Then you realize I have to interact with the physical world so that he can't see what I'm doing
But that one [?
cannot do, ?]
because in that particular example that one gave you a very big clue because right at the beginning of that match that boss tells you to put your control on the ground.
And he says, I'm going to control this controller with my mind.
And actually, just [INAUDIBLE]
That's only if you have dualtech.
So, because on PlayStation, if you didn't he just ignored that part.
And you didn't get that clue at all
Yeah, and that's a problem.
But it's still very cool
There were other clues, like you would talk about how you had been playing certain games by reading saves on your [INAUDIBLE].
At least not in Castlevania.
I just remember being like-- [INAUDIBLE]
So basically it's giving you a clue that there's got to be something better about the way that you're currently playing.
The funny thing about the radio channel is that one really did not give you any clue that there was something better.
It just told you about the back of the box.
It was like, it didn't tell you the back of the video game case, right.
It's not going to be that specific
It said CD case, specifically.
I was really confused, because I think mine came in a slip of paper or something
Yeah, if you buy it used or something
Yeah, so it's anti-used game protection, as well.
But yeah, mine was I think I actually said on the back of the CD case, but I was playing like a [INAUDIBLE] or something and that came in the DVD case.
So I didn't make a connection.
But if you look at the back of it, you see what the radio frequency is in the screenshot.
One of the eight screenshots that they show to demonstrate this is what the game is all about, there's the number and it's just right in there.
And then key it in
Has anyone played any of those breakout games?
Where it's lock you in a room and you have to-- [INTERPOSING VOICES]
Yeah, you find the thing, you find a slip of paper and escape [INAUDIBLE]
So, actually who doesn't know what these are about?
Escape the room.
OK, so escape the room-- actually, could we bring up something on your computer?
I have to go grab the video thing
Oh, the video cable
It's back there in my office
Oh, OK cool.
Call [?
Becky ?]
to send it.
Escape the room games are typically flash games.
And they're kind of like adventure games because there's usually only one room, although sometimes you have multiple rooms.
It's a much smaller world in most adventure games.
But the basic idea is the same.
You pick up objects, you use objects in a room, and you try to make things happen.
You know, if you haven't made anything, you are not progressing in the game.
If you made something happen, you are almost always making forward progress in the game.
And there is, actually, a guide for playing adventure games written by Roberta Williams, who did the Original King's Quest series.
And her guide is like, pick-up everything, go everywhere, use everything on everything.
That's actually what escape the room really is.
But there's a little bit more to escape the room games, but there's just not that many things that you can explore.
That's the idea is that you're stuck in a room.
And basically it comes down to just use everything on everything until you figure out what comes up.
Rarely is it logical, often pretty entertaining, because they're animation heavy.
So every time you do something right, you get some crazy animation.
I put the fish into the mail slot and out appears an octopus.
It's fun to watch.
For us, the kind of comparatively opposite of those are the Grow games.
And they're a series of games called Grow.
I think the guy who does them is called Eyezmaze.
And the whole basic idea is that imagine that you have a flower pot.
And every time you add something to it something happens.
But the order in which you are adding things to it makes different things happen.
And there are solutions, basically, in making the optimal number of things happen if you do it right.
So what you can do is not a mystery at all.
You have these six things that I could add to my pot.
And I know those are the only six things I could add.
The question is what order do I do them?
And doing them in different orders give you different results.
And I feel that these two genres are kind of linked, but they're playing off very different things.
One of them is, I don't know what I'm going to do.
But I know that whatever I do has got to leave this space.
And the other one is very much, I know exactly what I can possibly do, I just don't know what order to do them in.
So a couple of puzzles that you see online.
These are fairly encapsulated things.
One game is one series of puzzles.
There's not like a professor-made encyclopedia puzzle kind of thing.
What time is it?
OK, I think I'm actually going to grab a few more puzzles and bring them in.
But I do actually want to try to see whether we can put together a list of the kinds of puzzles that we haven't talked about today.
These could be physical puzzles, these could be puzzles that you've seen in a book by their genre.
Logic puzzles, you know, of a certain kind that you either really, really like or you really, really loathe.
And just break up in groups of-- three, six, nine, Heather [INAUDIBLE], ten, eleven, twelve, seventeen people.
Seventeen people?
That doesn't divide at all.
OK, two or three.
Groups of two or three and just like--
Jason should be back
OK, all right, then groups of three.
How about we-- just like-- you know, just a couple of puzzles that we haven't discussed and a description about what they're like.
And we'll quickly run through the class to sort of give everyone an idea of what-- As a toy, playing with a Rubik's Cube is more fun than the algorithm to solve it.
That's not fun at all, in my opinion
It's fascinating, you can also solve it.
But it's [INAUDIBLE], very tempting when you look at it also
So, one of the things you were talking about was interactive fiction.
Text adventures.
And how sometimes they could be considered just like if you're playing a regular adventure game but just with text instead.
And so the puzzle could be how you interact with the world, or it could be trying to figure out what actions are available to the user.
Like sometimes not all actions are there.
But also in some games the nature of text gives you different ways to have a puzzle.
Like in Lost Pig, because you're playing an orc he doesn't know English very well.
Or doesn't have full control of language, so you're trying to decipher his words to figure out what he's actually seeing
Do you have an example you can do?
To give with that to make it clearer?
Because I haven't played it
In that game, there is one part where he's describing a box and it has a slot.
And you put something in it and then it gives you something out.
But the way he's describing it, you don't really know what he's describing until you realize he's talking about a vending machine.
He describes it as a tall box with a slot in it.
And another example we were talking about was the game Aisle, which is you're this guy in a grocery store.
And it gives you this big prologue.
And you have one action that you can do.
So after you type in your one action, it tells you what happens and gives you an epilogue.
And then the game starts over.
But then using the information from that epilogue, you have a different choice of actions that you can take and different key words to use.
And so by playing it over and over you're uncovering more of the story until you get somewhere.
[INTERPOSING VOICES]
Aisle.
A-I-S-L-
We also thought about puzzles that were games put into a very contrived scenario.
Like bridge puzzles, or chess puzzles, or back when the Duelist magazine still existed they had Magic the Gathering puzzles
So like chess puzzles are like, here's this board.
What was the move that led to this board
Or like, black to win in three moves.
Something like that
So that's like, you are now placing additional constraints on the game
Yeah, exactly.
It already puts you in a situation like, pretend you're in this game.
Now, we promise you there's a way you can automatically win, you have to find it
There's a number of computer games like ChuChu Rocket and everything where it does like completely separate modes.
ChuChu Rocket is a frenetic both player--
It's a mice-herding game.
You have to get them into rocket ships, so they go in space.
It was for the Dreamcast.
It's so good.
The idea's really simple.
A bunch of mice wander around the streets, and each player gets to place three arrows.
And when the mice run under the arrow they follow the arrow.
So you want to get all your mice on your ship, and want to get the cats on the other ship so that they eat their mice
In puzzle-mode, you have three arrows in these particular directions.
But the way you place them, you have as much time as you want, but there's only one right answer.
And that immediately shows you the difference between a game and puzzle.
Right there.
Even though they are all in the same package
So, first off I don't like jigsaw puzzles, but Michelle loves them.
And then we also talked about whodunit's.
And I really love whodunit's because you have to pay attention to the detail in the stories.
Like the one thing that-- I guess the example that we were talking about is, if there's a mailman and it happens to be on a Sunday, he probably committed the crime.
Because they don't have mail on Sundays.
And it's like small details like that
Are you talking about mystery books or just like little paragraphs?
Well, it's usually like a story.
And they say, oh, here's the scenario, here's the background story, who is the murderer?
They compare it though, [INAUDIBLE]
People have parties like that, like full-on whodunit parties
That's very true.
I didn't see who was talking for a second.
Who's talking?
Another puzzle we came up with, was the ones that you did in elementary school with like the big grid.
And you would have someone's name, and maybe their favorite food, that you would give them a bunch of clues and you have to match them.
Yes, with something like the checks and the xs and stuff
So, that's just generally called logic puzzles, I believe.
It's just you are given the incomplete set of information but enough constraints that you can figure out the rest of the information.
And really the way to solve it is just write everybody's name, all the characters that you're given in the prose.
Write down all the characters on one column.
Write down all the things you're trying to figure out, like who's wearing what colored shirt, for instance, so just write out all the colors on the other column.
And just by the process of elimination, just figuring out, usually the answers are mutually exclusive.
So if somebody is wearing a red shirt, can not be wearing a green shirt
So part of the much harder ones are every fourth statement is false.
Like you don't know if one statement is definitely false.
And I've seen one where every statement is false except for one.
And I can bring in my book and there's like [INAUDIBLE]
The worst version of it is figuring out how puzzles see people in the world
They wouldn't fix it before because he pushed it out last week
You guys want to throw in any?
Sure, we started by trying to think of all the puzzles that we really dislike.
And we came up with a couple here.
The sliding square puzzles, you know where you have like-- Basically just a variation on a jigsaw puzzle where the rules are more constrained.
I don't like jigsaw puzzles either.
So we're like-minded there.
And then John had a couple.
I guess there's a Light's Out puzzle.
Inside the image you touch certain areas and [INAUDIBLE]
So you've got like a grid of buttons and every button you push will invert all those neighboring buttons.
Like implanting the top button, left and right, will-- [UNINTELLIGIBLE]
And then I guess I already described the concentric circle thing.
Yeah, their puzzles are similar that we're taking an action on one item in the scene.
These are incredibly difficult, the concentric circle ones, will have an effect on other things in the system.
And typically these are used in games where there will be a center podium with a pathway leading off.
On one side of the podium, then there will be a concentric circle around the concentric ring with a pathway off of that ring and another interval.
Then more and more circles to each side and you get to rotate individual circles and that will rotate the other circles different ways.
And you need to create a pathway
Every time I rotate one, something else is going to rotate
Then it's always going right before you're [INAUDIBLE]
I guess three of those are examples of games where you have to take action on an environment, but your action has direct effects throughout the environment.
And it's just really annoying figuring out how to optimize that
But they are predictable effects.
It's like every time I rotate this one left, that one's going to rotate right
I think it's kind of funny how the context has a lot to do with whether these puzzles are annoying or not.
Because I do enjoy Professor Layton quite a bit.
And it's the most, through an arbitrary character in order to justifying that you're trying [INAUDIBLE].
It's just fun.
Your intention on even playing the game is that you're going to be solving puzzles.
Not as [INAUDIBLE].
[INTERPOSING VOICES]
Also, the way that is presented makes it funny in that as well, right.
So the mafia goons come up to you and say, we're going to stop you from coming in here unless you solve this puzzle.
Shouldn't you just shoot me?
It would solve all your problems.
No, no, solve this puzzle
It's like they're making fun of themselves even.
All right, cool
I'm not sure if we really commented on what we liked or didn't like.
But some things that we came up with that haven't been brought up yet are cryptics, which are kind of a combination between crosswords and riddles, riddles, combinatory games as a puzzle apparatus.
Logic puzzles in general, not just like typical logic puzzles like that.
You can have state machines, or Sudoku, or any type of numerical puzzle.
We also want to differentiate that a lot of those games, or some of those games, are spatially oriented and some aren't.
Which we thought was an important aspect, but not necessarily definitive of a completely different genre
I'm not sure I've seen a state machine puzzle, actually, outside of a computer science textbook
I know some
I'll never ask you.
I expect it's going to be more involved than we have time for.
By combinatory puzzles you mean stuff like--
Combinatory games.
All the games that fall under combinatory game theory
Like Nim's.
[INAUDIBLE]
These are sort of exploration, so games where you have to see everything around you to know what is available for use.
On the [INAUDIBLE] so in the book they talked about actually Zelda, and just-- So a situation in a game that I didn't like is, you're playing a game and something happens you don't step too fast or something.
And because of that, you get too inert.
You're in a spot where you don't have the tools necessary to beat the enemy right in front of you.
But you also are trapped and can't go back.
It happened to me one time.
It was the Star Wars: Knights of the Old Republic.
There's an enemy, you couldn't go forward, and there was a cliff so you couldn't get back up.
And so, since I couldn't do anything I was forced to completely go back to the last save
But is that a puzzle, or is that just a bug?
I mean, it's time worth expiration and it's an item use.
So you need to get an item to go ahead.
If you have this item, then the enemy that you-- I think it was like a Rancor or something-- you could beat the Rancor or at least put it to sleep and walk by it.
But without it, you had to try and beat it or it would kill you
OK, so it was a item that would have made combat easier, basically.
But they kind of made it this one way with-- yeah.
So you see a lot of puzzle games where they have easy reset, right.
And it's like you can stop trying to solve it and you really screw up something along the way You hit reset and go right back to the beginning.
Nice that the Republic has no easy reset.
You live with the consequences, I guess that's what they were trying to go for
All right, so we've also had a loathing sliding puzzles--
Either with numbers or pictures, both of them are equally annoying.
But, on the other hand, Alec loves Rush Hour
It's the traffic red light game where you have to fly cars out of the way to form a path
In Professor Layton there are an awful lot of sliding puzzles that have differently shaped pieces, or oddly shaped pieces.
Which is slightly more interesting, or less annoying than regular sliding puzzles but it's still incredibly frustrating
Well, I think the difference there is that when you have differently sized pieces-- When you're doing just like a sliding puzzle where you're trying to recreate an image there's definite patterns that you keep using over and over again, because you're just like a rotating spaces.
So it's not actually interesting.
Whereas Rush Hour you actually have to think about things fit together.
So one other puzzle type that we haven't really mentioned, that I actually really enjoy, is pictographic puzzles.
Where it relies on sort of your association with visual images and wordplay and stuff.
Like, for example, you'll have a puzzle that's just an image of an eyeball, a bumblebee, and a screen shot of Judi Dench in a James Bond movie.
And the idea is it's IBM.
So I love puzzles like that.
[INTERPOSING VOICES]
Yeah, so stuff like that where it relies on someone, like a human being, actually associating information with something visual.
Rather than it being just a straight modern puzzle where you have this information and you use this to derive the rest of the information
There's R-E-B-U-S, and of course those are very language focused even though they're image-based
What one has the two lines and then [INAUDIBLE] in the middle?
Well, things like that, yeah
You had Rebus where you are trying to basically reconstruct a sentence.
And then I forget what's the classification of puzzles where you're given a certain lead of images and you're trying to write enough.
But if I wrote it in-- this is not a good puzzle but just to give an example-- if I put [INAUDIBLE].
I think there is a specific word for these kind of things.
But the whole idea is you're trying to reconstruct these sayings or these [INAUDIBLE] from individual theories.
So I mean, these are usually spatial to begin with.
I think there was a whole game show in the '80s that was just all-- [INTERPOSING VOICES]
Australia had a show called Catch Phrase which was--
Yeah, I think Catch Phrase was the show
So we wrote a B-list of different types of puzzles, so let's just run through them.
All right, we've got lots of different sliding puzzle variants.
We've got hidden object finding, the variance on Where's Waldo and then it's kind of [INAUDIBLE].
We have riddles, various kinds of logic puzzles, particularly two of which we've mentioned before.
Several people, witnesses, give statements.
One of them is a liar, which one's the liar.
So find the hidden sinister person.
And then there's the figment property x is next to figment property y, what order are they sitting in?
That kind of puzzle.
There's a million of them in Professor Layton.
There's these kinds of unraveling, or reverse-engineering type, puzzles.
Where some system has been permuted and you have to restore it to its original state.
Which Rubik's Cube is like the canonical example, I guess.
But then there's a bunch of other different good ones
I mean that chess games is kind of in that category too
There's tile rotation and swapping puzzles.
One example of this is Bioshock hacking, but then there are 1 million other variations on exchanging tiles or swapping them around or this kind of thing.
Lots of different kinds of math puzzles.
This particular word transformation puzzle type that I used to see all the time when I was a kid, where by changing one letter or word per step, you had to transform one word into another.
Mazes, jigsaw puzzles, code breaking nobody mentioned yet.
And then also, these kind of full experiment type puzzles.
I don't know how else you categorize them
Give us an example
All right, so I'm trying to remember how this works but I think it is-- this one popped up on Day(9)s podcast the other day.
I only give it because otherwise if I get it wrong then people will go oh my God, and try and puzzle it out.
So it has to do with the piece of string and burning
Oh, the 13 minutes?
Yeah, do you remember exactly how it goes?
OK, you have two pieces of string.
You know that the whole string takes an hour to burn but it's not linear.
You have an infinite supply of matches, how can you measure 15 minutes with this string?
One of those lateral thinking puzzles
I would classify this under riddles, but-- [INTERPOSING VOICES]
Let them go, but they both take an hour-- [INAUDIBLE] but they both take an hour to burn, non-linearly
Couldn't you just be lighting matches the entire time, and see how many matches you can burn an hour.
After that, just take four to that number--
Then how are you going to measure that original hour?
[INAUDIBLE]
But you know that the thing takes an hour to burn, yeah?
The whole thing
It will take slightly longer than the amount of time it takes to get to class.
We close by then.
It would be nice to give people time to think about it too.
So answer on Wednesday.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information, see the course materials on the MIT OpenCourseWare website
A bunch of puzzles
The first one's easy, and then everything after that is such a scam
Is it like a series, or is it just one book?
I don't know.
I found it.
[LAUGHTER]
[INAUDIBLE] Everyone's looking at him and they're like, you can do a practice leg in a one-person [UNINTELLIGIBLE PHRASE].
We'll just give you any puzzle that gets progressively more impossible
Actually, I want to throw out one suggestion.
If anybody's having trouble thinking about what to put up for a program post, and you haven't been able to keep up with your program posts, one thing that you might want to attempt is just a quick postmortem on the previous project.
What went right, what you would rather have gone right, what went wrong, and just a couple of other points.
Just sort of a reflection, just thinking about what you might want to try to change or what you want to make sure that you keep doing for the next project.
That's just a good idea.
So if you could do that next week.
It will be done today, so after this week's program post, you might want to try to do the exercise
So today, well first of all I want to say if you can't see the screen, you might want to position yourself so that you can, because most of everything is going to be up here for today.
So I don't know if, for those of you who have taken or are taking video game studies, how much you're going into this.
But some of today's talk is definitely going to be more on the video games studies theoretical side.
And we're not actually requiring you to implement this stuff, so a lot what we are going to be talking about today is rhetoric and meaning and how that relates to simulation.
If you're not conscious of this, or you don't put this in your game, that's fine.
This is just something that I think is really interesting, and I hope you guys will as well.
So, first of all, I want to talk about what simulation means, and it would help if I did this correctly.
There we go.
OK.
So, Gonzalo Frasca says, to simulate is to model a source system or different system which maintains for someone some of the behaviors of the original system.
And so to me, this definition is really important for a couple of reasons.
First of all, he puts source in parentheses.
For him and in his writing, he doesn't necessarily think you need one, which is a whole other argument all together.
I want to point out that all of this will assume that there is a star system and that the design is consciousness.
That's part of the board game assignment, to make a game based on a real world system.
But this technician has a lot of space in it.
It maintains for somebody some of the behaviors.
It's intentionally very loose.
And this'll come into to play shortly, because a big part we are going to talk about today is the relationship with subjectivity and simulation, which I hope you guys saw a little bit of in the reading for today.
So to give a more rigorous definition, this is a model taken from an extremely dry textbook called Theory of Modeling and Simulation.
Zeigler and all have [UNINTELLIGIBLE] of what it offers, and basically they present a simulation that's like this.
So the idea is, you have a star system that exists within the experimental frame.
This frame refers to, these are the conditions under which we are interested in the star system.
Right?
So if you want to make a game, say about life at MIT, there's some things you are going to include, some things you're going to exclude, right?
You probably don't care about the effect of the moon's gravity on the campus, for example.
And so then there's the modelling relation.
So we observed the star system under certain conditions, and then we focused on the elements that are important to us, and from that we construct the model.
And you notice, here they distinguish between a simulator and a model, which is not something that I think we do necessarily.
Because we tend to think that simulations are complex technological software entities.
So separating this indicates that multiple things can be a simulator.
For example, in terms of games, you can have a board game.
You can have a video game.
And these could be based on the same model, or they could be derived from the same system, but the simulator exactly can be a couple of different things.
And so this is a sort of technical way of looking at it.
What I want to talk about specifically is what I refer to in my writing as a simulation gap.
And this is another way of looking at the elements in a simulation.
So, major components such as star system, the simulation, the user, and I have all of these super intuitive arrows.
So we'll go through them quickly.
So the arrow at A, this is kind of the abstraction process.
We are looking at a star system we're considering, presumably within some sort of experimental frame that we care about.
And then from that we derive behaviors and rules for what we want to implement into the simulation.
So the arrow with B represents the user.
So this might be me.
And this is me interacting with the game.
This is me playing with it, seeing how the simulation works, watching it run.
But then, importantly, there's also arrow C my recognition that it's based on something.
If you can't tell that simulation has a star system, you're going to think of it very differently than if you know where that star system is.
And so D, then, is the proverbial simulation gap.
And this is the space in between all of them.
And this is a model of how users, in working with a simulation, can create a subjective interpretation of it.
They assign values to it.
They look at are there value judgments in the game.
They may consider various parts of the game to have certain meaning or express certain ideas.
And this arises from understanding the difference between the star system and the simulation.
I wasn't able to compress the quote down into a slide, but Ian Bogost said that the simulation gap is kind of-- a user's interpretation of a simulation converges-- all right, let's see if I can get this right.
It's horribly wordy in the book.
But he's basically saying your interpretation is based on what the simulation includes as much as what it excludes.
We look at these design decisions as having meaning.
And so we saw this in the star article.
I'm sorry, this picture is lo-res, but the game is lo-res.
I don't know if you remember this.
There's this key point in the article for today when [?
Star ?]
is talking about discussing SimCity 2000 with his daughter.
And if you remember, he and his daughter react differently to the game's exclusion of mixed-use field.
Actually, I probably want to check.
Has anyone not played SimCity?
Any of the versions?
No experience?
OK.
So SimCity basically is a-- he's called it a software toy.
It's a simulation of a city.
That's the name.
And the idea is you start with an empty space, and are trying to build it up into a burgeoning metropolis, or whatever.
One of the defining features of the games is it's very open.
There's no real set goals.
And this is primarily done through zoning.
And zoning is designating land for specific uses.
So the main zones in the game, and in actual urban planning are residential, commercial, and industrial.
So to mark a segment of land as residential means, this land is approved for building housing, basically apartments, houses, etc.
Commercial zones are things like commerce, offices, retailers, those kind of things, whereas industry is manufacturing, mostly.
And so in the game, you can basically say this chunk of land is for this use.
But in reality, it's usually not that cut and dry.
And so mixed-use development is a particular kind of zoning where you're saying this land can be used for multiple things at the same time.
And you see this a lot, actually, in Boston, especially down Newbury Street, Back Bay, where the ground levels of these apartment buildings are restaurants or stores.
And then there's space to let apartments built right above them.
And that's not possible in any SimCity.
And so to get back to the article then, the author is saying that he thinks the simulation is prejudiced against this kind.
It's against having this kind of zoning because the game just leaves it out.
Whereas his daughter says, well this is just how it works.
So for the two of them, they both interpret this exclusion differently.
The daughter thinks, well, OK, this is just how the game is.
It's not important.
And again the author says well this is actually a political decision.
So if any of you are familiar with the game, can you think of other games that are examples of politics in SimCity?
It's written about a fair amount, actually
There's the energy scope, where you can choose between coal or nuclear and whatnot.
I think when you do get to the [UNINTELLIGIBLE PHRASE]
Yeah.
They had their own agendas, certainly
Well, it's not only that.
I think it's cheaper to build a hydroelectric plant than it is to build a nuclear plant.
Because the material cost of building it in SimCity is cheaper
That's certainly part of it.
Although it's interesting to know that in the game, I think part of that is that the hydro plants don't generate as much, so you need a lot of them
Maybe.
I haven't played in a long time
Yes.
I think they took the hydro plants out in 2000
Oftentimes it's a lot of rules, anyways.
It was often to your interest to build industrial zones near a border of your city, so that your city wouldn't be polluted as much.
[UNINTELLIGIBLE PHRASE]
Yeah.
And it's a result of the pollution model.
The industrial zones pollute in a circle, so if you pollute on the edge of the map, half that circle is non-existent.
But there's a question.
Is the simulation arguing that you should send your pollution to other cities, or is it just a result of how it works?
Jason?
The decision on how to follow things like how taxes affect group rates, and how different sections interact.
All those types of things definitely have models from what they come from, which have certain slightly bigger political slants to them
I was just wondering.
I think the difference in how the father and the daughter viewed the system is that the father already had a preconception of what the city should be like.
But the kid wouldn't necessarily know because they haven't been exposed to all of this.
So to them, it's like oh, OK, this is how it works.
But the father knows that there are other things in the model system as opposed to the simulation
Yeah.
And that's, again, emphasizing the fact that the user has their own subjectivity.
Different people are going to assign different values to these things
Mass transit in SimCity is universally considered to be a part of the [UNINTELLIGIBLE PHRASE]
In some games they'll complain about the lack of roads, but they don't do anything about it.
They keep moving in
How are they moving in?
Yeah.
And actually, in the first one, you could just have trains.
You could just have rail lines, and there weren't even train stations.
Everything was built up against the tracks.
Which in reality, I've lived next to the commuter rail lines.
It's not pleasant
In come cities, a significant portion of the population does make do with only public transport, and you can get a taxi
Just one final point.
It's a very small detail, but a lot of the games will let you declare your city a nuclear-free zone, which just basically means you can't build a nuclear power plant.
But that stimulates growth.
People want to move there more because there's no nuclear power plants.
So I want to go through a few more really short examples.
Civilization Revolution is more like a stripped down Civilization.
Is anyone familiar with this version?
Now, specifically are you familiar with the DS version?
That's the only one I've played that's actually based in [INAUDIBLE]
It's on iPhone, too
Oh yeah, it's probably the same thing on the iPhone.
So I've noticed, having played this game a lot, in the game, they have all these different civilizations, and all represented by a famous leader.
So the Indians are lead by Gandhi for however many thousands of years the game goes on.
And I've noticed that Gandhi is incredibly aggressive.
Almost to the point where he declares war on someone.
He will stay at war the whole time.
I usually end up back again, honestly.
And you can try to make peace, and they'll never do it.
They'll say, oh yeah, we'll be friends.
And then three turns later, they're declaring war.
But on the other hand, Alexander the Great, who conquered much of Greece and Macedonia, and was very expansionist, is a total pacifist.
He'll never fight you.
He never wants to declare war.
And not only that, but he never expands.
They almost always just have their one city, and they never try to grow and spread, which is totally not what the Greeks did.
So there's this question here.
Is this the result, then, of bias in the simulation?
Do the designers just think, is it bad history?
I think that's backwards.
Wrong pointer
That might be it, because in, for example, Civilization 4, and I think 5, they have the same characters, where if you're being India, your player can be Gandhian.
He's pacifistic in that.
And I'm pretty sure Alexander the Great is probably also expansionist.
I'm not sure why they did that in this version
Maybe they messed up
Gandhi, in number 5, is still really aggressive
Maybe they just thought it was funny
Maybe they like irony
I actually have my own theory on this.
Part of the game is your civilization has a certain government, and governments get different benefits.
Actually, [UNINTELLIGIBLE] benefit in this game.
Indians start with fundamentalism, which gives you a bonus to attack.
Whereas the Greeks start with a democracy, and then a democracy that people won't declare war in.
So there's this kind of weird, and my theory is that Indians always attack you because they always have an attack bonus.
And the Greeks never attack you because they're a democracy.
As we know, democracies aren't necessarily pacifistic.
And again this is my thinking on it.
This understanding the rules of the system, and then I kind of infer this AI behavior based on that.
But again, this is my position.
It's easy to find people complaining about historical inaccuracies in Civilization.
There's always this question.
Is it part of the system?
Is the system just behaving in a certain way because of certain properties?
Is there another intent there?
We can infer all these different meanings from what the simulation includes and what it excludes.
So I want to ask, has anyone not played September 12?
Can I get a volunteer to play?
Go for it, Jeremy
Do you want to play?
So you can control with the mouse.
Can everyone read this OK?
Should I read it.
It's not a game.
You can't win or lose.
It's a simulation.
It has no ending.
It's already begun.
And here it goes.
So, all you can do is-- [INTERPOSING  VOICES]
So do you see what's happening to the people that you just hit with this?
[VIDEO GAME SOUNDS] [UNINTELLIGIBLE] --turn into errors
You can read in the intro screen that this is meant to be a simulation of the United States' response to 9/11
There's a puppy!
Do you think you're making things better?
[PLAYING VIDEO GAME] [INTERPOSING VOICES]
-- just to kill the puppy?
That is one crazy dog
What the heck?
Do you notice anything about when you [UNINTELLIGIBLE]?
Anything unusual?
So apparently as soon as you click, there is a delay.
You can never specifically target.
Even if you say, I want to get these terrorists, the delay means the odds are you're not going to.
Because they're going to move away
Did you bomb the other terrorists?
No.
If you kill a terrorist, you don't bomb more terrorists
Oh really?
Leopard terrorists
Ooh, it's a kid
It's a kid terrorist?
You shouldn't have brought adult terrorists
Great strategy.
If you turn all of the regular civilians into terrorists, then you can just kill everyone.
[LAUGHTER]
I think they also come in from the side there
Well you've got to kill them faster
Last night, someone figured out that if you camped out in the corner, eventually one would wander into the corner without any [UNINTELLIGIBLE] on it and you completely could probably kill only terrorists in the entire game
That's boring
How about just kill everyone, become terrorists and then chase them
So I want to say that this game has received a lot of criticism.
So this is actually Gonzalo Frasca's game, if you still remember the quote from the beginning.
Notable designer Greg Costikyan has really criticized this, saying this is really harsh.
How can you call this a simulation.
This is so insensitive.
And you notice that this game leaves out a lot of the aspects of terrorism and how peaceful interactions work.
You don't see 9/11 happening here.
It really simplifies the act of what makes someone become a terrorist.
You don't have all of the other pressures in the region, all of the other countries' influences.
It's very stripped down.
And I think Frasca talked about part of the point here is that it's ignoring so much.
Because that seems to be the way that the US government has responded.
It's like this happened, and now we're just going to start bombing them.
And so this is a really great example of how what has been stripped out and what's been left in really creates a strong statement.
And you can interpret it in a lot of different ways.
I personally think that what he's going for is this idea that the rhetoric surrounding the War on Terror and the political actions are ignoring a lot of factors, including our own role in the region of attack.
But anyways, I don't mean to turn this into a subplot.
Does anyone else have any other observations or anything to say about it?
I know that, as a result of this, or inspired by this, there is a great [UNINTELLIGIBLE] article.
That tells us we can expect a new mother of all bombs developed generates 10,000 terrorists with every blast
So another example, has anyone not played Every Day the Same Dream?
Does anyone want to play?
I'll try

[INAUDIBLE] Gandhi is not part of the system.
Gandhi is just not like a player, he's not part the system
In Civilization?
Well, the AI controls him
There was some music.
[MUSIC PLAYING]
Did you hit a car?
Aw, you can't jump?
[LAUGHTER]
[UNINTELLIGIBLE] a cubicle!
What is the biggest cubicle in this room?
Computer science.
Software
I get it.
Anything more to it?
You probably won't have enough time to do everything
Keep going forward.
You can just keep going forward
Oh, you [INAUDIBLE]
Oh my god!
[INAUDIBLE] Go left, go left
Left, left?
[INAUDIBLE] spent a day
You guys don't usually spend a day with a homeless guy in the graveyard?
Why wouldn't you?
It's cool
Don't get dressed
Next time don't put your clothes on
I'm going to get dressed like a homeless guy
Wow!
OK, do I crash the car, or do I--
Dude, get out of the car
Let's power through it.
Just get out of the car.
Just get out.
Go left
See that guy.
He abandoned his car
Oh, a cow.
Sweet
I can't?
Aww
Hold up traffic, [?
Kev.
?]
Aw.
Leaf is gone
I want to see what my boss is going to say.
[UNINTELLIGIBLE].
What?
You can't go left
You can't go left?
I think you can.
I think there's a car there.
[LAUGHTER]
Car's are inside-- [INTERPOSING VOICES]
You are late?
He didn't notice I missed day of work?
This is why you're late
The [?
graphic's ?]
going down
It's all your fault
Oh.
I'm going to get fired because the company is going to go down the drain.
In three days
Walk past your office
Hmm?
Walk past your office.
Or your cubicle, around it.
The experience of the game is a little bit stronger when you don't have people talking.
We don't have enough time [INAUDIBLE].
[LAUGHTER]
Push it
Should I?
Do it
Do it
All I see is [INAUDIBLE]
I told you I was going forward
What?
It's Groundhog's Day
It is the Matrix.
All right.
We're going naked this time
You're still wearing underwear.
There, I watched TV.
Now we've got feeling this [UNINTELLIGIBLE]?
I guess not
Did this skip, too?
Yeah, because we-- [INTERPOSING VOICES]
Everything different you do is another step
So this is the last step then
Naked.
Naked
HIt a car
I can't
You can't hit it?
I'm jamming it
Are the cars the same everyday?
And the leap was something, too.
[LAUGHTER]
Now what?
I assume you have to go past your office again.
But you're going to want to get dressed, or you're going to get fired again
No, he won't
No, I won't
Oh, right
[UNINTELLIGIBLE] left me
You keep on reminding me, because I had a job
Where are you going?
You're fired
I'm going to go postal at work.
Don't worry about it
He's going to kill himself
Am I doing the right thing here?
Yeah
We're the homeless guys
It's Saturday!
Good call
Yeah.
He got fired, too.
Ha ha.
I can't crash cars now.
Really not one's going out because day 6 is Saturday.
So no one's going to work
Are we not supposed to get up yet?
Oh, look.
You have no cubicle anymore
You're retired
That would be funny
You finished the program
That's you
I don't know.
Everyone else looks the same
Yeah, they all look the same
Deep.
[LAUGHTER]
It reminds me of that one game where you walk into a bar that's right across, and you can pick up a wife
Oh, Passage.
That's a weird game
It reminds me of that game a lot
So there's a couple things that I want to say about this game.
First of all, if you remember [?
Prospect's ?]
definition of a simulation, it's talking about the space in it.
And I would consider this a simulation.
This is based on a source system.
But it's not the kind of number crunching CS thing that we associate with the word.
For me, personally, I worked a pretty horrible dead-end cubicle job for about 18 months, before I came in here.
For me, this game really resonates, because it really closely matches the experience.
You're just doing the same thing every day, and any sort of deviation from the norm stands out.
But in terms of exclusion inclusion, one of the interesting things about this game, [UNINTELLIGIBLE] makes its point.
There is nothing else.
It's all a workday.
There is no weekend.
There's no I got home and there's something different.
And that's the limited agency in the game.
By that I mean it can be easy to forget about, especially in a place like MIT, but a lot of people I know end up in these very dead-end jobs.
This is a huge part of any major economy.
There are people who just go to their cubicle day after day.
When I was at that crap job, I knew people who had been there for 40 years
What kind of job was it?
It was for a mutual fund subsidiary at Bank of America.
Back in the financial district.
I'm not sure if it would have been better for me to stay, given the whole, you know, we tanked the economy but we awarded ourselves billions of dollars.
Maybe I should have stuck around
So it was a well-paid type of job.
I mean, in this climate, it might have become well-paid.
But it wasn't at the time.
It was actually, a lot of what I did was trade corrections.
This is kind of a tangent, but it drives me bonkers.
Financial representatives and advisers make an obscene amount of money, yet most of them don't know the difference between dollars and shares in stock.
And most of my job is correcting that for them.
It's unbelievable.
And it's no surprise to me that the thing is all fucked up.
Anyway, so any other reactions to this game or thoughts about this game?
The color [UNINTELLIGIBLE]
What do you mean color?
[UNINTELLIGIBLE PHRASE]
There's also nothing meaningful on the screen, right?
There was some kind of static noise, right?
But you turned it off?
I think so.
I forgot to close the September 12 window, so there was some noise from that
The use of color, just in general, was really interesting.
In particular, the only thing at your work that is colorful is the plumbing stock sign, the exit sign which tells you where to go kill yourself.
The leap is colorful
I think the cows.
And the hobos.
I think it's interesting.
It's kind of from the heritage of the game Another World.
Because sort of an early [UNINTELLIGIBLE] look at what it would be like to be a human transplant onto another planet.
You know [UNINTELLIGIBLE] that, in the beginning your character is extremely vulnerable and slow and clumsy.
And it uses the same control schemes on the same [UNINTELLIGIBLE] channel endings.
And by the same token, you're playing an ordinary person.
It's not the boss.
Not this person coming in.
And you don't go into a postal-style power trip, either
And so, I just wanted to say, I actually haven't seen the talk, but apparently, someone has argued that this game is just a critique of the art game movement.
Right, that it's making fun of the whole pretentiousness of that.
I don't really see that, but again, it's subjectivity.
Like where does this actually go
Well, what do the creators say to that?
You'll find that really creative people like that never want to talk about it
I thought there was an artist statement on the website
I'm sorry?
I thought there was an artist statement for this somewhere
There could be.
I honestly haven't looked it up.
OK, so moving on.
So anyway, good theory.
If you have a theory about how things work, the best way to test it is to figure out what's a horrible example.
So we've been talking about simulations and subjectivity, and saying, OK, how does this model explain the way meaning gets invented in a game and Mario is a really good example to throw with that.
So what are some things that are abstracted into the simulation?
Like what's here?
What's not?
It's physics.
You can jump to the moon
You can move in mid-air.
You can slam against extremely hard things and break them
[UNINTELLIGIBLE] things and kill them.
Technically [INAUDIBLE]
You can punch hard things and breaking them
You can get bigger.
You can shoot fire out of your hand
Still, the plumbers don't seem to really do any work.
They do get out of fights and clean up coins here and there, so that technically counts as points
Why is it significant that he's a plumber?
Is it so he can clean pipes?
He was the one who [UNINTELLIGIBLE]
And this is the point.
He's an Italian, and he's the plumber.
So if you focus on him, what can we say this game is arguing?
That Italian plumbers have hard heads?
He doesn't even eat.
Do they not need to eat?
How do you--?
He eats mushrooms
So to me, Mario is a really interesting example because we think of it as a simulation because it has physics.
When it came out, it had the most advanced jumping physics ever designed.
It's interesting.
So there's a couple of things here.
There's so many of the fantastical elements.
Like walking mushrooms-
He can jump like five times
-and flying turtles and stuff.
Yeah.
Just the way that he jumps.
You also notice that unlike the other games, it's hard to find anything political in Mario.
You could say he's hoarding money.
Maybe he's a capitalist
There's this whole long essay about how he's a communist
Interesting.
Could you post that on the forum?
I will
One thing again, just [UNINTELLIGIBLE] to notice is it's gender politics.
So the only female is the princess.
And what does she do?
She sits and waits.
She has no agency, and it's all about the masculine.
So you can have a feminist reading
Is Bowser masculine?
That's an interesting question.
Because it's also how much do you bring in other elements?
Do you think about the cartoon when you think about this?
Do you think about the other games?
Do you just look at Mario as an isolated event?
Peach kicks ass, man
That's true
[INAUDIBLE]
Well there's the Super Princess Peach, actually.
But her [UNINTELLIGIBLE PHRASE]
She gets very upset.
She cries
[UNINTELLIGIBLE]
[INAUDIBLE] competitive development.
I think there's something you said about the idea that snappers-- is that a Braunembaun?
It's Powell
Powell and Braunembaun are working together.
Then [UNINTELLIGIBLE] but has Nintendo ever made a proper Peach game?
I don't think so
Especially in the American Super Mario II, when you can control Peach.
And she is popular because she floats.
So I mean again, this is to say that this theory emphasizes subjectivity.
There's just no lacking here that we would find meaningful or that we would actually pick apart.
With that said, I certainly think it works, especially with more political games, games that have more of a message.
And again, this is all stuff to think about.
It's not something we're necessarily looking for.
So anything else with [UNINTELLIGIBLE] in there?
There's one probable reason why Mario's club-- actually the original pre-Super Mario Brothers where basically there is one screen, and all this bad stuff is coming out of pipes.
And the idea is that you are trying to clear all the pipes of all the bad stuff
I never got that from it
The pipes are there
I guess in that case what you are asking what do I bring from outside.
And I recall the typing from that being a big point of what [INAUDIBLE]
So it's like a Donkey Kong, Mario was actually originally a carpenter.
And they retroactively changed him into a plumber because Mario Brothers, the original, was about plumbing.
But they wanted to keep the character the same
Mario's in Donkey Kong?
Yes
They retroactively changed his name and his profession
--Donkey Kong on Super Nintendo
So that's the example.
Anyone not played this one?
You want to give it a shot?
Surprise.
You have to play this game again
Do you want to shoot, Jeremy?
I was actually just thinking of their Cooking Mama simulation that started this
Oh, gosh.
Princess Pamela Andersen has been captured by evil Ronald McDonald, who plans on making her a part of his Unhappy Meals, along with the chickens who are tortured for McDonald's Restaurants.
Help save the princess, blah blah blah.
Arrow key is move, space bar to jump.
To beat enemies, jumping on their heads.
You can also hit the down arrow to do the proverbial heads thing.
[LAUGHTER]
Ah.
Pamela Andersen
I thought they weren't nuggets.
[MUSIC] [LAUGHTER]
I want to thank all of you for being here for the opening of my new vegetarian restaurant, [UNINTELLIGIBLE].
Ronald McDonald-- Finally I've found the perfect toy for my next Unhappy Meal.
[INAUDIBLE] Looks like we have to take the princess once again [UNINTELLIGIBLE].
Let's go.
[LAUGHTER]
Wow
Did you know McDonald's is one of the biggest sellers of chicken flesh?
And they use an outdated method of slaughter that leads to extreme suffering for birds
So.
Wow.
It's not obvious at all that it's from
You think it's from PETA?
Yeah
Ooh, a kitten.
They're cooking hamburgers
Are you getting any carnivorous responses?
[INTERPOSING VOICES]
I think Super Meat Boy is a response
You worried about [UNINTELLIGIBLE]?
What was that, spinach?
I like growth hormones as the reason for that
There it is.
I don't know if this goes against one of PETA's earlier ads.
It was a sticker that had a chick that looked like that and it said I'm not a nugget.
This character's name is Nugget
There's also a KFC version of this.
But I think this one has more.
[INTERPOSING VOICES]
Not really.
[UNINTELLIGIBLE]
You are the best Super Chicken Sisters player.
OK. didn't
Even the mustache odds.
Wait, why is Wario a bad guy?
What do they have against mustaches?
OK, anyways.
That goes on.
So this game is really different than the other games we've looked at.
Does anyone have a sense of that as well?
Slightly political
Not subtle
For the most part it's just a re-skinning.
And so we were discussing this in our other class, about Ian Bogost's perspective games.
We looked at, what was it called, it was Space Invaders
Tax Invaders?
No.
It was a McCain ad.
And it was about corporal spending
Oh yeah.
I know of that one
[?
CorpWars, ?]
I think it was called.
It's just a re-skinning, and so they are using the same metaphors to just get people to play it, and it's not really improving their point or it's not doing anything for their cause
The simulation of this game seems to not really apply to their point at all.
You're really just playing Mario, and it happens to look different.
As opposed to September 12, evolve where the actual simulation itself had meaning with what they were trying to say
[INAUDIBLE].
The mechanics are having to do with the political point, or the narrative
This is a secondary critique of Mario
Yeah.
What was up with that?
I don't even know, what were they trying to say about Mario?
They're kind of enunciating the fact that it's basically a Mario ripoff.
Nintendo [UNINTELLIGIBLE].
Yes, we know it's a Mario ripoff, so we're going to tell you that we're aware of it
PETA also has an established problem with Nintendo, with their Cooking Mama re-skin
[UNINTELLIGIBLE] technically --although Nintendo has licenses
Well I guess Nintendo received the brunt of PETA's wrath, in that case anyways
What is Cooking Mama?
It's a DS game that's about cooking.
[UNINTELLIGIBLE] because they're working out vegetarian options
Wow
So they made another flash game similar to this one where Cooking Mama kills animals and the features are really bloody and slitting their throats and stuff.
[LAUGHTER]
Remember we played the video game yesterday in [UNINTELLIGIBLE] Video Game Studies?
Was that a McDonald's video game?
Yeah.
It was at videogame.com.
And that one is a simulation of running McDonald's.
And it presents itself as serious, and at first, most of us in the class, as I was, thought it was produced at McDonald's.
And as it goes on, you can manage the beef farms and the soy farms, and then your slaughterhouse, and then your stores, and then your corporate area.
And at first it seems like, hey we're going to show you how the business at McDonald's works.
And then you get this farm portion, and there's the option to use [UNINTELLIGIBLE] crops.
And you think, oh I can do that.
It's a trade off.
Then you have to bribe the mayor so you can destroy his crops to put yours.
And you're like, wait.
Then you can choose to feed your cows sludge to make them fatter.
There's all these little things that you can add on.
You can bribe a climate expert to say that you're environmentally friendly
Are these bribes based on any fact?
I don't know.
It was a good game, too.
We played it, and it was really challenging
[UNINTELLIGIBLE PHRASE]
You know it's the same company
[UNINTELLIGIBLE]
CNS 590, and apparently it's almost impossible.
They do their own act as they corrupt everything
And that's advanced rhetoric, right?
To be successful, you have to--
The guy next to me was playing all-natural organic without corrupting anyone.
And he was-- he went to [UNINTELLIGIBLE]
Yeah.
But at least learn the actions you're taking in the game.
You're supposed to hear when you're walking past someone who just talked to you about how McDonald's sucks
That wasn't very effective.
This was just like, oh, I'm hungry now
If it's the difference between a simulation and a game, this is really more of a game than a real simulation.
In a real simulation, the mechanics have to relate to whatever it is
But again, who's to say?
This isn't any more or less a simulation than Mario was
I don't think Mario is a simulation either
So that's exactly what it is.
This game has a point to make, but it's doing so in a very narrative fictional way.
It's not leveraging the simulation.
And generally speaking, these games are just not as effective.
And that's one of the cool things about simulations.
You get to see how it works and experience things first hand.
Whereas, is this any more or less effective than a pamphlet or a brochure?
Maybe in terms of exposure.
A lot of people can play this game, but it's not doing anything that any other type of media could do.
And that's what I got.
So we're going to do some of the brainstorm exercise again and talk about projects for the next assignment.
Does anyone else have anything they want to add before we move on
There's a lot [UNINTELLIGIBLE] the issue of [UNINTELLIGIBLE] simulation.
I mean we'll be playing games that are ostensibly simulations, or at least satires
Which are a different crunch
[UNINTELLIGIBLE PHRASE] And I want to point out -- back to one of the earlier points-- [INAUDIBLE] system.
It doesn't necessarily-- it does make a game.
[UNINTELLIGIBLE] system doesn't have to be a real world focus.
It could be any system.
[UNINTELLIGIBLE] They're all very much based on fictional systems.
But Michael's game is particularly good, but they're trying to figure out what the system is in those storybooks.
And make them, relevent for those players.
[UNINTELLIGIBLE PHRASE]
So we can do fictional systems?
Not for this particular one.
Not for the assignment.
[INAUDIBLE]
Does anyone know why they came out with Lego Star Wars?
[UNINTELLIGIBLE] of the Lego games?
Because they're awesome
It's a funny idea [UNINTELLIGIBLE]
But I think Lego was not doing very well until they got the Star Wars license to make actual Lego sets.
And that was a big revenue [UNINTELLIGIBLE].
That was last year.
If you have a toy store in Europe, it will be half Lego.
I've had that experience of, wow, I remember when I wanted this when I was six.
And you can't get it.
Just everything that's sold in the US, legal basis kind of-- [INTERPOSING VOICES]
It just seems so funny that they have Lego Star Wars video game
Well, there's also Lego Harry Potter
Well, they make Lego everything.
They have Lego [UNINTELLIGIBLE]
Indiana Jones.
[INTERPOSING VOICES]
--Lego Star Wars.
It's probably more on topic for stuff like video game studies, but it probably works well with a number of classes offered.
that the video game-- [UNINTELLIGIBLE] It will talk about different licensing.
We'll talk about different franchises and how we [INAUDIBLE].
So if you are interested there was another similar point that I wanted to make.
Oh, all right, simulations.
There's actually a games simulations class that's being offered in DUSP here.
[UNINTELLIGIBLE] largely using simulations for educational games.
You saw [INAUDIBLE].
It's a really, really good class.
And he approaches it in a very similar way.
For instance, let's look at a model.
Let's look at those assumptions and what you're choosing to simulate, what you're choosing to leave out.
And he'll get to things like game play, [INAUDIBLE].
He's pretty well equipped after taking this class to start thinking about those issues.
But he looks a lot [UNINTELLIGIBLE].
How does someone constructing [UNINTELLIGIBLE] the basic first step of how to release simulation.
[INAUDIBLE]
That's for CNS 590 Education Exploration for Games and Simulations
It's multiple courses
Yeah.
It's definitely crossed with 11.
I think it's crossed with at least one other thing
CNS [UNINTELLIGIBLE]
Actually we can [UNINTELLIGIBLE].
So this time I'm going to do it is a brainstorming similarly as last time.
But instead of brainstorming the whole game ideas, which is what we did last time, I want people to pull out systems.
Pull out the real world systems that you might want to write a game about.
Because what if [UNINTELLIGIBLE] near the end up reciting two models, and one of the game play, it's probably going to be [UNINTELLIGIBLE] determined by who your teammates are.
And who already has a team?
We do
We had a different idea
I think I have an idea for us to do
It's possible.
So I will recommend being friends with it and sharing it.
You don't have to.
But if you are going to hold back, try to contribute a couple of ideas [INAUDIBLE].
So, as you know [INAUDIBLE].
Anyone or any ideas?
[UNINTELLIGIBLE] I'll start with Jeremy and then I'll work my way back again
[UNINTELLIGIBLE] trying to build a pyramid
OK, pyramid building
I told you the green one was good
That was a lot more than [INAUDIBLE] at
Rennaisance Italian political machinations
OK. Renaissance Italian politics
Your four years here at MIT.
MIT undergrad
A real-time tower of defense, but let's say you're in a tower.
Think of who saw Inglorious Basterds.
I said think Inglorious Basterds.
Remember the guy in the tower?
Something like that.
Sparking a siege or something
So American sniper or German plot base
Like tower of defense.
Very similar, except you're not military
Running a marathon?
National politics
From Las Vegas
What?
Surviving the Department of Motor Vehicles
Middle School popularity contest.
[INTERPOSING VOICES]
I'm not supposed to make any jokes [UNINTELLIGIBLE]
Being a cook in a really busy restaurant
Making nuclear fusion work.
Finding lab scale scientific endeavors.
I'm thinking NASA.
NASA planning
NASA scale types
The immigration process to America
High school student body election or student government
Evolution.
And also I give-- I know I love this idea-- ants trying to find their way home
Developing a music career
A career in academia.
Also immune system
Academia and immune system.
I like that
Give myself a fever, ah!
Auction house
Lost in Japan.
[INTERPOSING VOICES]
[UNINTELLIGIBLE]
Patrick
Making photos in a darkroom
Culturing bacteria
Transfat
Similar to the immigration to the US, crossing the Caribbean Sea in a boat or a raft
Or one, then the other
How many r's in Caribbean?
Two
I'm pretty sure it's one
I'm pretty sure it's two r's and two- let's see.
Oh yeah.
It's one r and two b's.
Damn
Learning [INAUDIBLE]
Westward expansion of America
Westward expansion
Travelers to Oregon.
[LAUGHTER]
The Oregon Trail
Tech support
Ooh.
You have committed suicide
Advertising executives at Apple
I read was this article about how Steve Jobs was [UNINTELLIGIBLE]
I'd think about a person's journey of understanding self
Self discovery
Also a presidential election
A presidential election
Shepherding.
[INTERPOSING VOICES]
Mechanics are kind of interesting when you look at them
Viking colony in the new world before they left
I don't know if you guys still have it, but I made a game [UNINTELLIGIBLE].
It's probably indoors somewhere.
You stole my idea
If two people have the same idea, maybe you could work on a team together
Making a movie.
Like pre-production
Being a castaway
Running a religious industry.
[INTERPOSING VOICES]
Raising a kid?
Right in the middle of that right now
Piracy without getting caught
As in high seas piracy?
No.
software piracy
Designing a board game
Brainstorming
Building a computer
Waking up
Waking up
Acknowledging a [INAUDIBLE]
A couple.
Imperialism.
Well it's another religious one.
Organizing a mass suicide.
[LAUGHTER]
Filing a divorce
I called finding a divorce
I said finding a horse.
[INTERPOSING VOICES]
Running away from the police
Crime scene investigation
Firefighting
Falling asleep
Waking up.
Falling asleep
Organizing a heist
Running the Catholic Church
[UNINTELLIGIBLE PHRASE]
Money laundering.
Tax evasion
Mafia
Is that the same idea?
They could be two separate ideas
They could be separate ideas.
Tax evasion happens small scale and large scale
Picking up kids flying into a summer camp at the airport and not trying to lose them
What?
Not losing-
Not losing arriving children
I could see that would be at an airport meeting kids game
A gang or mafia organization
Mafia organization
The spread of fire through different materials
Fire spread
Would probably work very well with the firefighting game
Carrying out a hit with a small team of assassins.
Like you go into a hotel in Dubai or something
This is the point of the game
Yeah.
Or dot-com boom and bust
Dot-com boom and bust
Rescuing Chilean miners
Running a city
We talked about [UNINTELLIGIBLE]
Putting on and taking off clothes.
Or putting on clothes
I'm pretty sure there are games where you are taking off your clothes.
[INTERPOSING VOICES]
Hot sauce eating contest
Which is already a game
Well, sure
Start a company, or a Silicon Valley bubble
So Silicon, dot-com boom and bust as well
Surviving a family reunion
Containing an oil spill
Making a [UNINTELLIGIBLE] lab.
[INTERPOSING VOICES]
Cooking's a game.
There's flavors and such
Getting drunk, or trying not to
Make a [UNINTELLIGIBLE]
Nuclear proliferation
I mean, bomb making plus international diplomacy
Living alone on an island
That's kind of like castaway as well.
Yeah, so island
[INAUDIBLE]
So bomb disarming.
Also Congress and Supreme Court as two separate entities
Sorry, you have to repeat that.
What now?
Bomb disarming.
Congress and the Supreme Court as very separate ideas
Oh.
Congress.
We've had very many presidents in the Congress Supreme Court area.
Let's see.
I lost track.
Calvin
Surgery
Operation
Yeah, I love that game
Running and orphanage
If I might, running a charity
Creating and spreading [?
means.
?]
It'd be great if you could actually spread [INAUDIBLE]
Being a tourist in Amsterdam
Very specific.
OK. [LAUGHTER]
Basic programs at
That's worse
Playing games
Wait, how do you imagine that you're playing a game.
And you play too many games.
And you [UNINTELLIGIBLE].
[INTERPOSING VOICES]
[INAUDIBLE] Being an
Twitter
Being a pro gamer
The pro gamer game
Managing traffic intersections
Traffic intersections
Being a programmer.
A software designer
We have programmer and pro gamer.
Two different things
Fishing.
And losing weight
The historical whaling industry
It's not a current system, right?
[INTERPOSING VOICES]
Chastity and candle making
Unicorns
Wait, unicorns?
Maybe we should combine that into medieval guild operations
Running railroads
Railroads
Lumberjacking
Spelunking
Stock market speculation
Stock market?
Or speculation in particular
Salem witch trials.
Or witch hunts in general
Witch hunts
Unions
I'm starting to think that [UNINTELLIGIBLE]
Building a house
FedEx
FedEx
Spreading a mass hysteria.
Like McCarthy style
Also known as [INAUDIBLE]
Cold War spying and the internet
Cold War espionage.
And the internet
Communism
Computer virus
Hurricanes and oceans
Could you do something like astrology, which is not a real-world system, but is-- [INTERPOSING VOICES]
Yes.
You can do astrology
[INAUDIBLE] virus
Yeah.
But that would be more like a real virus
I would suggest limiting yourselves to just the real world.
I mean, coming up with things like zombie outbreaks, sure there is a system you can model.
But if you're just trying to push on the edges of the constraints, then you're just trying to give yourself an easier time.
And more often I see the results actually turn out to be less interesting because people are just going for the ideas.
They just [UNINTELLIGIBLE] them really, really quickly.
People aren't actually trying to work it.
But that being said, even with a lot of the ideas up here, a lot of them have connections.
Even there will be the obvious ones.
Like oil spill containing and oil spill starting could practically be the same thing.
Everything with the presidential election.
Spreading mass hysteria [UNINTELLIGIBLE].
And these ideas are now up on the board.
You have until the end of Friday, I guess.
We don't really need to know until the beginning of next week.
So actually by the end of Friday, you should know who is on your team and what project you're working on.
Over the weekend, you know who's on your team.
You know to email people on your team.
And what idea you're starting with.
Again, the idea can diverge a little bit, but I like the previous project.
We do want you to keep with whatever you started with and use it as core system.
You may choose presidential election.
It may be something about winning votes.
And then you decide no, you want to make a game of gerrymandering.
That's fine.
Because it's still keeping in with the same core of the system.
But say oh, we want to do something about winning votes and then instead it became about religion or something like that.
I could see how you got there, but that's not really keeping with the spirit at this particular time.
So stick with the original core system that you're working with.
And try to find the fun in it.
Don't try to like, oh, this is a really dull simulator mechanic, but if we add a bunch of [UNINTELLIGIBLE] then it becomes funny.
No.
Try to find it fun in the original system.
And if you can't try to pick apart the system and find the one component that is fun.
So all of bomb making might not be so much fun as as [UNINTELLIGIBLE] the blood-- Yeah, that might be the fun part.
The one part, I'll be, oops my hand slipped, and therefore I am now dead.
That part might be actually kind of fun, as opposed to OK, now I need to figure out how to get [UNINTELLIGIBLE].
Cool.
All these ideas are going to stay up on the board for a while.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
Ok, cool.
So we've been reading the assignments.
And we realize that there were a couple of things about how to write rules that we hadn't neccesarily gone through.
One thing that we would like to see in the next assignment are a couple things that I'm going to read out now.
But we'll probably send out an email, as well.
So you can have it for reference.
So having a paragraph or an overview that broadly explains what the game is about is actually really, really useful.
Just jumping straight into the-- this is how you set up a border and this is how you take a turn.
That's the meat of the assignment.
But just to help people get a grasp, especially if you thought about your theme really, really hard.
And this next coming assignment, your theme is right there, explaining who are you in this game?
What are you trying to do?
What's your overall goal in broad terms?
Not necessarily very specific game terms.
Just having one paragraph is a really, really good idea.
And really, that [UNINTELLIGIBLE] paragraph.
Two, list the components.
In a card game, it would be like we have this many red cards, this many blue cards.
Just so we know that we've got everything.
It also helps, sometimes, to be able to establish the vocabulary of the game really, really early.
So if all your red cards are attack cards, and all your blue cards are defense cards, you want to say your deck should have 25 red attack cards and 25 blue defense cards.
And now we know if you use the words attack card and defense card later, we now know what it means.
So this is obvious from just looking at currents.
But frankly, when we are opening up these cards, usually, there is one person looking at the cards and somebody else looking at the rules.
And just having the information redundant is kind of nice.
Let me see.
The number of players-- in this particular assignment, I'm trying to remember what the requirements are for.
It was like two to four?
Two to four
Two to four.
A number of your previous projects actually allow less players.
We want an explicit line that says this is how many players your game supports.
Just one line.
Make it like a bullet point.
By the way, bullet points-- be concise.
Making things really easy to read is highly valued.
One thing that you absolutely should do is make sure you're testing your written rules as well as your game.
Don't just test the rules of your game.
And then, you write it down.
And you're never actually testing the written version of your rules on someone.
You need to be able to test your rules, as well.
There are some games that clearly need that.
And some games that clearly don't.
And it showed.
By the way, no one really did badly on the last assignment.
I'm nitpicking because this is, obviously, from B to A.
Let's see.
Expected playing time.
Especially in this next assignment, if a game tends to take 30 minutes, or if your game tends to take 10.
Just state it.
In some games, it takes a really long time to set up and a really fast time to play.
Then, you just state expected set up time, expected playing time [UNINTELLIGIBLE].
There was a lot of assumption in the rules and in the design of the game that your players are male.
This is a very male-proportioned class right now.
But don't assume that your players are all going to be male.
It's actually really kind of off-putting to read a set of rules that just assumes that if you're playing this game, you are a guy.
There are a couple of things that people have done in academia to address that issue.
I've seen papers where they've used nonspecific third person pronouns.
They just [UNINTELLIGIBLE] he/she all the time unless they're talking about a specific person.
But in your case, when you're thinking about your themes, don't assume that your players are male.
You can cast them in the role of a male.
But don't assume that a player, itself is male
So I'm just going to say-- put this out there.
In the English language, it's actually very masculine oriented.
So the proper way to say in a third person, if you're not saying one says this, it's actually appropriate to say he.
That's actually proper in the English language
And I'm saying it's off-putting from a marketing standpoint to assume that your players is male.
It goes more than just gender specific pronouns.
It goes way more than that.
There were a lot of assumptions-- there were specific rules that assumed that you were a straight male in a couple of games.
So let's be clear about that.
[UNINTELLIGIBLE]
I was just going to mention that from what I've seen in a lot of rule books, it seems like a lot of people solve this problem by just replacing the assumption that the player's male with the assumption that the player's female.
Yeah, I've noticed this.
Every single pronoun in every like post second edition Dungeons and Dragons book is female.
It's kind of interesting
That's actually kind of an interesting way to deal with it.
It's just like you have a lampshade.
You've got to use a pronoun, but we don't want to use language that's going to turn people off the game.
So we just want to swap all the pronouns around.
And all of a sudden, you actually have a more marketable game.
You have a game where someone is going to read the rules.
And if they're a straight male reading the rules, it's like OK. Clearly it just was a gender pronoun swap.
But I can read these, and it makes sense.
Whereas, someone who is not a straight male reading the rules.
They're going, oh, this game is not for me.
So I'm not going to play the game.
That kind of reaction does happen.
Board games are supposed to be inclusive.
Board games and card cames are supposed to kind of like be these huge socially enabling games and [UNINTELLIGIBLE].
So keep that in mind.
You want people to feel welcome when they read the rules
If I could just interject.
So something that I've seen that happens a lot in like [?
bridge ?]
literature, is they make your partner-- so one person, specifically, is female and the others are male.
And that way, you kind of have this inclusivity.
But that helps clarify who you're talking about.
So if you are writing up examples, or sample turns, or that kind of thing, you can think about doing it that way as well.
Because it does the whole inclusivity, but then again kind of clarifies who you're talking about in the set
And we're in university.
We're trying to be professors about this
I find it, generally, people's aversion to writing using pronouns, which are identity neutral specifically, people worry that there's an ambiguity between singular and plural.
And that's why you shouldn't use that kind of pronoun.
But, for the most part, it's almost always obvious when you're talking about one person [INAUDIBLE]
I don't even see really why this is still an issue
Yeah.
It's one of those things where it's not technically correct to say if I'm referring to a single person to say they or their.
But then, if you're reading, other people will say do that.
A lot of people are just still using he or she now.
It's certainly awkward.
It's kind of [INAUDIBLE]
You can also use second person language-- you.
When it's your turn, you blah blah blah.
And also, the one player is a great substitue for all that.
Say player one, player two.
[UNINTELLIGIBLE].
And it's tough.
It's tough to write stuff like this.
But sometimes I find writing constraints, writing gets better because you're more focused on how your writing is going to be received.
And we're, frankly, trying to encourage everything in your games to be better.
I guess this is something that I hit on earlier.
Try to go for short sentences.
Not paragraphs or prose.
Those are actually difficult to read when you're trying to also process what the moves are going to be.
Go for active voice, rather than passive voice.
I personally, automatically, write passive voice when I'm writing in a report.
And I have to physically change myself and tell Microsoft Word to highlight every single instance of passive voice so that I can accurately correct that.
Microsoft Word doesn't do a great job in highlighting every single instance.
But if you do this often enough, you start picking it out yourself.
It's actually easier to read.
It's uncomfortable if you're moving from the field of writing chemical papers into writing for general public.
But active voice is easier for someone to read.
[UNINTELLIGIBLE] This one I might need more help and detail.
These substeps of actions
When you're going through the instructions, it's really helpful to have numbered sequences instead of having a chunk of prose that's saying first do this.
Then you do this.
Just bullet point it.
Or make it a numbered list.
And it's a lot easier for people to follow.
Generally speaking, bullets versus numbers.
You should use a numbered list if a sequence really matters versus bulleted lists are more like these are the things that are important.
But there's not necessarily a temporal relationship [UNINTELLIGIBLE]
So just some tips.
In the end, every single game that we played, we could play.
It didn't take us long to figure out what was necessary.
The games were good.
The games were entertaining.
[UNINTELLIGIBLE] extra people to come in and play.
And we don't [UNINTELLIGIBLE] to play again just the two of us.
Yeah.
I'm actually really, really impressed because it's a marketing group.
I know with previous classes [UNINTELLIGIBLE].
[APPLAUSE] [UNINTELLIGIBLE] Yay.
All right.
So that was the stuff on your assignment.
And now onto intellectual property.
Actually how many people figured out that we had given you the wrong chapter to read
Yeah
OK. All right.
For those of you who ended up rereading chapter three, I'm sorry.
It's not a huge chapter.
But I kind of want to talk about two different things, both of which are related to intellectual property.
What is the version of intellectual property that presented to you in the book.
In that sense, that's kind of the game designer concept of what intellectual property is.
Often it means a franchise.
Often it means somebody else's story wall or characters.
Maybe it's real people and images, like the faces and the bodies.
In sports games, you have to deal with that a lot.
There was a conference this last weekend.
And one of the old guards of the Boston game development team, who now runs iRacing, was talking about the amount of effort it takes to actually license a NASCAR car.
Because it's not just a car.
But every single logo that's on their car, and the person who's inside it, and the uniform that they're wearing, every single logo on their uniform, and their face.
There's a lot that goes into licensing for that.
It helps that the person who now owns iRacing is also the same person who owns the Red Sox.
So they've got the bank roll to be able to do all of this.
But that's the idea of what IP is from industry point of view.
And there's a couple of things that you can do to consider that.
Not just all right, how do we design this game so that we can minimize how much IP we need to license?
Licensing IP generally means licensing money that needs to be exchanged.
That means that your splitting up their royalties or you're just putting up money up front.
Of course, this goes for things like licensed music, for instance.
EA has an entire music department.
Sega has an entire music department.
Its job is either to create their own music, or more likely, license popular music.
And, of course, with Harmonix, that's half the business right there.
Probably be bought, at this point.
The other thing they can do is that, assuming that you are working with a license.
And the reason why you're working with licensing-- think of Star Wars, which came up in the last class.
How do we do justice to it?
How do we sort of make sure that the fans of those particular intellectual property franchises are going to look at this game that we're creating, and say, yeah, that's faithful?
That makes sense.
We'll be actually playing a lot more of those games, I think, later on.
For things like Battlestar Gallactica.
But let's take LEGO Star Wars as one example.
If you're a fan of LEGO then, you're probably a fan of building things up with little tiny blocks.
You like making stuff combine.
And you kind of like the whole blockiness, that look of it.
So the folks who made LEGO Star Wars-- I think they are called Traveller's Tales.
And LEGO Indiana Jones and those there was a bunch of games-- kind of had to play that up.
They made a lot of obstacles in the game.
Things that you had to put together with parts.
Sure, it is holding one button into assemble it from a bunch of disconnected parts.
But creating all that animation and sort of making-- this is one of the main game mechanics of the entire game was a decision they had to make.
and they're very consciously making that because with LEGO, we're going to assume people are going to make things.
And then you find parts in the entire game that you can later go back to a trophy room and can see your trophies, statues, and objects from the game builds that you're trying to do sort of built up.
So the more parts you find-- they're like the hidden collectibles that you find in Grand Theft Auto.
They're kind of hidden away in secret locations.
And you find more of them-- you go back to your trophy room and oh, wow, your X-wing.
Or you have part of an X-wing.
You haven't found half of it.
So all you've got are the wings.
And none of the guns or something like that.
They've actually figured out a way how that weaves back into basic game mechanics but is still appealing to LEGO fans.
If you get chopped in half by a lightsaber, you fall into pieces.
You don't actually bleed.
When the characters walk, they actually walk like that.
They're a little bit more flexible than actual LEGO pieces.
They don't just move down like the right angles.
But they try.
They try really, really hard to make everything look plasticy.
And it's [UNINTELLIGIBLE].
The same thing goes with Star Wars.
Right?
What are the things that people expect from Star Wars?
Lightsabers
Lightsabers.
OK.
Anything else?
The Force
The Force
Darth Vader
Darth Vader.
You've got to have Darth-- Well, they're doing it on the Episodes Four, Five, and Six.
Actually, I believe Darth Vader is still unlockable on Episode One, Two, and Three because people expect there to be a Darth Vader
Generally, they use the main characters from the films
Yeah, the main characters from films
Dog fights
Dog fights?
You mean like-- that's actually interesting because I believe they actually didn't do that.
They didn't manage to put that in.
But in a lot of other Lucas games they did.
They focused mostly on space fights
[INAUDIBLE]
Catch phrases
Catch phrases.
I think they've weaved the catch phrases into the names of the levels because they can't-- not that [UNINTELLIGIBLE]
Nobody ever says, I've got a bad feeling about this?
I think there's a title of a level that says, I've got a bad feeling about this.
Because there's no dialog.
No one says anything.
The only thing that says anything is R2-D2.
And that's just a voice over
Actually, I have a really good example of catch phrases in, I think Rogue Squadron on Nintendo 64.
There's this line in the Star Wars movies, where Luke is like, that stabilizer's broken loose.
If you can, lock it down.
Every like second time you get shot, they play that recording.
[LAUGHTER] And it just even like cut scene you where this other pilot is like, Luke, my stabilizer's broken loose.
And he's like, see if you can't have your R2 unit lock it down.
[LAUGHTER] So yeah, I mean a good example of the worst end is that game
You can over do it.
I'm trying to remember how the force works.
I think you can assemble LEGO blocks from a distance in Star Wars if you're playing Obi-Wan.
It's kind of, "oooo."
And then, [UNINTELLIGIBLE]
[LAUGHTER]
Call Yoda.
Obviously, Darth Vader is in there.
Darth Vader is in the first cut scene, obviously
Yoda is?
Yeah.
All the big characters have got to be there, right?
You've got to have your Luke.
You've got to have your C-3PO.
And the interesting thing about every LEGO game-- this is a nice mix between LEGO and Star Wars is that you can play any level with any character.
So after you've beaten it once, I think you unlock the ability to swap characters in, which is more of a LEGO kind of thing, right?
I'm just going to take the characters from the Harry Potter set and put them in the LEGO Star Wars game and see what happens.
But that's one thing that you get.
You get to unlock all these characters.
So you can play the whole game with [UNINTELLIGIBLE]
You want to add the locales?
[UNINTELLIGIBLE]
Oh yeah.
The locations.
At least some acknowledgment that there is some sequence of when people go to each locale
Yeah, the LEGO games, in particular, basically, follow the events of the movies, specifically.
So it's like each separate event in the movie is kind of it's own level
Mos Eisley Cantina is one chunk of this one level.
And you beat that, and you're going to-- what happens after the cantina?
[INTERPOSING VOICES] Tatooine
Probably in some Death Star
Death Star yeah
Oh right, right.
I was going to say, someone should invent a game where it has all the LEGO games.
And you can any of the characters, interchangably between them
Well, there's one company that already has technology for that.
And I would to see LEGO Star Wars slash Indiana Jones.
Even better you can change [UNINTELLIGIBLE] Indiana Jones, which is about it
Like take off the hat
Blaster?
Whip!
One thing which these particular games have done a really good job with is they'll always bring the music over.
Actually, every single Star Wars game has been very, very careful about making sure that it sounds like Star Wars.
The sound effects and the music.
[INAUDIBLE] A lot of it is actually developed with the Lucas Arts license, which is, obviously, under the Lucas umbrella.
So it makes it a little bit easier.
The company of the whole Lucas family, Lucas Arts, Lucas Film is actually really, really tightly centrally controlled by Lucas, himself.
Partly for quality control, partly for control freak issues.
In the end, what it does make it really, really easy as for say-- how many folks paid for The Force Unleashed?
The live action trailer was awesome.
The second one?
Oh, I haven't seen live action trailer
[UNINTELLIGIBLE] they blend the in-game footage and live action to get back and forth.
And it's really hard to tell where [UNINTELLIGIBLE]
I actually wonder whether the character is actually based on the case of the real world
It must be because it's the exact same face
OK. A lot of the assets that are used in the movies were brought over into the game.
Obviously with with some tweaks to make it run better in the game.
A lot of stuff in the force unleashed, like when you use your force power and smash things, which is actually most of the game, if I recall.
A lot of that same technology is actually technology that they use in the films to simulate things being smashed.
That being said, that particular example is probably not even a good instance of something that any player is going to notice.
It's like, why do I care that things that are breaking in the game, break exactly the same way in the movie?
But they do.
More usefully you see art assets.
You see the same concept art moving between the games, moving smoothly.
Traveller's Tales is actually an illegal Star Wars game.
They're a little bit further out.
Obviously, those are third party.
Those are separate from the Lucas family.
They had to actually license both LEGO and Star Wars.
But they already had a relationship with Star Wars.
[UNINTELLIGIBLE] So it was a little bit easier for them to negotiate that.
And as a game developer issue issue, I believe, with the [UNINTELLIGIBLE] and how they actually managed that.
Again, they're going to be very, very capital.
Even though it's not Lucas's own company, they're going to be very, very capital and at least matching up with Lucas's control of the product.
And if you're a developer working with a license, often, it's just a long process of just earning trust back from the original license holders.
[?
Jason Beam ?]
is actually a partner entering this lab.
And he used to work with THQ, which made a lot of DS and Wii titles, specifically on Disney and Pixar products.
One problem with the DS is that it doesn't have a huge amount of 3D processing quality.
Whereas, obviously Pixar movies have all the power in the world to do awesome looking 3D.
So some of the tricks that they have to do to be able to get the characters-- to be faithful to the characters in the movie are kind of interesting.
So the folks who work for Pixar films-- a lot of them have these opening and closing sequences that are actually animated in completely different style.
Maybe it's hand drawn, maybe it's paper cutouts.
And those are still emblematic of Pixar style, even though they're not super 3D, realistically rendered.
And a lot of DS games that [?
Jason Beam ?]
worked out will use those tricks.
They'll make a game.
Instead of making it look like it's all 3D, you can make it look like it's all cut out of cardboard because that's part of the movies.
And the DS has a much easier time with that.
One thing that's kind of useful when you're thinking about what to take from a franchise or license is to look at the verbs that come out from that franchise.
I think we have enough time.
I'd like to do an exercise where we pick out a franchise.
And you just try to come up with a bunch of verbs that make that franchise unique or at least you expect to see in a franchise.
But I'd like some examples.
I'm going to just start with movies and see if you can get at least one that everyone's fairly familiar with.
And if not, then I'll try books or other kinds of media properties.
Anyone want to throw out a movie that maybe everyone in here should be familiar with?
Independence Day
Independence Day?
Has anyone not seen Independence Day?
[INTERPOSING VOICES]
Actually, let's just throw out a few brainstorming ideas.
There's Independence Day
[INTERPOSING VOICES]
I'm going to go this way
Ferris Bueller's Day Off
Twilight
[INTERPOSING VOICES]
Lord of the Rings
Inception
Iron Man
The Matrix
OK. We hit some in there.
And I think that's all I can remember.
So Inception, Iron Man, The Matrix, Ferris Bueller's Day Off, Independence Day, Twilight, Lord of the Rings
I've never seen Twilight.
I'm going to pick Inception.
Who has not seen Inception?
You should
You haven't seen Inception?
[INTERPOSING VOICES]
Which have you seen?
I've seen all the other ones
[INTERPOSING VOICES]
My problem with The Matrix [UNINTELLIGIBLE]
Has everyone seen Ferris Bueller's Day Off?
Nope
Have you seen it?
Nope
[INTERPOSING VOICES]
Monty Python, yeah
Monty Python?
Does anyone not know Monty Python?
Wait--
[INTERPOSING VOICES]
--Holy Grail
[INTERPOSING VOICES]
Home Alone
[INTERPOSING VOICES]
There might have been a Home Alone game, but I bet no one here has played it
I have
[LAUGHTER]
[INTERPOSING VOICES]
That is impressive
They're for the original Game Boy
Oh, okay
It's actually one of the first Game Boy games my brother ever got
That's old
Yeah
OK. Let me try Lord of the Rings but maybe a specific Lord of the Rings movie
[TRANSPOSING VOICES]
--The Matrix.
How about The Matrix
No, the problem with The Matrix is that there are too many games on it
The Two Towers?
I'm really curious about Ferris Bueller's Day Off as a game.
If somebody would like to volunteer a quick summary of what Ferris Bueller's Day Off is about?
Which is Matthew --
[INTERPOSING VOICES]
Well, OK.
So the basic idea is the main character is Ferris Bueller played by Matthew Broderick.
The idea is that he's the infamously super popular like class clown kid in school.
And he decides to just play hooky with, essentially, his girlfriend and his best friend.
And the whole movie is them going around the city getting into wacky misadventures while the principal tries to catch him playing hooky because he hates him
That is awesome
It is incredible
So actually that white board would be--
Has anyone heard the Fight Club theory on Ferris Bueller's Day Off?
No
[INTERPOSING VOICES]
Ferris Bueller and his best friend
[UNINTELLIGIBLE]
Yeah, Cameron
[LAUGHTER]
I'm going to stand at this screen
[INTERPOSING VOICES]
Wait, who hasn't seen Fight Club
Oh, Fight Club.
That's a pretty good movie
Has everyone seen Fight Club?
That's another one
[INTERPOSING VOICES]
--because I think, that game would have benefitted from the exercise that we're about to do
[INTERPOSING VOICES]
Like do both controllers control the same character?
[LAUGHTER]
That's some Metal Gear Solid stuff right there
Psycho [UNINTELLIGIBLE]
[INTERPOSING VOICES]
So I just got to bring out the note pad thing.
And I want people to start thinking about verbs.
What are the verbs that make Ferris Bueller's Day Off kind of unique?
Or really that's what you think of when you think of Ferris Bueller's Day Off?
Verbs?
Yeah, verbs.
What do they do?
What do the characters do?
Bad karoake
[UNINTELLIGIBLE]
Bad Karoake
Sing
[INTERPOSING VOICES]
Let me make sure we've got [UNINTELLIGIBLE]
Live life
Lie.
They lie
OK.
So they sing.
They deceive or lie
They run
They run?
Elude.
[LAUGHTER]
They definitely elude
[INTERPOSING VOICES]
They drive
Destroy
They [UNINTELLIGIBLE PHRASE].
They trick
They trick?
That's deceive
[UNINTELLIGIBLE] be spontaneous
They be spontaneous
[LAUGHTER]
They improvise
They get in trouble
They get in trouble.
There's not a real verb for that
And get out of trouble
They mischief
[LAUGHTER]
[INTERPOSING VOICES]
Well, mischief is a word that should probably be in there.
That might be an actually a back of the box or tagline.
[UNINTELLIGIBLE PHRASE].
What else?
Anything else they do there?
They hide
They hide?
[UNINTELLIGIBLE]
Hide, run, elude.
Those kind of [UNINTELLIGIBLE]
Exploit maybe?
Exploit?
Yeah, they take advantage
OK.
They exploit [UNINTELLIGIBLE]
They impress
They impress
Relax
That's good.
At some point they relax
[
One guy is very relaxed
They kiss
They kiss
They hack
Do they hack into a computer?
Into the school computer
[INTERPOSING VOICES]
Oh yeah.
I forgot about that
[INTERPOSING VOICES]
It's going to take a little bit more interest
[LAUGHTER]
[INTERPOSING VOICES]
The school has a [UNINTELLIGIBLE] That's too easy.
There's a movie about that
[UNINTELLIGIBLE PHRASE]
That's War Games
You ruined it
[INTERPOSING VOICES]
Let's see.
What else?
It's a classic line.
[UNINTELLIGIBLE]
Do they ever support each other?
Or do they just stress each other out?
They may just stress each other out
They plan
[INTERPOSING VOICES]
Scheme.
No, they scheme
Scheme
Scheme is so much better than plan
I don't like these negative words for him
This may not be what the main characters do, but they campaign.
They run a campaign to save Ferris.
So the idea is that Ferris is basically pretending that he's deathly ill. And so a bunch of the students in the school are running a campaign to raise funds for his medical expenses
Well, everyone says it.
Like the police station says, tell your brother feel better.
And also the water tower has Save Ferris printed on it.
[UNINTELLIGIBLE]
Yeah.
And you can think about what those verbs associated with like putting signs up and painting.
And if you look at this list, the list can probably get much longer.
But this is a fairly unique collection.
Maybe it's all unique if you compare it to other, sort of, hijinks movies.
But it's not like we've had a lot of video games, in particular, that are in that game.
And we probably should have because it sounds like it's going to be a heck of a lot of fun.
Actually, the one theme that reminds me of this is actually Bully, which was released by Rockstar.
Where you sort of do these things.
But it's much more vengeful.
It's much more I'm going to show the other bullies in the school how [UNINTELLIGIBLE].
Whereas this one is kind of more good natured.
They parade
[UNINTELLIGIBLE]
[UNINTELLIGIBLE] is also part of the singing badly part
Yeah
You get the girlfriend and then they sing [UNINTELLIGIBLE]
But singing badly wouldn't have the same impact if there wasn't a brass band behind him.
So you can think about how any of these verbs-- and we've done this exercise by taking verbs and turning them into game mechanics before.
And you can think about how any of these verbs could actually be turned into a game.
Kiss might be a goal or it might be a rule, a mechanic, to be able to accomplish something.
Parade might be a setting.
Campaign might be a collection of mechanics that feed off each other.
It could be a whole system that tells you how successful your Save Ferris campaign is going.
You could make a game that's just all about saving Ferris, as opposed to playing Ferris Bueller himself.
And that's kind of one exercise that you can do when you're thinking about your reward systems and all the systems that you guys have [UNINTELLIGIBLE] as far as I can recall.
You don't have to worry about licensing the idea.
But you're basically working with a real live franchise.
And if you're doing something based on political diplomacy or something like that, then there are real life source material things that you're trying to be faithful to.
And going through this exercise-- OK. What are all the verbs in this idea that we're working with?
This isn't going to work-- What do people actually do?
It's a good exercise to be able to try to be faithful to [UNINTELLIGIBLE].
And then, you don't have to pick all of them, of course.
You're just going to cherry pick.
And you want to prioritize a little bit.
OK. What things can we get rid of and very still clearly be Ferris Bueller game.
Running.
I think as long as we have some kind of evation think, physical running may not be necessary.
You could do something like as long as they're not in one location where the principal is at any given time, you're good.
But you don't have to physically move through [UNINTELLIGIBLE] basically in all that
Drive, probably
Driving.
Yeah.
It's such a minor part of the film.
Yeah.
The actual driving is such a minor part of the film that you can probably do without that.
And you might actually want to think about what if instead of this being Ferris Bueller's day off into Ferris Bueller's multiple days off.
Maybe just play each day differently.
And just trying to stay out more and more days.
Maybe a score or number of days you don't go to school or something.
So that's an idea.
That's one approach that you can use in your projects.
Before we hit two, I just want to touch on another point.
Before I transition, any questions about dealing with IP ?
There is a point that Brenda makes in her book-- the Brenda [UNINTELLIGIBLE] book-- that if you go professionally in the game industry, you will most likely at some point in time, probably many times in your career, work with a licensed product.
And it can be painful because you do not have greater control yet trying to be faithful to what you're working with.
And if you succeed, you make a lot of money.
And maybe you get enough money to start working on your own projects.
But it's a reality because franchises do sell.
And they're a lot lower risk than dealing with a completely new idea.
By the way, if you haven't played the new Tomb Raider game, which is actually kind of interesting because it's like Tomb Raider meets Diablo.
If people have never... Whose played or seen some one play Tomb Raider before?
OK. Who knows about this latest game that I'm talking about.
It's kind of like 3/4 isometric view.
Yeah.
It is Tomb Raider meets Diablo but it still keeps in with the puzzle solving.
It's cooperative.
It's a very, very different game in terms of how you play it.
But it will be an interesting exercise to sort of play that game.
Maybe I'll load it up here sometime.
And you can take a look and how does that resemble the Tomb Raider franchise so that people who look at the game and still say, yeah that's Tomb Raider even though it's a very new game.
You can imagine like a Tomb Raider board game, for instance, and how would that be different.
And how could that still be faithful?
That's World of Warcraft games and Starcraft board games that we invent.
[UNINTELLIGIBLE PHRASE] But I'd love to try it out.
Let's see.
So the other point I want to get to is kind of the legalistic definition of IP.
And I don't want to spend an entire class on this because I know you will all fall asleep.
I just want to point out that there's quite a lot of different ways that intellectual property is protected.
People who are creating stuff professionally, creatively, want to make sure that they can finish their product, they can make sequels of their product, they can maybe even license their ideas to someone else.
Make IP for some company without having ideas copied outright and not seeing any compensation for that.
If it is copyrights, there's trademarks, there's patents.
I feel like I'm missing something.
I guess we have those three first.
And if anything-- those are definitely the three big ones.
Copyright.
Copyright basically protects an expression of an idea.
The idea, itself, is not protected.
If your basic idea is in a world where aliens are attacked by space marines, the idea, itself, is not protected, which is a relief because a lot of people have many games on that idea.
But the actual expression is.
So the actual-- say that you're an artist.
And you're creating concept art based on that theme.
That piece of concept art is copyrighted to either you or if you're an employee of a company then you've probably signed over your rights to the company.
That whatever you're creating on their dime belongs to the company.
And that's defensible in court.
It's actually an automatic protection.
But it's one of those things where it helps to be able to document that you've actually created something.
There's a copyright office that you can send things to.
But for the most part, internal company documentation saying, this piece of art was created on the so and so date by so and so person.
It's usually good enough to stand up in court if there's ever a dispute.
So the same thing goes for writing.
The same thing goes for audio and music.
A lot of music is protected under copyright.
So if somebody creates a piece of music that sounds an awful lot like your piece of music.
There are specific rules that allow for that to happen.
But I think, if you completely reperform the piece, you are allowed to do that.
But you may have to pay the original composers the royalties from the music.
This means that if person A composes a song, then B performs a song and releases an album, then C could actually reperform that song.
And the only royalties goes back to the composer.
It doesn't necessarily go back to Band B.
However, Band B-- it depends on really how close they sound to each other.
And, of course, if Band B happens to be also the composers, then they're going to get royalties and win.
But you see this idea that copyright protects the expression of an idea.
The composers, at some point in time, had to write down the music, the notes, and the lyrics.
And that is what is protected.
And you can't just copy that without entering into some sort of contract saying that you can.
We have systems like Creative Commons for instance that basically, people who are creating your IP say, under these conditions, I am letting people do whatever they want with it.
And there are different kinds of Creative Commons licensing.
Some of them that you do-- literally whatever you want with it.
They're like open source licenses.
And in some cases, it's like well, you can do whatever you want with it as long as it's for non-commercial use and you have to credit the origin [UNINTELLIGIBLE] So that's why Creative Commons evolved.
So to enable people to freely do stuff without necessarily having to pay and enter into separate contracts with everybody who is creating IP.
So the problem is with code.
If code and game mechanics actually for this class is something I want to consider.
Because you can think of code in two ways.
One, you can think of it as a chunk of text.
It's a chunk of ASCII symbols that you type into a development environment and you compile it.
And that's an expression of an idea.
But the other way that you can think of it is it's a machine.
It's a bunch of instructions that you feed into a machine for it to operate in a certain way.
And those things, machines, inventions basically, can be protected by patents.
That's pretty much what the patent system is designed for.
Patent system is a lot slower because you actually have to apply to central US Patent Office to be able to secure those, assuming that you're looking for patent protection in the US.
Every single country, every single region has it's own patent office and it's own process.
It's really expensive.
What it does is basically give you a temporary monopoly on that invention.
At least, that's what they were originally designed for.
You make an invention, and you've got like 25 years or something to be able to capitalize on it before anyone can use your idea.
And the whole reason why it was invented is that you now have an incentive to be able to come up with inventions.
You might make some profit off it.
And after the patent expires-- 25 years or something-- because you've released your information to the patent office, now, all of society can benefit.
Can make derivatives of your invention.
So in a sense, it was invented to eventually release great monopolies.
I got this idea, and I am going to be upfront and very clear about how I make this thing work.
Say Amazon patents 1-click.
And I'm going to say for the next 25 years, Amazon now has this lock on 1-click shipping and purchasing, which is [UNINTELLIGIBLE] but it's been [UNINTELLIGIBLE].
But after 25 years, anyone can do 1-click shipping.
But that's the problem, right?
25 years is a heck of a long time.
And by then, whatever is invented is kind of obsolete.
And a lot of the ideas that tend to get approved by the patent office are pretty damn obvious.
I'm not going to discourage the amount of work that Amazon has actually put into 1-click shipping.
It's actually a really complex process.
But I am going to discourage, say there is a patent on the giant floating arrow that tells you where you're supposed to go next.
They have a patent on this.
Namco has a patent on mini games that you play while your game is loading.
That's why only Namco games have mini games in those screens because they have a patent on that.
When it comes to game ideas and game mechanics, I find this really annoying.
It's a personal rant because I think that's [UNINTELLIGIBLE] creation is that opening it up is really, really tough.
I don't really think game mechanics fit in the [UNINTELLIGIBLE]
[UNINTELLIGIBLE] Apple has is on all the touching.
On their gestures, a lot of people can't get off the same thing
A lot of people have the same issue that I have with game mechanics, with software, in general.
Software happens so quickly that this kind of temporary monopoly basically means a real monopoly over that idea
I thought that game rules were not patented or covered by a copyright.
I seem to remember hearing that somewhere.
And this was an issue which came up when Scrabulous and Scrabble were at loggerheads with one another.
There's some weird technical detail that I don't really know, but it kind of comes down to the actual rules of Scrabble, the actual game rules themselves [UNINTELLIGIBLE] are not copyrightable.
But somehow, the look and feel of your game is.
Do you know what the specific details are?
I don't know the specific details.
But how you outlined it is one very clear way on how someone who's created a game can protect a game without necessarily being protective of the rules.
And Scrabble can protect the layout of their board.
Because in order to come to design the layout, they had to express the idea of what the board was with graphic design.
It's like here's where all the triple word scores are.
Here are where the double word scores.
That's an image.
Images can be copyrighted.
And that's protected under copyrighting law, not under patent law
And I think [UNINTELLIGIBLE] of copyrights and trademarks, it comes down to like confusion, right?
Like if someone could-- if Scrabulous was similar enough to Scrabble, superficially, that if someone could use their products.
I think that is like a major criteria, right?
And then again, it comes down to things like font, colors, titles, [INAUDIBLE]
It's even more specific when it comes to trademarks.
If you can claim that the image or a particular saying or something is closely associated with your company's core brand, you can defend that as a trademark if you registered it with the trademark office.
GAMBIT, the Singapore [UNINTELLIGIBLE] game lab is actually a registered trademark in Singapore.
And actually, we had some trouble registering that because well, [UNINTELLIGIBLE] not the first company to come along with the word GAMBIT.
There's actually another MIT product called GAMBIT.
It's a game theory modeling system.
So we actually had to negotiate some fine words to be able to separate what we were doing was not going to be confused with what they were doing.
[UNINTELLIGIBLE]?
Are copyrights and trademarks also for 25 years?
Or are they different?
[INTERPOSING VOICES]
Copyrights last as long as Walt Disney is dead.
That is because the Disney corp keeps lobbying the various branches of the United States to extend copyrights so that they can lock down Mickey.
That's the easiest way to tell exactly how long.
In other words, right now, you've got something copyrighted, it's in perpetuity until Disney loses.
This is actually really annoying.
Which means there's a whole bunch of works, literature [UNINTELLIGIBLE] that date back to the elite 20th century that by all means really should be public domain at this point.
And then, they will pass into public use.
But they aren't.
I know one thing for sure.
H.P.
Lovecraft is definitely in public domain.
That's why there are so many games based on H.P.
Lovecraft material.
Whatever is based on Disney material, has to be made by Disney.
[UNINTELLIGIBLE] of the copyright law.
Images.
If the Scrabble board is identified as being key to the Scrabble brand, you might be able to protect it under trademark.
And trademark is one of those things where all you're saying is that this is how our company is identifying ourselves.
And if that is confused with somebody else's product, that product is now hurting [UNINTELLIGIBLE] to make money.
So there are a lot of steps to allow you to fight that.
The problem with trademark is that you have to keep fighting back.
So Xerox, for instance, constantly fights to keep Xerox as a trademark because people keep using the word, Xerox, to mean photocopy, which is a great problem for a company to have.
But it's actually a cash drain as well.
Because everyone's associating your company's name-- you're thinking Google, right?
I think [UNINTELLIGIBLE].
Too many people or lose it.
Band-aid would for sure
So you have to fight it
And Kleenex, yeah
As long as you keep fighting it, you stand a chance of hanging onto it.
Like [UNINTELLIGIBLE] you blink
[INAUDIBLE]
I remember hearing about this, so I was looking to find it.
But Richard Garfield, who created Magic, has a patent on methods of play for trading card games
Like the [UNINTELLIGIBLE], for instance?
And the one who owns the patent is playing a card in a certain orientation and then changing it's orientation in the game field choosing to buy a different state of that character's embodiment
What?
[LAUGHTER]
[INTERPOSING VOICES]
That is stupid
He has a patent on tapping cards?
He has a patent on tapping cards, among many other things that are basically-- So I suspect that he doesn't enforce this patent or that it is not enforceable because if you try to enforce it, it gets [UNINTELLIGIBLE PHRASE]
It is possible, but it's not [UNINTELLIGIBLE]
I heard that the actual manifestation of that is that-- I don't know whether [UNINTELLIGIBLE] contested people over it.
But definitely, they're anti anyone else using the term tap.
So which is why whenever you go to look at Legend of the Five Rings or whatever, they have the turned sideways [UNINTELLIGIBLE].
But it's called bowing.
And the same is true in any collectible card game that has a tap mechanic, they're not allowed to use the word, tap
But that's a copyright
Yeah, it's a copyright
[INTERPOSING VOICES]
So I don't know it's been tested because I don't know if that patent has been contested
I would suspect not
Probably not.
But it's interesting.
If you look at any of the [UNINTELLIGIBLE], so [UNINTELLIGIBLE] Pokemon and I don't recall what the others are.
They all use the term tap.
But because of the copyright in which they hold, they're allowed to do it, but nobody else--
[INTERPOSING VOICES]
The copyright violation probably gives evidence for a patent violation.
So if you can stay clear of the copyright violation, the case that Wizards has to be able to contest the same mechanic that's under [UNINTELLIGIBLE].
But they might still have a case
I suspect that they think that their patent is not enforceable
It's possible
Otherwise, you wouldn't even be able to get away with not even calling it tapping.
As long as you turn a card, you'd be able to say, hey, do you have a method?
It's so ridiculous
Entering [UNINTELLIGIBLE PHRASE] the first orientation at the player's option.
Turning the trick card another orientation on the playing surface.
It doesn't even say for any reason
In Naval Battles, you have to turn the card upside down.
Like submarine cards, you can put them upside down to [UNINTELLIGIBLE]
[INTERPOSING VOICES]
--method of playing games when two or more players blah, blah, blah
[INTERPOSING VOICES]
So you can see how this gets annoying quick.
But there's actually an, unfortunately, good reason why things like game mechanics and more often implementation of game mechanisms code are protected by patents.
Is in that because there are already so many out there, if your company is just trying to make a game, the likelihood that you're going to run into a pile of patents is actually pretty high.
So what do you do?
If you've got something that you know that you created, a lot of companies just go and patent that.
In the event that it turns out to be a case that you run afoul of a different patent, they might actually look at the company that's suing them and say, but you were running to file our patents because we just also patented something pretty obvious.
EA, Activision what?
It's all just mutually assured destruction
Exactly.
I mean, well, it's not destruction.
It's what they do is they trade licenses.
It's like we will let you use these patents if you will let us use that patent and thus, avoid a legal fight.
Final costs.
Lawyers are getting paid throughout this entire process
[LAUGHTER]
It's cheaper than actually bringing it to court because--
Your patent isn't validated if you can be shown someone else came up with it and patented it earlier in time.
You have to be the first person to come up with it
[INTERPOSING VOICES]
And that's why I'm sure that Wizard has never tried to use their patent against people because they suspect that it's not going to hold up [UNINTELLIGIBLE]
It's possible.
You may not want to fight it.
But sometimes, the threat of a lawsuit is a big enough threat.
It's like you don't even have to be able to win it.
You just-- we are a big company.
We have lawyers.
And we can pay them.
Do you?
So that's one reason I mean why companies that are otherwise opposed to legal patents will still secure patents to protect themselves.
It's kind of sad.
It's like one of those things where in a city where everyone owns a gun, you want to own one, too.
So that's patents.
We talked about trademarks.
We talked about copyright.
One last thing I kind of want to talk about is fair use.
Fair use is kind of the defense that a lot of schools fall back on.
The idea that if you're doing something, or if you're appropriating some sort of material, or some sort of IP for political or educational reasons, you have a defense.
Should the case be brought to court, you might be able to defend yourself and win that case by saying well, this was the reason.
So am I doing something for satirical purposes.
Am I doing something to teach people about games, for instance.
I'm going to reprint rules or something.
And you guys can look at it.
And you guys can sue me.
And I can claim fair use.
I may not win.
That's the problem.
Fair use is only a defense.
Fair use is not a law that automatically protects you from being taken to court.
So lawyer's fees.
Who pays the lawyers' fees depends on the case.
But frankly, I would rather not be put in that position.
So there is that chilling effect.
Just keep that in mind of the games you guys are making for the class assignments.
But, of course, we don't expect you to be say, copying text verbatim from other rule sets.
In fact, I doubt any of you would be particularly interested in doing so.
Certainly not copying art.
But also, keep in mind that also applies for things like art that you find online.
You could protect yourself under fair use.
This is for a class assignment, darnit.
And for the most part, no one is really going to notice.
But, of course, if you want to take these games a little bit further, maybe sell them in the future, you'll want to rethink about every single piece of art, every single piece of text that you're using in your games.
Is that coming from somewhere else or can we claim that we invented that?
Have any past students or groups tried to pitch their games anywhere?
Oh, you mean like turn it commercial?
Not to my knowledge
[INTERPOSING VOICES]
Yeah, do it.
I mean, based on the games that we played on Tuesday, absolutely.
Actually, one of the pieces of feedback that we got from the GAMBIT [UNINTELLIGIBLE] every single one of those games has something they have not seen before in a card game.
Every one of those projects had a really interesting mechanic.
And as far as I can tell, they've all really been tested.
All right.
OK.
So I'm not going anymore into IP discussions.
I spend too much time talking to IP lawyers because we create IP in this lab.
We create games.
And if anyone happens to be interested.
I was considering a career in law.
I'll be happy to go into greater detail.
But for now, anymore questions?
OK. Let's just play games.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information, see the course materials on the MIT OpenCourseWare website
OK.
So today we're going to, I guess, kind of [UNINTELLIGIBLE] folks from [UNINTELLIGIBLE] which is actually very closely related.
These are folks that, [UNINTELLIGIBLE] the Comparative Media Studies Research Group, based in the old Media Lab building right now.
And you work with the shared education program.
For folks who're unaware of this, MIT actually has a teacher education program.
So if you want to be a certified teacher to teach--
Middle school math and science.
Secondary math and science
Then you can take classes and basically get your certification of [UNINTELLIGIBLE] certification at the end of those classes.
But they also make games.
Games using portable devices like iPhones and PocketPC games that run on Flash to go and prototype stuff before.
And focusing a lot on games for learning.
So the education [UNINTELLIGIBLE] the research group.
The Learning Games Network is kind of like a spinoff company of the [UNINTELLIGIBLE].
It basically sort of works on projects that will be too expensive to do inside the university.
Because the university is really inefficient.
Anyway.
Maybe you can start by introducing yourself?
I'm the latest in the group, so OK.
I am Constantine.
I come from Austria.
I'm at the moment a visiting researcher at the Education Arcade.
And so I'd say I come from an educational background.
So I did my Ph.D on learning from failure.
And I worked on theories that explain why we use, why we learn through mistakes and why most of them don't want to learn from failure.
But in games we think it's really funny, or entertaining, or challenging.
So I was really interested in how games can be tools to understand learning and also, is it possible-- this is one of my research questions-- is it possible that people may learn, through playing games, how to cope with failure better?
Like, being more open to realizing that sometimes in life we make failure and our expectations are not true, or sometimes maybe wrong.
So I do research on the theoretical level with that, and I do interviews with players on learning biographies and on how learning experiences through games were important in their lives.
And in the summer I was happy to work for Gambit and design a game in the summer production.
Maybe you heard about that anyway, or--
[UNINTELLIGIBLE]
So it's just like 8 weeks.
I was on a team and my theory was used as the basis to develop a learning game called Afterland.
And of course it's free online.
And I think our students did a really good job on this game.
And it's a platformer where-- I will not tell you what it's about because then I would ruin it, but there is a twist.
And what I'm going to do now in the next year is, it looks like I will try for a few months and do research on how the game is used by players.
How they learn in the game, how they reflect on the game and I will work with [UNINTELLIGIBLE] Johnson and hopefully even [UNINTELLIGIBLE] in that.
Thank you
[UNINTELLIGIBLE]
Hi, I'm Dan Reilly.
I'm a CMS alum.
I did a masters in, finished in 07.
Worked with Philip a little bit here and there.
Mostly I worked with the Education Arcade during my time at CMS and with Scott.
And we worked on a project called Lunar the Labyrinth which you may have heard of.
It teaches middle school math and literacy skills, particularly to the at-risk population that I believe [UNINTELLIGIBLE].
It's free and online
[UNINTELLIGIBLE]
Better than after me.
Got to keep it interesting
Is there a twist?
Is there a twist?
Yes, well-- [UNINTELLIGIBLE] twist that you originally wanted.
[INTERPOSING VOICES]
There was a twist in the development process.
So, that one was feted by the Department of Education.
And it was a larger scale project, but the grant didn't go to MIT, it went to Maryland Public Television.
So that was part of the, I believe, impetus, for the spinoff of the Learning Games Network, is to house a structure that can accomodate the sorts of larger scale projects and think about them in more sustainable ways.
I'll let Scott talk about that.
So, how did I get into this?
My interest in learning games is mostly around engagement.
I feel like people who are extremely motivated and enthralled with a particular subject will find a way to learn it.
And that a game may or may not be the best way to learn anything in particular.
But that there are many people out there who struggle to get interested in a topic they're even trying.
And so a game can be useful there.
So that's an exploratory tool to say, well, I never really thought about history, but I like this game and I want to spend some time with it and I want to be good at it, and it's social, so now I'm finding myself asking the kinds of questions that other people are telling me historians ask.
And I don't know what label they want to put on it, but I think it's interesting, and just sort of having that introduction to a topic.
Also I think games are helpful for-- I guess I'd call it identity exploration.
So for people, or particular learners, who may not think of themselves as learners or as confident learners, that they can put that enemy behind them when they enter the game because, it's a game.
And anyone can play it.
Or, maybe I'm particularly good at games, so I can play games.
And then I can suddenly realize that I might be good at other things as well, or I might be interested in other things that I wouldn't allow myself to discover whether or not I was interested, because it was a risky sphere to enter.
So, current projects.
We're doing a language learning game.
It's attached to a massively multiplayer online game, with a grant targeting a population of Spanish-speaking high school students in the US.
And that grant is from the Hewlett Foundation.
And then we have a couple of other projects going as well.
We've got science and some other stuff.
That's about it
So I'm Scott Osterweil.
I'm the research director of the Education Arcade.
I have two titles, research director and creative director of the Education Arcade.
But also one of the co-founders of the Learning Games Network.
And I don't want to repeat too much of what's been said.
Except that at the same time as I do this for a living, I still struggle with the term educational game.
Because it's not the way I think of what I do, primarily.
The first game I designed, before working in this setting-- the first game I did was a commercial game.
It wasn't designed as an educational game, it was designed just as a puzzle adventure game.
It happened to be about math and logic, but that was just because that's what we thought was interesting.
It got marketed because of the changes in the marketplace as edutainment.
But when I started in the business there was no such word as edutainment.
Most games, if you go back to the early 90s, were sort of exploratory environments.
Either they were twitch games or they were exploratory environments.
And I have nothing against twitch games, but the exploratory environments were all about thinking and problem solving.
And to me that's what's interesting.
No matter what domain you're working in, whether it's mathematical or whether it's historical, the processes are the same.
And one slight twist I want to put on what Dan said, I think we're on the same page, but because he led with the idea of engagement, it's possible to think that I'm talking about games as an inducement to do something you don't want to do.
And I think that games are an opportunity to discover how much fun something is.
Which is very different from sort of, ah, we tricked you .
But rather, this is really interesting, don't you like doing this and you can do more.
And if that's all you can get out of it, that's fine.
If you later realize, oh, I like doing that and that's math, and you do more of that, that's awesome.
That's even better.
And I think it's not so much because I just said it's fun, it's because I think the very process of thinking and problem-solving [UNINTELLIGIBLE] People do get that out of games, and I just think it's important.
It's how people realize, if you enjoyed it in a game, you can enjoy it in life.
And maybe this comes from my own realization .
But when I was in sixth grade, instead of them saying the scientific method, and writing this list of dull steps on the blackboard, they said, play this puzzle.
Now let's talk about how you solved the puzzle.
That's science.
I think that I might have ended up working at MIT.
So anyway, that's my spiel
So talking a little bit about identifying how you're going to locate what's enjoyable about the thing that you want the people to engage with.
Can you talk a little bit about, maybe in a recent project or in a past project, how you went about that process.
Let's talk about the things that you worked out.
And maybe sometimes it was more difficult and you had challenges.
Maybe sometimes it was just pretty obvious
I'll take it.
So--
[UNINTELLIGIBLE]
I'd like to congratulate Scott on his personal growth.
So, with the language learning game that we're working on at the moment, there are approaches to language learning out there in the world that emphasize vocabulary memorization or grammar.
Grammar, meaning, memorizing tables of conjugations and it's pretty much stripped of any sort of meaningful context.
So, and a meaningful context could either be a text, and I use that word loosely, that is engaging.
It could be a film that you want to see, that happens to be in a language that you are trying to learn.
Or it could be a means of communication.
After all, that's what language is.
So, if you're thinking about using language as a means of communication, then of course you need to think about, well, what are you communicating?
And, fortunately, games are full of communication.
Between the game and the player, and the player and the game.
And if it's a multiplayer game, between the players in the game.
So it was almost-- if you start from that philosophy, it's almost a trivial exercise to say, OK, well, what's something interesting for people to talk about in a game?
And so you come up with an interesting game.
And instead of having the mechanic be, click this button to activate whatever the nuclear missile, you have to, you build in collaboration.
So, I have part of the piece and you have part of the piece, and we need to talk about when to use what and in which context, and on whom.
And so, suddenly we're talking and it doesn't even feel like we're learning a language, it just feels like we're playing a game.
And then, if you're doing it in a language that you're attempting to learn, then there's the barrier there where you might want to say something that you just don't have the skill yet to say.
So it's not quite as simple as just saying, hey, go talk.
But there's slightly innovative ways that you can give people tools to allow themselves to express themselves in a fairly constrained context.
So, because you've designed the game, you sort of rig the system.
You know that people aren't going to talk about absolutely anything.
Maybe they will, in the chat, but primarily in the game you're incentivizing their discussion around a particular topic.
So if you give them tools to discuss that particular topic, not only does it make the gameplay more efficient, but that's also how the learning can happen.
Because there can be some sort of selection mechanism, whereby I am able to say something that I want to say, but without needing to bring all of the knowledge of how to construct that on my own.
There's some amount of, oh, I can just recognize that that's more or less what I want.
And then, by seeing it repeatedly and by seeing other players use the same tools to communicate, and knowing that the communication in a certain context is valid, that I can pay close attention to that, and start to become familiar and learn.
But it's not, I have to memorize it so that I can use it.
It's, I'm using it and I'm using it so much that I just can actually learn it
Question on that.
So, if you're trying to encourage me to talk Spanish and you talk Spanish and we're trying to work something out, what prevents you from just typing something in English and having you read it in English?
Right, so it depends on the game.
And in some games that's totally fine.
In other games we have teams moderate other teams.
In other games, we have some sort of communication tool that's not free text entry.
So it's not-- so one extreme is, here's a list of 10 sentences that you could say.
Click one of them.
And that's not the direction we tend to go in.
But that's an example of how to communicate something without free text entry.
So there could be some sort of sentence construction kit, let's say.
Where you have a pool of different words that you can assemble and a variety of orders to express a small set of meanings
And if you think about it, games like Charades or Pictionary, the need to communicate with constraints doesn't stop us from necessarily enjoying the game.
It could make it interesting.
So, that's part of the game.
Part of our insight was thinking about our own experiences learning language in terms of the game of say, and this gets at my core idea, which is to find the game that's already there.
So, a game that I always play when I'm traveling with [UNINTELLIGIBLE] when I travel to France, where I speak the language sort of, is how far can I get into a conversation before they figure out I'm American.
So that becomes a game.
And so the whole, and how can I have or if I notice someone who's French, how much of a conversation can we have in French?
That's a game for me.
At that point it becomes kind of fun.
So I think we were just really trying to access that.
And I think that really gets back to your larger question.
When you take on something new, I want to know where's the game already in this?
And I [UNINTELLIGIBLE] the first game I think, it was math, and the colleague with whom I co-designed the game, we both spent a lot of time saying, when we were kids, how did we play with numbers, what kind of things did we do, what was interesting us with numbers?
We really try to access our own play.
And similarly, now, if I can try to read practitioners in the field and say, what do you think about the shower, when you run it-- where's the-- because that's where the play is actually happening in the field.
And I think that's where people love their jobs, it's the play in it that they love about it.
And that's what I think you're trying to access in any simulation.
So Dan, talking about the language one, we're doing a game with the Royal Shakespeare Company, if we can get funding, around The Tempest.
And we're trying to figure out how to make a game of it.
And I realized that part of the change with Shakespeare is, to some degree, the language, too, and sort of making sense out of the language which is now somewhat archaic but really interesting.
And so we tried to make a game that's sort of a puzzle that you assemble by finding the pieces of the play and associative things.
So you find a hedgehog and you find some text that makes a reference, that somehow makes you think of the hedgehog and you put them together and you actually start building a visual puzzle.
So it's really a challenge of making sense of the significance of text.
It [UNINTELLIGIBLE] you and your picture of Shakespeare's play, The Tempest, starts to emerge through the gameplay.
So a story sort of appears that way, the way a picture appears in a jigsaw puzzle.
So it's a game you play without knowing, without ever having seen it.
But part of the experience of theater is the way stories emerge from different patterns in the game.
We're still trying to make a story [UNINTELLIGIBLE PHRASE] through exploration
Maybe if I talk about my career as coming-- I think I come from an area where we make really bad games.
There's a saying that education games are like chocolate-covered broccoli.
It's not healthy and not sweet.
And I actually come from a background where teachers or educators are asked to build games for somebody.
And what you have is contents.
And I can say that, because I think most of my colleagues seem to know better.
They thought, OK, I have content.
That content is good, and now I just make do with a song and the kids are going to love it.
And I think, as you all see, Jeremy, we all know how quick kids realize it is not fun.
And I think there are tons of bad games out there that ruin the reputation of educational games in a sense.
And coming here to MIT and working with Scott and people like Dan, that I realized more, the question is more, what makes us curious about the problem, about the learning content.
Of course, we can have the content of the game, something we don't even have to call it.
It's more abstract.
We look for metaphors that are kind of like, if you can understand how to be interested in history without really learning all the real data maybe you learn in other history, but you get interested in how history works.
And I think that's kind of like what I learned here and I realize now that it's really useful, that we don't use the chocolate covered broccoli any more.
We ask more about, what do we really want?
Do we want to have something that is healthy or sweet, or something different, let's cook it.
Let's not just combine two things.
And I think there's something that also-- the good educational games that are out there are heading in that direction
Do you have counter-examples from your work, of something that seemed fruitful but then kind of maybe you were trying to address some subject area and then were trying to marry some specific game mechanics you thought might work, but were disappointed?
I could talk about one that's not resolved yet, that I'm struggling with.
We're doing a game with a professor here at Ocean, in environmental engineering, about microbes.
He does all his work study on microbes in the ocean.
Which I've learned about.
Half the biomass of the ocean are microbes.
And we know almost nothing about them, because until people like him, he's one of a few, built apparatus to study ocean-like conditions in the laboratory, we had no way of knowing what was going on with them.
So that's interesting if you get into it.
It's not something that middle school kids are dying to know about.
So we're doing a simulation where the premise is that you've got a part of the ocean that's dead and you're trying to re-engineer it by introducing nano-bots, which you can build.
And these nano-bots would in effect simulate the functions of what looked like processing oxygen.
Or rather, plants processing either oxygen and producing CO2 and energy, or the reverse.
Taking CO2 and sunlight and-- Anyway.
So, the challenge, we made a sort of interesting playable simulation where you build the stuff.
But I haven't figured out yet I'm struggling with, have we found a way to entice you to do this, though, without having to give you too much information?
And I'm a big believer that a game-- this game does not start with instructions, because I don't believe games should ever start with instructions.
But I'm still struggling with the information we're providing along the way.
There's too much read this now, as opposed to-- and I think the problem is, because simulations are very wide open environments, and it's very hard to constrain a simulation enough that the player goes, ah, now I understand what variable I just used.
And particularly because kids have a tendency to see simulations as black boxes.
So they'll change three variables at once and not notice they've changed three variables, and not think about what each variable was.
They just don't have the practice of changing-- really, controlling three variables.
And if you don't do that, simulations do become black boxes, and I'm still struggling with how to-- not to, say, mark the kid down the road and say, OK, now make this choice, now make that choice.
Because that's too constrained and not fun.
I want to have them have a lot of choices but also have them understand the effect of the choices that they've made.
Maybe that's the best way to sum it.
So just processing the [UNINTELLIGIBLE PHRASE]
I can give a better example.
So, I was working on this game.
I wasn't even working on that, to my excuse.
I was beginning to work with these people, then realized it's not the way I want to go.
And then I stepped out of it.
And I think it's a good example of what you shouldn't do.
So this small town in Austria is using their waste to produce renewable energy.
It's really interesting.
The town is somewhere in the countryside and nobody was interested.
It makes so much power now that they can give power for free in their town to companies and the neighboring companies in neighboring towns.
So a lot of people are interested in how they do that.
And we thought, well, we want to be a cool town, what if we make a game instead of just providing the content.
But what they did, they gave us all the content and said, this is what you should teach the people.
And make it fun.
And in the end, these people were sitting together, OK, what do people like?
Comics?
And you have to a character that is moving around.
And they would make small multiple choice tests.
And afterwards they get their rewards, and it says your name and it says you're trained in that discipline.
And what was interesting for me, they built the game.
And the game's been bashed.
And it's very sad and it's horrible.
But the funniest thing for me was, who was the target group?
And they never talked about that.
They always thought who cares about target groups, we make this game, everybody's going to love it, everybody's so interested.
The idea was who decides if you can do that in the majors of towns.
They say, well, we want to have a company like that.
And so now we're talking about Austrian, European, majors of towns.
And they are normally fat and like to drink wine.
Normally they sit with another mayor and have lunch and then they say, wow, this works.
And then they decide to do it or not.
But now they were sending these games to majors who did not know at all what to do with this.
They couldn't get into the game.
And so, not that the game was bad.
They didn't even think about their target group.
And I think that one aspect that interested me the most is, you have to have the context of the game as a learning experience.
It has to be suitable.
If this is not happening, you can play the most interesting game.
If someone doesn't get the problem, doesn't get engaged to the problem, it's just something happening.
It's not really interesting.
And I think that's a major problem that this example was totally
So, know who you're designing your game for.
And how do you think they get engaged in the games
Yes
Just to do a simplistic example, but it's one [UNINTELLIGIBLE] example.
If I wanted a game for my mother, who occasionally would play a casual game.
I wouldn't do a massively multiplayer game with lots of stuff going around.
All these senior citizens are going to get into this massively multiplayer game.
I mean, it sounds like it would be really great, to come up with a massively multiplayer game for senior citizens, because of their eyesight.
But you also know that they didn't grow up with computers, and they're not going to be able to navigate a 3D space.
So, that's a simplified example of why it's important [INAUDIBLE]
And what is really good about it is that game like [?
Revish ?]
the cancer game, maybe some of you know about.
It's a cancer game [UNINTELLIGIBLE]
It was actually mentioned in the reading [UNINTELLIGIBLE]
But it's hopefully one of the best [UNINTELLIGIBLE] learning games out there.
But the good thing about them is, people that play that game have cancer.
So you have the problem to the game is so close that people who do not play the game say oh, I don't care about cancer.
They care about cancer because they have cancer.
The game is about cancer, and the way chemotherapy works.
So the context of the engagement is automatically, the problem is solved.
But what about this cancer.
What about a learning game where it's about language learning?
Are these people already interested to learn the language?
Do they come there with their language problem, or do we want to introduce them to the problem or show them maybe you don't have to be afraid to learn a language or whatever.
This question has to be addressed at a very early stage.
Otherwise, if you start designing and then address the question, it's late.
The question is sort of about the explicit versus the [UNINTELLIGIBLE].
So if you play [UNINTELLIGIBLE], that was really designed to introduce, basically, an idea that people have struggled with, which are what position and velocity graphs actually mean.
But the game doesn't mention velocity and it doesn't mention graphs.
And in fact in the same reviews that it got lots of good reviews when it came out, a lot of them said, I had no idea this was educational.
I wouldn't argue that that's fine, that people don't necessarily know that as they're playing the game, that they are still building up some conceptual knowledge.
I think that some large percentage of the people who play a game in that setting will probably play the game and do nothing else but then three months later they'll have gone.
That experience for most people will not be resilient.
Not because they didn't enjoy it, but because they have lots of things going on in life and there's more information coming out of [UNINTELLIGIBLE].
On the other hand, the kid plays that game and then you sit down and have a conversation about velocity and position graphs, there might be some real meaningful learning that occurs.
And so I think it's really important to not overreach in that sense.
I try to avoid, although we still do the saying that our games teach.
Because I think teaching is a much more active and explicit activity to learn games too.
I think our games let people play with them, let people learn from them.
And then just like if you read a book, the book doesn't teach anything.
The book puts information [UNINTELLIGIBLE] And I think that's really part of, I think, we have to be modest and humble about it as we do this stuff.
But that was [UNINTELLIGIBLE] as the challenge [UNINTELLIGIBLE] of thinking about, if you do want some learning to occur, where do you want the learning to occur.
Is this for peoples' personal lives, is it a mediated setting like a hospital?
Or is it in a school-- those are really critical questions .
It's not just the learning but the setting
And who's introducing a game to them, for instance.
Is it a teacher, a doctor
And were they playing.
Like with Labyrinth, which is this middle school math game that Dan and I were working on.
We knew that if we didn't push in another direction, the implementation was going to be, well print this game and it gets played in classrooms.
If you've ever seen-- if you remember your own computer labs in school, that that was [UNINTELLIGIBLE], which is true actually for a lot of [UNINTELLIGIBLE].
It meant every now and then you had to line up and walk down the hall to this other place, that you would only get to occasionally.
And by the time you settled into your chairs and booted it up, you had maybe 20 minutes on the computer and then it was time to go.
So if you design something for schools, you're talking about a 20 minute experience once every two or three weeks.
I'm not sure that was worth thinking about anything for that.
So we design the game, we play it online at home.
And encourage teachers to let the kids just play it.
And there was another advantage of that, which was to go back to who introduces it.
If you play that-- even if we have the perfect 20 minute game for the computer lab, if the teacher's walking around-- I shouldn't say.
I've seen good uses of that space.
But there's some greater probability you're going to feel like this is just a lesson.
Whereas if the teacher says, go play this at home, there's some greater probability that you will sort of forget about the school and get [UNINTELLIGIBLE].
And again, it's not that we want to trick you into doing math.
It's that we want to engage with the math in the most pleasant spirit possible.
And the less we give the wrapper of school to it, the more playful you're likely to be.
Even if you're not denying that it's math or science or [UNINTELLIGIBLE].
So that was [UNINTELLIGIBLE PHRASE]
So Constantine, you had said the cancer game is really well evaluated.
So did any of you have stories or experiences from seeing your games out in the world that way.
You've gotten feedback from teachers who have tried this in the classrooms
Yeah
[UNINTELLIGIBLE] so from the game I just talked about, about renewable energies that totally failed.
People played it but didn't understand it, hated it, or [UNINTELLIGIBLE].
So I think, total failure.
And I think just to start with this conversation is, this is a problematic thing.
Because there is very little research going on on the iteration of games, because it's also very hard.
It takes lot of effort and it also could take a lot of money to evaluate the research, that the success of the game for learning purposes.
And I just know very few studies that do that, in the end.
And I think we've just reached that moment where people start to think that this is an interesting way to do it.
And Remission, as I said, is probably-- maybe you'll correct me in a second, but I think it's the only game I would put my hand into the fire for it.
But the question is, what did they learn in the game?
They learned about [UNINTELLIGIBLE].
They learned that there is something they can do.
They learned a more positive perspective, a more positive relation to medicine.
They didn't learn anything else.
In the research--
It doesn't really change their illness
And they didn't get happier.
It's not that they said, wow, now, I couldn't cope better with their illness.
They just took the medicine and so the doctors were really happy with that.
But--
That's the stated goal of Remission, actually.
It's, this is a 3D game where you play a woman in space or something.
Shot down and [UNINTELLIGIBLE] but the stated goal of the game, at least of the people who were making it, was to try to encourage people who are suffering from cancer to keep up with their medication.
Don't fail keeping up the schedule of taking your chemotherapy drugs, basically.
And the game was designed to show what happens, what these drugs actually try to do in some sort of fictional-- or fantastical, I'm not going to say fictional -- so that's Remission
Is this a free play game as well?
Yeah.
Yeah.
[UNINTELLIGIBLE]
So, on Labyrinth the original grant include an excess of one year evaluation on the game use.
And then when Congress cut back the funding of the larger program, of which Labyrinth was just one small piece, that got cut off.
And so the research didn't happen.
We're now looking for a grant, because Labyrinth has been used aggressively in Maryland schools [UNINTELLIGIBLE].
So we are going to look for funding.
But generally, there's less money.
As little money as there is you're developing this [UNINTELLIGIBLE] there's even less for researching them.
And I think some of what I think our work is designed [UNINTELLIGIBLE], we figure out what we can learn by making the game.
And we publish what we've learned in the hope that other people will pick up the ball, people who have more of a research bent in the traditional sense.
Will start to do the research about it.
I think we're on the brink of that.
I think with stuff like the Quest to Learn School.
Increasing numbers of educators are using [UNINTELLIGIBLE].
I do have only anecdotal evidence, but the very first game that I did, which was this massive logic oriented game, ended up being not particularly well marketed.
It nevertheless sold a million copies.
And the anecdotal evidence I got was that it was one of the rare cases where parents would bring it in and recommend it to teachers, and teachers would recommend it to parents.
Or kids would come home from school recommending it.
And there seemed to be this fairly broad acceptance by both parents and teachers and kids, without kids having the sense that, oh, my teacher's making me do this.
And I hear that now that everybody's rich and famous in their 20s, I occasionally hear from them.
And some of them say it's what interested them in math and stuff.
So that's all anecdotal, and maybe [UNINTELLIGIBLE] has some [UNINTELLIGIBLE].
But I think the million copies is beating towards the--
[UNINTELLIGIBLE PHRASE]
Sure.
I'll just add a little bit to that.
I also have some anecdotal stuff I want to bring in.
But the most interesting thing that I've discovered was with Labyrinth and a target audience that I've seen play it, that it was not specifically designed for.
Which is kids with some sort of attention disorder.
Whether it's ADD, or autism or something.
And seeing these kids engage with the game, even engage with the game in a social way, and be hyperfocused on the game.
And be highly competent with the game, and to really calm down and adopt pro-social behaviors that were not characteristic of them.
Or that their peers or families hadn't seen with them before.
I think there's a lot of power there
The first two teachers I've met who have used Labyrinth said they were seeing performance with kids they didn't expect to see before.
Because our sense was that lots of kids have competencies and skills that they don't show off in the format of school.
School being its own kind of game which only certain people are good at.
And so having a kid be able to demonstrate a competency at a different kind of game, and let the teacher see it.
Because part of the design of Labyrinth is that teachers can track progress online.
That was one of our goals.
And that was [UNINTELLIGIBLE]
This is a very good laptop actually.
So is it possible to look it up and show some of Labyrinth?
Because we've been talking about it in this aspect
Yeah, sure.
Absolutely.
[INTERPOSING VOICES]
It's just a notion of a game
But other people can [UNINTELLIGIBLE].
[INTERPOSING VOICES]
While we're setting that up, I'd like to ask a bit about prototyping.
[UNINTELLIGIBLE] So, in the process of designing-- these are mostly digital games that we've been talking about all the time.
But I'd like to talk about the early stage, when you're working with paper or simple digital prototypes.
Maybe you can talk a little bit about how that might be similar or different from designing other kinds of computer games, or even board games?
Do you test it differently because you're designing it explicitly for a specific audience?
Do you test it with instructors as well as players, or [UNINTELLIGIBLE]?
I think it's always best to try to test with your target audience.
The things you'd learn from [UNINTELLIGIBLE] there's thing you learn, you can learn from any audience.
Particularly usability, which is important.
Frequently the flaw in a game is not the core mechanic, but the ease with which the player can understand what they're supposed to do.
But you can test that with any wide range of people.
Anyone who you think is roughly as computer game literate as your target audience.
But I think if your goal is to interest middle-school kids like we do with this game-- let me just turn this on-- then you do want to, because there are questions about age appropriateness.
Or [UNINTELLIGIBLE PHRASE] All right.
So we're about to get this screen here.
So this is a login screen.
This is not the first screen you would come to if you'd started the game from scratch, you get some more information.
But I've already logged in.
So what you'll notice-- let me just log in.
So it's online.
You can play from any computer.
That was important because we were concerned about underserved populations who either don't have computers at home or who have less control over their time, don't have a computer in their room, certainly.
Or might do all their work in an after-school setting or a library.
So basically, once you have an identity you can log in from anywhere.
I'm going to briefly enter the game space but [UNINTELLIGIBLE].
So here we're already in the middle of the game.
This is a puzzle adventure game.
And the plot is, someone's stolen my pet.
I've tracked it down to this underground factory that's being run by monsters-- and there's middle schoolers-- with the help of a mysterious woman with bluish skin and wings.
I've been given a monster costume, so in the guise of a monster myself I can go around and do jobs in the factory.
Find my pet, figure out what the monsters are up to, which is indeed no good.
And I'm in the middle of this game, so I've already advanced some.
There is a sequence of rooms, which, part of the game is just finding the rooms where the puzzles are.
For example, we've got it here.
This is one of the wings of the factory.
There's a hundred doors here.
And I can click on every door if I have to.
But I have a map, which is itself a mathematical tool, to find where I want to go.
And as I go through, each room represents a different puzzle.
Anyway, having said that, we're going to actually go back-- And oh.
The other big feature in the game is that if you play on teams, then you can communicate with your teammates through this communicator.
And the reason there is to actually get kids reading and writing about what they're doing with math.
Because if you want to help your teammmates, you can't feed each other the answers [UNINTELLIGIBLE] puzzle.
Every puzzle, every time you play, it's a different solution.
So you can't tell each other the answer, all you can do is talk about your strategies.
We're going to leave the game and go to puzzle mode [UNINTELLIGIBLE]
This is a shortcut?
Yeah, shortcut to puzzles, right.
You don't have to play it in the context of the game.
You can play any puzzle that you want.
I'm going to try one that I've never played with a group before.
So usually I pick the same game, play it, and get used to playing it with people.
But let's just try this one.
[UNINTELLIGIBLE].
I think I've tried level one.
So I'm trying to remember what you know here.
You're in the back in the shipping area.
And you've got to come up with a manifest to ship the number of objects that you've got.
That's all you know.
You've basically got to give them the right instructions to ship it.
And so now, just tell me what to do here
Click something
Of course.
And that's [UNINTELLIGIBLE]
[UNINTELLIGIBLE]
[UNINTELLIGIBLE] Up into the slots up there.
There's ones and zeroes
Can we put that crazy fish head second to the right on every one in the equals spot?
Yes.
You can.
[INTERPOSING VOICES]
Want to try it in the equals spot
And go to the equals on this one
[UNINTELLIGIBLE] Just tell us what your thinking was
Well, it looks like we're trying to build math equations.
And usually they end with equals something
Right.
And you saw that.
Now, just so you know, the equation we're going to build-- you're right, and just as a shortcut-- is not about arithmetic.
The equations are simple.
I'm not interested in drilling kids on arithmetic.
This is really about number systems, and number systems are just simple systems.
But there's also some algebra that's going to go on this, we start to do this too.
And then in higher levels of the puzzle, [UNINTELLIGIBLE] you start doing stuff in base two and base six.
We sort of dovetailed with the notion that our representations of numbers are just symbolic.
So zero, or one, or one zero, doesn't have to always mean 10.
It's just a symbol.
Anyway.
So now what should we do?
Pac-Man should be minus.
Because the only ones with six digits have Pac-Man as the third digit
Sounds good
I'm not sure which one is Pac-Man
Bottom right
This guy?
That guy's Pac-Man.
Pac-Man with an underline
I'm not sure if we can actually put that one there.
[INTERPOSING VOICES]
If you look at the top left, then what is the number two digit number that, minus the last digit, gives a single-digit number.
[UNINTELLIGIBLE]
[UNINTELLIGIBLE]
Oh, that's a good point, yeah.
It has to be divided by [UNINTELLIGIBLE]
But now what you're really doing is thinking about the structure of math.
Which is exactly what--
So when you say that's divided--
So, I'm going to guess the banana is four?
[UNINTELLIGIBLE]
You want to say why?
Because using that same logic, that first version, that something, n x divided by x equals some single digit number.
And I'm guessing x is 4
[UNINTELLIGIBLE]
By the way, you guys have avoided, and not noticed a piece of information which would make your work even more--
Oh, the thing at the top
What's the thing at the top?
The thing at the bottom
It could be two, actually, as well
What are all the green things?
Those are the things you're shipping.
Those are jars of stuff, of pepper
Oh, those make sense
Banana something banana
Banana something--
Equals something banana.
So-- [INTERPOSING VOICES]
Banana could actually be two.
Because 12 divided by two is equal to six.
six.
[INTERPOSING VOICES]
Oh, and in that respect, that means a little--
Yeah, the four squares are signs.
You try that?
[INTERPOSING VOICES]
It has to be zero on--
Every time you play the game the symbols are different.
They're the same symbols but they're a different--
Triple backlash should be-- oh.
[INTERPOSING VOICES]
By the way, you're still missing a piece of information.
But that's OK. That's what games are all about.
[INTERPOSING VOICES]
So what's this thing in the top left?
Can we put things in there and press enter, and will it evaluate them for us?
Yeah, but right now-- you're not done yet .
[UNINTELLIGIBLE]
Yeah, triple dash is definitely one
Triple dash is--
Oh, triple dash is one, because one times banana is banana
Unless banana's zero.
[INTERPOSING VOICES]
Banana's six, I think
It could be five
I've never done this game before-- no.
The fact that you're naming all these symbols, because they were just symbols.
[INTERPOSING VOICES]
I think banana's [UNINTELLIGIBLE], because banana-- [INTERPOSING VOICES]
[UNINTELLIGIBLE] banana squared is equal to something times banana
Not something times--
So, yeah, it could be 25 actually.
[INTERPOSING VOICES]
Remember, I didn't know the solution to this when it came in, but I actually have enough data to know that you're wrong.
It's on the screen
Banana definitely is-- [INTERPOSING VOICES]
Five because banana squared is anchor banana.
But flower banana divided by banana is also an integer.
So if flower banana and anchor banana are local banana, then banana should be five and not six.
because flower minus anchor times banana
I love your mental calculation.
[INTERPOSING VOICES]
So you're saying it's what?
Five
Well, that could work, but you could have done it in a simpler way.
Or you could have surmised it, not known it, but-- remember, this is only the first time you played, subsequent times you'd have other information.
[INTERPOSING VOICES]
You're a member of the group, so don't give away [UNINTELLIGIBLE] but in fact you're making-- [UNINTELLIGIBLE]
Anchor is 20
Anchor is two
Sorry, two.
Not 20
Anchor's two.
Claw is zero
Where's claw?
Claw's the one on the top left.
Of the squares that are left.
Left
Oh, you mean brass circles.
That makes sense
Oh, that one
That's zero.
Has to be zero
Wait, why is it zero?
Because if you look at the equation at the bottom--
Oh, I'm looking at the wrong spot.
[INTERPOSING VOICES]
Yeah, sigma's nine
[UNINTELLIGIBLE]
OK, so [UNINTELLIGIBLE] scythe is plus.
Scythe is plus
No, it has to be minus.
[INTERPOSING VOICES]
Anyone who is having trouble, and this happens in a group.
So, anyone having trouble following it as people were making stuff up or finding stuff.
Because if you did it's OK.
In fact, sometimes you've got to play the game yourself for it to make sense.
[UNINTELLIGIBLE] I could have told you-- I actually think I could have told you a lot about the game going into it.
And there would still be a high probability that when you started playing it you would have not absorbed anything I'd said.
That's my experience.
I could almost give it all away but you still don't get it until they play it
So they didn't know--
[UNINTELLIGIBLE]
Beta delta upside down is eight.
[INTERPOSING VOICES]
Can we enter that now?
Oh, so now you have to type in one times whatever-- equals--
You've got to type in what this is.
So and this is actually getting at another representation
Eight plus nine
Eight plus nine is 17
How, here you don't have to actually remember the [UNINTELLIGIBLE]
Just divide it by six
[UNINTELLIGIBLE]
It's puzzle mode
And by the way, it could take it in either order
Yeah, I agree.
Equals [UNINTELLIGIBLE]
I think this level's only addition [UNINTELLIGIBLE].
[INTERPOSING VOICES]
Or we could do 10 minus two
Anyway.
[UNINTELLIGIBLE] you were doing algebra.
When you were solving for those numbers, you were doing algebra.
You were really thinking about x and the structure of the math and the numerical quality of it.
This is simpler, but it's still getting at the fact that there are multiple ways of representing numbers [UNINTELLIGIBLE]
Is there any mechanic to prevent brute force?
Yes, those lights go out.
But, you do earn points along the way in the game if you make some progress.
And so, none of these puzzles are brick walls.
Because you can always play multiple puzzles at once.
You can leave this puzzle, go to another puzzle.
And, if you make any progress at all, you're accumulating points for the end of the game.
You could actually get to the end of the game story without having solved any puzzles.
Takes a long time.
The rewards are there for doing well.
And there are rewards for doing well, but there isn't punishment for not doing well.
At least in sort of a-- we don't say you can't, we don't make you stop in your tracks
[UNINTELLIGIBLE] you can do it again
Yes, that's right.
You can do it again instantaneously.
And there is no cost in the game to having failed repeatedly.
If you eventually solve it, you end up with the same score whether you solved it on the 20th time or the first time.
That's really what problem solving is about.
It's not about being the fastest.
It's just about learning how to solve things.
Speed is overrated [UNINTELLIGIBLE].
And in math it's very overrated.
Kids [UNINTELLIGIBLE] calculations in their head.
It's just more of this.
And the entire level's-- we came up with representations of subtraction, which is that you have a large number, you have 10 jars and five of them are empty.
So it's 10 minus five.
[INTERPOSING VOICES]
How is division?
Are they 10 rows of jars?
And the rows are empty?
Yeah, I think it's the same-- I think that's it.
I think that's the same.
It's in a grid pattern with some [UNINTELLIGIBLE]
And I guess there's [UNINTELLIGIBLE] but do you have any algebra as such in this format?
No.
But what we encourage is that-- so what comes with this is material for teachers to relate this to the curriculum.
And what we really want to happen is that kids have played the game before they get the subject.
And when the teacher introduces symbol systems in base six or whatever relates to this, instead of saying, all right, we're starting a new unit, you again know nothing, you're starting again at square one and have to start all over again with something new.
Instead, the teacher comes in and says, I want to introduce a new concept, but it's one that you've really already begun to master.
And it's called bases.
[UNINTELLIGIBLE] so how did you solve this?
What was going on in this game?
And they have a conversation about what they already know, as a way of leveraging into-- I mean, things that I arguably don't know.
Again, I would argue that [UNINTELLIGIBLE] you have things in cartons.
So a carton of six jars plus some left over would be-- a carton of six jars plus three left over would be one, three, basically.
So anyway.
The point is that the kids begin to build this robust conceptual understanding.
And hopefully there's a playfulness so that when the teacher talks about it, that some of them may actually feel good about it.
Instead of, here's one more reading --you'll never be a mathematician.
I think math education is largely designed to gate everybody so that at the end of high school, [UNINTELLIGIBLE] population is good at math.
Which means that we're taking 12 years teaching 93% of the population that's no good at math and they should stop doing that.
I don't know why we'd waste 12 years if you could just tell them to go to kindergarten.
But that's what what we end up doing.
Anyway, I think that's true of every subject.
I'd like to turn it around.
Everyone says there's things about math I can do.
[UNINTELLIGIBLE]
Can you tell us a little bit more about how that process went?
Sure.
And I also wanted to provide a little extra nuance to one of the questions about what if you just brute-force your way through it.
In the higher levels, it won't tell you when you put a symbol in a slot, whether it's correct or not.
So--
Which actually, you could learn something interesting.
Because you could start doing something and [UNINTELLIGIBLE] something that doesn't add up, that's actually new.
You're learning something.
So, there's 10 puzzles or-- how many?
Nine
Nine puzzles, and we came up with nine plus x, and some of that was slated for school admission and we never got there.
And some of that x was puzzles or activities that we decided were not as compelling as the others.
In our process for that, we did a hybrid approach with paper and computer prototyping.
In general the rule is, if there can be paper prototypes, paper prototype it.
Because it's so much faster.
But there are certain kinds of activities that really can't.
So for instance, this one is perfect for paper prototyping.
You just take out a piece of paper and a pen and you create-- I mean, you have to create some of these equations so that you then have a system of equations to solve.
But once you've created some of those is, as would need to be done whether it's a paper or a computer prototype, then it's trivial to put it in front of someone with paper.
So this is an example of something where we would, for the first round, test internally with just symbols and numbers.
And then probably we would do-- we might even test that version with people in our target audience.
But probably we would try to do some rudimentary art treatment before we put it in front of people, just so there's a little bit of context and it looks a little bit more like a game.
And then--
So that we try to-- if there's story, this one has less story about it, but if there's story, as there are in some puzzles, we actually try to act it out.
[UNINTELLIGIBLE] resistance to doing it.
So act out the character.
You're the computer, when [UNINTELLIGIBLE] and so even if it's just a character giving a response to something that the player does.
You try to do it in character, you try to get a sense of what-- because I think, even in that kind of rudimentary form you're engaging the player's imagination so you know whether or not they're interested in this character or the idea [UNINTELLIGIBLE]
Because I do master [UNINTELLIGIBLE]
And like a dungeon master you need to prepare, because it's hard to be the computer without also being the bottleneck if you're a human.
And in fact we've done some games, not represented here, we've done some games that are real time, that if you want to prototype with people, then especially on paper, and you want to keep things moving, then you need to have more than one person be the computer, because there's just too much to track for one person to do it all.
Or if you want to do-- sometimes we do hybrids, where we'll do a computer prototype but there's aspects of the game that are not represented through the interface or that logically we have a model, so we'll do players directing human interactions with the computer.
Not players, designers, influencing those interactions.
So anyway, this is a game that moved pretty much all the way to final design on paper, and then went straight to implementation.
Some of the other games-- for instance, there's a game-- do you want to show [UNINTELLIGIBLE]
[UNINTELLIGIBLE], yeah
That's an example of a game that's a little bit harder to think about playing just on paper
[UNINTELLIGIBLE] we implemented this early
Yeah, we did
So this is a game about modular arithmetic and common multiples and that sort of thing.
So it's very hard to see on the screen.
In the upper left there is an avatar
That's you
[UNINTELLIGIBLE]
And you know from the story that your goal is to get to the golden eggs and avoid the [UNINTELLIGIBLE].
That much you know.
And that you're in a factory where they make stuff
Do you move your avatar?
[UNINTELLIGIBLE] [INTERPOSING VOICES]
There is one flaw, by the way, in the [UNINTELLIGIBLE] interface, because people don't realize they have to click to get their guy onto that thing.
And it was supposed to start with the [UNINTELLIGIBLE] right where he was, that people would [UNINTELLIGIBLE] click.
Because now by-- what should I try now
Four
We're going to go--
Four or nine
Six
Four.
Four
Six.
Then you can let him get the [UNINTELLIGIBLE]
I'll go for six.
Bam.
Four is a subject that--
Oh, can we move [UNINTELLIGIBLE]
But you'll notice that that's all in different amounts.
And so it's really that modular arithmetic.
So 12 will take me once around on one of them, but [UNINTELLIGIBLE] on another one.
And you start doing simultaneous equations to solve this.
This level's hard.
I think maybe the hardest puzzle we designed in the whole game is level three of this, where every step you've got to align two bridges.
It's serious work
Are you trying to avoid that monster?
Yeah, you've got to avoid the monster
Which isn't always hard.
But you have to keep in mind, sometimes it needs, instead of going around once, particularly one on the nine thing, maybe you go 10 and he'll always go on [UNINTELLIGIBLE] shows up, to make [UNINTELLIGIBLE] to show up and [UNINTELLIGIBLE].
So this is really the [UNINTELLIGIBLE]
As of what?
Yes
Anyway, again [UNINTELLIGIBLE] you can say, hey, [UNINTELLIGIBLE] math, school math at all.
There's no school math in this game.
But I think there are really interesting things to think about
[UNINTELLIGIBLE] said with that character.
If he walks or not.
I think we [UNINTELLIGIBLE] anyway.
But you have to keep in mind, there are two forms of learning.
One is learning the game, how to play the game.
And the other one is learning-- there's a problem solving, you've got a puzzle in the game.
And I think it's sometimes like special education games.
It's so easy to mix this.
Because sometimes you make the coolest educational game and it's really interesting.
But [UNINTELLIGIBLE] character wouldn't move if you don't press the mouse button.
So he would always stand there in the beginning and he would get attacked.
And I think the mixture of both forms of learning is quite interesting to keep in mind when you're prototyping
Yeah, there's two answers to that.
One is, I never think.
just because we want people to think hard, I don't think the interface should be the thing that they're thinking hard about.
It's always tempting to say, oh, that's a really hard thing to solve.
How do I make him jump?
But no, I think generally it should not be hard to figure out how to make him jump.
Or walk left or right, or how to open a door.
Unless that's a puzzle, and it's easy to delude yourself that by making the interface hard... On the other hand, there is this other challenge, which is that kids do have different degrees of literacy around games.
And it's hard to figure out where the sweet spot is.
That's one I don't have an easy answer for.
Because some large percentage of the audience naturally knows, try and get to the end.
Knows what winning means.
But there's some kids who don't.
And the question is, if you make the game, if you bend over backwards-- I watched my mother play this, and it was horrible.
Because she just was totally clueless about where to go.
But I think if I try to set up the game to work for my mother, you can only work with all this explicit instruction.
So hopefully with kids it's less of an issue.
There's more, I think, uniform game literacy.
But it's always a challenge.
No game appeals to everybody.
And you do want-- I don't think those of us who make learning games have a good answer to that.
I don't think we can ever make a game that every kid would play with equal-- we should never claim that we're going to get 100% on that
One thing that I know that you've noticed is the amount of tolerance that kids and adults have for frustration, when-- I've seen kids say, I hate this game.
This game is too hard and they'll just keep going at it.
Whereas the adults have this tendency to get really embarrassed when they can't solve it first time
It happens when you're 26.
You suddenly sit down and say, well, I'm not very good at this.
You may be a gamer all your life, but it's having another adult watching you.
There's some point or another where you suddenly say, I'm not good at this.
I don't know why it happens.
[UNINTELLIGIBLE]
[UNINTELLIGIBLE]
Can you do negative numbers?
Not in this one, no.
That's a good idea, though
[UNINTELLIGIBLE]
In much of the arithmetic you could always do it as put in a larger number and [UNINTELLIGIBLE]
That's right
I have a question for [UNINTELLIGIBLE].
Because in some of them we were hearing from you guys about coming, more or less from a game design angle into designing games as a learning purpose.
And you kind of [UNINTELLIGIBLE] the other way around.
You came in from the [UNINTELLIGIBLE].
Are there things to keep in mind when you're working across disciplines, that game designers know?
So, I think one more difference that I still have, or see between the way I approach games and the way educational games approach games design, I'm more focused on learning as an experience, more in abstract form.
That doesn't mean that I wouldn't be invested in that, but that's not what I've been working on yet here.
So for instance, after that it was-- so the research question was how, how can players overcome expectations that are wrong.
And so the idea was not to use educational stuff like numbers or whatever, but more, can we develop patterns in games that players who realize very quickly, and are they ready to give them up if they realize that they think what they've seen is wrong.
So, for me it was very interesting because I come at an abstract theory inside.
And so we were developing prototypes.
And it's very interesting when you come from the abstract idea to prototypes.
But paper prototyping is perfect because it really made me as a scientist come to the ground.
Because of course I can read [UNINTELLIGIBLE] and all these people and say, wow, experiences this and that.
And then you suddenly have pen and paper and sit there and think about a game that, is it a metaphor for your theory.
And so I think my takeaway was that paper prototyping was very healthy for me.
And with the team to come down to the fact, what is really [UNINTELLIGIBLE] in the game.
What can we really-- I think at the beginning I was a bit inexperienced in prototyping.
Wow, we can all do that later.
We can do the cool stuff when we get to programming.
But then I realized, no, we can't.
We cannot do it as a paper prototype, how should we do it in programming.
But what is changing?
And I think for us it was very interesting.
So I think one experience we had is, we had them develop different prototypes.
And so I think that was healthy for us not to make one paper prototype but then think how we can do it better.
But that's what [UNINTELLIGIBLE] a lot of prototypes.
And to stay open.
And even if you spend three or four days on a prototype, if it doesn't work with others the moment you show them.
And they go like, what, these are the rules.
Are you sure?
I don't get it.
If you explain the tenth time, maybe-- maybe it was a cool idea back home or with your friend, maybe, but it just doesn't work.
And so this experience in the prototypes was very healthy for us.
And then coming to the point.
And I realized for me it was important that somebody coming in the room, playing a prototype for five minutes, kind of understands the core mechanic.
If this is not happening-- who takes the five minutes to understand a core mechanic?
Nobody, today
That would be, like in this game, just walking onto a [INAUDIBLE]
Right.
That's right.
And so, yeah.
I think, coming from a different perspective, again, prototyping is kind of like the way to get in the game.
Even as the designer.
Because from then on, you know what you're talking about.
And so I had to lose a lot of my abstract thoughts in the process.
And then the playtesting.
So it was interesting sometimes to see, even if the prototype is cool, then you have a prototype.
And then people come in and say, [UNINTELLIGIBLE] again.
[UNINTELLIGIBLE] it's so easy.
But then you can start to stick with your idea or say no, maybe I have to go down with the theoretical aspect and really look [UNINTELLIGIBLE] for the players
And another thing about prototyping, it's not focus groups.
So it's not a case of, --sure you want to put it in front of as many people as possible.
But you really want to, I think, avoid stuff like what did you like, what didn't you like.
Those kind of questions.
Because people want to be helpful.
And sometimes the piece they didn't like was actually the part where they were working hardest.
So it's like I didn't like that I had to jump over the wall.
[INAUDIBLE] jump over the wall.
And it may not really mean that if you took that out, that they would enjoy the game more.
In fact, they might not enjoy it at all.
So you have to be really careful about that.
And people want to please you.
So they'll say oh, it's good, and stuff like that.
I think it's really important that what you do with prototyping is watch the player.
And then trust yourself as a designer.
Rather than ask them what you should do, because they will try and be helpful and they'll come up with wrong suggestions.
Watch them and use your best judgment as designer as to what they're reacting to, and why they're reacting the way they are.
And try to fix it.
And then you try to get some questions, try to maybe get some access to their thinking.
But don't think that they necessarily know what was interesting to them, or what would make them happy.
Don't ask those questions.
Those are focus group questions, and you shouldn't ask them because it's not going to give you the answers you want.
You've got to trust yourself at some level as to what's happening
And something that I also had to learn was, if you have the focus test or the playtest, and then you realize you have problems.
And it's really healthy to write all the problems.
And then brainstorm different solutions.
Don't ever, when you discuss it, try to come up with a solution immediately.
Because what happens is, you quickly develop one solution.
And then you go and end up with another problem, but you don't realize that sometimes, some problems stick together.
Or sometimes you realize, well, we have so many problems, just because of this little aspect of the game.
Why don't we just let it [UNINTELLIGIBLE].
If you don't do that, you start to spend two hours in solving a problem then realize it was wrong.
Or you never realize it again.
So have [UNINTELLIGIBLE] problems and quick solutions where you [UNINTELLIGIBLE] or not.
Make solution, solution, solution and then with the same problem a few weeks later again
And I just want to clarify, focus tests and focus groups are two very different things.
The kind of resource we're talking about is focus testing, which is specifically identifying individual problems in the game.
And a focus group is asking them what they liked, which is actually very not useful for most product design in fact.
Games in particular
One piece of advice that [UNINTELLIGIBLE] really important if you're a smart college student, like an MIT student, is that you've probably learned in spite of your education, by having someone sit in front of a class and tell you what was going on.
And you somehow absorbed it.
And that's why you've got here.
But lots of people don't learn that.
And it's really hard to avoid the tendency in the game to just tell people what they should know.
Which is, sort of for my money, like when you meet someone who speaks a foreign language.
You don't understand each other so you both start speaking slower and louder.
And I think that's what a lot of education-- it's all a bad education in general does.
Speaks slower and louder.
And [UNINTELLIGIBLE] educational games.
So just remember that just because you got it the first time someone told you doesn't mean that everybody else-- it's not just about explaining it better.
It's about giving them a different way of learning
Any questions for our guests?
I just want to say something.
Have you ever heard of firehose games [UNINTELLIGIBLE] forum.
So I told them, I know a guy who works there.
And he said, we've got a focus group tomorrow night.
They're PS3 set to release in a few months.
Does anyone want to go tomorrow night?
Seven o'clock, about an hour to an hour and a half
That includes [UNINTELLIGIBLE] and [UNINTELLIGIBLE] are part of Firehose Games
Are they looking for people who haven't played the game before?
Maybe.
We're going to have [UNINTELLIGIBLE]
Can I ask a question, [UNINTELLIGIBLE]?
So the question I would have to use is, [UNINTELLIGIBLE] in his play biography, educational games?
Have you played it, would you say it was an important experience to you?
That even, maybe it's embarrassing now when the other ones are listening.
But does anybody out there [UNINTELLIGIBLE] Zambonis or whatever that game was?
Best game of all time.
[INTERPOSING VOICES]
It was similar in that you had a [UNINTELLIGIBLE] you were taking around and had to solve little puzzles.
And underlying them was math and simple math equations.
But-- [INTERPOSING VOICES]
Pizza.
You divide a pizza, it's cool
We can't let that slip.
That's Scott's game.
[INTERPOSING VOICES]
I think it's interesting to think about, if you have any experience like that, what would make you like the game.
I think it's very rich, because of all the people who have that experience with educational games, and then think about them when you design it.
Think about the influence
There's this series of games called Thinking Things.
And one of them had a mode that basically was kind of like programming, in that you could place different types of units on a grid, and then give them instructions in a loop.
So, say, go forward then left then right.
And they could paint a square.
And so what you'd end up doing is, if it's a starting configuration, a certain set of rules, you could get them to draw patterns and stuff like that.
And what used to happen is, my dad would say, can you get two inverted triangles drawn on the screen.
And I would sit there and try to basically program it.
I realized that it was actually a programming game, so
LOGO does a lot of that sort of thing too [UNINTELLIGIBLE PHRASE]
There's a bunch of games that have done that.
The last puzzle [UNINTELLIGIBLE] some of that stuff too.
And there's a product out just now called, that I'm hooked on right now, called Train Yard, which is on the iPad and the iPhone
[UNINTELLIGIBLE]
What?
I thought that would be your game
And it really is a programming game, basically.
It's kind of cool
Robuzzle.
[UNINTELLIGIBLE]
[UNINTELLIGIBLE]
I was just going to say that [UNINTELLIGIBLE] educational context.
Where-- Dinosaur Discovery, which is an ancient game, I played it when I was in kindergarten [UNINTELLIGIBLE].
And it was an adventure text puzzle game.
Which was Band Leader.
And then Where in the World is Carmen Sandiego
Another classic game that I played, back when-- about the time when things started getting popular, is Midnight East.
It was a math busters game
[UNINTELLIGIBLE] [INTERPOSING VOICES]
The Incredible Machine
That's like the [UNINTELLIGIBLE].
It's really [UNINTELLIGIBLE]
I loved that game
[UNINTELLIGIBLE PHRASE] back in the good days.
I really enjoyed that game that sounds like, I think we all want to add to the education game.
It's not like, oh my god, I had to play an education game, but more in, they could be really in [UNINTELLIGIBLE]
[UNINTELLIGIBLE] and it came with almanac.
They encouraged you to to do research, in fact.
It wasn't cheating to look up the answers.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
OK, so I'm actually to keep the discussion today pretty short because the games that we've got lined up are pretty long.
Two of them I know are really good.
One of them was good last year so we'll see again this year.
[UNINTELLIGIBLE PHRASE].
So we'll see.
Are there 4 games at least
I'm [UNINTELLIGIBLE] Mexico and Puerto Rico which will be five.
I just haven't pulled it out yet
I call Thebes
Cool.
For people-- have you played Thebes before
No
Good.
OK.
So for people who played San Juan last time, I think Puerto Rico's an interesting comparison.
Because, San Juan actually, it's a sequel to Puerto Rico.
So you kind of-- [INTERPOSING VOICES]
Which one was San Juan?
That was the one where, how would describe it?
Is it related to Race for the Galaxy as well?
Yeah
You choose the action
Ah, Race?
So that takes us right into today's game.
So Puerto Rico is kind of what was [INAUDIBLE].
Puerto Rico is the, I guess it was actually the first [UNINTELLIGIBLE PHRASE] we needed previous [UNINTELLIGIBLE] inspirations.
And San Juan's kind of the card game of this, which is why we introduced it earlier.
Because you guys were working on card games.
Race to the Galaxy is more of a spiritual succession as described in the reading.
But you could also think of it as a clone with a few upgrades.
Because it wasn't the same people.
It wasn't the same people who made--
It was [UNINTELLIGIBLE]
To clarify, the guy who did San Juan and the guy who did Race to the Galaxy were kind of collaborating.
So the games are really similar because they influenced each other.
But they were not strictly the same project
So you can think of it as multiple people from the same artist community turning out similar games at the same time
Nice
But, yeah, informed by each other, and when Race came out it was very much seen as this is Puerto Rico with problems fixed, or the game has expanded my bad--
And problems added, too
--and problems added, too, and, of course, completely different scenes.
But at least in terms of game mechanics, that's a nice connection that you can actually see among the games that you played before and you're going to be playing today.
Not every single game that we've got here has got that kind of heritage factor.
These games aren't specifically picked, to me, to match up with previous games or sequels.
The only reason why I'm talking about sequels because we had to talk about them sooner or later.
Of course, Battlestar Galactica is really a [?
rotation ?]
of the TV series.
And I wouldn't quite call this a sequel, in a sense, because simply, this is not based on a previous Battlestar Galactica game.
I'm sure there were previous Battlestar Galactica games that were based on the old '70s TV series that have nothing to do with this one.
This is more of the adaptation.
And the reason why I'm using that upgrade is so that we can be very clear in our vocabulary.
This is taking elements of something that's in a different medium, like the film or the TV series, and trying to figure out what in that TV series, in that world, that has been created, that fictional world, might yield interesting game playing.
And this game is pretty much about figuring out who is the Cylon in and among all the players.
Among all the players, there's going to be traitors.
And you try to hide your activities, and you try to help people, help the human race survive, if you happen not to be a traitor.
But some of the things that you end up doing in order to help the human race, not just surviving and not looking suspicious.
If you played Mafia it kind of has a little bit of that feel.
Only this is way more heavy [?
weighing ?]
of a game mechanics because they really, really wanted to make you think like the characters, that you have the same dilemmas that the characters in the TV show have
Does the Cylon know that he or she is a Cylon?
I believe that, I believe you can find out halfway

Yeah, if I recall the rules correctly.
And then there are players who will know right from the beginning.
So there may be more than one.
In fact, there is really more than one
I'm psyched
OK, so there were a bunch of different things that were listed in-- they were given as reasons why people want to do sequels.
But for the most part the big one is just you're building off an established base.
You're not having to rebuild everything from scratch.
In many cases, a sequel doesn't actually cost less or take less time than making your original game.
You just end up spending, hopefully, spending that development time a little bit more wisely.
You avoid a lot of pitfalls because you're pretty much making a very similar game.
The game industry's been one of those fields where, oddly enough, part two, the follow up, tends to be better then the original.
As opposed to how many movie sequels are generally better than the original?
Godfather, that's one of them
Back to the Future
It's rare
It's really rare.
I'm thinking the Clint Eastwood man with no name series, Godfather.
Any other?
Back to the Future
Back to the Future?
I don't know.
I could go either way
I don't agree
It could go either way
It's arguable.
It's arguable.
[INTERPOSING VOICES]
You mean like Empire Strikes Back
OK, The Empire Strikes Back
Spider-Man, perhaps?
Spider-Man 2
I don't know.
I felt like they were all bad
They were all bad
That's true
It was better
But, yeah, as we-- [INTERPOSING VOICES]
Dark Knight?
Yeah, I will, yeah, I'll give you that.
But it's still kind of tough.
Whereas how many good games sequels-- How many games can you think of where you played both the first, the original, and then the sequel to it?
[INTERPOSING VOICES]
Just the sequel
OK, well, some of them are spaced apart so far that you actually only ended up playing--
Well, I mean like Half-Life
Half-Life, Half-Life 2, for instance
Star Trek
Star Trek, Star Trek 2
Age of Empires
Age of Empires I, II, and then
Shogun - Total War
Shogun.
[INTERPOSING VOICES]
The Total War series
Yeah, Total War is kind of an interesting thing because they don't pitch them as sequels.
But they very much pitch them as a series.
And the whole idea is that if you liked one game, you actually know a lot about what any of the other games in that series do.
Sure, graphically they're upgrading.
Game mechanics wise they're trying to portal a lot of things.
Some of them are medieval.
Some of them are Japanese
In the readings, they mentioned that Final Fantastic VII was the sequel to VI.
Is that correct?
Is that correct nomenclature?
And I will disagree with that, actually.
Final Fantasy VII actually has a sequel, for instance
Right
What was it, Crisis Core?
[INTERPOSING VOICES]
There's a couple, actually.
[INTERPOSING VOICES]
More than one-- [INTERPOSING VOICES]
First of all, there's the movie.
That sequel was the movie.
And then they did a bunch of things
Oh God, there were--
Final Fantasy [UNINTELLIGIBLE]--
[UNINTELLIGIBLE] Final Fantasy--
--an entry in a series but-- [INTERPOSING VOICES]
But I think when it comes to, this is what we're going to be pitching to a publisher-- I'm not sure that Square Enix needs to pitch Final Fantasy themselves-- Think of another game that has an [?
all ?]
series, if anyone can come up with one
Baldur's Gate
Baldur's Gate might be actually a good one because different companies end up working on different types of revisions.
Actually, most of these-- well, actually, Fallout and Baldur's Gate both come from Interplay, I believe, originally
They went out of business after Bloodlands.
We were talking about this yesterday
I suspect that was before that--
That was Vampires Bloodlines
Yes, this came up yesterday
Yeah
But basically-- leaving aside what makes a company go out of business-- the issue of when you have a series and then you have different companies making different versions, Guitar Hero III, for instance--
Knights of Republic
Knights of the Old--
The second one was [UNINTELLIGIBLE PHRASE]
So BioWare was the first one and Obsidian did the second one.
Fallout was Bethesda, Obsidian, I believe.
Obsidian's really built a whole business model around making part two of something
Or expansion packs and stuff
Expansion packs, no.
You have closely related to that.
And in those cases, sometimes, like in the case of BioWare, Obsidian connections, they have a good relationship, they've all written a lot of technology, and they are getting an advantage of we've kind of ironed out the kinks, we're just sort of continuing the story, especially the story based game.
Knights of the Old Republic, clearly you've got to fit it within the Star Wars story.
And they had the interest to basically keep the tale of these characters going, even more so for-- well, Mass Effect was by Bioware, but that sort of thing.
But then you have the other angle where you just want to get the same success but you want to tell a different story, or in the case of Guitar Hero III have a completely different set of music.
And in Final Fantasy VII, you had certain game mechanics and certain things that carry on from game to game.
And definitely you want to carry the fans from game to game.
But part of the appeal of the Final Fantasy series is that every single one is going to be a different story, not dramatically different but it's going to be a different story with different characters and a different world.
So but this is pitched very similarly when you go to a game company and your whole idea is how do we get the fans to buy even more, or even more fans, to buy version two.
There are two edges to that sword, though, because what typically happens in a lot of sequel development is, they are trying so hard to please the original fans often by adding complexity, by adding things that build on things that players of the first game already know, that you actually start alienating new players.
The game becomes just so complex or just complex enough that people start getting turned off.
A couple of weeks ago over here we played Guitar Freaks on Friday Games for Gambit.
And the first version is actually extremely accessible.
Once again, from the second version all the way up to, I think, the version 4 that we were playing, it is almost impossible for a new player to jump in because they've got one beginning friendly song and everything else is based for fans of the game.
So you imagine a Guitar Hero game that's built entirely for people who've already mastered expert on the previous game.
That's kind of what happened with Guitar Freaks.
And in many cases, that might be enough if the game was particularly successful the first time around, and you only get 80% of the fan base to buy the second version, that's still a lot of money.
And it's a way lower risk compared to coming up with new IP from scratch.
And a lot of companies, when they're creating IP, they're very much specifically thinking, right off the bat, what are the sequel opportunities.
Even now, because you've got downloadable content, so it's even narrower than that, is what are downloadable content opportunities?
How do we release extension packs really, really quickly?
And how do we create stories, worlds, technology that's just going to be able to anticipate all that.
There's a game up there now on Xbox Live Arcade called Monday Night Combat.
And they did some of the old clever hacks to basically allow them to update the game on a week to week basis.
Whereas typically that kind of a certification that's required from Microsoft is really expensive.
Every time you want to release a new version, you have to put it through an entire rigorous QA testing schedule, again.
But they came up with some technology that let them make at least game tuning changes on the fly without having to go through the Microsoft testing series.
So it's not technically releasing the sequel but they were very much thinking about how do we keep updating this game.
I wouldn't be surprised if Steam started off as that.
There's a battle net kind of started off as just an easy way for them to get up patches and to keep tuning this game, and eventually someone involved decides, hey, we can sell games this way.
And launch was kind of bumpy.
But it's fine now.
There is one thing which the book didn't mention which is the reboot, the sort of franchise reboot.
This can sometimes happen when you have a publisher who's got control of an RP that kind of floundered halfway.
Maybe there was a bad execution on creation and possible circumstance.
Or they just play out the formula to death.
And then they want to try to get a new batch of fans, a few people who liked the original one.
And I was talking with the Tomb Raider game where they're just trying to-- they've rebooted the Tomb Raider franchise several times, at this point, each time trying to say, OK, you've probably heard from Tomb Raider from the huge amount of advertising that we've done when you may not have actually liked the previous versions of the games.
And we're doing to reboot this franchise by making a different game in that same world.
In Tomb Raider we'll use characters and we'll use the fiction, but we'll change the game play significantly.
Because we're not going after the old fans anymore.
We're going after a new batch.
It's probably about the same technical challenge as updating, as trying to make a sequel for a completely new platform.
Because they're basically having to write everything from scratch.
But the good news, of course, is that all that previous advertising that you did a decade ago still works.
Let me think of-- Can anyone think of a good example of a good reboot?
Is Mario 64 a reboot?
No
I think it was seen to a lot of people that it was because it wasn't clearly a sequel to any of the previous Mario games.
And it was kind of Nintendo kind of saying, hey, it's not--
It's not really a clone of anything
It's not really-- It invented a lot of things for the first time, in fact.
I would say that Mario 64 and, of course, the way how they branded it, they didn't brand it Super Mario World, what 5 or Super Mario 4 or anything, they called it Mario 64 to sort of distinguish it from some of the--
I would almost call IT an upgrade more than a reboot, though.
Because it was mostly just a move from 2D to 3
It was more than that, I think
It's a good challenge because to the player it may seem like it's not that but for a technical point of view they had to continue to reinvent how Mario moves, the way how Mario jumps frankly, the way how the camera moves, it never used to be a huge problem in the 2D games, but was the primary thing that they've managed to figure out how to get working in 3D was the camera.
I wouldn't be surprised if their marketing pitched it a little bit like a sequel.
But it was also very, very clear of this is going to be a whole new generation, you've never played Mario on anything like this, so don't expect your previous Mario experience to carry over necessarily
I was going to say Metroid, as well
Metroid Prime?
Yeah
Yeah, that's a really good example.
In fact, Metroid Prime was an instance where they gave the franchise to a completely different developer.
Previously, it's all internal Nintendo.
And then Nintendo went to a western developer, Retro Studios over in Texas and basically had them make the first western version of Metroid.
But they had a very heavy hand in controlling that.
Whereas-- I'm not sure we're going to be playing this later-- Castlevania: Lords of Shadow which just came out, kind of went the other way around.
Western developers, a Spanish developer, decided to make a Castlevania clone.
They loved Castlevania.
And they were going to make a game that was just like Castlevania but they don't call it Castlevania because they didn't have the rights for it.
And they showed it off at E3 or something, and Konami basically said, why don't we just turn this into a Castlevania game, Hideo Kojima to [INAUDIBLE] making it work
It must have been their dream come true
That was probably their dream come true.
That was probably what they wanted.
And you had the advantage of the all the Castlevania fans now saying OK, this is going to be in canon, if we liked Castlevania, we'll probably like this one, which is exactly what they were trying to get across.
It's like imagine the people who made Torchlight being approached from Blizzard saying why don't we just turn this into Diablo 3
A bunch of people who made Torchlight were from Blizzard, right?
I wouldn't be surprised
Were they from the Diablo team?
I wouldn't be surprised.
In the same sense that a lot of people who built GuildWars were from the World of Warcraft team.
So that's less an issue of sequels and more an issue of that's just how the game industry works.
People move from project to project and carry knowledge with them
Like Call of Duty was [UNINTELLIGIBLE PHRASE]
Yes, Call of Duty, yes and--
They got tired of it and were like we can do better
That's an interesting discussion of instead of sequels you have companies making effectively clones competing against the original franchise.
And in this product case they're trying-- there's people who left EA on 2015, I believe, and then they started Infinity Ward.
And they made a game that basically competed with the original
The first Call of Duty was so ridiculously good, at least, it marked me when I played it the first time
But it's very clearly trying to be Medal of Honor the next version
Yeah, clearly, yeah, no, of course, of course
So that's an interesting situation where it's hard to be a spiritual successor when you're competing against the original product.
But you can think of them--
BioWare did that with Dragon-- I mean not competing against [UNINTELLIGIBLE PHRASE] Dragon Age because they don't have the rights to [UNINTELLIGIBLE] game anymore
Well, Dragon Age is more of a DND clone, I would say, which-- [INTERPOSING VOICES]
Empire Earth reinvented itself with--
Which one?
--with Empire Earth--
Empire Earth
--with zero success.
It didn't do [UNINTELLIGIBLE]
I kind of lost track of that series
In the whole game, the first game was like, you can go through 15 ages.
You can literally play a game against someone and go from prehistoric units to nukes in the same game with everything in between.
And then they released the sequel which, I think, overseas and then it's like they [UNINTELLIGIBLE] themselves and go, we're going to simplify everything, there's just going to be four ages, kind of like in Ancient Empires, and we're not going to have this massive progression.
And they tried to simplify everything.
And it totally bombed
I think that's a rebooting.
But I think that's a reboot gone wrong.
That's a, our original game is not selling well enough among new players, maybe, and it's not entirely satisfying our old fans, so we're going to take this franchise that we advertised really, really well, and we're going to make it completely, effectively a new game, maybe with the same theme but a new game that targeted at a new set of players.
Sometimes it works.
Sometimes it bombs.
Might and Magic released a game called Clash of Heroes, Might and Magic Clash of Heroes for the Nintendo DS.
Who's played any of the original Might and Magic games?
Different from Heroes of Might and Magic?
Either of them, actually.
So who's played Heroes of Might and Magic?
Can you describe the game?
What's it like?
Want to go, [?
Fred?
?]
Well, it's like essentially just a turn based strategy game where you have cities and you have heroes.
And these heroes lead armies.
And you encounter other smaller armies and try to defeat them and stuff.
And you have--
[UNINTELLIGIBLE] city, you start with-- you have an economy
Yeah, you have an economy and you build things in your city that can then produce better units for your armies and stuff.
You can take over other cities and-- [INTERPOSING VOICES]
Like Civilization, I would say
Yeah, it's like Civilization but if you took Civilization and removed everything that wasn't military
Yeah
And that was what played out and gave it its distinctive arena--
Yeah
--turn based--
You're trying to do stuff like battlefields where the turn based, you have your units on one side and the opponent's units on the other side.
And you move units together.
So if you have pikemen and bowmen, you have the clump of all bowmen who then move together as a unit
So Clash of Heroes, a Nintendo DS game in the Might and Magic series, and what they did was, they made a match three game.
They made a game where you-- it's like Bejeweled.
You take three bowmen, put them in a row, and then they become a bunch of bowmen who are going to attack.
So they get the same idea of let's take-- and because the Nintendo DS has two screens, so the enemy's armies are on one and yours are on the other.
It's kind of like sending troops up and down.
But the game play's dramatically different.
It's effectively a franchise reboot.
They want to get the same idea of leading massive armies against each other.
I think they take some of the characters, at least some of the technologies that came with the original Heroes of Might and Magic.
But that's as far as it got.
Everything else they changed.
Because they were going for a completely different audience, the assumption being Nintendo DS players liked match three games like Bejeweled, actually really good.
I'd encourage it if you like match three games.
I was going to put Sarah on the spot for having worked on Thief II, and Thief Gold, and Thief 2 Gold, and a number of releases.
But maybe we can start telling us a little bit about your original involvement on Thief
Sure.
Do you want me to stand up to actually hear me?
I can bring the mike closer
So Thief: The Dark Project, I came on as actually as team tester.
Part way through it I became community manager, essentially.
And then I actually stepped into a design role for about the second half of the project.
And I continued to work as a designer and project manager on Thief II and Thief Gold
What were you designing in Thief?
In Thief I, I was pretty specifically designing game play for a level that someone else had already built.
Building levels took a lot longer than putting in game play, at least from my point of view.
I believe the person who built it thought that building levels was a lot faster than laying game play.
So it was a good combination.
But it was built by an artist who didn't have the scripting experience or the testing visions to go through and make sure that all the [UNINTELLIGIBLE] for the game play experience is working.
So I was basically handed a level and said, OK, make it fun
Now Thief is a first person stealth game.
Basically you're looking through the eyes of a thief who is not physically all that robust.
But he can sneak.
And he can shoot arrows, usually at your back, and hide
And the AI is, unlike in most games where the AI is tuned sort of aha, we noticed you, we charge you, the AI is carefully tuned to both tell the player if the AI sort of notices you, if the AI thinks he's looking for you and is looking for, and if he's actually coming after you.
So that a player who makes a noise while sneaking up on them will realize that, oh, he noticed that, I should go be more quiet.
So the AI feedback and [UNINTELLIGIBLE] was actually really pretty important to playing experience.
If you didn't give the player an avenue to approach a guard from behind or to sneak up on them in some way, you'll pretty much guarantee the player will get killed.
Because the player could not, in fact, take one guard on one to one many times.
You could probably kill one guard face to face, but then he'd be sufficiently wounded that the second fight would finish him off.
So you had to be giving the player channels and ways to sneak up and places to hide and things like that if you wanted to have an interesting game and a challenging game
And that's the game play we say you're talking about
Those are the game plays we've been talking about, yeah, placing AIs determining their path, putting-- this actually particularly had a lot of traps in it-- so creating the traps and giving clues to the player in games so that they could, A, either notice the trap was there before getting hit by it and finding some other way to trigger it, or notice that it was there or notice it was there and then discover how to get out of it
Thief is a 199--
Oh God, 19-- ancient at this point
199--
199--, 1996, it was released around the same time Doom was
And it's pretty impressive because in Doom you can't look up, I believe.
You can shoot up but you can't--
Yeah.
I think you can crouch.
You can lean around corners.
You can, in fact, shoot your bow while leaning around corners.
And, yeah, you can look up and down and all around
So that's Thief I.
What happens when Thief II gets approved?
Well, the first thing I had to do was convince people they wanted to do Thief II.
Because while Thief was a great game and it's actually sold fairly well over time, its initial sales were not what publishers want to see.
Because the initial sales a publisher wants to see is, oh my God, it sold 10 million copies, awesome.
Instead what we did was we sold a fairly good first initial sell through and then it just kind of kept going.
So the first thing we had to do was convince them that, yes, Thief II would be a good idea, enough people are continuing to buy Thief I.
Those same people will all run out and buy Thief II on the very first day.
And the way we did that was, first of all, taking a look at forums and saying, hey look, people are pretty excited about this.
They're enjoying it.
It's got good lasting power.
Our fans are going to stick around to play with it a second time.
And that was sort of a marketing end analysis that led into a publisher.
And the other part was saying, OK, what are the improvements we're going to make to this game?
How are we going to change this game to make it better than Thief I while still taking advantage of everything we did?
So we created basically a big design doc with, here are the areas that we thought we could have done better in Thief I.
Here's what we're going to improve.
And here are the changes we're going to make.
So we're not going to make Thief I, again.
We're going to take what Thief I was, add some improvements to it, make a deeper story, and hopefully get a more variety of game play was what we actually ended up aiming for in our initial docs.
We wanted to give Garrett additional abilities, vary the game play.
The game play in Thief I was all speed.
We tried to have some additional types of missions in Thief II.
But there was a little more face to face fighting.
There was a little more subterfuge.
Instead of just sneaking around, we also had, OK, so now you need to disguise yourself as someone and sneak in, sneak out again.
You need to pass for them while getting a goal done and things like that.
So we tried to find, what are additional stealth and subterfuge issues and game play we can have?
How many of those features in Thief II were inherited from dropped features from Thief I?
A lot of them inherited from dropped ideas from Thief I.
But when they first started designing Thief-- and I was not on the initial Thief design team so now I'm making up what they were talking about.
It wasn't clear that it was going to be-- It was sort of like, it's going to be a stealthy game, but we hadn't actually defined what stealthy game play was yet.
So it was Thief I that kind of told us what, this is what stealth game play that works.
And once we found the one or two things that worked, we kind of went with that and put in that.
It was in Thief II, we got to play around what are some other ideas of stealthy game play?
What are some other ways to trick people into [INAUDIBLE]?
And people that played Metal Gear Solid or any of the games in Metal Gear Solid series, it owes a lot to Thief, especially the really, really noisy end you see.
It wasn't, I heard something, I wonder what's over there.
[INTERPOSING VOICES]
DS-X also
But DS-X is also [UNINTELLIGIBLE PHRASE]
But that's what I'm saying is that DS-X also actually took a lot of Thief's lessons and went on with them from there
Again, a lot of people working on [UNINTELLIGIBLE PHRASE]
DS-X2, more DS-X2, but yeah.
Looking Glass folded and went out of business.
And a lot of people moved to Austin to then work on Thief III and DS-X2 and those projects there
I have a question about the Gold version
The Gold version?
The Gold versions of the games are really a way to try to convince, from the marketing point of view, is a way to convince players to buy the same material again.
I'm being a little cynical here, and I admit that.
But we added new, we did, in fact, add, what we did was, we added several new missions.
So it was expanded play.
And then we went back and problems that we knew were in the missions, we fixed them.
And we actually added some richer game play to several of the missions that, oh God, we have to get this out the door, we have to get this out the door, this would be so cool if we could set this up but we don't have time to, and out it goes.
And so we went back and we touched up over half of the missions.
We added some additional game play.
We added extra goals.
We took areas where we thought the game play was particularly weak because we'd gotten rushed on finishing it and we made the levels a bit longer and a little more interesting in our opinion.
And then we also added three new levels.
And it was kind of a chance, it was a do over.
We had very limited programming resources so we didn't really get to add much in the way of new features.
But we were able to go back and tune it and make significant changes to the levels, to the way that the guards worked, to the scripting behind the levels.
Vin was working on a 3D mod.
There's usually a fair bit the designer can do behind the walls of telling the AIs what to do and how to think and how the game play flows.
And Thief, especially, had an extremely rich editor that-- more powerful, I think, than any other editor I ever worked with, even though it is now 15 years old, 15 years old, I think.
It's still the most powerful editor for a designer I have ever seen
So did you feel-- You mentioned you looked at forums and you look at fan input.
And how much of that, of what went into making the sequel, actually came from fan input?
Or was it just like verifying what you already wanted to do
Part of it was to verify what we wanted to do.
Part of it was to verify there was enough interest that people would want to do it.
Part of it was to see, what did people like most?
Which of the missions did people like most?
Are there any good ideas out there for how to play a game that we think, yes, they're right, we never thought of that.
I think that one of the things people talked about a lot was actually disguise missions.
And I think we had two disguise missions in Thief II where you're kind of disguising yourself as, no, no, no, no one looked at me, I'm just another guard walking around, don't worry about me.
And that was something that we said, oh, yeah, that would be really cool.
And that's something we had no time to implement for Thief but, if we're choosing new features to implement in Thief II, that was actually not an amazingly difficult one.
Because you just, if you've got a way to put a tag on the play that says, oh, guards don't get upset when they see you
Did you have a situations where players came out with exploits that you tried to address in the Gold version?
There were not, no, actually.
There were a couple of places where people found bugs that were level enders that got people stuck that we went in and fixed.
There were not a whole lot of exploits that I recall us trying to go in and fix
So if somebody deliberately found something that put them on the roof of the castle or something like that, then you didn't want them trying to [INAUDIBLE]?
If it allowed the player to get to a place when he broke the game, we did try to fix it.
Because we tried to fix anything that was game breaker where you could get up on the wall and then leap off into the void.
But sometimes what we'd do is, rather than stop you from getting up on the castle and proving how cool you were, we'd let you get up on the castle but we'd put in invisible barriers so you couldn't hurl yourself into the darkness and then crash the game.
If players enjoyed finding those things so taking them out doesn't enhance the game play.
And the point of the Gold project was to try to make something the game play was sufficiently enhanced that it was worthwhile to go out and buy again
So now we're in an environment where patches pretty much is downloaded, right?
Yeah
We don't necessarily have that situation in the year 1996.
Is there still a role of [UNINTELLIGIBLE PHRASES] new releases?
I think now again that would be an expansion pack.
I don't think that you would do a Gold version.
You'd do an expansion pack.
And with an expansion pack patch you could fix old problems and you might introduce new games and of levels, but I don't think you do it with the hassle of doing a Gold.
I think you just do an expansion pack.
[APPLAUSE]
Questions for Sarah and her experience on it?
Pretty much all of your data points just came from forums where you--
Oh yeah, oh yeah, that was just one of the places where we looked.
We also looked at sales and continuing sales and things like that.
Like I said, that was actually more of a marketing sales to sort of do, here's why we think players will like it.
Here's some experiences we got.
Here's the hard data on our sales, how our sales are continuing, and the fact that while it did not come out and sell 100,000 copies the first month, it's continuing to sell 20,000 or 30,000 copies a month, which means that word of mouth is pretty strong, and people are continuing to go out and buy it and go out and play it.
And here's the forum where a whole bunch of people are using the editor released.
And the activity in that forum is going up over time not down over time which indicates you've got more people coming into it, people are coming in, they're staying at and using it, and you've got more people continuing to come in, the community's growing, which means that there's interest there and there's people who are paying attention to project and are going to continue to pay attention to your product
But also it gave for finding dead ends, for finding-- You said you fixed some of the dead games [UNINTELLIGIBLE]
Yeah, oh yeah, yeah.
Well, the difference between a-- let me think-- At our height, we had a 12 person QA department.
And 50,000 users is pretty big.
And we did our best on testing it.
And we had another 10 or 15 helpers through our publisher.
But we had comparatively speaking a ridiculously small QA department for the game
But then you have the situation where someone notes something on the forum and then you can verify it with your QA team
Yeah, yeah, yeah
And you can actually figure out whether what they were seeing was exactly the problem they were describing
And it being a solely single player game-- I know a lot of games do large beta tests where they ask 500, or 200, or 1,000 people to play the game.
We did not do any external testing like that at all.
Yeah
And this was also back in the '90s that wasn't being done
Oh, yeah, it was not the sort of thing that was really done in the '90s as well.
It's very much a developmental shift
Did you ever explicitly ask on the forums if there were another sequel, would you buy it?
I don't know.
I did not do that.
I was not on the forums.
I helped with some of the forum data in [?
reforms ?]
but I didn't really post on the forums a lot
Do you think that would be viable tactic, or would you just say, people would probably just say yes
I think that your data wouldn't be very good

The most you're probably going to get is the 1,000 people who read the forum say, yes.
And that's 1,000 people.
And that's not enough sales to justify doing anything
Gotcha
It's more interesting, I think it's more useful to look at trends, if you've got a survey out maybe, I don't know
I've seen companies bring in focus groups, people who played the first game to come in and bouncing off either prototypes or features that they're planning for the second version to see how they will respond to them.
I would say that the information you back from that is also not very good.
But it might be something that is done to gain traction in internal studio politics, and it was like, well, we did this exercise where we brought these people in and they said these features are OK, so we can take the risk of [UNINTELLIGIBLE PHRASE].
As opposed to driving the entire process [INAUDIBLE]
Games don't respond really well to focus test based development
Or focus group--
Or focus group based development.
Getting feedback on your games is great.
Finding out how many people are likely to like your game is a lot harder
Especially if they can't actually see it when you're asking them the question then they have no idea what they're commenting on.
They'll make something up that they think you want to hear, is, yes, of course, we'll buy a copy of the game.
We loved the first one.
Sure.
But if they haven't actually seen the second version of the game [UNINTELLIGIBLE PHRASE], so you don't actually know if that is the case.
For those of you who do missed [UNINTELLIGIBLE] talk last week, he was talking about how Inamplitude which was a sequel-- Was Amplitude-- [INTERPOSING VOICES]
Oh yeah
I think it was in the very first one.
So that was Frequency.
He was talking about how the folks from Sony were helping them run their focus groups.
And they did two tests and one of the tests was, after they played the game, they did a survey of how many people were planning on buying this when it came out.
And they had a particularly high score, like 80% or something like that, like the highest survey I've ever seen.
But they also did a survey of how many people were planning on buying the game just by looking at the box before they had played the game.
And that was a particularly low number.
And the Sony people called it, said this is not going to sell.
Because, sure, the people who played their game are going to enjoy the game.
But no one's going to play the game in the first place because people can't figure out what your game's about.
And they tried to address those problems with the sequel.
That's one thing which [?
Brenda ?]
talks about, your chance to do a do over, a chance for you to go back and fix the problems that you found in the first one, especially when you find those problems really close to ship date.
There's nothing you can do about the first time around.
But if you're lucky, you get a second chance to do it
I also noticed that Frequency was not called in, or rather, it wasn't Frequency
There was a Frequency II.
They called it Amplitude.
So it was actually kind of a reboot as well because they kind of just wanted people, OK, let's pretend Frequency didn't happen and [UNINTELLIGIBLE].
And we'll make sure that our Frequency people who loved the game know what it is
Ahead of time.
It's much easier to tell the small base who liked you, oh yeah, and buy this game, than it is to tell a large group of people who weren't interested the first time, no, no, you didn't like it the first time, but trust us, you'll like it this time
I've seen a couple of companies do effectively Gold versions every time they move to a new platform, like the latest platform which they release it on tends to become the definitive version that, if you're a big fan of the franchise and you're willing to buy this game multiple times, you want to get that because of the few extra features that they've put in.
Maybe it's different endings.
Maybe it's upgraded cutscenes.
If it's a role playing game I see that a lot.
Maybe they redid the cutscenes from scratch or redid the audio from scratch.
So it's the same assets but they're higher resolution.
And, of course, you have games that started out there like this online flash games and stuff like that and then you've got to move them to other platforms like DS or PSP.
And those games tend to have not only completely reworked controls because the controls are different but also a lot more features because you have to now justify to people who may have loved the game but didn't pay any money for the first version to actually put down money for it now.
So [?
N ?]
is a good example of that.
There's a game that came out by [UNINTELLIGIBLE] and the DS and PSP what started life as a free flash game.
And lots of people loved it.
But lots of people didn't have to pay any money for their first version.
So how much do you love it is kind of a weird factor.
I guess we should start with the games.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information, see the course materials on the MIT OpenCourseWare website
OK, so today, we have a guest lecturer, Clara.
Some of you have probably already taken her class.
CLARA FERNANDEZ-
[UNINTELLIGIBLE].
My current victims
And past victims.
CLARA FERNANDEZ-
[UNINTELLIGIBLE]
We'll probably need to do something about the lights.
[INAUDIBLE].
I'm sorry, I'm sorry.
I'm still operating on Tylenol and caffeine.
So today we're going to talk about stories in games.
And Clara teaches not just the Intro to Video Game Studies class, but also the Writing for Video Games class in the spring.
I wanted her to come in and bring her expertise because she's [UNINTELLIGIBLE] probably a lot more than I have.
CLARA FERNANDEZ-
Hello, perfect.
[INAUDIBLE].
So yeah, Philip asked me to come here and talk about stories in games, and as far as I remember, he said you guys had to read two articles for today.
Chapter four in [INAUDIBLE] and Soren Johnson's Theme is Not Meaning.
Didn't I [INAUDIBLE], too?
Yes.
CLARA FERNANDEZ-
And he's late,
Shh!
CLARA FERNANDEZ-
[LAUGHS] Anyway, so preparing this has been kind of interesting because-- I thought about this quite a bit because there's this focus in my research studying how games and stories can be brought together.
What does it mean to have a story in a game?
Where do we put the story?
But the interesting challenge was thinking, well, the guys are working mostly on non-visual games.
And what do stories have to do with non-fictional games.
So here what I'm going to need is your input, because you guys have been playing all these board games, card games, and I want a bit of your input of what you think.
How you think these concepts apply to non-digital games, because the part of digital games I can do pretty confidently.
But I don't know as much about board games or card games.
So I need your expertise.
This is why you guys are here.
So the first thing that I wanted to ask you about is that we have two different meetings, and one is sort of about fiction and the other is sort of about theme.
But what is the fiction of a game?
I turn it to you
The fiction of a game is, well, whatever you want it to be.
It could be the characters in a digital game.
It could be a real environment, but you just have fictional characters.
It could be an environment that may not exist.
It's pretty much up to the designer to figure out what they want to-- like with [INAUDIBLE] be in the game that are real, or if they want it to be fiction.
CLARA FERNANDEZ-
Well, there's something there, but it might be a bit more intuitive than that.
I think that you're on the right track, yes
I'd say it's the setting or the universe in which the user perceives the events of the game to take place in, or the one that the designer intends the user to see the game take place in
OK, so you can mention environment as the big-- what was the word you used-- the universe, the setting, something like that.
Back there
Oh yeah, I was going to say that the word that is used over and over is world.
It's just a world, pretty much.
CLARA FERNANDEZ-
And that's the keyword.
When [UNINTELLIGIBLE] is talking about fiction, he's really talking all throughout about fictional worlds.
In a way, designing games and designing video games specifically, is always thinking about what is the world of this game.
And at times there might be games that don't have worlds.
You can make Tetris.
You could make-- There's very few video games without worlds.
But there are some.
Tetris--
Peggle.
So does Peggle.
CLARA FERNANDEZ-
Peggle?
Well, Peggle has characters, you know.
You were talking about this.
Characters are already [UNINTELLIGIBLE] in fiction
You said it has nothing to do with unicorns.
CLARA FERNANDEZ-
Yes.
Who wants to say something?
We have two hands here
I'm stealing it.
OK, in print or with board games, I think of fiction much more as the specific explicit things they give you, like the pictures, the rules, and all of that stuff versus what it feels like you're doing.
So if they don't specifically tell you you're in blah place, then that's not part of the fiction.
CLARA FERNANDEZ-
OK, yeah.
So you put your finger on it, but when we're talking about non-digital games, there are some non-digital games where thinking about worlds is a bit shaky.
Do you want to say something?
Yeah, just going to add that in an environment, you have a set of things you can do.
So you're basically given rules.
Like in a board game, you could be given a character that has to jump for some reason.
The board game could say that they could jump some insane distance.
Well, in real life, they can't do that.
That's just part of the fiction.
So it's whatever laws apply within that world, as well.
CLARA FERNANDEZ-
This is a very interesting point because when you were talking about-- what's that word that he uses-- incongruous worlds?
Incoherent, incoherent.
He talks about incoherent worlds, and it's like, well, incoherent according to what?
It's obviously not the rules that are also creating part of the fiction.
The rules are also creating what are the laws of gravity in a specific world that we are playing in, for example.
So when we're talking about fiction, we're mainly talking about fictional world.
Now, how does that relate to theme?
And you were already kind of hinting at that.
That in fiction and board games, it's kind of like, well, it almost can be about anything.
But what is the theme of a g-- when you-- if you remember Soren Johnson's discussion-- what does he refer to as a theme?
He is not an academic, so he has a more wobbly, loose definition.
What is the theme of a game?
The theme of the game is what the designer says to the player, this is what the game is about.
CLARA FERNANDEZ-
But when-- Soren Johnson is saying theme is not meaning.
So he's making a point that theme might not be what the game is about.
Right?
This is what the game designer tells you.
CLARA FERNANDEZ-
Yeah.
The example that he gives is Ticket to Ride, right?
And Ticket to Ride is not about riding trains.
It's about-- it's about-- well, the description is about going from one end to another, right?
Yeah.
And travelling across-- CLARA FERNANDEZ-
Is building--
It's about travelling across the United States.
CLARA FERNANDEZ-
But is it really travelling, or is it building the--
It's not even a story about-- the story's about travelling.
CLARA FERNANDEZ-
Yeah, the story's about travelling, that's the theme.
But what you actually do is--
The official story is that I think it's some number of years after the events that were Around the World in Eighty Days, and to commemorate that, the original people who were involved in that race decided they're going to race across America in trains.
And that is the theme of the game, according to-- CLARA FERNANDEZ-
Yeah, that's the theme of the game
Well, that sort of what drives to the point of what Sorenson's talking about, that when theme and meaning are incongruous-- as they are in Ticket to Ride to an extreme extent-- it can actually turn people off from the game sometimes.
CLARA FERNANDEZ-
Well, and it's not that it's a bad game--
Or Halo Wars.
CLARA FERNANDEZ-
Well, not necessarily
The thing about Halo Wars was that it sounded like you bought it thinking it was going to be an action game, then got disappointed and were like, blah!
In the case with Ticket to Ride, it is pretty much just what they were saying there, right?
That the theme seems to just be whoever it is that wrote the instruction booklet say that you're doing, which may not necessarily have anything to do with the actual mechanics of the game.
CLARA FERNANDEZ-
Yeah, I have the game.
I didn't know if you guys have played it.
I made you want to play it.
[INAUDIBLE] As of right now, [UNINTELLIGIBLE] is a game called Too Many Cooks, and it seems to be about cooking.
Yeah, you have ingredients, and you're making a soup of sorts.
But really what you're doing is resource management and exchanging.
Just imagine being a cook and having to, like, well, I have to wait until I get the right ingredient or maybe exchange it for another.
That's not how you cook really.
So there's always the clash of how the fiction of the game relates to the rules.
And the interesting distinction between fiction and theme is that theme seems to be like a layer, seems to be like a decoration.
At times, it can be part of the title.
It can be how the tokens look like, or what the board looks like.
That is setting up the fiction of the game.
Not the fiction, the theme.
But it doesn't really have a fictional world.
When when you're playing Ticket to Ride or when you're playing Settlers, you have a feeling like you're in a world
At least, more so than Ticket to Ride.
So, I will argue that maybe the world that you feel is not that they're telling you that you're in, right?
One of the problems of Ticket to Ride is that you never really feel these characters exist.
But the world of the United States is very, very present.
Like the positioning of the cities, the fact that these cities exist and can be connected by land is very-- CLARA FERNANDEZ-
You guys have read the levels of abstraction?
Talked about levels of abstraction here at all?
Not in great detail, but-- CLARA FERNANDEZ-
OK, it's the idea that it's a world, but the way that it works is so abstracted that it doesn't really-- you're not in a world anymore.
It's just these little bits and pieces.
OK, we have fiction and we have theme, but we also have stories.
And what do stories have to do with this?
How do we have the story in non-digital games?
What principle-- a story-- I'm going to find it for you to save time.
A story is a series of connected events that involve characters.
So those characters perform actions, or there are happenings, things that happen to them, and those actions happen in time and in a specific setting, in a specific place.
And in the case of games, they also involve objects.
That's like my telegraphic version of what a story is.
So what we do in games, in games we have changes of state, right?
Every step of a non-digital game, particularly.
We can think about non-digital games as changes of state, and having a change of state can constitute an event.
So that is one way.
So how do we generate rules that generate changes of state that can be thought of as an event in the story?
But I want to know a bit more.
Of the games that you've played so far, how can non-digital games-- and maybe you've played more board than card games than anything, but-- how can we have a story?
How can we have those events happen?
So one game that I've played that has a really nice story to it is Battlestar Galactica.
In a way, that places-- it works off of what the player already knows.
So it sort of assumes that the player has a knowledge of the Battlestar Galactica world, and so it keeps referring to that.
Whenever you play a card that has special abilities, [INAUDIBLE], so it relies on previous knowledge.
CLARA FERNANDEZ-
That's a very good point.
The point of-- in digital games and non-digital games, but is also in digital games-- a lot of storytelling in games happens in the player's head.
Is it evoking [INAUDIBLE] , is tapping on the player's knowledge.
Is not, as we've got to say, is not so much about storytelling, but like, OK, what do you know?
We are making the player fill in the gaps.
So that designing a game that involves storytelling is usually giving cues to the player to create the story, if that makes sense.
Patrick?
I was going to say that a lot of board and card games, a lot of the fiction just comes from the metaphors of the mechanics.
So, for example, in Settlers, when you want to build a road, if you strip away all the fiction, it's just I have these cards, I put them down, I can put this piece here.
But adding the fiction to it, you say now, I'm not just putting cards down, I'm using the resources to build a road.
And so that metaphorical action is another event in the story.
CLARA FERNANDEZ-
Well, it becomes a meaningful action.
That's another-- I'm sure that you guys have been talking about that.
What is a meaningful action?
And at times, meaningful action is it constitutes a story event.
Yeah
I think the whole point of Munchkin and and Shiggy and those games are to construct stories.
When we played it, there was a moment when someon trashed someone else's shrooms, and then the other person played a Go Dumpster Diving card to go pick up the shrooms and a bong from the dumpster.
And this whole sequence of events, the [UNINTELLIGIBLE] text on the cards, what they were, and the actions you were taking, and you string them all together, I did this, and this happened, and this happened with this object, and you've got a perfect story, because you know you're playing the geek, and that-- CLARA FERNANDEZ-
Well, there's that.
But the other thing that is good about Munchkin is that reading the cards is fun.
There are bits of the story you'll get in Munchkin that are like, [INAUDIBLE].
That is kind of like engaging already.
Oh, and you wanted to say something?
I was just going to say that there are some games which are entirely about telling stories.
CLARA FERNANDEZ-
Thank you
Role-playing games are the number one, and then another example is something like Once Upon a Time, which I think is [INAUDIBLE].
And then they have, also, The Adventures of Baron Munchausen, if you've ever seen-- CLARA FERNANDEZ-
Yes.
Oh, I should bring it, yes.
I have it in my office.
Thought that's kind of difficult to set up in half an hour.
But yes
And actually, a friend of mine invented a game which is vaguely based on The Adventures of Baron Munchausen, where each person is telling a story and they have cards in front of them which needs to be flipped over by incorporating the elements of the story into their hand of cards-- Anyway, don't worry about it
Yeah, you're going a bit faster, but that's fine because yeah, there are games that are about storytelling, communal storytelling, and role-playing games are about that.
And again, as game designers, what we are given when we have the rule book from D&D is a fictional world that has rules.
But then we have, as a designer, as a DM, we are coming up with what is the situation?
What is the world?
What is the the specific challenges that we're situating in this world?
How are we going to do that?
But it's about communal storytelling.
As we're going to see in a minute, there are also games that are about coming up with stories.
We're being thrown in a similar way-- well, in Munchkin, you're kind of constructing the story as you go.
But in the games that we're going to play today, or some of you are, like Gloom or Once Upon a Time, the game is being able to produce a narrative.
Being able to make sense given the specific cues.
You've had your hand up?
I was going to do the opposite extreme of a game with a very basic story, and you kind of make a story up as you go along, but it's not really that crazy.
This card game, Falling, where pretty much the pretense is you are [UNINTELLIGIBLE], and you're falling to your demise.
And so while you guys are falling, you guys decide we'll make a contest of who hits the ground last.
And so it's a real-time game, and you're dealt cards in order, and it keeps going until you reach the ground card.
And so you can attack other people.
OK, this is going to happen to you and different effects happen.
And so when you're actually playing it, it's kind of like the intensity when you're falling.
Like, oh, I'm going to die, I have to do this fast because, before you know it, the ground is already at you
It's interesting because it creates a longer event of something that's supposed is supposed to be, pop!
Gone.
Scratch, right?
But it's kind of zooming in on how long that would feel if you were actually falling, making a mini-story of that.
It's interesting because I don't know how strongly-- this is one of the questions we'll be talking about in stories in games is what kind of an event constitutes an event in the story?
Is opening a door an event in the story?
Well, in some games, it can be because it might give you access to new areas, there might be someone showing up that changes things.
But in other cases, like Monopoly, going around and getting the-- how many, $100?-- $200, is that an event?
Maybe it's not.
It doesn't seem like a story event because it's such a mundane thing that happens all the time, although clearly, it has [INAUDIBLE].
CLARA FERNANDEZ-
Yeah, it does have an effect, but as a storytelling art, is kind of questionable.
The other thing I wanted to ask you, seeing as you guys have been playing particular board games, there are board games that have similar mechanics.
So, I played Race to the Galaxy, which I really like.
And I've been told that [?
Potomac Coast ?]
is very similar, yes?
So does changing the theme-- those two games have different themes-- but does changing the theme affect the rules in any ways?
For those of you who might have played both?
Not really.
Not for-- San Juan, I think is closer.
San Juan is with the cards.
But it doesn't really change the game, because the mechanics are almost identical.
It's just covered in a different shell, or whatever it is, whatever the mechanic they're trying to mimic.
CLARA FERNANDEZ-
So it's like maybe you don't like colonial or whatever--
Yeah, if you don't like the-- CLARA FERNANDEZ-
The science fiction part--
The science fiction theme, you could go for this more Earth town building theme.
But if you don't like playing the game itself, switching to the other game is probably not going to be all that different.
CLARA FERNANDEZ-
Yes, yeah, which I think is kind of interesting, and it's like, so [INAUDIBLE].
Yes?
The interesting thing is that in neither of those cases, unlike, say, Ticket to Ride, do these things feel like they're tacked onto the theme?
And in each case, the actual mechanics of the card faithfully represent what they're supposed to be in the game fiction, right?
So I feel that the fact that the themes are essentially interchangeable, or that the same mechanics underlie both games, despite the fact that they have different themes, doesn't necessarily mean that there's something wrong with the theme.
It's just that--
But the meaning is similar--
It's just that the mechanics are flexible, or something like that
But what I mean, the meaning of what you actually do in the games is similar.
Resource management, planning ahead, knowing when you can deploy a certain-- they are about the same thing, even if they have different settings.
So that's probably the connection.
It's not about oh, if you like science fiction games, you can play the Battlestar Galactica game, Race to the Galaxy, whereas I'm guessing that for most people, it's, well, if I like Puerto Rico, or I'm thinking of San Juan, sorry, [UNINTELLIGIBLE] wrong place
Their themes are close enough.
CLARA FERNANDEZ-
But you would play something that is similar.
Does that make sense?
If the meaning of both games is really empire building in some sense, right?
In San Juan, it's got historic themes, and Race to the Galaxy it's futuristic.
But they're both empire building.
The theme is very well defined in both of these games.
That will make sense because both themes are about empire building, only one's in the future and one's in the past.
CLARA FERNANDEZ-
OK, so we have-- this has taken a bit of the time, but these are the foundations of the concepts that I wanted to get across today and discuss today.
When we talk about video games, there's an extra layer, which is video games that are story driven versus just games with a story.
An example I have here.
So this is Soul Calibur 2, as far as I remember--
[INAUDIBLE] Soul Calibur II-- [INAUDIBLE] I think so.
CLARA FERNANDEZ-
I don't know.
[INAUDIBLE] Soul Calibur hoping that it was familiar to [INAUDIBLE] designing.
Anyway, so in Soul Calibur, what's the game about?
Fighting.
CLARA FERNANDEZ-
It's about fighting hand-to-hand
It has some weird story.
CLARA FERNANDEZ-
And it does have a weird story that's like, well, does anybody know the story?
I know that there's something, and there's some reason why Sandra or Sophitia-- Sophitia, right?
I don't know.
I think her sister is Sandra
They're related somehow, they're mother and daughter or brothers or sisters or something.
CLARA FERNANDEZ-
Oh, OK. Maybe that's what she's asking for.
I didn't know.
I still like these games.
But do I know their stories?
No.
But does it?
Eh, not really.
Enough of these.
For some people, I'm sure that they do care about the stories of these, or the stories of the characters in Street Fighter, for example.
Yes?
Well I was going to say for games like this, I actually kind of like the story because it adds-- the game is not the story.
There are two separate entities.
But one kind of adds to the other.
Like Diablo, for example.
There's lots of people that have played Diablo II without really caring about the actual story--
Those cinematics were so cool back in the day
But even though you can pretty much separate the story from Diablo II and people would probably still play it, the story definitely adds a layer of, I am attached to this game.
CLARA FERNANDEZ-
But Diablo, in Diablo, the difference is that Diablo's closer to being a story-driven game.
It's closer to being an art-boutique game, where the quests, you have to know who you're going to talk to, what are you going to do.
I had a friend, we were working in this game service, and we had to test all these role-playing games.
And one of my coworkers was like, oh, I have to play-- it was Baldur's Gate II.
One of the cool ones.
And he was like, OK, I have to test this, and I just have to see that it works.
And he was not paying attention to any other text.
He was just doing the technical testing.
But then he got stuck, and we were supposed to finish the games, and he was stuck, and he was going slower because he didn't know what to do.
I said, well, if you had read what you actually had to do.
Well, for him, it was like, it was just a story!
Well, you're playing an RPG.
You can't leave the story out.
Somebody's telling you that you have to squash I don't know how many arts, be happy to know that you have to support the arts.
You have to know where to go.
So in story-driven games, what happens is that the story's really so tied to the rules and the goals of the game that you have, as you advance in the game, you're also advancing in the story.
That something that, here, when you beat up somebody in Soul Calibur or Street Fighter, whatever fighting game you can think of, there might be a little cut scene that tells us who they are.
One thing that I like about-- is it Soul Calibur?
Yeah.
There's a bit of taunting, like, oh, he beat me, or something like that.
But that is building character a bit, and that means writing.
But really, it's not really story-driven.
The fact that we're getting bits of the story as we complete challenges, doesn't mean that if we ignore the game, we can still complete the game without knowing the story.
Does it make sense?
So this is something that video games are doing that, to a certain extent, some of the board games that you might have been playing don't quite do.
Does it make sense?
We can still think of-- and there might be some people who play these board games still at the mathematical level without paying any attention to what the event that's representing of.
But that's one potential of video games, that idea of having a story-driven game where you're playing a game, you are interacting with the world with a strong-- in the cases of story-driven games, you also have the struggle of a fictional world that you are interacting with, that you are exploring, that you're poking around.
So one thing I wanted to make clear here is that when we talk about storytelling-- and let's try to use the term.
We're talking about the stories, but is it really storytelling?
And the thing is that technically games don't tell stories.
Who's telling the story?
Who's the narrator?
There might be games that have a kind of gimmick of having the narrator, like Prince of Persia: Sands of Time.
When you make a mistake, and he's supposed to be telling what's happened to him, and he's like, no, no, no, wait, when are you guys going to learn, no, that's not how it happened.
But really, most games, again like Baldur's Gate or Diablo, or more recently like Mass Effect, who is the storyteller?
And the thing is that there are also different types of stories.
When we are interacting with a game, what we are doing is performing, and as we perform, we are creating a story.
There's a story that has been maybe prepared for us.
Going back to the Munchkin thing, in every card, we have a bit of the story, and we are kind of putting it together, laying it out.
So the game designers have designed each game, each chunk of the story, what are the cues that we're giving the player?
But then there's also what else this player is bringing to the game.
As I was saying before, there's part that is the game itself telling its story, and the other is what we are evoking, what we are having the player bring to the game.
Is the story happening in that player's head?
So what I'd like to say is that it's not really storytelling, it's about story building, and as game designers, we are designing those little chunks.
We are creating stories that are meant to be reconstructed, that are meant to be together.
So the difference between storytelling and story building basically, when we have storytelling, we have somebody who's narrating, or maybe there's someone who's re-enacting what's happened, so a thing like theater, for example.
But this continuous, it's something that just happens and is not fragmented, which is what story building means.
We have something that has not already happened.
It's communicated one way, and what those events are and what order they're told, that's determined by the author.
If we think about stories and how stories apply to games in terms of story building, we're talking about fragmented stories.
Stories in pieces.
Stories that we have to put together like a puzzle.
And we're game designers.
We make puzzles.
That's one of the many things that we can do.
What those fragments are, how are we giving cues to the player to put together those fragments, or to come up with their own ways of coming up with the story.
In a way, it's a kind of collaboration between the designer and the player, because we rely on the player to put this story together, to come up with their own ways of filling the gaps, to really have a proactive role in figuring out what the story is, and what the order of events is.
But there are-- you guys have seen the games of progression and the games of emergence?
Yes?
You discussed that?
Could you [INAUDIBLE] I'm trying to remember where this-- CLARA FERNANDEZ-
It's like the [INAUDIBLE] in the first chapter, second chapter?
Well it's a seperate-- I think we had actually looked at the individual paper
Yeah, we have the individual, not the book.
CLARA FERNANDEZ-
Not the chapter?
But it's basically the same thing.
CLARA FERNANDEZ-
OK, yeah, because he talked about the same thing in a couple of different places.
So what happens is that at times, there are games that have a specific order of events, and there are some adventure games, like Syberia for example, where you have to go through a very specific sequence of events, and when you get to a scene it's like, you have to do this.
Things like Dragon's Lair, the laser disc DVD.
When you go to a place and you have to figure out what is the action that you have to do, and if you don't do it, you die in horrible ways.
So those are games that set the order of events very specifically.
But most of the games tend to have some more flexibility, and let the players explore the world.
Things that happen in the world are generated by how the player is interacting with the world.
So in RPGs, for example, depending on the RPG, there might be the main quest and the side quests, and there might be flexibility in what order that happens.
Even how a player might solve a specific quest.
In terms of table top, who here has been dungeon master?
OK, a bunch of you
[LAUGHTER] CLARA FERNANDEZ-
OK, so we have a corner of dungeon masters.
So you probably know that when you design a quest, and then you set it up, and you have your players, and then your players come up with a couple of different ways of tackling your quest, whatever they can come up with.
And you have to improvise and try to-- you're building the rails of their living.
It's like they're trying to anticipate what they're going to do.
So we're trying to get that into video games, too, but we don't have the improvisational capacity built into computers yet.
But as you see, it's like the dungeon masters, as dungeon masters, we're responding to the actions of the player.
Does it make sense?
When we're talking about story building, there are games, as I was saying, that are about story building, where the challenge is constructing a story, improvising as you go.
You're given chunks, and you have to be able to be a storyteller.
But you'll play these later.
We've actually talked about this before.
So when we're talking about building the fictional world of the game-- I'm going to [UNINTELLIGIBLE] through this a bit fast.
We have until 4:30, right?
Yes.
CLARA FERNANDEZ-
OK.
So there are also very minimal ways in which we can construct our fiction, and these are lessons that I've taken from digital games.
But if you think about it, many of these also apply to non-digital games.
So for example, we have the title of a game can tell us something about the fictional world, right?
What about board games or card games?
How do the titles build the fictional world?
The board game Mafia Wars is pretty clear about what you're doing.
CLARA FERNANDEZ-
So that's a way.
It's evoking some things, like the type or kind of world on itself.
And also setting the situation.
So here, Mafia Wars is about
Mafia having-- CLARA FERNANDEZ-
There's different factions
Getting in trouble-- CLARA FERNANDEZ-
There's going-- yeah.
There's going to be betrayal probably, shooting down people in horrible ways
[INAUDIBLE]
Any licensed game?
CLARA FERNANDEZ-
Any licensed game?
Yes.
Thought at times, things like versions of Monopoly or Trivial Pursuit where it's kind of like--
Star Wars-- CLARA FERNANDEZ-
Star Wars everything
Risk gets kind of broad.
CLARA FERNANDEZ-
And there are also skimmings of Risk, right?
Yeah, lots
Lord of the Rings.
CLARA FERNANDEZ-
Lord of the Rings everything, too.
Kind of like Barbie
I just feel like there are so many Risks
I know there's [?
God's ?]
which takes place back in like mythological, [INAUDIBLE]?
CLARA FERNANDEZ-
Yeah, apart from franchises, what other titles are--
The Settlers of Catan.
CLARA FERNANDEZ-
Settlers of Catan, which is evoking--?
Settling.
[LAUGHTER]
Building cities.
CLARA FERNANDEZ-
Building, sure
San Juan.
CLARA FERNANDEZ-
Yeah, San Juan's going to-- that's what you told me, too
Chinatown.
CLARA FERNANDEZ-
Chinatown?
It's set in Chinatown.
CLARA FERNANDEZ-
And what do you do?
You build businesses that are stereotypically associated with being in Chinatown.
[LAUGHTER] CLARA FERNANDEZ-
So with one title, it's not only we're evoking the world, but we can also evoke rules.
What do you have to do to here?
So like Mafia Wars, you kill people.
But then, Settlers of Catan
[UNINTELLIGIBLE].
CLARA FERNANDEZ-
But you are maybe playing what some of the rules are, what the game is about, which is what is important here.
And again, Soren Johnson calls attention to the fact that if you're announcing that your game is about something in the description booklet, the people might not always read, or at least a part of people don't read.
You should follow up with that, because it's helping, it's giving cues to the player about how is the story of the game going to build up.
What is the key?
I like talking about giving keys to the player a lot because I come from theater.
It's a very intuitive way of understanding.
It's not about us selling the story, it's about helping the player construct the story.
So I have two examples from video games I really like about how to build a fictional world with a title.
One obvious one in video games is Space Invaders, right?
Space Invaders means we're on Earth and--
Invaders, aliens.
CLARA FERNANDEZ-
Invaders from space, and we'll probably have a fight
Invading from space.
CLARA FERNANDEZ-
Another good one, Zombies Ate My Neighbors.
There's a whole story right there.
So what's the story of this game?
Zombies ate the neighbors?
CLARA FERNANDEZ-
The zombies ate the neighbors?
So what do I do in this game?
Probably kill zombies.
CLARA FERNANDEZ-
So probably killing zombies
Are zombies eating my neighbors?
Or the zombies ate my neighbors and then just left
Maybe the zombies are your neighbors.
CLARA FERNANDEZ-
Yeah, maybe you have to kill your neighbors, too, now.
If there's anything left, because they ate them.
But this is a whole situation.
What do I have to do?
OK, zombies.
Whacking zombies, I'm sure, or running away from them
The fact that you have neighbors in the neighborhood, right?
It's not like a military zombie shooter.
CLARA FERNANDEZ-
Yes, it's not Resident Evil
They looked vaguely like '50s.
CLARA FERNANDEZ-
Yeah, I'm not sure about that.
This is like a Genesis game
I just played a game with an old, like-- [UNINTELLIGIBLE] CLARA FERNANDEZ-
Yeah, yeah, right
Harkening back to the zombie movies of the day.
CLARA FERNANDEZ-
Yeah.
Another great title that I found not too long ago, Earth Dies Screaming.
[LAUGHTER]
Wow.
CLARA FERNANDEZ-
Which is actually about the '70s--
Whoa.
CLARA FERNANDEZ-
The title is from a '70s British movie that I think has nothing to do with the game.
This is like Space Invaders in a way.
You are defending the earth, it kind of looks like that.
You have your space ship, and you have invaders, and you have to destroy it before it lands on earth.
But again, it's very evocative.
It's more memorable than Space Invaders in a way, because it's got-- there's people attacking earth, whoever it is.
And this is going to end badly.
That's another thing, but Space Invaders and things like [UNINTELLIGIBLE] and all those arcade games that do have an ending, you know the ending.
You cannot really save earth.
It's really ominous in a way.
But again, we have a whole.
We have a whole situation is about-- in the title.
It's also very memorable.
I really like this title.
Another way in which we can give cues to players to build worlds-- and I had more of this, but this applies more to video games than to board games, card games, and even role-playing games, is character design.
Because when we are making-- and tell me from your own experience-- but when you're making non-digital games, and even when you're preparing your Dungeons & Dragons campaign, you don't really come up with the characters.
The characters is something that the player brings.
For card games, if you have any examples that are counter examples for this, please let me know.
But character design is not something that you do so much in non-digital games.
Do you have any examples of that?
Well, [UNINTELLIGIBLE] can be.
CLARA FERNANDEZ-
Yeah, in a way, yes
I was thinking [?
Acompora ?]
and other games where you have a character which is assigned to you and often they have back story.
In [?
Acompora ?]
there's [UNINTELLIGIBLE], Byron, or what have you
I was going to say the game where you're trying to-- Clue, Clue.
Where you choose a character and they all have their [INAUDIBLE] and whatnot.
CLARA FERNANDEZ-
The things that I-- As a player, are you really involved with those characters?
I think I can think of one game which is an example where you are, and it's called Android.
This game was released, I think, two years ago now, and part of the-- actually, you should probably play it-- part of the point is that it was trying to include developments of-- resolving characters' personal stories through mechanics in the game.
CLARA FERNANDEZ-
Oh, yeah
So you're actually playing through this choose your own adventure-ish type scenario.
And the choices that you make throughout the game influence what directions your character's personal story arc.
It's a very good game.
But it's interesting, right?
CLARA FERNANDEZ-
Yeah, what I'm trying to get at here is that at times, the characters are-- and again, evoking actions, evoking certain emotions, too.
I think that that detachment with non-digital games-- with role-playing games, it's different.
But again, when you're building your own characters in role-playing, you have an investment when you're making it yourself.
I'm talking more of the point of view of the designer.
Here?
So another [UNINTELLIGIBLE] game again, Battlestar Galactica.
You are assigned a character, and if you know Battlestar Galactica, you're probably going to go for certain characters, because like, oh, that character's hilarious, I want to be him, or, I'm going to be Starbuck, Starbuck's awesome.
And then each player actually also has certain traits, which are characteristic of their-- for the player in the [UNINTELLIGIBLE], like [?
tie ?
], except he's now taller, so there's a certain negative attribute to him if you're in a certain situation.
CLARA FERNANDEZ-
OK. All right, we have-- near the back
I was going to say in live-action role-playing games, like expansion, Assassin's Guild, and stuff, the characters are entirely designed by the game designers.
But you still get the players who take up this character and become involved with it, because all of their actions are based on what the designer chose to make that character.
CLARA FERNANDEZ-
Yep, yep, that's true, that's true.
And one?
Two other games.
Cosmic Encounter is where probably a lot closer to the thousand island kind of situation you've got.
You pick a race at the beginning of the game.
So it's not a single character.
But that's effectively the same thing in that, if you pick something like the Cubans, you have different goals, and you have different powers that you can use in the game.
Whereas if you play a game like Illuminati, instead of playing an individual you're playing a corporation, or actually a [UNINTELLIGIBLE] for society, for the most part, and your actual game winning goals are different, depending on who you pick.
And part of the game is also figuring out what certain characters are trying-- how certain characters are trying to win the game, because not everybody has the same game winning goal.
CLARA FERNANDEZ-
OK. Patrick, last one
In the card game Saboteur, if your role is the saboteur, then it is in the player's best interest to act in that role.
CLARA FERNANDEZ-
To perform like what?
Yes, to be some person to the other players without letting them catch on.
CLARA FERNANDEZ-
Well, awesome.
You see, and this is why, I think, [UNINTELLIGIBLE] I was asking.
Yeah, I think the LARPing one is the obvious case, yeah.
In video games, what we do, we can build character in cinematic ways and cut scenes and whatever, but really, it's very difficult to build a character that is at the same time you, and not you are the player and not the player.
And what we can do is give them cues about the personality of that character.
And one of the examples I wanted to give was Sonic, because Sonic is not really you.
This is always the-- a bit of this trouble in a lot of the examples that you were doing, except for LARPing, where LARPing is a bit more involved, is that it's kind of like you but not you.
Who am I?
I'm the saboteur, because I'm doing this.
In games, we have Sonic, for example.
We are Sonic.
What are we?
We are a blue hedgehog.
And what do we do?
Collect rings
Run.
CLARA FERNANDEZ-
So, run and collect coins, and liberating cute animals
[UNINTELLIGIBLE] CLARA FERNANDEZ-
Yes, that too.
And one of the things, one of the goals of designing a character, particularly in video games, is making it seem alive.
Making it go beyond the functional part.
Like, oh, we are the saboteur, we are whatever character in Clue, for example.
Having that emotional involvement, that is something that we are trying to achieve in video games.
And Sonic does it in a very subtle, very cool way.
Does anybody know what this picture is of?
What he's doing?
When you don't do anything, he starts tapping his foot and looking at you.
CLARA FERNANDEZ-
So it's kind of like, you're not controlling me.
Come on.
I wanna go.
I really have to go.
I have to run, because what I do is running.
So, let's go
Mario goes to sleep.
CLARA FERNANDEZ-
Yeah, Mario goes to sleep, that's right
Starts-- [INAUDIBLE] CLARA FERNANDEZ-
And do you remember any of the Earthworm Jim games?
He does crazy stuff.
He starts-- he flips his head, and he dresses up, like he is thinking, why don't I-- Yeah.
Or he gets an afro wig at some point.
What are you doing?
But it's pulling attention to, who am I?
When you're not controlling me, I'm gonna do it myself.
And is the case of Sonic, it's like, OK, I'm a running guy, but I need you to run, so come on.
And it's very subtle, something that is very minimal, but is building character, but is also telling the player who this character is.
The last thing-- and this is something that, again, video games can do very well-- is something that role-playing games can do very well, or LARPing, but I'm not sure that card games or board games can do this, is environmental storytelling.
It's telling the story in the space as you go around, as you're examining the space and what's going on.
Video games, what they do very well, is construction of a virtual world that we can navigate, that we can explore, that we can interact with.
And one of-- those of you who've been in my classes before, you know that I love the beginning of Bioshock as an example of environmental story telling, because you know what is this world, what has happened before, and they don't tell it with a word.
It's just going to the space and figuring out what has happened.
Another example that I really love is Portal where a lot of the story is told, or you figure it out, as you explore the world.
So what is this in Portal?
So, first of all, what is the premise of Portal, the story?
You're a test subject testing a new device at Aperture Labs.
CLARA FERNANDEZ-
Yes, so you're a test subject.
But very soon, you realize that there's something kind of wrong.
This voice that keeps talking to, kind of warps around, and she goes and comes back.
But there's something broken, and you just go on because you don't know what else to do.
But here, what is this?
Once you start realizing, OK, there's something really wrong, what is this?
This is when you're trying to escape
The hole in the wall.
CLARA FERNANDEZ-
There's a hole--
One of the things is stuck forward, and you can sneak under it or something?
CLARA FERNANDEZ-
Yeah, you go behind the scenes and you realize, OK, so there's something really wrong here, I kind of guessed it, but here we have what looks like somebody's refuge, like where somebody was hiding, and they were writing all these messages.
And, of course, it's very funny, because the famous "The cake is a lie," is not something that everybody has to go through.
You have to find it.
You have to find it in the game.
It's part of exploring the game.
So the reason why I like-- I like Portal for many, many reasons-- but one is because the storytelling is very brilliant--and is not something that we're told, this is what happened.
We figure it out.
by listening to the voice, by listening to GlaDOS, by going behind the scenes.
We are piecing the story together.
Yes?
If you ever pay attention to what's written on the wall in the safe houses in Left for Dead, there's a lot of funny stuff, as well.
And I initially hadn't realized Left for Dead had a story, really.
And then, when you actually pause and read what's written on the walls in the safe houses, there's almost flame wars between different people who run to the same houses and are writing facts about the zombie apocalypse, and saying, oh, that's not true, that town is completely overrun, no, you're totally stupid, I was there last week, and it had these things going on and various two bits of trivia and facts about the world.
And it's very, very clever, and often quite hilarious.
CLARA FERNANDEZ-
Because what environment storytelling is doing, after all, we're telling the story of the fictional world.
That can be part of the goal of solving games, figuring out what is the story of the fictional world?
Yes, Andrew?
Was there a conscious decision to use environmental storytelling as opposed to story building?
Well, you mentioned the difference between storytelling and story building earlier, and you mentioned that that was environmental story telling as opposed to story building.
CLARA FERNANDEZ-
Yeah, and probably is really story building.
So the thing is that environmental storytelling is what people call it.
I'm using a phrase.
But yeah, this is a type of story building.
We're building a [UNINTELLIGIBLE], build it for the player.
But yeah, this is story building
More precisely, you will see talks on environmental storytelling when you go to a conference.
But what they're really talking about, so you know, is just that [INAUDIBLE] involved in research
So maybe the closest to this kind of thing is in flavor text in CCG's and other such stuff.
So if you think, for instance, of Magic the Gathering, then there is a back story behind each of these expansions which comes out.
And that's told through actual fiction, where you can buy a really trashy novel.
[LAUGHTER] But it's also told through little quotes between-- by significant the characters in the fiction, also the images which are presented on the cards.
So, maybe that's the closest thing to environmental story telling in-- CLARA FERNANDEZ-
Yes, yeah.
Magic the Gathering has this world, and it seems to be very easy expand it.
I don't know if there's a bible of the Magic the Gathering world
Yeah, yeah, there's an entire creative department at Wizards of the Coast that's responsible for maintaining the story and making sure it's all consistent and stuff.
CLARA FERNANDEZ-
So it's not really something-- it's almost 4:00, so I have still something, but we can go through it very fast if that works?
If we run out of time, we can play these games on Friday or something.
CLARA FERNANDEZ-

If that sounds like a good idea.
All right.
CLARA FERNANDEZ-
So, we've been talking so far about designing the story of world, designing the story of the fictional world.
What we also have, as I was saying before, the players also construction a story as we go.
Things that happen to the player, things that happen in the world, because they happen to the player.
And again, one of the reasons why I was talking about games of progressions versus games of emergence is that in stories, we have a similar parallel of stories that have been designed for the player, things that they have to do in role-playing games, and I'm talking about pen and pencil-- pen and paper role-playing games, we have the story that as the end we have designed, and then there's what the players are trying to do.
In video games, because computers don't have the capacity for improvisation, what the player has to do is a lot more constrained in a way.
So what are the bosses that they have to-- how do they solve a specific problem, or how do we get over a specific problem.
So I like giving the example of Legend of Zelda games, where we do have a story that we're completing almost in a specific order, because if you don't have certain objects, you can't go to the next stage until you have certain properties.
So even though we have a really nice, relatively open world, a world that we can navigate, we do have a specific order of events that we have to follow.
Adventure games, it might be something like point and click adventure games, again, things like Syberia, things like Myst, where there is a pre-established sequence of events that we have to go settle.
And then, the other extreme, we have games that are closer to these games like Once Upon a Time, or Gloom where the system itself is producing stories of sorts, like in The Sims.
And in The Sims, the fact that we have a way to record our own stories and share them is proof that that's a kind of system closer to things like Once Upon a Time.
That is, the system can generate narratives, and that playing with the system will generate some narratives.
So we have ways in which, as designers, we can provide cues or ways to design the story of the player.
And the first one is the game premise, what is the goal of the game?
Game events, I don't know if I'll let that one, because that was too long.
And we have [UNINTELLIGIBLE] a narrative.
The game premise is who are you in the game, what do you do in it.
It's kind of similar to the title, but in this case, is very tied to who is the player, what is the role of the player in the game?
So things like Cooking Mama, for example.
You're Cooking Mama, so who are you, Cooking Mama?
You're not mama.
Mama's complaining about you.
You are the person who's cooking for Mama.
That's the title they should have, Cooking for Mama.
And then now they're getting pissed off at you because you didn't do it right.
But it tells you basically so you're a cook.
Or what is it that you do.
You're trying to please Mama by cooking something nice and yummy.
Usually, also kind of weird.
A really good example of creating the story of the player by the premise of who you are in the game is this.
CLARA FERNANDEZ-
I love this example
[UNINTELLIGIBLE] CLARA FERNANDEZ-
Because it's like, who are you?
You're a hacker.
And it gives you-- the cool thing about this trailer is it gives you a list of what you do.
This is your story.
You are framing people, you are framing innocent people, of course.
Maintaining people's academic records, building their devices, and all those are activities in the game, but it's also who you are.
That is your story.
Those are the kinds of things that you're going to be doing in the game.
So again, we were talking about Saboteur before.
Well, as saboteur, you already have a series of actions that you associated with the your character.
That is way that we can design the story of the player, at least at the higher level, and give it that cue.
I'm skipping game events because it's kind of long and it's very, very strong.
Two basic video games, and mostly [UNINTELLIGIBLE].
The other example that I want to make here is micronarratives.
And again, all these are more video game things, but micronarratives are little events, they're not even cut scenes, they're just little events that happen as reactions to the player's actions.
In a way, Sonic tapping his foot is a kind micronarrative.
You're not doing anything, come on, I want to go.
Or Mario falling asleep is kind of like a micronarrative of sorts.
This is one of my favorite micronarratives in these video games.
This is the game Jet Set Willy, which is from 1985, maybe, something, middle '80s.
It was for Commodore 64.
Well, it was originally for the--
Spectrum.
CLARA FERNANDEZ-
Yes, thank you.
He's prepared a presentation on the Spectrum, yes.
Commodore 64 and surrounding this world is Z80 platforms in the '80s, and this was a sequel to Manic Miner.
So Manic Miner, you were Miner Willy, yes?
Miner Willy.
And you go down into the earth, and you find a treasure, and you become immensely rich.
So with the money that you get from the first game, you throw a party, and you throw such a big party that your house becomes invaded by the weirdest creatures ever.
There are flying toilet seats, mutant pigs, I don't know
Sounds like a good party.
CLARA FERNANDEZ-
Yeah, such a good party that you get people from other worlds to come to your party.
But the thing was that it was such a good party, you wanted to go to bed.
This is Willy.
This is you in the game.
So in Jet Set Willy, this is the day after, and Willy is is hungover probably after the super party.
And he was in a bed.
In the game, to the left, you have the picture of the bed, and that's your goal, go to bed.
But this is your landlady, who's seen how you trashed the place, and she doesn't want you to go to bed until you've cleaned it up.
So the cool thing, they really want to get-- and then, the animation here-- is that she has her arms like this.
She's holding her arms like this, and whenever you get near her-- which arm is it?-- her left arm, she goes like, ha!
Go.
And it's a reaction to what you do.
With this, it's a few pixels.
It's probably four or five pixels.
And in four or five pixels, this game is telling you this is what you have to do.
Go that way.
This is your goal, you asked me.
You've gotta come here, go that way, and clean it up.
And this is eight bit pixels!
And she's telling you all that with just a few pixels.
You don't really need that much.
But with that kind of reaction, we always want to, again, gives cues.
What is it that the player has to do?
We don't need a cut scene.
With a few pixels, we know what we have to do, right?
And it's a very difficult game, too, for today's standards.
Anyway, so the conclusion here is that why do we need stories?
Why do we need fictions?
What do fictional worlds do for us?
What do themes do for us?
As we were saying, humans have a tendency the story-icize everything.
We are natural storytellers.
Some of what these games like Once Upon a Time or [?
Bloom ?]
are playing with is because we tell other people what has happened to us.
If you complain about something, you're complaining in the form of a story if something might have happened to you.
We understand our experiences as stories, and there are different disciplines that have dealt with this from computer science to psychology.
How we are making stories about ourselves, how we are making stories about our worlds.
So having a story, having a game that has a fictional world in which a story can take place is a way of understanding the underlying system.
As I was saying, if we have a game that is called Mafia Wars, we already have a goal, we have sets of rules, who we are, what is it that we have to do.
We don't need a lot of-- two words have been enough to evoke a lot of that information.
So the last thing, and this applies a bit more to video games, is that by having a story, we're also providing consistency to the world.
When Jesper Juul is talking about incoherent worlds, he's really talking about worlds where there's not really a story, and there are video games that have awful stories, that's true.
But when you're forced, or when you're thinking about your world not only as a fictional world as a simulation, but also as a world that has to produce a story that makes sense, that is going to provide more coherence to the world hopefully.
We have to [UNINTELLIGIBLE].
But that means that you can take the [UNINTELLIGIBLE] for games, [UNINTELLIGIBLE].
And there are things that yes there are complains about, like why does Mario have to realize?
Well, because it's Mario, right?
Is that part of the story?
That's one of those no-man's land issues.
Does that undermine the consistency of the world?
Maybe the anthological level, the level of the world itself.
But really, it's more about making sense about the rules.
What is it-- being able to look at the world in a critical way.
I'm trying to think of an example of-- apart from the three lives of Mario--
He was a plumber.
CLARA FERNANDEZ-
What?
I thought you said he was a plumber.
Oh no, that was in the theme.
CLARA FERNANDEZ-
No, but for example, Kirby.
He eats other creatures and spits them.
Where does it come from?
Why does it do that?
What is the story of Kirby?
He's kind of cute, but hey, I would watch out with him, because he can eat almost anything, and then he just dumps it?
Kirby's a weirdly disturbing character.
[LAUGHTER] CLARA FERNANDEZ-
Yes, if you think about it in terms of stories.
But this is kind of wow
Yoshi, too, right?
CLARA FERNANDEZ-
Yes.
And I love Yoshi
Yeah, and that's another video game character-- it seems like one of the reasons why these arguments have been raised is because there have been these huge piles of inconsistencies.
CLARA FERNANDEZ-
But the inconsistency has become-- I think that when you try to write the story, if that makes sense.
So Yoshi eats creatures and--
Then he spits them out.
Each one.
Depends on which Mario you're talking about, because if it's Super Mario World-- CLARA FERNANDEZ-
I'm thinking of Yoshi's Island
Okay, yeah, in Yoshi's Island, yeah, he poops eggs.
CLARA FERNANDEZ-
Yeah, and is like, hm, and then he throws it, and they're like bombs?
Yeah, he throws his eggs at-- he uses his eggs as weapons.
CLARA FERNANDEZ-
Yeah, but all of those things become evident the moment that we start his story [UNINTELLIGIBLE].
So who is this person, what do they do, why do they do this?
Wow, it's kind of falling apart.
So the incoherence-- and I'm just thinking that I don't know if you guys will really like this or not-- but the incoherence comes from the impossibility-- or not impossibility, but the problems of trying to make this into a story and realizing, well, this has not been-- this has been thought out as a cool game, and I love Yoshi's Island, but as a story, it kind of falls apart.
Yes?
I think one of the really glaring inconsistencies in a lot of RPGs in particular is that in most of these worlds, there's some kind of resurrection card, and there are a lot of plots which revolve around character dying and that being a bad thing, but then you're like, why don't you just use your Phoenix Down or-- why do we care if the human is being assassinated, I've got this thing in my inventory which will bring him back.
CLARA FERNANDEZ-
Yes, I wished that about Aeris like a thousand times, and no, because this guy kills her?
Then what kind of--
The actual conceit there is you're not dead, you're just knocked out.
So it's like smelling salts or something.
CLARA FERNANDEZ-
But there are games like Planescape: Torment where that is incorporated in the-- you're immortal, and that's the thing, figuring out why you're immortal.
They make fun of you waking up, and you go like, oh my goodness, this really hurts, I feel terrible!
And then making that problem into an essential part of the game.
Yes?
So, with the RPG thing, the incoherency is how so many times when you're in these battles, you're doing these incredible things.
Like you're summoning asteroids or jumping in the [UNINTELLIGIBLE].
But then in the game world, you get to a ledge, and they're like, oh, it's a ledge.
I have to go around.
Or it's a gap like this wide.
I can't step over it because of a bush, oh no!
What am I going to do?
There's a bush in the way.
CLARA FERNANDEZ-
But you're thinking diagonals, too.
What?
No, I cannot do diagonals
I think another egregious inconsistency that comes up a lot is the-- and I think it's explicitly mentioned in the reading-- that you're the world savior usually, and on the way, you're going to pillage, steal, kill innocent people, and it's not going to be a big deal.
And even in games where they make the endpoint variable, so all right, you can either save the world to or be a total jerk, and that fixes one problem, they still have the problem that during the game, you can alternate back and forth between being really good and really bad, and all it does is it kind of swings you one way or the other.
It's one dimensional.
So you're sort of just in a spectrum, and if you do five really bad things and then five really good things, you're back to where you started.
And that's not necessarily consistent in terms storytelling
Well, there's also the classic inconsistency of if I don't save the world, everything will be destroyed, but you're still going to charge me 100 gil for this phoenix?
[LAUGHTER]
It's still inconsistent
Or, I have to save the world now because bad things are happening.
Let me go do these 20 side quests first.
CLARA FERNANDEZ-
The whole world is that--
This cat's stuck in a tree!
[LAUGHTER]
There're rats in the cellar, man, rats in the cellar.
CLARA FERNANDEZ-
Yes, but the end of the world is nigh.
Like in Final Fantasy, the end of the world is nigh, but let me go around--
Blitzball!
CLARA FERNANDEZ-
--and get more random encounters so that I can bump up my stats
So that actually reminds me of-- I think it was a web-comic-- that I read around a month ago, and it was based around, oh my god, you're our savior!
You know how to solve it!
Yes, I am, and so the whole time, he's just going on, like, what are you doing?
They're raping and pillaging everything, they're destroying everything, what are you doing?
Oh, I'll do that in a second, I just have to get this really, really cool shield.
And then at one point it's like, why, what are you doing?
It's like, I'm fishing, so I can get this lure, so I can catch another fish.
Why would you ever be doing this right now when the world's about to end, and evil is spreading across everything?
He's like, don't worry about it, I'm on it.
CLARA FERNANDEZ-
Well, it's very interesting.
And this is my last slide, so if you want to-- we can go on discussing
Sure.
CLARA FERNANDEZ-
So at times, the little cues that are building world or building character can also backfire.
So I've been playing one of the DS Zelda games, and one of the nicest touches is that whenever you meet a character, Link is going to look at them in the eye.
Hi!
You're walking, and it's like, hi.
And at times it's super weird, because the characters look at each other.
They acknowledge that they're there, so they feel kind of alive with a very simple algorithm, where there's another character, they look at each other.
It's like, wow, they're alive, cool, and they react.
But then, you go into somebody's house, and they're looking at you and you're like, hi!
[BREAKING SOUND] You know?
You break their pots to get rupees.
And they're looking at you, and you're breaking everything.
And it's just like, this is kind of weird.
They should react, because they're looking at you while you're breaking their house.
And that's, again, is giving a touch that is making those characters alive, but it's also like breaking it, like, I'm still breaking your pots.
I don't know what's better
I think that was subverted in-- no, in another Zelda game, Link's Awakening?
You go into the shop, and the shop owner is always looking at you, so if you take something off the shelf and try to exit, he says, no, you can't do that, you can't steal it.
But if you run around him, he's not fast enough to keep his eye on you, so you run around him until he's looking away, and you can leave.
But if you come back in, he says, you're the guy who stole something, and he shoots you with a laser.
[LAUGHTER]
And then every character in the game calls you "thief" for the rest of the game.
CLARA FERNANDEZ-
Ah, that's really funny
Yeah, it's awesome
I was going to say, Oblivion and Fallout do the same, where everyone watches you if they're within-- if they're acknowledging you, and then you try to take something, they're run up and be like, why the hell are you taking my stuff, or start beating you up or whatever
Or shooting you
Or calling the town guard.
But it's problematic when you're powerful enough and you can take the town guard.
Then it's just really annoying because everywhere you go, there's guys you can kill that are kind of there, and you're like, all right, come on, get out of my way
So there was a Mega Man Legends which I don't know if anybody ever played.
I don't think anyone ever played that game.
But essentially it was like Mega Man, but the RPG style of game.
But it was actually slightly different from most RPGs.
Most RPGs, you just walk around, walking into people's houses, looking for loot.
But you actually couldn't just walk into people's houses.
Their doors would be locked.
And you could knock and see if anybody was there, but it was kind of interesting, because it was actually consistent, in the sense that you couldn't just randomly walk into somebody's house and steal their crap.
CLARA FERNANDEZ-
So one thing that I wanted to ask-- because all these things in video games, after a while, they result in [UNINTELLIGIBLE], right?
Because we have the possibility of having a fleshed out world that we can interact with, that is simulated, and the times where those cracks are showing, where the system is is showing, when it's showing that this is limited, that we've chosen a specific level of abstraction to simulate the world.
But do those incongruencies bother people in board games or card games so much?
I would argue that sometimes it's a little bit of a speed bump to learning what's going on.
Assuming some [UNINTELLIGIBLE], assuming that is incoherent.
It seems like because it's almost impossible to play a card game or a board game without first understanding the rules, unlike in video games where you can just jump right in, that is usually something that can be surmounted if people say, oh, the rules say this, but the theme says that, OK, I understand how to play by the rules, and be able to step over that.
CLARA FERNANDEZ-
Yeah, and I think that that's an excellent point, and when we start playing a board game, probably what you guys do in this class, is you get a new board game.
What do you do?
You pick up the instructions and you're figuring out the rules first, and not the world.
In a lot of video games, we figure out the world first, and then by exploring the world, we're figuring out the rules.
And even if there are games that are giving us tutorials or manuals, who-- I don't know, not many people read the manuals, if at all
While the game is installing.
CLARA FERNANDEZ-
Yes, [INAUDIBLE].
But manuals is not something that-- even though it's going to be the first thing to look at in a board game and a card game.
And even, well, as a player, you also look at the rule book at least to built your own character most times
You mean on PC and computer games?
CLARA FERNANDEZ-
On role-playing games
Oh, are you talking about table top role-play games?
CLARA FERNANDEZ-
Yes, table top
Oh yeah, even table top role-playing games you try to start.
People start by looking at their handbooks.
CLARA FERNANDEZ-
LARPing might be a good example of how-- LARPing, you're just given your
Your character sheet.
You have to read it.
CLARA FERNANDEZ-
Your character sheet, which is very interesting
For role-playing, I know a lot of people who can't really be bothered to learn the rules that thoroughly and prefer just to experience the story and have somebody tell them how to resolve the mechanics and that kind of thing.
CLARA FERNANDEZ-
But it's funny, because when you're building your character in a table top role-playing game, you still have to have a look at what is the race that you want, what is your religion, what is your powers that you want to choose?
Even if you're only playing for kicks, and you don't care so much about the rule.
And I'm the type of table top role-player that prefers the story, and I don't care so much about combat.
But you still have to know the rules even before you start playing, if that makes sense
There are situations [UNINTELLIGIBLE] described where people don't look at the rules, but then they're usually relying on the GM to tell them what's allowed and what's not allowed, which means someone read the rules.
It's no different from someone explaining to you a card game, as opposed to having the rules--
It actually makes the experience a bit like playing a computer game, that the GM does the [UNINTELLIGIBLE]
Yeah, the GM is your computer, and in fact it's more powerful than a computer, because a GM can figure out what you're actually asking.
[LAUGHTER] [INAUDIBLE]
I had a question about the [UNINTELLIGIBLE], actually.
So one thing about card games and board games is that almost everything that gets revealed or that happens in the middle of the game is usually a player mechanics.
Sometimes there's some loose flexibility, like games where they allow open negotiation between specific players.
But most of it's actually in the play of a card, the move of a piece, something like that.
And especially when you're playing cards, you have to play flavor text, especially for cards where you can just play them at any time.
And I've seen micronarratives done-- at least what I think a micronarrative seems to be-- done pretty well in those instances.
In fact, in the example of cards, right, someone tries to do something that might win them the game, but you play this card, and that denies them the win, but it is tied up with some sort of flavor text, some sort of thematic element to that card.
CLARA FERNANDEZ-
Well, it could be like-- in Monopoly, what is the type of card?
Those two types of cards
Go to jail?
CLARA FERNANDEZ-
No
Community Chest and Chance?
CLARA FERNANDEZ-
Yeah, like the Chance.
There's Chance when there's something that happens to you, but there's also--
Community Chest.
CLARA FERNANDEZ-
--the Community Chest, thank you, which is something that you play.
The differences in micronarratives here, what it means is something, it's a reaction to player's events.
It's not something that the player chooses to do.
Those reaction cards, those are also part of the player's narrative, and in a way, the player is choosing, this is going to happen now.
This is what Gloom is about, and these are what these two games are about most of the time.
It's like you're trying to construct a story, and you're trying to tamper into somebody else's story, so it's that kind of reaction.
Yeah, we would have to find a name for it.
But yeah, the micronarrative is referring to something that happens as reaction.
It's not something that the player does
It's often in reaction to what another player is doing.
So it depends on whose point of view you are taking.
So, I've got a Community Chest card, and I'm holding it until somebody else is about to take advantage of a position, and then I deny them that.
So it's still a reaction to what the other player is doing
Well, I just wanted to mention, in terms of micronarrative, I don't know, would this be a possible example?
In the board game that we're designing, there's a computer in the center that you want to hack, and every time someone tries to hack it, it gets weaker.
So it's sort of a response to someone doing an action on it.
Would that count as a micronarrative, or is that more of a mechanic?
CLARA FERNANDEZ-
It seems as an event it's an important event.
It's a change of state.
But I don't know how much of a story it is.
And there's no science to what constitutes an event or not.
It's something that's part of my research, it's something that I'm going to figure out.
How can we identify what's an event that is a rule and is also an event in the story?
Because things like disconnecting the computer, or hacking the computer, yeah, that seems to be clearly an event.
What if we had each time that it becomes weaker, there is something else that you can do.
So by hacking the computer, there is a new action.
You get access to new areas of the game, or you get a new power up.
Or it's easier-- for example, if-- I'm totally hypothesizing about what your game is about-- but if you will have a computer that is controlling the security of a place, whenever you hack a certain part of the system, it's letting you move faster through the board.
In a way, that might be closer to a micronarrative, because it's a reaction, but it's also a consequence
In that case, in Careers for Girls, when you choose to go down a path, is that a micronarrative?
Because the player chooses and then sort of changes the state, [INAUDIBLE].
CLARA FERNANDEZ-
I don't know the choices, but the events that happen, like what happens when you land on something, might be closer.
And again, it's kind of like, it's a micronarrative?
Careers and Careers for Girls, which I both see as really, really appalling-- CLARA FERNANDEZ-
I agree
Seems to be built on micronarratives, the whole idea that everywhere you land, something's happening-- you're granted something.
CLARA FERNANDEZ-
But the choice-- what I'm saying is that the choice itself is not the micronarrative, it's the landing and what happens when you land.
Oh wow.
It's almost time.
Anything else?
I actually wanted to make one last comment about emergent stories, particularly the example of The Sims.
Is anybody familiar with Alice and Kev?
Yeah
Yeah, so Alice and Kev was this really interesting experiment that this guy did where he created two homeless Sims, a father and-- CLARA FERNANDEZ-
Ah, yes!
--a daughter
I think we talked about this--
And just did a sort of a web-comic style display of the lives of these two homeless Sims as she grew up and he got older.
And it almost seemed like it became a role-playing game, where the guy playing The Sims became the dungeon master, who had created these characters, and then the AIs were playing these characters for him.
In particular, there was this one moment where he gets to a part of the story where the daughter has just gotten her first paycheck, and she decides to donate it to charity.
And he keeps trying to keep her from donating it to charity, but she keeps saying I want to donate it to charity.
So eventually he lets her donate it to charity.
And it was almost like it was the character's choice to donate it.
At the end of this blog that he did about this specific event, he's talking about what does it mean when a video game character that you created makes you question your own life choices?
[LAUGHTER]
This is a really interesting thing.
If you haven't read it, you should check it out.
CLARA FERNANDEZ-
Yeah, on what blog?
CLARA FERNANDEZ-
I think it's aliceandkev.com, something like that.
CLARA FERNANDEZ-
It's a story of the Sims 3
Yeah.
CLARA FERNANDEZ-
Any more?
Talking about non-digital games being very involved in your character, and I felt like one game I've played where I've been very involved with a character is Mafia, where there's not a lot of actual information given to you.
People are yelling at you, like, you're the Mafia, the way you smiled at him!
Clearly you guys-- I don't know.
CLARA FERNANDEZ-
So you become the character more by how people see you than how you play it?
I guess you're also assigning character to other people, reading into their actions and things
That's a game where the roles are designed by the designer, right?
Yeah, but there's only two roles.
It's not like--
Well no, there can be a lot of roles
There's a few roles
I've seen websites with a whole list of roles, and some of them are hilarious
Seems good.
In digital games, I feel like there are the games where you have a very defined character, unlike this.
If you have a very defined character in a video game, you tend to get into it more, whereas Fable, your character isn't defined, and changes based on the actions you use.
If you're evil, he becomes ugly and he gets [UNINTELLIGIBLE] things.
But I never felt like I identified with the character at all versus games that already give me a very strong character, and I play a certain way
That's odd because that's completely-- that runs against what you would be trying to do, at least, that's why they're marketing games that are trying to do it, to make you feel closer to the character because it's the reflection of who you are
I think that Bioshock does a better job of that, where you never see the guy.
So you're always just looking through his eyes, and he has essentially no personality, at least, no personality is displayed in the game.
So it's almost as if you are directly interacting with the world
You're given a space to to be--
To actually project yourself into the game, which I think is sort of what Fable was trying to do, but doesn't really do a good job of it.
Because you see him right there, and he's clearly not you
I feel like Torment is a really-- Planescape: Torment has a really interesting take on this, where the character is essentially who it is that you play in that, but there's more to it than that, because there's also back story to the character, which you, the character, have forgotten, and it's revealed to you over time.
But you know more of that character, who your character was back then, than you know about your character now.
So your nature changes over time, and you've gone through several iterations of this.
And none of them is the truer or more correct you
The funny thing about Planescape: Torment, it's kind of like a personality test.
I've been playing it again, and the first time I played through, I was chaotic good?
And I was trying to play different.
I'm still chaotic good.
So I'm kind of freaking out.
OK, I'm really trying to do something different, but I'm still in the same alignment.
Because a lot of your definition of-- and actually, your alignment can also decide items that you can get or things that you can do-- but it's based on your choices and how you tackle a quest.
If you lie, for example, or if you do crazy things.
It's not something-- once you realize what the effect of your choices is, you can kind of control it.
But there's-- playing Planescape is-- it's going to be about you and how your game playing style is [UNINTELLIGIBLE] humanity
Yeah, there's interesting effects in Knights of the Old Republic if anybody's played that.
You can explicitly make decisions to go to the dark side.
I remember when I played it, every time it came up with a decision where I could-- the dark side has all the awesome powers, right?
So I kept thinking, I'm gonna be on the dark side, I'm gonna have all the awesome powers, it's gonna be grand.
Then you get to the decisions, and I was almost uncomfortable with choosing the evil options, such that I couldn't actually bring myself to go to the dark side
We are actually out of time.
But I wouldn't say that-- the options are thankfully all-- or, actually, let's save discussion of Knights of the Old Republic for another class [UNINTELLIGIBLE]
Yeah.
OK, all right, well, thank you.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information see the course materials on the MIT OpenCourseWare website
--to Abe, who is going to give us a guest lecture on art games.
And today's reading, the proctor reading is kind of like part two of a lot of the things that Jason maybe talked about when we did simulations.
If you guys remember, is a quality reading that a [INAUDIBLE] question, where that came from.
One thing that I think was of interest was that there are a couple of responses to that article.
If you scroll all the way down to the bottom of the article that these responses vary [INAUDIBLE] back to them.
And those are kind of interesting, because it just carries on a conversation.
People actually just turning his article into basically a blog post and then write on it.
And they were pretty good.
But a lot of that was kind of the state of art game discussion back in 1999, I think, I think a pretty old article, where they were very much thinking about games have to be simulations.
Games and narratives, blaaach.
Storytelling is like some other mediums' domain.
It's all about the simulation.
Arguably at that time there wasn't a big discussion of an art games movement.
There were other games movements, like new games which were really more about people interacting with you, trying to get games to where people were more cooperative, less competitive, and less cutthroat.
There were other movements about people-- they were probably advertising game movements.
And there was a lot of work in people making months of computer games that effectively made it unplayable, but made really pretty images in the process.
So that [UNINTELLIGIBLE], for instance, where everything becomes black and white and is striped, including the monsters.
There's no way you can see what the hell's going on, but they look pretty cool
Have you seen-- I think it's 96 kilobytes-- there was a competition in full game [INAUDIBLE]-- no it's 4 kilobytes, I think.
It's a 4-kilobyte game in full 3-D, with multiple levels, extremely clever, including the music and everything
Yeah, yeah
Have you ever heard of this or seen this?
There are a couple of those.
A lot of those come up with the demoscene.
Demoscene, which I will put off to some other time--
It's very impressive, too.
It's like very--
Yeah, yeah.
It's kind of this weird thing of computer hacking, really, really fast and really, really compact coding, and visualized statics and music, which goes more like a digital music video.
Only there were some people who added mouse control, and game play into it and made these kind of games.
But that's very different from the art game scene that we have today, or at least what's commonly described as art games.
So I'd like Abe to talk about what happened since then, since 1999
Yeah, yeah.
Hi, guys.
I'm Abe.
I work here at the lab.
I play a bunch of your games.
I've met a bunch of you.
It's nice to be here.
Phillip asked me to come in to talk about, quote/unquote, art games.
I think largely because it's a kind of, I'll say, sector of game development that my work at Gambit tends to intersect with, especially recently.
And I think, too, that it's--
Sorry.
I was choking from--
I think, too, that it's a sector that's a little bit confusing-- and possibly confusing, too.
And so, just to give you guys a sense-- when you asked me about this it actually made me think, like, oh, yeah, cool, I'll talk about this art games movement or whatever, but that of course-- ooh, wait.
I'm locked into some kind of weird mode right now.
Sorry.
OK. That brings up this question, which is, are games art?
And just to give you a sense for how this little talk thing, whatever, is going to go, I'm going to talk a little bit about this question, give you examples of what people have said about this, and opinions that I think are particularly important, particularly valid on this subject.
I also after that want to show you some examples, some historical and then kind of moving forward examples.
I'm actually going to get some of you guys to just come up here and maybe even play some of these examples that I think are particularly important.
And I'd really like it if we can keep this less of a me talking at you and you engaging me as you have questions on this.
Because me blabbing at you as you fall asleep is maybe good for a nap, but less useful for getting anything out of this.
So we'll try to structure this as a conversation, and then I'll try to pull back, hearken back, to any of my teacher instincts and see if I can get you engaged.
But we'll see.
So are games art?
I hate this question.
Let's start off by saying that I fundamentally find it abhorrent, and I hear it all the time.
And it's aggravating to me because I think it's incontrovertibly yes.
I think that you can't possibly come up with any other answer other than yes, games are art, games have art, yes, yes, yes, right?
But it keeps coming up.
And it keeps becoming a question-- and I should say, too, that this is a wholly biased and slanted talk.
This is my perspective on this, and hopefully by the time-- you'll be like, wow, he's absolutely right.
There's no way that games could be anything but art.
Or at least they are art, and then some.
So this guy, who's a little hard to see right now, but he's handily brought up, recently brought into the fore, and that's Roger Ebert of Siskel and Ebert fame.
Or since you guys are, some of you, younger than me, probably Ebert and Roeper.
He had some interesting, controversial comments recently.
And I pulled this quote, "Nevertheless, I'm being convinced that in principle--" I love that he put that in italics. "
--video games cannot be art.
Perhaps it is foolish for me to say never, because never as Rick Wakeman informs us, is a long, long time.
Let me just say that no video gamer now living will survive long enough to experience the medium as an art form."
So apparently we should all be dead already, according to Ebert, because we wouldn't have lived long enough to experience the medium as an art form.
But here's a guy who, even though I don't think he's a particularly good film critic, is a popular film critic.
And he's weighing in on a medium that he doesn't particularly understand.
And I think it's important to look at what exactly is he doing when he makes this statement.
He's in this business of classification, right?
He's in this business of saying, all right, we've got this subject called art.
What can we say falls under that subset and what does not?
So he says things like painting, sculpture, dance, opera, literature, film can all be classified as art and that games are not.
Probably some other things.
I don't know where he stands on comic books.
I don't think he's written about it yet, but I'm sure he'd probably-- I don't know what he thinks about pornography.
Although, since it's in film form, maybe he does think it's art.
It's unclear.
He hasn't weighed in on that yet.
But the interesting thing about this is, this is not new.
This is like a high-culture, low-culture debate that's been going on for a really long time.
And I think it's particularly hilarious because he would use films.
And films had to fight out of the low-culture phenomenon to gain respect as a high-culture medium.
So he's working-- reviewing a medium that was once conceived as low culture, had to work really hard to be considered high culture.
And now he's saying, look, my films are high culture, your video games are low culture.
Which is totally hilarious because in the past 50 years, this has been completely collapsed by post-modernism.
That's a whole 'nother talk altogether.
Maybe high culture and low culture don't even exist anymore.
So Ebert brings us all back.
And you get everybody upset.
And a bunch of trolls are on his website saying really terrible things about him and about what he said.
But he kind of reinvigorated this talk.
And so new scientists, inspired by this Ebert controversy and looking for con tips on CultureLab asked the question, can games be art?
And they sent it out to a bunch of game scholars saying, what is your opinion on this?
Ian Bogost is a very smart guy and says some really interesting things.
This is really why I'm not gonna read the whole thing, but in the first paragraph he brings up things like Fountain by Marcel Duchamp, which is a urinal in a museum, for those of you who don't know about it.
And that was a statement, a piece of art that he was literally trying to confront, what high culture defines as art by taking, literally, a urinal, putting it upside down on a pedestal, entitling it Fountain, and putting it in a museum, which is the pinnacle of high culture.
If you can make it in a museum, you're art.
And so this is something that-- He mentions Yoko Ono's performance art, where she's sitting in a chair, and she has scissors next to her and the audience is invited to come cut her clothes off one piece at a time.
A lot of this kind of stuff.
Andy Warhol pop culture paintings, Mondrian's abstract paintings.
And he says a lot in that first paragraph.
But I think this interesting part that I've pseudo-highlighted down here is interesting.
I'm starting in the middle.
He says if the purpose of art is indeed to force us to see something we thought we understood in a new light, perhaps the most fundamental move video games have made in the artistic tradition is in the very eliciting of the question, can video games be art?
I think that's really interesting.
So the fact that people are even asking this question suggests that video games are doing something that we traditionally conceive of art doing.
That's, I think, a really powerful statement, that we're even having this discussion renders the question moot in some senses, which is I would say classic Bogostian wordplay.
I think I just made something up called Bogostian.
I should tweet all about it.
John Sharp, from the Savannah College of Art and Design, has had a lot of thoughts on this.
He's actually a great person to talk about this, because he studies video games, but he does it from an art history background as well.
That's pretty good and useful.
He talks about some interesting things here.
Similarly to what Ian is doing, he's talking a bit about how video games and art are doing some similar things at times, and yet games have certain affordances that other media do not.
He says at the end here, which I think is interesting, all this poses the question of why are we concerned with giving video games the status of art?
This is one of the reasons why I hate this question.
Are we simply trying to legitimize them?
Does being called, quote/unquote art change the qualities of video games.
Only if we can answer these questions can we move the conversation forward.
I think he's reacting here to the question itself.
Actually, he and Ian both are reacting to the question, Ian saying the question invalidates itself and John Sharp now saying that this question, it's not helping us.
Asking how does calling it art, what does that do for the medium, that's the important question.
Here's some thoughts about the question and some more evidence for why I think the question, are video games art is just quite wrong.
Mary Flanagan takes it as being assumed that video games are art, which I think is actually great.
In Critical Play she doesn't go to great lengths to define video games as art.
She just looks at art and she looks at games and compares them.
She just assumes that this is so.
I think this is a really nice quote from the beginning of her Critical Play book, which is that, "Critical Play is built on the premise that, as with other media, games carry beliefs within their representation systems and mechanics."
This is really important.
"Artists using games as a medium of expression--" Again, good use of italics. "
--,then, manipulate elements common to games-- representation systems and styles, rules of progress, codes of conduct, context of reception, winning and losing paradigms, ways of interacting in a game-- for they are the material properties of games, much like marble and chisel or pen and ink bring with them their own intended possibilities, limitations, and conventions."
It's a great point.
You would have a really hard time making a painting with a chisel and no paint, like you were using marble and a chisel, a you were trying to make a painting.
Two different things.
Both are art.
It's the same thing with games.
Games have rule sets, games have systems, games have win-and-loss paradigms.
They have all these things that are conventions and things that we associate with even our definition of what a game is, which is at times faulty.
But those are our tools.
Those are how we communicate things.
I think her first sentence is really important.
I'm guessing with the stuff that you've been talking about-- them and Gonzalo Frasca-- that's coming through.
I actually see that in your games.
You're trying to-- so, for example I play-tested that FedEx game, and I play-tested the game about marriage.
There's all kinds of meaning in those systems and meanings in those choices that you get that are being communicated.
And that's very similar to what an artist would do with any other medium.
You're communicating a message through those.
And sometimes we do it accidentally.
It's not always explicit.
We make something-- I would say that the graveyard and when you get divorced and you put your divorced person in the graveyard is an interesting choice for me, because it was creating an equivalence between divorce and death, which I think is kind of interesting.
That's just one interesting example.
So Mary Flanagan in her book talks about this art movement called Fluxus.
Before I go into-- any kind of thoughts or questions?
Does anybody else think-- does anybody think video games aren't art still?
It's okay.
Oh, crap, we're gonna fight
[INAUDIBLE] when we were designing art, so our art project now is you have to choose a real-world system and relate it.
That's what we're all working on.
And one of the big things that really helped us get started is, what is the message we want to get across to each set of ablutions?
What is a bad ablution?
Do you want to say that's random, do you want to say that's this?
And how do we express that, which [UNINTELLIGIBLE]
Totally.
And it gets even crazier when you take-- when you try to look at systems that are not particularly tangible.
You maybe looked at it, but Elude is a game about depression.
There aren't exactly tangible kind of road maps the way there are with FedEx systems, for clinical depression.
And yet you're looking at things like emotions, feelings, stuff like that, that we might call abstract, or just not immediate.
How do you communicate those in systems?
It's really tricky
I have friends that talk about all their games which are like more --genic or more abstract.
Does Ebert [UNINTELLIGIBLE] art?
I mean, it's a really good question
What would he say about that?
I think that falls along the same lines as, like I said, does Ebert consider pornography art?
It's a really difficult question.
That's a much larger art history debate, about has high culture and low culture completely collapsed?
Has mass-media proliferation deteriorated conceptions of what really high culture art are, versus low culture.
I don't have particularly good answers for that.
I think just as all of the other media that we looked at, does a Jerry Bruckheimer film count as art?
Just like we would look at that and ask that question, we don't then deny the entire medium of film from the possibility of being artistic.
There seems to be this desire, because the majority of games in our sector-- which I actually do, too, because if you go far enough back-- I should say video games-- Because most video games are commercially intended and involve fairly similar things, does that preclude the medium from the possibility of being artistic?
I go back to that Ebert comment about he says they can't possibly be, or at least he says never is a long time.
But not in our lifetime, it apparently is impossible.
It's a good point
There's actually interesting effects with the game we're designing which is essentially city building and city creation in the time of the Roman Empire.
And we actually didn't intend to instill any meaning in the game.
But once we actually designed the mechanics and stuff, we realized that it was essentially a game about the rich getting richer and trying to control the board.
So it's like there's almost this inherent meaning that came out from just designing the game
Yeah, yeah.
You almost can't help but have it happen in some instances.
It reminds me of a good thing that Brenda Brathwaite put in one of her talks once, which was about the game Civilization, the mod Colonization for sig IV, which is a great mod in which you play one of the European colonizers and you come over to the New World.
And there's indigenous peoples there.
You have to negotiate with them at first, but eventually you just beat them up and take their land, or they just give it to you because you're more powerful than they are.
And she was making the point, there's this great box art, the Native Americans in the corner and the Conquistador in the middle, the Spanish, and the Dutch.
And it's like, if you were a Native American how would you respond to this game?
And I'm like, yeah, I'm totally making America again.
And other people would be like, oh my god, you basically just wiped out my ancestors, thanks.
So it's true, sometimes you falling into meaning.
It's something.
And I think, again, a lot of art does that, right?
And some art did it quite purposely, where they weren't divulging the meaning, they were inviting readings on it.
So, Fluxus.
Fluxus is this really interesting mid-20th-century art-- they rejected the term "movement," they called it an "attitude"-- that Mary Flanagan talks about a bit in her Critical Play book.
It's actually quite interesting.
It really starts out early with John Cage.
Have you guys heard of him, the composer?
He's most famous for this piece of music that he wrote where, I forget the duration,
4'33"
4'33", yeah.
it's 4 minutes and 33 seconds of not, well, not playing
He rests
He rests.
Exactly.
So you'd sit down at the piano.
You'd pull out the music and everyone would hush.
And then for 4 minutes and 33 seconds you'd sit there, possibly with your hands on the keys.
And then after the time, you would stand up, and you'd take your bow, and the audience would erupt into incredible applause.
He was really testing the idea of what they eventually called scores, or event scores.
This idea that music is an art form where you can write it down, and then it becomes performable.
It becomes this thing that someone can engage with.
We don't think of paintings that way.
We don't think of paintings as having a score, having some kind of, how do I engage with this?
It sounds kind of like games, in figuring out how does interactivity work in artistic mediums.
Really interested in indeterminacy, too.
I think you'll see in some of these examples, they're really kind of pushing on the boundaries of what art has been doing to this point.
They call it intermedia, meaning that the more multiple modes that you can use for communicating it was really important to the attitude.
Simplicity was really big, having basic systems and basic scores.
And then this idea of play was really important.
Yoko Ono, it's funny we sort of just attach her to John Lennon, but she was an artist before she was in a relationship with him.
She's actually well known as one of these kind of Fluxus practitioners.
This is interesting.
This is the Fluxus manifesto.
I wanted to put this up because I think you can get a little bit of a sense for what they were trying to go for.
They were taking the word "flux" and putting definition-- but then they were mending it here, so, purge the world of Bourgeois sickness, intellectual, professional, and commercialized culture.
Purge the world of dead art, imitation, artificial art, abstract art, illusionistic art, mathematical art.
I like this, purge the world of Europeanism.
I don't think that's a word
[UNINTELLIGIBLE]
Promote a revolutionary flood and tide in art.
Promote living art, anti-art, non-art reality.
These things sound totally crazy.
I love the, "to be fully grasped," but then it crosses out "fully."
I don't doubt that was completely intentional, or if they wrote fully grasped and were like, oh, this is gonna be great.
I'm crossing it out so you can see that I crossed out fully grasped.
We want it to be not fully grasped.
We want it to be partially grasped by all people.
Not only critics feel this class is professional.
It's just a really interesting thing, and it sort of screams the avant-garde.
They were trying to push against boundaries.
They're trying to figure out what is the edge of the medium and then how can we jump way off it?
Some of the classic Fluxus works were these things called Flux boxes or Flux kits.
Flux kits tended to have event scores.
This is one of the most famous called Water Yam by George Brecht.
And basically this is a box with 70 event scores.
These are an example of these event scores.
It's interesting.
The event scores really seem like rule systems.
They're like the instructions to your board game, except they're kind of whacky.
If you look at air conditioning, the rule is move through the place.
Which is really interesting.
If you encounter this game and you're like, all right, I'm gonna play Water Yam.
And you're like, move through the place.
I think that's pretty characteristic of movement, right, present you with some sort of objective or some sort of requirement, but one that is abstract or nebulous, or just kind of strange.
This one's even better.
On a white chair, a great tape measure, alphabet, flag, black, and spectral color.
In this instance, it's not even clear what you're necessarily supposed to do.
Are you supposed to compile these things on a white chair?
Are you supposed to sit on a white chair and hold all of these things?
It's not particularly clear.
This is the playfulness of this movement.
This is the energy of this of avant-garde movement.
You can see the relationship, I hope, between rule systems and these.
The next example is, I think, an even better one.
But you can see how they're playing around with these ideas.
At least they're incorporating interactivity into these art movements.
And keep in mind, these are being shown in galleries These are being shown in high-art places.
These are being sent out to their friends as works of art.
This one's great.
This is White Chess by Yoko Ono.
It's often depicted on its own and you have to imagine what the rules are.
But I actually think her rules as they are written are actually pretty important to it.
She says, play it for as long as you remember who is your opponent and who is your own self.
Very interesting wordage.
What are the complications with white chess.
Anyone.
Any thoughts?
Based on your perceptions of non-white chess or white-and-black chess?
You'd get into arguing about is that my pawn or your pawn?
Yes, you definitely-- it will be hard to keep track of, I would think.
That's why she said, "as long as you can remember."
Any other weirdness from this?
Who is friend and who is enemy?
Yeah, certainly.
This sort of like, wait, was that my piece?
Was that your piece?
Not only does that complicate your relationship to the other player, but now it's like, to the game itself, it's like, wait, what was the strategy I was doing?
Friendly or unfriendly fire?
I mean, it's really hard to keep track of stuff in chess.
What about the board?
He writes off the king and the queen by mistake because you don't have to--
There's not really any actual boxes on the board
There are boxes--
Yeah, they're all white.
That complicates things a little bit, too, because you need to remember how you were moving
How large is--
This one's really big.
This is like a garden installment of White Chess.
And I think the cool thing, too, this is another example of the movement.
As long as you have the components you could play this game.
You could create this art piece.
I think the intent is not go play my installation of White Chess.
It's that this is the rules, these are the pieces, it can be played.
That screams game, that screams that portability of a game.
Again, this wasn't considered anything but art, except probably by people who turned their nose at this sort of thing.
they've Any thoughts on what White Chess could be about, in a--
Two different sides merging to one.
And so you eventually get, why am I fighting?
Who is my enemy in this situation?
Sure, sure.
Any other thoughts?
Racial relations
Sure, racial relations
It's probably a whole antiwar thing of, like, are we actually destroying ourselves?
Yeah, and that's an interesting thing.
Yoko Ono was a decidedly antiwar activist at the time she was making this in the early '60s.
One thing-- you can't take anything for granted, but the pieces aren't all black, they're all white.
So I think the race thing does play a factor.
The inability to remember who you're attacking and who you are, even.
I think that's actually why I really like her words.
Because, "for as long as you can remember who's your opponent and who is your own self," there's something about incorporating your own self in there, not just saying what your pieces are.
She's not saying can you remember what your pieces are.
She's saying remember who you're playing against versus who you are.
That's very different verbiage.
I think it's very much a game about who are you fighting, anyway?
What is the point of war, what is the purpose of war?
And how does war particularly function?
And in the rhetoric of the time it makes a lot of sense, right?
So that's Yoko Ono's White Chess, which is part of the Fluxus [UNINTELLIGIBLE].
Yes, it's pretty cool.
Actually, Fluxus and artists in this period have a really interesting relationship with chess.
There's like some quote, and I'm gonna butcher it, but it's-- I think it was Marcel Duchamp saying that not all chess players are artists, but all artists are chess players, or something like that.
He is particularly fascinated with chess, and you can see elements of chess in his work, ought to be painted or whatever
He was completely obsessed with chess
Yeah
He played it really hard.
He wasn't necessarily a very good player, but he played a lot
Very into it.
So now I'm gonna move out of this Fluxus movement.
I just wanted to show you a couple examples.
But I think this next one is actually an interesting-- this is a current theme that I'm really excited about, which is called Sixteen Tons.
This is a game I unfortunately have not been able to play yet, but I've witnessed play because it's a game for four players.
This was created by Eric Zimmerman and Nathalie Pozzi.
It's a really interesting game.
It's a little hard to see from these pictures.
But it's this giant circle, I would say almost the size of this room, of corrugated cardboard, maybe even bigger, a little bit bigger than this room.
Inside you've got this array of colored and circles in these pretty hefty steel tubes that have color.
Let's go through the rules.
The first thing you notice is that they contextualize the title, Sixteen Tons, with a quote from the song, "You load 16 tons, and what do you get?
Another day older and deeper in debt."
This is doing an interesting thing that John Sharp, who you can see in the middle, thinks is actually particularly important for games and games that are working this art movement is that they declare their intentionality.
Whether they declare what they're meaning is or not is less important.
To be taken seriously, you take yourself seriously, I think was one of things he said.
And I think this is another good example when you talk about how games maybe in the digital space do this.
But right now they're already priming you for reading the rules in a way that is different than just, ooh, let's play.
Could you imagine if Monopoly had some sort of Marxist quote in the front of it, or something like that, your experience of it would be completely different.
You set up, you put the pieces in this array, you stand on a number and that indicates what color you are.
And this is great, you take out $3.00.
Although you can see in this second picture they're playing what Eric the designer has deemed high stakes.
Sixteen Tons where you pull out three five-dollar bills.
It says, how to win.
You win when the two pieces of your color are directly adjacent to each other.
So diagonally adjacent does not count for a win.
The way you play is, players take turns, starting with player one.
When it's your turn, you say, put me to work.
Players may then offer to pay you one or more dollars in any order.
So anyone can just say, I'll give you this.
You have to accept the highest payment, except if there's a tie.
Because with specific denominations there is very likely to be a tie.
Then you get to choose which of the highest bidders you can take.
The person who pays then tells you what move to make.
And if no one offers to pay you, you get to decide what to do.
And then it's the next player's turn, and you would continue until one player wins.
Just off the top of your heads, having not even played it-- it's tough to critique games when you haven't played.
But just looking at the rules and hearing about it and even-- I love that picture on the left, because it embodies some of the spirit of the play, too.
What are some interesting things about this rule set of that you notice?
You play with money, but you won't win money?
Well, that's an interesting question.
What happens to the money at the end of the game?
Who knows?
Do you keep it?
It's unclear
Very unclear.
That's one of the things that's really great about this game, is that it puts you to into these social situations where now you're playing with real currency that was your own.
And now it's like, what happens to this?
It says you win if your piece-- but maybe you feel like you win if you moved those pieces and you have the most money at the end of the game.
You've got this kind of weird economy in here.
I think the fact that it's not explicit in this rule set is great.
That's what creates this potential for an interesting social dynamic.
Other things that are interesting about this game?
If you're the only person who has money, then you can basically win
You can basically win if you're the only person who has money, yeah.
So it does require a little bit of strategy, too
Technically, from that set of-- could you win on the first turn, because it seems you only have two pieces of a certain color
Yeah, but the right is still the starting position
Yeah, Iknow.
And so on your turn, you could take one-- your piece and have it be moved to the square right next to yours
No
Move to any adjacent--
Oh, you have to move it--
Read the rule in the red there
Oh, any adjacent number,okay.
I didn't read the adjacent part
Yeah, it's any adjacent or empty square
It's actually really interesting
Yeah, it's a really weird-- I also think you're not-- the agency you have in the game to win is not by moving your own pieces, it's by paying someone to do the work for you.
So then we get back to this quote at the beginning, "You load 16 tons and what do you get?
Another day older and deeper in debt."
It's like, what is this game about.
It could be about a lot of things.
And we honestly probably shouldn't dissect it until we've played it.
I think, too, the corrugated cardboard, it gets hot inside of there, if you're not outside.
And that's a really interesting thing, too.
You're in there, it's hot, it's sweaty, you've got dirty money in your hands, you're moving around these sizable-- they have weight to them.
That was really important, too.
This could have been played around a table with go-pieces or something, painted go-pieces.
But these are substantial steel pieces that you've got to pick up and move.
Obviously the song was about manual labor.
So this is a really interesting game.
This was released at the Art History of Games conference last year.
And it did well at IndieCade this year.
It won the Developers Choice Award.
And it's really cool.
I want to try to get it up to MIT so that we can all play it
It occurs to me that if you went by rules where you keep the money you get to the end of the game, you basically could throw the game to get all the money
Yeah, you could try.
But people are gonna know what you're doing.
Because they're going to see you're shiny.
So it's--
I'm not gonna pay you back
Yeah, the multiplayer gets-- the multiplayer, the game gets in it.
Just cycling back to what we're talking about, is this art?
It would be hard for me to call it much of anything else.
So this is one of my favorite games.
It's certainly my favorite that Jason's ever made.
Who's played Gravitation here?
A couple of games?
Alright, cool.
And a bunch of you haven't.
That's awesome.
So I'll get one of you to play it.
One thing I want to point out with this-- we can talk after someone's played Gravitation, a little bit about what we think about the game itself.
But I think this quote is interesting.
He has artist statements with all of his games.
Some of them say more, some of them say less.
This one responded briefly 'cause this was after Passage was released.
And I think you guys have all seen Passage, right?
You can immediately identify the aesthetic similarities because the character is exactly the same.
He was responding to criticism from people about his pixel art, and why did he go for Pixel art.
He says, "I now see ultra-low res pixel art as a kind of digital cartooning.
It stands right on the line between the symbolic and the representational, leaving plenty of room for viewer interpretation.
Why draw sideburns when the viewers can imagine sideburns on their own.
You just need to give them something to pin their imagination on, something to guide them a bit.
Cartoons are perfect for that.
I think it's a really interesting point.
And I think what he's also suggesting here, which is maybe not clear, is that this is doing a similar thing to what that quote from the music does.
It's priming your experience of the play.
You load up this game, you see that it's got this low-res pixel art that we now associate with this kind of art game movement, and you're ready to read this in another way.
If I gave you a first-person shooter and told you nothing about it except that it was gonna be the fastest, awesomest, multiplayer first-person shooter experience, but it looked like this, would you be inclined to read it like an art game?
I don't know.
But certainly in this instance with these kind of little games, this pixel-art aesthetic that you see being duplicated in a lot of places, is sort of priming the pump.
It's establishing some sort of context in which you can read this game.
So who wants to play Gravitation?
Oops, no, that's not what's supposed to happen.
Why did it click on link.
Oh, 'cause I clicked on my screen, not on this screen.
All that does is take me out of here.
Yay
Me not being here, it might be a problem.
Let's see.
Let's see what screen it's logged up on.
One
Hold on one sec.
I've got a mirror on my screen.
I should not be choosing who plays.
Who wants to-- I can't make the choice.
Who wants to do it?
Interesting that it's a version zero.
You have to explicitly make that the case
Anybody.
It's free to download, so if you don't play it now, you can play it on your own
We have a few games, right?
Yeah, we have a few.
We have a few opportunities, so anybody jump over and play
Just talk with Jason, and then we'll do join me in Dubai--
And actually, if you wouldn't mind talking about your experience to play Gravitation as you play it.
Let's play, but live, say.
[GAME MUSIC]
I seem to have a restricted view, and it's falling.
I caught it
She stole your ball
She seemed to like you, but she stole it for the moment.
Now she seems to be playing, which is better
We seem to-- he bats it back and forth and my screen gets bigger whenever I especially do so.
And it's getting brighter
It gets bigger when she loves you
It changes this fiery thing and it seems to be batting the ball more.
[UNINTELLIGIBLE]
You're on fire.
[UNINTELLIGIBLE]
I'm on fire.
This seems bad.
Now I'm really confused
You should go to the furnace
Oh, now you're not on fire.
That makes sense.
Whoa!
Wait, can you bounce the ball up there?
I didn't see anything that showing me how to get up there.
Oh, here we go
Isn't that always the case?
What's that star?
Uh-oh.
Well, now you can bounce the ball up here
Oh, you can jump higher when she loves you
--the size of the screen
Yeah, that's how high you can jump .
OK, so-- I guess maybe bounce it more to get higher?
You're losing her
I'm on fire again.
Oh, but you're gonna be on fire in a while.
She's not gonna love you very soon
Yeah, apparently getting the stars is also good
It seems it goes faster and faster the higher up you get, I think.
Or maybe the more stars you eat
And that's all ice.
Oh, yes.
She seems happier when I bounce the ball.
I guess-- so I don't have a which is I think the start of it because the stars into the fire so that they melt or something.
[UNINTELLIGIBLE]
Oh, now it's a star.
Interesting
Well, there's been [UNINTELLIGIBLE]
What?
All right.
Look what's [UNINTELLIGIBLE] into the fire
Otherwise, he's gonna have a hammer
Yeah, you're dead
The numbers of the stars are all going down.
The ball can't get to you.
It's [UNINTELLIGIBLE]
OK, now I can-- --push the ice into the fire?
And the number on the top goes up every time you-- yeah, the numbers on the star corresponds to how many, like--
You know, fire seems like it would be bad
Yeah
Try to get rid of the ice--
But she doesn't want to play
That's because I got rid of the ice.
He was happy I think
Oh, it's getting icy, though.
It's getting colder.
There's no way to recharge my thing, though
I think you just have to go higher.
This is where you were last time, I think.
Yeah, there
That was really like--
Do you think [UNINTELLIGIBLE] too fast, and your hair's burning, that means you just gave him too much love, and he starts falling really fast
Oh, no
But there's a a counter on the ice blocks of the stars, and you get more points if you get them into the furnace faster.
I think they went down and he started warming up in the furnace.
The warmer the world, the less points you get for de-icing?
Is this about global warming?
I think he's on drugs.
He gets love from this girl, and then the ice and the stars make you happy, but only for a little bit.
Yeah.
Yeah
So if I'm on fire it seems to reduce at a much faster rate.
[INAUDIBLE] said
I think it's about drugs.
I think everything's about drugs.
I mean, yeah, obviously
You can actually catch the purple guy into that cave
Oh, no
It's a really stupid design, going all the way back down there
There's a [INAUDIBLE] on the right, you can fall faster
The higher you climb, the further you fall
Ooh, poetic
Now I probably have a [UNINTELLIGIBLE] stars, so I think they need to play--
She's gone
She's dead.
You killed her
So sad.
And it's not the fire left
Oh, no
And the fire isn't bringing things back
Oh, my god.
You didn't get any stars for her, so she died.
She eats stars
Oh, there you go.
Is she there now?
The fire started warming you again
Nope.
She's gone forever.
You can't play ball alone
When she's gone.
Maybe she only comes in the spring.
Maybe winter's too cold.
So maybe if I wait by the fire
Does is go faster right there?
Yeah, why is it getting bigger?
'Cause he's by the fire.
It's getting warmer
That wasn't working earlier
Now it is
He's at rock bottom
Looks like it's spring
But when she--
How high can you jump right now?
Can you hit the ball?
Pretty high.
That's a lot of jumbos
No, I can't hit the ball
Only she can hit the ball
While you're increasing, you should probably go up and get--
Go up.
Go up.
Go up
Ah, there you go
[INAUDIBLE] stop [INAUDIBLE].
It stopped contracting, too
You're on fire now
He's on fire
Can you go [UNINTELLIGIBLE] high if he's on fire?
Uh-oh.
Whoa.
Four stars
Are you getting all of them?
That might have been a bit of overdose
You totally OD'd on stars just there
Or, or, you can only hold four, and then it retracts and you get four that turn to ice blocks
What?
That was intense
That's a lot of falling
There was so much energy
Wait.
No.
No
You won.
Yay!
Now what you won is a good question
That's Gravitation.
Actually, I'm not even kidding.
That's certainly my favorite Jason Rohrer game.
It's one of my favorite games.
And I think even just the example of you guys playing is great, because you're just shouting out all these different ideas for what it could be about.
There's some clear intentionality in his design with this game, in terms of the system's trying to communicate something that's other than just engagement or fun.
Was it fun?
Yeah
Oh, all right.
Cool.
Yeah, it's pretty good.
Thoughts on what it's about?
I heard a couple people be like, drugs
It's definitely drugs.
I saw tons of analogies
Yeah, it's kind of like drugs
Or love.
Or both
Love is a drug
Your relationship versus whatever you're trying to do for your career
Yeah.
I think it's like the ultimate MIT game.
If I told you that this was his daughter, would it change your interpretations of the game?
Yes
So now, if you know that that's his daughter, what do you think it's about?
Neglect
Job versus family
He says explicitly, before you play, that this is a game of motivation.
He doesn't go too far into telling you what the game is specifically about.
I think playing through it a couple times-- I will tell you the first time I played and I came back down and she was gone, it was very upsetting for me.
I was like, and I don't know if it's just I was ready to play it that way, or my life stage or something like that.
But the idea of he's playing with his daughter, that inspires him, he travels, and then one of these days if you're gone too long, she grows up.
I don't know if she dies, but--
He didn't bring her any money.
Because she had no money, she dies [INAUDIBLE]
So I think this is a really--
What did they say, it's a game about motivation?
It is, yeah
Or she OD'd on drugs
It's free.
If you just Google-search Gravitation Jason Rohrer, you'll find the links to it.
You'll find his artist statement, which I think is really good to read.
Again, with John Sharp's point about declare your intentionality.
Whatever you say what game's about is important.
I think this qualifies, at least for me, as art.
It's doing things that I see other piece of art doing
OK, so Paolo Pedercini-- how many over time?
Oh, good.
Paolo Pedercini is one of my favorite game designers right now.
This game actually is one of his newest ones, and I think it's hilarious and great.
It's called Memory Reloaded The Downfall.
Test your concentration with Memory Reloaded matching game for these unstable times.
History may be different from that which you remember.
Who would like to play Memory Reloaded?
I'm laughing already, because it's funny.
Has anybody played it?
I need somebody who's not played it.
OK, good.
Do you [UNINTELLIGIBLE]?
Sure
Awesome.
OK. You're up
I will click
Don't do it.
Oh, you're on
You're done.
You lost already
--go away.
It's kind of scary.
I could already see what they can do.
You're gonna play a game switch on me.
Afghan Freedom Fighters.
He's gonna be a terrorist later.
Migrant Workforce.
Migrant workforce.
Iraqi Oil.
Afghan Freedom Fighter
I bet if you grab Iraqi Oil before Afghan Freedom Fighter, it becomes terrorist.
Global Inequality.
Peace in Palestine.
Global Inequality.
Global Inequality.
Yes.
Global Inequality.
Yes
All right
I'm doing well.
Ruthless Banker.
Loan shark.
Was Iraqi Oil here?
No.
Universal Health Care.
Man-made Global Warming.
Natural Global Warming.
Ha!
Hey, not cool.
Universal Health Care.
Universal Health Care.
Sweet.
Oh.
One over.
There.
Peace in Palestine.
There we go.
Wait.
Needy Banker.
Ruthless Banker.
Does it matter, the order you click them in?
Ruthless Banker.
What?
What did the other one [UNINTELLIGIBLE]?
There we go.
Nice.
So that means now-- He's never a satisfied banker.
What the hell?
Oh, dear.
Did we?
Shoot.
Better move
This-- [INAUDIBLE] Natural Climate [INAUDIBLE]
Oh, I get it.
That's fun.
Is that the entire game?
Yes, that is the whole game.
Of course, you get different outcomes.
Yeah, that's the idea.
I think you guys have seen some of the Molleindustria's [UNINTELLIGIBLE] You guys saw Every Day the Same Dream?
I like that one
Which is interesting.
Yeah, that one's also one of my favorite games.
This is looking at another kind of sector of art, the activist political statement sector of art is doing that through games, and it's something-- this music too [INAUDIBLE]-- something that he does particularly well with his games.
And I think this one's particularly interesting and kind of funny.
I love the reworking of a really classic game mechanic.
The entire point of memory is your expectation that they would be the same the second time you turn it over.
And when it changes on you, it really worsens that kind of perspective, I suppose imbalance, that can exist when you're not just turning over Toadstools and Bowsers, but actual, interesting things.
I love the crying, sweating polar bear with a beach chair.
It actually cracks me up.
Humor is actually something that shouldn't be ignored here.
If we remember, that was one of the central Fluxus things, this idea of satire.
Humor is important.
So I think Paolo Pedercini's work is really awesome.
Again, for me, it's certainly arcing up.
This game's really cool.
Who's played Love?
A few people who's played Love?
All right, cool.
I'd love someone who has not played Love to play for a little bit.
I don't know if we can play the whole game.
This is actually a game that I really don't like, but I like the things that it does, if that makes any sense.
It bothers me for some reason.
And yet-- it's such an articulate statement, this game bothers me-- but I think it's worth looking at, because he's doing a particularly interesting thing, Alexander Ocias.
I suppose it's a soft C, Ocias.
I have the sound turned off.
I'm sorry, top right.
[GAME MUSIC]
If you say, yes, will he like not teach you how to play?
Don't touch the buttons.
It's like [UNINTELLIGIBLE]
There's two paths
Can you find them?
[UNINTELLIGIBLE].
I'm not gonna listen--
It says travel the lower paths
Who's gonna be able to listen to all this?
What's appearing?
Weird slowdown
You [UNINTELLIGIBLE] disobey him
What?
I can't hit the red blocks now?
Disgusting.
[INAUDIBLE]
Just give us not only art, it's all so [UNINTELLIGIBLE]
Patience.
Yay
Those are red
I know.
That means they can't touch him
Don't do it.
Don't do it, dude
Do it
Do it
Do it
You're just gonna have to go back to the part that was just hard
You get red blocks when you disobey
You still can't touch them
So if you disobey, it hides from you what things actually are.
Or if you obey, they're getting the victory
It's so funny.
Just that little--
What are the controls for this?
You should disobey
What?
Can you no longer jump?
I'm just screwing around
He's [UNINTELLIGIBLE]
Oh, I thought that was bad
This game is well aggravatingly tuned, because it's difficult enough
Are you doing that intentionally?
I just like to
It's actually a pretty long game.
I encourage you guys to-- it's pretty- it's all flash-based
What's the message?
It's a really good question.
Let me just jump in and go back to that place where-- thank you for playing.
Thank you for playing
Disgusting
You are a changed person because of it.
So the question then becomes what's the point, which already we're in a position now where we're asking questions like that.
Which is, if you were playing Modern Warfare II, you might say what's the point, and be like, to blow a lot of stuff up, right?
This is the response he had in one of his interviews he gave after his release and said, I wanted to get people to wake up and think about the games they are playing.
Much of the industry appears to be pushing in the opposite direction, as we'll have tutorials, holding your hand from beginning to end in subtext, screens for pointless exposition.
It is treating players like idiots.
It is absolutely infuriating.
When you think about it in terms of the tutorial, if this is the tutorial for playing the game, it's pretty obnoxious.
But it's not that much different than the kind of tutorials we encounter in a lot of games that just basically hold your hand and tell you how to do everything.
I think he's really confronting that.
I guess if I were to say the reasons that I don't like this game, I think the control scheme is actually, it's a weakness.
The game is so hard that, I mean, you saw you couldn't even get to the things that he was trying to communicate.
It gets weirder, the further along you get, I can promise you.
It's just you have to get really skilled at doing it.
There's kind of like a play barrier for entry on it.
It's also, for me, I think dare I say, it almost gets too abstract for me at certain points.
Clearly he's trying to say something, but there's nothing I can latch onto to understand what it is.
That's entirely his prerogative, but for me, I'm just like, what?
I have no clue what this is about at all.
So that's a little tricky, not fully grasping.
Clearly I'm not-- I'm partially, so it's like the perfect Fluxus piece of art.
OK, so I've only got a couple more to talk about.
I've got about 15 minutes.
This, I think, is actually one of the most important games we can talk about today, which is Cow Clicker.
Who's played Cow Clicker?
Yay.
This is by Ian Bogost-- that's the guy we saw earlier.
And he said, Cow Clicker is a Facebook game about Facebook games.
It's partly a satire and partly a playable theory of today's social games, and partly an earnest example of that genre.
So this is kind of like a classic sort of art thing to do.
You take something that you're going to satire, and you make a version of that, that maybe does something differently, but it's fundamentally what you are satiring.
It's like the Andy Kaufman comedic kind of style.
You just assume that kind of character or role.
It's something that I think is like joyfully subversive in doing satire.
So hopefully I'm logged into Facebook so you can see.
I haven't clicked recently, so you guys can witness one of my clicks.
[UNINTELLIGIBLE] Facebook is down.
Facebook is canceled forever
Yes.
[LAUGHTER] It actually has no friends.
It's also possible that the wireless-- the wireless in that corner is kind of bad
Should I switch to Google Guest?
I can actually, before we can see it, I can talk a little bit about the sort of interesting controversy that Cow Clicker is creating.
It's making money.
Let me try Google Guest.
I'm not connected.
How about now?
It seems to be working better
You're still not--
The last part of his statement is about an earnest effort at a game.
It is fundamentally a Facebook game.
You can see I've got 5 mooney and my next click is in a minute and 40-- ah, man, I missed one of my clicks.
But maybe if we wait while I talk about it you can witness one of my clicks.
Oh, wait, I actually have some clicks left.
I've got one this week
That's in two hours?
Yeah, it looks like that's when I get my next one.
So basically you can click your cow.
Why can I not click my cow here.
But you can click your cow, and then you can-- oh, it's because I'm on scroll?
Anyway, you can click your cow, and then you can use your mooney to buy new cows that are fundamentally the same thing as other cows, just with aesthetic differences.
So plain cow.
These are the cows I have.
I have a plain cow and a steel cow.
I haven't been playing a lot of Cow Clicker.
If I go to buy a cow, I can buy a new cow.
So you can see lots of different cows.
They cost you in game currency
Paisley cow
Holy cow.
Oil cow [UNINTELLIGIBLE].
It's got a BP logo.
Cute.
Like the [UNINTELLIGIBLE] cows.
That's pretty good.
So there's lots of different cows.
You can buy cows, you can give cows, you can invite a cow, there's cow rankings
You can buy a cow
So you can see, mooney's actually pretty inflated at this point.
You can get 10 mooney for 50 cents, which is pretty good, 1,000 for $10, which is awesome.
And collect a lot of cows.
You can trade cows.
And you can go to your friends barns and see how many cows they have.
And you can click on your cows when you make your appointment to actually visit a cow.
So I think you can see how he's poking fun at things like FarmVille things like Ravenwood Fair, which I'm obsessed with now, actually, these kind of games where all you're fundamentally doing is clicking and acquiring an in-game currency that's you can then use for in-game goods.
But then, of course, you reach this barrier where you don't have enough in-game currency and you use moo currency.
Of course the interesting thing is there are 500,000 people playing this.
I don't know if it's that many.
It's a lot.
If you go to the Cow Clicker front page it'll tell us.
It just [UNINTELLIGIBLE].
Where does it say how many players there are?
Ranking, probably in ranking.
He's milking the cow clicker.
You can see this person's-- Monica's--
Is the cow--
Is that their all-time clicks or this week's clicks?
Is there a--
Here we go, yeah, yeah.
12,000 monthly active users.
That's 500,000 in a business [UNINTELLIGIBLE].
But that's a lot of people.
And he's making some interesting money on it.
And I think one of the things that was important that he did is there's an active forum where people can talk about it.
So it is a little bit weirdly controversial, 'cause he made this game that is poking fun at Facebook games.
And yet it is a Facebook game and presumably making some money, too.
But I think it's really good.
This is the kind of art that's also important, where it's taking what we do and purposely looking at how is it constructed and how it's set up and forcing us to kind of be introspective about the things that we do and the way that we engage.
So this is a kind of subversive game design that I think along the lines of Paolo Pedercini's work and even Gonzalo Frasca if forcing us to look at how we interact.
And Cow Clicker-- So that's that one.
And I've got like 15 minutes.
Phillip asked me to talk a little bit about some work that I'm doing in this space.
So I'm gonna talk to you about a game, it's still a work in progress.
It's tentatively entitled Before and Now, and it's by me.
And the idea from it is it's inspired by Fluxus boxes and those Flux kits.
I'm really interested in physically tangible games.
I don't know why I'm all of a sudden obsessed with holding something.
But I was at IndieCade and I heard again-- and this is the question that drives me more nuts than the are video games art question, although they're interestingly related.
But it's when will we have the Citizen Kane in video games?
It drives me absolutely nuts.
I think 80% of the time people who say that have never seen Citizen Kane.
I don't really know that.
But I heard it for the umpteenth time at IndieCade.
I wanted to throw the television out of my hotel room window.
And I was going nuts.
And I was like, all right, what is Citizen Kane about?
I don't want to spoil it.
Who's seen Citizen Kane in this room actually?
All right, good.
So I see Citizen Kane and it's about this very interesting idea about presents and mementos and happiness in a very kind of real way.
And the temporality of happiness, like are you happy now?
Were you happy then?
Do you remember being happy?
When is happiness?
And so I was like, well, I'm going to make a Citizen Kane in video games, but it's not gonna be known for being epically innovative with camera movement.
It's going to be about the things I think Citizen Kane are about.
So it's basically in this form, although I'm rethinking what kind of box it's gonna be in.
But in this form it's a box.
And inside of it are various objects that start out as-- the starting objects of the box are the things that have personal significance to me.
One of the rules of the game, in fact, I think the most important rule of the game is this one.
Rule number one, you may permanently exchange an object of personal significance for any object of equal value.
So when you're playing this game, you can take something that belongs to you, and you can exchange it for anything in the box.
And of course, the interesting thing is these objects in the box are of varying degrees of interactivity.
You can [INAUDIBLE] so you can draw something on the Etch A Sketch.
The rules are kind of veiled at this moment now, where's there's like a couple other rules that are causing to reflect.
This is actually an extension of Yup on Word which is a project we worked on in the summer, which was a game that as you're playing it's a platform where you're typing, and the game stops and asks you personal questions.
What I really liked about that is it was doing something that I was referring to as a mechanic of reflection, where the activity of playing is reflecting, is thinking, is contemplating.
It's not interactive in a very active way as we think of it, it's more reflective.
I liken it to the moments in chess where you're planning your move.
There's not a lot of action in chess.
But it's a lot of contemplation in chess.
So if it's personal-- what if that were personal.
And so what I didn't particularly care about with Yup on Word it inspires you to reflect, but it doesn't do it with tremendous grace.
It just asks you a question.
It just comes up out of the blue and and says, hey, when was the last time you lied?
Sometimes you're surprised by that, and that's a good effect, but it's also not subtle.
So I'm trying to explore how can we be subtle in causing reflection.
So there are a few things here.
The value question is interesting.
How do you know what the value of the object is in the box?
Do you think of it as monetary value?
Do you think of it as value in terms of significance to the person who put that in there?
Why is it significant to the person who put that in there?
What is this thing?
You can look at it and reflect.
And then when you leave a trace of yourself in this box, the next person who comes and plays is going to see that.
And they're going to wonder what was that.
So I'm also kind of interested the organicness, the kind of never-ending, interminable nature of a game that carries the effect of the person who's played it on and on and on.
The next iteration of this that I want to do-- actually this was an idea that Jason Bean gave me was to use an apothecary chest.
Because one of the things that we play-tested it here with-- I left it out for two weeks and people-- you probably saw it.
It's right there.
People were engaging in-- this is a photo with objects that people had left in.
So it was really fun for me after two weeks to go into this box and be like, what is all this stuff.
What got left?
I was actually surprised the Etch A Sketch was left.
And then I was like, what is this stuff?
And to give you one really good example, one person at the lab put in a Nintendo DS flash card, which was really a wrapped DS flash card.
So now there's this kind of next level.
The game has changed with another level of interactivity, where it's like, if I want to get at what was the significance of this object to this person, I need to get a DS and play this.
So I open up the-- it's got folders, and it's got a copy of the game Lemming.
It's got some music in it.
So now it's kind of like you've uncovered this next layer.
And it's like the interesting traces of the people who played this game are now marked onto this box forever.
Marley even said to me-- I was talking to her about her play after she had done it, and I was like-- She told me she wanted to take the instructions.
She wanted to take the rules.
That's one of the objects that she could take out of that.
And I told her I had thought about that actually when I was designing it, that someone might be like, ha, I'm going to take the rules.
And I was going to put in a rule that said you can permanently exchange for an object that's not the rules, so that the game could persist.
But I didn't want to artificially impose that.
If someone takes the rules because that's what they want, and now the game has changed.
And maybe someone comes along later and modifies it.
And maybe they put something in that makes it a whole new, different game.
And I really liked the idea that you could encounter this game and that it's gonna be in this state of flux, I guess you could say.
That's what I'm working on now.
And like I said, I think one of the things I didn't like is that all of the objects are accessible all at once.
So if it's an apothecary chest with lots of drawers, this was Jason Bean's idea, then you have to do a little exploring.
And maybe the rules are in one of those drawers.
And maybe you see the rules first.
Maybe you see the rules way after you've looked at a lot of stuff.
And now you've seen all this stuff and played with it all and you really like something.
And now it's like, ooh, what do I trade?
And yeah, I also am trying to find a way to give the box a history as well.
I've contemplated some ideas of putting in some kind of clues that you could then, if you felt, if you thought about it or looked it up, you might be able to find a story to this box.
Like if you did an Internet search you might come up on a fake website-- well, it wouldn't be fake, it'd just be a website-- but a website with a fake story that tells you a little bit about this box, and then it gives you a few more hooks into what the meaning of this game or the inspiration for this game was.
And so this is one of my projects that I'm working on right now, like I said, I'm interested in essentially.
But you can see where a lot of the other things that I was talking about are inspiring this work.
And so we come back to that question of, are games art?
Of course we only have five minutes.
I don't know if you guys have administrative stuff afterwards.
But this is, like I said, this is my viewpoint.
It's certainly shared with a lot of people.
But it's not guaranteed truth.
This is an opinion that I have just shared.
Any thoughts you guys have about this subject or these ideas or these games?
Anything at all?
Yesterday's football game?
[UNINTELLIGIBLE]
Well, cool.
I hope this was useful.
I hope it was helpful.
I think if, at the very least, it causes you to continue to think about your own games that you're developing and how meaning can be embedded in the play that you might not even realize is there.
And also just to count the possibility space.
I really am excited by-- I'm discouraged by the safe choices that a lot of independent game developers make and excited when some game developers really push on the edge.
I don't think we have a clear understanding of what the rules are or what even constitutes a game.
Or what degree of interactivity is appropriate for a quote/unquote game.
And I think some of these designers are really testing the waters of the medium, you know, what can we really do with this?
Where can we go?
How edgy and satirical and avant-garde can we be?
I think that's always good.
'Cause I'm a revolutionary, crazy person
Crazy person
Hope 7 played yet one more this year, which were some of the games that Abe worked with the team on over the summer.
Those are all on our GAMBIT website, just until 2010 and as yet [UNINTELLIGIBLE] which we're booked about this summer.
This sounds like tri-season will be digital work
Yeah, and please play them and give us feedback, too, right?
Because that's the other thing.
These are not perfect articulations of game as the pure art form.
We're constantly evolving and trying to improve, so any feedback you can give us on this stuff is greatly appreciated
Do you still have that box, like somewhere?
Yeah, yeah.
It's in my office.
Yeah, totally.
I have the box and the other part of it is when I do the second version of it in the apothecary kind of chest-- The one thing that's important to me is that each of the-- For the prototype I just sort of threw some reasonably meaningful objects in there.
But I want it to be populated with objects that are personally significant to myself or symbolic to myself.
And populating that box at the outset with meaningful objects I think is important, because once it's released into the world for play that it's constantly changing.
So that first instantiation of the game is the one that's about me in some sense, and about my experiences and then it becomes the world's to just play
So the way that your game kind of reminds me of is I'm pretty sure I heard somewhere that some person had a pen.
He started with a pen and kept trading--
The red paper--
No.
This is a different one.
Not the red paper clip for a car.
I think someone started with a pen or something and kept trading it.
And over the course of a year or two, he'd got-- From that initial pen, he was able to trade for a house or something
It was [UNINTELLIGIBLE].
And that was sort of an interesting-- I think it was on eBay, too, or something like that.
And that's even using a pretty well-designed game system-- 'cause eBay is totally a game-- and using a game system and subverting the system itself to do something really interesting.
Yeah, that's a really great-- it's a game inside of a game.
I love that.
It's kind of like the Cow Clicker, like making a game inside of the game that subverts--
Are you familiar with the concept of geo-caching?
When I first saw-- the game you were talking about using [UNINTELLIGIBLE] reminded me of it.
So initially what it was, was when GPS's first came out, people were like, oh, hey, this is really cool technology.
We should think of something fun to do with it.
So what they did was, these people started hiding empty tins of little trinkets and stuff on various hiking trails or other locations.
And you would look up the coordinates on your GPS finder, and you would find this box.
And then you'd trade trinkets, and you get them.
And you keep traveling to these other--
Yeah, and in that sense this whole worldwide game becomes-- There's some shared ideas with some location-based games that do some really interesting things with that, whether it's alternate reality, but not even with the narrative.
There was one I was reading about that was, come out and play, I think, in New York City at one point.
It was Cruel to be Kind, it was called.
It was a game where you don't know who's playing, you don't know who's on what team, but you get missions and you get a weapon that's a complement, like to kill someone who's on the other team, mistake them for a celebrity, or something like that.
So you're using these random acts of kindness-- you know, helping somebody across the street-- and you don't know, it's in New York City, so you don't know if someone's playing the game.
So now all of a sudden you're wandering around doing really nice things for people on the chance, on a hunch that you have that maybe they're playing the game.
And maybe they're not, so you've just helped somebody across the street, which is totally nice.
And it comes to this massive conclusion where you just have these two giant teams who meet up in a location to "nice" each other to death, basically.
There's a lot of really, really interesting stuff, especially in the location-based, around.
Big games are really cool.
There's a lot of people doing really neat stuff with big games
I think in particular with the box and the geo-caching idea, and talking about the paperclip-for-a-house thing, it's really interesting how people value things.
In particular, you have this very vague rule, and it just says something of equal value.
And what does it actually mean?
Like what someone is willing to trade for something else
Yeah, those are the kind of implicit rules like you see in Sixteen Tons, where it's like, what happens with the money?
Jason Bean really pointed it out, 'cause he was thinking about value when he was doing the trading.
And it's such a good word, because value is such a different thing to different people.
Someone might see the flash card for the DS, which has a very clear, higher monetary value than an Etch A Sketch.
And yet, maybe you played with an Etch A Sketch when you were a kid, and all of a sudden it floods back these memories that you have.
There's also a component to it that I didn't really talk about, which is the sacrificing of an object.
And I think, too, that we tend to be beholden to our material objects too much.
In giving it up, in sacrificing it to this game so that you don't have it anymore, if it's something of personal significance, you're letting go and being able to say, OK, the memory is one thing.
I don't need to-- And that's how it relates a bit back to Citizen Kane, which is a movie you should see, because it is good.
It's everything they said it is, depending on who they are.
Cool.
Thanks, guys.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information, see the course materials on the MIT OpenCourseWare website
[UNINTELLIGIBLE] and we will spend the next couple of weeks grading them.
But that was pretty fun on Friday so [UNINTELLIGIBLE].
We've been talking these past couple of weeks-- we've been talking the past couple of weeks about serious games, games for education, games as an art form, as a means of expression.
And we're going to be continuing some of those discussions later on.
But the whole idea is to sort of give you, to sort of prep you, for a situation where you're going to make a game for a client.
And for this last assignment, you're actually going to be making a game based on a problem that Lisa is going to be talking about.
GUEST SPEAKER 1: My name is Jim [UNINTELLIGIBLE].
I'm a clinical psychologist.
I work at-- I was at Beth Israel, I'm just starting at Brigham and Women's Hospital, one of the Harvard mafia hospitals.
And my area is using interactive media for clinical training but especially for treatment of clinical problems.
So we make computer programs to help treat clinical disorders in psychology.
GUEST SPEAKER 2: It's my turn to stand up.
I'm [UNINTELLIGIBLE].
I'm a psychiatrist at UMass.
I trained in the same place like Jim used to work, Beth Israel.
And my focus is primarily clinical.
I'm an attending psychiatrist in a variety of settings, inpatient, outpatient.
And what I brought up to Jim and then Phillip was a practical question that I will try to present, which is helping psychiatry residents deal with angry, agitated, and potentially [UNINTELLIGIBLE] patients.
Later this is a significant and largely unknown issue.
Certainly the research is picking up, and we don't quite understand the extent of the problem.
And but I'll say more.
I'll save it for our presentation.
Thank you for coming here
So when they contacted me, there was the usual interest in trying to make some sort of digital piece of software to help the training residents in learning how to deal with violence in mental institutions.
And immediately the first thing that I thought, well before we jump right into digital, let's try doing something in analog first.
Let's try prototyping something first.
But it also occurred to me that, in the real world of game design, there's actually a lot more jobs in game design outside of the game industry than inside the game industry.
There are a lot of companies out there that are basically looking for solutions.
Some of which may be games.
And you might end up working in some of those capacities.
A lot of folks in advertising for instance are involved in games.
A lot of folks who are interested in improving teaching methods, or improving simulations, working on-- trying to think of an example-- public awareness.
And people who are also now working in website design and in various online services, who are trying to think about how to make their experiences a little bit more game like.
There are really really good ways of doing that, and there are really really bad ways of doing that.
And for the most part what we're hoping is that, if you end up getting one of those jobs, you're going to be able to bring some of your personal insight as having actually done game design, as opposed to just copying what games have done, and just assuming, well, it kind of looks like a game, therefore it is a game.
You guys know that now.
So this is kind of get your feet wet, in terms of designing something, designing for somebody else's needs.
The important thing to keep in mind is, who is going to be playing your game, or who's this game intended for.
And in what context are they going to be playing your game.
One of the particular problems that you're trying to address, what's the part of the system that you're trying to simulate.
And what kinds of insights do you hope people are going to get out of that.
The problem they're going to talk about it huge.
If you noticed I uploaded actually four readings to the class website.
Only one of them is required, but all of them are useful.
I will be uploading more.
These are all reference material for the team to use however you wish.
Because the problem is so huge and vast, you don't have to solve the whole problem.
You pick one thing that you're excited about, that you're interested about, and that you think that you might have a good strategy.
Do the usual brainstorm.
We're going to be going through brainstorming and team selection pretty much just this week.
And try to identify some small part that might be addressable with a card game or a board game, or I'll talk a little bit about role-playing games in a couple of weeks.
So with that being said I'm going to turn it over to you guys.
GUEST SPEAKER 2: OK so we'll try to take turns to talk about violence in psychiatry, in particular.
So as Jim has pointed out, violence is a large issue.
What I'm going to try to focus on, as I mentioned earlier, rather than engaging with psychiatric patients, especially psychiatric patients on an inpatient program, which is where the question for me came from.
And obviously violence can be looked at in a variety of ways.
And I think in one of the handouts that I sent to Philip, Violence in Mental Illness, they talk about the biological, psychological theories that the social dimension of violence and how it can be understood.
If you look at the human development, and no matter what theory you consider, either a traditional drive-based theory like Freud or Erikson's stages, each seems to define a certain developmental goal, and often, achieving or achieving in a faulty way that goal can create potential for aggression and sometimes violence.
A little child or an adult who's dependent or needs are frustrated may try to takeover and get those met, in one way or another.
So what I use as a [UNINTELLIGIBLE] metaphor for the origins of violence is a communication breakdown.
At some level, something doesn't happen right between the person who gets angry and those around him or her.
So in a most generic way, basically communication is less about who says what to whom, than in what channel, and with what affect.
The breakdown can occur at many levels.
Again, this is the generic form.
I hope I'll [UNINTELLIGIBLE] what I mean.
And especially what can be done at a practical level, let's say when you are the resident on call at night.
And the staff calls you up and says so and so is getting out of control.
She's raising her voice, threatening, pounding walls, wanting to get out, refusing medications.
How do you approach such a person?
And again, the intervention can happen at different levels.
And actually one goal of the training, or potentially the game that you might consider, is what to do.
How do you establish when do you engage fully, by yourself, with outside help, when intervention is minimal and you just bring pens with you, restrain the patient, use pharmacology.
So it's a continuum of interventions.
I'm going to just focus on what to do when you can actually engage the patient.
Again, continuing the metaphor is what Walter Fisher has described as this type of narrative, rationality, as opposed to the traditional rational paradigm that defines people as thinking beings that actually do make decisions and base their actions on evidential reasoning.
Or rational people.
We analyze the situation.
We come up with the best possible outcome.
Walter Fisher talks about that fact that we are all storytellers, that we try to interact and understand others based on the narrative that we have about ourselves and about others.
My assumption, and again, this is just the introductory metaphor if you want, that at some point violence can be caused by a disruption in this narrative.
Things don't make sense anymore.
We feel misunderstood.
We feel the victim of an injustice.
And that those principles apply to somebody who is struggling with a mental illness.
Where things have a much narrower window of opportunity than for people who are able to flexibly adapt to the demands of reality.
So we all know who this is, and the reason I put it up-- and I have to admit I haven't looked into the copyright, I know there are longer and longer copyright rules for using cartoons, so you'll have to forgive me.
My point is that this is an angry person.
And as someone who is communicating.
I think we get quickly and clearly the fact that his emotional range may be intense, but it's fairly narrow.
It doesn't cover like a spectrum.
And the communication is fairly simple.
He's in distress.
He is imposing his point of view, his space.
He could become very aggressive and assaultive.
I think that the language is very simplistic.
How would we approach the Hulk in trying to calm him down and change his color?
That's the end of my introductory metaphor.
And please, feel free to interrupt, jump in, and make this presentation as long or as short as you want me to.
There are many slides, but I'm not going to go through every single line.
I'm just using this as a stimulus for the conversation especially from your perspective and from your needs.
As I said, I'm a clinical provider, I'm a psychiatrist.
I deal with patients all the time, many of whom are angry or potentially angry or potentially assaultive.
And I may do things after so many years instinctively.
But sometimes I have to think through, OK, this is the situation, how do I do to maximize the benefit for the patient I'm treating and to keep myself safe.
I've been assaulted just once, in 15 years.
It was like in the movies.
A VA patient, former Green Beret, who got upset with the system, and picked up a very heavy armchair and lifted it above his head.
And we established eye contact then.
I think he wondered, what should I do with this puny little doctor?
Should I squash him or not?
And at the end, I guess it was a last, you know, split second moment, he decided to throw it to my side.
A large section of the wall came down, so I was looking at it wondering, well I could have been that wall coming down.
He ran out.
It took him a while-- for people to catch up with him.
And two cruisers full of police officers to subdue him.
So I guess, no communication there.
He was just subdued.
What's the magnitude of the problem?
I think it's substantial.
And actually more than 10 years ago, there were some isolated reports of how significant the problem was in terms of the trainees, and especially psychiatry trainees, being the target of verbal and physical violence.
On this first page I talk about medical students in one study.
Numbers are relatively small.
But again, the study is 14 years old.
A survey of psychiatry residents showed that assaults or threats of violence are clearly one of the most difficult events in their training.
And some were wondering introspectively if this is the right field for them.
Actually some students may wonder reading this literature whether psychiatry is the right field for them to go into.
And as you see in the fourth article, almost two-thirds of psychiatry residents were assaulted at least once during their training.
And obviously like with any dangerous situation, the risk of post-traumatic symptoms can be substantial.
Either acute stress response or longer lasting issues.
The idea of improving their training and their ability to deal with the situation has multiple benefits.
Those who know what to do are never or almost never assaulted.
Not only that, if they happened to be exposed, the likelihood of developing post-traumatic goes down significantly.
So I don't think I need to demonstrate that point any further.
The assault rate in a more [UNINTELLIGIBLE] relatively recent study, 10 years old, pretty old, you see that overall the number of assault rate was 30% to 40%.
Men tended to be slightly more physically assaulted than women
Can you define what assault means, exactly?
What qualifies as assault?
GUEST SPEAKER 2: Assault basically means kind of laying hands on the person.
And the assault resulting in an injury, of any kind.
Skin being kind of broken through or--
Just holding the clinician wouldn't be an assault?
GUEST SPEAKER 2: Well it would be an assault because that kind of grabbing action could actually result in injury.
And I think it would be probably very limiting to talk only about physical injuries, especially since the psychological consequences are often the longer lasting and much more significant.
I think what the surveys have shown also is that how unprepared trainees in the mental health and outside the mental health field are actually.
Often this is reduced to a few lectures, especially during orientation.
And then they are thrown into the lion cage and they learn on the job.
And many of these findings are introspective.
Many trainees felt that well, I didn't say anything, because I thought I was supposed to toughen up and bite the bullet and move on.
Or this is part of the job, I chose this, it's my fault.
Or maybe I did something wrong.
Maybe I was the one who made it so the assault happened.
So a lot of reasons that were just, again, not looked into.
Fear of being scrutinized.
How come that I'm the one that was assaulted, and not 10 trainees.
How I'm going to explain this.
I was so frightened that I cannot even remember clearly what happened to me.
So there are a variety of reasons why people who are in training-- and again, I'm focusing on a again a very small segment, a specific group of trainees in medical and psychiatric training.
But I could easily think of other situations of people who are exposed to the potential for violence and where such training would be helpful.
Up until now, and for the most part the data that I was able to collect, and most people in looking at this have been able to collect is fairly limited.
There are certain limited number of programs.
Methodologically the research is quite faulty.
You can just talk about how limited the generalizability of the findings are.
And often people have talked about what happened in the past.
So clearly the recall bias is a major limiting factor in understanding what happened.
Also, as Jim asked me, so how do you define assault.
One of the biggest problems with most research in this field is that defining assault, aggression, is done in very idiosyncratic ways.
It's hard to compare different studies.
So this is a compilation of what people in different areas have thought about in terms of improving the psychiatric training.
And as you see most often, in the last few years, the idea of using simulation to expose the trainees to a protected environment that has clearly defined boundaries and consequences, and basically that can be repeated almost in identical fashion in order to gain proficiency, this coming up more and more.
What many training programs are doing is to use standardized patients that use actors.
And that of course would behave in a programmed fashion.
They make get up to you and make fists, but hopefully they'll not get so much into the role and then they're punching you.
GUEST SPEAKER 1: I think this is the real challenge, is that there's different levels of learning.
You can learn kind of a list of facts, things to do if your patient starts to get agitated and threatening.
And you can learn how to take a test on it and forget it the next day, especially if it doesn't come up for you right away.
And then there's the question of internalizing it so you actually have not just kind of a working knowledge but really understand what you should do in different kinds of circumstances.
And that's where the idea of role playing with live role plays for the trainees-- it could be residents, and mental health providers-- can really be helpful so they have some sense, some semblance of this kind of training.
But the tricky things with these role plays is it's not easy to ensure that everybody receives high quality access to well trained actors.
Here at Harvard Medical School, they probably do.
At UMass, they probably do.
But there's a lot of places.
And if you expand beyond psychiatry to every mental health center in the country, every caseworker who is going out to visit a patient at their home, or social worker, they're definitely not getting this kind of training.
And so the ones who really I think you said before are really well trained are the nurses, actually.
Because they get a different kind of training that's really pretty effective, it sounds like.
But the mental health care providers often have no training.
And then they're the ones who are kind of in a closed room with the person who can become really violent.
Especially if it's at their home, or an inpatient unit.
GUEST SPEAKER 2: Jim was making reference to that psychiatrist at MGH who last year was stabbed by a patient in the outpatient clinic.
GUEST SPEAKER 1: You all hear about this last year?
Can you just describe the situation?
GUEST SPEAKER 2: Well, it was late in the evening, when most things like this have the potential of happening.
This was a neuro evaluation or [UNINTELLIGIBLE] in the Bipolar Clinic at MGH.
Basically he was angry from the beginning and was able to stab repeatedly.
An off guard person who just happened to walk by.
GUEST SPEAKER 1: It was a police officer, right?
GUEST SPEAKER 2: Yeah.
He was one of the security people for that particular clinic.
He came and he had a gun.
I don't know how often off-duty security people carry their guns.
But he opened and killed the patient.
So he was charged.
Eventually was cleared.
For using excessive force.
GUEST SPEAKER 1: What happened to the psychiatrist?
GUEST SPEAKER 2: Well, the psychiatrist was admitted into the hospital and required extensive surgeries.
She was stabbed in the chest and in the neck.
It was a fairly gruesome-- she survived and I think she's back at work.
But this is an extreme case.
And I'm afraid that many of the things that we talk about, the principles that we may come up with, may not necessarily be so helpful, other than thinking back at a level of intervention.
It is quite conceivable that for somebody who charges at you, you won't be able to talk him down, you won't be able to use any of the nice de-escalation principles that I'm going to talk about in a few minutes.
But, figuring out what do, I think it means [UNINTELLIGIBLE]
You mentioned that nurses have a different training system for this, and it seems to be effective.
What is that training, like what is different about what nurses get taught?
GUEST SPEAKER 2: Well, there are some, in most states, there are some regulations that try to implement some level of training or are designed specifically for patients who get out of control.
And that's not only psychiatry.
Emergency room teams have probably the most highest rate of assaultive patients, across diagnostic departments.
Often psychiatric patients, especially patients using substance abuse, will get out of control.
So if you want to become an emergency physician, or if you work in the field as an EMT or another capacity, you'll have to deal with these situations.
What makes the nurses somewhat better trained is that they often are the frontline staff dealing with out of control patients.
And I think that's probably pretty sad, because physicians should be equally, and trainees, physicians and trainees should be equally well trained.
But on the nursing front, there has been some progress.
In Massachusetts, the Department of Mental Health has some mandatory training.
But there's a lot of red tape there, but there a lot of helpful aspects, including aspects related to stop attacks.
And I wish I could have presented a recording of a training session for staff.
When I went to it, I thought it would be something like a very spectacular kind of kung-fu, how you do it.
When in fact, the emphasis is how to keep yourself safe, and believe it or not how to keep the person safe, the person that you're blocking the punches, the person that may be trying to strangle you, or keep you in a stronghold, or scratch you, or rip your hair.
Again, even when you're trying to be intervene or you're trying to protect yourself, and keep the other person safe.
No punching, kicking, no kind of spectacular steps.
But again, that's also I feel could be an interesting topic for a simulation.
How do you interact with-- especially since again I'm a totally kind of outsider in the field.
Having kids, they were asking me the other day about Microsoft's Kinect.
And I thought that was really interesting because the level of interaction with computer generated interfaces is very interesting, and certainly would be a safe bet to be attacked by somebody from the
It sounds like the nurses get better training because the states require them to have better training.
GUEST SPEAKER 2: That's correct.
That's a part of it.
I've been trying at UMass to suggest that not only the staff physicians get exposed to the training but even the psychiatry trainees and other trainees, especially in emergency medicine.
They should all really know about it.
GUEST SPEAKER 1: I suspect some of this have to do with the benefits of being in a union.
So a union can demand that nurses receive this better training, where the physicians don't have a union.
GUEST SPEAKER 2: It's also about numbers.
On any unit, you have six, seven nurses for one physician.
And the nurses are there in shifts around the clock.
So their likelihood of being exposed to an out-of-control patient are much higher.
Usually on these units people intervene as a team.
And as a team leader that tries to apply so many of the principles I'm going to talk about.
And then if there is need for seclusion or restraints, then that has to be done in a coordinated fashion.
And things are much clearer, as I said, on the nursing front.
Maybe because they are doing it all the time and because there are some tight regulations, because they're unionized.
And at least half of the literature really comes from the nursing direction.
So that's a tangent
Just want to ride off the Kinect comment that you were making.
We will be talking about a little bit more about live action games so that's kind of how you prototype something that's going to be a motion gesture computer game.
You don't obfuscate.
Actually having to write code to do that, that sort of body recognition, is kind of annoying and expensive.
But you can test off a lot of ideas just by having two people play with coding.
Safety becomes the biggest concern for the players as well.
It's kind of interesting in this particular case that you are trying to teach someone how to prevent damage to both parties.
That actually kind of helps you with the design a lot actually.
That's only one problem.
You'll go into more details about the kinds of things to prevent it escalating into actual conflict.
GUEST SPEAKER 2: And the idea is basically to do your best to not end up at that particular extreme of direct combat, which is not-- So ideally you don't have to go there.
And as I said in 15 years I've been assaulted once.
I'm sure I was lucky.
People think that my eyebrows are intimidating.
So don't tend to get very [UNINTELLIGIBLE].
Again I'm going to tell a few, you know, tell you about a few pearls and a few tricks that I personally use.
It may not be helpful for everybody.
Over time, you develop you own style.
The idea is to be able to make an assessment and try to take preventative steps.
And I think that's the secret.
And this is what I hope better training may be the user simulation and gaming would be helpful.
Now I'm going to just, I'm going to jump into talking about prevention and [UNINTELLIGIBLE] but identifying how you assess and how you think about the potential for violence.
And these are some known risk factors for aggression.
Obviously the history of violence is the top.
How recent, how frequently, those people become assaultive.
What's the pattern of escalation.
What are the associated symptoms.
What happens before somebody becomes assaultive.
Why is this important?
Because you can do, and probably should do before, you engage with the patient an assessment at that level.
And it will help you figure out how deep to get involved, how likely you are to [UNINTELLIGIBLE] the patient.
Versus saying, oh, I don't need this, I need help, let's go in as a team.
So that-- that makes it important.
People often talk about violence and a rise in tendencies.
Young people tend to be more assaultive, and there's no surprise.
Also older people who have dementia and get more disinhibited and more likely to become reactive.
I always like to tell the story about a patient with dementia.
I was moonlighting, and he swung at me as I was passing.
And then I went to talk to him.
Well, I don't know, you just passed by, and I was upset, and you were the closest.
So I thought I would try to-- So there was no way for me to predict that.
It wasn't a serious thing, it wasn't a predatory type of assaultiveness where somebody plans.
Doesn't like you, goes after you.
A colleague of mine was attacked by a patient.
The patient watched when people were not around, went into the doctors room, locked the door, and started punching.
So that's a totally different animal.
So the distinction between impulsive, affective if you want or the type of aggression and the predatory is essential to keep in mind.
In some subcultures, violence is not allowed certainly much more frequent.
People of a lower socioeconomic status may be more likely to become assaultive.
Patients of a limited cognitive ability, especially when institutionalized, are way more likely to become assaultive.
And then it's many, many diagnoses increase the likelihood of assaultiveness.
The top are psychotic disorders, like schizophrenia.
Mood disorders like bipolar.
I told you about what we call cognitive disorders, especially dementia and confusional states.
Patients that have had traumatic brain injuries or that have had epilepsy.
GUEST SPEAKER 1: There's a real problem with Iraq or Afghanistan veterans, when they have had an IE explode under their vehicle.
And they had a brain injury.
Now they have a lot of disinhibition about violence.
And they can just go off sometimes very unexpectedly.
GUEST SPEAKER 2: The frontal lobe exerts a sort of inhibitory influence on our ability to function.
And that allows us to plan, to think ahead, to sequence events.
This is the executive function of our frontal lobe.
When it's impaired either in a trauma, or after surgery, or being in an accident, or becoming demented, that ability to operate in that fashion may be impaired.
Violence is a possible side effect.
Substance abuse are at the top of the list, because of the disinhibition, both during the intoxication phase and during the withdrawal people tend to become very agitated and assaultive.
There are certain personality disorders.
So when we think about the percentage order, we have this enduring maladaptive patterns of relating to you and to others and to the world.
The obsessive compulsive personality disorder likes to have things organized, planned.
Nothing-- even going on vacation is an exhausting task.
Nothing has to happen randomly.
The grandiose personalities really need this kind of feeling coming from all directions.
They need to be in the limelight.
They use people, not as real people but as people that provide sustenance to their own self esteem.
Two type of personality disorders are particularly likely to promote violence.
The antisocial and you may have heard the term borderline personality disorder, which is basically at core has significant instability and emotions and interpersonal relationships and identity and basically their lives are a mess.
Often they feel abandoned, they feel put down.
They may get briefly paranoid and often they become assaultive.
So again, I'm throwing a lot of terms and definitions at you.
I don't want to make this a lecture about psychiatry.
But these are conditions that I think are helpful to think about
When people with delusional disorders like schizophrenia get violent, is it usually because they are frustrated about something or is it because they suddenly have some sort of delusion about their clinician that involves requiring them to take some sort of action that is violent?
GUEST SPEAKER 2: I think both.
I think even people that have a psychotic disorder have the right to feel the victim of an injustice or feel frustrated or-- many of the reasons that I talked about could happen.
But in particular, I think the other point is equally important.
That they may act on a delusional conviction.
Or their psychosis may also have hallucinations.
And command auditory hallucinations are known to often be the forerunners of a violent attack.
You talk about me and say things.
Or the voice tells me, I'm going to go after you.
And the person feels hopeless to resist that.
I discharged a patient, was doing great.
Next day he comes back.
Went home, the voice told him that he needs to hurt his mother, otherwise he'll get disemboweled.
So he started punching her and then chased her with a knife.
She had to hide in a neighbor's house until the police arrived.
There is an interesting threat control override here that people who have looked at violence have prescribed as being sometimes the proximal cause for violence.
People feel directly threatened.
And they also have this overpowering urge that they need to act on that they cannot resist.
It's a controversial concept.
I'm not so sure why we should give this particular association a higher status than other delusional convictions about the neighbors watching me or planning to kill me, or putting together a variety of clues that my friends are really trying to get rid of me and then I need to protect myself.
But you are absolutely right.
What's interesting even though patients with 20 psychiatric disorders may have their reasons to become assaultive, in the overall bigger scheme of things, when you look at actual violent assaults committed in society, they're responsible for a tiny one.
So it's an interesting way of looking at numbers.
Yes they could become violent, but the reality is that their numbers, the actual violent acts, are very small.
But anyway this may just be a kind of statistical trick
You're talking about the number of cases of violent episodes compared to the general population.
GUEST SPEAKER 2: Exactly.
That's exactly the problem right there.
And then there are various acute findings when you examine the patient.
On what we call the mental status examination.
Obviously, hostilities, suspiciousness, agitation are all associated with violence.
I want us to focus one minute on the so-called aggressive attributional style, how do we respond to stress when we have a perception of threat.
This is the threat override delusion that I mentioned.
I think that's quite relevant.
What happens in conditions of uncertainty, and many of these patients may have to think in a distorted fashion.
I was talking at the beginning about the communication breakdown.
You have the layer of the mental illness that shrinks your options.
You have a particular situation to deal with that makes you even more likely to become assaulted.
And this constellation of cognitive distortions may make it more likely for you to become assaulted.
Neglecting the base information, which assigns a high weight to certain events that otherwise are low frequency, attending only to facts that would confirm your assumptions, I think--
So if the psychiatrist's office calls to reschedule an appointment, the person assumes that this is because the psychiatrist doesn't like him.
This person doesn't want to treat me, they don't like me, they don't want me to be their patient because they rescheduled me.
GUEST SPEAKER 2: Ignoring facts that could disconfirm your hypothesis.
I just talked to the psychiatrist who was nice to me, but still the fact that he called and canceled is the overriding explanation that he doesn't like me.
Or focusing on common events just because you remember them, which is an interesting way of explaining.
The fact that you remember them and it brings them up in the rack.
And other disconfirming evidence is totally kind of ignored
In spite the assumption that the train on the other platform always seems to arrive first.
Because those are the ones you remember.
GUEST SPEAKER 2: Or the garden variety, the grass is always greener.
And I talked about medications and street drugs.
Some are more likely to be associated.
Alcohol just by the sheer volume.
And especially because withdrawal is so violent.
It's a significant issue.
The intoxication phase, your level of coordination, your perception of facts, it's just they're so off that you are more likely to lose control.
Antidepressants have an interesting impact, because in people that have an underlying tendency to swing in their moods may actually all [?
blend those ?]
things.
You may have heard of the news of antidepressants promoting violence and suicide in adolescents.
And while statisticians are still scratching their heads to figure out the exact numbers, the truth is that mood is very unstable in adolescents.
And often initial depression may turn out to be something else.
You give somebody an antidepressant, call him or her for followup in six or eight months.
In the meanwhile, lots of things could happen.
And I think in those cases, when actually those young adolescents committed suicide, it was not so much about antidepressants causing it.
Probably the antidepressants made the symptoms worse.
It was very poor followup and poor diagnostic.
But anyway, this is getting too technical.
I included this last minute.
This is an interesting summary of a nationwide study focusing on violence risk assessment.
Let me just take two minutes about useful measures in assessing dynamic to the violence.
People have looked at violence in two ways.
They have put together all those factors that I presented initially.
And they have looked at the perspective of those who have become violent.
And they compiled the history, clinical risk assessment scale.
The problem is that when you're dealing with somebody, those scales, even though they may give you a good image of what happens in the long run, they may not help you deal with the immediate situation.
So clinical scales have been used in [UNINTELLIGIBLE].
One developed in Norway, the other one, the dynamic appraisal, developed in Australia.
These are the items for the HCR-20.
And it's almost like a repetition of what I said earlier in terms of risk factors.
This is the historical risk factors.
And these are validated, because they have been used for more than 15 years.
The clinical scale is the dynamic part of the HCR.
And then the risk management scale is trying to project in the future what you have to do in order to ensure safety, what could go wrong.
GUEST SPEAKER 1: So the value of these scales is kind of sizing up the likelihood of this person coming violent, in the absence of other information.
The best predictor is probably what they're actually doing behaviorally in front of you.
They're reaching for something heavy, I take that in and weigh that heavily.
But in the absence of anything else, you know that the people who meet a certain profile are more likely to be violent as a psychiatric patient than others.
GUEST SPEAKER 2: This is a dynamic scale that looks at what happens in front of you, as you pointed out.
You're considering confusion or debility, how boisterous and how verbally threatening the person is.
The most recent one is the dynamic appraisal of the situation aggression that has seven factors that seems to be the best one [UNINTELLIGIBLE] assessment or prediction of violence over the next 24 hours.
And this is an interesting psychopathy and an index.
Psychopathy has an interesting history.
I think the current version that we all talk about is antisocial personality.
But basically refers to people with minimal capacity for empathy, whose moral dimensions is fairly primitive, who engage in stealing, cheating, violence, and don't have a lot of moral drawbacks to do it.
It seems to be associated with violence, not surprisingly.
And this is actually a factor analysis that identifies those two dimensions.
Now just at the end I want to talk about the de-escalation principles that I am actually now trying to incorporate in the standardized patients, because these should be fairly easy to at least present, and then maybe hopefully to teach the residents how to use in terms of dealing with agitated patients.
So actually many of them are used by staff.
How to start interaction with a violent patient.
How to position yourself.
Within arm's length.
Probably at 45 degrees.
For two reasons.
One, in case you're attacked, to offer a much smaller surface of the impact and also to allow you a way to escape, you have to.
Not obviously to block the exit door.
Stand over if the person is leaning down or in the bed.
Never turn your back.
What to do with you your hands.
Never make fists, or kind of keep your arms crossed, or behind you, or in your pockets, to kind of appear as nonthreatening as possible.
How to regulate the eye contact.
If it's too intense, it can become bothersome.
If it lacks completely, then people could conclude that you are not really interested or involved.
GUEST SPEAKER 1: The thing that's important to keep in mind with a psychiatric patient is, well, a couple things.
One is that mental health patients are much more likely to be the victims of violence than actually to commit violence.
And this is really the outlier, the unusual case where the psychiatric patient becomes violent.
But obviously it's an important situation.
It does happen frequently enough to be a real concern for the providers, for their well being.
Basically there's different levels of what you have to understand how to do.
One, how to protect yourself, and then two, how to protect the patient.
It's almost like if your child who's 11 years old comes at you swinging, it's really different than if you're getting bumped on the subway in terms of what you're going to do.
Now the person who's sitting in your office might be a lot bigger than your 11 year old.
But you kind of get the idea.
You're not trying to punish them, you're not trying to inflict any more harm on them than is absolutely needed in order to stop the situation and for you to get out, or let them out.
GUEST SPEAKER 2: Yes.
And that's important, and that's somewhat counter-intuitive.
I'm just thinking, I'm trying to think in terms of potential simulation, and targets for our training paradigm.
I think an important aspect is for the person who is going to the training to understand himself or herself.
How what's the reaction when you are threatened.
And how to continue to think on your feet.
And take decisions that are not based on a flight or fight or tend and befriend attitude.
This is not about dealing with a common situation.
This is when you have to maintain your ability to remain therapeutic and helpful and to minimize harm to yourself and certainly harm to the other person.
And the truth of the matter is that many people in these situations react the way that they've always done.
I gave Jim the example of a social worker I work with, who every time when she is confronted by a patient, she gets angry and she gets defensive and tells things to her that I find are hurtful.
And they are just adding gas to the fire.
I need to intervene because you feel that it's almost like, you know, I'm there to umpire.
OK, time out.
Stop.
We need to do something else.
And I try to explain to her.
She just doesn't get it.
Or can theoretically figure it out but doesn't do a very good job at it.
When we are angry or frightened, our behavior, the way we talk, the way we track, changes.
And those things can be major triggers for the other person.
GUEST SPEAKER 1: If you're staying really calm, and the other person's escalating, that's a real dampening force on that other person from escalating even more.
And it's really, really hard to do, to stay professional.
And recognizing that this aggression is really a medical symptom.
It's like if they are bleeding on the floor.
The difference obviously is that it could actually hurt you, the provider.
But you need to recognize it as a medical symptom that needs to be dealt with as opposed to taking it really personally, or you being violent back.
GUEST SPEAKER 2: How do you appear.
How do you-- obviously, being calm, centered, self assured is always helpful.
It's the message that you're conveying.
Often if you respond to anger by being fearful or angry yourself, obviously these will just escalate.
Or if you're fidgety and pacing, and gesturing yourself, and wondering, so what do I do now, I think the other person would clearly pick up that feeling.
Touching most often than not is not a good idea.
Even though instinctively we would like to calm the other person down or wonder what it is.
This is how we learn to show empathy and that we care, we want to help.
Well, it may not be a good idea.
And then this could be a topic in itself.
Active listening would be the most-- if you want the buzzword, in terms of the verbal de-escalation principle.
Often when I'm called, and many of my colleagues are called in such situations.
I introduce myself.
I'm telling the person no matter how agitated he is why I'm here.
And I'm trying to understand.
I may not respond or react to the person's curses or threatening behavior.
Of course I try to maintain a safe distance.
And I continue my assessment.
Can I deal with this, or I need help, or I need to stop and find another way.
And I give that person feedback about what I see, how he or she makes me feel.
I feel you are threatening, I feel uncomfortable, I think we need to stop.
Do you want to talk?
What are the issues that you're dealing with?
And I also explain and set limits on what is acceptable and what is not acceptable.
Often, that combination works.
I listed pharmacotherapy and seclusion restraint because they are part of the process, and I think I would like people who are trained for this to understand that those things are available.
Any questions so far?
I've just got a couple of comments on [UNINTELLIGIBLE].
GUEST SPEAKER 1: Yeah.
We can do that.
And also I wanted to suggest and see what you think about this, if someone would like to play the role of a patient, I can meet with you briefly and just kind of prepare you to do that if anybody is interested in taking the role of the aggressive patient.
You'd like to do that?
Why don't you and I step out for just a couple of minutes.
And then [UNINTELLIGIBLE]?
[INTERPOSING VOICES]
So there's a couple of things that have been brought up already, immediately seem to be possible places where you guys can think of what you're doing to do your project on.
There's identifying factors.
There's a whole bunch of different schemes and different metrics that people have been using.
All the way from known personality types to socioeconomic factors and personal history.
You could make some kind of [UNINTELLIGIBLE].
I can imagine, some sort of like can you figure out enough information before it's too late.
The dynamic assessment of violence seems to be more of a time pressure kind of thing.
Because you almost have to figure out on the spot.
But factors are very different.
But similarly I can find de-escalation.
It seems kind of interesting, because it almost seems like you could make a word game.
Or a face to face kind of thing out of that.
Can you remember what you're supposed to do in conversation with each other without missing any of the [UNINTELLIGIBLE].
And that might be something.
That would be like a live action game but not necessarily a physical game.
It would be more like a party game.
GUEST SPEAKER 2: One way-- this is, by the way, I didn't want to go there to make this too long.
But these are some parts of a standardized patient that I'm putting together, then I'm going to try actually to talk to the simulation department at UMass, to be played by an actor.
And this would be the training goals.
And I'm really particularly interested in how people respond when they are exposed to an angry situation or even to an angry face.
What do you have to do in order to de-escalate the patient.
And what I think I'm going to suggest to the actor is that, you are in the room and you're pacing up and down.
And appearing angry.
And you're allowed to use some generic swear words.
And to have a number of complaints.
Could be a number of them.
Nurse was too late to give me the medications.
The girlfriend didn't come in time.
You owe money.
You just had a job interview today and you missed it because you're in the hospital.
Could be a number of things.
And then the person approaches you.
The trainee should follow some rules in terms of the verbal and nonverbal de-escalation.
If that person is able to engage you, you can become less angry, less agitated, more interested.
Pacing less, lower your voice.
If the trainee is just doing the right or wrong things, you either stay at the same level or get more and more angry.
Of course, it's fairly demanding for a standardized for an actor, because the nonverbal elements are easy.
How you regulate the distance.
How you look, how you approach.
You inhibit your natural tendencies to pat the person on the shoulder, whatever.
But the verbal aspects are more difficult
Improvising dialog on the spot is hard.
GUEST SPEAKER 2: Exactly.
But at the same time, the goals of the dialog are not.
Because you should probably find out what's going on, even though, let's say, the actor will try to tell you about his life, and how miserable things are.
You know, I lost my job, and I was kicked out.
But you need to try to bring the person to what really happened.
I think that's very, very helpful because you're pinning down a particular conflict that triggered it.
And if you are able to do that, then you can take some steps.
OK.
I have a patient right now who has difficulty expressing herself.
And systematically there's one particular shift where the staff is not patient, doesn't wait for her.
You just need to give her a little bit more time.
And she gets upset and angry and starts pacing.
And I have to every time say, please, listen to her.
Or she comes up to me, she tells me the story and then I have to go back to the staff.
People just don't have the patience or don't care.
So that's just a simple intervention.
This is why you need to find out what's going on.
And then, if you can get more information about what brought the person to this place.
GUEST SPEAKER 1: OK.
I think we're prepared.
She did role play before I showed this, because [UNINTELLIGIBLE].
GUEST SPEAKER 2: Do you want to do it?
You've been trained.
GUEST SPEAKER 1: No.
You're the expert actually.
GUEST SPEAKER 2: OK. Alright.
Let's see what happens to me.
[AUDIENCE LAUGHTER] So, I'm Dr. [?
See.
?]
Where's John?
I want to speak to John.
I respond to him and I want him.
Why isn't he here?
Why?
GUEST SPEAKER 2: Who's this person?
I don't know
Where's John?
I only want to speak to him.
GUEST SPEAKER 2: Well, Nurse John is not available
[BANGING NOISE] Why isn't he available?
Why?
Only [UNINTELLIGIBLE].
I don't want to talk to you right now.
GUEST SPEAKER 2: How can I help you?
Get Johnny.
GUEST SPEAKER 2: OK.
I can try to see if he's around
[BANGING NOISE] You could [UNINTELLIGIBLE] now.
GUEST SPEAKER 2: OK.
But what else can I do?
Um.
An education.
Stop it.
Not good.
Try to poison me.
You all want me dead.
GUEST SPEAKER 2: What am I trying to do to you?
Doesn't [UNINTELLIGIBLE].
GUEST SPEAKER 2: OK.
I can talk about that.
So tell me what kind of problems do you have with the medication?
Trying to poison.
You all want me dead.
GUEST SPEAKER 2: So you don't think the medication is helpful?
Course not.
Trying to kill me.
[TEARING NOISE] [LAUGHTER]
They're watching me.
GUEST SPEAKER 2: What else can I do to help?
[INAUDIBLE] family.
GUEST SPEAKER 2: You want me to contact your family?
No.
I hate them all.
They're all against me.
GUEST SPEAKER 2: It must be very hard, to not have anyone to talk to or feel comfortable with
That's why I want to talk [UNINTELLIGIBLE] with John.
GUEST SPEAKER 2: Well, I'm sorry you're struggling so much.
And I know we haven't met before.
But I'm here and trying to be helpful to you
You want to kill me, don't you?
GUEST SPEAKER 2: No, I don't think I want to
[GRUNTING] GUEST SPEAKER 2: So we have a number of options.
And I think you look angry.
And you're scaring people and you're scaring me
Fine.
GUEST SPEAKER 2: You're scaring me
[UNINTELLIGIBLE] You're just laughing behind me.
You're not scared.
You're just laughing.
GUEST SPEAKER 2: No, I think you're a big, strong guy.
And when you get angry and upset, you get to be intimidating.
And there are other--
Is that why you're going to poison me?
Scared of me?
GUEST SPEAKER 2: No.
Well, I'm not trying to hurt you.
But we need to come up with a solution.
Because you could get hurt and other people could get hurt.
So we need to do something about that
I would be better off dead.
Everyone wants me dead.
GUEST SPEAKER 2: Well.
I understand you're quite distraught today.
And I want to help you.
I may not be able to, right away.
I'm going to try to get the nurse that you want.
But what if I do not get him?
What's the next best thing?
[INAUDIBLE] I don't know.
GUEST SPEAKER 2: Would it help to contact your family?
Maybe mom.
I want my mom.
GUEST SPEAKER 2: OK.
I can try to reach out to her.
And do you think that if you change the medications maybe you can try a combination that could help?
As long as it's not poison.
GUEST SPEAKER 2: OK.
I hear you.
You don't like the medication.
You think that it's hurting you.
But it's helped in the past.
And maybe a different combination would be better.
Hmm?
Maybe.
GUEST SPEAKER 2: Maybe?
Well, thank you.
We can go on like this.
[LAUGHTER, APPLAUSE] [INTERPOSING VOICES]
I have a question concerning about the disruption of narrative model of why violence occurs.
Is that pretty much accepted as the way-- GUEST SPEAKER 2: I just made it up for this.
[LAUGHTER]
I was just wondering if there were competing, different theories, it might be useful for people to understand more than one model, or is that [INAUDIBLE].
GUEST SPEAKER 2: The narrative perspective is, if you want, a cry from the past.
I think the last 20 years, in science and certainly medicine, have been dominated by hard evidence.
Actually, the evidence-based medicine has been taking over the entire world, as the ultimate standard.
So there are lists of the strength of the evidence, and randomized controlled designs are the best, and clinical impression anecdotes are the worst.
So on this map, the narrative or a person-like story about the patient may not be if you want the accepted model today.
But I think in dealing with individual cases, I feel that makes a lot of sense.
When you have to deal with somebody who is very distraught, if I tell him that the best available evidence was published two years ago and showed that, you know, 20 milligrams of Abilify would work best for psychosis, I'm not so sure that you would fuel the warmth and the empathy.
But the reality, it could be that that evidence is is right.
He's psychotic.
And he tried to a variety of medications.
Some of them made him fat and pushed his triglycerides up and raised his blood pressure.
If I switch him to Abilify in two months, he'll get better.
But dealing with the immediate issue, I think it's just, it's something else.
And that knowledge about medication, while helpful in the long run, will not make it very good.
So this is why I came up with if you want, for the purpose of this meeting, with a way of creating a metaphor that could sustain my verbal and nonverbal de-escalation.
I'm trying to create some sense, some coherence, some meaning, some meaning to that interaction.
And in doing that, I'm empathic and I'm trying to channel the energies in a way that it's almost like a beginning.
He's distraught.
He's ready to give up.
He's ready to shut himself away from the world.
I'm trying to reach out to him, without becoming his friend, without becoming his brother or father.
And I think in a sense I'm restoring the meaning of his narrative.
Somebody cares, somebody reaches out
Because it seems like a very interesting system for people, for designers to play with.
The idea of trying, even though there is this historic narrative, and you may not know enough about the narrative to be able to act on it.
But you'd find out things about it by talking to the patient.
And what you're really trying to do is make things fit within that person's narrative.
GUEST SPEAKER 2: You're filling the blanks that were kind of somewhat destroyed by events.
And it could be a variety of events.
And at times you have to use your imagination.
It's almost like intuition.
I mean, I know the principles from every case, but how I'm going to do it, it's not pre-programmed
Seems like that's also something that you could then play with from a design point of view.
This idea of disruptive narrative, and then how do you put it back.
Just like putting the pieces back together [INAUDIBLE].
GUEST SPEAKER 2: That may be the reason-- and that's another speculation, I'm just making this up-- why adventure games are so powerful.
Because, in a sense, you're finding a substitute narrative that is much more controllable than your own.
And for the time of the game you're there.
It's almost like you're buying a few minutes, 10 minutes, 20 minutes of a different story line
Actually, one of our researchers here looks specifically at adventure games.
And her point of view is that playing adventure games is like being a performer who hasn't been given the script.
You're still an actor in the story, and you have to figure out what that story is in order to move it along.
There's one other thing that I do want to bring up.
Obviously, I don't want any of these games that you guys are working on to be quizzes, to be really fun quizzes, even.
[INAUDIBLE] all this stuff is about.
I do want you guys to be learning games.
And not just Trivial Pursuit, Psychiatric Version.
That's kind of pointless.
We have had games in the past in this class where people basically pulled questions out of a hat and had to answer them, with various penalties.
Kind of fun, but that's not what I've been teaching you guys.
So try to think about systems.
Try to think about all these things that they want their trainees to learn.
Think about how they can engage with the system, in some small way, for your project
Can I ask a question?
What are your thoughts, or what are the field's thoughts on lying to people, so mitigate a situation.
GUEST SPEAKER 2: Well, I would call lying a distortion, in a very general way.
And when people lie, they have to make a decision.
Often, people lie for a variety of reasons.
What I always am surprised by is the reason to make friends, to get friendly, to resolve a situation.
And I don't believe in lying as a communication tool.
Given, it may make things easier in the short run.
Certainly clinically it's not helpful.
I was telling you about patients with borderline pathology that has this affective and relational instability that is basically taking over their lives.
They are some of the most accurate lie detectors that you can imagine.
You cannot design something more perfect than that.
Especially having been in the system for a long time.
They can feel the lie even before it comes up.
And if you're going that path with them
So this role play, for example, if you told Bobby, well, let me go get John.
You wait here for a minute.
And then you know that John is not working right now.
He's nowhere in the hospital.
But you just told him that to safely get out, then you get security to come back in with you, and you can subdue him or something.
GUEST SPEAKER 2: That's a great example.
And that happens all the time.
Because often when you're in this place, many of your requests are unreasonable.
So you need to set some limits.
And eventually you'll end up setting some limits.
He could have continued to escalate until attacking me, breaking everything in the room.
I usually tell them about how they make others feel.
How unacceptable some of their behaviors are, and what are the consequences.
So I'm going to probably tell you if this continues, I don't have any other choice but to call security and you may end up in restraints.
And with a chemical medication, it could be injectable, you can take it by mouth.
So I'm coming up front with how the next few minutes, the proximal future, looks like.
As opposed to kind of fudge it and get on his friendly side, and, oh I don't mean anything, but the security will put you in restraints.
So I don't want to buy him out by just becoming his friend.
If that involves lying-- that's-- some people do it.
When you're in the corner, you're-- OK, so what do I do now.
A little lie won't hurt.
And you do it.
Maybe it's just my-- GUEST SPEAKER 1: What you just mentioned too, that sort of like good cop, bad cop.
Well, I wish we could have more of a conversation here, but security needs to put you in restraints.
Security's the bad guy.
When that's exactly where you want the patient to be anyways.
GUEST SPEAKER 2: I'm OK, but you know Jim.
He doesn't tolerate it.
He's the director and he's going to-- [LAUGHTER] GUEST SPEAKER 2: Does that answer your question?
Yeah.
Would you-- [INTERPOSING VOICES]
I guess my followup is would you ever sneakily put medicine in someone's food or do something [UNINTELLIGIBLE].
GUEST SPEAKER 2: It's not only a lawsuit waiting to happen, of a major magnitude, but I don't think it's right.
Even though probably reality is that family members and sometimes even providers in nursing homes may do that.
But I think it's-- GUEST SPEAKER 1: You may need to treat the person against their will, which is often meaning putting them into physical restraints, which is usually like leather kind of things, kind of a straitjacket thing, or maybe to a bed.
And maybe physically inject them with something to really sedate them.
But you should always be telling them what you're going to do, right.
And the reason why you're doing it.
Even if they don't have any choice in the matter.
GUEST SPEAKER 2: And if you feel that the person is unable to comprehend, then you may need to work towards appointing a guardian who will take decisions for that person.
And it's not necessarily that the legal field has pushed itself very hard in field.
But I think it's fair.
You meet the person where that person is at, and try to be helpful.
And I think, in my view, there is no substitute for being upfront and saying this is what's going to happen.
He may not be my best friend, but he knows exactly.
I'm telling the patients often ahead what's going to happen.
In that sense, I'm giving them choices.
So if I end up taking them to court for commitment, they know that.
And you would be surprised how angry they are at the beginning and how OK they are later.
And how often they come up to me and say, thank you, I was in a very bad space and I didn't want to, and I fought you all the way but I think that was the best solution.
I don't necessarily do that to get that thanks.
I just do it as a principle.
Often it doesn't come.
It's not like people are grateful.
Often they leave the hospital.
GUEST SPEAKER 1: Let me show a little of this training, just a few minutes.
This is a project I've been working on a lot for NASA, which is basically to develop a mental health support system for the long duration space cruise.
And this is kind of a whole other topic.
But just one piece of this was teaching the astronauts how to manage conflict, interpersonal conflict more effectively or safely.
Astronauts being a very different population from what we were talking about.
[VIDEO PLAYBACK] --Welcome to the virtual space station, where you can access training and resources to help deal with the stresses of long duration space flights.
In the training simulator, you'll interact with virtual crew members to learn and practice managing some of the problems that might come up on long duration missions.
The resources modules has more advanced training that you can do to learn about adapting to the stresses of living in isolation, plus interviews with long duration flyers.
It also has a number of ebooks and audio books on dealing with stress.
Plus survival stories about polar missions and other adventures.
In the self-assessment module, you'll find questionnaires-- [END VIDEO PLAYBACK] GUEST SPEAKER 1: This is basically the whole interface.
What I'll show you here is our simulator.
[VIDEO PLAYBACK] --This is the training simulator.
Click on the topic you would like to learn about.
Then the coach or mentor for that topic will takeover and guide you through.
-- This is the conflict management part of the simulator.
We recommend that you do the 10 minute briefing first.
After you complete each simulation-- [INTERPOSING VOICES] Here's how these simulations work.
You're flight engineer one, or FE1.
Your crew mate, Chuck, is flight engineer two.
We'll be giving you three choices of what to say or do.
You can roll your mouse over them to hear how they're said.
Even though some options are more likely than others to reduce the conflict, I recommend that you try out options you might not choose in real life.
That's because all paths in these simulations lead to teaching points.
You can click the backup button anytime if you want to change your choice and go down another pathway.
You can click on the fair fighting manual at any time to read some tips.
You'll also find this manual in the resource module's library.
Now as you go through the simulation, I'll be right here coaching you along the way.
OK. Back to the simulation.
It's four months into your mission, with five and a half left to go.
So far, the mission's been going pretty well.
But you're both tired, having had to sleep shift last night.
-- Is your display getting squirrelly?
Shoot, mine too.
Damn, I think the whole system's down.
It was the wrong cable.
When I was installing the recorder I disconnected the A10 cable.
I fixed it right away but I think it screwed up the whole system.
Look, if ground calls, don't tell them about the cable.
-- Ah, this is a tricky situation.
You believe it's important to tell the ground what happened.
At the same time, you don't want Chuck to get in trouble.
What's your response?
-- They'll spend hours trying to figure this out.
We got to tell them.
What's wrong with telling the ground.
-- You're saying make a choice.
-- OK, but you're putting me in a bad position.
-- I know.
But thanks you're a real friend.
I owe you big time.
-- Houston, this is FE2.
Go ahead.
No.
Negative.
I have no idea.
Yeah I know.
I already got the reboot started.
No, I guess it's too early to say what the impact is.
Roger that.
I'll let you know if it happens again.
FE2 out.
-- You're feeling very anxious having just heard Chuck lie to the ground.
You're worried that this might backfire on you, if the ground finds out that you knew what happened.
This makes you very angry at Chuck.
How would you like to respond?
[END VIDEO PLAYBACK] GUEST SPEAKER 1: The reason I'm showing you this is that this is one approach that I've taken to teaching how to handle an interaction with somebody through sort of a simulation.
Now this is obviously not a game.
There's no score.
There's no time limit.
It's more of an interactive movie.
But with the teaching points here, just like what Cesar's given you is, we can give this to an astronaut and they'll read it in 5-10 minutes and say, yup, got it, what's the next thing.
That doesn't mean they actually can use it, remember it, absorbed it.
and I'm not saying that this is the way they should do your games.
But that this is one approach that we've taken to try to get them more involved with the material in more of a simulated way.
We can just do maybe one or two more of this.
What would you like to do here?
[VIDEO PLAYBACK] -- Chuck, we need to talk about this.
-- What is there to talk about?
Ground doesn't need to know every single little detail, alright?
Plus if we do tell them, next thing you know we're going to be all over the news.
-- This was a good choice.
Even though Chuck seems exasperated, you know what's going on in his mind.
Understanding what each other is thinking goes a long way to solving conflict.
Chuck is concerned about the ruckus this might cause about negative media attention.
But now you have something to work with.
What's your next move?
-- I know you-- you've been doing excellent work so far.
I can see you're really scared of them.
I know you don't want to draw a lot of attention.
What'll you think will happen if they figure it out?
-- OK. You got a point.
I'll get fried.
Especially if it looked like I was trying to cover it up.
You know, this really is a team issue.
-- On one hand, you seemed to get Chuck to come around to your perspective.
But the ground really needs to know what happened.
On the other hand, he's asking you to share the responsibility, even though it really wasn't your fault.
You have mixed feelings.
You don't want to be blamed for a mistake you didn't make.
But you're concerned about keeping a good relationship with Chuck.
How are you going to respond?
-- I'm sorry, Chuck.
But this was your mistake.
[INTERPOSING VOICES] [LAUGHTER] -- Houston, this is FE2.
-- Go ahead, FE2.
-- Houston, I think we found the problem.
The instructions you sent up for the recorder had some serious problems.
When you follow the recorder connect procedure, you can end up disconnecting A10.
It may need to be validated.
-- Copy that, FE2.
The procedures team ran through this in the mock-up, but they can give it another look.
-- In any organization [LAUGHTER] Sometimes this can help to create a bond within the workgroup, because everybody has a common enemy.
But, even though scapegoating the ground may keep the crew unified for a while, [END VIDEO PLAYBACK] GUEST SPEAKER 1: OK. You kind of get the idea.
We'd like to be available by email, in case you guys have any questions
I'll sent out some tech information to the class list and we're probably going to spend the rest of this week being around all of you so we're not quite sure yet exactly what we're going to do yet.
Hopefully [UNINTELLIGIBLE PHRASE].
I just have a couple more [UNINTELLIGIBLE] things to do.
I wanted to thank you guys first of all.
[APPLAUSE] So let's see.
On next Wednesday, which is the Wednesday right before Thanksgiving, I think a couple of folks are probably going to be getting out-- because I've been hearing some daffy stuff about what the TSA might be doing, and what people have been doing in response to TSA [LAUGHTER].
Getting to the airport early, if you are flying out, might be a good idea.
Because if everybody decides to refuse the bag scanner scans and the body searches, then we're talking about some real delays.
Anyway.
So I think what I'm going to do is [UNINTELLIGIBLE] this class.
If there are teams who are really wanting to do a live action game, and they want to get a head start on that, talk to me directly.
And maybe what we can do is, meet up with you as a team.
And try to get some of ideas across.
Do a brainstorming with me.
I will also get into talking about live action games the following week.
I'll schedule one of the classes for that.
So we're not going to miss that topic.
I just worried that if we get to it too late, and you are making live action games, then you don't have enough material to work with.
Finally, about this assignment, if the subject matter is at all uncomfortable for you to work with, and you can't really find something in this fairly large topic that you will be comfortable working with and you can't find a team, talk to me directly.
And we can work out some other assignment.
I can't say that I have an assignment in my back pocket all ready to go, and more likely than not, I'm half-assing it too.
I would much prefer folks to work on this problem that I think is fascinating.
And a pretty big problem that you can find some small project that you're comfortable with.
But if you are, I don't want to make anyone uncomfortable due to [INAUDIBLE].
So just talk to me.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu
Today is pretty much just set aside for brainstorming.
So we're probably going to do about an hour of that.
Just to throw the ideas that people already have in their heads about what could be with the problem space that we're working on.
This whole preparing people to deal with violence and folks suffering from an illness.
And there's a whole bunch of different things that people may be working on.
And the idea is that hopefully we can get this all out and then we can spend the next half-hour, and frankly the rest of this week, trying to figure out who's in which team.
Who's collaborating on a similar topic and who's interested in what.
And then, anyone actually get through any of the readings?
OK, great.
So there's a few things up there.
I realize that we don't have a lot of time to read.
And there's probably more stuff up there to read that needs to be read necessarily.
But it's all there as resources.
Any ideas that people thought might be interesting to work on?
Andrew?
So one of the basic issues I heard from the presentation, and the papers, was communication.
So this may work as a simulation, but the real world system would be communication between the psychiatrist and the patient.
Along with the psychiatrist with the person above them
Psychiatrist, patient, communication?
Yes.
What do they call them?
Not the patient.
Patient, psychiatrist?
Orderly?
Nurse?
What function do they have?
Give us more information
Psychiatrist-Patient for now that's fine
You know how talking about psychiatrists there is patient.
Psychiatrist, resident?
Maybe like nurse, patient, and then nurse reporting incidents to supervisors?
Three levels?
Yes, that's the one
So it would go supervisor, nurse--
I'm sorry, guys
--and you have the patient.
Those kinds of reporting up and reporting down.
All right, yeah?
So I had a couple ideas.
One of them was, every idea you have a patient and there's two doctors or psychiatrists.
And so one psychiatrist is trying to calm the patient while the other is doing the opposite and trying to make this patient as angry as possible.
So the way I was thinking was, the psychiatrist that's trying to enrage the patient?
That's training in what not to say.
While the doctor who is trying to calm them down is--
So by showing what's the worst possible way?
OK, I'm going to call that good cop, bad cop
--Yeah, and then another idea I had which I thought of with [UNINTELLIGIBLE] yesterday was have a team of psychiatrists working with one patient.
But interlaced in one of the psychiatrists is a saboteur trying to enrage the patient as well
And what are the target audiences going to be like actual residents.
I'm just going to say a saboteur in a group of psychiatrists.
Because maybe there's an idea under that dynamic that we can play with.
OK?
OK, so a few quick things.
One would be, I'm not exactly sure how this fits into a game.
But simulating people who are delusional schizophrenic by either having every player have a separate audio track playing in their ear as they play?
So there's voices speaking to them and they have different audio experiences.
Or having just one speaker that everyone can hear that's playing audio.
So incorporating audio into the experience to simulate in some way the internal thought processes and perhaps these delusional thoughts.
The voice that tells you to go kill your mother and what not.
Which is a documented issue
This particular game doesn't have to be a board game, doesn't have to be a card game, you have a lot more flexibility in what you want to do.
I would suggest steering away from doing anything that requires digital logic.
But MP3 players would be perfect
Exactly, something like that.
So the second thing would be a board game where you're in charge of managing a ward.
It's just a logistical game.
Making sure that you don't leave anyone neglected or forget anyone.
So your patients have their favorite nurse, and they have issues that you have to uncover and deal with.
And if you leave somebody unattended and someone goes over they may become violent.
So you've got to try and make sure.
And then the third idea would be to have one player play as the patient and to simulate their inability to communicate properly for whatever reason.
The information of their condition is hidden from the player.
And so try to create a game experience that is genuinely frustrating because the player doesn't quite know what's wrong with them and can't articulate it properly.
And the other players are in charge of finding it out.
And putting some kind of time-constraint things where it gets heated
There are a bunch of improv games that are exactly that.
I have something about me but can't tell you what it is
But the players know that it can get really, yeah
That's like the game where you have, I can't remember what it's called, but the one where you have to try to make the other person say a word.
But you can't use a certain series of key words
I'm simulating schizophrenia with audio tape.
Being part of RPG's which have done that well.
So like live-action role playing stuff.
Which did it really well.
It actually works.
So I recommend that.
It's been tried in a setting which I've seen before and worked really well.
So I think it could work out
You could do a refinement on that idea
So I know one of the main things that he was talking about was knowing what to do and just going through the process of what happens when somebody does something.
And so that brought to mind D & D, and how you have to prepare for the encounter and then in the encounter you have certain actions that you can do.
Calm through speech, calm through presenting some object, or some idea or whatever.
Which I think would map pretty easily into the D & D space.
And then you could also have certain abilities which are natural to your character like you're very charismatic which makes some things easier and such.
So simplify D & D to handle interactions with patients
So table-top RPG style.
And the kind of things you want to emphasize are things like preparation and playing up your actual skills?
Right
Things that you're good at.
So natural skills and preparation
So I had an idea for a game that could potentially be a card game.
Where the player who's playing as the patient and the player who's playing as the doctor are operating under different rules.
And one of the main points of the game is they're not actually aware of the other player's rules.
So, for instance the patient could have certain goals which he is trying to achieve.
But he is unaware which of these goals is completely unreasonable to the doctor.
And the doctor has certain actions he can take and is unaware which of these are going to be threatening to the patient.
And so, then there could be an instance where the patient actually believes the only way he is capable of winning the game is by using violence.
And so that could be one of the end conditions
So, sure.
And you have to deduce that out?
Moving on to Michelle?
I like the idea that a psychiatrist has to really keep their composure when dealing with a violent person.
They can't raise their voice.
They have to keep at a certain distance.
And they can't seem upset for whatever reason.
Sort of a party game idea, you could have two people.
Basically, one person would just have to be throwing insults at the other person.
And the person who's receiving the insults would just have to stay calm the whole time and continue to talking.
Which is really hard to do even if you know it's coming
So keep your cool.
I mean, that could be a game where you're literally insulting someone or it could be a thing where you're playing a character with a certain amount of composure points or something and you're planning actions and maintaining notes.
And different tactics to be able to increase those or decrease those
A co-op type of game where you are a bunch of nurses trying to take care of the ward.
This is a lot like Jeremy's Diner Dash style game.
Except, you have a limitation to how you can actually do it once.
So you will be forced to neglect one often
A patient, one?
Neglect one patient?
So nurses will get sick and they'll have to leave
So, rather you have limited resources.
Not quite nearly enough to deal with every single crisis that's going to happen
So the one thing I found interesting in reading this stuff is statistically the amount of violence that happens with mental patients is not actually that different from the amount of violence that happens in the general population.
And it talks about how people going through this, and deciding to go into the profession, might actually second guess themselves because they have this perception of there's more violence.
So I think incorporating something where you have life in the ward and life outside of it.
So you incorporate both the things where you're trying to deal with patients but you're also reinforcing this idea that it's not significantly more dangerous than real life is
None of your patients stab you but you get mugged on the way home
Yeah, something like that.
It's just life
One thing that could be really attractive is that now broadens your audience to people who aren't just psychiatric residents
So, I like the idea of the patient and doctor having different roles, rules, goals, views, et cetera.
But I also like the idea of incorporating an escalation and de-escalation into the conflict between the two of them.
So I feel like part of the thing which was really compelling about the role-playing scenario which they showed was there was then sense in which the doctor can say things which can escalate or de-escalate the situation.
Make it more or less likely that things will go a bad way.
And one way which that could be done partially through roles would be to have hidden keywords which the patient responds to or what have you and that moves them into a different state.
And then transitioning between these two states changes the goals which the patient has so maybe the intent on trying to get Nurse-- what was the nurse's name?
Dave?
John
Nurse John.
But as they get de-escalated then they just want to change their meds.
But then as they get de-escalated from that then they just want to go run back out
Just coming up with the rules that model what an unreasonable patient would do or what are you willing to give up.
And actually putting somebody in the shoes of that, is essentially something to think about what I can do as a doctor
So, there was this guy we used to play in my family where you're given a scenario and certain number of words you can't use.
Essentially, you choose random words and you have a scenario.
Basically what you were saying where certain keywords do certain things.
But you have to somehow create the best story or the best calm-down passage to read to the patient.
And then there's some arbitrator, the GM of the game, that decides which of the three people that were playing which one had the best story out of those.
To get you to have to come up with something to deal with, You need Nurse John right away
You could do a thing where you had to create a story about a random explanation of elements or cards.
But then some number of those cards, specifically the other player or maybe the game of other players specifically just saying.
Those internally are supposed to be our keywords.
And there's some 20 questions type of way to try and figure out what those cards are.
So that would be like a doctor trying to figure out what they're dealing with while they're [INAUDIBLE]
You can make that game like Guess Who.
Where each person draws a card and you have to give a statement to the other person or ask them something.
And then you see whether they escalate or de-escalate and each individual card has a trade story.
If you talk about this that's bad.
If you talk about this that's good
I'm not familiar with Guess Who
OK, so it's not really that much like Guess Who
20 questions?
What I can think of is everyone just has a whole bunch of character cards and each of them have traits.
Like, watch this.
If you you say this they'll get upset.
If you say this, it's not very fully fleshed out.
But everyone just randomly draws traits.
And you just take turns speaking to each other.
And then the patients either will respond or get more angry
So it's almost like your role is a collection of randomized traits or randomized behavioral rules.
So, rules determined by randomized rules.
For those people in the Assassin's Guild, or are familiar with the Assassin's Guild, there's the concept of assignments.
Things that you as an individual player are given at the beginning of the game as things that you can do and have to do.
Or if you do something you have to do it in these specific conditions.
Psychological limitations is what they're called.
It could be something as simple as you have to speak in an accent all the time or you have to limp.
It could be something more like every time you see blood you have to run away because there's a [INAUDIBLE].
It might be interesting to think of a collection of those things and that crafts how we play.
So this makes me think I should probably do the live-action class at some point and talk about party games and role-playing games.
I think you had your hands up?
No, my hands are in my pocket
Your hands are just up, correct?
Another thing that psychiatrists do when they're talking to a patient is to try and get the story out of them as to why they're upset.
So you could have a game where the patient has some set of cards that has some set of story with some keywords and the doctor has to try and get these words out of him.
But the patient, himself, doesn't fully know his story so it keeps on changing.
You can keep modifying your story as you go by drawing cards and putting them away.
And you can only really get the full picture once you get all the cards from the patient that you can
So, does the patient have all the cards?
The patient has his full story but it keeps changing
So, ever-changing story and you want to stay on top of [UNINTELLIGIBLE]
Yes
This isn't really a fleshed out idea.
But as a mechanic?
Say there's a doctor and a patient and the only communication is through cards.
So the doctor will give the patient two cards, and then the third card is randomly inserted off of a deck and shuffled in.
The patient receives all three.
So the patient might be schizophrenic and imagining things.
And they get what the doctor is saying but they also get other random things as points
So as a mechanic that would be-- man, that's hard to describe.
The idea that you will be adding the randomized element
So you get information that is both what the other person played, and something random, and you can't distinguish which is which
You have a communication channel and you're adding random noise
Yeah.
Is that a question [INAUDIBLE]?
I am running out of space, so that's a good thing.
Any other ideas?
That's creepy.
Anyone else?
One concept I thought was interesting, though I'm not really sure how to turn it into a game immediately, is the idea of the patient and specifically the doctor or the rest of the world having at the moment irreconcilable narratives.
And the goal of bringing those into some type of medium in which the patient can comfortably exist with this narrative that's in line with the rest of the world
So the idea, again, of narrative disruption is why people get violent in the first place.
So maybe even taking out the question entirely of is this person going to act violently, but can you even work together with somebody to change how they see the world until it makes sense on both sides
So with that, something I can't actually think of a game for it right now, be something like you have the patient and doctor.
One person trying to figure out the other person, the other person trying to trigger the other.
But there's something in the middle that's changing everything.
So it's just trying to reconcile the difference.
Figuring out the connection and communication between the two
So you have a situation where you actually have three people playing.
And there's one person who's changing information on some predictable or unpredictable way
Maybe they have their own objective in mind.
The two people on the ends are trying to figure out what the other one's saying and then one person in the middle has their own agenda, representing schizophrenia
So there's a noisy communication channel with an agenda --I like that.
That could be an algorithm or that could be a person.
It could be a person running an algorithm in their head
I mean the noisy communication channel, or the actual state of mind of the patient, that is the point right?
And then depending on their level of arousal, that could change how everything gets screened into being very combative.
And so everything the doctor says is taken as being
Some sort of parable that's actually going to affect how they were able to [INAUDIBLE]
This would probably only work once, the first time you play through, but again you've concealed from the players, or certain players, themselves what their goals are.
Or what they believe think about the game.
Say, it is in fact a delusion or not true.
So you might tell them that it's about a game set in 2100 when you've been wrongly imprisoned in a psychiatric ward and you must fight your way out or something like that.
And there's a voice that will guide you, or whatever.
And then you don't tell that player that they're in fact playing a schizophrenic player who will [UNINTELLIGIBLE] these things.
And then you use some sort of live-action role playing or not, I'm not exactly sure, but this other guy is convinced that this is what the game is about
I could write a [UNINTELLIGIBLE] rivals scenario.
The player's don't know that going in, but folks have seen the film The Cabinet of Dr. Caligari.
It's an old German expressionist [UNINTELLIGIBLE] film.
I could spoil it?
Does anyone care?
It's an old 1920s
I think you're fine
But the basic idea comes down to the hero of the horror story is actually just a patient in a psychiatric ward.
And everybody who he believes is part of the story he's conquering is somebody else in the psychiatric ward.
The big villain is actually the person running the ward
That's like Shutter Island
Yeah, it's like Shutter Island
Not entirely unlike a lot of different stories.
But similar idea of the information being given is not to be trusted
Something to go along like that, a game mechanic for that would be everyone's assigned a different role.
And one of those roles is patient.
So you don't know if you're the patient or not.
And so in time as you do your objectives, and as you play along you may start to realize things aren't actually as they seem.
So eventually you may figure out I'm actually the patient here
Right, almost like primeval authority in that sense
Going along with that, I think an interesting mechanic would be like the Battlestar Gallactica.
The person knows that they're a Cylon but they can infiltrate by doing that voting system at the end.
What is that?
The [UNINTELLIGIBLE].
So you know that someone's trying to sabotage you but you don't know who
So that's who's the patient from a different way, right?
One is, player knows who-- at least one person knows and the other one is trying to figure out whether they are it.
[INAUDIBLE]
This is just an interesting mechanic.
There was this RPG called, Great [INAUDIBLE] Game that had this really interesting mechanic for handling the dark side that each player had.
Which might be relevant in this situation.
Essentially you had a different person who's also playing another protagonist also play your antagonist.
And so, they would either talk to you and tell you things or they would just directly control your character like if you fail at the composure role.
And taking the adversarial side of your character away from your control, it means that your character is more likely to behave in a really disruptive way.
Rather than just a controllably disruptive way would be if you had a vested interest in your character
So, open end case control of your character.
And that might work really well with some of the ideas of the good cop bad cop idea that someone thought of already
So, actually going off of the audio idea of someone is actually hearing voices?
If it was the patient thing.
You could actually have things where you have one player is actually playing a patient who is schizophrenic and another player is playing the voices in his head.
But they don't really know that.
They are playing the scenario where they have some delusion that they're trapped in whatever.
So having a person actually being the-- as you're saying-- having an actual person be the dark side of the patient
But neither of them know who's the dark side?
The patient, yeah, you could have it
One way you could arbitrate it is you could say, the patient player is just basically able to do as they please.
But there are rules which govern whether the dark side character can take over.
And that means that the patient is genuinely driven by trying to reach a compromise but they can just be overruled by the dark side
There could also be things where it could be more like a strong role-playing thing?
Where the voice is actually just trying to convince them that yes, this delusion is real
So, do you still think you have a recorded thing?
Or two different?
You could actually have two people just actually standing there talking
So two people playing one character at once
It would potentially require a lot of suspension of disbelief on the part of the person playing the psychiatrist
It might be one of those things that's easier to mediate by having people talk on screen or something like that?
Or a microphone in a speaker type thing?
[INAUDIBLE PHRASE] I've not seen many games that are designed specifically to be played on the [INAUDIBLE] function
I definitely like that idea, and I think it could work well in a card game as well.
If you have two players sharing the same hand, and the player who actually gets to play the cards from the hand is whoever's in control of the patient.
And so the state basically changes depending on what the doctor does.
Different players come in control of the patient's hand
So it seems like we actually have a lot of different options on how we could mechanic someone who's going to be unstable.
And also, a bunch of ideas on how that state switch could happen or what makes it take over.
So that's pretty cool.
And that's not mutually exclusive from any other ideas that are specifically about the patient and the doctor, right?
So we're talking about what happens to the patient.
These are the rules a patient could be operating under.
The doctor could have a different set of rules
I was reading about this, and I think that it turns out most violence in psychiatric wards happens to the patients themselves or to other patients by patients.
Much more than it actually happens towards practitioners.
And I'm just noticing that I don't think any of our ideas address any of those issues
Ward management maybe gets a little closer
People start turning on themselves.
And I was just thinking another possible way you could apply a lot of these role-playing multiple role auditory mechanics is to a situation.
We should probably do research in what situations lead to patients coming in contact with each other and the encounter turning violent.
And all players be patients that have various different mental disorders, and just to help you understand as a patient what kinds of behaviors and situations would lead you to going violent on someone else.
Just like one of those, be in someone else's shoes helps you understand the situation better
So we're just putting them in an environment where everything that is necessary for a different character to become upset and violent and just see how it naturally occurs.
[INAUDIBLE]
They gave a persuasion that's a cross between BS and Uno.
So you have two doctors who are trying to get rid of cards in their hand, and the patient wants a certain set of cards.
And he's saying, I'm looking for John.
And the two doctors pull out a hard and say, oh this is John.
Or they can be like, oh this isn't John but I can help you out later
That almost seems like it's intended to teach people about lying
Yes
So, I'm going to say it's almost like BS and Go Fish.
So we have a pretty good pile here.
A lot of these can obviously be combined.
And there's no reason why multiple groups couldn't attempt to do similar topics.
In fact, you could take exactly the same set and I'm pretty sure you're going to [INAUDIBLE].
So that's fine.
Again, a lot of things that could probably be easier done live-action.
Or easier to do a prototype live-action.
The folks who were here on Monday, they are actually focused mostly on trying to see what could be moved out to more resonant things in the future.
Your games don't necessarily need to do that.
But you might want to keep that in mind.
Because this assignment is a game for a client after all.
And if you manage to address that need-- here's something that doesn't require one person from the original GM team.
You can imagine the acting exercise that we ran through, beyond talented improvising on their feet.
Requires people and requires responses that may not necessarily every gaming environment's going to have.
Something that can be turned into a very understandable set of rules or using technologies that are very accessible.
They're going to play your voice and its going to be a lot easier than something like formulas or something like a phone as opposed to [INAUDIBLE]
It was a phone conversation actually going?
So if it's live, and people are in different rooms, and actually talking to each other over the phone or Skype or something like that.
But you're talking to someone?
You have the phone-ups and then that other person can hear what you're saying and be the evil voice?
That's the thing, you don't even need to do it on Skype.
You just get two cell phones with [INTERPOSING VOICES] Now, the technology just becomes that much easier for games [INAUDIBLE PHRASE]
Or you could make it like if there's some game that two people are playing.
The two groups that are playing the same game but they both hear the other group.
Like this group would hear the other group's playing the game.
So it's not even just a random conversation.
They're playing the same game you're playing but you're hearing them play games
Wow.
Wow
That's deep
I could say two groups eavesdropping same game
I just wanted to say that I think the thing which is really hard if you're working with people who like don't really want to do a role-playing thing is building a level of emotional intensity and the fear of violence.
Which is the point of the training.
They want them to be able to deal with that kind of emotional stress on your feet.
So I was just wondering.
Because I don't have any good ideas.
Can anyone think of something which works out?
Which could genuinely create a situation where one could be afraid of violence?
Without actually having to threaten another person
I don't think you're going to be able to do that
Well, if one player's role is actually not to understand but to provide the experience for the other?
And they are hearing the recording that's giving them cues as to what to do?
And the other player's not agreed to this, and one of their cues is like and now, grab the other player by the shoulders and shake them violently and say something.
You could possibly create a situation where you're like, whoa
The problem is you have to be able to get somebody who can do that without cracking up in the middle of it.
Because people do that when they're nervous and uncomfortable.
So you'd pretty much be reduced to finding professional actors.
And that's a limited resource that puts a limitation on how widespread your game is
Even people who are [INAUDIBLE]
I had a thought where you have a patient to patient interaction.
So you have two people in different rooms on the phone with each other trying to negotiate something.
I don't know what it would be
And some other people in each of their rooms that are talking in their ear and trying to distract them from the conversation
OK, so patient talks to patient
Maybe it could be something like they're trying to negotiate [INAUDIBLE]
With voices in-person?
There is a specific article on schizophrenia that I uploaded in your resources.
[INTERPOSING VOICES] Patrick first, then Michelle?
So I was thinking something along the lines of Battleship?
If you have a doctor player and a patient player and they're both looking at their own version of the same board.
And if the patient wants to get from one end to the other and the doctor has to allow whether or not they can make their move.
But on the doctor's board they have different obstacles.
And so the patient will say, I want to move from here to here.
And the doctor says, I can't let you do that.
I can let you go from here to here.
It'd be a game about compromise
OK, so sort of a doctor mediated [UNINTELLIGIBLE PHRASE].
And the Battleship analogy is good to help us remember where the idea came from.
Michelle?
Yeah, so just something that Owen said about trying to simulate fear.
I mean, if you can add time pressure to any game then I think that strong of a stress situation can almost be fear.
It can get [UNINTELLIGIBLE]
Because what you're really going for is stress in making decisions
[UNINTELLIGIBLE] created stress without the [INTERPOSING VOICES]
So, an idea of how to take the doctor mediated board movement into the real world?
We're talking about these phone to phone conversations.
You could have the patient in a room but he's blindfolded.
And he's trying to move through the room towards an exit in the room or something.
And the doctor has some limited information about the room and has to help him navigate through the room.
And he'll say, I want to move forward.
And he'll say, well I can't really let you do that because there's a pit fill of spikes in front of you.
And you wouldn't necessarily actually have a real pit full of spikes in front of them.
But that kind of thing where you're trying to gather information and help them through this.
And you could put time pressure on it and say, well I have to make a decision now because he wants to move now.
But I might accidentally move him into an obstacle and have to deal with that
You could have a thing where there are some things that the patient needs to do that the patient can't talk about.
And it might be something like the patient has to move every 10 minutes.
And only the patient knows this.
The doctor doesn't
The patient has a fear of moving to the left
Or something like that
Things like that
Patient has to move left, right
I think it also emphasizes the thing they were talking about on Monday in that you are both trying to not-- if it escalates to the point where violence happens-- you're both trying to not let yourself get hurt but you're also trying to not hurt them.
And I think incorporating that in some way might be cool
Definitely a lot of these ideas are sort of mechanicing the patient's limitations are really, really, cool.
But again, remember the audience is the residents.
So understanding what a patient is going through is useful.
But also thinking of rules that help a resident understand what they can do is also really important.
So don't just think about throwing a ton of limitations on them without also given them a release.
Things that they can actively do.
Words that they can carry out
So, working off the time pressure thing, I think that it might make sense for these kinds of escalation, negotiation type, situations for the person who's controlling the patient to know what their triggers are.
And every time a bad thing is said, they just reduce the amount of time that the doctor has to deal with them.
And they say that, they just say minus 10 seconds.
And they have a timer and it ticks down to reach the conclusion of the doctor getting them to where they want to be.
And rather than like raising their voice, or being physically in some way, they just adjust a timer.
And when it gets down to zero it's over
I can see that.
[UNINTELLIGIBLE PHRASE]
I'd just like to say, one way you could make your players feel pressure is to give them a really flat goal hierarchy.
Right?
So a good example is Missile Command.
Does everyone know that game?
Missile Command is just six cities.
And there's missiles coming in and you have to shoot them all down.
And each city is equally valuable.
So if you lose one you're in a drastically reduced position.
Whereas games that tend to have a more structured goal hierarchy basically means you can afford a loss.
Right?
So like chess is a good example.
Where you can say, OK I can afford to lose this.
Whereas in Missile Command you can't afford to lose anything.
So by really capping what the player has, what they can afford to lose, and then making sure that all of their goals have equal value, that means that every loss and every failure is felt much more strongly.
And I felt like, Dion when you were doing your role play, there was this sense, especially coming from Caesar, that any one thing that goes wrong is really bad.
There's a really thin line there
That's especially applicable when talking about the doctor's goals, not necessarily the patient's goals
I mean, from the doctors' position right?
You can't really afford a lot of mistakes in these situations
I'm kind of aghast, but I was listening to what you were saying and I have a thought on how to combine lots of these different ideas.
I'm envisioning a game where there'd be two types of players, like you were mentioning, doctors and patients.
And there'd be one doctor and multiple patients.
It would be a turn based game where you go around and give each person a turn.
The doctor would always be in relationship to each individual one.
And limited vocabulary I thought was important, taking that from a couple of the ideas.
The patient and the doctor each have their own set of vocabularies and the clash of that would introduce mechanic there.
Where the patient has their own vocabulary describing to the doctor what they like, what they don't like.
They know what they like.
You could extract it to describe anything the doctor would have in their vocabulary.
So, as one example the doctor could give the patient a card with a shape or some item on it.
And the item has attributes like color, shape, something that somebody could like or dislike.
In their list they could look at this and see one of these things clashes with something in my list of dislikes.
I don't like that card.
I'm going to escalate my anger, my displeasure, with the situation.
And so, the doctor has a limited amount of time to deal with each patient.
As the patient's escalation increases, they have less and less time to be dealt with.
After a certain point the doctor loses that patient, so you have that loss brought in.
And then after all the patients have been lost the game is over.
The way that it would be won is that the doctor is discovering the profile of each patient.
Because they are trying to, like in a normal setting or a real world setting, you're trying to discover the profile that allows the doctor to deal with the patient in the best possible way so that they aren't violent
So that seems to play off the patient and doctor have different rules and triggers.
You've got a situation where, in what you're suggesting, you have one doctor and multiple patients.
Which is accurate.
I'm not quite sure that they'd be dealing with them simultaneously but it might not matter for the purposes of the game.
It might be like those improv games where you manage to figure out before time runs out what a person's profile is.
That could be the win condition.
So it's equal win [UNINTELLIGIBLE] and equal loss if you don't actually get it incorporated with something like you give a trigger and reduce the amount of time you have available, the number of turns available.
It doesn't have to be real time.
So you can use it with multiple patients.
So that doesn't even necessarily have to deal with talking about actual psychiatric ward situations.
That could be something like objects and colors.
It's more of a guessing game.
That might be too abstract, possibly, because in the end you want people to be able to tie it back to the experience.
To be able to make an easy connection to what it's like to be a resident or what it's going to be like to be a resident.
So this is where the theme needs to be pretty clear.
But in [UNINTELLIGIBLE] talked, fear is not meaning.
Meaning is also tied pretty well with your mechanics.
So that's just something to keep in mind.
OK?
We probably have another half-an-hour class.
I think the plan was just to let everyone figure out teams you wanted to be in.
This is going to be class time to talk to each other and see if there's stuff in here that people are really psyched about.
Is there anyone in this class that's already like, this is something I really want to do?
Well, my preference is towards the ward advantage.
To ward advantage?
I don't really have a plan [INAUDIBLE]
I'd be interested in voices and hidden information.
So fooling players and that result in them behaving in erratic ways.
And as from the point of view of other players, and then the point of view of player's behaving erratically, other players are behaving erratically because it's not in line with what they think is actually going on.
Or voices, or both, or something like that
So that's the general territory you're interested in going into?
Who wants to?
I'm very interested in the escalation and de-escalation by hidden keywords.
[INAUDIBLE]
I like basically hidden rules, goals, and keywords, and two different players playing the same character.
Those are two mechanics I really like
We have a couple of seats out there.
Caffeine helps, there's a bunch.
What do we have on Friday?
On Friday we mostly just have brainstorming and team-building scheduled?
That time is pretty flexible
S0 what I can do maybe, if folks can plan to come into Wednesday with a team, what I'm going to do is I might move my--
You mean Friday?
--Sorry, Friday, yeah.
Then I will move my live-action talk up to Friday instead.
So Wednesday we can just-- since it seems like a lot of people are going in that direction.
You don't have to do a live-action game, but it's one of my favorite games.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation, or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu.
So some of my perspective on what kinds of things you can do in relation to game mechanics is kind of like seeing [INAUDIBLE].
But it's not the be all and end all.
I'm basically showing you my perspective but it's not the only perspective out there.
I have very little that I feel qualified to say about things like sports.
Things where we are talking about athletic competitions.
Faster, higher, stronger, that sort of thing.
But I'm a little bit more engaged in games where there's role-playing.
Where you're taking on a persona or at least some sort of behavior that you aren't just normally doing in real-life.
It's not just about how fast you can run or how skilled you are at hitting the ball.
That sort of thing.
Let's see, and not every single live-action game needs to necessarily fit within role-playing sports.
You've got things like party games.
This afternoon we'll have the lead designer for Dance central coming in to do a little Q & A about Dance Central around 4 o'clock, 4:30, in the TV lounge.
You guys are all welcome to do that.
And that's arguably a live-action game.
Sure you've got a computer, you've got a-- how much is the Kinect?
$150 piece of equipment connected to a $250 Xbox hooked to a screen that's probably more than that.
You've got all this technology, but in the end, what's that game about?
What's Rock Band about?
It's about the people who are in the living room.
And what is one of the arguments that [INAUDIBLE], in a different chapter in Casual Revolution, he argues that one of the big things that's happened with gaming is this re-introduction of the space in front of the screen.
The living room space has come back into play.
You haven't lost that with board games or card games, but for electronic games really it used to be something that existed with arcade games.
You've got these giant motorcycles or steering wheels and stuff like that.
And you still had it with some really expensive computer games like flight simulators where people buy the joysticks.
I'm a big train fan, so I've got a Playstation One train controller.
It's throttle and brakes.
That's all it is.
No steering when it comes to trains.
It's a lot of fun.
With arcade games, and of course [INAUDIBLE], there's this whole idea that you're going to be engaging your entire body.
And you could argue that those are live-action games.
And you take the technology out and what you've got is just this human body playing with all it's constraints, beholden to gravity, it can't move faster than a certain speed, it's kind of ungamely, and it's got roughly a certain size.
Gives you constraints that you can work with in terms of game mechanics.
One of the things that we did last year was we had a live-action game assignment in this class.
And the constraint was you had to be able to play the game in this room.
Was a newspaper a constant?
No
That was one huge decision to use newspaper as a constraint.
But they did things like blindfold people.
They were playing around with the senses on the body so that you could hear.
When one person had unlimited ammunition, in terms of low vaulted newspaper and the bucket, but couldn't see where everybody else was.
And everybody else could move around the room on sheets of newspaper.
They had to step on sheets of newspaper.
So you could always hear where they were stepping.
And the person with the bucket was trying to pelt the people who were stepping on the ground.
If you got hit you had to go out of the magic circle and then start coming in.
The whole idea was to attack the person who was actually throwing all these newspaper balls.
We had a little newspaper hat that you could grab.
It was pretty awesome, because what you need to play that game?
You need one set of rules, and a chair, a recycle bin--specifically it was that bin-- and newspaper.
But there were all these things in play about what you can do.
So OK, your target is going to be roughly the size of a human body, and roughly the weight of a human body.
It's going to be able to move at roughly the same speed of a human body.
And you cannot have multiple targets in the same place at once.
In fact it's hilarious when you try, because you've got these [INAUDIBLE].
And because you're playing with the humans, all five of the human senses, usually just like hearing.
I don't know that anyone's made a smell game.
But you will have the situation where the people who are standing on the newspapers will try to make noise near the other people who are playing the game.
So they were drawing attention away from themselves.
Sometimes it works.
Sometimes it doesn't.
Sometimes it leads to hilarious collisions --a good time was had by all.
And those kinds of party games.
You know, King of the Hill?
Can anyone think of any other playground type games that are personal favorites?
Freeze tag
Freeze tag?
[?
Lava
What's?
[?
Lava's where you have swings and all things you can climb on.
And you're playing generally on the little pebble-like rocky stuff.
And if you step on the pebble stuff that's like the lava and you get burnt.
So you have to stay hanging on the metal part.
And usually there's someone who's immune to the lava trying to tag you.
So they can walk over the lava but they can't go in the structure
Lava monster
Does the game ever stop?
Just out of curiosity?
Yeah, someone becomes the next lava monster.
Once you get tagged you switch roles and keep doing it
But the thing is, I never played that on my playground.
I always played in my living room and throw couch cushions on the floor and stuff
That sounds really dangerous
We'd have wheely chairs and push off and yeah
I used to have a water bed.
So there was some unpredictability that I could have in my bedroom with my two brothers.
But they were both bigger than me so I always lost.
I mean, you can think of-- on one hand, these are athletic games because they do test your ability to move and control your body.
Which is why they're so popular around kids who are trying to master what their bodies are capable of.
But on the other hand, they're also games of mimicry going back to [INAUDIBLE] definition.
Because you're pretending that it's lava, and pretending that you're a person that cares that it's lava, or a monster that's immune to it.
And these games don't necessarily-- they may have some rules about, OK you can tag now we swap.
I've seen games where sure, I've technically been tagged out of this game but since this game really has no end state that just means that oh, I lost.
Now I'm back in the game again.
It's like playing a Doom death match and you die and you're just right back in the game again.
Let's see, anyone play those-- what's the name of the football variant where you just have a scarf attached to you?
Flag football
For those of you-- everyone knows what it is?
Isn't flag football where you just have a flag and instead of being tackled you?
Yeah, instead of tackling you just grab the flag.
And that's an abstraction.
That's an abstraction of something that you well could physically do.
You could grab this person and pull them to the ground
Does touch football exist?
Yes
That's a smaller level of abstraction because you can really hit someone pretty damn hard with touch football.
But it's an abstraction of, here's this thing that I personally, physically, could do.
But I'm going to introduce this level of abstraction for-- I don't know-- safety.
That's usually the reason.
And so you've got the human body and you've got all of its values and virtues and all of its curses.
You've got issues of safety.
You don't want people to hurt each other, but yet human beings and their bodies are extremely capable of hurting each other with no added implement at all.
Even worse when it gets to a game and someone's trying to win something.
If you don't have a specific rule that says you can't punch this person, but you have a game that's very athletic, someone's going to try.
That's why you get these sort of abstractions to pretend in that military form that all right, I've grabbed this thing off you.
Now you have just been tagged.
I've effectively tackled you [INAUDIBLE].
So, what I'm trying to draw attention to with these examples is that with live-actions games the thing that you have to keep in mind all the time is at what point is the person who's playing actually the character who's playing?
It's really, really, hard to tell when it comes to live-action games.
If you are playing in sports it's the same thing.
Any given athlete and their performance in the sport is who they are.
It's the same person.
If I'm role-playing then the person that I'm playing is not supposed to be the same.
But there needs to be a whole bunch of rules that basically specify that the character that I'm playing is not actually the person that you're seeing in front of you.
For that time, the player is not the Count of Monte Cristo or something like that
One interesting exception to that is wrestling.
Not wrestling like actual high-school wrestling, but wrestling on TV-- I don't remember what it's called.
Because everybody puts on a persona.
They're never the same person that they are.
I just thought it was interesting when you said that any sports person, but that's one exception
That's true.
And that creates a lot of discussion about whether wrestling is a sport.
It's athletic.
Everyone agrees that it's athletic.
But so are trapeze artists athletic but you wouldn't call it a sport.
And in fact, if I understand correctly, wrestlers are insured as circus performers, because they have the same risks and are playing the same role as a performance.
But you're right, that's one of those things where it's a persona.
The really, really, good wrestling networks play around with what's real and what's not.
Yeah sure, these two people are putting on these ridiculous personalities.
Threatening to kill each other over the most trivial and made-up of the offenses, and crazy accents, and costumes, and everything.
But when they're bleeding they're actually bleeding.
There's no denying and no doubt they actually get angry with each other and people actually get hurt.
And playing around with that line is actually some of the appeal that we get in things like professional wrestling.
I guess that's what the terminology that they use.
It's very different from something like mixed martial arts.
I don't watch mixed martial arts
No, I don't think so, I'm pretty sure it's just flat out
[INAUDIBLE] are very associated with the person
So they are the persona
They don't have stage personalities, but they do have strong personalities
That's definitely a sport.
I mean, they're actually fighting
I mean let's take a look at Muhammad Ali for instance.
Extremely athletic person but his personality had a lot to do with it as well with how successful that he was.
The way that he taunted people on TV and on cameras before the game, so that people would get all worked up for the game, made him a star.
Sure he was athletic, but that was part of it.
So where that line is, and as designers you get some control over that through the rules that you're writing.
And there are a couple of tactics that you can focus on.
But I want to segue into-- that's the tension.
That is one thing that's pulling your design in one direction.
Which is, how am I going to figure out what the difference between the person who's playing this game and the character that they're playing, or the role that they're playing, is going to be.
In your games, for this next assignment, if you've got people playing mental patients then you've got to figure out how you're going to draw that line.
You've got to be clear whether it's the player who's actually upset at you or the character who's actually upset at you.
But if you draw that line too broadly, then you've got the situation where both be angry at you [INAUDIBLE].
Because this is a ridiculous situation and emotional.
Pretending to be really emotional can usually make a lot of people react by laughing just to release the tension in space.
In fact, that happened when Dion was role-playing with Caesar, right?
Sure, they were laughing.
They were trying to level that and take it as seriously as possible, and everybody else in the room was cracking up.
To release that tension.
And sometimes that's OK playing games.
In fact, sometimes that's great for the game.
And sometimes that's detrimental.
Depends on what you're playing for.
But you have to understand that is something that's pulling the overall design, why do rules exist, and why your game mechanics exist, and why your game mechanic is pretending a certain way.
That's one of the tensions that's pulling it in a certain direction.
And in reality, that's actually two different tensions.
Because in one sense, they're trying to make an experience as aversive, as feeling like you're actually doing it as possible.
And on the other hand, you don't want to make it so associated with who you are, and what you're actually doing, that people start to think, oh wow.
This person, this player, is actually mad at me.
Or, this player could actually hit me at this point.
While as, no the character [INAUDIBLE].
I call that verisimilitude.
These are just words that I picked up from dictionaries.
This one, you were in my masters thesis.
Verisimilitude is basically a tension when you're designing a game mechanic.
And that tension is telling you, I want to make this experience as immersive and as true to life as possible.
And the opposite of that is what I call dissociation.
Which is really hard to pronounce.
It's not like I just call it separation.
But, how do you draw that line between who I am and who my character is?
And I'll give you a couple of examples just to get inside this now.
Who doesn't know what a live-action role-playing game is?
Or doesn't have a good clear idea?
OK.
So maybe, a little bit of background just so that I can use the examples.
Do you folks know how a table-top role-playing game generally works?
OK.
So, start from scratch.
Imagine improv theatre.
Imagine you have an environment where you have a bunch of people who want to enact some sort of scene together.
In theatre, a scene is usually something like five minutes, maybe half an hour.
In a live-action role-playing game, that's more like an hour to 10 days to 30 days long.
A much longer period of time.
But the motivation is the same.
We're in this space to enact some sort of shared reality, shared creative reality, together.
In a role-playing game that's done by the usual addition of rules.
And usually some sort of world setting and some sort of character description.
And how these things are told to you could be as simple as the game master walks up to each person and explains what's going on.
You are all cats in an alley-way and some of you are hungry and looking for food.
Some of you are on top of the pecking order and make sure you get all of the food.
Here's how you eat food.
Here's how you move food around.
Go.
That sort of thing.
A game master could just do it at that level, and you've got a live-action role-playing game.
There's usually some sort of end condition or at least some sort of time limit to tell you when the game's over.
When you get a little bit more detailed, and when you start mixing and matching it with the conventions of other kind of role-playing games-- computer and table-top role-playing games-- you have things like character sheets where you're given your information on a piece of paper, or you create your information on a piece of paper.
And that information is information that's only available to you and the game master.
A secret information for all intents and purposes.
You will use different means to convey chunks of the information to the rest of the world.
Like I'm actually not female, but I'm playing a female character.
So maybe I'll wear a badge or something that has my gender on it.
And that tells the rest of the world treat me like a female.
That's one example.
I may actually be psychic, but the rest of the world can't necessarily see that.
Except for maybe other psychic people.
So maybe I'll have a little Greek symbol or something-- not Psi, that would be too obvious-- or maybe my badge is just a different color or something.
And people who are not psychic have no idea what that means.
And people who are playing other psychic characters now see, [INAUDIBLE] has a red badge.
I am sensing psychically.
And then there are other game mechanics that I can do with that person.
Actually all the psychic people know each other's cell phone numbers or something and can talk to each other.
That's an example
That's awesome
That would be to your advantage.
So the connection with improv theatre is actually stronger than a lot of people may generally assume.
Because a lot of people think of live-action role-playing games and they focus on the games part of it.
And they think of it as a RPG in the computer sense.
So your role is your character in the sense of, I have my character's stats, my character's abilities, a lot of these steps and abilities don't necessarily need to be written down.
Because, again, you're dealing with a human body.
So, how strong is my character?
I don't know.
How strong are you?
How long a sword do I have?
Well, how long is that piece of foam over there?
That's your sword now
[INAUDIBLE].
I played a character that was supposed to be very, very, strong.
And I am not very, very, strong.
But I had a disclaimer badge and the that it manifested itself in game play was that at some point they challenged someone to an arm-wrestling duel.
And they made sure-- they looked at my stat and saw, oh I know what this means.
And so even though they were about twice my size they pretended to lose.
It was very entertaining
So the spectrum of on one hand there are these games.
The type of games that Jeremy describes allows you to play things that are physically very different from you.
And then there are some live-action role-playing games--
Where it's just how strong you are
--Yeah, it's just how strong you are.
A lot of those are like [INAUDIBLE] LARPs, where people will dress up in actual armor that they've decided to make out of leather or duct tape.
And they will swing around swords instead of foam padded.
They'll go out to a forest somewhere-- typically in New England-- and spend two days, three days, in the middle of a forest sleeping in a cabin somewhere and pretending to fantasy adventure it.
And every once in awhile there will be GM controlled volunteers who will be wearing [INAUDIBLE] masks.
That's on the other spectrum.
It's a spectrum.
It's not like the [INAUDIBLE] LARPs where you run around with sticks and are Assassins Guild type LARPs where most of your abilities are governed by numbers and stats.
Kind of this spectrum.
You can think of assassin guild type thing as being very much inspired by what you could do on a computer or what you could do on a table-top LARP.
Because on a table-top LARP, how strong your character is has nothing to do with how strong the player is.
It's what [INAUDIBLE]
Playing a game just recently, I found out that in New England there are basically LARPs which get done with rifles
They're military level
But they're not paintball.
So you shoot people
With plastic pellets?
Yeah.
Yeah, air-soft is what they call them.
And so one team will dress up as US soldiers, and the other ones are like middle-eastern soldiers.
It's elaborate and slightly distasteful.
You can go and look at their website.
I think it's the New England Airsoft Association or something like that.
And they have excerpts of a video cam of these.
And there's negotiation between the two sides and what have you.
They actually take on the roles of the insurgents or the [INAUDIBLE].
Sometimes they're just shooting each other.
Other times they're trying to find out who in the town who is the bad guy
Sometimes it gets more elaborate.
And some of us are actually going to be [INAUDIBLE] and some of us are actually going to be combatants and you have to figure out who's who.
That one will be really, really, interesting [INAUDIBLE] for IEDs.
That's something that annoys me about games that involve Middle Eastern combatants.
Is that, they don't have rules for IEDs.
And that's the primary form of contact in those states.
That's to the point.
Military exercises a lot of them are basically live-action role-playing games.
All the way from I'm actually running around with a gun full of blanks or shooting pellets or shooting a laser beam that is going to register on some other soldier's gear
We call them war games
We call them--
[INAUDIBLE]
--Yes, that's a good thing.
Then you also have the other end of military simulations where it's just a bunch of people in a room looking at a screen.
Or not even a screen, a white board with a world map on it, and they're moving forces around like a table player.
But those are still considered live-action action things, because every person in the room is representing someone in the chain of command that would normally be in a war room attack.
I am the commander.
You are the lieutenant, or something like that, and I'm going to say, get me the status of Washington D.C. Has it been nuked yet?
And the lieutenant says we have three missiles coming in based on some look up.
The term for that is typically called creep [INAUDIBLE], especially when it's real people playing against real people.
It's two or more player game where not everyone is one the same side.
If it's just on one side then just the war game terminology tends to be more a catch all.
But when you have two players on each side, and are playing through these things, how results get resolved also can vary.
There are obviously digital and mathematical ways to compute.
OK, I've given an order to fire a missile at Singapore.
And it's going to take two hours for it to land.
And there's a percentage that it might land.
You could do it.
You could do that way.
But traditionally, what's actually been done is that you say this is what I'm going to do.
And someone far more experienced than you, usually your instructor or one of several instructors, takes that information, goes into another room, and talks to the people who are actually doing the same thing with your opposing team.
And they're just saying, OK in this event what's going to happen?
And then they go back to each team and they report, missile missed.
Or, your troops engaged.
You found your opponent's troops at this location and you've lost 50% of your troops.
That sort of thing.
And that's the traditional [INAUDIBLE].
The nice thing about that is that takes advantage of the fact that not just the players but everybody is a live-action human being.
Because you have flexibility.
You can do ridiculous things that programmers or game designers may not necessarily have assumed were possible, but happen to make sense in the heat of that particular game.
And you have real people deciding, this is what's going to happen.
So we'll make up a logical solution.
In the military, that's why you want people who are really experienced.
In a non-military live-action role-playing game, like the kind that's played around MIT, people will still do that.
So I'm in a game where I have a steampunk setting.
And I don't have all the equipment I need to build this machine of death that I'm supposed to be completing.
But I could take apart some of the other machines that are in the world.
And then the game master says, wait a minute.
We've got to put rules about that.
Or, we don't want to start anyone taking apart [INAUDIBLE].
But it makes perfect sense in this world.
So, OK, it's going to take you two hours and you have to be in this room.
But after that, you'll have the parts you need.
The other side of live-action role-playing is that often the people who are controlling the game of live-action say that wasn't fair.
And again, they can improvise along with the players.
So you have that bit of flexibility.
You're limited by how experienced most people are.
People who don't tend to be terribly experienced tend to make rash decisions that end up hurting the playability of the game.
Sometimes they get lucky.
Sometimes they don't.
Let me see, one example early on when I was making live-action role-playing games, was I had a game mechanic where if one of the police officers in this sort of detective scenario wanted to do a background check they could ask the game master.
Some amount of time-- actually it was fairly instant.
Because the very molded kind of system.
The assumption was that they will find a computer and they'll look it up.
And we'll give them information.
And that led to two problems was that, first of all what happens if I'm not physically in the same location as the person who wants the information?
And this is weird to think of a computer with legs walking around.
It sounds very science fiction.
And on the other hand, it means that I needed to have all the information about everybody at my finger tips at all times.
Which often wasn't true.
So someone would come in, I need the information about this character.
I'm performing a look up.
And it's like, I'll get back to you in two hours.
Because I don't have the information here with me I'm going to look it up and I'll give it to you later.
Now I don't make that mistake anymore, but that was the kind of mistake I made when I was a [INAUDIBLE].
Because I hadn't really thought about what that would require from me as a game master.
Now that's a different tension.
I am actually going to read a chunk of this-- [INAUDIBLE].
I talked about dissociation or separation.
And what I'm talking about with that mistake that I made is just plain old feasibility.
Something that you probably have already run into it with the games that you've designed
Yeah
And many of the games that you've designed [INAUDIBLE] could have thought of as just plane of scope.
It's like, how big is this game?
How long is it going to take us to put all these bits together?
[INAUDIBLE].
Can we order these bits in time?
And are they going to be delivered in time for the class?
I know multiple teams had this problem
[INAUDIBLE]
Let me see if I can combine some of these ideas together.
One thing that the Assassin Guild likes to have are people who have locked memories.
They know something, but they don't know that they know it.
Little memory packets, little pieces of paper that they printed out and folded up and stapled.
Takes you about 20 seconds to make one of those.
Now imagine a game that has maybe 60 people.
And each person maybe has five locked memories because the GM's really love the idea.
And 5 times 3 is
That's only about an hour.
Only about an hour
Only about an hour.
Does it ever only actually take an hour to actually produce that?
You get staple cuts and you have to write them all
It's a little less than two hours actually
No way
20 seconds per 60 people
But it's not 20 seconds each
And then what happens is that oh, wow, we have 300 pieces of paper all of which are now folded and almost indistinguishable from each other.
And we've got to figure out who gets which.
And these are the concerns.
That's a feasibility issue.
Doing something like that for one or two players is extremely feasible.
It takes almost no time for the game master.
Doing that for your entire player base, if you have a 60 person game, on the large scale becomes the majority of your prep time.
Suddenly you're setting aside a whole day just to deal with that.
And a game's only three days long or something like that.
With that level of effort.
Things to preserve the game master's sanity.
The MIT janitorial staff, facility staff, are pretty ruthless when it comes to picking up stuff that has fallen on the floor.
So say that's a really important piece of paper that you've taped up on the wall somewhere's here at MIT.
And there's information that this is the place that your players are looking for.
They've combed all of campus and they need to find this location.
And that tape that you used just wasn't that strong and it fell.
And the janitorial staff just picks it up and throws it away.
Now what happens to your game?
Can your game even end?
Are you just going to have a whole bunch of really tired players chasing you down as the game master?
So the way that you resolve this can be built into the game mechanics.
You can build things like multiple redundancy.
Like in addition to a sign there will usually be this other thing in this same area.
If you find one and you don't find the other, it means something has dropped off.
Involve the game master and the game master will go and fix it at some point in time in the future.
If you have a game that has no secret locations, and you have a game that's campus wide, you could just give people a map of all the information.
Sure, you still want stuff on the wall because it's neat to be able to walk into a classroom like this and say, oh mortuary.
That's where I am.
And then suddenly reminded, oh maybe I should be role-playing back then and that sort of thing.
But it's sort of like walking through a space and you have a map that told you you're in a mortuary.
But you weren't looking at your space, so you just walked through the space in entry to the classroom.
Yes, I am going to do my tutoring in this room.
It's a morgue, why are you doing tutoring?
So it's neat to be able to have those things to give players information about what's going on in the space.
And to help them make believe, help them imagine what they could possibly do in the space.
Maybe even give them clues about what they need to be doing in order to advance their goals.
But then you also want to be able to have backup plans for when those things fail.
That's part of feasibility.
You don't want the game to break halfway just because janitorial staff came through or the wind blew or something and now your game sucks.
That's another type of information.
And that's not entirely unlike the information that we've discussed in this class back earlier this semester.
But with live-action games, especially live-action games that don't have any kind of turn based thing, they're happening in real time and they're moving in real-time.
Information overload on the player becomes something that you have to be very careful of when you're designing your game mechanic.
How much is someone expected to know at any given time?
This is going to be very applicable for any of the games that you design for the third assignment.
If you decide to do a live-action game, think about here's somebody who's moving around in a space.
Maybe it's a hospital or it may be the doctor's office.
And this person is trying to remember X numbers of statistics about their character.
How agitated I am, the number that determines how many drugs are part of me right now, and that sort of thing.
You've got these stats.
And once again, too high, then what happens is you've got a character that's standing there trying to do all the math in their head and not role-playing.
If they're role-playing anything, they're role-playing a mathematician or a [INAUDIBLE] in their minds.
And you're losing the immersion, you're losing the [INAUDIBLE] to be I'm supposed to be an angry person.
But I'm standing here trying to do math about how angry I am.
And you've got attention of information and very [INAUDIBLE].
There are times when you actually want that though.
There are ways to take that and turn it into your advantage.
In particular, when the thing that you want people to be doing-- the kind of activity that you want people to engage in-- are the things that players would generally be uncomfortable doing with each other.
Physical violence would be a good example of that.
I want to be able to hurt this other character, but I don't want to hurt this other player.
So what are the things that I can do to bridge that gap?
Insult his mother
Verbally, sure, there are ways to do that.
But you might need rules or at least some guidance from the GM to say, take that seriously.
Because that can very easily drift into the realm of, yeah that's just a joke
Rock, paper, scissors?
Rock, paper, scissors?
That could be a mechanic for a fist fight, for instance.
Or [INAUDIBLE] that you start see them avoiding that
Nerf guns
Nerf guns?
Things that shoot little pieces of plastic.
But odds and evens
How about tokens like in [INAUDIBLE]
Say what?
Health tokens like in [INAUDIBLE]
Give me some more detail
Like if someone was a certain number of flags, and so you hurt them by taking one
So the player might have multiple flags for instance.
OK, gets them angry.
It's like health bar almost, actually
[INAUDIBLE]
So dice rolling is under the general term like things randomizing numbers.
In fact, rock, paper, scissors, is randomizing things almost.
And then you have toy weapons, right?
Probably not so relevant for the game violence.
But you could imagine setting up a room full of foam things which you will be quite happy to throw at each other.
You know that this cylinder of foam, of pipe-insulating material, is not going to hurt anyone when you throw it.
Which makes it more likely that I'm going to pick it up and throw it at someone.
Clearly it's something that's designed for me to throw that the GMs put there.
And if you want your players to be engaging in that kind of activity that's technically-- you could have put a real bottle there.
You could have put a real mug or computer or something.
That's the reason why you as the GM could have, but they are deliberately designing to put something fake there to clue the player into maybe there's something I can do with that I wouldn't necessarily want to do if it was the real thing.
I wouldn't want to throw a real bottle at somebody, but I would easily throw a foam bottle at somebody.
Same thing for Nerf guns, dart guns, this guns, it's like you put something with a trigger in somebody's hands that's going to launch a little cheap piece of plastic at someone?
They're itching for a chance to use it, really.
Let's see, what else?
Dice are difficult.
The problem with dice is you need a flat surface to roll them on, and you might not have that
[INAUDIBLE]
Still a little bit risky, because they could end up going all over the floor
Another thing that I've seen people do is some kind of [INAUDIBLE] style thing.
Where essentially they have [INAUDIBLE] or something, you might have created them.
You can beat a challenge by giving one to the other player that you're fighting with.
And then [INAUDIBLE]
So it's like the amount of energy you've got
Yeah, it's like the amount of energy you've got.
And also, if you discard them or give them to another player then it's the difference between these things are just gone and they circulate around
[INAUDIBLE]
Yeah, so if there's the circulation thing than you can get whooped a few times.
But after being whooped then suddenly you get a chance to whoop back.
And then it becomes less [INAUDIBLE]
That was interesting.
This is the good part that I want to talk about which is [INAUDIBLE].
A topic that you guys are already very familiar with.
But, at what point do your game mechanics-- are you designing your game mechanics not necessarily to enhance the realism or to encourage certain behaviors or to give the player enough information about what they're allowed to do or what they should be doing?
And you're just going into the realm of how do I make this competitively interesting?
Real fights, if you get a bunch of blows, you do not get stronger.
Well, if you're Bruce [?
Bannon ?]
maybe.
But from a competitive point of view, or from a traditional game play point of view, that sounds like a great idea.
That gets you back into the game, whereas any very realistic fight mechanic is really probably going to be a positive feedback mechanic.
You win, you keep winning.
You can think of these five different things as--its not like game balance, but these are decisions that you have to make either as an individual designer or as a team about what any given mechanic in your game is trying to accomplish.
It's rarely mutually exclusive.
It's usually trying to involve multiple one of these things.
You don't want to fall down on any of these, especially not feasibility.
It's like, yes this is the perfect game mechanic!
And we had two years to be able to put together all the things that we had.
And for the games that have failed to run because of that idea.
Especially live-action games that have a hidden computer component, especially if they're designed by MIT students, rapidly designed into a software engineering project that never finishes.
The game kind of disappears and it's all about this damn website that would have solved everything else.
It's completely realistic because of the hacking mechanic that actually happens on the computer.
And the skills scale to your ability as a character which the computer already knows, and you don't have to remember, so you can feel really neat.
And it's competitive because you know what the opponents are doing, but not everything they're doing, and it's going to take us five years to write this code.
The game's never going to happen.
Let's see, let me think about an example just to sort of put some ideas in your head.
Back to the idea of things that players might be uncomfortable doing with each other.
Seduction is one of those things that's-- if you don't have a game mechanic for it, it never happens in the way that you want it to that helps the game.
If it happens, it's happening between the players and that causes all kinds of problems.
So you come up with a rule, come up with a game mechanic, that feels nothing like real-life seduction.
We're going to play a game of blackjack now, if I win you're now in love with me.
And it's like, that's not how seduction actually works, but it actually makes it easier for players to say, I can do that.
I can play a game of blackjack.
There's no social awkwardness in playing a game of blackjack
You can associate that with, I win this game so I slipped inside your glass a love potion so now you're in love with me.
Something like that
But see, in that particular case you're describing an action that people might be more actually willing to do through some sort of simple mechanic.
Like OK, instead of love potion we're going to mechanic that enables you to something.
I'm going to actually give you a file.
And if you drink something in the tea, like maple syrup, talk to the GM.
Because you probably just drank some magic potion.
But if the goal is, all right we're playing a film noire game.
And you're supposed to be engaging in constant witty banter, laden with sexual innuendo, a lot of people have a lot of difficulty with that.
Sometimes it works.
I made it work in some games.
But it makes it a lot easier if the success or failure in that is not going to be bottlenecked by whether the players feel comfortable doing it.
So creating a game mechanic that actually feels a little bit less than what activity you're trying to represent may make it easier for people to engage in the activity.
Same thing goes in combat, playing rock, paper, scissors, feels nothing like an actual fight.
But people will be much more willing to play rock, paper, scissors with each other than to punch each other.
So come up a rule, give them a little plastic gun, give them something to make it easier, and they'll be willing to engage in that activity more often.
Violence in the rule will probably be the thing that comes up a lot in this setting.
So, let me see what else we can get into.
It's about 2 o'clock.
How many folks here have played some variant of Mafia or Werewolf or?
Who's played Mafia in particular?
OK, who's played Werewolf?
So those are live-action games too.
And those are live-action games that specifically don't try to draw too sharp a division between who you are and who your player is.
You are given a role, a card, that basically says whether you're Mafia or whether you're a villager I think
Civilian
And the whole idea is that everybody who's-- that's a character sheet that gives you private, secret, information about the role that you're supposed to play.
But in my opinion, these particular games-- and we're going to play some variants of them-- what you are told by the cards is only what you're allowed to do.
What actually gives you the role is what you decide to do.
I play Mafia and I'm going to say, Patrick needs to die.
And other people in the room agree with me, and Patrick's character gets taken out of the game.
That decision is the decision that made me Mafia.
And that decision is the decision that other people are trying to catch, because they are trying to find a person who's actually responsible for various acts when everybody's eyes are closed.
The general model for these games is there are cycles.
For Werewolf it's a day and night cycle, and for Mafia it's also day and night.
But the idea is that during the night everybody who's not Mafia gets to have their eyes open.
Everybody who is Mafia-- everybody who's not Mafia keeps their eyes closed.
Everyone who is Mafia keeps their eyes open.
And people who are Mafia can communicate through some GM mediator, some game master mediated way, to specify the player to be taken out of the game at this point.
It's usually a point.
I see things with blinks.
Because as the GM moves around sometimes that gives you clues, because you can hear.
Even with your eyes closed you can hear where the GM is.
And then that's like the night cycle.
And during the day cycle everyone opens their eyes and whoever's been taken out of the game informs you.
And everyone knows and everybody starts discussing and accusing each other of being responsible for it.
And the whole point of the game is for the villagers to identify who all the Mafia is before everyone finishes.
It's a live-action game that doesn't have a lot of action.
I would say, it's one action.
There's closing your eyes, opening your eyes, and there's pointing.
There are variants where sometimes there is a Godfather
I've seen websites with whole lists of different roles that are just ridiculous.
Like there's grave keeper, there's necromancers
There's people who can bring characters back from the dead.
There are people who can basically do a background check.
I think that's--
Oh, that's right you told us
-- angels and cops.
And you go from two cycles to three cycles per round.
Four cycles per round where different people have their eyes open.
And you get different bits of information about what the world state is like.
The fact that killing somebody is pointing at them, that's a dissociation mechanic right?
It's not like you're actually being asked to engage in physical combat.
Even though the fiction of the game is that you're assassinating this person in the middle of the night.
The verisimilitude, and competition, will largely come down to the discussion phase.
They are trying to put players in an environment where you've got a paranoid of villagers, a paranoid town, and they're all mutually accusing each other.
And both the act of trying to not be accused, and trying to deduce who is the right person to accuse, is a compelling competitive experience.
That's what all these games are really about.
So all the emphasis, all the time of the game frankly, is devoted to that phase when everyone sits down and discusses with each other who could be the Mafia.
You want to walk us through one of the variants maybe?
Yeah, do you want to actually play it?
Or just describe it?
We have 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.
We could have two groups or one big group
One big group may take awhile
Let's do two groups but how many different rule variants?
I only found one that I thought was really interesting, although we can certainly-- even the Wikipedia page has a million variants
The one I normally would play would be one where there's mafia and there's cops.
There might be a medic
Inspector, angel, and prostitute
What's the prostitute do again?
If you sleep with the mafia they don't kill anybody.
If you sleep with an inspector they get incorrect information.
If you sleep with a nurse they don't heal
This is it's very important to make sure you know what rules that you're playing with before the game starts.
So walk us through the roles, and then what we'll do is we'll break this into two groups.
And we'll have two groups play separately and that's probably the end of the class activity
I had found a variant called Thing, which is based on the John Carpenter movie.
Which is interesting because it takes out one of the roles.
The idea is one person is the shape shifting thing, and at night they infect someone else and that person becomes a thing as well.
So they're not knocked out, out right.
And then during the day the group votes and decides on one person to test.
And if they test a thing that person is killed and they get to test another one.
And the idea is that the things are trying to outnumber the people.
Which is interesting because it takes away that inspector role
Do you know who gets thinged in the middle of the night?
So, I was thinking about this.
What you would need is for the person who is the thing to indicate to the moderator.
And then maybe the moderator could tap them or something
Ok, so if you're a thing you know you're a thing
Yeah.
The things are spreading as opposed to just--
And then if there's two things, do they both just agree on another?
On a third?
Presumably.
So this is really interesting because apparently this is played at a conference for science fiction writers.
So there's little snippets of it on the web about the rules from different blogs and stuff.
And it's not as interesting, but of course the problem is how do you Google for mafia thing?
It's not interesting to me, and then it seems like there's [INAUDIBLE]
So it might be fine actually to have two groups figure out how to play that game just from the description that you've just been given.
I think [INAUDIBLE] there's going to be enough people who've played traditional Mafia to know how it works.
And then we'll just reconvene at the end of it and see what each version people came up with.
What were the similarities and what were the differences.
Again, that's the flexibility of doing a live-action game when the people who are actually coming up the the rules are actually in the same room as the players, or are the players themselves.
You get to make stuff up on the fly as long as everyone agrees.
So let's give it a shot.
While Jason is sorting out, any questions about the stuff?
The two biggest challenges that I've come across in the block mechanics have to do with surgical challenges and intellectual skill.
So let's go intellectual.
It's easy to play a character that's stupider than you.
It's really hard to play a character that's smarter than you.
In particular, it's really hard to play a character that's smarter than you if you don't have a GM around all the time to tell you, stop.
So, if you have any ideas about how to deal with that.
The other one is, I'm creating social conflicts.
Like, this is harder than any role-playing game because it feels like if you want to be able to persuade a character or something you just want to persuade them because you're role-playing right?
But you're supposed to be playing a character who is more persuasive than you.
But it feels really cheap.
If you can just sort of
You believe me now!
It also ties into another area that frequently pops up in LARP's and role-playing games in general which is use of metagame knowledge.
So frequently if you've been persuaded to do something then you've come across players using their metagame knowledge that they don't want to do the thing that they're being mechanically forced to do.
And they're trying to find loopholes constantly.
I don't know if you can really force people to abandon their metagame knowledge mechanically?
But I don't know
I have some examples on how the Assassin's Guild in particular has dealt with that.
And in some ways it's cheating, because we're not going to try to do some of those things.
In particular, let's take persuasion for one of those things.
The problem with persuasion is that whatever mechanic you come to, other than what you see is what you get-- [INAUDIBLE]-- any other mechanic that you come up with runs that risk of the, wait a minute this is not what I really, really, want to do.
So the Assassin's Guild has replaced that with brainwashing.
Which is the whole idea of, you're not being persuaded.
You goals are being forcibly changed, but now you have other goals.
And that's the big difference.
Your old goals are no longer your win conditions.
Your new goals are now your win conditions.
You can still win the game by obeying your new goals
That's cool
The brainwashing mechanism
So you've aligned the interests of the player with the interest of the character
You want to go with your new goals now because-- how this came up, typically in the Assassin's Guild, the Assassin's Guild actually grew out from circle of death games.
Which is, getting to know you.
I have the name of somebody, and I need to shoot them with this water gun.
Who is this person again?
I don't know.
I'm a freshman.
I haven't met everyone on campus.
I'm going to figure out who this person is.
That's where those games came from.
The Assassin's Guild kind of grew up from that.
Then it became teams.
And brainwashing was basically the set of game mechanics that was used for recruiting people to your team.
So it's like, all right everyone's got one target.
But there are a few people out there who's job is destroying everybody on this opposing faction.
And there's only one person in that opposing faction right now.
There's only one of me.
But I have this brainwashing mechanic that recruits more and more people.
So what you thought was one, I'm just against another individual, becomes my group is against that group.
And it feels pretty awesome, actually.
Because what it actually feels-- it's a weird game mechanic way of making your character have some sort of development in the game.
If you think about a role-playing game, character development means your goals now change somehow.
But it's really difficult to give players a means of doing that themselves.
Brainwashing was one of the few ways that seems to actually have worked.
I'm actively going to knock out this character, put this blinky helmet on them for five seconds, and if nobody catches me in this process I give them new goals.
And the person who's [INAUDIBLE], dude this is way more exciting than anything [INAUDIBLE].
And you also avoid the metagame problem of loyalty.
It's like, I got this information from this other group but I've been brainwashed and I don't want to tell you.
It's like, no you want to tell me now because it'll actually help you win.
But metagaming in all kinds of live-action games is a big thing.
You know things about the player or you know things about the game state that you're not really supposed to know.
If a person is supposed to be able to teleport from one end of campus to the other end of campus, the character may be able to do that.
But the damn player's got to walk all the way in between.
And you have to drive the player halfway to the, I now know that player is probably on this side of campus.
Even though your character really had no idea how to deal with that.
That's just one of those-- there are ways to design game mechanics to avoid those situations.
But in many, many, live-action role playing there is a honor system that really has to kick in.
The nice thing about it is that if everyone decides to go by the honor system, and agrees that this honor system is necessary for the world to be reinforced, it actually helps reinforce that mutual illusion that this world is real.
By saying, I know that I'm not supposed to know those things.
But I am going to play this game as hard as I can in a way that's [INAUDIBLE].
Actually reinforces the world for everybody else.
It's the same for improv theatre.
Anyone here involved in improv theatre or improv comedy?
So, you know yes, and?
Sure
What would be the concept behind that?
Yes, and, is an idea where if you give someone an offer they'll take your offer.
So if I say, man we're all out of tomatoes.
Someone will say, yes and the big tomato cook-off is tomorrow or something like this
You cannot contradict what was said
Not only do you not contradict, you always add information.
Every line, at least at the beginning of the scene, should be adding information to the scene.
And then after a few lines you have a very, very, elaborate set up that's kind of ridiculous
But, mutually shared?
Exactly
So, in that same situation.
Metagaming is one of those things that's actively working to destroy the reality of your world.
And if all the players agree to try their damn hardest to not let it interfere with the world [INAUDIBLE].
The illusion in a way that it strengthens the scene for improv theatre
Hold on, just one sec, the other concern that you had before persuasion issue which was?
Characters who, in particular this comes up a lot in this [INAUDIBLE] type situations or what have you, where you have somebody's who either a super genius
Oh, so someone who's better than you at something
Much better than you.
How do you mechanic that?
The best way that you can do that is to cast the right person.
But what I have generally found is that old role-playing game chestnut of separating intelligence from wisdom makes it a little bit easier.
There's intelligence, which is the ability to think on the feet, and there's wisdom which is how much you know about the world.
And you as the game master designing a game can figure out, when I say someone's really smart, what do I mean?
Intelligence or wisdom?
It helps you a little bit on thinking how I can use game mechanics or game writing to fix that problem.
So if it's intelligence it's just like, oh I have the ability to do stuff that other people can't do because I'm so smart.
And you can literally just give them abilities to act on things that other people do not have.
I'm able to build a telephone out of this Coke because I'm a professor of Gilligan's Island.
I'm smart.
It doesn't mean I have to act terribly smart.
I'm capable of doing things that other people can't because I've been told specifically I have these abilities.
But that's the intelligence side of it.
The wisdom side of it is a little bit harder.
Which is basically, you dump a whole bunch of information on somebody.
And you hope that they are capable of making associations between what's happening in game and what they read.
That's a lot harder.
It is a problem, but I find that by separating those two it makes it an easier problem.
Patrick?
Yeah, this is a bit of a noobish question.
Are LARPs usually played in real time?
Typically, they have rules about when to hit the pause button.
Basically at MIT those are usually enacted when safety becomes an issue.
People are running too fast.
Somebody just fell down and you want to be able to stop everyone from running, and so they shout halt.
Typically they're happening in real time because it's really hard to do turn based stuff that doesn't quickly feel like a board game
I mean like, when you say someone would want to do a background check and you'd say I don't know, come back in two hours.
You'd literally wait two hours?
So, yeah, often that really means just two hours.
And that's an interesting thing of that it's real time but it's kind of a time delay at the same time.
So as a game master you can build in game mechanics that take advantage of real time delays to be able to give you as game master some time to breathe.
Typically they're happening in real time.
I wrote a game where the clock stopped somewhere between midnight and 8:00 AM.
The game world literally froze, well not literally figuratively froze, as well.
And one minute after 8:00 the clock would practically, in game terms, have moved one minute even though there were eight intervening hours in between so that people could go and take a break.
Which would be great for an Army of Darkness type of scenario.
[INAUDIBLE].
But those three films as opposed to it all happened within 48 hours.
So Andrew?
[INAUDIBLE]
So I guess two things, one is in terms of wisdom.
Could you possibly have little note cards that are flipped over everywhere?
And then the wisdom person, the person with high wisdom, could read about information in there?
You could certainly have taken advantage of a piece of paper taped on the wall just put in the back, right?
Some people can arguably be able to read off the back of the paper, some people aren't.
Which will be a little bit easier than trying to keep it all in your head at once.
You can also give very limited wisdom, like the whole psychic example, right?
I am sensitive in some way.
I can tell when certain things are cursed.
By giving them clues on printed pieces of paper that nobody else knows how to interpret.
I see there is a number associated with each one of these rooms.
That number happens to be even.
This is not a place that any sane person would want to be.
And there are people who have the wisdom to tell that and there are some people who don't.
That's another way to put those environmental clues.
Particularly in MIT, hiding clues in plain sight, and every single way that you can possibly do that, is really exploited.
So the numbers for instance.
I've seen games where everyone has five digits on their badge.
And every single one of those digits means something.
If you happen to have the ability to interpret them, but each one of the digits means something different.
So that's an example
And just really quickly.
In terms of space, how much space do Assassin's Guild games take?
MIT Assassin's Guild has the advantage of the MIT campus.
So it can go anywhere from main campus mounted by Mass Ave. and Main Street, to all over campus.
But players hate that.
Players hate walking all over campus.
It kills their feet.
So often it just turns out to be the few rooms that they could get room reservations for, the MIT campus activities complex, and the sort of general vicinity around there.
Outdoors is really not used much, because MIT Assassin's Guild loves shooting little plastic things at each other.
And when the wind blows that just sucks.
Jeremy?
Do those eight story [INAUDIBLE] games that have gone way horribly, fallen apart
Oh, absolutely.
How much time have we got?
Can you highlight what are the mistakes that have made-- everyone who's doing these, especially on my team, they're all smart people.
And they're written by smart people.
So what hazard mistakes do smart people still make that results in these games falling apart?
Or bad things happen?
Well, the most common thing that happens is not even an issue of how smart the game writers were but how much experience they've had in seeing things go horribly wrong
Right, which is why I'm asking
Just like this class has given you experience making games, you've seen how some projects can go wrong.
A lot of it is just system dynamics, right?
I put together a bunch of rules.
I didn't expect those rules to interact in that way.
But because a live-action game is difficult to run multiple times, the first time you see--
How do you play a live-action game that lasts to a month?
That actually came up in the forums I believe.
Somebody was asking about this.
You can at best to a test--
Give them mechanics?
--You can play the sort of game mechanics.
Write down the stripped down version which takes out all of the plot motivations and just has the combat.
This is how combat plays.
I did that for a Matrix game once.
And it's like, I'm writing a game of a Matrix.
Combat is really freaking important.
So I just made an entire game where all it was, you want to go in and get this thing and get out.
That was the whole rule.
And here's your three pages of contact notes.
And people were like holy shit, this is a lot to remember.
And that showed me the problems that I had.
Unfortunately, it wasn't so catastrophic that I would use it as a catastrophic game example
Tell us about a catastrophic
Catastrophic game examples
Tell us someone tripped and hurt themselves.
[INAUDIBLE]
No, that's not what you're looking for I understand.
It was a game where everyone was playing undead.
I think they were vampires.
So they could get knocked down if they got shot, and they would get up again in five minutes because they were immortal.
Unless they got staked or something they would get up again.
Unfortunately, there were a lot of vampires in the game.
Often many of them were in different factions.
So they were going after each other with guns.
So when two people who were obviously against each other shot each other, fell over, they would get up in five minutes and shoot each other again.
Repeatedly begging for someone to show up and do something to break this deadlock.
And that's one of those things where it's like game mechanically you know what's supposed to happen, and it sucks.
So that's one example
Do we have to wait five minutes?
Can we just continue to shoot each other?
But the issue is, again, these games are happening in real time.
So the five minute rule is there possibly for a reason.
While you are down for five minutes those five minutes become very valuable for someone who wants you down for those five minutes.
You can kill me.
I can kill you.
But both of us can knock each other down for five minutes.
And that's a great delaying tactic.
And I can send one of my friends and do something that I don't want you interfering with.
And if that's the case then that's great.
But often it was just, oh crap there's no way to break the deadlock
You'd figure the limiting factor would be ammunition though
I think that game actually let you pick up your ammunition all you want.
Because it can be a really annoying limitation to run out of ammunition.
Especially if you are playing with the little rubber dart guns because carrying a whole pile of them is really annoying.
So if the rule is, I get to pick them up, then you have the allusion of having a lot of ammunition without having to carry a lot of ammunition.
So that's one example of a catastrophic failure.
There's a really funny quote regarding brainwashing actually, which is where were you?
I was not being brainwashed.
Who were you with?
Not with the people who were brainwashing me.
Back to the problem with the persuasion, right?
It's like, I want to tell you something because my goals haven't changed far enough to want to align me to my new goals.
So I'm going to do everything that I can without breaking the rules.
I'm not allowed to tell you that I was being brainwashed.
So I can say that I wasn't being brainwashed.
Other catastrophic?
How long do you think the thing will take probably?
I don't know.
Probably about as long as a game of Mafia
OK, so like half an hour?
Depends on the people
If you put a time limit on discussions then, because I've played in the past where it's lasted hours.
Because there's no cap and people just went into these long discussions in between rounds.
So if you put a cap like OK, it's going to be this long a discussion and then we're going to kill someone off
Let's see, I had wanted to read
Played online Mafia?
It's a totally different experience
Chat involved or something?
Forums usually.
So basically, when you can look at the history of what people have said and who they've accused and what order they did it in, it completely changes how Mafia turns out, right?
Because all of a sudden, people start going, back in the history and compiling statistics [INAUDIBLE].
And who said what and then who must be on their side if that's the case.
But there's this kind of meta knowledge that you can use with this if you are Mafia because you can tell [INAUDIBLE].
We were able to communicate at the Mafia privately.
That meant we could for instance do things like, make sure we were never the first people to accuse anyone.
So that's a Mafia truth, never do that.
Because it immediately makes you look suspicious later on if somebody turns out to be the Mafia.
There's no way that you can be the first person who accuses.
But bandwagoning somebody who's not in the Mafia is always great.
So you can coordinate to do the sort of thing to try and defeat the statistical analysis that somebody else is doing
You've got this whole shadow governance theory
That's a great [INAUDIBLE] tactic to try and get you away from my team is to actually accuse someone on your team.
And hope that they don't actually vote them off.
I've definitely been in a game where we purposely killed off one of the people on our team just so that we wouldn't be accused.
Oh, he cannot be on Mafia because he killed off a Mafia.
OK, and then we end up winning because no one figured that out
I mean, that's one of the best things about all kinds of live-action games is that it almost encourages people to look for these slightly out of the box illusions.
These sort of emergent I'm not really going to expect that but it still fits within the rules.
Because we are dealing with real people for the most part and you're not limited to the cards you're allowed to play or the turns you're allowed to take.
So taking advantage of that.
I have one more disaster story
I want to hear it
I'll limit it to just one for the people who don't.
This was a game where there was a virus that was spreading.
People who are in Assassin's Guild might have already heard this one.
The whole basic idea is that there was a explosive nano-virus.
People who got infected after a certain amount of time would explode and anybody in the room with them would now be infected.
Of course this was a game-- a game set in science fiction kind of thing-- and we've got doctors who are capable.
Who have mechanics to be able to research cures.
However when the doctor happened to be one of the first people to die, what rapidly happens is that what should have been a one night game turned out to be a one hour game and everybody exploded.
I guess that might come down to feasibility.
But you can think about it as you need a certain amount of redundancy when you're designing any game, but particularly live-actions games.
Because shit happens.
And you've got to expect that to happen.
There's always this fear about, oh I put in too many resources into this game and the players are going to find it too easy.
And more often than not, yeah, there's some players that that are going to say we breezed through that.
But, rarely do people get terribly upset in a live-action game when something is really too easy.
They get really upset when it comes to the end and it seems to be impossible.
Oh, so the GMs only made it possible for one person to fix this problem?
And they didn't think about that person just dying in the beginning of the game?
So that was another example.
That [INAUDIBLE], I'm sorry.
I feel really bad about bringing up examples from my games, but I don't know which ones are the ones that I genuinely-- Oh, I do have one more.
So, this was after The Sims came out.
And people should really be worried about that.
I thought having a bladder stat would be a really good idea for a game.
This was a game that was set in a train that was steam pumped.
So tea was the game.
Tea was in fact the combat drug.
If you drank tea, your combat status would increase but so would your bladder stat.
Once your bladder stat was passed a certain point all your combat stats went down and you really had to go to the restroom.
Which was really interesting, most people just reviled that game.
Because there was one more number they had to hit that was stupid annoying and really didn't do terribly much for the immersion of the game.
I was thinking, it's a game about travelling on the train.
So you use the restroom all the time.
That's the only thing I do when I'm travelling.
I use the restroom.
[INAUDIBLE].
But that led to some interesting emergent situations.
Because when this train gets attacked, and everybody needs to find a safe place to hide, what they all do is they head into the washroom, barricade it, and drink tea until all their combat stats go out the roof and then win the fight.
Because they were right there in the restroom they could afford to do that.
It was hilarious
Some of these people come up with exploits
They may hate the rules, but once they understand how the systems work they're going to come up with ways to use it to their advantage.
And that can be some of the most enjoyable moments
I saw a squirrel this past weekend, and he figured out if he stood under a bird feeding thing up in the tree it could just get free food because when the birds ate it once every two to three minutes they would drop a grain inevitably.
And there was this really fat squirrel just sitting there.
He spent the entire day sitting right [INAUDIBLE].
So it's like it's so natural to try and find exploits of everything and work it to your advantage
There is a moral--
It doesn't work for [INAUDIBLE] but you still do it anyway
There is a moral to that.
Is that the live-action games, especially if you're playing in the live-action game community, there is whatever you write down as a rule we put it up on the wall and they put it on a piece of paper.
Someone's going to try to do it.
And you have to think of, I'm going to put in a rule that's so ridiculous that people are not going to want to try and do it because it's going to hurt them or something.
Like jump on this spot 100 times and you will unlock this door.
Or you could find someone with lock picking, and you'll just assume that person's going to find someone with lock picking.
No, someone's going to jump on the spot 100 times and they're going to hate you for it because you put the rule in there.
So try not to hurt your players, would be the moral, because they will do what you tell them to do
Do you guys sign disclosure forms?
All right, that's a [INAUDIBLE]
Before we go, I want to just clarify this version of the investigation mechanic.
So, the group of scientists gets to test two people.
If you catch the thing then they're dead and out.
If you catch them on the second time-- this is what I've read-- if you catch them on second guess you get another guess.
I'm not sure if that means you can have infinite guesses or not.
So that's something to play around with
If you test the wrong person who's not the thing they don't die?
No
So I test someone the first time, they're the thing, then they're out.
But I test someone the second time, regardless of whether or not I tested a thing the first time
Yes.
So you always get at least two guesses
But if you test someone on the second one they don't die?
If they're the thing they're dead
The thing dies.
If you test someone on the first time and they're the thing you don't get an extra guess?
Well, not necessarily
You get two
You always get two
So if you guess any time [INAUDIBLE] but if you guess a thing and then not a thing you get a player
I think you can catch the thing really quickly
I think you can catch the thing without actually doing anything sort of like making a system where you automatically catch the thing.
You're getting two guesses so in the ratio of randomly guessing you'll--
And you can certainly change it right?
This is just what I've found.
And it seemed like an interesting thing, the whole point of the part was
Which thing do you start with?
No, you're not [INAUDIBLE].
Because as I recall you would infect someone first
Because you're going to have two unprescribed
This seems really like biased towards non-infected
Well we'll find out.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
Some content in this audio is not covered under a Creative Commons license.
For more information see the course materials on the MIT OpenCourseWare website
All right.
So this, of course, does not actually mean that you have more time for assignment three.
In fact it eats a little bit of your time for assignment three.
Sorry about that, but I think this is a net gain for everyone.
Right.
With that out of the way, Mia Consalvo.
Actually, a lot of you probably are already familiar, you might have taken any of her classes.
She is the visiting lecturer-- visiting professor, visiting professor-- here at GAMBIT.
Are you teaching something in the spring?
Yes.
Cheating and ethics in games.
Brian can tell you all about it
You can take this as the sneak preview--
Yes.
This is a taster of a little bit of what we do.
Thanks.
OK.
So it sounds like you all have lots of opinions, which will be good.
So to get started-- and actually, I don't like the term lecturer because I don't get to lecture very much-- I'm going to start by handing around some index cards and I want you to take a few minutes and generate some examples of ethical elements, procedures, episodes, games, the more specific you can be the better.
And I'm actually going to collect these at the end because, as Phil mentioned, I'm teaching this ethics and games class and I want to bring in more examples of this stuff.
So this will actually be useful for me
Talking about in games
Yeah.
So we were just talking about Fable, but if you could think of a more specific example from it, a level, different choices that you get with characters in certain games, whatever you can come up with in terms of are there interesting ethical dilemmas, situations
Wait.
Is this within the context of the game, or are we talking about in a multiplayer game your conduct with other players, and this kind of thing?
Or, when I was in Owen's office whether or not to steal the swag
You can do it within MMOs as well, or a multiplayer game.
You can continue to write examples [UNINTELLIGIBLE] if you can think of them.
But just to get you started--
[UNINTELLIGIBLE]
Well then you can start by giving me an example
Sure.
OK.
So I have a couple.
Fighting games, obviously, there's always kind of questions and debate about whether something's cheap or viable or should you be allowed to do it.
Yeah, so if maybe-- I'm trying to think how to describe it-- so, people who play casually will tend to say, oh, if you do this with this character then it's cheap and you shouldn't do that, whereas the community at large and people who play seriously tend to say, well, if you can do something you should be allowed to do it, and if you can't overcome it that's your own fault
OK.
So violating a community norm?
Yeah [INTERPOSING VOICES]
Even if it's allowed in the rules but it's against a community norm.
OK.
It's like using [UNINTELLIGIBLE].
You hadn't thought about that.
Evan?
[UNINTELLIGIBLE]
Oh, sorry
I have a bunch of stuff, which is related to griefing just generally.
So let's just go for the one which is one of my favorites.
So this is griefing for comic effect, or role playing griefing but doing it in a way which you are then going to write a blog about.
And from your character's perspective the griefing makes sense.
So there is a particular player that played EverQuest.
His name is Fansy the Famous Bard.
He was on the good team's side and he griefed evil because evil is bad.
So people who had played evil characters, he would run around and annoy in a really frustrating way, which eventually caused the makers of EverQuest to change the rules of the server so that people could kill him, where previously they weren't allowed to
What did he do?
There was this mechanic in EverQuest where if you were close to a monster it became aggressive and attacked you.
So he had a low level character who was able to run really fast and--
I have heard of him
--he would run around and then run over the top of evil people, and then the monsters would detach from him and then kill them.
Oh yeah, and the catch was that the server was a mobile server.
Aside from-- that is why-- so you can't complain to GMs about a player's behavior, aside from there was a built-in protection for players so that if they're level four or below-- so a very low level-- they can't be killed by higher level players.
And so he just sat on the cusp of that.
Yet there's still a way to use his ability to go around and grief people from a higher level.
It was pretty great.
And he did it all totally in character as well, which is what makes this a good grief
Is that me?
OK.
I think of Portal when they ask you to use your portable [UNINTELLIGIBLE]
[UNINTELLIGIBLE].
You get to burn up the Cube
Yeah.
You don't want to, and they make you.
And you've tried to leave without it a few times and you try to break the machine and it doesn't work.
It's very sad
I feel bad.
[INAUDIBLE] OK. How about you?
I thought of Jason Rohrer's Sleep Is Death
What is that like?
It's a more of like a story telling tool where you have one player that can control the 2D world and then you have one player who is the player, and they can interact with it.
And each player has 30 seconds to make some kind of action, and then you make a story going back and forth.
And what happens is-- you can play online with people, and the controller character or player will have some kind of story that they want to tell.
And so it's up to the player whether or not they want to go along with the story and act like a character in that world or try and break the story
OK. What's the name of it again?
Sleep Is Death
[INAUDIBLE].
I forgot what it's called
Actually that's more griefing, would you say?
Kind of
Is that by the same person as [INAUDIBLE] Passage?
Yes
In Ocarina of Time there's the treasure box gambling game.
[UNINTELLIGIBLE] achieve if you use the Lens of Truth.
[UNINTELLIGIBLE] So there's a built-in mechanic that allows you to cheat
[UNINTELLIGIBLE] legit
Yeah [UNINTELLIGIBLE] use it anyway so it's not like a [UNINTELLIGIBLE]
I just like include the [UNINTELLIGIBLE]
Lucky man
So in Civ IV for each and every sector there's some sort of slavery mechanic, but I guess in Civ IV's example I know.
Many players don't seem to mind the fact that what is considered slavery is ethically wrong, but in the game it's affectionately called whipping, and it's considered a very efficient way of converting food--
What's it called?
Whipping
Whipping
You whip and then you [INTERPOSING VOICES]
OK.
So it's efficient but it's slavery
You use granaries to offset the population loss, and if you [UNINTELLIGIBLE] farms it becomes a very effective way to produce, especially in the [UNINTELLIGIBLE]
[UNINTELLIGIBLE] a lot of people that way
So in The Sims [INAUDIBLE]
But you have to watch them flail around and die
And they get tired and [INAUDIBLE]
A more obscure example that goes along with [UNINTELLIGIBLE] griefing is discussions over the point in the game the community is in MMOs.
Specifically, there was apparently a huge problem in an obscure MMO called Skyress where a large community of the players was there to basically anthropomorph and RP.
A large community of the players was there to play to win because it had a large combat economy system.
And this, tactically, almost tore the game apart because it was such a small number of people over what should be the point to this game when, mechanically, somebody thinks they're alive.
Also in Canara Rey you can blow up people, and they'll even put the star.
It was traumatizing
Yeah, but they're in a giant ball of awesome
That's true
Well, it's not really murder but--
I didn't want to get starred.
[INTERPOSING VOICES]
It's starification
If they don't suffocate on the inside of the kevlar first.
[INTERPOSING VOICES]
It's like the Borg, once you're in there you know it's a better way of [UNINTELLIGIBLE].
You're grateful
So in F.E.A.R there's a nail gun, and when you kill people with it the game does a really good job of spectacularly nailing them to surfaces in the environment.
So if someone jumps and you shoot them in the shoulder you can nail them up to the ceiling.
And if you nail them multiple times you can have them hanging by a hand, and by their head, and by a foot from the ceiling or from the wall.
Anyway, [INTERPOSING VOICES] on forums there were entire threads devoted to sharing screenshots of your most spectacular nails, just the absolutely, most gruesome-- five nails to the eye suspending a guy from [UNINTELLIGIBLE] kind of thing.
And it's pretty-- Anyway, and there's a lot of debate on these threads because most people will be like, yeah, look at this picture this one's awesome and everything it has blood spatters everywhere and all that.
But every third or fourth post it would be like, you guys are so messed up, this is wrong.
And it's not real, you idiot, but that fact that you like it is still real.
And people would argue at great lengths over this.
And I remember chancing upon these forums and being like, is this messed up?
I'm not sure
So is it more efficient in the game to use the nail gun to kill people than other [INTERPOSING VOICES]
It's a hell of a lot more fun
OK.
But it's not like the game forces you to do it if you would not want to
No it doesn't.
You could use other guns, but at some point, if you don't have enough ammo for the other guns you would switch to the nail gun, I guess.
it's a good gun.
Shotgun is fun too.
It disintegrates people at short range
I've now heard they're [UNINTELLIGIBLE].
[INTERPOSING VOICES]
[UNINTELLIGIBLE] turn people into skeletons
Yes.
That one was in the expansion.
Yes.
that was the original, though.
The sniper rifle is a rail gun that shoots a beam of electricity and turns people into big [UNINTELLIGIBLE] of flesh off of their bones
Delightful.
Grand Theft Auto, you can get a prostitute for money, you get help.
And then eventually you kill the prostitute, and get your money back
[UNINTELLIGIBLE]
And then some of the ones related to that is BioShock where you can either kill the little sisters and harvest them for lots of good, or lots of points, rather, or save them and get a small amount of points but you feel better about yourself, which actually in the end, ends us being the better way to go
It's like you get more if you save them?
If you kill them and harvest them you get 10 points, let's say.
If you save them you get five points.
But then after 10 saves you get a bonus of 100 points.
So it ends up being more efficient in the long run to save them, more efficient in the short run to kill them
And also if you're going to kill them you should kill them all.
Don't just knock off one
In Age of Empires III when you are colonizing your colony you have the option of exploiting the native people to help you fight.
They just help you fight
How is this presented to exploit natives?
You establish an outpost inside of their residence area and you supposedly train with them.
There's nothing really there in the game, you just literally put a house of yours inside of their colony
Also Microsoft's selling that for $0.10
Empires III?
Yeah
Why?
I think it was just yesterday.
I'm not sure.
[INTERPOSING VOICES]
In inFAMOUS you can choose whether to be bad or good, and [UNINTELLIGIBLE] other people or to help them survive in a post-apocalyptic world.
And I guess you can do the same thing in Fallout
Well also, inFAMOUS forces you more to choose.
Remember Ash was playing that in our class?
And she said you really had to go one way or the other to get-- on either end of the spectrum there were special abilities.
If you stayed in the middle you got nothing.
Although there are certain things you can build off of
So there's also getting a following.
You can gain people that travel with you.
Like based off if you're good, neutral, or bad
I'm sorry.
I was going to say, in Fallout 3 there's one particularly really good example where you get trapped in this dream world that this guy is creating for you and a bunch of other people, which you don't know that it's a dream world at first.
You know it's a dream or something but you don't know that these are real people and they're dreaming with you.
And he tells you to kill the other people in the dream in order to let you get out because he is in complete control of it.
You can kill them and you'll get out fine and all the people will be dead.
Or there's ways that aren't-- you're not told but there are ways to just go around all of it and just get out without killing anyone
They stole that from-- in Baldur's Gate 2 there's a similar thing
I was actually going to talk [INTERPOSING VOICES]
So the first thing I would mention is a Emerald Items
Just one thing
God of War, [UNINTELLIGIBLE], how so many times you'll be fighting monsters and all of a sudden a bunch of civilians come by and you just let them go, but there's no fun there.
If you kill them you get life and experience so it's--
[INTERPOSING VOICES] reward you for killing them?
Yeah
Oh that's bizarre
You get your life back
Maybe some [INTERPOSING VOICES] do that in the story
Huh?
You're Kratos
Oh, you are the God of War
Well, you become the God of-- well, pretty much you just go around killing, hacking and slashing everything up
OK.
So these are just civilians who are doing nothing
Yes, they're just running away from everybody for their lives from scary monsters, and then you come and just hack them all into pieces.
Also in the game, the second one, in order to pass on you have to translate something.
And to translate it you smash your head into it and then they say you have to sacrifice it in blood so you just smash them into pieces until they have the blood sacrifice
It feels so good
Do you have to do that?
Yeah
Oh,
In Indigo Prophecy you switch between different sets of characters.
And one of them is a guy who commited a murder, and then you switch to the cops who are trying to catch him.
In one of the earlier scenes you play as a cop you bring in a witness and you're trying to make up the artist rendering of what the murderer looks like.
And so you can choose whether or not to make a faithful rendering and bring the cops closer to catching him or to make one that looks nothing like him so you can evade them when you play as the other guy
Interesting.
If you make it not look like you as killer--
Then there are other clues he has to find in order to finally track him down
Can you--
How do you win this game?
Yeah
[UNINTELLIGIBLE] events
Yeah.
[INTERPOSING VOICES]
I mean is the goal to catch the killer, or for the killer to get away, or you just--
It's actually an adventure game so the goal is to maintain the story.
[UNINTELLIGIBLE]
OK. Philip
In a LARP that I played a long time ago I was allowed to draw one item from a location every hour.
The game ran from noon to midnight on three different days but I couldn't play before 2:00 PM.
So I come in at 2:00 PM.
How many items was I allowed to draw when I came into the game given that I already missed two hours of the game?
And if the answer is, well, I can make that up, then what happens if I miss an hour half way through the-- and if I forget.
No one's looking into the box
What did you do?
What did I do?
Yes
What did I do?
I think I drew more than one item at the beginning of the game and I made sure I never forgot.
So, honor system.
A lot of LARPs are based on the honor system
So in Final Fantasy VII, like a lot of Japanese RPGs, you get used to the idea of walking into people's houses and just digging in the items you find, except in FF VII they actually mix it up a couple of times.
There are a couple of instances where you go in there and do that and maybe it's fairly insignificant, you could buy it easily with a couple pieces of gold or gil or whatever it is.
And then the person walks out of their kitchen or something, and it's, did you just take my item?
And you have the option saying yes or no.
And if you say no then you get to keep it but if you say yes they're, what the hell?
Give it back.
I thought it was interesting
I have something that hasn't really been touched on.
When you're running a very large scale Guild in an MMO you reach a point where you need to distribute from these high end The community optimal thing to do is for the people who are in charge of the Guild to allocate who gets what so that you can, for instance, kick out the characters that's most important for the Guild to have for the [?
game ?]
to progress.
But for the most part, actual members of the Guild don't really like that because you would like to receive some kind of reward for having put in time to the Guild.
And going a long time without ever having gotten anything is really frustrating, so people come up with these intricate systems for DKP for distributing your resources, essentially, to try to balance the interests of the individual and the interest of the community
How many of you are unsure what DKP is?
Dragon Kill Points.
There are these sometimes elaborate schemes for if you participate in these raids, you get so many points.
And the idea is the Guild claims everything that drops physically.
And you build up points, and at some point you have enough to buy a certain item.
So let's say you're on a raid and you just got something really cool last time.
If something drops this time that you could really use, even though you're there you don't necessarily get it because you just spent all your DKP.
And there's lots of Guild drama over how these DKPs--
Primary source of Guild drama
Have you been playing?
Hm?
Dan, have you been playing this?
You end up by raids and then you catch them before breakfast
And there are black out dates
Running one of these things is actually surprisingly complicated and hard, particularly-- so my little experience, I ran a Guild which incorporated this.
One problem that happens is if you have fixed prices for things then you get DKP inflation as prices change.
And what happens is-- and it's incredibly complicated-- to where, now, you have to reprice everything constantly.
So instead we went for an auction-based system where people bid on things.
And then prices correct themselves.
However, people can collude in auctions, which then causes a resentment in people who aren't in on the collusion, and the whole thing is just a nightmare.
And now I don't run a Guild anymore and I'm really happy
In games like Pokemon or Animal Crossing you can decide whether or not to change the time on your DNS or game cube.
[UNINTELLIGIBLE] score came out I was like 10 I had a bed time.
I didn't have a lot of time to catch up on Pokemon
You can do that too and manipulate the RNG and get the stats you want.
Perfect for [INTERPOSING VOICES]
There's also-- besides whenever you do the random number generator hacks in games in order to get a certain drop that's normally 1% but you can increase it to up to almost 100-- but what I was going to say was that in Fallout 3 you can pickpocket people.
And something you do when you pickpocket people, as long as you don't get caught, you can take a grenade from your inventory and place it in their pocket, and all of a sudden they're, oh my god, there's a grenade, run away.
And it will blow up inside their pocket and they will explode
What game?
Fallout 3.
And the funny thing is that that's actually an achievement to blow someone up
Do you get, other than the joy and giggles--
Well besides from the joy and giggles, [INTERPOSING VOICES] you can pick their pockets, whatever they have in their pocket you can get
That is still left
Well you take it out as you're putting the grenade in
You wouldn't want their [UNINTELLIGIBLE] to be damaged by the grenade.
[INTERPOSING VOICES]
Yeah, you're just putting them in at the same time
[UNINTELLIGIBLE]
And it can be a little tricky.
[INTERPOSING VOICES]
OK.
Anybody have any representative examples we haven't touched on?
I used to play a lot of [UNINTELLIGIBLE] online and they had a bunch of really sophisticated rules that applied in certain situations.
So there were a lot of times in the game where you just flat out lie to people and have them go into situations that were unsafe for them.
For example, if people were in your house-- in an urban [UNINTELLIGIBLE] game if people are inside your house they are automatically flagged as attackable by you because they're in your house and you might not have let them in.
And they can run outside their house and they'll be unattackable by you.
But you can also put a trash barrel in front of your door, which only you can destroy and nobody else can destroy, and they can't run through that.
So you would invite them in to trade them something or enjoying your meal.
And they're in your house, then you put a trash barrel there so they can never leave and just kill them
Or just leave them there, right?
Or just leave them.
There are tons and tons of rules.
[INTERPOSING VOICES] It's really bad, It's really bad.
I was part of the Guild that did this to me and then I [UNINTELLIGIBLE] I was like, that is pretty sick.
There are a bunch of way to steal tons and tons of money from people and they didn't realize that you did
Did you get that [UNINTELLIGIBLE]?
Yeah.
Yes.
But I feel like just blatantly lying to people as a tactic instead of just being [UNINTELLIGIBLE]
The question about real world money for in game items especially for competitive games whether or not it's allowed by the GMs.
Those are two different questions.
One is the ethical question from the player's standpoint.
And the other is what does it mean in terms of the player character relation in the game
I was going to say one thing that comes up especially in board games that have a lot of positive feedback, it's called king making.
Good examples are like in Monopoly, or Settlers or Munchkin.
And how it works, let's say you and Owen and I are playing Settlers and typically what will happen is say Owen is winning but I'm typically really far behind, or someone is going to be in a position where they have no chance of winning.
So if I decide he's winning and you're close second and I'm distant third, what people do is say, hey Mia, I will give you everything for your one little card and then you can win because I'm mad at Owen.
And then it comes up, like you said, in Monopoly there's this question of is that a really jerky thing to do?
The rules don't disallow it.
It's an ongoing debate.
[UNINTELLIGIBLE]
Don't do that
Similarly in Monopoly, the rules don't specifically allow If I have two [UNINTELLIGIBLE] we'll exchange that and promise each other immunity on our properties or something that you're not allowed to actually promise each other.
Or if [UNINTELLIGIBLE] will always do it to you and will promise me half the profits or something like that where clearly the third person is now shut out of the game and can't win because [UNINTELLIGIBLE] advantage even though we're [UNINTELLIGIBLE] with things that aren't explicitly allowed in the rules
How many of you have played Dragon Age?
I've seen it played
OK.
I've just, I'm playing through it right now.
I've been thinking about these issues a lot.
Actually on my first play through there's a scene where there's a kid possessed by a demon.
You have the choice whether you kill him or not.
The first time I killed him and-- I thought everybody did.
I had no compunction about it.
It was like die demon, demon die.
It turns out everybody really hated that, all the other characters, my companions.
And this play time through I was I'll try it.
And it was all this extra work involved.
But it seems like in the game whether you want to do the good thing or not it generally works to your advantage to do the good thing because you get more stuff, either better approval ratings from your companions who then unlock bonuses or you get stuff.
Whereas, being the jerk--
It kind of depends on who your companions are
Does it?
Who approves?
People like the--
Like Elven assassin, I can't remember what his name is
Oh, Zevran
[UNINTELLIGIBLE] but she likes all the jerky things
OK.
I didn't have Zevran the first time through.
That's why [UNINTELLIGIBLE]
Also that one guy, [UNINTELLIGIBLE] that can spot [INTERPOSING VOICES] No, no, another guy.
I think you can get him on your team at least.
He's in the temple where there's that big dragon and where you're doing some ash quest or something.
And he wants you to pour dragon blood on the ashes or something.
I'm pretty sure you can get him on your team.
Am I right or am I wrong?
But I think you can get them as part of your final army.
And they're pretty good, and they're bad
I just chose poorly, apparently, my companions [INAUDIBLE] moralistic ones like kill a kid.
OK.
In the reading you did [UNINTELLIGIBLE] talks about different choices and the way that game designers are putting games together.
And he talks about two different categories, the open and the closed ethical systems.
The open was more letting players determine for themselves [UNINTELLIGIBLE] they want to take and kind of seeing how that goes through, whereas the closed is the system that forces a particular way of playing upon you.
And so, just kind of going through these here, where do you see them falling in terms of closed and open?
Wait.
You confused me with those [UNINTELLIGIBLE].
You said open means that you have the decision on how to value actions or on what to do?
Well he says, for example, in The Sims with drowning, he mentions The Sims is an open game where you can do what you want and think about what happens
A definitely closed example, it's not up there, but I'll give you a closed example up there [UNINTELLIGIBLE].
In the first Modern Warfare game they have a flashback sequence where you have to snipe the featured bad guy, [UNINTELLIGIBLE].
And even though in the game story he survives your attempt to kill him, if you don't actually hit him with the bullet the game won't let you progress.
So you have to go and basically snipe this dude no matter what, the game won't let you do it.
And that's just part of the story, that's just how it works.
And it's dubiously moral that you should be meddling with such affairs but you know.
And it actually comes back to bite you later in the game because you're at the place where it's causing problems for you the fact that you tried snipe this guy.
And they had you play the flashback but you have to do it.
And it's all directed [UNINTELLIGIBLE]
Isn't like the Cube murder closed as well because you don't have a choice.
You have to kill him to move on
The what?
The Cube murder in [INTERPOSING VOICES]
Oh, the Cube.
I'm thinking Q. Yeah, that would be closed because you simply can't move on, and you just have to deal with your--
But that's the joke
Yeah, yeah.
Exactly
Starification is fairly closed
Yeah.
You can bad mouth those people you just can't play Katamari I'm sorry
I have a question actually about when we passed Sleep Is Death griefing.
It seems to me that was intended by design
Which one?
The Sleep Is Death griefing directly under [UNINTELLIGIBLE].
Like the idea that I am a GM, you're a player, and part of the challenge is not knowing whether your player is going to play along with you.
But that's part of the game.
So I don't know that [UNINTELLIGIBLE] closed or open.
That tension's intended
I don't know
It's still open, right?
I would say this one is open, I guess, because you are allowed to do what you want.
The example he gives for closed ended is-- one of them-- is Manhunt where you have to kill the bull and the more grisly you kill them the better in some ways you don't like it.
There's just no way to get through the game.
The other example he also gives is Shadow of the Colossus.
And that one, especially, it's ambiguous in terms of, OK, I have to kill these things, and if you play through you realize well, wait a minute, they're just hanging out they're not really doing anything necessarily bad but I have to keep killing them anyhow.
And you can start to feel guilty but it's what you have to do to play the game.
So you would say that's a closed system
[UNINTELLIGIBLE] and Indigo Prophecy might be closed because you have to go on with the story, you have to make decisions such in order to progress on
I feel like it isn't-- is that right?
I think the idea is that closed would be you don't get to make decisions.
Closed is like if you want to play the game you have to make this moral decision-- you don't have a choice in the moral decision, you have to be evil.
And that's one thing.
And another game could be you have to be good but if you have the decision between both paths then that's open, I believe, right?
Yes
The premise behind Inidgo Prophecy was that you can choose to do any action in the game it would just have a different ending as a result.
But in practice it only had a few endings but the premise was that it would have countless endings depending on your actions
[UNINTELLIGIBLE] and Age of Empires III is going to have some closed-- Maid exploitation because for the campaigns at least [UNINTELLIGIBLE] there are conditions for completing a scenario [INTERPOSING VOICES]
Oh, so you have to do [UNINTELLIGIBLE]?
The [UNINTELLIGIBLE] is open.
[?
QTA ?]
is open.
[INTERPOSING VOICES]
Most of the rest are open
Yeah.
Fallout 3 is definitely open
The western style one
The F.E.A.R nail gun is actually--
The nail gun is what?
I don't know of anyone who's gotten through F.E.A.R without killing someone with a nail gun at some point
But is that [INTERPOSING VOICES]
Can you punch things?
Yeah you can
You can't run at that level
You can actually kung fu kick things, which is pretty cool, in slow mo
In fact, I don't say all the rest of them seem open
Pretty open, yeah
[UNINTELLIGIBLE] kicking people
You don't have to do this at any point, do you?
No.
[INAUDIBLE]
OK.
So most of these-- what, do we say about [UNINTELLIGIBLE]?
Oh
Open
Regarding BioShock, would that be open or closed?
Well [UNINTELLIGIBLE] open--
You have a choice but it's in your best interests to do one or the other.
[INTERPOSING VOICES]
So if you have only one thing you can do it's fairly closed.
If you have only two things you can do that are very clearly defined is it really an open system or is it just a closed system with two paths
I was under the impression that [UNINTELLIGIBLE] because you weren't self-guiding the system was determining whether what you were doing was good or bad
I was thinking [UNINTELLIGIBLE].
I thought inFAMOUS [UNINTELLIGIBLE] would also be closed
[UNINTELLIGIBLE] I didn't get the difference between only having one option and the system telling you which option is the right option.
So in that case some of these turn from open to close
Like this one.
{INTERPOSING VOICES]
BioShock would then be closed.
Fallout 3 The Dream Land, would that be closed?
It isn't telling you which choice is the right option.
You needn't be morally [UNINTELLIGIBLE]
What do you think?
Do you think he's saying-- I know what you're saying because he also mentioned Ice Field Republic as a closed option.
Is it that there are two discrete, like good or evil, like the Jedi or the dark path?
Is Fallout that concrete?
Fallout 3.
I know inFAMOUS is.
You're either going one way or the other.
But I'm not as sure, and I'm just asking people who've played it, is Fallout 3 that particular in is it setting up a dichotomy of the good character versus evil, or is it much--
It's much more of a gradient
Yeah.
[UNINTELLIGIBLE]
In Baldur's Gate there's two things.
There's your alignment, which you pick at the beginning of the game.
It's more like D&D style but it's fixed.
But then there's this thing that's your reputation, which can be decreased if you do certain things, from the game point of view, in view others.
So you can get into doing bad stuff sometimes without actually decreasing your reputation.
You can also just pick a evil alignment and then do only good things, and that's just very weird.
So I don't exactly remember how the alignment really factors into game play.
But there's a moment, for example, in the game, there's a big story for-- it rejoins it afterwards-- but you you have to ally yourself with either the Thieves Guild or the vampires in town who are competing affections.
So you have the thieves against the vampires.
If it's the thieves against the vampires you got to pick one to help you raid a certain area, and you don't really have other choices.
And one doesn't make you gooder or eviler.
They're both kind of bad.
But you have to do it to find the person that you're trying to get.
That's kind of closed
So in terms of Baldur's Gate II, I would say, one, there's the reputation stats, which I've found never really into the [UNINTELLIGIBLE], which is some type of metric for your--
That's not true
You kill everyone and you have to go [UNINTELLIGIBLE] to do that.
People in your party can duel you for your honor if they get upset at you.
Kaldor dueled me, left my party, and then tried to attack me because I was too evil
[UNINTELLIGIBLE]
[UNINTELLIGIBLE] I stood out.
Like very, very specific actions that will affect your alignment, and there aren't very many of those.
In addition you have these cut scene decisions, which are also very black or white, good or evil.
But I would say there's a lot of decisions in the game that don't [UNINTELLIGIBLE], that have both that ethical choice but not nearly as much of a mechanical impact
One of the things he also mentions in relation to the open system, going back to The Sims, is that you can do what you want and the game doesn't necessarily punish you.
You can murder and you can drown your Sims, and the rest of the Sims in the house don't move away in fear or disapprove of your action.
It's just up to you whether you want to feel bad about it or not.
Or the game has no larger response other than let a Sim die once the site lit a Sim on fire.
And I wish they rolled for his life and got a zombie back.
It turns out the only way zombies happen in the game is if you make somebody else unhappy.
So then I had to kill a zombie.
It was sad.
So that was another element to it.
One of the things that he talks about in general in relation to all of this, and a lot of these fit is ethics to your statistics.
That somehow ethical systems are on this point or sliding scale and that you're heading towards good or you're heading towards evil.
We were even mentioning this before class started that you can blow up an entire town but then if you give three beggars $0.50 cents each suddenly you're back at neutral because of the way that these sliding systems work in bizzarely mechanical ways
Yeah, you can buy reputation back by stoning two temples
It's like indulgences
Yeah it is, exactly
Indulgences [INAUDIBLE]
I have to close this now because I'm getting high off of marker fumes.
I wanted to show you an example of one that we haven't talked about yet.
Somebody mentioned Call of Duty.
This one is actually from last year.
It's not Castro, which is the new one, but--
Is it the Russian hostage thing?
Yes.
It's the new Russian level--
[UNINTELLIGIBLE]
From Modern Warfare 2
It's like a competition
So this was last year's Call of Duty, and this was the optional level.
You did not have to play it.
And of course it caused a big storm when it was released.
I'm just going to play it and then we can talk about it afterwards.
Has anyone played this?
Yeah
Want to provide a little context or explain the a little bit more?
So the convoluted plot is that this is a false flag attack, which is staged by a terrorist organization, which is meant to frame the CIA to start a war between Russia and the US.
And so the idea here is that you are playing a character who is undercover as part of the CIA who is part of this terrorist organization.
And then you end up being double crossed because they, as it was just revealed at the end, they knew you were a CIA agent and they're trying to pin the attack on America.
So that's the--
Do you have the option [INTERPOSING VOICES] of shooting Makarov in the back or--
No, I don't believe so
If you tried and shoot Makarov then the level ends with a fail, and you have to arrest him again.
If you try and get through the level without shooting any civilians the level also ends.
Makarov decides that you're a traitor and shoots you before doing anything else--
Before he's supposed to shoot you later
Before he's supposed to shoot you, I seem to remember
I think you can get through the mission by not shooting any civilians but you can't get through without shooting any of the cops
Yeah, there's--
If you don't shoot the cops then you can't [UNINTELLIGIBLE]
Yeah that's true.
[INAUDIBLE]
Although that guy was incredibly diligent about killing civilians.
I did not play that level that way
Oh, god.
I did
Yeah, that guy took his work to heart.
I don't think that I would have-- like Owen says-- I haven't played Modern Warfare 2 yet or Black Ops, for that matter-- but I don't feel like it would come to me that it was really necessary to just hunt down every last [INTERPOSING VOICES]
He's thorough.
He's very thorough.
But it is also interesting that you-- [INTERPOSING VOICES] and that's all right-- that you think, well, I'm killing all of these people--
So that I can then kill him, probably, or something
Right, or it's for a greater good, right, that I'm going to get in with this group and help bring down this terrorist cell, whereas, ha, ha, not so much.
Call of Duty, has anyone played Black Ops?
Not yet
There's a scene in it, another level where you can assassinate Castro.
And I was just watching it--
It pissed off Castro
--this morning
It did.
They--
No the Cuba objected to the video game.
I watched that level.
It's not nearly as exciting as this one.
Although it's interesting in that you infiltrate his compound, you get up to-- he's in his bedroom with his whatever she is-- and you shoot him.
And in this clip the guy shot him in the head and it slows down to bullet time actually, and you see it enter his forehead and he falls over.
And then the enraged lover grabs the machine gun and starts firing at you in her negligee.
So you have to kill her too.
And, yeah, Cuba is the operation.
But I don't think it takes a lot to get Cuba mad at the United States anyway
Can you score that?
I don't see the statistics, the like statistics.
[INAUDIBLE]
[UNINTELLIGIBLE] see if I dislike
It's pretty high [INAUDIBLE]
In the previous Modern Warfare game there's actually-- I haven't seen anyone draw this parallel, but there is a level where you are in the middle of a firefight, a US Marine in some Middle Eastern war zone, and you are told to shoot at enemies who are coming onto your center but at such a range that you cannot tell friend from foe.
That it's absolutely certain that whatever it is you can get there's a 50% chance that there's actually one of your allies
There's also the really creepy--
Gunship sequence
--the gunship sequence, which is--
Which gets most of the attention.
But I was surprised by the subtlety of the sequence where you are just in the middle of a firefight in Modern Warfare 1 because it has that situation of you are being told to kill but you may not, even though everyone that you might be shooting at is a combatant, you may not want to shoot everyone in this space.
I felt that made a more interesting, subtle point [UNINTELLIGIBLE].
That's kind of what the situation of being in the Middle East right now is like, as far as you can tell
I was thinking about this because I actually did an interview last week, two weeks ago, with some radio people and it was about Black Ops being released because it's the best selling entertainment media ever.
And they introduced it, of course, on the segment as the ultra violent video game, Call of Duty
That's pretty accurate
It is please.
And one of the questions that each affiliate asked over and over is they mentioned the Castro level and they said, have video games gone too far?
Is there a line that shouldn't be crossed?
And I'm sure if they had-- none of these people actually played video games so they were just reading their scripts-- this might have even fit even better in terms of [INTERPOSING VOICES]
I'm surprised this got through
Well it's an optional level.
You don't have to do it.
Interesting comment right there.
If it was JFK airport or US citizens-- But leaving aside just the knee jerk gamer defense, well, it's just for fun or it's just a video game, how do you answer that question?
I know what I spew to them but I'm just curious in terms of what does it mean beyond just games as statistics for the good or the dark side?
Is there a point you think where it can go too far with the game?
What's the assumption behind that question?
There's a really disturbing news report I was reading about this Japanese game called RapeLay.
And for me it was clear this is totally messed up, I can't believe this exists.
But the part that was disturbing was the defense put forth by the developers.
And they were saying who are you guys to be telling us this is messed up.
You Americans, you make games all the time where you have to kill people.
Murder is a much worse crime than rape.
Our game is not nearly as bad as, say, this.
And I was, wait, no, but that's more messed up.
But, what?
And I don't know what to think of it
[INAUDIBLE]
What?
In a South Park episode they address it [UNINTELLIGIBLE]
Actually, just online on that game in particular there was a backlash that I heard in Japan about RapeLay mainly because the game had been out for maybe two years and gone completely unnoticed until, basically, the West heard about it, American and Europe.
Because I think it showed up on Amazon.com or something at some point.
And of course, there was a gigantic controversy.
The media blew it up completely, and so then, in Japan-- was it banned in Japan, I think?
No.
[INTERPOSING VOICES]
But it had been out for a while and no one had paid attention.
All of a sudden--
Apparently Japanese politicians started cracking down on these types of games.
And a lot of fans got really angry and they said this is just like your Western Euro-centric view, and why is this backlashing onto us kind of thing
The only reason why there was a backlash was because of political attention not necessarily because of moral--
A lot of our moral framework is Judeo-Christian in nature.
I read an entire Wikipedia article on such things afterwards.
Child pornography, as long as it doesn't actually show the actors, so representations, animated, 3D, whatever is not illegal in Japan.
And there's no real political pressure either to make it illegal.
It's just not part of the public consciousness that it's something that needs to be made illegal.
And the point is not a weird point, completely crazy, that killing people is probably the worst crime.
And the West makes games that advocate that all the time
So the question of [INTERPOSING VOICES]
Would there probably be a radio interview with that answer?
In terms of the question of whether or not a game crosses the line first assumes that there is a line to be crossed.
And second, I guess, my answer would be yes there is a line but that line is basically whether or not people buy the game.
If people buy the game the game hasn't really crossed the line
So [UNINTELLIGIBLE], the video reminds me of the game of 9/11, where you are a person on one of the top floors--
Oh, and you could jump
Yeah.
Well, that's the only way to end the game.
You can go any way you want but there's no escape.
And so the only way to end the game is to jump out a window
That had the same response
I was just going to say, back on the point of is there a line or not?
I think the line is not a line on whether it becomes OK or not.
It's a line between-- I know we mentioned this before, but-- between pornography and art.
There's ways to take pictures of naked human bodies and it's artful and people respect it and all that.
And there's ways in games to represent killing and rape and all that kind of stuff in a way that provides a message within this framework that makes it OK. And there's games where-- I think Manhunt is one of these-- where there's no real point other than just displaying just terrible, terrible images for the sake of displaying that.
In my mind it's very, very similar to the discussion of is that image pornography or art?
See, I think that's the right way to go with this kind of a thing in particular.
I feel that there's this question of is it doing anything meaningful in the game other than just being gratuitous?
And in this case it's definitely, yes.
It progresses the plot, it makes you think, I guess, a bit about the characters that are involved.
There's character development with Makarov here, it's showing the shady dealings of the CIA and all this kind of business.
It's also making a political statement at the same time.
So yeah, it's not just nailing people's eyes to the ceiling with a nail gun and what have you
I was thinking, are there any games that are over the line?
I could definitely imagine a game that would be over the line.
You could imagine in RapeLay, or whatever it's called, there were a customizable character thing that you were advertising, rape your neighbors or something like that.
I would be uncomfortable with something like that.
I almost wanted to say that there is no line if people buy it, but I feel like there are some things that they would be over the line regardless of whether people buy it or not
People might even buy it because they are over this line
Yeah, of course.
That's why you would buy that
How many of you have played or seen Super Columbine Massacre RPG?
How many of you have heard of it?
OK.
Does that game cross the line?
Can somebody describe this?
[INTERPOSING VOICES]
I mean, I get the idea.
from the title
Somebody want to?
It's kind of a Super Nintendo US-style RPG.
It has you walking around in this sort of Final Fantasy with Nintendo era kind of graphics with quotes and dialogue that have been collected from the actual news reports of the [UNINTELLIGIBLE] diaries and stuff like that.
And yeah, it is largely about the events leading up to and the massacre itself.
Yes, and you do go around-- and I can't remember whether you actually--
You do everything
You shoot everybody
You have to sneak into the school.
I forget which one you start off as, it's either Kinkel or Harris, but you pick up your friend.
It's very RPGish in the beginning where you're in the room because you touch objects and it gives flashbacks to your friendship with this guy like learning how to make pipe bombs behind the convenience store.
And you pick him up, you go to the school.
You have to sneak in and plant some bombs.
And after those go off then you move through the school and you're shooting.
And it's interesting [UNINTELLIGIBLE].
I've showed it to a class once in Ohio, my old school, and I'm sneaking into the school, right, and there are security cameras you have to avoid.
And students, you can't be seen by them either.
And there's a point where I'm trying to manipulate the character and everyone in class is like, oh, that way, that way.
Oh, oh, you blew it, you got seen, you got kicked out.
And it was like this moment of typical gaming, like, oh, you did it.
And then you realize, I got kicked out of the school because I was trying to plant a bomb.
And this was based on something that really happened.
And it was that moment of like you were saying with the friendly fire and not sure who you're shooting.
And it was like, wow, this is something that made me think about what about all those other games where I'm happy to kill people, and I don't really think about the consequences.
And this is a game where you are forced to
[UNINTELLIGIBLE] a really interesting parallel in a film that I could think of, The Battle of Algiers.
I don't know if you've seen this but there's this scene where you are following the insurgents who are these two women who've managed to smuggle bombs into a cafe they think leaders of the civilians hang out, which is largely frequented by the ruling class French section of society there.
And there's this incredibly tense scene where they're trying to leave their bombs in the cafe without being detected and get out of there.
And you're like, oh my god, are they going to be able to make it?
I hope that they can bomb this cafe.
What am I thinking?
This is terrible.
And you definitely have that same kind of tension
I feel like people devote too much emotional energy into getting offended.
They pour their heart and souls into getting offended by things.
And the line for me is is it does it advocate and is it instructional?
If it's not either of those two things it doesn't affect me really, and if you really want to go out and play some messed up shit it's none of my business.
And my line is not going to be the same as your line, it's not going to be the same as someone else's line.
And because there is no human definable line it really scares me that someone could define a line.
And since I don't like that idea I'd rather say that as long as it's not like and then you pull back the safety, and here's how you sneak into the airport, and then here's how you shoot this.
As long as it's not something like that, and now please go do it, here's why you should because America is evil, or whatever.
That would scare me because I think that would be increasing the chances that someone shoots me up at an airport.
But otherwise, because my attitudes are so liberal and someone else's attitude is going to be so conservative, and oh god, someone else's attitude is going to be even more liberal than mine and be OK with things that I wouldn't feel comfortable with.
If I was allowed to draw a line, someone else could draw that somewhere else, either before or after, and that would just mess everything up.
And so that's how I feel about it.
And I think that the environment right now kind of tends towards that in that--
[UNINTELLIGIBLE]?
Huh?
The line drawing?
Yeah, in that--
Depends on which line drawing?
Yeah.
People who make games are not intrinsically, on average, really messed up, and so tend to make stuff that they feel generally comfortable with.
And those who are messed up, you just draw attention to them, really, by giving them publicity anyway.
For every person who says that-- Diablo 2 got an M rating because of violence.
Now if you look at it today it's like little sprites whacking at each other.
But that was the line back then.
And if that's the line today then this stuff is absolutely horrendous and should not be disseminated.
But I think it should be, and someone else might think what I think is OK is not OK.
It's just going to be what I think is not OK is OK.
It's very complicated
This guy got death threats.
Apparently seen this game
I'd strongly like to agree with the a lot of effort being put into the line drawing especially the public display of, hey, I'm drawing the line here, mostly by politicians but also by people with some sort of possibly, often self-interested agenda.
And both by the game industry and by people who [UNINTELLIGIBLE] the game industry.
It seems like there's more effort either being used to figure out what is offensive and what is not offensive than to discuss the need that some things should be able to offend.
That we should have stuff that needs [UNINTELLIGIBLE] There needs to be some stuff that crosses because maybe the line [INAUDIBLE]
And it's a method to sensitize adults to violence.
There's so much violence these days it'd be better if you didn't freak out at every little--
I think it's totally OK to be freaked out at things.
I think that you should definitely say when you're offended by something, and I think that should be open to debate like whether something is a decent artistic expression or [INTERPOSING VOICES] So there's a difference between staging that you find something offensive and attempting to have these things banned or [INTERPOSING VOICES]
I agree
I'm totally fine with people wasting words on saying whether it's--
No clearly.
But as long as because you are offended you think I shouldn't be able to look at it, that's the problem
OK.
I think it's--
Or even better, if by being offended you get to now make your point.
This is why it is offensive to me and these are my reasons for it.
It might be more informative and actually better for society to get those ideas out into the open
Suppose you get offended by proxy?
It doesn't offend me but I can see how it offends others
I think there's also a conflation of taking offense and discomfort because playing these games might be uncomfortable.
And I don't think that's necessarily a bad thing if it makes you think about, like [UNINTELLIGIBLE] was saying, or like [UNINTELLIGIBLE] game, this Super Columbine Massacre, if it makes you think about things in a different way, or like that film.
And especially games have the ability to provoke that in you to make you think because you're the one.
Especially these games that force you to do certain things.
It's like, well, I don't even think about is there a game that you absolutely refuse to play because there was something in it that you just really did not want to do?
That's fine
OK?
I think we'll end there.
The following content is provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free.
To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu
I know I'm meeting with two of the teams today, one on Wednesday and one on Friday.
So each of those meetings will take about an hour.
And we'll be covering both some individual feedback on the presentations themselves, as well as talking in more depth about how to narrow down your focus.
So right now it feels like you guys have gotten a pretty good understanding of what everybody else says.
And now the trick is, the real trick of X Prize, is to figure out within all those things that are clearly important to somebody, what's the most important thing that you can change to change the conversation.
So if you can only change one parameter-- start yourself there, constrain yourself as far as you can-- if you can only change one parameter and really demonstrate that in an interesting way, which parameter would it be?
Which one is most important to you?
And also we're going to start thinking about not what is it that the field tells you is important.
There are self-perpetuating myths within any field.
So not necessarily what is all the standard literature tells you is important, but what are you starting to gather from your conversations with your advisers, from your readings, from your creative thoughts within your own group?
What is it that is really a game changer?
Is it all about cost?
Is it all about power?
Is it all about energy?
What is really an interesting conversation in your group?
For those of you who are extremely left brained, the next part of the semester is going to be very, very hard.
We're going to go from being very logical and data-driven to asking you to stand on your heads for a while and think about this problem in a totally different way.
And I know that that's going to be challenging for some folks.
So what I'm going to ask you to do today is we're going to start to step of the logical box just a little bit and start to play a little bit more.
Because what we'd like to do is start to figure out what are the creative solutions, what are the interesting solutions to these problems.
So today we're talking about engagement strategies.
And by engagement strategies, I mean how do we involve everyone who needs to be involved in this X Prize.
And that's a pretty broad set of people.
If we started and talk about our prize economics 101-- we've talked about the Ortega Prize and the Ansari Prize a number of times.
And we know that if you look at the amount of money that the prize purse offered and the amount that the winner spent, the amount that the total field spent, those numbers don't necessarily line up in an equal one.
We have a nice leverage that we get from prizes.
People are spending more than the prize purse, certainly in aggregate, and often even just the winner is spending more than the prize purse is worth.
And why does that happen?
Why do you spend more than the prize purse is worth?
Because you think that you get some benefit over and beyond what the prize is worth
Yes.
So what do you get besides the purse?
Their sort of internal utility saying, sort of I really just want to do this
It feels good to win
Yes
Yeah.
So what else?
Free marketing
Free marketing.
What else?
Credibility?
Credibility.
What else?
The establishment of an industry?
The establishment of an industry, and hopefully follow on market benefits if the prize is designed in that way
You keep you intellectual properties
You often keep your intellectual property.
Certainly in the X Prizes you do.
One of the big things that teams value is the PR, free PR, massive amounts.
And this is Ansari, we're talking 5.5 billion media impressions.
If you value that on the free market, they call it $120 million worth of free press for your team.
That's pretty good.
Name an ad campaign.
Let me be a little more specific.
Name an ad campaign that sticks.
Anyone, Paul?
Apple
Apple.
Which one?
The skinny guy and the fat guy
The Apple/PC.
Yeah
Got milk?
Got milk?
What else?
Just do it
Just do it, big one.
What else?
Terry Tate, Office Linebacker
What do these have in common?
What do these catchy ad campaigns have in common?
Taglines
Taglines.
What else?
Short and fun
What else?
Same over time
Say that again
It stays over the time.
It's the same message over time
So the same message over time
[INAUDIBLE PHRASE]
Yeah, so the little jingle
[INAUDIBLE PHRASE].
Simple and smart
Yeah, simple and smart.
Is there and ad campaign that you can tell me that you really remember that doesn't fit these things?
Is there an ad campaign that shocked you
I'd definitely call some of the beer advertisements certainly not simple, maybe not smart.
You remember them anyway
Short, fun, and stupid
Yeah, that'll work too
Yeah?
Billy Mays, I don't know
Yeah.
So what is it about Billy Mays?
Well he's just sort of yelling at you for one to five minutes and saying, this is awesome.
You have to buy it now, now, now
Yeah, so there's repetitiveness there.
There's also emotional content.
Billy Mays is very excitable, very excited.
my mom used to cry every season when political commercials came on.
It didn't matter what candidate it was for.
They always made her cry.
And she remembers those.
They've got an emotional heart string component.
There are commercials that are shocking.
There are commercials that make you laugh.
But they all have some sort of emotional connection.
Name an ad campaign that got you to do or buy something.
Because you saw that ad you actually did something
Well there was the..I remember an ad for World Lights Out Day.
Granted I'm sort of an environmentalist myself.
But I know that I had a bunch of psets due that week.
The last thing I wanted to do is turn off my laptop and just sit around in the dark for an hour.
But amazingly enough, I pulled myself to do it anyway
OK, what else?
Who else actually did something because of an ad?
If you see something on sale.
You click on it to check the sale prices
Yeah totally.
What else?
Online banner ads, anyone ever clicked on one?
Google ads?
I click on Google ads a lot if I'm shopping for something.
I find that the ads that come up are relevant.
So it has some relevance to it.
It has some relevance to the demographic that it's aiming at
They tend to [INAUDIBLE PHRASE]
All right.
Paul?
I was just going to say one is that those with a social cause and especially ones right after 9/11.
I throw up so many checks that I have to slap myself not to
The breast cancer awareness month is October, we just got through.
And the number of a pink ribbons I saw was incredible.
And the amount of money that that ad campaign has raised for that organization is huge.
So some of them make you do things.
But figuring out what sticks and translates to what actually makes you take action is not a simple thing.
Is there any ad that you've ever seen that gets you to successfully change your behavior?
At MIT there was a group called SAVE.
They're an environment group.
But they have a really clever ad campaign.
They call it Do It in the Dark.
They have these glow in the dark stickers that you put on the light switch.
But at the same time they're also distributing glow in the dark condoms and really stupid stuff like that.
But it was funny.
It was lighthearted.
The use the circular doors instead of--
Yeah.
I find those little stickers to be really effective.
Those little nudges, right?
anyone actually read Nudge?
Great book came out, top book for The Economist Magazine this year.
It talks about how the messages in the right place at the right time can really change behavior.
The way that you arrange the food in a middle school cafeteria can really determine what kids choose to eat, little nudges.
Have you ever had and ad that you loved and you couldn't tell me what the product was for?
For instance, there's a number of ad campaigns out right now for car insurance.
There's the one with the anime girl.
Can anyone tell me which company that's for?
Yeah, that's-- Progressive
Not Progressive
No it's online.
It's like Esurance
Esurance.
What about the white room with all the white?
Progressive
That one is Progressive.
I'm sure there's others.
What about the gecko?
Geico
Geico.
Would you remember it if it didn't say Geico and gecko sounded like the same thing.
I don't know if I would.
I'm spending so much time paying attention to get the gecko.
So I think that there's a lot of things that we can think about in terms of advertising.
We sort of know intuitively what gets us, what misses us, and what it is catchy and fun but ultimately never left a mark on our behavior.
This was a nice slide that Dr. [UNINTELLIGIBLE] put up a couple weeks ago, last week.
So we're talking about the stages of prizes, right?
So we have an idea.
We do the prize design.
We do validation.
We talk about the prize.
We kick it off.
We do some monitoring.
We have our proof of concept or our actual competition.
And then there's a new market that comes.
So which of these require advertising?
The stuff under prize communication
OK, so who are you communicating to?
The people who going to compete for the prize
So prize communication, you're going to communicate to potential competitors.
If we consider this to be the whole period of time before the prize is launched, who else do we have to communicate to?
Who else do we have to sell to?
Sponsors
Sponsors.
Who's going to put up that $10 million prize first and all the operating expenses?
Who else?
Media
I would say media is a little one in this phase.
We're sort of cultivating relationships.
But we're not trying to tell a story to the public just yet, not until we have a person-hand.
But who else are we really looking to talk to at this point?
Judges?
No, I guess that's too simple
Let me broaden that out and say the industry.
So in this phase, When you're doing your prize design, your validating it, that's where expert opinions, community opinions really matter.
This is where it mattered how we measured 100 miles per gallon equivalent.
It mattered where the race was going to be.
It mattered what kinds of vehicles were going to be loud in the competition no matter what the safety standards were.
And these kind of inputs all came from the community at large.
Not necessarily the people who were going to compete, though some of them were involved.
Burt Rutan was involved in the formation of the rules for the Ansari X Prize.
So he was someone that they went out and asked.
So were a lot of his competitors.
But this is the stage where we're thinking about how we formulate the prize in the right way, who's going to compete, who's going to sponsor.
Where else does communication come into play?
Prize kickoff
Yeah.
So the kickoff is big.
So the kickoff Progressive Automotive, they were on stage at the New York Auto Show with Michael Bloomberg.
Jay Leno had them on TV that night, so pretty big media events.
So who are you communicating to?
The media is communicating to the public at large
Yeah, the public.
Who else do we care about at the kickoff?
Media
Yeah media is big, and in some respects because they are an angle to the public and everybody else.
But in some respects because you're starting to cultivate a relationship with the media that's going to last throughout your competition.
And this is where you really start to engage them.
What else?
Competitors
Yeah, competitors.
If the day of competition is launched no one knows about it, you fail.
So your competitors, anybody else?
So those are your primary audiences.
What about during the prize competition?
So your teams have registered, things are going along.
No one has won yet.
Are you doing any message managing, any media work?
Still media and public [INAUDIBLE PHRASE] interested in what the competitors are doing and making sure that it's still interesting
Another thing that's in the space for X Prize is education, we'll say educators and students, as part of their public campaign, but also as an explicit additional item
Is it important doing the prize formations where you create these checkgates so that you can get the attention of the media and continue to say
Yes, exactly
Now you have only 500 companies, now you're down to 100 companies
Yeah, so stage-gates are really nice for drawing the attention of the media.
You can only send out so many press releases the say, we're having a competition, before people start to ignore them.
If you send out one that says, now we have 20 competitors, now those 20 competitors are going to be putting their cars on dynamos.
Now we have ten competitors including this really interesting story from West Philly High School.
You can start to build your messaging in that way
And also the different trials or stages make what is happening [UNINTELLIGIBLE PHRASE]
Yeah, exactly.
So if we have, let's call it the prize competition itself, so now you're launching your rockets, or you're racing your cars, or whatever's going on.
Anyone different this time?
Probably work with the investors involved now, not so much the sponsors, but the people who might taket his idea
Yes.
So the future investors.
And some of that's happening up here as well.
During the prize most of the teams are still raising capital throughout the course of the prize.
And there's 21 teams in the Google Lunar X Prize.
And I don't believe any of them have enough money to launch a rocket right now.
So they're still raising money.
So here you've got future investors in the technology.
You definitely want the big automakers to be at the race the day of Progressive Auto.
Who else?
Anyone disagree with me copying this list up here?
Just to refresh my memory, what is the educator's role in sort of being reached out during the prize?
So from the perspective of the X Prize Foundation, a prize is not just about the new technology.
It's also about inspiring the next generation, and insofar as educators play a formal role in that.
The other thing that happens the day of your prize competition is that you're starting to look for future sponsors.
This is where all the excitement is happening, right?
If you have the Progressive Auto X Prize Race, and you don't have the next set of sponsors lined up and watching that race and getting excited you've done something wrong, so future donors, future sponsors of prizes.
What about afterwards?
Is there anything to be done after a prize is won?
Yes
What?
Continue trumpeting the winner as look at this spectacular thing we've done.
And also from this, we've created these new products.
And the point [UNINTELLIGIBLE PHRASE] to have one prize go out and then never hear about it ever again
So post-prize, you're building the team, building up the team, and not even necessarily just the winner.
There may be reasons to be telling stories of those who competed and failed or competed and got close.
You're trying to build your brand.
And insofar as your brand is tied up with these teams, that's a part of the reason for supporting the teams.
But you're also trying to support the industry.
Because ultimately, the success of Burt Rutan in selling SpaceShipOne, the success of Virgin Galactic in launching people into space, comes back in benefits to the X Prize.
All right, so you can tell that there's a lot of PR work that goes on throughout the course of a prize.
In fact, if I were going to name the X Prize's core competency, I would say it was PR and marketing.
That's fundamentally what the organization is organized to do.
We're going to talk about four pieces of an engagement strategy today, namely audience, which we've started to talk about a bit here, the message, strategy, and the delivery.
Who's this?
iPod and Bono
iPod and Bono.
So there's two things going on there, right?
You can all tell me who it is.
Why is that important?
It could be you.
It could be me
Yeah.
But why else is it important?
Some of the iPod videos are very generic people.
But this one we all know is Bono.
Why does that matter?
That's the same reason why everyone puts stars in their ads, because everyone wants to be like Mike
So there's a demographic that you're shooting to capture here.
You're affiliating the Apple name with U2.
And you're saying that that affiliation is somehow meaningful.
So we're going to come back to that.
Some of what we were talking about over on that list that's happening here, these are really simple ads.
They're not graphically intensive.
They're not heavy on the messaging.
It's typically a song or an image like this.
And we still know that it's iPod.
Why?
Because they print out so many of these and it's the same style each time
So that stylistic branding and building a message, not even necessarily a tagline, but a visual tagline over time is really impactful.
So let's talk about audiences.
We said over here that we wanted to talk to the public.
Right?
So let's talk about a prize in energy?
Who is the public?
Because you can't talk to everyone.
You can't to six billion people on the planet all at once.
So if you're going to be talking about energy, are any of these people particularly relevant?
Are there other audiences?
Environmentalists
Environmentalists clearly are
Techies can be
Techies can be
Entrepreneurs
Entrepreneurs.
Who else?
I remember a demographic that they used in better places that they called Scuppies-- socially conscious young urban professionals
Socially conscious yuppies, all right.
Scuppies, I like it.
Who else?
Who else is relevant in an energy prize?
Well, in some cases, it would also be the developmental people in a third world prize
What if you're trying to reach out to the people that are going to be taking advantage of your global prize?
What's your demographic there?
The government.
NGOs
Governments, NGOs.
Is there any reason that don't want to reach out to the users?
You can't because it's very hard to reach out to them
It's doable though.
It's harder, but it's doable
[INAUDIBLE PHRASE]?
It's a good question
But also it has a risk of promise
It does have the risk of a promise.
So reaching out and engaging the end users for a product that doesn't yet exist is building a market, which can be valuable.
And it can be dangerous as well.
Who else?
Are there are other segments of let's say, the sort of Western media savvy market that are relevant to an energy prize?
Well it's going to be whoever is going to be using it at the end.
So Green Aviation, the aviation hobbyist market will be interested in this.
Home-scale storage, the people who are in the markets that have time-of-day pricing will be interested.
Grid-scale storage, it's going to be more utilities rather than so much like the public as we tend to think of the public
So there's clearly the user communities.
What I want to push a little bit today is to remember that the broader community is actually really important for X Prize.
We talk about 5.5 billion media impressions.
Those don't go to only the competitors and the users.
Particular in the case of commercial space we have a total user market estimated at around 13,000.
You've far oversampled if you're trying to push out six billion media impressions.
But that's important, right?
You're building a name for the prize.
You're building an audience.
Competition is worth far less without people watching the competition.
Imagine a scrimmage football game versus an NFL game on game day, really big difference in terms of the amount of dollars you can bring in for ads, in terms of the sponsors that you can engage.
So if we're going to think about energy, is there another demographic that we need to make sure we engage?
People who think technology is cool or neat
Yeah.
Just neato.
How would you find them?
What's the salient feature of those people?
Disposable income
So they have some level of disposable income
What they read
Yeah, like what?
Popular Science, Popular Mechanics.
Wired
Wired
Yeah, Wired
So popular print magazines that are technology-related.
TV shows?
Yeah, like the fix up your home shows would probably be a reasonable place to look.
Discovery Channel kinds of things, [Big Bang Theory] MythBusters
MythBusters yeah.
So is there another way that you find this people?
Are there things that they own, places that they shop?
Tools, like any sort of do-it-yourself stuff.
Probably newer media displays, like Pandora for instance, or Grooveshark or something like that
So insofar as you're reaching out to the people that tend to be the early adopters, I think that that's very true.
And you can sort of start to think about how early in the marketplace you want to be reaching out to.
We have that nice bell chart, right?
So you know that your early adopters are the most passionate.
And they're really useful to tap into.
But they're also a very small chunk of your market.
So if you're talking about going viral, they may or may not be the right people.
So what about teams?
We're going to talk to teams
Given with the current technology though, [INAUDIBLE PHRASE]?
Yeah, please
For example, [UNINTELLIGIBLE PHRASE] with those guys it's a long tail
That's true
It's huge
With the right Twitter users, right?
Twitter like Wikipedia, there's a very small fraction that control a very large audience.
If you can get to the gate keepers, then you have a pretty broad distribution.
The average Twitter user and the average Facebook user is a fairly small distribution
I think it behooves us to find out who is that person to try to get a prize across
If you're engaged teams-- Yeah, Rob?
I just have another question.
So can you tap into nationalistic things?
Totally
So what kind of people would you target for that?
Well, what do you think?
Yeah, I'm not really sure.
Like for our prize, I'm not quite sure who you would tap into
So people get engaged in energy for reasons of pure environmentalism, the sort of tree huggers.
You also get a couple of different nationalistic angles on it in the US.
You get the dependence on foreign oil.
So there's a definite market there.
And that's somewhat segmented by politics, but not exclusively.
You can also tap into that group by thinking about the made in America pride, which was a much stronger movement in the late 80s, early 90s than it is now.
There's still some things going on there.
And we also realize that we're not only reaching out to American audiences and American competitors.
It may behoove us to really tap into and call in the nationalistic tendencies in Japan, or Israel, or Romania, or wherever the markets are evolving.
Yeah, Francois?
I think we can also try to advocate to people who are in power from the community.
Like Bob was here last week talking about Antarctica.
If you can get [INAUDIBLE PHRASE].
Because people will try to get news from those very [UNINTELLIGIBLE PHRASE] in markets.
And they get involved [INAUDIBLE PHRASE]
Right, definitely, definitely.
So let's talk about teams in particular.
So we've talked about segmenting the public.
How do we find the people who are going to compete?
So there's obviously the people that are already in your industry.
They're pretty easy to reach out to through trade publications, and industry conferences, and you sort of know who's in that space.
If you want to bring someone into a space, how do you do that?
There's one, the shotgun approach, where you just try and reach the largest number of people.
Like you go on Leno and you advertise your automotive prize
Right.
But how do you find it relevant?
General technology, publications like Technology Review
Yeah, so you can reach out into the general technology community.
What do you think of as adjacent markets to energy, energy storage in particular.
What are things that people may be working on that's not storage but could be storage if they thought about it differently
Chemicals
What was that?
It might being involving chemicals
Yeah, chemical.
Right.
What else?
Electrical engineering sorts of things
Electrical engineer
If you're familiar with laptops you're probably familiar with energy storage.
Small-scale electronics
Small-scale electronics.
Bob Metcalfe was talking about the role of inconnectivity and the internet in bringing together smart grids.
And so if we think that IT has a role to play in our technology, that might be an interesting community to bring in.
What else?
Architects
Architects, if you're going home-scale.
If you're going larger-scale, maybe civil engineers
The teams should be the ones who should make dream, kind of.
The question is who can we make...[INAUDIBLE PHRASE] It might be a different angle than only the technological side.
There might be some mavericks that I say, I want to change the world.
I want to
I don't care what my prize is, I want Dean Kamen competing for it.
Or I don't care what my prize is, I want-- who's another name out there that you would want to just get in on your action?
Branson
Richard Branson.
Who else?
A college team from
Yeah, a college team from MIT.
Who else?
Amory Lovins.
Amory Lovins.
So already, without even going to Google, you can come up with six names off the top of your head.
Add those to the ones you're already expecting from the industry, broaden your thought about a few more university labs, a few more hobbyist teams, and you've actually got a pretty wide field.
OK, so these are audiences.
Next thing is the message.
We were talking over there about taglines.
A short, concise message is really, really important for our prize.
And we're going to talk about this from a few different perspectives.
But when I talk about messages, Ansari is the first private space flight.
The genome has sequenced 100 human genomes in ten days.
Progressive, I'm talking about a race of cars that are getting 100 miles to the gallon or better.
You're able to capture it in a really succinct phrase.
And most people you say, cars getting 100 miles per gallon in a race, that's enough to sort grab their attention.
And then you can tell them a more in-depth story.
It helps to have a sticky image or tagline, to have this image of a race or to have something that's technologically interesting.
Energy is tougher.
Saying I want a battery with x power density doesn't stick.
And it doesn't capture what people already intuitively understand.
So thinking about how you tell that story, not only what you want them to know, but what you want them to do.
Is your tagline about them becoming involved?
This is an all teams can play?
Excuse me.
Thank you.
Or is it that you want them to donate money?
Is it that you want them to tell their kids?
Is it that you want them to tell their neighbors, so thinking about how you build that message.
For prizes we can think of this in a few different parts.
One is we can talk about the rationale.
Why was the Ansari X Prize there?
Well, sort of at the peak, it was because Peter wants to see the day when humidity is left to earth as its only place of existence.
It's hard to tell that story.
That's a big story to tell.
And there's a personal reason he had, which is to get himself and his friends closer to traveling to space.
And that was meaningful.
And he told that story personally a lot.
And then this was a quote from a newspaper article that came out at the time.
"To open a new era where space is no longer the exclusive domain of massive governments, space programs and ordinary people can now realistically dream of one day reaching for the stars."
It feels good.
It's not a concise message.
But it feels good.
So figuring out, are you telling the story of the rationale?
Or are you telling the story of the challenge?
So we've talked a lot about where on the scale of difficulty you place your challenge.
Are we talking about affordable and available access to space, wide-scale commercialization of the space frontier?
We weren't giving a prize for that.
We're weren't giving a prize for orbital spaceflight.
We were giving a prize for suborbital spaceflight because we thought it opened up these other markets.
So you can start to talk about the challenge that you're offering.
And you can start to talk about the prize.
And this is the messaging you guys are going to be building over the next few weeks.
Ansari, you can say X Prize for commercial spaceflight.
People talk about it as the space X Prize.
You can say it was for the first private spaceflight.
You can tell someone that it was for the first private team to go to space twice in two weeks.
You can say it was for the first team and site all the variables and numbers that were relevant.
You can hand them the master team agreement, which was the 60-page contract the team signed when they came into play.
So depending on who you're talking to will depend on how much they want to know about what your prize really is.
So as you're starting to craft your prize, you're going to want to think about what's the name of it, first all of.
That's actually going to go a long way in your marketing.
If you can capture in three to five words, what's the challenge?
It's the first private spaceflight.
It's a race for green cars.
It's sequencing my genome.
How do you put that in a really tight phrase that's meaningful.
And then how do you build that out to tell more about what the competition actually entails.
Is the relevant thing that it's a private team in space twice in two weeks?
Is it that it carries three passengers?
It's some subset of the full rule set that starts to deliver your message in a meaningful way.
So we talked about the strategies here, the different elements of the prize that we're going to try to build a media communication strategy around.
For X Prize, launch is sort of the big first thing.
So we talked about the launch for the Progressive Auto X Prize at the New York Auto Show.
For the Ansari X Prize it was standing under the arch at St. Louis with the head of the FAA, the head of NASA, 12 astronauts on stage, launching with the new Spirit of St. Louis, which is mimicking the Spirit of St. Louis that Charles Lindbergh was sponsored by.
Sustaining interest, building that media story, the only thing when X Prize signs contracts with the team, the only thing that they maintain rights, none of the IP.
You have your own technology.
But X Prize gets the rights to all the media attention.
They get the right to tell the story.
They get the right to tell you how big the X Prize logo is going to be on your jacket when you're racing your car.
They get to manage all the PR.
So they're going to be thinking about how they're going to use that and build the interest up to the day of the competition.
Obviously the competitions themselves can be massive media events, particularly if you have a date-certain prize, a little bit harder when you have a first pass at the post prize.
And then building on success, here we have SpaceShipOne hanging in the National Air and Space Museum.
This is the most frequented museum in the world.
And SpaceShipOne is hanging in the main lobby.
So that was a piece of the PR strategies that were built up for Ansari.
Delivery: how you deliver your message is just as important as what you're saying.
We can think of ad campaigns that are very guerrilla.
So I don't know if any of you in Boston when the little Cartoon Network guys went up around.
It closed down the city.
For those that weren't around, they were little Lite-Brites, little children's toys that had the outline of a new Cartoon Network character on them.
And they were placed all around the city.
People thought they were bombs.
There was a major bomb scare.
The police came in.
It had a news coverage for hours and hours.
Roads were closed.
It ended up being extremely effective market, but at a cost.
So the guerrilla campaigns can be really great.
And viral campaigns can be really great.
The question is, do you want to do it that way?
Or do you want to have the mayor of Boston at your side saying, we're having a race in our city?
Which one is more useful to you?
So you'll have to think about what's the strategy.
Who do you want to involve?
Who do you want to engage?
Who do you want to alienate?
Because you may want to alienate people.
That may actually be of use to you.
You may say all those existing utilities, forget about them.
They're dinosaurs.
We have nothing useful for them.
Or you may want them in your back pocket.
And those are two really different ways of crafting your communication strategies.
The medium that you're using: are you out there communicating on blogs, and Twitter, and Facebook?
Or are you on the evening news?
The average age of broadcast evening news watchers in the US is like 64 years old
A quick question about guerrilla versus mainstream, is it possibly to do both?
It is possible to do both.
But it's harder.
It depends on what your message is that you're delivering in your guerrilla tactics.
We're fighting the proliferation of AIDS by dumping body bags on the stairs of the UN was actually intended to engage the UN, but by engaging the public's ire in doing that.
So there are ways to do that
So, for instance, they can't be mutually exclusive such that like the guerrilla message could be, screw the utilities and take your home off grid.
Whereas mainstream message would be like, benefit the utilities and help them reduce
I think you'd find that that was difficult to engage both audiences like that
Maybe if you run it through the CIA [INAUDIBLE PHRASE]
So which medium you choose will be based on who your audience is, who are you trying to reach, how are you trying to reach them, how do you want to come across.
Do want to come across as young and fresh?
Do you want to come across as established and credible?
Not that you can't do some of both, but again, the medium that you use will drive some of that.
The location, are you delivering this at the largest energy trade show, at the IEEE Conference, at Menlo Park where Edison had his first light bulb?
There's very different images and emotions that are evoked by using different locations.
What are other locations that are relative to energy?
Power plants.
Hoover Dam
Power plants, Hoover Dam, what else?
Niagara Falls
Niagara Falls
Independence Hall
Independence Hall
Little poor villages
Little poor villages.
Anything else that comes to mind immediately?
Coal mine
Coal mine
Benjamin Franklin's lab
Yeah
Las Vegas Strip
So places where energy is used, places where energy is produced.
Land mines, if you're talking about batteries.
There's lots of different ways.
And they all evoke very different mental images
Antarctica
Antarctica
You mean landfills not land mines
Sorry, landfills, yes.
Not land mines.
Two totally different things.
Dates: if I say, what's a date that's relevant to energy?
Anniversary of a blackout
Anniversary of a blackout
If you want to pull up the nationalist no foreign oil angle and do it on July 4
Independence day
I sort of disagree.. there is bias in that which is America's symbol
Yeah.
It's a choice.
It's very much a choice.
You wouldn't want to launch on July 4 if that wasn't your intention
Bastille Day.
Yeah.
Bastille Day, yeah.
[INAUDIBLE PHRASE]
So the choice of date is not arbitrary.
Do you want to do this in the middle of the Olympics?
Do you want to do it during the holiday season?
Do you want to do it when you're likely to get a lot of news, when you're not likely to get a lot of new?
Just choosing when you launch something actually has a lot of play into both who you're reaching and how you're going to reach them.
Choice of a spokesperson: we sort of talked about that with Bono and U2.
You don't have to give me names.
But what are the categories of spokespeople that we might think about?
Al Gore
What was that?
Al Gore
So politicians, has-been politicians, Nobelists
Green, right.
He's personified
So you have you could pick an environmentalist.
You could pick a politician.
You could pick a very public figure.
We've talked about picking celebrities, movie stars, TV stars, rock stars
Descendants of technology heroes
Descendants of technology heroes, so Erik Lindbergh involved with the launch of the Ansari X Prize
You might be able to pick living technology heroes
Living technology heroes.
What are other groups that might be interesting to have on stage with you?
But if you do those, the challenges, the mass audience will not know who they are.
And that's why guys like Al Gore, he's a former politician.
He's not just a pitch man
But even Al Gore is polarizing.
So your choice of a politicized spokesperson is very much going to drive your audience as well.
And that may be in line with the people you're trying to reach.
It's something you'll want to ask
Michael Jackson
Michael Jackson, unfortunately.
Anyone else that comes to mine
A rural villager
A rural villager.
If you're trying to reach emerging economies, a cricket player, a soccer player.
There are big public figures in every culture.
And knowing who those are and who you're trying to reach, gimmicks.
We had one team last year that wanted to light the White House green so it became the greenhouse.
There have been groups that have draped the Hollywood sign in black in mourning.
What else?
You guys know more ad gimmicks than I do.
What are other big things that have happened, things that you noticed, the equivalent of MIT hacks on the world stage?
You already had the Cartoon Network one
Cartoon Network
[INAUDIBLE PHRASE] pulled off a few MIT hacks
There's been lots of body bag campaigns for various causes.
What else?
Artificial blackouts
Artificial blackouts, real blackouts.
What are the pluses and minuses of doing a gimmick.
The plus is you get a lot of media attention.
What are the minuses
You get a lot of media attention
You get a lot of media attention.
What are the other minuses
You might get thrown in jail
You might get throw in jail.
You might get negative publicity.
What else?
People might not know what you're doing
They might miss the message entirely.
What else?
We already said it might backfire.
We might lose some sense of legitimacy
It may lose some sense of legitimacy.
So again, gimmicks may go better with guerrilla ad campaigns, not necessarily, but possibly in terms of trying to come across as edgy and not established
Yeah there was an ad campaign in Canada for the Nissan Cube where they said this is going to be all Web 2.0, et cetera, et cetera, et cetera.
And it kind of backfired when they noticed that several of the winners had some form of personal relationship to the judges
Yep.
So you have to be careful about what your claiming and how credible you are if your are going to put yourself out on the stage like that.
The other thing is that these things can be really expensive.
Just in terms of costs, cost can be really big depending on how you do them, so deciding is it more worthwhile to pay all expenses to blackout every building in downtown New York or to have a massive ad campaign through an ad agency.
And everything you're choosing.
And I think this is sort of the core message about today's lecture.
We want to reach the whole world.
We want to reach all the public that's relevant.
We want to reach all the competitors that are relevant.
We want to do it in a way that's exciting and engaging.
And you can only do so much.
So when you pitch your final X Prize to the board members, you're going to have to say, you know, these are the three most important audiences and three most important messages.
And here's how I'm going to reach them.
It's not, I'm going to reach the world.
If it is I'm going to reach the world, it's going to be very shallow.
Because a given dollar only spreads so thin.
So you want to be able to be targeted with your message and figure out who's most important to you and in what way
I was going to say in terms of gimmicks, another way a very good PR is the Chicago Irish Parade.
They literally dye the whole river
They dye the river green for St. Patty's Day
Is that a gimmick?
Yeah, it's totally a gimmick
But can you piggyback on an existing?
Sure, as long as you can control the messaging.
If everyone thinks that you're celebrating St. Patrick's Day instead of green energy, clean energy, that might be a backfired message.
OK, so what I'd like you guys to do is get in your teams.
And we're going to take about the next 13 minutes.
We'll take until 3:40.
And what I want you to do is to come up with an X Prize marketing campaign for a green battery prize.
So this is a prize, let's say it's $25 million for a replacement for toxic batteries.
We're going to create batteries that are be thrown in landfills, no evil effects.
So I want you to think about who your audience is, your most important audience or audiences, one or two, what you're going to try to be saying as your message, what's your strategy, what's you're delivery.
So what I'd like to do is sort of go around the room and talk about each of these things step by step.
So we're going to take one group and we'll sort of run it all the way through on one message.
And we'll take some more and see where we land.
So Francois, let's start with your group.
What's your audience that you decided to target?
Well there are two audiences
Let's pick one, one to start with.
We'll come back
Environmentalists
All right.
And what are you trying to tell them?
[INAUDIBLE PHRASE].
[INAUDIBLE PHRASE]
So we talked about what you want them to know and what you want them to do.
So you're telling them that the prize exist.
And do you want them to do anything about it?
Just be excited?
Yeah, just be excited.
And to support them
What was your strategy for engaging the environmentalists?
So one big kickoff followed by like a regular traditional media campaign
OK, and your opening kickoff, how are you going to deliver that?
There will be a green Energizer Bunny tour
Green Energizer Bunny tour.
OK, So you've chosen the Energizer Bunny as your spokesman.
And what is he doing on his tour?
He is touring the city, dropping batteries... collecting them.
Beating his drum
Beating his drum.
OK. Did you have any city in particular, any date in particular?
Did it not matter at all?
A lot of cities [INAUDIBLE PHRASE]
Worldwide launch
We did not quite [UNINTELLIGIBLE PHRASE] We want the vertical instead of the horizontal
What do you guys have?
Who's your first audience?
Large device manufacturers
Large device manufacturers.
OK. What do you want them to know or do?
To realize the actual environmental costs of [UNINTELLIGIBLE PHRASE]
And what do you hope they do once they know that environmental cost?
To get them to start with environmentally friendly batteries
Do you want them to buy you new batteries?
Or do you want them to change other things?
Do you care?
Both actually
Did you have a particular piece of the strategy you were focusing on?
We were going to do a long-term guerrilla smear campaign
Long guerrilla smear campaign
[INAUDIBLE PHRASE]
Plus launch of new batteries.
All right.
Is there anything else?
Is there any salient big features of your long guerrilla smear campaign?
We were going to distribute Energizer death bunnies
Energizer death bunnies.
And what is an Energizer death bunny?
We haven't really thought through the details on that concept [INAUDIBLE PHRASE]
OK, Energizer death bunnies, fantastic.
OK, Paul, what did your group focus on?
Let me try.
We were focused on the consumers.
[INAUDIBLE PHRASE]
What did you want to tell them?
Don't pollute the planet and buy new batteries
Don't pollute it.
Buy new batteries.
And how are you going to do that?
With was your strategy?
We were trying to place ads in things like newborn baby magazines and science magazines
Parenting magazines and Popular Science magazines.
But while I read Popular Science magazines while I was pregnant and with young kids, I don't think your demographic overlap is probably very large there
Well the parenting will
Parenting will
Some of those, popular magazines, science magazines is more for the well-off, socially conscious
Exactly.
So when we broaden that definition of audience for sure.
What's your delivery strategy?
We were saying a baby playing on a pile of batteries
Baby on batteries
And an announcement on Earth Day
Announcement on Earth Day.
So you're going to have a big print ad campaign announced on Earth Day.
And your image was a baby crawling around on batteries.
So you're drawing on their mother's heartstrings.
Don't poison my baby
It always works
It's true.
It almost always works
So we talked a bit about how to get the competitors involved.
But then we also were interested in focusing on consumers.
And for consumers, basically we were thinking kids, parents, environmentalists, and miscellaneous other [UNINTELLIGIBLE PHRASE]
OK. And what was your message to them?
The basic message is that we're coming up with these new green batteries.
And you should switch to them because the current batteries are super bad for the environment
OK. strategy?
Our strategy is we're going to have Captain Planet be our spokesman.
We were thinking maybe some sort of roll-out campaign probably at a Superfund site, and then follow that up with ads probably online and the traditional media too.
Maybe an after school special, yes.
We're going to, to some extent, target children.
And the children will go to their parents and be like, oh my god.
Why are we doing these bad things?
It's a long-standing ad technique.
It's completely reasonable.
So Captain Planet at a Superfund site
Yes
Did anyone have a strategy for engaging competitors?
Yes.
There's a couple of obvious groups that would get involved.
And it was probably fairly easier to reach those people.
Put advertisements in The Chronicle of Higher Education saying, whip our this prize.
We want competitors.
What are your ideas to win it all?
Universities are an obvious place to look for chemical engineers and electrical engineers.
So hit up their trade magazines and websites that they're most likely to visit.
Put ads on Facebook for everyone who's a chemE or electrical engineer major
Anyone else think about engaging competitors at all?
Let's be totally ridiculous.
We're going to be totally ridiculous with this.
I like the Energizer death bunnies
[INAUDIBLE PHRASE]
Captain Planet is great.
He's not ridiculous
I don't think you've seen those cartoons
I have.
I assure you I have.
We actually reviewed them last year as part of our PR campaign.
Let's say you wanted to go totally rogue.
You want nothing to do with the current battery manufacturers.
You're looking for something totally different
Depending on how green these batteries are, you could give them out to random passerbyers as candy or something
Battery candy
Drinkable batteries
Drinking battery acid
Have a battery smoothie or something like that [INAUDIBLE PHRASE]
Drinking the electrolytes
Get Gatorade on there
Gatorade, I like it.
I like it
Powerade.
Or Powerade
Oh, that's awful.
Powerade
You could also have a concert, Poweraid, A-I-D. Get Willie Nelson and Bono
[INAUDIBLE PHRASE].
What if we wanted to be very conservative?
What if our goal was to impact Energizer, and Duracell, and the biggest manufacturers in the world?
And we really wanted them on our side.
We really wanted them engaged.
How does that change your ad campaign?
I guess you have to demonstrate that what you have is literally no different than what's already out there, it's just the quality is better
So there's a degree to which your demonstration becomes critical.
The demo equals credibility.
What else changes?
You want to bring them on board and have them involved.
So maybe if you can work out ahead of time with them that if the batteries meet this level, that they will produce them
So we talked about a little bit at the beginning of class about advance market commitments, advance purchase commitments.
Engaging a major manufacturer in one of these could be a really great way to really set some credibility.
So we're going to make sure that they're going to make the first million units and sell them.
Is there something other than the battery manufactures that grants you credibility?
Are there partners besides battery manufacturers?
Yeah, something like Energy Star
So something like Energy Star ratings
Some sort of [UNINTELLIGIBLE]
What?
Greenpeace
Greenpeace.
Yeah.
Greenpeace is definitely a two-edged sword.
It comes across as being both green and crazy.
And not so peaceful, I think, is a fair way of putting it.
But it definitely polarizes
You could go the regulatory route and say, these toxic batteries are going to get [UNINTELLIGIBLE]
Yeah.
So you can try to get a regulatory reform passed.
It's an interesting strategy.
So when X Prize flew the Ansari flights, the FAA pushed through an incredible set of legislation that allowed that to happen in the US in very short time.
If you wanted to have a regulatory piece to your prize, when would you want to do that in the course of the prize
As early as possible
Yeah.
So ideally, you would launch the prize and you would have your choice of a bipartisan set of senators and representatives on stage with you saying, we're passing this bill or we're putting forward this bill.
If you can do it early enough, then you can really impact who is going to compete
Although if you do that, then doesn't it kind of negate the purpose of the prize because the Senate or Congress is then creating the incentive for you
You can use it as an extra piece of your incentive.
If you think that they're likely to do it anyway, then you're right.
You're stepping out in front of the parade with a prize purse you don't need.
If you think your prize is what's allowing them to sponsor the legislation given the confidence to sponsor that legislation, there may be some argument there.
It's a fair question.
OK, so as you move forward-- and you guys are going to be narrowing down your focus-- we need to remember that the conversation isn't just about batteries.
It isn't just about storage devices.
But it's about the people around the technology.
And we didn't talk a lot about that today.
It sort of came out in some of your delivering messages here.
It's about the kids and the pollution.
It's about the parents and their kids.
Someone was saying, we want your ideas.
It's about the personal pride of being a competitor.
At some point, the actual technology itself is completely minimized behind what's happening in your public relations stage.
Because we're making heroes out of the teams.
We're making messages that are related to you and me personally.
And we're drawing on that emotional connection that we talked about for ad campaigns.
And you don't have emotional connections to batteries in most cases.
As you start to think about how you're going to craft your prize, have these things in mind.
You want to make something that's exciting, that's engaging, that's meaningful to an individual level.
If you have questions, feel free to ask.
Otherwise I'm going to be meeting with team Grid Storage I think next.
You guys are next.
All right, thanks very much.
Wednesday we'll be meeting at IDEO or at the Kendall T station.